<?xml version="1.0" encoding="utf-8"?>
<posts>
  <row Id="1" PostTypeId="1" AcceptedAnswerId="3" CreationDate="2017-05-16T18:01:06.383" Score="51" ViewCount="6454" Body="&lt;p&gt;I'd like to learn which format is most commonly used for storing the full human genome sequence (4 letters without a quality score) and why.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume that storing it in plain-text format would be very inefficient. I expect a binary format would be more appropriate (e.g. 2 bits per nucleotide).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which format is most common in terms of space efficiency?&lt;/p&gt;&#xA;" OwnerUserId="43" LastEditorUserId="6330" LastEditDate="2019-10-11T10:16:08.443" LastActivityDate="2022-02-23T16:01:18.060" Title="What's the most efficient file format for the storage of DNA sequences?" Tags="|human-genome|storage|file-formats|" AnswerCount="9" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2" PostTypeId="2" ParentId="1" CreationDate="2017-05-16T18:08:41.067" Score="4" Body="&lt;p&gt;In terms of raw storage capacity 2 bits per nucleotide, and then further compressed with standard compression techniques would be the most efficient. However, you'd still have other storage considerations. Like what to do about non-standard bases: like if you want to indicate a gap or ambiguity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd also query if it is really necessary to store them as binary since it reduces the readability of the data. It is quite convenient having a whole bunch of unix and programming tools that can operate on the string level in text files.&lt;/p&gt;&#xA;" OwnerUserId="62" LastEditorUserId="62" LastEditDate="2017-05-16T18:19:38.200" LastActivityDate="2017-05-16T18:19:38.200" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3" PostTypeId="2" ParentId="1" CreationDate="2017-05-16T18:09:58.197" Score="42" Body="&lt;p&gt;Genomes are commonly stored as either fasta files (.fa) or twoBit (.2bit) files. Fasta files store the entire sequence as text and are thus not particularly compressed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;twoBit files store each nucleotide in two bits and contain additional metadata that indicates where there's regions containing &lt;code&gt;N&lt;/code&gt; (unknown) bases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information, see the documentation on the twoBit format at the &lt;a href=&quot;http://genome.ucsc.edu/FAQ/FAQformat.html#format7&quot; rel=&quot;noreferrer&quot;&gt;UCSC genome browser&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can convert between twoBit and fasta format using the &lt;a href=&quot;https://genome.ucsc.edu/goldenpath/help/twoBit.html&quot; rel=&quot;noreferrer&quot;&gt;faToTwoBit and twoBitToFa utilities&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the human genome, you can download it in either fasta or twoBit format here: &lt;a href=&quot;http://hgdownload.cse.ucsc.edu/goldenPath/hg38/bigZips/&quot; rel=&quot;noreferrer&quot;&gt;http://hgdownload.cse.ucsc.edu/goldenPath/hg38/bigZips/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2017-05-16T18:21:25.940" LastActivityDate="2017-05-16T18:21:25.940" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4" PostTypeId="2" ParentId="1" CreationDate="2017-05-16T18:11:54.510" Score="25" Body="&lt;p&gt;The standard formats for storing sequence data are &lt;a href=&quot;https://en.wikipedia.org/wiki/FASTA_format&quot; rel=&quot;noreferrer&quot;&gt;fasta&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/FASTQ_format&quot; rel=&quot;noreferrer&quot;&gt;fastq&lt;/a&gt;.  Fasta is used if you only need the raw sequence data, fastq is used if you want to store the sequence data along with the quality information from base calling.  Each of these can be compressed using gzip or another standard compression algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically we want to keep the quality information along with the raw sequence data, but the quality information accounts for half the storage space required.  Some people have developed &lt;a href=&quot;https://web.stanford.edu/~mainakch/papers/QualComp_Final_v2.pdf&quot; rel=&quot;noreferrer&quot;&gt;algorithms&lt;/a&gt; for lossy compression of the quality data that allow us to reduce the storage requirements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested in storing variant calling data, the standard format for that is &lt;a href=&quot;http://www.internationalgenome.org/wiki/Analysis/vcf4.0/&quot; rel=&quot;noreferrer&quot;&gt;VCF&lt;/a&gt;.  VCF is useful if you want to store quality information of the variant calls, genomic positions, and any annotations you might have about the position.  VCFs can be compressed and indexed using &lt;a href=&quot;http://www.htslib.org/doc/tabix.html&quot; rel=&quot;noreferrer&quot;&gt;bgzip and tabix&lt;/a&gt;.  Many tools require variant data to be compressed and indexed using these tools.&lt;/p&gt;&#xA;" OwnerUserId="59" LastActivityDate="2017-05-16T18:11:54.510" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="5" PostTypeId="1" AcceptedAnswerId="13" CreationDate="2017-05-16T18:14:20.903" Score="19" ViewCount="654" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Human_Genome_Project&quot; rel=&quot;noreferrer&quot;&gt;The Human Genome Project&lt;/a&gt; was the project of 'determining the sequence of nucleotide base pairs that make up human DNA, and of identifying and mapping all of the genes of the human genome'. It was declared complete in 2003, i.e. 99% of the euchromatic human genome completed with 99.99% accuracy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are the datasets provided by HGP still accurate, or as accurate as was claimed in 2003? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the technology in the past (such as using old techniques), or any other reason (newer research studies), is it possible that the datasets are not as accurate as originally expected?&lt;/p&gt;&#xA;" OwnerUserId="43" LastEditorUserId="73" LastEditDate="2017-12-28T11:43:48.257" LastActivityDate="2017-12-28T11:43:48.257" Title="Accuracy of the original human DNA datasets sequenced by Human Genome Project?" Tags="|hgp|phylogenetics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="6" PostTypeId="5" CreationDate="2017-05-16T18:16:13.057" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-16T18:16:13.057" LastActivityDate="2017-05-16T18:16:13.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="7" PostTypeId="4" CreationDate="2017-05-16T18:16:13.057" Score="0" Body="The Human Genome Project was a scientific research project with the goal of determining the sequence of human DNA." OwnerUserId="43" LastEditorUserId="43" LastEditDate="2017-05-16T18:41:13.577" LastActivityDate="2017-05-16T18:41:13.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="9" PostTypeId="1" CreationDate="2017-05-16T18:28:49.047" Score="13" ViewCount="863" Body="&lt;p&gt;I'm interested working with the medication information provided by the &lt;a href=&quot;http://biobank.ctsu.ox.ac.uk/crystal/field.cgi?id=20003&quot; rel=&quot;noreferrer&quot;&gt;UK Biobank&lt;/a&gt;.  In order to get these into a usable form I would like to map them to &lt;a href=&quot;https://en.wikipedia.org/wiki/Anatomical_Therapeutic_Chemical_Classification_System&quot; rel=&quot;noreferrer&quot;&gt;ATC codes&lt;/a&gt;.  Since many of the drugs listed in the data showcase include dosage information, doing an exact string match between drug names is not very effective.  I've considered using something like &lt;a href=&quot;https://pypi.python.org/pypi/fuzzywuzzy&quot; rel=&quot;noreferrer&quot;&gt;fuzzywuzzy&lt;/a&gt; to do string matching between the medications in the data showcase and the ATC drug names but validating the matches could still be a laborious process.  Does anyone know of a tool that can match drug names to ATC codes or some other drug ontology?  If not, maybe there's a better way to do it that I haven't thought of.&lt;/p&gt;&#xA;" OwnerUserId="59" LastActivityDate="2017-05-16T19:36:42.037" Title="Mapping drug names to ATC codes" Tags="|drugs|ontology|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="11" PostTypeId="2" ParentId="1" CreationDate="2017-05-16T18:31:11.587" Score="13" Body="&lt;p&gt;There are several things to consider when asking for &quot;the most efficient&quot; way to store data, it all depends on your use case. Do you just need ACGT, or are there also IUPAC codings for combinations? Do you need additional data (like quality values)? What kind of application are you using the data for (does it need to load all at once or in in chunks? Once or multiple times? Sequential or random access? etc.pp)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g., most efficient for:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Lowest footprint on disk, without a lot of hassle: use either FASTA or 2bit, but run through standard compressor (gzip, bzip2, others). The literature you want to consult here is that of standard text compression I think. Also of interest &lt;a href=&quot;http://mattmahoney.net/dc/text.html&quot; rel=&quot;noreferrer&quot;&gt;Large Text Compression Benchmark&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Keeping the file on disk, but ultra-fast loading small subsets into memory, being able to work in memory with character sized entities: a simple dump of the DNA as characters to disk, maybe combined with an index file to know which chromosome starts where. Then use &lt;a href=&quot;http://pubs.opengroup.org/onlinepubs/009695399/functions/mmap.html&quot; rel=&quot;noreferrer&quot;&gt;mmap&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Storing quality values: See papers like &lt;a href=&quot;http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0059190&quot; rel=&quot;noreferrer&quot;&gt;Compression of FASTQ and SAM Format Sequencing Data&lt;/a&gt; or &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3637481/&quot; rel=&quot;noreferrer&quot;&gt;Sequence squeeze: an open contest for sequence compression&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Any combination of the above use cases + a lot more&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="44" LastActivityDate="2017-05-16T18:31:11.587" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="12" PostTypeId="1" CreationDate="2017-05-16T18:31:28.570" Score="11" ViewCount="121" Body="&lt;p&gt;I'm looking to dock a large ligand (~90kDa) to a receptor slightly larger receptor  (~125kDa) using Hex. If anyone is familiar with docking large structures, are there any recommended parameters for finding the best docking solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parameters in particular:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Number of Solutions  &lt;/li&gt;&#xA;&lt;li&gt;N order of correlation for initial and final&#xA;searches &lt;/li&gt;&#xA;&lt;li&gt;Receptor Range &lt;/li&gt;&#xA;&lt;li&gt;Ligand Range&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="65" LastEditorUserId="48" LastEditDate="2017-05-18T00:01:22.427" LastActivityDate="2017-05-18T00:01:22.427" Title="What are the optimal parameters for docking a large ligand using Hex?" Tags="|proteins|docking|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="13" PostTypeId="2" ParentId="5" CreationDate="2017-05-16T18:37:07.157" Score="18" Body="&lt;p&gt;The HGP developed the first &quot;reference&quot; human genome - a genome that other genomes could be compared to, and was actually a composite of multiple human genome sequences. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The standard human reference genome is actually continually updated with major and minor revisions, a bit like software. The &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/grc/human&quot; rel=&quot;noreferrer&quot;&gt;latest major version&lt;/a&gt; is called GRCh38, was released in 2013, and has since had a number of minor updates. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Are the datasets provided by HGP still accurate?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, in a sense, but we certainly have better information now. One way to measure the quality of the assembly is that the initial release from the HGP &lt;a href=&quot;https://www.nature.com/nmeth/journal/v7/n5/full/nmeth0510-331.html&quot; rel=&quot;noreferrer&quot;&gt;had hundreds of thousands&lt;/a&gt; of gaps - sequences that could not be resolved (this often occurs because of repetitive sequences). The newest reference genome has less than 500 gaps.&lt;/p&gt;&#xA;" OwnerUserId="8" LastActivityDate="2017-05-16T18:37:07.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="14" PostTypeId="1" AcceptedAnswerId="385" CreationDate="2017-05-16T18:37:27.773" Score="41" ViewCount="60491" Body="&lt;p&gt;I'd like to learn the differences between 3 common formats such as &lt;a href=&quot;https://en.wikipedia.org/wiki/FASTA_format&quot; rel=&quot;noreferrer&quot;&gt;FASTA&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/FASTQ_format&quot; rel=&quot;noreferrer&quot;&gt;FASTQ&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/SAM_(file_format)&quot; rel=&quot;noreferrer&quot;&gt;SAM&lt;/a&gt;. How they are different? Are there any benefits of using one over another?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on Wikipedia pages, I can't tell the differences between them.&lt;/p&gt;&#xA;" OwnerUserId="43" LastEditorUserId="3848" LastEditDate="2020-12-07T12:01:25.873" LastActivityDate="2022-08-18T18:58:12.410" Title="What is the difference between FASTA, FASTQ, and SAM file formats?" Tags="|fasta|fastq|file-formats|sam|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="15" PostTypeId="1" CreationDate="2017-05-16T18:41:23.147" Score="11" ViewCount="3295" Body="&lt;p&gt;Many of my colleagues recommend I use BWA-MEM instead of regular old BWA. The problem is I don't understand why and reading the BWA man page doesn't seem to help the matter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference between BWA and BWA-MEM? And, in which instances would you employ one over the other?&lt;/p&gt;&#xA;" OwnerUserId="88" LastEditorUserId="294" LastEditDate="2018-04-10T07:43:04.913" LastActivityDate="2018-04-10T07:43:04.913" Title="Difference between BWA-backtrack and BWA-MEM" Tags="|sequence-alignment|bwa|read-mapping|reads|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="16" PostTypeId="2" ParentId="1" CreationDate="2017-05-16T18:46:51.883" Score="7" Body="&lt;p&gt;It is not yet standardized, but &lt;a href=&quot;https://www.sevenbridges.com/graph/better-reference/&quot; rel=&quot;noreferrer&quot;&gt;graph format&lt;/a&gt; has the potential for being the most space-efficient method for storing genomes.  The idea is this: rather than store a genome as a linear string of sequenced nucleotides, genomes are stored as overlapping graphs, where sequence variants branch off from the reference genome, and then rejoin when the alignment continues.  Basically, you start with a reference genome, and for every subsequent genome added to the graph, only the differences are stored.  This could allow for an enormous gain in space efficiency. &lt;/p&gt;&#xA;" OwnerUserId="47" LastActivityDate="2017-05-16T18:46:51.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="17" PostTypeId="2" ParentId="14" CreationDate="2017-05-16T18:53:38.760" Score="7" Body="&lt;p&gt;FASTA (officially) just stores the name of a sequence and the sequence, unofficially people also add comment fields after the name of the sequence. FASTQ was invented to store both sequence and associated quality values (e.g. from sequencing instruments). SAM was invented to store alignments of (small) sequences (e.g. generated from sequencing) with associated quality values and some further data onto a larger sequences, called reference sequences, the latter being anything from a tiny virus sequence to ultra-large plant sequences.&lt;/p&gt;&#xA;" OwnerUserId="44" LastEditorUserId="73" LastEditDate="2021-09-29T04:38:50.373" LastActivityDate="2021-09-29T04:38:50.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18" PostTypeId="2" ParentId="14" CreationDate="2017-05-16T18:57:28.007" Score="19" Body="&lt;p&gt;In a nutshell, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;FASTA&lt;/code&gt; file format is a DNA sequence format for specifying or representing DNA sequences and was first described by Pearson &lt;code&gt;(Pearson,W.R. and Lipman,D.J. (1988) Improved tools for biological sequence comparison. Proc. Natl Acad. Sci. USA, 85, 2444–2448)&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;FASTQ&lt;/code&gt; is another DNA sequence file format that extends the FASTA format with the ability to store the sequence quality. The quality scores are often represented in  ASCII characters which correspond to a phred score)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both FASTA and FASTQ are common sequence representation formats and have emerged as key data interchange formats for molecular biology and bioinformatics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;SAM&lt;/code&gt; is format for representing sequence alignment information from a read aligner. It represents sequence information in respect to a given reference sequence. The information is stored in a series of tab delimited ascii columns. The full SAM format specification is available at &lt;a href=&quot;http://samtools.sourceforge.net/SAM1.pdf&quot; rel=&quot;noreferrer&quot;&gt;http://samtools.sourceforge.net/SAM1.pdf&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2017-05-16T18:57:28.007" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="19" PostTypeId="1" AcceptedAnswerId="293" CreationDate="2017-05-16T19:15:57.957" Score="23" ViewCount="2402" Body="&lt;p&gt;A common bioinformatics task is to decompose a DNA sequence into its constituent k-mers and compute a hash value for each k-mer. &lt;a href=&quot;https://en.wikipedia.org/wiki/Rolling_hash&quot; rel=&quot;noreferrer&quot;&gt;Rolling hash functions&lt;/a&gt; are an appealing solution for this task, since they can be computed very quickly. A rolling hash does not compute each the hash value from scratch with each k-mer: rather it updates a running hash value using an update strategy and a sliding window over the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's also very useful for many applications to have a k-mer hash to the same value as its reverse complement. Unless the data were generated using a strand-specific sample prep, it's impossible to distinguish a k-mer from its reverse complement, and they should be treated as the same sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any rolling hashes that will map reverse complements to the same value? If not, how would we develop such an algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;: Ideally the hash function would be able to support k &gt; 32, which would be lossy unless using something larger than a 64-bit integer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;ANOTHER UPDATE&lt;/strong&gt;: I don't think it's necessary to &lt;em&gt;store&lt;/em&gt; both the running k-mer and its reverse complement in a single value. If storing two k-mer strings and/or two hash values makes this easier, I'm totally cool with that.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2017-05-16T20:50:34.373" LastActivityDate="2017-07-26T16:10:55.117" Title="Are there any rolling hash functions that can hash a DNA sequence and its reverse complement to the same value?" Tags="|hashing|k-mer|algorithms|" AnswerCount="4" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="20" PostTypeId="2" ParentId="1" CreationDate="2017-05-16T19:17:38.697" Score="20" Body="&lt;p&gt;The standard and the most common sequence format is FASTA for sure. You can compress it with a compressor. For the ~3GB human genome, gzip reduces the size to ~900MB, depending on the option in use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another often used format is UCSC's 2-bit format. This format keeps each A/C/G/T with 2 bits. As I remember, it keeps non-A/C/G/T bases and lowercases in two separate lists. These lists basically tell you that bases between offset x and y are all &quot;N&quot;/lowercase. The 2-bit format loses IUB codes that GRCh37 has. UCSC's hg19 differs from GRCh37 at a few bases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BWA also produces its own 2-bit format with indexing. You can generate it separately with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bwa fa2pac -f hg19.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unlike UCSC, BWA keeps all IUB codes but loses letter cases. BWA does not provide utilities to convert its 2-bit representation to FASTA, either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 2-bit format typically reduces the file size down to 1/4 of its original size, unless there are too many scattered ambiguous bases. For human genome, you get a file ~784MB in size. You can compress it further with gzip, but that actually doesn't work well. A gzip'd 2-bit file is only ~5-10% smaller.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to achieve an even smaller file size, you can compress the BWT of 2-bit file. This gives you a ~633MB file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bwa pac2bwtgen hg19.fa.pac tmp.bwt &amp;amp;&amp;amp; gzip tmp.bwt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A bit-aware compression algorithm may achieve an even higher compression ratio. However, such BWT-based compression prevents you from extracting subsequences. It is probably of little use in practice.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-05-16T19:17:38.697" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="21" PostTypeId="1" AcceptedAnswerId="64" CreationDate="2017-05-16T19:24:16.303" Score="48" ViewCount="14099" Body="&lt;p&gt;What are the actual differences between different annotation databases? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My lab, for reasons still unknown to me, prefers Ensembl annotations (we're working with transcript/exon expression estimation), while some software ship with RefSeq annotations. Are there significant differences between them today, or are they, for all intents and purposes, interchangeable (e.g., are exon coordinates between RefSeq and Ensembl annotations interchangeable)?&lt;/p&gt;&#xA;" OwnerUserId="82" LastActivityDate="2024-03-28T23:20:07.193" Title="Feature annotation: RefSeq vs Ensembl vs Gencode, what's the difference?" Tags="|sequence-annotation|ensembl|refseq|gencode|" AnswerCount="6" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="22" PostTypeId="2" ParentId="9" CreationDate="2017-05-16T19:33:06.700" Score="5" Body="&lt;p&gt;The &lt;a href=&quot;http://cart.embl.de/&quot; rel=&quot;noreferrer&quot;&gt;CART&lt;/a&gt; tool let's you upload a set of names and map them (optionally in a fuzzy way) to STITCH 4 identifiers, and then use those to map to ATC codes (using the chemicals sources &lt;a href=&quot;http://stitch4.embl.de/cgi/show_download_page.pl&quot; rel=&quot;noreferrer&quot;&gt;download file&lt;/a&gt;). It's a bit indirect, and I'm not sure what CART will do with the dosage info you mention.&lt;/p&gt;&#xA;" OwnerUserId="112" LastActivityDate="2017-05-16T19:33:06.700" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="23" PostTypeId="2" ParentId="9" CreationDate="2017-05-16T19:36:42.037" Score="5" Body="&lt;p&gt;&lt;a href=&quot;https://www.drugbank.ca&quot; rel=&quot;noreferrer&quot;&gt;DrugBank&lt;/a&gt; seems to have a &lt;a href=&quot;https://www.drugbank.ca/atc&quot; rel=&quot;noreferrer&quot;&gt;tool to map ATC codes to drug names&lt;/a&gt; and DrugBank IDs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A quick look in the XSD schema on the &lt;a href=&quot;https://www.drugbank.ca/releases/latest&quot; rel=&quot;noreferrer&quot;&gt;release page&lt;/a&gt; suggests the complete database includes ATC codes for drugs, you could then do a fuzzy match of the BioBank names against all of DrugBank's synonyms, or match on some other data (e.g. canonicalised SMILES) if available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The downside is that there may not be complete overlap between UK BioBank and DrugBank. Additionally, DrugBank is under licence for commercial use.&lt;/p&gt;&#xA;" OwnerUserId="109" LastActivityDate="2017-05-16T19:36:42.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="24" PostTypeId="2" ParentId="5" CreationDate="2017-05-16T19:40:38.183" Score="8" Body="&lt;p&gt;While the quality of the reference human assembly keeps improving, there are still misassemblies in it. A common problem is recent segmental duplications are occasionally collapsed into one sequence in the reference. Another issue is that the centromeric sequences in the reference are computationally generated, which are probably different from real sequences. Issues like these often complicate data analyses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should also beware that each human has a different genome. A large region having one copy in the reference genome may have two copies in a specific sample. While this is not really the problem with the reference, such copy-number changes will have the same effect as reference errors and mess up your pipelines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is still room for improvement to the human reference genome. In some regions, the CHM1 and CHM13 PacBio assemblies are better than the current reference genome at the larger scale. Illumina population data can produce better consensus at the single base level. GRC is continuously releasing new patches to the latest assembly.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-05-16T19:40:38.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="25" PostTypeId="2" ParentId="14" CreationDate="2017-05-16T19:50:20.683" Score="3" Body="&lt;p&gt;FASTA and FASTQ formats are both file formats that contain sequencing reads while SAM files are these reads aligned to a reference sequence. In other words, FASTA and FASTQ are the &amp;quot;raw data&amp;quot; of sequencing while SAM is the product of aligning the sequencing reads to a refseq.&lt;/p&gt;&#xA;&lt;p&gt;A FASTA file contains a read name followed by the sequence. An example of one of these reads for RNASeq might be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Flow cell number: lane number: chip coordinates etc.&#xA;ATTGGCTAATTGGCTAATTGGCTAATTGGCTAATTGGCTAATTGGCTAATTGGCTAATTGGCTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The FASTQ version of this read will have two more lines, one + as a place holder and then a line of quality scores for the base calls. The qualities are given as characters with '!' being the lowest and '~' being the highest, in increasing ASCII value. It would look something like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@Flow cell number: lane number: chip coordinates etc.&#xA;ATTGGCTAATTGGCTAATTGGCTAATTGGCTAATTGGCTAATTGGCTAATTGGCTAATTGGCTA&#xA;+&#xA;!''*((((***+))%%%++)(%%%%).1***-+*''))**55CCF&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;CCCCCCC65&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A SAM file has many fields for each alignment, the header begins with the @ character. The alignment contains 11 mandatory fields and various optional ones. You can find the spec file here: &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://samtools.github.io/hts-specs/SAMv1.pdf&lt;/a&gt; .&lt;/p&gt;&#xA;&lt;p&gt;Often you'll see BAM files which are just compressed binary versions of SAM files. You can view these alignment files using various tools, such as SAMtools, IGV or USCS Genome browser.&lt;/p&gt;&#xA;&lt;p&gt;As to the benefits, FASTA/FASTQ vs. SAM/BAM is comparing apples and oranges. I do a lot of RNASeq work so generally we take the FASTQ files and align them the a refseq using an aligner such as STAR which outputs SAM/BAM files. There's a lot you can do with just these alignment files, looking at expression, but usually I'll use a tool such as RSEM to &amp;quot;count&amp;quot; the reads from various genes to create an expression matrix, samples as columns and genes as rows. Whether you get FASTQ or FASTA files just depends on your sequencing platform. I've never heard of anybody really using the quality scores.&lt;/p&gt;&#xA;" OwnerUserId="94" LastEditorUserId="73" LastEditDate="2021-09-29T04:39:22.363" LastActivityDate="2021-09-29T04:39:22.363" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="26" PostTypeId="2" ParentId="19" CreationDate="2017-05-16T19:51:29.843" Score="8" Body="&lt;p&gt;Let &lt;code&gt;f&lt;/code&gt; and &lt;code&gt;r&lt;/code&gt; be two integers. They always keep the k-mer on the forward and reverse strand, respectively. At a new base &lt;code&gt;c&lt;/code&gt; in a proper 2-bit encoding, we update the two integers as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f = (f&amp;lt;&amp;lt;2|c) &amp;amp; ((1ULL&amp;lt;&amp;lt;2*k) - 1)&#xA;r = r&amp;gt;&amp;gt;2 | (3ULL-c)&amp;lt;&amp;lt;2*(k-1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With this updating rule, &lt;code&gt;f&lt;/code&gt; keeps the forward strand k-mer ending at &lt;code&gt;c&lt;/code&gt; and &lt;code&gt;r&lt;/code&gt; is &lt;code&gt;f&lt;/code&gt;'s reverse complement. The hash of the k-mer can be &lt;code&gt;min(f,r)&lt;/code&gt;. If integer operations take constant time, computing the hashes of all k-mers of a sequence of length L takes O(L) time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above only works if k-mer fits a word, which means &lt;code&gt;k&amp;lt;=32&lt;/code&gt; on x86. For long k-mers, you can overload bit operators in C++. In case of &lt;code&gt;32&amp;lt;k&amp;lt;=64&lt;/code&gt;, a simpler solution is to split into lower and higher bits:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;c1 = c&amp;amp;1, c2 = c&amp;gt;&amp;gt;1&#xA;f1 = (f1&amp;lt;&amp;lt;1|c1) &amp;amp; ((1ULL&amp;lt;&amp;lt;k) - 1)&#xA;f2 = (f2&amp;lt;&amp;lt;1|c2) &amp;amp; ((1ULL&amp;lt;&amp;lt;k) - 1)&#xA;r1 = r1&amp;gt;&amp;gt;1 | (1ULL-c1)&amp;lt;&amp;lt;(k-1)&#xA;r2 = r2&amp;gt;&amp;gt;1 | (1ULL-c2)&amp;lt;&amp;lt;(k-1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is twice as slow as the &lt;code&gt;k&amp;lt;=32&lt;/code&gt; version. It is possible to implement these lines with SSE2 intrinsics, but that is overkilling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you want to hash a long k-mer into fewer bits, there are a few options:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;xor:      f^r&#xA;min:      min(f,r)&#xA;min+hash: hash(min(f,r))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;hash()&lt;/code&gt; is a generic randomization hash function, which could be Murmur3, Wang's integer hash function, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is a microbenchmark. Here, we squeeze all k-mers into a 32-bit hash table. &lt;code&gt;collision_rate = 1 - #distinct_hash/#distinct_kmer&lt;/code&gt;. This is not the best metric, but it somehow measures the randomness, which is better than nothing.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;====================================================&#xA; Algorithm    data  max-k  k   %collision  CPU time&#xA;----------------------------------------------------&#xA; xor          chr11  32    31     8.5%       0.9s&#xA; xor+Wang     chr11  32    31     9.6%       1.3s&#xA; min+Wang     chr11  32    31     1.4%       1.3s&#xA; min+Wang     chr11  64    31     1.4%       1.9s&#xA; min+murmur3  chr11  inf   31     1.4%       5.7s&#xA; min+Wang     chr11  64    51     1.5%       2.1s&#xA; min+murmur3  chr11  inf   51     1.5%       6.8s&#xA; min+Wang     chr11  64    63     1.5%       2.1s&#xA; min+murmur3  chr11  inf   63     1.5%       7.5s&#xA; radix sort   chr11  -     -      -        5.6-7.3s&#xA;----------------------------------------------------&#xA; min+Wang     hg38   64    63    26.2%        37s&#xA; min+murmur3  hg38   inf   63    26.2%       192s&#xA; radix sort   hg38   -     -      -         ~138s&#xA;====================================================&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Observations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Don't use XOR. In fact, XOR hashes all palindromes to &lt;code&gt;0xffff...&lt;/code&gt;. This is already worrying enough.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Unless you want to work with very long k-mers, don't use generic string hash functions like FNV and Murmur. Murmur is even slower than radix sorting all hashes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For &lt;code&gt;k&amp;lt;=64&lt;/code&gt;, min+Wang is the best here. It is fast and simple to compute and has randomness nearly as good as murmur.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2017-05-18T16:54:03.623" LastActivityDate="2017-05-18T16:54:03.623" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="27" PostTypeId="1" AcceptedAnswerId="32" CreationDate="2017-05-16T20:00:43.050" Score="17" ViewCount="4680" Body="&lt;p&gt;I have a set of BAM files that are aligned using the NCBI GRCh37 human genome reference (with the chromosome names as NC_000001.10) but I want to analyze it using a BED file that has the UCSC hg19 chromosome names (e.g. chr1). I want to use bedtools to pull out all the on-target and off-target reads.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Are NCBI and UCSC directly comparable? Or do I need to re-align the BAM/lift-over the BED to the UCSC reference?&lt;/li&gt;&#xA;&lt;li&gt;Should I convert the BED file or the BAM file? Everyone here uses the UCSC chromosome names/positions so I'll need to convert the eventual files to UCSC anyway.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="110" LastEditorUserId="37" LastEditDate="2017-06-25T02:10:00.757" LastActivityDate="2017-08-08T11:02:02.630" Title="Convert a BAM file from one reference to another?" Tags="|sam|file-formats|bed|format-conversion|reference-genome|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="28" PostTypeId="2" ParentId="19" CreationDate="2017-05-16T20:17:30.960" Score="6" Body="&lt;p&gt;If your goal is to minimise storage by just having one hash per kmer and its reverse complement, there is a simple solution for non-rolling hashes. For any sequence S, you compute and store the hash of the smaller of S and its reverse complement. A simple lexicographical comparison for &quot;smaller&quot; is enough. In programming terms:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;hash=computeHash(min(S,rev(S));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If your goal is to minimise computing time via rolling hashes AND store only one kmer value, this will be hard. Hashes are normally designed to minimise collisions and you are asking it not only to collide, but also to collide in very, very specific circumstances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question is: is the computation of a hash a bottleneck for an application? Maybe, maybe not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://softwareengineering.stackexchange.com/questions/49550/which-hashing-algorithm-is-best-for-uniqueness-and-speed/145633#145633&quot;&gt;This&lt;/a&gt; post on StackExchange seems to imply that, in 2012, MurmurHash was able to compute de-novo a hash for a UUID (similar to a standard kmer in size and complexity) in ~250ns, i.e., ~4 million hashes per second. That's already quite fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guess is that most applications will use a hash to look up things in memory or disk. And here, even with look-ups in memory, real life data will let you run into CPU cache miss speed penalties very quickly and I expect the impact of this to be higher than the speed of the hash computation itself.&lt;/p&gt;&#xA;" OwnerUserId="44" LastActivityDate="2017-05-16T20:17:30.960" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="29" PostTypeId="1" CreationDate="2017-05-16T20:22:01.540" Score="6" ViewCount="200" Body="&lt;p&gt;What's the most widely accepted tool for doing pseudo-temporal ordering from scRNAseq data? Also is there away to separate differential expression that occurs based on &quot;cell identity&quot; or maybe more accurately cell type fate from that which arises from cells being in different stages differentiation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more concrete, lets say there's a population of cells, some of which were born at time 1, time 2, and time 3. The progression along the temporal trajectory can be described via a set of genes that are fluctuating as the cell matures. So you might have the same cell type which was born at time 1 be transcriptionally distinct from a younger one born at time 3.  On the other hand, within this population there are subpopulations which will have different cell fates and are transcriptionally distinct. Is there away to reliably separate the temporal axis from the &quot;cell fate axes&quot;. If not is this something people are working on or is flawed logic to think this kind of thing is possible?&lt;/p&gt;&#xA;" OwnerUserId="94" LastActivityDate="2017-06-05T17:42:37.170" Title="Pseudo-temporal ordering in heterogeneous populations" Tags="|scrnaseq|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="30" PostTypeId="5" CreationDate="2017-05-16T20:52:13.140" Score="0" Body="&lt;p&gt;The &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&amp;amp;PAGE_TYPE=BlastDocs&amp;amp;DOC_TYPE=BlastHelp&quot; rel=&quot;nofollow noreferrer&quot;&gt;FASTA format&lt;/a&gt; has a single-line definition (defline), followed by one or more lines of DNA, RNA or amino acid sequences. The definition line is specified by the greater than (&lt;code&gt;&amp;gt;&lt;/code&gt;) character.&lt;/p&gt;&#xA;" OwnerUserId="8" LastEditorUserId="8" LastEditDate="2017-05-18T00:02:17.593" LastActivityDate="2017-05-18T00:02:17.593" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="31" PostTypeId="4" CreationDate="2017-05-16T20:52:13.140" Score="0" Body="To be used for questions specific tothe  sequence file format `.fasta`.&#xD;&#xA;&#xD;&#xA;Please minimise usage if the question is more generally about sequence formats." OwnerUserId="57" LastEditorUserId="3848" LastEditDate="2020-11-29T15:18:47.417" LastActivityDate="2020-11-29T15:18:47.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="32" PostTypeId="2" ParentId="27" CreationDate="2017-05-16T21:04:42.503" Score="24" Body="&lt;p&gt;You're the second person I have ever seen using NCBI &quot;chromosome names&quot; (they're more like supercontig IDs). Normally I would point you to &lt;a href=&quot;https://github.com/dpryan79/ChromosomeMappings&quot; rel=&quot;noreferrer&quot;&gt;a resource providing mappings between chromosome names&lt;/a&gt;, but since no one has added NCBI names (yet, maybe I'll add them now) you're currently out of luck there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, the quickest way to do what you want is to &lt;code&gt;samtools view -H foo.bam &amp;gt; header&lt;/code&gt; to get the BAM header and then change each NCBI &quot;chromosome name&quot; to its corresponding UCSC chromosome name. DO NOT REORDER THE LINES! You can then use &lt;code&gt;samtools reheader&lt;/code&gt; and be done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why, you might ask, would this work? The answer is that chromosome/contig names in BAM files aren't stored in each alignment. Rather, the names are stored in a list in the header and each alignment just contains the integer index into that list (read group IDs are similar, for what it's worth). This also leads to the warning above against reordering entries, since that's a VERY convenient way to start swapping alignments between chromosomes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an aside, you'd be well served switching to Gencode or Ensembl chromosome names, they're rather more coherent than the &lt;code&gt;something_random&lt;/code&gt; mess that's present in hg19 from UCSC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;: Because I'm nice, &lt;a href=&quot;https://github.com/dpryan79/ChromosomeMappings/blob/master/GRCh37_NCBI2UCSC.txt&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; is the conversion between NCBI and UCSC. Note that if you have any alignments to patches that there is simply no UCSC equivalent. One of the many reasons not to use UCSC (avoid their annotations too).&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-05-16T21:46:53.327" LastActivityDate="2017-05-16T21:46:53.327" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="35" PostTypeId="2" ParentId="15" CreationDate="2017-05-16T22:49:39.000" Score="9" Body="&lt;p&gt;To quote the Introduction to &lt;code&gt;BWA&lt;/code&gt; on &lt;a href=&quot;http://bio-bwa.sourceforge.net/&quot; rel=&quot;noreferrer&quot;&gt;sourceforge&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;BWA is a software package for mapping low-divergent sequences against a large reference genome, such as the human genome. It consists of three algorithms: BWA-backtrack, BWA-SW and BWA-MEM. The first algorithm is designed for Illumina sequence reads up to 100bp, while the rest two for longer sequences ranged from 70bp to 1Mbp. BWA-MEM and BWA-SW share similar features such as long-read support and split alignment, but BWA-MEM, which is the latest, is generally recommended for high-quality queries as it is faster and more accurate. BWA-MEM also has better performance than BWA-backtrack for 70-100bp Illumina reads.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In short, for anything where you have read lengths over 70bp BWA-MEM is faster, and more accurate.&lt;/p&gt;&#xA;" OwnerUserId="149" LastActivityDate="2017-05-16T22:49:39.000" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="36" PostTypeId="1" AcceptedAnswerId="42" CreationDate="2017-05-17T02:18:01.720" Score="13" ViewCount="237" Body="&lt;p&gt;I have run Oxford Nanopore Technologies' MinION sequencing on the same DNA sample using three flowcells, each aligned against the same reference genome (E.coli K12 MG1655) using both BWA MEM and GraphMap and stored as BAM files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I quantitatively and efficiently analyse the quality of alignment (percentage identity, insertion rate, deletion rate) of each of these files?&lt;/p&gt;&#xA;" OwnerUserId="163" LastEditorUserId="163" LastEditDate="2017-05-17T02:23:22.180" LastActivityDate="2017-05-18T00:01:18.567" Title="Compare alignment quality of multiple sequencing runs aligned against the same reference genome" Tags="|sam|sequence-alignment|nanopore|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="38" PostTypeId="2" ParentId="21" CreationDate="2017-05-17T02:29:23.840" Score="12" Body="&lt;h2&gt;Ensembl vs Gencode&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.gencodegenes.org/faq.html&quot; rel=&quot;noreferrer&quot;&gt;https://www.gencodegenes.org/faq.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The GENCODE annotation is made by merging the Havana manual gene annotation and the Ensembl automated gene annotation. [...] In practical terms, the GENCODE annotation is identical to the Ensembl annotation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Further, for the GTF file differences:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The only exception is that the genes which are common to the human chromosome X and Y PAR regions can be found twice in the GENCODE GTF, while they are shown only for chromosome X in the Ensembl file.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h2&gt;Gencode(Ensembl) vs RefSeq&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Gencode is in almost all cases &lt;a href=&quot;https://bmcgenomics.biomedcentral.com/articles/10.1186/1471-2164-16-S8-S2&quot; rel=&quot;noreferrer&quot;&gt;more comprehensive&lt;/a&gt;. For example, this is NCBI RefSeq vs Ensembl (v24, release 83) for BRCA gene:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/MO5Gb.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MO5Gb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RefSeq and Gencode are not interchangeable in most cases, though RefSeq annotations will often be a subset of the Gencode ones.&lt;/p&gt;&#xA;" OwnerUserId="161" LastActivityDate="2017-05-17T02:29:23.840" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="42" PostTypeId="2" ParentId="36" CreationDate="2017-05-17T03:10:02.837" Score="6" Body="&lt;p&gt;Qualimap will do this for you. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Go to &lt;a href=&quot;http://qualimap.bioinfo.cipf.es&quot; rel=&quot;nofollow noreferrer&quot;&gt;qualimap.bioinfo.cipf.es&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Run qualimap (default params are fine) on each BAM file&lt;/li&gt;&#xA;&lt;li&gt;Open up the HTML output, and you can read off the %identity (they measure the opposite, i.e. mismatch rate, but 100% - mismatch rate is %identity of course), indel rate, etc.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;One thing to watch out for (you don't mention it in your question, but just in case) is that you cannot directly compare Q scores - these are a bit of a mess and calculated very differently in each piece of software. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unsolicited suggestion: you might also try &lt;a href=&quot;https://github.com/philres/ngmlr&quot; rel=&quot;nofollow noreferrer&quot;&gt;NGM-LR&lt;/a&gt; for mapping MinION data. We've found it beats the others for our data (though we map to a distant reference).&lt;/p&gt;&#xA;" OwnerUserId="156" LastEditorUserId="163" LastEditDate="2017-05-18T00:01:18.567" LastActivityDate="2017-05-18T00:01:18.567" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="43" PostTypeId="2" ParentId="1" CreationDate="2017-05-17T03:20:00.923" Score="8" Body="&lt;p&gt;I think the question is a bit ambiguous so please excuse this answer that's a bit redundant from the rest of the ones provided.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As others have mentioned, if you want to store a full genome, &lt;a href=&quot;https://en.wikipedia.org/wiki/FASTA_format&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;FASTA&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;https://genome.ucsc.edu/FAQ/FAQformat.html#format7&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;2bit&lt;/code&gt;&lt;/a&gt; formats are appropriate.  For some context, &lt;a href=&quot;http://hgdownload.cse.ucsc.edu/goldenPath/hg19/bigZips/&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;hg19&lt;/code&gt;&lt;/a&gt; is about 900Mb compressed for the &lt;code&gt;FASTA&lt;/code&gt; file and about 780Mb compressed for the &lt;code&gt;2bit&lt;/code&gt; file .  &lt;code&gt;hg19&lt;/code&gt; is a reference and is haploid so doesn't represent a &quot;full&quot; human genome that would normally have two alleles for the autosome (non-sex chromosomes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A common format for representing variant information is Variant Call Format (&lt;a href=&quot;https://vcftools.github.io/specs.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;VCF&lt;/code&gt;&lt;/a&gt;). The &lt;code&gt;VCF&lt;/code&gt; format represent differences from a reference (&lt;code&gt;hg19&lt;/code&gt;, say) that can be used to recover the original full sequence by using the reference and the differences encoded in the &lt;code&gt;VCF&lt;/code&gt; file.  I've seen &lt;code&gt;VCF&lt;/code&gt; files in the range of 100Mb, but a reference file is still needed to recover the full genome sequence which is the range of 800Mb+, as mentioned above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're considering just one &quot;whole genome&quot; in isolation, then the answer is pretty clear: &lt;code&gt;2bit&lt;/code&gt; format is probably approaching the entropy limit of the human genome and you probably won't be able to do much better.&#xA;The reason why your question is a bit ambiguous is that as soon as you start encoding more than one genome, a population of genomes, say, then you can start exploiting the redundancy of the genome as shared by the population.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, say you want to store two &quot;whole genomes&quot;.  You could download the &lt;code&gt;hg19&lt;/code&gt; reference and download two &lt;code&gt;VCF&lt;/code&gt; files which would give around 1Gb worth of data (around 800Mb for the &lt;code&gt;2bit&lt;/code&gt; file and around 200Mb for both of the &lt;code&gt;VCF&lt;/code&gt; files).  Now you've been able to represent a &quot;whole genome&quot; in 500Mb instead of the 800Mb.  You can see a similar argument for downloading 3 &lt;code&gt;VCF&lt;/code&gt; files and more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The minimum amount of information needed to represent a population of genomes is, as far as I know, unknown, but I would guess in the 2.5Mb-5Mb range.  For example, see &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/25/2/274/218156/Human-genomes-as-email-attachments&quot; rel=&quot;noreferrer&quot;&gt;&quot;Human genomes as email attachments&quot; by Christley, Lu, Li and Xie&lt;/a&gt; which claims a 4Mb encoding of a genome.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things get tricky because you have to ask what you're claiming as a &quot;whole genome&quot;.  &lt;code&gt;VCF&lt;/code&gt; files are notoriously bad because older versions of the specification only store high quality differences from reference, throwing away high quality called sections.  If you want to store low quality information, the encoding is now going to depend on the sequencing technology in weird ways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Insertions, deletions, mobile insertion elements, copy number variants, other structural variants, etc. all complicate this matter further.  &lt;a href=&quot;http://biorxiv.org/content/early/2017/01/18/101378&quot; rel=&quot;noreferrer&quot;&gt;Genome Graphs&lt;/a&gt; are trying to tackle at least some of these problems but the focus is on variant calling rather than efficient individual whole genome representation, though perhaps can be adapted in the future.&lt;/p&gt;&#xA;" OwnerUserId="113" LastEditorUserId="113" LastEditDate="2017-05-19T10:36:42.200" LastActivityDate="2017-05-19T10:36:42.200" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="44" PostTypeId="2" ParentId="12" CreationDate="2017-05-17T03:45:52.300" Score="1" Body="&lt;p&gt;While I have no experience with this &lt;em&gt;specific&lt;/em&gt; question you have, according to Feinstein &amp;amp; Brylinski (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4468813/&quot; rel=&quot;nofollow noreferrer&quot;&gt;2015&lt;/a&gt;), a &quot;fully automated procedure&quot;, i.e. a Perl script, can be used to optimize the box size itself, and it can be found &lt;a href=&quot;http://brylinski.cct.lsu.edu/content/docking-box-size&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="172" LastEditorUserId="172" LastEditDate="2017-05-17T04:01:09.563" LastActivityDate="2017-05-17T04:01:09.563" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="45" PostTypeId="1" AcceptedAnswerId="68" CreationDate="2017-05-17T04:38:52.420" Score="27" ViewCount="9268" Body="&lt;p&gt;I have a single ~10GB FASTA file generated from an Oxford Nanopore Technologies' MinION run, with &amp;gt;1M reads of mean length ~8Kb. How can I quickly and efficiently calculate the distribution of read lengths?&lt;/p&gt;&#xA;&lt;p&gt;A naive approach would be to read the FASTA file in Biopython, check the length of each sequence, store the lengths in a numpy array and plot the results using matplotlib, but this seems like reinventing the wheel.&lt;/p&gt;&#xA;&lt;p&gt;Many solutions that work for short reads are inadequate for long reads. If I'm hey output a single (text) line per 1/10 bases, which would lead to a text output of upwards of 10,000 lines (and potentially more than 10x that) for a long read fasta.&lt;/p&gt;&#xA;" OwnerUserId="163" LastEditorUserId="73" LastEditDate="2021-06-28T22:59:24.490" LastActivityDate="2021-06-28T22:59:24.490" Title="Read length distribution from FASTA file" Tags="|fasta|nanopore|" AnswerCount="7" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="46" PostTypeId="1" AcceptedAnswerId="47" CreationDate="2017-05-17T04:46:54.870" Score="4" ViewCount="16679" Body="&lt;p&gt;I have a computer engineering background, not biology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I started working on a bioinformatics project recently, which involves &lt;em&gt;de-novo&lt;/em&gt; assembly. I came to know the terms &lt;code&gt;Transcriptome&lt;/code&gt; and &lt;code&gt;Genome&lt;/code&gt;, but I cannot identify the difference between these two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know a transcriptome is the set of all messenger RNA molecules in a cell, but am not sure how this is different from a genome.&lt;/p&gt;&#xA;" OwnerUserId="173" LastEditorUserId="131" LastEditDate="2017-06-21T10:24:47.297" LastActivityDate="2017-06-21T10:24:47.297" Title="What is the difference between a transcriptome and a genome?" Tags="|genome|transcriptome|" AnswerCount="2" CommentCount="4" ClosedDate="2017-07-11T10:57:28.610" ContentLicense="CC BY-SA 3.0" />
  <row Id="47" PostTypeId="2" ParentId="46" CreationDate="2017-05-17T04:52:48.293" Score="18" Body="&lt;p&gt;In brief, the  “genome”  is the collection of all  DNA  present  in  the  nucleus  and  the  mitochondria of a  somatic  cell. The initial product of genome expression is the “transcriptome”, a collection of RNA molecules derived from those genes.&lt;/p&gt;&#xA;" OwnerUserId="23" LastEditorUserId="141" LastEditDate="2017-05-18T00:01:35.137" LastActivityDate="2017-05-18T00:01:35.137" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="49" PostTypeId="2" ParentId="45" CreationDate="2017-05-17T05:09:23.507" Score="5" Body="&lt;p&gt;There are several potential approaches. For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://biopython.org/DIST/docs/tutorial/Tutorial.html#htoc295&quot; rel=&quot;noreferrer&quot;&gt;histogram of sequence lengths&lt;/a&gt; in the Biopython tutorial&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/maasha/biopieces/wiki/plot_distribution&quot; rel=&quot;noreferrer&quot;&gt;plot_distribution&lt;/a&gt; from the Ruby-based biopieces framework&lt;/li&gt;&#xA;&lt;li&gt;various solutions to get sequence length &lt;a href=&quot;https://gif.biotech.iastate.edu/calculate-sequence-lengths-fasta-file&quot; rel=&quot;noreferrer&quot;&gt;including bioawk&lt;/a&gt; and &lt;a href=&quot;https://www.biostars.org/p/118954/#119066&quot; rel=&quot;noreferrer&quot;&gt;EMBOSS infoseq&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As to which of these are &quot;quick and efficient&quot; using a 10 GB file...it's hard to say in advance. You may have to try and benchmark a few of them.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2017-05-17T05:09:23.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="50" PostTypeId="2" ParentId="46" CreationDate="2017-05-17T05:13:07.323" Score="5" Body="&lt;p&gt;They are two very different things. Your genome is a large section of about 3 billion DNA nucleotide bases. It has no concept of exon and introns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Transcriptome is a study of &lt;a href=&quot;https://en.wikipedia.org/wiki/Transcription_(biology)&quot; rel=&quot;noreferrer&quot;&gt;transcriptions&lt;/a&gt;. You have introns and exons. We can now talk about alternative splicing and gene expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can think your genome is like a cooking recipe. While it's good to have a good recipe, you can't do much if you don't use it for cooking. &lt;/p&gt;&#xA;" OwnerUserId="174" LastActivityDate="2017-05-17T05:13:07.323" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="51" PostTypeId="1" AcceptedAnswerId="59" CreationDate="2017-05-17T05:19:48.690" Score="15" ViewCount="1871" Body="&lt;p&gt;I want to focus on transcriptome analysis. We know it's possible to analyze RNA-Seq experiment based on alignment or k-mers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Possible alignment workflow:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Align sequence reads with &lt;a href=&quot;https://ccb.jhu.edu/software/tophat/index.shtml&quot; rel=&quot;noreferrer&quot;&gt;TopHat2&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Quantify the gene expression with &lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/install/&quot; rel=&quot;noreferrer&quot;&gt;Cufflinks&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Possible reference-free workflow:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Quantify sequence reads with &lt;a href=&quot;https://pachterlab.github.io/kallisto/&quot; rel=&quot;noreferrer&quot;&gt;Kallisto&lt;/a&gt; reference-free index&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Both strategy generate gene expression table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q:&lt;/strong&gt; What are pros and cons for each of the approach? Can you give guideline?&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="174" LastEditDate="2017-05-17T05:40:44.417" LastActivityDate="2017-05-18T00:01:14.437" Title="Alignment based vs reference-free (transcriptome analysis)?" Tags="|transcriptome|rna-seq|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="52" PostTypeId="1" AcceptedAnswerId="54" CreationDate="2017-05-17T05:24:25.273" Score="13" ViewCount="3988" Body="&lt;p&gt;&lt;a href=&quot;https://www.thermofisher.com/order/catalog/product/4456740&quot; rel=&quot;noreferrer&quot;&gt;ERCC spike-in&lt;/a&gt; is a set of synthetic controls developed for RNA-Seq. I'm interested in using it to normalize my RNA-Seq samples. In particular, I'd like to use the spike-ins to remove technical bias and any variation that should not be part of my analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The site doesn't give any details on how I can do that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q:&lt;/strong&gt; What are the possible normalization strategies? Can you briefly describe them?&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="203" LastEditDate="2017-07-28T11:18:36.177" LastActivityDate="2017-08-01T22:16:10.137" Title="Normalization methods with RNA-Seq ERCC spike in?" Tags="|rna-seq|transcriptome|normalization|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="54" PostTypeId="2" ParentId="52" CreationDate="2017-05-17T05:43:40.693" Score="9" Body="&lt;p&gt;You may consider using &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/RUVSeq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;RUVSeq&lt;/a&gt;. Here is an excerpt from the &lt;a href=&quot;http://www.nature.com/nbt/journal/v32/n9/full/nbt.2931.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;2013 Nature Biotechnology publication&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We evaluate the performance of the External RNA Control Consortium (ERCC) spike-in controls and investigate the possibility of using them directly for normalization. We show that the spike-ins are not reliable enough to be used in standard global-scaling or regression-based normalization procedures. We propose a normalization strategy, called remove unwanted variation (RUV), that adjusts for nuisance technical effects by performing factor analysis on suitable sets of control genes (e.g., ERCC spike-ins) or samples (e.g., replicate libraries).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;RUVSeq essentially fits a generalized linear model (GLM) to the expression data, where your expression matrix $Y$ is a $m$ by $n$ matrix, where $m$ is the number of samples and $n$ the number of genes. The model boils down to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$Y = X*\beta + Z*\gamma + W*\alpha + \epsilon$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $X$ describes the conditions of interest (e.g., treatment vs. control), $Z$ describes observed covariates (e.g., gender) and $W$ describes unobserved covariates (e.g., batch, temperature, lab). $\beta$, $\gamma$ and $\alpha$ are parameter matrices which record the contribution of $X$, $Z$ and $W$, and $\epsilon$ is random noise. For subset of carefully selected genes (e.g., ERCC spike-ins, housekeeping genes, or technical replicates) we can assume that $X$ and $Z$ are zero, and find $W$ - the &quot;unwanted variation&quot; in your sample. &lt;/p&gt;&#xA;" OwnerUserId="163" LastEditorUserId="163" LastEditDate="2017-07-28T02:29:08.227" LastActivityDate="2017-07-28T02:29:08.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="55" PostTypeId="2" ParentId="51" CreationDate="2017-05-17T06:29:37.357" Score="7" Body="&lt;p&gt;I wouldn't say &lt;a href=&quot;https://pachterlab.github.io/kallisto/&quot; rel=&quot;noreferrer&quot;&gt;Kallisto&lt;/a&gt; (or &lt;a href=&quot;http://salmon.readthedocs.io/en/latest/&quot; rel=&quot;noreferrer&quot;&gt;Salmon&lt;/a&gt;) are reference-free. They use a transcriptome as reference anda concept called &lt;em&gt;pseudo-alignment&lt;/em&gt; which greatly speed up the process of assigning your reads to a transcript.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, both approaches of (i) mapping against a reference genome (what you called &lt;em&gt;alignment workflow&lt;/em&gt; ) and (ii) mapping against a reference transcriptome will serve different purposes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Transcriptome mapping using pseudoalignent is becoming the method of choice for gene/transcript quantification and differential expression analysis. The drawback is that you only focus on known transcripts&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two typical workflows are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Kallisto followed by &lt;a href=&quot;http://pachterlab.github.io/sleuth/&quot; rel=&quot;noreferrer&quot;&gt;sleuth&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Salmon, followed by tximport and DESeq2/EdgeR&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Genome mapping is useful for, per example discovery of new isoforms. You shouldn't use TopHat anymore as it has been discontinued by the author.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A typical workflow would be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Hisat2 (alignment)&lt;/li&gt;&#xA;&lt;li&gt;StringTie (transcript assembly and abundance estimation)&lt;/li&gt;&#xA;&lt;li&gt;Ballgown (differential expression)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="179" LastEditorUserId="179" LastEditDate="2017-05-17T08:11:35.430" LastActivityDate="2017-05-17T08:11:35.430" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="56" PostTypeId="2" ParentId="45" CreationDate="2017-05-17T06:38:49.653" Score="2" Body="&lt;p&gt;It is not exactly what you asked, but you can generate a histogram of read length distribution of your nanopore data directly from the HDF5 files using &lt;a href=&quot;https://poretools.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;poretools&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="179" LastActivityDate="2017-05-17T06:38:49.653" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="57" PostTypeId="1" AcceptedAnswerId="80" CreationDate="2017-05-17T07:03:57.863" Score="9" ViewCount="230" Body="&lt;p&gt;A popular framework to analyze differences between groups, either experiments or diseases, in transcriptomics is using linear models (&lt;a href=&quot;http://bioconductor.org/packages/limma&quot; rel=&quot;noreferrer&quot;&gt;limma&lt;/a&gt; is a popular choice). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance we have a disease D with three stages as defined by clinicians, A, B and C. 10 samples each stage and the healthy H to compare with is RNA-sequenced. A typical linear model would be to observe the three stages&lt;code&gt;~A+B+C&lt;/code&gt; independently. The data of each stage is not from the same person. (but for the question assume it isn't)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding is that such a model would not take into account that stage C appears only on 30% of patients in stage B. And that a healthy patient upon external factors can jump to stage B. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we want to find the role of a gene in the disease we should include somehow this information in the model. Which makes me think about mixing linear models and hidden Markov chains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can such a disease be described in terms of linear models with such data and information?&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-05-17T13:04:24.530" LastActivityDate="2017-05-19T11:58:27.690" Title="Linear models of complex diseases" Tags="|transcriptome|" AnswerCount="1" CommentCount="16" ContentLicense="CC BY-SA 3.0" />
  <row Id="58" PostTypeId="2" ParentId="45" CreationDate="2017-05-17T07:32:20.717" Score="7" Body="&lt;p&gt;Using Biopython and matplotlib would seem like the way to go, indeed.&#xA;It really just boils down to three lines of code to get that graph:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import Bio, pandas&#xA;lengths = map(len, Bio.SeqIO.parse('/path/to/the/seqs.fasta', 'fasta'))&#xA;pandas.Series(lengths).hist(color='gray', bins=1000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Of course you might want to make a longer script that's callable from the command line, with a couple options. You are welcome to use mine:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python2&#xA;&#xA;&quot;&quot;&quot;&#xA;A custom made script to plot the distribution of lengths&#xA;in a fasta file.&#xA;&#xA;Written by Lucas Sinclair.&#xA;Kopimi.&#xA;&#xA;You can use this script from the shell like this:&#xA;$ ./fastq_length_hist --input seqs.fasta --out seqs.pdf&#xA;&quot;&quot;&quot;&#xA;&#xA;###############################################################################&#xA;# Modules #&#xA;import argparse, sys, time, getpass, locale&#xA;from argparse import RawTextHelpFormatter&#xA;from Bio import SeqIO&#xA;import pandas&#xA;&#xA;# Matplotlib #&#xA;import matplotlib&#xA;matplotlib.use('Agg', warn=False)&#xA;from matplotlib import pyplot&#xA;&#xA;################################################################################&#xA;desc = &quot;fasta_length_hist v1.0&quot;&#xA;parser = argparse.ArgumentParser(description=desc, formatter_class=RawTextHelpFormatter)&#xA;&#xA;# All the required arguments #&#xA;parser.add_argument(&quot;--input&quot;, help=&quot;The fasta file to process&quot;, type=str)&#xA;parser.add_argument(&quot;--out&quot;, type=str)&#xA;&#xA;# All the optional arguments #&#xA;parser.add_argument(&quot;--x_log&quot;, default=True, type=bool)&#xA;parser.add_argument(&quot;--y_log&quot;, default=True, type=bool)&#xA;&#xA;# Parse it #&#xA;args        = parser.parse_args()&#xA;input_path  = args.input&#xA;output_path = args.out&#xA;x_log       = bool(args.x_log)&#xA;y_log       = bool(args.y_log)&#xA;&#xA;################################################################################&#xA;# Read #&#xA;lengths = map(len, SeqIO.parse(input_path, 'fasta'))&#xA;&#xA;# Report #&#xA;sys.stderr.write(&quot;Read all lengths (%i sequences)\n&quot; % len(lengths))&#xA;sys.stderr.write(&quot;Longest sequence: %i bp\n&quot; % max(lengths))&#xA;sys.stderr.write(&quot;Shortest sequence: %i bp\n&quot; % min(lengths))&#xA;sys.stderr.write(&quot;Making graph...\n&quot;)&#xA;&#xA;# Data #&#xA;values = pandas.Series(lengths)&#xA;&#xA;# Plot #&#xA;fig   = pyplot.figure()&#xA;axes  = values.hist(color='gray', bins=1000)&#xA;fig   = pyplot.gcf()&#xA;title = 'Distribution of sequence lengths'&#xA;axes.set_title(title)&#xA;axes.set_xlabel('Number of nucleotides in sequence')&#xA;axes.set_ylabel('Number of sequences with this length')&#xA;axes.xaxis.grid(False)&#xA;&#xA;# Log #&#xA;if x_log: axes.set_yscale('symlog')&#xA;if y_log: axes.set_xscale('symlog')&#xA;&#xA;# Adjust #&#xA;width=18.0; height=10.0; bottom=0.1; top=0.93; left=0.07; right=0.98&#xA;fig.set_figwidth(width)&#xA;fig.set_figheight(height)&#xA;fig.subplots_adjust(hspace=0.0, bottom=bottom, top=top, left=left, right=right)&#xA;&#xA;# Data and source #&#xA;fig.text(0.99, 0.98, time.asctime(), horizontalalignment='right')&#xA;fig.text(0.01, 0.98, 'user: ' + getpass.getuser(), horizontalalignment='left')&#xA;&#xA;# Nice digit grouping #&#xA;sep = ('x','y')&#xA;if 'x' in sep:&#xA;    locale.setlocale(locale.LC_ALL, '')&#xA;    seperate = lambda x,pos: locale.format(&quot;%d&quot;, x, grouping=True)&#xA;    axes.xaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(seperate))&#xA;if 'y' in sep:&#xA;    locale.setlocale(locale.LC_ALL, '')&#xA;    seperate = lambda x,pos: locale.format(&quot;%d&quot;, x, grouping=True)&#xA;    axes.yaxis.set_major_formatter(matplotlib.ticker.FuncFormatter(seperate))&#xA;&#xA;# Save it #&#xA;fig.savefig(output_path, format='pdf')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;EDIT - an example output:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5kmaX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5kmaX.png&quot; alt=&quot;Sequence length distribution&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="40" LastEditorUserId="40" LastEditDate="2017-05-17T18:09:52.143" LastActivityDate="2017-05-17T18:09:52.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="59" PostTypeId="2" ParentId="51" CreationDate="2017-05-17T07:58:46.987" Score="11" Body="&lt;p&gt;First of all, I would emphasize that &quot;alignment-free&quot; quantification tools like Salmon and Kallisto are &lt;em&gt;not&lt;/em&gt; reference-free. The basic difference between them and more traditional aligners is that they do not report a specific position (either in a genome or transcriptome) to which a read maps. However, their overall purpose is still to quantify the expression levels (or differences) of a known set of transcripts; hence, they require a reference (which could be arbitrarily defined).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most important criterion for deciding which approach to use (and this is true of almost everything in genomics) is exactly what question you would like to answer. If you are primarily interested in quantifying and comparing expression of mature mRNA from known transcripts, then a transcriptome-based alignment may be fastest and best. However, you may miss potentially interesting features outside of those known transcripts, such as new isoforms, non-coding RNAs, or information about pre-mRNA levels, which can often be gleaned from intronic reads (see the &lt;a href=&quot;https://www.nature.com/nbt/journal/v33/n7/full/nbt.3269.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;EISA&lt;/a&gt; method).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4729156/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; also has some good considerations about which tools may work best depending on the question you want to answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, another fast and flexible aligner (which can be used with or without a reference transcriptome) is &lt;a href=&quot;https://github.com/alexdobin/STAR&quot; rel=&quot;nofollow noreferrer&quot;&gt;STAR&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="182" LastEditorUserId="141" LastEditDate="2017-05-18T00:01:14.437" LastActivityDate="2017-05-18T00:01:14.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="60" PostTypeId="5" CreationDate="2017-05-17T07:58:51.323" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-17T07:58:51.323" LastActivityDate="2017-05-17T07:58:51.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="61" PostTypeId="4" CreationDate="2017-05-17T07:58:51.323" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-17T07:58:51.323" LastActivityDate="2017-05-17T07:58:51.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="62" PostTypeId="1" AcceptedAnswerId="63" CreationDate="2017-05-17T09:15:38.363" Score="6" ViewCount="1033" Body="&lt;p&gt;I would like to test for positive selection in a large set of genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to have a yes/no answer to the question if gene was evolving under positive selection. Therefore I chose model &lt;a href=&quot;https://doi.org/10.1093/molbev/msv035&quot; rel=&quot;noreferrer&quot;&gt;BUSTED&lt;/a&gt; for my analysis, which is implemented in &lt;a href=&quot;http://www.hyphy.org/&quot; rel=&quot;noreferrer&quot;&gt;HYPHY&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I launch HYPHY in the command-line, I get a series of question I  have to answer in the interactive form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there way to perform a batch analysis involving multiple datasets, i.e. many sequence alignments?&lt;/p&gt;&#xA;" OwnerUserId="191" LastEditorUserId="191" LastEditDate="2018-07-19T11:48:05.030" LastActivityDate="2018-07-19T11:48:05.030" Title="How to run HYPHY on multiple files non-interactively?" Tags="|codon|positive-selection|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="63" PostTypeId="2" ParentId="62" CreationDate="2017-05-17T09:15:38.363" Score="5" Body="&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I made the following bash script to generate an input file for HYPHY.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;cat &amp;lt;&amp;lt; EOF&#xA;inputRedirect = {};&#xA;inputRedirect[&quot;01&quot;]=&quot;Universal&quot;; // genetic code&#xA;inputRedirect[&quot;02&quot;]=&quot;$(readlink -f $1)&quot;; // codon data&#xA;inputRedirect[&quot;03&quot;]=&quot;$(readlink -f $2)&quot;; // tree&#xA;inputRedirect[&quot;04&quot;]=&quot;${3:-All}&quot;; // Test for selection on a branch&#xA;inputRedirect[&quot;05&quot;]=&quot;&quot;; // complete selection&#xA;&#xA;&#xA;ExecuteAFile (HYPHY_LIB_DIRECTORY+&quot;TemplateBatchFiles/BUSTED.bf&quot;, inputRedirect);&#xA;EOF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want to test all the branches jointly run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./gen_busted.sh alignment.phy tree.nwk &amp;gt; script.bf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or for a particular branch:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./gen_busted.sh alignment.phy tree.nwk branchLabel &amp;gt; script.bf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;HYPHYMP script.bf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now you can use the script to generate a &lt;code&gt;.bf&lt;/code&gt; file for every sequence alignment you have, and run HYPHY for every file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Longer version&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will describe how to make a similar script for any model, such as MEME, RELAX or PARRIS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, find the &lt;code&gt;.bf&lt;/code&gt; script which performs the analysis. On GNU/Linux it should be at &lt;code&gt;HYPHY_INSTALLATION_PATH/lib/hyphy/TemplateBatchFiles&lt;/code&gt; (it might be different between different OS). In case of BUSTED your script has a name &lt;code&gt;BUSTED.bf&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now run this model in the interactive mode in order to record all the inputs required by the model. Make sure to specify &lt;strong&gt;full paths&lt;/strong&gt; to all the files.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;HYPHYMP HYPHY_INSTALLATION_PATH/lib/hyphy/TemplateBatchFiles/BUSTED.bf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In case of BUSTED the inputs are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Genetic code.&lt;/li&gt;&#xA;&lt;li&gt;Codon data (sequence alignment in phylip format).&lt;/li&gt;&#xA;&lt;li&gt;Tree (in newick format).&lt;/li&gt;&#xA;&lt;li&gt;Which branches to test.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now for every input alignment you need to generate a batch file which looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;inputRedirect = {};&#xA;inputRedirect[&quot;01&quot;]=&quot;Universal&quot;; // genetic code&#xA;inputRedirect[&quot;02&quot;]=&quot;/path/to/alignment.phy&quot;; // codon data&#xA;inputRedirect[&quot;03&quot;]=&quot;/path/to/tree.nwk&quot;; // tree&#xA;inputRedirect[&quot;04&quot;]=&quot;All&quot;; // Test for selection on all branches&#xA;inputRedirect[&quot;05&quot;]=&quot;BRANCH1&quot;; // Test for selection on branch1 &#xA;inputRedirect[&quot;06&quot;]=&quot;BRANCH2&quot;; // Test for selection on branch2 &#xA;inputRedirect[&quot;07&quot;]=&quot;&quot;; // complete selection&#xA;&#xA;&#xA;ExecuteAFile (HYPHY_LIB_DIRECTORY+&quot;TemplateBatchFiles/BUSTED.bf&quot;, inputRedirect);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now you can create a script which generates the file. The most important thing, don't forget to use full paths in the &lt;code&gt;.bf&lt;/code&gt; file.&lt;/p&gt;&#xA;" OwnerUserId="191" LastEditorUserId="191" LastEditDate="2017-05-18T17:39:37.100" LastActivityDate="2017-05-18T17:39:37.100" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="64" PostTypeId="2" ParentId="21" CreationDate="2017-05-17T09:34:08.350" Score="24" Body="&lt;p&gt;To add to &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/38/203&quot;&gt;rightskewed answer&lt;/a&gt;:&#xA;While it is true that:&lt;/p&gt;&#xA;&lt;p&gt;Gencode is an additive set of annotation (the manual one done by Havana and an automated one done by Ensembl),&lt;/p&gt;&#xA;&lt;p&gt;the annotation (GTF) files are quite similar for a few exceptions involving the X chromosome and Y par and additional remarks in the Gencode file (see more at &lt;a href=&quot;https://www.gencodegenes.org/faq.html&quot; rel=&quot;noreferrer&quot;&gt;FAQ - Gencode&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What are the actual differences between different annotation databases?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;They are a few differences, but the main one  &lt;strong&gt;for me (and it could be stupid)&lt;/strong&gt; is&lt;/p&gt;&#xA;&lt;p&gt;that &lt;strong&gt;Refseq is developed by the American NCBI&lt;/strong&gt; and&lt;/p&gt;&#xA;&lt;p&gt;the &lt;strong&gt;ENSEMBL is mainly developed by the European EMBL-EBI.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Often, labs or people will just start using what is the best known to them (because of a course or workshop) or because they start working with one of the databases with one specific tool and keep with it later.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;My lab, for reasons still unknown to me, prefers Ensembl annotations (we're working with transcript/exon expression estimation), while some software ship with RefSeq annotations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Your lab might be mostly European based people or they might also have read papers like the one from Frankish et al.  Comparison of GENCODE and RefSeq gene annotation and the impact of reference geneset on variant effect prediction. BMC Genomics 2015; 16(Suppl 8):S2 - DOI: 10.1186/1471-2164-16-S8-S2&lt;/p&gt;&#xA;&lt;p&gt;From the &lt;a href=&quot;https://bmcgenomics.biomedcentral.com/articles/10.1186/1471-2164-16-S8-S2&quot; rel=&quot;noreferrer&quot;&gt;Frankish et al. paper&lt;/a&gt; paper:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The GENCODE Comprehensive transcripts contain more exons, have greater genomic coverage and capture many more variants than RefSeq in both genome and exome datasets, while the GENCODE Basic set shows a higher degree of concordance with RefSeq and has fewer unique features.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As for:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Are there significant differences between them today, or are they, for all intents and purposes, interchangeable (e.g., are exon coordinates between RefSeq and Ensembl annotations interchangeable)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No. I don't think they are great differences between them as that the global picture should stay the same (although you will see different results if you are interested in a small set of genes). However, &lt;strong&gt;they are not directly interchangeable&lt;/strong&gt;. Particularly as there are many versions of Ensembl and Refseq based on different genome annotations (and those won't be interchangeable between themselves either in most cases).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;However, you can easily translate&lt;/strong&gt; most[1] of your Refseq IDs to ENSEMBL IDs and vice-versa with tools as &lt;a href=&quot;http://www.ensembl.org/biomart/martview&quot; rel=&quot;noreferrer&quot;&gt;http://www.ensembl.org/biomart/martview&lt;/a&gt; for example (there are devoted libraries/API as well like &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/biomaRt.html&quot; rel=&quot;noreferrer&quot;&gt;Biocondutor: biomaRt&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;[1] Most as sometimes, they might be annotated in one of the database but haven't (yet) an equivalent in the other.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In fine, even if people tends to keep to what they are used to (and that the annotations are constantly expanded and corrected) depending on the research subject one might be interested in using one database over another:&lt;/p&gt;&#xA;&lt;p&gt;From &lt;a href=&quot;https://bmcgenomics.biomedcentral.com/articles/10.1186/s12864-015-1308-8&quot; rel=&quot;noreferrer&quot;&gt;Zhao S, Zhang B. A comprehensive evaluation of ensembl, RefSeq, and UCSC annotations in the context of RNA-seq read mapping and gene quantification. BMC Genomics. 2015;16: 97.&lt;/a&gt; paper:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;When choosing an annotation database, researchers should keep in mind that no database is perfect and some gene annotations might be inaccurate or entirely wrong. [..] Wu et al. [27] suggested that when conducting research that emphasizes reproducible and robust gene expression estimates, a less complex genome annotation, such as RefGene, might be preferred. When conducting more exploratory research, a more complex genome annotation, such as Ensembl, should be chosen.&lt;/p&gt;&#xA;&lt;p&gt;[..]&lt;/p&gt;&#xA;&lt;p&gt;[27] Wu P-Y, Phan JH, Wang MD. Assessing the impact of human genome annotation choice on RNA-seq expression estimates. BMC Bioinformatics. 2013;14(Suppl 11):S8. doi: 10.1186/1471-2105-14-S11-S8.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="203" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-05-17T10:16:19.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="65" PostTypeId="2" ParentId="19" CreationDate="2017-05-17T10:18:12.290" Score="4" Body="&lt;p&gt;At its easiest, you just store the &lt;em&gt;forward&lt;/em&gt; ($F$) and &lt;em&gt;reverse&lt;/em&gt; ($R$) hash value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You update the forward hash value by conventional means, e.g. bit-shifting the base value into its lower bits:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;F_{n + 1} = ((F_n \ll B \mid x) \mathop\&amp;amp; M,&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$B$ is the bit size of the base encoding, $M$ is the word mask for a word of length $W$ bits, and can be omitted if the width of the hash is the width of the machine word; and the reverse hash value by doing the mathematical inverse, e.g. bit-shifting the value of the  reverse complement of the base into the upper bits:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;R_{n + 1} = ((R_n \gg B) \mid (\operatorname{compl}x \ll (W - B))) \mathop\&amp;amp; M.&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then you compute a &lt;em&gt;combined hash&lt;/em&gt; value by xoring the two hashes, $H  = F \oplus R$.&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2017-07-26T16:10:55.117" LastActivityDate="2017-07-26T16:10:55.117" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="66" PostTypeId="1" AcceptedAnswerId="69" CreationDate="2017-05-17T10:28:32.203" Score="17" ViewCount="11319" Body="&lt;p&gt;I have the following data of fragment counts for each gene in 16 samples:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; str(expression)&#xA;'data.frame':   42412 obs. of  16 variables:&#xA; $ sample1 : int  4555 49 122 351 53 27 1 0 0 2513 ...&#xA; $ sample2 : int  2991 51 55 94 49 10 55 0 0 978 ...&#xA; $ sample3 : int  3762 28 136 321 94 12 15 0 0 2181 ...&#xA; $ sample4 : int  4845 43 193 361 81 48 9 0 0 2883 ...&#xA; $ sample5 : int  2920 24 104 151 50 20 32 0 0 1743 ...&#xA; $ sample6 : int  4157 11 135 324 58 26 4 0 0 2364 ...&#xA; $ sample7 : int  3000 19 155 242 57 12 18 2 0 1946 ...&#xA; $ sample8 : int  5644 30 227 504 91 37 11 0 0 2988 ...&#xA; $ sample9 : int  2808 65 247 93 272 38 1108 1 0 1430 ...&#xA; $ sample10: int  2458 37 163 64 150 29 729 2 1 1049 ...&#xA; $ sample11: int  2064 30 123 51 142 23 637 0 0 1169 ...&#xA; $ sample12: int  1945 63 209 40 171 41 688 3 2 749 ...&#xA; $ sample13: int  2015 57 432 82 104 47 948 4 0 1171 ...&#xA; $ sample14: int  2550 54 177 59 201 36 730 0 0 1474 ...&#xA; $ sample15: int  2425 90 279 73 358 34 1052 3 3 1027 ...&#xA; $ sample16: int  2343 56 365 67 161 43 877 3 1 1333 ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How do I compute RPKM values from these?&lt;/p&gt;&#xA;" OwnerUserId="191" LastEditorUserId="29" LastEditDate="2018-05-09T20:16:53.577" LastActivityDate="2018-07-26T14:43:27.470" Title="How to compute RPKM in R?" Tags="|rna-seq|r|transcriptome|fpkm|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="67" PostTypeId="2" ParentId="66" CreationDate="2017-05-17T10:28:32.203" Score="8" Body="&lt;p&gt;RPKM is defined as:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;RPKM =   numberOfReads / ( geneLength/1000 * totalNumReads/1,000,000 )&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As you can see, you need to have gene lengths for every gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say &lt;code&gt;geneLength&lt;/code&gt; is a vector which have the same number of rows as your &lt;code&gt;data.frame&lt;/code&gt;, and every value of the vector corresponds to a gene (row) in &lt;code&gt;expression&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;expression.rpkm &amp;lt;- data.frame(sapply(expression, function(column) 10^9 * column / geneLength / sum(column)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Regarding numerical stability&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is suggested in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/69/191&quot;&gt;one of the answers&lt;/a&gt;, that all the computations should be performed in a log-transformed scale. In my opinion there is absolutely no reason for doing that. IEEE &lt;a href=&quot;https://en.wikipedia.org/wiki/Double-precision_floating-point_format&quot; rel=&quot;nofollow noreferrer&quot;&gt;binary64&lt;/a&gt; stores a number as binary number 1.b_{51}b{50}...b_0 times 2^{e-1023}. The &lt;em&gt;relative&lt;/em&gt; precision doesn't depend on the exponent value given that a number is in the range [~10^-308; 10^308].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case of RPKM we can only get out of the range if total number of reads is around 10^300, which is not realistic at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the bright site there is not much harm in doing computations in the log-scale either. Worst case you loose a bit of precision.&lt;/p&gt;&#xA;" OwnerUserId="191" LastEditorUserId="191" LastEditDate="2018-07-26T14:43:27.470" LastActivityDate="2018-07-26T14:43:27.470" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="68" PostTypeId="2" ParentId="45" CreationDate="2017-05-17T10:43:50.363" Score="20" Body="&lt;p&gt;If you want something quick and dirty you could rapidly index the FASTA with &lt;code&gt;samtools faidx&lt;/code&gt; and then put the lengths column through R (other languages are available) on the command line.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools faidx $fasta&#xA;cut -f2 $fasta.fai | Rscript -e 'data &amp;lt;- as.numeric (readLines (&quot;stdin&quot;)); summary(data); hist(data)'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This outputs a statistical summary, and creates a PDF in the current directory called Rplots.pdf, containing a histogram.&lt;/p&gt;&#xA;" OwnerUserId="215" LastActivityDate="2017-05-17T10:43:50.363" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="69" PostTypeId="2" ParentId="66" CreationDate="2017-05-17T10:46:41.653" Score="31" Body="&lt;p&gt;First off,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Don’t use RPKMs&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://haroldpimentel.wordpress.com/2014/05/08/what-the-fpkm-a-review-rna-seq-expression-units/&quot; rel=&quot;noreferrer&quot;&gt;They are truly deprecated&lt;/a&gt; because they’re confusing once it comes to paired-end reads. If anything, use &lt;em&gt;FPKM&lt;/em&gt;s, which are mathematically the same but use a more correct name (do we count paired reads separately? No, we count &lt;em&gt;fragments&lt;/em&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even better, &lt;a href=&quot;http://rpubs.com/klmr/rnaseq-norm&quot; rel=&quot;noreferrer&quot;&gt;use TPM (= transcripts per million), or an appropriate cross-library normalisation method&lt;/a&gt;. TMP is defined as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\text{TPM}_\color{orchid}i =&#xA;    {\color{dodgerblue}{\frac{x_\color{orchid}i}{{l_\text{eff}}_\color{orchid}i}}}&#xA;    \cdot&#xA;    \frac{1}{\sum_\color{tomato}j \color{dodgerblue}{\frac{x_\color{tomato}j}{{l_\text{eff}}_\color{tomato}j}}}&#xA;    \cdot&#xA;    \color{darkcyan}{10^6}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$\color{orchid}i$: transcript index,&lt;/li&gt;&#xA;&lt;li&gt;$x_i$: transcript raw count,&lt;/li&gt;&#xA;&lt;li&gt;$\color{tomato}j$ iterates over all (known) transcripts,&lt;/li&gt;&#xA;&lt;li&gt;$\color{dodgerblue}{\frac{x_k}{{l_\text{eff}}_k}}$: rate of fragment coverage per nucleobase ($l_\text{eff}$ being the &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/367/29&quot;&gt;effective length&lt;/a&gt;),&lt;/li&gt;&#xA;&lt;li&gt;$\color{darkcyan}{10^6}$: scaling factor (= “per millions”).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;That said, FPKM an be calculated in R as follows. Note that most of the calculation happens in log transformed number space, to avoid &lt;a href=&quot;https://en.wikipedia.org/wiki/Numerical_stability&quot; rel=&quot;noreferrer&quot;&gt;numerical instability&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;fpkm = function (counts, effective_lengths) {&#xA;    exp(log(counts) - log(effective_lengths) - log(sum(counts)) + log(1E9))&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here, the &lt;em&gt;effective length&lt;/em&gt; is the transcript length minus the mean fragment length plus 1; that is, all the possible positions of an average fragment inside the transcript, which equals the number of all distinct fragments that can be sampled from a transcript.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This function handles &lt;em&gt;one&lt;/em&gt; library at a time. I (&lt;a href=&quot;http://varianceexplained.org/r/tidy-genomics-biobroom/&quot; rel=&quot;noreferrer&quot;&gt;and others&lt;/a&gt;) argue that this is the way functions should be written. If you want to apply the code to multiple libraries, nothing is easier using &lt;a href=&quot;http://dplyr.tidyverse.org/&quot; rel=&quot;noreferrer&quot;&gt;‹dplyr›&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tidy_expression = tidy_expression %&amp;gt;%&#xA;    group_by(Sample) %&amp;gt;%&#xA;    mutate(FPKM = fpkm(Count, col_data$Lengths))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, the data in the question isn’t in tidy data format, so we first need to transform it accordingly using &lt;a href=&quot;http://tidyr.tidyverse.org/&quot; rel=&quot;noreferrer&quot;&gt;‹tidyr›&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tidy_expression = expression %&amp;gt;%&#xA;    gather(Sample, Count)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This equation fails if all your counts are zero; instead of zeros you will get a vector of NaNs. You might want to account for that.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;And I mentioned that TPMs are superior, so here’s their function as well:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;tpm = function (counts, effective_lengths) {&#xA;    rate = log(counts) - log(effective_lengths)&#xA;    exp(rate - log(sum(exp(rate))) + log(1E6))&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2017-07-26T16:13:26.200" LastActivityDate="2017-07-26T16:13:26.200" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="70" PostTypeId="1" AcceptedAnswerId="77" CreationDate="2017-05-17T11:51:45.250" Score="12" ViewCount="526" Body="&lt;p&gt;I have two annotations of the same genome generated with different annotation pipelines. I want to identify overlapping gene models. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;An important feature of this genome is that there are many 'genes within genes', i.e. a genemodel in the intron of another genemodel&lt;/em&gt;. Therefore, I only want to count two genemodels as overlapping when their coding sequence exon annotations overlap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using something like &lt;code&gt;bedtools intersect&lt;/code&gt; it is straightforward to calculate overlap between the gene-level annotations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However: I am not sure how to select genes as overlapping when only their coding sequence exons (CDS features) overlap.&lt;/p&gt;&#xA;" OwnerUserId="196" LastActivityDate="2017-05-17T14:56:36.030" Title="How to calculate overlapping genes between two genome annotation versions" Tags="|sequence-annotation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="71" PostTypeId="5" CreationDate="2017-05-17T11:57:13.740" Score="0" Body="&lt;h1&gt;R Programming Language&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.r-project.org&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;R project homepage&quot;&gt;R&lt;/a&gt; is a free, open-source programming language and software environment for &lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_statistics&quot; rel=&quot;nofollow noreferrer&quot;&gt;statistical computing&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Bioinformatics&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioinformatics&lt;/a&gt;, and &lt;a href=&quot;https://en.wikipedia.org/wiki/Graphics&quot; rel=&quot;nofollow noreferrer&quot;&gt;graphics&lt;/a&gt;. It is a multi-paradigm language and dynamically typed. R is an implementation of the &lt;a href=&quot;http://lib.stat.cmu.edu/S/Spoetry/Tutor/slanguage.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;S programming language&lt;/a&gt; combined with lexical scoping semantics inspired by &lt;a href=&quot;http://stackoverflow.com/tags/scheme/info&quot; title=&quot;Scheme on StackOverflow&quot;&gt;Scheme&lt;/a&gt;.&lt;br&gt;&#xA;R was created by &lt;a href=&quot;http://www.stat.auckland.ac.nz/~ihaka/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ross Ihaka&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Robert_Gentleman_%28statistician%29&quot; rel=&quot;nofollow noreferrer&quot;&gt;Robert Gentleman&lt;/a&gt; and is now developed by the &lt;a href=&quot;http://www.r-project.org/contributors.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;R Development Core Team&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The R environment is easily extended through a packaging system on &lt;a href=&quot;http://cran.r-project.org&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;CRAN The Comprehensive R Archive Network&quot;&gt;CRAN&lt;/a&gt;, the Comprehensive R Archive Network, where one can find several bioinformatics packages, and &lt;a href=&quot;http://bioconductor.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioconductor&lt;/a&gt; which is focused in bioinformatics.&lt;/p&gt;&#xA;" OwnerUserId="131" LastEditorUserId="298" LastEditDate="2018-03-15T16:21:00.787" LastActivityDate="2018-03-15T16:21:00.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="72" PostTypeId="4" CreationDate="2017-05-17T11:57:13.740" Score="0" Body="Please supplement your question with a minimal reproducible example. Use dput() for data and specify all non-base packages with library calls. For statistical questions please use http://stats.stackexchange.com." OwnerUserId="131" LastEditorUserId="48" LastEditDate="2018-03-15T16:21:31.320" LastActivityDate="2018-03-15T16:21:31.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="73" PostTypeId="1" AcceptedAnswerId="76" CreationDate="2017-05-17T13:02:17.453" Score="20" ViewCount="3844" Body="&lt;p&gt;I am trying to understand the benefits of joint genotyping and would be grateful if someone could provide an argument (ideally mathematically) that would clearly demonstrate the benefit of joint vs. single-sample genotyping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I've gathered from other resources (Biostars, GATK forums, etc.)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Joint-genotyping helps control FDR because errors from individually genotyped samples are added up, and amplified when merging call-sets (by Heng Li on &lt;a href=&quot;https://www.biostars.org/p/10926/&quot; rel=&quot;noreferrer&quot;&gt;https://www.biostars.org/p/10926/&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If someone understands this, can you please clarify what is the difference on the overall FDR rate between the two scenarios (again, with an example ideally)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Greater sensitivity for low-frequency variants - By sharing information across all samples, joint calling makes it possible to “rescue” genotype calls at sites where a carrier has low coverage but other samples within the call set have a confident variant at that location. (from &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/article.php?id=4150&quot; rel=&quot;noreferrer&quot;&gt;https://software.broadinstitute.org/gatk/documentation/article.php?id=4150&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I don't understand how the presence of a confidently called variant at the same locus in another individual can affect the genotyping of an individual with low coverage. Is there some valid argument that allows one to consider reads from another person as evidence of a particular variant in a third person? What are the assumptions for such an argument? What if that person is from a different population with entirely different allele frequencies for that variant?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having read several of the papers (or method descriptions) that describe the latest haplotype-aware SNP calling methods (HaplotypeCaller, freebayes, Platypus) the overall framework seems to be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;ol&gt;&#xA;&lt;li&gt;Establish a prior on the allele frequency distribution at a site of interest using one (or combination) of: non-informative prior, population genetics model-based prior like Wright Fisher, prior based on established variation patterns like dbSNP, ExAC, or gnomAD.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Build a list of plausible haplotypes in a region around the locus of interest using local assembly.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Select haplotype with highest likelihood based on prior and reads data and infer the locus genotype accordingly.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;At which point(s) in the above procedure can information between samples be shared or pooled? Should one not trust the AFS from a large-scale resource like gnomAD much more than the distribution obtained from other samples that are nominally party of the same &quot;cohort&quot; but may have little to do with each other because of different ancestry, for example?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I really want to understand the justifications and benefits offered by multi-sample genotyping and would appreciate your insights.    &lt;/p&gt;&#xA;" OwnerUserId="133" LastActivityDate="2017-05-17T14:08:21.663" Title="Single-sample vs. joint genotyping" Tags="|genotyping|gatk|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="74" PostTypeId="2" ParentId="73" CreationDate="2017-05-17T13:13:18.863" Score="1" Body="&lt;p&gt;The benefit to additional samples is seen in your point 1. The likelihood of making a variant call is a function of (1) the depth of coverage supporting a given variant (ignoring mapping/base quality considerations) and (2) the likelihood of that variant existing given background knowledge. With low depth and no background knowledge, poorly covered variants will be assumed to be sequencing errors. Adding more samples can just serve then to increase the background knowledge on a position.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-05-17T13:13:18.863" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="75" PostTypeId="2" ParentId="21" CreationDate="2017-05-17T13:34:35.107" Score="6" Body="&lt;p&gt;To add practical advice to what others have said:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a practical sense, I think the biggest difference between RefSeq and Ensembl/GENCODE is in the sensitivity/specificity trade off. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ensembl aims more towards the inclusive end, including a far larger number of transcript variants, many of which are only weakly supported.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RefSeq trades some of this sensitivity for specificity - you can be more confident that a RefSeq transcript exists, but less confident that the ReqSeq annotation includes all of the real transcripts for a gene. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2017-05-17T13:34:35.107" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="76" PostTypeId="2" ParentId="73" CreationDate="2017-05-17T14:08:21.663" Score="10" Body="&lt;p&gt;Say you are sequencing to 2X coverage. Suppose at a site, sample S has one reference base and one alternate base. It is hard to tell if this is a sequencing error or a heterozygote. Now suppose you have 1000 other samples, all at 2X read depth. One of them has two ALT bases; 10 of them have one REF and one ALT. It is usually improbable that all these samples have the same sequencing error. Then you can assert sample S has a het. Multi-sample calling helps to increase the sensitivity of not so rare SNPs. Note that what matters here is the assumption of error independency. Ancestry only has a tiny indirect effect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Multi-sample calling penalizes very rare SNPs, in particular singletons. When you care about variants only, this is for good. Naively combining single-sample calls yields a higher error rate. Multi-sample calling also helps variant filtering at a later stage. For example, for a sample sequenced to 30X coverage, you would not know if a site at 45X depth is caused by a potential CNV/mismapping or by statistical fluctuation. When you see 1000 30X samples at 45X depth, you can easily know you are looking at a CNV/systematic mismapping. Multiple samples enhance most statistical signals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Older methods pool all BAMs when calling variants. This is necessary because a single low-coverage sample does not have enough data to recover hidden INDELs. However, this strategy is not that easy to massively parallelized; adding a new sample triggers re-calling, which is very expensive as well. As we are mostly doing high-coverage sequencing these days, the old problem with INDEL calling does not matter now. GATK has this new single-sample calling pipeline where you combine per-sample gVCFs at a later stage. Such sample combining strategy is perhaps the only sensible solution when you are dealing with 100k samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The so-called haplotype based variant calling is a separate question. This type of approach helps to call INDELs, but is not of much relevance to multi-sample calling. Also, of the three variant callers in your question, only GATK (and Scalpel which you have not mentioned) use assembly at large. Freebayes does not. Platypus does but only to a limited extent and does not work well in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess what you really want to talk about is imputation based calling. This approach further improves sensitivity with LD. With enough samples, you can measure the LD between two positions. Suppose at position 1000, you see one REF read and no ALT reads; at position 1500, you see one REF read and two ALT reads. You would not call any SNPs at position 1000 even given multiple samples. However, when you know the two positions are strongly linked and the dominant haplotypes are REF-REF and ALT-ALT, you know the sample under investigation is likely to have a missing ALT allele. LD transfers signals across sites and enhances the power to make correct genotyping calls. Nonetheless, as we are mostly doing high-coverage sequencing nowadays, imputation based methods only have a minor effect and are rarely applied.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-05-17T14:08:21.663" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="77" PostTypeId="2" ParentId="70" CreationDate="2017-05-17T14:56:36.030" Score="6" Body="&lt;p&gt;&lt;strong&gt;Short Answer:&lt;/strong&gt;&#xA;In my opinion, my approach would be to pull out the CDS exons and run bedtools on those.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A Few More Details:&lt;/strong&gt;&#xA;When you pull out the exons, make sure that you assign them all IDs if the don't already have them assigned and record which IDs &quot;belong&quot; to which genes. Now when you get exons that overlap, you know that they are coding and you can tie them back to which genes they originate from.&lt;/p&gt;&#xA;" OwnerUserId="243" LastActivityDate="2017-05-17T14:56:36.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="78" PostTypeId="2" ParentId="66" CreationDate="2017-05-17T15:16:10.923" Score="0" Body="&lt;p&gt;If you are looking for a more visual solution (in addition to the other answers), &lt;a href=&quot;https://docs.gdc.cancer.gov/Data/Bioinformatics_Pipelines/Expression_mRNA_Pipeline/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCI Genomic Data Commons (TCGA repository)&lt;/a&gt; offers a nice formula:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RY53Q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RY53Q.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Where:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;RCg: Number of reads mapped to the gene&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;RCpc: Number of reads mapped to all protein-coding genes&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;RCg75: The 75th percentile read count value for genes in the sample&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;L: Length of the gene in base pairs&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As others have pointed out, FPKMs have some problems. GDC also calculates FPKM-UQ values that are upper quartile normalized. Those are recommended for cross-sample comparison and differential expression analysis.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2017-05-17T15:16:10.923" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="79" PostTypeId="2" ParentId="66" CreationDate="2017-05-17T15:33:34.013" Score="1" Body="&lt;p&gt;If you are planning to do a differential expression analysis, you will probably don't need the RPKM calculation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RPK= No.of Mapped reads/ length of transcript in kb (transcript length/1000)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RPKM = RPK/total no.of reads in million (total no of reads/ 1000000)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The whole formula together:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RPKM = (10^9 * C)/(N * L)&#xA;Where,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C = Number of reads mapped to a gene&lt;/p&gt;&#xA;&#xA;&lt;p&gt;N = Total mapped reads in the experiment&lt;/p&gt;&#xA;&#xA;&lt;p&gt;L = exon length in base-pairs for a gene&lt;/p&gt;&#xA;" OwnerUserId="206" LastActivityDate="2017-05-17T15:33:34.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="80" PostTypeId="2" ParentId="57" CreationDate="2017-05-17T15:55:59.503" Score="4" Body="&lt;p&gt;There are two potential sources of bias in this design.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;We cannot distinguish correlation from causation. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Imagine two cases. In the first, the disease progression is inducing immune response. Later stages will be associated with the higher gene expression levels. In the second scenario, the disease is caused by overexpression of a gene. Later stages will be also associated with the higher expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is typical for observational studies. But I just want to mention that a special care should be taken during interpretation of the results.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;If we are not following our individuals, we cannot distinguish correlation and avoidance.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Let's say the disease is lethal in certain cases; the survival is negatively correlated with the disease stage. Now imagine there is a gene, which causes severe symptoms when highly expressed. On the later stages you will only observe those patients in which the gene was not highly expressed. From that you would conclude that gene expression is decreasing with the disease progression. In reality this gene is very important and causal, you just do not have patients which are alive to see this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is similar to &lt;a href=&quot;https://en.wikipedia.org/wiki/Abraham_Wald&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wald's studies of aircrafts&lt;/a&gt; (&lt;a href=&quot;https://en.wikipedia.org/wiki/Survivorship_bias&quot; rel=&quot;nofollow noreferrer&quot;&gt;Survival bias&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Researchers from the Center for Naval Analyses had conducted a study&#xA;  of the damage done to aircraft that had returned from missions, and&#xA;  had recommended that armor be added to the areas that showed the most&#xA;  damage.&#xA;  Wald proposed that the Navy instead reinforce the areas where the&#xA;  returning aircraft were unscathed, since those were the areas that, if&#xA;  hit, would cause the plane to be lost.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think the second point is crucial, and can and will lead to false conclusions. I do not see immediately how to avoid this bias if you study different individuals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore I suggest that the same individuals are followed for a long time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are different approaches you can use later. For example you can have two-stage procedure:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Identify genes which are differentially expressed between healthy (H) and sick (A, B or C).&lt;/li&gt;&#xA;&lt;li&gt;Build a linear model of disease stage stage ~ gene1 + gene2 + ..., using genes identified at step 1.&lt;/li&gt;&#xA;&lt;li&gt;Similarly build a linear model of survival as a function gene expression.&lt;/li&gt;&#xA;&lt;li&gt;It is possible you can use logistic regression to infer probability of transition from stage B to stage C.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This is not the only possible approach of course. As you suggested, Markov model is also applicable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While following the same individuals it is possible to use continuous-time Markov model for such a study. In this case discrete or continuous measurements are recorded at certain time points, and the model parameters are inferred using maximum likelihood.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not an expert in this field, but I think &lt;a href=&quot;https://dx.doi.org/10.18637/jss.v038.i08&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; describing the &lt;a href=&quot;https://cran.r-project.org/web/packages/msm/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;msm&lt;/a&gt; package for R will be useful. It also supports hidden Markov models.&lt;/p&gt;&#xA;" OwnerUserId="191" LastEditorUserId="191" LastEditDate="2017-05-19T11:58:27.690" LastActivityDate="2017-05-19T11:58:27.690" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="81" PostTypeId="1" AcceptedAnswerId="82" CreationDate="2017-05-17T16:26:16.783" Score="14" ViewCount="2878" Body="&lt;p&gt;I have a FASTA file with 100+ sequences like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Sequence1&#xA;GTGCCTATTGCTACTAAAA ...&#xA;&amp;gt;Sequence2&#xA;GCAATGCAAGGAAGTGATGGCGGAAATAGCGTTA&#xA;......&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also have a text file like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sequence1 40&#xA;Sequence2 30&#xA;......&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to simulate next-generation paired-end reads for all the sequences in my FASTA file. For &lt;code&gt;Sequence1&lt;/code&gt;, I would like to simulate at 40x coverage. For &lt;code&gt;Sequence2&lt;/code&gt;, I would like to simulate at 30x coverage. In other words, I want to control my sequence coverage for each sequence in my simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q:&lt;/strong&gt; What is the simplest way to do that? Any software I should use? Bioconductor?&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="298" LastEditDate="2017-07-05T08:32:11.957" LastActivityDate="2022-03-04T15:29:42.903" Title="How to simulate NGS reads, controlling sequence coverage?" Tags="|fasta|ngs|simulated-data|" AnswerCount="10" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="82" PostTypeId="2" ParentId="81" CreationDate="2017-05-17T16:36:36.997" Score="2" Body="&lt;p&gt;I am not aware of any software that can do this directly, but I would split the fasta file into one sequence per file, loop over them in BASH and invoke ART the sequence simulator (or another) on each sequence.&lt;/p&gt;&#xA;&lt;p&gt;For more information about ART, please see their paper here: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3278762/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3278762/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="31" LastEditorUserId="73" LastEditDate="2022-03-04T09:52:25.653" LastActivityDate="2022-03-04T09:52:25.653" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="83" PostTypeId="1" CreationDate="2017-05-17T16:52:03.917" Score="7" ViewCount="285" Body="&lt;p&gt;This question pertains to iCLIP, but it could just as easily be ChIP-seq or ATAC-seq or mutation frequencies. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have iCLIP read counts across the transcriptome and I wish to know if the signals are correlated - that is, where one of them is high, is the other likely to be high.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Often when dealing with such data (e.g. iCLIP data) we know that the data is generally sparse - that is at most positions both signals are zero and this is correct, and also zero-inflated - that is many bases that &quot;should&quot; have a signal are missing that data. So just calculating the Spearman's correlation is likely to give an artificially low value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What might be a way to asses the association? I should add that the aim is the assess association of binding patterns within genes, rather than (or as well as) between genes.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things I have thought of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Apply some sort of smoothing to the data (eg a rolling mean). Remove any bases with 0 in both samples. Compute the spearmans. &lt;/li&gt;&#xA;&lt;li&gt;Calculate the average pairwise distance between every read in sample one and every read in sample two. Compare this to data where the reads have been randomised within genes. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In the first case removing all bases with 0 in both samples seems wrong. But if 99.99% of all bases have zero in both samples, then this seems like its necessary for Spearman. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the second case, the result seems like it would be non-intuitive to interpret. And also calculating this would be massively computationally intensive. &lt;/p&gt;&#xA;" OwnerUserId="235" LastEditorUserId="235" LastEditDate="2017-05-18T08:41:44.793" LastActivityDate="2017-07-26T16:30:47.080" Title="How to correlate two zero inflated bedgraph-like signals?" Tags="|statistics|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="84" PostTypeId="2" ParentId="45" CreationDate="2017-05-17T17:33:19.747" Score="12" Body="&lt;p&gt;Statistics for nanopore reads are tricky because of the huge range of read lengths that can be present in a single run. I have found that the best way to display lengths is by using a log scale on both the x axis (length) and the y axis (sequenced bases, or counts, depending on preference).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have written my own scripts for doing this: one for generating the read lengths, and another for plotting the length distribution in various ways. The script that generates read lengths also spits out basic length summary statistics to standard error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ~/scripts/fastx-length.pl &amp;gt; lengths_mtDNA_called.txt&#xA;Total sequences: 2110&#xA;Total length: 5.106649 Mb&#xA;Longest sequence: 107.414 kb&#xA;Shortest sequence: 219 b&#xA;Mean Length: 2.42 kb&#xA;Median Length: 1.504 kb&#xA;N50: 336 sequences; L50: 3.644 kb&#xA;N90: 1359 sequences; L90: 1.103 kb&#xA;&#xA;$ ~/scripts/length_plot.r lengths_mtDNA_called.txt&#xA;lengths_mtDNA_called.txt ... done&#xA;Number of sequences: 2110 &#xA;Length quantiles:&#xA;      0%      10%      20%      30%      40%      50%      60%      70% &#xA;   219.0    506.9    724.4    953.0   1196.2   1503.0   1859.2   2347.3 &#xA;     80%      90%     100% &#xA;  3128.2   4804.7 107414.0 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here are a couple of of the produced graphs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CnECX.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CnECX.png&quot; alt=&quot;Digital electrophoresis plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/el6d3.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/el6d3.png&quot; alt=&quot;Read length distribution plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The scripts to generate these can be found here:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/gringer/bioinfscripts/blob/master/fastx-length.pl&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/gringer/bioinfscripts/blob/master/fastx-length.pl&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/gringer/bioinfscripts/blob/master/length_plot.r&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/gringer/bioinfscripts/blob/master/length_plot.r&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-05-17T17:33:19.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="85" PostTypeId="2" ParentId="81" CreationDate="2017-05-17T17:55:24.600" Score="2" Body="&lt;p&gt;If you're ok with some randomness you can generate reads from your sequence file using a Poisson random variable.  You'll need to do some math to figure out what value of lambda to use in order for the expected coverage at each base pair in your read to match what you set in your text file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example you have a sequence S of length 1,000, a read length of 50, and an insert size of 100.  For each base b in S generate a Poisson random variable p.  You will then generate p reads from base b to b+50.  Then, generate the paired read starting at b+50+100.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, you would have to play with it to figure out what lambda to use but this would give you basically what you want, as long as you're ok with not having exactly the coverage you're targeting for each read.&lt;/p&gt;&#xA;" OwnerUserId="59" LastActivityDate="2017-05-17T17:55:24.600" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="86" PostTypeId="2" ParentId="83" CreationDate="2017-05-17T18:04:16.737" Score="4" Body="&lt;p&gt;Honestly I'd just use &lt;code&gt;multiBigwigSummary&lt;/code&gt; and then &lt;code&gt;plotCorrelation&lt;/code&gt; from deepTools for this, but I'm a bit biased. There, the idea would be to consider each gene as a unit (you could instead use bins, but I don't think that would as nicely do what you want), namely by giving the tools a BED or GTF file input. It would then calculate the average signal in each gene/transcript and you could do your spearman's correlation. Features with 0 in all samples could optionally be removed (&lt;code&gt;plotCorrelation --skipZeros&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While you certainly could go the whole 9 yards and use per-base comparisons, that seems a bit overkill and I suspect that it won't really yield appreciably more information (especially when one considers the additional time overhead).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-05-17T18:04:16.737" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="87" PostTypeId="2" ParentId="83" CreationDate="2017-05-17T18:08:38.830" Score="3" Body="&lt;p&gt;Rather than working on the base level, you could probably work on say gene level counts. &lt;a href=&quot;https://en.wikipedia.org/wiki/Kendall_rank_correlation_coefficient&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kendall's tau&lt;/a&gt;, an ordinal association metric, can then be used as an appropriate correlation measure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $X$ and $Y$ are your iCLIP replicates, $i$ represents gene index and $(x_i, y_i)$ represents the number of RBP binding sites in $X$ and $Y$ respectively for the $i^{th}$ gene, Kendall's tau is defined as :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;\frac{\text{#(concordant pairs)} - \text{#(discordant pairs)}}{n(n-1)/2}&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where any two pairs $(x_i, y_i)$ and $(x_j, y_j)$ are concordant if:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$x_i &amp;gt; x_j$ AND $y_i &amp;gt; y_j$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;OR&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$x_i &amp;lt; x_j$ AND $y_i &amp;lt; y_j$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Correspondingly they are discordant if:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$x_i &amp;lt; x_j$ AND $y_i &amp;gt; y_j$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;OR&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;$x_i &amp;gt; x_j$ AND $y_i &amp;lt; y_j$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="161" LastEditorUserId="29" LastEditDate="2017-07-26T16:30:47.080" LastActivityDate="2017-07-26T16:30:47.080" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="88" PostTypeId="2" ParentId="27" CreationDate="2017-05-17T18:56:13.670" Score="4" Body="&lt;p&gt;The &quot;right&quot; solution would be realignment, but that's expensive and most of us would not go that route. My preferred solution would be to convert the bed file, as opposed to the bam. Here's why:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Reheadering the bam means that you may have reads aligned to contigs without a corresponding entry in UCSC (see &lt;a href=&quot;https://github.com/dpryan79/ChromosomeMappings/blob/master/GRCh37_NCBI2UCSC.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;Devon's list for the mappings&lt;/a&gt;). This is a problem because:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Some of those reads would likely have been mapped elsewhere if a reference without those contigs was used. &lt;/li&gt;&#xA;&lt;li&gt;I'm not even sure what happens to those reads after reheadering - I guess they would need to be marked as unmapped? Lots of potential for screwiness there.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;2) It seems cleaner to convert the bed file from UCSC-&gt;NCBI, where you are guaranteed that every entry has a &quot;home&quot;. Then, after you pull your info from the bam, you can always convert chromosome names back if you need to.&lt;/p&gt;&#xA;" OwnerUserId="74" LastActivityDate="2017-05-17T18:56:13.670" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="89" PostTypeId="1" CreationDate="2017-05-17T19:20:51.663" Score="7" ViewCount="149" Body="&lt;p&gt;I have SNP data from several cultivars of rice which I have used to produce alignments, but I don't think that the usual models and algorithms used for generating phylogenetic trees are appropriate, because these cultivars are not the result of speciation events and have been interbred in their histories. How can I best calculate and visualize their degree of relatedness?&lt;/p&gt;&#xA;" OwnerUserId="83" LastEditorUserId="3848" LastEditDate="2020-04-15T06:31:16.770" LastActivityDate="2020-04-15T06:31:16.770" Title="How does one construct a cladogram of intraspecies relationships?" Tags="|fasta|phylogenetics|snp|phylogeny|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="90" PostTypeId="2" ParentId="83" CreationDate="2017-05-17T20:18:09.070" Score="3" Body="&lt;p&gt;It depends whether you want to treat the peak intensities as binary (comparing presence/absence of peaks in the sets) or continuous (comparing the relative magnitudes of the peaks).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Binary&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For starting out, a simple binary comparison may be appropriate. You can use a &lt;a href=&quot;http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0096303&quot; rel=&quot;nofollow noreferrer&quot;&gt;peak caller&lt;/a&gt; of your choosing to identify peaks in each sample according to your desired criteria. Then you can use a similarity metric such as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Jaccard_index&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jaccard index&lt;/a&gt; to quantify the level of agreement among the peaks in the two samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One potential obstacle is that defining the boundaries of your peaks won't be totally straightforward. For example, a peak in one sample might have 2 overlapping peaks in the other sample, one on each end. A rough solution for this is to divide the genome into bins (maybe around 100-1000 bp, depending on your desired resolution). You can treat a peak as present in a bin if more than half of the peak lies in the bin. That way, bins in one sample can be directly compared to the corresponding bins in the other sample. Obviously, this isn't the only way to do this; other appropriate methods exist too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Continuous&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to treat the peak intensities as continuous, you could apply a similar binning method, taking the &quot;score&quot; of a bin to be the average peak intensity at positions within that bin. You could then throw away all bins with no peaks or only low-intensity peaks throughout the genome. Then you could compute the Spearman's correlation for the remaining bins. I'm guessing it will be harder to find a strong correlation for continuous intensities, because of the amount of experimental variability that is inherently present.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If, after following these steps, the Spearman's correlation is still &quot;artificially low&quot; as you suggested, then this is likely a problem with the underlying data, not the overall analysis; maybe your two datasets actually don't agree that well.&lt;/p&gt;&#xA;" OwnerUserId="177" LastActivityDate="2017-05-17T20:18:09.070" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="91" PostTypeId="1" AcceptedAnswerId="133" CreationDate="2017-05-18T00:14:58.347" Score="18" ViewCount="16069" Body="&lt;p&gt;I have a FASTA file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; Sequence_1&#xA;GCAATGCAAGGAAGTGATGGCGGAAATAGCGTTAGATGTATGTGTAGCGGTCCC...&#xA;&amp;gt; Sequence_2&#xA;GCAATGCAAGGAAGTGATGGCGGAAATAGCGTTAGATGTATGTGTAGCGGTCCC....&#xA;....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to generate a BED file for each sequence like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sequence_1 0 1500&#xA;Sequence_2 0 1700&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The BED regions will simply be the size of the sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q:&lt;/strong&gt; I did that before with a one-line command. I don't remember what that is, it was on Biostars. I can't find the post now. What's the simplest way to do the conversion?&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="96" LastEditDate="2017-06-08T07:04:56.403" LastActivityDate="2017-06-08T07:04:56.403" Title="How to convert FASTA to BED" Tags="|fasta|bed|file-formats|format-conversion|" AnswerCount="7" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="92" PostTypeId="1" AcceptedAnswerId="93" CreationDate="2017-05-18T00:28:24.290" Score="19" ViewCount="536" Body="&lt;p&gt;I am working with a set of (bulk) RNA-Seq data collected across multiple runs, run at different times of the year. I have normalized my data using library size / quantile / RUV normalization, and would like to check (quantitatively and/or qualitatively) whether or not normalization has succeeded in removing the batch effects. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is important to note that by &quot;normalization has succeeded&quot;, I simply mean that unwanted variation has been removed - further analysis is required to ensure that biological variation has not been removed. What are some plots / statistical tests / software packages which provide a first-step QC for normalization?&lt;/p&gt;&#xA;" OwnerUserId="163" LastEditorUserId="163" LastEditDate="2019-09-11T15:53:23.030" LastActivityDate="2019-09-11T15:53:23.030" Title="Confirm success or failure of RNA-Seq normalization" Tags="|rna-seq|normalization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="93" PostTypeId="2" ParentId="92" CreationDate="2017-05-18T00:49:44.233" Score="17" Body="&lt;p&gt;You should use box plots and PCA plot. Let's take a look at the RUV paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4404308/&quot; rel=&quot;noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4404308/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Before normalization and after UQ normalization&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aZBua.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aZBua.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Libraries do not cluster as expected according to treatment. ... for UQ-normalized counts. UQ normalization does not lead to better clustering of the samples...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Before normalization, the medians in the box-plot obviously look very different among replicates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After UQ normalization, the medians look closer but Trt.11 look like an outlier. Furthermore, the treatments aren't clustered on the PCA plot. Since they are replicates, you'd like them be close on the plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;After RUV normalization&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/m4tYK.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m4tYK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;... RUVg shrinks the expression measures for Library 11 towards the median across libraries, suggesting robustness against outliers. ... Libraries cluster as expected by treatment. ...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The RUV has made the distribution more robust and the samples closer on the PCA plot. However, it's still not perfect as one of the treatments is not close to the other two on the first PC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The vignettes for Bioconductor &lt;code&gt;RUVSeq&lt;/code&gt; describes the two functions: &lt;code&gt;plotRLE&lt;/code&gt; and &lt;code&gt;plotPCA&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="174" LastEditDate="2017-05-18T00:55:59.730" LastActivityDate="2017-05-18T00:55:59.730" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="94" PostTypeId="2" ParentId="91" CreationDate="2017-05-18T01:01:14.320" Score="6" Body="&lt;p&gt;You could adapt this &lt;a href=&quot;http://www.danielecook.com/generate-fasta-sequence-lengths/&quot; rel=&quot;noreferrer&quot;&gt;awk one-liner&lt;/a&gt;. Note that it assumes that sequence IDs are not longer than 100 characters and that there is no description following the sequence ID on the header line.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat myseqs.fasta | awk '$0 ~ &quot;&amp;gt;&quot; {print c; c=0;printf substr($0,2,100) &quot;\t0\t&quot;; } $0 !~ &quot;&amp;gt;&quot; {c+=length($0);} END { print c; }'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Otherwise, any Bio* library (Perl, Python, Ruby) provides FASTA format parsers which will extract sequence IDs and lengths.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd point out that whilst this resembles BED it is not, strictly-speaking, since BED maps to coordinates on a chromosome or some longer sequence object.&lt;/p&gt;&#xA;" OwnerUserId="150" LastEditorUserId="150" LastEditDate="2017-05-18T01:07:42.050" LastActivityDate="2017-05-18T01:07:42.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="95" PostTypeId="2" ParentId="91" CreationDate="2017-05-18T01:04:48.900" Score="0" Body="&lt;p&gt;If the FASTA sequences are all on a single line, then the following perl one-liner should work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat myseqs.fasta | perl -ne 'if(/^&amp;gt;([^ ]+)/){print $1} else {print &quot; 0 &quot;,length,&quot;\n&quot;}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Explanation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If the line begins with a '&gt;', then print everything up to the first space (but don't put a line break at the end)&lt;/li&gt;&#xA;&lt;li&gt;Otherwise, print &quot; 0 &quot;, followed by the length of the line, followed by a line break&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-05-18T01:04:48.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="96" PostTypeId="2" ParentId="91" CreationDate="2017-05-18T01:08:32.233" Score="4" Body="&lt;p&gt;Inspired by &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/45/read-length-distribution-from-fasta-file/84#84&quot;&gt;this answer&lt;/a&gt; to a related question on read length distributions, you could do this with Biopython:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.SeqIO import parse&#xA;with open(&quot;regions.bed&quot;, &quot;w&quot;) as bed:&#xA;    for record in parse(&quot;regions.fasta&quot;, &quot;fasta&quot;):&#xA;        print(record.id, 0, len(record.seq), sep=&quot;\t&quot;, file=bed)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="163" LastEditorUserId="132" LastEditDate="2017-05-18T13:55:20.790" LastActivityDate="2017-05-18T13:55:20.790" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="97" PostTypeId="1" AcceptedAnswerId="172" CreationDate="2017-05-18T03:11:37.330" Score="12" ViewCount="352" Body="&lt;p&gt;I have a DNA sample which I know doesn't &lt;em&gt;quite&lt;/em&gt; match my reference genome - my culture comes from a subpopulation which has undergone significant mutation since the reference was created.&lt;/p&gt;&#xA;&lt;p&gt;The example I have in mind is E.coli. We've tried assembly using a couple of different tools and the de-novo assembly isn't as high quality as we would like, despite having tonnes of data. Approaching this from a Bayesian point of view, the reference genome provides a very good prior if we could use it wisely&lt;/p&gt;&#xA;&lt;p&gt;From visual inspection with IGV, a significant number of both SNPs and SVs appear to be present, but an assembly built entirely from my own sequencing data is not high enough quality for my purposes.&lt;/p&gt;&#xA;&lt;p&gt;How can I modify this reference genome to match my sample with new sequencing data (preferably with Oxford Nanopore Technologies long reads, but I can also use these to scaffold short reads if necessary), taking advantage of my knowledge that the existing reference is &lt;em&gt;mostly&lt;/em&gt; very good, without having to access the reads which were originally used to construct the reference genome?&lt;/p&gt;&#xA;&lt;p&gt;The goal of the project isn't to determine where the SVs are, I just need a reference that accurately represents my sample in order to use the data for downstream analysis (as the training set for machine learning.) So by a high quality reference, I mean one which represents as well as possible the sample that was sequenced. To make matters worse, this may not be the one which has the highest alignment identity if there are systematic sequencing errors, as in nanopore sequencing!&lt;/p&gt;&#xA;" OwnerUserId="163" LastEditorUserId="73" LastEditDate="2023-01-09T19:23:06.157" LastActivityDate="2023-01-09T19:23:06.157" Title="Improve a reference genome with sequencing data" Tags="|genome|phylogenetics|assembly|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="98" PostTypeId="1" AcceptedAnswerId="99" CreationDate="2017-05-18T03:14:29.460" Score="8" ViewCount="3454" Body="&lt;p&gt;After DNA sequencing, I generated a sam file through alignment of a fastq file. &#xA;Before using well known variant calling programs (eg. Annovar etc.), I want to pick some reads and know what kinds of mutations are there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any quick way of doing it?&lt;/p&gt;&#xA;" OwnerUserId="274" LastActivityDate="2017-05-21T17:47:47.147" Title="How to quickly determine mutations in a read of a sam file?" Tags="|variant-calling|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="99" PostTypeId="2" ParentId="98" CreationDate="2017-05-18T03:24:19.127" Score="8" Body="&lt;p&gt;For qualitative analysis, you're probably better off using something less granular like &lt;a href=&quot;http://software.broadinstitute.org/software/igv/&quot; rel=&quot;noreferrer&quot;&gt;IGV&lt;/a&gt; or &lt;a href=&quot;http://bioviz.org/igb/overview.html&quot; rel=&quot;noreferrer&quot;&gt;IGB&lt;/a&gt;. However, if you really want to look at a couple of reads:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're willing to ignore sequencing errors, you can inspect the CIGAR string or the MD tag, both of which give information on the alignment of a single read.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The CIGAR string gives details on insertions, deletions, clippings, matches and mismatches. From &lt;a href=&quot;http://genome.sph.umich.edu/wiki/SAM&quot; rel=&quot;noreferrer&quot;&gt;Genome Analysis Wiki&lt;/a&gt;,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The sequence being aligned to a reference may have additional bases that are not in the reference or may be missing bases that are in the reference. The CIGAR string is a sequence of of base lengths and the associated operation. They are used to indicate things like which bases align (either a match/mismatch) with the reference, are deleted from the reference, and are insertions that are not in the reference. For example:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;RefPos:     1  2  3  4  5  6  7     8  9 10 11 12 13 14 15 16 17 18 19&#xA;Reference:  C  C  A  T  A  C  T     G  A  A  C  T  G  A  C  T  A  A  C&#xA;Read:                   A  C  T  A  G  A  A     T  G  G  C  T&#xA;With the alignment above, you get:&#xA;POS: 5&#xA;CIGAR: 3M1I3M1D5M&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Most common usage of the CIGAR string uses M (match/mismatch), I (insertion), D (deletion), S (soft clipping), and H (hard clipping). Note that = (match) and X (mismatch) are available as alternatives to the less informative M, but they are less widely used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The MD tag gives specific details on mismatches and deletions. From the &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMtags.pdf&quot; rel=&quot;noreferrer&quot;&gt;SAMtools tags specification&lt;/a&gt;,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The MD field aims to achieve SNP/indel calling without looking at the reference. For example, a string ‘10A5^AC6’ means from the leftmost reference base in the alignment, there are 10 matches followed by an A on the reference which is different from the aligned read base; the next 5 reference bases are matches followed by a 2bp deletion from the reference; the deleted sequence is AC; the last 6 bases are matches. The MD field ought to match the CIGAR string.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Note that neither of these will give you any idea of structural variants in short reads, and neither will be particularly readable (or helpful, due to the higher error rate) in long reads.&lt;/p&gt;&#xA;" OwnerUserId="163" LastEditorUserId="163" LastEditDate="2017-05-18T03:33:12.387" LastActivityDate="2017-05-18T03:33:12.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="100" PostTypeId="2" ParentId="89" CreationDate="2017-05-18T04:30:57.133" Score="3" Body="&lt;p&gt;Even with inbreeding and other genetic phenomena that might mask actual evolution of these cultivars, any phylogenetic methodology would be capable of determining relationships accurately. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try creating a Neighbour Joining tree with &lt;a href=&quot;http://www.megasoftware.net/download_form&quot; rel=&quot;nofollow noreferrer&quot;&gt;MEGA&lt;/a&gt;, which is one of the simplest methods available. This should give you enough to check the relationships of the cultivars.&lt;/p&gt;&#xA;" OwnerUserId="280" LastActivityDate="2017-05-18T04:30:57.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="101" PostTypeId="2" ParentId="97" CreationDate="2017-05-18T05:29:39.427" Score="3" Body="&lt;p&gt;You can use &lt;a href=&quot;https://github.com/jts/nanopolish&quot; rel=&quot;nofollow noreferrer&quot;&gt;nanopolish&lt;/a&gt; using the illumina reads. Also have a look at &lt;a href=&quot;https://github.com/broadinstitute/pilon/wiki&quot; rel=&quot;nofollow noreferrer&quot;&gt;pilon&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="282" LastEditorUserId="77" LastEditDate="2017-05-18T13:58:52.057" LastActivityDate="2017-05-18T13:58:52.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="102" PostTypeId="2" ParentId="97" CreationDate="2017-05-18T05:38:32.473" Score="3" Body="&lt;p&gt;If it is a short-read draft assembly and you have long-reads (ONT or Pacbio) run &lt;a href=&quot;https://github.com/warrenlr/LINKS&quot; rel=&quot;nofollow noreferrer&quot;&gt;links&lt;/a&gt; to scaffold the genome and then run &lt;a href=&quot;https://github.com/broadinstitute/pilon&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pilon&lt;/a&gt; iteratively to try to polish and fill gaps using the short-reads. &lt;/p&gt;&#xA;" OwnerUserId="223" LastActivityDate="2017-05-18T05:38:32.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="103" PostTypeId="1" AcceptedAnswerId="104" CreationDate="2017-05-18T07:17:15.830" Score="4" ViewCount="4271" Body="&lt;p&gt;&lt;em&gt;Bootstrap analysis&lt;/em&gt;, &lt;em&gt;bootstrapping&lt;/em&gt; etc are quite common jargons of bioinformatics and phylogenetics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is not very much clear to us, what exactly being meant by &lt;em&gt;&quot;a boot's straps&quot;.&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it means a &lt;em&gt;&quot;comparison&quot;&lt;/em&gt;? (Such as we hold the two straps of a boot (shoe) in &quot;two hands&quot;; may be an analogy to comparing 2 things (say left-'hand' side and right-'hand' side of an algebraic proof-task)) or it is something about &lt;em&gt;sequence alignment&lt;/em&gt; (since we 'tie' a knot in boot (shoe) so that the 2 ends of the strap remain attached to each-other). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of my classmates even assumed it may be a process during computer booting (but I could not agree with that). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've searched internet, but could not found anything very helpful. This &lt;a href=&quot;https://en.wikipedia.org/wiki/Bootstrapping#Etymology&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia&lt;/a&gt; page (&lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Bootstrapping&amp;amp;oldid=776992221#Etymology&quot; rel=&quot;nofollow noreferrer&quot;&gt;permalink&lt;/a&gt;) about bootstrapping, tells, &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;sup&gt; Tall boots may have a tab, loop or handle at the top known as a bootstrap, allowing one to use fingers or a boot hook tool to help pulling the boots on. The saying &quot;to pull oneself up by one's bootstraps&quot; was already in use during the 19th century as an example of an impossible task. The idiom dates at least to 1834, when it appeared in the Workingman's Advocate: &quot;It is conjectured that Mr. Murphee will now be enabled to hand himself over the Cumberland river or a barn yard fence by the straps of his boots.&quot; In 1860 it appeared in a comment on philosophy of mind: &lt;strong&gt;&lt;em&gt;&quot;The attempt of the mind to analyze itself an effort analogous to one who would lift himself by his own bootstraps&quot;&lt;/em&gt;&lt;/strong&gt;.  &lt;strong&gt;Bootstrap as a metaphor, meaning to better oneself by one's own unaided efforts&lt;/strong&gt;, was in use in 1922. This metaphor spawned additional metaphors for a series of self-sustaining processes that proceed without external help.&lt;/sup&gt;  &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;sup&gt;The term is sometimes attributed to a story in Rudolf Erich Raspe's The Surprising Adventures of Baron Munchausen, but in that story Baron Munchausen pulls himself (and his horse) out of a swamp by his hair (specifically, his pigtail), not by his bootstraps – and no explicit reference to bootstraps has been found elsewhere in the various versions of the Munchausen tales.&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(In the quotation, highlighted portion seemed to tried to tell a meaning), but it wasn't also very much helpful. Could not correlate with bioinformatics. (Btw the same wikipedia article mentions computer booting). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wikipedia article about &lt;a href=&quot;https://en.wikipedia.org/wiki/Bootstrapping_(statistics)#History&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bootstrapping (statistics)&lt;/a&gt; (&lt;a href=&quot;https://en.wikipedia.org/w/index.php?title=Bootstrapping_(statistics)&amp;amp;oldid=780324952#History&quot; rel=&quot;nofollow noreferrer&quot;&gt;permalink&lt;/a&gt;) tells: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;sup&gt;The bootstrap was published by Bradley Efron in &quot;Bootstrap methods: another look at the jackknife&quot; (1979), inspired by earlier work on the jackknife.Improved estimates of the variance were developed later. A Bayesian extension was developed in 1981. The bias-corrected and accelerated (BCa) bootstrap was developed by Efron in 1987, and the ABC procedure in 1992.&lt;/sup&gt; &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Still that is not very helpful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the textbook, &lt;em&gt;Introduction to Bioinformatics/ Arthur M. Lesk/ 3rd Edition&lt;/em&gt;/ Oxford / (Low Price edition) ; in its &lt;em&gt;chapter 5&lt;/em&gt; (Alignments and phylogenetic trees), section The problem of varying rates of evolution -&gt; &lt;em&gt;computational consideration&lt;/em&gt; (page 296); a clearer definition has been given: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;3&quot;&gt;&#xA;  &lt;li&gt;Formal statistical tests, involving rerunning the calculation on subsets of the original data, are known as &lt;strong&gt;jackknifing&lt;/strong&gt; and &lt;strong&gt;bootstrapping&lt;/strong&gt;.     &#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;&lt;strong&gt;jackknifing&lt;/strong&gt; is calculation with data sets sampled randomly from the original data. For phylogeny calculations from from multiple sequence alignments, select different subsets of the position in the alignment, and rerun the calculation. Finding that each subset gives the same phylogenetic tree lends it credibility. If each subset gives a different tree, none of them are trustworthy. &lt;/li&gt;&#xA;  &lt;li&gt;&lt;strong&gt;Bootstrapping&lt;/strong&gt; is  similar to jackknifing except that the position chosen at random may include multiple copies of the same position, to form data sets of the same size as original, to preserve statistical properties of data sampling. &lt;/li&gt;&#xA;  &lt;/ul&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;More clear definition of &quot;Boot-strapping&quot;; but it does not explain, where the relation exists with the boot (shoe) and its straps. &lt;/p&gt;&#xA;" OwnerDisplayName="user286" LastActivityDate="2022-07-07T15:46:00.673" Title="Bootstrap analysis - why it is called bootstrap?" Tags="|terminology|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="104" PostTypeId="2" ParentId="103" CreationDate="2017-05-18T07:30:52.260" Score="9" Body="&lt;p&gt;The method itself has nothing really to do with boots or straps, just as the jack-knife method also has nothing to do with knives. In the case of bootstrapping, the goal is to determine the accuracy of an estimate from random subsets of a population. Normally estimating something like the variance of the mean requires multiple independent samples, but bootstrapping allows you to perform estimates from a single population. This is essentially allowing the estimate to &quot;&lt;a href=&quot;https://en.wiktionary.org/wiki/pull_oneself_up_by_one%27s_bootstraps&quot; rel=&quot;nofollow noreferrer&quot;&gt;pull itself up by its bootstraps&lt;/a&gt;&quot;, which is the only reasonable source for the term. See also the more general wikipedia article on &lt;a href=&quot;https://en.wikipedia.org/wiki/Bootstrapping&quot; rel=&quot;nofollow noreferrer&quot;&gt;bootstrapping outside of statistics&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="43" LastEditDate="2017-05-18T19:59:53.980" LastActivityDate="2017-05-18T19:59:53.980" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="105" PostTypeId="2" ParentId="103" CreationDate="2017-05-18T07:43:12.813" Score="6" Body="&lt;p&gt;In general, branch support values indicate how much of the total data support that branch in a proposed tree. But what do you do if you used all of the available data to infer the tree in the first place? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With Bootstrap supports, you &lt;em&gt;generate&lt;/em&gt; new alignments (almost) &lt;em&gt;ex nihilo&lt;/em&gt;. You calculate a trees support values, using its own data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the connection to &quot;pulling yourself up by your bootstraps&quot;: you don't &quot;pull&quot; on additionaly collected data (a rope or something in the metaphor), but rather on the very thing that your observation is based on (your boots).&lt;/p&gt;&#xA;" OwnerUserId="204" LastActivityDate="2017-05-18T07:43:12.813" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="106" PostTypeId="2" ParentId="91" CreationDate="2017-05-18T08:04:25.233" Score="4" Body="&lt;p&gt;Here is an approach with &lt;a href=&quot;http://biopython.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;BioPython&lt;/code&gt;&lt;/a&gt;. The &lt;code&gt;with&lt;/code&gt; statement ensures both the input and output file handles are closed and a &lt;a href=&quot;https://stackoverflow.com/questions/20535342/lazy-evaluation-python&quot;&gt;&lt;em&gt;lazy&lt;/em&gt;&lt;/a&gt; approach is taken so that only a single fasta record is held in memory at a time, rather than reading the whole file into memory, which is a bad idea for large input files. The solution makes no assumptions about the sequence ID lengths or the number of lines that the sequences are spread across:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;with open('sequences.fasta') as in_f, open('sequences.bed','w') as out_f:&#xA;    for record in SeqIO.parse(in_f, 'fasta'):&#xA;        out_f.write('{}\t0\t{}\n'.format(record.id, len(record)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastEditorUserId="-1" LastEditDate="2017-05-23T12:41:55.037" LastActivityDate="2017-05-18T08:56:58.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="107" PostTypeId="1" CreationDate="2017-05-18T08:05:10.337" Score="9" ViewCount="148" Body="&lt;p&gt;What are the best tools to design gRNAs in a high-throughput way for a CRISPR screen, e.g. targeting all protein-coding genes in a genome? I would like to take into account possible off-target effects, as well as to allow for flexibility in the PAM sequence, to have the possibility of using non-canonical PAMs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm aware of &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/CRISPRseek.html&quot; rel=&quot;noreferrer&quot;&gt;CRISPRseek&lt;/a&gt;, but I'm afraid it might be quite slow to run for a genome-scale search.&lt;/p&gt;&#xA;" OwnerUserId="182" LastActivityDate="2017-05-27T16:05:14.763" Title="Large-scale gRNA design for a CRISPR screen" Tags="|crispr|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="108" PostTypeId="2" ParentId="91" CreationDate="2017-05-18T08:40:08.150" Score="13" Body="&lt;p&gt;It's good practice to have your &lt;code&gt;FASTA&lt;/code&gt; indexed, so you can leverage the &lt;code&gt;.fai&lt;/code&gt; you are likely to already have. If not, you can just generate the index with &lt;code&gt;samtools&lt;/code&gt; and use some &lt;code&gt;awk&lt;/code&gt; to make your &lt;code&gt;BED&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools faidx $fasta&#xA;awk 'BEGIN {FS=&quot;\t&quot;}; {print $1 FS &quot;0&quot; FS $2}' $fasta.fai &amp;gt; $fasta.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will maintain tab separation but you can drop the &lt;code&gt;BEGIN&lt;/code&gt; statement to use spaces. The &lt;a href=&quot;https://genome.ucsc.edu/FAQ/FAQformat.html#format1&quot; rel=&quot;noreferrer&quot;&gt;BED spec&lt;/a&gt; only requires &quot;whitespace&quot; for the simple &lt;code&gt;BED&lt;/code&gt; format.&lt;/p&gt;&#xA;" OwnerUserId="215" LastEditorUserId="215" LastEditDate="2017-05-18T09:06:45.380" LastActivityDate="2017-05-18T09:06:45.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="109" PostTypeId="2" ParentId="81" CreationDate="2017-05-18T08:57:06.240" Score="4" Body="&lt;p&gt;The &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/polyester.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;polyester&lt;/a&gt; bioconductor package can do this. It says it simulates RNA-seq reads, but I don't know if that's really any different from other NGS reads. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can use a range of error and bias models, or learn them from a dataset.&lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2017-05-18T08:57:06.240" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="110" PostTypeId="1" AcceptedAnswerId="121" CreationDate="2017-05-18T09:11:09.517" Score="7" ViewCount="644" Body="&lt;p&gt;I'm attempting to reconstruct the read fragments that form a hit to a target, excluding/skipping gaps. &lt;em&gt;BLAST&lt;/em&gt;'s &lt;code&gt;outfmt 6&lt;/code&gt;, and alternatives such as Diamond's &lt;code&gt;M8&lt;/code&gt; format only give the number of gap opens on the alignment. One can't just use the &lt;code&gt;start&lt;/code&gt; and &lt;code&gt;end&lt;/code&gt; positions of the alignment as indices to slice the read DNA sequence, as this will include the nucleotides that were gapped by the alignment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;PSL&lt;/code&gt; appeared to be the only format I could find that gave the actual start positions (and lengths) of each of the pieces that formed a hit (that is, each gap forces a new block).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The regular &lt;em&gt;BLAST&lt;/em&gt; output is human readable and is a nightmare to parse. The &lt;code&gt;XML&lt;/code&gt; format mentioned has a structure for each HSP, but an HSP can still include gaps:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  [...]&#xA;  &amp;lt;Hit_hsps&amp;gt;&#xA;    &amp;lt;Hsp&amp;gt;&#xA;      &amp;lt;Hsp_num&amp;gt;1&amp;lt;/Hsp_num&amp;gt;&#xA;      &amp;lt;Hsp_bit-score&amp;gt;51.6&amp;lt;/Hsp_bit-score&amp;gt;&#xA;      &amp;lt;Hsp_score&amp;gt;122&amp;lt;/Hsp_score&amp;gt;&#xA;      &amp;lt;Hsp_evalue&amp;gt;9.8e-06&amp;lt;/Hsp_evalue&amp;gt;&#xA;      &amp;lt;Hsp_query-from&amp;gt;2&amp;lt;/Hsp_query-from&amp;gt;&#xA;      &amp;lt;Hsp_query-to&amp;gt;98&amp;lt;/Hsp_query-to&amp;gt;&#xA;      &amp;lt;Hsp_hit-from&amp;gt;35&amp;lt;/Hsp_hit-from&amp;gt;&#xA;      &amp;lt;Hsp_hit-to&amp;gt;65&amp;lt;/Hsp_hit-to&amp;gt;&#xA;      &amp;lt;Hsp_query-frame&amp;gt;2&amp;lt;/Hsp_query-frame&amp;gt;&#xA;      &amp;lt;Hsp_hit-frame&amp;gt;0&amp;lt;/Hsp_hit-frame&amp;gt;&#xA;      &amp;lt;Hsp_identity&amp;gt;24&amp;lt;/Hsp_identity&amp;gt;&#xA;      &amp;lt;Hsp_positive&amp;gt;27&amp;lt;/Hsp_positive&amp;gt;&#xA;      &amp;lt;Hsp_gaps&amp;gt;1&amp;lt;/Hsp_gaps&amp;gt;&#xA;      &amp;lt;Hsp_align-len&amp;gt;32&amp;lt;/Hsp_align-len&amp;gt;&#xA;         &amp;lt;Hsp_qseq&amp;gt;ITAIGAGLQGPAGCEVIDAGGLLVMPGGIDTH&amp;lt;/Hsp_qseq&amp;gt;&#xA;         &amp;lt;Hsp_hseq&amp;gt;IAAVGTGLE-PAGAEIIDAGGLLVMPGGIDVH&amp;lt;/Hsp_hseq&amp;gt;&#xA;      &amp;lt;Hsp_midline&amp;gt;I A+G GL+ PAG E+IDAGGLLVMPGGID H&amp;lt;/Hsp_midline&amp;gt;&#xA;    &amp;lt;/Hsp&amp;gt;&#xA;  &amp;lt;/Hit_hsps&amp;gt;&#xA;&amp;lt;/Hit&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm trying to be lazy, I don't want to have to parse the &lt;code&gt;Hsp_qseq&lt;/code&gt; and &lt;code&gt;Hsp_hseq&lt;/code&gt; to determine exactly where my gaps are. I just want co-ordinates of where my read hits a protein. The &lt;code&gt;PSL&lt;/code&gt; format (below) tells me this (&lt;code&gt;tStarts&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;match   mis-    rep.    N's     Q gap   Q gap   T gap   T gap   strand  Q               Q       Q       Q       T               T       T       T       block   blockSizes      qStarts  tStarts&#xA;        match   match           count   bases   count   bases           name            size    start   end     name            size    start   end     count&#xA;---------------------------------------------------------------------------------------------------------------------------------------------------------------&#xA;[...]&#xA;21      4       0       0       1       8       1       24      ++      &amp;lt;PROTEIN&amp;gt;   662     321     354     &amp;lt;READ&amp;gt;      101     2       101     2       8,17,   321,337,        2,50,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As an alternative, I've executed the &lt;code&gt;BLAT&lt;/code&gt; aligner as a means to find alignments of translated DNA reads against a small protein database. I selected &lt;code&gt;BLAT&lt;/code&gt; as it provided the easiest means to access information on the actual alignment blocks (with the smallest amount of parsing). That is, the PSL has a comma delimited field that enumerates each of the starting positions of a query to a target.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, &lt;code&gt;BLAT&lt;/code&gt; outputs a significantly greater number of alignments than alternative tools. This is likely due to its identity percentage threshold defaulting to 25% when emulating a &lt;code&gt;blastx&lt;/code&gt; style query.&#xA;I can cope with this (apologies to my disks for the I/O), but for filtering purposes, the output &lt;code&gt;PSL&lt;/code&gt; format appears to leave me nothing but raw match counts to work with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;PSL&lt;/code&gt; gives me what I want, except now I don't have information on its significance. I know that the &lt;a href=&quot;http://genome.ucsc.edu/FAQ/FAQblat.html#blat4&quot; rel=&quot;nofollow noreferrer&quot;&gt;BLAT FAQ&lt;/a&gt; describes a Perl script for adding &quot;web-based identity and score calculations&quot;. I can actually get identity percentages and scores from this script, but I cannot tell what the score actually represents. Is it comparable to bit-score? How can I recover something like an e-value from a &lt;code&gt;PSL&lt;/code&gt; output file?&lt;/p&gt;&#xA;" OwnerUserId="215" LastEditorUserId="73" LastEditDate="2017-12-29T01:04:21.610" LastActivityDate="2017-12-29T01:04:21.610" Title="Understanding the significance of BLAT score calculations for read fragment alignment" Tags="|sequence-alignment|blat|blast|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="111" PostTypeId="2" ParentId="45" CreationDate="2017-05-18T09:23:17.877" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/lh3/bioawk&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioawk&lt;/a&gt; could be reasonably efficient for this kind of task.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bioawk -c fastx '{histo[length($seq)]++} END {for (l in histo) print l,histo[l]}' \&#xA;    | sort -n&#xA;0   33270&#xA;1   1542&#xA;2   1132&#xA;3   3397&#xA;4   8776&#xA;5   11884&#xA;6   12474&#xA;7   14341&#xA;8   13165&#xA;9   15467&#xA;10  21089&#xA;11  30469&#xA;12  45204&#xA;13  62311&#xA;14  88744&#xA;15  115767&#xA;16  140770&#xA;17  191810&#xA;18  313088&#xA;19  518111&#xA;20  1097867&#xA;21  4729715&#xA;22  6575557&#xA;23  2734062&#xA;24  1015476&#xA;25  493323&#xA;26  323827&#xA;27  164419&#xA;28  107120&#xA;29  72487&#xA;30  40120&#xA;31  24538&#xA;32  22295&#xA;33  13121&#xA;34  9382&#xA;35  4847&#xA;36  3858&#xA;37  3161&#xA;38  2852&#xA;39  2388&#xA;40  1639&#xA;41  961&#xA;42  686&#xA;43  377&#xA;44  199&#xA;45  114&#xA;46  78&#xA;47  59&#xA;48  50&#xA;49  52&#xA;50  48&#xA;51  42&#xA;52  39&#xA;53  28&#xA;54  49&#xA;55  59&#xA;56  55&#xA;57  51&#xA;58  55&#xA;59  43&#xA;60  52&#xA;61  56&#xA;62  48&#xA;63  67&#xA;64  95&#xA;65  488&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;-c fastx&lt;/code&gt; tells the program to parse the data as fastq or fasta. This gives access to the different parts of the records as &lt;code&gt;$name&lt;/code&gt;, &lt;code&gt;$seq&lt;/code&gt; (and &lt;code&gt;$qual&lt;/code&gt; in the case of fastq format) in the awk code (bioawk is based on awk, so you can use whatever language features you want from &lt;a href=&quot;http://www.grymoire.com/Unix/Awk.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;awk&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Between the single quotes come a series of &lt;code&gt;&amp;lt;condition&amp;gt; {&amp;lt;action&amp;gt;}&lt;/code&gt; blocks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first one has no &lt;code&gt;&amp;lt;condition&amp;gt;&lt;/code&gt; part, which mean it is executed for each record. Here, it updates the lengths counts in a table which I named &quot;histo&quot;. &lt;code&gt;length&lt;/code&gt; is a predefined function in awk.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the second block, the &lt;code&gt;END&lt;/code&gt; condition means we want it to be executed after all the input has been processed. The action part consists in looping over the recorded length values and print them together with the associated count.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output is piped to &lt;code&gt;sort -n&lt;/code&gt; in order to sort the results numerically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On my workstation, the above code took 20 seconds to execute for a 1.2G fasta file.&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2017-05-18T09:23:17.877" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="112" PostTypeId="1" AcceptedAnswerId="16226" CreationDate="2017-05-18T09:27:51.110" Score="27" ViewCount="1362" Body="&lt;p&gt;I am currently looking for a system which will allow me to version both the code and the data in my research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think my way of analyzing data is not uncommon, and this will be useful for many people doing bioinformatics and aiming for the reproducibility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the requrements:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Analysis is performed on multiple machines (local, cluster, server).&lt;/li&gt;&#xA;&lt;li&gt;All the code is transparently synchronized between the machines.&lt;/li&gt;&#xA;&lt;li&gt;Source code versioning.&lt;/li&gt;&#xA;&lt;li&gt;Generated data versioning.&lt;/li&gt;&#xA;&lt;li&gt;Support for large number of small generated files (&gt;10k). These also could be deleted.&lt;/li&gt;&#xA;&lt;li&gt;Support for large files (&gt;1Gb). At some point old generated files can  permanently deleted. It would be insane to have &lt;em&gt;transparent&lt;/em&gt; synchronization of those, but being able to synchronize them on demand would be nice.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So far I am using &lt;strong&gt;git&lt;/strong&gt; + rsync/scp. But there are several downsides to it.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Synchronization between multiple machines is a bit tedious, i.e. you have to git pull before you start working and git push after each update. I can live with that.&lt;/li&gt;&#xA;&lt;li&gt;You are not supposed to store large generated data files or large number of files inside your repository.&lt;/li&gt;&#xA;&lt;li&gt;Therefore I have to synchronize data files manually using rsync, which is error prone.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There is something called &lt;a href=&quot;https://git-annex.branchable.com/&quot; rel=&quot;noreferrer&quot;&gt;git annex&lt;/a&gt;. It seems really close to what I need. But:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A bit more work than git, but that's ok.&lt;/li&gt;&#xA;&lt;li&gt;Unfortunately it seems it does not work well with the large number of files. Often I have more that 10k small files in my analysis. There are some tricks to &lt;a href=&quot;http://git-annex.branchable.com/tips/Repositories_with_large_number_of_files/&quot; rel=&quot;noreferrer&quot;&gt;improve indexing&lt;/a&gt;, but it doesn't solve the issue. What I need is one symlink representing the full contents of directory.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;One potential solution is to use Dropbox or something similar (like &lt;a href=&quot;https://syncthing.net/&quot; rel=&quot;noreferrer&quot;&gt;syncthing&lt;/a&gt;) in combination with git. But the downside is there will be no connection between the source code version and the data version.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any versioning system for the code and the data meeting the requirements you can recommend?&lt;/p&gt;&#xA;" OwnerUserId="191" LastEditorUserId="191" LastEditDate="2017-05-18T13:50:39.270" LastActivityDate="2021-06-25T13:18:25.937" Title="How to version the code and the data during the analysis?" Tags="|reproducibility|data-management|git|" AnswerCount="8" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="113" PostTypeId="2" ParentId="112" CreationDate="2017-05-18T09:51:51.780" Score="10" Body="&lt;p&gt;Your question is somewhat open, but I think it could prove an interesting discussion. I don't believe in many cases it is worth storing the data you have created in &lt;code&gt;git&lt;/code&gt;. As you've noted, it isn't designed for large files (although we have &lt;code&gt;git-lfs&lt;/code&gt;) and it's definitely not designed for binary formats such as &lt;code&gt;BAM&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm of the opinion that &lt;code&gt;how&lt;/code&gt; a file was created and &lt;code&gt;what&lt;/code&gt; has been done to it since is key. Large files that took much effort to create should be mirrored somewhere (but not necessarily in a version control system). Other less-important (or less difficult to create) files that have been lost, clobbered or otherwise tainted can be regenerated as long as you know how they came to be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For what it's worth, I've been working on &lt;a href=&quot;https://github.com/SamStudio8/chitin/&quot; rel=&quot;noreferrer&quot;&gt;a piece of software called &lt;code&gt;chitin&lt;/code&gt;&lt;/a&gt; (self described as a shell for disorganised bioinformaticians). I also wrote &lt;a href=&quot;https://samnicholls.net/2016/11/16/disorganised-disaster/&quot; rel=&quot;noreferrer&quot;&gt;a long blog post&lt;/a&gt; on why I thought this was a necessary project for me, but the main reason was that despite my attempts to organise my filesystem and make good archives of my experiments, over time I forget what my shorthand directory names meant, or exactly what program generated which data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;chitin&lt;/code&gt;'s goal is to automatically capture changes made to the file system during the execution of a command. It knows what commands to run to re-create a particular file, what commands have used that file and can tell you when and why that file was changed (and by who ;) too).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not finished (nothing ever is), but I feel that you might be going down the wrong road by wanting to store all your data and its versions when really, I think most people just want to know the commands that instigated changes. If the data history is important (and your code is well versioned), then you can simply check out any commit and execute your analysis to re-generate data.&lt;/p&gt;&#xA;" OwnerUserId="215" LastActivityDate="2017-05-18T09:51:51.780" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="114" PostTypeId="2" ParentId="110" CreationDate="2017-05-18T10:05:47.703" Score="1" Body="&lt;p&gt;I would suggest using blastx and parsing the output XML. As an example, have a look at this: &lt;a href=&quot;http://www.perlmonks.org/?node_id=1006367&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.perlmonks.org/?node_id=1006367&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The relevant part of the hit would be the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;Hit&amp;gt;&#xA;  &amp;lt;Hit_num&amp;gt;1&amp;lt;/Hit_num&amp;gt;&#xA;  &amp;lt;Hit_id&amp;gt;gnl|BL_ORD_ID|1515029&amp;lt;/Hit_id&amp;gt;&#xA;  &amp;lt;Hit_def&amp;gt;43989.cce_0262 (Cyanothece ATCC 51142)&amp;lt;/Hit_def&amp;gt;&#xA;  &amp;lt;Hit_accession&amp;gt;1515029&amp;lt;/Hit_accession&amp;gt;&#xA;  &amp;lt;Hit_len&amp;gt;65&amp;lt;/Hit_len&amp;gt;&#xA;  &amp;lt;Hit_hsps&amp;gt;&#xA;    &amp;lt;Hsp&amp;gt;&#xA;      &amp;lt;Hsp_num&amp;gt;1&amp;lt;/Hsp_num&amp;gt;&#xA;      &amp;lt;Hsp_bit-score&amp;gt;40.0466&amp;lt;/Hsp_bit-score&amp;gt;&#xA;      &amp;lt;Hsp_score&amp;gt;92&amp;lt;/Hsp_score&amp;gt;&#xA;      &amp;lt;Hsp_evalue&amp;gt;0.00664016&amp;lt;/Hsp_evalue&amp;gt;&#xA;      &amp;lt;Hsp_query-from&amp;gt;155&amp;lt;/Hsp_query-from&amp;gt;&#xA;      &amp;lt;Hsp_query-to&amp;gt;253&amp;lt;/Hsp_query-to&amp;gt;&#xA;      &amp;lt;Hsp_hit-from&amp;gt;12&amp;lt;/Hsp_hit-from&amp;gt;&#xA;      &amp;lt;Hsp_hit-to&amp;gt;44&amp;lt;/Hsp_hit-to&amp;gt;&#xA;      &amp;lt;Hsp_query-frame&amp;gt;-1&amp;lt;/Hsp_query-frame&amp;gt;&#xA;      &amp;lt;Hsp_hit-frame&amp;gt;0&amp;lt;/Hsp_hit-frame&amp;gt;&#xA;      &amp;lt;Hsp_identity&amp;gt;17&amp;lt;/Hsp_identity&amp;gt;&#xA;      &amp;lt;Hsp_positive&amp;gt;27&amp;lt;/Hsp_positive&amp;gt;&#xA;      &amp;lt;Hsp_gaps&amp;gt;0&amp;lt;/Hsp_gaps&amp;gt;&#xA;      &amp;lt;Hsp_align-len&amp;gt;33&amp;lt;/Hsp_align-len&amp;gt;&#xA;      &amp;lt;Hsp_qseq&amp;gt;LRGAICSMEHIEEALGKLKDWARKLIELLLGPR&amp;lt;/Hsp_qseq&amp;gt;&#xA;      &amp;lt;Hsp_hseq&amp;gt;ITGAVCLMDYLEKVLEKLRELAQKLIETLLGPQ&amp;lt;/Hsp_hseq&amp;gt;&#xA;      &amp;lt;Hsp_midline&amp;gt;+ GA+C M+++E+ L KL++ A+KLIE LLGP+&amp;lt;/Hsp_midline&amp;gt;&#xA;    &amp;lt;/Hsp&amp;gt;&#xA;  &amp;lt;/Hit_hsps&amp;gt;&#xA;&amp;lt;/Hit&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You should be able to get the position information from the following tags: &lt;code&gt;&amp;lt;Hsp_query-from&amp;gt;&lt;/code&gt;,&lt;code&gt;&amp;lt;Hsp_query-to&amp;gt;&lt;/code&gt;,&lt;code&gt;&amp;lt;Hsp_hit-from&amp;gt;&lt;/code&gt;,&lt;code&gt;&amp;lt;Hsp_hit-to&amp;gt;&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To translate back to you original DNA sequence I can imagine you need the &lt;code&gt;&amp;lt;Hsp_query-frame&amp;gt;&lt;/code&gt; tag.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then of course you get the evalue for free: &lt;code&gt;&amp;lt;Hsp_evalue&amp;gt;&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="196" LastActivityDate="2017-05-18T10:05:47.703" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="115" PostTypeId="2" ParentId="97" CreationDate="2017-05-18T11:07:13.690" Score="4" Body="&lt;p&gt;One approach to this is to use whatever data you have to iteratively update the reference genome. You can keep chain files along the way so you can convert coordinates (e.g. in gff files) from the original reference to your new pseudoreference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple approach might be:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Align new data to existing reference&lt;/li&gt;&#xA;&lt;li&gt;Call variants (e.g. samtools mpileup, GATK, or whatever is best for you)&lt;/li&gt;&#xA;&lt;li&gt;Create new reference incorporating variants from 2&lt;/li&gt;&#xA;&lt;li&gt;Rinse and repeat (i.e. go to 1)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You can track some simple stats as you do this - e.g. the number of new variants should decrease, the number of reads mapped should increase, and the mismatch rate should decrease, with every iteration of the above loop. Once the pseudoreference stabilises, you know you can't do much more. &lt;/p&gt;&#xA;" OwnerUserId="156" LastActivityDate="2017-05-18T11:07:13.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="116" PostTypeId="2" ParentId="91" CreationDate="2017-05-18T11:33:23.620" Score="15" Body="&lt;p&gt;You can do this easily with &lt;a href=&quot;https://github.com/lh3/bioawk&quot; rel=&quot;noreferrer&quot;&gt;bioawk&lt;/a&gt;, which is a version of awk with added features facilitating bioinformatics:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bioawk -c fastx '{print $name&quot;\t0\t&quot;length($seq)}' test.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;-c fastx&lt;/code&gt; tells the program that the data should be parsed as fasta or fastq format. This makes the &lt;code&gt;$name&lt;/code&gt; and &lt;code&gt;$seq&lt;/code&gt; variables available in the awk commands.&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2017-05-18T11:33:23.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="117" PostTypeId="2" ParentId="112" CreationDate="2017-05-18T12:46:12.660" Score="5" Body="&lt;p&gt;The Open Science Framework uses versioning for all files and is free to use: &lt;a href=&quot;https://osf.io&quot; rel=&quot;noreferrer&quot;&gt;https://osf.io&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can integrate data or code from various sources such as github, dropbox, google drive, figshare or amazon cloud&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also store files on their server using OSF data storage, but I do not know exactly what the file size limit is.&lt;/p&gt;&#xA;" OwnerUserId="179" LastActivityDate="2017-05-18T12:46:12.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="118" PostTypeId="1" AcceptedAnswerId="129" CreationDate="2017-05-18T12:52:23.073" Score="8" ViewCount="649" Body="&lt;p&gt;I am interested in identifying mappings between different types of Affymetrix arrays. I am aware that mappings between gene and probeset can be extracted using Ensembl's Biomart database. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Ensembe gene id ENSG00000181019 maps to &#xA;1. AFFY HG-U133_Plus_2's 210519_s_at&#xA;2. AFFY HuGene-1_0-st-v1's 8002303&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there any way to extract mappings of probeset ids between two arrays(eg: HG-U133_Plus_2, HuGene-1_0-st-v1)?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;eg: 210519_s_at and 8002303&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="23" LastEditorUserId="73" LastEditDate="2017-06-07T13:11:07.443" LastActivityDate="2017-06-07T13:11:07.443" Title="probeset to probeset mappings between Affymetrix arrays" Tags="|ensembl|biomart|microarray|" AnswerCount="3" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="119" PostTypeId="2" ParentId="112" CreationDate="2017-05-18T13:44:31.117" Score="3" Body="&lt;p&gt;Using Git for version-controlling code is a good practice, but it does not lend itself well to versioning large data files.  Manually syncing data across multiple nodes is asking for trouble, you want this syncing to either be handled automatically in a managed environment, or just keep the files on a single network-attached storage device.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One tool you might want to look into is &lt;a href=&quot;https://arvados.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Arvados&lt;/a&gt;, which is designed for syncing bioinformatics data and workflows across multiple machines.  From the project website:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Arvados is a platform for storing, organizing, processing, and sharing genomic and other big data. The platform is designed to make it easier for data scientists to develop analyses, developers to create genomic web applications and IT administers to manage large-scale compute and storage genomic resources. The platform is designed to run in the cloud or on your own hardware.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="47" LastActivityDate="2017-05-18T13:44:31.117" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="120" PostTypeId="1" AcceptedAnswerId="144" CreationDate="2017-05-18T14:28:47.653" Score="15" ViewCount="438" Body="&lt;p&gt;I have a reference genome and now I would like to call structural variants from Illumina pair-end whole genome resequencing data (insert size 700bp). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many tools for SV calls (I made an incomplete list of tools bellow). There is also a tool for merging SV calls from multiple methods / samples - &lt;a href=&quot;https://github.com/fritzsedlazeck/SURVIVOR&quot; rel=&quot;noreferrer&quot;&gt;SURVIVOR&lt;/a&gt;. Is there a combination of methods for SV detection with optimal balance between sensitivity and specificity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a &lt;a href=&quot;http://www.ijpmbs.com/uploadfile/2016/1017/20161017025004545.pdf&quot; rel=&quot;noreferrer&quot;&gt;benchmarking paper&lt;/a&gt;, evaluating sensitivity and specificity of SV calls of individual methods using simulated pair-end reads. However, there is no elaboration on the combination of methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;List of tools for calling structural variants:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/arq5x/lumpy-sv&quot; rel=&quot;noreferrer&quot;&gt;Lumpy&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://breakdancer.sourceforge.net/&quot; rel=&quot;noreferrer&quot;&gt;BreakDancer&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/Illumina/manta&quot; rel=&quot;noreferrer&quot;&gt;Manta&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/dellytools/delly&quot; rel=&quot;noreferrer&quot;&gt;Delly&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/PapenfussLab/gridss&quot; rel=&quot;noreferrer&quot;&gt;GRIDSS&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://compbio.med.harvard.edu/Meerkat/&quot; rel=&quot;noreferrer&quot;&gt;Meerkat&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/genome/pindel&quot; rel=&quot;noreferrer&quot;&gt;Pindel&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://sourceforge.net/projects/softsv/&quot; rel=&quot;noreferrer&quot;&gt;Softsv&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://compbio.cs.toronto.edu/prism/&quot; rel=&quot;noreferrer&quot;&gt;Prism&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2017-06-04T14:08:11.640" LastActivityDate="2017-06-04T14:08:11.640" Title="How can I call structural variants (SVs) from pair-end short read resequencing data?" Tags="|ngs|structural-variation|variant-calling|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="121" PostTypeId="2" ParentId="110" CreationDate="2017-05-18T15:44:11.847" Score="2" Body="&lt;p&gt;It would appear that &lt;code&gt;BLAT&lt;/code&gt; and &lt;a href=&quot;http://www.ebi.ac.uk/about/vertebrate-genomics/software/exonerate&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;exonerate&lt;/code&gt;&lt;/a&gt; (as suggested by &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/298/terdon&quot;&gt;@terdon&lt;/a&gt;) are the only pieces of software that provide enough information for &quot;HSPFragments&quot; to be parsed from their outputs, at least according to &lt;a href=&quot;http://biopython.org/DIST/docs/api/Bio.SearchIO._model-module.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioPython's documentation&lt;/a&gt;, anyway:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Most search programs only have HSPs with one HSPFragment in them,&#xA;  making these two objects inseparable. However, there are programs&#xA;  (e.g. BLAT and Exonerate) which may have more than one HSPFragment&#xA;  objects in any given HSP. If you are not using these programs, you can&#xA;  safely consider HSP and HSPFragment as a single union.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It would seem the options are sticking with &lt;code&gt;BLAT&lt;/code&gt; and generating scores afterwards with the &lt;a href=&quot;http://genome.ucsc.edu/FAQ/FAQblat.html#blat4&quot; rel=&quot;nofollow noreferrer&quot;&gt;script described in their FAQ&lt;/a&gt;, or using &lt;code&gt;exonerate&lt;/code&gt; which provides a raw score as part of its default output. What these scores actually mean I will have to investigate, but for now I think the case is closed!&lt;/p&gt;&#xA;" OwnerUserId="215" LastEditorUserId="215" LastEditDate="2017-05-18T15:50:52.953" LastActivityDate="2017-05-18T15:50:52.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="122" PostTypeId="2" ParentId="98" CreationDate="2017-05-18T16:23:09.657" Score="4" Body="&lt;p&gt;&lt;code&gt;samtools mpileup&lt;/code&gt; can do this quickly:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools mpileup -f reference.fasta -uv input.sam &amp;gt; variants.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will produce a &lt;a href=&quot;http://vcftools.sourceforge.net/VCF-poster.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VCF-formatted&lt;/a&gt; file containing information about what variants have been seen in the SAM file, aggregated for all the mapped reads.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-05-18T16:23:09.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="123" PostTypeId="2" ParentId="112" CreationDate="2017-05-18T16:57:57.520" Score="15" Body="&lt;p&gt;There is a couple of points to consider here, which I outline below. The goal here should be to find a workflow that is minimally intrusive on top of already using &lt;code&gt;git&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As of yet, there is no ideal workflow that covers all use cases, but what I outline below is the closest I could come to it.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Reproducibility is not just keeping all your data&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;You have got your raw data that you start your project with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All other data in your project directory should never just &quot;be there&quot;, but have some record of where it comes from. Data processing scripts are great for this, because they &lt;em&gt;already document how&lt;/em&gt; you went from your raw to your analytical data, and then the files needed for your analyses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And those scripts can be versioned, with an appropriate single entry point of processing (e.g. a &lt;code&gt;Makefile&lt;/code&gt; that describes how to run your scripts).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This way, the state of all your project files is defined by the raw data, and the version of your processing scripts (and versions of external software, but that's a whole different kind of problem).&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;What data/code should and should not be versioned&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Just as you would not version generated code files, you should not want to version 10k intermediary data files that you produced when performing your analyses. The data that &lt;em&gt;should be&lt;/em&gt; versioned is your &lt;em&gt;raw data&lt;/em&gt; (at the start of your pipeline), not automatically generated files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might want to take snapshots of your project directory, but not keep every version of every file ever produced. This already cuts down your problem by a fair margin.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Approach 1: Actual versioning of data&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;For your &lt;strong&gt;raw or analytical data&lt;/strong&gt;, &lt;a href=&quot;https://git-lfs.github.com/&quot; rel=&quot;noreferrer&quot;&gt;Git LFS&lt;/a&gt; (and alternatively &lt;a href=&quot;https://git-annex.branchable.com/&quot; rel=&quot;noreferrer&quot;&gt;Git Annex&lt;/a&gt;, that you already mention) is designed to solve exactly this problem: add tracking information of files in your Git tree, but do not store the content of those files in the repository (because otherwise it would add the size of a non-diffable file with every change you make).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your &lt;strong&gt;intermediate files&lt;/strong&gt;, you do the same as you would do with intermediate code files: add them to your &lt;code&gt;.gitignore&lt;/code&gt; and do not version them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This begs a couple of considerations:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Git LFS is a paid service from Github (the free tier is limited to 1 GB of storage/bandwidth per month, which is very little), and it is more expensive than other comparable cloud storage solutions. You could consider paying for the storage at Github or running your own LFS server (there is a reference implementation, but I assume this would still be a substantial effort)&lt;/li&gt;&#xA;&lt;li&gt;Git Annex is free, but it replaces files by links and hence changes time stamps, which is a problem for e.g. GNU Make based workflows (major drawback for me). Also, fetching of files needs to be done manually or via a commit hook&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Approach 2: Versioning code only, syncing data&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If your analytical data stays the same for most of your analyses, so the actual need to version it (as opposed to back up and document data provenance, which is essential) may be limited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key to get this this working is to put all &lt;strong&gt;data files&lt;/strong&gt; in your &lt;code&gt;.gitignore&lt;/code&gt; and ignore all your &lt;strong&gt;code files&lt;/strong&gt; in &lt;code&gt;rsync&lt;/code&gt;, with a script in your project root (extensions and directories are an example only):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;cd $(dirname $0)&#xA;rsync -auvr \&#xA;    --exclude &quot;*.r&quot; \&#xA;    --include &quot;*.RData&quot; \&#xA;    --exclude &quot;dir with huge files that you don't need locally&quot; \&#xA;    yourhost:/your/project/path/* .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The advantage here is that you don't need to remember the &lt;code&gt;rsync&lt;/code&gt; command you are running. The script itself goes into version control.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is especially useful if you do your heavy processing on a computing cluster but want to make plots from your result files on your local machine. I argue that you &lt;em&gt;generally don't need&lt;/em&gt; bidirectional sync.&lt;/p&gt;&#xA;" OwnerUserId="69" LastActivityDate="2017-05-18T16:57:57.520" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="124" PostTypeId="2" ParentId="81" CreationDate="2017-05-18T17:32:11.053" Score="4" Body="&lt;p&gt;This python script takes a fasta file and tsv file with counts and prints the sequences in the fasta files that many times as it is specified in the tsv file (assuming the format in the question). So if &lt;code&gt;bar.tsv&lt;/code&gt; and &lt;code&gt;foo.fasta&lt;/code&gt; will be your files:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;repeat = {}&#xA;for line in open(&quot;bar.tsv&quot;):&#xA;    seq_id, coverage = line.split()&#xA;    repeat[seq_id] = int(coverage)&#xA;&#xA;for seq_record in SeqIO.parse(foo.fasta, &quot;fasta&quot;):&#xA;    for i in range(repeat.get(seq_record.name, 0)):&#xA;        print(&quot;&amp;gt;&quot;,seq_record.name,&quot;_&quot;,i,sep='')&#xA;        print(seq_record.seq)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2017-05-30T15:24:35.630" LastActivityDate="2017-05-30T15:24:35.630" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="125" PostTypeId="1" AcceptedAnswerId="127" CreationDate="2017-05-18T18:52:27.690" Score="13" ViewCount="177" Body="&lt;p&gt;Let's say I want to construct a phylogenetic tree based on orthologous nucleotide sequences; I do not want to use protein sequences to have a better resolution. These species have different GC-content.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we use a straightforward approach like maximum likelihood with JC69 or any other classical nucleotide model, conserved protein coding sequences of distant species with similar GC-content will artificially cluster together. This will happen because GC-content will mainly affect wobbling codon positions, and they will look similar on the nucleotide level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are possible ways to overcome this? I considered the following options so far:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Using protein sequence. This is possible of course, but we lose a lot of information on the short distance. Not applicable to non-coding sequences. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1186/1471-2105-15-S2-S8&quot; rel=&quot;noreferrer&quot;&gt;Recoding&lt;/a&gt;. In this approach C and T can be combined into a single pyrimidine state Y (G and A could be also combined in some implementations). This sounds interesting, but, first, we also lose some information here. Mathematical properties of the resulting process are not clear. As a result, this approach is not widely used.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Excluding third codon position from the analysis. Losing some short-distance information again. Also, not all synonymous substitution are specific to the third codon positions, so we still expect to have some bias. Not applicable to non-coding sequence.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It should be possible in theory to have a model which allows shifts in GC-content. This will be a non time-reversible Markov process. As far as I understand there are some computational difficulties estimating likelihood for such models.&lt;/p&gt;&#xA;" OwnerUserId="191" LastEditorUserId="3848" LastEditDate="2019-08-20T20:20:12.453" LastActivityDate="2019-08-20T20:20:12.453" Title="What is the best way to account for GC-content shift while constructing nucleotide-based phylogenetic tree?" Tags="|phylogenetics|phylogeny|nucleotide-models|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="127" PostTypeId="2" ParentId="125" CreationDate="2017-05-18T22:38:52.100" Score="5" Body="&lt;p&gt;There are models that take into account compositional heterogeneity both under the &lt;a href=&quot;https://doi.org/10.1080/10635150600975218&quot; rel=&quot;nofollow noreferrer&quot;&gt;maximum likelihood&lt;/a&gt; and &lt;a href=&quot;https://doi.org/10.1093/molbev/msl091&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayesian&lt;/a&gt; frameworks. Although the substitution process is not time-reversible, the computations are &lt;a href=&quot;https://doi.org/10.1080/10635150490445779&quot; rel=&quot;nofollow noreferrer&quot;&gt;simplified by assuming&lt;/a&gt; that the instantaneous rate matrix can be decomposed into an &quot;equilibrium frequency vector&quot; (non-homogeneous) and a symmetric, constant exchange rate matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess all your suggestions are also valid, and I remember recoding being used successfully to reduce the GC-content bias (examples in the references above and &lt;a href=&quot;https://doi.org/10.1093/molbev/msh137&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).  &lt;/p&gt;&#xA;" OwnerUserId="45" LastEditorUserId="191" LastEditDate="2017-05-19T09:24:00.620" LastActivityDate="2017-05-19T09:24:00.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="129" PostTypeId="2" ParentId="118" CreationDate="2017-05-19T01:18:14.417" Score="9" Body="&lt;p&gt;If your question is: can probeset IDs from different platforms be mapped to one another in a similar way as mapping probesets to genes, then the answer is: Yes. BioMart allows you to map almost anything that has an ID to anything else that has an ID.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use BioMart either via the web interface or programatically. A brief guide to using the web interface, for mapping HG U133 Plus 2 to HuGene 1.0:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Go to &lt;a href=&quot;http://www.ensembl.org/biomart/martview&quot; rel=&quot;noreferrer&quot;&gt;the start page&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;Select H. sapiens Ensembl genes for your database; Human genes (GRCh38.p10) for your dataset&lt;/li&gt;&#xA;&lt;li&gt;Click Filters in the left-hand column&lt;/li&gt;&#xA;&lt;li&gt;Expand &quot;REGION&quot;, scroll down to GENE and select &quot;Input microarray probes/probesets ID list [Max 500 advised]&quot; &lt;/li&gt;&#xA;&lt;li&gt;Select AFFY HG U133 PLUS 2 probe ID(s) and either copy/paste or upload a list, one per line&lt;/li&gt;&#xA;&lt;li&gt;Click Attributes in the left-hand column&lt;/li&gt;&#xA;&lt;li&gt;Scroll through, uncheck what you don't want to see and choose what you do, for example Gene Stable ID, AFFY HuGene 1 0 st v1 probe and AFFY HG U133 Plus 2 probe&lt;/li&gt;&#xA;&lt;li&gt;Finally click &quot;Results&quot; in the menu at the top of the left-hand column&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;And you should see &lt;a href=&quot;http://www.ensembl.org/biomart/martview/5e02aa3df275d697dcd2c806c6e5c1a0&quot; rel=&quot;noreferrer&quot;&gt;a result like this&lt;/a&gt; (you'll need to click the &quot;Results&quot; button).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should not expect that there be a one-to-one mapping, for two reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Genes have multiple transcripts and probesets are mapped to each transcript&lt;/li&gt;&#xA;&lt;li&gt;Some transcripts have more than one probeset: in this case, the HuGene IDs 8002301 and 8002303 map to transcripts for this gene&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Finally: here's a programmatic example using &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/biomaRt.html&quot; rel=&quot;noreferrer&quot;&gt;R/BioMart&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(biomaRt)&#xA;mart.hs &amp;lt;- useMart(&quot;ensembl&quot;, &quot;hsapiens_gene_ensembl&quot;)&#xA;results &amp;lt;- getBM(attributes = c(&quot;ensembl_gene_id&quot;, &quot;affy_hugene_1_0_st_v1&quot;, &quot;affy_hg_u133_plus_2&quot;),&#xA;                 filters = &quot;affy_hg_u133_plus_2&quot;,&#xA;                 values = c(&quot;210519_s_at&quot;),&#xA;                 mart = mart.hs)&#xA;&#xA;results&#xA;  ensembl_gene_id affy_hugene_1_0_st_v1 affy_hg_u133_plus_2&#xA;1 ENSG00000181019               8002301         210519_s_at&#xA;2 ENSG00000181019               8002303         210519_s_at&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="150" LastEditorUserId="150" LastEditDate="2017-05-19T01:39:33.997" LastActivityDate="2017-05-19T01:39:33.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="130" PostTypeId="2" ParentId="118" CreationDate="2017-05-19T01:53:56.150" Score="7" Body="&lt;p&gt;Instead of biomaRt, it is also possible to use the mapping databases built into Bioconductor itself, and map from probe to gene, and then from gene to probe in the second. Some R code to convert between hgu133 and hgu95 using the same probe ID provided in another:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(hgu133plus2.db)&#xA;library(hgu95av2.db)&#xA;&#xA;query_probe &amp;lt;- &quot;210519_s_at&quot;&#xA;&#xA;hgu133_ensembl &amp;lt;- select(hgu133plus2.db, keys = query_probe, columns = &quot;ENSEMBL&quot;)&#xA;&#xA;ensembl_hgu95 &amp;lt;- select(hgu95av2.db, keys = hgu133_ensembl$ENSEMBL, keytype = &quot;ENSEMBL&quot;, columns = &quot;PROBEID&quot;)&#xA;&#xA;dplyr::inner_join(hgu133_ensembl, ensembl_hgu95, by = &quot;ENSEMBL&quot;, suffix = c(&quot;.133&quot;, &quot;.95&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="51" LastActivityDate="2017-05-19T01:53:56.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="131" PostTypeId="2" ParentId="118" CreationDate="2017-05-19T01:57:32.150" Score="2" Body="&lt;p&gt;Other answers explain why there might not be one to one mapping between the probes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://bioinformatics.louisville.edu/abid/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AbsID database&lt;/a&gt; does conversion based on mapping the probe sequences to a genome build, and then determines mappings based on overlapping genome alignment coordinates. This is really useful if you want to be sure that two probes are &lt;em&gt;actually likely&lt;/em&gt; measuring the same transcript.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is dependent on both probes aligning to the genome, however.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I worked in the group that provides the AbsID mapping tool.&lt;/p&gt;&#xA;" OwnerUserId="51" LastEditorUserId="51" LastEditDate="2017-05-19T02:16:24.147" LastActivityDate="2017-05-19T02:16:24.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="133" PostTypeId="2" ParentId="91" CreationDate="2017-05-19T05:14:21.100" Score="7" Body="&lt;p&gt;We have many excellent answers! This will be an excellent reference for future users.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found what exactly what I was asking in my question:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/191052/&quot; rel=&quot;noreferrer&quot;&gt;https://www.biostars.org/p/191052/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ pip install pyfaidx  &#xA;$ faidx --transform bed test.fasta &amp;gt; test.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the one-line command I was asking. The other answers also work, but I want to accept my own answer.&lt;/p&gt;&#xA;" OwnerUserId="174" LastActivityDate="2017-05-19T05:14:21.100" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="134" PostTypeId="1" AcceptedAnswerId="155" CreationDate="2017-05-19T05:49:46.937" Score="6" ViewCount="496" Body="&lt;p&gt;I have a BAM file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@SQ SN:chr1 LN:248956422&#xA;@SQ SN:chrx LN:248956423&#xA;ST-E00110:348:HGVKKALXX:1:1201:5822:48670   323 chr1    9999    0   67H66M16H   chrx    1000    0   GATAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAAC  JJJJJJJJJJJJJJJJAJJJJJJJJJJJJFJJJJJJFJFJJJJJJFJJJJJJJJJJJA77FJFJJJ  NM:i:0  MD:Z:66 AS:i:66 XS:i:65 SA:Z:chr5,18606834,-,73S76M,34,0;   RG:Z:g1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There is a read aligned to &lt;code&gt;chr1&lt;/code&gt;, and it's mate aligned to &lt;code&gt;chrx&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a BED file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    0   100000  TestOnly&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to filter out everything that falls outside my BED region, that includes cross-alignments. In my example, although my read aligned to &lt;code&gt;chr1&lt;/code&gt; but it's mate is not. I don't want this read.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I do:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;samtools view -L test.bed test.bam&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;the command gives me the read because it doesn't check cross-alignments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My solution:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;samtools view -L test.bed test.bam | grep -v chrx&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;but this is very slow and clumsy. In my production pipeline I would have to do something like:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;samtools view -L test.bed test.bam | grep -v chrx | grep -v ... | grep -v ... | grep -v ... | grep -v ...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q:&lt;/strong&gt; Is there a better solution?&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="174" LastEditDate="2017-05-19T09:14:23.940" LastActivityDate="2020-01-11T08:35:34.760" Title="How to filter out cross alignments from a BED file?" Tags="|sam|bed|genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="135" PostTypeId="1" AcceptedAnswerId="137" CreationDate="2017-05-19T05:50:24.497" Score="16" ViewCount="10009" Body="&lt;p&gt;I am working with over a million (long) reads, and aligning them to a large genome. I am considering running my alignment jobs in parallel, distributing horizontally across hundreds of nodes rather than trying to run a single job with dozens of cores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to merge the sorted BAM files together for further downstream analysis. What is the most efficient way to do so while maintaining a valid file header and taking advantage of the fact that the input bam files are already sorted?&lt;/p&gt;&#xA;" OwnerUserId="163" LastEditorUserId="2085" LastEditDate="2018-11-16T03:25:49.653" LastActivityDate="2023-04-11T02:26:01.513" Title="Merge hundreds of small BAM files into a single BAM file" Tags="|sequence-alignment|sam|samtools|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="137" PostTypeId="2" ParentId="135" CreationDate="2017-05-19T07:06:35.520" Score="15" Body="&lt;p&gt;&lt;code&gt;samtools merge merged.bam *.bam&lt;/code&gt; is efficient enough since the input files are sorted. You can get a bit faster with sambamba and/or biobambam, but they're not typically already installed and IO quickly becomes a bottleneck anyway.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-05-19T07:06:35.520" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="138" PostTypeId="5" CreationDate="2017-05-19T07:08:03.053" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-19T07:08:03.053" LastActivityDate="2017-05-19T07:08:03.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="139" PostTypeId="4" CreationDate="2017-05-19T07:08:03.053" Score="0" Body="The set of all transcripts of a biological entity, e.g. the transcriptome of a single cell, of a tissue, of an organ, of an individual, of a species. Can also be used to refer to a subset of the transcriptome, e.g. the polyadenylated transcriptome, given that often experiments ignore the 99% of transcripts that are ribosomal RNAs." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T07:59:40.603" LastActivityDate="2018-03-04T07:59:40.603" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="140" PostTypeId="2" ParentId="135" CreationDate="2017-05-19T07:18:49.330" Score="8" Body="&lt;p&gt;Merging sorted files is a linear operation, so any well-implemented tools that do it will do it with approximately the same efficiency.  So &lt;code&gt;samtools merge&lt;/code&gt; (use the most up-to-date version, as there have been improvements in merge header handling in the 1.3.x and 1.4.x versions), &lt;code&gt;picard MergeSamFiles&lt;/code&gt;, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These tools need to hold all the input BAM files open simultaneously, so depending on how many hundred of input files you have you may run into the system limit on open file descriptors.  Use &lt;code&gt;ulimit&lt;/code&gt; to maximise this first; if there are still too many, you may need to merge the first 500 files, then merge the next 500 into that, etc.  Samtools does not do this internally; I'm not sure whether any of the other merge implementations do.&lt;/p&gt;&#xA;" OwnerUserId="134" LastActivityDate="2017-05-19T07:18:49.330" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="141" PostTypeId="2" ParentId="125" CreationDate="2017-05-19T09:18:45.637" Score="3" Body="&lt;p&gt;The following 2004 paper describes a way to model compositional changes across the tree, in a Bayesian framework: &lt;a href=&quot;https://doi.org/10.1080/10635150490445779&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1080/10635150490445779&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A python package implementing this (&quot;p4&quot;), and improvements added along the years, is available here: &lt;a href=&quot;https://github.com/pgfoster/p4-phylogenetics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/pgfoster/p4-phylogenetics&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get started, you may find useful examples here: &lt;a href=&quot;http://p4.nhm.ac.uk/scripts.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://p4.nhm.ac.uk/scripts.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has been used in a few large-scale phylogenetic analyses.&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2017-05-19T09:18:45.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="142" PostTypeId="2" ParentId="112" CreationDate="2017-05-19T09:33:58.507" Score="4" Body="&lt;p&gt;The way we deal with this is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;All work is done in a single filesystem mounted on the cluster&lt;/li&gt;&#xA;&lt;li&gt;This file system is mounted on local machines via sshfs/samba (depending on the location of the current &quot;local&quot; machine on the network).&lt;/li&gt;&#xA;&lt;li&gt;Code is versioned with git hub&lt;/li&gt;&#xA;&lt;li&gt;All computation is carried out via light-weight automated pipelines. We use ruffus in combination with an in-house utility layer. The system doesn't really matter as long is it no more work to add another step to the pipeline than it would be to execute it manually. &lt;/li&gt;&#xA;&lt;li&gt;All questionably design decisions are encoded in configuration files. These configuration files, along with a very detailed log output by the pipeline (what was run, what was the git commit of the code run, what was the time stamp of the files it was run on, etc) and the initial input files are version controlled.&lt;/li&gt;&#xA;&lt;li&gt;The benefit of this is that code + configuration + time = output. It is not expected that the whole pipeline will be rerun everytime anything is changed, but the pipeline will tell you if something is out of date (it can use timestamps or file hashes), and it can all be run in one go before publication.&lt;/li&gt;&#xA;&lt;li&gt;Any other analysis is carried out in juptyer notebooks. These are version controlled.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To summarise, we don't synchronise because we only ever work from one disk location even if we use multiple CPU locations. We version control:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Code&lt;/li&gt;&#xA;&lt;li&gt;Inputs, configuration, logs&lt;/li&gt;&#xA;&lt;li&gt;Juptyer notebooks&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Log records the git commits used to produce the current outputs. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2017-05-19T09:33:58.507" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="143" PostTypeId="1" CreationDate="2017-05-19T10:09:11.327" Score="4" ViewCount="115" Body="&lt;p&gt;I currently have ~180 whole germlines and around 10M SNPs/indels. I would like to build a predictive model using Machine Learning (ML) techniques to predict cancer risk according to these germline variants. The thing is, most of these 10M variants are not relevant to cancer, so first I have to annotate them and remove non-relevant SNPs/indels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which tools/measurements do you think are important to use in order to filter out irrelevant variants and keep only, let's say, around 10,000 SNPs/indels?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess the most obvious is CADD score, so I could keep those SNPs/indels with PHRED &gt;= 10.&#xA;Which one would you suggest taking into account that downstream analysis would be ML algorithms that take as features variants that &lt;strong&gt;really&lt;/strong&gt; contribute to cancer?&lt;/p&gt;&#xA;" OwnerUserId="328" LastEditorUserId="298" LastEditDate="2017-05-20T21:54:37.907" LastActivityDate="2018-03-07T10:47:55.103" Title="How can I use annotations to remove variants not relevant to cancer risk?" Tags="|ngs|sequence-annotation|machine-learning|cancer|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="144" PostTypeId="2" ParentId="120" CreationDate="2017-05-19T10:18:40.500" Score="6" Body="&lt;p&gt;I think the best method or combination of methods will depend on aspects of the data that might vary from one dataset to another. E.g. the type, size, and frequency of structural variants, the number SNVs, the quality of the reference, contaminants or other issues (e.g. read quality, sequencing errors) etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For that reason, I'd take two approaches:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Try a lot of methods, and look at their overlap&lt;/li&gt;&#xA;&lt;li&gt;Validate a subset of calls from different methods by wet lab experiments - in the end this is the only real way of knowing the accuracy for a particular case.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="156" LastEditorUserId="57" LastEditDate="2017-05-22T03:36:59.117" LastActivityDate="2017-05-22T03:36:59.117" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="145" PostTypeId="2" ParentId="143" CreationDate="2017-05-19T10:24:51.223" Score="5" Body="&lt;p&gt;While your question is specific to cancerous germline mutations, I'd suggest you look at the &lt;a href=&quot;http://cancer.sanger.ac.uk/cosmic&quot; rel=&quot;noreferrer&quot;&gt;COSMIC&lt;/a&gt; database of somatic mutations to include in your analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other factors to include in this kind of analysis you're suggesting, such as predictive deleterious effects (PolyPhen for example can perform such predictions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have 10M variants/indels, for the same form of cancer, then look for common variants, or maybe the frequency of variants identified across exons of a gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you've asked is highly broad and we'd need more specific details about your cohort to suggest more specific approaches. &lt;/p&gt;&#xA;" OwnerUserId="222" LastActivityDate="2017-05-19T10:24:51.223" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="146" PostTypeId="1" CreationDate="2017-05-19T10:32:45.047" Score="14" ViewCount="506" Body="&lt;p&gt;In Computer Science a &lt;a href=&quot;https://en.wikipedia.org/wiki/De_Bruijn_graph&quot; rel=&quot;noreferrer&quot;&gt;De Bruijn graph&lt;/a&gt; has (1) &lt;code&gt;m^n&lt;/code&gt; vertices representing all possible  sequences of length &lt;code&gt;n&lt;/code&gt; over &lt;code&gt;m&lt;/code&gt; symbols, and (2) directed edges connecting nodes that differ by a shift of &lt;code&gt;n-1&lt;/code&gt; elements (the successor having the new element at the right).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However in Bioinformatics while condition (2) is preserved, what is called a De Bruijn graph &lt;a href=&quot;http://www.homolog.us/Tutorials/index.php?p=2.1&amp;amp;s=1&quot; rel=&quot;noreferrer&quot;&gt;doesn't seem to respect condition&lt;/a&gt; (1). In some cases the graph doesn't look anything like a de Bruijn graph at all (e.g. &lt;a href=&quot;http://genome.cshlp.org/content/18/5/821.full&quot; rel=&quot;noreferrer&quot;&gt;http://genome.cshlp.org/content/18/5/821.full&lt;/a&gt;).     &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, if I want to make it explicit that I am using the Bioinformatics interpretation of a de Bruijn graph, is there a term for it? Something like &quot;simplified de Bruijn graph&quot;, &quot;projection of a de Bruijn graph&quot;, or &quot;graph of neighbouring k-mers&quot;? Are there any papers making this distinction, or did I get it all wrong?&lt;/p&gt;&#xA;" OwnerUserId="45" LastEditorUserId="96" LastEditDate="2019-01-15T14:50:16.330" LastActivityDate="2019-01-15T14:50:16.330" Title="How to make a distinction between the &quot;classical&quot; de Bruijn graph and the one described in NGS papers?" Tags="|ngs|assembly|k-mer|de-bruijn-graphs|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="147" PostTypeId="2" ParentId="146" CreationDate="2017-05-19T11:07:00.467" Score="4" Body="&lt;p&gt;In addition to the regular De Bruijn graph as depicted on the wikipedia, some implementations in bioinformatics feature additional processing. I guess the main reason figure 1 in the paper you linked (concerning the Velvet genome assembler) is slightly different is that a node represents &lt;em&gt;a series of overlapping k-mers&lt;/em&gt;. In order to visualize this as a more classic De Bruin graph you would have to connect the k-mers depicted &lt;em&gt;above&lt;/em&gt; the nodes. The caption next to figure one describes the processing quite clearly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As per your last question: I don't think there is a 'Bioinformatic interpretation of a De Bruijn graph'. There are different implementations, which all have there specifics. Thus it would be best to refer to the actual implementation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/27587666&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; is a nice paper on how to construct a pan-genome De Bruijn graph of multiple genomes simultaneously.  &lt;/p&gt;&#xA;" OwnerUserId="196" LastActivityDate="2017-05-19T11:07:00.467" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="149" PostTypeId="1" AcceptedAnswerId="166" CreationDate="2017-05-19T12:32:54.203" Score="13" ViewCount="3708" Body="&lt;p&gt;Several gene set enrichment methods are available, the most famous/popular is the &lt;a href=&quot;http://software.broadinstitute.org/gsea/index.jsp&quot; rel=&quot;noreferrer&quot;&gt;Broad Institute tool&lt;/a&gt;. Many other tools are available (See for example the &lt;a href=&quot;https://bioconductor.org/packages/release/BiocViews.html#___GeneSetEnrichment&quot; rel=&quot;noreferrer&quot;&gt;biocView of GSE&lt;/a&gt; which list 82 different packages). There are several parameters in consideration :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the statistic used to order the genes, &lt;/li&gt;&#xA;&lt;li&gt;if it competitive or self-contained,&lt;/li&gt;&#xA;&lt;li&gt;if it is supervised or not,&lt;/li&gt;&#xA;&lt;li&gt;and how is the enrichment score calculated.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am using the &lt;a href=&quot;http://bioconductor.org/packages/fgsea&quot; rel=&quot;noreferrer&quot;&gt;fgsea - Fast Gene Set Enrichment Analysis&lt;/a&gt; package to calculate the enrichment scores and someone told me that the numbers are different from the ones on the Broad Institute despite all the other parameters being equivalent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are these two methods (fgsea and Broad Institute GSEA) equivalent to calculate the enrichment score?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I looked to the algorithms of both papers, and they seem fairly similar, but I don't know if in real datasets they are equivalent or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;Is there any article reviewing and comparing how does the enrichment score method affect to the result?&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="131" LastEditDate="2017-05-22T09:32:02.147" LastActivityDate="2017-05-22T09:32:02.147" Title="Are fgsea and Broad Institute GSEA equivalent?" Tags="|r|gse|bioconductor|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="150" PostTypeId="1" CreationDate="2017-05-19T13:57:30.553" Score="6" ViewCount="426" Body="&lt;p&gt;I have an &lt;a href=&quot;http://datadryad.org/bitstream/handle/10255/dryad.122664/obitools_script.txt?sequence=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;obitools script&lt;/a&gt; (de Barba et al. 2016) that I would like to run faster. How would you run it in parallel to cut down on time?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;illuminapairedend -r rawdata_scandinavia_R2.fastq rawdata_scandinavia_R1.fastq | tee rawdata_scandinavia.fastq | obiannotate -S goodAli:'&quot;Alignement&quot; if score&amp;gt;40.00 else &quot;Bad&quot;' | obisplit -t goodAli -p rawdata_scandinavia.&#xA;touch rawdata_scandinavia.Bad.fastq&#xA;touch rawdata_scandinavia.Alignement.fastq&#xA;touch rawdata_scandinavia.fastq&#xA;ngsfilter -t rawdata_scandinavia.ngsfilter -u rawdata_scandinavia.unidentified.fastq rawdata_scandinavia.Alignement.fastq &amp;gt; rawdata_scandinavia.filtered.fastq&#xA;obisplit -p MICROSAT.PCR_ -t experiment rawdata_scandinavia.filtered.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can someone describe what tools and steps one needs to take to run this job in parallel, given that the functions are not designed to be run in parallel out of the box?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;De Barba, M., Miquel, C., Lobréaux, S., Quenette, P. Y., Swenson, J. E., &amp;amp; Taberlet, P. (2016). High-throughput microsatellite genotyping in ecology: improved accuracy, efficiency, standardization and success with low-quantity and degraded DNA. Molecular Ecology Resources, 1–16. &lt;a href=&quot;https://doi.org/10.1111/1755-0998.12594&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1111/1755-0998.12594&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="200" LastEditorUserId="200" LastEditDate="2017-06-09T08:05:55.257" LastActivityDate="2017-06-09T08:05:55.257" Title="Parallel processing of scripts that use obitools" Tags="|ngs|linux|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="151" PostTypeId="2" ParentId="150" CreationDate="2017-05-19T14:03:08.770" Score="3" Body="&lt;p&gt;Your only option here is to first split the fastq files into small chunks, process each and then merge them back together. Alternatively, find something else that runs faster.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-05-19T14:03:08.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="152" PostTypeId="2" ParentId="146" CreationDate="2017-05-19T14:14:34.317" Score="3" Body="&lt;p&gt;Let's first assume DNA only has one strand. An assembly de Bruijn graph is a subgraph of a complete de Bruijn graph. It contains a vertex u if u is a k-mer in reads; it contains an edge u-&gt;v, if u and v are adjacent k-mers on a read. Alternatively, we note that an edge u-&gt;v is represented by a (k+1)-mer. An assembly de Bruijn graph can be considered a subgraph edge induced from all (k+1)-mers in reads – in fact, some assemblers take the list of (k+1)-mer as a succinct representation of de Bruijn graphs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DNA has two strands. We just need to induce an assembly de Bruijn graph from all (k+1)-mers and their reverse complement. It is still a subgraph of a complete de Bruijn graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because an assembly de Bruijn graph is just a subgraph. It is not necessary to give it a new name.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I deleted my old answer as that was not what you are asking for based on your comments. I was confused by your mentioning velvet. Velvet uses an equivalent but uncommon representation of de Bruijn graphs, which complicates your question.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-05-19T14:14:34.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="153" PostTypeId="1" AcceptedAnswerId="192" CreationDate="2017-05-19T14:40:13.787" Score="19" ViewCount="10688" Body="&lt;p&gt;I have a set of high-troughput experiments with 2 genotypes (&amp;quot;WT&amp;quot; and &amp;quot;prg1&amp;quot;) and 3 treatments (&amp;quot;RT&amp;quot;, &amp;quot;HS30&amp;quot; and &amp;quot;HS30RT120&amp;quot;), and there are 2 replicates for each of the genotype x treatment combinations.&lt;/p&gt;&#xA;&lt;p&gt;The read counts for the genes are summarized in a file that I load as follows in R:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; counts_data &amp;lt;- read.table(&amp;quot;path/to/my/file&amp;quot;, header=TRUE, row.names=&amp;quot;gene&amp;quot;)&#xA;&amp;gt; colnames(counts_data)&#xA; [1] &amp;quot;WT_RT_1&amp;quot;          &amp;quot;WT_HS30_1&amp;quot;        &amp;quot;WT_HS30RT120_1&amp;quot;   &amp;quot;prg1_RT_1&amp;quot;       &#xA; [5] &amp;quot;prg1_HS30_1&amp;quot;      &amp;quot;prg1_HS30RT120_1&amp;quot; &amp;quot;WT_RT_2&amp;quot;          &amp;quot;WT_HS30_2&amp;quot;       &#xA; [9] &amp;quot;WT_HS30RT120_2&amp;quot;   &amp;quot;prg1_RT_2&amp;quot;        &amp;quot;prg1_HS30_2&amp;quot;      &amp;quot;prg1_HS30RT120_2&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I describe the experiments as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; col_data &amp;lt;- DataFrame(&#xA;    geno = c(rep(&amp;quot;WT&amp;quot;, times=3), rep(&amp;quot;prg1&amp;quot;, times=3), rep(&amp;quot;WT&amp;quot;, times=3), rep(&amp;quot;prg1&amp;quot;, times=3)),&#xA;    treat = rep(c(&amp;quot;RT&amp;quot;, &amp;quot;HS30&amp;quot;, &amp;quot;HS30RT120&amp;quot;), times=4),&#xA;    rep = c(rep(&amp;quot;1&amp;quot;, times=6), rep(&amp;quot;2&amp;quot;, times=6)),&#xA;    row.names = colnames(counts_data))&#xA;&amp;gt; col_data&#xA;DataFrame with 12 rows and 3 columns&#xA;                        geno       treat         rep&#xA;                 &amp;lt;character&amp;gt; &amp;lt;character&amp;gt; &amp;lt;character&amp;gt;&#xA;WT_RT_1                   WT          RT           1&#xA;WT_HS30_1                 WT        HS30           1&#xA;WT_HS30RT120_1            WT   HS30RT120           1&#xA;prg1_RT_1               prg1          RT           1&#xA;prg1_HS30_1             prg1        HS30           1&#xA;...                      ...         ...         ...&#xA;WT_HS30_2                 WT        HS30           2&#xA;WT_HS30RT120_2            WT   HS30RT120           2&#xA;prg1_RT_2               prg1          RT           2&#xA;prg1_HS30_2             prg1        HS30           2&#xA;prg1_HS30RT120_2        prg1   HS30RT120           2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to build a DESeq2 object that I could use to either:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;find differentially expressed genes when the treatment varies for a given fixed genotype&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;or:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;find differentially expressed genes when the genotype varies for a given fixed treatment&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In the &lt;a href=&quot;https://support.bioconductor.org/p/74101/#74163&quot; rel=&quot;noreferrer&quot;&gt;bioconductor help forum&lt;/a&gt; I think I've found a somewhat similar situation, and I read the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Try a design of ~ genotype + genotype:condition&lt;/p&gt;&#xA;&lt;p&gt;Then you will have a condition effect for each level of genotype, including the reference level.&lt;/p&gt;&#xA;&lt;p&gt;You can constrast pairs of them using the list style of the 'contrast' argument.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However, this doesn't explain how to apply this &amp;quot;list style&amp;quot; to the &amp;quot;contrast&amp;quot; argument. And the above situation seems to be asymmetrical. By that I mean that genotype and condition do not seem to have an interchangeable role.&lt;/p&gt;&#xA;&lt;p&gt;So I tried the following more symmetric formula:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dds &amp;lt;- DESeqDataSetFromMatrix(&#xA;                                countData = counts_data,&#xA;                                colData = col_data,&#xA;                                design = ~ geno + treat + geno:treat)&#xA;&amp;gt; dds &amp;lt;- DESeq(dds)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, can I for instance &lt;strong&gt;get the differential expression results when comparing treatment &amp;quot;HS30&amp;quot; against &amp;quot;RT&amp;quot; as a reference, in genotype &amp;quot;prg1&amp;quot;&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;p&gt;And how?&lt;/p&gt;&#xA;&lt;p&gt;If I understand correctly, the above-mentioned &amp;quot;list style&amp;quot; uses names given by the &lt;code&gt;resultsNames&lt;/code&gt; function. In my case, I have the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; resultsNames(dds)&#xA;[1] &amp;quot;Intercept&amp;quot;               &amp;quot;geno_WT_vs_prg1&amp;quot;        &#xA;[3] &amp;quot;treat_HS30RT120_vs_HS30&amp;quot; &amp;quot;treat_RT_vs_HS30&amp;quot;       &#xA;[5] &amp;quot;genoWT.treatHS30RT120&amp;quot;   &amp;quot;genoWT.treatRT&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I guess I would need a contrast between &amp;quot;genoprg1.treatRT&amp;quot; and a &amp;quot;genoprg1.treatHS30&amp;quot;, but these are not in the above results names.&lt;/p&gt;&#xA;&lt;p&gt;I'm lost.&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-06-02T16:04:30.200" Title="Understanding DESeq2 design, contrast and results" Tags="|r|bioconductor|deseq2|differential-expression|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="154" PostTypeId="2" ParentId="153" CreationDate="2017-05-19T14:51:01.017" Score="16" Body="&lt;p&gt;The simplest manner is to not use a wald test, but rather an LRT with a reduced model lacking the factor of interest:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dds = DESeq(dds, test=&quot;LRT&quot; reduced=~geno+geno:Treatment)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above would give you results for Treatment regardless of level while still accounting for a possible interaction (i.e., a &quot;main effect of treatment, regardless of the type of treatment&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an aside, this is probably a case where the edgeR-preferred way of creating groups of genotype-treatment combinations and then using a model of &lt;code&gt;~0+group&lt;/code&gt; might make your life a bit easier. You'll get the same results (more or less) regardless, but it'll probably be easier for you to think in those terms rather than remembering that the base level will be &lt;code&gt;treatment HS30&lt;/code&gt; and &lt;code&gt;geno prg1&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-05-19T14:51:01.017" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="155" PostTypeId="2" ParentId="134" CreationDate="2017-05-19T17:44:29.193" Score="6" Body="&lt;p&gt;According to the &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAM specification&lt;/a&gt;, the 3rd field of a SAM line (&lt;code&gt;RNAME&lt;/code&gt;) is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;RNAME: Reference sequence NAME of the alignment. If @SQ header lines&#xA;  are present, RNAME (if not ‘*’) must be present in one of the SQ-SN&#xA;  tag. An unmapped segment without coordinate has a ‘*’ at this field.&#xA;  However, an unmapped segment may also have an ordinary coordinate such&#xA;  that it can be placed at a desired position after sorting. If RNAME is&#xA;  ‘*’, no assumptions can be made about POS and CIGAR.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And the 7th field is (emphasis mine, missing &quot;to&quot; theirs):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;RNEXT: Reference sequence name of the primary alignment of the NEXT&#xA;  read in the template. For the last read, the next read is the first&#xA;  read in the template. If @SQ header lines are present, RNEXT (if not&#xA;  ‘*’ or ‘=’) must be present in one of the SQ-SN tag. This field is set&#xA;  as ‘*’ when the information is unavailable, &lt;strong&gt;and set as ‘=’ if RNEXT is&#xA;  identical RNAME&lt;/strong&gt;. If not ‘=’ and the next read in the template has one&#xA;  primary mapping (see also bit 0x100 in FLAG), this field is identical&#xA;  to RNAME at the primary line of the next read. If RNEXT is ‘*’, no&#xA;  assumptions can be made on PNEXT and bit 0x20&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, you want to remove those lines whose 7th field isn't &lt;code&gt;=&lt;/code&gt; and, just in case, those lines whose 7th field isn't &lt;code&gt;=&lt;/code&gt; &lt;em&gt;and&lt;/em&gt; isn't the same as the 3rd field. You can therefore use something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -L test.bed test.bam | awk '$7==&quot;=&quot; || $3==$7&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And, to save as a bam file again:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -L test.bed test.bam | awk '$7==&quot;=&quot; &amp;amp;&amp;amp; $3==$7 | &#xA;    samtolls view -b &amp;gt; fixed.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;On a separate note, there's very rarely a need to chain multiple grep commands like that. You can just use &lt;code&gt;\|&lt;/code&gt; (or &lt;code&gt;|&lt;/code&gt; with the &lt;code&gt;-E&lt;/code&gt; or &lt;code&gt;-P&lt;/code&gt; options) to separate them. Something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -L test.bed test.bam | grep -v 'chrx\|chr2\|chr10\|chrN'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -L test.bed test.bam | grep -Ev 'chrx|chr2|chr10|chrN'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-05-21T23:21:14.657" LastActivityDate="2017-05-21T23:21:14.657" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="156" PostTypeId="1" AcceptedAnswerId="158" CreationDate="2017-05-19T18:34:20.767" Score="35" ViewCount="2602" Body="&lt;p&gt;Why do some assemblers like &lt;a href=&quot;https://github.com/aquaskyline/SOAPdenovo2&quot; rel=&quot;noreferrer&quot;&gt;SOAPdenovo2&lt;/a&gt; or &lt;a href=&quot;https://github.com/dzerbino/velvet&quot; rel=&quot;noreferrer&quot;&gt;Velvet&lt;/a&gt; require an odd-length &lt;em&gt;k&lt;/em&gt;-mer size for the construction of de Bruijn graph, while some other assemblers like &lt;a href=&quot;https://github.com/bcgsc/abyss&quot; rel=&quot;noreferrer&quot;&gt;ABySS&lt;/a&gt; are fine with even-length &lt;em&gt;k&lt;/em&gt;-mers?&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="96" LastEditDate="2019-02-28T13:12:20.163" LastActivityDate="2020-04-24T12:03:22.037" Title="Why do some assemblers require an odd-length kmer for the construction of de Bruijn graphs?" Tags="|ngs|assembly|k-mer|de-bruijn-graphs|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="157" PostTypeId="1" AcceptedAnswerId="160" CreationDate="2017-05-19T18:50:05.307" Score="14" ViewCount="1725" Body="&lt;p&gt;If there are soft clipped base pairs specified in the CIGAR string for a read in a SAM/BAM file, will these be used for variant calling in a &lt;a href=&quot;http://www.htslib.org/doc/samtools.html&quot; rel=&quot;noreferrer&quot;&gt;samtools&lt;/a&gt; + &lt;a href=&quot;http://www.htslib.org/doc/bcftools.html&quot; rel=&quot;noreferrer&quot;&gt;bcftools&lt;/a&gt; workflow? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/tooldocs/current/org_broadinstitute_gatk_tools_walkers_haplotypecaller_HaplotypeCaller.php&quot; rel=&quot;noreferrer&quot;&gt;GATK HaplotypeCaller&lt;/a&gt;, for example, has an explicit option &lt;code&gt;--dontUseSoftClippedBases&lt;/code&gt; for whether to use soft clipped bases. The samtools documentation does not mention clipped bases. &lt;/p&gt;&#xA;" OwnerUserId="272" LastActivityDate="2017-05-19T20:15:11.303" Title="Are soft-clipped bases used for variant calling in samtools + bcftools?" Tags="|sam|samtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="158" PostTypeId="2" ParentId="156" CreationDate="2017-05-19T18:52:34.737" Score="31" Body="&lt;p&gt;From the &lt;a href=&quot;http://www.ebi.ac.uk/~zerbino/velvet/Manual.pdf&quot; rel=&quot;noreferrer&quot;&gt;manual of Velvet&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;it must be an odd number, to avoid palindromes. If you put in an even&#xA;  number, Velvet will just decrement it and proceed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;the &lt;a href=&quot;https://en.wikipedia.org/wiki/Palindromic_sequence&quot; rel=&quot;noreferrer&quot;&gt;palindromes&lt;/a&gt; in biology are defined as reverse complementary sequences. The problem of palindromes is explained in this &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2874646/&quot; rel=&quot;noreferrer&quot;&gt;review&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Palindromes induce paths that fold back on themselves. At least one&#xA;  assembler avoids these elegantly; Velvet requires K, the length of a&#xA;  K-mer, to be odd. An odd-size K-mer cannot match its reverse&#xA;  complement.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is possible to construct graph with palindromes, but then the interpretation will be harder. Allowing only graphs of odd &lt;em&gt;k&lt;/em&gt;-mers is just an elegant way to avoid writing a code for interpretation of a more complicated graph.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="96" LastEditDate="2019-02-28T13:13:44.453" LastActivityDate="2019-02-28T13:13:44.453" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="159" PostTypeId="1" AcceptedAnswerId="163" CreationDate="2017-05-19T18:59:48.617" Score="8" ViewCount="955" Body="&lt;p&gt;Does anyone have experience generating pdb structures with Phyre and ITasser online tools. The results generated from each given the same amino acid sequence input are very different and I am wondering whether or not this is a usual experience. I know ITasser was the number 1 rated in the CASP trails, but should the results really be this disparate?&lt;/p&gt;&#xA;" OwnerUserId="65" LastEditorUserId="57" LastEditDate="2017-05-24T12:13:12.043" LastActivityDate="2019-04-04T08:13:28.417" Title="Phyre2 vs ITasser, completely different models generated" Tags="|proteins|protein-structure|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="160" PostTypeId="2" ParentId="157" CreationDate="2017-05-19T19:19:13.093" Score="15" Body="&lt;p&gt;No, samtools (and therefore bcftools) does not use soft-clipped bases. You can quickly confirm this by using either &lt;code&gt;samtools depth&lt;/code&gt; or &lt;code&gt;samtools mpileup&lt;/code&gt; to look at a region with a soft-clipped alignment. You'll note that the soft-clipped region isn't used in the depth/pileup (both tools use the same underlying code, so it doesn't matter which you use). If you're curious, samtools ignores soft-clipped bases because it's based on making a per-base stack of alignments covering each position. In the BAM format, alignments are sorted and assigned to bins according to their start/end positions, which won't include soft-clipping. Consequently, when samtools is making the pileup it won't even see the alignments that would overlap a given base if soft-clipped bases were included.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This then sort of begs the question of what GATK's HaplotypeCaller is doing differently. There, regions in the genome are essentially assembled in a small de Bruijn graph, which allows for soft-clipped bases around indels to then be resolved, given that the graph would start/end a little-way on past each side of indels. This is also why you don't need to do indel realignment with the HaplotypeCaller (this was needed in the old UnifiedGenotyper).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: For more details regarding the HaplotypeCaller, see &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/article.php?id=4146&quot; rel=&quot;noreferrer&quot;&gt;this nice page&lt;/a&gt; on GATK's website, which goes into much more detail than I did here.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-05-19T20:15:11.303" LastActivityDate="2017-05-19T20:15:11.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="161" PostTypeId="1" AcceptedAnswerId="167" CreationDate="2017-05-20T00:21:08.950" Score="9" ViewCount="6908" Body="&lt;p&gt;The peak calling tool MACS2 can call peaks in either narrow peak mode (for focused signals like transcription factor ChIPseq) or broad peak mode (for more defuse signals, like certain histone modifications). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm for narrow peak calling is well described in the MACS publication. But I don't find much documentation for how peak calling is different in broad peak mode. The manual only contains the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;--broad&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;When this flag is on, MACS will try to composite broad regions in&#xA;  BED12 ( a gene-model-like format ) by putting nearby highly enriched&#xA;  regions into a broad region with loose cutoff. The broad region is&#xA;  controlled by another cutoff through --broad-cutoff. The maximum&#xA;  length of broad region length is 4 times of d from MACS&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But this doesn't really describe exactly how this is performed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what is the algorithm MACS uses for calling broad peaks?&lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2017-05-24T10:01:42.090" Title="How are MACS2's narrow peak and broad peak algorithms different?" Tags="|algorithms|chip-seq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="163" PostTypeId="2" ParentId="159" CreationDate="2017-05-20T04:11:29.220" Score="5" Body="&lt;p&gt;I'm less familiar with Phyre, but I-TASSER is a really sophisticated system that takes the results of a search using multiple threaders and plugs them into an ab initio simulation which tries to minimize the energy of the models by sampling many possible 3D conformations, which I don't think Phyre does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/I-TASSER#/media/File:I-TASSER-pipeline.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/I-TASSER#/media/File:I-TASSER-pipeline.jpg&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compare with a similar workflow schematic for Phyre:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.nature.com/nprot/journal/v10/n6/images/nprot.2015.053-F1.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.nature.com/nprot/journal/v10/n6/images/nprot.2015.053-F1.jpg&lt;/a&gt;&#xA;&lt;a href=&quot;http://www.nature.com/nprot/journal/v10/n6/fig_tab/nprot.2015.053_F2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.nature.com/nprot/journal/v10/n6/fig_tab/nprot.2015.053_F2.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Structure prediction still has a long way to go, and you'll always get better results if there are close homologues available in the PDB, but given the consistent high performance of I-TASSER in CASP I would treat those results as more significant. That said, it can't hurt to consider multiple answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: included blmoore's link to second half of Phyre protocol schematic&lt;/p&gt;&#xA;" OwnerUserId="138" LastEditorUserId="138" LastEditDate="2017-05-24T18:17:03.157" LastActivityDate="2017-05-24T18:17:03.157" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="164" PostTypeId="2" ParentId="120" CreationDate="2017-05-20T08:33:58.813" Score="3" Body="&lt;p&gt;In case if you are really dedicated to obtain perfect results, you can use strategy described &lt;a href=&quot;http://www.nature.com/nature/journal/v526/n7571/full/nature15394.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;there&lt;/a&gt;, in 1000GP 3rd Phase SV detection paper - use these tools, validate your calls with IRS test, merge calls into one callset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do not wanna spend thousands human-hours as was spent during this paper preparation, from my experience, it is better to use 1 paired-end insert distance method and one read-depth based method. Each of them cover &quot;different&quot; regions in the genome. (even thou they have huge overlap, paired-end detection requires both SV breakpoints to be located within the regions with good mappability which is not always the case, but resolution of read-depth methods is lower in general, paired-ends works well for deletions/tandem duplications/inversions, but have troubles with non-tandem duplications).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps.&lt;/p&gt;&#xA;" OwnerUserId="349" LastActivityDate="2017-05-20T08:33:58.813" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="165" PostTypeId="1" CreationDate="2017-05-20T11:31:59.123" Score="9" ViewCount="222" Body="&lt;p&gt;The completeness of a module can easily be checked by looking at the &lt;code&gt;Definition&lt;/code&gt; entry associated with the module. For example, in module &lt;a href=&quot;http://www.genome.jp/kegg-bin/show_module?M00010&quot; rel=&quot;nofollow noreferrer&quot;&gt;M00010&lt;/a&gt;, it is given as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Definition  K01647 (K01681,K01682) (K00031,K00030)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which can be translated to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;K01647 AND (K01681 OR K01682) AND (K00031 OR K00030)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If this expression evaluates to &lt;code&gt;TRUE&lt;/code&gt;, the module is considered complete.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering whether analogous information exists for a single &lt;em&gt;reaction&lt;/em&gt;. For example, when looking at reaction &lt;a href=&quot;http://www.genome.jp/dbget-bin/www_bget?rn:R00352&quot; rel=&quot;nofollow noreferrer&quot;&gt;R00352&lt;/a&gt;, one finds the following information about its Orthology:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9jRBq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9jRBq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I can't easily find the logical relation that correspond to the KOs for this reaction. It could be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;K01648 AND K15230 AND K15231 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;K01648 OR K15230 OR K15231&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;K01648 OR (K15230 AND K15231)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the above example, I know that the correct expression is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;K01648 OR (K15230 AND K15231)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;i.e. one either needs either &lt;code&gt;K01648&lt;/code&gt; or both of the other two subunits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can this information be retrieved from KEGG for each reaction and if so, how?&lt;/p&gt;&#xA;" OwnerUserId="351" LastEditorUserId="73" LastEditDate="2017-12-28T11:54:32.443" LastActivityDate="2019-05-26T07:07:10.983" Title="How to retrieve logical expressions (KO based) for reactions from KEGG?" Tags="|kegg|database|data-retrieval|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="166" PostTypeId="2" ParentId="149" CreationDate="2017-05-20T20:24:45.047" Score="7" Body="&lt;p&gt;According to the &lt;a href=&quot;http://biorxiv.org/content/early/2016/06/20/060012&quot; rel=&quot;nofollow noreferrer&quot;&gt;FGSEA preprint&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We ran reference GSEA with default parameters. The permutation number&#xA;  was set to 1000, which means that for each input gene set 1000&#xA;  independent samples were generated. The run took 100 seconds and&#xA;  resulted in 79 gene sets with GSEA-adjusted FDR q-value of less than&#xA;  10−2. All significant gene sets were in a positive mode. First, to get&#xA;  a similar nominal p-values accuracy we ran FGSEA algorithm on 1000&#xA;  permutations. This took 2 seconds, but resulted in no significant hits&#xA;  due after multiple testing correction (with FRD ≤ 1%).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Thus, FGSEA and GSEA are not identical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And again in the conclusion:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Consequently, gene sets can be ranked more precisely in the results&#xA;  and, which is even more important, standard multiple testing&#xA;  correction methods can be applied instead of approximate ones as in&#xA;  [GSEA].&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The author argues that FGSEA is more accurate, so it can't be equivalent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested specifically in the enrichment score, that was &lt;a href=&quot;http://disq.us/p/1f683mc&quot; rel=&quot;nofollow noreferrer&quot;&gt;addressed by the author in the preprint comments&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Values of enrichment scores and normalized enrichment scores are the&#xA;  same for both broad version and fgsea.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So that part seems to be the same.&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="35" LastEditDate="2017-05-21T19:52:04.773" LastActivityDate="2017-05-21T19:52:04.773" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="167" PostTypeId="2" ParentId="161" CreationDate="2017-05-20T21:38:54.743" Score="10" Body="&lt;p&gt;The key function is &lt;a href=&quot;https://github.com/taoliu/MACS/blob/master/MACS2/IO/CallPeakUnit.pyx#L1443&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;call_broadpeaks&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The description attached to the function says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This function try to find enriched regions within which, scores are&#xA;  continuously higher than a given cutoff for level 1, and link them&#xA;  using the gap above level 2 cutoff with a maximum length of&#xA;  lvl2_max_gap.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;scoring_function_s: symbols of functions to calculate&#xA;  score. 'p' for pscore, 'q' for qscore, 'f' for fold change, 's' for&#xA;  subtraction. for example: ['p', 'q']&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;lvl1_cutoff_s:  list of cutoffs&#xA;  at highly enriched regions, corresponding to scoring functions.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;lvl2_cutoff_s:  list of cutoffs at less enriched regions,&#xA;  corresponding to scoring functions.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;min_length :minimum peak length,&#xA;  default 200.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;lvl1_max_gap   :  maximum gap to merge nearby enriched&#xA;  peaks, default 50.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;lvl2_max_gap   :  maximum length of linkage&#xA;  regions, default 400.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Return both general PeakIO object for highly&#xA;  enriched regions and gapped broad regions in BroadPeakIO.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To give some basic explanation, the algorithm (briefly) appears to be as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Two separate levels of peaks are called, level 1 (a higher pval, ie more significant) and level 2 (a lower pval). Level 1 is controlled by &lt;code&gt;-p&lt;/code&gt; and level 2 is controlled by &lt;code&gt;--broad-cutoff&lt;/code&gt;. When each peakset is called, they are immediately linked by the max gap parameter for each set.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then, assuming that all level 1 peaks should be inside level 2 peaks (this is an explicit assumption by MACS2), the algorithm groups level 1 peaks inside level 2 peaks to output a broad peak.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has a few implications:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The broad peak calls really come from the level 2 peaks alone (+ linking). The level 1 peak calls allow you to distinguish sub peaks (so that you can have gapped peaks).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Aside from the linking, the broad peak calls would be the same as narrow peak calls, if you called both with the same pval threshold (for example, if you set &lt;code&gt;--broad-cutoff 0.1&lt;/code&gt; in broad peak mode, and the &lt;code&gt;-p 0.1&lt;/code&gt; for narrow peak mode)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="13" LastEditorUserId="29" LastEditDate="2017-05-24T10:01:42.090" LastActivityDate="2017-05-24T10:01:42.090" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="168" PostTypeId="1" AcceptedAnswerId="175" CreationDate="2017-05-20T21:49:51.210" Score="13" ViewCount="370" Body="&lt;p&gt;I'm looking for tools to check the quality of a VCF I have of a human genome. I would like to check the VCF against publicly known variants across other human genomes, e.g. how many SNPs are already in public databases, whether insertions/deletions are at known positions, insertion/deletion length distribution, other SNVs/SVs, etc.? I suspect that there are resources from previous projects to check for known SNPs and InDels by human subpopulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What resources exist for this, and how do I do it? &lt;/p&gt;&#xA;" OwnerUserId="146" LastEditorUserId="96" LastEditDate="2019-01-15T14:30:07.880" LastActivityDate="2019-01-15T14:30:07.880" Title="Given a VCF of a human genome, how do I assess the quality against known SNVs?" Tags="|vcf|public-databases|variation|structural-variation|snv|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="169" PostTypeId="2" ParentId="168" CreationDate="2017-05-21T01:12:08.377" Score="5" Body="&lt;p&gt;The greatest &lt;strong&gt;protein&lt;/strong&gt; coding variant catalogue is definitely &lt;a href=&quot;http://www.nature.com/nature/journal/v536/n7616/full/nature19057.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ExAC&lt;/a&gt; (&gt;65k individuals). They also published a &lt;a href=&quot;https://macarthurlab.org/2016/03/17/reproduce-all-the-figures-a-users-guide-to-exac-part-2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blogpost&lt;/a&gt; where they describe how to reproduce figures in the paper (it is a good start how to get familiar with the dataset).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the &lt;strong&gt;whole-genome&lt;/strong&gt; variants I would look at the data created by &lt;a href=&quot;http://www.internationalgenome.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;1000 genomes&lt;/a&gt; project (the latest release has more than 3k individuals). The integrated variant call sets can be downloaded though the &lt;a href=&quot;http://www.internationalgenome.org/data-portal/data-collection/phase-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;portal&lt;/a&gt; and catalogue of SVs can be found &lt;a href=&quot;http://www.internationalgenome.org/phase-3-structural-variant-dataset&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;http://www.nature.com/nature/journal/v526/n7571/full/nature15394.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper (also 1000 genomes project) they speak about non-precise placement of SVs by SV callers. I would keep this in mind for the comparison of your genome to the known variants.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2017-05-21T11:13:48.910" LastActivityDate="2017-05-21T11:13:48.910" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="170" PostTypeId="2" ParentId="168" CreationDate="2017-05-21T05:18:15.573" Score="4" Body="&lt;p&gt;Your best bet is to use programs that provide you an complete annotation of variants present in your VCF. Two examples are &lt;a href=&quot;http://snpeff.sourceforge.net&quot; rel=&quot;nofollow noreferrer&quot;&gt;snpEff&lt;/a&gt; and &lt;a href=&quot;http://annovar.openbioinformatics.org/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Annovar&lt;/a&gt;. These programs work on known variants deem different sources and provide you with information on each item in your file, that you can filter after to try to understand the effects of each variant.&lt;/p&gt;&#xA;" OwnerUserId="280" LastActivityDate="2017-05-21T05:18:15.573" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="171" PostTypeId="2" ParentId="112" CreationDate="2017-05-21T07:27:58.337" Score="6" Body="&lt;p&gt;First of all, kudos to you for taking versioning seriously. The fact that you're mindful of this issue is a good sign that you want to do responsible research!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For many bioinformatics projects, data files are so large that versioning the data directly with a tool like git is impractical. But your question is really getting at a couple of different issues.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;How do I do my research reproducibly, and show full provenance for each data point and result I produce?&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;How do I manage and synchronize my research work across multiple machines?&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The short answer&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Archive the primary data.&lt;/li&gt;&#xA;&lt;li&gt;Place your workflow under version control.&lt;/li&gt;&#xA;&lt;li&gt;Version checksums of large data files.&lt;/li&gt;&#xA;&lt;li&gt;Use GitHub to synchronize your workflow between machines.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The long answer&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Archive the primary data&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;As far as reproducibility is concerned, what matters most is the primary data: the raw, unprocessed data you collect from the instrument. If you are analyzing data that has been published by others, then write a script that will automate the task of downloading the data from its primary official source, and place that script under version control.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you or a lab mate or a colleague produced the data and it is not yet published, then you should already have plans for submitting it to an archive. Indeed, most journals and funding agencies now require this prior to publication. I'd even go so far as to say the data should be submitted as soon as it's collected. Scientists worry a lot about having their data stolen and their ideas scooped, but statistically speaking, getting scooped is much less likely than nobody ever touching your data or reading your paper. But if you or an advisor insists, most data archives allow you to keep data private for an extended period of time until a supporting manuscript is published.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Putting (for example) Fastq files in a git repository is a bad idea for a lot of reasons. No hosting service will support files that big, git will be very slow with files that big, but most importantly git/GitHub is not archival! Use a proper data archive!&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Place your workflow under version control&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Treat your raw data as read-only. Only process the raw data using scripts, and keep these scripts under version control. Vince Buffalo describes this well in his book &lt;a href=&quot;http://shop.oreilly.com/product/0636920030157.do&quot; rel=&quot;noreferrer&quot;&gt;Bioinformatics Data Skills&lt;/a&gt;. Check it out!&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Version checksums of large data files&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If there are any data files that you want to track but are too big to place under version control, compute checksums and place &lt;em&gt;these&lt;/em&gt; under version control. Checksums are very small alphanumeric strings that are, for all practical purposes, unique for each data file. So instead of putting that 5GB trimmed Fastq file or the 7GB BAM file under version control, compute their checksums and put the &lt;em&gt;checksums&lt;/em&gt; under version control. The checksums won't tell you the contents of your files, but they can tell you when the file contents change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This should give full disclosure and complete provenance for every data point in your analysis. The workflow has a scripts/command for downloading the primary data, scripts/commands for processing the data, and checksums that serve as a signature to validate intermediate and final output files. With this, anyone should be able to reproduce you analysis!&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Use GitHub to synchronize your workflow between machines&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If your workflow is already under version control with git, it's trivial to push this to a hosting service like GitHub, GitLab, or BitBucket. Then it's just a matter of using &lt;code&gt;git push&lt;/code&gt; and &lt;code&gt;git pull&lt;/code&gt; to keep your code up-to-date on your various machines.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2017-05-21T07:27:58.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="172" PostTypeId="2" ParentId="97" CreationDate="2017-05-21T07:43:43.940" Score="4" Body="&lt;p&gt;Depending on the coverage of your data and the complexity of the genome, you could either reassemble the genome &lt;em&gt;de novo&lt;/em&gt; or run a reference guided (or reference assisted) assembly. It sounds like you're leaning more towards the latter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a couple of reference-guided assembly tools available: &lt;a href=&quot;https://doi.org/10.1093/bioinformatics/btu291&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlignGraph&lt;/a&gt; and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/24931998&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ragout&lt;/a&gt;. These may or may not be appropriate depending on the organism of interest and your data types. For example, these tools are very unlikely to work well on Oxford Nanopore reads which have not been error corrected using &lt;a href=&quot;https://github.com/jts/nanopolish&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nanopolish&lt;/a&gt; or &lt;a href=&quot;http://canu.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Canu -correct&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="163" LastEditDate="2017-05-21T21:36:50.493" LastActivityDate="2017-05-21T21:36:50.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="173" PostTypeId="2" ParentId="149" CreationDate="2017-05-21T10:41:33.910" Score="4" Body="&lt;p&gt;I found that the author responded that in the &lt;a href=&quot;http://disq.us/p/1f7wolq&quot; rel=&quot;nofollow noreferrer&quot;&gt;discussion&lt;/a&gt; in the preprint,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I wonder is the enrichment score calculated the same way as in broad gsea?&lt;/p&gt;&#xA;&lt;p&gt;Values of enrichment scores and normalized enrichment scores are the same for both broad version and fgsea.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="48" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-05-21T10:41:33.910" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="174" PostTypeId="1" CreationDate="2017-05-21T11:49:58.693" Score="19" ViewCount="697" Body="&lt;p&gt;All the long-read sequencing platforms are based on single-molecule sequencing which causes higher per-base error rates. For this reason a polishing step was added to genome assembly pipelines - mapping raw reads back to assembly and correcting details of the assembly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have decent PacBio RSII dataset of single individual genome of heavily heterozygous non-model species. Assembly went well, but when I tried to polish the assembly using &lt;a href=&quot;https://github.com/PacificBiosciences/GenomicConsensus&quot; rel=&quot;noreferrer&quot;&gt;quiver&lt;/a&gt; it could not converge over a couple of iterations and I bet it is because of too great divergence of haplotypes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any other way to polish a genome with such properties?&#xA;For instance, is there a way to separate long reads by haplotype, so I could polish using one haplotype only?&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="73" LastEditDate="2017-05-22T09:28:42.417" LastActivityDate="2021-01-04T12:43:00.533" Title="How to deal with heterozygosity during polishing of genome assembly based on long reads?" Tags="|ngs|assembly|long-reads|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="175" PostTypeId="2" ParentId="168" CreationDate="2017-05-21T14:48:39.303" Score="7" Body="&lt;p&gt;To achieve (at least some of) your goals, I would recommend the &lt;a href=&quot;http://grch37.ensembl.org/Homo_sapiens/Tools/VEP&quot; rel=&quot;noreferrer&quot;&gt;Variant Effect Predictor (VEP)&lt;/a&gt;. It is a flexible tool that provides several types of annotations on an input .vcf file.  I agree that ExAC is the &lt;em&gt;de facto&lt;/em&gt; gold standard catalog for human genetic variation in coding regions.  To see the frequency distribution of variants by global subpopulation make sure &quot;ExAC allele frequencies&quot; is checked in addition to the 1000 genomes. &lt;a href=&quot;https://i.stack.imgur.com/zDUTT.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zDUTT.jpg&quot; alt=&quot;VEP ExAC&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Output in the web-browser:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/0OCCY.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0OCCY.jpg&quot; alt=&quot;VEP_ExAC_res&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you download the annotated .vcf, frequencies will be in the &lt;code&gt;INFO&lt;/code&gt; field:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##INFO=&amp;lt;ID=CSQ,Number=.,Type=String,Description=&quot;Consequence annotations from Ensembl VEP. Format: Allele|Consequence|IMPACT|SYMBOL|Gene|Feature_type|Feature|BIOTYPE|EXON|INTRON|HGVSc|HGVSp|cDNA_position|CDS_position|Protein_position|Amino_acids|Codons|Existing_variation|DISTANCE|STRAND|FLAGS|SYMBOL_SOURCE|HGNC_ID|TSL|SIFT|PolyPhen|AF|AFR_AF|AMR_AF|EAS_AF|EUR_AF|SAS_AF|AA_AF|EA_AF|ExAC_AF|ExAC_Adj_AF|ExAC_AFR_AF|ExAC_AMR_AF|ExAC_EAS_AF|ExAC_FIN_AF|ExAC_NFE_AF|ExAC_OTH_AF|ExAC_SAS_AF|CLIN_SIG|SOMATIC|PHENO|MOTIF_NAME|MOTIF_POS|HIGH_INF_POS|MOTIF_SCORE_CHANGE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The previously mentioned Annovar can also &lt;a href=&quot;http://doc-openbio.readthedocs.io/projects/annovar/en/latest/user-guide/filter/#exac-annotations&quot; rel=&quot;noreferrer&quot;&gt;annotate with ExAC allele frequencies&lt;/a&gt;.  Finally, should mention the newest whole-genome resource, &lt;a href=&quot;http://gnomad.broadinstitute.org/&quot; rel=&quot;noreferrer&quot;&gt;gnomAD&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="84" LastActivityDate="2017-05-21T14:48:39.303" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="176" PostTypeId="2" ParentId="98" CreationDate="2017-05-21T17:47:47.147" Score="7" Body="&lt;p&gt;I wrote a program, &lt;a href=&quot;http://asciigenome.readthedocs.io/en/latest/description.html&quot; rel=&quot;noreferrer&quot;&gt;ASCIIGenome&lt;/a&gt;, that I find handy in cases where you want to have a quick look at genomic data. It's a genome browser for the command line. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To view only reads containing mismatches you can use the internal function &lt;code&gt;awk&lt;/code&gt;. To filter for reads where the NM tag (number of mismatches) is &gt;0:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ASCIIGenome -fa genome.fa aln.bam&#xA;...&#xA;&#xA;[h] for help: awk 'getSamTag(&quot;NM&quot;) &amp;gt; 0'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The view on terminal screen may look something like this:&lt;a href=&quot;https://i.stack.imgur.com/qXKkp.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qXKkp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, to get only reads containing indels you can use &lt;code&gt;awk '$6 ~ &quot;D|I&quot;'&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps and feel free to report bugs &amp;amp; issues.&lt;/p&gt;&#xA;" OwnerUserId="339" LastActivityDate="2017-05-21T17:47:47.147" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="177" PostTypeId="2" ParentId="174" CreationDate="2017-05-22T03:36:11.637" Score="4" Body="&lt;p&gt;A few possibilities:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Falcon&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try falcon and falcon-unzip. These are designed exactly for your problem and your data: &lt;a href=&quot;https://github.com/PacificBiosciences/FALCON&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/PacificBiosciences/FALCON&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Not Falcon&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you think you have assembled haplotypes (which seems reasonable to expect given enough coverage), you should be able to see the two haplotypes by just doing all pairwise alignments of your contigs. Haplotypes should show up as pairs of contigs that are MUCH more similar (even with a lot of between-haplotype divergence) than other pairs. Once you have all such pairs, you can simply select one of each pair to polish.&lt;/p&gt;&#xA;" OwnerUserId="156" LastActivityDate="2017-05-22T03:36:11.637" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="178" PostTypeId="5" CreationDate="2017-05-22T06:32:44.180" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-22T06:32:44.180" LastActivityDate="2017-05-22T06:32:44.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="179" PostTypeId="4" CreationDate="2017-05-22T06:32:44.180" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-22T06:32:44.180" LastActivityDate="2017-05-22T06:32:44.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="180" PostTypeId="5" CreationDate="2017-05-22T06:33:31.493" Score="0" Body="&lt;p&gt;Bioconductor is &quot;is a free, open source and open development software project for the analysis and comprehension of genomic data generated by wet lab experiments in molecular biology.&quot; (&lt;a href=&quot;https://en.wikipedia.org/wiki/Bioconductor&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Learn more on the project's &lt;a href=&quot;https://www.bioconductor.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;home page&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="559" LastEditorUserId="559" LastEditDate="2017-06-03T18:40:03.477" LastActivityDate="2017-06-03T18:40:03.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="181" PostTypeId="4" CreationDate="2017-05-22T06:33:31.493" Score="0" Body="Bioconductor provides tools for the analysis and comprehension of high-throughput genomic data in the R language." OwnerUserId="131" LastEditorUserId="131" LastEditDate="2017-05-25T14:11:27.163" LastActivityDate="2017-05-25T14:11:27.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="182" PostTypeId="5" CreationDate="2017-05-22T06:38:11.750" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-22T06:38:11.750" LastActivityDate="2017-05-22T06:38:11.750" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="183" PostTypeId="4" CreationDate="2017-05-22T06:38:11.750" Score="0" Body="use for general applications of high-throughput nucleotide sequencing methods that use short-read technology (e.g. Illumina, IonTorrent). For long-read sequencing, use 'long-read', or a more specific tag if applicable (e.g. nanopore, pacbio)." OwnerUserId="57" LastEditorUserId="73" LastEditDate="2018-01-02T21:05:35.280" LastActivityDate="2018-01-02T21:05:35.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="184" PostTypeId="5" CreationDate="2017-05-22T06:42:42.650" Score="0" Body="&lt;p&gt;BED is a file format that has been adapted for a variety of other more generic genome annotation use cases.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://genome.ucsc.edu/FAQ/FAQformat.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC description&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.ensembl.org/info/website/upload/bed.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl description&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="131" LastEditorUserId="37" LastEditDate="2017-06-25T02:12:47.427" LastActivityDate="2017-06-25T02:12:47.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="185" PostTypeId="4" CreationDate="2017-05-22T06:42:42.650" Score="0" Body="Browser Extensible Data, a collection of related plain text formats for describing genome features for visualization with genome browsers." OwnerUserId="131" LastEditorUserId="96" LastEditDate="2017-06-15T13:53:14.837" LastActivityDate="2017-06-15T13:53:14.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="186" PostTypeId="5" CreationDate="2017-05-22T06:43:49.197" Score="0" Body="&lt;p&gt;VCF, acronym for Variant Calling Format, is a text file type used to store information regarding genetic variants. Use this tag for questions related to the use of this file type.&lt;/p&gt;&#xA;" OwnerUserId="1140" LastEditorUserId="1140" LastEditDate="2017-08-02T18:25:20.317" LastActivityDate="2017-08-02T18:25:20.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="187" PostTypeId="4" CreationDate="2017-05-22T06:43:49.197" Score="0" Body="VCF, acronym for Variant Calling Format, is a text file type used to store information regarding genetic variants." OwnerUserId="1140" LastEditorUserId="1140" LastEditDate="2017-08-02T18:25:24.307" LastActivityDate="2017-08-02T18:25:24.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="188" PostTypeId="5" CreationDate="2017-05-22T06:46:31.737" Score="0" Body="&lt;p&gt;The central biochemical basis to life on earth, providing the foundational code required for evolution from the simplist single cellular organism to higher eukaryotes. Provides the molecular basis to the Darwinian theory of evolution. The entire content of DNA in an organism is referred to as the 'genome'.&lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="3848" LastEditDate="2020-07-07T20:24:34.443" LastActivityDate="2020-07-07T20:24:34.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="189" PostTypeId="4" CreationDate="2017-05-22T06:46:31.737" Score="0" Body="Acronym for  deoxyribonucleic acid, the unit of inheritance for all eukaryotic and prokaryotic organisms" OwnerUserId="-1" LastEditorUserId="3848" LastEditDate="2020-07-07T20:24:34.443" LastActivityDate="2020-07-07T20:24:34.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="190" PostTypeId="2" ParentId="174" CreationDate="2017-05-22T08:12:38.290" Score="3" Body="&lt;p&gt;You could also have a go at &lt;a href=&quot;https://github.com/marbl/canu/releases&quot; rel=&quot;nofollow noreferrer&quot;&gt;Canu&lt;/a&gt;. It's designed for long-read assembly (both PacBio and Nanopore), although not specifically for complex population sequencing. It tries to strip a genome down into its unique components, and generates paths from those components that are well-supported from the reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With regards to polishing, it seems to be the case that polishing doesn't converge, and there will be a lot of variants that just oscillate between two possibilities. For me and at least one other person at London Calling this year, there was basically no gain in accuracy for polishing past the third iteration. I used my own error correction algorithm, but they used the more &quot;standard&quot; polishing with Pilon. For what it's worth, the nanopore WGS consortium used Racon for polishing their Canu assemblies.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-05-23T14:18:02.627" LastActivityDate="2017-05-23T14:18:02.627" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="191" PostTypeId="1" CreationDate="2017-05-22T10:47:44.397" Score="9" ViewCount="160" Body="&lt;p&gt;I have a huge amount of ~20x human WGS samples, aligned, and all SNVs that were called with GATK under standard germline parameters set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I need to do is to model SNVs Allele Frequency (AF) for different underlying Copy Numbers. I'd better provide a toy example. For particular genomic region X:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;If X is presented by &lt;strong&gt;2&lt;/strong&gt; copies for the particular samples, we expect AF to be super-close to 1 or to 0.5.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;If X is presented by &lt;strong&gt;4&lt;/strong&gt; copies, I expect any particular AF to be close to 0.25, 0.5, 0.75 or 1.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, I can use &lt;em&gt;Binomial Distribution&lt;/em&gt; for these purposes. However, as we know, the distribution is not exactly Binomial due to alignment/sequencing biases and the median AF for all heterozygous SNVs is more close to 0.48 but not to 0.5 as we would expect. Another thing: for high copy numbers we expect higher coverages. And GATK use several filters so I suppose that we will not see SNVs with AF like 0.125 (in case if the segment has ploidy 8) - despite the super high coverage there GATK may reject this &quot;weird&quot; AF.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read several papers that model SNVs AFs (and I agree that Beta Binomial Distribution may be quite accurate), however, I was not convinced enough that I should use the particular modelling. From your experience (in case if you do SNVs calling), which probabilistic distribution should I use? How should I estimate parameters for each of them (should I expect for CN4 AF=0.5 more frequent than AF=0.75 or vice versa, how to estimate this from data)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPD:&lt;/strong&gt; For simplicity we can say that we have a lot of previously identified regions with ploidy different from CN2, and I can take these coordinates from &lt;a href=&quot;http://www.nature.com/ng/journal/v47/n3/full/ng.3200.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. So I can use more or less &quot;supervised&quot; learning for parameters' estimation.&lt;/p&gt;&#xA;" OwnerUserId="349" LastEditorUserId="73" LastEditDate="2017-06-01T19:58:14.903" LastActivityDate="2017-06-04T01:17:18.017" Title="Expected allele frequency distribution of SNVs in real NGS data" Tags="|snv|modelling|statistics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="192" PostTypeId="2" ParentId="153" CreationDate="2017-05-22T12:05:18.343" Score="10" Body="&lt;p&gt;It seems that the &quot;combining factors&quot; trick described in part 3.3 of DESeq2 current &quot;vignette&quot; (as of may 2017) under the title &quot;Interaction&quot; is a way to access to the desired contrasts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems possible do do it directly when building the &lt;code&gt;colData&lt;/code&gt; and when calling &lt;code&gt;DESeqDataSetFromMatrix&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's add a combined &quot;geno&quot; and &quot;treat&quot; factors to the future &lt;code&gt;colData&lt;/code&gt; parameter:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; col_data$geno_treat &amp;lt;- as.factor(paste(col_data$geno, col_data$treat, sep=&quot;_&quot;))&#xA;&amp;gt; col_data&#xA;DataFrame with 12 rows and 4 columns&#xA;                        geno       treat         rep     geno_treat&#xA;                 &amp;lt;character&amp;gt; &amp;lt;character&amp;gt; &amp;lt;character&amp;gt;       &amp;lt;factor&amp;gt;&#xA;WT_RT_1                   WT          RT           1          WT_RT&#xA;WT_HS30_1                 WT        HS30           1        WT_HS30&#xA;WT_HS30RT120_1            WT   HS30RT120           1   WT_HS30RT120&#xA;prg1_RT_1               prg1          RT           1        prg1_RT&#xA;prg1_HS30_1             prg1        HS30           1      prg1_HS30&#xA;...                      ...         ...         ...            ...&#xA;WT_HS30_2                 WT        HS30           2        WT_HS30&#xA;WT_HS30RT120_2            WT   HS30RT120           2   WT_HS30RT120&#xA;prg1_RT_2               prg1          RT           2        prg1_RT&#xA;prg1_HS30_2             prg1        HS30           2      prg1_HS30&#xA;prg1_HS30RT120_2        prg1   HS30RT120           2 prg1_HS30RT120&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We can now use a design where differential expression will be explained by these combined factors:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dds &amp;lt;- DESeqDataSetFromMatrix(&#xA;    countData = counts_data,&#xA;    colData = col_data,&#xA;    design = ~ geno_treat)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We run the analysis:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dds &amp;lt;- DESeq(dds)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then we can query results for a particular contrast between such factor combinations. For instance, to have the results for the effect of treatment &quot;HS30&quot; against the reference state &quot;RT&quot; in genotype &quot;prg1&quot;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;res &amp;lt;- results(dds, contrast=c(&quot;geno_treat&quot;, &quot;prg1_HS30&quot;, &quot;prg1_RT&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="292" LastActivityDate="2017-05-22T12:05:18.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="193" PostTypeId="1" AcceptedAnswerId="201" CreationDate="2017-05-22T14:51:27.550" Score="19" ViewCount="11485" Body="&lt;p&gt;The results obtained by running the &lt;code&gt;results&lt;/code&gt; command from DESeq2 contain a &quot;baseMean&quot; column, which I assume is the mean across samples of the normalized counts for a given gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How can I access the normalized counts proper?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried the following (continuing with the example used &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/192/292&quot;&gt;here&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dds &amp;lt;- DESeqDataSetFromMatrix(countData = counts_data, colData = col_data, design = ~ geno_treat)&#xA;&amp;gt; dds &amp;lt;- DESeq(dds)&#xA;estimating size factors&#xA;estimating dispersions&#xA;gene-wise dispersion estimates&#xA;mean-dispersion relationship&#xA;final dispersion estimates&#xA;fitting model and testing&#xA;&amp;gt; res &amp;lt;- results(dds, contrast=c(&quot;geno_treat&quot;, &quot;prg1_HS30&quot;, &quot;prg1_RT&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is what I have for the first gene:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; res[&quot;WBGene00000001&quot;,]$baseMean&#xA;[1] 181.7862&#xA;&amp;gt; mean(assays(dds)$mu[&quot;WBGene00000001&quot;,])&#xA;[1] 231.4634&#xA;&amp;gt; mean(assays(dds)$counts[&quot;WBGene00000001&quot;,])&#xA;[1] 232.0833&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;assays(dds)$counts&lt;/code&gt; corresponds to the raw counts. &lt;code&gt;assays(dds)$mu&lt;/code&gt; seems to be a transformation of these counts approximately preserving their mean, but this mean is very different from the &quot;baseMean&quot; value, so these are likely not the normalized values.&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2017-05-24T10:17:43.520" Title="How can I extract normalized read count values from DESeq2 results?" Tags="|r|bioconductor|deseq2|normalization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="194" PostTypeId="1" CreationDate="2017-05-22T15:02:28.473" Score="6" ViewCount="3134" Body="&lt;p&gt;In a weighted gene co-expression network analysis (using &lt;a href=&quot;https://cran.r-project.org/package=WGCNA&quot; rel=&quot;noreferrer&quot;&gt;WGCNA&lt;/a&gt;), the soft-threshold power is recommended as a noise filtering. It consists on raising the correlation to a certain number. To decide this power the scale-free topology is estimated for some powers. The function to estimate this prints:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   Power SFT.R.sq  slope truncated.R.sq mean.k. median.k. max.k.&#xA;1      1   0.9300  3.110          0.996  2960.0    3060.0   3970&#xA;2      2   0.7510  1.010          0.964  1750.0    1780.0   2900&#xA;3      3   0.1730  0.258          0.806  1170.0    1150.0   2280&#xA;4      4   0.0942 -0.183          0.713   833.0     782.0   1870&#xA;5      5   0.3800 -0.463          0.777   623.0     559.0   1580&#xA;6      6   0.5350 -0.656          0.834   481.0     412.0   1360&#xA;7      7   0.6270 -0.797          0.872   381.0     312.0   1190&#xA;8      8   0.6870 -0.910          0.900   307.0     241.0   1050&#xA;9      9   0.7270 -1.000          0.918   252.0     189.0    936&#xA;10    10   0.7490 -1.080          0.928   210.0     150.0    841&#xA;11    12   0.7850 -1.190          0.948   150.0      98.0    693&#xA;12    14   0.8090 -1.280          0.958   111.0      65.9    582&#xA;13    16   0.8290 -1.360          0.968    84.0      45.6    497&#xA;14    18   0.8410 -1.410          0.973    65.2      32.1    429&#xA;15    20   0.8490 -1.450          0.977    51.6      23.0    375&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The recommendations of the &lt;a href=&quot;https://labs.genetics.ucla.edu/horvath/CoexpressionNetwork/Rpackages/WGCNA/faq.html&quot; rel=&quot;noreferrer&quot;&gt;FAQ&lt;/a&gt; indicate that a SFT.R.sq value should be above&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;0.8 for reasonable powers (less than 15 for unsigned or signed hybrid networks, and less than 30 for signed networks) &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and the mean connectivity below the hundreds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Others have used the power just as noise filtering without caring much about the scale-free topology fit. I would pick the first power even if the mean connectivity is in the order of thousands because the scale-free topology fit is pretty high, however the slope is puzzling me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should the soft-threshold power be selected?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Based on an example &lt;a href=&quot;https://area51.stackexchange.com/proposals/109245/bioinformatics/109254#109254&quot;&gt;question&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2022-12-20T18:41:21.107" Title="How to select a power for a scale-free topology network" Tags="|statistics|networks|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="195" PostTypeId="1" AcceptedAnswerId="220" CreationDate="2017-05-22T16:14:17.213" Score="20" ViewCount="469" Body="&lt;p&gt;&lt;strong&gt;Background:&lt;/strong&gt; We're increasingly needing some way of storing lots of variant data associated with lots of subjects: think clinical trials and hospital patients, looking for disease-causing or relevant genes. A thousand subjects is where we'd start, there's talk of millions on the horizon. With various genomic medicine initiatives, this is likely a wider need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The problem:&lt;/strong&gt; While there's plenty of platforms out there, it's a rapidly evolving field. It's difficult to get a feel for how (and if) they perform and how they line up against each other:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What's scalable and can handle a lot of data? What sort of limits?&lt;/li&gt;&#xA;&lt;li&gt;What's robust and not a teetering pile of hacked-together components?&lt;/li&gt;&#xA;&lt;li&gt;What has a large community behind it and is actually used widely?&lt;/li&gt;&#xA;&lt;li&gt;What makes for easy access and search from another service? (Commandline, REST or software APIs)&lt;/li&gt;&#xA;&lt;li&gt;What sort of variants they handle?&lt;/li&gt;&#xA;&lt;li&gt;What sort of parameters can be used in searching? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Solutions I've seen so far:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;BigQ: used with i2b2, but its wider use is unclear&lt;/li&gt;&#xA;&lt;li&gt;OpenCGA: looks the most developed, but I've heard complaints about the size of data it spits out&lt;/li&gt;&#xA;&lt;li&gt;Using BigQuery over a Google Genomics db: doesn't seem to be a general solution&lt;/li&gt;&#xA;&lt;li&gt;Gemini: recommended but is it really scalable and accessible from other services?&lt;/li&gt;&#xA;&lt;li&gt;SciDb: a commercial general db&lt;/li&gt;&#xA;&lt;li&gt;Quince&lt;/li&gt;&#xA;&lt;li&gt;LOVD&lt;/li&gt;&#xA;&lt;li&gt;Adam&lt;/li&gt;&#xA;&lt;li&gt;Whatever platform DIVAS &amp;amp; RVD run on: which may not be freely available&lt;/li&gt;&#xA;&lt;li&gt;Several graphical / graph genome solutions: We (and most other people) are probably not dealing with graph genome data at the moment, but is this a possible solution?&lt;/li&gt;&#xA;&lt;li&gt;Roll your own: Frequently recommended but I'm sceptical this is a plausible solution for a large dataset.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Anyone with experience give a review or high-level guide to this platform space?&lt;/p&gt;&#xA;" OwnerUserId="377" LastEditorUserId="96" LastEditDate="2019-01-15T14:29:49.963" LastActivityDate="2019-01-15T14:29:49.963" Title="The state, limitations and comparisons of large variant stores" Tags="|genomics|variation|human-genome|data-management|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="196" PostTypeId="1" AcceptedAnswerId="217" CreationDate="2017-05-22T16:32:40.663" Score="16" ViewCount="690" Body="&lt;p&gt;I am the resident Bioinfo Geek in a hospital academic lab that routinely employs NGS as well as CyTOF and other large volume data producing technologies. I am sick of our current &quot;protocol&quot; for metadata collection and association with the final products (miriad excel sheets and a couple poorly designed RedCap DBs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to implement a central structured, controlled datastore that will take care of this. I know that the interface to the technicians how will be inputing the data is crucial to its adoption, but this is not the focus of THIS particular question: &lt;strong&gt;Does there exist a schema or schema guidelines for this type of database?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would rather use a model that has been developed by people who know how to do this well. I know of BioSQL but it seems more geared towards full protein/nucleotide records like those found in uniprot or genbank. That is not what we have here. What I want is something similar to the system touched on in this preprint: &lt;a href=&quot;http://biorxiv.org/content/early/2017/05/10/136358&quot; rel=&quot;noreferrer&quot; title=&quot;Managing The Analysis Of High-Throughput Sequencing Data&quot;&gt;http://biorxiv.org/content/early/2017/05/10/136358&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, can anyone provide links to where I might find relevant guidelines or supply personal advice?&lt;/p&gt;&#xA;" OwnerUserId="243" LastEditorUserId="243" LastEditDate="2017-05-22T17:01:44.230" LastActivityDate="2017-06-02T12:27:05.573" Title="Designing a lab NGS file database schema" Tags="|ngs|database|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="197" PostTypeId="2" ParentId="196" CreationDate="2017-05-22T17:01:00.507" Score="9" Body="&lt;p&gt;The &lt;a href=&quot;http://ga4gh.org/&quot; rel=&quot;noreferrer&quot;&gt;Global Alliance for Genomics and Health&lt;/a&gt; has been working on the issue of representing sequencing data and metadata for storage and sharing for quite some time, though with mixed results.  They do offer a model and API for storing NGS data in their &lt;a href=&quot;https://github.com/ga4gh/ga4gh-schemas&quot; rel=&quot;noreferrer&quot;&gt;GitHub repository&lt;/a&gt;, but it can be a bit of a pain to get a high-level view.  I am not sure if any better representation of this exists elsewhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can say from personal experience (having built over a dozen genomic databases), there is no ideal data model and storage best practices.  Genomic data comes in many shapes and sizes, and your needs are going to vary from every other organization, so what works for one bioinformatics group won't necessarily work for you.  The best thing to do is design and implement a model that will cover all of the data types in your workflow and downstream analyses you might do with the data and metadata.&lt;/p&gt;&#xA;" OwnerUserId="47" LastEditorUserId="47" LastEditDate="2017-05-22T18:08:04.757" LastActivityDate="2017-05-22T18:08:04.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="199" PostTypeId="2" ParentId="196" CreationDate="2017-05-22T18:04:31.807" Score="5" Body="&lt;p&gt;I agree that there is no ideal data model that is going to be stable for very long in a quick-moving field like genome informatics. Perhaps a schema-less (NoSQL or some other document-based system, such as &lt;a href=&quot;https://www.mongodb.com/&quot; rel=&quot;noreferrer&quot;&gt;MongoDB&lt;/a&gt;) database approach would work better? This gives you ultimate flexibility to attach whatever information is relevant to database entries you're adding to your database now, without the need to rebuild the database later if you want to attach more/different information to subsequent database entries.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2017-05-22T18:04:31.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="200" PostTypeId="2" ParentId="1" CreationDate="2017-05-22T18:11:10.693" Score="3" Body="&lt;p&gt;In all seriousness, the most efficient way to store DNA sequence data is...you guessed it...in DNA. &lt;a href=&quot;http://dx.doi.org/10.1126/science.1226355&quot; rel=&quot;nofollow noreferrer&quot;&gt;(Church, Gao, and Kasuri, 2012)&lt;/a&gt; and others have used DNA synthesis and sequencing as an information writing/reading mechanism.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Practical? Not yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Storage efficiency? Unparalleled!&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2017-05-22T18:11:10.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="201" PostTypeId="2" ParentId="193" CreationDate="2017-05-22T18:17:38.167" Score="13" Body="&lt;p&gt;The normalized counts themselves can be accessed with &lt;code&gt;counts(dds, normalized=T)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now as to what the baseMean actually means, that will depend upon whether an &quot;expanded model matrix&quot; is in use or not. Given your previous question, we can see that &lt;code&gt;geno_treat&lt;/code&gt; has a bunch of levels, which means that expanded models are not in use. In such cases, the baseMean should be the mean of the base factor in &lt;code&gt;geno_treat&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-05-22T18:17:38.167" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="202" PostTypeId="1" AcceptedAnswerId="273" CreationDate="2017-05-22T18:19:53.483" Score="24" ViewCount="888" Body="&lt;p&gt;Are there any free open source software tools available for simulating Oxford Nanopore reads?&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="73" LastEditDate="2017-06-08T11:52:35.820" LastActivityDate="2019-11-17T00:06:26.630" Title="Tools for simulating Oxford Nanopore reads" Tags="|nanopore|genome-sequencing|software-recommendation|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="203" PostTypeId="5" CreationDate="2017-05-22T18:22:02.093" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-22T18:22:02.093" LastActivityDate="2017-05-22T18:22:02.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="204" PostTypeId="4" CreationDate="2017-05-22T18:22:02.093" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-22T18:22:02.093" LastActivityDate="2017-05-22T18:22:02.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="205" PostTypeId="5" CreationDate="2017-05-22T18:24:39.177" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-22T18:24:39.177" LastActivityDate="2017-05-22T18:24:39.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="206" PostTypeId="4" CreationDate="2017-05-22T18:24:39.177" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-22T18:24:39.177" LastActivityDate="2017-05-22T18:24:39.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="207" PostTypeId="5" CreationDate="2017-05-22T18:29:07.460" Score="0" Body="&lt;p&gt;Many bioinformatics analyses involve decomposing long nucleotide (or, more rarely, peptide) sequences can into their constituent &lt;em&gt;k&lt;/em&gt;-mers, overlapping subsequences of length &lt;em&gt;k&lt;/em&gt;. For sequence &lt;code&gt;D = GATTACA&lt;/code&gt; and &lt;code&gt;k = 4&lt;/code&gt;, the &lt;em&gt;k&lt;/em&gt;-mers in &lt;code&gt;D&lt;/code&gt; are &lt;code&gt;{GATT, ATTA, TTAC, TACA}&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2017-05-25T14:18:50.583" LastActivityDate="2017-05-25T14:18:50.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="208" PostTypeId="4" CreationDate="2017-05-22T18:29:07.460" Score="0" Body="for questions related to k-mer based algorithms, data structures, or concepts" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2018-09-27T10:38:06.703" LastActivityDate="2018-09-27T10:38:06.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="209" PostTypeId="5" CreationDate="2017-05-22T18:34:08.353" Score="0" Body="&lt;p&gt;Hash functions map data of arbitrary size to a value (usually an integer value) of fixed size. In bioinformatics, hash functions are often used to convert DNA sequences (especially &lt;em&gt;k&lt;/em&gt;-mers) into values that can be stored and analyzed efficiently.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2017-05-25T14:11:44.250" LastActivityDate="2017-05-25T14:11:44.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="210" PostTypeId="4" CreationDate="2017-05-22T18:34:08.353" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-22T18:34:08.353" LastActivityDate="2017-05-22T18:34:08.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="211" PostTypeId="5" CreationDate="2017-05-22T18:36:31.463" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-22T18:36:31.463" LastActivityDate="2017-05-22T18:36:31.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="212" PostTypeId="4" CreationDate="2017-05-22T18:36:31.463" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-22T18:36:31.463" LastActivityDate="2017-05-22T18:36:31.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="213" PostTypeId="5" CreationDate="2017-05-22T18:53:31.967" Score="0" Body="&lt;p&gt;Reproducibility refers to the ability to repeat a scientific result that was reported previously. Several terms are frequently used synonymously with reproducibility, such as &lt;em&gt;replicability&lt;/em&gt; or &lt;em&gt;repeatability&lt;/em&gt;. These various terms come with subtle differences in connotation, although there is no universally accepted interpretation of each term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of computation in biology, these terms could refer to any of the following, which constitute a continuum of reproducibility.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Given a scientist's exact code and data, the ability to produce the same exact result as the original scientist.&lt;/li&gt;&#xA;&lt;li&gt;Given a scientist's code and a description of the data, the ability to produce a similar and consistent result with similar data.&lt;/li&gt;&#xA;&lt;li&gt;Given a scientist's data and a description of the code, the ability to produce a similar and consistent result with a similar independent implementation of the code.&lt;/li&gt;&#xA;&lt;li&gt;Given a description of the code and the data, the ability to produce a similar result or reach a similar conclusion based on independent data collection and analysis.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There is some debate as to what type/level of reproducibility can and should be expected of scientists. The strongest scientific claims come from completely reproducing a result from independent data collection and software implementation, but resources and career incentives make it difficult to reproduce studies/results in this way. On the other hand, reproducing a precise result can sometimes be difficult even with the exact same code and data as the original author, due to differences in computing platforms and the lack of computational training for many biologists.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2017-05-25T14:12:34.707" LastActivityDate="2017-05-25T14:12:34.707" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="214" PostTypeId="4" CreationDate="2017-05-22T18:53:31.967" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-22T18:53:31.967" LastActivityDate="2017-05-22T18:53:31.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="215" PostTypeId="2" ParentId="92" CreationDate="2017-05-22T19:07:50.327" Score="8" Body="&lt;p&gt;Visual inspection with histograms, boxplots, or some other distribution visualization is the way to go. Prior to normalization, your abundances may look something like this.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/VFVcC.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VFVcC.jpg&quot; alt=&quot;Pre-norm&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Post-normalization, they should look something like this.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/XrgU2.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XrgU2.jpg&quot; alt=&quot;Post-norm&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://biowize.wordpress.com/2013/12/12/normalization-for-differential-expression-analysis/&quot; rel=&quot;noreferrer&quot;&gt;this blog post&lt;/a&gt; for example code.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2017-05-22T19:07:50.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="216" PostTypeId="2" ParentId="81" CreationDate="2017-05-22T19:13:49.987" Score="4" Body="&lt;p&gt;The &lt;a href=&quot;https://github.com/lh3/wgsim&quot; rel=&quot;nofollow noreferrer&quot;&gt;wgsim package&lt;/a&gt; by Heng Li (of BWA and samtools fame) is my go-to tool for simulating Illumina reads. It doesn't provide any convenient way to simulate differential coverage across different sequences, but it shouldn't be to hard to run wgsim multiple times, generating the desired level of coverage for each sequence of interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would implement a Python script to slurp up your test file, and call wgsim (using the &lt;code&gt;subprocess&lt;/code&gt; module) for each sequence. This will probably require you to have each sequence in a separate file. :-(&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2017-05-22T19:13:49.987" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="217" PostTypeId="2" ParentId="196" CreationDate="2017-05-22T19:31:41.353" Score="5" Body="&lt;p&gt;For &lt;em&gt;metadata&lt;/em&gt;, I would use a SQL schema something like the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CREATE TABLE Project (&#xA;    ac TEXT, -- project/Study accession&#xA;    PRIMARY KEY (ac)&#xA;);&#xA;CREATE TABLE Sample ( -- biological sample/biopsy&#xA;    ac TEXT,&#xA;    PRIMARY KEY (ac)&#xA;);&#xA;CREATE TABLE AnalysisSample (&#xA;    prj_ac TEXT, -- project acccession (Project.ac)&#xA;    symbol TEXT, -- a short name unique in the project&#xA;    sample_ac TEXT, -- sample accession (Sample.ac)&#xA;    PRIMARY KEY (prj_ac, symbol)&#xA;);&#xA;CREATE TABLE Collection ( -- a BAM file&#xA;    ac TEXT, -- collection/alignment file accession&#xA;    prj_ac TEXT, -- project accession (Project.ac)&#xA;    PRIMARY KEY (ac)&#xA;);&#xA;CREATE TABLE ReadGroup (&#xA;    cl_ac TEXT, -- collection accession (Collection.ac)&#xA;    rg_id TEXT, -- @RG-ID&#xA;    sample_sym TEXT, -- @RG-SM; matching AnalysisSample.symbol&#xA;    PRIMARY KEY (cl_ac, rg_id)&#xA;);&#xA;CREATE TABLE VariantSet ( -- a VCF file&#xA;    ac TEXT, -- VCF file accession&#xA;    prj_ac TEXT, -- project accession (Project.ac)&#xA;    PRIMARY KEY (ac)&#xA;);&#xA;CREATE TABLE VariantSample (&#xA;    vs_ac TEXT, -- VCF file accession (VariantSet.ac)&#xA;    sample_sym TEXT, -- sample symbol in the VCF file; matching AnalysisSample.symbol&#xA;    PRIMARY KEY (vs_ac, sample_sym)&#xA;);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the schema, you have &lt;code&gt;Project&lt;/code&gt; and biological &lt;code&gt;Sample&lt;/code&gt; tables, which are independent of each other at the high level. An &lt;code&gt;AnalysisSample&lt;/code&gt; describes a sample used in BAM or VCF and connects &lt;code&gt;Project&lt;/code&gt; and biological &lt;code&gt;Sample&lt;/code&gt;. Importantly, each &lt;code&gt;AnalysisSample&lt;/code&gt; has a symbol unique in a project (see the primary index). This is the symbol on a BAM read group line or on a VCF sample line. A &lt;code&gt;Collection&lt;/code&gt; is in effect a BAM/CRAM file. In theory, a BAM file may contain more than one samples (though rare in practice), which is addressed by a separate &lt;code&gt;ReadGroup&lt;/code&gt; table. Finally, a &lt;code&gt;VariantSet&lt;/code&gt; is a VCF file. &lt;code&gt;VariantSample&lt;/code&gt; tells you which samples are included in each VCF file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the skeleton of a full schema. You can add extra fields to appropriate tables (e.g. file path and hg19/hg38/etc to &lt;code&gt;Collection&lt;/code&gt;, read length to &lt;code&gt;ReadGroup&lt;/code&gt; and family ID to &lt;code&gt;Sample&lt;/code&gt;). You also need indices for efficient table joining and perhaps more tables for complex structures (e.g. pedigree).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the projects I have participated, this schema should work most of time. It is inspired by GA4GH's JSON schema, but my version is in SQL, is simpler and also has a slightly different structure which I think is better.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-05-22T19:31:41.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="218" PostTypeId="2" ParentId="146" CreationDate="2017-05-22T20:33:56.417" Score="7" Body="&lt;p&gt;Several papers have made this distinction, and a few indeed use different terms to distinguish between them. For example, &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0022000016300502&quot; rel=&quot;noreferrer&quot;&gt;Kazaux et al. (2016)&lt;/a&gt; acknowledge that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;These constraints favour the use of a version of the de Bruijn Graph (dBG) dedicated to genome assembly – a version which differs from the combinatorial structure invented by N.G. de Bruijn.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-11-21&quot; rel=&quot;noreferrer&quot;&gt;Kingsford et al. (2010)&lt;/a&gt; also recognise the distinction:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Note that this definition of a de Bruijn graph differs from the traditional definition described in the mathematical literature in the 1940s that requires the graph to contain all length-k strings that can be formed from an alphabet (rather than just those strings present in the genome). &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The oldest reference I found for a specific term to refer to the assembly-related structure is &lt;a href=&quot;http://online.liebertpub.com/doi/abs/10.1089/cmb.1995.2.333&quot; rel=&quot;noreferrer&quot;&gt;Skiena and Sundaram (1995)&lt;/a&gt;, where they call it a &lt;strong&gt;subgraph of the de Bruijn digraph&lt;/strong&gt;. Later, in 2002, &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S0012365X01001339&quot; rel=&quot;noreferrer&quot;&gt;Błażewicz et al.&lt;/a&gt; will refer to it as a &lt;strong&gt;de Bruijn induced subgraph&lt;/strong&gt;.  The term &lt;strong&gt;de Bruijn subgraph&lt;/strong&gt; is also formally defined in &lt;a href=&quot;https://pub.uni-bielefeld.de/publication/2302499&quot; rel=&quot;noreferrer&quot;&gt;Quitzau’s thesis (2009)&lt;/a&gt;. There, and also in the article (&lt;a href=&quot;http://link.springer.com/chapter/10.1007/978-3-540-87361-7_29&quot; rel=&quot;noreferrer&quot;&gt;Quitzau and Stoye, 2008)&lt;/a&gt; the authors describe the &lt;strong&gt;sequence graph&lt;/strong&gt; as a modification of the sparse de Bruijn subgraph (commonly used in assembly problems), where non-branching paths are replaced by a single vertex. The term &lt;strong&gt;sparse de Bruijn graph&lt;/strong&gt; is also used by &lt;a href=&quot;https://arxiv.org/abs/1306.4353&quot; rel=&quot;noreferrer&quot;&gt;Chauve et al. (2013)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another term that I found was &lt;strong&gt;word graph&lt;/strong&gt;, described by both &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/21/8/1371/249821/A-graph-based-algorithm-for-generating-EST&quot; rel=&quot;noreferrer&quot;&gt;Malde et al. (2005)&lt;/a&gt; and by &lt;a href=&quot;http://link.springer.com/chapter/10.1007/978-3-540-72031-7_29&quot; rel=&quot;noreferrer&quot;&gt;Heath and Pati (2007)&lt;/a&gt; as a &lt;em&gt;subgraph&lt;/em&gt; or as a &lt;em&gt;generalization&lt;/em&gt; of a de Bruijn graph. &lt;a href=&quot;http://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-14-313&quot; rel=&quot;noreferrer&quot;&gt;Rødland (2013)&lt;/a&gt; summarises some of the terms used for this data structure:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The data structure is best understood in terms of the de Bruijn subgraph representation of S[k]. (...) Some authors may refer to this as a word graph, or even just a de Bruijn graph.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Although we can recognise that the distinction is not very relevant, the question is asking specifically for the situation where one wants to make such a distinction. &lt;/p&gt;&#xA;" OwnerUserId="45" LastActivityDate="2017-05-22T20:33:56.417" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="219" PostTypeId="2" ParentId="202" CreationDate="2017-05-22T20:43:59.303" Score="9" Body="&lt;p&gt;By chance, just today I've heard of a nanopore read simulator, &lt;a href=&quot;http://biorxiv.org/content/early/2016/03/18/044545.1&quot; rel=&quot;noreferrer&quot;&gt;NanoSim&lt;/a&gt;. It is  &lt;a href=&quot;https://github.com/bcgsc/NanoSim&quot; rel=&quot;noreferrer&quot;&gt;released under a GPL license&lt;/a&gt;. I have never used it, though...&lt;/p&gt;&#xA;" OwnerUserId="45" LastActivityDate="2017-05-22T20:43:59.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="220" PostTypeId="2" ParentId="195" CreationDate="2017-05-22T22:13:52.937" Score="14" Body="&lt;p&gt;An epic question. Unfortunately, the short answer is: no, there are no widely used solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For several thousand samples, BCF2, the binary representation of VCF, should work well. I don't see the need of new tools at this scale. For a larger sample size, ExAC people are using spark-based hail. It keeps all per-sample annotations (like GL, GQ and DP) in addition to genotypes. Hail is at least something heavily used in practice, although mostly by a few groups so far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simpler problem is to store genotypes only. This is sufficient to the majority of end users. There are better approaches to store and query genotypes. GQT, developed by the Gemini team, enables fast query of samples. It allows you to quickly pull samples under certain genotype configurations. As I remember, GQT is orders of magnitude faster than google genomics API to do PCA. Another tool is BGT. It produces a much smaller file and provides fast and convenient queries over sites. Its paper talks about ~32k whole-genome samples. I am in the camp who believe specialized binary formats like GQT and BGT are faster than solutions built on top of generic databases. I would encourage you to have a look if you only want to query genotypes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intel's GenomicDB approaches the problem in a different angle. It does not actually keep a &quot;squared&quot; multi-sample VCF internally. It instead keeps per-sample genotypes/annotations and generates merged VCF on the fly (this is my understanding, which could be wrong). I don't have first-hand experience with GenomicDB, but I think something in this line should be the ultimate solution in the era of 1M samples. I know GATK4 is using it at some step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to others in your list, Gemini might not scale that well, I guess. It is partly the reason why they work on GQT. Last time I checked, BigQuery did not query individual genotypes. It only queries over site statistics. Google genomics APIs access individual genotypes, but I doubt it can be performant. Adam is worth trying. I have not tried, though.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-05-22T22:13:52.937" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="221" PostTypeId="2" ParentId="202" CreationDate="2017-05-22T23:05:06.307" Score="7" Body="&lt;p&gt;In addition to the already mentioned &lt;a href=&quot;https://github.com/bcgsc/NanoSim&quot; rel=&quot;noreferrer&quot;&gt;NanoSim&lt;/a&gt;, there is also &lt;a href=&quot;https://github.com/ethanagbaker/SiLiCO&quot; rel=&quot;noreferrer&quot;&gt;SiLiCO&lt;/a&gt; and &lt;a href=&quot;https://sourceforge.net/p/readsim/wiki/Home/&quot; rel=&quot;noreferrer&quot;&gt;ReadSim&lt;/a&gt; (although it hasn't been updated in over 2 years, so I am not sure how relevant it is at this point considering how fast the technology is progressing).&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2017-05-22T23:05:06.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="223" PostTypeId="1" AcceptedAnswerId="224" CreationDate="2017-05-23T18:43:01.447" Score="-5" ViewCount="115" Body="&lt;p&gt;As a newcomer, what are some influential research papers published in the past few years that I should read to catch up in the field?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: @Devon Ryan answered it perfectly. I tried a google search but it turned up a bunch of opinionated articles that varied. The biostar discussions are perfect and was the link I was searching for. I hope someone else new to the field finds this useful!&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="1157" LastEditDate="2018-06-14T02:07:26.363" LastActivityDate="2018-06-14T02:07:26.363" Title="Influential papers in bioinformatics" Tags="|literature-search|" AnswerCount="1" CommentCount="6" ClosedDate="2017-05-23T22:06:43.123" ContentLicense="CC BY-SA 3.0" />
  <row Id="224" PostTypeId="2" ParentId="223" CreationDate="2017-05-23T19:15:08.387" Score="4" Body="&lt;p&gt;Over on biostars there's a thread like this every year or so. I'll link to &lt;a href=&quot;https://www.biostars.org/p/229807/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the 2016 edition&lt;/a&gt; and the (much shorter) &lt;a href=&quot;https://www.biostars.org/p/171860/&quot; rel=&quot;nofollow noreferrer&quot;&gt;2015 edition&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My personal picks from those would be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.nature.com/nature/journal/v536/n7616/full/nature19057.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ExAC&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://biorxiv.org/content/early/2015/06/27/021592&quot; rel=&quot;nofollow noreferrer&quot;&gt;salmon&lt;/a&gt;, which is now published&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://arxiv.org/abs/1505.02710&quot; rel=&quot;nofollow noreferrer&quot;&gt;kallisto&lt;/a&gt;, which is also now published&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-05-23T19:15:08.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="225" PostTypeId="1" AcceptedAnswerId="227" CreationDate="2017-05-24T03:26:50.163" Score="33" ViewCount="24937" Body="&lt;p&gt;I am using a reference genome for mm10 mouse downloaded from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genome?term=mus%20musculus&quot; rel=&quot;noreferrer&quot;&gt;NCBI&lt;/a&gt;, and would like to understand in greater detail the difference between lowercase and uppercase letters, which make up roughly equal parts of the genome. I understand that N is used for 'hard masking' (areas in the genome that could not be assembled) and lowercase letters for 'soft masking' in repeat regions.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What does this soft masking actually mean? &lt;/li&gt;&#xA;&lt;li&gt;How confident can I be about the sequence in these regions?&lt;/li&gt;&#xA;&lt;li&gt;What does a lowercase n represent? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="163" LastEditorUserId="163" LastEditDate="2017-05-24T11:56:09.067" LastActivityDate="2017-05-24T14:24:15.477" Title="Uppercase vs lowercase letters in reference genome" Tags="|fasta|genome|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="226" PostTypeId="2" ParentId="202" CreationDate="2017-05-24T03:36:03.760" Score="3" Body="&lt;p&gt;The best nanopore read simulators would be associated with the best base-callers. For a base-caller to effectively model the DNA strand, it needs to take into account the expected underlying electrical model together with the associated signal noise (both in the time dimension as well as the amplitude dimension). In theory, it should be possible to reverse the algorithm and generate an electrical signal given an underlying sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, I'm not aware of any tools that attempt to simulate nanopore reads at the electrical level. Any &quot;nanopore read simulator&quot; that concentrates only on base sequence would need to encompass all the possible base-calling software models that exist, which is an impossible task (particularly given how quickly ONT updates their own base callers).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-05-24T03:36:03.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="227" PostTypeId="2" ParentId="225" CreationDate="2017-05-24T06:01:32.763" Score="29" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What does this soft masking actually mean?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A lot of the sequence in genomes are repetitive. Human genome, for example, has (at least) two-third repetitive elements.[1]. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These repetitive elements are soft-masked by converting the upper case letters to lower case. An important use-case of these soft-masked bases will be in homology searches: An &lt;code&gt;atatatatatat&lt;/code&gt; will tend to appear both in human and mouse genomes but is likely non-homologous.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How confident can I be about the sequence in these regions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As you can be about in non soft-masked based positions. Soft-masking is done after determining portions in the genome that are likely repetitive. There is no uncertainty whether a particular base is 'A' or 'G', just that it is part of a repeat and hence should be represented as an 'a'.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What does a lowercase n represent?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;UCSC uses &lt;a href=&quot;https://tandem.bu.edu/trf/trfdesc.html&quot; rel=&quot;noreferrer&quot;&gt;Tandom Repeat Finder&lt;/a&gt; and &lt;a href=&quot;http://www.repeatmasker.org/&quot; rel=&quot;noreferrer&quot;&gt;RepeatMasker&lt;/a&gt; for soft-masking potential repeats. NCBI most likely uses &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3045581/&quot; rel=&quot;noreferrer&quot;&gt;TANTAN&lt;/a&gt;. 'N's represents no sequence information is available for that base. It being replaced by 'n' is likely an artifact of the repeat-masking software where it soft-masks an 'N' by an 'n' to indicate that portion of the genome is likely a repeat too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &lt;a href=&quot;http://journals.plos.org/plosgenetics/article?id=10.1371/journal.pgen.1002384&quot; rel=&quot;noreferrer&quot;&gt;http://journals.plos.org/plosgenetics/article?id=10.1371/journal.pgen.1002384&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="161" LastActivityDate="2017-05-24T06:01:32.763" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="228" PostTypeId="1" CreationDate="2017-05-24T06:41:10.320" Score="7" ViewCount="647" Body="&lt;p&gt;&lt;em&gt;This question has also been posted on &lt;a href=&quot;https://www.biostars.org/p/254308/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have sequenced numerous multiplexed pools of BS amplicon-seq libraries derived from human samples on a MiSeq over the past few weeks. I have been utilising trim-galore and Bismark for alignment and am finding the mapping efficiency to be really low for two pools (55% &amp;amp; 30% respectively) both of which had a cytosine per base sequence of around 10-20% throughout the entire read in their fastqc files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The high C content visible in the fastqc files makes me think the poor mapping efficiency is due to poor bisulfite conversion, as this would be expected to be close to zero is bisulfite conversion had actually took place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am new to the field so any help would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="88" LastEditorUserId="73" LastEditDate="2018-01-16T20:58:01.850" LastActivityDate="2018-01-16T20:58:01.850" Title="The effects of incomplete bisulfite conversion upon mapping efficiency" Tags="|sequence-alignment|fastq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="229" PostTypeId="2" ParentId="228" CreationDate="2017-05-24T06:46:36.547" Score="7" Body="&lt;p&gt;Bisulfite conversion efficiency has no effect on the mapping rate in bismark and similar tools. The reason is that the reads are fully bisulfite converted &lt;em&gt;in silico&lt;/em&gt; before alignment to minimize mapping bias.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest that you play around with the settings handed to bowtie2, such as using local alignment and modifying the &lt;code&gt;--score-min&lt;/code&gt; option to allow more mismatches. Alternatively, you might try a different aligner like &lt;a href=&quot;https://github.com/brentp/bwa-meth&quot; rel=&quot;noreferrer&quot;&gt;bwameth&lt;/a&gt;, which will always do local alignment.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-05-24T06:46:36.547" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="230" PostTypeId="1" AcceptedAnswerId="240" CreationDate="2017-05-24T07:35:31.097" Score="16" ViewCount="7305" Body="&lt;p&gt;As enrichment analysis a usual step is to infer the pathways enriched in a list of genes. However I can't find a discussion about which database is better. Two of the most popular (in my particular environment) are Reactome and KEGG (Maybe because there are tools using them in Bioconductor).&#xA;KEGG requires a subscription for ftp access, and for my research I would need to download huge amounts of KGML files I am now leaning towards Reactome&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is the one with more genes associated to pathways ? &#xA;Which is more completely annotated ?&#xA;Is there any paper comparing them ? &lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-05-24T10:16:12.237" LastActivityDate="2020-04-25T22:38:26.933" Title="What are the advantages and disadvantages between using KEGG or Reactome?" Tags="|database|" AnswerCount="3" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="231" PostTypeId="2" ParentId="225" CreationDate="2017-05-24T08:16:19.220" Score="13" Body="&lt;p&gt;The use of lower/upper case letters and &lt;code&gt;N&lt;/code&gt;/&lt;code&gt;n&lt;/code&gt; letters in genomes sequences is not completely standardised and you should always check the specification of the resource you are using.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lower case letters are most commonly used to represent “soft-masked sequences”, a convention popularised by &lt;a href=&quot;http://www.repeatmasker.org/&quot; rel=&quot;noreferrer&quot;&gt;RepeatMasker&lt;/a&gt;, where interspersed repeats (which covers transposons, retrotransposons and processed pseudogenes) and low complexity sequences are marked with lower case letters. Note that larger repeats, such as sizable tandem repeats, segmental duplications, and whole gene duplications are not generally masked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are other uses for lower/upper case letters, for example, &lt;a href=&quot;http://www.ensembl.org/Help/View?id=155&quot; rel=&quot;noreferrer&quot;&gt;Ensembl have used&lt;/a&gt; upper/lower case letters to represent exonic and intronic sequences respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;N&lt;/code&gt; and &lt;code&gt;n&lt;/code&gt; nucleotides may represent “hard masked sequences”, where interspersed repeats and low complexity sequences are replaced by &lt;code&gt;N&lt;/code&gt;s. But &lt;code&gt;N&lt;/code&gt;/&lt;code&gt;n&lt;/code&gt;s may alternatively represent ambiguous nucleotides, indeed this is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Nucleic_acid_notation#IUPAC_notation&quot; rel=&quot;noreferrer&quot;&gt;IUPAC&lt;/a&gt; specification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note occasionally (although fortunately rarely) &lt;code&gt;X&lt;/code&gt;/&lt;code&gt;x&lt;/code&gt; is used to represent ambiguous nucleotides or “hard-masked sequences” too.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2017-05-24T08:16:19.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="232" PostTypeId="1" CreationDate="2017-05-24T09:32:26.523" Score="11" ViewCount="1020" Body="&lt;p&gt;I’d normally use &lt;code&gt;collapseReplicates&lt;/code&gt; (or do the collapsing upstream) to handle technical replicates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in my current RNA-seq experimental design, samples were sequenced twice using different library preparation protocols, leading to marked differences in the resulting count estimates (in fact, the choice of different protocol explains most of the variance in the bulk data according to PCA). I would therefore like to model these differences by adding them as a covariate to the DESeq2 model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I &lt;em&gt;thought&lt;/em&gt; I could just add another factor to the design formula, equivalent to &lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#multi-factor-designs&quot; rel=&quot;noreferrer&quot;&gt;the “pasilla” example in the DESeq2 vignette&lt;/a&gt;, which adds the factor &lt;code&gt;type&lt;/code&gt; for the library type (single-end vs paired-end). However, the &lt;a href=&quot;http://bioconductor.org/packages/release/data/experiment/vignettes/pasilla/inst/doc/create_objects.html&quot; rel=&quot;noreferrer&quot;&gt;pasilla vignette&lt;/a&gt; states that these different libraries are actually independent &lt;em&gt;biological&lt;/em&gt; replicates, not technical replicates as I had previously assumed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’m now concerned that having technical replicates in the design might artificially inflate the power. Can the design be salvaged?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It’s worth noting that we &lt;em&gt;also&lt;/em&gt; have biological replicates, and a full rank design; i.e. every combination of treatment and library prep, with biological replicates.&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2017-05-24T10:02:03.190" LastActivityDate="2017-05-25T10:04:02.870" Title="Can I model technical replicates in DESeq2?" Tags="|rna-seq|deseq2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="233" PostTypeId="2" ParentId="193" CreationDate="2017-05-24T10:17:43.520" Score="6" Body="&lt;p&gt;It depends what you mean by “normalised”. As Devon said, the &lt;code&gt;normalized = TRUE&lt;/code&gt; argument to the &lt;code&gt;count&lt;/code&gt; function gives you normalised counts. However, these are “only” library-size normalised (i.e. divided by the &lt;code&gt;sizeFactors(dds)&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, &lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#data-transformations-and-visualization&quot; rel=&quot;noreferrer&quot;&gt;as the vignette explains&lt;/a&gt;, downstream processing generally requires more advanced normalisation, to account for the &lt;a href=&quot;https://en.wikipedia.org/wiki/Heteroscedasticity&quot; rel=&quot;noreferrer&quot;&gt;heteroscedasticity&lt;/a&gt; of the counts. This is often done by simply &lt;code&gt;log&lt;/code&gt;ging the counts but this has obvious drawbacks (most trivially, what do we do with 0 counts? A workaround is to add a pseudocount but that’s problematic too).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DESeq2 offers two different methods to perform a more rigorous analysis:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;rlog&lt;/code&gt; — a &lt;em&gt;regularised log&lt;/em&gt;, and&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;vst&lt;/code&gt; — a &lt;em&gt;variance stabilising transformation&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You’d generally use either of these for downstream analysis, not &lt;code&gt;count(dds, normalized = TRUE)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2017-05-24T10:17:43.520" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="234" PostTypeId="2" ParentId="232" CreationDate="2017-05-24T10:20:25.597" Score="5" Body="&lt;p&gt;Practically speaking, there's no way to include the technical replicates in that design (in DESeq2 at least). Your concern regarding inflating the power is exactly correct and the only way to combat that would be to add a pairing factor like one might do with case-control or tumor-normal studies. That is, something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  group libraryPrep sample&#xA;1    WT           A      1&#xA;2    WT           B      1&#xA;3   MUT           A      2&#xA;4   MUT           B      2&#xA;5    WT           A      3&#xA;6    WT           B      3&#xA;7   MUT           A      4&#xA;8   MUT           B      4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here &lt;code&gt;sample&lt;/code&gt; pairs the technical replicates together. However this ends up either being rank deficient or in the end not more informative.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-05-24T10:20:25.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="235" PostTypeId="2" ParentId="232" CreationDate="2017-05-24T11:09:19.483" Score="6" Body="&lt;p&gt;If they're truly technical replicates, then &lt;strike&gt;there's no way to model them using DESeq2&lt;/strike&gt;*, as you've alluded to with the &lt;code&gt;collapseReplicates&lt;/code&gt; function. DESeq2/ Mike Love's general recommendation with &lt;code&gt;collapseReplicates&lt;/code&gt; is to just add the reads together for technical replicates. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to model them instead of collapsing them down, you can &lt;code&gt;voom&lt;/code&gt; transform your data and follow an example similar to section 11.3 in the &lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/limma/inst/doc/usersguide.pdf&quot; rel=&quot;noreferrer&quot;&gt;Limma users guide&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;(Code example from 11.3):&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Design&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;| FileName  | Cy3   | Cy5  |&#xA;| --------- |:-----:| ----:|&#xA;| File1     | wt1   | mu1  |&#xA;| File2     | wt1   | mu1  |&#xA;| File3     | wt2   | mu2  |&#xA;| File4     | wt2   | mu2  |&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Example&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; biolrep &amp;lt;- c(1, 1, 2, 2)&#xA;&amp;gt; corfit &amp;lt;- duplicateCorrelation(MA, ndups = 1, block = biolrep)&#xA;&amp;gt; fit &amp;lt;- lmFit(MA, block = biolrep, cor = corfit$consensus)&#xA;&amp;gt; fit &amp;lt;- eBayes(fit)&#xA;&amp;gt; topTable(fit, adjust = &quot;BH&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;*Edit: As @Devon Ryan mentions, you can design a paired model in &lt;code&gt;DESeq2&lt;/code&gt;, but beware of making this full rank. &lt;/p&gt;&#xA;" OwnerUserId="222" LastEditorUserId="222" LastEditDate="2017-05-25T10:04:02.870" LastActivityDate="2017-05-25T10:04:02.870" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="236" PostTypeId="2" ParentId="225" CreationDate="2017-05-24T12:29:31.210" Score="4" Body="&lt;ol&gt;&#xA;&lt;li&gt;Lower case nucleotides commonly denotes a &lt;strong&gt;soft&lt;/strong&gt; masked sequences. How exactly the genome was masked you can find in FAQ of &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genome/doc/ftpfaq/#masking&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI&lt;/a&gt;:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Are repetitive sequences in eukaryotic genomes masked?&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Repetitive sequences in eukaryotic genome assembly sequence files, as&#xA;  identified by &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/16287941&quot; rel=&quot;nofollow noreferrer&quot;&gt;WindowMasker&lt;/a&gt;, have been masked to lower-case.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The location and identity of repeats found by &lt;a href=&quot;http://www.repeatmasker.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RepeatMasker&lt;/a&gt; are also&#xA;  provided in a separate file. These spans could be used to mask the&#xA;  genomic sequences if desired. Be aware, however, that many less&#xA;  studied organisms do not have good repeat libraries available for&#xA;  RepeatMasker to use.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;IMHO, low complexity regions are always more likely to be missassembled than high complexity sequences. However, this will be problem for non-model organisms. I would guess that the reliability of the softmasked regions of mouse genome will very high.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;No idea, looks like an artefact.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;An example of usage of the soft mask&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mapping of sequence to reference usually starts with perfect matches of seeds (substrings) of the mapped reads and the reference sequence. Soft masked (low complexity) regions are not used for matches of seeds, but they are used only for the extension of the alignment if there was a seed in a neighbouring region. This application of softmasking applied to problem of long read assembly is described on this &lt;a href=&quot;https://dazzlerblog.wordpress.com/2016/04/01/detecting-and-soft-masking-repeats/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2017-05-24T14:24:15.477" LastActivityDate="2017-05-24T14:24:15.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="237" PostTypeId="1" AcceptedAnswerId="239" CreationDate="2017-05-24T15:37:17.157" Score="7" ViewCount="485" Body="&lt;p&gt;I produced a bam file by aligning reads to a small set of synthetic sequences using bwa-mem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am heavily filtering reads that are not paired and of a certain orientation.&#xA;Applying the filtering, I get a few thousands of reads:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h $myfilebam | \&#xA;samtools view -h -F4 - | \&#xA;samtools view -h -F8 - | \&#xA;samtools view -h -F256 - | \&#xA;samtools view -h -F512 - | \&#xA;samtools view -h -F1024 - | \&#xA;samtools view -h -F2048 - | \&#xA;samtools view -h -f16 - | \&#xA;samtools view -h -f32 -  | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Gives me &lt;code&gt;89502&lt;/code&gt; reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I then pipe this into &lt;code&gt;samtools mpileup&lt;/code&gt;, I get no results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h $myfilebam | \&#xA;samtools view -h -F4 - | \&#xA;samtools view -h -F8 - | \&#xA;samtools view -h -F256 - | \&#xA;samtools view -h -F512 - | \&#xA;samtools view -h -F1024 - | \&#xA;samtools view -h -F2048 - | \&#xA;samtools view -h -f16 - | \&#xA;samtools view -h -f32 -  | \&#xA;samtools mpileup --excl-flags 0 -Q0 -B -d 999999 - | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Returns 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried different combinations of filtering, and when I do both &lt;code&gt;-f 16&lt;/code&gt; and &lt;code&gt;-f 32&lt;/code&gt; returns empty, but if I do either of those, then it works:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h $myfilebam | \&#xA;samtools view -h -F4 - | \&#xA;samtools view -h -F8 - | \&#xA;samtools view -h -F256 - | \&#xA;samtools view -h -F512 - | \&#xA;samtools view -h -F1024 - | \&#xA;samtools view -h -F2048 - | \&#xA;samtools view -h -f16 - | \&#xA;samtools mpileup --excl-flags 0 -Q0 -B -d 999999 - | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Returns &lt;code&gt;1056&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas why? My thinking was that it would work with &lt;code&gt;--excl-flags 0&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: substituting &lt;code&gt;mpileup&lt;/code&gt; for &lt;code&gt;depth&lt;/code&gt; does work, and prints out each position and the depth as expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT2: adding &lt;code&gt;-q 0&lt;/code&gt; to &lt;code&gt;mpileup&lt;/code&gt; gives the same empty result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="73" LastEditDate="2018-01-15T01:39:24.533" LastActivityDate="2024-02-29T20:34:37.437" Title="samtools mpileup empty when filtering out flags" Tags="|sam|samtools|mpileup|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="238" PostTypeId="1" AcceptedAnswerId="241" CreationDate="2017-05-24T15:41:28.400" Score="11" ViewCount="311" Body="&lt;p&gt;I have a few sets of marker genes that I can classify RNA-seq samples using semi-supervised clustering.  I would like to automate the process, however, I am struggling to find the ideal algorithm that could generate some kind of score for marker gene set from a given sample.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I presume that this is a standard analyses in many groups but I am not sure which method(s) are yielding good results in practice.&lt;/p&gt;&#xA;" OwnerUserId="64" LastActivityDate="2017-05-24T23:32:41.013" Title="Classifying samples based on marker gene expression" Tags="|rna-seq|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="239" PostTypeId="2" ParentId="237" CreationDate="2017-05-24T22:35:24.987" Score="2" Body="&lt;p&gt;By using &lt;code&gt;-h&lt;/code&gt; in the &lt;code&gt;samtools view&lt;/code&gt; command, you're including all the header lines in your word count. If you happen to have about 89500 reference sequences, then the lengths of those would all appear in the header and inflate the &lt;code&gt;-h&lt;/code&gt; word count, but not the mpileup count. Try piping it through an additional &lt;code&gt;samtools view&lt;/code&gt; (i.e. without &lt;code&gt;-h&lt;/code&gt;) and see if the counts change:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;samtools view -h -f32 -  | \&#xA;samtools view | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Also, &lt;code&gt;samtools mpileup&lt;/code&gt; by default only considers high-quality bases and concordant reads. Try adding a &lt;code&gt;-A&lt;/code&gt; to your mpileup line (which stops anomalous read pairs from being discarded):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;samtools mpileup -A -Q0 -B -d 999999 - | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Whether or not this is actually a good idea will be dependent on what you want to get out of the analysis, and what the downstream programs / analyses are expecting.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2024-02-29T20:34:37.437" LastActivityDate="2024-02-29T20:34:37.437" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="240" PostTypeId="2" ParentId="230" CreationDate="2017-05-24T22:59:54.537" Score="4" Body="&lt;p&gt;One big downside of KEGG is the licensing issue. One big advantage of Reacome are various crosslinks to other databases and data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;ad 1,&lt;/strong&gt; This depends on which pathway, they are both primary databases. Sometimes other databases that for instance combine data of primary databases have better annotation of pathways (there is an example in the review paper bellow)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;ad 3,&lt;/strong&gt; There is very extensive relatively new (2015) review on this topic focused on human pathways: &lt;a href=&quot;https://academic.oup.com/database/article/2433126/Comparison-of-human-cell-signaling-pathway&quot; rel=&quot;nofollow noreferrer&quot;&gt;Comparison of human cell signaling pathway databases—evolution, drawbacks and challenges&lt;/a&gt;. However I could not find there which one is more complete ...&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2017-05-24T22:59:54.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="241" PostTypeId="2" ParentId="238" CreationDate="2017-05-24T23:32:41.013" Score="7" Body="&lt;p&gt;I would consider using gene expression signatures to classify samples (especially cancer subtypes but the same principles apply to other problems of this type) one of the classic problems of bioinformatics. Quite a lot of work has been done on methods to derive gene sets that provide good classification performance. This is slightly different from your problem since you already have a gene signature but it may still prove useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These methods will typically fit a model that selects a (small) number of genes from genome-wide expression data that distinguish between the cell types/conditions in question, i.e. they derive a gene signature. The resulting model then allows for the classification of new samples. I've had success using &lt;a href=&quot;http://bioinformatics.csiro.au/generave/&quot; rel=&quot;noreferrer&quot;&gt;GeneRave&lt;/a&gt; for this purpose (but note that this was designed for microarray data, I haven't used it with RNA-seq data and don't know how well it holds up there). A more recent paper relating to this issue can be found &lt;a href=&quot;https://www.nature.com/articles/srep32976.&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how does that help you? One option would be to fit one of these classifiers to gene expression data for the genes you already know to obtain a model that can then be applied to new samples automatically.&lt;/p&gt;&#xA;" OwnerUserId="154" LastActivityDate="2017-05-24T23:32:41.013" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="242" PostTypeId="1" AcceptedAnswerId="276" CreationDate="2017-05-25T11:46:36.237" Score="18" ViewCount="642" Body="&lt;p&gt;I'm currently trying to assembly a genome from a rodent parasite, &lt;em&gt;Nippostrongylus brasiliensis&lt;/em&gt;. This genome does have an existing reference genome, but it is highly fragmented. Here are some continuity statistics for the scaffolds of the current Nippo reference genome (assembled from Illumina reads):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Total sequences: 29375&#xA;Total length: 294.400206 Mb&#xA;Longest sequence: 394.171 kb&#xA;Shortest sequence: 500 b&#xA;Mean Length: 10.022 kb&#xA;Median Length: 2.682 kb&#xA;N50: 2024 sequences; L50: 33.527 kb&#xA;N90: 11638 sequences; L90: 4.263 kb&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This genome is most likely difficult to assemble because of the highly repetitive nature of the genomic sequences. These repetitive sequences come in (at least) three classes:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Tandem repeats with a &lt;em&gt;repeat unit&lt;/em&gt; length greater than the read length of Illumina sequencers (e.g. 171bp)&lt;/li&gt;&#xA;&lt;li&gt;Tandem repeats with a &lt;em&gt;cumulative&lt;/em&gt; length greater than the fragment length of Illumina sequencers, or the template length for linked reads (e.g. 20kb)&lt;/li&gt;&#xA;&lt;li&gt;Complex (i.e. non-repetitive) sequence that appears at multiple places throughout the genome&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/marbl/canu&quot; rel=&quot;nofollow noreferrer&quot;&gt;Canu&lt;/a&gt; seems to deal quite well with the first two types of repeats, despite the abundance of repetitive structure in the genome. Here's the unitigging summary produced by Canu on one of the assemblies I've attempted. Notice that about 30% of the reads either span or contain a long repeat:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;category            reads     %          read length        feature size or coverage  analysis&#xA;----------------  -------  -------  ----------------------  ------------------------  --------------------&#xA;middle-missing        694    0.07     7470.92 +- 5552.00        953.06 +- 1339.13    (bad trimming)&#xA;middle-hump           549    0.05     3770.05 +- 3346.10         74.23 +- 209.86     (bad trimming)&#xA;no-5-prime           3422    0.33     6711.32 +- 5411.26         70.92 +- 272.99     (bad trimming)&#xA;no-3-prime           3161    0.30     6701.35 +- 5739.86         87.41 +- 329.42     (bad trimming)&#xA;&#xA;low-coverage        27158    2.59     3222.51 +- 1936.79          4.99 +- 1.79       (easy to assemble, potential for lower quality consensus)&#xA;unique             636875   60.76     6240.20 +- 3908.44         25.22 +- 8.49       (easy to assemble, perfect, yay)&#xA;repeat-cont         48398    4.62     4099.55 +- 3002.72        335.54 +- 451.43     (potential for consensus errors, no impact on assembly)&#xA;repeat-dove           135    0.01    16996.33 +- 6860.08        397.37 +- 319.52     (hard to assemble, likely won't assemble correctly or even at all)&#xA;&#xA;span-repeat        137927   13.16     9329.94 +- 6906.27       2630.06 +- 3539.53    (read spans a large repeat, usually easy to assemble)&#xA;uniq-repeat-cont   155725   14.86     6529.83 +- 3463.16                             (should be uniquely placed, low potential for consensus errors, no impact on assembly)&#xA;uniq-repeat-dove    28248    2.70    12499.99 +- 8446.95                             (will end contigs, potential to misassemble)&#xA;uniq-anchor          5721    0.55     8379.86 +- 4575.71       3166.22 +- 3858.35    (repeat read, with unique section, probable bad read)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, the third type of repeat is giving me a bit of grief. Using the above assembly, here are the continuity parameters from the assembled contigs:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Total sequences: 3505&#xA;Total length: 322.867456 Mb&#xA;Longest sequence: 1.762243 Mb&#xA;Shortest sequence: 2.606 kb&#xA;Mean Length: 92.116 kb&#xA;Median Length: 42.667 kb&#xA;N50: 417 sequences; L50: 194.126 kb&#xA;N90: 1996 sequences; L90: 35.634 kb&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It's not a &lt;em&gt;bad&lt;/em&gt; assembly, particularly given the complexity of the genome, but I feel like it could be improved by tackling the complex genomic repeats in some fashion. About 60Mb of the contigs in this assembly are linked with each other in a huge web (based on the GFA output from Canu):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Tw8SQ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Tw8SQ.jpg&quot; alt=&quot;60Mb linked structure from Canu GFA&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The repetitive regions are typically over 500bp in length, average about 3kb, and I've seen at least one case which seems to be a 20kb sequence duplicated in multiple regions.&lt;/p&gt;&#xA;&lt;p&gt;The Canu defaults seem to give the best assembly results for the few parameters that I've tried, with one exception: trimming. I've tried playing around a little bit with the trimming parameters, and curiously a trimming coverage of 5X (with overlap of 500bp) seems to give a more contiguous assembly than with a trimming coverage of 2X (with the same overlap).&lt;/p&gt;&#xA;&lt;p&gt;If anyone is interested in having a look at these data themselves, called FASTQ files from Nippo sequencing runs can be found &lt;a href=&quot;https://www.ncbi.nlm.nih.gov//sra/?term=SRP092357&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Raw nanopore signal files are available within ENA project &lt;a href=&quot;https://www.ebi.ac.uk/ena/browser/view/PRJEB20824?show=reads&quot; rel=&quot;nofollow noreferrer&quot;&gt;PRJEB20824&lt;/a&gt;. There's also a Zenodo archive &lt;a href=&quot;https://doi.org/10.5281/zenodo.569899&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; that contains the GFA and assembly contigs.&lt;/p&gt;&#xA;&lt;p&gt;I can use Illumina data to correct the Canu assembly, but that doesn't help with resolving the &amp;quot;type 3&amp;quot; repeats. The regions are sufficiently similar that illumina reads get mapped to multiple points in the genome. The Illumina contigs are high quality (i.e. they have good BUSCO scores, indicating few variant errors), but quite short. Any sniff of a repeat and the contig ends. I've got more than a few examples of regions that would make an Illumina read (even 10x linked reads) cower in fear.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have any other suggestions on how I could resolve these complex repeats?&lt;/p&gt;&#xA;&lt;p&gt;Computational solutions would be preferred, but resequencing is not out of the question.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-06-29T05:21:46.870" LastActivityDate="2022-06-29T05:23:03.237" Title="How can I improve a long-read assembly with a repetitive genome?" Tags="|assembly|nanopore|long-reads|canu|repeat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="243" PostTypeId="5" CreationDate="2017-05-25T17:51:23.720" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-25T17:51:23.720" LastActivityDate="2017-05-25T17:51:23.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="244" PostTypeId="4" CreationDate="2017-05-25T17:51:23.720" Score="0" Body="When dealing with associating sequencing reads to a matching position in a set of reference sequences (typically a genome)." OwnerUserId="292" LastEditorUserId="292" LastEditDate="2017-05-31T13:48:58.563" LastActivityDate="2017-05-31T13:48:58.563" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="245" PostTypeId="5" CreationDate="2017-05-25T17:59:21.297" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-25T17:59:21.297" LastActivityDate="2017-05-25T17:59:21.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="246" PostTypeId="4" CreationDate="2017-05-25T17:59:21.297" Score="0" Body="The tricky art of scaling quantitative data across libraries, typically to account for differences in sequencing depth. This can also be about scaling for read source length, like transcript or gene length, in order to enable comparisons across genes." OwnerUserId="292" LastEditorUserId="292" LastEditDate="2017-05-31T13:46:57.663" LastActivityDate="2017-05-31T13:46:57.663" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="247" PostTypeId="5" CreationDate="2017-05-25T18:05:03.600" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-25T18:05:03.600" LastActivityDate="2017-05-25T18:05:03.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="248" PostTypeId="4" CreationDate="2017-05-25T18:05:03.600" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-05-25T18:05:03.600" LastActivityDate="2017-05-25T18:05:03.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="249" PostTypeId="1" AcceptedAnswerId="250" CreationDate="2017-05-25T18:21:49.743" Score="7" ViewCount="727" Body="&lt;p&gt;&lt;a href=&quot;https://genome.ucsc.edu/ENCODE/dataMatrix/encodeDataMatrixHuman.html&quot; rel=&quot;noreferrer&quot;&gt;The ENCODE Experiment Matrix at UCSC&lt;/a&gt; lists the different available cell types under the categories &quot;Tier 1&quot;, &quot;Tier 2&quot; and &quot;Tier 3&quot;. What is the difference between these classifications?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What, for example, makes GM12878 a Tier 1 cell type and A549 a Tier 2 cell type?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ejhqx.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ejhqx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="61" LastEditDate="2017-05-26T09:05:34.530" LastActivityDate="2017-05-26T20:27:47.253" Title="What is the difference between ENCODE Tier 1, 2 and 3 cell types?" Tags="|encode|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="250" PostTypeId="2" ParentId="249" CreationDate="2017-05-25T18:44:04.180" Score="6" Body="&lt;p&gt;The tiers denote the original priority of sequencing and processing the samples. This is understandable given the number of marks and experiment types that ENCODE tried to sequence. Further details are available on &lt;a href=&quot;http://genome.ucsc.edu/ENCODE/cellTypes.html&quot; rel=&quot;noreferrer&quot;&gt;ENCODE's website&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: Just to expand a bit, tier one cells were supposed to get processed first. Then tier 2 came second. Of course tier 2 had a LOT more cells, so it got split into 2 and 2.5, since there were already apparently tier 3 cells.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-05-25T18:44:04.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="251" PostTypeId="1" AcceptedAnswerId="263" CreationDate="2017-05-25T22:47:46.183" Score="6" ViewCount="62" Body="&lt;p&gt;Does &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3218867/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GISTIC 2.0&lt;/a&gt; estimate the background model:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;G = -log(Probability | Background)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;by permuting within the sample or across all samples in the set?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper describes the probabilistic scoring method based on permutations, but I could not understand if this permutation is performed within the sample only. The &lt;a href=&quot;ftp://ftp.broadinstitute.org/pub/GISTIC2.0/GISTICDocumentation_standalone.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation page&lt;/a&gt; seems to suggest across samples, but this would mean that different sized sets might lead to different outcomes on the sample sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, does it matter if the set is constituted by, for example, 10 samples of the same tissue of origin or ~1000 from multiple tissues?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;" OwnerUserId="193" LastEditorUserId="215" LastEditDate="2017-05-29T06:36:18.757" LastActivityDate="2017-05-29T21:34:18.543" Title="Does GISTIC (v 2.0) estimate amplified/deleted probabilities on a single sample basis?" Tags="|cnv|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="252" PostTypeId="2" ParentId="230" CreationDate="2017-05-26T09:41:03.727" Score="4" Body="&lt;p&gt;For you the main point would be whether an enrichment analysis is going to give you an &lt;em&gt;informative&lt;/em&gt; answer. That's what will make a particular database better. And there's all sorts of subjective decisions made in their construction as to what what a pathway is, what to include, where to draw boundaries, etc. so different databases will give different answers and it will not be clear which is more correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So browse over the references / pointers people have given, select a service and use it and no others. Don't jump around databases until you get an answer you like, that's just fishing. &lt;/p&gt;&#xA;" OwnerUserId="377" LastActivityDate="2017-05-26T09:41:03.727" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="253" PostTypeId="1" AcceptedAnswerId="257" CreationDate="2017-05-26T13:23:00.853" Score="7" ViewCount="316" Body="&lt;p&gt;I am running samtools mpileup (v1.4) on a bam file with very choppy coverage (ChIP-seq style data). I want to get a first-pass list of positions with SNVs and their frequency as reported by the read counts, but no matter what I do, I keep getting all SNVs filtered out as not passing QC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the magic parameter set for an initial list of SNVs and frequencies?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: this is a question I posted on &quot;the other&quot; website, but didn't get a reply there.&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2017-05-29T14:47:26.867" Title="variant calling on ChIP-seq style data: samtools mpileup with minimal filters" Tags="|variant-calling|samtools|chip-seq|mpileup|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="254" PostTypeId="1" AcceptedAnswerId="3055" CreationDate="2017-05-26T14:06:59.617" Score="8" ViewCount="709" Body="&lt;p&gt;I have perform an enrichment analysis to a cluster of genes. The output is a list of pathways and their p-value (the pathways are selected because p-value &amp;lt; 0.05). The list is still quite long, so I want to reduce it. For that purpose I have a calculated the Dice coefficient of the pathways in a matrix $p$x$p$ where $p$ is the number of pathways in the list. I want both the ones that are more different (they overlap less, their Dice coefficient is lower) and the pathways more representative of the most similar pathways (So if a there is a group of 5 pathways that overlap over 0.8 take just one).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I select the most representatives pathways? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a similar &lt;a href=&quot;http://revigo.irb.hr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tool&lt;/a&gt; for GO but it relays on discarding not significant GO, while here all the initial pathways are already significant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I do a clustering of the pathways using the Dice coefficient matrix I don't know where (or how) to cut.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mMdEH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mMdEH.png&quot; alt=&quot;circular dendrogara&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried using the height to select the pathways. But I am unsure of the interpretation of height. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some other tools I have seen use a multidimensional scaling plot, but I am not sure if performing it and cutting at certain point of the first dimension would help.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/CeetJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CeetJ.png&quot; alt=&quot;MDS plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-12-14T07:53:35.587" LastActivityDate="2017-12-14T07:53:35.587" Title="How to select the most representative pathways from a gene enrichment analysis?" Tags="|pathway|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="255" PostTypeId="2" ParentId="249" CreationDate="2017-05-26T20:27:47.253" Score="1" Body="&lt;p&gt;From &lt;a href=&quot;https://www.genome.gov/26524238/encode-project-common-cell-types/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Encode's website&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;... To aid in the integration and&#xA;  comparison of data produced using different technologies and&#xA;  platforms, the ENCODE Consortium has designated cell types that will&#xA;  be used by all investigators. These common cell types include both&#xA;  cell lines and primary cell types, and plans are being made to explore&#xA;  the use of primary tissues and embryonic stem (ES) cells.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Cell types were selected largely for practical reasons, including&#xA;  their wide availability, the ability to grow them easily, and their&#xA;  capacity to produce sufficient numbers of cells for use in all&#xA;  technologies being used by ENCODE investigators. Secondary&#xA;  considerations were the diversity in tissue source of the cells, germ&#xA;  layer lineage representation, the availability of existing data&#xA;  generated using the cell type, and coordination with other ongoing&#xA;  projects. Effort was also made to select at least some cell types that&#xA;  have a relatively normal karyotype...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="99" LastActivityDate="2017-05-26T20:27:47.253" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="257" PostTypeId="2" ParentId="253" CreationDate="2017-05-27T01:16:15.507" Score="7" Body="&lt;p&gt;I used this in the past for ChIP-seq data and it generated SNVs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools mpileup \&#xA;--uncompressed --max-depth 10000 --min-MQ 20 --ignore-RG --skip-indels \&#xA;--fasta-ref ref.fa file.bam \&#xA;| bcftools call --consensus-caller \&#xA;&amp;gt; out.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This was samtools 1.3 in case that makes a difference.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2017-05-27T01:16:15.507" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="259" PostTypeId="2" ParentId="254" CreationDate="2017-05-27T10:43:34.673" Score="2" Body="&lt;p&gt;If you're happy with a more confident ranking of the most representative gene sets, rather than necessarily cutting down the list, you might try EGSEA. It uses an ensemble approach to give a ranking of the most relevant gene sets, and also produces an interactive HTML output with statistics, heatmaps, pathway maps, summary plots and GO graphs which allows you to examine the output at varying levels of granularity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can read the paper on &lt;a href=&quot;http://biorxiv.org/content/early/2016/07/08/042580&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioRxiv&lt;/a&gt; or download the package from &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/EGSEA.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioconductor&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="163" LastEditorUserId="163" LastEditDate="2017-05-27T23:58:34.373" LastActivityDate="2017-05-27T23:58:34.373" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="261" PostTypeId="2" ParentId="107" CreationDate="2017-05-27T16:05:14.763" Score="2" Body="&lt;p&gt;Perhaps &lt;a href=&quot;http://genomecrispr.dkfz.de/#!/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GenomeCRISPR&lt;/a&gt;, I've personally never used it, but it has a well-documented API and appears you could automate it to go through the whole genome.&lt;/p&gt;&#xA;" OwnerUserId="33" LastActivityDate="2017-05-27T16:05:14.763" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="262" PostTypeId="2" ParentId="254" CreationDate="2017-05-28T10:23:33.793" Score="4" Body="&lt;p&gt;This sounds like something that might be amenable to a clustered heatmap plot, or a correlation matrix plot, or something similar. Have you looked at a correlation matrix of the dice coefficient matrix (or maybe just a heatmap plot of that matrix without the correlation matrix)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;corrplot&lt;/code&gt; package looks like it might be useful, in particular &lt;a href=&quot;https://cran.r-project.org/web/packages/corrplot/vignettes/corrplot-intro.html&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;corrplot vignette&quot;&gt;the &lt;code&gt;hclust&lt;/code&gt; / drawing rectangles presentation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't vouch for this package though; it's just something I found by a search for &quot;R plot correlation matrix&quot;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-12-12T12:56:22.340" LastActivityDate="2017-12-12T12:56:22.340" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="263" PostTypeId="2" ParentId="251" CreationDate="2017-05-28T11:40:16.593" Score="3" Body="&lt;p&gt;There doesn't seem to be much differences between GISTIC 1.0 and 2.0 as it says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;As with GISTIC 1.0, we obtain P-values for each marker by comparing the score at each locus to a background score distribution generated by random permutation of the marker locations in each sample&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But on the &lt;a href=&quot;http://www.pnas.org.sare.upf.edu/content/suppl/2007/11/20/0710052104.DC1#ST&quot; rel=&quot;nofollow noreferrer&quot;&gt;supplementary material&lt;/a&gt; of the GISTIC 1.0 there is a more detailed explanation of the method. See the &quot;Stage 2&quot; section:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Second, we compare these G-scores to the distribution of scores expected if only random aberrations were observed. This distribution can be determined by rescoring the genome after permuting marker locations within each sample; we instead derive a semiexact estimate.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Furthermore in another section (&quot;Stage 2: Aggregation of Data from Different Tumors to Differentiate Between Driver and Passenger Aberrations&quot;) it says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To determine which of the aberrations identified in Stage 1 are likely to represent driver events, we aggregate the data from all tumors used in the analysis to generate summary scores for amplifications, deletions, and LOH. The statistical significance of each score is determined by comparison to the distribution of scores obtained by all permutations of the data (using a semiexact approximation), with correction for multiple hypothesis testing. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;However, the relevant section (of the supplementary materials) seems to be &quot;Null Hypothesis Generation: An Analytic Derivation of the Null Distribution&quot;, where it describes the semiexact approximation used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a &lt;a href=&quot;http://www.pnas.org.sare.upf.edu/content/suppl/2007/11/20/0710052104.DC1/10052Fig9.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;supplementary file&lt;/a&gt; it describes it as &quot;Generate all permutations of SNP labels within each sample to simulate datasets with random aberrations&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that the background probabilities are calculated within sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't say if it matters how the sets are constituted but I would say that the broader the sets are, the better estimation of the structural variations it will perform but the same estimation of the background will be done. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately you can check the code of the program, or test with 10 samples and replace one of them to see if the results are changing accordingly. &lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-05-29T21:34:18.543" LastActivityDate="2017-05-29T21:34:18.543" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="264" PostTypeId="1" AcceptedAnswerId="270" CreationDate="2017-05-28T21:40:13.940" Score="17" ViewCount="10365" Body="&lt;p&gt;I got a bunch of vcf files (v4.1) with structural variations of bunch of non-model organisms (i.e. there are no known variants). I found there are quite a some tools to manipulate vcf files like &lt;a href=&quot;https://vcftools.github.io/&quot; rel=&quot;noreferrer&quot;&gt;VCFtools&lt;/a&gt;, R package &lt;a href=&quot;https://cran.r-project.org/web/packages/vcfR/index.html&quot; rel=&quot;noreferrer&quot;&gt;vcfR&lt;/a&gt; or python library &lt;a href=&quot;https://pypi.python.org/pypi/PyVCF&quot; rel=&quot;noreferrer&quot;&gt;PyVCF&lt;/a&gt;. However none of them seems to provide a quick summary, something like (preferably categorised by size as well):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;type    count&#xA;DEL     x&#xA;INS     y&#xA;INV     z&#xA;....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there any tool or a function I overlooked that produces summaries of this style?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that vcf file is just a plain text file and if I will dissect &lt;code&gt;REF&lt;/code&gt; and &lt;code&gt;ALT&lt;/code&gt; columns I should be able to write a script that will do the job, but I hoped that I could avoid to write my own parser.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;--- edit ---&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far it seems that only tool that aims to do summaries (@gringer answer) is not working on vcf v4.1. Other tools would provide just partial solution by filtering certain variant type. Therefore I accept my own parser perl/R solutions, till there will be a &lt;strong&gt;working&lt;/strong&gt; tool for &lt;strong&gt;stats&lt;/strong&gt; of vcf with &lt;strong&gt;structural variants&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2017-06-03T11:03:39.833" LastActivityDate="2022-11-04T03:23:43.010" Title="Is there an easy way to create a summary of a VCF file (v4.1) with structural variations?" Tags="|vcf|structural-variation|" AnswerCount="5" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="266" PostTypeId="1" CreationDate="2017-05-29T00:01:03.100" Score="6" ViewCount="338" Body="&lt;p&gt;I'm just getting started with the Mauve aligner and I'm finding the documentation a bit lacking. I'm using the &lt;code&gt;progressiveMauve&lt;/code&gt; tool from the command line and would like to output an identity matrix file. The set of output files it generates by default for &lt;code&gt;progressiveMauve --output=outfile&lt;/code&gt; are &lt;code&gt;outfile&lt;/code&gt;, &lt;code&gt;outfile.backbone&lt;/code&gt; and &lt;code&gt;output.bbcols&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://darlinglab.org/mauve/user-guide/progressivemauve.html&quot; rel=&quot;noreferrer&quot;&gt;documentation&lt;/a&gt; for the tool lists an &lt;code&gt;--input-id-matrix&lt;/code&gt; option but this doesn't seem to have any effect. The &lt;a href=&quot;http://darlinglab.org/mauve/user-guide/files.html&quot; rel=&quot;noreferrer&quot;&gt;documentation page for output files&lt;/a&gt; does describe an identity matrix file but doesn't say anything about how to generate it.&lt;/p&gt;&#xA;" OwnerUserId="126" LastActivityDate="2017-08-31T09:18:38.437" Title="How can I output an identity matrix in progressiveMauve?" Tags="|sequence-alignment|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="268" PostTypeId="2" ParentId="264" CreationDate="2017-05-29T05:16:36.277" Score="9" Body="&lt;p&gt;According to the &lt;code&gt;bcftools&lt;/code&gt; man page, it is able to produce statistics using the command &lt;code&gt;bcftools stats&lt;/code&gt;. Running this myself, the statistics look like what you're asking for:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# This file was produced by bcftools stats (1.2-187-g1a55e45+htslib-1.2.1-256-ga356746) and can be plotted using plot-vcfstats.&#xA;# The command line was: bcftools stats  OVLNormalised_STARout_KCCG_called.vcf.gz&#xA;#&#xA;# Definition of sets:&#xA;# ID    [2]id   [3]tab-separated file names&#xA;ID  0   OVLNormalised_STARout_KCCG_called.vcf.gz&#xA;# SN, Summary numbers:&#xA;# SN    [2]id   [3]key  [4]value&#xA;SN  0   number of samples:  108&#xA;SN  0   number of records:  333&#xA;SN  0   number of no-ALTs:  0&#xA;SN  0   number of SNPs: 313&#xA;SN  0   number of MNPs: 0&#xA;SN  0   number of indels:   20&#xA;SN  0   number of others:   0&#xA;SN  0   number of multiallelic sites:   0&#xA;SN  0   number of multiallelic SNP sites:   0&#xA;# TSTV, transitions/transversions:&#xA;# TSTV  [2]id   [3]ts   [4]tv   [5]ts/tv    [6]ts (1st ALT) [7]tv (1st ALT) [8]ts/tv (1st ALT)&#xA;TSTV    0   302 11  27.45   302 11  27.45&#xA;# SiS, Singleton stats:&#xA;...&#xA;# IDD, InDel distribution:&#xA;# IDD   [2]id   [3]length (deletions negative)  [4]count&#xA;IDD 0   -9  1&#xA;IDD 0   -2  4&#xA;IDD 0   -1  6&#xA;IDD 0   1   4&#xA;IDD 0   2   1&#xA;IDD 0   3   3&#xA;IDD 0   4   1&#xA;# ST, Substitution types:&#xA;# ST    [2]id   [3]type [4]count&#xA;ST  0   A&amp;gt;C 2&#xA;ST  0   A&amp;gt;G 78&#xA;ST  0   A&amp;gt;T 2&#xA;ST  0   C&amp;gt;A 5&#xA;ST  0   C&amp;gt;G 0&#xA;ST  0   C&amp;gt;T 66&#xA;ST  0   G&amp;gt;A 67&#xA;ST  0   G&amp;gt;C 0&#xA;ST  0   G&amp;gt;T 1&#xA;ST  0   T&amp;gt;A 1&#xA;ST  0   T&amp;gt;C 91&#xA;ST  0   T&amp;gt;G 0&#xA;# DP, Depth distribution&#xA;# DP    [2]id   [3]bin  [4]number of genotypes  [5]fraction of genotypes (%)    [6]number of sites  [7]fraction of sites (%)&#xA;DP  0   &amp;gt;500    0   0.000000    333 100.000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is annotating what is explicitly in the VCF file, and that's SNVs (and INDELs). If you want a structural variant analysis (i.e. on a larger scale than single nucleotides), then you'll need to use something that does more than a summary of the VCF file.&lt;/p&gt;&#xA;&lt;p&gt;While Inversions, large-scale deletions, and breakpoints are part of the 4.1 specifications, they are unfortunately &lt;a href=&quot;https://github.com/samtools/bcftools/issues/623&quot; rel=&quot;nofollow noreferrer&quot;&gt;not currently supported by bcftools&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-11-04T03:21:17.493" LastActivityDate="2022-11-04T03:21:17.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="269" PostTypeId="2" ParentId="264" CreationDate="2017-05-29T05:23:32.083" Score="2" Body="&lt;p&gt;You might want to try &lt;a href=&quot;https://github.com/pjotrp/bioruby-vcf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bio-VCF&lt;/a&gt;. From the authors description &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Bio-vcf is a new generation VCF parser, filter and converter. Bio-vcf&#xA;  is not only very fast for genome-wide (WGS) data, it also comes with a&#xA;  really nice filtering, evaluation and rewrite language and it can&#xA;  output any type of textual data, including VCF header and contents in&#xA;  RDF and JSON.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In addition it is a very fast parser. The rewrite DSL might help you customise your filtering and needs. &lt;/p&gt;&#xA;" OwnerUserId="27" LastActivityDate="2017-05-29T05:23:32.083" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="270" PostTypeId="2" ParentId="264" CreationDate="2017-05-29T06:22:59.417" Score="6" Body="&lt;p&gt;The &amp;quot;my own parser&amp;quot; solutions. The information I was searching for in part of column &lt;code&gt;INFO&lt;/code&gt;, namely in variables &lt;code&gt;SVLEN&lt;/code&gt; and &lt;code&gt;SVTYPE&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;very fast SV types + counts (by @user172818 in comment) :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zcat var.vcf.gz | perl -ne 'print &amp;quot;$1\n&amp;quot; if /[;\t]SVTYPE=([^;\t]+)/' | sort | uniq -c&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;quite slow SV types + counts + sizes :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SV_colnames &amp;lt;- c('CHROM', 'POS', 'ID', 'REF', 'ALT', 'QUAL', 'FILTER', 'INFO', 'FORMAT', 'SAMPLE1')&#xA;&#xA;ssplit &amp;lt;- function(s, split = '='){&#xA;    unlist(strsplit(s, split = split))&#xA;}&#xA;&#xA;# note, capital letters just respect original naming conventions of the VCF file&#xA;getSVTYPE &amp;lt;- function(info){&#xA;    ssplit(grep(&amp;quot;SVTYPE&amp;quot;, info, value = T))[2]&#xA;}&#xA;&#xA;getSVLEN &amp;lt;- function(info){&#xA;    SVLEN &amp;lt;- ssplit(grep(&amp;quot;SVLEN&amp;quot;, info, value = T))&#xA;    ifelse(length(SVLEN) == 0, NA, as.numeric(SVLEN[2]))&#xA;}&#xA;&#xA;load_sv &amp;lt;- function(file){&#xA;    vcf_sv_table &amp;lt;- read.table(file, stringsAsFactors = F)&#xA;    colnames(vcf_sv_table) &amp;lt;- SV_colnames&#xA;    # possible filtering&#xA;    # vcf_sv_table &amp;lt;- vcf_sv_table[vcf_sv_table&lt;span class=&quot;math-container&quot;&gt;$FILTER == 'PASS',]&#xA;    vcf_sv_table_info &amp;lt;- strsplit(vcf_sv_table$&lt;/span&gt;INFO, ';')&#xA;    vcf_sv_table&lt;span class=&quot;math-container&quot;&gt;$SVTYPE &amp;lt;- unlist(lapply(vcf_sv_table_info, getSVTYPE))&#xA;    vcf_sv_table$&lt;/span&gt;SVLEN &amp;lt;- unlist(lapply(vcf_sv_table_info, getSVLEN))&#xA;    return(vcf_sv_table)&#xA;}&#xA;&#xA;sv_df &amp;lt;- load_sv('my_sv_calls.vcf')&#xA;table(sv_df$SVTYPE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="57" LastEditorUserId="73" LastEditDate="2022-11-04T03:23:43.010" LastActivityDate="2022-11-04T03:23:43.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="271" PostTypeId="1" AcceptedAnswerId="466" CreationDate="2017-05-29T06:25:30.990" Score="13" ViewCount="1828" Body="&lt;p&gt;The IGSR has a &lt;a href=&quot;http://www.internationalgenome.org/wiki/Analysis/Variant%20Call%20Format/VCF%20%28Variant%20Call%20Format%29%20version%204.0/encoding-structural-variants&quot; rel=&quot;noreferrer&quot;&gt;sample&lt;/a&gt; for encoding structural variants in the VCF 4.0 format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example from the site (the first record):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS   ID  REF ALT   QUAL  FILTER  INFO  FORMAT  NA00001&#xA;1 2827693   . CCGTGGATGCGGGGACCCGCATCCCCTCTCCCTTCACAGCTGAGTGACCCACATCCCCTCTCCCCTCGCA  C . PASS  SVTYPE=DEL;END=2827680;BKPTID=Pindel_LCS_D1099159;HOMLEN=1;HOMSEQ=C;SVLEN=-66 GT:GQ 1/1:13.9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How to read it? From what I can see:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This is a deletion (&lt;code&gt;SVTYPE=DEL&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;The end position of the variant comes before the starting position (reverse strand?)&lt;/li&gt;&#xA;&lt;li&gt;The reference starts from &lt;code&gt;2827693&lt;/code&gt; to &lt;code&gt;2827680&lt;/code&gt; (13 bases on the reverse strand)&lt;/li&gt;&#xA;&lt;li&gt;The difference between reference and alternative is &lt;code&gt;66&lt;/code&gt; bases (&lt;code&gt;SVLEN=-66&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This doesn't sound right to me. For instance, I don't see where exactly the deletion starts. The &lt;code&gt;SVLEN&lt;/code&gt; field says &lt;code&gt;66&lt;/code&gt; bases deleted, but where? &lt;code&gt;2827693&lt;/code&gt; to &lt;code&gt;2827680&lt;/code&gt; only has 13 bases between.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q:&lt;/strong&gt; How to read the deletion correctly from this structural VCF record? Where is the missing 66-13=53 bases?&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="96" LastEditDate="2019-01-15T14:29:28.820" LastActivityDate="2019-01-15T14:29:28.820" Title="How to read structural variant VCF?" Tags="|vcf|variation|structural-variation|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="272" PostTypeId="2" ParentId="253" CreationDate="2017-05-29T14:47:26.867" Score="5" Body="&lt;p&gt;Another approach is &lt;a href=&quot;https://github.com/lh3/htsbox&quot; rel=&quot;noreferrer&quot;&gt;htsbox&lt;/a&gt;. You can get a candidate list with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;htsbox pileup -Cvcf ref.fa -q20 -Q20 -s5 file.bam &amp;gt; out.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here, &lt;code&gt;-q&lt;/code&gt; sets min mapping quality, &lt;code&gt;-Q&lt;/code&gt; sets min base quality, &lt;code&gt;-v&lt;/code&gt; outputs variants only &lt;code&gt;-c&lt;/code&gt; outputs VCF, &lt;code&gt;-C&lt;/code&gt; gives you base counts on both strands and finally &lt;code&gt;-s5&lt;/code&gt; requires at least 5 high-quality bases to call out an allele. It is useful when your data are failing the assumptions made by typical variant callers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why not samtools+bcftools or varscan? Transparency and speed. This command line simply counts based on the parameters you use. It does not apply additional operations. And because of this, it is over an order of magnitude faster than samtools mpileup or varscan. It is worth noting that samtools uses BAQ by default, which reduces FPs occasionally. However, BAQ is not quite necessary for longer Illumina reads and it hurts sensitivity at the same time.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-05-29T14:47:26.867" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="273" PostTypeId="2" ParentId="202" CreationDate="2017-05-29T19:09:21.650" Score="11" Body="&lt;p&gt;&lt;strong&gt;Simulators designed specifically for Oxford Nanopore:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/bcgsc/NanoSim&quot; rel=&quot;nofollow noreferrer&quot;&gt;NanoSim&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/karel-brinda/nanosim-h&quot; rel=&quot;nofollow noreferrer&quot;&gt;NanoSim-H&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/ethanagbaker/SiLiCO&quot; rel=&quot;nofollow noreferrer&quot;&gt;SiLiCO&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://sourceforge.net/p/readsim/wiki/Home/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ReadSim&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/lykaust15/DeepSimulator&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepSimulator&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;General long read simulators:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/gt1/loresim&quot; rel=&quot;nofollow noreferrer&quot;&gt;Loresim&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/gt1/loresim2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Loresim 2&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://sourceforge.net/projects/fastqsim/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FASTQsim&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/bioinform/longislnd&quot; rel=&quot;nofollow noreferrer&quot;&gt;LongISLND&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For an exhaustive list of existing read simulators, see page 15 of my thesis, &lt;a href=&quot;https://doi.org/10.5281/zenodo.1045317&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Novel computational techniques for mapping and&#xA;classifying Next-Generation Sequencing data&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="425" LastEditorUserId="3144" LastEditDate="2019-11-17T00:06:26.630" LastActivityDate="2019-11-17T00:06:26.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="274" PostTypeId="2" ParentId="81" CreationDate="2017-05-29T19:46:25.187" Score="-1" Body="&lt;p&gt;You can split your FASTA file sequence-by-sequence using&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;split -a 6 -p '^&amp;gt;' your_file.fa seq_&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then use any existing read simulator supporting coverage (ART, DWGsim, etc.). If you want to have all the reads mixed (not ordered by the original sequence), you can use RNFtools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 1:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As @terdon pointed out, the previous command works on OS X only. An analogical one liner for Linux (but with a slightly different naming scheme using numbers rather than letters) can be&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;csplit -f seq_ -n 6 your_file.fa '/^&amp;gt;/' {*}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To make this command work also on OS X, one needs to install coreutils (e.g., using brew) and then use &lt;code&gt;gcsplit&lt;/code&gt; instead of &lt;code&gt;csplit&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 2:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once FASTA is split by sequences, the simulation becomes straightforward and many different approaches can be used. My favorite one is using GNU Parallel. Imagine that you have your coverages in a text file called &lt;code&gt;covs.txt&lt;/code&gt; on separate lines and in the same order as the sequences in &lt;code&gt;your_file.fa&lt;/code&gt;, e.g.,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;40&#xA;30&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you can simulate reads from the original sequences using DWGsim by&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ls -1 seq_* | paste covs.txt - \&#xA;    | parallel -v --colsep '\t' dwgsim -1 100 -2 100 -C {1} {2} sim_{2}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and merge the obtained FASTQ files using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat sim_seq_*.bwa.read1.fastq &amp;gt; reads.1.fq&#xA;cat sim_seq_*.bwa.read2.fastq &amp;gt; reads.2.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One possible danger of this approach is that we have assumed that the number of seq_* files is the same as the number of lines in &lt;code&gt;covs.txt&lt;/code&gt;, which might not be true (by mistake). We should check this prior to the simulation step, e.g., by:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[[ &quot;$(ls -1 seq_* | wc -l)&quot; == &quot;$(cat covs.txt | wc -l)&quot; ]] \&#xA;    || echo &quot;Incorrent number of lines in covs.txt&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Another caveat is that the simulated reads are not in a random order (they are grouped by source sequences).&lt;/p&gt;&#xA;" OwnerUserId="425" LastEditorUserId="425" LastEditDate="2017-05-30T18:09:35.290" LastActivityDate="2017-05-30T18:09:35.290" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="275" PostTypeId="2" ParentId="271" CreationDate="2017-05-29T22:34:37.427" Score="4" Body="&lt;p&gt;So, first off, as others have pointed out, I'm pretty sure that example is just wrong. At least, the numbers don't match as you've pointed out. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, it is impossible to be sure without showing us the header of the VCF file as well. The INFO field (the 5th field of a VCF file) is very, very variable and depends entirely on the header lines. Each program (or human) implementing a VCF is free to choose to have whatever they feel like in the INFO field. However, each &lt;code&gt;IDENTIFIER=&lt;/code&gt; needs to have an associated INFO line at the beginning of the file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the &lt;code&gt;SVTYPE&lt;/code&gt;, &lt;code&gt;SVLEN&lt;/code&gt;, &lt;code&gt;HOMLEN&lt;/code&gt; etc will have commented (start with a &lt;code&gt;#&lt;/code&gt;) lines at the beginning of the file explaining what these values are. So check those, even though they're relatively standard, you never know, the obvious reading you used might be wrong despite its seeming so reasonable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a newer example of a VCF line for an SV taken from the current &lt;a href=&quot;https://github.com/samtools/hts-specs/blob/master/VCFv4.3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VCF specification&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.1&#xA;##fileDate=20100501&#xA;##reference=1000GenomesPilot-NCBI36&#xA;##assembly=ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/release/sv/breakpoint_assemblies.fasta&#xA;##INFO=&amp;lt;ID=BKPTID,Number=.,Type=String,Description=&quot;ID of the assembled alternate allele in the assembly file&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=CIEND,Number=2,Type=Integer,Description=&quot;Confidence interval around END for imprecise variants&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=CIPOS,Number=2,Type=Integer,Description=&quot;Confidence interval around POS for imprecise variants&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=END,Number=1,Type=Integer,Description=&quot;End position of the variant described in this record&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=HOMLEN,Number=.,Type=Integer,Description=&quot;Length of base pair identical micro-homology at event breakpoints&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=HOMSEQ,Number=.,Type=String,Description=&quot;Sequence of base pair identical micro-homology at event breakpoints&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=SVLEN,Number=.,Type=Integer,Description=&quot;Difference in length between REF and ALT alleles&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=SVTYPE,Number=1,Type=String,Description=&quot;Type of structural variant&quot;&amp;gt;&#xA;##ALT=&amp;lt;ID=DEL,Description=&quot;Deletion&quot;&amp;gt;&#xA;##ALT=&amp;lt;ID=DEL:ME:ALU,Description=&quot;Deletion of ALU element&quot;&amp;gt;&#xA;##ALT=&amp;lt;ID=DEL:ME:L1,Description=&quot;Deletion of L1 element&quot;&amp;gt;&#xA;##ALT=&amp;lt;ID=DUP,Description=&quot;Duplication&quot;&amp;gt;&#xA;##ALT=&amp;lt;ID=DUP:TANDEM,Description=&quot;Tandem Duplication&quot;&amp;gt;&#xA;##ALT=&amp;lt;ID=INS,Description=&quot;Insertion of novel sequence&quot;&amp;gt;&#xA;##ALT=&amp;lt;ID=INS:ME:ALU,Description=&quot;Insertion of ALU element&quot;&amp;gt;&#xA;##ALT=&amp;lt;ID=INS:ME:L1,Description=&quot;Insertion of L1 element&quot;&amp;gt;&#xA;##ALT=&amp;lt;ID=INV,Description=&quot;Inversion&quot;&amp;gt;&#xA;##ALT=&amp;lt;ID=CNV,Description=&quot;Copy number variable region&quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&quot;Genotype&quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GQ,Number=1,Type=Float,Description=&quot;Genotype quality&quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=CN,Number=1,Type=Integer,Description=&quot;Copy number genotype for imprecise events&quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=CNQ,Number=1,Type=Float,Description=&quot;Copy number genotype quality for imprecise events&quot;&amp;gt;&#xA;#CHROM POS     ID        REF             ALT QUAL FILTER INFO                                               FORMAT NA00001&#xA;1      2827694 rs2376870 CGTGGATGCGGGGAC C   .    PASS   SVTYPE=DEL;END=2827708;HOMLEN=1;HOMSEQ=G;SVLEN=-14 GT:GQ  1/1:13.9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note how the numbers do match and also note how each of the subfields in the INFO field is explained with an &lt;code&gt;##INFO&lt;/code&gt; line. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-05-29T22:34:37.427" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="276" PostTypeId="2" ParentId="242" CreationDate="2017-05-29T23:41:52.653" Score="7" Body="&lt;p&gt;&amp;quot;A few&amp;quot; 100kb reads won't help much. You need to apply the ultra-long protocol, which is different from the standard protocol.&lt;/p&gt;&#xA;&lt;p&gt;You can't resolve 20kb near identical repeats/segdups with 10kb reads. All you can do is to bet your luck on a few excessively long reads spanning some units by chance. For divergent copies, it is worth looking at &lt;a href=&quot;http://biorxiv.org/content/early/2016/05/14/053256&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;. It uses Illumina reads to identify k-mers in unique regions and ignores non-unique k-mers at the overlapping stage. The paper said that this strategy is better than using standard overlappers, which I buy, but probably it can't resolve a 20kb segdup with a handful of mismatches, either.&lt;/p&gt;&#xA;&lt;p&gt;Such mismatch-based approaches always have limitations and may not work for recent segdups/repeats. The ultimate solution is to get long reads, longer than your repeat/segdup units. The ~100kb reads in the &lt;a href=&quot;http://biorxiv.org/content/early/2017/04/20/128835&quot; rel=&quot;nofollow noreferrer&quot;&gt;recent preprint&lt;/a&gt; will be a game changer for you. If your ~20kb repeats are not tandem, 10X's ~100kb linked reads may help, too.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="73" LastEditDate="2022-06-29T05:23:03.237" LastActivityDate="2022-06-29T05:23:03.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="277" PostTypeId="2" ParentId="81" CreationDate="2017-05-30T07:06:02.543" Score="7" Body="&lt;p&gt;I am working on a Illumina sequencing simulator for metagenomics: &lt;a href=&quot;https://github.com/HadrienG/InSilicoSeq&quot; rel=&quot;noreferrer&quot;&gt;InSilicoSeq&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is still in alpha release and very experimental, but given a multi-fasta and an abundance file, it will generate reads from your input genomes with different coverages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the documentation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;iss generate --genomes genomes.fasta --abundance abundance_file.txt \&#xA;    --model_file HiSeq2500 --output HiSeq_reads&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# multi-fasta file&#xA;&amp;gt;genome_A&#xA;ATGC...&#xA;&amp;gt;genome_B&#xA;CCGT...&#xA;...&#xA;&#xA;# abundance file (total abundance must be 1!)&#xA;genome_A    0.2&#xA;genome_B    0.4&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I didn't design it to work with coverage but rather abundance of the genome in a metagenome, so you might have to do a tiny bit of math ;)&lt;/p&gt;&#xA;" OwnerUserId="179" LastActivityDate="2017-05-30T07:06:02.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="278" PostTypeId="1" AcceptedAnswerId="340" CreationDate="2017-05-30T07:30:54.223" Score="8" ViewCount="320" Body="&lt;p&gt;Using a &lt;a href=&quot;https://en.wikipedia.org/wiki/Laser_capture_microdissection&quot; rel=&quot;noreferrer&quot;&gt;laser-capture microdissection&lt;/a&gt; of cells a group of cells stained with the marker of interest was sequenced. In another cohort of patients (this is all  human liver tissue) the whole tissue was sequenced (RNA-seq in both cases)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I estimate the contribution of the cells marked in the whole liver (&quot;weight of these cells&quot; in the liver in words of my PI)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My gut feeling is that it can't be done this way, it would require both single cell sequencing and whole tissue sequencing to estimate the contribution of each cell line. But perhaps there is a tool that given the cell lines or the main expression of other cells lines it can be compared to using &lt;a href=&quot;http://www.bioconductor.org/packages/GSVA/&quot; rel=&quot;noreferrer&quot;&gt;GSVA&lt;/a&gt; or some similar tool. &lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="73" LastEditDate="2017-06-01T12:28:58.280" LastActivityDate="2020-08-18T16:31:11.030" Title="How can the cell line contribution be estimated from RNASeq data?" Tags="|rna-seq|cell-line|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="280" PostTypeId="1" AcceptedAnswerId="281" CreationDate="2017-05-30T09:05:00.953" Score="11" ViewCount="1022" Body="&lt;p&gt;I have been using &lt;a href=&quot;https://github.com/alexdobin/STAR&quot; rel=&quot;noreferrer&quot;&gt;STAR&lt;/a&gt; for our RNA-Seq samples. The &lt;code&gt;final.out&lt;/code&gt; log file reports percentage of uniquely mapped reads along with percentage of reads that map to &lt;code&gt;multiple loci&lt;/code&gt; (less than or equal to 10) and percentage of reads mapping to &lt;code&gt;too many loci&lt;/code&gt; (greater than 10). However, I want to break down the &lt;code&gt;multiple loci&lt;/code&gt; part to individual counts: Reads mapping to 2 locations, 3 locations, 4 locations .. 10 locations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;NH&lt;/code&gt; tag seems to be used by &lt;code&gt;STAR&lt;/code&gt;. However a naive read counting approach results in it reporting more number of reads than total reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, my &lt;code&gt;final.out&lt;/code&gt; looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   Mapping speed, Million of reads per hour |       1403.36&#xA;&#xA;                      Number of input reads |       53015978&#xA;                  Average input read length |       26&#xA;                                UNIQUE READS:&#xA;               Uniquely mapped reads number |       368916&#xA;                    Uniquely mapped reads % |       0.70%&#xA;                      Average mapped length |       26.45&#xA;                   Number of splices: Total |       1057&#xA;        Number of splices: Annotated (sjdb) |       0&#xA;                   Number of splices: GT/AG |       802&#xA;                   Number of splices: GC/AG |       1&#xA;                   Number of splices: AT/AC |       0&#xA;           Number of splices: Non-canonical |       254&#xA;                  Mismatch rate per base, % |       0.31%&#xA;                     Deletion rate per base |       0.00%&#xA;                    Deletion average length |       1.45&#xA;                    Insertion rate per base |       0.00%&#xA;                   Insertion average length |       1.00&#xA;                         MULTI-MAPPING READS:&#xA;    Number of reads mapped to multiple loci |       45766732&#xA;         % of reads mapped to multiple loci |       86.33%&#xA;    Number of reads mapped to too many loci |       3757890&#xA;         % of reads mapped to too many loci |       7.09%&#xA;                              UNMAPPED READS:&#xA;   % of reads unmapped: too many mismatches |       0.00%&#xA;             % of reads unmapped: too short |       5.89%&#xA;                 % of reads unmapped: other |       0.00%&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Counting histogram of number of positions a read maps to using &lt;code&gt;pysam&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def get_reads_hist(bam): &#xA;    bam = pysam.AlignmentFile(bam, 'rb')&#xA;    counts = Counter()&#xA;    for query in bam.fetch():&#xA;        nh_count = Counter(dict(query.get_tags())['NH'])&#xA;        counts += nh_count            &#xA;    return counts&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;results in &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Counter({1: 330606,&#xA;     2: 86772164,&#xA;     3: 329,&#xA;     4: 38083,&#xA;     5: 31,&#xA;     6: 1094,&#xA;     7: 129,&#xA;     8: 50,&#xA;     10: 50})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The count &lt;code&gt;1&lt;/code&gt; reads are fine even though they do not match the counts in &lt;code&gt;final.out&lt;/code&gt; file since I am counting a certain category of reads (say those mapping to &lt;code&gt;tRNA&lt;/code&gt; only), but the reads mapping to 2 locations are highly overestimated. Why is that?&lt;/p&gt;&#xA;" OwnerUserId="161" LastEditorUserId="77" LastEditDate="2017-05-30T09:33:43.417" LastActivityDate="2017-05-30T09:33:43.417" Title="Quantifying reads mapping to multiple loci" Tags="|rna-seq|sam|samtools|star|pysam|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="281" PostTypeId="2" ParentId="280" CreationDate="2017-05-30T09:08:11.583" Score="9" Body="&lt;p&gt;You almost had the correct python code already, you just need to filter out secondary alignments:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def get_reads_hist(bam): &#xA;    bam = pysam.AlignmentFile(bam, 'rb')&#xA;    counts = Counter()&#xA;    for query in bam.fetch():&#xA;        if query.is_secondary:&#xA;            continue&#xA;        nh_count = Counter(dict(query.get_tags())['NH'])&#xA;        counts += nh_count            &#xA;    return counts&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-05-30T09:08:11.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="282" PostTypeId="1" AcceptedAnswerId="305" CreationDate="2017-05-30T12:16:20.780" Score="10" ViewCount="2084" Body="&lt;p&gt;How can I manipulate protein-interaction network graph from the String database using &lt;code&gt;STRINGdb&lt;/code&gt; bioconductor package and R?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have downloaded the whole graph for &lt;em&gt;Homo sapiens&lt;/em&gt; from STRING, which has about 20.000 proteins.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How do I read the file using that package? &lt;/li&gt;&#xA;&lt;li&gt;How do I filter things I don't need? Supposing that I want to keep tumor data, as an example.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="416" LastEditorUserId="191" LastEditDate="2017-05-31T17:40:17.260" LastActivityDate="2017-06-08T08:20:19.730" Title="How to manipulate protein interaction network from String database in R?" Tags="|r|bioconductor|database|networks|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="283" PostTypeId="1" AcceptedAnswerId="284" CreationDate="2017-05-30T12:28:59.600" Score="5" ViewCount="1720" Body="&lt;p&gt;I am searching for a multi-omics dataset which may include genomics, transcriptomics and proteomics (eg. &lt;a href=&quot;http://snyderome.stanford.edu/&quot; rel=&quot;noreferrer&quot;&gt;snyderome&lt;/a&gt;).&#xA;I need such a dataset for an introductory data-exploration purpose. So it can be from any host organism and any experimental methods/conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any database/repository where I can find it?&lt;/p&gt;&#xA;" OwnerUserId="274" LastEditorUserId="57" LastEditDate="2017-05-30T19:07:49.063" LastActivityDate="2017-06-03T23:06:42.720" Title="Is there any publicly available multi-omics dataset?" Tags="|multi-omics|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="284" PostTypeId="2" ParentId="283" CreationDate="2017-05-30T12:53:39.583" Score="10" Body="&lt;p&gt;There is a database called &lt;a href=&quot;http://www.omicsdi.org/search?q=omics_type:%22Multi-Omics%22&quot; rel=&quot;nofollow noreferrer&quot;&gt;OmicsDI&lt;/a&gt;, where one can search for multi-omics datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a &lt;a href=&quot;https://doi.org/10.1101/049205&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; of the associated publication (Perez-Riverol, Yasset, et al. &quot;Omics Discovery Index-Discovering and Linking Public Omics Datasets.&quot; bioRxiv (2016): 049205.) for more details. &lt;/p&gt;&#xA;" OwnerUserId="274" LastEditorUserId="191" LastEditDate="2017-06-02T08:05:19.020" LastActivityDate="2017-06-02T08:05:19.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="285" PostTypeId="1" AcceptedAnswerId="286" CreationDate="2017-05-30T14:35:07.713" Score="13" ViewCount="402" Body="&lt;p&gt;I'm interested in obtaining coding sequences of my favourite gene in all individuals from the 1000Genomes (and similar projects). I use GATK to get the right subset of variants, vcf-consensus to map these variants onto the reference genome and finally samtools to extract the individual exons. This works fine if the variants are SNPs but if there are any indels, this changes the coordinates of exons and I end up getting the wrong region. Is there any generic way of remapping genomic coordinates to account for the changes created by indels?&lt;/p&gt;&#xA;" OwnerUserId="148" LastEditorUserId="96" LastEditDate="2019-01-15T14:28:49.213" LastActivityDate="2019-01-15T19:17:46.453" Title="Remapping genomic coordinates to account for indels" Tags="|variation|structural-variation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="286" PostTypeId="2" ParentId="285" CreationDate="2017-05-30T15:23:33.850" Score="8" Body="&lt;p&gt;I think that you need a LiftOver Chain file to transform your coordinates. You can obtain such a file using &lt;code&gt;bcftools consensus&lt;/code&gt; with the &lt;code&gt;-c&lt;/code&gt; parameter:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-c, --chain &amp;lt;file&amp;gt;         write a chain file for liftover&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you can use it to transform coordinates in various genomic formats using &lt;a href=&quot;http://crossmap.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CrossMap&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="425" LastEditorUserId="425" LastEditDate="2017-05-31T01:03:59.797" LastActivityDate="2017-05-31T01:03:59.797" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="287" PostTypeId="2" ParentId="83" CreationDate="2017-05-30T19:18:22.740" Score="2" Body="&lt;p&gt;It is one of my favorite stories. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Drop a glance to &lt;a href=&quot;http://stereogene.bioinf.fbb.msu.ru/&quot; rel=&quot;nofollow noreferrer&quot;&gt;StereoGene software&lt;/a&gt;, it for genomic track correlation, it described in a &lt;a href=&quot;http://biorxiv.org/content/early/2017/05/25/059584&quot; rel=&quot;nofollow noreferrer&quot;&gt;preprint&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also can run MACS or another peak caller and estimate the correlation of two interval sets using the &lt;a href=&quot;http://genometricorr.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GenomtriCorr&lt;/a&gt; package.&lt;/p&gt;&#xA;" OwnerUserId="318" LastEditorUserId="57" LastEditDate="2017-06-15T18:16:11.983" LastActivityDate="2017-06-15T18:16:11.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="288" PostTypeId="1" CreationDate="2017-05-30T21:18:07.883" Score="8" ViewCount="167" Body="&lt;p&gt;I have about 200 short nucleotide motifs (6-12 bp in length) from the human genome, and I'm trying to see how conserved they are across vertebrates. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking that I'd need to make a bed file for each motif that lists all of its occurrences in the human genome. From there, I could map the beds to a bigwig files of &lt;a href=&quot;http://compgen.cshl.edu/phast/phastCons-HOWTO.html&quot; rel=&quot;noreferrer&quot;&gt;PhastCons&lt;/a&gt; scores (essentially doing the reverse of what the PhastCons software was designed to do). Does that sound like the best approach? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm getting stuck at the step of going from motifs to bed files. I've tried using BLAST to find all occurrences of motifs, but their short length is causing issues.&lt;br&gt;&#xA;I've tried messing with the e-value threshold, word size, and filter parameters, but I still don't get any hits. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a work-around for this issue, or should I just rethink my entire approach?&lt;/p&gt;&#xA;" OwnerUserId="250" LastEditorUserId="44" LastEditDate="2017-05-30T23:42:19.883" LastActivityDate="2018-03-07T00:10:38.133" Title="Have DNA motifs 6-12bp long, trying to get conservation scores" Tags="|k-mer|motifs|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="289" PostTypeId="1" CreationDate="2017-05-30T21:36:46.213" Score="14" ViewCount="3922" Body="&lt;p&gt;Is there any resource (paper, blogpost, Github gist, etc.) describing the BWA-MEM algorithm for assigning mapping qualities? I vaguely remember that I have somewhere seen a formula for SE reads, which looked like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$C * (s_1 - s_2) / s_1,$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $s_1$ and $s_2$ denoted the alignment scores of two best alignments and &lt;code&gt;C&lt;/code&gt; was some constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that a reimplementation of this algorithm in some scripting language could be very useful for the bioinfo community. For instance, I sometimes test various mapping methods and some of them tend to find good alignments, but fail in assigning appropriate qualities. Therefore, I would like to re-assign all the mapping qualities in a SAM file with the BWA-MEM algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Btw. This algorithm must already have been implemented outside BWA, see the BWA-MEM paper: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;GEM does not compute mapping quality. Its&#xA;  mapping quality is estimated with a BWA-like algorithm with suboptimal&#xA;  alignments available.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the &lt;a href=&quot;https://github.com/lh3/mem-paper/&quot; rel=&quot;noreferrer&quot;&gt;BWA-MEM paper repo&lt;/a&gt; contains only the resulting &lt;code&gt;.eval&lt;/code&gt; files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; The question is &lt;em&gt;not&lt;/em&gt; about the algorithm for computing alignment scores. Mapping qualities and alignment scores are two different things:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Alignment score quantifies the similarity between two sequences (e.g., a read and a reference sequence)&lt;/li&gt;&#xA;&lt;li&gt;Mapping quality (MAQ) quantifies the probability that a read is aligned to a wrong position.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Even alignments with high scores can have a very low mapping quality.&lt;/p&gt;&#xA;" OwnerUserId="425" LastEditorUserId="298" LastEditDate="2017-07-26T16:07:05.550" LastActivityDate="2023-11-09T08:52:32.393" Title="How does the BWA-MEM algorithm assign its mapping qualities?" Tags="|sequence-alignment|bwa|read-mapping|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="290" PostTypeId="2" ParentId="288" CreationDate="2017-05-30T21:53:30.923" Score="7" Body="&lt;p&gt;&lt;strong&gt;In case you have only ACGT in your motifs&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The short motifs make it sound as if you are in the business of looking for a kmer counter. You can choose to use existing software or build your own.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Using existing software might be your easiest path. An older post from 2014 will probably give you a first idea what's out there: &lt;a href=&quot;http://homolog.us/blogs/blog/2014/04/07/kmer-counting-a-2014-recap/&quot; rel=&quot;noreferrer&quot;&gt;http://homolog.us/blogs/blog/2014/04/07/kmer-counting-a-2014-recap/&lt;/a&gt; . Note that a couple of algorithms mentioned there got successors, so it is worthwhile digging a bit around. The small kmer size will make most of them usable for your needs.&lt;/li&gt;&#xA;&lt;li&gt;As the maximum size of your kmers is comparatively small (12 nt need 24 bits, i.e., max 16.7 million entries in your kmer table), you should be able to easily roll your own kmer counting in about any language you like and on about any of nowadays computer. The pseudocode section on the &lt;a href=&quot;https://en.wikipedia.org/wiki/K-mer&quot; rel=&quot;noreferrer&quot;&gt;Wikipedia&lt;/a&gt; entry for kmers will give you first pointers for that. Might be a bit more work, but maybe more flexible depending on your needs.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In case you have IUPAC bases (N, W, etc.) in your motifs&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know any any pre-existing software doing what you need. I could imagine that the short motifs make using &lt;a href=&quot;https://en.wikipedia.org/wiki/Regular_expression&quot; rel=&quot;noreferrer&quot;&gt;regular expressions&lt;/a&gt; doable for this kind of search, but I may be wrong. Testing this should be easy in a simple script as all major programming languages have modules or libraries for REs. Even if it should take a couple of hours to run on your data set, that would be good enough for a one-off calculation.&lt;/p&gt;&#xA;" OwnerUserId="44" LastActivityDate="2017-05-30T21:53:30.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="291" PostTypeId="2" ParentId="289" CreationDate="2017-05-30T22:15:53.533" Score="3" Body="&lt;p&gt;Yes, there bwa-mem was published as a &lt;a href=&quot;https://arxiv.org/pdf/1303.3997.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;preprint&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;BWA-MEM’s seed extension differs from the standard seed extension in two aspects. Firstly, suppose at a certain extension step we come to reference&#xA;  position x with the best extension score achieved at query position y.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;...&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Secondly, while extending a seed, BWA-MEM tries to keep track of the&#xA;  best extension score reaching the end of the query sequence&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And there is a description of the scoring algorithm directly in the source code of &lt;a href=&quot;https://github.com/lh3/bwa/blob/master/bwamem.c&quot; rel=&quot;nofollow noreferrer&quot;&gt;bwa-mem&lt;/a&gt; (lines 22 - 44), but maybe the only solution is really to go though the source code.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2017-06-02T18:39:54.083" LastActivityDate="2017-06-02T18:39:54.083" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="292" PostTypeId="2" ParentId="288" CreationDate="2017-05-30T22:46:52.913" Score="3" Body="&lt;p&gt;To scan motifs in a genome (or database) I would use &lt;a href=&quot;http://meme-suite.org/tools/fimo&quot; rel=&quot;nofollow noreferrer&quot;&gt;FIMO&lt;/a&gt; which will give you the exact locations of these motifs in your genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have the locations, you can use a &lt;code&gt;phastCons&lt;/code&gt; bigiwig from UCSC to calculate the basewise conservation scores. However, please remember that &lt;code&gt;phastCons&lt;/code&gt; scores are smoothed across windows and it might not be the best metric if you are trying to compare the conservation levels at your motif matching sites as compared to the sequences flanking them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wrote a &lt;a href=&quot;https://github.com/saketkc/moca&quot; rel=&quot;nofollow noreferrer&quot;&gt;package&lt;/a&gt; a while back to do this, including doing de-novo motif discovery. However, it might be an overkill for your use case. &lt;/p&gt;&#xA;" OwnerUserId="161" LastEditorUserId="161" LastEditDate="2017-05-31T01:51:14.620" LastActivityDate="2017-05-31T01:51:14.620" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="293" PostTypeId="2" ParentId="19" CreationDate="2017-05-31T02:30:14.807" Score="8" Body="&lt;p&gt;A rolling hash function for DNA sequences called &lt;a href=&quot;http://www.bcgsc.ca/platform/bioinfo/software/nthash&quot; rel=&quot;noreferrer&quot;&gt;ntHash&lt;/a&gt; has recently been &lt;a href=&quot;https://doi.org/10.1093/bioinformatics/btw397&quot; rel=&quot;noreferrer&quot;&gt;published in Bioinformatics&lt;/a&gt; and the authors dealt with reverse complements:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Using this table, we can easily compute the hash value for the reverse-complement (as well as the canonical form) of a sequence efficiently, without actually reverse- complementing the input sequence, as follows:&#xA;  ...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;EDIT (by @user172818): I will add more details about how ntHash works. The notations used in its paper are somewhat uncommon. The source code is more informative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's first define rotation functions for 64-bit integers:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rol(x,k) := x &amp;lt;&amp;lt; k | x &amp;gt;&amp;gt; (64-k)&#xA;ror(x,k) := x &amp;gt;&amp;gt; k | x &amp;lt;&amp;lt; (64-k)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We then define a hash function &lt;code&gt;h()&lt;/code&gt; for each base. In the implementation, the authors are using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;h(A) = 0x3c8bfbb395c60474&#xA;h(C) = 0x3193c18562a02b4c&#xA;h(G) = 0x20323ed082572324&#xA;h(T) = 0x295549f54be24456&#xA;h(N) = 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The rolling hash function of a forward k-mer &lt;code&gt;s[i,i+k-1]&lt;/code&gt; is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f(s[i,i+k-1]) := rol(h(s[i]),k-1) ^ rol(h(s[i+1]),k-2) ^ ... ^ h(s[i+k-1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;^&lt;/code&gt; is the XOR operator. The hash function of its reverse complement is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;r(s[i,i+k-1]) := f(~s[i,i+k-1])&#xA;               = rol(h(~s[i+k-1]),k-1) ^ rol(h(~s[i+k-2]),k-2) ^ ... ^ h(~s[i])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;~&lt;/code&gt; gives the reverse complement of a DNA sequence. Knowing &lt;code&gt;f(s[i,i+k-1])&lt;/code&gt; and &lt;code&gt;r(s[i,i+k-1])&lt;/code&gt;, we can compute their values for the next k-mer:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f(s[i+1,i+k]) = rol(f(s[i,i+k-1]),1) ^ rol(h(s[i]),k)  ^ h(s[i+k])&#xA;r(s[i+1,i+k]) = ror(r(s[i,i+k-1]),1) ^ ror(h(~s[i]),1) ^ rol(h(~s[i+k]),k-1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This works because &lt;code&gt;rol&lt;/code&gt;, &lt;code&gt;ror&lt;/code&gt; and &lt;code&gt;^&lt;/code&gt; can all be switched in order. Finally, for a k-mer &lt;code&gt;s&lt;/code&gt;, the hash function considering both strands is the smaller between &lt;code&gt;f(s)&lt;/code&gt; and &lt;code&gt;r(s)&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;h(s) = min(f(s),r(s))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is a linear algorithm regardless of the k-mer length. It only uses simple arithmetic operations, so should be fairly fast. I have briefly tested its randomness. It seems comparable to murmur. ntHash is probably the best algorithm so far if you want to hash an arbitrarily long k-mer into 64 bits.&lt;/p&gt;&#xA;" OwnerUserId="425" LastEditorUserId="425" LastEditDate="2017-05-31T19:53:13.073" LastActivityDate="2017-05-31T19:53:13.073" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="294" PostTypeId="1" AcceptedAnswerId="304" CreationDate="2017-05-31T06:22:09.180" Score="13" ViewCount="3619" Body="&lt;p&gt;I vaguely remember, that the original plan of Oxford Nanopore was to provide cheap sequencers (MinION), but charge for base-calling. For that reason the base-calling was performed in the cloud, and the plan was to make it commercial once the technology is established.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, this limits the potential uses of MinION in the field, since huge areas do not have decent internet connection. Also, not all the data can be legally transferred to a third-party company in the clinical studies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example for the &lt;a href=&quot;https://dx.doi.org/10.1038/nature16996&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ebola paper&lt;/a&gt;, they had to create a special version of their software:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An offline-capable version of MinKNOW, with internet ‘ping’ disabled&#xA;  and online updates disabled was made available to us by Oxford&#xA;  Nanopore Technologies specifically for the project&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are couple of third-party base-callers available today. I am aware of &lt;a href=&quot;https://nanoporetech.com/publications/nanocall-open-source-basecaller-oxford-nanopore-sequencing-data&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nanocall&lt;/a&gt; and &lt;a href=&quot;https://nanoporetech.com/publications/deepnano-deep-recurrent-neural-networks-base-calling-minion-nanopore-reads&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepNano&lt;/a&gt;, but since they are not official, it can be hard for them to keep-up with the latest versions of sequencers and cells.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is it possible as of today to sequence offline without a special arrangement (like the Ebola one).&lt;/li&gt;&#xA;&lt;li&gt;If not, what's the policy of Oxford Nanopore toward third-party base-callers? Are they going to help them, or to sue them eventually?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="191" LastEditorUserId="73" LastEditDate="2017-06-08T20:21:12.793" LastActivityDate="2017-06-08T20:21:12.793" Title="Is it possible to perform MinION sequencing offline?" Tags="|ngs|minion|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="295" PostTypeId="2" ParentId="294" CreationDate="2017-05-31T07:09:09.510" Score="5" Body="&lt;p&gt;In short, yes offline basecalling is supported.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Oxford nanopore has made available an official offline basecaller, called albacore. If you have an account to the nanpore community website you can download it from &lt;a href=&quot;https://community.nanoporetech.com/downloads&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concerning running it in the field, last time I checked live basecalling was still requiring a lot of computing power, so you might want to run the MinION in the field, but basecall once back a your lab / computing facility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not aware of any plans to sue the third-party basecallers, I think Oxford Nanopore is pretty positive about them :-) &lt;/p&gt;&#xA;" OwnerUserId="179" LastActivityDate="2017-05-31T07:09:09.510" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="296" PostTypeId="1" AcceptedAnswerId="298" CreationDate="2017-05-31T09:48:18.563" Score="14" ViewCount="1685" Body="&lt;p&gt;This is a frequently-asked question within the nanopore community. Oxford Nanopore currently claims that they are able to generate run yields of 10-15 gigabases (e.g. see &lt;a href=&quot;https://twitter.com/Clive_G_Brown/status/865211116208697344&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://twitter.com/Clive_G_Brown/status/860106072081739778&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;), and yet it's more common to see users only managing in the 1-5 gigabase range.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So why the big difference in yield?&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-08T20:23:34.517" LastActivityDate="2018-10-26T20:28:34.207" Title="How can I improve the yield of MinION sequencing runs?" Tags="|fastq|minion|" AnswerCount="3" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="297" PostTypeId="1" AcceptedAnswerId="384" CreationDate="2017-05-31T09:49:29.980" Score="12" ViewCount="988" Body="&lt;p&gt;I have often downloaded datasets from the SRA where the authors failed to mention which adapters were trimmed during the processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Local alignments tend to overcome this obstacle, but it feels a bit barbaric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;fastQC works occasionally to pick them up, but sometimes fails to find the actual adapter sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually, I ended up looking up the kits they used and trying to grep for all the possible barcodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a more robust/efficient way to do this?&lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="298" LastEditDate="2017-05-31T13:17:49.717" LastActivityDate="2017-06-03T09:09:53.957" Title="How can I systematically detect unknown barcode/adapter sequences within a set of samples?" Tags="|barcode|adapter|trimming|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="298" PostTypeId="2" ParentId="296" CreationDate="2017-05-31T10:02:35.193" Score="13" Body="&lt;p&gt;I attended a talk by Josh Quick at PoreCampAU 2017, in which he discussed some common barriers to getting both good sequencing yield and long read length. It mostly boils down to being more careful with the sample preparation. Bear in mind that the MinION will still sequence a dirty sample, it will just be at a reduced yield. Here are my notes from that talk:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The hardest thing about MinION sequencing is getting the sample in the first place&lt;/li&gt;&#xA;&lt;li&gt;There are lots of different sample types and extraction methods&lt;/li&gt;&#xA;&lt;li&gt;You can't get longer reads than what you put in; shit in leads to shit out&lt;/li&gt;&#xA;&lt;li&gt;DNA is very stable when not moving, but very sensitive to lateral damage&lt;/li&gt;&#xA;&lt;li&gt;The phenol chloroform method of DNA extraction is very good, and can be used with a phase-locked gel to make extraction easier&lt;/li&gt;&#xA;&lt;li&gt;A simple salt + alcohol extraction might be the best method for extraction (because it involves the least amount of work on the DNA)&lt;/li&gt;&#xA;&lt;li&gt;EDTA (e.g. as found in TE buffer, and many extraction kits) is not compatible with the rapid kit&lt;/li&gt;&#xA;&lt;li&gt;The most consistently good Nanopore runs produced by Josh's lab were 1D ligations runs on R9.4; the best overall run was a phenol-chloroform extraction + rapid kit&lt;/li&gt;&#xA;&lt;li&gt;John Tyson can tune himself out of a low-yield hole (via software)&lt;/li&gt;&#xA;&lt;li&gt;Getting small numbers of short reads is very important&lt;/li&gt;&#xA;&lt;li&gt;Suggested (and mostly untested) purification techniques: spin column (60-100kb); ethanol extraction (100-150kb), dialysis (150-250kb); low melting-point agarose plug (~1Mb, DNA extraction in situ)&lt;/li&gt;&#xA;&lt;li&gt;The nanopore protocol input is in nanograms, but should really be stated as molarity; the kit expects about 0.2 pmol input&lt;/li&gt;&#xA;&lt;li&gt;Picture molecules tethered to the surface of the membrane. You can then see that the flow cell density is independent of the sequence length&lt;/li&gt;&#xA;&lt;li&gt;Tapestation, Qubit and Nanodrop are all a good idea; a DNA sample that can pass all three tests will work well: no short-length shoulders by Tapestation, sufficient DNA by Qubit, high purity by Nanodrop&lt;/li&gt;&#xA;&lt;li&gt;RNA can interfere with sequencing; digesting RNA is recommended&lt;/li&gt;&#xA;&lt;li&gt;Freezing DNA is a really bad idea. The ice crystals are very good at chopping DNA up into small pieces&lt;/li&gt;&#xA;&lt;li&gt;DNA that is kept in the fridge is remarkably stable; can be kept for over two years (and probably indefinitely)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Update on yield:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We had a run in June 2018 which produced about 3 million reads from PCR-amplified mouse cDNA (read N50 of ~1kb), and that was in a situation where the it was the first sample prepared by someone, the input DNA amount was probably about 20% of what it should have been, and we suffered a dataloss incident. If that's what happened with a bad run, I have high hopes for our runs in the future.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our next cDNA run in August 2018 produced about 7 million reads and a similar read N50 (i.e. total yield of about 7Gb). The run was interrupted by a Windows update during the first night (I had disabled automatic updates previously, but they had been re-enabled), and there have since been additional software and hardware updates since to improve sequencing yield. If we have a good flow cell and similar sample prep, I expect that the next cDNA run that we'll do should produce over 8 million reads, and possibly over 10 million.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-10-10T00:58:02.720" LastActivityDate="2018-10-10T00:58:02.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="299" PostTypeId="1" CreationDate="2017-05-31T10:05:01.120" Score="7" ViewCount="364" Body="&lt;p&gt;Given WGS data or RNA-seq data, which tools can I use to detect gene fusions?&lt;/p&gt;&#xA;" OwnerUserId="283" LastActivityDate="2017-05-31T16:58:30.973" Title="Which tools can detect chimeric RNA (fusion genes) from WGS or RNA-Seq data?" Tags="|rna-seq|wgs|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="300" PostTypeId="2" ParentId="299" CreationDate="2017-05-31T10:05:01.120" Score="9" Body="&lt;p&gt;Most of these use RNA-seq data, some use WGS data, and some use both. They are listed alphabetically. I will add to the list when I discover more.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Barnacle: &lt;a href=&quot;http://bmcgenomics.biomedcentral.com/articles/10.1186/1471-2164-14-550&quot; rel=&quot;noreferrer&quot;&gt;http://bmcgenomics.biomedcentral.com/articles/10.1186/1471-2164-14-550&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Bellerophontes: &lt;a href=&quot;http://bioinformatics.oxfordjournals.org/content/28/16/2114.long&quot; rel=&quot;noreferrer&quot;&gt;http://bioinformatics.oxfordjournals.org/content/28/16/2114.long&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;BreakDancer: &lt;a href=&quot;http://www.nature.com/nmeth/journal/v6/n9/abs/nmeth.1363.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.nature.com/nmeth/journal/v6/n9/abs/nmeth.1363.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;BreakFusion: &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3389765/&quot; rel=&quot;noreferrer&quot;&gt;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3389765/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;BreakPointer: &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3561864/&quot; rel=&quot;noreferrer&quot;&gt;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3561864/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;ChimeraScan: &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3187648/&quot; rel=&quot;noreferrer&quot;&gt;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3187648/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Comrad: &lt;a href=&quot;http://bioinformatics.oxfordjournals.org/content/27/11/1481.long&quot; rel=&quot;noreferrer&quot;&gt;http://bioinformatics.oxfordjournals.org/content/27/11/1481.long&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;CRAC: &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4053775/&quot; rel=&quot;noreferrer&quot;&gt;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4053775/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;deFuse: &lt;a href=&quot;http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1001138&quot; rel=&quot;noreferrer&quot;&gt;http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1001138&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Dissect: &lt;a href=&quot;http://bioinformatics.oxfordjournals.org/content/28/12/i179.abstract&quot; rel=&quot;noreferrer&quot;&gt;http://bioinformatics.oxfordjournals.org/content/28/12/i179.abstract&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;EBARDenovo: &lt;a href=&quot;http://bioinformatics.oxfordjournals.org/content/early/2013/03/01/bioinformatics.btt092&quot; rel=&quot;noreferrer&quot;&gt;http://bioinformatics.oxfordjournals.org/content/early/2013/03/01/bioinformatics.btt092&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;EricScript: &lt;a href=&quot;http://bioinformatics.oxfordjournals.org/content/28/24/3232&quot; rel=&quot;noreferrer&quot;&gt;http://bioinformatics.oxfordjournals.org/content/28/24/3232&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;FusionAnalyser: &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3439881/&quot; rel=&quot;noreferrer&quot;&gt;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3439881/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;FusionCatcher: &lt;a href=&quot;http://biorxiv.org/content/early/2014/11/19/011650.full-text.pdf+html&quot; rel=&quot;noreferrer&quot;&gt;http://biorxiv.org/content/early/2014/11/19/011650.full-text.pdf+html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;FusionFinder: &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3384600/&quot; rel=&quot;noreferrer&quot;&gt;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3384600/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;FusionHunter: &lt;a href=&quot;http://bioinformatics.oxfordjournals.org/content/27/12/1708.long&quot; rel=&quot;noreferrer&quot;&gt;http://bioinformatics.oxfordjournals.org/content/27/12/1708.long&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;FusionMap: &lt;a href=&quot;http://bioinformatics.oxfordjournals.org/content/27/14/1922&quot; rel=&quot;noreferrer&quot;&gt;http://bioinformatics.oxfordjournals.org/content/27/14/1922&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;FusionQ: &lt;a href=&quot;http://www.biomedcentral.com/1471-2105/14/193&quot; rel=&quot;noreferrer&quot;&gt;http://www.biomedcentral.com/1471-2105/14/193&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;FusionSeq: &lt;a href=&quot;http://www.genomebiology.com/2010/11/10/R104&quot; rel=&quot;noreferrer&quot;&gt;http://www.genomebiology.com/2010/11/10/R104&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;IDP-fusion: &lt;a href=&quot;http://nar.oxfordjournals.org/content/early/2015/06/03/nar.gkv562.full&quot; rel=&quot;noreferrer&quot;&gt;http://nar.oxfordjournals.org/content/early/2015/06/03/nar.gkv562.full&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;iFUSE: &lt;a href=&quot;http://bioinformatics.oxfordjournals.org/content/29/13/1700.long&quot; rel=&quot;noreferrer&quot;&gt;http://bioinformatics.oxfordjournals.org/content/29/13/1700.long&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;InFusion: &lt;a href=&quot;https://bitbucket.org/kokonech/infusion/wiki/Home&quot; rel=&quot;noreferrer&quot;&gt;https://bitbucket.org/kokonech/infusion/wiki/Home&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;INTEGRATE: &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pubmed/26556708&quot; rel=&quot;noreferrer&quot;&gt;http://www.ncbi.nlm.nih.gov/pubmed/26556708&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;JAFFA: &lt;a href=&quot;http://www.genomemedicine.com/content/7/1/43&quot; rel=&quot;noreferrer&quot;&gt;http://www.genomemedicine.com/content/7/1/43&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;LifeScope: &lt;a href=&quot;http://www.thermofisher.com/no/en/home/life-science/sequencing/next-generation-sequencing/solid-next-generation-sequencing/solid-next-generation-sequencing-data-analysis-solutions/lifescope-data-analysis-solid-next-generation-sequencing/lifescope-genomic-analysis-software-solid-next-generation-sequencing.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.thermofisher.com/no/en/home/life-science/sequencing/next-generation-sequencing/solid-next-generation-sequencing/solid-next-generation-sequencing-data-analysis-solutions/lifescope-data-analysis-solid-next-generation-sequencing/lifescope-genomic-analysis-software-solid-next-generation-sequencing.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;MapSplice: &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pubmed/20802226&quot; rel=&quot;noreferrer&quot;&gt;http://www.ncbi.nlm.nih.gov/pubmed/20802226&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;MOJO &lt;a href=&quot;https://github.com/cband/MOJO&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/cband/MOJO&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;nFuse: &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3483554/&quot; rel=&quot;noreferrer&quot;&gt;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3483554/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Pegasus: &lt;a href=&quot;http://www.biomedcentral.com/1752-0509/8/97&quot; rel=&quot;noreferrer&quot;&gt;http://www.biomedcentral.com/1752-0509/8/97&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;PRADA: &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pubmed/24695405&quot; rel=&quot;noreferrer&quot;&gt;http://www.ncbi.nlm.nih.gov/pubmed/24695405&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;ShortFuse: &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3072550/&quot; rel=&quot;noreferrer&quot;&gt;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3072550/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;SnowShoes-FTD: &lt;a href=&quot;http://nar.oxfordjournals.org/content/39/15/e100&quot; rel=&quot;noreferrer&quot;&gt;http://nar.oxfordjournals.org/content/39/15/e100&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;SOAPFuse: &lt;a href=&quot;http://www.genomebiology.com/2013/14/2/R12&quot; rel=&quot;noreferrer&quot;&gt;http://www.genomebiology.com/2013/14/2/R12&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;SOAPFusion: &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pubmed/24123671&quot; rel=&quot;noreferrer&quot;&gt;http://www.ncbi.nlm.nih.gov/pubmed/24123671&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;STAR: &lt;a href=&quot;http://bioinformatics.oxfordjournals.org/content/29/1/15&quot; rel=&quot;noreferrer&quot;&gt;http://bioinformatics.oxfordjournals.org/content/29/1/15&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;STAR-Fusion: &lt;a href=&quot;https://github.com/STAR-Fusion/STAR-Fusion/wiki&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/STAR-Fusion/STAR-Fusion/wiki&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;TopHat-Fusion: &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3245612/&quot; rel=&quot;noreferrer&quot;&gt;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3245612/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;TRUP: &lt;a href=&quot;http://www.genomebiology.com/2015/16/1/7&quot; rel=&quot;noreferrer&quot;&gt;http://www.genomebiology.com/2015/16/1/7&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;ViralFusionSeq: &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3582262/&quot; rel=&quot;noreferrer&quot;&gt;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3582262/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Other useful programs:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chimeraviz (visualization tools for gene fusions): &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/28525538&quot; rel=&quot;noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pubmed/28525538&lt;/a&gt; (disclaimer: I created this)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chimera: &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4253834/&quot; rel=&quot;noreferrer&quot;&gt;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4253834/&lt;/a&gt;&lt;br&gt;&#xA;OncoFuse: &lt;a href=&quot;http://bioinformatics.oxfordjournals.org/content/29/20/2539.long&quot; rel=&quot;noreferrer&quot;&gt;http://bioinformatics.oxfordjournals.org/content/29/20/2539.long&lt;/a&gt;&lt;br&gt;&#xA;FuMa: &lt;a href=&quot;http://bioinformatics.oxfordjournals.org/content/early/2015/12/09/bioinformatics.btv721.abstract&quot; rel=&quot;noreferrer&quot;&gt;http://bioinformatics.oxfordjournals.org/content/early/2015/12/09/bioinformatics.btv721.abstract&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="283" LastActivityDate="2017-05-31T10:05:01.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="301" PostTypeId="2" ParentId="297" CreationDate="2017-05-31T10:28:26.410" Score="1" Body="&lt;p&gt;If you happen to know a sequence that should be highly abundant in the library, you can grep its beginning or end (with pattern match highlighting) and see if the same sequence systematically comes just before or just after respectively. This kind of visual inspection can help you finding the adaptor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, in a previous lab, we were working on &lt;em&gt;D. melanogaster&lt;/em&gt; small RNA sequencing data and my colleague knew from previous experience with this kind of data that the following small RNA was likely to be abundant: &lt;a href=&quot;http://flybase.org/reports/FBgn0065042.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://flybase.org/reports/FBgn0065042.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We just had to grep it in the fastq file to see many lines with this sequence, next to another sequence that happened to be always the same: the unknown adapter.&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2017-06-03T09:09:53.957" LastActivityDate="2017-06-03T09:09:53.957" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="302" PostTypeId="2" ParentId="297" CreationDate="2017-05-31T10:45:23.277" Score="2" Body="&lt;p&gt;I'm not aware of any existing methods to do this, but here are a couple of ideas about how it might be done:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Canu has a method of adapter trimming which involves looking for the absence of overlap for reads. If there are no other reads which share sequence across a particular region, then the read is broken up at the point of low coverage, and small pieces are discarded. It would be possible to use a method like this to hunt for possible adapter/barcode sequences by preserving the short reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option is to do a kmer search at the start of reads, and see if any of the high-abundance kmers can be assembled together and/or matched to existing known adapters or barcodes.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-05-31T10:45:23.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="303" PostTypeId="1" CreationDate="2017-05-31T11:01:10.113" Score="9" ViewCount="504" Body="&lt;p&gt;Microbial genomes can contain extensive duplications. Often we'd like to transfer annotations from an annotated species to one that is newly sequenced. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Existing tools (e.g. RATT, LiftOver, Kraken) either make specific assumptions about how closely related the species are or fail to transfer when multiple matches are found in the new genome, especially if the sequences are highly similar. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, I have a synthetic biology application where genes can duplicate extensively. They are identical in sequence but duplicated many times and be relocated (i.e., not just adjacent to each other). None of the above mentioned tools are able to transfer coordinates of annotations to genomes with multiple copies of features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any pre-existing tools or software that transfer annotations in this scenario? Ideas for ways to do this robustly?&lt;/p&gt;&#xA;" OwnerUserId="442" LastEditorUserId="77" LastEditDate="2017-05-31T14:31:42.897" LastActivityDate="2017-05-31T17:43:32.240" Title="How to transfer gff annotations in genome with extensive duplications?" Tags="|sequence-annotation|ngs|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="304" PostTypeId="2" ParentId="294" CreationDate="2017-05-31T11:09:16.597" Score="8" Body="&lt;p&gt;Short answer: yes, but you need to get permission (and modified software) from ONT before doing that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;... but that doesn't tell the whole story. This question has the potential to be very confusing, and that's through no fault of the questioner. The issue is that for the MinION, sequencing (or more specifically, generating the raw data in the form of an electrical signal trace) is distinct and separable from base calling. Many other sequencers also have distinct raw data and base-calling phases, but they're not democratised to the degree they are on the MinION.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;sequencing&quot; part of MinION sequencing is carried out by ONT software, namely MinKNOW. As explained to me during PoreCampAU 2017, when the MinION is initially plugged into a computer it is missing the firmware necessary to carry out the sequencing. The most recent version of this firmware is usually downloaded at the start of a sequencing run by sending a request to ONT servers. In the usual case, you can't do sequencing without being able to access those servers, and you can't do sequencing without ONT knowing about it. However, ONT acknowledge that there are people out there who won't have Internet access when sequencing (e.g. sequencing Ebola in Africa, or metagenomic sequencing in the middle of the ocean), and an email to &lt;code&gt;&amp;lt;support@nanoporetech.com&amp;gt;&lt;/code&gt; with reasons is likely to result in a quick software fix to the local sequencing problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once the raw signals are acquired, the &quot;base-calling&quot; part of MinION sequencing can be done anywhere. The ONT-maintained basecaller is Albacore, and this will get the first model updates whenever the sequencing technology is changed (which happens a lot). Albacore is a local basecaller which can be obtained from ONT by browsing through their community pages (available to anyone who has a MinION); ONT switched to only allowing people to do basecalling locally in about April 2017, after establishing that using AWS servers was just too expensive. Albacore is open source and free-as-in-beer, but has a restrictive licensing agreement which limits the distribution (and modification) of the program. However, Albacore is not the only available basecaller. ONT provide a FOSS basecaller called &lt;a href=&quot;https://github.com/nanoporetech/nanonet&quot; rel=&quot;nofollow noreferrer&quot;&gt;nanonet&lt;/a&gt;. It's a little bit behind Albacore on technology, but ONT have said that all useful Albacore changes will eventually propagate through to nanonet. There is another non-ONT basecaller that I'm aware of which uses a neural network for basecalling: &lt;a href=&quot;https://bitbucket.org/vboza/deepnano&quot; rel=&quot;nofollow noreferrer&quot;&gt;deepnano&lt;/a&gt;. Other basecallers exist, each varying distances away technology-wise, and I expect that more will appear in the future as the technology stabilises and more change-resistant computer scientists get in on the act.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: ONT has just pulled back the curtain on their basecalling software; all the repositories that I've looked at so far (except for the Cliveome) have been released under the Mozilla Public License (free and open source, with some conditions and limitations). Included in &lt;a href=&quot;https://github.com/nanoporetech/&quot; rel=&quot;nofollow noreferrer&quot;&gt;that software repository&lt;/a&gt; is Scrappie, which is their testing / bleeding-edge version of Albacore.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-07T22:47:06.100" LastActivityDate="2017-06-07T22:47:06.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="305" PostTypeId="2" ParentId="282" CreationDate="2017-05-31T11:12:06.970" Score="4" Body="&lt;p&gt;I think the easiest way is to download the graph using &lt;code&gt;STRINGdb&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(STRINGdb)&#xA;string_db &amp;lt;- STRINGdb$new(version=&quot;10&quot;, species=9606,&#xA;                          score_threshold=400, input_directory=&quot;&quot; )&#xA;full.graph &amp;lt;- string_db$get_graph()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now you can use &lt;a href=&quot;http://igraph.org/r/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;igraph&lt;/code&gt;&lt;/a&gt;, to manipulate the graph. Let's assume you want to take 200 proteins with the highest degree, i.e. number of edges they have.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(igraph)&#xA;&#xA;# see how many proteins do you have    &#xA;vcount(full.graph)&#xA;&#xA;# find top 200 proteins with the highest degree&#xA;top.degree.verticies &amp;lt;- names(tail(sort(degree(full.graph)), 200))&#xA;&#xA;# extract the relevant subgraph&#xA;top.subgraph &amp;lt;- induced_subgraph(full.graph, top.degree.verticies)&#xA;&#xA;# count the number of proteins in it&#xA;vcount(top.subgraph)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How to get disease specific genes?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's no GO annotation for cancer or Alzheimer's disease. It is out of scope of the GO consortium.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you can do, you can either take KEGG Pathways annotation, or manually select list of relevant GO-terms. Or acquire the list from one of the papers. For example annotation term &lt;code&gt;05200&lt;/code&gt; corresponds to the cancer KEGG pathway. You can easily retrieve proteins associated with the annotation:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cancer.pathway.proteins &amp;lt;-&#xA;    string_db$get_term_proteins('05200')$STRING_id&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then perform subgraphing as described above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively you can try to get an enrichment score for an every gene given it's neighbors (the way enrichment is shown on the string-db website). Then you can keep only those having top enrichment scores. Probably &lt;code&gt;get_ppi_enrichment_full&lt;/code&gt; or &lt;code&gt;get_ppi_enrichment&lt;/code&gt; functions will help you to do that.&lt;/p&gt;&#xA;" OwnerUserId="191" LastEditorUserId="191" LastEditDate="2017-06-08T08:20:19.730" LastActivityDate="2017-06-08T08:20:19.730" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="306" PostTypeId="1" CreationDate="2017-05-31T12:17:46.297" Score="5" ViewCount="217" Body="&lt;p&gt;I recently used the minION (Nanopore, 9.4 flow cell, RAD001 kit) to generate a metagenome out of environmental samples.&#xA;Passed reads weren't brilliant (196, average 1,594bp lenght), but working with &lt;a href=&quot;https://github.com/infphilo/centrifuge&quot; rel=&quot;noreferrer&quot;&gt;centrifuge&lt;/a&gt; the classification outputs turned out to have quite low hitLength to queryLength scores (average 2%, max 14%). This plus score values don't give me a lot of confidence towards the results I got.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone else used &lt;em&gt;centrifuge&lt;/em&gt; and experienced the same?&lt;/p&gt;&#xA;" OwnerUserId="445" LastEditorUserId="77" LastEditDate="2017-05-31T12:22:34.797" LastActivityDate="2017-05-31T15:45:09.357" Title="Rapid metagenomics classifiers on long read data" Tags="|nanopore|long-reads|minion|metagenome|" AnswerCount="1" CommentCount="2" ClosedDate="2017-06-01T15:05:02.230" ContentLicense="CC BY-SA 3.0" />
  <row Id="307" PostTypeId="1" AcceptedAnswerId="310" CreationDate="2017-05-31T13:40:38.773" Score="11" ViewCount="2610" Body="&lt;p&gt;I have the following FASTA file, &lt;code&gt;original.fasta&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;foo&#xA;GCTCACACATAGTTGATGCAGATGTTGAATTCACTATGAGGTGGGAGGATGTAGGGCCA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I need to change the record id from &lt;code&gt;foo&lt;/code&gt; to &lt;code&gt;bar&lt;/code&gt;, so I wrote the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;original_file = r&quot;path\to\original.fasta&quot;&#xA;corrected_file = r&quot;path\to\corrected.fasta&quot;&#xA;&#xA;with open(original_file) as original, open(corrected_file, 'w') as corrected:&#xA;    records = SeqIO.parse(original_file, 'fasta')&#xA;    for record in records:&#xA;        print record.id             # prints 'foo'&#xA;        if record.id == 'foo':&#xA;            record.id = 'bar'&#xA;        print record.id             # prints 'bar' as expected&#xA;        SeqIO.write(record, corrected, 'fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We printed the record id before and after the change, and get the expected result. We can even doublecheck by reading in the corrected file again with BioPython and printing out the record id:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;with open(corrected_file) as corrected:&#xA;    for record in SeqIO.parse(corrected, 'fasta'):&#xA;        print record.id                  # prints 'bar', as expected&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, if we open the corrected file in a text editor, we see that the record id is not &lt;code&gt;bar&lt;/code&gt; but  &lt;code&gt;bar foo&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;bar foo&#xA;GCTCACACATAGTTGATGCAGATGTTGAATTCACTATGAGGTGGGAGGATGTAGGGCCA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We can confirm that this is what is written to the file if we read the file using plain Python:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;with open(corrected_file) as corrected:&#xA;    print corrected.readlines()[0][1:] # prints 'bar foo'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is this a bug in BioPython? And if not, what did I do wrong and how do I change the record id in a FASTA file using BioPython?&lt;/p&gt;&#xA;" OwnerUserId="450" LastActivityDate="2017-05-31T19:39:46.323" Title="Changing the record id in a FASTA file using BioPython" Tags="|fasta|python|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="308" PostTypeId="1" AcceptedAnswerId="389" CreationDate="2017-05-31T13:47:17.630" Score="5" ViewCount="480" Body="&lt;p&gt;&lt;a href=&quot;http://www.sanger.ac.uk/science/tools/smalt-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;SMALT&lt;/a&gt; seems to be one of the most used read mappers for bacterial data, see, e.g., &lt;a href=&quot;https://scholar.google.com/scholar?q=%22smalt%22+bacteria&amp;amp;as_ylo=2016&quot; rel=&quot;nofollow noreferrer&quot;&gt;this query&lt;/a&gt;. I do not say that it is not a great mapper, but I cannot easily see what are its main strengths compared to mappers such as BWA-MEM, Bowtie2, NovoAlign or GEM. Moreover, it is not even published. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you name some of its distinguishing features (e.g., user support by Sanger Pathogens)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have heard only arguments like &quot;We use SMALT because everyone does it.&quot;, but this is not convincing enough for me.&lt;/p&gt;&#xA;" OwnerUserId="425" LastEditorUserId="298" LastEditDate="2017-06-07T12:39:23.120" LastActivityDate="2017-06-07T12:39:23.120" Title="Why is SMALT better for microbial genomics than other mappers?" Tags="|read-mapping|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="309" PostTypeId="2" ParentId="303" CreationDate="2017-05-31T14:04:22.683" Score="3" Body="&lt;p&gt;I think you will have to first identify the regions homologous to the ones defined in your GFF and then transfer the annotations. Of course, the assumption there is that the homolog will also have the same annotation which is often not true. However, I don't see how you can do it in any other way since you cannot use genomic coordinates (and you would still be making the same assumption even if you could, anyway) when the genomes are so different.&lt;/p&gt;&#xA;&lt;p&gt;For a very simplistic approach (which might be enough if, as you say, your sequences are almost identical), you can do something like:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Collect the sequences of interest from your already annotated species.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a tool like &lt;a href=&quot;https://www.ebi.ac.uk/Tools/psa/genewise/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;genewise&lt;/code&gt;&lt;/a&gt; or &lt;a href=&quot;https://www.ebi.ac.uk/about/vertebrate-genomics/software/exonerate&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;exonerate&lt;/code&gt;&lt;/a&gt; to map these into the target genome. Both tools can return gff-formatted output and both can find multiple hits in the target genome. For what you want, I would suggest using a very high threshold of sequence similarity and query coverage (where the target sequence found covers all or most of the query sequence used).&lt;/p&gt;&#xA;&lt;p&gt;Since these are microbial genomes and therefore splicing isn't a problem, you could do the same thing with even a simple BLASTn or tBLASTn if you start from protein sequences.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;At this point, you should have a list of homologs (some of which will be &lt;a href=&quot;https://biology.stackexchange.com/a/4964/1306&quot;&gt;orthologs and others paralogs&lt;/a&gt;) and you can transfer the annotations of the query sequence over to the target.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Again, I stress that this is making a whopping huge assumption: that homologous sequences have the same function and can automatically be annotated as whatever you had in the query genome. This is going to be true for many cases but it will also be false for others. Especially if you are looking at paralogs (genes whose duplication occurred after the speciation event and are therefore likely to have diverged in function).&lt;/p&gt;&#xA;&lt;p&gt;However, as I said before, this problem would be exactly the same even if you did manage to transfer annotations just by identifying the syntenic regions of the genomes&lt;sup&gt;1&lt;/sup&gt;, so there's not much difference there.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;sup&gt;1&lt;/sup&gt; &lt;sub&gt;As I said in the comments, I don't see how this could be possible. By definition, if you have extensive duplications, the genomic coordinates will be completely different and it is impossible to map from one genome into the other. &lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-05-31T14:04:22.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="310" PostTypeId="2" ParentId="307" CreationDate="2017-05-31T14:36:06.500" Score="11" Body="&lt;p&gt;If I use &lt;code&gt;SeqIO.parse(filehandle, 'fasta')&lt;/code&gt; to parse a FASTA file, then it will return a &lt;code&gt;SeqRecord&lt;/code&gt; object where the &lt;code&gt;id&lt;/code&gt; and &lt;code&gt;name&lt;/code&gt; are the first word (everything before the first whitespace) of the line beginning with &lt;code&gt;&amp;gt;&lt;/code&gt; and the &lt;code&gt;description&lt;/code&gt; is the complete line (all not including the initial &lt;code&gt;&amp;gt;&lt;/code&gt;). (This behaviour can overruled by providing &lt;a href=&quot;https://github.com/biopython/biopython/blob/dbdf7c155af3e7f203ea966acedc12740271ed73/Bio/SeqIO/FastaIO.py#L102&quot; rel=&quot;noreferrer&quot;&gt;a custom &lt;code&gt;title2ids&lt;/code&gt; function&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if I for example have a &lt;code&gt;original.fasta&lt;/code&gt; file like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Lorem|ipsum&amp;gt;dolor sit amet&#xA;GCTCACACATAGTTGATGCAGATGTTGAATTCACTATGAGGTGGGAGGATGTAGGGCCA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I will get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; from Bio import SeqIO&#xA;&amp;gt;&amp;gt;&amp;gt; path = r'C:\path\to\original.fasta'&#xA;&amp;gt;&amp;gt;&amp;gt; records = SeqIO.parse(open(path), 'fasta')&#xA;&amp;gt;&amp;gt;&amp;gt; record = next(records)&#xA;&amp;gt;&amp;gt;&amp;gt; record.id&#xA;'Lorem|ipsum&amp;gt;dolor'&#xA;&amp;gt;&amp;gt;&amp;gt; record.name&#xA;'Lorem|ipsum&amp;gt;dolor'&#xA;&amp;gt;&amp;gt;&amp;gt; record.description&#xA;'Lorem|ipsum&amp;gt;dolor sit amet'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, what happens in my case when you do &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    if record.id == 'foo':&#xA;        record.id = 'bar'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is that the &lt;code&gt;record.id&lt;/code&gt; is successfully changed from &lt;code&gt;foo&lt;/code&gt; to &lt;code&gt;bar&lt;/code&gt;, but that the &lt;code&gt;record.description&lt;/code&gt; is not changed and stays &lt;code&gt;foo&lt;/code&gt;. That's why, when the FASTA file is printed out, I see the described behaviour of &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;bar foo&#xA;GCTCACACATAGTTGATGCAGATGTTGAATTCACTATGAGGTGGGAGGATGTAGGGCCA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, the solution to my problem is to both change the &lt;code&gt;id&lt;/code&gt; AND the &lt;code&gt;description&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;original_file = r&quot;path\to\original.fasta&quot;&#xA;corrected_file = r&quot;path\to\corrected.fasta&quot;&#xA;&#xA;with open(original_file) as original, open(corrected_file, 'w') as corrected:&#xA;    records = SeqIO.parse(original_file, 'fasta')&#xA;    for record in records:&#xA;        if record.id == 'foo':&#xA;            record.id = 'bar'&#xA;            record.description = 'bar'&#xA;        SeqIO.write(record, corrected, 'fasta')&#xA;&#xA;with open(corrected_file) as corrected:&#xA;    records = SeqIO.parse(corrected, 'fasta')&#xA;    for record in records:&#xA;        print record.id             # prints bar&#xA;        print record.name           # prints bar&#xA;        print record.description    # prints bar&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="450" LastEditorUserId="280" LastEditDate="2017-05-31T19:39:46.323" LastActivityDate="2017-05-31T19:39:46.323" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="311" PostTypeId="2" ParentId="303" CreationDate="2017-05-31T15:34:47.833" Score="5" Body="&lt;p&gt;There is one very simplistic way I use which &lt;em&gt;might&lt;/em&gt; work for what you are doing, it is similar to what terdon proposed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take a de-novo microbial genome annotation tool (I have my own, but you could use/modify &lt;a href=&quot;http://www.vicbioinformatics.com/software.prokka.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;prokka&lt;/a&gt;). Tools like these often first predict gene boundaries (with other tools like &lt;a href=&quot;http://prodigal.ornl.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;prodigal&lt;/a&gt; or &lt;a href=&quot;https://ccb.jhu.edu/software/glimmer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;glimmer&lt;/a&gt;) and then try to assign a function to found genes. This function assignment is often done with BLAST and other tools ... and that is where you can go in and modify to do what you need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use a &quot;knowledge&quot; protein database of genes I want to have very strictly annotated as a first line of annotation (e.g. in your case: the annotated genomes). For that I loop through very strict identity/similarity parameters which get gradually relaxed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g.:&#xA;Loop 0: only transfer annotations at 100% DNA identity, same length.&#xA;Loop 1: only transfer annotations at 100% similarity, same length.&#xA;Loop 2: only transfer annotations at 99% similarity, length +/- 1%.&#xA;...&#xA;Loop n: only transfer annotations at 100-(n-1)% similarity, length +/- (n-1)%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In each loop, obviously only annotate what has not been annotated in previous loops.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that, use &quot;normal&quot; annotation pipeline of the tool to annotate the rest.&lt;/p&gt;&#xA;" OwnerUserId="44" LastEditorUserId="44" LastEditDate="2017-05-31T17:43:32.240" LastActivityDate="2017-05-31T17:43:32.240" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="312" PostTypeId="2" ParentId="306" CreationDate="2017-05-31T15:45:09.357" Score="1" Body="&lt;p&gt;You should also keep in mind that single read accuracy of ONT still is a bit lacking, a 2D or 1D^2 accuracy of 95% still means that on average there's one error every 20 bp, and due to the nature of the data some stretches may be more junky than others. Maybe centrifuge doesn't like that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Getting a feel of what the data can tell you (and what not) is pretty important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may want to try other classification tools to get an idea how well centrifuge works, Kraken comes to mind. But also: what about BLASTing some of these reads at NCBI and simply look at the results to get a first impression?&lt;/p&gt;&#xA;" OwnerUserId="44" LastActivityDate="2017-05-31T15:45:09.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="313" PostTypeId="2" ParentId="299" CreationDate="2017-05-31T16:58:30.973" Score="0" Body="&lt;p&gt;1- Chimerascan + Star&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2- Tophat Fusion&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two pipelines I have used extensively and recommend. The others listed by @L42 I can't speak to but Star Fusion sounds promising. Star is fast. &lt;/p&gt;&#xA;" OwnerUserId="460" LastActivityDate="2017-05-31T16:58:30.973" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="314" PostTypeId="1" CreationDate="2017-05-31T18:06:26.410" Score="6" ViewCount="131" Body="&lt;p&gt;I have:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A list of differentially phosphorylated sites in a knockout condition. Some genes contain as many as 70 possible phosphorylation sites; others contain only one.&lt;/li&gt;&#xA;&lt;li&gt;A list of genes belonging to a specific gene set annotation.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How can I test the differentially phosphorylated proteins for enrichment of this annotation?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few ideas I’ve considered:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Ignore the number of phosphorylation events detected within a gene and simply count the gene as differentially phosphorylated if it contains at least a single site that is differentially phosphorylated. Compare the enrichment score for this selected set to the enrichment score for the set genes containing at least one site that is not differentially phosphorylated. The problem here is that genes with a large number of phosphorylation sites have almost no influence in the enrichment score, since it’s almost certain that they’ll have at least one site that is differentially phosphorylated and at least one that is not.&lt;/li&gt;&#xA;&lt;li&gt;Mark each candidate phosphorylation site as either “in the set” or “not in the set” based on the protein in which it’s found. Then perform the enrichment analysis using the set of annotated phosphorylation sites instead of using the traditional enrichment analysis performed at the gene level. The potential problem with this approach is that it may place too much influence on genes with many potential phosphorylation sites.&lt;/li&gt;&#xA;&lt;li&gt;Aggregate all candidate phosphorylation sites within a gene and use some numerical threshold to determine whether the gene is differentially phosphorylated or not. (There are various ways that this could be done.) Then perform enrichment analysis using the resulting set of differentially phosphorylated genes. A possible problem here is that some of the phosphorylation sites may be more functionally important than others, so it’s not clear how to weight the relative importance of individual phosphorylation sites within a gene.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I realize the goal here isn’t well-defined mathematically; I’m mainly curious what approach makes the most sense given the biological context. Of the above approaches, I’m currently leaning towards approach 2 because it’s straightforward to implement and it at least &lt;em&gt;attempts&lt;/em&gt; to account for the variable number of phosphorylation events within a gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOTE: I also have normalized phosphoprotein and protein abundances for all of these sites, obtained from mass spec. So if the solution requires an alternative method of computing differential phosphorylation, that’s fine.&lt;/p&gt;&#xA;" OwnerUserId="177" LastActivityDate="2017-05-31T19:05:58.757" Title="Gene set enrichment analysis on differential phosphorylation sites" Tags="|gse|differential-expression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="315" PostTypeId="2" ParentId="314" CreationDate="2017-05-31T18:35:14.477" Score="1" Body="&lt;p&gt;Your dataset looks perfect for the &lt;a href=&quot;https://dx.doi.org/10.1186/1753-6561-3-S7-S96&quot; rel=&quot;nofollow noreferrer&quot;&gt;SUMSTAT&lt;/a&gt; enrichment test.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You need to come up with a statistic representing your gene. The simplest ideas here are the number of sites or, probably better, a proportion of phosphorylated sites.&lt;/li&gt;&#xA;&lt;li&gt;Now you can compute a statistic for every gene set, for example just sum (SUMSTAT). You can also have average, sum of squares or something else.&lt;/li&gt;&#xA;&lt;li&gt;Geta a null distribution of your gene set statistics by permutations. You can either randomize statistics for genes, or just randomly assign phosphorylated sites across the genome keeping the total number of sites.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now you can compute p-value just by comparing your value with the null-distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have to be cautious of two things:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You are testing multiple gene sets, i.e. you perform a statistical test for every gene set. Therefore correct your significance for multiple testing. I suggest using FDR for this.&lt;/li&gt;&#xA;&lt;li&gt;Beware of potential biases when doing permutations. The obvious one is gene length, i.e. longer genes have higher chance of getting at least one site. But there can be other ones like GC-content, chromosomes, etc. You can overcome this by using more realistic permutations, or by controlling for the potential correlations. You can get a few ideas on fighting biases from &lt;a href=&quot;https://doi.org/10.1093/molbev/msx083&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; (sorry for the self-advertisement).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="191" LastEditorUserId="191" LastEditDate="2017-05-31T19:05:58.757" LastActivityDate="2017-05-31T19:05:58.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="316" PostTypeId="1" AcceptedAnswerId="320" CreationDate="2017-05-31T20:27:51.153" Score="3" ViewCount="46" Body="&lt;p&gt;We have arrayCGH (aCGH) results for one sample. There is a 0.5 Mb terminal duplication on chromosome 19 (62995490-63407936, according to NCBI36/hg18). The duplication is rare: a literature review suggests there are only 3-4 samples with clinical information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the steps to validate the results? How do we ascertain that this duplication is the cause of the clinical symptoms?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have some ideas:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;aCGH the parents. Not sure how this would help.   &lt;/li&gt;&#xA;&lt;li&gt;whole genome exome sequencing. Worried this might make it more difficult to pinpoint genetic cause.&lt;/li&gt;&#xA;&lt;li&gt;whole genome sequencing?&lt;/li&gt;&#xA;&lt;li&gt;other ideas?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; I am new to aCGH and high-throughput sequencing, any advice is welcome.&lt;/p&gt;&#xA;" OwnerUserId="131" LastEditorUserId="73" LastEditDate="2017-06-02T03:49:26.877" LastActivityDate="2017-06-02T03:49:26.877" Title="How do I validate a single sample ArrayCGH result?" Tags="|ngs|array-cgh|exome|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="317" PostTypeId="1" CreationDate="2017-05-31T20:39:57.870" Score="6" ViewCount="214" Body="&lt;p&gt;There are many posts on the web regarding QC steps pre and post-imputation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does applying below (new?) 10% &lt;a href=&quot;https://en.wikipedia.org/wiki/Minor_allele_frequency&quot; rel=&quot;nofollow noreferrer&quot;&gt;MAF&lt;/a&gt; difference rule make sense, pitfalls?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the process:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Get MAF for imputed set, using &lt;a href=&quot;https://mathgen.stats.ox.ac.uk/genetics_software/snptest/snptest.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SNPTEST&lt;/a&gt; with flag &lt;code&gt;-summary_stats_only&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Convert imputed set to hard-calls using &lt;a href=&quot;http://www.well.ox.ac.uk/~cfreeman/software/gwas/gtool.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;gtools&lt;/a&gt; with flag &lt;code&gt;--threshold 0.9&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;If the MAF from step 1 and step 2 differs more than 10% than exclude the variant.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;More info:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;GWAS is 50K vs 50K case control samples.&lt;/li&gt;&#xA;&lt;li&gt;This step is applied after &lt;code&gt;info &amp;gt; 0.4&lt;/code&gt; filter.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="131" LastEditorUserId="131" LastEditDate="2017-06-05T06:30:45.717" LastActivityDate="2018-01-16T21:12:58.123" Title="Filtering imputed GWAS SNPs based on a MAF difference of 10%" Tags="|gwas|imputation|qc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="318" PostTypeId="1" AcceptedAnswerId="319" CreationDate="2017-05-31T20:47:18.167" Score="8" ViewCount="153" Body="&lt;p&gt;Is working with and relying on old genome builds still valid?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example &lt;a href=&quot;https://genome-euro.ucsc.edu/cgi-bin/hgGateway?db=hg18&amp;amp;redirect=manual&amp;amp;source=genome.ucsc.edu&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI36/hg18&lt;/a&gt;. Would results from papers based on old builds require &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgLiftOver&quot; rel=&quot;nofollow noreferrer&quot;&gt;LiftOver&lt;/a&gt; and re-analysis to be useful?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A bit of context, this is related to other post, where we have aCGH results based on old build: &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/316/131&quot;&gt;How do I validate a single sample ArrayCGH result?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="131" LastEditorUserId="48" LastEditDate="2018-06-19T10:53:02.923" LastActivityDate="2018-06-19T10:53:02.923" Title="Working with old genome builds" Tags="|human-genome|array-cgh|liftover|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="319" PostTypeId="2" ParentId="318" CreationDate="2017-05-31T21:03:50.680" Score="8" Body="&lt;p&gt;In my opinion, it is not very reliable. LiftOver is very limited in terms of transformations it can support. The &lt;a href=&quot;https://genome.ucsc.edu/goldenpath/help/chain.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;LiftOver Chain format&lt;/a&gt; can capture only matching regions in the same order. It means that it can account for indels, but even simple structural variations become problematic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, when a newer assembly is available, it is usually a recommended practice to remap all the reads rather than transform the existing alignments.&lt;/p&gt;&#xA;" OwnerUserId="425" LastEditorUserId="425" LastEditDate="2017-06-07T21:09:58.510" LastActivityDate="2017-06-07T21:09:58.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="320" PostTypeId="2" ParentId="316" CreationDate="2017-05-31T21:28:07.403" Score="2" Body="&lt;p&gt;Why not just good old qPCR? That's (A) quick, (B) cheap and (C) easy to analyze. If you care about the exact location of the break point (I'm guessing from the context that you don't), then targeted sequencing with a custom capture kit would work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding validating the biological relevance of this, there are multiple (parallel) routes one can take. Firstly, screen unaffected family members for this same alteration. If you find this in unaffected individuals then it's obviously not the causative alteration. Ideally, one would also do either a cell-line experiment or a mouse (or other model organism) experiment to see if you can reconstitute at least some component of the clinical phenotype. This may not always be possible, of course.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-05-31T21:28:07.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="321" PostTypeId="1" CreationDate="2017-05-31T22:32:59.663" Score="8" ViewCount="1653" Body="&lt;p&gt;The most variant calling pipeline GATK include a &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/article?id=44&quot; rel=&quot;nofollow noreferrer&quot;&gt;Base Quality Score Recalibration (BQSR)&lt;/a&gt; which requires a list of known variants. Recently, some work has been done for reference-free recalibration of scores as well: &lt;a href=&quot;http://biorxiv.org/content/early/2017/04/27/130732&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lacer&lt;/a&gt; and &lt;a href=&quot;http://www.genetics.org/content/early/2016/11/07/genetics.116.189985&quot; rel=&quot;nofollow noreferrer&quot;&gt;atlas&lt;/a&gt;, which is motivated by making the most for aDNA and low coverage datasets. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The importance for aDNA is explained in &lt;a href=&quot;https://www.youtube.com/watch?v=oJ9pbQsyaUg&amp;amp;list=PLoCxWrRWjqB1JUCntl4X09ezmOtKx1Gke&amp;amp;index=4&quot; rel=&quot;nofollow noreferrer&quot;&gt;this lecture&lt;/a&gt;, but it is not clear to me if / how is important BQSR is for fresh DNA samples with decent (&gt;15x) coverage. Especially when I work with non-model organisms and I can not simply use the standard tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How big an impact does recalibration of scores have on variant calling? Is there a rule of thumb for which it is / it is not worth the effort?&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2018-06-19T14:24:12.543" LastActivityDate="2020-01-22T11:10:16.963" Title="Is there a point in recalibration of scores for variant calling?" Tags="|ngs|variant-calling|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="322" PostTypeId="1" AcceptedAnswerId="323" CreationDate="2017-05-31T22:56:26.803" Score="15" ViewCount="12884" Body="&lt;p&gt;How do I download a reference genome that I can use with bowtie2? Specifically HG19. On UCSC there are a lot of file options.&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="37" LastEditDate="2017-06-25T02:11:38.870" LastActivityDate="2017-06-25T02:11:38.870" Title="Downloading a reference Genome for Bowtie2" Tags="|human-genome|reference-genome|genome-sequencing|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="323" PostTypeId="2" ParentId="322" CreationDate="2017-05-31T23:21:53.943" Score="9" Body="&lt;p&gt;tl;dr: Just use the either the downloads on the &lt;a href=&quot;http://bowtie-bio.sourceforge.net/bowtie2/index.shtml&quot; rel=&quot;noreferrer&quot;&gt;Bowtie2 homepage&lt;/a&gt; or the &lt;a href=&quot;https://support.illumina.com/sequencing/sequencing_software/igenome.html&quot; rel=&quot;noreferrer&quot;&gt;Illumina iGenomes&lt;/a&gt;. Or just uncompress and concatenate the &lt;a href=&quot;http://hgdownload.soe.ucsc.edu/goldenPath/hg19/chromosomes/&quot; rel=&quot;noreferrer&quot;&gt;FASTA files found on UCSC goldenpath&lt;/a&gt; and then build the index.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A bit longer answer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two components to &quot;genome for a read mapper&quot; such as Bowtie or BWA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, you need to choose the actual sequence (genome release such as GRCh37/hg19 or GRCh38/hg38). There are patch releases such as GRCh37.p3 where some bases might be exchanged and depending on the release, some &quot;unmapped&quot; loci contigs might be added, but generally GRCh37.p1 is roughly the same as GRCh37.p2, for example. Usually, people have agreed on some specific patch version for each read and use this for read mapping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, there is the UCSC flavour hg19/hg38 etc. and the NCBI/GRC flavour GRCh37, GRCh38 etc. (similar with mouse). UCSC has no versioning besides the genome release and (to the best of my knowledge) does not update the genome sequence after releasing a hg19 FASTA file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, you have to build the index files for each genome. Depending on the read mapper you use, you might or might not need the original FASTA files for the alignment. For Bowtie and Bowtie 2, you don't need the original FASTA files after building the index as Bowtie 1/2 can reconstruct the sequence &quot;on the fly&quot; from the index files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;HTH&lt;/p&gt;&#xA;" OwnerUserId="247" LastActivityDate="2017-05-31T23:21:53.943" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="324" PostTypeId="2" ParentId="321" CreationDate="2017-05-31T23:27:28.080" Score="2" Body="&lt;p&gt;That's a good question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd say that you don't need to bother with variant recalibration for&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;low number of samples (e.g., just two trios); I could not get GTAK recalibration of variant scores to work anyway&lt;/li&gt;&#xA;&lt;li&gt;high-coverage samples (e.g., X Ten genomes with 30x coverage) where the DNA samples themselves are of high, comparable quality and have been sequenced with consistent technology.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Generally, it is my impression that a lot of the thoughts and advanced statistical models built into GATK come from the earlier phases of the 1000 Genomes project. This means (1) low-coverage, (2) different coverage genomes (3) sequenced with varying technology versions by (4) different samples and (5) population sequencing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are in a clinical setting where you do 30x sequencing on X Ten platforms only anyway, then variant recalibration will probably not help you that much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if you are integrating many data sets from different data centers and machine versions etc., variant recalibration might be worth a shot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good check would be looking at genotype quality distributions and other variant/quality related metrics before and after recalibration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone: please correct me if I'm wrong!&lt;/p&gt;&#xA;" OwnerUserId="247" LastActivityDate="2017-05-31T23:27:28.080" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="325" PostTypeId="2" ParentId="318" CreationDate="2017-05-31T23:34:31.387" Score="4" Body="&lt;p&gt;I think that right now, the only human builds that are worth considering are hg19/GRCh37 as many data bases such as gnomAD still exclusively use this release. On the other hand, hg38/GRCh8 has many important fixes and the useful (but yet underused) feature of alternative loci.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anything from older releases should be remapped to a more recent one.&lt;/p&gt;&#xA;" OwnerUserId="247" LastActivityDate="2017-05-31T23:34:31.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="326" PostTypeId="1" CreationDate="2017-05-31T23:36:32.283" Score="3" ViewCount="85" Body="&lt;p&gt;What is the &quot;best&quot; assembly for the popular model organisms:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;human (GRCh37 and GRCh38 are obvious, I'd pick whatever bwakit uses)&lt;/li&gt;&#xA;&lt;li&gt;mouse (GRCm37/GRCm38, OK)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;but what about non-human/mouse ones?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;fruit fly&lt;/li&gt;&#xA;&lt;li&gt;zebrafish&lt;/li&gt;&#xA;&lt;li&gt;E. coli&lt;/li&gt;&#xA;&lt;li&gt;any other idea?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="247" LastActivityDate="2017-06-01T16:50:34.150" Title="Which reference to use for read mapping for popular model organisms" Tags="|read-mapping|reference-genome|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="327" PostTypeId="1" CreationDate="2017-05-31T23:38:44.793" Score="5" ViewCount="618" Body="&lt;p&gt;What are good means for performing quality control (QC) or NGS reads?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm aware of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;FastQC&lt;/li&gt;&#xA;&lt;li&gt;NGS Screen&lt;/li&gt;&#xA;&lt;li&gt;Kraken (e.g., for screening against contaminants)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What are other popular means for such QC?&lt;/p&gt;&#xA;" OwnerUserId="247" LastEditorUserId="57" LastEditDate="2017-06-04T20:02:08.037" LastActivityDate="2017-06-20T16:44:39.103" Title="QC measures for NGS sequencing" Tags="|ngs|quality-control|" AnswerCount="5" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="328" PostTypeId="1" CreationDate="2017-05-31T23:40:03.393" Score="6" ViewCount="120" Body="&lt;p&gt;One big problem that I'm regularly facing is that URLs for downloading Bioinformatics data (e.g., RefSeq releases or NCBI genome releases) disappear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any good solution for this?&lt;/p&gt;&#xA;" OwnerUserId="247" LastActivityDate="2017-06-03T13:36:28.887" Title="Stable download URLs" Tags="|data-download|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="329" PostTypeId="1" CreationDate="2017-05-31T23:46:41.107" Score="6" ViewCount="162" Body="&lt;p&gt;PacBio is selling ~10x PacBio SEQUEL long reads as an upgrade to Illumina data for SV discovery.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a clinical setting, the main requirements are proper sensitivity and specificity but also the processing of cohorts, at least families. This requires a genotyping step, such that it can be identified whether a given variant is shared by two or more individuals or whether it is not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the tools of the trade for this task?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As having 50-60x PacBio reads is not an option from a economic point of view, one has to make do with 10x coverage.&lt;/p&gt;&#xA;" OwnerUserId="247" LastEditorUserId="57" LastEditDate="2017-06-01T15:11:31.337" LastActivityDate="2017-06-01T15:11:31.337" Title="Structural variant calling for low-coverage PacBio data" Tags="|structural-variation|long-reads|pacbio|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="330" PostTypeId="2" ParentId="329" CreationDate="2017-05-31T23:48:37.513" Score="1" Body="&lt;p&gt;I'm aware of the following (very few and suboptimal) options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/fritzsedlazeck/Sniffles/releases&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sniffles&lt;/a&gt; -- sadly not very reliable in my experience, also no genotyping step or multi-sample support&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-15-180&quot; rel=&quot;nofollow noreferrer&quot;&gt;PB Honey&lt;/a&gt; -- no genotyping step or multi-sample support&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="247" LastActivityDate="2017-05-31T23:48:37.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="332" PostTypeId="2" ParentId="326" CreationDate="2017-06-01T02:02:50.910" Score="2" Body="&lt;p&gt;If the programs you are using allow for it, take the most recent available genome, as it will be most likely to have the fewest errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many of the well-known model organisms have an official release site (e.g. &lt;a href=&quot;http://flybase.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;flybase&lt;/a&gt; for drosophila, &lt;a href=&quot;http://www.wormbase.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;wormbase&lt;/a&gt; for nematodes) from which genomes are fairly promptly propagated through to larger public databases, of which the most well-known are probably &lt;a href=&quot;http://www.ensembl.org/info/about/species.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl&lt;/a&gt; and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genome/browse/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefSeq Genome&lt;/a&gt;. There's usually enough cross-talk between the large data repositories that it doesn't matter all that much where the source is, as long as it is at least a few months since the last genome release.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-01T02:02:50.910" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="333" PostTypeId="1" AcceptedAnswerId="339" CreationDate="2017-06-01T03:27:11.300" Score="4" ViewCount="360" Body="&lt;p&gt;I would like to show that certain types of RBP motifs are enriched in RNA editing islands (i.e. clusters of RNA editing). However, I am unsure about how to think about sequence motifs with respect to their occurrence in other genomic features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand how to find the probability of a motif at a location. i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P(sequence is at position i) = P(A)^[A] * P(C)^[C] * P(G)^[G] * P(T)^[T]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where (.) is the base pair and [.] is the number of bps. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead what I would like to find is: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;P(sequence S is contained in a feature type T) = ???&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where feature type T is an gene, intron, editing island, etc. I think I should incorporate length since I will mainly be comparing genes or introns vs. editing islands. Also, I am not sure what to do about the editing islands being located within genes. How can I keep from counting the same motif twice?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas would be greatly appreciated. Thank you for your time. &lt;/p&gt;&#xA;" OwnerUserId="269" LastActivityDate="2018-03-08T12:08:47.270" Title="How to calculate statistical significance of sequence motifs" Tags="|statistics|motifs|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="334" PostTypeId="2" ParentId="326" CreationDate="2017-06-01T04:40:50.010" Score="5" Body="&lt;p&gt;I will answer one of the points – E.coli.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt; Bacteria, and in particular E.coli, are highly variable and there is usually no single best assembly. Large scale WGS studies should come with multiple assemblies for individual monophyletic clusters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Long answer:&lt;/strong&gt; Whereas a single reference sequence can make sense for human or mice (to a certain extent), bacteria are not the case. The variability between them even within a single species can be enormous and various lineages can contain very different genes. This is the reason why biologists usually distinguish a &lt;em&gt;core genome&lt;/em&gt; consisting of the genes shared among a vast majority (typically 95%) of individuals within a given phylogenetic clade, (i.e., of those genes which are essential for this clade – imagine a CPU and RAM in a computer), and an &lt;em&gt;accessory genome&lt;/em&gt; consisting of the other, unnecessary, genes – imagine a joystick in our analogy). The size of the accessory genome can vary a lot for different bacteria. For instance, Escherichia coli is said to have an &lt;em&gt;open&lt;/em&gt; accessory genome (i.e., much bigger than the core genome; imagine a Raspberry-PI and all the extension kits) whereas Chlamydia trachomatis has a &lt;em&gt;closed&lt;/em&gt; accessory genome (i.e., small compared to the core genome; imagine an ATM).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For species having a small accessory genome, a single reference might be sufficient. However, if a core genome is large, we can use either pan-genomes, or multiple references. The word pan-genome tends to be used for different things: the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/27769991&quot; rel=&quot;nofollow noreferrer&quot;&gt;computational biologists&lt;/a&gt; usually use it as a representation of all genomic content in a certain phylogenetic clade (which can be mathematically modeled using finite automata, hence as graphs) whereas some biologists define it as a concatenation of all (core and accessory) genes from the clade (without considering SNPs, etc.). Unfortunately, methods for graph references (especially for read mapping) are still not sufficiently developed, therefore, researchers still have to resort to reference sequence-based methods and use either concatenated genes or multiple reference sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When particular bacterial species are studied using massive whole-genome sequencing of many isolates, researchers usually cluster the isolates to monophyletic sequence clusters and infer a special reference sequence for each of these clusters (see, e.g., this &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3725542/&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt; for the pneumococcus). All of the inferred references should be then published together with the paper or in a separate data paper (see, e.g., the &lt;a href=&quot;http://datadryad.org/resource/doi:10.5061/dryad.t55gq/8&quot; rel=&quot;nofollow noreferrer&quot;&gt;sequences from the previous example&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Disclaimer:&lt;/strong&gt; I am not a biologist. If anything in this post is too simplified or incorrect, please, leave me a comment and I will try to fix it. I tried to make this text accessible for my younger self.&lt;/p&gt;&#xA;" OwnerUserId="425" LastEditorUserId="425" LastEditDate="2017-06-01T15:41:32.643" LastActivityDate="2017-06-01T15:41:32.643" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="335" PostTypeId="2" ParentId="230" CreationDate="2017-06-01T07:05:29.963" Score="1" Body="&lt;p&gt;One big advantage of Reactome, in my opinion, is its visualization using the web interface. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many pathways (in Reactome and KEGG) consist of genes / proteins that are up- and down-regulated through the respective pathway. If you do a simple overrepresentation analysis this is not taken into consideration. Therefore, you might end up seeing a pathway as &quot;overexpressed&quot; although only the down-regulated genes were observed more frequently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Reactome, you can zoom in on the different pathways very conveniently and then pick up these inconsistencies. I haven't really found a public database and tool that can take these different regulations into consideration. Therefore, you'll probably always will need some manual investigation of your data. In my opinion, this is easier with Reactome than with KEGG.&lt;/p&gt;&#xA;" OwnerUserId="476" LastActivityDate="2017-06-01T07:05:29.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="336" PostTypeId="1" AcceptedAnswerId="356" CreationDate="2017-06-01T07:27:25.760" Score="7" ViewCount="167" Body="&lt;p&gt;What is the current standard for imputing missing genotypes between two genotyping panels?  I have two populations genotyped using two different panels (A &amp;amp; B), and I would like to impute all the genotypes in population B for those positions on used in panel A.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've read the examples for impute2, and I think the closest thing to what I am looking for is &lt;a href=&quot;https://mathgen.stats.ox.ac.uk/impute/impute_v2.html#ex5&quot; rel=&quot;nofollow noreferrer&quot;&gt;this example&lt;/a&gt;, &quot;Imputation with one unphased reference panel&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simply put, I want to provide a list of SNPs, some variant file for population B, and haplotype information from 1,000 Genomes and get imputed genotypes for each SNP in the list.  Is impute2 the state of the art for this?&lt;/p&gt;&#xA;" OwnerUserId="59" LastEditorUserId="73" LastEditDate="2017-06-01T19:55:30.320" LastActivityDate="2017-06-01T21:36:32.873" Title="Imputing missing genotypes from separate genotyping panels" Tags="|imputation|impute2|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="338" PostTypeId="1" CreationDate="2017-06-01T07:33:33.863" Score="6" ViewCount="744" Body="&lt;p&gt;I find that in many contexts, the terms computational biology, bioinformatics and biostatistics are often treated as functionally equivalent, and yet for students selecting PhD programs and the like the difference could be quite significant. Is there a standard or rigorous definition of these terms and the difference between them?&lt;/p&gt;&#xA;" OwnerUserId="163" LastEditorUserId="93" LastEditDate="2017-06-03T17:33:30.207" LastActivityDate="2017-06-03T17:33:30.207" Title="Difference between computational biology, bioinformatics and biostatistics" Tags="|statistics|terminology|computation|" AnswerCount="0" CommentCount="6" FavoriteCount="0" ClosedDate="2017-06-01T08:33:53.843" ContentLicense="CC BY-SA 3.0" />
  <row Id="339" PostTypeId="2" ParentId="333" CreationDate="2017-06-01T08:22:35.933" Score="4" Body="&lt;p&gt;As far as I remember the exact probability computation is an open problem. The reason is that potential motifs can overlap, which makes probability computations for an arbitrary string non-trivial, and depends on the motif.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example if he have a binary string of four digits, the probability of &quot;01&quot; will be 11/16, while the probability of &quot;11&quot; will be 8/16=1/2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest approximation is to assume that probability of the motif on every position of the sequence is equal and independent. In &lt;a href=&quot;https://stepik.org/lesson/Some-Hidden-Messages-are-More-Surprising-than-Others-3/step/5?course=Bioinformatics-Algorithms&amp;amp;unit=7&quot; rel=&quot;nofollow noreferrer&quot;&gt;such case&lt;/a&gt; (this comes from &lt;a href=&quot;https://stepik.org/course/Bioinformatics-Algorithms-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;this course&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WZH86.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WZH86.png&quot; alt=&quot;Pr(N,A,Pattern,t)&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Probability of text of length N having at least t occurrences of a k-mer Pattern, A is the number of letters in the alphabet, n = N – t * k)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approximation is very rough, there are better &lt;a href=&quot;https://doi.org/10.1016/0097-3165(81)90005-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="191" LastActivityDate="2017-06-01T08:22:35.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="340" PostTypeId="2" ParentId="278" CreationDate="2017-06-01T09:01:04.053" Score="6" Body="&lt;p&gt;There are couple of computational methods which try to do this (I never used them, so no experience):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1093/bioinformatics/btt351&quot; rel=&quot;noreferrer&quot;&gt;CellMix&lt;/a&gt;, based on sets of marker gene lists&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://dx.doi.org/10.1186/1471-2105-12-258&quot; rel=&quot;noreferrer&quot;&gt;Subset Prediction from Enrichment Correlation&lt;/a&gt;, which is based on correlations with subset-specific genes across a set of samples.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://dx.doi.org/10.1186/1471-2164-13-460&quot; rel=&quot;noreferrer&quot;&gt;Cell type enrichment&lt;/a&gt;, which uses our highly expressed, cell specific gene database&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://dx.doi.org/10.1038/nmeth.1439&quot; rel=&quot;noreferrer&quot;&gt;Cell type-specific significance analysis&lt;/a&gt; using differential gene expression for each cell type&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You might have to get some reference expression levels from public databases or papers for some of the methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing to keep in mind: you cannot really compute the cells proportion, only RNA proportion. If you have a good reason to assume that RNA quantity &#xA;per cell is very similar, this is a good proxy for the cells proportion in a tissue.&lt;/p&gt;&#xA;" OwnerUserId="191" LastActivityDate="2017-06-01T09:01:04.053" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="341" PostTypeId="2" ParentId="322" CreationDate="2017-06-01T09:38:53.087" Score="11" Body="&lt;p&gt;It’s a matter of preference I guess but I recommend the &lt;a href=&quot;http://grch37.ensembl.org/index.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;Ensembl&lt;/strong&gt; builds&lt;/a&gt;. Decide whether you want the toplevel or primary assembly, and whether you want soft-masked, repeat-masked or unmasked files. The naming schema is very straightforward; the combinations are described in the &lt;a href=&quot;ftp://ftp.ensembl.org/pub/release-75/fasta/homo_sapiens/dna/README&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;README&lt;/code&gt; file&lt;/a&gt;, and all files &lt;a href=&quot;ftp://ftp.ensembl.org/pub/release-75/fasta/homo_sapiens/dna/&quot; rel=&quot;noreferrer&quot;&gt;reside in one directory&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if you want the unmasked primary assembly, the file to download would be &lt;a href=&quot;ftp://ftp.ensembl.org/pub/release-75/fasta/homo_sapiens/dna/Homo_sapiens.GRCh37.75.dna.primary_assembly.fa.gz&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;Homo_sapiens.GRCh37.75.dna.primary_assembly.fa.gz&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for &lt;strong&gt;GoldenPath/UCSC&lt;/strong&gt;, there’s no need to download and concatenate separate chromosomes (contrary to what the other answer said); you can download the whole (toplevel) reference &lt;a href=&quot;http://hgdownload.soe.ucsc.edu/goldenPath/hg19/bigZips/&quot; rel=&quot;noreferrer&quot;&gt;from the &lt;code&gt;bigZips&lt;/code&gt; directory&lt;/a&gt;; from the &lt;code&gt;README&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This directory contains the Feb. 2009 assembly of the human genome (hg19,&#xA;  GRCh37 Genome Reference Consortium Human Reference 37 (GCA_000001405.1)),&#xA;  as well as repeat annotations and GenBank sequences.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are essentially three options here:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;chromFa.tar.gz&lt;/code&gt;, which contains the whole genome in one chromosome per file;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;chromFaMasked.tar.gz&lt;/code&gt;, the same with repeats masked by &lt;code&gt;N&lt;/code&gt;;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;hg19.2bit&lt;/code&gt;, which is the whole genome in one file, but needs to be extracted using the utility program &lt;code&gt;twoBitToFa&lt;/code&gt;, which needs to be downloaded &lt;a href=&quot;http://hgdownload.cse.ucsc.edu/admin/exe&quot; rel=&quot;noreferrer&quot;&gt;separately&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In any case, I always download the reference and build my own index for mapping, since this allows me more control; not everybody might need this much control, but then building the index once is fairly fast anyway.&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2017-06-01T09:49:08.560" LastActivityDate="2017-06-01T09:49:08.560" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="342" PostTypeId="2" ParentId="327" CreationDate="2017-06-01T09:56:56.927" Score="12" Body="&lt;p&gt;&lt;a href=&quot;http://multiqc.info/&quot; rel=&quot;noreferrer&quot;&gt;MultiQC&lt;/a&gt; can merge all your different reports into a single one.&#xA;Which could be useful once you manage to know which QC tools to use.&lt;/p&gt;&#xA;" OwnerUserId="485" LastActivityDate="2017-06-01T09:56:56.927" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="343" PostTypeId="1" CreationDate="2017-06-01T11:11:09.893" Score="19" ViewCount="3027" Body="&lt;p&gt;I’m using the &lt;a href=&quot;http://www.girinst.org/server/RepBase/index.php&quot; rel=&quot;noreferrer&quot;&gt;RepBase libraries&lt;/a&gt; in conjunction with RepeatMasker to get genome-wide repeat element annotations, in particular for transposable elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This works well enough, and seems to be the de facto standard in the field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are two issues with the use of RepBase, which is why I (and others) have been looking for alternatives (so far without success):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.girinst.org/accountservices/register.php?commercial=0&quot; rel=&quot;noreferrer&quot;&gt;RepBase isn’t open data&lt;/a&gt;. Their academic license agreement includes a clause that &lt;em&gt;explicitly forbids dissemination of data derived from RepBase&lt;/em&gt;. It’s unclear to what extent this is binding/enforceable, but it effectively prevents publishing at least some of the data I’m using and generating. This is unacceptable for &lt;a href=&quot;https://en.wikipedia.org/wiki/Open_science&quot; rel=&quot;noreferrer&quot;&gt;open science&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Subordinate to this, the subscription model of RepBase also makes it impossible to integrate RepBase into fully automated pipelines, because user interaction is required to subscribe to RepBase, and to provide the login credentials.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;RepBase is heavily manually curated. This is both good and bad. Good, because manual curation of sequence data is often the most reliable form of curation. On the flip side, manual curation is inherently biased; and worse, it’s hard to quantify this bias — &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-7-474&quot; rel=&quot;noreferrer&quot;&gt;this is acknowledged by the RepBase maintainers&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2017-12-11T17:16:03.307" LastActivityDate="2019-02-27T18:42:54.970" Title="Are there any RepBase alternatives for genome-wide repeat element annotations?" Tags="|sequence-annotation|database|repeat-elements|transposable-elements|" AnswerCount="6" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="344" PostTypeId="1" AcceptedAnswerId="345" CreationDate="2017-06-01T11:15:55.563" Score="13" ViewCount="4011" Body="&lt;p&gt;What are the key differences between VCF versions 4.1 and 4.2?&#xA; It looks like v4.3 contains a changelog (specs available &lt;a href=&quot;http://samtools.github.io/hts-specs/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;) but earlier specifications do not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/211152/&quot; rel=&quot;noreferrer&quot;&gt;This&lt;/a&gt; biostar post points out one difference: the introduction of &lt;code&gt;Number=R&lt;/code&gt; for fields with one value per allele including REF — can anyone enumerate the other changes between these two versions?&lt;/p&gt;&#xA;" OwnerUserId="61" LastEditorUserId="96" LastEditDate="2017-06-08T08:47:34.387" LastActivityDate="2017-06-08T08:47:34.387" Title="What's the difference between VCF spec versions 4.1 and 4.2?" Tags="|vcf|htslib|file-formats|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="345" PostTypeId="2" ParentId="344" CreationDate="2017-06-01T11:40:33.550" Score="16" Body="&lt;p&gt;This is easy to check, you can download both specs in .tex format and do &lt;code&gt;diff&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Changes to the v4.2 compared to v4.1:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Information field format: adding source and version as recommended fields.&lt;/li&gt;&#xA;&lt;li&gt;INFO field can have one value for each possible allele (code &lt;code&gt;R&lt;/code&gt;).&lt;/li&gt;&#xA;&lt;li&gt;For all of the ##INFO, ##FORMAT, ##FILTER, and ##ALT metainformation, extra fields can be included after the default fields.&lt;/li&gt;&#xA;&lt;li&gt;Alternate base (ALT) can include &lt;code&gt;*&lt;/code&gt;: missing due to a upstream deletion.&lt;/li&gt;&#xA;&lt;li&gt;Quality scores, a sentence removed: &lt;em&gt;High QUAL scores indicate high confidence calls. Although traditionally people use integer phred scores, this field is permitted to be a floating point to enable higher resolution for low confidence calls if desired.&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;Examples changed a bit.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="191" LastActivityDate="2017-06-01T11:40:33.550" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="346" PostTypeId="2" ParentId="266" CreationDate="2017-06-01T11:52:11.200" Score="1" Body="&lt;p&gt;mauve documentation have a similar command but the usage is &lt;code&gt;--id-matrix=&amp;lt;file&amp;gt;&lt;/code&gt;. So very similar to the progressivemauve command (just doesn't include --input). Try it and see if it works. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it's a bug for sure as several other people on biostar have complained. &lt;/p&gt;&#xA;" OwnerUserId="490" LastActivityDate="2017-06-01T11:52:11.200" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="347" PostTypeId="2" ParentId="343" CreationDate="2017-06-01T12:09:58.040" Score="5" Body="&lt;p&gt;You could use &lt;a href=&quot;http://bix.ucsd.edu/repeatscout/&quot; rel=&quot;noreferrer&quot;&gt;RepeatScout&lt;/a&gt;, which has defined repeat libraries for a limited number of species (including human, mouse, and rat). If your taxon is not represented, you can also do de novo repeat prediction with RepeatScout to build your own library to feed to RepeatMasker. The &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.517.8206&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;noreferrer&quot;&gt;RepeatScout publication&lt;/a&gt; includes some comparisons with RepBase. Another related tool is &lt;a href=&quot;http://www.repeatmasker.org/RepeatModeler/&quot; rel=&quot;noreferrer&quot;&gt;RepeatModeler&lt;/a&gt;, which wraps RepeatScout with &lt;a href=&quot;http://eddylab.org/software/recon/&quot; rel=&quot;noreferrer&quot;&gt;RECON&lt;/a&gt; and some other programs, and shares authors with the RepeatMasker team.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the plus side RepeatScout/RepeatModeler are &lt;a href=&quot;https://github.com/mmcco/RepeatScout&quot; rel=&quot;noreferrer&quot;&gt;open source&lt;/a&gt; and do not use manual curation, meeting your criteria. On the negative, I'm not sure exactly how RepeatModeler and the component tools are maintained. The RepeatScout web and github pages have not been updated for several years, although the RepeatModeler page shows its latest release was in 2017. Anyway, I know that some combination of RepeatScout/RepeatModeler have been used to annotate repeats for some &lt;em&gt;fairly recent&lt;/em&gt; newly sequenced genomes, e.g. for &lt;a href=&quot;https://www.nature.com/nature/journal/v513/n7518/full/nature13726.html&quot; rel=&quot;noreferrer&quot;&gt;cichlids&lt;/a&gt;, &lt;a href=&quot;https://www.nature.com/nature/journal/v496/n7445/full/nature12027.html&quot; rel=&quot;noreferrer&quot;&gt;coelacanth&lt;/a&gt;, and &lt;a href=&quot;https://bmcgenomics.biomedcentral.com/articles/10.1186/1471-2164-14-95&quot; rel=&quot;noreferrer&quot;&gt;Darwin's finch&lt;/a&gt;, so I think it's fair to say this kind of approach is accepted in the field, at least for vertebrate genome projects.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2017-06-01T12:09:58.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="348" PostTypeId="2" ParentId="343" CreationDate="2017-06-01T12:15:47.873" Score="14" Body="&lt;p&gt;Dfam has recently launched a sister resource, &lt;a href=&quot;http://www.dfam-consensus.org/&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;Dfam_consensus&lt;/strong&gt;&lt;/a&gt;, whose &lt;a href=&quot;https://xfam.wordpress.com/2017/05/18/introducing-dfam_consensus-dfams-consensus-sequence-twin/&quot; rel=&quot;noreferrer&quot;&gt;stated aim&lt;/a&gt; is to replace RepBase. From the annoucement:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Dfam_consensus provides an open framework for the community to store both seed alignments (multiple alignments of instances for a given family) and the corresponding consensus sequence model.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Both RepeatMasker and RepeatModeler have been updated to support Dfam_consensus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven’t tried it yet but it looks promising.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2017-06-01T12:15:47.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="349" PostTypeId="2" ParentId="278" CreationDate="2017-06-01T12:48:19.277" Score="4" Body="&lt;p&gt;Cell deconvolution is mentioned in &lt;a href=&quot;https://www.biostars.org/p/160961/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Biostars post&lt;/a&gt;, which mentions CIBERSORT for immune cell mixes, and the Bioconductor package &lt;a href=&quot;http://www.bioconductor.org/packages/release/bioc/html/DeconRNASeq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeconRNASeq&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I'm aware, it is only possible at best to get proportional representation for transcript expression from standard high-throughput sequencing results, because the sequencers and sample preparation workflow are designed in such a way that the same number of reads are output regardless of the input amount.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-01T12:48:19.277" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="350" PostTypeId="1" AcceptedAnswerId="353" CreationDate="2017-06-01T13:04:41.200" Score="12" ViewCount="482" Body="&lt;p&gt;Models of structures deposited in the Protein Data Bank vary in the quality, depending both on the data quality and expertise and patience of the person who built the model. Is there a well-accepted subset of the PDB entries that has only &quot;high quality&quot; structures? Ideally these structures would be representative for classes of proteins in the whole PDB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;based on a &lt;a href=&quot;https://biology.stackexchange.com/questions/56946/subset-of-protein-crystal-structures-from-pdb/&quot;&gt;real question&lt;/a&gt; from biology.SE&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2017-06-20T08:27:16.083" Title="How to select high quality structures from the Protein Data Bank?" Tags="|protein-structure|pdb|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="351" PostTypeId="1" AcceptedAnswerId="420" CreationDate="2017-06-01T13:08:19.863" Score="8" ViewCount="1079" Body="&lt;p&gt;Can anyone recommend a good tool for estimating the tumor content given a matched tumor and normal file for DNA NGS whole genome sequencing data or whole exome data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to estimate this without a normal sample as well?&lt;/p&gt;&#xA;" OwnerUserId="374" LastEditorUserId="374" LastEditDate="2017-06-02T08:06:06.787" LastActivityDate="2017-06-04T13:37:19.093" Title="Tumor purity/contamination/admixture estimation" Tags="|cancer|wgs|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="352" PostTypeId="2" ParentId="351" CreationDate="2017-06-01T13:38:07.977" Score="2" Body="&lt;p&gt;It's usually CNV callers that make use of Tumour/Normal WGS pairs to estimate purity. It can also be done with WES (exome) Tumour/Normal pairs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several tools out there, I have some experience with the one written by Illumina (public on Github):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/Illumina/canvas&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Illumina/canvas&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It requires realigning things with bowtie2, so I don't think it can take existing data aligned with other aligners. Not sure why.&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2017-06-01T13:38:07.977" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="353" PostTypeId="2" ParentId="350" CreationDate="2017-06-01T13:55:32.567" Score="9" Body="&lt;p&gt;There is a very nice database, &lt;a href=&quot;http://dunbrack.fccc.edu/Guoli/pisces_download.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;pdbcull&lt;/a&gt; (also known as the PISCES server in the literature). It filters the PDB for high resolution and reduced sequence identity. It also seems to be updated regularly. Depending on the cut-offs, you get between 3000 and 35000 structures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are specifically interested in rotamers, you may want to look at &lt;a href=&quot;http://kinemage.biochem.duke.edu/databases/top8000.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;top8000&lt;/a&gt; instead, where they have checked for high resolution, and good MolProbity scores. They also provide a rotamer database.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PDB also provides &lt;a href=&quot;http://www.rcsb.org/pdb/statistics/clusterStatistics.do&quot; rel=&quot;nofollow noreferrer&quot;&gt;their own clustering.&lt;/a&gt; They first cluster the sequences, and then extract a representative structure for each one, based on the quality factor (&lt;code&gt;1/resolution - R_value&lt;/code&gt;). This has the advantage of being comprehensive, but you will have bad structures when no good ones were ever obtained.&lt;/p&gt;&#xA;" OwnerUserId="501" LastEditorUserId="501" LastEditDate="2017-06-20T08:27:16.083" LastActivityDate="2017-06-20T08:27:16.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="355" PostTypeId="2" ParentId="329" CreationDate="2017-06-01T15:11:07.053" Score="2" Body="&lt;p&gt;There is an evaluation of PB Honey and Sniffles algorithms for low coverage PacBio datasets in &lt;a href=&quot;http://www.biorxiv.org/content/early/2016/12/17/092544.full.pdf+html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this preprint&lt;/a&gt; and another evaluation is shown on &lt;a href=&quot;http://www.pacb.com/wp-content/uploads/2016-ashg-poster-andrew-carroll-dnanexus.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this poster&lt;/a&gt;. Both reports agree that optimal is (surprisingly) combination of &lt;a href=&quot;https://sourceforge.net/projects/pb-jelly/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PB Honey&lt;/a&gt; and &lt;a href=&quot;https://github.com/fritzsedlazeck/Sniffles&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sniffles&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Author of Sniffles have benchmarked Sniffles against PB Honey, where he shown that Sniffles performs significantly better. Take a look on &lt;a href=&quot;http://schatzlab.cshl.edu/presentations/2016/2016.10.28.BIODATA.PacBioSV.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this presentation (slide 15)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option is &lt;a href=&quot;https://github.com/EichlerLab/pacbio_variant_caller&quot; rel=&quot;nofollow noreferrer&quot;&gt;SMRT-SV&lt;/a&gt;, but I am not aware of any benchmarking.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2017-06-01T15:11:07.053" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="356" PostTypeId="2" ParentId="336" CreationDate="2017-06-01T15:22:21.113" Score="4" Body="&lt;p&gt;Given that you mention wanting to use 1000 Genomes as a reference panel for imputing genotypes into your two SNP chip panels, I am going to assume that you are working with human data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that case there are several options you can go with:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If your two panels are of European descent, then you are probably best off using the &lt;a href=&quot;http://www.haplotype-reference-consortium.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HRC reference panel&lt;/a&gt; together with a fast genotype imputation tool such as &lt;a href=&quot;http://www.cell.com/ajhg/fulltext/S0002-9297(15)00491-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;Beagle 4.1&lt;/a&gt; to impute genotypes in each of your two SNP chip panels separately.&lt;/li&gt;&#xA;&lt;li&gt;If your panels are not of European descent, then you will likely want to use the 1000 Genomes phase 3 reference panel with Beagle 4.1, Impute2, or Minimac3.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In either case, there are two phasing services available that will do much of the heavy lifting for you &lt;a href=&quot;https://imputationserver.sph.umich.edu/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;,&lt;a href=&quot;https://imputation.sanger.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second &lt;a href=&quot;https://www.wtccc.org.uk/ccc2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wellcome Trust Case-Control Consortium paper&lt;/a&gt; performed a cross-imputation analysis as you describe.  I don't see many studies using multiple SNP chip panels.  You will need to take care in your analysis that you are not hit by batch effects from using two different SNP chip panels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, none of these methods will work if the region you are imputing into has too few variants.  I'm not sure what the minimum number of variants is, but if you are using a whole genome genotyping panel of at least 500k SNPs, then you should be ok if you impute a whole chromosome at a time.&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="492" LastEditDate="2017-06-01T21:36:32.873" LastActivityDate="2017-06-01T21:36:32.873" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="357" PostTypeId="1" AcceptedAnswerId="359" CreationDate="2017-06-01T15:29:48.250" Score="9" ViewCount="1162" Body="&lt;p&gt;As someone who's beginning to delve into bioinformatics, I'm noticing that like biology there are industry standards here, similar to Illumina in genomics and bowtie for alignment, many people use bash as shell. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is using a shell besides bash going to cause issues for me?&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="77" LastEditDate="2017-06-06T09:09:06.957" LastActivityDate="2018-03-02T00:21:03.057" Title="Using shells other than bash" Tags="|linux|shell|" AnswerCount="5" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="358" PostTypeId="2" ParentId="357" CreationDate="2017-06-01T15:44:33.430" Score="5" Body="&lt;p&gt;I would not say bash as a &quot;standard&quot;, but it is indeed likely to be the most widely used unix shell and available by default on most modern unix/linux distros. There are a few other more convenient shells like &lt;a href=&quot;http://www.zsh.org&quot; rel=&quot;noreferrer&quot;&gt;zsh&lt;/a&gt; that are broadly compatible with &lt;code&gt;/bin/sh&lt;/code&gt;, but they are not as widely available. There is also C-shell and in particular its open-source implementation &lt;a href=&quot;https://en.wikipedia.org/wiki/Tcsh&quot; rel=&quot;noreferrer&quot;&gt;tcsh&lt;/a&gt;. C-shell is quite different from bash. Over ten years ago, I saw it was used from time to time, but nowadays, I rarely see its use, except by programmers from older generations.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-06-01T15:44:33.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="359" PostTypeId="2" ParentId="357" CreationDate="2017-06-01T15:53:21.003" Score="18" Body="&lt;p&gt;Bioinformatics tools written in shell and other shell scripts generally specify the shell they want to use (via &lt;code&gt;#!/bin/sh&lt;/code&gt; or e.g. &lt;code&gt;#!/bin/bash&lt;/code&gt; if it matters), so won't be affected by your choice of user shell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are writing significant shell scripts yourself, there are reasons to do it in a Bourne-style shell.  See &lt;em&gt;Csh Programming Considered Harmful&lt;/em&gt; and other essays/polemics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A Bourne-style shell is pretty much the industry standard, and if you choose a substantially different shell you'll have to translate some of the documentation of your bioinformatics tools.  It's not uncommon to have things like&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Set some variables pointing at reference data and add the script to your PATH to run it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;export FOO_REF=/path/to/stuff&#xA;export PATH=/path/to/foo-x.y:$PATH&#xA;foo blah blah&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;These will typically be shown in Bourne-shell syntax.  By using a different shell you have to translate the &lt;code&gt;export&lt;/code&gt; commands to your local syntax, and especially PATH munging is somewhat shell-dependent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're experienced in Unix, this will be only a minor niggle.  If you're a beginner, IMHO this will add a non-negligible amount of friction on top of all the other things you're learning.&lt;/p&gt;&#xA;" OwnerUserId="134" LastActivityDate="2017-06-01T15:53:21.003" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="360" PostTypeId="2" ParentId="357" CreationDate="2017-06-01T15:59:22.890" Score="11" Body="&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SH adheres to an official industry standard, but it is not suitable for scientific computing. Bash is considered an informal standard (e.g., by Google). Bash 3 is preferable in most of situations in the world of bioinformatics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Long answer&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As already described in other answers, SH (&lt;code&gt;/bin/sh&lt;/code&gt;, plain Bourne shell, the original UNIX shell) should fully adhere to POSIX which is a real industry standard. However, SH is too limited for scientific computing since many key features were incorporated later in SH successors, especially in Bash (&lt;code&gt;/bin/bash&lt;/code&gt;, Bourne Again Shell): &lt;code&gt;set -o pipefail&lt;/code&gt;, &lt;code&gt;[[ ... ]]&lt;/code&gt;, or process substitutions &lt;code&gt;&amp;lt; ()&lt;/code&gt; to name at least few.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, it is much more difficult to write &quot;safe&quot; scripts in pure SH and only shell experts are usually capable to prevent unexpected behavior. For example, it may be hard to ensure that no command in a pipeline failed in the middle of computation. For Bash, various easy-to-follow defensive-programming recommendations have been developed and they should prevent all these problems. From this reason, many computer scientists, software engineers and companies use Bash as a kind of a standard. For instance, Google internal policy allows &lt;a href=&quot;https://google.github.io/styleguide/shell.xml?showone=Which_Shell_to_Use#Which_Shell_to_Use&quot; rel=&quot;nofollow noreferrer&quot;&gt;only Bash&lt;/a&gt; for writing any shell scripts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even though we cannot expect that Bash is present on completely every Unix machine (e.g., on mobile devices as @terdon pointed out), a vast majority of *nix machines used for scientific computation should have it. We also should be aware of the fact that Bash can be slower than SH and that it has recently suffered from  major &lt;a href=&quot;https://www.symantec.com/connect/blogs/shellshock-all-you-need-know-about-bash-bug-vulnerability&quot; rel=&quot;nofollow noreferrer&quot;&gt;security issues&lt;/a&gt;. Moreover, various Bash versions exist and scripts working on modern Linux machines with &lt;a href=&quot;http://wiki.bash-hackers.org/bash4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bash 4&lt;/a&gt; might not work on OS X, which is still based on Bash 3. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To sum up, Bash 3 is probably be the most reasonable choice for scientific computing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I addressed the comments from @terdon and @John Marshall. In particular, I added an explanation why Bash is more suitable for scientific computing than SH (in my opinion).&lt;/p&gt;&#xA;" OwnerUserId="425" LastEditorUserId="425" LastEditDate="2018-03-02T00:21:03.057" LastActivityDate="2018-03-02T00:21:03.057" CommentCount="15" ContentLicense="CC BY-SA 3.0" />
  <row Id="361" PostTypeId="1" AcceptedAnswerId="369" CreationDate="2017-06-01T16:47:02.727" Score="21" ViewCount="6692" Body="&lt;p&gt;I used to work with publicly available genomic references, where basic statistics are usually available and if they are not, you have to compute them only once so there is no reason to worry about performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recently I started sequencing project of couple of different species with mid-sized genomes (~Gbp) and during testing of different assembly pipelines I had compute number of unknown nucleotides many times in both raw reads (in fastq) and assembly scaffolds (in fasta), therefore I thought that I would like to optimize the computation.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For me it is reasonable to expect 4-line formatted fastq files, but general solution is still prefered&lt;/li&gt;&#xA;&lt;li&gt;It would be nice if solution would work on gzipped files as well&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Q : What is the fastest way (performance-wise) to compute the number of unknown nucleotides (Ns) in fasta and fastq files?&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2017-06-02T08:28:52.683" LastActivityDate="2020-08-27T15:33:15.750" Title="What is the fastest way to calculate the number of unknown nucleotides in FASTA / FASTQ files?" Tags="|fasta|fastq|benchmarking|" AnswerCount="10" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="362" PostTypeId="2" ParentId="326" CreationDate="2017-06-01T16:50:34.150" Score="0" Body="&lt;p&gt;The &quot;best&quot; assembly depends on the species. For more common species, Ensembl/UCSC/NCBI has their own version that corresponds to the most popular assembly. Usually the actual genetic sequence is identical across all of them, but the chromosome names and gene annotations will vary. If those three sources agree for a species, there is a &quot;standard&quot; reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For fruit fly specifically, FlyBase Consortium/Berkeley Drosophila Genome Project is the gold standard. Other species have their own organizations.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2017-06-01T16:50:34.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="363" PostTypeId="2" ParentId="361" CreationDate="2017-06-01T17:00:25.523" Score="8" Body="&lt;p&gt;&lt;strong&gt;FASTQ&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As it was pointed out, fastq can be complicated. But in a simple case when you have four lines per record, one possible solution in bash is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -n '2~4p' seqs.fastq | grep -io N | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;sed -n '2~4p'&lt;/code&gt; will print every fourth line&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;grep -o N&lt;/code&gt; will output a line with &lt;code&gt;N&lt;/code&gt; for every matching symbol&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;wc -l&lt;/code&gt; will count the lines&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I suspect this python approach will work faster:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat seqs.fastq | python3 -c &quot;import sys; print(sum(line.upper().count('N') for line in sys.stdin))&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;FASTA&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coreutils:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -v &quot;^&amp;gt;&quot; seqs.fasta | grep -io N | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Python:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat seqs.fasta | python3 -c &quot;import sys; print(sum(line.upper().count('N') for line in sys.stdin if not line.startswith('&amp;gt;')))&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="191" LastEditorUserId="298" LastEditDate="2017-06-02T13:41:57.900" LastActivityDate="2017-06-02T13:41:57.900" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="364" PostTypeId="2" ParentId="361" CreationDate="2017-06-01T17:10:02.993" Score="5" Body="&lt;p&gt;Honestly, the easiest way (especially for FASTQ) is probably to use a dedicated parsing library, such as R/Bioconductor:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;suppressMessages(library(ShortRead))&#xA;&#xA;seq = readFasta(commandArgs(TRUE)[1]) # or readFastq&#xA;cat(colSums(alphabetFrequency(sread(seq))[, 'N', drop = FALSE]), '\n')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This may not be the fastest, but it’s &lt;em&gt;pretty fast&lt;/em&gt;, since the relevant methods are highly optimised. The biggest overhead is actually probably from loading &lt;code&gt;ShortRead&lt;/code&gt;, which is an unjustifiable slog.&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2017-06-02T14:53:35.553" LastActivityDate="2017-06-02T14:53:35.553" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="365" PostTypeId="2" ParentId="361" CreationDate="2017-06-01T17:15:22.907" Score="5" Body="&lt;p&gt;If it is raw speed you're after, then writing an own little C/C++ program is probably what you need to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fortunately, the worst part (a fast and reliable parser) has already been tackled: the &lt;a href=&quot;https://github.com/lh3/readfq&quot; rel=&quot;noreferrer&quot;&gt;readfq&lt;/a&gt; from Heng Li is probably the fastest FASTA/FASTQ parser around.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And it's easy to use, the example on GitHub can easily be expanded to do what you need. Just add in a parameter parser (for the filename you want to analyse), program a simple 'N'-counter loop and have the results printed to stdout. Done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Else, if it is simplicity you are after, see Konrad's answer for an R based solution. Or a very simple script using Biopython.&lt;/p&gt;&#xA;" OwnerUserId="44" LastActivityDate="2017-06-01T17:15:22.907" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="366" PostTypeId="2" ParentId="361" CreationDate="2017-06-01T17:22:20.160" Score="10" Body="&lt;p&gt;I think that this should be pretty fast:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FASTA:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -v &quot;^&amp;gt;&quot; seqs.fa | tr -cd N | wc -c&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;FASTQ:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -n '1d;N;N;N;P;d' seqs.fq | tr -cd N | wc -c&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;See this &lt;a href=&quot;https://stackoverflow.com/a/16679640/4641846&quot;&gt;answer on SO&lt;/a&gt; about how to count characters in BASH using different approaches.&lt;/p&gt;&#xA;" OwnerUserId="425" LastActivityDate="2017-06-01T17:22:20.160" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="367" PostTypeId="1" AcceptedAnswerId="795" CreationDate="2017-06-01T19:49:22.177" Score="20" ViewCount="8054" Body="&lt;p&gt;According to &lt;a href=&quot;https://haroldpimentel.wordpress.com/2014/05/08/what-the-fpkm-a-review-rna-seq-expression-units/&quot; rel=&quot;noreferrer&quot;&gt;this famous blog post&lt;/a&gt;, the effective transcript length is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\tilde{l}_i = l_i - \mu$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $l_i$ is the length of transcript and $\mu$ is the average fragment length. However, typically fragment length is about 300bp. What if when the transcript $l_i$ is smaller than 300? How do you compute the effective length in this case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A related question: when computing the FPKM of a gene, how to choose a transcript? Do we choose a &quot;canonical&quot; transcript (how?) or combine the signals from all transcripts to a gene-level FPKM?&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="298" LastEditDate="2017-07-26T16:06:30.160" LastActivityDate="2017-07-26T16:15:39.647" Title="How exactly is &quot;effective length&quot; used in FPKM calculated?" Tags="|rna-seq|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="368" PostTypeId="2" ParentId="367" CreationDate="2017-06-01T20:03:41.287" Score="15" Body="&lt;p&gt;The effective length is $\tilde{l}_i = l_i - \mu + 1$ (note the R code at the bottom of Harold's blog post), which in the case of $\mu &amp;lt; l_i$ should be 1. Ideally, you'd use the mean fragment length mapped to the particular feature, rather than a global $\mu$, but that's a lot more work for likely 0 benefit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding choosing a particular transcript, ideally one would use a method like salmon or kallisto (or RSEM if you have time to kill). Otherwise, your options are (A) choose the major isoform (if it's known in your tissue and condition) or (B) use a &quot;union gene model&quot; (sum the non-redundant exon lengths) or (C) take the median transcript length. None of those three options make much of a difference if you're comparing between samples, though they're all inferior to a salmon/kallisto/etc. metric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why are salmon et al. better methods? They don't use arbitrary metrics that will be the same across samples to determine the feature length. Instead, they use expectation maximization (or similarish, since at least salmon doesn't actually use EM) to quantify individual isoform usage. The effective gene length in a sample is then the average of the transcript lengths after weighting for their relative expression (yes, one should remove $\mu$ in there). This can then vary between samples, which is quite useful if you have isoform switching between samples/groups in such a way that methods A-C above would miss (think of cases where the switch is to a smaller transcript with higher coverage over it...resulting in the coverage/length in methods A-C to be tamped down).&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="29" LastEditDate="2017-07-26T16:04:12.083" LastActivityDate="2017-07-26T16:04:12.083" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="369" PostTypeId="2" ParentId="361" CreationDate="2017-06-01T21:46:22.750" Score="26" Body="&lt;p&gt;For FASTQ:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seqtk fqchk in.fq | head -2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It gives you percentage of &quot;N&quot; bases, not the exact count, though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For FASTA:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;seqtk comp in.fa | awk '{x+=$9}END{print x}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This command line also works with FASTQ, but it will be slower as awk is slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: ok, based on @BaCH's reminder, here we go (you need &lt;a href=&quot;https://github.com/lh3/readfq/blob/master/kseq.h&quot; rel=&quot;noreferrer&quot;&gt;kseq.h&lt;/a&gt; to compile):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;// to compile: gcc -O2 -o count-N this-prog.c -lz&#xA;#include &amp;lt;zlib.h&amp;gt;&#xA;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;stdint.h&amp;gt;&#xA;#include &quot;kseq.h&quot;&#xA;KSEQ_INIT(gzFile, gzread)&#xA;&#xA;unsigned char dna5tbl[256] = {&#xA;    0, 1, 2, 3,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4, &#xA;    4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4, &#xA;    4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 5, 4, 4,&#xA;    4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4, &#xA;    4, 0, 4, 1,  4, 4, 4, 2,  4, 4, 4, 4,  4, 4, 4, 4, &#xA;    4, 4, 4, 4,  3, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4, &#xA;    4, 0, 4, 1,  4, 4, 4, 2,  4, 4, 4, 4,  4, 4, 4, 4, &#xA;    4, 4, 4, 4,  3, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4, &#xA;    4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4, &#xA;    4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4, &#xA;    4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4, &#xA;    4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4, &#xA;    4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4, &#xA;    4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4, &#xA;    4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4, &#xA;    4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4&#xA;};&#xA;&#xA;int main(int argc, char *argv[]) {&#xA;    long i, n_n = 0, n_acgt = 0, n_gap = 0;&#xA;    gzFile fp;&#xA;    kseq_t *seq;&#xA;    if (argc == 1) {&#xA;        fprintf(stderr, &quot;Usage: count-N &amp;lt;in.fa&amp;gt;\n&quot;);&#xA;        return 1;&#xA;    }&#xA;    if ((fp = gzopen(argv[1], &quot;r&quot;)) == 0) {&#xA;        fprintf(stderr, &quot;ERROR: fail to open the input file\n&quot;);&#xA;        return 1;&#xA;    }&#xA;    seq = kseq_init(fp);&#xA;    while (kseq_read(seq) &amp;gt;= 0) {&#xA;        for (i = 0; i &amp;lt; seq-&amp;gt;seq.l; ++i) {&#xA;            int c = dna5tbl[(unsigned char)seq-&amp;gt;seq.s[i]];&#xA;            if (c &amp;lt; 4) ++n_acgt;&#xA;            else if (c == 4) ++n_n;&#xA;            else ++n_gap;&#xA;        }&#xA;    }&#xA;    kseq_destroy(seq);&#xA;    gzclose(fp);&#xA;    printf(&quot;%ld\t%ld\t%ld\n&quot;, n_acgt, n_n, n_gap);&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It works for both FASTA/Q and gzip'ed FASTA/Q. The following uses SeqAn:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;seqan/seq_io.h&amp;gt;&#xA;&#xA;using namespace seqan;&#xA;&#xA;int main(int argc, char *argv[]) {&#xA;    if (argc == 1) {&#xA;        std::cerr &amp;lt;&amp;lt; &quot;Usage: count-N &amp;lt;in.fastq&amp;gt;&quot; &amp;lt;&amp;lt; std::endl;&#xA;        return 1;&#xA;    }&#xA;    std::ios::sync_with_stdio(false);&#xA;    CharString id;&#xA;    Dna5String seq;&#xA;    SeqFileIn seqFileIn(argv[1]);&#xA;    long i, n_n = 0, n_acgt = 0;&#xA;    while (!atEnd(seqFileIn)) {&#xA;        readRecord(id, seq, seqFileIn);&#xA;        for (i = beginPosition(seq); i &amp;lt; endPosition(seq); ++i)&#xA;            if (seq[i] &amp;lt; 4) ++n_acgt;&#xA;            else ++n_n;&#xA;    }&#xA;    std::cout &amp;lt;&amp;lt; n_acgt &amp;lt;&amp;lt; '\t' &amp;lt;&amp;lt; n_n &amp;lt;&amp;lt; std::endl;&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;On a FASTQ with 4-million 150bp reads:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The C version: ~0.74 sec&lt;/li&gt;&#xA;&lt;li&gt;The C++ version: ~2.15 sec&lt;/li&gt;&#xA;&lt;li&gt;An older C version without a lookup table (see &lt;a href=&quot;https://bioinformatics.stackexchange.com/posts/369/revisions&quot;&gt;the previous edit&lt;/a&gt;): ~2.65 sec&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2017-06-09T01:02:55.120" LastActivityDate="2017-06-09T01:02:55.120" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="370" PostTypeId="2" ParentId="321" CreationDate="2017-06-01T22:07:27.130" Score="5" Body="&lt;p&gt;I personally don't think BQSR has a huge impact on variant calling, but you don't really need to guess. If you run GATK BQSR, it outputs a table and charts of exactly how much quality scores are adjusted. The adjustment will vary depending on the position in the read and genomic context (previous and following base). In my experience, the difference is a few points at most, but it's certainly noticeable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GATK recommends BQSR for both genome and exome data, which is normally much higher than 15x.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2017-06-01T22:07:27.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="371" PostTypeId="2" ParentId="361" CreationDate="2017-06-01T22:20:21.293" Score="18" Body="&lt;p&gt;5 hours and no benchmarks posted? I am sorely disappointed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll restrict the comparison to just be fasta files, since fastq will end up being the same. So far, the contenders are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;R with the &lt;code&gt;ShortRead&lt;/code&gt; package (even if not the fastest, certainly a super convenient method).&lt;/li&gt;&#xA;&lt;li&gt;A pipeline of &lt;code&gt;grep -v &quot;^&amp;gt;&quot; | tr -cd A | wc -c&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;A pipeline of &lt;code&gt;grep -v &quot;^&amp;gt;&quot; | grep -io A | wc -l&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;A pipeline of &lt;code&gt;seqtk comp | awk&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Something custom in C/C++&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The R code can't actually count N records for some reason when I try it, so I counted &lt;code&gt;A&lt;/code&gt; for everything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll exclude all python solutions, because there's no conceivable universe in which python is fast. For the C/C++ solution I just used what @user172818 posted (everything I tried writing took the same amount of time).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Repeating 100 times and taking the average (yes, one should take the median):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;1.7 seconds&lt;/li&gt;&#xA;&lt;li&gt;0.65 seconds&lt;/li&gt;&#xA;&lt;li&gt;15 seconds&lt;/li&gt;&#xA;&lt;li&gt;1.2 seconds&lt;/li&gt;&#xA;&lt;li&gt;0.48 seconds&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Unsurprisingly, anything in straight C or C++ is going to win. &lt;code&gt;grep&lt;/code&gt; with &lt;code&gt;tr&lt;/code&gt; is surprisingly good, which surprises me since even though &lt;code&gt;grep&lt;/code&gt; is very very optimized I still expected it to have more overhead. Piping to &lt;code&gt;grep -io&lt;/code&gt; is a performance killer. The R solution is surprisingly good given the typical R overhead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update1:&lt;/strong&gt; As suggested in the comments&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;6&quot;&gt;&#xA;&lt;li&gt;&lt;code&gt;sum(x.count('A') for x in open('seqs.fa','r') if x[0] != '&amp;gt;')&lt;/code&gt; in python&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This yields a time of&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;6&quot;&gt;&#xA;&lt;li&gt;1.6 seconds (I'm at work now, so I've adjusted the time as best I can to account for the different computers)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update2:&lt;/strong&gt; More benchmarks from the comments&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;7&quot;&gt;&#xA;&lt;li&gt;&lt;code&gt;awk -F A '!/^&amp;gt;&quot;/ {cnt+=NF-1}END{print cnt}' foo.fa&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;perl -ne 'if(!/^&amp;gt;/){$count += tr/A//} END{print &quot;$count\n&quot;}' foo.fa&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;These yield times of:&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;7&quot;&gt;&#xA;&lt;li&gt;5.2 seconds&lt;/li&gt;&#xA;&lt;li&gt;1.18 seconds&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The awk version is the best hack I could come up with, there are likely better ways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update 3&lt;/strong&gt;: Right, so regarding fastq files, I'm running this on a 3.3GB gzipped file with 10 repetitions (this takes a bit of time to run), so I'm not going to initially limit tests to commands that I can trivially modify to handle compressed files (after all, uncompressed fastq files are an abomination).&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;seqtk fqchk&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;sed -n '2~4p' &amp;lt;(zcat foo.fastq.gz) | grep -io A | wc -l&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;sed -n '1d;N;N;N;P;d' &amp;lt;(zcat Undetermined_S0_R1_001.fastq.gz) | tr -cd A | wc -c&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;The C example from user1728181 (the comparison to C++ is already provided there, so no need to include it).&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;bioawk -c fastx '{n+=gsub(/A/, &quot;&quot;, $seq)} END {print n}' foo.fastq.gz&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The average times are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;1 minute 44 seconds&lt;/li&gt;&#xA;&lt;li&gt;2 minutes 6 seconds&lt;/li&gt;&#xA;&lt;li&gt;1 minute 52 seconds&lt;/li&gt;&#xA;&lt;li&gt;1 minute 15 seconds&lt;/li&gt;&#xA;&lt;li&gt;3 minutes 8 seconds&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;N.B., these will often not handle fastq files with entries spanning more than 4 lines. However, such files are the dodo birds of bioinformatics, so that's not practically important. Also, I couldn't get &lt;code&gt;ShortRead&lt;/code&gt; to work with the compressed fastq file. I suspect there's a way to fix that.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-06-02T11:41:38.503" LastActivityDate="2017-06-02T11:41:38.503" CommentCount="24" ContentLicense="CC BY-SA 3.0" />
  <row Id="372" PostTypeId="1" AcceptedAnswerId="373" CreationDate="2017-06-01T22:57:13.273" Score="7" ViewCount="565" Body="&lt;p&gt;Without going into too much background, I just joined up with a lab as a bioinformatics intern while I'm completing my masters degree in the field. The lab has data from an RNA-seq they outsourced, but the only problem is that the only data they have is preprocessed from the company that did the sequencing: filtering the reads, aligning them, and putting the aligned reads through RSEM. I currently have output from RSEM for each of the four samples consisting of: gene id, transcript id(s), length, expected count, and FPKM. I am attempting to get the FASTQ files from the sequencing, but for now, this is what I have, and I'm trying to get something out of it if possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found &lt;a href=&quot;https://biowize.wordpress.com/2014/03/04/understanding-rsem-raw-read-counts-vs-expected-counts/&quot; rel=&quot;noreferrer&quot;&gt;this article&lt;/a&gt; that talks about how expected read counts can be better than raw read counts when analyzing differential expression using EBSeq; it's just one guy's opinion, and it's from 2014, so it may be wrong or outdated, but I thought I'd give it a try since I have the expected counts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I have just a couple of questions about running EBSeq that I can't find the answers to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1: In the output RSEM files I have, not all genes are represented in each, about 80% of them are, but for the ones that aren't, should I remove them before analysis with EBSeq? It runs when I do, but I'm not sure if it is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2: How do I know which normalization factor to use when running EBSeq? This is more of a conceptual question rather than a technical question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="506" LastEditorUserId="73" LastEditDate="2017-06-02T07:50:21.887" LastActivityDate="2017-06-02T07:50:21.887" Title="Missing genes and normalisation of RSEM output using EBSeq" Tags="|rna-seq|differential-expression|ebseq|rsem|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="373" PostTypeId="2" ParentId="372" CreationDate="2017-06-02T00:14:20.617" Score="6" Body="&lt;p&gt;Yes, that blog post &lt;em&gt;does&lt;/em&gt; represent just one guy's opinion (hi!) and it does date &lt;em&gt;all the way back to 2014&lt;/em&gt;, which is, like, decades in genomics years. :-) By the way, there is quite a bit of literature discussing the improvements that expected read counts derived from an Expectation Maximization algorithm provide over raw read counts. I'd suggest reading the RSEM papers for a start&lt;sup&gt;[&lt;a href=&quot;http://dx.doi.org/10.1093/bioinformatics/btp692&quot; rel=&quot;noreferrer&quot;&gt;1&lt;/a&gt;][&lt;a href=&quot;http://dx.doi.org/10.1186/1471-2105-12-323&quot; rel=&quot;noreferrer&quot;&gt;2&lt;/a&gt;]&lt;/sup&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But your main question is about the mechanics of running RSEM and EBSeq. First, RSEM was written explicitly to be compatible with EBSeq, so I'd be very surprised if it does not work correctly out-of-the-box. Second, EBSeq's &lt;code&gt;MedianNorm&lt;/code&gt; function worked very well in my experience for normalizing the library counts. Along those lines, the blog you mentioned above has &lt;a href=&quot;https://biowize.wordpress.com/2013/12/12/normalization-for-differential-expression-analysis/&quot; rel=&quot;noreferrer&quot;&gt;another post&lt;/a&gt; that you may find useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But all joking aside, these tools &lt;em&gt;are indeed&lt;/em&gt; dated. Alignment-free RNA-Seq tools provide orders-of-magnitude improvements in runtime over the older alignment-based alternatives, with comparable accuracy. &lt;a href=&quot;https://github.com/kingsfordgroup/sailfish&quot; rel=&quot;noreferrer&quot;&gt;Sailfish&lt;/a&gt; was the first in a growing list of tools that now includes &lt;a href=&quot;https://github.com/COMBINE-lab/salmon&quot; rel=&quot;noreferrer&quot;&gt;Salmon&lt;/a&gt; and &lt;a href=&quot;https://github.com/pachterlab/kallisto&quot; rel=&quot;noreferrer&quot;&gt;Kallisto&lt;/a&gt;. When starting a new analysis from scratch (i.e. if you ever get the original FASTQ files), there's really no good reason not to estimate expression using these much faster tools, followed by a differential expression analysis with DESeq2, edgeR, or sleuth.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;1&lt;/sup&gt;&lt;strong&gt;Li B, Ruotti V, Stewart RM, Thomson JA, Dewey CN&lt;/strong&gt; (2010) RNA-Seq gene expression estimation with read mapping uncertainty. &lt;em&gt;Bioinformatics&lt;/em&gt;, 26(4):493–500, &lt;a href=&quot;http://dx.doi.org/10.1093/bioinformatics/btp692&quot; rel=&quot;noreferrer&quot;&gt;doi:10.1093/bioinformatics/btp692&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;2&lt;/sup&gt;&lt;strong&gt;Li B, Dewey C&lt;/strong&gt; (2011) RSEM: accurate transcript quantification from RNA-Seq data with or without a reference genome. &lt;em&gt;BMC Bioinformatics&lt;/em&gt;, 12:323, &lt;a href=&quot;http://dx.doi.org/10.1186/1471-2105-12-323&quot; rel=&quot;noreferrer&quot;&gt;doi:10.1186/1471-2105-12-323&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2017-06-02T02:54:54.193" LastActivityDate="2017-06-02T02:54:54.193" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="374" PostTypeId="2" ParentId="372" CreationDate="2017-06-02T01:40:28.860" Score="3" Body="&lt;ol&gt;&#xA;&lt;li&gt;Include all genes/transcripts in your analysis.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;A transcript that is not detected could be undetected through sampling error (i.e. the sequencer / library prep just happened to miss that transcript), or it could be because the transcript isn't generated in a particular sample. It's not uncommon for genes to be switched off in response to different biological factors, so zero-count genes shouldn't be ignored. I can't speak from experience with EBSeq, but as long as the analysis package treats a zero count as &quot;unobserved&quot; rather than &quot;absent&quot; (and makes relevant corrections), it's a good idea to keep them in.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-02T01:40:28.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="375" PostTypeId="1" AcceptedAnswerId="4974" CreationDate="2017-06-02T02:48:04.570" Score="16" ViewCount="511" Body="&lt;p&gt;After discovering a few difficulties with genome assembly, I've taken an interest in finding and categorising repetitive DNA sequences, such as this one from &lt;em&gt;Nippostrongylus brasiliensis&lt;/em&gt; [each base is colour-coded as A: green; C: blue; G: yellow; T: red]:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QSk09.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QSk09.png&quot; alt=&quot;Repeat sequence represented in a rectangular fashion&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/RjVLd.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RjVLd.png&quot; alt=&quot;Repeat sequence represented in a circular fashion&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[FASTA file associated with this sequence can be found &lt;a href=&quot;http://www.gringene.org/data/longRepeat_ch192_r13894.fa&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These sequences with large repeat unit sizes are only detectable (and assembleable) using long reads (e.g. PacBio, nanopore) because any subsequence smaller than the unit length will not be able to distinguish between sequencing error and hitting a different location within the repeat structure. I have been tracking these sequences down in a bulk fashion by two methods:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Running an all-vs-all mapping, and looking for sequences that map to themselves lots of times&lt;/li&gt;&#xA;&lt;li&gt;Carrying out a compression of the sequence (e.g. bzip2), and finding sequences that have a compression rate that is substantially higher than normal&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;After I've found a suspicious sequence, I then want to be able to categorise the repeat (e.g. major repeat length, number of tandem repeats, repetitive sequence). This is where I'm getting stuck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For doing a &quot;look, shiny&quot; demonstration, I currently have a very manual process of getting these sequences into a format that I can visualise. My process is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use LAST to produce a dot plot of self-mapping for the mapping&lt;/li&gt;&#xA;&lt;li&gt;Visually identify the repetitive region, and extract out the region from the sequence&lt;/li&gt;&#xA;&lt;li&gt;Use a combination of &lt;code&gt;fold -w &amp;lt;width&amp;gt;&lt;/code&gt; and &lt;code&gt;less -S&lt;/code&gt; to visually inspect the sequence with various potential repeat unit widths to find the most likely repeat unit size&lt;/li&gt;&#xA;&lt;li&gt;Display the sequence in a rectangular and circular fashion using &lt;a href=&quot;https://github.com/gringer/bioinfscripts/blob/master/seqmat.r&quot; rel=&quot;noreferrer&quot;&gt;my own script&lt;/a&gt;, wrapping at the repeat unit length&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;But that process is by no means feasible when I've got thousands of potential repetitive sequences to fish through.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any better way to do this? Given an arbitrary DNA sequence of length &gt;10kb, how can I (in an automated fashion) find both the location of the repeat region, and also the unit length (bearing in mind that there might be multiple repeat structures, with unit lengths from 30bp to 10kb)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example sequence can be found &lt;a href=&quot;http://www.gringene.org/data/fd2.fa&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, which has a ~21kb repeat region with ~171bp repeat units about 1/3 of the way into the sequence.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;A Kmer-based Analysis&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I've now seen human sequences with repetitive regions &lt;a href=&quot;https://twitter.com/gringene_bio/status/919190635306876928&quot; rel=&quot;noreferrer&quot;&gt;in excess of 10kb&lt;/a&gt; (i.e. out of the range of most linked-read applications). My &lt;a href=&quot;https://github.com/gringer/bioinfscripts/blob/master/fastx-rlength.pl&quot; rel=&quot;noreferrer&quot;&gt;current idea&lt;/a&gt; is centred around creating hash tables of short sequences (currently 13-mers) and tracking their location:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Process the sequence, storing the location of each kmer&lt;/li&gt;&#xA;&lt;li&gt;For each kmer, find out how many times it appears in the sequence&lt;/li&gt;&#xA;&lt;li&gt;For repeated kmers, find out how much of a gap there is between the next time that kmer occurs&lt;/li&gt;&#xA;&lt;li&gt;Report the median and modal gap length of repeated kmers, and statistics associated with their frequency in the sequence&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Some local repetitive regions may be lost in the statistics with this approach, it's hard to tell if there are multiple repetitive regions within a single sequence, and if the repeat units are themselves slightly repetitive (enough that a kmer is duplicated within a repeat unit), then the algorithm will under-report the repetitiveness (see step 3).&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-09-04T11:09:40.187" LastActivityDate="2020-10-04T10:15:41.157" Title="Finding the location and unit length of repetitive sequences within a long read" Tags="|nanopore|long-reads|repeat-elements|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="376" PostTypeId="2" ParentId="357" CreationDate="2017-06-02T03:42:33.323" Score="5" Body="&lt;p&gt;The generic command &lt;code&gt;sh&lt;/code&gt; is quite literally an industry standard, a POSIX standard, to be precise (IEEE 1003.2 and 1003.2a, available for purchase for hundreds of dollars at various websites). In theory, any script that starts with &lt;code&gt;#!/bin/sh&lt;/code&gt; should conform to this standard. In practise, most Linux systems have a shell that is close to this standard, but has a few quirks and extensions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problems crop up when these quirks and extensions become standard practise in shell scripts. The Debian operating system changed to &lt;code&gt;dash&lt;/code&gt; as their &lt;code&gt;sh&lt;/code&gt; shell to encourage people to stop using &quot;bashisms&quot; in shell scripts that didn't specify a particular shell, i.e. those that began with &lt;code&gt;#!/bin/sh&lt;/code&gt;. The &lt;code&gt;dash&lt;/code&gt; shell tries to be as standards-compliant as possible:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;dash is the standard command interpreter for the system.  The current version of dash is in the process of being changed to&#xA;       conform with the POSIX 1003.2 and 1003.2a specifications for the shell.  This version has many features which make it&#xA;       appear similar in some respects to the Korn shell, but it is not a Korn shell clone (see ksh(1)).  Only features designated&#xA;       by POSIX, plus a few Berkeley extensions, are being incorporated into this shell.  This man page is not intended to be a&#xA;       tutorial or a complete specification of the shell.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not familiar with the differences, and generally try to stick to the &lt;code&gt;sh&lt;/code&gt; man pages to instruct me with regards to correct standards-compliant shell scripts.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-06T19:45:54.183" LastActivityDate="2017-06-06T19:45:54.183" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="377" PostTypeId="2" ParentId="357" CreationDate="2017-06-02T06:54:02.307" Score="7" Body="&lt;p&gt;&lt;a href=&quot;http://pubs.opengroup.org/onlinepubs/9699919799.2016edition/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Open Group Base Specifications Issue 7&#xA;IEEE Std 1003.1™-2008, 2016 Edition&lt;/a&gt;, or &quot;The POSIX Standard&quot; for short, is the standard that defines the interfaces and utilities provided by a Unix system.  Among these is the command line shell language and tools (see &quot;Shell &amp;amp; Utilities&quot; in the main index on the page linked above).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know, there is no shell that implement &lt;em&gt;exactly&lt;/em&gt; what's specified by the standard, but both &lt;code&gt;bash&lt;/code&gt; and &lt;code&gt;ksh93&lt;/code&gt; does a pretty good job of adhering to the standard along with their own, sometimes conflicting, extensions. The &lt;code&gt;ksh93&lt;/code&gt; shell in particular has had a big impact on the past development of the POSIX shell specification, but future POSIX specifications &lt;em&gt;may&lt;/em&gt; borrow more from &lt;code&gt;bash&lt;/code&gt; due to its wide use on Linux.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;bash&lt;/code&gt; shell is pretty much ubiquitous on Linux systems, and may be installed on all other Unices too. &lt;code&gt;ksh93&lt;/code&gt; is also available for most Unices but is usually not installed by default on Linux. &lt;code&gt;ksh93&lt;/code&gt; is available by default on at least macOS (as &lt;code&gt;ksh&lt;/code&gt;) and Solaris.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are concerned about portability when writing a shell script (which is IMHO a good thing to be concerned about), you should make sure that you use only the POSIX utilities and their POSIX command line flags, as well as only use POSIX shell syntax. You should then ensure that you script is executed by &lt;code&gt;/bin/sh&lt;/code&gt; which is supposed to be a shell that understands the POSIX specification.  &lt;code&gt;/bin/sh&lt;/code&gt; is often implemented by &lt;code&gt;bash&lt;/code&gt; running in &quot;POSIX mode&quot;, but it may also be &lt;code&gt;dash&lt;/code&gt;, &lt;code&gt;ash&lt;/code&gt; or &lt;code&gt;pdksh&lt;/code&gt; (or something else) depending on what Unix you are using.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a Linux user, the most difficult bit in writing a portable script is often not the shell per se, but the multitude of non-standard command line flags provided by the GNU implementation of the many shell utilities.  The GNU coreutils (basic shell utilities) may, like &lt;code&gt;bash&lt;/code&gt;, be installed on all Unices though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note that &lt;code&gt;bash&lt;/code&gt;, when running in POSIX mode (either when invoked as &lt;code&gt;/bin/sh&lt;/code&gt; or with its &lt;code&gt;--posix&lt;/code&gt; command line flag), is not &lt;em&gt;strict&lt;/em&gt; about its POSIX conformance and may accept some syntax extensions to the POSIX standard.&lt;/p&gt;&#xA;" OwnerUserId="434" LastEditorUserId="434" LastEditDate="2018-01-18T06:46:26.083" LastActivityDate="2018-01-18T06:46:26.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="378" PostTypeId="1" AcceptedAnswerId="386" CreationDate="2017-06-02T07:13:13.107" Score="14" ViewCount="334" Body="&lt;p&gt;Hidden Markov models (HMMs) are used extensively in bioinformatics, and have been adapted for gene prediction, protein family classification, and a variety of other problems. Indeed, &lt;a href=&quot;http://www.cambridge.org/gb/academic/subjects/life-sciences/genomics-bioinformatics-and-systems-biology/biological-sequence-analysis-probabilistic-models-proteins-and-nucleic-acids&quot; rel=&quot;noreferrer&quot;&gt;the treatise by Durbin, Eddy and colleagues&lt;/a&gt; is one of the defining volumes in this field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although the details of each of these different applications of HMMs differ, the core mathematical model remains unchanged, and there are efficient algorithms for computing the probability of the observed sequence given the model, or (perhaps more useful) the most likely hidden sequence given the sequence of observed states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Accordingly, it seems plausible that there could be a generic software library for solving HMMs. As far as I can tell that's not the case, and most bioinformaticians end up writing HMMs from scratch. Perhaps there's a good reason for this? (Aside from the obvious fact that it's already difficult, nigh impossible, to get funding to build and provide long-term support for open source science software. Academic pressures incentivize building a new tool that you can publish a paper on much more than building on and extending existing tools.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do any generic HMM solver libraries exist? If so, would this be tempting enough for bioinformaticians to use rather than writing their own from scratch?&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2017-06-04T13:37:40.803" Title="Generic HMM solvers in bioinformatics?" Tags="|hidden-markov-models|sequence-analysis|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="379" PostTypeId="2" ParentId="297" CreationDate="2017-06-02T07:52:08.220" Score="4" Body="&lt;p&gt;You mention that FastQC &lt;em&gt;&quot;fails to find the actual adapter sequences&quot;&lt;/em&gt; - I guess you mean in the Adapter Sequence Contamination plot. However, the kmer and Sequence Content Plots are often useful even when the former fails. I've used these in the past - you can sometimes just read off the adapter sequence from the start of the Sequence Content Plot (or at least see how many bases to trim).&lt;/p&gt;&#xA;" OwnerUserId="224" LastActivityDate="2017-06-02T07:52:08.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="380" PostTypeId="2" ParentId="361" CreationDate="2017-06-02T09:28:46.290" Score="6" Body="&lt;h2&gt;Using bioawk&lt;/h2&gt;&#xA;&lt;p&gt;With &lt;a href=&quot;https://github.com/lh3/bioawk&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioawk&lt;/a&gt; (counting &amp;quot;A&amp;quot; in the &lt;em&gt;C. elegans&lt;/em&gt; genome, because there seem to be no &amp;quot;N&amp;quot; in this file), on my computer:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ time bioawk -c fastx '{n+=gsub(/A/, &quot;&quot;, $&lt;/span&gt;seq)} END {print n}' genome.fa &#xA;32371810&#xA;&#xA;real    0m1.645s&#xA;user    0m1.548s&#xA;sys     0m0.088s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;bioawk is an extension of awk with convenient parsing options. For instance &lt;code&gt;-c fastx&lt;/code&gt; makes the sequence available as &lt;code&gt;$seq&lt;/code&gt; in fasta and fastq format (&lt;strong&gt;including gzipped versions&lt;/strong&gt;), so the above command &lt;strong&gt;should also handle fastq format robustly&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;gsub&lt;/code&gt; command is a normal awk function. It returns the number of substituted characters (got it from &lt;a href=&quot;https://unix.stackexchange.com/a/169575/55127&quot;&gt;https://unix.stackexchange.com/a/169575/55127&lt;/a&gt;). I welcome comments about how to count inside awk in a less hackish way.&lt;/p&gt;&#xA;&lt;p&gt;On this fasta example it is almost 3 times slower than the &lt;code&gt;grep&lt;/code&gt;, &lt;code&gt;tr&lt;/code&gt;, &lt;code&gt;wc&lt;/code&gt; pipeline:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ time grep -v &amp;quot;^&amp;gt;&amp;quot; genome.fa | tr -cd &amp;quot;A&amp;quot; | wc -c&#xA;32371810&#xA;&#xA;real    0m0.609s&#xA;user    0m0.744s&#xA;sys     0m0.184s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(I repeated the timings, and the above values seem representative)&lt;/p&gt;&#xA;&lt;h2&gt;Using python&lt;/h2&gt;&#xA;&lt;h3&gt;readfq&lt;/h3&gt;&#xA;&lt;p&gt;I tried the python readfq retrieved from the repository mentioned in this answer: &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/365/292&quot;&gt;https://bioinformatics.stackexchange.com/a/365/292&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ time python -c &amp;quot;from sys import stdin; from readfq import readfq; print sum((seq.count('A') for _, seq, _ in readfq(stdin)))&amp;quot; &amp;lt; genome.fa&#xA;32371810&#xA;&#xA;real    0m0.976s&#xA;user    0m0.876s&#xA;sys     0m0.100s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3&gt;&amp;quot;pure python&amp;quot;&lt;/h3&gt;&#xA;&lt;p&gt;Comparing with something adapted from this answer: &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/363/292&quot;&gt;https://bioinformatics.stackexchange.com/a/363/292&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ time python -c &amp;quot;from sys import stdin; print sum((line.count('A') for line in stdin if not line.startswith('&amp;gt;')))&amp;quot; &amp;lt; genome.fa&#xA;32371810&#xA;&#xA;real    0m1.143s&#xA;user    0m1.100s&#xA;sys     0m0.040s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(It is slower with python 3.6 than with python 2.7 on my computer)&lt;/p&gt;&#xA;&lt;h3&gt;pyGATB&lt;/h3&gt;&#xA;&lt;p&gt;I just learned (08/06/2017) that &lt;a href=&quot;http://gatb.inria.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATB&lt;/a&gt; includes a fasta/fastq parser and has recently released a python API. I tried to use it yesterday to test another answer to the present question and &lt;a href=&quot;https://github.com/GATB/pyGATB/issues/2&quot; rel=&quot;nofollow noreferrer&quot;&gt;found a bug&lt;/a&gt;. This bug is now fixed, so here is a &lt;a href=&quot;https://pypi.python.org/pypi/pyGATB/0.1.2&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyGATB&lt;/a&gt;-based answer:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ time python3 -c &amp;quot;from gatb import Bank; print(sum((seq.sequence.count(b\&amp;quot;A\&amp;quot;) for seq in Bank(\&amp;quot;genome.fa\&amp;quot;))))&amp;quot;&#xA;32371810&#xA;&#xA;real    0m0.663s&#xA;user    0m0.568s&#xA;sys     0m0.092s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(You can also do &lt;code&gt;sequence.decode(&amp;quot;utf-8&amp;quot;).count(&amp;quot;A&amp;quot;)&lt;/code&gt; but this seems &lt;a href=&quot;https://github.com/GATB/pyGATB/issues/2#issuecomment-307131176&quot; rel=&quot;nofollow noreferrer&quot;&gt;a little slower&lt;/a&gt;.)&lt;/p&gt;&#xA;&lt;p&gt;Although I used python3.6 here (pyGATB seems python3-only), this is faster than the other two python approaches (for which the reported timings are obtained with python 2.7). This is even almost as fast as the &lt;code&gt;grep&lt;/code&gt;, &lt;code&gt;tr&lt;/code&gt;, &lt;code&gt;wc&lt;/code&gt; pipeline.&lt;/p&gt;&#xA;&lt;h3&gt;Biopython&lt;/h3&gt;&#xA;&lt;p&gt;And, to have even more comparisons, here is a solution using &lt;code&gt;SeqIO.parse&lt;/code&gt; from Biopython (with python2.7):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ time python -c &amp;quot;from Bio import SeqIO; print(sum((rec.seq.count(\&amp;quot;A\&amp;quot;) for rec in SeqIO.parse(\&amp;quot;genome.fa\&amp;quot;, \&amp;quot;fasta\&amp;quot;))))&amp;quot;&#xA;32371810&#xA;&#xA;real    0m1.632s&#xA;user    0m1.532s&#xA;sys     0m0.096s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is a bit slower than the &amp;quot;pure python&amp;quot; solution, but perhaps more robust.&lt;/p&gt;&#xA;&lt;p&gt;There seems to be a slight improvement with @peterjc's suggestion to use the lower level &lt;code&gt;SimpleFastaParser&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ time python -c &amp;quot;from Bio.SeqIO.FastaIO import SimpleFastaParser; print(sum(seq.count('A') for title, seq in SimpleFastaParser(open('genome.fa'))))&amp;quot;&#xA;32371810&#xA;&#xA;real    0m1.618s&#xA;user    0m1.500s&#xA;sys     0m0.116s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(I did a series of timings and tried to take one that seemed representative, but there's a lot of overlap with the higher-level parser's timings.)&lt;/p&gt;&#xA;&lt;h3&gt;scikit-bio&lt;/h3&gt;&#xA;&lt;p&gt;I read about &lt;a href=&quot;http://scikit-bio.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;scikit-bio&lt;/a&gt; today (23/06/2017), which has a sequence reader.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ time python3 -c  &amp;quot;import skbio; print(sum(seq.count('A') for seq in skbio.io.read('genome.fa', format='fasta', verify=False)))&amp;quot;&#xA;32371810&#xA;&#xA;real    0m3.643s&#xA;user    0m3.440s&#xA;sys     0m1.228s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3&gt;pyfastx&lt;/h3&gt;&#xA;&lt;p&gt;I read about &lt;a href=&quot;https://pyfastx.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyfastx&lt;/a&gt; today (27/08/2020), which has a sequence reader.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ time python3 -c &amp;quot;from pyfastx import Fasta; print(sum(seq.count('A') for (_, seq) in Fasta('genome.fa', build_index=False)))&amp;quot;&#xA;32371810&#xA;&#xA;real    0m0.781s&#xA;user    0m0.740s&#xA;sys     0m0.040s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(The timing is done using python 3.6, on the same machine as previous timings from 3 years ago, so it should be comparable.)&lt;/p&gt;&#xA;&lt;h3&gt;&lt;code&gt;fastx_read&lt;/code&gt; from mappy&lt;/h3&gt;&#xA;&lt;p&gt;I realize (27/08/2020) that I hadn't added a benchmark for &lt;code&gt;fastx_read&lt;/code&gt; from &lt;a href=&quot;https://pypi.org/project/mappy/&quot; rel=&quot;nofollow noreferrer&quot;&gt;mappy&lt;/a&gt; which I regularly use since quite some time already:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ time python3 -c &amp;quot;from mappy import fastx_read; print(sum(seq.count('A') for (_, seq, _) in fastx_read('genome.fa')))&amp;quot;&#xA;32371810&#xA;&#xA;real    0m0.731s&#xA;user    0m0.648s&#xA;sys     0m0.080s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Benchmarks on a fastq.gz file&lt;/h2&gt;&#xA;&lt;p&gt;I tested some of the above solutions (or adaptations thereof), counting &amp;quot;N&amp;quot; in the same file that was used here: &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/400/292&quot;&gt;https://bioinformatics.stackexchange.com/a/400/292&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h3&gt;bioawk&lt;/h3&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ time bioawk -c fastx '{n+=gsub(/N/, &quot;&quot;, $&lt;/span&gt;seq)} END {print n}' SRR077487_2.filt.fastq.gz&#xA;306072&#xA;&#xA;real    1m9.686s&#xA;user    1m9.376s&#xA;sys     0m0.304s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3&gt;pigz + readfq python module&lt;/h3&gt;&#xA;&lt;p&gt;readfq doesn't complain and is very fast when I pass directly the compressed fastq, but returns something wrong, so don't forget to manually take care of the decompression.&lt;/p&gt;&#xA;&lt;p&gt;Here I tried with &lt;code&gt;pigz&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ time pigz -dc SRR077487_2.filt.fastq.gz | python -c &amp;quot;from sys import stdin; from readfq import readfq; print sum((seq.count('N') for _, seq, _ in readfq(stdin)))&amp;quot;&#xA;306072&#xA;&#xA;real    0m52.347s&#xA;user    1m40.716s&#xA;sys     0m8.604s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The computer has 16 cores, but I suspect the limiting factor for &lt;code&gt;pigz&lt;/code&gt; is reading from the disk: the processors are very far from running full speed.&lt;/p&gt;&#xA;&lt;p&gt;And with &lt;code&gt;gzip&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ time gzip -dc SRR077487_2.filt.fastq.gz | python -c &amp;quot;from sys import stdin; from readfq import readfq; print sum((seq.count('N') for _, seq, _ in readfq(stdin)))&amp;quot;&#xA;306072&#xA;&#xA;real    0m49.448s&#xA;user    1m31.984s&#xA;sys     0m2.312s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here gzip and python both used a full processor. Resource usage balance was slightly better. I work on a desktop computer. I suppose a computing server would take advantage of &lt;code&gt;pigz&lt;/code&gt; better.&lt;/p&gt;&#xA;&lt;h3&gt;pyGATB&lt;/h3&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ time python3 -c &amp;quot;from gatb import Bank; print(sum((seq.sequence.count(b\&amp;quot;N\&amp;quot;) for seq in Bank(\&amp;quot;SRR077487_2.filt.fastq.gz\&amp;quot;))))&amp;quot;&#xA;306072&#xA;&#xA;real    0m46.784s&#xA;user    0m46.404s&#xA;sys     0m0.296s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3&gt;biopython&lt;/h3&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ time python -c &amp;quot;from gzip import open as gzopen; from Bio.SeqIO.QualityIO import FastqGeneralIterator; print(sum(seq.count('N') for (_, seq, _) in FastqGeneralIterator(gzopen('SRR077487_2.filt.fastq.gz'))))&amp;quot;&#xA;306072&#xA;&#xA;real    3m18.103s&#xA;user    3m17.676s&#xA;sys     0m0.428s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3&gt;pyfastx (added 27/08/2020)&lt;/h3&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ time python3 -c &amp;quot;from pyfastx import Fastq; print(sum(seq.count('N') for (_, seq, _) in Fastq('SRR077487_2.filt.fastq.gz', build_index=False)))&amp;quot;&#xA;306072&#xA;&#xA;real    0m51.140s&#xA;user    0m50.784s&#xA;sys     0m0.352s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3&gt;&lt;code&gt;fastx_read&lt;/code&gt; from mappy (added 27/08/2020)&lt;/h3&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ time python3 -c &amp;quot;from mappy import fastx_read; print(sum(seq.count('N') for (_, seq, _) in fastx_read('SRR077487_2.filt.fastq.gz')))&amp;quot;&#xA;306072&#xA;&#xA;real    0m52.336s&#xA;user    0m52.024s&#xA;sys     0m0.300s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Conclusion&lt;/h2&gt;&#xA;&lt;p&gt;pyGATB and bioawk both handle transparently compression (gzipped or not) and format differences (fasta or fastq). pyGATB is quite a new tool, but seems more efficient compared to the other python modules I tested.&lt;/p&gt;&#xA;&lt;p&gt;(Update 27/08/2020) pyfastx and mappy seem quite convenient too, and are only slightly slower than pyGATB.&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2020-08-27T15:33:15.750" LastActivityDate="2020-08-27T15:33:15.750" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="382" PostTypeId="2" ParentId="150" CreationDate="2017-06-02T09:35:00.113" Score="3" Body="&lt;p&gt;Here is a bash script and GNU &lt;code&gt;parallel&lt;/code&gt; along with some other tools to split-apply-combine the data. It may or may not work out of the box, but it's not far off and it should give the general idea of how one way of how to approach this.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;# ====== USER PROVIDED PARAMETERS ========&#xA;N=20 # number of cores to use&#xA;ngsfilter=&quot;rawdata_scandinavia.ngsfilter&quot;&#xA;unidentified=&quot;unidentified_samples.fastq&quot;&#xA;&#xA;R1=&quot;rawdata_scandinavia_R1.fastq&quot;&#xA;R2=&quot;rawdata_scandinavia_R2.fastq&quot;&#xA;# ==== END USER PROVIDED PARAMETERS =====&#xA;&#xA;# fastqutils available @ http://ngsutils.org/modules/fastqutils&#xA;fastqutils split $R1 to_illumina_R1_ $N &amp;amp;&#xA;fastqutils split $R2 to_illumina_R2_ $N &amp;amp;&#xA;wait&#xA;&#xA;R1=$( ls | grep -P &quot;to_illumina_R1_&quot; )&#xA;R2=$( ls | grep -P &quot;to_illumina_R2_&quot; )&#xA;&#xA;# Do alignment of two reads.&#xA;for i in $( seq $N )&#xA;do&#xA;    partR1=$( ls | grep -P &quot;^to_illumina_R1_\\.$i\\.fastq$&quot; )&#xA;    partR2=$( ls | grep -P &quot;^to_illumina_R2_\\.$i\\.fastq$&quot; )&#xA;    illuminapairedend -r $partR1 $partR2 | tee to_ngsfilter_$i.fastq | obiannotate -S goodAli:'&quot;Align&quot; if score&amp;gt;40.00 else &quot;Bad&quot;' | obisplit -t goodAli -p to_ngsfilter_$i. &amp;amp;&#xA;done&#xA;&#xA;# Remove intermediate aligned files.&#xA;echo $R1 | xargs rm -r&#xA;echo $R2 | xargs rm -r&#xA;ls | grep -P &quot;\\.seq$|\\.err$&quot; | xargs rm -r&#xA;&#xA;input=$( ls | grep -P &quot;to_ngsfilter_[0-9]+\\.Align\\.fastq&quot; )&#xA;&#xA;# For some reasons, files need to be &quot;touched&quot; in order to work. Perhaps&#xA;# it's not closing them...&#xA;touch $unidentified&#xA;&#xA;touchedbyangel=$( ls | grep -P &quot;to_ngsfilter_&quot;  )&#xA;for i in $touchedbyangel&#xA;do&#xA;    touch $i&#xA;done&#xA;&#xA;parallel -j$N --result filtered_{} ngsfilter -t $ngsfilter -u $unidentified {} ::: $input&#xA;&#xA;# Remove intermediate results.&#xA;ls | grep -P &quot;\\.seq$|\\.err$&quot; | xargs rm -r&#xA;&#xA;ls | grep -P &quot;filtered_.*\\.fastq$&quot; | xargs cat &amp;gt; filtered_data.fastq&#xA;&#xA;# remove intermediate results to conserve space&#xA;rm to_ngsfilter*&#xA;&#xA;# Split reads by locus.&#xA;filtered=$( ls | grep -P &quot;^filtered_&quot;  )&#xA;parallel -j$N --result to_split_{} obisplit -p MS.PCR_ -t experiment {} ::: $filtered&#xA;&#xA;ls | grep -P &quot;\\.err$|\\.seq$&quot; | xargs rm -r&#xA;ls | grep -P &quot;^to_split_&quot; | xargs rm -r&#xA;&#xA;# Wasn't able to parallelize this, so it's just pushing tasks into the background.&#xA;obiuniq -m sample MS.PCR_UA_MxRout1_03.fastq &amp;gt; MS.PCR_UA_MxRout1_03.uniq.fasta &amp;amp;&#xA;obiuniq -m sample MS.PCR_UA_MxRout1_06.fastq &amp;gt; MS.PCR_UA_MxRout1_06.uniq.fasta &amp;amp;&#xA;obiuniq -m sample MS.PCR_UA_MxRout1_14.fastq &amp;gt; MS.PCR_UA_MxRout1_14.uniq.fasta &amp;amp;&#xA;obiuniq -m sample MS.PCR_UA_MxRout1_16.fastq &amp;gt; MS.PCR_UA_MxRout1_16.uniq.fasta &amp;amp;&#xA;obiuniq -m sample MS.PCR_UA_MxRout1_17.fastq &amp;gt; MS.PCR_UA_MxRout1_17.uniq.fasta &amp;amp;&#xA;obiuniq -m sample MS.PCR_UA_MxRout1_25.fastq &amp;gt; MS.PCR_UA_MxRout1_25.uniq.fasta &amp;amp;&#xA;obiuniq -m sample MS.PCR_UA_MxRout1_51.fastq &amp;gt; MS.PCR_UA_MxRout1_51.uniq.fasta &amp;amp;&#xA;obiuniq -m sample MS.PCR_UA_MxRout1_57.fastq &amp;gt; MS.PCR_UA_MxRout1_57.uniq.fasta &amp;amp;&#xA;obiuniq -m sample MS.PCR_UA_MxRout1_63.fastq &amp;gt; MS.PCR_UA_MxRout1_63.uniq.fasta &amp;amp;&#xA;obiuniq -m sample MS.PCR_UA_MxRout1_64.fastq &amp;gt; MS.PCR_UA_MxRout1_64.uniq.fasta &amp;amp;&#xA;obiuniq -m sample MS.PCR_UA_MxRout1_65.fastq &amp;gt; MS.PCR_UA_MxRout1_65.uniq.fasta &amp;amp;&#xA;obiuniq -m sample MS.PCR_UA_MxRout1_67.fastq &amp;gt; MS.PCR_UA_MxRout1_67.uniq.fasta &amp;amp;&#xA;obiuniq -m sample MS.PCR_UA_MxRout1_68.fastq &amp;gt; MS.PCR_UA_MxRout1_68.uniq.fasta &amp;amp;&#xA;obiuniq -m sample MS.PCR_UA_MxRout1_ZF.fastq &amp;gt; MS.PCR_UA_MxRout1_ZF.uniq.fasta &amp;amp;&#xA;wait&#xA;&#xA;uniqfiles=$( ls | grep -P &quot;.{2}+\\.uniq\\.fasta$&quot; )&#xA;&#xA;for i in $uniqfiles&#xA;do&#xA;    # this section removes the file suffix&#xA;    # https://stackoverflow.com/questions/125281/how-do-i-remove-the-file-suffix-and-path-portion-from-a-path-string-in-bash&#xA;    tab=${i%.fasta}&#xA;    tab=${tab##*/}&#xA;    obigrep -p 'count&amp;gt;1' $i | obiannotate -k merged_sample -k count | obiannotate --length | obisort -r -k seq_length | obitab --no-definition --output-seq &amp;gt; $tab.tab&#xA;done&#xA;&#xA;ls | grep -P &quot;\\.uniq\\.fasta&quot; | xargs rm -r&#xA;&#xA;echo &quot;Done. I think.&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="200" LastEditorUserId="292" LastEditDate="2017-06-02T13:18:13.977" LastActivityDate="2017-06-02T13:18:13.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="383" PostTypeId="2" ParentId="378" CreationDate="2017-06-02T10:59:48.487" Score="5" Body="&lt;p&gt;There are certainly software libraries for working with HMMs. For a general-purpose implementation in C++, take a look at the &lt;a href=&quot;http://docs.seqan.de/seqan/2.1.1/group_HmmAlgorithms.html&quot; rel=&quot;noreferrer&quot;&gt;SeqAn HMM algorithms&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your purposes, i.e. “computing … the most likely hidden sequence given the sequence of observed states”, you’d invoke &lt;code&gt;viterbiAlgorithm&lt;/code&gt; with your observed sequence and the HMM graph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More fundamentally I think that most existing, mature implementations are probably found in the domain of signal processing, which has been using them longer than biology, and where most of the underlying theory was developed.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2017-06-02T10:59:48.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="384" PostTypeId="2" ParentId="297" CreationDate="2017-06-02T11:41:15.883" Score="1" Body="&lt;p&gt;The &lt;code&gt;minion&lt;/code&gt; utility from the kraken/reaper toolkit may be helpful for this: &lt;a href=&quot;http://wwwdev.ebi.ac.uk/enright-dev/kraken/reaper/src/reaper-latest/doc/minion.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://wwwdev.ebi.ac.uk/enright-dev/kraken/reaper/src/reaper-latest/doc/minion.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="306" LastActivityDate="2017-06-02T11:41:15.883" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="385" PostTypeId="2" ParentId="14" CreationDate="2017-06-02T12:16:24.783" Score="62" Body="&lt;p&gt;Let’s start with what they have in common: All three formats store&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;sequence data, and&lt;/li&gt;&#xA;&lt;li&gt;sequence metadata.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Furthermore, all three formats are text-based.&lt;/p&gt;&#xA;&lt;p&gt;However, beyond that all three formats are different and serve different purposes.&lt;/p&gt;&#xA;&lt;p&gt;Let’s start with the simplest format:&lt;/p&gt;&#xA;&lt;h3&gt;FASTA&lt;/h3&gt;&#xA;&lt;p&gt;FASTA stores a variable number of sequence records, and for each record it stores the sequence itself, and a sequence ID. Each record starts with a header line whose first character is &lt;code&gt;&amp;gt;&lt;/code&gt;, followed by the sequence ID. The next lines of a record contain the actual sequence.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/FASTA_format&quot; rel=&quot;noreferrer&quot;&gt;Wikipedia artice&lt;/a&gt; gives several examples for peptide sequences, but since FASTQ and SAM are used exclusively (?) for nucleotide sequences, here’s a nucleotide example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Mus_musculus_tRNA-Ala-AGC-1-1 (chr13.trna34-AlaAGC)&#xA;GGGGGTGTAGCTCAGTGGTAGAGCGCGTGCTTAGCATGCACGAGGcCCTGGGTTCGATCC&#xA;CCAGCACCTCCA&#xA;&amp;gt;Mus_musculus_tRNA-Ala-AGC-10-1 (chr13.trna457-AlaAGC)&#xA;GGGGGATTAGCTCAAATGGTAGAGCGCTCGCTTAGCATGCAAGAGGtAGTGGGATCGATG&#xA;CCCACATCCTCCA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The ID can be in any arbitrary format, although &lt;a href=&quot;https://en.wikipedia.org/wiki/FASTA_format#Sequence_identifiers&quot; rel=&quot;noreferrer&quot;&gt;several conventions exist&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In the context of nucleotide sequences, FASTA is mostly used to store reference data; that is, data extracted from a curated database; the above is adapted from &lt;a href=&quot;http://gtrnadb.ucsc.edu/&quot; rel=&quot;noreferrer&quot;&gt;GtRNAdb&lt;/a&gt; (a database of tRNA sequences).&lt;/p&gt;&#xA;&lt;h3&gt;FASTQ&lt;/h3&gt;&#xA;&lt;p&gt;FASTQ was conceived to solve a specific problem arising during sequencing: Due to how different sequencing technologies work, the confidence in each &lt;a href=&quot;https://biology.stackexchange.com/a/1873/166&quot;&gt;base call&lt;/a&gt; (that is, the estimated probability of having correctly identified a given nucleotide) varies. This is expressed in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Phred_quality_score&quot; rel=&quot;noreferrer&quot;&gt;Phred quality score&lt;/a&gt;. FASTA had no standardised way of encoding this. By contrast, a FASTQ record &lt;a href=&quot;https://en.wikipedia.org/wiki/FASTQ_format#Encoding&quot; rel=&quot;noreferrer&quot;&gt;contains a sequence of quality scores&lt;/a&gt; for each nucleotide.&lt;/p&gt;&#xA;&lt;p&gt;A FASTQ record has the following format:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A line starting with &lt;code&gt;@&lt;/code&gt;, containing the sequence ID.&lt;/li&gt;&#xA;&lt;li&gt;One or more lines that contain the sequence.&lt;/li&gt;&#xA;&lt;li&gt;A new line starting with the character &lt;code&gt;+&lt;/code&gt;, and being either empty or repeating the sequence ID.&lt;/li&gt;&#xA;&lt;li&gt;One or more lines that contain the quality scores.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Here’s an example of a FASTQ file with two records:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@071112_SLXA-EAS1_s_7:5:1:817:345&#xA;GGGTGATGGCCGCTGCCGATGGCGTC&#xA;AAATCCCACC&#xA;+&#xA;IIIIIIIIIIIIIIIIIIIIIIIIII&#xA;IIII9IG9IC&#xA;@071112_SLXA-EAS1_s_7:5:1:801:338&#xA;GTTCAGGGATACGACGTTTGTATTTTAAGAATCTGA&#xA;+&#xA;IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII6IBI&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;FASTQ files are mostly used to store short-read data from high-throughput sequencing experiments. The sequence and quality scores are usually put into a single line each, and indeed many tools assume that each record in a FASTQ file is exactly four lines long, even though this isn’t guaranteed.&lt;/p&gt;&#xA;&lt;p&gt;As with FASTA, the format of the sequence ID isn’t standardised, but different producers of FASTQ use &lt;a href=&quot;https://en.wikipedia.org/wiki/FASTQ_format#Illumina_sequence_identifiers&quot; rel=&quot;noreferrer&quot;&gt;fixed notations that follow strict conventions&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;SAM&lt;/h3&gt;&#xA;&lt;p&gt;SAM files are so complex that a &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;noreferrer&quot;&gt;complete description&lt;/a&gt; &lt;sup&gt;[PDF]&lt;/sup&gt; takes 15 pages. So here’s the short version.&lt;/p&gt;&#xA;&lt;p&gt;The original purpose of SAM files is to store mapping information for sequences from high-throughput sequencing. As a consequence, a SAM record needs to store more than just the sequence and its quality, it also needs to store information about where and how a sequence maps into the reference.&lt;/p&gt;&#xA;&lt;p&gt;Unlike the previous formats, SAM is tab-based, and each record, consisting of either 11 or 12 fields, fills exactly one line. Here’s an example (tabs replaced by fixed-width spacing):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;r001  99  chr1  7 30  17M         =  37  39  TTAGATAAAGGATACTG   IIIIIIIIIIIIIIIII&#xA;r002  0   chrX  9 30  3S6M1P1I4M  *  0   0   AAAAGATAAGGATA      IIIIIIIIII6IBI    NM:i:1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For a description of the individual fields, refer to the documentation. The relevant bit is this: SAM can express exactly the same information as FASTQ, plus, as mentioned, the mapping information. However, SAM is also used to store read data &lt;em&gt;without&lt;/em&gt; mapping information.&lt;/p&gt;&#xA;&lt;p&gt;In addition to sequence records, SAM files can also contain a &lt;em&gt;header&lt;/em&gt;, which stores information about the reference that the sequences were mapped to, and the tool used to create the SAM file. Header information precede the sequence records, and consist of lines starting with &lt;code&gt;@&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;SAM itself is almost never used as a storage format; instead, files are stored in BAM format, which is a compact, gzipped, binary representation of SAM. It stores the same information, just more efficiently. And, in conjunction with a &lt;a href=&quot;https://en.wikipedia.org/wiki/Database_index&quot; rel=&quot;noreferrer&quot;&gt;search index&lt;/a&gt;, allows fast retrieval of individual records from the middle of the file (= fast &lt;a href=&quot;https://en.wikipedia.org/wiki/Random_access&quot; rel=&quot;noreferrer&quot;&gt;random access&lt;/a&gt;). BAM files are also much more compact than compressed FASTQ or FASTA files.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The above implies a &lt;em&gt;hierarchy&lt;/em&gt; in what the formats can store: FASTA ⊂ FASTQ ⊂ SAM.&lt;/p&gt;&#xA;&lt;p&gt;In a typical high-throughput analysis workflow, you will encounter all three file types:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;FASTA to store the reference genome/transcriptome that the sequence fragments will be mapped to.&lt;/li&gt;&#xA;&lt;li&gt;FASTQ to store the sequence fragments before mapping.&lt;/li&gt;&#xA;&lt;li&gt;SAM/BAM to store the sequence fragments after mapping.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2021-09-22T09:02:45.223" LastActivityDate="2021-09-22T09:02:45.223" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="386" PostTypeId="2" ParentId="378" CreationDate="2017-06-02T13:16:00.703" Score="7" Body="&lt;p&gt;I would also recommend to take a look at &lt;a href=&quot;https://pomegranate.readthedocs.io/en/latest/&quot; rel=&quot;noreferrer&quot;&gt;pomegranate&lt;/a&gt;, a nice Python package for probabilistic graphical models. It includes solvers for HMMs and much more. Under the hood it uses Cythonised code, so it's also quite fast.&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2017-06-02T13:16:00.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="388" PostTypeId="1" AcceptedAnswerId="12782" CreationDate="2017-06-02T14:08:49.563" Score="13" ViewCount="404" Body="&lt;p&gt;I want some templates of different file formats that I can use to test my scripts and identify possible bugs in my code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider nucleotide FASTA, a simple but often abused format, I would want templates to capture regular and irregular formats, like I have seen all of these:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Single line sequence&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1&#xA;ATG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;2) Multi-line sequence&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1&#xA;AT&#xA;G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;3) Upper and lower case letters in sequence&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1&#xA;Atg&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;4) Ns and Xs (and possibly other letters) in sequence&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1&#xA;ANnxX&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;5) Unusual headers (sometimes non-ASCI characters, need to consider the encoding)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;ATG &amp;gt;汉字&#xA;ATG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;6) Whitespace between records&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1&#xA;ATG&#xA;&#xA;&amp;gt;2&#xA;ATG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;7) Duplicated headers&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1&#xA;ATG&#xA;&amp;gt;1&#xA;ATC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;8) Empty headers or sequences (valid FASTA?)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;9) No new line &lt;code&gt;'\n'&lt;/code&gt; character on last line (can mess-up file concatenation)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1&#xA;A# &amp;lt; no new-line here&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;10) Different newline characters depending on the OS&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1&#xA;A# \r\n vs \n&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There should be separate templates for nucleotide and protein FASTA, and separate ones for aligned FASTA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would ideally include other aspects too, like different compression formats (such as &lt;code&gt;.gz&lt;/code&gt;, &lt;code&gt;.bzip2&lt;/code&gt;) and different file extensions (such as &lt;code&gt;.fa&lt;/code&gt;, &lt;code&gt;.fasta&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have never seen resources that provide templates covering these, but I think it would be useful. Of course I could build my own templates but it would take time to capture all the likely variations of the formats, particularly for more complex file formats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, I am not just interested in FASTA format, it was an example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note, I know about tools (such as &lt;code&gt;BioPython&lt;/code&gt;) that should handle many formats well, but they may also have bugs. Anyway, in practice sometimes I end up parsing files myself directly because I don't want the overhead or dependency of an external package.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;EDIT: Please don't answer this question to say you don't know of any such resources, me neither, hence the question. &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/393/104&quot;&gt;bli's helpful answer&lt;/a&gt; shows that there is at least one test suite that could be used as a starting point. I know that it is normally easy to look up the specification of any particular file format.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="104" LastEditDate="2018-11-19T09:27:48.003" LastActivityDate="2020-03-31T11:26:12.363" Title="Are there any databases of templates for common bioinformatic file formats?" Tags="|fasta|database|file-formats|" AnswerCount="6" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="389" PostTypeId="2" ParentId="308" CreationDate="2017-06-02T14:20:08.550" Score="3" Body="&lt;p&gt;I have used smalt a couple of times. Smalt is more flexible than bwa/bowtie and can be tuned to be more sensitive to divergent hits, which is useful to certain applications. I heard Sanger did evaluate a few mappers for some non-typical applications (not sure what). They found smalt to perform better. Also smalt was developed at Sanger. I guess they could modify smalt based on their needs, though I don't know the details.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-06-02T14:20:08.550" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="390" PostTypeId="2" ParentId="375" CreationDate="2017-06-02T14:20:35.313" Score="2" Body="&lt;p&gt;It could be an idea to fragment the long reads into small sequences, like simulating Illumina reads of 150 bp, and then map these small sequences against the original long reads and extract regions with a high coverage? &lt;/p&gt;&#xA;" OwnerUserId="446" LastActivityDate="2017-06-02T14:20:35.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="391" PostTypeId="2" ParentId="388" CreationDate="2017-06-02T14:24:46.343" Score="0" Body="&lt;p&gt;These may not be exactly what your are looking for, but they do contain a wide range of formats with examples. If you want more, you might try searching other major databases hosting other types of into e.g. Uniprot, PDB, NCBI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://genome.ucsc.edu/FAQ/FAQformat.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genome.ucsc.edu/FAQ/FAQformat.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ensembl.org/info/website/upload/bed.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ensembl.org/info/website/upload/bed.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="269" LastActivityDate="2017-06-02T14:24:46.343" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="392" PostTypeId="2" ParentId="388" CreationDate="2017-06-02T14:24:55.750" Score="1" Body="&lt;p&gt;No. At least none that I've heard of and I doubt there ever will be. There is no central repository for formats and each tool, community, field etc have their own. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best you can do is look up the official standard for each format and hope they include examples. Having a truly comprehensive collection of all possible variations is basically impossible for &lt;del&gt;horrible&lt;/del&gt; complex formats like VCF. Just consider the simple fact that it allows user-defined &lt;code&gt;INFO&lt;/code&gt; fields with arbitrary contents. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the best you can do is make sure your scripts conform with the standard and hope your input does as well. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-06-02T14:24:55.750" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="393" PostTypeId="2" ParentId="388" CreationDate="2017-06-02T16:18:48.537" Score="7" Body="&lt;p&gt;You mention Biopython, which contains tests: &lt;a href=&quot;https://github.com/biopython/biopython/tree/master/Tests&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/biopython/biopython/tree/master/Tests&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the tests consist in reading files present in the folders listed in the above link. These files could be a starting point for a database of test files. Whenever one comes across a test case not covered with these files, one could construct a new test file and contribute it to Biopython, along with a test, or at least file an issue: &lt;a href=&quot;https://github.com/biopython/biopython/issues&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/biopython/biopython/issues&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That would be a way to contribute to Biopython while constituting a database of test files.&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2017-06-02T16:18:48.537" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="394" PostTypeId="2" ParentId="388" CreationDate="2017-06-02T17:08:55.260" Score="2" Body="&lt;p&gt;As far as I know, there is no single repository that collects all of the common data formats used in bioinformatics.  Typically, you have to go to the source to find the specifications for each format.  There are a few places that collect descriptions of file formats, though:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://software.broadinstitute.org/software/igv/FileFormats&quot; rel=&quot;nofollow noreferrer&quot;&gt;IGV File Formats&lt;/a&gt;, coveres all of the formats usable in the Broad Institute's Integrative Genomics Viewer software (which is a lot).&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://wiki.nci.nih.gov/display/TCGA/File+Format+Specifications&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCI File Formats&lt;/a&gt;, mostly formats used by TGCA (including MAF and VCF).&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://cgwb.nci.nih.gov/FAQ/FAQformat.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC Genomics&lt;/a&gt;, covers BED, MAF, and a few others.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://software.broadinstitute.org/cancer/software/genepattern/file-formats-guide&quot; rel=&quot;nofollow noreferrer&quot;&gt;GenePattern&lt;/a&gt;, covers many of the file formats related to microarray data.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://software.broadinstitute.org/cancer/software/gsea/wiki/index.php/Data_formats&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSEA&lt;/a&gt;, Broad Gene Set Enrichment Analysis documentation.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="47" LastActivityDate="2017-06-02T17:08:55.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="395" PostTypeId="1" AcceptedAnswerId="399" CreationDate="2017-06-02T19:22:34.750" Score="6" ViewCount="180" Body="&lt;p&gt;In the analyses of single-cell RNA-seq data there are different unsupervised approaches to identify putative subpopulations (e.g. as available with Suerat or SCDE packages). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a good way of computationally validating the cluster solutions? Different methods may results in slightly different clustering results. How to know which one is the best i.e. representative of biological sub-populations?&lt;/p&gt;&#xA;" OwnerUserId="467" LastEditorUserId="57" LastEditDate="2017-06-02T19:37:06.107" LastActivityDate="2017-06-14T01:38:01.883" Title="validating identified sub-populations of cells in scRNA-seq" Tags="|scrnaseq|clustering|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="396" PostTypeId="1" AcceptedAnswerId="398" CreationDate="2017-06-02T19:36:07.733" Score="9" ViewCount="3294" Body="&lt;p&gt;I have indexed a gzipped reference with bwa: &lt;code&gt;bwa index reference.fa.gz&lt;/code&gt;, which produces a series of other files &lt;code&gt;reference.fa.gz.{amb,ann,bwt,pac,sa}&lt;/code&gt;. These are working fine with bwa alignment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have discovered that samtools does not take a gzipped reference, so I am planning to use an unzipped version of the reference for my workflow instead of dealing with two separate representations of the reference. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are the files &lt;code&gt;reference.fa.gz.{amb,ann,bwt,pac,sa}&lt;/code&gt; dependent upon the reference being gzipped? Do I need to reindex the unzipped reference.fa with bwa, or can I just rename the current files to remove the &lt;code&gt;.gz&lt;/code&gt; portion of the filename? &lt;/p&gt;&#xA;" OwnerUserId="272" LastEditorUserId="37" LastEditDate="2017-06-25T02:11:15.843" LastActivityDate="2017-06-25T02:11:15.843" Title="What are all the reference files produced by bwa index, and are these dependent upon whether the reference is zipped?" Tags="|bwa|samtools|reference-genome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="397" PostTypeId="2" ParentId="388" CreationDate="2017-06-02T19:44:00.670" Score="5" Body="&lt;p&gt;Not that I am aware. It is best to go with format specifications when coding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also it may be good to look at the example files that come together with various tools performing file conversions and handling. E.g. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Trimmomatic comes with few .fa files (&lt;a href=&quot;http://www.usadellab.org/cms/?page=trimmomatic&quot; rel=&quot;noreferrer&quot;&gt;http://www.usadellab.org/cms/?page=trimmomatic&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Samtools with a toy.sam (&lt;a href=&quot;https://github.com/lh3/samtools-legacy/blob/master/examples/toy.sam&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/lh3/samtools-legacy/blob/master/examples/toy.sam&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="375" LastEditorUserId="77" LastEditDate="2017-06-02T19:58:28.837" LastActivityDate="2017-06-02T19:58:28.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="398" PostTypeId="2" ParentId="396" CreationDate="2017-06-02T19:46:16.340" Score="7" Body="&lt;p&gt;You'll get the exact same index (the &lt;code&gt;amb&lt;/code&gt;, &lt;code&gt;ann&lt;/code&gt;, &lt;code&gt;bwt&lt;/code&gt;, &lt;code&gt;pac&lt;/code&gt; and &lt;code&gt;sa&lt;/code&gt; files) whether the reference is gzipped or not. BWA also makes its own packed reference sequence (the .pac file) so you don't even need the genome around after you index.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-06-02T19:46:16.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="399" PostTypeId="2" ParentId="395" CreationDate="2017-06-02T19:50:21.880" Score="6" Body="&lt;p&gt;A SC3, single-cell consensus clustering, approach could be helpful here. It aims at achieving &quot;high accuracy and robustness by combining multiple clustering solutions through a consensus approach&quot; &lt;a href=&quot;https://www.nature.com/nmeth/journal/v14/n5/full/nmeth.4236.html&quot; rel=&quot;noreferrer&quot;&gt;https://www.nature.com/nmeth/journal/v14/n5/full/nmeth.4236.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="375" LastActivityDate="2017-06-02T19:50:21.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="400" PostTypeId="2" ParentId="361" CreationDate="2017-06-02T20:28:26.517" Score="6" Body="&lt;h3&gt;Decompression of gzipped FASTQ is the main issue&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If we take real world gzipped FASTQ (which as the OP suggested would be beneficial) rather than trivial FASTA as the starting point then the real issue is actually decompressing the file not counting the Ns and in this case the C program &lt;code&gt;count-N&lt;/code&gt; is no longer the fastest solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally it would be good to use a specific file for benchmarking which actually has Ns, because you'll get some quite interesting execution time differences with some methods counting the more frequently occurring As rather than Ns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One such file is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/phase3/data/HG00096/sequence_read/SRR077487_2.filt.fastq.gz&quot; rel=&quot;noreferrer&quot;&gt;http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/phase3/data/HG00096/sequence_read/SRR077487_2.filt.fastq.gz&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's also worth checking the various solutions return the correct answer there should be 306072 Ns in the above file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next note that decompression of this file redirected to &lt;code&gt;/dev/null&lt;/code&gt; is slower with &lt;code&gt;zcat&lt;/code&gt; and &lt;code&gt;gzip&lt;/code&gt; (which are both gzip 1.6 on my system) than say a parallel implementation of gzip like &lt;a href=&quot;http://zlib.net/pigz/&quot; rel=&quot;noreferrer&quot;&gt;Mark Adler's&lt;/a&gt; &lt;code&gt;pigz&lt;/code&gt;, which appears to use 4 threads for decompression.  All timings represent an average of 10 runs reporting real (wall clock time).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time pigz -dc SRR077487_2.filt.fastq.gz &amp;gt; /dev/null&#xA;&#xA;real    0m29.0132s&#xA;&#xA;time gzip -dc SRR077487_2.filt.fastq.gz &amp;gt; /dev/null&#xA;&#xA;real    0m40.6996s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There is an ~11.7 second difference between the two.  Next if I then try to benchmark a one-liner which performs on FASTQ and gives the correct answer (Note I've yet to encounter FASTQ which is not 4 line, and seriously who generates these files!)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time pigz -dc SRR077487_2.filt.fastq.gz | awk 'NR%4==2{print $1}' | tr -cd N | wc -c&#xA;306072&#xA;&#xA;real    0m34.793s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see the counting adds ~5.8 second to the total run time versus &lt;code&gt;pigz&lt;/code&gt; based decompression.  Additionally this time delta is higher when using &lt;code&gt;gzip&lt;/code&gt; ~6.7 seconds above gzip decompression alone.   &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time gzip -dc SRR077487_2.filt.fastq.gz | awk 'NR%4==2{print $1}' | tr -cd N | wc -c                                                                &#xA;306072&#xA;&#xA;real    0m44.399s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;pigz awk tr wc&lt;/code&gt; based solution is however ~4.5 seconds faster than the &lt;code&gt;count-N&lt;/code&gt; based C code solution:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time count-N SRR077487_2.filt.fastq.gz &#xA;2385855128      306072  0&#xA;&#xA;real    0m39.266s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This difference appears to be robust to re-running as many times as I like.  I expect if you could use pthread in the C based solution or alter it to take the standard out from pigz it would also show an increase in performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Benchmarking another alternative &lt;code&gt;pigz grep&lt;/code&gt; variant appears to take more or less the same time as the &lt;code&gt;tr&lt;/code&gt; based variant:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time pigz -dc SRR077487_2.filt.fastq.gz | awk 'NR%4==2{print $1}' | grep -o N | wc -l&#xA;306072&#xA;&#xA;real    0m34.869s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that the &lt;code&gt;seqtk&lt;/code&gt; based solution discussed above is noticeably slower,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time seqtk comp  SRR077487_2.filt.fastq.gz | awk '{x+=$9}END{print x}'&#xA;306072&#xA;&#xA;real    1m42.062s &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However it's worth noting &lt;code&gt;seqtk comp&lt;/code&gt; is doing a bit more than the other solutions.&lt;/p&gt;&#xA;" OwnerUserId="1769" LastEditorUserId="1769" LastEditDate="2017-06-02T20:57:56.203" LastActivityDate="2017-06-02T20:57:56.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="401" PostTypeId="1" CreationDate="2017-06-02T23:38:13.547" Score="9" ViewCount="1573" Body="&lt;p&gt;I have a BAM created by Picard. I want to filter alignments by flags with &lt;code&gt;samtools view&lt;/code&gt;. However, I noticed that even if I apply no filters, the output BAM is different from my input BAM. Are BAMs produced by different tools also different in size? How can I check if two BAMs are the same?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="73" LastEditDate="2017-06-03T02:05:48.400" LastActivityDate="2017-06-03T12:14:11.773" Title="Why do BAM files created by different tools have different file sizes?" Tags="|sam|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="402" PostTypeId="1" AcceptedAnswerId="406" CreationDate="2017-06-02T23:44:44.637" Score="20" ViewCount="14759" Body="&lt;p&gt;I know how to downsample a BAM file to lower coverage. I know I can randomly select lines in SAM, but this procedure can't guarantee two reads in a pair are always sampled the same time. Is there a way to downsample BAM while keeping pairing information intact?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2023-09-15T00:17:45.753" Title="How can I downsample a BAM file while keeping both reads in pairs?" Tags="|sam|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="403" PostTypeId="1" CreationDate="2017-06-02T23:50:11.167" Score="18" ViewCount="5140" Body="&lt;p&gt;After some google searches, I found multiple tools with overlapping functionality for viewing, merging, pileuping, etc. I have not got time to try these tools, so will just see if anyone already know the answer: what is the difference between them? Performance? Features? Or something else? Which one is generally preferred? Samtools?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="2085" LastEditDate="2018-12-09T06:25:02.533" LastActivityDate="2018-12-09T06:25:02.533" Title="What is the difference between samtools, bamtools, picard, sambamba and biobambam?" Tags="|sequence-alignment|sam|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="404" PostTypeId="1" AcceptedAnswerId="421" CreationDate="2017-06-02T23:58:10.703" Score="8" ViewCount="742" Body="&lt;p&gt;As I read the SAM spec, the &quot;X&quot; CIGAR operator represents a mismatch. This seems useful as we can know where are the mismatches without looking at the reference genome. However, many popular aligners such as BWA do not output &quot;X&quot;. Why do they omit &quot;X&quot;?&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2017-06-08T09:10:32.403" Title="Why most aligners do not output the &quot;X&quot; CIGAR operation?" Tags="|bwa|sam|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="405" PostTypeId="2" ParentId="401" CreationDate="2017-06-03T00:20:02.360" Score="4" Body="&lt;p&gt;It's unlikely that two different mapping tools will give exactly the same alignment, scores, and match strings for the same sequence mapped to the same reference. For some sequence/reference alignments, it's impossible to determine which is the &quot;best&quot; alignment, and slight differences in code can have large effects on the chosen alignment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, even if the actual mapping location and match string are exactly the same (for example when using a tool like Picard to filter BAM/SAM files), different tools will incorporate different metadata with each mapping. This is allowed in the &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAM file format specification&lt;/a&gt; by the addition of optional fields beyond the 11th column. There are a few standard optional tags that can be used in these fields, and additional custom non-standard tags can be used as well. It is very likely that Picard is adding additional metadata to the alignments in the BAM/SAM file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's an additional complication in that the underlying &lt;em&gt;SAM&lt;/em&gt; alignment (and metadata) could be identical, but the &lt;em&gt;BAM&lt;/em&gt; file can still have different file sizes. One reason for this is that BAM file compression methods can be changed. For example, alignment tools might choose a quick compression method, while filtering tools might choose a method that results in better compression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Checking for alignment similarity is more difficult than just comparing files at a binary level, and your particular application (or context, or story) will change what the best method of comparison is. It would be helpful to know why you want to compare BAM files in order to provide a better answer to your question.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-03T00:20:02.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="406" PostTypeId="2" ParentId="402" CreationDate="2017-06-03T01:08:48.127" Score="17" Body="&lt;p&gt;&lt;a href=&quot;http://www.htslib.org/doc/samtools.html&quot; rel=&quot;noreferrer&quot;&gt;samtools&lt;/a&gt; has a subsampling option:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;-s FLOAT: &#xA;  Integer part is used to seed the random number generator [0]. Part after the decimal point sets the fraction of templates/pairs to subsample [no subsampling]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -bs 42.1 in.bam &amp;gt; subsampled.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;will subsample 10 percent mapped reads with 42 as the seed for the random number generator.&lt;/p&gt;&#xA;" OwnerUserId="161" LastActivityDate="2017-06-03T01:08:48.127" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="407" PostTypeId="1" AcceptedAnswerId="416" CreationDate="2017-06-03T01:22:37.273" Score="5" ViewCount="254" Body="&lt;p&gt;I am calling SNPs from WGS samples produced at my lab. I am currently using bwa-mem for mapping Illumina reads as it is recommended by GATK best practice. However, bwa is a bit slow. I heard from my colleague that &lt;a href=&quot;https://github.com/amplab/snap&quot; rel=&quot;noreferrer&quot;&gt;SNAP&lt;/a&gt; is much faster than bwa. I tried it on a small set of reads and it is indeed faster. However, I am not sure how it works with downstream SNP callers, so here are my questions: have you used SNAP for short-read mapping? What is your experience? Does SNAP work well with SNP callers like GATK and freebayes? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2017-06-04T00:35:38.640" Title="Do you use SNAP for short-read mapping?" Tags="|bwa|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="408" PostTypeId="2" ParentId="403" CreationDate="2017-06-03T01:50:55.283" Score="16" Body="&lt;p&gt;The obvious answer is that different people wrote them. It's fairly common in bioinformatics for people with a computer science background to get frustrated with existing tools and create their own alternative tool (rather than improving an existing tool). Over time, tools with similar initial aims will have popular functionality implemented in them (and eventually have bugs fixed), such that it matters less which particular tool is used for common methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's my impression of the tools:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/samtools/&quot; rel=&quot;noreferrer&quot;&gt;samtools&lt;/a&gt; -- originally written by Heng Li (who also wrote BWA). The people who now work on samtools also maintain the alignment file format specification for &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;noreferrer&quot;&gt;SAM, BAM, and CRAM&lt;/a&gt;, so any new file format features are likely to be implemented in samtools first.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/pezmaster31/bamtools&quot; rel=&quot;noreferrer&quot;&gt;bamtools&lt;/a&gt; -- this looks like it was written by Derek Barnett, Erik Garrison, Gabor Marth, Michael Stromberg to mirror the samtools toolkit, but using C++ instead of C&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://broadinstitute.github.io/picard/&quot; rel=&quot;noreferrer&quot;&gt;picard&lt;/a&gt; -- Java tools written by the Broad Institute for manipulating BAM/SAM files. Being written in Java makes it easier to port to other operating systems, so it may work better on Windows systems. I'm more familiar with picard being used at a filtering level (e.g. removing PCR duplicates), and for statistical analysis, but it links in with the Java HTS library from samtools, so probably shares a lot of the functionality.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/lomereiter/sambamba&quot; rel=&quot;noreferrer&quot;&gt;sambamba&lt;/a&gt; -- a GPL2-licensed toolkit written in the D programming language (presumably by Artem Tarasov and Pjotr Prins). I haven't used it (and don't know people who have used it), but the github page suggests &quot;For almost 5 years the main advantage over samtools was parallelized BAM reading. Finally in March 2017 samtools 1.4 was released, reaching parity on this.&quot;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/gt1/biobambam2&quot; rel=&quot;noreferrer&quot;&gt;biobambam&lt;/a&gt; -- written by German Tischler in C++. I also have no experience with this toolkit. This seems to have some multithreading capability, but is otherwise similar to other toolkits.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-03T01:50:55.283" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="409" PostTypeId="2" ParentId="327" CreationDate="2017-06-03T09:06:32.320" Score="7" Body="&lt;p&gt;We routinely run both &lt;a href=&quot;http://www.bioinformatics.babraham.ac.uk/projects/fastqc/&quot; rel=&quot;noreferrer&quot;&gt;FastQC&lt;/a&gt; and &lt;a href=&quot;http://www.bioinformatics.babraham.ac.uk/projects/fastq_screen/&quot; rel=&quot;noreferrer&quot;&gt;FastQ Screen&lt;/a&gt; on all of our raw sequencing reads. FastQ Screen is a tool for detecting cross-species contamination. &lt;a href=&quot;https://github.com/crukci-bioinformatics/MGA&quot; rel=&quot;noreferrer&quot;&gt;MGA&lt;/a&gt; is another similar tool.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are then lots of QC tools specific to different types of data, most of which run after alignment. For example &lt;a href=&quot;http://rseqc.sourceforge.net/&quot; rel=&quot;noreferrer&quot;&gt;RSeQC&lt;/a&gt; (RNA data), &lt;a href=&quot;http://qualimap.bioinfo.cipf.es/&quot; rel=&quot;noreferrer&quot;&gt;Qualimap&lt;/a&gt; and many many others. Without specifying what kind of data you have this is a bit difficult to make recommendations for though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Phil&lt;/p&gt;&#xA;" OwnerUserId="224" LastActivityDate="2017-06-03T09:06:32.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="410" PostTypeId="2" ParentId="283" CreationDate="2017-06-03T11:31:03.630" Score="2" Body="&lt;p&gt;Next to OmicsDI the EBI has a special repository for multi-omics datasets: &lt;a href=&quot;https://www.ebi.ac.uk/biosamples&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/biosamples&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It links the different datasets between repositories, ie. PRIDE for MS/MS based data and ArrayExpress for RNASeq data.&lt;/p&gt;&#xA;" OwnerUserId="476" LastActivityDate="2017-06-03T11:31:03.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="411" PostTypeId="1" CreationDate="2017-06-03T11:43:30.357" Score="17" ViewCount="4659" Body="&lt;p&gt;I'm trying to download three WGS datasets from the SRA that are each between 60 and 100GB in size.  So far I've tried:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fetching the .sra files directly from NCBI's ftp site&lt;/li&gt;&#xA;&lt;li&gt;Fetching the .sra files directly using the aspera command line (&lt;code&gt;ascp&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Using the SRA toolkit's &lt;code&gt;fastqdump&lt;/code&gt; and &lt;code&gt;samdump&lt;/code&gt; tools&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It's excruciatingly slow.  I've had three &lt;code&gt;fastqdump&lt;/code&gt; processes running in parallel now for approximately 18 hours.  They're running on a large AWS instance in the US east (Virginia) region, which I figure is about as close to NCBI as I can get.  In 18 hours they've downloaded a total of 33GB of data.  By my calculation that's ~500kb/s.  They do appear to still be running - the fastq files continue to grow and their timestamps continue to update.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this rate it's going to take me days or weeks just to download the datasets.  Surely the SRA must be capable of moving data at higher rates that this?  I've also looked, and unfortunately the datasets I'm interested have not been mirrored out to ENA or the Japanese archive, so it looks like I'm stuck working with the SRA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better way to fetch this data that wouldn't take multiple days?&lt;/p&gt;&#xA;" OwnerUserId="556" LastEditorUserId="1157" LastEditDate="2018-05-19T07:13:03.150" LastActivityDate="2023-08-03T10:26:47.190" Title="What's the best way to download data from the SRA? Is it really this slow?" Tags="|data-download|" AnswerCount="6" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="412" PostTypeId="2" ParentId="411" CreationDate="2017-06-03T12:10:48.387" Score="12" Body="&lt;p&gt;Proximity to NCBI may not necessarily give you the fastest transfer speed. AWS may be deliberately throttling the Internet connection to limit the likelihood that people will use it for undesirable things. There's a chance that a home network might be faster, but you're likely to get the fastest connection to NCBI by using an academic system that is linked to NCBI via a research network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility is using Aspera for downloads. This is unlikely to help if bandwidth is being throttled, but it might help if there's a bit of congestion through the regular methods:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/public/&quot; rel=&quot;noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/public/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NCBI also has an online &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK51062/?report=reader&quot; rel=&quot;noreferrer&quot;&gt;book&lt;/a&gt; about best practises for downloading data from their servers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a related note, just in case someone sees this and EBI/ENA is an option, there's a great guide for how to do file transfer using Aspera on the EBI web site:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ebi.ac.uk/ena/browse/read-download#downloading_files_aspera&quot; rel=&quot;noreferrer&quot;&gt;https://www.ebi.ac.uk/ena/browse/read-download#downloading_files_aspera&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Your command should look similar to this on Unix:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ascp -QT -l 300m -i &amp;lt;aspera connect installation directory&amp;gt;/etc/asperaweb_id_dsa.openssh era-fasp@fasp.sra.ebi.ac.uk:&amp;lt;file or files to download&amp;gt; &amp;lt;download location&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In my case, I've just started downloading some files from a MinION sequencing run. The estimated completion time via standard FTP was 12 hours for about 32GB of data; &lt;code&gt;ascp&lt;/code&gt; has reduced that estimated download time to about an hour. Here's the command I used for downloading:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ascp -QT -l 300m -i ~/.aspera/connect/etc/asperaweb_id_dsa.openssh era-fasp@fasp.sra.ebi.ac.uk:/vol1/ERA932/ERA932268/oxfordnanopore_native/20160804_Mock.tar.gz .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-05T02:20:43.573" LastActivityDate="2017-06-05T02:20:43.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="413" PostTypeId="2" ParentId="401" CreationDate="2017-06-03T12:14:11.773" Score="7" Body="&lt;p&gt;It's worth bearing in mind that when outputting compressed BAM, as most tools do by default, they may well be using different levels of compression and/or different libraries, or versions of said, libraries for doing (de)compression which will result in different file sizes.  Additionally coordinate sorted BAM will compress more than unsorted BAM. The current version of Picard uses &lt;a href=&quot;https://github.com/samtools/htsjdk&quot; rel=&quot;noreferrer&quot;&gt;HTSJDK&lt;/a&gt; which in turn uses java.util.zip.Deflater/Inflater, current versions of samtools should be using the &lt;a href=&quot;https://github.com/samtools/htslib&quot; rel=&quot;noreferrer&quot;&gt;HTSlib&lt;/a&gt; library which in turn depends on the &lt;a href=&quot;http://zlib.net/&quot; rel=&quot;noreferrer&quot;&gt;standard zlib library&lt;/a&gt;. You can see the effect of different implementations of zlib have on file size and execution time in benchmarking done by the &lt;a href=&quot;http://www.htslib.org/benchmarks/zlib.html&quot; rel=&quot;noreferrer&quot;&gt;samtools team&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in your case the best way to see if there is any difference between the BAM files is to rule out the effect of different levels of compression or libraries used for compression and save both BAM files as uncompressed.  Both samtools and Picard have options for disabling or changing the levels of compression, since the BAM compression standard BGZF is implemented on top of the gzip format it has inherited the ability, just like with gzip, to change the compression level from 0 to 9.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;samtools view -bu&lt;/code&gt; will allow you to produce uncompressed BAM output (which is also handy for piping into other programs as it saves time wasted compressing decompressing what is essentially a stream).  Also note that &lt;code&gt;samtools sort&lt;/code&gt; has a &lt;code&gt;-l INT&lt;/code&gt; setting where INT can be set between 0 (compression off, as with &lt;code&gt;-u&lt;/code&gt;) 1 (for fastest compression, but increased file size) or -9 (for maximal compression, with increased run time).  Some of the effects of increased runtime for higher compression settings might be ameliorated using the &lt;code&gt;-@&lt;/code&gt; argument which allows you to set the number of extra threads used for BAM compression, by default samtools won't use any.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Picard tools has a general setting &lt;code&gt;COMPRESSION_LEVEL&lt;/code&gt; which is applicable to most of its tools setting this to 0, &lt;code&gt;COMPRESSION_LEVEL=0&lt;/code&gt; should disable compression. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So re-running whatever Picard tool you used in the first instance with &lt;code&gt;COMPRESSION_LEVEL=0&lt;/code&gt; will then enable you to check that the file is not further modified by &lt;code&gt;samtools view -bu&lt;/code&gt;. The assumption here being that if both files have exactly the same content they should be the same size uncompressed, of course if they have trivial differences with regard to white space formatting things may still differ.&lt;/p&gt;&#xA;" OwnerUserId="1769" LastActivityDate="2017-06-03T12:14:11.773" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="414" PostTypeId="1" AcceptedAnswerId="415" CreationDate="2017-06-03T12:42:57.640" Score="7" ViewCount="940" Body="&lt;p&gt;I need to merge sequencing data from different sequencing runs but for the same ChiP-seq library (HiSeq 2000).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any potential advantages or disadvantages when merging files at .fastq or .BAM stage (alignment with Bowite/1.1.2)?&lt;/p&gt;&#xA;" OwnerUserId="375" LastActivityDate="2017-06-06T17:40:13.550" Title="Merging sequencing data for ChIP-seq experiments" Tags="|chip-seq|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="415" PostTypeId="2" ParentId="414" CreationDate="2017-06-03T12:48:05.047" Score="7" Body="&lt;p&gt;I don’t think it matters. Both are easy to merge (BAM via &lt;code&gt;samtools merge&lt;/code&gt;, and (gzipped) FASTQ via &lt;code&gt;cat&lt;/code&gt;), and neither method has specific disadvantages, unless your FASTQ files are sorted for some reason (but they generally shouldn’t be).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One advantage of keeping the FASTQ files separate is that it makes it slightly easier to parallelise the mapping step: just run the mapper in parallel on the separate FASTQ files. Although &lt;code&gt;bowtie&lt;/code&gt; has an option (&lt;code&gt;-p&lt;/code&gt;) for this, throughput from that is slightly worse than running the mapping on split files.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2017-06-03T12:48:05.047" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="416" PostTypeId="2" ParentId="407" CreationDate="2017-06-03T13:07:50.953" Score="5" Body="&lt;p&gt;GATK best practices are explicably meant to consume BWA MEM generated BAM.  Whilst SNAP may be faster, the Broad will not have tested it for compatibility with GATK as such you can't guaranty using it won't have unexpected consequences.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As such you'd be better off using BWA MEM because I assume accurately called variation is always better than fast and incorrectly called variation.  The main issue you'll have is ensuring shorter split hits and mapping quality are reported in the same way as &lt;code&gt;bwa MEM -M&lt;/code&gt; which GATK/Picard is expecting.  Ultimately however you'd be better off posting this question on the &lt;a href=&quot;http://gatkforums.broadinstitute.org/gatk&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATK forum&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's also worth noting that the soon to be released GATK 4 will utilise bwaspark which can distribute it's alignment processes across Apache Spark for increase performance.  Consequently I can't see SNAP being adopted anytime soon.&lt;/p&gt;&#xA;" OwnerUserId="1769" LastEditorUserId="1769" LastEditDate="2017-06-04T00:35:38.640" LastActivityDate="2017-06-04T00:35:38.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="417" PostTypeId="2" ParentId="328" CreationDate="2017-06-03T13:36:28.887" Score="4" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Persistent_uniform_resource_locator&quot; rel=&quot;nofollow noreferrer&quot;&gt;Persistent uniform resource locator or PURL&lt;/a&gt; is one such solution, these are designed to be a bit more robust than permalinks in so much as they are supposed to survive the change of domain name.  The bio ontology community already use them &lt;a href=&quot;http://purl.bioontology.org/docs/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://purl.bioontology.org/docs/index.html&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="1769" LastActivityDate="2017-06-03T13:36:28.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="418" PostTypeId="1" AcceptedAnswerId="433" CreationDate="2017-06-03T13:58:26.980" Score="15" ViewCount="4581" Body="&lt;p&gt;The industry standard for aligning short reads seems to be bwa-mem. However, in my tests I have seen that using bwa backtrack (bwa-aln + bwa-sampe + bwa-samse) performs better. It is slightly slower, but gives significantly better results in terms of both sensitivity and specificity. I have tested it using the genome in a bottle data and public samples (NA12878 and NA12877 among others) and found that backtrack consistently outperformed bwa-mem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So why is bwa-mem the standard? Am I wrong and other tests have shown the opposite? I don't really see how since I tested using the most common datasets and validation data. Is it that the slight increase in efficiency outweighs the decrease in performance? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only other explanation I can see is that bwa backtrack is designed specifically for Illumina reads and all my tests have been on Illumina data. Is it just that bwa-mem is &quot;sequencer agnostic&quot;? So that we can use the same algorithm irrespective of what sequencing platform is used? In that case, it makes sense to use backtrack for if we only deal with Illumina data and mem if we can have different sequencers. But, if so, seeing as Illumina is so widespread, why isn't backtrack used more often on Illumina data? I feel I must be missing something.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="96" LastEditDate="2017-08-31T03:23:56.190" LastActivityDate="2017-08-31T03:23:56.190" Title="Why is bwa-mem the standard algorithm when using bwa?" Tags="|ngs|read-mapping|bwa|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="420" PostTypeId="2" ParentId="351" CreationDate="2017-06-03T16:18:46.493" Score="3" Body="&lt;p&gt;I have previously estimated tumour purity with the &lt;a href=&quot;http://dna-discovery.stanford.edu/software/expands/&quot; rel=&quot;nofollow noreferrer&quot;&gt;EXPANDS&lt;/a&gt; an inferred tumour heterogeneity program which is designed to calculate the number of clonal subpopulations in matched tumour/normal samples.  The purity is essentially the size of the largest subpopulation identified in that sample - this is discussed in the programs &lt;a href=&quot;http://dna-discovery.stanford.edu/software/expands/expands_faq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;FAQ&lt;/a&gt;.  In addition to a matched tumour / normal exome or genome sequencing you'll also need match somatic copy number as a seg file as input.  Other programs also exist for this sort of inferred heterogeneity analysis - some of which may also give you a measure of purity are: &lt;a href=&quot;https://www.nature.com/nbt/journal/v30/n5/full/nbt.2203.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Absolute&lt;/a&gt;, &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/gb-2013-14-7-r80&quot; rel=&quot;nofollow noreferrer&quot;&gt;ThetA&lt;/a&gt;, &lt;a href=&quot;http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003665&quot; rel=&quot;nofollow noreferrer&quot;&gt;SciClone&lt;/a&gt;, &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-014-0473-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;CHAT&lt;/a&gt;, &lt;a href=&quot;https://www.nature.com/nmeth/journal/v11/n4/full/nmeth.2883.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyClone&lt;/a&gt; and &lt;a href=&quot;http://www.pnas.org/content/113/37/E5528.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;Canopy&lt;/a&gt;. A more complete list looks to be &lt;a href=&quot;https://omictools.com/tumor-purity-and-heterogeneity-category&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only other thing I'd suggest is have an estimate via some orthogonal measure of the purity to try at least judge how the different methods are performing with you data, &lt;em&gt;e&lt;/em&gt;.&lt;em&gt;g&lt;/em&gt;. you might have clonality indications from &lt;a href=&quot;https://en.wikipedia.org/wiki/Cytogenetics&quot; rel=&quot;nofollow noreferrer&quot;&gt;cytogenetics/FISH&lt;/a&gt; or histology work, or perhaps &lt;a href=&quot;https://en.wikipedia.org/wiki/Flow_cytometry#Fluorescence-activated_cell_sorting_.28FACS.29&quot; rel=&quot;nofollow noreferrer&quot;&gt;FACS&lt;/a&gt;.  Picking samples known to be pure / impure via one or more of these might help you get a handle on how well various methods are performing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As regards estimation without a normal sample the only program which looks like it might help is &lt;a href=&quot;https://github.com/DeveauP/QuantumClone/&quot; rel=&quot;nofollow noreferrer&quot;&gt;QuantumClone&lt;/a&gt;, but that requires more than one tumour sample form said patient either spatially or temporally distinct to perform the analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also If you have low coverage DNA sequencing &lt;a href=&quot;http://www.precancer.leeds.ac.uk/software-and-datasets/cnanorm/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CNAnorm&lt;/a&gt; looks like it could be useful with just a single sample. &lt;/p&gt;&#xA;" OwnerUserId="1769" LastActivityDate="2017-06-03T16:18:46.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="421" PostTypeId="2" ParentId="404" CreationDate="2017-06-03T18:39:37.780" Score="10" Body="&lt;p&gt;The SAM format originally had only &lt;strong&gt;M&lt;/strong&gt;, &lt;strong&gt;I&lt;/strong&gt;, &lt;strong&gt;D&lt;/strong&gt;, &lt;strong&gt;N&lt;/strong&gt;, &lt;strong&gt;S&lt;/strong&gt;, &lt;strong&gt;H&lt;/strong&gt;, and &lt;strong&gt;P&lt;/strong&gt; CIGAR operators.  See the &lt;a href=&quot;https://github.com/samtools/hts-specs/tree/a9b16d4c54d8b2bf900b210b8319474163d7ce2a&quot; rel=&quot;nofollow noreferrer&quot;&gt;original SAM specification&lt;/a&gt; (if you can view Apple Pages documents) and Table 1 in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/19505943&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;The Sequence Alignment/Map format and SAMtools&lt;/em&gt;&lt;/a&gt; (Li &lt;em&gt;et al&lt;/em&gt;, 2009).  This was in line with previous tools using CIGAR strings, notably &lt;a href=&quot;http://www.ebi.ac.uk/about/vertebrate-genomics/software/exonerate&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;exonerate&lt;/em&gt;&lt;/a&gt; which introduced them with just the &lt;strong&gt;M&lt;/strong&gt;, &lt;strong&gt;I&lt;/strong&gt;, and &lt;strong&gt;D&lt;/strong&gt; operators.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BWA-backtrack was written contemporaneously with the SAM format in 2008 and 2009 (and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/19451168&quot; rel=&quot;nofollow noreferrer&quot;&gt;published in May 2009&lt;/a&gt;).  Its &lt;em&gt;ChangeLog&lt;/em&gt; shows that it was outputting mismatch information in an &lt;strong&gt;MD&lt;/strong&gt; tag from January 2009 and that &lt;strong&gt;MD&lt;/strong&gt; was defined in the SAM specification of the time (and that the tag value's syntax was somewhat in flux in February 2009).  The &lt;strong&gt;MD&lt;/strong&gt; tag is also described in that early v1.0 Pages-formatted SAM specification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;=&lt;/strong&gt; and &lt;strong&gt;X&lt;/strong&gt; CIGAR operators were introduced later in &lt;a href=&quot;https://github.com/samtools/hts-specs/commit/07dc1c67a717a6c5cc9d65eeb8b3c99612744cde&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAM v1.3&lt;/a&gt; as a result of &lt;a href=&quot;https://sourceforge.net/p/samtools/mailman/samtools-devel/thread/ee957a250907301016y22c96c5x2907a33c2e719da2%40mail.gmail.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this lengthy samtools-devel mailing list thread&lt;/a&gt;.  The characters used for the operators and the initial implementations were essentially in place by November 2009.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since then, the &lt;strong&gt;=&lt;/strong&gt;/&lt;strong&gt;X&lt;/strong&gt; operators have not really taken over from &lt;strong&gt;M&lt;/strong&gt;, as you've seen.  There's probably a number of factors contributing to this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Their later introduction, long after the basic &lt;strong&gt;M&lt;/strong&gt;/&lt;strong&gt;I&lt;/strong&gt;/&lt;strong&gt;D&lt;/strong&gt; operators and &lt;strong&gt;MD&lt;/strong&gt; tag were well-established;&lt;/li&gt;&#xA;&lt;li&gt;Conceivably their being specific to SAM and unavailable in other CIGAR flavours;&lt;/li&gt;&#xA;&lt;li&gt;The &lt;strong&gt;MD&lt;/strong&gt; tag still provides more information — &lt;strong&gt;X&lt;/strong&gt; doesn't tell you what the mismatched reference bases were.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="134" LastEditorUserId="134" LastEditDate="2017-06-08T09:10:32.403" LastActivityDate="2017-06-08T09:10:32.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="422" PostTypeId="1" CreationDate="2017-06-03T20:57:33.417" Score="7" ViewCount="1520" Body="&lt;p&gt;I've come across a bit of confusion about the initialism NGS, so think it would be a good idea to clarify this term (and similar terms like 2GS, SBS, and HTS) for this site with a bit of discussion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the most common initialisms used to describe different sequencing technologies?&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-05T13:19:33.923" LastActivityDate="2019-01-15T15:15:20.917" Title="What is the difference between NGS, 2GS, SBS and HTS?" Tags="|ngs|terminology|hts|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="423" PostTypeId="2" ParentId="422" CreationDate="2017-06-03T20:57:33.417" Score="5" Body="&lt;p&gt;Here are my attempts at definitions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Sanger_sequencing&quot; rel=&quot;noreferrer&quot;&gt;Sanger&lt;/a&gt;: A method of sequencing that depends on chain-terminatiing dideoxynucleotides. This sequencing uses the differential flow of DNA sequences of different lengths through a gel to determine the original DNA sequence, producing a single sequence per reaction container.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NGS: Next-generation sequencing, also referred to as 2GS (second-generation sequencing). This term is used to describe the first wave of sequencing technologies that followed Sanger sequencing technology. The use of NGS has become more confusing with the advent of long-read sequencing, because it's a common assumption that &quot;next-generation&quot; refers to the most recent technology (which is incorrect in this case).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;HTS: High-throughput sequencing. This term describes any type of sequencing technology that produces large amounts of data, usually in the form of millions of different sequences produced from the same sequencing run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SBS: Sequencing by synthesis. This term describes a method of sequencing that depends on the synthesis of [DNA] bases in order for sequencing to be carried out. This definition can extend into long-read sequencing (e.g. PacBio sequencers depend on synthesis during sequencing), but is more typically associated with only the second-generation sequencing technology.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-03T20:57:33.417" CommentCount="6" CommunityOwnedDate="2017-06-04T07:42:56.527" ContentLicense="CC BY-SA 3.0" />
  <row Id="424" PostTypeId="1" CreationDate="2017-06-03T23:01:33.863" Score="6" ViewCount="168" Body="&lt;p&gt;Blast reports E-values, but short-read mappers report mapping qualities. Are they the same thing? Can they be converted to each other? If not, why blast doesn't report mapping quality while short-read mappers do not report E-values?&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="73" LastEditDate="2017-06-04T10:57:51.070" LastActivityDate="2017-06-04T10:57:51.070" Title="What is the difference between SAM mapping quality and Blast E-value?" Tags="|terminology|blast|read-mapping|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="425" PostTypeId="2" ParentId="283" CreationDate="2017-06-03T23:06:42.720" Score="3" Body="&lt;p&gt;There are several &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/browse/?view=series&amp;amp;display=20&amp;amp;zsort=samples&quot; rel=&quot;nofollow noreferrer&quot;&gt;datasets available on GEO&lt;/a&gt;, though you do have to search for them. For example, here are three data sets that have both Illumina methylation and gene expression microarray profiling:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE87650&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSE87650&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE72874&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSE72874&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE56047&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSE56047&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="563" LastActivityDate="2017-06-03T23:06:42.720" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="426" PostTypeId="1" AcceptedAnswerId="430" CreationDate="2017-06-03T23:10:32.853" Score="4" ViewCount="376" Body="&lt;p&gt;I learned that GATK 4 is using &lt;a href=&quot;https://spark.apache.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spark&lt;/a&gt; for parallelization. I googled around, though I am still not quite sure how spark really works and how to use it in practice. Besides GATK 4, are any other bioinformatics tool using spark? Generally, is spark widely used? Is it a necessary skill to learn? Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="549" LastEditorUserId="292" LastEditDate="2017-06-05T10:13:33.140" LastActivityDate="2017-06-06T21:40:05.000" Title="Is spark widely used in bioinformatics?" Tags="|gatk|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="427" PostTypeId="1" CreationDate="2017-06-03T23:19:39.530" Score="3" ViewCount="507" Body="&lt;p&gt;I was given a list of target regions in BED and many exome alignments in BAM. I was asked to extract on-target alignments from these BAMs to save disk space. I know I can use bedtools to extract sub-BAMs. I am thinking to write a script to apply bedtools to all BAMs at my hand, but I speculate there may be some more convenient command lines to achieve my goal. How would you do? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="549" LastActivityDate="2017-06-04T11:30:33.870" Title="How to extract exome on-target reads in batch?" Tags="|sam|linux|bedtools|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="428" PostTypeId="2" ParentId="191" CreationDate="2017-06-04T01:17:18.017" Score="4" Body="&lt;p&gt;I don't have enough experience to answer which probabilistic distribution should be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this questions also also asks how to estimate parameters of the distributions. If a binomial distribution is chosen, then Heng Li's paper titled &quot;A statistical framework for SNP calling, mutation discovery, association mapping and population genetical parameter estimation from sequencing data&quot; &lt;a href=&quot;https://doi.org/10.1093/bioinformatics/btr509&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; is probably the definitive one. Section 2.3.1 of that paper describes an EM algorithm for estimating allele frequencies from multiple samples under the assumption of Hardy-Weinberg equilibrium for arbitrary but constant ploidy.&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2017-06-04T01:17:18.017" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="429" PostTypeId="2" ParentId="427" CreationDate="2017-06-04T01:31:45.110" Score="2" Body="&lt;p&gt;It sounds like &lt;code&gt;bedtools intersect&lt;/code&gt; will work for you:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools intersect -wa -a &amp;lt;alignment.bam&amp;gt; -b &amp;lt;region.bed&amp;gt; &amp;gt; &amp;lt;intersect_alignment.bam&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When a BAM file is used for the A file, the alignment is retained if overlaps exist, and exlcuded if an overlap cannot be found.  If multiple overlaps exist, they are not reported, as we are only testing for one or more overlaps.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's easy enough to wrap this up into a &lt;code&gt;for&lt;/code&gt; loop for batch processing (line breaks can be removed if desired):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; for alnBam in alignmentFiles*.bam;&#xA;   do echo ${alnBam};&#xA;   bedtools intersect -wa -a ${alnBam} -b &amp;lt;region.bed&amp;gt; &amp;gt; intersect_${alnBam};&#xA; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are a lot of other options which should cover most related uses. For more details, just run &quot;bedtools intersect&quot; (with no arguments).&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-04T11:30:33.870" LastActivityDate="2017-06-04T11:30:33.870" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="430" PostTypeId="2" ParentId="426" CreationDate="2017-06-04T01:33:49.913" Score="8" Body="&lt;p&gt;The only bioinformatics tool other than GATK4 that I am aware of that uses spark is &lt;a href=&quot;https://github.com/hail-is/hail&quot; rel=&quot;noreferrer&quot;&gt;Hail&lt;/a&gt; (a Spark based replacement for Plink). Hail is also supported by researchers at the Broad.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most places I have worked at have not switched over to Spark.  As such, I don't think it is widely used generally. Therefore, I don't think knowing Spark qualifies as a necessary skill to learn for bioinformatics at this time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I think Spark is a superior paradigm for handling the large amount of data that bioinformaticians routinely deal with, and I think we will soon see the field move towards using it more.  Any bioinformatician would do well to acquaint themselves with Spark and to play around with Hail.&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2017-06-04T01:33:49.913" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="431" PostTypeId="5" CreationDate="2017-06-04T07:23:24.257" Score="0" Body="&lt;h2&gt;&lt;code&gt;minion&lt;/code&gt; -- The Oxford Nanopore MinION&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;A small, portable sequencer that uses electrical current flowing through small molecules (e.g. DNA nucleotides) to determine the underlying sequence.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;What questions should have this tag?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Anything that is associated with the generation or analysis of data from the MinION, e.g. long-read FASTQ files, raw signal data, nanopore sample prep QC. If questions are not specific to the &lt;em&gt;MinION&lt;/em&gt; (e.g. also applicable to PromethION), then the &lt;code&gt;nanopore&lt;/code&gt; tag should be used.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Brief Introduction&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The MinION's flow cell is comprised of 2048 wells containing a membrane perforated by nanopores. Ligated with a molecular motor, a single stranded DNA molecule passes through the pore, altering the recorded current. After the electronic sequencing is carried out, a software basecalling algorithm transforms the current trace into a modelled DNA sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantages of the MinION are rapid library preparation, portability (&lt;a href=&quot;http://dx.doi.org/10.1016/j.jbiotec.2016.12.006&quot; rel=&quot;nofollow noreferrer&quot;&gt;Walter &lt;em&gt;et al.&lt;/em&gt;, 2016&lt;/a&gt;; &lt;a href=&quot;http://biorxiv.org/content/early/2016/09/27/077651&quot; rel=&quot;nofollow noreferrer&quot;&gt;Castro-Wallace &lt;em&gt;et al.&lt;/em&gt;, 2016&lt;/a&gt;), long molecule sequencing (&lt;a href=&quot;http://dx.doi.org/10.1101/019281&quot; rel=&quot;nofollow noreferrer&quot;&gt;Urban &lt;em&gt;et al.&lt;/em&gt;, 2015&lt;/a&gt;), and sequencing of non-model modifications of the DNA strand (&lt;a href=&quot;http://dx.doi.org/10.1038/nmeth.4184&quot; rel=&quot;nofollow noreferrer&quot;&gt;Simpson &lt;em&gt;et al.&lt;/em&gt;, 2017&lt;/a&gt;). With the recent improvement in the chemistry of the MinION, Oxford Nanopore has overcome the majority of issues associated with low yield and high error rates that have limited the range of its application. The MinION sequencing device has now been successfully applied to sequence genomes of a wide range of sizes, from bacterial and viral genomes (&lt;a href=&quot;http://dx.doi.org/10.1038/srep28625&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deschamps &lt;em&gt;et al.&lt;/em&gt;, 2016&lt;/a&gt;; &lt;a href=&quot;http://dx.doi.org/10.1101/098913&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quick &lt;em&gt;et al.&lt;/em&gt;, 2017&lt;/a&gt;), amplicon sequencing like bacterial 16S rRNA sequencing (&lt;a href=&quot;http://dx.doi.org/10.1186/s13742-016-0111-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;Benitez-paez &lt;em&gt;et al.&lt;/em&gt;, 2016&lt;/a&gt;), and more recently a human genome (&lt;a href=&quot;http://dx.doi.org/10.1101/128835&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jain &lt;em&gt;et al.&lt;/em&gt;, 2017&lt;/a&gt;). The MinION has also been used for cDNA sequencing (&lt;a href=&quot;http://dx.doi.org/10.7717/peerj.1441&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hargreaves &lt;em&gt;et al.&lt;/em&gt;, 2015&lt;/a&gt;), for detecting DNA methylation patterns without chemical treatment (&lt;a href=&quot;http://dx.doi.org/10.1038/nmeth.4184&quot; rel=&quot;nofollow noreferrer&quot;&gt;Simpson &lt;em&gt;et al.&lt;/em&gt;, 2017&lt;/a&gt;; &lt;a href=&quot;http://dx.doi.org/10.1038/nmeth.4189&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rand &lt;em&gt;et al.&lt;/em&gt;, 2017&lt;/a&gt;), and for direct RNA sequencing with detection of modified 16S rRNA nucleotides (&lt;a href=&quot;http://dx.doi.org/10.1101/132274&quot; rel=&quot;nofollow noreferrer&quot;&gt;Smith &lt;em&gt;et al.&lt;/em&gt;, 2017&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[CC-by-4.0 text source by White &lt;em&gt;et al.&lt;/em&gt;, 2017 &lt;a href=&quot;https://f1000research.com/articles/6-631/v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;]&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Important links for learning more&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://nanoporetech.com/how-it-works&quot; rel=&quot;nofollow noreferrer&quot;&gt;Summary of nanopore DNA sequencing&lt;/a&gt; (official ONT website)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://f1000research.com/gateways/nanoporeanalysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;F1000 Research Nanopore Analysis gateway&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-15T13:53:23.610" LastActivityDate="2017-06-15T13:53:23.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="432" PostTypeId="4" CreationDate="2017-06-04T07:23:24.257" Score="0" Body="Questions associated with the generation or analysis of data from sequencer MinION. For questions about long reads (e.g. including PacBio), use `long-reads` instead. For questions that are more generally about nanopore devices (e.g. including PromethION), use `nanopore` instead." OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-15T13:54:11.900" LastActivityDate="2017-06-15T13:54:11.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="433" PostTypeId="2" ParentId="418" CreationDate="2017-06-04T07:38:44.210" Score="7" Body="&lt;p&gt;&lt;code&gt;bwa mem&lt;/code&gt; is newer, faster, and [should be] more accurate, particularly for longer reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the &lt;code&gt;bwa&lt;/code&gt; man page (presumably in Heng Li's own words):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;BWA is a software package for mapping low-divergent sequences against a large reference genome, such as the human genome. It consists of three algorithms:  BWA-backtrack, BWA-SW and BWA-MEM. The first algorithm  is designed for Illumina sequence reads up to 100bp, while the rest two for longer sequences ranged from 70bp to 1Mbp. BWA-MEM and BWA-SW share similar  features such as long-read support and split alignment, but &lt;strong&gt;&lt;em&gt;BWA-MEM, which is the latest, is generally recommended for high-quality queries as it is faster and  more  accurate.&lt;/em&gt;&lt;/strong&gt; BWA-MEM also has better performance than BWA-backtrack for 70-100bp Illumina reads.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-04T07:38:44.210" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="434" PostTypeId="1" CreationDate="2017-06-04T09:28:02.797" Score="2" ViewCount="160" Body="&lt;p&gt;Our research institute processes a lot of flow cytometry data, but the produced data is under-utilised due to the effort required to process it. A typical run will produce 5 million events (ideally one event per cell), with up to 14 dimensions of [ideally] log-normal fluorescence values for particular groups of cells (&quot;populations&quot;). Due to various systematic errors, negative values, scatter, and a non-zero &quot;zero&quot; value can happen, but I'm going to ignore those for the purpose of this question and assume that the data are well-distributed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Researchers will typically probe these data using manual filters (set up specifically for each experiment) to find populations of interest. I suppose a picture might help. This one shows three identifiable cell populations, in order of size one at about (X/Ly6C:2,Y/CD86:2), one at about (4.5,3), and one small population at (2,4).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ErbhE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ErbhE.png&quot; alt=&quot;CD86 vs Ly6C flow plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's another plot that has two cell populations that are close to each other, such that the population &quot;humps&quot; overlap substantially.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jJ72F.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jJ72F.png&quot; alt=&quot;B220 vs CD11b flow plot&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Manual filters are typically used because it can be very difficult to distinguish between a noisy data point from a large cell population and a less noisy data point from a small cell population, particularly when considering populations that make up about 0.01% of the total cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an additional complication, counting these populations can be difficult when populations overlap (as in the second image). A filter/slice through the plot that separates populations could count many cells as being members of the wrong population.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can these populations be detected in an automated way? If it is assumed that the populations are spread in a gaussian fashion at some point in each dimension, is there some method that can be used to approximate the number of cells in each population, even when populations are close by?&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="96" LastEditDate="2019-01-15T15:00:35.647" LastActivityDate="2019-01-15T15:00:35.647" Title="Finding peaks and estimating cell population sizes in multi-dimensional flow cytometry data" Tags="|flow-cytometry|peak-calling|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="435" PostTypeId="2" ParentId="424" CreationDate="2017-06-04T10:24:42.397" Score="9" Body="&lt;p&gt;The E-value and the mapping qualities are two very different things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The E-value is &quot;a parameter that describes the number of hits one can 'expect' to see by chance when searching a database of a particular size&quot;. More details can be found here: &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&amp;amp;PAGE_TYPE=BlastDocs&amp;amp;DOC_TYPE=FAQ#expect&quot; rel=&quot;noreferrer&quot;&gt;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&amp;amp;PAGE_TYPE=BlastDocs&amp;amp;DOC_TYPE=FAQ#expect&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mapping quality is an attempt to estimate the probability that a given base from a given read is mapped correctly to a particular place in the reference genome. Different aligners calculate mapping qualities very differently, so there is no simple way to compare them across aligners. But in general the mapping quality will consider things like: (i) the quality of the base call; (ii) the repeat structure of the reference; (iii) the alignment algorithm; (iv) whether the read has a mapped pair; (v) anything else the author of the mapping software thought might help. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;E-values and mapping qualities are, therefore, measuring two fundamentally different properties with very different uses.&lt;/p&gt;&#xA;" OwnerUserId="156" LastActivityDate="2017-06-04T10:24:42.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="436" PostTypeId="2" ParentId="434" CreationDate="2017-06-04T11:13:06.240" Score="2" Body="&lt;p&gt;There are quite a few algorithms developed for the automatic classification of multidimensional flow cytometry data, you can see a (not so recent?) review here: &lt;a href=&quot;http://www.nature.com/nmeth/journal/v10/n3/full/nmeth.2365.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.nature.com/nmeth/journal/v10/n3/full/nmeth.2365.html&lt;/a&gt;. In your case, you are interested in the &lt;strong&gt;unsupervised&lt;/strong&gt; methods, since you do not have data from single populations (which would provide you &quot;labels&quot; associated to each sample), as I understood. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, if you have your data set as a matrix of 5 million samples (cells) with 14 dimensions (FC values) each, then you are looking basically for algorithms that will do a &lt;em&gt;clustering&lt;/em&gt; of the samples into groups. By the way your description of each sample belonging to a multivariate Normal distribution leads to a &lt;strong&gt;Gaussian Mixture Model&lt;/strong&gt;, which can be estimated from your data using e.g. the scikit-learn library for python: &lt;a href=&quot;http://scikit-learn.org/stable/modules/mixture.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://scikit-learn.org/stable/modules/mixture.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="45" LastActivityDate="2017-06-04T11:13:06.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="437" PostTypeId="2" ParentId="434" CreationDate="2017-06-04T12:09:34.170" Score="2" Body="&lt;p&gt;Automated gating methods are gaining popularity over manual and time consuming analysis in say &lt;a href=&quot;https://www.flowjo.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FlowJo&lt;/a&gt;, you should take a look at the &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/openCyto.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;openCyto&lt;/a&gt; bioconductor package this is a framework which builds on top of existing Bioconductor infrastructure for flow cytometry &lt;a href=&quot;http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003806&quot; rel=&quot;nofollow noreferrer&quot;&gt;(PLOS compbio paper)&lt;/a&gt; with the aim of making various analysis routes more accessible to the researcher by assisting getting data into and out various gating algorithms. You can find a nice intro &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/vignettes/openCyto/inst/doc/openCytoVignette.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; which starts with importing a manual gating scheme from FlowJo and a how-to on 1D and 2D gating can also be found &lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/openCyto/inst/doc/HowToAutoGating.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An excellent YouTube intro to analysing flow data in R by Ryan Duggan &lt;a href=&quot;https://www.youtube.com/watch?v=_B7mo6dB3BU&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cytometry on Air&lt;/a&gt; is worth watching especially for those which have no R experience and are curious as to what can be achieved.  Additionally it's worth having a look at the growing list of Bioconductor packages and datasets that exists for flow data: &lt;a href=&quot;https://www.bioconductor.org/help/search/index.html?q=cytometry/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioconductor.org/help/search/index.html?q=cytometry/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1769" LastActivityDate="2017-06-04T12:09:34.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="438" PostTypeId="1" AcceptedAnswerId="447" CreationDate="2017-06-04T13:21:05.390" Score="6" ViewCount="225" Body="&lt;p&gt;Working on various cancers I have an interest in detecting structural variation (SV) in human, we've successfully used various tools like &lt;a href=&quot;http://gmt.genome.wustl.edu/packages/pindel/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pindel&lt;/a&gt;, &lt;a href=&quot;http://svdetect.sourceforge.net/Site/Home.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SVDetect&lt;/a&gt;, &lt;a href=&quot;https://github.com/Illumina/manta&quot; rel=&quot;nofollow noreferrer&quot;&gt;Manta&lt;/a&gt;, and &lt;a href=&quot;https://github.com/arq5x/lumpy-sv&quot; rel=&quot;nofollow noreferrer&quot;&gt;LUMPY&lt;/a&gt;, to name a few for detecting SVs in illumina short-read sequencing.  I curious if anyone has successfully used ONTs &lt;a href=&quot;https://nanoporetech.com/products/minion&quot; rel=&quot;nofollow noreferrer&quot;&gt;MinION&lt;/a&gt; sequencer for detecting SV, as there are many cases where longer reads would be beneficial for SV detection especially where events are long and/or occur in repetitive regions.  Has anyone tried or had success with tools previously designed for PacBio data such as &lt;a href=&quot;https://github.com/fritzsedlazeck/Sniffles&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sniffles&lt;/a&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Supplementary question, obviously the MinION has not quite got the throughput of say the PacBio SEQUEL so can't generate low-pass coverage for the whole human genome as easily, has anyone got experience trying to generate reads targeting specific areas of the genome in areas that are known for translations, long inversions, duplications, &lt;em&gt;etc&lt;/em&gt;?&lt;/p&gt;&#xA;" OwnerUserId="1769" LastEditorUserId="73" LastEditDate="2017-06-08T05:32:58.860" LastActivityDate="2017-06-08T05:32:58.860" Title="Detecting structural variants with MinION data" Tags="|nanopore|structural-variation|long-reads|cancer|" AnswerCount="1" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="439" PostTypeId="2" ParentId="378" CreationDate="2017-06-04T13:31:13.200" Score="4" Body="&lt;p&gt;If I remember correctly Ewan Birney's Dynamite (a &lt;a href=&quot;https://en.wikipedia.org/wiki/Compiler-compiler&quot; rel=&quot;nofollow noreferrer&quot;&gt;compiler-compiler&lt;/a&gt;) as presented at &lt;a href=&quot;https://pdfs.semanticscholar.org/2345/f68bb0b2f079f9f201034da462d69a7817b1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ISMB 1997&lt;/a&gt; had this functionality, there is also some code here on GitHub &lt;a href=&quot;https://github.com/birney/wise3&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/birney/wise3&lt;/a&gt; which at least mentions Dynamite.  Suspect Ewan is too busy these days to work on this, although he has tweeted about blowing dust of his old Dynamite, sorry Dynamite code: &lt;a href=&quot;https://twitter.com/ewanbirney/status/788121636973142016&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://twitter.com/ewanbirney/status/788121636973142016&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1769" LastEditorUserId="1769" LastEditDate="2017-06-04T13:37:40.803" LastActivityDate="2017-06-04T13:37:40.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="440" PostTypeId="2" ParentId="351" CreationDate="2017-06-04T13:37:19.093" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/mathieu-lemire/celluloid_0.11&quot; rel=&quot;nofollow noreferrer&quot;&gt;Celluloid&lt;/a&gt; emits copy number profiles and tumour purity / ploidy information.  There's a nice tutorial https://github.com/mathieu-lemire/celluloid_0.11_tutorial. One thing to keep in mind is that celluloid will find multiple solutions and taking advantage of its plotting tools is essential to determining which solution is correct. Typically the first solution is correct, but occasionally it may not be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another tool similar is &lt;a href=&quot;http://genome.cshlp.org/content/early/2014/07/24/gr.180281.114&quot; rel=&quot;nofollow noreferrer&quot;&gt;TITAN&lt;/a&gt;, which may also require hand annotated the ideal solution.&lt;/p&gt;&#xA;" OwnerUserId="64" LastActivityDate="2017-06-04T13:37:19.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="441" PostTypeId="1" CreationDate="2017-06-04T13:50:23.627" Score="10" ViewCount="962" Body="&lt;p&gt;Some of the work in our lab requires a comparison of a strain across several experimental conditions. We are looking to identify most similar experimental conditions based on the gene transcription response similarity from the cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While we could easily invent and create home-grown methods to do it, their implementation and testing are a laborious project in themselves and are outside the scope of our current work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any methods for RNA expression profile similarity computation that has been already published? If yes, what is your experience using them?&lt;/p&gt;&#xA;" OwnerUserId="569" LastEditorUserId="163" LastEditDate="2017-06-07T06:19:11.957" LastActivityDate="2017-12-07T13:27:28.263" Title="What methods exist to calculate RNA expression profile similarity" Tags="|rna-seq|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="442" PostTypeId="1" CreationDate="2017-06-04T14:13:44.537" Score="5" ViewCount="324" Body="&lt;p&gt;I use SOAPdenovo2 to assemble a large genome (4.8G) using ~20X paired-end reads. The  total length of contig sizes is 6.3G while total length of scaffolds is 2.7G. Note that this is a haploid genome, so there is no issue of heterozygosity for scaffolding. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering what happened during the scaffolding procedure? How comes that sum of scaffolds is so much smaller than sum of contigs?&lt;/p&gt;&#xA;" OwnerUserId="570" LastEditorUserId="73" LastEditDate="2017-06-05T02:22:40.943" LastActivityDate="2017-06-05T10:36:44.423" Title="What causes the difference in total length of assembled contigs and scaffolds in SOAPdenovo2?" Tags="|assembly|scaffold|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="443" PostTypeId="2" ParentId="327" CreationDate="2017-06-04T17:58:23.480" Score="5" Body="&lt;p&gt;The quality control of ngs reads is heavily dependent on type of the project. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;For genome assembly&lt;/strong&gt; projects based on short reads, beside already covered checking quality of sequencing, you would like to look at the kmer spectra to find out, if your reads are going to make sense when they will be translated to De Brujin graph. You will get also a clues about genome coverage, genome size, repetitive content and small clue about heterozygocity. A lot of useful info about interpretation you can find in the README of GenomeScope.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A list tools I used:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.genome.umd.edu/jellyfish.html&quot; rel=&quot;noreferrer&quot;&gt;jellyfish&lt;/a&gt; - to count k-mer frequencies&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/schatzlab/genomescope&quot; rel=&quot;noreferrer&quot;&gt;GenomeScope&lt;/a&gt; - a package for analysis and visualisation of k-mer frequencies (they recommend to use jellyfish for counting k-mer frequencies)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://kmergenie.bx.psu.edu/&quot; rel=&quot;noreferrer&quot;&gt;kmergenie&lt;/a&gt; - for prediction of optimal kmer for assembly&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Using these tools can save you a lot of frustration if you accidentally sequence a contaminated sample or if your colleagues / a sequencing company have sent you a wrong file!&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2017-06-04T17:58:23.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="444" PostTypeId="2" ParentId="426" CreationDate="2017-06-04T20:22:22.863" Score="6" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/bigdatagenomics/adam&quot; rel=&quot;noreferrer&quot;&gt;ADAM&lt;/a&gt; and &lt;a href=&quot;https://github.com/bigdatagenomics/avocado&quot; rel=&quot;noreferrer&quot;&gt;avocado&lt;/a&gt; are Spark-based alignment and variant calling tools under active development by a collaboration (&lt;a href=&quot;http://bdgenomics.org/&quot; rel=&quot;noreferrer&quot;&gt;http://bdgenomics.org&lt;/a&gt;) that also includes the Broad, but I do not believe they have wide adoption.&lt;/p&gt;&#xA;" OwnerUserId="462" LastEditorUserId="462" LastEditDate="2017-06-04T21:17:09.970" LastActivityDate="2017-06-04T21:17:09.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="445" PostTypeId="2" ParentId="441" CreationDate="2017-06-04T22:35:10.487" Score="5" Body="&lt;p&gt;There's a &lt;a href=&quot;http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1004075&quot; rel=&quot;nofollow noreferrer&quot;&gt;new paper&lt;/a&gt; that's just appeared on the subject of &quot;proportionality&quot;, including a method by which RNA expression might be compared.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a new concept for me, and the article is not easy enough for me to read that I can write up a quick summary; the authors don't seem to devote a section in the paper to defining &quot;proportionality&quot;. However, here's an interesting chunk from the article:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We graphed the network of relationships between these mRNAs (S5 Fig.), an approach similar to gene co-expression network [12] or weighted gene co-expression analysis [13] but founded on proportionality and therefore valid for relative data. The network revealed one cluster of 96, and many other smaller clusters of mRNAs behaving proportionally across conditions.&lt;br&gt;&#xA;  ...&lt;br&gt;&#xA;  We are also keen to raise awareness that correlation (and other statistical methods that assume measurements come from real coordinate space) should not be applied to relative abundances. This is highly relevant to gene coexpression networks [12]. Correlation is at the heart of methods like Weighted Gene Co-expression Network Analysis [13] and heatmap visualization [14]. These methods are potentially misleading if applied to relative data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastEditorUserId="48" LastEditDate="2017-12-07T13:26:46.570" LastActivityDate="2017-12-07T13:26:46.570" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="446" PostTypeId="2" ParentId="441" CreationDate="2017-06-04T22:42:23.850" Score="3" Body="&lt;p&gt;If you have a lot of samples with very different environmental conditions, then a Weighted Gene Correlation Network Analysis (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2631488/&quot; rel=&quot;nofollow noreferrer&quot;&gt;WGCNA&lt;/a&gt;) might be appropriate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This type of analysis looks for genes that trace similar (or opposing) expression patterns throughout different conditions (e.g. high-medium-medium-low-absent-high would be highly negatively correlated with low-medium-medium-high-very high-low).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That particular paper introduces the concept of &quot;modules&quot;, which are groups of genes that share similar expression patterns. Functions are available for plotting how the expression of canonical module members changes throughout the different conditions, and for identifying which module (or modules) a particular gene is likely to be a member of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;WGCNA works best when there's a lot of different expression changes in the different conditions, which sounds like it would fit well with your project. However, it concentrates more on the genes, rather than the conditions (which seems like it would be less useful for you).&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="48" LastEditDate="2017-12-07T13:27:28.263" LastActivityDate="2017-12-07T13:27:28.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="447" PostTypeId="2" ParentId="438" CreationDate="2017-06-04T23:02:47.750" Score="5" Body="&lt;p&gt;There was a Structural Variant breakout session at the London Calling conference this year. Unfortunately I didn't attend that session, but MinION community members have access to Constance Donnell's summary of that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://community.nanoporetech.com/posts/breakout-structural-varia&quot; rel=&quot;noreferrer&quot;&gt;https://community.nanoporetech.com/posts/breakout-structural-varia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are my attempts at grabbing non-creative chunks from those notes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Professor Wigard Kloosterman has been developing a bioinformatics pipeline called NanoSV for mapping genomic structural variants in patients with congenital abnormalities&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Tomas Sesani at the University of Utah studied the precise copy number of duplicated genes in individual virus genes over time, and tracked the SNP within copy number variable regions of the genome [no public tool mentioned]&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Dr. Sudha Rao from Genotypic Technology in India used the MinION sequencer with Sniffles for structural variation calling&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;ONT have said that the sessions/talks from London Calling 2017 will all be made publicly available at some point in the future.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-04T23:02:47.750" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="448" PostTypeId="2" ParentId="317" CreationDate="2017-06-04T23:11:39.370" Score="3" Body="&lt;p&gt;I cannot think of any principled rationale for choosing this filtering strategy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am going to take a guess that this filtering strategy is supposed to filter out SNPs for which imputation did not work well?  In that case the appropriate statistic to filter on is the INFO score as described &lt;a href=&quot;https://mathgen.stats.ox.ac.uk/genetics_software/snptest/snptest.html#data_summaries&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might consider picking a higher threshold than 0.4 if you feel that you aren't filtering out enough variants.&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="77" LastEditDate="2018-01-16T21:12:58.123" LastActivityDate="2018-01-16T21:12:58.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="449" PostTypeId="1" AcceptedAnswerId="450" CreationDate="2017-06-04T23:25:12.173" Score="7" ViewCount="182" Body="&lt;p&gt;The &lt;code&gt;samtools mpileup&lt;/code&gt; command has quite a neat feature that it is able to correct mapping errors associated with misalignment of INDELs. By default, the &lt;code&gt;mpileup&lt;/code&gt; command will not work for reads that have more than 250X coverage of the reference genome. While this limit can be increased, very high coverage causes the mpileup program to grind to a halt, so it'd be nice to know if there's some easy way to make that faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To add a bit more context, I've been doing this with mitochondrial genome reads that were extracted from both Illumina whole-genome sequencing (coverage ~1000X), and from targeted amplicon sequencing done on the IonTorrent (coverage up to ~4000X).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see that @rightskewed has mentioned the downsampling ability of samtools with &lt;code&gt;samtools view -s &amp;lt;float&amp;gt;&lt;/code&gt; (see &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/406/73&quot;&gt;here&lt;/a&gt;), which seems like it might work as a solution for this if used prior to the mpileup operation.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-05T10:26:20.287" Title="How can I speed up INDEL calling/correction on BAM files?" Tags="|sam|samtools|variant-calling|indel|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="450" PostTypeId="2" ParentId="449" CreationDate="2017-06-04T23:34:21.633" Score="4" Body="&lt;p&gt;I wasn't aware of the samtools subsampling when I had this problem a couple of years ago, so ended up writing my own digital normalisation method to deal with &lt;em&gt;mapped&lt;/em&gt; reads. This method reduces the genome coverage, but preserves reads where coverage is low.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because I was working with IonTorrent reads (which have variable length), I came up with the idea of selecting the longest read that mapped to each location in the genome (assuming such a read existed). This meant that the highly variable coverage for different samples (sometimes as low as 200X, sometimes as high as 4000X) was flattened out to a much more consistent coverage of about 100-200X. Here's the core of the Perl code that I wrote:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  if(($F[2] ne $seqName) || ($F[3] != $pos) || (length($bestSeq) &amp;lt;= length($F[9]))){&#xA;    if(length($bestSeq) == length($F[9])){&#xA;      ## reservoir sampling with a reservoir size of 1&#xA;      ## See https://en.wikipedia.org/wiki/Reservoir_sampling&#xA;      ## * with probability 1/i, keep the new item instead of the current item&#xA;      $seenCount++;&#xA;      if(!rand($seenCount)){&#xA;        ## i.e. if rand($seenCount) == 0, then continue with replacement&#xA;        next;&#xA;      }&#xA;    } else {&#xA;      $seenCount = 1;&#xA;    }&#xA;    if(($F[2] ne $seqName) || ($F[3] != $pos)){&#xA;      if($output eq &quot;fastq&quot;){&#xA;        printSeq($bestID, $bestSeq, $bestQual);&#xA;      } elsif($output eq &quot;sam&quot;){&#xA;        print($bestLine);&#xA;      }&#xA;    }&#xA;    $seqName = $F[2];&#xA;    $pos = $F[3];&#xA;    $bestLine = $line;&#xA;    $bestID = $F[0];&#xA;    $bestFlags = $F[1];&#xA;    $bestSeq = $F[9];&#xA;    $bestQual = $F[10];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The full code (which works as a filter on uncompressed SAM files) can be found &lt;a href=&quot;https://github.com/gringer/bioinfscripts/blob/master/sam2LongestBase.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-05T10:26:20.287" LastActivityDate="2017-06-05T10:26:20.287" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="451" PostTypeId="2" ParentId="442" CreationDate="2017-06-05T02:34:11.050" Score="1" Body="&lt;p&gt;Something else that could be happening is that contigs that are being collapsed into &quot;heterozygous&quot; groups. This would be a particular problem when a genome has a substantial amount of repeated sequence. Digging deep into the supplementary information of the &lt;a href=&quot;https://academic.oup.com/gigascience/article-lookup/doi/10.1186/2047-217X-1-18&quot; rel=&quot;nofollow noreferrer&quot;&gt;SOAPdenovo2 paper&lt;/a&gt;, I've found the following information:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In SOAPdenovo2, heterozygous contig pairs are recognized by utilizing the information of contig depth and the locality of contig. The recognized heterozygous contig pairs should obey the following rules: 1) the similarity between contigs should be high enough, for example, ≥ 95%; 2) the depth of both contigs should be near half of the average depth or all contigs, complying Poisson distribution; 3) the two contigs should be located adjacently in a scaffold and have no relationship to each other inferred by paired-end reads information. The normal contigs neighboring the heterozygous regions, if they exist, could be connected to both of the heterozygous contig pairs (H1 and H2). Only the contig with relatively higher depth in a heterozygous contig pair were kept for scaffolding. The method reduces the influence of genome heterozygosity on final scaffold length. All heterozygous contig pairs were outputted to a file to facilitate further analysis. &lt;strong&gt;However, the trade-off of this method is that it might incorrectly remove paralogous contigs&lt;/strong&gt;. This problem could be relieved by a gap-filling procedure while the removed copy of paralogous contigs would be represented by gaps during scaffolding.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-05T10:36:44.423" LastActivityDate="2017-06-05T10:36:44.423" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="453" PostTypeId="1" CreationDate="2017-06-05T12:49:24.763" Score="13" ViewCount="780" Body="&lt;p&gt;This has come up repeatedly recently: I have a very large text file (in the order of several GiB) and I need to perform line-based subsetting for around 10,000 lines. There exist solutions for specific scenarios (e.g. &lt;code&gt;samtools view -s&lt;/code&gt; for randomly sampling BAM files) but sometimes my use-case doesn’t fit into these categories.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately a naïve &lt;code&gt;sed&lt;/code&gt;-based solution is &lt;em&gt;extremely&lt;/em&gt; slow:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time sed -n -f &amp;lt;(awk -vOFS='' '{print $0, &quot;p&quot;}' line_numbers.txt) input_file &amp;gt; selected_lines.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where &lt;code&gt;line_numbers.txt&lt;/code&gt; is a file containing one line number per line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Forget running this for 10,000 lines; it’s already grinding to a halt for a mere 1000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I speed this up, ideally so that it scales only with the size of the input file, and has more or less constant runtime n the number of lines that I subset?&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="73" LastEditDate="2017-06-05T19:44:59.737" LastActivityDate="2018-01-28T23:06:13.360" Title="How do I efficiently subset a very large line-based file?" Tags="|shell|benchmarking|subset|text|" AnswerCount="4" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="454" PostTypeId="2" ParentId="453" CreationDate="2017-06-05T12:49:24.763" Score="7" Body="&lt;p&gt;Turns out, simply keeping track of the next candidate line (after sorting the sample line numbers) fixes the performance issue, and most of the remaining slowness seems to be due to the overhead of actually reading the file so there’s not very much to improve.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I don’t know how how to do this in &lt;code&gt;sed&lt;/code&gt;, and it’s not trivial in &lt;code&gt;awk&lt;/code&gt; either, here’s a Perl script:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env perl&#xA;&#xA;use strict;&#xA;use warnings;&#xA;&#xA;my $file = $ARGV[0];&#xA;my $lines_file = $ARGV[1];&#xA;&#xA;open my $lines_fh, '&amp;lt;', $lines_file or die &quot;Cannot read file $lines_file&quot;;&#xA;chomp (my @lines = &amp;lt;$lines_fh&amp;gt;);&#xA;close $lines_fh;&#xA;&#xA;@lines = sort {$a &amp;lt;=&amp;gt; $b} @lines;&#xA;&#xA;open my $fh, '&amp;lt;', $file or die &quot;Cannot read file $file&quot;;&#xA;my $line = 1;&#xA;my $next_line = 0;&#xA;while (&amp;lt;$fh&amp;gt;) {&#xA;    last if $next_line == scalar @lines;&#xA;    if ($line++ == $lines[$next_line]) {&#xA;        $next_line++;&#xA;        print;&#xA;    }&#xA;}&#xA;close $fh;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I’ve implemented a &lt;a href=&quot;https://gist.github.com/klmr/a3e88c0ce102be0d3b2ce52ae83825e5&quot; rel=&quot;nofollow noreferrer&quot;&gt;similar function in C++&lt;/a&gt; for an &lt;a href=&quot;https://github.com/HannahVMeyer/PhenotypeSimulator&quot; rel=&quot;nofollow noreferrer&quot;&gt;R package&lt;/a&gt;, that's only slightly longer than the Perl script. It is ~3 times faster than the Perl script on my test file.&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="298" LastEditDate="2017-06-09T01:58:30.053" LastActivityDate="2017-06-09T01:58:30.053" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="455" PostTypeId="2" ParentId="453" CreationDate="2017-06-05T14:17:42.013" Score="4" Body="&lt;p&gt;Some related questions appear in other sites, with potentially interesting solutions, which I report here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To sample approximately 1% of the non-empty lines:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN {srand()} !/^$/ { if (rand() &amp;lt;= .01) print $0}' input_file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(from &lt;a href=&quot;https://stackoverflow.com/a/692321/1878788&quot;&gt;https://stackoverflow.com/a/692321/1878788&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To select 1000 random lines:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;shuf -n 1000 input_file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(from &lt;a href=&quot;https://stackoverflow.com/a/15065490/1878788&quot;&gt;https://stackoverflow.com/a/15065490/1878788&lt;/a&gt;, and &lt;a href=&quot;https://unix.stackexchange.com/a/108604/55127&quot;&gt;https://unix.stackexchange.com/a/108604/55127&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Edit: Python solutions using a list of lines&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Using a set of line indices and selecting lines by testing set membership:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;import sys&#xA;&#xA;with open(sys.argv[2], &quot;r&quot;) as line_numbers_file:&#xA;    line_indices = set(int(line) - 1 for line in line_numbers_file)&#xA;&#xA;with open(sys.argv[1], &quot;r&quot;) as input_file:&#xA;    print(*(line.strip() for (idx, line) in enumerate(input_file)&#xA;            if idx in line_indices), sep=&quot;\n&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using a numpy boolean array together with &lt;code&gt;itertools.compress&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;import sys&#xA;from itertools import compress&#xA;from numpy import zeros&#xA;&#xA;with open(sys.argv[2], &quot;r&quot;) as line_numbers_file:&#xA;    line_indices = [int(line) - 1 for line in line_numbers_file]&#xA;&#xA;selector = zeros(max(line_indices) + 1, dtype=bool)&#xA;selector[line_indices] = 1&#xA;&#xA;with open(sys.argv[1], &quot;r&quot;) as input_file:&#xA;    print(*(line.strip() for line in compress(input_file, selector)), sep=&quot;\n&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I did some tests on a file containing 15774756 sam records and a list of 10000 pre-generated random line numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The perl script proposed by Konrad Rudolph (&lt;a href=&quot;https://bioinformatics.stackexchange.com/a/454/292&quot;&gt;https://bioinformatics.stackexchange.com/a/454/292&lt;/a&gt;) runs in about 5.3 seconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The set membership testing python solution runs in about 4.45 seconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The compress based solution runs in about 3.4 seconds.&#xA;I suspect this may vary a lot depending on the highest line number we want, since the number of iterations will depend on the length of the boolean array. Here the highest line number was 15773768, so pretty high compared with the total number of lines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried with python 3.6. I suspect that python 2.7 could be slightly faster, but haven't tested.&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2017-06-06T07:10:27.073" LastActivityDate="2017-06-06T07:10:27.073" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="456" PostTypeId="1" CreationDate="2017-06-05T16:36:54.550" Score="8" ViewCount="614" Body="&lt;p&gt;We are designing a CRISPR/Cas9 experiment and thinking of the down-stream data analyses. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any R packages for analysing raw NGS read count data from pooled genetic screens using CRIPSR/Cas9 to disrupt gene expression in a population of cells? I guess we will need to start with basics, i.e. sequences processing, data exploration, visualisation etc. &lt;/p&gt;&#xA;" OwnerUserId="573" LastEditorUserId="2719" LastEditDate="2018-07-04T07:08:37.583" LastActivityDate="2018-07-04T07:16:08.893" Title="R packages for data analyses of pooled CRISPR screens" Tags="|r|crispr|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="457" PostTypeId="1" CreationDate="2017-06-05T16:51:28.363" Score="9" ViewCount="643" Body="&lt;p&gt;Is it possible to draw e.g. 95% confidence ellipses around samples from the same group on the results from the plotMDS function under edgeR? If so, how?&lt;/p&gt;&#xA;" OwnerUserId="467" LastActivityDate="2017-06-06T14:18:36.500" Title="confidence ellipses for MDS plot in edgeR?" Tags="|r|edger|" AnswerCount="1" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="458" PostTypeId="1" CreationDate="2017-06-05T17:15:39.767" Score="10" ViewCount="1030" Body="&lt;p&gt;We have heard in the group that it is important to keep track of and to filter artifact regions when analysing data from functional genomics experiments, especially ChIP-seq. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, we have seen pipelines that remove the ENCODE tracks i) before cross-correlations QCs, ii) after cross-correlations QC but before peak calling and iii) after peak calling. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have noticed that removal of the tracks does not affect significantly cross-correlation and peak-independent QCs. However, we are not sure whether peak calling should be done on the filtered tracks or not?&lt;/p&gt;&#xA;" OwnerUserId="375" LastActivityDate="2023-05-21T22:08:42.160" Title="When to account for the blacklisted genomic regions in ChIP-seq data analyses?" Tags="|chip-seq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="459" PostTypeId="2" ParentId="458" CreationDate="2017-06-05T17:41:43.290" Score="10" Body="&lt;p&gt;Aside: Cross-correlation is largely meaningless, regardless of what some of the ENCODE folks might argue. When we process our DEEP samples we don't even look at that value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regardless, if you're using SPP/phantomPeakQual for cross-correlation then note that it already removes the highest peaks from your dataset before computing the cross-correlation (in fact, it can remove most of the actual peaks too, which makes one further wonder what it's actually telling you). I don't know that this is actually documented anywhere, it's something I noticed when going through the code while pondering whether to implement it in deepTools. But at least it's ignoring these regions already :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, it's most convenient to just remove peaks overlapping blacklisted regions. In an ideal world you'd filter out the blacklisted reads before peak calling, but (1) this is really inconvenient (more time and disk required) and (2) I've never seen an appreciable gain in peak calling performance. In theory at least you should be losing sensitivity right around blacklisted regions if you don't remove reads overlapping blacklisted regions, but you have to ask yourself whether you want to trust such peaks anyway. For other QC steps, at least with deepTools we provide a parameter with every tool to specify a BED file of blacklisted regions to skip.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an aside, there are many fewer blacklisted regions in more recent genome builds (GRCh38 and GRCm38 at least), so this is less of an issue in general with them.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-06-05T17:41:43.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="460" PostTypeId="2" ParentId="29" CreationDate="2017-06-05T17:42:37.170" Score="3" Body="&lt;p&gt;Regarding the first part of the question: scRNA-seq is a rapidly developing field so it may be hard to talk about &quot;widely accepted tool for doing pseudo-temporal ordering from scRNAseq data&quot;. Few of the tools aiming to do just that include Monocole, Waterfall or Sincell (see this paper for references &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4122333/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4122333/&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second part is a bit more complex. Many aspects have come together here, e.g. cell-cycles, sub-populaitons identification and pseudo-temporal ordering, to get a true reflection of the biological processes. There are efforts on all (and more) these fronts (see again e.g. the above paper, though not the latest) and there are most likely people working on their integration. I'm not aware of any studies published yet at this depth though&lt;/p&gt;&#xA;" OwnerUserId="375" LastActivityDate="2017-06-05T17:42:37.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="461" PostTypeId="2" ParentId="422" CreationDate="2017-06-05T17:43:34.723" Score="5" Body="&lt;p&gt;I am not sure how appropriate it is at this point to still refer to sequencing as next-generation sequencing. The leading NGS technology is Illumina/Solexa that has been around for over 10 years at this point. 454 was around even earlier. It's not really &quot;next&quot; at this point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Opinions aside, I would also add &quot;third-generation sequencing&quot; to that list, referring to long read technologies like PacBio and Oxford Nanopore. See &lt;a href=&quot;https://biology.stackexchange.com/questions/21080/what-is-the-difference-between-second-and-third-generation-sequencing&quot;&gt;this question on Biology SE&lt;/a&gt; for more details.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2017-06-05T17:43:34.723" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="462" PostTypeId="1" CreationDate="2017-06-05T18:06:35.393" Score="8" ViewCount="152" Body="&lt;p&gt;We have whole genome sequencing data for patients (not-cancer) (n=60) and for healthy controls (n=20). The sequencing centre has provided us with the best practice bioinformatics analyses including reads mapping (.BAM) and variant calling using GATK (.vcf) as well as annotation (annotated .vcf and .gVCF). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should be our next steps? We are interested to see if there are any differences (global and/or specific) between the groups. &lt;/p&gt;&#xA;" OwnerUserId="589" LastActivityDate="2017-06-06T15:26:23.913" Title="How to compare groups using WGS data?" Tags="|wgs|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="463" PostTypeId="5" CreationDate="2017-06-05T19:19:33.367" Score="0" Body="&lt;p&gt;BAM was the first widely-adopted binary standard for storing NGS alignments. Its specification is openly maintained &lt;a href=&quot;https://github.com/samtools/hts-specs&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. BAM is a compressed format, using bgzf to compress reads in blocks. This is convenient, as it allows sorted files to be indexed and then rapid queried.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions with this tag should broadly fall into one of the following categories:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Questions regarding the standard itself (e.g., &quot;What is the relationship between the BAM header and alignments?&quot;).&lt;/li&gt;&#xA;&lt;li&gt;Questions regarding using BAM files (e.g., &quot;How do I sort a BAM file?&quot; or &quot;How do I ensure that all reads in one/more fastq files are contained in a BAM file?).&lt;/li&gt;&#xA;&lt;li&gt;When seeing errors, such as corrupt or incomplete BAM files.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Particularly for category 3, it is then vital to provide information regarding exactly how the BAM file was made, ideally including the version of upstream tools as well as the exact command used.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-06-05T20:44:04.017" LastActivityDate="2017-06-05T20:44:04.017" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="464" PostTypeId="4" CreationDate="2017-06-05T19:19:33.367" Score="0" Body="The &quot;Binary Alignment Map&quot; (BAM) format is one of the common binary formats used to store sequence alignment information. Questions should include this tag if they directly pertain to the format itself, details of BAM file usage, or errors relating to likely malformed BAM files." OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-06-05T20:43:58.390" LastActivityDate="2017-06-05T20:43:58.390" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="465" PostTypeId="2" ParentId="453" CreationDate="2017-06-05T19:54:57.300" Score="5" Body="&lt;p&gt;Perl should be fairly fast with this when using a hash set to store the list of lines. A structure like this also works for subsetting based on a field value, where the comparison would be with the field rather than &quot;$.&quot;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/perl&#xA;&#xA;use strict;&#xA;use warnings;&#xA;&#xA;my $lines_file = $ARGV[0];&#xA;my %include_lines = ();&#xA;&#xA;open my $lines_fh, '&amp;lt;', $lines_file or die &quot;Cannot read file $lines_file&quot;;&#xA;while(&amp;lt;$lines_fh&amp;gt;){&#xA;  chomp;&#xA;  $include_lines{$_} = 1;&#xA;}&#xA;close $lines_fh;&#xA;&#xA;while(&amp;lt;&amp;gt;){&#xA;  if($include_lines{$.}){ # &quot;$.&quot; -- line number of current file&#xA;    print;&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that according to &lt;a href=&quot;https://stackoverflow.com/questions/5920686/how-to-get-the-current-line-number-of-a-file-open-using-perl#comment6821096_5920709&quot;&gt;this SO answer&lt;/a&gt;, the &quot;$.&quot; operator is not strictly the current line number, and can be influenced by different file operations or other settings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: just saw your comment about speed in your answer, comparing hash sets to a sorted list. The &lt;code&gt;$lines[$next_line]&lt;/code&gt; bit feels a bit odd to me. Have you tried out using &lt;code&gt;shift&lt;/code&gt; or &lt;code&gt;pop&lt;/code&gt; on a sorted list to fetch the next line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/perl&#xA;&#xA;use strict;&#xA;use warnings;&#xA;&#xA;my $lines_file = $ARGV[0];&#xA;&#xA;open my $lines_fh, '&amp;lt;', $lines_file or die &quot;Cannot read file $lines_file&quot;;&#xA;chomp (my @lines = &amp;lt;$lines_fh&amp;gt;);&#xA;close $lines_fh;&#xA;&#xA;@lines = sort {$a &amp;lt;=&amp;gt; $b} @lines;&#xA;my $next_line = shift(@lines);&#xA;&#xA;while (&amp;lt;&amp;gt;) {&#xA;    if ($. == $next_line) {&#xA;        $next_line = shift(@lines);&#xA;        print;&#xA;        last if (!@lines);&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I changed the &lt;code&gt;shift&lt;/code&gt; to a &lt;code&gt;pop&lt;/code&gt; (reversing the sort order to match), and got times of 4.8s, 5.8s and 4.1s for Konrad's original code, my hash code, and my pop code respectively, fetching 10,000 lines from &lt;code&gt;/usr/share/dict/british-english-insane&lt;/code&gt; 25 times over (after copying input files to &lt;code&gt;/tmp&lt;/code&gt;). They're all the same from my perspective: quick enough that it would take me longer to type out the command than it would to run it. Using &lt;code&gt;shift&lt;/code&gt; instead of &lt;code&gt;pop&lt;/code&gt; doesn't seem to change the time noticeably.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-01-28T23:06:13.360" LastActivityDate="2018-01-28T23:06:13.360" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="466" PostTypeId="2" ParentId="271" CreationDate="2017-06-05T21:05:51.263" Score="8" Body="&lt;p&gt;I just received a reply from 1000Genomes regarding this. I'll post it in its entirety below:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Looking at the example you mention, I find it difficult to come up with an&#xA;  interpretation of the information whereby the stated end seems to be correct,&#xA;  so believe that this may indeed be an error.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Since the v4.0 was created, however, new versions of VCF have been introduced,&#xA;  improving and correcting the specification. The current version is v4.3&#xA;  (&lt;a href=&quot;http://samtools.github.io/hts-specs/&quot; rel=&quot;noreferrer&quot;&gt;http://samtools.github.io/hts-specs/&lt;/a&gt;). I believe the first record shown on&#xA;  page 11 provides an accurate example of this type of deletion.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;I will update the web page to include this information.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So we can take this as official confirmation that we were all correct in suspecting the example was just wrong.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-06-05T21:05:51.263" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="467" PostTypeId="2" ParentId="462" CreationDate="2017-06-05T21:16:01.070" Score="2" Body="&lt;p&gt;I'm not familiar with the program, but apparently &lt;a href=&quot;https://hail.is/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hail&lt;/a&gt; is setting itself up as a swiss-army chainsaw project for doing downstream analysis on variant-called datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An overview of Hail can be found here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://blog.cloudera.com/blog/2017/05/hail-scalable-genomics-analysis-with-spark/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://blog.cloudera.com/blog/2017/05/hail-scalable-genomics-analysis-with-spark/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A tutorial on association testing can be found here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://hail.is/hail/tutorial.html#Association-testing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hail.is/hail/tutorial.html#Association-testing&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-05T21:16:01.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="468" PostTypeId="2" ParentId="411" CreationDate="2017-06-06T02:47:36.653" Score="6" Body="&lt;p&gt;In general, the best way to download SRA data is: don't download from SRA. However, as ENA has not be sync'd yet, I would recommend to download from SRA ftp and then convert to fastq locally. You can find files in the SRA format &lt;a href=&quot;https://ftp-trace.ncbi.nih.gov/sra/sra-instant/reads/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. Downloading and then converting locally is much faster than direct retrieval from NCBI for some mysterious reasons.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-06-06T02:47:36.653" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="469" PostTypeId="2" ParentId="411" CreationDate="2017-06-06T07:56:48.763" Score="3" Body="&lt;p&gt;By far the fastest method in my experience has been to use the &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/SRAdb.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SRAdb&lt;/a&gt; library in R. For most entries, you can download fastq files directly. Some older experiments don't have them, but I've still found it much faster to download SRA files via &lt;code&gt;getSRAfile()&lt;/code&gt; and then to convert them using &lt;code&gt;fastqdump&lt;/code&gt; than to use &lt;code&gt;fastqdump&lt;/code&gt; directly. &lt;/p&gt;&#xA;" OwnerUserId="182" LastActivityDate="2017-06-06T07:56:48.763" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="470" PostTypeId="5" CreationDate="2017-06-06T08:05:14.003" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-06-06T08:05:14.003" LastActivityDate="2017-06-06T08:05:14.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="471" PostTypeId="4" CreationDate="2017-06-06T08:05:14.003" Score="0" Body="These questions are about sequence alignment. " OwnerUserId="450" LastEditorUserId="450" LastEditDate="2017-06-06T09:16:52.090" LastActivityDate="2017-06-06T09:16:52.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="472" PostTypeId="5" CreationDate="2017-06-06T08:08:22.230" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-06-06T08:08:22.230" LastActivityDate="2017-06-06T08:08:22.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="473" PostTypeId="4" CreationDate="2017-06-06T08:08:22.230" Score="0" Body="Questions specific to nanopore sequencing. For general question about long reads, use tag long-reads instead and for questions about specific sequencer use a specific sequencer tag (i.e. minion, gridion)" OwnerUserId="450" LastEditorUserId="57" LastEditDate="2017-06-08T11:42:01.353" LastActivityDate="2017-06-08T11:42:01.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="474" PostTypeId="5" CreationDate="2017-06-06T08:10:34.667" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-06-06T08:10:34.667" LastActivityDate="2017-06-06T08:10:34.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="475" PostTypeId="4" CreationDate="2017-06-06T08:10:34.667" Score="0" Body="Questions specific to interacting with and post-processing sequence alignments using the SAMtools package. For questions specifically about formats SAM, BAM or CRAM use tags sam or bam," OwnerUserId="450" LastEditorUserId="57" LastEditDate="2018-05-03T07:09:55.937" LastActivityDate="2018-05-03T07:09:55.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="476" PostTypeId="2" ParentId="414" CreationDate="2017-06-06T08:19:37.980" Score="3" Body="&lt;p&gt;For ChIP-seq it shouldn't really matter. But do be aware that by default, &lt;code&gt;samtools merge&lt;/code&gt; retains read group information (the &lt;code&gt;@RG&lt;/code&gt; field in the header) from each input file. This could pose a problem for some downstream analyses (e.g. for the GATK HaplotypeCaller) if you want the merged data to be considered as all part of the same sample. You can change this behaviour using the &lt;code&gt;-c&lt;/code&gt; option. &lt;/p&gt;&#xA;" OwnerUserId="182" LastActivityDate="2017-06-06T08:19:37.980" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="477" PostTypeId="2" ParentId="456" CreationDate="2017-06-06T09:05:07.703" Score="5" Body="&lt;p&gt;Assuming you mean CRISPR screens targeting many loci (for example using the GeCKO library) there is an R package &lt;a href=&quot;https://git.embl.de/msmith/geckoR&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It uses a linear mixed effect model to compare guide counts before and after a selection step, allowing for multiple guides/gene and multiple replicates. &#xA;It can also do the initial read alignment AFAIK and the author should be quite responsive in case you have questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option could be the &lt;em&gt;python&lt;/em&gt; program &lt;a href=&quot;https://bitbucket.org/liulab/mageck-vispr&quot; rel=&quot;nofollow noreferrer&quot;&gt;MAGeCK&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="595" LastEditorUserId="48" LastEditDate="2018-07-04T07:16:08.893" LastActivityDate="2018-07-04T07:16:08.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="478" PostTypeId="2" ParentId="441" CreationDate="2017-06-06T10:57:34.970" Score="5" Body="&lt;p&gt;The suggestion to use proportionality is probably the correct one if you are interested in similar &lt;em&gt;patterns&lt;/em&gt; between samples. But not if you are interested in &lt;em&gt;absolute&lt;/em&gt; differences. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example: the following two samples are similar in pattern, but not similar on absolute levels:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;        Sample 1    Sample 2    Sample 3&#xA;Gene A    10          100           80&#xA;Gene B     8           80          100  &#xA;Gene C    12          120          120&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Samples 1 and 2 have a perfect proportionality (phi is 0) and also a perfect correlation (as a side note any pair with a perfect proportionality will always have a perfect correlation). However, in terms of logfold changes, samples 2 and 3 are more like each other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course in real life you would never see a comparison like the sample 1 - sample 2 one because normalisation would have removed the scale difference. This was exactly the point brought up by the proportionality paper. But normalisation methods don't normally guarantee that the sum of expression for each sample is identical, and such differences can still occur. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative that might suit more in the second case is either the euclidean distance between the samples, or the euclidean distance on the first two components of a principle component or multi-dimensional scaling. The latter is effectively using mean logFC between samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that &lt;code&gt;x&lt;/code&gt; is a matrix containing normalised, log transformed expression values, you could use R and limma to calculate distance in multi-dimensional scaled space as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(limma)&#xA;mds &amp;lt;- plotMDS(x, plot=FALSE)&#xA;mds &amp;lt;- data.frame(mds$x, mds$y)&#xA;distances &amp;lt;- dist(mds)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="235" LastActivityDate="2017-06-06T10:57:34.970" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="479" PostTypeId="2" ParentId="321" CreationDate="2017-06-06T11:07:55.663" Score="1" Body="&lt;p&gt;Ideally these BQSR methods were made keeping in mind how technical errors will actually screw up the base quality calls and when the machines were still more on development phase while being used for the 1000G project. As of now machines are more powerful and strong where it will be unlikely to use it but still we use with listed SNPs to find the covariates and build a model around the data using the information with machine learning tricks to improve the quality of those base calls. Ideally it should be more appropriate when old machines from Illumina or other standard companies are being used but with new machines which are much powerful and having high throughput they should tend to go down. I do not recall if such tests have been made but obviously I know new sequencing machine always make such tests to show that they have reduced such errors but still recommend such BQSR for variant calls. Now the problem is the list of SNPs, this to me is the real problem since the list we use is far from being Gold standard and if that is not properly taken care of everything we infer about quality is still shaky. &lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/44/base-quality-score-recalibration-bqsr&quot; rel=&quot;nofollow noreferrer&quot;&gt;This link&lt;/a&gt; is pretty informative but it's an old one. I would really see improvements with new sequencers.  However very less people care about such tests in academic research and also translational lab will really not invest time and money on such unless the facility has some bioinformaticians who always does such testing while buying a new sequencer for the institute. In terms of clinical genomics for finding variants I reckon most powerful and up-to-date sequencers should be used but not sure if they still use BQSR and if so what is the list they use to build model of covariation around the data.&lt;/p&gt;&#xA;" OwnerUserId="451" LastEditorUserId="1157" LastEditDate="2019-04-19T07:48:46.800" LastActivityDate="2019-04-19T07:48:46.800" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="480" PostTypeId="2" ParentId="457" CreationDate="2017-06-06T11:28:46.080" Score="2" Body="&lt;p&gt;I can see that there would be a three step process to doing this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Merge counts from all samples in the group and then resample pseudo-replicates from this. If &lt;code&gt;x&lt;/code&gt; is a matrix with samples in the group being columns and genes being rows&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;s &amp;lt;- sample(row.names(x), n = mean(colSums(x)), probs=rowSums(x)/sum(rowSums(x))&#xA;stab &amp;lt;- table(s)&#xA;s &amp;lt;- as.vector(stab)&#xA;names(s) &amp;lt;- names(stab)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Do this thousands of times.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You would then want to project those onto your MDS plot space - i'm less clear about how you could do this without perturbing the space itself.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate the parameter for an ellipise that would contain 95% of these points. Again, I'm not so sure about how to do this, but I think the &lt;code&gt;car&lt;/code&gt; pacakge might be a good place to start looking.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="235" LastEditorUserId="29" LastEditDate="2017-06-06T14:18:36.500" LastActivityDate="2017-06-06T14:18:36.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="481" PostTypeId="1" AcceptedAnswerId="3142" CreationDate="2017-06-06T12:44:58.087" Score="15" ViewCount="2674" Body="&lt;p&gt;&lt;em&gt;This is a question from &lt;a href=&quot;https://www.reddit.com/user/beneficii9&quot; rel=&quot;noreferrer&quot;&gt;/u/beneficii9&lt;/a&gt; on reddit. The original post can be found &lt;a href=&quot;https://www.reddit.com/r/bioinformatics/comments/6f040g/how_do_you_do_admixture_testing_with_a_whole/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Through the Personal Genome Project, I have had my whole genome sequenced by Veritas, and have it in the form of a single VCF file for the whole genome and one BAS file for each chromosome. The reference genome associated with the VCF file is hg19. It has been helpful in health data; for example, I discovered I'm homozygous for the non-functional variant CYP-2D6 gene (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/projects/SNP/snp_ref.cgi?rs=3892097&quot; rel=&quot;noreferrer&quot;&gt;rs3892097&lt;/a&gt;), which can render several common medications useless, and helps explain why some medicines didn't really work for me. My doctor has found this information very helpful.&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, I can't find any way of looking at admixture or ancestry. I've tried setting everything up using a combination of VCFTools, Plink1.9, and ADMIXTURE, but I can't get it to work. I think for ADMIXTURE you have to have a bunch of genomes sorted by geographical origin to compare your genome against, but I'm not sure how to do that, and what's online isn't very clear to me. So scratch that one off.&lt;/p&gt;&#xA;&lt;p&gt;I've tried converting the file to 23andme format (and at this &lt;a href=&quot;https://www.reddit.com/u/psychosomaticism&quot; rel=&quot;noreferrer&quot;&gt;/u/psychosomaticism&lt;/a&gt; has been very helpful). I did that (though it seems there were problems because of the way the VCF file was set up). But the websites that take the data want you to point them to your 23andme account, and that doesn't really work if you only have the file. 23andme doesn't provide for people who had their whole genomes sequenced. They want you to give them a saliva sample like everyone else.&lt;/p&gt;&#xA;&lt;p&gt;So, what can I do?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2019-01-18T17:39:58.820" Title="How do I carry out an ancestry/admixture test on a single VCF file?" Tags="|vcf|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="482" PostTypeId="5" CreationDate="2017-06-06T12:47:47.490" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-06-06T12:47:47.490" LastActivityDate="2017-06-06T12:47:47.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="483" PostTypeId="4" CreationDate="2017-06-06T12:47:47.490" Score="0" Body="Posts originally from reddit, where the question is well-structured and fits in with the StackExchange question/answer format. StackExchange questions should link to the original post at the top of the question." OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-06T12:56:19.700" LastActivityDate="2017-06-06T12:56:19.700" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="484" PostTypeId="2" ParentId="481" CreationDate="2017-06-06T13:03:11.313" Score="4" Body="&lt;p&gt;A suggestion from the UK Biobank QC methods white paper:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Create a set of SNPs common to your VCF and the 1000 genomes phase 3 call set.&lt;/li&gt;&#xA;&lt;li&gt;Perform PCA of the 1000 genomes samples using eigenstrat smartpca. You might have to convert to binary plink format.&lt;/li&gt;&#xA;&lt;li&gt;Project your genotypes onto this pre-computed PCA space and visualize using ggplot to see which cluster you fall in.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="601" LastActivityDate="2017-06-06T13:03:11.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="485" PostTypeId="2" ParentId="481" CreationDate="2017-06-06T13:16:05.703" Score="2" Body="&lt;p&gt;Ancestry testing is a tricky subject. I spent a good chunk of &lt;a href=&quot;http://researcharchive.vuw.ac.nz/handle/10063/1987&quot; rel=&quot;nofollow noreferrer&quot;&gt;my PhD project&lt;/a&gt; on questions similar to this, and haven't really found a good answer for how to detect unmodelled ancestry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general idea of how I tried to approach ancestry determination was to create a model set containing well-defined groups of individuals with a known, specific ancestral background. A query individual (or individuals) were then added to this group, and an ancestry-estimating program was run (specifically &lt;a href=&quot;http://web.stanford.edu/group/pritchardlab/structure.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Structure&lt;/a&gt;) to work out what proportion of the unknown individual was attributable to each of the known groups. I would expect that most genetic ancestry tests follow a similar approach to this, although perhaps with a little bit less care about the definition of the model population groups.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Problems arise when individuals are incorrectly assigned to a particular group, when groups are present in the model set that represent a larger proportion of individuals than most other groups, when groups are closely-related to other groups, and when an ancestral history is present in a test individual that doesn't match any of the model groups. And all that assumes that the marker set used for ancestry determination is perfect: no bias towards any particular group, and no systematic genotyping error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This doesn't mean that ancestry testing won't work, but it's a good idea to take the results with a large grain of salt. There's a &lt;a href=&quot;https://www.theguardian.com/world/2017/apr/13/a-dna-test-showed-im-100-maori-many-thought-there-were-none-of-us-left&quot; rel=&quot;nofollow noreferrer&quot;&gt;good example&lt;/a&gt; of a media personality in New Zealand who was told that she had a very high probability of being 100% Māori, despite having a good knowledge of her own family history that indicated one European ancestor a few generations back on both sides of her family.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-06T13:16:05.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="486" PostTypeId="1" CreationDate="2017-06-06T13:27:44.753" Score="1" ViewCount="258" Body="&lt;p&gt;&lt;em&gt;This is a question from &lt;a href=&quot;https://www.reddit.com/user/wipeyourmit&quot; rel=&quot;nofollow noreferrer&quot;&gt;/u/wipeyourmit&lt;/a&gt; on reddit. The original post can be found &lt;a href=&quot;https://www.reddit.com/r/bioinformatics/comments/6dfu3c/question_about_eggnog_mapper_and_annotation_in/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If I have a metagenomic dataset that contains reads from both&#xA;  eukaryotes and prokaryotes and then I annotate by running DIAMOND or&#xA;  HMMER against a bacterial database how much of a risk do I run of&#xA;  eukaryotic reads being annotated in the process?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;I was hoping to use the eggNOG mapper to search against the bacterial&#xA;  and archaeal databases and to exclude the eukaryotic portion of my&#xA;  dataset. Is the eukaryotic filtering something that I would need to do&#xA;  in a step prior to this?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastEditorUserId="55" LastEditDate="2017-06-08T11:43:24.667" LastActivityDate="2017-06-08T11:43:24.667" Title="Accidental mapping of eukaryotic reads in a metagenomic dataset" Tags="|read-mapping|metagenome|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="487" PostTypeId="2" ParentId="486" CreationDate="2017-06-06T13:31:44.300" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from &lt;a href=&quot;https://www.reddit.com/user/Romanticon&quot; rel=&quot;nofollow noreferrer&quot;&gt;/u/Romanticon&lt;/a&gt; on reddit. The original answer can be found &lt;a href=&quot;https://www.reddit.com/r/bioinformatics/comments/6dfu3c/question_about_eggnog_mapper_and_annotation_in/di2qv4a/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Depending on the length of your reads, the level of error in the&#xA;  reads, the quality of the database, and the specificity settings you&#xA;  give DIAMOND, you can change the level of &quot;bleed&quot; that you get - but&#xA;  you'll always have a couple reads, especially at metagenome dataset&#xA;  sizes, that will be annotated incorrectly.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;But you'll always have this happen, whenever you annotate any large&#xA;  dataset. The best way to handle it is to set a threshold after&#xA;  annotation that removes wrongly annotated reads (looking for 'clearly&#xA;  wrong' organisms in your annotated dataset can help you figure out&#xA;  where to set the threshold level - there's probably no Bos taurus&#xA;  reads in a mouse metagenome).&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;You can do things like enable the sensitive flag on the DIAMOND&#xA;  annotation search to help improve annotation accuracy.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastEditorUserId="298" LastEditDate="2017-06-07T10:45:03.643" LastActivityDate="2017-06-07T10:45:03.643" CommentCount="0" CommunityOwnedDate="2017-06-06T13:31:44.300" ContentLicense="CC BY-SA 3.0" />
  <row Id="488" PostTypeId="1" AcceptedAnswerId="490" CreationDate="2017-06-06T13:38:19.540" Score="13" ViewCount="2118" Body="&lt;p&gt;&lt;em&gt;This is a question from &lt;a href=&quot;https://www.reddit.com/user/apivan19&quot; rel=&quot;noreferrer&quot;&gt;/u/apivan19&lt;/a&gt; on reddit. The original post can be found &lt;a href=&quot;https://www.reddit.com/r/bioinformatics/comments/6ddvkv/converting_gene_names/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have some proteomics data that was given to me with the UniProt gene identifiers in column 1. I've been trying to convert these to normal gene symbols using various programs, but it is proving to be difficult.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Uniprot website does it fairly decently, but it is not able to convert all of them and then adds some unknown genes into my list.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I will give it 5439 genes in UniProt notation, and will say &quot;5420 of 5439 UniProt identifiers have been converted to 5450 gene symbols&quot;... which is ridiculous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried using David to change the symbols, but it returns them to me in some ridiculous, random order and there's no way I can sort... actually there might be but it'll take a second.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are some of the easiest ways to do this? It's already very time consuming and am looking for simpler solutions&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="55" LastEditDate="2017-06-08T11:43:09.197" LastActivityDate="2017-07-11T09:18:11.997" Title="Converting gene names from one public database format to another" Tags="|database|gene|conversion|" AnswerCount="6" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="489" PostTypeId="2" ParentId="488" CreationDate="2017-06-06T13:46:44.057" Score="3" Body="&lt;p&gt;My favourite gene database conversion site is &lt;a href=&quot;https://biodbnet-abcc.ncifcrf.gov/db/db2db.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;db2db&lt;/a&gt;. You provide a list of IDs in one of a large number of different public formats, and can select one or more IDs as translation targets. It will then walk through various known paths to do the translation, picking what it determines to be the most reliable route to get the information you have asked for. Results appear in the browser as a table, but can also be exported as an Excel file, or as a tab-separated text file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the mapping of genes from one database to another is not a one-to-one mapping. It is likely the case that there will be some genes in the source database that map to multiple genes in the target database (and &lt;em&gt;vice versa&lt;/em&gt;), and some genes that aren't present in the target database. These phenomena probably account for the &quot;ridiculous&quot; results that have been seen here.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-06T13:46:44.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="490" PostTypeId="2" ParentId="488" CreationDate="2017-06-06T14:16:38.920" Score="9" Body="&lt;p&gt;I tend to use &lt;a href=&quot;http://www.ensembl.org/biomart&quot; rel=&quot;noreferrer&quot;&gt;Ensembl Biomart&lt;/a&gt; for such queries since there are APIs for various programming languages, e.g. &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/biomaRt.html&quot; rel=&quot;noreferrer&quot;&gt;biomaRt&lt;/a&gt;, and, maybe more interestingly, via a &lt;a href=&quot;http://www.ensembl.org/info/data/biomart/biomart_restful.html&quot; rel=&quot;noreferrer&quot;&gt;REST API&lt;/a&gt; (although it’s a pretty terrible one).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To translate identifiers from different databases, proceed as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Choose database “Ensembl genes”&lt;/li&gt;&#xA;&lt;li&gt;Choose dataset &lt;em&gt;your desired oganism&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;Go on “Filters” › “Gene:” › “Input external reference ID list”&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Select the chosen source database&lt;/li&gt;&#xA;&lt;li&gt;Provide a list of IDs, delimited by newline&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;Go to “Attributes” › “Gene:” › untick “Transcript stable ID”&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If Ensembl IDs are desired, leave “Gene stable ID” ticked …&lt;/li&gt;&#xA;&lt;li&gt;Otherwise untick it; go to “External:”, tick your desired identifier format&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;Click “Results” at the top left. This gives a preview that can be exported into various formats; alternatively the top centre buttons “XML” and “Perl” provide the query in XML (for SOAP/REST requests) and as a (horrendously formatted) executable Perl script.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="29" LastActivityDate="2017-06-06T14:16:38.920" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="491" PostTypeId="1" AcceptedAnswerId="501" CreationDate="2017-06-06T14:29:59.870" Score="7" ViewCount="1056" Body="&lt;p&gt;I'm fairly new to &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/&quot; rel=&quot;noreferrer&quot;&gt;snakemake&lt;/a&gt; and I'm trying to understand the difference between the &lt;code&gt;--cluster&lt;/code&gt; and &lt;code&gt;--drmaa&lt;/code&gt; flags, both of which are used to submit jobs to compute clusters/nodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The docs give a few hints about the advantages of using &lt;code&gt;--drmaa&lt;/code&gt; &lt;a href=&quot;http://snakemake.readthedocs.io/en/stable/executable.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If your cluster system supports DRMAA, Snakemake can make use of that&#xA;  to increase the control over jobs. E.g. jobs can be cancelled upon&#xA;  pressing Ctrl+C, which is not possible with the generic --cluster&#xA;  support.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And &lt;a href=&quot;http://snakemake.readthedocs.io/en/stable/tutorial/additional_features.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If available, DRMAA is preferable over the generic cluster modes&#xA;  because it provides better control and error handling.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So I have a conceptual understanding of the advantages of using &lt;code&gt;--drmaa&lt;/code&gt;. However, I don't consider the above a very complete explanation, and I don't know how these flags are implemented in snakemake under the hood, can someone elaborate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, that although this could be considered a more general programming question, snakemake is predominantly used in bioinformatics; I was persuaded that this question would be considered on-topic because of &lt;a href=&quot;https://bioinformatics.meta.stackexchange.com/questions/61/serving-as-a-support-forum-for-specific-tools&quot;&gt;this meta-post&lt;/a&gt; and &lt;a href=&quot;https://bioinformatics.meta.stackexchange.com/a/66/104&quot;&gt;this answer&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="73" LastEditDate="2017-06-06T21:58:34.553" LastActivityDate="2017-06-06T21:58:34.553" Title="How are snakemake's --cluster and --drmaa options implemented?" Tags="|snakemake|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="492" PostTypeId="2" ParentId="488" CreationDate="2017-06-06T14:37:23.240" Score="3" Body="&lt;p&gt;I'm not a huge fan of the Ensembl BioMart system because I find it difficult to use. The &lt;a href=&quot;http://llama.mshri.on.ca/synergizer/doc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Synergizer&lt;/a&gt; has a very straightforward interface and works pretty well for most lists. Note: it hasn't been updated in a while.&lt;/p&gt;&#xA;" OwnerUserId="110" LastActivityDate="2017-06-06T14:37:23.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="493" PostTypeId="2" ParentId="327" CreationDate="2017-06-06T15:04:52.107" Score="2" Body="&lt;p&gt;The kind of QC you do routinely depends on what your lab's focus is. We do a lot of low-quality, multiplexed DNA and RNA. If you routinely do fresh frozen whole genomes, your QC will be different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Weighing in from the resequencing side of things (i.e. sequencing an organism that has a good reference), there are several things we are testing with quality control:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Did the library preparation succeed? Was the capture successful? Is our input material of high enough quality? &lt;/li&gt;&#xA;&lt;li&gt;Did the sequencing work? For Illumina, did enough clusters pass filter? What's our Q30? Did we sequence into adapters?&lt;/li&gt;&#xA;&lt;li&gt;Can we proceed with analysis? Are the indices on multiplexed samples correct?&#xA;Do we have the correct reference sequence? Do we have the right targets?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You may use a combination of tools:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://support.illumina.com/sequencing/sequencing_software/bcl2fastq-conversion-software.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bcl2fastq&lt;/a&gt; : if you multiplexed, check the undetermined indices file after converting to fastq to make sure you caught all of the indexed reads&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.bioinformatics.babraham.ac.uk/projects/fastqc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FastQC&lt;/a&gt;: General sequencing quality, sequencing into adapters, handy flagging for obvious problems&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/Illumina/interop&quot; rel=&quot;nofollow noreferrer&quot;&gt;Illumina interop&lt;/a&gt;/SAV: Cluster pass filter, Q30&lt;/li&gt;&#xA;&lt;li&gt;Alignment (with &lt;a href=&quot;http://bio-bwa.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BWA&lt;/a&gt;, bowtie, etc), &lt;a href=&quot;http://bedtools.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools&lt;/a&gt;, &lt;a href=&quot;http://www.htslib.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools&lt;/a&gt;, &lt;a href=&quot;https://broadinstitute.github.io/picard/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Picard&lt;/a&gt;: check for contamination, make sure we're using the right reference, overlap with target regions, depth of coverage, soft-clipping for low-quality sequence&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="110" LastActivityDate="2017-06-06T15:04:52.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="494" PostTypeId="2" ParentId="462" CreationDate="2017-06-06T15:26:23.913" Score="2" Body="&lt;p&gt;If you have gVCFs, the first thing you should try is joint variant calling. According to GATK, joint variant calling &quot;empowers variant discovery by providing the ability to leverage population-wide information from a cohort of multiple sample[sic], allowing us to detect variants with great sensitivity and genotype samples as accurately as possible.&quot; &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/article.php?id=4150&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have two sets of high-quality variants, what you do next depends on your research question. Are you looking for druggable mutations? Underlying causes? Biomarkers? Patient prognosis? You can look at the most frequently mutated positions in the patient cohort and compare them to your reference cohort, check for co-occurring mutations, cluster them, do principal component analysis, do some machine learning to stratify them, etc. &lt;/p&gt;&#xA;" OwnerUserId="110" LastActivityDate="2017-06-06T15:26:23.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="495" PostTypeId="2" ParentId="491" CreationDate="2017-06-06T15:31:11.623" Score="4" Body="&lt;p&gt;&lt;strong&gt;DRMAA&lt;/strong&gt; (Distributed Resource Management Application API) appears to be an open API that describes a specification for submission and management of work submitted to some grid/cluster. &lt;a href=&quot;https://en.wikipedia.org/wiki/DRMAA#Implementations&quot; rel=&quot;nofollow noreferrer&quot;&gt;If your scheduler is DRMAA compliant&lt;/a&gt;, my guess would be that using &lt;code&gt;snakemake&lt;/code&gt;'s &lt;code&gt;--drmaa&lt;/code&gt; flag will afford you the additional controls exposed by that API.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned in your question with DRMAA support, a Ctrl+C on one's console would be passed on to the grid to kill or stop submitted jobs. A task that would otherwise require use of the &lt;code&gt;qdel&lt;/code&gt; command on &lt;code&gt;Sun Grid Engine&lt;/code&gt;, for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could see why the suggestion would be to enable &lt;code&gt;--drmaa&lt;/code&gt; where possible as it's likely to expose functionality (such as control of jobs, and availability of logs and errors) that makes the submission and management of jobs a little easier for you and &lt;code&gt;snakemake&lt;/code&gt;. Perhaps someone with &lt;code&gt;snakemake&lt;/code&gt; experience can explain a little more, but I hope this helps?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For what it's worth, I use Sun Grid Engine without DRMAA and it's quite a pain to get good information out of &lt;code&gt;qstat&lt;/code&gt;, and its &lt;code&gt;qacct&lt;/code&gt; log format is quite possibly the worst file format I have ever encountered. I suspect DRMAA provides a nice (or at least more reasonable) API for &lt;code&gt;snakemake&lt;/code&gt; (and others) to get at this information more readily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Appended:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under the hood, the flags are parsed by Python's &lt;code&gt;argparse&lt;/code&gt; module in the &lt;code&gt;__init__.py&lt;/code&gt;. &lt;a href=&quot;https://bitbucket.org/snakemake/snakemake/src/e7bdc7587c6e41113b972e3eba1b9404e741873c/snakemake/__init__.py?fileviewer=file-view-default#__init__.py-747&quot; rel=&quot;nofollow noreferrer&quot;&gt;A mutually exclusive group of options&lt;/a&gt; forces a selection of &lt;code&gt;--cluster&lt;/code&gt; or &lt;code&gt;--drmaa&lt;/code&gt; (or &lt;code&gt;--cluster-sync&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;--cluster&lt;/code&gt; or &lt;code&gt;-c&lt;/code&gt; requires you to specify an argument; the command for which to submit a job on your cluster (the example names &lt;code&gt;qsub&lt;/code&gt;). The &lt;code&gt;--drmaa&lt;/code&gt; flag just seems to indicate to &lt;code&gt;snakemake&lt;/code&gt; that DRMAA is to be used, which doesn't change a lot in terms of how the command you run from your console is processed in the &lt;code&gt;__init__.py&lt;/code&gt; or &lt;a href=&quot;https://bitbucket.org/snakemake/snakemake/src/e7bdc7587c6e41113b972e3eba1b9404e741873c/snakemake/workflow.py?at=master&amp;amp;fileviewer=file-view-default&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;workflow.py&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when it is time to interface with your scheduler (as specified in &lt;a href=&quot;https://bitbucket.org/snakemake/snakemake/src/e7bdc7587c6e41113b972e3eba1b9404e741873c/snakemake/scheduler.py?at=master&amp;amp;fileviewer=file-view-default&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;scheduler.py&lt;/code&gt;&lt;/a&gt;), an &lt;code&gt;elif&lt;/code&gt; statement checks whether you're in some form of cluster mode and the &lt;code&gt;else&lt;/code&gt; catches the case where you've raised the &lt;code&gt;--drmaa&lt;/code&gt; flag instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's where the magic happens, as now your job is submitted with the &lt;code&gt;DRMAAExecutor&lt;/code&gt; as specified in &lt;a href=&quot;https://bitbucket.org/snakemake/snakemake/src/e7bdc7587c6e41113b972e3eba1b9404e741873c/snakemake/executors.py?at=master&amp;amp;fileviewer=file-view-default&quot; rel=&quot;nofollow noreferrer&quot;&gt;executors.py&lt;/a&gt;, instead of the default executor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without investigating much more, I can see that the &lt;code&gt;DRMAAExecutor&lt;/code&gt; features some more class attributes and exposes additional functions when compared to the others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this is of more use? Again, I've only just looked at the code myself but in lieu of a &lt;code&gt;snakemake&lt;/code&gt; user chiming in, I thought I'd have a nose.&lt;/p&gt;&#xA;" OwnerUserId="215" LastEditorUserId="215" LastEditDate="2017-06-06T18:54:34.823" LastActivityDate="2017-06-06T18:54:34.823" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="496" PostTypeId="2" ParentId="414" CreationDate="2017-06-06T17:40:13.550" Score="2" Body="&lt;p&gt;Agree with the others that it doesn't really matter. One thing to note though - if you're deduplicating your BAM files (you probably should for ChIP-seq data), make sure that you do this &lt;em&gt;after&lt;/em&gt; merging.. :)&lt;/p&gt;&#xA;" OwnerUserId="224" LastActivityDate="2017-06-06T17:40:13.550" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="497" PostTypeId="2" ParentId="488" CreationDate="2017-06-06T17:52:17.670" Score="8" Body="&lt;p&gt;If you're comfortable doing a little programming, check out &lt;a href=&quot;http://mygene.info&quot; rel=&quot;noreferrer&quot;&gt;mygene.info&lt;/a&gt; (web services for gene annotations of all sorts).  ID translation is specifically one of the use cases addressed in the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/mygene.html&quot; rel=&quot;noreferrer&quot;&gt;bioconductor client&lt;/a&gt; (see the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/mygene/inst/doc/mygene.pdf&quot; rel=&quot;noreferrer&quot;&gt;vignette&lt;/a&gt;), and there is a &lt;a href=&quot;https://pypi.python.org/pypi/mygene&quot; rel=&quot;noreferrer&quot;&gt;python client&lt;/a&gt; as well available through pypi. The documentation for mygene can be found &lt;a href=&quot;http://mygene-py.readthedocs.io/en/latest/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="612" LastEditorUserId="73" LastEditDate="2017-06-06T19:53:55.077" LastActivityDate="2017-06-06T19:53:55.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="498" PostTypeId="2" ParentId="488" CreationDate="2017-06-06T18:32:15.170" Score="4" Body="&lt;p&gt;You can do the same using &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/AnnotationDbi.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;AnnotationDbi&lt;/a&gt; Package from Bioconductor. Download the organism specific annotation file like &lt;a href=&quot;https://bioconductor.org/packages/release/data/annotation/html/org.Mm.eg.db.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;org.Mm.eg.db&lt;/a&gt; for mouse and map current gene ids to the gene names/gene symbols.&lt;/p&gt;&#xA;" OwnerUserId="206" LastActivityDate="2017-06-06T18:32:15.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="499" PostTypeId="2" ParentId="441" CreationDate="2017-06-06T19:05:40.117" Score="2" Body="&lt;p&gt;The euclidean distance is probably the simplest, both conceptually and in terms of implementation. It's by no means an elegant solution, and it may not perform well in certain circumstances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Euclidean distance is easiest to conceptualize as the distance between two points in a two dimensional space.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Y&#xA;^&#xA;|&#xA;|&#xA;|&#xA;|            * p = (3, 3)&#xA;|&#xA;|&#xA;|&#xA;|        * q = (2, 1)&#xA;|&#xA;----------------------------&amp;gt; X&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this example, the distance between the two points is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;d(p, q) = sqrt( (p_x-q_x)^2 + (p_y-q_y)^2 )&#xA;        = sqrt( (2-3)^2 + (1-3)^2 )&#xA;        = sqrt(5)&#xA;        ≈ 2.24&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For a gene expression profile with two genes, this is exactly how the euclidean distance would be calculated, using expression values from one gene as the X axis and expression values from the other gene as the Y axis. Realistically, though, gene expression profiles typically contain thousands or tens of thousands of genes, so instead we use the generalization of the distance calculation for N dimensions.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;d(p, q) = sqrt( (p_1-q_1)^2 + (p_2-q_2)^2 + ... + (p_N-q_N)^2 )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Packages for R and Python make these types of calculations trivial once you have the data loaded into the correct data structure. See Ian's answer for some example R code.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2017-06-06T19:05:40.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="500" PostTypeId="1" AcceptedAnswerId="503" CreationDate="2017-06-06T19:19:47.827" Score="13" ViewCount="627" Body="&lt;p&gt;I am writing a software tool to which I would like to add the ability to compute alignments using the efficient Burrows-Wheeler Transform (BWT) approach made popular by tools such as BWA and Bowtie. As far as I can tell, though, both of these tools and their derivatives are invoked strictly via a command-line interface. Are there any libraries that implement BWT-based read alignment with a C/C++ API?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python bindings would also be great, but probably too much to expect.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="298" LastEditDate="2017-06-07T12:39:30.463" LastActivityDate="2019-04-29T00:39:30.740" Title="Library for computing BWT-based alignments" Tags="|sequence-alignment|bwa|read-mapping|api|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="501" PostTypeId="2" ParentId="491" CreationDate="2017-06-06T19:38:15.237" Score="5" Body="&lt;p&gt;I'd always kind of wondered how this worked too, so I took this as an excuse to look into the snakemake code. At the end of the day this becomes a question of (1) how are jobs actually submitted and (2) how is it determined if jobs are done (and then whether they failed)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For DRMAA, python has a module (appropriately named &quot;drmaa&quot;) that wraps around the libdrmaa library that comes with most schedulers. This is a very popular route, for example the Galaxy project uses this for dealing with most clusters (e.g., I use it to connect our internal Galaxy instance to our slurm cluster). The huge benefit here is that DRMAA does magic to allow you to simply submit commands to your cluster without needing to know whether you should execute &lt;code&gt;qsub&lt;/code&gt; or &lt;code&gt;srun&lt;/code&gt; or something else. Further, it then provides methods to simply query whether a job is running or not and what its exit status was.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the &lt;code&gt;--cluster&lt;/code&gt; command requires a LOT more magic on the side of snakemake. At the end of the day it creates a shell script that it then submits using the command you provided. Importantly, it includes some secret files in that script that it can then watch for (ever noticed that &lt;code&gt;.snakemake&lt;/code&gt; directory where you execute a script? That appears to be among the things it's used for.). These are named &lt;code&gt;{jobid}.jobfinished&lt;/code&gt; and &lt;code&gt;{jobid}.jobfailed&lt;/code&gt; and one of them will get touched, depending on the exit status of your command/script. Once one of those is there, then snakemake can move on in its DAG (or not, if there's a failure). This is obviously a LOT more to keep track of and it's then not possible for snakemake to cancel running jobs, which is something one can do easily with DRMAA.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-06-06T19:38:15.237" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="502" PostTypeId="2" ParentId="500" CreationDate="2017-06-06T19:44:56.890" Score="4" Body="&lt;p&gt;&lt;a href=&quot;http://seqan.de/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SeqAn&lt;/a&gt; supports BWT tables for use with their parametrisable alignment algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To use it, &lt;a href=&quot;http://seqan.readthedocs.io/en/seqan-v1.4.2/Tutorial/SimpleReadMapping.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;follow the general outline for building a SeqAn short-read aligner&lt;/a&gt;, and use the &lt;a href=&quot;http://docs.seqan.de/seqan/2.2.0/index.html?p=FMIndex&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;FMIndex&lt;/code&gt;&lt;/a&gt; specialisation instead of — as in the example — the &lt;code&gt;IndexQGram&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="73" LastEditDate="2017-06-06T19:50:47.123" LastActivityDate="2017-06-06T19:50:47.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="503" PostTypeId="2" ParentId="500" CreationDate="2017-06-06T20:17:34.890" Score="11" Body="&lt;p&gt;First, let us remark that there exist several hundred read mappers, most of which have been even published (see, e.g., pages 25-29 of &lt;a href=&quot;http://brinda.cz/publications/brinda_phd.pdf&quot; rel=&quot;noreferrer&quot;&gt;this thesis&lt;/a&gt;). Developing a new mapper probably makes sense only as a programming exercise. Whereas developing a quick proof-of-concept read mapper is usually easy, turning it into a real competitor of existing and well-tuned mappers can last for years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not clear from the provided description how long is your reference, how many alignments you need to compute, etc. In certain situations, it may be useful to write a wrapper over existing mappers (e.g., using &lt;a href=&quot;https://docs.python.org/3/library/subprocess.html#subprocess.Popen&quot; rel=&quot;noreferrer&quot;&gt;subprocess.Popen&lt;/a&gt; for running the mapper and &lt;a href=&quot;http://pysam.readthedocs.io/&quot; rel=&quot;noreferrer&quot;&gt;PySam&lt;/a&gt; for parsing the output); while in some other situations, standard dynamic programming may be sufficient (e.g., using &lt;a href=&quot;https://github.com/mengyao/complete-striped-smith-waterman-library&quot; rel=&quot;noreferrer&quot;&gt;SSW&lt;/a&gt; with its Python binding).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let assume that you want to develop a toy read mapper. Most of read mappers are based on a so called seed-and-extend paradigm. Simply speaking, first you detect candidates for alignments, usually as exact matches between a read and the reference (using either a hash table or some full-text index – e.g., BWT-index). Then you would need to compute alignments for these candidates, typically using some algorithm based on dynamic programming, and report the best obtained alignments (e.g., the ones with the highest alignment score).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There exist two big, powerful and well debugged libraries implementing BWT-indexes which can be easily used for building a read mapper:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://seqan.rtfd.io&quot; rel=&quot;noreferrer&quot;&gt;SeqAn&lt;/a&gt;. See the &lt;a href=&quot;http://seqan.readthedocs.io/en/master/Tutorial/DataStructures/Indices/FMIndex.html?highlight=bwt&quot; rel=&quot;noreferrer&quot;&gt;FMIndex tutorial&lt;/a&gt; and the &lt;a href=&quot;http://seqan.readthedocs.io/en/master/Tutorial/Algorithms/Alignment/PairwiseSequenceAlignment.html&quot; rel=&quot;noreferrer&quot;&gt;Pairwise Sequence Alignment tutorial&lt;/a&gt; for quick examples of how to detect exact matches and how to do pairwise alignments. Also, they provide a tutorial about a &lt;a href=&quot;http://seqan.readthedocs.io/en/seqan-v1.4.2/Tutorial/SimpleReadMapping.html&quot; rel=&quot;noreferrer&quot;&gt;quick development of a read mapper&lt;/a&gt;, but the resulting mapper seems to be hash-table based, not BWT-based. SeqAn is used, e.g., in &lt;a href=&quot;https://github.com/seqan/seqan/tree/master/apps/yara&quot; rel=&quot;noreferrer&quot;&gt;YARA&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/simongog/sdsl-lite&quot; rel=&quot;noreferrer&quot;&gt;SDSL-Lite&lt;/a&gt;. This is a general library for succinct data structures. See the &lt;a href=&quot;http://simongog.github.io/assets/data/sdsl-slides/tutorial#1&quot; rel=&quot;noreferrer&quot;&gt;tutorial slides&lt;/a&gt; for an idea of how it works. For instance, &lt;a href=&quot;https://github.com/iqbal-lab/gramtools&quot; rel=&quot;noreferrer&quot;&gt;GramTools&lt;/a&gt; are based on SDSL.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="425" LastEditorUserId="425" LastEditDate="2017-06-06T20:46:40.420" LastActivityDate="2017-06-06T20:46:40.420" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="504" PostTypeId="2" ParentId="426" CreationDate="2017-06-06T20:53:30.430" Score="6" Body="&lt;p&gt;I think others have provided fine answers to your question already. I just thought it might be relevant to mention about a tool that allows using Spark to distribute computations that re-use existing commandline bioinformatics software ... so, avoiding the need to re-implement proven algorithms in a language that Spark supports, &lt;a href=&quot;https://github.com/mcapuccini/EasyMapReduce&quot; rel=&quot;nofollow noreferrer&quot;&gt;EasyMapReduce&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This perceived need to re-implement algorithms, has otherwise been one of the biggest road blocks for wider adoption of Spark in the bioinformatics community in my understanding. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EasyMapReduce was presented SparkSummit East this year (video link &lt;a href=&quot;https://www.youtube.com/watch?v=4C4R9qptUQo&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, as shown on the repository home page).&lt;/p&gt;&#xA;" OwnerUserId="630" LastEditorUserId="73" LastEditDate="2017-06-06T21:40:05.000" LastActivityDate="2017-06-06T21:40:05.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="505" PostTypeId="1" AcceptedAnswerId="512" CreationDate="2017-06-06T20:58:15.443" Score="6" ViewCount="552" Body="&lt;p&gt;I recently got some &lt;a href=&quot;https://www.illumina.com/products/by-type/microarray-kits/infinium-methylation-epic.html&quot; rel=&quot;noreferrer&quot;&gt;EPIC DNA methylation&lt;/a&gt; data and I was wondering what are some good practices to follow? &#xA;I am interested in knowing about normalization and differential analysis. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="626" LastEditorUserId="73" LastEditDate="2017-06-06T21:50:14.550" LastActivityDate="2017-06-06T23:10:14.527" Title="What are some good practices to follow during EPIC DNA methylation data analysis?" Tags="|r|normalization|methylation|microarray|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="506" PostTypeId="2" ParentId="486" CreationDate="2017-06-06T21:32:30.483" Score="3" Body="&lt;p&gt;If you know the eukaryotic contaminant present you could use &lt;code&gt;bbsplit.sh&lt;/code&gt; from &lt;a href=&quot;https://sourceforge.net/projects/bbmap/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BBMap suite&lt;/a&gt; to split/bin reads first using a reference for that contaminant (into one or as many bins as reference sequences you provide). &lt;/p&gt;&#xA;" OwnerUserId="634" LastActivityDate="2017-06-06T21:32:30.483" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="508" PostTypeId="1" AcceptedAnswerId="519" CreationDate="2017-06-06T22:23:31.397" Score="19" ViewCount="13379" Body="&lt;p&gt;&lt;em&gt;This is based on a question from &lt;a href=&quot;https://www.biostars.org/u/39680/&quot; rel=&quot;nofollow noreferrer&quot;&gt;betsy.s.collins&lt;/a&gt; on BioStars. The original post can be found &lt;a href=&quot;https://www.biostars.org/p/256448/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Does anyone have any suggestions for other tags or filtering steps on BWA-generated BAM files that can be used so reads only map to one location? One example application would be to find seeds for the &lt;a href=&quot;https://github.com/Generade-nl/TULIP&quot; rel=&quot;nofollow noreferrer&quot;&gt;TULIP&lt;/a&gt; assembler/scaffolder, which works best for reads that map to unique genomic locations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The questioner is referring to genomically-unique alignments, which are something different from multiple possible alignments at a single location. If the only multiple alignment is at the same genomic location, those alignments are still unique in the genomic sense. The impossibility of finding the exact, correct, alignment is a well-known problem, and there are a few downstream methods (e.g. left normalisation) to make sure that multiple local alignments and/or sequencing errors have reduced impact.&lt;/p&gt;&#xA;&lt;p&gt;The concept of &amp;quot;uniquely mapped reads&amp;quot; is a loaded term, and most sources suggest filtering by MAPQ should do the trick. However, this approach doesn't seem to work when using BWA as a read mapper.&lt;/p&gt;&#xA;&lt;p&gt;Uniquely mapped reads (i.e. a read that maps to a single location in the genome) are sometimes preferred when running analyses that depend on the quantification of reads, rather than just the coverage (e.g. RNASeq). Duplicated reads require additional filtering or normalisation on the analysis side, and most downstream programs won't consider concepts like &amp;quot;a fifth of the read maps somewhere here on chromosome 5, and four-fifths of the read maps somewhere here on chromosome 14.&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-11-17T21:34:41.273" LastActivityDate="2022-08-19T08:45:41.163" Title="Obtaining uniquely mapped reads from BWA mem alignment" Tags="|sam|sequence-alignment|bwa|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="509" PostTypeId="5" CreationDate="2017-06-06T22:38:50.510" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-06-06T22:38:50.510" LastActivityDate="2017-06-06T22:38:50.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="510" PostTypeId="4" CreationDate="2017-06-06T22:38:50.510" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-06-06T22:38:50.510" LastActivityDate="2017-06-06T22:38:50.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="511" PostTypeId="2" ParentId="500" CreationDate="2017-06-06T23:07:51.320" Score="9" Body="&lt;p&gt;BWA-MEM can be used as a library. File &lt;a href=&quot;https://github.com/lh3/bwa/blob/master/example.c&quot; rel=&quot;noreferrer&quot;&gt;bwa/example.c&lt;/a&gt; shows the basic functionality for single-end mapping. It should give identical mapping to the bwa-mem command line. Header &lt;a href=&quot;https://github.com/lh3/bwa/blob/master/bwamem.h&quot; rel=&quot;noreferrer&quot;&gt;bwa/bwamem.h&lt;/a&gt; contains basic documentation. Paired-end mapping is doable, but not well exposed. Several teams, including GATK and MS genomics, have been using bwa-mem as a library.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-06-06T23:07:51.320" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="512" PostTypeId="2" ParentId="505" CreationDate="2017-06-06T23:10:14.527" Score="6" Body="&lt;p&gt;EPIC data can be processed in the same manner as the previous iteration of methylation array data from Illumina (450k). This means that starting with .idat files, normalization should be performed (for example, via the minfi package). A &lt;a href=&quot;http://biorxiv.org/content/early/2016/07/23/065490&quot; rel=&quot;noreferrer&quot;&gt;recent paper&lt;/a&gt; from the creators of minfi is particularly helpful because it makes clear that normalized EPIC data from their package can be immediately compared against, for example, &lt;a href=&quot;https://wiki.nci.nih.gov/display/TCGA/Data+level&quot; rel=&quot;noreferrer&quot;&gt;level 3 TCGA data&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that, I suggest using the manifest to attach genomic coordinates to your probes and segregating them into functional regions. By testing differential methylation in only regulatory regions, for example, you can increase the statistical power by reducing the overall number of tests to the ones you expect to yield major differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are existing packages out there for differential methylation analysis, but without knowing your replicate structure or aims, it is difficult to point you in the right direction.&lt;/p&gt;&#xA;" OwnerUserId="643" LastActivityDate="2017-06-06T23:10:14.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="513" PostTypeId="1" CreationDate="2017-06-06T23:44:21.290" Score="6" ViewCount="88" Body="&lt;p&gt;I'm working on a university project of predicting Translation Initiation Sites in human DNA. I searched the net for papers and documentation to get guidelines and inspiration, but I feel uncertain that I was able to find the state of the art in solving this problem, so I thought of asking the community here: What's the state of the art method of predicting Translation Initiation Sites?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I'm also posting the question to help this nascent Q&amp;amp;A site by participating, please close it if it completely misses the mark of being a good question.)&lt;/p&gt;&#xA;" OwnerDisplayName="user548" LastActivityDate="2017-06-07T01:53:56.240" Title="State of the art in predicting Translation Initiation Sites" Tags="|human-genome|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="515" PostTypeId="2" ParentId="513" CreationDate="2017-06-07T01:53:56.240" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://science.sciencemag.org/content/324/5924/218&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ribosome Profiling&lt;/a&gt; experiments profile the positions of ribosome-protected mRNA. a cycloheximide treatment fixes the translating ribosomes. A naive approach to define Translation Initiation Sites (TIS) would be to look at positions where you see distinct 'peaks' in the distribution of these fragments since the ribosome density will be higher at the start of the CDS (at the TIS).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://tisdb.human.cornell.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TISDB&lt;/a&gt; is a database comprised of &lt;a href=&quot;http://www.pnas.org/content/109/37/E2424.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;GTI-Seq&lt;/a&gt; global mapping of TIS codons at nearly single-nucleotide resolution. GTI-Seq uses two 'fixers' that fix ribosomes where one of them specifically fixes the initiating ribosomes. &lt;/p&gt;&#xA;" OwnerUserId="161" LastActivityDate="2017-06-07T01:53:56.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="516" PostTypeId="1" CreationDate="2017-06-07T03:39:25.560" Score="6" ViewCount="110" Body="&lt;p&gt;I'm trying to use HOMER to make a metagene profile over gene bodies using a bedgraph file I've generated. The problem is that every time I do, I get really weird scaling on the y-axis. I should be getting average values across the gene body on the order of 5-10, but instead I'm getting values on the order of 0.03-0.05 or less. The weird thing is, when I don't use metagene script or when I don't use histogram with -size given, I get perfectly normal values -- but that's not what I want, unfortunately, for the metagene profile.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I missing?&lt;/p&gt;&#xA;" OwnerUserId="651" LastEditorUserId="73" LastEditDate="2017-06-07T04:27:00.783" LastActivityDate="2017-06-07T06:15:01.453" Title="What's the scaling for HOMER metagenes?" Tags="|ngs|bed|metagenome|homer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="517" PostTypeId="2" ParentId="486" CreationDate="2017-06-07T03:49:20.207" Score="3" Body="&lt;p&gt;You can use &lt;a href=&quot;https://ccb.jhu.edu/software/centrifuge/&quot; rel=&quot;nofollow noreferrer&quot;&gt;centrifuge&lt;/a&gt; with the NT library to profile the taxonomy, and then remove the reads from eukaryotes. &lt;/p&gt;&#xA;" OwnerUserId="652" LastActivityDate="2017-06-07T03:49:20.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="518" PostTypeId="1" AcceptedAnswerId="520" CreationDate="2017-06-07T04:20:05.333" Score="10" ViewCount="893" Body="&lt;p&gt;I am aligning a dataset of 1,000,000 reads oh human mRNA sequenced on Oxford Nanopore Technologies' MinION, and would like to use the STAR aligner, using the &lt;a href=&quot;https://github.com/PacificBiosciences/cDNA_primer/wiki/Bioinfx-study:-Optimizing-STAR-aligner-for-Iso-Seq-data#param&quot; rel=&quot;noreferrer&quot;&gt;parameters recommended by Pacific Biosciences&lt;/a&gt; for long reads. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://groups.google.com/forum/#!searchin/rna-star/sjdboverhang%7Csort:relevance/rna-star/h9oh10UlvhI/ZwAs1xFVCAAJ&quot; rel=&quot;noreferrer&quot;&gt;this Google Groups thread&lt;/a&gt;, in setting up the genome index for short reads, the parameter &lt;code&gt;sjdbOverhang&lt;/code&gt; should be set to 1 less than the read length. Obviously, with long (mean 1.7Kb, max &gt;50Kb) reads, this doesn't make sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Running &lt;code&gt;STARlong --runMode genomeGenerate&lt;/code&gt; without setting &lt;code&gt;sjdbOverhang&lt;/code&gt; sets the parameter to &lt;code&gt;0&lt;/code&gt;. Does anyone know what this means, how it might affect mapping, and what I should set it to for my long reads?&lt;/p&gt;&#xA;" OwnerUserId="163" LastEditorUserId="73" LastEditDate="2017-06-07T04:50:06.280" LastActivityDate="2017-06-08T02:58:54.080" Title="Building STAR Genome Index for nanopore RNA sequencing" Tags="|rna-seq|sequence-alignment|nanopore|star|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="519" PostTypeId="2" ParentId="508" CreationDate="2017-06-07T04:21:28.520" Score="13" Body="&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt; - as of &lt;a href=&quot;https://github.com/samtools/samtools/pull/1346&quot; rel=&quot;nofollow noreferrer&quot;&gt;January 2021&lt;/a&gt;, samtools can now do filtering based on an expression that includes tag variables. In this case, this expression can be used to exclude any reads that have either an XA or SA tag:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -b mapped.bam -e '!([XA] | [SA])' &amp;gt; unique_mapped.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For more details on the samtools expression parameter, see the samtools documentation:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.htslib.org/doc/samtools.html#FILTER_EXPRESSIONS&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.htslib.org/doc/samtools.html#FILTER_EXPRESSIONS&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Original answer follows....&lt;/p&gt;&#xA;&lt;p&gt;To exclude &lt;em&gt;all possible&lt;/em&gt; multi-mapped reads from a BWA-mapped BAM file, it looks like you need to use &lt;code&gt;grep&lt;/code&gt; on the uncompressed SAM fields:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h mapped.bam | grep -v -e 'XA:Z:' -e 'SA:Z:' | samtools view -b &amp;gt; unique_mapped.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Explanation follows...&lt;/p&gt;&#xA;&lt;p&gt;I'm going to assume a situation in which a bioinformatician is presented with a mapped BAM file produced by BWA, and has no way of getting the original reads. One high-effort solution would be to extract the mapped reads from the BAM file and re-map with a different mapper that uses the MAPQ score to indicate multiple mappings.&lt;/p&gt;&#xA;&lt;p&gt;... but what if that were not possible?&lt;/p&gt;&#xA;&lt;p&gt;My understanding of BWA's output is that if a read maps perfectly to multiple genomic locations, it will be given a high mapping quality (MAPQ) score for both locations. Many people expect that a read that maps to at least two locations can have (at best) a 50% probability of mapping to one of those locations (i.e. MAPQ = 3). Because BWA doesn't do this, it makes it difficult to filter out multiply-mapped reads from BWA results using the MAPQ filter that works for other aligners; this is likely to be why the current answer on Biostars [&lt;code&gt;samtools view -bq 1&lt;/code&gt;] probably won't work.&lt;/p&gt;&#xA;&lt;p&gt;Here is an example line from a BWA mem alignment that I've just made. These are Illumina reads mapped to a parasite genome that has a &lt;em&gt;lot&lt;/em&gt; of repetitive sequence:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ERR063640.7     16      tig00019544     79974   21      21M2I56M1I20M   *       0       0       TATCACATATCATCCGACTCAGCTCGACGAGTACAATGCTAATTTAACACTTAGAATGCCCGGCAATGAAATTCGTTTTCCGTCAATTCTTGAAAATTTC    &amp;lt;AABBEGABFJKKKIM7GHKKJK&amp;gt;JLKLDGMHLIMIHHCGIJKKLJKLNJGLLLKLILKLMFNDLKGHJEKMKKMIJHGLOJLLLKIJLKKJEJLIGG&amp;gt;D    NM:i:4  MD:Z:83A13      AS:i:77 XS:i:67 XA:Z:tig00019544,-78808,21M2I56M1I20M,6;tig00019544,-84624,79M1I20M,6;tig00019544,-79312,33M4I42M1I20M,8;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;BWA mem has found that this particular read, ERR063460.7, maps to at least three different locations: tig00019544, tig00019544, and tig00019544. Note that the MAPQ for this read is 21, so even though the read maps to multiple locations, MAPQ can't be used to determine that.&lt;/p&gt;&#xA;&lt;p&gt;However, the alternative locations are shown by the presence of the &lt;code&gt;XA&lt;/code&gt; tag in the custom fields section of the SAM output. Perhaps just filtering on lines that contain the &lt;code&gt;XA&lt;/code&gt; tag will be able to exclude multiply-mapped reads. The &lt;code&gt;samtools view&lt;/code&gt; man page suggests that &lt;code&gt;-x&lt;/code&gt; will filter out a particular tag:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ samtools view -x XA output.bam | grep '^ERR063640\.7[[:space:]]'&#xA;ERR063640.7 16  tig00019544 79974   21  21M2I56M1I20M   *   0   0   TATCACATATCATCCGACTCAGCTCGACGAGTACAATGCTAATTTAACACTTAGAATGCCCGGCAATGAAATTCGTTTTCCGTCAATTCTTGAAAATTTC    &amp;lt;AABBEGABFJKKKIM7GHKKJK&amp;gt;JLKLDGMHLIMIHHCGIJKKLJKLNJGLLLKLILKLMFNDLKGHJEKMKKMIJHGLOJLLLKIJLKKJEJLIGG&amp;gt;D    NM:i:4  MD:Z:83A13  AS:i:77 XS:i:67&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... so it filtered out the tag (i.e. the tag no longer exists in the SAM output), but not the &lt;em&gt;read&lt;/em&gt;. There are no useful bits in the FLAG field to indicate multiple genomic mappings (which I know can be filtered to exclude the read as well), so I have to resort to other measures.&lt;/p&gt;&#xA;&lt;p&gt;In this particular case, I can use &lt;code&gt;grep -v&lt;/code&gt; on the uncompressed SAM output to exclude alignment lines that have the &lt;code&gt;XA&lt;/code&gt; tag (and re-compress to BAM afterwards, just to be tidy):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ samtools view -h output.bam | grep -v 'XA:Z:' | samtools view -b &amp;gt; output_filtered.bam&#xA;$&lt;/span&gt; samtools view output_filtered.bam | grep '^ERR063640\.7[[:space:]]'&#xA;&amp;lt;no output&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Hurray! reads filtered. As a little aside, this &lt;code&gt;grep&lt;/code&gt; search has a fairly substantial computational load: it's looking for some string with the text &lt;code&gt;XA:Z:&lt;/code&gt; somewhere in the line, and doesn't actually capture every situation. Some masochistic person might come along at a later date and decide that they're going to call all their reads &lt;code&gt;HAXXA:Z:AWESOME!:&amp;lt;readNumber&amp;gt;&lt;/code&gt;, in which case a tweak to this grep search would be needed to make sure that there's a space (or more specifically a tab character) prior to the &lt;code&gt;XA:Z:&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now I do a check for &lt;em&gt;any&lt;/em&gt; duplicated read names, just to be sure:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ samtools view output_filtered.bam | awk '{print $&lt;/span&gt;1}' | sort | uniq -d&#xA;ERR063640.1194&#xA;ERR063640.1429&#xA;ERR063640.1761&#xA;ERR063640.2336&#xA;ERR063640.2825&#xA;ERR063640.3458&#xA;ERR063640.4421&#xA;ERR063640.4474&#xA;ERR063640.4888&#xA;ERR063640.49&#xA;ERR063640.4974&#xA;ERR063640.5070&#xA;ERR063640.5130&#xA;ERR063640.5300&#xA;ERR063640.5868&#xA;ERR063640.6116&#xA;ERR063640.6198&#xA;ERR063640.6468&#xA;ERR063640.6717&#xA;ERR063640.6797&#xA;ERR063640.7322&#xA;ERR063640.750&#xA;ERR063640.7570&#xA;ERR063640.7900&#xA;ERR063640.8115&#xA;ERR063640.8405&#xA;ERR063640.911&#xA;ERR063640.9206&#xA;ERR063640.9765&#xA;ERR063640.9986&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Oh... damn. I wonder what they are:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ samtools view output_filtered.bam | grep '^ERR063640.3458[[:space:]]'&#xA;ERR063640.3458  16  tig00002961 5402    60  58S38M  *   0   0   AGGTACCATTCGATAGAGGGAGAAAGGCACTACTAAAGATTTTGCCACATTTGCTATATCCGTATCGCGAAGATCAGGACTTACTCCGCAGAAGAA    DD6HFFJBKFH=KDILKLGLJEKLKGFJIH8IKHLLMJEK:L:HBGJIHJKFLLKIHJDHLNKCK;KMKGMFKJILIIIMKI9JLKKHEJFII?CC    NM:i:0  MD:Z:38 AS:i:38 XS:i:0  SA:Z:tig00002377,202353,-,14M3I5M1I35M38S,19,5;&#xA;ERR063640.3458  2064    tig00002377 202353  19  14M3I5M1I35M38H *   0   0   AGGTACCATTCGATAGAGGGAGAAAGGCACTACTAAAGATTTTGCCACATTTGCTATA  DD6HFFJBKFH=KDILKLGLJEKLKGFJIH8IKHLLMJEK:L:HBGJIHJKFLLKIHJ  NM:i:5  MD:Z:5G48   AS:i:35 XS:i:27 SA:Z:tig00002961,5402,-,58S38M,60,0;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Aha! Supplemental alignments, which use the &lt;em&gt;official&lt;/em&gt; &lt;code&gt;SA&lt;/code&gt; tag [other canonical alignments in a chimeric alignment]. These appear to be situations where a single read has been split up, and maps to multiple locations. Note that in this case, &lt;em&gt;&lt;strong&gt;both of the alignments still have MAPQ scores of over 3&lt;/strong&gt;&lt;/em&gt;. It sounds like the questioner would also want to get rid of these situations as well. This time, there are standard flag fields as well to deal with these situations (0x800: secondary alignment). Except it's not enough to just filter the supplemental alignment, because &lt;em&gt;both&lt;/em&gt; read mappings should be removed, rather than just the one (or ones) that happened to be tagged as secondary.&lt;/p&gt;&#xA;&lt;p&gt;Luckily, BWA appears to put the &lt;code&gt;SA&lt;/code&gt; tag into all reads containing supplementary alignments (if this is not the case, I'm sure someone will correct me on that). So, I add in the &lt;code&gt;SA&lt;/code&gt; search as an additional &lt;code&gt;grep&lt;/code&gt; filter:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ samtools view -h output.bam | grep -v -e 'XA:Z:' -e 'SA:Z:' | samtools view -b &amp;gt; output_filtered.bam&#xA;$&lt;/span&gt; samtools view output_filtered.bam | awk '{print $1}' | sort | uniq -d&#xA;&amp;lt;no output&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Done. Easy peasy! &lt;code&gt;&amp;lt;/sarcasm&amp;gt;&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;... that original &amp;quot;high-effort&amp;quot; solution of using a different aligner doesn't look so bad now.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-08-19T08:45:41.163" LastActivityDate="2022-08-19T08:45:41.163" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="520" PostTypeId="2" ParentId="518" CreationDate="2017-06-07T04:50:54.920" Score="8" Body="&lt;p&gt;The parameter is used to determine how much sequence STAR indexes on each side of a splice junction to improve its alignment accuracy.  For very long reads, this may not be ideal. I am not sure if STAR is capable of including multiple splice junctions since a long read is more than likely to span more than one.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may be worthwhile to consider aligning your reads directly to transcripts and taking the reads that align poorly and using them to look for novel transcripts.&lt;/p&gt;&#xA;" OwnerUserId="64" LastEditorUserId="64" LastEditDate="2017-06-08T02:58:54.080" LastActivityDate="2017-06-08T02:58:54.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="521" PostTypeId="2" ParentId="518" CreationDate="2017-06-07T04:58:25.327" Score="4" Body="&lt;p&gt;I've found the &lt;a href=&quot;https://github.com/alexdobin/STAR/blob/master/doc/STARmanual.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;STAR manual&lt;/a&gt; to be incredibly helpful with trying to navigate my way round all the STAR command line parameters. Here's the section on the &lt;code&gt;sjdbOverhang&lt;/code&gt; parameter:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;--sjdbOverhang&lt;/code&gt; specifies the length of the genomic sequence around the annotated junction to be used in constructing the splice junctions database. Ideally, this length should be equal to the &lt;em&gt;ReadLength-1&lt;/em&gt;, where &lt;em&gt;ReadLength&lt;/em&gt; is the length of the reads. For instance, for Illumina 2x100b paired-end reads, the ideal value is 100-1=99. In case of reads of varying length, the ideal value is &lt;em&gt;max(ReadLength)-1&lt;/em&gt;. &lt;strong&gt;In most cases, the default value of 100 will work as well as the ideal value.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;[emphasis preserved from the original text]&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would recommend trying out running STAR with this value unset, or set to 100, and seeing how it goes. If you want to be particularly adventurous, you could set it to 1000 and see if anything changes.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-07T04:58:25.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="522" PostTypeId="1" CreationDate="2017-06-07T05:25:27.387" Score="6" ViewCount="1878" Body="&lt;p&gt;I'm using several datasets that are encoded using either Entrez or Ensembl IDs to specify genes, and need to decide on which to standardise on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any major reasons to use one over the other?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which tools and conversion tables are best to use for this?&lt;/p&gt;&#xA;" OwnerDisplayName="user657" LastEditorUserId="73" LastEditDate="2017-06-07T09:37:00.570" LastActivityDate="2017-06-07T09:37:00.570" Title="Entrez or Ensembl gene IDs?" Tags="|conversion|gene|public-databases|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="523" PostTypeId="2" ParentId="516" CreationDate="2017-06-07T06:15:01.453" Score="4" Body="&lt;p&gt;Figured it out! For anyone who finds this thread in the future and is wondering what is going on: HOMER, when using the -size given option, normalizes the y-axis to the number of basepairs in the intervals -- in this case, the length of the gene body. To get the &quot;raw&quot; values, as I wanted, you need to multiply by the length of the gene body to &quot;undo&quot; the normalization.&lt;/p&gt;&#xA;" OwnerUserId="651" LastActivityDate="2017-06-07T06:15:01.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="524" PostTypeId="1" AcceptedAnswerId="530" CreationDate="2017-06-07T06:21:23.533" Score="7" ViewCount="158" Body="&lt;p&gt;I have performed RNA-seq analysis using HISAT2 &amp;amp; StringTie workflow suggested in: &lt;a href=&quot;http://www.nature.com/nprot/journal/v11/n9/full/nprot.2016.095.html&quot; rel=&quot;noreferrer&quot;&gt;Transcript-level expression analysis of RNA-seq experiments with HISAT, StringTie and Ballgown&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the transcripts/exons have decimal coverage values (eg., 1.1 or 2.59).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is:&lt;/strong&gt; How can these tools report decimal coverage? If only half of the read overlaps exon will this exon have 0.5 coverage?&lt;/p&gt;&#xA;" OwnerUserId="186" LastActivityDate="2017-06-07T08:31:35.480" Title="How can HISAT2/StringTie report decimal coverage values" Tags="|rna-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="525" PostTypeId="1" AcceptedAnswerId="542" CreationDate="2017-06-07T07:20:47.170" Score="17" ViewCount="5803" Body="&lt;p&gt;I have a pipeline for generating a BigWig file from a BAM file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BAM -&amp;gt; BedGraph -&amp;gt; BigWig&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which uses &lt;code&gt;bedtools genomecov&lt;/code&gt; for the &lt;code&gt;BAM -&amp;gt; BedGraph&lt;/code&gt; part and &lt;code&gt;bedGraphToBigWig&lt;/code&gt; for the &lt;code&gt;BedGraph -&amp;gt; BigWig&lt;/code&gt; part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The use of &lt;code&gt;bedGraphToBigWig&lt;/code&gt; to create the &lt;code&gt;BigWig&lt;/code&gt; file requires a BedGraph file to reside on disk in uncompressed form as it performs seeks. This is problematic for large genomes and variable coverage BAM files when there are more step changes/lines in the &lt;code&gt;BedGraph&lt;/code&gt; file. My &lt;code&gt;BedGraph&lt;/code&gt; files are in the order of 50 Gbytes in size and all that IO for 10-20 BAM files seems unnecessary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Are there any tools capable generate &lt;code&gt;BigWig&lt;/code&gt; without having to use an uncompressed &lt;code&gt;BedGraph&lt;/code&gt; file on disk? I'd like this conversion to hapen as quickly as possible.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried the following tools, but they still create/use a &lt;code&gt;BedGraph&lt;/code&gt; intermediary file:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;deepTools&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h1&gt;Some Benchmarks, Ignoring IO&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Here are some timings I get for creating a &lt;code&gt;BigWig&lt;/code&gt; file from a &lt;code&gt;BAM&lt;/code&gt; file using 3 different pipelines. All files reside on a &lt;code&gt;tmpfs&lt;/code&gt; i.e. in memory.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;BEDTools and Kent Utils&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This is the approach taken by most.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time $(bedtools genomecov -bg -ibam test.bam -split -scale 1.0 &amp;gt; test.bedgraph \&#xA;  &amp;amp;&amp;amp; bedGraphToBigWig test.bedgraph test.fasta.chrom.sizes kent.bw \&#xA;  &amp;amp;&amp;amp; rm test.bedgraph)&#xA;&#xA;real    1m20.015s&#xA;user    0m56.608s&#xA;sys     0m27.271s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;SAMtools and Kent Utils&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Replacing &lt;code&gt;bedtools genomecov&lt;/code&gt; with &lt;code&gt;samtools depth&lt;/code&gt; and a custom &lt;code&gt;awk&lt;/code&gt; script (&lt;a href=&quot;https://gist.github.com/nathanhaigh/c8322002a42e40c479d3d766aab32547&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;depth2bedgraph.awk&lt;/code&gt;&lt;/a&gt;) to output &lt;code&gt;bedgraph&lt;/code&gt; format has a significant performance improvement:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time $(samtools depth -Q 1 --reference test.fasta test.bam \&#xA;  | mawk -f depth2bedgraph.awk \&#xA;  &amp;gt; test.bedgraph \&#xA;  &amp;amp;&amp;amp; bedGraphToBigWig test.bedgraph test.fasta.chrom.sizes kent.bw \&#xA;  &amp;amp;&amp;amp; rm test.bedgraph)&#xA;&#xA;real    0m28.765s&#xA;user    0m44.999s&#xA;sys     0m1.166s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Although it is has less features, we used &lt;code&gt;mawk&lt;/code&gt; here as it's faster than &lt;code&gt;gawk&lt;/code&gt; (we don't need those extra features here).&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Parallelising with xargs&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If you want to have a &lt;code&gt;BigWig&lt;/code&gt; file per chromosome, you can easily parallelise this across chromosome/reference sequences. We can use &lt;code&gt;xargs&lt;/code&gt; to run 5 parallel BAM-&gt;BedGraph-&gt;BigWig pipelines, each using the &lt;code&gt;tmpfs&lt;/code&gt; mounted &lt;code&gt;/dev/shm&lt;/code&gt; for the intermediary &lt;code&gt;BedGraph&lt;/code&gt; files.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cut -f1 test.fasta.chrom.sizes \&#xA;  | xargs -I{} -P 5  bash -c 'mkdir /dev/shm/${1} \&#xA;  &amp;amp;&amp;amp; samtools depth -Q 1 --reference test.fasta -r &quot;${1}&quot; test.bam \&#xA;  | mawk -f scripts/depth2bedgraph.awk \&#xA;  &amp;gt; &quot;/dev/shm/${1}/test.bam.bedgraph&quot; \&#xA;  &amp;amp;&amp;amp; mkdir &quot;./${1}&quot; \&#xA;  &amp;amp;&amp;amp; bedGraphToBigWig \&#xA;    &quot;/dev/shm/${1}/test.bam.bedgraph&quot; \&#xA;    test.fasta.chrom.sizes \&#xA;    &quot;./${1}/test.bam.bw&quot; \&#xA;  &amp;amp;&amp;amp; rm &quot;/dev/shm/${1}/test.bam.bedgraph&quot;' -- {}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;deepTools&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Let's see how &lt;code&gt;deepTools&lt;/code&gt; performs.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time bamCoverage --numberOfProcessors max \&#xA;  --minMappingQuality 1 \&#xA;  --bam test.bam --binSize 1 --skipNonCoveredRegions \&#xA;  --outFileName deeptools.bw&#xA;&#xA;real    0m40.077s&#xA;user    3m56.032s&#xA;sys     0m9.276s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="400" LastEditorUserId="96" LastEditDate="2017-06-08T08:47:24.203" LastActivityDate="2018-07-12T10:25:05.437" Title="BAM to BigWig without intermediary BedGraph" Tags="|sam|file-formats|format-conversion|" AnswerCount="3" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="526" PostTypeId="2" ParentId="522" CreationDate="2017-06-07T08:08:25.347" Score="2" Body="&lt;p&gt;I think there is no general answer to your question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which type of ID you will use depends on how practical it will be for your downstream processing. This depends on what other source of information you have, how much conversion work you will need to do in order to match your data to these information if you use one or the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the choice may emerge by trial and error, as you realize that a given choice is actually not convenient in your particular use case. Try to set up easy to use conversion procedures that you can easily insert in some steps of your workflow, so that you can easily switch from one choice to the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concerning the tools to do the conversion, see the following question and its answers: &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/488/292&quot;&gt;Converting gene names from one public database format to another&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2017-06-07T08:08:25.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="527" PostTypeId="1" AcceptedAnswerId="580" CreationDate="2017-06-07T08:24:32.383" Score="5" ViewCount="221" Body="&lt;p&gt;miRBase 21 was published June 26, 2014 and was still in its growth phase. Why is it not being updated anymore or the project declared officially dead? ENSEMBL also uses miRBase as a starting point (&lt;a href=&quot;http://www.ensembl.org/info/genome/genebuild/ncrna.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.ensembl.org/info/genome/genebuild/ncrna.html&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do people use now as a good reference for miRNAs especially for less well annotated/sequenced species?&lt;/p&gt;&#xA;" OwnerUserId="391" LastEditorUserId="391" LastEditDate="2017-06-07T08:34:20.397" LastActivityDate="2019-03-25T00:56:11.580" Title="Where is an up to date miRNA database and what happened to miRBase?" Tags="|sequence-annotation|database|" AnswerCount="5" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="528" PostTypeId="1" AcceptedAnswerId="533" CreationDate="2017-06-07T08:24:35.987" Score="11" ViewCount="1449" Body="&lt;p&gt;I'm trying to find a programmatic way to automatically extract the following information from a &lt;a href=&quot;http://www.wwpdb.org/documentation/file-format&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB file&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;RNA sequence &lt;/li&gt;&#xA;&lt;li&gt;Secondary structure restraints in bracket format, e.g. &lt;code&gt;. (( . ( . ) . ))&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Does software exist that can take a PDB file as input and generate these two pieces of information? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g. file. &lt;a href=&quot;http://www.rcsb.org/pdb/explore.do?structureId=3NDB&quot; rel=&quot;nofollow noreferrer&quot;&gt;3NDB_ba.pdb&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The nucleotide sequence is: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;gUCUCGUCCCGUGGGGCUCGGCGGUGGGGGAGCAUCUCCUGUAGGGGAGAUGUAACCCCCUUUACCUGCCGAACCCCGCCAGGCCCGGAAGGGAGCAACGGUAGGCAGGACGUCGGCGCUCACGGGGGUGCGGGAC&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the secondary structure :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;.(((.(..(((((((((.(((((..(((((.(((((((((....)))))))))..)))))....((((((.....(((.....(((....))).....)))..)))))).))))))).))))))).....).))).&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="682" LastEditorUserId="640" LastEditDate="2017-06-08T18:31:15.580" LastActivityDate="2017-06-08T18:31:15.580" Title="How to extract RNA sequence and secondary structure restrains from a PDB file" Tags="|public-databases|pdb|rna|rna-structure|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="529" PostTypeId="2" ParentId="456" CreationDate="2017-06-07T08:28:15.090" Score="1" Body="&lt;p&gt;From my experience, there is few about CRISPR pooled screenings in R. One of them is &lt;a href=&quot;https://github.com/jyyu/ScreenBEAM&quot; rel=&quot;nofollow noreferrer&quot;&gt;ScreenBEAM&lt;/a&gt;. It uses a linear mixture model to determine significance at gene level by considering all the guides together at the same step in the analysis. However, it does not work very well for me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other ways of analyze these screenings outside of R that are easy to use such the suggested MAGeCK or others such HiTSelect, BAGEL or &lt;a href=&quot;http://crispr-analyzer.dkfz.de/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CRISPR-Analyzer&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CRISPR-Analyzer is an online tool that could be useful and that recapitulates some of the mentioned methods in a single analysis. However, if you want to work from command line I would recommend you BAGEL for gene essentiality experiments and MAGeCK for other experimental setup (i.e, control vs treatment)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry for not putting all the links but I can only put two at the moment!&lt;/p&gt;&#xA;" OwnerUserId="678" LastEditorUserId="678" LastEditDate="2017-06-07T12:08:54.520" LastActivityDate="2017-06-07T12:08:54.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="530" PostTypeId="2" ParentId="524" CreationDate="2017-06-07T08:31:35.480" Score="6" Body="&lt;p&gt;I suspect this is average coverage across the transcript or exon. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Formula for average coverage:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://latex.codecogs.com/gif.latex?Cov%20%3D%20N%20%5Ccdot%20%5Cfrac%7BL%7D%7BT%7D&quot; alt=&quot;coverage-formula&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;N: number of reads mapping/aligning to your transcript/exon&lt;/li&gt;&#xA;&lt;li&gt;L: average read length&lt;/li&gt;&#xA;&lt;li&gt;T: Transcript/exon length&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="81" LastActivityDate="2017-06-07T08:31:35.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="531" PostTypeId="1" AcceptedAnswerId="547" CreationDate="2017-06-07T08:39:33.900" Score="9" ViewCount="5653" Body="&lt;p&gt;I am trying to use &lt;code&gt;samtools depth&lt;/code&gt; (v1.4) with the -a option and a bed file listing the human chromosomes chr1-chr22, chrX, chrY, and chrM to print out the coverage at every position:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat GRCh38.karyo.bed | awk '{print $3}' | datamash sum 1&#xA;3088286401&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to know how to run &lt;code&gt;samtools depth&lt;/code&gt; so that it produces 3,088,286,401 entries when run against a GRCh38 bam file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools depth -b $bedfile -a $inputfile&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried it for a few bam files that were aligned the same way, and I get differing number of entries:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;3087003274&#xA;3087005666&#xA;3087007158&#xA;3087009435&#xA;3087009439&#xA;3087009621&#xA;3087009818&#xA;3087010065&#xA;3087010408&#xA;3087010477&#xA;3087010481&#xA;3087012115&#xA;3087013147&#xA;3087013186&#xA;3087013500&#xA;3087149616&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a special flag in &lt;code&gt;samtools depth&lt;/code&gt; so that it reports all entries from the bed file?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;code&gt;samtools depth&lt;/code&gt; is not the best tool for this, what would be the equivalent with &lt;code&gt;sambamba depth base&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sambamba depth base --min-coverage=0 --regions $bedfile $inputfile&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any other options?&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="57" LastEditDate="2017-06-07T22:19:56.217" LastActivityDate="2017-06-07T22:34:32.847" Title="samtools depth print out all positions" Tags="|sam|samtools|sambamba|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="532" PostTypeId="2" ParentId="527" CreationDate="2017-06-07T10:04:21.943" Score="2" Body="&lt;p&gt;I would guess DIANA-Tarbase ( evidences, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/25416803&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pubmed/25416803&lt;/a&gt; ) and TargetScan (predictions, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/26267216&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pubmed/26267216&lt;/a&gt; ) are good starting points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Especially for the less annotated species, TargetScan might be helpful.&lt;/p&gt;&#xA;" OwnerUserId="690" LastActivityDate="2017-06-07T10:04:21.943" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="533" PostTypeId="2" ParentId="528" CreationDate="2017-06-07T10:06:32.310" Score="8" Body="&lt;p&gt;I suggest you take a look at &lt;code&gt;rna-pdb-tools&lt;/code&gt; we do way more than you need! :-) The tools can get you a sequence, secondary structure and much more using various algorithms, and all is well documented &lt;a href=&quot;http://rna-pdb-tools.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://rna-pdb-tools.readthedocs.io/en/latest/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get sequence &lt;a href=&quot;http://rna-pdb-tools.readthedocs.io/en/latest/main.html#get-sequence&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://rna-pdb-tools.readthedocs.io/en/latest/main.html#get-sequence&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ rna_pdb_tools.py --get_seq 5_solution_1.pdb&#xA;&amp;gt; 5_solution_1.pdb A:1-576&#xA;CAUCCGGUAUCCCAAGACAAUCUCGGGUUGGGUUGGGAAGUAUCAUGGCUAAUCACCAUGAUGCAAUCGGGUUGAACACUUAAUUGGGUUAAAACGGUGGGGGACGAUCCCGUAACAUCCGUCCUAACGGCGACAGACUGCACGGCCCUGCCUCAGGUGUGUCCAAUGAACAGUCGUUCCGAAAGGAAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or you can get sequence and secondary structure via x3dna (&lt;a href=&quot;http://x3dna.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://x3dna.org/&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[mm] py3dna$ git:(master) ✗ ./rna_x3dna.py test_data/1xjr.pdb&#xA;test_data/1xjr.pdb&#xA; &amp;gt;1xjr nts=47 [1xjr] -- secondary structure derived by DSSR&#xA; gGAGUUCACCGAGGCCACGCGGAGUACGAUCGAGGGUACAGUGAAUU&#xA; ..(((((((...((((.((((.....))..))..))).).)))))))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The problem is not trivial. I hope rna-pdb-tools has stuff that can solve this issue for you (&lt;a href=&quot;http://rna-pdb-tools.readthedocs.io/en/latest/want.html#module-rna_pdb_tools.utils.rna_x3dna.rna_x3dna&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://rna-pdb-tools.readthedocs.io/en/latest/want.html#module-rna_pdb_tools.utils.rna_x3dna.rna_x3dna&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="640" LastEditorUserId="640" LastEditDate="2017-06-07T18:36:43.480" LastActivityDate="2017-06-07T18:36:43.480" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="534" PostTypeId="5" CreationDate="2017-06-07T11:15:08.160" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-06-07T11:15:08.160" LastActivityDate="2017-06-07T11:15:08.160" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="535" PostTypeId="4" CreationDate="2017-06-07T11:15:08.160" Score="0" Body="A file format used in structural biology to describe the 3-D coordinates of atoms of a biological structure, often a protein. Can also refer to the Protein Data Bank database, which stores all the submitted biological structures, most of which were obtained using X-ray crystallography." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-03T16:14:55.517" LastActivityDate="2018-03-03T16:14:55.517" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="537" PostTypeId="2" ParentId="525" CreationDate="2017-06-07T12:07:52.207" Score="2" Body="&lt;p&gt;The &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/QuasR.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;QuasR&lt;/a&gt; package in R has a useful function called &lt;code&gt;qExportWig&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To use it you first need to create a sample sheet and a &lt;code&gt;qProject&lt;/code&gt; object - this can be done starting with bam files or fastq files (in which case it also  performs alignments). You can then create wig or bigWig files without writing a bedGraph file to disk.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example would look something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(QuasR)&#xA;&#xA;# Create qProject object using qAlign&#xA;# Sample file should contain tab-separated file names (BAM or fastq) and sample names&#xA;myproj &amp;lt;- qAlign(&quot;samples.txt&quot;, genome=mygenome, paired=&quot;no&quot;)&#xA;&#xA;# Export all samples in project as bigWig, scaling to mean number of alignments by default&#xA;qExportWig(myproj, binsize=50, createBigWig=T)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It has a lot of flexible options, including scaling or log-transforming counts, shifting reads and changing the bin size.&lt;/p&gt;&#xA;" OwnerUserId="182" LastEditorUserId="182" LastEditDate="2017-06-08T06:56:00.910" LastActivityDate="2017-06-08T06:56:00.910" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="538" PostTypeId="5" CreationDate="2017-06-07T13:13:19.417" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-06-07T13:13:19.417" LastActivityDate="2017-06-07T13:13:19.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="539" PostTypeId="4" CreationDate="2017-06-07T13:13:19.417" Score="0" Body="questions involving microarray analysis (e.g. Illumina BeadChip, Affymetrix GeneChip)" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-08T11:39:46.183" LastActivityDate="2017-06-08T11:39:46.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="540" PostTypeId="1" AcceptedAnswerId="545" CreationDate="2017-06-07T13:23:52.380" Score="23" ViewCount="15041" Body="&lt;p&gt;When you look at all the genome files available from Ensembl. You are presented with a bunch of options. Which one is the best to use/download?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have a combination of choices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First part options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;dna_sm&lt;/strong&gt; - Repeats soft-masked (converts repeat nucleotides to lowercase)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;dna_rm&lt;/strong&gt; - Repeats masked (converts repeats to to N's)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;dna&lt;/strong&gt; - No masking&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Second part options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;.toplevel&lt;/strong&gt; - Includes haplotype information (not sure how aligners deal with this)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;.primary_assembly&lt;/strong&gt; - Single reference base per position&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Right now I usually use a non-masked primary assembly for analysis, so in the case of humans:&#xA;&lt;em&gt;Homo_sapiens.GRCh38.dna.primary_assembly.fa.gz&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this make sense for standard RNA-Seq, ChIP-Seq, ATAC-Seq, CLIP-Seq, scRNA-Seq, etc... ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In what cases would I prefer other genomes? Which tools/aligners take into account softmasked repeat regions?&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2023-08-11T10:27:14.867" Title="What Ensembl genome version should I use for alignments? (e.g. toplevel.fa vs. primary_assembly.fa)" Tags="|fasta|genome|ensembl|biomart|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="541" PostTypeId="2" ParentId="540" CreationDate="2017-06-07T13:29:18.277" Score="13" Body="&lt;p&gt;There's rarely a good reason to use a hard-masked genome (sometimes for blast, but that's it). For that reason, we use soft-masked genomes, which only have the benefit of showing roughly where repeats are (we never make use of this for our *-seq experiments, but it's there in case we ever want to).&lt;/p&gt;&#xA;&lt;p&gt;For primary vs. toplevel, very few aligners can properly handle additional haplotypes. If you happen to be using BWA, then the toplevel assembly would benefit you, but only if you use a dedicated wrapper to handle the ALT information, see &lt;a href=&quot;https://github.com/lh3/bwa/blob/master/README-alt.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;bwakit&lt;/a&gt;. If you use BWA (bwa-mem) right from the command line without this wrapper then &lt;strong&gt;do not use&lt;/strong&gt; the toplevel assembly. For STAR/hisat2/bowtie2/BBmap/etc. the haplotypes will just cause you problems due to increasing multimapper rates incorrectly. Note that none of these actually use soft-masking.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="16676" LastEditDate="2023-08-11T10:27:14.867" LastActivityDate="2023-08-11T10:27:14.867" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="542" PostTypeId="2" ParentId="525" CreationDate="2017-06-07T13:32:48.943" Score="11" Body="&lt;p&gt;This can be done in R very easily from an indexed .bam file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given single-end file for &lt;strong&gt;sample1&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(GenomicAlignments)&#xA;library(rtracklayer)&#xA;&#xA;## read in BAM file (use readGAlignmentPairs for paired-end files)&#xA;gr &amp;lt;- readGAlignments('sample1.bam')&#xA;&#xA;## convert to coverages&#xA;gr.cov &amp;lt;- coverage(gr)&#xA;&#xA;## export as bigWig&#xA;export.bw(gr.cov,'sample1.bigwig')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Be aware that this method doesn't include normalization steps (such as normalizing to total coverage). Most of these additional steps can be added if necessary. &lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2017-06-07T13:32:48.943" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="543" PostTypeId="2" ParentId="540" CreationDate="2017-06-07T14:08:52.377" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Which tools/aligners take into account softmasked repeat regions?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you're doing whole genome - whole genome alignment (rather than read alignment) then using the softmasked genome is definitely best. Tools suitable for such large scale alignments task tend to skip marked repeats completely in their initial steps to prevent the build up of bogus short alignments that can have a massive performance impact in terms of time and memory usage. For example, &lt;a href=&quot;http://www.bx.psu.edu/miller_lab/dist/README.lastz-1.02.00/README.lastz-1.02.00a.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;LASTZ&lt;/a&gt; skips lower case letters during the seeding stage. &lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2017-06-07T14:08:52.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="544" PostTypeId="2" ParentId="540" CreationDate="2017-06-07T14:44:42.053" Score="2" Body="&lt;p&gt;TOPLEVEL&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These files contains all sequence regions flagged as toplevel in an Ensembl&#xA;schema. This includes chromsomes, regions not assembled into chromosomes and&#xA;N padded haplotype/patch regions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g: I used the soft masked assemblies for genome annotation pipelines like MAKER, also toplevel unmasked ones for RNA-seq, ChipSeq analysis&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PRIMARY ASSEMBLY&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Primary assembly contains all toplevel sequence regions excluding haplotypes&#xA;and patches. This file is best used for performing sequence similarity searches&#xA;where patch and haplotype sequences would confuse analysis.&lt;/p&gt;&#xA;" OwnerUserId="705" LastActivityDate="2017-06-07T14:44:42.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="545" PostTypeId="2" ParentId="540" CreationDate="2017-06-07T15:04:08.313" Score="9" Body="&lt;p&gt;Generally, you should use the soft-masked or unmasked primary assembly. Cross-species whole-genome aligners, especially older ones, do need to know soft-masked regions; otherwise they can be impractically slow for mammalian genomes. Modern read aligners are designed to work with repeats efficiently and therefore they don't need to see the soft mask.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For GRCh38, though, I would recommend to use the official build &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/001/405/GCA_000001405.15_GRCh38/seqs_for_alignment_pipelines.ucsc_ids/&quot; rel=&quot;noreferrer&quot;&gt;at GRC FTP&lt;/a&gt;. Most people will probably choose &quot;no_alt_analysis_set&quot;. Using the Ensembl version is discouraged due to its chromosome naming. We more often use &quot;chr1&quot; instead of &quot;1&quot; for GRCh38. At one point, Ensembl actually agreed to use &quot;chr1&quot; as well, but didn't make that happen due to technical issues, I guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to alternate haplotypes, most aligners can't work with them; no variant callers can take the advantage of these sequences, either. When you align to a reference genome containing haplotypes with an aligner not supporting these extra sequences, you will get poor mapping results.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-06-07T15:04:08.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="546" PostTypeId="2" ParentId="15" CreationDate="2017-06-07T15:34:51.743" Score="12" Body="&lt;p&gt;&lt;strong&gt;TL;DR:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BWA-backtrack is based on &lt;a href=&quot;https://en.wikipedia.org/wiki/Backtracking&quot; rel=&quot;noreferrer&quot;&gt;backtracking&lt;/a&gt;.&#xA;This approach is appropriate only when the dissimilarity between the reads and the reference is low,&#xA;or when you want to find all best hits or enumerate all possible alignments up to a specified&#xA;number of errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In all other situations, BWA-MEM is preferable as it can,&#xA;thanks to its sophisticated strategy based on maximum exact matches,&#xA;deal with errors better and also automatically switch between local and&#xA;global alignment modes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Long description:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to provide some algorithmic insight since I believe that it&#xA;can be very useful in this case.&#xA;Both BWA-backtrack and BWA-MEM use the same indexing strategy&#xA;(heavily relying on BWT-index), but the actual algorithms are quite different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/FM-index&quot; rel=&quot;noreferrer&quot;&gt;BWT-index&lt;/a&gt; (and also other full-text indexes such as&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Suffix_array&quot; rel=&quot;noreferrer&quot;&gt;suffix arrays&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Suffix_tree&quot; rel=&quot;noreferrer&quot;&gt;suffix trees&lt;/a&gt;)&#xA;can easily find exact matches (imagine Ctrl+F-like search in a text editor),&#xA;but any differences between the read and the reference,&#xA;such as sequencing errors or genomic variants,&#xA;make the situation complicated. One then needs to&#xA;somehow transform inexact matching to exact matching,&#xA;and all three BWA mappers (note that there exists also BWA-SW, but it is deprecated)&#xA;use quite different strategies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;BWA-backtrack&lt;/strong&gt; looks for substrings of the reference, which would be similar&#xA;to the entire read (end-to-end) using an algorithm&#xA;called &lt;a href=&quot;https://en.wikipedia.org/wiki/Backtracking&quot; rel=&quot;noreferrer&quot;&gt;backtracking&lt;/a&gt;.&#xA;First, it searches occurences of the read without any &quot;corrections&quot;.&#xA;If nothing found, it consideres all possible single edits; then two edits, etc.&#xA;To make mapping efficient, one usually wants to stop with the first&#xA;found alignment as this would be the best one.&#xA;When required, it is also possible to find the other equally good alignments&#xA;or to enumerate all alignments up to some edit distance or withing some divergance rate&#xA;(see the -N option of BWA-backtrack).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It turns out that&#xA;the time required for finding an alignment&#xA;can be exponential in the number of errors, which is probably the main problem of&#xA;backtracking-based approaches.&#xA;To prevent huge overheads due to dissimilar reads,&#xA;one needs to limit the number of allowed errors to some reasonable number&#xA;(see maxDiff in the &lt;a href=&quot;http://bio-bwa.sourceforge.net/bwa.shtml&quot; rel=&quot;noreferrer&quot;&gt;BWA man page&lt;/a&gt;)&#xA;and consider the other reads unaligned.&#xA;In the case of BWA-backtrack,&#xA;the minimum required identity level is ~97% with the default options (see the -n option).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, the algorithm is more complicated and uses various heuristics such as&#xA;seed-and-extend or Z-dropoff in order to make the computation fast enough (at the price of lower accuracy). If you are interested&#xA;in more details, all these tricks are well described in the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/19451168&quot; rel=&quot;noreferrer&quot;&gt;paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;BWA-MEM&lt;/strong&gt; uses quite a different strategy. It detects long exact matches&#xA;between the read and the reference, and then chains them into local or global alignments,&#xA;based on what is more appropriate in that specific case.&#xA;Such an automatic local-global switching can be very powerful and&#xA;BWA-MEM works well with various types of data&#xA;(short reads, long reads, low error rates, high error rates, etc.).&lt;/p&gt;&#xA;" OwnerUserId="425" LastActivityDate="2017-06-07T15:34:51.743" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="547" PostTypeId="2" ParentId="531" CreationDate="2017-06-07T16:20:19.863" Score="6" Body="&lt;p&gt;You might try using &lt;a href=&quot;http://bedtools.readthedocs.io/en/latest/content/tools/genomecov.html&quot; rel=&quot;noreferrer&quot;&gt;bedtools genomecov&lt;/a&gt; instead. If you provide the -d option, it reports the coverage at every position in the BAM file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools genomecov -d -ibam $inputfile &amp;gt; &quot;${inputfile}.genomecov&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can also provide a BED file if you just want to calculate in the target  region.&lt;/p&gt;&#xA;" OwnerUserId="110" LastActivityDate="2017-06-07T16:20:19.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="548" PostTypeId="1" AcceptedAnswerId="550" CreationDate="2017-06-07T16:54:42.373" Score="15" ViewCount="965" Body="&lt;p&gt;As a small introductory project, I want to compare genome sequences of  different strains of influenza virus.&lt;/p&gt;&#xA;&lt;p&gt;What are the publicly available databases of influenza virus gene/genome sequences?&lt;/p&gt;&#xA;" OwnerUserId="711" LastEditorUserId="3848" LastEditDate="2020-11-30T20:23:43.250" LastActivityDate="2020-11-30T20:23:43.250" Title="Publicly available genome sequence database for viruses?" Tags="|database|public-databases|covid-19|genome-sequencing|sars-cov-2|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="549" PostTypeId="2" ParentId="548" CreationDate="2017-06-07T17:31:26.533" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genomes/FLU/Database/nph-select.cgi?go=database&quot; rel=&quot;nofollow noreferrer&quot;&gt;Influenza virus resource&lt;/a&gt; at NCBI or &lt;a href=&quot;https://www.fludb.org/brc/home.spg?decorator=influenza&quot; rel=&quot;nofollow noreferrer&quot;&gt;FluDB&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="634" LastActivityDate="2017-06-07T17:31:26.533" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="550" PostTypeId="2" ParentId="548" CreationDate="2017-06-07T17:45:42.067" Score="17" Body="&lt;p&gt;There area few different influenza virus database resources:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;http://www.fludb.org&quot; rel=&quot;noreferrer&quot;&gt;Influenza Research Database (IRD)&lt;/a&gt; (a.k.a FluDB - based upon URL)&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A NIAID Bioinformatics Resource Center or BRC which highly curates the data brought in and integrates it with numerous other relevant data types&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genomes/FLU/Database/nph-select.cgi?go=database&quot; rel=&quot;noreferrer&quot;&gt;NCBI Influenza Virus Resource&lt;/a&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A sub-project of the NCBI with data curated over and above the GenBank data that is part of the NCBI&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;http://platform.gisaid.org/&quot; rel=&quot;noreferrer&quot;&gt;GISAID EpiFlu Database&lt;/a&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A database of sequences from the Global Initiative on Sharing All Influenza Data. Has unique data from many countries but requires user agree to a data sharing policy.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;http://openflu.vital-it.ch/browse.php&quot; rel=&quot;noreferrer&quot;&gt;OpenFluDB&lt;/a&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Former GISAID database that contains some sequence data that GenBank does not have.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For those who also may be interested in other virus databases, there are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.viprbrc.org&quot; rel=&quot;noreferrer&quot;&gt;Virus Pathogen Resource (VIPR)&lt;/a&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A companion portal to the IRD, which hosts curated and integrated data for most other NIAID A-C virus pathogens including (but not limited to) Ebola, Zika, Dengue, Enterovirus, and Hepatitis C&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.hiv.lanl.gov/&quot; rel=&quot;noreferrer&quot;&gt;LANL HIV database&lt;/a&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Los Alamos National Laboratory HIV database with HIV data and many useful tools for all virus bioinformatics&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pave.niaid.nih.gov&quot; rel=&quot;noreferrer&quot;&gt;PaVE: Papilloma virus genome database&lt;/a&gt; (from quintik comment)&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;NIAID developed and maintained Papilloma virus bioinformatics portal&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I used to work for the IRD / VIPR and currently work for NIAID.&lt;/p&gt;&#xA;" OwnerUserId="714" LastEditorUserId="-1" LastEditDate="2017-08-22T21:01:56.533" LastActivityDate="2017-08-22T21:01:56.533" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="551" PostTypeId="2" ParentId="528" CreationDate="2017-06-07T19:22:15.420" Score="4" Body="&lt;h2&gt;Context&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;PDB file format&lt;/strong&gt; is a fixed-column file format designed in 1970s for storing structural models of macromolecules. The format has been around for long time, has many uses, and although it has &lt;a href=&quot;https://www.wwpdb.org/documentation/file-format-content/format33/v3.3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;official spec&lt;/a&gt; the files in circulation may not strictly conform to it. It always has a list of atoms with coordinates&#xA;(the first two lines are added to stress that it's a fixed-column format, they are not part of the file):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;         1         2         3         4         5         6         7         8&#xA;12345678901234567890123456789012345678901234567890123456789012345678901234567890&#xA;ATOM      1  N   VAL A   1       3.320  14.780   4.844  1.00 35.53           N&#xA;ATOM      2  CA  VAL A   1       3.577  16.239   4.984  1.00 35.39           C&#xA;ATOM      3  C   VAL A   1       4.896  16.398   5.727  1.00 30.43           C&#xA;ATOM      4  O   VAL A   1       5.143  15.702   6.732  1.00 30.51           O&#xA;ATOM      5  CB  VAL A   1       2.343  16.975   5.494  1.00 45.39           C&#xA;ATOM      6  CG1 VAL A   1       2.586  18.497   5.590  1.00 60.06           C&#xA;ATOM      7  CG2 VAL A   1       1.103  16.811   4.634  1.00 53.93           C&#xA;ATOM      8  N   LEU A   2       5.748  17.241   5.158  1.00 28.04           N&#xA;ATOM      9  CA  LEU A   2       7.116  17.471   5.661  1.00 24.31           C&#xA;ATOM     10  C   LEU A   2       7.166  18.490   6.792  1.00 24.00           C&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or, to show some RNA:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ATOM     42  N3    G B   2       9.252  12.871  -1.168  1.00 36.98           N&#xA;ATOM     43  C4    G B   2       8.424  12.964  -2.233  1.00 34.09           C&#xA;ATOM     44  P     A B   3      10.376   8.321  -4.834  1.00 40.53           P&#xA;ATOM     45  OP1   A B   3      11.773   8.279  -5.364  1.00 38.97           O&#xA;ATOM     46  OP2   A B   3       9.396   7.283  -5.218  1.00 39.32           O&#xA;ATOM     47  O5'   A B   3      10.429   8.473  -3.211  1.00 36.55           O&#xA;ATOM     48  C5'   A B   3      11.698   8.232  -2.554  1.00 35.13           C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;Protein Data Bank&lt;/strong&gt; -- international institution that archives and annotates structural models of biological molecules that anyone can &lt;a href=&quot;https://www.wwpdb.org/deposition/deposition-resources&quot; rel=&quot;nofollow noreferrer&quot;&gt;deposit&lt;/a&gt; -- is now using &lt;strong&gt;mmCIF&lt;/strong&gt; as the primary format. The fixed-column PDB format had inherent limitations (max. 99,999 atoms, but also it was hard to include additional info). They still generate PDB files if possible (i.e. except the largest structures), but the only format accepted now for depositions is mmCIF.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;mmCIF has a rather not intuitive &lt;a href=&quot;https://www.iucr.org/resources/cif/spec/version1.1/cifsyntax&quot; rel=&quot;nofollow noreferrer&quot;&gt;CIF syntax&lt;/a&gt; (think JSON or XML, but designed before XML),&#xA;plus ontology defined by &lt;a href=&quot;http://mmcif.wwpdb.org/dictionaries/mmcif_pdbx_v50.dic/Groups/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDBx/mmCIF dictionary&lt;/a&gt; (think XML Schema).&#xA;The content of mmCIF is divided into table-like categories with relations between tables (it was designed at the peak of RDMBS popularity) and is a bit harder to work with, so the old PDB format is more popular.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Sequence from PDB&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The PDB format has a record called &lt;code&gt;SEQRES&lt;/code&gt; that explicitly lists the sequence, for example (5NEO):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SEQRES   1 A   18    G   G   U   G   G   G   G   A   C   G   A   C   C          &#xA;SEQRES   2 A   18    C   C   A CBV   C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The mmCIF format has more details:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;_entity_poly.pdbx_seq_one_letter_code       'GGUGGGGACGACCCCA(CBV)C' &#xA;_entity_poly.pdbx_seq_one_letter_code_can   GGUGGGGACGACCCCACC &#xA;_entity_poly.pdbx_strand_id                 A &#xA;&#xA;loop_&#xA;_entity_poly_seq.entity_id &#xA;_entity_poly_seq.num &#xA;_entity_poly_seq.mon_id &#xA;_entity_poly_seq.hetero &#xA;1 1  G   n &#xA;1 2  G   n &#xA;1 3  U   n &#xA;1 4  G   n &#xA;1 5  G   n &#xA;1 6  G   n &#xA;1 7  G   n &#xA;1 8  A   n &#xA;1 9  C   n &#xA;1 10 G   n &#xA;1 11 A   n &#xA;1 12 C   n &#xA;1 13 C   n &#xA;1 14 C   n &#xA;1 15 C   n &#xA;1 16 A   n &#xA;1 17 CBV n &#xA;1 18 C   n &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The sequence is conveniently extracted for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, &lt;code&gt;_entity_poly_seq&lt;/code&gt; in mmCIF includes information about microheterogeneity, and the SEQRES record doesn't. It is relevant when the model has two alternative residues in the same place, because part of the sample had a point mutation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PDB files that didn't come from PDB usually don't have the SEQRES record.&#xA;You may get the sequence from the list of atoms (residue names in columns 18-20), but some residues may be missing in the atom list if atomic positions could not be determined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it should be easy to extract the sequence either way.&#xA;For example, BioPython has module BioSeqIO with two PDB pseudo-formats:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;pdb-seqres - Reads a Protein Data Bank (PDB) file to determine the complete protein sequence as it appears in the header (no dependencies).&lt;/li&gt;&#xA;&lt;li&gt;pdb-atom - Uses Bio.PDB to determine the (partial) protein sequence as it appears in the structure based on the atom coordinate section of the file (requires NumPy for Bio.PDB).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;I just tried BioPython 1.66 and while &lt;code&gt;SeqIO.parse()&lt;/code&gt; can extract protein sequence, it fails with RNA, such as the 5NEO above. Ooops.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Secondary structure restraints&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;We may be thinking about different restraints here. I'll write about restraints used in refinement.&#xA;The experimental data alone is normally not sufficient to refine a model, so one needs restraints that represent prior knowledge - lengths of atomic bonds, angles, planarity restraints, restraints based on local similarity to other structures that were determined from higher resolution data, etc, whatever can help to make a sensible model. Refinement programs (such as Refmac, BUSTER, Phenix.refine) try to fit the model to the data and satisfy geometrical restraints at the same time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Macromolecular crystallographic software is quite fragmented and one normally uses several different programs in the process. It happens that the programs to generate secondary structure restraints are separate from the actual refinement programs. (Secondary structure restraints are less essential than restraints for covalent bonds). CCP4 has a program called LibG that makes DNA/RNA restraints for Refmac. Phenix has a program called &lt;a href=&quot;https://www.phenix-online.org/documentation/reference/secondary_structure_restraints.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;phenix.secondary_structure_restraints&lt;/a&gt;,&#xA;or you can use a &lt;a href=&quot;http://rna.ucsc.edu/pdbrestraints/&quot; rel=&quot;nofollow noreferrer&quot;&gt;server from UCSC&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These restraints don't have a &quot;bracket format&quot;, but they explicitly specify expected distances and angles between atoms. For example (for phenix.refine):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  bond { &#xA;    action = *add &#xA;    atom_selection_1 = chain  A and resid    1  and name  O6 &#xA;    atom_selection_2 = chain  A and resid   18  and name  N4 &#xA;    distance_ideal = 2.91&#xA;    sigma = 0.1&#xA;  }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or (for Refmac):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;exte dist first chain A resi 16 ins  . atom N6 second chain A resi 3 ins . atom O4 value 2.94 sigma 0.15 type 1 &#xA;exte dist first chain A resi 16 ins  . atom N1 second chain A resi 3 ins . atom N3 value 2.84 sigma 0.1 type 1 &#xA;exte torsion first chain A resi 16 ins  . atom C2 second chain A resi 16 ins . atom N1 third chain A resi 3 ins . atom N3 fourth chain A resi 3 ins . atom C4 value 180 sigma 15 type 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="266" LastEditorUserId="266" LastEditDate="2017-06-07T23:53:58.087" LastActivityDate="2017-06-07T23:53:58.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="552" PostTypeId="2" ParentId="548" CreationDate="2017-06-07T19:58:48.613" Score="4" Body="&lt;p&gt;In addition to what others have suggested, I would also recommend &lt;a href=&quot;https://pave.niaid.nih.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PaVE&lt;/a&gt; as a resource. This is a curated database maintained by the NIAID and current holds over 300 papilloma virus genomes.&lt;/p&gt;&#xA;" OwnerUserId="712" LastEditorUserId="77" LastEditDate="2017-06-08T06:47:19.023" LastActivityDate="2017-06-08T06:47:19.023" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="553" PostTypeId="1" AcceptedAnswerId="555" CreationDate="2017-06-07T22:30:55.380" Score="6" ViewCount="82" Body="&lt;p&gt;I'm a newcomer to the world of bioinformatics, and in need of help solving a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to take a list of human proteins, and identify segments (13-17aa in length) with a high degree of similarity to microbial sequences. Ideally, I would like to start with list of FASTA sequences, and have an easy way to generate an output of the corresponding high similarity segments of each protein.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there existing tools or software that I should be aware of that will make my life easier?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="727" LastEditorUserId="45" LastEditDate="2017-06-10T14:39:04.273" LastActivityDate="2017-06-10T14:39:04.273" Title="Detecting portions of human proteins with high degree of microbial similarity" Tags="|fasta|sequence-homology|" AnswerCount="1" CommentCount="8" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="554" PostTypeId="2" ParentId="531" CreationDate="2017-06-07T22:34:32.847" Score="2" Body="&lt;p&gt;Just replace the &lt;code&gt;-a&lt;/code&gt; option with &lt;code&gt;-aa&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools depth -b $bedfile -aa $inputfile&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I see that you're using the GRCh38 human reference genome build, which includes alternate scaffolds that represent a wider variety of genomic variation in the human genome. This was quite a substantial change from &lt;a href=&quot;https://www.quora.com/What-are-the-differences-between-GRCh38-and-GRCh37&quot; rel=&quot;nofollow noreferrer&quot;&gt;previous reference genome builds&lt;/a&gt;, and many bioinformatics tools still haven't been updated to accept the GRCh38 reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The discrepancy in entry size is probably because there are alternate contigs that are uncovered by the reads that have been mapped. This is expected, but can cause a bit of confusion.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-07T22:34:32.847" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="555" PostTypeId="2" ParentId="553" CreationDate="2017-06-07T22:51:54.867" Score="4" Body="&lt;p&gt;Sounds like precisely the job BLAST was developed for. Now, which flavor will depend on what you want to do and what data you have available. Some options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;PSI-BLAST&lt;/strong&gt;: this is usually the best choice if you are trying to find protein homologs. It works by building a hidden markov model describing your query sequence and using that model to query a database of proteins. The advantage is that it is run in multiple iterations, giving you the chance to add or remove results (so you add the ones that are true positives and remove false ones), eventually building a pretty good model of your protein. This is far moer powerful than a simple homology-based approach since proteins work via protein domains and simple homology is not as important as specific conserved functional residues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this, go to the NCBI &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PROGRAM=blastp&amp;amp;PAGE_TYPE=BlastSearch&amp;amp;LINK_LOC=blasthome&quot; rel=&quot;nofollow noreferrer&quot;&gt;protein blast page&lt;/a&gt; and select PSI-BLAST:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/q4yAK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/q4yAK.png&quot; alt=&quot;psi-blast option at ncbi&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;BLASTp&lt;/strong&gt;: Simple protein-protein blast. It will identify homologous proteins based on sequence similarity. Whether or not that also implies functional homology is not that simple and will depend on each case you investigate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As above, go to the NCBI &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PROGRAM=blastp&amp;amp;PAGE_TYPE=BlastSearch&amp;amp;LINK_LOC=blasthome&quot; rel=&quot;nofollow noreferrer&quot;&gt;protein blast page&lt;/a&gt;, but this time use the defaults.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;tBLASTn&lt;/strong&gt;: this is a tool that takes protein sequences as input and compares them to a database of DNA which is dynamically translated in all 6 possible reading frames. Very good for finding homologous sequences when you don't have well annotated protein information for the target species. It has the benefit of being more sensitive and able to find more distant homologies than basic nucleotide BLASTn and the go-to approach when your target species is distant and not well annotated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PROGRAM=tblastn&amp;amp;PAGE_TYPE=BlastSearch&amp;amp;BLAST_SPEC=&amp;amp;LINK_LOC=blasttab&amp;amp;LAST_PAGE=blastp&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI's tBLASTn page&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;All of these can be run online through the &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI's BLAST page&lt;/a&gt;. If you want to investigate hundreds of proteins, I suggest you install blast locally. You can then either download the relevant target sequences from NCBI and rebuild the blast database locally (if so, I suggest you ask a new question about how to do that) or, use NCBI's remote blast client which lets you use a locally stored query file and will run blast on the NCBI's servers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, these programs will return what are known as High Scoring Pairs (HSPs), the regions of your query sequence(s) that align well to the target. There are various options you can play with to improve the sensitivity or the specificity but a discussion of those would require far more details about what you're doing and would be best in a new question as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have your HSPs, you can relatively easily parse them to select regions with a given range of sequence similarity values and of a specific length. Once again, that would be better discussed in a separate question once you have your results and can show an example. &lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-06-07T23:04:27.220" LastActivityDate="2017-06-07T23:04:27.220" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="556" PostTypeId="2" ParentId="528" CreationDate="2017-06-07T23:31:46.327" Score="1" Body="&lt;p&gt;If you know how to use python:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1 - Download the ModeRNA module from &lt;a href=&quot;https://github.com/lenarother/moderna&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and install&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2 - from a python IDLE execute:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from moderna import *&#xA;m = load_model('file.pdb', 'A') #A is the chain&#xA;seq = m.get_sequence()&#xA;sec = m.get_secstruc()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the variable &lt;code&gt;sec&lt;/code&gt; store the secondary structure in dot-bracket notation.&lt;/p&gt;&#xA;" OwnerUserId="729" LastActivityDate="2017-06-07T23:31:46.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="557" PostTypeId="2" ParentId="318" CreationDate="2017-06-08T00:09:09.243" Score="0" Body="&lt;p&gt;For mouse, I still see people using mm9/NCBI37 in high profile publications even though mm10/GRCm38 was released more than 5 years ago (2011). I personally don't think that's a great idea, but it's certainly valid according to the peer reviewers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It also depends on your application. If you are working with coding regions (likely well known for a long time) or extracting genome-wide stats (enrichment at TSS, for example), the differences should be negligible.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2017-06-08T00:09:09.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="558" PostTypeId="1" CreationDate="2017-06-08T01:03:40.633" Score="2" ViewCount="940" Body="&lt;p&gt;It's supposed that quantum computers could help to find the native state of all human DNA proteins.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know how to formulate the protein folding problem into a decision problem solvable either by &lt;a href=&quot;https://en.wikipedia.org/wiki/Quantum_annealing&quot; rel=&quot;nofollow noreferrer&quot;&gt;quantum annealers&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Quantum_Turing_machine&quot; rel=&quot;nofollow noreferrer&quot;&gt;universal quantum computers&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would also be interesting to know, how to model a decision problem for classical computers, like &lt;a href=&quot;http://folding.stanford.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Folding@Home&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="732" LastEditorUserId="96" LastEditDate="2017-06-08T03:39:52.487" LastActivityDate="2017-06-08T17:54:28.040" Title="Protein folding problem model for quantum computers?" Tags="|proteins|computation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="559" PostTypeId="1" AcceptedAnswerId="560" CreationDate="2017-06-08T02:25:46.173" Score="11" ViewCount="957" Body="&lt;p&gt;I have a gene expression quantification file from &lt;a href=&quot;https://portal.gdc.cancer.gov/files/92e73892-811b-4edd-b3db-d452bc5d28e0&quot; rel=&quot;noreferrer&quot;&gt;TCGA&lt;/a&gt; that contains the following lines:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ENSG00000242268.2   591.041000514&#xA;ENSG00000270112.3   0.0&#xA;ENSG00000167578.15  62780.6543066&#xA;ENSG00000273842.1   0.0&#xA;ENSG00000078237.5   36230.832883&#xA;ENSG00000146083.10  189653.152706&#xA;ENSG00000225275.4   0.0&#xA;ENSG00000158486.12  420.761140072&#xA;ENSG00000198242.12  2914738.3675&#xA;ENSG00000259883.1   1632.83700531&#xA;ENSG00000231981.3   0.0&#xA;ENSG00000269475.2   0.0&#xA;ENSG00000201788.1   0.0&#xA;ENSG00000134108.11  925529.547944&#xA;ENSG00000263089.1   2646.63769677&#xA;ENSG00000172137.17  23162.6989867&#xA;ENSG00000167700.7   291192.25157&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the &lt;code&gt;.number&lt;/code&gt; that are added to the Gene e.g. the &lt;code&gt;.2&lt;/code&gt; in &lt;code&gt;ENSG00000242268.2&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Why isn't the quantified value an integer; what does &lt;code&gt;591.041000514&lt;/code&gt; mean?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In case someone one wants more info &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene/?term=100507661&quot; rel=&quot;noreferrer&quot;&gt;about the gene LINC02082&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="292" LastEditDate="2017-06-08T10:28:06.803" LastActivityDate="2017-06-08T10:28:06.803" Title="How to read and interpret a gene expression quantification file?" Tags="|gene|normalization|identifiers|quantification|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="560" PostTypeId="2" ParentId="559" CreationDate="2017-06-08T03:05:39.547" Score="9" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The first column contains Ensembl gene identifiers, and the suffix is a version number that can be used to track changes to the gene annotations over time. From the &lt;a href=&quot;http://www.ensembl.org/info/genome/stable_ids/index.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;Ensembl Stable IDs&lt;/strong&gt; documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Ensembl annotation uses a system of stable IDs that have prefixes based on the species scientific name plus the feature type, followed by a series of digits and a version e.g. ENSG00000139618.1. The version may be omitted.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Following the &lt;a href=&quot;https://portal.gdc.cancer.gov/files/92e73892-811b-4edd-b3db-d452bc5d28e0&quot; rel=&quot;noreferrer&quot;&gt;first link&lt;/a&gt; you provided leads to a page with details for the file &lt;code&gt;2edcaaa7-63b4-40b4-abbe-5d7a84012e60.FPKM-UQ.txt.gz&lt;/code&gt;. The first thing that caught my eye about this filename was &lt;code&gt;FPKM&lt;/code&gt;, or &quot;fragments per kilobase of exon per million reads&quot;, which is a commonly used unit of RNA expression. Since these are not raw read counts, there's no expectation that these values should be integers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best explanation I've seen of FPKM comes from &lt;a href=&quot;https://haroldpimentel.wordpress.com/2014/05/08/what-the-fpkm-a-review-rna-seq-expression-units/&quot; rel=&quot;noreferrer&quot;&gt;a blog post&lt;/a&gt; written by Harold Pimentel of &lt;a href=&quot;http://pachterlab.github.io/kallisto/&quot; rel=&quot;noreferrer&quot;&gt;kallisto&lt;/a&gt; and &lt;a href=&quot;https://pachterlab.github.io/sleuth/&quot; rel=&quot;noreferrer&quot;&gt;sleuth&lt;/a&gt; fame. From the blog post:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The interpretation of FPKM is as follows: if you were to sequence this pool of RNA again, you expect to see &lt;code&gt;FPKM_i&lt;/code&gt; fragments for each thousand bases in the feature for every &lt;code&gt;N/10^6&lt;/code&gt; fragments you’ve sequenced. It’s basically just the rate of fragments per base multiplied by a big number (proportional to the number of fragments you sequenced) to make it more convenient.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;More generally though, even when FPKM is not the unit of expression abundance used, most quantification methods and associated units of expression will not produce integer estimates.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2017-06-08T06:45:32.780" LastActivityDate="2017-06-08T06:45:32.780" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="561" PostTypeId="1" AcceptedAnswerId="562" CreationDate="2017-06-08T04:54:36.177" Score="9" ViewCount="11290" Body="&lt;p&gt;I have some FASTQ sequence files and a FASTA file for some regions I'm interested in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Build an index for the FASTA file&lt;/li&gt;&#xA;&lt;li&gt;Use the index to count number of k-mers occurred in my sequence files&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I know how to do this in many k-mer counting tools, such as Kallisto, Jellyfish, DSK, Salmon. However, I'm more interested in Python (khmer is a Python package, all the other k-mer counting tools are C++).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, I can't figure out how to do that in khmer from the documentation:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;http://khmer.readthedocs.io/en/v2.1.1/introduction.html&quot; rel=&quot;noreferrer&quot;&gt;http://khmer.readthedocs.io/en/v2.1.1/introduction.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It talks about RNA-Seq, metagenomics, de Bruijn graph etc... All I need are just counts. Khmer is a popular k-mer tool, so it should be able to do it, but how?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q:&lt;/strong&gt; How to build an index and count k-mers using khmer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't need number of unique k-mers, I'm looking for a solution like Jellyfish where we build an index and count k-mers to the index.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MY_SEQ_1  100 counts&#xA;MY_SEQ_2  134 counts&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="174" LastEditorUserId="73" LastEditDate="2018-03-11T01:06:50.860" LastActivityDate="2018-03-11T01:06:50.860" Title="How to use Python to count k-mers?" Tags="|genome|k-mer|metagenome|khmer|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="562" PostTypeId="2" ParentId="561" CreationDate="2017-06-08T05:03:42.780" Score="5" Body="&lt;p&gt;If you want to count the number of unique k-mers that occur in your data set, you should use the &lt;a href=&quot;http://khmer.readthedocs.io/en/v2.1.1/user/scripts.html#unique-kmers-py&quot; rel=&quot;noreferrer&quot;&gt;unique-kmers.py&lt;/a&gt; script, which implements a HyperLogLog-based cardinality estimator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the number of unique k-mers is not what you're after, please clarify.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;khmer uses probabilistic data structures internally, which store k-mers as hashed values that cannot be un-hashed back into k-mer sequences uniquely. Querying a k-mer's abundance requires you to know which k-mer(s) you're looking for, which in this case would require a second pass over the reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The closest thing to what you're asking provided in khmer's command-line scripts is &lt;a href=&quot;http://khmer.readthedocs.io/en/v2.1.1/user/scripts.html#abundance-dist-py&quot; rel=&quot;noreferrer&quot;&gt;abundance-dist.py&lt;/a&gt; (or the alternative abundance-dist-single.py), which will produce a k-mer abundance histogram but not per-kmer abundance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://khmer.readthedocs.io/en/v2.1.1/user/scripts.html#load-into-counting-py&quot; rel=&quot;noreferrer&quot;&gt;load-into-counting.py&lt;/a&gt; scripts computes the k-mer abundances and stores them in a probabilistic data structure, which is written to disk and can subsequently be re-loaded into memory quickly. Many of the scripts in khmer expect the reads to have been pre-processed by &lt;code&gt;load-into-counting.py&lt;/code&gt;, while others will invoke the counting routines directly themselve. It's also fairly easy to do this with the Python API. In every case, the accuracy of the k-mer counts is dependant on &lt;a href=&quot;http://khmer.readthedocs.io/en/v2.1.1/user/choosing-table-sizes.html&quot; rel=&quot;noreferrer&quot;&gt;memory considerations&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; import khmer&#xA;&amp;gt;&amp;gt;&amp;gt; counts = khmer.Counttable(31, 1e7, 4)&#xA;&amp;gt;&amp;gt;&amp;gt; counts.consume_seqfile('reads.fq.gz')&#xA;(25000, 1150000)&#xA;&amp;gt;&amp;gt;&amp;gt; counts.get('TGACTTTCTTCGCTTCCTGACGGCTTATGCC')&#xA;117&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To check the false positive rate:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; fpr = khmer.calc_expected_collisions(counts)&#xA;&amp;gt;&amp;gt;&amp;gt; fpr&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you &lt;em&gt;must&lt;/em&gt; have the counts of each k-mer, it's not much more work...although reporting each k-mer's abundance only once would consume a lot of memory with a naive approach.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; outfile = open('outfile.txt', 'w')&#xA;&amp;gt;&amp;gt;&amp;gt; seenkmers = set()  # Consumes a lot of memory for large input!!!&#xA;&amp;gt;&amp;gt;&amp;gt; for read in khmer.ReadParser('reads.fq.gz):&#xA;...     for kmer in counts.get_kmers(read.sequence):&#xA;...         if kmer not in seenkmers:&#xA;...             print(kmer, counttable.get(kmer), file=outfile)&#xA;...             seenkmers.add(kmer)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2017-06-08T06:04:19.577" LastActivityDate="2017-06-08T06:04:19.577" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="563" PostTypeId="2" ParentId="527" CreationDate="2017-06-08T05:48:07.953" Score="3" Body="&lt;p&gt;I mostly use TargetScan predictions. But here are few curated databases:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://mirgate.bioinfo.cnio.es/miRGate/&quot; rel=&quot;nofollow noreferrer&quot;&gt;miRGate&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;miRGate is a curated database of human, mouse and rat miRNAs/mRNAs targets. It is designed to analyze miRNA and gene isoforms lists under a common and consistent space of annotations. Including all existing 3 UTR and the entirely known miRNAs. All Havana biotypes and ENCODE principal isoforms for the three organisms are also included. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/25858286&quot; rel=&quot;nofollow noreferrer&quot;&gt;Publication&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.mir2disease.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;miR2Disease&lt;/a&gt; : &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;a manually curated database, aims at providing a comprehensive resource of miRNA deregulation in various human diseases. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2686559/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Publication&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://pcsb.ahau.edu.cn:8080/PASmiR/f&quot; rel=&quot;nofollow noreferrer&quot;&gt;PASmiR&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;PASmiR, a literature-curated and web-accessible database, was thus developed to provide a solid platform for the collection, standardization, and searching of these miRNA-stress regulation data in plants. &lt;a href=&quot;https://bmcplantbiol.biomedcentral.com/articles/10.1186/1471-2229-13-33&quot; rel=&quot;nofollow noreferrer&quot;&gt;Publication&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="161" LastActivityDate="2017-06-08T05:48:07.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="564" PostTypeId="2" ParentId="318" CreationDate="2017-06-08T06:38:40.213" Score="2" Body="&lt;p&gt;You could use liftOver which isn't always great.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whenever I encounter this (especially NGS data readily available on the SRA), I often just get the raw files (e.g. fastqs) and re-align/re-map.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case (arrays) it may be a bit tough. Not impossible though, as I recently took some old yeast DNA/RNA microarray data and updated it to the newest genome. Just requires the right data (like DNA for normalization) and a good understanding of the entire process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A last resort/alternative is to align your &lt;strong&gt;new&lt;/strong&gt; data to the old genome to be able to make comparisons. This isn't ideal but works in cases where upgrading one source isn't possible or is a HUGE amount of time/effort. I've done this for a few fly experiments where all the available/previous data was done in dm3. All the old genomes can generally be found on &lt;a href=&quot;http://archive.ensembl.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://archive.ensembl.org&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2017-06-08T06:38:40.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="565" PostTypeId="1" CreationDate="2017-06-08T07:06:29.710" Score="10" ViewCount="270" Body="&lt;p&gt;Bashing file formats is a favorite pastime in bioinformatics, and annotation file formats such as GFF and BED seem to get special attention. A lot of this frustration stems from community's shockingly inconsistent adherence to specifications and conventions, but there are also some (dare I say objectively) problematic design choices in each of these formats.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;GFF (and its more common derivatives GTF and GFF3) use 1-based closed interval notation, which optimizes for human comprehension but is far inferior to 0-based half-open interval notation (such as used by BED) for computations involving interval arithmetic.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Although BED and GTF were designed for very specific use cases (visualization and gene prediction, respectively), they have been used and abused in a much wider set of contexts. For example, the BED fields related to the &lt;strong&gt;thick part&lt;/strong&gt; are irrelevant if you're not plotting them in a genome browser.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;BED supports a single level of feature decomposition (a feature can be broken up into blocks). GTF supports two levels (exons grouped by transcript_id, transcripts grouped by gene_id). In contrast, GFF3 supports an arbitrary number of levels, and uses parent/child relationships defined by &lt;code&gt;ID&lt;/code&gt; and &lt;code&gt;Parent&lt;/code&gt; attributes to declare a directed acyclic graph of features.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Data that does not fit into mandatory pre-defined fields must be relegated to optional fields or free-form attribute key/value pairs. While this flexibility is powerful, a common complaint is that &quot;all the action&quot; happens in these optional/free-form fields.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is a dearth of validation tools, and those that do exist focus primarily on validating syntax and not semantics. To use an aging analogy, it's one thing to say an XML file is valid, but it's completely different to validate it against a schema. There are essentially no widely used tools that do the latter for annotation files.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If we were tasked with creating a new annotation format, and if we were guaranteed the resources needed to develop it, and interest and wide adoption from the wider community (one can dream!), what design criteria should be considered in the development of this new format? &lt;strong&gt;What, if anything, makes an objectively good annotation data format?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="73" LastEditDate="2017-06-08T14:46:51.810" LastActivityDate="2017-06-09T01:59:00.970" Title="Annotation format design" Tags="|file-formats|bed|gff3|sequence-annotation|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="566" PostTypeId="2" ParentId="565" CreationDate="2017-06-08T07:18:13.830" Score="4" Body="&lt;p&gt;Presuming we consider &quot;human readable&quot;, &quot;easily parsable&quot;, and &quot;quickly queryable&quot; to be objectively good qualities (and if not, I worry for the future):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Text-based: It's absurdly common to want to use &lt;code&gt;grep&lt;/code&gt; or &lt;code&gt;awk&lt;/code&gt; on annotations. Sure, one could make variants of these that are binary-format aware, but why reinvent the wheel. Of course text files don't innately allow region-based queries of their contents, so on to point 2...&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This should further explicitly be line-based. Fields would need to be tab-separated (ideally it'd use the ascii record separator character instead, but I fear that ship has long ago sailed).&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;A strictly defined line order: This is one of my personal pet-peeves about BED/GFF/GTF files. If you're going to make a text-based annotation format, everyone would end up ahead if said format made explicit that it should be sorted. This then allows things like tabix to be used so the &quot;query a region&quot; problem is then solved. But I would go further than that. The issue with things like GFF is that there are multiple inter-dependent lines and there's no strict rule about whether a parent line absolutely must come before a child line. This just makes implementing things a nightmare and more often than not a randomly ordered file will just break tools. Since GTF or GFF-style interdependent lines likely be how any annotation format works, this ordering between lines with the same start position should be made explicit in the format.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-06-08T08:50:10.537" LastActivityDate="2017-06-08T08:50:10.537" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="567" PostTypeId="1" AcceptedAnswerId="569" CreationDate="2017-06-08T07:21:27.027" Score="10" ViewCount="858" Body="&lt;p&gt;Assume I have found the top 0.01% most frequent genes from a &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/559/how-to-read-and-interpret-a-gene-expression-quantification-file&quot;&gt;gene expression file&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say, these are 10 genes and I want to study the &lt;a href=&quot;https://en.wikipedia.org/wiki/Protein%E2%80%93protein_interaction&quot; rel=&quot;nofollow noreferrer&quot;&gt;protein protein interactions&lt;/a&gt;, the protein network and pathway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought to use &lt;a href=&quot;https://string-db.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;string-db&lt;/a&gt; or &lt;a href=&quot;http://interactome.dfci.harvard.edu&quot; rel=&quot;nofollow noreferrer&quot;&gt;interactome&lt;/a&gt;, but I am not sure what would be a plausible way to approach this problem and to build the protein network etc. Are there other more suitable databases?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I build a mathematical graph or network for these data?&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2017-06-10T06:04:33.900" LastActivityDate="2017-06-16T10:50:45.063" Title="How can I build a protein network pathway from a gene expression quantification file?" Tags="|proteins|networks|gene|pathway|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="568" PostTypeId="1" AcceptedAnswerId="571" CreationDate="2017-06-08T07:57:21.707" Score="7" ViewCount="1199" Body="&lt;p&gt;This question is based on &lt;a href=&quot;https://www.biostars.org/p/129057/&quot; rel=&quot;noreferrer&quot;&gt;a question&lt;/a&gt; on BioStars  posted &gt;2 years ago by user jack.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It describes a very frequent problem of generating GO annotations for non-model organisms. While it is based on some specific format and single application (Ontologizer), it would be useful to have a general description of the pathway to getting to a GAF file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, that the input format is lacking a bit of essential information, like how it was obtained. Therefore, it is har to assign evidence code. Therefore, lets assume that the assignments of GO terms were done automagically. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I want to do the Gene enrichment using Ontologizer without a&#xA;  predefined association file(it's not model organism). &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;I have parsed a file with two columns for that organism like this : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;geneA  GO:0006950,GO:0005737&#xA;geneB  GO:0016020,GO:0005524,GO:0006468,GO:0005737,GO:0004674,GO:0006914,GO:0016021,GO:0015031&#xA;geneC  GO:0003779,GO:0006941,GO:0005524,GO:0003774,GO:0005516,GO:0005737,GO:0005863&#xA;geneD  GO:0005634,GO:0003677,GO:0030154,GO:0006350,GO:0006355,GO:0007275,GO:0030528&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;I have downloaded the .ob file from Gene ontology file which contain&#xA;  this information (from &lt;a href=&quot;http://www.geneontology.org/doc/GO.terms_and_ids&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;) : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;!&#xA;! GO IDs (primary only) and name text strings&#xA;! GO:0000000 [tab] text string [tab] F|P|C&#xA;! where F = molecular function, P = biological process, C = cellular component&#xA;!&#xA;GO:0000001  mitochondrion inheritance   P&#xA;GO:0000002  mitochondrial genome maintenance    P&#xA;GO:0000003  reproduction    P&#xA;GO:0000005  ribosomal chaperone activity    F&#xA;GO:0000006  high affinity zinc uptake transmembrane transporter activity    F&#xA;GO:0000007  low-affinity zinc ion transmembrane transporter activity    F&#xA;GO:0000008  thioredoxin F&#xA;GO:0000009  alpha-1,6-mannosyltransferase activity  F&#xA;GO:0000010  trans-hexaprenyltranstransferase activity   F&#xA;GO:0000011  vacuole inheritance P&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;What I need as output is .gaf file in the following format (in the&#xA;  format of the files &lt;a href=&quot;http://geneontology.org/page/download-annotations&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;!gaf-version: 2.0&#xA;&#xA;!Project_name: Leishmania major GeneDB&#xA;&#xA;!URL: http://www.genedb.org/leish&#xA;&#xA;!Contact Email: mb4@sanger.ac.uk&#xA;&#xA; GeneDB_Lmajor    LmjF.36.4770    LmjF.36.4770        GO:0003723    PMID:22396527    ISO    GeneDB:Tb927.10.10130    F    mitochondrial RNA binding complex 1 subunit, putative    LmjF36.4770    gene    taxon:347515    20120910    GeneDB_Lmajor       &#xA; GeneDB_Lmajor    LmjF.36.4770    LmjF.36.4770        GO:0044429    PMID:20660476    ISS        C    mitochondrial RNA binding complex 1 subunit, putative    LmjF36.4770    gene    taxon:347515    20100803 GeneDB_Lmajor             GeneDB_Lmajor    LmjF.36.4770    LmjF.36.4770        GO:0016554    PMID:22396527    ISO    GeneDB:Tb927.10.10130    P    mitochondrial RNA binding complex 1 subunit, putative    LmjF36.4770    gene   taxon:347515    20120910    GeneDB_Lmajor       &#xA; GeneDB_Lmajor    LmjF.36.4770    LmjF.36.4770        GO:0048255    PMID:22396527    ISO    GeneDB:Tb927.10.10130    P    mitochondrial RNA binding complex 1 subunit, putative    LmjF36.4770    gene    taxon:347515    20120910    GeneDB_Lmajor  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;How to create your own GO association file (gaf)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="311" LastEditorUserId="298" LastEditDate="2017-06-08T11:20:04.643" LastActivityDate="2017-06-08T20:43:01.263" Title="How can I create my own GO association file (gaf)?" Tags="|sequence-annotation|gene-ontology|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="569" PostTypeId="2" ParentId="567" CreationDate="2017-06-08T08:24:57.997" Score="9" Body="&lt;p&gt;There are multiple ways to do this, and multiple protein interaction databases besides the ones you mentioned, such as &lt;a href=&quot;https://thebiogrid.org/&quot; rel=&quot;noreferrer&quot;&gt;BioGRID&lt;/a&gt; or &lt;a href=&quot;http://www.ebi.ac.uk/intact/&quot; rel=&quot;noreferrer&quot;&gt;IntAct&lt;/a&gt;. Interaction databases are different in how interactions are defined, sometimes it can be experimental evidence of interaction, sometimes coexpression, orthology-based predictions, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no single solution to your problem. For String-DB you can use their R package &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/STRINGdb.html&quot; rel=&quot;noreferrer&quot;&gt;STRINGdb&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# string-db&#xA;library(STRINGdb)&#xA;&#xA;# retrieve full graph for H. sapiens&#xA;string_db &amp;lt;- STRINGdb$new(version=&quot;10&quot;, species=9606,&#xA;                          score_threshold=400, input_directory=&quot;&quot; )&#xA;&#xA;# define genes of interest&#xA;genes.of.interest &amp;lt;- ...&#xA;&#xA;# get their neighbors&#xA;neighbors &amp;lt;- string_db$get_neighbors(genes.of.interest)&#xA;&#xA;# get the subgraph of interest (your genes + their neighbors)&#xA;my.subgraph &amp;lt;- string_db$get_subnetwork(c(genes.of.interest, neighbors))&#xA;&#xA;# look how many genes and interactions you have&#xA;my.subgraph&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you prefer GUI, you might consider using &lt;a href=&quot;http://www.cytoscape.org/&quot; rel=&quot;noreferrer&quot;&gt;Cytoscape&lt;/a&gt;. They have &lt;a href=&quot;http://apps.cytoscape.org/apps/with_tag/interactiondatabase&quot; rel=&quot;noreferrer&quot;&gt;multiple addons for interaction databases&lt;/a&gt;, e.g. &lt;a href=&quot;http://apps.cytoscape.org/apps/stringapp&quot; rel=&quot;noreferrer&quot;&gt;for STRINGdb&lt;/a&gt;. There you can just provide your gene list and get a network in a few clicks.&lt;/p&gt;&#xA;" OwnerUserId="191" LastEditorUserId="191" LastEditDate="2017-06-08T08:49:31.847" LastActivityDate="2017-06-08T08:49:31.847" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="570" PostTypeId="1" AcceptedAnswerId="574" CreationDate="2017-06-08T09:00:53.407" Score="13" ViewCount="509" Body="&lt;p&gt;I have many alignments from Rfam Database, and I would like to edit them. &#xA;I saw that many tools are used for Protein sequence alignments, but there is something specific to edit RNA alignments ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g. Stockholm Alignment of Pistol (just few entries). &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FP929053.1/1669026-1668956   AGUGGUCACAGCCACUAUAAACA-GGGCUU-UAAGCUGUG-AGCGUUGACCGUC----------ACAA-----CGGCGGUCAGGUAGUC&#xA;AFOX01000025.1/1981-1912     ACUCGUCUGAGCGAGUAUAAACA-GGUCAU-UAAGCUCAG-AGCGUUCACCGGG----------AUCA------UUCGGUGAGGUUGGC&#xA;HE577054.1/3246821-3246752   ACUCGUCUGAGCGAGUAUAAACA-GGUCAU-UAAGCUCAG-AGCGUUCACCGGG----------AUCA------UGCGGUGAGGUUGGC&#xA;CP000154.1/3364237-3364168   GUUCGUCUGAGCGAACGCAAACA-GGCCAU-UAAGCUCAG-AGCGUUCACUGGA----------UUCG------UCCAGUGAGAUUGGC`&#xA;`#=GC SS_cons                 &amp;lt;&amp;lt;&amp;lt;&amp;lt;__AAAAA_&amp;gt;&amp;gt;&amp;gt;&amp;gt;-------..&amp;lt;&amp;lt;&amp;lt;&amp;lt;-.----aaaaa.----&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;..........____....._&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;--&amp;gt;&amp;gt;&amp;gt;&amp;gt;`&#xA;`#=GC RF                      acUCGUCuggGCGAguAUAAAuA..cgCaU.UAgGCccaG.AGCGUcccggcgg..........uUau.....uccgccgggGGUuGcg&#xA;//&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="682" LastEditorUserId="640" LastEditDate="2017-06-08T20:08:04.130" LastActivityDate="2018-12-31T05:43:06.480" Title="What to use to edit RNA alignments?" Tags="|rna|rna-alignment|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="571" PostTypeId="2" ParentId="568" CreationDate="2017-06-08T09:08:04.750" Score="8" Body="&lt;p&gt;Here's a Perl script that can do this:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env perl &#xA;use strict;&#xA;use warnings;&#xA;&#xA;## Change this to whatever taxon you are working with&#xA;my $taxon = 'taxon:1000';&#xA;chomp(my $date = `date +%Y%M%d`);&#xA;&#xA;my (%aspect, %gos);&#xA;## Read the GO.terms_and_ids file to get the aspect (sub ontology)&#xA;## of each GO term. &#xA;open(my $fh, $ARGV[0]) or die &quot;Need a GO.terms_and_ids file as 1st arg: $!\n&quot;;&#xA;while (&amp;lt;$fh&amp;gt;) {&#xA;    next if /^!/;&#xA;    chomp;&#xA;    my @fields = split(/\t/);&#xA;    ## $aspect{GO:0000001} = 'P'&#xA;    $aspect{$fields[0]} = $fields[2];&#xA;}&#xA;close($fh);&#xA;&#xA;## Read the list of gene annotations&#xA;open($fh, $ARGV[1]) or die &quot;Need a list of gene annotattions as 2nd arg: $!\n&quot;;&#xA;while (&amp;lt;$fh&amp;gt;) {&#xA;    chomp;&#xA;    my ($gene, @terms) = split(/[\s,]+/);&#xA;    ## $gos{geneA} = (go1, go2 ... goN)&#xA;    $gos{$gene} = [ @terms ];&#xA;}&#xA;close($fh);&#xA;&#xA;foreach my $gene (keys(%gos)) {&#xA;    foreach my $term (@{$gos{$gene}}) {&#xA;        ## Warn and skip if there is no aspect for this term&#xA;        if (!$aspect{$term}) {&#xA;            print STDERR &quot;Unknown GO term ($term) for gene $gene\n&quot;;&#xA;            next;&#xA;        }&#xA;        ## Build a pseudo GAF line &#xA;        my @out = ('DB', $gene, $gene, ' ', $term, 'PMID:foo', 'TAS', ' ', $aspect{$term},&#xA;                             $gene, ' ', 'protein', $taxon, $date, 'DB', ' ', ' ');&#xA;        print join(&quot;\t&quot;, @out). &quot;\n&quot;;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Make it executable and run it with the &lt;code&gt;GO.terms_and_ids&lt;/code&gt; file as the 1st argument and the list of gene annotations as the second. Using the current &lt;code&gt;GO.terms_and_ids&lt;/code&gt; and the example annotations in the question, I get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ foo.pl GO.terms_and_ids file.gos &#xA;DB  geneD   geneD       GO:0005634  PMID:foo    TAS     C   geneD       protein taxon:1000  20170308    DB       &#xA;DB  geneD   geneD       GO:0003677  PMID:foo    TAS     F   geneD       protein taxon:1000  20170308    DB       &#xA;DB  geneD   geneD       GO:0030154  PMID:foo    TAS     P   geneD       protein taxon:1000  20170308    DB       &#xA;Unknown GO term (GO:0006350) for gene geneD&#xA;DB  geneD   geneD       GO:0006355  PMID:foo    TAS     P   geneD       protein taxon:1000  20170308    DB       &#xA;DB  geneD   geneD       GO:0007275  PMID:foo    TAS     P   geneD       protein taxon:1000  20170308    DB       &#xA;DB  geneD   geneD       GO:0030528  PMID:foo    TAS     F   geneD       protein taxon:1000  20170308    DB       &#xA;DB  geneB   geneB       GO:0016020  PMID:foo    TAS     C   geneB       protein taxon:1000  20170308    DB       &#xA;DB  geneB   geneB       GO:0005524  PMID:foo    TAS     F   geneB       protein taxon:1000  20170308    DB       &#xA;DB  geneB   geneB       GO:0006468  PMID:foo    TAS     P   geneB       protein taxon:1000  20170308    DB       &#xA;DB  geneB   geneB       GO:0005737  PMID:foo    TAS     C   geneB       protein taxon:1000  20170308    DB       &#xA;DB  geneB   geneB       GO:0004674  PMID:foo    TAS     F   geneB       protein taxon:1000  20170308    DB       &#xA;DB  geneB   geneB       GO:0006914  PMID:foo    TAS     P   geneB       protein taxon:1000  20170308    DB       &#xA;DB  geneB   geneB       GO:0016021  PMID:foo    TAS     C   geneB       protein taxon:1000  20170308    DB       &#xA;DB  geneB   geneB       GO:0015031  PMID:foo    TAS     P   geneB       protein taxon:1000  20170308    DB       &#xA;DB  geneA   geneA       GO:0006950  PMID:foo    TAS     P   geneA       protein taxon:1000  20170308    DB       &#xA;DB  geneA   geneA       GO:0005737  PMID:foo    TAS     C   geneA       protein taxon:1000  20170308    DB       &#xA;DB  geneC   geneC       GO:0003779  PMID:foo    TAS     F   geneC       protein taxon:1000  20170308    DB       &#xA;DB  geneC   geneC       GO:0006941  PMID:foo    TAS     P   geneC       protein taxon:1000  20170308    DB       &#xA;DB  geneC   geneC       GO:0005524  PMID:foo    TAS     F   geneC       protein taxon:1000  20170308    DB       &#xA;DB  geneC   geneC       GO:0003774  PMID:foo    TAS     F   geneC       protein taxon:1000  20170308    DB       &#xA;DB  geneC   geneC       GO:0005516  PMID:foo    TAS     F   geneC       protein taxon:1000  20170308    DB       &#xA;DB  geneC   geneC       GO:0005737  PMID:foo    TAS     C   geneC       protein taxon:1000  20170308    DB       &#xA;DB  geneC   geneC       GO:0005863  PMID:foo    TAS     C   geneC       protein taxon:1000  20170308    DB       &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that this is very much a pseudo-GAF file since most of the fields apart from the gene name, GO term and sub-ontology are fake. It should still work for what you need, however.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-06-08T20:43:01.263" LastActivityDate="2017-06-08T20:43:01.263" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="572" PostTypeId="2" ParentId="527" CreationDate="2017-06-08T09:18:07.773" Score="3" Body="&lt;p&gt;For miRNA precursor predictions in species in Ensembl but not in miRBase, you can have a go to MapMi (&lt;a href=&quot;http://www.ebi.ac.uk/enright-srv/MapMi/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ebi.ac.uk/enright-srv/MapMi/&lt;/a&gt;) . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can easily run it locally for other species or if you have smallRNA sequencing done on something new. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me know if you run into difficulties or have ideas to improve it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ensembl itself has its own miRNA predictions. &lt;/p&gt;&#xA;" OwnerUserId="693" LastEditorUserId="77" LastEditDate="2017-06-08T10:23:38.590" LastActivityDate="2017-06-08T10:23:38.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="573" PostTypeId="2" ParentId="570" CreationDate="2017-06-08T09:22:17.537" Score="7" Body="&lt;p&gt;There's nothing really special about RNA alignments, you can use any alignment editor, including whichever one you use for protein. That said, a classic and very useful tool for this sort of thing is &lt;a href=&quot;http://www.jalview.org/&quot; rel=&quot;noreferrer&quot;&gt;JalView&lt;/a&gt;. It can be installed locally or run as a Java webapp from your browser.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Jalview has built in DNA, RNA and protein sequence and structure visualisation and analysis capabilities. It uses Jmol to view 3D structures, and VARNA to display RNA secondary structure. The Jalview Desktop can also connect with databases and analysis services, and provides a graphical interface to the alignment and analysis services provided by the JavA Bioinformatics Analysis Web Services framework.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/I76Yq.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/I76Yq.png&quot; alt=&quot;jalview image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-06-08T09:22:17.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="574" PostTypeId="2" ParentId="570" CreationDate="2017-06-08T09:23:18.300" Score="11" Body="&lt;p&gt;I would suggest use RALEE—RNALignment Editor in Emacs. It can get for you the consensus secondary structure, you can move left/right sequences and their secondary structures (you can't do it in JalView!), and more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's an Emacs mode, so could be a bit hard to start off, but just try, you don't have to use all Emacs features to edit your alignments! &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The RALEE (RNA ALignment Editor in Emacs) tool provides a simple&#xA;  environment for RNA multiple sequence alignment editing, including&#xA;  structure-specific colour schemes, utilizing helper applications for&#xA;  structure prediction and many more conventional editing functions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1093/bioinformatics/bth489&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sam Griffiths-Jones Bioinformatics (2005) 21 (2): 257-259.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XOWVV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XOWVV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UPcwI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UPcwI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Fig. You can move left/right sequences and their secondary structures (you can't do it in JalView!)&lt;/p&gt;&#xA;" OwnerUserId="640" LastEditorUserId="964" LastEditDate="2018-12-31T05:43:06.480" LastActivityDate="2018-12-31T05:43:06.480" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="575" PostTypeId="5" CreationDate="2017-06-08T09:35:17.007" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-06-08T09:35:17.007" LastActivityDate="2017-06-08T09:35:17.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="576" PostTypeId="4" CreationDate="2017-06-08T09:35:17.007" Score="0" Body="When dealing with associating sequencing reads to a matching position in a set of reference sequences (typically a genome)." OwnerUserId="292" LastEditorUserId="292" LastEditDate="2017-06-23T15:05:12.343" LastActivityDate="2017-06-23T15:05:12.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="577" PostTypeId="2" ParentId="567" CreationDate="2017-06-08T10:31:33.980" Score="6" Body="&lt;p&gt;Many interaction databases now work with &lt;a href=&quot;http://psidev.sourceforge.net/molecular_interactions/xml/doc/user/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PSI format&lt;/a&gt; files. Most of the main databases can do this and the EBI has set up &lt;a href=&quot;http://www.ebi.ac.uk/Tools/webservices/psicquic/view/main.xhtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;PSICQUIC View&lt;/a&gt;, a very useful page where you can query multiple databases at once. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that it is very important to limit the results according to the detection method. There is a lot of noise in protein interaction databases. Depending on what you want to do you could limit to only experimentally verified interactions or to only direct, binary interactions &#xA;(so exclude the results of, for example, ChIP analyses which can also find complexes) etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, here's a simple example script that will query the APID database using its PSICQUIC service:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/perl&#xA;use strict;&#xA;use warnings;&#xA;use LWP::Simple;&#xA;&#xA;my @proteins;&#xA;## Read a list of target proteins, one per line (this expects UniProt names)&#xA;open(my $fh, &quot;$ARGV[0]&quot;) or die &quot;Need a list of proteins as the 1st argument: $!\n&quot;;&#xA;while (&amp;lt;$fh&amp;gt;) {&#xA;    chomp;&#xA;    push @proteins, $_;&#xA;}&#xA;close($fh);&#xA;## Get human interactions only&#xA;my $species=&quot;9606&quot;;&#xA;&#xA;## Get the interactions for each target protein&#xA;foreach my $protein (@proteins) {&#xA;    my $queryUrl= &quot;http://cicblade.dep.usal.es/psicquic-ws/webservices/current/search/query/$protein&quot;;&#xA;    $queryUrl  .= &quot;?taxidA:$species%20ANDtaxidB:$species&quot;;&#xA;&#xA;    my $tries=1;&#xA;    my $content = get $queryUrl;&#xA;    while ($tries&amp;lt;=10) {&#xA;        if (defined($content)) {&#xA;            $tries=11;&#xA;        } else {&#xA;            print STDERR &quot;Could not retrieve $queryUrl, retrying($tries)...\n&quot;;&#xA;            $content = get $queryUrl;&#xA;        }&#xA;        $tries++;&#xA;    }&#xA;&#xA;    # Now list all interactions&#xA;    my @lines = split(/\n/, $content);&#xA;    my $LINES= @lines;&#xA;    my $count = 0;&#xA;    for my $line (@lines) {&#xA;        $count++;&#xA;        my @flds = split(/\t/, $line); # split tab delimited lines      &#xA;        # split fields of a PSIMITAB 2.5 line&#xA;        my ($idA, $idB, $altIdA, $altIdB, $aliasA, $aliasB, $detMethod, $author, $pub, $orgA, $orgB, $intType, $sourceDb, $intID, $conf) = @flds;&#xA;        ## Here you can add logic to limit the interactions by specific detection method codes ($detMethod)&#xA;        ## or database of origin etc. &#xA;&#xA;        ## Print&#xA;        print &quot;$line\n&quot;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tested it by using a file with the UniProt ID for human TP53 (P04637) and it returns a list of 3988 interactions (showing 10 randomly selected results below):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ foo.pl names.txt | shuf -n 10&#xA;uniprotkb:Q9BWC9    uniprotkb:P04637    -   -   uniprotkb:CCDC106(gene_name)    uniprotkb:TP53(gene_name)   psi-mi:&quot;MI:0007&quot;(anti tag coimmunoprecipitation)    Zhou, J. et al.(2010)   pubmed:20159018 taxid:9606(Homo sapiens)    taxid:9606(Homo sapiens)    -   psi-mi:&quot;MI:0469&quot;(intact)    intact:EBI-7812926  -&#xA;uniprotkb:P04637    uniprotkb:O15126    -   -   uniprotkb:TP53(gene_name)   uniprotkb:SCAMP1(gene_name) psi-mi:&quot;MI:0018&quot;(two hybrid)    Lim, J. et al.(2006)    pubmed:16713569 taxid:9606(Homo sapiens)    taxid:9606(Homo sapiens)    -   psi-mi:&quot;MI:0463&quot;(biogrid)   biogrid:720622  -&#xA;uniprotkb:P63165    uniprotkb:P04637    -   -   uniprotkb:SUMO1(gene_name)  uniprotkb:TP53(gene_name)   psi-mi:&quot;MI:0018&quot;(two hybrid)    Minty, A. et al.(2000)  pubmed:10961991 taxid:9606(Homo sapiens)    taxid:9606(Homo sapiens)    -   psi-mi:&quot;MI:0463&quot;(biogrid)   biogrid:262339  -&#xA;uniprotkb:P04637    uniprotkb:P31350    -   -   uniprotkb:TP53(gene_name)   uniprotkb:RRM2(gene_name)   psi-mi:&quot;MI:0416&quot;(fluorescence microscopy)   Xue, L. et al.(2003)    pubmed:12615712 taxid:9606(Homo sapiens)    taxid:9606(Homo sapiens)    -   psi-mi:&quot;MI:0465&quot;(dip)   dip:DIP-40167E  -&#xA;uniprotkb:P04637    uniprotkb:Q00987    -   -   uniprotkb:TP53(gene_name)   uniprotkb:MDM2(gene_name)   psi-mi:&quot;MI:0004&quot;(affinity chromatography technology)    Dai, MS. et al.(2004)   pubmed:15308643 taxid:9606(Homo sapiens)    taxid:9606(Homo sapiens)    -   psi-mi:&quot;MI:0463&quot;(biogrid)   biogrid:478073  -&#xA;uniprotkb:Q99576    uniprotkb:P04637    -   -   uniprotkb:TSC22D3(gene_name)    uniprotkb:TP53(gene_name)   psi-mi:&quot;MI:0428&quot;(imaging technique) Ayroldi, E. et al.(2015)    pubmed:25168242 taxid:9606(Homo sapiens)    taxid:9606(Homo sapiens)    -   psi-mi:&quot;MI:0463&quot;(biogrid)   biogrid:1255896 -&#xA;uniprotkb:P04637    uniprotkb:Q00987    -   -   uniprotkb:TP53(gene_name)   uniprotkb:MDM2(gene_name)   psi-mi:&quot;MI:0415&quot;(enzymatic study)   Lui, K. et al.(2013)    pubmed:23572512 taxid:9606(Homo sapiens)    taxid:9606(Homo sapiens)    -   psi-mi:&quot;MI:0463&quot;(biogrid)   biogrid:859223  -&#xA;uniprotkb:P25685    uniprotkb:P04637    -   -   uniprotkb:DNAJB1(gene_name) uniprotkb:TP53(gene_name)   psi-mi:&quot;MI:0004&quot;(affinity chromatography technology)    Qi, M. et al.(2014) pubmed:24361594 taxid:9606(Homo sapiens)    taxid:9606(Homo sapiens)    -   psi-mi:&quot;MI:0463&quot;(biogrid)   biogrid:938952  -&#xA;uniprotkb:P04637    uniprotkb:Q8IW41    -   -   uniprotkb:TP53(gene_name)   uniprotkb:MAPKAPK5(gene_name)   psi-mi:&quot;MI:0424&quot;(protein kinase assay)  Sun, P. et al.(2007)    pubmed:17254968 taxid:9606(Homo sapiens)    taxid:9606(Homo sapiens)    -   psi-mi:&quot;MI:0469&quot;(intact)    intact:EBI-1202077  -&#xA;uniprotkb:Q13526    uniprotkb:P04637    -   -   uniprotkb:PIN1(gene_name)   uniprotkb:TP53(gene_name)   psi-mi:&quot;MI:0096&quot;(pull down) Mantovani, F. et al.(2007)  pubmed:17906639 taxid:9606(Homo sapiens)    taxid:9606(Homo sapiens)    -   psi-mi:&quot;MI:0469&quot;(intact)    intact:EBI-6112688  -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-06-09T00:07:33.717" LastActivityDate="2017-06-09T00:07:33.717" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="578" PostTypeId="1" AcceptedAnswerId="582" CreationDate="2017-06-08T10:38:42.810" Score="11" ViewCount="4225" Body="&lt;p&gt;I’m trying to find common names from a list of scientific names (not all will have them though).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was attempting to use &lt;code&gt;taxize&lt;/code&gt; in R but it aborts if it doesn’t find an entry in EOL and I don’t know a way around this other than manually editing the list - in which case I might as well check all this way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have a better way of doing this?&lt;/p&gt;&#xA;" OwnerUserId="691" LastEditorUserId="191" LastEditDate="2017-06-08T11:33:58.550" LastActivityDate="2022-04-21T20:54:19.087" Title="How to convert species names into common names?" Tags="|r|database|taxonomy|" AnswerCount="5" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="579" PostTypeId="2" ParentId="578" CreationDate="2017-06-08T10:44:47.550" Score="14" Body="&lt;p&gt;use the NCBI taxon dump under &lt;a href=&quot;ftp://ftp.ncbi.nih.gov/pub/taxonomy&quot; rel=&quot;noreferrer&quot;&gt;ftp://ftp.ncbi.nih.gov/pub/taxonomy&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in taxdmp.zip you'll find all the names for a given NCBI taxon&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -w ^9606 names.dmp &#xA;9606    |   Homo sapiens    |       |   scientific name |&#xA;9606    |   Homo sapiens Linnaeus, 1758 |       |   authority   |&#xA;9606    |   human   |       |   genbank common name |&#xA;9606    |   man |       |   common name |&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastActivityDate="2017-06-08T10:44:47.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="580" PostTypeId="2" ParentId="527" CreationDate="2017-06-08T10:51:40.737" Score="3" Body="&lt;p&gt;miRbase is a database of miRNA sequences, their genomic locations and the evidence for their existence. It is not a database of their targets, so I am assuming that this is the information you are asking for. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite not having been updated since 2014, as far as I am aware miRBase is still the go to place for this data. For example, &lt;a href=&quot;http://blog.rnacentral.org/2015/04/new-training-course-online-resources.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;courses at the European Bioinformatics Institute&lt;/a&gt; were still teaching the use of miRBase in April this year (2017). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://rnacentral.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNACentral&lt;/a&gt; is repository for all non-coding RNAs, but as it is a synthesis of other databases, including miRBase, I suspect data their will still be largely the same as miRBase. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2017-06-08T10:51:40.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="581" PostTypeId="2" ParentId="578" CreationDate="2017-06-08T10:57:53.833" Score="1" Body="&lt;p&gt;As Pierre &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/579/298&quot;&gt;suggested&lt;/a&gt;, you can get a dump of such names from &lt;a href=&quot;ftp://ftp.ncbi.nih.gov/pub/taxonomy&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI&lt;/a&gt;. Then, to query it for the common name for a species using its Latin name, you can do:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;$ awk -F'\t' -vname=&quot;Mus musculus&quot; '($7==&quot;scientific name&quot; &amp;amp;&amp;amp; $3==name){&#xA;                                      a[$1]=$3&#xA;                                    } &#xA;                                    ($1 in a &amp;amp;&amp;amp; /genbank common name/){  &#xA;                                      print $3&#xA;                                    }' names.dmp&#xA;house mouse&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-06-08T21:12:29.263" LastActivityDate="2017-06-08T21:12:29.263" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="582" PostTypeId="2" ParentId="578" CreationDate="2017-06-08T11:03:51.673" Score="15" Body="&lt;p&gt;As &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/579/191&quot;&gt;Pierre mentioned&lt;/a&gt;, NCBI is a good resource for this kind of transformation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can still use &lt;code&gt;taxize&lt;/code&gt; to perform the conversion:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(&quot;taxize&quot;)&#xA;&#xA;species &amp;lt;- c('Helianthus annuus', 'Mycobacterium bovis', 'Rattus rattus', 'XX', 'Mus musculus')&#xA;&#xA;uids &amp;lt;- get_uid(species)&#xA;&#xA;# keep only uids which you have in the database&#xA;uids.found &amp;lt;- as.uid(uids[!is.na(uids)])&#xA;# keep only species names  corresponding to your ids&#xA;species.found &amp;lt;- species[!is.na(uids)]&#xA;&#xA;common.names &amp;lt;- sci2comm(uids.found, db = 'ncbi')&#xA;names(common.names) &amp;lt;- species.found&#xA;&#xA;common.names&#xA;&#xA;## output:&#xA;## $`Helianthus annuus`&#xA;## [1] &quot;common sunflower&quot;&#xA;## &#xA;## $`Mycobacterium bovis`&#xA;## character(0)&#xA;## &#xA;## $`Rattus rattus`&#xA;## [1] &quot;black rat&quot;&#xA;## &#xA;## $`Mus musculus`&#xA;## [1] &quot;house mouse&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This works correctly if no common name is available, or if there is no species in the database. You can also try a different database, e.g. &lt;code&gt;db='itis'&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantage of this approach is that you do not have to parse a file yourself. The downside is that it can be slow for large list, since for every species a database request is performed.&lt;/p&gt;&#xA;" OwnerUserId="191" LastEditorUserId="298" LastEditDate="2017-06-08T21:44:35.360" LastActivityDate="2017-06-08T21:44:35.360" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="583" PostTypeId="1" AcceptedAnswerId="602" CreationDate="2017-06-08T12:07:20.897" Score="4" ViewCount="445" Body="&lt;p&gt;I calculated an AMOVA from a genind object, with one hierarchical factor. In the table I obtain there are SSD values (for my grouping factor,&quot;Error&quot; and total) and sigma2 values (for my grouping factor and &quot;Error&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What does &quot;error&quot; stand for?&lt;/li&gt;&#xA;&lt;li&gt;How do I calculate Fst? Which values do I have to use?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A genind object in R is an object which contains allelic information about a set of individuals.&lt;br&gt;&#xA;This is the table that I get: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Analysis of Molecular Variance&#xA;&#xA;Call: amova(formula = plantula.dist ~ g, nperm = 100)  &#xA;&#xA;           SSD         MSD  df  &#xA;g     354992.9 354992.9297   1  &#xA;Error 310814.9    693.7834 448 &#xA;Total 665807.9   1482.8683 449 &#xA;&#xA;Variance components:  &#xA;       sigma2 P.value  &#xA;g     1574.69       0  &#xA;Error  693.78         &#xA;&#xA;Variance coefficients:  &#xA;       a   &#xA;224.9956   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="752" LastEditorUserId="775" LastEditDate="2017-06-08T21:33:04.827" LastActivityDate="2017-06-09T07:07:13.327" Title="How to calculate Fst from AMOVA" Tags="|r|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="584" PostTypeId="1" AcceptedAnswerId="585" CreationDate="2017-06-08T12:07:22.530" Score="5" ViewCount="805" Body="&lt;p&gt;We are studying six different human macrophage/dendritic cell types isolated from healthy skin. They all differ from each other in a few cell surface markers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are interested in the characteristics of each cell type (&quot;marker&quot; genes or biological processes), the differences between them (especially cell surface proteins), and their relations (e.g. &quot;which are the closest relatives?&quot;) as can be inferred from the transcriptome, from an immunological point of view. The wider context is HIV-infection, thus HIV infection-related differences (and similarities) are of particular interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One naive approach is to contrast two of the most similar cell types (as inferred from previous knowledge, the sorting strategy and the PCA plot), and produce a list of differentially expressed (DE) genes.&#xA;I have now a list of ~500 DE genes between two cell types, together with fold changes, expression means etc., produced with DESeq2 on bulk RNA-seq data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the best practices to process this list to answer some of the above?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've found PPI analysis on &lt;a href=&quot;http://networkanalyst.ca&quot; rel=&quot;noreferrer&quot;&gt;NetworkAnalyst&lt;/a&gt;, and  GO and pathway analysis on &lt;a href=&quot;http://innatedb.com&quot; rel=&quot;noreferrer&quot; title=&quot;InnateDB&quot;&gt;InnateDB&lt;/a&gt; useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What other options are available?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm particularly interested in approaches using &lt;code&gt;R&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="208" LastEditorUserId="208" LastEditDate="2017-06-08T12:53:17.433" LastActivityDate="2017-06-08T14:37:56.240" Title="What are the ways to process a list of differentially expressed genes?" Tags="|rna-seq|r|deseq2|networks|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="585" PostTypeId="2" ParentId="584" CreationDate="2017-06-08T12:38:57.067" Score="7" Body="&lt;p&gt;You originally had asked a very broad question, so I'll try to demonstrate why that is such a hard question to answer. I've done two fairly large differential analysis studies (and a few smaller ones) covering very different areas of research, and the approaches that other researchers used subsequent to my differential expression calculations were unsurprisingly also very different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S221112471401047X&quot; rel=&quot;noreferrer&quot;&gt;first study&lt;/a&gt;, I did a &lt;em&gt;de-novo&lt;/em&gt; transcriptome assembly of &lt;em&gt;Schmidtea mediterranea (Smed)&lt;/em&gt; from Illumina RNASeq reads, then carried out a DE analysis (DESeq) on a &lt;em&gt;β-catenin&lt;/em&gt; knockout mutant population compared to wildtype. The other researchers had some very impressive tools available at their disposal, allowing them to subsequently visualise the positional expression of differentially-expressed genes in the knockout and wildtype populations after amputation. They found a few candidate genes that had graded expression along the anterior-posterior axis in the wildtype population, and this expression disappeared in the knockout mutants. One of these genes was a &lt;em&gt;teashirt (tsh)&lt;/em&gt; family member, and a good candidate for further discovery based on the researchers' existing knowledge of the gene pathways associated with cellular regeneration. To cut a very long story short, they experimentally validated the &lt;em&gt;teashirt&lt;/em&gt; link through further RNAi studies, generating a double-headed &lt;em&gt;Smed&lt;/em&gt; as a result. This was followed up in Zebrafish &lt;em&gt;(Danio rerio)&lt;/em&gt;, because there happened to be another building about 100m away that specialised in Zebrafish development.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://www.growkudos.com/publications/10.1084%252Fjem.20160470&quot; rel=&quot;noreferrer&quot;&gt;second study&lt;/a&gt; was/is a bit more similar to your current situation. We looked at sorted dendritic cell expression in mice after the application of two Th2-inducing treatment conditions (injected &lt;em&gt;Nippostrongylus brasiliensis&lt;/em&gt;; applied DBP+FITC) and one control condition (injected PBS). I did a differential expression analysis with &lt;a href=&quot;http://www.bioconductor.org/packages/release/bioc/html/DESeq2.html&quot; rel=&quot;noreferrer&quot;&gt;DESeq2&lt;/a&gt;, and developed a &lt;a href=&quot;http://shiny.rstudio.com/gallery/&quot; rel=&quot;noreferrer&quot;&gt;Shiny App&lt;/a&gt; for result exploration after I got tired of getting asked to do a whole bunch of mechanical follow-up tasks (mostly generating graphs, lists, and heatmaps for different gene subsets). We ended up with thousands of differentially expressed genes, so it wasn't feasible to look individually at every gene for research evidence and pathways -- one scientist spent a couple of weeks looking at one interesting-looking candidate gene. The differentially-expressed genes were fed into the oh-so-expensive &lt;a href=&quot;https://www.qiagenbioinformatics.com/products/ingenuity-pathway-analysis/&quot; rel=&quot;noreferrer&quot;&gt;IPA&lt;/a&gt; (which gives results, but not necessarily good ones) and &lt;a href=&quot;https://david.ncifcrf.gov/&quot; rel=&quot;noreferrer&quot;&gt;DAVID&lt;/a&gt;, among a few others. These helped a little bit, and led to what was the central idea of our paper (i.e. that there were at least two different Th2 responses), but we still had the problem of too many &quot;significant&quot; pathways, and lots of differentially-expressed genes that were members of a whole bunch of other pathways. We tried &lt;a href=&quot;https://labs.genetics.ucla.edu/horvath/htdocs/CoexpressionNetwork/Rpackages/WGCNA/&quot; rel=&quot;noreferrer&quot;&gt;WGCNA&lt;/a&gt;; we did a bit of &lt;a href=&quot;http://software.broadinstitute.org/gsea/index.jsp&quot; rel=&quot;noreferrer&quot;&gt;GSEA&lt;/a&gt;; and we compared &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/limma.html&quot; rel=&quot;noreferrer&quot;&gt;Limma&lt;/a&gt; and DESeq2, but still couldn't find anything concrete or obvious to demonstrate what was triggering the Th2 response. There were experimental studies as well: blocking and mutant studies to investigate the two types of Th2 response (and experimentally confirm the differential expression). Our experiments and exploration are still ongoing, but we felt it was necessary to get one paper out at the intermediate stage, so just ended up taking the top 25 differentially-expressed genes in each experiment (by fold change), and finding some way of grouping some of them together (via expert knowledge and literature searches). It kind-of-sort-of worked, but we're left with a bit of frustration in that we (and the tools we used) were basically stumped by having too many biologically-relevant results.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-08T14:37:56.240" LastActivityDate="2017-06-08T14:37:56.240" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="586" PostTypeId="2" ParentId="565" CreationDate="2017-06-08T14:32:07.633" Score="4" Body="&lt;p&gt;The problem is that GFF, fundamentally, is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Relational_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;relational&lt;/a&gt; format: it provides tags that relate individual rows via one-to-many relationships (e.g. gene–exon). This indirectly highlights the second complication: individual rows have different types, and therefore store different attributes in the 9th column.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Over the last few decades (!), we have accumulated a wealth of theory and tools to work with this kind of data. And the usual solution is to create a database schema for a relational database, and to use database drivers and database query languages (e.g. SQL, but increasingly also data-relational mappers such as LINQ and dplyr).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a text-based format is attractive for many reasons that Devon mentions, but it is fundamentally at odds with a lot of the theory and tools for relational data. This creates an &lt;a href=&quot;https://en.wikipedia.org/wiki/Object-relational_impedance_mismatch&quot; rel=&quot;nofollow noreferrer&quot;&gt;impedance mismatch&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’m convinced that, in the long run, the solution is going to be to revert to using relational databases for complex annotations. I say “revert” even though these databases already exist, they are simply often ignored in bioinformaticians’ day-to-day work (&lt;em&gt;I&lt;/em&gt; never use them). Because this is objectively the best technical solution, and we have the research to back this up.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2017-06-08T14:32:07.633" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="587" PostTypeId="1" AcceptedAnswerId="592" CreationDate="2017-06-08T14:54:57.103" Score="11" ViewCount="425" Body="&lt;p&gt;We're considering switching our storage format from BAM to CRAM. We work with human cancer samples, which may have very low prevalence variants (i.e. not diploid frequency).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we use lossy CRAM to save more space, how much will variants called from those CRAM files change? Which compression strategy has the lowest impact? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any other impacts on downstream tools that we're not considering?&lt;/p&gt;&#xA;" OwnerUserId="110" LastActivityDate="2017-06-09T16:28:26.360" Title="Do variant calls change when you call from CRAM?" Tags="|sam|file-formats|variant-calling|storage|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="588" PostTypeId="2" ParentId="558" CreationDate="2017-06-08T15:00:23.893" Score="1" Body="&lt;p&gt;I would expect that without any further insights into protein folding, a quantum method would look fairly similar to existing methods, but might have a bit more capability of exploring multiple states at the same time (without multitasking).&lt;/p&gt;&#xA;&lt;p&gt;Here's the approach used by &lt;a href=&quot;http://boinc.bakerlab.org/rosetta/rah_graphics.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rosetta&lt;/a&gt;, which is used in both the &lt;a href=&quot;http://boinc.bakerlab.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rosetta@home&lt;/a&gt; distributed computing project, and the &lt;a href=&quot;http://fold.it/portal/&quot; rel=&quot;nofollow noreferrer&quot;&gt;foldit&lt;/a&gt; distributed gaming project:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Rosetta's strategy for finding low energy shapes looks like this:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Start with a fully unfolded chain (like a metal chain with its ends pulled).&lt;/li&gt;&#xA;&lt;li&gt;Move a part of the chain to create a new shape.&lt;/li&gt;&#xA;&lt;li&gt;Calculate the energy of the new shape.&lt;/li&gt;&#xA;&lt;li&gt;Accept or reject the move depending on the change in energy.&lt;/li&gt;&#xA;&lt;li&gt;Repeat 2 through 4 until every part of the chain has been moved a lot of times.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;We call this a trajectory. The end result of a trajectory is a predicted structure.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-06-08T15:00:23.893" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="589" PostTypeId="2" ParentId="194" CreationDate="2017-06-08T15:27:39.173" Score="2" Body="&lt;p&gt;I would be concerned about the input data that produced these numbers. This dataset has a really odd shape with a dip at the start:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data.df &amp;lt;- read.table(&amp;quot;soft_threshold.txt&amp;quot;, header=TRUE);&#xA;png(&amp;quot;scale-free-194.png&amp;quot;);&#xA;plot(data.df$SFT.R.sq);&#xA;dummy &amp;lt;- dev.off();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/F1MT9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/F1MT9.png&quot; alt=&quot;scale-free power with an odd inflection&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here's what I expect this to look like (from the supplementary information in &lt;a href=&quot;https://www.pnas.org/doi/10.1073/pnas.0605938103&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Vp4Pk.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Vp4Pk.gif&quot; alt=&quot;normal scale-free topology&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It's easier to select the power/threshold when the plot looks like the second situation. With the data as presented, it doesn't look like any value would be suitable. but if I were forced to pick, then I'd choose 12 because it is the first number above the magic threshold (ignoring the weirdness at the start). However, I'll admit that I don't understand the concepts behind the power, and am just following the recommended instructions, so can't explain why it is what it is.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="1157" LastEditDate="2022-12-20T18:41:21.107" LastActivityDate="2022-12-20T18:41:21.107" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="590" PostTypeId="1" CreationDate="2017-06-08T15:46:03.133" Score="2" ViewCount="78" Body="&lt;p&gt;I want to calculate thermodynamic paremeter enthalpy change of cellulase, but I don't know what software can do this work. Please tell me, thank you very much !&lt;/p&gt;&#xA;" OwnerUserId="762" LastEditorUserId="96" LastEditDate="2017-06-08T20:07:15.413" LastActivityDate="2017-06-16T15:33:36.017" Title="Is there a software or web server to calculate thermodynamic parameter enthalpy change of cellulase?" Tags="|protein-structure|thermodynamics|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="591" PostTypeId="2" ParentId="565" CreationDate="2017-06-08T15:49:51.480" Score="4" Body="&lt;p&gt;I quite like BED and GFF3 (I don't like GTF/GFF2, though). As text-based formats, I don't think they leave us much room for improvement. Anyway, if you want a new format, here is one. The following is a hybrid between GFF3 and BED. It is a TAB-delimited text-based format with the following fields:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;chr, required&lt;/li&gt;&#xA;&lt;li&gt;start (0-based), required&lt;/li&gt;&#xA;&lt;li&gt;end, required&lt;/li&gt;&#xA;&lt;li&gt;strand&lt;/li&gt;&#xA;&lt;li&gt;ID&lt;/li&gt;&#xA;&lt;li&gt;type (see below)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Like BED, only the first 3 columns are required, the rest are optional. What is different starts with the &quot;type&quot; field as in GFF. This &quot;type&quot;, when present, defines the columns following it. For example, if type==coding, we could have cdsStart, cdsEnd, blockCount, blockSizes and blockStarts like BED; if type==exon, column 7 could be a &quot;phase&quot;, indicating the phase of the first base. This way, &quot;type&quot; makes this format highly extensible while still relatively easy to parse in comparison to using optional tags all the way. In addition, we may have semi-colon-separated &quot;key&quot;=&quot;value&quot; pairs at the end of each line as in GFF3. Example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1   10000  50000  -  x1 coding   10100   40800  2  1000,2000   10000,48000&#xA;chr1   10000  11000  -  *  exon     *       foo1=bar1;foo2=bar2&#xA;chr1   48000  50000  -  *  exon     2&#xA;chr2   10000  50000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above only gives a bare bone of the format. There are subtle questions such as: 1) the uniqueness/scope of ID; 2) whether to have parent ID as a fixed field or a type-specific field; 3) where to put display name; 4) sorting order. These are also important in practice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I like SQL a lot and think it should get used more often in bioinformatics, but I don't think it replaces text formats completely. Formats are useful for serialization and data transfer. They require less skill to work with and less software/hardware resource to deploy. Carefully engineered binary representations of formats can be much more efficient in a lot of use cases.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-06-08T15:49:51.480" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="592" PostTypeId="2" ParentId="587" CreationDate="2017-06-08T16:20:00.480" Score="6" Body="&lt;p&gt;By default, a CRAM you create with samtools is lossless. It typically halves the input BAM in terms of file size. If you want to compress more, you can let samtools convert most read names to integers. You won't be able to tell optical duplicates from read names, but this is a minor concern. You can also drop useless tags depending on your mapper and the downstream caller in use. For cancer data, I wouldn't reduce the resolution of base quality without comprehensive benchmarks. Unfortunately, base quality takes most of space in CRAM. Discarding the original read names and some tags probably won't save you much space.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-06-08T16:20:00.480" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="593" PostTypeId="2" ParentId="558" CreationDate="2017-06-08T17:54:28.040" Score="5" Body="&lt;p&gt;Protein folding problem can be viewed as a minimization problem. One can use &lt;a href=&quot;https://en.wikipedia.org/wiki/Quantum_annealing&quot; rel=&quot;noreferrer&quot;&gt;quantum annealing&lt;/a&gt; to perform the minimization. Running this on quantum computers would improve the performance since they can perform the tunneling directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, quantum annealing was &lt;a href=&quot;https://dx.doi.org/10.1038/srep00571&quot; rel=&quot;noreferrer&quot;&gt;used&lt;/a&gt; (&lt;a href=&quot;http://blogs.nature.com/news/2012/08/d-wave-quantum-computer-solves-protein-folding-problem.html&quot; rel=&quot;noreferrer&quot;&gt;blog post&lt;/a&gt;) for lattice protein folding on the the D-Wave quantum computer (128 qubits). But this was an extremely simplified model, with only 40 discrete states.&lt;/p&gt;&#xA;" OwnerUserId="191" LastActivityDate="2017-06-08T17:54:28.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="594" PostTypeId="1" AcceptedAnswerId="601" CreationDate="2017-06-08T20:21:18.697" Score="7" ViewCount="603" Body="&lt;p&gt;I have around ~3,000 short sequences of approximately ~10Kb long. What are the best ways to find the motifs among all of these sequences? Is there a certain software/method recommended?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several ways to do this. My goal would be to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) Check for motifs repeated within individual sequences&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) Check for motifs shared among all sequences&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(3) Check for the presence of &quot;expected&quot; or known motifs&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With respect to #3, I'm also curious if I find e.g. trinucleotide sequences, how does one check the context around these regions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for the recommendations/help!&lt;/p&gt;&#xA;" OwnerUserId="146" LastEditorUserId="146" LastEditDate="2017-06-08T21:34:24.920" LastActivityDate="2017-06-13T22:20:11.820" Title="What motif finding software is available for multiple sequences ~10Kb?" Tags="|motifs|software-recommendation|" AnswerCount="5" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="595" PostTypeId="1" AcceptedAnswerId="603" CreationDate="2017-06-08T21:34:33.547" Score="9" ViewCount="3751" Body="&lt;p&gt;Is there a convenient way to extract the longest isoforms from a transcriptome fasta file? I had found some &lt;a href=&quot;https://www.biostars.org/p/107759/&quot; rel=&quot;nofollow noreferrer&quot;&gt;scripts on biostars&lt;/a&gt; but none are functional and I'm having difficulty getting them to work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm aware that the longest isoforms aren't necessarily 'the best' but it will suit my purposes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fasta was generated via Augustus. Here is what the fasta file looks like currently (sequence shortened to save space)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Doug_NoIndex_L005_R1_001_contig_2.g7.t1&#xA;atggggcataacatagagactggtgaacgtgctgaaattctacttcaaagtctacctgattcgtatgatcaactcatca&#xA;ttaatataaccaaaaacctagaaattctagccttcgatgatgttgcagctgcggttcttgaagaagaaagtcggcgcaagaacaaagaagatagaccg&#xA;&amp;gt;Doug_NoIndex_L005_R1_001_contig_2.g7.t2&#xA;atggggcataacatagagactggtgaacgtgctgaaattctacttcaaagtctacctgattcgtatgatcaactcatca&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The format is as such:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Gene 1 isoform 1  &#xA;Gene 1 isoform 2  &#xA;Gene 2 isoform 1  &#xA;Gene 2 isoform 2   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and so forth. There are several genes that have more than one pair of isoforms (up to 3 or 4). There are roughly 80,000 total transcripts, probably 25,000 genes. I would like to extract the single longest isoform for each gene.&lt;/p&gt;&#xA;" OwnerUserId="779" LastEditorUserId="73" LastEditDate="2017-06-09T15:15:22.383" LastActivityDate="2017-06-13T00:51:03.047" Title="How can longest isoforms (per gene) be extracted from a FASTA file?" Tags="|fasta|isoform|filtering|" AnswerCount="5" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="597" PostTypeId="2" ParentId="595" CreationDate="2017-06-08T22:03:44.497" Score="1" Body="&lt;p&gt;An ex-coworker (Josep Avril) has written a couple of very useful little scripts that convert fasta to tbl (&lt;code&gt;seqID&amp;lt;TAB&amp;gt;Sequence&lt;/code&gt;) and back again. These are extremely handy for this sort of thing (and are included at the end of this answer). Using them, you can convert your fasta to a one sequence per line format, keep the longest sequence with a simple awk script and convert back to fasta.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am assuming that the part after the last &lt;code&gt;.&lt;/code&gt; in the ID line should be removed (so that &lt;code&gt;Doug_NoIndex_L005_R1_001_contig_2.g7.t1&lt;/code&gt; and &lt;code&gt;Doug_NoIndex_L005_R1_001_contig_2.g7.t2&lt;/code&gt; both map to &lt;code&gt;Doug_NoIndex_L005_R1_001_contig_2.g7&lt;/code&gt;). If that is a correct assumption, this should work for you:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ FastaToTbl file.fa | sed 's/\.[^.]*\t/\t/' | &#xA;    awk -F&quot;\t&quot; '{&#xA;                    if(length($2)&amp;gt;length(a[$1])){&#xA;                        a[$1]=$0&#xA;                    }&#xA;                }&#xA;                END{&#xA;                    for(i in a){&#xA;                        print a[i]&#xA;                    }&#xA;                }' | TblToFasta&#xA;&amp;gt;Doug_NoIndex_L005_R1_001_contig_2.g7 &#xA;atggggcataacatagagactggtgaacgtgctgaaattctacttcaaagtctacctgat&#xA;tcgtatgatcaactcatcattaatataaccaaaaacctagaaattctagccttcgatgat&#xA;gttgcagctgcggttcttgaagaagaaagtcggcgcaagaacaaagaagatagaccg&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One possible issue with that approach is that it needs to keep one sequence per ID in memory. If you have a huge fasta file and not much memory, that might be a problem. If that's the case, you can first sort the file to ensure that similar IDs always appear together and then print each line as soon as you find the next ID:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FastaToTbl file.fa | sed 's/\.[^.]*\t/\t/' | LC_ALL=C sort -t$'\t' -k1,1 |  &#xA;    awk -F&quot;\t&quot; '{&#xA;                    if(NR&amp;gt;1 &amp;amp;&amp;amp; prev!=$1){&#xA;                        print a[$1]; &#xA;                        prev=$1&#xA;                    } &#xA;                    if(length($2)&amp;gt;length(a[$1])){&#xA;                        a[$1]=$0&#xA;                    }&#xA;                }&#xA;                END{&#xA;                    print a[$1]&#xA;                }' | TblToFasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;FastaToTbl&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/awk -f&#xA;{&#xA;        if (substr($1,1,1)==&quot;&amp;gt;&quot;)&#xA;        if (NR&amp;gt;1)&#xA;                    printf &quot;\n%s\t&quot;, substr($0,2,length($0)-1)&#xA;        else &#xA;            printf &quot;%s\t&quot;, substr($0,2,length($0)-1)&#xA;        else &#xA;                printf &quot;%s&quot;, $0&#xA;}END{printf &quot;\n&quot;}&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;TblToFasta&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;#! /usr/bin/awk -f&#xA;{&#xA;  sequence=$NF&#xA;&#xA;  ls = length(sequence)&#xA;  is = 1&#xA;  fld  = 1&#xA;  while (fld &amp;lt; NF)&#xA;  {&#xA;     if (fld == 1){printf &quot;&amp;gt;&quot;}&#xA;     printf &quot;%s &quot; , $fld&#xA;&#xA;     if (fld == NF-1)&#xA;      {&#xA;        printf &quot;\n&quot;&#xA;      }&#xA;      fld = fld+1&#xA;  }&#xA;  while (is &amp;lt;= ls)&#xA;  {&#xA;    printf &quot;%s\n&quot;, substr(sequence,is,60)&#xA;    is=is+60&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-06-10T07:35:23.737" LastActivityDate="2017-06-10T07:35:23.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="598" PostTypeId="2" ParentId="561" CreationDate="2017-06-08T22:04:16.040" Score="7" Body="&lt;p&gt;I wrote a command-line k-mer counter called &lt;code&gt;kmer-counter&lt;/code&gt; that will output results in a form that your Python script can consume: &lt;a href=&quot;https://github.com/alexpreynolds/kmer-counter&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/alexpreynolds/kmer-counter&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can grab, build and install it like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ git clone https://github.com/alexpreynolds/kmer-counter.git&#xA;$ cd kmer-counter&#xA;$ make&#xA;$ cp kmer-counter /usr/local/bin&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Once the binary is in your path, you might use it in Python like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;k = 6&#xA;fastaFile = '/path/to/some/seqs.fa'&#xA;kmerCmd = 'kmer-counter --fasta --k=%d %s' % (k, fastaFile)&#xA;try:&#xA;    output = subprocess.check_output(kmerCmd, shell=True)&#xA;    result = {}&#xA;    for line in output.splitlines():&#xA;        (header, counts) = line.strip().split('\t')&#xA;        header = header[1:]&#xA;        kmers = dict((k,int(v)) for (k,v) in [d.split(':') for d in counts.split(' ')])&#xA;        result[header] = kmers&#xA;    sys.stdout.write(&quot;%s&quot; % (str(result)))&#xA;except subprocess.CalledProcessError as error:&#xA;    sys.stderr.write(&quot;%s&quot; % (str(error)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Given example FASTA like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;foo&#xA;TTAACG&#xA;&amp;gt;bar&#xA;GTGGAAGTTCTTAGGGCATGGCAAAGAGTCAGAATTTGAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For k=6, you would get an iterable Python dictionary like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{'foo': {'TTAACG': 1, 'CGTTAA': 1}, 'bar': {'GTTCTT': 1, 'AGAACT': 1, 'GAGTCA': 1, 'ATGGCA': 1, 'GAACTT': 1, 'ATTCTG': 1, 'CTAAGA': 1, 'CTTCCA': 1, 'ATTTGA': 1, 'GGAAGT': 1, 'AGGGCA': 1, 'CCTAAG': 1, 'CTCTTT': 1, 'AATTTG': 1, 'TCTGAC': 1, 'TTTGCC': 1, 'CTTAGG': 1, 'TTTGAC': 1, 'GAAGTT': 1, 'CCCTAA': 1, 'AGAATT': 1, 'AGTCAG': 1, 'CTGACT': 1, 'TCTTAG': 1, 'CGTTAA': 1, 'GTGGAA': 1, 'TGCCAT': 1, 'ACTCTT': 1, 'GGGCAT': 1, 'TTAGGG': 1, 'CTTTGC': 1, 'TGGAAG': 1, 'GACTCT': 1, 'CATGCC': 1, 'GCAAAG': 1, 'AAATTC': 1, 'GTCAAA': 1, 'TGACTC': 1, 'TAGGGC': 1, 'AAGTTC': 1, 'ATGCCC': 1, 'TCAAAT': 1, 'CAAAGA': 1, 'AACTTC': 1, 'GTCAGA': 1, 'CAAATT': 1, 'TAAGAA': 1, 'CATGGC': 1, 'AAGAAC': 1, 'AAGAGT': 1, 'TCTTTG': 1, 'TTCCAC': 1, 'TGGCAA': 1, 'GGCAAA': 1, 'AGTTCT': 1, 'AGAGTC': 1, 'TCAGAA': 1, 'GAATTT': 1, 'AAAGAG': 1, 'TGCCCT': 1, 'CCATGC': 1, 'GGCATG': 1, 'TTGCCA': 1, 'CAGAAT': 1, 'AATTCT': 1, 'GCATGG': 1, 'ACTTCC': 1, 'TTCTTA': 1, 'GCCATG': 1, 'GCCCTA': 1, 'TTCTGA': 1}}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can use standard Python calls to manipulate this dictionary object and get sums of counts per record, for sequence, etc. which seems to answer your question. Please feel free to clarify what you're looking for if this object representation is not clear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a fully-fleshed out demonstration, see: &lt;a href=&quot;https://github.com/alexpreynolds/kmer-counter/blob/master/test/kmer-test.py&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/alexpreynolds/kmer-counter/blob/master/test/kmer-test.py&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2017-07-27T20:08:44.917" LastActivityDate="2017-07-27T20:08:44.917" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="599" PostTypeId="2" ParentId="595" CreationDate="2017-06-08T22:06:35.127" Score="1" Body="&lt;p&gt;This solution is will work for your example and probably all Augustus-derived Fastas, but mileage will vary beyond that.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python&#xA;from __future__ import print_function&#xA;import sys&#xA;&#xA;def parse_fasta(data):&#xA;    &quot;&quot;&quot;Stolen shamelessly from http://stackoverflow.com/a/7655072/459780.&quot;&quot;&quot;&#xA;    name, seq = None, []&#xA;    for line in data:&#xA;        line = line.rstrip()&#xA;        if line.startswith('&amp;gt;'):&#xA;            if name:&#xA;                yield (name, ''.join(seq))&#xA;            name, seq = line, []&#xA;        else:&#xA;            seq.append(line)&#xA;    if name:&#xA;        yield (name, ''.join(seq))&#xA;&#xA;isoforms = dict()&#xA;for defline, sequence in parse_fasta(sys.stdin):&#xA;    geneid = '.'.join(defline[1:].split('.')[:-1])&#xA;    if geneid in isoforms:&#xA;        otherdefline, othersequence = isoforms[geneid]&#xA;        if len(sequence) &amp;gt; len(othersequence):&#xA;            isoforms[geneid] = (defline, sequence)&#xA;    else:&#xA;        isoforms[geneid] = (defline, sequence)&#xA;&#xA;for defline, sequence in isoforms.values():&#xA;    print(defline, sequence, sep='\n')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Save as &lt;code&gt;longest.py&lt;/code&gt;, and invoke on the command line like so.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python longest.py &amp;lt; intput.fasta &amp;gt; output.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The line &lt;code&gt;geneid = '.'.join(defline[1:].split('.')[:-1])&lt;/code&gt; is key. Let me break it down.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;defline[1:]&lt;/code&gt;: ignore the first character of the defline (the &lt;code&gt;&amp;gt;&lt;/code&gt; symbol)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;defline[1:].split('.')&lt;/code&gt;: split the string on &lt;code&gt;.&lt;/code&gt; symbol&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;defline[1:].split('.')[:-1]&lt;/code&gt;: ignore the last value after the split (the isoform name)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;'.'.join(defline[1:].split('.')[:-1])&lt;/code&gt;: join the split values again&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This value will be the same for all isoforms for the same gene, so we use it in the script to keep track of the longest sequence associated with each gene ID.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="191" LastEditDate="2017-06-09T08:07:00.293" LastActivityDate="2017-06-09T08:07:00.293" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="600" PostTypeId="2" ParentId="565" CreationDate="2017-06-08T22:09:13.897" Score="0" Body="&lt;p&gt;Just throwing this out there, as all the best stuff has already been said, but why does one need to specify a strict specification for a bioinformatic data format? As you say, what typically happens is all the action ends up in the optional fields.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a lot to be said for the libertarian values of just letting people figure it out on their own. Take optional fields in the BAM tag spec. for example. Here you can have your own tags, but they must start with an &quot;X&quot;, a &quot;Y&quot; or a &quot;Z&quot; and be followed by &quot;[A-Za-z0-9]&quot;. Why? Why can't people use their own tag names like &quot;read is unique&quot; or &quot;edit distance to genome&quot; or whatever they want. Are we not to be trusted with the power of naming things and recalling things arbitrarily? The result is one must look up the tag name's actual meaning in the publication of the tool that produced it, or ask on certain forums, etc. And this is assuming one knows the tool that produced the reads - if not then who knows wtf 'XT' stands for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, downstream readers of optional fields already trust that 'XT' is what it thinks it is. So if we're happy to use optional fields in any capacity, why stop at fields?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Extend this logic to the columns of the data format. Let users determine column names. You may once in a blue moon come across a file with the chromosome column as &quot;chr&quot; rather than &quot;chromosome&quot;, but most of the time you'll be looking for &quot;read is quantum-radioactive&quot;, and detecting and fixing naming-errors is not going to be as much of a problem as using a dataformat that does not store what it is you want to store. Or worse, it can store it, but in a really illogical way that results in everyone who uses the dataformat to have to ask at least 3 questions here or on other forums before they understand whats really going on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you end up with a general solution to the problem of storing tabular information in a compatible way across many different types of software, also known as SQL (or any other kind of general-purpose database such as REDIS, Neo4J, Numpy, etc). In fact, does it even matter what the datastore is, so long as it's tabular and has the &quot;chromosome&quot; and &quot;position&quot; values for each item?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TL;DR - We will not think of tomorrow's best data format today, because the nature of tomorrow's data is not yet known. Less policing in this area would most likely result in more robust software, where nothing is taken for granted and no assumptions about the data can be made until the schema has been parsed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In truth, the only reason we think we need to specify our dataformats is because we know if we didn't, other people will do something we didn't think of - and we, i think falsely, assume that will be bad.&lt;/p&gt;&#xA;" OwnerUserId="672" LastEditorUserId="205" LastEditDate="2017-06-09T01:59:00.970" LastActivityDate="2017-06-09T01:59:00.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="601" PostTypeId="2" ParentId="594" CreationDate="2017-06-09T02:19:01.120" Score="1" Body="&lt;p&gt;For (3), &lt;a href=&quot;http://www.geneinfinity.org/sp/sp_motif.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt; has a lot of links to pattern/motif finding tools. Following through the &lt;a href=&quot;http://bio.cs.washington.edu/YMF/YMFWeb/YMFInput.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;YMF&lt;/a&gt; link on that page, I came across the University of Washington &lt;a href=&quot;http://bio.cs.washington.edu/software/motif_discovery#Motif%20Discovery&quot; rel=&quot;nofollow noreferrer&quot;&gt;Motif Discovery&lt;/a&gt; section. Of these &lt;a href=&quot;http://www1.cse.wustl.edu/~jbuhler/pgt/&quot; rel=&quot;nofollow noreferrer&quot;&gt;projection&lt;/a&gt; seemed to be the only downloadable tool. I find it interesting how old all these tools are; maybe the introduction of microarrays and NGS has made them all redundant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your sub-problem (2) seems similar to the problem I'm having with &lt;em&gt;Nippostrongylus brasiliensis&lt;/em&gt; genome sequences, where I'd like to find regions of very high homology (length 500bp to 20kb or more, 95-99% similar) that are repeated throughout the genome. These sequences are killing the assembly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main way I can find these regions is by looking at a coverage plot of long nanopore reads mapped to the assembled genome (using GraphMap or BWA). Any regions with substantially higher than median coverage are likely to be shared repeats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've played around in the past with chopping up the reads to smaller sizes, which works better for hitting smaller repeated regions that are such a small proportion of most reads that they are never mapped to all the repeated locations. I wrote &lt;a href=&quot;https://github.com/gringer/bioinfscripts/blob/master/normalise_seqlengths.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;my own script&lt;/a&gt; a while back to chop up reads (for a different purpose), which produces a FASTA/FASTQ file where all reads are exactly the same length. For some unknown reason I took the time to document that script &quot;properly&quot; using POD, so here's a short summary:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Converts all sequences in the input FASTA file to the same length.&#xA;     Sequences shorter than the target length are dropped, and sequences longer&#xA;     than the target length are split into overlapping subsequences covering&#xA;     the entire range. This prepares the sequences for use in an&#xA;     overlap-consensus assembler requiring constant-length sequences (such as&#xA;     edena).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And here's the syntax:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ./normalise_seqlengths.pl -h&#xA;Usage:&#xA;    ./normalise_seqlengths.pl &amp;lt;reads.fa&amp;gt; [options]&#xA;&#xA;  Options:&#xA;    -help&#xA;      Only display this help message&#xA;&#xA;    -fraglength&#xA;      Target fragment length (in base-pairs, default 2000)&#xA;&#xA;    -overlap&#xA;      Minimum overlap length (in base-pairs, default 200)&#xA;&#xA;    -short&#xA;      Keep short sequences (shorter than fraglength)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-09T04:14:39.550" LastActivityDate="2017-06-09T04:14:39.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="602" PostTypeId="2" ParentId="583" CreationDate="2017-06-09T07:07:13.327" Score="1" Body="&lt;p&gt;Error is the estimation that can't be fitted by your parameters (in this case g). &#xA;The amova function in pegas package does the following: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This function performs a hierarchical analysis of molecular variance as described in Excoffier et al.&#xA;  (1992). This implementation accepts any number of hierarchical levels&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And as per the definition of Fixation index, you need the variance in the frequency of the allele between different subpopulations, weighted by the sizes of the subpopulations, the variance of the allelic state in the total population. Which is not provided by amova. Thus you cannot compute the fixation index from amova results. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2017-06-09T07:07:13.327" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="603" PostTypeId="2" ParentId="595" CreationDate="2017-06-09T07:17:47.383" Score="6" Body="&lt;p&gt;While the solution from &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/96/daniel-standage&quot;&gt;https://bioinformatics.stackexchange.com/users/96/daniel-standage&lt;/a&gt; should work (after adjusting for possible python3 incompatibility), the following is a shorter and less memory demanding method that uses biopython:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python&#xA;from Bio import SeqIO&#xA;import sys&#xA;&#xA;lastGene = None&#xA;longest = (None, None)&#xA;for rec in SeqIO.parse(sys.argv[1], &quot;fasta&quot;):&#xA;    gene = &quot;.&quot;.join(rec.id.split(&quot;.&quot;)[:-1])&#xA;    l = len(rec)&#xA;    if lastGene is not None:&#xA;        if gene == lastGene:&#xA;            if longest[0] &amp;lt; l:&#xA;                longest = (l, rec)&#xA;        else:&#xA;            lastGene = gene&#xA;            SeqIO.write(longest[1], sys.stdout, &quot;fasta&quot;)&#xA;            longest = (l, rec)&#xA;    else:&#xA;        lastGene = gene&#xA;        longest = (l, rec)&#xA;SeqIO.write(longest[1], sys.stdout, &quot;fasta&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you saved this as &lt;code&gt;filter.py&lt;/code&gt;, then &lt;code&gt;filter.py original.fa &amp;gt; subset.fa&lt;/code&gt; would be the command to use.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="191" LastEditDate="2017-06-09T08:06:41.350" LastActivityDate="2017-06-09T08:06:41.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="604" PostTypeId="1" CreationDate="2017-06-09T09:15:26.540" Score="6" ViewCount="238" Body="&lt;p&gt;I got a customized GRCh38.79 .gtf file (modified to have no MT genes) and I need to create a reference genome out of it (for 10xGenomics CellRanger pipeline). I suspect that the .79 part is the Ensembl number, which according this &lt;a href=&quot;http://www.ensembl.org/Help/ArchiveList&quot; rel=&quot;nofollow noreferrer&quot;&gt;ensembl archive list&lt;/a&gt; is paired with the GRCh38.p2 patch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I use this patch's fasta file, or it would be fine to use any of the GRCh38 patches?&lt;/p&gt;&#xA;" OwnerUserId="263" LastEditorUserId="37" LastEditDate="2017-06-09T16:09:16.243" LastActivityDate="2017-09-09T01:21:12.907" Title="Can a customized GRCh38 .gtf file be used with any of the GRCh38 released patches?" Tags="|human-genome|reference-genome|gtf|10x-genomics|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="605" PostTypeId="2" ParentId="594" CreationDate="2017-06-09T09:41:25.517" Score="1" Body="&lt;p&gt;Most tools I know of looks for enrichment of specific motifs - but that requires that you have a set of sequences which are of special interest and a background set to test against.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is that your case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update after comments 12th June 2017.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could try the &lt;a href=&quot;http://meme-suite.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;meme suite&lt;/a&gt; more specifically &lt;a href=&quot;http://meme-suite.org/tools/meme&quot; rel=&quot;nofollow noreferrer&quot;&gt;the motif finder&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="599" LastEditorUserId="57" LastEditDate="2017-06-13T16:21:27.300" LastActivityDate="2017-06-13T16:21:27.300" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="606" PostTypeId="2" ParentId="594" CreationDate="2017-06-09T10:20:12.173" Score="1" Body="&lt;p&gt;it is under development, but maybe &lt;a href=&quot;https://github.com/soedinglab/BaMMmotif&quot; rel=&quot;nofollow noreferrer&quot;&gt;BaMMmotif!&lt;/a&gt; is something for you? Its main selling point is that it can look for motifs enriched in a set of sequences of equal length &lt;em&gt;de novo&lt;/em&gt;. If you can't/don't want to supply a negative set it learns one from the positive sequences. There is a wealth of options to choose from if you have more information about your sequences: there are different models for &quot;zero or one&quot;, &quot;one&quot; and &quot;multiple&quot; occurrences of the motif.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use it to look for known motifs, if you encode them as an XXmotif PWM. If you have a file with motifs (like binding sites) you can use that as initialization as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I have not used the software myself, the authors are very responsive on git and the installation instructions seem pretty straightforward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Apparently the software was developed with ChIP experiments as the usual use case scenario, it might misbehave for larger sequences or take super long to run.&lt;/p&gt;&#xA;" OwnerUserId="787" LastEditorUserId="787" LastEditDate="2017-06-12T09:44:17.427" LastActivityDate="2017-06-12T09:44:17.427" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="607" PostTypeId="2" ParentId="595" CreationDate="2017-06-09T10:51:42.197" Score="3" Body="&lt;p&gt;Here is a solution in R. Could get really slow with big files. Works for the example you posted.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Biostrings)&#xA;&#xA;## read your fasta in as Biostrings object&#xA;fasta.s &amp;lt;- readDNAStringSet(&quot;sample.fa&quot;)&#xA;&#xA;## get the read names (in your case it has the isoform info)&#xA;names.fasta &amp;lt;- names(fasta.s)&#xA;&#xA;## extract only the relevant gene and isoform id (split name by the period symbol)&#xA;gene.iso &amp;lt;- sapply(names.fasta,function(j) cbind(unlist(strsplit(j,'\\.'))[2:3]))&#xA;&#xA;## convert to good data.frame = transpose result from previous step and add relevant column names&#xA;gene.iso.df &amp;lt;- data.frame(t(gene.iso))&#xA;colnames(gene.iso.df) &amp;lt;- c('gene','isoform')&#xA;&#xA;## and length of isoforms&#xA;gene.iso.df$width &amp;lt;- width(fasta.s)&#xA;&#xA;## split data.frame into list with entry for each gene&#xA;gene.iso.df.split &amp;lt;- split(gene.iso.df,gene.iso.df$gene)&#xA;&#xA;## optional to keep all the information but really you just need indices&#xA;##gene.iso.df.split.best &amp;lt;- lapply(gene.iso.df.split,function(x) x[order(x$width)[1],])&#xA;&#xA;## pull out the longest isoform ID for each gene (in case of a tie just take the first one)&#xA;best.id &amp;lt;- sapply(gene.iso.df.split,function(x) row.names(x)[order(x$width)[1]])&#xA;&#xA;## subset your original reads with the subset&#xA;fasta.s.best &amp;lt;- fasta.s[best.id]&#xA;&#xA;## export new fastafile containing longest isoform per gene&#xA;writeXStringSet(fasta.s, filepath='sample_best_isoform.fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="383" LastActivityDate="2017-06-09T10:51:42.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="608" PostTypeId="1" AcceptedAnswerId="622" CreationDate="2017-06-09T15:08:40.710" Score="7" ViewCount="845" Body="&lt;p&gt;I start with a sorted and indexed bam file (&quot;mapped.bam&quot;) representing the mapping of small reads on a reference genome, and a bed file (&quot;genes.bed&quot;) containing the coordinates of a set of features of interest (let's say they are genes), for which I want to compute an average profile using programs from &lt;a href=&quot;http://deeptools.readthedocs.io/en/latest/content/list_of_tools.html&quot; rel=&quot;noreferrer&quot;&gt;deeptools&lt;/a&gt;. &lt;strong&gt;I would like to understand the steps involved to be sure of what the vertical axis of the final profile represents.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;First step: making a bigwig file&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I create a bigwig file (&quot;mapped.bw&quot;) from the bam file using &lt;code&gt;bamCoverage&lt;/code&gt; as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bamCoverage -b mapped.bam -bs 10 -of=bigwig -o mapped.bw&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The help of &lt;code&gt;bamCoverage&lt;/code&gt; says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The coverage is calculated as the number of reads per bin, where bins are short consecutive counting windows of a defined size.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In my case, the bins are 10 bp long. My reads are longer than that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a given bin, a given read can:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;completely overlap the bin&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;overlap the bin on n bp, n &amp;lt; 10&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;not overlap the bin at all&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Please correct me if I'm wrong: My guess it that the read is counted as 1 in cases 1. and 2., and 0 otherwise, and I also suppose that a read can be counted for several successive bins if it is long enough.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Second step: averaging over genes and plotting&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I compute a &quot;meta profile matrix&quot; (&quot;mapped_on_genes.gz&quot;) using &lt;code&gt;computeMatrix scale-regions&lt;/code&gt; as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;computeMatrix scale-regions \&#xA;    -S mapped.bw \&#xA;    -R genes.bed \&#xA;    --upstream 300 \&#xA;    --unscaled5prime 500 \&#xA;    --regionBodyLength 2000 \&#xA;    --unscaled3prime 500 \&#xA;    --downstream 300 \&#xA;    -out mapped_on_genes.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(There is a &lt;code&gt;-bs&lt;/code&gt; parameter which default value is 10 according to the help of the command.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use this to plot a profile using &lt;code&gt;plotProfile&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plotProfile -m mapped_on_genes.gz \&#xA;    -out mapped_on_genes_meta_profile.pdf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I obtain a profile in with values on the y axis. In what units are these values?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guess is the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the upstream (300 bp) and internal 5-prime (500 bp), since the bin size was the same in &lt;code&gt;bamCoverage&lt;/code&gt; and &lt;code&gt;computeMatrix&lt;/code&gt;, each point on the x axis probably represents a 10 bp window, and its y coordinate is the average over the regions present in the bed file of the corresponding bins in the bigwig file, so it is an average number of reads overlapping a 10 bp bin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Same thing at the 3-prime and downstream side.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the central 100 bp portion, before averaging over regions some shrinking or spreading of the bins must have been performed, I guess by averaging between neighbouring bins. So the final unit is still a &lt;strong&gt;number of reads overlapping a 10 bp bin&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And if I use larger bins, I should end up with proportionally higher values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Am I correct?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2017-06-09T15:30:37.097" LastActivityDate="2017-06-09T17:04:19.710" Title="What unit do I get on the y-axis of a metagene profile plot?" Tags="|hts|deeptools|bigwig|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="609" PostTypeId="1" AcceptedAnswerId="632" CreationDate="2017-06-09T15:09:55.103" Score="6" ViewCount="85" Body="&lt;p&gt;I have two different sets of internal standards in a sample (1), and a sample (2) with these internal standards as well as about 30 peptides. Each peptide has one of each of these kind of standards. From sample 1, I can get the true ratio between standard 1 (S1) and standard 2 (S2), and from sample 2 I can get the ratio between the analyte (L) to each of the standards:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sample 1: S1/S2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sample 2: L/S1, L/S2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can thus calculate the ratio S1/S2 from the measured ratios in sample 2. Each sample is run in 5 technical replicates (that is; 5 injections into the LC-mass spectrometer), so S1/S2 ratios for one particular analyte could look like this (mock data):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;measured    calculated&#xA;0.967       0.987&#xA;1.007       0.967&#xA;1.044       1.012&#xA;1.041       1.025&#xA;1.048       1.046&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to compare the measured ratios to the calculated, to see if there are any bias in my acquisition of data. As the calculated and the measured ratio are from different samples, the replicates aren't &quot;linked&quot; so I could not use for example Bland Altman analysis. Should I use some kind of ANOVA? Or do you have any other recommendation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also like to make an overall assessment of the ratios of all 30 peptides to see if the bias is different for different peptides. How could I do that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any input would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="796" LastEditorUserId="57" LastEditDate="2017-06-09T22:15:09.710" LastActivityDate="2017-06-10T15:40:21.290" Title="How do I compare measured ratios to calculated ratios in peptide mass spectrometry?" Tags="|proteins|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="610" PostTypeId="1" CreationDate="2017-06-09T15:37:49.977" Score="6" ViewCount="877" Body="&lt;p&gt;I'm thinking that this isn't actually possible, but I'd like to check before I write it off.  I have RNA-seq data from cells under three different conditions, there are no replicates for any of the conditions.  The data has been processed with RSEM, and log2 fold changes have been calculated for each control-test pairing using the normalized expected read counts using EBseq.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If possible, I'd like to also calculate the p-value for each of these fold-changes, however, because there are no replicates I don't think that this is possible.  Is there any way to do this?&lt;/p&gt;&#xA;" OwnerUserId="506" LastActivityDate="2017-06-09T15:53:02.983" Title="Is it possible to calculate p-values for fold changes of single replicate RNA-seq samples?" Tags="|ebseq|rsem|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="611" PostTypeId="1" AcceptedAnswerId="618" CreationDate="2017-06-09T15:51:24.883" Score="10" ViewCount="291" Body="&lt;p&gt;The &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;noreferrer&quot;&gt;SAM specification&lt;/a&gt; indicates that each read group must have a unique ID field, but does not mark any other field as required. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also discovered that htsjdk throws exceptions if the sample (SM) field is empty, though there is no indication in the specification that this is required. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there other read group fields that I should expect to be required by common tools? &lt;/p&gt;&#xA;" OwnerUserId="272" LastEditorUserId="73" LastEditDate="2017-06-09T16:52:01.640" LastActivityDate="2017-06-09T16:52:01.640" Title="What are the de facto required fields in a SAM/BAM read group?" Tags="|file-formats|sam|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="612" PostTypeId="2" ParentId="610" CreationDate="2017-06-09T15:53:02.983" Score="3" Body="&lt;p&gt;DESeq2 is able to adjust p-values for single-replicate samples by estimating shared dispersion across all samples. DESeq2 will give a warning, but try its best to carry out the analysis. More information on doing that can be found in the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#can-i-use-deseq2-to-analyze-a-dataset-without-replicates&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2 vignette&lt;/a&gt; and the DESeq2 manual page.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The results will not be anywhere near the sensitivity that can be achieved when replicate information is available, but it might be good enough if only very obvious large-scale changes are of importance.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-09T15:53:02.983" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="613" PostTypeId="2" ParentId="611" CreationDate="2017-06-09T16:01:18.067" Score="3" Body="&lt;p&gt;The read group identifier needs to be specified in both the header lines of the BAM/SAM file and the alignment line. No other fields are required, but note that because the additional information is only stored once (i.e. in the header), it won't add much to file sizes or processing time if additional fields are included. If any of the other optional fields cause problems (either by inclusion or exclusion), it would be helpful to &lt;a href=&quot;https://github.com/samtools/htsjdk/issues&quot; rel=&quot;nofollow noreferrer&quot;&gt;report an issue&lt;/a&gt; about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Older tools required you to specify both a header read group ID and an alignment read group ID, but most tools of the recent tools I've used seem to be intelligent enough that they will add headers for any read groups without that information.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-09T16:01:18.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="614" PostTypeId="5" CreationDate="2017-06-09T16:08:03.673" Score="0" Body="&lt;p&gt;The GFF3 formal specification is available at &lt;a href=&quot;https://github.com/The-Sequence-Ontology/Specifications/blob/master/gff3.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/The-Sequence-Ontology/Specifications/blob/master/gff3.md&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2017-06-09T17:20:16.950" LastActivityDate="2017-06-09T17:20:16.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="615" PostTypeId="4" CreationDate="2017-06-09T16:08:03.673" Score="0" Body="Generic Feature Format version 3, a common tab-delimited plain-text format for annotating genes and other genomic features." OwnerUserId="96" LastEditorUserId="96" LastEditDate="2017-06-15T13:53:54.950" LastActivityDate="2017-06-15T13:53:54.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="616" PostTypeId="5" CreationDate="2017-06-09T16:16:17.263" Score="0" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.ensembl.org/info/website/upload/gff.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl spec&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://mblab.wustl.edu/GTF22.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;WUSTL spec&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2017-06-09T17:20:38.977" LastActivityDate="2017-06-09T17:20:38.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="617" PostTypeId="4" CreationDate="2017-06-09T16:16:17.263" Score="0" Body="Gene Transfer Format, a derivative of GFF, a tab-delimited plain-text format for describing gene annotations." OwnerUserId="96" LastEditorUserId="96" LastEditDate="2017-06-15T13:53:49.963" LastActivityDate="2017-06-15T13:53:49.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="618" PostTypeId="2" ParentId="611" CreationDate="2017-06-09T16:28:09.460" Score="9" Body="&lt;p&gt;The sample tag (i.e. SM) was a mandatory tag in the &lt;a href=&quot;https://sourceforge.net/p/samtools/code/HEAD/tree/trunk/sam-spec/&quot; rel=&quot;noreferrer&quot;&gt;initial SAM spec&lt;/a&gt; (see the &lt;code&gt;.pages&lt;/code&gt; file; you need a mac to open it). When transitioned to Latex, this requirement was mysteriously dropped. Picard is conforming to the initial spec. Anyway, the sample tag is important to quite a few tools. I would encourage you to add it.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-06-09T16:28:09.460" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="619" PostTypeId="2" ParentId="587" CreationDate="2017-06-09T16:28:26.360" Score="3" Body="&lt;p&gt;The main concern has always been the &quot;binning&quot; of quality scores that occurs via CRAM compression (and is also standard on the HiSeqX, HiSeq4000, and NovaSeq platforms). Anecdotally, I can report very little difference between 4-bin quality scores and full quality scores on cancer samples, though I don't know if I've seen a direct head-to-head comparison. &lt;/p&gt;&#xA;" OwnerUserId="74" LastActivityDate="2017-06-09T16:28:26.360" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="620" PostTypeId="1" CreationDate="2017-06-09T16:40:08.390" Score="7" ViewCount="1744" Body="&lt;p&gt;Is there a computational tool for measuring what percentage of RNA is spliced in an RNAseq experiment?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not particularly interested in complicated analyses that give ratios for all possible alternative splicing variations. I'd rather have a binary classification. For example: (RNA1 = 35% spliced, 65% unspliced)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit: (Additional information)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We're very early in the exploratory stage, so a specific hypothesis is still forming. But we're interested in the sequence content of lncRNAs, many of which are poorly spliced. So we want to get a sense for how prevalent unspliced version of these transcript are. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because this is currently exploratory, estimations and noise are fairly acceptable. A very basic approach might be to calculate the ratio of reads that map to exon-to-exon junctions, verses those that map from exon-to-intro junctions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more advanced approach I'm also considering is &lt;a href=&quot;https://combine-lab.github.io/salmon/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Salmon&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="800" LastEditorUserId="800" LastEditDate="2017-06-09T18:33:13.550" LastActivityDate="2017-06-13T19:14:44.307" Title="Spliced vs. unspliced ratios for transcripts in RNA-seq data" Tags="|rna-seq|rna-splicing|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="621" PostTypeId="2" ParentId="620" CreationDate="2017-06-09T16:48:33.833" Score="2" Body="&lt;p&gt;Take this first comment with a grain of salt, since this isn't an area I've worked in much, but: is binary classification possible? If a gene has 3 introns, and 2 are spliced out but 1 is retained, is this &quot;spliced&quot; or &quot;unspliced&quot;. My first impression is that an analysis would be a bit more nuanced than a binary classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I'm aware of a tool called Keep Me Around for intron retention analysis (&lt;a href=&quot;https://arxiv.org/abs/1510.00696&quot; rel=&quot;nofollow noreferrer&quot;&gt;preprint&lt;/a&gt;, &lt;a href=&quot;https://github.com/pachterlab/kma&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;). I've never used this software, but it looks like it's actively maintained and was created by a research group with a lot of experience and influence in this area, for whatever that's worth. :-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;: It looks like the original post was modified since I submitted my answer, but I'll leave the text about binary classification around for posterity. :-)&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2017-06-09T16:48:33.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="622" PostTypeId="2" ParentId="608" CreationDate="2017-06-09T17:04:19.710" Score="4" Body="&lt;p&gt;Feel free to @ me in deepTools questions, since I'm the primary developer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a given bin, the count assigned to it is the number of reads overlapping it, regardless of whether they overlap by 1 or 10 bases. So a read overlapping only partially and one overlapping completely are treated the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since your bigWig file is in units of &quot;alignments&quot; (i.e., it's not 1x normalized), the resulting profile will also be in units of &quot;alignments&quot; (i.e., profiles and heatmaps are in whatever units the input files are in).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Upstream/downstream regions and unscaled regions are also 10 base bins. Note that these are then the average of the per-base value, since the bins here may not perfectly correspond to the bins in the bigWig files. The line in the profile plot is indeed the average (by default, you can choose median, max, min, etc.) of the underlying regions for each bin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the scaled section in the middle, the the number of genomic bases per bin is changed such that the region will have &lt;code&gt;&quot;length&quot;/(regionBodyLength/binSize)&lt;/code&gt; bases each. As above, the per-base value is then averaged (or whatever you specify) to derive the per-bin value. The &lt;code&gt;length&lt;/code&gt; here is decreased if you have unscaled regions, since otherwise bases would get counted twice.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-06-09T17:04:19.710" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="623" PostTypeId="1" CreationDate="2017-06-09T18:17:58.273" Score="7" ViewCount="249" Body="&lt;p&gt;I am using &lt;strong&gt;PythonCyc&lt;/strong&gt; API in order to write a query for metabolites in &lt;strong&gt;BioCyc&lt;/strong&gt;. The purpose of this API is to communicate with the database software of BioCyc- Pathway Tools. Pathway Tools is in lisp therefore, PythonCyc creates a bridge between python and common lisp. To use this this API one must first create a PGDB object with a specified organism ID (orgid). In the example below I create a PGDB with orgid &quot;meta&quot;. After this, I am able to call methods from PythonCyc with this object:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import sys&#xA;import pythoncyc&#xA;#this creates PGDB object associated with meta(MetaCyc)&#xA;meta = pythoncyc.select_organism('meta')&#xA;#lists pathways of specified compound&#xA;pathways = meta.pathways_of_compound('sucrose')&#xA;print pathways&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The query for the metabolite above, 'sucrose', provides a list a pathways:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[u'|PWY-7347|', u'|SUCSYN-PWY|', u'|PWY-7238|', u'|PWY-6527|', u'|PWY-5343|', u'|PWY-3801|', u'|PWY-7345|', u'|PWY-862|', u'|SUCUTIL-PWY|', u'|PWY66-373|', u'|PWY-621|', u'|PWY-822|', u'|SUCROSEUTIL2-PWY|', u'|PWY-6525|', u'|PWY-6524|', u'|PWY-5337|', u'|PWY-5384|']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However if I switch this metabolite name to a common amino acid name, such as 'valine':&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import sys&#xA;import pythoncyc&#xA;#this creates PGDB object associated with meta(MetaCyc)&#xA;meta = pythoncyc.select_organism('meta')&#xA;#lists pathways of specified compound&#xA;pathways = meta.pathways_of_compound('valine')&#xA;print pathways&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The query provides an error message that states &quot;non coercible frame&quot;, which means it cannot find this ID. This is a common metabolite, just as sucrose, which should very well have pathway entries in database, however there are none found with this method. I have also tried synonyms of valine and other methods such as &quot;all_pathways(cpds)&quot; stated in API, which give me the same error message.&#xA;What is the reason for this? Does BioCyc not list any pathways for amino acids? Or am I using an incorrect method to access amino acid information?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://pythoncyc.readthedocs.io/en/latest/&quot; rel=&quot;noreferrer&quot;&gt;Link to PythonCyc API&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="803" LastActivityDate="2017-07-17T23:46:37.833" Title="What methods should I use from PythonCyc API to query metabolites in BioCyc database?" Tags="|python|api|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="624" PostTypeId="2" ParentId="453" CreationDate="2017-06-09T20:27:11.420" Score="1" Body="&lt;p&gt;I wrote a command-line (C++14) tool called &lt;code&gt;subset&lt;/code&gt; which is up on Github: &lt;a href=&quot;https://github.com/alexpreynolds/subset&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/alexpreynolds/subset&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This should be reasonably memory efficient and fast. The &lt;code&gt;subset&lt;/code&gt; tool does not store input lines in a table, but instead streams through the file once, storing a 4 or 8k buffer chunk of the input file (depending on OS). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It stores line numbers in an array, but eight bytes per integer * 100k is 800kB, for that use case — not very much memory there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's an O(nlogn) sort penalty on the line number array, but again this list will be much smaller than the query file, and integer sorting is fairly optimized, so the hit should be small. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your line number list is already sorted, I could add an option to skip sorting; let me know if that would be useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The filtering step walks through the line number array and input file linearly, printing lines where there are index matches, and skipping over the rest. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed, &lt;code&gt;subset&lt;/code&gt; will quit early in parsing the input file if there are no more line numbers to query. So this feature is especially useful for speeding up filtering of very large query files. (If your query file has 1M rows, say, and your last line number of interest is 12345, there's no reason to read through the rest of the file.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can grab, build and install it like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ git clone https://github.com/alexpreynolds/subset.git&#xA;$ cd subset&#xA;$ make&#xA;$ cp subset /usr/local/bin&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Once the binary is in your path, there are a couple ways to use it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, you can specify a start index and length value. The following grabs seven lines starting with the 33rd line (32 as a 0-indexed value):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ subset --prefix-with-indices -s 32 -n 7 -i query.txt &amp;gt; answer.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or you can specify a text file containing line numbers, each on a separate line. The following reads in a file called &lt;code&gt;line-numbers.txt&lt;/code&gt; and uses that to filter &lt;code&gt;query.txt&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ subset --prefix-with-indices -l line-numbers.txt -i query.txt &amp;gt; answer.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The indices in &lt;code&gt;line-numbers.txt&lt;/code&gt; should be positive, 0-indexed integers. The list of numbers does not need to be sorted, as &lt;code&gt;subset&lt;/code&gt; will sort the list of numbers for you. This is so that an efficient single pass through the input/query file can be done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can leave out &lt;code&gt;--prefix-with-indices&lt;/code&gt; to leave out the debug prefix. This is there so that you can do a sanity check on the result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;test/makefile&lt;/code&gt; tests demonstrate options and usages for the two types of filtering.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2017-06-10T01:29:46.010" LastActivityDate="2017-06-10T01:29:46.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="625" PostTypeId="2" ParentId="296" CreationDate="2017-06-10T03:52:26.343" Score="2" Body="&lt;p&gt;My bet is that the nanopore folks have (of course!) done a ton of optimisation on two key things:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The DNA cleanup&lt;/li&gt;&#xA;&lt;li&gt;The library preps&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I think that for most DNA extractions cleanups can vary a lot, but something like a Blue Pippin will do an exceptional job if you have access to one and can use it (they won't work for SUPER long DNA fragments which can't move through a gel of course). The Blue Pippin also does your size selection, so you don't burn out pores sequencing lots of little stuff. This should help yield too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then it's all down to the library prep, and @gringer's answer has some good tips on that. &lt;/p&gt;&#xA;" OwnerUserId="156" LastEditorUserId="73" LastEditDate="2017-06-10T05:01:59.620" LastActivityDate="2017-06-10T05:01:59.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="626" PostTypeId="1" AcceptedAnswerId="629" CreationDate="2017-06-10T03:57:26.523" Score="11" ViewCount="2129" Body="&lt;p&gt;I have a large phylogenomic alignment of &amp;gt;1000 loci (each locus is ~1000bp), and &amp;gt;100 species. I have relatively little missing data (&amp;lt;10%).&lt;/p&gt;&#xA;&lt;p&gt;I want to estimate a maximum-likelihood phylogenetic tree from this data, with measures of statistical support on each node.&lt;/p&gt;&#xA;&lt;p&gt;There are many phylogenetics programs that claim to be able to analyse datasets like this (e.g. RAxML, ExaML, IQtree, FastTree, PhyML etc). Given that I have access to a big server (512GB RAM, 56 cores), what are the pros and cons of each program.&lt;/p&gt;&#xA;&lt;p&gt;Which programme likely to give me the most accurate estimate of the ML tree for a dataset of this size within a reasonable timeframe?&lt;/p&gt;&#xA;" OwnerUserId="156" LastEditorUserId="-1" LastEditDate="2021-05-09T10:06:13.610" LastActivityDate="2021-05-09T10:06:13.610" Title="What is the best method to estimate a phylogenetic tree from a large dataset of &gt;1000 loci and &gt;100 species" Tags="|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="627" PostTypeId="1" AcceptedAnswerId="630" CreationDate="2017-06-10T10:41:51.353" Score="4" ViewCount="126" Body="&lt;p&gt;I've been processing data for a consortium project that uses the &lt;a href=&quot;https://github.com/zhanxw/rvtests&quot; rel=&quot;nofollow noreferrer&quot;&gt;rvtests&lt;/a&gt; toolkit. Our data analysis process uses a kinship matrix and the calculation of inverse-normalised transformed statistics after covariate adjustment (for 1kG-imputed data from a SNPchip). One of the operations we have been asked to do is to analyse males and females separately for a number of different phenotypes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example of the command line that I'm running:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rvtest --inVcf chr${ch}.imputed.poly.vcf.gz \&#xA; --pheno ../scripts/phenotypes.raw.2017-May-01.ped \&#xA; --pheno-name ${traitName} \&#xA; --out rv_results/${traitName}_FEMALE_chr${ch} \&#xA; --kinship kinship_matrix.kinship \&#xA; --meta score,cov[windowSize=500000] \&#xA; --peopleIncludeFile ../scripts/female.iids.txt \&#xA; --covar ../scripts/covar.2017-May-11.ped \&#xA; --covar-name ${covs} --useResidualAsPhenotype --dosage DS&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the kinship matrix doesn't interact well with the &lt;code&gt;--peopleinclude&lt;/code&gt; parameter. When I run this, the program complains about unexpected columns in the kinship matrix (starting at one column greater than the number of females in the dataset). I suspect what is happening is that rvtests is assuming that the kinship matrix applies to the sample set &lt;em&gt;after&lt;/em&gt; filtering, rather than applying to the total sample set. This sets off a few warning bells in my head, because it means that rvtest is probably ignoring the column/row labels in the kinship matrix that it has generated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I work around this problem?&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-09-08T15:14:45.393" Title="How can I combine a kinship matrix with subset individuals when using rvtests?" Tags="|microarray|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="628" PostTypeId="1" AcceptedAnswerId="2045" CreationDate="2017-06-10T11:14:30.313" Score="5" ViewCount="302" Body="&lt;p&gt;I used a script in R language that uses nnet library to predict promoter bacteria and i would like to know how to extract rules from this neural network results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As my input of the neural network i have n positive examples of promoter and n false examples. I use as input these examples and the FASTA file with the genome of the bacteria. As a result, i have a value of 0 to 1 for each network tested corresponding to its learning. I want to discover roles that can improve my network in future experiments in the same bacteria.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which algorithms or softwares could i use?&lt;/p&gt;&#xA;" OwnerUserId="813" LastEditorUserId="96" LastEditDate="2017-06-10T19:20:57.900" LastActivityDate="2017-07-11T15:01:38.167" Title="Rule Extraction from nnet results" Tags="|r|fasta|machine-learning|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="629" PostTypeId="2" ParentId="626" CreationDate="2017-06-10T11:58:10.863" Score="11" Body="&lt;p&gt;&lt;a href=&quot;http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0027731&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper claims&lt;/a&gt; that FastTree is almost as accurate as RAxML, while being much faster. You just have to be careful, however, that the support values output by FastTree are not bootstrap values, they are &lt;a href=&quot;http://www.microbesonline.org/fasttree/&quot; rel=&quot;nofollow noreferrer&quot;&gt;based on the Shimodaira-Hasegawa test&lt;/a&gt;. (Also, &lt;a href=&quot;http://darlinglab.org/blog/2015/03/23/not-so-fast-fasttree.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;see this comment&lt;/a&gt; for the case you have very short branch lengths).   [&lt;strong&gt;update:&lt;/strong&gt; However, according to &lt;a href=&quot;http://biorxiv.org/content/early/2017/05/25/142323&quot; rel=&quot;nofollow noreferrer&quot;&gt;the recent comparison paper mentioned below&lt;/a&gt; FastTree performed quite poorly in comparison to RAxML or IQ-tree.]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I understand, you should use ExaML only if your data is too large to be handled by RAxML in a single node. ExaML should perform like RAxML but with some parallelization overhead. For all effects I treat them as the same. I don't know of relevant advantages of phyML over RAxML (for me, it's easier to use but I am very used to phyML).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not familiar with IQ-tree, but &lt;a href=&quot;https://academic.oup.com/mbe/article-lookup/doi/10.1093/molbev/msu300&quot; rel=&quot;nofollow noreferrer&quot;&gt;its authors claim&lt;/a&gt; that even given the same time as RAxML or phyML, IQ-tree already finds better likelihoods more often than not (although by default it takes a bit longer to converge). &lt;a href=&quot;http://biorxiv.org/content/early/2017/05/25/142323&quot; rel=&quot;nofollow noreferrer&quot;&gt;A recent comparison between all these programs&lt;/a&gt; favoured IQ-TREE for both single-gene and concatenation analyses (with RAxML very close). It may also estimate branch support through a SH-like test only, but I'm not sure.   [&lt;strong&gt;update:&lt;/strong&gt; IQ-tree offers 3 measures of support, standard bootstrap, aLRT, and ultrafast bootstrap. See OP's comment below for details.]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, since you have few missing data, you might also want to try a single-locus tree inference followed by gene tree clustering (using &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1111/1755-0998.12676/abstract&quot; rel=&quot;nofollow noreferrer&quot;&gt;treescape&lt;/a&gt; or &lt;a href=&quot;https://academic.oup.com/mbe/article-lookup/doi/10.1093/molbev/msw038&quot; rel=&quot;nofollow noreferrer&quot;&gt;treeCL&lt;/a&gt;) to see how spread your data is, or to see the effect of removal of outliers, or to use ideas similar to &lt;a href=&quot;http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0129183&quot; rel=&quot;nofollow noreferrer&quot;&gt;statistical binning&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="45" LastEditorUserId="298" LastEditDate="2017-10-27T16:10:32.497" LastActivityDate="2017-10-27T16:10:32.497" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="630" PostTypeId="2" ParentId="627" CreationDate="2017-06-10T13:12:47.700" Score="2" Body="&lt;p&gt;Adam Locke (a collaborator of this project) suggests that removing covariate information for the unselected individuals (i.e. setting it to NA) works around this problem:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I believe the problem is that he is using a pre-computed kinship matrix including both males and females, and when using the “—peopleIncludeFile” it can’t properly select the right people from the kinship matrix. I don’t know that others have tried to do analysis this way. An alternative, that I think has worked for others, is to generate phenotype variables that have values only for males or females (so, e.g., BMI, BMI_females, BMI_males). For whatever reason, this appears to correctly only use the and select the samples desired without the kinship matrix.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Presumably a kinship matrix that is generated for just the subset individuals will work as well, although I don't like the idea that rvtests isn't able to do a label lookup on the kinship matrix to find the correct individuals.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-10T13:12:47.700" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="631" PostTypeId="2" ParentId="628" CreationDate="2017-06-10T14:14:17.493" Score="7" Body="&lt;p&gt;R's nnet package only supports fully connected neural networks with one hidden layer. This is the most primitive type of network. I doubt it will work well for promotors finding. In addition, such network won't give you useful interpretations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to explore neural networks, you should use a one-dimension convolution layer as is described in &lt;a href=&quot;http://www.nature.com/nbt/journal/v33/n8/full/nbt.3300.html&quot; rel=&quot;noreferrer&quot;&gt;this paper&lt;/a&gt;. This layer effectively represents a positional weight matrix (PWM). You can know which motifs are used. With backtracking, you can also identify the coordinates of motifs. To deploy such a model, you need to be fairly familiar with deep learning and to learn one deep learning framework. For framework, you may start with &lt;a href=&quot;https://keras.io&quot; rel=&quot;noreferrer&quot;&gt;keras&lt;/a&gt;. Once you get used to keras, you can implement your full promotor finder in less than 100 lines of python code. Alternatively, you may try &lt;a href=&quot;http://kundajelab.github.io/dragonn/&quot; rel=&quot;noreferrer&quot;&gt;dragonn&lt;/a&gt;. It is supposed to simplify deploying models for DNA sequences. I have no experience with it, though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For promotor finding, it is also worth trying traditional methods. It is interesting that few/no neuralNet-based publications have evaluated traditional methods, probably because many ML people know little about classical motif finding. These methods could work well, too.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-06-10T14:14:17.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="632" PostTypeId="2" ParentId="609" CreationDate="2017-06-10T15:40:21.290" Score="2" Body="&lt;p&gt;From a statistical perspective ratios are icky because the variance of the sampling distribution of a ratio decreases with the size of the numerator and denominator being sampled to create the ratio. If you can get at the numerator and denominator separately, then you should absolutely do that and look at count based tests like the Chi-square test, Fisher's exact test, or Poisson regression. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, a t-test would be appropriate in the two-sample case if its assumptions are met.  You need to check that both samples are normally distributed, ideally via a qq-plot. You also need to check the variance of both samples and choose a variant of the t-test that does or does not assume equal variances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the samples are not normally distributed, then the next approach would be a &lt;a href=&quot;https://en.wikipedia.org/wiki/Mann%E2%80%93Whitney_U_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mann-Whitney U&lt;/a&gt; test.&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2017-06-10T15:40:21.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="633" PostTypeId="1" AcceptedAnswerId="828" CreationDate="2017-06-10T20:24:19.480" Score="3" ViewCount="41" Body="&lt;p&gt;I have seen one of &lt;a href=&quot;https://discover.nci.nih.gov/cellminer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cellminer tools&lt;/a&gt;. I am not sure how do they calculate the cross correlation of the genes, what does it actually mean? Based on what databases? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example if I take their example input (&lt;a href=&quot;http://www.genenames.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HUGO&lt;/a&gt; format): &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;abcb1&#xA;BRCA2&#xA;CNBP&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get the following cross correlation matrix (based on what signals).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Identifier      abcb1   BRCA2   CNBP&#xA;abcb1           1      -0.142   0.069&#xA;BRCA2          -0.142   1       0.176&#xA;CNBP           0.069    0.176   1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2017-06-11T05:26:04.453" LastActivityDate="2017-06-20T08:34:17.377" Title="How does Cellminer's &quot;Cross-correlations of transcripts, drugs, and microRNAs&quot; work" Tags="|statistics|gene|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="634" PostTypeId="2" ParentId="604" CreationDate="2017-06-10T23:09:04.270" Score="2" Body="&lt;p&gt;You can see the exact assembly by looking in the README file in Ensembl's genome download page. &lt;a href=&quot;ftp://ftp.ensembl.org/pub/release-89/fasta/homo_sapiens/dna/README&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ensembl.org/pub/release-89/fasta/homo_sapiens/dna/README&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see the current assembly is GCA_000001405.25. Ensembl 79 uses version GCA_000001405.17. You can expect some differences between assembly versions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you take a look at &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly?term=GRCh38&amp;amp;cmd=DetailsSearch&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/assembly?term=GRCh38&amp;amp;cmd=DetailsSearch&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that 20,466,394 bp have been added since 17 -&gt; 25. This is only about 0.6% change so I would guess there is not that much difference between these versions. However, if I were you I would use the corresponding version of Ensembl. Even better would be using the latest version of Ensembl, as not only the genomes primary sequence but also the positions of genes can change between versions (&lt;a href=&quot;https://doi.org/10.1093/bib/bbw017&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1093/bib/bbw017&lt;/a&gt;). &lt;/p&gt;&#xA;" OwnerUserId="269" LastActivityDate="2017-06-10T23:09:04.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="637" PostTypeId="1" CreationDate="2017-06-10T23:27:13.180" Score="8" ViewCount="288" Body="&lt;p&gt;I read lecture notes about &lt;a href=&quot;http://mcb.berkeley.edu/courses/mcb142/lecture%20topics/Dernburg/Lecture6_Chapter8_screenviewing.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;mutations&lt;/a&gt;, and wondered what kind of algorithms are there to detect mutations?&lt;/p&gt;&#xA;&lt;p&gt;How do we know if the gene is mutated or whether it is a sequencing error?&lt;/p&gt;&#xA;&lt;p&gt;I saw &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/98/how-to-quickly-determine-mutations-in-a-read-of-a-sam-file&quot;&gt;this&lt;/a&gt; thread which is related, but I am not sure how to use the CIGAR information, e.g., is it 100% accurate? What is the underline mechanism to detect mutation? Is there a way to predict what the mutation will cause?&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2021-01-05T16:59:51.013" LastActivityDate="2021-01-05T16:59:51.013" Title="How to detect a mutation and predict its consequence?" Tags="|variant-calling|gene|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="638" PostTypeId="2" ParentId="637" CreationDate="2017-06-10T23:58:21.667" Score="9" Body="&lt;p&gt;&lt;strong&gt;Part 1 : how to detect mutations&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The keywords you are searching for are &quot;variant calling&quot;. Basically you have to map sequencing reads to a reference genome (or gene) and then estimate for each position of the genome if the observed difference of mapped reads and the reference is more likely a sequencing error or a mutation (in genomic glossary - variant). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Popular tools for variant calling are &lt;a href=&quot;https://software.broadinstitute.org/gatk/&quot; rel=&quot;noreferrer&quot;&gt;GATK&lt;/a&gt;, &lt;a href=&quot;https://github.com/ekg/freebayes&quot; rel=&quot;noreferrer&quot;&gt;FreeBayes&lt;/a&gt; or &lt;a href=&quot;https://samtools.github.io/bcftools/howtos/variant-calling.html&quot; rel=&quot;noreferrer&quot;&gt;bcftools&lt;/a&gt; (previously part as Samtools package).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question you linked asks for quick alternatives to simply looking for variants. Indeed, you can just visualise the mapped reads to the reference sequence and see if the variant is there or not. CIGAR is just a notation of read alignment used in sam files (files with mapped reads), you can find good explanation of CIGAR strings &lt;a href=&quot;http://genome.sph.umich.edu/wiki/SAM&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The follow-up about &quot;How to estimate an effect of mutation&quot; is in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/644/292&quot;&gt;@DevonRyan 's awesome answer&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="292" LastEditDate="2017-06-12T11:36:07.287" LastActivityDate="2017-06-12T11:36:07.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="639" PostTypeId="1" AcceptedAnswerId="641" CreationDate="2017-06-11T01:28:57.607" Score="17" ViewCount="2051" Body="&lt;p&gt;What are the advantages of having Bioconductor, for the bioinformatics community?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've read the '&lt;a href=&quot;https://bioconductor.org/about/&quot; rel=&quot;noreferrer&quot;&gt;About&lt;/a&gt;' section and skimmed the &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/gb-2004-5-10-r80&quot; rel=&quot;noreferrer&quot;&gt;paper&lt;/a&gt;, but still cannot really answer this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand Bioconductor is released twice a year (unlike &lt;code&gt;R&lt;/code&gt;), but if I want to use the latest version of a package, I'll have to use the dev version anyway. A stamp of &lt;em&gt;approval&lt;/em&gt; could be achieved much easier with a tag or something, so it sounds just like an extra (and unnecessary) layer to maintain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related to this, what are the advantages as a developer to have your package accepted into Bioconductor?&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2017-06-12T18:31:09.837" Title="Why Bioconductor?" Tags="|r|bioconductor|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="641" PostTypeId="2" ParentId="639" CreationDate="2017-06-11T07:13:30.900" Score="18" Body="&lt;p&gt;&lt;strong&gt;Benefits of central repository for Community&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having a central repository for packages is very useful. For couple of reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It makes very easy to resolve &lt;strong&gt;dependencies&lt;/strong&gt;. Installing all the dependencies manually would be exhausting but also dangerous (point 2).&lt;/li&gt;&#xA;&lt;li&gt;Package &lt;strong&gt;compatibility&lt;/strong&gt;! If I install package with dependencies, I would like to be sure that I install correct versions of all the dependencies.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Reliability&lt;/strong&gt; thanks to unified and integrated testing. &lt;code&gt;Bioconductor&lt;/code&gt; is trying really hard to force developers to write good test, they also have people manually testing submitted packages. They also remove packages that are not maintained. Packages in &lt;code&gt;Bioconductor&lt;/code&gt; are (reasonably) reliable.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In the end, installing dev versions of R packages is in my opinion &lt;strong&gt;very bad practise&lt;/strong&gt; for reproducible science. If developers delete GitHub repo, commit hash you have used won't be enough to get the code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Benefits of central repository for developers&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I forgot about the advantages for you as developer to submit your package to &lt;code&gt;Bioconductor&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Your package will be more visible&lt;/li&gt;&#xA;&lt;li&gt;users will have a guarantee that your code was checked by third person&lt;/li&gt;&#xA;&lt;li&gt;Your package will be for users easier to install&lt;/li&gt;&#xA;&lt;li&gt;Your package will be forced to use standardized vignettes, version tags and tests -&gt; will be more accessible by community to build on your code&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Bioconductor specific advantages over CRAN&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see the big advantage in the &lt;a href=&quot;https://support.bioconductor.org/&quot; rel=&quot;noreferrer&quot;&gt;community support page&lt;/a&gt;, provided by &lt;code&gt;Bioconductor&lt;/code&gt;. &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/643/57&quot;&gt;@Llopis' comprehensive elaboration&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2017-06-12T18:31:09.837" LastActivityDate="2017-06-12T18:31:09.837" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="642" PostTypeId="2" ParentId="639" CreationDate="2017-06-11T10:09:41.653" Score="3" Body="&lt;p&gt;Regarding what the advantage is to you as a developer of having a bioconductor package rather than using CRAN:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;There's a hierarchy of package quality, with bioconductor on the top (followed by CRAN and then &quot;random github repos&quot;). While there are many excellent CRAN packages, the average bioconductor package is better tested and documented. So if I as a user have two different packages that I can use and one is on bioconductor and the other CRAN, then I use the bioconductor package.&lt;/li&gt;&#xA;&lt;li&gt;Higher visibility. Since bioconductor packages are held in higher regard, they also become more visible. Further, the different &quot;views&quot; (e.g., &quot;Alternative Splicing&quot; and &quot;Transcription&quot;) make it convenient to find relevant packages. You can always search CRAN, but allowing tagging like this aids in discovery.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-06-11T10:09:41.653" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="643" PostTypeId="2" ParentId="639" CreationDate="2017-06-11T10:13:24.243" Score="10" Body="&lt;p&gt;Here is a list of the advantages of having Bioconductor for the bioinformatic community:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Outreach&lt;/strong&gt;: You have a &lt;strong&gt;repository for the field&lt;/strong&gt;, in that language. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Some packages related to bioinformatics (in R) are distributed through personal repositories, CRAN, github, bitbucket, sourceforge, but they are less used and harder to find.&lt;br&gt;&#xA;There are such efforts in other languages too: Biopython, Bioperl, Biojava, ...&lt;br&gt;&#xA;Also is harder to find the repositories related to a subject in CRAN, you don't have the BiocViews, the equivalent is optional and not usually filled,  which is quite useful when looking for a method.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Quality&lt;/strong&gt;: In Bioconductor each package is tested in Linux, Windows, and iOS, to make sure it works in all major operative systems (with all the dependencies). &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In some rare cases like &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/Rsubread.html&quot; rel=&quot;noreferrer&quot;&gt;this one&lt;/a&gt;, a packages is not supported for certain platform, but you can known it by checking the &lt;a href=&quot;http://bioconductor.org/checkResults/release/bioc-LATEST/Rsubread/&quot; rel=&quot;noreferrer&quot;&gt;build report&lt;/a&gt;.&lt;br&gt;&#xA;You are required to provide a vignette and examples in every exported element (and the vignette, examples and tests should pass). You are required to be able to install the package with stricter quality than CRAN, because there is a manual review (They pointed out a &lt;em&gt;comment&lt;/em&gt; in one of my functions!).&lt;br&gt;&#xA;They also provide docker images of the base packages. You don't need to install the latest R version to use the Bioconductor! But developers do so (at least when checked by Bioconductor servers) to ensure that the package will keep working in next R release. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Reusing&lt;/strong&gt;: Bioconductor provides the basic elements to a big number of applications. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For example the summarizedExperiment class is provided so that any package that needs a similar object can (should) use it. Or GSEABase is the base package to deal with GSEA enrichment analysis, providing functions, methods and classes for gene sets, and collections, making easier for anyone to create their own GSE analysis.&lt;br&gt;&#xA;It is easier to build upon the work of others if you know you are following the same quality standards. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Support&lt;/strong&gt;: To support Q&amp;amp;A is mandatory, the package maintainer must be registered in the &lt;a href=&quot;https://support.bioconductor.org&quot; rel=&quot;noreferrer&quot;&gt;webpage&lt;/a&gt;. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;While in CRAN usually the support is given by each package in its own way, in Bioconductor you can directly reach the maintainer and the users by posting in the same central place. &lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-06-11T22:00:10.837" LastActivityDate="2017-06-11T22:00:10.837" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="644" PostTypeId="2" ParentId="637" CreationDate="2017-06-11T10:15:56.960" Score="10" Body="&lt;p&gt;I'll follow up to the &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/638/292&quot;&gt;great answer&lt;/a&gt; from &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/57/kamil-s-jaron&quot;&gt;Kamil S Jaron&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding predicting what the variant (&quot;mutation&quot; is a very loaded term) will do, there are a variety of tools. Chief among these are &lt;a href=&quot;http://annovar.openbioinformatics.org/en/latest/&quot; rel=&quot;noreferrer&quot;&gt;annovar&lt;/a&gt; and &lt;a href=&quot;http://www.ensembl.org/info/docs/tools/vep/index.html&quot; rel=&quot;noreferrer&quot;&gt;VEP&lt;/a&gt;. The general idea behind these is to classify the variants according to their overlap with genes, which codons they change (if any), how big that change is (e.g., changes in charge are more likely detrimental) and so on. One could also consider conservation, since if a position is highly conserved then changes in it are more likely to be detrimental.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really want to predict how a variant will change a protein's function then that usually requires prior knowledge about the proteins in question. Eventually someone will use machine learning to cull the literature and provide good predictions, but I haven't seen that yet.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="48" LastEditDate="2017-08-14T09:51:31.817" LastActivityDate="2017-08-14T09:51:31.817" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="645" PostTypeId="2" ParentId="639" CreationDate="2017-06-11T10:18:55.697" Score="2" Body="&lt;p&gt;The most important reason is that Bioconductor has growing set of common data structures and base packages. If package X and package Y needs to work with the same type of data, having a common data structure in the core Bioconductor package Z makes our lives so much easier. I could do something in package X, take out the results and keep working on my data with package Y. This works because I'm using versions of package X and Y that are compatible with the data structure defined in package Z. The Bioconductor team makes sure that all packages use the common data structures and already existing packages where possible, so that we're all on the same page and so that people don't reinvent the wheel again and again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, there's a sophisticated process around &lt;a href=&quot;https://www.bioconductor.org/developers/package-submission/&quot; rel=&quot;nofollow noreferrer&quot;&gt;getting your package accepted to Bioconductor&lt;/a&gt;. This ensures that packages use data structures and functions already available in other Bioconductor packages (where reasonable), it ensures that packages are well written, that they have good documentation, and that they are well tested.&lt;/p&gt;&#xA;" OwnerUserId="283" LastEditorUserId="283" LastEditDate="2017-06-12T10:22:33.963" LastActivityDate="2017-06-12T10:22:33.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="646" PostTypeId="1" AcceptedAnswerId="647" CreationDate="2017-06-11T13:49:45.317" Score="8" ViewCount="259" Body="&lt;p&gt;I have two kinds of interactions: transient and stable. We are supposed to work on stable interaction, like interactions between two monomers in a heterodimer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a heterodimer there are two chains in which there are some residues between monomers which take part in the interaction of monomers and building the structure of heterodimer, and there are other residues interacting inside each monomer for building the structure of that monomer. There are some PDB IDs related to heterodimers in PDB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now imagine that we are trying to show which residues are physically interacting between two monomers in a heterodimer, and then we are trying two show the site of these physically interacting residues in the sequence related to that special structure. For that, we need to find the sequence related to that structure and we can download it from PDB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if we want to make a multiple sequence alignment of that sequence, first we need to find the refseq sequence of that and then we must blast the sequence and find homologues of that sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here there are some challenges:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;we must find residues interacting in the structure of heterodimer between two monomers.&lt;/li&gt;&#xA;&lt;li&gt;we must find the sequences of monomer chains of the heterodimer and map the interacting residues in the structure to the sequence.&lt;/li&gt;&#xA;&lt;li&gt;we know that when a structure is solved and its monomers are sequenced, may be they are not able to completely sequence the heterodimer and residue numbers are maybe different from the related refseq sequence. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I would like to know how can we find the related refseq sequence (or sequences, in the case of heterodimer) and then how can we map the physically interacting residues in the structure of PDB ID to refseq sequence (finding the sites of these interacting residues in that refseq sequence) regarding three challenges mentioned above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine that I have 100 PDB IDs of 100 nonredundant heterodimers and I would like to find and download the structures, sequence and finally refseq sequence. What should I do? &lt;/p&gt;&#xA;" OwnerUserId="818" LastEditorUserId="73" LastEditDate="2017-06-13T02:55:06.973" LastActivityDate="2017-12-22T14:48:11.110" Title="Getting structure and sequence related to PDB IDs" Tags="|protein-structure|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="647" PostTypeId="2" ParentId="646" CreationDate="2017-06-11T14:55:08.643" Score="9" Body="&lt;p&gt;You can download seqs and structures based on a list of PDB ids using &lt;a href=&quot;http://www.rcsb.org/pdb/download/download.do#FASTA&quot; rel=&quot;noreferrer&quot;&gt;http://www.rcsb.org/pdb/download/download.do#FASTA&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="640" LastActivityDate="2017-06-11T14:55:08.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="649" PostTypeId="2" ParentId="81" CreationDate="2017-06-11T20:27:23.053" Score="0" Body="&lt;p&gt;Another next-gen read simulation tool is &lt;a href=&quot;https://popmodels.cancercontrol.cancer.gov/gsr/packages/gemsim/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gemsim&lt;/a&gt;.  I haven't tested it, but I would be interested if anyone has had any experience with it.&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2017-06-11T20:27:23.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="650" PostTypeId="1" AcceptedAnswerId="670" CreationDate="2017-06-11T20:33:33.147" Score="1" ViewCount="717" Body="&lt;p&gt;I am going to describle the problem completely. First I should say that there are two kinds of interactions: transient and stable interactions. We are supposed to work on stable interaction like interactions between two monomers in a heterodimer. In a heterodimer there are two chains in which there are some residues between monomers which take part in the interaction of monomers and building the structure of heterodimer and also there are other residues interacting inside each monomer for building the structure of that monomer. There are some PDB IDs related to heterodimers in PDB. Now imagine that we are trying to show which residues are physically interacting between two monomers in a heterodimer and then we are trying two show the site of these physically interacting residues in the sequence related to that special structure. For that, we need to find the sequence related to that structure and we can download it from PDB. But if we want to make a multiple sequence alignment of that sequence, first we need to find the refseq sequence of that and then we must blast the sequence and find homologues of that sequence. Here there are some challenges: first, we must find residues interacting in the structure of heterodimer between two monomers. Second, we must find the sequences of monomer chains of the heterodimer and map the interacting residues in the structure to the sequence. Third, we know that when a structure is solved and its monomers are sequenced, may be they are not able to completely sequence the heterodimer and residue numbers are maybe different from the related refseq sequence. I would like to know how can we find the related refseq sequence (or sequences, in the case of heterodimer) and then how can we map the physically interacting residues in the structure of PDB ID to refseq sequence (finding the sites of these interacting residues in that refseq sequence) regarding three challenges mentioned above. Imagine that I have 100 PDB IDs of 100 nonredundant heterodimers and I would like to find and download the structures, sequence and finally refseq sequence. What should I do?&lt;/p&gt;&#xA;" OwnerUserId="818" LastActivityDate="2017-06-12T22:39:40.763" Title="Finding residues physically interacting in a PDB structure and sequence related to that structure" Tags="|protein-structure|" AnswerCount="1" CommentCount="1" ClosedDate="2017-06-13T12:36:21.717" ContentLicense="CC BY-SA 3.0" />
  <row Id="651" PostTypeId="2" ParentId="623" CreationDate="2017-06-11T21:57:03.280" Score="3" Body="&lt;p&gt;&lt;strong&gt;UPD&lt;/strong&gt;: As &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/623/what-methods-should-i-use-from-pythoncyc-api-to-query-metabolites-in-biocyc-data/651?noredirect=1#comment3035_651&quot;&gt;suggested&lt;/a&gt; by Llopis, maybe the correct identifier is VAL?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think there is no metabolite with such name in the database. I tried &lt;a href=&quot;https://biocyc.org/ECOLI/search-query?type=COMPOUND&amp;amp;name=Valine&quot; rel=&quot;nofollow noreferrer&quot;&gt;their search&lt;/a&gt;, the correct name seems to be &lt;code&gt;L-valine&lt;/code&gt;. I expect that they use similar naming for other amino acids. If you cannot find something, you can use their compound search through the website. &lt;/p&gt;&#xA;" OwnerUserId="191" LastEditorUserId="191" LastEditDate="2017-07-13T11:02:02.967" LastActivityDate="2017-07-13T11:02:02.967" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="653" PostTypeId="1" CreationDate="2017-06-12T06:08:10.227" Score="4" ViewCount="157" Body="&lt;p&gt;I would like to visualize &lt;a href=&quot;http://interactome.dfci.harvard.edu/H_sapiens/index.php?page=shownetwork&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; interaction list.  Is there an online/web based way to do it? Is there a way to analyze the data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming I have exported the list into lines such as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;2  A2M 348 APOE&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2019-11-14T13:31:31.663" LastActivityDate="2019-11-14T13:57:17.357" Title="What are the standard ways to visualize protein-protein or gene-gene interactions?" Tags="|gene|proteins|interactions|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="654" PostTypeId="1" AcceptedAnswerId="656" CreationDate="2017-06-12T07:49:43.730" Score="9" ViewCount="2823" Body="&lt;p&gt;&lt;em&gt;This is a problem I have to solve frequently, and I'd be interested in knowing what other methods people use to solve the same problem.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About twice a year, I get asked to determine variants from Illumina reads, usually from either mouse or human. These are things that have good reference genomes (e.g. &lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Info/Index&quot; rel=&quot;noreferrer&quot;&gt;human genome from ensembl&lt;/a&gt;, &lt;a href=&quot;http://www.ensembl.org/Mus_musculus/Info/Index&quot; rel=&quot;noreferrer&quot;&gt;mouse genome from ensembl&lt;/a&gt;), and are frequently tested out on various bioinformatics tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have at my disposal three computers, in order of usage preference: a laptop with 4 processing threads and 8GB of memory, a desktop with 12 processing threads and 64GB of memory, and a server with 24 processing threads and 256GB of memory. Occasionally I get access to better computers when I need to do lots of processing, or to do things quickly, but these comprise my basic bioinformatics team.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've had whole-genome projects that involved processing data from 4 individuals, up to projects with about 120 individuals, with each individual having about 10-40M paired-end reads (100-125bp each). Beyond this level, I suspect other researchers will typically have their own bioinformatics team and/or capabilities, which is why I don't get any of the larger-scale jobs that I hear about from salaried bioinformaticians.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the standard approach that you would use to process these reads into reference-mapped variant data (i.e. gzipped VCF) for use in downstream analysis tools (e.g. &lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Info/Index&quot; rel=&quot;noreferrer&quot;&gt;VEP&lt;/a&gt;)?&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="1157" LastEditDate="2020-07-08T09:19:56.533" LastActivityDate="2022-06-29T06:02:39.947" Title="How do I generate a variant list (i.e. VCF file) using Illumina reads from a human genome?" Tags="|variant-calling|read-mapping|human-genome|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="655" PostTypeId="2" ParentId="653" CreationDate="2017-06-12T08:34:12.630" Score="2" Body="&lt;p&gt;since I am not yet allowed to comment, I will have to pots this as an answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess Cytoscape (&lt;a href=&quot;http://cytoscape.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cytoscape.org/&lt;/a&gt;) would be a nice way to analyse such interaction lists/networks (still depends a bit on what you plan to do).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for this you will have to preprocess the downloadable excel sheet a bit.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;save as tab-separated file (tsv)&lt;/li&gt;&#xA;&lt;li&gt;run the below python3 script to format into an easily importable format for cytoscape. You can run the script as &lt;code&gt;python3 script.py &amp;lt;path-to-tsv-file&amp;gt; &amp;lt;delimeter for cytoscape&amp;gt;&lt;/code&gt; . The new format is written to std out on the console, therefore maybe you need to write the output to a new file. A sample call could look like &lt;code&gt;python3 testbench.py human_interactome_2017-06-12.tsv.csv &quot;;&quot;&lt;/code&gt;. Make sure to add the delimeter in quotes.&lt;/li&gt;&#xA;&lt;li&gt;you can then import the new file into cytoscape (import network as table) where the first column is your source, the third column your target and the middle column your interaction type.&lt;/li&gt;&#xA;&lt;li&gt;Within cytoscape you can then adapt your edge style to vary the line type in a discrete mapping depending on the value of the interaction.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Screenshot from cytoscape:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/FlSN5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FlSN5.png&quot; alt=&quot;Cytoscape screenshot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import sys&#xA;&#xA;HIIIidx = 4&#xA;HII5idx = 5&#xA;venkatesanidx = 6&#xA;yuidx = 7&#xA;Litidx = 8&#xA;&#xA;print(sys.argv)&#xA;&#xA;if len(sys.argv) &amp;lt;= 1:&#xA;    print(&quot;Usage: script.py file delimeter&quot;)&#xA;&#xA;delim = ';'&#xA;inFile = sys.argv[1]&#xA;&#xA;if len(sys.argv) &amp;gt; 2:&#xA;    delim = sys.argv[2]&#xA;&#xA;&#xA;def printInteraction(syma, symb, inter):&#xA;    print(syma + delim + inter + delim + symb)&#xA;&#xA;&#xA;with open( inFile , 'r') as file:&#xA;&#xA;    file.readline()&#xA;&#xA;    for line in file:&#xA;&#xA;        aline = line.strip('\n').split('\t')&#xA;&#xA;        if aline[HIIIidx] == '1':&#xA;            printInteraction(aline[1], aline[3], 'HIII')&#xA;&#xA;        if aline[HII5idx] == '1':&#xA;            printInteraction(aline[1], aline[3], 'HII5')&#xA;&#xA;        if aline[venkatesanidx] == '1':&#xA;            printInteraction(aline[1], aline[3], 'VEN')&#xA;&#xA;        if aline[yuidx] == '1':&#xA;            printInteraction(aline[1], aline[3], 'YU')&#xA;&#xA;        if aline[Litidx] == '1':&#xA;            printInteraction(aline[1], aline[3], 'LIT')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="690" LastEditorUserId="690" LastEditDate="2017-06-12T12:59:17.563" LastActivityDate="2017-06-12T12:59:17.563" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="656" PostTypeId="2" ParentId="654" CreationDate="2017-06-12T08:41:12.713" Score="9" Body="&lt;h2&gt;1. Adapter Trimming&lt;/h2&gt;&#xA;&lt;p&gt;One of the first things I do after encountering a set of reads is to remove the adapter sequences from the start and end of reads. Most basecalling software includes some amount of built-in adapter trimming, but it is almost always the case that some adapter sequence will remain. Removing adapters is helpful for mapping because it reduces the effort the mapper needs to go through to make a match (which may help some borderline reads to be mapped properly).&lt;/p&gt;&#xA;&lt;p&gt;My preferred adapter trimmer is &lt;a href=&quot;http://www.usadellab.org/cms/?page=trimmomatic&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trimmomatic&lt;/a&gt;. It has the ability to search for palindromic adapter read through, and includes an adaptive sliding window option. Trimmomatic is threaded, and seems to work reasonably fast when run through files one at a time in threaded mode. Here's an example script that I might run to trim reads from some samples. This puts trimmed FASTQ files into a &lt;code&gt;trimmed&lt;/code&gt; subdirectory:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TRIMPATH=/data/all/programs/trimmomatic/Trimmomatic-0.36;&#xA;JARPATH=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${TRIMPATH}/trimmomatic-0.36.jar&quot;;&#xA;ADAPTfile=&quot;$&lt;/span&gt;{TRIMPATH}/adapters/TruSeq3-PE-2.fa&amp;quot;;&#xA;CMDS=&amp;quot; ILLUMINACLIP:${ADAPTfile}:2:30:10:5:true LEADING:3 TRAILING:3  SLIDINGWINDOW:10:20 MINLEN:40 &amp;quot;;&#xA;&#xA;mkdir -p trimmed;&#xA;&#xA;for INFILE1 in *_1.fastq.gz; do&#xA;  base=&lt;span class=&quot;math-container&quot;&gt;$(basename &quot;$&lt;/span&gt;INFILE1&amp;quot; _1.fastq.gz);&#xA;  echo &lt;span class=&quot;math-container&quot;&gt;${base};&#xA;  INFILE2=&quot;$&lt;/span&gt;{base}_2.fastq.gz&amp;quot;;&#xA;  OUTFILE_P1=&amp;quot;trimmed/&lt;span class=&quot;math-container&quot;&gt;${base}_P1.fastq.gz&quot;;&#xA;  OUTFILE_P2=&quot;trimmed/$&lt;/span&gt;{base}_P2.fastq.gz&amp;quot;;&#xA;  OUTFILE_U1=&amp;quot;trimmed/&lt;span class=&quot;math-container&quot;&gt;${base}_U1.fastq.gz&quot;;&#xA;  OUTFILE_U2=&quot;trimmed/$&lt;/span&gt;{base}_U2.fastq.gz&amp;quot;;&#xA;  java -jar &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${JARPATH}&quot; PE -threads 20 -phred33 \&#xA;        &quot;$&lt;/span&gt;{INFILE1}&amp;quot;  &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${INFILE2}&quot; \&#xA;        &quot;$&lt;/span&gt;{OUTFILE_P1}&amp;quot; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${OUTFILE_U1}&quot; &quot;$&lt;/span&gt;{OUTFILE_P2}&amp;quot;  &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${OUTFILE_U2}&quot; \&#xA;        &quot;$&lt;/span&gt;{CMDS}&amp;quot;;&#xA;done;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Trimmomatic is somewhat sensitive to bad input data, and will loudly fail (i.e. stop running) if it sees quality strings of different length to sequence strings:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Exception in thread &amp;quot;Thread-1&amp;quot; java.lang.RuntimeException: Sequence and quality length don't match: 'TACATGGCCCTGAAATGACTTTCACCCAGGCAACCAGTGCCCCCTGTATAGACACATGCCTTGGGCGCTCCCCACCCTTCCTCGCGTGGCCACACCTCTGT' vs '-AAFFJFJJ7A-FJJJFJFJFJJFFA-FFAF-&amp;lt;A-&amp;lt;FFFJA-&amp;lt;-A7-F&amp;lt;&amp;lt;FFJJAJJJJJJJJJ--&amp;lt;--7-7AJ7&amp;lt;AAJA--J7&amp;lt;ACTGCTGTGGGGCACCCAGCCCCCCAGATAGCCTGGCAGAAGGATGGGGGCACAGACTTCCCAGCTGCACGGGAGAGAC'&#xA;        at org.usadellab.trimmomatic.fastq.FastqRecord.&amp;lt;init&amp;gt;(FastqRecord.java:25)&#xA;        at org.usadellab.trimmomatic.fastq.FastqParser.parseOne(FastqParser.java:89)&#xA;        at org.usadellab.trimmomatic.fastq.FastqParser.next(FastqParser.java:179)&#xA;        at org.usadellab.trimmomatic.threading.ParserWorker.run(ParserWorker.java:42)&#xA;        at java.lang.Thread.run(Thread.java:745)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;2. Read Mapping&lt;/h2&gt;&#xA;&lt;p&gt;Read mapping finds the most likely location for a read within a target genome.&#xA;There are extremely fast approximate mappers available, but these don't yet work for variant calling, and an exact mapping approach is necessary. My current preferred mapper is &lt;a href=&quot;https://ccb.jhu.edu/software/hisat2/index.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;HISAT2&lt;/a&gt;. I use this instead of BWA due to the double-read issue, and because of the local variant-aware mapping. HISAT2 is made by the same computing group as Bowtie2 and Tophat2 (JHUCCB), and is their recommended tool for replacing those other programs.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;HISAT2 is a successor to both HISAT and TopHat2. We recommend that HISAT and TopHat2 users switch to HISAT2.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The HISAT2 page includes a link to a &lt;a href=&quot;ftp://ftp.ccb.jhu.edu/pub/infphilo/hisat2/data/grch38_snp.tar.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;genomic index file&lt;/a&gt; for the human genome, including SNP variants, which I use when mapping human reads. HISAT2 is also threaded, so I run it on the files one at a time and pipe through samtools to create a sorted BAM file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mkdir mapped&#xA;for r1 in trimmed/*_P1.fastq.gz;&#xA;  do base=&lt;span class=&quot;math-container&quot;&gt;$(basename &quot;$&lt;/span&gt;{x}&amp;quot; _P1.fastq.gz);&#xA;  r2=&amp;quot;trimmed/&lt;span class=&quot;math-container&quot;&gt;${base}_P2.fastq.gz&quot;;&#xA;  echo $&lt;/span&gt;{base};&#xA;  hisat2 -p 20 -t -x /data/all/genomes/hsap/hisat_grch38_snp/genome_snp -1 \&#xA;    &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${r1}&quot; -2 &quot;$&lt;/span&gt;{r2}&amp;quot; 2&amp;gt;&amp;quot;mapped/hisat2_&lt;span class=&quot;math-container&quot;&gt;${y}_vs_grch38_stderr.txt&quot; | \&#xA;    samtools sort &amp;gt; &quot;mapped/hisat2_$&lt;/span&gt;{y}_vs_grch38.bam&amp;quot;;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;3. Variant Calling&lt;/h2&gt;&#xA;&lt;p&gt;I currently use samtools/bcftools to do this, but would be interested in other opinions as we recently had a grant reviewer response that samtools was a dinosaur program and better approaches were available. Samtools doesn't currently work with threads for variant calling, so I save the commands to a text file and then run them through &lt;a href=&quot;https://www.gnu.org/software/parallel/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GNU Parallel&lt;/a&gt;. This requires the genome files to be first downloaded and indexed via &lt;code&gt;samtools faidx&lt;/code&gt;. This produces a set of gzipped VCF files in the &lt;code&gt;variants&lt;/code&gt; directory:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mkdir -p variants;&#xA;(for x in mapped/*.bam;&#xA;   do echo samtools mpileup -v -f /data/all/genomes/hsap/hisat_grch38_snp/Homo_sapiens.GRCh38.dna.primary_assembly.fa &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${x}&quot; \| \&#xA;   bcftools call --ploidy GRCh38 -v -m -O z -o variants/$&lt;/span&gt;(basename &amp;quot;${x}&amp;quot; .bam).vcf.gz; done) &amp;gt; call_jobs.txt&#xA;cat call_jobs.txt | parallel -u;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As a slow, but more accurate alternative, variants can be called for all samples at the same time:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools mpileup -v -f /data/all/genomes/hsap/hisat_grch38_snp/Homo_sapiens.GRCh38.dna.primary_assembly.fa mapped/*.bam | \&#xA;  bcftools call --ploidy GRCh38 -v -m -O z -o variants/hisat2_allCalled_vs_grch38.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If a faster processing is desired, then the variant calling can be parallelised across chromosomes using &lt;code&gt;parallel&lt;/code&gt; and merged afterwards using &lt;code&gt;bcftools norm&lt;/code&gt;. The implementation of this is left as an exercise to the reader.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Aside: I see that &lt;a href=&quot;https://software.broadinstitute.org/gatk/best-practices/workflow?id=11145&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATK's best practise guidelines&lt;/a&gt; call per-sample with HaplotypeCaller, then process those VCF files as aggregated data. @Geraldine-vdauwera says this two-step joint calling workflow was designed to be scalable for any size cohort, so you can apply it pretty much the same way whether you have two samples or twenty thousand. Once you go higher than that there are some additional tweaks to keep scaling up.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-06-29T06:02:39.947" LastActivityDate="2022-06-29T06:02:39.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="657" PostTypeId="2" ParentId="653" CreationDate="2017-06-12T09:26:50.953" Score="1" Body="&lt;p&gt;Usually interactions are represented in a graph, with edges representing the interactions (colors, thickness or values on the edges represent different properties of those edges).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know, there aren't on line programs to represent this kind of information. For an off line representation, you can use the &lt;a href=&quot;https://cran.r-project.org/package=igraph&quot; rel=&quot;nofollow noreferrer&quot;&gt;igraph&lt;/a&gt; package in R, or &lt;a href=&quot;http://www.cytoscape.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cytoscape&lt;/a&gt; program. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2017-06-12T09:26:50.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="658" PostTypeId="1" AcceptedAnswerId="718" CreationDate="2017-06-12T11:51:35.007" Score="8" ViewCount="1610" Body="&lt;p&gt;I'm looking for cloud computing services that can be used for doing bioinformatics. An example I found is &lt;a href=&quot;https://insidedna.me&quot; rel=&quot;noreferrer&quot;&gt;InsideDNA&lt;/a&gt; and there is &lt;a href=&quot;https://aws.amazon.com/health/genomics/&quot; rel=&quot;noreferrer&quot;&gt;Amazon&lt;/a&gt; of course.&#xA;A little description of these would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2017-08-15T17:50:38.403" Title="What are the available cloud computing services for bioinformatics?" Tags="|linux|" AnswerCount="7" CommentCount="10" ClosedDate="2017-11-02T00:41:33.210" ContentLicense="CC BY-SA 3.0" />
  <row Id="659" PostTypeId="2" ParentId="658" CreationDate="2017-06-12T12:51:03.997" Score="9" Body="&lt;p&gt;I have trialed the free version of InsideDNA, and these were my notes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Cost: \$225/month for a team of 5 with 50TB storage or \$45/month with 10TB storage for individuals (assuming 6 month package: &lt;a href=&quot;https://insidedna.me/pricing&quot; rel=&quot;noreferrer&quot;&gt;https://insidedna.me/pricing&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;li&gt;Software installed: Around 600 bioinformatic tools available and standard command line tools; some popular tools missing (like &lt;a href=&quot;http://weizhongli-lab.org/cd-hit/&quot; rel=&quot;noreferrer&quot;&gt;CD-HIT&lt;/a&gt;), but should be possible to install on request.&lt;/li&gt;&#xA;&lt;li&gt;Jobs: Maximum of 32 CPUs and 208 RAM per job submission. Test jobs generally worked, although a larger test job failed.&lt;/li&gt;&#xA;&lt;li&gt;Other points: Command line was sometimes slow, &lt;code&gt;wget&lt;/code&gt; queries were slow, and &lt;code&gt;scp&lt;/code&gt; was blocked. However, these may be resolvable issues.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Overall, I felt InsideDNA could be useful for groups without their own computational infrastructure and could be used for easily sharing resources between groups. The packages on offer seem not expensive, but I had a few issues, and I don't know how good their sys admin support would be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not used the Amazon service, so can't comment beyond the details on their website. Also there are a few alternative companies, such as &lt;a href=&quot;https://genestack.com/&quot; rel=&quot;noreferrer&quot;&gt;Genestack&lt;/a&gt; and &lt;a href=&quot;https://www.dnanexus.com/&quot; rel=&quot;noreferrer&quot;&gt;DNAnexus&lt;/a&gt;, but I haven't directly tested them either.&lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="240" LastEditDate="2017-08-03T06:09:39.177" LastActivityDate="2017-08-03T06:09:39.177" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="660" PostTypeId="2" ParentId="658" CreationDate="2017-06-12T13:56:06.117" Score="4" Body="&lt;p&gt;Depending on your applications and uses, you might be interested in checking out &lt;a href=&quot;http://www.cyverse.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;CyVerse&lt;/a&gt;. It is an NSF funded initiative that provides you with data storage, high performance computing resources, and easy access to commonly used tools. As far as I know, it is free to use once you have an account. I also usually encounter it being used with plant and microbial genomics, so not sure how it will work with something like human genomics projects. But might be worth checking out at least. :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More information: &lt;a href=&quot;http://www.cyverse.org/about&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cyverse.org/about&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="824" LastActivityDate="2017-06-12T13:56:06.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="661" PostTypeId="1" AcceptedAnswerId="665" CreationDate="2017-06-12T15:30:36.277" Score="4" ViewCount="345" Body="&lt;p&gt;I was wondering if I could use the gff parsing capability of bioawk to facilitate the parsing of gtf files, and I looked at the following help message:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bioawk -c help&#xA;bed:&#xA;    1:chrom 2:start 3:end 4:name 5:score 6:strand 7:thickstart 8:thickend 9:rgb 10:blockcount 11:blocksizes 12:blockstarts &#xA;sam:&#xA;    1:qname 2:flag 3:rname 4:pos 5:mapq 6:cigar 7:rnext 8:pnext 9:tlen 10:seq 11:qual &#xA;vcf:&#xA;    1:chrom 2:pos 3:id 4:ref 5:alt 6:qual 7:filter 8:info &#xA;gff:&#xA;    1:seqname 2:source 3:feature 4:start 5:end 6:score 7:filter 8:strand 9:group 10:attribute &#xA;fastx:&#xA;    1:name 2:seq 3:qual 4:comment &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I see that there are 10 fields defined for gff parsing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I look at the pages for gtf and gff in the ensembl website (&lt;a href=&quot;http://www.ensembl.org/info/website/upload/gff.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;gff/gtf&lt;/a&gt; and &lt;a href=&quot;http://www.ensembl.org/info/website/upload/gff3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;gff3&lt;/a&gt;), all have 9 fields.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm curious about these &quot;filter&quot; and &quot;group&quot; fields. What are they meant for? Are they extracted from some of the columns mentioned in the above pages?&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2017-06-13T07:02:39.247" Title="What kind of &quot;gff&quot; format does bioawk parse?" Tags="|file-formats|gff3|gtf|bioawk|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="662" PostTypeId="2" ParentId="594" CreationDate="2017-06-12T16:02:21.920" Score="2" Body="&lt;p&gt;Check out &lt;a href=&quot;http://homer.ucsd.edu/homer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HOMER&lt;/a&gt;. &quot;Software for motif discovery and next generation sequencing analysis&quot;, it's what my lab uses currently for finding eRNA motifs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: For @ShanZhengYang &quot;HOMER was designed as a de novo motif discovery algorithm...&quot; &#xA;&lt;a href=&quot;http://homer.ucsd.edu/homer/introduction/basics.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;HOMER De Novo Motif&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="33" LastEditorUserId="33" LastEditDate="2017-06-12T16:48:00.760" LastActivityDate="2017-06-12T16:48:00.760" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="663" PostTypeId="1" AcceptedAnswerId="666" CreationDate="2017-06-12T17:06:26.053" Score="7" ViewCount="839" Body="&lt;p&gt;I have a set of differentially methylated/expressed/whatever entities with p-values attached (example below).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;entity_name    p-value    magnitude&#xA;entity1        0.04459    0.68&#xA;entity2        0.02283    0.99&#xA;...&#xA;entity_n       0.78       0.025&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Typically, I apply the p.adjust function in R with the &quot;fdr&quot; (Benjamini-Hochberg) approach to leave me with p-values adjusted to control the FDR.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;adjusted &amp;lt;- p.adjust(mydata,&quot;fdr&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I am interested in showing a volcano plot with the unadjusted p-values, and two alpha levels: 0.05 and one that corresponds to the correction. What is the best way to get this alpha? Is it appropriate to set the &quot;corrected alpha&quot; to the lowest original p-value that doesn't pass FDR correction?&lt;/p&gt;&#xA;" OwnerUserId="643" LastEditorUserId="57" LastEditDate="2017-06-12T22:05:16.573" LastActivityDate="2017-06-13T11:14:15.507" Title="How to correct alpha, and not p-values themselves, for visualization purposes" Tags="|r|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="664" PostTypeId="1" CreationDate="2017-06-12T17:45:52.033" Score="7" ViewCount="3081" Body="&lt;p&gt;From an RNA-seq experiment I have about 17000 gene ids for 2 sample conditions arranged according to their log2 fold changes when compared to a control. I need to annotate these, but I've never done annotation before and am wondering how to do this in R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm interested in up and down regulation of the pathways of the involved genes. I'd love to get enrichment and p-value analysis too. I'm interested in whether or not a package similar to PANTHER exists in R for this analysis. I've been reading how different annotation programs can be better or worse; it seems PANTHER is much better than DAVID, so I was wondering if there is a package that provides PANTHER-like analyses in R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There seems to be multiple packages available, are there any that stand out as being the best?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm primarily interested in human samples and annotated pathways.&lt;/p&gt;&#xA;" OwnerUserId="506" LastEditorUserId="73" LastEditDate="2018-01-11T04:11:04.200" LastActivityDate="2018-05-03T22:28:29.710" Title="How to perform functional analysis on a gene list in R?" Tags="|r|sequence-annotation|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="665" PostTypeId="2" ParentId="661" CreationDate="2017-06-12T18:00:21.667" Score="5" Body="&lt;p&gt;I would consider the description there a bug. The &lt;code&gt;filter&lt;/code&gt; is actually the strand, &lt;code&gt;strand&lt;/code&gt; is the frame, &lt;code&gt;group&lt;/code&gt; is the attribute, and &lt;code&gt;attribute&lt;/code&gt; does nothing. These are really meant to be the 9 columns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: There's a &lt;a href=&quot;https://github.com/lh3/bioawk/issues/16&quot; rel=&quot;nofollow noreferrer&quot;&gt;bug report&lt;/a&gt; related to this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 2&lt;/strong&gt;: I've made &lt;a href=&quot;https://github.com/lh3/bioawk/pull/21&quot; rel=&quot;nofollow noreferrer&quot;&gt;a pull request&lt;/a&gt; to clarify this and fix the aforementioned bug report.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 3&lt;/strong&gt;: I realized that I never directly answered the title of your question (&lt;em&gt;mea culpa&lt;/em&gt;). bioawk itself will work with gff, gff3, or gtf files. It really is just treating them as tab-separated files with named columns (this is surprisingly convenient, since it's a PITA to remember what column does what).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 4&lt;/strong&gt;: The PR has been merged. If you install from github then you'll see corrected field names.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-06-13T07:02:39.247" LastActivityDate="2017-06-13T07:02:39.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="666" PostTypeId="2" ParentId="663" CreationDate="2017-06-12T18:22:42.213" Score="7" Body="&lt;p&gt;The only way to get the alpha levels is to determine what they will be with &lt;code&gt;p.adjust()&lt;/code&gt;, since they will depend on the distribution of your unadjusted p values. The general steps you should follow will be:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Add a column of adjusted p-values to your dataframe (&lt;code&gt;mydata$padj = p.adjust(mydata, method=&quot;BH&quot;)&lt;/code&gt;, which is the same as FDR and saves a character).&lt;/li&gt;&#xA;&lt;li&gt;Use &lt;code&gt;which&lt;/code&gt; and &lt;code&gt;max&lt;/code&gt; to determine your two alpha threshold (e.g., &lt;code&gt;max(mydata$pvalue[mydata$padj &amp;lt; 0.05])&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Then you can adjust your plots however you like (presumably with some horizontal lines at the various alphas). Whether you take the smallest non-significant value or the largest significant value is up to you, just describe what &quot;dots on the line&quot; represent.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-06-13T11:14:15.507" LastActivityDate="2017-06-13T11:14:15.507" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="667" PostTypeId="2" ParentId="658" CreationDate="2017-06-12T20:17:58.167" Score="1" Body="&lt;h3&gt;Google Genomics&lt;/h3&gt;&#xA;&lt;p&gt;Google has an API called &lt;a href=&quot;https://cloud.google.com/genomics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Google Genomics&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;SNPedia&lt;/h3&gt;&#xA;&lt;p&gt;&amp;quot;SNPedia is a wiki investigating human genetics.&amp;quot;&#xA;snpedia.com&lt;/p&gt;&#xA;&lt;h3&gt;Promethease&lt;/h3&gt;&#xA;&lt;p&gt;&amp;quot;Promethease is a literature retrieval system that builds a personal DNA report&amp;quot;&#xA;promethease.com&lt;/p&gt;&#xA;&lt;h3&gt;DNA Land&lt;/h3&gt;&#xA;&lt;p&gt;&amp;quot;Compare DNA with reference data from different populations&amp;quot;&#xA;dna.land&lt;/p&gt;&#xA;&lt;h3&gt;The CyDAS Project&lt;/h3&gt;&#xA;&lt;p&gt;And, there's the CyDAS project which has an API that can analyze ISCN formulae. Per their &lt;a href=&quot;http://www.cydas.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;web site&lt;/a&gt;: their API &amp;quot;lets you analyze a Karyotype for virtually all information which can be extracted from karyotypes and the rearrangements therein: gains and losses of chromosomal material, break points, junctions...&amp;quot; It's a &lt;strong&gt;free&lt;/strong&gt; service, but I don't know how up to date it is.&lt;/p&gt;&#xA;" OwnerUserId="831" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-06-13T19:07:49.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="668" PostTypeId="2" ParentId="663" CreationDate="2017-06-12T20:51:34.303" Score="3" Body="&lt;p&gt;When you're doing a p-value adjustment, the same unadjusted p-value in different genes can be given different adjusted p-values depending on other factors. That means that you can't directly draw a line associated with the FDR on a plot of unadjusted p-value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possibility would be to take a range of values that are close to the FDR threshold (e.g. the 20 values closest to threshold), and draw a p-value greyzone within that region:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/Rscript&#xA;values &amp;lt;- c(rnorm(10000),rnorm(100, mean=1.5));&#xA;val.mean &amp;lt;- median(values);&#xA;val.diffs &amp;lt;- abs(values - median(values));&#xA;val.reldiffs &amp;lt;- (values - median(values));&#xA;&#xA;val.pval &amp;lt;- pnorm(val.diffs, mean = mean(val.diffs),&#xA;                   sd=sd(val.diffs), lower.tail=FALSE);&#xA;val.padj &amp;lt;- p.adjust(val.pvals, method=&quot;BH&quot;);&#xA;fdr.threshold &amp;lt;- 0.1;&#xA;close.bh &amp;lt;- order(abs(val.padj - fdr.threshold))[1:20];&#xA;&#xA;png(&quot;SE.663.png&quot;);&#xA;plot(val.reldiffs, -log10(val.pval),&#xA;     col=ifelse(1:10100 &amp;lt;= 10000,&quot;darkblue&quot;,&quot;darkgreen&quot;));&#xA;abline(h=-log10(0.05), col=&quot;red&quot;);&#xA;text(0,-log10(0.05),&quot;p=0.05&quot;, pos=1);&#xA;abline(h=range(-log10(val.pval[close.bh])), col=&quot;#00000040&quot;, lty=&quot;dashed&quot;);&#xA;rect(xleft=min(val.reldiffs)*2, xright=max(val.reldiffs)*2,&#xA;     ytop=max(-log10(val.pval[close.bh])),&#xA;     ybottom=min(-log10(val.pval[close.bh])), col=&quot;#00000020&quot;, border=NA);&#xA;text(0,min(-log10(val.pval[close.bh])),&quot;FDR=0.1&quot;, pos=1);&#xA;invisible(dev.off());&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/l8Ldk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/l8Ldk.png&quot; alt=&quot;Volcano plot of p-values with FDR greyzone&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-13T10:21:37.260" LastActivityDate="2017-06-13T10:21:37.260" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="669" PostTypeId="2" ParentId="664" CreationDate="2017-06-12T21:52:41.197" Score="7" Body="&lt;p&gt;N.B., I'm avoiding discussion of &quot;best&quot;, since that's more or less impossible to answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your question can actually be divided into two:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What's a good tool for pathways analysis (ideally in R)?&lt;/li&gt;&#xA;&lt;li&gt;What are good sources of pathway information?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For (1), there are a few different possibilities, but I prefer either roast or camera from limma or goseq (a stand-alone bioconductor package). Of those, I expect what roast or goseq are doing are the most similar to what panther is doing in the sense that they're not competitive. However, since genes do correlate with each other, I prefer competitive tests like that offered by camera.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For (2), your options are basically &lt;a href=&quot;http://bioconductor.org/packages/release/data/annotation/html/KEGG.db.html&quot; rel=&quot;noreferrer&quot;&gt;KEGG&lt;/a&gt; or &lt;a href=&quot;http://bioconductor.org/packages/release/data/annotation/html/KEGG.db.html&quot; rel=&quot;noreferrer&quot;&gt;reactome&lt;/a&gt;. KEGG is problematic, since you need a license for anything remotely recent (there are some R packages to get around this, but I haven't a clue how kosher they are legally). Given that, reactome might be your best bet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having written all of that, the best database out there is what &lt;a href=&quot;https://www.qiagenbioinformatics.com/products/ingenuity-pathway-analysis/&quot; rel=&quot;noreferrer&quot;&gt;IPA&lt;/a&gt; uses. This is a commercial product (I have no affiliation), but you can get a demo license that will work for a couple weeks at least. If you're doing analyses relatively often then it makes sense to spend a bit of cash on a license (maybe share with neighboring groups).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regardless of what you use, you might also want to use &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/pathview.html&quot; rel=&quot;noreferrer&quot;&gt;pathview&lt;/a&gt;, which can produce easily interpretable plots related to pathway enrichment.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-06-12T21:52:41.197" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="670" PostTypeId="2" ParentId="650" CreationDate="2017-06-12T22:39:40.763" Score="2" Body="&lt;p&gt;My reputation isn't high enough to make a comment, but have you considered a modelling software such as PyMol or Chimera? You can execute a command line script with softwares such as these and these have the functionality to predict H-bonds. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the PDB files there is information regarding the GenBank and UNIPROT info of the protein, and you can find more information here: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.wwpdb.org/documentation/file-format-content/format33/sect3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.wwpdb.org/documentation/file-format-content/format33/sect3.html&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want RefSeq ID, you can use GenBank info you extract and convert by piping into a biomaRt (Bioconducor) script, although maybe there is a more direct way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for finding which residues in the sequence are interacting, I'm fairly certain that the software packages like PyMol or Chimera will give you information about the interacting residues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I understand, you will want to filter out only the residues that interact and are on different chains (e.g, Chain A vs. Chain B). This process will require you to work with the syntax of the different software packages when you are scripting. For example, Chimera has a basic primer here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.cgl.ucsf.edu/chimera/current/docs/ProgrammersGuide/basicPrimer.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cgl.ucsf.edu/chimera/current/docs/ProgrammersGuide/basicPrimer.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I only gave a reference to Chimera because I am more comfortable with that then other packages and there is likely a more efficient way to do this. With Chimera, you can find the position within the chain of residues participating in the interaction and then you'd store these in a file to do whatever you want later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="823" LastActivityDate="2017-06-12T22:39:40.763" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="671" PostTypeId="2" ParentId="595" CreationDate="2017-06-13T00:28:05.247" Score="3" Body="&lt;p&gt;Late to the party here, but I like to try to avoid writing scripts when some command line magic will do. It's good practice to index your &lt;code&gt;FASTA&lt;/code&gt; so use it.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools faidx &amp;lt;myfasta.fa&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A little bit of &lt;code&gt;awk&lt;/code&gt; and &lt;code&gt;sort&lt;/code&gt; can determine the largest isoform of each gene (and they don't even have to be sorted by name in the source &lt;code&gt;FASTA&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -F'[\t.]' '{print $1,$2,$3,$4}' &amp;lt;myfasta.fa&amp;gt;.fai | sort -k4nr,4 | sort -uk1,2 | cut -f1-3 -d' '| tr ' ' '.' &amp;gt; selection.ls&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;code&gt;-F[\t.]&lt;/code&gt; specifies both the &lt;code&gt;fai&lt;/code&gt; tab delimiters, and the full stops in your contig names (&lt;code&gt;Doug_NoIndex_L005_R1_001_contig_2.g7.t1&lt;/code&gt;) as delimiters for &lt;code&gt;awk&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;For each line we print four columns. &lt;code&gt;$1&lt;/code&gt; to &lt;code&gt;$3&lt;/code&gt; are the three components of your contig name (after splitting on the &lt;code&gt;.&lt;/code&gt;), and column $4 is the column in the index for sequence length.&lt;/li&gt;&#xA;&lt;li&gt;We use sort to organise the lines by &lt;code&gt;$4&lt;/code&gt; (the contig length). Reverse sort with &lt;code&gt;-r&lt;/code&gt; so the longest genes appear first.&lt;/li&gt;&#xA;&lt;li&gt;We then subsort by columns &lt;code&gt;$1&lt;/code&gt; and &lt;code&gt;$2&lt;/code&gt; (the contig and gene name (&lt;code&gt;g1&lt;/code&gt;...)). &lt;code&gt;-u&lt;/code&gt; then drops any repeat of column &lt;code&gt;$1&lt;/code&gt; and &lt;code&gt;$2&lt;/code&gt; pairs (that is, all lines after the first pair -- the longest length).&lt;/li&gt;&#xA;&lt;li&gt;Finally, we reassemble &lt;code&gt;$1&lt;/code&gt; to &lt;code&gt;$3&lt;/code&gt; with &lt;code&gt;cut&lt;/code&gt; and &lt;code&gt;tr&lt;/code&gt; to give you a list of sequence names.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A loop with our old friend &lt;code&gt;samtools faidx&lt;/code&gt; can put your chosen sequences out to a new &lt;code&gt;FASTA&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while read contig;&#xA;    do samtools faidx &amp;lt;myfasta.fa&amp;gt; $contig &amp;gt;&amp;gt; selection.fa;&#xA;done &amp;lt; selection.ls&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Don't forget to index it, they are useful!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools faidx selection.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="215" LastEditorUserId="215" LastEditDate="2017-06-13T00:51:03.047" LastActivityDate="2017-06-13T00:51:03.047" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="672" PostTypeId="1" AcceptedAnswerId="681" CreationDate="2017-06-13T01:33:49.053" Score="4" ViewCount="1149" Body="&lt;p&gt;My data is a VCF file generated from an exome sequencing variant call pipeline. I'm not very familiar with the sequencing and variant calling process. I noticed that there are some missing genotypes, which are recorded as &quot;./.&quot; at the GT field. From googling I learned that they're not homozygous reference genotype (&quot;0/0&quot;), but missing calls due to some sequencing failure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to make my data &quot;cleaner&quot;, I thought it would be better to filter out the loci with missing calls if there're not too many of them. I also checked the corresponding &quot;DP&quot; of the loci with and without missing calls. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1:123 GT:DP 0/0:2 1/1:4 ./.&#xA;chr1:234 GT:DP 0/0:10 1/1:11 1/1:20&#xA;chr1:345 GT:DP 0/1:40 1/1:37 0/0:78&#xA;chr1:456 GT:DP 0/1:7 0/0:23 ./.&#xA;chr1:567 GT:DP 0/1:34 1/1:39 0/0:58&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the above toy example, there are 3 people and 5 loci. I checked the mean DP of all 5 loci and I found that the mean DP of loci with missing calls (1st and 4th loci) to be significantly lower than the mean DP of loci without missing calls (2nd, 3rd and 5th loci). This happens to my real data with a lot more loci and samples than this toy example. Is that a coincidence? Or are there any specific reason that the loci with missing calls have lower coverage than the &quot;normal&quot; loci? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="832" LastEditorUserId="73" LastEditDate="2017-06-13T02:34:51.760" LastActivityDate="2017-06-13T19:21:01.190" Title="Why are there missing calls in a VCF file from exome sequencing?" Tags="|variant-calling|vcf|ngs|exome|reads|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="673" PostTypeId="1" CreationDate="2017-06-13T01:38:38.563" Score="7" ViewCount="509" Body="&lt;p&gt;I have a list of transcription factors and I am interested in finding out which which genes might be transcribed as a result of the formation of transcription factor complexes with that transcription factor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas on good databases? I've seen that ENCODE hosts data from CHIPseq experiments, but I'm not sure if I can find conclusions regarding interactions from this site. I'm looking for a database with known or putative transcription/gene interactions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="823" LastEditorUserId="823" LastEditDate="2017-06-13T23:23:39.460" LastActivityDate="2017-06-14T09:50:01.067" Title="Given a transcription factor, what genes does it regulate?" Tags="|chip-seq|encode|multi-omics|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="674" PostTypeId="2" ParentId="673" CreationDate="2017-06-13T01:53:41.920" Score="5" Body="&lt;p&gt;&lt;a href=&quot;http://software.broadinstitute.org/gsea/msigdb/collections.jsp#C3&quot; rel=&quot;noreferrer&quot;&gt;MSigDB&lt;/a&gt; has a collection (C3:TFT) of gene sets corresponding to transcription factor targets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://amp.pharm.mssm.edu/Harmonizome/&quot; rel=&quot;noreferrer&quot;&gt;Harmonizome&lt;/a&gt; has functional terms for genes extracted from over a hundred publicly available resources.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2017-06-13T01:53:41.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="675" PostTypeId="2" ParentId="672" CreationDate="2017-06-13T02:49:37.973" Score="3" Body="&lt;p&gt;I've just been generating data like this, so can tell you about why/how missing calls are created in my dataset. There are two main reasons:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;1. Sequencing failure&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;When reads don't map across the variant region, then it's impossible to accurately determine a genotype for that region. This will commonly happen just outside the borders of the selected regions for exome sequencing, but can also happen through natural random sampling of the genome, or through repeated sequences coupled with systematic error in either the assembled genome or the sequencer. In this case, it would be expected that called variant regions including missing data would be lower coverage. It may be helpful to plot the coverage within a particular region to work out if this is likely to be happening.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;2. Merged non-variant data&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Variant calling can be processed in two different modes, one which takes into account &lt;em&gt;all&lt;/em&gt; samples when doing calling, and another which does the calling one sample at a time, followed by subsequent merging of the results. Multi-sample variant calling is more accurate, but computational limits (or experimental design) might mean that it is more appropriate for samples to be called one-by-one. It's fairly common (for the purpose of preserving space and computational power) for calling algorithms to only report variant information so if a single sample is identical to the reference, and that sample is the only one being called, then the variant and coverage information for that sample will be lost. When samples are merged after being separately called, the declared call for the &quot;same as reference&quot; samples will be set to missing. In this case, it would be generally expected that the other called samples would have high-coverage within the variant region.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-13T19:21:01.190" LastActivityDate="2017-06-13T19:21:01.190" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="676" PostTypeId="1" CreationDate="2017-06-13T02:51:01.843" Score="1" ViewCount="1232" Body="&lt;p&gt;Are there any advantages to learning Biopython instead of learning Bioperl?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, we would learn both, but someone starting out in bioinformatics may have to choose what to learn first depending on the kind of problems actually encountered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Are there problems for which Biopython is better than Bioperl (or vice-versa)?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="823" LastEditorUserId="37" LastEditDate="2017-06-25T02:02:35.567" LastActivityDate="2017-06-25T02:02:35.567" Title="For what bioinformatics tasks is Biopython more adapted than Bioperl?" Tags="|python|perl|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="677" PostTypeId="1" CreationDate="2017-06-13T03:03:52.503" Score="7" ViewCount="46" Body="&lt;p&gt;For my undergraduate research I'm looking for a database that gives the bound form of a particular protein structure. Is there any database that provide us with such data? So far I've found following proteins related to my work &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ascorbate peroxidase (APX)&lt;/li&gt;&#xA;&lt;li&gt;Beta-glucosidase&lt;/li&gt;&#xA;&lt;li&gt;Calcineurin b-like protein-interacting protein kinase.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I've searched in PDB and UNIPROT. &lt;/p&gt;&#xA;" OwnerUserId="833" LastEditorUserId="175" LastEditDate="2017-06-13T04:08:37.613" LastActivityDate="2017-06-21T13:36:08.693" Title="how to find the bound form of an enzyme structure?" Tags="|database|protein-structure|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="678" PostTypeId="1" CreationDate="2017-06-13T03:48:15.283" Score="7" ViewCount="4810" Body="&lt;p&gt;I have a BAM file, and I have a read ID. What is the simplest way to get mapping statistics of that read in human-readable format?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g. I might want: % identity of aligned bases; number of insertions and deletions; number of bases aligned to the reference; etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Extra points for one-liners that give detailed summary statistics, and extra-extra points if you can extend the answer to output summaries for all reads longer than some predetermined length.&lt;/p&gt;&#xA;" OwnerUserId="156" LastActivityDate="2017-06-13T14:01:53.477" Title="Get the mapping statistics of a single read from a BAM file" Tags="|sam|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="679" PostTypeId="2" ParentId="677" CreationDate="2017-06-13T04:45:30.723" Score="4" Body="&lt;p&gt;Are you aware of &lt;a href=&quot;http://www.brenda-enzymes.org/index.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;BRENDA&lt;/a&gt;? I was just introduced to it today for a completely separate reason (looking at carbohydrate enzyme families in the &lt;em&gt;Nippostrongylus brasiliensis&lt;/em&gt; proteome), and it seems to be a fairly comprehensive database. There is at least a literature link there for the &lt;a href=&quot;http://www.brenda-enzymes.org/enzyme.php?ecno=1.11.1.11#Crystallization/COMMENTARY&quot; rel=&quot;nofollow noreferrer&quot;&gt;ascorbate-complexed crystalisation of Ascorbate peroxidase&lt;/a&gt; on that site. Following through the paper, I see that &lt;a href=&quot;http://www.rcsb.org/pdb/explore/explore.do?pdbId=1OAF&quot; rel=&quot;nofollow noreferrer&quot;&gt;the complex has been uploaded to PDB&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-13T04:45:30.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="680" PostTypeId="2" ParentId="678" CreationDate="2017-06-13T05:32:15.000" Score="7" Body="&lt;p&gt;&lt;code&gt;samtools stats&lt;/code&gt; seems to be able to do most of this, excluding the CIGAR-string parsing stuff (i.e. INDELs):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ samtools view -h mapped.bam | grep -e '^@' -e 'readName' |&#xA;    samtools stats | grep '^SN' | cut -f 2-&#xA;&#xA;raw total sequences:    2&#xA;filtered sequences: 0&#xA;sequences:  2&#xA;is sorted:  1&#xA;1st fragments:  2&#xA;last fragments: 0&#xA;reads mapped:   2&#xA;reads mapped and paired:    0   # paired-end technology bit set + both mates mapped&#xA;reads unmapped: 0&#xA;reads properly paired:  0   # proper-pair bit set&#xA;reads paired:   0   # paired-end technology bit set&#xA;reads duplicated:   0   # PCR or optical duplicate bit set&#xA;reads MQ0:  0   # mapped and MQ=0&#xA;reads QC failed:    0&#xA;non-primary alignments: 0&#xA;total length:   13632   # ignores clipping&#xA;bases mapped:   13632   # ignores clipping&#xA;bases mapped (cigar):   13502   # more accurate&#xA;bases trimmed:  0&#xA;bases duplicated:   0&#xA;mismatches: 3670    # from NM fields&#xA;error rate: 2.718116e-01    # mismatches / bases mapped (cigar)&#xA;average length: 6816&#xA;maximum length: 6816&#xA;average quality:    8.9&#xA;insert size average:    0.0&#xA;insert size standard deviation: 0.0&#xA;inward oriented pairs:  0&#xA;outward oriented pairs: 0&#xA;pairs with other orientation:   0&#xA;pairs on different chromosomes: 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For a length filter, an awk length check can be slotted in instead of the grep:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ samtools view -h mapped.bam |&#xA;    awk -F'\t' '{if((/^@/) || (length($10)&amp;gt;500)){print $0}}' |&#xA;    samtools stats | grep '^SN' | cut -f 2-&#xA;&#xA;raw total sequences:    131&#xA;filtered sequences: 0&#xA;sequences:  131&#xA;is sorted:  1&#xA;1st fragments:  131&#xA;last fragments: 0&#xA;reads mapped:   131&#xA;reads mapped and paired:    0   # paired-end technology bit set + both mates mapped&#xA;reads unmapped: 0&#xA;reads properly paired:  0   # proper-pair bit set&#xA;reads paired:   0   # paired-end technology bit set&#xA;reads duplicated:   0   # PCR or optical duplicate bit set&#xA;reads MQ0:  0   # mapped and MQ=0&#xA;reads QC failed:    0&#xA;non-primary alignments: 0&#xA;total length:   569425  # ignores clipping&#xA;bases mapped:   569425  # ignores clipping&#xA;bases mapped (cigar):   453884  # more accurate&#xA;bases trimmed:  0&#xA;bases duplicated:   0&#xA;mismatches: 113755  # from NM fields&#xA;error rate: 2.506257e-01    # mismatches / bases mapped (cigar)&#xA;average length: 4346&#xA;maximum length: 16909&#xA;average quality:    9.1&#xA;insert size average:    0.0&#xA;insert size standard deviation: 0.0&#xA;inward oriented pairs:  0&#xA;outward oriented pairs: 0&#xA;pairs with other orientation:   0&#xA;pairs on different chromosomes: 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-13T12:16:07.970" LastActivityDate="2017-06-13T12:16:07.970" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="681" PostTypeId="2" ParentId="672" CreationDate="2017-06-13T08:01:30.593" Score="4" Body="&lt;p&gt;Missing variant calls due to lack of coverage shouldn't happen in the targeted capture region and I'd think most of these would come from off-target regions where some samples had reads mapped. I'd filter out the VCF to only include on-target loci before proceeding with further analyses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you did your variant calling on samples separately and then merged them together, you'll have missing calls at loci called variant in one sample and homozygous reference in the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can get over this by asking the variant caller to emit all sites within the target region, which would lead to a large VCF or do multi-sample calling, which is more computationally intensive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One other way is to do variant calling on all samples separately, generating a BED file of all variant regions in the cohort and then asking the variant caller to genotype those loci across all samples before merging them together.&lt;/p&gt;&#xA;" OwnerUserId="601" LastActivityDate="2017-06-13T08:01:30.593" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="682" PostTypeId="2" ParentId="676" CreationDate="2017-06-13T08:12:58.063" Score="5" Body="&lt;p&gt;Regading the perl vs python discussion, there is no final answer which language is better, but I have some advice for you:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Learn the language your colleagues or your advisor use&lt;/strong&gt;. This way you are able to discuss your code with them and also get help if you run into problems.&lt;/p&gt;&#xA;" OwnerUserId="668" LastEditorUserId="73" LastEditDate="2017-06-13T13:09:37.473" LastActivityDate="2017-06-13T13:09:37.473" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="683" PostTypeId="2" ParentId="676" CreationDate="2017-06-13T08:44:30.520" Score="4" Body="&lt;p&gt;This usually comes down to religious issues, so let me try and steer it back to more objective grounds:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What language do you know (better)? Use the library for that one.&lt;/li&gt;&#xA;&lt;li&gt;If you know neither and will be learning a language to use the library, the majority opinion would be that Python is easier to learn.&lt;/li&gt;&#xA;&lt;li&gt;However, some people say that they &quot;click&quot; with Perl better&lt;/li&gt;&#xA;&lt;li&gt;Python is what the majority of bioinformaticians are using at the moment (about 60%). And there is a virtue and aid in using what everyone else is.&lt;/li&gt;&#xA;&lt;li&gt;What are your colleagues / collaborators going to be using?&lt;/li&gt;&#xA;&lt;li&gt;Broadly, newer libraries tend to be written in Python.&lt;/li&gt;&#xA;&lt;li&gt;Conversely, many sing the praises of regular expressions and string handling in Perl, citing speed and ease of use.&lt;/li&gt;&#xA;&lt;li&gt;I think the numerics support in Python may be markedly better than that in Perl.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="377" LastActivityDate="2017-06-13T08:44:30.520" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="684" PostTypeId="2" ParentId="673" CreationDate="2017-06-13T09:06:00.697" Score="4" Body="&lt;p&gt;&lt;a href=&quot;http://iregulon.aertslab.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;iRegulon&lt;/a&gt; takes a sequence-based approach to finding transcription factor targets. There's a Cytoscape app that you can use to find the regulators of a given gene list, or the targets of a particular transcription factor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Transcription factor binding sites are predicted using a collection of position weight matrices (PWMs) from &lt;a href=&quot;http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003731#pcbi-1003731-t001&quot; rel=&quot;nofollow noreferrer&quot;&gt;a number of sources&lt;/a&gt;, including Jaspar and TRANSFAC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;iRegulon: From a Gene List to a Gene Regulatory Network Using Large Motif and Track Collections PLoS Comput Biol. 2014 Jul 24;10(7):e1003731. doi: &lt;a href=&quot;http://dx,doi.org/10.1371/journal.pcbi.1003731&quot; rel=&quot;nofollow noreferrer&quot;&gt;10.1371/journal.pcbi.1003731&lt;/a&gt;. eCollection 2014 Jul.&lt;/p&gt;&#xA;" OwnerUserId="194" LastActivityDate="2017-06-13T09:06:00.697" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="685" PostTypeId="2" ParentId="673" CreationDate="2017-06-13T09:25:06.267" Score="4" Body="&lt;p&gt;In general, there is two options to identify targets for transcription factors: experimental (ChIP-seq) and sequence-based predictions.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;TF binding from experimental data&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The are multiple projects that produce binding data of transcription factors and quantify their peaks across the genome. The advantage here is that you know that binding actually occurs, as opposed to motif predictions. But you need the corresponding experiments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You already mentioned &lt;a href=&quot;https://www.encodeproject.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENCODE&lt;/a&gt;, which is probably the biggest producer of such data. NIH's &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/20709693&quot; rel=&quot;nofollow noreferrer&quot;&gt;Roadmap Epigenomics&lt;/a&gt; is another one, but that focusses less on TFs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a gene list and would like to know which transcription factor was likely involved, you can do an enrichment test of your gene set in known TF targets. The &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/20709693&quot; rel=&quot;nofollow noreferrer&quot;&gt;ChEA (ChIP Enrichment Analysis) database&lt;/a&gt; does this.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Prediction using motifs&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Another possibility is to look at &lt;a href=&quot;https://en.wikipedia.org/wiki/Sequence_motif&quot; rel=&quot;nofollow noreferrer&quot;&gt;binding motifs&lt;/a&gt; and see whether your gene list is enriched in those. These will, however, be inactive in a given tissue or cell type if the chromatin is packed or the methylation state of the promotor is unfavorable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples for those are the &lt;a href=&quot;http://jaspar.genereg.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;JASPAR&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/TRANSFAC&quot; rel=&quot;nofollow noreferrer&quot;&gt;TRANSFAC&lt;/a&gt; databases, or the &lt;a href=&quot;http://software.broadinstitute.org/gsea/msigdb/collections.jsp#C3&quot; rel=&quot;nofollow noreferrer&quot;&gt;MSigDB motif gene set&lt;/a&gt; (as burger mentioned). You can also query those features using the Ensembl genome database (&lt;a href=&quot;http://www.ensembl.org/biomart/martview/c48f212aea63b05694341e6ead06d756&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioMart&lt;/a&gt;, &lt;a href=&quot;https://rest.ensembl.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;REST&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Calculating enrichment in gene sets&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Most likely, an analysis you will want to perform is to calculate the enrichment in your gene list, e.g. that you got from differential expression in two conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most convenient way is to use the &lt;a href=&quot;http://amp.pharm.mssm.edu/Enrichr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Enrichr platform&lt;/a&gt;, which is a web page that accepts a gene list and will compute enrichment in ChEA, JASPAR, TRANSFAC, etc. You can also &lt;a href=&quot;http://amp.pharm.mssm.edu/Enrichr/#stats&quot; rel=&quot;nofollow noreferrer&quot;&gt;download their gene sets&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="69" LastActivityDate="2017-06-13T09:25:06.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="686" PostTypeId="2" ParentId="676" CreationDate="2017-06-13T09:49:44.683" Score="5" Body="&lt;p&gt;Currently you could use either but a major question is which platform will others be using in the future. AFAIK Perl is only superior to Python for regex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the trend I see for new programmers and new software being released: Perl is on the way out and Python is still growing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://trends.google.com/trends/explore?date=all&amp;amp;q=bioperl,biopython&quot; rel=&quot;noreferrer&quot;&gt;https://trends.google.com/trends/explore?date=all&amp;amp;q=bioperl,biopython&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2017-06-13T09:49:44.683" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="687" PostTypeId="1" CreationDate="2017-06-13T10:23:17.233" Score="13" ViewCount="3866" Body="&lt;p&gt;I have a gene expression count matrix produced from bulk RNA-seq data. I'd like to find genes that were &lt;strong&gt;not&lt;/strong&gt; expressed in a group of samples and were expressed in another group.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem of course is that not all &lt;em&gt;effectively&lt;/em&gt; non-expressed genes will have 0 counts due to sequencing errors, or because they were expressed in a small subset of cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm interested in solutions using &lt;code&gt;R&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2017-06-15T09:53:50.680" Title="What methods are available to find a cutoff value for non-expressed genes in RNA-seq?" Tags="|rna-seq|r|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="688" PostTypeId="2" ParentId="653" CreationDate="2017-06-13T10:32:44.133" Score="2" Body="&lt;p&gt;Your list has two identifiers for the same node per line. In order to use it, you will need to change that. If you want to use the gene name (2nd and 4th fields, in your example), just run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk 'print $2,$4' netw.txt &amp;gt; netw.gr&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want to use the Entrez geneIDs instead, run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk 'print $1,$3' netw.txt &amp;gt; netw.gr&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, as others already mentioned, install &lt;a href=&quot;http://cytoscape.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;Cytoscape&lt;/code&gt;&lt;/a&gt;, launch it and import your gene list:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;In the welcome screen, select &quot;From Network File...&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EeYPU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EeYPU.png&quot; alt=&quot;Cytoscape welcome screen&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then, select your network file (&lt;code&gt;netw.gr&lt;/code&gt; in the example above) and choose &quot;Advanced Options&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nDM2t.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nDM2t.png&quot; alt=&quot;Advanced options&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Set the delimiter to SPACE and uncheck &quot;Use first line as column names&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YOnzG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YOnzG.png&quot; alt=&quot;select the delimiter&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Finally, click on the header of each column and set the first to &quot;source node&quot; and the second to &quot;target node&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pUBYI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pUBYI.png&quot; alt=&quot;source node&quot;&gt;&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LbSm3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LbSm3.png&quot; alt=&quot;target node&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now, click &quot;OK&quot; and you will have successfully imported your network into Cytoscape. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-06-13T10:32:44.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="689" PostTypeId="2" ParentId="687" CreationDate="2017-06-13T11:09:16.817" Score="10" Body="&lt;p&gt;A common method is to use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3870982/&quot; rel=&quot;noreferrer&quot;&gt;zFPKMs&lt;/a&gt;, which you can find implemented in R &lt;a href=&quot;https://github.com/severinEvo/gene_expression/blob/master/zFPKM.R&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, there's an inherent problem in declaring a difference between two things on either side of a given threshold. Given that, I would encourage you to use more than &quot;expressed in one and not in another&quot;, likely adding at least a &quot;with a minimal difference of X between them&quot; metric. You may also find the tau metric useful, which is implemented in R &lt;a href=&quot;https://github.com/severinEvo/gene_expression/blob/master/tau.R&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. This is meant to measure tissue-specificity, which is more akin to what you're probably interested in doing.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-06-13T11:09:16.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="690" PostTypeId="2" ParentId="687" CreationDate="2017-06-13T11:13:08.053" Score="13" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I'd like to find genes that were not expressed in a group of samples and were expressed in another group.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is, fundamentally, a differential expression analysis, with a twist. To solve this, you’d first use a differential expression library of your choice (e.g. DESeq2) and perform a one-tailed test of differential expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Briefly, you’d perform the normal setup and then use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;results(dds, altHypothesis = 'greater')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To perform a one-tailed test. This will give you only those genes that are significantly upregulated in one group. Check chapter 3.9 of the vignette for details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course this won’t tell you that the genes are &lt;em&gt;unexpressed&lt;/em&gt; in the other group. Unfortunately I don’t know of a good value to threshold the results; I would start by plotting a histogram of the (variance stabilised) expression values in your first group, and then visually choose an expression threshold that cleanly separates genes that are clearly expressed from zeros:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;vst_counts = assay(vst(dds))&#xA;dens = density(vst_counts[, replicate])&#xA;plot(dens, log = 'y')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(This merges the replicates in the group, which should be fine.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Counts follow a multimodal distribution, with one mode for unexpressed and one or more for expressed genes. The expression threshold can be set somewhere between the clearly unexpressed and expressed peaks:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MF0vy.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MF0vy.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here I used &lt;code&gt;identify(dens)&lt;/code&gt; to identify the threshold interactively but you could also use an analytical method:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;threshold = identify(dens)&#xA;quantile = sum(dens$x &amp;lt; dens$x[threshold]) / length(dens$x)&#xA;&#xA;# Using just one replicate here; more robust would be to use a mean value.&#xA;nonzero_counts = counts(dds, normalized = TRUE)[, replicates[1]]&#xA;nonzero_counts = nonzero_counts[nonzero_counts &amp;gt; 0]&#xA;&#xA;(expression_threshold = quantile(nonzero_counts, probs = quantile))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;26.5625%&#xA;4.112033&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2017-06-14T12:23:37.687" LastActivityDate="2017-06-14T12:23:37.687" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="691" PostTypeId="2" ParentId="687" CreationDate="2017-06-13T11:30:59.540" Score="0" Body="&lt;p&gt;Unfortunatly that is not doable - there is now way of distinguishing between features not expressed and features not expressed for technical reasons. Furthermore a lot of genes might seem lowly expressed to to technical noise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This also means that you will have to choose a cutoff and simply use that. For choosing a cutoff looking at the replicate distribution of log10(counts) and log10(FPKM) are probably the best options.&lt;/p&gt;&#xA;" OwnerUserId="599" LastActivityDate="2017-06-13T11:30:59.540" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="692" PostTypeId="1" AcceptedAnswerId="698" CreationDate="2017-06-13T11:51:32.633" Score="8" ViewCount="1875" Body="&lt;p&gt;I have a large collection of sequence trace files which are in &lt;a href=&quot;http://staden.sourceforge.net/manual/formats_unix_2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SCF file format&lt;/a&gt; (binary files that in addition to the string of DNA bases also contain the electropherogram of the sample and quality information about the base calls).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have an existing BioPython pipeline that accepts files in &lt;a href=&quot;http://www6.appliedbiosystems.com/support/software_community/ABIF_File_Format.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;AB1 format&lt;/a&gt; (pdf), so I need to convert my SCF files to AB1 files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found a question on Biostars that asks for the opposite: &lt;a href=&quot;https://www.biostars.org/p/622/&quot; rel=&quot;nofollow noreferrer&quot;&gt;converting from SCF to AB1&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The accepted answer there was to use &lt;a href=&quot;http://staden.sourceforge.net/manual/manpages_unix_1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;convert_trace&lt;/code&gt;&lt;/a&gt; from the Staden package.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./convert_trace -out_format scf &amp;lt; trace.ab1 &amp;gt; trace.scf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So I tried the opposite:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./convert_trace -out_format abi &amp;lt; trace.scf &amp;gt; trace.ab1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This did indeed produce a file with &lt;code&gt;*.ab1&lt;/code&gt; extension, but upon importing it to my BioPython pipeline, I got the error&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;File should start ABIF, not '\xaeZTR'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So apparently, &lt;code&gt;convert_trace&lt;/code&gt; converted my files to &lt;a href=&quot;http://staden.sourceforge.net/manual/formats_unix_12.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ZTR format&lt;/a&gt;, even altough I specified &lt;code&gt;-out_format abi&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Closer reading of the manual states that &lt;code&gt;[convert_exe] can write CTF, EXP, PLN, SCF and ZTR formats.&lt;/code&gt; (so indeed no ABI), but it confuses me why &lt;code&gt;ABI&lt;/code&gt; is still a valid output format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my question is, how do I convert SCF files into ABI files?&lt;/p&gt;&#xA;" OwnerUserId="450" LastEditorUserId="73" LastEditDate="2018-01-15T01:35:29.013" LastActivityDate="2018-10-02T18:13:03.940" Title="How can I convert SCF trace files to ABI files?" Tags="|format-conversion|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="693" PostTypeId="1" AcceptedAnswerId="704" CreationDate="2017-06-13T13:03:09.127" Score="18" ViewCount="3232" Body="&lt;p&gt;In single-cell RNA-seq data we have an inflated number of 0 (or near-zero) counts due to low mRNA capture rate and other inefficiencies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can we decide which genes are 0 due to gene dropout (lack of measurement sensitivity), and which are genuinely not expressed in the cell?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deeper sequencing does not solve this problem as shown on the below saturation curve of 10x Chromium data:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cBXmF.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cBXmF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also see &lt;a href=&quot;https://academic.oup.com/biostatistics/article/doi/10.1093/biostatistics/kxx053/4599254&quot; rel=&quot;noreferrer&quot;&gt;Hicks et al. (2017)&lt;/a&gt; for a discussion of the problem:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Zero can arise in two ways:&lt;/p&gt;&#xA;  &#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;&lt;p&gt;the gene was not expressing any RNA (referred to as &lt;em&gt;structural zeros&lt;/em&gt;) or&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;the RNA in the cell was not detected due to limitations of current experimental protocols (referred to as &lt;em&gt;dropouts&lt;/em&gt;)&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="208" LastEditorUserId="208" LastEditDate="2017-11-13T12:28:39.757" LastActivityDate="2019-05-12T11:00:50.710" Title="How can we distinguish between true zero and dropout-zero counts in single-cell RNA-seq?" Tags="|rna-seq|ngs|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="694" PostTypeId="1" AcceptedAnswerId="742" CreationDate="2017-06-13T13:29:05.017" Score="9" ViewCount="1321" Body="&lt;p&gt;I'd like to test &lt;a href=&quot;http://mmtf.rcsb.org&quot; rel=&quot;noreferrer&quot;&gt;MMTF&lt;/a&gt;, a new format for storing biomolecular structures which is promoted by RCSB as a more compact alternative to mmCIF and PDB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;http://mmtf.rcsb.org/faq.html&quot; rel=&quot;noreferrer&quot;&gt;MMTF FAQ&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;&lt;p&gt;How do I convert a PDBx/mmCIF file to an MMTF file?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The BioJava library contains methods to read and write PDBx/mmCIF files and MMTF files.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Can I do such a conversion, ideally from command line, but without writing my own Java program?&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2021-10-24T05:17:41.950" Title="Converter between PDB or mmCIF and MMTF" Tags="|file-formats|pdb|3d-structure|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="695" PostTypeId="1" AcceptedAnswerId="697" CreationDate="2017-06-13T13:43:41.210" Score="5" ViewCount="266" Body="&lt;p&gt;I have a set of genomic ranges that are potentially overlapping. I want to count the amount of ranges at certain positions using R. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm Pretty sure there are good solutions, but I seem to be unable to find them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solutions like &lt;a href=&quot;https://stackoverflow.com/questions/26140151/binning-two-vectors-of-different-ranges-using-r?rq=1&quot;&gt;cut&lt;/a&gt; or &lt;a href=&quot;https://stackoverflow.com/questions/11963508/generate-bins-from-a-data-frame&quot;&gt;findIntervals&lt;/a&gt; don't achieve what I want as they only count on one vector or accumulate by &lt;code&gt;all values &amp;lt;= break&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also &lt;code&gt;countMatches {GenomicRanges}&lt;/code&gt; doesn't seem to cover it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably one could use Bedtools, but I don't want to leave R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could only come up with a hilariously slow solution&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# generate test data&#xA;testdata &amp;lt;- data.frame(chrom = rep(seq(1,10),10),&#xA;                       starts = abs(rnorm(100, mean = 1, sd = 1)) * 1000,&#xA;                       ends = abs(rnorm(100, mean = 2, sd = 1)) * 2000)&#xA;&#xA;# make sure that all end coordinates are bigger than start&#xA;# this is a requirement of the original data&#xA;testdata &amp;lt;- testdata[testdata$ends - testdata$starts &amp;gt; 0,]&#xA;&#xA;# count overlapping ranges on certain positions&#xA;count.data &amp;lt;- lapply(unique(testdata$chrom), function(chromosome){&#xA;    tmp.inner &amp;lt;- lapply(seq(1,10000, by = 120), function(i){&#xA;        sum(testdata$chrom == chromosome &amp;amp; testdata$starts &amp;lt;= i &amp;amp; testdata$ends &amp;gt;= i)&#xA;    })&#xA;    return(unlist(tmp.inner))&#xA;})&#xA;&#xA;# generate a data.frame containing all data&#xA;df.count.data &amp;lt;- ldply(count.data, rbind)&#xA;&#xA;# ideally the chromosome will be columns and not rows&#xA;t(df.count.data)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="248" LastEditorUserId="298" LastEditDate="2017-06-13T14:14:59.873" LastActivityDate="2017-06-13T14:14:59.873" Title="Count genomic ranges" Tags="|r|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="696" PostTypeId="2" ParentId="678" CreationDate="2017-06-13T14:01:53.477" Score="2" Body="&lt;p&gt;Use &lt;a href=&quot;https://github.com/lh3/htsbox&quot; rel=&quot;nofollow noreferrer&quot;&gt;htsbox&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;htsbox samview -p in.bam | less -S&#xA;htsbox samview -pS in.sam | less -S&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It outputs mapping positions in the &lt;a href=&quot;https://github.com/lh3/miniasm/blob/master/PAF.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;PAF&lt;/a&gt; format, which looks something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;read1  4983  774  4982  +  chr18  80373285  26911072  26915544  3835  4631  60 \&#xA;       mm:i:214  io:i:119  in:i:159  do:i:339  dn:i:423&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is one long line in terminal. I folded the line for display purposes. For the first 12 fixed fields, see the table in &lt;a href=&quot;https://github.com/lh3/miniasm/blob/master/PAF.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;PAF page&lt;/a&gt;. They give mapping positions and identity. The optional fields tell you #mismatches (mm), #insOpens (io), #insertions (in), #delOpens (do) and #deletions (dn). Note that the alignment MUST have an &quot;NM&quot; tag; otherwise the number of matching bases (col. 11) is overestimated and &quot;mm&quot; will always be zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can easily compute summary statistics from PAF if you are familiar with command lines. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-shell prettyprint-override&quot;&gt;&lt;code&gt;htsbox samview -p in.bam | awk '{x+=$10;y+=$11}END{print x/y}'  # identity&#xA;htsbox samview -p in.bam | awk '$2&amp;gt;1000{x+=$10;y+=$11}END{print x/y}'  # identity for &amp;gt;1000bp&#xA;htsbox samview -p in.bam \&#xA;  | perl -ane '{$g+=$1 while /[id]n:i:(\d+)/g;$y+=$F[10]}END{print $g/$y,&quot;\n&quot;}' # gap rate&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you don't like long one-liners, feel free to write a proper script to collect all statistics in one go.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-06-13T14:01:53.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="697" PostTypeId="2" ParentId="695" CreationDate="2017-06-13T14:04:02.240" Score="6" Body="&lt;p&gt;&lt;a href=&quot;https://www.rdocumentation.org/packages/GenomicRanges/versions/1.24.1/topics/findOverlaps-methods&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;GenomicRanges::countOverlaps&lt;/code&gt;&lt;/a&gt; seems to be what you’re after:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;position_range = GRanges(position$chrom, IRanges(position, position, width = 1))&#xA;ranges_at_position = countOverlaps(position_ranges, granges)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29" LastActivityDate="2017-06-13T14:04:02.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="698" PostTypeId="2" ParentId="692" CreationDate="2017-06-13T14:06:16.443" Score="4" Body="&lt;p&gt;Apparently &lt;code&gt;convert_trace&lt;/code&gt; does not do a good parameter checking and silently sets ZTR as default if it does not recognise a valid output format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know of any freely available command-line converters in that specific direction. Even at the height of Sanger sequencing (late 90s, early 2000s), pipelines usually tried to reduce complexity and footprint by converting AB1 to anything else, often SCF (ZTR came too late and never got enough traction).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That being said, I dug out an SCF I had from 1997, loaded it into &lt;a href=&quot;http://www.geneious.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Geneious&lt;/a&gt;, and re-exported it as .ab1. I was a bit astonished to see that this worked. So Geneious (or similar packages from other companies) may be your best bet for a quick, one-off thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on your needs, you might want to think about other ways: either expand BioPython to read SCFs, or maybe to expand the Staden io_lib to be able to write .ab1 (James still maintains that package and is extremely helpful).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck.&lt;/p&gt;&#xA;" OwnerUserId="44" LastActivityDate="2017-06-13T14:06:16.443" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="699" PostTypeId="2" ParentId="676" CreationDate="2017-06-13T14:59:54.690" Score="1" Body="&lt;p&gt;I agree with @story and @agapow here. As it stands right now BioPerl is still much more mature than BioPython. If there's a tool or method that only exists in the BioPerl library then you should probably use BioPerl to accomplish what you need to accomplish. But it's undeniable that BioPython is progressing very quickly and will soon overtake BioPerl. That being said there are still a lot of plus side regarding Perl in the Bioinformatics realm and it will definitely help to be somewhat competent in it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in short my answer is to use both! I find myself mostly using Python and BioPython but if I need to accomplish something using a tool that isn't available in the BioPython library or can be accomplished more quickly using Perl I will look towards BioPerl/Perl. &lt;/p&gt;&#xA;" OwnerUserId="712" LastActivityDate="2017-06-13T14:59:54.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="700" PostTypeId="2" ParentId="693" CreationDate="2017-06-13T15:21:16.947" Score="1" Body="&lt;p&gt;Some people use imputation to differentiate between true zeros and dropout in single-cell data. Some approaches you can look into:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/VCCRI/CIDR&quot; rel=&quot;nofollow noreferrer&quot;&gt;CIDR&lt;/a&gt; which was published in Genome Biology, &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-017-1188-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/Vivianstats/scImpute&quot; rel=&quot;nofollow noreferrer&quot;&gt;scImpute&lt;/a&gt; , published in Nature, &lt;a href=&quot;https://www.nature.com/articles/s41467-018-03405-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Seurat (&lt;a href=&quot;http://satijalab.org/seurat/seurat_spatial_tutorial_part2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;addImputedScore&lt;/a&gt;) , i.e. the famous R package, also &lt;a href=&quot;https://cran.r-project.org/web/packages/Seurat/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/mohuangx/SAVER/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAVER&lt;/a&gt; which was reviewed in Frontiers of Genetics, &lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fgene.2019.00317/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="35" LastEditorUserId="3848" LastEditDate="2019-05-12T11:00:50.710" LastActivityDate="2019-05-12T11:00:50.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="701" PostTypeId="2" ParentId="664" CreationDate="2017-06-13T15:23:02.040" Score="4" Body="&lt;p&gt;You requested a tool similar to &lt;a href=&quot;http://pantherdb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PANTHER&lt;/a&gt; but in R. First, the PANTHER (Protein ANalysis THrough Evolutionary Relationships) tool does a classification  based on evolutionarily related proteins, gene ontologies ( molecular function, and biological process) and pathways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AFAIK there isn't a tool in R that integrates all these into one, but several packages do pieces of it. However, my observations after using it a couple of times is that is based a lot in gene ontologies. Gene Ontologies (GO) are not pathways, but the best tool I found for GO are &lt;a href=&quot;http://bioconductor.org/packages/topGO&quot; rel=&quot;nofollow noreferrer&quot;&gt;topGO&lt;/a&gt; (A bit hard to use but useful to get insights on the biology) which test over representations of a term in a given list. &lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2018-01-11T07:45:34.283" LastActivityDate="2018-01-11T07:45:34.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="702" PostTypeId="1" CreationDate="2017-06-13T15:38:05.440" Score="4" ViewCount="200" Body="&lt;p&gt;I made this post in regular stack overflow but I was told about this awesome feature by @nbryans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a researcher (my programming knowledge is small) conducting analysis on a set of antibiotic (methicillin) resistant and a set of antibiotic (methicillin) susceptible. The sole difference between these two sets is assumed to be the resistance (R) vs susceptibility (S). I want to find if a genetic element in the resistant or susceptible genomes is correlated to resistance (positively or negatively). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have done a complete fisher exact test already and will detail how I did so below. However I would like to figure out how to prepare my data for a more complex learning algorithm such as those offered by sci-kit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(many of my techniques were learned from this &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/27297683&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First I used &lt;a href=&quot;http://sun.aei.polsl.pl/REFRESH/index.php?page=projects&amp;amp;project=kmc&amp;amp;subpage=about&quot; rel=&quot;nofollow noreferrer&quot;&gt;KMC&lt;/a&gt; to split each fasta file for both R and S genomes into k-mers (I don't have enough &quot;reputation&quot; so I can't use &gt;2 links but k-mers are essentially genetic sequences of k length). The output for KMC was a dump of k-mers present in a genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: kmc dump&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AAAAAAAAAACGAATGTACACAATCGAA    1&#xA;AAAAAAAAACGAATGTACACAATCGAAC    1&#xA;AAAAAAAAATCAAATCCTGACTATTTAG    1&#xA;AAAAAAAAATCGGTCAATTCATTAAAAG    1&#xA;AAAAAAAACAAACATGAAGACCTTGTTA    1&#xA;AAAAAAAACGAATGTACACAATCGAACA    1&#xA;AAAAAAAACGTGTTAAAGTGAATCACAC    1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I then merged the data into one binary matrix which had columns as genomes and rows as k-mers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: 1 resistant genome and 1 susceptible genome&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;kmer    R_1    S_1&#xA;AAAAAAAAAACGAATGTACACAATCGAA    1   0&#xA;AAAAAAAAACGAATGTACACAATCGAAC    1   0&#xA;AAAAAAAAATCAAATCCTGACTATTTAG    1   0&#xA;AAAAAAAAATCGGTCAATTCATTAAAAG    1   0&#xA;AAAAAAAAATTCCCTTCTAATCTTGAAT    0   1&#xA;AAAAAAAACAAAAATTATATAAAGCGAA    0   1&#xA;AAAAAAAACAAACATGAAGACCTTGTTA    1   1&#xA;AAAAAAAACAACCACCCATACATTGAGT    0   1&#xA;AAAAAAAACCCTTACAACAAATATGTAA    0   1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the data I have been working with and would like to analyze further with sci kit or other classification algorithms. Essentially each k-mer is a feature whose correlation with resistance is based on its presence in R vs S genomes. This is where I would like help transitioning this data into a dataset format suitable for sci kit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I conducted a Fisher Exact test on each k-mer using R. For each k-mer a 2x2 matrix was created with the first column as resistant and the second column as susceptible. The two rows were present or not. So four numbers (# times k-mer was present in R, # in susceptible, # not in R, # not in S).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;      R    S&#xA;yes   #    #&#xA;no    #    #&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I used the following code in R:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;phenotype = as.numeric(grepl('^R_', colnames(raw_kmer_table)[2:ncol(raw_kmer_table)]))&#xA;fe_results = apply(kmer_pres_abs_matrix, 1, &#xA;               FUN = function(row) {&#xA;                 fe_mat = matrix(0, ncol = 2, nrow = 2)&#xA;                 fe_mat[1,1] = sum(row == 1 &amp;amp; phenotype == 1)&#xA;                 fe_mat[1,2] = sum(row == 1 &amp;amp; phenotype == 0)&#xA;                 fe_mat[2,1] = sum(row == 0 &amp;amp; phenotype == 1)&#xA;                 fe_mat[2,2] = sum(row == 0 &amp;amp; phenotype == 0)&#xA;&#xA;                 fe = fisher.test(fe_mat)&#xA;                 return(fe$p.value)&#xA;               }&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now that you have read how my data is currently formatted I would like to hear suggestions on how I can fit this data into a more complex test using sci kit or another resource. Algorithms I am interested in using are any you see fit and previously field-tested algorithms such as adaboost and forest learning algorithms. Also please ask me any clarifications or things I may have left out of this post!&lt;/p&gt;&#xA;" OwnerUserId="842" LastActivityDate="2017-06-14T15:33:46.443" Title="Preparing binary matrix data for Scikit classification algorithms" Tags="|k-mer|python|machine-learning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="703" PostTypeId="1" AcceptedAnswerId="739" CreationDate="2017-06-13T16:02:47.477" Score="5" ViewCount="115" Body="&lt;p&gt;We sort different populations of blood cells using a number of fluorescent flow cytometry markers and then sequence RNA. We want to see what the transcriptome tells us about the similarity and relation between these cells. In my experience on bulk RNA-seq data, there is a very good agreement between flow cytometry and mRNA expression for the markers. It's good to remember we sort using a few markers only, while there are hundreds if not thousands of cell surface proteins.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should we exclude genes of these sorting (CD) marker proteins when we perform PCA or other types of clustering?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The argument is that after sorting, these genes may dominate clustering results, even if the rest of the transcriptome would tell otherwise, and thus falsely confirm similarity relations inferred from sorting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should we at least check the influence of these genes on clustering results?&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2017-06-14T16:34:25.797" Title="Should the cell sorting marker genes be excluded during clustering?" Tags="|rna-seq|cell-line|clustering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="704" PostTypeId="2" ParentId="693" CreationDate="2017-06-13T16:03:16.817" Score="15" Body="&lt;p&gt;Actually this is one of the main problems you have when analyzing scRNA-seq data, and there is no established method for dealing with this. Different (dedicated) algorithms deal with it in different ways, but mostly you rely on how good the error modelling of your software is (a great read is the &lt;a href=&quot;http://dx.doi.org/10.1038/nbt.3711&quot; rel=&quot;noreferrer&quot;&gt;review&lt;/a&gt; by Wagner, Regev &amp;amp; Yosef, esp. the section on &quot;False negatives and overamplification&quot;). There are a couple of options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You can &lt;strong&gt;impute values&lt;/strong&gt;, i.e. fill in the gaps on technical zeros. &lt;a href=&quot;https://github.com/VCCRI/CIDR&quot; rel=&quot;noreferrer&quot;&gt;CIDR&lt;/a&gt; and &lt;a href=&quot;https://github.com/Vivianstats/scImpute&quot; rel=&quot;noreferrer&quot;&gt;scImpute&lt;/a&gt; do it directly. &lt;a href=&quot;http://biorxiv.org/content/early/2017/02/25/111591&quot; rel=&quot;noreferrer&quot;&gt;MAGIC&lt;/a&gt; and &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-015-0805-z&quot; rel=&quot;noreferrer&quot;&gt;ZIFA&lt;/a&gt; project cells into a lower-dimensional space and use their similarity there to decide how to fill in the blanks.&lt;/li&gt;&#xA;&lt;li&gt;Some people straight up &lt;strong&gt;exclude genes&lt;/strong&gt; that are expressed in very low numbers. I can't give you citations off the top of my head, but many trajectory inference algorithms like &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/monocle.html&quot; rel=&quot;noreferrer&quot;&gt;monocle2&lt;/a&gt; and &lt;a href=&quot;https://github.com/jw156605/SLICER&quot; rel=&quot;noreferrer&quot;&gt;SLICER&lt;/a&gt; have heuristics to choose informative genes for their analysis.&lt;/li&gt;&#xA;&lt;li&gt;If the method you use for analysis doesn't model gene expression explicitly but &lt;strong&gt;uses some other distance method&lt;/strong&gt; to quantify similarity between cells (like cosine distance, euclidean distance, correlation), then the noise introduced by dropout can be covered by the signal of genes that are highly expressed. Note that this is dangerous, as genes that are highly expressed are not necessarily informative.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;ERCC spike&lt;/strong&gt; ins can help you reduce technical noise, but I am not familiar with the Chromium protocol so maybe it doesn't apply there (?)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;since we are speaking about noise, you might consider using a protocol with unique molecular identifiers. They remove the amplification errors almost completely, at least for the transcripts that you capture...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Also, I would highly recommend using something more advanced than PCA to do the analysis. Software like the above-mentioned Monocle or &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/destiny.html&quot; rel=&quot;noreferrer&quot;&gt;destiny&lt;/a&gt; is easy to operate and increases the power of your analysis considerably.&lt;/p&gt;&#xA;" OwnerUserId="787" LastActivityDate="2017-06-13T16:03:16.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="705" PostTypeId="1" CreationDate="2017-06-13T16:24:28.293" Score="6" ViewCount="982" Body="&lt;p&gt;I am trying to understand how the MD:Z tag is used. The following is from the &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMtags.pdf&quot; rel=&quot;noreferrer&quot;&gt;SAM Optional Fields Specification&lt;/a&gt;, which gives an example but is not thorough. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The MD field aims to achieve SNP/indel calling without looking at the&#xA;  reference. For example, a string ‘10A5^AC6’ means from the leftmost&#xA;  reference base in the alignment, there are 10 matches followed by an A&#xA;  on the reference which is different from the aligned read base; the&#xA;  next 5 reference bases are matches followed by a 2bp deletion from the&#xA;  reference; the deleted sequence is AC; the last 6 bases are matches.&#xA;  The MD field ought to match the CIGAR string.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Suppose I have a read that I want to soft clip at both ends. If the read starts with CIGAR &lt;code&gt;100M&lt;/code&gt; and MD &lt;code&gt;50G49&lt;/code&gt;, and I want to change the CIGAR to &lt;code&gt;7S86M7S&lt;/code&gt;, what should the MD field become? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a more complete explanation of what appears in the MD tag field?&lt;/p&gt;&#xA;" OwnerUserId="272" LastActivityDate="2017-06-13T17:01:26.013" Title="How should the SAM MD tag match the CIGAR string?" Tags="|file-formats|sam|cigar|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="707" PostTypeId="2" ParentId="705" CreationDate="2017-06-13T16:50:57.050" Score="5" Body="&lt;p&gt;The MD string doesn't apply to soft or hard clipped regions,. so your example read becomes &lt;code&gt;43G42&lt;/code&gt;. Since variant calling using something simplistic like this is only ever going to use the aligned portion, I guess there was never much reason to bother with adding in soft-clipping information. This also means that you need to parse the CIGAR string if you're using the MD string.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-06-13T17:01:26.013" LastActivityDate="2017-06-13T17:01:26.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="708" PostTypeId="1" AcceptedAnswerId="710" CreationDate="2017-06-13T17:07:48.073" Score="5" ViewCount="531" Body="&lt;p&gt;I generate bigWig files using &lt;code&gt;bamCoverage&lt;/code&gt; from deeptools, in part for my colleagues to visualize their mapped libraries in the IGV viewer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A problem is that the displayed track name is apparently the file name, which is not convenient because some files for different libraries have the same name. The reason is that the files are organized with one directory per library. I would prefer to avoid changing the way I name and organize the files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there a way to edit my bigWig files in order to have control of the track name independently from the file name?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I understand, bigWig format derives from wiggle format, which have a track definition line that can contain a &lt;code&gt;name&lt;/code&gt; and a &lt;code&gt;definition&lt;/code&gt; attribute: &lt;a href=&quot;https://genome.ucsc.edu/goldenpath/help/customTrack.html#TRACK&quot; rel=&quot;noreferrer&quot;&gt;https://genome.ucsc.edu/goldenpath/help/customTrack.html#TRACK&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe there is a way to convert a bigWig file back to wiggle, edit the track definition, and then rebuild an updated bigWig file.&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2017-06-13T17:41:31.517" Title="Customizing bigWig file" Tags="|file-formats|format-conversion|bigwig|visualization|genome-browser|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="709" PostTypeId="1" CreationDate="2017-06-13T17:15:34.113" Score="3" ViewCount="204" Body="&lt;p&gt;When I use blastn and prokka (I will detail exactly how I did so below) on a 2.8 million bp fasta file I get output start/end numbers that do not seem to cover the entire genome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Starting with a .fna genome such as genome.fna I ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1 - blastn&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Searched the genome for 3135 different 28-mers using BLASTn.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;makeblastdb -in genome.fna -dbtype nucl -parse_seqids -out ./output/genome&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;next command in python&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blast_tsv = NcbiblastnCommandline(query=Q, db=DB, perc_identity=100, outfmt=6, out=(OUT))&#xA;    stdout, stderr = blast_tsv()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Q is this list of k-mers. DB is the database created&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This outputs a list of search results but the important thing is that none of the start/end (columns 7 and 8) integers were greater than 100,000 yet the entire genome is 2.8 million base pairs long. I can provide the link to this file in the comments. Does this have to do with blastn stopping searching after finding one match? And if so how can I tell blastn to search for every match for each k-mer? (I'm open to using other blastn programs other than biopython)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2 - prokka&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used prokka to create gff files for the genome. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;prokka-1.12/bin/prokka —setupdb&#xA;prokka-1.12/bin/prokka -kingdom Bacteria -rfam -outdir ./prokka_database/genome -force -prefix &quot;${genome/.fna/}&quot; genome.fna&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Th gff file produced by this command only described genes with start and end positions less than 200k. I can provide the link to this file as well. &lt;/p&gt;&#xA;" OwnerUserId="842" LastActivityDate="2017-06-13T18:22:13.263" Title="Why do BLASTn and prokka not seem to be searching the whole fasta file?" Tags="|sequence-annotation|genome|k-mer|python|blast|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="710" PostTypeId="2" ParentId="708" CreationDate="2017-06-13T17:41:31.517" Score="4" Body="&lt;p&gt;There's no equivalent to the wiggle header in bigWig (or bigBed) files, which is why UCSC uses the file name. This is actually the reason for the track line stuff that you linked to, since you can then specify a name and just point to where the bigWig (or other format) file is on the internet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, you can certainly convert your bigWig to wiggle, add the track line, and convert it back with UCSC tools (e.g., &lt;code&gt;bigWigToWig&lt;/code&gt; and &lt;code&gt;wigToBigWig&lt;/code&gt;). It shouldn't change the track name when you display it in UCSC (though if it does, then let me know, since I'll modify pyBigWig and libBigWig to support that then).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-06-13T17:41:31.517" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="711" PostTypeId="2" ParentId="709" CreationDate="2017-06-13T18:22:13.263" Score="6" Body="&lt;p&gt;With a k-mer size of 28 it shouldn't be finding that many matches. And the prokka results are suspicious as well. Maybe you have multiple contigs (none larger than 100kb) in that file? What is the result of &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;grep ^'&gt;' fasta_file | wc -l&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;? This would show how many contigs you have in the file.&lt;/p&gt;&#xA;" OwnerUserId="848" LastActivityDate="2017-06-13T18:22:13.263" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="712" PostTypeId="2" ParentId="687" CreationDate="2017-06-13T18:46:27.060" Score="5" Body="&lt;p&gt;Depending on how much effort you wish to put into this, here is one suggestion I have see used before (uses more than just R). Steps 1-4 come from &lt;a href=&quot;http://genome.cshlp.org/content/24/12/1918.long#ref-55&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; (see supplimentary material section &quot;Calculation of per gene Local FDR&quot;).&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Take your gene models and collapse the introns down to maybe 100bp&lt;/em&gt;&lt;br&gt;&#xA;Thus if we have a gene with two exons and a 1kb intron eg. exons (1000, 1100) and (2000,2100), we reduce the intron size so that the exons are (1000,1100) and (1200,1300). This is because we need to find sufficient gene free space to fit the null gene in.    &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Shift each gene into the nearest genomic space at least 5 kp from a genomic annotation and free of ESTs.&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Calculate the FPKM distribution of this shifted null set&lt;/em&gt;  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;From these two distributions it should be possible to calculate a &lt;a href=&quot;https://cran.r-project.org/web/packages/locfdr/vignettes/locfdr-example.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;local FDR&lt;/a&gt; for expression.&lt;/em&gt;&lt;br&gt;&#xA;You will need to use a &quot;mixing proportion&quot; to do this. The above reference used &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2660870/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Qvality&lt;/a&gt; to do this.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Carry out differential expression analysis as described in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/690/235&quot;&gt;Konrad's answer&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Subset the differentially expressed genes to only consider those that are not expressed in the control condition&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;You might want to also recalculate the FDRs for differential expression as you are only considering a subset of the tests you might have.&lt;/em&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="235" LastEditorUserId="235" LastEditDate="2017-06-15T09:53:50.680" LastActivityDate="2017-06-15T09:53:50.680" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="713" PostTypeId="2" ParentId="694" CreationDate="2017-06-13T18:55:36.833" Score="2" Body="&lt;p&gt;You looked at this &lt;a href=&quot;https://github.com/rcsb/mmtf-python&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/rcsb/mmtf-python&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;The python implementation of the MMTF API, decoder and encoder.&quot;&lt;/p&gt;&#xA;" OwnerUserId="640" LastActivityDate="2017-06-13T18:55:36.833" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="714" PostTypeId="2" ParentId="620" CreationDate="2017-06-13T19:14:44.307" Score="3" Body="&lt;p&gt;I don't know about tools, but i've used the following python code to calculate the ratio of reads that overlap the 5' or 3' ends of introns or that are spliced. We sum these across all introns in a gene set (we actaully use this for iCLIP analysis to see if RNA binding proteins bind pre-mRNA or spliced RNA).&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pysam&#xA;from collections import Counter&#xA;from CGAT import GTF, IOTools&#xA;&#xA;def calculateSplicingIndex(bamfile, gtffile, outfile):&#xA;&#xA;    bamfile = pysam.AlignmentFile(bamfile)&#xA;&#xA;    counts = Counter()&#xA;&#xA;    for transcript in GTF.transcript_iterator(&#xA;            GTF.iterator(IOTools.openFile(gtffile))):&#xA;&#xA;        introns = GTF.toIntronIntervals(transcript)&#xA;&#xA;        for intron in introns:&#xA;            reads = bamfile.fetch(&#xA;                reference=transcript[0].contig,&#xA;                start=intron[0], end=intron[1])&#xA;&#xA;            for read in reads:&#xA;                if 'N' in read.cigarstring:&#xA;                    blocks = read.get_blocks()&#xA;                    starts, ends = zip(*blocks)&#xA;                    if intron[0] in ends and intron[1] in starts:&#xA;                        counts[&quot;Exon_Exon&quot;] += 1&#xA;                    else:&#xA;                        counts[&quot;spliced_uncounted&quot;] += 1&#xA;                elif (read.reference_start &amp;lt;= intron[0] - 3&#xA;                      and read.reference_end &amp;gt;= intron[0] + 3):&#xA;                    if transcript[0].strand == &quot;+&quot;:&#xA;                        counts[&quot;Exon_Intron&quot;] += 1&#xA;                    else:&#xA;                        counts[&quot;Intron_Exon&quot;] += 1&#xA;                elif (read.reference_start &amp;lt;= intron[1] - 3&#xA;                      and read.reference_end &amp;gt;= intron[1] + 3):&#xA;                    if transcript[0].strand == &quot;+&quot;:&#xA;                        counts[&quot;Intron_Exon&quot;] += 1&#xA;                    else:&#xA;                        counts[&quot;Exon_Intron&quot;] += 1&#xA;                else:&#xA;                    counts[&quot;unspliced_uncounted&quot;] += 1&#xA;&#xA;    header = [&quot;Exon_Exon&quot;,&#xA;              &quot;Exon_Intron&quot;,&#xA;              &quot;Intron_Exon&quot;,&#xA;              &quot;spliced_uncounted&quot;,&#xA;              &quot;unspliced_uncounted&quot;]&#xA;&#xA;    with IOTools.openFile(outfile, &quot;w&quot;) as outf:&#xA;&#xA;        outf.write(&quot;\t&quot;.join(header)+&quot;\n&quot;)&#xA;        outf.write(&quot;\t&quot;.join(map(str, [counts[col] for col in header]))&#xA;                   + &quot;\n&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unfortunately this uses a bunch of libraries you may or may not have, including CGAT (for the GTF parser and IOTools package) and pysam.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you've got these statistics you can calculate the &quot;splicing index&quot; as &#xA;the log2 ratio of 2 times the number of spliced reads divided by the number of reads overlapping the 3' and 5' ends of introns. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2017-06-13T19:14:44.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="715" PostTypeId="2" ParentId="594" CreationDate="2017-06-13T19:28:19.813" Score="3" Body="&lt;p&gt;The &lt;a href=&quot;http://meme-suite.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;MEME Suite&lt;/a&gt; web site contains a collection of tools for motif analysis (I'm one of the maintainers). It contains two de novo motif discovery tools: &lt;a href=&quot;http://meme-suite.org/tools/meme&quot; rel=&quot;nofollow noreferrer&quot;&gt;MEME&lt;/a&gt; and &lt;a href=&quot;http://meme-suite.org/tools/dreme&quot; rel=&quot;nofollow noreferrer&quot;&gt;DREME&lt;/a&gt;. Public web applications are provided, but you can also download and build command line tools for a local installation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your first goal you could use &lt;a href=&quot;http://meme-suite.org/tools/meme&quot; rel=&quot;nofollow noreferrer&quot;&gt;MEME&lt;/a&gt; and select the &quot;Any number of repetitions model&quot; (ANR). For your second goal, you'd use MEME with the &quot;Zero or One Occurrences Per Sequence&quot; (ZOOPS) model. For your third goal you could use FIMO (Find Individual Motif Occurrences), and one or more of the motif databases provided on the &lt;a href=&quot;http://meme-suite.org/doc/download.html?man_type=web&quot; rel=&quot;nofollow noreferrer&quot;&gt;software and database download page&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It sounds like your sequence data is about 30Mb. The MEME web application is limited to 60kb of sequence data, so you'd have to install a local copy of the MEME Suite. MEME would take a long time to analyze a 30Mb sequence database unless you have MPI configured, and lots of cores available. You might want to consider analyzing a randomly selected subset of your sequences. The running time of MEME grows as the cube of the number of sequences. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For short motifs, you may want to use &lt;a href=&quot;http://meme-suite.org/tools/dreme&quot; rel=&quot;nofollow noreferrer&quot;&gt;DREME&lt;/a&gt; rather than MEME. DREME is better than MEME at identifying short motifs, but is limited to motifs &amp;lt;= 8 positions wide.&lt;/p&gt;&#xA;" OwnerUserId="313" LastEditorUserId="313" LastEditDate="2017-06-13T22:20:11.820" LastActivityDate="2017-06-13T22:20:11.820" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="716" PostTypeId="1" AcceptedAnswerId="717" CreationDate="2017-06-13T19:39:13.280" Score="3" ViewCount="302" Body="&lt;p&gt;I am trying to calculate the mappability adjusted length of introns as described by &lt;a href=&quot;http://genesdev.cshlp.org/content/29/1/63.abstract&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boutz et al&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Briefly, for each intron I wish to calculate the length minus the number of bases that are non-uniquely mappable. Mappability tracks can be downloaded as bigWigs from &lt;a href=&quot;http://genome.ucsc.edu/cgi-bin/hgFileUi?db=hg19&amp;amp;g=wgEncodeMapability&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The the score at each base is 1/number of mapping positions, so 1 indicates a read can be uniquely mapped at this location. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am struggling to do this computation in either a reasonable amount of time, or a reasonable amount of memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First I tried importing the whole bigwig:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(rtracklayer)&#xA;mappability &amp;lt;- import(BigWigFile(&quot;wgEncodeCrgMapabilityAlign50mer.bigWig&quot;),&#xA;                      as=&quot;NumericList&quot;,&#xA;                      selection=BigWigSelection(intron_ranges))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;intron_ranges&lt;/code&gt; is a &lt;code&gt;GRanges&lt;/code&gt; object with the introns (about 800,000 of them).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This used &lt;strong&gt;way&lt;/strong&gt; too much memory, and soon caused by machine to fall over.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second I tried processing one intron at a time:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;mappability_file = BigWigFile(&quot;wgEncodeCrgMapabilityAlign50mer.bigWig&quot;)&#xA;effective_length &amp;lt;- function (gr) {&#xA;  intron_selection &amp;lt;- BigWigSelection(gr)&#xA;  scores &amp;lt;- import(mappability_file, as=&quot;NumericList&quot;, selection=intron_selection)&#xA;  non_unique &amp;lt;- sum(scores[[1]] &amp;lt; 1.0)&#xA;  eff_len = width(gr)[1] - non_unique&#xA;  return(eff_len)&#xA;}    &#xA;&#xA;mappability &amp;lt;- sapply(intron_ranges, effective_length)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This has been running for hours and shows no sign of finishing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to do this that uses less than, say 4GB of RAM, but finishes in say less than 30 minutes? It feels like there should be. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not tied to R, just what I've tried so far. Happy with answers using binary packages, common shell tools, python or R. &lt;/p&gt;&#xA;" OwnerUserId="235" LastEditorUserId="292" LastEditDate="2017-06-14T11:44:08.643" LastActivityDate="2017-06-14T11:44:08.643" Title="Time and memory efficient processing of many intervals for a bigWig file" Tags="|bigwig|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="717" PostTypeId="2" ParentId="716" CreationDate="2017-06-13T20:43:29.640" Score="2" Body="&lt;p&gt;Assuming you can make a BED file of introns, you can then use the &lt;code&gt;pyBigWig&lt;/code&gt; module in python:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pyBigWig&#xA;import numpy&#xA;&#xA;bw = pyBigWig.open(&quot;some file.bw&quot;)&#xA;bed = open(&quot;introns.bed&quot;)&#xA;for line in bed:&#xA;    cols = line.strip().split(&quot;\t&quot;)&#xA;    vals = bw.values(cols[0], int(cols[1]), int(cols[2]), numpy=True)&#xA;    effLen = cols[2] - cols[1] - (vals &amp;lt; 1.0).sum()&#xA;    # Do something with this.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One can make that parallel (with the deeptools API) should it prove too slow. This will only use an appreciable amount of memory for very large introns, since there you need to store the value at each base. One could make this much more memory efficient by using &lt;code&gt;bw.intervals()&lt;/code&gt; instead, but that'd require quite a bit more code (you'd need to determine the number of bases overlap per interval and then assume that either each base you want to query has an overlapping interval or, if not, keep track of that fact).&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-06-13T20:52:36.063" LastActivityDate="2017-06-13T20:52:36.063" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="718" PostTypeId="2" ParentId="658" CreationDate="2017-06-13T20:46:40.447" Score="4" Body="&lt;p&gt;It really depends on what you are trying to do, but here are a few services that I know of.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://cloud.google.com/genomics/v1alpha2/gatk&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATK on Google Genomics Cloud&lt;/a&gt;: Google and the Broad offer a cloud instance tailored to GATK pipelines.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://aws.amazon.com/health/genomics/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genomics on Amazon Web Services&lt;/a&gt;: I don't think there is anything that makes this unique, but Amazon offers some resources to help get started with genomics/life sciences-centered cloud solutions.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.illumina.com/informatics.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Illumina Bioinformatics&lt;/a&gt;: Illumina is working on a whole suite of bioinformatics software for the cloud.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.cancergenomicscloud.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cancer Genomics Cloud&lt;/a&gt;: This is specific to cancer genomics, but I believe Seven Bridges allows you to push all sorts of data into the tool and analyze it.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="47" LastActivityDate="2017-06-13T20:46:40.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="719" PostTypeId="1" AcceptedAnswerId="726" CreationDate="2017-06-13T22:20:23.237" Score="5" ViewCount="182" Body="&lt;p&gt;From &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMtags.pdf&quot; rel=&quot;noreferrer&quot;&gt;SAM Optional Fields Specification&lt;/a&gt; the NM field is &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Edit distance to the reference, including ambiguous bases but excluding clipping&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Assuming both the MD and CIGAR are present, is the edit distance simply the number of characters &lt;code&gt;[A-Z]&lt;/code&gt; appearing in the MD field plus the number of bases inserted (&lt;code&gt;xI&lt;/code&gt;, if any) from the CIGAR string? Are there any other complications? &lt;/p&gt;&#xA;" OwnerUserId="272" LastActivityDate="2017-06-14T06:43:46.623" Title="Is the optional SAM NM field strictly computable from the MD and CIGAR?" Tags="|file-formats|sam|cigar|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="721" PostTypeId="2" ParentId="395" CreationDate="2017-06-14T01:38:01.883" Score="3" Body="&lt;p&gt;While better methods of evaluating your clusters would be to use an external dataset or a dataset with known truth, there are a variety of internal validation metrics that can be used to compare clustering solutions without another dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are a few metrics:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Davies-Bouldin Index&lt;/li&gt;&#xA;&lt;li&gt;Calinski-Harabasz Index&lt;/li&gt;&#xA;&lt;li&gt;Root-Mean-Square Standard Deviation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Many more can be found in this clustering review: &lt;a href=&quot;http://stke.sciencemag.org/content/9/432/re6&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://stke.sciencemag.org/content/9/432/re6&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These internal validation metrics grade your clustering solution based on three measures: compactness, connectedness, and separation. When using these metrics to compare clustering solutions, be sure to consider which metric is appropriate for your results as some algorithms work by optimizing certain measures.&lt;/p&gt;&#xA;" OwnerUserId="853" LastActivityDate="2017-06-14T01:38:01.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="722" PostTypeId="1" AcceptedAnswerId="825" CreationDate="2017-06-14T03:16:07.773" Score="9" ViewCount="944" Body="&lt;p&gt;I have a dataset of Oxford Nanopore cDNA reads. Many of my reads are full-length or close to full-length transcripts, and I and am interested in examining alternative splicing. For this, I would like to begin by visualising my reads and comparing variants qualitatively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried visualising my data in both IGV and SeqMonk, but neither has given a satisfying result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;IGV&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zM3Yn.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zM3Yn.png&quot; alt=&quot;IGV Visualisation of NOTCH2&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IGV shows some links between aligned sections, shown as fine blue lines and thick black lines. I cannot work out what the difference between these lines is, and even more confusingly, we have numerous alignments where two aligned exons come from the same sequence but are not joined by any line. This means that, to confirm or deny a spliced variant, I need to manually examine the read ID of each exon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;SeqMonk&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ugYDT.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ugYDT.png&quot; alt=&quot;SeqMonk Visualisation of NOTCH2&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SeqMonk gives a much cleaner visualisation, but unfortunately shows no links between aligned exons, and worse, I cannot find the read IDs. This means there is functionally (as far as I can see) no way to tell which exons come from the same read.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT: Sashimi Plot&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As recommended in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/723/163&quot;&gt;this answer&lt;/a&gt;, I have tried using IGV's Sashimi Plot. Unfortunately, I believe the high error rate and relatively low coverage is causing this to create somewhat confusing output. The Sashimi Plot shows mostly nearly every junction to have just a single read supporting it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5ah7z.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5ah7z.png&quot; alt=&quot;IGV Sashimi Plot of NOTCH2&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a visualisation tool which allows simple examination of splicing of full-length transcripts?&lt;/p&gt;&#xA;" OwnerUserId="163" LastEditorUserId="163" LastEditDate="2017-06-20T02:13:57.857" LastActivityDate="2017-06-20T02:13:57.857" Title="Visualisation of long read RNA-Seq splicing" Tags="|rna-seq|nanopore|read-mapping|visualization|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="723" PostTypeId="2" ParentId="722" CreationDate="2017-06-14T03:28:48.103" Score="3" Body="&lt;p&gt;The &lt;a href=&quot;https://software.broadinstitute.org/software/igv/Sashimi&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sashmi Plot&lt;/a&gt; feature built into IGV. It gives a nice summary of the spliced transcripts and the coverage of each exon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6MMjw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6MMjw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="64" LastActivityDate="2017-06-14T03:28:48.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="724" PostTypeId="1" CreationDate="2017-06-14T04:12:13.013" Score="2" ViewCount="75" Body="&lt;p&gt;I'm working on a project where I have to assemble sequences generated by RADseq. At the end I hope to compare two species of woodpeckers in Sri Lanka by using SNPs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to assemble it using Stacks pipeline, but it crushed on the way with:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;*** Error in `ustacks': malloc(): memory corruption: 0x000055db58024be0 ***&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Are there are any other tools or pipelines that I can use for assembling my data instead?&lt;/p&gt;&#xA;" OwnerUserId="409" LastEditorUserId="48" LastEditDate="2017-11-16T11:05:35.393" LastActivityDate="2017-11-16T11:05:35.393" Title="Assembling sequence data generated by RADseq" Tags="|ngs|assembly|radseq|" AnswerCount="0" CommentCount="9" ClosedDate="2018-01-18T09:41:21.150" ContentLicense="CC BY-SA 3.0" />
  <row Id="725" PostTypeId="2" ParentId="722" CreationDate="2017-06-14T05:23:42.993" Score="5" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xS23U.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xS23U.png&quot; alt=&quot;GenVision Pro Sashimi Plot&quot;&gt;&lt;/a&gt;DNASTAR's software is for purchase, but high quality. GenVision Pro does genomic visualization, including Sashimi plots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: not sure why this answer is being downvoted, unless it's because the software isn't free. OP has tried IGV and SeqMonk, I mentioned an alternative he might not have heard of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a video demonstrating the use of Sashimi plots in GenVision Pro:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.dnastar.com/t-support-videos.aspx?video=YJvcERoSIsg&quot; rel=&quot;noreferrer&quot;&gt;http://www.dnastar.com/t-support-videos.aspx?video=YJvcERoSIsg&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="138" LastEditorUserId="138" LastEditDate="2017-06-14T22:46:43.843" LastActivityDate="2017-06-14T22:46:43.843" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="726" PostTypeId="2" ParentId="719" CreationDate="2017-06-14T06:43:46.623" Score="4" Body="&lt;p&gt;Assuming both the MD and CIGAR are present &lt;strong&gt;and correct&lt;/strong&gt;, then yes, you can parse both to get the edit distance (&lt;code&gt;NM&lt;/code&gt; auxiliary tag). One big caveat to this is that there's a reason that the &lt;code&gt;samtools calmd&lt;/code&gt; command exists, since it's historically been the case that not all aligners have output correct MD strings. It's rare for the CIGAR string to be wrong and that'd be more of a catastrophic error on the part of an aligner. For what it's worth, if the &lt;code&gt;NM&lt;/code&gt; auxiliary is absent on a given alignment but present on others produced by the same aligner then it's fair to assume &lt;code&gt;NM:i:0&lt;/code&gt; for a given alignment by default (many aligners only produce &lt;code&gt;NM:i:XXX&lt;/code&gt; if the edit distance is at least 1).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-06-14T06:43:46.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="727" PostTypeId="5" CreationDate="2017-06-14T09:40:29.803" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-06-14T09:40:29.803" LastActivityDate="2017-06-14T09:40:29.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="728" PostTypeId="4" CreationDate="2017-06-14T09:40:29.803" Score="0" Body="Use this tag for questions related to single-cell RNA-seq." OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-06-14T10:58:45.880" LastActivityDate="2017-06-14T10:58:45.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="729" PostTypeId="2" ParentId="673" CreationDate="2017-06-14T09:50:01.067" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/26066708&quot; rel=&quot;nofollow noreferrer&quot;&gt;TRUSST&lt;/a&gt; is a manually curated database with around 790 TFs and their target genes. &lt;a href=&quot;http://www.grnpedia.org/trrust/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TRUSST Website&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="161" LastActivityDate="2017-06-14T09:50:01.067" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="731" PostTypeId="1" AcceptedAnswerId="758" CreationDate="2017-06-14T11:10:24.310" Score="7" ViewCount="2010" Body="&lt;p&gt;I have a list of 100 genes that are called as hits in a genetic screening. I want to have a network of the interactions between the proteins of these 100 genes. I am using both &lt;a href=&quot;https://string-db.org/&quot; rel=&quot;noreferrer&quot;&gt;STRINGdb web&lt;/a&gt; and its &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/STRINGdb.html&quot; rel=&quot;noreferrer&quot;&gt;R API&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both situations you can select the threshold of confidence of interaction (default 0.4) for displaying the network. Changing this parameter drastically changes the network that I am generating. I have read the FAQ section of STRINGdb and they recommend to choose some arbitrary number based on the number of interactions you need for you analysis. If I use the default threshold, I have few interactions and I don't know if lowering the threshold would be correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, my question is, is there any established threshold based on habitual user's experience used for that, beyond the recommendation of setting an arbitrary one? &lt;/p&gt;&#xA;" OwnerUserId="678" LastEditorUserId="292" LastEditDate="2017-06-14T12:10:48.763" LastActivityDate="2021-06-08T13:39:03.683" Title="How to select a cutoff for interaction confidence in STRINGdb?" Tags="|proteins|public-databases|networks|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="732" PostTypeId="1" AcceptedAnswerId="735" CreationDate="2017-06-14T11:26:35.647" Score="2" ViewCount="421" Body="&lt;p&gt;I've been asked to evaluate whether the MinION will be sufficient to distinguish between 20bp CRISPR guide RNAs from &lt;a href=&quot;https://www.addgene.org/pooled-library/zhang-mouse-gecko-v2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the GeCKO v2 set&lt;/a&gt;. I know that this set has some &lt;a href=&quot;https://groups.google.com/forum/#!topic/crispr/qxEi_jL7jls&quot; rel=&quot;nofollow noreferrer&quot;&gt;exactly identical sequences&lt;/a&gt;, which wouldn't be distinguishable regardless of the sequencing method used, so I would like to know whether any two non-indentical sequences differ by less than 15% of the 20bp (i.e. only 3 bases different).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are around 130k 20bp sequences in the dataset, and I'm pretty sure that an all-vs-all approach is not feasible, but perhaps there's some trick that I'm missing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I thought of doing was looking at dimer and trimer/codon counts to hunt down sequences that were most similar. I found about 100 sequences that shared the same trimer count signature (but were not identical), and within this group found [only] one pair with 85% identity; all the remainder were 70% or less.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;... but I'm having trouble ignoring the doubt I have that these two sequences are the most similar. It might be possible that there are sequences in the dataset that don't have identical trimer count signatures, but differ by less than three bases. I'll give a very obvious example (which isn't in the dataset):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1: AAAAA AAAAA AAAAA AAAAA&#xA;2: AAAAC AAAAA AAAAA AAAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The trimer counts for these two sequences are different (Sequence 1 -- AAA: 18; Sequence 2 -- AAA: 15; AAC: 1; ACA: 1; CAA: 1), and yet the sequences are only 1 base different (which might be a problem for the MinION). How can I work out if any such sequence pairs exist in the dataset?&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-14T15:00:55.240" Title="How do I find the most similar sequences from a large set of short sequences?" Tags="|minion|crispr|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="733" PostTypeId="2" ParentId="732" CreationDate="2017-06-14T12:20:49.173" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I'm pretty sure that an all-vs-all approach is not feasible, but perhaps there's some trick that I'm missing.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The &quot;trick&quot; is that I hadn't actually tried this, and Bowtie2 happens to be quite good at doing this. When there are other similar matches, the MAPQ score that Bowtie2 produces are reduced, so all that is needed is to identify those sequences with less than a MAPQ of 42. I've found 2810 such sequences that share some similarity to others (2416 if I exclude reverse-complement matches):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bowtie2-build Mouse_GeCKOv2_Library.fasta Mouse_GeCKOv2_Library.fasta&#xA;$ bowtie2 -f -x Mouse_GeCKOv2_Library.fasta -U Mouse_GeCKOv2_Library.fasta | &#xA;   samtools view | awk '{if($5 != 42){print &quot;&amp;gt;&quot;$1&quot;\n&quot;$10}}' &amp;gt; similar_seqs.fasta&#xA;125795 reads; of these:&#xA;  125795 (100.00%) were unpaired; of these:&#xA;    1 (0.00%) aligned 0 times&#xA;    122982 (97.76%) aligned exactly 1 time&#xA;    2812 (2.24%) aligned &amp;gt;1 times&#xA;100.00% overall alignment rate&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-14T12:20:49.173" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="734" PostTypeId="2" ParentId="732" CreationDate="2017-06-14T12:42:31.153" Score="2" Body="&lt;p&gt;For this application, you could probably also do something like calculate the Hamming distances between all of the strings in an all-vs-all approach (it should not take too long or too much overhead). You could use something like the Hamming distance tools in &lt;a href=&quot;https://julialang.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Julia&lt;/a&gt;. Here is an example of what I mean (using Julia):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using StringDistances&#xA;&#xA;k = [&quot;AATTGGCC&quot;, &quot;AATTGGCA&quot;, &quot;AATTCCGG&quot;]&#xA;&#xA;for s in k&#xA;    for y in k&#xA;        print(s, &quot; compared to &quot;, y, &quot;:  &quot;, compare(Hamming(), s, y), &quot;\n&quot;)&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AATTGGCC compared to AATTGGCC:  1.0&#xA;AATTGGCC compared to AATTGGCA:  0.875&#xA;AATTGGCC compared to AATTCCGG:  0.5&#xA;AATTGGCA compared to AATTGGCC:  0.875&#xA;AATTGGCA compared to AATTGGCA:  1.0&#xA;AATTGGCA compared to AATTCCGG:  0.5&#xA;AATTCCGG compared to AATTGGCC:  0.5&#xA;AATTCCGG compared to AATTGGCA:  0.5&#xA;AATTCCGG compared to AATTCCGG:  1.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is just a simple small example but the core is there. Should not be too bad scaling it up. Hope this helps and provides another helpful option. :)&lt;/p&gt;&#xA;" OwnerUserId="824" LastEditorUserId="298" LastEditDate="2017-06-14T15:00:55.240" LastActivityDate="2017-06-14T15:00:55.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="735" PostTypeId="2" ParentId="732" CreationDate="2017-06-14T14:14:33.443" Score="3" Body="&lt;p&gt;130k * 20bp is a small data set. At this scale, SSE2 Smith-Waterman may work well:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-shell prettyprint-override&quot;&gt;&lt;code&gt;git clone https://github.com/attractivechaos/klib&#xA;cd klib &amp;amp;&amp;amp; gcc -O2 -D_KSW_MAIN ksw.c -o ksw -lz&#xA;./ksw -a1 -b1 -q1 -r1 -t14 20bp.fa 20bp.fa &amp;gt; out.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I simulated 130k 20bp reads from E. coli K-12. The last command line takes about ~2 CPU hours based on the partial output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible to make this much faster by using 5bp seeds. You can also try ssearch and fasta from the &lt;a href=&quot;http://fasta.bioch.virginia.edu/fasta_www2/fasta_list2.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;fasta aligner package&lt;/a&gt;. They are slower, probably because they are not optimized for such input. Computing Hamming distance with SSE2 will be much faster, but is probably of little use as it does not allow gaps. There are also Gene Myers' edit-distance based algorithms, which can be faster than SSE2-SW.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bowtie2 (as well as most short-read mappers) won't work well at the default setting. Bowtie2 uses long exact seeds. It will miss many 1-mismatch hits, let alone 3-mismatch ones. The number you get from bowtie2 is an underestimate. You might be able to tune bowtie2, but to find 3-mismatch hits, you have to make it a lot slower.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-06-14T14:14:33.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="736" PostTypeId="1" AcceptedAnswerId="737" CreationDate="2017-06-14T14:22:55.960" Score="9" ViewCount="559" Body="&lt;p&gt;I'm faced with having to align many (some 100s) bacterial genomes, where the genome length is in the millions. Obviously, this is beyond normal alignment techniques and it's unclear to me what the best practice is for such circumstances:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;conventional alignment on a very powerful computer with lots of memory&lt;/li&gt;&#xA;&lt;li&gt;break up the genome into smaller fragments and align them individually&lt;/li&gt;&#xA;&lt;li&gt;some exotic different procedure&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What possible avenues of attack are there?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I've attempted to use Mafft and Clustal with little success)&lt;/p&gt;&#xA;" OwnerUserId="377" LastActivityDate="2018-01-16T20:29:51.697" Title="Aligning many long sequences" Tags="|sequence-alignment|genome|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="737" PostTypeId="2" ParentId="736" CreationDate="2017-06-14T15:12:37.680" Score="9" Body="&lt;p&gt;Whole genome aliment can be done using &lt;a href=&quot;http://darlinglab.org/mauve/user-guide/progressivemauve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Progressive Mauve&lt;/a&gt;, &lt;a href=&quot;http://last.cbrc.jp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LAST&lt;/a&gt; or &lt;a href=&quot;https://github.com/mummer4/mummer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mummer&lt;/a&gt;. For bacteria I used Mauve since it has also very nice visualisation engine. A very new tool is &lt;a href=&quot;https://github.com/lh3/minimap2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Minimap2&lt;/a&gt;, a super fast mapper that supposed beside read mapping be able to handle reference vs reference. However, I do not know how performance of it compares to the tools mentioned above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested in a rough idea of the shared genome regions, you can use &lt;a href=&quot;https://github.com/zeeev/bevel&quot; rel=&quot;nofollow noreferrer&quot;&gt;bevel&lt;/a&gt;. Bevel is not really an aligner, it is more like a dot-plot, but it is super fast (even for mammalian sized genomes).&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2018-01-16T20:29:51.697" LastActivityDate="2018-01-16T20:29:51.697" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="738" PostTypeId="2" ParentId="702" CreationDate="2017-06-14T15:33:46.443" Score="2" Body="&lt;p&gt;Since you are using R, you probably don't want to use scikit-learn, which is for Python. However, there is a similar R library &lt;a href=&quot;https://mlr-org.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;mlr&lt;/a&gt; (&quot;R package to make machine learning in R easy&quot;) that provides a unified interface to all popular machine learning methods. Check their &lt;a href=&quot;https://mlr-org.github.io/mlr-tutorial/devel/html/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; on how to get started.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2017-06-14T15:33:46.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="739" PostTypeId="2" ParentId="703" CreationDate="2017-06-14T16:34:25.797" Score="3" Body="&lt;p&gt;I do not think there is a simple &quot;yes&quot; or &quot;no&quot; answer here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good starting point would be, as you suggest, use all the genes and assess the results in the light of the marker genes and expected results. This could both serve as as good quality control as well as give you overview of all the processes happening in the cells. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the marker genes effect and biological question you may then want to remove the marker genes with potential ordering effect, or even other genes, e.g. by GO terms or pathways. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You most likely want to account for the cell cycle phases as well. And check for other technical factors. I recommend Bioconductor pipeline to get inspired &lt;a href=&quot;https://www.bioconductor.org/help/workflows/simpleSingleCell/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioconductor.org/help/workflows/simpleSingleCell/&lt;/a&gt; when it comes to scRNA-seq analyses. &lt;/p&gt;&#xA;" OwnerUserId="375" LastActivityDate="2017-06-14T16:34:25.797" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="740" PostTypeId="2" ParentId="731" CreationDate="2017-06-14T17:09:49.490" Score="5" Body="&lt;p&gt;This depends on what you are trying to do and whether you value specificity over sensitivity. We can't tell you since it is entirely dependent on the biological question you want to answer.&lt;/p&gt;&#xA;&lt;p&gt;However, I would recommend two things:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Don't use STRING. The creators of STRING made the choice to value sensitivity over all else, so they include &lt;em&gt;any&lt;/em&gt; interaction they can get their hands on. That is sometimes fine, depending on what you want to do, but is more often a problem. Personally, I tend to avoid STRING as much as possible and instead use more curated databases like &lt;a href=&quot;http://cicblade.dep.usal.es:8080/APID/init.action&quot; rel=&quot;nofollow noreferrer&quot;&gt;APID&lt;/a&gt; or &lt;a href=&quot;https://www.ebi.ac.uk/intact/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IntAct&lt;/a&gt;. You can find a list of many PPI databases in the &lt;a href=&quot;http://www.ebi.ac.uk/Tools/webservices/psicquic/view/main.xhtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;EBI's Psiqcuick View&lt;/a&gt; page. You might also be interested in my answer &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/577/298&quot;&gt;here&lt;/a&gt; which gives an example script using APID which can easily be modified to query other DBs.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Whatever you use, I recommend you filter by detection method. There are many different interaction detection methods ranging from those that only identify direct, binary interactions between two proteins (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Two-hybrid_screening&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yeast two-hybrid&lt;/a&gt;), those that also find proteins that are in the same complex even if there is no &lt;em&gt;direct&lt;/em&gt; interaction between them (e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Chromatin_immunoprecipitation&quot; rel=&quot;nofollow noreferrer&quot;&gt;ChIP&lt;/a&gt;), to various non-experimental methods which are used to &lt;em&gt;infer&lt;/em&gt; interactions.&lt;/p&gt;&#xA;&lt;p&gt;Luckily, all of these have been collected into a controlled vocabulary which you can see in the &lt;a href=&quot;http://www.ebi.ac.uk/ols/ontologies/MI/terms?obo_id=MI:0001&quot; rel=&quot;nofollow noreferrer&quot;&gt;Interaction Detection Method&lt;/a&gt; Ontology Lookup Service page. Go there, collect the interaction detection methods you consider good enough for whatever you want to do and filter your interactions using those.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;For example, for a high quality human interactome I built for a project I used to work on, I was only interested in direct binary interactions, so I only kept interactions detected by the following methods:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    &amp;quot;MI:0008&amp;quot; =&amp;gt; &amp;quot;array technology&amp;quot;,&#xA;    &amp;quot;MI:0009&amp;quot; =&amp;gt; &amp;quot;bacterial display&amp;quot;,&#xA;    &amp;quot;MI:0010&amp;quot; =&amp;gt; &amp;quot;beta galactosidase complementation&amp;quot;,&#xA;    &amp;quot;MI:0011&amp;quot; =&amp;gt; &amp;quot;beta lactamase complementation&amp;quot;,&#xA;    &amp;quot;MI:0012&amp;quot; =&amp;gt; &amp;quot;bioluminescence resonance energy transfer&amp;quot;,&#xA;    &amp;quot;MI:0013&amp;quot; =&amp;gt; &amp;quot;biophysical&amp;quot;,&#xA;    &amp;quot;MI:0014&amp;quot; =&amp;gt; &amp;quot;adenylate cyclase complementation&amp;quot;,&#xA;    &amp;quot;MI:0016&amp;quot; =&amp;gt; &amp;quot;circular dichroism&amp;quot;,&#xA;    &amp;quot;MI:0017&amp;quot; =&amp;gt; &amp;quot;classical fluorescence spectroscopy&amp;quot;,&#xA;    &amp;quot;MI:0018&amp;quot; =&amp;gt; &amp;quot;two hybrid&amp;quot;,&#xA;    &amp;quot;MI:0020&amp;quot; =&amp;gt; &amp;quot;transmission electron microscopy&amp;quot;,&#xA;    &amp;quot;MI:0030&amp;quot; =&amp;gt; &amp;quot;cross-linking study&amp;quot;,&#xA;    &amp;quot;MI:0031&amp;quot; =&amp;gt; &amp;quot;protein cross-linking with a bifunctional reagent&amp;quot;,&#xA;    &amp;quot;MI:0034&amp;quot; =&amp;gt; &amp;quot;display technology&amp;quot;,&#xA;    &amp;quot;MI:0040&amp;quot; =&amp;gt; &amp;quot;electron microscopy&amp;quot;,&#xA;    &amp;quot;MI:0041&amp;quot; =&amp;gt; &amp;quot;electron nuclear double resonance&amp;quot;,&#xA;    &amp;quot;MI:0042&amp;quot; =&amp;gt; &amp;quot;electron paramagnetic resonance&amp;quot;,&#xA;    &amp;quot;MI:0043&amp;quot; =&amp;gt; &amp;quot;electron resonance&amp;quot;,&#xA;    &amp;quot;MI:0047&amp;quot; =&amp;gt; &amp;quot;far western blotting&amp;quot;,&#xA;    &amp;quot;MI:0048&amp;quot; =&amp;gt; &amp;quot;filamentous phage display&amp;quot;,&#xA;    &amp;quot;MI:0049&amp;quot; =&amp;gt; &amp;quot;filter binding&amp;quot;,&#xA;    &amp;quot;MI:0051&amp;quot; =&amp;gt; &amp;quot;fluorescence technology&amp;quot;,&#xA;    &amp;quot;MI:0052&amp;quot; =&amp;gt; &amp;quot;fluorescence correlation spectroscopy&amp;quot;,&#xA;    &amp;quot;MI:0053&amp;quot; =&amp;gt; &amp;quot;fluorescence polarization spectroscopy&amp;quot;,&#xA;    &amp;quot;MI:0055&amp;quot; =&amp;gt; &amp;quot;fluorescent resonance energy transfer&amp;quot;,&#xA;    &amp;quot;MI:0065&amp;quot; =&amp;gt; &amp;quot;isothermal titration calorimetry&amp;quot;,&#xA;    &amp;quot;MI:0066&amp;quot; =&amp;gt; &amp;quot;lambda phage display&amp;quot;,&#xA;    &amp;quot;MI:0073&amp;quot; =&amp;gt; &amp;quot;mrna display&amp;quot;,&#xA;    &amp;quot;MI:0081&amp;quot; =&amp;gt; &amp;quot;peptide array&amp;quot;,&#xA;    &amp;quot;MI:0084&amp;quot; =&amp;gt; &amp;quot;phage display&amp;quot;,&#xA;    &amp;quot;MI:0089&amp;quot; =&amp;gt; &amp;quot;protein array&amp;quot;,&#xA;    &amp;quot;MI:0090&amp;quot; =&amp;gt; &amp;quot;protein complementation assay&amp;quot;,&#xA;    &amp;quot;MI:0091&amp;quot; =&amp;gt; &amp;quot;chromatography technology&amp;quot;,&#xA;    &amp;quot;MI:0092&amp;quot; =&amp;gt; &amp;quot;protein in situ array&amp;quot;,&#xA;    &amp;quot;MI:0095&amp;quot; =&amp;gt; &amp;quot;proteinchip(r) on a surface-enhanced laser desorption/ionization&amp;quot;,&#xA;    &amp;quot;MI:0097&amp;quot; =&amp;gt; &amp;quot;reverse ras recruitment system&amp;quot;,&#xA;    &amp;quot;MI:0098&amp;quot; =&amp;gt; &amp;quot;ribosome display&amp;quot;,&#xA;    &amp;quot;MI:0099&amp;quot; =&amp;gt; &amp;quot;scintillation proximity assay&amp;quot;,&#xA;    &amp;quot;MI:0107&amp;quot; =&amp;gt; &amp;quot;surface plasmon resonance&amp;quot;,&#xA;    &amp;quot;MI:0108&amp;quot; =&amp;gt; &amp;quot;t7 phage display&amp;quot;,&#xA;    &amp;quot;MI:0111&amp;quot; =&amp;gt; &amp;quot;dihydrofolate reductase reconstruction&amp;quot;,&#xA;    &amp;quot;MI:0112&amp;quot; =&amp;gt; &amp;quot;ubiquitin reconstruction&amp;quot;,&#xA;    &amp;quot;MI:0114&amp;quot; =&amp;gt; &amp;quot;x-ray crystallography&amp;quot;,&#xA;    &amp;quot;MI:0115&amp;quot; =&amp;gt; &amp;quot;yeast display&amp;quot;,&#xA;    &amp;quot;MI:0226&amp;quot; =&amp;gt; &amp;quot;ion exchange chromatography&amp;quot;,&#xA;    &amp;quot;MI:0227&amp;quot; =&amp;gt; &amp;quot;reverse phase chromatography&amp;quot;,&#xA;    &amp;quot;MI:0231&amp;quot; =&amp;gt; &amp;quot;mammalian protein protein interaction trap&amp;quot;,&#xA;    &amp;quot;MI:0232&amp;quot; =&amp;gt; &amp;quot;transcriptional complementation assay&amp;quot;,&#xA;    &amp;quot;MI:0255&amp;quot; =&amp;gt; &amp;quot;post transcriptional interference&amp;quot;,&#xA;    &amp;quot;MI:0369&amp;quot; =&amp;gt; &amp;quot;lex-a dimerization assay&amp;quot;,&#xA;    &amp;quot;MI:0370&amp;quot; =&amp;gt; &amp;quot;tox-r dimerization assay&amp;quot;,&#xA;    &amp;quot;MI:0397&amp;quot; =&amp;gt; &amp;quot;two hybrid array&amp;quot;,&#xA;    &amp;quot;MI:0398&amp;quot; =&amp;gt; &amp;quot;two hybrid pooling approach&amp;quot;,&#xA;    &amp;quot;MI:0399&amp;quot; =&amp;gt; &amp;quot;two hybrid fragment pooling approach&amp;quot;,&#xA;    &amp;quot;MI:0400&amp;quot; =&amp;gt; &amp;quot;affinity technology&amp;quot;,&#xA;    &amp;quot;MI:0401&amp;quot; =&amp;gt; &amp;quot;biochemical&amp;quot;,&#xA;    &amp;quot;MI:0405&amp;quot; =&amp;gt; &amp;quot;competition binding&amp;quot;,&#xA;    &amp;quot;MI:0406&amp;quot; =&amp;gt; &amp;quot;deacetylase assay&amp;quot;,&#xA;    &amp;quot;MI:0410&amp;quot; =&amp;gt; &amp;quot;electron tomography&amp;quot;,&#xA;    &amp;quot;MI:0411&amp;quot; =&amp;gt; &amp;quot;enzyme linked immunosorbent assay&amp;quot;,&#xA;    &amp;quot;MI:0415&amp;quot; =&amp;gt; &amp;quot;enzymatic study&amp;quot;,&#xA;    &amp;quot;MI:0416&amp;quot; =&amp;gt; &amp;quot;fluorescence microscopy&amp;quot;,&#xA;    &amp;quot;MI:0419&amp;quot; =&amp;gt; &amp;quot;gtpase assay&amp;quot;,&#xA;    &amp;quot;MI:0420&amp;quot; =&amp;gt; &amp;quot;kinase homogeneous time resolved fluorescence&amp;quot;,&#xA;    &amp;quot;MI:0423&amp;quot; =&amp;gt; &amp;quot;in-gel kinase assay&amp;quot;,&#xA;    &amp;quot;MI:0424&amp;quot; =&amp;gt; &amp;quot;protein kinase assay&amp;quot;,&#xA;    &amp;quot;MI:0425&amp;quot; =&amp;gt; &amp;quot;kinase scintillation proximity assay&amp;quot;,&#xA;    &amp;quot;MI:0426&amp;quot; =&amp;gt; &amp;quot;light microscopy&amp;quot;,&#xA;    &amp;quot;MI:0428&amp;quot; =&amp;gt; &amp;quot;imaging technique&amp;quot;,&#xA;    &amp;quot;MI:0432&amp;quot; =&amp;gt; &amp;quot;one hybrid&amp;quot;,&#xA;    &amp;quot;MI:0434&amp;quot; =&amp;gt; &amp;quot;phosphatase assay&amp;quot;,&#xA;    &amp;quot;MI:0435&amp;quot; =&amp;gt; &amp;quot;protease assay&amp;quot;,&#xA;    &amp;quot;MI:0437&amp;quot; =&amp;gt; &amp;quot;protein three hybrid&amp;quot;,&#xA;    &amp;quot;MI:0440&amp;quot; =&amp;gt; &amp;quot;saturation binding&amp;quot;,&#xA;    &amp;quot;MI:0508&amp;quot; =&amp;gt; &amp;quot;deacetylase radiometric assay&amp;quot;,&#xA;    &amp;quot;MI:0509&amp;quot; =&amp;gt; &amp;quot;phosphatase homogeneous time resolved fluorescence&amp;quot;,&#xA;    &amp;quot;MI:0510&amp;quot; =&amp;gt; &amp;quot;homogeneous time resolved fluorescence&amp;quot;,&#xA;    &amp;quot;MI:0511&amp;quot; =&amp;gt; &amp;quot;protease homogeneous time resolved fluorescence&amp;quot;,&#xA;    &amp;quot;MI:0512&amp;quot; =&amp;gt; &amp;quot;zymography&amp;quot;,&#xA;    &amp;quot;MI:0513&amp;quot; =&amp;gt; &amp;quot;collagen film assay&amp;quot;,&#xA;    &amp;quot;MI:0514&amp;quot; =&amp;gt; &amp;quot;in gel phosphatase assay&amp;quot;,&#xA;    &amp;quot;MI:0515&amp;quot; =&amp;gt; &amp;quot;methyltransferase assay&amp;quot;,&#xA;    &amp;quot;MI:0516&amp;quot; =&amp;gt; &amp;quot;methyltransferase radiometric assay&amp;quot;,&#xA;    &amp;quot;MI:0655&amp;quot; =&amp;gt; &amp;quot;lambda repressor two hybrid&amp;quot;,&#xA;    &amp;quot;MI:0657&amp;quot; =&amp;gt; &amp;quot;systematic evolution of ligands by exponential enrichment&amp;quot;,&#xA;    &amp;quot;MI:0678&amp;quot; =&amp;gt; &amp;quot;antibody array&amp;quot;,&#xA;    &amp;quot;MI:0695&amp;quot; =&amp;gt; &amp;quot;sandwich immunoassay&amp;quot;,&#xA;    &amp;quot;MI:0696&amp;quot; =&amp;gt; &amp;quot;polymerase assay&amp;quot;,&#xA;    &amp;quot;MI:0726&amp;quot; =&amp;gt; &amp;quot;reverse two hybrid&amp;quot;,&#xA;    &amp;quot;MI:0727&amp;quot; =&amp;gt; &amp;quot;lexa b52 complementation&amp;quot;,&#xA;    &amp;quot;MI:0728&amp;quot; =&amp;gt; &amp;quot;gal4 vp16 complementation&amp;quot;,&#xA;    &amp;quot;MI:0809&amp;quot; =&amp;gt; &amp;quot;bimolecular fluorescence complementation&amp;quot;,&#xA;    &amp;quot;MI:0813&amp;quot; =&amp;gt; &amp;quot;proximity enzyme linked immunosorbent assay&amp;quot;,&#xA;    &amp;quot;MI:0824&amp;quot; =&amp;gt; &amp;quot;x-ray powder diffraction&amp;quot;,&#xA;    &amp;quot;MI:0825&amp;quot; =&amp;gt; &amp;quot;x-ray fiber diffraction&amp;quot;,&#xA;    &amp;quot;MI:0827&amp;quot; =&amp;gt; &amp;quot;x-ray tomography&amp;quot;,&#xA;    &amp;quot;MI:0841&amp;quot; =&amp;gt; &amp;quot;phosphotransferase assay&amp;quot;,&#xA;    &amp;quot;MI:0870&amp;quot; =&amp;gt; &amp;quot;demethylase assay&amp;quot;,&#xA;    &amp;quot;MI:0872&amp;quot; =&amp;gt; &amp;quot;atomic force microscopy&amp;quot;,&#xA;    &amp;quot;MI:0879&amp;quot; =&amp;gt; &amp;quot;nucleoside triphosphatase assay&amp;quot;,&#xA;    &amp;quot;MI:0880&amp;quot; =&amp;gt; &amp;quot;atpase assay&amp;quot;,&#xA;    &amp;quot;MI:0887&amp;quot; =&amp;gt; &amp;quot;histone acetylase assay&amp;quot;,&#xA;    &amp;quot;MI:0889&amp;quot; =&amp;gt; &amp;quot;acetylase assay&amp;quot;,&#xA;    &amp;quot;MI:0892&amp;quot; =&amp;gt; &amp;quot;solid phase assay&amp;quot;,&#xA;    &amp;quot;MI:0894&amp;quot; =&amp;gt; &amp;quot;electron diffraction&amp;quot;,&#xA;    &amp;quot;MI:0895&amp;quot; =&amp;gt; &amp;quot;protein kinase A complementation&amp;quot;,&#xA;    &amp;quot;MI:0899&amp;quot; =&amp;gt; &amp;quot;p3 filamentous phage display&amp;quot;,&#xA;    &amp;quot;MI:0900&amp;quot; =&amp;gt; &amp;quot;p8 filamentous phage display&amp;quot;,&#xA;    &amp;quot;MI:0905&amp;quot; =&amp;gt; &amp;quot;amplified luminescent proximity homogeneous assay&amp;quot;,&#xA;    &amp;quot;MI:0916&amp;quot; =&amp;gt; &amp;quot;lexa vp16 complementation&amp;quot;,&#xA;    &amp;quot;MI:0920&amp;quot; =&amp;gt; &amp;quot;ribonuclease assay&amp;quot;,&#xA;    &amp;quot;MI:0921&amp;quot; =&amp;gt; &amp;quot;surface plasmon resonance array&amp;quot;,&#xA;    &amp;quot;MI:0946&amp;quot; =&amp;gt; &amp;quot;ping&amp;quot;,&#xA;    &amp;quot;MI:0947&amp;quot; =&amp;gt; &amp;quot;bead aggregation assay&amp;quot;,&#xA;    &amp;quot;MI:0949&amp;quot; =&amp;gt; &amp;quot;gdp/gtp exchange assay&amp;quot;,&#xA;    &amp;quot;MI:0953&amp;quot; =&amp;gt; &amp;quot;polymerization&amp;quot;,&#xA;    &amp;quot;MI:0968&amp;quot; =&amp;gt; &amp;quot;biosensor&amp;quot;,&#xA;    &amp;quot;MI:0969&amp;quot; =&amp;gt; &amp;quot;bio-layer interferometry&amp;quot;,&#xA;    &amp;quot;MI:0972&amp;quot; =&amp;gt; &amp;quot;phosphopantetheinylase assay&amp;quot;,&#xA;    &amp;quot;MI:0976&amp;quot; =&amp;gt; &amp;quot;total internal reflection fluorescence spectroscopy&amp;quot;,&#xA;    &amp;quot;MI:0979&amp;quot; =&amp;gt; &amp;quot;oxidoreductase assay&amp;quot;,&#xA;    &amp;quot;MI:0984&amp;quot; =&amp;gt; &amp;quot;deaminase assay&amp;quot;,&#xA;    &amp;quot;MI:0989&amp;quot; =&amp;gt; &amp;quot;amidase assay&amp;quot;,&#xA;    &amp;quot;MI:0990&amp;quot; =&amp;gt; &amp;quot;cleavage assay&amp;quot;,&#xA;    &amp;quot;MI:0991&amp;quot; =&amp;gt; &amp;quot;lipid cleavage assay&amp;quot;,&#xA;    &amp;quot;MI:0992&amp;quot; =&amp;gt; &amp;quot;defarnesylase assay&amp;quot;,&#xA;    &amp;quot;MI:0993&amp;quot; =&amp;gt; &amp;quot;degeranylase assay&amp;quot;,&#xA;    &amp;quot;MI:0994&amp;quot; =&amp;gt; &amp;quot;demyristoylase assay&amp;quot;,&#xA;    &amp;quot;MI:0995&amp;quot; =&amp;gt; &amp;quot;depalmitoylase assay&amp;quot;,&#xA;    &amp;quot;MI:0996&amp;quot; =&amp;gt; &amp;quot;deformylase assay&amp;quot;,&#xA;    &amp;quot;MI:0997&amp;quot; =&amp;gt; &amp;quot;ubiquitinase assay&amp;quot;,&#xA;    &amp;quot;MI:0998&amp;quot; =&amp;gt; &amp;quot;deubiquitinase assay&amp;quot;,&#xA;    &amp;quot;MI:0999&amp;quot; =&amp;gt; &amp;quot;formylase assay&amp;quot;,&#xA;    &amp;quot;MI:1000&amp;quot; =&amp;gt; &amp;quot;hydroxylase assay&amp;quot;,&#xA;    &amp;quot;MI:1001&amp;quot; =&amp;gt; &amp;quot;lipidase assay&amp;quot;,&#xA;    &amp;quot;MI:1002&amp;quot; =&amp;gt; &amp;quot;myristoylase assay&amp;quot;,&#xA;    &amp;quot;MI:1003&amp;quot; =&amp;gt; &amp;quot;geranylgeranylase assay&amp;quot;,&#xA;    &amp;quot;MI:1004&amp;quot; =&amp;gt; &amp;quot;palmitoylase assay&amp;quot;,&#xA;    &amp;quot;MI:1005&amp;quot; =&amp;gt; &amp;quot;adp ribosylase assay&amp;quot;,&#xA;    &amp;quot;MI:1006&amp;quot; =&amp;gt; &amp;quot;deglycosylase assay&amp;quot;,&#xA;    &amp;quot;MI:1007&amp;quot; =&amp;gt; &amp;quot;glycosylase assay&amp;quot;,&#xA;    &amp;quot;MI:1008&amp;quot; =&amp;gt; &amp;quot;sumoylase assay&amp;quot;,&#xA;    &amp;quot;MI:1009&amp;quot; =&amp;gt; &amp;quot;desumoylase assay&amp;quot;,&#xA;    &amp;quot;MI:1010&amp;quot; =&amp;gt; &amp;quot;neddylase assay&amp;quot;,&#xA;    &amp;quot;MI:1011&amp;quot; =&amp;gt; &amp;quot;deneddylase assay&amp;quot;,&#xA;    &amp;quot;MI:1016&amp;quot; =&amp;gt; &amp;quot;fluorescence recovery after photobleaching&amp;quot;,&#xA;    &amp;quot;MI:1019&amp;quot; =&amp;gt; &amp;quot;protein phosphatase assay&amp;quot;,&#xA;    &amp;quot;MI:1024&amp;quot; =&amp;gt; &amp;quot;scanning electron microscopy&amp;quot;,&#xA;    &amp;quot;MI:1026&amp;quot; =&amp;gt; &amp;quot;diphtamidase assay&amp;quot;,&#xA;    &amp;quot;MI:1030&amp;quot; =&amp;gt; &amp;quot;excimer fluorescence&amp;quot;,&#xA;    &amp;quot;MI:1031&amp;quot; =&amp;gt; &amp;quot;protein folding/unfolding&amp;quot;,&#xA;    &amp;quot;MI:1036&amp;quot; =&amp;gt; &amp;quot;nucleotide exchange assay&amp;quot;,&#xA;    &amp;quot;MI:1037&amp;quot; =&amp;gt; &amp;quot;Split renilla luciferase complementation&amp;quot;,&#xA;    &amp;quot;MI:1038&amp;quot; =&amp;gt; &amp;quot;silicon nanowire field-effect transistor&amp;quot;,&#xA;    &amp;quot;MI:1087&amp;quot; =&amp;gt; &amp;quot;monoclonal antibody blockade&amp;quot;,&#xA;    &amp;quot;MI:1088&amp;quot; =&amp;gt; &amp;quot;phenotype-based detection assay&amp;quot;,&#xA;    &amp;quot;MI:1089&amp;quot; =&amp;gt; &amp;quot;nuclear translocation assay&amp;quot;,&#xA;    &amp;quot;MI:1111&amp;quot; =&amp;gt; &amp;quot;two hybrid bait or prey pooling approach&amp;quot;,&#xA;    &amp;quot;MI:1112&amp;quot; =&amp;gt; &amp;quot;two hybrid prey pooling approach&amp;quot;,&#xA;    &amp;quot;MI:1113&amp;quot; =&amp;gt; &amp;quot;two hybrid bait and prey pooling approach&amp;quot;,&#xA;    &amp;quot;MI:1137&amp;quot; =&amp;gt; &amp;quot;carboxylation assay&amp;quot;,&#xA;    &amp;quot;MI:1138&amp;quot; =&amp;gt; &amp;quot;decarboxylation assay&amp;quot;,&#xA;    &amp;quot;MI:1142&amp;quot; =&amp;gt; &amp;quot;aminoacylation assay&amp;quot;,&#xA;    &amp;quot;MI:1145&amp;quot; =&amp;gt; &amp;quot;phospholipase assay&amp;quot;,&#xA;    &amp;quot;MI:1147&amp;quot; =&amp;gt; &amp;quot;ampylation assay&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you also want those that might not be direct binary but simply in the same complex, include:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;      &amp;quot;MI:0019&amp;quot; =&amp;gt; &amp;quot;coimmunoprecipitation&amp;quot;,&#xA;      &amp;quot;MI:0006&amp;quot; =&amp;gt; &amp;quot;anti bait coimmunoprecipitation&amp;quot;,&#xA;      &amp;quot;MI:0007&amp;quot; =&amp;gt; &amp;quot;anti tag coimmunoprecipitation&amp;quot;,&#xA;      &amp;quot;MI:0858&amp;quot; =&amp;gt; &amp;quot;immunodepleted coimmunoprecipitation&amp;quot;,&#xA;      &amp;quot;MI:0096&amp;quot; =&amp;gt; &amp;quot;pull down&amp;quot;,&#xA;      &amp;quot;MI:0963&amp;quot; =&amp;gt; &amp;quot;interactome parallel affinity capture&amp;quot;,&#xA;      &amp;quot;MI:0676&amp;quot; =&amp;gt; &amp;quot;tandem affinity purification&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-06-14T17:09:49.490" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="741" PostTypeId="1" AcceptedAnswerId="744" CreationDate="2017-06-14T21:02:35.707" Score="7" ViewCount="494" Body="&lt;p&gt;Google searching for &lt;strong&gt;NM_002084&lt;/strong&gt; gives the following result:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NM_002084&quot; rel=&quot;nofollow noreferrer&quot;&gt;NM_002084.4&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This, I assume, is the latest version v4, hence the &lt;code&gt;.4&lt;/code&gt; suffix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Searching for previous versions I get the following results, along with notes saying it was updated or removed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/89903006&quot; rel=&quot;nofollow noreferrer&quot;&gt;NM_002084.3&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This sequence has been updated. See current version.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sviewer/viewer.fcgi?val=NM_002084.2&quot; rel=&quot;nofollow noreferrer&quot;&gt;NM_002084.2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sviewer/viewer.fcgi?val=NM_002084.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;NM_002084.1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Record removed. This record was replaced or removed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Using &lt;em&gt;biomaRt&lt;/em&gt; I can get the latest(&lt;strong&gt;?&lt;/strong&gt;) version as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(&quot;biomaRt&quot;)&#xA;&#xA;# define db&#xA;ensembl &amp;lt;- useMart(&quot;ensembl&quot;, dataset = &quot;hsapiens_gene_ensembl&quot;)&#xA;&#xA;# get refseqs&#xA;getBM(attributes = c('refseq_mrna',&#xA;                     'chromosome_name',&#xA;                     'transcript_start',&#xA;                     'transcript_end',&#xA;                     'strand'),&#xA;      filters = c('refseq_mrna'),&#xA;      values = list(refseq_mrna = &quot;NM_002084&quot;),&#xA;      mart = ensembl)&#xA;#  refseq_mrna chromosome_name transcript_start transcript_end strand&#xA;#1   NM_002084               5        151020438      151028992      1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But querying for specific versions gives nothing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;getBM(attributes = c('refseq_mrna',&#xA;                     'chromosome_name',&#xA;                     'transcript_start',&#xA;                     'transcript_end',&#xA;                     'strand'),&#xA;      filters = c('refseq_mrna'),&#xA;      values = list(refseq_mrna = c(&quot;NM_002084.1&quot;, &quot;NM_002084.2&quot;, &quot;NM_002084.3&quot;, &quot;NM_002084.4&quot;)),&#xA;      mart = ensembl)&#xA;# [1] refseq_mrna      chromosome_name  transcript_start transcript_end   strand          &#xA;# &amp;lt;0 rows&amp;gt; (or 0-length row.names)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; How can I get all versions (preferably using R)?&lt;/p&gt;&#xA;" OwnerUserId="131" LastEditorUserId="292" LastEditDate="2017-06-15T09:40:25.250" LastActivityDate="2017-06-15T10:17:12.697" Title="Get RefSeq accession numbers with versions" Tags="|r|bioconductor|biomart|refseq|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="742" PostTypeId="2" ParentId="694" CreationDate="2017-06-14T22:42:30.370" Score="2" Body="&lt;p&gt;I wrote a very quick and dirty script to handle conversion between file types using BioJava.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/eedlund/Utils/tree/master/BioUtils&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/eedlund/Utils/tree/master/BioUtils&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Download the jar file &lt;a href=&quot;https://github.com/eedlund/Utils/raw/master/BioUtils/BioUtils.jar&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To run: &lt;code&gt;java -jar BioUtils.jar $FILE $TYPE&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where \$FILE is a PDB or mmCIF file you'd like to convert and \$TYPE is the format of the output file [PDB, CIF, MMTF].&lt;/p&gt;&#xA;" OwnerUserId="138" LastEditorUserId="77" LastEditDate="2018-04-29T06:03:10.153" LastActivityDate="2018-04-29T06:03:10.153" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="743" PostTypeId="2" ParentId="350" CreationDate="2017-06-14T22:56:46.020" Score="5" Body="&lt;p&gt;If you choose to perform your own culling of the PDB, resolution is probably the first thing you'll want to look at, which as Davidmh mentions is the main selection criteria for PISCES. High quality structures will also have better R-factor values. You can also give preference based on experimental technique, in descending order of quality:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neutron diffraction, X-ray diffraction, solution/solid state NMR, electron microscopy/crystallography, fiber diffraction, solution scattering.&lt;/p&gt;&#xA;" OwnerUserId="138" LastActivityDate="2017-06-14T22:56:46.020" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="744" PostTypeId="2" ParentId="741" CreationDate="2017-06-15T05:40:30.137" Score="6" Body="&lt;p&gt;I don't believe this is possible using &lt;code&gt;biomaRt&lt;/code&gt;, nor using &lt;code&gt;AnnotationHub&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two suggestions, neither of them very satisfactory. First, you can specify an Ensembl archive for &lt;code&gt;biomaRt&lt;/code&gt;, for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mart72.hs &amp;lt;- useMart(&quot;ENSEMBL_MART_ENSEMBL&quot;, &quot;hsapiens_gene_ensembl&quot;, &#xA;                      host = &quot;jun2013.archive.ensembl.org&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Of course, that requires that you have some idea of the date for each accession version and that the archives span that date - so not especially useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other option is to access EUtils using &lt;em&gt;e.g.&lt;/em&gt; &lt;code&gt;rentrez&lt;/code&gt;, which does allow search by version number:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(rentrez)&#xA;es &amp;lt;- entrez_search(&quot;nuccore&quot;, &quot;NM_002084.1 NM_002084.2&quot;)&#xA;es$ids&#xA;&#xA;[1] &quot;4504104&quot; &quot;6006000&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So knowing the accession, you could simply append 1, 2, 3... to it, run the search and see if UIDs come back, then get them using &lt;code&gt;entrez_fetch&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: here's a quick and dirty function which takes an accession as query, appends version = 1, fetches the ID, then increments the version and repeats until no more results are returned. It is not well-tested!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;getVersions &amp;lt;- function(accession) {&#xA;  require(rentrez)&#xA;  ids &amp;lt;- character()&#xA;  version &amp;lt;- 1&#xA;  repeat({&#xA;    es &amp;lt;- entrez_search(&quot;nuccore&quot;, paste0(accession, &quot;.&quot;, version))&#xA;    if(length(es$ids) == 0) {&#xA;      break&#xA;    }&#xA;    ids[version] &amp;lt;- es$ids&#xA;    version &amp;lt;- version + 1&#xA;  })&#xA;  ids&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;getVersions(&quot;NM_002084&quot;)&#xA;[1] &quot;4504104&quot;    &quot;6006000&quot;    &quot;89903006&quot;   &quot;1048339180&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="150" LastEditorUserId="150" LastEditDate="2017-06-15T10:17:12.697" LastActivityDate="2017-06-15T10:17:12.697" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="745" PostTypeId="1" AcceptedAnswerId="749" CreationDate="2017-06-15T06:29:13.993" Score="3" ViewCount="553" Body="&lt;p&gt;The latest version of Albacore from Oxford Nanopore Technologies calls bases from raw fast5 files. A useful piece of output is the &lt;code&gt;sequence_summary.txt&lt;/code&gt;, which is a big tab-delimited file with information on each read. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the columns in there is the &lt;code&gt;channel&lt;/code&gt;, which references the channel number from 1 to 512 on the flowcell. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These channels obviously have a physical layout on the flowcell. What is the layout? I.e. how many channels are there per row and per column, and how do the channel numbers map to the rows and columns?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My best guess is that it's a 16*32 array, but I don't know for sure, and even if I did I wouldn't know whether the &lt;code&gt;channels&lt;/code&gt; map across the shorter or the longer axis in order (or maybe they are not in order at all, but that would be silly).&lt;/p&gt;&#xA;" OwnerUserId="156" LastEditorUserId="156" LastEditDate="2017-06-15T06:34:33.327" LastActivityDate="2017-06-19T09:06:55.287" Title="Minion channel ID's from Albacore" Tags="|nanopore|minion|albacore|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="746" PostTypeId="2" ParentId="745" CreationDate="2017-06-15T08:09:40.980" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/mw55309/poRe_docs&quot; rel=&quot;nofollow noreferrer&quot;&gt;poRe&lt;/a&gt; has a &lt;code&gt;show.layout()&lt;/code&gt; function which shows you the 32*16 grid on which the channels are arranged. EDIT: THIS APPEARS NOT TO BE THE CORRECT PHYSICAL LAYOUT.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [,11] [,12] [,13]&#xA; [1,]  125  121  117  113  109  105  101   97   93    89    85    81    77&#xA; [2,]  126  122  118  114  110  106  102   98   94    90    86    82    78&#xA; [3,]  127  123  119  115  111  107  103   99   95    91    87    83    79&#xA; [4,]  128  124  120  116  112  108  104  100   96    92    88    84    80&#xA; [5,]  253  249  245  241  237  233  229  225  221   217   213   209   205&#xA; [6,]  254  250  246  242  238  234  230  226  222   218   214   210   206&#xA; [7,]  255  251  247  243  239  235  231  227  223   219   215   211   207&#xA; [8,]  256  252  248  244  240  236  232  228  224   220   216   212   208&#xA; [9,]  381  377  373  369  365  361  357  353  349   345   341   337   333&#xA;[10,]  382  378  374  370  366  362  358  354  350   346   342   338   334&#xA;[11,]  383  379  375  371  367  363  359  355  351   347   343   339   335&#xA;[12,]  384  380  376  372  368  364  360  356  352   348   344   340   336&#xA;[13,]  509  505  501  497  493  489  485  481  477   473   469   465   461&#xA;[14,]  510  506  502  498  494  490  486  482  478   474   470   466   462&#xA;[15,]  511  507  503  499  495  491  487  483  479   475   471   467   463&#xA;[16,]  512  508  504  500  496  492  488  484  480   476   472   468   464&#xA;      [,14] [,15] [,16] [,17] [,18] [,19] [,20] [,21] [,22] [,23] [,24] [,25]&#xA; [1,]    73    69    65    61    57    53    49    45    41    37    33    29&#xA; [2,]    74    70    66    62    58    54    50    46    42    38    34    30&#xA; [3,]    75    71    67    63    59    55    51    47    43    39    35    31&#xA; [4,]    76    72    68    64    60    56    52    48    44    40    36    32&#xA; [5,]   201   197   193   189   185   181   177   173   169   165   161   157&#xA; [6,]   202   198   194   190   186   182   178   174   170   166   162   158&#xA; [7,]   203   199   195   191   187   183   179   175   171   167   163   159&#xA; [8,]   204   200   196   192   188   184   180   176   172   168   164   160&#xA; [9,]   329   325   321   317   313   309   305   301   297   293   289   285&#xA;[10,]   330   326   322   318   314   310   306   302   298   294   290   286&#xA;[11,]   331   327   323   319   315   311   307   303   299   295   291   287&#xA;[12,]   332   328   324   320   316   312   308   304   300   296   292   288&#xA;[13,]   457   453   449   445   441   437   433   429   425   421   417   413&#xA;[14,]   458   454   450   446   442   438   434   430   426   422   418   414&#xA;[15,]   459   455   451   447   443   439   435   431   427   423   419   415&#xA;[16,]   460   456   452   448   444   440   436   432   428   424   420   416&#xA;      [,26] [,27] [,28] [,29] [,30] [,31] [,32]&#xA; [1,]    25    21    17    13     9     5     1&#xA; [2,]    26    22    18    14    10     6     2&#xA; [3,]    27    23    19    15    11     7     3&#xA; [4,]    28    24    20    16    12     8     4&#xA; [5,]   153   149   145   141   137   133   129&#xA; [6,]   154   150   146   142   138   134   130&#xA; [7,]   155   151   147   143   139   135   131&#xA; [8,]   156   152   148   144   140   136   132&#xA; [9,]   281   277   273   269   265   261   257&#xA;[10,]   282   278   274   270   266   262   258&#xA;[11,]   283   279   275   271   267   263   259&#xA;[12,]   284   280   276   272   268   264   260&#xA;[13,]   409   405   401   397   393   389   385&#xA;[14,]   410   406   402   398   394   390   386&#xA;[15,]   411   407   403   399   395   391   387&#xA;[16,]   412   408   404   400   396   392   388&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="681" LastEditorUserId="681" LastEditDate="2017-06-19T09:06:55.287" LastActivityDate="2017-06-19T09:06:55.287" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="747" PostTypeId="1" CreationDate="2017-06-15T08:49:32.977" Score="8" ViewCount="404" Body="&lt;p&gt;Since I have seen NCBI gene names were called &quot;Entrez ID&quot; for the first time, I am wondering where that comes from. Such a weird name!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anybody know where that originates?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My hypothesis is: in French, &quot;entrez&quot; can be translated to &quot;please enter&quot; (it is &lt;code&gt;to enter&lt;/code&gt; in the imperative mood), so I'm guessing that might come from a past website/database in French where &lt;code&gt;Entrez ID&lt;/code&gt; as in &quot;Please enter the ID&quot; was written in the search bar...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I could not find any reference... I cannot be the only one to ask himself this!&lt;/p&gt;&#xA;" OwnerUserId="868" LastActivityDate="2017-06-15T14:40:09.540" Title="What is the etymology of &quot;Entrez ID&quot;?" Tags="|gene|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="748" PostTypeId="2" ParentId="747" CreationDate="2017-06-15T09:03:44.387" Score="3" Body="&lt;h2&gt;Update&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/747/what-is-the-etymology-of-entrez-id/748#comment1340_747&quot;&gt;As pointed out by neilfws&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://en.wikipedia.org/wiki/Entrez&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; and Practical bioinformatics by Michael Agostino: The name &quot;Entrez&quot; means &quot;Come in!&quot; in French (pronounced “on-tray”); it was chosen to reflect the spirit of welcoming the public to search the content.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Old (probably incorrect) theory&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The first version of Entrez database was &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK3837/&quot; rel=&quot;nofollow noreferrer&quot;&gt;distributed&lt;/a&gt; by NCBI in 1991 (on CD-ROM). At that time, it consisted of nucleotide sequences, protein sequences and associated citations and abstracts from MEDLINE. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So even at the early stage it contained objects (entries) of very different types (sequences and abstracts). It is hard to find a word which will accurately and briefly describe all the types of data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I don't know the correct answer and this is purely a speculation, my guess is that Entrez is an intentionally misspelled word &quot;entries&quot;; which accurately describes all the datatypes in the database.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a misspelled word as a name is not uncommon, e.g. Google comes from misspelled &quot;googol&quot; and Flickr is a misspelled version of &quot;flicker&quot;.&lt;/p&gt;&#xA;" OwnerUserId="191" LastEditorUserId="191" LastEditDate="2017-06-15T14:40:09.540" LastActivityDate="2017-06-15T14:40:09.540" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="749" PostTypeId="2" ParentId="745" CreationDate="2017-06-15T10:08:22.437" Score="3" Body="&lt;p&gt;There are actually 2048 usable sequencing wells, hexagonally packed with four wells connected to the same sequencing sensor/channel via a multiplex (mux) selector. The combination of the mux and the channel number determines the physical location of the well. Unfortunately, the association between channel number and physical location is not obvious, and within each channel the muxes are also not in an obvious order, i.e. &lt;code&gt;[3,4,1,2,2,1,4,3]&lt;/code&gt; for two adjacent channels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an &quot;official&quot; text description of the layout as it was for R7; it hasn't changed for the most recent flow cells (R9.5):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Channels 1-64 occur at the top of the chip (the other low channel numbers are at the bottom)&lt;/li&gt;&#xA;&lt;li&gt;Channels order down the chip: 1-64, 449-512, 385-448, 321-384, 257-320, 193-256, 129-192, 65-128&lt;/li&gt;&#xA;&lt;li&gt;Muxes run from left to right in the order: 3, 4, 1, 2, 2, 1, 4, 3&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To confirm this layout, I pulled the channel/position lookup table definitions out of MinKNOW, and it seems to be the same as this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AGcU3.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AGcU3.jpg&quot; alt=&quot;Current MinION channel layout&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the code to produce this image (I haven't adjusted it to have a hexagonal structure):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/Rscript&#xA;library(rjson);&#xA;&#xA;layout.fname &amp;lt;- &quot;/opt/ONT/MinKNOW/Client/resources/app/resources/channels.json&quot;;&#xA;&#xA;data.list &amp;lt;- fromJSON(file=layout.fname);&#xA;&#xA;png(&quot;MinION_FC_Layout.png&quot;, width=3072, height=1536, pointsize=24);&#xA;par(mar=c(4,4,0.5,0.5), lwd=3);&#xA;data.mat &amp;lt;- matrix(unlist(data.list), ncol=2, byrow=TRUE);&#xA;plot(data.mat, pch=21,&#xA;     bg=(colorRampPalette(c(&quot;white&quot;,&quot;grey&quot;))(2048))[1:2048],&#xA;     col=c(&quot;red&quot;,&quot;yellow&quot;,&quot;green&quot;,&quot;blue&quot;),&#xA;     xlab=&quot;locX&quot;, ylab=&quot;locY&quot;, cex=2,&#xA;     xlim=c(7,76), ylim=c(0,33));&#xA;text(x=data.mat, labels=rep(1:512, each=4), cex=0.4);&#xA;legend(&quot;top&quot;,legend=c(1,2,3,4), fill=c(&quot;red&quot;,&quot;yellow&quot;,&quot;green&quot;,&quot;blue&quot;),&#xA;       inset=0.05, title=&quot;MUX&quot;);&#xA;text(x=6, y=data.mat[data.mat[,1] == 8,2],&#xA;     labels = ceiling(which(data.mat[,1] == 8) / 4));&#xA;text(x=77, y=data.mat[data.mat[,1] == 75,2],&#xA;     labels = ceiling(which(data.mat[,1] == 75) / 4));&#xA;arrows(x0=c(8,75), x1=c(39,44), y0=33, length=0.2);&#xA;invisible(dev.off());&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-16T05:49:50.170" LastActivityDate="2017-06-16T05:49:50.170" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="750" PostTypeId="1" CreationDate="2017-06-15T10:31:01.647" Score="1" ViewCount="50" Body="&lt;p&gt;I saw &lt;a href=&quot;http://www.nature.com/news/us-cancer-institute-to-overhaul-tumour-cell-lines-1.19364&quot; rel=&quot;nofollow noreferrer&quot;&gt;this nature news item&lt;/a&gt;, it sounds that &lt;a href=&quot;https://discover.nci.nih.gov/cellminer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cellminer&lt;/a&gt; is obsolete, is it right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the new tools to analyze the &quot;&lt;a href=&quot;https://www.jax.org/jax-mice-and-services/in-vivo-pharmacology/oncology-services/pdx-tumors#&quot; rel=&quot;nofollow noreferrer&quot;&gt;new cell lines&lt;/a&gt;&quot;? Where's the PDX repository?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about the European initiative that the article refers to? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be happy to see correlation between drugs and genes and similar capabilities as we can see on Cellminer. &lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2017-06-20T02:35:28.307" LastActivityDate="2017-06-20T02:35:28.307" Title="How to access the patient-derived xenografts (PDXs) repository?" Tags="|cancer|cell-line|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="751" PostTypeId="1" AcceptedAnswerId="753" CreationDate="2017-06-15T11:13:40.643" Score="8" ViewCount="508" Body="&lt;p&gt;I'm going through an RNA-seq pipeline in &lt;code&gt;R/Bioconductor&lt;/code&gt; and want to try multiple parameters at subsequent steps, for example, running clustering with different settings, running RegressOut or not on unwanted effects etc. That's a lot of &quot;versions&quot;, even if I don't do combinations of these steps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I keep track of this, and my conclusions?&#xA;Not necessarily want to save the results.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Save the different scripts with git (seems overkill)&lt;/li&gt;&#xA;&lt;li&gt;Make notes in the script itself&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="208" LastEditorUserId="292" LastEditDate="2017-06-15T15:06:38.017" LastActivityDate="2022-05-13T08:47:38.670" Title="What are the ways to keep track of branches in the analysis?" Tags="|r|bioconductor|git|workflow-management|best-practice|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="752" PostTypeId="1" AcceptedAnswerId="755" CreationDate="2017-06-15T11:56:51.837" Score="7" ViewCount="487" Body="&lt;p&gt;Is there a vcf file on the GRCh38 assembly with common cancer mutations I can download somewhere? Maybe from one of the big international cancer genomics consortia?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By common, I mean whichever mutations have been found recurrent in different types of cancer. &lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2017-06-18T17:04:43.097" Title="GRCh38 vcf file with common cancer mutations" Tags="|vcf|cancer|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="753" PostTypeId="2" ParentId="751" CreationDate="2017-06-15T12:26:26.903" Score="5" Body="&lt;p&gt;The main purpose of git is to version code, which usually means sequential improvement of the codebase. While it is possible to use branches for multiple variants of the software, permanent branches are traditionally used for gradual integration of new features (i.e. dev/testing/master branches). Supporting multiple independent branches requires some investment, i.e. distributing common changes among branches via merge or cherry-pick. This is hard to manage when you have more than two-three branches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you compare different methods of analysis, you probably want to compare the results between methods. Having them on separate branches makes it hard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion you should integrate all methods of analysis into the master branch. To avoid copy &amp;amp; paste, it is better to put common code in a library or an independent script. You can also specify a method as a run-time parameter of your pipeline, and create a meta-script which will execute all methods of interests.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you performed benchmarking, you shouldn't remove unused methods from you master branch. Having them is important for reproducible research, and your scripts could be used in the future for new datasets.&lt;/p&gt;&#xA;" OwnerUserId="191" LastActivityDate="2017-06-15T12:26:26.903" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="754" PostTypeId="2" ParentId="751" CreationDate="2017-06-15T12:47:47.027" Score="9" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Save the different scripts with git (seems overkill)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Whoa. I did an actual double take when reading this:&lt;sup&gt;1&lt;/sup&gt; it’s the &lt;em&gt;opposite&lt;/em&gt; of overkill.&lt;/p&gt;&#xA;&lt;p&gt;Version controlling your scripts (using Git or something similar) is the absolute minimum, and should become completely automatic. For every new project I begin, one of the very first steps is to issue the &lt;code&gt;git init&lt;/code&gt; command, and to set up a remote repository (on Github).&lt;/p&gt;&#xA;&lt;p&gt;To keep track of &lt;em&gt;different&lt;/em&gt; analyses, I use a combination of the following approaches:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Write reusable functions/scripts and parametrise. The parameters are kept either inside the script itself (that then calls the relevant function repeatedly), or in a declarative workflow.&lt;sup&gt;2&lt;/sup&gt;&lt;/li&gt;&#xA;&lt;li&gt;Document the alternative analysis approaches; once again, this could be a declarative workflow with different rules for alternative analyses, or a set of notebooks (via &lt;a href=&quot;http://rmarkdown.rstudio.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;R Markdown&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;li&gt;Have different Git branches for mutually exclusive approaches. At the end of the analysis one of these branches gets merged into the main branch, and published. If I want to publish several analysis approaches, I merge all these branches into the main branch, and use approaches (1) or (2) enable them simultaneously.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In fact, I recommend creating a declarative workflow script for &lt;em&gt;every&lt;/em&gt; analysis; I have found that this is the most practical, self-documenting way to run an analysis. It most closely resembles a wet-lab lab notebook. The advantage over a single R Markdown document is that rerunning just parts of the analysis can be completely automated, and dependencies in the workflow are apparent from the dependencies of the workflow rules. This is much harder in R Markdown.&lt;/p&gt;&#xA;&lt;p&gt;Some time ago I create an &lt;a href=&quot;https://github.com/klmr/example-r-analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;example analysis workflow&lt;/a&gt; to show how this can be structured.&lt;/p&gt;&#xA;&lt;p&gt;Regarding your other point:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Make notes in the script itself&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;“Notes” are a dangerous beasts: documentation is important, but experience shows that it’s sometimes very hard to keep documentation synchronised with the code. There is no mechanism that ensures that documentation and code actually agree. Differences between presumed analysis and actually executed analysis can become very problematic.&lt;/p&gt;&#xA;&lt;p&gt;People therefore prefer using self-documenting code as much as possible; that is: writing code so that its meaning becomes immediately clear, without comments, even to somebody who hasn’t worked on the code before. Doing this well is hard and takes practice, but improves overall code quality. Once again, using a declarative workflow helps here because the dependencies between the rules are self-documenting the kind of analysis that was performed.&lt;/p&gt;&#xA;&lt;p&gt;Jeff Atwood has written two seminal essays on this subject:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://blog.codinghorror.com/coding-without-comments/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Coding without comments&lt;/a&gt;, and&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://blog.codinghorror.com/code-tells-you-how-comments-tell-you-why/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Code tells you how, comments tell you why&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;They are two of the best pieces of advice on programming that I could give.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;sup&gt;1&lt;/sup&gt; To emphasise: take a look at the &lt;a href=&quot;https://bioinformatics.stackexchange.com/revisions/754/1&quot;&gt;edit history&lt;/a&gt; of this answer.&lt;/p&gt;&#xA;&lt;p&gt;&lt;sup&gt;2&lt;/sup&gt; I recommend &lt;a href=&quot;https://docs.ropensci.org/targets/&quot; rel=&quot;nofollow noreferrer&quot;&gt;‘targets’&lt;/a&gt; (for R), &lt;a href=&quot;https://www.nextflow.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nextflow&lt;/a&gt; or &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Snakemake&lt;/a&gt;. I used to use (and recommend) GNU make, but &lt;a href=&quot;https://twitter.com/klmr/status/1156949097116590080&quot; rel=&quot;nofollow noreferrer&quot;&gt;I no longer believe this to be an adequate tool&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2022-05-13T08:47:38.670" LastActivityDate="2022-05-13T08:47:38.670" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="755" PostTypeId="2" ParentId="752" CreationDate="2017-06-15T12:55:47.280" Score="8" Body="&lt;p&gt;Your question doesn't give enough information for a specific answer but this should do for a start.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Get the VCF file describing all variants in Clinvar from NCBI:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wget ftp://ftp.ncbi.nlm.nih.gov/pub/clinvar/vcf_GRCh38/clinvar.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Extract any variants whose VCF line contains the word &quot;cancer&quot;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zgrep -iE '^#|CLNDBN=[^;]*cancer' clinvar.vcf.gz &amp;gt; cancer.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now, you should probably filter that further based on variant frequency, type of cancer, etc etc but that should be a good start. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-06-15T12:55:47.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="756" PostTypeId="1" CreationDate="2017-06-15T13:09:16.177" Score="6" ViewCount="159" Body="&lt;p&gt;I want to find all experiments in GEO that are associated with a drug (for example tolvaptan). Is there any quick and scalable way to to this? I want to query more than 100 drugs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to use bioDBnet to map Drug Bank ID to look up data sets, but I don't know what output format should I select.&lt;/p&gt;&#xA;" OwnerUserId="872" LastEditorUserId="57" LastEditDate="2017-06-15T17:48:38.077" LastActivityDate="2018-09-24T17:22:05.357" Title="How to find GEO data sets using Drug Bank ID and bioDBnet" Tags="|database|gene|drugs|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="757" PostTypeId="2" ParentId="752" CreationDate="2017-06-15T13:22:38.700" Score="3" Body="&lt;p&gt;The trouble with looking for cancer-associated variants is that it can be difficult to tease out spurious effects (e.g. ethnicity) from causative variants. If you're interested in what &lt;em&gt;genes&lt;/em&gt; are implicated in most types of cancer, the annotations for the NanoString panels are quite good:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.nanostring.com/products/gene-expression-panels/hallmarks-cancer-gene-expression-panel-collection/pancancer-pathways-panel&quot; rel=&quot;nofollow noreferrer&quot;&gt;cancer pathways&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.nanostring.com/products/gene-expression-panels/hallmarks-cancer-gene-expression-panel-collection/pancancer-progression-panel&quot; rel=&quot;nofollow noreferrer&quot;&gt;cancer progression&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The &quot;Support documents&quot; section of those pages has a link to an Excel file with gene lists and probe sequences. While these lists don't give information about variants that disrupt function, I'm struggling a bit to think through why it would be useful in advance to know that, unless the aim is to induce cancer through CRISPR or something like that.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-15T13:22:38.700" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="758" PostTypeId="2" ParentId="731" CreationDate="2017-06-15T13:45:38.370" Score="5" Body="&lt;p&gt;I have used STRING pretty heavily, and have compared it to various other databases of protein interactions and signaling pathways.  I do feel like it has a lot of quality interaction annotations, but you have to sift through a lot of noise to get to them.  The simplest method I have found for doing this is to look at the individual scores for each interaction, and accept it if it passes one of the following tests:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Experiment Score &gt; 0.4&lt;/li&gt;&#xA;&lt;li&gt;Database Score &gt; 0.9&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Anything that passes one of these thresholds we consider at least an interaction of acceptable quality.  Those interactions with Experiment Scores &gt; 0.9 are high-quality, and have been experimentally validated.  The other scores represent inaccurate methods for determining signaling events, and should be ignored.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are not going to catch every actual protein signaling event this way, but you will at least be spared a lot of false positives.  The best way to construct an actual network of signaling events is to combine interaction records from multiple databases.&lt;/p&gt;&#xA;" OwnerUserId="47" LastEditorUserId="47" LastEditDate="2017-06-15T13:55:11.447" LastActivityDate="2017-06-15T13:55:11.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="759" PostTypeId="2" ParentId="751" CreationDate="2017-06-15T15:01:31.443" Score="1" Body="&lt;p&gt;I quite agree with &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/754/292&quot;&gt;this answer&lt;/a&gt; by @Konrad Rudolph.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to emphasis that using parametrization for your scripts is what will help you avoid multiplying branches in git. So yes, use git, but you don't necessarily have to go &quot;overkill&quot; creating lots of branches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, I would command these scripts from a workflow managment tool that will somehow take care of generating the branches of your analyses. If you use Snakemake, the various options taken along the path from your data to your results will be represented by the wildcards system, and this will be visible in the structure of your folders and file names, due to the fact that Snakemake works by inferring what should be done to produce a file based on its name.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This of course is not an excuse for not using other documentation approaches: Comments in the Snakefile and in the scripts, README files explaining how the workflow was run, using what configuration file. Put your scripts, the Snakefile, its configuration files and the README files under version control and document again using commit messages.&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2017-06-15T15:01:31.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="760" PostTypeId="2" ParentId="361" CreationDate="2017-06-15T15:47:20.327" Score="3" Body="&lt;p&gt;For FASTA files, I've implemented a relatively efficient method in &lt;a href=&quot;https://github.com/mdshw5/pyfaidx&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyfaidx&lt;/a&gt; v0.4.9.1. This post made me realize that my previous code was quite slow and easy to replace:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ pip install pyfaidx&#xA;$ time faidx -i nucleotide ~/Downloads/hg38.fa&#xA;name    start   end A   T   C   G   N   others&#xA;chr1    1   248956422   67070277    67244164    48055043    48111528    18475410    &#xA;chr10   1   133797422   38875926    39027555    27639505    27719976    534460  &#xA;chr11   1   135086622   39286730    39361954    27903257    27981801    552880  &#xA;chr11_KI270721v1_random 1   100316  18375   19887   31042   31012   0   &#xA;...&#xA;chrX    1   156040895   46754807    46916701    30523780    30697741    1147866 &#xA;chrY    1   57227415    7886192 7956168 5285789 5286894 30812372    &#xA;chrY_KI270740v1_random  1   37240   8274    12978   7232    8756    0   &#xA;&#xA;real    2m28.251s&#xA;user    2m15.548s&#xA;sys 0m9.951s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="267" LastActivityDate="2017-06-15T15:47:20.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="761" PostTypeId="1" AcceptedAnswerId="764" CreationDate="2017-06-15T17:51:10.583" Score="3" ViewCount="425" Body="&lt;p&gt;I am looking at the presence of viral genotypes within individual samples within an assay. Often times there is a sample whose read counts are firing off the charts and this sample tends to &quot;bleed through&quot; to the other samples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have recently tried confronting this problem with chi square analysis of conditional probability. For instance let's say I observe with 10 sample viral genotypes A and B. I then calculate the expected number of samples having both genotypes A and B from the number of A positive samples and number of B positive samples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far this is all I have. I do not know if anyone else has any ideas on how to determine and quantify bleed through in viral genotyping. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Slightly more information about this situation: these samples were run on Illumina HiSeq, we are planning on doing another HiSeq run with even more samples shortly so would like to take proper precautions with that, additionally the current pipeline utilizes Novobarcode as a demultiplexer so if anyone has suggestions on what I can implement during processing to minimize the effect of cross contamination that'd be helpful. &lt;/p&gt;&#xA;" OwnerUserId="712" LastEditorUserId="712" LastEditDate="2017-06-16T16:00:47.170" LastActivityDate="2017-06-16T16:00:47.170" Title="Techniques for analyzing and quantifying sample bleed through in genotyping with Illumina" Tags="|genotyping|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="762" PostTypeId="2" ParentId="756" CreationDate="2017-06-15T19:10:23.020" Score="2" Body="&lt;p&gt;One of the problems I've found with GEO and even ArrayExpress is that it seems like there are tons of false positives that come up in the search. Additionally, there may be a ton of results that you miss because you didn't include the synonym of the drug. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to make sure I have all the information relating to name, etc, you can write a script which downloads the result of a search in PubChem (this can be done with drugbank as well I think) and then parse the search result, extracting important information such as synonyms of drug name.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, take those synonyms and use them in an Entrez (EUtils) search for the experiments. Then try to filter the results to keep only what you're actually looking for. One bad way to do this is just by making sure the experiment summary contains the term your looking for. Another way to potentially do this, for example if there is a large experiment which tests serveral different factors is to programmatically search through associated files to ensure they contain a desired keyword. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll get different experiment types and so the files associated with the results will depend on the type of experiment in the search result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you get have a list of IDs of the experiments you want, you can feed them through to GEOquery in Bioconductor.&lt;/p&gt;&#xA;" OwnerUserId="823" LastActivityDate="2017-06-15T19:10:23.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="763" PostTypeId="2" ParentId="567" CreationDate="2017-06-15T19:21:31.780" Score="2" Body="&lt;p&gt;If you run this Rscript using the gene name as an argument, you'll get a file with the pathway written to the directory.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/Rscript&#xA;args = commandArgs(trailingOnly=TRUE)&#xA;library(paxtoolsr)&#xA;id &amp;lt;- args[1]&#xA;write.table(graphPc(source=id,kind='neighborhood',&#xA;            format='BINARY_SIF',verbose=TRUE), &#xA;            file=paste(id,'pathway',sep='_'), sep='\t',quote=FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="823" LastActivityDate="2017-06-15T19:21:31.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="764" PostTypeId="2" ParentId="761" CreationDate="2017-06-15T20:17:45.203" Score="1" Body="&lt;p&gt;Are you using dual-index barcodes with different barcodes at each end? There is a known phenomena of &quot;index switching&quot; that occurs in Illumina reads. One way to control for this is to add in unamplified water samples that have unused barcode combinations where each of the indexes is shared with another sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More information can be found on &lt;a href=&quot;http://seqanswers.com/forums/showthread.php?p=205928&quot; rel=&quot;nofollow noreferrer&quot;&gt;this SeqAnswers thread&lt;/a&gt;. Here are some useful associated links:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.biorxiv.org/content/early/2017/04/09/125724&quot; rel=&quot;nofollow noreferrer&quot;&gt;Signal-spreading preprint&lt;/a&gt; -- detailed investigation of the phenomenom, including methods, graphs, and associated data&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://seqanswers.com/forums/showthread.php?t=73736&quot; rel=&quot;nofollow noreferrer&quot;&gt;Crossblock&lt;/a&gt; -- tool by Brian Bushnell that attempts to remove cross contamination&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://seqanswers.com/forums/showthread.php?p=205928&quot; rel=&quot;nofollow noreferrer&quot;&gt;Summary by James Hadfield&lt;/a&gt; -- mentions a few fixes suggested by Illumina, many centred around not using barcode combinations that could conflict&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.illumina.com/content/dam/illumina-marketing/documents/products/whitepapers/index-hopping-white-paper-770-2017-004.pdf?linkId=36607862&quot; rel=&quot;nofollow noreferrer&quot;&gt;Illumina's quick-fire response&lt;/a&gt; -- they've known about it for a long time, and point out that it shouldn't be a problem for most purposes&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I have suspicions that at least some of the index switching is being caused by ligation of two separate barcoded fragments during the sample prep (which definitely happens &lt;a href=&quot;https://f1000research.com/articles/6-631/v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;during sample prep for nanopore&lt;/a&gt;), but I haven't done any experimentation to work out if this is also the case for Illumina (due to money/time/goal constraints). The experimental design would be to take a prepared Illumina library just prior to sequencing, ligate on ONT adapters, then sequence on both the ONT and Illumina machines to see if chimeric reads discovered via nanopore sufficiently matches the index switching observed via short-read sequencing.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-15T23:54:16.543" LastActivityDate="2017-06-15T23:54:16.543" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="765" PostTypeId="1" CreationDate="2017-06-16T02:59:51.210" Score="-2" ViewCount="88" Body="&lt;p&gt;How can I work out at which sites on the Drosophila melanogaster CG2316 mRNA the RNAi`s: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;12170/FBti0089992, &lt;/li&gt;&#xA;&lt;li&gt;12168/FBtp0030589, &lt;/li&gt;&#xA;&lt;li&gt;107343/FBtp0042163 and &lt;/li&gt;&#xA;&lt;li&gt;41984/FBst0034349 &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;(stock numbers/flybase ID) cleave? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I attempted to work this out myself some time ago but with limited success. I found the Fasta sequence of CG2316 and searched for the two complementary sites of the RNAi hairpin using the word doc find function. I feel this rudimentary method was flawed on a few levels, not least because I only got a single hit. As such I have posted the problem here.&lt;/p&gt;&#xA;" OwnerUserId="881" LastEditorUserId="48" LastEditDate="2018-01-16T21:37:47.463" LastActivityDate="2018-01-16T21:37:47.463" Title="Can you tell me where these RNAi’s cleave on the Drosophila melanogaster CG2316 mRNA?" Tags="|fasta|transcriptome-regulation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="767" PostTypeId="2" ParentId="750" CreationDate="2017-06-16T10:12:05.340" Score="2" Body="&lt;p&gt;I already see for links when I try to search. Take a look at them below. Also if you can be precise about kind of data are you trying to find, it would be better&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://pdmr.cancer.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pdmr.cancer.gov/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://cdt.northwestern.edu/news/patient-derived-xenograft-repository-now-available-researchers&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cdt.northwestern.edu/news/patient-derived-xenograft-repository-now-available-researchers&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.epo-berlin.com/epo-tumor-models-xenografts.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.epo-berlin.com/epo-tumor-models-xenografts.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.proxe.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.proxe.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://data-analysis.charite.de/care/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://data-analysis.charite.de/care/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.oasis-genomics.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.oasis-genomics.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cbioportal.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cbioportal.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope one of these above will serve the purpose for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: added some more links that might serve. As for the Cellminer the last update was 2012.&lt;/p&gt;&#xA;" OwnerUserId="451" LastEditorUserId="451" LastEditDate="2017-06-16T12:18:46.110" LastActivityDate="2017-06-16T12:18:46.110" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="768" PostTypeId="2" ParentId="567" CreationDate="2017-06-16T10:50:45.063" Score="1" Body="&lt;p&gt;Why not just use string-db's &lt;a href=&quot;http://string-db.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;online tool&lt;/a&gt;? There you can adjust all parameters of interest such as interaction confidence as well as pull in extra protein interactions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you make a user you can also upload a custom background gene-set set which means the analysis of go-terms and interconnectedness that sting provides is usable.&lt;/p&gt;&#xA;" OwnerUserId="599" LastActivityDate="2017-06-16T10:50:45.063" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="769" PostTypeId="1" AcceptedAnswerId="786" CreationDate="2017-06-16T10:53:49.660" Score="10" ViewCount="3684" Body="&lt;p&gt;I'm looking to subset a standard VCF file to generate one which only includes insertions (i.e. not indels). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can get part of the way there with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view -v indels &amp;lt;vcf&amp;gt; | awk '{if(length($4) == 1) print}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However this wouldn't catch an insertion that was part of a multi-allelic record with an insertion and deletion, where the reference length would be greater than 1 bp. Potentially then one way to go is a chain of decomposition, normalisation and then the same reference length filtering — surely there's a better way in one of the many VCF manipulation utils?&lt;/p&gt;&#xA;" OwnerUserId="61" LastEditorUserId="61" LastEditDate="2017-06-16T12:28:03.227" LastActivityDate="2018-07-02T13:49:52.417" Title="How can I extract only insertions from a VCF file?" Tags="|vcf|" AnswerCount="5" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="770" PostTypeId="2" ParentId="367" CreationDate="2017-06-16T11:01:15.400" Score="4" Body="&lt;p&gt;For the effective length part please see to Devons answer. I just have a small addition: Kallisto/Salmon/RSEM incorporate all bias estimates into the effective length meaning the effective length not only represent the length bias if you take the values from those tools (given that they were run with the bias algorithms enabled naturally).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With regards to getting gene level estimates you should not choose a specific transcript. Instead you should extract/calculate the RPKM/FPKM/TxPM (transcript per million that Kallisto/Salmon/RSEM outputs) for each transcript and sum them up to get the gene level estimate.&lt;/p&gt;&#xA;" OwnerUserId="599" LastActivityDate="2017-06-16T11:01:15.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="771" PostTypeId="1" AcceptedAnswerId="775" CreationDate="2017-06-16T11:22:34.610" Score="4" ViewCount="1740" Body="&lt;p&gt;Provide an overview of 10x data analysis packages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;10x provides Cell Ranger which prepares a count matrix from the bcl sequencer output files and other files (see bottom of page &lt;a href=&quot;https://support.10xgenomics.com/docs/license&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.10xgenomics.com/docs/license&lt;/a&gt; for the programs it uses).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can we do with the output files?&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2017-06-16T14:26:03.450" Title="10x Genomics Chromium single-cell RNA-seq data analysis options?" Tags="|rna-seq|r|bioconductor|scrnaseq|10x-genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="772" PostTypeId="2" ParentId="769" CreationDate="2017-06-16T11:24:00.487" Score="3" Body="&lt;p&gt;One method is to decompose multi-allelic records so that they're represented as one-allele, one-record using &lt;a href=&quot;https://github.com/atks/vt&quot; rel=&quot;nofollow noreferrer&quot;&gt;vt&lt;/a&gt; or similar:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view -v indels &amp;lt;vcf&amp;gt; |&#xA;  vt decompose - |&#xA;  bcftools view -H |&#xA;  awk '{if(length($5)&amp;gt;length($4)) print}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Some decomposed alleles will come with excess reference padding. To left-shift and trim these, add a &lt;code&gt;normalize&lt;/code&gt; step (NB matching these back to your input VCF becomes non-trivial):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view -v indels &amp;lt;vcf&amp;gt; |&#xA;  vt decompose - |&#xA;  vt normalize -r &amp;lt;reference.fasta - |&#xA;  awk '{if(length($4)==1) print}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;edit:&lt;/strong&gt; As gringer suggests, this can also be done without vt:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view -Ou -v indels &amp;lt;vcf&amp;gt; |&#xA;  bcftools norm -Ou -Nm - |&#xA;  bcftools view -H |&#xA;  awk '{if(length($5)&amp;gt;length($4)) print}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To also include complex alleles, use &lt;code&gt;view -V snps&lt;/code&gt; (!snvs) instead of &lt;code&gt;-v indels&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="61" LastEditorUserId="61" LastEditDate="2017-06-16T14:21:01.260" LastActivityDate="2017-06-16T14:21:01.260" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="773" PostTypeId="2" ParentId="769" CreationDate="2017-06-16T11:24:31.330" Score="2" Body="&lt;p&gt;Using &lt;a href=&quot;http://lindenb.github.io/jvarkit/VCFFilterJS.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;vcffilterjs&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;get the length of the REF;&lt;/li&gt;&#xA;&lt;li&gt;loop over the ALT, ignore the symbolic&lt;/li&gt;&#xA;&lt;li&gt;accept the variant if it's an insertion , eq:  len(ALT)&gt;len(REF) &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar dist/vcffilterjs.jar -e 'function accept(vc){var a=vc.getAlleles();var lenRef=a.get(0).length();for(i=1;i&amp;lt;a.size();++i) {var alt=a.get(i);if(alt.isSymbolic()) continue;var lenAlt=alt.length(); if(lenRef&amp;lt;lenAlt) return true; } return false; }accept(variant);' input.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;.&lt;/p&gt;&#xA;" OwnerUserId="71" LastActivityDate="2017-06-16T11:24:31.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="774" PostTypeId="1" CreationDate="2017-06-16T11:31:55.983" Score="4" ViewCount="788" Body="&lt;p&gt;Recent breakthroughs in bioinformatics tools for quantification (e.g.&#xA;&lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cufflinks&lt;/a&gt;/&lt;a href=&quot;http://pachterlab.github.io/kallisto/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kallisto&lt;/a&gt;/&lt;a href=&quot;https://github.com/COMBINE-lab/salmon&quot; rel=&quot;nofollow noreferrer&quot;&gt;Salmon&lt;/a&gt; etc.) and tools which can identify differential transcript usage (DTU) (e.g. &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/DRIMSeq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DRIMSeq&lt;/a&gt;, &lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cufflinks&lt;/a&gt; etc.) mean that from RNA-seq data we can now relatively easy obtain a genome wide analysis of transcripts that are differentially used betwen conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What can you use these results for? What systematic analysis does this transcript level data enable?&lt;/p&gt;&#xA;" OwnerUserId="599" LastEditorUserId="73" LastEditDate="2018-01-11T03:50:29.150" LastActivityDate="2022-01-06T17:00:55.717" Title="Analysis of differential transcript usage (DTU)" Tags="|rna-seq|transcriptome|software-recommendation|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="775" PostTypeId="2" ParentId="771" CreationDate="2017-06-16T11:35:51.537" Score="4" Body="&lt;h1&gt;Data preparation&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Cell Ranger uses the Illumina sequencing output (&lt;code&gt;.bcl&lt;/code&gt;) files&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/mkfastq&quot; rel=&quot;nofollow noreferrer&quot;&gt;Make fastq files&lt;/a&gt;:&#xA;&lt;code&gt;cellranger mkfastq&lt;/code&gt; ==&gt; &lt;code&gt;.fastq&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Prepare count matrix: &lt;code&gt;cellranger count&lt;/code&gt; ==&gt; &lt;code&gt;matrix.mtx, web_summary.html, cloupe.cloupe&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;Optional: combine multiple &lt;code&gt;matrix.mtx&lt;/code&gt; files (libraries): &lt;code&gt;cellranger aggr&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h1&gt;Data analysis&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/visualization/latest/what-is-loupe-cell-browser&quot; rel=&quot;nofollow noreferrer&quot;&gt;Loupe Cell Browser&lt;/a&gt; visualization of&lt;/strong&gt; &lt;code&gt;cloupe.cloupe&lt;/code&gt; &lt;strong&gt;files&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Count table&lt;/strong&gt; &lt;code&gt;matrix.mtx&lt;/code&gt; &lt;strong&gt;analysis options:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/python&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;Python&lt;/code&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;R&lt;/code&gt; &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/rkit&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cell Ranger R Kit&lt;/a&gt;: &lt;code&gt;cellrangerRkit::load_cellranger_matrix()&lt;/code&gt; ==&gt; ExpressionSet&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;R&lt;/code&gt; &lt;a href=&quot;http://hemberg-lab.github.io/scRNA.seq.course/scater-package.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scater&lt;/a&gt;: &lt;code&gt;scater::read10XResults()&lt;/code&gt; ==&gt; SCESet object&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;R&lt;/code&gt; &lt;a href=&quot;http://satijalab.org/seurat/pbmc-tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seurat&lt;/a&gt;: &lt;code&gt;Seurat::Read10X()&lt;/code&gt; ==&gt; Seurat object&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="208" LastEditorUserId="208" LastEditDate="2017-06-16T14:26:03.450" LastActivityDate="2017-06-16T14:26:03.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="776" PostTypeId="2" ParentId="774" CreationDate="2017-06-16T11:38:59.103" Score="4" Body="&lt;p&gt;I will take the liberty of giving one possible answers to my own question – but I’m very interested in other answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One analysis type that such data enables is the analysis of transcript switches with predicted potential consequences. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I myself have recently developed such a tool called &lt;a href=&quot;http://bioconductor.org/packages/IsoformSwitchAnalyzeR/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IsoformSwitchAnalyzeR&lt;/a&gt;. IsoformSwitchAnalyzeR enables statistical identification (via &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/DRIMSeq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DRIMSeq&lt;/a&gt;) of isoform switches with predicted functional consequences. The consequences analyzed can be chosen from a long list which includes gain/loss of protein domains, signal peptides changes in NMD sensitivity etc. The R package also enables easy visualization of isoform switches along with their consequences and it directly supports the output of &lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cufflinks/Cuffdiff&lt;/a&gt;, &lt;a href=&quot;http://deweylab.github.io/RSEM/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RSEM&lt;/a&gt;, &lt;a href=&quot;https://github.com/COMBINE-lab/salmon&quot; rel=&quot;nofollow noreferrer&quot;&gt;Salmon&lt;/a&gt; and &lt;a href=&quot;http://pachterlab.github.io/kallisto/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kallisto&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from enabling identification of interesting examples switch identification also enables systematic analysis of what genes are affected. For inspiration I will recommend &lt;a href=&quot;http://mcr.aacrjournals.org/content/early/2017/06/02/1541-7786.MCR-16-0459&quot; rel=&quot;nofollow noreferrer&quot;&gt;one of my own articles&lt;/a&gt; (describing results obtained with IsoformSwitchAnalyzeR) as well Hector et al’s &lt;a href=&quot;http://biorxiv.org/content/early/2016/09/21/076653&quot; rel=&quot;nofollow noreferrer&quot;&gt;recent paper&lt;/a&gt; (which is not using IsoformSwitchAnalyzeR). Of particular interest and finesse is Hector’s analysis of how isoform switches can disrupt protein-protein interactions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking forward to hear more ideas. &lt;/p&gt;&#xA;" OwnerUserId="599" LastEditorUserId="599" LastEditDate="2017-06-16T11:55:50.510" LastActivityDate="2017-06-16T11:55:50.510" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="777" PostTypeId="1" AcceptedAnswerId="778" CreationDate="2017-06-16T11:42:56.740" Score="6" ViewCount="714" Body="&lt;p&gt;I have two cancer cell lines (OCI-Ly18 &amp;amp; riva) that I want to find gene expression data for, but I'm not aware of many gene expression databases that allow searching by cell-line without searching by gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried &lt;a href=&quot;https://genevestigator.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genevestigator&lt;/a&gt; on the recommendation of this &lt;a href=&quot;https://www.researchgate.net/post/Is_there_a_database_to_find_out_which_genes_are_expressed_in_which_cell_lines&quot; rel=&quot;nofollow noreferrer&quot;&gt;thread&lt;/a&gt;&#xA;but found no data on either cell line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the best way to find gene expression data for these cell lines?&lt;/p&gt;&#xA;" OwnerUserId="885" LastEditorUserId="180" LastEditDate="2017-06-16T14:49:44.300" LastActivityDate="2017-06-16T14:49:44.300" Title="Searching for gene expression data by cell line" Tags="|database|cancer|differential-expression|cell-line|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="778" PostTypeId="2" ParentId="777" CreationDate="2017-06-16T11:59:37.793" Score="5" Body="&lt;p&gt;Try the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/&quot; rel=&quot;noreferrer&quot;&gt;Gene Expression Omnibus&lt;/a&gt; - it looks like they have some datasets.&lt;/p&gt;&#xA;" OwnerUserId="599" LastActivityDate="2017-06-16T11:59:37.793" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="779" PostTypeId="1" AcceptedAnswerId="985" CreationDate="2017-06-16T12:17:34.883" Score="3" ViewCount="3485" Body="&lt;p&gt;&lt;code&gt;cellranger aggr&lt;/code&gt; can combine multiple libraries (samples), and appends each barcode with an integer (e.g. AGACCATTGAGACTTA-1). The sample identity is not recorded in the combined &lt;code&gt;matrix.mtx&lt;/code&gt; file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/aggregate#gem_groups&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/aggregate#gem_groups&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can we keep and assign sample information to each cell after loading the data into &lt;code&gt;R&lt;/code&gt;? (e.g. &lt;code&gt;Seurat::Read10X()&lt;/code&gt;)&lt;/p&gt;&#xA;" OwnerUserId="208" LastEditorUserId="208" LastEditDate="2017-07-04T13:52:23.310" LastActivityDate="2018-06-14T21:48:40.630" Title="Handling sample identity in aggregated 10x libraries?" Tags="|rna-seq|r|scrnaseq|10x-genomics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="780" PostTypeId="2" ParentId="777" CreationDate="2017-06-16T12:30:32.550" Score="0" Body="&lt;p&gt;Try &lt;a href=&quot;https://portals.broadinstitute.org/ccle/data/browseData?conversationPropagation=begin&quot; rel=&quot;nofollow noreferrer&quot;&gt;in this section&lt;/a&gt; of &lt;a href=&quot;https://portals.broadinstitute.org/ccle/home&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cancer Cell Line Encyclopedia&lt;/a&gt;. It has expression data for a lot of cancer cell lines. I have tried few times and found all cell lines I wanted, except HeLa. You just have to select mRNA expression data and download the file you prefer (raw data, gene-centric, etc.)&lt;/p&gt;&#xA;" OwnerUserId="678" LastActivityDate="2017-06-16T12:30:32.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="781" PostTypeId="2" ParentId="779" CreationDate="2017-06-16T13:17:22.843" Score="1" Body="&lt;p&gt;You have to keep track of the file order you used for &lt;code&gt;aggr&lt;/code&gt;.  The suffix number represents which represents the aggregated sample. You could store this information in a text file and load it into R independently of the &lt;code&gt;Read10X&lt;/code&gt; function and combine it with the cell names to get their sample names. From the page you &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/aggregate#gem_groups&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Gem Groups&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;This is an integer that is appended to each barcode in the&#xA;  gene-barcode matrix. For example, &lt;em&gt;AGACCATTGAGACTTA-1&lt;/em&gt; and&#xA;  &lt;em&gt;AGACCATTGAGACTTA-2&lt;/em&gt; are distinct cell barcodes from different&#xA;  libraries, despite having the same nucleotide sequence.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The numbering of the GEM groups will reflect the order that the&#xA;  libraries were provided in the &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/aggregate#csv_setup&quot; rel=&quot;nofollow noreferrer&quot;&gt;Aggregation CSV&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="64" LastEditorUserId="48" LastEditDate="2017-06-16T13:44:38.527" LastActivityDate="2017-06-16T13:44:38.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="782" PostTypeId="1" AcceptedAnswerId="819" CreationDate="2017-06-16T13:39:16.387" Score="8" ViewCount="983" Body="&lt;p&gt;I am analysing 142 samples belonging to 6 batches. Additionally, those samples belong to 72 strains, which means that for most of the strains there are two samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could fit simple models (for strain and batches for instance), but when I get to the &quot;full&quot; model (~batch+strain), I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;so &amp;lt;- sleuth_fit(so, ~strain+batch, 'full')&#xA;Error in solve.default(t(X) %*% X) :&#xA;  system is computationally singular: reciprocal condition number = 5.2412e-19&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I should point out that of the 72 strains, only 15 have samples in distinct batches. This means that most strains (57) have both samples in the same batch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the error due to an unknown bug or rather to the experimental design? Does it mean that the information on batches cannot be used?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT I've posted the experimental design in a &lt;a href=&quot;https://gist.github.com/mgalardini/7553e14ead89a7f020f2b0a610086805&quot; rel=&quot;nofollow noreferrer&quot;&gt;gist&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;batch   strain  replica&#xA;batch_1     strain_41   1&#xA;batch_4     strain_41   2&#xA;batch_1     strain_28   1&#xA;batch_4     strain_28   2&#xA;batch_1     strain_26   1&#xA;[...]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2017-06-19T13:44:25.840" LastActivityDate="2017-06-20T11:00:23.183" Title="Getting a &quot;system is computationally singular&quot; error in sleuth" Tags="|rna-seq|r|differential-expression|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="783" PostTypeId="1" CreationDate="2017-06-16T13:56:08.720" Score="10" ViewCount="10295" Body="&lt;p&gt;If we have a PDB structrure, how can we find residues physically interacting with each other in space? I know that we must find the distance between residues and if the distance is less than 5-6 Angstrom, we say that residues are physically interacting. But how can we find the distance between all residues and how can we finally determine the distances between all residues? Is there a software or webserver for that?&lt;/p&gt;&#xA;" OwnerUserId="818" LastActivityDate="2019-09-10T21:28:12.907" Title="How can we find the distance between all residues in a PDB file?" Tags="|protein-structure|pdb|" AnswerCount="5" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="784" PostTypeId="2" ParentId="783" CreationDate="2017-06-16T14:22:17.763" Score="10" Body="&lt;p&gt;If you need to process multiple files, you could use &lt;a href=&quot;http://biopython.org/wiki/The_Biopython_Structural_Bioinformatics_FAQ&quot; rel=&quot;noreferrer&quot;&gt;Biopython&lt;/a&gt; to parse a PDB structure.&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;from Bio.PDB import PDBParser&#xA;&#xA;# create parser&#xA;parser = PDBParser()&#xA;&#xA;# read structure from file&#xA;structure = parser.get_structure('PHA-L', '1fat.pdb')&#xA;&#xA;model = structure[0]&#xA;chain = model['A']&#xA;&#xA;# this example uses only the first residue of a single chain.&#xA;# it is easy to extend this to multiple chains and residues.&#xA;for residue1 in chain:&#xA;    for residue2 in chain:&#xA;        if residue1 != residue2:&#xA;            # compute distance between CA atoms&#xA;            try:&#xA;                distance = residue1['CA'] - residue2['CA']&#xA;            except KeyError:&#xA;                ## no CA atom, e.g. for H_NAG&#xA;                continue&#xA;            if distance &amp;lt; 6:&#xA;                print(residue1, residue2, distance)&#xA;        # stop after first residue&#xA;        break&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you need to look at one structure, using a viewer perhaps would be easier. You could try &lt;a href=&quot;https://www.pymol.org/&quot; rel=&quot;noreferrer&quot;&gt;PyMOL&lt;/a&gt;: (&lt;a href=&quot;https://pymolwiki.org/index.php/Distance&quot; rel=&quot;noreferrer&quot;&gt;how to measure distance&lt;/a&gt;). There are other PDB viewers, some of which can work even through a browser.&lt;/p&gt;&#xA;" OwnerUserId="191" LastActivityDate="2017-06-16T14:22:17.763" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="785" PostTypeId="1" AcceptedAnswerId="788" CreationDate="2017-06-16T14:32:43.610" Score="3" ViewCount="109" Body="&lt;p&gt;I am looking for the positions of annotated regulatory sequences (promoters, enhancers and suppressors) in the human genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I looked at &lt;a href=&quot;http://www.ensembl.org/info/genome/funcgen/regulatory_build.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl regulatory Build&lt;/a&gt; and &lt;a href=&quot;http://www.pazar.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PAZAR&lt;/a&gt; but I am not used to look for datasets and I failed to find what I was looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Can you help to find annotated regulatory sequences (start and end positions) in the human genome?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="888" LastEditorUserId="888" LastEditDate="2017-06-16T14:42:31.880" LastActivityDate="2017-06-16T19:40:35.053" Title="Dataset: Locations of regulatory sequences in the human genome?" Tags="|database|human-genome|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="786" PostTypeId="2" ParentId="769" CreationDate="2017-06-16T14:45:22.267" Score="4" Body="&lt;p&gt;A one-liner:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;zcat my.vcf.gz |&#xA;  perl -ane '$x=0;for $y (split(&quot;,&quot;,$F[4])){$x=1 if length($y)&amp;gt;length($F[3])}print if /^#/||$x'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or equivalently&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;zcat my.vcf.gz |&#xA;  perl -ane '$x=0;map{$x=1 if length&amp;gt;length($F[3])}split(&quot;,&quot;,$F[4]);print if /^#/||$x'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For simple VCF operations, I generally recommend to write a script. This may be faster than those using heavy libraries. With a script, you only parse fields you care about; most libraries unnecessarily parse every field.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;On a related note, I recommend &lt;strong&gt;not&lt;/strong&gt; to decompose multi-allelic sites unless necessary. Decomposing is tricky, makes VCF harder to parse and to understand and may be a potential source of errors. Here is an example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM POS ID REF  ALT               QUAL FILTER  INFO  FORMAT  S1  S2  S3  S4&#xA;11     101 .  GCGT G,GCGA,GTGA,CCGT  199  PASS    .     GT      0/1 1/2 2/3 2/4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;vt decompose+normalize produces the following VCF:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM POS ID REF  ALT QUAL FILTER INFO FORMAT  S1   S2   S3   S4&#xA;11     101 .  GCGT G   199  PASS   .    GT      0/1  1/.  ./.  ./.&#xA;11     101 .  G    C   199  PASS   .    GT      0/.  ./.  ./.  ./1&#xA;11     102 .  CGT  TGA 199  PASS   .    GT      0/.  ./.  ./1  ./.&#xA;11     104 .  T    A   199  PASS   .    GT      0/.  ./1  1/.  1/.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In theory, you can reconstruct the original VCF from this output. However, it is very challenging for a program to do that. When you compute allele frequency line-by-line, this VCF will give you wrong results. &lt;code&gt;bcftools norm -m-&lt;/code&gt; replaces &quot;.&quot; with &quot;0&quot;. You can get a correct ALT allele frequency from the bcftools output, but a wrong REF allele frequency. Furthermore, vt is also imperfect in that &quot;CGT=&gt;TGA&quot; is not decomposed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My preferred output is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM POS    ID     REF    ALT    QUAL   FILTER INFO   FORMAT S1     S2     S3     S4&#xA;11     101    .      GCGT   G,&amp;lt;M&amp;gt;  0      .      .      GT     0/1    1/2    2/2    2/2&#xA;11     101    .      G      C,&amp;lt;M&amp;gt;  0      .      .      GT     0/2    2/0    0/0    0/1&#xA;11     102    .      C      T,&amp;lt;M&amp;gt;  0      .      .      GT     0/2    2/0    0/1    0/0&#xA;11     104    .      T      A,&amp;lt;M&amp;gt;  0      .      .      GT     0/2    2/1    1/1    1/0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here we use a symbolic allele &lt;code&gt;&amp;lt;M&amp;gt;&lt;/code&gt; to represent &quot;another ALT allele&quot;. You can calculate the allele frequency by looking at one line, and won't confuse other ALT alleles with REF. bgt can produce such a VCF indirectly. However, it discards all INFO, so is not a practical solution, either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, it is very difficult to decompose multi-allelic sites. When you get decomposing wrong, your downstream analyses may be inaccurate. Decomposition should be used with caution.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-06-16T14:45:22.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="787" PostTypeId="2" ParentId="783" CreationDate="2017-06-16T14:48:10.087" Score="4" Body="&lt;p&gt;Could you use CCP4's NCONT program? There's a GUI and a command line interface, whatever suits. You can specify which chains you want to target and interact with and set a cut off for distance. The bonus here is once you're in you have a nice suite of other structural tools to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're just doing it once, the GUI is friendly enough to work things out, if you're doing a batch then you can run it across several files via the command line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ccp4.ac.uk/download/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Download&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="252" LastEditorUserId="252" LastEditDate="2017-06-16T14:57:04.067" LastActivityDate="2017-06-16T14:57:04.067" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="788" PostTypeId="2" ParentId="785" CreationDate="2017-06-16T14:53:13.360" Score="5" Body="&lt;p&gt;This can be done quite easily using Ensebl's &lt;a href=&quot;http://www.ensembl.org/biomart/martview&quot; rel=&quot;noreferrer&quot;&gt;BioMart&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Choose the Ensembl Regulation database:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tm8MK.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tm8MK.png&quot; alt=&quot;Ensembl screenshot&quot;&gt;&lt;/a&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Select the &quot;Human Regulatory Features&quot; dataset:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6zTm8.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6zTm8.png&quot; alt=&quot;human regulatory features&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;That's basically it right there, just click on &quot;Results&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oPctm.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oPctm.png&quot; alt=&quot;results link&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Export to file and click &quot;Go&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SYI1k.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SYI1k.png&quot; alt=&quot;GO link&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This will download a file called &lt;code&gt;mart_export.txt&lt;/code&gt; which looks like this (I chose TSV for tab separated values):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ head mart_export.txt &#xA;Chromosome/scaffold name    Start (bp)  End (bp)    Feature type&#xA;18  76429380    76430144    Open chromatin&#xA;8   66405962    66406502    Open chromatin&#xA;4   61184401    61184600    CTCF Binding Site&#xA;X   40733600    40737000    Promoter&#xA;5   97407001    97407200    CTCF Binding Site&#xA;X   73946201    73946600    Promoter Flanking Region&#xA;15  19948201    19949200    CTCF Binding Site&#xA;5   11302601    11303143    Open chromatin&#xA;2   208407801   208408000   CTCF Binding Site&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see the format is pretty straightforward. The first column has the chromosome or scaffold name (it should always be a chromosome for human), the second and third are the star and end positions and the last field is the type of region. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also choose to limit by region type by choosing a filter (click the &quot;Filters&quot; link) in BioMart before downloading or by simply parsing the file once you've downloaded it. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-06-16T14:53:13.360" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="789" PostTypeId="2" ParentId="782" CreationDate="2017-06-16T15:10:27.567" Score="5" Body="&lt;p&gt;This happen when the variables (strain +batch) create a design matrix like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;batch strain&#xA;1 1 #&#xA;1 1 #&#xA;1 2&#xA;2 2&#xA;3 3&#xA;4 3&#xA;...&#xA;16 72&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which means that some of the covariates are not linearly independent (ie batch 1 and strain 1), all the strain 1 is in batch 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can correct for batch effects, but not in this design of the linear model (if you want to take into account the strain). You could do one batch more with those 15 strains that are in a single batch (if they are in different batch between them) that way you would get an independent design.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many strains in a single batch are in the same batch. You need to increase the number of samples (recommended anyway due to the low number of samples per strain) to avoid this problem. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;There are a lot of related question in Bioconductor &lt;a href=&quot;https://www.google.es/search?q=site%3Asupport.bioconductor.org%20%22system%20is%20computationally%20singular%22&amp;amp;rlz=1C1WPZB_enES727ES727&amp;amp;oq=site%3Asupport.bioconductor.org%20%22system%20is%20computationally%20singular%22&amp;amp;ie=UTF-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;support forum&lt;/a&gt;, from where I expanded an &lt;a href=&quot;https://support.bioconductor.org/p/62058/#62059&quot; rel=&quot;nofollow noreferrer&quot;&gt;answer&lt;/a&gt;.&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-06-20T07:33:09.990" LastActivityDate="2017-06-20T07:33:09.990" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="790" PostTypeId="2" ParentId="590" CreationDate="2017-06-16T15:33:36.017" Score="2" Body="&lt;p&gt;You could use &lt;a href=&quot;http://www.ccp4.ac.uk/MG/ccp4mg_help/pisa.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PISA&lt;/a&gt; assuming you have a PDB file of cellulase? Sorry it's a short answer and should be a comment, I don't have the reputation, however.&lt;/p&gt;&#xA;" OwnerUserId="252" LastActivityDate="2017-06-16T15:33:36.017" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="791" PostTypeId="2" ParentId="785" CreationDate="2017-06-16T19:40:35.053" Score="0" Body="&lt;p&gt;Promoters and enhancers based on CAGE-seq from FANTOM5:&#xA;&lt;a href=&quot;http://fantom.gsc.riken.jp/5/data/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://fantom.gsc.riken.jp/5/data/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just choose the category and download bed-file, e.g. CAGE-peaks would be promoters. There are READMEs in each folder that explain the data. &lt;/p&gt;&#xA;" OwnerUserId="894" LastActivityDate="2017-06-16T19:40:35.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="792" PostTypeId="2" ParentId="783" CreationDate="2017-06-16T20:09:38.157" Score="0" Body="&lt;p&gt;You can use &lt;a href=&quot;http://mdtraj.org/1.8.0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MDtraj&lt;/a&gt;. The package is easy to install using Anaconda.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can get the interacting residues using the following snippet (taken from &lt;a href=&quot;http://mdtraj.org/1.6.2/examples/native-contact.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mdtraj.org/1.6.2/examples/native-contact.html&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;heavy_pairs = np.array(&#xA;    [(i,j) for (i,j) in combinations(heavy, 2)&#xA;        if abs(native.topology.atom(i).residue.index - \&#xA;               native.topology.atom(j).residue.index) &amp;gt; 3])&#xA;&#xA;# compute the distances between these pairs in the native state&#xA;heavy_pairs_distances = md.compute_distances(native[0], heavy_pairs)[0]&#xA;# and get the pairs s.t. the distance is less than NATIVE_CUTOFF&#xA;native_contacts = heavy_pairs[heavy_pairs_distances &amp;lt; NATIVE_CUTOFF]&#xA;print(&quot;Number of native contacts&quot;, len(native_contacts))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="895" LastActivityDate="2017-06-16T20:09:38.157" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="793" PostTypeId="2" ParentId="769" CreationDate="2017-06-16T23:03:39.673" Score="0" Body="&lt;p&gt;If you're doing set operations, you could use &lt;a href=&quot;http://bedops.readthedocs.io/en/latest/content/reference/file-management/conversion/vcf2bed.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;vcf2bed&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ vcf2bed --insertions &amp;lt; in.vcf &amp;gt; out.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Based on the VCF v4.2 specification, &lt;code&gt;--snvs&lt;/code&gt;, &lt;code&gt;--insertions&lt;/code&gt;, and &lt;code&gt;--deletions&lt;/code&gt; are options available to filter input. In each case, the length of the reference and alternate alleles is used to determine which type of variant is being handled.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2017-06-16T23:03:39.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="794" PostTypeId="2" ParentId="752" CreationDate="2017-06-18T17:04:43.097" Score="8" Body="&lt;p&gt;If you are looking for cancer mutations, the primary resource is COSMIC and they provide GRCh38 VCFs. The download page is here: &lt;a href=&quot;http://cancer.sanger.ac.uk/cosmic/download&quot; rel=&quot;noreferrer&quot;&gt;http://cancer.sanger.ac.uk/cosmic/download&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It'll be up to you how you define &quot;common&quot;, but the VCF includes a lot of information you can use.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2017-06-18T17:04:43.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="795" PostTypeId="2" ParentId="367" CreationDate="2017-06-18T18:04:09.980" Score="12" Body="&lt;p&gt;I have a &lt;a href=&quot;http://robpatro.com/blog/?p=235&quot; rel=&quot;noreferrer&quot;&gt;blog post&lt;/a&gt; that describes the effective length (as well as these different relative abundance units).  The short explanation is that what people refer to as the &quot;effective length&quot; is actually the &lt;em&gt;expected&lt;/em&gt; effective length (i.e., the expectation, in a statistical sense, of the effective length).  The notion of effective length is actually a property of a transcript, fragment pair, and is equal to the number of potential starting locations for a fragment of this length on the given transcript.  If you take the average, over all fragments mapping to a transcript (potentially weighted by the conditional probability of this mapping), this quantity is the expected effective length of the transcript.  This is often approximated as simply $l_i - \mu$, or $l_i - \mu_{l_i}$ --- where $\mu_{l_i}$ is the mean of the &lt;em&gt;conditional&lt;/em&gt; fragment length distribution (conditioned on the fragment length being &amp;lt; $l_i$ to account for exactly the issue that you raise).&lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="29" LastEditDate="2017-07-26T16:15:39.647" LastActivityDate="2017-07-26T16:15:39.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="796" PostTypeId="1" CreationDate="2017-06-19T00:24:14.877" Score="6" ViewCount="213" Body="&lt;p&gt;&lt;strong&gt;The setup&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine that I work on an organism without a reference genome, and that the closest reference genome I can get is quite diverged. E.g. ~10% diverged in terms of SNVs when measured with short reads, and also has a lot of structural variants too. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now imagine I get a 1 million base-pair long-read (e.g. from Nanopore data) for my organism. The question is this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I estimate the proportion of the read that is meaningful sequence vs garbage? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Some things that probably won't work&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most standard approaches won't work here. E.g. I could try mapping the read to the reference, but &lt;em&gt;even if the read was perfectly good&lt;/em&gt; I wouldn't expect most of it to map thanks to true structural variations between the read and the reference. The same goes for standard alignment or BLAST. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Some things that might work&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best naive method here seems to be to cut the read up into smaller pieces (either overlapping or not) and use standard approaches to map/align each of these. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what have people tried for this? And what tools have you used and why?&lt;/p&gt;&#xA;" OwnerUserId="156" LastEditorUserId="73" LastEditDate="2017-06-19T03:42:46.103" LastActivityDate="2017-06-19T04:32:07.353" Title="How to estimate whether a long-read is meaningful sequence?" Tags="|sequence-alignment|nanopore|read-mapping|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="797" PostTypeId="2" ParentId="796" CreationDate="2017-06-19T00:34:27.393" Score="4" Body="&lt;p&gt;As a first pass, you could check if the read is chimeric. &lt;a href=&quot;https://github.com/rrwick/Porechop&quot; rel=&quot;nofollow noreferrer&quot;&gt;Porechop&lt;/a&gt; searches for known nanopore adaptors both on the ends and through the middle of the read. This won't resolve issues around blocked or empty pores, but it will at least check if you have found two long-ish reads lumped into the same file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By default, Porechop splits chimeric reads into two (or more, I suppose!) non-chimeric reads, but the option &lt;code&gt;--discard-middle&lt;/code&gt; would be a quick and easy way to check - run it on a fasta file containing only your long read, and if the output is empty, the read is chimeric.&lt;/p&gt;&#xA;" OwnerUserId="163" LastActivityDate="2017-06-19T00:34:27.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="798" PostTypeId="1" AcceptedAnswerId="811" CreationDate="2017-06-19T00:50:23.480" Score="7" ViewCount="3956" Body="&lt;p&gt;I have a refseq ID of a protein from E.coli and I want to find homologs of this protein. I ran Blast against refseq database but I got a lot of sequences most of which were from Ecoli again. I decided to run PSI-Blast to get more divergent species, but I do not exactly know if my result are real homologs or false positives. what is your idea for finding homologous protein sequences from more divergent species? And what can I do for selecting the real (but not false positive) hits?&lt;/p&gt;&#xA;" OwnerUserId="818" LastActivityDate="2017-06-19T13:54:59.920" Title="Finding homologs of a protein sequence" Tags="|sequence-homology|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="799" PostTypeId="2" ParentId="796" CreationDate="2017-06-19T04:32:07.353" Score="2" Body="&lt;p&gt;If you're looking at a single organism, in the absence of a reference genome you can map other reads to suspect reads and look at coverage. Looking at the actual sequence can also be useful: real DNA usually doesn't have an abundance of two different bases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nanopore reads also give another way to see if the read looks weird by having a look at the raw signal. Our &lt;a href=&quot;https://f1000research.com/articles/6-631/v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;chimeric reads paper&lt;/a&gt; gives a few examples of what DNA sequence should look like under normal circumstances. Here's the first raw signal figure from that paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JOCQu.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JOCQu.gif&quot; alt=&quot;good raw signal&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there's lots of contiguous signal with very similar current levels (i.e. it looks like the &quot;stall&quot; region, then it's not a good read and any basecalls should be ignored.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-19T04:32:07.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="800" PostTypeId="2" ParentId="798" CreationDate="2017-06-19T04:39:10.010" Score="4" Body="&lt;p&gt;It sounds like &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/smartblast/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Smart BLAST&lt;/a&gt; might do what you want. Here's the description of it's goal:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;SmartBLAST is a new and experimental NCBI tool that makes it easier to&#xA;  complete common sequence analysis tasks, such as finding a candidate&#xA;  protein name for a sequence, locating regions of high sequence&#xA;  conservation, or identifying regions covered by database sequences but&#xA;  missing from the query.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;To do this, SmartBLAST performs the following tasks in much less time&#xA;  than it takes to run a typical BLASTp search:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;a BLASTp comparison of the query with the closest matching sequences available;&lt;/li&gt;&#xA;  &lt;li&gt;a parallel BLASTp search to find the closest matches to high quality sequences from model organisms;&lt;/li&gt;&#xA;  &lt;li&gt;a multiple alignment between the query and five of the closest matching sequences (usually including two high quality sequences);&lt;/li&gt;&#xA;  &lt;li&gt;an analysis that produces a phylogenetic tree from the multiple sequence alignment.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;[from &lt;a href=&quot;https://ncbiinsights.ncbi.nlm.nih.gov/2015/07/29/smartblast/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Insights&lt;/a&gt;]&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-19T04:39:10.010" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="802" PostTypeId="1" CreationDate="2017-06-19T10:19:44.403" Score="4" ViewCount="586" Body="&lt;p&gt;I'm looking for the exact invocation used to generate the 16SMicrobial database that you can download from here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/blast/db/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nlm.nih.gov/blast/db/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm hoping to create the same type of blastdb with the same type of metadata with custom sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Platform isn't an issue, but let's say on ubuntu 14.04 or 16.04.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to replicate the creation of the database as closely as possible. &#xA; The most important feature is the taxonomic information as can be seen &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK279688/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The databases on the FTP site contain taxonomic information for each&#xA;  sequence, include the identifier indices for lookups, and can be up to&#xA;  four times smaller than the FASTA. The original FASTA can be generated&#xA;  from the BLAST database using blastdbcmd&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Creation of a blastdb using makeblastdb from a set of a fasta sequences is not an issue and can be achieved via:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;makeblastdb -in &amp;lt;your_file.fasta&amp;gt; -dbtype nucl -out &amp;lt;database_name&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question is specifically about the invocation NCBI uses to add the metadata that is present in the NCBI's 16SMicrobial blast database as I am keen to make sure I have replicated the process as closely as possible.&lt;/p&gt;&#xA;" OwnerUserId="302" LastEditorUserId="302" LastEditDate="2017-06-19T16:11:40.913" LastActivityDate="2018-11-30T08:59:00.007" Title="What command/invocation is used to generate NCBI 16SMicrobial blastdb" Tags="|blast|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="803" PostTypeId="2" ParentId="783" CreationDate="2017-06-19T11:18:01.990" Score="2" Body="&lt;p&gt;As part of a project me and some teammates did a &lt;a href=&quot;https://github.com/llrs/PYT-SBI&quot; rel=&quot;nofollow noreferrer&quot;&gt;script&lt;/a&gt; that outputs visual maps of distances between residues. It uses Biopython. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The module contact_map.py does what you are looking for. As an example, if you want to find the residues whose CA are below 5 you can run the following command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python3 contact_map.py pdb1cd8.ent -a CA -CA 5 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will produce three files:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;distance_map_pdb1cd8_CA.png # Heatmap of the distance between the residues&#xA;contact_map_pdb1cd8_CA.png # Black/White heatmap: If it is at that min distance&#xA;contact_map.log  # The actions taken&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you don't have downloaded already the pdb structue you can use the main module:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python3 cozmic.py real 1cd8 -a CA -CA 5 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="48" LastActivityDate="2017-06-19T11:18:01.990" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="805" PostTypeId="2" ParentId="802" CreationDate="2017-06-19T12:17:45.603" Score="0" Body="&lt;p&gt;I don't know what specific properties you are considering. As far as I know, that's just a normal blast database, like any other. These are produced using the &lt;code&gt;formatdb&lt;/code&gt; command which is part of the &lt;code&gt;ncbi-blast&lt;/code&gt; software. You haven't specified what system you are using so I can't help you find and install it, but the command will be something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;formatdb -p F -i yourSeqs.fasta -n yourDBName&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;formatdb&lt;/code&gt; : the command&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;-p F&lt;/code&gt; : protein: false. This is needed to make databases of nucleotide sequences. &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;-i yourSeqs.fasta&lt;/code&gt; : your input file with all your sequences in fasta format. &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;-n yourDBName&lt;/code&gt; : the name (choose whatever you like) for your database.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can then use &lt;code&gt;yourDBName&lt;/code&gt; as a database to blast against. The above assumes yuou are &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-06-19T12:17:45.603" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="807" PostTypeId="1" AcceptedAnswerId="861" CreationDate="2017-06-19T13:15:22.910" Score="20" ViewCount="1994" Body="&lt;p&gt;I would like to select a random record from a large set of &lt;code&gt;n&lt;/code&gt; unaligned sequencing reads in &lt;code&gt;log(n)&lt;/code&gt; time complexity (&lt;a href=&quot;https://en.wikipedia.org/wiki/Big_O_notation&quot; rel=&quot;noreferrer&quot;&gt;big O notation&lt;/a&gt;) or less. A record is defined as the equivalent of four lines in FASTQ format. The records do not fit in RAM and would need to be stored on disk. Ideally, I would like to store the reads in a compressed format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would prefer a solution that does not require any extra files such as for example a reference genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The title of this question mentions a FASTQ only because FASTQ is a common format for storing unaligned reads on disk. I am happy with answers that require a single limited transformation of the data to another file format in time complexity order &lt;code&gt;n&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Update&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;A clarification: I want the random record to be selected with probability &lt;code&gt;1/n&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="492" LastEditDate="2017-06-22T16:32:55.203" LastActivityDate="2021-04-27T08:46:13.457" Title="Random access on a FASTQ file" Tags="|sam|fastq|reads|benchmarking|" AnswerCount="12" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="808" PostTypeId="2" ParentId="807" CreationDate="2017-06-19T13:15:22.910" Score="5" Body="&lt;p&gt;You could shuffle the FASTQ once and then read sequences off the top of the file as you need them:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;gzip -dc input.fastq.gz | paste - - - - | shuf | tr '\t' '\n'| gzip -c &amp;gt; output.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would recommend &lt;a href=&quot;https://zlib.net/pigz/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pigz&lt;/a&gt; as a replacement for gzip in the compression step if you have it available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The downside of this approach is that you only get &lt;code&gt;n&lt;/code&gt; reads before you need to run the shuffle again, and &lt;a href=&quot;https://stackoverflow.com/a/24492814/528691&quot;&gt;apparently&lt;/a&gt; &lt;code&gt;shuf&lt;/code&gt; holds all data in RAM, so it would die with an out of memory error if the FASTQ file does not fit into RAM as is specified in the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using &lt;code&gt;sort -R&lt;/code&gt; is complexity &lt;code&gt;n log(n)&lt;/code&gt; and uses temporary files, so it &lt;a href=&quot;https://vkundeti.blogspot.co.uk/2008/03/tech-algorithmic-details-of-unix-sort.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;should not&lt;/a&gt; run out of memory:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;gzip -dc input.fastq.gz | paste - - - - | nl | sort -R | perl -pe 's/\s*\d+\t//' | tr '\t' '\n'| gzip -c &amp;gt; output.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;nl&lt;/code&gt; and &lt;code&gt;perl&lt;/code&gt; commands are necessary to make sure that identical records are not sorted next to each other.&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="191" LastEditDate="2017-06-22T11:06:17.097" LastActivityDate="2017-06-22T11:06:17.097" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="809" PostTypeId="2" ParentId="807" CreationDate="2017-06-19T13:44:42.180" Score="3" Body="&lt;p&gt;One possibility is to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;reformat the data such that each record is a single line containing the read description, bases, and quality scores&lt;/li&gt;&#xA;&lt;li&gt;pad out each record to a maximum length in each field such that every record in the file is the same number of bytes&lt;/li&gt;&#xA;&lt;li&gt;the total number of records can now be calculated as file size / record size&lt;/li&gt;&#xA;&lt;li&gt;choose a random record number between 0 and the total number of records&lt;/li&gt;&#xA;&lt;li&gt;binary search over the reformatted file until you obtain your read&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This would get you the log(n) lookup time you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, the data wouldn't be compressed.  You could 2-bit encode the bases and quantize the quals to save some space, but that'd be lossy and is perhaps not what you're looking for.  Alternatively, you could block-gzip the reformatted data and keep a record of how many blocks are in the file and how many reads are in each block (since the filesize will no longer reflect the number of records in the file).  Then to obtain a specific read, you'd calculate the block number it'll appear in, decompress the block, and return the appropriate read.&lt;/p&gt;&#xA;" OwnerUserId="822" LastActivityDate="2017-06-19T13:44:42.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="810" PostTypeId="2" ParentId="807" CreationDate="2017-06-19T13:51:48.503" Score="0" Body="&lt;p&gt;I suppose I can provide code for this if needed, but keep in mind that it'd probably have to be done in C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One could make this easier by using a bgzf compressed fastq file. Yes, BAM uses that already, but since Illumina's software now defaults to producing that there should be a reasonable amount of it already available.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Count the number of blocks in the file.&lt;/li&gt;&#xA;&lt;li&gt;Randomly select one of these blocks.&lt;/li&gt;&#xA;&lt;li&gt;Perform reservoir selection on the entries in that block.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;That won't be strictly O(log N), but it's lower on memory and IO requirements than parsing through the whole file and doing shuffling or otherwise completely munging the data into an otherwise not-terribly-useful format. This also has the benefit of allowing selecting more than one random entry a bit faster, since you can memoize things.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-06-19T13:51:48.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="811" PostTypeId="2" ParentId="798" CreationDate="2017-06-19T13:54:59.920" Score="5" Body="&lt;p&gt;It depends on what you're looking for. If you're just looking for sequence homology, then you can simply pick the best hits from a blast search. If, however, you are referring to &lt;em&gt;functional&lt;/em&gt; homology, if you are looking for the protein which has the same functions as your query, then it's more complicated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sequence homology is not enough to infer functional homology. For example, you can have cases of gene duplication and subsequent functional divergence. Such &lt;em&gt;paralogs&lt;/em&gt; are still &lt;em&gt;homologs&lt;/em&gt; (&lt;a href=&quot;https://biology.stackexchange.com/a/4964/1306&quot;&gt;paralogs are a subset of homologs&lt;/a&gt;), but they don't necessarily have the same function. It is also often the case that the homolog (be it orthologous or paralogous) of a protein in species B has a completely different function than its homolog in species A despite a high level of sequence similarity. This is usually very hard to determine &lt;em&gt;in silico&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find the functionally homologous protein(s), you would ideally need to identify the essential residues that allow your protein to perform its function. This could be done using something like &lt;a href=&quot;http://pfam.xfam.org/&quot; rel=&quot;noreferrer&quot;&gt;PFam&lt;/a&gt; which will identify protein domains. You can then check whether the homologs you find also have this domain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is essentially what PSI-blast does. Although it doesn't take domains into account, each successive iteration is used to build a model of your proptein. The model is built under the reasonable assumption that highly conserved residues are important. So it will consider more diverged sequences as homologous if those residues are conserved. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know how your protein works and what residues are important, you can use that knowledge to refine the results of your PSI-blast. If you don't, you'll have to use only &quot;good&quot; hits to make the model. One way to do this, for well studied proteins, is to only add proteins that are already annotated as homologs of what you are looking for to build your model, then use that model to search in un-annotated species. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't know, you could try looking for recognizable protein domains in your query protein (use PFam) and then use the HMM (hidden markov model) of the domain to identify important residues. For example, &lt;a href=&quot;http://pfam.xfam.org/family/PF01754#tabview=tab4&quot; rel=&quot;noreferrer&quot;&gt;this is&lt;/a&gt; the HMM logo for the zf-A20 zinc finger domain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/90nPc.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/90nPc.png&quot; alt=&quot;zinc finger sequence logo&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The huge cysteine (C) residues are shown that size because they are very conserved across proteins carrying this domain and, presumably, are functionally important for the domain. So, if you pass your protein through PFam and identify domains, find the important residues and make sure all your homologs have those conserved. If using PSI-blast, only include sequences where those residues are conserved in the results you keep.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, another useful tool that works in the same way is &lt;a href=&quot;http://hmmer.org/&quot; rel=&quot;noreferrer&quot;&gt;HMMER&lt;/a&gt;. This takes a protein alignment as input, like PSI-blast builds an HMM model from it and then can use this model to query a protein database for more hits. Methods like HMMER and PSI-blast are far better than simple sequence similarity approaches when looking for homologs.  &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-06-19T13:54:59.920" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="812" PostTypeId="2" ParentId="807" CreationDate="2017-06-19T13:57:44.610" Score="1" Body="&lt;p&gt;To emphasise the issue (as outlined in the &lt;code&gt;1/n&lt;/code&gt; update), consider an input file with one record that is 5 million bases long, and one records that is 100 bases long. You want an equal probability of selecting either of these two records. Any random seek methods will overwhelmingly pick out the long record.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I expect that indexing the locations of record starts is really the only workable option here, particularly if multi-line records are possible. Create an index file containing the start locations of each record (as identically sized integers, e.g. 64-bit), then sample from the index file (which is an identical length for each record) to fetch the start location. I'd envisage that this file would &lt;em&gt;only&lt;/em&gt; contain the start locations; any additional metadata (including sequence name) would require seeking in the original file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once the indexing is done, the file can be compressed with bgzip, with specific offsets retrieved using the &lt;code&gt;-b&lt;/code&gt; and &lt;code&gt;-s&lt;/code&gt; options. However, I expect that compression would not be particularly efficient if multiple random records were desired.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-20T10:13:08.820" LastActivityDate="2017-06-20T10:13:08.820" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="813" PostTypeId="2" ParentId="807" CreationDate="2017-06-19T14:06:33.253" Score="11" Body="&lt;h1&gt;Arbitrary record access in constant time&lt;/h1&gt;&#xA;&lt;p&gt;To get a random record in constant time, it is sufficient to get an arbitrary record in constant time.&lt;/p&gt;&#xA;&lt;p&gt;I have two solutions here: One with &lt;code&gt;tabix&lt;/code&gt; and one with &lt;code&gt;grabix&lt;/code&gt;. I think the &lt;code&gt;grabix&lt;/code&gt; solution is more elegant, but I am keeping the &lt;code&gt;tabix&lt;/code&gt; solution below because &lt;code&gt;tabix&lt;/code&gt; is a more mature tool than &lt;code&gt;grabix&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Thanks to &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/37/user172818&quot;&gt;user172818&lt;/a&gt; for suggesting &lt;code&gt;grabix&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;Update&lt;/h3&gt;&#xA;&lt;p&gt;This answer previously stated that &lt;code&gt;tabix&lt;/code&gt; and &lt;code&gt;grabix&lt;/code&gt; perform lookups in &lt;code&gt;log(n)&lt;/code&gt; time. After taking a closer look at the grabix source code and the tabix paper I am now convinced that lookups are independent of &lt;code&gt;n&lt;/code&gt; in complexity. However, both tools use an index that scales in size proportionally to &lt;code&gt;n&lt;/code&gt;. So, the loading of the index is order &lt;code&gt;n&lt;/code&gt;. However, if we consider the loading of the index as &amp;quot;...a single limited transformation of the data to another file format...&amp;quot;, then I think this answer is still a valid one. If more than one record is to be retrieved, then the index needs to be stored in memory, perhaps with a framework such as pysam or htslib.&lt;/p&gt;&#xA;&lt;h2&gt;Using &lt;code&gt;grabix&lt;/code&gt;&lt;/h2&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Compress with &lt;code&gt;bgzip&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Index the file and perform lookups with &lt;a href=&quot;https://github.com/arq5x/grabix&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;grabix&lt;/code&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In bash:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;gzip -dc input.fastq.gz | bgzip -c &amp;gt; output.fastq.gz&#xA;&#xA;grabix index output.fastq.gz&#xA;&#xA;# retrieve 5-th record (1-based) in log(n) time&#xA;# requires some math to convert indices (4*4 + 1, 4*4 + 4) = (17, 20)&#xA;grabix grab output.fastq.gz 17 20&#xA;&#xA;# Count the number of records for part two of this question&#xA;export N_LINES=$(gzip -dc output.fastq.gz | wc -l)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Using &lt;code&gt;tabix&lt;/code&gt;&lt;/h2&gt;&#xA;&lt;p&gt;The tabix code is more complicated and relies on the iffy assumption that &lt;code&gt;\t&lt;/code&gt; is an acceptable character for replacement of &lt;code&gt;\n&lt;/code&gt; in a FASTQ record.  If you are happy with a file format that is close to but not exactly FASTQ, then you could do the following:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Paste each record into a single line.&lt;/li&gt;&#xA;&lt;li&gt;Add a dummy chromosome and line number as the first and second column.&lt;/li&gt;&#xA;&lt;li&gt;Compress with &lt;code&gt;bgzip&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Index the file and perform lookups with &lt;a href=&quot;http://www.htslib.org/doc/tabix.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;tabix&lt;/code&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Note that we need to remove leading spaces introduced by &lt;code&gt;nl&lt;/code&gt; and we need to introduce a dummy chromosome column to keep &lt;code&gt;tabix&lt;/code&gt; happy:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;gzip -dc input.fastq.gz | paste - - - - | nl | sed 's/^ *//' | sed 's/^/dummy\t/' | bgzip -c &amp;gt; output.fastq.gz&#xA;tabix -s 1 -b 2 -e 2 output.fastq.gz &#xA;&#xA;# now retrieve the 5th record (1-based) in log(n) time&#xA;tabix output.fastq.gz dummy:5-5 &#xA;&#xA;# This command will retrieve the 5th record and convert it record back into FASTQ format&#xA;tabix output.fastq.gz dummy:5-5 | perl -pe 's/^dummy\t\d+\t//' | tr '\t' '\n'&#xA;&#xA;# Count the number of records for part two of this question&#xA;export N_RECORDS=$(gzip -dc output.fastq.gz | wc -l)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h1&gt;Random record in constant time&lt;/h1&gt;&#xA;&lt;p&gt;Now that we have a way of retrieving an arbitrary record in &lt;code&gt;log(n)&lt;/code&gt; time, retrieving a random record is simply a matter of getting a good random number generator and sampling. Here is some example code to do this in python:&lt;/p&gt;&#xA;&lt;h2&gt;Using grabix&lt;/h2&gt;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;# random_read.py&#xA;import os&#xA;import random&#xA;&#xA;n_records = int(os.environ[&amp;quot;N_LINES&amp;quot;]) // 4&#xA;rand_record_start = random.randrange(0, n_records) * 4 + 1&#xA;rand_record_end = rand_record_start + 3&#xA;os.system(&amp;quot;grabix grab output.fastq.gz {0} {1}&amp;quot;.format(rand_record_start, rand_record_end))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Using tabix&lt;/h2&gt;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;# random_read.py&#xA;import os&#xA;import random&#xA;&#xA;n_records = int(os.environ[&amp;quot;N_RECORDS&amp;quot;])&#xA;rand_record_index = random.randrange(0, n_records) + 1&#xA;# super ugly, but works...&#xA;os.system(&#xA;    &amp;quot;tabix output.fastq.gz dummy:{0}-{0} | perl -pe 's/^dummy\t\d+\t//' | tr '\t' '\n'&amp;quot;.format(&#xA;        rand_record_index)&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And this works for me:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;python3.5 random_read.py&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h1&gt;Disclaimer&lt;/h1&gt;&#xA;&lt;p&gt;Please note that &lt;code&gt;os.system&lt;/code&gt; calls a system shell and is vulnerable to &lt;a href=&quot;https://en.wikipedia.org/wiki/Code_injection#Shell_injection&quot; rel=&quot;nofollow noreferrer&quot;&gt;shell injection vulnerabilities&lt;/a&gt;. If you are writing production code, then you probably want to &lt;a href=&quot;https://docs.python.org/3/library/subprocess.html#security-considerations&quot; rel=&quot;nofollow noreferrer&quot;&gt;take extra precautions&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Thanks to &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/104/chris-rands&quot;&gt;Chris_Rands&lt;/a&gt; for raising this issue.&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="492" LastEditDate="2021-04-27T08:46:13.457" LastActivityDate="2021-04-27T08:46:13.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="814" PostTypeId="2" ParentId="807" CreationDate="2017-06-19T15:02:15.580" Score="0" Body="&lt;p&gt;A quick and dirty solution could be to convert the FASTQ file to two FASTAs, one of the them storing the bases and the other one storing the qualities, and then use standard methods for random access for FASTA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only complication is that base qualities can contain &lt;code&gt;&amp;gt;&lt;/code&gt;. However, this is a problem only when it is the first character of a line and we can fix it by prepending, e.g., &lt;code&gt;I&lt;/code&gt; to each quality sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If also other people think that this could be a sufficient solution, I will expand the answer and add all the commands.&lt;/p&gt;&#xA;" OwnerUserId="425" LastActivityDate="2017-06-19T15:02:15.580" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="815" PostTypeId="1" AcceptedAnswerId="816" CreationDate="2017-06-19T15:25:03.003" Score="10" ViewCount="6542" Body="&lt;p&gt;How to extract the sequence used to create a blast database.  This is useful when you download a blastdb from somewhere else e.g. one of the &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/blast/db&quot; rel=&quot;noreferrer&quot;&gt;databases provided by NCBI&lt;/a&gt; including the 16SMicrobial database. Or alternatively, when you want to double check which version of a sequence you have included in a blastdb.&lt;/p&gt;&#xA;" OwnerUserId="302" LastActivityDate="2017-06-19T15:25:03.003" Title="How to extract fasta from a blastdb" Tags="|fasta|blast|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="816" PostTypeId="2" ParentId="815" CreationDate="2017-06-19T15:25:03.003" Score="9" Body="&lt;p&gt;You can extract fasta sequence from a blastdb constructed from a fasta file using blastdbcmd which should be installed when you install blast/makeblastdb.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blastdbcmd -entry all -db &amp;lt;database label&amp;gt; -out &amp;lt;outfile&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you had a database called &lt;code&gt;my_database&lt;/code&gt; which contained the files:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;my_database.nhr&lt;/code&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;my_database.nsq&lt;/code&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;my_database.nin&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and you wanted your fasta output file to be called &lt;code&gt;reference.fasta&lt;/code&gt; you would run the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blastdbcmd -entry all -db my_database -out reference.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="302" LastActivityDate="2017-06-19T15:25:03.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="817" PostTypeId="1" AcceptedAnswerId="846" CreationDate="2017-06-19T15:37:17.460" Score="2" ViewCount="2518" Body="&lt;p&gt;I used blastn to search one a genome database for the sequences in a &lt;a href=&quot;http://ix.io/xFG&quot; rel=&quot;nofollow noreferrer&quot;&gt;file&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blastn -out output.txt -outfmt 6 -query sequence_list -db genome_database -perc_identity 100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These sequences are all from one contig of the genome so the output was all hits in one contig. I then &lt;strong&gt;appended&lt;/strong&gt; more sequences to the sequence file and used blastn again. However the weird thing is the output did not start with the same hits as the last blast. I expected if blastn iterated through sequences in a query file that it would output the original sequences first then go through the appended sequences. Since this was not the case, how does blast order its output?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also checked if it was sorted by which contig the hit was in and this is not the case. &lt;/p&gt;&#xA;" OwnerUserId="842" LastActivityDate="2017-06-21T18:01:18.207" Title="How does blastn sort output?" Tags="|sequence-alignment|blast|sequence-analysis|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="818" PostTypeId="1" AcceptedAnswerId="831" CreationDate="2017-06-19T17:41:36.857" Score="5" ViewCount="191" Body="&lt;p&gt;I had a protein Refseq ID and I PSI-BLASTed this sequence against Refseq database. We all know that the Refseq is a Reference sequence database and it shouldn't have redundancy. After BLASTing my sequence, at first iteration I got 1000 hits and among them there were a lot of redundant sequences! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My sequence had 241 amino acids and I found a lot of sequences with 100% identity, 100% cover and 0 E-value exactly the same as my sequence but with different IDs. All of these IDs were from RefSeq! In other iterations and after adjusting format options, I got this redundancy with other sequences from other species. My sequence is related to a chain of a multichain protein (E.coli fumarate reductase)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, when using NP_418578 as a query, I found WP_078165098.1, WP_064226696.1, WP_062863447.1, WP_001401474.1 and other that were identical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know what is wrong with Refseq. Is it really a Reference sequence database? If it is, where does this redundancy come from and why?&lt;/p&gt;&#xA;" OwnerUserId="818" LastEditorUserId="818" LastEditDate="2017-06-20T14:30:41.707" LastActivityDate="2017-06-20T14:30:41.707" Title="Duplicate long hits from PSI-BLAST" Tags="|blast|sequence-homology|refseq|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="819" PostTypeId="2" ParentId="782" CreationDate="2017-06-19T19:55:37.043" Score="2" Body="&lt;p&gt;You should be able to remove any one of the following strains to end up with a rank-sufficient model matrix: 5, 10, 12, 13, 14, 15, 19, 26, 28, 3, 30, 32, 36, 39, 41, 45, 46, 49, 5, 50, 52, 53, 58, 59, 60, 69, 8. As an aside you can figure this sort of thing out as follows (I read your dataframe in a the &lt;code&gt;d&lt;/code&gt; object):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; m = model.matrix(~batch+strain, d)&#xA;&amp;gt; dim(m) # 142 row, 77 columns, so minimum rank is 77&#xA;&amp;gt; qr(m)$rank # 76, so just barely rank insufficient&#xA;&amp;gt; #see if we can remove a single column and still get rank 76&#xA;&amp;gt; colnames(m)[which(sapply(1:77, function(x) qr(m[,-x])$rank) == 76)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You obviously don't want to remove the batch columns or the intercept. The normal tricks that you can sometimes use to get around this issue with case-control studies don't appear to help here, which is why I would just drop a strain and call it done. Keep in mind that your power is still likely terrible. I generally recommend at least 6 replicates per group (scale down the number of groups to fit your budget).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once the desired strain is removed from the model, it can be fit directly into the &lt;code&gt;sleuth_fit&lt;/code&gt; function to obtain the full model:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; m = m[, -9] # whatever column to drop to get the appropriate rank&#xA;&amp;gt; so &amp;lt;- sleuth_fit(so, m, 'full')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastEditorUserId="123" LastEditDate="2017-06-20T11:00:23.183" LastActivityDate="2017-06-20T11:00:23.183" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="820" PostTypeId="2" ParentId="807" CreationDate="2017-06-19T20:19:14.163" Score="2" Body="&lt;p&gt;I wrote &lt;a href=&quot;https://github.com/alexpreynolds/sample&quot; rel=&quot;nofollow noreferrer&quot;&gt;a tool called &lt;code&gt;sample&lt;/code&gt;&lt;/a&gt; that you can use to do random sampling without reading the entire file into memory. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can be used where GNU &lt;code&gt;shuf&lt;/code&gt; fails for lack of sufficient memory. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It requires two passes through the file to do a random sample, but the second pass is generally fast(er) as it uses &lt;code&gt;mmap&lt;/code&gt; routines to do cached reads. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do repeated samples, the repeated samples are also &lt;code&gt;mmap&lt;/code&gt;-ed (cached) and will run quickly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might use it on a FASTQ file like so:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$ sample -k 1234 -l 4 in.fq &amp;gt; out.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It parses the input file into records by every four newline characters (such as the format of a FASTQ file), reading line offset positions into memory. So the memory overhead is relatively very low.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It then applies reservoir sampling on those line offsets to write out a random sample (say, &lt;code&gt;1234&lt;/code&gt; records in this example) to standard output.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="191" LastEditDate="2017-06-22T11:07:39.083" LastActivityDate="2017-06-22T11:07:39.083" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="821" PostTypeId="1" AcceptedAnswerId="823" CreationDate="2017-06-19T20:49:17.313" Score="6" ViewCount="436" Body="&lt;p&gt;My understanding is that indels are from 1bp to 10Kb, and a healthy genome has ~400K-500K Indels. Surely most of these are small. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the distribution of insertion sizes in a healthy human genome? What is the distribution of deletion sizes? What is an average ratio of insertions:deletions? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not found a definitive reference.&lt;/p&gt;&#xA;" OwnerUserId="146" LastEditorUserId="96" LastEditDate="2019-01-15T14:28:19.817" LastActivityDate="2019-01-15T14:28:19.817" Title="What is the distribution of indel sizes in a healthy human genome? of insertion:deletion ratios?" Tags="|variation|indel|literature-search|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="822" PostTypeId="2" ParentId="807" CreationDate="2017-06-19T21:30:20.647" Score="1" Body="&lt;p&gt;Here's another approach that doesn't require any indexing, using &lt;a href=&quot;http://bedops.readthedocs.io/en/latest/content/reference/set-operations/bedextract.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS &lt;code&gt;bedextract&lt;/code&gt;&lt;/a&gt; to do a &lt;code&gt;log(n)&lt;/code&gt; sample on a sorted BED file. Your sample contains random records with equal probability &lt;code&gt;1/n&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach requires a single &lt;code&gt;O(n)&lt;/code&gt; pass through the file to transform it to a BED file:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$ cat records.fastq | paste - - - - | awk '{ print &quot;chrZ\t&quot;s&quot;\t&quot;(s+1)&quot;$0 }' &amp;gt; records.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Store the intervals in a separate file:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$ cut -f1-3 records.bed &amp;gt; intervals.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To do a random sample of &lt;code&gt;k&lt;/code&gt; elements, shuffle the intervals file and preserve the order of shuffled elements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do this with the &lt;a href=&quot;https://github.com/alexpreynolds/sample&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;sample&lt;/code&gt; tool&lt;/a&gt; I outlined earlier:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$ sample -k ${K} -s intervals.bed &amp;gt; intervals-sample.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or you can &lt;code&gt;shuf&lt;/code&gt; and &lt;code&gt;sort-bed&lt;/code&gt; to do the same thing:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$ shuf -n ${K} intervals.bed | sort-bed - &amp;gt; intervals-sample.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There's an &lt;code&gt;O(klog(k))&lt;/code&gt; cost here, but if &lt;code&gt;k &amp;lt;&amp;lt;&amp;lt; n&lt;/code&gt;, i.e., you're working with whole-genome scale input, this cost is amortized over the &lt;code&gt;log(n)&lt;/code&gt; search performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, use &lt;code&gt;bedextract&lt;/code&gt; to do a binary search on the records, and delinearize to get back to FASTQ:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$ bedextract records.bed intervals-sample.bed | cut -f4 | tr '\t' '\n' &amp;gt; sample.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With Unix I/O streams, this can be done in one pass:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$ sample -k ${K} -s intervals.bed | bedextract records.bed - | cut -f4 | tr '\t' '\n' &amp;gt; sample.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;By baking the sort order into &lt;code&gt;records.bed&lt;/code&gt;, you're guaranteed the ability to do a binary search, which is &lt;code&gt;log(n)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note:&lt;/em&gt; Further, by linearizing the FASTQ input to a BED file and querying on BED intervals, you have equal probability of picking any one interval (interval == FQ record). You can draw an unbiased sample without the hassle of creating and storing a separate index.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="191" LastEditDate="2017-06-22T11:07:13.503" LastActivityDate="2017-06-22T11:07:13.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="823" PostTypeId="2" ParentId="821" CreationDate="2017-06-19T21:43:05.247" Score="6" Body="&lt;p&gt;One of the &lt;a href=&quot;https://www.nature.com/nature/journal/v526/n7571/full/nature15394.html&quot; rel=&quot;noreferrer&quot;&gt;2015 papers from the 1000 genomes project&lt;/a&gt; has a nice figure (figure 1) showing the size distribution of medium to large sized insertions and deletions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bFMCX.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bFMCX.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;https://www.nature.com/nature/journal/v526/n7571/full/nature15393.html&quot; rel=&quot;noreferrer&quot;&gt;another 2015 1000 genomes paper&lt;/a&gt;, one can see that the absolute number of smaller indels is much larger, though an exact size range isn't given (as far as I saw). If you really want to know that, just download &lt;a href=&quot;ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/&quot; rel=&quot;noreferrer&quot;&gt;the most recent 1000 genomes VCF file(s)&lt;/a&gt; and compute the exact median size and/or distribution that you want.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-06-19T21:43:05.247" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="824" PostTypeId="2" ParentId="807" CreationDate="2017-06-20T00:43:57.250" Score="1" Body="&lt;p&gt;I wrote a tool called &lt;a href=&quot;https://github.com/mdshw5/strandex&quot; rel=&quot;nofollow noreferrer&quot;&gt;strandex&lt;/a&gt; that matches entire FASTQ records starting at a random offset inside a decompressed file (since non-block compressed gzip files are not seek-able). It was a response to &lt;a href=&quot;https://github.com/dib-lab/khmer/issues/1002#issuecomment-102649433&quot; rel=&quot;nofollow noreferrer&quot;&gt;this comment&lt;/a&gt; about the &lt;code&gt;screed&lt;/code&gt; FASTQ parser in &lt;a href=&quot;https://github.com/dib-lab/khmer&quot; rel=&quot;nofollow noreferrer&quot;&gt;khmer&lt;/a&gt;, and was initially just a proof of concept but it's been useful for me and a few other people. The general idea is to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;seek to a random offset in the file&lt;/li&gt;&#xA;&lt;li&gt;read a chunk of the file&lt;/li&gt;&#xA;&lt;li&gt;test if the regex pattern &lt;code&gt;@.+[\n\r]+.+[\n\r]+\+.*?[\n\r].+[\n\r]&lt;/code&gt; matches&lt;/li&gt;&#xA;&lt;li&gt;if so, extract the matching FASTQ record using regex capture groups&lt;/li&gt;&#xA;&lt;li&gt;if no match, goto step 2&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The CLI script and python library can be installed with &lt;code&gt;pip install strandex&lt;/code&gt;, and by default the script samples &lt;code&gt;-n&lt;/code&gt; reads using file offsets starting at a random number (reproducable by setting &lt;code&gt;-s&lt;/code&gt; seed), then moves a number of bytes to uniformly sample the entire file, determined by the file size. In this way the process is not truly random, but random enough, and avoids sampling any part of the file too much. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you specify &lt;code&gt;-n&lt;/code&gt; reads less than the total number in the file you get &lt;em&gt;down&lt;/em&gt;sampling, and if &lt;code&gt;-n&lt;/code&gt; is greater than the total number of reads you get &lt;em&gt;up&lt;/em&gt;sampling. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantage of this method over others is - it's a one-pass method, and since it's using the python C internals regex engine it's quite fast.&lt;/p&gt;&#xA;" OwnerUserId="267" LastActivityDate="2017-06-20T00:43:57.250" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="825" PostTypeId="2" ParentId="722" CreationDate="2017-06-20T01:49:07.367" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://bioviz.org/igb/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IGB&lt;/a&gt; gives the intended result without much hassle. It doesn't have the bells and whistles of IGV, but presents a clean and intuitive view of the individual reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In IGB, each read is presented as a single, contiguous line. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/n80lR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/n80lR.png&quot; alt=&quot;IGB zoomed out view&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you zoom in you can see the read ID for each read.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ntXKs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ntXKs.png&quot; alt=&quot;IGB zoomed in view&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a more holistic view, IGV's Sashimi plot can be modified to exclude spurious junctions supported by few reads. Here is a section of the above gene with default settings:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uuaLY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uuaLY.png&quot; alt=&quot;Sashimi plot with default settings&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The minimum junction coverage can be set by right-clicking on the Sashimi plot itself, or application-wide under View -&gt; Preferences -&gt; Alignments -&gt; Splice Junction Track Options. With minimum junction coverage 3, the noisy plot above now looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/A4GTP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/A4GTP.png&quot; alt=&quot;Sashimi plot with min junction coverage 3&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The numbers are still hard to read due to numerous overlapping almost-identical junctions, but at least the plot is now readable.&lt;/p&gt;&#xA;" OwnerUserId="163" LastEditorUserId="163" LastEditDate="2017-06-20T02:12:39.600" LastActivityDate="2017-06-20T02:12:39.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="826" PostTypeId="2" ParentId="821" CreationDate="2017-06-20T02:17:29.127" Score="5" Body="&lt;p&gt;Genome-In-A-Bottle (GIAB; version 3.3.2) contains 3.21M SNPs on auto+X chromosomes and 0.51M INDELs in 2.58Gb confident regions. The ins:del ratio is 0.92. On the CHM1-CHM13 pacbio assembly (European ancestry), there are 3.57M auto+X SNPs and 0.58M INDELs in 2.71Gb confident regions [&lt;a href=&quot;https://rdcu.be/232R&quot; rel=&quot;nofollow noreferrer&quot;&gt;reference&lt;/a&gt;] with an ins:del ratio 0.99. These give you an idea of relative short INDELs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For indels longer than 50bp, I would highly recommend to read &lt;a href=&quot;https://www.nature.com/nature/journal/v517/n7536/full/nature13907.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; by Chaisson et al. This call set is constructed from the whole-genome assembly of the CHM1 genome. It is far more comprehensive and probably more accurate than all the other call sets. The following is Table 1 from the paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/om410.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/om410.png&quot; alt=&quot;CHM1-SVs&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to say that the high ins-to-del ratio is surprising to me, but this is what the data tells us. I have done a similar analysis and reached a similar ratio.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: These numbers are essentially from one haplotype. They will be higher for a diploid sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: some numbers in the original post are wrong.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2018-07-20T01:28:43.777" LastActivityDate="2018-07-20T01:28:43.777" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="827" PostTypeId="2" ParentId="817" CreationDate="2017-06-20T03:16:04.833" Score="2" Body="&lt;p&gt;BLASTN should definitely report alignments in the order that the query sequences were provided.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on your description and my personal experience, I think the most likely explanation is some kind of mix up. It happens to all of us, especially when we're in a rush to get an answer, but even sometimes when we're being careful and disciplined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can go back and recreate the result, I would be very interested to see the two pairs of input and output files. If not, I wouldn't spend too much more time worrying about it. :-)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt; Alignments are reported in the order that the query sequences are provided, even in tabular mode. The order of each query's hits is sorted by bitscore or e-value, but that is a secondary sorting. I've included an example below that has two queries, &lt;code&gt;seq1&lt;/code&gt; and &lt;code&gt;seq2&lt;/code&gt; with &lt;code&gt;-outfmt 6&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seq1    scaffold2       100.000 280     0       0       1       280     11131   11410   7.73e-146       518&#xA;seq1    scaffold294     88.693  283     25      4       1       279     12160   12439   6.46e-92        339&#xA;seq1    scaffold1525    85.315  286     29      9       1       277     27142   26861   3.07e-75        283&#xA;seq1    scaffold1874    83.505  291     32      10      1       280     8558    8273    1.86e-67        257&#xA;seq1    scaffold147     82.653  294     34      9       1       279     10797   10506   1.45e-63        244&#xA;seq1    scaffold2478    82.374  278     38      8       11      280     7622    7896    1.13e-59        231&#xA;seq1    scaffold3405    87.940  199     17      5       1       196     14390   14584   1.46e-58        228&#xA;seq1    scaffold395     83.060  183     23      4       106     280     12021   11839   5.40e-38        159&#xA;seq1    scaffold2853    85.965  114     13      3       1       113     20864   20753   9.16e-26        119&#xA;seq2    scaffold9       100.000 140     0       0       1       140     1121    1260    2.34e-68        259&#xA;seq2    scaffold2950    90.909  110     6       3       1       106     110     1       6.84e-34        145&#xA;seq2    scaffold3416    95.385  65      3       0       1       65      11882   11818   1.16e-21        104&#xA;seq2    scaffold3103    95.385  65      3       0       1       65      9736    9672    1.16e-21        104&#xA;seq2    scaffold5297    88.608  79      7       1       64      140     150     228     6.99e-19        95.3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2017-06-21T17:39:45.060" LastActivityDate="2017-06-21T17:39:45.060" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="828" PostTypeId="2" ParentId="633" CreationDate="2017-06-20T08:34:17.377" Score="2" Body="&lt;p&gt;In the &lt;a href=&quot;http://cancerres.aacrjournals.org/content/72/14/3499.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; describing Cellminer says about the cross-correlations:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;... a cross-correlation table of the resultant z-scores can be&#xA;  generated ...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;On page 3503&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the z-scores are defined previously as:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The tool output includes relative transcript intensity presented&#xA;  as z-scores...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="48" LastActivityDate="2017-06-20T08:34:17.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="829" PostTypeId="1" CreationDate="2017-06-20T08:58:36.057" Score="3" ViewCount="162" Body="&lt;p&gt;I would like to look if there are mutations in residues of human histones associated with any disease. For instance, if a mutation in residue K6 (lysine 6) of histone &lt;a href=&quot;http://www.uniprot.org/uniprot/Q96QV6#function&quot; rel=&quot;nofollow noreferrer&quot;&gt;H2A1A&lt;/a&gt; is associated with any human disease, (with PUBMED evidence, preferably), but in a systematic way (for all types of histones and all residues). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The purpose of this is to get some candidate residues to study. I have checked the &lt;a href=&quot;https://www.omim.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OMIM&lt;/a&gt; database but I didn't find anything relevant. It seems that there is no systematic way to connect specific residues to diseases and search for them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any web database or API available for this purpose?&lt;/p&gt;&#xA;" OwnerUserId="678" LastEditorUserId="96" LastEditDate="2019-01-15T14:27:58.787" LastActivityDate="2019-01-15T14:27:58.787" Title="How can I find mutations associated with disease in human histone residues?" Tags="|database|variation|api|" AnswerCount="2" CommentCount="7" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="831" PostTypeId="2" ParentId="818" CreationDate="2017-06-20T12:11:46.953" Score="2" Body="&lt;p&gt;This is what is known as a feature, not a bug. Note that your identical proteins all have accessions starting with &lt;code&gt;WP_&lt;/code&gt;. These are special &amp;quot;non-redundant proteins&amp;quot;. Many sequences—particularly bacterial sequences—are identical between various different species so having a separate RefSeq entry for each of them would be inefficient. Therefore, RefSeq combines multiple such proteins into a single &lt;code&gt;WP_&lt;/code&gt; record. This is documented &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/refseq/about/nonredundantproteins/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (emphasis mine):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A new type of RefSeq protein record which represents non-redundant protein sequences was introduced in mid-2013. This record type was introduced to address a growing issue with redundancy in the Prokaryotic RefSeq protein dataset that coincided with a significant increase in bacterial genome submissions from individual isolates and closely related bacterial strains. For example, a large number of high-quality bacterial genomes may be submitted during a disease outbreak. The submitted sequences may reflect pathogen evolution during the course of the outbreak but the majority of the encoded proteins from these genomes may be identical to each other. As RefSeq includes these genomes, per community requests, this resulted in increased redundancy. &lt;strong&gt;By representing identical proteins using a single non-redundant protein accession number (with the prefix 'WP_'), redundancy in the database is significantly reduced.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;[ . . . ]&lt;/p&gt;&#xA;&lt;p&gt;Because a non-redundant protein sequence may be found in RefSeq genomes from multiple species, &lt;strong&gt;the organism information provided on the protein record reflects the lowest-common taxonomic node ranging from the genus species level to super-kingdom&lt;/strong&gt;. A non-redundant protein record that provides organism information at the level of a genus, family, or even super-kingdom does not mean that the protein is found in all RefSeq genomes below that taxonomic classification. It only indicates that the protein is found in more than one genome of different species for which the genus, family, or super-kingdom classification is the lowest common taxonomic node.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, your query was &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/protein/NP_418578&quot; rel=&quot;nofollow noreferrer&quot;&gt;NP_418578.1&lt;/a&gt;, anaerobic fumarate reductase catalytic and NAD/flavoprotein subunit from &lt;em&gt;E. coli&lt;/em&gt; strain K-12, substrain MG1655. The first thing to notice is how specific that is. This is the protein found from one specific substrain of one specific strain of one specific bacterial species. It is reasonable to expect that there will be identical sequences from many, many closely related species. Both from, most probably, all other strains and substrains of &lt;em&gt;E. coli&lt;/em&gt; and from other, related bacteria.&lt;/p&gt;&#xA;&lt;p&gt;Now, the specific sequences you mention are actually &lt;em&gt;slightly&lt;/em&gt; different and not 100% identical. Below is a multiple alignment of NP_418578.1 and the 4 WP_ sequences you mentioned. Note that each of the 5 entries is slightly different. Each has one residue that differs from the rest. Look for the &lt;code&gt;:&lt;/code&gt; in the identity line, there are 4 &lt;code&gt;:&lt;/code&gt; and all others are &lt;code&gt;*&lt;/code&gt; (I am only showing the relevant alignment blocks here and have removed those where all 4 sequences were identical):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;WP_001401474.1      MQTFQADLAIVGAGGAGLRAAIAAAQANPNAKIALISKVYPMRSHTVAAEGGSAAVAQDH&#xA;WP_062863447.1      MQTFQADLAIVGAGGAGLRAAIAAAQANPNAKIALISKVYPMRSHTVAAEGGSAAVAQDH&#xA;WP_064226696.1      MQTFQADLAIVGAGGAGLRAAIAAAQANPNAKIALISKVYPMRSHTVAAEGGSAAVAQDH&#xA;NP_418578.1         MQTFQADLAIVGAGGAGLRAAIAAAQANPNAKIALISKVYPMRSHTVAAEGGSAAVAQDH&#xA;WP_078165098.1      MQTFQADLAIVGAGGAGLRAAIAAAQANPNAKIALISKVYPMRSHTVAAEGGSAAIAQDH&#xA;                    *******************************************************:****&#xA;&#xA;[ . . . ]&#xA;&#xA;WP_001401474.1      KIERTWFAADKTGFHMLHTLFQTSLQFPQIQRFDEHFVLDILVDDGHVRGLVAMNMMEGT&#xA;WP_062863447.1      KIERTWFAADKTGFHMLHTLFQTSLQFPQIQRFDEHFVLDILVDDGHVRGLVAMNMMEGT&#xA;WP_064226696.1      KIERTWFAADKTGFHMLHTLFQTSLQFPQIQRFDEHFVLDILVDDGHIRGLVAMNMMEGT&#xA;NP_418578.1         KIERTWFAADKTGFHMLHTLFQTSLQFPQIQRFDEHFVLDILVDDGHVRGLVAMNMMEGT&#xA;WP_078165098.1      KIERTWFAADKTGFHMLHTLFQTSLQFPQIQRFDEHFVLDILVDDGHVRGLVAMNMMEGT&#xA;                    ***********************************************:************&#xA;&#xA;[ . . . ]&#xA;&#xA;WP_001401474.1      GILMTEGCRGEGGILVNKNGYRYLQDYGMGPETPLGEPKNKYMELGPRDKVSQAFWHEWR&#xA;WP_062863447.1      GILMTEGCRGEGGILVNKNGYRYLQDYGMGPETPLGEPKNKYMELGPRDKISQAFWHEWR&#xA;WP_064226696.1      GILMTEGCRGEGGILVNKNGYRYLQDYGMGPETPLGEPKNKYMELGPRDKVSQAFWHEWR&#xA;NP_418578.1         GILMTEGCRGEGGILVNKNGYRYLQDYGMGPETPLGEPKNKYMELGPRDKVSQAFWHEWR&#xA;WP_078165098.1      GILMTEGCRGEGGILVNKNGYRYLQDYGMGPETPLGEPKNKYMELGPRDKVSQAFWHEWR&#xA;                    **************************************************:*********&#xA;&#xA;WP_001401474.1      KGNTISTPRGDVVYLDLRHLGEKKLHERLPFICELAKAYVGIDPVKEPIPVRPTAHYTMG&#xA;WP_062863447.1      KGNTISTPRGDVVYLDLRHLGEKKLHERLPFICELAKAYVGVDPVKEPIPVRPTAHYTMG&#xA;WP_064226696.1      KGNTISTPRGDVVYLDLRHLGEKKLHERLPFICELAKAYVGVDPVKEPIPVRPTAHYTMG&#xA;NP_418578.1         KGNTISTPRGDVVYLDLRHLGEKKLHERLPFICELAKAYVGVDPVKEPIPVRPTAHYTMG&#xA;WP_078165098.1      KGNTISTPRGDVVYLDLRHLGEKKLHERLPFICELAKAYVGVDPVKEPIPVRPTAHYTMG&#xA;                    *****************************************:******************&#xA;&#xA;[ . . . ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Your sequence (NP_418578.1) is only identical to one &lt;code&gt;WP_*&lt;/code&gt; multi-species sequence, WP_001192973:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;WP_001192973.1      MQTFQADLAIVGAGGAGLRAAIAAAQANPNAKIALISKVYPMRSHTVAAEGGSAAVAQDH&#xA;NP_418578.1         MQTFQADLAIVGAGGAGLRAAIAAAQANPNAKIALISKVYPMRSHTVAAEGGSAAVAQDH&#xA;                    ************************************************************&#xA;&#xA;WP_001192973.1      DSFEYHFHDTVAGGDWLCEQDVVDYFVHHCPTEMTQLELWGCPWSRRPDGSVNVRRFGGM&#xA;NP_418578.1         DSFEYHFHDTVAGGDWLCEQDVVDYFVHHCPTEMTQLELWGCPWSRRPDGSVNVRRFGGM&#xA;                    ************************************************************&#xA;&#xA;WP_001192973.1      KIERTWFAADKTGFHMLHTLFQTSLQFPQIQRFDEHFVLDILVDDGHVRGLVAMNMMEGT&#xA;NP_418578.1         KIERTWFAADKTGFHMLHTLFQTSLQFPQIQRFDEHFVLDILVDDGHVRGLVAMNMMEGT&#xA;                    ************************************************************&#xA;&#xA;WP_001192973.1      LVQIRANAVVMATGGAGRVYRYNTNGGIVTGDGMGMALSHGVPLRDMEFVQYHPTGLPGS&#xA;NP_418578.1         LVQIRANAVVMATGGAGRVYRYNTNGGIVTGDGMGMALSHGVPLRDMEFVQYHPTGLPGS&#xA;                    ************************************************************&#xA;&#xA;WP_001192973.1      GILMTEGCRGEGGILVNKNGYRYLQDYGMGPETPLGEPKNKYMELGPRDKVSQAFWHEWR&#xA;NP_418578.1         GILMTEGCRGEGGILVNKNGYRYLQDYGMGPETPLGEPKNKYMELGPRDKVSQAFWHEWR&#xA;                    ************************************************************&#xA;&#xA;WP_001192973.1      KGNTISTPRGDVVYLDLRHLGEKKLHERLPFICELAKAYVGVDPVKEPIPVRPTAHYTMG&#xA;NP_418578.1         KGNTISTPRGDVVYLDLRHLGEKKLHERLPFICELAKAYVGVDPVKEPIPVRPTAHYTMG&#xA;                    ************************************************************&#xA;&#xA;WP_001192973.1      GIETDQNCETRIKGLFAVGECSSVGLHGANRLGSNSLAELVVFGRLAGEQATERAATAGN&#xA;NP_418578.1         GIETDQNCETRIKGLFAVGECSSVGLHGANRLGSNSLAELVVFGRLAGEQATERAATAGN&#xA;                    ************************************************************&#xA;&#xA;WP_001192973.1      GNEAAIEAQAAGVEQRLKDLVNQDGGENWAKIRDEMGLAMEEGCGIYRTPELMQKTIDKL&#xA;NP_418578.1         GNEAAIEAQAAGVEQRLKDLVNQDGGENWAKIRDEMGLAMEEGCGIYRTPELMQKTIDKL&#xA;                    ************************************************************&#xA;&#xA;WP_001192973.1      AELQERFKRVRITDTSSVFNTDLLYTIELGHGLNVAECMAHSAMARKESRGAHQRLDEGC&#xA;NP_418578.1         AELQERFKRVRITDTSSVFNTDLLYTIELGHGLNVAECMAHSAMARKESRGAHQRLDEGC&#xA;                    ************************************************************&#xA;&#xA;WP_001192973.1      TERDDVNFLKHTLAFRDADGTTRLEYSDVKITTLPPAKRVYGGEADAADKAEAANKKEKA&#xA;NP_418578.1         TERDDVNFLKHTLAFRDADGTTRLEYSDVKITTLPPAKRVYGGEADAADKAEAANKKEKA&#xA;                    ************************************************************&#xA;&#xA;WP_001192973.1      NG&#xA;NP_418578.1         NG&#xA;                    **&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, in summary, RefSeq will combine multiple identical sequences into a single &lt;code&gt;WP_*&lt;/code&gt; multi-species accession. You should therefore expect to find one 100% identical &lt;code&gt;WP_*&lt;/code&gt; sequence for your query and multiple, &lt;em&gt;almost&lt;/em&gt; identical &lt;code&gt;WP_*&lt;/code&gt; entries. And that's precisely what you see here.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-06-20T12:11:46.953" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="834" PostTypeId="2" ParentId="817" CreationDate="2017-06-20T16:44:09.773" Score="0" Body="&lt;p&gt;I have analyzed the situation more and found that blastn automatically sorts by e-value. &lt;a href=&quot;http://ix.io/xLh&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ix.io/xLh&lt;/a&gt; (column 11)&lt;/p&gt;&#xA;" OwnerUserId="842" LastActivityDate="2017-06-20T16:44:09.773" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="835" PostTypeId="2" ParentId="327" CreationDate="2017-06-20T16:44:39.103" Score="0" Body="&lt;p&gt;If you are looking for NGS QC for your fastq, bam, bed and vcf files I would suggest a commercial tool called &lt;a href=&quot;https://om.euformatics.com/apex/f?p=118:1:&quot; rel=&quot;nofollow noreferrer&quot;&gt;omnomicsQ&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It automates coverage analysis and sample pass, warn fail according according to your defined SOPs. Unlike fastqc it also comes with a database so that you can compare protocols, samples, and your performance to peer organisations. It also charts performance over time, exceptions,and  correlations between metrics&lt;/p&gt;&#xA;" OwnerUserId="924" LastActivityDate="2017-06-20T16:44:39.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="836" PostTypeId="2" ParentId="807" CreationDate="2017-06-20T17:20:24.900" Score="2" Body="&lt;p&gt;One of the most thorough treatments of this question (or a similar question: grabbing a random subset of reads) was given by Jared Simpson in a blog post a few years ago. &lt;a href=&quot;http://simpsonlab.github.io/2015/05/19/io-performance/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://simpsonlab.github.io/2015/05/19/io-performance/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you just want to grab a single random read, Jared's benchmarks suggest that seeking to a random position in the file and then retrieving the next complete read should be the most performant option. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, more generally, if you want to take a random subset of reads, there are many factors that affect performance.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2017-06-20T17:20:24.900" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="837" PostTypeId="1" AcceptedAnswerId="843" CreationDate="2017-06-20T19:23:49.107" Score="4" ViewCount="272" Body="&lt;p&gt;I'm beginning with the reference genome in FASTA format, hg19. I am reading the sequence into a Python dictionary with &lt;code&gt;BioPython&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;genome_dictionary = {} &#xA;for seq_record.id in SeqIO.parse(input_fasta_file, &quot;fasta&quot;):&#xA;    genome_dictionary[seq_record.id] = seq_record&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This creates a Python dictionary with keys as chromosome names and values as strings from the FASTA. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to take the FASTA, and simulate artificial SNPs and InDels along the FASTA to benchmark various algorithms. If I was only creating SNPs in the FASTA, this would be a trivial problem. The reference FASTA has the original coordinates, and the SNPs would also be in a coordinate system which is the same. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I randomly insert an insertion (e.g. an insertion of 12 bp), I move all events into a new coordinate system such that &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;new coordinate system = old coordinate system + 15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This becomes very difficult to keep track off throughout the entire genome after several interations. I therefore cannot keep track of the new and old coordinate system without running an algorithm and creating a VCF, which may have errors. This defeats the purpose of the simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output would be either a BED file or VCF which has keep track of the changes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What method/data structure is used to solve this problem? Would LiftOver work? Maybe CrossMap? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://genome.sph.umich.edu/wiki/LiftOver&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genome.sph.umich.edu/wiki/LiftOver&lt;/a&gt;&#xA;&lt;a href=&quot;http://crossmap.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://crossmap.sourceforge.net/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="146" LastEditorUserId="298" LastEditDate="2017-07-05T08:32:22.120" LastActivityDate="2017-07-05T08:32:22.120" Title="After artificially creating events in a FASTA file, how do I keep track of the old coordinates?" Tags="|python|bed|structural-variation|indel|simulated-data|" AnswerCount="2" CommentCount="19" ContentLicense="CC BY-SA 3.0" />
  <row Id="838" PostTypeId="1" AcceptedAnswerId="10656" CreationDate="2017-06-20T19:26:57.350" Score="5" ViewCount="1614" Body="&lt;p&gt;I have two files &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;s3.txt :&#xA;1   10  20&#xA;1   5   20&#xA;2   20  30&#xA;2   25  30&#xA;1   10  50&#xA;2   20  60&#xA;1   14  17&#xA;&#xA;s4.txt:&#xA;1   10  20&#xA;2   20  30&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am trying to match col0 of both the files and get rows that fall between range(inclusive of themselves) 10-20 and 20-30 as seen in s4 file.&#xA;file s4 has co ordinates which can be used as reference range (chrom start and end) and s3 has list of co ordinates from an experimental condition what I am trying to achieve is to which co-ordinates from my file s3 fall on or between my reference co ordinates in s4.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;code so far:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;containing_ranges = []&#xA;with open('s4.txt', 'r') as f:&#xA;    for line in f:&#xA;        fields = line.strip().split('\t')&#xA;        containing_ranges.append(fields)&#xA;&#xA;tested_ranges = []       &#xA;with open('s3.txt', 'r') as f:&#xA;    for line in f:&#xA;        fields = line.strip().split('\t')&#xA;        tested_ranges.append(fields)&#xA;&#xA;for c_range in containing_ranges:&#xA;    for t_range in tested_ranges:&#xA;        tst = int(t_range[1])&#xA;        ten = int(t_range[2])&#xA;        cst = int(c_range[1])&#xA;        cen = int(c_range[2])&#xA;        if  c_range[0] == t_range[0]:&#xA;            included = cst &amp;gt;= tst and cen &amp;lt;= ten&#xA;            if included == True:&#xA;               print t_range&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output with missing row(1   14  17) : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;['1', '10', '20']&#xA;['1', '5', '20']&#xA;['1', '10', '50']&#xA;['2', '20', '30']&#xA;['2', '20', '60']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Desired output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   10  20&#xA;2   20  30&#xA;2   25  30&#xA;1   14  17&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Not sure if my logic is wrong and why does it miss 14-17 as it falls between 10-20&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[EDIT] using pybedtools&#xA;&#xA;&amp;gt;&amp;gt;&amp;gt; print(s4.intersect(s3, wb=True))&#xA;1   10  20  1   10  20&#xA;1   10  20  1   5   20&#xA;1   10  20  1   10  50&#xA;1   14  17  1   14  17&#xA;2   20  30  2   20  30&#xA;2   25  30  2   25  30&#xA;2   20  30  2   20  60&#xA;&#xA;&amp;gt;&amp;gt;&amp;gt; print(s4.intersect(s3, wa=True, wb=True, F=1))&#xA;1   10  20  1   10  20&#xA;1   10  20  1   14  17&#xA;2   20  30  2   20  30&#xA;2   20  30  2   25  30&#xA;&#xA;&#xA;using bedops &#xA;bin$ less answer.bed &#xA;1       5       20&#xA;1       10      20&#xA;1       10      50&#xA;1       14      17&#xA;2       20      30&#xA;2       20      60&#xA;2       25      30&#xA;&#xA;using @bli code(on python2.7)&#xA;('1', 10, 20)&#xA;('1', 14, 17)&#xA;('2', 20, 30)&#xA;('2', 25, 30)&#xA; why can I not see the interval 1 5 20&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="926" LastEditorUserId="926" LastEditDate="2017-06-21T14:47:38.947" LastActivityDate="2019-10-30T08:15:23.480" Title="Range overlap python error with genomic regions" Tags="|bed|genomics|python|homework|" AnswerCount="4" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="839" PostTypeId="2" ParentId="838" CreationDate="2017-06-20T19:35:59.390" Score="1" Body="&lt;p&gt;You could use &lt;a href=&quot;http://bedops.readthedocs.io/en/latest/#&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS&lt;/a&gt;, instead:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sort-bed s3.txt &amp;gt; s3.bed&#xA;$ sort-bed s4.txt &amp;gt; s4.bed&#xA;$ bedops --element-of 1 s3.bed s4.bed &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you need to run it from within Python, you could use &lt;code&gt;subprocess.check_output()&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import subprocess&#xA;...&#xA;try:&#xA;    result = subprocess.check_output(&quot;bedops --element-of 1 %s %s &amp;gt; %s&quot; % (set_a_fn, set_b_fn, answer_fn), shell=True)&#xA;except subprocess.CalledProcessError as err:&#xA;    raise SystemExit(&quot;Could not run bedops\n&quot;)&#xA;# do stuff with 'result'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="776" LastActivityDate="2017-06-20T19:35:59.390" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="840" PostTypeId="2" ParentId="838" CreationDate="2017-06-20T19:38:59.547" Score="5" Body="&lt;p&gt;You're reinventing &lt;code&gt;bedtools intersect&lt;/code&gt; (or bedops), for which there's &lt;a href=&quot;https://daler.github.io/pybedtools/&quot; rel=&quot;nofollow noreferrer&quot;&gt;already a convenient python module&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;from pybedtools import BedTool&#xA;&#xA;s3 = BedTool('s3.bed')&#xA;s4 = BedTool('s4.bed')&#xA;&#xA;print(s4.intersect(s3, wa=True, wb=True, F=1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;wb=True&lt;/code&gt; is equivalent to &lt;code&gt;-wb&lt;/code&gt; with &lt;code&gt;bedtools intersect&lt;/code&gt; on the command line. Similarly, &lt;code&gt;F=1&lt;/code&gt; is the same as &lt;code&gt;-F 1&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-06-22T11:22:23.687" LastActivityDate="2017-06-22T11:22:23.687" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="841" PostTypeId="2" ParentId="837" CreationDate="2017-06-20T20:04:39.883" Score="4" Body="&lt;p&gt;If you look for a program which would randomly introduce SNPs + short indels and then would save everything into a VCF file, &lt;a href=&quot;https://github.com/nh13/DWGSIM&quot; rel=&quot;nofollow noreferrer&quot;&gt;DWGsim&lt;/a&gt; or &lt;a href=&quot;https://github.com/seqan/seqan/blob/master/apps/mason2/README.mason_variator&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mason Variator&lt;/a&gt; could be a good choice. Then you can create a corresponding Chain file using &lt;code&gt;bcftools consensus -c&lt;/code&gt; and transform various formats between these two coordinate systems using &lt;a href=&quot;http://crossmap.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CrossMap&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="425" LastActivityDate="2017-06-20T20:04:39.883" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="842" PostTypeId="1" CreationDate="2017-06-21T02:20:33.860" Score="7" ViewCount="143" Body="&lt;p&gt;I'm currently attempting association analysis with an extremely small set of patient exomes (n=10), with no control or parental exomes available. Downloading the ExAC VCF of variant sites (&lt;a href=&quot;http://exac.broadinstitute.org/downloads&quot; rel=&quot;noreferrer&quot;&gt;http://exac.broadinstitute.org/downloads&lt;/a&gt;) or the 1000G integrated call sets (&lt;a href=&quot;http://ftp.1000genomes.ebi.ac.uk/&quot; rel=&quot;noreferrer&quot;&gt;http://ftp.1000genomes.ebi.ac.uk/&lt;/a&gt;) and combining this with our pooled patient VCFs has not been successful (I suspect the approach of attempting to merge such large VCFs generated from different pipelines is rather naive).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at the primary literature, I have gathered it should be possible to use these resources to help increase statistical power for our analysis. My question is how do I take these large .vcfs with many samples and successfully merge them to our patient .vcfs, such that the combined VCF can be used downstream to run analysis packages? (PODKAT, PLINK, etc.)&lt;/p&gt;&#xA;" OwnerUserId="927" LastEditorUserId="96" LastEditDate="2019-01-15T14:27:41.027" LastActivityDate="2019-01-15T14:27:41.027" Title="What is a good pipeline for using public domain exomes as controls?" Tags="|ngs|public-databases|variation|exome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="843" PostTypeId="2" ParentId="837" CreationDate="2017-06-21T02:49:42.480" Score="5" Body="&lt;p&gt;I've written a handful of programs from scratch to simulate mutations and variations in real or simulated sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trick has always been to sort the variants by genomic coordinate, &lt;strong&gt;apply the variant with the largest coordinate first&lt;/strong&gt;, then apply the variant with the second largest coordinate, all the way down to the variant with the smallest coordinate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indels and other variants that affect sequence structure only affect &lt;em&gt;subsequent&lt;/em&gt; coordinates. So going in reverse order ensures variants at the beginning of the sequence don't throw off variants at the end of the sequence.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2017-06-22T20:27:43.903" LastActivityDate="2017-06-22T20:27:43.903" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="844" PostTypeId="1" CreationDate="2017-06-21T04:38:04.053" Score="0" ViewCount="117" Body="&lt;p&gt;I have some difficulties to understand/interpret the &lt;a href=&quot;https://en.wikipedia.org/wiki/MAPK/ERK_pathway&quot; rel=&quot;nofollow noreferrer&quot;&gt;pathway map&lt;/a&gt; and how a gene-gene interaction list or DNA sequencing can map into pathways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition what's the difference between &lt;a href=&quot;https://en.wikipedia.org/wiki/MAPK/ERK_pathway&quot; rel=&quot;nofollow noreferrer&quot;&gt;MARK/ERK pathway&lt;/a&gt; and the output of &lt;a href=&quot;http://www.kegg.jp/kegg-bin/show_pathway?hsa05200&quot; rel=&quot;nofollow noreferrer&quot;&gt;KEGG&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2017-06-22T03:02:52.990" LastActivityDate="2017-06-22T03:02:52.990" Title="How can I find the relevant pathway map from gene-gene or protein-protein interaction list?" Tags="|gene|ngs|phylogenetics|pathway|kegg|" AnswerCount="2" CommentCount="8" ClosedDate="2017-06-23T15:02:07.927" ContentLicense="CC BY-SA 3.0" />
  <row Id="845" PostTypeId="2" ParentId="774" CreationDate="2017-06-21T08:07:33.170" Score="3" Body="&lt;p&gt;I am not sure if this has been done, is common, or are research lines, but here is what I think can be done with transcripts' differences (aside from comparing the change of the expression of each transcript):&lt;/p&gt;&#xA;&lt;p&gt;Having the different transcripts, enables us to study also the relation between transcripts, which are co-expressed together or differently co-expressed. Maybe at the individual level the switch between transcripts of a gene is not related to a condition, but together with other transcripts it is.&lt;/p&gt;&#xA;&lt;p&gt;Transcripts usage could be also linked to structural or genomic changes. Either short/small changes or big. At genomic level SNP, inversions or methylation..., or structural changes such as compression of a region, different distances between chromosomes/regions...&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="7340" LastEditDate="2020-06-20T13:36:55.597" LastActivityDate="2020-06-20T13:36:55.597" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="846" PostTypeId="2" ParentId="817" CreationDate="2017-06-21T08:42:53.453" Score="1" Body="&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt; I stand corrected. @Daniel Standage correctly pointed out that the below is only true for a single query, or for the sorting of hits within those found by a given query. Otherwise they're sorted in the order they were given.&lt;/p&gt;&#xA;&lt;p&gt;BLAST results are sorted in descending order by the &amp;quot;Bit score&amp;quot; (column 12). Sorting by E. value would not be quite the same.&lt;/p&gt;&#xA;&lt;p&gt;i.e. column 11 (E value) will be presented in ascending order (an E. value of 0 is as good as it gets), and as it gets higher, your hit is getting worse. Likewise, bitscores are highest at the top and decrease down the list. A hit with E. value = 0 should have a high score. How high exactly depends on the hit length, so it's common to see many E values with 0.0, but the scores differ. Here's a snippet of one of my blast results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;PVCcif_ATCC43949  PAU_06042014  100.000  24180  0     0   1      24180  2233012  2257191  0.0  43606&#xA;PVCcif_ATCC43949  PAU_06042014  78.098   5593   1164  16  2765   8323   3219956  3225521  0.0  4547&#xA;PVCcif_ATCC43949  PAU_06042014  77.549   5394   1139  16  2948   8323   2376163  2370824  0.0  4260&#xA;PVCcif_ATCC43949  PAU_06042014  74.140   5669   1342  34  8490   14100  3225688  3231290  0.0  3573&#xA;PVCcif_ATCC43949  PAU_06042014  76.835   3665   825   9   16507  20150  3234191  3237852  0.0  2764&#xA;PVCcif_ATCC43949  PAU_06042014  76.871   2646   571   11  16625  19248  2362597  2359971  0.0  2001&#xA;PVCcif_ATCC43949  PAU_06042014  70.733   4131   1005  51  9047   13063  2368869  2364829  0.0  1954&#xA;PVCcif_ATCC43949  PAU_06042014  75.040   1891   448   7   8500   10383  2370771  2368898  0.0  1272&#xA;PVCcif_ATCC43949  PAU_06042014  78.986   1361   269   7   22766  24119  890051   891401   0.0  1150&#xA;PVCcif_ATCC43949  PAU_06042014  77.819   1082   217   5   536    1610   3217983  3219048  0.0  868&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see, they all have E values of 0.0, but they differ quite a lot in score, this is because of the length of the alignment, and the different percentage IDs.&lt;/p&gt;&#xA;" OwnerUserId="929" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-06-21T18:01:18.207" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="847" PostTypeId="1" AcceptedAnswerId="852" CreationDate="2017-06-21T08:45:44.693" Score="3" ViewCount="615" Body="&lt;p&gt;We believe that if after running blast, the global identity between a resulting sequence and our query is at least 30%, we can say that two sequences are homologs. what is the difference between local identity and global identity and how can we calculate them?&#xA;the following file is the global alignment result of one of my PSI-BLAST hits against query:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Aligned_sequences: 2&#xA;# 1: NP_418577.1&#xA;# 2: WP_036312822.1&#xA;# Matrix: EBLOSUM62&#xA;# Gap_penalty: 10.0&#xA;# Extend_penalty: 0.5&#xA;#&#xA;# Length: 264&#xA;# Identity:      85/264 (32.2%)&#xA;# Similarity:   125/264 (47.3%)&#xA;# Gaps:          29/264 (11.0%)&#xA;# Score: 376.5&#xA;# &#xA;#&#xA;#=======================================&#xA;&#xA;NP_418577.1        1 -----------MAEMKNLKIE-------VVRYNPEVDTAPHSAFYEVPYD     32&#xA;                                ..|..:..|:       :.|::||||..|....|:|...&#xA;WP_036312822.      1 MTIVDSGAPADTQEANDSGIQSYLVTFIIRRFDPEVDAEPRWVDYDVEMY     50&#xA;&#xA;NP_418577.1       33 ATTSLLDALGYIKDNLAPDLSYRWSCRMAICGSCGMMVNNVPKLACKTFL     82&#xA;                     .|..:||||..||.::...||:|.||...||||..|.:|...:|||||.:&#xA;WP_036312822.     51 PTDRVLDALHRIKWDVDGTLSFRRSCAHGICGSDAMRINGRNRLACKTLI    100&#xA;&#xA;NP_418577.1       83 R--DYTDGMKVEALANFPIERDLVVDMTHFIESLEAIKPYIIGNSRTADQ    130&#xA;                     :  |.:..:.|||:...|:|:||:|||..|.||...::|::...|.....&#xA;WP_036312822.    101 KDLDISKPIYVEAIKGLPLEKDLIVDMDPFFESFRDVQPFLQPKSAPEPG    150&#xA;&#xA;NP_418577.1      131 GTNIQTPAQMAKYHQFSGCINCGLCYAACPQFGLNPEFIGPAAITLAHRY    180&#xA;                     ....|:....|.|...:.||.|..|.::||.|..:.::.|||||..|||:&#xA;WP_036312822.    151 KERFQSIKDRAVYDDTTKCILCAACTSSCPVFWTDGQYFGPAAIVNAHRF    200&#xA;&#xA;NP_418577.1      181 NEDSRDHGKKERMAQLNSQNGVWSCTFVGYCSEVCPKHVDPAAAIQQGKV    230&#xA;                     ..||||.....|:..||.:.|||.|.....|:|.||:.::...||.:.|.&#xA;WP_036312822.    201 IFDSRDDAADVRLDILNDKEGVWRCRTTFNCTEACPRGIEITKAIAEVKQ    250&#xA;&#xA;NP_418577.1      231 ESSKDFLIATLKPR    244&#xA;                     ...:.         &#xA;WP_036312822.    251 AVLRG---------    255&#xA;&#xA;&#xA;#---------------------------------------&#xA;#---------------------------------------&#xA;the following file is the local alignment of that hit against query:&#xA;Aligned_sequences: 2&#xA;# 1: NP_418577.1&#xA;# 2: WP_036312822.1&#xA;# Matrix: EBLOSUM62&#xA;# Gap_penalty: 10.0&#xA;# Extend_penalty: 0.5&#xA;#&#xA;# Length: 219&#xA;# Identity:      83/219 (37.9%)&#xA;# Similarity:   119/219 (54.3%)&#xA;# Gaps:           2/219 ( 0.9%)&#xA;# Score: 390.5&#xA;# &#xA;#&#xA;#=======================================&#xA;&#xA;NP_418577.1       13 RYNPEVDTAPHSAFYEVPYDATTSLLDALGYIKDNLAPDLSYRWSCRMAI     62&#xA;                     |::||||..|....|:|....|..:||||..||.::...||:|.||...|&#xA;WP_036312822.     31 RFDPEVDAEPRWVDYDVEMYPTDRVLDALHRIKWDVDGTLSFRRSCAHGI     80&#xA;&#xA;NP_418577.1       63 CGSCGMMVNNVPKLACKTFLR--DYTDGMKVEALANFPIERDLVVDMTHF    110&#xA;                     |||..|.:|...:|||||.::  |.:..:.|||:...|:|:||:|||..|&#xA;WP_036312822.     81 CGSDAMRINGRNRLACKTLIKDLDISKPIYVEAIKGLPLEKDLIVDMDPF    130&#xA;&#xA;NP_418577.1      111 IESLEAIKPYIIGNSRTADQGTNIQTPAQMAKYHQFSGCINCGLCYAACP    160&#xA;                     .||...::|::...|.........|:....|.|...:.||.|..|.::||&#xA;WP_036312822.    131 FESFRDVQPFLQPKSAPEPGKERFQSIKDRAVYDDTTKCILCAACTSSCP    180&#xA;&#xA;NP_418577.1      161 QFGLNPEFIGPAAITLAHRYNEDSRDHGKKERMAQLNSQNGVWSCTFVGY    210&#xA;                     .|..:.::.|||||..|||:..||||.....|:..||.:.|||.|.....&#xA;WP_036312822.    181 VFWTDGQYFGPAAIVNAHRFIFDSRDDAADVRLDILNDKEGVWRCRTTFN    230&#xA;&#xA;NP_418577.1      211 CSEVCPKHVDPAAAIQQGK    229&#xA;                     |:|.||:.::...||.:.|&#xA;WP_036312822.    231 CTEACPRGIEITKAIAEVK    249&#xA;&#xA;&#xA;#---------------------------------------&#xA;#---------------------------------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="818" LastEditorUserId="298" LastEditDate="2017-06-21T09:58:00.247" LastActivityDate="2017-06-21T09:58:00.247" Title="what is the difference between local identity and global identity for homolog finding?" Tags="|sequence-homology|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="848" PostTypeId="2" ParentId="847" CreationDate="2017-06-21T09:15:19.583" Score="1" Body="&lt;p&gt;Defining homology isn't as simple as an identity cutoff, however, you're right that homology seems likely given a certain identity. This is where e-value comes into play. See:&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Expected_value&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Expected_value&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer the question, global-global requires that the alignment account for every base in the query and the target sequence. local-global requires a the full length of the query on a part of the target. local-local allows only the highest scoring part of the alignment between the query and target to be reported. i.e. one functional domain against another. these hits are often called HSPs.&lt;/p&gt;&#xA;" OwnerUserId="931" LastActivityDate="2017-06-21T09:15:19.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="849" PostTypeId="1" CreationDate="2017-06-21T09:18:16.357" Score="4" ViewCount="111" Body="&lt;p&gt;For example, see this gene (nad1) in ENA:&#xA;&lt;a href=&quot;http://www.ebi.ac.uk/ena/data/view/ABI60879&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ebi.ac.uk/ena/data/view/ABI60879&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at the XML for that gene you see the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;join(&#xA;             DQ984518.1: 324706 .. 325091 ,&#xA;  complement(DQ984518.1:  24417 ..  24498),&#xA;  complement(DQ984518.1:  22828 ..  23019),&#xA;             DQ984518.1:   3484 ..   3542 ,&#xA;  complement(DQ984518.1: 153702 .. 153960)&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which shows 5 exons joined out of phase and out of order. Is there a valid GTF representation of this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to dump a 'non-canonically spliced' gene into GTF? i.e. what's the recommendation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cross-posted on &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/849/how-to-represent-trans-spliced-genes-in-gtf/855#855&quot;&gt;biostars&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="931" LastEditorUserId="96" LastEditDate="2017-06-21T19:46:55.577" LastActivityDate="2017-06-21T19:46:55.577" Title="How to represent trans-spliced genes in GTF?" Tags="|file-formats|gtf|rna-splicing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="850" PostTypeId="2" ParentId="844" CreationDate="2017-06-21T09:21:15.913" Score="2" Body="&lt;p&gt;See a project like Plant reactome, where they infer the pathways of all sequenced plant genomes using orthology to well annotated species.&#xA;&lt;a href=&quot;http://plantreactome.gramene.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://plantreactome.gramene.org/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="931" LastActivityDate="2017-06-21T09:21:15.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="851" PostTypeId="2" ParentId="842" CreationDate="2017-06-21T09:26:47.787" Score="1" Body="&lt;p&gt;I gather that you want to use the background SNP frequency as a prior for input to your SNP calling algorithm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure of a canned algorithm for doing this, but a quick google shows up some promising links:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3593722/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3593722/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/article.php?id=4723&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://software.broadinstitute.org/gatk/documentation/article.php?id=4723&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-015-0489-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-015-0489-0&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="931" LastEditorUserId="292" LastEditDate="2017-06-21T14:00:21.773" LastActivityDate="2017-06-21T14:00:21.773" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="852" PostTypeId="2" ParentId="847" CreationDate="2017-06-21T09:41:44.030" Score="3" Body="&lt;p&gt;If you want a purely alignment based approach to begin to &lt;em&gt;infer&lt;/em&gt; homology (note I say &lt;em&gt;infer&lt;/em&gt;, as we stated in the comments, this isn't sufficient to say 2 sequences are homologs), I would use a Multiple Sequence Alignment (MSA) instead of a pairwise like you've done so far, and include as many similar proteins as you can from BLAST.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The choice of algorithm for the MSA probably won't matter too much, as you will be looking for conserved regions anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have your MSA, you can easily spot conserved domains. If anything is known about this protein structurally, it's quite trivial to work out whether a conserved region is a functional domain (active site, that kind of thing).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this information, you would be somewhat justified in calling these proteins structural homologs at least if they share domains, but, to reiterate, without doing phylogenetic analyses, you &lt;em&gt;cannot&lt;/em&gt; call them true homologs.&lt;/p&gt;&#xA;" OwnerUserId="929" LastActivityDate="2017-06-21T09:41:44.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="853" PostTypeId="1" AcceptedAnswerId="862" CreationDate="2017-06-21T10:17:57.267" Score="4" ViewCount="615" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/258878&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioStars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I look at this record in GenBank I see about 6k genes:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/CM000760?report=gbwithparts&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/CM000760?report=gbwithparts&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd really like to be able to dump those genes in GFF3 format, but I'm guessing I can't do that in a single step, so...&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How to dump all genes from a GenBank record in 'whatever' format?&lt;/li&gt;&#xA;&lt;li&gt;How to convert 'whatever' format to GFF3?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="931" LastEditorUserId="73" LastEditDate="2018-01-11T04:49:40.997" LastActivityDate="2018-01-11T04:49:40.997" Title="How to dump genes from GenBank in GFF3 format?" Tags="|gff3|data-download|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="854" PostTypeId="2" ParentId="829" CreationDate="2017-06-21T10:27:10.710" Score="3" Body="&lt;p&gt;I found &lt;a href=&quot;http://www.disgenet.org/web/DisGeNET/menu&quot; rel=&quot;nofollow noreferrer&quot;&gt;DisGeNET&lt;/a&gt; useful for my purpose, a database that associates genes with diseases and, if known, gene variants with diseases. It integrates several sources of information for computing a score of association of gene/variation with the disease ( &lt;a href=&quot;https://academic.oup.com/nar/article/45/D1/D833/2290909&quot; rel=&quot;nofollow noreferrer&quot;&gt;Piñero et al., 2016&lt;/a&gt;, &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pubmed/25877637&quot; rel=&quot;nofollow noreferrer&quot;&gt;Piñero et al., 2015&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Gene Disease Associations&lt;/strong&gt; from UniProt, PsyGeNET, ClinVar, Orphanet, the GWAS Catalog, CTD (human data), and Human Phenotype Ontology, RGD, MGD, and CTD (mouse and rat data) GAD, LHGDN and BeFree&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Variants Disease Associations&lt;/strong&gt; from ClinVar, the GWAS Catalog, Uniprot, GAD, and from BEFREE&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If available, it provides the Pubmed ID of articles supporting the proposed association and the rs id from the dbSNP for variants associated to disease.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition it provides an experimental R package, &lt;a href=&quot;https://bitbucket.org/ibi_group/disgenet2r&quot; rel=&quot;nofollow noreferrer&quot;&gt;disgenet2r&lt;/a&gt;, that allows to query the database in a programmatic manner and that provides some plot functions to generate representations of the associations founded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this approach, I have been able to found several human histone-disease associations (not my original purpose), together with some concrete histone residues that are mutated in specific diseases. &lt;/p&gt;&#xA;" OwnerUserId="678" LastEditorUserId="678" LastEditDate="2017-06-21T12:40:37.340" LastActivityDate="2017-06-21T12:40:37.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="855" PostTypeId="2" ParentId="849" CreationDate="2017-06-21T10:33:07.927" Score="3" Body="&lt;p&gt;To my knowledge there's no defined way to deal with that in GTF. &lt;a href=&quot;https://github.com/The-Sequence-Ontology/Specifications/blob/master/gff3.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;GFF3 handles trans-splicing&lt;/a&gt; (you'll have to scroll down to &quot;trans-spliced transcript&quot;) by giving an individual transcript multiple parents (e.g., &lt;code&gt;ID=some_transspliced_gene;Parent=gene1,gene2&lt;/code&gt;). You could use the same methodology with GTF files, but just note that it'll break most downstream programs.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-06-21T10:33:07.927" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="856" PostTypeId="1" AcceptedAnswerId="858" CreationDate="2017-06-21T10:57:29.787" Score="4" ViewCount="682" Body="&lt;p&gt;I have a VCF file with SNPs from a bacterial genome and want to find if the SNPs are located inside genes, is there some CLI-tool where you can pass a VCF file and a gff or gbk file and it returns the name of the genes?&lt;/p&gt;&#xA;" OwnerUserId="223" LastEditorUserId="223" LastEditDate="2017-06-21T11:45:19.827" LastActivityDate="2017-06-21T11:45:19.827" Title="Find gene at position from gff or gbk file" Tags="|snp|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="857" PostTypeId="1" CreationDate="2017-06-21T11:08:11.633" Score="4" ViewCount="161" Body="&lt;p&gt;I have a gene expression dataset that I want to investigate. Particularly, I would like to understand whether there is any correlation between each gene's expression and some quantitative or qualtitative data (say, correlation between gene 'XPTO' , body mass index, and race). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possible way to test this would be through logistic regression, but is this a good approach or are there caveats that I should know about using such a statistic?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is the following: which methods would you advise to measure such correlations, and why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(this post was &lt;a href=&quot;https://www.biostars.org/p/258889/&quot; rel=&quot;noreferrer&quot;&gt;crossposted on Biostars&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="499" LastActivityDate="2017-06-21T11:44:01.653" Title="Correlating gene expression with qualitative variables" Tags="|rna-seq|r|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="858" PostTypeId="2" ParentId="856" CreationDate="2017-06-21T11:10:55.837" Score="3" Body="&lt;p&gt;Via &lt;a href=&quot;https://github.com/bedops/bedops&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ gff2bed &amp;lt; annotations.gff &amp;gt; annotations.bed&#xA;$ vcf2bed &amp;lt; snps.vcf &amp;gt; snps.bed&#xA;$ bedmap --echo --echo-map-id-uniq snps.bed annotations.bed &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This can be reduced to a one-liner if you're using &lt;code&gt;bash&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --echo --echo-map-id-uniq &amp;lt;(vcf2bed &amp;lt; snps.vcf) &amp;lt;(gff2bed &amp;lt; annotations.gff) &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="776" LastActivityDate="2017-06-21T11:10:55.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="859" PostTypeId="2" ParentId="857" CreationDate="2017-06-21T11:44:01.653" Score="3" Body="&lt;p&gt;Logistic regression would generally be a bad choice for non-binary outcomes. In such cases, linear regression (or a GLM more generally) still works fine. You can already do that in the standard R packages for RNAseq (DESeq2, edgeR, and limma), where the fold-change is in whatever units you're measuring your quantitative trait in.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-06-21T11:44:01.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="860" PostTypeId="1" AcceptedAnswerId="865" CreationDate="2017-06-21T12:19:12.830" Score="5" ViewCount="1543" Body="&lt;p&gt;We can calculate the distance between residues in a PDB file regarding different parameters like closest atoms, alpha carbon, beta carbon, centroid and etc. Which one of these parameters are better to show physical interaction between residues in a PDB file?&lt;/p&gt;&#xA;" OwnerUserId="818" LastActivityDate="2017-10-11T16:15:21.360" Title="Best distance parameter for estimating physical interaction between residues in a PDB file" Tags="|protein-structure|pdb|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="861" PostTypeId="2" ParentId="807" CreationDate="2017-06-21T12:36:08.063" Score="7" Body="&lt;p&gt;As &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/492/wkretzsch&quot;&gt;wkretzsch&lt;/a&gt; suggested this was worthy of an actual answer, I feel the obvious solution is missing here; index the &lt;code&gt;FASTQ&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Index it&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;As much as I typically hesitate to jump to a solution that requires a script or framework (as opposed to just unix command line tools), there is sadly no &lt;code&gt;samtools fqidx&lt;/code&gt; (perhaps there should be), and existing answers suggest a lot of munging. Whilst they probably work, some appear cumbersome and have many steps in which you could make a mistake.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, to keep things simple - a quick and dirty alternative approach might be to use &lt;code&gt;biopython&lt;/code&gt;, seeing as it &lt;a href=&quot;http://biopython.org/DIST/docs/tutorial/Tutorial.html#htoc291&quot; rel=&quot;nofollow noreferrer&quot;&gt;already has this functionality implemented&lt;/a&gt; to do this, and if installed, is trivial to use:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;from Bio import SeqIO&#xA;fq = SeqIO.index(&quot;myfastq.fq&quot;, &quot;fastq&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Once you've acquired an index, you'll get fast random access for any read:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;# Random access by read name (I like owls)&#xA;record = fq[&quot;HOOT&quot;]&#xA;record&#xA;#&amp;gt; SeqRecord(seq=Seq('ACGTACGT', SingleLetterAlphabet()), id='HOOT', name='HOOT', description='HOOT', dbxrefs=[])&#xA;&#xA;# We can get the sequence&#xA;record.seq&#xA;#&amp;gt; Seq('ACGTACGT', SingleLetterAlphabet())&#xA;&#xA;# and qualities&#xA;record.letter_annotations&#xA;#&amp;gt; {'phred_quality': [41, 41, 41, 41, 41, 41, 41, 41]}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want to select arbitrary random records, you could use something like &lt;a href=&quot;https://docs.python.org/3.1/library/random.html#random.randrange&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;randrange&lt;/code&gt;&lt;/a&gt; to select between 0 and the length of the references list.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;from random import randint&#xA;&#xA;# Coerce keys to a list, as `dictionary-keyiterator` has no&#xA;#   __getitem__ attribute that would allow integer indices&#xA;# Note also that this doesn't necessarily guarantee a sorted order&#xA;#  but I guess that doesn't matter if you just want random records&#xA;key_list = list(fq.keys())&#xA;&#xA;# Select a random key&#xA;# Note we use len(key_list)-1 as randint endpoints are inclusive&#xA;random_readname = key_list[ randint(0, len(key_list)-1) ]&#xA;&#xA;# Get your record&#xA;rand_record = fq.get( random_readname )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want multiple records, you'll probably want &lt;a href=&quot;https://docs.python.org/2/library/random.html#random.sample&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;sample&lt;/code&gt;&lt;/a&gt; (to avoid replacement) instead:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;from random import sample&#xA;N = 100    &#xA;random_indices = sample(xrange(len(key_list)), N)&#xA;&#xA;for key_i in random_indices:&#xA;    random_readname = key_list[ key_i ]&#xA;    rand_record = fq.get( random_readname )&#xA;    # ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For what it's worth, I think &lt;code&gt;biopython&lt;/code&gt; holds this index in RAM, so if your file is absolutely massive, you might need to be more clever. If that's the case, you could iterate through the &lt;code&gt;FASTQ&lt;/code&gt; once, and output the readname, file offset and length - akin to a &lt;code&gt;FASTA&lt;/code&gt; &lt;code&gt;fai&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&#xA;See &lt;code&gt;samtools fqidx&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="215" LastEditorUserId="215" LastEditDate="2018-08-30T11:10:07.147" LastActivityDate="2018-08-30T11:10:07.147" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="862" PostTypeId="2" ParentId="853" CreationDate="2017-06-21T13:27:37.420" Score="3" Body="&lt;p&gt;Turns out you can just grab the GFF3 from the NCBI's FTP site!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/003/195/GCA_000003195.3_Sorghum_bicolor_NCBIv3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/003/195/GCA_000003195.3_Sorghum_bicolor_NCBIv3/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genome/doc/ftpfaq/#files&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/genome/doc/ftpfaq/#files&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Or &quot;Access the data&quot; on the right here:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly/GCF_000003195.3&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/assembly/GCF_000003195.3&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="931" LastActivityDate="2017-06-21T13:27:37.420" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="863" PostTypeId="2" ParentId="677" CreationDate="2017-06-21T13:36:08.693" Score="5" Body="&lt;p&gt;Tried looking for an explicit database? i.e.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ComSin: database of protein structures in bound (complex) and unbound (single) states in relation to their intrinsic disorder: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2808974/&quot; rel=&quot;noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2808974/&lt;/a&gt; or&lt;/li&gt;&#xA;&lt;li&gt;LigASite: a database of biologically relevant binding sites in proteins with known apo-structures &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/17933762&quot; rel=&quot;noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pubmed/17933762&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Else the PDB itself has a very powerful advanced search that should allow you to do this: &lt;a href=&quot;http://www.rcsb.org/pdb/search/advSearch.do&quot; rel=&quot;noreferrer&quot;&gt;http://www.rcsb.org/pdb/search/advSearch.do&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="931" LastActivityDate="2017-06-21T13:36:08.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="864" PostTypeId="2" ParentId="860" CreationDate="2017-06-21T13:38:50.203" Score="1" Body="&lt;p&gt;You should read my &lt;a href=&quot;https://bmcstructbiol.biomedcentral.com/articles/10.1186/1472-6807-8-53&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, where I did a comprehensive parameter sweep across different thresholds and definitions and benchmarked the resulting empirical potentials on a decoy ranking challenge:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The relevant phrase from the abstract is &quot;compared using 90 different definitions of residue-residue contact&quot;. However, it boils down (as always) to the question, &quot;what do you want to do&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cite me if you use my paper for your research. &lt;/p&gt;&#xA;" OwnerUserId="931" LastEditorUserId="48" LastEditDate="2017-06-22T08:56:15.450" LastActivityDate="2017-06-22T08:56:15.450" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="865" PostTypeId="2" ParentId="860" CreationDate="2017-06-21T13:48:17.480" Score="2" Body="&lt;p&gt;Use a distance cutoff of 12Å between Cᵦ atoms.&lt;/p&gt;&#xA;" OwnerUserId="931" LastActivityDate="2017-06-21T13:48:17.480" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="867" PostTypeId="2" ParentId="81" CreationDate="2017-06-21T14:46:48.633" Score="2" Body="&lt;p&gt;Simulating NGS reads while controlling sequence coverage is now easy with &lt;a href=&quot;http://karel-brinda.github.io/rnftools/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNFtools&lt;/a&gt; (from version 0.3.1). See the &lt;a href=&quot;http://rnftools.readthedocs.io/en/latest/tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt;, especially section &lt;a href=&quot;http://rnftools.readthedocs.io/en/latest/tutorial/02_simulation.html#sequence-extraction&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sequence extraction&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Environment preparation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, install &lt;a href=&quot;https://bioconda.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioConda and add the required channels&lt;/a&gt;. Then either install RNFtools in the default Conda environment&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;conda install rnftools&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or create and activate a separate Conda environment (preferable)&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;conda create -n rnftools rnftools&#xA;source activate rnftools&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Simulation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that you have a reference file &lt;code&gt;ref.fa&lt;/code&gt; and a tab-separated coverage file &lt;code&gt;coverage.tsv&lt;/code&gt; (e.g., those from your example). Then the following RNFtools &lt;code&gt;Snakefile&lt;/code&gt; will do the job you want:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import rnftools&#xA;import csv&#xA;&#xA;rnftools.mishmash.sample(&quot;simulation_with_coverage_control&quot;, reads_in_tuple=1)&#xA;&#xA;fa = &quot;ref.fa&quot;&#xA;tsv = &quot;coverage.tsv&quot;&#xA;&#xA;with open(tsv) as f:&#xA;    table = csv.reader(f, delimiter='\t')&#xA;    for seqname, cov in table:&#xA;&#xA;        rnftools.mishmash.DwgSim(&#xA;            fasta=fa,&#xA;            sequences=[seqname],&#xA;            coverage=float(cov),&#xA;            read_length_1=10, # quick test with supershort reads&#xA;            read_length_2=0,&#xA;        )&#xA;&#xA;include: rnftools.include()&#xA;rule: input: rnftools.input()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When you save this file (&lt;code&gt;Snakefile&lt;/code&gt;) and run &lt;code&gt;snakemake&lt;/code&gt;, RNFtools will simulate reads using DWGsim with the coverages defined your text file, and save all the simulated reads in &lt;code&gt;simulation_with_coverage_control.fq&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can play with all the parameters. In particular, you can use a different simulator (e.g., Art-Illumina using &lt;code&gt;rnftools.mishmash.ArtIllumina&lt;/code&gt;). See the &lt;a href=&quot;http://rnftools.rtfd.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNFtools documentation&lt;/a&gt; for more information.&lt;/p&gt;&#xA;" OwnerUserId="425" LastEditorUserId="425" LastEditDate="2017-06-21T15:32:54.330" LastActivityDate="2017-06-21T15:32:54.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="868" PostTypeId="2" ParentId="838" CreationDate="2017-06-21T15:27:07.430" Score="3" Body="&lt;h3&gt;Regarding your python code&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If you want the &lt;strong&gt;experimental ranges that are entirely contained in one of the reference ranges&lt;/strong&gt;, you need to have the coordinates in the following order:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cst &amp;lt;= tst &amp;lt; ten &amp;lt;= cen&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If what you want are the &lt;strong&gt;experimental ranges that overlap one of the reference ranges&lt;/strong&gt;, you need to have either the start or the end of the experimental range fall within the reference range:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(cst &amp;lt;= tst &amp;lt; cen) or (cst &amp;lt; ten &amp;lt;= cen)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Your code is equivalent to neither possibilities:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cst &amp;gt;= tst and cen &amp;lt;= ten&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is equivalent to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tst &amp;lt;= cst and cen &amp;lt;= ten&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Or &lt;code&gt;tst &amp;lt;= cst &amp;lt; cen &amp;lt;= ten&lt;/code&gt;, since we know that &lt;code&gt;cst &amp;lt; cen&lt;/code&gt;, by definition of a bed interval).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this rewriting, we can more easily see that you are actually selecting the &lt;strong&gt;experimental ranges that contain a reference range&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is some (python3) code that gives you the results in the other two situations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;ref_intervals = []&#xA;with open(&quot;s4.txt&quot;, &quot;r&quot;) as f:&#xA;    for line in f:&#xA;        (chr, start, end) = line.strip().split(&quot;\t&quot;)&#xA;        ref_intervals.append((chr, int(start), int(end)))&#xA;&#xA;exp_intervals = []&#xA;with open(&quot;s3.txt&quot;, &quot;r&quot;) as f:&#xA;    for line in f:&#xA;        (chr, start, end) = line.strip().split(&quot;\t&quot;)&#xA;        exp_intervals.append((chr, int(start), int(end)))&#xA;&#xA;contained = []&#xA;overlapping = []&#xA;for (r_chr, r_start, r_end) in ref_intervals:&#xA;    for (e_chr, e_start, e_end) in exp_intervals:&#xA;        if e_chr == r_chr:&#xA;            if r_start &amp;lt;= e_start &amp;lt; r_end or r_start &amp;lt; e_end &amp;lt;= r_end:&#xA;                overlapping.append((e_chr, e_start, e_end))&#xA;            if r_start &amp;lt;= e_start &amp;lt; e_end &amp;lt;= r_end:&#xA;                contained.append((e_chr, e_start, e_end))&#xA;&#xA;print(&quot;overlapping&quot;)&#xA;for (chr, start, end) in overlapping:&#xA;    print(chr, start, end, sep=&quot;\t&quot;)&#xA;&#xA;print(&quot;contained&quot;)&#xA;for (chr, start, end) in contained:&#xA;    print(chr, start, end, sep=&quot;\t&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I run it, I obtain the following results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ./overlap.py &#xA;overlapping&#xA;1   10  20&#xA;1   5   20&#xA;1   10  50&#xA;1   14  17&#xA;2   20  30&#xA;2   25  30&#xA;2   20  60&#xA;contained&#xA;1   10  20&#xA;1   14  17&#xA;2   20  30&#xA;2   25  30&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is probably an good exercise to program this, but as the other answers point out, &lt;strong&gt;there are efficient tools that would be a better solution in a professional setting&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2017-06-21T15:34:15.073" LastActivityDate="2017-06-21T15:34:15.073" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="869" PostTypeId="1" AcceptedAnswerId="870" CreationDate="2017-06-21T16:30:49.260" Score="3" ViewCount="842" Body="&lt;p&gt;I am analyzing data from a quantitative polymerase chain reaction (qPCR) using R. After cleaning the raw data, it looks something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dput(x)&#xA;structure(list(Reporter = c(&quot;FAM&quot;, &quot;FAM&quot;, &quot;FAM&quot;, &quot;FAM&quot;, &quot;FAM&quot;, &#xA;&quot;FAM&quot;, &quot;FAM&quot;, &quot;FAM&quot;, &quot;FAM&quot;, &quot;FAM&quot;, &quot;FAM&quot;, &quot;FAM&quot;, &quot;VIC&quot;, &quot;VIC&quot;, &#xA;&quot;VIC&quot;, &quot;VIC&quot;, &quot;VIC&quot;, &quot;VIC&quot;, &quot;VIC&quot;, &quot;VIC&quot;, &quot;VIC&quot;, &quot;VIC&quot;, &quot;VIC&quot;, &#xA;&quot;VIC&quot;), Number = c(1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L, 11L, &#xA;12L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L, 11L, 12L), A = c(22.19, &#xA;22.24, 22.5, 22.54, 22.6, 22.59, 23.07, 23.46, 22.43, 22.74, &#xA;24.09, 23.91, 24.52, 25.03, 25.25, 25.82, 25.13, 24.71, 25.34, &#xA;25.85, 25.25, 26.15, 25.81, 25.29), B = c(21.72, 21.78, 22.86, &#xA;22.73, 19.88, 20.07, 21.06, 21.06, 20.96, 21.11, 19.46, 19.43, &#xA;24.75, 24.69, 25.64, 25.19, 23.76, 23.69, 24.35, 25.05, 24.1, &#xA;23.81, 22.81, 23.13), C = c(21.37, 21.56, 20.07, 20.01, 21.17, &#xA;21.08, 20.54, 20.36, 33, NA, NA, NA, 23.91, 24.31, 23.61, 23.88, &#xA;24.33, 24.31, 23.37, 23.53, 33, NA, NA, NA), E = c(26.26, 27.33, &#xA;25.93, 26.56, 25.76, 23.03, 24.72, 25.27, 24.43, 24.31, 26.98, &#xA;23.33, 24.04, 25.02, 25.1, 25.1, 24.68, 25.48, 25.87, 26.22, &#xA;25.35, 25.36, 25.11, 25.98), F = c(25.81, 26.9, 25.58, 26.61, &#xA;25.06, 21.85, 23.59, 24.04, 23.19, 23.19, 25.17, 20.8, 24.12, &#xA;24.26, 25.32, 25.25, 24, 23.78, 24.7, 24.48, 23.52, 23.87, 23.05, &#xA;23.05), G = c(26.12, 27.02, 24.08, 25.15, 25.99, 23.18, 24.2, &#xA;24.05, 33, NA, NA, NA, 23.47, 23.45, 23.7, 23.74, 24.46, 24.19, &#xA;23.56, 23.53, 33, NA, NA, NA)), .Names = c(&quot;Reporter&quot;, &quot;Number&quot;, &#xA;&quot;A&quot;, &quot;B&quot;, &quot;C&quot;, &quot;E&quot;, &quot;F&quot;, &quot;G&quot;), class = c(&quot;tbl_df&quot;, &quot;tbl&quot;, &quot;data.frame&quot;&#xA;), row.names = c(NA, 24L))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think that each well measures two genes: target which is 12 different genes (color FAM), and the reference or housekeeping gene, GAPDH (color VIC). Also, control is triplicate A, B, C (3 x 12 wells), and treatment is E, F, G (3 x 12 wells). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have created a function &lt;code&gt;ddCt_&lt;/code&gt; for analyzing the data with the ddCt algorithm (Livak &amp;amp; Schmittgen, 2001). It takes one argument &lt;code&gt;x&lt;/code&gt; which is a data.frame of the form exemplified above. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ddCt_ &amp;lt;- function(x) {&#xA;  # Subset x by control/treatment and target/reference&#xA;  # Then, calculate Ct averages for each triplicate&#xA;  TC &amp;lt;- x %&amp;gt;% &#xA;    select(Reporter, Number, A, B, C) %&amp;gt;% &#xA;    filter(Reporter == &quot;FAM&quot;) %&amp;gt;% &#xA;    rowwise() %&amp;gt;%&#xA;    mutate(Ct_Avg = mean(c(A, B, C), na.rm = TRUE))&#xA;&#xA;  RC &amp;lt;- x %&amp;gt;% &#xA;    select(Reporter, Number, A, B, C) %&amp;gt;% &#xA;    filter(Reporter == &quot;VIC&quot;) %&amp;gt;% &#xA;    rowwise() %&amp;gt;%&#xA;    mutate(Ct_Avg = mean(c(A, B, C), na.rm = TRUE))&#xA;&#xA;  TT &amp;lt;- x %&amp;gt;% &#xA;    select(Reporter, Number, E, F, G) %&amp;gt;% &#xA;    filter(Reporter == &quot;FAM&quot;) %&amp;gt;% &#xA;    rowwise() %&amp;gt;%&#xA;    mutate(Ct_Avg = mean(c(E, F, G), na.rm = TRUE))&#xA;&#xA;  RT &amp;lt;- x %&amp;gt;% &#xA;    select(Reporter, Number, E, F, G) %&amp;gt;% &#xA;    filter(Reporter == &quot;VIC&quot;) %&amp;gt;% &#xA;    rowwise() %&amp;gt;%&#xA;    mutate(Ct_Avg = mean(c(E, F, G), na.rm = TRUE))&#xA;&#xA;  # Normalize Ct of the target gene to the Ct of the reference gene&#xA;  dCt_control   &amp;lt;- TC$Ct_Avg - RC$Ct_Avg&#xA;  dCt_treatment &amp;lt;- TT$Ct_Avg - RT$Ct_Avg&#xA;&#xA;  # Normalize dCt of the treatment group to the dCt of the control group&#xA;  ddCt &amp;lt;- dCt_treatment - dCt_control&#xA;&#xA;  # Calculate fold change&#xA;  fc &amp;lt;- 2^(-ddCt)&#xA;&#xA;  # Calculate avg and sd&#xA;  dCt_control_avg   &amp;lt;- mean(dCt_control)&#xA;  dCt_control_sd    &amp;lt;- sd(dCt_control)&#xA;  dCt_treatment_avg &amp;lt;- mean(dCt_treatment)&#xA;  dCt_treatment_sd  &amp;lt;- sd(dCt_treatment)&#xA;&#xA;  # Create output&#xA;  df &amp;lt;- data_frame(&#xA;    Sample = 1:12, &quot;dCt Control&quot; = dCt_control, &quot;dCt Treatment&quot; = dCt_treatment, &#xA;    &quot;ddCt&quot; = ddCt, &quot;Fold Change&quot; = fc, &quot;dCt Control Avg&quot; = dCt_control_avg, &#xA;    &quot;dCt Control SD&quot; = dCt_control_sd, &quot;dCt Treatment Avg&quot; = dCt_treatment_avg, &#xA;    &quot;dCt Treatment SD&quot; = dCt_treatment_sd&#xA;    ) %&amp;gt;% round(2)&#xA;  write.csv(x = df, file = &quot;result.csv&quot;)&#xA;  saveRDS(object = df, file = &quot;result.rds&quot;)&#xA;  df&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I am not a biochemist or molecular biologist, so I am unsure of the basic concepts involved and the overall approach. So, my questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is the function correct?&lt;/li&gt;&#xA;&lt;li&gt;Why is it important to calculate fold change and standard deviation after normalization?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="935" LastEditorUserId="935" LastEditDate="2017-06-21T21:05:36.370" LastActivityDate="2017-06-21T21:05:36.370" Title="qPCR: Why is fold change and standard deviation calculated after transformation?" Tags="|r|normalization|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="870" PostTypeId="2" ParentId="869" CreationDate="2017-06-21T16:43:27.950" Score="4" Body="&lt;ol&gt;&#xA;&lt;li&gt;The functions look correct, but calculate a few by hand and ensure they match. One thing I should note is that the subtraction of the Ct values usually happens before an average is made, since generally both are in the same well. In other word, subtract the reference Ct from the gene of interest Ct from the same well and then average the dCts. This order is important since one usually does a multiplexed qPCR.&lt;/li&gt;&#xA;&lt;li&gt;You have to normalize to a reference gene to control for how much cDNA was used, since that will alter the Ct values. If you calculated the fold-changes without normalization then they could be purely due to using more/less cDNA in the reaction (i.e., the output would be meaningless).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-06-21T19:37:17.810" LastActivityDate="2017-06-21T19:37:17.810" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="871" PostTypeId="2" ParentId="853" CreationDate="2017-06-21T18:31:20.883" Score="1" Body="&lt;p&gt;When accessing all of the annotated genes for a reference genome, downloading a GFF3 file directly from the &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/genomes/genbank&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genbank&lt;/a&gt; or &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/genomes/refseq&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefSeq&lt;/a&gt; FTP sites is definitely the way to go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when submitting a search query into NCBI Entrez, there is no easy way to download the results in GFF3 format. Probably the best route is to download the results in ASN.1 format, and then use &lt;code&gt;annotwriter&lt;/code&gt; from the NCBI C++ toolkit to convert to GFF3. But getting the latter to compile and run properly isn't trivial in my experience.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2017-06-21T18:31:20.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="872" PostTypeId="2" ParentId="844" CreationDate="2017-06-21T19:34:57.207" Score="2" Body="&lt;p&gt;As you alluded to in your question, KEGG does provide a curated set of enzyme and metabolite information. This information can be parsed and used to create a network that you can analyze to look at how gene products could be working together. Additional steps could include mapping transcriptome data to the map.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;http://biorxiv.org/content/early/2016/12/07/092304&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; as an example of how this can be done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this helps. :)&lt;/p&gt;&#xA;" OwnerUserId="824" LastActivityDate="2017-06-21T19:34:57.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="873" PostTypeId="2" ParentId="802" CreationDate="2017-06-22T09:17:53.090" Score="1" Body="&lt;p&gt;Running &lt;code&gt;blastdbcmd -db foo -info&lt;/code&gt; provides a little information but I haven't seen anything which will report exactly how a blastdb was created.&lt;/p&gt;&#xA;&lt;p&gt;A &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/blast/documents/blastdb.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;blastdb readme&lt;/a&gt; suggests that only the &lt;code&gt;parse_seqids&lt;/code&gt; option has been added to the standard parameters.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For those from NCBI, the following makeblastdb commands are recommended:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For nucleotide fasta file:   makeblastdb -in input_db -dbtype nucl -parse_seqids&lt;/li&gt;&#xA;&lt;li&gt;For protein fasta file:      makeblastdb -in input_db -dbtype prot -parse_seqids&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The &lt;code&gt;-parse_seqids&lt;/code&gt; requires NCBI fasta headers in the correct format (spec can be found &lt;a href=&quot;http://ncbi.github.io/cxx-toolkit/pages/ch_demo#ch_demo.id1_fetch.html_ref_fasta&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="941" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-06-22T19:56:20.457" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="874" PostTypeId="1" AcceptedAnswerId="875" CreationDate="2017-06-22T10:17:23.743" Score="10" ViewCount="2577" Body="&lt;p&gt;I am trying to find intersection of two genomic ranges (gr1 and gr2) and keep metadata from one of them&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gr1&#xA;chrI [1, 100] * | 0.1&#xA;chrI [101, 200] * | 0.2&#xA;&#xA;gr2&#xA;chrI [50, 150] + | &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;intersect(gr1, gr2)&lt;/code&gt; will remove metadata, where&#xA;&lt;code&gt;subsetByOverlaps(gr1, gr2)&lt;/code&gt; return:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chrI [1, 100] * | 0.1&#xA;chrI [101, 200] * | 0.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where metadata is saved, but intersection is bigger than real (the same as in initial gr1 start/end)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I get an intersection range and keep metadata?&#xA;to get the range:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chrI [50, 100] * | 0.1&#xA;chrI [101, 150] * | 0.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="943" LastEditorUserId="131" LastEditDate="2017-06-22T12:25:41.493" LastActivityDate="2017-06-22T12:25:41.493" Title="Intersection of two genomic ranges to keep metadata" Tags="|r|bioconductor|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="875" PostTypeId="2" ParentId="874" CreationDate="2017-06-22T11:14:42.393" Score="4" Body="&lt;p&gt;Presuming you don't have cases of multiple ovelaps&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; library(GenomicRanges)&#xA;&amp;gt; gr1 = GRanges(c(&quot;chrI&quot;, &quot;chrI&quot;), IRanges(start=c(1, 101), width=c(100, 100)), mcols=data.frame(foo=c(0.1, 0.2)))&#xA;&amp;gt; gr2 = GRanges(c(&quot;chrI&quot;), IRanges(start=c(50), width=c(101)))&#xA;&amp;gt; o = findOverlaps(gr1, gr2)&#xA;&amp;gt; grl1 = split(gr1[queryHits(o)], 1:length(o)) # You can't mendoapply on a GRanges object&#xA;&amp;gt; grl2 = split(gr2[subjectHits(o)], 1:length(o))&#xA;&amp;gt; foo = function(x, y) {&#xA;+     rv = x&#xA;+     start(rv) = max(start(x), start(y))&#xA;+     end(rv) = min(end(x), end(y))&#xA;+     return(rv)&#xA;+ }&#xA;&amp;gt; unlist(mendoapply(foo, grl1, y=grl2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GRanges object with 2 ranges and 1 metadata column:&#xA;    seqnames     ranges strand | mcols.foo&#xA;       &amp;lt;Rle&amp;gt;  &amp;lt;IRanges&amp;gt;  &amp;lt;Rle&amp;gt; | &amp;lt;numeric&amp;gt;&#xA;  1     chrI [ 50, 100]      * |       0.1&#xA;  2     chrI [101, 150]      * |       0.2&#xA;  -------&#xA;  seqinfo: 1 sequence from an unspecified genome; no seqlengths&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It'd be more memory efficient to iterate over &lt;code&gt;o&lt;/code&gt;, but this gives you the idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In cases of multiple overlaps, one could use &lt;code&gt;intersect()&lt;/code&gt;, then &lt;code&gt;findOverlaps()&lt;/code&gt; with that and then continue on with the procedure demonstrated above (or iterate over the overlaps).&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-06-22T11:32:27.683" LastActivityDate="2017-06-22T11:32:27.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="877" PostTypeId="1" CreationDate="2017-06-22T18:05:28.307" Score="4" ViewCount="239" Body="&lt;p&gt;There are wet methods: &lt;a href=&quot;https://deainfo.nci.nih.gov/advisory/bsa/1016/3_PDX.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Patient-derived models&lt;/a&gt;: &lt;a href=&quot;https://en.wikipedia.org/wiki/Patient-derived_tumor_xenograft&quot; rel=&quot;nofollow noreferrer&quot;&gt;Patient Derived Xenograft (PDX)&lt;/a&gt; and &lt;a href=&quot;http://www.nature.com/ng/journal/v47/n4/fig_tab/ng.3225_SF8.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Patient Derived Organoids (PDO)&lt;/a&gt; to reflect tumor biology. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any databases or computational tools that use the outcomes from PDO/PDX experiments to create a predictive computational model for cancer or other diseases?&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2020-01-04T14:21:42.907" LastActivityDate="2020-01-04T14:21:42.907" Title="How are PDO and PDX used in computational and predicative models for tumour biology?" Tags="|cancer|drugs|modelling|" AnswerCount="1" CommentCount="15" ContentLicense="CC BY-SA 4.0" />
  <row Id="878" PostTypeId="1" AcceptedAnswerId="880" CreationDate="2017-06-22T18:57:54.120" Score="8" ViewCount="608" Body="&lt;p&gt;This is a tough one I think: is there a publicly available, up-to-date, free, complete database for antibiotics names and classes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am specifically looking for information like, e.g., &lt;code&gt;cefpirome (is a) cephalosporin&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've looked in a couple of different places, but nothing really comes close. E.g.:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://card.mcmaster.ca/&quot; rel=&quot;noreferrer&quot;&gt;CARD&lt;/a&gt; comes closest, but is not free (companies need to pay a license) and not complete/up-to-date (e.g., cefpirome is missing)&lt;/li&gt;&#xA;&lt;li&gt;@Karel mentioned &lt;a href=&quot;http://www.kegg.jp/&quot; rel=&quot;noreferrer&quot;&gt;KEGG&lt;/a&gt; in his answer, but this also needs a license (companies for sure, academics if they download data via FTP).&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.antibiotics.or.jp/journal/database/database-top.htm&quot; rel=&quot;noreferrer&quot;&gt;Novel Antiobiotics Database&lt;/a&gt; has no class ... and stops in 2003&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://antibiotics.toku-e.com/&quot; rel=&quot;noreferrer&quot;&gt;The Antimicrobial Index&lt;/a&gt; is web only, and terms of use apparently prohibit any kind of web-scraping.&lt;/li&gt;&#xA;&lt;li&gt;A suggestion by @Pierre on Twitter was to try &lt;a href=&quot;https://en.wikipedia.org/w/api.php?action=query&amp;amp;list=categorymembers&amp;amp;cmtitle=Category:Antibiotic%20stubs&amp;amp;format=json&amp;amp;cmlimit=500&quot; rel=&quot;noreferrer&quot;&gt;Antimicrobial stubs&lt;/a&gt; on Wikipedia. Good idea, but the class is not stored in the ChemBox and must be free-text parsed. Or writing a script which evaluates the Wikipedia category tags.&lt;/li&gt;&#xA;&lt;li&gt;I drew blank at different government agencies (EMA, FDA, EFSA, NIH, etc.). The closest I found was from the WHO a list of critically important antibiotics. But not complete, and in a PDF.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any pointer appreciated.&lt;/p&gt;&#xA;" OwnerUserId="44" LastEditorUserId="44" LastEditDate="2017-06-22T21:43:20.580" LastActivityDate="2021-03-10T13:40:03.430" Title="Publicly available, free, complete database for antibiotics names and classes?" Tags="|database|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="880" PostTypeId="2" ParentId="878" CreationDate="2017-06-22T21:33:58.947" Score="6" Body="&lt;p&gt;&lt;strong&gt;ChEBI&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should be possible to get this information from the &lt;a href=&quot;https://www.ebi.ac.uk/chebi/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ChEBI database&lt;/a&gt;, see the &lt;a href=&quot;https://www.ebi.ac.uk/chebi/downloadsForward.do&quot; rel=&quot;nofollow noreferrer&quot;&gt;exported tables&lt;/a&gt;. You could download the ontologies (in OWL / OBO), parse them using some ad-hoc parser or using a dedicated library (e.g., &lt;a href=&quot;https://pypi.python.org/pypi/pronto&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pronto&lt;/a&gt;), build a directed acyclic graph based on the &lt;code&gt;is_a&lt;/code&gt; relationship, and extract the antibiotic subtree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;KEGG&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also use &lt;a href=&quot;http://www.genome.jp/kegg/&quot; rel=&quot;nofollow noreferrer&quot;&gt;KEGG: Kyoto Encyclopedia of Genes and Genomes&lt;/a&gt;, but a &lt;a href=&quot;http://www.kegg.jp/kegg/legal.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;subscription&lt;/a&gt; is probably required. The &lt;a href=&quot;ftp://ftp.genome.jp/pub/kegg/medicus/drug/drug&quot; rel=&quot;nofollow noreferrer&quot;&gt;exported drug table&lt;/a&gt; should be easy to parse (e.g., in Python).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a quick verification of the completeness, you can download the table, filter out everything except antibacterial agents and check the number of records:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;curl &quot;ftp://ftp.genome.jp/pub/kegg/medicus/drug/drug&quot; &amp;gt; keg_all.txt&#xA;&#xA;cat keg_all.txt \&#xA;   | perl -pe 's/\n/BREAK/g' \&#xA;   | perl -pe 's@///BREAK@///\n@g' \&#xA;   | grep 'CLASS       Antibacterial' \&#xA;   | perl -pe 's/BREAK/\n/g' \&#xA;   &amp;gt; keg_antibacterial.txt&#xA;&#xA;cat keg_antibacterial.txt | grep &quot;///&quot; | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are 646 records. However, some of them are almost identical (e.g., only 1 atom difference).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you add &lt;code&gt;| grep -i &quot;antibiotic&quot; \&lt;/code&gt; to the pipeline, then you get only 445 records.&lt;/p&gt;&#xA;" OwnerUserId="425" LastEditorUserId="425" LastEditDate="2017-06-23T15:55:14.457" LastActivityDate="2017-06-23T15:55:14.457" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="882" PostTypeId="1" AcceptedAnswerId="883" CreationDate="2017-06-22T23:42:46.283" Score="6" ViewCount="1264" Body="&lt;p&gt;I'm contributing to a python-based project that uses &lt;code&gt;Biopython&lt;/code&gt; to analyze fastq files. It currently uses &lt;code&gt;SeqIO.parse&lt;/code&gt;, which populates various structures with all of the fastq information (including converting quality scores). There is apparently a faster (lighter-weight) parser called &lt;a href=&quot;http://biopython.org/DIST/docs/api/Bio.SeqIO.QualityIO-module.html#FastqGeneralIterator&quot; rel=&quot;nofollow noreferrer&quot;&gt;FastqGeneralIterator&lt;/a&gt; that doesn't populate all of these items.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to use &lt;code&gt;FastqGeneralIterator&lt;/code&gt;, but be able to perform some of the functions &lt;code&gt;SeqIO.parse&lt;/code&gt; offers on the fly (not for &lt;em&gt;every&lt;/em&gt; sequence). For example, I'd like to convert base quality to PHRED scores for specific sequences, but I don't see that function available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an easy way to take a tuple output by &lt;code&gt;FastqGeneralIterator&lt;/code&gt; and convert it into a proper &lt;code&gt;Bio.SeqIO&lt;/code&gt; &lt;code&gt;SeqRecord&lt;/code&gt;? &lt;/p&gt;&#xA;" OwnerUserId="136" LastEditorUserId="492" LastEditDate="2017-06-23T15:04:59.877" LastActivityDate="2019-07-01T07:35:48.570" Title="How to maximize fastq parsing with FastqGeneralIterator (Bio.SeqIO.QualityIO)" Tags="|fastq|python|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="883" PostTypeId="2" ParentId="882" CreationDate="2017-06-23T13:01:54.793" Score="3" Body="&lt;p&gt;I had not come across &lt;code&gt;FastqGeneralIterator&lt;/code&gt; before, but I will start using it for some of my work!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One answer is to replicate the code in the &lt;a href=&quot;http://biopython.org/DIST/docs/api/Bio.SeqIO.QualityIO-pysrc.html#FastqPhredIterator&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;FastqPhredIterator&lt;/code&gt;&lt;/a&gt;, which does exactly what you are looking to do.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, My personal preference would be to use the biopython infrastructure as much as possible, especially when it comes to the tricky aspects of record parsing.  If the records you want extra info on are uncommon, then using &lt;code&gt;FastqPhredIterator&lt;/code&gt; for phred quality extraction might work for you:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;import io&#xA;from Bio.SeqIO.QualityIO import FastqGeneralIterator, FastqPhredIterator&#xA;&#xA;with open(&quot;input.fastq&quot;, &quot;rU&quot;) as handle:&#xA;    for title, sequence, quality in FastqGeneralIterator(handle):&#xA;        # do something with title, sequence, and quality&#xA;        # Set special_case to True when you want more info on the record&#xA;        special_case = False&#xA;        if special_case:&#xA;            record_stream = io.StringIO(&quot;\n&quot;.join([title, sequence, &quot;+&quot;, quality]))&#xA;            record = next(FastqPhredIterator(record_stream))&#xA;            # do something special with record&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What we are doing here is combining the record strings into one and dumping them into a &lt;code&gt;StringIO&lt;/code&gt; object, which implements the file handle duck-type. We then hand the in-memory handle to &lt;code&gt;FastqPhredIterator&lt;/code&gt; to parse the FASTQ record again. Make sure to set &lt;code&gt;special_case&lt;/code&gt; to &lt;code&gt;True&lt;/code&gt; only when you want extra info on a record, as the &lt;code&gt;special_case&lt;/code&gt; branch makes at least two extra copies of the FASTQ record. &lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="492" LastEditDate="2019-07-01T07:35:48.570" LastActivityDate="2019-07-01T07:35:48.570" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="884" PostTypeId="1" AcceptedAnswerId="906" CreationDate="2017-06-23T15:59:35.730" Score="9" ViewCount="240" Body="&lt;p&gt;I would like to add a text to PDB files that I'm processing with my tool, rna-pdb-tools. Someone points that the way I'm using it right now it's not correct (&lt;a href=&quot;https://github.com/mmagnus/rna-pdb-tools/issues/48&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/mmagnus/rna-pdb-tools/issues/48&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use HEADER right now which is not correct.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;HEADER Generated with rna-pdb-tools&#xA;HEADER ver 37c5b4e-dirty &#xA;HEADER https://github.com/mmagnus/rna-pdb-tools &#xA;HEADER Mon Oct 10 22:18:19 2016&#xA;ATOM      1  P     C A   1     -19.687  -3.296  65.469  1.00  0.00           P &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Do you happen to know which remark number to use? (&lt;a href=&quot;http://www.wwpdb.org/documentation/file-format-content/format33/remarks1.html&quot; rel=&quot;noreferrer&quot;&gt;http://www.wwpdb.org/documentation/file-format-content/format33/remarks1.html&lt;/a&gt;)?&lt;/p&gt;&#xA;" OwnerUserId="640" LastEditorUserId="96" LastEditDate="2017-07-27T04:05:00.873" LastActivityDate="2020-03-30T17:29:18.263" Title="PDB format: remark number for free text" Tags="|file-formats|pdb|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="885" PostTypeId="1" AcceptedAnswerId="888" CreationDate="2017-06-23T20:03:46.210" Score="11" ViewCount="4142" Body="&lt;p&gt;Do you know which quality score encoding PacBio uses now? I know some of their file formats have changed in the past year or two, but I haven't found much on their quality score encoding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most recent answer I found is from 2012, where one user states it's &quot;phred style&quot;, but doesn't state whether it's 33 or 64: &lt;a href=&quot;http://seqanswers.com/forums/archive/index.php/t-16895.html&quot; rel=&quot;noreferrer&quot;&gt;DeNovo assembly using pacBio data&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="136" LastEditorUserId="136" LastEditDate="2017-06-23T20:30:21.750" LastActivityDate="2017-06-24T20:23:05.160" Title="Which quality score encoding does PacBio use?" Tags="|ngs|pacbio|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="886" PostTypeId="1" AcceptedAnswerId="889" CreationDate="2017-06-23T21:07:18.203" Score="5" ViewCount="574" Body="&lt;p&gt;I have FPKM-UQ data from COAD-TCGA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I generated an expression set of this data using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; edata = log2(data + 1)&#xA;&amp;gt; edata[1:2,1:2]&#xA;      X01240896.3f3f.4bf9.9799.55c87bfacf36&#xA;1                                  8.540967&#xA;10                                13.528968&#xA;100                               16.296422&#xA;1000                              13.658199&#xA;10000                             15.143788&#xA;      X01ad5016.f691.4bca.82a0.910429d8d25b&#xA;1                                  9.886537&#xA;10                                16.719682&#xA;100                               17.212312&#xA;1000                              10.317842&#xA;10000                             13.166767&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Row name is entrezid, and column names are case_ids.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The batch data provided by TCGA which indicates a total batch num = 42&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; omics[1:5,]&#xA;                                      snf    boo order batch&#xA;X01240896.3f3f.4bf9.9799.55c87bfacf36   3 192807     1    22&#xA;X01ad5016.f691.4bca.82a0.910429d8d25b   2  82213     2     2&#xA;X01f493d4.229d.47a6.baa8.32a342c65d01   1 A08907     3    29&#xA;X022f39e9.57ee.4b2b.8b3a.8929e3d69a37   2 A16S13     4    32&#xA;X02f9668c.71e6.485f.88b1.b37dc8bdd2ab   3 102113     5     7&#xA;&amp;gt; batch = omics$batch&#xA;&amp;gt; batch&#xA;  [1] 22  2 29 32  7  1  8  8  7 41 26 14 26 26  1  5 22  5 15 18 28 12 12  8 19&#xA; [26] 13 38  5 27  7 41  1 22 26 28 22 33 14 13  8 10 13 11 17  5 38  5  7  7 31&#xA; [51] 22 42 11  8  7 17 15 17 12 17  2 19  2 12 18 14 22 11 21 15 17 17 33 15 19&#xA; [76] 15  6  1 10 11  2 28 36 17  9 15 12 17 15 10  6  8 13 25 11 15  1 13 24  8&#xA;[101] 13 15 17  8  2 22 17 11 13 37 19 38 13 19  5 16  5  5 22 35 35 19 13 17 22&#xA;[126]  7  8  8 41 39 14  6 19 18 38 22 14 30 19 24 11 14 13 16  7 13  8 22 12  8&#xA;[151] 22 17  2 40 13 35 15  5 24 19 41 22 22 19 17  8 13 33 10 17 29 12  9 11 24&#xA;[176] 27  5 14  7 25 13 35  9 18 16 13 31 25 18 10 35 18 14 33  5 19 17  8 11 11&#xA;[201] 42  7 10  8 39 19  8  7 11 19 31 35 28 36 13 37 10 38  3 18 13 34 39 13  3&#xA;[226] 40 31 17 19 26  7 19 12 22 19 17 12 17 13 12 28 33 37 17 35 18 40 13 20 35&#xA;[251] 17  6 15 14 12 27  5 41 31 41 18 21 19 17 40  8 41  2  5 15  5 15 15  5 39&#xA;[276]  1 22 33 13 17  7 11 15  3  7 38 15  7 14 14 22 31 14 18  3 19  7  9  6  5&#xA;[301] 22  8 11  1 13 10 19  8 14 15  6 32  1  2 19  4  5 13 18 42 11 17  8 36 19&#xA;[326] 15 22 28 15  8 11  8 15 38 19 39  7 11 42 23  5  1 22 17 35 13 40 25  7 41&#xA;[351] 38  5  8 19 31  6 38  8 36  4 15 15  7 12 22 38 29 35 15  2 10  2 19 15 25&#xA;[376]  5 39 30 19 18 15 10 10 33  1 19 32 19 19 15 20 15  7 22 35 22 30 14 12 16&#xA;[401] 21 13 15 18  7  5  6 31 12  7 15 32 33 18 14 15 15 26 31  1 14 19 17 18  1&#xA;[426]  5 10 32&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, when I run the ComBat analysis it generates all NANs, and no values:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; modcombat = model.matrix(~1, data=omics)&#xA;&amp;gt; combat_edata = ComBat(dat=edata, batch=batch, mod=modcombat, par.prior=TRUE, prior.plots=FALSE)&#xA;&amp;gt; combat_edata[1:5,1:2]&#xA;      X01240896.3f3f.4bf9.9799.55c87bfacf36&#xA;1                                       NaN&#xA;10                                      NaN&#xA;100                                     NaN&#xA;1000                                    NaN&#xA;10000                                   NaN&#xA;      X01ad5016.f691.4bca.82a0.910429d8d25b&#xA;1                                       NaN&#xA;10                                      NaN&#xA;100                                     NaN&#xA;1000                                    NaN&#xA;10000                                   NaN&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="962" LastEditorUserId="73" LastEditDate="2017-06-23T21:48:47.553" LastActivityDate="2017-06-23T22:07:12.503" Title="NaN values after ComBat analysis on TCGA COAD RNAseq" Tags="|r|combat|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="887" PostTypeId="1" AcceptedAnswerId="890" CreationDate="2017-06-23T21:12:05.243" Score="7" ViewCount="1297" Body="&lt;p&gt;I am attempting to run a BLAST search remotely using BLAST+.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can get search to work correctly at the command line with the following commands:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blastp -query proteins.fasta -remote -db nr -out proteins_nr.txt -outfmt 6 -evalue 1e-30&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I would like to us the remote database titled &quot;Microbial proteins from nr&quot; which is what would be used for the microbial blast search accessing BLAST from the NCBI site.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm struggling to find the correct code to access this database instead of just &quot;nr&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a list with the codes of different databases?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="727" LastActivityDate="2017-06-25T07:25:07.777" Title="how to set database other than nr for remote blast+ search" Tags="|database|blast|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="888" PostTypeId="2" ParentId="885" CreationDate="2017-06-23T21:30:22.757" Score="12" Body="&lt;p&gt;PacBio does use PHRED 33, &lt;em&gt;but it turns out the question may be irrelevant&lt;/em&gt; for the newer PacBio Sequel Sequencer, because it reports all base qualities as PHRED 0 (ASCII &lt;code&gt;!&lt;/code&gt;). The RS-II reports PHRED 33 quality scores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Sequel provides data in the form of &lt;em&gt;subreads&lt;/em&gt;, which are the circular consensus sequences (CCS) from a single zero-mode waveguide (ZMW). Essentially, a single circularized DNA molecule enters a ZMW well, and is sequenced as many times as possible. (Sometimes multiple molecules get into a single well, but that's not the normal case.) The number of times this single molecule can be sequenced depends on several variables, including molecule length. The consensus sequence for this single molecule is then reported out as a subread.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I started playing with the quality scores in my data, I realized the quality score for every base is &lt;code&gt;!&lt;/code&gt;, which equates to a PHRED quality of 0. I contacted someone at PacBio who informed me that they learned the base quality values were not any more useful than the bases themselves for generating a high-quality circular consensus sequence. Thus, rather than spend the computational effort to calculate the scores, they skipped it, and reported &lt;code&gt;!&lt;/code&gt; for every base. They also report an overall read quality of 0.8, but this is simply a placeholder. They are working to make this information more readily available.&lt;/p&gt;&#xA;" OwnerUserId="136" LastEditorUserId="136" LastEditDate="2017-06-24T20:23:05.160" LastActivityDate="2017-06-24T20:23:05.160" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="889" PostTypeId="2" ParentId="886" CreationDate="2017-06-23T22:07:12.503" Score="2" Body="&lt;p&gt;I think I might know what the problem is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I plotted the log2 corrected values, and the distribution may not be normal because of the 0 values present in RNASeq data. If I understand correctly, this data is actually missing, whereas with my current analysis it is viewed by R as an actual value of 0. So either the missing data needs to be imputed, or the data needs to be scaled and normalized to mean of 0, then subsequently log2 transformed.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; data &amp;lt;- standardNormalization(data)&#xA;&amp;gt; data &amp;lt;- log2(data + 1)&#xA;&amp;gt; combat_data = ComBat(dat=data, batch=batch, mod=modcombat, par.prior=TRUE, prior.plots=FALSE)&#xA;&amp;gt; combat_data[1:5,1:2]&#xA;          X01240896.3f3f.4bf9.9799.55c87bfacf36&#xA;100507661                           -0.13857428&#xA;57103                               -0.06700506&#xA;22838                                0.16295096&#xA;55567                               -0.13579991&#xA;6147                                 1.64990772&#xA;          X01ad5016.f691.4bca.82a0.910429d8d25b&#xA;100507661                           -0.13789534&#xA;57103                               -0.06210232&#xA;22838                                0.01546224&#xA;55567                               -0.13443289&#xA;6147                                 1.48473072&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="962" LastActivityDate="2017-06-23T22:07:12.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="890" PostTypeId="2" ParentId="887" CreationDate="2017-06-23T23:18:32.853" Score="9" Body="&lt;p&gt;I believe you're looking for &lt;code&gt;env_nr&lt;/code&gt;? It's listed as such, under &lt;code&gt;Metagenomic proteins&lt;/code&gt; in the blastp webpage. It appears that the word within brackets should be supplied alongside the &lt;code&gt;-db&lt;/code&gt; parameter. A quick test with a dummy amino acid fasta file does turn up a result to a valid NCBI protein accession.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hNm3k.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hNm3k.png&quot; alt=&quot;partial screenshot from NCBI&amp;#39;s blastp webpage&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I followed up on this with a little more digging around, and it appears the database you're looking for is called &lt;code&gt;Microbial_proteins&lt;/code&gt;. I ended up quote-searching &lt;code&gt;Microbial proteins from nr&lt;/code&gt; on Google and exactly 2 hits turn up. The first hit, I think, is what the OP posted a screenshot of in a comment to this answer. The second is a list of databases along with keywords from a Spain-based bioinformatics firm: &lt;a href=&quot;http://data.biobam.com/ncbi_blast_dbs_protein.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://data.biobam.com/ncbi_blast_dbs_protein.pdf&lt;/a&gt;. I retried the dummy example with the &lt;code&gt;Microbial_proteins&lt;/code&gt; database, and it seems to work. Again, I am not sure why I can't easily find this on an NCBI resource. Also, I am curious as to how different this database is from &lt;code&gt;env_nr&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="541" LastEditorUserId="541" LastEditDate="2017-06-25T07:25:07.777" LastActivityDate="2017-06-25T07:25:07.777" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="892" PostTypeId="1" AcceptedAnswerId="894" CreationDate="2017-06-24T03:12:53.440" Score="14" ViewCount="5878" Body="&lt;p&gt;How do you write a &lt;code&gt;.gz&lt;/code&gt; (or &lt;code&gt;.bgz&lt;/code&gt;) &lt;code&gt;fastq&lt;/code&gt; file using &lt;code&gt;Biopython&lt;/code&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd rather avoid a separate system call.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The typical way to write an &lt;code&gt;ASCII&lt;/code&gt; &lt;code&gt;.fastq&lt;/code&gt; is done as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;for record in SeqIO.parse(fasta, &quot;fasta&quot;):&#xA;    SeqIO.write(record, fastq, &quot;fastq&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;record&lt;/code&gt; is a &lt;code&gt;SeqRecord&lt;/code&gt; object, &lt;code&gt;fastq&lt;/code&gt; is the &lt;code&gt;file handle&lt;/code&gt;, and &lt;code&gt;&quot;fastq&quot;&lt;/code&gt; is the requested file format. The file format may be &lt;code&gt;fastq&lt;/code&gt;, &lt;code&gt;fasta&lt;/code&gt;, etc., but I do not see an option for &lt;code&gt;.gz&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the &lt;a href=&quot;http://biopython.org/DIST/docs/api/Bio.SeqIO-module.html&quot; rel=&quot;noreferrer&quot;&gt;SeqIO&lt;/a&gt; API.&lt;/p&gt;&#xA;" OwnerUserId="136" LastActivityDate="2017-08-09T20:19:15.707" Title="How do you write a .gz fastq file with Biopython?" Tags="|file-formats|python|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="893" PostTypeId="2" ParentId="892" CreationDate="2017-06-24T04:11:01.577" Score="0" Body="&lt;p&gt;As far as I know you can not write compressed fastq on the fly. You will have to write the entire fastq and then compress it.&lt;/p&gt;&#xA;" OwnerUserId="964" LastEditorUserId="73" LastEditDate="2017-06-24T12:41:55.300" LastActivityDate="2017-06-24T12:41:55.300" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="894" PostTypeId="2" ParentId="892" CreationDate="2017-06-24T06:49:27.180" Score="19" Body="&lt;p&gt;I'm not sure I'm doing it the best way, but here is an example where I read a compressed gzip fastq file and write the records in block gzip fastq:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio import SeqIO, bgzf&#xA;# Used to convert the fastq stream into a file handle&#xA;from io import StringIO&#xA;from gzip import open as gzopen&#xA;&#xA;records = SeqIO.parse(&#xA;    # There is actually simpler (thanks @peterjc)&#xA;    # StringIO(gzopen(&quot;random_10.fastq.gz&quot;).read().decode(&quot;utf-8&quot;)),&#xA;    gzopen(&quot;random_10.fastq.gz&quot;, &quot;rt&quot;),&#xA;    format=&quot;fastq&quot;)&#xA;&#xA;with bgzf.BgzfWriter(&quot;test.fastq.bgz&quot;, &quot;wb&quot;) as outgz:&#xA;    SeqIO.write(sequences=records, handle=outgz, format=&quot;fastq&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2017-06-26T17:18:28.973" LastActivityDate="2017-06-26T17:18:28.973" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="895" PostTypeId="1" AcceptedAnswerId="897" CreationDate="2017-06-24T11:19:23.527" Score="15" ViewCount="14463" Body="&lt;p&gt;I want to get a .bed file with the genes' names and canonical coordinates, also I would like to have coordinates of exons, too. I can get the list from UCSC, however, if I choose UCSC Genes - knownCanonical, I can not extract coordinates of exons. If I use other options - I am getting coordinates of as many transcriptional isoforms as were detected while I need only one canonical form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I get such BED file?&lt;/p&gt;&#xA;" OwnerUserId="349" LastEditorUserId="96" LastEditDate="2017-07-03T21:24:03.917" LastActivityDate="2022-01-07T21:28:26.383" Title="How to obtain .bed file with coordinates of all genes" Tags="|bed|public-databases|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="897" PostTypeId="2" ParentId="895" CreationDate="2017-06-24T15:37:00.853" Score="16" Body="&lt;p&gt;Via Gencode and BEDOPS &lt;code&gt;convert2bed&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ wget -qO- ftp://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_28/gencode.v28.annotation.gff3.gz \&#xA;    | gunzip --stdout - \&#xA;    | awk '$&lt;/span&gt;3 == &amp;quot;gene&amp;quot;' - \&#xA;    | convert2bed -i gff - \&#xA;    &amp;gt; genes.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can modify the &lt;code&gt;awk&lt;/code&gt; statement to get exons, by replacing &lt;code&gt;gene&lt;/code&gt; with &lt;code&gt;exon&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To get HGNC symbol names in the ID field, you can add the &lt;code&gt;--attribute-key=&amp;quot;gene_name&amp;quot;&lt;/code&gt; option to v2.4.40 or later of &lt;code&gt;convert2bed&lt;/code&gt;. This slight modification extracts the &lt;code&gt;gene_name&lt;/code&gt; attribute from the annotation record and puts it in the fourth (ID) column:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ wget -qO- ftp://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_28/gencode.v28.annotation.gff3.gz \&#xA;    | gunzip --stdout - \&#xA;    | awk '$&lt;/span&gt;3 == &amp;quot;gene&amp;quot;' - \&#xA;    | convert2bed -i gff --attribute-key=&amp;quot;gene_name&amp;quot; - \&#xA;    &amp;gt; genes.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;BEDOPS: &lt;a href=&quot;https://github.com/bedops/bedops&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/bedops/bedops&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is based off an answer I wrote on Biostars, which includes a Perl script for generating a BED file of introns from gene and exon annotations: &lt;a href=&quot;https://www.biostars.org/p/124515/#124522&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/124515/#124522&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2022-01-07T21:28:26.383" LastActivityDate="2022-01-07T21:28:26.383" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="898" PostTypeId="2" ParentId="45" CreationDate="2017-06-24T17:29:15.333" Score="5" Body="&lt;p&gt;&lt;strong&gt;&lt;em&gt;You specifically asked about FASTA files, but it's important to always consider read length and quality jointly when assessing high-error long-read data. FASTA files do not provide the quality. This information will help you determine how successful the run was, how many reads were 'high quality', etc.&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I originally posted a full answer here, suggesting &lt;a href=&quot;https://github.com/conchoecia/pauvre&quot; rel=&quot;noreferrer&quot;&gt;pauvre&lt;/a&gt;, but I decided it was a little off topic since you seem to only have the FASTA files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend generating FASTQ files, but I'm not sure whether you have the original base-called fast5 files. If so, generate FASTQs using &lt;code&gt;poretools&lt;/code&gt; as follows (&lt;a href=&quot;https://poretools.readthedocs.io/en/latest/&quot; rel=&quot;noreferrer&quot;&gt;poretools doc for generating FASTQ files&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;poretools fastq fast5/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I recommend &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/899/how-do-you-generate-read-length-vs-read-quality-plot-for-long-read-sequencing-da/900#900&quot;&gt;generating a heat map and histogram margin plot&lt;/a&gt; using &lt;a href=&quot;https://github.com/conchoecia/pauvre&quot; rel=&quot;noreferrer&quot;&gt;pauvre&lt;/a&gt; with both read length and quality like shown below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Jl7JS.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Jl7JS.png&quot; alt=&quot;My description&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;[Note: I am not the original author for pauvre, but I am now contributing to this project]&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="136" LastEditorUserId="136" LastEditDate="2017-06-24T18:33:29.243" LastActivityDate="2017-06-24T18:33:29.243" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="899" PostTypeId="1" CreationDate="2017-06-24T18:16:49.600" Score="10" ViewCount="2205" Body="&lt;p&gt;How do you generate read-length vs read-quality plot (heat map with histograms in the margin) for long-read sequencing data from the Oxford Nanopore Technologies (ONT) MinION? The MinKNOW software from ONT provides a plot like this during base calling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would also be very helpful for PacBio data.&lt;/p&gt;&#xA;" OwnerUserId="136" LastEditorUserId="6251" LastEditDate="2024-01-26T15:47:50.070" LastActivityDate="2024-01-26T15:47:50.070" Title="How do you generate read-length vs read-quality plot for long-read sequencing data (e.g., MinION)?" Tags="|python|ngs|fastq|nanopore|pacbio|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="900" PostTypeId="2" ParentId="899" CreationDate="2017-06-24T18:16:49.600" Score="4" Body="&lt;p&gt;It's important to always consider read length and quality jointly with high-error read data, and current long-read technologies (e.g., MinION and PacBio) have high error rates. &lt;strong&gt;&lt;em&gt;Considering read length and quality jointly will help you determine how successful the run was, how many reads were 'high quality', whether the longer reads are 'real' (or just pore noise), etc.&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've had a recent spike of interest in similar plots and came across a project called &lt;a href=&quot;https://github.com/conchoecia/pauvre&quot; rel=&quot;nofollow noreferrer&quot;&gt;pauvre&lt;/a&gt; (french for 'poor', play on 'pore') through the Oxford Nanopore Technologies (ONT) community that I think is even better than MinKNOW's base calling plot.  Plus, you can generate these plots from a fastq file when ever you want to, unlike with MinKNOW. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;[Note: I am not the original author, but I am now contributing because I liked (and needed) it.]&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Jl7JS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Jl7JS.png&quot; alt=&quot;My description&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pauvre will also report useful statistics:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fastq stats for fastq_runid_bb8b8ddedb22bdd6802b2bfa2b4e424c92c30d28_0.fastq&#xA;numReads: 2164829&#xA;numBasepairs: 4970615217&#xA;meanLen: 2296.077527139557&#xA;medianLen: 1495.0&#xA;minLen: 5&#xA;maxLen: 392031&#xA;N50: 3450&#xA;L50: 402786&#xA;&#xA;                               Basepairs &amp;gt;= bin by mean PHRED and length&#xA;minLen          Q0          Q5         Q10         Q15       Q17.5         Q20      Q21.5  Q25  Q25.5  Q30&#xA;     0  4970615217  4970611559  4835461771  3889995868  2900103275  1087779109  165637656  429      0    0&#xA; 50000    11531044    11531044      270324      160128       50729       50729          0    0      0    0&#xA;100000     6260554     6260554           0           0           0           0          0    0      0    0&#xA;150000     3504240     3504240           0           0           0           0          0    0      0    0&#xA;200000     2501101     2501101           0           0           0           0          0    0      0    0&#xA;250000     1609592     1609592           0           0           0           0          0    0      0    0&#xA;300000     1033423     1033423           0           0           0           0          0    0      0    0&#xA;350000      392031      392031           0           0           0           0          0    0      0    0&#xA;&#xA;                    Number of reads &amp;gt;= bin by mean Phred+Len&#xA;minLen       Q0       Q5      Q10      Q15    Q17.5     Q20  Q21.5  Q25  Q25.5  Q30&#xA;     0  2164829  2164605  2083436  1626706  1183812  435687  77341    1      0    0&#xA; 50000      109      109        5        3        1       1      0    0      0    0&#xA;100000       36       36        0        0        0       0      0    0      0    0&#xA;150000       15       15        0        0        0       0      0    0      0    0&#xA;200000        9        9        0        0        0       0      0    0      0    0&#xA;250000        5        5        0        0        0       0      0    0      0    0&#xA;300000        3        3        0        0        0       0      0    0      0    0&#xA;350000        1        1        0        0        0       0      0    0      0    0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;These plots and stats would be equally useful with PacBio, but that's not super easy (though it is possible) with current raw output from the Sequel sequencer: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/885/which-quality-score-encoding-does-pacbio-use/888&quot;&gt;Which quality score encoding does PacBio use?&lt;/a&gt;&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pauvre currently uses Biopython to parse the fastq and &lt;code&gt;matplotlib&lt;/code&gt; for the actual plot, and will let you choose the output image format (e.g., .png, .pdf, etc.). You can also choose whether the background is transparent or white (for .png output). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The parser is currently super slow because it's using &lt;code&gt;SeqIO.parse&lt;/code&gt;, but we're changing parsers to speed that up. We're also adding some extra features (e.g., choose whether to include y-axes in margin histograms, print some stats directly to the plot for documentation, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Purple is currently the only color choice (which I personally love), but adding options to change that will be super easy.&lt;/p&gt;&#xA;" OwnerUserId="136" LastEditorUserId="136" LastEditDate="2017-06-24T22:07:08.513" LastActivityDate="2017-06-24T22:07:08.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="901" PostTypeId="1" AcceptedAnswerId="902" CreationDate="2017-06-24T23:33:03.743" Score="8" ViewCount="1012" Body="&lt;p&gt;At the moment, the standard reference genomes (e.g. hg19, hg38) are haploid genomes. We know that the human genome is diploid. Naturally, the latter would be the respectively correct representation of the human genome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More and more biologists are using emerging technologies into order to capture the diploid nature of genetic information, e.g. phasing SNPs between mother and father chromosomes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the standard way that bioinformaticians generated a standard diploid reference genome? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually, reference genomes are not truly haploid (by my understanding). Given that reference genomes are 5'-3' coordinated, in order to create a complementary strand, one would need to take the 3'-5' complement. In order to have a diploid genome, you need two reference genomes and two 3'-5' complements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More importantly, how have large-scale genomic studies dealt with the fact that the haploid reference genome is a consensus-based &quot;half&quot; of a human genome? &lt;/p&gt;&#xA;" OwnerUserId="146" LastEditorUserId="37" LastEditDate="2017-06-25T02:07:50.473" LastActivityDate="2018-01-25T12:41:34.407" Title="What is the standard way to work with a diploid reference genome? Complementary strands?" Tags="|human-genome|genome|reference-genome|" AnswerCount="5" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="902" PostTypeId="2" ParentId="901" CreationDate="2017-06-25T02:00:16.993" Score="5" Body="&lt;p&gt;For calling small variants, the &lt;em&gt;standard way&lt;/em&gt; is to simply call diploid genotypes. You can already do a variety of research with unphased genotypes. You may further phase genotypes with imputation, pedigree or with long reads/linked reads, but not many are doing this because phasing is more difficult, may add cost and may not always give you new insight into your data. For these analyses, we use a haploid genome. For human samples, the vast majority of &quot;large-scale genomic studies&quot; are done this way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A diploid reference actually doesn't help much with reference-based analysis; it only complicates algorithms. What could help a lot is a population reference, which may be represented by a graph or a compressed full-text index or both. In theory, if you have a comprehensive population reference and a capable mapping algorithm, you may call extra variants that would not be callable with short reads. In practice, however, there are quite a few technical challenges. Handling population references is a research topic. There are no &quot;standards&quot; yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the goal is to assemble a new reference genome from a diploid sample, we almost always prefer to produce a diploid assembly. Unfortunately, I believe there are no &quot;standard&quot; procedures, either. SuperNova from 10x genomics builds the diploid information into a graph. Falcon from PacBio uses &quot;unzip&quot;. I don't think they have got widely used and evaluated so far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: saw your edit while writing the above. The fact that the genome only represents one strand does not mean we have to create the complement strand explicitly in analyses. We do most of reverse complement on the fly in algorithms as well as in mind.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;reference genomes are not truly haploid&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That depends on how the reference is assembled. If you sequence a haploid sample (e.g. bacteria), your assembly will be haploid. If you sequence an inbreed lab strain that is almost homozygous (e.g. mouse and fruit fly), your assembly will be nearly haploid. If you sequence a diploid sample, your assembly is very likely to be a mosaic of the two haplotypes. In case of the human reference genome, it is more complicated. It is largely a mosaic of several humans by stitching ~150kb haplotypes from these samples.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-06-25T02:00:16.993" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="903" PostTypeId="5" CreationDate="2017-06-25T02:14:49.307" Score="0" Body="" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2017-06-25T02:14:49.307" LastActivityDate="2017-06-25T02:14:49.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="904" PostTypeId="4" CreationDate="2017-06-25T02:14:49.307" Score="0" Body="An insertion/deletion sequence variation." OwnerUserId="37" LastEditorUserId="37" LastEditDate="2017-06-25T02:14:49.307" LastActivityDate="2017-06-25T02:14:49.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="905" PostTypeId="2" ParentId="901" CreationDate="2017-06-25T03:48:52.500" Score="2" Body="&lt;p&gt;There are some assemblers that produce assembly graphs that attempt to describe all the possible haploid paths within a set of reads. Such an assembly attempts to capture all the diploid variation (and/or population variation) in a sample at the expense of not having full-length chromosomes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Canu (for example) will produce contigs that are extended as long as consensus is maintained across different reads, but when there is a reliable break in coverage (i.e. an area where chromosomes are heterozygous) then the contigs will be broken up. Canu provides as output a GFA file (assembly graph) that can be used to determine which paths might combine together into a single chromosome.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-25T03:48:52.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="906" PostTypeId="2" ParentId="884" CreationDate="2017-06-25T22:03:05.130" Score="4" Body="&lt;p&gt;From &lt;a href=&quot;http://www.bmsc.washington.edu/CrystaLinks/man/pdb/part_31.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this document&lt;/a&gt;, it looks like REMARK 250 is the way to go.&lt;/p&gt;&#xA;" OwnerUserId="931" LastActivityDate="2017-06-25T22:03:05.130" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="907" PostTypeId="2" ParentId="901" CreationDate="2017-06-26T07:23:26.087" Score="6" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;At the moment, the standard reference genomes (e.g. hg19, hg38) are haploid genomes. We know that the human genome is diploid. Naturally, the latter would be the respectively correct representation of the human genome.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The premise of the OP's question is false.  The natural reference representation of the human genome is not diploid.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Think of a reference genome as a map, and not as a specific example of a human being's DNA.&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Not only is the human genome reference haploid, but &lt;a href=&quot;https://en.wikipedia.org/wiki/Human_genome#Human_reference_genome&quot; rel=&quot;nofollow noreferrer&quot;&gt;it is also a composite genome&lt;/a&gt;. This means that the human genome reference sequence is composed of sequences from multiple individuals.  In other words, the human reference does not correspond to any one human sequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any particular read from a DNA sequencer will be a read from a human genome that diverges from the reference genome. So an algorithm that tries to match the read to the reference genome will always need to handle potential discrepancies. Adding a second map against which to match a read would not change that fact.  Therefore, there is little value in providing a second haploid reference genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Side note&lt;/strong&gt;:&#xA;There are parts of the human genome that &quot;are too complex to be represented by a single path&quot;, and the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/grc/human&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genome Reference Consortium&lt;/a&gt; provides &quot;alternate loci&quot; for such regions of the genome.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Answer to the OP's second question&lt;/h3&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;More importantly, how have large-scale genomic studies dealt with the fact that the haploid reference genome is a consensus-based &quot;half&quot; of a human genome?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I interpret this question as &quot;How do large genomic studies represent genome diversity that cannot be represented by a haploid reference genome?&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One standard approach that has been very popular in genome-wide association studies is the use of a haplotype reference panel such as for example from the &lt;a href=&quot;http://www.internationalgenome.org/about&quot; rel=&quot;nofollow noreferrer&quot;&gt;1000 Genomes Project&lt;/a&gt;.  Modern phasing programs exploit the shared ancestry of the samples in a study and reference haplotypes to phase a study's samples. One popular phasing program is &lt;a href=&quot;https://mathgen.stats.ox.ac.uk/impute/impute.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Impute2&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="492" LastEditDate="2018-01-25T10:57:23.753" LastActivityDate="2018-01-25T10:57:23.753" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="908" PostTypeId="1" AcceptedAnswerId="916" CreationDate="2017-06-26T08:06:51.857" Score="8" ViewCount="3469" Body="&lt;p&gt;I have three sequencing libraries of single individual mapped to a reference using &lt;code&gt;bwa-mem&lt;/code&gt;. I would like to merge the three unsorted &lt;code&gt;.sam&lt;/code&gt; files I have so, I can call variants and heterozygosity estimates using &lt;a href=&quot;https://bitbucket.org/phaentu/atlas/wiki/Home&quot; rel=&quot;nofollow noreferrer&quot;&gt;atlas&lt;/a&gt;. Atlas requires one input mapping file (&lt;code&gt;bam&lt;/code&gt;) with defined read groups because it estimates the error profiles of different libraries separately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I merge multiple sam files? Preferably avoiding java (Picard tools).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to figure out a solution using &lt;code&gt;samtools 1.3&lt;/code&gt;. I sorted individual files using &lt;code&gt;samtools sort&lt;/code&gt;, then I used &lt;code&gt;samtools merge -r merged.bam s1.sort.sam s2.sort.sam s3.sort.sam&lt;/code&gt; to merge the sorted files. However, the read group didn't make it to the header (and the variant caller I use is complaining about it), also the read group is stupidly the file name.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to define meaningful readgroup names using procedure described on &lt;a href=&quot;https://www.biostars.org/p/47487/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioStars&lt;/a&gt;, but I found that this will just change the header, it does not adjust the names of read groups defined by &lt;code&gt;samtools merge&lt;/code&gt; (the file names).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following this related thread on &lt;a href=&quot;http://seqanswers.com/forums/showthread.php?t=4180&quot; rel=&quot;nofollow noreferrer&quot;&gt;SeqAnswers&lt;/a&gt;, I tried to define the correct header with read groups corresponding to merged file names:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools -rh rg.txt merged.bam s1.sort.sam s2.sort.sam s3.sort.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;rg.txt&lt;/code&gt; is a file with header&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@RG s1.sort&#xA;@RG s2.sort&#xA;@RG s3.sort&#xA;...&#xA;output of samtools view -H s1.sort&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, the header still had not the read group, I guess because sam header accepts only tagged items to be specified (something like &lt;code&gt;@RG    XY:s1.sort&lt;/code&gt;). So, I looked into the merged &lt;code&gt;bam&lt;/code&gt; and I found that the tag of RG is &lt;code&gt;Z:&lt;/code&gt;. So I tried to just rename header of the merged file using &lt;code&gt;samtools reheader&lt;/code&gt;, but then samtools complain about the fact that a tag needs to be of length 2:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Malformed key:value pair at line 123: &quot;@RG    Z:s1.sort&quot;&#xA;Segmentation fault (core dumped)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have opened an &lt;a href=&quot;https://github.com/samtools/samtools/issues/698&quot; rel=&quot;nofollow noreferrer&quot;&gt;issue&lt;/a&gt; to report this strange incompatibility of read groups generated by &lt;code&gt;samtools merge&lt;/code&gt; with &lt;code&gt;samtools reheader&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like solution to :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;create a bit more standardized read group names (&lt;code&gt;SM:Sample\tLB:library&lt;/code&gt; format)&lt;/li&gt;&#xA;&lt;li&gt;avoid pointless writing to disk like in sam -&gt; sorted.sam -&gt; merged.bam case (can be probably achieved through &quot;pipes and tees&quot;, thanks @bli)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I also know that I can specify RG to bwa, so the sam files will have read groups defined in the first place. But I do not like the idea of remapping three libraries just to create correct formatting of read groups.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2017-06-26T14:37:37.387" LastActivityDate="2022-08-20T06:38:35.903" Title="How to merge sam files together with adding read groups" Tags="|sam|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="909" PostTypeId="1" AcceptedAnswerId="912" CreationDate="2017-06-26T09:14:18.923" Score="6" ViewCount="1675" Body="&lt;p&gt;Note: I also posted this issue (with less context) in the bioconductor support site: &lt;a href=&quot;https://support.bioconductor.org/p/97424/&quot; rel=&quot;noreferrer&quot;&gt;https://support.bioconductor.org/p/97424/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm working on a snakemake workflow that identifies various small RNA species in &lt;em&gt;C. elegans&lt;/em&gt; small RNA-seq libraries. Some are endogenous siRNAs supposedly generated from RNA templates (through RNA-dependant RNA polymerases (RdRP)) that can be variously classified (protein-coding genes, transposons, and other repeat types).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I count such small RNA reads in a set of libraries and try to identify differentially producing sources. For this, I use DESeq2 (that I run &lt;a href=&quot;https://stackoverflow.com/q/42157335/1878788&quot;&gt;using rpy2&lt;/a&gt; from within snakemake).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure DESeq2 is always appropriate for this kind of data, but so far, the analyses would at least complete.&#xA;However, I recently added new potential types of small RNA source (simple repeats and satellites), and these happen to have low counts. I'm not 100% sure, but I suspect these low counts are the reason for failures during DESeq2 analyses (for debugging purposes, I ran this manually in R):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; dds &amp;lt;- DESeq(dds, betaPrior=T)&#xA;estimating size factors&#xA;estimating dispersions&#xA;gene-wise dispersion estimates&#xA;mean-dispersion relationship&#xA;-- note: fitType='parametric', but the dispersion trend was not well captured by the&#xA;   function: y = a/x + b, and a local regression fit was automatically substituted.&#xA;   specify fitType='local' or 'mean' to avoid this message next time.&#xA;Error in lfproc(x, y, weights = weights, cens = cens, base = base, geth = geth,  : &#xA;  newsplit: out of vertex space&#xA;In addition: There were 12 warnings (use warnings() to see them)&#xA;&amp;gt; warnings()&#xA;Warning messages:&#xA;1: In lfproc(x, y, weights = weights, cens = cens, base = base,  ... :&#xA;  procv: no points with non-zero weight&#xA;2: In lfproc(x, y, weights = weights, cens = cens, base = base,  ... :&#xA;  procv: no points with non-zero weight&#xA;3: In lfproc(x, y, weights = weights, cens = cens, base = base,  ... :&#xA;  procv: no points with non-zero weight&#xA;4: In lfproc(x, y, weights = weights, cens = cens, base = base,  ... :&#xA;  procv: no points with non-zero weight&#xA;5: In lfproc(x, y, weights = weights, cens = cens, base = base,  ... :&#xA;  procv: no points with non-zero weight&#xA;6: In lfproc(x, y, weights = weights, cens = cens, base = base,  ... :&#xA;  procv: no points with non-zero weight&#xA;7: In lfproc(x, y, weights = weights, cens = cens, base = base,  ... :&#xA;  procv: no points with non-zero weight&#xA;8: In lfproc(x, y, weights = weights, cens = cens, base = base,  ... :&#xA;  procv: no points with non-zero weight&#xA;9: In lfproc(x, y, weights = weights, cens = cens, base = base,  ... :&#xA;  procv: no points with non-zero weight&#xA;10: In lfproc(x, y, weights = weights, cens = cens, base = base,  ... :&#xA;  procv: no points with non-zero weight&#xA;11: In lfproc(x, y, weights = weights, cens = cens, base = base,  ... :&#xA;  procv: no points with non-zero weight&#xA;12: In lfproc(x, y, weights = weights, cens = cens, base = base,  ... :&#xA;  procv: no points with non-zero weight&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The count matrix indeed contains a high proportion of zeroes, the rest are mostly ones, but there are some sources that seem a little more significantly producing small RNAs:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; mean(counts_data == 0)&#xA;[1] 0.7488889&#xA;&amp;gt; mean(counts_data == 1)&#xA;[1] 0.1507407&#xA;&amp;gt; max(counts_data)&#xA;[1] 34&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How would you recommend handling such kind of data?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I for instance discard rows without enough counts?&#xA;If so, what would be a reasonable threshold?&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Edits: trying to get a dispersion plot:&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The same error occurs when trying to estimate dispersion:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; dds &amp;lt;- estimateSizeFactors(dds)&#xA;&amp;gt; dds &amp;lt;- estimateDispersions(dds)&#xA;gene-wise dispersion estimates&#xA;mean-dispersion relationship&#xA;-- note: fitType='parametric', but the dispersion trend was not well captured by the&#xA;   function: y = a/x + b, and a local regression fit was automatically substituted.&#xA;   specify fitType='local' or 'mean' to avoid this message next time.&#xA;Error in lfproc(x, y, weights = weights, cens = cens, base = base, geth = geth,  : &#xA;  newsplit: out of vertex space&#xA;In addition: There were 12 warnings (use warnings() to see them)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This seems to prevent the generation of dispersion plots:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; plotDispEsts(dds)&#xA;Error in min(py[py &amp;gt; 0], na.rm = TRUE) : &#xA;  invalid 'type' (list) of argument&#xA;In addition: Warning message:&#xA;In structure(x, class = unique(c(&quot;AsIs&quot;, oldClass(x)))) :&#xA;  Calling 'structure(NULL, *)' is deprecated, as NULL cannot have attributes.&#xA;  Consider 'structure(list(), *)' instead.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using &lt;code&gt;fitType=&quot;local&quot;&lt;/code&gt; also fails:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; dds &amp;lt;- DESeq(dds, betaPrior=T, fitType=&quot;local&quot;)&#xA;using pre-existing size factors&#xA;estimating dispersions&#xA;gene-wise dispersion estimates&#xA;mean-dispersion relationship&#xA;Error in lfproc(x, y, weights = weights, cens = cens, base = base, geth = geth,  : &#xA;  newsplit: out of vertex space&#xA;In addition: There were 12 warnings (use warnings() to see them)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But &lt;code&gt;fitType=&quot;mean&quot;&lt;/code&gt; works:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; dds &amp;lt;- DESeq(dds, betaPrior=T, fitType=&quot;mean&quot;)&#xA;estimating size factors&#xA;estimating dispersions&#xA;gene-wise dispersion estimates&#xA;mean-dispersion relationship&#xA;final dispersion estimates&#xA;fitting model and testing&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The dispersion plot then looks as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/647fF.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/647fF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What do these &lt;code&gt;fitType&lt;/code&gt; options mean, and are there good reasons why they would have to be changed in order to accomodate for low counts?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2017-06-26T09:48:42.643" LastActivityDate="2017-06-26T10:55:41.433" Title="Running differential expression analyses on count matrices with many zeroes" Tags="|r|bioconductor|differential-expression|deseq2|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="910" PostTypeId="2" ParentId="909" CreationDate="2017-06-26T09:44:14.527" Score="5" Body="&lt;p&gt;Remove low-count features in advance. This is standard for most tools including &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#pre-filtering&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2&lt;/a&gt; and &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/vignettes/edgeR/inst/doc/edgeRUsersGuide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;edgeR&lt;/a&gt; (see section 2.6). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will keep you from testing a lot of features that cannot be differentially expressed (using NB model you need at least four reads - below that the uncertainty is too high to ever be DE) and you will make the false discovery correction unnecessarily hard.&lt;/p&gt;&#xA;" OwnerUserId="599" LastEditorUserId="73" LastEditDate="2017-06-26T10:48:12.570" LastActivityDate="2017-06-26T10:48:12.570" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="911" PostTypeId="2" ParentId="899" CreationDate="2017-06-26T09:51:54.100" Score="6" Body="&lt;p&gt;I also wrote a package to create various plots from Oxford Nanopore sequencing data and alignments: &lt;a href=&quot;https://github.com/wdecoster/NanoPlot&quot; rel=&quot;noreferrer&quot;&gt;NanoPlot&lt;/a&gt;. It can be installed through pip (see also &lt;a href=&quot;https://github.com/wdecoster/NanoPlot&quot; rel=&quot;noreferrer&quot;&gt;the README on Github&lt;/a&gt;). In addition to multiple plots also a limited NanoStats output is created (see also &lt;a href=&quot;https://github.com/wdecoster/nanostat&quot; rel=&quot;noreferrer&quot;&gt;NanoStat&lt;/a&gt;). Data can be presented using:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A fastq file (optionally compressed)&lt;/li&gt;&#xA;&lt;li&gt;A bam file&lt;/li&gt;&#xA;&lt;li&gt;The sequencing_summary.txt file generated by albacore&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Using optional flags you can:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Log transform the read lengths&lt;/li&gt;&#xA;&lt;li&gt;Use aligned reads rather than sequenced reads&lt;/li&gt;&#xA;&lt;li&gt;Downsample the reads&lt;/li&gt;&#xA;&lt;li&gt;Set a maximum read length&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I've added an example below, plotting log transformed read length vs average read quality (using a kernel density estimate). More examples can be found in the &lt;a href=&quot;https://gigabaseorgigabyte.wordpress.com/2017/06/01/example-gallery-of-nanoplot/&quot; rel=&quot;noreferrer&quot;&gt;gallery on my blog&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I welcome all feedback and suggestions!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZwsHX.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZwsHX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2017-06-26T09:51:54.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="912" PostTypeId="2" ParentId="909" CreationDate="2017-06-26T10:55:41.433" Score="6" Body="&lt;p&gt;I don't think that the issue is the low counts, but rather the number of features without any real variance (the black dots at the bottom).&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;So what the heck is the dispersion plot and why does one need to fit it anyway?&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;In a typical RNAseq experiment, one measures many thousands of genes with only a few replicates per biological group. This then leads to issues when performing testing, since your variance measurement for any given gene will be larger or smaller than reality, simply due to the lack of a large number of replicates. Consequently, at least since the &lt;code&gt;limma&lt;/code&gt; package came out, most RNAseq packages have tried to fit the mean-dispersion with some sort of line and then used that as a background distribution toward which the variance assigned to each gene should be &quot;shrunken&quot; (i.e., moved toward). Getting this at least approximately correct is important, since if you have few replicates and the final &quot;shrunken variance&quot; is significantly wrong, then your p-values will also be wrong.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;What are the various fitType options and what are they doing?&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The three options that one can use to fit this relationship are &lt;code&gt;parametric&lt;/code&gt;, &lt;code&gt;local&lt;/code&gt;, and &lt;code&gt;mean&lt;/code&gt;. &lt;code&gt;parametric&lt;/code&gt; is the default and basically results in fitting to a bent line. This usually works quite well, but you can see that you have two clouds of data, one shaped like a bent line (at the top) and another (the black dots at the bottom) with no variance. That's causing this fit type to fail. Likewise, &lt;code&gt;local&lt;/code&gt; is doing a local fit and failing for the same reason. &lt;code&gt;mean&lt;/code&gt; is just taking the mean (average) of all of the estimates. This is fool-proof, but you're then regressing to something that really doesn't fit your data. &lt;code&gt;mean&lt;/code&gt; is really a last choice if nothing else works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your particular dataset, my main concerns would be figuring out (1) why the maximum mean counts are around 20 (that's really low) and (2) why so many of these features have incredibly low variance.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-06-26T10:55:41.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="913" PostTypeId="2" ParentId="908" CreationDate="2017-06-26T16:50:19.213" Score="3" Body="&lt;p&gt;&lt;strong&gt;Not so elegant but working solution&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found a solution that satisfy several of my conditions, basically I just have to assign read groups to individual mapping files, which can be just added to sorting process and then merge the sorted sam files with read groups.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-js prettyprint-override&quot;&gt;&lt;code&gt;function sort_and_assign_RG {&#xA;    # $1 input file&#xA;    # $2 read group id&#xA;    # $3 library&#xA;&#xA;    OFILE=$(basename $1 .sam).sort.RG.sam&#xA;    HEADER=$(basename $1 .sam).header.sam&#xA;&#xA;    # since I take header from unsorted file, so I need to add this information to header as well as the readgroup&#xA;    echo -e &quot;@HD\tVN:1.3\tSO:coordinate&quot; &amp;gt; $HEADER&#xA;    # add read group to header&#xA;    echo -e &quot;@RG\tID:&quot;$2&quot;\tLB:&quot;&quot;$3&quot; &amp;gt;&amp;gt; $HEADER&#xA;    # and the rest of the header&#xA;    samtools view -H $1 &amp;gt;&amp;gt; $HEADER&#xA;&#xA;    # now sort the input sam, remove header, &#xA;    # attach to each alignment read group ID (RGID) &#xA;    # and cat header and alignment together&#xA;    samtools sort -- $1 | samtools view - | \&#xA;      awk -v RGID=&quot;$2&quot; '{ printf &quot;%s\tRG:Z:%s\n&quot;, $0, RGID; }' | \&#xA;      cat $HEADER - &amp;gt; $OFILE&#xA;&#xA;    # remove temp header&#xA;    rm $HEADER&#xA;}&#xA;&#xA;sort_and_assign_RG s1.sam s1 is180&#xA;sort_and_assign_RG s2.sam s2 is350&#xA;sort_and_assign_RG s3.sam s3 is550&#xA;&#xA;samtools merge merged.bam s{1,2,3}.sort.RG.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It would be nice to avoid creation of three &lt;code&gt;sam&lt;/code&gt; files on the way, but at least I get standardized read groups in a merged bam tile.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;--- edit ---&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also found where my confusion is coming from. The read group in header is in format &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@RG    ID:foo    LB:lib_foo    ...`&#xA;@RG    ID:bar    LB:lib_bar    ...`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;ID&lt;/code&gt; is read group ID (not sample ID as I thought) and other tags are just specifications of the read group. Then individual mapping have read group assigned in different format&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{aliment1}    RG:Z:foo&#xA;{aliment2}    RG:Z:bar&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;RG&lt;/code&gt; is just a tag for read group and &lt;code&gt;Z&lt;/code&gt; is just a mark that says that this tag is just a &quot;printable string&quot;. Therefore I think that another solution would just to merge these sorted files and then just adding three &lt;strong&gt;correct&lt;/strong&gt; lines to the header.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to say, &lt;code&gt;.sam&lt;/code&gt; is a unbelievable intuitive format.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2017-06-26T17:23:21.913" LastActivityDate="2017-06-26T17:23:21.913" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="914" PostTypeId="1" CreationDate="2017-06-26T20:06:27.333" Score="5" ViewCount="250" Body="&lt;p&gt;What tools are available to predict, based on the structure of a certain protein, its interactions within a cell?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I am considering the split GFP protein and I am trying to predict if there will be any interactions which might hinder its binding efficiency to the other half of the GFP protein. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I am not only looking for protein interactions, but other molecules as well.&lt;/p&gt;&#xA;" OwnerUserId="823" LastEditorUserId="823" LastEditDate="2017-06-27T15:34:30.763" LastActivityDate="2017-10-11T18:22:13.730" Title="Tool for predicting interactions in the cell" Tags="|protein-structure|interactions|3d-structure|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="915" PostTypeId="2" ParentId="901" CreationDate="2017-06-26T20:07:16.807" Score="3" Body="&lt;p&gt;To add to all the other great answers, I would mention that the question is somewhat misleading. If the reference genome is for a single individual, then it should be diploid. However, it's a reference for all humans. It should really contain billions of copies to fully account for all the diversity. Since that is not realistic, the reference serves as a simple approximation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This point was addressed by the recent &lt;a href=&quot;https://www.nature.com/articles/ncomms13637&quot; rel=&quot;nofollow noreferrer&quot;&gt;Korean genome paper&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Human genomes are routinely compared against a universal reference.&#xA;  However, this strategy could miss population-specific and personal&#xA;  genomic variations, which may be detected more efficiently using an&#xA;  ethnically relevant or personal reference. ... Systematic comparison&#xA;  of human assemblies shows the importance of assembly quality,&#xA;  suggesting the necessity of new technologies to comprehensively map&#xA;  ethnic and personal genomic structure variations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="35" LastActivityDate="2017-06-26T20:07:16.807" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="916" PostTypeId="2" ParentId="908" CreationDate="2017-06-27T04:10:47.523" Score="5" Body="&lt;p&gt;I have used &lt;a href=&quot;https://github.com/ekg/bamaddrg&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/ekg/bamaddrg&lt;/a&gt; to add read groups quickly to multiple sam files.  And then you can do a samtools merge of the tagged files.&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2017-06-27T04:10:47.523" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="917" PostTypeId="2" ParentId="343" CreationDate="2017-06-27T08:06:22.357" Score="7" Body="&lt;p&gt;For pre-existing reliabe TE libraries it is a bit of a mess, because not everybody deposits the species-specific TE libraries to a database like RepBase. And as far as I know DFAM contains only human resources, or am I wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for &lt;em&gt;de novo&lt;/em&gt; generation of species-specific TE libraries (which should be done for any species not already present in eg. RepBase):&#xA;There is no &quot;gold-standard&quot; how to tackle this best.&#xA;In principle one has to think about two main parts&#xA;-repeat detection&#xA;-annotation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;strong&gt;repeat detection&lt;/strong&gt; I would recommend using a combination of two things (which is necessary, because TE copies might miss in the assemblies as repetitive regions tend to be difficult to assemble and thrown away in the final assembly).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I) Repeat detection from raw reads (as with e.g. DNApipeTE or tedna or RepeatExplorer). For me, DNAPipeTE worked quite nicely, but everything has pros and cons.&#xA;II) Repeat detection from assemblies (as with e.g. REPET or as mentioned before RepeatModeler)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the &lt;strong&gt;annotation&lt;/strong&gt; of these repeats is tricky too, because most methods are relying on homology between the &lt;em&gt;de novo&lt;/em&gt; TEs and the TEs from some (probably distantly) related species. But some programs take also structure into account (like REPCLASS). REPET can do both detection and annotation, but is a pain to get to run. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would recommend using some programs to do de novo repeat detection on your species of interest on both the raw reads and assembly, clustering these libraries together (with e.g. uclust and 95% identity) and then run an annotation with homology and structural identification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably the programs will &lt;strong&gt;not give you complete, full-length TE&lt;/strong&gt;s but rather consensus sequences of several copies from TE families.&#xA;If you want, you could search all copies of one family, extract them from the contigs plus boundaries and align them manually and curate boundaries manually. Then extend boundaries if not hitting the surrounding (non-alignable) regions or landmarks of TEs like LTRs or TIRs or so. But this is very time consuming if you only want to compare TE abundance between species for example, I would not do this and rather compare the abundance using read coverage (as in &lt;a href=&quot;https://doi.org/10.1093/molbev/msv261&quot; rel=&quot;noreferrer&quot;&gt;Bast et al. 2016&lt;/a&gt;). Depends all on the questions you want to ask.&lt;/p&gt;&#xA;" OwnerUserId="518" LastActivityDate="2017-06-27T08:06:22.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="918" PostTypeId="2" ParentId="914" CreationDate="2017-06-27T08:17:20.487" Score="7" Body="&lt;p&gt;You could try one of these tools to predict protein-protein interactions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://cb.csail.mit.edu/cb/struct2net/webserver/about.html&quot; rel=&quot;noreferrer&quot;&gt;Struct2Net&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Given two protein sequences, the structure-based interaction prediction technique threads these two sequences to all the protein complexes in the PDB and then chooses the best potential match. Based on this match, the method generates alignment scores, z-scores, and an interfacial energy for the sequence pair. Logistic regression is then used to evaluate whether a set of scores corresponds to an interaction or not. The algorithm is also extended to find all potential partners given a single protein sequence. Further details about the method are described &lt;a href=&quot;http://helix-web.stanford.edu/psb06/singh.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://cic.scu.edu.cn/bioinformatics/predict_ppi/default.html&quot; rel=&quot;noreferrer&quot;&gt;PRED&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Pred_PPI is a web-based system that serves for predicting PPIs from different organisms. This server is freely available to any researcher wishing to use it for non-commercial purposes. Based on auto covariance (AC) and support vector machine (SVM), this tool is capable of predicting PPIs for any target protein pair only using their primary sequences, and assigning an interaction probability to each SVM prediction as well. So the user can use this tool to predict novel PPIs with high confidence.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;For protein-RNA interactions, you can try:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://service.tartaglialab.com/page/catrapid_omics_group&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;cat&lt;/em&gt;Rapid&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Through the calculation of secondary structure, hydrogen bonding and van der Waals contributions, catRAPID is able predict protein-RNA interaction propensities with great accuracy (up to 89% on the ncRNA-protein interaction database, NPinter).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="298" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-06-28T08:57:51.490" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="920" PostTypeId="1" AcceptedAnswerId="931" CreationDate="2017-06-27T10:17:45.383" Score="10" ViewCount="4196" Body="&lt;p&gt;I want to compare two phylogenies and colour the association lines based on some metadata I have. I have been using ape cophyloplot but I have not had any success in getting the lines to colour accurately according to my data (&lt;a href=&quot;https://stackoverflow.com/questions/44006500/r-coloured-lines-for-tangelgram-package-ape-function-cophyloplot&quot;&gt;see previous question&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that in my actual work flow I define the colour scheme using a palette to control the colour outcome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want a means to make a tangle using phylogenies which I can format. Preferably in R. I like to get an output like this: &lt;img src=&quot;https://i.stack.imgur.com/HTDfR.jpg&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="982" LastEditorUserId="982" LastEditDate="2017-06-29T15:49:07.763" LastActivityDate="2017-06-29T15:49:07.763" Title="How do I generate a color-coded tanglegram?" Tags="|r|phylogeny|software-recommendation|phylogenetics|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="921" PostTypeId="2" ParentId="920" CreationDate="2017-06-27T12:30:55.550" Score="11" Body="&lt;p&gt;I think you can try dendextend, in this &lt;a href=&quot;https://cran.r-project.org/web/packages/dendextend/vignettes/Cluster_Analysis.html&quot; rel=&quot;noreferrer&quot;&gt;manual&lt;/a&gt; there is an example of coloring the branches. I don't think it is exactly like your coloring, but with a little tweaking you might get your colorscheme in there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The manual mentions an argument called &lt;code&gt;color_lines&lt;/code&gt; for the function  &lt;code&gt;tanglegram()&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# The `which` parameter allows us to pick the elements in the list to compare&#xA;iris_dendlist %&amp;gt;% dendlist(which = c(1,4)) %&amp;gt;% ladderize %&amp;gt;% &#xA;   # untangle(method = &quot;step1side&quot;, k_seq = 3:20) %&amp;gt;%&#xA;   set(&quot;rank_branches&quot;) %&amp;gt;%&#xA;   tanglegram(common_subtrees_color_branches = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;According to the manual, the code above produces an image like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/v8kWS.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/v8kWS.png&quot; alt=&quot;example of a colored tanglegram&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try to see if you can fit your color palette in there.&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="298" LastEditDate="2017-06-27T13:35:46.243" LastActivityDate="2017-06-27T13:35:46.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="922" PostTypeId="2" ParentId="920" CreationDate="2017-06-27T13:44:49.057" Score="2" Body="&lt;p&gt;As an alternative (though I realise this doesn't really answer the OPs question directly), &lt;a href=&quot;http://dendroscope.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dendroscope&lt;/a&gt; and &lt;a href=&quot;https://sites.google.com/site/cophylogeny/treemap&quot; rel=&quot;nofollow noreferrer&quot;&gt;Treemap&lt;/a&gt; can also produce these types of plots. I don't know if they'll do coloured connections off the top of my head, but I think they can differentially colour the tree branches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just in case that is of use!&lt;/p&gt;&#xA;" OwnerUserId="929" LastActivityDate="2017-06-27T13:44:49.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="923" PostTypeId="1" AcceptedAnswerId="930" CreationDate="2017-06-27T16:18:53.597" Score="4" ViewCount="4347" Body="&lt;p&gt;I want to convert my phylogeny into a dendrogram so I can use it with dendextend in R to produce a tanglegram. I have made some progress but I keep encountering errors, see below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(ape)&#xA;library(dendextend)&#xA;&#xA;Tree &amp;lt;- rtree(10, rooted=F)&#xA;&#xA;Tree &amp;lt;- read.tree(file=&quot;clipboard&quot;, text=NULL) &#xA;# test &amp;lt;- as.dendrogram(Tree) &#xA;## Error in ape::as.hclust.phylo(object) : the tree is not ultrametric&#xA;&#xA;Tree.ultra &amp;lt;- chronos(Tree)  &#xA;# test &amp;lt;- as.dendrogram(Tree.ultra) &#xA;## Error in ape::as.hclust.phylo(object) : the tree is not rooted&#xA;&#xA;Tree.ultra$root.edge &amp;lt;- 0&#xA;# test &amp;lt;- as.dendrogram(Tree.ultra) &#xA;## Error in ape::as.hclust.phylo(object) : the tree is not binary&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;https://www.biostars.org/p/4576/&quot; rel=&quot;nofollow noreferrer&quot;&gt;thread&lt;/a&gt; seems to be the closest to answering my issues but the workflow is different and Im not sure how to address this binary issue. I have tried using as.hclust, as.hclust.phylo and hclust. &lt;/p&gt;&#xA;" OwnerUserId="982" LastEditorUserId="3848" LastEditDate="2019-08-21T13:26:09.930" LastActivityDate="2019-08-21T13:26:09.930" Title="How to convert a phylogeny to a dendrogram in R" Tags="|r|phylogenetics|phylogeny|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="924" PostTypeId="1" AcceptedAnswerId="927" CreationDate="2017-06-27T16:55:20.077" Score="5" ViewCount="137" Body="&lt;p&gt;I have a protein of interest and I would like to now how it interacts with RNA. I have structures of both molecules.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What tool can I use?&lt;/p&gt;&#xA;" OwnerUserId="988" LastEditorUserId="77" LastEditDate="2017-06-27T21:26:30.170" LastActivityDate="2017-06-28T09:05:20.913" Title="How can I dock a protein to a nucleic acid?" Tags="|proteins|software-recommendation|rna|docking|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="925" PostTypeId="2" ParentId="923" CreationDate="2017-06-27T19:56:19.770" Score="2" Body="&lt;p&gt;For your example to work you should replace the &lt;code&gt;rtree&lt;/code&gt; function with the &lt;code&gt;rcoal&lt;/code&gt; function, see example below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(ape)  &#xA;library(dendextend)   &#xA;Tree &amp;lt;- rtree(10, rooted=F)&#xA;is.ultrametric(Tree) &#xA;[1] FALSE &#xA;is.binary.tree(Tree) &#xA;[1] TRUE&#xA;&#xA;Tree_rcoal &amp;lt;- rcoal(10, rooted=F) &#xA;is.ultrametric(Tree_rcoal) &#xA;[1] TRUE&#xA;is.binary.tree(Tree_rcoal) &#xA;[1] TRUE&#xA;&#xA;Tree_rcoal2 &amp;lt;- rcoal(10, rooted=F)&#xA;&#xA;tanglegram(Tree_rcoal, Tree_rcoal2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To make use of a file in newick format, you can try to convert with the &lt;code&gt;chronos&lt;/code&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;s &amp;lt;- &quot;owls(((Strix_aluco:4.2,Asio_otus:4.2):3.1,Athene_noctua:7.3):6.3,Tyto_alba:13.5);&quot;&#xA;cat(s, file = &quot;ex.tre&quot;, sep = &quot;\n&quot;)&#xA;tree.owls &amp;lt;- read.tree(&quot;ex.tre&quot;)&#xA;is.ultrametric(tree.owls)&#xA;[1] FALSE&#xA;dendrogram &amp;lt;- chronos(tree.owls)&#xA;&#xA;Setting initial dates...&#xA;Fitting in progress... get a first set of estimates&#xA;         Penalised log-lik = -24.04992 &#xA;Optimising rates... dates... -24.04992 &#xA;Optimising rates... dates... -24.04908 &#xA;&#xA;Done.&#xA;is.ultrametric(dendrogram)&#xA;[1] TRUE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2017-06-28T08:04:56.623" LastActivityDate="2017-06-28T08:04:56.623" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="926" PostTypeId="2" ParentId="765" CreationDate="2017-06-27T21:11:07.147" Score="1" Body="&lt;p&gt;Firstly, I would download some sort of editor that is built to handle nucleotide sequences. There are a handful out there, but APE is a fairly popular software designed for plasmid editing. It is much more efficient at searching for nucleotide strings as it can search for reverse complements and to a certain extent, mismatches. It's also free!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://biologylabs.utah.edu/jorgensen/wayned/ape/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://biologylabs.utah.edu/jorgensen/wayned/ape/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your question, I'm not exactly sure what you are asking, but I would use the BLAST alignment tool to try to see similarity between your RNAi sequences and the target mRNA. You input fasta files and the program searches for any possible overlap between the two inputs. The higher the &quot;max-score&quot; the more likely the two strands (or subsections of strands) have high similarity. This program also considers the complement&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PROGRAM=blastn&amp;amp;PAGE_TYPE=BlastSearch&amp;amp;LINK_LOC=blasthome&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PROGRAM=blastn&amp;amp;PAGE_TYPE=BlastSearch&amp;amp;LINK_LOC=blasthome&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use the results from BLAST to annotate in APE using different colors which is extremely helpful. You can also use BLAST in the user interface of APE. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also keep in mind RNAi's are not always effective, even though they are predicted to bind at a given location, that may not be a biological reality &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck! &lt;/p&gt;&#xA;" OwnerUserId="470" LastActivityDate="2017-06-27T21:11:07.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="927" PostTypeId="2" ParentId="924" CreationDate="2017-06-27T23:24:38.537" Score="3" Body="&lt;p&gt;I'm not sure what you meant but you can take a look at &lt;a href=&quot;http://genesilico.pl/NPDock&quot; rel=&quot;nofollow noreferrer&quot;&gt;NPDock&lt;/a&gt; (disclaimer we wrote that tool). If you have a structure of your protein of interest, you can dock it to the structure of your DNA/RNA of interest. Mind that this is a rigid body docking which means that the structure will not change upon binding. &lt;/p&gt;&#xA;" OwnerUserId="640" LastEditorUserId="48" LastEditDate="2017-06-28T09:05:20.913" LastActivityDate="2017-06-28T09:05:20.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="928" PostTypeId="1" AcceptedAnswerId="929" CreationDate="2017-06-28T05:28:39.973" Score="6" ViewCount="165" Body="&lt;p&gt;I have a distance matrix generated by hierfstat, thusly, with a link to the fasta file &lt;a href=&quot;https://www.dropbox.com/s/z9n89y080hvl3s2/2017.06.13.strucutre_test.masked.snp_sites.v2_3_2.subsample.fa?dl=0&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(adegenet)&#xA;library(hierfstat)&#xA;snps &amp;lt;- fasta2DNAbin('test.fa', chunkSize = 50)&#xA;gi &amp;lt;- DNAbin2genind(snps)&#xA;# manually define the populations for passing to pairwise.fst&#xA;p &amp;lt;- c('Botswana', 'Botswana', 'Botswana', 'Botswana', 'France', 'France', 'Vietnam', 'Vietnam', 'Uganda', 'Uganda', 'Uganda', 'Uganda', 'Vietnam', 'Vietnam', 'Laos', 'Laos', 'Laos', 'Vietnam', 'Vietnam', 'Vietnam', 'Vietnam', 'Vietnam')&#xA;f &amp;lt;- pairwise.fst(gi, p, res.type = c('dist', 'matrix'))&#xA;as.matrix(f)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which produces&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    1   2   3   4   5&#xA;1   0   0.2189008   0.225567409 0.1821518   0.259409722&#xA;2   0.2189008   0   0.130736953 0.1648034   0.191050772&#xA;3   0.2255674   0.130737    0   0.1669077   0.006396789&#xA;4   0.1821518   0.1648034   0.166907683 0   0.203931457&#xA;5   0.2594097   0.1910508   0.006396789 0.2039315   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is it safe to assume that 1 = Botswana, 2 = France, 3 = Vietnam, 4 = Uganda and 5 = Laos? i.e. that the distance matrix rows/columns follow the order that each population first appeared in &lt;code&gt;p&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to determine for certain, which row/column in the distance matrix corresponds to which population?&lt;/p&gt;&#xA;" OwnerUserId="993" LastEditorUserId="993" LastEditDate="2017-06-28T08:04:01.480" LastActivityDate="2017-06-28T08:23:44.230" Title="What are the labels in my rstats distance matrix?" Tags="|r|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="929" PostTypeId="2" ParentId="928" CreationDate="2017-06-28T08:23:44.230" Score="2" Body="&lt;p&gt;Disclaimer: I don't have adagenet and hierfstat installed, I just looked at the source code. It seems like pairwise.fst (the deprecated code from the adagenet package, which you seem to be using) already returns a matrix. Set res.type=&quot;matrix&quot; and try calling &lt;code&gt;rownames(f)&lt;/code&gt; and &lt;code&gt;colnames(f)&lt;/code&gt; afterwards.&lt;/p&gt;&#xA;" OwnerUserId="787" LastActivityDate="2017-06-28T08:23:44.230" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="930" PostTypeId="2" ParentId="923" CreationDate="2017-06-28T09:23:27.727" Score="1" Body="&lt;p&gt;After searching high and low I have found an answer from this &lt;a href=&quot;https://stackoverflow.com/questions/7445684/how-to-convert-a-tree-to-a-dendrogram-in-r&quot;&gt;thread&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Workflow goes like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(DECIPHER)&#xA;dend1 &amp;lt;- ReadDendrogram(file=&quot;clipboard&quot;) &#xA;&#xA;# Alternatively to load from existing environment &#xA;Tree$node.label &amp;lt;- NULL # Need to remove node labels&#xA;dend1 &amp;lt;- ReadDendrogram(textConnection(write.tree(Tree))) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This object can then be used in the dendextend package and tanglegram function. &lt;/p&gt;&#xA;" OwnerUserId="982" LastEditorUserId="982" LastEditDate="2017-10-04T15:45:16.353" LastActivityDate="2017-10-04T15:45:16.353" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="931" PostTypeId="2" ParentId="920" CreationDate="2017-06-28T11:19:21.197" Score="3" Body="&lt;p&gt;Thanks everyone for the suggestions @b.nota answer is useful to colour according to clade groups but does not address my actual question on using metadata to colour the lines. For this answer see below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;dendextend states for &lt;code&gt;color_lines&lt;/code&gt;&#xA;a vector of colors for the lines connected the labels. If the colors are shorter than the number of labels, they are recycled (and a warning is issued). The colors in the vector are applied on the lines from the bottom up.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Extract labels from dendrogram on the left&#xA;labels &amp;lt;- dendA %&amp;gt;% set(&quot;labels_to_char&quot;) %&amp;gt;% labels &#xA;&#xA;#Using a metadata table with colours create a vector of colours&#xA;labels &amp;lt;- as.data.frame(labels)&#xA;labels2 &amp;lt;- merge(labels, metadata, by.x=&quot;labels&quot;, by.y=&quot;Sample.name&quot;, sort=F)&#xA;cols &amp;lt;- as.character(labels2$Colours) &#xA;&#xA;# Make tanglegram&#xA;tanglegram(dendA, dendC, color_lines = cols)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Side note if anyone with a phylogeny needs to convert it to a dendrogram to try this on their data see &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/923/how-to-convert-a-phylogeny-to-a-dendrogram-in-r/930?noredirect=1#comment1714_930&quot;&gt;my other question&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="982" LastEditorUserId="982" LastEditDate="2017-06-29T15:48:38.730" LastActivityDate="2017-06-29T15:48:38.730" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="932" PostTypeId="2" ParentId="920" CreationDate="2017-06-28T12:40:25.230" Score="3" Body="&lt;p&gt;To get your colors in the original order, you can make a dataframe with the labels in the right order with your metadata. You can use &lt;code&gt;merge&lt;/code&gt; for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So from your example code it would be something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;site &amp;lt;- structure(list(name = structure(c(1L, 3L, 4L, 5L, 6L, 7L, 8L,9L, 10L, 2L), .Label = c(&quot;t1&quot;, &quot;t10&quot;, &quot;t2&quot;, &quot;t3&quot;, &quot;t4&quot;, &quot;t5&quot;,&quot;t6&quot;, &quot;t7&quot;, &quot;t8&quot;, &quot;t9&quot;), class = &quot;factor&quot;), site = c(1L, 1L,1L, 2L, 2L, 3L, 1L, 3L, 2L, 2L)), .Names = c(&quot;name&quot;, &quot;site&quot;), row.names = c(NA,10L), class = &quot;data.frame&quot;) &#xA;&#xA;library(ape)&#xA;library(dendextend)&#xA;&#xA;t1 &amp;lt;- rcoal(10)&#xA;t2 &amp;lt;- rcoal(10)&#xA;&#xA;str(site)&#xA;# The name are factors now, so make chr first&#xA;site$name &amp;lt;- as.character(site$name)&#xA;&#xA;# make a data.frame of your labels&#xA;labels_df &amp;lt;- data.frame(t1$tip.label)&#xA;&#xA;#merge the 2 data.frames in the right order (hence sort=F)&#xA;colors &amp;lt;- merge(labels_df,site,by.x=&quot;t1.tip.label&quot;, by.y=&quot;name&quot;,all.x=T, all.y=F,sort=F)&#xA;&#xA;tanglegram(t1, t2, color_lines=colors$site)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastActivityDate="2017-06-28T12:40:25.230" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="933" PostTypeId="1" AcceptedAnswerId="934" CreationDate="2017-06-28T12:53:59.550" Score="14" ViewCount="3240" Body="&lt;p&gt;I currently find Harvard's RESTful API for &lt;a href=&quot;http://exac.hms.harvard.edu/&quot; rel=&quot;noreferrer&quot;&gt;ExAC&lt;/a&gt; extremely useful and I was hoping that a similar resource is available for &lt;a href=&quot;http://gnomad.broadinstitute.org/&quot; rel=&quot;noreferrer&quot;&gt;Gnomad&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know of a public access API for Gnomad or possibly any plans to integrate Gnomad into the Harvard API?&lt;/p&gt;&#xA;" OwnerUserId="195" LastEditorUserId="96" LastEditDate="2019-01-15T14:27:14.393" LastActivityDate="2022-10-20T07:22:41.927" Title="Is there public RESTful api for Gnomad?" Tags="|variation|api|" AnswerCount="7" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="934" PostTypeId="2" ParentId="933" CreationDate="2017-06-28T13:04:58.663" Score="15" Body="&lt;p&gt;As far as I know, no but the vcf.gz files are behind a http server that supports &lt;em&gt;Byte-Range&lt;/em&gt;, so you can use &lt;code&gt;tabix&lt;/code&gt; or any related API:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ tabix &quot;https://storage.googleapis.com/gnomad-public/release-170228/vcf/exomes/gnomad.exomes.r2.0.1.sites.vcf.gz&quot; &quot;22:17265182-17265182&quot;&#xA;22  17265182    .   A   T   762.04  PASS    AC=1;AF=4.78057e-06;AN=209180;BaseQRankSum=-4.59400e+00;ClippingRankSum=2.18000e+00;DP=4906893;FS=1.00270e+01;InbreedingCoeff=4.40000e-03;MQ=3.15200e+01;MQRankSum=1.40000e+00;QD=1.31400e+01;ReadPosRankSum=2.23000e-01;SOR=9.90000e-02;VQSLOD=-5.12800e+00;VQSR_culprit=MQ;GQ_HIST_ALT=0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|1;DP_HIST_ALT=0|0|0|0|0|0|0|0|0|0|0|1|0|0|0|0|0|0|0|0;AB_HIST_ALT=0|0|0|0|0|0|0|0|0|0|0|0|1|0|0|0|0|0|0|0;GQ_HIST_ALL=1591|589|120|301|650|589|1854|2745|1815|4297|5061|2921|10164|1008|6489|1560|7017|457|6143|52950;DP_HIST_ALL=2249|1418|6081|11707|16538|9514|28624|23829|7391|853|95|19|1|0|0|1|0|1|0|0;AB_HIST_ALL=0|0|0|0|0|0|0|0|0|0|0|0|1|0|0|0|0|0|0|0;AC_AFR=0;AC_AMR=0;AC_ASJ=0;AC_EAS=0;AC_FIN=1;AC_NFE=0;AC_OTH=0;AC_SAS=0;AC_Male=1;AC_Female=0;AN_AFR=11994;AN_AMR=31324;AN_ASJ=7806;AN_EAS=13112;AN_FIN=20076;AN_NFE=94516;AN_OTH=4656;AN_SAS=25696;AN_Male=114366;AN_Female=94814;AF_AFR=0.00000e+00;AF_AMR=0.00000e+00;AF_ASJ=0.00000e+00;AF_EAS=0.00000e+00;AF_FIN=4.98107e-05;AF_NFE=0.00000e+00;AF_OTH=0.00000e+00;AF_SAS=0.00000e+00;AF_Male=8.74386e-06;AF_Female=0.00000e+00;GC_AFR=5997,0,0;GC_AMR=15662,0,0;GC_ASJ=3903,0,0;GC_EAS=6556,0,0;GC_FIN=10037,1,0;GC_NFE=47258,0,0;GC_OTH=2328,0,0;GC_SAS=12848,0,0;GC_Male=57182,1,0;GC_Female=47407,0,0;AC_raw=1;AN_raw=216642;AF_raw=4.61591e-06;GC_raw=108320,1,0;GC=104589,1,0;Hom_AFR=0;Hom_AMR=0;Hom_ASJ=0;Hom_EAS=0;Hom_FIN=0;Hom_NFE=0;Hom_OTH=0;Hom_SAS=0;Hom_Male=0;Hom_Female=0;Hom_raw=0;Hom=0;POPMAX=FIN;AC_POPMAX=1;AN_POPMAX=20076;AF_POPMAX=4.98107e-05;DP_MEDIAN=58;DREF_MEDIAN=5.01187e-84;GQ_MEDIAN=99;AB_MEDIAN=6.03448e-01;AS_RF=9.18451e-01;AS_FilterStatus=PASS;CSQ=T|missense_variant|MODERATE|XKR3|ENSG00000172967|Transcript|ENST00000331428|protein_coding|4/4||ENST00000331428.5:c.707T&amp;gt;A|ENSP00000331704.5:p.Phe236Tyr|810|707|236|F/Y|tTc/tAc||1||-1||SNV|1|HGNC|28778|YES|||CCDS42975.1|ENSP00000331704|Q5GH77||UPI000013EFAE||deleterious(0)|benign(0.055)|hmmpanther:PTHR14297&amp;amp;hmmpanther:PTHR14297:SF7&amp;amp;Pfam_domain:PF09815||||||||||||||||||||||||||||||,T|regulatory_region_variant|MODIFIER|||RegulatoryFeature|ENSR00000672806|TF_binding_site|||||||||||1||||SNV|1||||||||||||||||||||||||||||||||||||||||||||,T|regulatory_region_variant|MODIFIER|||RegulatoryFeature|ENSR00001729562|CTCF_binding_site|||||||||||1||||SNV|1||||||||||||||||||||||||||||||||||||||||||||&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;: 2019: the current server for gnomad doesn't support Byte-Range requests.&lt;/p&gt;&#xA;" OwnerUserId="71" LastEditorUserId="71" LastEditDate="2019-04-16T19:24:20.167" LastActivityDate="2019-04-16T19:24:20.167" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="935" PostTypeId="1" AcceptedAnswerId="937" CreationDate="2017-06-28T14:50:27.330" Score="13" ViewCount="15508" Body="&lt;p&gt;I am looking for a tool, preferably written in C or C++, that can quickly and efficiently count the number of reads and the number of bases in a compressed fastq file. I am currently doing this using &lt;code&gt;zgrep&lt;/code&gt; and &lt;code&gt;awk&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zgrep . foo.fasq.gz |&#xA;     awk 'NR%4==2{c++; l+=length($0)}&#xA;          END{&#xA;                print &quot;Number of reads: &quot;c; &#xA;                print &quot;Number of bases in reads: &quot;l&#xA;              }'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;zgrep .&lt;/code&gt; will print non-blank lines from the input file and the &lt;code&gt;awk 'NR%4==2&lt;/code&gt; will process every 4th line starting with the second (the sequence). &#xA;This works fine, but can take a &lt;em&gt;very&lt;/em&gt; long time when dealing with large files such as WGS data. Is there a tool I can use (on Linux) that will give me these values? Or, if not, I'm also open to suggestions for speeding up the above command.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="73" LastEditDate="2017-06-29T23:50:12.637" LastActivityDate="2021-12-17T12:06:15.390" Title="Fast way to count number of reads and number of bases in a fastq file?" Tags="|ngs|fastq|software-recommendation|benchmarking|" AnswerCount="8" CommentCount="15" ContentLicense="CC BY-SA 3.0" />
  <row Id="936" PostTypeId="2" ParentId="935" CreationDate="2017-06-28T15:22:24.697" Score="7" Body="&lt;p&gt;The following is more than twice as fast; however, &lt;code&gt;wc&lt;/code&gt; counts newline characters as well. We thus need to subtract the line count from the base count (using Bash):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fix_base_count() {&#xA;    local counts=(&lt;span class=&quot;math-container&quot;&gt;$(cat))&#xA;    echo &quot;$&lt;/span&gt;{counts[0]} $((counts[1] - counts[0]))&amp;quot;&#xA;}&#xA;&#xA;gunzip -c &amp;quot;$file&amp;quot; \&#xA;| awk 'NR % 4 == 2' \&#xA;| wc -cl \&#xA;| fix_base_count&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;All the caveats from Simon’s comment apply: this assumes the “simple” FASTQ format, where each record consists of exactly four lines. I think this is true for all files produced by Illumina sequencers and downstream tools.&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2021-12-17T12:06:15.390" LastActivityDate="2021-12-17T12:06:15.390" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="937" PostTypeId="2" ParentId="935" CreationDate="2017-06-28T16:15:06.267" Score="14" Body="&lt;p&gt;It's difficult to get this to go massively quicker I think - as with &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/361/what-is-the-fastest-way-to-calculate-the-number-of-unknown-nucleotides-in-fasta&quot;&gt;this question&lt;/a&gt; working with large gzipped FASTQ files is mostly IO-bound. We could instead focus on making sure we are getting the &lt;em&gt;right&lt;/em&gt; answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;People deride them too often, but this is where a well-written parser is worth it's weight in gold. Heng Li gives us this &lt;a href=&quot;http://lh3lh3.users.sourceforge.net/parsefastq.shtml&quot; rel=&quot;noreferrer&quot;&gt;FASTQ Parser in C&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I downloaded the &lt;a href=&quot;http://lh3lh3.users.sourceforge.net/download/kseq.tar.bz2&quot; rel=&quot;noreferrer&quot;&gt;example tarball&lt;/a&gt; and modified the example code (excuse my C...):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;zlib.h&amp;gt;&#xA;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &quot;kseq.h&quot;&#xA;KSEQ_INIT(gzFile, gzread)&#xA;&#xA;int main(int argc, char *argv[])&#xA;{&#xA;    gzFile fp;&#xA;    kseq_t *seq;&#xA;    int l;&#xA;    if (argc == 1) {&#xA;        fprintf(stderr, &quot;Usage: %s &amp;lt;in.seq&amp;gt;\n&quot;, argv[0]);&#xA;        return 1;&#xA;    }&#xA;    fp = gzopen(argv[1], &quot;r&quot;);&#xA;    seq = kseq_init(fp);&#xA;    int seqcount = 0;&#xA;    long seqlen = 0;&#xA;    while ((l = kseq_read(seq)) &amp;gt;= 0) {&#xA;        seqcount = seqcount + 1;&#xA;        seqlen = seqlen + (long)strlen(seq-&amp;gt;seq.s);&#xA;    }&#xA;    kseq_destroy(seq);&#xA;    gzclose(fp);&#xA;    printf(&quot;Number of sequences: %d\n&quot;, seqcount);&#xA;    printf(&quot;Number of bases in sequences: %ld\n&quot;, seqlen);&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then &lt;code&gt;make&lt;/code&gt; and &lt;code&gt;kseq_test foo.fastq.gz&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my example file (~35m reads of ~75bp) this took:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;real    0m49.670s&#xA;user    0m49.364s&#xA;sys     0m0.304s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Compared with your example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;real    0m43.616s&#xA;user    1m35.060s&#xA;sys     0m5.240s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Konrad's solution (in my hands):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;real    0m39.682s&#xA;user    1m11.900s&#xA;sys     0m5.112s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(By the way, just zcat-ing the data file to /dev/null):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;real    0m38.736s&#xA;user    0m38.356s&#xA;sys     0m0.308s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, I get pretty close in speed, but am likely to be more standards compliant. Also this solution gives you more flexibility with what you can do with the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And my horrible C can almost certainly be optimised.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Same test, with &lt;code&gt;kseq.h&lt;/code&gt; from Github, as suggested in the comments:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My machine is under different load this morning, so I've retested. Wall clock times:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OP: 0m44.813s&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Konrad: 0m40.061s&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;zcat &amp;gt; /dev/null&lt;/code&gt;: 0m34.508s&lt;/p&gt;&#xA;&#xA;&lt;p&gt;kseq.h (Github): 0m32.909s&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So most recent version of &lt;code&gt;kseq.h&lt;/code&gt; is faster than simply zcat-ing the file (consistently in my tests...).&lt;/p&gt;&#xA;" OwnerUserId="194" LastEditorUserId="298" LastEditDate="2017-06-29T09:28:15.997" LastActivityDate="2017-06-29T09:28:15.997" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="938" PostTypeId="2" ParentId="895" CreationDate="2017-06-28T16:15:13.173" Score="4" Body="&lt;p&gt;You can indeed get this information from the UCSC Table Browser. Select knownGene as your primary table, make a filter, add knownCanonical to as a linked table to filter on, then in the free-form query section add &quot;1&quot; without the quotes. Then click submit and select Bed output, where you can choose &quot;exons plus&quot; as an output option. This will lead to output like the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    17368   17436   uc031tla.1_exon_0_0_chr1_17369_r    0   -&#xA;chr1    29553   30039   uc057aty.1_exon_0_0_chr1_29554_f    0   +&#xA;chr1    30563   30667   uc057aty.1_exon_1_0_chr1_30564_f    0   +&#xA;chr1    30975   31097   uc057aty.1_exon_2_0_chr1_30976_f    0   +&#xA;chr1    30365   30503   uc031tlb.1_exon_0_0_chr1_30366_f    0   +&#xA;chr1    34553   35174   uc001aak.4_exon_0_0_chr1_34554_r    0   -&#xA;chr1    35276   35481   uc001aak.4_exon_1_0_chr1_35277_r    0   -&#xA;chr1    35720   36081   uc001aak.4_exon_2_0_chr1_35721_r    0   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For more step-by-step information, please check the answer from the mailing list archives here (in particular the Exon Method 1 section):&#xA;&lt;a href=&quot;https://groups.google.com/a/soe.ucsc.edu/d/msg/genome/BJ-6DlaZNCY/grgGIpuJAwAJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://groups.google.com/a/soe.ucsc.edu/d/msg/genome/BJ-6DlaZNCY/grgGIpuJAwAJ&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note that UCSC provides several mailing lists for support, if you have futher questions please send them there:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;General questions: genome@soe.ucsc.edu &lt;/li&gt;&#xA;&lt;li&gt;Questions involving private&#xA;data: genome-www@soe.ucsc.edu &lt;/li&gt;&#xA;&lt;li&gt;Questions involving mirror sites:&#xA;genome-mirror@ose.ucsc.edu&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1000" LastActivityDate="2017-06-28T16:15:13.173" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="939" PostTypeId="1" CreationDate="2017-06-28T17:56:42.243" Score="7" ViewCount="1187" Body="&lt;p&gt;I have a matrix of gene counts which I'm going to use as input for DESeq.  Right now, each gene is labeled by its Ensemble transcript ID, but I'd like to convert these to their HGNC symbols before I input them into DESeq for analysis. I'm attempting to do this conversion using biomaRt, however, this is my first time using the program, and I'm running into issues.  Here is my current code:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-R prettyprint-override&quot;&gt;&lt;code&gt;library(&quot;biomaRt&quot;)&#xA;mart &amp;lt;- useMart(biomart = &quot;ENSEMBL_MART_ENSEMBL&quot;, dataset = &quot;mmusculus_gene_ensembl&quot;)&#xA;transcript.ids &amp;lt;- rownames(txi.kallisto$counts)&#xA;hgnc_symbols &amp;lt;- getBM(attributes = &quot;hgnc_symbol&quot;, filters = &quot;ensembl_transcript_id&quot;, values = transcript.ids, mart = mart)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Printing hgnc_symbols shows that it isn't what I want:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-R prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt;ghnc_symbols&#xA;[1] hgnc_symbol&#xA;&amp;lt;0 rows&amp;gt; (or 0-length row.names)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Lastly, here is what the input transcript IDs look like:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-R prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; head(rownames(txi.kallisto$counts))&#xA;[1] &quot;ENSMUST00000178862.1&quot; &quot;ENSMUST00000178537.1&quot; &quot;ENSMUST00000196221.1&quot;&#xA;[4] &quot;ENSMUST00000179664.1&quot; &quot;ENSMUST00000177564.1&quot; &quot;ENSMUST00000179520.1&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="506" LastEditorUserId="298" LastEditDate="2017-06-28T21:40:49.913" LastActivityDate="2017-06-29T07:18:05.463" Title="Trouble using biomaRt to retrieve hgnc symbols from Ensembl transcript ids" Tags="|r|bioconductor|deseq2|biomart|ensembl|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="940" PostTypeId="2" ParentId="935" CreationDate="2017-06-28T20:00:12.073" Score="5" Body="&lt;p&gt;I get fairly quick results with my &lt;a href=&quot;https://github.com/gringer/bioinfscripts/blob/master/fastx-fetch.pl&quot; rel=&quot;noreferrer&quot;&gt;fastx-length.pl&lt;/a&gt; script, with the added bonus of being able to handle multi-line FASTQ files and displaying additional read-length QC statistics:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time zcat albacored_all.fastq.gz | /bioinf/scripts/fastx-length.pl &amp;gt; /dev/null&#xA;Total sequences: 301135&#xA;Total length: 283.902419 Mb&#xA;Longest sequence: 5.601 kb&#xA;Shortest sequence: 6 b&#xA;Mean Length: 942 b&#xA;Median Length: 999 b&#xA;N50: 111835 sequences; L50: 1.103 kb&#xA;N90: 245243 sequences; L90: 608 b&#xA;&#xA;real    0m8,802s&#xA;user    0m16,584s&#xA;sys 0m0,260s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Versus the script you have provided:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zcat albacored_all.fastq.gz | awk 'NR%4==2{c++; l+=length($0)}&#xA;          END{&#xA;                print &quot;Number of reads: &quot;c; &#xA;                print &quot;Number of bases in reads: &quot;l&#xA;              }'&#xA;Number of reads: 301135&#xA;Number of bases in reads: 283902419&#xA;&#xA;real    0m8,382s&#xA;user    0m10,216s&#xA;sys 0m0,332s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Cat to &lt;code&gt;/dev/null&lt;/code&gt; for comparison:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time zcat albacored_all.fastq.gz &amp;gt; /dev/null&#xA;&#xA;real    0m7,877s&#xA;user    0m7,856s&#xA;sys 0m0,020s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I suspect that something using &lt;a href=&quot;https://github.com/lh3/bioawk&quot; rel=&quot;noreferrer&quot;&gt;bioawk&lt;/a&gt; might be a bit faster (and similarly FASTQ-compliant).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-28T20:00:12.073" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="941" PostTypeId="2" ParentId="939" CreationDate="2017-06-28T21:16:04.060" Score="8" Body="&lt;p&gt;You need to specify the number without the version. Instead of &quot;ENSMUST00000178862.1&quot; just &quot;ENSMUST00000178862&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do this with one more line:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-R prettyprint-override&quot;&gt;&lt;code&gt;g &amp;lt;- gsub(&quot;\\..*&quot;, &quot;&quot;, rownames(txi.kallisto$counts))&#xA;(hgnc_symbols &amp;lt;- getBM(attributes = c(&quot;hgnc_symbol&quot;, &quot;chromosome_name&quot;, &quot;ensembl_transcript_id&quot;), filters = &quot;ensembl_transcript_id&quot;, values = g, mart = mart))&#xA;##  hgnc_symbol chromosome_name ensembl_transcript_id&#xA;##1          NA              14    ENSMUST00000177564&#xA;##2          NA               6    ENSMUST00000178537&#xA;##3          NA               6    ENSMUST00000178862&#xA;##4          NA              12    ENSMUST00000179520&#xA;##5          NA              14    ENSMUST00000179664&#xA;##6          NA              14    ENSMUST00000196221&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With these transcripts it doesn't seem that there is an equivalent hgnc_symbol. But is recognized, otherwise they are not recognized:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-R prettyprint-override&quot;&gt;&lt;code&gt;(hgnc_symbols &amp;lt;- getBM(attributes = c(&quot;hgnc_symbol&quot;, &quot;chromosome_name&quot;, &quot;ensembl_transcript_id&quot;), filters = &quot;ensembl_transcript_id&quot;, values = rownames(txi.kallisto$counts), mart = mart))&#xA;## [1] hgnc_symbol           chromosome_name       ensembl_transcript_id&#xA;##&amp;lt;0 rows&amp;gt; (or 0-length row.names)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However you might be interested in doing something like &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/vignettes/biomaRt/inst/doc/biomaRt.html#given-the-human-gene-tp53-retrieve-the-human-chromosomal-location-of-this-gene-and-also-retrieve-the-chromosomal-location-and-refseq-id-of-its-homolog-in-mouse.&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-R prettyprint-override&quot;&gt;&lt;code&gt;human = useMart(&quot;ensembl&quot;, dataset = &quot;hsapiens_gene_ensembl&quot;)&#xA;mouse = useMart(&quot;ensembl&quot;, dataset = &quot;mmusculus_gene_ensembl&quot;)&#xA;getLDS(attributes = &quot;ensembl_transcript_id&quot;,&#xA;       filters = &quot;ensembl_transcript_id&quot;, values = g,mart = mouse,&#xA;      attributesL = &quot;hgnc_symbol&quot;, martL = human)&#xA;## The query to the BioMart webservice returned an invalid result: the number of &#xA;## columns in the result table does not equal the number of &#xA;## attributes in the query. Please report this to the mailing list.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-06-29T07:18:05.463" LastActivityDate="2017-06-29T07:18:05.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="942" PostTypeId="1" CreationDate="2017-06-29T08:42:12.243" Score="6" ViewCount="718" Body="&lt;p&gt;I have done de novo assembly of pair end raw read sequences and resulted transcripts sequences were separated based on coding potential into two categories: long non-coding RNA transcripts and coding RNA transcripts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find how many of long non coding can make hairpin loop structure and how can I annotate these miRNA which are generated from long non coding RNA in plants.&lt;/p&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="37" LastEditDate="2017-06-30T12:04:12.420" LastActivityDate="2017-06-30T12:04:12.420" Title="How to find hairpin loop structure in a large set of long non coding RNA transcripts" Tags="|bioconductor|rna-structure|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="943" PostTypeId="1" AcceptedAnswerId="946" CreationDate="2017-06-29T08:58:04.413" Score="6" ViewCount="379" Body="&lt;p&gt;I have the following mwe for filtering a Swissprot file based on a certain feature, in this case, transmembrane regions. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;records=[]&#xA;for record in SeqIO.parse(&quot;Input.txt&quot;, &quot;swiss&quot;):&#xA;    transmembrane_protein=False&#xA;    print record.id&#xA;    for i, feature in enumerate(record.features):&#xA;        if feature.type == &quot;TRANSMEM&quot;:&#xA;            transmembrane_protein=True&#xA;    if transmembrane_protein==True:&#xA;        records.append(record)&#xA;SeqIO.write(records, &quot;Output.txt&quot;, &quot;swiss&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The script works when &lt;code&gt;SeqIO.write(records, &quot;Output.txt&quot;, &quot;swiss&quot;)&lt;/code&gt; becomes &lt;code&gt;SeqIO.write(records, &quot;Output.txt&quot;, &quot;fasta&quot;)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, such a method is not yet supported.  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ValueError: Reading format 'swiss' is supported, but not writing&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;From the docs, I see that &lt;a href=&quot;http://biopython.org/DIST/docs/api/Bio.SeqIO-module.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;writing is not supported for swiss&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Note that while Bio.SeqIO can read all the above file formats, it cannot write to all of them.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Is there any unofficial way of using biopython/python to write swissprot files from parsed swissprot files? &lt;/p&gt;&#xA;" OwnerUserId="747" LastEditorUserId="747" LastEditDate="2017-06-30T05:31:46.470" LastActivityDate="2017-06-30T05:31:46.470" Title="Is there any way of using biopython to write Swissprot files?" Tags="|database|python|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="944" PostTypeId="2" ParentId="935" CreationDate="2017-06-29T09:05:43.780" Score="2" Body="&lt;p&gt;I hava implemented &lt;a href=&quot;https://github.com/jameslz/seqtk_utils/blob/master/seqtk_counts&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqtk_counts&lt;/a&gt; using kseq.h from  &lt;a href=&quot;https://github.com/attractivechaos/klib&quot; rel=&quot;nofollow noreferrer&quot;&gt;klib&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just a few line of Codes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;zlib.h&amp;gt;&#xA;#include &quot;kseq.h&quot;&#xA;&#xA;KSEQ_INIT(gzFile, gzread)&#xA;&#xA;int main(int argc, char *argv[]){&#xA;&#xA;gzFile fp;&#xA;kseq_t *seq;&#xA;&#xA;int l = 0;&#xA;&#xA;int64_t total = 0;&#xA;int64_t lines = 0;&#xA;&#xA;if (argc == 1) {&#xA;    fprintf(stderr, &quot;Usage: %s &amp;lt;fastq&amp;gt; &amp;lt;sample&amp;gt;\n&quot;, argv[0]);&#xA;&#xA;    return 1;&#xA;}&#xA;&#xA;fp = strcmp(argv[1], &quot;-&quot;)? gzopen(argv[1], &quot;r&quot;) : gzdopen(fileno(stdin), &quot;r&quot;);&#xA;seq = kseq_init(fp);&#xA;&#xA;while ((l = kseq_read(seq)) &amp;gt;= 0){&#xA;    total += seq-&amp;gt;seq.l;&#xA;    lines += 1;&#xA;}&#xA;&#xA;printf(&quot;%s\t%lld\t%lld\n&quot;, argv[2] ,(long long)lines, (long long)total);&#xA;kseq_destroy(seq);&#xA;gzclose(fp);&#xA;return 0;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compile it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gcc  -O2  seqtk_counts.c  -o  seqtk_counts  -Iklib  -lz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Usage:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seqtk_counts foo.fasq.gz foo &#xA;or&#xA;cat foo.fasq.gz | seqtk_counts  - foo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="652" LastEditorUserId="652" LastEditDate="2017-06-29T09:33:47.703" LastActivityDate="2017-06-29T09:33:47.703" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="946" PostTypeId="2" ParentId="943" CreationDate="2017-06-29T11:14:06.497" Score="6" Body="&lt;p&gt;Using &lt;code&gt;SeqIO.index&lt;/code&gt; rather than &lt;code&gt;SeqIO.parse&lt;/code&gt; lets you read all the records into a &lt;code&gt;dict&lt;/code&gt;, from which you can then extract the raw entry:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;record_dict = SeqIO.index('Input.txt', 'swiss')&#xA;for key in record_dict:&#xA;    print(record_dict.get_raw(key).decode())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now you should be able to apply your test for a transmembrane protein to each entry, and write out only the ones you want to keep.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;record_dict = SeqIO.index('Input.txt', 'swiss')&#xA;out = open('Output.txt', 'wb')&#xA;for key in record_dict:&#xA;    record = record_dict[key]&#xA;    for i, feature in enumerate(record.features):&#xA;    if feature.type == &quot;TRANSMEM&quot;:&#xA;        out.write(record_dict.get_raw(key).decode())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="194" LastActivityDate="2017-06-29T11:14:06.497" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="947" PostTypeId="2" ParentId="943" CreationDate="2017-06-29T11:26:35.603" Score="2" Body="&lt;p&gt;I don't know how to do this with biopython, but it's simple enough in Perl using &lt;a href=&quot;http://swissknife.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Swisskinfe&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env perl&#xA;use strict;&#xA;use warnings;&#xA;use SWISS::Entry;&#xA;&#xA;my $flat=$ARGV[0]||die(&quot;Need a flat file\n&quot;);&#xA;&#xA;## Change the line termination string so we read an &#xA;## entire entry at a time&#xA;local $/ = &quot;\n//\n&quot;;&#xA;&#xA;my $fh;&#xA;## Deal with compressed or uncompressed files&#xA;if($flat=~/\.gz$/){&#xA;    open($fh,&quot;zcat $flat |&quot;)|| die(&quot;cannot open flat file $flat : $! \n&quot;);&#xA;}&#xA;else{&#xA;    open($fh,&quot;$flat&quot;)|| die(&quot;cannot open flat file $flat : $!: $@\n&quot;);&#xA;}&#xA;## Parse the file. &#xA;entry: while (my $current_entry = &amp;lt;$fh&amp;gt;) {&#xA;    ## Read the current entry as an &quot;entry&quot; object&#xA;    my $entry = SWISS::Entry-&amp;gt;fromText($current_entry);&#xA;    ## Iterate over the entry's features&#xA;    foreach my $feature ($entry-&amp;gt;FTs-&amp;gt;elements()) {&#xA;        ## If any of the features are 'TRANSMEM', print the entry&#xA;        if ($feature-&amp;gt;[0] eq 'TRANSMEM') {&#xA;            print &quot;$current_entry\n&quot;;&#xA;            ## Skip to the next entry&#xA;            next entry;&#xA;        }&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You need to install the Swissknife module first:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Download Swissknife_1.73.tar.gz from &lt;a href=&quot;https://sourceforge.net/projects/swissknife/files/latest/download&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (SourceForge link).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Install&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tar xvzf Swissknife.tar.gz&#xA;cd Swissknife_1.73/&#xA;perl Makefile.PL&#xA;sudo make install&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now, make the script above executable and run with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;foo.pl file.flat&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-06-29T11:33:38.093" LastActivityDate="2017-06-29T11:33:38.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="949" PostTypeId="1" CreationDate="2017-06-29T15:45:09.317" Score="8" ViewCount="2685" Body="&lt;p&gt;I have the genotyped data from impute2 output in .gen format (imputed to 1000G P3). The file has genotype posterior probabilities (GP:3 values per variant). I have converted .gen to .vcf using qctools and the .vcf file has GT:GP format. I need to convert the .vcf file with GT:GP format to GT:DS. Genotype dosages are recommended for use in qtltools/fastqtl analysis. However, I cannot find a tool that would keep the .vcf format and convert GP to DS. Any help much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="1012" LastActivityDate="2017-07-24T19:52:43.587" Title="How to convert the .vcf (imputed) file with GT:GP format to GT:DS?" Tags="|file-formats|vcf|format-conversion|impute2|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="950" PostTypeId="1" CreationDate="2017-06-29T16:20:35.530" Score="7" ViewCount="69" Body="&lt;p&gt;This is an interesting problem - I was wondering if anyone has a creative solution.&lt;/p&gt;&#xA;&lt;p&gt;So I have a vector of vertices representing atoms in a protein, as well as 6 variables containing the absolute minimum/maximum bound of the set at each direction.&lt;/p&gt;&#xA;&lt;p&gt;I need to build a grid that surrounds the protein/vertices in a homogeneous layer.&lt;/p&gt;&#xA;&lt;p&gt;Right now I just loop through every square of the grid bound by the minimum-maximum values and then loop through every vertex to calculate the minimum distance between my point and every atom in the protein, and see if that point is at an acceptable distance. This process is quite expensive.&lt;/p&gt;&#xA;&lt;p&gt;Do you guys have any creative answers as to how minimize the time?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&lt;p&gt;P.S. the list is ordered by phyiical linkage of amino acids in a  protein, which means it does not always have a relationship to the location of the amino acid in protein space.&lt;/p&gt;&#xA;" OwnerUserId="1016" LastEditorUserId="10182" LastEditDate="2021-05-11T09:59:28.667" LastActivityDate="2021-05-11T09:59:28.667" Title="Minimizing particle-flow grid mapping time of a protein surface using spherical-coordinates and differentiation encoding" Tags="|protein-structure|proteins|3d-structure|" AnswerCount="0" CommentCount="1" FavoriteCount="0" ClosedDate="2021-05-06T09:57:06.797" ContentLicense="CC BY-SA 4.0" />
  <row Id="951" PostTypeId="1" CreationDate="2017-06-29T16:24:27.427" Score="10" ViewCount="710" Body="&lt;p&gt;I have 12 human gut microbiome WGS Nextseq reads (151 bp paired end). What will be an effective strategy to assemble a metagenome?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let us say I have already filtered the fastq for quality, adapter sequence and host contamination (human, in this case). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Should I concatenate all the R1 reads as one single R1 read and one Single R2 read?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat Sample[1..12].R1 &amp;gt; Single_R1.fastq&#xA;cat Sample[1..12].R2 &amp;gt; Single_R2.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then use Diginorm to normalize the Single_R1.fastq and Single_R2.fastq. Subsequently, feed these fastq files into any metagenome assembler such as Megahit, MetaSPAdes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normalize the output by using CD-HIT or similar tool to remove duplicates and filter by contig length. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;OR &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Perform metagenome assembly for each of the samples individually after applying filtering, removing adapters and host contamination. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R1=(*_R1_001.filtered.fastq)&#xA;R2=(*_R2_001.filtered.fastq)&#xA;for ((i=0;i&amp;lt;=${#R1[@]};i++)); do  &#xA;  /bin/metagenome-assembler -1 &quot;${R1[i]}&quot; -2 &quot;${R2[i]}&quot; -o ${R1[i]%.*}.contigs.fa;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Followed by combining all the contigs.fa into one mega_contigs.fa &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat *.contigs.fa &amp;gt; Mega_contigs.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and use CD-HIT or similar tool to remove duplicates. &lt;/p&gt;&#xA;" OwnerUserId="626" LastEditorUserId="298" LastEditDate="2017-06-29T18:03:20.433" LastActivityDate="2018-03-01T07:24:44.267" Title="Pooling data in metagenome assembly" Tags="|assembly|metagenome|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="952" PostTypeId="2" ParentId="949" CreationDate="2017-06-29T16:57:57.123" Score="5" Body="&lt;p&gt;You can do this in &lt;a href=&quot;https://hail.is/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hail&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the rough code to do it (0.1 versions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Setup:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from hail import *&#xA;hc = HailContext()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Import the .gen file. VCF works too:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dataset = hc.import_gen(&#xA;    'src/test/resources/example.gen', &#xA;    'src/test/resources/example.sample')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Remap the genotype schema and export to VCF:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dataset.annotate_genotypes_expr('g = {GT: g.call(), DS: g.dosage()}')\&#xA;    .export_vcf('/tmp/out.vcf.bgz')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Take a look at the &lt;a href=&quot;https://hail.is/hail/getting_started.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;getting started page&lt;/a&gt; if you want to try it out!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should note that you may be able to do QTL analyses in Hail, depending on the method you want to use. See &lt;a href=&quot;http://discuss.hail.is/t/fast-linear-regression-for-multiple-phenotypes-eqtls/190&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1017" LastEditorUserId="1017" LastEditDate="2017-06-30T13:12:31.343" LastActivityDate="2017-06-30T13:12:31.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="953" PostTypeId="2" ParentId="942" CreationDate="2017-06-29T19:20:15.247" Score="4" Body="&lt;p&gt;For the first question: &quot;how many of long non coding can make hairpin loop structure?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest thing to would be to run your sequences through an RNA secondary structure prediction tool. There's a few tools for doing this but the most commonly used are &lt;a href=&quot;http://rna.tbi.univie.ac.at/cgi-bin/RNAWebSuite/RNAfold.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNAfold&lt;/a&gt; from the &lt;a href=&quot;https://www.tbi.univie.ac.at/RNA/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ViennaRNA package&lt;/a&gt; and &lt;a href=&quot;http://unafold.rna.albany.edu/?q=mfold&quot; rel=&quot;nofollow noreferrer&quot;&gt;mfold&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will give you an output that looks like this, where the parentheses indicate which bases are paired with each other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&#xA;GGGCUAUUAGCUCAGUUGGUUAGAGCGCACCCCUGAUAAGGGUGAGGUCGCUGAUUCGAAUUCAGCAUAGCCCA&#xA;(((((((..((((.........)))).(((((.......))))).....(((((.......)))))))))))).&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To see if there's hairpins, you'll need to annotate the elements within it. Take a look at &lt;a href=&quot;https://www.biostars.org/p/4300/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this question on biostars&lt;/a&gt; for an example of how to do this. In short, you can use the &lt;a href=&quot;https://github.com/ViennaRNA/forgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;forgi&lt;/a&gt; library to produce an annotation like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&#xA;(((((((..((((.........)))).(((((.......))))).....(((((.......)))))))))))).&#xA;sssssssmmsssshhhhhhhhhssssmssssshhhhhhhsssssmmmmmssssshhhhhhhsssssssssssse&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there's an &lt;code&gt;h&lt;/code&gt; in the output annotation, you have a hairpin. The remainder of your question about annotating miRNAs will be left to somebody else.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2017-06-29T19:20:15.247" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="954" PostTypeId="1" AcceptedAnswerId="968" CreationDate="2017-06-29T21:19:48.177" Score="9" ViewCount="4126" Body="&lt;p&gt;I am on a Mac using UNIX. I am trying to use the kallisto quant command on all files in a directory (instead of manually entering them). Because I'm running the analysis against the same index file, I first enter the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./kallisto index -i --index --make-unique index.fa &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This successfully creates an index file. Then, I tried this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for file in *.fasta; do kallisto quant --single -l 200 -s 0.1 -o $file-aligned &#xA;&quot;$file&quot;; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;While the $file-aligned folders are created, they're empty. I get an error that states that the index file is missing. So I assumed I would need to specify the index file in the command line. I then tried this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for file in *.fasta; do kallisto quant -i index.fa --single -l 200 -s 0.1 -o&#xA;$file-aligned  &quot;$file&quot;; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get an error that says &quot;Error: incompatible indices. Found version 0, expected version 10. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have now also tried the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for file in *.fasta; do ./kallisto quant -i index --single - 200 -s 0.01 -o &#xA;&quot;${file}-aligned&quot; &quot;${file}&quot;; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but I get an error saying the index file is not found.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I am now running &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;or file in *.fasta; do ./kallisto quant -i index --single -l 200 -s 0.1 -o $file-aligned &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&quot;$file&quot;; done&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This sort of works. The index file error messages are gone, but it runs quant on the same file 10 times instead of each file once. If I run echo $file it spits back the file that my command is running repeatedly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I close terminal, start a new session and run the same command, I get&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error: file not found &quot;&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="779" LastEditorUserId="779" LastEditDate="2017-06-30T18:52:35.010" LastActivityDate="2017-07-03T20:50:27.253" Title="Run kallisto iteratively across many samples" Tags="|rna-seq|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="955" PostTypeId="2" ParentId="954" CreationDate="2017-06-29T23:43:58.127" Score="5" Body="&lt;p&gt;For generating the index, you should be using one of &lt;code&gt;-i&lt;/code&gt; or &lt;code&gt;--index&lt;/code&gt;, but not both:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./kallisto index --make-unique -i index.fa.idx index.fa &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You need to point kallisto to the generated index file (usually has a &lt;code&gt;.idx&lt;/code&gt; extension), rather than the fasta file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for file in *.fasta;&#xA;  do kallisto quant -i index.fa.idx --single -l 200 -s 0.1 -o &quot;${file}-aligned&quot; &quot;${file}&quot;;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-06-30T16:13:19.153" LastActivityDate="2017-06-30T16:13:19.153" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="956" PostTypeId="1" CreationDate="2017-06-30T10:37:39.143" Score="8" ViewCount="3387" Body="&lt;p&gt;FCS is a patented data format used for storing flow cytometry data. The most recent version is FCS3.1. There is some &lt;a href=&quot;http://isac-net.org/PDFS/90/9090600d-19be-460d-83fc-f8a8b004e0f9.pdf&quot; rel=&quot;noreferrer&quot;&gt;documentation&lt;/a&gt; on the format, but there is no information on how to read these files. There are some R packages and a MATLAB code to read an FCS file, but I am looking for standard libraries developed either by the FCS consortium or any other group. I also wish to know if FCS is a subset of an existing standard data format that can be read by a standard library using any programming language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I would want to convert these files to an easily readable format like HDF5. &lt;/p&gt;&#xA;" OwnerUserId="345" LastEditorUserId="345" LastEditDate="2017-06-30T10:41:19.763" LastActivityDate="2019-02-03T02:49:28.850" Title="How can I read FCS files using open source libraries?" Tags="|file-formats|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="957" PostTypeId="2" ParentId="956" CreationDate="2017-06-30T11:56:45.483" Score="3" Body="&lt;p&gt;R/Bioconductor has a number of different flow cytometry processing packages. One place to start for looking at cytometry data from a high level would be &lt;a href=&quot;http://opencyto.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;openCyto&lt;/a&gt; (or its &lt;a href=&quot;http://opencyto.org/openCytoVignette.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;vignette&lt;/a&gt;), which is a large set of tools for basic extraction and analysis of FCS files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have looked in the past at the FCS files as an R structure using &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/flowCore.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;flowCore&lt;/a&gt;. Loading a single FCS file is fairly straightforward and follows a familiar R pattern:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;file.name &amp;lt;- &quot;/dir/file.fcs&quot;&#xA;x &amp;lt;- read.FCS(file.name, transformation=FALSE)&#xA;summary(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Asking about converting FCS files into an &quot;easily readable format like HDF5&quot; doesn't seem like the right question. &lt;a href=&quot;https://en.wikipedia.org/wiki/Hierarchical_Data_Format#HDF5&quot; rel=&quot;nofollow noreferrer&quot;&gt;HDF5&lt;/a&gt; is a container format and shares a lot of similarity to file systems. I've found it best to keep FCS files as they are, as it is a compact, standardised binary format.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-06-30T11:56:45.483" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="958" PostTypeId="2" ParentId="949" CreationDate="2017-06-30T12:44:22.447" Score="0" Body="&lt;p&gt;There's &lt;a href=&quot;https://samtools.github.io/bcftools/bcftools-man.html#plugin&quot; rel=&quot;nofollow noreferrer&quot;&gt;the dosage plugin for bcftools&lt;/a&gt;, but it only outputs tab separated values. It would not be too hard to extend the plugin to output a VCF with the DS tag instead, but it has not been done yet.  There's a good chance the bcftools devs would respond to a &lt;a href=&quot;https://github.com/samtools/bcftools/issues/new&quot; rel=&quot;nofollow noreferrer&quot;&gt;feature request&lt;/a&gt;...&lt;/p&gt;&#xA;&lt;p&gt;In any case, this code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;curl https://raw.githubusercontent.com/samtools/bcftools/develop/test/convert.vcf &amp;gt; convert.vcf&#xA;bcftools +dosage convert.vcf &amp;gt; output.tsv&#xA;head -2 output.tsv &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;has the output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#[1]CHROM       [2]POS  [3]REF  [4]ALT  [5]NA00001      [6]NA00002      [7]NA00003      [8]NA00004      [9]NA00005      [10]NA00006     [11]NA00007     [12]NA00008    [13]NA00009     [14]NA00010&#xA;X       2698560 G       A       0.1     0.0     0.1     0.2     0.3     0.2     0.2     0.2     0.2     0.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is using bcftools version 1.3.1.&lt;/p&gt;&#xA;&lt;p&gt;Here is an excerpt from the bcftools manual for the dosage plugin:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;dosage&lt;/p&gt;&#xA;&lt;p&gt;print genotype dosage. By default the plugin searches for PL, GL and GT, in that order.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="492" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-06-30T12:44:22.447" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="959" PostTypeId="2" ParentId="956" CreationDate="2017-06-30T12:51:15.207" Score="5" Body="&lt;p&gt;A few years ago, I wrote a python script to convert FCS files into tab-separated format. It was far from handling all the possibilities that the format description offers, but at least it worked for some of the files produced on one of our machine: &lt;a href=&quot;http://www.igh.cnrs.fr/equip/Seitz/en_equipe-programmes.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.igh.cnrs.fr/equip/Seitz/en_equipe-programmes.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The format documentation I found enabled decoding (see section 3 of the pdf you mention), but it requires reading data in binary mode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general idea of this format (and, I guess, many other binary formats), is that there is a header zone at the beginning of the file with a defined number of fields encoding numbers indicating how the rest of the file is structured. So a first phase is to parse this header, following the description given in the documentation of the format. The information extracted from the header tells where to find the data and how it is encoded, still according to rules described in the format documentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case this may be useful, and for the record, here is the code from the above-mentioned script (after stripping comments, some of which are merely copied from the format documentation, and adding a few ones):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python&#xA;&quot;&quot;&quot;This script tries to read FCS flow cytometry data.&#xA;Format parsing inspired by information found here:&#xA;http://isac-net.org/Resources-for-Cytometrists/Data-Standards/Data-File-Standards/Flow-Cytometry-Data-File-Format-Standards.aspx&#xA;&quot;&quot;&quot;&#xA;&#xA;import re&#xA;# To decode binary-encoded data&#xA;import struct&#xA;import sys&#xA;&#xA;class Parameter(object):&#xA;    &quot;&quot;&quot;This object represents one of the parameter types that are present in a DATA segment of a FCS file.&quot;&quot;&quot;&#xA;    __slots__ = (&quot;p_name&quot;, &quot;p_bits&quot;, &quot;p_range&quot;, &quot;p_ampl&quot;, &quot;parser&quot;)&#xA;    def __init__(self, p_name, p_bits, p_range, p_ampl):&#xA;        self.p_name = p_name&#xA;        self.p_bits = p_bits&#xA;        self.p_range = p_range&#xA;        self.p_ampl = p_ampl&#xA;        # Function for parsing a value of the parameter in the data segment&#xA;        self.parser = None&#xA;&#xA;##############################################&#xA;# Here starts the parsing of the header part #&#xA;# which tells where the other parts are.     #&#xA;##############################################&#xA;&#xA;f = open(sys.argv[1], &quot;rb&quot;)&#xA;# The format name is encoded in 6 letters&#xA;# An ASCII letter is coded with one octet&#xA;file_format = &quot;&quot;.join([f.read(1) for __ in range(6)])&#xA;sys.stdout.write(&quot;Format: %s\n&quot; % file_format)&#xA;# The format descriptions reserves 4 octets that we skip&#xA;skip = f.read(4)&#xA;# 8 octet chunks encode the start and end positions&#xA;# of different parts of the data&#xA;text_start = int(f.read(8).strip(&quot; &quot;))&#xA;text_end = int(f.read(8).strip(&quot; &quot;))&#xA;data_start = int(f.read(8).strip(&quot; &quot;))&#xA;data_end = int(f.read(8).strip(&quot; &quot;))&#xA;analysis_start = int(f.read(8).strip(&quot; &quot;))&#xA;analysis_end = int(f.read(8).strip(&quot; &quot;))&#xA;&#xA;if (analysis_start and analysis_end):&#xA;    sys.stderr.write(&quot;Cannot deal with ANALYSIS segment of an FCS file.\n&quot;)&#xA;&#xA;####################################################&#xA;# Here starts the parsing of the &quot;TEXT&quot; portion    #&#xA;# which describes how the data proper is organized #&#xA;####################################################&#xA;f.seek(text_start)&#xA;# The first character in the primary TEXT segment is the ASCII delimiter character.&#xA;sep = f.read(1)&#xA;if sep not in [&quot;_&quot;, &quot;@&quot;]:&#xA;    alt_sep = &quot;_@_&quot;&#xA;elif sep not in [&quot;_&quot;, &quot;|&quot;]:&#xA;    alt_sep = &quot;_|_&quot;&#xA;else:&#xA;    assert sep not in [&quot;+&quot;, &quot;|&quot;]&#xA;    alt_sep = &quot;+|+&quot;&#xA;text_segment = f.read(text_end - text_start)&#xA;&#xA;fields = text_segment.split(sep)&#xA;&#xA;info = {}&#xA;&#xA;i = 0&#xA;while i &amp;lt; len(fields) - 1:&#xA;    key = fields[i]&#xA;    i += 1&#xA;    val = fields[i]&#xA;    i += 1&#xA;    # Keywords are case insensitive, they may be written in a file in lower case, upper case, or a&#xA;    # mixture of the two. However, an FCS file reader must ignore keyword case. A keyword value may&#xA;    # be in lower case, upper case or a mixture of the two. Keyword values are case sensitive.&#xA;    info[key.upper()] = val&#xA;print &quot;%s events were detected.&quot; % info[&quot;$TOT&quot;]&#xA;print &quot;Each event is characterized by %s parameters&quot; % info[&quot;$PAR&quot;]&#xA;&#xA;if info[&quot;$NEXTDATA&quot;] != &quot;0&quot;:&#xA;    sys.stderr.write(&quot;Some other data exist in the file but hasn't been parsed.\n&quot;)&#xA;&#xA;# L - List mode. For each event, the value of each parameter is stored in the order in which the&#xA;# parameters are described. The number of bits reserved for parameter 1 is described using the&#xA;# $P1B keyword. There can be only one set of list mode data per data set. The $DATATYPE&#xA;# keyword describes the data format. This is the most versatile mode for the storage of flow&#xA;# cytometry data because mode C and mode U data can be created from mode L data.&#xA;assert info[&quot;$MODE&quot;] == &quot;L&quot;&#xA;&#xA;parameters = []&#xA;&#xA;# indices of the parameters&#xA;p_indices = range(1, int(info[&quot;$PAR&quot;]) + 1)&#xA;for i in p_indices:&#xA;    p_name = info[&quot;$P%dN&quot; % i]&#xA;    p_bits =  info[&quot;$P%dB&quot; % i]&#xA;    p_range = info[&quot;$P%dR&quot; % i]&#xA;    p_ampl =  info[&quot;$P%dE&quot; % i]&#xA;    parameters.append(Parameter(p_name, p_bits, p_range, p_ampl))&#xA;&#xA;sys.stdout.write(&quot;The parameters are:\n%s\n&quot; % &quot;\t&quot;.join([par.p_name for par in parameters]))&#xA;&#xA;# How are 32 bit words organized&#xA;if info[&quot;$BYTEORD&quot;] == &quot;4,3,2,1&quot;:&#xA;    endianness = &quot;&amp;gt;&quot;&#xA;else:&#xA;    endianness = &quot;&amp;lt;&quot;&#xA;    assert info[&quot;$BYTEORD&quot;] == &quot;1,2,3,4&quot;&#xA;&#xA;    # I stripped a long comment which is just a copy of the documentation&#xA;# Type of data:&#xA;if info[&quot;$DATATYPE&quot;] == &quot;I&quot;:&#xA;    for par in parameters:&#xA;        nb_bits = int(par.p_bits)&#xA;        assert nb_bits % 8 == 0&#xA;        nb_bytes = nb_bits / 8&#xA;        # Determine format string for unpacking (see https://docs.python.org/2/library/struct.html)&#xA;        if nb_bytes == 1:&#xA;            c_type = &quot;B&quot; # unsigned char&#xA;        elif nb_bytes == 2:&#xA;            c_type = &quot;H&quot; # unsigned short&#xA;        elif nb_bytes == 4:&#xA;            c_type = &quot;L&quot; # unsigned long&#xA;        elif nb_bytes == 8:&#xA;            c_type = &quot;Q&quot; # unsigned long long&#xA;        else:&#xA;            raise ValueError, &quot;Number of bytes (%d) not valid for an integer (see https://docs.python.org/2/library/struct.html#byte-order-size-and-alignment).&quot; % nb_bytes&#xA;        fmt = &quot;%s%s&quot; % (endianness, c_type)&#xA;        p_range = int(par.p_range)&#xA;        def parser(data):&#xA;            value = struct.unpack(fmt, data.read(nb_bytes))[0]&#xA;            try:&#xA;                assert value &amp;lt; p_range&#xA;            except AssertionError:&#xA;                print &quot;Value %s higher than %d&quot; % (str(value), p_range)&#xA;            return value&#xA;        par.parser = parser&#xA;    pass&#xA;else:&#xA;    raise NotImplementedError, &quot;Only the parsing of integer value has been implemented so far.&quot;&#xA;&#xA;&#xA;out_file = open(sys.argv[2], &quot;w&quot;)&#xA;out_file.write(&quot;#amplification_types\t&quot; + &quot;\t&quot;.join([par.p_ampl for par in parameters]) + &quot;\n&quot;)&#xA;out_file.write(&quot;parameters\t&quot; + &quot;\t&quot;.join([par.p_name for par in parameters]) + &quot;\n&quot;)&#xA;i = 1&#xA;##############################################&#xA;# Here starts the parsing of the data proper #&#xA;##############################################&#xA;f.seek(data_start)&#xA;while f.tell() &amp;lt; data_end:&#xA;    values = []&#xA;    for par in parameters:&#xA;        values.append(par.parser(f))&#xA;    out_file.write(&quot;%d\t&quot; % i + &quot;\t&quot;.join(map(str, values)) + &quot;\n&quot;)&#xA;    i += 1&#xA;out_file.close()&#xA;f.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2017-07-04T07:01:36.560" LastActivityDate="2017-07-04T07:01:36.560" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="960" PostTypeId="1" CreationDate="2017-06-30T13:19:17.627" Score="8" ViewCount="1983" Body="&lt;p&gt;I have some protein sequences and I want to build a position-specific scoring matrix (PSSM) for them and then upload this PSSM to &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&amp;amp;PAGE=Proteins&amp;amp;PROGRAM=blastp&amp;amp;RUN_PSIBLAST=on&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI PSI-BLAST&lt;/a&gt;. I used &lt;a href=&quot;http://fasta.bioch.virginia.edu/fasta_www2/chaps.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;CHAPS&lt;/a&gt; program for this pupose but uploading the output PSSM gave me an error in NCBI PSI-BLAST. Do you know any tool or webserver for getting PSSMs for a group of protein sequences which then can work in NCBI PSI-BLAST?&#xA;This is a part of my PSSM file: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PssmWithParameters ::= {&#xA;pssm {&#xA;    isProtein TRUE ,&#xA;    numRows 28 ,&#xA;    numColumns 131 ,&#xA;    byRow FALSE ,&#xA;    query&#xA;    seq {&#xA;        id {&#xA;            other {&#xA;                accession &quot;WP_000208753&quot; } } ,&#xA;        inst {&#xA;            repr raw ,&#xA;            mol aa ,&#xA;            length 131 ,&#xA;            seq-data&#xA;            ncbieaa &quot;MTTKRKPYVRPMTSTWWKKLPFYRFYMLREGTAVPAVWFSIELIFGLFALKNGPEAW&#xA;AGFIDFLQNPVIVIINLITLAAALLHTKTWFELAPKAANIIVKDEKMGPEPIIKSLWAVTVVATIVILFVALYW&quot; } } ,&#xA;    intermediateData {&#xA;        freqRatios {&#xA;            { 0, 10, 0 } ,&#xA;            { 564418841, 10, -10 } ,&#xA;            { 0, 10, 0 } ,&#xA;            { 11768571, 10, -9 } ,&#xA;            { 185838265, 10, -10 } ,&#xA;            { 31496547, 10, -9 } ,&#xA;            { 3872857, 10, -8 } ,&#xA;            { 291750464, 10, -10 } ,&#xA;            { 128450763, 10, -10 } ,&#xA;            { 759856221, 10, -10 } ,&#xA;            { 359173937, 10, -10 } ,&#xA;            { 179865517, 10, -9 } ,&#xA;            { 14537895, 10, -8 } ,&#xA;            { 212921456, 10, -10 } ,&#xA;            { 220554141, 10, -10 } ,&#xA;            { 368516324, 10, -10 } ,&#xA;            { 319343525, 10, -10 } ,&#xA;            { 426173953, 10, -10 } ,&#xA;            { 463659523, 10, -10 } ,&#xA;            { 817322186, 10, -10 } ,&#xA;            { 811693964, 10, -11 } ,&#xA;            { 0, 10, 0 } ,&#xA;            { 227810064, 10, -10 } ,&#xA;            { 0, 10, 0 } ,&#xA;            { 0, 10, 0 } ,&#xA;            { 0, 10, 0 } ,&#xA;            { 0, 10, 0 } ,&#xA;            { 0, 10, 0 } ,&#xA;            { 0, 10, 0 } ,&#xA;            { 768325726, 10, -10 } ,&#xA;            { 0, 10, 0 } ,&#xA;            { 1425562, 10, -8 } ,&#xA;            { 372685285, 10, -10 } ,&#xA;            { 466727421, 10, -10 } ,&#xA;            { 185741084, 10, -10 } ,&#xA;            { 427328646, 10, -10 } ,&#xA;            { 122594965, 10, -10 } , &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="818" LastEditorUserId="3848" LastEditDate="2019-08-21T13:25:26.220" LastActivityDate="2019-08-21T13:25:26.220" Title="A tool or webserver for building PSSM matrix" Tags="|blast|phylogenetics|phylogeny|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="961" PostTypeId="2" ParentId="935" CreationDate="2017-06-30T13:50:55.503" Score="1" Body="&lt;h3&gt;Using pyGATB&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;(I use the same file as in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/400/292&quot;&gt;https://bioinformatics.stackexchange.com/a/400/292&lt;/a&gt;, same workstation as in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/380/292&quot;&gt;https://bioinformatics.stackexchange.com/a/380/292&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ time python3 -c &quot;from gatb import Bank; seq_lens = [len(seq) for seq in Bank('SRR077487_2.filt.fastq.gz')]; print('Number of reads: %d' % len(seq_lens), 'Number of bases in reads: %d' % sum(seq_lens), sep='\n')&quot;&#xA;Number of reads: 23861612&#xA;Number of bases in reads: 2386161200&#xA;&#xA;real    0m41.122s&#xA;user    0m40.788s&#xA;sys     0m0.312s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is quite faster than bioawk:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ time bioawk -c fastx '{nb_seq+=1; nb_char+=length($seq)} END {print &quot;Number of reads: &quot;nb_seq&quot;\nNumber of bases in reads: &quot;nb_char}' SRR077487_2.filt.fastq.gz&#xA;Number of reads: 23861612&#xA;Number of bases in reads: 2386161200&#xA;&#xA;real    1m3.182s&#xA;user    1m2.916s&#xA;sys     0m0.268s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But not so much than the OP example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ time zgrep . SRR077487_2.filt.fastq.gz | awk 'NR%4==2{c++; l+=length($0)} END{print &quot;Number of reads: &quot;c; print &quot;Number of bases in reads: &quot;l}'&#xA;Number of reads: 23861612&#xA;Number of bases in reads: 2386161200&#xA;&#xA;real    0m47.127s&#xA;user    1m36.292s&#xA;sys     0m6.796s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or than &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/936/292&quot;&gt;the &lt;code&gt;wc&lt;/code&gt; based solution&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ fix_base_count() {&#xA;&amp;gt;     local counts=($(cat))&#xA;&amp;gt;     echo &quot;${counts[0]} $((${counts[1]} - ${counts[0]}))&quot;&#xA;&amp;gt; }&#xA;$ time gunzip -c SRR077487_2.filt.fastq.gz | awk 'NR % 4 == 2' | wc -cl | fix_base_count&#xA;23861612 2386161200&#xA;&#xA;real    0m44.915s&#xA;user    1m12.000s&#xA;sys     0m6.972s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I didn't compare with C-based solutions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/937/292&quot;&gt;&lt;code&gt;zcat&lt;/code&gt; to &lt;code&gt;/dev/null&lt;/code&gt; reference&lt;/a&gt; is the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ time zcat SRR077487_2.filt.fastq.gz &amp;gt; /dev/null&#xA;&#xA;real    0m39.745s&#xA;user    0m39.464s&#xA;sys     0m0.252s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm still impressed by pyGATB speed&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2017-06-30T13:50:55.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="962" PostTypeId="1" AcceptedAnswerId="964" CreationDate="2017-06-30T16:21:57.023" Score="5" ViewCount="237" Body="&lt;p&gt;For a study in GEO, I would like to obtain the data table header descriptions, specifically the &quot;VALUE&quot; column for all samples in the study.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you go here: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE99511&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE99511&lt;/a&gt;)&#xA;and then scroll down and click one of the samples: let's choose &quot;GSM2644971&quot;. Then scroll down and you should see &quot;Data table header descriptions&quot; and below that you should see &quot;&lt;strong&gt;VALUE&lt;/strong&gt; Normalized (provided the normalization method) Average Beta&quot;. That information is what I want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried using assayData() from the Biobase package but I didn't know whether the method takes a sample, a matrix of samples, or something else as a parameter. Apparently it takes a S4 and then returns some random text but I don't know how to use that random text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDITED:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if I do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Biobase)&#xA;library(GEOquery)&#xA;&#xA;getgeo&amp;lt;-getGEO(&quot;GSE99511&quot;)&#xA;assayData(getgeo$GSE99511_series_matrix.txt.gz)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I use assayData() I would want it to return &quot;Normalized (provided the normalization method) Average Beta&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead it returns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;environment: 0x110674178&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Please let me know if the question doesn't make sense or if there's another method that I should be using.&lt;/p&gt;&#xA;" OwnerUserId="1025" LastEditorUserId="1025" LastEditDate="2017-06-30T23:45:53.233" LastActivityDate="2017-07-02T12:54:12.990" Title="Obtaining data table headers from GEO using GEOquery" Tags="|r|database|bioconductor|text|" AnswerCount="1" CommentCount="6" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="964" PostTypeId="2" ParentId="962" CreationDate="2017-07-02T12:54:12.990" Score="3" Body="&lt;p&gt;The short answer is that if you are seeing information in a GEO sample, then it is the sample that you need to download and access using the appropriate method. In this case, that method is &lt;code&gt;Columns()&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(GEOquery)&#xA;gsm &amp;lt;- getGEO(&quot;GSM2644971&quot;)&#xA;Columns(gsm)&#xA;&#xA;          Column      Description&#xA;1         ID_REF                                                            &#xA;2         VALUE       Normalized (provided the normalization method) Average Beta&#xA;3         Detection   Pval&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The longer answer: you will really benefit from sitting down with the GEOquery documentation and trying to understand the objects that it uses and their methods. It is not something you can work out by trial and error for every case-by-case basis. &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/vignettes/GEOquery/inst/doc/GEOquery.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;The tutorial&lt;/a&gt; is a really good starting point (you will find &lt;code&gt;Columns()&lt;/code&gt; in there, for example).&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2017-07-02T12:54:12.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="965" PostTypeId="2" ParentId="960" CreationDate="2017-07-02T21:04:35.520" Score="2" Body="&lt;p&gt;If you take a look at my answer in this &lt;a href=&quot;https://www.biostars.org/p/259190/#259265&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioStars post&lt;/a&gt;, you can generate a PSSM using &lt;code&gt;AlignIO&lt;/code&gt; in Biopython:&lt;/p&gt;&#xA;" OwnerUserId="929" LastEditorUserId="48" LastEditDate="2018-07-31T08:20:59.287" LastActivityDate="2018-07-31T08:20:59.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="966" PostTypeId="2" ParentId="935" CreationDate="2017-07-02T21:56:20.447" Score="7" Body="&lt;h1&gt;pigz | awk | wc is the fastest method&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;First off for benchmarks with FASTQ it's best to use a specific real-world example with a known answer.  I've chosen this file:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/phase3/data/HG01815/sequence_read/ERR047740_1.filt.fastq.gz&quot; rel=&quot;noreferrer&quot;&gt;ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/phase3/data/HG01815/sequence_read/ERR047740_1.filt.fastq.gz&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;as my test file, the correct answers being:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Number of reads: 67051220&#xA;Number of bases in reads: 6034609800&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Next we want to find the fastest way possible to count these, all timings are the average wall-clock time (real) of 10 runs collected with the bash &lt;code&gt;time&lt;/code&gt; on an otherwise unloaded system:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;zgrep&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zgrep . ERR047740_1.filt.fastq.gz |&#xA;     awk 'NR%4==2{c++; l+=length($0)}&#xA;          END{&#xA;                print &quot;Number of reads: &quot;c; &#xA;                print &quot;Number of bases in reads: &quot;l&#xA;              }'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the slowest method with an average run-time of 125.35 seconds&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;gzip awk&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Using &lt;code&gt;gzip&lt;/code&gt; we gain about another 10 seconds:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gzip -dc ERR047740_1.filt.fastq.gz |&#xA;     awk 'NR%4==2{c++; l+=length($0)}&#xA;          END{&#xA;                print &quot;Number of reads: &quot;c; &#xA;                print &quot;Number of bases in reads: &quot;l&#xA;              }'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Average run-time is 116.69 seconds&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Konrad's gzip awk wc variant&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fix_base_count() {&#xA;    local counts=($(cat))&#xA;    echo &quot;${counts[0]} $((${counts[1]} - ${counts[0]}))&quot;&#xA;}&#xA;&#xA;gzip -dc ERR047740_1.filt.fastq.gz \&#xA;    | awk 'NR % 4 == 2' \&#xA;    | wc -cl \&#xA;    | fix_base_count&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This runs slower on this test file than the &lt;code&gt;gzip awk&lt;/code&gt; variant of the solution, average run-time is 122.28 seconds.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;kseq_test using latest &lt;code&gt;kseq.h&lt;/code&gt; from &lt;a href=&quot;https://github.com/attractivechaos/klib&quot; rel=&quot;noreferrer&quot;&gt;klib&lt;/a&gt;&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Code compiled with: &lt;code&gt;gcc -O2 -o kseq_test kseq_test.c -lz&lt;/code&gt; where &lt;code&gt;kseq_test.c&lt;/code&gt; is Simon's adaptation of Heng Li's FASTQ parser. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;kseq_test ERR047740_1.filt.fastq.gz&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Average run-time is 99.14 seconds, which is better than the &lt;code&gt;gzip&lt;/code&gt; core utilities based solution so far, but we can do better! &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;piz awk&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Using Mark Adler's &lt;a href=&quot;https://zlib.net/pigz/&quot; rel=&quot;noreferrer&quot;&gt;pigz&lt;/a&gt; as a drop-in replacement for &lt;code&gt;gzip&lt;/code&gt;, note that &lt;code&gt;pigz&lt;/code&gt; gives us a speed gain as on top of &lt;code&gt;gzip&lt;/code&gt; as in addition to the main deflate thread it uses another 3 threads for reading, writing and checksum calculations, see the &lt;a href=&quot;https://zlib.net/pigz/pigz.pdf&quot; rel=&quot;noreferrer&quot;&gt;man page&lt;/a&gt; for details.    &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pigz -dc ERR047740_1.filt.fastq.gz |&#xA;     awk 'NR%4==2{c++; l+=length($0)}&#xA;          END{&#xA;                print &quot;Number of reads: &quot;c; &#xA;                print &quot;Number of bases in reads: &quot;l&#xA;              }'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Average run-time is now 93.86 seconds, &lt;strong&gt;this is ~5 seconds faster than the kseq based C code but we can further improve the benchmark&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;pigz awk wc&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Next we use &lt;code&gt;pigz&lt;/code&gt; as a drop in replacment for Konrad's &lt;code&gt;wc&lt;/code&gt; variant of the &lt;code&gt;awk&lt;/code&gt; based solution.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fix_base_count() {&#xA;    local counts=($(cat))&#xA;    echo &quot;${counts[0]} $((${counts[1]} - ${counts[0]}))&quot;&#xA;}&#xA;&#xA;gzip -dc ERR047740_1.filt.fastq.gz \&#xA;    | awk 'NR % 4 == 2' \&#xA;    | wc -cl \&#xA;    | fix_base_count&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Average run-time is now down to 83.03 seconds, this is ~16 seconds faster than the kseq based solution and ~42 seconds faster than the OPs &lt;code&gt;zgrep&lt;/code&gt; based solution.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next as a baseline lets see just how much of this run-time is due to decompression of the input &lt;code&gt;fastq.gz&lt;/code&gt; file.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;gzip alone&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;gzip -dc ERR047740_1.filt.fastq.gz &amp;gt; /dev/null&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Average run-time: 105.95 seconds, so the &lt;code&gt;gzip&lt;/code&gt; based solutions (which also includes &lt;code&gt;zcat&lt;/code&gt; and &lt;code&gt;zgrep&lt;/code&gt; as these are provided by &lt;code&gt;gzip&lt;/code&gt;) are never going to be faster than &lt;code&gt;kseq_test&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;pigz alone&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;pigz -dc ERR047740_1.filt.fastq.gz &amp;gt; /dev/null&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Average run-time: 77.66 seconds, so quite clearly the additional three threads for read, write and checksum calculation offer a useful advantage&lt;/strong&gt;.  What's more this speed-up is greater when leveraging the &lt;code&gt;awk | wc&lt;/code&gt; based solution, it's not clear why, but I expect this is due to the extra write thread.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interestingly average CPU usage across all threads is quite revealing for the various answers, I've collated these stats using GNU time &lt;code&gt;/usr/bin/time --verbose&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;zgrep&lt;/code&gt; based solution 133% - must be more than one thread somehow&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;gzip | awk&lt;/code&gt; based solution 99% - all &lt;code&gt;gzip&lt;/code&gt; based solutions run single-threaded at 99% CPU usage&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;pigz | awk&lt;/code&gt; 147% &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;gzip | awk | wc&lt;/code&gt; 99% as with &lt;code&gt;gzip&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;pgiz | awk | wc&lt;/code&gt; 155%&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;kseq_test&lt;/code&gt; 99%&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;gzip &amp;gt; dev/null&lt;/code&gt; 99%&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;pigz &amp;gt; dev/null&lt;/code&gt; 155%&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whilst the main deflate thread in &lt;code&gt;pigz&lt;/code&gt; will run at 100% CPU load the extra 3 don't quite fully occupy additional cores to 100% (as is evidenced by average CPU usage of ~150%) they do however clearly result in reduced run-time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using Ubuntu 16.04.2 LTS**, my &lt;code&gt;gzip&lt;/code&gt;, &lt;code&gt;zcat&lt;/code&gt;, &lt;code&gt;zgrep&lt;/code&gt; versions are all gzip 1.6 and &lt;code&gt;pigz&lt;/code&gt; is version 2.3.1.  &lt;code&gt;gcc&lt;/code&gt; is version 5.4.0 &lt;/p&gt;&#xA;&#xA;&lt;p&gt;** I think my patch level is actually 16.04.4 but I've not rebooted for 170 days :p&lt;/p&gt;&#xA;" OwnerUserId="1769" LastEditorUserId="1769" LastEditDate="2017-07-02T22:46:28.427" LastActivityDate="2017-07-02T22:46:28.427" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="967" PostTypeId="2" ParentId="877" CreationDate="2017-07-02T23:22:34.500" Score="1" Body="&lt;p&gt;&lt;strong&gt;Potential pitfall!&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure about predictive models, but you need to be aware of a potential pitfall in blindly aligning PDX or PDO based sequencing data without first removing contaminating host organism reads, as otherwise these will lead to a lot of false positive variants caused by miss alignment.  In my experience even a small mount of host material can lead to a ten fold increase in called variants due to miss aligned host reads looking like true variants.  I'd recommend using &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/28/12/i172/269972&quot; rel=&quot;nofollow noreferrer&quot;&gt;Xenome&lt;/a&gt;, &lt;a href=&quot;https://github.com/data61/gossamer/blob/master/docs/xenome.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note Xenome was designed for DNA sequencing it's not splice-aware, so for RNA-Seq there is no optimal solution other than removing reads which align to the host organisms genome.  Although the issue here is that in conjunction with a splice-aware read aligner synteny between the host and grafted genomes might create some interesting problems.  However for RNA-Seq provided &lt;a href=&quot;https://en.wikipedia.org/wiki/Flow_cytometry#Fluorescence-activated_cell_sorting_.28FACS.29&quot; rel=&quot;nofollow noreferrer&quot;&gt;FACS&lt;/a&gt; or similar confirms low-levels of host contamination I expect levels of expression will not be badly affected.  Although this really needs investigating.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally and rather annoyingly Xenome produces none-standard FASTQ so you'll need to fix it's output with:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;awk '{if (NR % 4 == 1) print \&quot;@\&quot;$0; else if (NR % 4 == 3) print \&quot;+\&quot;$0; else print $0 }'&lt;/code&gt; as reported &lt;a href=&quot;http://seqanswers.com/forums/showthread.php?t=43872&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; on seqanswers.&lt;/p&gt;&#xA;" OwnerUserId="1769" LastActivityDate="2017-07-02T23:22:34.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="968" PostTypeId="2" ParentId="954" CreationDate="2017-07-03T09:02:32.610" Score="7" Body="&lt;p&gt;&lt;strong&gt;Running the jobs in parallel&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Iteratively is one solution, but since you're not using bootstrapping, which can be run multithread (and is not needed if you're taking the popular &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/vignettes/tximport/inst/doc/tximport.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tximport&lt;/a&gt; route) you can do better using &lt;a href=&quot;https://www.gnu.org/software/parallel/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GNU parallel&lt;/a&gt;, this will enable you to run as many jobs as you have execution threads on your system simultaneously.  Note that Kalliso only runs multithreaded with &lt;code&gt;-t&lt;/code&gt; if you're using bootstrapping.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You'll need to make the index first:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;kallisto index --make-unique -I index.fa.idx index.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Next lets define the input FASTQ files and the sample IDs you want to use using a tab-delimited input file like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sample_1    sample_1_R1.fastq.gz&#xA;sample_2    sample_2_R1.fastq.gz&#xA;sample_3    sample_3_R1.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here the first column is the sample ID which will be used for output and the second separated by a tab character is the input file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can produce this using a text editor but as a short-cut you can feed the list of fastq.gz you have into a list via:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ls -1 *.fastq.gz &amp;gt; editme&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Next we create a list of jobs to run on our system using a utility script &lt;code&gt;Make_job_list_kallisto.sh&lt;/code&gt; which consumes our tab delimited input file list above.  I've used your &lt;code&gt;-l&lt;/code&gt; and &lt;code&gt;-s&lt;/code&gt; parameters below, note that with paired end data normally these are estimated for you.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash -eu&#xA;&#xA;[ $# -ne 2 ] &amp;amp;&amp;amp; { echo -en &quot;\n*** This script generates jobs for GNU parallel. *** \n\n Error Nothing to do, usage: &amp;lt; input tab delimited list &amp;gt; &amp;lt; output run list file &amp;gt;\n\n&quot; ; exit 1; }&#xA;set -o pipefail&#xA;&#xA;# Get command-line args&#xA;INPUT_LIST=$1&#xA;OUTPUT=$2&#xA;&#xA;# Set counter&#xA;COUNT=1&#xA;END=$(wc -l $INPUT_LIST | awk '{print $1}')&#xA;&#xA;echo &quot; &quot;&#xA;echo &quot; * Input file is: $INPUT_LIST&quot;&#xA;echo &quot; * Number of runs: $END&quot;&#xA;echo &quot; * Output job list for GNU parallel saved to: $OUTPUT&quot;&#xA;echo &quot; &quot;&#xA;&#xA;# Main bit of command-line for job&#xA;CMD=&quot;kallisto quant -i index.fa.idx --single -l 200 -s 0.1&quot;&#xA;&#xA;# Main Loop&#xA;[ -e $OUTPUT ] &amp;amp;&amp;amp; rm $OUTPUT&#xA;while [ $COUNT -le $END ];&#xA;do&#xA;    LINE=( $(awk &quot;NR==$COUNT&quot; $INPUT_LIST) )&#xA;    # Make file list&#xA;    echo &quot;Working on $COUNT of $END Sample ID: ${LINE[0]}, Files ${LINE[@]:1}&quot;&#xA;    echo &quot;$CMD -o ${LINE[0]} ${LINE[@]:1}&quot; &amp;gt;&amp;gt; $OUTPUT&#xA;    ((COUNT++))&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We use the script like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; Make_job_list_kallisto.sh input_file_list.txt job_list.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now we have a job list for GNU parallel we can run our jobs in parallel like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;parallel --progress --jobs 4 --joblog kallisto_joblog.txt &amp;lt; job_list.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will run 4 of the kallisto jobs simultaneously (Increase this if you have more threads - don't mind consuming all the IO/CPU threads on your system) producing a nice progress report of those completed, additionally a job log will be written to the file &lt;code&gt;kallisto_joblog.txt&lt;/code&gt;.  &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;For paired end data I already have these scripts + similar instructions on &lt;a href=&quot;https://github.com/MattBashton/grolar&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt; which people might find useful. &lt;/p&gt;&#xA;" OwnerUserId="1769" LastEditorUserId="1769" LastEditDate="2017-07-03T20:50:27.253" LastActivityDate="2017-07-03T20:50:27.253" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="969" PostTypeId="1" AcceptedAnswerId="1005" CreationDate="2017-07-03T09:09:33.200" Score="6" ViewCount="385" Body="&lt;p&gt;I would like to ask if anyone has experience in running a subset of the &lt;a href=&quot;https://github.com/PASApipeline/PASApipeline/wiki&quot; rel=&quot;nofollow noreferrer&quot;&gt;PASA pipeline&lt;/a&gt;, in particular for the reconciliation of some experimental 'transcripts' with the reference annotation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In more detail, I am working with RNA-seq data from &lt;em&gt;D. melanogaster&lt;/em&gt;. I have reconstructed the 'transcripts' using Trinity. I have aligned these 'transcripts' to the reference genome using GMAP. Now I would like to match these 'experimental transcripts' with the reference annotation, to see how they compare.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if I can run just a subset of the whole PASA pipeline, so basically skipping the step of alignment to the reference and using the alignment file that I generated externally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I would be glad to receive more general redirecting to any released software allowing the comparison of experimental transcripts to the reference annotation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cross-posted on the &lt;a href=&quot;https://groups.google.com/forum/#!topic/pasapipeline-users/cyG9e8-HjG8&quot; rel=&quot;nofollow noreferrer&quot;&gt;PASA Google Group pasapipeline-users&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cross-posted &lt;a href=&quot;https://www.biostars.org/p/260594/&quot; rel=&quot;nofollow noreferrer&quot;&gt;on biostars&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="294" LastEditorUserId="294" LastEditDate="2018-04-10T07:44:32.357" LastActivityDate="2018-04-10T07:44:32.357" Title="PASA pipeline: compare experimental transcripts to the reference annotation" Tags="|rna-seq|assembly|transcriptome|isoform|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="970" PostTypeId="1" AcceptedAnswerId="971" CreationDate="2017-07-03T11:49:21.247" Score="1" ViewCount="157" Body="&lt;p&gt;For a diet analysis of an insect-eating animal, all species in a sample shall be identified. For this a sequencing of the metagenomic sample was done, where the COI/COX region was used as a barcode and amplified. This resulted in 20 million reads (after quality control). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While looking for an apropriate workflow, I am faced with these issues:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;For many tools 20 million reads are to much to handle (BLAST for&#xA;instance)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For the majority of metagenomic tools the focus is on micro-organisms, not insects&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The database to search against must contain COI/COX data, mainly from insects, however, the animal could also eat snails or frog eggs...&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Since the reads are amplicons some analysis are not recommended&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Leaving me with these questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What are the typical analysis steps in a workflow like this?&lt;/li&gt;&#xA;&lt;li&gt;Which tools could I use for my purpose?&lt;/li&gt;&#xA;&lt;li&gt;Where could I get a database of COI/COX data?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1037" LastActivityDate="2017-07-03T15:34:15.717" Title="Workflow of metabarcoding analysis" Tags="|metagenome|barcode|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="971" PostTypeId="2" ParentId="970" CreationDate="2017-07-03T12:13:22.150" Score="0" Body="&lt;p&gt;I'm working on something similar as a side project. I'll detail the steps that I'm using, but I'm curious if others can suggest a better way.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;You don't mention if your data are paired-end, but in my case I have paired-end sequences that overlap in the middle. I used &lt;a href=&quot;https://ccb.jhu.edu/software/FLASH&quot; rel=&quot;nofollow noreferrer&quot;&gt;FLASh&lt;/a&gt; to merge the reads and &lt;a href=&quot;https://github.com/lh3/seqtk&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqtk&lt;/a&gt; to convert the reads to fasta format.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I used the cluster_fast algorithm in &lt;a href=&quot;https://github.com/torognes/vsearch&quot; rel=&quot;nofollow noreferrer&quot;&gt;vsearch&lt;/a&gt; to cluster sequences at 100% identity, then filtered out rare sequence types (&amp;lt;10 sequences) to account for sequencing errors.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This resulted in a manageable number of unique sequences, which I searched against a custom database using vsearch, but which could also have been searched against the nr database using blast.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="581" LastActivityDate="2017-07-03T12:13:22.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="972" PostTypeId="1" AcceptedAnswerId="993" CreationDate="2017-07-03T14:25:39.637" Score="5" ViewCount="1142" Body="&lt;p&gt;I want to get the alignment of chain A of 1kf6 (PDB ID) from the pfam database &lt;a href=&quot;http://pfam.xfam.org/protein/FRDA_ECOLI&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. This protein chain has two main domains (FAD_binding_2 and    Succ_DH_flav_C). In pfam there is a link to one of these domains and after clicking in one of the mentioned domains in the table below the page, another page comes at the top of which there is a link to architectures. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if we click on &quot;FAD_binding_2&quot; in the table, there are 220 architectures that have this domain and after clicking on that button (at the top of page), a page opens &lt;a href=&quot;http://pfam.xfam.org/family/Succ_DH_flav_C#tabview=tab1&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. In this page 7471 sequences with the following architecture: FAD_binding_2, Succ_DH_flav_C, resembles most to my chain (because it has both FAD_binding_2 and Succ_DH_flav_C domains). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we want to see all 7471 sequences with that architecture we can click on the show button under that architecture and finally we can see all of that 7471 architecture resembling our chain A sequence. Down the page we can see other architectures that all have FAD_binding_2, but some of them do not have both FAD_binding_2 and Succ_DH_flav_C domains (or some of them have other extra domains that we are not interested in) and so we do not need their sequences and their alignments. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we want to get the alignment of full sequences at the left side of the page, we can see that we can just get the alignment of all 15696 sequences and it seems that there is no way to get the alignment of our favourite 7471 sequences. I would like to know if there is any way to get the alignment or at least the fasta file of our favourite 7471 sequences (which have both of our favourite domains not just one of them)?&lt;/p&gt;&#xA;" OwnerUserId="818" LastEditorUserId="298" LastEditDate="2017-07-03T16:20:39.590" LastActivityDate="2017-07-18T13:01:54.007" Title="Obtaining all protein sequences with a particular domain architecture from Pfam" Tags="|sequence-homology|domains|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="973" PostTypeId="2" ParentId="970" CreationDate="2017-07-03T14:28:22.810" Score="0" Body="&lt;p&gt;Regarding your third question (&quot;Where could I get a database of COI/COX data?&quot;), you can apparently search and download fasta sequences from &lt;a href=&quot;http://www.boldsystems.org/index.php/Public_SearchTerms&quot; rel=&quot;nofollow noreferrer&quot;&gt;BOLD&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't see a &quot;select all&quot; button to select all records found by one search in a single fasta file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There seems to exist an R API: &lt;a href=&quot;https://cran.r-project.org/web/packages/bold/bold.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cran.r-project.org/web/packages/bold/bold.pdf&lt;/a&gt; which refers to this: &lt;a href=&quot;http://www.boldsystems.org/index.php/resources/api#sequenceParameters&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.boldsystems.org/index.php/resources/api#sequenceParameters&lt;/a&gt;&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this I seem to understand that one can directly query using urls:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wget -O Gastropoda.fa http://www.boldsystems.org/index.php/API_Public/sequence?taxon=Gastropoda&amp;amp;marker=COI&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I get a fasta file containing COI sequences for Gastropoda. There may be ways to refine the query geographically, using the &lt;code&gt;geo&lt;/code&gt; parameter.&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2017-07-03T15:34:15.717" LastActivityDate="2017-07-03T15:34:15.717" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="974" PostTypeId="1" AcceptedAnswerId="987" CreationDate="2017-07-03T14:41:40.387" Score="10" ViewCount="2072" Body="&lt;p&gt;I have high-depth variant calling created using the HaplotypeCaller with &lt;code&gt;--output_mode EMIT_ALL_SITES&lt;/code&gt; I'm interested in finding all sites (regardless of genotype call heterozygous or homozygous) where at least one of the &lt;strong&gt;alternative&lt;/strong&gt; alleles have an &lt;code&gt;AD&lt;/code&gt; value (Allelic Depth) greater than 10, &lt;em&gt;I&lt;/em&gt;.&lt;em&gt;e&lt;/em&gt;. are supported by more than 10 reads. Also ideally I want back more than just the first alternative allele.  Note that I don't want back lines of VCF were we only see an AD count for the ref allele only.&lt;/p&gt;&#xA;&lt;p&gt;So in the example VCF snippet below I'm wanting to select lines: 6,7,8,12,13 and 14, which have GT:AD values &lt;code&gt;1/1:1,988:989&lt;/code&gt; &lt;code&gt;0/1:116,92&lt;/code&gt; &lt;code&gt;0/1:220,234&lt;/code&gt; &lt;code&gt;0/1:62,611&lt;/code&gt; &lt;code&gt;1/1:0,109&lt;/code&gt; respectively.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  12908_DIAG&#xA;3   187446740   .   T   .   Infinity    .   AN=2;DP=1095;MQ=60.00   GT:AD:DP    0/0:1095:1095&#xA;3   187446741   .   C   .   Infinity    .   AN=2;DP=1117;MQ=60.00   GT:AD:DP    0/0:1117:1117&#xA;3   187446752   .   A   .   Infinity    .   AN=2;DP=1297;MQ=60.00   GT:AD:DP    0/0:1297:1297&#xA;3   187446763   .   C   .   Infinity    .   AN=2;DP=1494;MQ=60.00   GT:AD:DP    0/0:1494:1494&#xA;3   187451574   .   C   .   Infinity    .   AN=2;DP=1493;MQ=60.00   GT:AD:DP    0/0:1493:1493&#xA;3   187451609   rs1880101   A   G   39794.03    .   AC=2;AF=1.00;AN=2;BaseQRankSum=1.859;ClippingRankSum=0.000;DB;DP=995;ExcessHet=3.0103;FS=0.000;MLEAC=2;MLEAF=1.00;MQ=60.00;MQRankSum=0.000;QD=24.56;ReadPosRankSum=0.406;SOR=8.234  GT:AD:DP:GQ:PL  1/1:1,988:989:99:39808,2949,0&#xA;4   1803279 .   T   G   0   .   AC=0;AF=0.00;AN=2;BaseQRankSum=-6.652;ClippingRankSum=0.000;DP=245;ExcessHet=3.0103;FS=89.753;MLEAC=0;MLEAF=0.00;MQ=59.97;MQRankSum=0.000;ReadPosRankSum=-2.523;SOR=6.357   GT:AD:DP:GQ:PL  0/0:211,23:234:99:0,364,6739&#xA;4   1803307 rs2305183   T   C   2486.60 .   AC=1;AF=0.500;AN=2;BaseQRankSum=-5.049;ClippingRankSum=0.000;DB;DP=215;ExcessHet=3.0103;FS=1.110;MLEAC=1;MLEAF=0.500;MQ=59.97;MQRankSum=0.000;QD=11.95;ReadPosRankSum=-0.045;SOR=0.809  GT:AD:DP:GQ:PL  0/1:116,92:208:99:2494,0,3673&#xA;4   1803671 .   C   A   0   .   AC=0;AF=0.00;AN=2;BaseQRankSum=-0.880;ClippingRankSum=0.000;DP=450;ExcessHet=3.0103;FS=0.000;MLEAC=0;MLEAF=0.00;MQ=60.00;MQRankSum=0.000;ReadPosRankSum=-0.953;SOR=0.572    GT:AD:DP:GQ:PL  0/0:445,2:447:99:0,1272,15958&#xA;4   1803681 .   T   C   0   .   AC=0;AF=0.00;AN=2;BaseQRankSum=-1.654;ClippingRankSum=0.000;DP=483;ExcessHet=3.0103;FS=0.000;MLEAC=0;MLEAF=0.00;MQ=60.00;MQRankSum=0.000;ReadPosRankSum=-0.422;SOR=0.664    GT:AD:DP:GQ:PL  0/0:479,2:481:99:0,1408,18538&#xA;4   1803703 .   A   G   0   .   AC=0;AF=0.00;AN=2;BaseQRankSum=-1.704;ClippingRankSum=0.000;DP=458;ExcessHet=3.0103;FS=0.000;MLEAC=0;MLEAF=0.00;MQ=60.00;MQRankSum=0.000;ReadPosRankSum=0.299;SOR=0.497 GT:AD:DP:GQ:PL  0/0:454,2:456:99:0,1325,18095&#xA;4   1803704 rs2234909   T   C   6676.60 .   AC=1;AF=0.500;AN=2;BaseQRankSum=-2.605;ClippingRankSum=0.000;DB;DP=456;ExcessHet=3.0103;FS=1.753;MLEAC=1;MLEAF=0.500;MQ=60.00;MQRankSum=0.000;QD=14.71;ReadPosRankSum=0.324;SOR=0.849   GT:AD:DP:GQ:PL  0/1:220,234:454:99:6684,0,6366&#xA;4   1803824 rs2305184   C   G   2030.60 .   AC=1;AF=0.500;AN=2;BaseQRankSum=8.083;ClippingRankSum=0.000;DB;DP=124;ExcessHet=3.0103;FS=6.128;MLEAC=1;MLEAF=0.500;MQ=60.00;MQRankSum=0.000;QD=16.51;ReadPosRankSum=0.180;SOR=0.096    GT:AD:DP:GQ:PL  0/1:62,61:123:99:2038,0,1766&#xA;4   1805296 rs3135883   G   A   3876.03 .   AC=2;AF=1.00;AN=2;DB;DP=110;ExcessHet=3.0103;FS=0.000;MLEAC=2;MLEAF=1.00;MQ=60.00;QD=29.22;SOR=9.401    GT:AD:DP:GQ:PL  1/1:0,109:109:99:3890,326,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/gm4pftuk9i1gx6p/test.vcf?dl=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;A dropbox link for file&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'd initially considered using GATK's SelectVariants but I'm not sure JEXL has the ability to select out what I want specifically other than a blanket AD &amp;gt; 10 which will give me both ref and alt alleles with AD &amp;gt; 10.  Perhaps there is a bioawk solution or something more elaborate with coreutils which could successfully return sites with an alt AD count &amp;gt; 10?&lt;/p&gt;&#xA;" OwnerUserId="1769" LastEditorUserId="1769" LastEditDate="2020-12-03T20:43:25.947" LastActivityDate="2020-12-03T20:45:58.427" Title="Selecting sites from VCF which have an alt AD &gt; 10" Tags="|variant-calling|vcf|bioawk|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="975" PostTypeId="2" ParentId="974" CreationDate="2017-07-03T15:41:14.367" Score="5" Body="&lt;p&gt;using &lt;a href=&quot;http://lindenb.github.io/jvarkit/VCFFilterJS.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;vcfilterjs&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the following script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function accept(vc)&#xA;    {&#xA;    var i,j;&#xA;    for(i=0;i&amp;lt; vc.getNSamples();++i)&#xA;        {&#xA;        var genotype = vc.getGenotype(i);&#xA;        if(!genotype.hasAD()) continue;&#xA;        var ad = genotype.getAD();&#xA;        /* loop over AD starting from '1' =  first ALT */&#xA;        for(j=1  ;j&amp;lt; ad.length ;++j)&#xA;            {&#xA;            if(ad[j]&amp;gt;10) return true;&#xA;            }&#xA;        }&#xA;    return false;&#xA;    }&#xA;accept(variant);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;usage:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar dist/vcffilterjs.jar -f script.js Test.vcf &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastEditorUserId="71" LastEditDate="2017-07-05T16:24:32.853" LastActivityDate="2017-07-05T16:24:32.853" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="977" PostTypeId="2" ParentId="951" CreationDate="2017-07-03T21:16:19.757" Score="1" Body="&lt;p&gt;If your goal is to bin the resulting contigs into genomes, then you should do option #1, pooling the reads and assembling into one set of contigs. &lt;/p&gt;&#xA;" OwnerUserId="1044" LastActivityDate="2017-07-03T21:16:19.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="979" PostTypeId="1" AcceptedAnswerId="980" CreationDate="2017-07-04T11:53:41.607" Score="5" ViewCount="2030" Body="&lt;p&gt;I have a reference database with contains 100s of sequences in fasta format. Some of these sequences have duplicate names like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1_uniqueGeneName&#xA;atgc&#xA;&amp;gt;1_anotherUniqueGeneName&#xA;atgc&#xA;&amp;gt;1_duplicateName&#xA;atgc&#xA;&amp;gt;1_duplicateName&#xA;atgc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is is possible to run through a large file like this and change the names of only the duplicates?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1_uniqueGeneName&#xA;atgc&#xA;&amp;gt;1_anotherUniqueGeneName&#xA;atgc&#xA;&amp;gt;1_duplicateName_1&#xA;atgc&#xA;&amp;gt;1_duplicateName_2&#xA;atgc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="982" LastEditorUserId="48" LastEditDate="2017-07-05T17:52:29.090" LastActivityDate="2017-07-05T17:52:29.090" Title="How to append numbers only on duplicates sequence names?" Tags="|fasta|text|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="980" PostTypeId="2" ParentId="979" CreationDate="2017-07-04T11:58:43.597" Score="5" Body="&lt;p&gt;Sure, this little Perl snippet should do it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ perl -pe 's/$/_$seen{$_}/ if ++$seen{$_}&amp;gt;1 and /^&amp;gt;/; ' file.fa &#xA;&amp;gt;1_uniqueGeneName&#xA;atgc&#xA;&amp;gt;1_anotherUniqueGeneName&#xA;atgc_2&#xA;&amp;gt;1_duplicateName&#xA;atgc_3&#xA;&amp;gt;1_duplicateName_2&#xA;atgc_4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, to make the changes in the original file, use &lt;code&gt;-i&lt;/code&gt;:    &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl -i.bak -pe 's/$/_$seen{$_}/ if ++$seen{$_}&amp;gt;1 and /^&amp;gt;/; ' file.fa &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that the first occurrence of a duplicate name isn't changed, the second will become &lt;code&gt;_2&lt;/code&gt;, the third &lt;code&gt;_3&lt;/code&gt; etc. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Explanation&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;perl -pe&lt;/code&gt; : &lt;strong&gt;p&lt;/strong&gt;rint each input line after applying the script given by &lt;code&gt;-e&lt;/code&gt; to it. &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;++$seen{$_}&amp;gt;1&lt;/code&gt; : increment the current value stored in the hash &lt;code&gt;%seen&lt;/code&gt; for this line (&lt;code&gt;$_&lt;/code&gt;) by 1 and compare it to &lt;code&gt;1&lt;/code&gt;. &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;s/$/_$seen{$_}/ if ++$seen{$_}&amp;gt;1 and /^&amp;gt;/&lt;/code&gt; : if the current line starts with a &lt;code&gt;&amp;gt;&lt;/code&gt; and the  value stored in the hash &lt;code&gt;%seen&lt;/code&gt; for this line is greater than 1 (if this isn't the first time we see this line), replace the end of the line (&lt;code&gt;$&lt;/code&gt;) with a &lt;code&gt;_&lt;/code&gt; and the current value in the hash&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Alternatively, here's the same idea in &lt;code&gt;awk&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk '(/^&amp;gt;/ &amp;amp;&amp;amp; s[$0]++){$0=$0&quot;_&quot;s[$0]}1;' file.fa &#xA;&amp;gt;1_uniqueGeneName&#xA;atgc&#xA;&amp;gt;1_anotherUniqueGeneName&#xA;atgc&#xA;&amp;gt;1_duplicateName&#xA;atgc&#xA;&amp;gt;1_duplicateName_2&#xA;atgc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To make the changes in the original file (assuming you are using GNU &lt;code&gt;awk&lt;/code&gt; which is the default on most Linux versions), use &lt;code&gt;-i inplace&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -iinplace '(/^&amp;gt;/ &amp;amp;&amp;amp; s[$0]++){$0=$0&quot;_&quot;s[$0]}1;' file.fa &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Explanation&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In awk, the special variable &lt;code&gt;$0&lt;/code&gt; is the current line. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;(/^&amp;gt;/ &amp;amp;&amp;amp; s[$0]++)&lt;/code&gt; : if this line starts with a &lt;code&gt;&amp;gt;&lt;/code&gt; and incrementing the value stored in the array &lt;code&gt;s&lt;/code&gt; for this line by 1 evaluates to true (is greater than 0).&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$0=$0&quot;_&quot;s[$0]&lt;/code&gt; :  make the current line be itself with a &lt;code&gt;_&lt;/code&gt; and the value from &lt;code&gt;s&lt;/code&gt; appended.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;1;&lt;/code&gt; : this is just shorthand for &quot;print this line&quot;. If an expression evaluates to true, awk will print the current line. Since &lt;code&gt;1&lt;/code&gt; is always true, this will print every line. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If you want all of the duplicates to be marked, you need to read the file twice. Once to collect the names and a second to mark them:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk '{&#xA;    if (NR==FNR){&#xA;        if(/^&amp;gt;/){&#xA;            s[$0]++&#xA;        }&#xA;        next;&#xA;    }&#xA;    if(/^&amp;gt;/){&#xA;        k[$0]++;&#xA;        if(s[$0]&amp;gt;1){&#xA;            $0=$0&quot;_&quot;k[$0]&#xA;        }&#xA;    }&#xA;    print&#xA;}' file.fa file.fa&#xA;&amp;gt;1_uniqueGeneName&#xA;atgc&#xA;&amp;gt;1_anotherUniqueGeneName&#xA;atgc&#xA;&amp;gt;1_duplicateName_1&#xA;atgc&#xA;&amp;gt;1_duplicateName_2&#xA;atgc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;IMPORTANT&lt;/strong&gt;: note that all of these approaches assume you don't already have sequence names ending with &lt;code&gt;_N&lt;/code&gt; where &lt;code&gt;N&lt;/code&gt; is a number. If your input file has 2 sequences called &lt;code&gt;foo&lt;/code&gt; and one called &lt;code&gt;foo_2&lt;/code&gt;, then you will end up with two &lt;code&gt;foo_2&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat test.fa&#xA;&amp;gt;foo_2&#xA;actg&#xA;&amp;gt;foo&#xA;actg&#xA;&amp;gt;foo&#xA;actg&#xA;$ perl -pe 's/$/_$seen{$_}/ if ++$seen{$_}&amp;gt;1 and /^&amp;gt;/; ' test.fa &#xA;&amp;gt;foo_2&#xA;actg&#xA;&amp;gt;foo&#xA;actg&#xA;&amp;gt;foo_2&#xA;actg&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If this can be an issue for you, use one of the more sophisticated approaches suggested by the other answers.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-07-04T13:59:59.517" LastActivityDate="2017-07-04T13:59:59.517" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="981" PostTypeId="2" ParentId="979" CreationDate="2017-07-04T12:02:17.610" Score="6" Body="&lt;p&gt;Sure, using biopython:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;from Bio import SeqIO&#xA;&#xA;records = set()&#xA;of = open(&quot;output.fa&quot;, &quot;w&quot;)&#xA;for record in SeqIO.parse(&quot;foo.fa&quot;, &quot;fasta&quot;):&#xA;    ID = record.id&#xA;    num = 1&#xA;    while ID in records:&#xA;        ID = &quot;{}_{}&quot;.format(record.id, num)&#xA;        num += 1&#xA;    records.add(ID)&#xA;    record.id = ID&#xA;    record.name = ID&#xA;    record.description = ID&#xA;    SeqIO.write(record, of, &quot;fasta&quot;)&#xA;of.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Change &lt;code&gt;output.fa&lt;/code&gt; and &lt;code&gt;foo.fa&lt;/code&gt;. One doesn't need to explicitly change the &lt;code&gt;.name&lt;/code&gt; and &lt;code&gt;.description&lt;/code&gt;, but that's handy to prevent the original ID from not appearing still (after a space).&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Explanation&lt;/h1&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;records = set()&lt;/code&gt; This will create a lookup table of all IDs &lt;strong&gt;written&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;for record in SeqIO.parse(&quot;foo.fa&quot;, &quot;fasta&quot;):&lt;/code&gt; Open &lt;code&gt;foo.fa&lt;/code&gt; as a fasta file and iterate over the &lt;strong&gt;entries&lt;/strong&gt; in it. These are objects with an ID, name, description, and sequence attribute (the name and description are the same as the ID if not present).&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;ID = record.id&lt;/code&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Memoization&quot; rel=&quot;nofollow noreferrer&quot;&gt;Memoize&lt;/a&gt; the entry ID (e.g., &lt;code&gt;1_duplicateName&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;while ID in records:&lt;/code&gt; As long as the ID has already been seen, keep looping.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;ID = &quot;{}_{}&quot;.format(record.id, num)&lt;/code&gt; Start adding increasing numbers after the ID, such as &lt;code&gt;1_duplicateName_1&lt;/code&gt; and &lt;code&gt;1_duplicateName_2&lt;/code&gt;. This will continue until the ID has not been seen.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;records.add(ID)&lt;/code&gt; Add the unseen ID to the set.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;record.id = ID&lt;/code&gt; Update the ID, the &lt;code&gt;.name&lt;/code&gt; and &lt;code&gt;.description&lt;/code&gt; are the same. If you don't do that, then you get output like &lt;code&gt;&amp;gt;1_duplicateName_1 1_duplicateName&lt;/code&gt;. That's not really a problem, but it's excessive.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;SeqIO.write(record, of, &quot;fasta&quot;)&lt;/code&gt; Write the record to the output file in fasta format.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A benefit of biopython is the easy ability to change formats, so instead of fasta one could have substituted Genbank or another format if needed.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-07-05T15:49:58.337" LastActivityDate="2017-07-05T15:49:58.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="982" PostTypeId="1" CreationDate="2017-07-04T12:53:14.190" Score="6" ViewCount="939" Body="&lt;p&gt;I was wondering how I can calculate the charge of a protein peptide (e.g. &quot;RKTTLVPNTQTASPR&quot;) computationally in R or another tool.&lt;/p&gt;&#xA;" OwnerUserId="704" LastEditorUserId="298" LastEditDate="2017-07-04T14:56:02.313" LastActivityDate="2017-07-18T22:47:05.673" Title="Calculating the charge of a peptide computationally" Tags="|r|proteins|" AnswerCount="3" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="983" PostTypeId="2" ParentId="982" CreationDate="2017-07-04T13:05:35.823" Score="3" Body="&lt;p&gt;A quick google search turns up &lt;a href=&quot;http://protcalc.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;protcalc&lt;/a&gt;, which is able to give a nice pH-dependent table of peptide charges (yours ranges from 3.1 at pH 4 to 1.5 at pH 10). It's on sourceforge, so hopefully the source code (and maybe a publication) is somewhere in there. Granted, this isn't R, but it's not clear from your post how necessary that really is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, in general &quot;best and most accurate&quot; turns out to be highly subjective (at least the &quot;best&quot; part of that).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-07-04T13:05:35.823" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="984" PostTypeId="2" ParentId="979" CreationDate="2017-07-04T13:10:58.163" Score="7" Body="&lt;p&gt;And here’s a solution using R (with the Bioconductor):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;fa = ShortRead::readFasta(infile)&#xA;ids = as.character(ShortRead::id(fa))&#xA;fa@id = Biostrings::BStringSet(make.unique(ids, sep = '_'))&#xA;ShortRead::writeFasta(fa, outfile)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2017-07-04T17:14:57.440" LastActivityDate="2017-07-04T17:14:57.440" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="985" PostTypeId="2" ParentId="779" CreationDate="2017-07-04T14:13:22.690" Score="1" Body="&lt;h3&gt;An example solution for Seurat:&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Retrieve sample IDs from the .csv used with &lt;code&gt;cellranger&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samples &amp;lt;- read.csv(file.path(&quot;/path/to/csv&quot;, &quot;nameof.csv&quot;), stringsAsFactors=F)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Load the 10x dataset and initialize the Seurat object:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cells.data &amp;lt;- Read10X(&quot;path/to/filtered_gene_bc_matrices&quot;)&#xA;&#xA;cells &amp;lt;- new(&quot;seurat&quot;, raw.data=cells.data)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Get barcodes and suffix:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cellcodes &amp;lt;- as.data.frame(cells@raw.data@Dimnames[[2]])&#xA;colnames(cellcodes) &amp;lt;- &quot;barcodes&quot;&#xA;rownames(cellcodes) &amp;lt;- cellcodes$barcodes&#xA;&#xA;cellcodes$libcodes &amp;lt;- as.factor(gsub(pattern=&quot;.+-&quot;, replacement=&quot;&quot;, cellcodes$barcodes))&#xA;cellcodes$samples &amp;lt;- as.vector(samples$library_id[cellcodes$libcodes])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Create dataframe for &lt;code&gt;meta.data&lt;/code&gt; argument and set up object:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sampleidentity &amp;lt;- cellcodes[&quot;samples&quot;]&#xA;&#xA;cells &amp;lt;- Setup(cells,&#xA;               meta.data=sampleidentity,&#xA;               min.cells=3,&#xA;               min.genes=200,&#xA;               do.logNormalize=T, total.expr=1e4, project=&quot;projectname&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Use the &lt;code&gt;group.by&lt;/code&gt; argument of various functions.&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2017-07-04T14:13:22.690" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="986" PostTypeId="2" ParentId="895" CreationDate="2017-07-04T16:06:20.003" Score="2" Body="&lt;p&gt;You can download a list of transcript annotations as a flat file from UCSC:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://hgdownload.cse.ucsc.edu/goldenPath/hg19/database/refGene.txt.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://hgdownload.cse.ucsc.edu/goldenPath/hg19/database/refGene.txt.gz&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not exactly a BED file, but it does contain information about the known transcripts for this assembly (hg19, in this case):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;585 NR_046018   chr1    +   11873   14409   14409   14409   3   11873,12612,13220,  12227,12721,14409,  0   DDX11L1 unk unk -1,-1,-1,&#xA;585 NR_024540   chr1    -   14361   29370   29370   29370   11  14361,14969,15795,16606,16857,17232,17605,17914,18267,24737,29320,  14829,15038,15947,16765,17055,17368,17742,18061,18366,24891,29370,  0   WASH7P  unk unk -1,-1,-1,-1,-1,-1,-1,-1,-1,-1,-1,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The gene name can be found in column 13. The exon start and end positions are in columns 10 and 11, respectively. More information can be found in the other columns:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;bin&lt;/li&gt;&#xA;&lt;li&gt;name&lt;/li&gt;&#xA;&lt;li&gt;chrom&lt;/li&gt;&#xA;&lt;li&gt;strand&lt;/li&gt;&#xA;&lt;li&gt;txStart&lt;/li&gt;&#xA;&lt;li&gt;txEnd&lt;/li&gt;&#xA;&lt;li&gt;cdsStart&lt;/li&gt;&#xA;&lt;li&gt;cdsEnd&lt;/li&gt;&#xA;&lt;li&gt;exonCount&lt;/li&gt;&#xA;&lt;li&gt;exonStarts&lt;/li&gt;&#xA;&lt;li&gt;exonEnds&lt;/li&gt;&#xA;&lt;li&gt;score&lt;/li&gt;&#xA;&lt;li&gt;name2&lt;/li&gt;&#xA;&lt;li&gt;cdsStartStat&lt;/li&gt;&#xA;&lt;li&gt;cdsEndStat&lt;/li&gt;&#xA;&lt;li&gt;exonFrames&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(The column information was obtained from the &lt;a href=&quot;http://hgdownload.cse.ucsc.edu/goldenpath/hg19/database/refGene.sql&quot; rel=&quot;nofollow noreferrer&quot;&gt;sql file here&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2017-07-04T16:06:20.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="987" PostTypeId="2" ParentId="974" CreationDate="2017-07-04T16:24:13.710" Score="4" Body="&lt;p&gt;This now works with the development version of Bcftools v1.5 (commit 4f134df). Thanks to Petr Danecek for &lt;a href=&quot;https://github.com/samtools/bcftools/issues/639&quot; rel=&quot;nofollow noreferrer&quot;&gt;adding the feature&lt;/a&gt;. I expect this feature to make its way into the next release of Bcftools:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;git clone git://github.com/samtools/htslib.git&#xA;git clone git://github.com/samtools/bcftools.git&#xA;(cd bcftools; make)&#xA;&#xA;bgzip Test.vcf&#xA;./bcftools/bcftools index Test.vcf.gz&#xA;./bcftools/bcftools filter -i 'AD[0:1-] &amp;gt; 10' Test.vcf.gz &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output without header (I have modified the second line to be tri-allelic to demonstrate the filtering works):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;3   187451609   rs1880101   A   G   39794   PASS    AC=2;AF=1;AN=2;BaseQRankSum=1.859;ClippingRankSum=0;DB;DP=995;ExcessHet=3.0103;FS=0;MLEAC=2;MLEAF=1;MQ=60;MQRankSum=0;QD=24.56;ReadPosRankSum=0.406;SOR=8.234   GT:AD:DP:GQ:PL  1/1:1,988:989:99:39808,2949,0&#xA;4   1803279 .   T   G   0   PASS    AC=0;AF=0;AN=2;BaseQRankSum=-6.652;ClippingRankSum=0;DP=245;ExcessHet=3.0103;FS=89.753;MLEAC=0;MLEAF=0;MQ=59.97;MQRankSum=0;ReadPosRankSum=-2.523;SOR=6.357 GT:AD:DP:GQ:PL  0/0/0:211,3,34:234:99:0,364,6739&#xA;4   1803307 rs2305183   T   C   2486.6  PASS    AC=1;AF=0.5;AN=2;BaseQRankSum=-5.049;ClippingRankSum=0;DB;DP=215;ExcessHet=3.0103;FS=1.11;MLEAC=1;MLEAF=0.5;MQ=59.97;MQRankSum=0;QD=11.95;ReadPosRankSum=-0.045;SOR=0.809   GT:AD:DP:GQ:PL  0/1:116,92:208:99:2494,0,3673&#xA;4   1803704 rs2234909   T   C   6676.6  PASS    AC=1;AF=0.5;AN=2;BaseQRankSum=-2.605;ClippingRankSum=0;DB;DP=456;ExcessHet=3.0103;FS=1.753;MLEAC=1;MLEAF=0.5;MQ=60;MQRankSum=0;QD=14.71;ReadPosRankSum=0.324;SOR=0.849  GT:AD:DP:GQ:PL  0/1:220,234:454:99:6684,0,6366&#xA;4   1803824 rs2305184   C   G   2030.6  PASS    AC=1;AF=0.5;AN=2;BaseQRankSum=8.083;ClippingRankSum=0;DB;DP=124;ExcessHet=3.0103;FS=6.128;MLEAC=1;MLEAF=0.5;MQ=60;MQRankSum=0;QD=16.51;ReadPosRankSum=0.18;SOR=0.096    GT:AD:DP:GQ:PL  0/1:62,61:123:99:2038,0,1766&#xA;4   1805296 rs3135883   G   A   3876.03 PASS    AC=2;AF=1;AN=2;DB;DP=110;ExcessHet=3.0103;FS=0;MLEAC=2;MLEAF=1;MQ=60;QD=29.22;SOR=9.401 GT:AD:DP:GQ:PL  1/1:0,109:109:99:3890,326,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edited&lt;/strong&gt; (Dec 2020): bcftools filter syntax has been updated, answer now reflects changes found in version v1.11 (not sure at what point these were introduced).  The old query &lt;code&gt;'AD[1-] &amp;gt; 10'&lt;/code&gt; would produce the error &lt;code&gt;The FORMAT tag AD can have multiple subfields, run as AD[sample:subfield]&lt;/code&gt;. A discussion of these changes can be found &lt;a href=&quot;https://github.com/samtools/bcftools/issues/757&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="1769" LastEditDate="2020-12-03T20:45:58.427" LastActivityDate="2020-12-03T20:45:58.427" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="989" PostTypeId="1" AcceptedAnswerId="2396" CreationDate="2017-07-05T06:55:02.657" Score="7" ViewCount="760" Body="&lt;p&gt;I'd like to simulate 10% sequencing error using &lt;code&gt;art_illumina&lt;/code&gt;. The simulator doesn't have a parameter that I can just give the 10%, but it has this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-qs  --qShift   the amount to shift every first-read quality score by&#xA;-qs2 --qShift2  the amount to shift every second-read quality score by&#xA;                NOTE: For -qs/-qs2 option, a positive number will shift up &#xA;                quality scores (the max is 93) that reduce substitution sequencing &#xA;                errors and a negative number will shift down quality scores that &#xA;                increase sequencing errors. If shifting scores by x, the error&#xA;                rate will be 1/(10^(x/10)) of the default profile.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Heng Li's &lt;code&gt;wgsim&lt;/code&gt; has an option for base error rate. Can I do the same for &lt;code&gt;art_illumina&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q:&lt;/strong&gt; Does the &quot;error rate&quot; in the description mean sequencing error? If I set both &lt;code&gt;qShift&lt;/code&gt; and &lt;code&gt;qShift2&lt;/code&gt; to 10, &lt;code&gt;1/(10^(x/10)) == 0.1, where x is 10&lt;/code&gt;, does that mean 10 is the value I should give to the simulator?&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="298" LastEditDate="2017-07-05T08:31:34.743" LastActivityDate="2017-08-29T00:01:24.383" Title="How to simulate &quot;base error rate&quot; in art_illumina?" Tags="|ngs|simulated-data|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="990" PostTypeId="1" CreationDate="2017-07-05T08:56:46.243" Score="2" ViewCount="67" Body="&lt;p&gt;I am interested in collecting the reference genomes of marine protists and would like to know which databases can be used for such tasks, and is there a single, centralized database of protistological genomic data?&lt;/p&gt;&#xA;" OwnerUserId="369" LastEditorUserId="369" LastEditDate="2017-07-05T09:24:30.257" LastActivityDate="2017-07-05T09:28:22.160" Title="Genome databases of microbial eukaryotes" Tags="|database|genome|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="991" PostTypeId="2" ParentId="990" CreationDate="2017-07-05T09:28:22.160" Score="2" Body="&lt;p&gt;You can find many protist genomes on ensembl&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://protists.ensembl.org/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://protists.ensembl.org/index.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2017-07-05T09:28:22.160" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="992" PostTypeId="2" ParentId="972" CreationDate="2017-07-05T10:01:28.437" Score="2" Body="&lt;p&gt;That does not appear to be possible using PFAM. I would suggest trying a different database. I managed to perform the same query using SMART (&lt;a href=&quot;http://smart.embl-heidelberg.de/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://smart.embl-heidelberg.de/&lt;/a&gt;), via entering &quot;Pfam:FAD_binding_2 AND Pfam:Succ_DH_flav_C&quot; under &quot;Domain selection&quot; in the &quot;Architecture Analysis&quot; section of the search screen, which resulted in 9921 protein sequences with those two domains as you can see &lt;a href=&quot;http://smart.embl-heidelberg.de/smart/selective.cgi?domains=Pfam%3AFAD_binding_2+AND+Pfam%3ASucc_DH_flav_C&amp;amp;terms=&amp;amp;taxon_text=&amp;amp;input=Architecture+query&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the results screen you can select all the sequences and under &quot;Action&quot; at the top of the screen choose &quot;Download Protein Sequences as FASTA file&quot;.&lt;/p&gt;&#xA;" OwnerUserId="369" LastActivityDate="2017-07-05T10:01:28.437" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="993" PostTypeId="2" ParentId="972" CreationDate="2017-07-05T12:00:40.840" Score="2" Body="&lt;p&gt;I believe you can do this using SQL files of the database.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First you need to get the integer accession of this architecture. &lt;a href=&quot;ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/database_files/architecture.txt.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;architecture&lt;/code&gt; table&lt;/a&gt; has the following columns:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;auto_architecture&lt;/code&gt; (integer id, &lt;strong&gt;this is what we need&lt;/strong&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;architecture&lt;/code&gt; (a string describing the architecture)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;type_example&lt;/code&gt; (example sequence)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;no_seqs&lt;/code&gt; (number of sequences)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;architecture_acc&lt;/code&gt; (accessions of the architectures)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You need the following line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;3900719357      FAD_binding_2~Succ_DH_flav_C    Z9JRB3  7471    PF00890 PF02910&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now you need to get the sequences with this architecture id. First, download the &lt;a href=&quot;ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/database_files/pfamseq.txt.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;file with all the sequences&lt;/a&gt; (&lt;strong&gt;warning&lt;/strong&gt;, this one is more than 7Gb). With this one you need column #1 (pfam sequence accesion), #12 (sequence itself) and #16 (architecture integer id).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do not want to import files in the database, you can use &lt;code&gt;awk&lt;/code&gt; to export all the sequences to &lt;code&gt;fasta&lt;/code&gt; format.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;zcat pfamseq.txt.gz | awk -F\\t '{ if ($16==&quot;3900719357&quot;) print &quot;&amp;gt;&quot; $1 &quot;\n&quot; $12}' &amp;gt; sequences.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Of couse, if you need to perform multiple queries like this, it is much easier to import tables into the database. Then you can perform both queries in a single SQL operation (assuming you know the architecture string):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sql prettyprint-override&quot;&gt;&lt;code&gt;SELECT pfamseq_acc, sequence&#xA;    FROM architecture&#xA;    JOIN pfamseq ON pfamseq.auto_architecture=architecture.auto_architecture&#xA;    WHERE architecture=&quot;FAD_binding_2~Succ_DH_flav_C&quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(I haven't tried this SQL query, so maybe you need to correct it a bit)&lt;/p&gt;&#xA;" OwnerUserId="191" LastEditorUserId="191" LastEditDate="2017-07-05T14:35:18.587" LastActivityDate="2017-07-05T14:35:18.587" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="994" PostTypeId="1" CreationDate="2017-07-05T14:20:21.857" Score="8" ViewCount="494" Body="&lt;p&gt;I have 446 whole Klebsiella Pneumoniae genomes I want to build a phylogenetic tree from. After reading about constructing phylogenetic trees it seems the only option for large numbers of genomes is to isolate a gene with low variability from generation to generation and use this gene to build a tree. For example Lars Jensen recommends using &quot;16S rRNA [or] all ribosomal-protein-coding genes&quot; &lt;a href=&quot;https://www.biostars.org/p/1930/&quot; rel=&quot;noreferrer&quot;&gt;https://www.biostars.org/p/1930/&lt;/a&gt;. What program isolates these genes of interest from the whole genome fasta files and can put them into a multiple alignment file? Or outputs them in a formate ready for a multiple alignment program such as Muave? The reason I say a multiple alignment file is because this it the type of file most phylogenetic tree programs take (I.E. clonalframe).&lt;/p&gt;&#xA;" OwnerUserId="842" LastEditorUserId="3848" LastEditDate="2019-08-21T13:27:50.990" LastActivityDate="2019-08-21T13:27:50.990" Title="How to isolate genes from whole genomes for phylogenetic tree analysis?" Tags="|gene|phylogenetics|phylogeny|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="995" PostTypeId="2" ParentId="994" CreationDate="2017-07-05T14:50:16.667" Score="3" Body="&lt;p&gt;There are a lot of ways to do this. I suggest using Prokka/Roary to produce a core genome alignment. There's a useful tutorial on the &lt;a href=&quot;https://sanger-pathogens.github.io/Roary/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Roary&lt;/a&gt; website:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for file in *&#xA;do&#xA;    prokka --kingdom Bacteria --outdir &quot;${file%%.*}&quot;  --genus Listeria --locustag &quot;${file%%.*}&quot; &quot;$file&quot;&#xA;    mv &quot;${file%%.*}&quot;/PROKKA_07052017.gff GFF/&quot;${file%%.*}&quot;.gff # use current date&#xA;done&#xA;roary -f Alignment -e -n -v GFF/*.gff&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Alignment/core_gene_alignment.aln can be used as input for phylogenetic analyses&lt;/p&gt;&#xA;" OwnerUserId="581" LastEditorUserId="298" LastEditDate="2017-07-05T20:39:21.993" LastActivityDate="2017-07-05T20:39:21.993" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="996" PostTypeId="2" ParentId="994" CreationDate="2017-07-05T14:54:52.480" Score="3" Body="&lt;p&gt;Extract desired gene sequences using &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;standalone blast&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simply provide a reference database with your desired output. &#xA;Set up your command and away you go. You can set the search up with a for loop for a batch of sequences. Command may look like &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for f in *.fasta; do&#xA;   f=$(basename $f .fasta)&#xA;   blastn \&#xA;   -outfmt &quot;6 sseqid qseq %&quot; \&#xA;   -query $f.fasta \&#xA;   -subject reference.fna \&#xA;   &amp;gt; out/$f.fas&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Watch the output as blast will spit out the gene as detected in + or - sense. If you want to only gather positive sense use &lt;code&gt;-strand&lt;/code&gt;option. &#xA;The default output I have here is tab output which requires a few sed commands to make into fasta.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -i \&#xA;-e 's/\s*$//g' \&#xA;-e 's/^/&amp;gt;/g'  \&#xA;-e 's/\s\+/\n/g' \&#xA;*.fas &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Online alignment servers are an easy way to align small datasets e.g. &lt;a href=&quot;http://www.ebi.ac.uk/Tools/msa/&quot; rel=&quot;nofollow noreferrer&quot;&gt;EBI&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="982" LastEditorUserId="982" LastEditDate="2017-07-05T20:13:16.533" LastActivityDate="2017-07-05T20:13:16.533" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="997" PostTypeId="1" CreationDate="2017-07-05T14:55:12.183" Score="10" ViewCount="3682" Body="&lt;p&gt;Sometimes it useful to perform a nucleotide protein coding gene sequence alignment based on codons, not on individual nucleotides. For example for further codon model analysis it is important to have full codons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A widely used approach here is to perform a protein sequence alignment first and then impose this alignment to the nucleotide sequences using &lt;a href=&quot;http://www.bork.embl.de/pal2nal/&quot; rel=&quot;noreferrer&quot;&gt;PAL2NAL&lt;/a&gt;, &lt;a href=&quot;http://wbiomed.curtin.edu.au/bioinf/CodonAlign.php&quot; rel=&quot;noreferrer&quot;&gt;CodonAlign&lt;/a&gt; or something similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how &lt;a href=&quot;https://dx.doi.org/10.1186/1471-2105-6-156&quot; rel=&quot;noreferrer&quot;&gt;transAlign&lt;/a&gt; or &lt;a href=&quot;http://guidance.tau.ac.il/ver2/&quot; rel=&quot;noreferrer&quot;&gt;GUIDANCE&lt;/a&gt; (in codon mode) work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem here is that you are discarding part of the information which could be potentially used for the sequence alignment. E.g. if you have slowly evolving low-complexity region adjacent to a quickly evolving one, the amino acid induced alignment could be wrong, while incorporating nucleotide sequence potentially allows to make the alignment more accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm aware of two programs which can do true codon alignment. First, &lt;a href=&quot;http://wasabiapp.org/software/prank/&quot; rel=&quot;noreferrer&quot;&gt;PRANK&lt;/a&gt; has a dedicated codon model, but it is rather slow and using it is overkill for certain problems. Second, &lt;a href=&quot;http://www.bioinformatics.org/sms2/pairwise_align_codons.html&quot; rel=&quot;noreferrer&quot;&gt;Sequence Manipulation Suite&lt;/a&gt; can perform codon alignments, but only for a pair of sequences; also it's javascript based, therefore it is hard to run it for a large number of sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you recommend any software for multiple codon sequence alignment? Preferably available for offline use.&lt;/p&gt;&#xA;" OwnerUserId="191" LastEditorUserId="3848" LastEditDate="2020-11-30T20:26:52.883" LastActivityDate="2020-11-30T20:26:52.883" Title="Which sequence alignment tools support codon alignment?" Tags="|sequence-alignment|covid-19|codon|sars-cov-2|virulign|" AnswerCount="3" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="998" PostTypeId="1" AcceptedAnswerId="2284" CreationDate="2017-07-05T15:23:26.553" Score="6" ViewCount="118" Body="&lt;p&gt;I'm looking for a lightweight knowledgebase describing the human body to annotate disease sites. &#xA;I do not need a great level of detail, I just need a kind of basic organs/sub-organs taxonomy. I checked out the &lt;a href=&quot;http://si.washington.edu/projects/fma&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Foundational Model of Anatomy ontology&lt;/em&gt;&lt;/a&gt;, but it's over-kill for my purposes and really cumbersome (Protege fails when it tries to load FMA.owl).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a simple online knowlegebase available for this?&lt;/p&gt;&#xA;" OwnerUserId="1063" LastEditorUserId="298" LastEditDate="2017-07-05T15:28:58.790" LastActivityDate="2017-09-12T15:34:33.303" Title="Human body sites knowledge base" Tags="|database|ontology|" AnswerCount="2" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="999" PostTypeId="2" ParentId="998" CreationDate="2017-07-05T15:33:55.510" Score="3" Body="&lt;p&gt;Search in &lt;a href=&quot;https://bioportal.bioontology.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioportal&lt;/a&gt;: For example:  &lt;strong&gt;&lt;a href=&quot;https://bioportal.bioontology.org/ontologies/SNOMEDCT?p=summary&quot; rel=&quot;nofollow noreferrer&quot;&gt;SNOMED&lt;/a&gt;&lt;/strong&gt;, &lt;strong&gt;&lt;a href=&quot;https://bioportal.bioontology.org/ontologies/UBERON?p=summary&quot; rel=&quot;nofollow noreferrer&quot;&gt;UBERON&lt;/a&gt;&lt;/strong&gt;.&#xA;Please, note that &lt;strong&gt;&lt;a href=&quot;https://bioportal.bioontology.org/ontologies/FMA?p=summary&quot; rel=&quot;nofollow noreferrer&quot;&gt;FMA&lt;/a&gt;&lt;/strong&gt; is also available as RDF, csv , etc...  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also try to convert FMA to obo and try to open it with &lt;a href=&quot;http://oboedit.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OBO-edit&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="71" LastEditorUserId="48" LastEditDate="2017-09-12T15:34:33.303" LastActivityDate="2017-09-12T15:34:33.303" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1000" PostTypeId="2" ParentId="982" CreationDate="2017-07-05T20:28:07.993" Score="1" Body="&lt;p&gt;I've used PropKa in the past to get isoelectric points. Pretty simple to use:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/jensengroup/propka-3.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/jensengroup/propka-3.1&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="929" LastActivityDate="2017-07-05T20:28:07.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1002" PostTypeId="1" AcceptedAnswerId="1009" CreationDate="2017-07-05T22:18:06.357" Score="4" ViewCount="3523" Body="&lt;p&gt;I was wondering if there is a tutorial or a small code snippet to understand how to write bioinformatics pipeline using python, for example&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;use a aligner (say hisat) &lt;/li&gt;&#xA;&lt;li&gt;get the output and process it using samtools &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I was able to use subprocess from python2.7 for this purpose using samtools but i am not able to link both the processes.i.e given path(which I can use argparse) for directory with fastq files output would be processed bam. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;sample code for samtools sam to bam :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import subprocess&#xA;subprocess.run(['samtools','view', '-bS',&#xA;                '../some.sam',&#xA;                '&amp;gt;',&#xA;                '../some.bam'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="926" LastEditorUserId="931" LastEditDate="2017-07-06T11:22:58.393" LastActivityDate="2017-07-06T11:22:58.393" Title="using python to write bioinformatics pipelines tutorial" Tags="|rna-seq|samtools|python|genomics|" AnswerCount="5" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="1003" PostTypeId="2" ParentId="1002" CreationDate="2017-07-05T22:30:23.793" Score="5" Body="&lt;p&gt;BioPython has some good tools for processing reads and alignments.&#xA;&lt;a href=&quot;http://biopython.org/DIST/docs/tutorial/Tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://biopython.org/DIST/docs/tutorial/Tutorial.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a python library wrapping samtools so many of the samtools calls can be used directly as python objects and calls&#xA;&lt;a href=&quot;https://pysam.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pysam.readthedocs.io/en/latest/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would use subprocess to call the aligner and specify the output to a bam file that you have named and then read that bam file with pysam to do the analysis that you are interested in.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you provide more specifics about the analysis or aligners I can help you more but you are in the right track just need to connect the two calls together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import subprocess&#xA;import pysam&#xA;import os &#xA;&#xA;for root, dirs, files in os.walk(rootPath):&#xA;    for filename in fnmatch.filter(files, &quot;.fastq&quot;):&#xA;&#xA;        subprocess.run([`hisat`, `-f`, filename `-o`, `some.bam`])&#xA;&#xA;        #get depth&#xA;        depth = pysam.depth('some.bam')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="964" LastEditorUserId="964" LastEditDate="2017-07-06T02:35:12.113" LastActivityDate="2017-07-06T02:35:12.113" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1004" PostTypeId="2" ParentId="1002" CreationDate="2017-07-06T00:25:58.437" Score="4" Body="&lt;p&gt;If you're just doing alignment and conversion to a sorted BAM file, there's no need to run it through python. A simple pipe on the unix command line works just as well (and probably runs faster):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;hisat2 -x genome.index -1 reads_R1.fq.gz -2 reads_R2.fq.gz | &#xA;  samtools sort &amp;gt; reads_vs_genome.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-07-06T02:51:18.897" LastActivityDate="2017-07-06T02:51:18.897" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1005" PostTypeId="2" ParentId="969" CreationDate="2017-07-06T07:35:34.347" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://groups.google.com/d/msg/pasapipeline-users/cyG9e8-HjG8/jSwb9o4kAAAJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;Answer from the authors&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;PASA ends up using gff3 format instead of sam format for uploading the&#xA;  alignments.  If somebody has the gmap gff3 output, then can be done to&#xA;  just upload that directly using the custom alignment importer, but not&#xA;  with sam. They suggest just having PASA rerun GMAP as part of its&#xA;  regular routine.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="294" LastEditorUserId="48" LastEditDate="2017-12-04T21:15:12.990" LastActivityDate="2017-12-04T21:15:12.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1006" PostTypeId="2" ParentId="1002" CreationDate="2017-07-06T09:00:02.707" Score="0" Body="&lt;p&gt;Have a look at &lt;a href=&quot;https://github.com/spotify/luigi&quot; rel=&quot;nofollow noreferrer&quot;&gt;luigi&lt;/a&gt; for writing pipelines in python. You can found BioLuigi if you like :-P&lt;/p&gt;&#xA;" OwnerUserId="931" LastActivityDate="2017-07-06T09:00:02.707" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="1007" PostTypeId="1" AcceptedAnswerId="2050" CreationDate="2017-07-06T09:03:18.220" Score="4" ViewCount="324" Body="&lt;p&gt;I have a list of about 50k 'Protein IDs' from Reactome. Is there a simple way to get all the corresponding 'Pathway IDs' for each protein? What is the best service to use? (I'm guessing I can use the Reactome API, but I don't necessarily want to hit that 50k times...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Protein IDs from reactome look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R-HSA-49155&#xA;R-HSA-199420&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The corresponding Reactome 'Pathway IDs' for those Protein IDs would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R-HSA-49155  R-HSA-110331&#xA;R-HSA-49155  R-HSA-110330&#xA;R-HSA-49155  R-HSA-110357&#xA;R-HSA-199420 R-HSA-1660499&#xA;R-HSA-199420 R-HSA-202424&#xA;R-HSA-199420 R-HSA-199418&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="931" LastEditorUserId="931" LastEditDate="2017-07-07T09:00:10.593" LastActivityDate="2017-07-13T09:04:28.000" Title="Convert Reactome Protein IDs to Pathway IDs?" Tags="|conversion|identifiers|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="1008" PostTypeId="2" ParentId="1002" CreationDate="2017-07-06T09:12:57.450" Score="13" Body="&lt;p&gt;Taking a different tack from other answers, there's lots of tools for pipelines in Python. Note: there was a time when people would use &quot;pipeline&quot; to refer to a shell script. I'm talking about something more sophisticated that helps you decompose an analysis into parts and  runs it robustly.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Snakemake&lt;/strong&gt; is my favourite. It's (nearly) pure Python and can generate reports.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Nextflow&lt;/strong&gt; is growing in popularity and is pretty straightforward&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Ruffus&lt;/strong&gt; used to be reasonably popular, seemed fine when I used it&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Bpipe&lt;/strong&gt; is for bioinformatics&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Airflow&lt;/strong&gt; is a more industrial &quot;big&quot; solution but in wide use&lt;/li&gt;&#xA;&lt;li&gt;See this &lt;a href=&quot;https://github.com/pditommaso/awesome-pipeline&quot; rel=&quot;noreferrer&quot;&gt;big list of pipeline systems&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'm sure you can find something there.&lt;/p&gt;&#xA;" OwnerUserId="377" LastActivityDate="2017-07-06T09:12:57.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1009" PostTypeId="2" ParentId="1002" CreationDate="2017-07-06T09:53:12.087" Score="4" Body="&lt;p&gt;I &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/1008/191&quot;&gt;agree&lt;/a&gt; that using a specialized tool is probably a good idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless if you want to stick with Python, I suggest using &lt;a href=&quot;https://plumbum.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;plumbum&lt;/code&gt;&lt;/a&gt; instead of &lt;code&gt;subprocess&lt;/code&gt;. It has a very nice syntax for this kind of problems.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;from plumbum import local&#xA;&#xA;# load a command&#xA;samtools = local['samtools']&#xA;&#xA;# you can then easily redirect&#xA;(samtools['view', '-bS', '../some.sam'] &amp;gt; '../some.bam')()&#xA;&#xA;# it is also easy to pipe &#xA;hisat2 = local['hisat2']&#xA;chain = hisat2['-x', 'genome.index', '-1', 'reads_R1.fq.gz', '-2', 'reads_R2.fq.gz'] | samtools['sort'] &amp;gt; 'reads_vs_genome.bam'&#xA;&#xA;chain()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="191" LastEditorUserId="191" LastEditDate="2017-07-06T11:16:07.240" LastActivityDate="2017-07-06T11:16:07.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1010" PostTypeId="1" AcceptedAnswerId="1011" CreationDate="2017-07-07T10:02:31.870" Score="7" ViewCount="325" Body="&lt;p&gt;I have this read in my BAM file. It maps on chromosome 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I open this BAM file in IGV, and I can see the alignment on chromosome 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when I open this file in R with Rsamtools:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bamContigsCel &amp;lt;- Rsamtools::scanBam('output/alignment/pacbio/bwa/ref     /bristolAssemblySorted.bam', param = Rsamtools::ScanBamParam(what = Rsamtools::scanBamWhat(), flag = Rsamtools::scanBamFlag(isMinusStrand = FALSE), tag = bamTags))[[1]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I then check if the read maps to chromosome 1 in my R object but I cannot find it.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bamContigsCel$rname[bamContigsCel$qname == '000000F|arrow']&#xA;[1] II II IV&#xA;Levels: I II III IV MtDNA V X&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But if I look at the BAM file, it's there. Why isn't Rsamtools importing my read into R?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ samtools view bristolAssemblySorted.bam | grep -n '000000F' | head  -c80&#xA;000000F|arrow   2064    I   336331  7   2926310H260M1774267H    *   0 0   GAAGCTGTCTAAACTTTGGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Cross-posted on &lt;a href=&quot;https://www.biostars.org/p/261365/&quot; rel=&quot;noreferrer&quot;&gt;biostars&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="77" LastEditDate="2017-07-07T11:46:03.383" LastActivityDate="2017-07-09T20:07:48.103" Title="scanBam from Rsamtools is not importing one of my reads into R" Tags="|r|sam|samtools|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1011" PostTypeId="2" ParentId="1010" CreationDate="2017-07-07T10:35:44.040" Score="4" Body="&lt;p&gt;Note the flag; that read is mapped in a reverse-complemented manner, so &lt;code&gt;isMinusStrand = FALSE&lt;/code&gt; is filtering it out. I tested this by making a BAM file with only that read:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@SQ SN:I    LN:1000000&#xA;000000F|arrow   2064    I   336331  7   2926310H260M1774267H    *   00  *   *&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then in R:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; library(Rsamtools)&#xA;&amp;gt; blah = scanBam(&quot;foo.bam&quot;, param=ScanBamParam(what=scanBamWhat(), flag=scanBamFlag(isMinusStrand = TRUE)))[[1]]&#xA;&amp;gt; blah$qname&#xA;[1] &quot;000000F|arrow&quot;&#xA;&amp;gt; blah = scanBam(&quot;foo.bam&quot;, param=ScanBamParam(what=scanBamWhat(), flag=scanBamFlag(isMinusStrand = FALSE)))[[1]]&#xA;&amp;gt; blah$qname&#xA;character(0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastEditorUserId="48" LastEditDate="2017-07-09T20:07:48.103" LastActivityDate="2017-07-09T20:07:48.103" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="1012" PostTypeId="1" AcceptedAnswerId="1013" CreationDate="2017-07-07T12:43:54.240" Score="4" ViewCount="1522" Body="&lt;p&gt;I am dealing with a list of genes which have been selected from a gene enrichment analysis. In order to see what kind of genes are overrepresented, I ran &lt;a href=&quot;https://github.com/jhcepas/eggnog-mapper&quot; rel=&quot;nofollow noreferrer&quot;&gt;eggnog-mapper&lt;/a&gt; to do an orthology assignment of each gene against a bacterial database (provided in eggnog). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After running eggnog you get a .csv file containing for each gene which is the closest ortholog, and a list of GO, KEGG and COG terms associated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From there, I would like to do a Fisher-test (also considering Bonferroni correction) to highlight COG categories overrepresented in our dataset compared to our reference genome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally my question: How do you deal with genes assigned to multiple COG categories (same would apply for GO terms). You count +1 to each of the COG categories assigned? Do you just choose one category per gene? &lt;/p&gt;&#xA;" OwnerUserId="446" LastEditorUserId="298" LastEditDate="2017-07-07T12:45:38.550" LastActivityDate="2017-07-07T12:48:31.647" Title="COG Annotation - Dealing with genes assigned to two or more COG categories" Tags="|sequence-annotation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1013" PostTypeId="2" ParentId="1012" CreationDate="2017-07-07T12:48:31.647" Score="3" Body="&lt;p&gt;You add 1 to the count of each COG category. You are looking for over represented &lt;em&gt;COG categories&lt;/em&gt; so you must count them all. Many genes will be assigned to multiple categories. In fact, a majority of human genes are given multiple GO cellular function annotations. I have worked considerably in this field and one of the surprises I found is how common protein multifunctionality is. But I digress. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point here is that you want to see whether any COG categories are over represented in your test set. Whether a gene is annotated to one or to a hundred categories is irrelevant. You are counting genes per COG, so you need to count each of the COGs assigned to a gene separately. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-07-07T12:48:31.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1014" PostTypeId="1" AcceptedAnswerId="1015" CreationDate="2017-07-07T12:52:14.280" Score="6" ViewCount="296" Body="&lt;p&gt;I'm working with scATAC-Seq data on the K562 cell line, which is supposed to be derived from a female patient. &lt;a href=&quot;https://www.nature.com/article-assets/npg/nature/journal/v523/n7561/extref/nature14590-s1.pdf&quot; rel=&quot;noreferrer&quot;&gt;While following the scATAC-seq data analysis pipeline&lt;/a&gt;, after performing bowtie alignment they recommend filtering out all reads aligned to mitochondrial DNA and on the Y chromosome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Out of curiosity, I decided to count how many reads aligned to chrY and found that it can be quite high - chrY has approximately 10% as many reads as a similarly-sized chromosome (chr19 in hg19).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it normal to find reads mapping to chrY when the cell line is female-derived, or did I mess up somehow?&lt;/p&gt;&#xA;" OwnerUserId="1076" LastActivityDate="2017-07-07T12:58:00.847" Title="Y Chromosome Aligned Reads in scATAC-seq data from a female-derived cell line?" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1015" PostTypeId="2" ParentId="1014" CreationDate="2017-07-07T12:58:00.847" Score="7" Body="&lt;p&gt;There are homologous regions between X an Y chromosomes: &lt;a href=&quot;https://en.wikipedia.org/wiki/Pseudoautosomal_region&quot; rel=&quot;noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Pseudoautosomal_region&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is therefore normal to have some female-derived reads mapping in Y chromosome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should probably check what proportion of such reads fall in other parts of the Y chromosome than pseudoautosomal regions.&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2017-07-07T12:58:00.847" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1016" PostTypeId="1" CreationDate="2017-07-07T19:10:32.453" Score="4" ViewCount="67" Body="&lt;p&gt;Does anyone know if there is a program/library/script in R or Python that takes as input a list of proteins/peptides and a list of post-translational modifications (PTMs; like oxidation of methionine and acetylation of cysteine), and as output returns a matrix/list with the imported peptides and all possible locations of the selected modifications?&lt;/p&gt;&#xA;" OwnerUserId="704" LastEditorUserId="96" LastEditDate="2017-07-08T11:43:12.400" LastActivityDate="2017-07-08T12:50:08.357" Title="Software to produce a table of post-translational modifications from a peptide list" Tags="|r|proteins|python|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="1017" PostTypeId="2" ParentId="1016" CreationDate="2017-07-08T12:50:08.357" Score="2" Body="&lt;p&gt;I don't know of a tool that can do exactly what you want. However, there are various tools that can predict specific classes of post translational modifications. For example:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Phosphorylation: &lt;a href=&quot;http://gps.biocuckoo.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GPS&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Computational prediction of phosphorylation sites with their cognate protein kinases (PKs) is greatly helpful for further experimental design. Although ~10 online predictors were developed, the PK classification and control of false positive rate (FPR) were not well addressed. Here we adopted a well-established rule to classify PKs into a hierarchical structure with four levels.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Mitochondrial import: &lt;a href=&quot;http://ihg.gsf.de/ihg/mitoprot.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mitoprot&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;MitoProt calculates the N-terminal protein region that can support a Mitochondrial Targeting Sequence and the cleavage site. A complete description of the method to make the prediction is available in: M.G. Claros, P. Vincens. Computational method to predict mitochondrially imported proteins and their targeting sequences. Eur. J. Biochem. 241, 779-786 (1996).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Acetylation: &lt;a href=&quot;http://bdmpail.biocuckoo.org/prediction.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;PAIL: Prediction of Acetylation on Internal Lysines&amp;quot;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In this work, we present a novel online predictor for protein acetylation sites prediction of PAIL, Prediction of Acetylation on Internal Lysines. We have manually mined scientific literature to collect 249 experimentally verified acetylation sites of 92 distinct proteins. Then the BDM (Bayesian Discriminant Method) algorithm has been employed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;You can use tools like the above and then combine the results to get the sort of matrix you are asking for.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-07-08T12:50:08.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="1018" PostTypeId="1" CreationDate="2017-07-08T13:35:12.137" Score="7" ViewCount="104" Body="&lt;p&gt;I'm using velvet to align given reads of RNA to given CDSs (i.e. coding areas and genes) of an organism, so I can generate gene-expression profiles. But after using &lt;code&gt;velvetg out-dir/ -alignment yes&lt;/code&gt;, velvet produces &lt;code&gt;contig-alignment.psa&lt;/code&gt; which is a file in a strange format. &#xA;The file for every contig contains zero or more records of 7 integers, and I cannot understand their meaning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is an excerpt of the file, and you can see the entire file &lt;a href=&quot;https://paste.ofcode.org/d9599cGH3aXnbdGqtWqrMs&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;contig_1&#xA;1    193    1939    1939    6   194 2&#xA;195 244 1   1   6   22  71&#xA;&amp;gt;contig_2&#xA;1   84  1   1   6   59  142&#xA;86  170 1935    1935    6   22  106&#xA;172 285 1935    1935    6   108 221&#xA;&amp;gt;contig_3&#xA;1   98  2   2   6   1   98&#xA;100 321 2   2   6   100 321&#xA;334 415 1204    1204    6   1   82&#xA;&amp;gt;contig_15&#xA;1   23  3   3   6   84  106&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="872" LastEditorUserId="96" LastEditDate="2017-08-31T03:23:35.837" LastActivityDate="2018-11-18T08:39:39.300" Title="How to interpret contig-alignment.psa produced by velvet" Tags="|read-mapping|rna-alignment|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="1019" PostTypeId="1" AcceptedAnswerId="2019" CreationDate="2017-07-08T14:24:13.377" Score="6" ViewCount="131" Body="&lt;p&gt;I'm trying to compute a gene expression profile for an organism. I have gene nucleotide sequences of the mentioned organism stored in a fasta file and a set of paired reads stored in two separate files with the same fasta format. Now I want to compute the coverage of every gene by the given reads. That is, on average how many reads cover a given segment of the sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any tool for this specific purpose?&#xA;I'm trying to use the velvet sequencer but I have encountered a &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/1018/how-to-interpret-contig-alignment-psa-produced-by-velvet&quot;&gt;few problems&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="872" LastEditorUserId="872" LastEditDate="2017-07-08T19:08:23.327" LastActivityDate="2017-07-08T21:50:31.803" Title="How can I compute gene expression for a set of RNA reads?" Tags="|rna-seq|rna-alignment|" AnswerCount="3" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2019" PostTypeId="2" ParentId="1019" CreationDate="2017-07-08T16:22:43.643" Score="8" Body="&lt;p&gt;It's unclear if your paired-end reads are actually in fasta format, I'll presume that they're in fastq instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest tool to use is &lt;a href=&quot;http://salmon.readthedocs.io/en/latest/&quot; rel=&quot;noreferrer&quot;&gt;salmon&lt;/a&gt;, which nicely deals with things like multimapping. If you're trying to judge the quality of an assembly, then I recommend having a look at &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/27252236&quot; rel=&quot;noreferrer&quot;&gt;transrate&lt;/a&gt;, which uses some related methodology for assessing contig quality.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-07-08T16:22:43.643" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2020" PostTypeId="2" ParentId="1019" CreationDate="2017-07-08T19:17:51.123" Score="3" Body="&lt;p&gt;In addition to salmon that was already mentioned, you can also try &lt;a href=&quot;https://pachterlab.github.io/kallisto/starting&quot; rel=&quot;nofollow noreferrer&quot;&gt;kallisto&lt;/a&gt; or &lt;a href=&quot;https://github.com/bli25ucb/RSEM_tutorial&quot; rel=&quot;nofollow noreferrer&quot;&gt;RSEM&lt;/a&gt;, which are also fairly popular/respectable and will work with a transcriptome FASTA.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2017-07-08T19:17:51.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2021" PostTypeId="2" ParentId="1019" CreationDate="2017-07-08T21:50:31.803" Score="2" Body="&lt;p&gt;It's not possible to compute absolute expression from RNASeq reads if they are processed in the usual way, where a sequencer produces the same number of reads regardless of the input RNA amount. At best, RNASeq will give you an indication of &lt;a href=&quot;http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1004075&quot; rel=&quot;nofollow noreferrer&quot;&gt;proportional expression&lt;/a&gt; within a single sample. For this reason, relative expression (i.e. that used by differential expression tests) is easier to determine than absolute expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The closest approximation to absolute expression is to generate an expression relative to the average expression of a set of housekeeping genes, but I don't think there's a universal set that has been decided on. Gene expression, even for common housekeeping genes, can vary depending on the environmental conditions of the cell. For example, GAPDK is involved in &lt;a href=&quot;http://www.nature.com/nri/journal/v16/n9/full/nri.2016.70.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;immune cell activation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as long as experimental conditions are similar, and you're not planning on looking for statistical significance, the proportional expression can still give qualitative insights into how cell populations behave in relation to other populations. DESeq2 provides a &lt;a href=&quot;https://rdrr.io/bioc/DESeq2/man/varianceStabilizingTransformation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;variance-stabilising transformation function&lt;/a&gt; that minimises variation for small-count genes, assuming that each sample has roughly the same total expression. I have found that I get better outcomes / comparisons from this transformation when carrying out a further adjustment to account for gene length (i.e. divide by the length of the longest transcript for each gene). See our &lt;a href=&quot;http://jem.rupress.org/content/early/2016/12/01/jem.20160470#materials-methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;Th2 paper&lt;/a&gt;, section &quot;Read mapping and differential expression analysis&quot; for more information. The &quot;transcripts-per-million&quot; values produced by &lt;a href=&quot;https://pachterlab.github.io/kallisto/about.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kallisto&lt;/a&gt; and &lt;a href=&quot;https://combine-lab.github.io/salmon/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Salmon&lt;/a&gt; provide measures similar to this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If, on the other hand, you were able to modify the experimental design, single cell sequencing (or &quot;known cell count&quot; sequencing) can be used for determining absolute expression: use a spiked-in transcript that is added in proportion to the &lt;em&gt;cell count&lt;/em&gt;, so that results can be compared in proportion to the expression for that transcript.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-07-08T21:50:31.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2022" PostTypeId="1" AcceptedAnswerId="2030" CreationDate="2017-07-09T03:06:16.293" Score="5" ViewCount="216" Body="&lt;p&gt;I have been searching for a long time and the furthest I got is some database with the functional description for genes. Then, I have to parse these descriptions manually to figure out the association. Moreover, for some species, there are not even any functional descriptions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need this information for mice, rice, and Arabidopsis Thaliana. Can anyone help me? &lt;/p&gt;&#xA;" OwnerUserId="1082" LastEditorUserId="57" LastEditDate="2017-07-09T16:27:02.363" LastActivityDate="2017-07-11T09:10:06.243" Title="Where can I find a database that has phenotype information together with associated SNPs?" Tags="|gene|gwas|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2023" PostTypeId="2" ParentId="974" CreationDate="2017-07-09T19:15:01.990" Score="4" Body="&lt;p&gt;You can do this in &lt;a href=&quot;https://hail.is&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hail&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from hail import *&#xA;hc = HailContext()&#xA;(hc.import_vcf('test.vcf')&#xA; .filter_variants_expr('gs.exists(g =&amp;gt; g.ad[1:].exists(d =&amp;gt; d &amp;gt; 10))')&#xA; .export_vcf('filtered.vcf'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This works with any number of samples and will keep the variants where at least one sample has a genotype with an alternate allele support by more than 10 reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To verify we got the expected 6 variants:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; hc.import_vcf('filtered.vcf').count()&#xA;(1L, 6L)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://hail.is/hail/hail.VariantDataset.html#hail.VariantDataset.count&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;count&lt;/code&gt;&lt;/a&gt; returns the number of samples (1) and number of variants (6).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take a look at the &lt;a href=&quot;https://hail.is/hail/getting_started.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;getting started page&lt;/a&gt; or &lt;a href=&quot;https://hail.is/hail/tutorials-landing.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorials&lt;/a&gt; if you want to try it out!&lt;/p&gt;&#xA;" OwnerUserId="1024" LastActivityDate="2017-07-09T19:15:01.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2024" PostTypeId="1" CreationDate="2017-07-09T21:31:41.673" Score="2" ViewCount="72" Body="&lt;p&gt;Are there any examples of researchers data mining articles and papers (e.g., from pubmed or google scholar) in order to derive relations between genes and diseases?  &lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="77" LastEditDate="2017-07-09T21:59:22.660" LastActivityDate="2017-07-09T22:50:16.190" Title="Using data mining of papers in order to derive genomic connections" Tags="|gene|genome|cancer|data-mining|" AnswerCount="2" CommentCount="9" ClosedDate="2017-07-09T23:00:13.187" ContentLicense="CC BY-SA 3.0" />
  <row Id="2025" PostTypeId="2" ParentId="2024" CreationDate="2017-07-09T22:32:46.213" Score="4" Body="&lt;p&gt;There are many databases that have used publication scraping for oncogenic gene fusions. There are publications for the individual methods they used for their scraping and aggregation.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;COSMIC - &lt;a href=&quot;http://cancer.sanger.ac.uk/cosmic&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cancer.sanger.ac.uk/cosmic&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;TICdb - &lt;a href=&quot;http://www.unav.es/genetica/TICdb/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.unav.es/genetica/TICdb/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;ChimerDB - &lt;a href=&quot;http://ercsb.ewha.ac.kr/fusiongene/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ercsb.ewha.ac.kr/fusiongene/&lt;/a&gt; -&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/27899563&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pubmed/27899563&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="964" LastActivityDate="2017-07-09T22:32:46.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2026" PostTypeId="2" ParentId="2024" CreationDate="2017-07-09T22:50:16.190" Score="3" Body="&lt;p&gt;This existed as a closed silo, at least in 2015. Qiagen has a team of hired students and Post-Docs for collating research papers into their &lt;a href=&quot;https://www.qiagenbioinformatics.com/products/qiagen-knowledge-base/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Knowledge Base&lt;/a&gt;, an extensive database that is integrated into a few of their commercial products. Qiagen's claim is that by providing a consistently-structured and well-formatted database, the process of research discovery is accelerated. I've got a few notes from a seminar they gave us a couple of years ago:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The curation is better than a free / public database because free databases typically have wrong or inconsistent data&lt;/li&gt;&#xA;&lt;li&gt;Their knowledge base is an army of hired MDs and PhDs, 5.3M &quot;findings&quot; as of Nov 05 2015&lt;/li&gt;&#xA;&lt;li&gt;Scientific articles have unstructured data, so expert knowledge is required to process the data into a &quot;finding&quot; for their knowledge base&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I was already doubtful about their approach, and this doubt was magnified when they started getting into the details of their process. I started to lose trust and write my own thoughts about it:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The knowledge base they have created is private&lt;/li&gt;&#xA;&lt;li&gt;The approach is to use a function (i.e. MDs/PhDs) to convert research articles into scientifically-validated associations. However, the function is effectively a black box; Qiagen allow the experts to decide for themselves how a particular article should be converted into a finding.&lt;/li&gt;&#xA;&lt;li&gt;The findings in the knowledge base have no strength or likelihood.&lt;/li&gt;&#xA;&lt;li&gt;The Experts were encouraged to only enter findings that were &quot;generally accepted by the scientific community. but it seemed like only a single expert was required to get a finding into the knowledge base (i.e. findings weren't curated by other people).&lt;/li&gt;&#xA;&lt;li&gt;When using their software (e.g. IPA), p-values of 10^-15 and 10^-22 are reported; these numbers are meaningless.&lt;/li&gt;&#xA;&lt;li&gt;The team didn't look at methylation, phosphorylation, etc.; plans are in place to hire additional experts to look at this at a later date.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;After listening to their seminar, I realised that Qiagen's idea of a general database that attempts to capture everything isn't going to work well. Experts will disagree about whether a particular association is valid, relevance (or correctness) will differ depending on the area of interest, and the established correctness of a piece of information can change over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be possible to create something that might work by adding a lot more metadata to all the &quot;findings&quot; in such a database, but it would need to be a huge, public, collaborative effort. Something on the scale of SRA/ENA, with encouragement from the scientific community to enter details into the database prior to publications being accepted/indexed. If the people who wrote papers entered their findings in such a database, other researchers wouldn't need to be hired to try to understand papers and guess at the findings of published research.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-07-09T22:50:16.190" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2027" PostTypeId="2" ParentId="2022" CreationDate="2017-07-10T06:56:52.493" Score="4" Body="&lt;p&gt;A few days ago, I was trying to find some GWAS datasets to download. I hope this site for 3000 rice will help: &lt;a href=&quot;http://snp-seek.irri.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://snp-seek.irri.org/&lt;/a&gt;. You can click &lt;a href=&quot;http://snp-seek.irri.org/_snp.zul&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genotype&lt;/a&gt; in the index page for &quot;Query for SNPs from the 3000 genome project&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt; With the help of user manual, I can get IRIS ID from &lt;a href=&quot;http://snp-seek.irri.org/_snp.zul&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genotype&lt;/a&gt;. According to this &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/12855438&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, iris id may associate with some phenotype information. But the website of IRIS is under maintenance, I can not get more information about phenotype.  &lt;/p&gt;&#xA;" OwnerUserId="1085" LastEditorUserId="1085" LastEditDate="2017-07-11T06:26:50.940" LastActivityDate="2017-07-11T06:26:50.940" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2028" PostTypeId="1" AcceptedAnswerId="2033" CreationDate="2017-07-10T09:40:34.390" Score="4" ViewCount="70" Body="&lt;p&gt;MEDIPS is an established tool with functions for the quality control and analysis of data derived from immunoprecipitation (IP)-seq samples, like Methylation IP sequencing datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know if there are any other tools I should consider nowadays that could do a better job and hence replace MEDIPS for the analysis of MeDIP datasets.&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2017-07-11T02:29:51.997" Title="alternatives to MEDIPS to analyse MeDIP datasets" Tags="|methylation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2029" PostTypeId="1" CreationDate="2017-07-10T12:02:01.193" Score="2" ViewCount="61" Body="&lt;p&gt;I'm looking for a Linked Open Data approach to annotate a dataset with human genome and drugs information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the &lt;a href=&quot;http://lod-cloud.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Linked Open Data cloud&lt;/a&gt;, there are a lot of interconnected &lt;code&gt;RDF&lt;/code&gt; vocabularies devoted to life sciences but, unfortunately, a lot of them seem out of date.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recently found &lt;a href=&quot;https://www.wikidata.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikidata&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Wikidata is a free and open knowledge base that can be read and edited&#xA;  by both humans and machines.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It seems exactly what Linked Data should be and it seems continuously updated. At a first sight, it covers very well genomic and drugs knowledge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does Anyone know if Wikidata is a reliable source for genomic and drugs annotation?&lt;/p&gt;&#xA;" OwnerUserId="1063" LastEditorUserId="1063" LastEditDate="2017-07-10T22:59:23.797" LastActivityDate="2017-07-10T22:59:23.797" Title="Is Wikidata a reliable KB for genomic and drugs annotations?" Tags="|sequence-annotation|public-databases|data-management|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2030" PostTypeId="2" ParentId="2022" CreationDate="2017-07-10T16:25:32.420" Score="8" Body="&lt;p&gt;&lt;a href=&quot;http://www.mousephenotype.org/&quot; rel=&quot;noreferrer&quot;&gt;International Mouse Phenotyping Consortium&lt;/a&gt; is building a database of phenotypes and knock-outs of mouse. I believe that this database will be fairly complete (20000 knock-outs), but these are knock-outs, not SNPs... There are several mouse GWAS studies, but I am not aware of a database that would pull all the results together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Arabidopsis&lt;/em&gt; big GWAS project is called &lt;a href=&quot;http://1001genomes.org/tools/&quot; rel=&quot;noreferrer&quot;&gt;1001 genomes project&lt;/a&gt;. A lot more information about &lt;em&gt;Arabidopsis&lt;/em&gt; from individual experiments is collected and maintained by &lt;a href=&quot;https://www.arabidopsis.org/&quot; rel=&quot;noreferrer&quot;&gt;The Arabidopsis Information Resource&lt;/a&gt;, however this resource is subscription based, therefore what you need might not be available for free.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned, SNPs of rice were identified and linked to different rice groups in &lt;a href=&quot;https://gigascience.biomedcentral.com/articles/10.1186/2047-217X-3-7&quot; rel=&quot;noreferrer&quot;&gt;The 3,000 rice genomes project&lt;/a&gt;. There is also &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/27322545&quot; rel=&quot;noreferrer&quot;&gt;GWAS&lt;/a&gt; done on 512 individuals, not sure if it is sufficient for reliable conclusions (even they claim that they found supersigificant links).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this what you were searching for?&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2017-07-10T16:25:32.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2031" PostTypeId="1" CreationDate="2017-07-10T17:05:03.750" Score="5" ViewCount="2008" Body="&lt;p&gt;I need to install &lt;a href=&quot;http://www.ub.edu/dnasp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DnaSp&lt;/a&gt;, a nice GUI originally written for Windows for the population genetics of nucleotide sequence data.&lt;/p&gt;&#xA;&lt;p&gt;However, its not working on my mac. I'm using wine to install it using &lt;a href=&quot;http://www2.ub.es/dnasp/download.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;these instructions for wine&lt;/a&gt;. Wine usually successfully ports Windows programs.&lt;/p&gt;&#xA;&lt;p&gt;it successfully installs but then I get this error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Mollies-MacBook-Air:dnasp51001 molliepassacantando&lt;span class=&quot;math-container&quot;&gt;$ WINEPREFIX=~/.wine64 wine regsvr32.exe scrrun.dll mfc40.dll threed32.ocx&#xA;regsvr32: Successfully registered DLL 'scrrun.dll'&#xA;regsvr32: Successfully registered DLL 'mfc40.dll'&#xA;err:typelib:sltg_get_typelib_ref Unable to find reference&#xA;err:typelib:sltg_get_typelib_ref Unable to find reference&#xA;err:typelib:sltg_get_typelib_ref Unable to find reference&#xA;err:typelib:sltg_get_typelib_ref Unable to find reference&#xA;err:typelib:sltg_get_typelib_ref Unable to find reference&#xA;err:typelib:sltg_get_typelib_ref Unable to find reference&#xA;regsvr32: Successfully registered DLL 'threed32.ocx'&#xA;Mollies-MacBook-Air:dnasp51001 molliepassacantando$&lt;/span&gt; winedevice.exe(30828,0x405ed000) malloc: *** error for object 0x401cea08: pointer being freed was not allocated&#xA;*** set a breakpoint in malloc_error_break to debug&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and cannot run it.&lt;/p&gt;&#xA;&lt;p&gt;Has anyone successfully installed DnaSp on a Mac or know of an alternate program I could use? I need to be able to calculate haplotype frequencies.&lt;/p&gt;&#xA;" OwnerUserId="1088" LastEditorUserId="3848" LastEditDate="2022-02-21T16:37:41.363" LastActivityDate="2024-03-12T21:02:00.360" Title="How to install DnaSP on a Mac" Tags="|genome|phylogenetics|software-installation|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="2032" PostTypeId="2" ParentId="2029" CreationDate="2017-07-10T17:15:42.777" Score="1" Body="&lt;p&gt;The question is what you mean with &quot;reliable&quot;. For bacterial genome annotations there is one Wikidata based platform called &lt;a href=&quot;http://wikigenomes.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;WikiGenomes&lt;/a&gt; (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/28365742&quot; rel=&quot;nofollow noreferrer&quot;&gt;publications&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="579" LastActivityDate="2017-07-10T17:15:42.777" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2033" PostTypeId="2" ParentId="2028" CreationDate="2017-07-11T02:29:51.997" Score="2" Body="&lt;p&gt;I guess QSEA should be an obvious answer:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;QSEA (quantitative sequencing enrichment analysis) was developed as&#xA;  the successor of the MEDIPS package for analyzing data derived from&#xA;  methylated DNA immunoprecipitation (MeDIP) experiments followed by&#xA;  sequencing (MeDIP-seq).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/vignettes/qsea/inst/doc/qsea_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://bioconductor.org/packages/release/bioc/vignettes/qsea/inst/doc/qsea_tutorial.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2017-07-11T02:29:51.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2034" PostTypeId="1" AcceptedAnswerId="2046" CreationDate="2017-07-11T07:03:13.770" Score="4" ViewCount="424" Body="&lt;p&gt;I am trying to reorder scaffolds of a rice species, but no genetic map is available right now. &lt;em&gt;Oryza sativa Japonica&lt;/em&gt; is a close relative of this rice species. Mummer was used to do a whole genome alignment, and I am trying to reorder scaffolds according the mummer result. But I found it's too complicated to do this by myself, many scaffolds may align to multiple Oryza sativa Japonica chromosomes or multiple loci in the same chromosome.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What software can I use to handle this task?&lt;/p&gt;&#xA;" OwnerUserId="1085" LastEditorUserId="298" LastEditDate="2017-07-11T08:49:31.670" LastActivityDate="2017-07-11T15:15:50.387" Title="Reordering scaffolds according to a reference without a genetic map" Tags="|sequence-alignment|assembly|software-recommendation|scaffold|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2035" PostTypeId="2" ParentId="2034" CreationDate="2017-07-11T07:17:43.397" Score="5" Body="&lt;p&gt;Have you tried &lt;a href=&quot;http://darlinglab.org/mauve/mauve.html&quot; rel=&quot;noreferrer&quot;&gt;Mauve&lt;/a&gt; alignment? Its pretty easy once you become familiar and has a GUI for further ease of use. Additionally there are a few online tutorials on how to re-order contigs/ scaffolds using this software. Heres &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2723005/&quot; rel=&quot;noreferrer&quot;&gt;one I use&lt;/a&gt; when in need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mauve contains a function called &lt;a href=&quot;http://darlinglab.org/mauve/user-guide/reordering.html&quot; rel=&quot;noreferrer&quot;&gt;Mauve Contig Mover (MCM)&lt;/a&gt; which can be used to a) compare an assembly to a reference and/or b) rearrange contigs to improve assembly quality.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use the tutorial as above which uses the GUI for Mauve. I&lt;/p&gt;&#xA;" OwnerUserId="982" LastEditorUserId="982" LastEditDate="2017-07-11T09:55:52.823" LastActivityDate="2017-07-11T09:55:52.823" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2036" PostTypeId="1" AcceptedAnswerId="2039" CreationDate="2017-07-11T07:47:50.980" Score="6" ViewCount="510" Body="&lt;p&gt;Looking for tools to reconcile alignment file of experimental transcripts mapped to genome (SAM/BAM) with the reference transcriptome annotation (GTF) from Ensembl (organism: &lt;em&gt;D. melanogaster&lt;/em&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The aim would be to check which transcripts reported in the alignment file are also reported in the reference annotation, generating a new annotation (e.g., GTF) including the new transcripts as seen in the alignment file (experimental transcripts to genome).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- Edit: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Up to now, I tried an in-house script that does the following: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;generate a BED file from the BAM file with the exon coordinates (start/end) per transcript - using &lt;a href=&quot;https://github.com/bedops/bedops&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS v2p4p27&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;convert the reference GTF to BED to retrieve the coordinates per annotated feature &lt;/li&gt;&#xA;&lt;li&gt;compare the experimental vs the reference BED intervals&lt;/li&gt;&#xA;&lt;li&gt;infer the suite of annotated exons per experimental transcript, thus, ideally, the isoform it corresponds to&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;However, with this simple approach, I only get around 20% of the experimental transcripts that can be identified in all their length using reference annotation. This might be due either to a too simple approach (e.g. if the start/end of the exon in the experimental transcript is shifted by 1 base as compared to the reference, it won't be detected), or to a real biological signal (since I am also using data from long-read sequencing technologies).&lt;/p&gt;&#xA;" OwnerUserId="294" LastEditorUserId="294" LastEditDate="2017-07-11T08:30:36.360" LastActivityDate="2018-04-10T23:42:16.903" Title="tools to reconcile experimental transcripts with reference annotation" Tags="|sequence-alignment|transcriptome|software-recommendation|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2038" PostTypeId="2" ParentId="2036" CreationDate="2017-07-11T08:44:12.137" Score="5" Body="&lt;p&gt;As per my answer to @_julien_roux on twitter:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Trying to find novel transcripts within the context of an existing annotation is much less straightforward. You probably need to do a &quot;genome-guided assembly&quot; with Trinity and PASA:&#xA;&lt;a href=&quot;http://pasapipeline.github.io/#A_ComprehensiveTranscriptome&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://pasapipeline.github.io/#A_ComprehensiveTranscriptome&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We did something similar in much simpler organisms in our recent paper: &#xA;&lt;a href=&quot;http://dx.doi.org/10.1186/s12864-017-3505-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dx.doi.org/10.1186/s12864-017-3505-0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your biggest problems are going to be alternative spliced gene models, the large amount of fragmentation inherent in de novo transcriptome assembly and largely underreported non-coding RNAs in most genomes. It was easier for us as the dictyostelids have few and small introns - fly is going to be a different ball-game. Be prepared to spend a lot of time optimising parameters before finding a result you're happy with (maybe) ;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may think the fly genome pretty good and complete, but be prepared to find lots of errors, corrections and novelty. We were very surprised at the amount of new annotation we were able to find in &lt;em&gt;D. discoideum&lt;/em&gt; - an organism where every single gene had been manually curated!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="1092" LastEditorUserId="1092" LastEditDate="2017-07-11T09:01:26.063" LastActivityDate="2017-07-11T09:01:26.063" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2039" PostTypeId="2" ParentId="2036" CreationDate="2017-07-11T08:44:31.813" Score="6" Body="&lt;p&gt;I've never tried this myself, so I don't know how easy this is...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One option would be to start with &lt;a href=&quot;http://research-pub.gene.com/gmap/&quot; rel=&quot;noreferrer&quot;&gt;GMAP&lt;/a&gt;, which is meant to align whole transcripts against the genome. The really nice thing about this is that it can directly produce GFF3 files. You can then use that with your Ensembl GTF with &lt;code&gt;cuffcompare&lt;/code&gt; or whatever the equivalent is in stringTie. You should then be able to get the information you want from the &lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/cuffcompare/#transfrag-class-codes&quot; rel=&quot;noreferrer&quot;&gt;transfrag codes&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-07-11T08:44:31.813" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2040" PostTypeId="2" ParentId="2022" CreationDate="2017-07-11T09:10:06.243" Score="1" Body="&lt;p&gt;Have you looked at Ensembl? At least for mice, they have pretty rich data for SNPs e.g.&#xA;&lt;a href=&quot;http://www.ensembl.org/Mus_musculus/Variation/Explore?db=core;r=2:3225281-3226281;v=rs27096498;vdb=variation;vf=802773&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ensembl.org/Mus_musculus/Variation/Explore?db=core;r=2:3225281-3226281;v=rs27096498;vdb=variation;vf=802773&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assumes you have known dbSNP ids (or 'rs' numbers) for your SNPs. If not you could use their VEP tool to annotate them:&#xA;&lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Tools/VEP?db=core&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ensembl.org/Homo_sapiens/Tools/VEP?db=core&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;VEP is also available for rice and Arabidopsis:&#xA;&lt;a href=&quot;http://plants.ensembl.org/Oryza_sativa/Tools/VEP?db=core&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://plants.ensembl.org/Oryza_sativa/Tools/VEP?db=core&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1092" LastActivityDate="2017-07-11T09:10:06.243" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2041" PostTypeId="2" ParentId="488" CreationDate="2017-07-11T09:18:11.997" Score="2" Body="&lt;p&gt;By far my preferred option for doing this manually is PICR:&#xA;&lt;a href=&quot;http://www.ebi.ac.uk/Tools/picr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ebi.ac.uk/Tools/picr/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW it is not &quot;ridiculous&quot; to get different numbers of genes reported for a given set of proteins. For several reasons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Uniprot IDs can disappear, merge or be split&lt;/li&gt;&#xA;&lt;li&gt;not all uniprot and gene IDs have a 1-to-1 relationship&lt;/li&gt;&#xA;&lt;li&gt;depending on species some gene symbols can be ambiguous or synonymous.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1092" LastActivityDate="2017-07-11T09:18:11.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2043" PostTypeId="1" AcceptedAnswerId="2044" CreationDate="2017-07-11T13:10:56.437" Score="13" ViewCount="1687" Body="&lt;p&gt;Biopython's &lt;code&gt;.count()&lt;/code&gt; methods, like Python's &lt;code&gt;str.count()&lt;/code&gt;, perform a non-overlapping count, how can I do an overlapping one?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, these code snippets &lt;code&gt;return 2&lt;/code&gt;, but I want the answer &lt;code&gt;3&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; from Bio.Seq import Seq&#xA;&amp;gt;&amp;gt;&amp;gt; Seq('AAAA').count('AA')&#xA;2&#xA;&amp;gt;&amp;gt;&amp;gt; 'AAAA'.count('AA')&#xA;2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastEditorUserId="104" LastEditDate="2017-08-10T14:41:41.410" LastActivityDate="2017-08-14T08:58:49.933" Title="How can I do an overlapping sequence count in Biopython?" Tags="|python|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2044" PostTypeId="2" ParentId="2043" CreationDate="2017-07-11T13:10:56.437" Score="18" Body="&lt;p&gt;For Biopython 1.70, there is a new &lt;code&gt;Seq.count_overlap()&lt;/code&gt; method, which includes optional &lt;code&gt;start&lt;/code&gt; and &lt;code&gt;end&lt;/code&gt; arguments:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; from Bio.Seq import Seq&#xA;&amp;gt;&amp;gt;&amp;gt; Seq('AAAA').count_overlap('AA')&#xA;3&#xA;&amp;gt;&amp;gt;&amp;gt; Seq('AAAA').count_overlap('AA', 1, 4)&#xA;2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This method is also implemented for the &lt;code&gt;MutableSeq&lt;/code&gt; and &lt;code&gt;UnknownSeq&lt;/code&gt; classes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; from Bio.Seq import MutableSeq, UnknownSeq&#xA;&amp;gt;&amp;gt;&amp;gt; MutableSeq('AAAA').count_overlap('AA')&#xA;3&#xA;&amp;gt;&amp;gt;&amp;gt; UnknownSeq(4, character='A').count_overlap('AA')&#xA;3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I co-contributed the &lt;code&gt;.count_overlap()&lt;/code&gt; methods with Peter Cock, see &lt;a href=&quot;https://github.com/biopython/biopython/commit/97709cc7a4b8591e794b442796d41e4f7b855a0f&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;97709cc&lt;/code&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2017-07-11T13:10:56.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2045" PostTypeId="2" ParentId="628" CreationDate="2017-07-11T15:01:38.167" Score="4" Body="&lt;p&gt;Short answer: you can't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neural networks use positive and negative examples to add weights to the neural network architecture that is provided to it. Trying to deconvolve the meaning behind the weights is nigh-on impossible except for the simplest perceptron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is common to many machine learning algorithms: they work very much like black boxes. One exception is decision trees. They will report what features are used to classify the positive vs negative datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as with all ML methods, you need to be very careful of your training datasets. This is especially true of motif searching and especially for negative datasets. It's quite easy to find a negative dataset which is completely inappropriate for learning and give you misleading accuracies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like @user172818, I would try traditional methods as they can work well, again, if given appropriate data. The &lt;a href=&quot;http://meme-suite.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MEME-suite&lt;/a&gt; would be a good start.&lt;/p&gt;&#xA;" OwnerUserId="1092" LastActivityDate="2017-07-11T15:01:38.167" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2046" PostTypeId="2" ParentId="2034" CreationDate="2017-07-11T15:15:50.387" Score="2" Body="&lt;p&gt;I've been using &lt;a href=&quot;http://kmer.sourceforge.net/wiki/index.php/Getting_Started_with_ATAC&quot; rel=&quot;nofollow noreferrer&quot;&gt;ATAC&lt;/a&gt; to successfully align different versions of the rice genome assembly. There is a chance it will work for you between two closely related species (potato and tomato work well for example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not massively user friendly, but it's very fast, and it produces long, 'clean' alignments (i.e. almost all 1 to 1 alignments). The output format is a bit cryptic (zero-based start position and length instead of the more usual 1-based start and end positions), but as I said, it should basically output the order for you in about 30 minutes.&lt;/p&gt;&#xA;" OwnerUserId="931" LastActivityDate="2017-07-11T15:15:50.387" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2048" PostTypeId="1" CreationDate="2017-07-11T20:06:56.393" Score="3" ViewCount="178" Body="&lt;p&gt;UPDATE: I found the solution. I was using normalized values and GEO was using raw beta values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to link GEOquery and minfi. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically I want to obtain beta values from the idat files on GEOquery. I was following this guide: &lt;a href=&quot;https://kasperdanielhansen.github.io/genbioconductor/html/minfi.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://kasperdanielhansen.github.io/genbioconductor/html/minfi.html&lt;/a&gt;, up until the preprocessing part. Meaning that I was able to obtain the RGset. Then I used my own preprocessing code to obtain the beta values. However, I cross checked them with the beta values that were on GEO and they weren't consistent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the accession number I used was GSE68777. So I went to that study on GEO and clicked on the first sample: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM1681154&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM1681154&lt;/a&gt;. Then I scrolled down and clicked &quot;Download full table&quot; to download the samples and beta values in a text file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I typed head(beta) and chose the first sample. Then I did command F in the text file for that sample and it's value there wasn't the same as the value from the beta data table. Hopefully you can help find the error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code I'm using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(GEOquery)&#xA;library(minfi)&#xA;library(&quot;IlluminaHumanMethylation450kanno.ilmn12.hg19&quot;)&#xA;library(&quot;IlluminaHumanMethylation450kmanifest&quot;)&#xA;&#xA;######## Code copied from 450k Guide ########&#xA;&#xA;getGEOSuppFiles(&quot;GSE68777&quot;)&#xA;untar(&quot;GSE68777/GSE68777_RAW.tar&quot;, exdir = &quot;GSE68777/idat&quot;)&#xA;head(list.files(&quot;GSE68777/idat&quot;, pattern = &quot;idat&quot;))&#xA;idatFiles &amp;lt;- list.files(&quot;GSE68777/idat&quot;, pattern = &quot;idat.gz$&quot;, full = TRUE)&#xA;sapply(idatFiles, gunzip, overwrite = TRUE)&#xA;rgSet &amp;lt;- read.metharray.exp(&quot;GSE68777/idat&quot;)&#xA;geoMat &amp;lt;- getGEO(&quot;GSE68777&quot;)&#xA;pD.all &amp;lt;- pData(geoMat[[1]])&#xA;pD &amp;lt;- pD.all[, c(&quot;title&quot;, &quot;geo_accession&quot;, &quot;characteristics_ch1.1&quot;, &quot;characteristics_ch1.2&quot;)]&#xA;names(pD)[c(3,4)] &amp;lt;- c(&quot;group&quot;, &quot;sex&quot;)&#xA;pD$group &amp;lt;- sub(&quot;^diagnosis: &quot;, &quot;&quot;, pD$group)&#xA;pD$sex &amp;lt;- sub(&quot;^Sex: &quot;, &quot;&quot;, pD$sex)&#xA;sampleNames(rgSet) &amp;lt;- sub(&quot;.*_5&quot;, &quot;5&quot;, sampleNames(rgSet))&#xA;rownames(pD) &amp;lt;- pD$title&#xA;pD &amp;lt;- pD[sampleNames(rgSet),]&#xA;pData(rgSet) &amp;lt;- pD&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1025" LastEditorUserId="1025" LastEditDate="2017-07-17T18:58:28.627" LastActivityDate="2017-07-17T18:58:28.627" Title="Minfi returning incorrect beta values" Tags="|r|bioconductor|normalization|methylation|data-mining|" AnswerCount="0" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2049" PostTypeId="1" AcceptedAnswerId="2075" CreationDate="2017-07-11T21:14:20.717" Score="2" ViewCount="285" Body="&lt;p&gt;My goal is to make a conservation plot between bacterial sequences and a human protein.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, I have a FASTA file of the protein, and a FASTA file with the sequences of the proteins from the BLAST results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my initial attempts, I have tried to do this using msa software:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have downloaded clustalx and run a profile alignment using the single protein sequence as Profile 1 and the sequences from the BLAST results as Profile 2, and selected the option &quot;Align Sequences to Profile 1&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm having difficulty interpreting and dealing with the results. Since the majority of the sequences aren't aligned, the results are a total mess, with a huge number of dashes added.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to visualize the number of BLAST hits per amino acid in my protein of interest. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;ie: &lt;img src=&quot;https://i.stack.imgur.com/L8jL5.jpg&quot; alt=&quot;alignment&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to have a graph with my protein on the x-axis and a plot like the regions of high conservation for the multiple alignments, except with the spikes corresponding to a high number of BLAST hits. This would allow me to identify regions of my protein that have higher sequence similarity to bacteria than others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better way to achieve this or to salvage the results from the profile alignment?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="727" LastEditorUserId="272" LastEditDate="2017-07-15T20:47:11.947" LastActivityDate="2017-07-16T13:37:45.500" Title="Profile of conservation between bacterial sequences and human protein" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2050" PostTypeId="2" ParentId="1007" CreationDate="2017-07-11T21:19:03.517" Score="6" Body="&lt;p&gt;If you don't mind hitting it 50k times and are OK with python3...&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from urllib import request&#xA;import json&#xA;&#xA;def getPathways(proteinID):&#xA;    baseURL = 'http://reactome.org/ContentService/data/query'&#xA;    PathwayIDs = set()&#xA;    try:&#xA;        response = request.urlopen('{}/{}'.format(baseURL, proteinID)).read().decode()&#xA;        data = json.loads(response)&#xA;        if 'consumedByEvent' in data:&#xA;            for event in data['consumedByEvent']:&#xA;                PathwayIDs.add(event['stId'])&#xA;        if 'producedByEvent' in data:&#xA;            for event in data['producedByEvent']:&#xA;                PathwayIDs.add(event['stId'])&#xA;    except:&#xA;        pass&#xA;    return PathwayIDs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Usage would then be something like:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;l = ['R-HSA-49155', 'R-HSA-199420', '']&#xA;for rid in l:&#xA;    ids = getPathways(rid)&#xA;    for _ in ids:&#xA;        print(&quot;{}\t{}&quot;.format(rid, _))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which would produce:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R-HSA-49155 R-HSA-110239&#xA;R-HSA-49155 R-HSA-110240&#xA;R-HSA-49155 R-HSA-110238&#xA;R-HSA-49155 R-HSA-110356&#xA;R-HSA-199420    R-HSA-8948800&#xA;R-HSA-199420    R-HSA-6807106&#xA;R-HSA-199420    R-HSA-6807206&#xA;R-HSA-199420    R-HSA-6807126&#xA;R-HSA-199420    R-HSA-8847968&#xA;R-HSA-199420    R-HSA-8850997&#xA;R-HSA-199420    R-HSA-2321904&#xA;R-HSA-199420    R-HSA-8948775&#xA;R-HSA-199420    R-HSA-8944497&#xA;R-HSA-199420    R-HSA-6807134&#xA;R-HSA-199420    R-HSA-8850945&#xA;R-HSA-199420    R-HSA-8873946&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that this will silently ignore invalid or missing IDs such as &lt;code&gt;''&lt;/code&gt; (that's the &lt;code&gt;try&lt;/code&gt; and &lt;code&gt;except&lt;/code&gt; above. Note also that these are different pathway IDs than what you provided in your example. The main reason is that the protein IDs you showed are not always involved in the pathways IDs you showed (in my example, they always are).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-07-11T21:19:03.517" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2051" PostTypeId="1" AcceptedAnswerId="2052" CreationDate="2017-07-11T22:01:38.863" Score="3" ViewCount="442" Body="&lt;p&gt;I'm looking at Reduced representation bisulfite sequencing (RRBS) data from ENCODE, and to align the FASTQ files I've used Bismark with Bowtie 1. When I load the resulting BAM file into &lt;code&gt;R&lt;/code&gt; with &lt;code&gt;Rsamtools&lt;/code&gt; and &lt;code&gt;GenomicRanges&lt;/code&gt;, I get something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; reads&#xA;GRanges object with 2 ranges and 5 metadata columns:&#xA;      seqnames         ranges strand |                     seq                                   XM          XR          XG        NM&#xA;         &amp;lt;Rle&amp;gt;      &amp;lt;IRanges&amp;gt;  &amp;lt;Rle&amp;gt; |          &amp;lt;DNAStringSet&amp;gt;                          &amp;lt;character&amp;gt; &amp;lt;character&amp;gt; &amp;lt;character&amp;gt; &amp;lt;integer&amp;gt;&#xA;  [1]     chrM [16523, 16558]      - | ATAAAACCTA...CCCTTAAATA .....h........h.......z.............          CT          GA         3&#xA;  [2]     chrM [16524, 16559]      + | TAAAGTTTAA...TTTTAAATAA .....hh.......hhh.h.z...hhhh........          CT          CT        11&#xA;  -------&#xA;  seqinfo: 25 sequences from an unspecified genome&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For the read on the - strand (which is aligned to the &quot;G-&gt;A&quot;-converted reference), how should I compare methylation calls to those on the + strand, since they don't line up?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the &lt;code&gt;z&lt;/code&gt; in the methylation string of the - strand is one position ahead of the &lt;code&gt;z&lt;/code&gt; in the + strand (which makes sense because of symmetric CpG methylation). But how should I determine whether these two methylation calls are &quot;essentially the same&quot; or not?&lt;/p&gt;&#xA;" OwnerUserId="240" LastEditorUserId="57" LastEditDate="2017-07-14T21:06:46.883" LastActivityDate="2017-07-14T21:06:46.883" Title="How to interpret methylation calls from Bismark on opposite strands?" Tags="|r|samtools|methylation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2052" PostTypeId="2" ParentId="2051" CreationDate="2017-07-11T22:22:01.657" Score="4" Body="&lt;p&gt;The strand the bismark reports is related to the strand from which the read originated, not necessarily how it's aligned. So, alignments on the + strand shouldn't have calls overlapping those on the - strand, since you can't have a C in the same place on the same strand. One should often see Z/z next to each other on opposite strand, like in your example, since these are CpG (so it should be a Z/z on the + strand and then a Z/z on the following base on the - strand). Thus, in your example you have two reads supporting unmethylation for the CpG as a whole (1 read for each of the Cs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most confusing thing about BSseq is that reads (or paired-end reads) are only ever informative for a single strand. This is actually true for all sequencing with Illumina instruments (single-stranded fragments are loaded, after all), but since strands are almost always complementary we can usually ignore this fact.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an aside, you might find &lt;a href=&quot;https://github.com/dpryan79/MethylDackel&quot; rel=&quot;nofollow noreferrer&quot;&gt;MethylDackel&lt;/a&gt; useful (full disclosure, I wrote it). It'll be much faster at extracting methylation calls than bismark and supports nice things like excluding regions of bias methylation and likely variant positions.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-07-11T22:22:01.657" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2053" PostTypeId="2" ParentId="2043" CreationDate="2017-07-12T03:18:23.897" Score="9" Body="&lt;p&gt;I've encountered this problem before, and used python &lt;code&gt;re&lt;/code&gt; module to solve this problem.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;all = re.findall(r'(?=(AA))','AAAA')&#xA;counts = len(all)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can get more details in this &lt;a href=&quot;https://stackoverflow.com/questions/11430863/how-to-find-overlapping-matches-with-a-regexp&quot;&gt;thread&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1085" LastEditorUserId="1085" LastEditDate="2017-07-12T05:17:53.587" LastActivityDate="2017-07-12T05:17:53.587" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2054" PostTypeId="1" CreationDate="2017-07-12T08:14:19.597" Score="6" ViewCount="366" Body="&lt;p&gt;I have recently generated a genome-guided transcriptome with &lt;a href=&quot;https://github.com/trinityrnaseq/trinityrnaseq/wiki&quot; rel=&quot;noreferrer&quot;&gt;Trinity&lt;/a&gt;, and would like to apply an additional filter to exclude transcripts that don't have good support from the RNASeq reads. This is with the goal of trying to reduce the initial dataset down to something a bit more manageable (I have about 300k transcripts covering 250Mb in total, but would prefer about 1/10 of that number).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see that Trinity has a &lt;a href=&quot;https://github.com/trinityrnaseq/trinityrnaseq/wiki/RNA-Seq-Read-Representation-by-Trinity-Assembly&quot; rel=&quot;noreferrer&quot;&gt;workflow using Bowtie2&lt;/a&gt; for evaluating the quality of assembled transcripts, but Bowtie2 is getting a bit old in the mapping world now. Are any of the super-fast transcript mappers (e.g. &lt;a href=&quot;https://combine-lab.github.io/salmon/&quot; rel=&quot;noreferrer&quot;&gt;Salmon&lt;/a&gt; or &lt;a href=&quot;https://pachterlab.github.io/kallisto/about&quot; rel=&quot;noreferrer&quot;&gt;Kallisto&lt;/a&gt;) appropriate to use for transcript filtering?&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-12-20T16:18:01.330" Title="Filter Trinity transcriptome based on RNASeq reads" Tags="|rna-seq|read-mapping|" AnswerCount="3" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2055" PostTypeId="2" ParentId="2054" CreationDate="2017-07-12T08:41:00.213" Score="6" Body="&lt;p&gt;I used a combination of BUSCO and Salmon to filter transcripts based on their abundance in the RNASeq read dataset. The approach was roughly as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Run BUSCO in short/transcript mode on the Trinity-generated sequences&lt;/li&gt;&#xA;&lt;li&gt;Run Salmon on the Trinity-generated sequences, using the RNASeq reads that were used to generate the transcriptome&lt;/li&gt;&#xA;&lt;li&gt;Merge the BUSCO full results table with the Salmon results table to determine distribution of the number of reads mapped for BUSCO genes&lt;/li&gt;&#xA;&lt;li&gt;Choose an appropriate &quot;real&quot; read count threshold based on the read count distribution for BUSCO genes&lt;/li&gt;&#xA;&lt;li&gt;Sub-select the transcripts based on this threshold, using the Salmon results table&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h2&gt;Step 1 -- BUSCO&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python ~/install/busco/BUSCO.py -i ../Trinity-GG.fasta -o BUSCO_Trinity-GG_nematodes -l ~/install/busco/nematoda_odb9 -m tran -c 10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Step 2 -- Salmon&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;~/install/salmon/Salmon-0.8.2_linux_x86_64/bin/salmon index -t Trinity-GG.fasta -i Trinity-GG.fasta.sai&#xA;~/install/salmon/Salmon-0.8.2_linux_x86_64/bin/salmon quant -i Trinity-GG.fasta.sai -1 reads-all_R1_trimmed.fastq.gz -2 reads-all_R2_trimmed.fastq.gz -p 10 -o quant/reads-all_quant -l A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Step 3 -- Results merge (using R)&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/Rscript&#xA;busco.table &amp;lt;-&#xA;    read.delim(&quot;BUSCO/run_BUSCO_Trinity-GG_nematodes/full_table_BUSCO_T-BNOCFED_nematodes.tsv&quot;, comment.char=&quot;#&quot;,&#xA;               header=FALSE, stringsAsFactors = FALSE,&#xA;               col.names=c(&quot;Busco id&quot;,&quot;Status&quot;,&quot;Sequence&quot;,&quot;Score&quot;,&quot;Length&quot;));&#xA;busco.table &amp;lt;- subset(busco.table, Status != &quot;Missing&quot;);&#xA;&#xA;count.table &amp;lt;- read.delim(&quot;quant/reads-all_quant_ISR/quant.sf&quot;,&#xA;                          stringsAsFactors=FALSE);&#xA;&#xA;busco.expr.df &amp;lt;- merge(busco.table, count.table,&#xA;                       by.x=&quot;Sequence&quot;, by.y=&quot;Name&quot;, all.x=TRUE);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Step 4 -- Count thresholding (visual exploration)&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pdf(&quot;busco_NumReads.pdf&quot;, paper=&quot;a4r&quot;, width=11, height=8);&#xA;options(scipen=10);&#xA;busco.numReads &amp;lt;- sort(tapply(busco.expr.df$NumReads, busco.expr.df$Busco.id,&#xA;                              function(x){&#xA;                                  exp(mean(log(x[x&amp;gt;0])))}), decreasing=TRUE);&#xA;par(mfrow=c(1,2));&#xA;plot(x=seq(0,1,length.out=length(busco.numReads)),&#xA;     busco.numReads, log=&quot;y&quot;, ylab=&quot;Number of Mapped Reads&quot;,&#xA;     yaxt=&quot;n&quot;,&#xA;     xlab=&quot;Proportion of Complete BUSCO sequences&quot;);&#xA;axis(2, at=rep(c(1,2,5),each=6) * 10^(0:5), las=2, cex.axis=0.71);&#xA;abline(col=&quot;#00000020&quot;, h=rep(1:9,each=6) * 10^(0:5), lwd=0.1);&#xA;abline(col=&quot;#80808080&quot;, h=rep(1,each=6) * 10^(0:5), lwd=2);&#xA;rect(ybottom=10^(par(&quot;usr&quot;)[3]), ytop=10^(par(&quot;usr&quot;)[4]),&#xA;     xleft=0.9, xright=par(&quot;usr&quot;)[2],&#xA;     col=&quot;#80808020&quot;, border=NA);&#xA;plot(x=seq(0,1,length.out=length(busco.numReads)), xlim=c(0.9,1),&#xA;     busco.numReads, log=&quot;y&quot;, ylab=&quot;Number of Mapped Reads&quot;, yaxt=&quot;n&quot;,&#xA;     xlab=&quot;Proportion of Complete BUSCO sequences&quot;);&#xA;axis(2, at=rep(c(1,2,5),each=6) * 10^(0:5), las=2, cex.axis=0.71);&#xA;abline(col=&quot;#00000020&quot;, h=rep(1:9,each=6) * 10^(0:5), lwd=0.1);&#xA;abline(col=&quot;#80808080&quot;, h=rep(1,each=6) * 10^(0:5), lwd=2);&#xA;invisible(dev.off());&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7iw47.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7iw47.png&quot; alt=&quot;choosing read count threshold from BUSCO genes&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looks like a value of 50 will be suitable.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Step 5 -- Sub-select the transcripts&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat(file=&quot;HighCount_Transcripts_Num50.txt&quot;,&#xA;    subset(count.table, NumReads &amp;gt;= 50)$Name, sep=&quot;\n&quot;);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then filter using a suitable &lt;a href=&quot;https://github.com/gringer/bioinfscripts/blob/master/fastx-fetch.pl&quot; rel=&quot;noreferrer&quot;&gt;FASTA selection utility&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pv Trinity-GG.fasta | ~/scripts/fastx-fetch.pl -i HighCount_Transcripts_Num50.txt &amp;gt; HighCount50_Trinity-GG.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-07-12T08:54:27.503" LastActivityDate="2017-07-12T08:54:27.503" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2056" PostTypeId="1" CreationDate="2017-07-12T09:36:21.243" Score="4" ViewCount="2692" Body="&lt;p&gt;For DNA/RNA quantification machines like the Bioanalyzer or TapeStation, the DNA Integrity Number (DIN) or RNA Integrity Number (RIN) numbers are quoted as a measure of the fragmentation of the material.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is the DNA Integrity Number (DIN) calculated in Bioanalyzer/TapeStation? Is there any open-source software to parse/collate the data from the machines?&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2017-07-17T02:10:55.023" Title="how is the DNA Integrity Number (DIN) calculated in Bioanalyzer/TapeStation?" Tags="|phylogenetics|quantification|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2057" PostTypeId="2" ParentId="2056" CreationDate="2017-07-12T09:38:45.987" Score="1" Body="&lt;p&gt;The best I could find by looking at github.com is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/brianfleharty/Agilent-BioAnalyzer-RIN-R-code-for-Fly/blob/master/RNA_RIN.R&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/brianfleharty/Agilent-BioAnalyzer-RIN-R-code-for-Fly/blob/master/RNA_RIN.R&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;#set the working directory&#xA;dir&amp;lt;- &quot;H:\\B_RIN\\emd&quot;&#xA;setwd(dir)&#xA;myfiles&amp;lt;-list.files()&#xA;which(substr(myfiles,69,79)==&quot;Results.csv&quot;)&#xA;results&amp;lt;-myfiles[which(substr(myfiles,69,79)==&quot;Results.csv&quot;)]&#xA;n &amp;lt;- which(substr(myfiles,69,79)==&quot;Results.csv&quot;)&#xA;targets &amp;lt;- read.delim(myfiles[n[1]], sep=&quot;,&quot;,header=F,skip=14,as.is=T)&#xA;length(n)&#xA;targets2&amp;lt;- read.delim(myfiles[n[1]], sep=&quot;,&quot;,header=F,skip=0,as.is=T)&#xA;str1&amp;lt;-substr(targets2[1,2],1,67)&#xA;str1&#xA;str2&amp;lt;-&quot;_S&quot;&#xA;str3&amp;lt;-paste(str1,str2,sep=&quot;&quot;)&#xA;str3&#xA;chip_num&amp;lt;-length(n)&#xA;samp_num_chip&amp;lt;-length(grep(str3,myfiles))&#xA;samp_num&amp;lt;-length(which(substr(myfiles,69,74)==&quot;Sample&quot;))&#xA;samples1&amp;lt;- matrix(NA, ncol=chip_num, nrow=samp_num)&#xA;for(i in 1:chip_num)&#xA;{&#xA;#mod chip numb&#xA;targets2&amp;lt;- read.delim(myfiles[n[i]], sep=&quot;,&quot;,header=F,skip=0,as.is=T)&#xA;str1&amp;lt;-substr(targets2[1,2],1,67)&#xA;str2&amp;lt;-&quot;_S&quot;&#xA;str3&amp;lt;-paste(str1,str2,sep=&quot;&quot;)&#xA;samp_num_chip&amp;lt;-length(grep(str3,myfiles))&#xA;#mod sample numb&#xA;str1&amp;lt;-substr(targets2[1,2],1,67)&#xA;str4&amp;lt;-&quot;_Sample&quot;&#xA;for(j in 1:samp_num_chip)&#xA;{&#xA;iii&amp;lt;-j&#xA;end.st&amp;lt;-&quot;.csv&quot;&#xA;str5&amp;lt;-paste(str1,str4,iii,end.st,sep=&quot;&quot;)&#xA;samples1[j,i]&amp;lt;-str5&#xA;}&#xA;}&#xA;samples1&#xA;grep(&quot;2100&quot;,samples1)&#xA;samples1&amp;lt;-samples1[grep(&quot;2100&quot;,samples1)]&#xA;samples1&#xA;write.table(samples1,file=paste(dir,&quot;\\ba_lane.txt&quot;,sep=&quot;&quot;), sep=&quot;\t&quot;,col.names=NA)&#xA;#RIN Calculator#######refined peakfinder####################&#xA;#read bioanalyzer data into a matrix called dta&#xA;#since the total RNA and mRNA assays run differently&#xA;#skip more lines for the total RNA assay&#xA;ramp &amp;lt;- colorRamp(c(&quot;darkmagenta&quot;,&quot;white&quot;))&#xA;elec &amp;lt;- read.delim(&quot;ba_lane.txt&quot;, sep=&quot;\t&quot;,header=T,as.is=T)&#xA;qc.mat &amp;lt;- matrix(NA, ncol=1, nrow=nrow(elec))&#xA;dta &amp;lt;- matrix(NA, nrow=1060, ncol=nrow(elec))&#xA;for(i in 1:nrow(elec))&#xA;{&#xA;x &amp;lt;- read.csv(as.character(elec[i,2]), header=F, skip=18, nrows=1060)&#xA;dta[,i] &amp;lt;- x[,2]&#xA;}&#xA;time&amp;lt;- x[,1]&#xA;#plot ladder and define peak threshold in HD&#xA;par(mfrow=c(3,4))&#xA;for(samples in 1:length(samples1))&#xA;#for(samples in 1:nrow(elec))&#xA;{&#xA;#which(dta[,lad] &amp;gt; 5)&#xA;#draw some segments and store max fluor in max.peaks matrix&#xA;lad&amp;lt;-samples&#xA;ti1&amp;lt;-17&#xA;ti2&amp;lt;-65&#xA;p.row&amp;lt;-length(dta[which(time==ti1):which(time==ti1+1),lad])*(ti2-ti1+1)&#xA;peaks &amp;lt;- matrix(NA, nrow=p.row, ncol=1)&#xA;max.peaks &amp;lt;- matrix(NA, nrow=length(seq(ti1,ti2,.05)), ncol=1)&#xA;for (i in 1:length(seq(ti1,ti2,.05)))&#xA;{&#xA;ii&amp;lt;-seq(ti1,ti2,.05)[i]&#xA;x0&amp;lt;- ii&#xA;x0&amp;lt;-round(x0,digits=3)&#xA;y0&amp;lt;- ii&#xA;x&amp;lt;- ii+.05&#xA;x&amp;lt;-round(x,digits=3)&#xA;y&amp;lt;- ii&#xA;#segments(x0,y0,x,y)&#xA;max.peaks[i]&amp;lt;- max(dta[which(time==x0):which(time==x),lad])&#xA;}    &#xA;#human or mouse&#xA;cent&amp;lt;-max(max.peaks)*.6&#xA;#flatworm&#xA;#cent&amp;lt;-max(max.peaks)*.05&#xA;plot(time,dta[,lad],type=&quot;h&quot;,col=&quot;dodgerblue&quot;,xlab=&quot;seconds&quot;,ylab=&quot;Fluorescence Units&quot;,xlim=c(17,75),ylim=c(0,(max(max.peaks)*1.2)),main=elec[lad,1])&#xA;abline(h=cent,col=&quot;grey&quot;)&#xA;#define the middle of the segments&#xA;max.avg&amp;lt;-seq(ti1+.025,ti2+.025,.05)&#xA;#plot the maximum value of each segment&#xA;max.peaks&#xA;lines(max.avg,max.peaks,type=&quot;b&quot;,col=&quot;deepskyblue&quot;)&#xA;#lines(max.avg,min.peaks,type=&quot;b&quot;,col=&quot;blue&quot;)&#xA;#find time associated with peaks &amp;gt; 5 fu and fill seconds matrix with them&#xA;#12.5% of max fu incase chip is fu&#xA;p1&amp;lt;-which(max.avg==38)&#xA;p2&amp;lt;-which(max.avg==55)&#xA;sizes &amp;lt;- which(max.peaks&amp;gt;cent)&#xA;max.avg[sizes]&#xA;gt35&amp;lt;-which(max.avg[sizes]&amp;gt;35)&#xA;lt55&amp;lt;-which(max.avg[sizes]&amp;lt;55)&#xA;#as.numeric(duplicated(c(gt35,lt55)))&#xA;tfr&amp;lt;-which(duplicated(c(gt35,lt55))==TRUE)&#xA;c(gt35,lt55)[tfr]&#xA;max.avg[sizes][c(gt35,lt55)[tfr]]&#xA;max.peaks[sizes][c(gt35,lt55)[tfr]]&#xA;points(max.avg[sizes+1][c(gt35,lt55)[tfr]],max.peaks[sizes+1][c(gt35,lt55)[tfr]],col=&quot;lawngreen&quot;,pch=19)&#xA;points(max.avg[sizes-1][c(gt35,lt55)[tfr]],max.peaks[sizes-1][c(gt35,lt55)[tfr]],col=&quot;lawngreen&quot;,pch=19)&#xA;points(max.avg[sizes][c(gt35,lt55)[tfr]],max.peaks[sizes][c(gt35,lt55)[tfr]],col=&quot;deeppink3&quot;,pch=19)&#xA;#colnms2&amp;lt;-c(&quot;max.avg[sizes]&quot;,&quot;max.peaks[sizes]&quot;,&quot;max.avg[sizes+1]&quot;,&quot;max.peaks[sizes+1]&quot;,&quot;max.avg[sizes-1]&quot;)&#xA;max.avg[sizes]&#xA;max.peaks[sizes]&#xA;max.avg[sizes+1]&#xA;max.peaks[sizes+1]&#xA;max.avg[sizes-1]&#xA;max.peaks[sizes-1]&#xA;area&amp;lt;-cbind(max.peaks[sizes-1][c(gt35,lt55)[tfr]],max.peaks[sizes][c(gt35,lt55)[tfr]],max.peaks[sizes+1][c(gt35,lt55)[tfr]])&#xA;time2&amp;lt;-cbind(max.avg[sizes-1][c(gt35,lt55)[tfr]],max.avg[sizes][c(gt35,lt55)[tfr]],max.avg[sizes+1][c(gt35,lt55)[tfr]])&#xA;p1&amp;lt;-area[which(area[,1]&amp;lt;cent),1][1]&#xA;t1&amp;lt;-time2[which(area[,1]&amp;lt;cent),1][1]&#xA;p2&amp;lt;-area[which(area[,1]&amp;lt;cent),1][2]&#xA;t2&amp;lt;-time2[which(area[,1]&amp;lt;cent),1][2]&#xA;#area[which(area[,2]&amp;lt;cent),2]&#xA;#area[which(area[,2]&amp;lt;cent),2]&#xA;p3&amp;lt;-area[which(area[,3]&amp;lt;cent),3][1]&#xA;t3&amp;lt;-time2[which(area[,3]&amp;lt;cent),3][1]&#xA;p4&amp;lt;-area[which(area[,3]&amp;lt;cent),3][2]&#xA;t4&amp;lt;-time2[which(area[,3]&amp;lt;cent),3][2]&#xA;#abline(v=area[which(area[,3]&amp;lt;cent),3])&#xA;#abline(h=11)&#xA;#abline(v=42.75)&#xA;v1&amp;lt;-t1&#xA;v2&amp;lt;-t2&#xA;v3&amp;lt;-t3&#xA;v4&amp;lt;-t4&#xA;abline(v=v1)&#xA;abline(v=v2)&#xA;abline(v=v3)&#xA;abline(v=v4)&#xA;#time[area[which(area[,1]&amp;lt;cent),1][1]==dta[,lad]]&#xA;#time[area[which(area[,3]&amp;lt;cent),3][1]==dta[,lad]]&#xA;#time[area[which(area[,1]&amp;lt;cent),1][2]==dta[,lad]]&#xA;#time[area[which(area[,3]&amp;lt;cent),3][2]==dta[,lad]]&#xA;#abline(v=time[area[which(area[,1]&amp;lt;cent),1][1]==dta[,lad]])&#xA;#abline(v=time[area[which(area[,3]&amp;lt;cent),3][1]==dta[,lad]])&#xA;#abline(v=time[area[which(area[,1]&amp;lt;cent),1][2]==dta[,lad]])&#xA;#abline(v=time[area[which(area[,3]&amp;lt;cent),3][2]==dta[,lad]])&#xA;#abline(h=area[which(area[,3]&amp;lt;cent),3][1])&#xA;#abline(h=area[which(area[,3]&amp;lt;cent),3][2]+3)&#xA;s18a&amp;lt;-round(t1,digits=1)&#xA;s18b&amp;lt;-round(t3,digits=1)&#xA;s28a&amp;lt;-round(t2,digits=1)&#xA;s28b&amp;lt;-round(t4,digits=1)&#xA;if(is.na(s18a)){s18a&amp;lt;-30}&#xA;if(is.na(s18b)){s18b&amp;lt;-35.05}&#xA;if(is.na(s28a)){s28a&amp;lt;-30}&#xA;if(is.na(s28b)){s28b&amp;lt;-30.05}&#xA;eta&amp;lt;-which(time==s18a)&#xA;etb&amp;lt;-which(time==s18b)&#xA;tea&amp;lt;-which(time==s28a)&#xA;teb&amp;lt;-which(time==s28b)&#xA;#dta[eta:etb,lad]&#xA;#dta[tea:teb,lad]&#xA;s18&amp;lt;-sum(dta[eta:etb,lad])&#xA;s28&amp;lt;-sum(dta[tea:teb,lad])&#xA;s28/s18&#xA;qc.mat[samples]&amp;lt;-  (-1*exp((s28/s18)*-1)+1)*10&#xA;text(60,cent*.75,c(format(qc.mat[samples],digits=2),&quot;\n\nB-RIN&quot;))&#xA;}&#xA;qc.mat&#xA;###end of ratio calc&#xA;###&#xA;###&#xA;#barplot(qc.mat,beside=T,ylim=c(0,11))&#xA;#set the working directory&#xA;#results &amp;lt;- read.delim(&quot;results-files.txt&quot;, sep=&quot;,&quot;,header=F,skip=0,as.is=T)&#xA;#targets &amp;lt;- read.delim(results[1,], sep=&quot;,&quot;,header=F,skip=14,as.is=T)&#xA;#targets[1:5,]&#xA;tbl&amp;lt;-which(targets[,1]==&quot;Overall Results:&quot;)&#xA;which(targets[,1]==&quot;Sample Name&quot;)&#xA;length(which(targets[,1]==&quot;Sample Name&quot;))&#xA;samples&amp;lt;-as.character(targets[which(targets[,1]==&quot;Sample Name&quot;),2])&#xA;#read in the targets file                               &#xA;colnms&amp;lt;-as.character(targets[(tbl[1]+1):(tbl[1]+4),1])&#xA;mat1 &amp;lt;- matrix(NA, ncol=length(colnms), nrow=length(samples)*length(results))&#xA;colnames(mat1)&amp;lt;-colnms&#xA;rownames(mat1)&amp;lt;-rep(samples,length(results))&#xA;for(i in 1:length(results)) &#xA;{&#xA;targets &amp;lt;- read.delim(results[i], sep=&quot;,&quot;,header=F,skip=14,as.is=T)&#xA;targets[1:5,]&#xA;which(targets[,1]==&quot;Sample Name&quot;)&#xA;length(which(targets[,1]==&quot;Sample Name&quot;))&#xA;samples&amp;lt;-as.character(targets[which(targets[,1]==&quot;Sample Name&quot;),2])&#xA;tbl&amp;lt;-which(targets[,1]==&quot;Overall Results:&quot;)&#xA;length(which(targets[,1]==&quot;Overall Results:&quot;))&#xA;tbl[1]&#xA;tbl[1]&#xA;colnms&amp;lt;-as.character(targets[(tbl[1]+1):(tbl[1]+4),1])&#xA;for (j in 1:length(samples))&#xA;{&#xA;mat1[((length(samples)*(i-1))+j),]&amp;lt;-as.matrix(targets[(tbl[j]+1):(tbl[j]+4),2])&#xA;}&#xA;}&#xA;mat1&#xA;cbind(mat1,qc.mat)&#xA;colnames(qc.mat)&amp;lt;-&quot;B-RIN&quot;&#xA;write.table(cbind(mat1,qc.mat),file=paste(dir,&quot;\\RIN_VS_B-RIN.txt&quot;,sep=&quot;&quot;), sep=&quot;\t&quot;,col.names=NA)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="180" LastEditorUserId="298" LastEditDate="2017-07-12T10:23:21.807" LastActivityDate="2017-07-12T10:23:21.807" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2059" PostTypeId="1" CreationDate="2017-07-13T08:24:21.383" Score="3" ViewCount="123" Body="&lt;p&gt;I am looking for advice on how to best approach a problem I am faced with. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a dataset of numerous degradative biomarkers, clinical information and various other measures (from clinical trials). I would like to see how different biomarkers relate to each other, and effect their levels (high/low etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to gain an understanding of how these biomarker levels fluctuate in different disease profiles - and understand if there is a relationship between each of them (co/independent). The limitation is that I do not have access to genetic or environmental data so this will be a &quot;random effect&quot;. There are around 1000 marker samples and 10's of biomarkers mixed between clinical and biochemical. There are also numerous time points to be incorporated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have thought about implementing something similar to a protein-protein interaction network, although this will of course not involve physical interactions, but more pathway interactions for further hypothesis driven research. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to firstly: validate if this project actually makes sense and secondly: ask for some advice around how to best implement this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have experience implementing neural/deep nets, directed and undirected networks (Bayes) in python, but not sure exactly how to go about this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="367" LastEditorUserId="73" LastEditDate="2018-01-10T06:20:01.163" LastActivityDate="2018-01-10T06:20:01.163" Title="Building network for biomarker interaction analysis" Tags="|proteins|networks|pathway|interactions|" AnswerCount="1" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2060" PostTypeId="2" ParentId="2054" CreationDate="2017-07-13T08:35:41.727" Score="3" Body="&lt;p&gt;I would recommend &lt;a href=&quot;http://hibberdlab.com/transrate/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Transrate&lt;/a&gt;. It generates a score for each transcript based on many metrics (read their paper &lt;a href=&quot;http://dx.doi.org/10.1101/gr.196469.115&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) and, using that score, gives a predicted cut-off to produce the optimal transcript set.&lt;/p&gt;&#xA;" OwnerUserId="1092" LastEditorUserId="1092" LastEditDate="2017-07-13T08:44:04.673" LastActivityDate="2017-07-13T08:44:04.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2061" PostTypeId="2" ParentId="1007" CreationDate="2017-07-13T09:04:28.000" Score="3" Body="&lt;p&gt;On Reactome website they have at the download &lt;a href=&quot;http://reactome.org/pages/download-data/&quot; rel=&quot;nofollow noreferrer&quot;&gt;page&lt;/a&gt; mapping files (uniprot, ensembl, etc.), but unfortunately not for the protein IDs you are using (stable identifiers).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had contact with their helpdesk, and they sent me a file containing all protein IDs to the pathways. Exactly what you need. I have asked them if they wanted to put it on their download page as well, but not sure if they want to do this. Meanwhile you can ask for the file as well, or get it from my google &lt;a href=&quot;https://drive.google.com/open?id=0ByVMqlqqt462enVaNkZZSU9iMjg&quot; rel=&quot;nofollow noreferrer&quot;&gt;drive&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume you know how to get your Protein IDs and pathways from this file, using e.g., R?&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2017-07-13T09:04:28.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2062" PostTypeId="1" CreationDate="2017-07-13T12:22:24.687" Score="2" ViewCount="190" Body="&lt;p&gt;I would like to be able to batch download FASTA files from ENSEMBL. I normally would use the API to download them from the ENSEMBL gene IDs. However, the IDs aren't ones I know, only ones I would get from searching on the ENSEMBL website and collecting the IDs manually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way I can use the API to query search terms and collect all results that are returned?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for example, if I use the ENSEMBL website to search &quot;HLA&quot; I could collect the list of IDs (ENSG00000204252 etc. etc.) manually. HLA works okay using the API to query gene IDs because there's only a few terms (A, B, C plus non-classical for class I) but I was wondering if there was a way to directly access the search query one the homepage programmatically, as I may need to do it for some messier examples.&lt;/p&gt;&#xA;" OwnerUserId="252" LastEditorUserId="252" LastEditDate="2017-07-14T08:59:31.373" LastActivityDate="2017-07-14T08:59:31.373" Title="Is there a way to retrieve ENSEMBL IDs from a search query?" Tags="|database|reference-genome|ensembl|data-mining|" AnswerCount="1" CommentCount="7" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2063" PostTypeId="2" ParentId="2062" CreationDate="2017-07-13T17:47:10.213" Score="1" Body="&lt;p&gt;I would download the complete geneset FASTA file from &lt;a href=&quot;http://www.ensembl.org/info/data/ftp/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ensembl&lt;/a&gt; and then parse out the genes you want with a script.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you could do similarly with API. Get all the gene names, select the ones you want, and then pull down each sequence.&lt;/p&gt;&#xA;" OwnerUserId="1092" LastActivityDate="2017-07-13T17:47:10.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2065" PostTypeId="1" CreationDate="2017-07-14T10:46:42.500" Score="3" ViewCount="191" Body="&lt;p&gt;I'm new to bioinformatics and Im starting a new microbial sequencing project and I'd like to record all of the qc data correctly. This is research and I'm not sure what I'll need later. Is there any consensus on useful qc metrics to record in a database for each sequencing run?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No rnaseq, running FastQC, assembling with a defined reference for a single species. &lt;/p&gt;&#xA;" OwnerUserId="167" LastEditorUserId="167" LastEditDate="2017-07-14T10:55:05.473" LastActivityDate="2018-01-28T22:09:15.393" Title="What sequencing data metrics should I record?" Tags="|database|ngs|qc|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2066" PostTypeId="2" ParentId="2059" CreationDate="2017-07-14T10:54:33.113" Score="2" Body="&lt;p&gt;You can start by looking at the correlation (if data is non-parametric) of each variable with each variable of each data point you have. One thing that might change is the correlation between variables along the disease.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next you can determine which relationships are constant along the disease progression/clinical trial. Identify also the ones that don't show a constant relationship. To do so you can use WGCNA (although with just 10 variables is too low to expect a scale free network you can use to correlate the variables and group them by correlation profile)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the question is which variables are more important, those who kept the same relationship or those which change? &#xA;Use the eigengenes of those variables (separately by those that have a constant relationship and those which don't) to estimate the dependency with the parametric variables. Or you simply can do an ANOVA of all the variables with each non-parametric variable. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2017-07-14T10:54:33.113" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2067" PostTypeId="1" CreationDate="2017-07-14T11:13:42.640" Score="6" ViewCount="461" Body="&lt;p&gt;I want to perform a genome comparison on a group of isolates. I want to look into two broad groups of taxa and compare the accessory genome in each group. I have been using prokka (v1.12) and roary (v3.8.2) to do this but it appears the accessory_binary_genes.fa file is actually an untrue representation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; gene_presence_absence.Rtab does contain all the full presence/ absence for accessory gene sets. Despite this Im still unhappy with the nomenclature of the gene groups [issue for another day] &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://github.com/sanger-pathogens/Roary/issues/335&quot; rel=&quot;nofollow noreferrer&quot;&gt;[github issue 335]&lt;/a&gt; Your best to ignore the accessory_binary_genes.fa file. It is just for creating a quick and dirty tree with FastTree. The file itself is filtered to remove very common and not common variation to speedup the tree generation, hence the difference in numbers.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The top 5% and bottom 5% are excluded. It is truncated at 4000 genes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I've been looking into alternative pipelines and a new software &lt;a href=&quot;https://www.nature.com/articles/srep24373&quot; rel=&quot;nofollow noreferrer&quot;&gt;BPGA&lt;/a&gt; looks promising. Does anyone have experience with this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I essentially want a tool which will give me the core and accessory gene sets, without the noise from partial gene hits.&lt;/p&gt;&#xA;" OwnerUserId="982" LastEditorUserId="982" LastEditDate="2017-07-20T11:05:50.983" LastActivityDate="2017-11-27T15:50:38.583" Title="What tools can I use for a bacterial core/pan genome pipeline?" Tags="|genome|software-recommendation|" AnswerCount="2" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2068" PostTypeId="1" AcceptedAnswerId="2072" CreationDate="2017-07-14T13:40:47.287" Score="9" ViewCount="3505" Body="&lt;p&gt;I would like to convert a file in &lt;code&gt;gff3&lt;/code&gt; format to a &lt;code&gt;gtf2.2&lt;/code&gt; format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why I would like to do this is: I have a set of transcripts assembled by a bunch of different software (and using RNA-seq data from different sequencing technologies) and I would like to compare each of those sets with the reference transcriptome annotation for that species (&lt;em&gt;D. melanogaster&lt;/em&gt;). I &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2036/tools-to-reconcile-experimental-transcripts-with-reference-annotation&quot;&gt;already asked for the community advice&lt;/a&gt; about how to proceed on that, but to run the suggested software (&lt;a href=&quot;https://tacorna.github.io/&quot; rel=&quot;noreferrer&quot;&gt;TACO&lt;/a&gt; and &lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/cuffmerge/&quot; rel=&quot;noreferrer&quot;&gt;cuffmerge&lt;/a&gt;) I need to have a GTF file containing the experimental transcripts to compare to the reference transcripts (GTF).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Up to now, I &lt;strong&gt;unsuccessfully&lt;/strong&gt; tried: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;gffread&lt;/code&gt; utility in the &lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/&quot; rel=&quot;noreferrer&quot;&gt;Cufflinks&lt;/a&gt; package (&lt;code&gt;gffread input.gff3 -T -o output.gtf&lt;/code&gt;): this results in an empty &lt;code&gt;output.gtf&lt;/code&gt; file and an empty &lt;code&gt;log&lt;/code&gt; file (used &lt;code&gt;Cufflinks&lt;/code&gt; v.2.2.1) - I contacted the authors via their &lt;a href=&quot;https://groups.google.com/d/msg/tuxedo-tools-users/T-qE4P67-HU/O_up6Yh6BgAJ&quot; rel=&quot;noreferrer&quot;&gt;Google group&lt;/a&gt; but haven't heard of them yet &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;gff3_to_gtf&lt;/code&gt; utility in the &lt;a href=&quot;http://genometools.org/&quot; rel=&quot;noreferrer&quot;&gt;GenomeTools&lt;/a&gt; (&lt;code&gt;gt&lt;/code&gt;) package (&lt;code&gt;gt gff3_to_gtf input.gff3 -o output.gtf&lt;/code&gt;): the output is not created and the log file is not informative - I contacted the authors via their &lt;a href=&quot;http://gt-users@genometools.org&quot; rel=&quot;noreferrer&quot;&gt;mailing list&lt;/a&gt; but haven't heard of them yet &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;GFF3_to_GTF&lt;/code&gt; utility in the &lt;a href=&quot;https://testtoolshed.g2.bx.psu.edu/repository/display_tool?repository_id=afcb6456d8e300ed&amp;amp;tool_config=database%2Fcommunity_files%2F000%2Frepo_21%2Ffml_gff_converter_programs%2Fgalaxy%2Fgff3_to_gtf.xml&amp;amp;changeset_revision=4c459744cab1&quot; rel=&quot;noreferrer&quot;&gt;FML&lt;/a&gt; package (&lt;code&gt;./gff3_to_gtf_converter.pl input.gff3 output.gtf&lt;/code&gt;): the output just contains a header (&lt;code&gt;##gff-version 2.5&lt;/code&gt;) and the &lt;code&gt;log&lt;/code&gt; is empty&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;gff3&lt;/code&gt; file was created as output of &lt;a href=&quot;https://wiki.gacrc.uga.edu/wiki/Gmap&quot; rel=&quot;noreferrer&quot;&gt;GMAP&lt;/a&gt;, and contains the transcripts as found by alignment to the reference (specifying option &lt;code&gt;-f gff3_match_cdna&lt;/code&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Edit: what I found out a poteriori, is that this format is not a standard gff3, thus the conversion is not trivial...]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the beginning of the gff3 file I tried to convert: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ head r9_gmap_match-cdna.gff&#xA;##gff-version   3&#xA;# Generated by GMAP version 2017-04-24 using call:  /home/aechchik/software/gmap-2017-04-24/bin/gmap.sse42 -d gmapidx -D /scratch/beegfs/monthly/aechchik/isoforms/ref/chromosomes/ -f gff3_match_cdna -n 0 -t 20 r9_2d.fasta&#xA;2L    gmapidx    cDNA_match    18442664    18443024    79    -    . ID=ae6a7818-85b5-4739-8031-e58f4462ad41_Basecall_2D_2d.path1;Name=ae6a7818-85b5-4739-8031-e58f4462ad41_Basecall_2D_2d;Target=ae6a7818-85b5-4739-8031-e58f4462ad41_Basecall_2D_2d 141 489;Gap=M13 I10 M8 D3 M8 D3 M32 D1 M4 D2 M5 D3 M34 I3 M6 D1 M7 D1 M5 D1 M30 D2 M16 I1 M8 D3 M10 D1 M5 D1 M6 I1 M8 I1 M8 D1 M33 D3 M33 I1 M28 D3 M25&#xA;###&#xA;3R    gmapidx    cDNA_match    15853880    15855465    96    +    . ID=dd2444cf-34d6-4cd3-87ab-0ae1f3cb1f96_Basecall_2D_2d.path1;Name=dd2444cf-34d6-4cd3-87ab-0ae1f3cb1f96_Basecall_2D_2d;Target=dd2444cf-34d6-4cd3-87ab-0ae1f3cb1f96_Basecall_2D_2d 80 1645;Gap=M46 D2 M12 D2 M157 D3 M4 D1 M50 I2 M3 I1 M12 I1 M68 I1 M66 I1 M53 D2 M47 D1 M16 D1 M35 D1 M155 D1 M28 D1 M166 D2 M47 D1 M8 D4 M69 D1 M28 D1 M5 D1 M12 D1 M202 I1 M115 D1 M61 I2 M7 D1 M7 I1 M36 D2 M41&#xA;3R    gmapidx    cDNA_match    15855529    15855742    97    +    . ID=dd2444cf-34d6-4cd3-87ab-0ae1f3cb1f96_Basecall_2D_2d.path1;Name=dd2444cf-34d6-4cd3-87ab-0ae1f3cb1f96_Basecall_2D_2d;Target=dd2444cf-34d6-4cd3-87ab-0ae1f3cb1f96_Basecall_2D_2d 1646 1856;Gap=M21 D1 M68 D1 M85 D1 M37&#xA;###&#xA;X    gmapidx    cDNA_match    14837810    14838142    93    -    . ID=960b50cd-945e-4c12-b9bc-367f965575bb_Basecall_2D_2d.path1;Name=960b50cd-945e-4c12-b9bc-367f965575bb_Basecall_2D_2d;Target=960b50cd-945e-4c12-b9bc-367f965575bb_Basecall_2D_2d 74 406;Gap=M13 D1 M182 I1 M14 I2 M56 I1 M30 D2 M21 D1 M13&#xA;X    gmapidx    cDNA_match    14837470    14837753    92    -    . ID=960b50cd-945e-4c12-b9bc-367f965575bb_Basecall_2D_2d.path1;Name=960b50cd-945e-4c12-b9bc-367f965575bb_Basecall_2D_2d;Target=960b50cd-945e-4c12-b9bc-367f965575bb_Basecall_2D_2d 407 688;Gap=I2 M5 I1 M64 I1 M44 D1 M9 D5 M31 D3 M23 I1 M19 I1 M25 I1 M26 D1 M19 I1 M9&#xA;###&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="294" LastEditorUserId="294" LastEditDate="2017-07-18T16:41:08.203" LastActivityDate="2021-07-09T10:13:02.447" Title="How to convert GFF3 to GTF2" Tags="|software-recommendation|format-conversion|gff3|gtf|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2069" PostTypeId="2" ParentId="2068" CreationDate="2017-07-14T13:46:53.463" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;http://genometools.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GenomeTools&lt;/a&gt; package has lots of small applications that let you massage GFF3. It may have a converter, or it may let you tweak your existing GFF3 so that gffread works&lt;/p&gt;&#xA;" OwnerUserId="931" LastActivityDate="2017-07-14T13:46:53.463" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2070" PostTypeId="2" ParentId="829" CreationDate="2017-07-14T16:12:50.580" Score="5" Body="&lt;p&gt;&lt;strong&gt;Ensembl&lt;/strong&gt; contains this information: When you &lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Gene/Phenotype?db=core;g=ENSG00000164508;r=6:25726132-25726527;t=ENST00000297012#ALL_tablePanel&quot; rel=&quot;noreferrer&quot;&gt;go to the “phenotype” menu item of a given gene&lt;/a&gt;, you will see a list of variants (potentially after clicking on “ALL associated variants”) with their associated genomic position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can subtract the transcript’s start position from that position to find out the residue (of the translated sequence, and be careful to subtract the lengths of all preceding introns!). Unfortunately this seems to require some manual work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively you can &lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Variation/Phenotype?db=core;g=ENSG00000164508;r=6:25726132-25726527;t=ENST00000297012;v=rs2275906;vdb=variation;vf=1677930&quot; rel=&quot;noreferrer&quot;&gt;query the individual SNPs&lt;/a&gt;, which will tell you their position in all transcripts (in this case: Leu 298/68/244):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/zr2Q4.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zr2Q4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2017-07-14T16:12:50.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2072" PostTypeId="2" ParentId="2068" CreationDate="2017-07-15T11:30:13.013" Score="5" Body="&lt;p&gt;The following bit of python code should work:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python&#xA;import sys&#xA;&#xA;lastTranscript = [None, None, None, []]  # ID, chrom, strand, [(start, end, score), ...]&#xA;&#xA;&#xA;def getID(s):&#xA;    &quot;&quot;&quot;Parse out the ID attribute&quot;&quot;&quot;&#xA;    s = s.split(&quot;;&quot;)&#xA;    for k in s:&#xA;        if k.startswith(&quot;ID=&quot;):&#xA;            return k[3:]&#xA;    return None&#xA;&#xA;&#xA;def dumpLastTranscript():&#xA;    &quot;&quot;&quot;Print the last transcript&quot;&quot;&quot;&#xA;    bounds = sorted(lastTranscript[3])&#xA;    print(&quot;{}\tgmapidx\tgene\t{}\t{}\t.\t{}\t.\tgene_id \&quot;{}\&quot;; transcript_id \&quot;{}\&quot;;&quot;.format(lastTranscript[1], bounds[0][0], bounds[-1][1], lastTranscript[2], lastTranscript[0], lastTranscript[0]))&#xA;    print(&quot;{}\tgmapidx\ttranscript\t{}\t{}\t.\t{}\t.\tgene_id \&quot;{}\&quot;; transcript_id \&quot;{}\&quot;;&quot;.format(lastTranscript[1], bounds[0][0], bounds[-1][1], lastTranscript[2], lastTranscript[0], lastTranscript[0]))&#xA;    for start, end, score in bounds:&#xA;        print(&quot;{}\tgmapidx\texon\t{}\t{}\t{}\t{}\t.\tgene_id \&quot;{}\&quot;; transcript_id \&quot;{}\&quot;;&quot;.format(lastTranscript[1], start, end, score, lastTranscript[2], lastTranscript[0], lastTranscript[0]))&#xA;&#xA;&#xA;def handleLine(cols):&#xA;    &quot;&quot;&quot;Handle a single line, appending the exon bounds to the previous if relevant&quot;&quot;&quot;&#xA;    ID = getID(cols[8])&#xA;    assert(ID is not None)&#xA;    if lastTranscript[0] is not None and lastTranscript[0] != ID:&#xA;        dumpLastTranscript()&#xA;        lastTranscript[3] = []&#xA;    lastTranscript[0] = ID&#xA;    lastTranscript[1] = cols[0]&#xA;    lastTranscript[2] = cols[6]&#xA;    lastTranscript[3].append((int(cols[3]), int(cols[4]), cols[5]))&#xA;&#xA;&#xA;f = open(sys.argv[1])&#xA;for line in f:&#xA;    if line.startswith(&quot;#&quot;):&#xA;        continue&#xA;    cols = line.strip().split(&quot;\t&quot;)&#xA;    handleLine(cols)&#xA;&#xA;dumpLastTranscript()&#xA;f.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you saved that as &lt;code&gt;gff2gtf.py&lt;/code&gt; and made it executable, the usage would be &lt;code&gt;./gff2gtf.py foo.gff3 &amp;gt; foo.gtf&lt;/code&gt;. With the example that you provided in your post, the result is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;2L  gmapidx gene    18442664    18443024    .   -   .   gene_id &quot;ae6a7818-85b5-4739-8031-e58f4462ad41_Basecall_2D_2d.path1&quot;; transcript_id &quot;ae6a7818-85b5-4739-8031-e58f4462ad41_Basecall_2D_2d.path1&quot;;&#xA;2L  gmapidx transcript  18442664    18443024    .   -   .   gene_id &quot;ae6a7818-85b5-4739-8031-e58f4462ad41_Basecall_2D_2d.path1&quot;; transcript_id &quot;ae6a7818-85b5-4739-8031-e58f4462ad41_Basecall_2D_2d.path1&quot;;&#xA;2L  gmapidx exon    18442664    18443024    79  -   .   gene_id &quot;ae6a7818-85b5-4739-8031-e58f4462ad41_Basecall_2D_2d.path1&quot;; transcript_id &quot;ae6a7818-85b5-4739-8031-e58f4462ad41_Basecall_2D_2d.path1&quot;;&#xA;3R  gmapidx gene    15853880    15855742    .   +   .   gene_id &quot;dd2444cf-34d6-4cd3-87ab-0ae1f3cb1f96_Basecall_2D_2d.path1&quot;; transcript_id &quot;dd2444cf-34d6-4cd3-87ab-0ae1f3cb1f96_Basecall_2D_2d.path1&quot;;&#xA;3R  gmapidx transcript  15853880    15855742    .   +   .   gene_id &quot;dd2444cf-34d6-4cd3-87ab-0ae1f3cb1f96_Basecall_2D_2d.path1&quot;; transcript_id &quot;dd2444cf-34d6-4cd3-87ab-0ae1f3cb1f96_Basecall_2D_2d.path1&quot;;&#xA;3R  gmapidx exon    15853880    15855465    96  +   .   gene_id &quot;dd2444cf-34d6-4cd3-87ab-0ae1f3cb1f96_Basecall_2D_2d.path1&quot;; transcript_id &quot;dd2444cf-34d6-4cd3-87ab-0ae1f3cb1f96_Basecall_2D_2d.path1&quot;;&#xA;3R  gmapidx exon    15855529    15855742    97  +   .   gene_id &quot;dd2444cf-34d6-4cd3-87ab-0ae1f3cb1f96_Basecall_2D_2d.path1&quot;; transcript_id &quot;dd2444cf-34d6-4cd3-87ab-0ae1f3cb1f96_Basecall_2D_2d.path1&quot;;&#xA;X   gmapidx gene    14837470    14838142    .   -   .   gene_id &quot;960b50cd-945e-4c12-b9bc-367f965575bb_Basecall_2D_2d.path1&quot;; transcript_id &quot;960b50cd-945e-4c12-b9bc-367f965575bb_Basecall_2D_2d.path1&quot;;&#xA;X   gmapidx transcript  14837470    14838142    .   -   .   gene_id &quot;960b50cd-945e-4c12-b9bc-367f965575bb_Basecall_2D_2d.path1&quot;; transcript_id &quot;960b50cd-945e-4c12-b9bc-367f965575bb_Basecall_2D_2d.path1&quot;;&#xA;X   gmapidx exon    14837470    14837753    92  -   .   gene_id &quot;960b50cd-945e-4c12-b9bc-367f965575bb_Basecall_2D_2d.path1&quot;; transcript_id &quot;960b50cd-945e-4c12-b9bc-367f965575bb_Basecall_2D_2d.path1&quot;;&#xA;X   gmapidx exon    14837810    14838142    93  -   .   gene_id &quot;960b50cd-945e-4c12-b9bc-367f965575bb_Basecall_2D_2d.path1&quot;; transcript_id &quot;960b50cd-945e-4c12-b9bc-367f965575bb_Basecall_2D_2d.path1&quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That at least looks like correct gtf 2.2 format to me.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-07-17T07:29:29.317" LastActivityDate="2017-07-17T07:29:29.317" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2073" PostTypeId="1" CreationDate="2017-07-15T15:02:59.803" Score="3" ViewCount="748" Body="&lt;p&gt;I'm currently learning about Gene Set Enrichment Analysis (&lt;a href=&quot;http://software.broadinstitute.org/gsea/index.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSEA&lt;/a&gt;) in the hopes of using it in my analysis of differentially expressed genes, and I just had a few questions about the program, specifically about GSEAPreranked, which I need cleared up.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;On the ranked list needed for GSEA input, should the list include all genes, or only those that pass a certain threshold of significance (i.e. fold change higher than 2, p value less than 0.05, etc.)? Ideally I'd like to sort the genes by fold change alone as I don't trust my p values as much, so should I only include genes with high fold changes?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I am comparing multiple conditions of disease with different treatments. Am I correct that GSEA only compares two conditions? If this is the case should I run GSEA for each control/treatment comparison? Would this be conventional?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="506" LastEditorUserId="57" LastEditDate="2017-07-17T15:17:56.247" LastActivityDate="2018-04-13T00:42:26.610" Title="Basic questions about GSEA" Tags="|gene|sequence-annotation|rna|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2074" PostTypeId="2" ParentId="2073" CreationDate="2017-07-15T15:46:52.880" Score="6" Body="&lt;p&gt;You seem to refer to the GSEA provided by the &lt;a href=&quot;http://software.broadinstitute.org/gsea/index.jsp&quot; rel=&quot;noreferrer&quot;&gt;Broad institute&lt;/a&gt;, (there are other GSEA algorithms). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) You can provide whatever you wish, but if you want to know if those gene sets in which side of the ordered list are they, then provide all the list (of genes) you have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) GSEA analyize if the order of a given list distributes in certain way the elements of another (unordered) list. In your case you would need to compute the GSEA for each treatment vs control comparison.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2017-07-15T15:46:52.880" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2075" PostTypeId="2" ParentId="2049" CreationDate="2017-07-16T13:37:45.500" Score="2" Body="&lt;p&gt;I'm not sure what you mean by it being a mess? That looks like a pretty good alignment to me, and it most certainly has aligned all the sequences. You are nearly always going to have gaps (see my MSA below, which are all paralogs from the &lt;em&gt;same genome&lt;/em&gt; so they couldn't really be more closely related, and yet, there are gaps). That comes with its own set of problems mind you, as you need to decide how you're going to deal with them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also don't necessarily have to do a profile alignment. Those sequences don't look massively divergent to me, so straight forward sequence alignment would probably give you more or less the same result.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My goal is to visualize the number of BLAST hits per amino acid in my protein of interest.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This doesn't really make sense. You should really be searching for the number of BLAST hits with a particular domain, if you want to infer homology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want a per-position visualisation of the conservation, you could look at the shannon entropy for each column and plot that. I wrote a script to do just that a little while ago: &lt;a href=&quot;https://github.com/jrjhealey/bioinfo-tools/blob/master/Shannon.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/jrjhealey/bioinfo-tools/blob/master/Shannon.py&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just beware it's not super well tested yet. Feed an MSA in with as many sequences as you want to analyse, but you'll have to have identified the sequences and done the alignment first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, given this MSA:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    16    149&#xA;PAU_02775  MSTTPEQIAV EYPIPTYRFV VSLGDEQIPF NSVSGLDISH DVIEYKDGTG&#xA;PLT_01696  MSTTPEQIAV EYPIPTYRFV VSIGDEQIPF NSVSGLDISH DVIEYKDGTG&#xA;PAK_02606  MSTTPEQIAV EYPIPTYRFV VSIGDEQVPF NSVSGLDISH DVIEYKDGTG&#xA;PLT_01736  MSTTPEQIAV EYPIPTYRFV VSIGDEKVPF NSVSGLDISH DVIEYKDGTG&#xA;PAK_01896  MTTTT----V DYPIPAYRFV VSVGDEQIPF NNVSGLDITY DVIEYKDGTG&#xA;PAU_02074  MATTT----V DYPIPAYRFV VSVGDEQIPF NSVSGLDITY DVIEYKDGTG&#xA;PLT_02424  MSVTTEQIAV DYPIPTYRFV VSVGDEQIPF NNVSGLDITY DVIEYKDGTG&#xA;PLT_01716  MTITPEQIAV DYPIPAYRFV VSVGDEKIPF NNVSGLDVHY DVIEYKDGTG&#xA;PLT_01758  MAITPEQIAV EYPIPTYRFV VSVGDEQIPF NNVSGLDVHY DVIEYKDGIG&#xA;PAK_03203  MSTSTSQIAV EYPIPVYRFI VSIGDDQIPF NSVSGLDINY DTIEYRDGVG&#xA;PAU_03392  MSTSTSQIAV EYPIPVYRFI VSVGDEKIPF NSVSGLDISY DTIEYRDGVG&#xA;PAK_02014  MSITQEQIAA EYPIPSYRFM VSIGDVQVPF NSVSGLDRKY EVIEYKDGIG&#xA;PAU_02206  MSITQEQIAA EYPIPSYRFM VSIGDVQVPF NSVSGLDRKY EVIEYKDGIG&#xA;PAK_01787  MSTTADQIAV QYPIPTYRFV VTIGDEQMCF QSVSGLDISY DTIEYRDGVG&#xA;PAU_01961  MSTTADQIAV QYPIPTYRFV VTIGDEQMCF QSVSGLDISY DTIEYRDGVG&#xA;PLT_02568  MSTTVDQIAV QYPIPTYRFV VTVGDEQMSF QSVSGLDISY DTIEYRDGIG&#xA;&#xA;           NYYKMPGQRQ AINISLRKGV FSGDTKLFDW INSIQLNQVE KKDISISLTN&#xA;           NYYKMPGQRQ AINISLRKGV FSGDTKLFDW INSIQLNQVE KKDISISLTN&#xA;           NYYKMPGQRQ AINISLRKGV FSGDTKLFDW INSIQLNQVE KKDISISLTN&#xA;           NYYKMPGQRQ AINITLRKGV FSGDTKLFDW LNSIQLNQVE KKDISISLTN&#xA;           NYYKMPGQRQ LINITLRKGV FPGDTKLFDW LNSIQLNQVE KKDVSISLTN&#xA;           NYYKMPGQRQ LINITLRKGV FPGDTKLFDW LNSIQLNQVE KKDVSISLTN&#xA;           NHYKMPGQRQ LINITLRKGV FPGDTKLFDW LNSIQLNQVE KKDVSISLTN&#xA;           NYYKMPGQRQ SINITLRKGV FPGDTKLFDW INSIQLNQVE KKDIAISLTN&#xA;           NYYKMPGQRQ SINITLRKGV FPGDTKLFDW INSIQLNQVE KKDIAISLTN&#xA;           NWFKMPGQSQ LVNITLRKGV FPGKTELFDW INSIQLNQVE KKDITISLTN&#xA;           NWFKMPGQSQ STNITLRKGV FPGKTELFDW INSIQLNQVE KKDITISLTN&#xA;           NYYKMPGQIQ RVDITLRKGI FSGKNDLFNW INSIELNRVE KKDITISLTN&#xA;           NYYKMPGQIQ RVDITLRKGI FSGKNDLFNW INSIELNRVE KKDITISLTN&#xA;           NWLQMPGQRQ RPTITLKRGI FKGQSKLYDW INSISLNQIE KKDISISLTD&#xA;           NWLQMPGQRQ RPTITLKRGI FKGQSKLYDW INSISLNQIE KKDISISLTD&#xA;           NWLQMPGQRQ RPSITLKRGI FKGQSKLYDW INSISLNQIE KKDISISLTD&#xA;&#xA;           EAGTEILMTW SVANAFPTSL TSPSFDATSN EVAVQEITLT ADRVTIQAA&#xA;           EAGTEILMTW SVANAFPTSL ISPSFDATSN EVAVQEITLT ADRVTIQAA&#xA;           EAGTEILMTW SVANAFPTSL TSPSFDATSN EVAVQEITLT ADRVTIQAA&#xA;           EAGTEILMTW SVANAFPTSL TAPAFDATSN EVAVQEISLT ADRVTIQAA&#xA;           ETGTEILMSW SVANAFPTSL TSPSFDATSN DIAVQEIKLT ADRVTIQAA&#xA;           EVGTEILMTW SVANAFPTSL TSPSFDATSN DIAVQEIKLT ADRVTIQAA&#xA;           EAGTEILMSW SVANAFPTSL TSPSFDATSN DIAVQEIKLT ADRVMIQAA&#xA;           ETGSQILMTW NVANAFPTSF TSPSFDAASN DIAIQEIALV ADRVTIQAP&#xA;           EAGTEILMTW NVANAFPTSF TSPSFDATSN EIAVQEIALT ADRVTIQAA&#xA;           DAGTELLMTW NVSNAFPTSL TSPSFDATSN DIAVQEITLT ADRVIMQAV&#xA;           DAGTELLMTW NVSNAFPTSL TSPSFDATSN DIAVQEITLM ADRVIMQAV&#xA;           DTGSEVLMSW VVSNAFPSSL TAPSFDASSN EIAVQEISLV ADRVTIQVP&#xA;           DTGSKVLMSW VVSNAFPSSL TAPSFDASSN EIAVQEISLV ADRVTIQVP&#xA;           ETGSNLLITW NIANAFPEKL TAPSFDATSN EVAVQEMSLK ADRVTVEFH&#xA;           ETGSNLLITW NIANAFPEKL TAPSFDATSN EVAVQEISLK ADRVTVEFH&#xA;           ETGSNLLITW NIANAFPEKL TAPSFDATSN EVAVQEISLK ADRVTVEFH&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You'd get this plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zCFTz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zCFTz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I would like to have a graph with my protein on the x-axis and a plot&#xA;  like the regions of high conservation for the multiple alignments,&#xA;  except with the spikes corresponding to a high number of BLAST hits.&#xA;  This would allow me to identify regions of my protein that have higher&#xA;  sequence similarity to bacteria than others.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not sure your logic is quite right here though. Blast won't give you hits depending on a particular position. It's a local aligner, so it'll just return you hits where at least some part of your query matches at least some part of another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you could do is take the logic in the script above, and just use a different metric. For example, perhaps you could count the proportion of sequences which have the most common amino acid at a given position within your MSA. That would be fairly crude though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you say in the comments,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;My final goal is to produce a plot visualizing regions of high bacterial sequence similarity to my human protein of interest.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;your original alignment will show you this intrinsically, if only you include the sequences of all the BLAST hits in the first place. Thus your work flow will be:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Blast sequence of interest.&lt;/li&gt;&#xA;&lt;li&gt;Download all/as many hits as you want (bear in mind the E-value/bitscore and the number of hits you get. It might only be a few dozen, in which case you can use the lot, but if not, just take all the hits below a certain cut-off.)&lt;/li&gt;&#xA;&lt;li&gt;Align all the sequences.&lt;/li&gt;&#xA;&lt;li&gt;Look at the column scores for the whole MSA. You can use whatever metric of conservation you like really. Might be as simple as proportion of sequences with the most common residue, or something more complex like Shannon entropy (though as you can see in the graph above Shannon entropy can be kinda noisy) etc.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="929" LastActivityDate="2017-07-16T13:37:45.500" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2076" PostTypeId="2" ParentId="997" CreationDate="2017-07-17T00:31:47.860" Score="4" Body="&lt;p&gt;I don't know of any transcript-to-transcript aligners that are able to do this, but &lt;a href=&quot;http://last.cbrc.jp/doc/lastal.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;LAST&lt;/a&gt; can align transcript queries to protein reference sequences using a specified frameshift cost. Here's the specific documentation for that option:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;-F COST   &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Align DNA queries to protein reference sequences, using the specified&#xA;  frameshift cost. A value of 15 seems to be reasonable. (As a special&#xA;  case, -F0 means DNA-versus-protein alignment without frameshifts,&#xA;  which is faster.) The output looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a score=108 s prot 2  40 + 649&#xA;FLLQAVKLQDP-STPHQIVPSP-VSDLIATHTLCPRMKYQDD s dna  8 117 + 999&#xA;FFLQ-IKLWDP\STPH*IVSSP/PSDLISAHTLCPRMKSQDN&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;The \ indicates a forward shift by one nucleotide, and the / indicates&#xA;  a reverse shift by one nucleotide. The * indicates a stop codon. The&#xA;  same alignment in tabular format looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;108 prot 2 40 + 649 dna 8 117 + 999 4,1:0,6,0:1,10,0:-1,19&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;The &quot;-1&quot; indicates the reverse frameshift.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I sent an email to the &lt;a href=&quot;https://groups.google.com/forum/#!forum/last-align&quot; rel=&quot;nofollow noreferrer&quot;&gt;LAST mailing list&lt;/a&gt; about adding a frameshift penalty for transcript-to-transcript matching; I've been pleasantly surprised with the requested features that Martin Frith has added to LAST in the past. Unfortunately, in this case the problem is too difficult to sort out due to all the possible combinations that could happen, so it's unlikely to be implemented in LAST in the forseeable future (unless someone else writes that code).&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-07-22T11:05:35.870" LastActivityDate="2017-07-22T11:05:35.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2077" PostTypeId="2" ParentId="2056" CreationDate="2017-07-17T02:10:55.023" Score="2" Body="&lt;p&gt;Note that DIN (DNA Integrity Number) and RIN (RNA Integrity Number) are different scores. I've had trouble in the past finding code or formulas for calculating both of these scores, which is very frustrating considering how frequently they are used in research papers and for NGS QC. The closest that I have been able to get to this are demonstrative graphs for various &lt;a href=&quot;http://bmcmolbiol.biomedcentral.com/articles/10.1186/1471-2199-7-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;RIN numbers&lt;/a&gt; and &lt;a href=&quot;http://www.agilent.com/cs/library/applications/5991-5258EN.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DIN numbers&lt;/a&gt;, with a description of the variables that are included in the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code that @719016 provided appears to attempt to calculate the RIN, not the DIN.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just in case it's helpful, here is an extract of that code, only including lines that seem to be directly involved in calculating the actual B_RIN values. This is not runnable, because it depends on particular input files, and it's still pretty unintelligible:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;myfiles&amp;lt;-list.files()&#xA;results&amp;lt;-myfiles[which(substr(myfiles,69,79)==&quot;Results.csv&quot;)]&#xA;n &amp;lt;- which(substr(myfiles,69,79)==&quot;Results.csv&quot;)&#xA;targets &amp;lt;- read.delim(myfiles[n[1]], sep=&quot;,&quot;,header=F,skip=14,as.is=T)&#xA;targets2&amp;lt;- read.delim(myfiles[n[1]], sep=&quot;,&quot;,header=F,skip=0,as.is=T)&#xA;str3&amp;lt;-paste0(substr(targets2[1,2],1,67),&quot;_S&quot;)&#xA;chip_num&amp;lt;-length(n)&#xA;samp_num_chip&amp;lt;-length(grep(str3,myfiles))&#xA;samp_num&amp;lt;-length(which(substr(myfiles,69,74)==&quot;Sample&quot;))&#xA;samples1&amp;lt;- matrix(NA, ncol=chip_num, nrow=samp_num)&#xA;for(i in 1:chip_num)&#xA;{&#xA;    ##mod chip numb&#xA;    targets2&amp;lt;- read.delim(myfiles[n[i]], sep=&quot;,&quot;,header=F,skip=0,as.is=T)&#xA;    str3&amp;lt;-paste0(substr(targets2[1,2],1,67),&quot;_S&quot;)&#xA;    samp_num_chip&amp;lt;-length(grep(str3,myfiles))&#xA;    ##mod sample numb&#xA;    str1&amp;lt;-substr(targets2[1,2],1,67)&#xA;    for(j in 1:samp_num_chip)&#xA;    {&#xA;        iii&amp;lt;-j&#xA;        str5&amp;lt;-paste0(str1,&quot;_Sample&quot;,iii,&quot;.csv&quot;)&#xA;        samples1[j,i]&amp;lt;-str5&#xA;    }&#xA;}&#xA;samples1&amp;lt;-samples1[grep(&quot;2100&quot;,samples1)]&#xA;##RIN Calculator#######refined peakfinder####################&#xA;##read bioanalyzer data into a matrix called dta&#xA;##since the total RNA and mRNA assays run differently&#xA;##skip more lines for the total RNA assay&#xA;elec &amp;lt;- read.delim(&quot;ba_lane.txt&quot;, sep=&quot;\t&quot;,header=T,as.is=T)&#xA;qc.mat &amp;lt;- matrix(NA, ncol=1, nrow=nrow(elec))&#xA;dta &amp;lt;- matrix(NA, nrow=1060, ncol=nrow(elec))&#xA;for(i in 1:nrow(elec))&#xA;{&#xA;    x &amp;lt;- read.csv(as.character(elec[i,2]), header=F, skip=18, nrows=1060)&#xA;    dta[,i] &amp;lt;- x[,2]&#xA;}&#xA;time&amp;lt;- x[,1]&#xA;for(samples in 1:length(samples1))&#xA;{&#xA;    ##store max fluor in max.peaks matrix&#xA;    lad&amp;lt;-samples&#xA;    ti1&amp;lt;-17&#xA;    ti2&amp;lt;-65&#xA;    p.row&amp;lt;-length(dta[which(time==ti1):which(time==ti1+1),lad])*(ti2-ti1+1)&#xA;    peaks &amp;lt;- matrix(NA, nrow=p.row, ncol=1)&#xA;    max.peaks &amp;lt;- matrix(NA, nrow=length(seq(ti1,ti2,.05)), ncol=1)&#xA;    for (i in 1:length(seq(ti1,ti2,.05)))&#xA;    {&#xA;        ii&amp;lt;-seq(ti1,ti2,.05)[i]&#xA;        x0&amp;lt;- ii&#xA;        x0&amp;lt;-round(x0,digits=3)&#xA;        y0&amp;lt;- ii&#xA;        x&amp;lt;- ii+.05&#xA;        x&amp;lt;-round(x,digits=3)&#xA;        y&amp;lt;- ii&#xA;        max.peaks[i]&amp;lt;- max(dta[which(time==x0):which(time==x),lad])&#xA;    }    &#xA;    ##human or mouse&#xA;    cent&amp;lt;-max(max.peaks)*.6&#xA;    ##flatworm&#xA;    ##cent&amp;lt;-max(max.peaks)*.05&#xA;    ##define the middle of the segments&#xA;    max.avg&amp;lt;-seq(ti1+.025,ti2+.025,.05)&#xA;    ##find time associated with peaks &amp;gt; 5 fu and fill seconds matrix with them&#xA;    ##12.5% of max fu incase chip is fu&#xA;    sizes &amp;lt;- which(max.peaks&amp;gt;cent)&#xA;    gt35&amp;lt;-which(max.avg[sizes]&amp;gt;35)&#xA;    lt55&amp;lt;-which(max.avg[sizes]&amp;lt;55)&#xA;    tfr&amp;lt;-which(duplicated(c(gt35,lt55))==TRUE)&#xA;    area&amp;lt;-cbind(max.peaks[sizes-1][c(gt35,lt55)[tfr]],&#xA;                max.peaks[sizes][c(gt35,lt55)[tfr]],&#xA;                max.peaks[sizes+1][c(gt35,lt55)[tfr]])&#xA;    time2&amp;lt;-cbind(max.avg[sizes-1][c(gt35,lt55)[tfr]],&#xA;                 max.avg[sizes][c(gt35,lt55)[tfr]],&#xA;                 max.avg[sizes+1][c(gt35,lt55)[tfr]])&#xA;    t1&amp;lt;-time2[which(area[,1]&amp;lt;cent),1][1]&#xA;    t2&amp;lt;-time2[which(area[,1]&amp;lt;cent),1][2]&#xA;    t3&amp;lt;-time2[which(area[,3]&amp;lt;cent),3][1]&#xA;    t4&amp;lt;-time2[which(area[,3]&amp;lt;cent),3][2]&#xA;    s18a&amp;lt;-ifelse(is.na(t1,-30,round(t1,digits=1)))&#xA;    s18b&amp;lt;-ifelse(is.na(t3,-35.05,round(t3,digits=1)))&#xA;    s28a&amp;lt;-ifelse(is.na(t2,-30,round(t2,digits=1)))&#xA;    s28b&amp;lt;-ifelse(is.na(t4,-30.05,round(t4,digits=1)))&#xA;    eta&amp;lt;-which(time==s18a)&#xA;    etb&amp;lt;-which(time==s18b)&#xA;    tea&amp;lt;-which(time==s28a)&#xA;    teb&amp;lt;-which(time==s28b)&#xA;    s18&amp;lt;-sum(dta[eta:etb,lad])&#xA;    s28&amp;lt;-sum(dta[tea:teb,lad])&#xA;    qc.mat[samples]&amp;lt;-  (-1*exp((s28/s18)*-1)+1)*10&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Mostly based on that last line, the RIN seems to be associated with the ratio of the 18s peak to the 28s peak.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-07-17T02:10:55.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2078" PostTypeId="1" AcceptedAnswerId="2112" CreationDate="2017-07-16T20:23:10.543" Score="4" ViewCount="2067" Body="&lt;p&gt;I've got a decent knowledge of programming (incl. bash scripting) but I fail to understand how Arlequin works. Could you please help me with a very simple reproducible example on how to use Arlequin via the command line?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the .zip file comes with its pre-compiled version, I'll just note that I'm on Mac OS X but of course, a reproducible example with Linux (or at worst with Windows) would also be helpful.&lt;/p&gt;&#xA;" OwnerUserId="888" OwnerDisplayName="Remi.b" LastEditorUserId="888" LastEditDate="2017-07-20T23:10:33.560" LastActivityDate="2019-04-16T00:13:57.217" Title="How can I use Arlequin via the command line?" Tags="|genome|statistics|positive-selection|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2079" PostTypeId="2" ParentId="2073" CreationDate="2017-07-17T06:35:19.213" Score="6" Body="&lt;p&gt;Use the following R package for Gene Set Enrichment analysis of RNA-seq data: &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/SeqGSEA.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqGSEA&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is another R package (&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/fgsea.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;fgsea&lt;/a&gt;) recently published called &quot;Fast Gene Set Enrichment Analysis&quot; by Alexey Sergushichev.&lt;/p&gt;&#xA;" OwnerUserId="206" LastEditorUserId="48" LastEditDate="2017-12-06T22:31:01.777" LastActivityDate="2017-12-06T22:31:01.777" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2080" PostTypeId="2" ParentId="2067" CreationDate="2017-07-17T11:21:39.053" Score="2" Body="&lt;p&gt;Roary also takes into account paralogs, so sometimes two core genes are split into different groups based on their neighbour genes and they end up with different nomenclature (group_*...). As suggested by Andrew Page in the github issue I would consider the gene_presence_absence.Rtab (this contains all the orthologous genes) and remove rows corresponding to vectors only containing 1s (core genes). In this way you will have a matrix of 1 and 0 corresponding to presence/absence of a particular accessory gene in your isolates. &lt;/p&gt;&#xA;" OwnerUserId="446" LastActivityDate="2017-07-17T11:21:39.053" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2081" PostTypeId="1" CreationDate="2017-07-17T12:22:15.753" Score="4" ViewCount="583" Body="&lt;p&gt;When doing Illumina 2x150bp sequencing of genomic DNA, and after aligning the reads to GRCh38, what percentage of the non-N fraction of the human genome is MAPQ=0? This is, what part corresponds to regions that can't be uniquely mapped with 2x150bp reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And, how many genes are affected by MAPQ=0 regions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I presume the numbers will dance around depending on fragment sizes, read quality, etc. but I am happy with some starting numbers.&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="57" LastEditDate="2017-07-20T12:56:04.567" LastActivityDate="2024-01-02T11:28:56.360" Title="what percentage of the human genome is MAPQ=0?" Tags="|genome|read-mapping|human|illumina|mapq|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2082" PostTypeId="2" ParentId="2067" CreationDate="2017-07-17T14:19:35.680" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/jasonsahl/LS-BSR&quot; rel=&quot;nofollow noreferrer&quot;&gt;LS-BSR&lt;/a&gt; should be able to give you what you're looking for. See the &lt;a href=&quot;https://peerj.com/articles/332/&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After you run the primary analysis, there is a simple documented workflow for splitting the pangenome into core and accessory, based on a user defined threshold. I'm the developer, so can help if you run into any problems.&lt;/p&gt;&#xA;" OwnerUserId="1126" LastEditorUserId="48" LastEditDate="2017-11-27T15:50:38.583" LastActivityDate="2017-11-27T15:50:38.583" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2083" PostTypeId="1" CreationDate="2017-07-17T15:57:38.820" Score="6" ViewCount="1835" Body="&lt;p&gt;I am trying to translate (lift over) bed files describing genomic regions from hg37 to hg38. I have tried both UCSC's &lt;a href=&quot;http://genome.ucsc.edu/cgi-bin/hgLiftOver&quot; rel=&quot;noreferrer&quot;&gt;LiftOver&lt;/a&gt; tool and &lt;a href=&quot;http://crossmap.sourceforge.net/&quot; rel=&quot;noreferrer&quot;&gt;CrossMap&lt;/a&gt; but saw that they give me different results. I therefore need a way of assessing how correct the results are. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I decided to test by downloading a bed file of the RefSeq genes for hg37 and one for hg38 from UCSC. Then, I run &lt;code&gt;liftOver&lt;/code&gt; to translate the hg37 file to hg39 coordinates and now I want to compare the results of the liftover to the bed file I downloaded for hg38.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I usefully compare these files? I am hoping for a tool that can report how similar the two are. Ideally, I would like to see three things in the output:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Matching regions: those that are shown with identical positions in the liftover results and the UCSC bed file. &lt;/li&gt;&#xA;&lt;li&gt;Non-matching regions: those whose coordinates do not match between the two files. &lt;/li&gt;&#xA;&lt;li&gt;(if possible) fuzzy matches: those that are not identical but pretty close (off by a few nucleotides).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I thought it would be easy to do with a little awk script but it gets quite complicated because certain RefSeq genes are mapped to multiple positions (e.g. &lt;a href=&quot;http://www.genenames.org/cgi-bin/gene_symbol_report?hgnc_id=41553&quot; rel=&quot;noreferrer&quot;&gt;MIR4454&lt;/a&gt;). And the 3rd requirement (fuzzy) is not trivial to do in awk. I can live without that though, if a tool can give me the other two. &lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="73" LastEditDate="2023-01-15T20:20:41.453" LastActivityDate="2023-01-15T20:20:41.453" Title="How can I compare two bed files?" Tags="|bed|awk|liftover|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2084" PostTypeId="2" ParentId="2083" CreationDate="2017-07-17T18:09:10.553" Score="2" Body="&lt;p&gt;I wrote &lt;a href=&quot;http://aegean.readthedocs.io/en/stable/parseval.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ParsEval&lt;/a&gt; to handle these types of comparisons. ParsEval reports a variety of similarity statistics at both the nucleotide level and feature (exon) level. It doesn't explicitly support &quot;fuzzy&quot; matches, but it does report scaled values between 0.0 and 1.0 for similarity, so you could get a similar feel by definig a similarity cutoff for filtering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Caveat: ParsEval doesn't handle BED files, only GFF3.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2017-07-17T18:09:10.553" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2085" PostTypeId="2" ParentId="2083" CreationDate="2017-07-17T19:06:47.940" Score="3" Body="&lt;p&gt;From a set operations viewpoint, consider your hg37 regions as a &quot;reference&quot; set and hg38 as a &quot;map&quot; set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have these sets in BED format, &lt;a href=&quot;http://bedops.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS&lt;/a&gt; lets you do set operations on them with various overlap criteria.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, you can use &lt;a href=&quot;http://bedops.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS &lt;code&gt;bedmap&lt;/code&gt;&lt;/a&gt; to map hg38 regions to hg37 regions. The &lt;code&gt;bedmap&lt;/code&gt; tool lets you assign overlap criteria from as little as one base to as much as full, mutual overlap. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you do mapping, you can do operations, like counting the number of regions that meet the overlap criteria between reference and map sets, and calculate percentages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, use &lt;code&gt;--exact&lt;/code&gt; to get exactly-matching regions between hg37 and hg38:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --count --exact hg37.bed hg38.bed | awk '{s+=$1}END{print s;}' &amp;gt; exact_counts.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can do fractional (&quot;fuzzy&quot;) overlaps, where (for example) at least 50% of an hg38 region has to overlap an hg37 region:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --count --fraction-map 0.5 hg37.bed hg38.bed | awk '{s+=$1}END{print s;}' &amp;gt; fuzzy_0p5_counts.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Getting a percentage out of a &quot;fuzzy&quot; result can be complicated by overlaps between hg38 regions that overlap a reference hg37 region. A more complicated procedure can be implemented with the &lt;code&gt;--echo-map&lt;/code&gt; option and some command-line work to implement logic to deal with that case more stringently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get a straight-up count of no overlaps, build the inverse — or mask — of your hg38 regions, using &lt;a href=&quot;https://github.com/ENCODE-DCC/kentUtils&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kent tools&lt;/a&gt; &lt;code&gt;fetchChromSizes&lt;/code&gt; and BEDOPS &lt;code&gt;bedops --difference&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ fetchChromSizes hg38 | awk '{print $1&quot;\t0\t&quot;$2}' | sort-bed - &amp;gt; hg38.bounds.bed&#xA;$ bedops --difference hg38.bounds.bed hg38.bed &amp;gt; hg38.masked.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The regions in &lt;code&gt;hg38.masked.bed&lt;/code&gt; make up all of the genomic space in hg38, except for your lift-over regions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apply &lt;code&gt;--count --fraction-ref 1.0&lt;/code&gt; to count the number of reference (hg37) elements which completely overlap these masked hg38 regions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --count --fraction-ref 1.0 hg37.bed hg38.masked.bed | awk '{s+=$1}END{print s;}' &amp;gt; no_counts.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://bedops.readthedocs.io/en/latest/content/reference/statistics/bedmap.html#overlap-criteria&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation for &lt;code&gt;bedmap&lt;/code&gt;&lt;/a&gt; describes these overlap criteria and operations in more detail.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2017-07-17T19:37:29.100" LastActivityDate="2017-07-17T19:37:29.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2086" PostTypeId="2" ParentId="623" CreationDate="2017-07-17T23:46:37.833" Score="3" Body="&lt;p&gt;Pathway-tools has a unique identifier (called a Frame-ID) for each metabolite in its database.  You can get the list of all compounds with this api call:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;all_cpds = meta.get_class_all_instances('|Compounds|')&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for each compound, you can also get its common name, or all its synonyms:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&#xA;cpd_names = {}&#xA;for cpd_frame_id in all_cpds:&#xA;      cpd_names[cpd_frame_id] = meta.get_name_string(cpd_frame_id)&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="1131" LastActivityDate="2017-07-17T23:46:37.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2087" PostTypeId="2" ParentId="972" CreationDate="2017-07-18T13:01:54.007" Score="3" Body="&lt;p&gt;An alternative way is using the Hmmer website, and do a hmmsearch with the Pfam accession, forex. (PF02910). Navigate to the domain tab and find the domain architecture that you are interested in and press view scores. That way, only the sequences you are interested in will be downloadable under the tab downloads. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Step-by-Step guide:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1- Go to &lt;a href=&quot;http://www.ebi.ac.uk/Tools/hmmer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ebi.ac.uk/Tools/hmmer/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2- Search with hmmsearch &lt;a href=&quot;http://www.ebi.ac.uk/Tools/hmmer/search/hmmsearch&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ebi.ac.uk/Tools/hmmer/search/hmmsearch&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3- Use Accession search and type in PF02910&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4- Navigate to Domain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5- To the right side of &#xA;9742 sequences with domain architecture: FAD_binding_2, Succ_DH_flav_C &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Press View scores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;6- It should show (Your results have been filtered) and Query matches (9742)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;7- Navigate to Download and download the sequences in the format of interest. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;8- Repeat for domain architectures of interest. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope you find this helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kind Regards,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sara&lt;/p&gt;&#xA;" OwnerUserId="1049" LastActivityDate="2017-07-18T13:01:54.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2088" PostTypeId="1" CreationDate="2017-07-18T13:26:44.693" Score="1" ViewCount="462" Body="&lt;p&gt;Suppose I have single-end RNA-seq data for which the reads in the fastq file are reversed with respect to the original extracted RNAs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose I have the following workflow:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Map the reads on the reference genome, sort and index the resulting sam output.&lt;/li&gt;&#xA;&lt;li&gt;Use the bam file to make a bigwig file showing reads mapping on the plus and minus strands of the chromosomes.&lt;/li&gt;&#xA;&lt;li&gt;Use the bam file and an annotation file to quantify the gene expression.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Steps 2 and 3 depend on step 1, but are not dependant on one another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sure that strandedness needs to be taken into account no later than at step 3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The questions are the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it common to have mapping tools with an option to &quot;correct&quot; for strandedness?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Assuming the bam file has been built ignoring library strandedness information, is it good practice to correct for strandedness when building bigwig files?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Regarding this second point, I'm split between two attitudes:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;There should be no correction, the bigwig files are there to represent the &quot;raw&quot; mapping results.&lt;/li&gt;&#xA;&lt;li&gt;The bigwig files are there to give an idea of the origin and abundance of the originally extracted RNAs, so a strandedness correction should be used if necessary.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Which one is the most common attitude?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the second one is preferred, then wouldn't it be &quot;safer&quot; / &quot;cleaner&quot; to handle strandedness as early as possible? Possibly at the mapping, or even using a pre-processing step?&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="73" LastEditDate="2017-12-10T02:25:38.367" LastActivityDate="2017-12-10T02:51:36.053" Title="At what processing step should library strandedness type be taken into account?" Tags="|read-mapping|strandedness|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2089" PostTypeId="2" ParentId="2088" CreationDate="2017-07-18T15:07:12.493" Score="2" Body="&lt;p&gt;I usually take into account library strandness during the mapping to the reference step (but I never worked with bigwig files myself). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming RNA-seq data from Illumina, you can use &lt;a href=&quot;https://ccb.jhu.edu/software/hisat2/manual.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hisat2&lt;/a&gt; for alignment to the reference genome, and you have the option &lt;code&gt;--rna-strandness&lt;/code&gt; to specify the strand-specificity information (default unstranded). This is particularly important when you wish to have XS strand attribute in your bam file for easier compatibility with downstream software (especially important in my experience if you are planning to use some utility from &lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cufflinks&lt;/a&gt; for your step 3).&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2017-07-18T15:07:12.493" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2090" PostTypeId="1" AcceptedAnswerId="2094" CreationDate="2017-07-18T15:34:49.860" Score="6" ViewCount="830" Body="&lt;p&gt;It seems that in snakemake the script specified after the &lt;code&gt;--jobscript&lt;/code&gt; cannot be used properly if a &lt;code&gt;workdir:&lt;/code&gt; is specified in the snakefile. &#xA;The path of the script specified becomes relative to the workdir defined in the snakefile instead of being relative to the current working directory. I do not know if this is a feature or a bug !!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless, it becomes quite painful to workaround this problem. The only way I found was to parse the command line in the snakefile, for example:&#xA;&lt;code&gt;&#xA;snakemake -c &quot;qsub&quot; -j 30 --js ./sge.sh --latency-wait 30 -rp&#xA;&lt;/code&gt;&#xA;in order to copy &lt;code&gt;./sge.sh&lt;/code&gt; in the output directory defined by the keyword &lt;code&gt;workdir&lt;/code&gt; in the snakefile.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a side effect, if you specify option for qsub on the command line, for example:&#xA;&lt;code&gt;&#xA;snakemake -c &quot;qsub -e ./logs/ -o ./logs/&quot; -j 30 --js ./sge.sh --latency-wait 30 -rp&#xA;&lt;/code&gt;&#xA;the &lt;code&gt;logs&lt;/code&gt; folder must be created under the workdir directory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there something I don't understand with the &lt;code&gt;--jobscript&lt;/code&gt; option? Am I not using it the right way? Am I going against the best practice?&lt;/p&gt;&#xA;" OwnerUserId="373" LastActivityDate="2017-11-10T13:40:24.453" Title="combining the use of workdir and option --jobscript in snakemake" Tags="|snakemake|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2091" PostTypeId="2" ParentId="2088" CreationDate="2017-07-18T16:14:56.310" Score="1" Body="&lt;p&gt;Firstly, right at the start if the experiment it's important that your RNA samples are processed with a strand-specific protocol (e.g. Illumina's TruSeq Stranded) in order to produce stranded libraries for sequencing. If the samples haven't been treated as such, then there's nothing you can do to correct it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, I'm not sure stranded protocols are compatible with single-end reads. The aligners I'm aware of use the direction of aligned read pairs to assess the strand from which the RNA fragment has come from. How that works with single-end reads, I don't know?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, to answer your specific question, essentially step 1 is where you need to start with strand-specific options and do so at all subsequent steps as appropriate.&lt;/p&gt;&#xA;" OwnerUserId="1092" LastActivityDate="2017-07-18T16:14:56.310" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2092" PostTypeId="1" CreationDate="2017-07-18T17:21:35.507" Score="6" ViewCount="1007" Body="&lt;p&gt;I have found a blog post with a script that I would like to use for my current research project: &lt;a href=&quot;https://bcbio.wordpress.com/2009/02/07/automated-protein-conservation-display-from-blast-alignments/&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The script is incredibly fast and produces a smooth conservation plot. In the blog post, the author mentions that it would be totally possible to change the remote blast function to a local database. Instead of making a local database, I have opted to try to change the function to still use a remote blast, but with different parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found the following resource for learning about the NCBIWWW module: &lt;a href=&quot;http://biopython.org/DIST/docs/api/Bio.Blast.NCBIWWW-module.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this, I rewrote the line in the script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blast_handle = NCBIWWW.qblast(blast_method, &quot;nr&quot;, search_gi)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blast_handle = NCBIWWW.qblast(program=&quot;blastp&quot;, &#xA;                                      database=&quot;refseq_protein&quot;, &#xA;                                      sequence=search_gi, &#xA;                                      entrez_query=&quot; txid2 [ORGN] OR txid4751 [ORGN] &quot;, &#xA;                                      expect=20000.0, &#xA;                                      alignments=10000, &#xA;                                      descriptions=10000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Strangely, I get the exact same graph as an output even with the different parameters. Am I doing something about this search incorrectly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Update:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I got my BLAST to match the webpage. my current parameters are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blast_handle = NCBIWWW.qblast(program=&quot;blastp&quot;, database=&quot;refseq_protein&quot;, sequence=&quot;P02686.3&quot;, word_size=6, expect=20000.0, hitlist_size=10000, gapcosts=&quot;11 1&quot;, matrix_name=&quot;BLOSUM62&quot;, filter=&quot;F&quot;, genetic_code=1, threshold=21, composition_based_statistics=2, alignments=10000, descriptions=10000, entrez_query=&quot; txid2 [ORGN] OR txid4751 [ORGN] &quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With this new blast_handle, the script no longer works. I get the following error codes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;line 191, in &#xA;main(sys.argv[1])&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;line 27, in main&#xA;blast_rec = ncbi_manager.remote_blast(protein_gi, &quot;blastp&quot;)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;line 184, in remote_blast&#xA;return rec_it.next()&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;line 575, in parse&#xA;raise ValueError(&quot;Your XML file was empty&quot;)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;ValueError: Your XML file was empty&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="727" LastEditorUserId="3848" LastEditDate="2020-04-15T17:58:38.763" LastActivityDate="2020-04-15T17:58:38.763" Title="changing blast parameters using NCBIWWW module" Tags="|python|blast|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2093" PostTypeId="2" ParentId="2088" CreationDate="2017-07-18T17:59:44.230" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;If your aligner has a strandedness option then go ahead and use it. The general idea here being that you'll preferentially align correctly to genes.&lt;/li&gt;&#xA;&lt;li&gt;Whether to bother with strandedness here depends on your goal. Are you interested in anti-sense transcription? Do you need to use the bigWig track to accurately quantify translational pausing or some other strand-specific phenomenon? If yes, then go ahead and make different bigWig files for each strand. If you just need the bigWig file for convenience in IGV or creating heatmaps (e.g., with deepTools) then you'll generally be fine without accounting for strand. For well-annotated species, the rate of anti-sense transcription is generally low and the rate of genes overlapping isn't absurdly high. Consequently, you can usually get away with using an unstranded bigWig.&lt;/li&gt;&#xA;&lt;li&gt;If you're using featureCounts you'll certainly want to use the appropriate strandedness setting.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It's relatively common for RNAseq aligners/mappers to offer strandedness options. Certainly tophat2 and hisat(2) do, but also things like salmon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, for bigWig files the question becomes whether you really need two files to get the information you need. You're not going to delete the BAM files, so you can always make them if you need them.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-07-18T17:59:44.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2094" PostTypeId="2" ParentId="2090" CreationDate="2017-07-18T18:17:26.237" Score="2" Body="&lt;p&gt;To quote from the &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/snakefiles/configuration.html#configure-working-directory&quot; rel=&quot;nofollow noreferrer&quot;&gt;snakemake documentation&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;All paths in the snakefile are interpreted relative to the directory snakemake is executed in. This behaviour can be overridden by specifying a workdir in the snakefile:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;workdir: &amp;quot;path/to/workdir&amp;quot;&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Usually, it is preferred to only set the working directory via the command line, because above directive limits the portability of Snakemake workflows.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It's not explicitly stated, but it's sort of implied that all relative paths then become relative to the working directory. I would expect that specifying an absolute path would get around this.&lt;/p&gt;&#xA;&lt;p&gt;As an aside, in my mind setting the working directory is usually only needed on clusters without a shared file system (presumably shared between the worker nodes, but not with the head node), since there you can't &lt;code&gt;cd $WORKDIR&lt;/code&gt; before running snakemake. This is then normally done with your scheduler, in such cases.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-07-18T18:44:51.447" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2095" PostTypeId="1" AcceptedAnswerId="2099" CreationDate="2017-07-18T22:32:49.527" Score="2" ViewCount="256" Body="&lt;p&gt;I used prokka to create a database for some fasta files but I noticed a strange difference between the prokka and fasta files. Normally the seqid for the gff3 output for prokka has seqid's that match the sequence id's of the fasta files. The files I just produced with prokka have a different seqid than my fasta files. My fasta files look like this &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;NODE_108_length_645_cov_0.679537_ID_215&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However the prokka output gff3 file for the same fasta file uses a seqid like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gnl|X|JHIOGLGG_28&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What would cause this difference? And do the last digits still correlate to the same seqid's even though the string before it has changed?&lt;/p&gt;&#xA;" OwnerUserId="842" LastEditorUserId="298" LastEditDate="2017-07-18T23:24:55.483" LastActivityDate="2017-07-19T08:31:11.713" Title="Does Prokka or gff3 change the sequence / subject ID of fasta files?" Tags="|fasta|gff3|sequence-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2096" PostTypeId="2" ParentId="982" CreationDate="2017-07-18T22:47:05.673" Score="3" Body="&lt;p&gt;Another quick Google search reveals the &lt;a href=&quot;https://cran.r-project.org/web/packages/Peptides/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Peptides R package&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Peptides)&#xA;s &amp;lt;- &quot;RKTTLVPNTQTASPR&quot;&#xA;charge(s)&#xA;&#xA;[1] 2.997683&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Optional arguments to &lt;code&gt;charge()&lt;/code&gt; are &lt;code&gt;pH&lt;/code&gt; (default = 7) and &lt;code&gt;pKscale&lt;/code&gt; (choice of nine, default = &quot;Lehninger&quot;). See &lt;code&gt;?charge&lt;/code&gt; for details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another (non-R) option is the &lt;a href=&quot;http://emboss.sourceforge.net/apps/release/6.6/emboss/apps/iep.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;EMBOSS suite program iep&lt;/a&gt;. I highly recommend installing EMBOSS, it provides a suite of useful command-line tools with output that can be easily read and processed by other sofware, including R.&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2017-07-18T22:47:05.673" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2097" PostTypeId="1" CreationDate="2017-07-19T03:09:25.747" Score="4" ViewCount="85" Body="&lt;p&gt;I want to survival analysis using the subset of TCGA LUAD dataset, which identifier are located &lt;a href=&quot;https://xenabrowser.net/datapages/?host=https%3A%2F%2Ftcga.xenahubs.net&amp;amp;dataset=TCGA.LUAD.sampleMap%2FLUAD_clinicalMatrix&amp;amp;label=Phenotypes&amp;amp;allIdentifiers=true&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(LUAD_clinicalMatrix[,c(&quot;X_EVENT&quot;,&quot;X_OS_IND&quot;)])&#xA;  X_EVENT X_OS_IND&#xA;1      NA       NA&#xA;2       0        0&#xA;3       1        1&#xA;4       0        0&#xA;5       0        0&#xA;6       0        0&#xA;&#xA;&amp;gt; all(LUAD_clinicalMatrix$X_EVENT==LUAD_clinicalMatrix$X_OS_UNIT)&#xA;[1] FALSE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to know which identifier means the sample is alive or dead, and &quot;1&quot; means alive or dead?&lt;/p&gt;&#xA;" OwnerUserId="1133" LastEditorUserId="48" LastEditDate="2017-11-16T11:01:25.097" LastActivityDate="2017-11-16T11:01:25.097" Title="What does the &quot;X_OS_IND&quot; column mean?" Tags="|r|file-formats|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2098" PostTypeId="1" CreationDate="2017-07-19T04:55:16.090" Score="6" ViewCount="457" Body="&lt;p&gt;My goal is to work on GWAS method development as a research project. However, human genome data are usually confidential because of the identification problem, so it's very hard to get them. (application takes a long time and I may not get it at the end.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, maybe I can try to play with the data of other species first. So I wonder where I can find some GWAS data for Zebrafish and rice. (The reason I am asking for these two is that I find researchers tend to perform genome analysis to mice, Arabidopsis Thaliana, zebrafish, and rice, and I already found where to download the GWAS data for the first two.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can find several data sets (like &lt;a href=&quot;https://zfin.org/downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;ZFIN&lt;/a&gt;) that have the annotated genome information, but that's only one genome. Where can I download the raw data (like genotypes of 500 zebrafishes and corresponding phenotypes) to start my own analysis? &lt;/p&gt;&#xA;" OwnerUserId="1082" LastEditorUserId="1082" LastEditDate="2017-07-19T16:14:25.300" LastActivityDate="2017-09-17T20:17:47.040" Title="Where can I find a GWAS data set about genotypes and phenotypes for Zebrafish and rice?" Tags="|database|gwas|" AnswerCount="1" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2099" PostTypeId="2" ParentId="2095" CreationDate="2017-07-19T08:31:11.713" Score="3" Body="&lt;p&gt;Prokka is an annotation tool. It takes your contigs file with fasta headers &lt;code&gt;&amp;gt;NODE_108...&lt;/code&gt;and searchs for ORFs and their putative functions. Prokka produces submission ready output, the new fasta headers you see &lt;code&gt;&amp;gt;JHIOGLGG_28...&lt;/code&gt;are locus tags and gene description from the annotation and are not related to contig node data. Please see the &lt;a href=&quot;https://github.com/tseemann/prokka#output-files&quot; rel=&quot;nofollow noreferrer&quot;&gt;github information page&lt;/a&gt; for more info.&lt;/p&gt;&#xA;" OwnerUserId="982" LastActivityDate="2017-07-19T08:31:11.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2100" PostTypeId="2" ParentId="2097" CreationDate="2017-07-19T08:55:52.523" Score="5" Body="&lt;p&gt;According to &lt;a href=&quot;https://groups.google.com/forum/#!topic/ucsc-cancer-genomics-browser/YvKnWZSsw1Q&quot; rel=&quot;noreferrer&quot;&gt;this link&lt;/a&gt;, 1=death 0=censor and null=no data.&lt;/p&gt;&#xA;" OwnerUserId="931" LastActivityDate="2017-07-19T08:55:52.523" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2101" PostTypeId="1" AcceptedAnswerId="2108" CreationDate="2017-07-19T13:41:05.330" Score="3" ViewCount="2530" Body="&lt;p&gt;I have surprisingly low counts when running &lt;code&gt;featureCounts&lt;/code&gt; on some (single-end) RNA-seq data mapped on &lt;em&gt;C. elegans&lt;/em&gt; genome using &lt;code&gt;hisat2&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To more easily show the problem, I generated a small subset of the bam file and of the annotation file I'm using. Here is what I can see when loading these two files on IGV:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iijez.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iijez.png&quot; alt=&quot;View of the his-16 -&amp;gt; his-12 region in IGV&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The coverage for the &quot;his-11&quot; and &quot;his-15&quot; genes peak at around 3600. For &quot;his-12&quot; and &quot;his-16&quot;, it peaks at more than 1000&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the content of my annotation file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat his_11-16.gtf &#xA;II  ensembl transcript  13817759    13818143    .   -   .   gene_biotype &quot;protein_coding&quot;; gene_id &quot;WBGene00001890&quot;; gene_name &quot;his-16&quot;; gene_source &quot;ensembl&quot;; gene_version &quot;1&quot;; p_id &quot;P8354&quot;; transcript_biotype &quot;protein_coding&quot;; transcript_id &quot;ZK131.10&quot;; transcript_name &quot;ZK131.10&quot;; transcript_source &quot;ensembl&quot;; transcript_version &quot;1&quot;; tss_id &quot;TSS11539&quot;;&#xA;II  ensembl transcript  13818371    13818739    .   +   .   gene_biotype &quot;protein_coding&quot;; gene_id &quot;WBGene00001889&quot;; gene_name &quot;his-15&quot;; gene_source &quot;ensembl&quot;; gene_version &quot;1&quot;; p_id &quot;P8185&quot;; transcript_biotype &quot;protein_coding&quot;; transcript_id &quot;ZK131.9&quot;; transcript_name &quot;ZK131.9&quot;; transcript_source &quot;ensembl&quot;; transcript_version &quot;1&quot;; tss_id &quot;TSS52436&quot;;&#xA;II  ensembl transcript  13819626    13819937    .   -   .   gene_biotype &quot;protein_coding&quot;; gene_id &quot;WBGene00001888&quot;; gene_name &quot;his-14&quot;; gene_source &quot;ensembl&quot;; gene_version &quot;1&quot;; p_id &quot;P21890&quot;; transcript_biotype &quot;protein_coding&quot;; transcript_id &quot;ZK131.8&quot;; transcript_name &quot;ZK131.8&quot;; transcript_source &quot;ensembl&quot;; transcript_version &quot;1&quot;; tss_id &quot;TSS50308&quot;;&#xA;II  ensembl transcript  13820210    13820620    .   +   .   gene_biotype &quot;protein_coding&quot;; gene_id &quot;WBGene00001887&quot;; gene_name &quot;his-13&quot;; gene_source &quot;ensembl&quot;; gene_version &quot;1&quot;; p_id &quot;P2149&quot;; transcript_biotype &quot;protein_coding&quot;; transcript_id &quot;ZK131.7&quot;; transcript_name &quot;ZK131.7&quot;; transcript_source &quot;ensembl&quot;; transcript_version &quot;1&quot;; tss_id &quot;TSS35106&quot;;&#xA;II  ensembl transcript  13821198    13821581    .   -   .   gene_biotype &quot;protein_coding&quot;; gene_id &quot;WBGene00001886&quot;; gene_name &quot;his-12&quot;; gene_source &quot;ensembl&quot;; gene_version &quot;1&quot;; p_id &quot;P26082&quot;; transcript_biotype &quot;protein_coding&quot;; transcript_id &quot;ZK131.6&quot;; transcript_name &quot;ZK131.6&quot;; transcript_source &quot;ensembl&quot;; transcript_version &quot;1&quot;; tss_id &quot;TSS23693&quot;;&#xA;II  ensembl transcript  13821778    13822314    .   +   .   gene_biotype &quot;protein_coding&quot;; gene_id &quot;WBGene00001885&quot;; gene_name &quot;his-11&quot;; gene_source &quot;ensembl&quot;; gene_version &quot;1&quot;; p_id &quot;P10535&quot;; transcript_biotype &quot;protein_coding&quot;; transcript_id &quot;ZK131.5&quot;; transcript_name &quot;ZK131.5&quot;; transcript_source &quot;ensembl&quot;; transcript_version &quot;1&quot;; tss_id &quot;TSS52792&quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;featureCounts&lt;/code&gt; run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ featureCounts -a his_11-16.gtf -o his_11-16_counts.txt -t transcript -g gene_name -O his_11-16_sorted.bam&#xA;&#xA;        ==========     _____ _    _ ____  _____  ______          _____  &#xA;        =====         / ____| |  | |  _ \|  __ \|  ____|   /\   |  __ \ &#xA;          =====      | (___ | |  | | |_) | |__) | |__     /  \  | |  | |&#xA;            ====      \___ \| |  | |  _ &amp;lt;|  _  /|  __|   / /\ \ | |  | |&#xA;              ====    ____) | |__| | |_) | | \ \| |____ / ____ \| |__| |&#xA;        ==========   |_____/ \____/|____/|_|  \_\______/_/    \_\_____/&#xA;      v1.5.2&#xA;&#xA;//========================== featureCounts setting ===========================\\&#xA;||                                                                            ||&#xA;||             Input files : 1 BAM file                                       ||&#xA;||                           S his_11-16_sorted.bam                           ||&#xA;||                                                                            ||&#xA;||             Output file : his_11-16_counts.txt                             ||&#xA;||                 Summary : his_11-16_counts.txt.summary                     ||&#xA;||              Annotation : his_11-16.gtf (GTF)                              ||&#xA;||      Dir for temp files : ./                                               ||&#xA;||                                                                            ||&#xA;||                 Threads : 1                                                ||&#xA;||                   Level : meta-feature level                               ||&#xA;||              Paired-end : no                                               ||&#xA;||         Strand specific : no                                               ||&#xA;||      Multimapping reads : not counted                                      ||&#xA;|| Multi-overlapping reads : counted                                          ||&#xA;||   Min overlapping bases : 1                                                ||&#xA;||                                                                            ||&#xA;\\===================== http://subread.sourceforge.net/ ======================//&#xA;&#xA;//================================= Running ==================================\\&#xA;||                                                                            ||&#xA;|| Load annotation file his_11-16.gtf ...                                     ||&#xA;||    Features : 6                                                            ||&#xA;||    Meta-features : 6                                                       ||&#xA;||    Chromosomes/contigs : 1                                                 ||&#xA;||                                                                            ||&#xA;|| Process BAM file his_11-16_sorted.bam...                                   ||&#xA;||    Single-end reads are included.                                          ||&#xA;||    Assign reads to features...                                             ||&#xA;||    Total reads : 35037                                                     ||&#xA;||    Successfully assigned reads : 1849 (5.3%)                               ||&#xA;||    Running time : 0.00 minutes                                             ||&#xA;||                                                                            ||&#xA;||                         Read assignment finished.                          ||&#xA;||                                                                            ||&#xA;|| Summary of counting results can be found in file &quot;his_11-16_counts.txt&quot;    ||&#xA;||                                                                            ||&#xA;\\===================== http://subread.sourceforge.net/ ======================//&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the resulting counts file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat his_11-16_counts.txt&#xA;# Program:featureCounts v1.5.2; Command:&quot;featureCounts&quot; &quot;-a&quot; &quot;his_11-16.gtf&quot; &quot;-o&quot; &quot;his_11-16_counts.txt&quot; &quot;-t&quot; &quot;transcript&quot; &quot;-g&quot; &quot;gene_name&quot; &quot;-O&quot; &quot;his_11-16_sorted.bam&quot; &#xA;Geneid  Chr Start   End Strand  Length  his_11-16_sorted.bam&#xA;his-16  II  13817759    13818143    -   385 869&#xA;his-15  II  13818371    13818739    +   369 3&#xA;his-14  II  13819626    13819937    -   312 953&#xA;his-13  II  13820210    13820620    +   411 23&#xA;his-12  II  13821198    13821581    -   384 423&#xA;his-11  II  13821778    13822314    +   537 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why such a low assignment rate, why so little counts?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see that &lt;code&gt;hisat2&lt;/code&gt; decided to split an important proportion of the reads between orthologs, like &quot;his-16&quot; and &quot;his-12&quot;, but my understanding is that the default values for the &lt;code&gt;minOverlap&lt;/code&gt; and &lt;code&gt;fracOverlap&lt;/code&gt; parameters should ensure that such split reads will be counted:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  --minOverlap &amp;lt;int&amp;gt;  Minimum number of overlapping bases in a read that is&#xA;                      required for read assignment. 1 by default. Number of&#xA;                      overlapping bases is counted from both reads if paired&#xA;                      end. If a negative value is provided, then a gap of up&#xA;                      to specified size will be allowed between read and the&#xA;                      feature that the read is assigned to.&#xA;&#xA;  --fracOverlap &amp;lt;float&amp;gt; Minimum fraction of overlapping bases in a read that is&#xA;                      required for read assignment. Value should be within range&#xA;                      [0,1]. 0 by default. Number of overlapping bases is&#xA;                      counted from both reads if paired end. Both this option&#xA;                      and '--minOverlap' option need to be satisfied for read&#xA;                      assignment.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What did I get wrong?&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2017-07-19T21:16:01.000" Title="Counts obtained by featureCounts seem much less than observed coverage" Tags="|read-mapping|coverage|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2102" PostTypeId="2" ParentId="2081" CreationDate="2017-07-19T15:02:52.070" Score="1" Body="&lt;p&gt;I would suggest you to run some simulation. You can use &lt;a href=&quot;https://github.com/lh3/wgsim&quot; rel=&quot;nofollow noreferrer&quot;&gt;wgsim&lt;/a&gt; to simulate reads that are as much similar to what you want an answer for.&#xA;I don't think there are faster methods.&lt;/p&gt;&#xA;" OwnerUserId="1140" LastActivityDate="2017-07-19T15:02:52.070" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2103" PostTypeId="2" ParentId="2081" CreationDate="2017-07-19T19:20:48.830" Score="6" Body="&lt;p&gt;Mapping quality is determined by the repetitiveness of the genome, the sequencing error rate, insert size, the capability of the mapper and the nasty heuristics behind the mapper. MAPQ=0 to one mapper is not necessarily MAPQ=0 to another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I get what you mean. You want to know the uniqueness/repetitiveness. It is still hard if you want to get a useful answer. For 150bp reads, each reference position is covered by 150 reads. What if 50 of them have no other exact hits elsewhere, but the rest 100 have? Is this position a repeat or not? In addition, what if the 50 are unique only because one mismatch? If there is a variant at that mismatch, the 50 would become repeats or mapped elsewhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My preference is to say a position is &quot;unique&quot; under k-long reads if over k/2 reads overlapping the position have no other perfect or 1-mismatch/1-gap hits elsewhere in the genome. Under this definition, 79.3% of human genome are unique for 35bp reads. 92.4% for 75bp reads. I don't have the number for 150bp reads. I guess it will be around 95%. Empirically, 94-95% of human genome is callable with 100bp paired-end reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to other measurements, the most common one is the fraction of reads that has an exact hit elsewhere. You can use Fabio's method. It gives a good enough estimate once you simulate over 1 million reads. This fraction is around 86% for 35bp reads and 95% for 75bp reads, as I remember. The problem with this approach is this fraction is not very informative to variant calling due to the issues I talked about. Another way is to use RepeatMasker. It is worse. RepeatMasker masks 50% of human genome, but excludes segmental duplications where short reads can't be confidently mapped.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-07-19T19:20:48.830" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2104" PostTypeId="2" ParentId="2098" CreationDate="2017-07-19T19:39:13.950" Score="1" Body="&lt;p&gt;I think it will be more productive for you if you choose one of the public human datasets and practise your GWAS methods on that. A couple of examples examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.internationalgenome.org/data&quot; rel=&quot;nofollow noreferrer&quot;&gt;1000 genomes&lt;/a&gt; -- use &quot;population&quot; as your trait of association. You'll find that you get association across the whole genome, so the analysis method would change from &quot;what bits are associated&quot; to &quot;what are the bits that are most consistently associated&quot;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;NCI &lt;a href=&quot;https://portal.gdc.cancer.gov/repository&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genomic Data Commons&lt;/a&gt; -- open access (and controlled access) data associated with cancer&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-07-19T19:39:13.950" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2106" PostTypeId="1" AcceptedAnswerId="2107" CreationDate="2017-07-19T20:49:38.947" Score="3" ViewCount="820" Body="&lt;p&gt;Could some one please help me with understanding how to generate the ribosomal interval list that is required to when using picard metrics this step&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar picard.jar CollectRnaSeqMetrics \&#xA;      I=input.bam \&#xA;      O=output.RNA_Metrics \&#xA;      REF_FLAT=ref_flat.txt \&#xA;      STRAND=SECOND_READ_TRANSCRIPTION_STRAND \&#xA;      RIBOSOMAL_INTERVALS=ribosomal.interval_list&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have a couple of bam files that were aligned using mm9 ensembl gtf as annotation, i did a lot of search and found this link Ribosomal Intervals For Collectrnaseqmetrics but this link or any other source does not explain how to produce or find ribo interval list from ensembl annotation&lt;/p&gt;&#xA;" OwnerUserId="926" LastActivityDate="2017-07-19T21:20:24.753" Title="ribo interval list for picard ensembl mm9 gtf" Tags="|rna-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2107" PostTypeId="2" ParentId="2106" CreationDate="2017-07-19T21:08:28.883" Score="3" Body="&lt;p&gt;At least with Ensembl annotations, you can &lt;code&gt;grep rRNA foo.gtf &amp;gt; rRNA.gtf&lt;/code&gt; to get the annotated rRNAs. Note however, that the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene/100861531&quot; rel=&quot;nofollow noreferrer&quot;&gt;45S cassette&lt;/a&gt; isn't in the mouse reference genome, so you'll never have alignments to 18S, 5.8S, or 28S rRNA. I always get the feeling that the Collect*Metrics programs are made with human sequences in mind, since the cassette is included in the human reference genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, while you can use the annotation and reference sequence to make picard happy, make sure to append the Rn45S sequence to your genome if you want to actually measure rRNA levels. In my experience this is only useful for rRNA-depleted samples, since the rates can vary wildly there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: To make a file compatible with picard's &quot;interval list&quot;, one can do the following with the GTF file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep rRNA foo.gtf | awk 'BEGIN{OFS=&quot;\t&quot;}{print $1, $4-1, $5}' &amp;gt; foo.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;According to the &lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/1319/collected-faqs-about-interval-lists&quot; rel=&quot;nofollow noreferrer&quot;&gt;picard documentation&lt;/a&gt; a BED file can be used as input and the above &lt;code&gt;awk&lt;/code&gt; code extracts the appropriate columns and switches to 0-based coordinates used in the BED format.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-07-19T21:20:24.753" LastActivityDate="2017-07-19T21:20:24.753" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2108" PostTypeId="2" ParentId="2101" CreationDate="2017-07-19T21:16:01.000" Score="4" Body="&lt;p&gt;Given the high level of multimapping in this region, you'll need to use the &lt;code&gt;-M --primary&lt;/code&gt; options if you want to keep many of the alignments. I would be very hesitant to use these numbers as input for DESeq2 or similar programs, since it's fairly questionable whether one should fully trust the &quot;randomness&quot; of the aligner's assignments. I'm more comfortable using something like &lt;a href=&quot;http://salmon.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;salmon&lt;/a&gt; for such cases.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-07-19T21:16:01.000" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2109" PostTypeId="1" CreationDate="2017-07-20T12:38:01.380" Score="2" ViewCount="30" Body="&lt;p&gt;NCI-60 has additional &lt;a href=&quot;https://dtp.cancer.gov/discovery_development/nci-60/cell_list.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;cell lines&lt;/a&gt;, but when I get results from &lt;a href=&quot;https://discover.nci.nih.gov/cellminer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CellMiner&lt;/a&gt; they are not included in there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Namely, the genes such as:  &lt;code&gt;DLD-1&lt;/code&gt;(Colon), &lt;code&gt;KM20L2&lt;/code&gt;    (Colon), &lt;code&gt;M19-MEL&lt;/code&gt; (Melanoma), don't show in the results of CellMiner. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there away to include these additional lines in the correlation calculations?&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2017-07-20T13:08:03.397" LastActivityDate="2017-07-20T13:08:03.397" Title="Working with the additional cell lines in NCI-60" Tags="|cancer|cell-line|" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2110" PostTypeId="2" ParentId="2081" CreationDate="2017-07-20T14:51:07.630" Score="5" Body="&lt;p&gt;As has been said before, mappability to the 'human genome' depends on a number of factors, among these the reference version and type of reads, for which you are interested in GRCh38 and 2x150bp reads. Although I am not aware of numbers accounting for these particular reference and type of Illumina reads, the 1000 genomes project has provided the community with a similar and close estimation that you might be interested in considering regarding your inquiry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar to your question, the 1K project estimated 'the proportion of the human genome that is less accessible to short reads'. In these estimates the human genome is GRCh37 and the types of reads in question are mostly 2x Illumina with a mixture of lengths with the longest being up to 250bp. In these estimates each base in the human genome is considered (and marked) 'less accesible' according to these criteria:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;L - depth of coverage is much lower than average&lt;/p&gt;&#xA;&#xA;&lt;p&gt;H - depth of coverage is much higher than average&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Z - too many reads with zero mapping quality overlap this position&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q - the average mapping quality at the position is too low&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each of these criteria has &quot;standard&quot; and &quot;strict&quot; thresholds for a base to be considered - or not - in each category. You can read more in the link below: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/accessible_genome_masks/README.accessible_genome_mask.20140520&quot; rel=&quot;noreferrer&quot;&gt;ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/accessible_genome_masks/README.accessible_genome_mask.20140520&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the strict thresholds, the human genome has about 16.8% of &quot;Z&quot; and 3.1% of &quot;Q&quot; bases, respectively. Considering the &quot;Z&quot; and &quot;Q&quot; criteria as a proxy for ~ mapq=0, about 19.9% of the human genome can not be uniquely mapped. &lt;/p&gt;&#xA;" OwnerUserId="475" LastActivityDate="2017-07-20T14:51:07.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2111" PostTypeId="1" CreationDate="2017-07-20T15:53:54.547" Score="3" ViewCount="454" Body="&lt;p&gt;I'm using the &lt;em&gt;Peptides&lt;/em&gt; package in R to calculate the mass of peptides with the mw() method. The thing is that I want to calculate them for different pH values, and thus I wrote this little function to take the pH value into account according this image:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iA96t.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iA96t.jpg&quot; alt=&quot;Enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code is this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;phmw = function(pept_seq , unimod_location, pH ){&#xA;&#xA;  monoisotopic_hydrogen_mass = 1.00794&#xA;&#xA;  if(pH &amp;lt; 7){&#xA;&#xA;    # mol_weight = mol_weight_of_peptide + mol_weight_of_Hydrogen&#xA;    mol_weight = mw(pept_seq, monoisotopic = T) + monoisotopic_hydrogen_mass&#xA;&#xA;  }else if(pH &amp;gt; 7){&#xA;&#xA;    # mol_weight = mol_weight_of_peptide - mol_weight_of_Hydrogen&#xA;    mol_weight = mw(pept_seq, monoisotopic = T) - monoisotopic_hydrogen_mass&#xA;&#xA;  }else if(pH == 7){&#xA;&#xA;    # mol_weight = mol_weight_of_peptide &#xA;    mol_weight = mw(pept_seq, monoisotopic = T) &#xA;  }&#xA;&#xA;  return(mol_weight)&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Do you think that this approach is correct or not?&lt;/p&gt;&#xA;" OwnerUserId="704" LastEditorUserId="1775" LastEditDate="2018-02-22T08:39:43.730" LastActivityDate="2018-04-03T16:24:46.723" Title="Influence of pH on protein peptide mass" Tags="|r|proteins|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2112" PostTypeId="2" ParentId="2078" CreationDate="2017-07-20T23:10:02.150" Score="3" Body="&lt;p&gt;For MAC OSX, the executable found in &lt;code&gt;arlsumstat_macosx&lt;/code&gt; (&lt;code&gt;arlsumstatmac_64bit&lt;/code&gt;) seem to work appropriately. The directory for &lt;code&gt;arlsumstat_macosx&lt;/code&gt; does not contain any examples but you can use example files from &lt;code&gt;arlecore_macosx/Example files_linux&lt;/code&gt; using this syntax.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The complication is that &lt;code&gt;arlsumstatmac_64bit&lt;/code&gt; will look in the current directory for the file &lt;code&gt;arl_run.ars&lt;/code&gt;. There is a file with the same name in &lt;code&gt;arlecore_macosx&lt;/code&gt; but it is not compatible. So you have to be in the &lt;code&gt;arlsumstat_macosx&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, &lt;code&gt;cd&lt;/code&gt; to &lt;code&gt;arlsumstat_macosx&lt;/code&gt; and do&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./arlsumstatmac_64bit path/to/arlecore_macosx/Example\ files_linux/Freqncy/cohen.ars path/to/arlecore_macosx/Example\ files_linux/Freqncy/cohen.arp&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;, where the &lt;code&gt;.arp&lt;/code&gt; file contains the data and the &lt;code&gt;.ars&lt;/code&gt; file contains the settings (the description of what you want Arlequin to do).&lt;/p&gt;&#xA;" OwnerUserId="888" LastActivityDate="2017-07-20T23:10:02.150" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2113" PostTypeId="1" CreationDate="2017-07-21T12:33:07.187" Score="9" ViewCount="177" Body="&lt;p&gt;A different sort of problem: even a small 'omics lab generates a lot of data, raw, intermediate and processed. What (software) solutions exist for managing this data, such that &quot;old&quot; data can be retrieved and checked or re-analysed, even after people have left the lab? Important points would be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ease of installation&lt;/li&gt;&#xA;&lt;li&gt;ease of putting data in, in an appropriately tagged / labeled fashion (it's no good if the repository is just a centralised bad mess)&lt;/li&gt;&#xA;&lt;li&gt;useful search and exploration&lt;/li&gt;&#xA;&lt;li&gt;security (i.e. restricted to members of the lab)&lt;/li&gt;&#xA;&lt;li&gt;previews / summaries of the data&lt;/li&gt;&#xA;&lt;li&gt;can accommodate any dataset&lt;/li&gt;&#xA;&lt;li&gt;local, not SaaS&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It seems from my explorations of the topic that there's very little between the primitive (e.g. handrolled Access databases) and big industrial solutions. Things I have looked at include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Dataverse: very popular, installation seems complex and unclear if uploading is that easy&lt;/li&gt;&#xA;&lt;li&gt;DSpace: mostly for publications and documents&lt;/li&gt;&#xA;&lt;li&gt;CKAN&lt;/li&gt;&#xA;&lt;li&gt;OSF: used this for a while, integrates with a lot of services but uploading data seemed awkward&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="377" LastActivityDate="2017-07-25T22:01:40.587" Title="Solutions for managing data in a small bioinformatics / 'omics lab?" Tags="|database|data-management|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2114" PostTypeId="1" AcceptedAnswerId="2115" CreationDate="2017-07-21T13:03:59.900" Score="6" ViewCount="883" Body="&lt;p&gt;I have a binary matrix of gene presence or absence which looks like: [roary output]&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Gene    sample1 sample2 sample3 sample4&#xA;fliI        1       1       1        1&#xA;patB_1      1       1       1        1&#xA;pgpA        1       1       1        1&#xA;osmB        1       1       1        1&#xA;cspA        1       0       1        1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I convert to fasta so it looks like and technically aligned:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;sample1&#xA;11111&#xA;&amp;gt;sample2&#xA;11110&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is it possible to do this in R? &lt;/p&gt;&#xA;" OwnerUserId="982" LastActivityDate="2017-09-17T00:22:57.303" Title="How to convert a binary matrix of gene presence or absence into a fasta sequence" Tags="|r|fasta|format-conversion|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2115" PostTypeId="2" ParentId="2114" CreationDate="2017-07-21T13:20:17.730" Score="8" Body="&lt;p&gt;It's not a fasta file, but:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; m&#xA;       sample1 sample2 sample3 sample4&#xA;fliI         1       1       1       1&#xA;patB_1       1       1       1       1&#xA;pgpA         1       1       1       1&#xA;osmB         1       1       1       1&#xA;cspA         1       0       1       1&#xA;&amp;gt; # Collapse to labeled strings&#xA;&amp;gt; blah = apply(m, 2, function(x) paste(x, collapse=''))&#xA;&amp;gt; blah&#xA;sample1 sample2 sample3 sample4 &#xA;&quot;11111&quot; &quot;11110&quot; &quot;11111&quot; &quot;11111&quot; &#xA;&amp;gt; # Write that to a file with the appropriate format&#xA;&amp;gt; cat(paste(mapply(function(x, y) sprintf(&quot;&amp;gt;%s\n%s\n&quot;, x, y), names(blah), blah), collapse=&quot;&quot;), file=&quot;some file.fa&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Make sure to change &lt;code&gt;some file.fa&lt;/code&gt;. I have no idea why you would want to do all of this, but this will give you the output you want:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat blah.txt &#xA;&amp;gt;sample1&#xA;11111&#xA;&amp;gt;sample2&#xA;11110&#xA;&amp;gt;sample3&#xA;11111&#xA;&amp;gt;sample4&#xA;11111&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-07-21T13:20:17.730" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2116" PostTypeId="2" ParentId="2114" CreationDate="2017-07-21T18:05:08.483" Score="3" Body="&lt;p&gt;One could also use &lt;code&gt;cut&lt;/code&gt; and a pair of Python scripts:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;transpose.py&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python                                                                                                                                                                                                                                                                                                                       &#xA;&#xA;import sys&#xA;&#xA;for c in zip(*(l.split() for l in sys.stdin.readlines() if l.strip())):&#xA;    sys.stdout.write(&quot;%s\n&quot; % ('\t'.join(c)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;mock_fasta.py&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python                                                                                                                                                                                                                                                                                                                       &#xA;&#xA;import sys&#xA;&#xA;for e in (l.split() for l in sys.stdin.readlines() if l.strip()):&#xA;    sys.stdout.write(&quot;&amp;gt;%s\n%s\n&quot; % (e[0], ''.join(e[1:])))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cut -f2- in.mtx | transpose.py | mock_fasta.py &amp;gt; in.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which gives:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ more in.fa&#xA;&amp;gt;sample1&#xA;11111&#xA;&amp;gt;sample2&#xA;11110&#xA;&amp;gt;sample3&#xA;11111&#xA;&amp;gt;sample4&#xA;11111&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Why anyone would use R for text parsing is beyond me, but kids these days etc.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2017-09-17T00:22:57.303" LastActivityDate="2017-09-17T00:22:57.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2117" PostTypeId="1" AcceptedAnswerId="2394" CreationDate="2017-07-21T18:15:17.510" Score="7" ViewCount="231" Body="&lt;p&gt;I have identified a genomic region (CpG island) that is enriched for a characteristic of interest in the vast majority of my samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After viewing it in the UCSC genome browser, I have confirmed that it does not overlap any annotated gene promoters, gene bodies, or retrotransposons, but am convinced that it is functionally interacting with something long range.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a publicly available visualization tool where I can input my genomic region and get a list of topologically associated domains from 3C or other chromatin-chromatin interaction data?&lt;/p&gt;&#xA;" OwnerUserId="643" LastActivityDate="2017-11-19T20:42:24.850" Title="Visualization tools for 3C/Hi-C long-range interaction data?" Tags="|software-recommendation|visualization|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2118" PostTypeId="2" ParentId="2117" CreationDate="2017-07-21T18:47:06.840" Score="2" Body="&lt;p&gt;Depending on the genome you want, you may be able to use the &lt;a href=&quot;http://chorogenome.ie-freiburg.mpg.de/&quot; rel=&quot;nofollow noreferrer&quot;&gt;chorogeneome navigator&lt;/a&gt;, which is from some of my colleagues. This uses a number of public HiC datasets and allows you to view interactions.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-07-21T18:47:06.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2119" PostTypeId="1" AcceptedAnswerId="2120" CreationDate="2017-07-21T19:45:06.567" Score="7" ViewCount="674" Body="&lt;p&gt;I have a problem here with my rna seq data:&lt;/p&gt;&#xA;&lt;h1&gt;Sequencing details:&lt;/h1&gt;&#xA;&lt;p&gt;rRNA was removed, followed by cDNA preparation and generation of stranded libraries using the TruSeq Stranded Total RNA Sample Prep Kit. Sequencing performed on the HiSeq2500 platform (Illumina) to generate 2 × 125 bp paired-end reads&lt;/p&gt;&#xA;&lt;h1&gt;Alignment and preprocessing&lt;/h1&gt;&#xA;&lt;p&gt;reads were aligned using tophat2(std aligner in pipeline at core (&lt;code&gt;--library-type fr-firststrand&lt;/code&gt;), unfortunately the original unaligned files were purged and i only have access to this aligned file which i converted to fastq using following steps&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;samtools sort -n sample.bam -o sample_sorted.bam&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;bedtools bamtofastq -i sample_sorted.bam -fq sample_1.fq -fq2 sample_2.fq&lt;/code&gt; (get a lot of mate skipping errors here)&lt;/li&gt;&#xA;&lt;li&gt;check for these reads for adapters and trim them and again use hisat2 (with &lt;code&gt;--rna-strandness RF&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h1&gt;trimmed reads&lt;/h1&gt;&#xA;&lt;p&gt;Reads were trimmed by passing parameters &lt;code&gt;--adapters adapters.file --adapter-trim-end RIGHT --length-dist --threads 12 --adapter-min-overlap 7 --max-uncalled 250 --min-read-length 25 --&lt;/code&gt; to FLEXBAR version 2.4&lt;/p&gt;&#xA;&lt;h2&gt;adapters for all samples using multiqc&lt;/h2&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IsVW7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IsVW7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h2&gt;fastqc sequence length pre trimming&lt;/h2&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fPONe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fPONe.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h2&gt;fastqc sequence length post trimming&lt;/h2&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3nY5A.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3nY5A.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h2&gt;rmats error&lt;/h2&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Incorrect readLength. sample.bam has a read length of 114, while readLength param is 125&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h2&gt;picard error on aligned bam from hisat2)&lt;/h2&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;ProcessExecutor   &lt;a href=&quot;https://i.stack.imgur.com/fPONe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt; &amp;quot;Not creating insert size PDF as there are duplicated header names: All_Reads&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h2&gt;snapshot of picard insertsize metrics file&lt;/h2&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lgeHk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lgeHk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h1&gt;Questions&lt;/h1&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is my approach flawed at some step?&lt;/li&gt;&#xA;&lt;li&gt;why do i get different read lengths error for each sample when i process it through rmats when i specify len as 125?&lt;/li&gt;&#xA;&lt;li&gt;why is no histogram being produced when using picard insertsize metrics?&lt;/li&gt;&#xA;&lt;li&gt;is normal to see the graph shift to the right after trimming?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h1&gt;insertsize for all samples (data from picard collectinsertsize compiled using multiqc)&lt;/h1&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/l3A9I.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/l3A9I.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="926" LastEditorUserId="6251" LastEditDate="2021-09-02T20:50:48.347" LastActivityDate="2021-09-02T20:50:48.347" Title="insert size pre and post trimming" Tags="|rna-seq|rna-splicing|trimming|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2120" PostTypeId="2" ParentId="2119" CreationDate="2017-07-21T22:19:44.113" Score="4" Body="&lt;ol&gt;&#xA;&lt;li&gt;It's not an unreasonable approach. rMATs is rather picky about its input, but it seems you've noticed that.&lt;/li&gt;&#xA;&lt;li&gt;rMATs can't handle trimmed reads. It also can't handle soft-clipped reads. You might as well not trim. You'll get a lower alignment rate, but your only other choice would be to either exclude the trimmed reads or trim everything down to the same length.&lt;/li&gt;&#xA;&lt;li&gt;You have a mixture of RF and FR alignments and the plotting script can't handle that.&lt;/li&gt;&#xA;&lt;li&gt;Assuming you mean the read length distribution, it's actually shifted to the left, not the right. You had a single length before and a long left tail after trimming.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-07-21T22:19:44.113" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2121" PostTypeId="2" ParentId="2114" CreationDate="2017-07-22T10:53:27.133" Score="4" Body="&lt;p&gt;A mixture of &lt;code&gt;apply&lt;/code&gt;, &lt;code&gt;paste&lt;/code&gt; and &lt;code&gt;cat&lt;/code&gt; will do this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; data.mat&#xA;       sample1 sample2 sample3 sample4&#xA;fliI         1       1       1       1&#xA;patB_1       1       1       1       1&#xA;pgpA         1       1       1       1&#xA;osmB         1       1       1       1&#xA;cspA         1       0       1       1&#xA;&amp;gt; data.conv &amp;lt;- apply(data.mat,2,paste,collapse=&quot;&quot;) # aggregate along 2nd dim&#xA;&amp;gt; cat(paste0(&quot;&amp;gt;&quot;, names(data.conv), &quot;\n&quot;, data.conv), sep=&quot;\n&quot;)&#xA;&amp;gt;sample1&#xA;11111&#xA;&amp;gt;sample2&#xA;11110&#xA;&amp;gt;sample3&#xA;11111&#xA;&amp;gt;sample4&#xA;11111&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If writing to a file is required, just add a &lt;code&gt;file&lt;/code&gt; parameter to the &lt;code&gt;cat&lt;/code&gt; command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat(paste0(&quot;&amp;gt;&quot;, names(data.conv), &quot;\n&quot;, data.conv), &#xA;    sep=&quot;\n&quot;, file=&quot;out_fastaLike.txt&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-07-22T10:53:27.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2124" PostTypeId="2" ParentId="884" CreationDate="2017-07-24T15:56:04.047" Score="2" Body="&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually, after having a more thorough look into the documentation; &lt;code&gt;REMARK 3&lt;/code&gt; might be the more relevant to what you need. Other tools are reported in that section. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;REMARK 3 presents information on refinement program(s) used and related statistics. For non-diffraction studies, REMARK 3 is used to describe any refinement done, but its format is mostly free text.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.wwpdb.org/documentation/file-format-content/format33/remark3.html&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;More on `Remark 3`&quot;&gt;More on &lt;code&gt;Remark 3&lt;/code&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;From the same source as you reference yourself &lt;a href=&quot;http://www.wwpdb.org/documentation/file-format-content/format33/remarks1.html&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;v 3.3&quot;&gt;v 3.3&lt;/a&gt;, I would argue that &#xA;&lt;code&gt;REMARK 0&lt;/code&gt; would be more accurate than &lt;code&gt;REMARK 250&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;REMARK 0 (updated), Re-refinement notice&#xA;  REMARK 0 identifies entries in which a re-refinement has been performed using the data from an existing entry. This remark also describes the PDB code and the journal records for the original data set.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;While when you consider the definition of &lt;code&gt;REMARK 250&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- which follows directly:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;REMARK 205&lt;/code&gt; (specific to Fiber diffraction experiment), &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;REMARK 201&lt;/code&gt;(and &lt;code&gt;REMARK 215/217&lt;/code&gt; all specific to NMR experiment),&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;REMARK 230&lt;/code&gt; (neutron diffraction study), &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;REMARK 240&lt;/code&gt;(electron crystallography study) and &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;REMARK 245&lt;/code&gt;(and &lt;code&gt;REMARK 247&lt;/code&gt;: specific to EM study)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;REMARK 250, Other Type of Experiment Details&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;REMARKs specific to other kinds of studies, not listed above. &#xA;  REMARK 250 is mandatory if other than X-ray, NMR, neutron, or electron study. The format of the date in this remark is DD-MMM-YY. DD is the day of the month (a number 01 through 31), MMM is the English 3-letter abbreviation for the month, and YY is the year.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and then &lt;code&gt;REMARK 265&lt;/code&gt; is also about the crystallography experiment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hence, I know that personally, I usually assume that whatever is annotated in &lt;code&gt;REMARK 250&lt;/code&gt;is relevant to the first acquisition experiment to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another reason I am more inclined to think &lt;code&gt;REMARK 0&lt;/code&gt;is more accurate, it is due that the definition of &lt;code&gt;REMARK 0&lt;/code&gt; was &lt;strong&gt;updated&lt;/strong&gt; while &lt;code&gt;REMARK 6-99&lt;/code&gt; are &lt;em&gt;are no longer for use of free text annotation&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, I am not sure how &lt;a href=&quot;http://pdb-extract.wwpdb.org/&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Pdb_extract&quot;&gt;Pdb_extract&lt;/a&gt; would do with the remarks when converting the PDB file to CIF files. It would probably worth perusing  &lt;a href=&quot;http://mmcif.wwpdb.org/docs/pdb_to_pdbx_correspondences.html#XPLOR&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;PDB to PDBx/mmCIF Data Item Correspondences&quot;&gt;PDB to PDBx/mmCIF Data Item Correspondences&lt;/a&gt; page.&lt;/p&gt;&#xA;" OwnerUserId="203" LastEditorUserId="203" LastEditDate="2017-07-26T23:01:09.873" LastActivityDate="2017-07-26T23:01:09.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2125" PostTypeId="2" ParentId="949" CreationDate="2017-07-24T19:52:43.587" Score="2" Body="&lt;p&gt;Hm, I didn't know that the plugin existed so I wrote my own script to convert GP to minor allele dosage on github. Maybe someone else will find it useful :) &lt;a href=&quot;https://github.com/7methylg/VCF-GP-to-DS&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/7methylg/VCF-GP-to-DS&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1183" LastActivityDate="2017-07-24T19:52:43.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2128" PostTypeId="1" CreationDate="2017-07-25T09:34:43.760" Score="8" ViewCount="467" Body="&lt;p&gt;I would like to gather proteins FASTA sequence from Entrez with python 2.7. I am looking for any proteins that have the keywords: &quot;terminase&quot; and &quot;large&quot; in their name. So far I got this code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez&#xA;Entrez.email = &quot;example@example.org&quot;&#xA;&#xA;&#xA;searchResultHandle = Entrez.esearch(db=&quot;protein&quot;, term=&quot;terminase large&quot;, retmax=1000)&#xA;searchResult = Entrez.read(searchResultHandle)&#xA;ids = searchResult[&quot;IdList&quot;]&#xA;&#xA;handle = Entrez.efetch(db=&quot;protein&quot;, id=ids, rettype=&quot;fasta&quot;, retmode=&quot;text&quot;)&#xA;record = handle.read()&#xA;&#xA;out_handle = open('myfasta.fasta', 'w')&#xA;out_handle.write(record.rstrip('\n'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, it can get me several terminases from various organisms, while I need only terminase form bacteriophages (specificly Viruses [taxid 10239], host: bacteria). I've managed to get the nuccore accession ids from NCBI of the viruses I am intersted in, but I don't know how to combine those two informations. The id file looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NC_001341&#xA;NC_001447&#xA;NC_028834&#xA;NC_023556&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Do I need to access every gb file of every ID and search for my desired protein in it?&lt;/p&gt;&#xA;" OwnerUserId="1193" LastActivityDate="2017-07-25T12:28:13.403" Title="Getting protein FASTA sequence based on keyword with python" Tags="|python|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2130" PostTypeId="1" AcceptedAnswerId="2132" CreationDate="2017-07-25T11:11:26.397" Score="4" ViewCount="37" Body="&lt;p&gt;Can anyone explain me, why I don't find a specific protein with a blast that was took before from the NCBI refseq database?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, I was trying to blast the protein with the accession number &quot;NP_420767&quot; and its sequence, respectively, however that protein does not show up in the results. It not only happens when the standard options are chosen, but also, when &quot;Reference proteins (refseq_protein)&quot; as database in the blast options is selected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am really puzzled by that.. Shouldn't blast show the initial refseq entry in the results list as it is part of the refseq database and has the same sequence?&lt;/p&gt;&#xA;" OwnerUserId="1191" LastActivityDate="2017-07-25T11:27:52.257" Title="blasting a refseq protein does not show the protein in the result set" Tags="|blast|refseq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2131" PostTypeId="1" AcceptedAnswerId="2133" CreationDate="2017-07-25T11:26:33.690" Score="7" ViewCount="898" Body="&lt;p&gt;Is there a way to filter bases in BAM files based on phred quallities through python's pysam ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a code here that&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Takes the nucleobases per position from a BAM file using pysam's pileup function&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Saves it in ReverseList and ForwardList based on both strands (i.e forward and reverse), I want to reject those bases that have phred quality below 25 to be not stored in the ForwardList and ReverseList lists so that they are not used for further analysis.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; samfile = pysam.Samfile( filename, &quot;rb&quot; )&#xA; ReverseList = [''] * lenref&#xA; ForwardList = [''] * lenref&#xA; for pileupcolumn in samfile.pileup() :&#xA;     for pileupread in pileupcolumn.pileups:&#xA;         if (pileupread.alignment.mapping_quality &amp;lt;= 15):&#xA;               continue      &#xA;         if not pileupread.is_del and not pileupread.is_refskip:&#xA;               if pileupread.alignment.is_reverse: #negative&#xA;                      ReverseList[pileupcolumn.pos] += pileupread.alignment.query_sequence[pileupread.query_position]&#xA;               else:&#xA;                      ForwardList[pileupcolumn.pos] += pileupread.alignment.query_sequence[pileupread.query_position]&#xA;&#xA; samfile.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Where lenref = 16569(length of mitochondrial genome) and filename is the name of BAM file. I want to filter based of phred qualities of bases.&lt;/p&gt;&#xA;" OwnerUserId="1194" LastActivityDate="2017-07-25T11:42:57.597" Title="Filtering bases based on phred qualities with pysam" Tags="|sam|python|pysam|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2132" PostTypeId="2" ParentId="2130" CreationDate="2017-07-25T11:27:52.257" Score="2" Body="&lt;p&gt;NP_420767 is represented by the non-redundant refSeq protein WP_010919826, which has the same amino acid sequence. This is not very clearly annotated, but if you scroll down to the sequence in the GenPept entry for NP_420767, you'll see the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CONTIG      join(WP_010919826.1:1..799)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/refseq/about/nonredundantproteins&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for more info.&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2017-07-25T11:27:52.257" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2133" PostTypeId="2" ParentId="2131" CreationDate="2017-07-25T11:42:57.597" Score="4" Body="&lt;p&gt;The PileupRead object as a query_position attribute, which you can use for this:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;for pileupcolumn in samfile.pileup() :&#xA;    for pileupread in pileupcolumn.pileups:&#xA;         if (pileupread.alignment.mapping_quality &amp;lt;= 15):&#xA;             continue      &#xA;         if not pileupread.is_del and not pileupread.is_refskip:&#xA;             if pileupread.alignment.query_qualities[pileupread.query_position] &amp;lt; 10:&#xA;                 # Skip entries with base phred scores &amp;lt; 10&#xA;                 continue&#xA;             if pileupread.alignment.is_reverse: #negative&#xA;                   ReverseList[pileupcolumn.pos] += pileupread.alignment.query_sequence[pileupread.query_position]&#xA;             else:&#xA;                   ForwardList[pileupcolumn.pos] += pileupread.alignment.query_sequence[pileupread.query_position]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note the 6th line, which implements a filter with a threshold of 10.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-07-25T11:42:57.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2134" PostTypeId="2" ParentId="2113" CreationDate="2017-07-25T11:43:31.083" Score="4" Body="&lt;p&gt;I don't know of any prebuilt products, but I can describe how we managed this in my postdoc lab, and how I plan to manage it in my newly-started group. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rule 1: All work happens in the projects project directory on the central &#xA;filestore, not on your desktop or laptop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rule 2: Heavy computational work is done by the groups standard analysis pipelines. Interpretation is done in jupyter notebooks or Rmarkdown.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A project has a directory on the group's filestore. That directory has a fixed structure:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;proj001----raw_data             *&#xA;       |&#xA;        --external_datasets&#xA;       |&#xA;        --src                  *&#xA;       |&#xA;        --notebooks            *&#xA;       |&#xA;        --pipeline1&#xA;       |&#xA;        --pipeline2&#xA;       | &#xA;        -- etc&#xA; * link to a seperate, backed up filesystem.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Pipelines are where heavy duty analysis happens and everyone uses the same standard pipelines, producing the same standard output files and database structure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So common pipelines might be the mapping pipeline, the readqc pipeline, the differential expression pipeline, the exome pipeline, the chip-seq pipeline etc...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pipelines have three important outputs: an automatically generated html report, an sqlite database and files in the export directory. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we archive a project, this is what we save, along with the pipeline's configuration file and log file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if I know that Jane Bloggs did an RNAseq 5 years ago on a cell type i'm interested in, if I konw that that was project 5, I know that in the project 5 directory there will be a &lt;code&gt;diff_expression_pipeline&lt;/code&gt; directory, and that it will contain an sqlite database called &lt;code&gt;csvdb&lt;/code&gt; and that that will have a table called &lt;code&gt;refcoding_deseq_gene_diff&lt;/code&gt; and that table will follow a known format. There will be bigwigs in the &lt;code&gt;export&lt;/code&gt; directory. Or the BAMs will be in the &lt;code&gt;mapping_pipeline&lt;/code&gt; directory. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course the problem remains knowing that Jane Bloggs did this RNAseq and what the project was called. We use a Wiki for this, but its not ideal. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2017-07-25T11:43:31.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2135" PostTypeId="2" ParentId="2128" CreationDate="2017-07-25T12:28:13.403" Score="4" Body="&lt;p&gt;Found what I was looking for. In:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;searchResultHandle = Entrez.esearch(db=&quot;protein&quot;, term=&quot;terminase large&quot;, retmax=1000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've added:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;searchterm = &quot;(terminase large subunit AND viruses[Organism]) AND Caudovirales AND refseq[Filter]&quot;&#xA;searchResultHandle = Entrez.esearch(db=&quot;protein&quot;, term=searchterm, retmax=6000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which norrowed down my searches to the desired viruses. Granted it's not filtered by host, but by a taxonomy group, but it is enough for my work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you @Llopis for additional help&lt;/p&gt;&#xA;" OwnerUserId="1193" LastActivityDate="2017-07-25T12:28:13.403" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2136" PostTypeId="2" ParentId="658" CreationDate="2017-07-25T13:07:46.147" Score="5" Body="&lt;p&gt;I'm not sure what kinds of bioinformatics tasks you would like to perform, therefore it is difficult to give a good recommendation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're specifically working on statistical genetics, I can recommend &lt;a href=&quot;https://hail.is&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hail&lt;/a&gt; [1]. Hail is an open-source tool for analyzing genetics data at the tens of terabyte scale. Most of Hail's users do their science in Jupyter notebooks that are backed by Google Cloud Platform Dataproc clusters. Hail permits you to perform a variety of statistical genetics tasks including:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;filtering and aggregation for quality control&lt;/li&gt;&#xA;&lt;li&gt;subsetting, linear regression, linear mixed model regression, and linear burden testing&lt;/li&gt;&#xA;&lt;li&gt;utilities for computing various measures of relatedness&lt;/li&gt;&#xA;&lt;li&gt;principal components analysis&lt;/li&gt;&#xA;&lt;li&gt;variant splitting&lt;/li&gt;&#xA;&lt;li&gt;import/export from a variety of formats including PLINK, VCF, and BGEN, and&lt;/li&gt;&#xA;&lt;li&gt;a python API which enables the use of libraries like matplotlib for plotting analysis results&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To learn specifically about using Hail with the Google Cloud Platform and Jupyter notebooks, I strongly recommend &lt;a href=&quot;http://discuss.hail.is/t/using-hail-with-jupyter-notebooks-on-google-cloud/196/2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Liam's Hail forum post about his cloud-tools repository&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example, from the &lt;a href=&quot;https://hail.is/hail/tutorials/hail-overview.html#Quality-Control&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hail tutorial&lt;/a&gt;, of using Hail to perform some quality control and display a scatter plot of the first two principal components of the individuals:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from hail import *&#xA;import matplotlib.pyplot as plt&#xA;import matplotlib.patches as mpatches&#xA;&#xA;hc = HailContext()&#xA;&#xA;table = hc.import_table('data/1kg_annotations.txt', impute=True).key_by('Sample')&#xA;common_vds = (hc.read('data/1kg.vds')&#xA;              .annotate_samples_table(table, root='sa')&#xA;              .sample_qc()&#xA;              .filter_samples_expr('sa.qc.dpMean &amp;gt;= 4 &amp;amp;&amp;amp; sa.qc.callRate &amp;gt;= 0.97')&#xA;              .filter_genotypes('''let ab = g.ad[1] / g.ad.sum() in&#xA;                         ((g.isHomRef &amp;amp;&amp;amp; ab &amp;lt;= 0.1) ||&#xA;                          (g.isHet &amp;amp;&amp;amp; ab &amp;gt;= 0.25 &amp;amp;&amp;amp; ab &amp;lt;= 0.75) ||&#xA;                          (g.isHomVar &amp;amp;&amp;amp; ab &amp;gt;= 0.9))''')&#xA;              .variant_qc()&#xA;              .filter_variants_expr('va.qc.AF &amp;gt; 0.01')&#xA;              .ld_prune(memory_per_core=512, num_cores=4))&#xA;&#xA;pca = common_vds.pca('sa.pca', k=5, eigenvalues='global.eigen')&#xA;pca_table = pca.samples_table().to_pandas()&#xA;&#xA;colors = {'AFR': 'green', 'AMR': 'red', 'EAS': 'black', 'EUR': 'blue', 'SAS': 'cyan'}&#xA;plt.scatter(pca_table[&quot;sa.pca.PC1&quot;], pca_table[&quot;sa.pca.PC2&quot;],&#xA;            c = pca_table[&quot;sa.SuperPopulation&quot;].map(colors),&#xA;            alpha = .5)&#xA;plt.xlim(-0.6, 0.6)&#xA;plt.xlabel(&quot;PC1&quot;)&#xA;plt.ylabel(&quot;PC2&quot;)&#xA;legend_entries = [mpatches.Patch(color=c, label=pheno) for pheno, c in colors.items()]&#xA;plt.legend(handles=legend_entries, loc=2)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;[1] Disclaimer: I work on Hail&lt;/p&gt;&#xA;" OwnerUserId="1087" LastEditorUserId="1087" LastEditDate="2017-08-15T17:50:38.403" LastActivityDate="2017-08-15T17:50:38.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2137" PostTypeId="1" AcceptedAnswerId="2138" CreationDate="2017-07-25T15:56:54.430" Score="6" ViewCount="571" Body="&lt;p&gt;I have a gtf file from Ensembl, and I noticed that several &quot;transcript&quot; annotations have the exact same coordinates. See for instance the third and fourth transcripts (&quot;Y74C9A.2b.1&quot; and &quot;Y74C9A.2b.4&quot;) for this gene:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep &quot;WBGene00022276&quot; genes.gtf | awk '$3 == &quot;transcript&quot; {print}'&#xA;I   ensembl transcript  10413   16842   .   +   .   gene_biotype &quot;protein_coding&quot;; gene_id &quot;WBGene00022276&quot;; gene_name &quot;nlp-40&quot;; gene_source &quot;ensembl&quot;; gene_version &quot;1&quot;; p_id &quot;P24752&quot;; transcript_biotype &quot;protein_coding&quot;; transcript_id &quot;Y74C9A.2a.2&quot;; transcript_name &quot;Y74C9A.2a.2&quot;; transcript_source &quot;ensembl&quot;; transcript_version &quot;1&quot;; tss_id &quot;TSS7921&quot;;&#xA;I   ensembl transcript  11495   16793   .   +   .   gene_biotype &quot;protein_coding&quot;; gene_id &quot;WBGene00022276&quot;; gene_name &quot;nlp-40&quot;; gene_source &quot;ensembl&quot;; gene_version &quot;1&quot;; p_id &quot;P22572&quot;; transcript_biotype &quot;protein_coding&quot;; transcript_id &quot;Y74C9A.2b.2&quot;; transcript_name &quot;Y74C9A.2b.2&quot;; transcript_source &quot;ensembl&quot;; transcript_version &quot;1&quot;; tss_id &quot;TSS9220&quot;;&#xA;I   ensembl transcript  11499   16842   .   +   .   gene_biotype &quot;protein_coding&quot;; gene_id &quot;WBGene00022276&quot;; gene_name &quot;nlp-40&quot;; gene_source &quot;ensembl&quot;; gene_version &quot;1&quot;; p_id &quot;P22572&quot;; transcript_biotype &quot;protein_coding&quot;; transcript_id &quot;Y74C9A.2b.1&quot;; transcript_name &quot;Y74C9A.2b.1&quot;; transcript_source &quot;ensembl&quot;; transcript_version &quot;1&quot;; tss_id &quot;TSS9215&quot;;&#xA;I   ensembl transcript  11499   16842   .   +   .   gene_biotype &quot;protein_coding&quot;; gene_id &quot;WBGene00022276&quot;; gene_name &quot;nlp-40&quot;; gene_source &quot;ensembl&quot;; gene_version &quot;1&quot;; p_id &quot;P22572&quot;; transcript_biotype &quot;protein_coding&quot;; transcript_id &quot;Y74C9A.2b.4&quot;; transcript_name &quot;Y74C9A.2b.4&quot;; transcript_source &quot;ensembl&quot;; transcript_version &quot;1&quot;; tss_id &quot;TSS9215&quot;;&#xA;I   ensembl transcript  11505   16842   .   +   .   gene_biotype &quot;protein_coding&quot;; gene_id &quot;WBGene00022276&quot;; gene_name &quot;nlp-40&quot;; gene_source &quot;ensembl&quot;; gene_version &quot;1&quot;; p_id &quot;P24752&quot;; transcript_biotype &quot;protein_coding&quot;; transcript_id &quot;Y74C9A.2a.3&quot;; transcript_name &quot;Y74C9A.2a.3&quot;; transcript_source &quot;ensembl&quot;; transcript_version &quot;1&quot;; tss_id &quot;TSS16820&quot;;&#xA;I   ensembl transcript  11618   16842   .   +   .   gene_biotype &quot;protein_coding&quot;; gene_id &quot;WBGene00022276&quot;; gene_name &quot;nlp-40&quot;; gene_source &quot;ensembl&quot;; gene_version &quot;1&quot;; p_id &quot;P24752&quot;; transcript_biotype &quot;protein_coding&quot;; transcript_id &quot;Y74C9A.2a.1&quot;; transcript_name &quot;Y74C9A.2a.1&quot;; transcript_source &quot;ensembl&quot;; transcript_version &quot;1&quot;; tss_id &quot;TSS43845&quot;;&#xA;I   ensembl transcript  11623   16842   .   +   .   gene_biotype &quot;protein_coding&quot;; gene_id &quot;WBGene00022276&quot;; gene_name &quot;nlp-40&quot;; gene_source &quot;ensembl&quot;; gene_version &quot;1&quot;; p_id &quot;P22572&quot;; transcript_biotype &quot;protein_coding&quot;; transcript_id &quot;Y74C9A.2b.3&quot;; transcript_name &quot;Y74C9A.2b.3&quot;; transcript_source &quot;ensembl&quot;; transcript_version &quot;1&quot;; tss_id &quot;TSS49800&quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What does it mean? Do these two &quot;transcripts&quot; actually correspond to an exon, belonging to two different isoforms?&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2017-07-25T17:14:56.770" Title="Same transcript coordinates in gtf file, different transcript ID" Tags="|sequence-annotation|gtf|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2138" PostTypeId="2" ParentId="2137" CreationDate="2017-07-25T16:48:03.920" Score="7" Body="&lt;p&gt;&lt;code&gt;transcript&lt;/code&gt; objects cover the co-ordinates from the start of the first exon to the end of the last exon of a transcript (i.e. an isoform). If two different isoforms share the same first and last exons, but have a different set of internal exons, then their &lt;code&gt;transcript&lt;/code&gt; entries will be the same, but the set of &lt;code&gt;exon&lt;/code&gt; entires associated with each transcript will be different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider the two transcripts below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                      1         2&#xA;             12345678901234567890123 &#xA;transcript_1 |&amp;gt;&amp;gt;&amp;gt;|----|&amp;gt;&amp;gt;|----|&amp;gt;&amp;gt;&amp;gt;&amp;gt;|&#xA;&#xA;transcript_2 |&amp;gt;&amp;gt;|--|&amp;gt;&amp;gt;|------|&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;|&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These two transcripts would give the following GTF entires&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1   ensembl   transcript    1   23   .   +   .   gene_id &quot;gene_1&quot;; transcript_id &quot;transcript_1&quot;;&#xA;chr1   ensembl   exon    1   5   .   +   .   gene_id &quot;gene_1&quot;; transcript_id &quot;transcript_1&quot;;&#xA;chr1   ensembl   exon    10   13   .   +   .   gene_id &quot;gene_1&quot;; gene_id &quot;gene_1&quot;; transcript_id &quot;transcript_1&quot;;   &#xA;chr1   ensembl   exon    18   23   .   +   .   gene_id &quot;gene_1&quot;; transcript_id &quot;transcript_1&quot;;   &#xA;chr1   ensembl   transcript    1   23   .   +   .   gene_id &quot;gene_1&quot;; transcript_id &quot;transcript_2&quot;;&#xA;chr1   ensembl   exon    1   4   .   +   .   gene_id &quot;gene_1&quot;; transcript_id &quot;transcript_2&quot;;&#xA;chr1   ensembl   exon    7   10   .   +   .   gene_id &quot;gene_1&quot;; transcript_id &quot;transcript_2&quot;;   &#xA;chr1   ensembl   exon    17   23   .   +   .   gene_id &quot;gene_1&quot;; transcript_id &quot;transcript_2&quot;;   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="235" LastActivityDate="2017-07-25T16:48:03.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2139" PostTypeId="2" ParentId="2137" CreationDate="2017-07-25T17:07:37.253" Score="8" Body="&lt;p&gt;As Ian &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2138/298&quot;&gt;explained&lt;/a&gt;, these are different transcripts which happen to have the same start and end positions. You have no information on their exonic structure in that file. However, if you look them up at EnsEMBL, you will see:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://May2017.archive.ensembl.org/Caenorhabditis_elegans/Transcript/Summary?db=core;g=WBGene00022276;r=I:11499-16842;t=Y74C9A.2b.1&quot; rel=&quot;noreferrer&quot;&gt;Transcript Y74C9A.2b.1&lt;/a&gt; :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yYXlV.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yYXlV.png&quot; alt=&quot;exonic structure of Y74C9A.2b.1&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   Y74C9A.2b.1.e1  11,499  11,561  -   -   63&#xA;    Intron 1-2      11,562  11,617          56&#xA;2   Y74C9A.2b.1.e2  11,618  11,689  -   -   72&#xA;    Intron 2-3      11,690  14,950          3,261&#xA;3   Y74C9A.2b.1.e3  14,951  15,160  -   1   210&#xA;    Intron 3-4      15,161  16,472          1,312&#xA;4   Y74C9A.2a.1.e3  16,473  16,842  1   -   370&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This has 4 exons, two of which are non-coding (UTR). The second starts at &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://May2017.archive.ensembl.org/Caenorhabditis_elegans/Transcript/Summary?db=core;g=WBGene00022276;r=I:11499-16842;t=Y74C9A.2b.4&quot; rel=&quot;noreferrer&quot;&gt;Transcript Y74C9A.2b.4&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uZbYI.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uZbYI.png&quot; alt=&quot;exonic structure of Y74C9A.2b.4&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   Y74C9A.2b.4.e1  11,499  11,557  -   -   59&#xA;    Intron 1-2      11,558  11,617          60&#xA;2   Y74C9A.2b.1.e2  11,618  11,689  -   -   72&#xA;    Intron 2-3      11,690  14,950          3,261&#xA;3   Y74C9A.2b.1.e3  14,951  15,160  -   1   210&#xA;    Intron 3-4      15,161  16,472          1,312&#xA;4   Y74C9A.2a.1.e3  16,473  16,842  1   -   370&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Granted, the two transcripts are very simlar, they both have 4 exons, the first 2 of which are non coding (UTR). However, the 1st exon of transcript &#xA;Y74C9A.2b.1 is 63 nt long and ends at 11,561 while the 1st exon of transcript Y74C9A.2b.4 is 59 nts and ends at 11,557.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So there you go: two (slightly) different transcripts. &lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-07-25T17:14:56.770" LastActivityDate="2017-07-25T17:14:56.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2140" PostTypeId="1" AcceptedAnswerId="2144" CreationDate="2017-07-25T17:23:52.373" Score="7" ViewCount="559" Body="&lt;p&gt;We have an experimental design as seen below &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where we administered drug at 0 min for each mouse genotype and took them down at given below intervals. wt and ko mouse models  were administered only with the vehicle which would be our negative control.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Performed rnaseq (total rna) truseq stranded protocol would like to see differences between each time point compared to the vehicle, used star aligner to align to mm9 , htseq for gene counts . &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sample  type    time&#xA;WT  Vehicle     30 min&#xA;KO  Vehicle     30 min&#xA;WT  Drug    30min&#xA;KO  Drug    30min&#xA;WT  Drug    1 hour&#xA;KO  Drug    1 hour&#xA;WT  Drug    2 hour&#xA;KO  Drug    2hour&#xA;WT  Drug    3 hour&#xA;KO  Drug    3hour&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We do not have any replicates and hence cannot use any of the differential expression statistical tools (asked this question at bioconductor where Dr. Love himself replied &lt;a href=&quot;https://support.bioconductor.org/p/98429/#98456&quot; rel=&quot;noreferrer&quot;&gt;link to question asked&lt;/a&gt;) where he suggested a linear model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;we were wondering if someone could help us with any such design (papers/previously performed analysis) what we would like to see is comparison/differences (significant) between these groups &#xA;(veh vs 30min, veh vs 1 hour, beh vs 2 hours, veh vs 3 hours) &lt;/p&gt;&#xA;" OwnerUserId="926" LastEditorUserId="926" LastEditDate="2017-07-25T19:41:07.320" LastActivityDate="2017-07-26T14:28:53.783" Title="differential gene expression complex design no replicates" Tags="|rna-seq|deseq2|differential-expression|" AnswerCount="2" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2141" PostTypeId="2" ParentId="2113" CreationDate="2017-07-25T22:01:40.587" Score="2" Body="&lt;p&gt;We have been organizing data on a &quot;per project&quot; basis, using a GitHub repository for each project. Each project ends up being a paper (written in R Markdown), so you could have multiple &quot;projects&quot; based on the same datasets. See this as an example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/SchlossLab/new_project&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/SchlossLab/new_project&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The analyses are all managed with a Makefile that goes from loading the sequence data from the SRA, to performing analyses, to rendering the final manuscript. This way it is always clear how things were done. Here is an example of what that can look like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/SchlossLab/new_project/blob/master/Makefile&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/SchlossLab/new_project/blob/master/Makefile&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as organizing the raw data and whatnot, I have just used a simple MySQL database hosted on MAMP (Mac version, Linux is LAMP, Windows is WAMP). It has a GUI and everything, and is pretty easy to use. Here is a resource for getting started on that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://microbiology.github.io/PDFs/MySQL-Intro-Workshop-Hannigan.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://microbiology.github.io/PDFs/MySQL-Intro-Workshop-Hannigan.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All of this stuff is free to use, which is awesome! Hope this helps! :)&lt;/p&gt;&#xA;" OwnerUserId="824" LastActivityDate="2017-07-25T22:01:40.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2142" PostTypeId="1" AcceptedAnswerId="2143" CreationDate="2017-07-26T01:44:09.730" Score="14" ViewCount="303" Body="&lt;p&gt;I am doing a research project involving calculating k-mer frequencies and I am wondering if there is any standard file format for storing k-mer counts.&lt;/p&gt;&#xA;" OwnerUserId="1196" LastActivityDate="2017-07-28T01:33:46.580" Title="Is there a standard k-mer count file format?" Tags="|file-formats|k-mer|phylogenetics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2143" PostTypeId="2" ParentId="2142" CreationDate="2017-07-26T05:13:34.800" Score="6" Body="&lt;p&gt;Not as far as I am aware. The Ray assembler used to (and possibly still does) store the kmers as FASTA files where the header was the count of the sequence, which I thought was a pretty neat bastardisation of the FASTA file format. It looks like this format is also used by Jellyfish when reporting kmer frequencies by the &lt;code&gt;dump&lt;/code&gt; command (but its default output format is a custom binary format):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The dump subcommand outputs a list of all the k-mers in the file associated with their count. By default, the output is in FASTA format, where the header line contains the count of the k-mer and the sequence part is the sequence of the k-mer. This format has the advantage that the output contains the sequence of k-mers and can be directly fed into another program expecting the very common FASTA format. A more convenient column format (for human beings) is selected with the -c  switch.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.genome.umd.edu/jellyfish.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jellyfish&lt;/a&gt; changed their internal format between v1 and v2 (both not FASTA), because they changed to doing counts based on bloom filters. Jellyfish2 has an optional two-pass method that sets up a bloom filter intermediate file to record kmers, and multiple different final reporting formats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://khmer.readthedocs.io/en/v2.1.2/introduction.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Khmer&lt;/a&gt; also uses bloom filters, but in a slightly different way. It also has been extended to be useful for partitioning and comparing datasets.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-07-28T01:33:46.580" LastActivityDate="2017-07-28T01:33:46.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2144" PostTypeId="2" ParentId="2140" CreationDate="2017-07-26T07:38:16.753" Score="7" Body="&lt;p&gt;Although it is not recommended to use no replicates, in the edgeR manual they give some advice on how to go on with no replicates design. See page 21 of user &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/edgeR/inst/doc/edgeRUsersGuide.pdf&quot; rel=&quot;noreferrer&quot;&gt;guide&lt;/a&gt;. You can e.g. estimate a BCV value. Of course it is still a trick and not sound statistics.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2017-07-26T07:38:16.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2145" PostTypeId="1" AcceptedAnswerId="2147" CreationDate="2017-07-26T09:40:18.733" Score="8" ViewCount="973" Body="&lt;p&gt;Low coverage MinION reads should be useful to close gaps and resolve repeats left by short-read assemblers. However, I haven't had any success with the software I know about. I'm aware of the following packages, either for scaffolding or closing gaps in short-read assemblies using long reads:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/warrenlr/LINKS/&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;LINKS&lt;/code&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://sourceforge.net/projects/operasf&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;OPERA-LG&lt;/code&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/mdcao/npScarf&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;npScarf&lt;/code&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://sourceforge.net/projects/pb-jelly&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;PBJelly&lt;/code&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.baseclear.com/genomics/bioinformatics/basetools/SSPACE-longread&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;SSPACE-longread&lt;/code&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I've tried &lt;code&gt;npScarf&lt;/code&gt; and &lt;code&gt;LINKS&lt;/code&gt;, and they ran successfully but didn't resolve the gaps in my assembly. I couldn't get &lt;code&gt;PBJelly&lt;/code&gt; and &lt;code&gt;OPERA-LG&lt;/code&gt; to run with the current versions of &lt;code&gt;BLASR&lt;/code&gt; and &lt;code&gt;samtools&lt;/code&gt;, and both packages seem not to be maintained. I have not tried &lt;code&gt;SSPACE-longread&lt;/code&gt; because it's not open source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What software can I use to fill gaps and resolve repeats in a short-read assembly with low-coverage Nanopore data?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;More information&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I'm finishing a mitochondrial genome. I have a 17 kb short read assembly with one gap. This was made from 2x150b paired-end reads from a TruSeq PCR-free library with a 400 b insert size. Molecular data suggests the genome may be around 32 kb.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After mapping short reads against the short read assembly, &lt;code&gt;Pilon&lt;/code&gt; identified a tandem repeat of 156 bp in an AT-rich region, but wasn't able to close the gap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 1.4 Gb of MinION rapid reads with an L50 of 8808 b. I mapped these reads against the short read assembly, and I can see reads that span the gap. I seem to have the information required to close the gap, but I don't know how to do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The organism has a nuclear genome &gt; 600 Mb, so long read coverage is low. Despite this, I tried a &lt;em&gt;de novo&lt;/em&gt; &lt;code&gt;Canu&lt;/code&gt; assembly of the Nanopore reads, and I pulled out a 39 kb contig containing mitochondrial genes. Most genes on this contig are duplicated and fragmented, and &lt;code&gt;Pilon&lt;/code&gt; wasn't able to improve it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for reading!&lt;/p&gt;&#xA;" OwnerUserId="1021" LastEditorUserId="1021" LastEditDate="2017-07-26T10:34:48.093" LastActivityDate="2017-07-26T10:34:48.093" Title="How can I use Nanopore reads to close gaps or resolve repeats in a short-read assembly?" Tags="|nanopore|long-reads|repeat-elements|scaffold|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2147" PostTypeId="2" ParentId="2145" CreationDate="2017-07-26T10:25:29.833" Score="6" Body="&lt;p&gt;You might want to look into &lt;a href=&quot;https://github.com/rrwick/Unicycler&quot; rel=&quot;noreferrer&quot;&gt;Unicycler&lt;/a&gt; (manuscript with more information &lt;a href=&quot;http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005595&quot; rel=&quot;noreferrer&quot;&gt;can be found here&lt;/a&gt;); even though it is supposed to be used with bacterial genomes only, it might work well with a small genome such as a mitochondrion's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wq02F.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wq02F.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beware that if you happen to have very long reads, you might end up with an assembly with multiple copies of the circular genome: you might want to look into &lt;a href=&quot;https://github.com/sanger-pathogens/circlator&quot; rel=&quot;noreferrer&quot;&gt;circlator&lt;/a&gt; then.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="298" LastEditDate="2017-07-26T10:30:40.230" LastActivityDate="2017-07-26T10:30:40.230" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2148" PostTypeId="2" ParentId="2140" CreationDate="2017-07-26T11:42:11.777" Score="7" Body="&lt;p&gt;Mike Love is right. &lt;strong&gt;If&lt;/strong&gt; the response you are looking for is linear in terms of change per minute, the most productive approach is likely to be fitting a linear model. You &lt;strong&gt;might&lt;/strong&gt; get something out of this because the difference between successive time points represents multiple measurements of the rate of change over time. The biggest problem is that the time points aren't independent. You would need to fit a model that estimated that rate for both WT and KO and then test the difference in rates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see three ways forward, in all cases you want to test the last term in the model &lt;code&gt;~time + genotype + time:genotype&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Ideally you would like to fit a negative binomial linear model to the read counts. But that is going to be difficult as you have no replicates from which to assess the dispersion. Thus you could generate read counts using your favorite read count pipeline (STAR+featureCounts/Salmon/Kallisto) and take the approach in the edgeR guide noted by &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2144/235&quot; title=&quot;b.nota&quot;&gt;b.nota&lt;/a&gt; and then use edgeR to test the fitted model.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You could use something like salmon or kallisto to estimate the TPMs assume that log TPMs are normally distributed and model them with plain &lt;code&gt;lm&lt;/code&gt; in R, using something like &lt;code&gt;broom&lt;/code&gt; to model each gene separately. A tutorial by David Robinson, the author of &lt;code&gt;broom&lt;/code&gt; on doing this sort of thing can be found &lt;a href=&quot;http://varianceexplained.org/r/tidy-genomics-broom/&quot; rel=&quot;noreferrer&quot; title=&quot;here&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You could use the Salmon or Kallisto bootstraps to estimate &lt;em&gt;technical&lt;/em&gt; variation (but not biological) and then I suspect you could use Sleuth to fit the model.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In any case these results should be treated with scepticism andmostly treated as hypothesis generating. You might find genes that you want to go back and confirm and you might see some patterns of interest if you look at some enrichments etc. which you can then design experiments to test. &lt;/p&gt;&#xA;" OwnerUserId="235" LastEditorUserId="235" LastEditDate="2017-07-26T14:28:53.783" LastActivityDate="2017-07-26T14:28:53.783" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2149" PostTypeId="1" AcceptedAnswerId="2150" CreationDate="2017-07-26T11:54:47.203" Score="6" ViewCount="1279" Body="&lt;h3&gt;Question&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;How can I extract reads from a &lt;code&gt;bam&lt;/code&gt; file (produced by &lt;code&gt;bwa-mem&lt;/code&gt;) to &lt;code&gt;fastq&lt;/code&gt; given a list of reference sequences to filter out?&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Potential difficulties&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;maintaining FR orientation of pair end reads (in &lt;code&gt;bam&lt;/code&gt; all the sequences are reference sequences)&lt;/li&gt;&#xA;&lt;li&gt;keeping R1 and R2 reads &lt;/li&gt;&#xA;&lt;li&gt;keeping quality scores in the same encoding as original fastq (default illumina phred scores in my case)&lt;/li&gt;&#xA;&lt;li&gt;bam can be (ana usually is) sorted by coordinates&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;Almost there solutions&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/blaxterlab/blobology/blob/master/bowtie2_extract_reads_mapped_to_specific_contigs.pl&quot; rel=&quot;noreferrer&quot;&gt;Sujai's perl solution&lt;/a&gt; in &lt;a href=&quot;https://github.com/blaxterlab/blobology&quot; rel=&quot;noreferrer&quot;&gt;blobology&lt;/a&gt; does exact opposite - getting reads from list of references (so I could just reverse the list). The disadvantage is that the script outputs an interleaved &lt;code&gt;fq&lt;/code&gt; file; requires unique names of mates, otherwise R1/R2 information is lost.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;samtools + grep them all from fastq files&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;create a list of read names that do not map to filtered scaffolds. (cut will extract just read names, uniq will collapse pair end read names if they are the same). Then grep read names from fastq files and remove -- separator between hits&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view foo.bam | grep -vf list_of_scaffols_filter \&#xA;  | cut -f 1 | uniq &amp;gt; list_of_reads_to_keep&#xA;&#xA;grep -A 3 -f list_of_reads_to_keep foo_R1.fq | grep -v &quot;^--$&quot; &amp;gt; foo_R1_filtered_bash.fq&#xA;grep -A 3 -f list_of_reads_to_keep foo_R2.fq | grep -v &quot;^--$&quot; &amp;gt; foo_R1_filtered_bash.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;filter bam &amp;amp; picard tools&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Or I could do just the filtering part and use &lt;a href=&quot;http://broadinstitute.github.io/picard/command-line-overview.html&quot; rel=&quot;noreferrer&quot;&gt;Picard-tools&lt;/a&gt; (Picard.SamToFastq), but as usual I am avoiding java as much as I can. I guess&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view foo.bam | grep -vf list_of_scaffols_filter \&#xA;  | java -jar picard.jar SamToFastq INPUT=/dev/stdin \&#xA;  FASTQ=foo_R1_filtered_bash.fq SECOND_END_FASTQ=foo_R2_filtered_bash.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The first solution does not really work for me since I do not want to rename all the reads in bam file and I want to keep the R1/R2 information (since R1 and R2 have different error profiles). Both solutions 2 and 3 I find bit clumsy and I am not sure if they are general, I might get some unexpected behaviour if one of reads is mapped second is not... They both relay on the same filtering step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering about some pysam solution. I guess it will be much slower, but at least it will be much clearer and perhaps more general. Something like in &lt;a href=&quot;https://www.biostars.org/p/6970/&quot; rel=&quot;noreferrer&quot;&gt;Convert Bam File To Fasta File&lt;/a&gt; - there is pysam solution for fasta (not fastq), almost there...&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Story&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I have very fragmented reference genome. Some of scaffolds them are too small to works with, and some of them are contaminants (identified using blobtools).&#xA;I want to separate reads that are mapping to different groups to separate contaminants, short scaffolds and scaffold that will be used for downstream analysis. The reason is that if we remap all the reads to filtered reference (0.7 - 0.8 of original genome), the most of them (0.95 - 0.99) will still find a place where they map, therefore there is 0.2 - 0.3 of misplaced reads that will obviously have to affect downstream analysis, like variant calling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This filtering idea is based logic that if the filtered duplicated genomic region will contain some small differences, they will attract their reads (and if I filter them I will improve variant calling) and if they will be exactly same, they will get reads assigned at random, so there is no harm in doing that.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2017-07-26T13:17:23.220" Title="How to safely and efficiently convert subset of bam to fastq?" Tags="|sam|fastq|format-conversion|filtering|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2150" PostTypeId="2" ParentId="2149" CreationDate="2017-07-26T13:13:32.230" Score="5" Body="&lt;p&gt;I'm not aware of any pre-made program to do this, so I &lt;a href=&quot;https://github.com/dpryan79/Answers/tree/master/bioinfoSE_2149&quot; rel=&quot;noreferrer&quot;&gt;wrote one for you&lt;/a&gt;. This will take a BAM file with any ordering and produce properly ordered gzipped fastq files with the filtering as you requested. Internally, this iterates over all of the entries in the BAM file (ignoring secondary/supplemental entries and those where both mates map to your filter list), store the properly oriented sequence/quality/read name in a buffer, and then dumps that buffer entry to disk once the mate is found. This should be reasonably performant (hey, it's python, to don't expect too much), though if you happen to have indexed BAM files then one could think of ways to make this run faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do check the output, since I've only run one test.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-07-26T13:13:32.230" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2151" PostTypeId="2" ParentId="2149" CreationDate="2017-07-26T13:17:23.220" Score="2" Body="&lt;p&gt;Ok, I wrote a bit brute pysam / BioPython parser that uses index of bam to get proper order of read pair for R1 / R2 files and bitwise flag. It should not be too difficult to add more sophisticated filtering rules now.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;# 1. arg - indexed bam file&#xA;# 2. arg - list of headers to filter&#xA;# 3. arg - name pattern for the output reads&#xA;&#xA;import os&#xA;import sys&#xA;import pysam&#xA;from Bio import SeqIO, Seq, SeqRecord&#xA;&#xA;samfile = pysam.AlignmentFile(sys.argv[1], &quot;rb&quot;)&#xA;header_set = set(line.strip() for line in open(sys.argv[2]))&#xA;base = sys.argv[3]&#xA;&#xA;out_R1 = base + 'R1_filtered.fq'&#xA;out_R2 = base + 'R2_filtered.fq'&#xA;&#xA;with open(out_R1, mode='w') as R1, open(out_R2, mode='w') as R2:&#xA;    for entry in samfile:&#xA;        if entry.is_read1 and not entry.reference_name in header_set:&#xA;            # pait pair&#xA;            entry_R2 = samfile.mate(entry)&#xA;            # do some sequence gymnastics with R1&#xA;            seq_R1 = Seq.Seq(entry.seq)&#xA;            if entry.is_reverse :&#xA;                seq_R1 = seq_R1.reverse_complement()&#xA;            # do some sequence gymnastics with R2&#xA;            seq_R2 = Seq.Seq(entry_R2.seq)&#xA;            if entry_R2.is_reverse :&#xA;                seq_R2 = seq_R2.reverse_complement()&#xA;            R1.write('@' + entry.qname + '\n' + str(seq_R1) + '\n+\n' + entry.qqual + '\n')&#xA;            R2.write('@' + entry_R2.qname + '\n' + str(seq_R2) + '\n+\n' + entry_R2.qqual + '\n')&#xA;&#xA;&#xA;samfile.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are some ugly bits, please feel free to make it nicer.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2017-07-26T13:17:23.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2152" PostTypeId="1" AcceptedAnswerId="2162" CreationDate="2017-07-26T13:35:42.953" Score="8" ViewCount="3720" Body="&lt;p&gt;Let $C$ be base coverage, $R$ is the length of reads and $K$ is the length of $k$-mer. Then $k$-mer coverage $C_k$ can be computed as $C_k = C\cdot(R - K + 1)/R$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone please explain why is this equation valid (I'm mostly confused as why it is divided by $R$)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;http://www.ebi.ac.uk/~zerbino/velvet/Manual.pdf&quot; rel=&quot;noreferrer&quot;&gt;Velvet manual&lt;/a&gt; Section 5.1&lt;/p&gt;&#xA;" OwnerUserId="1199" LastEditorUserId="29" LastEditDate="2017-07-26T16:32:17.023" LastActivityDate="2017-07-27T10:54:57.303" Title="Formula for k-mer coverage" Tags="|k-mer|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2153" PostTypeId="1" AcceptedAnswerId="2154" CreationDate="2017-07-26T15:20:57.187" Score="6" ViewCount="97" Body="&lt;p&gt;I am interested in finding a database that takes a gene or protein name as input (possibly with the option to specify transcript) and gives information about the protein's functional domains in terms of either specific residue ranges, genomic coordinates, etc. Ideally, the database would include 3D conformational information as well as the functional domains. Does such a database exist? It seems that this information can be inferred for a given protein using various publications and references but I have yet to find a single collection containing these data all in one place.&lt;/p&gt;&#xA;" OwnerUserId="1201" LastEditorUserId="203" LastEditDate="2017-07-27T09:11:20.577" LastActivityDate="2019-05-28T09:30:22.383" Title="Do any publicly available databases detail protein structure and functional domains?" Tags="|database|gene|proteins|protein-structure|rna-structure|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2154" PostTypeId="2" ParentId="2153" CreationDate="2017-07-26T16:24:03.780" Score="6" Body="&lt;p&gt;Some of this information (at least some domains, active sites, etc) is available from &lt;a href=&quot;http://www.uniprot.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;UniProt&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to download their whole database, you can search without specifying any terms and then click the Download button.&lt;/p&gt;&#xA;" OwnerUserId="306" LastEditorUserId="29" LastEditDate="2017-07-26T17:12:17.537" LastActivityDate="2017-07-26T17:12:17.537" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2156" PostTypeId="1" AcceptedAnswerId="2158" CreationDate="2017-07-26T17:21:23.227" Score="5" ViewCount="1497" Body="&lt;p&gt;I have been analyzing some virus DNA from the NCBI databases. The fasta sequences that I recieve from them have header lines that look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;gi|61393989|gb|AY848686.1|&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think that the second number is the GenBank ID, but I have several questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) What does the &quot;gi&quot; in the first ID stand for?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Is this a standard format for storing a series of IDs associated with sequences? That is, should I expect to see this format elsewhere and be prepared to parse fasta headers like this in code that I write?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) It seems that &lt;code&gt;AY848686&lt;/code&gt; is also a valid GenBank ID, and I see the &lt;code&gt;.1&lt;/code&gt; missing/present in GenBank IDs in various places. What does the &lt;code&gt;.1&lt;/code&gt; mean at the end of GenBank IDs and why is it there/does it &lt;em&gt;need&lt;/em&gt; to be there?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for you answers.&lt;/p&gt;&#xA;" OwnerUserId="1196" LastActivityDate="2017-07-26T17:33:05.717" Title="Fasta Sequence Identifier format?" Tags="|fasta|identifiers|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2157" PostTypeId="2" ParentId="2153" CreationDate="2017-07-26T17:23:16.920" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://www.ensembl.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;EnsEMBL&lt;/a&gt; also has this. Search for your gene of interest, choose your transcript, go to the page of its protein product(s), and select &quot;Domains &amp;amp; Features&quot; from the right-hand menu (using human p53 as an example):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Domain source       Start End  Description                                            Accession InterPro  &#xA;PANTHER             3     331  -                                                      PTHR11447:SF6-         &#xA;Pfam                5     29   p53 transactivation domain                             PF08563   IPR013872 &#xA;PANTHER             3     331  p53 tumour suppressor family                           PTHR11447 IPR002117 &#xA;PRINTS              116   142  p53 tumour suppressor family                           PR00386   IPR002117 &#xA;PRINTS              158   179  p53 tumour suppressor family                           PR00386   IPR002117 &#xA;PRINTS              213   234  p53 tumour suppressor family                           PR00386   IPR002117 &#xA;PRINTS              236   258  p53 tumour suppressor family                           PR00386   IPR002117 &#xA;Prosite_patterns    237   249  p53 tumour suppressor family                           PS00348   IPR002117 &#xA;PRINTS              264   286  p53 tumour suppressor family                           PR00386   IPR002117 &#xA;Pfam                95    288  p53 DNA-binding domain                                 PF00870   IPR011615 &#xA;Superfamily         97    287  p53-like transcription factor DNA-binding              SSF49417  IPR008967 &#xA;Gene3D              94    297  p53/RUNT-type transcription factor DNA-binding domain  2.60.40.720IPR012346 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-07-26T17:23:16.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2158" PostTypeId="2" ParentId="2156" CreationDate="2017-07-26T17:25:01.930" Score="5" Body="&lt;ol&gt;&#xA;&lt;li&gt;The &lt;code&gt;gi&lt;/code&gt; is an abbreviation for &quot;Genbank identifier&quot;.&lt;/li&gt;&#xA;&lt;li&gt;This is a pretty standard convention used by data stored in NCBI databases. There used to be a pretty comprehensive description of the conventions used at NCBI (I wouldn't say it was a standard or specification, just convention) &lt;a href=&quot;ftp://ftp.ncbi.nih.gov/blast/documents/formatdb.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, but this page is no longer available it seems.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;code&gt;AY848686&lt;/code&gt; label is an accession value, and the appended number is a version number.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Sitemap/sequenceIDs.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt; for more info.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The short answer is: these are all internal conventions. None of it &lt;em&gt;needs&lt;/em&gt; to be there, except perhaps to ensure NCBI's internal tools work as expected.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;: I found a non-official copy of the old docs &lt;a href=&quot;http://structure.usc.edu/blast/formatdb.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Look for the section starting with &quot;FASTA Defline Format&quot;. I've copied it here to prevent further problems with bitrot in the future.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FASTA Defline Format&#xA;--------------------&#xA;The syntax of FASTA Deflines used by the NCBI BLAST server depends on&#xA;the database from which each sequence was obtained.  The table below lists&#xA;the identifiers for the databases from which the sequences were derived.&#xA;&#xA;&#xA;  Database Name                         Identifier Syntax&#xA;&#xA;  GenBank                               gb|accession|locus&#xA;  EMBL Data Library                     emb|accession|locus&#xA;  DDBJ, DNA Database of Japan           dbj|accession|locus&#xA;  NBRF PIR                              pir||entry&#xA;  Protein Research Foundation           prf||name&#xA;  SWISS-PROT                            sp|accession|entry name&#xA;  Brookhaven Protein Data Bank          pdb|entry|chain&#xA;  Patents                               pat|country|number &#xA;  GenInfo Backbone Id                   bbs|number &#xA;  General database identifier           gnl|database|identifier&#xA;  NCBI Reference Sequence               ref|accession|locus&#xA;  Local Sequence identifier             lcl|identifier&#xA;&#xA;&#xA;For example, an identifier might be &quot;gb|M73307|AGMA13GT&quot;, where the &quot;gb&quot; tag&#xA;indicates that the identifier refers to a GenBank sequence, &quot;M73307&quot; is its&#xA;GenBank ACCESSION, and &quot;AGMA13GT&quot; is the GenBank LOCUS.  &#xA;&#xA;&quot;gi&quot; identifiers are being assigned by NCBI for all sequences contained&#xA;within NCBI's sequence databases.  The 'gi' identifier provides a uniform&#xA;and stable naming convention whereby a specific sequence is assigned&#xA;its unique gi identifier.  If a nucleotide or protein sequence changes,&#xA;however, a new gi identifier is assigned, even if the accession number&#xA;of the record remains unchanged. Thus gi identifiers provide a mechanism&#xA;for identifying the exact sequence that was used or retrieved in a&#xA;given search.&#xA;&#xA;For searches of the nr protein database where the sequences are derived&#xA;from conceptual translations of sequences from the nucleotide databases&#xA;the following syntax is used:&#xA;&#xA;                     gi|gi_identifier&#xA;&#xA;An example would be:&#xA;&#xA;        gi|451623           (U04987) env [Simian immunodeficiency...&quot;&#xA;&#xA;where '451623' is the gi identifier and the 'U04987' is the accession&#xA;number of the nucleotide sequence from which it was derived.&#xA;&#xA;Users are encouraged to use the '-I' option for Blast output which will&#xA;produce a header line with the gi identifier concatenated with the database&#xA;identifier of the database from which it was derived, for example, from a&#xA;nucleotide database:&#xA;&#xA;        gi|176485|gb|M73307|AGMA13GT&#xA;&#xA;And similarly for protein databases: &#xA;&#xA;        gi|129295|sp|P01013|OVAX_CHICK&#xA;&#xA;The gnl ('general') identifier allows databases not on the above list to&#xA;be identified with the same syntax.  An example here is the PID identifier:&#xA;&#xA;        gnl|PID|e1632&#xA;&#xA;PID stands for Protein-ID; the &quot;e&quot; (in e1632) indicates that this ID&#xA;was issued by EMBL.  As mentioned above, use of the &quot;-I&quot; option&#xA;produces the NCBI gi (in addition to the PID) which users can also&#xA;retrieve on.&#xA;&#xA;The bar (&quot;|&quot;) separates different fields as listed in the above table.&#xA;In some cases a field is left empty, even though the original&#xA;specification called for including this field.  To make&#xA;these identifiers backwards-compatiable for older parsers&#xA;the empty field is denoted by an additional bar (&quot;||&quot;).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2017-07-26T17:33:05.717" LastActivityDate="2017-07-26T17:33:05.717" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2159" PostTypeId="2" ParentId="2156" CreationDate="2017-07-26T17:31:21.397" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;That is a GenBank gene ID and it (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Sitemap/sequenceIDs.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;apparently&lt;/a&gt;) stands for &amp;quot;GenInfo Identifier&amp;quot;. The full GenBank ID is &lt;code&gt;gi|61393989&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Yes, this is the standard format for sequences provided by NCBI. However, you can have all sorts of extra details appended to that. They (NCBI) generally use &lt;code&gt;|&lt;/code&gt; as a field separator, so you could try to write code that parse that, but I really wouldn't recommend it. The FASTA headers have traditionally been free. People can put anything they like there apart from a &lt;code&gt;\n&lt;/code&gt;, so I really wouldn't put any effort into writing code to parse these headers.&lt;/p&gt;&#xA;&lt;p&gt;If you need a short name for the sequence, just take the first N characters or, if it matches &lt;code&gt;^..|.+?|&lt;/code&gt;, take the characters until the second &lt;code&gt;|&lt;/code&gt;. Seriously though, don't bother parsing this it is very rarely helpful and is almost certain to break as soon as you use a different database.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The AY848686.1 is a GenBank accession. The &lt;code&gt;gb&lt;/code&gt; indicates that it's a GenBank accession and the &lt;code&gt;.1&lt;/code&gt; means it is the 1st version of that accession.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;As for why there are two (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Sitemap/sequenceIDs.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Unlike the gi number system, in which sequence identification numbers were not necessarily consistent across the databases (e.g., GenBank and EMBL could each assign their own gi number to a sequence), the new system is designed to ensure consistency. It is also designed to show a relationship between a sequence identification number and the accession number of the record in which it is found. In contrast, GI numbers are assigned consecutively and bear no resemblance to the accession number. Finally, the new system allows the assignment of alphanumeric protein IDs to proteins translations within nucleotide sequence records. The protein IDs contain three letters followed by five digits, a period, and a version number.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="298" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-07-26T17:31:21.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2160" PostTypeId="2" ParentId="2142" CreationDate="2017-07-26T17:39:51.333" Score="5" Body="&lt;p&gt;Based on my experience reviewing k-mer counting software and as a core contributor to the khmer project, I can confidently say that there is no widely used standard format.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2017-07-26T17:39:51.333" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2161" PostTypeId="2" ParentId="935" CreationDate="2017-07-26T17:47:55.083" Score="1" Body="&lt;p&gt;if the data is in SRA, there is sra-stat utility that returns reads,bases and quality distribution. these are stored in the SRA file. use --quick to get the stored stats or --statistics to calculate additional values broken down per readgroup/barcode.&#xA;sra-stat --quick SRR077487&lt;/p&gt;&#xA;" OwnerUserId="1203" LastActivityDate="2017-07-26T17:47:55.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2162" PostTypeId="2" ParentId="2152" CreationDate="2017-07-26T20:45:29.493" Score="3" Body="&lt;p&gt;$C_k$ is defined as the number of reads containing a k-mer. The fraction of a read available to contain a k-mer is $(R-K+1)/R$, which is the number of k-mers in the read divided by its length. That times the nucleotide coverage ($C$) is then the expected coverage of the k-mer.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-07-26T20:45:29.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2163" PostTypeId="1" CreationDate="2017-07-26T20:50:10.633" Score="7" ViewCount="633" Body="&lt;p&gt;I have a code below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def FilterReads(in_file, out_file):&#xA;&#xA;def read_ok(read):&#xA;    &quot;&quot;&quot;&#xA;    read_ok - reject reads with a low quality (&amp;lt;5) base call&#xA;    read - a PySam AlignedRead object&#xA;    returns: True if the read is ok&#xA;    &quot;&quot;&quot;&#xA;    if any([ord(c)-33 &amp;lt; _BASE_QUAL_CUTOFF for c in list(read.qual)]):&#xA;        return False&#xA;    else:&#xA;        return True&#xA;&#xA;_BASE_QUAL_CUTOFF = 30&#xA;&#xA;bam_in = pysam.Samfile(in_file, 'rb')&#xA;bam_out = pysam.Samfile(out_file, 'wb', template=bam_in)&#xA;&#xA;&#xA;for read in bam_in.fetch():&#xA;    if read_ok(read):&#xA;        bam_out.write(read)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This code works fine by &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Rejecting reads having a base with phred quality score below five&lt;/li&gt;&#xA;&lt;li&gt;But it first takes a BAM file &lt;/li&gt;&#xA;&lt;li&gt;And then creates a filtered BAM file for further analysis but this takes a lot of time. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So is there a way to reject these reads using pileup in pysam so that I may not have to create a file then sort it and again read it? &#xA;Or can I modify &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2131/filtering-bases-based-on-phred-qualities-with-pysam&quot;&gt;this code&lt;/a&gt; to perform the same function&lt;/p&gt;&#xA;" OwnerUserId="1194" LastActivityDate="2018-11-24T18:25:05.413" Title="Reject reads with low quality bases from a Bam file through pysam" Tags="|sam|ngs|sequence-alignment|python|pysam|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2164" PostTypeId="1" AcceptedAnswerId="2167" CreationDate="2017-07-26T22:03:37.427" Score="5" ViewCount="1375" Body="&lt;p&gt;I'm attempting to visualize the results of my BLAST search in a way similar to the graphical display of the distribution of blast hits from the web BLAST.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, from my BLAST search:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zVWjr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zVWjr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;except I would like the graph to contain information including information for the distribution of all 6139 hits across my query sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this is a large number of sequences to visualize a distribution, I am considering using a score of the number of hits on a certain region, and having an output more similar to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HhPYg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HhPYg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I were to run my blast using the BLAST+ command line tool, and read the results into R, which parts of the output should I plot in order to recreate the graphical display of the distribution? Ideally I would like to make a reusable object that can make an equivalent plot for any BLAST input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="727" LastEditorUserId="96" LastEditDate="2017-07-27T03:45:52.203" LastActivityDate="2017-07-28T19:18:37.343" Title="creating graph of distribution of blast hits on the query sequence" Tags="|r|blast|visualization|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2165" PostTypeId="1" AcceptedAnswerId="2166" CreationDate="2017-07-27T02:14:57.477" Score="8" ViewCount="418" Body="&lt;p&gt;I have RNA seq data which I've quantified using Kallisto.  I'd like to use tximport to transform the read count data into input for EdgeR, following the R code supplied in the tximport documentation:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;cts &amp;lt;- txi$counts&#xA;normMat &amp;lt;- txi$length&#xA;normMat &amp;lt;- normMat/exp(rowMeans(log(normMat)))&#xA;library(edgeR)&#xA;o &amp;lt;- log(calcNormFactors(cts/normMat)) + log(colSums(cts/normMat))&#xA;y &amp;lt;- DGEList(cts)&#xA;y$offset &amp;lt;- t(t(log(normMat)) + o)&#xA;# y is now ready for estimate dispersion functions see edgeR User's Guide&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would ideally follow this up with EdgeR's exactTest.  However, I'd also like to remove genes with low read counts using code such as this:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;filterCounts &amp;lt;- function(counts) {&#xA;  cpms &amp;lt;- cpm(counts)&#xA;  keep &amp;lt;- rowSums(cpms &amp;gt; 1) &amp;gt;= 1&#xA;  counts[keep,]&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The only issue is that I can't tell whether I should remove the genes with low read counts before or after I normality the counts while importing with tximport in the first code.  I'm thinking that it would be best to do this before, editing out the abundance, counts, and length rows of the txi data frame which correspond to lowly expressed genes, but I would like this to be confirmed before I proceed.&lt;/p&gt;&#xA;" OwnerUserId="506" LastEditorUserId="203" LastEditDate="2017-07-28T11:18:40.363" LastActivityDate="2017-07-28T11:18:40.363" Title="When performing differential expression analysis, should genes with low read counts be removed before or after normalization?" Tags="|rna-seq|statistics|edger|best-practice|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2166" PostTypeId="2" ParentId="2165" CreationDate="2017-07-27T06:37:14.703" Score="6" Body="&lt;p&gt;The more genes you have the more robust the scaling factor is (among the reason why one doesn't normalize to ERCC spike-ins without a compelling reason), so I suppose in theory it's better to filter after determining the scale factor. Having said that, I'd be surprised if the results changed much either way. Unless you end up filtering out a LOT of genes the scaling factors should be fairly robust to the change.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-07-27T06:37:14.703" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2167" PostTypeId="2" ParentId="2164" CreationDate="2017-07-27T08:55:09.647" Score="4" Body="&lt;p&gt;You probably want to include query start (qstart) and query end (qend) in your blast output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blastn -query your.fasta -out blast.out.txt -db your.db -outfmt '6 qseqid sseqid qstart qend length evalue'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In R you can take the &quot;qstart:qend&quot; from each line for density plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many ways in R to plot the densities of these start and end amino acids.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me show an example with a small data frame:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;qstart &amp;lt;- c(200, 300, 250, 400, 500)&#xA;qend &amp;lt;- c(300, 450, 400, 600, 650)&#xA;&#xA;df &amp;lt;- as.data.frame(cbind(qstart,qend))&#xA;&#xA;aa &amp;lt;- vector()&#xA;i=1&#xA;for(i in 1:5){&#xA;aa &amp;lt;- append(aa, c(df[i,1]:df[i,2]))&#xA;i+1&#xA;}&#xA;&#xA;hist(aa)&#xA;dens &amp;lt;- density(aa)&#xA;plot(dens)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2017-07-28T19:18:37.343" LastActivityDate="2017-07-28T19:18:37.343" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2168" PostTypeId="2" ParentId="2152" CreationDate="2017-07-27T10:02:08.870" Score="5" Body="&lt;p&gt;I was still puzzled from the answers, so I tried to calculate with all the steps. I take this definition &quot;$C_k$ is the number of reads containing a k-mer.&quot; and corresponding definition for coverage ($C$): &quot;$C$ is the number of reads covering a base&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coverage is $C = \frac{T \cdot R}{L}$, where $T$ is total number of reads, $R$ is read length and $L$ is length of genome. Given the $C_k$ definition, $C_k = \frac{T (R - K + 1)}{L-K+1}$, where $R - K + 1$ is just number of kmers in a read, and $L-K+1$ is number of kmers in a genome. Then,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$C_k = \frac{T (R - K + 1)}{L-K+1} = \frac{T (R - K + 1)}{L-K+1} \cdot \frac{R}{R} = \frac{R - K + 1}{R} \cdot \frac{T \cdot R}{L - K + 1}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;since $L &amp;gt;&amp;gt; K$, we can approximate $L - K + 1 \approx L$, then we reduce the expression to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{R - K + 1}{R} \cdot \frac{T \cdot R}{L} = \frac{R - K + 1}{R} \cdot C$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is the formula for $C_k$.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2017-07-27T10:54:57.303" LastActivityDate="2017-07-27T10:54:57.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2169" PostTypeId="1" AcceptedAnswerId="2171" CreationDate="2017-07-27T10:52:01.277" Score="3" ViewCount="128" Body="&lt;p&gt;I am a student trying to analyze GEO2r datas for one of my courses.&#xA;The IDs given in the output are different for different series. I need to convert all of them to a similar format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this process I encountered the following type of ID which I don't know its origin:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;”ID&quot;    &quot;logFC&quot;&#xA;&quot;SP_v2 4634&quot;    &quot;-0.9897758&quot;&#xA;&quot;SP_v2 3382&quot;    &quot;-0.8391782&quot;&#xA;&quot;SP_v2 4210&quot;    &quot;-1.1693583&quot;&#xA;&quot;SP_v2 2117&quot;    &quot;-1.0504727&quot;&#xA;&quot;SP_v2 3488&quot;    &quot;-0.9756444&quot;&#xA;&quot;SP_v2 1128&quot;    &quot;-0.8289103&quot;&#xA;&quot;SP_v2 2735&quot;    &quot;-0.8629999&quot;&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Each one of the rows represent a single gene.&#xA;My question is that what is this ID? The GEO accession is GSE97750.&lt;/p&gt;&#xA;" OwnerUserId="872" LastEditorUserId="298" LastEditDate="2017-07-27T13:03:53.463" LastActivityDate="2017-07-27T13:03:53.463" Title="What database do gene IDs starting with sp_v2 refer to?" Tags="|identifiers|" AnswerCount="1" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2170" PostTypeId="1" AcceptedAnswerId="2172" CreationDate="2017-07-27T12:21:49.380" Score="5" ViewCount="193" Body="&lt;p&gt;Is there any extensive documentation or description for each class of Transfrag class codes as reported by the cuffcompare tool in the Cufflinks package?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/cuffcompare/#transfrag-class-codes&quot; rel=&quot;noreferrer&quot;&gt;Official doc&lt;/a&gt; might not be the best. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g. what does &lt;em&gt;contained&lt;/em&gt; mean (class code: c)? Or, what is a &lt;em&gt;generic exonic overlap&lt;/em&gt; (class code: o)?&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2017-07-27T12:47:53.153" Title="Documentation for Transfrag class codes (cuffcompare)" Tags="|transcriptome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2171" PostTypeId="2" ParentId="2169" CreationDate="2017-07-27T12:27:29.010" Score="5" Body="&lt;p&gt;This is possibly a wild goose chase, but a lot of searching led me to this sample: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM2536852&quot; rel=&quot;noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM2536852&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which appears to exhibit accessions of the right format. This in turn leads to the platform:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GPL22166&quot; rel=&quot;noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GPL22166&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is a custom spotted cDNA array, which the metadata seems to suggest is a human platform.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The full annotation table gives gene symbols and Entrez IDs for most of the probes on the array:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?view=data&amp;amp;acc=GPL22166&amp;amp;id=53508&amp;amp;db=GeoDb_blob144&quot; rel=&quot;noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?view=data&amp;amp;acc=GPL22166&amp;amp;id=53508&amp;amp;db=GeoDb_blob144&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To take the example of the first line of your results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ID          ORF     Entrez gene SEQUENCE&#xA;SP_v2 4634  ETNK1   55500       AAAGCAGCTTCATCTTTCAAAATTGATTTGCTCTGGTTTT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The Entrez Gene record for that ID matches up:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene/?term=55500&quot; rel=&quot;noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/gene/?term=55500&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="194" LastActivityDate="2017-07-27T12:27:29.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2172" PostTypeId="2" ParentId="2170" CreationDate="2017-07-27T12:47:53.153" Score="2" Body="&lt;p&gt;You can get a bit more information by &lt;a href=&quot;https://github.com/cole-trapnell-lab/cufflinks/blob/b55bb214ad4abeb58a1435d8f7a7f0dd8bed8b76/src/gff_utils.cpp&quot; rel=&quot;nofollow noreferrer&quot;&gt;looking at the source code comments&lt;/a&gt;. &lt;a href=&quot;https://github.com/cole-trapnell-lab/cufflinks/blob/b55bb214ad4abeb58a1435d8f7a7f0dd8bed8b76/src/gff_utils.cpp#L140-L141&quot; rel=&quot;nofollow noreferrer&quot;&gt;Contained&lt;/a&gt; is for single exon transcripts that are primarily contained (&lt;a href=&quot;https://github.com/cole-trapnell-lab/cufflinks/blob/b55bb214ad4abeb58a1435d8f7a7f0dd8bed8b76/src/gff_utils.cpp#L145&quot; rel=&quot;nofollow noreferrer&quot;&gt;&gt;80% overlap&lt;/a&gt;) within an exon of an already annotated transcript. &lt;code&gt;Generic exonic overlap&lt;/code&gt; is for cases where a transfrag exon overlaps that of an annotated transcript. The strand doesn't seem to matter in such cases.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-07-27T12:47:53.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2174" PostTypeId="1" AcceptedAnswerId="2175" CreationDate="2017-07-28T08:30:21.540" Score="7" ViewCount="1375" Body="&lt;p&gt;I have a fastq file from minION (albacore) that contains information on the read ID and the start time of the read. I want to extract these two bits of information into a single csv file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been trying to figure out a grep/awk/sed solution, but without success. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@93a12f52-95e5-40c7-8c3e-70bf94ed0720 runid=17838b1d08f30a031bf60afabb146a8b0fba7486 read=12217 ch=492 start_time=2017-07-04T06:42:43Z&#xA;CTATTGTCCCCTGCCGCTGCCCCTCCTGCTACGCCCCACTGCCTCACCAGCCGTTACGGTCGCCCCCCATCGCATGCCTTTACACACACACTTCTTTACACATGCTATCTTCCC&#xA;+&#xA;&quot;&quot;*)&amp;amp;$-.,-(#&quot;&amp;amp;'$%''+16#&quot;$##&amp;amp;)%%/&quot;+*(*(&amp;amp;#&quot;&amp;amp;'%1&quot;+)#)&quot;&quot;%$$#&amp;amp;&amp;amp;'1%&quot;'8&amp;gt;MJ&amp;lt;#'&amp;amp;%'.2'.$(&amp;amp;#'()'&amp;amp;&amp;amp;%'$('%&quot;%%%..$#&quot;&amp;amp;&quot;#+&amp;amp;,*$%&quot;#&quot;&#xA;@ff37e422-a25f-404c-8314-ef1733f9c30c runid=17838b1d08f30a031bf60afabb146a8b0fba7486 read=8432 ch=200 start_time=2017-07-04T06:56:41Z&#xA;CGATGGCCGTATGCTTTTGTTATGAAGCGAAAAGCTGCTCGCTTCTCTAGATAATAATGATGTGGCGAAAACGCTATGCGATTCGTTGACATACATGATGGCGGATTTATCTACCACTTTGTGGCATGCTTTTCTCGCCAGATAATGGAATGTTTCTCTGGCGGTAATGGATAGTATCAAATCTCACTAGCCCATTCTATAAGCGCATCCGCATGCACTAGTTCTTGATTCGATCGTCCTCTAGCATGTTCGAAGAAGATAGCATTCACTATCATCATCGCCTCAGGTAAGTTTATTCGGTTGGGGCGTGTGAAGGCAAACACCTTGTTGTCCAGTAAGTTTTCAGTTACTATAACTTAAAGTCGCACATGAATCTAGTCTCCTATTCCCCACCCATGATCCCACTCACACATTTCTACAGAGATGTGGTTAGAAATTTTCATATTAGGTCAGCTTTGACTCAATAAGACATAATTCTTCACTGAATGACTTTTTAAGAACCACCAGGACCAGAGAGAACCAAGAGAGTGGTACCTCTTAAAACACAATAAAGTGATTCAGCCTTAGCCATTGGATTCTGGAGGACCTTGAACCATGTGGGAAGCAGCTCAGGGTGGCCATGTACTATACTGGCGGGTAAGCTTCTGGAGTGCTAGGTTCTTTTTGTCTTTTCTTAAGCATTGCCGCCAGTTGATTGGGTTTTGAACATAAAATAATGCGCCACCAGCAATTCCAGATTTGTTCCTACGGGATAGATTTGTTCAGTTCTAGCATTATGCTTCACTAACCAGATGCGGGCCCTAAGTCCTTCACTTGGAATATTGGATTGGATCATGAGAATATTCTGTCTGAAGCTCGTCATTAATTTTGTTACAAAATAGAGCTTTTTGACTGGAAGTACCACCATACGTGTTCTCAAACTTCAGCATTTTTAGAACTTCCCACGGCATCTTGACCCTTTTCACAGCATGGATAGTCAGGCAGCAGTGAACTTTGTGACTCTTTAATGCCTTCACTTTTCTCTCAGTTTCCCCGCCTTGCGTTATCTTTACTCGTCTTGGGACTTTTATCCCAATGCCAGCCTTCTACCCTGAGACCTCAGTGGGTCATCATCCCAGCCCGGGACATCTCATCCCATCATTTATGGGCTGTTGTGTTTTTTTCAAAACCTAGCCCTCTCAGGAGGAGGAGGAGTGGGAGTCAGTTCAGTGAGGAGGATTAGGATGATCTGAAATGTAAGCACATATAAGCGAAGCACTTATTTTGGGTTGGGTCCTCACGGTGGACATAAGATCGCCTTATGTGTTTAGTAAGCCATTCCTAGCTCTCAATGGCGTGATTACATAGAAGCGTGAGGGATCAGTCCTATGGAAGACTAGGAAGTAAATGAACAAAATATATTAACCATAGAAGTCTCATGGGTCGCTGTAGCCAAAAGATTAACACTTTTGACTACATTGTGGTTTTAGGCATTGAAACAAAAACTTTGAGTCTCCTAAACAAATGAATGGAAAATAGTAGCGAACTTCGATTCCTAACATTAAATCTAGAAATAGCAAGTTAGTTTAAAGACTTTATTTAGCTTTGCTTGCTATAATGAAAACCTTGCCTCCCGGTCGGGGCCATTGTGCCTGAAGCTAGCTTATTGTCTCCTCGAGCTCCCAGCTTCAGCAACTCCTTTTGAAGCGTTTGTCTCAGCTTGGATCTTCAGCAGCTCTTGGTGGCTCTTTTGAGCTAGCTCCTCTGAGATCTTGTATTTGGTAGGTCGCTTAGTCATAGTACTTTTCTTTTAACACCCTTCAGCTCTACGATTACATTTGGTTTTGTGGATATCATAATGATTGATGTGAAGATACATTGTACATGTG&#xA;+&#xA;&quot;#&quot;&quot;#&quot;##&quot;'&quot;&amp;amp;%&amp;amp;?JP7+80)'&amp;amp;*+&amp;amp;&amp;amp;(,.&amp;gt;3&amp;amp;*(#%(')*&amp;amp;'*#(-$)&quot;&amp;amp;,63;?844&amp;amp;&amp;amp;#$'3+.&amp;gt;@9;-/259...&amp;amp;:&quot;$&quot;/*&quot;#*#%(.&amp;amp;%)+,/76/+'4B93:;70.1)'1:)*#&amp;amp;()(+'03589/++&amp;amp;)'$-:;@,B5(9+.JAU;7-+&amp;amp;,1212.+329NPSDHRHH6),&amp;amp;)%)+).-*$,&amp;lt;1(.+&amp;amp;$$2&amp;amp;,-0+.&amp;amp;'//&quot;4(%&quot;+#,/)&amp;lt;56%*%*$&amp;lt;4*/;DI+6&amp;gt;((-%*).+--;2-'04-062&amp;lt;@-&amp;lt;&amp;gt;6CP)#.0/+,#(*2+#&quot;$$-2'5(%%&amp;amp;'5+)$$%8+0;&amp;gt;.@9*(,((,*-2/49G9/1/3./.0CQ93E+/-D-'67J*)($-2+'*/&amp;amp;*)+%&amp;amp;'092&amp;lt;C9/5()$&amp;amp;%(.,+-'**)'2/.(//6&amp;lt;59-5-)&quot;**40:A4+834)-(*+;K0-,&amp;amp;)+%)(#-&amp;amp;+&amp;amp;&amp;amp;))+$&amp;gt;9.6&amp;lt;?A;3'/85$/--..)/3DP99HJ9.&quot;2;I?@C/5%.4&amp;lt;0+0+&amp;amp;$-05C46208AFQE80&quot;%356+02*9I73//(-*/9706192*/)%)(&amp;amp;'($1749C*KKA*/#&quot;/*)5($'#+7@=$%)/&amp;amp;+%##*&amp;amp;&amp;amp;;.),#(4.,2/'-G03%&quot;&quot;#,(2)./,%%1;7#+%$'(03'5)+&amp;amp;.%1)&amp;amp;(%'#*,$3#''-3,),$&quot;),'+,,*$%'%$33ILADPPY@JW\O2-8'9[Y@--*.,&quot;&quot;$%$&quot;%#&quot;65',0/4(6.*43B1.#&amp;amp;)+),(6BI729=9&amp;amp;#$.)#,(,(/-286D&amp;amp;),'C;P=)'-'#$)*91*$%(.3LH31,($/(+)'$$*#+'((('*)00:7C1E24(%+$&amp;amp;-,$'++)&amp;lt;)'**&quot;&amp;amp;''&quot;(*(.14++$5&amp;amp;*,%?.F1&amp;lt;10&quot;&amp;amp;%$,)&amp;amp;&amp;amp;*(&amp;amp;$%$/.&quot;&quot;+3&quot;#&quot;#&quot;*%5*-&amp;gt;87FQ&amp;gt;AKY&amp;lt;9855,&amp;amp;(*@&amp;gt;HAZRQ)'%$&quot;--&amp;amp;/#%.+,/A9F8-1'+&quot;6--*34P-9;=997;7&amp;lt;SUU8((-+.:784/%).&amp;amp;)#*'(*/%7H(1JI-+&amp;amp;+)&amp;amp;9.,/N)&quot;%&quot;&amp;amp;))22/&amp;amp;,)%)'+*/17:8$''(*.*..(/.(%+*&amp;amp;22(+)7GM99A@9)0*1%/470((&amp;amp;((%(937'15084#,($&quot;&amp;amp;'%&quot;'4.,+5&amp;lt;7($*0+*291$*&amp;amp;$%%('+/)'.1-(-.7=K/'%&quot;'*)03&amp;lt;:/&amp;lt;=1(&amp;gt;.((-9-'+&amp;gt;&amp;amp;&amp;amp;&amp;amp;%';,)69V&amp;lt;'1:&amp;gt;FNXA+01:7*#&quot;$'$.%&amp;amp;4BQT7/&amp;amp;03++)'&amp;amp;'*3E.+0(*6..*&amp;lt;.)9*)))'%#(*.&amp;amp;&amp;amp;'&quot;*-IU=&amp;amp;,-))-.,-&amp;gt;'G--6..90067%(,0/*%9)&amp;amp;#'&quot;&quot;#$,5H/$/#-/&amp;amp;2(.0.+;&amp;gt;9--7.3@+2(/4.*0&amp;amp;,-&amp;amp;'/*%0-**-#,/2&amp;amp;-$*)'+#&quot;*)-*##&amp;amp;$3):C:D6H+',./2?&amp;lt;0&amp;gt;)%#&amp;amp;&amp;amp;3::9;8$#%#('$#$',,,*-346@=.%&amp;amp;&amp;amp;(&amp;amp;%('*+&amp;amp;*&amp;amp;1)'&amp;amp;-..#'+2*)/6499=-2*,0,82,$)($:2%*##%$,.5=-5/((#)2./('($@=4,)%1+176:AHQT,(,)0(()@2'+(18G?($)(*'.)&amp;amp;&amp;amp;+*12SD7,18I2)()*+C:01ETZIPDUOB+.55K7B:A5)*/('')/-1=7E&amp;lt;IGR=43@A68;K/:8.'--)BGEE5-0/5+,2''DXBLW7)#DHTC5AL82+--,&amp;amp;#);+).,'3)/.336/&amp;amp;'&amp;amp;(,+8E346-/%1-)1*$+),@-&amp;amp;-%'$+)5.&amp;amp;&amp;amp;%&amp;amp;#(,2&amp;amp;03&amp;amp;/((,(-5%%)?7I4*').+/7+&amp;amp;***845(*/.)(.0))&amp;lt;?YK3=GHH2&amp;amp;*)%$%#%KLY&amp;lt;9&amp;amp;&amp;amp;+,8;/2*./;+))-)/03$&amp;amp;*$+0'36&amp;lt;J+5/+&amp;amp;+(HGFFM57,59)-,0)/:**&quot;%$&amp;amp;#$'&amp;amp;$%#)*%33VX=@FOL00(''4IGIYOG0=1,,#(*$)-0CFM?8+31E8)(+)(%'&quot;'/&quot;&quot;%+3/,1;+7-*''&amp;amp;-%98.$$/'&amp;amp;)&amp;amp;+&amp;amp;(&amp;amp;&amp;amp;&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Should produce&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&#xA;93a12f52-95e5-40c7-8c3e-70bf94ed0720, 2017-07-04T06:42:43Z&#xA;ff37e422-a25f-404c-8314-ef1733f9c30c, 2017-07-04T06:56:41Z&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="156" LastEditorUserId="73" LastEditDate="2023-01-15T20:20:28.563" LastActivityDate="2023-01-15T20:20:28.563" Title="Extract nanopore read ID &amp; start times from fastq file" Tags="|fastq|nanopore|awk|" AnswerCount="3" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2175" PostTypeId="2" ParentId="2174" CreationDate="2017-07-28T08:37:41.497" Score="9" Body="&lt;pre&gt;&lt;code&gt;awk '{if(NR%4==1) print $1, $5}' file.fastq | sed -e &quot;s/ start_time=/, /&quot; -e &quot;s/^@//&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;awk&lt;/code&gt; command gets the first of every 4 lines, printing the first and fifth &quot;word&quot;. &lt;code&gt;sed&lt;/code&gt; is then used to strip the initial &lt;code&gt;@&lt;/code&gt; and replace &lt;code&gt;start_time=&lt;/code&gt; with &lt;code&gt;,&lt;/code&gt;. The output on your example file is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;93a12f52-95e5-40c7-8c3e-70bf94ed0720, 2017-07-04T06:42:43Z&#xA;ff37e422-a25f-404c-8314-ef1733f9c30c, 2017-07-04T06:56:41Z&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastEditorUserId="298" LastEditDate="2017-07-28T15:15:43.600" LastActivityDate="2017-07-28T15:15:43.600" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2176" PostTypeId="2" ParentId="2174" CreationDate="2017-07-28T15:10:39.743" Score="6" Body="&lt;p&gt;Since the string &lt;code&gt;start_time&lt;/code&gt; will only appear on the header line, or else you don't have a valid fastq file, you can simply do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ perl -ne '/^@(\S+).*start_time=(.*)/ &amp;amp;&amp;amp; print &quot;$1, $2\n&quot;' file.fastq &#xA;93a12f52-95e5-40c7-8c3e-70bf94ed0720,2017-07-04T06:42:43Z&#xA;ff37e422-a25f-404c-8314-ef1733f9c30c,2017-07-04T06:56:41Z&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Alternatively, since you mentioned &lt;code&gt;awk&lt;/code&gt; and &lt;code&gt;sed&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk -v OFS=&quot;, &quot; '/start_time/{print $1,$NF}' file.fastq | sed 's/start_time=//'&#xA;@93a12f52-95e5-40c7-8c3e-70bf94ed0720, 2017-07-04T06:42:43Z&#xA;@ff37e422-a25f-404c-8314-ef1733f9c30c, 2017-07-04T06:56:41Z&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, doing the whole thing in &lt;code&gt;awk&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk 'sub(/start_time=/,&quot;&quot;){print $1&quot;, &quot;$NF}' file.fastq &#xA;@93a12f52-95e5-40c7-8c3e-70bf94ed0720, 2017-07-04T06:42:43Z&#xA;@ff37e422-a25f-404c-8314-ef1733f9c30c, 2017-07-04T06:56:41Z&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And if the &lt;code&gt;@&lt;/code&gt; annoy you:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk 'sub(/^@/,&quot;&quot;) &amp;amp;&amp;amp; sub(/ .*start_time=/,&quot;, &quot;)' file.fastq &#xA;93a12f52-95e5-40c7-8c3e-70bf94ed0720, 2017-07-04T06:42:43Z&#xA;ff37e422-a25f-404c-8314-ef1733f9c30c, 2017-07-04T06:56:41Z&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And in &lt;code&gt;sed&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sed -n 's/^@\([^ ]*\).*start_time=\(.*\)/\1, \2/p' file.fastq &#xA;93a12f52-95e5-40c7-8c3e-70bf94ed0720, 2017-07-04T06:42:43Z&#xA;ff37e422-a25f-404c-8314-ef1733f9c30c, 2017-07-04T06:56:41Z&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, if your sed supports it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sed -En 's/^@(\S+).*start_time=(.*)/\1, \2/p' file.fastq &#xA;93a12f52-95e5-40c7-8c3e-70bf94ed0720, 2017-07-04T06:42:43Z&#xA;ff37e422-a25f-404c-8314-ef1733f9c30c, 2017-07-04T06:56:41Z&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, since &lt;code&gt;grep&lt;/code&gt; can't do replacements, to use it you would have to do something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -oP '^@\K\S+|start_time=\K.*' file.fastq | paste - - &#xA;93a12f52-95e5-40c7-8c3e-70bf94ed0720    2017-07-04T06:42:43Z&#xA;ff37e422-a25f-404c-8314-ef1733f9c30c    2017-07-04T06:56:41Z&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And to get the commas:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -oP '^@\K\S+|start_time=\K.*' file.fastq | paste - - | sed 's/\t/, /'&#xA;93a12f52-95e5-40c7-8c3e-70bf94ed0720, 2017-07-04T06:42:43Z&#xA;ff37e422-a25f-404c-8314-ef1733f9c30c, 2017-07-04T06:56:41Z&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-07-28T15:10:39.743" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2177" PostTypeId="1" AcceptedAnswerId="2178" CreationDate="2017-07-28T17:57:12.517" Score="8" ViewCount="1443" Body="&lt;p&gt;GFF3 files are in tabular format with 9 fields per line, separated by tabs. The first 8 fields share almost same data structure, but the 9th field varies a lot depending on feature type and gene prediction algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presently I am trying to build my own GFF module that will parse a GFF file and return one structured data object (a &lt;strong&gt;python dictionary structure&lt;/strong&gt;). For that, I need a good understanding of the 9th field of the GFF format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So can any one please help me by providing various GFF files?&lt;/p&gt;&#xA;" OwnerUserId="211" LastEditorUserId="96" LastEditDate="2019-07-24T15:23:28.437" LastActivityDate="2019-07-24T15:23:28.437" Title="How GFF3 attributes (9th column) varies from one gene prediction algorithm to another" Tags="|file-formats|python|gff3|sequence-annotation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2178" PostTypeId="2" ParentId="2177" CreationDate="2017-07-28T19:18:45.167" Score="8" Body="&lt;p&gt;The first place to start is the &lt;a href=&quot;https://github.com/The-Sequence-Ontology/Specifications/blob/master/gff3.md&quot; rel=&quot;noreferrer&quot;&gt;GFF3 specification&lt;/a&gt;. This is the official word on what is and is not allowed in a GFF3 file. For example, users can define arbitrary attribute keys, so long as they do not begin with an uppercase letter (these are reserved for &quot;official&quot; use).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But your question doesn't seem to be about what is &lt;em&gt;allowed&lt;/em&gt;, but what is &lt;em&gt;commonly used&lt;/em&gt;. I have a question for you: which gene predictors are YOU using? Or are you using gene annotations produced by others?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are a few examples.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;NCBI RefSeq&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If you're using GFF3 from NCBI, chances are it looks something like this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NC_007070.3     Gnomon  gene    329235  331223  .       +       .       ID=gene14;Dbxref=BEEBASE:GB42168,GeneID:551678;Name=LOC551678;gbkey=Gene;gene=LOC551678;gene_biotype=protein_coding&#xA;NC_007070.3     Gnomon  mRNA    329235  331223  .       +       .       ID=rna27;Parent=gene14;Dbxref=GeneID:551678,Genbank:XM_624067.4,BEEBASE:GB42168;Name=XM_624067.4;gbkey=mRNA;gene=LOC551678;model_evidence=Supporting evidence includes similarity to: 66 ESTs%2C 24 Proteins%2C and 99%25 coverage of the annotated genomic feature by RNAseq alignments%2C including 113 samples with support for all annotated introns;product=receptor expression-enhancing protein 5-like%2C transcript variant X1;transcript_id=XM_624067.4&#xA;NC_007070.3     Gnomon  exon    329235  329459  .       +       .       ID=id117;Parent=rna27;Dbxref=GeneID:551678,Genbank:XM_624067.4,BEEBASE:GB42168;gbkey=mRNA;gene=LOC551678;product=receptor expression-enhancing protein 5-like%2C transcript variant X1;transcript_id=XM_624067.4&#xA;NC_007070.3     Gnomon  exon    329850  330082  .       +       .       ID=id118;Parent=rna27;Dbxref=GeneID:551678,Genbank:XM_624067.4,BEEBASE:GB42168;gbkey=mRNA;gene=LOC551678;product=receptor expression-enhancing protein 5-like%2C transcript variant X1;transcript_id=XM_624067.4&#xA;NC_007070.3     Gnomon  exon    330166  330301  .       +       .       ID=id119;Parent=rna27;Dbxref=GeneID:551678,Genbank:XM_624067.4,BEEBASE:GB42168;gbkey=mRNA;gene=LOC551678;product=receptor expression-enhancing protein 5-like%2C transcript variant X1;transcript_id=XM_624067.4&#xA;NC_007070.3     Gnomon  exon    330376  331223  .       +       .       ID=id120;Parent=rna27;Dbxref=GeneID:551678,Genbank:XM_624067.4,BEEBASE:GB42168;gbkey=mRNA;gene=LOC551678;product=receptor expression-enhancing protein 5-like%2C transcript variant X1;transcript_id=XM_624067.4&#xA;NC_007070.3     Gnomon  CDS     329333  329459  .       +       0       ID=cds8;Parent=rna27;Dbxref=GeneID:551678,Genbank:XP_624070.1,BEEBASE:GB42168;Name=XP_624070.1;gbkey=CDS;gene=LOC551678;product=receptor expression-enhancing protein 5-like isoform X1;protein_id=XP_624070.1&#xA;NC_007070.3     Gnomon  CDS     329850  330082  .       +       2       ID=cds8;Parent=rna27;Dbxref=GeneID:551678,Genbank:XP_624070.1,BEEBASE:GB42168;Name=XP_624070.1;gbkey=CDS;gene=LOC551678;product=receptor expression-enhancing protein 5-like isoform X1;protein_id=XP_624070.1&#xA;NC_007070.3     Gnomon  CDS     330166  330301  .       +       0       ID=cds8;Parent=rna27;Dbxref=GeneID:551678,Genbank:XP_624070.1,BEEBASE:GB42168;Name=XP_624070.1;gbkey=CDS;gene=LOC551678;product=receptor expression-enhancing protein 5-like isoform X1;protein_id=XP_624070.1&#xA;NC_007070.3     Gnomon  CDS     330376  330416  .       +       2       ID=cds8;Parent=rna27;Dbxref=GeneID:551678,Genbank:XP_624070.1,BEEBASE:GB42168;Name=XP_624070.1;gbkey=CDS;gene=LOC551678;product=receptor expression-enhancing protein 5-like isoform X1;protein_id=XP_624070.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;MAKER&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The MAKER annotation workflow (&lt;a href=&quot;http://dx.doi.org/10.1101/gr.6743907&quot; rel=&quot;noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;http://www.yandell-lab.org/software/maker.html&quot; rel=&quot;noreferrer&quot;&gt;software&lt;/a&gt;) is a pretty commonly used gene annotation tool, and produces GFF3 output like this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;scaffold_12     maker   gene    652527  655343  .       +       .       ID=maker-scaffold_12-augustus-gene-0.959;Name=maker-scaffold_12-augustus-gene-0.959&#xA;scaffold_12     maker   mRNA    652527  655343  .       +       .       ID=maker-scaffold_12-augustus-gene-0.959-mRNA-1;Parent=maker-scaffold_12-augustus-gene-0.959;Name=maker-scaffold_12-augustus-gene-0.959-mRNA-1;_AED=0.24;_eAED=0.18;_QI=0|0|0|0.66|0.5|0.33|3|0|218&#xA;scaffold_12     maker   exon    652527  652817  .       +       .       ID=maker-scaffold_12-augustus-gene-0.959-mRNA-1:exon:1203;Parent=maker-scaffold_12-augustus-gene-0.959-mRNA-1&#xA;scaffold_12     maker   exon    654877  655170  .       +       .       ID=maker-scaffold_12-augustus-gene-0.959-mRNA-1:exon:1204;Parent=maker-scaffold_12-augustus-gene-0.959-mRNA-1&#xA;scaffold_12     maker   exon    655272  655343  .       +       .       ID=maker-scaffold_12-augustus-gene-0.959-mRNA-1:exon:1205;Parent=maker-scaffold_12-augustus-gene-0.959-mRNA-1&#xA;scaffold_12     maker   CDS     652527  652817  .       +       0       ID=maker-scaffold_12-augustus-gene-0.959-mRNA-1:cds;Parent=maker-scaffold_12-augustus-gene-0.959-mRNA-1&#xA;scaffold_12     maker   CDS     654877  655170  .       +       0       ID=maker-scaffold_12-augustus-gene-0.959-mRNA-1:cds;Parent=maker-scaffold_12-augustus-gene-0.959-mRNA-1&#xA;scaffold_12     maker   CDS     655272  655343  .       +       0       ID=maker-scaffold_12-augustus-gene-0.959-mRNA-1:cds;Parent=maker-scaffold_12-augustus-gene-0.959-mRNA-1&#xA;scaffold_12     maker   gene    941547  943897  .       +       .       ID=snap-scaffold_12-processed-gene-0.851;Name=snap-scaffold_12-processed-gene-0.851&#xA;scaffold_12     maker   mRNA    941547  943897  .       +       .       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1;Parent=snap-scaffold_12-processed-gene-0.851;Name=snap-scaffold_12-processed-gene-0.851-mRNA-1;_AED=0.22;_eAED=0.22;_QI=0|0|0.25|0.25|1|1|4|661|95&#xA;scaffold_12     maker   exon    941547  941631  .       +       .       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1:exon:1206;Parent=snap-scaffold_12-processed-gene-0.851-mRNA-1&#xA;scaffold_12     maker   exon    942343  942367  .       +       .       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1:exon:1207;Parent=snap-scaffold_12-processed-gene-0.851-mRNA-1&#xA;scaffold_12     maker   exon    942780  942920  .       +       .       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1:exon:1208;Parent=snap-scaffold_12-processed-gene-0.851-mRNA-1&#xA;scaffold_12     maker   exon    943200  943897  .       +       .       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1:exon:1209;Parent=snap-scaffold_12-processed-gene-0.851-mRNA-1&#xA;scaffold_12     maker   CDS     941547  941631  .       +       0       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1:cds;Parent=snap-scaffold_12-processed-gene-0.851-mRNA-1&#xA;scaffold_12     maker   CDS     942343  942367  .       +       2       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1:cds;Parent=snap-scaffold_12-processed-gene-0.851-mRNA-1&#xA;scaffold_12     maker   CDS     942780  942920  .       +       1       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1:cds;Parent=snap-scaffold_12-processed-gene-0.851-mRNA-1&#xA;scaffold_12     maker   CDS     943200  943236  .       +       1       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1:cds;Parent=snap-scaffold_12-processed-gene-0.851-mRNA-1&#xA;scaffold_12     maker   three_prime_UTR 943237  943897  .       +       .       ID=snap-scaffold_12-processed-gene-0.851-mRNA-1:three_prime_utr;Parent=snap-scaffold_12-processed-gene-0.851-mRNA-1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;GeneMark, SNAP, Augustus&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If we look at the output of several common &lt;em&gt;ab initio&lt;/em&gt; gene prediction tools, none of them actually produces GFF3 by default. Here is some output from GeneMark (&lt;a href=&quot;https://doi.org/10.1093/nar/26.4.1107&quot; rel=&quot;noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;http://exon.gatech.edu/GeneMark/&quot; rel=&quot;noreferrer&quot;&gt;software&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;543   1   +  Initial         799051     799097         47          1 2 - -&#xA;543   2   +  Terminal        799236     799266         31          3 3 - -&#xA;&#xA;544   3   -  Terminal        802357     802514        158          3 2 - -&#xA;544   2   -  Internal        802607     802685         79          1 1 - -&#xA;544   1   -  Initial         802829     802843         15          3 1 - -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is some output from SNAP (&lt;a href=&quot;http://dx.doi.org/10.1186/1471-2105-5-59&quot; rel=&quot;noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;http://korflab.ucdavis.edu/software.html&quot; rel=&quot;noreferrer&quot;&gt;software&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Einit   8230    8239    +       9.329   0       1       0       scaffold_12-snap.4&#xA;Exon    8848    8869    +       2.484   2       2       2       scaffold_12-snap.4&#xA;Exon    10121   10208   +       15.302  1       0       2       scaffold_12-snap.4&#xA;Exon    11361   11420   +       5.969   0       0       2       scaffold_12-snap.4&#xA;Exon    11471   11535   +       0.921   0       2       1       scaffold_12-snap.4&#xA;Eterm   12169   12187   +       18.163  1       0       1       scaffold_12-snap.4&#xA;Einit   14569   14668   +       -6.918  0       1       0       scaffold_12-snap.5&#xA;Exon    15029   15203   +       -1.023  2       2       0       scaffold_12-snap.5&#xA;Exon    16171   16307   +       -9.230  1       1       1       scaffold_12-snap.5&#xA;Eterm   16667   16698   +       9.829   2       0       0       scaffold_12-snap.5&#xA;Einit   17809   17898   -       -5.390  0       0       0       scaffold_12-snap.6&#xA;Exon    17299   17350   -       6.978   0       1       2       scaffold_12-snap.6&#xA;Eterm   17029   17129   -       -2.857  2       0       0       scaffold_12-snap.6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Both of these formats appear to be tab-delimited, but that's where the similarity with GFF3 ends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output of Augustus (&lt;a href=&quot;https://doi.org/10.1093/nar/gkl200&quot; rel=&quot;noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;http://bioinf.uni-greifswald.de/augustus/&quot; rel=&quot;noreferrer&quot;&gt;software&lt;/a&gt;) looks like GTF (a variant of the GFF format) once you lose all the lines beginning with a &lt;code&gt;#&lt;/code&gt; symbol, but Augustus also has a GFF3 output mode.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Predicted genes for sequence number 1 on both strands&#xA;# start gene g1&#xA;scaffold_12     AUGUSTUS        gene    2841    3400    1       +       .       g1&#xA;scaffold_12     AUGUSTUS        transcript      2841    3400    1       +       .       g1.t1&#xA;scaffold_12     AUGUSTUS        start_codon     2841    2843    .       +       0       transcript_id &quot;g1.t1&quot;; gene_id &quot;g1&quot;;&#xA;scaffold_12     AUGUSTUS        intron  3027    3097    1       +       .       transcript_id &quot;g1.t1&quot;; gene_id &quot;g1&quot;;&#xA;scaffold_12     AUGUSTUS        CDS     2841    3026    1       +       0       transcript_id &quot;g1.t1&quot;; gene_id &quot;g1&quot;;&#xA;scaffold_12     AUGUSTUS        CDS     3098    3400    1       +       0       transcript_id &quot;g1.t1&quot;; gene_id &quot;g1&quot;;&#xA;scaffold_12     AUGUSTUS        stop_codon      3398    3400    .       +       0       transcript_id &quot;g1.t1&quot;; gene_id &quot;g1&quot;;&#xA;# protein sequence = [MAIKNAEHDLRVIVDAIEGLGLKVAPHKTEAMAFPASALCGRRGAAPPKIRLGGSSILVGSRSRWYISGHSENSSKSP&#xA;# RTEGKETTPLQQRDPLDAPLWVSGVVAHCCGGPEGQEGCPGLAAQGSDQGVLRIRDGLLCGYDGCGDHRPRPSDSSAGGGLCRP]&#xA;# end gene g1&#xA;###&#xA;# start gene g2&#xA;scaffold_12     AUGUSTUS        gene    4712    15229   0.21    +       .       g2&#xA;scaffold_12     AUGUSTUS        transcript      4712    15229   0.21    +       .       g2.t1&#xA;scaffold_12     AUGUSTUS        start_codon     4712    4714    .       +       0       transcript_id &quot;g2.t1&quot;; gene_id &quot;g2&quot;;&#xA;scaffold_12     AUGUSTUS        intron  4858    5591    0.74    +       .       transcript_id &quot;g2.t1&quot;; gene_id &quot;g2&quot;;&#xA;scaffold_12     AUGUSTUS        intron  5686    15028   0.23    +       .       transcript_id &quot;g2.t1&quot;; gene_id &quot;g2&quot;;&#xA;scaffold_12     AUGUSTUS        CDS     4712    4857    0.72    +       0       transcript_id &quot;g2.t1&quot;; gene_id &quot;g2&quot;;&#xA;scaffold_12     AUGUSTUS        CDS     5592    5685    0.42    +       1       transcript_id &quot;g2.t1&quot;; gene_id &quot;g2&quot;;&#xA;scaffold_12     AUGUSTUS        CDS     15029   15229   0.43    +       0       transcript_id &quot;g2.t1&quot;; gene_id &quot;g2&quot;;&#xA;scaffold_12     AUGUSTUS        stop_codon      15227   15229   .       +       0       transcript_id &quot;g2.t1&quot;; gene_id &quot;g2&quot;;&#xA;# protein sequence = [MGRNSHRSCCVVNCKITSAKSDCKFYKFPTAKWKINQRKMWVAAVKRQKYIKDEISHAETQTEITEVTGATKVNYANK&#xA;# KYICLLFVRTYVLRMLVDVALSNLRFSLFGIRKSLEIFGQSEKADQTRWRLPSCEMEWIESRKGKMRE]&#xA;# end gene g2&#xA;###&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Parsing attributes&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;So...how should YOU handle attributes in GFF3's 9th column? That depends a lot on what you want to do with the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most important attributes are &lt;code&gt;ID&lt;/code&gt; and &lt;code&gt;Parent&lt;/code&gt;, which are used to define relationships between features and subfeatures. (These relationships implicitly define a directed acyclic graph of features, although most GFF3 parsers don't directly support traversal of this graph.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But not only do you have to handle the pre-defined attributes discussed in the GFF3 specification, you also have to be able to handle any number of arbitrary attributes whose keys you may not know beforehand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest way to handle this would be to parse the attribute column into a dictionary of key/value pairs. Once it's in this form, it's trivial to see what attributes are there and how to access them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if we have a feature with the following attributes...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ID=mRNA42;Parent=gene19;integrity=0.95;foo=bar&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;...we would want it in a dictionary like so.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;attributes = {&#xA;    'ID': 'mrna42',&#xA;    'Parent': 'gene19',&#xA;    'integrity': '0.95',&#xA;    'foo': 'bar',&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Python code to parse that might look something like this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;attributes = dict()&#xA;for keyvaluepair in attributestring.split(';'):&#xA;    for key, value in keyvaluepair.split('='):&#xA;        attributes[key] = value&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One thing to consider is that it's possible for an attribute to have multiple values (separated by commas). It's not commonly used, but it is valid and something that should be handled or at least checked for.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2017-07-28T19:18:45.167" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2179" PostTypeId="1" AcceptedAnswerId="2182" CreationDate="2017-07-28T23:41:39.557" Score="-1" ViewCount="70" Body="&lt;pre&gt;&lt;code&gt;use Bio::DB::GenBank;&#xA;use Bio::DB::Query::GenBank;&#xA;&#xA;$query = &quot;LEGK&quot;; &#xA;$query_obj = Bio::DB::Query::GenBank-&amp;gt;new(-db =&amp;gt; 'protein', &#xA;                                          -query =&amp;gt; $query );&#xA;&#xA;$gb_obj = Bio::DB::GenBank-&amp;gt;new;&#xA;&#xA;$stream_obj = $gb_obj-&amp;gt;get_Stream_by_query($query_obj);&#xA;&#xA;while ($seq_obj = $stream_obj-&amp;gt;next_seq) {&#xA;    # do something with the sequence object    &#xA;    print &quot;&amp;gt;$query&quot;,' ', $seq_obj-&amp;gt;display_id, ' ', $seq_obj-&amp;gt;desc,&quot;\n&quot;, $seq_obj-&amp;gt;seq[,'\n';&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I print first occurrence of protein sequence?&lt;/p&gt;&#xA;" OwnerUserId="1224" LastEditorUserId="48" LastEditDate="2017-08-01T12:57:03.253" LastActivityDate="2017-08-01T12:57:03.253" Title="Bioperl - how can i print first result of search sequence per iteration?" Tags="|database|public-databases|perl|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2180" PostTypeId="1" AcceptedAnswerId="2181" CreationDate="2017-07-29T00:06:37.310" Score="4" ViewCount="9056" Body="&lt;p&gt;I'm very new to bioinformatics in general, and I'm trying to understand some basic concepts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have RNAseq data, and bioinformatics people tell me that intensities cannot be compared across patients. So there are all of these pipelines to compare intensities to Z scores--are those as simple as just plugging data into a bioconductor package?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be great to get some overview description of why Z values are important/why you can't compare intensities across patients, and/or some pointers towards any resources that I can read to learn more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks much for your time!&lt;/p&gt;&#xA;" OwnerUserId="1226" LastActivityDate="2017-07-31T09:09:58.577" Title="RNAseq: Z score, Intensity, and Resources" Tags="|rna-seq|bioconductor|python|normalization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2181" PostTypeId="2" ParentId="2180" CreationDate="2017-07-30T08:57:43.057" Score="3" Body="&lt;p&gt;It depends on what test or analysis you want to do, whether you need intensities (expression values) or z-scores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to do statistical analysis, such as finding differentially expressed genes between groups of patients (e.g., with limma), you don't want to use z-scores. But you use normalized intensities (for microarrays) or start with raw counts for RNAseq. Then follow the user guide (limma has a great user guide, also for starters).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For visualization in heatmaps or for other clustering (e.g., k-means, fuzzy) it is useful to use z-scores. Z-scores are a form of transformation (scaling), where every genes is sort of &quot;reset&quot; to the mean of all samples, using also the standard deviation. If you want to know exactly what a z-score is, a simple google search can tell you the details. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In R you can use the scale function for z-score transformation. Be aware that the function works on columns though. Which means you have to transpose your matrix first if genes are in the rows, and then transpose them back after transformation.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2017-07-30T08:57:43.057" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2182" PostTypeId="2" ParentId="2179" CreationDate="2017-07-30T15:28:33.087" Score="4" Body="&lt;p&gt;In the simplest case, if you just want to stop after the first record was printed, you can just add &lt;code&gt;exit&lt;/code&gt; (I also corrected the syntax errors you had and added &lt;code&gt;use strict&lt;/code&gt; and &lt;code&gt;use warnings&lt;/code&gt;; I suggest you get into the habit of using those two, they save you from a lot of grief in the long run):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env perl&#xA;use Bio::DB::GenBank;&#xA;use Bio::DB::Query::GenBank;&#xA;use strict;&#xA;use warnings; &#xA;my $query = &quot;LEGK&quot;; &#xA;my $query_obj = Bio::DB::Query::GenBank-&amp;gt;new(-db =&amp;gt; 'protein', &#xA;                                          -query =&amp;gt; $query );&#xA;my $gb_obj = Bio::DB::GenBank-&amp;gt;new;&#xA;my $stream_obj = $gb_obj-&amp;gt;get_Stream_by_query($query_obj);&#xA;while (my $seq_obj = $stream_obj-&amp;gt;next_seq) {&#xA;	print &quot;&amp;gt;$query&quot;,' ', $seq_obj-&amp;gt;display_id, ' ', $seq_obj-&amp;gt;desc,&quot;\n&quot;, $seq_obj-&amp;gt;seq, &quot;\n&quot;;&#xA;    exit;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If instead, you want to only process the first result for a list of multiple queries:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env perl&#xA;use Bio::DB::GenBank;&#xA;use Bio::DB::Query::GenBank;&#xA;use strict;&#xA;use warnings; &#xA;my @queries = qw(LEGK TP53 ACT1); &#xA;query:foreach my $query (@queries) {&#xA;	my $query_obj = Bio::DB::Query::GenBank-&amp;gt;new(-db =&amp;gt; 'protein', &#xA;                                          -query =&amp;gt; $query );&#xA;	my $gb_obj = Bio::DB::GenBank-&amp;gt;new;&#xA;	my $stream_obj = $gb_obj-&amp;gt;get_Stream_by_query($query_obj);&#xA;	while (my $seq_obj = $stream_obj-&amp;gt;next_seq) {&#xA;		print &quot;&amp;gt;$query&quot;,' ', $seq_obj-&amp;gt;display_id, ' ', $seq_obj-&amp;gt;desc,&quot;\n&quot;, $seq_obj-&amp;gt;seq, &quot;\n&quot;;&#xA;        ## Move to the next query&#xA;        next query;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, to print only the first sequence if multiple quries have identical sequences:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env perl&#xA;use Bio::DB::GenBank;&#xA;use Bio::DB::Query::GenBank;&#xA;use strict;&#xA;use warnings; &#xA;my @queries = qw(LEGK TP53 ACT1); &#xA;foreach my $query (@queries) {&#xA;	my %seen;&#xA;	my $query_obj = Bio::DB::Query::GenBank-&amp;gt;new(-db =&amp;gt; 'protein', &#xA;                                          -query =&amp;gt; $query );&#xA;	my $gb_obj = Bio::DB::GenBank-&amp;gt;new;&#xA;	my $stream_obj = $gb_obj-&amp;gt;get_Stream_by_query($query_obj);&#xA;	while (my $seq_obj = $stream_obj-&amp;gt;next_seq) {&#xA;		if ($seen{$seq_obj-&amp;gt;seq}) {&#xA;			## Here, since we're not using an argument (next query), this will move&#xA;			## to the next iteration of the current loop and not the `foreach`. &#xA;			next;&#xA;		}&#xA;		print &quot;&amp;gt;$query&quot;,' ', $seq_obj-&amp;gt;display_id, ' ', $seq_obj-&amp;gt;desc,&quot;\n&quot;, $seq_obj-&amp;gt;seq, &quot;\n&quot;;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-07-30T15:28:33.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2183" PostTypeId="1" CreationDate="2017-07-31T06:45:17.843" Score="-3" ViewCount="64" Body="&lt;p&gt;I have the following Chip-Seq data and could not found a description of it. Can you help me with it? More, I would like to find out the order of the nucleotides in binding regions; that is, what number they are.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@GWZHISEQ01:319:HA1NPADXX:1:1101:1182:2196 1:N:0:ACTGAT&#xA;GGAGGAAGTGTGTTGCTGCAGACAGTAGCAACCAGACCCACACTGCGCGTA&#xA;+&#xA;CCCFFFFFHHFHHIJJJJIJJIIIIHGJJJJJJJCGHIJJIJJFHIIJJII&#xA;@GWZHISEQ01:319:HA1NPADXX:1:1101:1272:2109 1:N:0:ACTGAT&#xA;NCCACGCTAGGCTCAGCTTGTCGGCCTGGCTAAGCAGTTGCGAAAGTGCGC&#xA;+&#xA;#1=DDDFFHHHHHJJIJJJJJJIGEHGGJIGHJFHGJIJJJJJJJ@CHCBG&#xA;@GWZHISEQ01:319:HA1NPADXX:1:1101:1418:2141 1:N:0:ACTGAT&#xA;GCACGCACTACCCAGAGATCATCCAAAGCCTGAAGCCACAGGGCGCACTCG&#xA;+&#xA;CCCFFFFFGHGHHJJJIIJJJJIIHGGGIJIIJJIIIJJJIJJJIJIJIJH&#xA;@GWZHISEQ01:319:HA1NPADXX:1:1101:1302:2160 1:N:0:ACTGAT&#xA;ACACTCTTTCCCTACACGACGCTCTTCCGAGATCGGAAGAGCACACGTCTG&#xA;+&#xA;CCCFFFFFHDHHHJJJJIJJJJIIIIIJIHJIJJJIIJHIIJJJJJJJJHH&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1230" LastActivityDate="2017-08-04T10:20:55.950" Title="Chip-Seq data description" Tags="|chip-seq|" AnswerCount="1" CommentCount="4" ClosedDate="2017-07-31T08:31:28.373" ContentLicense="CC BY-SA 3.0" />
  <row Id="2184" PostTypeId="2" ParentId="2183" CreationDate="2017-07-31T07:23:14.257" Score="4" Body="&lt;p&gt;That's a &lt;a href=&quot;https://en.wikipedia.org/wiki/FASTQ_format&quot; rel=&quot;nofollow noreferrer&quot;&gt;fastq&lt;/a&gt; file, you will want to align it to the genome, call peaks, and then use something like &lt;a href=&quot;http://meme-suite.org/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MEME&lt;/a&gt; to determine binding motifs.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="292" LastEditDate="2017-08-04T10:20:55.950" LastActivityDate="2017-08-04T10:20:55.950" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2185" PostTypeId="2" ParentId="2180" CreationDate="2017-07-31T09:09:58.577" Score="2" Body="&lt;p&gt;In cancer genomics experiments often involve sequencing a large number of tumour samples, but few or no matched normals. This is partly due to financial constraints, but also due to ethical ones: it is easy to get ethical approval for tumour tissue, which is removed as a matter of course in a patient's care anyway, whereas it is hard to get approval for normal tissue, which would not normally be removed and would require an additional procedure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this presents a problem. Most of the procedures we use in standard gene expression analysis involve comparing one group to another after suitable normalisations of the data. But here we have no suitable comparison group. To get around this problem in cancer RNA-seq analysis we do something called outlier detection. That is, we assume that for a given gene, the majority of the patients have &quot;normal&quot; expression, and look for the few patients that don't look like the rest. We do this with the Z-score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To calculate the Z-score for an observation we subtract the mean of all observations and divide by the standard deviation. Thus the Z score of an observation is how many standard deviations an observation is from the mean of all observations - or how unusual it is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus for Gene A in Patient 1, we calculate how many standard deviations Gene A is from the mean of Gene A across all patients. A very big (or small/negative) value tells us that the expression of Gene A is unusual in patient 1 compared to the other patients. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(NB: in some cancer studies they use the median and median absolute deviation rather than the mean and standard deviation to compute a &quot;robust Z-score&quot; as the data is unlikely to be normally distributed). &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2017-07-31T09:09:58.577" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2186" PostTypeId="1" AcceptedAnswerId="2187" CreationDate="2017-07-31T09:43:42.450" Score="8" ViewCount="3474" Body="&lt;p&gt;I have a problem: I've managed to download a massive fasta file of 1500 sequences, but now I want to split them into separate fasta files based on the genus.&lt;/p&gt;&#xA;&lt;p&gt;EDIT&lt;br /&gt;&#xA;The fasta file looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;terminase_large.fasta&#xA;&amp;gt;YP_009300697.1 terminase large subunit [Arthrobacter phage Mudcat]&#xA;MGLSNTATPLYYGQF...&#xA;&#xA;&amp;gt;YP_009208724.1 hypothetical protein ADP65_00072 [Achromobacter phage phiAxp-3]&#xA;MSNVLLKQELDEWL...&#xA;&#xA;&amp;gt;YP_009148449.1 large terminase subunit [Delftia phage RG-2014]&#xA;MSEPRKLVKKTLD...&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to end up with something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mycobacterium_terminase.fasta&#xA;&amp;gt;Mycobacterium phage JAMaL&#xA;MVRKKPPPELE...&#xA;&#xA;&amp;gt;Mycobacterium phage Bruin&#xA;MEVCGYTLDDI...&#xA;&#xA;&amp;gt;Mycobacterium phage Zaka&#xA;MSLDNHLPELA...&#xA;&#xA;salmonella_terminase.fasta&#xA;&amp;gt;Salmonella virus SETP7&#xA;MNVDITATEPQ...&#xA;&#xA;&amp;gt;Salmonella virus SE2&#xA;MEGGSDSLIAM...&#xA;and so on...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So that the label on the alignment would have the name of the phage and not the protein's ID.&lt;/p&gt;&#xA;&lt;p&gt;I've managed to extract the genus names of my organism with this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;outfile = open('species.txt', 'w')&#xA;with open('terminase_large.fasta') as fd:&#xA;    for line in fd:&#xA;        if line.startswith('&amp;gt;'):&#xA;            if '[' in line:&#xA;                name=line.split('[')[-1]&#xA;                name=name.split(' ', 1)[0]&#xA;                outfile.write(name[:] + &amp;quot;\n&amp;quot;)&#xA;outfile.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And got to extract only the unique names with this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;lines_seen = set()&#xA;outfile = open('species2.txt', &amp;quot;w&amp;quot;)&#xA;for line in open(&amp;quot;species.txt&amp;quot;, &amp;quot;r&amp;quot;):&#xA;    if line not in lines_seen:  # not a duplicate&#xA;        outfile.write(line)&#xA;        lines_seen.add(line)&#xA;outfile.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(Can I merge those two scripts together?)&lt;/p&gt;&#xA;&lt;p&gt;Now, my genus names look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Arthrobacter&#xA;Achromobacter&#xA;Delftia&#xA;....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried automating my script to get the Entrez data, but it gives me the 'Supplied id parameter is empty' message&lt;/p&gt;&#xA;&lt;p&gt;My code looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez&#xA;Entrez.email = &amp;quot;example@example.org&amp;quot;&#xA;&#xA;for line in open(&amp;quot;species2.txt&amp;quot;, &amp;quot;r&amp;quot;):&#xA;    searchterm = &amp;quot;(terminase large subunit AND viruses[Organism]) AND&amp;quot; +line+ &amp;quot;AND refseq[Filter]&amp;quot;&#xA;    searchResultHandle = Entrez.esearch(db=&amp;quot;protein&amp;quot;, term=searchterm, retmax=1000)&#xA;    searchResult = Entrez.read(searchResultHandle)&#xA;    ids = searchResult[&amp;quot;IdList&amp;quot;]&#xA;&#xA;    handle = Entrez.efetch(db=&amp;quot;protein&amp;quot;, id=ids, rettype=&amp;quot;fasta&amp;quot;, retmode=&amp;quot;text&amp;quot;)&#xA;    record = handle.read()&#xA;&#xA;    out_handle = open('terminase_large_'+str(line[:-1])+'.fasta', 'w')&#xA;    out_handle.write(record.rstrip('\n'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can someone help me with it?&lt;/p&gt;&#xA;" OwnerUserId="1193" LastEditorUserId="1157" LastEditDate="2020-09-30T10:11:14.990" LastActivityDate="2020-09-30T10:11:14.990" Title="Subset FASTA file by species name" Tags="|fasta|python|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="2187" PostTypeId="2" ParentId="2186" CreationDate="2017-07-31T10:35:37.210" Score="7" Body="&lt;p&gt;Splitting into multiple files and changing the IDs can be easily done:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;perl -pe 'if(/&amp;gt;/){/\[(.*?)\]\s*$/; $_=&quot;&amp;gt; $1\n&quot;}' file.fa | &#xA;    awk '(/^&amp;gt;/){name=$2} {print &amp;gt;&amp;gt; name&quot;.fa&quot;}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That assumes all your FASTA headers have &lt;code&gt;[foo bar baz]&lt;/code&gt; as the last element of a line. It will create a file called &lt;code&gt;foo.fa&lt;/code&gt; (the bacterium's name) with all sequences saved there. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Explanation&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;perl -pe&lt;/code&gt; : run the script given by &lt;code&gt;-e&lt;/code&gt; on each line of the input file, and print the resulting line. &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;if(/&amp;gt;/)&lt;/code&gt; : if this line starts with a &lt;code&gt;&amp;gt;&lt;/code&gt;. \&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;/\[(.*?)\]\s*$/&lt;/code&gt; : match an opening bracket (&lt;code&gt;\[&lt;/code&gt;), then capture (that's what the parentheses do, they capture a pattern so we can refer to it as &lt;code&gt;$1&lt;/code&gt;) everything until the first &lt;code&gt;]&lt;/code&gt; (&lt;code&gt;.*?\]&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$_=&quot;&amp;gt; $1\n&quot;&lt;/code&gt; : the &lt;code&gt;$_&lt;/code&gt; special variable in Perl is (in this case) the current line. So, &lt;code&gt;$_=foo&lt;/code&gt; means &quot;make the current line read &lt;code&gt;foo&lt;/code&gt;. Since the &lt;code&gt;-p&lt;/code&gt; prints each input line, changing the value of &lt;code&gt;$_&lt;/code&gt; means the changed value will be printed. So here, we are printing &lt;code&gt;&amp;gt;&lt;/code&gt;, whatever was in the square brackets (&lt;code&gt;$1&lt;/code&gt;) and a newline character. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The output of the perl command alone on your example input file is:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;$ perl -pe 'if(/&amp;gt;/){/\[(.*)\]\s*$/; $_=&quot;&amp;gt; $1\n&quot;}' file.fa&#xA;&amp;gt; Arthrobacter phage Mudcat&#xA;MGLSNTATPLYYGQF...&#xA;&#xA;&amp;gt; Achromobacter phage phiAxp-3&#xA;MSNVLLKQELDEWL...&#xA;&#xA;&amp;gt; Delftia phage RG-2014&#xA;MSEPRKLVKKTLD...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, we now pass it through &lt;code&gt;awk&lt;/code&gt; which does:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;(/^&amp;gt;/){a=$2}&lt;/code&gt; : if this line starts with an &lt;code&gt;&amp;gt;&lt;/code&gt;, save the second field (the bacterial species) as the variable &lt;code&gt;name&lt;/code&gt;. &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;{print &amp;gt;&amp;gt; name&quot;.fa&quot;}&lt;/code&gt; : print each line into a file whose name is the current value of the variable &lt;code&gt;name&lt;/code&gt; with a &lt;code&gt;.fa.&lt;/code&gt; extension. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If you prefer python scripts to the one-liner approach, you can do the same thing with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;import re&#xA;outFile = None&#xA;with open(&quot;file.fa&quot;, &quot;r&quot;) as inFile, open(species + &quot;.fa&quot;, 'a') as outFile:&#xA;    for line in inFile:&#xA;        line = line.rstrip()&#xA;        if line.startswith('&amp;gt;'):&#xA;            regex = re.compile('.*\[((.+?)\s+.*?)\].*')&#xA;            matches = regex.search(line)&#xA;            species = matches[2]&#xA;            outFile.write('&amp;gt;%s\n' % matches[1])&#xA;        else:&#xA;            outFile.write(&quot;%s\n&quot; % line)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As for your script, you've got the right idea, but have a small bug. You forgot to remove the &lt;code&gt;\n&lt;/code&gt; from your input file, so it looks for &lt;code&gt;Arthrobacter\n&lt;/code&gt; instead of &lt;code&gt;Arthrobacter&lt;/code&gt;. The golden rule of debugging is &quot;print all the things&quot;. If you add &lt;code&gt;print(&quot;Searchterm: &quot;,searchterm)&lt;/code&gt;  you will see:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;Searchterm:  (terminase large subunit AND viruses[Organism]) ANDArthrobacter  &#xA;AND refseq[Filter]&#xA;Searchterm:  (terminase large subunit AND viruses[Organism]) ANDAchromobacter  &#xA;AND refseq[Filter]&#xA;Searchterm:  (terminase large subunit AND viruses[Organism]) ANDDelftia  &#xA;AND refseq[Filter]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, you need to remove the newline characters and add a space like so (I also made it a bit more &quot;pythonic&quot; and conforming to the Python syntax guidelines):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;from Bio import Entrez&#xA;Entrez.email = &quot;example@example.org&quot;&#xA;&#xA;with open(&quot;species2.txt&quot;, &quot;r&quot;) as in_handle:&#xA;    for line in in_handle:&#xA;        line = line.rstrip()&#xA;        searchterm = (&quot;(terminase large subunit AND viruses[Organism]) &quot; +&#xA;                      &quot;AND %s AND refseq[Filter]&quot; % line)&#xA;        print(&quot;Searchterm: &quot;, searchterm)&#xA;        searchResultHandle = Entrez.esearch(db=&quot;protein&quot;,&#xA;                                            term=searchterm, retmax=1000)&#xA;        searchResult = Entrez.read(searchResultHandle)&#xA;        ids = searchResult[&quot;IdList&quot;]&#xA;&#xA;        handle = Entrez.efetch(db=&quot;protein&quot;, id=ids,&#xA;                               rettype=&quot;fasta&quot;, retmode=&quot;text&quot;)&#xA;        record = handle.read()&#xA;&#xA;        with open('terminase_large_' + str(line[:-1]) + '.fasta', 'w') as out_handle:&#xA;            out_handle.write(record.rstrip('\n'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="450" LastEditDate="2017-08-11T13:19:48.687" LastActivityDate="2017-08-11T13:19:48.687" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2188" PostTypeId="2" ParentId="951" CreationDate="2017-07-31T21:12:27.573" Score="1" Body="&lt;p&gt;I also think pooling is the better option, followed by partitioning by coverage/taxonomy of contigs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe check out &lt;a href=&quot;https://github.com/DRL/blobtools&quot; rel=&quot;nofollow noreferrer&quot;&gt;BlobTools&lt;/a&gt;, which helps with filtering read-pairs by taxonomy of contigs they contribute to and also does nice visualisations of assemblies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://blobtools.readme.io/docs/the-blobtools-workflows&quot; rel=&quot;nofollow noreferrer&quot;&gt;Workflow B&lt;/a&gt; seems to be what you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Disclaimer: I am the developer of this tool.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1234" LastActivityDate="2017-07-31T21:12:27.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2189" PostTypeId="5" CreationDate="2017-08-01T10:33:20.660" Score="0" Body="&lt;p&gt;RNA-seq (RNA sequencing) is a widely used approach to study gene expression. Questions should include this tag if they pertain to issues related to bioinformatics analysis of RNA-seq data. &lt;/p&gt;&#xA;" OwnerUserId="1140" LastEditorUserId="1140" LastEditDate="2017-08-01T10:47:49.007" LastActivityDate="2017-08-01T10:47:49.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2190" PostTypeId="4" CreationDate="2017-08-01T10:33:20.660" Score="0" Body="Questions should include this tag if they pertain to issues related to bioinformatics analysis of RNA-seq data, e.g. normalization, differential expression analysis, sequencing and experimental design or transcriptome assembly..." OwnerUserId="1140" LastEditorUserId="57" LastEditDate="2018-05-03T07:09:43.600" LastActivityDate="2018-05-03T07:09:43.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2192" PostTypeId="5" CreationDate="2017-08-01T13:53:26.140" Score="0" Body="&lt;p&gt;bwa (Burrows-Wheeler Aligner) is a software for aligning reads obtained with Next Generation Sequencing. Questions related to the functionalities of bwa should include this tag. Please consider specifying if you are using bwa aln or bwa mem. Fro more info: &lt;a href=&quot;http://bio-bwa.sourceforge.net/bwa.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://bio-bwa.sourceforge.net/bwa.shtml&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1140" LastEditorUserId="1140" LastEditDate="2017-08-01T14:02:58.737" LastActivityDate="2017-08-01T14:02:58.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2193" PostTypeId="4" CreationDate="2017-08-01T13:53:26.140" Score="0" Body="bwa (Burrows-Wheeler Aligner) is a software for aligning reads obtained with Next Generation Sequencing. " OwnerUserId="1140" LastEditorUserId="1140" LastEditDate="2017-08-01T14:02:54.490" LastActivityDate="2017-08-01T14:02:54.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2194" PostTypeId="1" CreationDate="2017-08-01T16:24:34.650" Score="12" ViewCount="256" Body="&lt;p&gt;I’ve got an RNA-seq dataset with a large proportion of environmental RNA “contamination”. BLASTing random reads reveals that much of the data comes from bacterial, plant and viral RNA. My target organism only accounts for ~5% of the RNA-seq read data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to get a more or less (more, if possible) comprehensive overview of what species are found in my sample — bacteria, plants, animals (?), but also viruses. How can I perform this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been unable to find a “standard” way of performing a metagenome screen. BLAST online services all seem to be severely rate limited (certainly unable to handle uploading ~80 M reads). Installing BLAST (or a similar tool) locally is of course not an issue, but I can’t find a comprehensive database that spans all phyla — the best I’ve been able to find are databases, such as &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/refseq/about/nonredundantproteins/&quot; rel=&quot;noreferrer&quot;&gt;NCBI-NR&lt;/a&gt;, that are restricted to single phyla or classes, or to bacteria.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally I’d like a workflow that I feed with my RNA-seq data, and get out a list of species with % of reads mapped.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2017-09-15T14:10:56.140" Title="How do I efficiently perform a metagenome screen of “all” species?" Tags="|rna-seq|metagenome|" AnswerCount="5" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2195" PostTypeId="2" ParentId="2194" CreationDate="2017-08-01T17:36:43.390" Score="4" Body="&lt;p&gt;I think you're just looking for &lt;code&gt;nr&lt;/code&gt;. It is absolutely not limited to prokaryotes, far from it. It is, according to the blurb on the NCBI's blast page:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The nucleotide collection consists of GenBank+EMBL+DDBJ+PDB+RefSeq sequences, but excludes EST, STS, GSS, WGS, TSA, patent sequences as well as phase 0, 1, and 2 HTGS sequences. The database is non-redundant. Identical sequences have been merged into one entry, while preserving the accession, GI, title and taxonomy information for each entry.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The link you gave in your question doesn't describe the &lt;code&gt;nr&lt;/code&gt; database, but the collection of RefSeq non-redundant sequences. The two are not the same thing at all. The nr databases contains sequences from all available species so it should be perfect for what you're after. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I admit I haven't tried this with millions of redas, but I did used to run many blast searches with several thousand sequences using the command line blast tools and the remote nr database. I just checked the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK279690/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blast+ docs&lt;/a&gt; now and see no mention of a rate limit. Which isn't to say there isn't one, for sure, but since each query is run sequentially when you give a multifasta file, there seems to reason to limit since the servers should be able to queue the jobs as needed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, don't use the API or roll your own, just &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&amp;amp;PAGE_TYPE=BlastDocs&amp;amp;DOC_TYPE=Download&quot; rel=&quot;nofollow noreferrer&quot;&gt;download NCBI's blast client&lt;/a&gt; and try that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blastn -query query.fa -db nr -task blastn &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If that complains, you'll have to download the &lt;code&gt;nr&lt;/code&gt; DB (well, since you want nucleotide blast, you probably want its nucleotide cousin, &lt;code&gt;nt&lt;/code&gt;) which you can find here: &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/blast/db/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ncbi.nlm.nih.gov/blast/db/&lt;/a&gt;. Just get all the &lt;code&gt;foo.nt*&lt;/code&gt; files.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-08-01T17:36:43.390" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2196" PostTypeId="2" ParentId="2194" CreationDate="2017-08-01T21:54:51.507" Score="7" Body="&lt;p&gt;As you point out, BLAST is not going to work well for reads from a typical RNASeq experiment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From an RNASeq dataset, you might be able to get away with just a ribosomal screen using &lt;a href=&quot;https://www.arb-silva.de/projects/ssu-ref-nr/&quot; rel=&quot;noreferrer&quot;&gt;Silva&lt;/a&gt;, because the ribosome is the most abundantly transcribed gene. This will usually be the case even if polyA selection or ribosomal depletion was carried out on a sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, bear in mind that there will be a lot of false results due to database incompleteness. I used &lt;a href=&quot;http://ccb.jhu.edu/software/kraken/&quot; rel=&quot;noreferrer&quot;&gt;Kraken&lt;/a&gt; when I did something like this a few years ago, and ended up with one sequence annotated as a fruit bat (where my subject was a water-living planarian flatworm). &lt;a href=&quot;https://github.com/infphilo/centrifuge/&quot; rel=&quot;noreferrer&quot;&gt;Centrifuge&lt;/a&gt; would be a better option now; the authors even claim that &lt;a href=&quot;http://genome.cshlp.org/content/26/12/1721.full&quot; rel=&quot;noreferrer&quot;&gt;they can store the entirety of nr in a 70 GB index&lt;/a&gt; with Centrifuge (which would be better / more comprehensive than just a ribosomal screen).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-08-01T21:54:51.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2197" PostTypeId="2" ParentId="52" CreationDate="2017-08-01T22:16:10.137" Score="4" Body="&lt;p&gt;We have added ERCC spike-ins to all our RNASeq data, just in case other people might find it useful in the future. However, I have never used it in my own analyses because I can't think of a reasonable way that it could be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The typical recommendation for ERCC is to add it in proportion to the input RNA amount, but that makes an assumption that total cell RNA counts are similar across different cells (which is demonstrably false by looking at single cell RNASeq results).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have yet to think of a situation where ERCC would provide better results than a &quot;housekeeping&quot; gene set sampled from the original reads.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-08-01T22:16:10.137" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2198" PostTypeId="2" ParentId="2174" CreationDate="2017-08-01T22:25:13.990" Score="3" Body="&lt;p&gt;Albacore produces a &lt;code&gt;sequencing_summary.txt&lt;/code&gt; file (actually TSV, not CSV) in the same directory as the &lt;code&gt;workspace&lt;/code&gt; folder that might have the data that you want in it. Here are the fields present in that file:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;filename&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;read_id&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;run_id&lt;/li&gt;&#xA;&lt;li&gt;channel&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;start_time&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;duration&lt;/li&gt;&#xA;&lt;li&gt;num_events&lt;/li&gt;&#xA;&lt;li&gt;template_start&lt;/li&gt;&#xA;&lt;li&gt;num_events_template&lt;/li&gt;&#xA;&lt;li&gt;template_duration&lt;/li&gt;&#xA;&lt;li&gt;num_called_template&lt;/li&gt;&#xA;&lt;li&gt;sequence_length_template&lt;/li&gt;&#xA;&lt;li&gt;mean_qscore_template&lt;/li&gt;&#xA;&lt;li&gt;strand_score_template&lt;/li&gt;&#xA;&lt;li&gt;complement_start&lt;/li&gt;&#xA;&lt;li&gt;num_events_complement&lt;/li&gt;&#xA;&lt;li&gt;complement_duration&lt;/li&gt;&#xA;&lt;li&gt;num_called_complement&lt;/li&gt;&#xA;&lt;li&gt;sequence_length_complement&lt;/li&gt;&#xA;&lt;li&gt;mean_qscore_complement&lt;/li&gt;&#xA;&lt;li&gt;strand_score_complement&lt;/li&gt;&#xA;&lt;li&gt;sequence_length_2d&lt;/li&gt;&#xA;&lt;li&gt;mean_qscore_2d&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-08-01T22:25:13.990" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2199" PostTypeId="1" CreationDate="2017-08-02T05:27:40.667" Score="2" ViewCount="68" Body="&lt;p&gt;I have a question regarding the RNA-Seq analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have a time course data regarding a mouse model wt and mutant treated with a drug (10uM) and the taken down at different time points: 8 time points and did RNA-seq on these samples currently we have tpm values and raw gene counts from rna seq. Also we have only one sample for each time point. we would like to see for changes in genes across different time point but not sure statistical method should we use to determine this, should we consider for example a single time point wt vs mt, or see trends across wt and mt individually &#xA;I know that there are some bioc packages for NGS analysis (edgeR,&#xA;DESeq,etc). Is any package or method good for one-sample-per-time-point&#xA;data?&lt;/p&gt;&#xA;" OwnerUserId="315" LastEditorUserId="57" LastEditDate="2017-08-02T08:56:11.040" LastActivityDate="2017-08-02T08:56:11.040" Title="ngs time course experiment" Tags="|rna-seq|software-recommendation|statistics|" AnswerCount="0" CommentCount="4" ClosedDate="2017-08-02T11:48:59.337" ContentLicense="CC BY-SA 3.0" />
  <row Id="2200" PostTypeId="1" CreationDate="2017-08-02T17:24:24.340" Score="4" ViewCount="197" Body="&lt;p&gt;I have a PDB file, and I need to extract its residue sequence numbers (resSeq's). Based on manual inspection of the first few lines of the PDB file (pasted below), I would think that resSeq's should begin with 22, 23. However, Biopython's PDB module suggests otherwise (output attached below as well). I wonder if it's a Biopython bug or if I have problems understanding the PDB format. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ATOM      1  N   GLY A  22      78.171  89.858  59.231  1.00 21.24           N  &#xA;ATOM      2  CA  GLY A  22      79.174  88.827  58.999  1.00 20.87           C  &#xA;ATOM      3  C   GLY A  22      80.438  89.415  58.391  1.00 21.89           C  &#xA;ATOM      4  O   GLY A  22      80.362  90.202  57.440  1.00 23.18           O  &#xA;ATOM      5  N   LEU A  23      81.588  89.069  58.972  1.00 21.51           N  &#xA;ATOM      6  CA  LEU A  23      82.895  89.555  58.527  1.00 20.80           C  &#xA;ATOM      7  C   LEU A  23      83.288  89.020  57.162  1.00 22.41           C  &#xA;ATOM      8  O   LEU A  23      82.889  87.923  56.788  1.00 22.93           O  &#xA;ATOM      9  CB  LEU A  23      83.973  89.232  59.560  1.00 20.97           C  &#xA;ATOM     10  CG  LEU A  23      84.225  87.818  60.062  1.00 13.32           C  &#xA;ATOM     11  CD1 LEU A  23      85.448  87.888  60.939  1.00 15.24           C  &#xA;ATOM     12  CD2 LEU A  23      83.035  87.258  60.829  1.00 12.21           C  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&amp;nbsp;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;for i in chain:&#xA;    print i.get_full_id()&#xA;&#xA;OUT:('pdb', 0, 'A', (' ', 2, ' '))&#xA;    ('pdb', 0, 'A', (' ', 3, ' '))&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1249" OwnerDisplayName="Alex Mayorov" LastEditorUserId="298" LastEditDate="2017-08-03T08:28:43.720" LastActivityDate="2017-08-03T13:45:40.647" Title="Biopython: resseq doesn't match pdb file" Tags="|pdb|python|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2201" PostTypeId="1" CreationDate="2017-08-02T20:14:06.153" Score="0" ViewCount="127" Body="&lt;p&gt;This is a cross-post from &lt;a href=&quot;https://math.stackexchange.com/q/2380215&quot;&gt;Mathematics forum&lt;/a&gt;. As no one has answered it yet, I fIgured post here as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to describe a transform I used to rank my data points. I have recoded my variable columns from $10$ measurements with different range to $n$ (for each measurement) where $n \in \{0,0.5,1\}$.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For columns where measurements were expressed as categorical values I simply recoded the categorical value to either $0$, $0.5$ or $1$ and for continuous variables I recoded the lower quartile as $0$, interquartile as $0.5$ and upper quartile as $1$ and finally I summed the recoded values to produce a single score $\sum n_{i1,..i10} $ for each row. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to write my method and I'd like to have it in mathematical notation. I was wondering if anyone could help me with this!  &lt;/p&gt;&#xA;" OwnerUserId="1250" LastEditorUserId="77" LastEditDate="2017-08-18T13:29:16.580" LastActivityDate="2017-08-18T17:39:08.600" Title="Mathematical notation for formulating a rank score" Tags="|modelling|" AnswerCount="1" CommentCount="11" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2202" PostTypeId="1" AcceptedAnswerId="2211" CreationDate="2017-08-02T21:03:29.337" Score="4" ViewCount="82" Body="&lt;p&gt;I am interested in identifying indels in whole genome bisulfite sequencing data (76bp paired end). Currently, I do this by setting the &lt;code&gt;-rfg&lt;/code&gt; and &lt;code&gt;-rdg&lt;/code&gt; affine gap penalty scores for bowtie2 to more permissive values than the default 5+3N and mapping using the bisulfite sequencing alignment wrapper, bismark.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, what values of &lt;code&gt;-rfg&lt;/code&gt; and &lt;code&gt;-rdg&lt;/code&gt; will allow me to identify longest possible indels without sacrificing alignment quality? Is it better to set the affine penalty to zero with a high penalty for initially opening an indel (ex. 8+0N)? Or is it better to keep the initial penalty low and having a nonzero penalty for extension (ex. 2+1N)?&lt;/p&gt;&#xA;" OwnerUserId="643" LastEditorUserId="643" LastEditDate="2017-08-04T03:39:26.213" LastActivityDate="2017-08-04T03:39:26.213" Title="Best way to detect long insertions in bisulfite sequencing data?" Tags="|ngs|sequence-alignment|read-mapping|methylation|indel|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2203" PostTypeId="1" CreationDate="2017-08-03T08:54:48.673" Score="5" ViewCount="633" Body="&lt;p&gt;I want to incorporate Multiple sequence alignment into our Shiny app, i.e. allow the possibility of selecting different nucleotide sequences, and do a multiple sequence alignment in real-time. The user would define as input different nt. sequences and then a tree will pop up in the app. Do you think this is feasible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any recommendations and feedback are really appreciated :)&lt;/p&gt;&#xA;" OwnerUserId="446" LastEditorUserId="73" LastEditDate="2018-01-14T22:52:58.643" LastActivityDate="2018-01-14T22:52:58.643" Title="Multiple sequence alignment with R" Tags="|r|sequence-alignment|" AnswerCount="1" CommentCount="8" ClosedDate="2018-01-21T18:36:36.043" ContentLicense="CC BY-SA 3.0" />
  <row Id="2204" PostTypeId="1" AcceptedAnswerId="2262" CreationDate="2017-08-03T09:29:03.723" Score="7" ViewCount="292" Body="&lt;p&gt;Is the MUMmer suite capable of calculating reference sequence coverage statistics for all query sequences collectively? It would be possible to achieve by parsing the output of &lt;code&gt;nucmer&lt;/code&gt; / &lt;code&gt;show-coords&lt;/code&gt; / &lt;code&gt;show-tiling&lt;/code&gt; but it seems like there should be a better way. I currently do this using a sensitive read mapper, &lt;code&gt;samtools depth&lt;/code&gt; and some scripting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To clarify, I'd like to know the reference coverage achieved using all of the query sequences (i.e. the whole &lt;em&gt;de novo&lt;/em&gt; assembly, in my case).&lt;/p&gt;&#xA;" OwnerUserId="1258" LastEditorUserId="1258" LastEditDate="2017-09-08T11:59:22.017" LastActivityDate="2017-09-08T11:59:22.017" Title="How to calculate overall reference coverage with MUMmer?" Tags="|assembly|coverage|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2206" PostTypeId="2" ParentId="2203" CreationDate="2017-08-03T09:31:47.057" Score="4" Body="&lt;p&gt;The &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/msa.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;‹msa› package on Bioconductor&lt;/a&gt; does exactly that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It doesn’t hand the results on a silver platter, though: you’ll need to &lt;a href=&quot;https://bioconductor.org/packages/devel/bioc/vignettes/msa/inst/doc/msa.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;read the vignette&lt;/a&gt; carefully to learn how to use it. But after that it’s pretty powerful.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2017-08-03T09:31:47.057" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2207" PostTypeId="2" ParentId="2200" CreationDate="2017-08-03T13:45:40.647" Score="3" Body="&lt;p&gt;Let's check it. I copied your file fragment into &lt;code&gt;a.pdb&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; from Bio import PDB&#xA;&amp;gt;&amp;gt;&amp;gt; for res in PDB.PDBParser().get_structure('pdb', 'a.pdb')[0].get_residues():&#xA;...   print(res.get_full_id())&#xA;... &#xA;('pdb', 0, 'A', (' ', 22, ' '))&#xA;('pdb', 0, 'A', (' ', 23, ' '))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So BioPython (ver. 1.66) gives 22 and 23 as expected.&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2017-08-03T13:45:40.647" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2208" PostTypeId="5" CreationDate="2017-08-03T14:10:09.633" Score="0" Body="&lt;p&gt;python (&lt;a href=&quot;https://www.python.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.python.org/&lt;/a&gt;) is a programming language, widely used in bioinformatics. Use this tag for questions about programming in python.&lt;/p&gt;&#xA;" OwnerUserId="1140" LastEditorUserId="1140" LastEditDate="2017-08-03T18:11:21.103" LastActivityDate="2017-08-03T18:11:21.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2209" PostTypeId="4" CreationDate="2017-08-03T14:10:09.633" Score="0" Body="python is a programming language, widely used in bioinformatics" OwnerUserId="1140" LastEditorUserId="1140" LastEditDate="2017-08-03T18:11:15.430" LastActivityDate="2017-08-03T18:11:15.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2210" PostTypeId="2" ParentId="658" CreationDate="2017-08-03T15:23:45.557" Score="1" Body="&lt;p&gt;DNAnexus -- &lt;a href=&quot;http://dnanexus.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dnanexus.com&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BaseSpace -- &lt;a href=&quot;http://basespace.illumina.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://basespace.illumina.com&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seven Bridges Genomics -- &lt;a href=&quot;http://www.sbgenomics.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sbgenomics.com&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Curoverse &lt;a href=&quot;http://curoverse.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://curoverse.com&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;InsideDNA &lt;a href=&quot;http://insidedna.me/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://insidedna.me/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2017-08-03T15:23:45.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2211" PostTypeId="2" ParentId="2202" CreationDate="2017-08-03T18:30:18.183" Score="3" Body="&lt;p&gt;You'd be best off by starting with &lt;code&gt;-rfg&lt;/code&gt; and &lt;code&gt;-rdg&lt;/code&gt; as is and reverting bismark's change of &lt;code&gt;--score-min&lt;/code&gt; back to the default for bowtie2. That alone will allow for much longer indels. If that still doesn't suffice, then I'd play around more with &lt;code&gt;--score-min&lt;/code&gt; before messing with the gap open/extend penalties. If you do need to play with those, then increase the gap open penalty and decrease the gap extension to 1. Do check that the resulting alignments aren't nonsense though!&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-08-03T18:30:18.183" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2212" PostTypeId="2" ParentId="2194" CreationDate="2017-08-03T18:55:25.983" Score="1" Body="&lt;p&gt;I have had good success using the MEGAN software to detect and visualize contamination.  My input data was WGS not RNA-seq But I would check it out.  It can create a very nice weight phylogenetic tree for all the detected organisms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ab.inf.uni-tuebingen.de/software/megan6&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ab.inf.uni-tuebingen.de/software/megan6&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ttT6l.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ttT6l.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2017-08-03T18:55:25.983" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2213" PostTypeId="2" ParentId="2194" CreationDate="2017-08-03T21:38:09.977" Score="4" Body="&lt;p&gt;You might want to look into &lt;a href=&quot;https://en.wikipedia.org/wiki/MinHash&quot; rel=&quot;nofollow noreferrer&quot;&gt;minihashes&lt;/a&gt; as a more efficient way of clustering and comparing your sequences to known species and samples. &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-0997-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;In the mash paper&lt;/a&gt; the authors show how it would be feasible to create minihash sketches that you could search against from large metagenomics datasets (check the &quot;Clustering massive metagenomic datasets&quot; section in the results):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For a large-scale test, samples from the Human Microbiome Project [36]&#xA;  (HMP) and Metagenomics of the Human Intestinal Tract [37] (MetaHIT)&#xA;  were combined to create a ~10 TB 888-sample dataset. Importantly, the&#xA;  size of a Mash sketch is independent of the input size, requiring only&#xA;  70 MB to store the combined sketches (s = 10,000, k = 21) for these&#xA;  datasets.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Moreover, the creators of &lt;a href=&quot;https://sourmash.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;sourmash&lt;/a&gt; (a tool using the same approach) provide &lt;a href=&quot;https://sourmash.readthedocs.io/en/latest/databases.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;precomputed sketches for the whole of NCBI's RefSeq database&lt;/a&gt; (includes bacterial, viral and fungal genomes). Creating a similar sketch for the whole of ENA/SRA (or a subset of it) could require some time but would allow for very fast and accurate searches.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2017-08-03T21:38:09.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2215" PostTypeId="1" AcceptedAnswerId="2218" CreationDate="2017-08-04T13:17:53.547" Score="9" ViewCount="1851" Body="&lt;p&gt;I have a set of RNA-seq samples from different experiments (Single and Paired End, depending on the experiment). I ran &lt;a href=&quot;https://www.bioinformatics.babraham.ac.uk/projects/fastqc/&quot; rel=&quot;noreferrer&quot;&gt;FASTQC&lt;/a&gt; in all the samples and found overrepresented adapter sequences:&lt;a href=&quot;https://i.stack.imgur.com/RlZlZ.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RlZlZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I removed the adaptors (TruSeq adaptors) using &lt;a href=&quot;http://cutadapt.readthedocs.io/en/stable/guide.html&quot; rel=&quot;noreferrer&quot;&gt;Cutadapt&lt;/a&gt; (in addition, I removed low quality and N bases from the 3' end of the reads). After that, I ran again FASTQC and the output is the following (representative example) :&#xA;&lt;a href=&quot;https://i.stack.imgur.com/tpgVR.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tpgVR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know what is happening? Now I have an overrepresented sequence for which no sequence is provided. What does this mean? &lt;/p&gt;&#xA;" OwnerUserId="678" LastActivityDate="2017-08-04T17:36:59.603" Title="FASTQC overrepresented sequences after trimming" Tags="|ngs|fastq|trimming|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2216" PostTypeId="1" AcceptedAnswerId="2217" CreationDate="2017-08-04T15:10:40.400" Score="27" ViewCount="6614" Body="&lt;p&gt;A bit of a historical question on a number, 30 times coverage, that's become so familiar in the field: why do we sequence the human genome at 30x coverage?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question has two parts:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Who came up with the 30x value and why?&lt;/li&gt;&#xA;&lt;li&gt;Does the value need to be updated to reflect today's state-of-the-art?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In summary, if the 30x value is a number that was based on the old Solexa GAIIx 2x35bp reads and error rates, and the current standard Illumina sequencing is 2x150bp, does the 30x value need updating?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GGkq0.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GGkq0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="180" LastEditDate="2017-08-10T10:08:24.777" LastActivityDate="2017-08-10T18:12:40.087" Title="Why sequence the human genome at 30x coverage?" Tags="|genome|ngs|phylogenetics|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2217" PostTypeId="2" ParentId="2216" CreationDate="2017-08-04T15:46:51.593" Score="30" Body="&lt;p&gt;The earliest mention of the 30x paradigm I could find is in the original Illumina whole-genome sequencing paper: &lt;a href=&quot;https://www.nature.com/nature/journal/v456/n7218/full/nature07517.html&quot; rel=&quot;noreferrer&quot;&gt;Bentley, 2008&lt;/a&gt;. Specifically, in Figure 5, they show that most SNPs have been found, and that there are few uncovered/uncalled bases by the time you reach 30x: &lt;a href=&quot;https://i.stack.imgur.com/42eHQ.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/42eHQ.png&quot; alt=&quot;30xSequencingDepth&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These days, 30x is still a common standard, but large-scale germline sequencing projects are often pushing down closer to 25x and finding it adequate. Every group doing this seriously has done power calculations based on specifics of their machines and prep (things like error rates and read lengths matter!).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cancer genomics is going in the other direction. When you have to contend with purity, ploidy, and subclonal populations, much more coverage than 30x is needed. Our group showed in &lt;a href=&quot;http://www.cell.com/cell-systems/abstract/S2405-4712(15)00113-1&quot; rel=&quot;noreferrer&quot;&gt;this 2015 paper&lt;/a&gt; that even 300x whole-genome coverage of a tumor was likely missing real rare variants in a tumor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the whole, the sequence coverage you need really depends on what questions you're asking, and I'd recommend that anyone designing a sequencing experiment consult with both a sequencing expert and a statistician beforehand (and it's even better if those are the same person!)&lt;/p&gt;&#xA;" OwnerUserId="74" LastActivityDate="2017-08-04T15:46:51.593" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2218" PostTypeId="2" ParentId="2215" CreationDate="2017-08-04T17:36:59.603" Score="6" Body="&lt;p&gt;As @AaronBerlin mentioned, you didn't remove reads that were completely trimmed. Next time use the &lt;code&gt;--minimum-length&lt;/code&gt; option and set it to something reasonable, like 20. Alternatively, use &quot;Trim Galore!&quot;, which is a wrapper around cutadapt that has more reasonable defaults.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-08-04T17:36:59.603" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2219" PostTypeId="2" ParentId="97" CreationDate="2017-08-04T19:08:28.877" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/bede/kindel&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kindel&lt;/a&gt; (which I wrote) can infer consensus from low quality alignments of short reads to viral references, and extending it to work with single molecule reads and larger genomes is on my to-do list, though I imagine this will require some redesign.&lt;br&gt;&#xA;Presumably you're dealing with a bacterial or fungal genome in this case ? I do also have a basic C++ version, but it's a long way away from being user friendly. Anyway, it may be worth a look – feel free to get in touch with any issues you encounter. I'd use Pilon as mentioned above&lt;/p&gt;&#xA;" OwnerUserId="1258" LastEditorUserId="48" LastEditDate="2017-11-10T16:48:43.307" LastActivityDate="2017-11-10T16:48:43.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2220" PostTypeId="2" ParentId="2216" CreationDate="2017-08-04T20:51:38.387" Score="6" Body="&lt;p&gt;30 times coverage is not unique to this problem, but number &lt;strong&gt;30&lt;/strong&gt; has its empirical role in statistics:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In statistical analysis, the rule of three states that if a certain event did not occur in a sample with n subjects, the interval from 0 to 3/n is a 95% confidence interval for the rate of occurrences in the population. &lt;strong&gt;When n is greater than 30, this is a good approximation to results from more sensitive tests.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;source: &lt;a href=&quot;https://en.wikipedia.org/wiki/Rule_of_three_(statistics)&quot; rel=&quot;noreferrer&quot;&gt;Wikipedia: Rule of three (statistics)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly, you can search for related questions like this one:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.researchgate.net/post/What_is_the_rationale_behind_the_magic_number_30_in_statistics&quot; rel=&quot;noreferrer&quot;&gt;What is the rationale behind the magic number 30 in statistics?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In line with this, I have seen data processing in other disciplines which required &lt;strong&gt;n ≥ 30&lt;/strong&gt; for sufficient reliability of results.&lt;/p&gt;&#xA;" OwnerUserId="1267" LastActivityDate="2017-08-04T20:51:38.387" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2222" PostTypeId="2" ParentId="2216" CreationDate="2017-08-06T03:27:17.950" Score="7" Body="&lt;p&gt;Solexa Inc. sequenced NA12878 chrX to ~30x in early 2007, which later became part of &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/18987734&quot; rel=&quot;noreferrer&quot;&gt;Bentley (2008)&lt;/a&gt;. This, I believe, was the first time that 30x showed up. I don't recall they had a particular reason for that. Figure 5 in the published paper was just aftermath. It does not really explain why not 25x or 35x, given that the curves between 25x and 35x in that figure are about linear.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the abstract of &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/21771779&quot; rel=&quot;noreferrer&quot;&gt;Ajay et al (2011)&lt;/a&gt;, the authors argued &quot;the current recommendation of ~30x coverage is not adequate&quot;. Nonetheless, the discussion section seems to suggest 50–60x would be necessary with GAIIx, but 35x was adequate with HiSeq2000 plus better recent chemistry. Overall, this paper provides a more thorough analysis. The data quality at that time is also closer to data we produce today.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The required coverage is largely determined by two factors: read placement bias (e.g. GC bias) and base/mapping error rate. While GC bias has been reduced with the PCR-free protocol, base error rate has been going the downward since HiSeq2500. I guess 30x coverage would be necessary if you want to achieve the sensitivity with the older 30x data. Illumina, as a sequencing service provider, and our sequencing facility still insist on the 30x threshold.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-08-06T03:27:17.950" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2223" PostTypeId="1" AcceptedAnswerId="2228" CreationDate="2017-08-06T19:26:16.787" Score="8" ViewCount="1410" Body="&lt;p&gt;I have some software which takes fastas as the input. I need to include SNVs and InDels from a VCF into the reference hg38 and then use this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is, I don't know of an algorithmically sound way to do this.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there any existing software packages which could do this efficiently? Is it easier to output a FASTA, or a bam (and then convert to a FASTA)? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What about if I wanted to do the same with a bedpe of germline structural variants? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="146" LastEditorUserId="96" LastEditDate="2019-01-15T14:26:29.020" LastActivityDate="2019-01-15T20:10:30.333" Title="How to manipulate a reference FASTA or bam to include variants from a VCF?" Tags="|sam|fasta|vcf|reference-genome|variation|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2224" PostTypeId="1" AcceptedAnswerId="2225" CreationDate="2017-08-06T19:47:22.410" Score="3" ViewCount="518" Body="&lt;p&gt;I'm trying to use Annovar to annotate some variants with their CADD and FATHMM scores. I've downloaded the latest versions of the software and the databases but when I run it I get an error saying the index is out of date.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;WARNING: Your index file annovar_latest/humandb/hg19_cadd.txt.idx is out of date and will not be used. ANNOVAR can still generate correct results without index file.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It still runs without the index file but it is extremely slow. How can I update the index file?&lt;/p&gt;&#xA;" OwnerUserId="59" LastEditorUserId="2719" LastEditDate="2018-05-21T22:31:57.277" LastActivityDate="2018-05-21T22:31:57.277" Title="Annovar index out of date" Tags="|sequence-annotation|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2225" PostTypeId="2" ParentId="2224" CreationDate="2017-08-06T20:36:58.513" Score="2" Body="&lt;p&gt;Your safest bet is to just &lt;a href=&quot;http://annovar.openbioinformatics.org/en/latest/user-guide/download/#additional-databases&quot; rel=&quot;nofollow noreferrer&quot;&gt;redownload the annotation&lt;/a&gt;. I've not seen official documentation of the index used by annovar, but apparently it's just a text file with a single line header and chromosome-bin coordinates. Over on SEQanswers, &lt;a href=&quot;http://seqanswers.com/forums/showthread.php?t=23535&quot; rel=&quot;nofollow noreferrer&quot;&gt;there's a thread discussing this issue&lt;/a&gt; with a single &lt;a href=&quot;http://seqanswers.com/forums/attachment.php?attachmentid=3818&amp;amp;d=1432814490&quot; rel=&quot;nofollow noreferrer&quot;&gt;perl script&lt;/a&gt; that allegedly recreates the index.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-08-06T20:36:58.513" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2226" PostTypeId="2" ParentId="2223" CreationDate="2017-08-06T20:38:17.907" Score="7" Body="&lt;p&gt;GATK has a solution that might work for you:&#xA;&lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/tooldocs/current/org_broadinstitute_gatk_tools_walkers_fasta_FastaAlternateReferenceMaker.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;FastaAlternateReferenceMaker&lt;/a&gt;, which : &quot;Given a variant callset, this tool replaces the reference bases at variation sites with the bases supplied in the corresponding callset records.&quot; &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Input&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The reference, requested intervals, and any number of variant ROD files.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;A FASTA file representing the requested intervals.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Usage example&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar \&#xA;   -T FastaAlternateReferenceMaker \&#xA;   -R reference.fasta \&#xA;   -o output.fasta \&#xA;   -L input.intervals \&#xA;   -V input.vcf \&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="59" LastEditorUserId="48" LastEditDate="2017-11-13T22:48:19.573" LastActivityDate="2017-11-13T22:48:19.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2227" PostTypeId="2" ParentId="2223" CreationDate="2017-08-06T21:23:14.093" Score="3" Body="&lt;p&gt;There's a &lt;code&gt;vcf2fq&lt;/code&gt; &lt;a href=&quot;https://github.com/lh3/samtools/blob/master/bcftools/vcfutils.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;sub-program&lt;/a&gt; that was written as part of vcfutils to convert a VCF file into a fastq file given a reference sequence. Unfortunately this doesn't work properly with INDELs (it will just mask them, rather than actually converting them), so I wrote &lt;a href=&quot;https://github.com/gringer/bioinfscripts/blob/master/vcf2fq.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;a modification&lt;/a&gt; to implement INDEL correction as well:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./vcf2fq.pl -f &amp;lt;input.fasta&amp;gt; &amp;lt;all-site.vcf&amp;gt; &amp;gt; &amp;lt;output.fastq&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It adds and removes sequence from the reference, as specified by the INDEL information in the VCF file. The code keeps a record of where in the reference sequence the INDELs should go, then inserts (or deletes) the INDELs, updating the reference position as necessary. It should work with things like deletions right next to insertions, but hasn't been extensively tested.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-11-10T05:49:52.723" LastActivityDate="2017-11-10T05:49:52.723" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2228" PostTypeId="2" ParentId="2223" CreationDate="2017-08-06T21:47:56.003" Score="4" Body="&lt;p&gt;You could &lt;a href=&quot;http://bedops.readthedocs.io/en/latest/content/reference/file-management/conversion/vcf2bed.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;convert VCF to BED&lt;/a&gt; via &lt;code&gt;vcf2bed --snvs&lt;/code&gt;, &lt;code&gt;vcf2bed --insertions&lt;/code&gt;, and &lt;code&gt;vcf2bed --deletions&lt;/code&gt;, and then use &lt;code&gt;samtools faidx&lt;/code&gt; by way of a &lt;a href=&quot;https://gist.github.com/alexpreynolds/fa9b0f90e181e3b4f640&quot; rel=&quot;nofollow noreferrer&quot;&gt;wrapper script&lt;/a&gt; to convert BED to FASTA, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ vcf2bed --snvs &amp;lt; variants.vcf | bed2faidxsta.pl &amp;gt; snvs.fa&#xA;$ vcf2bed --insertions &amp;lt; variants.vcf | bed2faidxsta.pl &amp;gt; insertions.fa&#xA;$ vcf2bed --deletions &amp;lt; variants.vcf | bed2faidxsta.pl &amp;gt; deletions.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You need FASTA files for your reference genome, which have been indexed with &lt;code&gt;samtools faidx&lt;/code&gt;, e.g., for &lt;code&gt;hg38&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cd /foo/bar/baz&#xA;$ wget ftp://hgdownload.cse.ucsc.edu/goldenPath/hg38/chromosomes/*.fa.gz&#xA;$ for fn in `ls *.fa.gz`; do gunzip $fn; done&#xA;$ for fn in `ls *.fa`; do samtools faidx $fn; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Once you have indexed FASTA files somewhere on your file system, you can pipe BED to the &lt;code&gt;bed2faidxsta.pl&lt;/code&gt; script, to get out FASTA sequences.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2017-08-07T16:41:45.307" LastActivityDate="2017-08-07T16:41:45.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2229" PostTypeId="2" ParentId="658" CreationDate="2017-08-06T22:47:16.617" Score="0" Body="&lt;p&gt;There are tonnes of them. On top of the excellent ones everybody mentioned&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;iRods&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Arvados&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Galaxy&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1280" LastActivityDate="2017-08-06T22:47:16.617" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2230" PostTypeId="1" AcceptedAnswerId="2231" CreationDate="2017-08-07T09:44:43.227" Score="6" ViewCount="954" Body="&lt;p&gt;I am introducing SNVs into specific samples in order to estimate false negative rates for a variant calling pipeline. I know reads can be simulated but I would actually prefer to use the real data so as to keep everything else equal.&lt;/p&gt;&#xA;&lt;p&gt;Given a read ID, I want to edit a single basecall (e.g. the 12th base) for just that read from within a large FASTQ file containing millions of reads.&lt;/p&gt;&#xA;&lt;p&gt;example, i want to change the 12th base ('C') in read 31027 to a 'T':&lt;/p&gt;&#xA;&lt;pre&gt;@70630 1:N:0:ATCACG&#xA;GAAGGTCCATGGATAATACTCAATTTTCCACAACAGCTTTTGTACTCTAGATCATTGATATTTACCAAAAGTCACTCAAACTCATCCTATGCATAATTCTAGTCCACCAATCATGATATGATGGAGAACATGGTTGTAATCAGGAAGACAG&#xA;+&#xA;DDDDDHIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIHIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIGHHIIIIIIIIIIIIIIIIIIIIIIHIIHIHIIIHHIGIIIIHHHHIIH@&#xA;@31027 1:N:0:ATCACG&#xA;CAAAAGTCACTCAAACTCATCCTATGCATAATTCTAGTCCACCAATCATGATATGATGGAGAACATGGTTGTAATCAGGAAGACAGATAAAGCAGCAGACCAAAAGTAATCTGAGAAATTATATTTGAATCACTCAGATATACATCAAATA&#xA;+&#xA;DDBDDHC@GHFHHIIE@CEHHIHEFFCGFCH?HHEGFCCEHHCGH@HFHHHCHHIIIHEHIIHII@CECHIHIIIEECCDGEHFFHHHHEHHFHHGGIHHHDGHFHIIIGHHHHHHEHHIIHIIIGGHHHCHHHCHIIIHHIEH@GHIHIC&#xA;@87319 1:N:0:ATCACG&#xA;CAATTAAGCTTTGGCAACGGTGGTCAAGATGAGATGCATATGGAGATAATAACTAAAAGTCAATCGAGACTCATCGTATGCATATTTCTAGTCCATCGATCATGAAATGATAGGATAGCTAGAATGAAAAGTAAATTTCCAGAAGGTCCAT&#xA;+&#xA;DDDDDIIIIIIIIIIIIIHIHHIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIHHIIIIIIIIIIIIIIIIIHIIHIIIIIIIIIIIIHIIIIIIIIIHIHIH&#xA;&lt;/pre&gt;&#xA;&lt;p&gt;Of course I could stream through the entire FASTQ file rewriting everything to a new file until i get to the read of interest, do the edits to that read, write it out to the new file, and then keep going until the end. BioPython would work fine for this as would other approaches.&lt;/p&gt;&#xA;&lt;p&gt;However, is there an efficient way to do this without the full read/write stream? Could this be done with sed/awk ? What about an indexed fastq file?&lt;/p&gt;&#xA;&lt;p&gt;cheers&#xA;David&lt;/p&gt;&#xA;" OwnerUserId="1281" LastEditorUserId="73" LastEditDate="2023-01-15T20:27:43.183" LastActivityDate="2023-01-15T20:27:43.183" Title="How can I edit a specific FASTQ read in place, given the read ID?" Tags="|python|ngs|fastq|awk|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2231" PostTypeId="2" ParentId="2230" CreationDate="2017-08-07T09:52:10.770" Score="5" Body="&lt;p&gt;You'll first need to determine the appropriate line number, which you can do with &lt;code&gt;grep -m1 -nw &quot;@31027&quot; foo.fastq&lt;/code&gt;. After that, note that you can provide a line number to &lt;code&gt;sed&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -i '123456s/CAAAAGTCACTCA/CAAAAGTCACTTA/' foo.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That will do the replacement only on line 123456 and edit the file in place (the &lt;code&gt;-i&lt;/code&gt; option).&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="298" LastEditDate="2017-08-07T11:48:56.240" LastActivityDate="2017-08-07T11:48:56.240" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2232" PostTypeId="5" CreationDate="2017-08-07T11:06:35.623" Score="0" Body="&lt;p&gt;FASTQ (&lt;a href=&quot;https://en.wikipedia.org/wiki/FASTQ_format&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/FASTQ_format&lt;/a&gt;) is a file format use to store short reads and their quality values. Use this tag for questions related to fastq file format.&lt;/p&gt;&#xA;" OwnerUserId="1140" LastEditorUserId="1140" LastEditDate="2017-08-07T11:37:41.333" LastActivityDate="2017-08-07T11:37:41.333" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2233" PostTypeId="4" CreationDate="2017-08-07T11:06:35.623" Score="0" Body="FASTQ is a file format use to store short reads and their quality values" OwnerUserId="1140" LastEditorUserId="1140" LastEditDate="2017-08-07T11:37:44.543" LastActivityDate="2017-08-07T11:37:44.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2234" PostTypeId="1" AcceptedAnswerId="2236" CreationDate="2017-08-06T20:33:05.040" Score="5" ViewCount="285" Body="&lt;p&gt;I wish to adapt the r language function fgseaL, &lt;a href=&quot;https://github.com/ctlab/fgsea&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/ctlab/fgsea&lt;/a&gt; , to perform rapidGSEA, &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-016-1244-x&quot; rel=&quot;noreferrer&quot;&gt;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-016-1244-x&lt;/a&gt; , computation of inter-class  deviation per gene and the subsequent gene rank sorting operation on 9 different phenotype labels as illustrated in the diagram immediately below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kXvmr.gif&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kXvmr.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought of applying the R-language rank() function on the Expression Data Matrix D. If that is not correct, what sequence of R language commands should we&#xA;apply to the Expression Data Matrix D to calculate a key value sorted deviation measure  across 8 labeled human leukemia groups and a healthy labeled normal control group prior to running fgseaL? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I show below how fgseaL finds the correlation matrix between the R language variable , mat, which corresponds to the Expression Data Matrix D and the R language variable , labels , which is a vector of gene phenotype labels&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    tmatSc &amp;lt;- scale(t(mat))&#xA;    labelsSc &amp;lt;- scale(labels)[, 1]&#xA;&#xA;    minSize &amp;lt;- max(minSize, 1)&#xA;&#xA;    pathwaysFiltered &amp;lt;- lapply(pathways, function(p) { as.vector(na.omit(fmatch(p, rownames(mat)))) })&#xA;    pathwaysSizes &amp;lt;- sapply(pathwaysFiltered, length)&#xA;&#xA;    toKeep &amp;lt;- which(minSize &amp;lt;= pathwaysSizes &amp;amp; pathwaysSizes &amp;lt;= maxSize)&#xA;    m &amp;lt;- length(toKeep)&#xA;&#xA;    if (m == 0) {&#xA;        return(data.table(pathway=character(),&#xA;                          pval=numeric(),&#xA;                          padj=numeric(),&#xA;                          ES=numeric(),&#xA;                          NES=numeric(),&#xA;                          nMoreExtreme=numeric(),&#xA;                          size=integer(),&#xA;                          leadingEdge=list()))&#xA;    }&#xA;&#xA;    pathwaysFiltered &amp;lt;- pathwaysFiltered[toKeep]&#xA;    pathwaysSizes &amp;lt;- pathwaysSizes[toKeep]&#xA;&#xA;    corRanks &amp;lt;- var(tmatSc, labelsSc)[,1]&#xA;    ranksOrder &amp;lt;- order(corRanks, decreasing=T)&#xA;    ranksOrderInv &amp;lt;- invPerm(ranksOrder)&#xA;    stats &amp;lt;- corRanks[ranksOrder]&#xA;&#xA;    pathwaysReordered &amp;lt;- lapply(pathwaysFiltered, function(x) ranksOrderInv[x])&#xA;&#xA;    gseaStatRes &amp;lt;- do.call(rbind,&#xA;                           lapply(pathwaysReordered, calcGseaStat,&#xA;                                  stats=stats,&#xA;                                  returnLeadingEdge=TRUE))&#xA;&#xA;I found a problem with the algorithm shown immediately below.&#xA;correcttest &amp;lt;- data.frame(names = row.names(normal))&#xA;correcttest &amp;lt;- cbind(correcttest3, normal)&#xA;correcttest &amp;lt;- cbind(correcttest3, ALL3m)&#xA;rownames(correcttest) &amp;lt;- correcttest$names&#xA;correcttest$names &amp;lt;- NULL&#xA;correctlabelnormal &amp;lt;- rep(0:0, 73)&#xA;correctlabelALL3m &amp;lt;- rep(1:1, 122)&#xA;correctlabel &amp;lt;- as.vector(c(correctlabelnormal,correctlabelALL3m))&#xA;s &amp;lt;- apply(correcttest, 1, function(x) coef(lm(x~correctlabel))[2])&#xA;o &amp;lt;- rank(s)&#xA;o &amp;lt;- max(o) - o + 1&#xA;res &amp;lt;- fgseaL(df,o,correctlabel,nperm = 2000,minSize = 1, maxSize=50000)&#xA;empty data table (0 rows) of 8 columns:   pathway,pval,padj,ES,NES,nMoreExtreme,size&#xA;&#xA;I found the binary phenotype labeled group fgseaL test results below looked satisfactory.    &#xA;correcttest &amp;lt;- data.frame(names = row.names(normal))&#xA;correcttest &amp;lt;- cbind(correcttest3, normal)&#xA;correcttest &amp;lt;- cbind(correcttest3, ALL3m)&#xA;rownames(correcttest) &amp;lt;- correcttest$names&#xA;correcttest$names &amp;lt;- NULL&#xA;correctlabelnormal &amp;lt;- rep(0:0, 73)&#xA;correctlabelALL3m &amp;lt;- rep(1:1, 122)&#xA;correctlabel &amp;lt;- as.vector(c(correctlabelnormal,correctlabelALL3m))&#xA;fgseaL(df,correcttest,correctlabel,nperm = 2000,minSize = 1, maxSize=50000)&#xA;       pathway        pval        padj         ES       NES nMoreExtreme  size&#xA;1: Gene.Symbol 0.003940887 0.003940887 -0.2460126 -1.180009            3 45714&#xA;                                 leadingEdge&#xA;1: AKIRIN2,LRRC20,HSPA5,HSPA5,DTWD2,ZFYVE28,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thank you for your consideration.&lt;/p&gt;&#xA;" OwnerUserId="1283" OwnerDisplayName="Frank" LastEditorUserId="1283" LastEditDate="2017-08-08T00:28:04.583" LastActivityDate="2017-08-23T16:16:10.837" Title="How to adapt the fgseaL function to perform rapidGSEA computation of gene ranks across 9 different phenotype labels?" Tags="|r|genomics|gsea|" AnswerCount="3" CommentCount="12" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2235" PostTypeId="2" ParentId="2230" CreationDate="2017-08-07T11:50:46.357" Score="2" Body="&lt;p&gt;If your fastq file is simple, if each of your reads only has a single line of DNA, you could do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{&#xA;        if($1 == &quot;@31027&quot;){&#xA;            a=NR+1&#xA;        } &#xA;        if(NR==a){&#xA;            split($0,s,&quot;&quot;); &#xA;            s[12]=&quot;T&quot;; &#xA;            for(i in s){&#xA;                seq = sprintf(&quot;%s%s&quot;,seq,s[i]);&#xA;            } &#xA;            $0=seq&#xA;        }&#xA;    }1;' file.fastq &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-08-07T11:58:19.077" LastActivityDate="2017-08-07T11:58:19.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2236" PostTypeId="2" ParentId="2234" CreationDate="2017-08-07T12:14:39.873" Score="2" Body="&lt;p&gt;You rank the fit coefficient rather than the original score matrix. So, given a score matrix, &lt;code&gt;D&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;D = matrix(c(22,20,9,8,46,22,18,10,3,18,3,29,2,1,5,45,43,47,17,5,14,44,21,36), byrow=T, ncol=6)&#xA;cl = c(0,0,0,1,1,1)&#xA;s = apply(m, 1, function(x) coef(lm(x~cl))[2]) # [1]&#xA;o = rank(s)&#xA;o = max(o) - o + 1 # [2]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;o&lt;/code&gt; is then the rank of each row.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] This fits each row as a linear model of &lt;code&gt;cl&lt;/code&gt; and extracts the &lt;code&gt;cl&lt;/code&gt; coefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[2] This converts the ranking to be the same as shown in the figure. I don't know if this is important, but I would assume so given how GSEA methods tend to work.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-08-07T12:14:39.873" CommentCount="17" ContentLicense="CC BY-SA 3.0" />
  <row Id="2237" PostTypeId="1" AcceptedAnswerId="2239" CreationDate="2017-08-07T14:57:32.910" Score="4" ViewCount="493" Body="&lt;p&gt;I am using the following command to get all refseq genes from UCSC:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/usr/bin/mysql --user=genomep --password=password --host=genome-mysql.cse.ucsc.edu \&#xA;    -A -D hg38 -e 'select concat(t.name, &quot;.&quot;, i.version) name, \&#xA;    k.locusLinkId as &quot;EntrezId&quot;, t.chrom, t.strand, t.txStart, \&#xA;    t.txEnd, t.cdsStart, t.cdsEnd, t.exonCount, t.exonStarts, \&#xA;    t.exonEnds, t.score, t.name2 from refGene t join hgFixed.gbCdnaInfo i \&#xA;    on t.name = i.acc join hgFixed.refLink k on t.name = k.mrnaAcc'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That returns data in the following format (showing the 1st 5 lines):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;+-------------+-----------+-------+--------+---------+-------+----------+--------+-----------+--------------------+--------------------+-------+-----------+&#xA;| name        | EntrezId  | chrom | strand | txStart | txEnd | cdsStart | cdsEnd | exonCount | exonStarts         | exonEnds           | score | name2     |&#xA;+-------------+-----------+-------+--------+---------+-------+----------+--------+-----------+--------------------+--------------------+-------+-----------+&#xA;| NR_046018.2 | 100287102 | chr1  | +      |   11873 | 14409 |    14409 |  14409 |         3 | 11873,12612,13220, | 12227,12721,14409, |     0 | DDX11L1   |&#xA;| NR_106918.1 | 102466751 | chr1  | -      |   17368 | 17436 |    17436 |  17436 |         1 | 17368,             | 17436,             |     0 | MIR6859-1 |&#xA;| NR_107062.1 | 102465909 | chr1  | -      |   17368 | 17436 |    17436 |  17436 |         1 | 17368,             | 17436,             |     0 | MIR6859-2 |&#xA;| NR_107063.1 | 102465910 | chr1  | -      |   17368 | 17436 |    17436 |  17436 |         1 | 17368,             | 17436,             |     0 | MIR6859-3 |&#xA;| NR_128720.1 | 103504738 | chr1  | -      |   17368 | 17436 |    17436 |  17436 |         1 | 17368,             | 17436,             |     0 | MIR6859-4 |&#xA;+-------------+-----------+-------+--------+---------+-------+----------+--------+-----------+--------------------+--------------------+-------+-----------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also want to find the accession of the canonical transcript of each of the genes returned by the command above. Those seem to be stored in the &lt;code&gt;knownCanonical&lt;/code&gt; table:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ /usr/bin/mysql --user=genomep --password=password --host=genome-mysql.cse.ucsc.edu -A -D hg38 -e 'select * from knownCanonical limit 5'&#xA;+-------+------------+-----------+-----------+------------+--------------------+&#xA;| chrom | chromStart | chromEnd  | clusterId | transcript | protein            |&#xA;+-------+------------+-----------+-----------+------------+--------------------+&#xA;| chrX  |  100628669 | 100636806 |         1 | uc004ega.3 | ENSG00000000003.14 |&#xA;| chrX  |  100584801 | 100599885 |         2 | uc004efy.5 | ENSG00000000005.5  |&#xA;| chr20 |   50934866 |  50958550 |         3 | uc002xvw.2 | ENSG00000000419.12 |&#xA;| chr1  |  169853073 | 169893959 |         4 | uc001ggs.5 | ENSG00000000457.13 |&#xA;| chr1  |  169795048 | 169854080 |         5 | uc001ggp.4 | ENSG00000000460.16 |&#xA;+-------+------------+-----------+-----------+------------+--------------------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, there seems to be no obvious way to link the &lt;code&gt;knownCanonical&lt;/code&gt; table to the &lt;code&gt;refGene&lt;/code&gt;, &lt;code&gt;hgFixed.gbCdnaInfo&lt;/code&gt; and &lt;code&gt;hgFixed.refLink&lt;/code&gt; tables used above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, how can I modify my 1st query (or, if necessary write a new one) so that my results also include the accession of the gene's canonical transcript? &lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="96" LastEditDate="2017-08-08T03:13:26.597" LastActivityDate="2017-08-08T03:13:26.597" Title="Get canonical transcript from UCSC" Tags="|public-databases|identifiers|ucsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2238" PostTypeId="2" ParentId="2186" CreationDate="2017-08-07T15:06:06.557" Score="2" Body="&lt;h2&gt;A (bio)awk-based solution&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;Generate the list of genus&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bioawk -c fastx '{print $comment}' terminase_large.fasta \&#xA;    | sed -r 's/.*\[(\w+).*/\1/' \&#xA;    | sort -u &amp;gt; genus_names.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is how it works:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/lh3/bioawk&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;bioawk&lt;/code&gt;&lt;/a&gt; is like awk but with extra parsing capabilities. &lt;code&gt;-c fastx&lt;/code&gt; parses the fields in fasta or fastq records. &lt;code&gt;{print $comment}&lt;/code&gt; will print the &lt;code&gt;$comment&lt;/code&gt; field which contain the genus names you want to extract.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;code&gt;sed&lt;/code&gt; command captures (with parentheses) the word (&lt;code&gt;\w+&lt;/code&gt;) that is just after the opening square bracket (which should be the genus name, if there is only one opening square bracket on the line), and substitutes the whole line (that is, everything before the bracket (&lt;code&gt;.*&lt;/code&gt;), the bracket (&lt;code&gt;\[&lt;/code&gt;), the word, and everything after the word (&lt;code&gt;.*&lt;/code&gt; again)) with just the captured word (&lt;code&gt;\1&lt;/code&gt;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;All this is sorted, and the &lt;code&gt;-u&lt;/code&gt; option of &lt;code&gt;sort&lt;/code&gt; ensures there is only one occurrence of each genus name in the output, which we put in the &lt;code&gt;genus_names.txt&lt;/code&gt; file.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h3&gt;Extract the fasta records for each genus&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for genus in $(cat genus_names.txt)&#xA;do&#xA;    bioawk -c fastx '{print}' terminase_large.fasta \&#xA;        | grep ${genus} \&#xA;        | awk -F &quot;\t&quot; '{print &quot;&amp;gt;&quot;$4&quot;\n&quot;$2}' \&#xA;        | sed -r 's/^&amp;gt;[^\[]+\[(.*)\]/&amp;gt;\1/' &amp;gt; ${genus}.fasta&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is how it works:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;We loop on the content of the previously established list of genus names.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For each value of &lt;code&gt;${genus}&lt;/code&gt;, we parse again the fasta file with &lt;code&gt;bioawk&lt;/code&gt;. This time, for a given fasta record, we print all the elements that have been parsed. They appear on one line, in the following order (according to &lt;code&gt;bioawk -c help&lt;/code&gt;): &lt;code&gt;1:name 2:seq 3:qual 4:comment&lt;/code&gt; (we actually only need &lt;code&gt;seq&lt;/code&gt; and &lt;code&gt;comment&lt;/code&gt; for the following steps, but there are less risks of making errors at the later &lt;code&gt;awk&lt;/code&gt; step if we keep the four fields).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We use &lt;code&gt;grep&lt;/code&gt; to select the resulting lines that contain the genus name we're currently dealing with.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The lines that have been successfully selected are then parsed by awk and reformatted into fasta format using only the comment part in the header. The &lt;code&gt;-F &quot;\t&quot;&lt;/code&gt; is to indicate that the field delimiters are tabulations (which is how bioawk has written the records when we did &lt;code&gt;{print}&lt;/code&gt;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We use &lt;code&gt;sed&lt;/code&gt; to keep only the part inside the square brackets (again using a pattern capture between parentheses, that we use in the substitution part with &lt;code&gt;\1&lt;/code&gt;), and the output is written in a file named using the current value of &lt;code&gt;${genus}&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;More details on the last &lt;code&gt;sed&lt;/code&gt; command:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We match the whole header lines. They start with &quot;&gt;&quot; (&lt;code&gt;^&amp;gt;&lt;/code&gt;), then there are some non-&quot;[&quot; characters (&lt;code&gt;[^\[]+&lt;/code&gt;), then a &quot;[&quot; (&lt;code&gt;\[&lt;/code&gt;), then something we capture (&lt;code&gt;(.*)&lt;/code&gt;), then a &quot;]&quot; (&lt;code&gt;\]&lt;/code&gt;). We replace them with the captured part. The sequence lines are unaffected because they don't match the above regular expression.&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2017-08-09T12:18:26.183" LastActivityDate="2017-08-09T12:18:26.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2239" PostTypeId="2" ParentId="2237" CreationDate="2017-08-07T21:02:46.693" Score="3" Body="&lt;p&gt;You can link them with the &lt;code&gt;kgXref&lt;/code&gt; table, since &lt;code&gt;kgXref.refseq == refGene.name&lt;/code&gt; and &lt;code&gt;kgXref.kgID == knownCanonical.transcript&lt;/code&gt;. Since it seems that &lt;code&gt;knownCanonical.transcript&lt;/code&gt; is what you want anyway, you don't even need to join on it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mysql --user=genomep --password=password --host=genome-mysql.cse.ucsc.edu \&#xA;-A -D hg38 -e 'select concat(t.name, &quot;.&quot;, i.version) name, x.kgID, \&#xA;k.locusLinkId as &quot;EntrezId&quot;, t.chrom, t.strand, t.txStart, \&#xA;t.txEnd, t.cdsStart, t.cdsEnd, t.exonCount, t.exonStarts, \&#xA;t.exonEnds, t.score, t.name2 from refGene t join (hgFixed.gbCdnaInfo i, hgFixed.refLink k, kgXref x) \&#xA;on (t.name = i.acc and t.name = k.mrnaAcc and t.name = x.refseq) limit 10'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output is then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;+-------------+------------+-----------+-------+--------+---------+--------+----------+--------+-----------+--------------------+--------------------+-------+------------+&#xA;| name        | kgID       | EntrezId  | chrom | strand | txStart | txEnd  | cdsStart | cdsEnd | exonCount | exonStarts         | exonEnds           | score | name2      |&#xA;+-------------+------------+-----------+-------+--------+---------+--------+----------+--------+-----------+--------------------+--------------------+-------+------------+&#xA;| NR_106918.1 | uc031tla.1 | 102466751 | chr1  | -      |   17368 |  17436 |    17436 |  17436 |         1 | 17368,             | 17436,             |     0 | MIR6859-1  |&#xA;| NR_107062.1 | uc031tlm.1 | 102465909 | chr1  | -      |   17368 |  17436 |    17436 |  17436 |         1 | 17368,             | 17436,             |     0 | MIR6859-2  |&#xA;| NR_107063.1 | uc032cta.1 | 102465910 | chr1  | -      |   17368 |  17436 |    17436 |  17436 |         1 | 17368,             | 17436,             |     0 | MIR6859-3  |&#xA;| NR_128720.1 | uc032dmn.1 | 103504738 | chr1  | -      |   17368 |  17436 |    17436 |  17436 |         1 | 17368,             | 17436,             |     0 | MIR6859-4  |&#xA;| NR_036051.1 | uc031tlb.1 | 100302278 | chr1  | +      |   30365 |  30503 |    30503 |  30503 |         1 | 30365,             | 30503,             |     0 | MIR1302-2  |&#xA;| NR_036266.1 | uc033cjs.1 | 100422831 | chr1  | +      |   30365 |  30503 |    30503 |  30503 |         1 | 30365,             | 30503,             |     0 | MIR1302-9  |&#xA;| NR_036267.1 | uc032csz.1 | 100422834 | chr1  | +      |   30365 |  30503 |    30503 |  30503 |         1 | 30365,             | 30503,             |     0 | MIR1302-10 |&#xA;| NR_036268.1 | uc032hiw.1 | 100422919 | chr1  | +      |   30365 |  30503 |    30503 |  30503 |         1 | 30365,             | 30503,             |     0 | MIR1302-11 |&#xA;| NR_026822.1 | uc001aak.4 |    654835 | chr1  | -      |   34610 |  36081 |    36081 |  36081 |         3 | 34610,35276,35720, | 35174,35481,36081, |     0 | FAM138C    |&#xA;| NR_106918.1 | uc031tla.1 | 102466751 | chr1  | -      |  187890 | 187958 |   187958 | 187958 |         1 | 187890,            | 187958,            |     0 | MIR6859-1  |&#xA;+-------------+------------+-----------+-------+--------+---------+--------+----------+--------+-----------+--------------------+--------------------+-------+------------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want the protein ID then join &lt;code&gt;knownCanonical&lt;/code&gt; on &lt;code&gt;x.kgID&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-08-07T21:02:46.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2240" PostTypeId="2" ParentId="525" CreationDate="2017-08-07T21:13:05.843" Score="4" Body="&lt;p&gt;Just for reference, there is a not very in depth, but first-hand reasoning (Jim Kent) given here about why bedGraphToBigWig does not support streaming &lt;a href=&quot;http://genome.soe.ucsc.narkive.com/2S1Z3VpG/bedgraphtobigwig-reading-in-from-stdin&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genome.soe.ucsc.narkive.com/2S1Z3VpG/bedgraphtobigwig-reading-in-from-stdin&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: as noted wigToBigWig allows bedgraph streaming input via stdin but takes (possibly much) more memory than bedGraphToBigWig itself does&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat file.bedgraph | wigToBigWig stdin chrom.sizes output.bigwig&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note: You can also use &lt;code&gt;wiggletools bin 10&lt;/code&gt; for example to bin rows and downsample&lt;/p&gt;&#xA;" OwnerUserId="60" LastEditorUserId="60" LastEditDate="2017-08-09T16:13:38.660" LastActivityDate="2017-08-09T16:13:38.660" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2241" PostTypeId="1" CreationDate="2017-08-07T23:00:43.070" Score="6" ViewCount="346" Body="&lt;p&gt;I am looking the secretome profile and the membrane receptor profile for a given cell type. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my specific case, this should be the secretome and outer membrane receptor profiles of dorsal root ganglion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I've done in the past is taken proteomics datasets or RNAseq datasets and used this as a reference cell surface proteome or secretome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wasn't sure what the best way to filter secreted proteins or membrane receptors were, so I used the Human Protein Atlas and this turned out to be a big headache. A possible alternative method might be to use DAVID, but there must be a more efficient method via the command line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are these:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there a &quot;consensus&quot; database containing protein expression&#xA;profiles for different tissue types in human?&lt;/li&gt;&#xA;&lt;li&gt;If not, what is the best way to filter out the &lt;strong&gt;secreted&lt;/strong&gt; and &lt;strong&gt;transmembrane&lt;/strong&gt; proteins by gene symbol?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="823" LastActivityDate="2017-09-21T09:57:09.233" Title="Secretome and membrane receptor profiles" Tags="|public-databases|networks|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2242" PostTypeId="1" AcceptedAnswerId="2250" CreationDate="2017-08-08T09:59:08.317" Score="17" ViewCount="6080" Body="&lt;p&gt;I would like to convert a &lt;a href=&quot;https://genome.ucsc.edu/FAQ/FAQformat#format1&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;BED&lt;/code&gt;&lt;/a&gt; format to &lt;a href=&quot;http://www.ensembl.org/info/website/upload/gff3.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;GFF3&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only useful tool that I could find via a &lt;a href=&quot;https://www.google.ch/search?q=bed%20to%20gff3&quot; rel=&quot;noreferrer&quot;&gt;google search&lt;/a&gt; seems to be &lt;a href=&quot;https://galaxy.inf.ethz.ch/&quot; rel=&quot;noreferrer&quot;&gt;Galaxy&lt;/a&gt;, and I do not feel very comfortable with online tools, plus the webserver is currenlty under maintenance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone knows about a command-line tool that can handle this conversion?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: here are some lines of my BED file: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ head -4 last_minion-r7_sort.bed&#xA;211000022278137 175 211 8e5d0959-7cdb-49cf-9298-94ed3b2aedb5_Basecall_2D_000_2d 42  +&#xA;211000022279134 0   503 e8a9c6b8-bad2-4a7e-97d8-ca4acb34ff70_Basecall_2D_000_2d 69  -&#xA;211000022279134 24  353 e258783d-95a3-41f5-9ad5-bb12311dbaf4_Basecall_2D_000_2d 45  -&#xA;211000022279134 114 429 26601afb-581a-41df-b42b-b366148ea06f_Basecall_2D_000_2d 100 -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The bed file thus has 6 columns as for now: chromosome, start coordinate, end coordinate, read name, score, strand. This file was obtained from conversion of &lt;code&gt;MAF&lt;/code&gt; format (as output of alignment of RNA-seq reads to reference genome, using &lt;a href=&quot;http://last.cbrc.jp/&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;LAST&lt;/code&gt;&lt;/a&gt;) converted to &lt;code&gt;SAM&lt;/code&gt; using &lt;a href=&quot;http://last.cbrc.jp/doc/maf-convert.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;maf-convert&lt;/code&gt;&lt;/a&gt;, then to &lt;code&gt;BAM&lt;/code&gt; using &lt;a href=&quot;http://samtools.sourceforge.net/&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;samtools&lt;/code&gt;&lt;/a&gt;, finally to &lt;code&gt;BED&lt;/code&gt; using &lt;a href=&quot;http://bedtools.readthedocs.io/en/latest/content/bedtools-suite.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;bedtools&lt;/code&gt;&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The aim of my conversion is basically to convert &lt;code&gt;SAM&lt;/code&gt; -&gt; &lt;code&gt;GTF&lt;/code&gt;, for &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2036/tools-to-reconcile-experimental-transcripts-with-reference-annotation&quot;&gt;post-processing&lt;/a&gt;. Since there is no straightforward way to do this, I am going through steps, the only way to do this in my knowledge is : &lt;code&gt;SAM&lt;/code&gt; -&gt; &lt;code&gt;BAM&lt;/code&gt; -&gt; &lt;code&gt;BED&lt;/code&gt; -&gt; &lt;code&gt;GFF3&lt;/code&gt; -&gt; &lt;code&gt;GTF&lt;/code&gt; but for now I am stuck in the &lt;code&gt;BED&lt;/code&gt; -&gt; &lt;code&gt;GFF3&lt;/code&gt; part. &lt;/p&gt;&#xA;" OwnerUserId="294" LastEditorUserId="294" LastEditDate="2017-08-08T12:40:39.843" LastActivityDate="2023-10-26T12:38:25.933" Title="How to convert BED to GFF3" Tags="|software-recommendation|bed|format-conversion|gff3|" AnswerCount="7" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2243" PostTypeId="2" ParentId="27" CreationDate="2017-08-08T11:02:02.630" Score="0" Body="&lt;p&gt;Just to point out that if you want to follow @Devon Ryan's answer for a different organism/assembly, that is not in his very useful linked resource, you can download NCBI to UCSC contig to chromosome number mappings from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/assembly&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To the site and search for your assembly name. At the bottom of the page is a box called &quot;Global assembly definition&quot; containing a link titled &quot;Download full sequence report&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The downloaded file contains a table with:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Chromosome numbers in &quot;Sequence-Name&quot;/&quot;Assigned-molecule&quot;  &lt;/li&gt;&#xA;&lt;li&gt;NCBI names in Refseq-Accn  &lt;/li&gt;&#xA;&lt;li&gt;UCSC contig name in &quot;UCSC-style-name&quot;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="235" LastActivityDate="2017-08-08T11:02:02.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2244" PostTypeId="2" ParentId="2242" CreationDate="2017-08-08T11:42:33.840" Score="8" Body="&lt;p&gt;To answer the question as asked,  for people googling.&#xA;For BED6, in python:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#contigs.tsv contians chromosome names and lengths in two columns&#xA;for line in open(&amp;quot;contigs.tsv&amp;quot;):&#xA;   fields = line.strip().split(&amp;quot;\t&amp;quot;)&#xA;   print fields[0], &amp;quot;.&amp;quot;, &amp;quot;contig&amp;quot;,&amp;quot;1&amp;quot;,str(fields[1]), &amp;quot;.&amp;quot;, &amp;quot;+&amp;quot;, &amp;quot;.&amp;quot;, &amp;quot;ID=%s&amp;quot; % fields[0]&#xA;&#xA;for line in open(&amp;quot;my_bed_file.bed&amp;quot;):&#xA;   fields = line.strip().split(&amp;quot;\t&amp;quot;)&#xA;&#xA;   # note: BED is 0-based, half-open, GFF is 1-based, closed&#xA;   start = str(int(fields[1]) + 1)&#xA;   print fields[0], &amp;quot;bed&amp;quot;, &amp;quot;interval&amp;quot;, fields[1], fields[2], fields[4], fields[5], &amp;quot;.&amp;quot;, &amp;quot;ID=%s;parent=%s&amp;quot; % (fields[3], fields[0])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For bed12, in python:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#contigs.tsv contians chromosome names and lengths in two columns&#xA;for line in open(&amp;quot;contigs.tsv&amp;quot;):&#xA;   fields = line.strip().split(&amp;quot;\t&amp;quot;)&#xA;   print fields[0], &amp;quot;.&amp;quot;, &amp;quot;contig&amp;quot;,&amp;quot;1&amp;quot;,str(fields[1]), &amp;quot;.&amp;quot;, &amp;quot;+&amp;quot;, &amp;quot;.&amp;quot;, &amp;quot;ID=%s&amp;quot; % fields[0]&#xA;&#xA;for line in open(&amp;quot;my_bed12.bed&amp;quot;):&#xA;&#xA;   fields = line.strip().split(&amp;quot;\t&amp;quot;)&#xA;   contig = fields[0]&#xA;   # note: BED is 0-based, half-open, GFF is 1-based, closed&#xA;   start= int(fields[1]) + 1)&#xA;   end=fields[2]&#xA;   name=fields[3]&#xA;   score=fields[4]&#xA;   strand=fields[5]&#xA;   print contig, &amp;quot;bed&amp;quot;, &amp;quot;interval&amp;quot;, str(start), end, score, strand, &amp;quot;.&amp;quot;, &amp;quot;ID=%s;parent=%s&amp;quot; % (name, contig)&#xA;&#xA;   block_starts = map(int,fields[11].split(&amp;quot;,&amp;quot;))&#xA;   block_sizes = map(int, fields[10].split(&amp;quot;,&amp;quot;))&#xA;   &#xA;   for (block, (bstart, blen)) in enumerate(zip(block_starts, block_sizes)):&#xA;      bend = start + bstart + blen&#xA;      print contig, &amp;quot;bed&amp;quot;, &amp;quot;block&amp;quot;, str(start + bstart), str(bend), score, strand, &amp;quot;.&amp;quot;, &amp;quot;ID=%s_%i;parent=%s&amp;quot; %(name, block, name)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="235" LastEditorUserId="650" LastEditDate="2023-10-04T14:06:44.877" LastActivityDate="2023-10-04T14:06:44.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2245" PostTypeId="2" ParentId="2242" CreationDate="2017-08-08T12:00:46.733" Score="5" Body="&lt;p&gt;To convert BAM to GTF, which is the best way to get a file to compare with cuffcompare:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pysam&#xA;&#xA;bamfile=pysam.AlignmentFile(&amp;quot;my_bam_file.bam&amp;quot;)&#xA;&#xA;for alignment in bamfile.fetch():&#xA;&#xA;    if alignment.is_unmapped:&#xA;        continue&#xA;&#xA;    contig = bamfile.get_reference_name(alignment.reference_id)&#xA;    name = alignment.query_name&#xA;    &#xA;    if alignment.is_reverse:&#xA;        strand = &amp;quot;-&amp;quot;&#xA;    else: &#xA;        strand = &amp;quot;+&amp;quot;&#xA;&#xA;    for start, end in alignment.getblocks():&#xA;        # note: BED is 0-based, half-open, GFF is 1-based, closed&#xA;        print contig, &amp;quot;BAM&amp;quot;, &amp;quot;exon&amp;quot;, str(start + 1), str(end), &amp;quot;0&amp;quot;, strand, &amp;quot;.&amp;quot;, 'gene_id &amp;quot;%s&amp;quot;; transcript_id &amp;quot;%s&amp;quot;;' % (name, name)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;NB, this will also work with a SAM file as long as it is headered.&lt;/p&gt;&#xA;" OwnerUserId="235" LastEditorUserId="650" LastEditDate="2023-10-04T14:06:07.443" LastActivityDate="2023-10-04T14:06:07.443" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2250" PostTypeId="2" ParentId="2242" CreationDate="2017-08-08T20:27:37.457" Score="8" Body="&lt;p&gt;Galaxy has API and API-consuming libraries (such as BioBlend) that will allow you to interactively script against it without opening the graphical interface at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However you can also take almost any tool out of Galaxy and use it independently since everything is open source. The converter you mentioned is available as a Python script &lt;a href=&quot;https://github.com/galaxyproject/galaxy/blob/dev/tools/filters/bed_to_gff_converter.py&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; and the tool 'wrapper' which you can use to understand how to invoke the python script is &lt;a href=&quot;https://github.com/galaxyproject/galaxy/blob/dev/tools/filters/bed2gff.xml&quot; rel=&quot;noreferrer&quot;&gt;next to it&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="34" LastActivityDate="2017-08-08T20:27:37.457" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2251" PostTypeId="1" CreationDate="2017-08-09T15:32:05.220" Score="4" ViewCount="92" Body="&lt;p&gt;I am trying to install &lt;a href=&quot;https://github.com/gtamazian/chromosomer&quot; rel=&quot;noreferrer&quot;&gt;chromosomer&lt;/a&gt; but I fail. Can anybody help me, please?&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;$ pip install chromosomer&#xA;&#xA;Collecting chromosomer&#xA;  Could not find a version that satisfies the requirement chromosomer (from versions: )&#xA;No matching distribution found for chromosomer&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output &lt;code&gt;python&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;$ python&#xA;&#xA;Python 3.6.0 |Anaconda custom (x86_64)| (default, Dec 23 2016, 13:19:00) &#xA;[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin&#xA;Type &quot;help&quot;, &quot;copyright&quot;, &quot;credits&quot; or &quot;license&quot; for more information.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="298" LastEditDate="2017-08-09T16:39:29.187" LastActivityDate="2017-08-09T16:39:29.187" Title="Cannot install chromosomer" Tags="|python|software-installation|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2252" PostTypeId="2" ParentId="2242" CreationDate="2017-08-09T15:56:32.420" Score="4" Body="&lt;p&gt;Bioconductor makes this so easy. It does the coordinate conversion on import.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(rtracklayer)&#xA;&#xA;## import the bed file&#xA;bed.ranges &amp;lt;- import.bed('regions.bed')&#xA;&#xA;## export as a gff3 file&#xA;export.gff3(bed.ranges,'regions.gff3')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And people wonder why R is so popular for bioinformatics...&lt;/p&gt;&#xA;&lt;p&gt;Also if needed, you could go straight from &lt;strong&gt;BAM&lt;/strong&gt; file to &lt;strong&gt;gff3&lt;/strong&gt; in R as well.&lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="650" LastEditDate="2023-10-04T14:05:44.133" LastActivityDate="2023-10-04T14:05:44.133" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2253" PostTypeId="2" ParentId="2251" CreationDate="2017-08-09T16:21:04.900" Score="6" Body="&lt;p&gt;Chromosomer only exists for Python 2. You should thus be able to install it via&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pip2 install chromosomer&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;But:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your Python installation is a bit screwed up: there are Python 2 and Python 3, which are unfortunately incompatible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On your system &lt;code&gt;python&lt;/code&gt; and &lt;code&gt;pip&lt;/code&gt; seem to be aliases for Python 3, which I’d strongly recommend against (lots of tools will break). Instead, they should alias Python 2. Python 3 should be invoked via &lt;code&gt;python3&lt;/code&gt;/&lt;code&gt;pip3&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2017-08-09T16:21:04.900" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2254" PostTypeId="2" ParentId="892" CreationDate="2017-08-09T20:19:15.707" Score="3" Body="&lt;p&gt;Here is how I was able to export to a standard gzipped FASTQ file using Biopython. Basically, instead of using &lt;code&gt;SeqIO.write&lt;/code&gt;, I directly called the &lt;code&gt;.format&lt;/code&gt; method of the SeqRecord object. The example code below imports a gzipped FASTQ file, removes reads that do not contain a G in positions 7, 8, and 9, and writes the results to a gzipped FASTQ file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;import gzip&#xA;&#xA;path_in = &quot;path/to/in.fastq.gz&quot;&#xA;path_out = &quot;path/to/out.fastq.gz&quot;&#xA;handle_in = gzip.open(path_in, &quot;rt&quot;)&#xA;handle_out = gzip.open(path_out, &quot;wt&quot;)&#xA;&#xA;fq = SeqIO.parse(handle_in, &quot;fastq&quot;)&#xA;for read in fq:&#xA;    # Only export reads that have a G in positions 7, 8,&#xA;    # and 9&#xA;    if read.seq[6:9] == &quot;GGG&quot;:&#xA;        handle_out.write(read.format(&quot;fastq&quot;))&#xA;&#xA;handle_in.close()&#xA;handle_out.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using the code example from the original question that converts a FASTA file to a FASTQ file, it would look something like the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fastq = gzip.open(&quot;path/to/out.fastq.gz&quot;, &quot;wt&quot;)&#xA;for record in SeqIO.parse(fasta, &quot;fasta&quot;):&#xA;    fastq.write(record.format(&quot;fastq&quot;))&#xA;fastq.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Caveat:&lt;/strong&gt; This only works for Python 3. Python 3 differs from Python 2 in how it imports data from binary files (&lt;a href=&quot;http://python3porting.com/problems.html?highlight=decode#reading-from-files&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;). In fact, ideally you should be using Python 3.4 or greater since Biopython has compatibility issues with Python 3.3 (&lt;a href=&quot;https://github.com/biopython/biopython/pull/723&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="1302" LastActivityDate="2017-08-09T20:19:15.707" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2257" PostTypeId="1" AcceptedAnswerId="2258" CreationDate="2017-08-09T23:26:19.593" Score="5" ViewCount="85" Body="&lt;p&gt;I'm running blast+ searches for a list of proteins using specific parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my search&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; blastp -query protein.fasta -remote -db refseq_protein -out &#xA;protein_refseq_blast.txt -outfmt '6 qseqid sseqid qstart qend length evalue' -&#xA;entrez_query 'txid2 [ORGN] or txid4751 [ORGN]' -evalue 200000 -max_target_seqs 10000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For some proteins, this runs moderately quickly, ~5-10 mins. For other proteins, this command never stops running, even after an hour or so. I'm wondering if this variability in speed is due to any part of this choice in parameters, is simply variability in speed for the BLAST servers. Is it common for BLAST searches to take this long? Is there a way to run all of them in parallel using these parameters if this is the case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="727" LastActivityDate="2017-08-10T10:12:11.917" Title="Major variability in speed of BLAST between proteins" Tags="|blast|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2258" PostTypeId="2" ParentId="2257" CreationDate="2017-08-10T08:35:02.367" Score="5" Body="&lt;p&gt;Try changing you &lt;code&gt;evalue&lt;/code&gt; threshold to something like &lt;code&gt;0.001&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you blasted a sequence against a database containing 10,000 sequences and got a hit with an evalue of &lt;code&gt;1&lt;/code&gt;, this would mean that, given the size of your database and length of protein, you would expect to find a match as good as (or better than) the hit returned once purely by chance. If the &lt;code&gt;evalue&lt;/code&gt; threshold in your query was set to &lt;code&gt;0.01&lt;/code&gt; (for example) then this hit would not be returned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you got a hit with an &lt;code&gt;evalue&lt;/code&gt; of 200,000 this would mean you would expect to see 200,000 matches with a similar score simply by chance. If your &lt;code&gt;evalue&lt;/code&gt; threshold were set to 200000, then all of the hits with an evalue &amp;lt;=200000 would be considered significant. If you set it to &lt;code&gt;0.001&lt;/code&gt; then only those hits with &lt;code&gt;evalues&lt;/code&gt; &amp;lt;= &lt;code&gt;0.001&lt;/code&gt; would be considered significant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means that (depending on the size of the data base) almost every sequence in your query set will be considered as significant given your threshold, which will slow down your search hugely. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the the section on E values &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&amp;amp;PAGE_TYPE=BlastDocs&amp;amp;DOC_TYPE=FAQ#expect&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1148" LastEditorUserId="1148" LastEditDate="2017-08-10T10:12:11.917" LastActivityDate="2017-08-10T10:12:11.917" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2259" PostTypeId="1" AcceptedAnswerId="2261" CreationDate="2017-08-10T08:52:33.817" Score="4" ViewCount="220" Body="&lt;p&gt;I'm looking to generate data based on the DNA composition of a region of my genomes (data is incomplete genomes from HiSeq runs in fasta format). I'm looking for software which will give me sliding windows for GC content, GC skew, codon bias etc. I would like to use something which I can use with or plug into R. &lt;/p&gt;&#xA;" OwnerUserId="982" LastEditorUserId="77" LastEditDate="2017-08-10T12:36:33.297" LastActivityDate="2017-08-10T12:36:33.297" Title="Software recommendations - DNA composition" Tags="|r|software-recommendation|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2260" PostTypeId="2" ParentId="2257" CreationDate="2017-08-10T09:23:08.983" Score="3" Body="&lt;p&gt;The evalue associated with a blast HSP (High-scoring Segment Pair) is a measure of how often you would expect to find such an HSP in a database of this size by pure chance. For instance, if you blast the sequence &lt;code&gt;ACTG&lt;/code&gt; against the entirety of the NCBI's &lt;code&gt;nr&lt;/code&gt; database, you would expect several hundred thousand hits by pure chance. That is, several hundred thousand sequences that can be aligned to your query but are in no way related to it (not actual homologs). Those hits don't actually mean anything, so you use the evalue to limit the results to those that are unlikely to be found by chance and, therefore, more likely to have biological meaning if hits are found. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, setting the evalue to 200000 means you are interested even in HSPs whose characteristics (score, length, etc) mean you would expect to find &amp;lt;=200000 HSPs with such characteristics by pure chance. This evalue is pointless. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because your e-value is so high, BLAST will report absolutely anything and everything it finds, and the vast majority of your results will be meaningless and irrelevant. If you want a permissive e-value, you could use &lt;code&gt;1&lt;/code&gt; or even &lt;code&gt;10&lt;/code&gt; but 200000 is way too much. You usually only want negative e-values, cases where you wouldn't expect to find even one such HSP by chance. For highly dissimilar sequences, I would recommend you use 0.1 or something of that scale. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Changing the evalue will speed things up considerably. That said, you should always expect different times for different queries since the time it takes to find hits will always depend on the number of hits that can be found. If your sequence can be aligned to many of the sequences in the database, it will take longer to fetch all results. &lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-08-10T10:05:48.027" LastActivityDate="2017-08-10T10:05:48.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2261" PostTypeId="2" ParentId="2259" CreationDate="2017-08-10T09:58:50.333" Score="4" Body="&lt;p&gt;There are more R packages available that calculate GC content, for example Ape's &lt;code&gt;GC.content()&lt;/code&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; library(ape)&#xA;&amp;gt; data(woodmouse)&#xA;&amp;gt; GC.content(woodmouse)&#xA;[1] 0.3873347&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With a sliding window is in &lt;code&gt;Biostrings&lt;/code&gt; package.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; library(Biostrings) &#xA;&amp;gt; DNA &amp;lt;- DNAString(&quot;ACTGAAACCGTGGCAGTTTGAC&quot;)&#xA;&amp;gt; letterFrequencyInSlidingView(DNA, view.width=4,letters=&quot;CG&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For codon usage bias, I think &lt;code&gt;cubfits&lt;/code&gt; or &lt;code&gt;sscu&lt;/code&gt; packages might be useful. &lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2017-08-10T11:24:02.910" LastActivityDate="2017-08-10T11:24:02.910" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2262" PostTypeId="2" ParentId="2204" CreationDate="2017-08-10T12:19:49.280" Score="4" Body="&lt;p&gt;I believe all you need to do is to run &lt;code&gt;dnadiff&lt;/code&gt; from MUMmer. That will run a comparison and output a number of useful metrics.&lt;/p&gt;&#xA;" OwnerUserId="1305" LastEditorUserId="77" LastEditDate="2017-08-10T13:14:13.037" LastActivityDate="2017-08-10T13:14:13.037" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2263" PostTypeId="1" AcceptedAnswerId="2264" CreationDate="2017-08-10T12:24:36.120" Score="3" ViewCount="150" Body="&lt;p&gt;I noticed remote bigWig files (http URL) take sometimes about 1 minute to load on IGV. Once they loaded, jumping from one region to another can also take several seconds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding is that bigWig files have an internal index for different chunks of different sizes. Can the index be modified so that the speed of loading on IGV is improved? Does this need to happen at the point of creation of the bigWig file, or can the index of an existing bigWig be updated to speed up loading?&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2017-08-10T12:33:11.853" Title="IGV faster loading remote bigWig files" Tags="|bigwig|deeptools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2264" PostTypeId="2" ParentId="2263" CreationDate="2017-08-10T12:33:11.853" Score="1" Body="&lt;p&gt;You would need to do this when creating the bigWig file. Note that I'm not aware of anything that allows manually setting the various zoom levels (libBigWig/pyBigWig will allow you to set a max number, but that's it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This then raises the question of whether IGV is using an appropriate zoom level to begin with. I've never looked at how it chooses which of the zoom levels to use, but I know it uses them since if you don't have any it will fail to load bigWig files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Honestly, though, if it's taking a minute to jump to a new region then I suspect that the network is just slow or the remote server is overtaxed. Changing the internal indexing isn't going to fix that.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-08-10T12:33:11.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2265" PostTypeId="1" AcceptedAnswerId="2271" CreationDate="2017-08-10T12:46:40.243" Score="8" ViewCount="2973" Body="&lt;p&gt;I generated a file starting with the following bed lines:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ head -6 /tmp/bed_with_gene_ids.bed&#xA;I   3746    3909    &amp;quot;WBGene00023193&amp;quot;    .   -&#xA;I   3746    3909    &amp;quot;WBGene00023193&amp;quot;    .   -&#xA;I   4118    4220    &amp;quot;WBGene00022277&amp;quot;    .   -&#xA;I   4118    4358    &amp;quot;WBGene00022277&amp;quot;    .   -&#xA;I   4118    10230   &amp;quot;WBGene00022277&amp;quot;    .   -&#xA;I   4220    4223    &amp;quot;WBGene00022277&amp;quot;    .   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to merge them based on the name field (the 4-th column), taking the min for the start and the max for the end. Other fields are expected to be the same for all records having the same name.&lt;/p&gt;&#xA;&lt;p&gt;Expected result:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;I   3746    3909    &amp;quot;WBGene00023193&amp;quot;    .   -&#xA;I   4118    10230   &amp;quot;WBGene00022277&amp;quot;    .   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I found a potential solution based on &lt;code&gt;bedtools groupby&lt;/code&gt; here: &lt;a href=&quot;https://www.biostars.org/p/145751/#145775&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/145751/#145775&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Sample data:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat genes.bed&#xA;chr14    49894259    49895806    ENSMUST00000053290    0.000000    ...&#xA;chr14    49894873    49894876    ENSMUST00000053290    0.000000    ...&#xA;chr14    49894876    49895800    ENSMUST00000053291    0.000000    ...&#xA;chr14    49895797    49895800    ENSMUST00000053291    0.000000    ...&#xA;chr14    49901908    49901941    ENSMUST00000053291    0.000000    ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Example output:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;sort -k4,4 genes.bed \&#xA;| groupBy -g 1,4 -c 4,2,3 -o count,min,max \&#xA;| awk -v OFS='\t' '{print &lt;span class=&quot;math-container&quot;&gt;$1, $&lt;/span&gt;4, &lt;span class=&quot;math-container&quot;&gt;$5, $&lt;/span&gt;2, $3}'&#xA;&#xA;chr14    49894259    49895806    ENSMUST00000053290    2&#xA;chr14    49894876    49901941    ENSMUST00000053291    3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;However:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I don't understand the groupBy behaviour (Why &lt;code&gt;-g 1,4&lt;/code&gt; and not just &lt;code&gt;-g 4&lt;/code&gt;?, Why &lt;code&gt;-c 4,2,3&lt;/code&gt; in this order and then rearrange things using &lt;code&gt;awk&lt;/code&gt;?)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This code doesn't work for me.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Here is what happens when I try the solution given above:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ head -3 /tmp/bed_with_gene_ids.bed | bedtools groupby -g 1,4 -c 4,2,3 -o count,min,max | awk -v OFS='\t' '{print $1, $4, $5, $2, $&lt;/span&gt;3}'&#xA;3           3746    4220&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here are attempt based on what I thought could work according to &lt;a href=&quot;http://bedtools.readthedocs.io/en/latest/content/tools/groupby.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ head -6 /tmp/bed_with_gene_ids.bed | bedtools groupby -g 4 -c 1,2,3,4,5,6 -o first,min,max,distinct,first,first&#xA;I   3746    10230   &amp;quot;WBGene00022277&amp;quot;,&amp;quot;WBGene00023193&amp;quot;   .   -&#xA;&#xA;$ head -6 /tmp/bed_with_gene_ids.bed | bedtools groupby -g 4 -c 1,2,3,4,5,6 -o first,min,max,last,first,first&#xA;I   3746    10230   &amp;quot;WBGene00022277&amp;quot;    .   -&#xA;&#xA;$ head -6 /tmp/bed_with_gene_ids.bed | bedtools groupby -g 4 -c 1,2,3,5,6 -o first,min,max,first,first&#xA;I   3746    10230   .   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I don't get why when I group based on the 4-th column, for which I have two distinct values, I cannot obtain two lines in the resulting output.&lt;/p&gt;&#xA;&lt;p&gt;I understand based on the comments on the documentation page that the documentation is not up-to-date. In particular, there is a &lt;code&gt;-full&lt;/code&gt; option that is needed if one wants all fields to be outputted. Re-reading the solution mentioned above, I think I now understand the reason for the multiple columns for the &lt;code&gt;-g option&lt;/code&gt; and for the &lt;code&gt;awk&lt;/code&gt; rearrangement. Hence the following attempt.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ head -6 /tmp/bed_with_gene_ids.bed | bedtools groupby -g 1,4,5,6 -c 2,3 -o min,max -full&#xA;    I   3746    3909    &amp;quot;WBGene00023193&amp;quot;    .   -   3746    10230&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But this still doesn't give me two lines.&lt;/p&gt;&#xA;&lt;p&gt;Are there other tools that could do what I want efficiently?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Edit: Solution&lt;/h3&gt;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2271/292&quot;&gt;this answer&lt;/a&gt;, the problem with bedtools is that there is a bug in the latest release (2.26.0 as of august 2017). In order to have a functional &lt;code&gt;bedtools groupby&lt;/code&gt;, one needs to get the development version from github.&lt;/p&gt;&#xA;&lt;p&gt;With the &lt;a href=&quot;https://github.com/arq5x/bedtools2/commit/52db65490b196f95bcd141a20c79ed36d9989496&quot; rel=&quot;nofollow noreferrer&quot;&gt;github version of bedtools&lt;/a&gt;, I can now get the expected result as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ head -6 /tmp/bed_with_gene_ids.bed | bedtools groupby -g 1,4,5,6 -c 2,3 -o min,max | awk -v OFS=&quot;\t&quot; '{print $1,$5,$6,$2,$3,$&lt;/span&gt;4}'&#xA;I   3746    3909    &amp;quot;WBGene00023193&amp;quot;    .   -&#xA;I   4118    10230   &amp;quot;WBGene00022277&amp;quot;    .   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I include fields 1, 5 and 6 in &lt;code&gt;-g&lt;/code&gt; (besides field 4) in order to have them printed out. In my bed file, they should be the same for a given value of field 4. The &lt;code&gt;awk&lt;/code&gt; part is needed because one has apparently not total control on the output order: the &lt;code&gt;-g&lt;/code&gt; fields come before the &lt;code&gt;-c&lt;/code&gt; fields.&lt;/p&gt;&#xA;&lt;p&gt;Note (added 23/12/2020): As specified in &lt;code&gt;bedtools groupby&lt;/code&gt; documentation, the input bed file should be sorted according to (chromosome (1) and) name (4) field(s) for this to work properly. Otherwise, duplicate (i.e. non-merged) names may appear in the output.&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2020-12-23T15:08:16.737" LastActivityDate="2020-12-23T15:08:16.737" Title="Merging bed records based on name" Tags="|bed|bedtools|" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2266" PostTypeId="2" ParentId="2265" CreationDate="2017-08-10T13:45:37.637" Score="4" Body="&lt;p&gt;You could do this with the &lt;a href=&quot;https://github.com/CGATOxford/cgat&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;CGAT&lt;/code&gt; toolkit&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;cgat bed2bed --method=merge --merge-by-name -I bed_with_gene_ids.bed&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Installing such a massive package might be overkill for this task though. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2017-08-10T13:45:37.637" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2267" PostTypeId="2" ParentId="2265" CreationDate="2017-08-10T13:59:48.527" Score="0" Body="&lt;p&gt;If you're 100% sure that everything but the start and end positions will be the same for all lines sharing a name, you could just do it yourself. For example, in Perl:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ perl -lane '$start{$F[3]}||=$F[1]; &#xA;              if( $F[1] &amp;lt; $start{$F[3]} ){&#xA;                 $start{$F[3]} = $F[1]&#xA;              } &#xA;              if( $F[2] &amp;gt; $end{$F[3]} ){&#xA;                 $end{$F[3]} = $F[2]&#xA;              } &#xA;              $chr{$F[3]} = $F[0]; &#xA;              $rest{$F[3]} = join &quot;\t&quot;, @F[4,$#F]; &#xA;              END{&#xA;                 foreach $n (keys %chr){&#xA;                  print &quot;$chr{$n}\t$start{$n}\t$end{$n}\t$n\t$rest{$n}&quot;&#xA;                 }&#xA;              }' file.bed &#xA;I   3746    3909    &quot;WBGene00023193&quot;    .   -&#xA;I   4118    10230   &quot;WBGene00022277&quot;    .   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-08-10T13:59:48.527" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2268" PostTypeId="2" ParentId="2241" CreationDate="2017-08-10T15:10:04.033" Score="1" Body="&lt;p&gt;I don't think there is a database with expression data for your particular cell type. I am afraid your &quot;headache&quot; approach doesn't sound so bad...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So get expression data e.g., from GEO for your cell type, and define which genes/proteins you consider as &quot;expressed&quot; (e.g., minimal RPKM value).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, you can use GO annotation to find your genes involved with membrane or secretion (or what ever). You can use biomaRt in R for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, you want to know all &quot;membrane&quot; genes.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; library(biomaRt)&#xA;&amp;gt; &#xA;&amp;gt; ensembl = useMart(&quot;ensembl&quot;,dataset=&quot;hsapiens_gene_ensembl&quot;)&#xA;&amp;gt; &#xA;&amp;gt; membrane.genes &amp;lt;- getBM(attributes=c('hgnc_symbol',&#xA;&amp;gt; 'ensembl_transcript_id', 'go_id'),&#xA;&amp;gt; filters = 'go', values = 'GO:0016020', mart = ensembl)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And now you want to know it only from genes that are expressed in your ganglion cells (with dummy example of two genes).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; gene.list &amp;lt;- c(&quot;KIR2DL3&quot;, &quot;GPR1&quot;)&#xA;&amp;gt; &#xA;&amp;gt; membrane.genes.list &amp;lt;- membrane.genes[membrane.genes$hgnc_symbol ==&#xA;&amp;gt; gene.list,]&#xA;&amp;gt; &#xA;&amp;gt; membrane.only &amp;lt;- membrane.genes.list[membrane.genes.list$go_id ==&#xA;&amp;gt; 'GO:0016020',]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastActivityDate="2017-08-10T15:10:04.033" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2269" PostTypeId="1" CreationDate="2017-08-10T16:25:38.047" Score="8" ViewCount="80" Body="&lt;p&gt;I have some data I am working with, and I am curious if I am able to combine p-values from a paired t-test for CpG sites in the genome using Fisher's Method to get one p-value for each unique gene. Linked &lt;a href=&quot;https://en.wikipedia.org/wiki/Fisher%27s_method&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; is the Wikipedia page for Fisher's Method. I understand that an assumption of the method used is that each individual p-value being combined must be independent. I'm relatively new to biostatistics, so I'm curious if using CpGs from the same gene would violate this assumption.&lt;/p&gt;&#xA;" OwnerUserId="1309" LastActivityDate="2017-09-12T09:13:45.453" Title="Melt p-values for CpG sites mapping to the same gene" Tags="|r|bioconductor|gene|methylation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2270" PostTypeId="2" ParentId="2216" CreationDate="2017-08-10T16:58:43.040" Score="0" Body="&lt;p&gt;The point I always miss in the discussion about coverage is, that no one tells how it was calculated. Were duplicates removed? How do you count overlapping paired-end reads? As 2 or 1? Just to point out two things that have influence on the coverage.&lt;/p&gt;&#xA;" OwnerUserId="1310" LastEditorUserId="77" LastEditDate="2017-08-10T18:12:40.087" LastActivityDate="2017-08-10T18:12:40.087" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2271" PostTypeId="2" ParentId="2265" CreationDate="2017-08-10T17:40:41.057" Score="6" Body="&lt;p&gt;Although you don't mention it, I'm guessing you're using bedtools v2.26.0. Version 2.26.0 of groupBy has a bug in it, which you've encountered (it was fixed shortly after release, so you'll either have to use a version before the bug was introduced, or compile the current source code yourself from &lt;a href=&quot;https://github.com/arq5x/bedtools2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/arq5x/bedtools2&lt;/a&gt;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;v2.26.0: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;local10:~/Documents/tmp$ cat asdf.bed &#xA;I	3746	3909	WBGene00023193	.	-&#xA;I	3746	3909	WBGene00023193	.	-&#xA;I	4118	4220	WBGene00022277	.	-&#xA;I	4118	4358	WBGene00022277	.	-&#xA;I	4118	10230	WBGene00022277	.	-&#xA;I	4220	4223	WBGene00022277	.	-&#xA;local10:~/Documents/tmp$ groupBy -i asdf.bed -g 4 -c 2,3 -o min,max &#xA;3746    10230 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;v2.26.0-125-g52db654 (I.E. compiling the source code from github): &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;local10:~/Documents/tmp$ bedtools2/bin/groupBy -i asdf.bed -g 4 -c 2,3 -o min,max&#xA;WBGene00023193  3746    3909&#xA;WBGene00022277  4118    10230&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To answer your questions: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) You might notice that my output above gives the grouped columns first; you'll have to reorder the output via awk in order to get it back in order. As for why they chose to group on both columns 1 and 4: if you have the same name on multiple chromosomes, you may want to treat them as separate features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Version differences, as stated in the first part of my answer. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To actually merge the file: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Make sure to run this with a version other than v2.26.0 (As Devon Ryan writes in the comments, you may want to add column 6 to &lt;code&gt;-g&lt;/code&gt; to make it strand-specific):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./bedtools2/bin/groupBy -i asdf.bed -g 1,4 -c 2,3,5,6 -o min,max,first,first \&#xA;     | awk -v OFS='\t' '{print $1, $3, $4, $2, $5, $6}'&#xA;I   3746    3909    WBGene00023193  .   -&#xA;I   4118    10230   WBGene00022277  .   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1076" LastEditorUserId="1076" LastEditDate="2017-08-11T17:36:04.583" LastActivityDate="2017-08-11T17:36:04.583" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2272" PostTypeId="2" ParentId="2269" CreationDate="2017-08-10T18:11:16.970" Score="6" Body="&lt;p&gt;Methylation levels have high local correlation, so Fisher's method would be problematic. Having said that, you have no reason to use Fisher's method after a paired t-test. A paired t-test will give you a single p-value per gene, which is what you want. Do be sure to only include CpG with some minimal coverage in both group.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-08-10T18:11:16.970" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2273" PostTypeId="2" ParentId="2265" CreationDate="2017-08-10T18:48:21.233" Score="2" Body="&lt;pre&gt;&lt;code&gt;$ cut -f4-6 in.bed | sed 's/\t/_/g' | sort | uniq | awk -F'_' '{ system(&quot;grep &quot;$1&quot; in.bed | bedops --merge - &quot;); print $0; }' | paste -d &quot;\t&quot; - - | sed 's/_/\t/g' | sort-bed - &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Given your sample input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ more in.bed&#xA;I   3746    3909    &quot;WBGene00023193&quot;    .   -&#xA;I   3746    3909    &quot;WBGene00023193&quot;    .   -&#xA;I   4118    4220    &quot;WBGene00022277&quot;    .   -&#xA;I   4118    4358    &quot;WBGene00022277&quot;    .   -&#xA;I   4118    10230   &quot;WBGene00022277&quot;    .   -&#xA;I   4220    4223    &quot;WBGene00022277&quot;    .   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;answer.bed&lt;/code&gt; file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ more answer.bed&#xA;I   3746    3909    &quot;WBGene00023193&quot;    .   -&#xA;I   4118    10230   &quot;WBGene00022277&quot;    .   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Sorting with &lt;code&gt;sort-bed&lt;/code&gt; is useful at the end, so that you can pipe it or work with it with other BEDOPS tools, or other tools that now accept sorted BED input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Streaming is a pretty efficient way to do things, generally.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How this works&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the pipeline again:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cut -f4-6 in.bed | sed 's/\t/_/g' | sort | uniq | awk -F'_' '{ system(&quot;grep &quot;$1&quot; in.bed | bedops --merge - &quot;); print $0; }' | paste -d &quot;\t&quot; - - | sed 's/_/\t/g' | sort-bed - &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We start by cutting columns 4 through 6 (id, score and strand), replacing tabs with underscores, sorting and removing duplicates:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cut -f4-6 in.bed | sed 's/\t/_/g' | sort | uniq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What we get out of this is a sorted list of &quot;needles&quot; — one for each ID-score-strand combination: an &lt;em&gt;ID-needle&lt;/em&gt; — that we can use to &lt;code&gt;grep&lt;/code&gt; or filter the original BED file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This list is piped to &lt;code&gt;awk&lt;/code&gt; which, for each ID-needle, runs &lt;code&gt;grep&lt;/code&gt; against the original BED file and pipes the subset to &lt;code&gt;bedops --merge -&lt;/code&gt;, which merges overlapping intervals. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that merging only works for overlapping intervals. &lt;em&gt;Merging is not necessarily the same as returning a min-max pair, and this pipeline will break if there are intervals that do not overlap.&lt;/em&gt; But you could modify the &lt;code&gt;awk&lt;/code&gt; statement to process the input intervals and return the minimum and maximum interval coordinates, if that is really what you want, by tracking the min and max values over all intervals that come into &lt;code&gt;awk&lt;/code&gt;, and printing a final interval with an &lt;code&gt;END&lt;/code&gt; block.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;system&lt;/code&gt; command prints the merged interval on one line. The following &lt;code&gt;print $0&lt;/code&gt; statement prints the needle on the next line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -F'_' '{ system(&quot;grep &quot;$1&quot; in.bed | bedops --merge - &quot;); print $0; }'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We take each pair of alternating lines and re-linearize them with &lt;code&gt;paste&lt;/code&gt;. This result now contains four columns: the three columns of each merged interval, and the ID-needle. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We then use &lt;code&gt;sed&lt;/code&gt; to replace underscores with tabs, so that we turn the ID-needle back into three, tab-separated ID-score-strand columns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;paste -d &quot;\t&quot; - - | sed 's/_/\t/g'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output is now a six-column BED file, but it is ordered by the sort order we applied on ID-needles further up the pipeline, which we don't want. What we really want is BED that is sorted per BEDOPS &lt;code&gt;sort-bed&lt;/code&gt;, so that we can do more set operations and get a correct result. So we pipe this to &lt;code&gt;sort-bed -&lt;/code&gt; to write a sorted file to &lt;code&gt;answer.bed&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sort-bed - &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2017-08-11T16:51:49.820" LastActivityDate="2017-08-11T16:51:49.820" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2274" PostTypeId="1" AcceptedAnswerId="2281" CreationDate="2017-08-10T22:22:01.717" Score="4" ViewCount="363" Body="&lt;p&gt;Let's say I have the genome &lt;code&gt;hg19&lt;/code&gt; loaded into R via BSGenome&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(&quot;BSgenome&quot;)    &#xA;hg19genome = getBSgenome('BSgenome.Hsapiens.UCSC.hg19', masked=FALSE)   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I then have a list of SNPs loaded as a GRanges object, &lt;code&gt;gr&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(GenomicRanges)&#xA;&#xA; &amp;gt; gr&#xA; GRanges object with 212 ranges and 3 metadata columns:                                                                                                                                                                                                                                                                                                                                                &#xA;        seqnames               ranges strand |     width     REF    ALT                                                                                                                                                                                                                                                                                                                             &#xA;           &amp;lt;Rle&amp;gt;          v &amp;lt;IRanges&amp;gt;  &amp;lt;Rle&amp;gt; | &amp;lt;numeric&amp;gt;                                                                                                                                                                                                                                                                                                                        &#xA;    [1]        1 [86099032, 86099032]      * |         1     C      T                                                                                                                                                                                                                                                                                                                  &#xA;    [2]        1 [86099033, 86099033]      * |         1     C      A                                                                                                                                                                                                                                                                                                                   &#xA;    [3]        1 [86099199, 86099199]      * |         1     G      A  &#xA;  ....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a straightforward way to inject these SNPs into &lt;code&gt;hg19genome&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="146" LastEditorUserId="146" LastEditDate="2017-08-10T22:37:31.110" LastActivityDate="2017-08-11T13:12:29.063" Title="Given a Genomic Ranges of SNPs, how to inject these SNPs in genome via BSGenome?" Tags="|r|bioconductor|reference-genome|snp|biostrings|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2275" PostTypeId="1" AcceptedAnswerId="2277" CreationDate="2017-08-10T22:44:40.640" Score="10" ViewCount="1419" Body="&lt;p&gt;The problem: I have a VCF file, a reference genome, and a bunch of annotations for the reference (genes, repeat regions, etc.) as GFF or BED files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would like is a tool that takes all of this as input and outputs a tab- or comma-delimited table containing as much information as possible. Potential columns in the output include:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Type of variant (SNV, structural, etc)&lt;/li&gt;&#xA;&lt;li&gt;Details of variant (e.g. reference base, variant base, coverage, position, etc)&lt;/li&gt;&#xA;&lt;li&gt;Annotations overlapping variant&lt;/li&gt;&#xA;&lt;li&gt;Annotations near variant (e.g. is it just upstream of a gene)&lt;/li&gt;&#xA;&lt;li&gt;If it appears in a coding region, does it change the amino acid.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There are many tools that do &lt;em&gt;something like this&lt;/em&gt;. But to the newbie (like me) it is not clear which tools are worth starting with. Since the majority of tools take a little effort to get working in the first place, my question is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What tool comes closest to doing what I have described, is known to reliable, and is likely to be maintained for the next few years.&lt;/p&gt;&#xA;" OwnerUserId="156" LastActivityDate="2017-08-14T04:20:37.817" Title="Tools to create annotated table of variants from VCF" Tags="|sequence-annotation|vcf|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2277" PostTypeId="2" ParentId="2275" CreationDate="2017-08-11T06:50:40.807" Score="5" Body="&lt;p&gt;I have used FEATnotator and I think it can provide all of the columns you would like to see. There are many output files generated, but the consolidated output has the following columns:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Chromosome     &lt;/li&gt;&#xA;&lt;li&gt;Position       &lt;/li&gt;&#xA;&lt;li&gt;Column_3     &lt;/li&gt;&#xA;&lt;li&gt;Consensus_Allele &lt;/li&gt;&#xA;&lt;li&gt;Annotation_Signature   &lt;/li&gt;&#xA;&lt;li&gt;Reference_Base &lt;/li&gt;&#xA;&lt;li&gt;Alternate_Base &lt;/li&gt;&#xA;&lt;li&gt;Transition/Transversion SNP_Type       &lt;/li&gt;&#xA;&lt;li&gt;Premature_STOP_Gained   STOP_Lost      &lt;/li&gt;&#xA;&lt;li&gt;START_CODON    &lt;/li&gt;&#xA;&lt;li&gt;STOP_CODON     &lt;/li&gt;&#xA;&lt;li&gt;SPLICE_SITE &lt;/li&gt;&#xA;&lt;li&gt;InterGenic     &lt;/li&gt;&#xA;&lt;li&gt;Gene_Body      &lt;/li&gt;&#xA;&lt;li&gt;Intron &lt;/li&gt;&#xA;&lt;li&gt;Exon &lt;/li&gt;&#xA;&lt;li&gt;Coding &lt;/li&gt;&#xA;&lt;li&gt;UTR    &lt;/li&gt;&#xA;&lt;li&gt;Transcription_Start_Site      &lt;/li&gt;&#xA;&lt;li&gt;Nearest_gene   &lt;/li&gt;&#xA;&lt;li&gt;Distance&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;here is some example output records from using a VCF, reference genome and GFF annotation file (sorry about the crappy formatting...there are a lot of fields!):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Chr01   28177   T       G       NA      T     G TRANSVERSION    NA      NO      NO      NO      NO      NO      YES     NO      NO      NO      NO      NO      NO      Eucgr.A00211.v2.0       16054&#xA;Chr01   44876   C       G       gene:Eucgr.A00211.v2.0   mRNA:Eucgr.A00211.1.v2.0[intron]       C       G       TRANSVERSION    NA      NO      NO      NO      NO      NO      NO      YES     YES     NO      NO      NO      NO      NA  NA&#xA;Chr01   46819   A       G       gene:Eucgr.A00211.v2.0   mRNA:Eucgr.A00211.1.v2.0   CDS:Eucgr.A00211.1.v2.0.CDS.4[AAA - K =&amp;gt; GAA - E (MISSENSE)]   exon:Eucgr.A00211.1.v2.0.exon.4      A       G       TRANSITION      MISSENSE        NO  NO       NO      NO      NO      NO      YES   NO        YES     YES     NO      NO      NA      NA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1281" LastEditorUserId="298" LastEditDate="2017-08-11T13:28:51.170" LastActivityDate="2017-08-11T13:28:51.170" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2278" PostTypeId="2" ParentId="2275" CreationDate="2017-08-11T07:39:25.050" Score="5" Body="&lt;p&gt;snpEff is a great tool for annotating VCF files and you can add custom reference sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://snpeff.sourceforge.net/&quot; rel=&quot;noreferrer&quot;&gt;http://snpeff.sourceforge.net/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Guide to add custom annotation files in snpEff&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/50/adding-genomic-annotations-using-snpeff-and-variantannotator&quot; rel=&quot;noreferrer&quot;&gt;https://gatkforums.broadinstitute.org/gatk/discussion/50/adding-genomic-annotations-using-snpeff-and-variantannotator&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a bunch of pre curated annotation datasets available in their database.&lt;/p&gt;&#xA;" OwnerUserId="206" LastActivityDate="2017-08-11T07:39:25.050" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2279" PostTypeId="1" AcceptedAnswerId="2280" CreationDate="2017-08-11T07:44:08.053" Score="5" ViewCount="759" Body="&lt;p&gt;I've annotated VCF files using snpEff and looking for a tool or script to parse the VCF file and clean up the file to make it interpretable for a biologist. &lt;/p&gt;&#xA;" OwnerUserId="206" LastActivityDate="2017-08-11T08:04:59.083" Title="Tool or script to parse annotated VCF files" Tags="|variant-calling|snp|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2280" PostTypeId="2" ParentId="2279" CreationDate="2017-08-11T08:04:59.083" Score="5" Body="&lt;p&gt;(edit)you can filter the VCF annotations with &lt;strong&gt;snpsift&lt;/strong&gt;, I've also written a &lt;strong&gt;VcfFilterSequenceOntology&lt;/strong&gt; &lt;a href=&quot;http://lindenb.github.io/jvarkit/VcfFilterSequenceOntology.html&quot; rel=&quot;noreferrer&quot;&gt;http://lindenb.github.io/jvarkit/VcfFilterSequenceOntology.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've written vcf2table: &lt;a href=&quot;http://lindenb.github.io/jvarkit/VcfToTable.html&quot; rel=&quot;noreferrer&quot;&gt;http://lindenb.github.io/jvarkit/VcfToTable.html&lt;/a&gt;&#xA;It decodes VEP and SNPeff annotations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;chr1/10001/T (n 1)&#xA; Variant&#xA; +--------+--------------------+&#xA; | Key    | Value              |&#xA; +--------+--------------------+&#xA; | CHROM  | chr1               |&#xA; (....)&#xA; VEP&#xA; +--------------------------+------+----------------+------------+-----------------+--------+------------------+-----------------------------------------------+-------------+---------+-----------------+----------------------+&#xA; | PolyPhen                 | EXON | SIFT           | ALLELE_NUM | Gene            | SYMBOL | Protein_position | Consequence                                   | Amino_acids | Codons  | Feature         | BIOTYPE              |&#xA; +--------------------------+------+----------------+------------+-----------------+--------+------------------+-----------------------------------------------+-------------+---------+-----------------+----------------------+&#xA; | probably_damaging(0.956) | 8/9  | deleterious(0) | 1          | ENSG00000102967 | DHODH  | 346/395          | missense_variant                              | R/W         | Cgg/Tgg | ENST00000219240 | protein_coding       |&#xA; |                          | 3/4  |                | 1          | ENSG00000102967 | DHODH  |                  | non_coding_exon_variant&amp;amp;nc_transcript_variant |             |         | ENST00000571392 | retained_intron      |&#xA; |                          |      |                | 1          | ENSG00000102967 | DHODH  |                  | downstream_gene_variant                       |             |         | ENST00000572003 | retained_intron      |&#xA; |                          |      |                | 1          | ENSG00000102967 | DHODH  |                  | downstream_gene_variant                       |             |         | ENST00000573843 | retained_intron      |&#xA; |                          |      |                | 1          | ENSG00000102967 | DHODH  |                  | downstream_gene_variant                       |             |         | ENST00000573922 | processed_transcript |&#xA; |                          |      |                | 1          | ENSG00000102967 | DHODH  | -/193            | intron_variant                                |             |         | ENST00000574309 | protein_coding       |&#xA; | probably_damaging(0.946) | 8/9  | deleterious(0) | 1          | ENSG00000102967 | DHODH  | 344/393          | missense_variant                              | R/W         | Cgg/Tgg | ENST00000572887 | protein_coding       |&#xA; +--------------------------+------+----------------+------------+-----------------+--------+------------------+-----------------------------------------------+-------------+---------+-----------------+----------------------+&#xA; Genotypes&#xA; +---------+------+-------+----+----+-----+---------+&#xA; | Sample  | Type | AD    | DP | GQ | GT  | PL      |&#xA; +---------+------+-------+----+----+-----+---------+&#xA; | M10475  | HET  | 10,2  | 15 | 10 | 0/1 | 25,0,10 |&#xA; | M10478  | HET  | 10,4  | 16 | 5  | 0/1 | 40,0,5  |&#xA; | M10500  | HET  | 10,10 | 21 | 7  | 0/1 | 111,0,7 |&#xA; | M128215 | HET  | 15,5  | 24 | 0  | 0/1 | 49,0,0  |&#xA; +---------+------+-------+----+----+-----+---------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastActivityDate="2017-08-11T08:04:59.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2281" PostTypeId="2" ParentId="2274" CreationDate="2017-08-11T13:12:29.063" Score="3" Body="&lt;p&gt;This seems relatively complicated given the structure of a BSGenome object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The creator of the package answered this question previously on the Bioconductor support forums:&#xA;&lt;a href=&quot;https://support.bioconductor.org/p/86665/#86757&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.bioconductor.org/p/86665/#86757&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We don't provide an easy way to inject arbitrary SNPs in an arbitrary&#xA;  BSgenome at the moment. However, it should not be too hard to forge&#xA;  the BSgenome ... package. First you would need to&#xA;  compute the sequences of the mutated chromosomes (you can&#xA;  use replaceLetterAt for this), then write them to a 2bit file (put&#xA;  them in a DNAStringSet object and call rtracklayer::export on it),&#xA;  then use that 2bit file to forge the BSgenome ....&#xA;  package (see the BSgenomeForge vignette in the BSgenome package for&#xA;  how to do this).&#xA;  - Herve Pages&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here is another similar question with more info about replaceLetterAt: &lt;a href=&quot;https://support.bioconductor.org/p/26199/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.bioconductor.org/p/26199/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2017-08-11T13:12:29.063" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2282" PostTypeId="1" AcceptedAnswerId="2295" CreationDate="2017-08-11T18:47:12.110" Score="6" ViewCount="4711" Body="&lt;p&gt;After reading some of the forum posts in Biostar and SeqAnswers I find it very confusing whether to filter out the duplicate reads from aligned files or not. As far I understand it's very difficult to distinguish between highly expressed genes and duplicate reads and we may lose important information during the filtration process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, is it really necessary to remove the duplicates in differential expression analysis using RNA-seq data?&lt;/p&gt;&#xA;" OwnerUserId="206" LastEditorUserId="96" LastEditDate="2017-08-18T04:38:32.127" LastActivityDate="2019-05-29T14:28:18.280" Title="Removing PCR duplicates in RNA-seq Analysis" Tags="|rna-seq|differential-expression|quality-control|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2283" PostTypeId="2" ParentId="2282" CreationDate="2017-08-11T21:08:06.510" Score="3" Body="&lt;p&gt;Generally you should just leave them as is. One does remove/mark duplicates in DNA seq. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For further read check this &lt;a href=&quot;https://www.nature.com/articles/srep25533&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nature paper &lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="315" LastActivityDate="2017-08-11T21:08:06.510" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2284" PostTypeId="2" ParentId="998" CreationDate="2017-08-12T21:00:41.833" Score="6" Body="&lt;p&gt;I am the developer of Uberon and I would be happy to help you with what you need, either from Uberon, or from the FMA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mentioned you need something simpler than FMA. There are a variety of tools for creating custom subsets. Additionally, some ontologies provide ready-made subsets for particular purposes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, we make a subset called 'basic.obo' from Uberon:&#xA;&lt;a href=&quot;http://uberon.github.io/downloads.html#subsets&quot; rel=&quot;noreferrer&quot;&gt;http://uberon.github.io/downloads.html#subsets&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can examine this directly in either oboedit or Protege5 (note Protege5 parses obo so you don't need to convert).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From your question, you didn't indicate you needed programmatic access, but if you do there are a variety of options, as well as more programmer-friendly JSON exports.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most ontologies have a number of different options for web-browsing, you can see some of the options for Uberon here:&#xA;&lt;a href=&quot;http://obofoundry.org/ontology/uberon.html&quot; rel=&quot;noreferrer&quot;&gt;http://obofoundry.org/ontology/uberon.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You mentioned annotating disease sites. There may be a tools (either desktop or web-based) for helping with this - e.g. providing autocomplete over a desired subset of an ontology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also want to ask your question on a mailing list like obo-discuss. Unfortunately many ontology developers don't read stackexchange, but most are always keen to hear of community requirements, to help them provide better and more useful products.&lt;/p&gt;&#xA;" OwnerUserId="1322" LastActivityDate="2017-08-12T21:00:41.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2285" PostTypeId="5" CreationDate="2017-08-12T21:23:24.850" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-08-12T21:23:24.850" LastActivityDate="2017-08-12T21:23:24.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2286" PostTypeId="4" CreationDate="2017-08-12T21:23:24.850" Score="0" Body="Ontologies are collections of terms organized in a graph that can be used to annotate biological data, for example, tagging genes or variants with terms from disease or phenotype ontologies. The most well known bio ontology is the Gene Ontology (GO), used for functional annotation of genes. Sites such as the OBO Foundry and Bioportal can be used to find bio-ontologies." OwnerUserId="1322" LastEditorUserId="1322" LastEditDate="2017-08-13T07:14:05.840" LastActivityDate="2017-08-13T07:14:05.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2287" PostTypeId="5" CreationDate="2017-08-12T21:25:50.737" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-08-12T21:25:50.737" LastActivityDate="2017-08-12T21:25:50.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2288" PostTypeId="4" CreationDate="2017-08-12T21:25:50.737" Score="0" Body="The Gene Ontology (GO) is a collection of annotations of genes from multiple species to terms describing the functional role of those genes. The terms are organized in a graph structure that allows for propagation of information. See http://geneontology.org" OwnerUserId="1322" LastEditorUserId="1322" LastEditDate="2017-08-13T07:13:57.537" LastActivityDate="2017-08-13T07:13:57.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2289" PostTypeId="1" CreationDate="2017-08-12T22:46:42.073" Score="7" ViewCount="106" Body="&lt;p&gt;I'd like to know how common certain mutation &lt;em&gt;types&lt;/em&gt; are in public data sets like the 1000 Genomes, ExAC, and ESP6500. Specifically, I'd like to know the distribution of stop-gains, stop-losses, frameshift, and other mutation types. For example, what is the median number of stop-gains observed across individuals in the ExAC population? I'd like to have the full distribution, but median values would work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be clear, I am &lt;em&gt;not&lt;/em&gt; looking for allele frequencies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I didn't find this kind of summary data when I looked at the respective websites. Is there a good source for these summary data?&lt;/p&gt;&#xA;" OwnerUserId="136" LastActivityDate="2020-03-07T01:51:45.223" Title="Where can I find summary data for how common certain mutation *types* are?" Tags="|public-databases|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2290" PostTypeId="2" ParentId="2234" CreationDate="2017-08-13T21:44:02.207" Score="0" Body="&lt;p&gt;This weekend, I  installed the R language packages rapidGSEA and the Broad Institute GSEA , the Nvidia CUDA Toolkit 6.5-- Custom installation option  and MINGW makefiles on my son's Windows 8.1 notebook computer. &#xA;  This step was very tricky to accomplish because I had to maneuver around the &quot;NVIDIA Installer cannot continue. The NVIDIA graphics driver is not compatible with this version of Windows&quot; installation error message which prevents the NVIDIA compilers from being installed correctly.&#xA;  My objective is to run rapidGSEA without the CUDA enhancements since I am not using a computer with a NVIDIA GPU.&#xA;  I did this step because rapidGSEA and Broad Institute GSEA have a significantly richer set of gene set enrichment analytics than fgseaL as shown below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The GSEA method takes five default arguments and three optional arguments&#xA;GSEA &amp;lt;- function(exprsData, labelList, geneSets, numPermutations, metricString,&#xA;                 dumpFileName=&quot;&quot;, checkInput=TRUE, doublePrecision=FALSE) {...}&lt;/p&gt;&#xA;&#xA;&lt;p&gt;exprsData, labelList and geneSets refer to the data obtained in the previous section. numPermutations denotes the number of permutations in the resampling test, metricString denotes the local ranking measure (one of the following):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;naive_diff_of_classes&#xA;naive_ratio_of_classes&#xA;naive_log2_ratio_of_classes&#xA;stable_diff_of_classes&#xA;stable_ratio_of_classes&#xA;stable_log2_ratio_of_classes&#xA;onepass_signal2noise&#xA;onepass_t_test&#xA;twopass_signal2noise&#xA;twopass_t_test&#xA;stable_signal2noise&#xA;stable_t_test&#xA;overkill_signal2noise&#xA;overkill_t_test&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1283" LastEditorUserId="1283" LastEditDate="2017-08-13T21:52:33.080" LastActivityDate="2017-08-13T21:52:33.080" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2291" PostTypeId="2" ParentId="2265" CreationDate="2017-08-13T22:27:07.477" Score="3" Body="&lt;p&gt;You can do this easily with &lt;a href=&quot;https://hail.is&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hail&lt;/a&gt;.  Hail primarily uses BED files to annotate genetic datasets (see the last &lt;a href=&quot;https://hail.is/docs/stable/hail.VariantDataset.html#hail.VariantDataset.annotate_variants_table&quot; rel=&quot;nofollow noreferrer&quot;&gt;annotate_variants_table&lt;/a&gt; example), but you can manipulate BED files using Hail's general facilities for manipulating delimited text files.  For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat genes.bed&#xA;I   3746    3909    &quot;WBGene00023193&quot;    .   -&#xA;I   3746    3909    &quot;WBGene00023193&quot;    .   -&#xA;I   4118    4220    &quot;WBGene00022277&quot;    .   -&#xA;I   4118    4358    &quot;WBGene00022277&quot;    .   -&#xA;I   4118    10230   &quot;WBGene00022277&quot;    .   -&#xA;I   4220    4223    &quot;WBGene00022277&quot;    .   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The Hail script (python code):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from hail import *&#xA;hc = HailContext()&#xA;(hc&#xA; .import_table('genes.bed', impute=True, no_header=True)&#xA; .aggregate_by_key('f0 = f0, f3 = f3',&#xA;    'f1 = f1.min(), f2 = f2.max(), f4 = &quot;.&quot;, f5 = &quot;-&quot;')&#xA; .select(['f0', 'f1', 'f2', 'f3', 'f4', 'f5'])&#xA; .export('genes_merged.bed', header=False))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat genes_merged.bed &#xA;I   3746    3909    WBGene00023193  .   -&#xA;I   4118    10230   WBGene00022277  .   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I aggregate over chrom and name so this solution won't merge entries on different chromosomes. The &lt;code&gt;select&lt;/code&gt; is necessary to reorder the fields because &lt;code&gt;aggregate_by_key&lt;/code&gt; places the keys being aggregated over first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclosure: I work on Hail.&lt;/p&gt;&#xA;" OwnerUserId="1024" LastActivityDate="2017-08-13T22:27:07.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2292" PostTypeId="2" ParentId="2275" CreationDate="2017-08-14T04:20:37.817" Score="9" Body="&lt;p&gt;&lt;a href=&quot;https://hail.is&quot; rel=&quot;noreferrer&quot;&gt;Hail&lt;/a&gt; might be an option for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is actively developed by a growing team at the Broad.  It is rigorously tested (continuous integration, continuous deployment, bug reports get regression tests, blah blah blah).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was designed to solve this problem (among others).  It can import a variety of formats, including VCF, TSV, UCSC BED, JSON and interval files.  (We don't have explicit support for GFF, but we can probably handle them with general facilities.  If not, get in touch and we'll add support.)  It can call out to VEP (and soon Nirvana, Illumina's VEP rewrite).  It has general facilities to transform, filter, clean and query data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's more, we've curated a large collection of annotation resources (currently ~22 databases of annotations + VEP), hosted in a public bucket in Google cloud, and built an interactive query builder to select which resources you want to use.  Get in touch if you'd like us to add additional resources.  You can read more about it &lt;a href=&quot;https://hail.is/docs/stable/annotationdb.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I work on Hail.&lt;/p&gt;&#xA;" OwnerUserId="1024" LastActivityDate="2017-08-14T04:20:37.817" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2293" PostTypeId="1" AcceptedAnswerId="2325" CreationDate="2017-08-14T08:33:43.183" Score="5" ViewCount="347" Body="&lt;p&gt;Is the mappability of the centromeres in the GRCh38 genome reference similar to each other?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I can remember when GRCh38 came out, the sequence of the centromeres was determined by a combination of sequencing data and software prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the way the sequence of the centromeres has been determined, should we expect Illumina 2x150bp (or shorter, 2x75bp) reads to map relatively equally to all centromere sequences?&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2017-08-18T04:54:39.033" Title="GRCh38 centromeres mappability" Tags="|illumina|centromere|mappability|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2294" PostTypeId="2" ParentId="2043" CreationDate="2017-08-14T08:53:07.083" Score="4" Body="&lt;p&gt;You can use &lt;code&gt;finditer&lt;/code&gt; from python's &lt;a href=&quot;https://docs.python.org/2/library/re.html#module-re&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;re&lt;/code&gt;&lt;/a&gt; module. The advantage of this approach is it allows for getting the indices of those matches, which could be handy down the track.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; import re&#xA;&amp;gt;&amp;gt;&amp;gt; matches = re.finditer(r'(?=(AA))', 'AAAA')&#xA;&amp;gt;&amp;gt;&amp;gt; indices = [match.span(1) for match in matches]&#xA;&amp;gt;&amp;gt;&amp;gt; indices&#xA;[(0, 2), (1, 3), (2, 4)]&#xA;&amp;gt;&amp;gt;&amp;gt; num_matches = len(indices)&#xA;&amp;gt;&amp;gt;&amp;gt; num_matches&#xA;3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="149" LastEditorUserId="149" LastEditDate="2017-08-14T08:58:49.933" LastActivityDate="2017-08-14T08:58:49.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2295" PostTypeId="2" ParentId="2282" CreationDate="2017-08-14T10:33:13.743" Score="9" Body="&lt;p&gt;For normal RNA-seq PCR duplicates are normally kept in, but the duplication rate can be used as a quality control: The higher the duplication rate, the lower the quality. For expression analysis, it is probably best to discard high duplication rate samples, rather than deduplicate them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, the smaller the amount of RNA input into the library prepartion the worst the duplication. Many protocols for very low input quantities (such as single cell) include random barcodes called UMIs (Unique Molecular Identifiers). These allow PCR duplicates to be distinguished from genuinely independent molecules that just happen to have the sample mapping position. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2017-08-14T10:33:13.743" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2297" PostTypeId="2" ParentId="2293" CreationDate="2017-08-14T10:49:09.487" Score="3" Body="&lt;p&gt;I doubt it, unless you are asking if mapping would be similarly bad for all centromeres. Here are some repetitive structures (probably not centromeric) that I've found in the nanopore reads for &quot;human&quot; sample NA12878, produced by the &lt;a href=&quot;https://github.com/nanopore-wgs-consortium/NA12878&quot; rel=&quot;nofollow noreferrer&quot;&gt;nanopore-WGS consortium&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2D3cq.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2D3cq.jpg&quot; alt=&quot;Repetitive human reads #1&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These structures are consistent in that they repeat lots of times, but the internal patterns can be quite different. Here are a few more:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SU65U.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SU65U.jpg&quot; alt=&quot;Repetitive human reads #2&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that centromeres need to be uniquely linkable to a single chromosome, it would make sense to me if the centromere internal structure is unique to each chromosome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's possible to have highly-repetitive structures which don't map to each other. While I haven't dug too deeply into the human reads, I've looked at the 5 most-compressible regions in an assembled rodent parasite (&lt;em&gt;Nippostrongylus brasiliensis&lt;/em&gt;) genome, and found no internal similarities between them:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GtvQz.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GtvQz.jpg&quot; alt=&quot;Nippo most-compressible regions&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the issues with assembly from Illumina reads is that these highly-repetitive regions get collapsed into a single repeat (or at best a region up to twice the fragment length). With internal repeat units that have identity of over 98%, assembling the true sequence is very difficult, even with an exact knowledge of paired read separation. Even if this were possible, it can be impossible to correctly place a read because multiple internal units could be identical (or similarly different) to the sequenced read.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-08-14T10:49:09.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2298" PostTypeId="1" AcceptedAnswerId="2299" CreationDate="2017-08-14T19:51:21.827" Score="16" ViewCount="23773" Body="&lt;p&gt;What the difference between TPM and CPM when dealing with RNA seq data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What metrics would you use if you have to perform some down stream analysis other than Differential expression for eg. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clustering analysis using Hclust function and then plotting heat map to find differences in terms of expression levels, correlation and pca&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it wrong to use TPM for such analysis, if yes then when does one use TPM versus CPM. &lt;/p&gt;&#xA;" OwnerUserId="926" LastActivityDate="2021-01-06T08:22:48.657" Title="Difference between CPM and TPM and which one for downstream analysis?" Tags="|rna-seq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2299" PostTypeId="2" ParentId="2298" CreationDate="2017-08-14T21:15:27.507" Score="13" Body="&lt;p&gt;You can find the various equations in &lt;a href=&quot;https://haroldpimentel.wordpress.com/2014/05/08/what-the-fpkm-a-review-rna-seq-expression-units/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this oft-cited blog post from Harold Pimentel&lt;/a&gt;. CPM is basically depth-normalized counts, whereas TPM is length-normalized (and then normalized by the length-normalized values of the other genes).&lt;/p&gt;&#xA;&lt;p&gt;If one has to choose between those two choices one typically chooses TPM for most things, since generally the length normalization is handy. Realistically, you probably want &lt;code&gt;log(TPM)&lt;/code&gt; since otherwise noise in your most highly expressed genes dominates over small expression signals.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="3355" LastEditDate="2021-01-06T08:22:48.657" LastActivityDate="2021-01-06T08:22:48.657" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="2300" PostTypeId="1" AcceptedAnswerId="2314" CreationDate="2017-08-15T07:11:42.977" Score="3" ViewCount="269" Body="&lt;p&gt;&lt;em&gt;Note: this question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/267545/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioStars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am getting peptides using biomaRt library in R for the gene 'BRCA1'. 27 different aminoacid sequences are returned, and 12 of those have an asterisk (*) in the end, while 15 do not. What does that mean? I know that asterisk is used to indicate stop codon. So, does that mean the peptides without an asterisk do not have a stop codon? Is that possible at all? Are all of those valid peptides or not?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is the R code I used: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(biomaRt);&#xA;mart = useMart('ensembl', dataset='hsapiens_gene_ensembl');&#xA;seq = getSequence(id='BRCA1', type='hgnc_symbol', seqType='peptide', mart = mart);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;MDLSALRVEEVQNVINAMQKILECPICLELIKEPVSTKCDHIFCKVLLCCPSWSTVVRS*&lt;/code&gt; is an example sequence with an asterisk in the end. &lt;code&gt;MDLSALRVEEVQNVINAMQKILECPICLELIKEPVSTKCDHIFCKSLQESTRFSQLVEEL‌​LKIICAFQLDTGLEYANSYN‌​FAKKENNSPEHLKDEVSI&lt;/code&gt; is an example without an asterisk.&lt;/p&gt;&#xA;" OwnerUserId="1331" LastEditorUserId="73" LastEditDate="2018-01-10T06:29:26.623" LastActivityDate="2018-01-10T07:56:14.690" Title="Why are some of the gene peptides returned by biomaRt missing an asterisk in the end?" Tags="|gene|proteins|biomart|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2301" PostTypeId="2" ParentId="2300" CreationDate="2017-08-15T07:25:59.873" Score="4" Body="&lt;p&gt;If you &lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Transcript/Summary?g=ENSG00000012048;r=17:43093586-43125364;t=ENST00000477152&quot; rel=&quot;nofollow noreferrer&quot;&gt;look at the original data at Ensembl&lt;/a&gt; you'll notice that most of these are labeled &quot;CDS 3' incomplete&quot; &lt;s&gt;and have a TSL (transcript support level) of 1, which is as low as it goes&lt;/s&gt;. It seems likely that that is simply an incomplete annotation. I'm not surprised that there are a bunch of these for BRCA1, since there was a long time when there were a LOT of legal issues surrounding working on it.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-08-15T09:11:38.693" LastActivityDate="2017-08-15T09:11:38.693" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2302" PostTypeId="1" AcceptedAnswerId="2303" CreationDate="2017-08-15T08:15:32.233" Score="4" ViewCount="305" Body="&lt;p&gt;Say I have reads that overlap some genes that produce small RNAs, but I want only those reads that start at exactly the TSS of the loci. In other words, reads whose 5' end match the 5' end of a genomic feature. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&#xA;   5'....3'&#xA;...+++++++... Gene&#xA;...0000000... R1&#xA;...00000000.. R2&#xA;..000000000.. R3&#xA;..00000000... R4&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output should be R1 and R2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe this is a fairly common bioinformatics operation but somehow it doesn't seem to be an option in the general use tools I looked at (py-bedtools, samtools, bedops). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A solution I thought of would be:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;reduce the gene coordinates to its 5' end, and &lt;/li&gt;&#xA;&lt;li&gt;using bedtools &lt;code&gt;bedClosest&lt;/code&gt; to annotate the distance between reads and genes, and finally &lt;/li&gt;&#xA;&lt;li&gt;select reads that overlap (d=0) and write them to a (bam) file.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This involves quite a bit of wrangling data with &lt;code&gt;pybedtools&lt;/code&gt; or &lt;code&gt;awk/bash&lt;/code&gt;, and I wonder is there is a more elegant solution to this. Solutions in &lt;code&gt;R/Bioconductor&lt;/code&gt; are also welcome.&lt;/p&gt;&#xA;" OwnerUserId="108" LastActivityDate="2017-08-15T08:33:27.510" Title="How to filter intervals (reads or genomic coordinates) that have the exact same 5' or 3' ends?" Tags="|bedtools|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2303" PostTypeId="2" ParentId="2302" CreationDate="2017-08-15T08:33:27.510" Score="1" Body="&lt;p&gt;Presuming you have deepTools installed:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#/!usr/bin/env python&#xA;from deeptoolsintervals.parse import GTF&#xA;import pysam&#xA;&#xA;anno = GTF([&quot;some/gtf/or/bed/file&quot;, &quot;or/more/than/one/if you like/&quot;])&#xA;bam = pysam.AlignmentFile(&quot;something.bam&quot;)&#xA;for read in bam:&#xA;    s = read.reference_start&#xA;    if read.is_reverse:&#xA;        s = read.reference_end&#xA;    e = s + 1&#xA;    overlaps = anno.findOverlaps(read.reference_name s, e, matchType=4)&#xA;    # For a GTF, parse the results to see if this is the correct type.&#xA;    # For a BED3/BED6 file there will be no exons, so anything other than&#xA;    # None indicates an appropriate overlap&#xA;&#xA;    do something&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;deeptoolsintervals&lt;/code&gt; in the GTF/BED parsing and custom interval tree library used by deepTools. &lt;code&gt;anno&lt;/code&gt; holds the interval tree and the associated information (gene/transcript name or ID, exons, strand, etc.). the &lt;code&gt;matchType=4&lt;/code&gt; is the pivotal part since that specifies that only overlaps having the exact same start position should be included. Assuming you have a stranded library, you might instead do something like switch between &lt;code&gt;matchType=4&lt;/code&gt; (the same start position) and &lt;code&gt;matchType=5&lt;/code&gt; (the same end position) depending on the orientation of the alignment and the the &lt;code&gt;strand=&lt;/code&gt; of the gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that you'll need to make a &lt;code&gt;set()&lt;/code&gt; to hold read names to find the mates (that or jump around the file to get them, which is probably slower).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-08-15T08:33:27.510" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2305" PostTypeId="2" ParentId="2298" CreationDate="2017-08-15T09:45:02.523" Score="3" Body="&lt;p&gt;Neither CPM nor TPM are well suited here, because neither performs robust cross-sample normalisation (see the blog post Devon linked to).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DESeq2 provides two robust log-space normalisation methods for downstream analysis, the &lt;em&gt;regularised log&lt;/em&gt; (&lt;code&gt;rlog&lt;/code&gt;), and the &lt;em&gt;variance stabilising transformation&lt;/em&gt; (&lt;code&gt;vst&lt;/code&gt;). The &lt;a href=&quot;http://bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#data-transformations-and-visualization&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2 vignette explains how to use these&lt;/a&gt; for things like hclust.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;On a more general note, CPM does not account for transcript length differences, while TPM does. If the choice is between TPM and CPM I would therefore use TPM. However, if you are only comparing the same transcripts across experiments, the transcript length is actually invariant so it doesn’t matter (but CPM is still not a good cross-experiment normalisation).&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2017-08-15T09:45:02.523" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="2306" PostTypeId="1" AcceptedAnswerId="2311" CreationDate="2017-08-15T10:46:20.913" Score="13" ViewCount="3282" Body="&lt;p&gt;&lt;code&gt;fast5&lt;/code&gt; is a variant of &lt;a href=&quot;https://support.hdfgroup.org/HDF5/whatishdf5.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;HDF5&lt;/code&gt;&lt;/a&gt; the native format in which raw data from Oxford Nanopore MinION are provided. You can easily extract the reads in fast5 format into a standard fastq format, using for example &lt;a href=&quot;https://poretools.readthedocs.io/en/latest/&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;poretools&lt;/code&gt;&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say I have aligned these reads in &lt;code&gt;fastq&lt;/code&gt; format to an external reference genome, resulting in a &lt;code&gt;SAM&lt;/code&gt; file. Say I have then taken a subset of the  &lt;code&gt;SAM&lt;/code&gt; file, according to the &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;noreferrer&quot;&gt;bitwise flag&lt;/a&gt;, to include only the reads that map to the reference. With the read ID, I can then grep them out from the file containing the reads in &lt;code&gt;fastq&lt;/code&gt; format, generating a subset file in &lt;code&gt;fastq&lt;/code&gt; format containing only the IDs that have mapped to the reference.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my question is, can we subset reads from the &lt;code&gt;fast5&lt;/code&gt; archive according to the list of mapping reads as taken from the file with reads in &lt;code&gt;fastq&lt;/code&gt; format? This is for educational purposes, so that we have a smaller starting archive, and the &lt;code&gt;fast5&lt;/code&gt; -&gt; &lt;code&gt;fastq&lt;/code&gt; extraction takes less cpu time. &lt;/p&gt;&#xA;" OwnerUserId="294" LastEditorUserId="149" LastEditDate="2017-08-16T08:52:58.910" LastActivityDate="2018-11-02T12:22:56.053" Title="How to convert fastq to fast5" Tags="|nanopore|software-recommendation|fastq|format-conversion|fast5|" AnswerCount="2" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2308" PostTypeId="1" CreationDate="2017-08-15T16:34:21.093" Score="8" ViewCount="708" Body="&lt;p&gt;I'm comparing the results that I obtain when doing a DE analysis with the Wald test and the likelihood-ratio test. One the thing that I've noticed is that there are many genes with 'beta' close to zero that are considered differentially expressed between the conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that the likelihood-ratio test does't use the beta values to calculate the p-values, but I find it strange that transcripts with similar expression values between the conditions are being considered differentially expressed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Volcano plot (Wald test q-values):&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ATPtp.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ATPtp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Volcano plot (likelihood-ratio test q-values):&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/hovqx.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hovqx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1334" LastEditorUserId="96" LastEditDate="2017-08-18T04:41:53.457" LastActivityDate="2017-09-26T00:10:53.477" Title="Sleuth: transcripts with beta close to 0 are considered differentially expressed in a likelihood-ratio test" Tags="|rna-seq|differential-expression|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2309" PostTypeId="2" ParentId="2306" CreationDate="2017-08-15T18:44:00.690" Score="3" Body="&lt;p&gt;I do this very frequently, using the read file name to identify FAST5 files associated with particular reads. If a FASTQ record includes the channel number and read number (and preferably the runID as well), then I use that information to find the associated FAST5 read.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If reads are called with Albacore, then the &lt;code&gt;sequencing_summary.txt&lt;/code&gt; file has additional information. The first column is the exact name of the file that was called, the second column is the read ID of the FASTQ sequence, and the third column is the run ID associated with that read ID. This is much easier to work with, but does require calling reads with Albacore (which, admittedly, is what seems to be producing the best results at the moment).&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-08-16T07:38:17.043" LastActivityDate="2017-08-16T07:38:17.043" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2310" PostTypeId="1" AcceptedAnswerId="2312" CreationDate="2017-08-16T01:27:43.727" Score="3" ViewCount="90" Body="&lt;p&gt;In my RNAseq dataset of differentiated stem-cell lines, some samples have far fewer significantly differentially expressed genes than others. QC shows that this is because there are way fewer reads for one sample than the others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can gene co-expression networks be used to infer differential expression? For example, if certain genes just miss the cutoff q-value cutoff  If so, what papers have done so? &lt;/p&gt;&#xA;" OwnerUserId="823" LastActivityDate="2017-08-16T08:27:56.623" Title="Can gene co-expression networks be used to help identify differentially expressed genes?" Tags="|rna-seq|differential-expression|networks|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2311" PostTypeId="2" ParentId="2306" CreationDate="2017-08-16T04:30:09.763" Score="6" Body="&lt;p&gt;&lt;strong&gt;NOTICE&lt;/strong&gt;:&#xA;I have altered my answer slightly from the original as I have turned the original script into a &lt;code&gt;pip&lt;/code&gt; installable program (with tests) and have updated the links and code snippets accordingly. The essence of the answer is still exactly the same.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This is something I have been meaning to get around to for a while, so thanks for the prompt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have &lt;a href=&quot;https://github.com/mbhall88/fast5seek&quot; rel=&quot;nofollow noreferrer&quot;&gt;created a python program called &lt;code&gt;fast5seek&lt;/code&gt;&lt;/a&gt; to do what (I think) you're after.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you have mentioned this is for educational purposes I have added a tonne of comments to the code too so I think you shouldn't have any issues following it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The docs on the &lt;a href=&quot;https://github.com/mbhall88/fast5seek&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub repo&lt;/a&gt; have all the info, but for those reading along at home&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pip3 install fast5seek&#xA;fast5seek -i /path/to/fast5s -r in.fastq in.bam in.sam -o out.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What it does is read in &lt;code&gt;&amp;lt;in.fastq|in.bam|in.sam&amp;gt;&lt;/code&gt; and extract the read id from each header. It then goes through all the fast5 files under &lt;code&gt;/path/to/fast5s&lt;/code&gt; and checks whether their read id is in the set of read ids from &lt;code&gt;&amp;lt;in.fastq|in.bam|in.sam&amp;gt;&lt;/code&gt;. If it is, the path to the file is written to it's own line in &lt;code&gt;out.txt&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If no output (&lt;code&gt;-o&lt;/code&gt;) is given, it will write the output to stdout.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you wanted to pipe these paths into another program, you could do something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mkdir subset_dir/&#xA;fast5seek -i /path/to/fast5s/ -r in.fastq | xargs cp -t subset_dir/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above example would copy the fast5 files that are found in your fastq/BAM/SAM to &lt;code&gt;subset_dir/&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="149" LastEditorUserId="149" LastEditDate="2018-11-02T12:22:56.053" LastActivityDate="2018-11-02T12:22:56.053" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2312" PostTypeId="2" ParentId="2310" CreationDate="2017-08-16T06:41:26.923" Score="2" Body="&lt;p&gt;While you can use networks to find differentially expressed genes (see the &lt;a href=&quot;https://labs.genetics.ucla.edu/horvath/CoexpressionNetwork/Rpackages/WGCNA/&quot; rel=&quot;nofollow noreferrer&quot;&gt;WGCNA&lt;/a&gt; package, which does this) in my experience this ends up largely matching what you'd get using a traditional package with a looser threshold for significance. Given the time savings of traditional packages, there's rarely any gain to using networks (it won't hurt to try, just note that it'll take some time). If some very interesting genes are just above your significance threshold then change your threshold. You need to validate your findings in some way anyway, so your p-value threshold is partly just a way to protect you from wasting your time (but if background knowledge suggests that your time wouldn't be wasted...).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-08-16T06:41:26.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2313" PostTypeId="2" ParentId="2310" CreationDate="2017-08-16T08:27:56.623" Score="2" Body="&lt;p&gt;Co-expression network will give you an idea about the genes having similar expression patterns and the nodes will be decided on the basis of correlation scores and nothing much you will get from differential gene expression perspective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest you to try some other models for differential expression analysis like &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/baySeq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;baySeq&lt;/a&gt; , &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/DESeq2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq&lt;/a&gt;, &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/edgeR.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;edgeR&lt;/a&gt;, &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/NOISeq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;NOIseq&lt;/a&gt;. Depending upon the model you use for analysis you might get some extra significant DEG. Also, try different significance cutoff and see how it's changing the analysis result to get the threshold that suits the data best.&lt;/p&gt;&#xA;" OwnerUserId="206" LastActivityDate="2017-08-16T08:27:56.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2314" PostTypeId="2" ParentId="2300" CreationDate="2017-08-16T08:49:48.813" Score="8" Body="&lt;p&gt;If it's 3' incomplete that means the evidence used to create it was a fragment. &lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Transcript/SupportingEvidence?db=core;g=ENSG00000012048;r=17:43093586-43125364;t=ENST00000477152&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt;'s the evidence used to construct BRCA1-214 ENST00000477152.5, a 3' incomplete.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that &lt;a href=&quot;http://www.ebi.ac.uk/ena/data/view/AK307553.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;there&lt;/a&gt;'s a full length cDNA from EMBL, AK307553.1, which was used to create this model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sequence was mapped against the genome sequence to create the transcript model. When searching for the translation, it was found that the open reading frame was started, but there was no stop codon. This suggests that the cDNA AK307553.1 is actually a fragment, that the mRNA was broken or cleaved before it was reverse-transcribed sequenced. We display it in Ensembl in the hope that this will lead someone to identify the full length transcript that it represents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many people choose not to work with these incomplete transcripts. You can do this by filtering by Gencode Basic in biomaRt.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastEditorUserId="48" LastEditDate="2018-01-10T07:56:14.690" LastActivityDate="2018-01-10T07:56:14.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2316" PostTypeId="1" AcceptedAnswerId="2317" CreationDate="2017-08-17T02:33:01.107" Score="7" ViewCount="547" Body="&lt;p&gt;I am running a slow downstream analysis on a large set of nanopore reads (approx 3 million), and would like to split them into smaller chunks, run the analysis in massively parallel, and then recombine. Originally I just split the FASTQ into chunks, re-aligned each chunk, and then merged the output, but here I would like to use an existing alignment so I can compare results with existing analyses (i.e. the alignments must be the same).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I efficiently split a FASTQ file and a BAM file giving the alignment of the FASTA file into chunks, ensuring that all of the reads in FASTQ chunk 1 are in BAM chunk 1, vice versa and so on?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My FASTQ is approximately 45GB and my BAM is 33GB, so I would prefer to avoid storing one of the two files in memory if possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Here's some pseudocode of exactly what I'm trying to do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# input: in.bam, in.fastq, chunk_size&#xA;i &amp;lt;- 0&#xA;for fastq_read in in.fastq:&#xA;    bam_read &amp;lt;- extract fastq_read.read_name from in.bam&#xA;    n &amp;lt;- i modulo chunk_size&#xA;    write fastq_read to out.n.fastq&#xA;    write bam_read to out.n.bam&#xA;    i &amp;lt;- i + 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I could swap the above to iterate through the bam file and fetch from the fastq if that's easier.&lt;/p&gt;&#xA;" OwnerUserId="163" LastEditorUserId="163" LastEditDate="2017-08-17T03:41:07.107" LastActivityDate="2021-07-23T01:55:27.753" Title="Split FASTQ and matching BAM into matching chunks" Tags="|sam|fasta|nanopore|fastq|subset|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2317" PostTypeId="2" ParentId="2316" CreationDate="2017-08-17T07:33:10.510" Score="5" Body="&lt;p&gt;Hmm, it's hard to think of a super efficient way of doing this (assuming the files aren't ordered the same - if they are then this whole answer is basically redundant). And also assuming the read ids for both files aren't a perfect intersection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Off the top of my head you probably want to build a set of read ids for the fastq file and another for the bam. Some python code to get started on that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pysam&#xA;import itertools&#xA;&#xA;def get_read_id_fastq(ref_path):&#xA;    &quot;&quot;&quot;Extracts the read ids from a fastq file.&quot;&quot;&quot;&#xA;    read_ids = set()&#xA;    with open(ref_path, 'r') as ref:&#xA;        for line in ref:&#xA;            if line.startswith('@'):  # i.e if line is header&#xA;                # split the line on spaces, take the first element, remove @&#xA;                read_id = line.split(' ')[0].replace('@', '')&#xA;                read_ids.add(read_id)&#xA;&#xA;    return read_ids&#xA;&#xA;def get_read_id_bam(ref_path):&#xA;    &quot;&quot;&quot;Extract the read ids from a BAM file.&quot;&quot;&quot;&#xA;&#xA;    read_ids = set()&#xA;    with pysam.AlignmentFile(ref_path, 'r', ignore_truncation=True) as ref:&#xA;        for read in ref:&#xA;            # query_name is the query template name&#xA;            read_ids.add(read.query_name)&#xA;&#xA;    return read_ids&#xA;&#xA;fastq_ids = get_read_id_fastq(fastq_path)&#xA;bam_ids = get_read_id_bam(bam_path)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then take the intersection of these two sets and that's your common read ids. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;common_ids = fastq_ids &amp;amp; bam_ids&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The next part will be a bit more involved. You will have to iterate through each file one at a time. I would suggest that for the first one you iterate through, create a running dictionary with the read id that was written as the key and the 'chuck number' it was written to as the value. You could create a cycle for your chunk size to manage this effectively&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chunk_cycle = itertools.chain(*zip(range(chunk_size)*len(common_ids)))&#xA;write_idx = {}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The next part will probably require you to put in some hard-fought times getting it to work. I'll put in some rough pseudo-code to give you an idea.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for line in file:&#xA;    read_id = # get line read_id&#xA;    if read_id in common_ids:&#xA;        chunk_num = chunk_cycle.next()&#xA;        write_idx[read_id] = chunk_num&#xA;        file_to_write_to = 'out.{}.bam'.format(chunk_num)&#xA;        # open this file or write to it if already open&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When you go to do the next file when you find a read id is in the common set, you look up it's value in &lt;code&gt;write_idx&lt;/code&gt; and this will give you the chunk number to write to. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reads wont be in the exact same order between the two files, but you could sort later if you needed it (not sure it would matter?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps. Sorry I couldn't give you more, but this should give you a head start hopefully.&lt;/p&gt;&#xA;" OwnerUserId="149" LastEditorUserId="149" LastEditDate="2017-08-17T13:00:35.147" LastActivityDate="2017-08-17T13:00:35.147" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2318" PostTypeId="1" AcceptedAnswerId="2319" CreationDate="2017-08-17T13:10:12.177" Score="9" ViewCount="5147" Body="&lt;p&gt;I recently installed Ubuntu 16.04 (because I was still using 12.04). But it seems my bedtools scripts don't work properly anymore. I can't figure out how to use the new bedtools for my old ways. What I want to do is get the number of reads from a bam file, per interval from a bed file. It was very simple with my old version of Ubuntu and bedtools:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools coverage -abam file.bam -b All_peaks.bed &amp;gt; file.cov.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But this doesn't seem to work anymore. I used to get small files as results (~3MB), but now huge files are created (~4GB) including read names??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using bedtools v2.25 now.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2017-08-17T13:21:05.450" Title="How to count reads in bam per bed interval with bedtools" Tags="|sam|bedtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2319" PostTypeId="2" ParentId="2318" CreationDate="2017-08-17T13:21:05.450" Score="10" Body="&lt;p&gt;The order of &lt;code&gt;-a&lt;/code&gt; and &lt;code&gt;-b&lt;/code&gt; switched at some point. You want:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools coverage -a All_peaks.bed -b file.bam &amp;gt; file.cov.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For reference, this is the end of the help output in version 2.25:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Default Output:  &#xA;     After each entry in A, reports: &#xA;       1) The number of features in B that overlapped the A interval.&#xA;       2) The number of bases in A that had non-zero coverage.&#xA;       3) The length of the entry in A.&#xA;       4) The fraction of bases in A that had non-zero coverage.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this is the equivalent output from version 2.19:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Default Output:  &#xA;     After each entry in B, reports: &#xA;       1) The number of features in A that overlapped the B interval.&#xA;       2) The number of bases in B that had non-zero coverage.&#xA;       3) The length of the entry in B.&#xA;       4) The fraction of bases in B that had non-zero coverage.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;: The change in behavior happened in version 2.24:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We have changed the behavior of the coverage tool such that it is consistent with the other tools. Specifically, coverage is now&#xA;  computed for the intervals in the A file based on the overlaps with the B file, rather than vice versa.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-08-17T13:21:05.450" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2321" PostTypeId="1" CreationDate="2017-08-17T16:42:31.383" Score="4" ViewCount="72" Body="&lt;p&gt;I have not found any work which investigates assessment of differences in levels of secreted proteins by taking advantage of differential expression of the genes which mediate the secretory pathway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, suppose the gene whose product binds specifically to a certain signal peptide is down-regulated. I would assume that this implies a down-regulation in all the levels of secreted proteins to which it binds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is such an analysis possible to date? If so, where? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="823" LastActivityDate="2018-10-28T15:00:03.200" Title="Using signal peptide and the expression levels of signal recognition particle in secretome analysis" Tags="|proteins|differential-expression|networks|interactions|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2322" PostTypeId="2" ParentId="2201" CreationDate="2017-08-17T19:47:04.523" Score="1" Body="&lt;p&gt;Here is my solution to the problem. I am posting it here in case anyone else come up with the same idea but did not know how to formulate it in mathematical notation!&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$$ \psi_i = \sum_{i=1}^9 S_i $$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;where $S_i$ is the score function and defined as:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$$&#xA;S_i = \cases{&#xA;  0 &amp;amp; \text{if $\theta(x_i) &amp;lt; Q_1$ or $\theta(x_i) = $ benign/neutral} \\&#xA;  0.5 &amp;amp; \text{if $Q_1 \leq \theta(x_i) &amp;lt; Q_3$ or $\theta(x_i) = $ possibly damaging/uncertain} \\&#xA;  1 &amp;amp; \text{if $\theta(x_i) \geq Q_3$ or $\theta(x_i) = $ damaging}&#xA;}&#xA;$$&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The $\theta(x_i)$ is the pathogenicity or conservation score for variant $x$ as defined by model $i$ and $Q$ denotes the quartile range for scores from M-CAP, CADD, GERP and Phylop models.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1250" LastEditorUserId="29" LastEditDate="2017-08-18T17:39:08.600" LastActivityDate="2017-08-18T17:39:08.600" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2323" PostTypeId="1" CreationDate="2017-08-18T00:23:15.570" Score="4" ViewCount="633" Body="&lt;p&gt;&lt;em&gt;Note: this question has also been asked on &lt;a href=&quot;https://www.reddit.com/r/bioinformatics/comments/6udxef/how_to_confirm_a_gene_is_derived_by_exon/?st=j6hk7ejm&amp;amp;sh=885530b6&quot; rel=&quot;nofollow noreferrer&quot;&gt;reddit&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to confirm that the sequence of a novel gene is derived by exon shuffling between several different genes. I have the promoter sequence, gene sequence, and mRNA (with defined exon/intro boundaries). I've tried performing several database searches with the sequences, but each search produces a set of different unrelated hits. How would one go about using this info to confirm the exon shuffling hypothesis? Any tips are appreciated. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1343" LastEditorUserId="73" LastEditDate="2017-12-06T00:26:39.520" LastActivityDate="2020-04-28T04:19:54.867" Title="How to confirm exon shuffling in a gene?" Tags="|ngs|sequence-analysis|molecular-genetics|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2324" PostTypeId="1" CreationDate="2017-08-18T03:12:20.163" Score="3" ViewCount="15741" Body="&lt;p&gt;I'm currently working with data from a Luminex multiplex assay. In this assay, the concentrations of 17 different analyte proteins were identified in 12 groups in triplicate. One of these 17 groups was used as the control, and the log2 fold changes were calculated for the analyte concentration of each sample in each group using the average control concentration for that analyte.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, now I would like to calculate a p-value for the identified fold changes if possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current preliminary idea is to perform the t test for each group compared with the control group for each analyte, but I don't think that my group sizes are large enough for this. My statistical knowledge is lacking, so if there is a better way of calculating these p-values I don't know about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better way of obtaining these p-values?&lt;/p&gt;&#xA;" OwnerUserId="506" LastActivityDate="2017-08-19T17:55:22.827" Title="How to calculate p-values for fold changes?" Tags="|statistics|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2325" PostTypeId="2" ParentId="2293" CreationDate="2017-08-18T04:54:39.033" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;should we expect Illumina 2x150bp (or shorter, 2x75bp) reads to map relatively equally to all centromere sequences?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No. It has long been established that different chromosomes are associated with different centromeric sequences. It is sometimes possible to tell which chr a read is originated from based on its sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The GRCh38 centromeres are trickier. As I remember, the centromeric sequences were computationally generated with a Markov chain (or something alike) modeled after Venter's genome. GRC can distinguish most chromosomes, but not all. Some alpha arrays are placed onto 2 or 4 chromosomes. The original GRCh38 keeps all 4 copies. When you download GRCh38 for mapping purposes, only one copy is retained; additional copies are hard masked along with PARs on chrY.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to know more, see &lt;a href=&quot;http://genome.cshlp.org/content/early/2017/04/07/gr.213611.116.abstract&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/24501022&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-08-18T04:54:39.033" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2326" PostTypeId="2" ParentId="2324" CreationDate="2017-08-18T05:08:01.100" Score="0" Body="&lt;p&gt;So to get a p value you must have one null hypothesis like the fold changes are significant or not. Then you test the hypothesis against the data and calculate the p value (likeliness or unlikeliness) for a given threshold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, t test is good enough to calculate the p-value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://willett.ece.wisc.edu/wp-uploads/2016/01/05b-TandP.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://willett.ece.wisc.edu/wp-uploads/2016/01/05b-TandP.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="206" LastActivityDate="2017-08-18T05:08:01.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2327" PostTypeId="2" ParentId="2324" CreationDate="2017-08-18T06:51:59.027" Score="3" Body="&lt;p&gt;Your null hypothesis would be that the fold-changes are 0, so you can either do the T-test accordingly or simply do away with the fold changes and perform the T-test between the raw values in the group (this is preferable to performing a T-test of one group vs. 0, since it allows you to assess the expected variability around 0). Note, however, that 3 samples is pretty much the bare minimum needed for any statistics, so your power will be terrible and your estimation of variance will likely be pretty inaccurate. Consequently, take any p-values with an appropriate grain of salt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my knowledge there aren't any good alternatives for your situation without having a good expected background distribution.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-08-18T06:51:59.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2328" PostTypeId="2" ParentId="2324" CreationDate="2017-08-18T08:58:29.290" Score="1" Body="&lt;p&gt;You can't calculate a p-value on the fold-change values, you need to use the  concentrations in triplicate thus giving a measure of the variance for the t-test to use. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;t-test assumes your data are normally distributed, if they aren't you're going to get spurious p-values. If you aren't sure a non-parametric test like Wilcoxon is better. It will be less sensitive although with only 3 replicates your experiment is low powered anyway.&lt;/p&gt;&#xA;" OwnerUserId="1092" LastActivityDate="2017-08-18T08:58:29.290" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2329" PostTypeId="1" AcceptedAnswerId="2330" CreationDate="2017-08-18T10:13:12.797" Score="8" ViewCount="3595" Body="&lt;p&gt;I'm trying to understand the magnitude of batch effects in my RNA-seq samples, and I was wondering which expression units are more suitable to draw a PCA. I'm thinking of either &lt;code&gt;counts&lt;/code&gt; or &lt;code&gt;TPM&lt;/code&gt;, but things like &lt;a href=&quot;http://bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#data-transformations-and-visualization&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;rlog&lt;/code&gt; or &lt;code&gt;vst&lt;/code&gt;&lt;/a&gt; could work too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, I'm wondering whether either of this units should be log-transformed first, to avoid high-abundance transcripts driving the PCA.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2017-08-18T21:13:03.153" Title="Which measure should be used in a PCA or RNA-seq data? TPM or counts?" Tags="|rna-seq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2330" PostTypeId="2" ParentId="2329" CreationDate="2017-08-18T10:17:53.487" Score="5" Body="&lt;p&gt;tldr: log transform counts and TPMs, but rlog/vst are preferred&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TPM should be log transformed to get more useful results. If you're using DESeq2 already (given the reference to &lt;code&gt;rlog&lt;/code&gt; and &lt;code&gt;vst&lt;/code&gt;, this seems likely), then please go ahead and use &lt;code&gt;rlog&lt;/code&gt; or &lt;code&gt;vst&lt;/code&gt;. That will give you more reasonable results than raw counts. If you're stuck with counts for some reason, then first use normalized counts so they're at least a bit more comparable and then log transform them so your highly expressed genes aren't driving everything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: As an aside, if you know what the batch effect is (e.g., library prep. date), it's sometimes convenient to include that in your model. You can then assess the genes that are actually changed due to that, which is sometimes handy to know (e.g., which genes might be more/less prone to degradation).&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-08-18T10:57:03.527" LastActivityDate="2017-08-18T10:57:03.527" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2331" PostTypeId="2" ParentId="2324" CreationDate="2017-08-18T10:56:05.363" Score="4" Body="&lt;p&gt;The key point here is whether or not the values are approximately normally distributed and whether any transformations can be applied to make them so. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a t-test, the most important thing is that there is no relationship between the mean and the variance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its also correct that you don't want to divide by the mean of the controls. You want to retain all the information. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So here is how i'd start, first calculate the mean and standard deviation of each of your 18 groups (17 treatments + control), and plot this on a scatter plot (i.e. mean on X, SD on Y).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a relationship between the two?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Repeat the exercise with log transformed values. Is the relationship more or less.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pick the set of values that has the weakest mean-SD relationship. I'm guessing the log transformed might well be better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $X = (x_1, x_2, x_3)$ is your controls and $Y_i = (y_{i,1}, y_{i,2}, y_{i,3})$ are the treated samples from treatment i, to test the difference between treatment and control, do a t-test on X and Y (not &lt;code&gt;T/mean(c)&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for example in R&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X = c(x_1, x_2, x_3)&#xA;Y_1 = c(y_11, y_12, y_13)&#xA;t.test (X, Y_1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or in excel is if the three values for X are in column A and the three values for $Y_1$ in column B&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;=T.TEST(A1:A3,B1:B3,2,3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you are going to do 17 of these, remember to do a multiple testing correction by multiplying the resulting p-values by 17. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to compare treatment 1 ($Y_1$) vs treatment 2 ($Y_2$) do the comparison directly, not taking the control into account:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;t.test(Y_1, Y_2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Remember that if you use log transformed values, you calculate log fold changes using &lt;code&gt;mean(Y)-mean(X)&lt;/code&gt; rather than &lt;code&gt;mean(Y)/mean(X)&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="235" LastEditorUserId="48" LastEditDate="2017-08-19T17:55:22.827" LastActivityDate="2017-08-19T17:55:22.827" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2332" PostTypeId="2" ParentId="2329" CreationDate="2017-08-18T11:13:10.100" Score="4" Body="&lt;p&gt;PCA works best when the input data is approximately normally distributed on each dimension. It would be a good idea to do some initial data quality checks to verify that this is the case (and transform the data appropriately if not), or at least verify that the data is approximately normally distributed in the aggregate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For looking at Illumina RNASeq data, what worked best for me (i.e. produced the most normal-looking data) was the following steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Removing genes that had low raw counts in all samples&lt;/li&gt;&#xA;&lt;li&gt;Using DESeq's variance-stabilized transform (which transforms counts into a log-like distribution)&lt;/li&gt;&#xA;&lt;li&gt;Further normalising the VST values by dividing by the longest transcript length within each gene (which I call VSTPk)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;These steps are stated in a bit more detail in our Th2 paper that was published at the end of last year:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://jem.rupress.org/content/early/2016/12/01/jem.20160470#materials-methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://jem.rupress.org/content/early/2016/12/01/jem.20160470#materials-methods&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-08-18T21:13:03.153" LastActivityDate="2017-08-18T21:13:03.153" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2333" PostTypeId="1" AcceptedAnswerId="2334" CreationDate="2017-08-18T12:00:33.640" Score="4" ViewCount="68" Body="&lt;p&gt;I'm pretty new to all of this--forgive me if this is a simple question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I download illumina counts from GEO (like the supplementary file in &#xA;GSE89225). Can I do comparisons directly on that file? Is there some normalization procedure I should go through?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1226" LastEditorUserId="1226" LastEditDate="2017-08-18T12:05:03.827" LastActivityDate="2017-08-18T12:09:02.667" Title="Analyzing Illumina Counts" Tags="|rna-seq|normalization|illumina|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2334" PostTypeId="2" ParentId="2333" CreationDate="2017-08-18T12:09:02.667" Score="5" Body="&lt;p&gt;The counts files for GSE89225 is the output of HTSeq-count as a large matrix. Unless you are developing a differential expression package yourself you should not attempt to directly use this. Rather, you should load it into R and use packages such as DESeq2, edgeR, or limma (those are the most popular ones).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For convenience, in DESeq2 you would want the &lt;code&gt;DESeqDatasetFromMatrix()&lt;/code&gt; function after loading this.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-08-18T12:09:02.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2335" PostTypeId="1" CreationDate="2017-08-19T01:05:10.987" Score="4" ViewCount="421" Body="&lt;p&gt;Is there a tool that can scan fastq files without assembling them for a custom list of user defined snps?&lt;/p&gt;&#xA;" OwnerUserId="167" LastEditorUserId="123" LastEditDate="2017-08-21T11:23:44.720" LastActivityDate="2017-08-21T11:23:44.720" Title="Is there a way to quickly verify the presence of some SNPs in Fastq files?" Tags="|fastq|vcf|snp|" AnswerCount="2" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2336" PostTypeId="2" ParentId="2335" CreationDate="2017-08-19T06:34:42.700" Score="4" Body="&lt;p&gt;Try this pipeline: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Clean raw fastq files (trimmomatic)&lt;/li&gt;&#xA;&lt;li&gt;Align clean fastq files to the reference genome (dna-to-dna aligner of your choice)&lt;/li&gt;&#xA;&lt;li&gt;Convert the alignment sam file to bam (samtools)&lt;/li&gt;&#xA;&lt;li&gt;Call SNPs from bam file (samtools)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Here's a start tutorial on this: &lt;a href=&quot;http://ged.msu.edu/angus/tutorials-2013/snp_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ged.msu.edu/angus/tutorials-2013/snp_tutorial.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2017-08-19T06:34:42.700" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2338" PostTypeId="1" AcceptedAnswerId="2388" CreationDate="2017-08-19T19:58:06.747" Score="2" ViewCount="1514" Body="&lt;p&gt;We all have used freemix score at some stage to check contamination or swaps in our sequencing experiment, but can anyone once for all explain how the score is calculated? &lt;/p&gt;&#xA;" OwnerUserId="1250" LastActivityDate="2017-08-25T21:30:44.297" Title="VerifyBamID freemix score" Tags="|sam|ngs|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2339" PostTypeId="1" CreationDate="2017-08-20T11:15:18.257" Score="5" ViewCount="256" Body="&lt;p&gt;What are the relative advantages / disadvantages of the TM (template modelling) and GDT (global distance test, total score) scores, used in protein structure prediction competitions, such as &lt;a href=&quot;https://www.google.ru/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;ved=0ahUKEwir2PqO1-XVAhVHPhQKHcFGBF0QFggmMAA&amp;amp;url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FCASP&amp;amp;usg=AFQjCNENea3DBFw-bFeW2BRImlsjhtnxDA&quot; rel=&quot;noreferrer&quot;&gt;CASP&lt;/a&gt;? When should I choose one over the other? &lt;/p&gt;&#xA;" OwnerUserId="1249" LastActivityDate="2017-08-20T11:15:18.257" Title="Differences between TM and GDT-TS scores for structure comparison" Tags="|protein-structure|statistics|modelling|" AnswerCount="0" CommentCount="1" ClosedDate="2021-05-06T10:12:43.257" ContentLicense="CC BY-SA 3.0" />
  <row Id="2340" PostTypeId="2" ParentId="2335" CreationDate="2017-08-20T22:16:29.723" Score="1" Body="&lt;p&gt;You could use &lt;a href=&quot;http://www.biorxiv.org/content/early/2017/04/07/118000&quot; rel=&quot;nofollow noreferrer&quot;&gt;ARIBA&lt;/a&gt;, which will map you reads to the regions of your interest (more on that in a second) and then assemble/call variants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZHRVk.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZHRVk.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tool is mostly used to find known variants related to AMR (antimicrobial resistance) from a set of databases, but &lt;a href=&quot;https://github.com/sanger-pathogens/ariba/wiki/Task%3A-prepareref#user-provided-data---fasta-and-metadata&quot; rel=&quot;nofollow noreferrer&quot;&gt;according to the project's wiki&lt;/a&gt; you can build your own database with your region of interest.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Additionally, reference sequences can be either of:&#xA;&#xA;    Presence/absence sequences. ARIBA will look for these sequences in the input reads and report any that it finds, and also any variants between these sequences and the input reads.&#xA;&#xA;    Variants only sequences. These should have known variant details specified in the metadata file (see below). ARIBA reports only when it finds at least one of the given variants in each of these these sequences. If a sample has one of these sequences, but does not have one of the supplied variants, then it is not reported. If you supply a variants only sequence, but no variant, then the sequence will be removed during sanity checks.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="123" LastActivityDate="2017-08-20T22:16:29.723" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2341" PostTypeId="1" AcceptedAnswerId="2364" CreationDate="2017-08-21T09:45:33.583" Score="5" ViewCount="8939" Body="&lt;p&gt;I would like to extract all the CDS from a batch of genomes. I have found a perl script from &lt;a href=&quot;https://www.biostars.org/p/46281/&quot; rel=&quot;noreferrer&quot;&gt;BioStars&lt;/a&gt; but this does not seem to work for me. I would preferably like to have a script/ method which will use the locus tag as a header.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My gff files are from prokka output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ATCC0000    Prodigal:2.6    CDS 243 434 .   +   0   ID=FKKLIMLP_00001;Parent=FKKLIMLP_00001_gene;inference=ab initio prediction:Prodigal:2.6;locus_tag=FKKLIMLP_00001;product=hypothetical protein&#xA;ATCC0000    prokka  gene    243 434 .   +   .   ID=FKKLIMLP_00001_gene;locus_tag=FKKLIMLP_00001&#xA;ATCC0000    Prodigal:2.6    CDS 1727    2131    .   -   0   ID=FKKLIMLP_00002;Parent=FKKLIMLP_00002_gene;inference=ab initio prediction:Prodigal:2.6;locus_tag=FKKLIMLP_00002;product=hypothetical protein&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="982" LastEditorUserId="982" LastEditDate="2017-08-21T09:51:14.530" LastActivityDate="2021-04-21T10:15:07.147" Title="How to extract / convert gff3 CDS sequences to multifasta" Tags="|fasta|software-recommendation|format-conversion|gff3|" AnswerCount="5" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2342" PostTypeId="2" ParentId="2341" CreationDate="2017-08-21T09:57:25.883" Score="3" Body="&lt;p&gt;Using python and &lt;a href=&quot;https://github.com/chapmanb/bcbb/tree/master/gff&quot; rel=&quot;nofollow noreferrer&quot;&gt;this GFF parser&lt;/a&gt; that mimics &lt;a href=&quot;http://biopython.org/wiki/SeqIO&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biopython's SeqIO parsers&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from BCBio import GFF&#xA;&#xA;# Read the gff&#xA;for seq in GFF.parse('my_file.gff'):&#xA;    # only focus on the CDSs&#xA;    for feat in filter(lambda x: x.type == 'CDS',&#xA;                       seq.features):&#xA;        # extract the locus tag&#xA;        locus_tag = feat.qualifiers.get('locus_tag',&#xA;                                        ['unspecified'])[0]&#xA;        # extract the sequence&#xA;        dna_seq = seq = str(feat.extract(seq).seq)&#xA;        # simply print the sequence in fasta format&#xA;        print('&amp;gt;%s\n%s' % (locus_tag, dna_seq))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="123" LastActivityDate="2017-08-21T09:57:25.883" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2343" PostTypeId="1" AcceptedAnswerId="2368" CreationDate="2017-08-21T10:36:56.920" Score="5" ViewCount="906" Body="&lt;p&gt;I have a lot of protein clusters. I want to perform an enrichment analysis of their functional annotations, against reference datasets or list of genes I select.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More precisely: a method yields cluster of proteins. I want to decide, for each cluster (which is a set of proteins identifiers), if it holds meaning regarding the proteins it contains, by comparing the annotations found in the cluster and the annotations found in common or specific datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Initially, I used &lt;a href=&quot;https://david.ncifcrf.gov&quot; rel=&quot;nofollow noreferrer&quot;&gt;DAVID&lt;/a&gt;, which compute the GO annotations from the protein list, then perform the enrichment analysis against common datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, DAVID suffer of the following drawbacks:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;since I didn't find any other way to use it, I'm limited by the web interface and its limitations (number of genes, url size, number of request/day).&lt;/li&gt;&#xA;&lt;li&gt;automation over the web interface is a hacky script.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The team behind seems to offer a way to run DAVID in-house (allowing, hopefully, to relax the limitations), but I can't find any way to download the database from their website.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the alternative ways to get enrichment analysis over proteins in a reliable and automated way ?&lt;/p&gt;&#xA;" OwnerUserId="1355" LastEditorUserId="1355" LastEditDate="2017-08-22T10:30:08.333" LastActivityDate="2017-08-23T10:49:18.397" Title="How to run enrichment analysis of protein functional annotation?" Tags="|go|go-enrichment|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2344" PostTypeId="2" ParentId="2341" CreationDate="2017-08-21T11:23:34.903" Score="4" Body="&lt;p&gt;The &lt;a href=&quot;http://ccb.jhu.edu/software/stringtie/gff.shtml#gffread&quot; rel=&quot;nofollow noreferrer&quot;&gt;gffread utility&lt;/a&gt; in Cufflinks package might be interesting for you. To generate a multi-fasta file with nucleotide sequences from your GFF file, then you can try: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gffread -w output_transcripts.fasta -g reference_genome.fa input_transcripts.gff&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="294" LastActivityDate="2017-08-21T11:23:34.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2345" PostTypeId="2" ParentId="2343" CreationDate="2017-08-21T11:26:13.097" Score="3" Body="&lt;p&gt;I would recommend R instead of sticking to websites.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many tools for enrichment analysis. I like to use &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/goseq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GOseq&lt;/a&gt;, which is initially made for RNAseq data, but can also be used for proteins (I have used it for proteomics). You can use the length of the protein instead of transcript length (to correct for length bias), or you can exclude the length information by just using the hypergeometric method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/goseq/inst/doc/goseq.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt; for code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Subsequently you can use &lt;a href=&quot;https://sourceforge.net/projects/gogadget/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gogadget&lt;/a&gt;, a tool I made for visualization of GOseq results with a heatmap or cytoscape network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, you might wanna take a look at &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/clusterProfiler.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;clusterProfiler&lt;/a&gt;. With this package you can do enrichment analysis for many clusters in parallel.&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2017-08-22T10:43:13.063" LastActivityDate="2017-08-22T10:43:13.063" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2346" PostTypeId="1" CreationDate="2017-08-21T12:05:47.623" Score="7" ViewCount="490" Body="&lt;p&gt;It is a well reported fact that GO analysis of RNAseq results is affected by a number of biases, including length bias and expression level bias. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;bioconductor&lt;/code&gt; package &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/goseq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;goseq&lt;/code&gt;&lt;/a&gt; allows you to correct for these biases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By default it corrects for length bias, but you can also get it to do read count bias. Using read counts to do the correction is attractive because in theory it should account for both sources of bias ($read counts\approx expression \times length$).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm doing an enrichment analysis were I have tried both options (length and read counts) and get very different answers. If I run a binomial regression on expression and length vs probability of being differential, I can see that both are &lt;em&gt;independently&lt;/em&gt; important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; model &amp;lt;- glm(sig ~  expression + log(length), data=retained_genes,  family=binomial(link=&quot;logit&quot;))&#xA;&amp;gt; print(anova(model, test=&quot;Chisq&quot;))&#xA;&#xA;Analysis of Deviance Table&#xA;&#xA;Model: binomial, link: logit&#xA;&#xA;Response: sig&#xA;&#xA;Terms added sequentially (first to last)&#xA;&#xA;                       Df Deviance Resid. Df Resid. Dev  Pr(&amp;gt;Chi)    &#xA;NULL                                    6676     4507.1              &#xA;expression              1  114.998      6675     4392.1 &amp;lt; 2.2e-16 ***&#xA;log(length)             1  102.553      6674     4289.5 &amp;lt; 2.2e-16 ***&#xA;expression:log(length)  1   34.094      6673     4255.4 5.252e-09 ***&#xA;---&#xA;Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So I'm know unsure what to do, should I use the analysis corrected for length or read count. Or perhaps take only terms significant in both? Or only in one?&lt;/p&gt;&#xA;" OwnerUserId="235" LastEditorUserId="235" LastEditDate="2017-08-21T15:51:45.610" LastActivityDate="2017-08-21T15:51:45.610" Title="Correct for gene length or read counts in GO enrichment analysis" Tags="|rna-seq|differential-expression|go|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2348" PostTypeId="2" ParentId="2346" CreationDate="2017-08-21T13:21:22.433" Score="1" Body="&lt;p&gt;I am not sure if it makes sense to use read counts as bias instead of gene length (and I certainly wouldn't expect the same results). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you use total read counts of all your samples (library size)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The correction for gene length is a pure technical one, the longer a gene the more reads will align (and higher read count genes are easier significant, since they are way above the noise threshold). If you use read counts, you also have a biological factor (expression) in there, which (I think) is the stuff you test with statistics (e.g., with edgeR) and thus not the bias you want to correct for. &lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2017-08-21T13:21:22.433" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2349" PostTypeId="1" CreationDate="2017-08-21T14:55:53.037" Score="4" ViewCount="56" Body="&lt;p&gt;I am helping a colleague to interpret protein multiple sequence alignment results. My colleague used a third-party proprietary solution (I know...) to calculate the alignment, and one of the resulting tables is a pairwise distance matrix with identity % in the top triangle and &lt;strong&gt;identity plus similarity&lt;/strong&gt; in the bottom triangle. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am struggling to interpret identity plus similarity, despite the fact I found the metric in several papers. &lt;/p&gt;&#xA;" OwnerUserId="1236" LastActivityDate="2017-08-21T23:06:08.790" Title="Identity plus Similarity Interpretation" Tags="|proteins|sequence-alignment|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2350" PostTypeId="2" ParentId="2346" CreationDate="2017-08-21T15:43:53.230" Score="2" Body="&lt;p&gt;My understanding of the subject is that the bias of the gene length (and other bias) should be taken care when analyzing the expression and &lt;strong&gt;before&lt;/strong&gt; enrichment analysis. The enrichment analysis should be done once the corrections has been performed. Because as the abstract of the &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/gb-2010-11-2-r14&quot; rel=&quot;nofollow noreferrer&quot;&gt;GOseq paper states&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;GO analysis is widely used ... but standard methods give biased results on RNA-seq data due to over-detection of differential expression for long and highly expressed transcripts.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So first, take care of the differential expression bias by length and then use the GO to reduce complexity. How you take care of the bias in the RNA-seq data is another question. But the &lt;a href=&quot;http://bioconductor.org/packages/cqn&quot; rel=&quot;nofollow noreferrer&quot;&gt;cqn&lt;/a&gt; package of Bioconductor can correct the expression by gene length and GC content. However, this correction might hurt the differential tool used (see &lt;a href=&quot;https://support.bioconductor.org/p/95683/#95713&quot; rel=&quot;nofollow noreferrer&quot;&gt;this discussion&lt;/a&gt; in Bioconductor), so it might be better in some cases to use GOSeq. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, to the question itself:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;So I'm know unsure what to do, should I use the analysis corrected for length or read count. Or perhaps take only terms significant in both? Or only in one?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Use whatever correction method that yields better differentially expressed genes (DEG. If you find that the correction for length improves the accuracy of the predictions of DEG better than correcting by length and GC, then use that one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option to obtain accurate GO terms, then you could use other testing procedures which don't rely uniquely in the Fisher test, such as those that take into account the structure of the GO graph. &lt;a href=&quot;http://bioconductor.org/packages/topGO&quot; rel=&quot;nofollow noreferrer&quot;&gt;TopGO&lt;/a&gt; use this approach (note that it is a bit difficult to work with this package),this will reduce the role of the gene length bias (and probably other bias) in the resulting significant GO.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2017-08-21T15:43:53.230" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2351" PostTypeId="2" ParentId="2349" CreationDate="2017-08-21T23:06:08.790" Score="3" Body="&lt;p&gt;I guess you already know what identity means. But for completeness, it is the % of pairwise alignment position which have an identical amino acid residue. For the identity+similarity it is important to know that amino acids can be grouped according to their physicochemical properties e.g. charge, size etc. While there are standard groupings for amino acids it may be that the proprietary algorithm uses different groupings. Either way, % identity+similarity would imply the number of pairwise alignment positions which contain either the same amino acid or amino acids from the same group (however they define groups).&lt;/p&gt;&#xA;" OwnerUserId="400" LastActivityDate="2017-08-21T23:06:08.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2352" PostTypeId="1" AcceptedAnswerId="2354" CreationDate="2017-08-22T06:29:21.313" Score="5" ViewCount="1784" Body="&lt;p&gt;I am using plink to check the association between a phenotype and the SNPs of a gene. plink says the phenotype is significantly associated with a SNP on that gene, and when I check the SNP alleles, for 6 out of 49 samples, the nucleotides are 'GA', and for the rest, they are 'AG', so only the order is different. I am not a biologist, but my basic genetics knowledge says that those two are semantically the same. However, obviously, plink does not agree with me and it associates this SNP with a small p-value since 'GA' samples have a higher value for the phenotype than 'AG' samples. I am very new to SNP analysis, any help is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="1331" LastEditorUserId="131" LastEditDate="2017-11-09T08:12:59.493" LastActivityDate="2017-11-09T08:12:59.493" Title="Is there any difference between SNPs 'AG' and 'GA' in association analyses?" Tags="|snp|gwas|plink|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2353" PostTypeId="2" ParentId="2352" CreationDate="2017-08-22T06:40:57.690" Score="4" Body="&lt;p&gt;Is &quot;user5054&quot; the same as &quot;user5504&quot;? No? Exactly. Not only does order matter, it's incredibly vitally important. &lt;code&gt;AG&lt;/code&gt; and &lt;code&gt;GA&lt;/code&gt; are completely and totally different from each other. If this is in a coding region, then the resulting amino acid is undoubtedly changed (fun fact, the &lt;a href=&quot;https://en.wikipedia.org/wiki/DNA_codon_table&quot; rel=&quot;nofollow noreferrer&quot;&gt;only exception&lt;/a&gt; is &lt;code&gt;TAG&lt;/code&gt; and &lt;code&gt;TGA&lt;/code&gt;). If it's at a &lt;a href=&quot;https://en.wikipedia.org/wiki/RNA_splicing&quot; rel=&quot;nofollow noreferrer&quot;&gt;splice site&lt;/a&gt; then splicing is likely altered (AG is part of a splice acceptor site). If it's at the binding site of something then it wouldn't be surprised if it didn't bind any more.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-08-22T06:40:57.690" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2354" PostTypeId="2" ParentId="2352" CreationDate="2017-08-22T07:53:00.347" Score="8" Body="&lt;p&gt;Could you please show us the context in which this appears, as you seem to be interpreting this differently to Devon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it's appearing as you say, GA and AG, then Devon is right, this usually means that the sequence goes ###AG### or ###GA###, which are two very different sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If, however, as you're implying, the sequences are actually ###G### or ###A###, then semantically, the two ways of writing it have the same outcome. Normally, however, this is written as A/G or G/A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you're looking at genotypes, which would usually appear as A|G and G|A. In this case, both individuals are heterozygotes, and in terms of the individual SNP they are the same. However the side of the pipe is relevant as this tells you which homologous chromosome it is on. This allows you to calculate LD with respect to neighbouring SNPs, and to determine the actual sequences of the proteins in that individual.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2017-08-22T07:53:00.347" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2355" PostTypeId="2" ParentId="2352" CreationDate="2017-08-22T09:19:27.233" Score="3" Body="&lt;p&gt;Most association analyses are carried out at a single SNP level, so AG and GA are likely to indicate a heterozygous genotype at a particular location. However, the precise notation matters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As @Emily_Ensembl has alluded to, for VCF files, A/G indicates unphased SNPs (order unknown, and shouldn't be considered in analyses), whereas A|G indicates phased SNPs (order matters).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible to carry out association analyses by combining multiple variants into a single haplotype, in which case the order would also matter. It's unlikely that this is the case, because such haplotype variant analyses are substantially more complicated to carry out, and will typically be paired with information about SNP linkage (statistic D´ or r²).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-08-22T09:19:27.233" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2356" PostTypeId="1" CreationDate="2017-08-22T10:11:37.243" Score="2" ViewCount="3588" Body="&lt;p&gt;I have multiple libraries of &lt;strong&gt;10x Chromium single-cell RNA-seq&lt;/strong&gt; data, which I'd like to combine. One option is using &lt;code&gt;cellranger aggr&lt;/code&gt; which by default does a depth normalization:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;mapped&lt;/code&gt;: (default) Subsample reads from higher-depth libraries until they all have an equal number of confidently mapped reads per cell. &lt;code&gt;raw&lt;/code&gt;: Subsample reads from higher-depth libraries until they all have an equal number of total (i.e. raw, mapping-independent) reads per cell.&#xA;&lt;code&gt;none&lt;/code&gt;: Do not normalize at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/aggregate&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/aggregate&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The new version of &lt;code&gt;Seurat&lt;/code&gt; (2.0) provides another way via the &lt;code&gt;MergeSeurat()&lt;/code&gt; (or &lt;code&gt;AddSamples()&lt;/code&gt;) functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://satijalab.org/seurat/merge_vignette.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://satijalab.org/seurat/merge_vignette.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These have the arguments:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;do.logNormalize whether to normalize the expression data per cell and transform to log space.&#xA;total.expr      scale factor in the log normalization &#xA;do.scale        In object@scale.data, perform row-scaling (gene-based z-score) &#xA;do.center       In object@scale.data, perform row-centering (gene-based centering)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm wondering how do these methods compare? What are the pros/cons etc? What do we need to consider when we combine samples?&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2017-08-24T00:15:42.657" Title="Cellranger aggr and Seurat merge difference?" Tags="|rna-seq|single-cell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2358" PostTypeId="2" ParentId="2234" CreationDate="2017-08-22T14:49:00.440" Score="3" Body="&lt;p&gt;I finally found another answer to my question. Please read this great article in May 12 2017 BioMed Central (BMC) Bioinformatics article titled &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-017-1674-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ranking metrics in gene set enrichment analysis: do they matter?&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, please read this blog , &lt;a href=&quot;http://crazyhottommy.blogspot.com/2016/08/gene-set-enrichment-analysis-gsea.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Diving into Genetics and Genomics: Gene Set Enrichment Analysis (GSEA) explained&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After reading these two articles, my choice for the best rapidGSEA local ranking measure is Minimum Significant Difference (i.e., MSD), because it has the best overall false positive rate (i.e., FPR) for larger sample sizes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, it is important to realize that fgseaL's phenotype labeling can hypothetically be emulated by GSEA with either one of it's sixteen possible ranking metrics or a custom ranking metric.&lt;/p&gt;&#xA;" OwnerUserId="1283" LastEditorUserId="1283" LastEditDate="2017-08-23T16:16:10.837" LastActivityDate="2017-08-23T16:16:10.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2359" PostTypeId="1" AcceptedAnswerId="2362" CreationDate="2017-08-22T16:39:42.897" Score="8" ViewCount="104" Body="&lt;p&gt;I would like to modify some reference transcripts from Ensembl (&lt;em&gt;D. melanogaster&lt;/em&gt;) to introduce a controlled rate of random errors in the sequences. The idea would be to introduce random base substitutions in these sequences, no indels for now, because I would like to keep the transcript sequence length as it is in the reference. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rate of error per transcript will be determined according to an error profile computed from an external set of RNA-seq reads (e.g., generated with ONT MinION)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The aim of this modification would be to establish a rough benchmark of the performances of aligners to use over transcripts from spliced reads (rna-to-genome), aka with more than one exon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea of which software would be best for this purpose?&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2017-08-25T12:23:33.937" Title="Introduce errors in reference transcripts according to external dataset error model" Tags="|rna-seq|software-recommendation|nanopore|simulated-data|" AnswerCount="4" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2360" PostTypeId="2" ParentId="2359" CreationDate="2017-08-22T17:39:19.703" Score="1" Body="&lt;p&gt;It sounds like what you're really looking for is a read simulator. A cursory search turns up &lt;a href=&quot;http://www.biorxiv.org/content/early/2016/03/18/044545&quot; rel=&quot;nofollow noreferrer&quot;&gt;NanoSim&lt;/a&gt;, which is designed to simulate reads from a MinION. This has the benefit of at least having been &lt;a href=&quot;https://gigascience.biomedcentral.com/articles/10.1186/s13742-016-0140-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;used in some of the published literature&lt;/a&gt;, which is always a nice sign.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also find &lt;a href=&quot;http://www.nature.com/nrg/journal/v17/n8/full/nrg.2016.57.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this review article&lt;/a&gt; on read simulators useful. It doesn't specifically mention NanoSim, but it should prove to be a useful review of the general concepts anyway if you need to read up on them.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-08-22T17:39:19.703" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2361" PostTypeId="1" CreationDate="2017-08-22T19:20:21.920" Score="5" ViewCount="321" Body="&lt;p&gt;I have data, obtained from a single metagenomic DNA sample, that consists of two MiSeq FASTQ files (R1 and R2) that I merged using &lt;a href=&quot;https://academic.oup.com/bioinformatics/article-lookup/doi/10.1093/bioinformatics/btt593&quot; rel=&quot;noreferrer&quot;&gt;PEAR&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want to estimate the abundances of the bacteria taxa to generate a figure like this one:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FlJxD.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FlJxD.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Figure from: &lt;em&gt;Panosyan, Hovik, and Nils‐Kåre Birkeland. &quot;Microbial diversity in an Armenian geothermal spring assessed by molecular and culture‐based methods.&quot; Journal of basic microbiology 54.11 (2014): 1240-1250.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that there wasn't a step of amplification of the 16S region as the goal of the sequencing was to discover new genes. I've already isolated 16S reads from my sample using &lt;a href=&quot;http://bioinfo.lifl.fr/RNA/sortmerna/&quot; rel=&quot;noreferrer&quot;&gt;SortMeRNA&lt;/a&gt;, but it seems like softwares that do OTU picking, taxonomic assignment and diversity analyses (such as mothur and QIIME) require that all the reads come from the same region of the 16S gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way of using these 16S reads that I've filtered using SortMeRNA in a diversity analysis using mothur/QIIME?&lt;/p&gt;&#xA;" OwnerUserId="1334" LastEditorUserId="1334" LastEditDate="2017-08-22T20:23:18.003" LastActivityDate="2022-03-16T22:11:15.387" Title="Microbial diversity analysis using whole-genome metagenomic data" Tags="|metagenome|taxonomy|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2362" PostTypeId="2" ParentId="2359" CreationDate="2017-08-22T19:43:37.373" Score="3" Body="&lt;p&gt;Do any of the answers for &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/202/tools-for-simulating-oxford-nanopore-reads/222&quot;&gt;this question&lt;/a&gt; help? Karel Brinda has mentioned a few read simulators in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/273/73&quot;&gt;the answer to that question&lt;/a&gt;, and has &lt;a href=&quot;http://brinda.cz/publications/brinda_phd.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;a thesis&lt;/a&gt; with more information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Excluding INDEL errors doesn't sound like a good idea; length can still be preserved even if doing that, it just needs an adjustment at the end of the sequence. Note that if you're trying to model nanopore reads, what you're really modelling is the base-caller, rather than the sequencer. I mention this in more detail in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/226/73&quot;&gt;my answer&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In most cases where errors are modelled, I find it better to use publicly-available data instead. Especially for nanopore data, there are unmodelled systematic errors in the base-callers and sequencer that can't be simulated using any programs (because they are unmodelled). The following paper would be a good place to start for cDNA sequences, which looks at single-cell data from mouse (C57Bl/6) B1a cells:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.biorxiv.org/content/early/2017/04/13/126847&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.biorxiv.org/content/early/2017/04/13/126847&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Illumina and ONT reads for that study can be found in SRA under accession number &lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/?study=SRP082530&quot; rel=&quot;nofollow noreferrer&quot;&gt;SRP082530&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know of any recent &lt;em&gt;D. melanogaster&lt;/em&gt; studies that have been done using nanopore. There's always the option of spending $1000 on a &lt;a href=&quot;https://store.nanoporetech.com/minion/rna/sets&quot; rel=&quot;nofollow noreferrer&quot;&gt;purchase of a MinION&lt;/a&gt; with an RNA starter kit to do the study yourself. Here's an older targeted gene study, but bear in mind that it was using an R7.3 flow cell, so errors rates will be much higher than what is currently available:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-015-0777-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-015-0777-z&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-08-22T19:52:49.440" LastActivityDate="2017-08-22T19:52:49.440" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2363" PostTypeId="1" AcceptedAnswerId="2365" CreationDate="2017-08-22T20:51:08.273" Score="7" ViewCount="775" Body="&lt;p&gt;I am constructing a bit of software which pipes the outputs of the bam file via &lt;code&gt;samtools view&lt;/code&gt; into a script for parsing. My goal is to (somehow) make this process more efficient, and faster than &lt;code&gt;samtools view&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am only using 3-4 fields in the bam. Naturally, if I only wanted those fields, I could do something like &lt;code&gt;samtools view file1.bam | cut -f ##&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm trying to be faster than this approach. More specifically: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) Given the bam file format, is it theoretically possible to output several requested SAM fields faster than &lt;code&gt;samtools view|cut -f&lt;/code&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) Is there software available to achieve (1), or would one need to hack samtools? &lt;/p&gt;&#xA;" OwnerUserId="146" LastEditorUserId="37" LastEditDate="2017-08-23T12:43:23.493" LastActivityDate="2017-08-25T14:27:47.847" Title="Is there a way to retrieve several SAM fields faster than `samtools view | cut -f`?" Tags="|sam|samtools|" AnswerCount="3" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2364" PostTypeId="2" ParentId="2341" CreationDate="2017-08-23T02:13:50.693" Score="6" Body="&lt;p&gt;I like &lt;code&gt;bedtools getfasta&lt;/code&gt;.  My typical option set is &lt;code&gt;bedtools getfasta -fi &amp;lt;reference&amp;gt; -bed &amp;lt;gff_file&amp;gt; -name -s&lt;/code&gt;.  Be aware of the &lt;code&gt;-s&lt;/code&gt; to make sure you are pulling the correct strand.  I like bedtools because it is a versatile tool overall for handling bed, gff and vcf file manipulations. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# bedtools getfasta&#xA;&#xA;Tool:    bedtools getfasta (aka fastaFromBed)&#xA;Version: v2.26.0&#xA;Summary: Extract DNA sequences from a fasta file based on feature coordinates.&#xA;&#xA;Usage:   bedtools getfasta [OPTIONS] -fi &amp;lt;fasta&amp;gt; -bed &amp;lt;bed/gff/vcf&amp;gt;&#xA;&#xA;Options: &#xA;    -fi Input FASTA file&#xA;    -bed    BED/GFF/VCF file of ranges to extract from -fi&#xA;    -name   Use the name field for the FASTA header&#xA;    -split  given BED12 fmt., extract and concatenate the sequencesfrom the BED &quot;blocks&quot; (e.g., exons)&#xA;    -tab    Write output in TAB delimited format.&#xA;        - Default is FASTA format.&#xA;&#xA;    -s  Force strandedness. If the feature occupies the antisense,&#xA;        strand, the sequence will be reverse complemented.&#xA;        - By default, strand information is ignored.&#xA;&#xA;    -fullHeader Use full fasta header.&#xA;        - By default, only the word before the first space or tab is used.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="964" LastActivityDate="2017-08-23T02:13:50.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2365" PostTypeId="2" ParentId="2363" CreationDate="2017-08-23T03:14:30.797" Score="6" Body="&lt;p&gt;The BAM file format is not a text-based format. It has a specific binary structure, specified in reasonable detail in the &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;noreferrer&quot;&gt;SAM file format specification&lt;/a&gt;. Whenever this information is displayed on a screen as text, it needs to be converted from the binary format to a text format, which takes a bit of time and processing power.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As this question suggests, if only a specific field from alignments is needed (or fields), then it will probably be better to extract just those fields and do any necessary conversion only on those fields. While this &lt;em&gt;can&lt;/em&gt; be done by writing a BAM parser from scratch, many developers have already written software libraries to process BAM files in this way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Devon Ryan has suggested &lt;a href=&quot;http://www.htslib.org/&quot; rel=&quot;noreferrer&quot;&gt;htslib&lt;/a&gt;, which is a C library written by a group that includes the people who wrote the SAM/BAM/CRAM file format specifications. There's also &lt;a href=&quot;https://github.com/pysam-developers/pysam&quot; rel=&quot;noreferrer&quot;&gt;pysam&lt;/a&gt;, which is a python wrapper around htslib.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The particular tool that is used will depend on your familiarity with programming and the specific thing that you want to do. If you just want to &quot;output a BAM file&quot;, then cat is the quickest:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat file1.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;... but you probably don't want to do that, because it seems like you want to process a text-based representation with a script. Because you haven't specified which fields you are interested in, it's not possible to suggest the best thing to use. In the end, I expect that an &quot;efficient&quot; solution to your problem would involve &lt;code&gt;htslib&lt;/code&gt; in some form. This is not really hacking &lt;em&gt;samtools&lt;/em&gt;, it's using the backend of samtools to process BAM data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it's useful to also think about the time cost of coding. What application do you have which means that the text-based processing method is not sufficient? It takes time to write code, and a lot more time to debug that code to make sure it's doing the right thing in all situations. If this is for a one-off thing, then &lt;code&gt;samtools view&lt;/code&gt; output fed into your script may be the quickest solution.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-08-23T03:14:30.797" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2366" PostTypeId="2" ParentId="2341" CreationDate="2017-08-23T04:12:35.133" Score="2" Body="&lt;p&gt;The &lt;code&gt;xtractore&lt;/code&gt; program from the &lt;a href=&quot;http://brendelgroup.github.io/AEGeAn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AEGeAn Toolkit&lt;/a&gt; was designed for this type of use case. Just set &lt;code&gt;--type=CDS&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ xtractore -h&#xA;&#xA;xtractore: extract sequences corresponding to annotated features from the&#xA;           given sequence file&#xA;&#xA;Usage: xtractore [options] features.gff3 sequences.fasta&#xA;  Options:&#xA;    -d|--debug            print debugging output&#xA;    -h|--help             print this help message and exit&#xA;    -i|--idfile: FILE     file containing a list of feature IDs (1 per line&#xA;                          with no spaces); if provided, only features with&#xA;                          IDs in this file will be extracted&#xA;    -o|--outfile: FILE    file to which output sequences will be written;&#xA;                          default is terminal (stdout)&#xA;    -t|--type: STRING     feature type to extract; can be used multiple&#xA;                          times to extract features of multiple types&#xA;    -v|--version          print version number and exit&#xA;    -V|--verbose          print verbose warning and error messages&#xA;    -w|--width: INT       width of each line of sequence in the Fasta&#xA;                          output; default is 80; set to 0 for no&#xA;                          formatting&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It looks like you're processing prokaryotic genomes, but this program works also on eukaryotic genomes where CDSs often have introns interspersed.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2017-08-23T04:12:35.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2367" PostTypeId="1" CreationDate="2017-08-23T06:18:51.097" Score="4" ViewCount="485" Body="&lt;p&gt;I want to use Canu to correct my nanopore long read (version: MinION R9.5), but I am not quite sure how to set the correctErrorRate. Should I follow the Canu manual (Nanopore R7 2D and Nanopore R9 1D Increase the maximum allowed difference in overlaps from the default of 14.4% to 22.5% with correctedErrorRate=0.225), or you guys have a better option? &lt;/p&gt;&#xA;" OwnerUserId="1364" LastEditorUserId="96" LastEditDate="2017-08-24T15:42:12.300" LastActivityDate="2017-09-08T08:07:38.117" Title="Error rate setting in Canu error correction" Tags="|nanopore|canu|quality-control|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2368" PostTypeId="2" ParentId="2343" CreationDate="2017-08-23T10:49:18.397" Score="3" Body="&lt;p&gt;DAVID depend on a couple of databases from the Gene Ontology Consortium, Reactome, KEGG,... most of them are accessible via Bioconductor. To perform an enrichment analysis you can have a look at the tutorial of the several &lt;a href=&quot;http://bioconductor.org/packages/release/BiocViews.html#___GeneSetEnrichment&quot; rel=&quot;nofollow noreferrer&quot;&gt;packages in Bioconductor&lt;/a&gt; that do this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the most important for analyzing enrichment in GO terms are &lt;a href=&quot;http://bioconductor.org/packages/topGO&quot; rel=&quot;nofollow noreferrer&quot;&gt;topGO&lt;/a&gt;, &lt;a href=&quot;http://bioconductor.org/packages/goseq&quot; rel=&quot;nofollow noreferrer&quot;&gt;goseq&lt;/a&gt;, &lt;a href=&quot;http://bioconductor.org/packages/GOstats&quot; rel=&quot;nofollow noreferrer&quot;&gt;GOstats&lt;/a&gt;. I would also recommend &lt;a href=&quot;http://bioconductor.org/packages/GOSemSim&quot; rel=&quot;nofollow noreferrer&quot;&gt;GOSemSim&lt;/a&gt; if you want to compare between GO  to focus on a specific GO terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other important packages are the &lt;a href=&quot;http://bioconductor.org/packages/fgsea&quot; rel=&quot;nofollow noreferrer&quot;&gt;fgsea&lt;/a&gt; to test any kind of gene set (which is similar to the one hosted by the Broad Institute), &lt;a href=&quot;http://bioconductor.org/packages/gsva&quot; rel=&quot;nofollow noreferrer&quot;&gt;gsva&lt;/a&gt; for enrichment analysis by sample, &lt;a href=&quot;http://bioconductor.org/packages/limma&quot; rel=&quot;nofollow noreferrer&quot;&gt;limma&lt;/a&gt; has some functions for functional enrichment too. &lt;a href=&quot;http://bioconductor.org/packages/piano&quot; rel=&quot;nofollow noreferrer&quot;&gt;Piano&lt;/a&gt;, &lt;a href=&quot;http://bioconductor.org/packages/GSCA&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSCA&lt;/a&gt;, &lt;a href=&quot;http://bioconductor.org/packages/SPIA&quot; rel=&quot;nofollow noreferrer&quot;&gt;SPIA&lt;/a&gt; are also worth mentioning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bioconductor has &quot;standard&quot; data sets of the expressions of some cells, like &lt;a href=&quot;http://bioconductor.org/packages/airway&quot; rel=&quot;nofollow noreferrer&quot;&gt;airway&lt;/a&gt; and &lt;a href=&quot;http://bioconductor.org/packages/ALL&quot; rel=&quot;nofollow noreferrer&quot;&gt;ALL&lt;/a&gt; frequently used in vignettes. They are not reference data sets because there isn't a reference expression for a cell of an organisms. It depends on the type of cell, the experiment, the conditions...&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2017-08-23T10:49:18.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2370" PostTypeId="2" ParentId="2367" CreationDate="2017-08-24T00:01:39.523" Score="3" Body="&lt;p&gt;It's generally a good idea to trust the &quot;official&quot; suggestions. You can also adjust the error rate based on coverage according to &lt;a href=&quot;http://canu.readthedocs.io/en/latest/parameter-reference.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;parameter reference&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For low coverage datasets (less than 30X), we recommend increasing&#xA;  correctedErrorRate slightly, by 1% or so.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;For high-coverage datasets (more than 60X), we recommend decreasing&#xA;  correctedErrorRate slighly, by 1% or so.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Raising the correctedErrorRate will increase run time. Likewise,&#xA;  decreasing correctedErrorRate will decrease run time, at the risk of&#xA;  missing overlaps and fracturing the assembly.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That being said, the sequencing quality can vary from between libraries or flow cells. The error rate is not a constant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the developer (&lt;a href=&quot;https://github.com/marbl/canu/issues/109&quot; rel=&quot;nofollow noreferrer&quot;&gt;full thread&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;if you want to assemble 1D data or have a bad run, it is possible you&#xA;  would need to increase the error rate. You could run with a high error&#xA;  rate (0.1) and look at the distribution of overlap error rates in the&#xA;  unitig step to look for a peak in the distribution. If you have a near&#xA;  neighbor you could also map the corrected reads to it and estimate the&#xA;  residual error that way.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="35" LastActivityDate="2017-08-24T00:01:39.523" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2371" PostTypeId="2" ParentId="2356" CreationDate="2017-08-24T00:15:42.657" Score="3" Body="&lt;p&gt;Cell Ranger aggregate subsamples reads (unless you select &lt;code&gt;none&lt;/code&gt;), so you will end up with less total reads in samples that have more initially. The output is still raw counts, but you will have more or less per cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seurat just merges the raw counts matrices and normalizes those.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2017-08-24T00:15:42.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2372" PostTypeId="2" ParentId="2359" CreationDate="2017-08-24T13:36:56.130" Score="1" Body="&lt;p&gt;The executable &lt;code&gt;fastq-sim&lt;/code&gt; in &lt;a href=&quot;http://cbrc3.cbrc.jp/~martin/dnemulator/README.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DNemulator&lt;/a&gt; package is able to modify a set of input sequences in &lt;code&gt;fasta&lt;/code&gt; format according to an external set of quality scores reported in a &lt;code&gt;fastq&lt;/code&gt; file. &lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2017-08-24T13:36:56.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2373" PostTypeId="1" AcceptedAnswerId="2374" CreationDate="2017-08-24T13:42:57.060" Score="7" ViewCount="790" Body="&lt;p&gt;I'm looking for a way to identify low complexity regions and other repeats in the genome of &lt;em&gt;Escherichia coli&lt;/em&gt;. I found that RepeatMasker may be used for example when drafting genomes of prokaryotes (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3587949/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;E. coli&lt;/em&gt; example&lt;/a&gt;). But RepeatMasker works on a limited dataset of species, neither of them being prokaryotes. By default, when running RepeatMasker, if no species is specified, it will compare with homo sapiens data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems rather inadequate, but the most relevent alternative, &lt;a href=&quot;https://sites.google.com/site/prapsoftware/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PRAP&lt;/a&gt;, requires a &quot;dead&quot; tool (VisCoSe, by Michael Spitzer).&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is it still wise to to use RepeatMasker on &lt;em&gt;Escherichia coli&lt;/em&gt;?&lt;/li&gt;&#xA;&lt;li&gt;If yes, which settings would maximise relevance ?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="522" LastEditorUserId="73" LastEditDate="2017-08-24T23:29:23.597" LastActivityDate="2020-09-07T15:40:28.573" Title="Is it wise to use RepeatMasker on prokaryotes?" Tags="|genome|repeat-elements|sequence-analysis|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2374" PostTypeId="2" ParentId="2373" CreationDate="2017-08-24T13:53:40.627" Score="5" Body="&lt;p&gt;If I understood correctly your question, you want to mask those regions in a (FASTA?) genome. I think you could identify those regions using &lt;a href=&quot;http://mummer.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;mummer&lt;/a&gt; and mask them using &lt;a href=&quot;http://bedtools.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# align genome against itself&#xA;nucmer --maxmatch --nosimplify genome.fasta genome.fasta&#xA;&#xA;# select repeats and convert the corrdinates to bed format&#xA;show-coords -r -T -H out.delta | awk '{if (&lt;span class=&quot;math-container&quot;&gt;$1 != $&lt;/span&gt;3 &amp;amp;&amp;amp; &lt;span class=&quot;math-container&quot;&gt;$2 != $&lt;/span&gt;4) print &lt;span class=&quot;math-container&quot;&gt;$0}' | awk '{print $8&quot;\t&quot;$1&quot;\t&quot;$&lt;/span&gt;2}' &amp;gt; repeats.bed&#xA;&#xA;# mask those bases with bedtools&#xA;bedtools maskfasta -fi genome.fasta -bed repeats.bed -fo masked.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Have a look at &lt;a href=&quot;http://mummer.sourceforge.net/manual/#identifyingrepeats&quot; rel=&quot;nofollow noreferrer&quot;&gt;nucmer&lt;/a&gt; and &lt;a href=&quot;http://bedtools.readthedocs.io/en/latest/content/tools/maskfasta.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools maskfasta&lt;/a&gt; options to fine-tune your analysis.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="9314" LastEditDate="2020-09-07T15:40:28.573" LastActivityDate="2020-09-07T15:40:28.573" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="2375" PostTypeId="1" AcceptedAnswerId="2376" CreationDate="2017-08-24T16:38:17.097" Score="6" ViewCount="846" Body="&lt;p&gt;I'm trying to run a differential gene expression analysis using DESeq2, with counts coming from kallisto. I have imported them using tximport and I'm creating the &lt;code&gt;DESeqDataSet&lt;/code&gt; (dds) using the &lt;code&gt;DESeqDataSetFromMatrix&lt;/code&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dds &amp;lt;- DESeqDataSetFromMatrix(counts,&#xA;                                samples,&#xA;                                ~ strain + batch)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I get the following error, expected given my experimental design:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in checkFullRank(modelMatrix) :&#xA;  the model matrix is not full rank, so the model cannot be fit as specified.&#xA;  One or more variables or interaction terms in the design formula are linear&#xA;  combinations of the others and must be removed.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, I know that I can just remove one column from the design matrix to make it work, but is there a way to supply my own design matrix to DESeq2? The following code raises an error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; design &amp;lt;- model.matrix(~strain+batch, s2c)&#xA;&amp;gt; design = design[, -9] # removing a column to avoid full-rank&#xA;&amp;gt; dds &amp;lt;- DESeqDataSetFromMatrix(counts, s2c, design=design)&#xA;converting counts to integer mode&#xA;Error: $ operator is invalid for atomic vectors&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a way to provide my own &lt;code&gt;model.matrix&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p.s. the modified model works in sleuth, but I would like to use DESeq2 for this particular analysis.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2018-10-22T23:25:35.040" Title="Is it possible to create a DESeqDataSet with a user-provided design matrix?" Tags="|rna-seq|deseq2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2376" PostTypeId="2" ParentId="2375" CreationDate="2017-08-24T17:32:11.120" Score="4" Body="&lt;p&gt;Provide rank sufficient design to &lt;code&gt;DESeqDataSetFromMatrix&lt;/code&gt; and then use your custom model matrix in &lt;code&gt;DESeq&lt;/code&gt;. In essence:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dds = DESeqDataSetFromMatric(counts, s2c, design=~batch)&#xA;design &amp;lt;- model.matrix(~strain+batch, s2c)&#xA;design = design[, -9]&#xA;DESeq(dds, full=design)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://support.bioconductor.org/p/64480/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this thread&lt;/a&gt; on the bioconductor site for details.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-08-24T17:32:11.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2377" PostTypeId="1" AcceptedAnswerId="2379" CreationDate="2017-08-24T21:07:58.983" Score="6" ViewCount="1128" Body="&lt;p&gt;I'm having a difficulty in grasping the general purpose and concept of indel calling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What exactly is this process? &lt;/p&gt;&#xA;" OwnerUserId="711" LastEditorUserId="96" LastEditDate="2017-08-31T03:02:24.887" LastActivityDate="2017-08-31T03:02:24.887" Title="What is indel calling and what is its purpose?" Tags="|genome|variant-calling|indel|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2378" PostTypeId="1" AcceptedAnswerId="2381" CreationDate="2017-08-24T21:26:20.367" Score="7" ViewCount="2416" Body="&lt;p&gt;Given an experiment consisting of an input (baseline RNA) and IP (pulldown to find RNAs bound to certain protein of interest)... Is a DE analysis performed over the RNA-seq data from the samples (lets say with EdgeR or DESEQ2) suitable to reveal the preferentially bound RNAs? What other software tools would you recommend?&lt;/p&gt;&#xA;" OwnerUserId="1375" LastActivityDate="2020-01-22T13:26:27.567" Title="RIP-seq analysis?" Tags="|rna-seq|software-recommendation|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2379" PostTypeId="2" ParentId="2377" CreationDate="2017-08-24T22:44:21.977" Score="9" Body="&lt;p&gt;Insertions and deletions (indels) are one type among many different types of &lt;em&gt;genetic variation&lt;/em&gt;, such as single nucleotide variants (SNVs), copy number variants (CNVs), and structural variants (SVs). I'll assume here that you know how indels are defined, but are simple trying to understand the importance of discovering and analyzing them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The goal of indel calling, like the goal of any variant calling, is to identify genetic variants that can subsequently be associated with important phenotypes, esp. disease. For example, if 60% of patients with disease XYZ have an indel in the promoter region of gene 123, then that is information of extreme interest and value in research and in clinical care.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Genome-wide association studies (GWAS) have been trying to correlate SNVs to disease and other phenotypes for years. Much less work has been done with indels, but their discovery and analysis remains an area of intense interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as the &lt;em&gt;process&lt;/em&gt; of indel calling, large indels can usually be found  by mapping paired reads and looking for large discrepancies in the expected distance between pairs and the observed distance.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Huh, my average insert size is 400bp, but the aligned read pairs flanking this area are 1200bp apart. Must be an 800bp deletion in there!&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Smaller indels are much more difficult to detect using this strategy, since they are harder to distinguish from noise (i.e. the variation in length of sequenced fragments). However, as another answer mentions, many short indels are simply reflected as short gaps in the alignment.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2017-08-31T03:01:23.080" LastActivityDate="2017-08-31T03:01:23.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2380" PostTypeId="2" ParentId="2378" CreationDate="2017-08-25T06:54:34.863" Score="5" Body="&lt;p&gt;You can try doing standard differential expression, but I worry that the between-sample normalization will work poorly. Personally, I would do peak calling instead, followed by diffBind. You have a few tools to choose from when it comes to this. In the past, I've rolled my own methods for this using MACS2 and genomic alignments (I then converted those to bedGraph files where entries are transcripts, so peak calling is then in transcript-coordinates).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's probably easier, though, to use premade software. There are a number of packages out there for RIP-seq. These include &lt;a href=&quot;https://academic.oup.com/bioinformatics/article-lookup/doi/10.1093/bioinformatics/bts569&quot; rel=&quot;noreferrer&quot;&gt;piranha&lt;/a&gt;, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3632129/&quot; rel=&quot;noreferrer&quot;&gt;RIPSeeker&lt;/a&gt; and &lt;a href=&quot;https://academic.oup.com/bioinformatics/article-lookup/doi/10.1093/bioinformatics/btt428&quot; rel=&quot;noreferrer&quot;&gt;ASPeak&lt;/a&gt;. Particularly if you're just starting out, you'd be well served with using one of those tools.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-08-25T06:54:34.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2381" PostTypeId="2" ParentId="2378" CreationDate="2017-08-25T11:15:54.753" Score="4" Body="&lt;p&gt;You need to be careful of terminology. To me, a RIP-seq experiment involves a pull down, followed by a RNAseq library prep. Thus, the whole transcript is captured, not just the binding site of the protein (as in CLIP-Seq, HITS-CLIP, PAR-CLIP, iCLIP or eCLIP). Thus &quot;peak-callers&quot; whether they be designed for calling protein-DNA or protein-RNA binding sites are not suitable as the signal will not be punctuate or peaky. Of the methods mentioned by @Devon Ryan, only &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3632129/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RIPSeeker&lt;/a&gt; seems setup to deal with this sort of data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some of the papers that talk about calling RNA-protein interactions they lump RIP-seq in with the CLIP techniques (see piranha and ASPeak papers referenced by @Devon Ryan). They appear to be talking about a technique where the RNA is fragmented before it is pulled down. Thus you would capture peaky binding sites (actually, the protocol would be remarkably similar to CLIP). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should work out which of these you have. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for using DESeq/EdgeR etc, in the RIPSeeker paper they say:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Furthermore, programs for de novo transcript assembly followed by&#xA;  differential expression (DE) analysis, such as the Cufflinks/Cuffdiff&#xA;  suite (15,16), and for DE on a set of known transcripts, such as DESeq&#xA;  (17), may appear applicable to RIP-seq analysis. Unlike peak-calling&#xA;  strategy, however, the transcript-based methods assume the full&#xA;  transcriptome being sequenced at a fairly deep coverage (as usually&#xA;  the case in RNA-seq) and thus may be sensitive to background noise&#xA;  typical to the IP-based protocols, which is due to both the&#xA;  non-specific RNA interactions with a protein of interest and the&#xA;  non-specific RNA input from the pull-down of the (mutant) control&#xA;  (Supplementary Figures S3 and S4).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't know to what extent RIPSeeker performs better than the naive DESeq approach as its not included as one of the comparators in the paper. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2017-08-25T11:15:54.753" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2382" PostTypeId="2" ParentId="2377" CreationDate="2017-08-25T11:31:05.573" Score="10" Body="&lt;p&gt;There are two types of INDELs: short indels and long indels. Some put the threshold at 50bp; others choose 1000bp. Short and long indels are called differently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&amp;lt;50bp short indels are called from read alignment directly. Modern indel callers essentially build a multi-alignment of reads and call an indel with enough read supports. Short indels may break open reading frames.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Illumina reads, long indels are usually called with split alignment or read pair alignment because mainstream aligners are unable to directly align through a long indel. Long indels are a type of structural variation. They may affect gene structures in a more dramatic way (e.g. delete a whole exon; screw up transcription by transposon insertion; create pseudogenes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, both short and long indels are types of genetic variants. Calling them helps to understand how genetics shapes phenotypes.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-08-25T11:31:05.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2383" PostTypeId="2" ParentId="2359" CreationDate="2017-08-25T11:42:24.700" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://www.biorxiv.org/content/early/2017/04/11/126656&quot; rel=&quot;nofollow noreferrer&quot;&gt;This preprint&lt;/a&gt; uses pbsim to simulate ONT RNA-seq reads for fruit fly. It is probably worth reading if you want to do the same thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should include INDEL errors. Those are what make RNA-seq alignment challenging. For the benchmark purpose, adding INDELs does not increase the complexity at all. You can parse splice junctions on the reference from CIGAR and compare them to the annotation. You don't need to worry about the base-level alignment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, there are &lt;a href=&quot;https://www.nature.com/articles/ncomms16027&quot; rel=&quot;nofollow noreferrer&quot;&gt;public real ONT data&lt;/a&gt; (AC:SRP082530) for &lt;a href=&quot;https://www.lexogen.com/sirvs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SIRV spike-in control&lt;/a&gt; and mouse B cells. You don't actually need simulation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: just noticed that you are an author of the first preprint I cited. I would use real data for evaluation.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2017-08-25T12:23:33.937" LastActivityDate="2017-08-25T12:23:33.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2384" PostTypeId="1" AcceptedAnswerId="2397" CreationDate="2017-08-25T11:46:38.820" Score="5" ViewCount="7133" Body="&lt;p&gt;I have 3224 Ensembl id's as rownames in a dataframe &quot;G&quot;. To convert Ensembl ids into Genesymbols I used biomart like following.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library('biomaRt')&#xA;mart &amp;lt;- useDataset(&quot;hsapiens_gene_ensembl&quot;, useMart(&quot;ensembl&quot;))&#xA;genes &amp;lt;- rownames(G)&#xA;G &amp;lt;-G[,-6]&#xA;G_list &amp;lt;- getBM(filters= &quot;ensembl_gene_id&quot;, attributes= c(&quot;ensembl_gene_id&quot;,&quot;hgnc_symbol&quot;),values=genes,mart= mart)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now in G_list I can see only 3200 ensembl ids showing Genesymbols / No Gene_symbols. Why the other 24 ensembl ids are not seen in G_list? If there are no gene_symbol for those 24 ensembl ids it should atleast show &quot;-&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples of problematic IDs are: ENSG00000257061, ENSG00000255778, ENSG00000267268. These are not at all shown in G_list (biomaRt). So, I gave them in biodbnet, which seems to handle them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;what is the problem here?&lt;/p&gt;&#xA;" OwnerUserId="1377" LastEditorUserId="77" LastEditDate="2017-08-25T12:07:08.273" LastActivityDate="2020-08-27T10:26:44.593" Title="Ensembl id to GeneSymbol with biomart" Tags="|r|bioconductor|biomart|ensembl|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2385" PostTypeId="2" ParentId="2384" CreationDate="2017-08-25T12:10:40.503" Score="6" Body="&lt;p&gt;It looks like you were using an old annotation. The problematic IDs you posted existed in the GRCh37 annotations, but don't in the most recent GRCh38 annotation. For that reason they were excluded. The IDs that have &lt;code&gt;-&lt;/code&gt; as symbols don't have associated symbols, but are present in the database.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To use an archived version in biomart:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mart = useDataset(&quot;hsapiens_gene_ensembl&quot;, useEnsembl(biomart=&quot;ensembl&quot;, version=84))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That's an example for release 84.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-08-25T13:03:35.240" LastActivityDate="2017-08-25T13:03:35.240" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="2386" PostTypeId="2" ParentId="2363" CreationDate="2017-08-25T12:17:12.407" Score="6" Body="&lt;p&gt;I modified your original question: as you are extracting 4 fields, you are not outputting BAM. The answer to the modified question is: yes, you can write a C program with htslib (or with bamtools, bioD, bioGo or rust-bio). Formatting an entire SAM is fairly expensive. You can see this by comparing &lt;code&gt;samtools view aln.bam &amp;gt; /dev/null&lt;/code&gt; and &lt;code&gt;samtools view -u aln.bam &amp;gt; /dev/null&lt;/code&gt;. With a C program, you can select fields to output. This will give you a noticeable performance boost, depending on the fields you extract.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, if you really care about performance, don't use script. The time you save from optimizing &lt;code&gt;samtools view|cut -f&lt;/code&gt; will be quickly neutralized by the inefficiency of most scripting languages.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-08-25T12:17:12.407" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2387" PostTypeId="2" ParentId="2363" CreationDate="2017-08-25T14:27:47.847" Score="3" Body="&lt;p&gt;Another library you can use for this purpose is the &lt;a href=&quot;https://github.com/samtools/htsjdk&quot; rel=&quot;nofollow noreferrer&quot;&gt;htsjdk&lt;/a&gt;, which is written in java. Using htsjdk with java is analogous to using htslib with C; the BAM format is already handled by the library and you can manipulate fields in your own code. The same basic analysis applies to java as C; you do not need to convert the BAM file to text and then parse it. Generally, a program implemented in java will be slower than C but still significantly faster than scripting. You do get language features of java: portability, array bounds checking, garbage collection, etc. &lt;/p&gt;&#xA;" OwnerUserId="272" LastActivityDate="2017-08-25T14:27:47.847" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2388" PostTypeId="2" ParentId="2338" CreationDate="2017-08-25T21:30:44.297" Score="4" Body="&lt;p&gt;Here's my current understanding of the freemix metric output by verifyBamID. The freemix score is a measure of contamination that is calculated without knowing the genotypes of the individuals. In the paper it is equation (2), and the authors refer to it as the &quot;sequence-only&quot; method. However, in addition to the sequencing reads, it requires some knowledge of SNPs and their allele frequency from the population. They demonstrate in the paper that using the SNPs and allele frequency estimates from a different population actually reduces the estimated level of contamination.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for intuition for what equation (2) is measuring, imagine what would happen to the number of heterozygous sites detected if DNA from two different individuals was sequenced together. The total number of heterozygous sites would increase because it would count all sites where the two individuals are homozygous for opposite alleles or at least one of the two individuals is heterozygous. This would lead to an increased fraction of heterozygous sites over that expected from &lt;a href=&quot;https://en.wikipedia.org/wiki/Hardy%E2%80%93Weinberg_principle&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hardy-Weinberg Equilibrium&lt;/a&gt; (i.e. 2pq).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the &lt;a href=&quot;https://groups.google.com/d/msg/verifybamid/oNMOfEZDwE4/AbVP_D-mbBEJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;explanation of freemix&lt;/a&gt; directly from the author, Hyun Min Kang:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The key idea of FREEMIX estimate is to use excessive heterozygosity to estimate the level of contamination. Especially for common SNPs, you will observe higher fraction of heterozygous alleles than 2*p*(1-p), and it turns out that you can quantify the contamination very well if you know the population allele frequency already. If you do not have accurate population allele frequency information, than it would be harder to estimate FREEMIX parameters using verifyBamID.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Additional sources:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://genome.sph.umich.edu/wiki/Verifying_Sample_Identities_-_Implementation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Verifying Sample Identities - Implementation&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://genome.sph.umich.edu/wiki/VerifyBamID#Interpreting_output_files&quot; rel=&quot;nofollow noreferrer&quot;&gt;Interpreting output files&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://groups.google.com/forum/#!forum/verifybamid&quot; rel=&quot;nofollow noreferrer&quot;&gt;verifyBamID Google Group&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1302" LastActivityDate="2017-08-25T21:30:44.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2390" PostTypeId="1" CreationDate="2017-08-26T21:14:09.057" Score="3" ViewCount="2617" Body="&lt;p&gt;In my greedy search with pseudocounts algorithm in my bioinformatics course, I did not follow the pseudocode since I wanted to solve the problem in my own way. Unfortunately, although my answer is correct, the answer we are supposed to give is the first correct answer, not the last.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem works as such: Given t strings, you make a profile matrix from the first k-mer in the first string, and then you look at the next string and using that profile matrix, you find the best k-mer match, and then you combine that k-mer with the one from the first string to make a new profile matrix, etc. You eventually do this for all k-mer windows in your first string of DNA. The way I did it was by making a probability profile matrix from the best k-mer groups in each case, and then multiplying the greatest probabilities in each column. This gives me a score, and if the next score is higher (thus, more probable) that group of k-mers is used. Below is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; def highest_probability(string, k, matrix):&#xA;    score = 1&#xA;    temp_string = &quot;&quot;&#xA;    c = 0&#xA;    best_string = &quot;&quot;&#xA;    temp = 0&#xA;    while c+k &amp;lt; len(string):&#xA;        for i in range(c, c+k):&#xA;            if string[i] == 'A':&#xA;                score *= matrix[i-c]['A']&#xA;                temp_string += 'A'&#xA;            elif string[i] == 'C':&#xA;                score *= matrix[i-c]['C']&#xA;                temp_string += 'C'&#xA;            elif string[i] == 'G':&#xA;                score *= matrix[i-c]['G']&#xA;                temp_string += 'G'&#xA;            elif string[i] == 'T':&#xA;                score *= matrix[i-c]['T']&#xA;                temp_string += 'T'&#xA;            if i == c+k-1:&#xA;                if score &amp;gt;= temp:&#xA;                    temp = score&#xA;                    best_string = temp_string&#xA;        temp_string=&quot;&quot;&#xA;        score=1&#xA;        c+=1&#xA;    print(best_string)&#xA;    return best_string&#xA;&#xA;&#xA;def GreedyMotifSearch(DNA, k, t):&#xA;    best_motifs = [i[0:k] for i in DNA]&#xA;    score=0&#xA;    c=0&#xA;    while c+k&amp;lt;len(DNA[0]):&#xA;        k_mer = DNA[0][c:c+k]&#xA;        motif=[k_mer]&#xA;        matrix = make_matrix(k_mer, matrix=[], i=0)&#xA;        for i in range(1, t):&#xA;            compare_string = DNA[i]&#xA;            good_motif = highest_probability(compare_string, k, matrix)&#xA;            motif.append(good_motif)&#xA;            matrix = make_matrix(good_motif, matrix, i)&#xA;        temp_score=1&#xA;        for dictionary in matrix:&#xA;            temp_score *= max(dictionary.values())&#xA;        print(temp_score)&#xA;        if temp_score &amp;gt; score:&#xA;            score = temp_score&#xA;            best_motifs = motif&#xA;        c+=1&#xA;    return best_motifs&#xA;&#xA;&#xA;&#xA;def make_matrix(string, matrix, i):&#xA;    if len(matrix)==0:&#xA;        for ch in string:&#xA;            new_dict = {'A':0, 'C':0, 'G':0, 'T':0}&#xA;            new_dict[ch] = 1&#xA;            matrix += [new_dict]&#xA;        print(matrix)&#xA;        return matrix&#xA;    elif len(matrix)==3:&#xA;        for j in range(len(string)):&#xA;            matrix[j][string[j]] += 1&#xA;        Sum = [sum(matrix[0].values()),sum(matrix[1].values()),sum(matrix[2].values())]&#xA;        for columns in range(len(matrix)):&#xA;            for keys in matrix[columns]:&#xA;                matrix[columns][keys] /= Sum[columns]&#xA;        print(matrix)&#xA;        return matrix &#xA;&#xA;DNA=['GGCGTTCAGGCA',&#xA;'AAGAATCAGTCA',&#xA;'CAAGGAGTTCGC',&#xA;'CACGTCAATCAC',&#xA;'CAATAATATTCG']&#xA;k=3&#xA;t=5&#xA;print(GreedyMotifSearch(DNA, k, t))  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output gives me ['TTC', 'GTC', 'TTC', 'GTC', 'TTC'] which is equally correct to the answer given in the problem output which is TTC,ATC,TTC,ATC,TTC. Unfortunately, my answer is the last correct occurrence. I tried adjusting my highest_probability function with if score is greater than temp instead of greater or equal, but that gives me the answer ['GGC', '', '', '', ''] which I don't understand at all. My question is 1) Why is this the case? and 2) How can I fix this? And does anyone have a simpler way of doing this?&lt;/p&gt;&#xA;" OwnerUserId="1385" LastActivityDate="2017-08-27T02:19:55.020" Title="Greedy Motif Search Using Probability Matrices" Tags="|python|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2391" PostTypeId="2" ParentId="2390" CreationDate="2017-08-27T02:19:55.020" Score="1" Body="&lt;p&gt;Assuming you're talking about &lt;a href=&quot;http://rosalind.info/problems/ba2d/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Implement GreedyMotifSearch&lt;/a&gt; from Rosalind, while I haven't actually finished this particular one, I've got an idea why &lt;code&gt;TTC,ATC,TTC,ATC,TTC&lt;/code&gt; doesn't work. Here's my explanation from the &quot;Questions&quot; section of that problem:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The profile is created from the first kmers in each string, and the most likely string from those is CAG. The answers will generally be similar to this.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I've been trying to do Rosalind by only using the information provided in the questions; this is one of the many problems where there wasn't enough information in the Question for me to arrive at a correct solution when I tried it. It's possible that the debug datasets were added to this question since I last looked at it, as they seem to describe this situation:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;TEST DATASET 1:&lt;/h2&gt;&#xA;&#xA;&lt;h2&gt;Input:&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;3 4&#xA;GCCCAA&#xA;GGCCTG&#xA;AACCTA&#xA;TTCCTT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Output:&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GCC&#xA;GCC&#xA;AAC&#xA;TTC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This dataset checks that your code always picks the first-occurring Profile-most Probable k-mer in a given sequence of Dna. In the first sequence (“GCCCAA”), “GCC” and “CCA” are both Profile-most Probable k-mers. However, you must return “GCC” since it occurs earlier than “CCA”. Thus, if the first sequence of your output is “CCA”, this test case fails your code.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-08-27T02:19:55.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2392" PostTypeId="1" AcceptedAnswerId="2393" CreationDate="2017-08-27T11:33:33.480" Score="2" ViewCount="319" Body="&lt;p&gt;I want to convert Ensembl ids to Gene_symbols. I used biomaRt package but all Ensembl ids are not seen in the results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples of problematic IDs are: ENSG00000255414, ENSG00000274358, ENSG00000235565, ENSG00000278309. I gave them in bioDBnet, which seems to handle them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know whether there is any R package for bioDBnet or not? &#xA;If so, could you please tell how to use it?&lt;/p&gt;&#xA;" OwnerUserId="1377" LastEditorUserId="48" LastEditDate="2017-08-28T06:33:05.113" LastActivityDate="2019-01-08T22:38:26.773" Title="Is there any R package for biodbnet?" Tags="|r|bioconductor|ensembl|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2393" PostTypeId="2" ParentId="2392" CreationDate="2017-08-27T13:52:35.477" Score="4" Body="&lt;p&gt;I couldn't find a package neither in Github nor in Bioconductor. Neither could I find any paper about a package linking to bioDBnet. So I don't believe there is any (R) package at the moment which links to &lt;a href=&quot;https://biodbnet-abcc.ncifcrf.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;biodbnet&lt;/a&gt; in order to retrieve the IDS.&lt;/p&gt;&#xA;&lt;p&gt;However it has a couple of APIs so you could build one or use them to retrieve the information you want.&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-08-27T13:52:35.477" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2394" PostTypeId="2" ParentId="2117" CreationDate="2017-08-27T22:41:15.197" Score="6" Body="&lt;p&gt;I would also recommend two very recent Hi-C visualization frameworks (with some public data available in both): &lt;a href=&quot;http://higlass.io/&quot; rel=&quot;noreferrer&quot;&gt;HiGlass&lt;/a&gt; and &lt;a href=&quot;http://www.aidenlab.org/juicebox/&quot; rel=&quot;noreferrer&quot;&gt;JuiceBox&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1389" LastActivityDate="2017-08-27T22:41:15.197" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2396" PostTypeId="2" ParentId="989" CreationDate="2017-08-29T00:01:24.383" Score="4" Body="&lt;p&gt;You need to specify an error profile to do this. However, you can shift quality scores of an existing profiles to get 10% error, which is easier than defining a complete profile.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, Illumina HiSeq 1000 has the error rate of 0.1% and you want 100 time more errors, so you should shift down the quality scores by 20 by setting both &lt;code&gt;qShift&lt;/code&gt; and &lt;code&gt;qShift2&lt;/code&gt; to -20. The following code will simulate paired end reads with 10% error.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;art_illumina -ss HS10 -i reference.fa -qs -20 -qs2 -20 -sam -p -l 100 -f 20 -m 200 -s 10 -o output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1392" LastActivityDate="2017-08-29T00:01:24.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2397" PostTypeId="2" ParentId="2384" CreationDate="2017-08-29T08:09:53.253" Score="2" Body="&lt;p&gt;I used &lt;a href=&quot;http://biit.cs.ut.ee/gprofiler/gconvert.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;gProfileR&lt;/a&gt; where there are no problematic IDs with that. It converted ENSEMBL IDs to Gene_symbols and made my work easier with GO analysis too. There is an R package and also API.&lt;/p&gt;&#xA;" OwnerUserId="1377" LastEditorUserId="48" LastEditDate="2017-08-29T18:58:59.800" LastActivityDate="2017-08-29T18:58:59.800" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2398" PostTypeId="1" CreationDate="2017-08-29T09:18:15.237" Score="4" ViewCount="1030" Body="&lt;p&gt;I have been using homer's &lt;code&gt;findMotifsGenome.pl&lt;/code&gt;, but with my new version (v4.9.1) of &lt;code&gt;homer&lt;/code&gt; I don't get &lt;code&gt;.motif&lt;/code&gt; files in the &lt;code&gt;knownResults&lt;/code&gt; folder. I do get them in the &lt;code&gt;homerResults&lt;/code&gt; (&lt;em&gt;de novo&lt;/em&gt;) folder, though. With my previous version of &lt;code&gt;homer&lt;/code&gt; I did get them and used them to find which peaks are in which motif. Which is what I would like to do&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know why there are no &lt;code&gt;.motif&lt;/code&gt; files? I have only &lt;code&gt;.svg&lt;/code&gt; files. I have seen that for the &lt;code&gt;homerResults&lt;/code&gt; the &lt;code&gt;.motif&lt;/code&gt; files are concatenated already into a file called &lt;code&gt;homerMotifs.all.motifs&lt;/code&gt;, but no such file for &lt;code&gt;knownResults&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS. I tried to call Chuck for help, but he was not answering his phone...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS2. I will ask this question on Biostars to hopefully get more response than here.&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2017-08-30T09:16:19.167" LastActivityDate="2017-11-26T16:18:00.837" Title="Where are .motif files from homer knownResults?" Tags="|chip-seq|motifs|homer|atac-seq|" AnswerCount="2" CommentCount="6" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2399" PostTypeId="1" CreationDate="2017-08-29T16:31:13.150" Score="5" ViewCount="125" Body="&lt;p&gt;Time for guess the bioinformatics file format... :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found this '.bed' file on GEO (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE84660&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSE84660&lt;/a&gt;), but it's clearly not a BED file. Anyone know what it &lt;em&gt;might&lt;/em&gt; be? And what can view it? It's from a HiC expt.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#column headers: bait1_chr, bait1_start, bait1_end, bait2_chr, bait2_start, bait2_end, contact_ID, NA, NA, B1_D0_counts, B1_D3_counts, B1_D6_counts, B2_D0_counts, B2_D3_counts, B2_D6_counts&#xA;&#xA;chr1    831895  848168  chr1    850619  874081  b2b_1   .       .       114     125     127     75      97      92&#xA;chr1    831895  848168  chr1    889424  903640  b2b_2   .       .       15      12      16      4       15      20&#xA;chr1    831895  848168  chr1    903641  927394  b2b_3   .       .       13      13      10      18      15      8&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1092" LastEditorUserId="57" LastEditDate="2017-08-31T13:55:04.510" LastActivityDate="2017-08-31T13:55:04.510" Title="What format is this? Pretty sure it's not a BED file" Tags="|file-formats|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2400" PostTypeId="2" ParentId="2399" CreationDate="2017-08-29T17:45:47.370" Score="6" Body="&lt;p&gt;It looks like a weird way to represent data. You're right, it does not at all look like a &lt;a href=&quot;http://genome.ucsc.edu/FAQ/FAQformat.html#format1&quot; rel=&quot;nofollow noreferrer&quot;&gt;BED&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM2247316&quot; rel=&quot;nofollow noreferrer&quot;&gt;Digging in GEO&lt;/a&gt; made me find this information:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Supplementary_files_format_and_content: Tab-delimited file reports one&#xA;  contact per row. Interacting HindIII fragments are represented in bed&#xA;  format (chr/start/stop) with bait (or upstream bait, in the case of&#xA;  b2b contacts) listed first. The following fields correspond to a&#xA;  unique contact ID, fragment IDs for first and second fragments, and&#xA;  raw read counts supporting contacts in each replicates.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So I think the authors merged a BED3 format plus additional BED-unrelated fields generating a tab-separated file - not clearly stated in their &lt;a href=&quot;http://www.nature.com/ng/journal/vaop/ncurrent/full/ng.3935.html?foxtrotcallback=true#methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;methods&lt;/a&gt; though. The easiest would be to contact the authors maybe?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on your aims, if you only need the first 3 fields (but I doubt it) I would reformat this &quot;bed&quot; file (to make it processable as a BED file) in two sub-files:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat original.bed | cut -f 1,2,3 &amp;gt; bait1.txt&#xA;cat bait1.txt | head -3&#xA;chr1    831895  848168 &#xA;chr1    831895  848168 &#xA;chr1    831895  848168&#xA;&#xA;cat original.bed | cut -f 4,5,6 &amp;gt; bait2.txt&#xA;cat bait2.txt | head -3&#xA;chr1    850619  874081  &#xA;chr1    889424  903640&#xA;chr1    903641  927394&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Not sure if it would fit your aims though. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, for visualization of contact matrices, maybe you'll find &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-017-1161-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; useful. &lt;/p&gt;&#xA;" OwnerUserId="294" LastEditorUserId="294" LastEditDate="2017-08-30T08:51:15.747" LastActivityDate="2017-08-30T08:51:15.747" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2403" PostTypeId="2" ParentId="2373" CreationDate="2017-08-29T22:20:51.273" Score="1" Body="&lt;p&gt;From your comment, it seems that masking regions are not your priority, but you would rather find them (correct me if I am wrong):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;not interested in masking, but really finding any type of repeats, from low complexity regions to small repeats&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To find these regions, you can try &lt;a href=&quot;http://www.cbcb.umd.edu/software/RepeatFinder/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RepeatFinder&lt;/a&gt;. From their &lt;a href=&quot;http://www.cbcb.umd.edu/software/RepeatFinder/RepeatFinderReprint.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, it seems that is suitable for bacterial genomes too. It also seems to be faster because based on suffix tree data structure instead of working with string-matching algorithm (as in RepeatMasker).  &lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2017-08-29T22:20:51.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2404" PostTypeId="1" CreationDate="2017-08-29T22:47:56.220" Score="4" ViewCount="734" Body="&lt;p&gt;I did a mapping of genomic paired-end reads to a reference assembly using bwa mem. I need to extract the reads that mapped only once to my reference.&#xA;For that, I have tried to follow the method proposed in another post &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/508/obtaining-uniquely-mapped-reads-from-bwa-mem-alignment/2402#2402&quot;&gt;Obtaining uniquely mapped reads from BWA mem alignment&lt;/a&gt; and unfortunately in my case, after using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ samtools view -h output.bam | grep -v -e 'XA:Z:' -e 'SA:Z:' | samtools view -b &amp;gt; output_filtered.bam&#xA;$ samtools view output_filtered.bam | awk '{print $1}' | sort | uniq -d&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I still obtain some outputs. Here is an example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NB501461:7:HH3GGAFXX:4:21612:26866:20400    99  ctg7180000256968    441 0   147M4S  =   560 230 AAGCTTTAGTCGCATGGCCATTCACAAGACCAGTGATTTTTTATAGAAGTCGTTCAAGAGATACTGATGAGGAGTACAGCCATTCTTTAAGCTATACAAAAAGCACCACCCACACCACACAATCCTCAAAAAGATCAAGGCAGTACAGATT AAAAAEEAEEAEAAEEEAAEEEAEEEEE/EEEEEEAA/EEEEEEEEEEEEEEAEE/EEEE///E6EAEEEEAEA&amp;lt;EE/EA/&amp;lt;/EEAE/EEEEE//EEEEEE6E/&amp;lt;EAE/EEE/EE&amp;lt;/E/E//AE&amp;lt;6EA//A&amp;lt;A///&amp;lt;A/6A/6//A///// NM:i:7  MD:Z:34A56T11A4A3A2A22T8    AS:i:112    XS:i:117&#xA;NB501461:7:HH3GGAFXX:4:21612:26866:20400    147 ctg7180000256968    560 0   32S111M8S   =   441 -230    CCAGCTCCACCACCCGACCCCCCCACCCACCCCAATCCTCAAACAGATCCATGCAGTACATTTTCATTTGTTCTGGCCCTGCGGCTCTTAGTTTAGGAAATATTCAATGACTTGCTACCTAGTAAGAACATTATTGTGCACAACGAATAGA ///E/////A////A/6A/EE/&amp;lt;//////A//AA/EEE/A/&amp;lt;E///A/A&amp;lt;//&amp;lt;/////A/////&amp;lt;///EE////E/E/E/E&amp;lt;/E//EA//E//A/&amp;lt;/E//E//AE&amp;lt;//A&amp;lt;///EE///////E//E//EA/A//EA/AE/A/E/E//6/A/ NM:i:7  MD:Z:11A5A22A4A4T27A7A24    AS:i:76 XS:i:79&#xA;NB501461:7:HH3GGAFXX:4:21612:6399:20400 113 ctg7180000282074    357 8   73S50M28S   ctg7180000432624    28  0   CTTATCCAATTTTTAAACATAGGATTAGGGTACCTTCCAAATGTTATTGAAACTTAATCTGAAGATGACCTCAAATGGAAGATGACCTCCAATGGAAGACAACCTTCCATGGAAGACGACCTCAAAAATGAAGAAGAAGAAAACATCTAAG /A//&amp;lt;//&amp;lt;&amp;lt;//E/66A//&amp;lt;A////A/&amp;lt;&amp;lt;/E/EA/EE//&amp;lt;E&amp;lt;E//&amp;lt;AE//A/EEAEEEE/E//A/A/&amp;lt;/EEE//E&amp;lt;EAEEEEE/EEEEEE&amp;lt;EEEEEEEEAEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE6EEEEEEEEEEEEEEAAAAA NM:i:2  MD:Z:32C1A15    AS:i:40 XS:i:35&#xA;NB501461:7:HH3GGAFXX:4:21612:6399:20400 177 ctg7180000432624    28  26  76S75M  ctg7180000282074    357 0   CTCCTTACTGTTGTTTCTTCTTTTTTCTTCTCTTTCATTTTTGAGGTTGTCTTCCATGTTTTGTTGTCTTCCATTTGAGGTCGTCTTCATATTAAGTCTCTATAACATTTGGCTTTGTCCCCTAAGCCTAAGTTTCAAATTTGGAAAAGAT /////&amp;lt;/////6/////A////EAEE&amp;lt;//////&amp;lt;A////E/E&amp;lt;//A//&amp;lt;///E//&amp;lt;E/////E/A/E//E/EAEEAE/EA///E//E/AE/&amp;lt;EAEE/6E6EEA/EE/AEEAE6EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEAAAAA NM:i:4  MD:Z:5T15T17G29G5   AS:i:55 XS:i:44&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As I do not see any XA or SA flags there anymore, I wonder what those can be?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to continue the steps I want to pursue until the &quot;ValidateSamFile&quot; from Picard tools, and there I do not know if it is linked or not, but I get the message:&#xA;&quot;Error: mate not found&quot; for a few thousands of reads.&#xA;Any idea what could be wrong, and how to fix it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope I was clear enough, otherwise please feel free to ask me for specific details.&#xA;Any help is welcome,&#xA;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1397" LastEditorUserId="964" LastEditDate="2017-08-29T22:53:47.450" LastActivityDate="2017-08-30T16:18:38.143" Title="Problems to Extract uniquely mapping reads from BWA MEM alignment" Tags="|read-mapping|bwa|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2405" PostTypeId="2" ParentId="2404" CreationDate="2017-08-29T22:58:37.240" Score="1" Body="&lt;p&gt;The issue is that you are filtering reads without regard to the pairing so you might filter out one of the mates if one aligns uniquely and the other does not.  You will need to filter out the un-mated reads from the file to have a valid BAM.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;As I do not see any XA or SA flags there anymore, I wonder what those can be?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am not sure what this means.  The grep should remove those flags and only leave records without them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a side note if you don't want split alignments I would recommend using a different aligner such as BWA aln or BowTie2&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2017-08-29T22:58:37.240" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2406" PostTypeId="1" AcceptedAnswerId="2407" CreationDate="2017-08-30T00:45:05.947" Score="6" ViewCount="1082" Body="&lt;p&gt;I am trying to convert DNAbin files to fasta format. Rationale: I want to use the fasta file to calculate non-synonymous/synonymous mutation rate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;my_dnabin1&lt;/code&gt; is a DNAbin file of 55 samples and I am using the following code to convert it into a fasta file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;dnabin_to_fasta &amp;lt;- lapply(my_dnabin1, function(x) as.character(x[1:length(x)]))&lt;/code&gt;&#xA;This generates a list of 55 samples which looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$SS.11.01&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &quot;t&quot; &quot;t&quot; &quot;a&quot; &quot;c&quot; &quot;c&quot; &quot;t&quot; &quot;a&quot; &quot;a&quot; &quot;a&quot; &quot;a&quot; &quot;a&quot; &quot;g&quot; &quot;c&quot; &quot;c&quot; &quot;g&quot; &quot;c&quot; &quot;t&quot; &quot;t&quot; &quot;c&quot; &quot;c&quot; &quot;c&quot; &quot;t&quot; &quot;c&quot; &quot;c&quot; &quot;a&quot; &quot;a&quot;&#xA; [27] &quot;c&quot; &quot;c&quot; &quot;c&quot; &quot;t&quot; &quot;a&quot; &quot;g&quot; &quot;a&quot; &quot;a&quot; &quot;g&quot; &quot;c&quot; &quot;a&quot; &quot;a&quot; &quot;a&quot; &quot;c&quot; &quot;c&quot; &quot;t&quot; &quot;t&quot; &quot;t&quot; &quot;c&quot; &quot;a&quot; &quot;a&quot; &quot;c&quot; &quot;c&quot; &quot;c&quot; &quot;c&quot; &quot;a&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$SS.11.02&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[1] &quot;t&quot; &quot;t&quot; &quot;a&quot; &quot;c&quot; &quot;c&quot; &quot;t&quot; &quot;a&quot; &quot;a&quot; &quot;a&quot; &quot;a&quot; &quot;a&quot; &quot;g&quot; &quot;c&quot; &quot;c&quot; &quot;g&quot; &quot;c&quot; &quot;t&quot; &quot;t&quot; &quot;c&quot; &quot;c&quot; &quot;c&quot; &quot;t&quot; &quot;c&quot; &quot;c&quot; &quot;a&quot; &quot;a&quot;&#xA; [27] &quot;c&quot; &quot;c&quot; &quot;c&quot; &quot;t&quot; &quot;a&quot; &quot;g&quot; &quot;a&quot; &quot;a&quot; &quot;g&quot; &quot;c&quot; &quot;a&quot; &quot;a&quot; &quot;a&quot; &quot;c&quot; &quot;c&quot; &quot;t&quot; &quot;t&quot; &quot;t&quot; &quot;c&quot; &quot;a&quot; &quot;a&quot; &quot;c&quot; &quot;c&quot; &quot;c&quot; &quot;c&quot; &quot;a&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and so on...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I want a fasta formatted file as the output that may look something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&amp;gt;SS.11.01 ttacctga&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&amp;gt;SS.11.02 ttacctga&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="1402" LastActivityDate="2017-08-30T01:40:35.883" Title="How to convert DNAbin to FASTA in R?" Tags="|r|fasta|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2407" PostTypeId="2" ParentId="2406" CreationDate="2017-08-30T01:40:35.883" Score="5" Body="&lt;p&gt;The following should work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat(file=&quot;out.fa&quot;, paste(paste0(&quot;&amp;gt;&quot;,names(dnabin_to_fasta)),&#xA;          sapply(dnabin_to_fasta, paste, collapse=&quot;&quot;), sep=&quot;\n&quot;), sep=&quot;\n&quot;);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A bit of explanation, starting from the middle outwards:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Collapse the letter arrays inside dnabin_to_fasta into single strings&lt;/li&gt;&#xA;&lt;li&gt;Append &quot;&gt;&quot; to the front of the names of each sequence&lt;/li&gt;&#xA;&lt;li&gt;Concatenate the results of steps 1 &amp;amp; 2 together, separating with a line break&lt;/li&gt;&#xA;&lt;li&gt;Write each concatenated string to a file, separating by a line break&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-08-30T01:40:35.883" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2408" PostTypeId="1" CreationDate="2017-08-30T07:15:15.767" Score="8" ViewCount="7139" Body="&lt;p&gt;I have downloaded &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE16146&quot; rel=&quot;noreferrer&quot;&gt;GSE16146&lt;/a&gt; dataset from GEO using GEOquery R package. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to extract &quot;Data table&quot; from downloaded GSE16146.  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;library(&quot;GEOquery&quot;)&#xA;&amp;gt;GSE16146 &amp;lt;- getGEO(&quot;GSE16146&quot;)&#xA;&amp;gt;Table(GSE16146)[1:5,]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This returns the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; Table(GSE16146)[1:5,]&#xA;Error in (function (classes, fdef, mtable)  :&#xA;unable to find an inherited method for function ‘Table’ for signature ‘&quot;list&quot;’&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can anyone help?&lt;/p&gt;&#xA;" OwnerUserId="1405" LastEditorUserId="294" LastEditDate="2017-08-30T09:42:33.323" LastActivityDate="2018-10-27T10:28:02.497" Title="Extracting expression data from GSE dataset downloaded from GEO" Tags="|r|bioconductor|microarray|geoquery|" AnswerCount="1" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2410" PostTypeId="1" AcceptedAnswerId="2444" CreationDate="2017-08-30T08:53:28.643" Score="3" ViewCount="312" Body="&lt;p&gt;Is there a database of somatic copy number alterations (CNAs) in different cancer types? That is, annotations of which CNAs are more commonly seen in different cancer types.&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="1771" LastEditDate="2018-02-09T08:45:25.553" LastActivityDate="2018-02-09T08:45:25.553" Title="Database of copy number alterations in different cancer types" Tags="|public-databases|cancer|cnv|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2411" PostTypeId="2" ParentId="2408" CreationDate="2017-08-30T09:30:20.967" Score="8" Body="&lt;p&gt;According to the &lt;a href=&quot;https://www.bioconductor.org/help/course-materials/2011/BioC2011/LabStuff/publicDataTutorial.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt;, all you need to do is: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library('GEOquery')&#xA;gseGSE16146 &amp;lt;- getGEO('GSE16146', GSEMatrix=FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As explanation, &lt;code&gt;getGEO()&lt;/code&gt; outputs by default to &lt;code&gt;GSEMatrix=TRUE&lt;/code&gt; and returns a list of &lt;code&gt;ExpressionSet&lt;/code&gt; objects. You should get what you were looking for with: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Table(GSMList(gseGSE16146)[[1]])[1:5,]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The manual has also a paragraph about this: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The GSE has  a  metadata section, just like the other classes. &#xA;  However, it doesn’t have a GEODataTable.  Instead, it contains two&#xA;  lists, accessible using GPLList and GSMList, that are each lists of&#xA;  GPL and GSM objects.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I tested the code I posted on &lt;code&gt;R 3.3.2&lt;/code&gt; and I get access to the Data Table just fine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;update: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Answering how to extract the actual expression data (not what asked in the original question): &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(GEOquery) &#xA;data = getGEO(&quot;GSE16146&quot;) &#xA;datExpr = exprs (data[[1]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="294" LastEditorUserId="294" LastEditDate="2018-10-27T10:28:02.497" LastActivityDate="2018-10-27T10:28:02.497" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="2412" PostTypeId="1" AcceptedAnswerId="5464" CreationDate="2017-08-30T13:24:41.623" Score="8" ViewCount="288" Body="&lt;p&gt;I mapped RNA reads to reference genome, using &lt;a href=&quot;http://last.cbrc.jp/&quot; rel=&quot;noreferrer&quot;&gt;LAST&lt;/a&gt; in &lt;a href=&quot;http://last.cbrc.jp/doc/last-split.html&quot; rel=&quot;noreferrer&quot;&gt;split mode&lt;/a&gt;, and converted the MAF alignment to SAM with &lt;a href=&quot;http://last.cbrc.jp/doc/maf-convert.html&quot; rel=&quot;noreferrer&quot;&gt;maf-convert&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is that the transcripts are not reported in a spliced manner, meaning that a &lt;code&gt;transcript_ID&lt;/code&gt; is reported several times in the alignment SAM file with an identical bitwise flag in &lt;code&gt;$2&lt;/code&gt;. From what I understand, this is due to the fact that only the exons are mapped (one exon per row) and reported as local alignments, as the software cannot deal with splicing patterns combination exon-intron (yet), which behaviour is clear from the CIGAR string. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a more concrete and visual example let's consider the mapping of transcript &lt;code&gt;FBtr0344900&lt;/code&gt; to the reference genome as given by LAST:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat last_aln.sam | grep FBtr0344900&#xA;FBtr0344900 0   4   42774   100 384=144H    *   0   0   TGCGACATTGTTCTACGATGACTACAAAAAATGACCAATAACTTCTATAAACCAATACGATATGTCAGGAGTTTCGGTCCCATACGAAGTCGCCGACTTAAGTATTTTATttttattttgatATGTGTTTGCTATTTTACCTTGTCGAATGCTTCCACACGCTATGAGAATACCATCGTGAGCGTAGCTTACTACTAGAATTTTGTTGAAGTTATTGACAAGCGATGTCTCAATATCTTCCGGACAGCCTCCAGCGTGACATTGCGGGGAATCATGTAACGGCCCAGTAACAGCCTCGGCCAGCACTCGAAGGTTTTCGTTAAGTTTAAGTATTTTATTTGTAGCACCCGCAAACAAAACATTGTGCATAAAGTCGAAGCTCAT    *   NM:i:0  AS:i:2304&#xA;FBtr0344900 0   4   43231   100 384H144=    *   0   0   CTGGAAGCTGTTGATTGAACTGGTATTGATGGCAAGTTAAACTGGGCGACTATGTCATTTAAGGGAGATAACGCCTGAGCCGGCAGTTCTTCAATGCAGTTAACGCAATAATGCTGAGAACCGAGTATGATAATAATACACAGT    *   NM:i:0  AS:i:864&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And here is the mapping of the same transcript &lt;code&gt;FBtr0344900&lt;/code&gt; as given by &lt;a href=&quot;https://github.com/alexdobin/STAR&quot; rel=&quot;noreferrer&quot;&gt;STAR&lt;/a&gt; - software which reports the alignment right the way I need it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat star_aln.sam | grep FBtr0344900&#xA;FBtr0344900    0    4    42774    255    384M73N144M    *    0    0TGCGACATTGTTCTACGATGACTACAAAAAATGACCAATAACTTCTATAAACCAATACGATATGTCAGGAGTTTCGGTCCCATACGAAGTCGCCGACTTAAGTATTTTATTTTTATTTTGATATGTGTTTGCTATTTTACCTTGTCGAATGCTTCCACACGCTATGAGAATACCATCGTGAGCGTAGCTTACTACTAGAATTTTGTTGAAGTTATTGACAAGCGATGTCTCAATATCTTCCGGACAGCCTCCAGCGTGACATTGCGGGGAATCATGTAACGGCCCAGTAACAGCCTCGGCCAGCACTCGAAGGTTTTCGTTAAGTTTAAGTATTTTATTTGTAGCACCCGCAAACAAAACATTGTGCATAAAGTCGAAGCTCATCTGGAAGCTGTTGATTGAACTGGTATTGATGGCAAGTTAAACTGGGCGACTATGTCATTTAAGGGAGATAACGCCTGAGCCGGCAGTTCTTCAATGCAGTTAACGCAATAATGCTGAGAACCGAGTATGATAATAATACACAGT    *    NH:i:1    HI:i:1    NM:i:0    MD:Z:528&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;https://groups.google.com/forum/#!topic/last-align/xpseSgE7Avs&quot; rel=&quot;noreferrer&quot;&gt;discussion&lt;/a&gt; with the author, it seems that I cannot get what I need straight from the current LAST version, and is not a technical problem. So, I have to modify the output myself. The aim would be to get at least a CIGAR line representing a complete transcript. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, do you know any software to do this? What I would need is a file containing one line per uniquely mapped transcripts, featuring three fields: &lt;code&gt;transcript_ID&lt;/code&gt;, &lt;code&gt;start position&lt;/code&gt; and &lt;code&gt;CIGAR string&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my side, I proceeded like this : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) extract the fields that I need for the  interested in from the SAM file: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cut -f1,4,6&#xA;FBtr0344900  42774  384=144H&#xA;FBtr0344900  43231  384H144=&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;2) split the CIGAR line to remove items I am not interested in - I simplify the command here, just assuming I only have perfect matches (which I am interested in) and hard-clipping (which I am not interested in):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cut -f3 | sed 's/H/_ /g'| sed 's/=/= /g' | sed 's/\w*_\s*//' | sed 's/=//g'&#xA;384&#xA;144&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;3) paste the modified CIGAR in the original file, with &lt;code&gt;paste&lt;/code&gt;, resulting in: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ paste (1) (2) | cut -f1,2,4&#xA;FBtr0344900  42774  384&#xA;FBtr0344900  43231  144&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;4) merge rows which start ba the same &lt;code&gt;transcript_id&lt;/code&gt;: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk -F'\t' -v OFS='\t' '{ x=$1 ; $1=&quot;&quot; ; a[x]=a[x]$0 } END { for(x in a) print x,a[x] }' | &#xA;FBtr0344900    42774    384    43231   144 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;5) compute the new cigar, calculating the intron length as the arithmetic formula &lt;code&gt;intron_length= (next_exon_start_coordinate - exon_length - previous_exon_start_coordinate)&lt;/code&gt;, in this simple case above: &lt;code&gt;intron_length= 43231-384-42774&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk '{ printf(&quot;%s&quot;, $1) }; { for (i=4; i&amp;lt;=NF; i+=2) { printf(&quot;%s%d%s%d&quot;, OFS, $(i-1), OFS, $i - $(i-1) - $(i-2)) } }; { printf(&quot;%s%d%s%s&quot;, OFS, $NF, OFS, RS) }'&#xA;FBtr0344900 384 73 144&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;6) ideally, with some method that I will figure out, I will modify the CIGAR string (adding alternate M,N at the end of each field except first one), this is how the final file should look like: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FBtr0344900 42774  384M73N144M&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The issue of my basic approach are that: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I am not sure how to account for the 1-based SAM : should I add &lt;code&gt;+1&lt;/code&gt; at every &lt;code&gt;exon_start_coordinate&lt;/code&gt;? Does not look like, as the STAR output has the exact same cigar string that I calculated on the STAR output. &lt;/li&gt;&#xA;&lt;li&gt;BIG issue: This will only work for the reads mapped in forward strand: how to make it feasible with reads mapped in the reverse strand? If I keep my current approach, I will have negative intron sizes...&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any suggestion is welcome!&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2018-11-18T08:35:09.460" Title="Convert local alignments to spliced alignments in SAM file" Tags="|sam|rna-alignment|rna-splicing|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2413" PostTypeId="1" AcceptedAnswerId="2414" CreationDate="2017-08-30T14:02:25.503" Score="4" ViewCount="81" Body="&lt;p&gt;I'm using JSmol for my &lt;a href=&quot;http://iimcb.genesilico.pl/SimRNAweb/jobs/260c8ff6-f24e-4eff-9760-1831407fc770/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SimRNAweb server&lt;/a&gt;. I would like to hide &quot;JSmol&quot; logo from all small visualizations that I have there. I would like to just put information about JSmol under the panel in text.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6yuoZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6yuoZ.png&quot; alt=&quot;image of RNA with a JSmol logo at the bottom right&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="640" LastEditorUserId="96" LastEditDate="2017-08-31T03:14:01.697" LastActivityDate="2017-08-31T03:14:01.697" Title="JSmol - hide &quot;JSmol&quot; logo" Tags="|rna-structure|3d-structure|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2414" PostTypeId="2" ParentId="2413" CreationDate="2017-08-30T15:17:32.770" Score="5" Body="&lt;p&gt;OK, I got an answer &lt;a href=&quot;https://sourceforge.net/p/jsmol/discussion/general/thread/35965248/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. Use:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;frank off&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4uWcP.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4uWcP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="640" LastActivityDate="2017-08-30T15:17:32.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2415" PostTypeId="2" ParentId="2404" CreationDate="2017-08-30T16:18:38.143" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;extract the reads that mapped only once to my reference&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Every read can be mapped to multiple places if you allow enough mismatches. You heard &quot;unique mapping&quot; from time to time. Strictly speaking, however, this concept is either ill defined or practically useless given 150bp reads. For a proper aligner, the best way to measure the reliability of an alignment is simply mapping quality. All the other approaches, including the answer in the other thread, have problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, many tools allow you to set a threshold on mapping quality. You can apply the threshold on the fly. It is a common and good practice to avoid writing filtered alignments to another BAM.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-08-30T16:18:38.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2417" PostTypeId="1" AcceptedAnswerId="2469" CreationDate="2017-08-30T20:36:46.323" Score="15" ViewCount="13257" Body="&lt;p&gt;Does anyone know what the &lt;code&gt;MAPQ&lt;/code&gt; values produced by &lt;a href=&quot;http://bio-bwa.sourceforge.net/&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;BWA-MEM&lt;/code&gt;&lt;/a&gt; mean? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for something similar to what &lt;a href=&quot;http://www.acgt.me/blog/2015/3/17/more-madness-with-mapq-scores-aka-why-bioinformaticians-hate-poor-and-incomplete-software-documentation&quot; rel=&quot;noreferrer&quot;&gt;Keith Bradnam discovered for Tophat v 1.4.1&lt;/a&gt;, where he realized that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;0 = maps to 5 or more locations &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;1 = maps to 3-4 locations &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;3 = maps to&#xA;  2 locations &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;255 = unique mapping&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm familiar with the notion that MAPQ should be theoretically be related to the probability of an &quot;incorrect&quot; alignment (10^(-&lt;code&gt;MAPQ&lt;/code&gt;/10)), although this is vaguely-specified enough that aligners in practice tend to actually just use something like the above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this is different than the &lt;code&gt;BWA&lt;/code&gt; &lt;code&gt;MAPQ&lt;/code&gt; scoring interpretation, because &lt;code&gt;BWA-MEM&lt;/code&gt; gives &lt;code&gt;MAPQ&lt;/code&gt; scores in the range $[0,60]$, rather than $[0,37]$ as has &lt;a href=&quot;http://www.acgt.me/blog/2014/12/16/understanding-mapq-scores-in-sam-files-does-37-42&quot; rel=&quot;noreferrer&quot;&gt;been established for &lt;code&gt;BWA&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1400" LastEditorUserId="1400" LastEditDate="2018-09-13T18:53:52.190" LastActivityDate="2020-01-04T04:21:25.790" Title="Meaning of BWA-MEM MAPQ scores" Tags="|sequence-alignment|read-mapping|bwa|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2418" PostTypeId="1" AcceptedAnswerId="3326" CreationDate="2017-08-30T22:25:47.747" Score="7" ViewCount="1962" Body="&lt;p&gt;Are there any suggested parameters to align ONT reads to the reference genome using &lt;a href=&quot;https://github.com/alexdobin/STAR/releases/tag/2.5.3a&quot; rel=&quot;noreferrer&quot;&gt;STAR-long&lt;/a&gt;? For now, I used the parameters suggested &lt;a href=&quot;https://github.com/PacificBiosciences/cDNA_primer/wiki/Bioinfx-study:-Optimizing-STAR-aligner-for-Iso-Seq-data#param&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, but I noticed a weird behaviour. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have RNA reads (&lt;em&gt;D. melanogaster&lt;/em&gt;) from &lt;code&gt;R7&lt;/code&gt; and &lt;code&gt;R9&lt;/code&gt; flowcells, separately. I only selected to analyze &lt;code&gt;2D&lt;/code&gt; reads in &lt;code&gt;pass&lt;/code&gt; category. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have, respectively, 113249 reads for &lt;code&gt;R7&lt;/code&gt; and 40318 reads for &lt;code&gt;R9&lt;/code&gt;. I aligned those reads and get (only!) 150 uniquely mapped reads for &lt;code&gt;R7&lt;/code&gt; data and 8017 uniquely mapped reads for &lt;code&gt;R9&lt;/code&gt; data. I tried to run again the same command on a different server with fresh compilation, but the output file is consistent with these 150 reads. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if I align the same with GMAP, I get 78016 uniquely mapped reads for &lt;code&gt;R7&lt;/code&gt; and 33523 uniquely mapped reads for &lt;code&gt;R9&lt;/code&gt;, so I suspect that something went wrong in the alignment run. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware that the two mappers behave very differently, STAR-long being  more precise and preferring to report mappings of fewer reads but at better loci, and GMAP being overall less precise, trying to map the most of the reads but at not-so-good loci. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if some of you had experience with this and could suggest me the best parameters for RNA reads from ONT?&lt;/p&gt;&#xA;" OwnerUserId="294" LastEditorUserId="294" LastEditDate="2017-08-30T22:43:23.157" LastActivityDate="2019-02-22T22:54:39.600" Title="STAR-long parameters for aligning RNA ONT reads to genome" Tags="|nanopore|rna-alignment|star|gmap|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2419" PostTypeId="1" CreationDate="2017-08-31T02:31:53.357" Score="4" ViewCount="2051" Body="&lt;p&gt;I know that the discrete raw counts from the RNA-Seq data are usually modeled by a negative binomial or a Poisson distribution, but what I am working on are the FPKM (Fragments Per Kilobase of transcript per Million mapped reads) values which represent a transformation of the raw values. They are continuous except many actual 0s. What is the probability distribution FPKM values are likely to follow, or are assumed to follow? Even after taking logarithm twice (after replacing 0s with 1, of course), the data does not follow a normal distribution since there are a lot of 0s.&lt;/p&gt;&#xA;" OwnerUserId="1331" LastActivityDate="2017-09-18T12:09:48.757" Title="What is the best distribution to model the FPKM values from normalized RNA-Seq data?" Tags="|rna-seq|statistics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2420" PostTypeId="2" ParentId="2417" CreationDate="2017-08-31T05:45:46.403" Score="7" Body="&lt;p&gt;Those numbers are not arbitrarily picked (well... maybe 255/60/40 is arbitrarily picked).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To convert from log10 Q values like these (also used for error rates in FASTQ files) to probabilities, divide the number by 10, negate it, then raise 10 to the power of the result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way of looking at it is to consider the decade to mean the number of 9s in the [mapping] accuracy, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q10: 90%&lt;br&gt;&#xA;Q20: 99%&lt;br&gt;&#xA;Q30: 99.9%&lt;br&gt;&#xA;Q40: 99.99%&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the formula adjusted to make all the numbers in between a smooth transition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this transformation, a MAPQ score of 3 corresponds to a probability of an incorrect mapping of almost exactly 50%, which is as confident as you can be about a specific mapping if you know that it maps equally well to two locations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; 10^-0.3&#xA;[1] 0.5011872&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A MAPQ score of 2 is an incorrect mapping probability of 63% (i.e. close to 3 different locations), and a MAPQ of 1 is an incorrect mapping probability of 79% (close to 5 different locations). These fit with the numbers that you have described.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as you point out, BWA doesn't do this; it seems to ignore the number of mappable locations when considering the MAPQ score. I explore the reason why this is a problem in &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/508/obtaining-uniquely-mapped-reads-from-bwa-mem-alignment/519#519&quot;&gt;my question about multi-mapped reads&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I notice that there's a bit of an attempt to answer this in a &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/289/how-does-the-bwa-mem-algorithm-assign-its-mapping-qualities&quot;&gt;similar previous question&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$C * (s_1 - s_2) / s_1,$&lt;/span&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$s_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$s_2$&lt;/span&gt; denoted the alignment scores of two best alignments and &lt;code&gt;C&lt;/code&gt; was some constant.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;See the answer by user172818 for more insight into the details of BWA-MEM MAPQ scores.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-01-04T04:21:25.790" LastActivityDate="2020-01-04T04:21:25.790" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2421" PostTypeId="1" CreationDate="2017-08-31T06:57:45.433" Score="4" ViewCount="332" Body="&lt;p&gt;I have a sequence file and want to count consecutive character &quot;N&quot;, with the tandem's position of occurrence and its length. Say I have a file named &lt;code&gt;mySequence.fasta&lt;/code&gt; like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;sequence-1 &#xA;ATCGCTAGCATNNNNNNNNNNNNNNCTAGCATCATGCNNNNNNATACGCATCACANNNNNNNNNCgcatATCAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and anticipated output should be like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Position 12 N 15&#xA;Position 38 N 6&#xA;Position 56 N 9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Kindly help me to solve this by &lt;code&gt;awk&lt;/code&gt; or &lt;code&gt;sed&lt;/code&gt; providing my file name &lt;code&gt;mySequence.fasta&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="1414" LastEditorUserId="96" LastEditDate="2017-08-31T19:30:05.920" LastActivityDate="2017-08-31T19:30:05.920" Title="Counting a specific consecutive character with its occurrence position and length" Tags="|fasta|text-processing|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2422" PostTypeId="1" AcceptedAnswerId="2454" CreationDate="2017-08-31T07:41:09.760" Score="6" ViewCount="162" Body="&lt;p&gt;I would like to know what do people verify when designing/using spike-in controls, to be used in sequencing experiments (mainly Illumina). So far I came up with this list:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does it align only to a given genome synthetic reference?&lt;/li&gt;&#xA;&lt;li&gt;Does it contain G-quadruplexes? E.g. &lt;a href=&quot;http://bioinformatics.ramapo.edu/QGRS/index.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;QGRS Mapper&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Does it contain known Illumina error motifs? (Do this matter anymore for recent platforms/chemistries?) See &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-016-0976-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Does it contain other motifs known to be polymerase inhibitors? &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4923904/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ref&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Anything else?&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="48" LastEditDate="2017-09-03T16:04:58.393" LastActivityDate="2017-09-04T06:09:45.713" Title="checks for spike-in sequence controls" Tags="|ngs|illumina|errors|spike-in|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2423" PostTypeId="2" ParentId="266" CreationDate="2017-08-31T09:18:38.437" Score="0" Body="&lt;p&gt;It indeed seems that the &lt;code&gt;--input-id-matrix&lt;/code&gt; option doesn't work as expected, as &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/346/123&quot;&gt;Amar has pointed out&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried the following command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./progressiveMauve genome1.fasta genome2.fasta genome3.fasta --output=test &amp;gt; test.log&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And saw the following bits in &lt;code&gt;test.log&lt;/code&gt; (truncated):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[...]&#xA;Genome conservation distance matrix:&#xA;0       0.0767545       0.187454&#xA;0.0767545       0       0.193385&#xA;0.187454        0.193385        0&#xA;&#xA;[...]&#xA;genome content distance matrix:&#xA;0       0.0767545       0.187454&#xA;0.0767545       0       0.193385&#xA;0.187454        0.193385        0&#xA;&#xA;bp distance matrix:&#xA;0.9     0.0125  0.075&#xA;0.0125  0.9     0.0375&#xA;0.075   0.0375  0.9&#xA;[...]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Could any of those three tables contain the information you are looking for?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;p.s. as an aside, if you are looking into a way to obtain a whole genome similarity matrix you might want to look into &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-0997-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;mash&lt;/a&gt; or &lt;a href=&quot;https://sourmash.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;sourmash&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2017-08-31T09:18:38.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2425" PostTypeId="1" CreationDate="2017-08-31T10:26:14.460" Score="9" ViewCount="145" Body="&lt;p&gt;Is there a way to run Cufflinks' &lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/cuffcompare/&quot; rel=&quot;nofollow noreferrer&quot;&gt;cuffcompare&lt;/a&gt; in a strand-agnostic mode?&lt;/p&gt;&#xA;&lt;p&gt;I would like to do this because I have some RNA-seq datasets derived from an unstranded run, that should be compared to a reference transcriptome. I think that it makes no reason to have some transcripts in the experimental dataset classified with transfrag code tag &lt;code&gt;x&lt;/code&gt; and &lt;code&gt;s&lt;/code&gt; as compared to the reference if the input dataset was not derived from stranded protocol.&lt;/p&gt;&#xA;&lt;p&gt;From &lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/cuffcompare/#transfrag-class-codes&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;x : Exonic overlap with reference on the opposite strand&lt;/p&gt;&#xA;&lt;p&gt;s : An intron of the transfrag overlaps a reference intron on the opposite strand (likely due to read mapping errors)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Conceptually, does it make sense?&lt;/p&gt;&#xA;&lt;p&gt;If so, and if the run by cuffcompare cannot be changed to a strand-agnostic manner, how should I interprete those transcripts classified as &lt;code&gt;x&lt;/code&gt;? I was thinking of merging them to &lt;code&gt;o&lt;/code&gt; and/or &lt;code&gt;c&lt;/code&gt; .&lt;/p&gt;&#xA;&lt;p&gt;Again, from manual:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;c :   Contained&lt;/p&gt;&#xA;&lt;p&gt;o :   Generic exonic overlap with a reference transcript&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="294" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2021-07-06T14:07:42.993" Title="Run cuffcompare in strand-agnostic mode" Tags="|rna-seq|transcriptome|strandedness|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2426" PostTypeId="2" ParentId="2421" CreationDate="2017-08-31T11:13:51.977" Score="2" Body="&lt;p&gt;I couldn't come up with a one liner but have a two liner instead...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get the start positions of the N repeat:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep N mySeq.fasta | grep -b -o N | awk -F: '{print $1}' |awk '$1!=p+1{print $1}{p=$1}'&#xA;12&#xA;38&#xA;56&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To get the length of the repeats:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep N mySeq.fasta | awk -F '[^N]+' '{for (i=1; i&amp;lt;=NF; i++) if ($i != &quot;&quot;) print length($i)}' &#xA;14&#xA;6&#xA;9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With thanks to stack overflow.&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2017-08-31T11:20:13.063" LastActivityDate="2017-08-31T11:20:13.063" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2427" PostTypeId="1" CreationDate="2017-08-31T12:30:28.843" Score="5" ViewCount="428" Body="&lt;p&gt;I have a very long list of prokaryote organism names. I would like to know their &lt;strong&gt;gram staining&lt;/strong&gt; (gram positive or gram negative) information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please let me know if there is any database that keeps this information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also trying to find a program that can find gram staining information via organism name only as my organism list is long.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: for each species the NCBI taxonomy lineage is also available&lt;/p&gt;&#xA;" OwnerUserId="211" LastEditorUserId="73" LastEditDate="2018-01-14T22:51:27.207" LastActivityDate="2022-09-23T21:24:23.113" Title="How can I work out gram staining information from the species name?" Tags="|database|public-databases|microbiology|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2428" PostTypeId="5" CreationDate="2017-08-31T13:08:02.013" Score="0" Body="&lt;p&gt;Biopython is a set of freely available tools for biological computation written in Python. It is developed by The Biopython Project, an international association of developers of Python tools for computational molecular biology. It includes a range of bioinformatics functionalities such as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Parsing bioinformatics files into data structures usable by Python&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Interfaces to commonly used bioinformatics programmes (BLAST,&#xA;Clustalw, EMBOSS among others)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Class for dealing with DNA, RNA and protein sequences. This includes&#xA;feature annotations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Tools for performing common operations on sequences, such as&#xA;translation, transcription and weight calculations&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;amongst many, many others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The biopython tag&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Questions with the biopython tag should relate to issues involving the Biopython package of tools.&#xA;Learning More&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The web site &lt;a href=&quot;http://www.biopython.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.biopython.org&lt;/a&gt; provides an online resource for modules, scripts, and web links for developers of Python-based software for life science research. It also has a useful &lt;a href=&quot;http://biopython.org/wiki/Main_Page&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki site&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://biopython.org/DIST/docs/tutorial/Tutorial.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biopython Cookbook&lt;/a&gt; provides many examples of BioPython being used as well as installation instructions and a FAQ section.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2017-08-31T17:30:01.963" LastActivityDate="2017-08-31T17:30:01.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2429" PostTypeId="4" CreationDate="2017-08-31T13:08:02.013" Score="0" Body="Biopython is a set of freely available tools for biological computation written in Python." OwnerUserId="123" LastEditorUserId="123" LastEditDate="2017-08-31T17:30:06.520" LastActivityDate="2017-08-31T17:30:06.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2430" PostTypeId="2" ParentId="2427" CreationDate="2017-08-31T13:23:21.527" Score="3" Body="&lt;p&gt;The &lt;a href=&quot;https://gold.jgi.doe.gov/index&quot; rel=&quot;nofollow noreferrer&quot;&gt;GOLD database&lt;/a&gt; can be used to retrieve a list of species names according to their Gram staining: go to the &quot;&lt;a href=&quot;https://gold.jgi.doe.gov/search&quot; rel=&quot;nofollow noreferrer&quot;&gt;Advanced search&lt;/a&gt;&quot; page and select the either &quot;gram-&quot; or &quot;gram+&quot; under &quot;Organism Fields&quot; -&gt; &quot;Gram Stain&quot;. Then on the results page click on the number under &quot;Organisms&quot;. On the results page you can select to have 1000 results per page, meaning that you would need to do some manual work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should add that as far as I know this information is based on what people submit when they register their genome projects, so I wouldn't expect this list to be curated or 100% accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have access to the genome of the species you are interested in (you could download a representative strain for each species using &lt;a href=&quot;https://github.com/kblin/ncbi-genome-download&quot; rel=&quot;nofollow noreferrer&quot;&gt;ncbi-genome-download&lt;/a&gt;), you could use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5192078/&quot; rel=&quot;nofollow noreferrer&quot;&gt;traitar&lt;/a&gt;, which can predict the gram staining, among other interesting broad phenotypes.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2017-09-01T08:27:33.743" LastActivityDate="2017-09-01T08:27:33.743" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2431" PostTypeId="1" AcceptedAnswerId="2433" CreationDate="2017-08-31T14:06:02.127" Score="9" ViewCount="1088" Body="&lt;p&gt;I'm looking for a convenient and reliable way to make an R analysis reproducible, either at different times or across collaborators.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Listing the package versions or a &lt;code&gt;sessionInfo()&lt;/code&gt; output is not very helpful, as it leaves the work of re-creating the environment entirely to you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I see, in order to make analysis reproducible, the following has to be the same:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the &lt;strong&gt;R environment&lt;/strong&gt; (R, repos (Bioconductor), packages, and maybe RStudio)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;data&lt;/strong&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;code&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;seed&lt;/strong&gt; for random processes: &lt;code&gt;set.seed()&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'll post an answer, but would like to hear about other options.&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2017-09-12T09:12:10.283" Title="Ultimate reproducibility in R?" Tags="|r|reproducibility|workflow-management|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2432" PostTypeId="2" ParentId="2431" CreationDate="2017-08-31T14:06:02.127" Score="1" Body="&lt;p&gt;One way to achieve this is via &lt;code&gt;packrat&lt;/code&gt;, a dependency management system developed by RStudio.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Setup:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) &lt;a href=&quot;https://support.rstudio.com/hc/en-us/articles/200526207-Using-Projects&quot; rel=&quot;nofollow noreferrer&quot;&gt;Create project&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) &lt;a href=&quot;https://stackoverflow.com/a/26598597&quot;&gt;Create script&lt;/a&gt; in project directory to start RStudio with specific R version:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;export RSTUDIO_WHICH_R=/path/to/R&#xA;rstudio&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;3) &lt;a href=&quot;https://rstudio.github.io/packrat/rstudio.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bring project&lt;/a&gt; under &lt;code&gt;packrat&lt;/code&gt; control. Install bioconductor, install packages...&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Every time you work on the project, run the above script in project directory.&#xA;&lt;a href=&quot;https://github.com/rstudio/packrat/issues/342#issuecomment-315059897&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/rstudio/packrat/issues/342#issuecomment-315059897&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2017-08-31T14:06:02.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2433" PostTypeId="2" ParentId="2431" CreationDate="2017-08-31T15:35:45.683" Score="16" Body="&lt;p&gt;You're right about trying to get the R environments to be as similar as possible, as easily as possible. It's not always necessary to have the exact environment to get the same results, but it definitely helps if you find yourself struggling to replicate results.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;General tips for ensuring reproducibility, not specific to R&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003285&quot; rel=&quot;noreferrer&quot;&gt;Sandve &lt;em&gt;et al.&lt;/em&gt;, PLoS Comp Bio, 2013&lt;/a&gt;. They mention a number of things that you do, like recording seed values and session info. I'll highlight a few points:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Environment&lt;/strong&gt;: Include a small script that lists all the necessary dependencies&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This should include all version numbers of languages and packages used&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Better implementation&lt;/em&gt;: make it a script that automatically installs dependencies&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Data&lt;/strong&gt;: Share raw and/or processed data where possible&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;At the very least, details on how to obtain the data you used&lt;/li&gt;&#xA;&lt;li&gt;This can be an accession number in a database, or a link to the source material&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Better implementation&lt;/em&gt;: again, make it a script that automatically downloads the raw data you used&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Code&lt;/strong&gt;: Comment and document your code&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;People (yourself included) need to understand what you've done and why&lt;/li&gt;&#xA;&lt;li&gt;Additionally, your work may be reproducible, but it may not be correct. If you (or anyone else) attempts to fix a problem in your code, they will have a difficult time doing so if they can't understand what your code is doing&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Code&lt;/strong&gt;: Use version control&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Services like GitHub and BitBucket make it simple to share your exact work with collaborators&lt;/li&gt;&#xA;&lt;li&gt;Version control also ensures that you have a history of all the changes done to your work&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;No manual manipulation of data&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If something is done to your data, write it down&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h1&gt;R-specific tips&lt;/h1&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The &lt;code&gt;packrat&lt;/code&gt; package, &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2432/240&quot;&gt;as you mentioned in your answer&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Using a number of custom functions for analyzing your data? Compile those functions into an R package&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Allows others to easily load/unload the exact functions you used through tools like CRAN, BioConductor, or &lt;code&gt;devtools::install_github()&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Makes it easy to see whether these functions are working as intended, since they are separated from other parts of the analysis&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Make unit tests for your work with the &lt;code&gt;testthat&lt;/code&gt; package&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Helps ensure that your code is working as intended, and gives others confidence in your work as well&lt;/li&gt;&#xA;&lt;li&gt;Write the tests using &lt;code&gt;testthat::test_that()&lt;/code&gt;, and check that they pass using &lt;code&gt;R CMD check&lt;/code&gt; or &lt;code&gt;devtools::check()&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;This works especially well with R packages&lt;/li&gt;&#xA;&lt;li&gt;This also works especially well with &lt;a href=&quot;https://travis-ci.com&quot; rel=&quot;noreferrer&quot;&gt;TravisCI&lt;/a&gt;, since you can automatically have your code get tested after you change it and make a commit&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Consider &lt;a href=&quot;https://docker.com&quot; rel=&quot;noreferrer&quot;&gt;Docker&lt;/a&gt; images for your environment&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This bundles everything about the R environment together into one easily executable image (and it's not just limited to R)&lt;/li&gt;&#xA;&lt;li&gt;These images can be easily shared through &lt;a href=&quot;https://hub.docker.com/&quot; rel=&quot;noreferrer&quot;&gt;Docker Hub&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Note&lt;/strong&gt; this might not always be the best solution, since few people know Docker and it's not always simple to use. Docker also requires admin privileges, which may not be possible for work on computer clusters&lt;/li&gt;&#xA;&lt;li&gt;There is lots of current discussion about Docker in science. See &lt;a href=&quot;https://arxiv.org/pdf/1410.0846.pdf&quot; rel=&quot;noreferrer&quot;&gt;Boettiger, 2014&lt;/a&gt;, for an example, using R&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="240" LastActivityDate="2017-08-31T15:35:45.683" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2434" PostTypeId="1" CreationDate="2017-09-01T02:55:19.517" Score="7" ViewCount="241" Body="&lt;p&gt;I found that several papers describe SNPs as a binomial distribution with the probability of &quot;success&quot; equals to minor allele frequency. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in my experiments, when I generate SNP array following this distribution, the simulation results behave very differently from the SNP array generated following the random mating procedures. I feel like there is some missing information about the description of a binomial distribution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder what will be a better mathematical description of the SNP array data? Besides a direct answer, any thoughts, or any suggestion of papers are also highly welcome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for these suggestions, after reading relevant materials, I think what I observe is about the Hardy-Weinberg equilibrium affected by evolutionary influences like mutation and migration. I will continue to look and list findings here if I find anything interesting. Please feel free to suggest some mathematical descriptions.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also thanks for the suggestion of tools. They look very helpful, but I really hope to understand some mathematics behind it. &lt;/p&gt;&#xA;" OwnerUserId="1082" LastEditorUserId="1082" LastEditDate="2017-09-01T15:20:56.907" LastActivityDate="2017-09-01T15:20:56.907" Title="What will be an appropriate mathematical distribution for SNP data?" Tags="|statistics|snp|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2435" PostTypeId="1" AcceptedAnswerId="2704" CreationDate="2017-09-01T08:53:28.960" Score="3" ViewCount="570" Body="&lt;p&gt;I would like to retrieve the InChI representation for a given KEGG compound but I fail to find a direct solution for this (preferably doing it via &lt;a href=&quot;https://pythonhosted.org/bioservices/&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioservices&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One could do it via ChEBI like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from bioservices import *&#xA;&#xA;kegg_con = KEGG()&#xA;&#xA;kegg_entry = kegg_con.parse(kegg_con.get('C00047'))&#xA;&#xA;chebi_con = ChEBI()&#xA;&#xA;chebi_entry = chebi_con.getCompleteEntity('CHEBI:' + kegg_entry['DBLINKS']['ChEBI'])&#xA;&#xA;print chebi_entry.smiles&#xA;print chebi_entry.inchi&#xA;print chebi_entry.inchiKey&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which will print&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NCCCC[C@H](N)C(O)=O&#xA;InChI=1S/C6H14N2O2/c7-4-2-1-3-5(8)6(9)10/h5H,1-4,7-8H2,(H,9,10)/t5-/m0/s1&#xA;KDXKERNSBIXSRK-YFKPBYRVSA-N&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a direct way which does not require to go via &lt;code&gt;ChEBI&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="351" LastEditorUserId="351" LastEditDate="2017-09-01T13:43:49.460" LastActivityDate="2017-10-25T19:40:31.017" Title="How to retrieve InChI key for KEGG compound?" Tags="|database|python|kegg|data-retrieval|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2436" PostTypeId="1" AcceptedAnswerId="2437" CreationDate="2017-09-01T09:52:07.090" Score="5" ViewCount="735" Body="&lt;p&gt;I am trying to read a fasta file, manipulate is in Python (using BioPython) and then write it back.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The format of my sequences is like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;k119_5 flag=0 multi=141.0706 len=473&#xA;AGGTTAGTCAGCACCGTTTCCGTGGTGCTGCCTTTCGCTTCAAAACCGACGGCGTCTATTACTGCATCCACGCCCCGGTGACCTGCCGTTTGTTCAATAATTGACTGTGCCGGATCGCTGTCTTCATCAAAATTAATCGGGATCGCGCCGTAGCGGTCGGCGGCGAAATGCAAGCGGTAGGGATGATGATCAACAACAAAAATCTGTTCCGCACCGAGCAACCGTGCACAGGCGATTGTCAACAATCCCACAGGACCAGCACCATAGACTGC&#xA;AACGCTTGAACCTTGTTGGATCTGCGCATTTTTTGCTGCCTGCCATGCCGTTGGCAGAATATCAGAAAGGAAAAGCGCTTTATCATCTGAAAGCAAAGGCGGTACTTTAAACGGCCCCACATTCCCTTTAGGGACGCGGACATATTCCGCCTGCCCACCAGGAACGCCGCCATACAGGTGACTATAACCAAACAATGCCGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which is the result of the assmebly using Megahit. While trying to read, it only recognizes &lt;code&gt;k119_5&lt;/code&gt; as the id, while I want the whole first line as the id of the sequence. Is vital information I want to keep! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas how to work with this?&lt;/p&gt;&#xA;" OwnerUserId="1417" LastActivityDate="2017-09-01T10:01:47.470" Title="Read and write FASTA files with more information than id and sequence" Tags="|fasta|python|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2437" PostTypeId="2" ParentId="2436" CreationDate="2017-09-01T09:55:54.997" Score="8" Body="&lt;p&gt;The &lt;code&gt;description&lt;/code&gt; field in the &lt;a href=&quot;http://biopython.org/wiki/SeqRecord&quot; rel=&quot;noreferrer&quot;&gt;SeqRecord&lt;/a&gt; object has the information you are looking for:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; from Bio import SeqIO&#xA;&amp;gt;&amp;gt; s = SeqIO.read('genome.fasta', 'fasta') # single sequence fasta file&#xA;&amp;gt;&amp;gt; s.id&#xA;k119_5&#xA;&amp;gt;&amp;gt; s.description&#xA;k119_5 flag=0 multi=141.0706 len=473&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Edit: as an aside, if you write a &lt;code&gt;SeqRecord&lt;/code&gt; object using &lt;code&gt;SeqIO&lt;/code&gt;'s &lt;code&gt;write&lt;/code&gt; method, the FASTA header should contain the whole original header.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="123" LastEditDate="2017-09-01T10:01:47.470" LastActivityDate="2017-09-01T10:01:47.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2438" PostTypeId="1" CreationDate="2017-09-01T09:56:33.900" Score="2" ViewCount="948" Body="&lt;p&gt;I would like to retrieve IDs from several databases using &lt;code&gt;InChI&lt;/code&gt; as an input, e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;InChI=1S/C6H14N2O2/c7-4-2-1-3-5(8)6(9)10/h5H,1-4,7-8H2,(H,9,10)/t5-/m0/s1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One can use &lt;a href=&quot;https://pythonhosted.org/bioservices/_modules/bioservices/unichem.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;unichem&lt;/a&gt; from &lt;code&gt;bioservices&lt;/code&gt; for this, however, these functions all require the &lt;code&gt;InChIKey&lt;/code&gt; as input, e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;KDXKERNSBIXSRK-YFKPBYRVSA-N&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is it possible to interconvert the two using &lt;code&gt;bioservices&lt;/code&gt; and if not is it possible to somehow use the functions in &lt;code&gt;unichem&lt;/code&gt; with &lt;code&gt;InChI&lt;/code&gt; rather than the &lt;code&gt;InChIKey&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from bioservices import *&#xA;u = UniChem()&#xA;u.get_src_compound_ids_from_inchikey('KDXKERNSBIXSRK-YFKPBYRVSA-N')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which works fine, however, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;u.get_src_compound_ids_from_inchikey('InChI=1S/C6H14N2O2/c7-4-2-1-3-5(8)6(9)10/h5H,1-4,7-8H2,(H,9,10)/t5-/m0/s1')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;does not work and returns &lt;code&gt;400&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is not passible via bioservices, is there any other python API available for this?&lt;/p&gt;&#xA;" OwnerUserId="351" LastEditorUserId="112" LastEditDate="2017-10-16T13:58:30.153" LastActivityDate="2017-10-18T05:07:44.617" Title="How to interconvert InChI and InChIKey?" Tags="|python|kegg|data-retrieval|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2439" PostTypeId="2" ParentId="2434" CreationDate="2017-09-01T10:03:37.913" Score="6" Body="&lt;p&gt;Simulating genotypes with realistic correlation structures is indeed not so simple, and there's quite a few papers dedicated entirely to that (e.g. &lt;a href=&quot;https://bmcgenet.biomedcentral.com/articles/10.1186/s12863-015-0173-4&quot; rel=&quot;noreferrer&quot;&gt;https://bmcgenet.biomedcentral.com/articles/10.1186/s12863-015-0173-4&lt;/a&gt;). Also, DEPICT (&lt;a href=&quot;https://data.broadinstitute.org/mpg/depict/index.html&quot; rel=&quot;noreferrer&quot;&gt;https://data.broadinstitute.org/mpg/depict/index.html&lt;/a&gt;) comes with a number of simulated GWASs to generate the nulls, so that's a simple way to obtain some ready-made fake data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, indepedent SNPs &lt;em&gt;should&lt;/em&gt; behave as draws from $Binomial(2, MAF)$. Note that the standard QC procedure of filtering out genotypes that deviate from Hardy-Weinberg equilibrium is just a goodness-of-fit test against $Binomial(2, \hat{MAF})$. If you employ this filter, some LD-pruning, and still get different results, consider posting more details about the discrepancies you see - that would be indeed unexpected.&lt;/p&gt;&#xA;" OwnerUserId="920" LastActivityDate="2017-09-01T10:03:37.913" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2440" PostTypeId="1" CreationDate="2017-09-01T12:03:45.913" Score="7" ViewCount="3529" Body="&lt;p&gt;I have around 1200 samples as columns and 60,000 genes with Htseq-Counts data. Before normalization with voom function I want to do filtering step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to remove genes whose expression is == 0 in at least 10 samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I do this with read counts itself or should I transform counts to cpm?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If filtering with counts data is fine May I know how to do this.&lt;/p&gt;&#xA;" OwnerUserId="1377" LastEditorUserId="294" LastEditDate="2017-09-03T16:04:42.897" LastActivityDate="2017-09-03T16:04:42.897" Title="Filtering step for read counts data" Tags="|r|rna-seq|reads|filtering|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2441" PostTypeId="2" ParentId="2440" CreationDate="2017-09-01T12:25:30.070" Score="8" Body="&lt;p&gt;If you have your counts in a data.frame called &lt;code&gt;counts&lt;/code&gt;, something like this might work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;filtered.counts &amp;lt;- counts[rowSums(counts==0)&amp;lt;10, ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For example lets assume the following data frame.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; A &amp;lt;- c(0,0,0,0,0)&#xA;&amp;gt; B &amp;lt;- c(0,1,0,0,0)&#xA;&amp;gt; C &amp;lt;- c(0,2,0,2,0)&#xA;&amp;gt; D &amp;lt;- c(0,5,1,1,2)&#xA;&amp;gt; &#xA;&amp;gt; counts &amp;lt;- data.frame(A=A, B=B, C=C, D=D)&#xA;&amp;gt; counts&#xA;  A B C D&#xA;1 0 0 0 0&#xA;2 0 1 2 5&#xA;3 0 0 0 1&#xA;4 0 0 2 1&#xA;5 0 0 0 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And now you would want to remove the genes with at least 3 zeroes or more:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; filtered.counts &amp;lt;- counts[rowSums(counts==0)&amp;lt;3, ]&#xA;&amp;gt; &#xA;&amp;gt; filtered.counts&#xA;  A B C D&#xA;2 0 1 2 5&#xA;4 0 0 2 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Every gene with 3 or more zeroes is now removed.&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2017-09-01T12:46:37.233" LastActivityDate="2017-09-01T12:46:37.233" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="2442" PostTypeId="2" ParentId="2438" CreationDate="2017-09-01T12:48:19.873" Score="1" Body="&lt;p&gt;Not sure if directly possible in &lt;code&gt;bioservices&lt;/code&gt; but one can do the following workaround using &lt;a href=&quot;https://www.chemspider.com/InChI.asmx&quot; rel=&quot;nofollow noreferrer&quot;&gt;chemspider&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;&#xA;host = &quot;http://www.chemspider.com&quot;&#xA;getstring = &quot;/InChI.asmx/InChIToInChIKey?inchi=&quot;&#xA;inchi = 'InChI=1S/C6H14N2O2/c7-4-2-1-3-5(8)6(9)10/h5H,1-4,7-8H2,(H,9,10)/t5-/m0/s1'&#xA;&#xA;r = requests.get('{}{}{}'.format(host, getstring, inchi))&#xA;if r.ok:&#xA;    res = str(r.text.replace('&amp;lt;?xml version=&quot;1.0&quot; encoding=&quot;utf-8&quot;?&amp;gt;\r\n&amp;lt;string xmlns=&quot;http://www.chemspider.com/&quot;&amp;gt;', '').replace('&amp;lt;/string&amp;gt;', '').strip())&#xA;else:&#xA;    print &quot;provide a valid inchi!&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will give the desired InChIKey&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;'KDXKERNSBIXSRK-YFKPBYRVSA-N'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which can be used in &lt;code&gt;unichem&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="351" LastActivityDate="2017-09-01T12:48:19.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2443" PostTypeId="2" ParentId="2419" CreationDate="2017-09-01T14:31:20.233" Score="5" Body="&lt;p&gt;FPKM/TPM values are generally log-normally distributed. Reference : &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1240081/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gene expression profiling in single cells from the pancreatic islets of Langerhans reveals lognormal distribution of mRNA levels&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="94" LastEditorUserId="57" LastEditDate="2017-09-17T17:00:08.440" LastActivityDate="2017-09-17T17:00:08.440" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2444" PostTypeId="2" ParentId="2410" CreationDate="2017-09-01T17:07:54.077" Score="7" Body="&lt;p&gt;I don't know if there's a database that does &lt;em&gt;exactly&lt;/em&gt; what you want, but there are some places that might help you figure this out, especially if you already have a list of CNAs/genes/regions in mind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://dcc.icgc.org/projects/details?filters=%7B%22project%22:%7B%22availableDataTypes%22:%7B%22is%22:%5B%22cnsm%22%5D%7D%7D%7D&amp;amp;projects=%7B%22from%22:1%7D&quot; rel=&quot;noreferrer&quot;&gt;ICGC&lt;/a&gt; is the first that comes to mind, as it has samples from many types of cancer and copy number data for most of them. Depending on what you want though, you might have to apply for access to download the data you want. It's rather annoying, and their data download tools are atrocious to use, but they have a &lt;strong&gt;lot&lt;/strong&gt; of data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://dgv.tcag.ca/dgv/app/home?ref=&quot; rel=&quot;noreferrer&quot;&gt;DGV&lt;/a&gt; might be another option, though their data browser is rather dated feeling. They publish a lot of gold-standard germline CNVs and such.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://cancer.sanger.ac.uk/cosmic&quot; rel=&quot;noreferrer&quot;&gt;COSMIC&lt;/a&gt; is a large database for cancer-associated mutations, including CNVs, so it might be worth a look for you.&lt;/p&gt;&#xA;" OwnerUserId="1269" LastActivityDate="2017-09-01T17:07:54.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2445" PostTypeId="1" AcceptedAnswerId="2447" CreationDate="2017-09-01T20:49:46.883" Score="3" ViewCount="4095" Body="&lt;p&gt;We all have been using samtools to convert between sam and bam files.  SAM is tab-delimited file. My question resides on the ability to visualize bam files as text.  Is there any way to visualize any bam file as text in linux? &#xA;Why not work only with sam files and we need to convert them to bam?&lt;/p&gt;&#xA;" OwnerUserId="1421" LastActivityDate="2017-09-04T15:15:06.783" Title="True bam file encoding and viewing as text" Tags="|sam|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2446" PostTypeId="2" ParentId="2445" CreationDate="2017-09-01T20:57:52.540" Score="3" Body="&lt;p&gt;bam file is a compressed sam file, comparable with zip or tar. They are binary, which means we (humans) cannot read them. So you need to unpack them (convert back to sam) to read it as a text file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;bam files are smaller than sam files, which is the reason most people like to use bam files instead of sam files.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2017-09-01T20:57:52.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2447" PostTypeId="2" ParentId="2445" CreationDate="2017-09-01T21:11:53.110" Score="8" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why not work only with sam files and we need to convert them to bam?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;SAM is for human only, a binary format like BAM is smaller and parsed much faster by a program (and we want speed and optimize spaces for those huge files, don't we ?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, bam files are compressed using a  method that allow them to be accessed using &lt;strong&gt;random-access&lt;/strong&gt;, that is how you can query a sub-part of bam file for a given genomic location.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you can always go back to SAM using samtools.&lt;/p&gt;&#xA;" OwnerUserId="71" LastActivityDate="2017-09-01T21:11:53.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2448" PostTypeId="2" ParentId="2445" CreationDate="2017-09-02T03:14:28.830" Score="3" Body="&lt;p&gt;The way to visualise BAM files as text in Linux is with tools like samtools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to see the raw binary information as slightly more human-readable, you can run &lt;code&gt;hexdump -C &amp;lt;file.bam&amp;gt;&lt;/code&gt;, but that type of text output is completely useless for further processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want more specifics about the BAM file format, you can have a look at the &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAM file format specification&lt;/a&gt;, which also includes details on the related BAM and CRAM formats.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-09-02T03:52:52.650" LastActivityDate="2017-09-02T03:52:52.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2449" PostTypeId="2" ParentId="2398" CreationDate="2017-09-02T10:18:37.603" Score="1" Body="&lt;p&gt;The relevant help page of homer seems to be this &lt;a href=&quot;http://homer.ucsd.edu/homer/motif/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;one&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On section 13 it says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;An HTML page is created in the output directory named homerResults.html along with a directory named &quot;homerResults/&quot; that contains all of the image and other support files to create the page.  These pages are explicitly created by running a subprogram called &quot;compareMotifs.pl&quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You seeem to have this images on the expected folder.&lt;br&gt;&#xA;Also on section 12 it says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;They [The *.motifs files] are reported in the output directories from findMotifs.pl and findMotifsGenome.pl&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So you'll need to specify the output directory for findMotifsGenome.pl.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2017-09-02T10:18:37.603" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2450" PostTypeId="2" ParentId="2440" CreationDate="2017-09-02T10:32:29.980" Score="3" Body="&lt;p&gt;While &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2441/48&quot;&gt;this answers&lt;/a&gt; explains how to do it I want to address when and why and which thresholds to do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Filtering the genes with low counts is usually done because the counts are not reliable it would be noise, specially when there are low number of samples these genes disturb power of the analysis. However with 1200 samples having 10 or more samples without any count is valuable information because you have enough samples to estimate if this is due to a latent or known variable or if it is due to noise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The CPM are used because the size of the library per sample affects the power of the experiment. The higher the library size the easier is to find if a gene is expressed or not. CPM are calculated to take into account that effect. Filtering by counts or CPM will result in different genes filtered if you use other thresholds than 0 (for instance I have filtered those genes that the average CPM where below 1, which would be different to filtering by those that the average counts is below 1.) &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2017-09-02T10:32:29.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2451" PostTypeId="1" AcceptedAnswerId="2457" CreationDate="2017-09-02T17:53:00.380" Score="7" ViewCount="1173" Body="&lt;p&gt;I'm trying to use the t-SNE algorithm on some microarrays data. More specifically my data frame has 18600 columns with genes (features) and 72 rows with conditions with replicates ( 10xWt , 10xTg , etc ). The expression values are in log2 scale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code that I'm trying to run.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# t-SNE implementation&#xA;library(Rtsne)&#xA;&#xA;set.seed(1)&#xA;&#xA;for(i in 1:15){&#xA;&#xA;  tsne = Rtsne(data.T[,-18601], dims = 2, perplexity=i, verbose=TRUE, max_iter = 1000, pca=T)&#xA;&#xA;  colors = rainbow(length(unique(data.T$classes)))&#xA;  names(colors) = unique(data.T$classes)&#xA;  plot(tsne$Y, t='n', main=&quot;tsne&quot;)&#xA;  text(tsne$Y, labels=data.T$classes, col=colors[data.T$classes])&#xA;&#xA;  readline(prompt=&quot;Press [enter] to continue&quot;)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Please note that I'm not counting the column 18601 because this colums contains the labels/classes for each condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The think here is that when I execute this script, R returns me this error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error: protect(): protection stack overflow&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Should I change the &lt;code&gt;--max-pp-size&lt;/code&gt; or it's a bug in Rtsne package?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also I was wondering if it is more meaningful to run the tSNE algorithm using not the log2 values of the expression level but the log fold change values in respect to the Wt (wild type) condition. I'm asking because I couldn't find a such other implementation of the tSNE on microarray data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the configuration of the Rtsne function I read &lt;a href=&quot;https://distill.pub/2016/misread-tsne/&quot; rel=&quot;noreferrer&quot;&gt;this article&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any other suggestion on the implementation is welcomed.&lt;/p&gt;&#xA;" OwnerUserId="704" LastEditorUserId="29" LastEditDate="2017-09-12T14:59:38.310" LastActivityDate="2017-09-12T14:59:38.310" Title="Using the t-SNE algorithm on microarray data + an error bonus" Tags="|r|visualization|microarray|clustering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2452" PostTypeId="1" CreationDate="2017-09-03T10:55:20.113" Score="2" ViewCount="280" Body="&lt;p&gt;I'm working with python vtkDelaunay3D (scipy.spatial.Delaunay) package for a special purpose. Normally the package is used for triangulation/tetrahedralisation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I measure euclidean distance between points with this package?&lt;/p&gt;&#xA;" OwnerUserId="818" LastEditorUserId="3848" LastEditDate="2020-04-21T14:27:42.293" LastActivityDate="2020-04-21T14:27:42.293" Title="How to measure euclidean distance between points with vtkDelaunay3D package?" Tags="|python|scipy|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="2454" PostTypeId="2" ParentId="2422" CreationDate="2017-09-04T06:04:41.237" Score="4" Body="&lt;p&gt;&lt;strong&gt;Disclaimer&lt;/strong&gt;: I'm a developer for &lt;a href=&quot;http://www.sequin.xyz&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sequin.xyz&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sequin is a new set of spiked-in controls for next-generation sequencing, and that includes Illumina. We design controls for RNA-Seq, genome sequencing, metagenomics etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please study our &lt;a href=&quot;http://www.sequin.xyz/about/papers/&quot; rel=&quot;nofollow noreferrer&quot;&gt;papers&lt;/a&gt; if you want more details.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Reference Standards For Next-Generation Sequencing&lt;/strong&gt; by Hardwick should be a good start. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here's my list:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Measured counts against input concentration. Pearson's correlation, spearman correlation and regression slope.&lt;/li&gt;&#xA;&lt;li&gt;PCA analysis&lt;/li&gt;&#xA;&lt;li&gt;Whether the synthetic aligned to the synthetic reference&lt;/li&gt;&#xA;&lt;li&gt;Spiked-in amount (e.g. dilution)&lt;/li&gt;&#xA;&lt;li&gt;Diagnostic statistics&lt;/li&gt;&#xA;&lt;li&gt;ROC Curve&lt;/li&gt;&#xA;&lt;li&gt;Detection limit&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(1) is important because the controls should give you correlation = 1, and slope = 1 in a perfect flawless experiment. This is a sample output (you can read more about it in our papers):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DjqEV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DjqEV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) This is our sample PCA (our paper has the details)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vGTh0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vGTh0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(3) is also important but you can only do that in simulations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(5)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zaEfm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zaEfm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(6)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cg6ss.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cg6ss.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(7)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NRF6p.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NRF6p.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="174" LastEditDate="2017-09-04T06:09:45.713" LastActivityDate="2017-09-04T06:09:45.713" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2455" PostTypeId="1" AcceptedAnswerId="2456" CreationDate="2017-09-04T07:54:45.077" Score="6" ViewCount="720" Body="&lt;p&gt;I'd like to use C++ to generate a new BAM file programmatically. This is an example how to use htslib to generate a new BCF file on the fly.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://github.com/samtools/htslib/blob/develop/test/test-bcf-translate.c&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/samtools/htslib/blob/develop/test/test-bcf-translate.c&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The two key functions are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;bcf_hdr_append&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;bcf_hdr_add_sample&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now when I navigate to the header file for SAM/BAM functions:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://github.com/samtools/htslib/blob/develop/htslib/sam.h&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/samtools/htslib/blob/develop/htslib/sam.h&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The only &quot;append&quot; function is &lt;code&gt;bam_aux_append&lt;/code&gt;. But that's for optional fields. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Given a &lt;code&gt;bam_hdr_t&lt;/code&gt; header (potentially empty), how to add a new chromosome reference? (For example, add a new chrAB to the BAM file)&lt;/li&gt;&#xA;&lt;li&gt;Given a non-empty &lt;code&gt;bam_hdr_t *&lt;/code&gt; header, how to remove a chromosome? (For example, remove chr4 from the BAM header)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My sample code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// Empty BAM header, nothing inside&#xA;bam_hdr_t * hdr = bam_hdr_init();&#xA;&#xA;// I'd like to add a new synthetic `chrAB` into the header, how?  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="174" LastEditorUserId="174" LastEditDate="2017-09-04T08:08:33.320" LastActivityDate="2017-09-04T08:12:16.353" Title="How to build a BAM header file with htslib in C++?" Tags="|samtools|c++|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2456" PostTypeId="2" ParentId="2455" CreationDate="2017-09-04T08:12:16.353" Score="4" Body="&lt;p&gt;You're just looking in the wrong header file. The BCF stuff will be in &lt;code&gt;vcf.h&lt;/code&gt; (you'll see the two functions you mentioned there).&lt;/p&gt;&#xA;&lt;p&gt;With BAM files one doesn't typically need to add/remove headers, which is why there are not convenience functions for them. To add a header:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Increment &lt;code&gt;hdr-&amp;gt;ntargets&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;realloc(hdr-&amp;gt;target_len, ...)&lt;/code&gt; and &lt;code&gt;hdr-&amp;gt;target_name&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;You likely want to append the name/length to the end of &lt;code&gt;hdr-&amp;gt;target_name&lt;/code&gt; and &lt;code&gt;hdr-&amp;gt;target_len&lt;/code&gt;, respectively.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I don't think you need to update the dictionary or plain text for most uses (or at least I've never needed to).&lt;/p&gt;&#xA;&lt;p&gt;Removing a chromosome would follow the reverse instructions, (you'd need to shift everything in the name and length arrays if you remove entries in the middle.&lt;/p&gt;&#xA;&lt;h1&gt;WARNING, doing this is prone to error!&lt;/h1&gt;&#xA;&lt;p&gt;I would strongly cautious you to be very careful when doing this. Please remember that the chromosome that entries are mapped to isn't actually stored in alignment entries themselves. Rather, they simply have an index into &lt;code&gt;hdr-&amp;gt;target_name&lt;/code&gt;, which saves space. Consequently, if you start adding/removing headers to/from a BAM file with alignments then you're going to have to modify &lt;strong&gt;all&lt;/strong&gt; of the alignments to keep the correct alignment:chromosome associations. Otherwise you'll end up with completely useless results (this is the same caveat as with &lt;code&gt;samtools reheader&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-09-04T08:12:16.353" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2457" PostTypeId="2" ParentId="2451" CreationDate="2017-09-04T12:07:15.483" Score="13" Body="&lt;p&gt;Converting your &lt;code&gt;data.frame&lt;/code&gt; to a &lt;code&gt;matrix&lt;/code&gt; (and then removing the &lt;code&gt;data.frame&lt;/code&gt;) will often free up enough memory that you won't run into this. Note that a &lt;code&gt;matrix&lt;/code&gt; is more memory efficient than a &lt;code&gt;data.frame&lt;/code&gt; and you're requiring &lt;code&gt;Rtsne()&lt;/code&gt; to hold both in memory at the same time (many math-centric functions will end up converting things to a matrix at some point for efficiency).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For what it's worth, it's never been entirely clear to me what the interaction is between a &lt;code&gt;data.frame&lt;/code&gt; and the pointer protection stack, but it's often the case that this solves this sort of error.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-09-04T12:07:15.483" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2458" PostTypeId="2" ParentId="2445" CreationDate="2017-09-04T15:15:06.783" Score="3" Body="&lt;p&gt;Assuming you want to visualize the pileup of reads across the reference, you may want to look into &lt;a href=&quot;https://github.com/dariober/ASCIIGenome&quot; rel=&quot;nofollow noreferrer&quot;&gt;ASCIIgenome&lt;/a&gt;, a command line visualization tool for sam/bam/cram files, similar in spirit as &lt;a href=&quot;http://software.broadinstitute.org/software/igv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IGV&lt;/a&gt;, but working in the command line.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ASCIIGenome my_sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ouCp4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ouCp4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2017-09-04T15:15:06.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2459" PostTypeId="1" AcceptedAnswerId="2460" CreationDate="2017-09-05T07:59:58.900" Score="5" ViewCount="1036" Body="&lt;p&gt;I have population SNP data in VCF format, and I found that some SNPs have a great similarity across samples(&gt; 99%). For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CHROM_POS   s0  s1  s2  s3  s4  s5  s6  s7  s8  s9&#xA;chr1_1  A   G   G   A   G   G   A   A   G   A&#xA;chr1_2  A   G   G   A   G   G   A   A   G   T&#xA;chr1_3  C   C   C   A   C   A   C   C   C   C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The similarity between chr1_1 and chr1_2 is 0.9, because they have a different SNP in sample &lt;strong&gt;s9&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a good way to remove these similar SNPs before I put the data into another pipeline?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Added: These SNPs will be used for GWAS analysis.&lt;/p&gt;&#xA;" OwnerUserId="1085" LastEditorUserId="240" LastEditDate="2017-09-05T18:49:52.513" LastActivityDate="2017-09-05T18:49:52.513" Title="How to deal with correlated SNPs in GWAS?" Tags="|snp|gwas|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2460" PostTypeId="2" ParentId="2459" CreationDate="2017-09-05T09:46:17.530" Score="6" Body="&lt;p&gt;What you are attempting to do is known as &lt;strong&gt;LD-pruning&lt;/strong&gt;.&lt;br&gt;&#xA;As @Emily_Ensembl said, it is not customary to do this for standard association tests: it is possible that one of the SNPs you remove is causal, or a better proxy for the causal locus, and would give (slightly) better association signal than the other. Even for SNPs in perfect LD, pruning is unwise because it would complicate interpretation: if SNP &lt;code&gt;chr1_2&lt;/code&gt; had some obvious function, but you removed it and are left only with its proxy &lt;code&gt;chr1_1&lt;/code&gt; (which could be far away), you will have a hard time connecting the dots. Typical GWAS tests are handled pretty quickly on modern computers, and are linear in SNPs, so there is no computational reason to prune either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, many fancier procedures would require LD-pruned data, because they either&lt;br&gt;&#xA;a) are very computationally expensive, or&lt;br&gt;&#xA;b) assume independent SNPs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suggest reading &lt;a href=&quot;https://www.cog-genomics.org/plink/1.9/ld&quot; rel=&quot;noreferrer&quot;&gt;PLINK's LD pruning documentation&lt;/a&gt;, as it pretty much covers the standard ways to do this.&lt;/p&gt;&#xA;" OwnerUserId="920" LastActivityDate="2017-09-05T09:46:17.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2461" PostTypeId="1" AcceptedAnswerId="2475" CreationDate="2017-09-05T10:53:59.850" Score="3" ViewCount="240" Body="&lt;p&gt;Which genes/transcripts show up as having the highest RPKM/FPKM on a human RNA-seq experiment? Is it usually the same genes/transcripts?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: mRNAs (e.g. poly-A RNA-seq preps) and not including small RNAs.&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="180" LastEditDate="2017-09-05T14:06:35.320" LastActivityDate="2017-09-06T13:04:56.537" Title="genes with highest RPKM/FPKM for a human RNA-seq experiment?" Tags="|rna-seq|ngs|illumina|human|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2462" PostTypeId="1" AcceptedAnswerId="2474" CreationDate="2017-09-05T16:07:52.180" Score="4" ViewCount="1031" Body="&lt;p&gt;I need to convert a lot of Ensembl Ids to the relative counterpart in Entrez (e.g., &lt;code&gt;ENSG00000157764&lt;/code&gt; &gt; &lt;code&gt;673&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found &lt;a href=&quot;http://mygene.info&quot; rel=&quot;nofollow noreferrer&quot;&gt;mygene.info&lt;/a&gt; and it seems what I needed. Let's see the query about &lt;a href=&quot;http://mygene.info/v3/gene/ENSG00000157764&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;ENSG00000157764&lt;/code&gt;&lt;/a&gt; in action. We can easily find the key-value &lt;code&gt;&quot;entrezgene&quot;: 673&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the mapping seems not complete in some cases or, maybe, I'm missing something...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Gene: &lt;code&gt;NDUFV2P1&lt;/code&gt; &lt;em&gt;(NADH:ubiquinone oxidoreductase core subunit V2&#xA;pseudogene 1)&lt;/em&gt;    &lt;/li&gt;&#xA;&lt;li&gt;Expected Mapping&#xA;&lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Gene/Summary?g=ENSG00000267809;r=19:53223973-53224879;t=ENST00000596415&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;ENSG00000267809&lt;/code&gt;&lt;/a&gt; &gt; &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene?cmd=retrieve&amp;amp;dopt=default&amp;amp;list_uids=4730&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;4730&lt;/code&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If I query &lt;em&gt;mygene&lt;/em&gt; for &lt;code&gt;ENSG00000267809&lt;/code&gt;, I get &lt;a href=&quot;http://mygene.info/v3/gene/ENSG00000267809&quot; rel=&quot;nofollow noreferrer&quot;&gt;this result&lt;/a&gt;, but this time no data about the entrez id.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What it sounds pretty odd is that if I query another &lt;em&gt;mygene&lt;/em&gt;'s endpoint for &lt;a href=&quot;http://mygene.info/v3/query?q=NDUFV2P1&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;NDUFV2P1&lt;/code&gt;&lt;/a&gt;, 2 identical records are returned, respectively with Ensembl and Entrez Ids.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to leverage such a service to get clean data? Otherwise, is there another robust conversion mapping strategy out there?&lt;/p&gt;&#xA;" OwnerUserId="1063" LastEditorUserId="1063" LastEditDate="2017-09-06T14:53:57.743" LastActivityDate="2017-09-06T14:53:57.743" Title="Missing data mappings in mygene.info while trying to convert Genes Ensembl Ids to Entrez Ids" Tags="|gene|public-databases|conversion|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2463" PostTypeId="1" AcceptedAnswerId="2464" CreationDate="2017-09-05T16:56:15.497" Score="4" ViewCount="1420" Body="&lt;p&gt;I'm trying to use the bfGWAS tool, which analyses GWAS data and integrates functional annotations to identify casual SNPs (&lt;a href=&quot;http://www.cell.com/ajhg/fulltext/S0002-9297(17)30324-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; and &#xA;&lt;a href=&quot;https://github.com/yjingj/bfGWAS&quot; rel=&quot;nofollow noreferrer&quot;&gt;github&lt;/a&gt;). In the user manual, it states:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We recommend first regressing covariants out from the original&#xA;  phenotypes, and then provide bfGWAS the corrected phenotypes (i.e.,&#xA;  residuals from the regression model with covariates).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is unclear to me what this means and how to go about determining the corrected phenotypes. Typically, in an association test with covariates, a regression would be performed using a model like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pheno ~ snp + covariate &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I could then get the residuals from this model. However, I'm not certain how to go about it for all SNPs, and then adjust the phenotypes. I have case-control data, so the phenotypes are binary.&lt;/p&gt;&#xA;" OwnerUserId="1443" LastEditorUserId="57" LastEditDate="2017-09-05T17:31:41.797" LastActivityDate="2017-09-05T18:49:25.670" Title="Adjusting phenotypes by regressing out covariates" Tags="|statistics|gwas|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2464" PostTypeId="2" ParentId="2463" CreationDate="2017-09-05T18:49:25.670" Score="2" Body="&lt;p&gt;Instead of the &quot;observed phenotype&quot;, it's asking for a &quot;corrected phenotype&quot;, which is the residuals from &lt;code&gt;pheno ~ covariates&lt;/code&gt;. In R, one would get that as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fit = lm(phenotype ~ covariates, data)&#xA;corrected_phenotype = resid(fit)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Of course &lt;code&gt;covariates&lt;/code&gt; is just a place hold for something like &lt;code&gt;age + gender&lt;/code&gt; or something along those lines.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-09-05T18:49:25.670" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2465" PostTypeId="1" AcceptedAnswerId="2496" CreationDate="2017-09-05T19:09:50.460" Score="5" ViewCount="749" Body="&lt;p&gt;Given a list of Uniprot IDs that are linked to an Ensembl gene ID, is there a way to systematically determine the UniProt accession corresponding to the protein product of the gene's primary transcript &lt;strong&gt;with no other information&lt;/strong&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://web.expasy.org/docs/userman.html#AC_line&quot; rel=&quot;nofollow noreferrer&quot;&gt;According to ExPasy&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Researchers who wish to cite entries in their publications should&#xA;  always cite the first accession number. This is commonly referred to&#xA;  as the 'primary accession number'. 'Secondary accession numbers' are&#xA;  sorted alphanumerically.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But what if the order has been jumbled or compiled from a different source and resorted?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ensembl:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ENSMUSG00000035642&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Uniprot: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q8R0P4, Q8CF11, D6RJK8, D6RJJ4, D3Z442, D3Z1Q3, D3YZD8, D3YY39, D3YX09, D3YWY5&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://biology.stackexchange.com/questions/65595/how-to-determine-the-primary-uniprot-accession-number-from-a-list-of-accession-n/65606#65606%22related%20ticket%20in%20the%20Biology%20section%22&quot;&gt;This question is cross listed on the Biology stack-exchange.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1446" LastEditorUserId="1446" LastEditDate="2017-09-08T14:09:07.213" LastActivityDate="2017-09-08T14:09:07.213" Title="How to determine the primary Uniprot accession number from a set of accession numbers?" Tags="|proteins|public-databases|identifiers|" AnswerCount="3" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="2466" PostTypeId="1" CreationDate="2017-09-05T20:05:49.490" Score="8" ViewCount="1517" Body="&lt;p&gt;I was following a tutorial on &quot;Tuxedo Genome Guided Transcriptome Assembly Workshop&quot; and was wondering how to interpret the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xfFmi.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xfFmi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I understand from 'Color Legends', the color blue represents something that is below normal and red represents something that is above normal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, my understanding is that merged.gtf is the reconstructed transcript and genes.bed is the reference. They look to be similar for the most part, but seems to have breaks between. What's the appropriate way of interpreting this result?&lt;/p&gt;&#xA;" OwnerUserId="711" LastActivityDate="2017-09-05T22:21:56.613" Title="Interpreting Intergrative Genomic Viewer (IGV)" Tags="|rna-seq|sequence-alignment|visualization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2467" PostTypeId="1" AcceptedAnswerId="2491" CreationDate="2017-09-05T21:44:41.777" Score="7" ViewCount="2178" Body="&lt;p&gt;I am interested in determining if any transcription factor binding site motifs are enriched in some BED files from a DNA methylation experiment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a database that has BED Files containing regions enriched for specific transcription factor binding motifs in homo sapiens.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that JASPAR contains TFBS motifs, but I can't find a download for regions of hg19 that are enriched with those motifs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:  I suppose I could determine TF motif enriched regions using HOMER if there are no curated databases with this information, but I would rather not.  Your help is appreciated!&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="39" LastEditDate="2017-09-06T19:34:13.750" LastActivityDate="2023-05-11T18:14:49.343" Title="Where to download JASPAR TFBS motif bed file?" Tags="|database|sequence-annotation|chip-seq|motifs|ucsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2468" PostTypeId="2" ParentId="2466" CreationDate="2017-09-05T22:21:56.613" Score="6" Body="&lt;p&gt;You've apparently colored your alignments by read strand. In this case, red indicates &quot;+ (watson) strand&quot; and blue indicates &quot;- (strand) strand&quot;. This strand association is determined by the orientation of read #1 in a pair (or just &quot;the read&quot; if you have single-end data). This isn't actually documented in the user manual, but you can find one of the Broad institute software engineers &lt;a href=&quot;https://groups.google.com/forum/#!searchin/igv-help/strand$20color%7Csort:relevance/igv-help/YiVzwnUuOZM/sUHc4V1OXZUJ&quot; rel=&quot;noreferrer&quot;&gt;replying here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your next question is likely why the color of the reads and the orientation of the genes are opposite of each other. This is because in modern directional protocols, read #2 in a pair matches the orientation of the gene from which it arose.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-09-05T22:21:56.613" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2469" PostTypeId="2" ParentId="2417" CreationDate="2017-09-06T03:13:46.477" Score="16" Body="&lt;p&gt;First of all, if you want to understand mapping quality (mapQ), ignore RNA-seq mappers. They often produce misleading mapQ because mapQ is not important to RNA-seq anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Strictly speaking, you have two questions, one in the title: the meaning of mapQ; and the other in a comment: how mapQ is computed. On the meaning, mapQ is nearly the same as baseQ – the phred scaled probability of the alignment/base being wrong. It often amuses me that we question mapQ but take baseQ for granted. BaseQ is also scaled and discretized differently; even fewer people know how Illumina/pacbio/nanopore/historical sequencers estimates baseQ.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the second question, Section 2 of the &lt;a href=&quot;http://genome.cshlp.org/content/suppl/2008/09/26/gr.078212.108.DC1/maq-supp.pdf&quot; rel=&quot;noreferrer&quot;&gt;MAQ supplementary&lt;/a&gt; explains the theoretical aspects of mapQ, which is still correct today. Briefly, mapping quality consists of three components: 1) the probability of contamination; 2) the effect of mapping heuristics and 3) the error due to the repetitiveness of the reference. Only 3) can be modeled theoretically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case of bwa-mem, if we assume the matching score is 1, type-3 error is estimated with:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$10/\log10\cdot[\log4\cdot(S_1-S_2)-\log n_{\rm sub}]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where $S_1$ is the best alignment score, $S_2$ is the second best and $n_{\rm sub}$ is the number of suboptimal alignments. Factor $\log 4$ comes from the scoring matrix. Factor $10/\log10$ is the Phred scale. This equation assumes gap-free alignment and is very close to Section 2.5.2 in the MAQ supplementary. It is ok-ish for short reads, but often overestimates for long reads. I am not aware of a practical approach in general cases. In addition to this method, you can estimate mapQ by read simulation: just try to find a function such that it fits empirical mapQ. Some have tried machine learning, too.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-09-06T03:13:46.477" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2470" PostTypeId="1" AcceptedAnswerId="2471" CreationDate="2017-09-06T10:54:04.097" Score="8" ViewCount="10707" Body="&lt;p&gt;Well, I'm trying to convert a list of Human Gene referenced by Ensembl Gene IDs to Entrez Gene IDs. I &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2462/missing-data-mappings-in-mygene-info-while-trying-to-convert-genes-ensembl-ids-t&quot;&gt;have been advised&lt;/a&gt; to use &lt;a href=&quot;http://www.ensembl.org/biomart/martview/&quot; rel=&quot;noreferrer&quot;&gt;biomart&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to get a kind of conversion table for all human genes. I don't know if my settings are wrong, but I didn't find any checkbox for Entrez Gene Id in the section &lt;code&gt;Attributes&lt;/code&gt; &gt; &lt;code&gt;External References&lt;/code&gt;. I've just found the Entrez transcript name ID (see below), but it's not what I need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VWtTT.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VWtTT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I use &lt;em&gt;biomart&lt;/em&gt; for such a conversion?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strike&gt;According to &lt;a href=&quot;https://youtu.be/DXPaBdPM2vs?t=1m50s&quot; rel=&quot;noreferrer&quot;&gt;this video tutorial&lt;/a&gt; EntrezGene ID should be an option in biomart (see screenshot below).&lt;/strike&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That video tutorial is outdated.&lt;/p&gt;&#xA;" OwnerUserId="1063" LastEditorUserId="1063" LastEditDate="2017-09-06T11:58:06.303" LastActivityDate="2023-09-07T16:15:48.133" Title="Converting Ensembl Gene IDs to Entrez Gene IDs through biomart" Tags="|gene|public-databases|biomart|conversion|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2471" PostTypeId="2" ParentId="2470" CreationDate="2017-09-06T11:22:30.280" Score="8" Body="&lt;p&gt;The ID you need is the NCBI gene ID, which is the same as the EntrezGene ID. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2017-09-06T11:22:30.280" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2474" PostTypeId="2" ParentId="2462" CreationDate="2017-09-06T11:58:22.727" Score="8" Body="&lt;p&gt;As others have mentioned in their answer, bioMart is usually the best place to go for this infomation because it draws its data directly from the Ensembl database. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you will find you do not get a full translation of Ensembl IDs to Entrez IDs even there. The reason for this is there simply does not exist a simple 1-to-1 mapping of Entrez IDs to Ensembl IDs: they are not just different names for the same thing, but are actually different things. At some point someone (or more likely some algo) has decided that the structure Ensembl ID A is referring to is similar enough to the structure that Enztrez ID B is referring to to link the two together, but such a linking is not always possible.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the example you gave, the Ensembl database does not list ENSG00000267809 as being equivalent to 4730, neither does the Entrez database list the opposite, even though both genes have the same name. Because of this, it is very unlikely that any service you use to convert the IDs will pick up this match. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this particular case, the reason for this is probably that the gene is a processed pseduo-gene and is not annotated in Refseq/NCBI as producing any transcripts. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2017-09-06T11:58:22.727" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2475" PostTypeId="2" ParentId="2461" CreationDate="2017-09-06T13:04:56.537" Score="6" Body="&lt;p&gt;As asked as it is, the answer is probably no.&lt;/p&gt;&#xA;&lt;p&gt;Indeed, the highest expressed RPKM/FPKMs will be different from one condition (or tissue) to another one for example.&lt;/p&gt;&#xA;&lt;p&gt;Then, you may also have technical artefacts due to the wet-lab part or to the normalisation. For example, mitochondrial genes are often reported in the top expressed genes.&lt;/p&gt;&#xA;&lt;p&gt;Now, if you want to compare your results to an outside source, I recommend checking the &lt;a href=&quot;https://www.ebi.ac.uk/gxa/home&quot; rel=&quot;noreferrer&quot;&gt;EBI gene expression Atlas&lt;/a&gt;* and more specifically the Baseline expression experiments.&lt;/p&gt;&#xA;&lt;p&gt;Pro-tips:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You can filter the genes based on their nature (eg, mRNA) and/or their expression.&lt;/li&gt;&#xA;&lt;li&gt;You can also visualise the overall expression of a specific gene across several experiments.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;*Disclaimer: I am related to this project myself.&lt;/p&gt;&#xA;" OwnerUserId="203" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-09-06T13:04:56.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2476" PostTypeId="1" AcceptedAnswerId="2477" CreationDate="2017-09-06T13:59:04.923" Score="7" ViewCount="2368" Body="&lt;p&gt;I am writing a small script to write a VCF file from short multiple sequence alignments, and stumbled into an interesting corner case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/variation/tools/reporter/docs/examples#vcf-section-3&quot; rel=&quot;noreferrer&quot;&gt;the way a deletion should be represented in VCF format&lt;/a&gt; (I assume version 4+):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NC_000016.9  2138199 rs1799758   GTGAG   G  .   .  RS=1799758;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But what happens if the deletion is at the first position? Immagine the following two aligned sequences:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;reference&#xA;AGTC&#xA;&amp;gt;sample&#xA;-GTC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How should it be encoded in VCF format? According to &lt;a href=&quot;http://seqanswers.com/forums/showthread.php?t=30884&quot; rel=&quot;noreferrer&quot;&gt;this thread on SEQAnswers&lt;/a&gt;, it might be necessary to include the first position after the end of the deletion, like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chromosome  1  AG G  .  .  [...]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is this a VCF 4+ correct way to encode such variants?&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="96" LastEditDate="2019-01-15T14:26:08.597" LastActivityDate="2019-01-15T14:26:08.597" Title="How to represent a deletion at position 1 in a VCF file?" Tags="|vcf|variation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2477" PostTypeId="2" ParentId="2476" CreationDate="2017-09-06T14:07:27.647" Score="7" Body="&lt;p&gt;From the latest &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.3.pdf&quot; rel=&quot;noreferrer&quot;&gt;VCF specs&lt;/a&gt; (page 8):&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;REF - reference base(s): Each base must be one of A,C,G,T,N (case insensitive). Multiple bases are permitted. The value in the POS field refers to the position of the first base in the String. For simple insertions and deletions in which either the REF or one of the ALT alleles would otherwise be null/empty, the REF and ALT Strings must include the base before the event (which must be reflected in the POS field), unless the event occurs at position 1 on the contig in which case it must include the base after the event;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So I think you're right&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2017-09-06T14:07:27.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2478" PostTypeId="1" AcceptedAnswerId="2479" CreationDate="2017-09-06T14:57:38.687" Score="3" ViewCount="109" Body="&lt;p&gt;I've been trying to figure out the correct way of extracting methylation status of particular elements from WGBS data, and found several possible approaches, but would like to discuss which one could be the best:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;using software dedicated to the extraction of these elements on WGS (like &lt;a href=&quot;https://github.com/cancerit/TraFiC&quot; rel=&quot;nofollow noreferrer&quot;&gt;TraFiC&lt;/a&gt; or &lt;a href=&quot;http://melt.igs.umaryland.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MELT&lt;/a&gt; and then quantify methylation (like &lt;a href=&quot;https://www.bioinformatics.babraham.ac.uk/projects/bismark/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bismark&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;using bedtools to scan bam file with a filtered version of &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTables&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC RepeatMasker table&lt;/a&gt; in bed format) and then quantify methylation&lt;/li&gt;&#xA;&lt;li&gt;I've seen the use of custom perl scripts on fastq files like in &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S221112471631631X#mmc1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, in which the Supplemental Methods Section states: &lt;em&gt;&quot;A custom Perl script was written to assay methylation of CpGs within a previously published motif of LINE-1 elements (Yang et al., 2004). Methylation levels for each of the four CpGs within the consensus LINE-1 element were determined globally for each WGBS sample prior to alignment using Line1_FASTQ.pl. Dup15q and control were compared and tested for significance using an unpaired t-test.&quot;&lt;/em&gt; &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any thoughts on this?&lt;/p&gt;&#xA;" OwnerUserId="1375" LastActivityDate="2017-11-04T06:34:33.567" Title="Extract LINE-1 methylation status from WGBS data" Tags="|methylation|transposable-elements|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2479" PostTypeId="2" ParentId="2478" CreationDate="2017-09-06T17:51:33.470" Score="0" Body="&lt;p&gt;Have you checked out &lt;a href=&quot;https://github.com/kwdunaway/WGBS_Tools&quot; rel=&quot;nofollow noreferrer&quot;&gt;WGBS Tools&lt;/a&gt; on github? All of the custom scripts used in the paper that you referenced are in that repository. Each tool has a brief description of how to use it.&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2017-09-06T17:51:33.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2480" PostTypeId="1" AcceptedAnswerId="2485" CreationDate="2017-09-06T18:00:39.713" Score="4" ViewCount="643" Body="&lt;p&gt;I have run a blastx of metagenomic databases (raw illumina reads) using the nr database. Unfortunately, I forgot to add the &lt;code&gt;--outfmt 6&lt;/code&gt; argument to the code and got the traditional output.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could I parse this output into &lt;code&gt;--outfmt 6&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="1421" LastEditorUserId="3848" LastEditDate="2020-05-03T02:02:36.487" LastActivityDate="2020-05-03T02:02:36.487" Title="Transform traditional blast output to `--outfmt 6`" Tags="|python|blast|metagenome|parsing|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="2481" PostTypeId="2" ParentId="2480" CreationDate="2017-09-06T18:31:24.607" Score="3" Body="&lt;p&gt;The &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK279697/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;blast_formatter&lt;/code&gt; command&lt;/a&gt; allows you to produce output in any of the formats supported by BLAST. However, it requires that you run &lt;code&gt;blastx&lt;/code&gt; with &lt;code&gt;--outfmt 11&lt;/code&gt; (ASN.1 format). I'm not sure there are any tools that will do the conversion from the full default BLAST report. This could potentially be scripted, but would be complex enough that you're probably better off re-running BLAST.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2017-09-06T18:31:24.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2482" PostTypeId="2" ParentId="2465" CreationDate="2017-09-06T20:22:17.557" Score="3" Body="&lt;p&gt;Consider the circumstances in which entries will have more than one accession (taken from the same page you linked in the original post).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Entries will have more than one accession number if they have been merged or split. For example, when two entries are merged into one, the accession numbers from both entries are stored in the AC line(s).&lt;/p&gt;&#xA;&lt;p&gt;If an existing entry is split into two or more entries (a rare occurrence), the original accession numbers are retained in all the derived entries and a new primary accession number is added to all the entries.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As mentioned in the comments, this &lt;em&gt;does not&lt;/em&gt; imply that there is or should be a single &amp;quot;primary&amp;quot; UniProt accession associated with (for example) an Ensembl gene ID. In some cases, a single protein record may have multiple accessions due to annotation changes, and the suggestion from your original post is to only report the primary accession. However, there may be multiple proteins associated with an Ensembl gene ID, and each protein will have its own accession (or set of accessions).&lt;/p&gt;&#xA;&lt;p&gt;Choosing a single representative isoform for a gene is an entirely different question.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-09-06T20:22:17.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2483" PostTypeId="1" AcceptedAnswerId="2487" CreationDate="2017-09-07T06:26:42.677" Score="5" ViewCount="137" Body="&lt;p&gt;For the purpose of testing phylogenetics software, we're using synthetic sequence data (i.e., it's computer generated, and there's no actual biological organism with that e.g. DNA sequence).  For real-world biological sequence data, we would say the sequence belongs to a &quot;species&quot; or &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Taxon&quot; rel=&quot;noreferrer&quot;&gt;taxon&lt;/a&gt;&quot; (or &quot;taxonomic group&quot;), but this feels inaccurate since there is no actual species or taxon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;: Can we describe synthetic biological sequence data as belonging to a &quot;species&quot; or &quot;taxon&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm wondering if this is standard practice, despite the synthetic nature of the data.&lt;/p&gt;&#xA;" OwnerUserId="1451" LastActivityDate="2017-09-08T12:27:29.890" Title="Can we describe synthetic biological sequence data as belonging to a &quot;species&quot; or &quot;taxon&quot;?" Tags="|terminology|phylogenetics|" AnswerCount="2" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2484" PostTypeId="2" ParentId="2483" CreationDate="2017-09-07T08:22:11.533" Score="2" Body="&lt;p&gt;When looking at genomic data for bacteria, taxa above the species level are operational taxonomic units (OTUs). OTUs are designated based on percent identity of the sequence (specifically the 16S rRNA gene sequences in most cases). The classification is usually based on rule of thumb, and some rough guidelines are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;97% for Genus&lt;/li&gt;&#xA;&lt;li&gt;94% for Family&lt;/li&gt;&#xA;&lt;li&gt;88% for Order&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Using OTUs is common practice. I don't see much of an ideological difference using OTUs with synthetic data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Konstantinidis, K. T., &amp;amp; Tiedje, J. M. (2005). Towards a Genome-Based Taxonomy for Prokaryotes. Journal of Bacteriology, 187(18), 6258–6264. &lt;a href=&quot;http://doi.org/10.1128/JB.187.18.6258-6264.2005&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://doi.org/10.1128/JB.187.18.6258-6264.2005&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2017-09-07T08:22:11.533" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2485" PostTypeId="2" ParentId="2480" CreationDate="2017-09-07T08:47:00.067" Score="3" Body="&lt;p&gt;Biopython has a parser for Blast's default text format (which I assume is the one you have). Just use the &lt;code&gt;blast-text&lt;/code&gt; string to specify this format in &lt;code&gt;SearchIO&lt;/code&gt;'s &lt;code&gt;parse&lt;/code&gt; method.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SearchIO&#xA;results = SearchIO.parse('myfile.txt', 'blast-text')&#xA;for entry in results:&#xA;    # extract the information you want&#xA;    query = entry.id&#xA;    for hit in entry.hits:&#xA;        subject = hit.id&#xA;        for hsp in hit.hsps:&#xA;            identity = hsp.ident_num *100 / entry.seq_len&#xA;            aln_len = hsp.aln.get_alignment_length()&#xA;            mismatches = aln_len - hsp.ident_num # not sure about this&#xA;            gaps = hsp.gap_num&#xA;            q_start = hsp.query_start + 1&#xA;            q_end = hsp.query_end&#xA;            s_start = hsp.hit_start + 1&#xA;            s_end = hsp.hit_end&#xA;            e_value = hsp.evalue&#xA;            bit_score = int(hsp.bitscore)&#xA;            # print in outfmt 6 format&#xA;            print('\t'.join([str(x) for x in [query,&#xA;                                      subject,&#xA;                                      identity,&#xA;                                      aln_len,&#xA;                                      mismatches,&#xA;                                      gaps,&#xA;                                      q_start,&#xA;                                      q_end,&#xA;                                      s_start,&#xA;                                      s_end,&#xA;                                      e_value,&#xA;                                      bit_score]]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Have a look at &lt;a href=&quot;http://biopython.org/DIST/docs/api/Bio.SearchIO.BlastIO-module.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biopython's docs&lt;/a&gt; for more information on how to parse Blast results.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2017-09-07T08:47:00.067" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2486" PostTypeId="1" AcceptedAnswerId="3177" CreationDate="2017-09-07T10:38:56.040" Score="7" ViewCount="641" Body="&lt;p&gt;How do I merge VCFs files containing CNVs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use &lt;a href=&quot;https://vcftools.github.io/perl_module.html#vcf-merge&quot; rel=&quot;nofollow noreferrer&quot;&gt;vcf-merge&lt;/a&gt;, a VCFtools function, and after bgzip and &lt;a href=&quot;http://www.htslib.org/doc/tabix.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tabix&lt;/a&gt;, SAMtools, to index and tab separate variants, but I don't know if it is the right way.&lt;/p&gt;&#xA;" OwnerUserId="1453" LastEditorUserId="73" LastEditDate="2018-01-14T22:53:43.873" LastActivityDate="2021-12-07T11:29:35.060" Title="VCF merge containing CNV" Tags="|vcf|cnv|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2487" PostTypeId="2" ParentId="2483" CreationDate="2017-09-07T12:59:16.677" Score="6" Body="&lt;p&gt;Phylogenies can be made with/of non-species groups. I think &quot;taxon&quot; is perfectly fine. You might also consider the use of the &quot;leaf / leaves&quot; or &quot;terminal&quot; if you want to emphasise the tip nature of the organisms. Even &quot;sequence&quot; would be fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ninja edit to my own suggestion: since you're making a phylogeny from sequences of unknown context or relationship (i.e. you can't say that they're sequences from a &quot;species&quot; or &quot;OTU&quot; or from anywhere), agnostic terms like &quot;taxon&quot; or &quot;sequence&quot; are more correct.&lt;/p&gt;&#xA;" OwnerUserId="377" LastEditorUserId="377" LastEditDate="2017-09-08T12:27:29.890" LastActivityDate="2017-09-08T12:27:29.890" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2488" PostTypeId="1" AcceptedAnswerId="2497" CreationDate="2017-09-07T13:46:01.060" Score="7" ViewCount="279" Body="&lt;p&gt;Is there a server/website somewhere where I can submit a list of DNA/RNA sequences and find the list of kmers hits and organisms where it's found? I checked the kraken website, but they don't have a webserver for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am specifically looking for kmers, as a complement of just doing BLAST on the NCBI nt database or similar. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: So far I have seen:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the Kraken app on Illumina BaseSpace, but it's not trivial to upload FASTQ data that hasn't been produced from an Illumina instrument.&lt;/li&gt;&#xA;&lt;li&gt;the CLARK trio of apps in InsideDNA, tried once with default parameters but produced empty results.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="180" LastEditorUserId="1813" LastEditDate="2023-04-18T10:11:02.303" LastActivityDate="2023-04-18T10:11:02.303" Title="Server for finding kmers in set of sequences" Tags="|k-mer|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2489" PostTypeId="2" ParentId="2465" CreationDate="2017-09-07T16:04:08.560" Score="0" Body="&lt;p&gt;Not with no other information, no. There is no way of looking at a list of accessions and guessing which one is primary. Primary accessions are just accessions like any other; they don't have any specific characteristics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is further complicated by the fact that each gene can (and usually does, for higher eukaryotes) have multiple transcripts, each of which can produce a different protein product. So in order to go from gene identifier to a protein, you need to first find the relevant transcript. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, as a first step, get the primary transcript IDs for each of your genes and then retrieve the protein accessions associated to them. &lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-09-07T16:11:13.207" LastActivityDate="2017-09-07T16:11:13.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2491" PostTypeId="2" ParentId="2467" CreationDate="2017-09-07T19:10:34.040" Score="3" Body="&lt;p&gt;You could make your own such file with &lt;a href=&quot;http://meme-suite.org/doc/fimo.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;FIMO&lt;/a&gt;, the &lt;a href=&quot;http://meme-suite.org/db/motifs&quot; rel=&quot;nofollow noreferrer&quot;&gt;JASPAR&lt;/a&gt; MEME files available via the &lt;a href=&quot;http://meme-suite.org/doc/download.html?man_type=web&quot; rel=&quot;nofollow noreferrer&quot;&gt;MEME download site&lt;/a&gt;, and your genome build of interest (per-chromosome FASTA files from UCSC goldenpath, say), e.g., for assembly &lt;code&gt;hg19&lt;/code&gt; and UCSC chromosome naming scheme:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ for chr in `seq 1 22` X Y; do echo $&lt;/span&gt;{chr}; wget -qO- http://hgdownload.cse.ucsc.edu/goldenpath/hg19/chromosomes/chr&lt;span class=&quot;math-container&quot;&gt;${chr}.fa.gz | gunzip -c - &amp;gt; hg19.chr$&lt;/span&gt;{chr}.fa; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For each chromosome (omitting the &lt;code&gt;for&lt;/code&gt; loop code for brevity), you can use the &lt;code&gt;fasta-get-markov&lt;/code&gt; in the MEME suite to generate a 5th-order Markov model of kmer frequencies (&lt;a href=&quot;https://biology.stackexchange.com/questions/41714/why-are-fifth-order-markov-models-the-ones-most-often-used-for-gene-prediction&quot;&gt;why 5th order?&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ fasta-get-markov -m 5 hg19.chr$&lt;/span&gt;{chr}.fa &amp;gt; hg19.chr${chr}.5th_order_background_model.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here is one way to download a &lt;a href=&quot;http://jaspar.genereg.net/downloads/&quot; rel=&quot;nofollow noreferrer&quot;&gt;non-redundant, vertebrate JASPAR database&lt;/a&gt; (MEME-formatted):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ wget http://jaspar.genereg.net/download/CORE/JASPAR2020_CORE_vertebrates_non-redundant_pfms_meme.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To run a FIMO query with the 5th-order background model and JASPAR database, using a p-value threshold of 1e-4:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ fimo --verbosity 1 --bgfile hg19.chr$&lt;/span&gt;{chr}.5th_order_background_model.txt --thresh 1e-4 --text JASPAR2018_CORE_vertebrates_non-redundant_pfms_meme.txt hg19.chr&lt;span class=&quot;math-container&quot;&gt;${chr}.fa \&#xA;  | tail -n+2 \&#xA;  | awk -vOFS=&quot;\t&quot; -vFS=&quot;\t&quot; '{ print $3, $4, $5, $1&quot;:&quot;$2&quot;&amp;gt;&quot;$10, $8, $6 }' \&#xA;  | sort-bed - \&#xA;  | starch --omit-signature - \&#xA;  &amp;gt; hg19.chr$&lt;/span&gt;{chr}.bed.starch&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output of a FIMO search is a text file which can be modified into a sorted BED file via &lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/file-management/sorting/sort-bed.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sort-bed&lt;/a&gt;, an example of which is shown above.&lt;/p&gt;&#xA;&lt;p&gt;To get a faster answer, each per-chromosome query can be put onto its own node on a computational cluster.&lt;/p&gt;&#xA;&lt;p&gt;Using a p-value threshold of 1e-4 will result in very large files (when uncompressed). Using tools to compress BED files is useful here, as is using a more stringent threshold (1e-5, etc.).&lt;/p&gt;&#xA;&lt;p&gt;Use of &lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/file-management/compression/starch.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS &lt;code&gt;starch&lt;/code&gt;&lt;/a&gt; to compress BED files (as shown above, for instance) will give better compression performance than &lt;code&gt;bgzip&lt;/code&gt; with per-chromosome random access and integration with BEDOPS &lt;code&gt;bedops&lt;/code&gt; and &lt;code&gt;bedmap&lt;/code&gt; CLI tools. The &lt;code&gt;bgzip&lt;/code&gt; option offers integration with &lt;code&gt;tabix&lt;/code&gt; and offers finer-grained random access, but it will use more disk space and doesn't integrate with BEDOPS tools.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2023-05-11T18:14:49.343" LastActivityDate="2023-05-11T18:14:49.343" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2492" PostTypeId="1" CreationDate="2017-09-08T03:26:17.487" Score="4" ViewCount="667" Body="&lt;p&gt;I have the nucleotide sequence:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;AATTGAGGCACATTTTTTTTTAGACAGTCTTGCTCTGTTGCCCAGGCTGGAGTGCAGTGGTGTGATCATAGCTCACTGCAGCCTCGACCTCCTGGGCTCAACAAAGCACACAGTGGGCGGATCCCCACCAG&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I blat this on UCSC Genome Browser, the first hit is a match which spans 6572 basepairs on chromosome 19, and essentially matches the first portion of the sequence to one spot on the genome, and the second portion to another part of the genome approximately 6.5kb downstream.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However when I use the BLAT command line tool, it doesn't find the second half of this match, it only matches the first portion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the flags I'm using:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;-t=dna&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;-q=dna&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;-minIdentity=90&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;-out=blast8&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I get the command line tool to mimic the behavior of BLAT on UCSC Genome Browser?&lt;/p&gt;&#xA;" OwnerUserId="1456" LastActivityDate="2017-09-08T07:35:46.933" Title="Using BLAT command line tool to blat split sequences" Tags="|human-genome|genomics|blat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2494" PostTypeId="2" ParentId="2492" CreationDate="2017-09-08T07:12:28.723" Score="5" Body="&lt;p&gt;&lt;a href=&quot;https://genome.ucsc.edu/FAQ/FAQblat.html#blat5&quot; rel=&quot;nofollow noreferrer&quot;&gt;The FAQ&lt;/a&gt; offers an answer:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;I'm setting up my own Blat server and would like to use the same parameter values that the UCSC web-based Blat server uses.&lt;/strong&gt; &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;We almost always expect there to be some small differences between the hgBlat/gfServer and the stand-alone command-line blat. The best matches can be found using pslReps and pslCDnaFilter utilities. The web-based blat is tuned permissively with a minimum cut-off score of 20, which will display most of the alignments. Other than to confirm that your command-line blat is working, there is little use in perfectly replicating the web-based blat results. We advise deciding which filtering parameters make the most sense for the experiment or analysis. Often these settings will be different and more stringent than those of the web-based blat. With that in mind, use the following settings to replicate the search results of the web-based blat:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;[ . . . ]&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;standalone blat:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;blat search:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blat -stepSize=5 -repMatch=2253 -minScore=0 -minIdentity=0 database.2bit query.fa output.psl &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="776" LastEditorUserId="298" LastEditDate="2017-09-08T07:35:46.933" LastActivityDate="2017-09-08T07:35:46.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2495" PostTypeId="2" ParentId="2367" CreationDate="2017-09-08T08:07:38.117" Score="0" Body="&lt;p&gt;There seems to be some confusion here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the first stage of the Canu pipeline (read correction), i think the parameter to use is &lt;strong&gt;rawErrorRate&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the docs: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;rawErrorRate&lt;/strong&gt;  The allowed difference in an overlap between two uncorrected reads, expressed as fraction error. Sets corOvlErrorRate and corErrorRate. The rawErrorRate typically does not need to be modified. It might need to be increased if very early reads are being assembled. The default is 0.300 For PacBio reads, and 0.500 for Nanopore reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that &lt;strong&gt;correctedErrorRate&lt;/strong&gt; is what you use at the next stages of the pipeline (trimming and assemble).&lt;/p&gt;&#xA;" OwnerUserId="1281" LastActivityDate="2017-09-08T08:07:38.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2496" PostTypeId="2" ParentId="2465" CreationDate="2017-09-08T08:12:31.687" Score="4" Body="&lt;p&gt;Reposting my answer from the &lt;a href=&quot;https://biology.stackexchange.com/questions/65595/how-to-determine-the-primary-uniprot-accession-number-from-a-list-of-accession-n/65606#65606%22related%20ticket%20in%20the%20Biology%20section%22&quot;&gt;related ticket in the Biology section&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think there is an issue with the terminology. The &quot;primary&quot; accession number, is the first accession number in cases where an entry has more than one accession number, as described in &lt;a href=&quot;http://www.uniprot.org/help/accession_numbers&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.uniprot.org/help/accession_numbers&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Entries can have more than one accession number. This can be due to&#xA;  two distinct mechanisms:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a) When two or more entries are merged, the accession numbers from all entries are kept. The first accession number is referred to as the&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;‘Primary (citable) accession number’, while the others are referred to&#xA;  as ‘Secondary accession numbers’. These are listed in alphanumerical&#xA;  order.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;b) If an existing entry is split into two or more entries (‘demerged’), new ‘primary’ accession numbers are attributed to all&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;the split entries while all original accession numbers are retained as&#xA;  ‘secondary’ accession numbers.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Example: P29358 which has been ‘demerged’ into P68250 and P68251.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Both reviewed and unreviewed entries can have primary accession numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you probably mean, as previous posters understood, are accession numbers of reviewed entries as opposed to unreviewed ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that case, you can indeed add &quot;reviewed:yes&quot; to your query, e.g. when you are using the UniProt ID mapping, &lt;a href=&quot;http://www.uniprot.org/help/uploadlists&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.uniprot.org/help/uploadlists&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1459" LastActivityDate="2017-09-08T08:12:31.687" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2497" PostTypeId="2" ParentId="2488" CreationDate="2017-09-08T12:06:56.250" Score="6" Body="&lt;p&gt;One Codex is powerful and user friendly, and works in a similar way to Kraken underneath&#xA;&lt;a href=&quot;https://www.onecodex.com/&quot; rel=&quot;noreferrer&quot;&gt;https://www.onecodex.com/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Kaiju web server might be of interest and is more sensitive than exclusively k-mer based methods&#xA;&lt;a href=&quot;http://kaiju.binf.ku.dk/server&quot; rel=&quot;noreferrer&quot;&gt;http://kaiju.binf.ku.dk/server&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The title of this question is misleading – it sounds like you're actually looking for a webserver for taxonomic classification using &lt;em&gt;k&lt;/em&gt;-mers.&lt;/p&gt;&#xA;" OwnerUserId="1258" LastEditorUserId="1258" LastEditDate="2017-09-08T16:18:02.587" LastActivityDate="2017-09-08T16:18:02.587" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2498" PostTypeId="1" AcceptedAnswerId="2511" CreationDate="2017-09-08T14:27:57.333" Score="6" ViewCount="294" Body="&lt;p&gt;When I run a GSEA analysis on two conditions from the same RNaseq (negative control PBS injection VS positive control CpG injection) from the same dataset/same gene list, I get results that look something like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lvm91.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lvm91.jpg&quot; alt=&quot;Example GSEA Image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice in my example that many gene sets are significantly enriched in PBS VS CpG, but none are very significantly enriched in the inverse, CpG VS PBS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a basic question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we're comparing two items, say A and B, shouldn't a up-regulated gene set that's enriched in A have a corresponding down-regulated gene set that's enriched in B?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I'm consistently confused about how to interpret the fact that one of my two conditions has many more enriched gene sets than the other.&lt;/strong&gt; What am I missing/misunderstanding?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1226" LastEditorUserId="1226" LastEditDate="2017-09-08T15:45:15.250" LastActivityDate="2017-09-15T07:04:13.467" Title="Clarification on Gene Enrichment" Tags="|rna-seq|gsea|go-enrichment|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2499" PostTypeId="1" AcceptedAnswerId="2500" CreationDate="2017-09-09T00:10:36.707" Score="5" ViewCount="167" Body="&lt;p&gt;I know that the GWAS association p-value threshold is 1e-8. This makes sense because in GWAS, you make ~1 million hypothesis (i.e. use that many SNPs in the association test). However, let's say I do not do genome-wide analysis but only use the 100 SNPs from a specific gene to check for association with my phenotype. Then, do you think a raw p-value of 1e-4 (which becomes 1e-2 after multiple hypothesis correction) is a reliable p-value to come to a conclusion that there is an association between that SNP and the checked phenotype? Obviously, if I checked all 1 million SNPs instead, this p-value would be insignificant after correction, but I am not checking 1 million SNPs, as I mentioned.&lt;/p&gt;&#xA;" OwnerUserId="1331" LastActivityDate="2017-09-09T04:01:29.850" Title="SNP-phenotype association analysis only using the SNPs on a specific gene" Tags="|gene|snp|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2500" PostTypeId="2" ParentId="2499" CreationDate="2017-09-09T04:01:29.850" Score="5" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I know that the GWAS association p-value threshold is 1e-8&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This may be a common threshold of statistical significance that is used, but it's definitely not an absolute value. It's a hack to try to work around many issues with GWAS associated with testing millions of SNPs. Unfortunately, the most relevant issue in GWAS (for spurious significance) is unexpected shared genetic structure in the cases or controls, and this cannot be excluded by a p-value threshold, regardless of how low it is set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your 100-SNP example, you are correct that the bonferroni correction would be a threshold of 1e-4 (for a desired significance threshold of 0.01), but it's important to be cautious about GWAS results, even when the p-value indicates otherwise. Trust no one, especially yourself. P-values should never be used as an indication of importance (which suggests that most GWAS results should probably be reconsidered), and are better used in conjunction with other evidence for identifying relevant / significant tests.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My recommendation for GWAS (or any similar bulk genotyping study, as in your 100-SNP subset) is that at least a bootstrap sub-sampling process is carried out, to make sure that observations found within the comparison of interest are at least well replicated when comparing smaller sub-groups of the cases and controls. More information on that can be found in &lt;a href=&quot;https://f1000research.com/posters/5-2190&quot; rel=&quot;noreferrer&quot;&gt;the poster&lt;/a&gt; I presented at Queenstown Research Week last year, and detail about the implementation in &lt;a href=&quot;http://www.biorxiv.org/content/early/2017/01/31/104497&quot; rel=&quot;noreferrer&quot;&gt;my draft preprint&lt;/a&gt; (which will probably never be properly published, given how old the research is).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-09-09T04:01:29.850" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2502" PostTypeId="2" ParentId="2431" CreationDate="2017-09-09T14:52:58.333" Score="5" Body="&lt;p&gt;Another option I'm trying at the moment is the combination of &lt;a href=&quot;https://snakemake.readthedocs.io&quot; rel=&quot;nofollow noreferrer&quot;&gt;snakemake&lt;/a&gt;, &lt;a href=&quot;https://conda.io&quot; rel=&quot;nofollow noreferrer&quot;&gt;conda.io&lt;/a&gt; and &lt;a href=&quot;https://yihui.name/knitr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;knitr&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Conda&lt;/strong&gt; is a &quot;Package, dependency and environment management for any language&quot; and could be used to set up the &lt;strong&gt;R-environment&lt;/strong&gt; as well as other software. Note, most bioinformatics software can be found in the related &lt;strong&gt;bioconda&lt;/strong&gt; project. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Snakemake&lt;/strong&gt; is a  workflow management system, which can be used to automate the preprocessing of your data in a reproducible manner (also it works nicely with conda since a conda environments can be specified in the Snake-Makefile) &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Knitr for dynamic report generation with R. Well suited to document and reproduce your R workflow into reports, ticking your &lt;strong&gt;code&lt;/strong&gt; and seed &lt;strong&gt;boxes&lt;/strong&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1461" LastEditorUserId="1461" LastEditDate="2017-09-11T15:44:39.477" LastActivityDate="2017-09-11T15:44:39.477" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2503" PostTypeId="1" AcceptedAnswerId="2505" CreationDate="2017-09-09T20:07:28.273" Score="7" ViewCount="7619" Body="&lt;p&gt;Is there a way for me to get a list of genes given a list of SNP rs ids? I found several questions asked with a similar goal years ago, and the answers are always about using multiple online tools, and/or R programming. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if there is a simpler solution recently? (and hopefully, I can solve it within the scope of python. )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The relevant question I found (that are asked and answered more than five years ago.)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://support.bioconductor.org/p/39278/&quot; rel=&quot;noreferrer&quot;&gt;https://support.bioconductor.org/p/39278/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.biostars.org/p/10437/&quot; rel=&quot;noreferrer&quot;&gt;https://www.biostars.org/p/10437/&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.biostars.org/p/413/&quot; rel=&quot;noreferrer&quot;&gt;https://www.biostars.org/p/413/&lt;/a&gt; &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1082" LastActivityDate="2020-07-20T16:38:44.867" Title="How to get a list of genes corresponding to the list of SNPs (rs ids)?" Tags="|database|gene|snp|" AnswerCount="6" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2504" PostTypeId="2" ParentId="2503" CreationDate="2017-09-10T08:34:51.797" Score="7" Body="&lt;p&gt;I can show you a simple way in R, using biomaRt. Let's say you have two snp ids you want to exmine.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SNPids &amp;lt;- c(&quot;rs431905509&quot;, &quot;rs431905511&quot;)&#xA;&#xA;library(&quot;biomaRt&quot;)&#xA;&#xA;# To show which marts are available&#xA;listMarts()&#xA;&#xA;# You need the SNP mart&#xA;mart &amp;lt;- useMart(&quot;ENSEMBL_MART_SNP&quot;)&#xA;&#xA;# Find homo sapiens&#xA;listDatasets(mart)&#xA;&#xA;# This will be the dataset we want to use&#xA;dataset &amp;lt;- useDataset(&quot;hsapiens_snp&quot;, mart=mart)&#xA;&#xA;# Show available filters&#xA;listFilters(dataset)&#xA;&#xA;# Now list all available attributes&#xA;listAttributes(dataset)&#xA;&#xA;# To get the ensembl gene id belonging to the SNPs&#xA;getBM(attributes=c('refsnp_id', 'ensembl_gene_stable_id'), &#xA;      filters = 'snp_filter', &#xA;      values = SNPids, &#xA;      mart = dataset)&#xA;&#xA;    refsnp_id ensembl_gene_stable_id&#xA;1 rs431905509        ENSG00000070371&#xA;2 rs431905509        ENSG00000100075&#xA;3 rs431905509        ENSG00000260924&#xA;4 rs431905511        ENSG00000145335&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2017-09-10T09:43:53.693" LastActivityDate="2017-09-10T09:43:53.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2505" PostTypeId="2" ParentId="2503" CreationDate="2017-09-10T14:38:44.270" Score="8" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I wonder if there is a simpler solution recently? (and hopefully, I can solve it within the scope of python. )&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A &lt;em&gt;simpler&lt;/em&gt; solution, I don't know... but this is at least &lt;em&gt;one&lt;/em&gt; Python solution using &lt;a href=&quot;http://biopython.org/DIST/docs/tutorial/Tutorial.html#htoc118&quot; rel=&quot;noreferrer&quot;&gt;Biopython's ELink method&lt;/a&gt; via &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK25499/&quot; rel=&quot;noreferrer&quot;&gt;NCBI's Entrez E-utils&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Biopython library is flexible enough (they have an in-depth tutorial worth reading) to modify the code below to to fit project-specific needs.  For example, the mapping between snp id and gene id is lost with this bulk request method.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez&#xA;Entrez.email = &quot;youremail@gmail.com&quot;&#xA;&#xA;snp_ids = [&quot;rs8179065&quot;, &#xA;           &quot;rs12899845&quot;, &#xA;           &quot;rs8179066&quot;, &#xA;           &quot;rs80357332&quot;, &#xA;           &quot;rs80356920&quot;]&#xA;&#xA;record = Entrez.read(Entrez.elink(dbfrom=&quot;snp&quot;, &#xA;                                  id=&quot;,&quot;.join(snp_ids).replace('rs', ''), &#xA;                                  db=&quot;gene&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This returns a &lt;code&gt;record&lt;/code&gt; object&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[{'LinkSetDb': [{'DbTo': 'gene', 'LinkName': 'snp_gene', 'Link': [{'Id': '374654'}, {'Id': '90381'}, {'Id': '672'}]}], 'ERROR': [], 'LinkSetDbHistory': [], 'DbFrom': 'snp', 'IdList': ['8179065', '8179066', '12899845', '80356920', '80357332']}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which contains several iterable fields -- 'Link' has a list of the Gene &lt;strong&gt;UID&lt;/strong&gt;'s that can be converted to names using &lt;code&gt;ESummary&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for gene_id in record[0]['LinkSetDb'][0]['Link']:&#xA;    handle = Entrez.esummary(db=&quot;gene&quot;, id=gene_id['Id'])&#xA;    uid_record = Entrez.read(handle)&#xA;    handle.close()&#xA;&#xA;    uid_summary = uid_record[&quot;DocumentSummarySet&quot;]['DocumentSummary'][0]&#xA;    print (gene_id['Id'], uid_summary['Name'])&#xA;    # ...or something more interesting than print&#xA;&#xA;&#xA;374654 KIF7&#xA;90381 TICRR&#xA;672 BRCA1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;An option to preserve discrete mapping, is to loop over each snp id and extract the gene(s):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;although I don't endorse the method below, especially for long lists of snps, you may flood the server with too many requests, maybe someone more familiar can comment on request limits or best practices for Entrez&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for snp_id in snp_ids:&#xA;    record = Entrez.read(Entrez.elink(dbfrom=&quot;snp&quot;, &#xA;                                  id=snp_id.replace('rs',''), &#xA;                                  db=&quot;gene&quot;))&#xA;    results = record[0]['LinkSetDb'][0]['Link']&#xA;    for result in results:&#xA;        uid = result['Id']&#xA;        handle = Entrez.esummary(db=&quot;gene&quot;, id=uid)&#xA;        uid_record = Entrez.read(handle)&#xA;        handle.close()&#xA;        uid_summary = uid_record[&quot;DocumentSummarySet&quot;]['DocumentSummary'][0]&#xA;        gene_name = uid_summary['Name']&#xA;&#xA;        print(snp_id, uid, gene_name)&#xA;&#xA;rs8179065 374654 KIF7&#xA;rs12899845 374654 KIF7&#xA;rs12899845 90381 TICRR&#xA;rs8179066 374654 KIF7&#xA;rs80357332 672 BRCA1&#xA;rs80356920 672 BRCA1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="84" LastActivityDate="2017-09-10T14:38:44.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2506" PostTypeId="2" ParentId="2241" CreationDate="2017-09-10T18:01:32.327" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there a &quot;consensus&quot; database containing protein expression profiles for different tissue types in human?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ebi.ac.uk/gxa/home&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/gxa/home&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If not, what is the best way to filter out the secreted and&#xA;  transmembrane proteins by gene symbol?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Personally I like to run a local copy of SignalP, as it will yield consistent results. However bioinformatic approaches tend not to capture highly unusual transmembrane proteins or membrane associations (e.g.: only one leaflet etc.). &lt;a href=&quot;http://www.cbs.dtu.dk/services/SignalP/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cbs.dtu.dk/services/SignalP/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the organisms, and scope of your analysis, further possibility could be to visit supplemental tables of recent publications containing somewhat supervised computational predictions on secretion, e.g.: &lt;a href=&quot;http://science.sciencemag.org/content/356/6340/eaal3321&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://science.sciencemag.org/content/356/6340/eaal3321&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I am looking the secretome profile and the membrane receptor profile for a given cell type.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The above links don't inform about cell-type specific retention of potentially secreted proteins. - but their combination will yield cell-type specific RNA and protein levels of transmembrane and secreted proteins.&lt;/p&gt;&#xA;" OwnerUserId="357" LastActivityDate="2017-09-10T18:01:32.327" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2507" PostTypeId="1" CreationDate="2017-09-10T19:37:07.150" Score="3" ViewCount="76" Body="&lt;p&gt;&lt;a href=&quot;https://www.medicaljournals.se/jrm/content/html/10.2340/16501977-2210&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.medicaljournals.se/jrm/content/html/10.2340/16501977-2210&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand the difference between statistical significance and effect size, but I am having trouble interpreting the data on Table III in the article linked above. I am evaluating the WOMAC scores. My first question is can effect size confidence intervals cross zero and still be valid? My second question is about the negative and positive values of the Hedges'g. The first study scores the WOMAC in a way where the higher the score, the better the outcome. The second two studies score the WOMAC in a way where the lower the score, the better the outcome. Does this mean that in the first study a positive Hedges'g indicate larger effect for the treatment group and a negative Hedges' g for the other two studies indicate larger effect for the treatment group?&lt;/p&gt;&#xA;" OwnerUserId="1462" LastActivityDate="2020-03-11T08:18:24.903" Title="What is the significance of the effect size on WOMAC scores for TKA?" Tags="|statistics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2508" PostTypeId="2" ParentId="2503" CreationDate="2017-09-10T21:57:31.637" Score="2" Body="&lt;p&gt;If you want a fast (binary) search with a locally-stored dataset, where you know exactly what SNPs and gene annotations are being queried, here's a way to set up files to do your own querying, which should scale up nicely if you have a lot of SNPs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Get annotations of interest and write them into a sorted BED file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, here's a way to get Gencode v26 gene annotations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ wget -qO- ftp://ftp.sanger.ac.uk/pub/gencode/Gencode_human/release_26/gencode.v26.basic.annotation.gff3.gz \&#xA;    | gunzip -c \&#xA;    | convert2bed --input=gff - \&#xA;    &amp;gt; gencode.v26.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;2) Get SNPs and write them into a text file sorted by SNP ID. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;code&gt;hg38&lt;/code&gt;, for instance:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ LC_ALL=C&#xA;$ wget -qO- http://hgdownload.cse.ucsc.edu/goldenpath/hg38/database/snp147.txt.gz \&#xA;   | gunzip -c \&#xA;   | awk -v OFS=&quot;\t&quot; '{ print $5,$2,$3,($3+1) }' \&#xA;   | sort -k1,1 \&#xA;   &amp;gt; hg38.snp147.sortedByName.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;3) Install &lt;a href=&quot;https://github.com/pts/pts-line-bisect&quot; rel=&quot;nofollow noreferrer&quot;&gt;pts-line-bisect&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ git clone https://github.com/pts/pts-line-bisect.git&#xA;$ cd pts-line-bisect &amp;amp;&amp;amp; make &amp;amp;&amp;amp; cd ..&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;4) Run a query:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ rs_of_interest=rs2814778&#xA;$ ./pts-line-bisect/pts_lbsearch -p hg38.snp147.sortedByName.txt ${rs_of_interest} \&#xA;   | head -1 \&#xA;   | cut -f2- \&#xA;   | bedmap --echo --echo-map-id-uniq --range 1000 - gencode.v26.bed \&#xA;   &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The file &lt;code&gt;answer.bed&lt;/code&gt; contains the genomic interval of the SNP of interest, along with the IDs of any Gencode v26 gene annotations within 1000 nt upstream and downstream of the SNP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To repeat step 4 within a Python script, you can use &lt;code&gt;subprocess.check_output()&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;pts_lbsearch = os.path.join(project_dir, &quot;pts-line-bisect&quot;, &quot;pts_lbsearch&quot;)&#xA;if not os.path.exists(pts_lbsearch):&#xA;  print &quot;Content-type:application/json\r\n\r\n&quot;&#xA;  print json.dumps({'error' : 'could not locate pts_lbsearch [%s]' % (pts_lbsearch)})&#xA;  sys.exit(errno.ENOENT)&#xA;&#xA;if term.startswith('rs'):&#xA;  query_fn = os.path.join(annotations_dir, &quot;hg38.snp147.sortedByName.txt&quot;)&#xA;  cmd = &quot;%s -p %s %s&quot; % (pts_lbsearch, query_fn, term)&#xA;  try:&#xA;    res = subprocess.check_output(cmd, shell=True)&#xA;    if res and len(res) &amp;gt; 0:&#xA;      (chr, start, stop) = res.strip().split('\n')[0].split('\t')&#xA;    else:&#xA;      pass&#xA;  except subprocess.CalledProcessError as err:&#xA;    sys.stderr.write(&quot;Query failed!\n&quot;)&#xA;    sys.exit(errno.EIO)&#xA;&#xA;#&#xA;# map 'chr', 'start', and 'stop' variables to annotations with &#xA;# another subprocess call (not shown -- use example above)&#xA;#&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2017-09-10T22:08:41.117" LastActivityDate="2017-09-10T22:08:41.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2509" PostTypeId="2" ParentId="2503" CreationDate="2017-09-11T08:50:28.680" Score="0" Body="&lt;p&gt;using ncbi-efetch and &lt;strong&gt;XSLT&lt;/strong&gt; with the following stylesheet:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;?xml version='1.0' encoding=&quot;UTF-8&quot;?&amp;gt;&#xA;&amp;lt;xsl:stylesheet&#xA;    xmlns:xsl='http://www.w3.org/1999/XSL/Transform'&#xA;    xmlns:r=&quot;https://www.ncbi.nlm.nih.gov/SNP/docsum&quot;&#xA;    version='1.1'&#xA;    &amp;gt;&#xA;&amp;lt;xsl:output method=&quot;text&quot;  encoding=&quot;UTF-8&quot;/&amp;gt;&#xA;&#xA;&amp;lt;xsl:template match=&quot;/&quot;&amp;gt;&#xA;&amp;lt;xsl:apply-templates select=&quot;r:ExchangeSet&quot;/&amp;gt;&#xA;&amp;lt;/xsl:template&amp;gt;&#xA;&amp;lt;xsl:template match=&quot;r:ExchangeSet&quot;&amp;gt;&#xA;&amp;lt;xsl:apply-templates select=&quot;r:Rs&quot;/&amp;gt;&#xA;&amp;lt;/xsl:template&amp;gt;&#xA;&amp;lt;xsl:template match=&quot;r:Rs&quot;&amp;gt;&#xA;&amp;lt;xsl:apply-templates select=&quot;r:Assembly/r:Component/r:MapLoc/r:FxnSet&quot;/&amp;gt;&#xA;&amp;lt;/xsl:template&amp;gt;&#xA;&#xA;&#xA;&amp;lt;xsl:template match=&quot;r:FxnSet&quot;&amp;gt;&#xA;&amp;lt;xsl:text&amp;gt;rs&amp;lt;/xsl:text&amp;gt;&#xA;&amp;lt;xsl:value-of select=&quot;../../../../@rsId&quot;/&amp;gt;&#xA;&amp;lt;xsl:text&amp;gt;  ncbi:&amp;lt;/xsl:text&amp;gt;&#xA;&amp;lt;xsl:value-of select=&quot;@geneId&quot;/&amp;gt;&#xA;&amp;lt;xsl:text&amp;gt;  &amp;lt;/xsl:text&amp;gt;&#xA;&amp;lt;xsl:value-of select=&quot;@symbol&quot;/&amp;gt;&#xA;&amp;lt;xsl:text&amp;gt;&#xA;&amp;lt;/xsl:text&amp;gt;&#xA;&amp;lt;/xsl:template&amp;gt;&#xA;&#xA;&amp;lt;/xsl:stylesheet&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;usage (group the rs# by 3):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ echo -e &quot;rs8179065\nrs12899845\nrs8179066\nrs80357332\nrs80356920\n&quot; | xargs  -L 3 echo  | tr &quot; &quot; &quot;,&quot; | while read R; do curl -sL &quot;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=snp&amp;amp;id=${R}&amp;amp;retmode=xml&quot; | xsltproc --novalid  transform.xsl -; done | uniq&#xA;rs8179065   ncbi:374654 KIF7&#xA;rs12899845  ncbi:90381  TICRR&#xA;rs8179066   ncbi:374654 KIF7&#xA;rs80357332  ncbi:672    BRCA1&#xA;rs80356920  ncbi:672    BRCA1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastEditorUserId="71" LastEditDate="2017-09-11T09:03:01.633" LastActivityDate="2017-09-11T09:03:01.633" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2510" PostTypeId="2" ParentId="2498" CreationDate="2017-09-11T15:55:23.543" Score="5" Body="&lt;p&gt;While I'm not familiar with GSEA software in particular, I believe your problem is that it only tests for &lt;em&gt;upregulated&lt;/em&gt; gene sets. Notice:  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;866/4408 gene sets are upregulated   &lt;/li&gt;&#xA;  &lt;li&gt;3542/4408 gene sets are upregulated  &lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;3542+866 = 4408. I.e., 866 sets have higher mean expression in positive condition, the rest have higher mean expression in negative condition. To avoid this confusion, I personally would reserve using the terms &quot;up/down-regulated&quot; only for sets where this change is at least remotely significant.&lt;/p&gt;&#xA;" OwnerUserId="920" LastActivityDate="2017-09-11T15:55:23.543" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2511" PostTypeId="2" ParentId="2498" CreationDate="2017-09-12T07:08:37.687" Score="8" Body="&lt;p&gt;If you compare A vs B the genes's fold change will have the opposite sign to B vs A. So will be the gene set up or down-regulated depending on the comparison to take.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The gene set test analyze if a given group of elements is sorted in a certain way in a list (I am talking about the GSE like the one you performed or the one in the Broad Institute). Usually it is used to say that gene set X is up-regulated in the comparison A vs B, that is, gene set X (in general) is more expressed in A than in B. The enrichment is measured in a enrichment score (or a normalized enrichment score, the higher, the clearer trend/distribution group X has. To asses X is really differentially expressed a p-value is also calculated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite that the number of the up-regulated gene sets sum up all the gene sets tested as pointed by &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2510/48&quot;&gt;juod&lt;/a&gt;, the number of gene sets with a a p-value below 5% and 1% is not the same in both comparisons which indicates that you are not really using the same samples in both gene sets. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check how you performed your comparison. Are the genes in each comparison with the same value and opposite sign?&lt;br&gt;&#xA;Check also the software used to run the GSEA analysis (if I recongize the type of report is from a package in Bioconductor [I don't recognize which one now]) as you (or me) might have misunderstood something or there is an error if the comparisons are done properly. For instance it could be that the report of p-values below the threshold would be from only the up-regulated gene sets. &lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-09-15T07:04:13.467" LastActivityDate="2017-09-15T07:04:13.467" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2512" PostTypeId="2" ParentId="2431" CreationDate="2017-09-12T09:12:10.283" Score="1" Body="&lt;p&gt;&lt;em&gt;checkpoint&lt;/em&gt; is a system for ensuring you get packages of the same time / version. However, I confess that I haven't used it and thus can't vouch for it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My feeling is that the rest will have to be handled outside this: code and data can be versioned in git (and dat). A whole solution might have use something like docker as others have suggested.&lt;/p&gt;&#xA;" OwnerUserId="377" LastActivityDate="2017-09-12T09:12:10.283" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2513" PostTypeId="2" ParentId="2269" CreationDate="2017-09-12T09:13:45.453" Score="1" Body="&lt;p&gt;What is typically done in methylation analysis is to assess the islands of methylations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check &lt;a href=&quot;https://www.bioconductor.org/help/workflows/methylationArrayAnalysis/#customising-visualisations-of-methylation-data&quot; rel=&quot;nofollow noreferrer&quot;&gt;this workflow&lt;/a&gt;, in the section linked it uses some predefined islands for instance. I am no expert on this area but you could asses if the islands or certain regions are more methylated than expected.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2017-09-12T09:13:45.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2514" PostTypeId="1" CreationDate="2017-09-13T11:26:55.327" Score="5" ViewCount="907" Body="&lt;p&gt;I know that Voom function from &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/limma.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;limma&lt;/a&gt; package from Bioconductor converts raw counts into log-CPM values and then Normalization is applied on that, with normalize.method argument.&lt;/p&gt;&#xA;&lt;p&gt;I would like to know clearly for RNA-Seq data:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;how this normalization is done (statistically) ?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;normalization across samples and normalization across genes. But don't the know how it is done statistically.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1377" LastEditorUserId="73" LastEditDate="2021-06-26T11:06:59.810" LastActivityDate="2021-06-26T11:06:59.810" Title="Voom function from limma package and Normalization on counts data" Tags="|r|bioconductor|normalization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2515" PostTypeId="1" AcceptedAnswerId="2532" CreationDate="2017-09-13T22:45:59.833" Score="5" ViewCount="720" Body="&lt;p&gt;I'm currently adding a few SNPs randomly into a FASTA within python using BioPython. In the following example from BioPython, I add an SNV at location &quot;5&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://biopython.org/DIST/docs/api/Bio.Seq.MutableSeq-class.html&quot; rel=&quot;noreferrer&quot;&gt;http://biopython.org/DIST/docs/api/Bio.Seq.MutableSeq-class.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.Seq import MutableSeq&#xA;from Bio.Alphabet import generic_dna&#xA;my_seq = MutableSeq(&quot;ACTCGTCGTCG&quot;, generic_dna)&#xA;my_seq[5] = &quot;A&quot;&#xA;print(my_seq)&#xA;## outputs ACTCGACGTCG &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In my case, I'm using the reference hg38 as the input FASTA to manipulate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After randomly inserting these SNPs, I would like to find a region whereby there are no SNPs within (let's say) +/- 50Kb, or the next nearest thing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to align the FASTA with variants against the reference within python? Otherwise, is there a quick way to check the variant FASTA against the reference FASTA, finding a region with no variants? What would be an efficient way to execute this task? &lt;/p&gt;&#xA;" OwnerUserId="146" LastActivityDate="2017-09-19T20:31:34.907" Title="Is it possible to do alignment within Python? Check variants against reference?" Tags="|sequence-alignment|python|reference-genome|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2516" PostTypeId="1" AcceptedAnswerId="2518" CreationDate="2017-09-14T09:24:26.730" Score="5" ViewCount="186" Body="&lt;p&gt;In my snakemake workflow, I use deeptools &lt;code&gt;bamCoverage&lt;/code&gt; (in a &lt;code&gt;make_bigwig&lt;/code&gt; rule), then &lt;code&gt;computeMatrix&lt;/code&gt; and &lt;code&gt;plotProfile&lt;/code&gt; (in different &lt;code&gt;plot_*_profile&lt;/code&gt; rules) to create coverage profiles for various kinds of small RNAs. It sometimes happens that there are very few reads in a particular category of small RNAs. This results in &lt;code&gt;bamCoverage&lt;/code&gt; exiting with an error (when I use the &lt;code&gt;--skipNAs&lt;/code&gt; option, in order to make coverage profiles not take into account empty regions):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error: The generated bedGraphFile was empty. Please adjust&#xA;your deepTools settings and check your input files.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The fact that &lt;code&gt;bamCoverage&lt;/code&gt; fails make things a little annoying for me, because the later the failure, the easier to handle things in my workflow: I will need to catch the error and generate a dummy output file in the &lt;code&gt;make_bigwig&lt;/code&gt; rule, and then handle the special dummy file in the &lt;code&gt;plot_*_profile&lt;/code&gt; rules.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be fine having an empty coverage file: the absence of reads (either in the bam file, or matching selection criteria in &lt;code&gt;bamCoverage&lt;/code&gt;) is not due to an error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is this just not possible, by the format definition?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Did I miss an option to have &lt;code&gt;bamCoverage&lt;/code&gt; accept making empty files?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Would this make &lt;code&gt;computeMatrix&lt;/code&gt; fail?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2017-09-15T07:43:52.817" Title="Dealing with absence of coverage" Tags="|snakemake|deeptools|workflow-management|errors|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2518" PostTypeId="2" ParentId="2516" CreationDate="2017-09-14T10:23:54.277" Score="3" Body="&lt;p&gt;I'll fix the &lt;code&gt;bamCoverage&lt;/code&gt; issue in the 2.6.0 release (it may already be fixed since I rewrite how bigWig files are made for that release). You can follow my status &lt;a href=&quot;https://github.com/fidelram/deepTools/issues/598&quot; rel=&quot;nofollow noreferrer&quot;&gt;on the issue page&lt;/a&gt;. In principle you should be able to do this, it's just a deepTools limitation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This should work fine with &lt;code&gt;computeMatrix&lt;/code&gt;, though keep in mind that you're just going to get a big block of NA values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;: This error ends up coming from pyBigWig. I've just found that &lt;code&gt;bedGraphToBigWig&lt;/code&gt; also can't make a bigWig file with now entries, so I suspect this is sort of undefined behavior. Regardless, I'll alter pyBigWig to produce a file that it can open and use in a meaningful manner.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-09-15T07:43:52.817" LastActivityDate="2017-09-15T07:43:52.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2519" PostTypeId="1" CreationDate="2017-09-14T13:07:53.857" Score="5" ViewCount="539" Body="&lt;p&gt;I'm analyzing a tumor sample and a healthy sample from the same patient. I want to use sciClone to look at tumor clonality. One input is the genomic regions that need to be excluded due to LOH. I have both WES and WGS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there currently a &quot;gold standard&quot; for which tool will return the most robust regions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sciClone paper mentions VarScan2, APOLLOH, HMMScan? Are these still state of the art?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it feasible to calculate LOH programmatically without the need of external software?&lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="298" LastEditDate="2017-09-15T07:40:29.007" LastActivityDate="2021-10-09T08:05:45.627" Title="How can I calculate loss of heterozygosity (LOH) in NGS sequencing data?" Tags="|cancer|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2520" PostTypeId="1" AcceptedAnswerId="3269" CreationDate="2017-09-14T13:24:44.840" Score="1" ViewCount="193" Body="&lt;p&gt;For calculating heteroplasmy in mtDNA I was using MitoSeek, the tool provides position wise heteroplasmy percentage. So, will it be wise to add up all sites to get a position-wise percentage in a single sample, or calculate it by adding up all counts (variants) from an aligned file and divide it by total read count?&lt;/p&gt;&#xA;" OwnerUserId="206" LastEditorUserId="73" LastEditDate="2018-01-10T06:12:40.003" LastActivityDate="2018-02-09T06:29:52.287" Title="Heteroplasmy Calculation" Tags="|mtdna|heteroplasmy|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2521" PostTypeId="2" ParentId="2194" CreationDate="2017-09-15T14:10:56.140" Score="4" Body="&lt;p&gt;One thing that hasn't been suggested so far would be phylogenetic placement. First option there would be placing against one larger tree roughly touching the domains you mentioned to get an overview. Second option would be having three trees (bacteria, plants, viruses) made up of relevant taxa for the type of sample you have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would give you a detailed and accurate view of your sample, but it does involve some work. Sadly the computational throughput for placement is still a bit behind BLAST, and not many complete pipelines exist, but in my opinion it is worth the extra mile (I might be biased. I'm working to resolve as many of these issues as possible).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to look into it, I can reccomend &lt;a href=&quot;https://academic.oup.com/sysbio/article/64/1/e26/2847641&quot; rel=&quot;nofollow noreferrer&quot;&gt;this excellent paper by Erick Matsen&lt;/a&gt;. Since your samples sound vaguely soil-ish, I will also link &lt;a href=&quot;https://www.nature.com/articles/s41559-017-0091&quot; rel=&quot;nofollow noreferrer&quot;&gt;this recent Nature paper&lt;/a&gt; using placement to analyze rainforest soil samples.&lt;/p&gt;&#xA;" OwnerUserId="204" LastActivityDate="2017-09-15T14:10:56.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2522" PostTypeId="1" AcceptedAnswerId="2569" CreationDate="2017-09-15T15:33:14.073" Score="5" ViewCount="259" Body="&lt;p&gt;I would like to create a density/histogram of the distribution of a particular DNA sequence over the entire transcript using R and/or command line tools. From here, I would like to use the coordinates of the bins to map the intron-exon diagram below the plot. In this way, I can tell the distribution of various DNA sequences across the entire transcript visually and see differences among alternatively spliced isoforms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already found the number of occurrences of the sequences I am interested in using the &lt;code&gt;seqinr&lt;/code&gt; and &lt;code&gt;biostrings&lt;/code&gt; packages. I have been using &lt;code&gt;biomaRt&lt;/code&gt; to get the exon start and stop sequences as well as the full gene sequence. Between all this, I feel I have all the tools I need but I am missing how to put it all together. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue I am having is that &lt;code&gt;biomaRt&lt;/code&gt; returns multiple exon sequences with identical start positions but different end positions for a total of 37 exons sequences. Meanwhile, &lt;a href=&quot;http://genatlas.medecine.univ-paris5.fr/fiche.php?symbol=grin1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genatlas&lt;/a&gt; and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/IEB/Research/Acembly/av.cgi?db=mouse&amp;amp;c=Gene&amp;amp;l=Grin1&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI&lt;/a&gt; are returning anywhere from 19 to 21 exons for Grin1, my gene of interest. How can I put all of these tools together to get the plot I need? Is there a better approach?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if there is an R package or command line tool that already has this function built in, that would be ideal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example code for one transcript. This code is actually from an earlier project where I used multiple transcripts but it works fine with one as well. I masked the sequences I'm searching for as &quot;XXXX&quot; at the request of my project manager.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(magrittr)&#xA;library(biomaRt)&#xA;library(org.Mm.eg.db)&#xA;library(biostrings)&#xA;library(seqinr)&#xA;&#xA;### Select only the longest sequences&#xA;getMaxLengthSequences &amp;lt;- function(x) {&#xA;  seqMax &amp;lt;- split(x, factor(x$mgi_symbol))&#xA;  seqMax &amp;lt;- lapply(seqMax, function(x) max(x$length)) %&amp;gt;% &#xA;    unlist() %&amp;gt;% &#xA;    as.data.frame()&#xA;  seqMax$mgi_symbol &amp;lt;- row.names(seqMax)&#xA;  row.names(seqMax) &amp;lt;- NULL&#xA;  names(seqMax) &amp;lt;- c(&quot;length&quot;, &quot;mgi_symbol&quot;)&#xA;  seqUnique &amp;lt;- merge(seqMax, x)&#xA;  seqUnique &amp;lt;- seqUnique[order(seqUnique$mgi_symbol), ]&#xA;&#xA;  return(seqUnique)&#xA;}&#xA;&#xA;### Get sequence for gene of interest&#xA;getSequences &amp;lt;- function(genelist, seqType) {&#xA;  ensembl &amp;lt;- useMart(&quot;ensembl&quot;, dataset = &quot;mmusculus_gene_ensembl&quot;)&#xA;&#xA;  sequ &amp;lt;- biomaRt::getSequence(id = genelist,&#xA;                               type = &quot;mgi_symbol&quot;,&#xA;                               seqType = seqType,&#xA;                               mart = ensembl)&#xA;&#xA;  ### Remove unavailable sequences&#xA;  sequ &amp;lt;- sequ[!grepl(&quot;Sequence unavailable&quot;, sequ[ ,seqType], fixed = TRUE), ]&#xA;&#xA;  ### Find the length of each sequence then pick the max length sequence if there are multiple sequences returned from BioMart&#xA;  sequ &amp;lt;- sequ[order(sequ$mgi_symbol), ]&#xA;  sequ$length &amp;lt;- sapply(sequ[ ,seqType], nchar)&#xA;  sequ &amp;lt;- getMaxLengthSequences(sequ)&#xA;&#xA;  return(sequ)&#xA;}&#xA;&#xA;### Get motif counts of interest (substrings are masked)&#xA;getmotifs &amp;lt;- function(x) {&#xA;&#xA;  seq &amp;lt;- x %&amp;gt;% &#xA;    tolower() %&amp;gt;% &#xA;    s2c()&#xA;&#xA;  output &amp;lt;- c(&quot;XXXX&quot; = count(seq, 4)[c(&quot;xxxx&quot;, &quot;xxxx&quot;)] %&amp;gt;% sum(),&#xA;              &quot;XXXX&quot; = count(seq, 4)[c(&quot;xxxx&quot;, &quot;xxxx&quot;)] %&amp;gt;% sum(),&#xA;              &quot;XXX&quot; = count(seq, 3)[&quot;xxx&quot;] %&amp;gt;% sum())&#xA;&#xA;  return(output)&#xA;}&#xA;&#xA;### Using the above functions: Get sequence and then find motifs&#xA;seq &amp;lt;- getSequences(&quot;Grin1&quot;)&#xA;targetMotifs &amp;lt;- matrix(sapply(seq[ ,&quot;coding&quot;], getmotifs), nrow = nrow(targets), ncol = 3, byrow = TRUE)&#xA;&#xA;targetMotifTable &amp;lt;- data.frame(Length = seq$length)&#xA;rownames(targetMotifTable) &amp;lt;- make.names(seq$mgi_symbol, unique = TRUE)&#xA;targetMotifTable &amp;lt;- cbind(targetMotifTable, targetMotifs) %&amp;gt;% as.data.frame()&#xA;names(targetMotifTable) &amp;lt;- c(&quot;Length&quot;, &quot;XXXX&quot;, &quot;XXXX&quot;, &quot;XXX&quot;)&#xA;&#xA;### Use biomaRt to get exons positions&#xA;ensembl &amp;lt;- useMart(&quot;ensembl&quot;, dataset = &quot;mmusculus_gene_ensembl&quot;)&#xA;&#xA;gb &amp;lt;- getBM(attributes = c('ensembl_exon_id', &quot;exon_chrom_start&quot;,&quot;exon_chrom_end&quot;,&quot;gene_exon&quot;), &#xA;            filters = &quot;mgi_symbol&quot;, &#xA;            values=&quot;Grin1&quot;, &#xA;            mart = ensembl, &#xA;            bmHeader = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1488" LastEditorUserId="1488" LastEditDate="2017-09-25T14:58:33.867" LastActivityDate="2017-09-26T11:49:15.910" Title="Software recommendation: find DNA sequence distribution over entire transcript" Tags="|r|software-recommendation|sequence-annotation|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2523" PostTypeId="1" CreationDate="2017-09-17T00:19:55.040" Score="4" ViewCount="358" Body="&lt;p&gt;I'm looking for some options for imputation for a high-dimensional dataset of DNA methylation (bisulfite sequencing) data.  Dimensions on the order of 50-100 samples x ~500,000 CpG loci/features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've used K-nearest neighbors, but It seems that this method is not very accurate. As far as I can tell, it limits the minimum number of &quot;genes/features&quot; to impute at one time to something like 1500. which is kind of a lot.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also used missForest to impute smaller datasets with greater accuracy, but it seems computationally infeasible to do it on the full 50x500,000 dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to impute values because I'm using the data for some statistical modeling which require complete cases.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone know of better alternatives than K-nearest neighbor for large scale imputation?  &lt;/p&gt;&#xA;" OwnerUserId="39" LastActivityDate="2017-09-18T10:24:31.850" Title="Recommendations for missing value imputation - DNA methylation data" Tags="|methylation|sequence-analysis|imputation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2525" PostTypeId="1" AcceptedAnswerId="2526" CreationDate="2017-09-17T23:06:41.723" Score="6" ViewCount="144" Body="&lt;p&gt;So I'm basically so new to this that I'm just trying to find out what tools, methods, and keywords I should go look up by myself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a unique strain of a bacteria.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was given RNAseq data for this unique strain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to analyze the RNAseq data, but I need an annotated genome to do that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a resource, I do have the annotated genome of the common lab strain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The common strain should be pretty similar to the unique strain, but I don't know how similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone tell me what tools/methods might be applicable so I can go look them up?&lt;/p&gt;&#xA;" OwnerUserId="1492" LastActivityDate="2023-12-22T08:14:13.557" Title="Bacterial genome annotation of a clinical isolate strain?" Tags="|rna-seq|sequence-annotation|reference-genome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2526" PostTypeId="2" ParentId="2525" CreationDate="2017-09-18T00:27:04.900" Score="5" Body="&lt;p&gt;For a quick (but reliable) analysis, I'd recommend using &lt;a href=&quot;https://pachterlab.github.io/kallisto/download&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kallisto&lt;/a&gt; or &lt;a href=&quot;https://github.com/COMBINE-lab/salmon/releases&quot; rel=&quot;nofollow noreferrer&quot;&gt;Salmon&lt;/a&gt; to quantify isoform read counts using the transcriptome of the lab strain.&lt;/p&gt;&#xA;&lt;p&gt;If you have a concern about transcripts that are in your sample but not in the lab strain, you can do two &lt;a href=&quot;https://github.com/trinityrnaseq/trinityrnaseq/releases&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trinity&lt;/a&gt; assemblies: one fully &lt;em&gt;de-novo&lt;/em&gt;, and one &lt;a href=&quot;https://github.com/trinityrnaseq/trinityrnaseq/wiki/Genome-Guided-Trinity-Transcriptome-Assembly&quot; rel=&quot;nofollow noreferrer&quot;&gt;genome-guided&lt;/a&gt;. These transcripts can then be used as a reference assembly for Kallisto/Salmon, and counts compared to the lab-strain results. Note that the Trinity assemblies will only apply to the particular environments that are in your samples; if a transcript isn't expressed, it won't be in the assembly.&lt;/p&gt;&#xA;&lt;p&gt;Any transcripts in the &lt;em&gt;de-novo&lt;/em&gt; assembly that are not in the genome-guided assembly are potentially novel to your strain. However, care must be taken in interpreting this: if there is any other contamination in the sample, those contaminant transcripts will also be assembled.&lt;/p&gt;&#xA;&lt;p&gt;I'm not too familiar with annotation pipelines, but based on a couple of Twitter posts it looks like &lt;a href=&quot;http://www.vicbioinformatics.com/software.prokka.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prokka&lt;/a&gt; might be a reasonable start. Torsten Seeman (lead developer of Prokka) has done a post about alternative annotation pipelines:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://thegenomefactory.blogspot.co.nz/2013/03/bacterial-genome-annotation-systems.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://thegenomefactory.blogspot.co.nz/2013/03/bacterial-genome-annotation-systems.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Update: Torsten Seeman &lt;a href=&quot;https://twitter.com/torstenseemann/status/1565471892840259585&quot; rel=&quot;nofollow noreferrer&quot;&gt;now recommends&lt;/a&gt; Bakta as a &amp;quot;worthy successor to Prokka&amp;quot;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/oschwengers/bakta&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/oschwengers/bakta&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Bakta is a tool for the rapid &amp;amp; standardized annotation of bacterial genomes and plasmids from both isolates and MAGs. It provides dbxref-rich, sORF-including and taxon-independent annotations in machine-readable JSON &amp;amp; bioinformatics standard file formats for automated downstream analysis.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-12-22T08:14:13.557" LastActivityDate="2023-12-22T08:14:13.557" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2528" PostTypeId="2" ParentId="2523" CreationDate="2017-09-18T10:24:31.850" Score="2" Body="&lt;p&gt;I think this is still an active field of research.  I have heard of &lt;a href=&quot;https://mathgen.stats.ox.ac.uk/genetics_software/phenix/phenix.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Phenix&lt;/a&gt;, which might be appropriate.&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2017-09-18T10:24:31.850" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2530" PostTypeId="2" ParentId="2419" CreationDate="2017-09-18T12:09:48.757" Score="0" Body="&lt;p&gt;Cuffdiff is one the tools that deals FPKM data for analysis of differential expression.By default they use the negative binomial with some condition specific parameter to avoid biases. There are some other models also available in Cuffdiff. Check the following documents for more details,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/cuffdiff/index.html#library-types&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cole-trapnell-lab.github.io/cufflinks/cuffdiff/index.html#library-types&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.reddit.com/r/bioinformatics/comments/3bx3em/fpkm_vs_raw_read_count_for_differential/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.reddit.com/r/bioinformatics/comments/3bx3em/fpkm_vs_raw_read_count_for_differential/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="206" LastActivityDate="2017-09-18T12:09:48.757" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2531" PostTypeId="2" ParentId="2503" CreationDate="2017-09-18T12:15:40.003" Score="1" Body="&lt;p&gt;Have you tried using SnpEff and SnpShift (&lt;a href=&quot;http://snpeff.sourceforge.net&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://snpeff.sourceforge.net&lt;/a&gt;) for annotation of the SNPs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if your desired organism available in their annotation database you can create them easily with the gene bank files. Other than annotation the tools features a bunch of other amazing functionalities.&lt;/p&gt;&#xA;" OwnerUserId="206" LastActivityDate="2017-09-18T12:15:40.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2532" PostTypeId="2" ParentId="2515" CreationDate="2017-09-18T12:40:38.123" Score="3" Body="&lt;p&gt;As others have said, doing whole-genome (or even whole-chromosome) alignments is the wrong solution. Simply track where you're creating SNVs. If you wrote your SNV locations to a sorted BED file you could use something like the following:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;minWidth = 100000  # Only report regions of &amp;gt;=100kb&#xA;last = [None, None]&#xA;for line in open(&quot;some bed file&quot;):&#xA;    cols = line.strip().split()&#xA;    if cols[0] != last[0]:&#xA;        if int(cols[2]) &amp;gt;= minWidth:&#xA;            print(&quot;{}:1-{}&quot;.format(cols[0], cols[2]))&#xA;    else:&#xA;        if int(cols[2]) - last[1] &amp;gt;= minWidth&#xA;            print(&quot;{}:{}-{}&quot;.format(cols[0], last[1], cols[2]))&#xA;    last = [cols[0], int(cols[2])]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will miss the last part of each chromosome/contig, since it doesn't know how long they are, but you should be able to figure out how to handle that. The memory requirements of this are trivially small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect there's also a bedtools or bedops command that will directly give you the distance to the next downstream region. One could alternatively parse that (it's all the above code is doing, but using &lt;code&gt;bedtools&lt;/code&gt; or &lt;code&gt;bedops&lt;/code&gt; would produce more validated results than a quickly thrown together script).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-09-18T12:40:38.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2533" PostTypeId="1" CreationDate="2017-09-18T16:46:02.140" Score="4" ViewCount="305" Body="&lt;p&gt;I need a command in R to retrieve all human genes associated with a Gene Ontology entry. I tried to look for it online but did not find it.&lt;/p&gt;&#xA;" OwnerUserId="1499" LastEditorUserId="96" LastEditDate="2017-09-20T07:04:58.623" LastActivityDate="2017-09-20T07:04:58.623" Title="Retrieving a list of human genes having GO associations" Tags="|r|human-genome|gene-ontology|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2534" PostTypeId="2" ParentId="2533" CreationDate="2017-09-18T17:05:28.830" Score="6" Body="&lt;p&gt;Here's an example for the mouse genome:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(org.Mm.eg.db)&#xA;select(org.Mm.eg.db, c(&quot;GO:0048406&quot;), c(&quot;GENENAME&quot;,&quot;SYMBOL&quot;), c(&quot;GO&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You get output like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                                                    GENENAME SYMBOL&#xA;1                                     pregnancy zone protein    Pzp&#xA;2 nerve growth factor receptor (TNFR superfamily, member 16)   Ngfr&#xA;3 nerve growth factor receptor (TNFR superfamily, member 16)   Ngfr&#xA;4                                             neurotrophin 3   Ntf3&#xA;5             neurotrophic tyrosine kinase, receptor, type 1  Ntrk1&#xA;6            furin (paired basic amino acid cleaving enzyme)  Furin&#xA;7              proprotein convertase subtilisin/kexin type 6  Pcsk6&#xA;8                                                 sortilin 1  Sort1&#xA;9                                      alpha-2-macroglobulin    A2m&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: Given your comment that you want instead a list of all genes with a GO term associated with them (for human):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(org.Hs.eg.db) # Install it from bioconductor yourself&#xA;unlist(as.list(org.Hs.egSYMBOL)[mappedkeys(org.Hs.egGO)])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;mappedkey()&lt;/code&gt; part gets the index into the bimap of all entries with a valid GO mapping. That's then used to subset the symbol bimap (after converting to a list).&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-09-18T20:38:20.467" LastActivityDate="2017-09-18T20:38:20.467" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2535" PostTypeId="2" ParentId="2533" CreationDate="2017-09-18T19:22:43.573" Score="6" Body="&lt;p&gt;I guess the following code will help,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;source(&quot;https://bioconductor.org/biocLite.R&quot;)&#xA;biocLite(&quot;biomaRt&quot;)&#xA;library(&quot;biomaRt&quot;)&#xA;ensembl = useMart(&quot;ensembl&quot;, dataset = &quot;hsapiens_gene_ensembl&quot;)&#xA;#listAttributes(ensembl)&#xA;mapping &amp;lt;- getBM(attributes = c(&quot;ensembl_gene_id&quot;, &quot;hgnc_symbol&quot;, &quot;go_id&quot;), mart = ensembl)&#xA;head(mapping)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="206" LastEditorUserId="206" LastEditDate="2017-09-18T19:40:03.890" LastActivityDate="2017-09-18T19:40:03.890" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2537" PostTypeId="2" ParentId="2515" CreationDate="2017-09-19T10:25:25.493" Score="2" Body="&lt;p&gt;If you store the positions of your random SNPs in a &lt;code&gt;sort-bed&lt;/code&gt; sorted BED file, you can filter with BEDOPS &lt;code&gt;bedmap&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --count --echo --range 50000 --delim '\t' SNPs.bed | awk '$1==1' | cut -f2- &amp;gt; SNPs.filtered.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The file &lt;code&gt;SNPs.filtered.bed&lt;/code&gt; will only contain those elements from &lt;code&gt;SNPs.bed&lt;/code&gt; which do not overlap within 50kb.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or you can use BEDOPS &lt;code&gt;bedops&lt;/code&gt; and &lt;code&gt;closest-features&lt;/code&gt; to do the same thing, but the logic isn't nearly as pretty:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops --everything --range 0:50000 test.bed | closest-features --dist test.bed - | awk -v FS='|' '{ if ($NR &amp;gt; 50000) { print $1; } }' &amp;gt; SNPs.filtered.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result is the same as with the &lt;code&gt;bedmap&lt;/code&gt; pipeline. IMO, the &lt;code&gt;bedmap&lt;/code&gt; approach is cleaner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have &lt;code&gt;SNPs.filtered.bed&lt;/code&gt;, you could convert it to FASTA via &lt;code&gt;samtools faidx&lt;/code&gt; and &lt;code&gt;bed2faidx.pl&lt;/code&gt; or similar. Then you have FASTA upon which you can apply your random perturbations.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2017-09-19T20:31:34.907" LastActivityDate="2017-09-19T20:31:34.907" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2538" PostTypeId="1" AcceptedAnswerId="2542" CreationDate="2017-09-19T10:58:22.457" Score="3" ViewCount="551" Body="&lt;p&gt;It seems there are two packages providing access to TCGA data these days:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/TCGAbiolinks.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;TCGAbiolinks (older)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/GenomicDataCommons.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GenomicDataCommons (newer)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Will &lt;em&gt;GenomicDataCommons&lt;/em&gt; supersede &lt;em&gt;TCGAbiolinks&lt;/em&gt;? Is there anything that I might miss if I decide to use only &lt;em&gt;GenomicDataCommons&lt;/em&gt;?&lt;/p&gt;&#xA;" OwnerUserId="1503" LastActivityDate="2017-09-20T06:59:49.197" Title="What is the R/Bioconductor preferred way of accessing TCGA data?" Tags="|r|bioconductor|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2542" PostTypeId="2" ParentId="2538" CreationDate="2017-09-19T20:25:22.940" Score="1" Body="&lt;p&gt;There isn't &lt;strong&gt;a&lt;/strong&gt; R/Bioconductor person or view, so there isn't a preferred way to access TCGA. Use what suits you. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bioconductor is build by contributors which are encourage to not overlap in their contributions. Usually the packages have different options/features even if they allow to access to the same data.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, the TCGAbiolinks package is specifically for TCGA dataset while GenomicDataCommons is for &quot;access the NIH / NCI Genomic Data Commons RESTful service&quot;. So the functionality they bring to R and Bioconductor is different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fact that one project is finishing doesn't mean that the data or the packages build around that project are obsolete. Some functionality different between them is the functions to harmonize and prepare (&lt;code&gt;GDCprepare&lt;/code&gt;) data from the TCGAbiolink package, or see a boxplot correlation and AAIC plot to define outliers &lt;code&gt;TCGAanalyze_Preprocessing&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-09-20T06:59:49.197" LastActivityDate="2017-09-20T06:59:49.197" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2543" PostTypeId="1" AcceptedAnswerId="2546" CreationDate="2017-09-20T02:22:25.160" Score="5" ViewCount="2127" Body="&lt;p&gt;So I have a list of start and stop positions along chromosomes in different species, and I'd like to get the corresponding DNA sequence for each set of coordinates. In the past, I've just download the genome as a fasta file and then use pyfaidx to extract the sequences at the given positions. But now that I'm working with several species at once, I was wondering if there's any kind of tool in Python or R that can fetch your sequences of interest without downloading a bunch of large files. Thanks&lt;/p&gt;&#xA;" OwnerUserId="250" LastActivityDate="2022-06-27T12:56:36.657" Title="Way to get genomic sequences at given coordinates without downloading fasta files of whole chromosomes/genomes first?" Tags="|r|python|genome|phylogenetics|sequence-analysis|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2544" PostTypeId="2" ParentId="2543" CreationDate="2017-09-20T02:56:05.820" Score="1" Body="&lt;p&gt;&lt;em&gt;I am not aware of any tools that offer this functionality out-of-the-box, although that doesn't necessarily mean it doesn't exist.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tool you describe would have to be some kind of web service, where you submit a request with something like the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;species&lt;/li&gt;&#xA;&lt;li&gt;assembly version&lt;/li&gt;&#xA;&lt;li&gt;list of intervals, each with:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;chromosome ID&lt;/li&gt;&#xA;&lt;li&gt;start position&lt;/li&gt;&#xA;&lt;li&gt;end position&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The response would be one or more subsequences in Fasta format that you could save to a file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach pre-supposes that there will be a relatively small number of subsequences, that those subsequences won't be too large, and you won't be requesting them very frequently. Otherwise, this approach will offer very little practical benefit over your current approach.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2017-09-20T07:03:28.920" LastActivityDate="2017-09-20T07:03:28.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2546" PostTypeId="2" ParentId="2543" CreationDate="2017-09-20T07:27:01.660" Score="8" Body="&lt;p&gt;using a http request.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if there is a DAS server, you can always use this protocol to download the xml -&gt; fasta. see &lt;a href=&quot;https://www.biostars.org/p/56/&quot; rel=&quot;noreferrer&quot;&gt;https://www.biostars.org/p/56/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ curl -s &quot;http://genome.ucsc.edu/cgi-bin/das/hg19/dna?segment=chrM:100,200&quot;  | xmllint --xpath '/DASDNA/SEQUENCE/DNA/text()' - | tr -d '\n'&#xA;ggagccggagcaccctatgtcgcagtatctgtctttgattcctgcctcattctattatttatcgcacctacgttcaatattacaggcgaacatacctacta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or use the UCSC utility &lt;strong&gt;twoBitToFa&lt;/strong&gt; which works with remote files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have an NCBI accession number you can use the E-utilities with seq_start and seq_end.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ wget -q  -O -  &quot;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=nuccore&amp;amp;id=AE014134.1&amp;amp;seq_start=100&amp;amp;seq_stop=300&amp;amp;rettype=fasta&quot;&#xA;&amp;gt;AE014134.1:100-300 Drosophila melanogaster chromosome 2L complete sequence&#xA;TGCCAACATATTGTGCTCTTTGATTTTTTGGCAACCCAAAATGGTGGCGGATGAACGAGATGATAATATA&#xA;TTCAAGTTGCCGCTAATCAGAAATAAATTCATTGCAACGTTAAATACAGCACAATATATGATCGCGTATG&#xA;CGAGAGTAGTGCCAACATATTGTGCTAATGAGTGCCTCTCGTTCTCTGTCTTATATTACCG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastActivityDate="2017-09-20T07:27:01.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2547" PostTypeId="1" AcceptedAnswerId="5264" CreationDate="2017-09-20T14:57:15.650" Score="5" ViewCount="567" Body="&lt;p&gt;I was wondering if some one can help me with pseudo r code for my approach here based on this &lt;a href=&quot;https://www.researchgate.net/publication/265550961_An_RNA-Sequencing_Transcriptome_and_Splicing_Database_of_Glia_Neurons_and_Vascular_Cells_of_the_Cerebral_Cortex&quot; rel=&quot;noreferrer&quot;&gt;paper&lt;/a&gt; to plot a heatmap&lt;/p&gt;&#xA;&lt;h1&gt;Figure 3&lt;/h1&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Hna8E.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Hna8E.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Figure description&lt;/strong&gt;&#xA;The top 40 enriched genes percelltypeare shown in a heat map.Onlyhighlyexpressed genes with FPKM ⬎20 areincludedin this analysis. Fold enrichment iscalculatedasFPKM of one cell type divided by the average FPKM of all other cell types. The majority of these genes showed specific expression by only one cell type, with the exception that some are expressed during more than one maturation stage in the oligodendrocyte lineage.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Similar description in their methods:&lt;/strong&gt;&#xA;Differential expression was calculated as the FPKM of a given cell type&#xA;divided by the average FPKM of all other cell types. Genes were ranked by&#xA;their fold enrichment in each cell type, and top enriched genes for each&#xA;cell type were identified&lt;/p&gt;&#xA;&lt;p&gt;In my case I have 9 cell types(columns) and ~24000 genes(rows) from mouse and the values are TPMs.&lt;/p&gt;&#xA;&lt;p&gt;I know to order celltype only based on one column like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;expr.order = expr[order(expr[,1],decreasing =T),]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which clearly not the right way and would need some help with code in r&lt;/p&gt;&#xA;" OwnerUserId="926" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-10-23T15:07:38.283" Title="cell type specific genes - heatmap using rank based approach" Tags="|r|rna-seq|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2548" PostTypeId="1" AcceptedAnswerId="2558" CreationDate="2017-09-21T01:17:29.437" Score="5" ViewCount="3640" Body="&lt;p&gt;I'm not sure what I'm missing, but I'm struggling to find an official &lt;code&gt;hg38&lt;/code&gt; &lt;code&gt;GTF&lt;/code&gt; file with RefSeq annotations. I'd like to provide the &lt;code&gt;GTF&lt;/code&gt; to &lt;a href=&quot;https://combine-lab.github.io/salmon/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Salmon&lt;/a&gt; to get gene-level annotations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's Salmon's help info for &lt;code&gt;--geneMap&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;File containing a mapping of&#xA;                                          transcripts to genes.  If this file is&#xA;                                          provided Salmon will output both&#xA;                                          quant.sf and quant.genes.sf files,&#xA;                                          where the latter contains aggregated&#xA;                                          gene-level abundance estimates.  The&#xA;                                          transcript to gene mapping should be&#xA;                                          provided as either a GTF file, or a in&#xA;                                          a simple tab-delimited format where&#xA;                                          each line contains the name of a&#xA;                                          transcript and the gene to which it&#xA;                                          belongs separated by a tab.  The&#xA;                                          extension of the file is used to&#xA;                                          determine how the file should be&#xA;                                          parsed.  Files ending in '.gtf', '.gff'&#xA;                                          or '.gff3' are assumed to be in GTF&#xA;                                          format; files with any other extension&#xA;                                          are assumed to be in the simple format.&#xA;                                          In GTF / GFF format, the&#xA;                                          &quot;transcript_id&quot; is assumed to contain&#xA;                                          the transcript identifier and the&#xA;                                          &quot;gene_id&quot; is assumed to contain the&#xA;                                          corresponding gene identifier.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here's what I've tried:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;NCBI seems to only provide &lt;code&gt;GFF&lt;/code&gt; files. I know you can convert between the two, but that hasn't worked well for me in the past. Both file formats allow a lot of freedom, which makes conversions sloppy. Salmon, specifically, is looking for the &lt;code&gt;gene_id&lt;/code&gt; and &lt;code&gt;transcript_id&lt;/code&gt; annotations.&lt;/li&gt;&#xA;&lt;li&gt;UCSC provides &lt;code&gt;GTF&lt;/code&gt; files from RefSeq, but the &lt;code&gt;gene_id&lt;/code&gt; annotation is identical to the &lt;code&gt;transcript_id&lt;/code&gt; annotation (i.e., it's the NM number). Or maybe there's an option I'm missing.&lt;/li&gt;&#xA;&lt;li&gt;Ensembl provides exactly the format and all of the information I could hope for, except the &lt;code&gt;transcript_id&lt;/code&gt; is the Ensembl ID (ENST#), not RefSeq (NM#).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Can anyone tell me how to get a &lt;code&gt;GTF&lt;/code&gt; file with hg38 RefSeq annotations?&lt;/p&gt;&#xA;" OwnerUserId="136" LastEditorUserId="136" LastEditDate="2017-09-21T15:15:18.287" LastActivityDate="2017-09-21T18:11:33.103" Title="hg38 GTF file with RefSeq annotations" Tags="|ngs|public-databases|sequence-annotation|human-genome|gtf|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2549" PostTypeId="1" CreationDate="2017-09-21T02:13:38.307" Score="5" ViewCount="447" Body="&lt;p&gt;Can FASTA files have nucleotide and protein sequences within them; or must they only have 1 type? For example, a FASTA file has 2 sequences. Can the first one encode amino acids while the second one encodes bases?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="1510" LastActivityDate="2017-09-21T04:28:55.450" Title="Can FASTA files have nucleotide and protein sequences within them; or must they only have 1 type?" Tags="|fasta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2550" PostTypeId="2" ParentId="2549" CreationDate="2017-09-21T04:28:55.450" Score="9" Body="&lt;p&gt;While there's nothing stopping anyone from doing that with the FASTA format (after all, it's just a text file with '&gt;' defining header lines), I don't know of any software that would support such a file structure. At best, it would interpret the nucleotide sequences as protein sequences (A/C/G/T are all valid 1-letter protein codes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A better question to ask would be &quot;Does ultra-cool-bioinformatics-tool X support combined nucleotide and protein sequences in the same FASTA file?&quot; In which case the answer would most likely be, &quot;No.&quot;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-09-21T04:28:55.450" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2551" PostTypeId="1" CreationDate="2017-09-21T09:30:54.787" Score="3" ViewCount="447" Body="&lt;p&gt;&lt;em&gt;Note: this question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/273873/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have performed a network recreation analysis based on the interactions of proteins from String db.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to know if the interactions have significance by reshuffling the proteins with permutation against the background network. Since, its first time for me to perform permutation test on PPI, Can someone guide me to a tool or a tutorial to perform permutation test on newly created PPI network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried looking at a &lt;a href=&quot;https://www.youtube.com/watch?v=GE2l3LYDQG0&amp;amp;index=51&amp;amp;list=PLA54E692040796EA5&quot; rel=&quot;nofollow noreferrer&quot;&gt;youtube tutorial&lt;/a&gt; and read some articles like &lt;a href=&quot;https://github.com/DavisLaboratory/prosperiti/blob/master/prosperiti.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; one, but they were not very useful for my purpose.&lt;/p&gt;&#xA;" OwnerUserId="1513" LastEditorUserId="73" LastEditDate="2019-01-27T20:49:05.187" LastActivityDate="2019-09-03T09:01:00.090" Title="pvalue calculation of protein-protein network with permutation test" Tags="|python|statistics|protein-protein-interaction|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2552" PostTypeId="2" ParentId="2241" CreationDate="2017-09-21T09:57:09.233" Score="0" Body="&lt;p&gt;You can get expression data from databases and for membrane proteins, run a transmembrane protein prediction method like CCTOP (&lt;a href=&quot;http://cctop.enzim.ttk.mta.hu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cctop.enzim.ttk.mta.hu/&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="11" LastActivityDate="2017-09-21T09:57:09.233" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2553" PostTypeId="1" AcceptedAnswerId="2555" CreationDate="2017-09-21T12:53:10.027" Score="7" ViewCount="2352" Body="&lt;p&gt;I am trying to make local databases for the ncbi-blast+ package (version 2.60). I am doing so for 4 T-cell receptor genes. 3 of the 4 (TRAV, TRAJ, TRBV) have worked fine, but I am having problems with TRBJ. I say this because they should be the same format and if one works the rest works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They take the format below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;K02545|TRBJ1-1*01|Homo sapiens|F|J-REGION|749..796|48 nt|3| | | |15 AA|15+0=15| | |&#xA;NTEAFFGQGTRLTVV&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I used the following command (like I say, it was fine for the rest, oddly enough):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;makeblastdb -in TRBJ_AA.fa -parse_seqids -dbtype prot&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I get the following error message:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BLAST options error: TRBJ_AA.fa does not match input format type, default input type is FASTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One issue I checked was the length of the sequences being too short, but I believe 11 is the minimum length required (for inputs, don't know if that is the same for databases)- but would someone be able to confirm, that strikes me as the most likely candidate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm happy to use some other metric, but I was hoping on keeping BLAST for consistency's sake for the time being.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit: I know it's to late, but for reference I'm on Ubuntu 16.04.2 LTS x86_64 for all steps. &lt;/p&gt;&#xA;" OwnerUserId="252" LastEditorUserId="252" LastEditDate="2017-09-21T15:10:55.407" LastActivityDate="2017-09-21T15:10:55.407" Title="Why is this makeblastdb command not working?" Tags="|fasta|blast|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2554" PostTypeId="1" CreationDate="2017-09-21T13:49:44.120" Score="6" ViewCount="1013" Body="&lt;p&gt;I'm doing some analysis and I need to subset a large VCF file (~8GB gziped) given a bed interval and identify within a subset of rsid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, both my normal choices to do this analysis (&lt;code&gt;snpSift&lt;/code&gt; and &lt;code&gt;bedtools&lt;/code&gt;) are taking way to long or failing due to memory issues in my local computer and a remote server.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you guys know any other options or suggestions to speed up this process?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Follow the commands I use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools intersect -a &amp;lt;myvcf&amp;gt;.vcf.gz -b &amp;lt;myinterval&amp;gt;.bed -wa | \&#xA;    java -Xmx10g -jar snpSift.jar filter --set &amp;lt;myrsid&amp;gt;.txt &quot;ID in SET[0]&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gzcat &amp;lt;myvcf&amp;gt;.vcf.gz | \&#xA;    java -Xmx10g -jar snpSift.jar intervals &amp;lt;mybed&amp;gt;.bed | \&#xA;    java -Xmx10g -jar snpSift.jar filter --set &amp;lt;myrsid&amp;gt;.txt &quot;ID in SET[0]&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;bedtools&lt;/code&gt; command usually fail due to unknown reason and &lt;code&gt;SnpSift&lt;/code&gt; runs over 6 hours even given 10GB of ram. My local machine have 8GB of RAM, but the server have 32GB.&lt;/p&gt;&#xA;" OwnerUserId="1515" LastEditorUserId="96" LastEditDate="2019-01-30T18:09:28.967" LastActivityDate="2019-03-01T15:34:47.657" Title="Any fast options to query large VCF bed intervals?" Tags="|vcf|bed|filtering|interval|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2555" PostTypeId="2" ParentId="2553" CreationDate="2017-09-21T14:18:48.550" Score="8" Body="&lt;p&gt;This looks like a bug in &lt;code&gt;makeblastdb&lt;/code&gt;. Removing the final &lt;code&gt;| | |&lt;/code&gt; from your sequence's description makes it work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;K02545|TRBJ1-1*01|Homo sapiens|F|J-REGION|749..796|48 nt|3| | | |15 AA|15+0=15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So does removing everything after the 1st space:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;K02545|TRBJ1-1*01|Homo sapiens|F|J-REGION|749..796|48&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There is nothing in the FASTA definition that would explain this and, even more weirdly, the presence of the &lt;code&gt;| | |&lt;/code&gt; alone isn't enough to trigger the issue. I played around with this a little and it looks like it doesn't like the combination of &lt;code&gt;+&lt;/code&gt; or &lt;code&gt;-&lt;/code&gt;along with empty &lt;code&gt;| |&lt;/code&gt;, but there's no clear pattern I can see:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These fail:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;| | |15-| | |&#xA;&amp;gt;| | |15+| | |&#xA;&amp;gt;| | |15-1| | |&#xA;&amp;gt;| | |15+1| | |&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;| |15+| |&#xA;&amp;gt;| | |151| | |&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So yeah, sounds like a bug. I suggest you report it to the NCBI devs. In the meantime, a simple fix would be to just keep the first space-separated field of your fasta header. Pass your fasta file through this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{print $1}' fasta.fa &amp;gt; new.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That will print the 1st field of each line and, assuming you don't have spaces in your sequence, it should effectively sort this out for you. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-09-21T14:18:48.550" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2556" PostTypeId="2" ParentId="2554" CreationDate="2017-09-21T15:20:23.450" Score="4" Body="&lt;p&gt;this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools intersect -a &amp;lt;myvcf&amp;gt;.vcf.gz -b &amp;lt;myinterval&amp;gt;.bed -wa | \&#xA;    java -Xmx10g -jar snpSift.jar filter --set &amp;lt;myrsid&amp;gt;.txt &quot;ID in SET[0]&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;can be replaced with one GATK SelectVariants &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/tooldocs/current/org_broadinstitute_gatk_tools_walkers_variantutils_SelectVariants.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://software.broadinstitute.org/gatk/documentation/tooldocs/current/org_broadinstitute_gatk_tools_walkers_variantutils_SelectVariants.php&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar gatk.jar -T SelectVariants -R ref.fa \&#xA;  -L myinterval.bed --keepIDs myrsid.txt -V myvcf.vcf.gz -o out.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and furthermore, you can also work in parallel by splitting your bed file (e.g. by chromosomes), and merge the results.&lt;/p&gt;&#xA;" OwnerUserId="71" LastActivityDate="2017-09-21T15:20:23.450" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2557" PostTypeId="2" ParentId="2554" CreationDate="2017-09-21T17:52:45.343" Score="4" Body="&lt;p&gt;Generically, with &lt;a href=&quot;http://bedops.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ vcf2bed &amp;lt; &amp;lt;(gunzip -c snps.vcf) | bedops -e 1 - myRegions.bed &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ vcf2bed &amp;lt; &amp;lt;(gunzip -c snps.vcf) | bedmap --echo --echo-map-id --delim '\t' myRegions.bed - &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BEDOPS supports input streams; use them where you can.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, if you're going to query the VCF data frequently, convert once and use it in downstream operations.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ vcf2bed &amp;lt; &amp;lt;(gunzip -c snps.vcf) &amp;gt; snps.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops -e 1 snps.bed myRegions.bed &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a computational cluster, you can further parallelize work by splitting SNPs on chromosomes with &lt;code&gt;bedextract&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ for chr in `bedextract --list-chr snps.bed`; do echo ${chr}; bedextract ${chr} snps.bed &amp;gt; snps.${chr}.bed; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then submit a batch job for each chromosome:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --chrom ${chr} --echo --echo-map-id --delim '\t' myRegions.bed snps.${chr}.bed &amp;gt; answer.${chr}.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using &lt;code&gt;bedextract&lt;/code&gt; and adding &lt;code&gt;--chrom ${chr}&lt;/code&gt; to your &lt;code&gt;bedmap&lt;/code&gt; call will yield large speed improvements over other toolkits, as these options focus work on the chromosome of interest and nothing else.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2017-09-21T17:52:45.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2558" PostTypeId="2" ParentId="2548" CreationDate="2017-09-21T18:11:33.103" Score="5" Body="&lt;p&gt;You will probably be interested in the following UCSC wiki page, which explains how to go from most of the UCSC tables to GTF/GFF:&lt;br&gt;&#xA;&lt;a href=&quot;http://genomewiki.ucsc.edu/index.php/Genes_in_gtf_or_gff_format&quot; rel=&quot;noreferrer&quot;&gt;http://genomewiki.ucsc.edu/index.php/Genes_in_gtf_or_gff_format&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic gist is that UCSC doesn't store any data internally as GTF or GFF, and so you will need to use our genePredToGtf utility in order to convert from our internally stored genePred format into GTF/GFF3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the command I usually give when we get this question on our mailing list:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;$ mysql --user=genome --host=genome-mysql.cse.ucsc.edu -A -N -e &quot;select * from refGene&quot; hg19 | cut -f2- | genePredToGtf -source=hg19.refGene.ucsc file stdin stdout&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Change stdout to the output filename you want in the last command to get an hg19 refGene GTF file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&#xA;chr1    hg19.refGene.ucsc   transcript  11869   14362   .   +   .   gene_id &quot;LOC102725121&quot;; transcript_id &quot;NR_148357&quot;;  gene_name &quot;LOC102725121&quot;;&#xA;chr1    hg19.refGene.ucsc   exon    11869   12227   .   +   .   gene_id &quot;LOC102725121&quot;; transcript_id &quot;NR_148357&quot;; exon_number &quot;1&quot;; exon_id &quot;NR_148357.1&quot;; gene_name &quot;LOC102725121&quot;;&#xA;chr1    hg19.refGene.ucsc   exon    12613   12721   .   +   .   gene_id &quot;LOC102725121&quot;; transcript_id &quot;NR_148357&quot;; exon_number &quot;2&quot;; exon_id &quot;NR_148357.2&quot;; gene_name &quot;LOC102725121&quot;;&#xA;chr1    hg19.refGene.ucsc   exon    13221   14362   .   +   .   gene_id &quot;LOC102725121&quot;; transcript_id &quot;NR_148357&quot;; exon_number &quot;3&quot;; exon_id &quot;NR_148357.3&quot;; gene_name &quot;LOC102725121&quot;;&#xA;chr1    hg19.refGene.ucsc   transcript  11874   14409   .   +   .   gene_id &quot;DDX11L1&quot;; transcript_id &quot;NR_046018&quot;;  gene_name &quot;DDX11L1&quot;;&#xA;...&#xA;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you have further questions about UCSC data or tools feel free to send your question to one of the below mailing lists:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;General questions: genome@soe.ucsc.edu&lt;/li&gt;&#xA;&lt;li&gt;Questions involving private data: genome-www@soe.ucsc.edu&lt;/li&gt;&#xA;&lt;li&gt;Questions involving mirror sites: genome-mirror@ose.ucsc.edu&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1000" LastActivityDate="2017-09-21T18:11:33.103" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2559" PostTypeId="2" ParentId="2554" CreationDate="2017-09-22T11:01:52.813" Score="3" Body="&lt;p&gt;I was tinkering with the command and was able to complete the execution inverting the order of rsid and bed intervals filtering. The command is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gzcat &amp;lt;myvcf&amp;gt;.vcf.gz | \&#xA;    java -Xmx10g -jar snpSift.jar filter --set &amp;lt;myrsid&amp;gt;.txt &quot;ID in SET[0]&quot; | \&#xA;    java -Xmx10g -jar snpSift.jar intervals &amp;lt;mybed&amp;gt;.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Maybe my rsid subset is smaller than my BED intervals and thus the process is quicker and more efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway thanks everyone&lt;/p&gt;&#xA;" OwnerUserId="1515" LastActivityDate="2017-09-22T11:01:52.813" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2560" PostTypeId="1" CreationDate="2017-09-22T14:12:51.740" Score="1" ViewCount="131" Body="&lt;p&gt;I have been thinking about this problem and reading some related papers for sometime.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem can be described as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose that each biometric template is a $d-$dimensional vector. A database is composed of $n$ such templates ( $n$ could be large, like 1 million). Given a query template $q$, the task is to identify the &quot;closest&quot; template according to some distance metric.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, since the dimension $d$ and the database size $n$ could be both large, it is not quite efficient to linearly scan the whole database.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, are there any algorithms to efficiently fulfill the above task? What are the state of the art algorithms for biometric identification?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Through some literature review, I can think of two approaches. But both of them seem to have some limitations.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Building space partitioning index like R-tree. The problem is that such index does not suit for high dimensional data. The final search efficiency would be close to linear search.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Locality sensitive hash. The problem is that this method can only provide &lt;strong&gt;approximate&lt;/strong&gt; searching result. For biometic identification, it is not guaranteed that the matched template could be found.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1523" LastActivityDate="2018-03-30T15:02:45.033" Title="What are the algorithms for biometric identification in large database?" Tags="|database|identifiers|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2561" PostTypeId="1" CreationDate="2017-09-24T11:14:21.517" Score="6" ViewCount="2276" Body="&lt;p&gt;&lt;em&gt;Note: this question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/274321/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need the get the list of gene names involved in glycolysis (to put an example). Not manually, I need to do this in a script. Ideally with Python. I think KEGG is the proper database to do this, but any other (free) suggestion would do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I do it?&lt;/p&gt;&#xA;" OwnerUserId="1525" LastEditorUserId="73" LastEditDate="2018-01-12T02:30:11.220" LastActivityDate="2021-10-29T13:49:09.147" Title="How can I extract gene names for a metabolic pathway from KEGG?" Tags="|python|pathway|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2562" PostTypeId="2" ParentId="2561" CreationDate="2017-09-24T12:39:48.910" Score="6" Body="&lt;p&gt;I whipped this little script up using the &lt;a href=&quot;http://www.kegg.jp/kegg/rest/&quot; rel=&quot;noreferrer&quot;&gt;KEGG API&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;import urllib.request&#xA;import re&#xA;import sys&#xA;pathway = 'hsa00010' # glycolysis&#xA;url = &quot;http://rest.kegg.jp/get/&quot; + pathway&#xA;with urllib.request.urlopen(url) as f:&#xA;    lines = f.read().decode('utf-8').splitlines()&#xA;    want = 0&#xA;    for line in lines:&#xA;        fields = line.split()&#xA;        ## The list of genes starts here&#xA;        if fields[0] == 'GENE':&#xA;            want = 1&#xA;            ## The line with GENE is different&#xA;            print(fields[2].rstrip(';'))&#xA;        ## We reached the next section of the file&#xA;        elif want == 1 and re.match('^\S', line):&#xA;            sys.exit();&#xA;        ## We're still in the list of genes&#xA;        if want == 1 and len(fields)&amp;gt;1:&#xA;            print(fields[1].rstrip(';'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-09-24T12:39:48.910" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2563" PostTypeId="2" ParentId="2561" CreationDate="2017-09-24T14:20:25.117" Score="6" Body="&lt;p&gt;using the REST API of &lt;a href=&quot;http://togows.dbcls.jp/&quot; rel=&quot;noreferrer&quot;&gt;togows&lt;/a&gt;: one can fetch a json structure of the genes associated to a kegg pathway. eg: &lt;a href=&quot;http://togows.org/entry/kegg-pathway/hsa00010/genes.json&quot; rel=&quot;noreferrer&quot;&gt;http://togows.org/entry/kegg-pathway/hsa00010/genes.json&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[&#xA;  {&#xA;    &quot;3101&quot;: &quot;HK3; hexokinase 3 [KO:K00844] [EC:2.7.1.1]&quot;,&#xA;    &quot;3098&quot;: &quot;HK1; hexokinase 1 [KO:K00844] [EC:2.7.1.1]&quot;,&#xA;    &quot;3099&quot;: &quot;HK2; hexokinase 2 [KO:K00844] [EC:2.7.1.1]&quot;,&#xA;    &quot;80201&quot;: &quot;HKDC1; hexokinase domain containing 1 [KO:K00844] [EC:2.7.1.1]&quot;,&#xA;    &quot;2645&quot;: &quot;GCK; glucokinase [KO:K12407] [EC:2.7.1.2]&quot;,&#xA;    &quot;2821&quot;: &quot;GPI; glucose-6-phosphate isomerase [KO:K01810] [EC:5.3.1.9]&quot;,&#xA;    &quot;5213&quot;: &quot;PFKM; phosphofructokinase, muscle [KO:K00850] [EC:2.7.1.11]&quot;,&#xA;    &quot;5214&quot;: &quot;PFKP; phosphofructokinase, platelet [KO:K00850] [EC:2.7.1.11]&quot;,&#xA;    &quot;5211&quot;: &quot;PFKL; phosphofructokinase, liver type [KO:K00850] [EC:2.7.1.11]&quot;,&#xA;    &quot;2203&quot;: &quot;FBP1; fructose-bisphosphatase 1 [KO:K03841] [EC:3.1.3.11]&quot;,&#xA;    &quot;8789&quot;: &quot;FBP2; fructose-bisphosphatase 2 [KO:K03841] [EC:3.1.3.11]&quot;,&#xA;    &quot;230&quot;: &quot;ALDOC; aldolase, fructose-bisphosphate C [KO:K01623] [EC:4.1.2.13]&quot;,&#xA;    &quot;226&quot;: &quot;ALDOA; aldolase, fructose-bisphosphate A [KO:K01623] [EC:4.1.2.13]&quot;,&#xA;    &quot;229&quot;: &quot;ALDOB; aldolase, fructose-bisphosphate B [KO:K01623] [EC:4.1.2.13]&quot;,&#xA;    &quot;7167&quot;: &quot;TPI1; triosephosphate isomerase 1 [KO:K01803] [EC:5.3.1.1]&quot;,&#xA;    &quot;2597&quot;: &quot;GAPDH; glyceraldehyde-3-phosphate dehydrogenase [KO:K00134] [EC:1.2.1.12]&quot;,&#xA;    &quot;26330&quot;: &quot;GAPDHS; glyceraldehyde-3-phosphate dehydrogenase, spermatogenic [KO:K10705] [EC:1.2.1.12]&quot;,&#xA;    &quot;5232&quot;: &quot;PGK2; phosphoglycerate kinase 2 [KO:K00927] [EC:2.7.2.3]&quot;,&#xA;    &quot;5230&quot;: &quot;PGK1; phosphoglycerate kinase 1 [KO:K00927] [EC:2.7.2.3]&quot;,&#xA;    &quot;5223&quot;: &quot;PGAM1; phosphoglycerate mutase 1 [KO:K01834] [EC:5.4.2.11]&quot;,&#xA;    &quot;5224&quot;: &quot;PGAM2; phosphoglycerate mutase 2 [KO:K01834] [EC:5.4.2.11]&quot;,&#xA;    &quot;441531&quot;: &quot;PGAM4; phosphoglycerate mutase family member 4 [KO:K01834] [EC:5.4.2.11]&quot;,&#xA;    &quot;2027&quot;: &quot;ENO3; enolase 3 [KO:K01689] [EC:4.2.1.11]&quot;,&#xA;    &quot;2026&quot;: &quot;ENO2; enolase 2 [KO:K01689] [EC:4.2.1.11]&quot;,&#xA;    &quot;2023&quot;: &quot;ENO1; enolase 1 [KO:K01689] [EC:4.2.1.11]&quot;,&#xA;    &quot;5315&quot;: &quot;PKM; pyruvate kinase, muscle [KO:K00873] [EC:2.7.1.40]&quot;,&#xA;    &quot;5313&quot;: &quot;PKLR; pyruvate kinase, liver and RBC [KO:K12406] [EC:2.7.1.40]&quot;,&#xA;    &quot;5161&quot;: &quot;PDHA2; pyruvate dehydrogenase alpha 2 [KO:K00161] [EC:1.2.4.1]&quot;,&#xA;    &quot;5160&quot;: &quot;PDHA1; pyruvate dehydrogenase alpha 1 [KO:K00161] [EC:1.2.4.1]&quot;,&#xA;    &quot;5162&quot;: &quot;PDHB; pyruvate dehydrogenase (lipoamide) beta [KO:K00162] [EC:1.2.4.1]&quot;,&#xA;    &quot;1737&quot;: &quot;DLAT; dihydrolipoamide S-acetyltransferase [KO:K00627] [EC:2.3.1.12]&quot;,&#xA;    &quot;1738&quot;: &quot;DLD; dihydrolipoamide dehydrogenase [KO:K00382] [EC:1.8.1.4]&quot;,&#xA;    &quot;160287&quot;: &quot;LDHAL6A; lactate dehydrogenase A like 6A [KO:K00016] [EC:1.1.1.27]&quot;,&#xA;    &quot;92483&quot;: &quot;LDHAL6B; lactate dehydrogenase A like 6B [KO:K00016] [EC:1.1.1.27]&quot;,&#xA;    &quot;3939&quot;: &quot;LDHA; lactate dehydrogenase A [KO:K00016] [EC:1.1.1.27]&quot;,&#xA;    &quot;3945&quot;: &quot;LDHB; lactate dehydrogenase B [KO:K00016] [EC:1.1.1.27]&quot;,&#xA;    &quot;3948&quot;: &quot;LDHC; lactate dehydrogenase C [KO:K00016] [EC:1.1.1.27]&quot;,&#xA;    &quot;124&quot;: &quot;ADH1A; alcohol dehydrogenase 1A (class I), alpha polypeptide [KO:K13951] [EC:1.1.1.1]&quot;,&#xA;    &quot;125&quot;: &quot;ADH1B; alcohol dehydrogenase 1B (class I), beta polypeptide [KO:K13951] [EC:1.1.1.1]&quot;,&#xA;    &quot;126&quot;: &quot;ADH1C; alcohol dehydrogenase 1C (class I), gamma polypeptide [KO:K13951] [EC:1.1.1.1]&quot;,&#xA;    &quot;131&quot;: &quot;ADH7; alcohol dehydrogenase 7 (class IV), mu or sigma polypeptide [KO:K13951] [EC:1.1.1.1]&quot;,&#xA;    &quot;127&quot;: &quot;ADH4; alcohol dehydrogenase 4 (class II), pi polypeptide [KO:K13980] [EC:1.1.1.1]&quot;,&#xA;    &quot;128&quot;: &quot;ADH5; alcohol dehydrogenase 5 (class III), chi polypeptide [KO:K00121] [EC:1.1.1.1 1.1.1.284]&quot;,&#xA;    &quot;130&quot;: &quot;ADH6; alcohol dehydrogenase 6 (class V) [KO:K13952] [EC:1.1.1.1]&quot;,&#xA;    &quot;10327&quot;: &quot;AKR1A1; aldo-keto reductase family 1 member A1 [KO:K00002] [EC:1.1.1.2]&quot;,&#xA;    &quot;217&quot;: &quot;ALDH2; aldehyde dehydrogenase 2 family (mitochondrial) [KO:K00128] [EC:1.2.1.3]&quot;,&#xA;    &quot;224&quot;: &quot;ALDH3A2; aldehyde dehydrogenase 3 family member A2 [KO:K00128] [EC:1.2.1.3]&quot;,&#xA;    &quot;219&quot;: &quot;ALDH1B1; aldehyde dehydrogenase 1 family member B1 [KO:K00128] [EC:1.2.1.3]&quot;,&#xA;    &quot;501&quot;: &quot;ALDH7A1; aldehyde dehydrogenase 7 family member A1 [KO:K14085] [EC:1.2.1.3 1.2.1.8 1.2.1.31]&quot;,&#xA;    &quot;223&quot;: &quot;ALDH9A1; aldehyde dehydrogenase 9 family member A1 [KO:K00149] [EC:1.2.1.3 1.2.1.47]&quot;,&#xA;    &quot;221&quot;: &quot;ALDH3B1; aldehyde dehydrogenase 3 family member B1 [KO:K00129] [EC:1.2.1.5]&quot;,&#xA;    &quot;222&quot;: &quot;ALDH3B2; aldehyde dehydrogenase 3 family member B2 [KO:K00129] [EC:1.2.1.5]&quot;,&#xA;    &quot;220&quot;: &quot;ALDH1A3; aldehyde dehydrogenase 1 family member A3 [KO:K00129] [EC:1.2.1.5]&quot;,&#xA;    &quot;218&quot;: &quot;ALDH3A1; aldehyde dehydrogenase 3 family member A1 [KO:K00129] [EC:1.2.1.5]&quot;,&#xA;    &quot;84532&quot;: &quot;ACSS1; acyl-CoA synthetase short chain family member 1 [KO:K01895] [EC:6.2.1.1]&quot;,&#xA;    &quot;55902&quot;: &quot;ACSS2; acyl-CoA synthetase short chain family member 2 [KO:K01895] [EC:6.2.1.1]&quot;,&#xA;    &quot;130589&quot;: &quot;GALM; galactose mutarotase [KO:K01785] [EC:5.1.3.3]&quot;,&#xA;    &quot;5236&quot;: &quot;PGM1; phosphoglucomutase 1 [KO:K01835] [EC:5.4.2.2]&quot;,&#xA;    &quot;55276&quot;: &quot;PGM2; phosphoglucomutase 2 [KO:K15779] [EC:5.4.2.7 5.4.2.2]&quot;,&#xA;    &quot;2538&quot;: &quot;G6PC; glucose-6-phosphatase catalytic subunit [KO:K01084] [EC:3.1.3.9]&quot;,&#xA;    &quot;57818&quot;: &quot;G6PC2; glucose-6-phosphatase catalytic subunit 2 [KO:K01084] [EC:3.1.3.9]&quot;,&#xA;    &quot;92579&quot;: &quot;G6PC3; glucose-6-phosphatase catalytic subunit 3 [KO:K01084] [EC:3.1.3.9]&quot;,&#xA;    &quot;83440&quot;: &quot;ADPGK; ADP dependent glucokinase [KO:K08074] [EC:2.7.1.147]&quot;,&#xA;    &quot;669&quot;: &quot;BPGM; bisphosphoglycerate mutase [KO:K01837] [EC:5.4.2.11 5.4.2.4]&quot;,&#xA;    &quot;9562&quot;: &quot;MINPP1; multiple inositol-polyphosphate phosphatase 1 [KO:K03103] [EC:3.1.3.80 3.1.3.62]&quot;,&#xA;    &quot;5105&quot;: &quot;PCK1; phosphoenolpyruvate carboxykinase 1 [KO:K01596] [EC:4.1.1.32]&quot;,&#xA;    &quot;5106&quot;: &quot;PCK2; phosphoenolpyruvate carboxykinase 2, mitochondrial [KO:K01596] [EC:4.1.1.32]&quot;&#xA;  }&#xA;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastActivityDate="2017-09-24T14:20:25.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2565" PostTypeId="2" ParentId="2522" CreationDate="2017-09-25T07:56:54.053" Score="1" Body="&lt;p&gt;I assume you have the sequence in seq&lt;sup&gt;1&lt;/sup&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Set the input&#xA;seq &amp;lt;- getSequences(&quot;GRIN1&quot;)&#xA;motif &amp;lt;- &quot;ggcc&quot;&#xA;# Convert to lower case&#xA;seq2 &amp;lt;- tolower(seqinr::s2c(as.character(seq)))&#xA;istarts &amp;lt;- seq(from = 1 , to = length(seq2), by = 1)&#xA;# Create words of the same size as the motif&#xA;oligos &amp;lt;- seq2[istarts]&#xA;for (i in 2:nchar(motif)) {&#xA;        oligos &amp;lt;- paste(seq2, seq2[istarts + i - 1], sep = &quot;&quot;)&#xA;}&#xA;# Remove exceeding words (words with NA, might need tweaking to motif size&#xA;oligos &amp;lt;- oligos[-c(length(oligos):(length(oligos)-nchar(motif)+2))]&#xA;# Find motif in the sequence&#xA;hits &amp;lt;- grep(motif, oligos, ignore.case = TRUE)&#xA;#Prepare the plot&#xA;positions &amp;lt;- ifelse(1:length(seq2) %in% hits, 1, 0)&#xA;plot(1:length(seq2), positions, col = c(&quot;white&quot;, &quot;black&quot;)[position+1], pch = 19)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The plot is not the density, but you can choose how to plot it now, as you might be interested in windows sizes of 5 nucleotides or 20. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/6CWmu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6CWmu.png&quot; alt=&quot;positions where the motif is found in a sequence&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From here you can iterate through multiple sequences. If you want to find several motifs you can take advantage of the oligos object if they are of the same size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is the &lt;a href=&quot;http://bioconductor.org/packages/seqPattern/&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqPattern&lt;/a&gt; package in Bioconductor for more advanced pattern matching. But surprisingly it doesn't have this simple motif search. I think it would be easier to do this with perl and pyhton than in R.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;1: In the code posted the function getSequence returns an error.&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-09-25T18:57:41.063" LastActivityDate="2017-09-25T18:57:41.063" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2566" PostTypeId="2" ParentId="2308" CreationDate="2017-09-26T00:10:53.477" Score="1" Body="&lt;p&gt;This question was answered &lt;a href=&quot;https://github.com/pachterlab/sleuth/issues/140&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="1334" LastActivityDate="2017-09-26T00:10:53.477" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2567" PostTypeId="1" CreationDate="2017-09-26T09:47:25.503" Score="11" ViewCount="4979" Body="&lt;p&gt;I have read counts data and I want to convert them into RPKM values. For this conversion I need the gene length. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does the gene length need to be calculated based on the sum of coding exonic lengths? Or are there any different ways for that? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that gene length can be taken from the Gencode GTF v19 file. Could you please tell me how that &lt;code&gt;Gene_length&lt;/code&gt; is calculated?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Data I'm having is RNA-Seq data. I don't have any idea whether I need to include UTR's in this calculation or only exons?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Github I have seen &lt;a href=&quot;https://github.com/santhilalsubhash/rpkm_rnaseq_count&quot; rel=&quot;noreferrer&quot;&gt;RPKM calculation from Counts data with the Gene_length from Gencode GTF file&lt;/a&gt;. Do you think this is the right way of calculation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And why RPKM is - Its not for differential analysis. For &lt;a href=&quot;http://cbc.mc.vanderbilt.edu/tnbc/&quot; rel=&quot;noreferrer&quot;&gt;TNBC subtyping&lt;/a&gt; they use microarray data. I would like to give a try with RNA-Seq data. So for this I'm trying out different and the right way. &lt;/p&gt;&#xA;" OwnerUserId="1377" LastEditorUserId="29" LastEditDate="2018-05-09T20:17:09.553" LastActivityDate="2018-05-09T20:17:09.553" Title="How can I calculate gene_length for RPKM calculation from counts data?" Tags="|gene|gtf|gencode|fpkm|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2568" PostTypeId="2" ParentId="2567" CreationDate="2017-09-26T10:02:37.753" Score="7" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/dpryan79/Answers/blob/master/SEQanswers_42420/GTF2LengthGC.R&quot; rel=&quot;noreferrer&quot;&gt;Here&lt;/a&gt; you can find some example R code to compute the gene length given a GTF file (it computes GC content too, which you don't need). This uses one of a number of ways of computing gene length, in this case the length of the &quot;union gene model&quot;. In this method, the non-duplicated exons for each gene are simply summed up (&quot;non-duplicated&quot; in that no genomic base is double counted). This is a very simple way of getting a gene length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are alternative methods that you should be aware of, among which are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Median transcript length: That is, the exonic lengths in each transcript are summed and the median across transcripts is used. This is probably a little more valid than the code that I linked to.&lt;/li&gt;&#xA;&lt;li&gt;Per-sample effective gene lengths: the optimal method, though it requires using something like RSEM, which will give you an effective gene length.&lt;/li&gt;&#xA;&lt;li&gt;Using the length of the &quot;major isoform&quot; in your tissue of interest. This isn't as good as method 2, but is more accurate than all of the others.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;At the end of the day, you're just coming up with a scale factor for each gene, so unless you intend to compare values across genes (this is problematic to begin with) then it's questionable if using some of the more correct but also more time-involved methods are really getting you anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: Note that if you want to plug these values into some sort of subtyping tool (TNBC in your case), you should first start with some samples for which you know the subtype. Then you can at least see if you're getting reasonable results. After that, do read up on how the method works and see if there's anything about RNAseq that makes it incompatible.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-09-26T10:58:34.450" LastActivityDate="2017-09-26T10:58:34.450" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2569" PostTypeId="2" ParentId="2522" CreationDate="2017-09-26T11:49:15.910" Score="5" Body="&lt;p&gt;If I understand the question correctly, you'd like to plot the positions of the matches to you motif along with a gene model that shows the positions of introns and exons for the different transcripts. This can be accomplished fairly easily with ggbio:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(EnsDb.Mmusculus.v79)&#xA;library(ggbio)&#xA;library(biomaRt)&#xA;library(stringr)&#xA;library(dplyr)&#xA;&#xA;# plot gene model with ggbio&#xA;gene_model &amp;lt;- autoplot(EnsDb.Mmusculus.v79, ~ symbol == &amp;quot;Grin1&amp;quot;)&#xA;&#xA;ensembl &amp;lt;- useMart(&amp;quot;ensembl&amp;quot;, dataset = &amp;quot;mmusculus_gene_ensembl&amp;quot;)&#xA;&#xA;# You need the full length gene sequence to match the gene model&#xA;seq &amp;lt;- biomaRt::getSequence(id = &amp;quot;Grin1&amp;quot;,&#xA;                         type = &amp;quot;mgi_symbol&amp;quot;,&#xA;                         seqType = &amp;quot;gene_exon_intron&amp;quot;,&#xA;                         mart = ensembl)&#xA;&#xA;# retrieve position of gene in genome to match up with gene model&#xA;gene_start=biomaRt::getBM(attributes=c(&amp;quot;start_position&amp;quot;), filters = 'external_gene_name', values = 'Grin1', mart=ensembl)&#xA;&#xA;# use stringR to find positions of exact matches&#xA;motif &amp;lt;- &amp;quot;ggcc&amp;quot;&#xA;motif_positions &amp;lt;- as.data.frame(str_locate_all(seq$gene_exon_intron, toupper(motif))) %&amp;gt;%&#xA;  mutate(start=start+gene_start[[1]]-1)&#xA;&#xA;# make plot similar to Llopis's answer &#xA;motif_plot &amp;lt;- ggplot(motif_positions, aes(x=start)) +&#xA;    geom_point(y=1) +&#xA;    scale_y_continuous(limits=c(0,1))&#xA;&#xA;# combine plots&#xA;tracks(motif= motifs, gene_model=gene_model)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JHJ8x.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JHJ8x.png&quot; alt=&quot;Gene Model with Motif matchs&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This gives all motif matches for the gene sequence, not just the transcripts. It would be quite straightforward to filter out positions that overlap introns&lt;/p&gt;&#xA;" OwnerUserId="581" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-09-26T11:49:15.910" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2570" PostTypeId="1" CreationDate="2017-09-26T23:50:05.087" Score="10" ViewCount="1529" Body="&lt;p&gt;I downloaded some publicly available RNA-seq data and want to compare those samples carrying a mutation (~4) against the rest (~800!). I ran both EdgeR and DESeq2, and the first results in an asymmetric volcano plot: skewed in one of the sides, meaning that it results in very large logFC that have large (NS) adjusted p-values. So next, in order to see if this asymmetry is persistant, I ran several random partitions of the data as groups (but keeping them highly unbalanced), obtaining the same behavior. Here an example of what I mean (I run it with 4, 20 and 100 to show how this effect decreases as I increase the small class size)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/rcYvB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rcYvB.png&quot; alt=&quot;upper panel: logFC vs logCPM, lower panel: logFC vs -log10(FDR)&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*I tried EdgeR-robust and I still get the same effect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not see this effect when running DESeq2, only very few DE genes.  What method would you recommend using when comparing such different class sizes? &lt;/p&gt;&#xA;" OwnerUserId="1375" LastEditorUserId="1375" LastEditDate="2017-09-28T18:02:42.440" LastActivityDate="2017-09-28T18:02:42.440" Title="*very* unbalanced group sizes for DE" Tags="|rna-seq|deseq2|edger|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2571" PostTypeId="2" ParentId="2570" CreationDate="2017-09-27T01:16:39.690" Score="3" Body="&lt;p&gt;Random partitioning (as you have done) seems like a reasonable thing to do to work out what's going on. From the results you have suggested, it looks like DESeq2 is performing better than EdgeR, so use DESeq2. Can you show the volcano plots and the partition sizes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I'm a little bit biased in this, because DESeq2 is what I prefer to use (particularly where Salmon is not appropriate).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-09-27T01:16:39.690" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2572" PostTypeId="1" CreationDate="2017-09-27T04:37:59.170" Score="1" ViewCount="2176" Body="&lt;p&gt;I am trying snakemake for my mRNA-seq pipeline and I got stuck in a simple thing.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from os.path import join&#xA;&#xA;&#xA;RSEM_INDEX = &quot;/app/ref/ensembl/human/idx/GRCh37&quot;&#xA;SAMPLES, = glob_wildcards(&quot;Align/{sample}/Aligned.toTranscriptome.out.bam&quot;)&#xA;&#xA;print(&quot;---Sample Names---&quot;)&#xA;&#xA;for i in SAMPLES:&#xA;    print(i)&#xA;&#xA;rule all:&#xA;    input:&#xA;        expand(&quot;Expression/{sample}/rsem.genes.results&quot;, sample=SAMPLES)&#xA;&#xA;&#xA;rule expression:&#xA;    input:&#xA;        bam = join(&quot;Align&quot;, &quot;{wildcards.sample}}&quot;, &quot;Aligned.toTranscriptome.out.bam&quot;)&#xA;    params:&#xA;        rsem_index = RSEM_INDEX&#xA;    output:&#xA;        join(&quot;Expression&quot;, &quot;{wildcards.sample}&quot;, &quot;rsem.genes.results&quot;)&#xA;    threads:&#xA;        8&#xA;    shell:&#xA;        &quot;&quot;&quot;&#xA;        rsem-calculate-expression --estimate-rspd --bam --no-bam-output -p {threads} --paired-end Align/{sample}/Aligned.toTranscriptome.out.bam {params.rsem_index} Expression/{wildcards.sample}/rsem.genes.results &amp;gt;&amp;amp; Expression/{wildcards.sample}/rsem.log&#xA;        &quot;&quot;&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is an error I got&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    MissingInputException in line 12 of /app/src/mRNAseq/TBD170049_20170320_RNAseq/quantify.snakemake:&#xA;Missing input files for rule all:&#xA;Expression/013_clamp_0m_buffy/rsem.genes.results&#xA;Expression/12_27_11_clamp_buffy_200/rsem.genes.results&#xA;Expression/010_200_buffy/rsem.genes.results&#xA;Expression/010_0_buffy/rsem.genes.results&#xA;Expression/12_27_11_clamp-p_buffy_0/rsem.genes.results&#xA;Expression/013_clamp_200m_buffy/rsem.genes.results&#xA;Expression/11_15_006_0m_buffy/rsem.genes.results&#xA;Expression/11_15_006_200m_buffy/rsem.genes.results&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, I thought rule all input for your final files you want to make, but it's looking for the final files that I want to make.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to get some help. would you help me please? Thank you in advance!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My file trees.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;drwxrwxr-x. 10 corea corea  4096 Sep 26 20:54 Align&#xA;-rw-rw-r--.  1 corea corea  1295 Sep 27 10:19 align.snakemake&#xA;-rw-rw-r--.  1 corea corea  1823 Sep 27 09:38 align.snakemake.test&#xA;-rw-rw-r--.  1 corea corea 31055 Sep 26 08:55 out&#xA;-rw-rw-r--.  1 corea corea   869 Sep 27 12:34 quantify.snakemake&#xA;drwxrwxr-x.  2 corea corea  4096 Sep 22 20:50 Rawdata&#xA;(mRNAseq) [corea@Biocore76 TBD170049_20170320_RNAseq]$ tree Align/&#xA;Align/&#xA;├── 010_0_buffy&#xA;│   ├── Aligned.out.bam&#xA;│   ├── Aligned.toTranscriptome.out.bam&#xA;│   ├── Log.final.out&#xA;│   ├── Log.out&#xA;│   ├── Log.progress.out&#xA;│   ├── SJ.out.tab&#xA;│   ├── _STARgenome&#xA;│   │   ├── sjdbInfo.txt&#xA;│   │   └── sjdbList.out.tab&#xA;│   └── _STARpass1&#xA;│       ├── Log.final.out&#xA;│       └── SJ.out.tab&#xA;├── 010_200_buffy&#xA;│   ├── Aligned.out.bam&#xA;│   ├── Aligned.toTranscriptome.out.bam&#xA;│   ├── Log.final.out&#xA;│   ├── Log.out&#xA;│   ├── Log.progress.out&#xA;│   ├── SJ.out.tab&#xA;│   ├── _STARgenome&#xA;│   │   ├── sjdbInfo.txt&#xA;│   │   └── sjdbList.out.tab&#xA;│   └── _STARpass1&#xA;│       ├── Log.final.out&#xA;│       └── SJ.out.tab&#xA;├── 013_clamp_0m_buffy&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1539" LastActivityDate="2017-09-27T06:36:55.567" Title="snakemake rule all" Tags="|snakemake|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2573" PostTypeId="2" ParentId="2572" CreationDate="2017-09-27T06:30:36.150" Score="3" Body="&lt;p&gt;The following (or something close) should work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule expression:&#xA;    input:&#xA;        &quot;Align/{sample}/Aligned.toTranscriptome.out.bam&quot;&#xA;    params:&#xA;        rsem_index = RSEM_INDEX&#xA;    output:&#xA;        &quot;Expression/{sample}/rsem.genes.results&quot;&#xA;    threads:&#xA;        8&#xA;    shell:&#xA;        &quot;&quot;&quot;&#xA;        rsem-calculate-expression --estimate-rspd --bam --no-bam-output -p {threads} --paired-end {input} {params.rsem_index} {output} &amp;gt;&amp;amp; Expression/{wildcards.sample}/rsem.log&#xA;        &quot;&quot;&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;BTW, the original issue was probably due to the &lt;code&gt;}}&lt;/code&gt; in your input line.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-09-27T06:36:55.567" LastActivityDate="2017-09-27T06:36:55.567" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2575" PostTypeId="1" CreationDate="2017-09-27T15:30:34.130" Score="2" ViewCount="348" Body="&lt;p&gt;I am interested on how profile hidden markov model is used for protein analysis in bioinformatics. Can you guys explain why phmm is more preferable method comparing to simple MSA(Multiple Sequence Alignment) scoring or simple MSA probability scoring? For example we have three MSA sequences as based model and a target to be scored.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;MSA Sequences&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;A B C B C A&lt;br&gt;&#xA;A B B C C A&lt;br&gt;&#xA;A B C B A A&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Target Sequence&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;A B B B C A&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By MSA scoring we can count score for target sequence comparing to a profile MSA. For example above target sequence protein column will counted as match(A), match(B), mismatch(B-majorityMSA(C)), match(B-majorityMSA(B)), match(C-majorityMSA(C)), match(A). Just give score for match and mismatch and we can get total score for a target sequence comparing to a profile MSA sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By MSA probability scoring we can count probability of each protein in each column so for example above &lt;br&gt; &#xA;&lt;b&gt;P(target_seq similar to MSA profile)&lt;/b&gt; = P(A in MSA sequences)*P(B in MSA sequences)*.... *P(A in MSA sequences) = 1*1*(1/3)*(2/3)*(2/3)*1 = 4/27 &lt;br&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Isn't MSA scoring like above examples powerful enough?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot !!&lt;/p&gt;&#xA;" OwnerUserId="1542" LastEditorUserId="1542" LastEditDate="2017-09-28T00:45:14.123" LastActivityDate="2017-09-28T08:11:58.033" Title="Why PHMM is used For Protein Analysis" Tags="|sequence-analysis|sequence-alignment|hidden-markov-models|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2576" PostTypeId="1" AcceptedAnswerId="2581" CreationDate="2017-09-27T17:31:17.290" Score="3" ViewCount="134" Body="&lt;p&gt;I am trying to use Ragout: &lt;a href=&quot;https://github.com/fenderglass/Ragout&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/fenderglass/Ragout&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to fill the gaps in my &lt;em&gt;de novo&lt;/em&gt; genome assembly. You can access the article freely here: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/24931998&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pubmed/24931998&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this, I first need to install Sibelia.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried doing so by cloning the GitHub repository into output/software/ragout/ and then running the following command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;git clone https://github.com/fenderglass/Ragout output/software/ragout&#xA;&#xA;python2 output/software/ragout/scripts/install-sibelia.py&#xA;&#xA;Installing Sibelia&#xA;Downloading source...&#xA;-- The CXX compiler identification is AppleClang 9.0.0.9000037&#xA;-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++&#xA;-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works&#xA;-- Detecting CXX compiler ABI info&#xA;-- Detecting CXX compiler ABI info - done&#xA;-- Detecting CXX compile features&#xA;-- Detecting CXX compile features - done&#xA;-- The C compiler identification is AppleClang 9.0.0.9000037&#xA;-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc&#xA;-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works&#xA;-- Detecting C compiler ABI info&#xA;-- Detecting C compiler ABI info - done&#xA;-- Detecting C compile features&#xA;-- Detecting C compile features - done&#xA;-- Performing Test HAVE_UNKNOWN_WALL&#xA;-- Performing Test HAVE_UNKNOWN_WALL - Success&#xA;-- Performing Test HAVE_UNKNOWN_FOMIT_FRAME_POINTER&#xA;-- Performing Test HAVE_UNKNOWN_FOMIT_FRAME_POINTER - Success&#xA;-- Looking for inttypes.h&#xA;-- Looking for inttypes.h - found&#xA;-- Looking for memory.h&#xA;-- Looking for memory.h - found&#xA;-- Looking for stddef.h&#xA;-- Looking for stddef.h - found&#xA;-- Looking for stdint.h&#xA;-- Looking for stdint.h - found&#xA;-- Looking for stdlib.h&#xA;-- Looking for stdlib.h - found&#xA;-- Looking for string.h&#xA;-- Looking for string.h - found&#xA;-- Looking for strings.h&#xA;-- Looking for strings.h - found&#xA;-- Looking for sys/types.h&#xA;-- Looking for sys/types.h - found&#xA;-- Performing Test HAVE_INLINE&#xA;-- Performing Test HAVE_INLINE - Success&#xA;-- Performing Test HAVE___INLINE&#xA;-- Performing Test HAVE___INLINE - Success&#xA;-- Performing Test HAVE___INLINE__&#xA;-- Performing Test HAVE___INLINE__ - Success&#xA;-- Performing Test HAVE___DECLSPEC_DLLEXPORT_&#xA;-- Performing Test HAVE___DECLSPEC_DLLEXPORT_ - Failed&#xA;-- Performing Test HAVE___DECLSPEC_DLLIMPORT_&#xA;-- Performing Test HAVE___DECLSPEC_DLLIMPORT_ - Failed&#xA;-- Check size of uint8_t&#xA;-- Check size of uint8_t - done&#xA;-- Check size of int32_t&#xA;-- Check size of int32_t - done&#xA;-- Looking for PRId32&#xA;-- Looking for PRId32 - found&#xA;-- Configuring done&#xA;-- Generating done&#xA;-- Build files have been written to: /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/build&#xA;Scanning dependencies of target divsufsort&#xA;[  4%] Building C object libdivsufsort-2.0.1/lib/CMakeFiles/divsufsort.dir/divsufsort.o&#xA;[  8%] Building C object libdivsufsort-2.0.1/lib/CMakeFiles/divsufsort.dir/sssort.o&#xA;[ 12%] Building C object libdivsufsort-2.0.1/lib/CMakeFiles/divsufsort.dir/trsort.o&#xA;[ 16%] Building C object libdivsufsort-2.0.1/lib/CMakeFiles/divsufsort.dir/utils.o&#xA;[ 20%] Linking C static library libdivsufsort.a&#xA;[ 20%] Built target divsufsort&#xA;Scanning dependencies of target Sibelia&#xA;[ 24%] Building CXX object CMakeFiles/Sibelia.dir/sibelia.cpp.o&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/sibelia.cpp:8:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/postprocessor.h:10:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/align.h:43:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/sequence.h:103:&#xA;/Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/sequence/string_set_dependent_generous.h:38:9: warning: &#xA;      'SEQAN_SEQUENCE_STRING_SET_DEPENDENT_GENEROUS_H_' is used as a header&#xA;      guard here, followed by #define of a different macro [-Wheader-guard]&#xA;#ifndef SEQAN_SEQUENCE_STRING_SET_DEPENDENT_GENEROUS_H_&#xA;        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;/Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/sequence/string_set_dependent_generous.h:39:9: note: &#xA;      'SEQAN_SEQUENCE_STRING_SET_DEPENDENT_GENEROUSH_' is defined here; did you&#xA;      mean 'SEQAN_SEQUENCE_STRING_SET_DEPENDENT_GENEROUS_H_'?&#xA;#define SEQAN_SEQUENCE_STRING_SET_DEPENDENT_GENEROUSH_&#xA;        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;        SEQAN_SEQUENCE_STRING_SET_DEPENDENT_GENEROUS_H_&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/sibelia.cpp:8:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/postprocessor.h:10:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/align.h:44:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/score.h:48:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/score/score_matrix.h:40:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/file.h:70:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/system.h:82:&#xA;/Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/system/system_sema.h:120:27: warning: &#xA;      'sem_init' is deprecated [-Wdeprecated-declarations]&#xA;            SEQAN_DO_SYS(!sem_init(hSemaphore, 0, init));&#xA;                          ^&#xA;/usr/include/sys/semaphore.h:55:42: note: 'sem_init' has been explicitly marked&#xA;      deprecated here&#xA;int sem_init(sem_t *, int, unsigned int) __deprecated;&#xA;                                         ^&#xA;/usr/include/sys/cdefs.h:176:37: note: expanded from macro '__deprecated'&#xA;#define __deprecated    __attribute__((deprecated))&#xA;                                       ^&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/sibelia.cpp:8:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/postprocessor.h:10:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/align.h:44:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/score.h:48:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/score/score_matrix.h:40:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/file.h:70:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/system.h:82:&#xA;/Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/system/system_sema.h:124:27: warning: &#xA;      'sem_destroy' is deprecated [-Wdeprecated-declarations]&#xA;            SEQAN_DO_SYS(!sem_destroy(hSemaphore));&#xA;                          ^&#xA;/usr/include/sys/semaphore.h:53:26: note: 'sem_destroy' has been explicitly&#xA;      marked deprecated here&#xA;int sem_destroy(sem_t *) __deprecated;&#xA;                         ^&#xA;/usr/include/sys/cdefs.h:176:37: note: expanded from macro '__deprecated'&#xA;#define __deprecated    __attribute__((deprecated))&#xA;                                       ^&#xA;3 warnings generated.&#xA;[ 28%] Building CXX object CMakeFiles/Sibelia.dir/postprocessor.cpp.o&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/postprocessor.cpp:9:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/postprocessor.h:10:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/align.h:43:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/sequence.h:103:&#xA;/Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/sequence/string_set_dependent_generous.h:38:9: warning: &#xA;      'SEQAN_SEQUENCE_STRING_SET_DEPENDENT_GENEROUS_H_' is used as a header&#xA;      guard here, followed by #define of a different macro [-Wheader-guard]&#xA;#ifndef SEQAN_SEQUENCE_STRING_SET_DEPENDENT_GENEROUS_H_&#xA;        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;/Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/sequence/string_set_dependent_generous.h:39:9: note: &#xA;      'SEQAN_SEQUENCE_STRING_SET_DEPENDENT_GENEROUSH_' is defined here; did you&#xA;      mean 'SEQAN_SEQUENCE_STRING_SET_DEPENDENT_GENEROUS_H_'?&#xA;#define SEQAN_SEQUENCE_STRING_SET_DEPENDENT_GENEROUSH_&#xA;        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;        SEQAN_SEQUENCE_STRING_SET_DEPENDENT_GENEROUS_H_&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/postprocessor.cpp:9:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/postprocessor.h:10:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/align.h:44:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/score.h:48:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/score/score_matrix.h:40:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/file.h:70:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/system.h:82:&#xA;/Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/system/system_sema.h:120:27: warning: &#xA;      'sem_init' is deprecated [-Wdeprecated-declarations]&#xA;            SEQAN_DO_SYS(!sem_init(hSemaphore, 0, init));&#xA;                          ^&#xA;/usr/include/sys/semaphore.h:55:42: note: 'sem_init' has been explicitly marked&#xA;      deprecated here&#xA;int sem_init(sem_t *, int, unsigned int) __deprecated;&#xA;                                         ^&#xA;/usr/include/sys/cdefs.h:176:37: note: expanded from macro '__deprecated'&#xA;#define __deprecated    __attribute__((deprecated))&#xA;                                       ^&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/postprocessor.cpp:9:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/postprocessor.h:10:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/align.h:44:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/score.h:48:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/score/score_matrix.h:40:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/file.h:70:&#xA;In file included from /Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/system.h:82:&#xA;/Users/cr517/Documents/phd/project/sibelia-build/Sibelia-master/src/include/seqan/system/system_sema.h:124:27: warning: &#xA;      'sem_destroy' is deprecated [-Wdeprecated-declarations]&#xA;            SEQAN_DO_SYS(!sem_destroy(hSemaphore));&#xA;                          ^&#xA;/usr/include/sys/semaphore.h:53:26: note: 'sem_destroy' has been explicitly&#xA;      marked deprecated here&#xA;int sem_destroy(sem_t *) __deprecated;&#xA;                         ^&#xA;/usr/include/sys/cdefs.h:176:37: note: expanded from macro '__deprecated'&#xA;#define __deprecated    __attribute__((deprecated))&#xA;                                       ^&#xA;3 warnings generated.&#xA;[ 32%] Building CXX object CMakeFiles/Sibelia.dir/indexedsequence.cpp.o&#xA;[ 36%] Building CXX object CMakeFiles/Sibelia.dir/util.cpp.o&#xA;[ 40%] Building CXX object CMakeFiles/Sibelia.dir/outputgenerator.cpp.o&#xA;[ 44%] Building CXX object CMakeFiles/Sibelia.dir/blockfinder.cpp.o&#xA;[ 48%] Building CXX object CMakeFiles/Sibelia.dir/blockinstance.cpp.o&#xA;[ 52%] Building CXX object CMakeFiles/Sibelia.dir/bifurcationstorage.cpp.o&#xA;[ 56%] Building CXX object CMakeFiles/Sibelia.dir/bulgeremoval.cpp.o&#xA;[ 60%] Building CXX object CMakeFiles/Sibelia.dir/dnasequence.cpp.o&#xA;[ 64%] Building CXX object CMakeFiles/Sibelia.dir/edge.cpp.o&#xA;[ 68%] Building CXX object CMakeFiles/Sibelia.dir/fasta.cpp.o&#xA;[ 72%] Building CXX object CMakeFiles/Sibelia.dir/serialization.cpp.o&#xA;[ 76%] Building CXX object CMakeFiles/Sibelia.dir/synteny.cpp.o&#xA;[ 80%] Building CXX object CMakeFiles/Sibelia.dir/test/unrolledlisttest.cpp.o&#xA;[ 84%] Building CXX object CMakeFiles/Sibelia.dir/platform.cpp.o&#xA;[ 88%] Building CXX object CMakeFiles/Sibelia.dir/stranditerator.cpp.o&#xA;[ 92%] Building CXX object CMakeFiles/Sibelia.dir/vertexenumeration.cpp.o&#xA;[ 96%] Building CXX object CMakeFiles/Sibelia.dir/resource.cpp.o&#xA;[100%] Linking CXX executable Sibelia&#xA;[100%] Built target Sibelia&#xA;[ 20%] Built target divsufsort&#xA;[100%] Built target Sibelia&#xA;Install the project...&#xA;-- Install configuration: &quot;Release&quot;&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/bin/Sibelia&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/NEWS.md&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/ANNOTATION.md&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/README.md&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/USAGE.md&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/INSTALL.md&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/SIBELIA.md&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/C-SIBELIA.md&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/LICENSE.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/C-Sibelia&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/C-Sibelia/Staphylococcus_aureus&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/C-Sibelia/Staphylococcus_aureus/NCTC8325.fasta&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/C-Sibelia/Staphylococcus_aureus/README.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/C-Sibelia/Staphylococcus_aureus/RN4220.fasta&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/C-Sibelia/Staphylococcus_aureus/variant.vcf&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori/blocks_coords.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori/circos&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori/circos/circos.conf&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori/circos/circos.highlight.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori/circos/circos.highlight1.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori/circos/circos.highlight2.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori/circos/circos.highlight3.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori/circos/circos.highlight4.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori/circos/circos.image.conf&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori/circos/circos.png&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori/circos/circos.segdup.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori/circos/circos.sequences.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori/circos/circos.svg&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori/coverage_report.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori/d3_blocks_diagram.html&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori/genomes_permutations.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori/Helicobacter_pylori.fasta&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Helicobacter_pylori/README.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Staphylococcus_aureus&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Staphylococcus_aureus/blocks_coords.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Staphylococcus_aureus/circos&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Staphylococcus_aureus/circos/circos.conf&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Staphylococcus_aureus/circos/circos.highlight.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Staphylococcus_aureus/circos/circos.image.conf&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Staphylococcus_aureus/circos/circos.png&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Staphylococcus_aureus/circos/circos.segdup.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Staphylococcus_aureus/circos/circos.sequences.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Staphylococcus_aureus/circos/circos.svg&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Staphylococcus_aureus/coverage_report.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Staphylococcus_aureus/d3_blocks_diagram.html&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Staphylococcus_aureus/genomes_permutations.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Staphylococcus_aureus/README.txt&#xA;-- Installing: /Users/cr517/Documents/phd/project/sibelia-build/share/Sibelia/doc/examples/Sibelia/Staphylococcus_aureus/Staphylococcus.fasta&#xA;Traceback (most recent call last):&#xA;  File &quot;output/software/ragout/scripts/install-sibelia.py&quot;, line 111, in &amp;lt;module&amp;gt;&#xA;    sys.exit(main())&#xA;  File &quot;output/software/ragout/scripts/install-sibelia.py&quot;, line 107, in main&#xA;    return int(not install_deps(args.prefix))&#xA;  File &quot;output/software/ragout/scripts/install-sibelia.py&quot;, line 31, in install_deps&#xA;    return install_sibelia(prefix)&#xA;  File &quot;output/software/ragout/scripts/install-sibelia.py&quot;, line 64, in install_sibelia&#xA;    shutil.copy(sibelia_bin_src, sibelia_bin_dst)&#xA;  File &quot;/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py&quot;, line 119, in copy&#xA;    copyfile(src, dst)&#xA;  File &quot;/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py&quot;, line 83, in copyfile&#xA;    with open(dst, 'wb') as fdst:&#xA;IOError: [Errno 2] No such file or directory: '/Users/cr517/Documents/phd/project/lib/Sibelia'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, I get a lot of errors. Could someone help me, please?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried to install Sibelia following their instructions here: &lt;a href=&quot;https://github.com/bioinf/Sibelia&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/bioinf/Sibelia&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cd Sibelia/build&#xA;➜  build git:(master) cmake ../src&#xA;-- The CXX compiler identification is AppleClang 9.0.0.9000037&#xA;-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++&#xA;-- Check for working CXX compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ -- works&#xA;-- Detecting CXX compiler ABI info&#xA;-- Detecting CXX compiler ABI info - done&#xA;-- Detecting CXX compile features&#xA;-- Detecting CXX compile features - done&#xA;-- The C compiler identification is AppleClang 9.0.0.9000037&#xA;-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc&#xA;-- Check for working C compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/cc -- works&#xA;-- Detecting C compiler ABI info&#xA;-- Detecting C compiler ABI info - done&#xA;-- Detecting C compile features&#xA;-- Detecting C compile features - done&#xA;-- Performing Test HAVE_UNKNOWN_WALL&#xA;-- Performing Test HAVE_UNKNOWN_WALL - Success&#xA;-- Performing Test HAVE_UNKNOWN_FOMIT_FRAME_POINTER&#xA;-- Performing Test HAVE_UNKNOWN_FOMIT_FRAME_POINTER - Success&#xA;-- Looking for inttypes.h&#xA;-- Looking for inttypes.h - found&#xA;-- Looking for memory.h&#xA;-- Looking for memory.h - found&#xA;-- Looking for stddef.h&#xA;-- Looking for stddef.h - found&#xA;-- Looking for stdint.h&#xA;-- Looking for stdint.h - found&#xA;-- Looking for stdlib.h&#xA;-- Looking for stdlib.h - found&#xA;-- Looking for string.h&#xA;-- Looking for string.h - found&#xA;-- Looking for strings.h&#xA;-- Looking for strings.h - found&#xA;-- Looking for sys/types.h&#xA;-- Looking for sys/types.h - found&#xA;-- Performing Test HAVE_INLINE&#xA;-- Performing Test HAVE_INLINE - Success&#xA;-- Performing Test HAVE___INLINE&#xA;-- Performing Test HAVE___INLINE - Success&#xA;-- Performing Test HAVE___INLINE__&#xA;-- Performing Test HAVE___INLINE__ - Success&#xA;-- Performing Test HAVE___DECLSPEC_DLLEXPORT_&#xA;-- Performing Test HAVE___DECLSPEC_DLLEXPORT_ - Failed&#xA;-- Performing Test HAVE___DECLSPEC_DLLIMPORT_&#xA;-- Performing Test HAVE___DECLSPEC_DLLIMPORT_ - Failed&#xA;-- Check size of uint8_t&#xA;-- Check size of uint8_t - done&#xA;-- Check size of int32_t&#xA;-- Check size of int32_t - done&#xA;-- Looking for PRId32&#xA;-- Looking for PRId32 - found&#xA;-- Configuring done&#xA;-- Generating done&#xA;-- Build files have been written to: /Users/cr517/Documents/phd/project/Sibelia/build&#xA;➜  build git:(master) make&#xA;Scanning dependencies of target lagan&#xA;[  3%] Creating directories for 'lagan'&#xA;[  6%] Performing download step (DIR copy) for 'lagan'&#xA;[  9%] No patch step for 'lagan'&#xA;[ 12%] No update step for 'lagan'&#xA;[ 15%] No configure step for 'lagan'&#xA;[ 18%] Performing build step for 'lagan'&#xA;rightinfluence.cpp:21:2: error: reference to 'end' is ambiguous&#xA;        end.score = -2;&#xA;        ^&#xA;rightinfluence.cpp:3:18: note: candidate found by name lookup is 'end'&#xA;Fragment origin, end;&#xA;                 ^&#xA;/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/iterator:1768:1: note: &#xA;      candidate found by name lookup is 'std::__1::end'&#xA;end(const _Cp&amp;amp; __c)&#xA;^&#xA;/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/iterator:1611:1: note: &#xA;      candidate found by name lookup is 'std::__1::end'&#xA;end(_Tp (&amp;amp;__array)[_Np])&#xA;^&#xA;/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/iterator:1760:1: note: &#xA;      candidate found by name lookup is 'std::__1::end'&#xA;end(_Cp&amp;amp; __c)&#xA;^&#xA;rightinfluence.cpp:22:22: error: reference to 'end' is ambiguous&#xA;        origin.totalScore = end.totalScore = 0;&#xA;                            ^&#xA;rightinfluence.cpp:3:18: note: candidate found by name lookup is 'end'&#xA;Fragment origin, end;&#xA;                 ^&#xA;/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/iterator:1768:1: note: &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2017-09-28T14:13:55.303" LastActivityDate="2017-09-28T14:47:48.720" Title="Installing Sibelia for Ragout on Mac OSX" Tags="|genomics|scaffold|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2577" PostTypeId="2" ParentId="2575" CreationDate="2017-09-27T23:31:36.657" Score="0" Body="&lt;p&gt;I recommend you to read &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2766791/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; in which the applications of HMM in protein sequence alignments are discussed.&lt;/p&gt;&#xA;" OwnerUserId="1375" LastActivityDate="2017-09-27T23:31:36.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2579" PostTypeId="2" ParentId="2575" CreationDate="2017-09-28T08:11:58.033" Score="1" Body="&lt;p&gt;The simple probabilistic scoring scheme you describe to query a target sequence against a (protein) family represented as a multiple sequence alignment (MSA) is actually similar in essence to a &lt;a href=&quot;https://en.wikipedia.org/wiki/Position_weight_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;Position Weight Matrix&lt;/a&gt; (PWM), although PWMs are more refined. PWMs are powerful rapid tools often used for motif analyses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, your approach (and PWMs) don't handle insertions or deletions (indels), and you can't easily add these into your model (for example, consider that 2 1bp indels should not score the same as 1 2bp indel, and indels could occur within your MSA as well as in your target sequence). You could try and add some arbitrary gap opening and extension penalties somehow (like BLAST) but then you'd abandon your probabilistic model! By contrast, pHMMs have a fully fledged probabilistic indel model with insertion and deletion states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, of practical note, there are some excellent implementations of pHMMs for protein sequence analysis, such as &lt;a href=&quot;http://hmmer.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HMMER3&lt;/a&gt; and &lt;a href=&quot;https://github.com/soedinglab/hh-suite&quot; rel=&quot;nofollow noreferrer&quot;&gt;HHSuite&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2017-09-28T08:11:58.033" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2580" PostTypeId="1" CreationDate="2017-09-28T10:20:22.817" Score="3" ViewCount="315" Body="&lt;p&gt;I'm using &lt;a href=&quot;https://bitbucket.org/young_computation/crcmapper/overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;CRCmapper&lt;/a&gt; on my data. There are several modules in the tool. I've used exactly instructed in the usage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Commands used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python CRCmapper.py -e MB12_noEncBlack_AllEnhancers.table.txt -b MB12.bam -g HG19 -f hg19-stdChr/ -s MB12_peaks.bed -n MB12 -o MB12/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the error I got:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Warning: text mode turns off computation of q-values&#xA;IDENTIFY TF-TF INTERACTIONS&#xA;Traceback (most recent call last):&#xA;  File &quot;CRCmapper.py&quot;, line 605, in &amp;lt;module&amp;gt;&#xA;    main()&#xA;  File &quot;CRCmapper.py&quot;, line 593, in main&#xA;    graph = buildNetwork(projectFolder, projectName, candidateGenes, refseqToNameDict, motifConvertFile)&#xA;  File &quot;CRCmapper.py&quot;, line 381, in buildNetwork&#xA;    target = refseqToNameDict[region[0]]&#xA;KeyError: 'AP1'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My expertise in python is very basic. I would be grateful if someone take a look at this error and suggest how to fix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. I contacted authors a long back, I didn't get any response.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="1545" LastEditorUserId="48" LastEditDate="2018-11-26T11:28:37.640" LastActivityDate="2019-07-24T17:52:19.263" Title="help with CRCmapper" Tags="|python|networks|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="2581" PostTypeId="2" ParentId="2576" CreationDate="2017-09-28T14:47:48.720" Score="4" Body="&lt;p&gt;There is a link to compiled binaries for OSX on the &lt;a href=&quot;http://bioinf.spbau.ru/sibelia&quot; rel=&quot;nofollow noreferrer&quot;&gt;project homepage&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2017-09-28T14:47:48.720" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2582" PostTypeId="1" AcceptedAnswerId="2583" CreationDate="2017-09-28T16:17:55.147" Score="4" ViewCount="734" Body="&lt;p&gt;I haven't been able to find the following in the documentation at &lt;a href=&quot;http://www.repeatmasker.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.repeatmasker.org/&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) Why are certain regions classified as &quot;Other&quot;? Are these regions impossible to concretely classify due to alignment issues?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) I believe that centromeric and telomeric regions would be classified as tandem repeats. Are telomeric/centromeric regions classified as &quot;DNA&quot; within the hg19 version of RepeatMasker? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are these documented somewhere? &lt;/p&gt;&#xA;" OwnerUserId="146" LastActivityDate="2017-10-04T01:27:16.643" Title="How are centromeric/telomeric regions categorized in RepeatMasker? &quot;Other&quot; category?" Tags="|repeat|centromere|telomere|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2583" PostTypeId="2" ParentId="2582" CreationDate="2017-09-28T16:31:43.167" Score="3" Body="&lt;p&gt;I'll leave question 1 to someone else since I don't know. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding question 2, centromeric and telomeric regions are hard masked in hg19, so there's no sequence to repeat mask. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For hg38, the centromeres are labeled &lt;code&gt;ALR/Alpha&lt;/code&gt; with a class of &lt;code&gt;centr&lt;/code&gt;. Telomeres are still hard-masked, but they tend to have simple repeats next to them (as annotated by repeatmasker, which makes sense).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apparently the &lt;code&gt;Other&lt;/code&gt; class in hg19 is comprised of &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/16288912&quot; rel=&quot;nofollow noreferrer&quot;&gt;SVA elements&lt;/a&gt;. In hg38, these are reclassified as &lt;code&gt;Retrotransposon&lt;/code&gt; class and &lt;code&gt;SVA&lt;/code&gt; family, which didn't exist in hg19. &lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="48" LastEditDate="2017-09-29T06:59:29.150" LastActivityDate="2017-09-29T06:59:29.150" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2584" PostTypeId="2" ParentId="2570" CreationDate="2017-09-28T17:08:13.823" Score="3" Body="&lt;p&gt;In general, you would expect roughly the same distribution on both sides of your volcano plot. The first two plots are concerning since there's so much on the right and not on the left. With such a small number of samples, it is hard to conclude if this is an artifact of not having enough samples, a technical artifact from the way the data was processed, or true biology. I remember reading that DESeq and other packages like a high number of samples for estimating dispersion or something, so this could also be contributing. I think you should find a better dataset (more samples) to prove or disprove your hypothesis.&lt;/p&gt;&#xA;" OwnerUserId="66" LastActivityDate="2017-09-28T17:08:13.823" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2585" PostTypeId="1" AcceptedAnswerId="2609" CreationDate="2017-09-29T00:18:43.927" Score="3" ViewCount="935" Body="&lt;p&gt;I am a complete newbie to MATLAB and the first task I have is to create a volcano plot. I have been using &lt;a href=&quot;https://www.mathworks.com/help/bioinfo/ref/mavolcanoplot.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt; to understand about it and get started.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to run it on dummy values - &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a=[1 2 3]&#xA;&#xA;b=[4.6 2.7 4.5]&#xA;&#xA;c=[0.05 0.33 0.45]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then I ran - &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SigStructure = mavolcanoplot(a, b, c)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My understanding is that &lt;code&gt;a&lt;/code&gt; represents the gene expression values for condition 1, &lt;code&gt;b&lt;/code&gt; for condition 2, and &lt;code&gt;c&lt;/code&gt; is the list of p-values for the 3 data points in &lt;code&gt;a&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However running this code gives me the error - &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Index exceeds matrix dimensions.&#xA;&#xA;Error in mavolcanoplot (line 127)&#xA;appdata.effect = X(paramStruct.goodVals) - Y(paramStruct.goodVals);&#xA;&#xA;Error in volc (line 4)&#xA;SigStructure = mavolcanoplot(a, b, c)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can anyone explain where I am going wrong?&lt;/p&gt;&#xA;" OwnerUserId="907" LastEditorUserId="48" LastEditDate="2017-09-29T08:54:52.387" LastActivityDate="2017-10-06T10:02:49.147" Title="Error in creating a volcano plot in MATLAB" Tags="|differential-expression|matlab|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2586" PostTypeId="1" AcceptedAnswerId="2587" CreationDate="2017-09-29T15:39:03.103" Score="10" ViewCount="4701" Body="&lt;p&gt;I see that &lt;a href=&quot;https://wiki.nci.nih.gov/display/TCGA/RNASeq+Version+2&quot; rel=&quot;noreferrer&quot;&gt;TCGA RNASeq V2 RSEM&lt;/a&gt; data is normalized with upper-quartile normalization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After doing Quantification with RSEM with the samples I have, I got &quot;genes.results&quot; as output which has gene id, transcript id(s), length, expected count, and FPKM. So, from all the sample output files I got the gene_id and expected_counts [from all samples]. for eg: it looks like following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Gene_id S1  S2  S3  S4  S5&#xA;ENSG00000000003 1355    1121    242 709 1071&#xA;ENSG00000000005 5   0   0   0   0&#xA;ENSG00000000419 839 1345    48  976 1234&#xA;ENSG00000000457 429 1803.08 386 1156    628&#xA;ENSG00000000460 404 1444    523 479 1260&#xA;ENSG00000000938 294 312 93  189 683&#xA;ENSG00000000971 3911    4633.92 264 2863    5194&#xA;ENSG00000001036 1546    2276    271 1641    2141&#xA;ENSG00000001084 1945    2125    490 980 1533&#xA;ENSG00000001167 2054    4069.72 3457    2075    2207&#xA;ENSG00000001460 153 339 77  221 273&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to apply &quot;upper-quartile&quot; normalization on this data. But doesn't know which R package I can use and which function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is anyone aware about this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. This is not for differential analysis&lt;/p&gt;&#xA;" OwnerUserId="1377" LastEditorUserId="48" LastEditDate="2017-12-18T10:44:59.223" LastActivityDate="2022-01-26T19:02:42.820" Title="How to apply upperquartile normalization on RSEM expected counts?" Tags="|r|rna-seq|normalization|rsem|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2587" PostTypeId="2" ParentId="2586" CreationDate="2017-09-29T18:58:48.477" Score="9" Body="&lt;p&gt;You can use the &lt;code&gt;quantile&lt;/code&gt; function in base R to get the value of a particular quantile (e.g. 0.75 for the upper quartile). This can then be used as a factor for normalisation: divide the observed expression by this number.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## set up simulated data&#xA;data.mat &amp;lt;- floor(10^data.frame(sapply(1:5,function(x){rnorm(100)})));&#xA;colnames(data.mat) &amp;lt;- paste0(&amp;quot;S&amp;quot;,1:5);&#xA;data.mat[data.mat &amp;gt; 0] &amp;lt;- floor(data.mat[data.mat &amp;gt; 0]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It's generally a good idea to first remove genes that have no counts in any samples. Rows can be removed using a logical selection based on the total expression in any given row.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## Only retain rows that have a positive total expression per gene&#xA;data.mat &amp;lt;- data.mat[rowSums(data.mat) &amp;gt; 0,];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Two methods of upper quartile determination are presented here, one which is based on the total counts, and one based on the expressed counts (i.e. excluding values with no expression). These both use the &lt;code&gt;quantile&lt;/code&gt; function of R. The &lt;code&gt;apply&lt;/code&gt; function calculates statistics that are based on a single specified dimension (or multiple dimensions) of the data. The first dimension of a 2D matrix in R corresponds to the row, while the second corresponds to the column. By using &lt;code&gt;2&lt;/code&gt; in the apply function, data are collected along the second dimension (i.e. the column), and statistics generated from that:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data.quantileAll &amp;lt;- apply(data.mat, 2, function(x){quantile(x, 0.75)});&#xA;data.quantileExpressed &amp;lt;-&#xA;  apply(data.mat, 2, function(x){quantile(x[x&amp;gt;0], 0.75)});&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Expressed counts may be preferred when read counts are low, so it's difficult to distinguish between a transcript with very low expression and a transcript with no expression.&lt;/p&gt;&#xA;&lt;p&gt;Normalisation is applied by dividing the values by the column-based upper quartile statistic. R will repeat values along the last dimension first (which in this case is the column), so the matrix needs to be transposed first (with columns as genes) to get the arithmetic working properly, with a final transposition to return the orientation back to the original direction (with columns as samples).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data.norm &amp;lt;- t(t(data.mat) / data.quantileExpressed);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-01-26T19:02:42.820" LastActivityDate="2022-01-26T19:02:42.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2588" PostTypeId="2" ParentId="2560" CreationDate="2017-09-30T00:46:36.770" Score="2" Body="&lt;p&gt;I think that a very nice example of what you are talking about saying the word bigger, is probably a microarray transciptional analysis of RNA extracted from a biological sample. Simplifying, in these analyis, each observation is analised on, up to 10000 genes expression levels of different mRNA coding genes. Thus, from these type of molecular biology experiments, we obtain matrices with few observations (d, the samples) respect to the dimensions observed (k, the target gene expression levels). Well, d &amp;lt;&amp;lt; k. In this situation, in order to perform a classification-like task, and to reduce the problem of the overfitting related with dimensionality maledition, the ways that we might follow are several: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) PCA: it is a good tool to visualize multidimensional datasets reducing dimensionality and noise level in the consolidated and standardized data. Problem: it is an unsupervised machine learning algorithm. Alone, it haven't the capacity of class discrimination. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) PCA-LDA is a good combination, where PCA eliminates  hyperdimensionality and background in our data sheets, and than LDA (linear discriminant analysis) finds the directions onto the new subspace of characteristics, that are able to maximize variances between classes and decrease the variances within these. For this reason LDA is known as a kind of supervised machine learning technique. Furthermore, PCA and LDA are usually used together because LDA applied on high dimensional datasets could lead us toward overfitting problems. After the training of PCA-LDA pipeline, being LDA a classificator object trained and shaped on the principal component subspace, you can use this to examine new unknown samples or observation and predict its class by similarity. e.g. recently, Martin et al., on a Nature article talk about PCA-LDA approach to identify and than to predict cancer cells from those are normal, starting from ranman spectra of tissues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) PCA-LDA on transpose matrix: to use as better as possible the huge dimensionality, some authors used to transpose the initial matrix in order to force positioning of genes as rows and samples as columns. Then, the subsequent LDA is performed not on PC scores, but on loading factors of the eigenfactor matrix obtaining thus the list of the most important discriminant gene expression levels to segregate normal samples to cancer ones.&lt;/p&gt;&#xA;" OwnerUserId="1057" LastEditorUserId="48" LastEditDate="2017-10-01T10:50:36.113" LastActivityDate="2017-10-01T10:50:36.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2590" PostTypeId="1" AcceptedAnswerId="2591" CreationDate="2017-10-01T20:20:34.690" Score="7" ViewCount="3727" Body="&lt;p&gt;I'm trying to investigate whether there are errors within my BAM. After looking at the BAM header to see whether the read groups exists (using samtools, i.e. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -H file1.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here, it appears that the header includes &lt;code&gt;@RG&lt;/code&gt; tags. However, I suspect that the BAM contains reads that either lack the &lt;code&gt;@RG&lt;/code&gt; tag or perhaps these read groups are not well-defined, i.e. it looks like some reads in the BAM may be missing an RG tag. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I explicitly check whether all reads have an RG tag, and whether these are well-defined?&lt;/p&gt;&#xA;" OwnerUserId="146" LastActivityDate="2017-10-02T19:22:57.313" Title="How to check whether all BAM read contain defined read groups?" Tags="|sam|samtools|reads|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2591" PostTypeId="2" ParentId="2590" CreationDate="2017-10-01T22:08:51.313" Score="6" Body="&lt;p&gt;According to the man page, running &lt;code&gt;samtools stats --split RG &amp;lt;file1.bam&amp;gt;&lt;/code&gt; should produce summary statistics separated by read group. If it doesn't produce a list of ungrouped reads, counts/statistics can be compared to running without the &lt;code&gt;--split RG&lt;/code&gt; argument.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's some example output from a BAM file with combined read groups:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ samtools stats --split RG both.bam | grep '^SN'&#xA;SN  raw total sequences:    2392&#xA;SN  filtered sequences: 0&#xA;SN  sequences:  2392&#xA;SN  is sorted:  1&#xA;SN  1st fragments:  2392&#xA;SN  last fragments: 0&#xA;SN  reads mapped:   2341&#xA;SN  reads mapped and paired:    0   # paired-end technology bit set + both mates mapped&#xA;SN  reads unmapped: 51&#xA;SN  reads properly paired:  0   # proper-pair bit set&#xA;SN  reads paired:   0   # paired-end technology bit set&#xA;SN  reads duplicated:   0   # PCR or optical duplicate bit set&#xA;SN  reads MQ0:  0   # mapped and MQ=0&#xA;SN  reads QC failed:    0&#xA;SN  non-primary alignments: 0&#xA;SN  total length:   2292231 # ignores clipping&#xA;SN  bases mapped:   2255361 # ignores clipping&#xA;SN  bases mapped (cigar):   1538560 # more accurate&#xA;SN  bases trimmed:  0&#xA;SN  bases duplicated:   0&#xA;SN  mismatches: 254720  # from NM fields&#xA;SN  error rate: 1.655574e-01    # mismatches / bases mapped (cigar)&#xA;SN  average length: 958&#xA;SN  maximum length: 9078&#xA;SN  average quality:    20.0&#xA;SN  insert size average:    0.0&#xA;SN  insert size standard deviation: 0.0&#xA;SN  inward oriented pairs:  0&#xA;SN  outward oriented pairs: 0&#xA;SN  pairs with other orientation:   0&#xA;SN  pairs on different chromosomes: 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;--split&lt;/code&gt; command has also produced additional &lt;code&gt;.bamstat&lt;/code&gt; files, which can be looked at for the summary statistics for individual read groups. In this case I'm only interested in the &quot;raw total sequences&quot; line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep 'raw total sequences' *.bamstat&#xA;both.bam_minusOnly.bamstat:SN   raw total sequences:    1901&#xA;both.bam_plusOnly.bamstat:SN    raw total sequences:    491&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are 1901 sequences from the &quot;minusOnly&quot; group, and 491 sequences from the &quot;plusOnly&quot; group, and I know from the previous statistics that there are 2392 total sequences. 1901 + 491 = 2392, so I know there are no ungrouped reads in this file.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-10-01T23:38:54.913" LastActivityDate="2017-10-01T23:38:54.913" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2592" PostTypeId="2" ParentId="2590" CreationDate="2017-10-02T07:07:44.277" Score="4" Body="&lt;p&gt;using &lt;a href=&quot;http://lindenb.github.io/jvarkit/SamJdk.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;samjdk&lt;/a&gt; and invoking the function &lt;a href=&quot;https://samtools.github.io/htsjdk/javadoc/htsjdk/htsjdk/samtools/SAMRecord.html#getReadGroup--&quot; rel=&quot;nofollow noreferrer&quot;&gt;getReadGroup()&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;getReadGroup() returns The SAMReadGroupRecord from the SAMFileHeader for this SAMRecord, or null if 1) this record has no RG tag, or 2) the header doesn't contain the read group with the given ID.or 3) this record has no SAMFileHeader&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar dist/samjdk.jar -e 'return record.getReadGroup()==null;'  input.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the output will be a sam/bam file with missing/bad RG .&lt;/p&gt;&#xA;" OwnerUserId="71" LastActivityDate="2017-10-02T07:07:44.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2593" PostTypeId="2" ParentId="2590" CreationDate="2017-10-02T19:22:57.313" Score="2" Body="&lt;p&gt;A complete solution to validate a SAM/BAM file would be picard's ValidateSamFile.  This will find many other issues with BAM files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://broadinstitute.github.io/picard/command-line-overview.html#ValidateSamFile&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://broadinstitute.github.io/picard/command-line-overview.html#ValidateSamFile&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2017-10-02T19:22:57.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2594" PostTypeId="1" AcceptedAnswerId="2599" CreationDate="2017-10-02T21:40:11.273" Score="7" ViewCount="1479" Body="&lt;p&gt;I am having trouble understanding the affine gap penalty in the following example -&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mZTnP.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mZTnP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/7f2Pr.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7f2Pr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Pt01s.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Pt01s.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure where the 3 and 4 come from or the 4 and 5 in cells 1,3 and 2,4. I'm not understanding how the affine gap penalty works here. I understand how the linear gap penalty worked in the Needlemen_Wunsch but not this.&lt;/p&gt;&#xA;" OwnerUserId="1565" LastActivityDate="2017-10-04T01:00:11.117" Title="Waterman-Smith-Beyer Sequence Alignment" Tags="|sequence-alignment|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2595" PostTypeId="1" CreationDate="2017-10-03T02:48:37.133" Score="3" ViewCount="158" Body="&lt;p&gt;I would like to find the identity of a protein, together with the organism name and kingdom.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The given DNA sequence is below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ACTGTTGTTCGGTGATCATCAGTTGTACAACGTCCTAACAACATCACATGCAATGCTTATGATATTCTTC&#xA;TTCATCATGCCAGGCACGATGGCAGGACTAGGCAACTTACTAGTGCCATTCCAGATGAGTGTACCGGAGT&#xA;TAGTATTCCCAAAGATTAATAACATCGGTATATGATTTTTAGTATGTGGTCTACTTTTGATTACGGGTTC&#xA;ATCTTGGATGGAGGAAGGTTCAGGAACGGCCTGAACCGTCTATCCACCACTAGCGCTCACTGCAAGTCAT&#xA;AGCGGACTTGCTGTAGATACGTTCATTATCGCATTGCACATGGCCGGTGCAAGCTCCCTTACAGGAAGCA&#xA;TCAACCTTATATGTACAATCGCCTATGCCCGCCGTTCACTCATGGCGATGCTGCAGTCATCACTTTATCC&#xA;CTGATCCATTACAATCACTGCAGCGTTACTCATAGGAGTTGTGCCTGTGCTAGCAGGTGCTATCACGATG&#xA;CTACTCACTGATAGAAGTTGGAGTACCAGCTTCTATGACAGTTCGGCAGGCGGTGATCCTATGTTGTATC&#xA;AGCACTTATTCTGGGTGTTTGGGCATCCAGAAGTCTATATCATCATACTTCCAGTATTCGGTATAGTCAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've been asked to translate this DNA sequence into the 6 possible frames which are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;5'3' Frame 1&#xA;TVVR-SSVVQRPNNITCNAYDILLHHARHDGRTRQLTSAIPDECTGVSIPKD--HRYMIF&#xA;SMWSTFDYGFILDGGRFRNGLNRLSTTSAHCKS-RTCCRYVHYRIAHGRCKLPYRKHQPY&#xA;MYNRLCPPFTHGDAAVITLSLIHYNHCSVTHRSCACASRCYHDATH--KLEYQLL-QFGR&#xA;R-SYVVSALILGVWASRSLYHHTSSIRYSQ&#xA;5'3' Frame 2&#xA;LLFGDHQLYNVLTTSHAMLMIFFFIMPGTMAGLGNLLVPFQMSVPELVFPKINNIGI-FL&#xA;VCGLLLITGSSWMEEGSGTA-TVYPPLALTASHSGLAVDTFIIALHMAGASSLTGSINLI&#xA;CTIAYARRSLMAMLQSSLYP-SITITAALLIGVVPVLAGAITMLLTDRSWSTSFYDSSAG&#xA;GDPMLYQHLFWVFGHPEVYIIILPVFGIV&#xA;5'3' Frame 3&#xA;CCSVIISCTTS-QHHMQCL-YSSSSCQARWQD-ATY-CHSR-VYRS-YSQRLITSVYDF-&#xA;YVVYF-LRVHLGWRKVQERPEPSIHH-RSLQVIADLL-IRSLSHCTWPVQAPLQEASTLY&#xA;VQSPMPAVHSWRCCSHHFIPDPLQSLQRYS-ELCLC-QVLSRCYSLIEVGVPASMTVRQA&#xA;VILCCISTYSGCLGIQKSISSYFQYSV-S&#xA;3'5' Frame 1&#xA;LTIPNTGSMMI-TSGCPNTQNKC-YNIGSPPAELS-KLVLQLLSVSSIVIAPASTGTTPM&#xA;SNAAVIVMDQG-SDDCSIAMSERRA-AIVHIRLMLPVRELAPAMCNAIMNVSTASPL-LA&#xA;VSASGG-TVQAVPEPSSIQDEPVIKSRPHTKNHIPMLLIFGNTNSGTLIWNGTSKLPSPA&#xA;IVPGMMKKNIISIACDVVRTLYN--SPNNS&#xA;3'5' Frame 2&#xA;-LYRILEV--YRLLDAQTPRISADTT-DHRLPNCHRSWYSNFYQ-VAS--HLLAQAQLL-&#xA;VTLQ-L-WIRDKVMTAASP-VNGGHRRLYI-G-CFL-GSLHRPCAMR--TYLQQVRYDLQ&#xA;-ALVVDRRFRPFLNLPPSKMNP-SKVDHILKIIYRCY-SLGILTPVHSSGMALVSCLVLP&#xA;SCLA--RRIS-ALHVMLLGRCTTDDHRTT&#xA;3'5' Frame 3&#xA;DYTEYWKYDDIDFWMPKHPE-VLIQHRITACRTVIEAGTPTSISE-HRDSTC-HRHNSYE&#xA;-RCSDCNGSGIK--LQHRHE-TAGIGDCTYKVDASCKGACTGHVQCDNERIYSKSAMTCS&#xA;ER-WWIDGSGRS-TFLHPR-TRNQK-TTY-KSYTDVINLWEY-LRYTHLEWH--VA-SCH&#xA;RAWHDEEEYHKHCM-CC-DVVQLMITEQQ&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I ran jobs on Uniprot for all these sequences. Here are the results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;www.uniprot.org/blast/uniprot/B20171003A7434721E10EE6586998A056CCD0537E082E0FZ&#xA;www.uniprot.org/blast/uniprot/B2017100383C3DD8CE55183C76102DC5D3A26728BEBF5BAE&#xA;www.uniprot.org/blast/uniprot/B20171003A7434721E10EE6586998A056CCD0537E082E5FP&#xA;www.uniprot.org/blast/uniprot/B20171003A7434721E10EE6586998A056CCD0537E082F08D&#xA;nothing&#xA;www.uniprot.org/blast/uniprot/B20171003F725F458AC8690F874DD868E4ED79B88CE32C4B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But how can I decide the identity of the protein and the name of the organism? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the first one, the organism is &lt;em&gt;Pacific ridley turtle&lt;/em&gt;  but in the second one it is Diplonema sp. ATCC 50224. So I can only decide at a kingdom level, but not to the organism, right?&lt;/p&gt;&#xA;" OwnerUserId="1567" LastEditorUserId="73" LastEditDate="2017-10-03T04:04:52.050" LastActivityDate="2017-10-03T16:32:05.147" Title="Uniprot: Determining protein and organism given a DNA sequence" Tags="|database|sequence-alignment|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2597" PostTypeId="2" ParentId="2595" CreationDate="2017-10-03T07:49:00.253" Score="1" Body="&lt;p&gt;Well if it is not a homework assignment, I would suggest to use blastx. Then you don't need to translate your DNA into protein first, but that will be covered by blastx.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is a homework assignment though, the teacher probably wants you to select the best blast hit. So which hit gives the highest score, and most matching (identical) amino acids between the alignment.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2017-10-03T07:49:00.253" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2598" PostTypeId="2" ParentId="2595" CreationDate="2017-10-03T10:40:49.387" Score="4" Body="&lt;p&gt;Well, let's see. The first step is to simply run a nucleotide blast (blastn) against the nr (non-redundant) database. I did this using the &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI blast tool&lt;/a&gt;. While, as expected, I got many hits, there was only one hit which matched the &lt;em&gt;entire length&lt;/em&gt; of the query sequence perfectly:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Diplonema sp. ATCC 50224 cytochrome c oxidase subunit 1 (cox1) mRNA, complete cds; mitochondrial&#xA;Sequence ID: JF698657.1&#xA;Length: 1612Number of Matches: 1&#xA;Range 1: 141 to 770&#xA;Alignment statistics for match #1&#xA;Score   Expect  Identities  Gaps    Strand&#xA;1164 bits(630)  0.0 630/630(100%)   0/630(0%)   Plus/Plus&#xA;Query  1    ACTGTTGTTCGGTGATCATCAGTTGTACAACGTCCTAACAACATCACATGCAATGCTTAT  60&#xA;            ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||&#xA;Sbjct  141  ACTGTTGTTCGGTGATCATCAGTTGTACAACGTCCTAACAACATCACATGCAATGCTTAT  200&#xA;&#xA;Query  61   GATATTCTTCTTCATCATGCCAGGCACGATGGCAGGACTAGGCAACTTACTAGTGCCATT  120&#xA;            ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||&#xA;Sbjct  201  GATATTCTTCTTCATCATGCCAGGCACGATGGCAGGACTAGGCAACTTACTAGTGCCATT  260&#xA;&#xA;Query  121  CCAGATGAGTGTACCGGAGTTAGTATTCCCAAAGATTAATAACATCGGTATATGATTTTT  180&#xA;            ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||&#xA;Sbjct  261  CCAGATGAGTGTACCGGAGTTAGTATTCCCAAAGATTAATAACATCGGTATATGATTTTT  320&#xA;&#xA;Query  181  AGTATGTGGTCTACTTTTGATTACGGGTTCATCTTGGATGGAGGAAGGTTCAGGAACGGC  240&#xA;            ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||&#xA;Sbjct  321  AGTATGTGGTCTACTTTTGATTACGGGTTCATCTTGGATGGAGGAAGGTTCAGGAACGGC  380&#xA;&#xA;Query  241  CTGAACCGTCTATCCACCACTAGCGCTCACTGCAAGTCATAGCGGACTTGCTGTAGATAC  300&#xA;            ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||&#xA;Sbjct  381  CTGAACCGTCTATCCACCACTAGCGCTCACTGCAAGTCATAGCGGACTTGCTGTAGATAC  440&#xA;&#xA;Query  301  GTTCATTATCGCATTGCACATGGCCGGTGCAAGCTCCCTTACAGGAAGCATCAACCTTAT  360&#xA;            ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||&#xA;Sbjct  441  GTTCATTATCGCATTGCACATGGCCGGTGCAAGCTCCCTTACAGGAAGCATCAACCTTAT  500&#xA;&#xA;Query  361  ATGTACAATCGCCTATGCCCGCCGTTCACTCATGGCGATGCTGCAGTCATCACTTTATCC  420&#xA;            ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||&#xA;Sbjct  501  ATGTACAATCGCCTATGCCCGCCGTTCACTCATGGCGATGCTGCAGTCATCACTTTATCC  560&#xA;&#xA;Query  421  CTGATCCATTACAATCACTGCAGCGTTACTCATAGGAGTTGTGCCTGTGCTAGCAGGTGC  480&#xA;            ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||&#xA;Sbjct  561  CTGATCCATTACAATCACTGCAGCGTTACTCATAGGAGTTGTGCCTGTGCTAGCAGGTGC  620&#xA;&#xA;Query  481  TATCACGATGCTACTCACTGATAGAAGTTGGAGTACCAGCTTCTATGACAGTTCGGCAGG  540&#xA;            ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||&#xA;Sbjct  621  TATCACGATGCTACTCACTGATAGAAGTTGGAGTACCAGCTTCTATGACAGTTCGGCAGG  680&#xA;&#xA;Query  541  CGGTGATCCTATGTTGTATCAGCACTTATTCTGGGTGTTTGGGCATCCAGAAGTCTATAT  600&#xA;            ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||&#xA;Sbjct  681  CGGTGATCCTATGTTGTATCAGCACTTATTCTGGGTGTTTGGGCATCCAGAAGTCTATAT  740&#xA;&#xA;Query  601  CATCATACTTCCAGTATTCGGTATAGTCAG  630&#xA;            ||||||||||||||||||||||||||||||&#xA;Sbjct  741  CATCATACTTCCAGTATTCGGTATAGTCAG  770&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, we now know the sequence is from the gene for mitochondrial Cytochrome C Oxidase subunit 1 from &lt;em&gt;Diplonema sp.&lt;/em&gt; ATCC 50224. That the sequence is mitochondrial, is very important since the mitochondrion of this species has a slightly different genetic code. If you go to the NCBI's page for this sequence (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/JF698657&quot; rel=&quot;nofollow noreferrer&quot;&gt;JF698657.1&lt;/a&gt;), you will see this line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/transl_table=4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That tells you it is using &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Taxonomy/Utils/wprintgc.cgi?mode=c#SG4&quot; rel=&quot;nofollow noreferrer&quot;&gt;translation table 4&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;4. The Mold, Protozoan, and Coelenterate Mitochondrial Code and the Mycoplasma/Spiroplasma Code (transl_table=4)&#xA;&#xA;    AAs  = FFLLSSSSYY**CCWWLLLLPPPPHHQQRRRRIIIMTTTTNNKKSSRRVVVVAAAADDEEGGGG&#xA;  Starts = --MM------**-------M------------MMMM---------------M------------&#xA;  Base1  = TTTTTTTTTTTTTTTTCCCCCCCCCCCCCCCCAAAAAAAAAAAAAAAAGGGGGGGGGGGGGGGG&#xA;  Base2  = TTTTCCCCAAAAGGGGTTTTCCCCAAAAGGGGTTTTCCCCAAAAGGGGTTTTCCCCAAAAGGGG&#xA;  Base3  = TCAGTCAGTCAGTCAGTCAGTCAGTCAGTCAGTCAGTCAGTCAGTCAGTCAGTCAGTCAGTCAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;See how all of your translated reading frames have in-frame stop codons? That's because you translated them using the standard genetic code instead of the correct mitochondrial one. As you can see above, in this code, the codon TGA is not a STOP, as in the standard code, and instead codes for tyrosine (W). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if you look at &lt;a href=&quot;http://www.uniprot.org/blast/uniprot/B2017100383C3DD8CE55183C76102DC5D3A26728BEBF5BAE?alignment=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;the result&lt;/a&gt; of the protein blast you ran on UniProt, you will see:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Query1   LLFGDHQLYNVLTTSHAMLMIFFFIMPGTMAGLGNLLVPFQMSVPELVFPKINNIGI-FL   59&#xA;         LLFGDHQLYNVLTTSHAMLMIFFFIMPGTMAGLGNLLVPFQMSVPELVFPKINNIGI FL&#xA;F5CI61   LLFGDHQLYNVLTTSHAMLMIFFFIMPGTMAGLGNLLVPFQMSVPELVFPKINNIGIWFL   107&#xA;&#xA;Query60  VCGLLLITGSSWMEEGSGTA-TVYPPLALTASHSGLAVDTFIIALHMAGASSLTGSINLI   118&#xA;         VCGLLLITGSSWMEEGSGTA TVYPPLALTASHSGLAVDTFIIALHMAGASSLTGSINLI&#xA;F5CI61   VCGLLLITGSSWMEEGSGTAWTVYPPLALTASHSGLAVDTFIIALHMAGASSLTGSINLI   167&#xA;&#xA;Query119 CTIAYARRSLMAMLQSSLYP-SITITAALLIGVVPVLAGAITMLLTDRSWSTSFYDSSAG   177&#xA;         CTIAYARRSLMAMLQSSLYP SITITAALLIGVVPVLAGAITMLLTDRSWSTSFYDSSAG&#xA;F5CI61   CTIAYARRSLMAMLQSSLYPWSITITAALLIGVVPVLAGAITMLLTDRSWSTSFYDSSAG   227&#xA;&#xA;Query178 GDPMLYQHLFWVFGHPEVYIIILPVFGIV                                  206&#xA;         GDPMLYQHLFWVFGHPEVYIIILPVFGIV&#xA;F5CI61   GDPMLYQHLFWVFGHPEVYIIILPVFGIV                                  256&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The only locations where there isn't a 100% identical match is where a W is aligned to a STOP (&lt;code&gt;-&lt;/code&gt; in the Uniprot output). So, your actual protein sequence is the 2nd frame of the 5' to 3' direction which, when translated using the correct code, is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;LLFGDHQLYNVLTTSHAMLMIFFFIMPGTMAGLGNLLVPFQMSVPELVFPKINNIGIWFL&#xA;VCGLLLITGSSWMEEGSGTAWTVYPPLALTASHSGLAVDTFIIALHMAGASSLTGSINLI&#xA;CTIAYARRSLMAMLQSSLYPWSITITAALLIGVVPVLAGAITMLLTDRSWSTSFYDSSAG&#xA;GDPMLYQHLFWVFGHPEVYIIILPVFGIV&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-10-03T16:32:05.147" LastActivityDate="2017-10-03T16:32:05.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2599" PostTypeId="2" ParentId="2594" CreationDate="2017-10-03T13:41:56.080" Score="5" Body="&lt;p&gt;&lt;strong&gt;The linear gap penalty&lt;/strong&gt; has a cost which is directly proportional to the length of the gap: the gap costs 1 unit per consecutive missing base.&lt;br&gt;&#xA;The simplest case is assigning each unit (= each missing base) a penalty of 1:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;g(k) = k&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Or more generally, if you want to increase the penalty per base to 2, 3 or n:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;g(k) = n * k&lt;br&gt;&#xA;  with k = length of the gap, and n = cost of 1 missing base&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The affine gap penalty&lt;/strong&gt; has a cost that follows -as its name conveniently states- an affine function.&lt;br&gt;&#xA;The idea behind it, is that any gap in an alignment has a cost with two components:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;the &quot;gap opening&quot; penalty&lt;/strong&gt;: it is a cost payed only once per gap, basically just for existing.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;the &quot;gap extension&quot; penalty:&lt;/strong&gt; equivalent to the linear gap penalty, it costs 1 unit for each consecutive missing base.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The general formula for the cost is then:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;g(k) = a + n*k&lt;br&gt;&#xA;  with a= opening penalty, n= extension penalty, k= length of the gap&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Compared to the linear gap penalty, this will give a small bonus to longer consecutive gaps compared to skipping one nucleotide, matching one, skipping one again etc (because in the case of longer gaps, you only pay the opening penalty once, so the global cost per base decreases).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Cells on your diagram:&lt;/strong&gt;&lt;br&gt;&#xA;From what I understand, you're working with an affine penalty of g(k) = 1 + 1*k. For the [3,4] you're &quot;coming&quot; from the score of 2 on the left (cell 1,2). You either:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;match the A (row 1) with G (column 3) and get a penalty of 1 for substitution -&gt; total of previous 2 + new 1 = 3&lt;/li&gt;&#xA;&lt;li&gt;open a gap to skip the G (column 3) and get a penalty of 2 (1 for opening + 1 because gap length of 1) -&gt; total of previous 2 + new 2 = 4.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So you choose the lesser penalty and go with 3, which is the number selected in the bigger box.&lt;br&gt;&#xA;The explanation is the exact same one for the other cell you mentioned. &lt;/p&gt;&#xA;" OwnerUserId="1452" LastEditorUserId="1452" LastEditDate="2017-10-03T14:42:31.397" LastActivityDate="2017-10-03T14:42:31.397" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2600" PostTypeId="2" ParentId="578" CreationDate="2017-10-03T23:20:00.667" Score="4" Body="&lt;p&gt;Getting back to the original question about using &lt;code&gt;taxize&lt;/code&gt; to search specifically in EOL, it looks like the most recent version of the package does not produce this error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current version of R I'm using (3.4.2) produces the following output:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;sci2comm( c(&quot;Cancer&quot;, &quot;Lophopanopeus&quot;), db=&quot;eol&quot; )&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$Cancer&#xA;   [1] &quot;En art taskekrabbe&quot;              &quot;Felsenkrabbe&quot;                    &quot;Atlantic rock crab&quot;&lt;br&gt;&#xA;   [4] &quot;Jaiba de roca amarilla&quot;          &quot;Tourteau poïnclos&quot;               &quot;Granciporro atlantico giallo&quot;&lt;br&gt;&#xA;   [7] &quot;Atlantische rotskrab&quot;            &quot;Sapateira de rocha do Atlântico&quot; &quot;Bogenkrabbe&quot;&lt;br&gt;&#xA;  [10] &quot;red rock crab&quot;                   &quot;Pacific rock crab&quot;               &quot;Jaiba del Pacífico&quot;&lt;br&gt;&#xA;  [13] &quot;Tourteau du Pacifique&quot;           &quot;Granciporro del Pacifico&quot;        &quot;Pacifische rotskrab&quot;&lt;br&gt;&#xA;  [16] &quot;Sapateira de rocha do Pacífico&quot;  &quot;Jonahkrabbe&quot;                     &quot;Jonah crab&quot;&lt;br&gt;&#xA;  [19] &quot;Jaiba de roca Jonás&quot;             &quot;Tourteau jona&quot;                   &quot;Grancipporo atlantico rosso&quot;&lt;br&gt;&#xA;  [22] &quot;Jonakrab&quot;                        &quot;Sapateira boreal&quot;                &quot;Bou&quot;&lt;br&gt;&#xA;  [25] &quot;Almindelig taskekrabbe&quot;          &quot;Taskekrabbe&quot;                     &quot;Taschenkrebs&quot;&lt;br&gt;&#xA;  [28] &quot;Edible crab&quot;                     &quot;rock crab&quot;                       &quot;Buey&quot;&lt;br&gt;&#xA;  [31] &quot;Buey de mar&quot;                     &quot;Jaiba de roca masera&quot;            &quot;Masera&quot;&lt;br&gt;&#xA;  [34] &quot;Paguro&quot;                          &quot;Pato&quot;                            &quot;Sabago&quot;&lt;br&gt;&#xA;  [37] &quot;Isotaskurapu&quot;                    &quot;Crabe dormeur&quot;                   &quot;Dormeur&quot;&lt;br&gt;&#xA;  [40] &quot;Tourteau&quot;                        &quot;crabe-dormeur&quot;                   &quot;Granchio di mare&quot;&lt;br&gt;&#xA;  [43] &quot;Granciporro&quot;                     &quot;Noordzeekrab&quot;                    &quot;grote krab&quot;&lt;br&gt;&#xA;  [46] &quot;grote noordzeekrab&quot;              &quot;hoofdkrab&quot;                       &quot;slagkrab&quot;&lt;br&gt;&#xA;  [49] &quot;steenkrab&quot;                       &quot;toogkrab&quot;                        &quot;zeekrab&quot;&lt;br&gt;&#xA;  [52] &quot;Sapateira&quot;                      &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$Lophopanopeus&#xA;  [1] &quot;blackclaw crestleg crab&quot; &quot;knobkneed crestleg crab&quot; &quot;knobknee crestleg crab&quot;  &quot;molarless crestleg crab&quot;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;$'Pugettia foliata'&#xA;  character(0)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A slightly older version of R (3.3.2) produces an error when one of the taxa was not found:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;sci2comm( c(&quot;Cancer&quot;, &quot;Lophopanopeus&quot;, &quot;Pugettia foliata&quot;), db=&quot;eol&quot; )&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Error in eol_pages(taxonconceptID = x, common_names = TRUE, ...) : &#xA;    Not Found (HTTP 404).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am very thankful for the package updates!&lt;/p&gt;&#xA;" OwnerUserId="1572" LastEditorUserId="1572" LastEditDate="2017-10-03T23:25:01.600" LastActivityDate="2017-10-03T23:25:01.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2601" PostTypeId="1" CreationDate="2017-10-04T00:31:22.883" Score="3" ViewCount="789" Body="&lt;p&gt;Starting with a chromosome # and position, I am trying to get chromStart and chromEnd values for a .bed file, but I am not sure how to calculate chromEnd when I have a variant that is an insertion or deletion.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My file is not a VCF file but it gives the chromosome number, chromosomal position (pos), reference (ref) allele, and alternate (alt) allele for each genetic variant, like you would find in a VCF file.  The tricky part is that not all of my variants are single nucleotide base changes--I also have insertions and deletions that can be multiple bases long.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For SNPs I am setting chromStart = pos - 1, and chromEnd = start + 1 = pos.  (Note, .bed format requires 0-based indexing and the end position is exclusive. See &lt;a href=&quot;https://genome.ucsc.edu/FAQ/FAQformat.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genome.ucsc.edu/FAQ/FAQformat.html&lt;/a&gt;) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am unsure of how to handle indels though (insertions and deletions).  Currently I am adding the length of the reference allele to the starting position to get the end position (chromEnd = start + len(ref)).  I am not sure if this is right though. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will be using this .bed file with a .bed file from Gencode and bedtools. Does anyone know the standard way the research community (or at least Gencode) treats indels when determining chromosomal position for .bed files? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1574" LastEditorUserId="1574" LastEditDate="2017-10-04T00:54:40.387" LastActivityDate="2017-10-06T22:02:32.183" Title="Dealing with indels when converting a position to chromStart and chromEnd for a bed file" Tags="|human-genome|bed|bedtools|gencode|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2602" PostTypeId="2" ParentId="2594" CreationDate="2017-10-04T01:00:11.117" Score="2" Body="&lt;p&gt;The formula your slide shows describes a dynamic programming with a general gap cost. Finding the optimal solution takes $O(n^3)$ time. No one is using the algorithm these days. This is the first time I see an attempt to fill a matrix with this algorithm. It is interesting, but I am not sure how it works. Although affine gap cost is a special case of general gap cost, the algorithm to find the best score is very different, so is the time complexity. With modern affine gap formulations, each cell holds three values:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The overall optimal score (or the optimal score if this cell is a match, which yields an equivalent but different formulation)&lt;/li&gt;&#xA;&lt;li&gt;The optimal score when the cell ends at a deletion&lt;/li&gt;&#xA;&lt;li&gt;The optimal score when the cell ends at an insertion&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The values in each cell only depends on the up, left and up-left cells in the matrix. You fill the matrix from the top left corner to the bottom right. This is an $O(n^2)$ algorithm, because you traverse each cell once. For more details, you can read &lt;a href=&quot;https://figshare.com/articles/Notes_on_pairwise_alignment_with_dynamic_programming/5223973&quot; rel=&quot;nofollow noreferrer&quot;&gt;this note&lt;/a&gt;. I would recommend to learn affine gap cost the normal way.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-10-04T01:00:11.117" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2603" PostTypeId="2" ParentId="2582" CreationDate="2017-10-04T01:27:16.643" Score="5" Body="&lt;p&gt;On (2), centromeres and telomeres primarily consist of &lt;a href=&quot;https://en.wikipedia.org/wiki/Satellite_DNA&quot; rel=&quot;noreferrer&quot;&gt;satellite&lt;/a&gt; sequences. In human centromeres, the dominant type is alpha satellite (aka alphoid). GGAAT micro-satellite (aka SATIII centromeric micro-satellite) is also very common. There are less common types such as beta satellite. Because there are all kinds of centromeric/telomeric sequences of different lengths and properties, RepeatMasker doesn't put them into one class. Note that RepeatMasker has a &quot;centr&quot; class but it does not include all centromeric satellites (no SATIII, for example). In general, it is not advised to find centromeres with RepeatMasker output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GRCh38 is an unusual reference genome in that its centromeric sequences are computationally generated modeled after Venter as I remember. They are mostly alpha satellite, but are not real sequences (which has always bugged me). GRC know exactly where they put these sequences. GRCh37 is a more typical reference genome. It lacks probably majority of alpha satellites, but still has many. When you map reads to GRCh37, you occasionally see ~10,000-fold read depth even though the average depth is ~30X. These regions are mostly alpha satellites. Mappers just pile all different copies of satellites onto the ones remained in GRCh37.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-10-04T01:27:16.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2604" PostTypeId="2" ParentId="2585" CreationDate="2017-10-04T05:53:01.870" Score="2" Body="&lt;p&gt;so while I am not so familiar with MATLAB, in the definition of the volcano plot it specifies that &lt;em&gt;PValues&lt;/em&gt; is supposed to be a column vector or a DataMatrix object. Your &lt;em&gt;c&lt;/em&gt; vector is a row vector, and this confuses MATLAB.&lt;/p&gt;&#xA;" OwnerUserId="787" LastActivityDate="2017-10-04T05:53:01.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2605" PostTypeId="1" CreationDate="2017-10-04T07:43:34.080" Score="5" ViewCount="214" Body="&lt;p&gt;The R package Biostrings has a function to create a pairwiseAlignment from pattern and subject sequences.&lt;/p&gt;&#xA;&lt;p&gt;So far I can save the result into a text file using writePairwiseAlignments. I would like to save the result into a SAM/BAM file, but couldn't find anything from the BioStrings package or any R package.&lt;/p&gt;&#xA;&lt;p&gt;What are the steps to convert the result from pairwiseAlignement into a SAM/BAM file?&lt;/p&gt;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-R prettyprint-override&quot;&gt;&lt;code&gt;library(Biostrings)&#xA;&#xA;patterns = DNAStringSet(c(&#xA;  'GCTCCCTGGCGGTAAGTTGATCAAAGGAACGCAAAGTTTTCAAG',&#xA;  'ACTTCACAGCTCTGGCGGTAAGTTGATCAAAGGAAACGCAAAGTTTTCAAG'&#xA;))&#xA;&#xA;subject = DNAString('ACTTCACCAGCTCCCTGGCGGTAAGTTGATCAAAGGAAACGCAAAGTTTTCAAG')&#xA;&#xA;pwa = pairwiseAlignment(patterns, subject)&#xA;&#xA;writePairwiseAlignments(pwa) # produces below&#xA;#=======================================&#xA;#&#xA;# Aligned_sequences: 2&#xA;# 1: P1&#xA;# 2: S1&#xA;# Matrix: NA&#xA;# Gap_penalty: 14.0&#xA;# Extend_penalty: 4.0&#xA;#&#xA;# Length: 54&#xA;# Identity:      44/54 (81.5%)&#xA;# Similarity:    NA/54 (NA%)&#xA;# Gaps:          10/54 (18.5%)&#xA;# Score: 27.19725&#xA;#&#xA;#&#xA;#=======================================&#xA;# P1 ---------GCTCCCTGGCGGTAAGTTGATCAAAGGAA-CGCAAAGTTTTCAAG&#xA;#             ||||||||||||||||||||||||||||| |||||||||||||||&#xA;# S1 ACTTCACCAGCTCCCTGGCGGTAAGTTGATCAAAGGAAACGCAAAGTTTTCAAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Essentially, I would like to convert the result from writePairwiseAlignments to a sam/bam format so I can view it in a GUI application. If I understand correctly, subject (referred from the Biostrings package) should be the reference, the patterns are the reads.&lt;/p&gt;&#xA;" OwnerUserId="1577" LastEditorUserId="73" LastEditDate="2021-07-04T04:45:39.020" LastActivityDate="2021-07-04T04:45:39.020" Title="R Biostrings pairwiseAlignment to BAM" Tags="|sequence-alignment|bioconductor|biostrings|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2607" PostTypeId="2" ParentId="21" CreationDate="2017-10-04T13:52:39.947" Score="1" Body="&lt;p&gt;While annotations between RefSeq and Gencode are not so different is the coding regions (genes), Gencode is much more rich in the intergenic regions. This could be very advantagous for epigenetic studies, where regulation is of interest.   &lt;/p&gt;&#xA;" OwnerUserId="1579" LastEditorUserId="73" LastEditDate="2017-10-05T11:56:57.857" LastActivityDate="2017-10-05T11:56:57.857" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2608" PostTypeId="2" ParentId="2585" CreationDate="2017-10-04T18:13:23.590" Score="2" Body="&lt;p&gt;Is there a reason why you are doing this in MATLAB instead of R?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In R with the ggplot2 package you can make a volcano plot pretty easily using something like this (may be errors as it is off the top of my head and untested):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# load libraries&#xA;library(ggplot2)&#xA;library(data.table)&#xA;&#xA;# create a dataframe/data.table of the data&#xA;a = c(1,2,3)&#xA;b = c(4.6, 2.7, 4.5)&#xA;c = c(0.05, 0.33, 0.45)&#xA;dt = data.table(A=a,B=b,pval=c)&#xA;&#xA;# add column to calculate log2 fold change between a/b&#xA;dt[,log2Diff := (log2(A)-log2(B))]&#xA;# add column to transform pvalue to -log scale&#xA;dt[,logPval := -log10(pval)]&#xA;&#xA;#make the plot&#xA;ggplot(dt, aes(x=log2Diff,y=logPval))+geom_point()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="39" LastActivityDate="2017-10-04T18:13:23.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2609" PostTypeId="2" ParentId="2585" CreationDate="2017-10-04T20:03:50.670" Score="3" Body="&lt;p&gt;I'm not sure if nowadays matlab is still a good option for informatics. I would rather go for a R script to make a volcano plot. If you don't want to have fancy code but functional one, you can even make the foldchange calculations and log2 transformation in excel and just plot the log2 fold change in x axis and the -log10 transform of the p/q values on the y axis.&#xA;If you need it, I could provide one of my own scripts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ymohs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ymohs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: here is the actual script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#load data into the 'res' variable&#xA;res &amp;lt;- read.table(&quot;genes.txt&quot;, header=TRUE)&#xA;head(res)&#xA;&#xA;# Make a basic volcano plot with grey dots&#xA;with(res, plot(log2FoldChange, -log10(qvalue), pch=19, col=&quot;#4A4E4D&quot;, xlab=&quot;Log2 Fold Change&quot;, xlim=c(-8,8)))&#xA;&#xA;# Add colored points: &quot;indianred&quot; if padj&amp;lt;0.0001, orange of log2FC&amp;gt;2, green if both)&#xA;with(subset(res, qvalue&amp;lt;.0001 ), points(log2FoldChange, -log10(qvalue), pch=19, col=&quot;#17B090&quot;))&#xA;with(subset(res, abs(log2FoldChange)&amp;gt;2), points(log2FoldChange, -log10(qvalue), pch=19, col=&quot;#F89538&quot;))&#xA;with(subset(res, qvalue&amp;lt;.0001 &amp;amp; abs(log2FoldChange)&amp;gt;2), points(log2FoldChange, -log10(qvalue), pch=19, col=&quot;#FA6367&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1568" LastEditorUserId="48" LastEditDate="2017-10-06T10:02:49.147" LastActivityDate="2017-10-06T10:02:49.147" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2611" PostTypeId="2" ParentId="2601" CreationDate="2017-10-06T22:02:32.183" Score="1" Body="&lt;p&gt;You can convert to VCF and use &lt;a href=&quot;http://bedops.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS&lt;/a&gt; &lt;code&gt;vcf2bed --insertions&lt;/code&gt;, &lt;code&gt;vcf2bed --deletions&lt;/code&gt; and &lt;code&gt;vcf2bed --snvs&lt;/code&gt; to segregate the VCF into three subsets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For convenience, BEDOPS conversion tools deal with index shifts, &lt;a href=&quot;http://bedops.readthedocs.io/en/latest/content/reference/file-management/conversion/vcf2bed.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;including &lt;code&gt;vcf2bed&lt;/code&gt;&lt;/a&gt;. The resulting BED file is 0-indexed, half-open.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2017-10-06T22:02:32.183" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2612" PostTypeId="1" CreationDate="2017-10-07T18:08:03.170" Score="4" ViewCount="184" Body="&lt;p&gt;I'm a computer scientist working with biologists at a small school that doesn't have dedicated bioinformatics staff. I apologize if I use incorrect terminology since I have limited bioinformatics background.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our biology staff is studying a soil sample that has been sequenced. The resulting sequences (in the hundreds of millions) were analyzed using mg-rast, but they didn't seem to have a high confidence in the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been asked to help analyze these sequences. I setup a local instance of NCBI blast, but it was far too slow to analyze that many sequences. We moved on to using BLAT. They've requested we use the nt database from NCBI. I converted the nt FASTA file to 2bit file, splitting the files into small chunks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;gfServer seems to have limitations however. When I run gfServer, I get the following error while gfServer attempts to start.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Exceeding 4 billion bases, sorry gfServer can't handle that.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Is it possible to use the full nt database using BLAT? Is there an alternative to BLAT that can handle that many sequences?&lt;/p&gt;&#xA;" OwnerUserId="1593" LastActivityDate="2017-10-08T09:03:03.213" Title="Using large databases with BLAT" Tags="|sequence-alignment|blast|metagenome|blat|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2613" PostTypeId="2" ParentId="2612" CreationDate="2017-10-08T09:03:03.213" Score="1" Body="&lt;p&gt;I'd recommend having a go with either &lt;a href=&quot;https://onecodex.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OneCodex&lt;/a&gt; (commercial / Freemium) or &lt;a href=&quot;http://ccb.jhu.edu/software/centrifuge/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Centrifuge&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chris Mason's lab has &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-017-1299-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;recently looked at metagenomic classification&lt;/a&gt; and OneCodex gave the most accurate result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Centrifuge uses &lt;a href=&quot;http://genome.cshlp.org/content/26/12/1721.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;Burrows-Wheeler lookups&lt;/a&gt;, but hasn't yet been compared by the Mason Lab.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-10-08T09:03:03.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2615" PostTypeId="1" CreationDate="2017-10-09T16:08:56.667" Score="5" ViewCount="265" Body="&lt;p&gt;Does anyone have a good method to find a signature where you combine the expression from multiple genes to predict a specific condition. &#xA;Assume you have performed sequencing of 48 cancer samples and 48 normal sample. Then find a combination of genes that predicts the cancer samples, using for instance AUC. &lt;/p&gt;&#xA;" OwnerUserId="1595" LastEditorUserId="1595" LastEditDate="2017-10-11T07:23:48.970" LastActivityDate="2017-11-11T10:20:44.730" Title="Gene expression signature (AUC) from gene expression data with 48 +48 samples" Tags="|r|rna-seq|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2616" PostTypeId="1" AcceptedAnswerId="2618" CreationDate="2017-10-09T17:55:19.570" Score="2" ViewCount="87" Body="&lt;p&gt;From what I was able to observe, the annotation files (CSV) for NetAffx Hg-U133 Plus 2.0 is updated regularly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, from glancing at the CSV files (NA35: 2015 &amp;amp; NA22: 2007), I can't seem to notice any glaring difference between the two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know what information is actually being updated? I assume it may be gene identifiers for different databases, but wanted to make sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below are the files I downloaded and examined from Affymetrix website (you need an account to download archived annotation files)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;HG-U133_Plus_2 Annotations, CSV format, Release 22 (20 MB, 5/31/07)&lt;/li&gt;&#xA;&lt;li&gt;HG-U133_Plus_2 Annotations, CSV format, Release 35 (33 MB, 4/16/15)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="711" LastActivityDate="2017-10-09T18:13:14.537" Title="Differences between NetAffx Hg-U133 Plus 2.0 Annotation file versions" Tags="|bioconductor|sequence-annotation|microarray|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2617" PostTypeId="1" AcceptedAnswerId="2622" CreationDate="2017-10-09T18:12:46.243" Score="4" ViewCount="1447" Body="&lt;p&gt;Is there a good way of obtaining the labels (e.g. genes) within individual clusters that haven been clustered hierarchically in Python (preferentially, but not necessarily, by seaborn)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found these difficulties highlighted elsewhere (&lt;a href=&quot;http://altanalyze.blogspot.com/2012/06/hierarchical-clustering-heatmaps-in.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, and &lt;a href=&quot;https://stackoverflow.com/questions/42470289/get-values-included-in-each-cluster&quot;&gt;here&lt;/a&gt;), and would hope for some simple option (e.g.: like in MATLAB where cutoff can be specified, and corresponding genes would be returned), without potential risk that the plotted hierarchical heatmap differed from the cluster assignment ( see &lt;a href=&quot;https://stackoverflow.com/questions/28687882/cutting-scipy-hierarchical-dendrogram-into-clusters-via-a-threshold-value&quot;&gt;here&lt;/a&gt;) &lt;/p&gt;&#xA;" OwnerUserId="357" LastEditorUserId="48" LastEditDate="2017-10-16T13:57:27.153" LastActivityDate="2017-10-16T13:57:27.153" Title="How to obtain clusters of hierarchical heatmap when using Python?" Tags="|python|clustering|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2618" PostTypeId="2" ParentId="2616" CreationDate="2017-10-09T18:13:14.537" Score="2" Body="&lt;p&gt;After checking the files myself, it does seem like gene identifiers indeed have been updated. Some that were missing have been added with identifiers and some have additional identifiers now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Among some other things that were updated from 2007 include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Genome version NCBI Version 36.1 =&gt; GRCh37&lt;/li&gt;&#xA;&lt;li&gt;Alignments&lt;/li&gt;&#xA;&lt;li&gt;Gene title&lt;/li&gt;&#xA;&lt;li&gt;Gene symbol&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thought this would be a good reference for those who may be looking into older researches involving microarrays. I will be sure to check the files more carefully before posting questions like this one.&lt;/p&gt;&#xA;" OwnerUserId="711" LastActivityDate="2017-10-09T18:13:14.537" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2619" PostTypeId="1" CreationDate="2017-10-09T18:30:54.410" Score="2" ViewCount="421" Body="&lt;p&gt;I am trying to find the number of silent mutations in for each position in codon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the &lt;strong&gt;homework&lt;/strong&gt;, I am asked:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For the codons and for each position, count the number of silent mutation. There are 64 codons, for each position there are 3 mutations leading to a change at the DNA level. There are thus 192 possible mutations for each of the three positions of the codons. For each position, print the number of silent mutations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I understand stand silent mutation is the mutation that doesn't lead to amino acid change. For example, &lt;strong&gt;GCC&lt;/strong&gt; can become &lt;strong&gt;GCT&lt;/strong&gt; but it is still &lt;strong&gt;Thr&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure what does &quot;calculating silent mutation for each position&quot; mean? I mean it is easy to find silent mutation for each codon like the example I gave above, but couldn't really understand how to calculate the silent mutation for each position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I change letters (nucleotides) one by one and see if it causing a silent mutation?&lt;/p&gt;&#xA;" OwnerUserId="1567" LastEditorUserId="77" LastEditDate="2017-10-09T18:39:12.367" LastActivityDate="2017-10-09T18:47:57.443" Title="How can I calculate the silent mutation for each position in codon?" Tags="|phylogenetics|codon|homework|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2620" PostTypeId="2" ParentId="2619" CreationDate="2017-10-09T18:47:57.443" Score="3" Body="&lt;p&gt;&lt;code&gt;Calculate&lt;/code&gt; in this context would just mean &lt;code&gt;determine&lt;/code&gt;. You don't actually have to change every base to determine whether a change is silent. Instead, try building a hash table of amino acids to codons, which you can then quickly iterate over to find all silent changes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, try to avoid loaded terms like &quot;mutation&quot;, even if your professor incorrectly used it. Terming things like this as &quot;silent polymorphism&quot; or &quot;silent change&quot; or something along those lines is preferable.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-10-09T18:47:57.443" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2621" PostTypeId="2" ParentId="2615" CreationDate="2017-10-09T18:51:06.700" Score="1" Body="&lt;p&gt;I have a method for doing this that uses a MCMC algorithm originally designed for population genetics (via the &lt;em&gt;structure&lt;/em&gt; program) to generate a statistic between 0 and 1 indicating genetic risk. This needs the use of reference populations for cases and controls that the query individuals are added to for classification. I use T1D as an example. My methods paper is not peer-reviewed, and probably doesn't go into sufficient detail for what you are looking for, but at least indicates one way this can be done:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biorxiv.org/content/early/2017/01/31/104497&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/early/2017/01/31/104497&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the work I've done with this, using 48 cases and 48 controls with a very specifically defined type of cancer should be sufficient to create a generalised signature from human populations. If a generic cancer profile is needed, the populations are not representative of a general human population, or there are some cases in the control group, then things get trickier and require more individuals. On the other hand, 1500 per group is probably overkill.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-11-11T10:20:44.730" LastActivityDate="2017-11-11T10:20:44.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2622" PostTypeId="2" ParentId="2617" CreationDate="2017-10-10T07:33:52.200" Score="6" Body="&lt;p&gt;Clustering like this is typically done with scipy. Here's the code we use in deepTools (original context &lt;a href=&quot;https://github.com/fidelram/deepTools/blob/master/deeptools/heatmapper.py#L1192-L1196&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.cluster.hierarchy import fcluster, linkage&#xA;Z = linkage(some_matrix)  # You might want to set `method` and `metric`&#xA;groups = fcluster(Z, nGroups)  # You might want to set `criterion`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;groups&lt;/code&gt; is then a vector containing the group assignment of the input matrix according to the clustering. As an aside, this is exactly the same as what's shown in the last link in your post, so I'm not sure how you didn't find it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, at least in python, hierarchical clustering is painfully slow with large datasets.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-10-10T07:33:52.200" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2623" PostTypeId="1" AcceptedAnswerId="3190" CreationDate="2017-10-11T15:48:25.090" Score="8" ViewCount="286" Body="&lt;p&gt;I want measure the hydrophobicity of each amino acid within a PDB structure file.  Since I have the PDB file I want to consider the 3D information, rather than sequence-only measures such as GRAVY.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not interested in measuring the SASA (solvent accessible surface area) or RD (residue depth) - although correlated, it is not the information I require.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I comparatively measure the hydrophobicity of the amino acids within a PDB structure file?  Is there an algorithm, library or program to do so?&lt;/p&gt;&#xA;" OwnerUserId="1606" LastActivityDate="2018-01-02T00:38:24.730" Title="How to measure or assign hydrophobicity score values to individual amino acids of a PDB structure?" Tags="|software-recommendation|pdb|3d-structure|data-mining|computation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2624" PostTypeId="2" ParentId="860" CreationDate="2017-10-11T16:15:21.360" Score="6" Body="&lt;p&gt;A popular (and I would say, respected and trusted) website is PDBsum &lt;a href=&quot;https://www.ebi.ac.uk/pdbsum&quot; rel=&quot;noreferrer&quot;&gt;https://www.ebi.ac.uk/pdbsum&lt;/a&gt; (which also has a Wikipedia article about it: &lt;a href=&quot;https://en.wikipedia.org/wiki/PDBsum&quot; rel=&quot;noreferrer&quot;&gt;https://en.wikipedia.org/wiki/PDBsum&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They measure protein-protein (prot-prot) contacts as any Nitrogen, Carbon, or Oxygen element atoms (N, C, O) within 4.0 Angstroms (Å) when measuring the 3d euclidean distance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your atomic contacts are not close enough, the amino acids may not actually interact at all, even though they theoretically could.  Since in PPIs you generally want to use reliable ground-truth information, it is better to use cautious parameters.  Often, in PPIs, the answer to &quot;does it interact?&quot; is not &quot;yes&quot; or &quot;no&quot; but rather &quot;yes&quot; or &quot;don't know / unknown&quot;.&lt;/p&gt;&#xA;" OwnerUserId="1606" LastActivityDate="2017-10-11T16:15:21.360" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2625" PostTypeId="2" ParentId="914" CreationDate="2017-10-11T18:22:13.730" Score="2" Body="&lt;p&gt;IBIS reports protein interaction with other biomolecules.  Your protein or homolog must exist in the Protein Data Bank.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IBIS @ NCBI: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Structure/ibis/ibis.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/Structure/ibis/ibis.cgi&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;For a given protein sequence or structure query, IBIS reports protein-protein, protein-small molecule, protein nucleic acids and protein-ion interactions observed in experimentally-determined structural biological assemblies. IBIS also infers/predicts interacting partners and binding sites by homology, by inspecting the protein complexes formed by close homologs of a given query. To ensure biological relevance of inferred binding sites, the IBIS algorithm clusters binding sites formed by homologs based on binding site sequence and structure conservation.&quot;&lt;/p&gt;&#xA;" OwnerUserId="1606" LastActivityDate="2017-10-11T18:22:13.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2626" PostTypeId="2" ParentId="2615" CreationDate="2017-10-11T23:14:27.220" Score="1" Body="&lt;p&gt;There are many different ways of doing it! I would recomend the well established R pacakge &lt;a href=&quot;http://topepo.github.io/caret/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;CARET&lt;/a&gt; which have a &lt;a href=&quot;https://topepo.github.io/caret/feature-selection-overview.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;whole chapter&lt;/a&gt; and many build in functions for it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition you probably want to do some low-level filtering based on variance and fold changes (or the combination - ad DE analysis) to get the number of features down before progressing to the more advanced methods.&lt;/p&gt;&#xA;" OwnerUserId="599" LastEditorUserId="73" LastEditDate="2017-11-11T10:19:34.903" LastActivityDate="2017-11-11T10:19:34.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2627" PostTypeId="1" CreationDate="2017-10-12T19:06:46.440" Score="-2" ViewCount="39" Body="&lt;p&gt;I am trying to download CopyNumber450k package in the R processor on my laptop in order to see if I could do a step of protocol for my research proffessor.  I was unable to successfully download the package I think (because it was too outdated for the newest version of R?) The version of R I have downloaded is 3.4.1. This package appears to work on the version 1.8.0 of R. I thought this was strange, as that seems really old. I then tried to see if this download was possible using some sort of e-version of an R-processor : &lt;a href=&quot;https://rdrr.io/bioc/CopyNumber450k/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rdrr.io/bioc/CopyNumber450k/&lt;/a&gt;   Even the e-version appeared to be unable to find and download this package of R. I have a feeling that I may be misinterpreting something? I tried to find an equivalent package that could run on the newer version of R and didn’t seem to find anything. Then, I tried to see if it was possible to download an old version of R. I couldn’t find a clear way to do that either. Does anyone on bioinformatics stack exchange have insight regarging what I may be doing wrong, something i may be misinterpreting, or how I may overcome this challenge?&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="1341" LastActivityDate="2017-10-13T07:16:41.770" Title="Downloading CopyNumber450k package from Bioconductor" Tags="|bioconductor|illumina|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2628" PostTypeId="1" CreationDate="2017-10-12T23:24:57.810" Score="5" ViewCount="786" Body="&lt;p&gt;I have some sorted and indexed alignment large BAM files. I'd like to read all the reads, and apply custom operation on each of them. The orders aren't important. I'm using &lt;code&gt;htslib&lt;/code&gt; in C++ to do the reading. I have a single powerful machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is ... reading a large BAM file is very slow even in C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q:&lt;/strong&gt; Is there any trick to speed up the reading? Reading BAM files multithreaded (I have many cores)?&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="174" LastEditDate="2017-10-13T00:35:30.090" LastActivityDate="2018-03-30T20:06:50.703" Title="Read sorted and indexed BAM files faster in C++?" Tags="|sam|c++|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2629" PostTypeId="2" ParentId="2628" CreationDate="2017-10-13T06:39:20.667" Score="2" Body="&lt;p&gt;In both &lt;a href=&quot;https://github.com/fidelram/deepTools&quot; rel=&quot;nofollow noreferrer&quot;&gt;deepTools&lt;/a&gt; and &lt;a href=&quot;https://github.com/dpryan79/MethylDackel&quot; rel=&quot;nofollow noreferrer&quot;&gt;MethylDackel&lt;/a&gt; we implement multithreading by having individual threads process separate genomic regions. At some point you hit an IO limit, but unless your processing is trivial then IO won't typically be limiting with single-threaded applications. Since you already have sorted/index BAM files, you can take advantage of the same strategy.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-10-13T06:39:20.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2630" PostTypeId="2" ParentId="2627" CreationDate="2017-10-13T07:16:41.770" Score="2" Body="&lt;p&gt;I think you need to explain better what you have tried, including some code. You say that you were unsuccessful to download it, because of an outdated version of R. How did you do that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure what you have tried, but if you read the instructions for installation &lt;a href=&quot;https://www.bioconductor.org/packages/3.0/bioc/html/CopyNumber450k.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, you need to type the following in R console:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;## try http:// if https:// URLs are not supported&#xA;source(&quot;https://bioconductor.org/biocLite.R&quot;) &#xA;biocLite(&quot;CopyNumber450k&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Show us what kind of error you get from this. Then we can maybe help you further.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2017-10-13T07:16:41.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2633" PostTypeId="2" ParentId="2438" CreationDate="2017-10-14T01:23:55.137" Score="4" Body="&lt;p&gt;Recently I've developed a python lib for converting InChI to InChIKey: &lt;a href=&quot;https://github.com/liwt31/chembl_ikey&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/liwt31/chembl_ikey&lt;/a&gt;. Maybe it will help.&lt;/p&gt;&#xA;" OwnerUserId="1621" LastEditorUserId="1621" LastEditDate="2017-10-18T05:07:44.617" LastActivityDate="2017-10-18T05:07:44.617" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2634" PostTypeId="1" AcceptedAnswerId="2635" CreationDate="2017-10-14T16:31:35.103" Score="7" ViewCount="4926" Body="&lt;p&gt;Given a list of gene IDs, how do you retrieve the gene description, summary and other detailed information in R?&lt;/p&gt;&#xA;" OwnerUserId="208" LastEditorUserId="298" LastEditDate="2017-10-15T12:46:55.897" LastActivityDate="2019-08-14T11:27:34.750" Title="Retrieve detailed gene descriptions" Tags="|r|gene|sequence-annotation|public-databases|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2635" PostTypeId="2" ParentId="2634" CreationDate="2017-10-14T16:31:35.103" Score="7" Body="&lt;h2&gt;MyGene.info&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Using the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/mygene.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;R package&lt;/a&gt; for the &lt;a href=&quot;http://mygene.info&quot; rel=&quot;nofollow noreferrer&quot;&gt;MyGene.info database&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;source(&quot;https://bioconductor.org/biocLite.R&quot;)&#xA;biocLite(&quot;mygene&quot;)&#xA;&#xA;library(mygene)&#xA;gene &amp;lt;- getGene(&quot;1017&quot;, fields=&quot;all&quot;)&#xA;gene$summary&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Multiple genes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gene &amp;lt;- getGenes(c(1347:1350), fields=&quot;summary&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Entrez database&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(devtools)&#xA;install_github(&quot;ropensci/rentrez&quot;)&#xA;&#xA;library(rentrez)&#xA;genes &amp;lt;- entrez_summary(db=&quot;gene&quot;, id=c(718,720))&#xA;genes$`720`$summary&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="208" LastEditorUserId="208" LastEditDate="2018-01-18T13:43:09.873" LastActivityDate="2018-01-18T13:43:09.873" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2636" PostTypeId="2" ParentId="2065" CreationDate="2017-10-15T11:43:28.523" Score="3" Body="&lt;p&gt;I would say that the following metrics are useful:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Sample source (Extraction place, person, date)&lt;/li&gt;&#xA;&lt;li&gt;Details about pre-HTS preparation (e.g. amplicon PCR, rRNA removal)&lt;/li&gt;&#xA;&lt;li&gt;Reads per sample (+ per barcode / lane, if available)&lt;/li&gt;&#xA;&lt;li&gt;Sequencing kit (e.g. TruSeq v3), useful for adapter trimming&lt;/li&gt;&#xA;&lt;li&gt;Sequence length (e.g. 125bp)&lt;/li&gt;&#xA;&lt;li&gt;Paired end / single end&lt;/li&gt;&#xA;&lt;li&gt;If paired end, target fragment length in sample prep (e.g. 650bp)&lt;/li&gt;&#xA;&lt;li&gt;Reference sequence metadata&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Scientific name&lt;/li&gt;&#xA;&lt;li&gt;Location of reference genome (preferably a stable web address)&lt;/li&gt;&#xA;&lt;li&gt;Version of reference genome&lt;/li&gt;&#xA;&lt;li&gt;Read mapping rate (per sample / barcode / lane), as a percentage of total reads&lt;/li&gt;&#xA;&lt;li&gt;Mapping program + version&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;If available, potential contaminating organisms&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Name / description&lt;/li&gt;&#xA;&lt;li&gt;Location of contaminant sequence (preferably a stable web address)&lt;/li&gt;&#xA;&lt;li&gt;Estimated mapping rate (bearing in mind that there may be some overlap with the target)&lt;/li&gt;&#xA;&lt;li&gt;Mapping program + version&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;More generally, ENA / EBI has an excellent reference in sample checklists, which indicate the minimum acceptable metadata for attaching to sequence archive data:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ebi.ac.uk/ena/submit/checklists&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/ena/submit/checklists&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-01-28T22:09:15.393" LastActivityDate="2018-01-28T22:09:15.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2637" PostTypeId="1" CreationDate="2017-10-15T15:50:16.317" Score="2" ViewCount="108" Body="&lt;p&gt;I plan on collecting some drop-seq data from brain tissue (from mice and humans). Using only the 75-85% of genes that have 1:1 orthology between mice and humans, I'd like to cluster the cells by transcriptomic similarity and get a dendrogram to use for cell type classification. There are tons of tools for processing and clustering single cell RNA-seq data (&lt;a href=&quot;https://www.scrna-tools.org/tools&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.scrna-tools.org/tools&lt;/a&gt;), but I get the impression that most of them were not designed with Drop-seq in mind. Anyone have any insight into the pros and cons of the best tools for Drop-seq analysis?&lt;/p&gt;&#xA;" OwnerUserId="1628" LastActivityDate="2022-01-19T10:02:06.957" Title="Getting hierarchy of cell populations with Drop-seq data" Tags="|rna-seq|clustering|single-cell|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2638" PostTypeId="1" CreationDate="2017-10-15T17:10:10.480" Score="5" ViewCount="1035" Body="&lt;p&gt;I am trying to make a phylogenetic tree from sequences obtained with blast. I have files that contains hundreds of fasta protein sequences which are all named like &lt;code&gt;&amp;gt;NP_567483.1 transcriptional regulator [Pseudomonas aeruginosa PAO1]&lt;/code&gt;. But if I use these files to do multiple alignments or phylogenetic trees using either ClustalX or MEGA7, every sequence are named after the first ID in the sequence names (for this example it would be &lt;code&gt;NP_567483.1&lt;/code&gt;), and what I want is to have the name of the strain instead (here &lt;code&gt;Pseudomonas aeruginosa PAO1&lt;/code&gt;) which is part of the sequence description. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I don't know whether the softwares I'm using understand that the information between the [ ] is the species and that I just didn't find where to set this as the name or if I have to write some kind of a script to change my fasta files before. I'm really new to this and don't know much about it. Thanks for your help in advance. Julian&lt;/p&gt;&#xA;" OwnerUserId="1629" LastEditorUserId="298" LastEditDate="2017-10-16T13:56:47.797" LastActivityDate="2017-10-18T02:12:59.017" Title="How can I only get the species name for fasta sequences from blast results?" Tags="|fasta|blast|sequence-alignment|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2639" PostTypeId="2" ParentId="2638" CreationDate="2017-10-15T19:39:57.407" Score="3" Body="&lt;p&gt;There might be a &quot;cleaner&quot; way to do this, but what I would do is first remove all of the spaces from the fasta sequence names and replace them with underscores using something like sed.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed 's/ /_/g'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Run your BLAST after you remove the spaces and you will get the whole ID string in the result instead of only that first part (e.g. &lt;code&gt;NP_567483.1&lt;/code&gt;). From here you can parse out the species name, or whatever other ID you would like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this is helpful! :)&lt;/p&gt;&#xA;" OwnerUserId="824" LastEditorUserId="824" LastEditDate="2017-10-15T20:31:39.220" LastActivityDate="2017-10-15T20:31:39.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2640" PostTypeId="2" ParentId="2567" CreationDate="2017-10-15T21:36:09.357" Score="1" Body="&lt;p&gt;I assume you are mapping against the genome rather the transcriptome, since for the later the length would be trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming the first, I think not only the coding sections should be included but also the UTR, since reads can map against them which is what we ultimately care about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, I found gene annotation files (e.g. gff or gtf) can be inconsistent in terms of naming, so it's good practice to inspect and double check. Below is some R code to import the annotation and calculate isoform lengths:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(rtracklayer)&#xA;# Reading into a GRanges object&#xA;anno &amp;lt;- import.gff3(&quot;annotation.gff&quot;)&#xA;# Filtering exons and UTRs&#xA;exons &amp;lt;- anno[anno@elementMetadata$type %in% c(&quot;exon&quot;,&quot;five_prime_UTR&quot;,&quot;three_prime_UTR&quot;),]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Depending on the annotation at hand, the most sensible is probably best to count the length of each isoform which are often contained in the &quot;Parent&quot; column of the annotation file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# splitting up isoforms as preparation for the next step&#xA;tmp &amp;lt;- split(exons,as.character(exons$Parent))&#xA;# for each isoform, calculate the sum of all reduced exons&#xA;Gene_length &amp;lt;- sum(width(reduce(tmp)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note, reduce merges overlapping intervals together, since UTRs can &quot;contain&quot; bits of exons which would be otherwise double counted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This code can of course be adapted mainly by changing the &quot;Parent&quot;, &quot;exon&quot; etc.&lt;/p&gt;&#xA;" OwnerUserId="1461" LastActivityDate="2017-10-15T21:36:09.357" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2641" PostTypeId="2" ParentId="2638" CreationDate="2017-10-16T04:23:31.660" Score="1" Body="&lt;p&gt;You can replace fasta header of each sequence with the string that is inside the brackets using awk:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -F '[][]' '{if ($0~/&amp;gt;/) {print &quot;&amp;gt;&quot;$2} else {print $0}}' input.fasta &amp;gt; output.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For lines starting with &lt;code&gt;&amp;gt;&lt;/code&gt; (fasta header), it replaces the line with the string inside the brackets and for other lines (which contain the sequence), it prints the line without making any changes.&lt;/p&gt;&#xA;" OwnerUserId="1392" LastActivityDate="2017-10-16T04:23:31.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2642" PostTypeId="2" ParentId="2634" CreationDate="2017-10-16T09:44:18.097" Score="8" Body="&lt;p&gt;Description of how to use biomaRt &lt;a href=&quot;http://www.ensembl.org/info/data/biomart/biomart_r_package.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. Filter by your list of gene IDs. Get the descriptions etc as attributes. Eg:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; library(biomaRt)&#xA;&amp;gt; ensembl = useEnsembl(biomart=&quot;ensembl&quot;, dataset=&quot;hsapiens_gene_ensembl&quot;)&#xA;&amp;gt; IDs &amp;lt;- c(&quot;BRCA2&quot;,&quot;BRAF&quot;)&#xA;genedesc &amp;lt;- getBM(attributes=c('external_gene_name','description'), filters = 'external_gene_name', values = IDs, mart =ensembl)&#xA;&amp;gt; genedesc&#xA;  external_gene_name                                                                      description&#xA;1               BRAF B-Raf proto-oncogene, serine/threonine kinase [Source:HGNC Symbol;Acc:HGNC:1097]&#xA;2              BRCA2                  BRCA2, DNA repair associated [Source:HGNC Symbol;Acc:HGNC:1101]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2017-10-16T09:44:18.097" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2643" PostTypeId="1" CreationDate="2017-10-16T17:01:25.237" Score="2" ViewCount="2300" Body="&lt;p&gt;I encoutered some technical problem when learning BLAST as below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After I use &lt;code&gt;conda install blast&lt;/code&gt; to install ncbi-blast, and run the command:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;update_blastdb.pl --decompress 16SMicrobial&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the shell complains: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can't locate List/MoreUtils.pm in @INC (you may need to install the&#xA;  List::MoreUtils module) (@INC contains: /etc/perl /usr/loc&#xA;  al/lib/x86_64-linux-gnu/perl/5.22.1 /usr/local/share/perl/5.22.1&#xA;  /usr/lib/x86_64-linux-gnu/perl5/5.22 /usr/share/perl5 /usr/li&#xA;  b/x86_64-linux-gnu/perl/5.22 /usr/share/perl/5.22&#xA;  /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base .) at&#xA;  /home/mck f111/miniconda3/envs/bioinfo/bin/update_blastdb.pl line 43.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;After I run this command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cpanm List::MoreUtils&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It appears some installation happened, however, the complains are still there.&#xA;Could anyone tell me how to fix this? Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="1634" LastEditorUserId="96" LastEditDate="2017-10-16T21:10:38.060" LastActivityDate="2017-10-16T21:10:38.060" Title="Error &quot;Can't locate List/MoreUtils.pm in @INC&quot; when I run update_blastdb.pl" Tags="|blast|perl|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2644" PostTypeId="1" AcceptedAnswerId="2646" CreationDate="2017-10-16T19:23:48.613" Score="6" ViewCount="14947" Body="&lt;p&gt;I want to download all SRA file from the following &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra?linkname=bioproject_sra_all&amp;amp;from_uid=201245&quot; rel=&quot;nofollow noreferrer&quot;&gt;project&lt;/a&gt;. Is there a method to download all the SRA files at the same time?&lt;/p&gt;&#xA;" OwnerUserId="1595" LastEditorUserId="48" LastEditDate="2017-10-17T14:21:54.980" LastActivityDate="2021-04-21T12:14:23.567" Title="Download multiple SRA files" Tags="|rna-seq|data-download|" AnswerCount="7" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2645" PostTypeId="2" ParentId="2644" CreationDate="2017-10-16T23:35:59.217" Score="3" Body="&lt;p&gt;Assuming you ultimately just want the fastq files and you know the SRR (run) numbers, I would download them from here: &lt;a href=&quot;ftp://ftp.sra.ebi.ac.uk/vol1/fastq/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.sra.ebi.ac.uk/vol1/fastq/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for downloading multiple files, I've just used multiple wget commands. I don't know of a way to download all of the files together in like a zipped folder or anything :/&lt;/p&gt;&#xA;" OwnerUserId="1628" LastActivityDate="2017-10-16T23:35:59.217" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2646" PostTypeId="2" ParentId="2644" CreationDate="2017-10-17T07:46:10.763" Score="4" Body="&lt;p&gt;A quick look at your link tells me the SRR numbers run from SRR837819 to SRR837856. You can use fastq-dump from the sratoolkit, and make a for loop around it in bash.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something like this should work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (( i = 19; i &amp;lt;= 56; i++ ))&#xA;  do&#xA;  fastq-dump --accession SRR8378$i&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After reading &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/77/devon-ryan&quot;&gt;Devon Ryan&lt;/a&gt;'s answer, I realize that you asked for SRA files instead of fastq. This can also be done with prefetch of the sratoolkit:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (( i = 19; i &amp;lt;= 56; i++ ))&#xA;  do&#xA;  prefetch SRR8378$i&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2017-10-17T09:48:28.950" LastActivityDate="2017-10-17T09:48:28.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2647" PostTypeId="2" ParentId="2644" CreationDate="2017-10-17T08:19:27.760" Score="3" Body="&lt;p&gt;I suggest you follow the advice in &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/1628/eric-a-brenner&quot;&gt;Eric A Brenner's&lt;/a&gt; answer and just download the fastq files. However, if you really really want to use the SRA files for some reason, note that you can use &lt;a href=&quot;https://github.com/rvalieris/parallel-fastq-dump&quot; rel=&quot;nofollow noreferrer&quot;&gt;parallel-fastq-dump&lt;/a&gt; to make things faster. Do follow its advice regarding using &lt;code&gt;prefetch&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'd need to combine that with the answer from &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/939/b-nota&quot;&gt;b.nota&lt;/a&gt; (i.e., put the commands in a &lt;code&gt;for&lt;/code&gt; loop).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-10-17T08:19:27.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2648" PostTypeId="1" CreationDate="2017-10-17T10:48:25.670" Score="5" ViewCount="72" Body="&lt;p&gt;I've made an application that at this point ranks all combinations of drug pairs relevant to a biological network/graph in the order of how disruptive the outcome of deleting the targets of a given drug pair is. So, in short, my applications objective is to kill/disrupt the network, but the optimal goal would be to disrupt nodes related to disease and have minimal impact on the functional part of the cell/network. I've read a few articles about network medicine and found that a diffusion-based method for identifying disease modules seems easier to understand coming from an informatics background.&#xA;I know how to implement the algorithm for random walkers with reset algorithm (RWR) but unsure how to get/ identify known disease associated genes/nodes of the network and these are used as seeds for the algorithm.&lt;/p&gt;&#xA;&lt;p&gt;If you guys have some advice on how to do this or even know of a better way to identify disease, functional and topological modules I would appreciate it.&lt;/p&gt;&#xA;&lt;p&gt;Some details of my project:&#xA;Application developed to run in Cytoscape, programmed in Java. Using Jung and JGraphT for graph algorithms.&lt;/p&gt;&#xA;&lt;p&gt;This is for exploring paired drug combinations for treating cancer. I retrieve drug-target relation from a refined list published in an article called: &amp;quot;A comprehensive map of molecular drug targets&amp;quot; from December.&lt;/p&gt;&#xA;" OwnerUserId="1633" LastEditorUserId="73" LastEditDate="2021-05-06T10:10:44.527" LastActivityDate="2021-05-06T10:10:44.527" Title="Help with identifying disease modules" Tags="|phylogenetics|phylogeny|algorithms|" AnswerCount="0" CommentCount="0" ClosedDate="2021-05-12T02:13:39.747" ContentLicense="CC BY-SA 4.0" />
  <row Id="2649" PostTypeId="1" AcceptedAnswerId="2651" CreationDate="2017-10-17T11:00:09.670" Score="17" ViewCount="9505" Body="&lt;p&gt;I have a fasta file like &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;sample 1 gene 1&#xA;atgc&#xA;&amp;gt;sample 1 gene 2&#xA;atgc&#xA;&amp;gt;sample 2 gene 1 &#xA;atgc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to get the following output, with one break between the header and the sequence.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;sample 1 gene 1   atgc&#xA;&amp;gt;sample 1 gene 2   atgc&#xA;&amp;gt;sample 2 gene 1   atgc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="982" LastActivityDate="2023-05-14T14:00:01.083" Title="How to convert fasta file to tab delimited file" Tags="|fasta|format-conversion|" AnswerCount="12" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2650" PostTypeId="2" ParentId="2649" CreationDate="2017-10-17T11:53:41.097" Score="9" Body="&lt;p&gt;assuming there is only one sequence line per record, use paste with two 'stdin'&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat your.fasta | paste - -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastActivityDate="2017-10-17T11:53:41.097" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2651" PostTypeId="2" ParentId="2649" CreationDate="2017-10-17T12:21:47.597" Score="13" Body="&lt;p&gt;If you have multi-line fasta files, as is very common, you can use these scripts&lt;sup&gt;1&lt;/sup&gt; to convert between fasta and tbl (sequence_name &lt;code&gt;&amp;lt;TAB&amp;gt;&lt;/code&gt; sequence) format:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;FastaToTbl&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/awk -f&#xA;{&#xA;        if (substr($1,1,1)==&quot;&amp;gt;&quot;)&#xA;    		if (NR&amp;gt;1)&#xA;                    	printf &quot;\n%s\t&quot;, substr($0,2,length($0)-1)&#xA;    		else &#xA;    			printf &quot;%s\t&quot;, substr($0,2,length($0)-1)&#xA;            else &#xA;                    printf &quot;%s&quot;, $0&#xA;}END{printf &quot;\n&quot;}&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;TblToFasta&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;#! /usr/bin/awk -f&#xA;{&#xA;  sequence=$NF&#xA;&#xA;  ls = length(sequence)&#xA;  is = 1&#xA;  fld  = 1&#xA;  while (fld &amp;lt; NF)&#xA;  {&#xA;     if (fld == 1){printf &quot;&amp;gt;&quot;}&#xA;     printf &quot;%s &quot; , $fld&#xA;&#xA;     if (fld == NF-1)&#xA;      {&#xA;        printf &quot;\n&quot;&#xA;      }&#xA;      fld = fld+1&#xA;  }&#xA;  while (is &amp;lt;= ls)&#xA;  {&#xA;    printf &quot;%s\n&quot;, substr(sequence,is,60)&#xA;    is=is+60&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Save those in your &lt;code&gt;$PATH&lt;/code&gt;, make them executable, and you can then do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat file.fa&#xA;&amp;gt;sequence1 &#xA;ATGCGGAGCTTAGATTCTCGAGATCTCGATATCGCGCTTATAAAAGGCCCGGATTAGGGC&#xA;TAGCTAGATATCGCGATAGCTAGGGATATCGAGATGCGATACG&#xA;&amp;gt;sequence2 &#xA;GTACTCGATACGCTACGCGATATTGCGCGATACGCATAGCTAACGATCGACTAGTGATGC&#xA;ATAGAGCTAGATCAGCTACGATAGCATCGATCGACTACGATCAGCATCAC&#xA;$ FastaToTbl file.fa &#xA;sequence1   ATGCGGAGCTTAGATTCTCGAGATCTCGATATCGCGCTTATAAAAGGCCCGGATTAGGGCTAGCTAGATATCGCGATAGCTAGGGATATCGAGATGCGATACG&#xA;sequence2   GTACTCGATACGCTACGCGATATTGCGCGATACGCATAGCTAACGATCGACTAGTGATGCATAGAGCTAGATCAGCTACGATAGCATCGATCGACTACGATCAGCATCAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And, to get the Fasta back:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ FastaToTbl file.fa | TblToFasta&#xA;&amp;gt;sequence1 &#xA;ATGCGGAGCTTAGATTCTCGAGATCTCGATATCGCGCTTATAAAAGGCCCGGATTAGGGC&#xA;TAGCTAGATATCGCGATAGCTAGGGATATCGAGATGCGATACG&#xA;&amp;gt;sequence2 &#xA;GTACTCGATACGCTACGCGATATTGCGCGATACGCATAGCTAACGATCGACTAGTGATGC&#xA;ATAGAGCTAGATCAGCTACGATAGCATCGATCGACTACGATCAGCATCAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This can be a very useful trick when searching a fasta file for a string:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TblToFasta file.fa | grep 'foo' | FastaToTbl&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If you really want to keep the leading &lt;code&gt;&amp;gt;&lt;/code&gt; of the header (which doesn't seem very useful), you could do something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ perl -0pe 's/\n//g; s/.&amp;gt;/\n&amp;gt;/g; s/$/\n/;' file.fa &#xA;&amp;gt;sequence1 ATGCGGAGCTTAGATTCTCGAGATCTCGATATCGCGCTTATAAAAGGCCCGGATTAGGGCTAGCTAGATATCGCGATAGCTAGGGATATCGAGATGCGATAC&#xA;&amp;gt;sequence2 GTACTCGATACGCTACGCGATATTGCGCGATACGCATAGCTAACGATCGACTAGTGATGCATAGAGCTAGATCAGCTACGATAGCATCGATCGACTACGATCAGCATCAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But that will read the entire file into memory. If that's an issue, add an empty line between each fasta record, and then use perl's paragraph mode to process each &quot;paragraph&quot; (sequence) at a time:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl -pe  's/&amp;gt;/\n&amp;gt;/' file.fa | perl -00pe 's/\n//g; s/.&amp;gt;/\n&amp;gt;/g; s/$/\n/;'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;1&lt;/sup&gt;&lt;sub&gt;Credit to Josep Abril who wrote these scripts more than a decade ago.&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-07-09T17:20:30.093" LastActivityDate="2018-07-09T17:20:30.093" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="2652" PostTypeId="2" ParentId="2637" CreationDate="2017-10-17T14:17:54.110" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://bmcbiol.biomedcentral.com/articles/10.1186/s12915-017-0383-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; article uses the freely available R package &lt;a href=&quot;https://github.com/rajewsky-lab/dropbead&quot; rel=&quot;nofollow noreferrer&quot;&gt;dropbead&lt;/a&gt; for filtering and then &lt;a href=&quot;https://www.nature.com/nbt/journal/v33/n5/full/nbt.3192.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seurat&lt;/a&gt; to perform a principal component analysis that groups together affine transcriptomes. It could be what you are looking for. &lt;/p&gt;&#xA;" OwnerUserId="1637" LastActivityDate="2017-10-17T14:17:54.110" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2653" PostTypeId="2" ParentId="2649" CreationDate="2017-10-17T14:20:51.570" Score="2" Body="&lt;p&gt;Remove empty records (description without sequence):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '$2{print RS}$2' FS='\n' RS=\&amp;gt; ORS= f1.fa &amp;gt; f2.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Remove blank lines:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed '/^$/d' f2.fa &amp;gt; f3.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Convert multi-line fasta to single-line fasta:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '/^&amp;gt;/ {printf(&quot;\n%s\n&quot;,$0);next; } { printf(&quot;%s&quot;,$0);} END {printf(&quot;\n&quot;);}' f3.fa &amp;gt; f4.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, @Pierre solution:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat f4.fa | paste - - &amp;gt; f.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="35" LastActivityDate="2017-10-17T14:20:51.570" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2654" PostTypeId="1" CreationDate="2017-10-17T15:46:47.770" Score="3" ViewCount="1140" Body="&lt;p&gt;I have a heatmap and I would like to find some rectangles.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/eUJrR.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eUJrR.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already used &lt;a href=&quot;http://seaborn.pydata.org/generated/seaborn.clustermap.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;clustermap&lt;/a&gt;. But here, I can not calculate these rectangles.&#xA;The order of the data should not be changed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This Code is not working: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;import matplotlib.pyplot as plt&#xA;import seaborn as sns&#xA;tips2 = pd.read_csv('heatmapExport.csv')&#xA;sns.set(color_codes=True)&#xA;g = sns.clustermap(tips2, cbar=&quot;false&quot;)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does somebody have an idea?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example of what I want:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3tgrc.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3tgrc.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1638" LastEditorUserId="292" LastEditDate="2017-10-19T10:55:28.067" LastActivityDate="2019-01-30T18:04:59.163" Title="Find Patterns in Cluster" Tags="|python|clustering|" AnswerCount="1" CommentCount="15" ContentLicense="CC BY-SA 3.0" />
  <row Id="2655" PostTypeId="1" AcceptedAnswerId="2660" CreationDate="2017-10-17T16:42:25.970" Score="2" ViewCount="172" Body="&lt;p&gt;There are 394 &lt;a href=&quot;https://www.genenames.org/cgi-bin/genefamilies/set/471&quot; rel=&quot;nofollow noreferrer&quot;&gt;CD marker genes&lt;/a&gt;. Various resources (&lt;a href=&quot;https://www.bdbiosciences.com/documents/cd_marker_handbook.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;BD handbook&lt;/a&gt;, &lt;a href=&quot;http://www.abcam.com/primary-antibodies/human-cd-antigen-guide&quot; rel=&quot;nofollow noreferrer&quot;&gt;Abcam guide&lt;/a&gt;) contain information about which cell types express which CD markers. What is the best way to obtain or download this data so that it can be used with programs such as &lt;code&gt;R&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can we download similar annotation data on expression for any gene or cell type?&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2017-10-18T06:33:57.653" Title="Obtain CD (cluster of differentiation) marker expression information" Tags="|r|sequence-annotation|gene-ontology|flow-cytometry|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2656" PostTypeId="2" ParentId="2649" CreationDate="2017-10-17T16:45:11.120" Score="4" Body="&lt;p&gt;Where possible, I recommend using a dedicated parsing library, rather than hacking a parser together: as you can see in the other answers, parsing even simple formats gets complex pretty quickly if you value correctness.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here’s a small R script that does what we need, using &lt;a href=&quot;https://cran.r-project.org/package=seqinr&quot; rel=&quot;nofollow noreferrer&quot;&gt;‘seqinr’&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env Rscript&#xA;suppressPackageStartupMessages(library(seqinr))&#xA;parsed = read.fasta(file('stdin'), as.string = TRUE)&#xA;table = data.frame(unlist(parsed), row.names = sapply(parsed, attr, 'Annot'))&#xA;write.table(table, stdout(), sep = '\t', quote = FALSE, col.names = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Save it as &lt;code&gt;fasta-to-tsv&lt;/code&gt;, make it executable, and use it as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fasta-to-tsv &amp;lt; input.fasta &amp;gt; output.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Equivalent code of similar length can be written in Python or Perl.&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2020-06-14T19:28:40.803" LastActivityDate="2020-06-14T19:28:40.803" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="2658" PostTypeId="2" ParentId="2649" CreationDate="2017-10-17T21:43:19.270" Score="12" Body="&lt;p&gt;There is a very simple &lt;a href=&quot;http://biopython.org/&quot; rel=&quot;noreferrer&quot;&gt;BioPython&lt;/a&gt; solution, that is minimal, readable, and handles multi-line fasta:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;for record in SeqIO.parse('example.fa', 'fasta'):&#xA;    print('&amp;gt;{}\t{}'.format(record.description, record.seq))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastActivityDate="2017-10-17T21:43:19.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2659" PostTypeId="2" ParentId="2638" CreationDate="2017-10-18T02:12:59.017" Score="1" Body="&lt;p&gt;With &lt;code&gt;sed&lt;/code&gt;, first test it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;echo '&amp;gt;NP_567483.1 transcriptional regulator [Pseudomonas aeruginosa PAO1]' | sed &quot;s/&amp;gt;.*\[\(.*\)\]/&amp;gt;\1/&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then modify your fasta:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -i.bak &quot;s/&amp;gt;.*\[\(.*\)\]/&amp;gt;\1/&quot; file.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1642" LastActivityDate="2017-10-18T02:12:59.017" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2660" PostTypeId="2" ParentId="2655" CreationDate="2017-10-18T03:12:21.230" Score="5" Body="&lt;p&gt;I can offer this as a way to scrape the data from the Abcam resource:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(rvest)&#xA;library(dplyr)&#xA;&#xA;antigens &amp;lt;- read_html(&quot;http://www.abcam.com/primary-antibodies/human-cd-antigen-guide&quot;) %&amp;gt;% &#xA;  html_node(&quot;table&quot;) %&amp;gt;% &#xA;  html_table(header = TRUE)&#xA;&#xA;# what targets expressed in &quot;Langerhans&quot; ?&#xA;antigens %&amp;gt;% &#xA;  filter(grepl(&quot;Langerhans&quot;, `Cellular Expression`)) %&amp;gt;% &#xA;  select(`Target Name`)&#xA;&#xA;   Target Name&#xA;1         CD1a&#xA;2         CD1b&#xA;3         CD1c&#xA;4         CD1e&#xA;5         CD14&#xA;6         CD39&#xA;7         CD50&#xA;8         CD69&#xA;9         CD74&#xA;10        CD83&#xA;11       CD101&#xA;12       CD206&#xA;13       CD207&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The data could be further processed by, for example, splitting the &lt;code&gt;Cellular Expression&lt;/code&gt; column on either &lt;code&gt;and&lt;/code&gt; or &lt;code&gt;,&lt;/code&gt; (comma) to try and make one row per cell type.&lt;/p&gt;&#xA;" OwnerUserId="150" LastEditorUserId="150" LastEditDate="2017-10-18T06:33:57.653" LastActivityDate="2017-10-18T06:33:57.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2663" PostTypeId="1" AcceptedAnswerId="2664" CreationDate="2017-10-19T02:32:40.177" Score="4" ViewCount="508" Body="&lt;p&gt;I'm planning on using Oxford Nanopore MinION sequencer to test a script that I'm working on. However, I would not like to run multiple MinION runs to validate the tool. I'd like to simulate various minION runs and get raw reads and all pore events in a fast5 file.  Do you know of any Oxford Nanopore (MinION) sequencing simulator that allows to get fast5 as output? &#xA;I looked into &lt;a href=&quot;https://github.com/bcgsc/NanoSim&quot; rel=&quot;nofollow noreferrer&quot;&gt;NanoSim&lt;/a&gt; but it seems like there is no option to get fast5 as an output.&lt;/p&gt;&#xA;" OwnerUserId="1421" LastEditorUserId="1421" LastEditDate="2017-10-19T02:51:50.680" LastActivityDate="2018-04-08T06:07:19.657" Title="Simulate fast5 MinION run" Tags="|minion|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2664" PostTypeId="2" ParentId="2663" CreationDate="2017-10-19T04:25:58.907" Score="1" Body="&lt;p&gt;Using a simulator for example MinION output doesn't make sense when there are an abundance of freely-available FASTQ and FAST5 sequences for general purpose use (and especially for the purposes of testing out algorithms). Simulation of sequences can only ever express the components of DNA that are explicitly modelled, which is never going to fully represent reality.&lt;/p&gt;&#xA;&lt;p&gt;As long as you're doing your own private research, you can use the FAST5 sequences from Clive Brown's own genome:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/nanoporetech/ONT-HG1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nanoporetech/ONT-HG1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Bear in mind the license attached to that data:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/nanoporetech/ONT-HG1/blob/master/LICENSE.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nanoporetech/ONT-HG1/blob/master/LICENSE.md&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;License to use for research use only (i.e. non-commercial) governed by the law of England and Wales.&lt;/p&gt;&#xA;&lt;p&gt;You agree to this License by downloading the data on this site whether in whole or in part.&lt;/p&gt;&#xA;&lt;p&gt;You cannot re-distribute modified copies of these data, or derivative works, without my express written permission. Where such permission is granted, you must cite the original work and its Copyrights and you must include this license with any work that includes any of my genome.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;For a more open license, you can use the nanopore-wgs reads here (derived from human cell line NA12878):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/nanopore-wgs-consortium/NA12878&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nanopore-wgs-consortium/NA12878&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We encourage the reuse of this data in your own analysis and publications which is released under the Creative Commons CC-BY license. Therefore we would be grateful if you would cite the reference below if you do.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If you don't want human data, there's mouse parasite data here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ebi.ac.uk/ena/data/view/PRJEB20824&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ebi.ac.uk/ena/data/view/PRJEB20824&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And &lt;em&gt;E. coli&lt;/em&gt; data here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ebi.ac.uk/ena/data/view/PRJEB22772&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/ena/data/view/PRJEB22772&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;... and if you just want 8h of sensor noise as a FAST5 file, I've got that here in one of my DropBox folders (as a ~70MB file):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/20j1anlz0motrs4/MINICOL233_20170118_FN__MN15625_sequencing_throughput_ONLL3127_84599_ch320_read10302_strand.fast5?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/20j1anlz0motrs4/MINICOL233_20170118_FN__MN15625_sequencing_throughput_ONLL3127_84599_ch320_read10302_strand.fast5?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Reminder: if you do ever present the results of analyses based on a data source, it's a good idea to acknowledge the source (even if not expected). If nothing else, it will let the creator know that someone cares about their data, and perhaps encourage them to release more in the future.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-10-24T09:06:06.900" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2665" PostTypeId="2" ParentId="2649" CreationDate="2017-10-19T09:04:01.870" Score="7" Body="&lt;p&gt;A very useful tool for this kind of data manipulation is &lt;a href=&quot;https://github.com/lh3/bioawk&quot; rel=&quot;noreferrer&quot;&gt;bioawk&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bioawk -c fastx '{print &quot;&amp;gt;&quot;$name&quot; &quot;$comment&quot;\t&quot;$seq}' test.fa&#xA;&amp;gt;sample 1 gene 1    atgc&#xA;&amp;gt;sample 1 gene 2    atgc&#xA;&amp;gt;sample 2 gene 1    atgc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;bioawk is based on awk, with added parsing capabilities. Here, we tell that the format is fasta or fastq with &lt;code&gt;-c fastx&lt;/code&gt;, and this makes the &lt;code&gt;$name&lt;/code&gt; (between &quot;&gt;&quot; and the first blank character), &lt;code&gt;$comment&lt;/code&gt; (after the first blank character) and &lt;code&gt;$seq&lt;/code&gt; (the sequence, in one line) variables available within awk instructions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See for instance &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/116/292&quot;&gt;this answer&lt;/a&gt; for another use case.&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2017-10-19T09:09:15.057" LastActivityDate="2017-10-19T09:09:15.057" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2666" PostTypeId="1" CreationDate="2017-10-19T09:27:30.943" Score="2" ViewCount="58" Body="&lt;p&gt;For my undergraduate research project, I'm trying to compare the structures of some proteins, specifically transcription factors crucial for dehydration tolerance in plants, in the hope of finding differences that make each protein more or less dehydration tolerant than the others. i want to compare the selected transcription factors at their structural level. structure of one transcription factor from different plant species such as rice,sorghum,maize etc. are considered. ultimately i'm hoping to find a relationship between these structures of proteins and their respective level of dehydration tolerance in each plant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any database or a site where I can find information regarding this?&lt;/p&gt;&#xA;" OwnerUserId="833" LastEditorUserId="833" LastEditDate="2017-10-19T10:27:48.210" LastActivityDate="2017-11-19T16:48:49.853" Title="How can I compare structures of proteins responsible for dehydration tolerance in plants?" Tags="|public-databases|protein-structure|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2667" PostTypeId="1" CreationDate="2017-10-19T10:23:04.197" Score="2" ViewCount="88" Body="&lt;p&gt;I am trying to use &lt;a href=&quot;http://catchenlab.life.illinois.edu/chromonomer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chromonomer&lt;/a&gt;. It should be possible (according to the software page and a paper) to add a GFF/GTF file to the analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alas, neither the built-in help nor some light source code browsing were enough to find a way of passing the file to the software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I use a GFF or a GTF file with Chromonomer?&lt;/p&gt;&#xA;" OwnerUserId="1649" LastEditorUserId="48" LastEditDate="2017-10-19T15:56:31.643" LastActivityDate="2017-10-26T06:47:59.880" Title="How to provide a GFF/GTF file to Chromonomer" Tags="|gff3|gtf|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2668" PostTypeId="1" CreationDate="2017-10-19T11:01:20.487" Score="10" ViewCount="1098" Body="&lt;p&gt;I want to collect data on how common different diseases are in different populations (or, at least, globally). Ideally, it would give me a way of querying the database with a disease name, and would return the number of cases per N population.&lt;/p&gt;&#xA;&lt;p&gt;My main objective is to have a general idea of how prevalent a disease is, I am not trying to track current outbreaks, so I am not too bothered if the information is a little out of date, as long as the database is still maintained and not abandoned.&lt;/p&gt;&#xA;&lt;p&gt;I have found &lt;a href=&quot;https://medicapps.cardiff.ac.uk/fidd/about.aspx&quot; rel=&quot;nofollow noreferrer&quot;&gt;FIDD&lt;/a&gt;, but it seems to be no longer maintained. Are there any actively maintained, similar tools that have the information I am after?&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2023-04-14T09:10:39.420" LastActivityDate="2023-04-14T09:10:39.420" Title="Is there a database of disease prevalence?" Tags="|database|public-databases|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2670" PostTypeId="1" CreationDate="2017-10-19T16:35:34.437" Score="0" ViewCount="122" Body="&lt;p&gt;Why is it that when doing paired end Radseq that R1 and R2 have different start points? For instance, while doing Radseq you do a restriction digestion of your DNA using enzyme X- shouldn't the sequencing reaction take place from either end of where the enzyme cut? If it doesn't- but happens randomly where do your multiplex sequences go?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance- if you are sequencing the sequence on an illumina machine with the &quot;&amp;lt;&gt;&quot; representing the cut site-&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where does the sequencing reaction start from in each direction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where does the multiplexing sequence go? - does it go on each?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;5-GGCTAGC&amp;lt;CCTGCA*GG&amp;gt;GCGCTT-3&#xA; 3-CCGATCG&amp;lt;GG*ACGTCC&amp;gt;CGCGAA-5&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="1652" LastEditorUserId="1652" LastEditDate="2017-10-19T18:18:57.070" LastActivityDate="2017-11-18T19:38:22.227" Title="Why do reverse radtags have different start points in radtag sequencing?" Tags="|ngs|radseq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2671" PostTypeId="1" AcceptedAnswerId="2673" CreationDate="2017-10-19T18:19:23.920" Score="4" ViewCount="141" Body="&lt;p&gt;I'm looking at some microarray data. For the first time I've calculated a &lt;a href=&quot;https://support.bioconductor.org/p/6124/&quot; rel=&quot;nofollow noreferrer&quot;&gt;moderated T statistic&lt;/a&gt; from &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/limma.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;limma&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any standard practice for where to cut off that value? For log2 fold change I usually cut off at +/- 1.5 and adjust accordingly; for adjusted p value I cut off at p&amp;lt;0.01 and adjust accordingly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These moderated T statistics seem a little higher than log2 fold change and I'm not sure what is generally accepted as significant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1226" LastActivityDate="2017-10-19T19:17:34.760" Title="Standard Cutoff for Moderated T-statistics" Tags="|bioconductor|statistics|differential-expression|microarray|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2672" PostTypeId="2" ParentId="2670" CreationDate="2017-10-19T19:14:33.197" Score="1" Body="&lt;p&gt;Assume that after digestion you're left with the following sequence:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;5' AGCTAGCTAGCTAGCTGACGTCGTAC 3'&#xA;3' GTACGACGTCAGCTAGCTAGCTAGCT 5'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then one read would start with &lt;code&gt;AGCTAG&lt;/code&gt; and the other would start with &lt;code&gt;TCGATC&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The barcodes (what are used for multiplexing) are outside of this and part of the illumina adapters. They're sequences in completely different reads. This is nicely illustrated below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Dknpn.jpg&quot; alt=&quot;Illumina fragment structure&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Index 1 and 2 don't actually need to be present. Even if they are, whether the sequencer actually sequences them is a user setting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For those curious, the order of the reads as they're sequenced are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;read #1&lt;/li&gt;&#xA;&lt;li&gt;Index 1 (optional)&lt;/li&gt;&#xA;&lt;li&gt;Index 2 (optional)&lt;/li&gt;&#xA;&lt;li&gt;read #2 (optional)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you ever get more than 2 reads from a sequencing provider, then it's likely that they included the index sequence (I have no clue why, but some sequencing centers like to do this...even though it confuses the heck out of their users).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-10-19T19:14:33.197" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2673" PostTypeId="2" ParentId="2671" CreationDate="2017-10-19T19:17:34.760" Score="3" Body="&lt;p&gt;You're misinterpreting the moderated T-statistic, it's basically the fold-change divided by its variance. The p-value comes directly from that, so if you filter by moderated fold-change you're just setting an unknown (unless you go through the trouble of figuring it out) p-value threshold. Instead, do exactly as you've been doing and set appropriate (adjusted) p-value and fold-change thresholds.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-10-19T19:17:34.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2674" PostTypeId="1" AcceptedAnswerId="2676" CreationDate="2017-10-20T07:29:16.213" Score="6" ViewCount="1175" Body="&lt;p&gt;I have a gff file like this (I edited the name):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;scaffold_x  source  exon    2987526 2987805 .   -   .   name &quot;foobar&quot;;transcriptId 68892&#xA;scaffold_x  source  CDS     2987526 2987805 .   -   1   name &quot;foobar&quot;;proteinId 68892;exonNumber 5&#xA;scaffold_x  source  exon    2988610 2988745 .   -   .   name &quot;foobar&quot;;transcriptId 68892&#xA;scaffold_x  source  CDS     2988610 2988745 .   -   2   name &quot;foobar&quot;;proteinId 68892;exonNumber 4&#xA;scaffold_x  source  exon    2989157 2989645 .   -   .   name &quot;foobar&quot;;transcriptId 68892&#xA;scaffold_x  source  CDS     2989157 2989645 .   -   2   name &quot;foobar&quot;;proteinId 68892;exonNumber 3&#xA;scaffold_x  source  exon    2990903 2991541 .   -   .   name &quot;foobar&quot;;transcriptId 68892&#xA;scaffold_x  source  CDS     2990903 2991541 .   -   2   name &quot;foobar&quot;;proteinId 68892;exonNumber 2&#xA;scaffold_x  source  stop_codon  2990903 2990905 .   -   0   name &quot;foobar&quot;&#xA;scaffold_x  source  exon    2991587 2992430 .   -   .   name &quot;foobar&quot;;transcriptId 68892&#xA;scaffold_x  source  CDS     2991587 2992430 .   -   0   name &quot;foobar&quot;;proteinId 68892;exonNumber 1&#xA;scaffold_x  source  start_codon 2992428 2992430 .   -   0   name &quot;foobar&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So the &lt;code&gt;start_codon&lt;/code&gt; is &lt;code&gt;2992428    2992430&lt;/code&gt; and the &lt;code&gt;stop_codon&lt;/code&gt; is &lt;code&gt;2990903   2990905&lt;/code&gt;; I would therefore expect all the exons within this range, however, e.g. exon 5 is found in &lt;code&gt;2987526   2987805&lt;/code&gt; which is clearly outside this range.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there something wrong with the gff file itself or is there a plausible explanation for that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the corresponding protein fasta I do find an amino acid sequence associated with this prtein ID and name:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;68892|foobar&#xA;MHTGD...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="351" LastEditorUserId="96" LastEditDate="2019-03-26T19:06:05.967" LastActivityDate="2019-03-26T19:06:05.967" Title="Can exons be located outside of the coding sequence?" Tags="|gene|gff3|molecular-genetics|exon|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2675" PostTypeId="2" ParentId="2674" CreationDate="2017-10-20T07:57:48.757" Score="8" Body="&lt;p&gt;Those are the untranslated regions (UTRs). All mRNAs have a 5' UTR and a 3' UTR. These give the ribosome something to grab onto and often contain important regulatory sites such as miRNA target sites.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2017-10-20T07:57:48.757" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2676" PostTypeId="2" ParentId="2674" CreationDate="2017-10-20T08:03:44.440" Score="11" Body="&lt;p&gt;Yes, of course. Exons are not limited to the protein coding regions. Many UTRs are in exons. In fact, you even have various cases of UTRs being multiple exons, and being spliced. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is strange in your file is not so much that you have exons beyond the stop codon, but that you also have them marked as CDS (coding sequence). That isn't possible, no. While there can indeed be exons in the UTRs, those are not coding and shouldn't be called CDS. That's probably just a minor nomenclature issue though and what you are looking at is a spliced UTR. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the exons (or the parts of the exons) that fall between the start and stop codons will be translated, while those that fall outside that regions will not be (UTRs). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, this is the exonic structure shown for &lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Transcript/Summary?db=core;g=ENSG00000141510;r=17:7661779-7687550;t=ENST00000617185&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENST00000617185&lt;/a&gt;, one of the transcripts of the human P53 gene:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/43fYt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/43fYt.png&quot; alt=&quot;TP53 exonic structure&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The boxes are exons and the lines are introns. The colored boxes are protein coding exons, while the empty ones are UTR exons. Note how they are also spliced and that this particular transcript has 3 non-coding exons on the 5' end and one on the 3' end. &lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-09-06T09:54:24.150" LastActivityDate="2018-09-06T09:54:24.150" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="2677" PostTypeId="1" AcceptedAnswerId="6879" CreationDate="2017-10-20T08:50:43.333" Score="22" ViewCount="2399" Body="&lt;p&gt;First, sorry if I am missing something basic - I am a programmer recently turned bioinformatician so I still don't know a lot of stuff. This is a cross post with a &lt;a href=&quot;https://www.biostars.org/p/278961/&quot; rel=&quot;noreferrer&quot;&gt;Biostars question&lt;/a&gt; hope that's not bad form.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;While it is obvious that scRNA-seq data contain lots of zeroes, I couldn't find any detailed explanation of why they occur except for short notices along the lines of &quot;substantial technical and biological noise&quot;. For the following text, let's assume we are looking at a single gene that is not differentially expressed across cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If zeroes were caused solely by low capture efficiency and sequencing depth, all observed zeroes should be explained by low mean expression across cells. This however does not seem to be the case as the distribution of gene counts across cells often has more zeroes than would be expected from a negative binomial model. For Example the &lt;a href=&quot;https://www.biorxiv.org/content/biorxiv/early/2017/08/09/173997.full.pdf&quot; rel=&quot;noreferrer&quot; title=&quot;ZIFA paper&quot;&gt;ZIFA paper&lt;/a&gt; explicitly uses a zero-inflated negative binomial distribution to model scRNA-seq data. Modelling scRNA-seq as zero-inflated negative binomial seems widespread throughout the literature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However assuming negative binomial distribution for the original counts (as measured in bulk RNA-seq) and assuming that every RNA fragment of the same gene from every cell has approximately the same (low) chance of being captured and sequenced, the distribution across single cells should still be negative binomial (see &lt;a href=&quot;https://math.stackexchange.com/questions/2476596/&quot;&gt;this question&lt;/a&gt;  for related math).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the only remaining possible cause is that inflated zero counts are caused by PCR. Only non-zero counts (after capture) are amplified and then sequenced, shifting the mean of the observed gene counts away from zero while the pre-PCR zero counts stay zero. Indeed some quick simulations show that such a procedure could occasionally generate zero-inflated negative binomial distributions. This would suggest that excessive zeroes should not be present when UMIs are used - I checked one scRNA-seq dataset with UMIs and it seems to be fit well by plain negative binomial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my reasoning correct? Thanks for any pointers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question&#xA;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/693/how-can-we-distinguish-between-true-zero-and-dropout-zero-counts-in-single-cell&quot;&gt;How can we distinguish between true zero and dropout-zero counts in single-cell RNA-seq?&lt;/a&gt; is related, but provides no clues to my present inquiry.&lt;/p&gt;&#xA;" OwnerUserId="1654" LastEditorUserId="1771" LastEditDate="2018-02-23T12:57:11.730" LastActivityDate="2019-03-17T20:45:22.530" Title="What is the actual cause of excessive zeroes in single cell RNA-seq data? Is it PCR?" Tags="|rna-seq|scrnaseq|single-cell|modelling|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2678" PostTypeId="2" ParentId="2677" CreationDate="2017-10-20T13:34:51.010" Score="5" Body="&lt;p&gt;I know of no references for this, but in general, I would say that your reasoning is sound. I would just add that in contrast to what I suspect you have simulated, not all transcripts are equally likely to be captured and amplified. We don't really understand what the determinants of this are, but for example, GC content is definitely related. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2017-10-20T13:34:51.010" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2679" PostTypeId="2" ParentId="2666" CreationDate="2017-10-20T15:59:41.620" Score="1" Body="&lt;p&gt;Have you explored in &lt;a href=&quot;https://www.rcsb.org/pdb/home/home.do&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB&lt;/a&gt;?. If you already have a starting sequence, you can use it as input to search with &lt;a href=&quot;https://www.ebi.ac.uk/Tools/hmmer/search/phmmer&quot; rel=&quot;nofollow noreferrer&quot;&gt;phmmer&lt;/a&gt; using PDB as database. Once you have a curated set to analyse, you can visualize and align the structures with &lt;a href=&quot;http://www.ks.uiuc.edu/Research/vmd/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VMD&lt;/a&gt;, for example. &lt;/p&gt;&#xA;" OwnerUserId="1375" LastActivityDate="2017-10-20T15:59:41.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2680" PostTypeId="1" AcceptedAnswerId="2683" CreationDate="2017-10-20T21:44:57.977" Score="3" ViewCount="558" Body="&lt;p&gt;I can’t quite understand the way inversions are defined. In particular I expect an inversion to be only a reversed version of the reference and not its reverse complement. Most sources use diagrams such as below which totally obscure the fact that it is also complemented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/l6xbU.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/l6xbU.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone provide an explanation on what causes an inversion and why it is defined the way it is?&lt;/p&gt;&#xA;" OwnerUserId="122" LastActivityDate="2017-10-21T13:42:45.293" Title="Why are inversions defined as the reverse complement and not just the reverse of the reference?" Tags="|ngs|structural-variation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2683" PostTypeId="2" ParentId="2680" CreationDate="2017-10-21T13:42:45.293" Score="5" Body="&lt;p&gt;If an inversion were just reversed then there would be 3' -&gt; 3' bonds and 5' -&gt; 5' bonds. That's why inversions are reverse complemented, you then maintain the normal 5'-&gt;3' direction. This is nicely covered &lt;a href=&quot;https://biology.stackexchange.com/questions/44550/how-to-read-this-dna-inversion-diagram&quot;&gt;over on our biology sister site&lt;/a&gt;, which also includes this image:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ijaLQ.png&quot; alt=&quot;DNA inversion&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-10-21T13:42:45.293" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2684" PostTypeId="1" AcceptedAnswerId="2685" CreationDate="2017-10-21T22:57:39.890" Score="3" ViewCount="172" Body="&lt;p&gt;Is it possible to make UCSC Genome Browser to show the sequence (nucleotide chain) of the current region?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I fiddled with the various viewing options and couldn't figure it out. &lt;/p&gt;&#xA;" OwnerUserId="122" LastEditorUserId="298" LastEditDate="2017-10-23T08:34:02.050" LastActivityDate="2021-11-15T11:27:19.953" Title="How to get UCSC Genome Browser to show the actual sequence?" Tags="|genome-browser|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2685" PostTypeId="2" ParentId="2684" CreationDate="2017-10-22T02:20:17.753" Score="5" Body="&lt;p&gt;If you zoom in at one specific region, it shows the nucleotide chain on the third line (below the chromosome number). It can't be there by default as the size of default region is more than 100,000bp. You can use the scale on the top of the page to see the size of the region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/csSI4.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/csSI4.png&quot; alt=&quot;UCSC Genome Browser Example&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1392" LastActivityDate="2017-10-22T02:20:17.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2686" PostTypeId="1" AcceptedAnswerId="2689" CreationDate="2017-10-22T12:20:58.257" Score="6" ViewCount="192" Body="&lt;p&gt;I understand how to assemble genome from error-free reads. I implemented like this: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Construct directed overlap graph with reads as vertices and edges as&#xA;  maximum overlap between two vertices. Edges represent the length of&#xA;  overlapping maximum substring of one read suffix with other read&#xA;  prefix. Find the path that visits all vertices only once in overlap&#xA;  graph. With small graph we can use greedy approach : Starting from&#xA;  first pattern always choose egde of maximum weight and store shortest&#xA;  common superstring spelled by this path in resulting string. Resulting&#xA;  string spelled by path is shortest common superstring of all reads,&#xA;  that is assembled genome.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For example &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AACTAG&#xA;  CTAGAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Gives the &lt;code&gt;AACTAGAT&lt;/code&gt; as a result&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now let's say that two error-prone reads have at most one error (mismatch):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AACTAG&#xA;  CTCGAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see second read has one substitution in third character from &lt;strong&gt;'A'&lt;/strong&gt; to &lt;strong&gt;'C'&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how assembled genome should look like for the error-prone reads? Do I take &lt;strong&gt;A&lt;/strong&gt; or &lt;strong&gt;C&lt;/strong&gt; to resulting string? Or I'm missing something?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I used short reads for example because of simplicity. I work with much larger number of reads which are 100 nucleotides long and have at most one substitution of character. So that's a big overlap graph. I know I can use De Bruijn graph, but can I use some greedy approach (like in error-free example)?  &lt;/p&gt;&#xA;" OwnerUserId="1663" LastActivityDate="2017-10-23T01:05:59.507" Title="Genome assembly from error-prone reads" Tags="|genome|assembly|reads|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2687" PostTypeId="2" ParentId="2686" CreationDate="2017-10-22T14:13:23.250" Score="2" Body="&lt;p&gt;For short reads, the typical and the most widely used solution is to correct away sequencing errors before assembly. You can correct errors with k-mer spectrum, a trie or multi-alignment. There are many papers on this topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Error correction alone won't fix all sequencing errors. Remaining errors may lead to bubbles and tips in the overlap/de Bruijn graph. You can identify them by inspecting the local topology of assembly graph.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-10-22T14:13:23.250" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2688" PostTypeId="2" ParentId="2668" CreationDate="2017-10-22T18:16:23.413" Score="5" Body="&lt;p&gt;For rare diseases, you can use &lt;a href=&quot;http://www.orpha.net/consor/cgi-bin/Disease_Search.php?lng=EN&quot; rel=&quot;noreferrer&quot;&gt;Orphanet&lt;/a&gt;. You can download an xml file with prevalence from &lt;a href=&quot;http://www.orpha.net/consor/cgi-bin/index.php?lng=EN&quot; rel=&quot;noreferrer&quot;&gt;orphadata&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an OWL version of Orphanet called &lt;a href=&quot;https://www.ebi.ac.uk/ols/ontologies/ordo&quot; rel=&quot;noreferrer&quot;&gt;ORDO&lt;/a&gt;, you can browse it on the EBI Ontology Lookup Service. See for example &lt;a href=&quot;https://www.ebi.ac.uk/ols/ontologies/ordo/terms?iri=http%3A%2F%2Fwww.orpha.net%2FORDO%2FOrphanet_247&quot; rel=&quot;noreferrer&quot;&gt;Arrhythmogenic right ventricular cardiomyopathy&lt;/a&gt; and point prevalence relationships in the lower right box. There are a variety of programmatic ways to access this, including the OLS web API, SPARQL queries, or downloading the OWL and using an ontology API.&lt;/p&gt;&#xA;" OwnerUserId="1322" LastActivityDate="2017-10-22T18:16:23.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2689" PostTypeId="2" ParentId="2686" CreationDate="2017-10-22T18:47:50.417" Score="1" Body="&lt;p&gt;It's common for assemblers to have a coverage threshold to make sure that there are enough reads across the same region to properly assemble a contig.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can't work out an ideal solution from a situation where the consensus sequence is ambiguous. If your overlap were representative (e.g. a heterozygous variant in a diploid genome, based on a few tens of reads of 100bp, rather than 6bp), then the resulting assembly would have a bubble in the assembly graph at the variant point.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;        A&#xA;       / \&#xA;A-A-C-T   G-A-T&#xA;       \ /&#xA;        C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A conservative assembler might produce two assembled contigs corresponding to these two paths, AACTAGAT and AACTCGAT. A greedy assembler would just pick one path and use that. Subsequent assembly steps might attempt to search for bubbles and annotate the assembled contigs with variant locations.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-10-23T01:05:59.507" LastActivityDate="2017-10-23T01:05:59.507" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2690" PostTypeId="2" ParentId="2668" CreationDate="2017-10-23T00:11:05.690" Score="5" Body="&lt;p&gt;Comment as an answer, as requested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.tdrdata.com/ipd/ipd_init&quot; rel=&quot;noreferrer&quot;&gt;Incidence and Prevalance Database&lt;/a&gt; (IPD - subscription required).&lt;/p&gt;&#xA;" OwnerUserId="150" LastActivityDate="2017-10-23T00:11:05.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2691" PostTypeId="1" CreationDate="2017-10-23T09:50:54.787" Score="4" ViewCount="137" Body="&lt;p&gt;I have a gff file like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;scaffold_1  source  mRNA    2987526 2992430 .   -   .   ID=protein_id_68892;Name=&quot;foobar&quot;;locus_tag=my_organism_68892;translation=MHTGDALEGSTGNVSILV...&#xA;scaffold_1  source  exon    2987526 2987805 .   -   .   name &quot;foobar&quot;; transcriptId 68892;locus_tag=my_organism_68892&#xA;scaffold_1  source  CDS     2987526 2987805 .   -   1   name &quot;foobar&quot;; proteinId 68892; exonNumber 5;locus_tag=my_organism_68892;Parent=protein_id_68892&#xA;scaffold_1  source  exon    2988610 2988745 .   -   .   name &quot;foobar&quot;; transcriptId 68892;locus_tag=my_organism_68892&#xA;scaffold_1  source  CDS     2988610 2988745 .   -   2   name &quot;foobar&quot;; proteinId 68892; exonNumber 4;locus_tag=my_organism_68892;Parent=protein_id_68892&#xA;scaffold_1  source  exon    2989157 2989645 .   -   .   name &quot;foobar&quot;; transcriptId 68892;locus_tag=my_organism_68892&#xA;scaffold_1  source  CDS     2989157 2989645 .   -   2   name &quot;foobar&quot;; proteinId 68892; exonNumber 3;locus_tag=my_organism_68892;Parent=protein_id_68892&#xA;scaffold_1  source  exon    2990903 2991541 .   -   .   name &quot;foobar&quot;; transcriptId 68892;locus_tag=my_organism_68892&#xA;scaffold_1  source  CDS     2990903 2991541 .   -   2   name &quot;foobar&quot;; proteinId 68892; exonNumber 2;locus_tag=my_organism_68892;Parent=protein_id_68892&#xA;scaffold_1  source  exon    2991587 2992430 .   -   .   name &quot;foobar&quot;; transcriptId 68892;locus_tag=my_organism_68892&#xA;scaffold_1  source  CDS     2991587 2992430 .   -   0   name &quot;foobar&quot;; proteinId 68892; exonNumber 1;locus_tag=my_organism_68892;Parent=protein_id_68892&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;mRNA&lt;/code&gt;entry was not present at first and I added it in. I also have the protein fast file available and would now like to add the amino acid sequences to the gff. I originally added them in column 9 to the &lt;code&gt;mRNA&lt;/code&gt;entry which is apparently incorrect. &lt;a href=&quot;https://github.com/The-Sequence-Ontology/Specifications/blob/master/gff3.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; it is written that the sequences should go at the end of the file in the &lt;code&gt;##Fasta&lt;/code&gt;section. However, I struggle to figure out which identifier I should use to link it to my &lt;code&gt;mRNA&lt;/code&gt; entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I have to use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##Fasta&#xA;&amp;gt;protein_id_68892&#xA;MHTGDALEGSTGNVSILV...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or does it work with the locus_tag&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##Fasta&#xA;&amp;gt;my_organism_68892&#xA;MHTGDALEGSTGNVSILV...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or is it something completely else? Where and how do can I add the translation information?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the link above (and also in the comments it is suggested) the identifier in the first column is used, in my case that would be &lt;code&gt;scaffold_1&lt;/code&gt;. However, I do have a lot of entries with &lt;code&gt;scaffold_1&lt;/code&gt;in my file so a link between &lt;code&gt;scaffold_1&lt;/code&gt;and an amino acid sequence would not be unique as I would have&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##Fasta&#xA;&amp;gt;scaffold_1&#xA;MHTGDALEGSTGNVSILV...&#xA;&amp;gt;scaffold_1&#xA;GDAHTEGSTGNVSILV...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;. In the end, I want to create a genbank file (which does not exist yet) where I can link a CDS to an amino acid sequence.&lt;/p&gt;&#xA;" OwnerUserId="351" LastEditorUserId="351" LastEditDate="2017-10-23T11:25:54.387" LastActivityDate="2018-02-25T19:58:25.507" Title="Which identifier do I have to use when I want to add a ##Fasta section to a gff?" Tags="|gff3|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2692" PostTypeId="2" ParentId="2691" CreationDate="2017-10-23T12:48:04.660" Score="0" Body="&lt;p&gt;To answer your specific question, you should use the identifier from the first column of the GFF in the ##Fasta section. As you note, this won't work if you are trying to add multiple protein sequence for each seqid. Try using the scaffold DNA sequences in this section and add amino acid translations as an attribute of the annotations.&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2017-10-23T12:48:04.660" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2693" PostTypeId="1" AcceptedAnswerId="2695" CreationDate="2017-10-23T15:57:31.047" Score="2" ViewCount="1118" Body="&lt;p&gt;I was looking BioPython to calculate the number of possible alignments between two sequences. Let's say there are two input sequences with the length of &lt;em&gt;m&lt;/em&gt; and &lt;em&gt;n&lt;/em&gt;. The program will count the possible number of alignments based on the length of two sequences. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to do that with BioPython? If not, how can I do that?&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="77" LastEditDate="2017-11-18T23:03:03.780" LastActivityDate="2017-11-18T23:03:03.780" Title="Counting number of possible alignments between two DNA sequences using python" Tags="|python|sequence-alignment|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2694" PostTypeId="1" CreationDate="2017-10-23T16:58:34.053" Score="9" ViewCount="233" Body="&lt;p&gt;I have a structure with two subunits. I am trying to show movement of the C-terminal subunit upon ligand binding by superposition with another structure from the same strain in the apo form. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to superpose the N-terminal subunits (A), note the root mean square of the atomic deviations (RMSD), then apply the resulting matrix to the superposition of both subunits of the molecule with both subunits of the other (B), then I want to superpose on the C-terminal subunits (C), and then measure the RMSD between (C) and the C-terminal subunits from (B). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know how can I do this in Chimera? I know how to superpose molecules and calculate the RMSD in Chimera, but I do not know how to apply the resulting matrix from (A) to (B). I tried to save the aligned structures from (A) as a PDB file to check and see if there was a matrix in the file, but an error message came up saying that I needed to save the file as &lt;code&gt;$name&lt;/code&gt; or &lt;code&gt;$number&lt;/code&gt;. I tried to save the file with &lt;code&gt;$&lt;/code&gt; in front of the name, but it didn't work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I do this correctly?&lt;/p&gt;&#xA;" OwnerUserId="1672" LastEditorUserId="3848" LastEditDate="2019-07-31T00:35:39.990" LastActivityDate="2022-01-31T11:03:45.360" Title="Chimera Alignments" Tags="|sequence-alignment|software-recommendation|protein-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2695" PostTypeId="2" ParentId="2693" CreationDate="2017-10-23T18:21:03.197" Score="10" Body="&lt;p&gt;A &lt;a href=&quot;https://en.wikipedia.org/wiki/Closed-form_expression&quot; rel=&quot;noreferrer&quot;&gt;closed-form&lt;/a&gt; solution is offered in &lt;em&gt;&lt;a href=&quot;https://www.researchgate.net/profile/Alberto_Cabada/publication/5346653_An_Exact_Formula_for_the_Number_of_Alignments_Between_Two_DNA_Sequences/links/02bfe50efedf9c053e000000.pdf&quot; rel=&quot;noreferrer&quot;&gt;An exact formula for the number of alignments&#xA;between two DNA sequences&lt;/a&gt;&lt;/em&gt; by Torres, Cabada, and Nieto:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$f(m,n)=\sum_{k=0}^{min(m,n)}2^{k}\binom{m}{k}\binom{n}{k}$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this solution seems reasonable, you could calculate this without BioPython, but with a simple &lt;code&gt;xrange&lt;/code&gt; loop, the &lt;a href=&quot;https://docs.python.org/2.0/ref/power.html&quot; rel=&quot;noreferrer&quot;&gt;power&lt;/a&gt; operator, and &lt;code&gt;scipy.special.binom&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;&#xA;import sys&#xA;import scipy.special&#xA;&#xA;def alignments(m, n):&#xA;    if m &amp;lt; 0 or n &amp;lt; 0:&#xA;        raise ValueError('m and n should be non-negative')&#xA;    s = 0&#xA;    b = min(m, n) + 1&#xA;    for k in xrange(0, b):&#xA;        s += (2 ** k) * scipy.special.binom(m, k) * scipy.special.binom(n, k)&#xA;    return s&#xA;&#xA;def main():&#xA;    m = 4&#xA;    n = 2&#xA;    a = alignments(m, n)&#xA;    assert (m == 4 and n == 2 and a == 41), &quot;something went wrong!&quot;&#xA;    sys.stdout.write(&quot;alignments(%d, %d) = %d\n&quot; % (m, n, a))&#xA;&#xA;if __name__ == &quot;__main__&quot;:&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't know what your &lt;code&gt;m&lt;/code&gt; and &lt;code&gt;n&lt;/code&gt; are, but keep in mind that this solution grows quickly. Also, I did some digging and scipy &lt;a href=&quot;https://stackoverflow.com/a/22966919&quot;&gt;approximates&lt;/a&gt; the calculation of the binomial coefficient, which is probably going to be an issue for large &lt;code&gt;m&lt;/code&gt; and &lt;code&gt;n&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're working with large values, you may want to look into using the Python &lt;code&gt;long&lt;/code&gt; type and consider the accuracy of the answer. The Stack Overflow link offers alternatives to scipy that generate an exact calculation of the binomial coefficient, at the cost of a longer runtime.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2017-10-24T18:37:58.030" LastActivityDate="2017-10-24T18:37:58.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2696" PostTypeId="1" AcceptedAnswerId="3260" CreationDate="2017-10-24T08:14:14.477" Score="5" ViewCount="133" Body="&lt;p&gt;This question is slightly related to this one:&#xA;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/97/improve-a-reference-genome-with-sequencing-data&quot;&gt;Improve a reference genome with sequencing data&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Only in my case, I have a starting genome reference A, of a short phage genome, and reads from a related genome B, for which I haven't been able to find a reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I took the Illumina 2x75bp paired end reads from genome B and ran Velvet on it (I ran Ray, but it didn't complete). It gives me a fragmented collection of contigs. The contigs align 99-98% to reference genome A. I don't have long reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: with the short reads and the contigs that I have, can I transform the reference genome A into a reference genome B?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: no, I don't have long reads, and there is no budget for them. The budget for long reads is zero and will remain zero. Even if we could sequence long reads for free, we would still not do it because of the lab work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tools that I have tried so far:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ococo:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$HOME/ococo/ococo -i contigs.bwamem.bam -c 1 -f $HOME/Genomes/T2_phage/starting.fasta -F consensus.fasta --vcf-cons - -V variants.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It produced a consensus.fasta, and checking how much it changes from the reference, it changes. There are one or more changes in about 33% of the 100bp windows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;diff -y &amp;lt;(seqtk seq -l 100 $HOME/Genomes/T2_phage/starting.fasta) &amp;lt;(seqtk seq -l 100 consensus.fasta) | grep '\ \|\ ' | wc -l&#xA;1272&#xA;diff -y &amp;lt;(seqtk seq -l 100 $HOME/Genomes/T2_phage/starting.fasta) &amp;lt;(seqtk seq -l 100 consensus.fasta) | grep -v '\ \|\ ' | wc -l&#xA;419&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="180" LastEditorUserId="48" LastEditDate="2018-01-10T08:02:56.010" LastActivityDate="2018-01-10T08:02:56.010" Title="Creating a new reference genome B by changing genome A with mismatches from BAM file without long reads" Tags="|sam|reference-genome|snp|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2698" PostTypeId="1" CreationDate="2017-10-24T12:41:25.180" Score="3" ViewCount="1266" Body="&lt;p&gt;I'm using the python API for the biom-format package. I read in two files in biom format and merged them, like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [1]: import biom&#xA;&#xA;In [2]: a = biom.load_table('./table-a.biom')&#xA;&#xA;In [3]: b = biom.load_table('./table-b.biom')&#xA;&#xA;In [4]: a&#xA;Out[4]: 51116 x 46 &amp;lt;class 'biom.table.Table'&amp;gt; with 185077 nonzero entries (7% dense)&#xA;&#xA;In [5]: b&#xA;Out[5]: 37479 x 26 &amp;lt;class 'biom.table.Table'&amp;gt; with 100124 nonzero entries (10% dense)&#xA;&#xA;In [6]: c = a.merge(b)&#xA;&#xA;In [7]: c&#xA;Out[7]: 67022 x 47 &amp;lt;class 'biom.table.Table'&amp;gt; with 254555 nonzero entries (8% dense)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How do I now export this biom object (&lt;code&gt;c&lt;/code&gt;) as a biom file? There are a few methods available, eg, &lt;code&gt;.to_tsv()&lt;/code&gt;, but I don't see a way to save this object in biom format.&lt;/p&gt;&#xA;" OwnerUserId="1475" LastEditorUserId="77" LastEditDate="2017-10-24T18:50:04.890" LastActivityDate="2018-08-02T20:50:44.240" Title="How do I export my biom object to a file in biom format?" Tags="|python|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2699" PostTypeId="2" ParentId="2677" CreationDate="2017-10-24T14:23:37.647" Score="8" Body="&lt;p&gt;The &lt;a href=&quot;https://www.biostars.org/p/278961/&quot; rel=&quot;noreferrer&quot;&gt;Biostars thread&lt;/a&gt; turned out helpful. The most interesting possible cause, not mentioned in the &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2678/1654&quot;&gt;Ian Subery's answer&lt;/a&gt;, is that due to bursty nature of transcription, the true distribution of transcript counts across cells can be bimodal with a peak at zero even assuming a simple model of transcription such as the random telegraph model. See for example &lt;a href=&quot;http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=PMC5310734&quot; rel=&quot;noreferrer&quot;&gt;Dattani &amp;amp; Barahona 2017&lt;/a&gt; for a more detailed discussion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the random telegraph model predicts that the even with peak at zero, there should usually be non-negligible probability of having small, but nonzero counts, which is not always the case in zero-inflated scRNA-seq data: see for example the Isl1 gene in &lt;a href=&quot;https://portals.broadinstitute.org/single_cell/study/retinal-bipolar-neuron-drop-seq&quot; rel=&quot;noreferrer&quot;&gt;Retina dataset&lt;/a&gt; (click Explore -&gt; search for &quot;Isl1&quot;)&lt;/p&gt;&#xA;" OwnerUserId="1654" LastActivityDate="2017-10-24T14:23:37.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2700" PostTypeId="1" AcceptedAnswerId="2701" CreationDate="2017-10-24T18:48:46.170" Score="5" ViewCount="3247" Body="&lt;p&gt;I have a lot of PDB IDs and I need to get uniprot fasta sequences of these PDB IDs special chains by API services. For example, imagine that I need to get fasta sequence of '1kf6' 'A' chain. The uniprot entry (Accession) for this '1kf6' chain is 'P00363'. If I find a way to get this entry, I can simply retrieve its fasta sequence using bioservices package:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from bioservices import *&#xA;u = UniProt()&#xA;sequence = u.retrieve(&quot;P00363&quot;,&quot;fasta&quot;)&#xA;print(sequence)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The main problem is that I don't know how can I find this uniprot entry(&quot;P00363&quot; here) for each chains in PDB IDs(here I imagine that I don't know this &quot;P00363&quot; is the uniprot entry for '1kf6' 'A' chain and I'm trying to find it using API services). I tried to use ID mapping. In ID mapping cross-references, the chain name for each entry is specified and if I can add a cross-reference column to u.search('1kf6', columns=&quot;id,entry name,length, genes,pdb&quot;), I can simply find the entry related to my favourite chain. The following code returns some entries each related to one of '1kf6' chains. The problem is that I don't know exactly which entry is related to 'A' chain that here is my favorite chain. In the following code, id searches for entry and other parameters search for sth else related to them:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from bioservices import *&#xA;u = UniProt()&#xA;res = u.search(&quot;1kf6&quot;,columns=&quot;id,entry name,length, genes&quot;)&#xA;print(res)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Each one of the following words can be used and added to columns part:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;'id', 'entry name', 'genes', 'genes(PREFERRED)', 'genes(ALTERNATIVE)', 'genes(OLN)', 'genes(ORF)', 'organism', 'organism-id', 'protein names', 'proteome', 'lineage(ALL)', 'lineage-id', 'virus hosts', 'fragement', 'sequence', 'length', 'mass', 'encodedon', 'comment(ALTERNATIVE PRODUCTS)', 'comment(ERRONEOUS GENE MODEL PREDICTION)', 'comment(ERRONEOUS INITIATION)', 'comment(ERRONEOUS TERMINATION)', 'comment(ERRONEOUS TRANSLATION)', 'comment(FRAMESHIFT)', 'comment(MASS SPECTROMETRY)', 'comment(POLYMORPHISM)', 'comment(RNA EDITING)', 'comment(SEQUENCE CAUTION)', 'feature(ALTERNATIVE SEQUENCE)', 'feature(NATURAL VARIANT)', 'feature(NON ADJACENT RESIDUES)', 'feature(NON STANDARD RESIDUE)', 'feature(NON TERMINAL RESIDUE)', 'feature(SEQUENCE CONFLICT)', 'feature(SEQUENCE UNCERTAINTY)', 'version(sequence)', 'domains', 'domain', 'comment(DOMAIN)', 'comment(SIMILARITY)', 'feature(COILED COIL)', 'feature(COMPOSITIONAL BIAS)', 'feature(DOMAIN EXTENT)', 'feature(MOTIF)', 'feature(REGION)', 'feature(REPEAT)', 'feature(ZINC FINGER)', 'ec', 'comment(ABSORPTION)', 'comment(CATALYTIC ACTIVITY)', 'comment(COFACTOR)', 'comment(ENZYME REGULATION)', 'comment(FUNCTION)', 'comment(KINETICS)', 'comment(PATHWAY)', 'comment(REDOX POTENTIAL)', 'comment(TEMPERATURE DEPENDENCE)', 'comment(PH DEPENDENCE)', 'feature(ACTIVE SITE)', 'feature(BINDING SITE)', 'feature(DNA BINDING)', 'feature(METAL BINDING)', 'feature(NP BIND)', 'feature(SITE)', 'go', 'go(biological process)', 'go(molecular function)', 'go(cellular component)', 'go-id', 'interpro', 'interactor', 'comment(SUBUNIT)', 'citation', 'citationmapping', 'created', 'last-modified', 'sequence-modified', 'version(entry)', '3d', 'feature(BETA STRAND)', 'feature(HELIX)', 'feature(TURN)', 'comment(SUBCELLULAR LOCATION)', 'feature(INTRAMEMBRANE)', 'feature(TOPOLOGICAL DOMAIN)', 'feature(TRANSMEMBRANE)', 'annotation score', 'score', 'features', 'comment(CAUTION)', 'comment(TISSUE SPECIFICITY)', 'comment(GENERAL)', 'keywords', 'context', 'existence', 'tools', 'reviewed', 'feature', 'families', 'subcellular locations', 'taxonomy', 'version', 'clusters', 'comments', 'database', 'keyword-id', 'pathway', 'score'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I used some of the above words in columns part but they couldn't retrieve chain name. Anyone know which one of the above parameters can give me the chain name for this PDB ID? or anyone has any other idea for finding uniprot entry(UniprotKB AC) related to each PDB ID chain by API services?&lt;/p&gt;&#xA;" OwnerUserId="818" LastEditorUserId="818" LastEditDate="2017-10-24T19:33:25.077" LastActivityDate="2022-10-04T14:01:42.167" Title="How to map PDB chains to Uniprot IDs using API services" Tags="|fasta|python|pdb|api|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2701" PostTypeId="2" ParentId="2700" CreationDate="2017-10-24T21:10:12.113" Score="3" Body="&lt;p&gt;I found the answer and I thought it may be useful for others. we can use pypdb package for that purpose (it works in unix systems):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pypdb&#xA;&#xA;all_info = pypdb.get_all_info('1kf6')&#xA;print(all_info)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the output is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{'polymer': [{'@entityNr': '1', '@length': '602', '@type': 'protein', '@weight': '66057.6', 'chain': [{'@id': 'A'}, {'@id': 'M'}], 'Taxonomy': {'@name': 'Escherichia coli', '@id': '562'}, 'macroMolecule': {'@name': 'Fumarate reductase flavoprotein subunit', 'accession': {'@id': 'P00363'}}, 'polymerDescription': {'@description': 'FUMARATE REDUCTASE FLAVOPROTEIN'}, 'enzClass': {'@ec': '1.3.5.4'}}, {'@entityNr': '2', '@length': '243', '@type': 'protein', '@weight': '27021.9', 'chain': [{'@id': 'B'}, {'@id': 'N'}], 'Taxonomy': {'@name': 'Escherichia coli', '@id': '562'}, 'macroMolecule': {'@name': 'Fumarate reductase iron-sulfur subunit', 'accession': {'@id': 'P0AC47'}}, 'polymerDescription': {'@description': 'FUMARATE REDUCTASE IRON-SULFUR PROTEIN'}, 'enzClass': {'@ec': '1.3.5.1'}}, {'@entityNr': '3', '@length': '130', '@type': 'protein', '@weight': '14898.8', 'chain': [{'@id': 'C'}, {'@id': 'O'}], 'Taxonomy': {'@name': 'Escherichia coli', '@id': '562'}, 'macroMolecule': {'@name': 'Fumarate reductase subunit C', 'accession': {'@id': 'P0A8Q0'}}, 'polymerDescription': {'@description': 'FUMARATE REDUCTASE 15 KDA HYDROPHOBIC PROTEIN'}}, {'@entityNr': '4', '@length': '119', '@type': 'protein', '@weight': '13118.9', 'chain': [{'@id': 'D'}, {'@id': 'P'}], 'Taxonomy': {'@name': 'Escherichia coli', '@id': '562'}, 'macroMolecule': {'@name': 'Fumarate reductase subunit D', 'accession': {'@id': 'P0A8Q3'}}, 'polymerDescription': {'@description': 'FUMARATE REDUCTASE 13 KDA HYDROPHOBIC PROTEIN'}}], 'id': '1KF6'}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This output gives all the information related to all chains and we can simply find uniprotkb AC of each chain in this output. I also found another answer for this question. This answer can be found &lt;a href=&quot;https://www.biostars.org/p/166134/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;from xml.etree.ElementTree import fromstring&#xA;&#xA;pdb_id = '1kf6.A'&#xA;pdb_mapping_url = 'http://www.rcsb.org/pdb/rest/das/pdb_uniprot_mapping/alignment'&#xA;uniprot_url = 'http://www.uniprot.org/uniprot/{}.xml'&#xA;&#xA;def get_uniprot_accession_id(response_xml):&#xA;    root = fromstring(response_xml)&#xA;    return next(&#xA;        el for el in root.getchildren()[0].getchildren()&#xA;        if el.attrib['dbSource'] == 'UniProt'&#xA;    ).attrib['dbAccessionId']&#xA;&#xA;def get_uniprot_protein_name(uniport_id):&#xA;    uinprot_response = requests.get(&#xA;        uniprot_url.format(uniport_id)&#xA;    ).text&#xA;    return fromstring(uinprot_response).find('.//{http://uniprot.org/uniprot}recommendedName/{http://uniprot.org/uniprot}fullName'&#xA;    ).text&#xA;&#xA;def map_pdb_to_uniprot(pdb_id):&#xA;    pdb_mapping_response = requests.get(&#xA;        pdb_mapping_url, params={'query': pdb_id}&#xA;    ).text&#xA;    uniprot_id = get_uniprot_accession_id(pdb_mapping_response)&#xA;    uniprot_name = get_uniprot_protein_name(uniprot_id)&#xA;    return {&#xA;        'pdb_id': pdb_id,&#xA;        'uniprot_id': uniprot_id,&#xA;        'uniprot_name': uniprot_name&#xA;    }&#xA;&#xA;print (map_pdb_to_uniprot(pdb_id))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output of this code is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{'pdb_id': '1kf6.A', 'uniprot_id': 'P00363', 'uniprot_name': 'Fumarate reductase flavoprotein subunit'}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="818" LastActivityDate="2017-10-24T21:10:12.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2702" PostTypeId="1" AcceptedAnswerId="2782" CreationDate="2017-10-25T03:00:18.583" Score="2" ViewCount="526" Body="&lt;p&gt;Is there a way to remove split alignments of single-end long reads from BWA-SW output? BWA-MEM has an option to include or flag split alignments in the result, but it seems BWA-SW method include them in the result unless the split occur in paired-end reads.&lt;/p&gt;&#xA;" OwnerUserId="1392" LastEditorUserId="1392" LastEditDate="2017-10-25T21:33:09.500" LastActivityDate="2019-04-19T13:23:00.707" Title="Mark BWA-SW split alignments in output for long reads" Tags="|sequence-alignment|bwa|sam|long-reads|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2703" PostTypeId="1" AcceptedAnswerId="2714" CreationDate="2017-10-25T03:34:22.433" Score="1" ViewCount="61" Body="&lt;p&gt;I am trying to find out that which DNA sequences(in the below) have a similar domain to a specific toxin which is &lt;em&gt;Clostridium botulinum&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find the fragment of the DNA sequence that belong to that toxin gene below:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;gi|27867582 (fragment of the known Clostridium botuninum toxin gene)&#xA;  GTGAATCAGCACCTGGACTTTCAGATGAAAAATTAAATTTAACTATCCAAAATGATGCTT&#xA;  ATATACCAAAATATGATTCTAATGGAACAAGTGATATAGAACAACATGATGTTAATGAAC&#xA;  TTAATGTATTTTTCTATTTAGATGCACAGAAAGTGCCCGAAGGTGAAAATAATGTCAATC&#xA;  TCACCTCTTCAATTGATACAGCATTATTAGAACAACCTAAAATATATACATTTTTTTCAT&#xA;  CAGAATTTATTAATAATGTCAATAAACCTGTGCAAGCAGC&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I ran a blastx job over the above sequence which translates nucleotide into protein. Here's the &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Get&amp;amp;RID=Z422R3SJ014&quot; rel=&quot;nofollow noreferrer&quot;&gt;result&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The result tells me that. The particular DNA sequence listed above, belongs to &lt;em&gt;botulinum neurotoxin type E [Clostridium botulinum]&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides, I am given two DNA sequences:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Sequence A&#xA;TCTATCAAGTAGATTATTAAATACTACTGCACAAAATGATTCTTACGTTCCAAAATATGA&#xA;TTCTAATGGTACAAGTGAAATAAAGGAATATACTGTTGATAAACTAAATGTATTTTTCTA&#xA;TTTATATGCACAAAAAGCTCCTGAAGGTGAAAGTGCTATAAGTTTAACTTCTTCAGTTAA&#xA;TACAGCATTATTAGATGCATCTAAAGTTTATACGTTTTTTTCTTCAGATTTTATTAATAC&#xA;&#xA;&amp;gt; Sequence B&#xA;TCCTGGCTCAGGACGAACGCTGGCGGCGTGTGCTTAACACATGCAAGTCGAGCGATGAAG&#xA;CTTCCTTCGGGAAGTGGATTAGCGGCGGACGGGTGAGTAACACGTGGGTAACCTGCCTCA&#xA;AAGTGGGGGATAGCCTTCCGAAAGGAAGATTAATACCGCATAACATAAGAGAATCGCATG&#xA;ATTTTCTTATCAAAGATTTATTGCTTTGAGATGGACCCGCGGCGCATTAGCTAGTTGGTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Again, I translated them into proteins. The result are listed respectively(A and B):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Get&amp;amp;RID=Z423E71C015&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Get&amp;amp;RID=Z423E71C015&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Get&amp;amp;RID=Z42AKVWC015&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Get&amp;amp;RID=Z42AKVWC015&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first job tells that, Sequence A belongs to &lt;em&gt;botulinum neurotoxin type F [Clostridium baratii]&lt;/em&gt; which means that similar domain with the above &lt;em&gt;botulinum neurotoxin&lt;/em&gt;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second job tells me that SequenceB belongs to &lt;em&gt;unknown protein [Clostridium botulinum NCTC 2916]&lt;/em&gt;. So that means it is not in the same domain with the above botulinum neurotoxin right?&lt;/p&gt;&#xA;" OwnerUserId="1671" LastEditorUserId="1671" LastEditDate="2017-10-26T17:24:44.000" LastActivityDate="2017-10-26T18:17:05.283" Title="Finding out proteins that have a similar domain to a specific toxin" Tags="|sequence-alignment|homework|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2704" PostTypeId="2" ParentId="2435" CreationDate="2017-10-25T09:14:41.480" Score="2" Body="&lt;p&gt;I'm not aware of a functionality in the KEGG service that would allow you to do that directly. I believe your solution (using e.g. ChEBI) is the correct one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may have other useful service such as unichem, chemspider (you would need a login though for that one) could help as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Disclaimer&lt;/strong&gt;: I'm the bioservices main author&lt;/p&gt;&#xA;" OwnerUserId="1678" LastEditorUserId="48" LastEditDate="2017-10-25T19:40:31.017" LastActivityDate="2017-10-25T19:40:31.017" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2705" PostTypeId="1" AcceptedAnswerId="2708" CreationDate="2017-10-25T13:22:09.093" Score="3" ViewCount="374" Body="&lt;p&gt;Is it possible to load a list of rsID identifiers (e.g. rs66823261) to IGV to then be able to jump to the location of each of them when copy+pasting the rsID on the locus text box?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For human and other genomes, there is currently the feature of typing in a gene name, then jumping to the gene location.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if there is a way of achieving the same for rsIDs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: thanks for the answers so far. I downloaded the Infinium GSA bed file from Illumina's website (&lt;a href=&quot;https://support.illumina.com/downloads/infinium-global-screening-array-v1-0-support-files.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.illumina.com/downloads/infinium-global-screening-array-v1-0-support-files.html&lt;/a&gt;), loaded on an IGV session, which asked to create an index. I did a liftover of the file to hg38 and sorted with bedtools sort. Then I went to the text box, introduced one of the rs ids:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;rs6569648&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Got a positive result. Thank you all!&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="180" LastEditDate="2017-10-27T08:48:35.390" LastActivityDate="2017-10-27T08:48:35.390" Title="IGV jump to rsID location" Tags="|snp|igv|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2706" PostTypeId="1" CreationDate="2017-10-25T15:46:32.350" Score="3" ViewCount="240" Body="&lt;p&gt;I am learning about generating heatmaps using R. For this example I am using randomly generated data to test the process, but I encountered what appears to be an error or perhaps there is something am missing. Specifically, the values of Npy2r in groups A &amp;amp; B are the same, yet the heat map colors them differently. I am trying to understand why. Am including the R code I am using to generate the heat map&lt;a href=&quot;https://i.stack.imgur.com/F9ToQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/F9ToQ.png&quot; alt=&quot;heatmap&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;heatmap.2(x, Rowv=F, Colv = F, scale=&quot;col&quot;, col=greenred(3), key=TRUE, symkey=FALSE,                             &#xA;      density.info=&quot;none&quot;,cexRow=1,cexCol=1,margins=c(6,11),  trace=&quot;none&quot;,srtCol=45)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1430" LastEditorUserId="57" LastEditDate="2017-10-25T15:58:04.250" LastActivityDate="2017-10-25T15:58:50.637" Title="error in heatmap using R" Tags="|r|differential-expression|microarray|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2707" PostTypeId="2" ParentId="2706" CreationDate="2017-10-25T15:51:57.207" Score="8" Body="&lt;p&gt;You have &lt;code&gt;scale=&quot;col&quot;&lt;/code&gt; in the code. What you are plotting is the z-score calculated based on the value distribution per column. Try changing it to &lt;code&gt;&quot;row&quot;&lt;/code&gt; or &lt;code&gt;&quot;none&quot;&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1375" LastEditorUserId="57" LastEditDate="2017-10-25T15:58:50.637" LastActivityDate="2017-10-25T15:58:50.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2708" PostTypeId="2" ParentId="2705" CreationDate="2017-10-25T18:37:27.600" Score="2" Body="&lt;p&gt;Yes, if you load the list you are interested in as a non-indexed &quot;BED&quot; file then you can jump to an rsID or any other name in the BED file. You'll need a targeted list, dbSNP for example is much too large to load in this way.&lt;/p&gt;&#xA;" OwnerUserId="1681" LastEditorUserId="77" LastEditDate="2017-10-25T19:42:00.760" LastActivityDate="2017-10-25T19:42:00.760" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2709" PostTypeId="2" ParentId="2649" CreationDate="2017-10-25T20:51:41.967" Score="8" Body="&lt;p&gt;You can use these commands:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl -pe 's/&amp;gt;(.*)/&amp;gt;\1\t/g; s/\n//g; s/&amp;gt;/\n&amp;gt;/g' sequences.fa | grep -v '^$'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Explanation:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Append a tab to every header line&lt;/li&gt;&#xA;&lt;li&gt;Join all lines&lt;/li&gt;&#xA;&lt;li&gt;Split the single obtained line by the '&gt;' character&lt;/li&gt;&#xA;&lt;li&gt;Remove the empty line (the first line is empty due to the fact that '&gt;' is the first character of the FASTA file)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="425" LastEditorUserId="298" LastEditDate="2019-05-23T08:17:25.103" LastActivityDate="2019-05-23T08:17:25.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2710" PostTypeId="2" ParentId="2674" CreationDate="2017-10-26T06:34:55.317" Score="2" Body="&lt;p&gt;Although the explanation that UTR (untranscribed) regions can consist of multiple exons covers most situations, I think it is good to mention &lt;a href=&quot;https://en.wikipedia.org/wiki/Stop_codon#Translational_readthrough&quot; rel=&quot;nofollow noreferrer&quot;&gt;stop codon readthrough&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would result in a CDS (coding sequence) part of your mRNA (messenger RNA) &lt;em&gt;after&lt;/em&gt; a stop codon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, following the &lt;a href=&quot;https://github.com/The-Sequence-Ontology/Specifications/blob/master/gff3.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;gff3 format specification&lt;/a&gt;, this should probably be encoded differently, using SO (&lt;a href=&quot;https://github.com/The-Sequence-Ontology/SO-Ontologies/blob/master/so-simple.obo&quot; rel=&quot;nofollow noreferrer&quot;&gt;sequence ontology&lt;/a&gt;) terms like SO:0000697 (gene with stop codon read through) and  SO:0000883 (stop codon readthrough).&lt;/p&gt;&#xA;" OwnerUserId="196" LastActivityDate="2017-10-26T06:34:55.317" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2711" PostTypeId="2" ParentId="2667" CreationDate="2017-10-26T06:47:59.880" Score="1" Body="&lt;p&gt;I think the part of the code that should allow you to do this is in &lt;code&gt;chromomer.cc&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lines 2857-2983 form the function &lt;code&gt;parse_command_line&lt;/code&gt;. Nowhere is there any reference to anything resembling parsing a GFF file, so I think the documentation is not in line with the code at this point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference, I looked at version 1.07. It would be helpful if the code were on github.&lt;/p&gt;&#xA;" OwnerUserId="196" LastActivityDate="2017-10-26T06:47:59.880" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2712" PostTypeId="2" ParentId="2705" CreationDate="2017-10-26T09:49:13.210" Score="0" Body="&lt;p&gt;you can use mysql ucsc to get the position of the SNP and then ask IGV to take some picures: see &lt;a href=&quot;https://software.broadinstitute.org/software/igv/batch&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://software.broadinstitute.org/software/igv/batch&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ echo -e 'rs66823261\nrs25' | awk '{printf(&quot;select concat(chrom,\&quot;:\&quot;,chromStart-10,\&quot;-\&quot;,chromEnd+10) from  snp150 where name=\&quot;%s\&quot;;\n&quot;,$1);}' | mysql --user=genome --host=genome-mysql.soe.ucsc.edu -A -D hg38 -N | awk '{printf(&quot;goto %s\nsnapshot\n&quot;,$1);}'&#xA;&#xA;goto chr8:170681-170702&#xA;snapshot&#xA;goto chr7:11584131-11584152&#xA;snapshot&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastActivityDate="2017-10-26T09:49:13.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2714" PostTypeId="2" ParentId="2703" CreationDate="2017-10-26T18:17:05.283" Score="1" Body="&lt;p&gt;A &lt;a href=&quot;https://en.wikipedia.org/wiki/Protein_domain&quot; rel=&quot;nofollow noreferrer&quot;&gt;protein domain&lt;/a&gt; is a conserved part of a given protein sequence and (tertiary) structure that can evolve, function, and exist independently of the rest of the protein chain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is different from global protein similarity, so similarity to neurotoxin type F doesn't necessarily mean that sequence A has similar domain to your known toxin gene. Your blast results do include conserved domain information though.&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2017-10-26T18:17:05.283" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2715" PostTypeId="2" ParentId="2705" CreationDate="2017-10-26T20:05:43.900" Score="0" Body="&lt;p&gt;If you want all rsIDs (complete dbSnp database) then Pierre's answer is the best option currently.    Any bed file with all the rsIDs would be too big to load.  For a targeted list of, say, 100,000 or fewer snps you can use a bed file.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW igv.js already does the equivalent of Pierre's suggestion, this will be ported to IGV desktop in the near future.  &lt;/p&gt;&#xA;" OwnerUserId="1681" LastActivityDate="2017-10-26T20:05:43.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2716" PostTypeId="2" ParentId="933" CreationDate="2017-10-26T21:48:12.580" Score="4" Body="&lt;p&gt;You can browse gnomAD variants with &lt;a href=&quot;http://reg.clinicalgenome.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;ClinGen Allele Registry&lt;/a&gt; (there is API spec available).&lt;/p&gt;&#xA;" OwnerUserId="1690" LastEditorUserId="48" LastEditDate="2017-10-27T16:09:32.620" LastActivityDate="2017-10-27T16:09:32.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2717" PostTypeId="2" ParentId="2698" CreationDate="2017-10-26T22:18:03.483" Score="1" Body="&lt;p&gt;There is a function in &lt;a href=&quot;http://picrust.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PiCRUST&lt;/a&gt; you can use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from picrust.util import write_biom_table&#xA;&#xA;write_biom_table(c,'table-c.biom')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note, however, that depending if you want json or hdf5 format and if your metadata are complex, you may need to use additional arguments. These are the arguments of the function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def write_biom_table(biom_table, biom_table_fp, compress=True,&#xA;                 write_hdf5=HAVE_H5PY, format_fs=None):&#xA;&quot;&quot;&quot;Writes a BIOM table to the specified filepath&#xA;&#xA;Parameters&#xA;----------&#xA;biom_table : biom.Table&#xA;    The table object to write out&#xA;biom_table_fp : str&#xA;    The path to the output file&#xA;compress : bool, optional&#xA;    Defaults to ``True``. If True, built-in compression on the output HDF5&#xA;    file will be enabled. This option is only relevant if ``write_hdf5`` is&#xA;    ``True``.&#xA;write_hdf5 : bool, optional&#xA;    Defaults to ``True`` if H5PY is installed and to ``False`` if H5PY is&#xA;    not installed. If ``True`` the output biom table will be written as an&#xA;    HDF5 binary file, otherwise it will be a JSON string.&#xA;format_fs : dict, optional&#xA;    Formatting functions to be passed to `Table.to_hdf5`&#xA;&#xA;Notes&#xA;-----&#xA;This code was adapted from QIIME 1.9&#xA;&quot;&quot;&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have some problems to perform a simple operation and write the output of biom data generated with picrust, it might be helpful as an &lt;a href=&quot;https://github.com/biocore/qiime/issues/2205&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt; of how to use the arguments.&lt;/p&gt;&#xA;" OwnerUserId="1691" LastActivityDate="2017-10-26T22:18:03.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2718" PostTypeId="2" ParentId="2398" CreationDate="2017-10-26T22:19:13.753" Score="3" Body="&lt;p&gt;I have provided a &lt;a href=&quot;https://www.biostars.org/p/269966/#280121&quot; rel=&quot;nofollow noreferrer&quot;&gt;solution&lt;/a&gt; in Biostars:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you edit the file &lt;code&gt;findKnownMotifs.pl&lt;/code&gt; and move the call to &lt;code&gt;printMotif&lt;/code&gt; to come before testing &lt;code&gt;$svgFlag&lt;/code&gt; you will find that &lt;code&gt;.motif&lt;/code&gt; files are printed again. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the following I have commented out the call from where it need not be, and introduced it as the first line, right before the &lt;code&gt;if&lt;/code&gt; block:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   printMotif($_, $directory . &quot;/$knownDir/$matrixFile&quot;); # this should be done regardless of $svgFlag                                                                                                        &#xA;    if ($svgFlag) {&#xA;        my $str = HomerSVGLogo::motif2svg($_,$bitFlag,$rnaFlag2);&#xA;            open SVG, &quot;&amp;gt;$directory/$knownDir/$logoFile.svg&quot;;&#xA;            print SVG $str;&#xA;            close SVG;&#xA;            print MAIN &quot;$str\n&quot;;&#xA;    } else {&#xA;		#printMotif($_, $directory . &quot;/$knownDir/$matrixFile&quot;); # this should be done regardless of $svgFlag                                                                                               &#xA;            `profile2seq.pl &quot;$directory/$knownDir/$matrixFile&quot; 100 &amp;gt; &quot;$tmpFile&quot;`;&#xA;            my $width = $widthFactor * $_-&amp;gt;{'len'};&#xA;            `seqlogo -a $bits -f &quot;$tmpFile&quot; -F PNG -S -c -o &quot;$directory/$knownDir/$logoFile&quot; -h $height -w $width`;&#xA;		`seqlogo -a $bits -f &quot;$tmpFile&quot; -F PDF -S -c -o &quot;$directory/$knownDir/$logoFile&quot; -h $height -w $width 2&amp;gt; /dev/null`;&#xA;		print MAIN &quot;&amp;lt;IMG src=\&quot;$knownDir/$logoFile.png\&quot;/&amp;gt;&quot;;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I expect this change was mistakenly introduced while refactoring the code to support svg as alternate output format for logos.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have reported this to the author of this code.&lt;/p&gt;&#xA;" OwnerUserId="1692" LastEditorUserId="776" LastEditDate="2017-10-27T16:09:43.360" LastActivityDate="2017-10-27T16:09:43.360" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2719" PostTypeId="2" ParentId="2700" CreationDate="2017-10-27T08:28:37.083" Score="2" Body="&lt;p&gt;An alternative solution involves the use of the &lt;a href=&quot;http://www.ebi.ac.uk/pdbe/docs/sifts/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SIFTS&lt;/a&gt; resource from the EBI. I wrote &lt;a href=&quot;https://github.com/mgalardini/pdb2uniprot/blob/master/parse_sifts.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;a python parser&lt;/a&gt; for their xml format, which will give you a mapping between a pdb and a uniprot entry at the single residue level.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ curl --silent &quot;ftp://ftp.ebi.ac.uk/pub/databases/msd/sifts/xml/1kf6.xml.gz&quot; | gunzip | python parse_sifts.py&#xA;1kf6    A       M       0       P00363  M       1&#xA;1kf6    A       Q       1       P00363  Q       2&#xA;1kf6    A       T       2       P00363  T       3&#xA;1kf6    A       F       3       P00363  F       4&#xA;1kf6    A       Q       4       P00363  Q       5&#xA;1kf6    A       A       5       P00363  A       6&#xA;1kf6    A       D       6       P00363  D       7&#xA;1kf6    A       L       7       P00363  L       8&#xA;1kf6    A       A       8       P00363  A       9&#xA;1kf6    A       I       9       P00363  I       10&#xA;[...]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The returned fields are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;pdb ID&lt;/li&gt;&#xA;&lt;li&gt;chain&lt;/li&gt;&#xA;&lt;li&gt;pdb residue&lt;/li&gt;&#xA;&lt;li&gt;pdb position&lt;/li&gt;&#xA;&lt;li&gt;uniprot ID&lt;/li&gt;&#xA;&lt;li&gt;uniprot residue&lt;/li&gt;&#xA;&lt;li&gt;uniprot position&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In the above snippet, just put your PDB ID of choice to retrieve the data you need.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2017-10-27T08:28:37.083" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2720" PostTypeId="1" CreationDate="2017-10-27T12:53:53.560" Score="0" ViewCount="622" Body="&lt;p&gt;I have a reference sequence of amino acids that I compare other sequences to using &lt;code&gt;Bio.pairwise2.align.globalmx&lt;/code&gt; of Biopython package (one to many compare - one by one [is there a better way?])&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's works fine but only returns strings of the alignment with very little metadata.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want to extract the indices of the amino acids that are different than the reference sequence and plot it in some kind of histogram according to protein features:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;pos 1-36: 3 mutations.&lt;/li&gt;&#xA;&lt;li&gt;pos 37-202: 4 mutations.&lt;/li&gt;&#xA;&lt;li&gt;pos 203: 1 mutation&lt;/li&gt;&#xA;&lt;li&gt;pos 204-660: 0 mutations&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is there any builtin package to do so?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I get the data of diff between 2 sequences in python? I mean which AA changed to which AA and at each position. &lt;code&gt;Bio.pairwise2.align.globalmx&lt;/code&gt; returns some data structures of strings (see below) that can be printed but does not contain the locations in a convenient way. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;LLGHKVNPFKALYNRIDMK-QVEALVEASKEE &#xA;|||||||||||||||||||||||||||||||| &#xA;LLGHKVNPFKALYNRIDM-RQVEALVEASKEE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1695" LastEditorUserId="48" LastEditDate="2018-02-03T11:47:13.507" LastActivityDate="2018-02-03T11:47:13.507" Title="Mutations in sequences relative to reference (histogram)" Tags="|sequence-alignment|python|sequence-analysis|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="2721" PostTypeId="1" CreationDate="2017-10-27T14:49:29.720" Score="5" ViewCount="182" Body="&lt;p&gt;I have a RNA-seq bam file and there are few reads that are puzzling me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the bam header, this bam file is sorted by coordinate, created using tophat and markduplicate step is not done. But some reads are marked for being duplicate in the samflag. What is worse is when I run picard markduplicate, these reads' pcr duplicate flag is toggled, marking them not a duplicate. Also I manually found the duplicate of this read (identical reads with same start positions and mates start position) so initial marking looks true.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my questions are: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea why would this happen?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does Tophat mark reads for being duplicate? ( I don't think so)&#xA;And does &lt;code&gt;picard markduplicate&lt;/code&gt; toggle the duplicate flag if the reads are already marked as being duplicates?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is how the read looks before and after mark duplicate step.&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C0RTF 1187 17 7579880 255 61M10754N40M = 7579927 10902 CTC... &#xA;0UNP1 163 17 7579880 255 61M10754N40M = 7579927 10902 CTC... &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After Markduplicate:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C0RTF 163 17 7579880 255 61M10754N40M = 7579927 10902 CTC... &#xA;0UNP1 163 17 7579880 255 61M10754N40M = 7579927 10902 CTC... &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1696" LastEditorUserId="298" LastEditDate="2017-10-30T13:27:51.183" LastActivityDate="2017-10-30T20:15:42.250" Title="Does picard markduplicate toggle PCR duplicate samflag" Tags="|rna-seq|samtools|markduplicate|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2722" PostTypeId="1" AcceptedAnswerId="2724" CreationDate="2017-10-27T14:50:55.833" Score="0" ViewCount="210" Body="&lt;p&gt;I try to find organism IDs used by UniProt (e.g. 9606 for &quot;Homo sapiens&quot;) and information about the 'taxonomic kingdom' this organism belongs to. The only source on UniProt I could find is &lt;a href=&quot;http://www.uniprot.org/docs/speclist&quot; rel=&quot;nofollow noreferrer&quot;&gt;this file&lt;/a&gt; which contains all required information. So in the Python world it would be a dictionary of the form:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{org_id: {'name': 'org_name', 'kingdom': 'org_kingdom'}}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or a dataframe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the file from above is not particular nice to parse due to its inconsistent column format (not impossible to parse though), so I am wondering whether there is an alternative source to retrieve this information or whether there is maybe even a webservice available that takes care of this parsing.&lt;/p&gt;&#xA;" OwnerUserId="351" LastEditorUserId="351" LastEditDate="2017-10-27T17:46:30.640" LastActivityDate="2017-10-30T09:27:52.827" Title="How to access organism IDS, taxon nodes from UniProt?" Tags="|database|data-retrieval|uniprot|webservice|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2723" PostTypeId="2" ParentId="2722" CreationDate="2017-10-27T15:01:58.770" Score="4" Body="&lt;p&gt;These are &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/taxonomy&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI taxonomy&lt;/a&gt; ID's rather than anything specific to UniProt. One way to parse them would be using BioPython via &lt;a href=&quot;http://biopython.org/DIST/docs/api/Bio.Entrez-module.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;Bio.Entrez&lt;/code&gt;&lt;/a&gt;, for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez&#xA;&#xA;def get_taxon_name_from_NCBI_id(id_):&#xA;    Entrez.email = 'name@provider.com'&#xA;    handle = Entrez.efetch(db='taxonomy', id= id_, retmode='xml')&#xA;    records = Entrez.read(handle)[0]&#xA;    return records['ScientificName'], records['TaxId']&#xA;&#xA;print(get_taxon_name_from_NCBI_id(9606))&#xA;# ('Homo sapiens', '9606')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Of course you could query Entrez directly:&#xA;&lt;a href=&quot;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=taxonomy&amp;amp;id=9606&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=taxonomy&amp;amp;id=9606&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or even more basically do a manual search of NCBI taxonomy:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/taxonomy/?term=9606&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/taxonomy/?term=9606&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="104" LastEditDate="2017-10-27T15:20:02.983" LastActivityDate="2017-10-27T15:20:02.983" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2724" PostTypeId="2" ParentId="2722" CreationDate="2017-10-27T16:04:58.860" Score="1" Body="&lt;p&gt;That file is actually trivial to parse. You only care about lines that have &lt;code&gt;N=$species&lt;/code&gt;, so you can simply do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -En 's/.* ([0-9]+): N=(.*)/\1\t\2/p' speclist &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That will return a tab separated list of taxID and species:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sed -En 's/.* ([0-9]+): N=(.*)/\1\t\2/p' speclist | head&#xA;648330  Aedes albopictus densovirus (isolate Boublik/1994)&#xA;10804   Adeno-associated virus 2&#xA;648242  Adeno-associated virus 2 (isolate Srivastava/1982)&#xA;118452  Abacion magnum&#xA;72259   Abaeis nicippe&#xA;102642  Abax parallelepipedus&#xA;392897  Abalistes stellaris&#xA;75332   Abbottina rivularis&#xA;515833  Abdopus aculeatus&#xA;56673   Antarctic bacterium DS2-3R&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So to get human, for example, you could do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sed -En 's/.* ([0-9]+): N=(.*)/\1\t\2/p' speclist | grep sapiens&#xA;63221   Homo sapiens neanderthalensis&#xA;9606    Homo sapiens&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And, for exact matches:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sed -En 's/.* ([0-9]+): N=(.*)/\1\t\2/p' speclist | &#xA;    awk -F'\t' '$2==&quot;Homo sapiens&quot;'&#xA;9606    Homo sapiens&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you also want to keep the taxon node (kingdom), you can use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -En 's/.* ([A-Z])  *([0-9]+): N=(.*)/\1\t\2\t\3/p' speclist &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which prints:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sed -En 's/.* ([A-Z])  *([0-9]+): N=(.*)/\1\t\2\t\3/p' speclist  | head&#xA;V   648330  Aedes albopictus densovirus (isolate Boublik/1994)&#xA;V   10804   Adeno-associated virus 2&#xA;V   648242  Adeno-associated virus 2 (isolate Srivastava/1982)&#xA;E   118452  Abacion magnum&#xA;E   72259   Abaeis nicippe&#xA;E   102642  Abax parallelepipedus&#xA;E   392897  Abalistes stellaris&#xA;E   75332   Abbottina rivularis&#xA;E   515833  Abdopus aculeatus&#xA;B   56673   Antarctic bacterium DS2-3R&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;sed&lt;/code&gt; command might appear a little daunting if you don't know &lt;code&gt;sed&lt;/code&gt;, but it's actually quite simple. The &lt;code&gt;-E&lt;/code&gt; flag enables extended regular expressions which give us a simpler syntax. The &lt;code&gt;-n&lt;/code&gt; suppresses normal output, so that &lt;code&gt;sed&lt;/code&gt; won't actually print anything unless explicitly told to. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;s/from/to/&lt;/code&gt; is the substitution operator and will replace &lt;code&gt;from&lt;/code&gt; with &lt;code&gt;to&lt;/code&gt;. Here, we are replacing zero or more characters (&lt;code&gt;.*&lt;/code&gt;) until a space followed by one or more numbers (&lt;code&gt;[0-9]+&lt;/code&gt;), a &lt;code&gt;:&lt;/code&gt;, another space, then the string &lt;code&gt;N=&lt;/code&gt; and then everything else (&lt;code&gt;.*&lt;/code&gt;). The parentheses let us &lt;em&gt;capture&lt;/em&gt; the matched strings, so that we can then refer to the 1st captured string as &lt;code&gt;\1&lt;/code&gt;, the second as &lt;code&gt;\2&lt;/code&gt; etc. We are therefore replacing the entire line (because the regex matches the entire line) with the 1st pattern captured (the taxID, &lt;code&gt;\1&lt;/code&gt;), a tab (&lt;code&gt;\t&lt;/code&gt;) and the species name (&lt;code&gt;\2&lt;/code&gt;). Finally, the &lt;code&gt;p&lt;/code&gt; at the end means &quot;print the resulting line if the substitution was successful&quot;, so only lines matching the desired pattern are actually printed. &lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-10-30T09:27:52.827" LastActivityDate="2017-10-30T09:27:52.827" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2725" PostTypeId="2" ParentId="2654" CreationDate="2017-10-28T10:00:39.167" Score="3" Body="&lt;p&gt;Based on your description I think you should have a look at a technique called 'biclustering'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The example on &lt;a href=&quot;https://en.wikibooks.org/wiki/Data_Mining_Algorithms_In_R/Clustering/Biclust&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt; defines the goal of this technique as 'Finding subgroups of rows and columns which are as similar as possible to each other and as different as possible to the rest.'&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since your examples are python-based, you could check out &lt;a href=&quot;http://scikit-learn.org/stable/modules/biclustering.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scikit-learn's implementations of biclustering&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="196" LastActivityDate="2017-10-28T10:00:39.167" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2726" PostTypeId="1" AcceptedAnswerId="2732" CreationDate="2017-10-28T18:44:10.307" Score="9" ViewCount="2498" Body="&lt;p&gt;People talk about deep sequencing. Is there any way to calculate how deep the sequencing is ? What should be the optimum depth to get reliable data ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am doing whole genome sequencing of a virus genome which is 10 kb long. I got 80000 reads from Illumina sequencing. Is there any way to tell about my sequencing depth ? &lt;/p&gt;&#xA;" OwnerUserId="1684" LastEditorUserId="3848" LastEditDate="2020-12-03T08:28:31.673" LastActivityDate="2020-12-03T08:28:31.673" Title="What is deep sequencing?" Tags="|ngs|covid-19|sars-cov-2|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2727" PostTypeId="2" ParentId="2726" CreationDate="2017-10-28T21:04:40.773" Score="8" Body="&lt;p&gt;Sequencing depth is typically calculated as the &lt;em&gt;number of total bases sequenced&lt;/em&gt; divided by the &lt;em&gt;number of bases in the target genome&lt;/em&gt;. An Illumina sequencing run with 2x125 bp reads and 500 million read pairs sequenced would be a sequencing depth of about 40X (assuming my calculations are correct for a 3 billion base genome).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sequencing depth depends on how long each read is, as well as whether it is a paired-end sequencing run. 80,000 read pairs with each having 125 bp would be a coverage of 2000X for a 10 kbp viral genome (assuming the sample were entirely composed of that virus).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The optimum depth changes depending on the application. A genotyping-by-sequencing analysis, combined with imputation, could work well with 96 samples at a 1X coverage (or possibly less). On the other end of the scale, a cell-free DNA analysis to identify structural variation in metastasing cancer may need a few thousand times coverage of the host DNA. &lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-10-29T02:26:10.997" LastActivityDate="2017-10-29T02:26:10.997" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2728" PostTypeId="2" ParentId="2726" CreationDate="2017-10-29T16:57:38.093" Score="9" Body="&lt;p&gt;There are several questions in your post I'll try to answer each one:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there any way to calculate how deep the sequencing is ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;See gringer's &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2727/48&quot;&gt;answer&lt;/a&gt;. TLDR: The depth of the sequencing is how many times each position has been sequenced. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What should be the optimum depth to get reliable data ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The optimum depth depends on what you want to do with that data. Usually,  30X or 50X is done, but in targeted sequencing you can sequence a lot more reaching 1000X (which would be really deep).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there any way to tell about my sequencing depth ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you know the length of the region sequenced and how many bases you have in the reads, you can calculate the general sequencing depth.&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="298" LastEditDate="2017-10-30T10:17:59.063" LastActivityDate="2017-10-30T10:17:59.063" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2729" PostTypeId="1" AcceptedAnswerId="2733" CreationDate="2017-10-30T10:39:10.967" Score="4" ViewCount="385" Body="&lt;p&gt;I generate bigwig files using a shell script based on &lt;code&gt;bedtools genomecov&lt;/code&gt; (to generate a bedgraph from a bam file) &lt;code&gt;bedmap&lt;/code&gt; (to compute means across 10 bp bins) and &lt;code&gt;bedGraphToBigWig&lt;/code&gt; to convert the binned bedgraph into bigwig.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes, the bam file has no data on some chromosomes, this results in a bedgraph file with no entries for the empty chromosomes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to add the missing entries. How could I do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to see what I could do with pyBigWig, opening the bedgraph file in &quot;a&quot; mode, but don't really know how to proceed:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# I have the chromosome sizes for my genome of interest already in a list:&#xA;chrom_sizes&#xA;#[('I', 15072434),&#xA;# ('II', 15279421),&#xA;# ('III', 13783801),&#xA;# ('IV', 17493829),&#xA;# ('MtDNA', 13794),&#xA;# ('V', 20924180),&#xA;# ('X', 17718942)]&#xA;bw_test = pyBigWig.open(&quot;path/to/bigwig&quot;, &quot;a&quot;)&#xA;bw_test.chroms()&#xA;# {'IV': 17493829, 'V': 20924180, 'X': 17718942}&#xA;bw_test.addHeader(chrom_sizes)&#xA;# RuntimeError: Received an error in bwCreateHdr&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="292" LastActivityDate="2017-10-30T19:37:06.637" Title="Adding entries to bigwig file" Tags="|python|bigwig|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2730" PostTypeId="1" AcceptedAnswerId="2734" CreationDate="2017-10-30T15:41:13.877" Score="5" ViewCount="134" Body="&lt;p&gt;I have a expression data from a small cohort of samples taken at baseline and after 2 independent treatments. I can do differential expression contrasting T1 and T2 or I can contrast T1 vs baseline and T2 vs baseline and look at the differences. What is the difference between these two analyses? Are they both valid? If so what inferences can be drawn from each?&lt;/p&gt;&#xA;" OwnerUserId="772" LastActivityDate="2017-10-31T05:19:17.837" Title="Differential Expression With 2 Treatments" Tags="|differential-expression|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2731" PostTypeId="1" CreationDate="2017-10-30T17:59:49.003" Score="2" ViewCount="489" Body="&lt;p&gt;I am trying to test a preexisting python machine learning script with a subset of my genetic data.  One of the feature columns I am using happens to only have NA values in it.  I lose this column when I use the following code:&#xA;&lt;code&gt;Imputer(missing_values='NaN', strategy='mean', axis=0).fit_transform(data)&lt;/code&gt; &#xA;This causes problems later on because my model expects the correct number of features, and I do not want to retrain my model at this time.  Is there any way to work with Imputer or fit_transform to prevent this column from getting dropped?&lt;/p&gt;&#xA;" OwnerUserId="1574" LastActivityDate="2017-10-30T17:59:49.003" Title="How to prevent sklearn Imputer(missing_values=&quot;NaN&quot;, strategy=&quot;mean&quot;, axis=0).fit_transform(data) from removing columns with only NA in them" Tags="|python|machine-learning|" AnswerCount="0" CommentCount="3" ClosedDate="2017-11-01T12:21:58.723" ContentLicense="CC BY-SA 3.0" />
  <row Id="2732" PostTypeId="2" ParentId="2726" CreationDate="2017-10-30T19:19:01.320" Score="11" Body="&lt;p&gt;I found &lt;a href=&quot;http://www.metagenomics.wiki/pdf/definition/coverage-read-depth&quot; rel=&quot;noreferrer&quot;&gt;a post useful for this topic&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It explains the difference of coverage and depth.  It also has a useful explanation on how to calculate coverage and depth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hZqvF.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hZqvF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a copy of what the link says just encase the post is removed:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Depth of coverage&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;em&gt;How strong is a genome &quot;covered&quot; by sequenced fragments (short reads)?&lt;/em&gt; &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Per-base coverage is the average number of times a base of a genome is&#xA;  sequenced. The coverage depth of a genome is calculated as the number&#xA;  of bases of all short reads that match a genome divided by the length&#xA;  of this genome. It is often expressed as 1X, 2X, 3X,... (1, 2, or, 3&#xA;  times coverage).&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Breadth of coverage&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;em&gt;How much of a genome is &quot;covered&quot; by short reads? Are there regions that are not covered, even not by a single read?&lt;/em&gt; &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Breadth of coverage is the percentage of bases of a reference genome&#xA;  that are covered with a certain depth. For example: 90% of a genome is&#xA;  covered at 1X depth; and still 70% is covered at 5X depth.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h1&gt;SAMtools: get breadth of coverage&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Get breadth of coverage of a reference genome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How many bases of my reference genome are covered by my short reads sample at a level of 5X or higher?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; # set shell variables to your file-names&#xA;&#xA; REF_GENOME_FILE=my_ref_genome.fna&#xA; WGS_SAMPLE=my_short_read_sample.fastq &#xA; MIN_COVERAGE_DEPTH=5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Map short reads against reference genome (bowtie2)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# create bowtie2 index database (database name: refgenome)&#xA;&#xA;bowtie2-build ${REF_GENOME_FILE} refgenome&#xA;ls&#xA;refgenome.1.bt2&#xA;refgenome.2.bt2&#xA;refgenome.3.bt2&#xA;refgenome.4.bt2&#xA;refgenome.rev.1.bt2&#xA;refgenome.rev.2.bt2&#xA;&#xA;&#xA;# map reads and sort bam file&#xA;bowtie2 -x refgenome --no-unal -U ${WGS_SAMPLE} -S - -p 12 | \&#xA;samtools view -bS - | \&#xA;samtools sort -m 5G - mapping_result_sorted.bam&#xA;&#xA;-U ${WGS_SAMPLE} consider all reads as unpaired reads&#xA;-S -             bowtie output in SAM format, write to standard out&#xA;-x refgenome     use reference genome bowtie2-database&#xA;--no-unal        suppress SAM records for reads that failed to align&#xA;-p 12            use 12 processors&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Get breadth of coverage (SAMtools)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# create samtools index&#xA;samtools index mapping_result_sorted.bam&#xA;&#xA;# get total number of bases covered at MIN_COVERAGE_DEPTH or higher&#xA;samtools mpileup mapping_result_sorted.bam | awk -v &#xA;X=&quot;${MIN_COVERAGE_DEPTH}&quot; '$4&amp;gt;=X' | wc -l&#xA;32876&#xA;&#xA;# get length of reference genome&#xA;bowtie2-inspect -s refgenome | awk '{ FS = &quot;\t&quot; } ; BEGIN{L=0}; {L=L+$3}; &#xA;END{print L}'&#xA;45678&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Result: breadth of reference genome coverage&#xA;total number of covered bases: 32876  (with &gt;= 5X coverage depth)&#xA;Depth of coverage (average per-base coverage): 0.719 X (32876 ÷ 45678) (total number of covered bases divided by reference genome length)&#xA;percent: 71.9%  (0.719 × 100) (coverage breadth in percent)&lt;/p&gt;&#xA;" OwnerUserId="1689" LastEditorUserId="29" LastEditDate="2017-11-01T17:28:45.277" LastActivityDate="2017-11-01T17:28:45.277" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2733" PostTypeId="2" ParentId="2729" CreationDate="2017-10-30T19:37:06.637" Score="3" Body="&lt;p&gt;Presuming you still have your BAM file sitting around somewhere:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;import pysam&#xA;import pyBigWig&#xA;&#xA;bam = pysam.AlignmentFile(&quot;alignments.bam&quot;)&#xA;ifile = pyBigWig.open(&quot;your current file.bw&quot;)&#xA;ofile = pyBigWig.open(&quot;your new file.bw&quot;, &quot;w&quot;)&#xA;&#xA;# Create/Add the header&#xA;hdr = []&#xA;for chrom in bam.header['SQ']: &#xA;    hdr.append((chrom['SN'], chrom['LN']))&#xA;bam.close()&#xA;ofile.addHeader(hdr)&#xA;&#xA;# Write the bigWig&#xA;for _ in hdr:&#xA;    try:&#xA;        ints = ifile.intervals(_[0])&#xA;        starts = [x[0] for x in ints]&#xA;        ends = [x[1] for x in ints]&#xA;        vals = [x[2] for x in ints]&#xA;        ofile.addEntries([_[0]] * len(starts), starts, ends=ends, values=vals)&#xA;    except:&#xA;        # intervals() will throw an error if the chromosome isn't present&#xA;        pass&#xA;ofile.close()&#xA;ifile.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The reason you can't append to a bigWig file is that it has a various indices and zoom levels typically built in. You'd have to add to them without overwriting neighboring blocks of data. In theory this might sometimes be theoretically possible, but not typically and I'm not aware of tools that support this.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-10-30T19:37:06.637" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2734" PostTypeId="2" ParentId="2730" CreationDate="2017-10-30T19:50:47.230" Score="4" Body="&lt;p&gt;If you're interested in looking at the differences between two treatments then you'll end up wanting to do both a direct contrast as well as the individual comparisons to baseline.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The direct contrast will give you the genes actually differentially expressed between the two conditions. In practice, you may want to filter this a bit so you only have genes differentially expressed vs. baseline in at least one condition (e.g., to get rid of genes only slightly higher due to T1 and slightly lower in T2, but not different enough in either case to be DE). Use a pretty lax p-value threshold (e.g., 0.1 or 0.2) for this.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-10-30T19:50:47.230" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="2735" PostTypeId="2" ParentId="2721" CreationDate="2017-10-30T20:15:42.250" Score="6" Body="&lt;p&gt;Yes, if MarkDuplicates encounters a pair that's marked as a duplicate that it considers (for whatever reason) to not be a duplicate then it will unset the duplicate mark. You can test this yourself by making a small BAM file either with or without duplicate entries but some marked regardless. In the case of actual duplicates, the duplicate flag will be maintained. In the case where there are no actual duplicates then the flag will be removed. At least that's the case in version 2.14.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-10-30T20:15:42.250" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2736" PostTypeId="2" ParentId="2720" CreationDate="2017-10-30T20:16:36.967" Score="1" Body="&lt;p&gt;In answer to the comments, you can do multiple pair wise alignments and then concatenate them to visualise, if MSA is not the way to go. I have a script you can find &lt;a href=&quot;https://github.com/jrjhealey/bioinfo-tools/blob/master/Shannon.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; which will draw a graph of the Shannon entropy (uncertainty) in a column. If you create a gapped FASTA alignment for each of your pairs, and concatenate it in to a multifasta, the script should still work with it. Though it may be noisy - best just to try it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;JalView might be an option for a one-stop GUI tool, but I’m not sure if it’ll require an MSA instead of multiple pair wise.&lt;/p&gt;&#xA;" OwnerUserId="929" LastEditorUserId="48" LastEditDate="2017-11-30T08:45:07.573" LastActivityDate="2017-11-30T08:45:07.573" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2737" PostTypeId="2" ParentId="2730" CreationDate="2017-10-31T05:19:17.837" Score="3" Body="&lt;p&gt;Depending upon the type of treatment used the set of DEGs will change.If the treatments have similar kind of effect you will get a small list (less variable genes will have higher p-val) using a cutoff of p&amp;lt;0.05. So, it's better to start with control vs treated comparisons then T1 vs T2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Compare the lists form CTRL vs T1 and CTRL vs T2 you will get the genes that are expressed in both conditions as well as unique to individual treatments.&lt;/p&gt;&#xA;" OwnerUserId="206" LastActivityDate="2017-10-31T05:19:17.837" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2738" PostTypeId="1" AcceptedAnswerId="2739" CreationDate="2017-10-31T08:15:38.057" Score="7" ViewCount="1249" Body="&lt;p&gt;Say your aim is to calculate the coverage of an RNA-seq experiment generated with long-read sequencing (so, uneven read length). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Up to now, I relied on the &lt;a href=&quot;https://www.illumina.com/documents/products/technotes/technote_coverage_calculation.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lander/Waterman equation&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$C = L*N / G$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; = final coverage &lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$G$&lt;/span&gt; = haploid genome (or transcriptome) length&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; = read length&lt;br&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; = number of reads&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two major conceptual issues with this formula: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is uneven in this case. Should we use the median/mean read length instead? If so, side question: if the RNA-seq experiment was executed following the &lt;a href=&quot;http://www.pacb.com/wp-content/uploads/Procedure-Checklist-Isoform-Sequencing-Iso-Seq-Analysis-using-the-Clontech-SMARTer-PCR-cDNA-Synthesis-Kit-and-SageELF-Size-Selection-System.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;size-fractionated protocol&lt;/a&gt; (PacBio), is it advisable to merge all the fractions prior to coverage calculation or should it be calculated separately for each fraction?&lt;/li&gt;&#xA;&lt;li&gt;Should &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; consist of the mapped reads to the reference or simply the number of reads as found in the subreads whole dataset?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="294" LastEditorUserId="1157" LastEditDate="2019-04-15T16:58:42.630" LastActivityDate="2019-04-15T16:58:42.630" Title="Coverage calculation: long reads (RNA-seq)" Tags="|rna-seq|transcriptome|long-reads|pacbio|coverage|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2739" PostTypeId="2" ParentId="2738" CreationDate="2017-10-31T09:49:04.327" Score="6" Body="&lt;p&gt;The read length is irrelevant when calculating the mean coverage statistic. It's simply the &lt;em&gt;total number of bases sequenced&lt;/em&gt; divided by the &lt;em&gt;target Xome length&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the example provided in the question, $L*N$ is equivalently expressed as $\sum^{N} l_x$, or the total length of sequenced reads. For equal-length reads, this is easier to calculate as &lt;em&gt;number of reads&lt;/em&gt; $*$ &lt;em&gt;length of reads&lt;/em&gt;, but the full sum is necessary if read lengths are not equal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For what it's worth, this value, $L*N$, is also equivalent to the average read length multiplied by the number of reads, i.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\frac{\sum^{N} l_x}{N} * N$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;... which makes it a bit more obvious why the total length of sequenced bases is &lt;em&gt;very slightly&lt;/em&gt; easier to calculate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In answer to the second part of the question, $N$ should ideally correspond to only the sequences that are from the reference. For an assembly, only the reads that were used to create the assembly should count when considering coverage. When there's no likelihood of contamination (and especially for RNASeq with variable coverage per transcript), a statistic using the total number of reads should be good enough for most purposes that I can think of. High-throughput sequencing is such a fuzzy process that a few tens of millions of bases here or there shouldn't make a difference.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-10-31T10:06:54.310" LastActivityDate="2017-10-31T10:06:54.310" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2740" PostTypeId="1" AcceptedAnswerId="2741" CreationDate="2017-10-31T14:58:23.897" Score="6" ViewCount="273" Body="&lt;p&gt;There is high throughput sequencing data &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM455387&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, and I don't know what format it is in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was submitted in 2009, and the description says the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Library strategy:     ncRNA-Seq&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Library source:   transcriptomic&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Library selection:    size fractionation&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Instrument model:     Illumina Genome Analyzer II&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Description:  CIPPNK, tar file of Illumina *_seq.txt files provided as supplementary file&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I got the archive here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/geo/samples/GSM455nnn/GSM455387/suppl/GSM455387%5FWT%5FCIPPNK%5Fseq%5Fs1%2Etar%2Egz&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ncbi.nlm.nih.gov/geo/samples/GSM455nnn/GSM455387/suppl/GSM455387%5FWT%5FCIPPNK%5Fseq%5Fs1%2Etar%2Egz&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inside, there are 330 files from &lt;code&gt;s_1_0001_seq.txt&lt;/code&gt; to &lt;code&gt;s_1_0330_seq.txt&lt;/code&gt; which are tab-delimited text files where the first column is always &lt;code&gt;1&lt;/code&gt;, the second has the number found in the file name, then 2 mysterious integers, and then what looks like a read of length 36, with sometimes a dot instead of a letter:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ head s_1_0330_seq.txt &#xA;1   330 690 785 TTCCTACATTGTTCCCCCATGCTGTTGGCACCATCA&#xA;1   330 44  145 TTTTTATCACGAGTTTTAAATCTGTAGTCACCATCA&#xA;1   330 53  141 AATAATGCATAACAAAACGGAATCTGTAGAA.AAA.&#xA;1   330 784 461 TAATTGTAGTGATTGATCAATCTGTAGGCACCATCA&#xA;1   330 588 634 TATTATGCACATTTTCTAGTTCACTGTAGGCACCAT&#xA;1   330 718 678 TTACATGTTTCGGGTAGGAGCCTGTAGGCACCATCA&#xA;1   330 635 834 TGTGATCATTAGTTCAAAGCCCCCTGTCGGCACCCT&#xA;1   330 494 523 TGAAAATCAAAAATGCTGAACTGTAGGCACCATCAA&#xA;1   330 393 783 TTTTTTTTTAAATTTAAAAAAACTGTAGGCACCATC&#xA;1   330 48  148 GTTTAACCGTGTAGACGTTGGTTTCTGTAGGCACCA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This seems to be some &quot;old&quot; high-throughput sequencing format. I think someone &lt;a href=&quot;http://seqanswers.com/forums/showthread.php?t=282&quot; rel=&quot;nofollow noreferrer&quot;&gt;in a 2008 message in sequanswers&lt;/a&gt; was dealing with this type of file:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://seqanswers.com/forums/showpost.php?p=1841&amp;amp;postcount=8&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://seqanswers.com/forums/showpost.php?p=1841&amp;amp;postcount=8&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is this format that seemed so standard back then, that the authors did not give more info than describing the files as &quot;Illumina *_seq.txt files&quot;? I do not dare ask them for such a trivial question (the given contact is a Nobel prize laureate and is probably too busy to answer random bioinformatics questions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;In particular, what are the columns 3 and 4, and what do the dots mean?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="298" LastEditDate="2017-10-31T17:57:08.250" LastActivityDate="2017-10-31T17:57:08.250" Title="What is this 5-column sequencing format from 2009?" Tags="|file-formats|hts|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2741" PostTypeId="2" ParentId="2740" CreationDate="2017-10-31T15:07:30.920" Score="7" Body="&lt;p&gt;This is an early Solexa/Illumina sequencer format. The columns are the identifier to location on the flowcell.  I believe the &amp;quot;.&amp;quot; was the original placeholder for an unread base, which has been replaced by and &amp;quot;N&amp;quot; in current Illumina sequencing output.&lt;/p&gt;&#xA;&lt;p&gt;From on &lt;a href=&quot;http://www.crg.eu/en/content/processing-and-analysis-illumina-sequencing-data&quot; rel=&quot;noreferrer&quot;&gt;http://www.crg.eu/en/content/processing-and-analysis-illumina-sequencing-data&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;seq.txt (Gerald)&lt;/p&gt;&#xA;&lt;p&gt;4 1 23 1566 ACCGCTCTCGTGCTCGTCGCTGCGTTGAGGCTTGCG&lt;/p&gt;&#xA;&lt;p&gt;The first 4 columns are an ID that provides location details of the cluster in the flow cell, followed by the read sequence. Seq.txt file size typically ranges between 250 Megabytes and 1 Gigabyte. One file is generated per lane.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="964" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-10-31T15:07:30.920" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2742" PostTypeId="1" AcceptedAnswerId="2773" CreationDate="2017-10-31T16:01:27.983" Score="6" ViewCount="1716" Body="&lt;p&gt;I have a long list of autoimmune-associated SNPs, and I want to boil it down so that I get one SNP representing each LD block. I chose to use PLINK's --clump option for this. I'm roughly following &lt;a href=&quot;https://neurogenetics.qimrberghofer.edu.au/iSECA/LD_clumping_tutorial.html&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; tutorial (but analyzing my own data).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have the full data from the original GWAS's, so I am using 1000 Genomes data as the reference for LD estimation. But, the 1000 Genomes data don't have all the SNPs I need. I end up a few hundred warnings:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Warning: 'rs12345678' is missing from the main dataset, and is a top variant.&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does PLINK handle this situation? More importantly, how should I handle this situation? I don't want to omit those SNPs, but I also don't want to take them all, for the same reason that I am doing this in the first place.&lt;/p&gt;&#xA;" OwnerUserId="1708" LastEditorUserId="131" LastEditDate="2017-11-07T22:50:01.247" LastActivityDate="2018-02-01T04:07:39.073" Title="PLINK clump behavior on missing SNPs?" Tags="|snp|gwas|plink|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2743" PostTypeId="2" ParentId="2637" CreationDate="2017-10-31T18:17:50.443" Score="1" Body="&lt;p&gt;SC3 is part of the Bioconductor library for R. It is based on PCA and spectral dimensionality reductions and utilizes k-means. In the introductory paper the authors perform analysis on a Drop-seq dataset of more than 44,000 cells. SC3 outputs several useful plots, including a consensus matrix, expression panel, and a marker genes heatmap. All of these utilize the hierarchical clustering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't speak from personal experience but the package seems very user friendly. Being a Bioconductor package means it has lots of support and excellent documentation. I am getting ready to use it myself for a small Drop-seq dataset of olfactory epithelial cells.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;SC3: consensus clustering of single-cell RNA-seq data : Nature Methods : Nature Research. Available at: &lt;a href=&quot;https://www.nature.com/nmeth/journal/v14/n5/full/nmeth.4236.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/nmeth/journal/v14/n5/full/nmeth.4236.html&lt;/a&gt;. (Accessed: 31st October 2017)&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2017-10-31T18:17:50.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2744" PostTypeId="1" AcceptedAnswerId="2746" CreationDate="2017-10-31T23:30:46.397" Score="6" ViewCount="1512" Body="&lt;p&gt;I'm evaluating possibilites for somatic tumor variant calling without paired-normal samples. I'm aware of the consequences without a normal sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the popular variant calling tools such as Strelka, VarScan etc require a normal sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the some of the possibilites I have? I'm looking for good sensitivity in exchange for higher false positives.&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="96" LastEditDate="2017-11-07T22:39:45.857" LastActivityDate="2018-05-30T12:15:35.770" Title="Somatic tumor only variant calling?" Tags="|variant-calling|cancer|genomics|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2745" PostTypeId="1" AcceptedAnswerId="2749" CreationDate="2017-10-31T23:59:16.627" Score="4" ViewCount="776" Body="&lt;p&gt;I want to retrieve abstracts/summaries for NCBI bookshelf entries, eg: &quot;NBK1440&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK25497/table/chapter2.T._entrez_unique_identifiers_ui/?report=objectonly&quot; rel=&quot;nofollow noreferrer&quot;&gt;Docs&lt;/a&gt; say the dbname is &quot;books&quot; and &lt;a href=&quot;https://dataguide.nlm.nih.gov/eutilities/utilities.html#efetch&quot; rel=&quot;nofollow noreferrer&quot;&gt;efetch guide&lt;/a&gt; says a rettype of &quot;docsum&quot; works for all databases. However when I make the query:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=books&amp;amp;id=NBK1440&amp;amp;rettype=docsum&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;returns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;ERROR&amp;gt;UID=1440: cannot get document summary&amp;lt;/ERROR&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The default query is useless (just echos back the ID): &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=books&amp;amp;id=NBK1440&amp;amp;retmode=text&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;returns&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1440&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Picking another 3:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=books&amp;amp;id=NBK1391,NBK410087,NBK1391&amp;amp;rettype=docsum&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Fails the same way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;ERROR&amp;gt;UID=1391: cannot get document summary&amp;lt;/ERROR&amp;gt;&#xA;&amp;lt;ERROR&amp;gt;UID=410087: cannot get document summary&amp;lt;/ERROR&amp;gt;&#xA;&amp;lt;ERROR&amp;gt;UID=1391: cannot get document summary&amp;lt;/ERROR&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The same goes for esummary:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=books&amp;amp;id=1440&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Dies on NBK1440 and removing NBK gives:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;ERROR&amp;gt;UID=1440: cannot get document summary&amp;lt;/ERROR&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The values come from the (9Mb) ClinVar citations file:  &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/pub/clinvar/tab_delimited/var_citations.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nlm.nih.gov/pub/clinvar/tab_delimited/var_citations.txt&lt;/a&gt; - the first is from tihs line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;15048   9   1800562     NCBIBookShelf   NBK1440&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;ANSWER&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks to arupgsh - yes the ClinVar citation_id was a RID for bookshelf not an ID like it is for Pubmed and PMC. FYI my solution was:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def get_summary_for_bookshelf_rid(bookshelf_rid):&#xA;    handle = Entrez.esearch(db=&quot;books&quot;, term=bookshelf_rid)&#xA;    search_results = Entrez.read(handle)&#xA;    handle = Entrez.esummary(db=&quot;books&quot;, id=','.join(search_results['IdList']))&#xA;    results = Entrez.read(handle)&#xA;&#xA;    for r in results:&#xA;        if r[&quot;RID&quot;] == bookshelf_rid:&#xA;            return r&#xA;&#xA;    return None&#xA;&#xA;get_summary_for_bookshelf_rid(&quot;NBK1440&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;returns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{'PubDate': '2000/04/03 00:00', 'Title': 'HFE-Associated Hereditary Hemochromatosis', 'Text': '', 'RType': 'chapter', u'Id': '1475938', u'Item': [], 'Book': 'gene', 'Parents': '', 'RID': 'NBK1440', 'ID': 'gene/chapter/hemochromatosis/PMC'}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1713" LastEditorUserId="1713" LastEditDate="2017-11-28T01:18:50.180" LastActivityDate="2017-11-28T01:18:50.180" Title="Retrieve abstract/summary for NCBI bookshelf via Entrez" Tags="|api|data-retrieval|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2746" PostTypeId="2" ParentId="2744" CreationDate="2017-11-01T02:33:18.947" Score="5" Body="&lt;p&gt;There is a &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/28659176&quot; rel=&quot;noreferrer&quot;&gt;recent paper&lt;/a&gt; that attempts to do this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;ISOWN: accurate somatic mutation identification in the absence of&#xA;  normal tissue controls.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;In this work, we describe the development, implementation, and&#xA;  validation of ISOWN, an accurate algorithm for predicting somatic&#xA;  mutations in cancer tissues in the absence of matching normal tissues.&#xA;  ISOWN is available as Open Source under Apache License 2.0 from&#xA;  &lt;a href=&quot;https://github.com/ikalatskaya/ISOWN&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/ikalatskaya/ISOWN&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="64" LastActivityDate="2017-11-01T02:33:18.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2748" PostTypeId="2" ParentId="2744" CreationDate="2017-11-01T11:33:47.077" Score="5" Body="&lt;p&gt;I've been using the &lt;a href=&quot;http://csb5.github.io/lofreq/&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;LoFreq*&quot;&gt;LoFreq* caller&lt;/a&gt; for exactly this. It is designed to find variants with very low frequency, so is well suited for this type of analysis.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;LoFreq* (i.e. LoFreq version 2) is a fast and sensitive variant-caller for inferring SNVs and indels from next-generation sequencing data. It makes full use of base-call qualities and other sources of errors inherent in sequencing (e.g. mapping or base/indel alignment uncertainty), which are usually ignored by other methods or only used for filtering.&lt;/p&gt;&#xA;&lt;p&gt;LoFreq* is very sensitive; most notably, it is able to predict variants below the average base-call quality (i.e. sequencing error rate). Each variant call is assigned a p-value which allows for rigorous false positive control. Even though it uses no approximations or heuristics, it is very efficient due to several runtime optimizations and also provides a (pseudo-)parallel implementation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;(In my experience it is not fast at all, but that's what they claim).&lt;/p&gt;&#xA;&lt;p&gt;I have used it many times to call somatic variants using the germline caller (and not its somatic one which requires a tumor/normal pair). Because the tool is so sensitive, its germline caller actually deals quite well with somatic samples.&lt;/p&gt;&#xA;&lt;p&gt;You can find instructions for its use &lt;a href=&quot;http://csb5.github.io/lofreq/commands/&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Commands for LoFreq*&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-05-30T12:15:35.770" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2749" PostTypeId="2" ParentId="2745" CreationDate="2017-11-01T13:22:36.060" Score="5" Body="&lt;p&gt;Entrez requires the unique identifiers(UID) for fetching related info. The id you are using in the query is RID. I guess that why you are getting &lt;code&gt;&amp;lt;ERROR&amp;gt;UID=1440: cannot get document summary&amp;lt;/ERROR&amp;gt;&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;List of example uids using the term &quot;ncbi+blast&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=books&amp;amp;term=ncbi+blast&quot; rel=&quot;noreferrer&quot;&gt;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=books&amp;amp;term=ncbi+blast&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Summary of the first item:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=books&amp;amp;id=4459608&quot; rel=&quot;noreferrer&quot;&gt;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=books&amp;amp;id=4459608&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the Entrez python package:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez&#xA;Entrez.email = &quot;youremail@domain.com&quot;&#xA;handle = Entrez.esummary(db=&quot;books&quot;, id=&quot;4459608,4456634&quot;, retmode=&quot;xml&quot;)&#xA;records = Entrez.parse(handle)&#xA;for record in records:&#xA;     print(record['Title'])&#xA;handle.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm not sure about the availability of book abstracts.&lt;/p&gt;&#xA;" OwnerUserId="206" LastEditorUserId="206" LastEditDate="2017-11-01T13:27:49.157" LastActivityDate="2017-11-01T13:27:49.157" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2750" PostTypeId="2" ParentId="2745" CreationDate="2017-11-01T14:54:09.100" Score="3" Body="&lt;p&gt;As arupgsh says, you need to use esearch to get a list of unique identifiers before using efetch to retrieve info about each result. I think the easiest way to do this is to use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez Direct&lt;/a&gt;, which allows you to simply pipe esearch output to efetch:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db books -query NBK1440 | efetch -format docsum&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db books -query NBK1440 | esummary&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="581" LastActivityDate="2017-11-01T14:54:09.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2751" PostTypeId="1" AcceptedAnswerId="2752" CreationDate="2017-11-02T19:12:13.773" Score="0" ViewCount="96" Body="&lt;p&gt;I want to make function get_abstract, but i don't know what this can look like if the function must return idlist. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez, Medline, SeqIO&#xA;&#xA;&#xA;list_of_ids = []&#xA;Entrez.email = 'ski89@gmail.com'&#xA;&#xA;handle_one = Entrez.esearch(db=&quot;pubmed&quot;, term='e coli', &#xA;                           retmode=&quot;text&quot;, retmax=100000)&#xA;record = Entrez.read(handle_one)&#xA;idlist = record[&quot;IdList&quot;]&#xA;for ID in idlist:&#xA;    list_of_ids.append(ID)&#xA;&#xA;    if len(list_of_ids) == 100000:&#xA;        handle_one = Entrez.esearch(db=&quot;pubmed&quot;, term='e coli', &#xA;                           retmode=&quot;text&quot;, retstart=100000, retmax=100000)&#xA;        record = Entrez.read(handle_one)&#xA;        idlist = record[&quot;IdList&quot;]&#xA;        for ID in idlist:&#xA;            list_of_ids.append(ID)&#xA;&#xA;            if len(list_of_ids) == 200000:&#xA;                handle_one = Entrez.esearch(db=&quot;pubmed&quot;, term='e coli', &#xA;                           retmode=&quot;text&quot;, retstart=200000, retmax=100000)&#xA;                record = Entrez.read(handle_one)&#xA;                idlist = record[&quot;IdList&quot;]&#xA;                for ID in idlist:&#xA;                    list_of_ids.append(ID)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;so the args will be the start and TERM:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;get_abstracts(start,TERM): &#xA;handle_one = Entrez.esearch(db=&quot;pubmed&quot;, term=TERM, &#xA;                               retmode=&quot;text&quot;, retstart=start, retmax=100000)&#xA;                    record = Entrez.read(handle_one)&#xA;                    idlist = record[&quot;IdList&quot;]&#xA;                    for ID in idlist:&#xA;                        list_of_ids.append(ID)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can someone help refactor this? cheers&lt;/p&gt;&#xA;" OwnerUserId="1224" LastEditorUserId="964" LastEditDate="2017-11-02T19:54:05.350" LastActivityDate="2017-11-02T20:20:32.367" Title="Biopython code refactoring" Tags="|python|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2752" PostTypeId="2" ParentId="2751" CreationDate="2017-11-02T20:20:32.367" Score="4" Body="&lt;p&gt;Here is a sample of code that should get you started.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I created a separate &lt;code&gt;get_abstracts&lt;/code&gt; function.  Then that function get called repetitively until less than 100K ids are returned.  If you wanted to get fancy this whole process could be called as a recursive function but that seemed overly complex.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BUFFER_SIZE = 100000&#xA;&#xA;def get_abstracts(start, term):&#xA;    &quot;&quot;&quot;Get N number of records from Entrez&#xA;    :returns list: List of ids returned from search&#xA;    &quot;&quot;&quot;&#xA;    handle_one = Entrez.esearch(db=&quot;pubmed&quot;, term=term, &#xA;                       retmode=&quot;text&quot;, retstart=start, retmax=BUFFER_SIZE)&#xA;    record = Entrez.read(handle_one)&#xA;    idlist = record[&quot;IdList&quot;]&#xA;    return idlist&#xA;&#xA;list_of_ids = []&#xA;start_at = 0&#xA;&#xA;# Iterate the search returning 100000&#xA;while true:&#xA;    ids = get_abstracts(start_at, 'e coli')&#xA;&#xA;    list_of_ids.extend(ids)&#xA;&#xA;    # Break if less than 100K results were returned &#xA;    if len(ids) &amp;lt; BUFFER_SIZE:&#xA;        break&#xA;    start_at += BUFFER_SIZE&#xA;&#xA;# Do something with list_of_ids&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="964" LastActivityDate="2017-11-02T20:20:32.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2753" PostTypeId="1" AcceptedAnswerId="2759" CreationDate="2017-11-03T00:33:45.017" Score="7" ViewCount="135" Body="&lt;p&gt;I'm currently creating an R package, and my scripts require before any analysis is done that 1-2 bed files be loaded. Normally, I would run the scripts with the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(data.table)&#xA;session_bed = read.table(&quot;/path/name/to/bedfile.bed&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then refer to &lt;code&gt;session_bed&lt;/code&gt; throughout the analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in order to release this as an R package, I believe that these files must be downloaded at the time on the package installation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) Normally, how are bed files distributed in R packages? Are these gzipped? Where are they placed? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;http://r-pkgs.had.co.nz/data.html&quot; rel=&quot;noreferrer&quot;&gt;http://r-pkgs.had.co.nz/data.html&lt;/a&gt;, all .bed files should be &lt;code&gt;inst/extdata&lt;/code&gt; I guess. However, it's not clear to me how to &quot;link&quot; to this data within the R package. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How have other packages dealt with this? (Is there a size limit?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) In this case, should the function loading the bed file be executed about loading the R library, &lt;code&gt;library(package_name)&lt;/code&gt;? Is there more appropriate option? &lt;/p&gt;&#xA;" OwnerUserId="146" LastActivityDate="2019-01-10T08:24:48.057" Title="How to release an R package with several bed files?" Tags="|r|bed|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2754" PostTypeId="1" AcceptedAnswerId="2755" CreationDate="2017-11-03T02:17:33.700" Score="2" ViewCount="44" Body="&lt;p&gt;I think I have a fairly easy-to-solve problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have an expression table generated from an Affy set. It has one row of gene names (already symbols) and then 180 sample rows. 181 rows total.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I made another excel file with one row with three different phenotypes I want to compare. That row has a label (&quot;classes&quot;) and 180 names. 181 rows total.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In R, I can import each of these &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dat &amp;lt;- read.csv(&quot;expression.csv&quot;)&#xA;dat &amp;lt;- data.matrix(dat)&#xA;pheno &amp;lt;- read.csv(&quot;pheno.csv&quot;)&#xA;pheno &amp;lt;- data.matrix(pheno)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can then fit them&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fit &amp;lt;- lmFit(dat,pheno)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And ask for an eBayes from limma&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fit &amp;lt;- eBayes(fit)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, when I view the topTable,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;table &amp;lt;- topTable(fit)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My first column has numbers (like &quot;834&quot;,&quot;1142&quot;), when I'd expect to have gene names there (like &quot;ACTR&quot;, &quot;CHRNA1&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I doing wrong? Or, rather, am I doing anything right? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1226" LastEditorUserId="1226" LastEditDate="2017-11-03T12:33:19.090" LastActivityDate="2017-11-03T12:33:19.090" Title="Simple Affy Differential Expression" Tags="|rna-seq|bioconductor|differential-expression|microarray|rna|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2755" PostTypeId="2" ParentId="2754" CreationDate="2017-11-03T02:55:36.503" Score="1" Body="&lt;p&gt;What do the first few lines of your lmFit input variables look like (i.e. &lt;code&gt;dat&lt;/code&gt; and &lt;code&gt;pheno&lt;/code&gt;)? Are the rows numbered, or do they use the actual symbols?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By default, I don't think &lt;code&gt;read.csv&lt;/code&gt; expects a file to have row numbers (even though &lt;code&gt;write.csv&lt;/code&gt; puts them in by default). Try adding a &lt;code&gt;row.names&lt;/code&gt; parameter to the expression matrix read call:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dat &amp;lt;- read.csv(&quot;expression.csv&quot;, row.names=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-11-03T02:55:36.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2756" PostTypeId="1" AcceptedAnswerId="2758" CreationDate="2017-11-03T03:56:13.380" Score="4" ViewCount="216" Body="&lt;p&gt;Given a BAM file, I'd like to calculate the sum, over all reads, of the mapping quality and the number of mapped bases (i.e. number of M's in the CIGAR string).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, given two reads like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;read1: mapq=40, mapped_bases=10&#xA;read2: mapq=20, mapped_bases=50&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The number I'm looking for is: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(40*10) + (20*50) = 1400&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One option here is to stream the file and get the two quantities line-by-line. But this seems likely to be slow. Thoughts welcome.&lt;/p&gt;&#xA;" OwnerUserId="156" LastEditorUserId="156" LastEditDate="2017-11-03T05:20:55.693" LastActivityDate="2017-11-03T08:21:24.693" Title="sum of products of mapq and mapped bases for each read in a from a BAM file" Tags="|sam|samtools|mapq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2757" PostTypeId="2" ParentId="2756" CreationDate="2017-11-03T04:13:52.200" Score="0" Body="&lt;p&gt;One possibility uses bamutils (&lt;a href=&quot;http://ngsutils.org/modules/bamutils/removeclipping/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ngsutils.org/modules/bamutils/removeclipping/&lt;/a&gt;) to remove the clipped bases, then awk to get the sum of the products, something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bamutils removeclipping val.bam val_rc.bam&#xA;samtools view val_rc.bam | awk '{print length($10)*$5}' | awk '{sum+=$0} END { print &quot;mapq*len sum =&quot;,sum}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the bamutils step is rather slow, but it will do the job.&lt;/p&gt;&#xA;" OwnerUserId="156" LastEditorUserId="156" LastEditDate="2017-11-03T06:00:07.460" LastActivityDate="2017-11-03T06:00:07.460" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2758" PostTypeId="2" ParentId="2756" CreationDate="2017-11-03T08:21:24.693" Score="3" Body="&lt;p&gt;BAM files are stored line by line, so your only option is to process it that way. In python with &lt;code&gt;pysam&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pysam&#xA;&#xA;bam = pysam.AlignmentFile(&quot;something.bam&quot;)&#xA;tot = 0&#xA;for read in bam.fetch():&#xA;    if read.is_unmapped:&#xA;        continue&#xA;    tl = 0&#xA;    for op, l in read.cigartuples:&#xA;        if op in [0, 7, 8]:&#xA;            tl += l&#xA;    tot += tl * read.mapping_quality&#xA;bam.close()&#xA;print(&quot;the total is {}&quot;.format(tot))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-11-03T08:21:24.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2759" PostTypeId="2" ParentId="2753" CreationDate="2017-11-03T08:33:00.317" Score="7" Body="&lt;p&gt;You can store it in &lt;code&gt;data/&lt;/code&gt; subfolder. If one of your functions need this bed file, you can include importing it in your function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some &lt;a href=&quot;http://www.davekleinschmidt.com/r-packages/&quot; rel=&quot;noreferrer&quot;&gt;examples&lt;/a&gt; how to get the data from &lt;code&gt;data/&lt;/code&gt; subfolder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are probably more ways to do this, this is just one way.&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2017-11-03T08:46:55.437" LastActivityDate="2017-11-03T08:46:55.437" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2761" PostTypeId="1" AcceptedAnswerId="2764" CreationDate="2017-11-03T11:47:22.323" Score="10" ViewCount="7560" Body="&lt;p&gt;I would like to do easily reproducible analysis using publicly available data from NCBI, so I have chosen a &lt;code&gt;snakemake&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to write a single rule, that would be able to download any genome given a species code name and separated table of species and their NCBI IDs. So I wrote a script &lt;code&gt;scripts/download.sh&lt;/code&gt; that takes a species code and download the genome to &lt;code&gt;data/&amp;lt;species_code&amp;gt;/genome.fa.gz&lt;/code&gt;. The script internally reads the table &lt;code&gt;tables/download_table.tsv&lt;/code&gt;, where are corresponding species code names and NCBI IDs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I tried to do a &lt;code&gt;snakemake&lt;/code&gt; like this :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;species='Cbir Avag Fcan Lcla Dcor Dpac Pdav Psp62 Psp79 Minc1 Minc2 Mjav Mare Mflo Mhap Pant'&#xA;&#xA;rule download:&#xA;    input :&#xA;        &quot;tables/download_table.tsv&quot;&#xA;    output :&#xA;        &quot;data/{sp}/genome.fa.gz&quot;&#xA;    shell :&#xA;        &quot;scripts/download.sh {sp}&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, &lt;code&gt;snakemake&lt;/code&gt; returned an error message I do not really understand :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Target rules may not contain wildcards. Please specify concrete files or a rule without wildcards.`. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a way to write a single rule for downloading all the genomes?&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2017-11-05T11:27:02.857" LastActivityDate="2017-11-05T11:52:23.763" Title="How to resolve in snakemake error : &quot;Target rules may not contain wildcards.&quot;" Tags="|snakemake|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2762" PostTypeId="2" ParentId="2753" CreationDate="2017-11-03T11:48:41.367" Score="6" Body="&lt;p&gt;As &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2759/48&quot;&gt;benn&lt;/a&gt; said you can store them in &lt;code&gt;data/&lt;/code&gt; subfolder of your package. There is no size limit (AFAIK) but if you plan to publish your package in some repositories they might have limits or other restrictions. Bioconductor requires to divide the data required for a software package at certain threshold, but for small amount of data it is fine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find them via &lt;code&gt;system.file&lt;/code&gt; like: &lt;code&gt;read.csv(system.file(&quot;data/mydata.csv&quot;, package = &quot;mypackage&quot;))&lt;/code&gt;.&#xA;Of course your bed files will need the apropiate function to load them.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can load when your library is called by modifying the &lt;code&gt;local&lt;/code&gt; variable, or using &lt;code&gt;.First&lt;/code&gt; (I am not so sure of the last one), but this pollutes the environment and sometimes if preferably to use &lt;code&gt;data&lt;/code&gt; function. You might need to convert your files to &lt;code&gt;.Rdata&lt;/code&gt; file.&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2019-01-10T08:24:48.057" LastActivityDate="2019-01-10T08:24:48.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2764" PostTypeId="2" ParentId="2761" CreationDate="2017-11-03T12:49:58.650" Score="9" Body="&lt;p&gt;The problem is that you need a master rule that requires all of your desired outputs as inputs, in your case it would be :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule all:&#xA;    input:&#xA;        expand(&quot;data/{sp}/genome.fa.gz&quot;, sp=species.split(' '))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You'll also need separate download link inputs for each species. You could make a separate &lt;code&gt;download_table.tsv&lt;/code&gt; for each species, but it would probably be easier to make a config file with this information, and add a params keyword to your rule. Something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule download:&#xA;   params:&#xA;       url=config['locations']['sp']&#xA;   output :&#xA;       &quot;data/{sp}/genome.fa.gz&quot;&#xA;   shell :&#xA;       &quot;scripts/download.sh {params.url}&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="581" LastEditorUserId="57" LastEditDate="2017-11-05T11:52:23.763" LastActivityDate="2017-11-05T11:52:23.763" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2766" PostTypeId="2" ParentId="2761" CreationDate="2017-11-03T18:01:05.533" Score="3" Body="&lt;p&gt;Given the Makefile you provided and which someone then deleted from the question, you should probably just add this line (this should be the first rule):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;rule all: input: [&quot;data/{}/genome.fa.gz&quot;.format(x) for x in species.split()]&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This rule just specifies a list of expected output files and corresponds to the following lines from the original &lt;code&gt;Makefile&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;GENOMES=$(patsubst %, data/%/genome.fa.gz, $(SPECIES))&#xA;all : $(GENOMES)&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="425" LastActivityDate="2017-11-03T18:01:05.533" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2767" PostTypeId="2" ParentId="2478" CreationDate="2017-11-03T19:49:54.830" Score="1" Body="&lt;p&gt;After a more extensive search, I found a useful answer that I'll share in case anyone else wonders how to do this. I'm using &lt;a href=&quot;https://github.com/al2na/methylKit&quot; rel=&quot;nofollow noreferrer&quot;&gt;MethylKit&lt;/a&gt; package for differential methylation expression and it is quite easy to integrate with &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/GenomicRanges.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genomic Ranges&lt;/a&gt; and &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/rtracklayer.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;rtracklayer&lt;/a&gt; in R to extract particular element methylation differences:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;element.of.interest &amp;lt;- import.bed(&quot;FILENAME.bed&quot;) #download custom track from Genome Browser&#xA;element.meth &amp;lt;- diffmeth[as(diffmeth, &quot;GRanges&quot;) %over% element.of.interest]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &quot;diffmeth&quot; is the resulting MethylKit object from the differential methylation analysis.&lt;/p&gt;&#xA;" OwnerUserId="1375" LastActivityDate="2017-11-03T19:49:54.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2768" PostTypeId="2" ParentId="2478" CreationDate="2017-11-04T06:34:33.567" Score="1" Body="&lt;p&gt;Via &lt;a href=&quot;http://bedops.readthedocs.io&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap —-echo —-echo-map-id-uniq &amp;lt;(bam2bed &amp;lt; reads.bam) &amp;lt;(rmsk2bed &amp;lt; repeats.rmsk) &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="776" LastActivityDate="2017-11-04T06:34:33.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2769" PostTypeId="1" CreationDate="2017-11-04T12:37:06.923" Score="5" ViewCount="434" Body="&lt;p&gt;I am currently performing a large RNA-seq analysis from mice PBMCs. The dataset contains around 6,000 transcriptomic profiles and I would like to use DESeq2 to identify the sets of differentially expressed genes in the different conditions. In total, I have 100 biological stimulations, and for each stimulation I have 30 control samples and 30 samples treated with a molecule of interests (so is have: (30 controls+30 treated) * 100 stimulations = 6,000 samples). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to identify, for each stimulation, the sets of differentially expressed genes between control samples and treated samples. I do not want to compare samples from the different stimulations. In total, I would like to have thus 100 lists of differentially expressed genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have started to use deseq2 to identify these lists but deseq2 is spending lot of time to perform comparisons on which I am not interested with (comparisons between the biological stimulations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For now, I have a table sampleTable which looks like that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/K95Y5.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/K95Y5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using DEseq2 using the following command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DESeqDataSetFromHTSeqCount(sampleTable = sampleTable, directory = directory, design= ~ condition)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Could you please help me with that? How can I specify to deseq2 to not perform the comparisons between the biological stimulations, but rather between control and treated sample within each stimulation ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you and best,&lt;/p&gt;&#xA;" OwnerUserId="1730" LastActivityDate="2017-11-04T19:09:02.183" Title="How can I specify to DEseq2 to only perform comparisons on which I am interested with?" Tags="|rna-seq|differential-expression|deseq2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2771" PostTypeId="2" ParentId="2769" CreationDate="2017-11-04T19:09:02.183" Score="3" Body="&lt;p&gt;You can specify the exact comparisons you want in the &lt;code&gt;results()&lt;/code&gt; function. So:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dds = DESeqDataSetFromHTSeqCount(sampleTable = sampleTable, directory = directory, design= ~ condition)&#xA;dds = DESeq(dds)&#xA;res = results(dds, contrast(&quot;condition&quot;, &quot;treatment1&quot;, &quot;control1&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;code&gt;condition&lt;/code&gt; should not be, &quot;&lt;code&gt;stim001-control1&lt;/code&gt;, &lt;code&gt;stim001-control2&lt;/code&gt;, etc.&quot;, but instead, &quot;&lt;code&gt;stim001control&lt;/code&gt;, &lt;code&gt;stim001control&lt;/code&gt;, &lt;code&gt;stim001control&lt;/code&gt;, &lt;code&gt;stim001treatment&lt;/code&gt;, etc.&quot;. Don't put minus signs in your levels and don't number your samples in them (they're not in different groups.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last command would be repeated for each of the comparisons. Note that this will still be quite slow due to the number of samples and the number of groups. With so many samples, you might just make a separate &lt;code&gt;sampleTable&lt;/code&gt; for each of the comparisons you want to make. Each would then only contain the samples relevant for that comparison, so 60 total in each.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, if you want to fit all of the groups at once then it's likely that limma/voom will prove to have better performance. It uses different math that happens to be quicker with large models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, if the slowness is coming from &lt;code&gt;DESeqDataSetFromHTSeqCount()&lt;/code&gt; then you can mere everything on the command line into a single matrix. A couple lines of python or even the &lt;code&gt;join&lt;/code&gt; command can do that quickly enough.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-11-04T19:09:02.183" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2772" PostTypeId="1" CreationDate="2017-11-04T23:16:21.203" Score="0" ViewCount="814" Body="&lt;p&gt;I want to assemble a linear genome and I have a reference genome too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran the following commands for de novo velvet assembly: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;velveth auto 31,45,2 -fastq -shortPaired1 -separate 2-CA-SVA-2_S2_L001_R1_001.fastq 2-CA-SVA-2_S2_L001_R2_001.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I chose auto_33 for further analysis&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;velvetg auto_33 -exp_cov auto&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now my output files inside auto_33 folder are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;contigs.fa&#xA;Graph2&#xA;LastGraph&#xA;Log&#xA;PreGraph&#xA;Roadmaps&#xA;Sequences&#xA;stats.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I have no clue how to move ahead. Can I do anything else with velvet or do I have to switch to another program?&lt;/p&gt;&#xA;" OwnerUserId="1684" LastEditorUserId="73" LastEditDate="2017-11-05T02:57:43.430" LastActivityDate="2017-11-05T13:46:05.793" Title="How to proceed after contigs.fa file obtained from VELVET assembly" Tags="|ngs|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2773" PostTypeId="2" ParentId="2742" CreationDate="2017-11-05T12:01:23.483" Score="6" Body="&lt;p&gt;Most likely, the GWASs that generated your summary statistics used other imputation panels than 1000G, like HRC. Clearly, PLINK can't estimate the LD for SNPs that aren't found in the reference, and most likely just ignores them. I think the easiest solution is to use whatever individual-level dataset you have to estimate the correlations - even if it's small or from a different population. The estimate will be biased, but you don't really need high precision for clumping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then again, you can skip PLINK and try searching for SNP prioritization methods based on summary statistics - it's an area of active development, and something might be available to suit your needs.&lt;/p&gt;&#xA;" OwnerUserId="920" LastActivityDate="2017-11-05T12:01:23.483" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2774" PostTypeId="2" ParentId="2772" CreationDate="2017-11-05T13:46:05.793" Score="3" Body="&lt;p&gt;The contigs.fa file is a FASTA formatted file of the final contigs generated by velvet.  I would start assessing the quality of the assembly that was generated. Here are some questions to get you started.  Really the next steps depend on how you plan to use the assembly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the distribution of contig lengths? Average and N50 contig length&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does the assembly compare to the reference?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How much of the reference does the assembly cover?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;When you align the original reads back to the contigs how well are they covered? &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Do any of the read alignments not make sense? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there sections of the contigs that are not covered?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="964" LastActivityDate="2017-11-05T13:46:05.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2775" PostTypeId="1" AcceptedAnswerId="2966" CreationDate="2017-11-05T15:53:07.910" Score="1" ViewCount="57" Body="&lt;p&gt;I am trying to collect the ec-code (and gene) of a reaction. I have tried the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pythoncyc&#xA;pgdb = pythoncyc.PGDB(&quot;sco&quot;)&#xA;r = &quot;RXN1A0-6128&quot;&#xA;gene = &quot;SCP1.233B&quot;&#xA;enzyme = pgdb.enzymes_of_reaction(r)&#xA;enzyme = pgdb.enzymes_of_gene(gene)&#xA;pgdb.full_enzyme_name(enzyme)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I only get an enzyme id, and not the ec-code, which is visible on &lt;a href=&quot;https://biocyc.org/SCO/NEW-IMAGE?type=REACTION&amp;amp;object=RXN1A0-6128&quot; rel=&quot;nofollow noreferrer&quot;&gt;biocyc&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1670" LastEditorUserId="48" LastEditDate="2017-12-01T08:24:49.540" LastActivityDate="2017-12-01T08:24:49.540" Title="Using pythoncyc to get ec-code of a reaction" Tags="|python|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2776" PostTypeId="2" ParentId="2323" CreationDate="2017-11-05T21:31:53.310" Score="1" Body="&lt;p&gt;These suggestions are based on your comment that you have the gene sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a statistical approach, you could formulate a null model and show that the sequence similarity you observe is much higher than expected under the null model, but reasonable under your exon shuffling model. Ideally, you would show this individually for each exon, to convince the reader about the exons' disparate origins. I'll propose a reasonable null model, though suggestions are gladly accepted here. Pick a random gene, assume it's built using exon shuffling, and search the rest of the transcriptome for sequences that best substantiate that argument. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an evolutionary approach, you could search for orthologs (homologues? I never remember the difference) of your gene in other species. Hopefully, the results would show that your gene formed more recently than the exons it's built out of. If your gene actually has some other origin, maybe you'll find out it's older than some of its putative components. This can't exactly &quot;confirm&quot; your hypothesis, but it will test your hypothesis and just be generally a cool thing to try.&lt;/p&gt;&#xA;" OwnerUserId="1708" LastActivityDate="2017-11-05T21:31:53.310" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2777" PostTypeId="1" CreationDate="2017-11-06T00:28:25.210" Score="6" ViewCount="604" Body="&lt;p&gt;I have asked this question in &lt;a href=&quot;https://www.biostars.org/p/281809/&quot; rel=&quot;nofollow noreferrer&quot;&gt;biostars&lt;/a&gt;, but I am trying here as well as people working with &quot;omics&quot; data might be able to help. I think my issue relates understanding how large data storage on online server works. I am trying to download large data from the Human Microbiome Project. These are available on online server and can be downloaded via aspera (ascp).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't figure out the right path to get the data. On &lt;a href=&quot;https://www.hmpdacc.org/hmp/resources/download.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;their website&lt;/a&gt;, they give an example of command to retreive an example dataset:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; ascp -l 100M -QT user@aspera.hmpdacc.org:/ReferenceGenomes/test_genome.tar /path/to/local/directory&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I executed this, using my own username, from this I am asked my password, and I can authenticate. But then I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(Error: Server aborted session: No such file or directory)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, I downloaded the manifest file via the HMP portal. In this manifest, they provide the path to which these files are supposed to be available. I tried to execute again the command with one of those path, provided by the manifest file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ascp -l 100M -QT user@aspera.hmpdacc.org:/files/5a950f27980b5d93e4c16da124000378  /path/to/local/directory&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I again get the same error of &quot;No such file or directory&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://downloads.hmpdacc.org/data2/hhs/genome/microbiome/wgs/analysis/hmwgsqc/v2/SRS013951.tar.bz2&quot; rel=&quot;nofollow noreferrer&quot;&gt;full url&lt;/a&gt; provided by the manifest&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried putting different chunks of this address into to ascp command but nothing works. I think there is something I don't understand about how to access the server via aspera and what is expected to go in this path. When I copy paste this url in google, this directly download the file I want. So I'm close, but I need to do it via command line as I need to download hundreds of these files and they are huge (6-10GB each).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I download the data of the human microbiome project using ascp?&lt;/p&gt;&#xA;" OwnerUserId="1741" LastEditorUserId="48" LastEditDate="2017-11-06T11:04:52.770" LastActivityDate="2018-07-04T16:09:17.127" Title="Download data from the Human Microbiome Project via ascp" Tags="|data-download|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2778" PostTypeId="5" CreationDate="2017-11-06T08:25:14.340" Score="0" Body="&lt;p&gt;Some database examples are GEO, NCBI, Uniprot.&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-11-06T11:04:38.787" LastActivityDate="2017-11-06T11:04:38.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2779" PostTypeId="4" CreationDate="2017-11-06T08:25:14.340" Score="0" Body="Use this tag for questions about how to use, download, retrieve, check and browse public databases. " OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-11-06T11:05:00.370" LastActivityDate="2017-11-06T11:05:00.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2780" PostTypeId="2" ParentId="2777" CreationDate="2017-11-06T08:34:54.207" Score="-1" Body="&lt;p&gt;I can't test for ascp because they do not longer support my current setup but you can follow the &lt;a href=&quot;https://www.hmpdacc.org/hmp/resources/ftp_public.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;FTP instructions&lt;/a&gt;. With FileZilla:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;url: public-ftp.ihmpdcc.org&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2017-11-06T08:34:54.207" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2782" PostTypeId="2" ParentId="2702" CreationDate="2017-11-06T11:02:28.493" Score="1" Body="&lt;p&gt;From &lt;a href=&quot;http://bio-bwa.sourceforge.net/bwa.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;BWA manual&lt;/a&gt;, it is advised to use BWA-MEM in your longs read case (unless you have a good reason not to):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;BWA-MEM and BWA-SW share similar features such as long-read support and split alignment, but BWA-MEM, which is the latest, is generally recommended for high-quality queries as it is faster and more accurate.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I guess that BWA-SW returns several lines per transcript (based on local alignments). In that case, the &lt;a href=&quot;https://sourceforge.net/p/bio-bwa/mailman/message/31199486/&quot; rel=&quot;nofollow noreferrer&quot;&gt;issue has already been reported&lt;/a&gt;. The user asks: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How can I differentiate between a read that has multiple hits in the reference and a read that has a split alignment?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The author replies: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In principle, it is hard to tell. Bwa-mem makes an arbitrary choice.&#xA;  Mappings flagged with 0x100 are multiple hits. Mappings flagged with&#xA;  0x800 are from a chimeric alignment.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So you could actually filter by &lt;a href=&quot;https://broadinstitute.github.io/picard/explain-flags.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;flag&lt;/a&gt;. But I am afraid the output could include something you do not necessarily want to (since there is no specific flag for the split reads).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe a better solution: you could switch to some mapper that will report the proper split in the hit of the read map, such as &lt;a href=&quot;https://github.com/lh3/minimap2&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimap2&lt;/a&gt; from the same author. It is suitable for long reads and comes with a &lt;code&gt;splice -x&lt;/code&gt; feature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, the laziest solution I see here would be to remove the split alignments exploiting the &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;sam format&lt;/a&gt; and in particular the CIGAR string. Meaning, to remove the sam fields containing a &lt;code&gt;N&lt;/code&gt; in their CIGAR (aka, skipped region from the reference). Something like this would work: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat alignment.sam | awk '$6 !~ /N/ {print $0}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Hope this can be somewhat helpful.&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2017-11-06T11:02:28.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2783" PostTypeId="1" AcceptedAnswerId="2784" CreationDate="2017-11-06T11:28:30.670" Score="9" ViewCount="2036" Body="&lt;p&gt;I am working on a project in which I need to compute several parameters (such GC content and length) of 5'UTR and 3'UTR sequences of &lt;em&gt;Saccharomyces cerevisiae&lt;/em&gt; yeast genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is finding a proper annotation for these regions in yeast. I have tried with BiomaRt and these sequences are not available for yeast. I have also tried with UCSC table browser to obtain a BED file with 5'UTR or 3'UTR sequences but it is returning only coordinates of tRNAs, pseudogenes, ncRNAs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, &lt;strong&gt;is there a straight-forward way to obtain an annotation for 5'UTR and 3'UTR regions in &lt;em&gt;Saccharomyces cerevisiae&lt;/em&gt; yeast&lt;/strong&gt;?&lt;/p&gt;&#xA;" OwnerUserId="678" LastEditorUserId="298" LastEditDate="2017-11-06T16:26:47.353" LastActivityDate="2021-11-01T20:34:23.587" Title="5'UTR and 3'UTR annotation in yeast" Tags="|sequence-annotation|bed|gtf|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2784" PostTypeId="2" ParentId="2783" CreationDate="2017-11-06T13:28:04.930" Score="9" Body="&lt;p&gt;I am unaware of any &quot;official&quot; or gold-standard UTR annotations in &lt;em&gt;S. cerevisiae&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One option is to &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE39128&quot; rel=&quot;noreferrer&quot;&gt;use the annotations&lt;/a&gt; from the TIF-Seq publication (&lt;a href=&quot;https://www.nature.com/nature/journal/v497/n7447/full/nature12121.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;Pelechano et al. 2013&lt;/strong&gt;&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;GSE39128_tsedall.txt.gz&lt;/code&gt; file contains the major isoforms identified. It would be up to you to computational associate each transcript with a given gene. It is also up to you to determine which major isoforms are present in your dataset. Furthermore based on my own experience there is often a significant number of discrepancies between what I see in an NGS dataset (e.g. RNA-Seq) and what is called as a possible isoform in these annotations. However, a lot of this could be related to the inherent technological limitations of the method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at the SGD genome browser they provide some additional UTR annotations. Derived from &lt;a href=&quot;http://science.sciencemag.org/content/320/5881/1344&quot; rel=&quot;noreferrer&quot;&gt;&lt;strong&gt;Nagalakshmi et al. 2008&lt;/strong&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find the relevant annotations for those UTR's in their supplemental data section. The file is &lt;code&gt;1158441_tables_s2_to_s6.zip&lt;/code&gt; and inside you will find &lt;em&gt;TableS4&lt;/em&gt; which seems to contain what you seek.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There might be more resources but these are the two main ones I'm aware of.&lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="29" LastEditDate="2017-11-06T14:04:50.190" LastActivityDate="2017-11-06T14:04:50.190" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2785" PostTypeId="1" AcceptedAnswerId="2787" CreationDate="2017-11-06T15:30:01.433" Score="4" ViewCount="930" Body="&lt;p&gt;I wanted to generate a clustering heat map for the microarray data. This is the first time I'm working on Microarray data. I read some tutorials but have few doubts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using microarray (Affymetrix SNP 6.0 data) gene expression data. For example the data looks like following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ProbeID S1  S2  S3  S4  S5  S6  S7&#xA;10008   131.4311    369.4926    222.0441    687.4181    176.8892    258.1233    316.5573&#xA;10010   78.73022    83.97501    81.56039    86.11443    78.09758    81.88231    84.17101&#xA;10014   90.02816    95.07267    101.1761    93.35585    81.96468    94.3553 93.89527&#xA;10017   79.86837    81.63064    88.19524    79.47265    76.3437 101.6351    93.71674&#xA;10019   99.03493    109.1835    104.97  102.7423    108.3677    98.93459    101.4052&#xA;10020   79.58075    84.28915    90.53562    74.47786    75.96112    96.39649    95.8828&#xA;10021   121.5373    149.9351    146.5956    122.8523    110.5759    132.4268    130.4409&#xA;10025   616.5994    1326.735    1358.187    2315.851    1068.745    3229.759    4435.021&#xA;10035   70.44073    69.56772    68.25446    68.35857    70.86771    74.3843 67.93569&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I created expression matrix and did log2 conversion. Now the data looks like following.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;            S1  S2  S3  S4  S5  S6  S7&#xA;Gene1   6.429276339 6.338451158 6.333760753 6.419191996 6.503471181 6.329103499 6.211373601&#xA;Gene2   6.379471993 6.296018518 6.237316465 6.2696332   6.329489132 6.359770303 6.240070336&#xA;Gene3   12.84498365 12.00265682 13.92741965 12.553162   13.39001307 13.8933423  12.58695704&#xA;Gene4   8.661860382 6.723004202 6.300975176 7.661012019 7.905219709 6.957578023 6.70945883&#xA;Gene5   6.945382967 6.814979733 6.575916303 6.63460198  6.627380524 6.733926424 6.280618235&#xA;Gene6   9.280222581 8.81560969  9.683073561 9.480038673 8.801438707 9.190943942 7.743705471&#xA;Gene7   6.593564871 6.63502488  6.389962535 6.511360029 6.694572404 6.6492763   6.527199544&#xA;Gene8   6.431615372 6.309515078 6.248153876 6.329288965 6.46768078  6.355268547 6.384284754&#xA;Gene9   7.513349406 7.234654595 7.490935892 6.801368215 7.323811386 7.018733196 7.055932044&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Before clustering for normalisation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I applied a function for normalisation &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data = t(apply(data, 1, function(x) {&#xA;  q10 = quantile(x, 0.1)&#xA;  q90 = quantile(x, 0.9)&#xA;  x[x &amp;lt; q10] = q10&#xA;  x[x &amp;gt; q90] = q90&#xA;  scale(x)&#xA;}))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I apply the above function I see there are values which are positive and also negative. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And when I used &quot;rma&quot; for background correction and &quot;normalise&quot; function from &quot;oligo&quot; package I don't see any negative values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which one should I use to check the clustering among samples? Any help is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="1369" LastActivityDate="2017-11-26T03:26:46.203" Title="Normalizing microarray data for clustering heat map" Tags="|r|bioconductor|normalization|microarray|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2786" PostTypeId="2" ParentId="2783" CreationDate="2017-11-06T16:43:28.647" Score="7" Body="&lt;p&gt;I found the following two files in &lt;a href=&quot;https://downloads.yeastgenome.org/sequence/S288C_reference/&quot; rel=&quot;noreferrer&quot;&gt;https://downloads.yeastgenome.org/sequence/S288C_reference/&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://downloads.yeastgenome.org/sequence/S288C_reference/SGD_all_ORFs_3prime_UTRs.fsa&quot; rel=&quot;noreferrer&quot;&gt;SGD_all_ORFs_3prime_UTRs.fsa&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://downloads.yeastgenome.org/sequence/S288C_reference/SGD_all_ORFs_5prime_UTRs.fsa&quot; rel=&quot;noreferrer&quot;&gt;SGD_all_ORFs_5prime_UTRs.fsa&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;According to the README files in the same directory, these are (the README for the 5' file is equivalent):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;SGD_all_ORFs_3prime_UTRs.README&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Information about the SGD_all_ORFs_3prime_UTRs.fsa file.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;This file contains sequences for predicted 3' UTR sequences for all&#xA;  characterized, uncharacterized and dubious ORFs in S288c.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;3' UTR coordinates were retrieved from YeastMine&#xA;  (yeastmine.yeastgenome.org) on July 3, 2017 using the stop coordinates&#xA;  of both the predicted transcript and gene sequences. A BED file with&#xA;  these coordinates was made. BED files were fixed on Sept 22, 2017 to&#xA;  account for strandedness. Sequences were batch downloaded again in&#xA;  FASTA format from USCS Genome Table Browser on Sept 22, 2017&#xA;  (&lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTables?hgsid=604915183_aUc9FZ1vwyAccGti1uI0wteYEM7x&quot; rel=&quot;noreferrer&quot;&gt;https://genome.ucsc.edu/cgi-bin/hgTables?hgsid=604915183_aUc9FZ1vwyAccGti1uI0wteYEM7x&lt;/a&gt;)&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;File created: 2017-08-25&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Those therefore seem to be the 3' and 5' UTR sequences of all yeast genes in FASTA format and should be exactly what you need. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-11-06T16:43:28.647" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2787" PostTypeId="2" ParentId="2785" CreationDate="2017-11-06T20:35:15.903" Score="5" Body="&lt;p&gt;You see negative values with your function because you're setting the average of each row to 0 and its standard deviation to 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, I would trust a standard normalization method (rma in this case) more than some random &quot;truncate and then scale the rows&quot; method. Your method isn't even doing any between-array normalization, which is the benefit of rma.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-11-06T20:35:15.903" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="2788" PostTypeId="1" CreationDate="2017-11-07T05:03:04.910" Score="1" ViewCount="572" Body="&lt;p&gt;I already know the Bash AWK solution and also R code for the question that I am asking. But, my file is so big and with R it takes very long time and I am afraid of AWK's mistake for this computation, so I wonder if there is a Perl or Python solution for my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a population allele count data, and it looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   0   0   0   0   0   0   0   0   0   1   2   1   0   0   0   0&#xA;0   2   0   0   0   0   0   0   0   0   0   4   0   2   0   0   0&#xA;0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0&#xA;2   2   0   0   2   1   0   0   0   0   2   4   0   0   0   2   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Columns are populations and rows are SNPs. I have two rows per SNP (one row for the number of copies of allele &quot;A&quot; in each population and one row for the number of copies of allele &quot;a&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the example above, the first and second rows are number of alleles for the SNP1, the third and fourth rows are alleles for SNP 2, and like this until SNP20000000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to calculate population allele frequency per SNP for all populations: frequency of allele A at SNP1 in population 1 = number of copies of allele A in population/total number of A/a gene copies in population and frequency of allele a at SNP1 in population 1 = number of copies of allele a in population/total number of A/a gene copies in the population. This means for each SNP number of copies of each allele should be divided to sum of the number of copies of allele &quot;A&quot; and &quot;a&quot; for each population. This is my desired output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   0   0   0   0   0   0   0   0   0   1   0.333333    1   0   0   0   0&#xA;0   1   0   0   0   0   0   0   0   0   0   0.666667    0   1   0   0   0&#xA;0   0   0   0   0   0   0   0   0   0   0   0           0   0   0   0   0&#xA;1   1   0   0   1   1   0   0   0   0   1   1           0   0   0   1   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As I said, I have an R and Bash solution, but is there a way to do this estimation in Perl or Python?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the Bash solution, but I do not know how to translate it into Perl or Python? I am learning Perl...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{for(i = 1; i &amp;lt;= NF; i++)&#xA;    tally[i] += $i};&#xA;    (NR%2) == 1&#xA;    {for(i=1; i&amp;lt;=NF; i++)&#xA;        allele1[i] = $i};&#xA;        (NR%2) == 0&#xA;        {for(i = 1; i &amp;lt;= NF; i++)&#xA;        allele2[i] = $i;&#xA;            for(i = 1; i &amp;lt;= NF; i++)&#xA;                if(tally[i] == 0)&#xA;                    tally[i] = 1;&#xA;                for(i = 1; i &amp;lt;= NF; i++)&#xA;                    printf allele1[i]/tally[i]&quot;\t&quot;;&#xA;                    printf &quot;\n&quot;;&#xA;                for(i = 1; i &amp;lt;= NF; i++)&#xA;                    printf allele2[i]/tally[i]&quot;\t&quot;;&#xA;                    printf &quot;\n&quot;;&#xA;                for(i = 1; i &amp;lt;= NF; i++)&#xA;                    tally[i]=0}' MyData |&#xA;sed 's/\t$//g'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1749" LastEditorUserId="1775" LastEditDate="2017-12-15T08:37:59.343" LastActivityDate="2017-12-15T09:10:32.840" Title="Convert population allele count into population allele frequency by using Perl or Python (translate available Bash into Perl or Python)" Tags="|python|perl|" AnswerCount="4" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2790" PostTypeId="1" CreationDate="2017-11-07T06:14:04.013" Score="5" ViewCount="641" Body="&lt;p&gt;I am experiencing an error while using getSequence() in biomaRt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code is given below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(&quot;biomaRt&quot;)&#xA;ensemblp &amp;lt;- useMart(biomart=&quot;plants_mart&quot;,host=&quot;plants.ensembl.org&quot;)&#xA;osj = useDataset(&quot;osativa_eg_gene&quot;, mart = ensemblp)&#xA;genes &amp;lt;- read.csv(&quot;osj.csv&quot;)&#xA;osj_id &amp;lt;- genes$ensembl_transcript_id&#xA;&#xA;os_genes &amp;lt;- getSequence(id = osj_id, type = &quot;ensembl_transcript_id&quot;, seqType = &quot;peptide&quot;, mart = ensemblp)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The error message I am getting is :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in martCheck(mart, c(&quot;ensembl&quot;, &quot;ENSEMBL_MART_ENSEMBL&quot;)) :   &#xA;This function only works when used with the ensembl BioMart.This&#xA;function only works when used with the ENSEMBL_MART_ENSEMBL BioMart.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Did I make any mistake in code? Please help me fix the error. I tried asking this in &lt;a href=&quot;https://support.bioconductor.org/p/102570/&quot; rel=&quot;noreferrer&quot;&gt;Bioconductor support&lt;/a&gt;, but no one responded yet.&lt;/p&gt;&#xA;" OwnerUserId="1751" LastEditorUserId="298" LastEditDate="2017-11-07T10:36:50.123" LastActivityDate="2017-11-08T18:17:44.650" Title="BiomaRt error: Error in martCheck" Tags="|r|biomart|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2791" PostTypeId="2" ParentId="2788" CreationDate="2017-11-07T08:17:52.723" Score="-1" Body="&lt;p&gt;Here's my quick Perl attempt after misreading the question, which divides each row elements by the row total:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat popCountData.txt | perl -lane '&#xA;  my $total=0;&#xA;  grep {$total += $_} @F; ## create per-row total&#xA;  foreach (@F) {printf(&quot;%0.2f &quot;, ($total) ? $_ / $total : 0)} @F; ## divide elements by total&#xA;  print &quot;&quot;; ## print a line break (note: line will end with a space)&#xA;'&#xA;0.20 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.20 0.40 0.20 0.00 0.00 0.00 0.00 &#xA;0.00 0.25 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.50 0.00 0.25 0.00 0.00 0.00 &#xA;0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 &#xA;0.13 0.13 0.00 0.00 0.13 0.07 0.00 0.00 0.00 0.00 0.13 0.27 0.00 0.00 0.00 0.13 0.00&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The challenge here is that the processing is in chunks of two lines, which means a couple of things:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The previous line (at least for even-numbered lines) needs to be stored&lt;/li&gt;&#xA;&lt;li&gt;The sum of counts (or whatever statistic is necessary) will have different processing on each line&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, I can't quite see how to get from your example input to your example output. For example, how does a count of 2 and 4 turn into a frequency of $1/3$ and $2/3$?&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-11-07T08:17:52.723" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2792" PostTypeId="2" ParentId="2790" CreationDate="2017-11-07T09:05:16.167" Score="9" Body="&lt;p&gt;getSequence has only been enabled for the main Ensembl (vertebrates) biomaRt. This is all at the Bioconductor end. You'll need to use a getBM instead, for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;os_genes &amp;lt;- getBM(attributes =c(&quot;ensembl_transcript_id&quot;, &quot;peptide&quot;), filters = &quot;ensembl_transcript_id&quot;, values = osj_id, mart = osj)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1332" LastEditorUserId="48" LastEditDate="2017-11-07T22:49:21.010" LastActivityDate="2017-11-07T22:49:21.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2793" PostTypeId="1" CreationDate="2017-11-07T11:13:48.060" Score="2" ViewCount="440" Body="&lt;p&gt;I am using DESeq2 to perform a differential expression analysis, but I obtained very strange results for some genes. For some genes, I have very high log2FoldChange with very low p-value as displayed in the following table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/V35Fp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/V35Fp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I look the normalized expression values for these genes.  I can see that these genes are expressed in only one single sample (condition1= samples from 1 to 7, condition2 = samples 8 to 13).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ooejw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ooejw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you please tell me how to avoid DESeq2 to detect these genes as differentially expressed ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using to following commands:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dds                      &amp;lt;- DESeqDataSetFromHTSeqCount(sampleTable = sampleTable, directory = directory, design= ~ condition)&#xA;dds                      &amp;lt;- DESeq(dds)&#xA;res                      &amp;lt;- results(dds, contrast=c(&quot;condition&quot;,condition1,condition2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1730" LastActivityDate="2017-11-17T09:44:42.133" Title="Why do I obtain very strange results with DESeq2?" Tags="|differential-expression|statistics|deseq2|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2794" PostTypeId="2" ParentId="2788" CreationDate="2017-11-07T16:01:29.857" Score="4" Body="&lt;p&gt;I don't know how fast you need it to be, but this R solution runs on 20 million SNPs in under 2 min on my laptop:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;allele_counts &amp;lt;-matrix(rep(c(1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 2, 1, 0, 0, 0, 0,&#xA;                     0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 0, 2, 0, 0, 0,&#xA;                     0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,&#xA;                     2, 2, 0, 0, 2, 1, 0, 0, 0, 0, 2, 4, 0, 0, 0, 2, 0), 10000000), &#xA;                   ncol=17, byrow=TRUE)&#xA;allele1&amp;lt;-allele_counts[ c(TRUE,FALSE), ]&#xA;allele2&amp;lt;-allele_counts[ c(FALSE,TRUE), ]&#xA;total_counts &amp;lt;- allele1+allele2&#xA;allele_counts[c(TRUE,FALSE), ] &amp;lt;-allele1/total_counts&#xA;allele_counts[c(FALSE,TRUE), ] &amp;lt;-allele2/total_counts&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I decided to a head-to-head test of this solution vs. &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2795/581&quot;&gt;bli's pandas solution&lt;/a&gt; and your awk solution&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used readr for file IO:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(readr)&#xA;allele_counts&amp;lt;-as.matrix(read_tsv(&quot;allele_counts.txt&quot;, col_names=F))&#xA;...&#xA;write_tsv(as.data.frame(allele_counts), &quot;allele_freq_R.txt&quot;, col_names=F)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here are the timings:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat Benchmarks/R.txt &#xA;s       h:m:s   max_rss max_vms max_uss max_pss io_in   io_out  mean_load&#xA;103.7556        0:01:43 2211.16 2518.59 2209.11 2209.15 0.00    27.58   73.62&#xA;&#xA;cat Benchmarks/Python.txt &#xA;s       h:m:s   max_rss max_vms max_uss max_pss io_in   io_out  mean_load&#xA;132.6425        0:02:12 4872.45 6042.28 4870.89 4870.92 2.67    46.59   99.92&#xA;cat Benchmarks/awk.txt &#xA;s       h:m:s   max_rss max_vms max_uss max_pss io_in   io_out  mean_load&#xA;126.4455        0:02:06 4.22    413.55  0.93    0.97    0.00    130.68  91.28&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The R solution is slightly faster than the pandas solution and uses half as much memory. Your awk one-liner is about the same speed as the pandas script, but requires a tiny fraction of the memory footprint.&lt;/p&gt;&#xA;" OwnerUserId="581" LastEditorUserId="581" LastEditDate="2017-11-08T16:21:06.937" LastActivityDate="2017-11-08T16:21:06.937" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2795" PostTypeId="2" ParentId="2788" CreationDate="2017-11-07T18:36:03.367" Score="0" Body="&lt;p&gt;In python, if your files can fit in RAM, you might try using pandas and cytoolz.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;import pandas as pd&#xA;from cytoolz import interleave&#xA;&#xA;def odd(num):&#xA;    return num % 2&#xA;&#xA;def even(num):&#xA;    return 1 - (num % 2)&#xA;&#xA;# Read in two separate dataframes&#xA;alleles_1 = pd.read_table(&quot;snps.txt&quot;, skiprows=odd, sep=&quot;\s+&quot;, header=None)&#xA;alleles_2 = pd.read_table(&quot;snps.txt&quot;, skiprows=even, sep=&quot;\s+&quot;, header=None)&#xA;sums = alleles_1 + alleles_2&#xA;# https://stackoverflow.com/a/45565745/1878788&#xA;pd.DataFrame(list(interleave([&#xA;    alleles_1.div(sums).fillna(0).values,&#xA;    alleles_2.div(sums).fillna(0).values]))).to_csv(&#xA;        &quot;freqs.txt&quot;, sep=&quot;\t&quot;, index=False, header=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't know how efficient this is. On my workstation it runs in 1m27 on data obtained by taking 1 million times the example given in the question. See &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2794/292&quot;&gt;heathobrien's answer&lt;/a&gt; for benchmarks on larger data.&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2017-11-09T09:54:43.703" LastActivityDate="2017-11-09T09:54:43.703" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2796" PostTypeId="2" ParentId="2744" CreationDate="2017-11-07T20:59:52.357" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;All the popular variant calling tools such as Strelka, VarScan etc&#xA;  require a normal sample.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Strelka and VarScan require a normal sample in somatic mode, but they both have a germline mode for unpaired analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the most popular somatic variant callers is MuTect and that does not require a normal.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2017-11-07T20:59:52.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2798" PostTypeId="1" CreationDate="2017-11-08T09:12:44.833" Score="1" ViewCount="174" Body="&lt;p&gt;I am a bioinformatics student and I was learning the basics of &lt;a href=&quot;https://en.wikipedia.org/wiki/Structural_equation_modeling&quot; rel=&quot;nofollow noreferrer&quot;&gt;structural equation modeling (SEM)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering, are there any applications of SEM on  bioinformatics? Are there any strong sides for use of structural equation models using data on bioinformatics?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did a quick search and found almost nothing. While it seemed like a learning algorithm, I can't say I figured it out completely.&lt;/p&gt;&#xA;" OwnerUserId="1757" LastEditorUserId="7374" LastEditDate="2020-03-12T20:51:42.840" LastActivityDate="2020-03-12T20:51:42.840" Title="Can structural - equation modeling be used in Bioinformatics?" Tags="|statistics|algorithms|structural-variation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="2799" PostTypeId="2" ParentId="2798" CreationDate="2017-11-08T11:01:28.997" Score="1" Body="&lt;p&gt;I have came across them in gene regulatory network analysis before. see &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2278111/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1003068&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. However I haven't seen them in other contexts, but that's not to say they don't exist.&lt;/p&gt;&#xA;" OwnerUserId="252" LastEditorUserId="252" LastEditDate="2017-11-08T11:10:56.987" LastActivityDate="2017-11-08T11:10:56.987" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2800" PostTypeId="1" AcceptedAnswerId="2801" CreationDate="2017-11-08T14:59:01.430" Score="2" ViewCount="2610" Body="&lt;p&gt;I'd like to threshold a bigWig file on a value and get all regions above or below that value in bed format. Ideally without too many file conversions. Are there any tools to do that?&lt;/p&gt;&#xA;" OwnerUserId="143" LastActivityDate="2023-09-13T06:41:09.323" Title="bigWig to bed for regions above/below threshold" Tags="|bed|bigwig|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2801" PostTypeId="2" ParentId="2800" CreationDate="2017-11-08T15:03:34.553" Score="1" Body="&lt;p&gt;You can do that with a bit of python:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python&#xA;import pyBigWig&#xA;threshold = 10  # Change me&#xA;bw = pyBigWig.open(&quot;file.bw&quot;)  # Change me&#xA;of = open(&quot;regions.bed&quot;, &quot;w&quot;)  # Change me&#xA;&#xA;for chrom, len in bw.chroms().items():&#xA;    intervals = bw.intervals(chrom)&#xA;    for interval in interals:&#xA;        if abs(interval[2]) &amp;gt; threshold:&#xA;            of.write(&quot;{}\t{}\t{}\n&quot;.format(chrom, interval[0], interval[1]))&#xA;bw.close()&#xA;of.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You'll need to install pyBigWig (it's available via &lt;code&gt;pip&lt;/code&gt; or &lt;code&gt;conda install&lt;/code&gt;) and change the lines with &lt;code&gt;Change me&lt;/code&gt; in them.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-11-08T15:03:34.553" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2802" PostTypeId="1" CreationDate="2017-11-08T16:59:56.727" Score="3" ViewCount="251" Body="&lt;p&gt;I have microarray z scores for gene set enriched (GSE) pathways that range approximately from -3 to 14. I understand that an absolute value of 1.96 has a p-value = .05 and significant for our purposes. My question is whether I can consider extreme z scores as more significant?&lt;/p&gt;&#xA;" OwnerUserId="1430" LastActivityDate="2017-11-09T08:08:56.320" Title="Interpreting large z scores from microarray data" Tags="|microarray|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2804" PostTypeId="2" ParentId="2790" CreationDate="2017-11-08T18:17:44.650" Score="1" Body="&lt;p&gt;Biomart seems to be down most of the time so i think you can parse the ensemble ID and names from the annotation file which i did as i was having similar problem .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use this to make a file with ensemble ID and gene names for you use &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{                                &#xA;    for (i = 1; i &amp;lt;= NF; i++) {&#xA;        if ($i ~ /gene_id|gene_name/) {&#xA;            printf &quot;%s &quot;, $(i+1)&#xA;        }&#xA;    }&#xA;    print &quot;&quot;&#xA;}' Homo_sapiens.GRCh37.70.gtf | sed -e 's/&quot;//g' -e 's/;//g' -e 's/ /\t/' | sort -k1,1 | uniq &amp;gt; Homo_sapiens.GRCh37.70.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="285" LastActivityDate="2017-11-08T18:17:44.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2806" PostTypeId="2" ParentId="2800" CreationDate="2017-11-08T18:22:51.137" Score="1" Body="&lt;p&gt;Convert bigWig to WIG via &lt;a href=&quot;https://www.encodeproject.org/software/bigwigtowig/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;bigWigToWig&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ bigWigToWig signal.bw signal.wig&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Convert WIG to BED via BEDOPS &lt;a href=&quot;http://bedops.readthedocs.io/en/latest/content/reference/file-management/conversion/wig2bed.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;wig2bed&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ wig2bed &amp;lt; signal.wig &amp;gt; signal.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Filter BED by its score column with &lt;code&gt;awk&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk '($&lt;/span&gt;5 &amp;lt; 1)' signal.bed &amp;gt; signal.filtered.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The test &lt;code&gt;$5 &amp;lt; 1&lt;/code&gt; filters elements from &lt;code&gt;signal.bed&lt;/code&gt; that have a score value less than &lt;code&gt;1&lt;/code&gt;. Change this to the threshold test you would want to apply.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2023-09-13T06:41:09.323" LastActivityDate="2023-09-13T06:41:09.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2807" PostTypeId="1" AcceptedAnswerId="2809" CreationDate="2017-11-08T19:48:24.113" Score="3" ViewCount="204" Body="&lt;p&gt;I'm reading &lt;a href=&quot;https://www.biorxiv.org/content/early/2017/10/24/196311&quot; rel=&quot;nofollow noreferrer&quot;&gt;a preprint on a new somatic variant discovery tool&lt;/a&gt;, and the first sentence of the Discussion section has me confused.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;Across the four datasets analyzed in this study, we discovered that the major source of disagreement between callers originates from somatic variants called within STRs, in particular if the motif is two base pairs or longer.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The &quot;&lt;em&gt;two base pairs or longer&lt;/em&gt;&quot; bit is what confuses me. I interpreted &quot;motif&quot; as referring to the short tandem repeats (STRs), which are by definition 2 bp or longer. A motif of &amp;lt; 2 bp is...a homopolymer, not really a motif in the common sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missing something here? Or is this simply an awkward word choice on the part of the authors?&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2017-11-09T08:12:52.643" Title="What does &quot;motif 2 bp or longer&quot; mean?" Tags="|variant-calling|genomics|repeat-elements|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2808" PostTypeId="1" AcceptedAnswerId="3035" CreationDate="2017-11-08T21:37:18.523" Score="5" ViewCount="192" Body="&lt;p&gt;I am very new to bioinformatics and trying to repeat the benchmark in the RapMap paper with an experimental tool working in a similar but different fashion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the &lt;a href=&quot;https://academic.oup.com/bioinformatics/article-lookup/doi/10.1093/bioinformatics/btw277&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; (taken from their &lt;a href=&quot;https://github.com/COMBINE-lab/RapMap&quot; rel=&quot;nofollow noreferrer&quot;&gt;github&lt;/a&gt;) about RapMap, an alignment/mapping tool for rna-seq reads to a reference genome, the dataset used for their benchmarks is described like this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To test the accuracy of different mapping and alignment tools in a&#xA;  scenario where we know the true origin of each read, we generated data&#xA;  using the Flux Simulator (Griebel et al., 2012). This synthetic&#xA;  dataset was generated for the human transcriptome from an annotation&#xA;  taken from the ENSEMBL (Cunningham et al., 2015) database consisting&#xA;  of 86090 transcripts corresponding to protein-coding genes. The&#xA;  dataset consists of  48 million 76 bp, paired-end reads. [...]&#xA;  When benchmarking these methods, reads were aligned directly to the transcriptome, rather than to the genome.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I understand that they generated reads using flux simulator, but I don't understand which transcriptome they mapped them against. I can't find any other information in the paper about which annotation they used either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am assuming one would need to generate an entire transcriptome in .fa format from the GTF and genome? Or did they generate a transcriptome from only the simulated reads and then mapped the same reads against it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GENCODE offers GTFs and FASTAs for gene-encoding transcripts &lt;a href=&quot;https://www.gencodegenes.org/releases/current.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, might that be the kind of thing they used? Would the reads simulated from these GTFs fit to these transcript FASTA files?&lt;/p&gt;&#xA;" OwnerUserId="1760" LastEditorUserId="48" LastEditDate="2017-11-09T08:13:13.510" LastActivityDate="2017-12-11T07:57:24.453" Title="RapMap: reference transcriptome for simulated reads" Tags="|sequence-alignment|read-mapping|transcriptome|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2809" PostTypeId="2" ParentId="2807" CreationDate="2017-11-08T22:41:11.450" Score="4" Body="&lt;p&gt;What's your reference for the definition of STRs? I think it is still ambiguous among the community. Wikipedia states motifs of &gt;=2 bp. However other references include homopolymers as well:&lt;br&gt;&#xA;&lt;a href=&quot;http://www.nature.com/nmeth/journal/v14/n6/full/nmeth.4267.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Source 1&lt;/a&gt;, &lt;a href=&quot;http://www.cell.com/ajhg/fulltext/S0002-9297(17)30382-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;source 2&lt;/a&gt; and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5054066/&quot; rel=&quot;nofollow noreferrer&quot;&gt;source 3&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1767" LastEditorUserId="48" LastEditDate="2017-11-09T08:12:52.643" LastActivityDate="2017-11-09T08:12:52.643" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2810" PostTypeId="2" ParentId="2808" CreationDate="2017-11-08T23:53:07.903" Score="1" Body="&lt;p&gt;If they generated reads from the transcriptome then the started with the FASTA version of the ENSEMBL transcriptome and then aligned the reads back to it.  For a project like this I don't think that you need the GTF files, since you don't need to know the location of the transcript on the larger genomic reference.  If you can't find an ENSEMBL transcriptome then you can use the GTF and the bedtools program to create a FASTA.   &lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2017-11-08T23:53:07.903" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2811" PostTypeId="2" ParentId="2802" CreationDate="2017-11-09T08:08:56.320" Score="2" Body="&lt;p&gt;This is a statistical question. What a higher z value means is that it is more extreme (if you assume a normal distribution), thus a more extreme value is less likely to happen by chance. Which is translated to a (lower) p-value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What it is significant and not is arbitrary. Prior to the experiment you must decide which is your threshold, how much chance you allow. Apparently you decided to select 0.05 as your threshold, which means that one of 20 results might be due to chance. But your extreme z score is not &lt;strong&gt;more&lt;/strong&gt; significant, it is &lt;strong&gt;less expected&lt;/strong&gt; (under the null hypothesis, those p-values might change if you change your null hypothesis, ie you think your genes expression don't follow a normal distribution) but &lt;strong&gt;as significant as&lt;/strong&gt; your 0.0499999 p-values. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2017-11-09T08:08:56.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2812" PostTypeId="1" CreationDate="2017-11-10T13:09:35.027" Score="2" ViewCount="1352" Body="&lt;p&gt;What's the most straightforward way to filter a BAM file for read pairs where one or both of the reads starts with a given sequence pattern? Would a tool that deals with UMIs work for this? Which one would be best?&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="1775" LastEditDate="2017-11-10T16:53:29.950" LastActivityDate="2017-11-15T12:47:09.740" Title="Filter BAM file for read pairs where one or both of the reads starts with a given sequence pattern" Tags="|sam|sequence-analysis|filtering|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2813" PostTypeId="2" ParentId="2090" CreationDate="2017-11-10T13:40:24.453" Score="1" Body="&lt;p&gt;It's just a guess, but I think you might want to use &lt;code&gt;--keep-target-files&lt;/code&gt; flag to not to adjust paths of targets. From &lt;a href=&quot;http://snakemake.readthedocs.io/en/stable/executable.html?highlight=keep-target-files&quot; rel=&quot;nofollow noreferrer&quot;&gt;snakemake manual&lt;/a&gt; :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;–keep-target-files&lt;/code&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Do not adjust the paths of given target files relative to the working&#xA;  directory.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Default: False&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So if I understand right &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2090/combining-the-use-of-workdir-and-option-jobscript-in-snakemake/2094#2094&quot;&gt;Devon's answer&lt;/a&gt;, combination of this flag and setting of &lt;code&gt;workdir&lt;/code&gt; could force compute target in different directory that the snakemake is executed in.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2017-11-10T13:40:24.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2814" PostTypeId="2" ParentId="2812" CreationDate="2017-11-10T13:54:58.450" Score="3" Body="&lt;p&gt;&lt;strong&gt;First option&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My solution uses the following steps:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;use picard sortsam to sort the records on query-name (not samtools sort because the order &lt;a href=&quot;https://github.com/samtools/hts-specs/issues/5&quot; rel=&quot;nofollow noreferrer&quot;&gt;is not the same&lt;/a&gt; between java and C ) &lt;/li&gt;&#xA;&lt;li&gt;use &lt;a href=&quot;https://docs.oracle.com/javase/8/docs/technotes/tools/unix/jjs.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;jjs&lt;/a&gt; (java scripting engine) and the &lt;a href=&quot;https://github.com/samtools/htsjdk&quot; rel=&quot;nofollow noreferrer&quot;&gt;htsjdk library&lt;/a&gt; to build a &lt;code&gt;buffer&lt;/code&gt;of reads having the same name. If any read starts with a pattern, print the whole &lt;code&gt;buffer&lt;/code&gt;. The htsjdk should be already contained in the picard.jar.&lt;/li&gt;&#xA;&lt;li&gt;sort back to coordinate order using samtools.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All in one:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar /path/to/picard.jar SortSam I=in.bam O=/dev/stdout  SO=queryname |\&#xA;jjs -cp /path/to/picard.jar  script.js |\&#xA;samtools sort -T TMP -o out.bam - &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with script.js&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;var out=java.lang.System.out;&#xA;var File = Java.type(&quot;java.io.File&quot;);&#xA;var SamReaderFactory = Java.type(&quot;htsjdk.samtools.SamReaderFactory&quot;);&#xA;var SamInputResource = Java.type(&quot;htsjdk.samtools.SamInputResource&quot;);&#xA;var SAMFileWriterFactory = Java.type(&quot;htsjdk.samtools.SAMFileWriterFactory&quot;);&#xA;var samReader =  SamReaderFactory.makeDefault().open(SamInputResource.of(java.lang.System.in));&#xA;var samWriter =  new SAMFileWriterFactory().makeBAMWriter(samReader.getFileHeader(),true,java.lang.System.out);&#xA;var iter=samReader.iterator();&#xA;var buffer=[];&#xA;for(;;)&#xA;        {&#xA;        var rec=null;&#xA;        if(iter.hasNext()) rec=iter.next();&#xA;        if(rec==null || (buffer.length&amp;gt;0 &amp;amp;&amp;amp; !rec.getReadName().equals(buffer[0].getReadName())))&#xA;            {&#xA;            var i=0;&#xA;            for(i=0;i&amp;lt; buffer.length;++i)&#xA;                {&#xA;                if(buffer[i].getReadString().match(/^AAAA/)) break;&#xA;                }&#xA;            if(i!=buffer.length)&#xA;                {&#xA;                for(i=0;i&amp;lt; buffer.length ;++i)&#xA;                    {&#xA;                    samWriter.addAlignment(buffer[i]);&#xA;                    }&#xA;                }&#xA;            if(rec==null) break;&#xA;            buffer=[];&#xA;            }&#xA;        buffer.push(rec);&#xA;        }&#xA;&#xA;samReader.close();&#xA;samWriter.close();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;A second option&lt;/strong&gt; is to use samtools view and awk to export the read names:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; samtools view  input.bam | awk -F '\t' '{if($10 ~ /^AAAA/) print $1;}' | sort | uniq &amp;gt; names.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and to use &lt;a href=&quot;http://broadinstitute.github.io/picard/command-line-overview.html#FilterSamReads&quot; rel=&quot;nofollow noreferrer&quot;&gt;Picard FilterSamReads&lt;/a&gt; and READ_LIST_FILE to filter the reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE: 3rd solution:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've updated my tool &lt;a href=&quot;http://lindenb.github.io/jvarkit/SamJdk.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;samjdk&lt;/a&gt;: there is now a new option &lt;code&gt;--pair&lt;/code&gt; to work with query-name-sorted BAM files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The command would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar picard.jar SortSam I=S1.bam O=/dev/stdout  SO=queryname |\&#xA;java -jar dist/samjdk.jar --samoutputformat BAM --pair \&#xA;     -e 'return records.stream().anyMatch(R-&amp;gt;R.getReadString().startsWith(&quot;AAAAAAG&quot;));' |\&#xA;samtools sort -T TMP -o out.bam - &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastEditorUserId="48" LastEditDate="2017-11-10T16:32:35.243" LastActivityDate="2017-11-10T16:32:35.243" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2815" PostTypeId="2" ParentId="2812" CreationDate="2017-11-10T14:03:42.830" Score="3" Body="&lt;p&gt;For at least &lt;strong&gt;one in a pair&lt;/strong&gt; matching the sequence, if the read names of pairs are identical, some grep magic could do the job:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Print the sequence at the beginning of the line and read the name in the second column&lt;/li&gt;&#xA;&lt;li&gt;Find all lines starting with the sequence&lt;/li&gt;&#xA;&lt;li&gt;Extract only read names&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;And then just grep all found reads and save into the BAM file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SEQ=&amp;lt;my_sequence&amp;gt;&#xA;samtools view unfiltered.bam | \&#xA;   awk '{print $10 &quot;\t&quot; $1}' | \&#xA;   grep &quot;^$SEQ&quot; | \&#xA;   cut -f 2 &amp;gt; list_of_matching_names&#xA;samtools view unfiltered.bam | \&#xA;   grep -f list_of_matching_names | \&#xA;   samtools view -b &amp;gt; filtered.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Two small notes:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You might want to consider also reverse complementary sequences.&lt;/li&gt;&#xA;&lt;li&gt;You also might want to keep header of BAM file (you can extract it at the beginning and add it in the end).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="57" LastEditorUserId="1775" LastEditDate="2017-11-10T16:53:20.583" LastActivityDate="2017-11-10T16:53:20.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2816" PostTypeId="1" CreationDate="2017-11-10T15:17:20.347" Score="1" ViewCount="340" Body="&lt;p&gt;I have 2 lists, &lt;code&gt;list1&lt;/code&gt; and &lt;code&gt;list2&lt;/code&gt;, of protein sequences of the same given gene in different strains. In &lt;code&gt;list1&lt;/code&gt; are reported amino-acid sequences from resistant strains, and in &lt;code&gt;list2&lt;/code&gt; are reported amino-acid sequences from susceptible strains. Each item of each list is an amino-acid sequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to filter out (in Python 3) from &lt;code&gt;list1&lt;/code&gt; all of the variants that are also present in &lt;code&gt;list2&lt;/code&gt;. The final aim is to have a list of the sequences that have unique mutations that are not in the 2nd list. The mutations can be of any type (e.g., indels, mismatches).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the sequences are of at least 90% identity to some reference sequence  (filtered out the low matches using seq alignment).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any enlightenment / package / tool for this job will be appreciated!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Toy example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;a&lt;/code&gt;, &lt;code&gt;b&lt;/code&gt;, &lt;code&gt;c&lt;/code&gt; : amino-acids&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this input: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Reference seq: &lt;code&gt;aaabbb&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Resistant strains: &lt;code&gt;list1&lt;/code&gt; = [aa&lt;strong&gt;c&lt;/strong&gt;bb&lt;strong&gt;c&lt;/strong&gt;, a&lt;strong&gt;b&lt;/strong&gt;abbb]&lt;/li&gt;&#xA;&lt;li&gt;Susceptible strains: &lt;code&gt;list2&lt;/code&gt; = [aa&lt;strong&gt;c&lt;/strong&gt;bbb, aaabb&lt;strong&gt;c&lt;/strong&gt;]&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The expected output would be: [a&lt;strong&gt;b&lt;/strong&gt;abbb], since the mutations in &lt;code&gt;list2&lt;/code&gt; cancels out the item aa&lt;strong&gt;c&lt;/strong&gt;bb&lt;strong&gt;c&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course this is a simple example and it is possible to have more complex scenarios (e.g., longer indels and mismatches).&lt;/p&gt;&#xA;" OwnerUserId="1695" LastEditorUserId="1695" LastEditDate="2017-11-14T14:24:14.693" LastActivityDate="2018-03-14T17:38:15.747" Title="Filtering out all seqs with mutations of list2 from list1" Tags="|python|proteins|sequence-alignment|filtering|" AnswerCount="1" CommentCount="17" ContentLicense="CC BY-SA 3.0" />
  <row Id="2817" PostTypeId="1" AcceptedAnswerId="2818" CreationDate="2017-11-10T15:28:11.673" Score="10" ViewCount="6993" Body="&lt;p&gt;I want to create a database for a proteomics study. Therefore, the mapping from a given sequence to a protein ID has to be unique. I am wondering whether there is already a built-in function in &lt;a href=&quot;https://en.wikipedia.org/wiki/Biopython&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biopython&lt;/a&gt; for that, but I could not find any. The duplicated sequences should be merged into a single entry and there respective IDs should be just concatenated (see expected outcome below).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the following example input&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;prot_id_1&#xA;MAWIGLDISKLFVENSRDAAA&#xA;&amp;gt;prot_id_2&#xA;MENVSRIWRYQQRRVLSRAFTHWYLMGLTKHNHPS&#xA;&amp;gt;prot_id_3&#xA;MRTRPSRDFHYIVGRITRESPEEKGIFYVFH&#xA;&amp;gt;prot_id_4&#xA;MEMVLSSANPSTTFADSYVV&#xA;&amp;gt;prot_id_5&#xA;MAWIGLDISKLFVENSRDAAA&#xA;&amp;gt;prot_id_6&#xA;MENVSRIWRYQQRRVLSRAFTHWYLMGLTKHNHPS&#xA;&amp;gt;prot_id_7&#xA;MAWIGLDISKLFVENSRDAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I currently do it as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;import numpy as np&#xA;from Bio.Seq import Seq&#xA;from Bio.SeqRecord import SeqRecord&#xA;from Bio.Alphabet import IUPAC&#xA;&#xA;# Read entries into dictionary, gi: sequence&#xA;record_dict = SeqIO.to_dict(SeqIO.parse(&quot;test.fasta&quot;, &quot;fasta&quot;))&#xA;&#xA;# Extract sequence as string&#xA;record_dict = {gi: str(record_dict[gi].seq) for gi in record_dict}&#xA;&#xA;# Check for duplicated sequences&#xA;seq_id_dict = {}&#xA;for k, v in record_dict.items():&#xA;    seq_id_dict.setdefault(v, []).append(k)&#xA;&#xA;# Filter sequences that appear more than once&#xA;filter_dict = {seqi: li for seqi, li in seq_id_dict.items() if len(li) &amp;gt; 1}&#xA;&#xA;# Merge identifiers to a single one&#xA;merged_ids = {&quot;_&quot;.join(sorted(li)): seqi for seqi, li in filter_dict.items()}&#xA;&#xA;# Remove all entries that are associated with duplicate IDs&#xA;for gi in set(np.concatenate(filter_dict.values())):&#xA;    record_dict.pop(gi, None)&#xA;&#xA;# Add the merged IDs again&#xA;record_dict.update(merged_ids)&#xA;&#xA;# Create list of Seq objects&#xA;final_seq = [SeqRecord(Seq(seqi, IUPAC.protein), id=gi, name='', description='') for gi, seqi in record_dict.items()]&#xA;&#xA;# Write file&#xA;SeqIO.write(final_seq, 'test_output.fasta', 'fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which gives me the desired output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;prot_id_4&#xA;MEMVLSSANPSTTFADSYVV&#xA;&amp;gt;prot_id_1_prot_id_5_prot_id_7&#xA;MAWIGLDISKLFVENSRDAAA&#xA;&amp;gt;prot_id_3&#xA;MRTRPSRDFHYIVGRITRESPEEKGIFYVFH&#xA;&amp;gt;prot_id_2_prot_id_6&#xA;MENVSRIWRYQQRRVLSRAFTHWYLMGLTKHNHPS&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a smarter way of accomplishing this?&lt;/p&gt;&#xA;" OwnerUserId="351" LastEditorUserId="1775" LastEditDate="2017-12-02T12:52:57.603" LastActivityDate="2020-01-19T23:17:45.590" Title="How do I find identical sequences in a FASTA file?" Tags="|fasta|python|filtering|" AnswerCount="6" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2818" PostTypeId="2" ParentId="2817" CreationDate="2017-11-10T15:45:20.237" Score="8" Body="&lt;p&gt;The trick would be to swap the key in the dictionary to be the sequence itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also I would recommend using a different separator that &quot;_&quot; since that is what the current ids have so that you can easily separate the individual ids from the concatenated id. I used a pipe &quot;|&quot; in this example. Also I just manually wrote the FASTA output instead of using Biopython to do it.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;from collections import defaultdict&#xA;&#xA;dedup_records = defaultdict(list)&#xA;for record in SeqIO.parse(&quot;test.fasta&quot;, &quot;fasta&quot;):&#xA;    # Use the sequence as the key and then have a list of id's as the value&#xA;    dedup_records[str(record.seq)].append(record.id)&#xA;with open(&quot;Output.fasta&quot;, 'w') as output:&#xA;    for seq, ids in dedup_records.items():&#xA;        # Join the ids and write them out as the fasta&#xA;        output.write(&quot;&amp;gt;{}\n&quot;.format('|'.join(ids)))&#xA;        output.write(seq + &quot;\n&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="964" LastEditorUserId="1775" LastEditDate="2017-12-02T12:53:15.567" LastActivityDate="2017-12-02T12:53:15.567" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="2819" PostTypeId="1" AcceptedAnswerId="2822" CreationDate="2017-11-10T16:55:50.093" Score="3" ViewCount="63" Body="&lt;p&gt;I am using Texshade to produce figures for MSA. Upon reading the documentation I could not find anyway to highlight how conserved CpG sites within the alignment. Does anyone know if this is possible? If not is there another MSA visualizer I can use that can produce a figure like this. Thanks! &lt;/p&gt;&#xA;" OwnerUserId="712" LastActivityDate="2017-11-10T22:27:08.123" Title="Highlight CpG sites with Texshade" Tags="|sequence-alignment|methylation|visualization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2820" PostTypeId="1" AcceptedAnswerId="2821" CreationDate="2017-11-10T19:37:20.257" Score="5" ViewCount="1584" Body="&lt;p&gt;I am trying to understand local realignment but I could not get a clear idea of what is the problem solved by it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, reading Homer and Nelson (2010):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Because alignment algorithms map each read independently to the reference genome, alignment artifacts could result, such that SNPs, insertions, and deletions are improperly placed relative to their true location. This leads to local alignment errors due to a combination of sequencing error, equivalent positions of the variant being equally likely, and adjacent variants or nearby errors driving misalignment of the local sequence. These local misalignments lead to false positive variant detection, especially at apparent heterozygous positions. For example, insertions and deletions towards the ends of reads are difficult to anchor and resolve without the use of multiple reads. In some cases, strict quality and filtering thresholds are used to overcome the false detection of variants, at the cost of reducing power [13]. Since each read represents an independent observation of only one of two possible haplotypes (assuming a diploid genome), multiple read observations could significantly reduce false-positive detection of variants.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't understand how the different artifacts (such as SNPs) can result from the alignment.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Homer, Nils, and Stanley F. Nelson. &quot;&lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/gb-2010-11-10-r99&quot; rel=&quot;noreferrer&quot;&gt;Improved variant discovery through local re-alignment of short-read next-generation sequencing data using SRMA.&lt;/a&gt;&quot; Genome biology 11.10 (2010): R99.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2017-11-10T19:50:55.617" Title="What is local realignment and what is the problem it solves?" Tags="|sequence-alignment|read-mapping|reads|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2821" PostTypeId="2" ParentId="2820" CreationDate="2017-11-10T19:50:55.617" Score="7" Body="&lt;p&gt;SNPs are likely to be created and InDels are likely to be missed. Suppose you have a read, &lt;code&gt;ACTGACTGACTGTAC&lt;/code&gt; and you align it to a reference sequence &lt;code&gt;ACTGACTGACTGTTAAGAACGACTACGAC&lt;/code&gt;. If you aligned that, you would either get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ACTGACTGACTGTac    (lower case denotes soft-clipping)&#xA;ACTGACTGACTGTTAAGAACGACTACGAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ACTGACTGACTGTAC    (N.B., you've created some possible SNPS)&#xA;ACTGACTGACTGTTAAGAACGACTACGAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But what happens if you look at a bunch of reads? See below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ACTGACTGACTGTAC&#xA;  TGACTGACTGTACTA&#xA;    ACTGACTGTACTAAG&#xA;ACTGACTGACTGT--TAAGAACGACTACGAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've made the 2 base insertion obvious. As you can see, using many alignments, we can &quot;realign&quot; our original read to see that it actually ends in a 2 base insertion, something that's completely impossible to find with a single read. This realignment is part of what the GATK haplotype caller does (it makes a local de Bruijn graph). That's also what my &lt;a href=&quot;https://github.com/dpryan79/MethIndelRealigner&quot; rel=&quot;noreferrer&quot;&gt;MethIndelRealigner&lt;/a&gt; does. I have another real-world example in its documentation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Raw alignments in IGV:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://raw.githubusercontent.com/dpryan79/MethIndelRealigner/master/images/InDelExample.annotated.png&quot; alt=&quot;Before realignment&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alignments after realignment:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://raw.githubusercontent.com/dpryan79/MethIndelRealigner/master/images/RealignedExample.png&quot; alt=&quot;After realignment&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can find plenty of similar scenarios that result in SNPs rather than simply losing coverage for InDels.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-11-10T19:50:55.617" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2822" PostTypeId="2" ParentId="2819" CreationDate="2017-11-10T21:34:53.757" Score="4" Body="&lt;p&gt;The best solution I could find was to use the &lt;code&gt;\shaderegion&lt;/code&gt; function on all the sequences so it ended up looking something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;\shaderegion{1}{CG}{Red}{OrangeRed}&#xA;&#xA;\shaderegion{2}{CG}{Red}{OrangeRed}&#xA;&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I cast the rest of shading similarity to grey so the CG pairs were more easily distinguishable.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;\shadingcolors{grays}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Two examples one with &lt;code&gt;\fingerprints&lt;/code&gt; and one regular.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/XDZ3T.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XDZ3T.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Hr7OY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Hr7OY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="712" LastEditorUserId="712" LastEditDate="2017-11-10T22:27:08.123" LastActivityDate="2017-11-10T22:27:08.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2823" PostTypeId="1" AcceptedAnswerId="2850" CreationDate="2017-11-11T21:58:49.980" Score="5" ViewCount="43" Body="&lt;p&gt;I have a list of gene/protein IDs and I could not query a network using String, BioGrid, Intact and IIS, for Streptomyces thioluteus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bacteria I am working with is the &lt;em&gt;Streptomyces caatingaensis&lt;/em&gt;. Since it is newer than others, I thought I would find an interaction network looking for &lt;em&gt;Streptomyces thioluteus&lt;/em&gt; (which shares one pathway I am analyzing in common, others do not share). As a result, I have &lt;a href=&quot;http://www.uniprot.org/uniprot/?query=streptomyces%20thioluteus&amp;amp;sort=score&quot; rel=&quot;nofollow noreferrer&quot;&gt;this list&lt;/a&gt; of proteins, but I could not query any network from the IDs. Sometimes I could not find the bacterium in the tools and when it was found, as in &lt;a href=&quot;http://bioinfo03.ibi.unicamp.br/lnbio/IIS2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IIS&lt;/a&gt; the result was &lt;strong&gt;zero&lt;/strong&gt; interactions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there anything I could use to have some network that could be used as insight for the analysis?&lt;/p&gt;&#xA;" OwnerUserId="1650" LastEditorUserId="48" LastEditDate="2017-11-12T12:38:36.297" LastActivityDate="2017-11-15T01:18:49.953" Title="How to query a network for a bacterium, specifically Streptomyces caatingaensis or Streptomyces thioluteus?" Tags="|database|proteins|networks|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2824" PostTypeId="1" CreationDate="2017-11-12T09:36:56.180" Score="3" ViewCount="1575" Body="&lt;p&gt;I have two columns of data, x and y, how can I make a ‘thermodynamic diagram’ with matlab? The z axis is the density value of dots, like below figure.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/LL0Qw.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LL0Qw.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/E0Ov7.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/E0Ov7.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The x-axis is angle, y-axis is distance.&lt;a href=&quot;https://i.stack.imgur.com/tjBpo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tjBpo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;I want to construct a heatmap like this figure:&lt;a href=&quot;https://i.stack.imgur.com/Vh8Iz.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Vh8Iz.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="762" LastEditorUserId="73" LastEditDate="2017-11-13T03:47:31.837" LastActivityDate="2017-11-13T12:55:24.213" Title="Matlab scatter diagram" Tags="|software-recommendation|statistics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2825" PostTypeId="1" AcceptedAnswerId="2833" CreationDate="2017-11-12T10:27:37.637" Score="11" ViewCount="6853" Body="&lt;p&gt;I am interested in converting a VCF file into a FASTA file given a reference sequence with Python or R. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Samtools/BCFtools (Heng Li) provides a Perl script &lt;a href=&quot;https://github.com/lh3/samtools/blob/master/bcftools/vcfutils.pl&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;vcfutils.pl&lt;/code&gt;&lt;/a&gt; which does this, the function &lt;code&gt;vcf2fq&lt;/code&gt; (lines 469-528)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This script has been modified by others to convert InDels as well, e.g. &lt;a href=&quot;https://github.com/gringer/bioinfscripts/blob/master/vcf2fq.pl&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; by David Eccles &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./vcf2fq.pl -f &amp;lt;input.fasta&amp;gt; &amp;lt;all-site.vcf&amp;gt; &amp;gt; &amp;lt;output.fastq&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Given that converting a VCF into a FASTA given some reference is a somewhat standard procedure in bioinformatics, are there standard R or Python implementations used by the community?&lt;/p&gt;&#xA;" OwnerUserId="146" LastEditorUserId="48" LastEditDate="2017-11-12T12:37:05.577" LastActivityDate="2021-07-03T15:12:45.313" Title="Converting a VCF into a FASTA given a reference with Python, R" Tags="|r|python|fasta|vcf|samtools|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2828" PostTypeId="2" ParentId="2817" CreationDate="2017-11-13T08:14:38.017" Score="4" Body="&lt;p&gt;Just as a small variation to &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2818/351&quot;&gt;@Bioathlete's answer&lt;/a&gt; in case you want to write the fasta using Biopython (e.g. to add names and description):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;from Bio.Seq import Seq&#xA;from Bio.SeqRecord import SeqRecord&#xA;from Bio.Alphabet import IUPAC&#xA;from collections import defaultdict&#xA;&#xA;dedup_records = defaultdict(list)&#xA;for record in SeqIO.parse(&quot;test.fasta&quot;, &quot;fasta&quot;):&#xA;    # Use the sequence as the key and then have a list of id's as the value&#xA;    dedup_records[str(record.seq)].append(record.id)&#xA;&#xA;# this creates a generator; if you need a physical list, replace the outer &quot;(&quot;, &quot;)&quot; by &quot;[&quot; and &quot;]&quot;, respectively&#xA;final_seq = (SeqRecord(Seq(seqi, IUPAC.protein), id=&quot;|&quot;.join(gi), name='', description='') for seqi, gi in dedup_records.items())&#xA;&#xA;# write file&#xA;SeqIO.write(final_seq, 'test_output_final.fasta', 'fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will give the desired output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;prot_id_1|prot_id_5|prot_id_7&#xA;MAWIGLDISKLFVENSRDAAA&#xA;&amp;gt;prot_id_2|prot_id_6&#xA;MENVSRIWRYQQRRVLSRAFTHWYLMGLTKHNHPS&#xA;&amp;gt;prot_id_4&#xA;MEMVLSSANPSTTFADSYVV&#xA;&amp;gt;prot_id_3&#xA;MRTRPSRDFHYIVGRITRESPEEKGIFYVFH&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="351" LastEditorUserId="351" LastEditDate="2017-11-13T15:14:57.760" LastActivityDate="2017-11-13T15:14:57.760" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2829" PostTypeId="1" AcceptedAnswerId="2830" CreationDate="2017-11-13T12:10:20.557" Score="6" ViewCount="163" Body="&lt;p&gt;I have two genome assemblies of the same non-model species, call them Assembly 1 (generated from Illumina data) and Assembly 2 (generated from PacBio data).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Assembly 1, I also have predicted proteome data, generated with &lt;a href=&quot;https://evidencemodeler.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;EVM&lt;/a&gt;. Say there is a protein, call it Protein X, for which I have the fasta sequence from predictions, but suspect an error in Assembly 1 (because the Protein X has a long segment of auto-identity). For this reason, I would like to verify the assembly of the locus coding for Protein X in Assembly 2 and/or verify the presence of Protein X in the Assembly 2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Data in my possession:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;both genome assemblies (fasta)&lt;/li&gt;&#xA;&lt;li&gt;raw rna-seq data &lt;/li&gt;&#xA;&lt;li&gt;reference-based transcriptome assembly for each genome assembly (fasta/gtf)&lt;/li&gt;&#xA;&lt;li&gt;set of predicted protein sequence based on Assembly 1 (fasta)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Things I was thinking of: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;build the EVM predictions on the Assembly 2 and verify the identity of Protein X by simple fasta identity check &lt;/li&gt;&#xA;&lt;li&gt;look for the protein sequence in the Assembly 2. I could use something similar to &lt;a href=&quot;http://www.bioinformatics.org/sms2/rev_trans.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reverse Translate&lt;/a&gt; to get the most likely nucleotide sequence from the input amino-acid sequence of Protein X, then align it to the Assembly 2 to check the coordinates of the locus it originates from - but I am not very sure of the relevance and/or correctness of this procedure&lt;/li&gt;&#xA;&lt;li&gt;something allowing 'blast' of the protein sequence in the Assembly 2 (but I am not aware of any software that could do that)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Any ideas on how I could do that?&lt;/p&gt;&#xA;" OwnerUserId="294" LastEditorUserId="298" LastEditDate="2017-11-13T20:10:58.897" LastActivityDate="2017-11-13T20:10:58.897" Title="Verify a predicted protein in one genome in a different genome of the same species" Tags="|proteins|assembly|validation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2830" PostTypeId="2" ParentId="2829" CreationDate="2017-11-13T12:18:18.610" Score="3" Body="&lt;p&gt;This should be very easy to do. Here are some options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a tool like &lt;a href=&quot;https://www.ebi.ac.uk/about/vertebrate-genomics/software/exonerate&quot; rel=&quot;nofollow noreferrer&quot;&gt;Exonerate&lt;/a&gt; or &lt;a href=&quot;https://www.ebi.ac.uk/Tools/psa/genewise/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GeneWise&lt;/a&gt; that can align protein sequences to genomic DNA while attempting to model splice sites etc. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As you said, blast. There's nothing special in what you describe, really, this is what tBLASTn is for. Just use your protein as the query sequence and tBLASTn as the blast flavor which will take a protein query and match it against all 6 possible reading frames of a DNA database. Set up your assembly as the blast DB, and then blast as you would normally. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Of the two, using the first approach is much better since it will try to build a valid gene model for you instead of simply finding the regions of high homology as blast will do. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, just for completeness, you can go old school and run a full de-novo gene prediction tool on your assembly. Something like &lt;a href=&quot;http://genome.crg.es/software/geneid/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GeneID&lt;/a&gt; for instance. But that's certainly overkill if you're just looking for a single gene.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-11-13T12:18:18.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2831" PostTypeId="2" ParentId="2824" CreationDate="2017-11-13T12:55:24.213" Score="4" Body="&lt;p&gt;A reverse image search on google leads to the &lt;a href=&quot;https://commons.wikimedia.org/wiki/File:BicubicInterpolationExample.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;exact code&lt;/a&gt; used to produce the second image:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[Xcoarse, Ycoarse] = meshgrid([0 1 2 3], [0 1 2 3]);&#xA;[Xfine, Yfine] = meshgrid(linspace(0,3,3000), linspace(0,3,3000));&#xA;DataCoarse = [ 1 2 4 1; ...&#xA;           6 3 5 2; ...&#xA;           4 2 1 5; ...&#xA;           5 4 2 3];&#xA;DataBicubicFine  = interp2(Xcoarse, Ycoarse, DataCoarse, Xfine, Yfine, 'bicubic');&#xA;figure&#xA;surf(Xfine, Yfine, DataBicubicFine); shading flat; colormap(jet); view(0, 90)&#xA; hold on; plot3(Xcoarse, Ycoarse, 10*ones(size(Xcoarse)), 'k.', 'MarkerSize', 20)&#xA;colorbar&#xA;print -dpng -r300 BicubicInterpolationExample.png &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="581" LastActivityDate="2017-11-13T12:55:24.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2833" PostTypeId="2" ParentId="2825" CreationDate="2017-11-13T18:51:09.507" Score="5" Body="&lt;p&gt;You can whip up something quite easily in Python using &lt;a href=&quot;https://github.com/mdshw5/pyfaidx&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyfaidx&lt;/a&gt;, which allows you to access a FASTA file by genomic coordinates. So you'd just have to pull the coordinates from the VCF to get the sequence, then output in FASTA format with whatever unique identifier for each sequence you want to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: The &lt;code&gt;FastaVariant&lt;/code&gt; class is new in &lt;code&gt;pyfaidx&lt;/code&gt;, I hadn't seen it before. It makes this even easier, as you won't have to manually mess with the VCF to get the positions. Something like the following should work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;consensus = FastaVariant('yourgenome.fasta', 'variants.vcf.gz', het=True, hom=True)&#xA;&#xA;out = open(&quot;variants.fasta&quot;, &quot;w&quot;)&#xA;&#xA;for chrom in consensus.keys:&#xA;    for var in consensus[chrom].variants_sites:&#xA;        record = consensus[chrom][var-1:var]&#xA;        print(record, file=out)&#xA;&#xA;out.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1269" LastEditorUserId="1269" LastEditDate="2017-11-14T18:51:42.240" LastActivityDate="2017-11-14T18:51:42.240" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2834" PostTypeId="2" ParentId="2825" CreationDate="2017-11-13T21:45:19.470" Score="2" Body="&lt;p&gt;Assuming you have &lt;code&gt;samtools&lt;/code&gt;-indexed FASTA files for your reference genome, and have &lt;a href=&quot;http://bedops.readthedocs.io&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS&lt;/a&gt; and &lt;a href=&quot;https://github.com/samtools/htslib&quot; rel=&quot;nofollow noreferrer&quot;&gt;htslib&lt;/a&gt; installed, you could use &lt;a href=&quot;https://gist.github.com/alexpreynolds/fa9b0f90e181e3b4f640&quot; rel=&quot;nofollow noreferrer&quot;&gt;bed2faidxsta.pl&lt;/a&gt; to convert BED to FASTA:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ vcf2bed &amp;lt; variants.vcf | bed2faidxsta.pl --fastaDir=/path/to/fasta &amp;gt; variants.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To index your FASTA files, you could do the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cd /path/to/fasta&#xA;$ bgzip -c my_genome.fa &amp;gt; my_genome.fa.bgz&#xA;$ samtools faidx my_genome.fa.bgz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="776" LastActivityDate="2017-11-13T21:45:19.470" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2835" PostTypeId="1" CreationDate="2017-11-14T00:26:35.937" Score="25" ViewCount="4256" Body="&lt;p&gt;Yesterday I was debugging some things in R trying to get a popular Flow Cytometry tool to work on our data. After a few hours of digging into the package I discovered that our data was hitting an edge case, and it seems like the algorithm wouldn't work correctly under certain circumstances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This bug is not neccessarily the most significant issue, but I'm confident that it has occurred for other users before me, perhaps in a more insidious and less obvious way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that since this tool was released, literature has been published that utiltised it, &lt;strong&gt;how does the profession handle the discovery of bugs in these widely used packages?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm sure that given the piecemeal nature of a lot of the libraries out there, this is going to happen in a much more significant way at some point (i.e. invalidating a large number of published results)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some context I'm a programmer who's been working across various other parts of software development, and is quite new to Bioinformatics.&lt;/p&gt;&#xA;" OwnerUserId="1706" LastEditorUserId="48" LastEditDate="2017-11-14T11:40:24.690" LastActivityDate="2017-11-15T12:05:23.593" Title="What happens if a major bug is discovered in a bioinformatic package that has been used in published literature?" Tags="|r|software-quality|" AnswerCount="5" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="2836" PostTypeId="2" ParentId="2835" CreationDate="2017-11-14T02:13:27.307" Score="21" Body="&lt;p&gt;I prefer to treat software tools and computers in a similar fashion to laboratory equipment, and in some sense biology in general. Biologists are used to unexpected things happening in their experiments, and it's not uncommon for a new discovery to change the way that people look at something. Things break down, cells die off quicker on a Wednesday afternoon, results are inconsistent, and that third reviewer keeps on about doing that thing that's been done a hundred times before without anything surprising happening (just not &lt;em&gt;this&lt;/em&gt; time). It's a good idea to record as much as can be thought of that might influence an experiment, and for software that includes any input data or command line options, and especially software version numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this sense, a discovered software bug can be treated as a new discovery of how the world works. If the discovery is made public, and other people consider that it's important enough, then some people might revisit old research to see if it changes things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, the nice thing about software is that bugs can be reported back to the creators of programs, and possibly fixed, resulting in an improved version of the software at a later date. If the bug itself doesn't spark interest and the program gets fixed anyway, people unknowingly use newer versions, and there might be a bit more confusion and discussion about why results don't match similar studies carried out before the software change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want a bit of an idea of the biological equivalent of a major software bug, have a look at the barcode &lt;a href=&quot;https://www.biorxiv.org/content/early/2017/04/09/125724&quot; rel=&quot;noreferrer&quot;&gt;index switching&lt;/a&gt; issue, or the &lt;a href=&quot;https://blogs.biomedcentral.com/on-biology/2016/03/22/mix-mayhem-cell-line-contamination-biological-research/&quot; rel=&quot;noreferrer&quot;&gt;cell line contamination&lt;/a&gt; issue.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-11-15T08:41:59.447" LastActivityDate="2017-11-15T08:41:59.447" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2837" PostTypeId="2" ParentId="2835" CreationDate="2017-11-14T08:15:17.873" Score="9" Body="&lt;p&gt;While I agree with &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2836/48&quot;&gt;gringer's answer&lt;/a&gt; I would first report the bug to the maintainers with a test case (if possible) in order to  have the bug corrected. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bug might be an erroneous implementation in a corner case or in a more common use case of the software. But it is a good opportunity to collaborate to increase the quality of the software, either by introducing tests (or expanding them), by re-evaluating the state of the art with other tools, or to communicate with users of other software that depends on the tool.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the bug and the time one is able to spend, do a collaboration to study the effect of such a bug in the published studies using the software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes the bug doesn't affect much, but sometimes it can have quite an impact which can be difficult to estimate. For instance, I &lt;a href=&quot;https://support.bioconductor.org/p/92979/&quot; rel=&quot;nofollow noreferrer&quot;&gt;found&lt;/a&gt; that the reactome.db package didn't have the same annotations that the available online database (in version 1.58.0), due to an incorrect mapping. However, it was impossible for me to evaluate the damage of such an error on the published literature, because gene set enrichment tools (usually) don't report the data they are based on and the versions of the databases used are often under reported in the literature.&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-11-15T09:32:32.397" LastActivityDate="2017-11-15T09:32:32.397" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2839" PostTypeId="2" ParentId="2835" CreationDate="2017-11-14T10:27:49.730" Score="4" Body="&lt;p&gt;It depends also where the package is available from. I mean if it is from &lt;a href=&quot;https://bioconductor.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioconductor&lt;/a&gt;, authors and maintainers are responsible to &lt;a href=&quot;https://www.bioconductor.org/developers/package-guidelines/#responsibilities&quot; rel=&quot;nofollow noreferrer&quot;&gt;respond to bug&lt;/a&gt; reports. It could be reported at &lt;a href=&quot;https://support.bioconductor.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;support sites&lt;/a&gt;, so others are also aware of the bug. If the maintainer doesn't fix it, it might lose its spot in bioconductor. However, other sites, such as CRAN or Sourgeforge might be less stringent.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2017-11-14T10:27:49.730" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2840" PostTypeId="5" CreationDate="2017-11-14T11:41:56.290" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-11-14T11:41:56.290" LastActivityDate="2017-11-14T11:41:56.290" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2841" PostTypeId="4" CreationDate="2017-11-14T11:41:56.290" Score="0" Body="Use this tag for methods to ensure quality of software or to compare the quality of software, not only for bugs in programs" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-11-21T08:33:10.730" LastActivityDate="2017-11-21T08:33:10.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2842" PostTypeId="2" ParentId="2835" CreationDate="2017-11-14T12:39:54.510" Score="7" Body="&lt;p&gt;If you have a software development background you probably know about filing bug reports. Your question seems to be more about how to correct the scientific literature which has been affected by the bug. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The traditional way that this has been done is by writing an paper that explains the bug and gives examples of how it may impact published results (see gringer's examples of index switching and HELA contamination). This way the community will be aware that older results using the implicated method need to be verified before building upon them. Unfortunately, awareness of these issues doesn't always diffuse through the community as quickly as it should, but it's probably still the best tool we have to ensure that science is self-correcting.&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2017-11-14T12:39:54.510" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2845" PostTypeId="2" ParentId="2816" CreationDate="2017-11-14T15:06:42.967" Score="1" Body="&lt;p&gt;Samtools/VCF is overkill for comparing amino acid sequences from a single protein. I would suggest making pairwise alignments between each sequence in list2 and the reference and making a list of mutations (2c and 6c in your example). Then repeat the procedure for list1, discarding sequences if they contain mutations in your stored list. This can all be done using biopython. &lt;a href=&quot;http://biopython.org/DIST/docs/tutorial/Tutorial.html#htoc86&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's&lt;/a&gt; how to make the alignments to get you started:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import pairwise2&#xA;from Bio import SeqIO&#xA;seq1 = SeqIO.read(&quot;alpha.faa&quot;, &quot;fasta&quot;)&#xA;seq2 = SeqIO.read(&quot;beta.faa&quot;, &quot;fasta&quot;)&#xA;alignments = pairwise2.align.globalxx(seq1.seq, seq2.seq)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="581" LastActivityDate="2017-11-14T15:06:42.967" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2846" PostTypeId="1" AcceptedAnswerId="2847" CreationDate="2017-11-14T15:21:31.560" Score="11" ViewCount="1846" Body="&lt;p&gt;Disclaimer: This is a self answered question for documentation purpose and I adapted this from the following &lt;a href=&quot;https://gist.github.com/sujaikumar/504b3b7024eaf3a04ef5&quot; rel=&quot;noreferrer&quot;&gt;github gist&lt;/a&gt;. Especially from users &lt;strong&gt;terrycojones&lt;/strong&gt; and &lt;strong&gt;peterjc&lt;/strong&gt; as well as &lt;strong&gt;sujaikumar&lt;/strong&gt; who raised the issue.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I have a strange situation. I have two BLAST databases (&lt;code&gt;20170821&lt;/code&gt; and &lt;code&gt;20170824&lt;/code&gt;) that both contain a certain subject sequence &lt;code&gt;LC074724&lt;/code&gt;. When I query them both with a given query, &lt;code&gt;Q&lt;/code&gt;, in one case (&lt;code&gt;20170824&lt;/code&gt;) there is a &lt;strong&gt;very strong match&lt;/strong&gt; and in the other (&lt;code&gt;20170821&lt;/code&gt;) &lt;strong&gt;no match at all&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is there not even a lower quality match found although the databases are nearly the same?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BLAST version&#xA;&#xA;$ blastn -version&#xA;blastn: 2.6.0+&#xA; Package: blast 2.6.0, build May 11 2017 22:22:40&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Both databases contain an identical subject sequence &lt;code&gt;LC074724&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ blastdbcmd -entry LC074724 -db 20170824 | md5&#xA;fcfbc167c7f81dfd75aad1bf3db6220b&#xA;&#xA;$ blastdbcmd -entry LC074724 -db 20170821 | md5&#xA;fcfbc167c7f81dfd75aad1bf3db6220b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Only difference for the databases is &lt;code&gt;20170821&lt;/code&gt; is just &lt;code&gt;20170824&lt;/code&gt; with an extra 643 sequences. Rest is completely identical.&lt;/p&gt;&#xA;" OwnerUserId="1786" LastEditorUserId="1786" LastEditDate="2017-11-14T15:39:14.313" LastActivityDate="2019-06-19T10:42:25.497" Title="Why does a very strong BLAST hit get lost when I change num_alignments, num_descriptions or max_target_seqs parameter?" Tags="|blast|sequence-alignment|filtering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2847" PostTypeId="2" ParentId="2846" CreationDate="2017-11-14T15:36:38.227" Score="11" Body="&lt;p&gt;I'm trying to figure out why. This will be a longer read, &lt;strong&gt;Tl;dr at the end&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Doing a match&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;There is a good match of &lt;code&gt;Q&lt;/code&gt; against &lt;code&gt;LC074724&lt;/code&gt; in &lt;code&gt;20170824&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ blastn -outfmt '6 qaccver saccver bitscore' -db 20170824 -query Q.fasta \&#xA;  -task blastn -max_hsps 1 | grep LC074724&#xA;Q   LC074724.1  2581&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that the bitscore of the &lt;code&gt;LC074724&lt;/code&gt; match is &lt;code&gt;2581&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there is &lt;strong&gt;no match&lt;/strong&gt; of &lt;code&gt;LC074724&lt;/code&gt; &lt;strong&gt;at all in&lt;/strong&gt; &lt;code&gt;20170821&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ blastn -outfmt '6 qaccver saccver bitscore' -db 20170821 -query Q.fasta \&#xA;-task blastn -max_hsps 1 | grep LC074724&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The option &lt;code&gt;-outfmt '6 qaccver saccver bitscore'&lt;/code&gt; tells BLAST to print the query accession number and version, the subject accession and version, and the bitscore of the match.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;What are the best hits for 20170821&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;So what are the bitscores of the Q matches in 20170821?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ blastn -outfmt '6 bitscore' -db 20170821 -query Q.fasta -task blastn -max_hsps 1 | sort -nr | head&#xA;3301&#xA;2616&#xA;2590&#xA;2590&#xA;2563&#xA;2554&#xA;2547&#xA;2544&#xA;2540&#xA;2540&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The bitscore of &lt;code&gt;Q&lt;/code&gt; against &lt;code&gt;LC074724&lt;/code&gt; in &lt;code&gt;20170824&lt;/code&gt; (2581) would have placed it &lt;strong&gt;in 5th position in the top 10 matches of Q&lt;/strong&gt;, but &lt;code&gt;LC074724&lt;/code&gt; is &lt;strong&gt;not matched at all&lt;/strong&gt; in &lt;code&gt;20170821&lt;/code&gt;! The lowest bitscores matched in &lt;code&gt;20170821&lt;/code&gt; is 2354.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;What's the difference between the databases?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Get the ids of the subjects in each database&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ blastdbcmd -entry all -db 20170824 | egrep '^&amp;gt;' | cut -c2- | sort &amp;gt; 20170824.ids&#xA;$ blastdbcmd -entry all -db 20170821 | egrep '^&amp;gt;' | cut -c2- | sort &amp;gt; 20170821.ids&#xA;&#xA;$ wc -l 2017082?.ids&#xA;    9754 20170821.ids&#xA;    9111 20170824.ids&#xA;   18865 total&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;20170821&lt;/code&gt; has 643 sequences that are not in &lt;code&gt;20170824&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ comm -23 20170821.ids 20170824.ids | wc -l&#xA;     643&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;20170824&lt;/code&gt; has no sequences that are not in &lt;code&gt;20170821&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ comm -13 20170821.ids 20170824.ids | wc -l&#xA;       0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the two have 9111 sequences in common:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ comm -12 20170821.ids 20170824.ids | wc -l&#xA;    9111&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These numbers match with the numbers from &lt;code&gt;wc -l&lt;/code&gt; above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So &lt;code&gt;20170821&lt;/code&gt; is just &lt;code&gt;20170824&lt;/code&gt; with an extra 643 sequences.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Maybe LC074724 matches, but with a low bit score?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;BLAST only shows the top 500 matched subjects by default. Let's ask it to show more:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ blastn -outfmt '6 qaccver saccver bitscore' -db 20170821 -query Q.fasta \&#xA;  -task blastn -max_hsps 1 -max_target_seqs 10000 &amp;gt; 20170821-10000-matches.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And....&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ head 20170821-10000-matches.txt&#xA;Q   Q   3301&#xA;Q   X234A   2616&#xA;Q   CS388973.1  2590&#xA;Q   DM059402.1  2590&#xA;Q   LC074724.1  2581&#xA;Q   KJ843188.1  2576&#xA;Q   AB697496.1  2576&#xA;Q   AB697499.1  2576&#xA;Q   AB697503.1  2576&#xA;Q   AB697508.1  2576&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There it is! And &lt;strong&gt;it's in 5th position, just as it should be&lt;/strong&gt; (based on the bitscores discussed above).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to confirm, taking away the &lt;code&gt;-max_target_seqs&lt;/code&gt; option gets no match:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ blastn -outfmt '6 qaccver saccver bitscore' -db 20170821 -query Q.fasta \&#xA;  -task blastn -max_hsps 1 | grep LC074724&#xA;$&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Three confusingly named BLAST options&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I've always found the names and descriptions of the following three BLAST command-line options very confusing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the Formatting options section of blastn -help:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;-num_descriptions =0&gt;   Number of database sequences to show one-line descriptions for   Not applicable for outfmt &gt; 4&lt;br&gt;&#xA;  Default = `500'    * Incompatible with:  max_target_seqs&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;-num_alignments =0&gt;   Number of database sequences to show alignments for   Default = `250'    * Incompatible with: &#xA;  max_target_seqs&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And from the Restrict search or results section:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;-max_target_seqs =1&gt;   Maximum number of aligned sequences to keep   Not applicable for outfmt &amp;lt;= 4   Default = `500'    *&#xA;  Incompatible with:  num_descriptions, num_alignments&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;From the section names, it seems like only the latter option (&lt;code&gt;-max_target_seqs&lt;/code&gt;) would have any effect on the search, and that the former two are &lt;strong&gt;just about what is displayed&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But... using either just &lt;code&gt;-num_descriptions&lt;/code&gt; or &lt;code&gt;-num_alignments&lt;/code&gt; with a big value also gets a match:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ blastn -db 20170821 -query Q.fasta -task blastn -max_hsps 1 -num_descriptions 10000 | grep LC074724&#xA;LC074724.1  Hepatitis B virus DNA, complete genome, isolate: p621     2581    0.0&#xA;&amp;gt;LC074724.1 Hepatitis B virus DNA, complete genome, isolate: p621&#xA;&#xA;$ blastn -db 20170821 -query Q.fasta -task blastn -max_hsps 1 -num_alignments 10000 | grep LC074724&#xA;LC074724.1  Hepatitis B virus DNA, complete genome, isolate: p621     2581    0.0&#xA;&amp;gt;LC074724.1 Hepatitis B virus DNA, complete genome, isolate: p621&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So BLAST is apparently finding the &lt;code&gt;LC074724&lt;/code&gt; match even without the high &lt;code&gt;-max_target_seqs&lt;/code&gt; option, but it's not displaying it unless there is a high &lt;code&gt;-num_alignments&lt;/code&gt; or &lt;code&gt;-num_descriptions&lt;/code&gt; value. (&lt;strong&gt;Keep in mind, this is the 5th best hit! Not somewhere &gt;500&lt;/strong&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to confirm, when none of the 3 options are given and no -outfmt option is either, the match is not found:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ blastn -db 20170821 -query Q.fasta -task blastn -max_hsps 1 | grep LC074724&#xA;$&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Tl;dr:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, the options &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; -evalue&#xA; -max_target_seqs&#xA; -num_alignments&#xA; -num_descriptions&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;are &lt;strong&gt;NOT&lt;/strong&gt; just filters applied to the results, but instead directly influence the results &lt;strong&gt;found&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This means you can End up with a &lt;strong&gt;bacterial&lt;/strong&gt; sequence as &lt;strong&gt;BEST hit&lt;/strong&gt; when using a value &lt;code&gt;x&lt;/code&gt;, and with no bacterial sequences &lt;strong&gt;at all&lt;/strong&gt; in your top20 when using value &lt;code&gt;y&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To quote the NCBI user service:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We don't consider this a bug, but I agree that we should document this&#xA;  possibility better. This can happen because limits, including max&#xA;  target sequences, are applied in an early ungapped phase of the&#xA;  algorithm, as well as later.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The cutoffs are applied during an ungapped extension phase, and some&#xA;  alignments may improve in score during gapped extension. The safest&#xA;  approach is to relax (increase) the threshold and deal with a larger&#xA;  result set.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This was totally unexpected behaviour for me and is as far as I know is nowhere documented in the BLAST help or similar.&lt;/p&gt;&#xA;" OwnerUserId="1786" LastEditorUserId="1786" LastEditDate="2017-11-14T15:45:06.537" LastActivityDate="2017-11-14T15:45:06.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2848" PostTypeId="1" CreationDate="2017-11-14T20:25:29.010" Score="3" ViewCount="143" Body="&lt;p&gt;&lt;em&gt;Note: this question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/283612/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are seeing a retained intron transcript event for some RNAseq samples, and we want to assess at the sequence level which intron retention events are actually occurring so that we can do protein domain predictions. For example, we have transcript level expression support for DROSHA-203, which according to Ensembl is a retained intron transcript that potential gives rise to multiple splicing variants via the cDNA sequence information: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OnAMx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OnAMx.png&quot; alt=&quot;DROSHA-203 cDNA&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In theory, the orange sites indicate where a mutation might occur which could obliterate a splicing recognition site, thereby causing an retained intron event. So in total there are 6 potential mutational events which might give rise to 4 possible retention events. Given that 4 intronic regions might be retained, the number of unique retention events is a combinatorics question equal to 2^4, e.g., intron1 retained only, vs intron1 + 2 retained, all possible unique combination, etc, correct?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better way to more granularly quantify these possible transcripts? Or do we have to PCR validate all of them to figure out which one's are actually occurring? Is there a way that I can look at the reads mapping to the corresponding intronic regions to make even a probabilistic assessment of which transcripts are present and to what extent they are expressed? The libraries are prepped for total RNA with a ribozero depletion, so I should still be able to quantify any enrichment to these intronic regions, correct?&lt;/p&gt;&#xA;" OwnerUserId="962" LastEditorUserId="73" LastEditDate="2017-11-14T23:11:56.270" LastActivityDate="2017-11-15T02:33:48.183" Title="Quantification of EST/cDNA based retained intron transcripts" Tags="|rna-seq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2849" PostTypeId="2" ParentId="2848" CreationDate="2017-11-14T23:08:43.433" Score="1" Body="&lt;p&gt;You can do full-length whole-transcriptome sequencing using either the &lt;a href=&quot;https://store.nanoporetech.com/catalog/product/view/id/167/s/direct-rna-sequencing-kit/category/28/&quot; rel=&quot;nofollow noreferrer&quot;&gt;direct RNA kit&lt;/a&gt; or a cDNA kit (&lt;a href=&quot;https://store.nanoporetech.com/catalog/product/view/id/177/s/cdna-pcr/category/28/&quot; rel=&quot;nofollow noreferrer&quot;&gt;with&lt;/a&gt; or &lt;a href=&quot;https://store.nanoporetech.com/catalog/product/view/id/178/s/direct-cdna/category/28/&quot; rel=&quot;nofollow noreferrer&quot;&gt;without&lt;/a&gt; PCR) on a MinION. This will allow you to quantify isoforms directly, rather than estimating by looking at read pairs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it'd be a bit expensive to do it that way if you're looking at a single transcript (and aren't barcoding to multiplex samples on the same run).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you've already done Illumina whole-transcriptome sequencing (e.g. strand-specific paired-end sequencing), tools like &lt;a href=&quot;https://pachterlab.github.io/kallisto/download&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kallisto&lt;/a&gt;, &lt;a href=&quot;https://github.com/deweylab/RSEM#introduction&quot; rel=&quot;nofollow noreferrer&quot;&gt;RSEM&lt;/a&gt;, or &lt;a href=&quot;https://github.com/COMBINE-lab/salmon/releases&quot; rel=&quot;nofollow noreferrer&quot;&gt;Salmon&lt;/a&gt; can be used to estimate isoform expression based on the mapped reads. These tools use the information about transcript placement and/or similarity to alter the observed read counts in an attempt to better represent reality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, if you want these tools to quantify intronic sequence (i.e. the $2^4$ possibilities you have suggested), then those sequences need to be included in the sequences provided. They won't estimate frequencies for non-existent sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an alternative to including every possible gene model, something like a &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-017-1284-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;SuperTranscript model&lt;/a&gt; might work. I'm not familiar with Alicia Oshlack's method, but it at least gives the appearance that it might reduce the complexity of searching for multiple unannotated isoforms within a gene.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-11-15T02:33:48.183" LastActivityDate="2017-11-15T02:33:48.183" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2850" PostTypeId="2" ParentId="2823" CreationDate="2017-11-15T01:09:10.590" Score="1" Body="&lt;p&gt;I realized that some tools work better with Gene Names instead of some type of ID.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using IIS I was capable of creating a network using &lt;a href=&quot;http://www.uniprot.org/uniprot/?query=streptomyces+caatingaensis&amp;amp;sort=score&quot; rel=&quot;nofollow noreferrer&quot;&gt;this list&lt;/a&gt; of proteins (&lt;em&gt;Streptomyces caatingaensis&lt;/em&gt;) by querying proteins by Genes Names having as base the E. Coli.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tool has 3 main tabs: proteins, annotation, and interactome.&#xA;Search the Gene Names in proteins section, go to Interactome, set the parameters and create the network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. If you are not used to the tool you should read the research article available on the home page before using the network for research purposes.&lt;/p&gt;&#xA;" OwnerUserId="1650" LastEditorUserId="1650" LastEditDate="2017-11-15T01:18:49.953" LastActivityDate="2017-11-15T01:18:49.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2851" PostTypeId="1" AcceptedAnswerId="2852" CreationDate="2017-11-15T09:43:16.840" Score="7" ViewCount="1075" Body="&lt;p&gt;I'm learning the Gviz bioconductor package, I generate a plot as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Gviz)&#xA;track &amp;lt;- AnnotationTrack(start=c(1,5,7), end=c(2,6,10), strand=c('*','*','*'), stacking=&quot;dense&quot;, showFeatureId=TRUE, id=c('red','blue', 'red'))&#xA;pdf(file=paste(&quot;test&quot;,&quot;pdf&quot;, sep=&quot;.&quot;))&#xA;plotTracks(track)&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;test.pdf&lt;/code&gt; looks like this: &lt;a href=&quot;https://i.stack.imgur.com/bXZPG.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bXZPG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, what I want is for the boxes to be coloured rather than labelled with the colour word, how to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that my real example is much more complex than this.&lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="298" LastEditDate="2017-11-15T09:59:19.887" LastActivityDate="2017-11-15T10:02:28.663" Title="How can I colour boxes in Gviz AnnotationTrack in R?" Tags="|r|bioconductor|gviz|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2852" PostTypeId="2" ParentId="2851" CreationDate="2017-11-15T10:02:28.663" Score="8" Body="&lt;p&gt;Did you try the &lt;code&gt;fill&lt;/code&gt; argument?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;track &amp;lt;- AnnotationTrack(start=c(1,5,7), end=c(2,6,10), strand=c('*','*','*'), stacking=&quot;dense&quot;, showFeatureId=TRUE, id=c('red','blue', 'red'), fill=c('red','blue', 'red'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastActivityDate="2017-11-15T10:02:28.663" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2853" PostTypeId="1" AcceptedAnswerId="2854" CreationDate="2017-11-15T10:22:05.147" Score="8" ViewCount="1041" Body="&lt;p&gt;What are the least present &lt;em&gt;k&lt;/em&gt;-mers in the human genome at different sizes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Starting with k=4 and going up in size until k=10, what are the &lt;em&gt;k&lt;/em&gt;-mers least seen (or not at all) in the human genome? I am only interested in the reference human genome, so I am discounting SNPs/Indels in the population.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is not pre-calculated somewhere, what tool is recommended to use that starts with the &lt;em&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Reference_genome#Human_reference_genome&quot; rel=&quot;noreferrer&quot;&gt;GRCh38&lt;/a&gt;&lt;/em&gt; reference as input?&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="1775" LastEditDate="2017-11-15T18:22:11.280" LastActivityDate="2017-11-15T21:57:50.937" Title="Least present k-mers in the human genome" Tags="|human-genome|sequence-analysis|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2854" PostTypeId="2" ParentId="2853" CreationDate="2017-11-15T11:56:29.283" Score="12" Body="&lt;p&gt;You can use the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/21217122/&quot; rel=&quot;noreferrer&quot; title=&quot;Link to the article&quot;&gt;Jellyfish&lt;/a&gt; &lt;a href=&quot;http://www.cbcb.umd.edu/software/jellyfish&quot; rel=&quot;noreferrer&quot; title=&quot;Link to the help page&quot;&gt;software&lt;/a&gt; to calculate the k-mer profiles up to length 31.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the instructions in the user &lt;a href=&quot;http://www.genome.umd.edu/docs/JellyfishUserGuide.pdf&quot; rel=&quot;noreferrer&quot;&gt;guide&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic command to count all k-mers is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;jellyfish count -m 21 -s 100M -t 10 -C reads.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To compute the histogram of the k-mer occurrences, use the histo subcommand (see section 3.1):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;jellyfish histo mer_counts.jf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To query the counts of a particular k-mer, use the query subcommand (see section 3.3):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;jellyfish query mer_counts.jf AACGTTG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To output all the counts for all the k-mers in the le, use the dump subcommand (see section 3.2):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;jellyfish dump mer_counts.jf &amp;gt; mer_counts_dumps.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="964" LastEditorUserId="48" LastEditDate="2017-11-15T16:21:05.233" LastActivityDate="2017-11-15T16:21:05.233" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2855" PostTypeId="2" ParentId="2835" CreationDate="2017-11-15T12:05:23.593" Score="7" Body="&lt;p&gt;In extreme cases, it can destroy careers:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;One of the most spectacular flameouts in science happened last year.&#xA;  In a short letter (barely over 300 words long) published in Science in&#xA;  the very last issue of 2006, Geoffrey Chang, a crystallographer,&#xA;  retracted 3 Science articles, a Nature article, a PNAS article and a&#xA;  JMB article. The sum of 5 years of work was destroyed, apparently,&#xA;  over a single sign error in a data-processing program.&#xA;  – &lt;a href=&quot;http://boscoh.com/protein/a-sign-a-flipped-structure-and-a-scientific-flameout-of-epic-proportions.html&quot; rel=&quot;noreferrer&quot;&gt;A Sign, a Flipped Structure, and a Scientific Flameout of Epic Proportions&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Most cases are not that extreme. In fact, realistically, I would guess that &lt;em&gt;most&lt;/em&gt; papers have some data processing errors of some kinds, yet still we muddle on towards truth. And this isn't unique to software – the whole idea of independent replication in science is that we're always probably messing things up in ways we don't realize, so we need to re-do things and hope that the errors are different this time. But software is unusual in how tiny errors can cause very large effects, and often the effects are hard to detect. If you drop a test tube on the floor, you'll probably notice. If a piece of new software gives incorrect results... well, we didn't know what the results were supposed to look like, that's what makes it science! There are things you can do to &lt;a href=&quot;https://www.nature.com/news/blind-analysis-hide-results-to-seek-the-truth-1.18510&quot; rel=&quot;noreferrer&quot;&gt;mitigate this somewhat&lt;/a&gt;, but to some extent it's just inherent to the scientific process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I still like to show that blog post to new grad students to scare them into writing tests. I actually think it's a good habit to, whenever you start using a new software package, start by writing some tests for it – it's a good way to check your assumptions and understanding about what it actually does, and even if it's a popular package by eminent researchers, you'll still sometimes be shocked by what you find.&lt;/p&gt;&#xA;" OwnerUserId="1801" LastActivityDate="2017-11-15T12:05:23.593" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2856" PostTypeId="2" ParentId="2853" CreationDate="2017-11-15T12:23:23.280" Score="10" Body="&lt;p&gt;You can also use R. I give you an example of only chr1 and only kmer=4.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(BSgenome.Hsapiens.UCSC.hg38)&#xA;library(Biostrings)&#xA;&#xA;genome &amp;lt;- BSgenome.Hsapiens.UCSC.hg38&#xA;kmers &amp;lt;- oligonucleotideFrequency(genome$chr1, 4)&#xA;kmers&#xA;&#xA;m &amp;lt;- as.matrix(kmers)&#xA;&#xA;m[order(m),]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2017-11-15T12:45:32.263" LastActivityDate="2017-11-15T12:45:32.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2857" PostTypeId="2" ParentId="2812" CreationDate="2017-11-15T12:47:09.740" Score="0" Body="&lt;p&gt;I am one of the authors of the &lt;code&gt;UMI-tools&lt;/code&gt; pacakge, and although you probably could hack together a way to do this with our package, its not what its designed for and the solutions above are probably better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best way to do this would be to filter the fastq before mapping if thats a possibility, if it has to be the BAM, then here is a python implementation similar to Pierres java solution above. This one fiddles about with strands and cigar operations a bit more. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First sort the reads by name with &lt;code&gt;samtools sort -n&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;import pysam&#xA;inbam = pysam.AlignmentFile(&quot;mybam.bam&quot;)&#xA;outbam = pysam.AlignmentFile(&quot;mybam.filtered.bam&quot;, &quot;wb&quot;, template=inbam)&#xA;seq_to_find = &quot;AAGG&quot;&#xA;rc_seq_to_find = &quot;CCTT&quot;&#xA;&#xA;def test_and_out_buffer(read_buffer, outbam, seq_to_find, rc_seq_to_find):&#xA;    '''This function takes a list of reads and check if any of them&#xA;    have the prespecific sequences at their start. If the sequence&#xA;    is detected the whole buffer is output. The start of the read &#xA;    is the end of the alignment if the read is on the '-' strand. &#xA;    Also checks that the read doesn't start with hard-clipped sequence'''&#xA;&#xA;    do_output=False&#xA;&#xA;    for segment in read_buffer:&#xA;&#xA;        if segment.is_reverse:&#xA;            # the last operation is stored in the first part of the&#xA;            #last of the cigar tuples. Operation 5 is Hard-clip&#xA;            output_condition = (not segment.cigartuples[-1][0] == 5) and \&#xA;              segment.query_sequence.endswith(rc_seq_to_find)&#xA;        else:&#xA;            output_condition = (not segment.cigartuples[0][0] == 5) and \&#xA;              segment.query_sequence.startswith(seq_to_find)&#xA;&#xA;        if output_condition:&#xA;            do_output=True&#xA;            break&#xA;&#xA;    if do_output:&#xA;       for segment in read_buffer:&#xA;           outbam.write(segment)&#xA;&#xA;last_name=&quot;&quot;&#xA;read_buffer=[]&#xA;&#xA;# Loop over reads in the file&#xA;for read in inbam.fetch(until_eof=True):&#xA;&#xA;    # if the name hasn't changed, the name is part of the &#xA;    # same &quot;pair&quot;&#xA;    if read.query_name==last_name:&#xA;        buffer.append(read)&#xA;        continue&#xA;&#xA;    # if we got here then the a new pair is being examined&#xA;    # test the old pair and output if neccesary. &#xA;    test_and_output_buffer(read_buffer, outbam, seq_to_find, rc_seq_to_find)  &#xA;&#xA;    # make a new buffer with just the single new read.&#xA;    last_name=read.query_name&#xA;    read_buffer=[read]&#xA;&#xA;# the last set of reds will not have been output.  &#xA;test_and_output_buffer(read_buffer, outbam, seq_to_find, rc_seq_to_find)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="235" LastActivityDate="2017-11-15T12:47:09.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2858" PostTypeId="1" CreationDate="2017-11-15T14:45:45.050" Score="0" ViewCount="2073" Body="&lt;p&gt;I have a multifasta file containing several genes from a genomic island. I want to know the total length of this island. How can I do this?&lt;/p&gt;&#xA;" OwnerUserId="982" LastEditorUserId="57" LastEditDate="2017-11-23T11:58:38.537" LastActivityDate="2017-11-23T12:01:27.103" Title="How to find the sum of all sequences in a multifasta file" Tags="|fasta|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2859" PostTypeId="1" AcceptedAnswerId="2860" CreationDate="2017-11-15T16:43:53.120" Score="1" ViewCount="9734" Body="&lt;p&gt;We can use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;color yellow, resi 3&#xA;color yellow, resi 3-8&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to colour one residue or a range of residues. But what if I want to colour residue 3-8 and 10-20, how to write it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, my protein has two chains, light chain (L) with 100 residues and heavy chain (H) with 100 residues. How to colour the residue 10-20, 30-40 of light chain, and the residue 5-10, 20-30 of heavy chain in one go?&lt;/p&gt;&#xA;" OwnerUserId="1806" LastEditorUserId="712" LastEditDate="2017-11-16T18:25:42.427" LastActivityDate="2017-11-16T18:25:42.427" Title="How to colour multiple residues in Pymol?" Tags="|pymol|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2860" PostTypeId="2" ParentId="2859" CreationDate="2017-11-15T18:28:58.057" Score="3" Body="&lt;p&gt;Note: as I don't have any working version of pymol at the moment,&#xA;I am not able to test the solution properly.&lt;br&gt;&#xA;However, an easy way is to first create a selection, and then to color it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;select toBecolored, resi 3-8&#xA;color yellow, toBecolored&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Regarding the L/H chain part, if they are effectively annotated as L and H, something like the following should work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;select toBeColored, ((i. 10-20 or i. 30-40) and c. L ) or ((i. 5-10+20-30) and c. H)&#xA;color yellow, toBeColored&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Combining both commands is probably possible as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;color yellow, ((i. 10-20 or i. 30-40) and c. L ) or ((i. 5-10+20-30) and c. H)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Pymol allows synonyms (eg. resi and i.  or chain and c.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following pages may be also helpful:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pymolwiki.org/index.php/Selection_Algebra&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pymol wiki: help on Selection&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://pymol.sourceforge.net/newman/user/S0220commands.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;SourceForge&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And the two following links, if you want to get a bit deeper in proper programming with python for pymol (as they can provide you with inspiration):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pymolwiki.org/index.php/Category:Plugins&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pymol wiki: plugins&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/Pymol-Scripts/Pymol-script-repo&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pymol scripts&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="203" LastEditorUserId="203" LastEditDate="2017-11-16T14:55:32.380" LastActivityDate="2017-11-16T14:55:32.380" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2861" PostTypeId="1" CreationDate="2017-11-15T20:31:28.080" Score="6" ViewCount="335" Body="&lt;p&gt;I am wondering about the appropriate workflow to determine a protein's cellular location based on its sequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say I have a sequence like this from a fasta file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MTIKEHKVVYEAHNVKALKAPQHFYNSQPGKGYVTDMQHYQEMYQQSINEPEKFFDKMAK&#xA;EYLHWDAPYTKVQSGSLNNGDVAWFLNGKLNASYNCVDRHAFANPDKPALIYEADDESDN&#xA;KIITFGELLRKVSQIAGVLKSWGVKKGDTVAIYLPMIPEAVIAMLAVARIGAIHSVVFAG&#xA;FSAGSLKDRVVDANSKVVITCDEGKRGGKTINTKKIVDEGLNGVDLVSRILVFQRTGTEG&#xA;IPMKAGRDYWWHEEAAKQRTYLPPVSCDAEDPLFLLYTSGSTGSPKGVVHTTGGYLLGAA&#xA;LTTRYVFDIHPEDVLFTAGDVGWITGHTYALYGPLTLGTASIIFESTPAYPDYGRYWRII&#xA;QRHKATHFYVAPTALRLIKRVGEAEIAKYDTSSLRVLGSVGEPISPDLWEWYHEKVGNKN&#xA;CVICDTMWQTESGSHLIAPLAGAVPTKPGSATVPFFGINACIIDPVTGVELEGNDVEGVL&#xA;AVKSPWPSMARSVWNHHDRYMDTYLKPYPGHYFTGDGAGRDHDGYYWIRGRVDDVVNVSG&#xA;HRLSTSEIEASISNHENVSEAAVVGIPDELTGQTVVAYVSLKDGYLQNNATEGDAEHITP&#xA;DNLRRELILQVRGEIGPFASPKTIILVRDLPRTRSGKIMRRVLRKVASNEAEQLGDLTTL&#xA;ANPEVVPAIISAVENQFFSQKKK&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can feed it to &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PAGE=Proteins&quot; rel=&quot;noreferrer&quot;&gt;NCBI's blastp&lt;/a&gt; which then gives me a list of genes. I can check e.g. the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/ipg/NP_013254.1&quot; rel=&quot;noreferrer&quot;&gt;first entry&lt;/a&gt; which also &lt;a href=&quot;http://www.uniprot.org/uniprot/P52910&quot; rel=&quot;noreferrer&quot;&gt;provides a link to SwissProt&lt;/a&gt;. There I finally see that this protein is located in the nucleus and also the cytosol.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Is there a more straightforward way of achieving this information?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Let's say someone annotated this protein to be a mitochondrial one. How could I check that i.e. (dis)prove that?&lt;/p&gt;&#xA;" OwnerUserId="351" LastActivityDate="2017-11-16T18:32:12.957" Title="How to determine a protein's cellular location based on its sequence?" Tags="|proteins|blastp|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2862" PostTypeId="2" ParentId="2853" CreationDate="2017-11-15T21:57:50.937" Score="4" Body="&lt;p&gt;Grabbing chromosomes for &lt;code&gt;hg38&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ wget ftp://hgdownload.cse.ucsc.edu/goldenPath/hg38/chromosomes/*.fa.gz&#xA;$ for fn in `ls *.fa.gz`; do gunzip $fn; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Via &lt;a href=&quot;https://github.com/alexpreynolds/kmer-counter&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;kmer-counter&lt;/code&gt;&lt;/a&gt; and Python, here's how to search for kmers of length 7 from chromosome &lt;code&gt;chrY&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python                                                                                                                                                                                                                                                                                                                                                                                                                    &#xA;&#xA;import sys&#xA;import subprocess&#xA;import itertools&#xA;&#xA;k = 7&#xA;chr = 'chrY'&#xA;fastaFile = '%s.fa' % (chr)&#xA;kmerCmd = 'kmer-counter --fasta --no-rc --k=%d %s' % (k, fastaFile)&#xA;&#xA;try:&#xA;    output = subprocess.check_output(kmerCmd, shell=True)&#xA;    result = {}&#xA;    for line in output.splitlines(): &#xA;        (header, counts) = line.strip().split('\t')&#xA;        header = header[1:]&#xA;        kmers = dict((key,int(val)) for (key,val) in [d.split(':') for d in counts.split(' ')])&#xA;        result[header] = kmers&#xA;except subprocess.CalledProcessError as error:&#xA;    sys.stderr.write(&quot;%s\n&quot; % (str(error)))&#xA;&#xA;kmers = result[chr]&#xA;comp = {'A':'T', 'C':'G', 'G':'C', 'T':'A'}&#xA;for kmerList in itertools.product('ACGT', repeat=k):&#xA;    kmerKey = ''.join(kmerList)&#xA;    kmerCompKey = ''.join(reversed([comp.get(b,b) for b in kmerList]))&#xA;    if kmerKey not in kmers and kmerCompKey not in kmers:&#xA;        kmers[kmerKey] = 0&#xA;&#xA;for key, val in sorted(kmers.iteritems(), key=lambda (key,val):(val,key)):&#xA;    sys.stdout.write(&quot;%s\t%s\n&quot; % (key, val))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This script will print a two-column tab-delimited text file to standard output, where the first column is the 7mer and the second column is the count of that 7mer and its reverse complement over &lt;code&gt;chrY&lt;/code&gt; (including zero-counts):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CGACGCG 20&#xA;CGTCGCG 20&#xA;CGCGATA 23&#xA;TACGCGC 25&#xA;...&#xA;AAATAAA 33521&#xA;TTTCTTT 34014&#xA;GAATGGA 35361&#xA;AATGGAA 36906&#xA;TTTTTTT 103093&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With a couple tweaks, this script could run over a range of &lt;em&gt;k&lt;/em&gt; and chromosomes for your reference genome.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2017-11-15T21:57:50.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2863" PostTypeId="2" ParentId="2861" CreationDate="2017-11-16T08:46:16.707" Score="6" Body="&lt;p&gt;You could use &lt;a href=&quot;https://www.blast2go.com/&quot; rel=&quot;noreferrer&quot; title=&quot;Blast2go company&quot;&gt;Blast2go&lt;/a&gt; to find the predicted relevant gene ontologies of your sequences. If you look for the cellular compartment sub ontology you should find the location of such protein. I have never used this tool, but here is the &lt;a href=&quot;https://academic.oup.com/nar/article/36/10/3420/2410320&quot; rel=&quot;noreferrer&quot; title=&quot;Blast2GO article&quot;&gt;paper&lt;/a&gt; describing how it works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wouldn't restrict to the first predicted match of blastp see if some more genes with a good fit are also located at the same sites. But otherwise your workflow seems right to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But to prove this is in X or Y you, or your team or collaborators, should always check with histochemistry if it really allocates on the predicted place. All the softwares predictions must be validated with some biological experiments to prove them. &lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-11-16T09:20:41.297" LastActivityDate="2017-11-16T09:20:41.297" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2864" PostTypeId="1" AcceptedAnswerId="2865" CreationDate="2017-11-16T14:17:55.213" Score="3" ViewCount="63" Body="&lt;p&gt;I went through the manuals and can't find the meaning of the below flag:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-filt_rules_l 'filter==0'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the link to manual for this flag: &lt;a href=&quot;https://mathgen.stats.ox.ac.uk/impute/filtering_options.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;filt_rules_l&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone explain?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IMPUTE version: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;impute_v2.3.1_x86_64_dynamic&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; The full IMPUTE2 code is irrelevant, I am &lt;strong&gt;not&lt;/strong&gt; using the command to impute, I received an imputed data and the script used to impute. This one flag was a mystery to me. I will update the post with impute software version when I find out.&lt;/p&gt;&#xA;" OwnerUserId="131" LastEditorUserId="131" LastEditDate="2017-11-20T08:21:12.647" LastActivityDate="2017-11-20T13:15:08.997" Title="IMPUTE2 missing filter option" Tags="|impute2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2865" PostTypeId="2" ParentId="2864" CreationDate="2017-11-16T14:41:49.077" Score="3" Body="&lt;p&gt;I have never used this program, but reading the manual in the link it has to mean the following.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The manual says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Writing &lt;code&gt;-filt_rules_l 'eur.maf&amp;lt;0.05'&lt;/code&gt; on the command line would tell the program to remove any variants with &lt;code&gt;eur.maf&lt;/code&gt; values less than 0.05 from the reference panel.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, &lt;code&gt;-filt_rules_l 'filter==0'&lt;/code&gt; would mean that the program removes any variants with &lt;code&gt;filter&lt;/code&gt; values of 0 from the reference panel. In the legend file there should be a column named &lt;code&gt;filter&lt;/code&gt;, which are filtered out when the value is 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the example legend file, in your case there must be a custom additional column named &lt;em&gt;&quot;filter&quot;&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ head ALL_1000G_phase1integrated_v3_chr1_impute.legend&#xA;&#xA;id position a0 a1 afr.aaf amr.aaf asn.aaf eur.aaf afr.maf amr.maf asn.maf eur.maf&#xA;rs58108140 10583 G A 0.0407 0.1713 0.1311 0.2071 0.0407 0.1713 0.1311 0.2071&#xA;rs189107123 10611 C G 0.0142 0.0276 0.0140 0.0211 0.0142 0.0276 0.0140 0.0211&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastEditorUserId="131" LastEditDate="2017-11-20T13:15:08.997" LastActivityDate="2017-11-20T13:15:08.997" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2866" PostTypeId="1" AcceptedAnswerId="2868" CreationDate="2017-11-16T16:55:21.330" Score="9" ViewCount="6925" Body="&lt;p&gt;I am trying to understand PCR duplicates in NGS analyses (actually whole-genome). I searched, and the best answer I found is in &lt;a href=&quot;http://www.cureffi.org/2012/12/11/how-pcr-duplicates-arise-in-next-generation-sequencing/&quot; rel=&quot;noreferrer&quot;&gt;this blog&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I don't understand if I understood how PCR duplicates arise  correctly because I cannot see the problem of having them in the downstream analysis - aside from computational problems, i.e. unnecessary redundancy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I understood correctly, PCR duplicates arise during library preparation when PCR amplifies the fragments with adapters. In this case, if you have duplicate fragments, you'll end up amplifying some of them twice or more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when you are doing the mapping, they should fall on the same region, most likely decreasing the quality of the mapping (since they increase the consensus on a specific sequence, which could have been subject to sequencing errors). But aside from that, you should have the same mapping with respect to the mapping where duplicates are removed, although with decreased quality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the quality problem the actual reason for removing PCR duplicates or is there anything that I'm missing?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="298" LastEditDate="2017-11-17T08:46:17.917" LastActivityDate="2017-11-17T08:46:17.917" Title="How do PCR duplicates arise and why is it important to remove them for NGS analysis?" Tags="|genome|ngs|genome-sequencing|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2867" PostTypeId="2" ParentId="2866" CreationDate="2017-11-16T17:20:57.743" Score="7" Body="&lt;p&gt;PCR polymerases introduce errors. When an error arises in the first few cycles of amplifications, it will appear in a reasonably high fraction of DNA fragments in the library. After sequencing, you may see the same error occur to multiple reads. If you remove PCR duplicates when calling variants, all errors are reduced down to one read. For high-coverage data, you won't call the error as a false variant. However, if you keep PCR duplicates, the multiple occurrences of the error may make the error look like a real variant. This is particularly bad for calling SNVs from impure tumor samples.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-11-16T17:20:57.743" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2868" PostTypeId="2" ParentId="2866" CreationDate="2017-11-16T17:22:53.677" Score="13" Body="&lt;p&gt;In any scenario where depth of coverage is an important factor, PCR duplicates erroneously inflate the coverage and, if not removed, can give the illusion of high confidence when it is not really there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, consider the following hypothetical scenario.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                                         *&#xA;TTTCATACTAACTAGCCTGCGGTCTGTGTTTCCCGACTTCTGAGTCATGGGGTTTCAATGCCTATAGATTC&#xA;               ..........................C.&#xA;                ............................&#xA;                      .............T..............&#xA;                       ..................C.........&#xA;                           ............................&#xA;                               ............................&#xA;                                   ............................&#xA;                                    .....C......................&#xA;                                       ..C.........................&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The marked position has a coverage of 9 (9 overlapping reads), 4 of which suggest the presence of an alternate allele. Since these 4 reads map to different positions, they are independent observations supporting the same putative single nucleotide variant (SNV). The &lt;code&gt;T&lt;/code&gt; in the third read can safely be ignored as a sequencing error, since it only appears once.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now consider the following hypothetical scenario.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                                         *&#xA;TTTCATACTAACTAGCCTGCGGTCTGTGTTTCCCGACTTCTGAGTCATGGGGTTTCAATGCCTATAGATTC&#xA;                ............................&#xA;                      .............T..............&#xA;                       ..................C.........&#xA;                       ..................C.........&#xA;                       ..................C.........&#xA;                       ..................C.........&#xA;                               ............................&#xA;                               ............................&#xA;                               ............................&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this case, the marked position also has a coverage of 9, but 7 of those reads appear to be PCR duplicates. If we remove the duplicates, the marked position has only a coverage of 4, and only one &lt;code&gt;C&lt;/code&gt; at the marked position, which is not enough confidence to call the SNV.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2017-11-16T17:22:53.677" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2869" PostTypeId="2" ParentId="2861" CreationDate="2017-11-16T18:32:12.957" Score="2" Body="&lt;p&gt;You can run the sequence through tools like SignalP to look for cell surface signals and so on, which will give you a clue about where I is bound for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;None of these tools will give you an absolutely definitive answer though, often because we simply don’t have the experimental data.&lt;/p&gt;&#xA;" OwnerUserId="929" LastActivityDate="2017-11-16T18:32:12.957" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2870" PostTypeId="2" ParentId="2858" CreationDate="2017-11-16T18:40:02.707" Score="2" Body="&lt;p&gt;Here’s a function that I keep around for calculating the individual lengths of fasta sequences&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;falens(){&#xA;awk '/^&amp;gt;/ {if (seqlen){print seqlen}; print ;seqlen=0;next; } { seqlen += length($0)}END{print seqlen}' $1&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And one I use for summing a column of numbers&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sum(){&#xA;awk '{ sum += $1 } END { print sum}'&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can combine these by piping the output to do the sum for you:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;falens myseqs.fa | grep -v “&amp;gt;” | awk '{ sum += $1 } END { print sum}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Can’t test this myself currently as I’m not at my laptop, but it’s based on 2 functions that work normally)&lt;/p&gt;&#xA;" OwnerUserId="929" LastActivityDate="2017-11-16T18:40:02.707" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2871" PostTypeId="1" AcceptedAnswerId="2876" CreationDate="2017-11-16T21:35:44.793" Score="2" ViewCount="714" Body="&lt;p&gt;I am trying to use Pilon to improve a reference based on some Illumina data I have got. So, I aligned the Illumina reads to the reference using bwa. Then, I want to use Pilon to improve the reference using the alignment file in BAM format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Information about Pilon is found &lt;a href=&quot;https://github.com/broadinstitute/pilon/wiki/Requirements-&amp;amp;-Usage&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Requirements of Pilon&quot;&gt;here&lt;/a&gt;. I installed Pilon using conda.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output of 'pilon' is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Pilon version 1.22 Wed Mar 15 16:38:30 2017 -0400&#xA;&#xA;    Usage: pilon --genome genome.fasta [--frags frags.bam] [--jumps jumps.bam] [--unpaired unpaired.bam]&#xA;                 [...other options...]&#xA;           pilon --help for option details&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output of:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pilon --genome ref.fa --frags readsSorted.bam --output pilo --outdir output/scaffold&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; Exception in thread &quot;main&quot; java.lang.reflect.InvocationTargetException&#xA;    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)&#xA;    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)&#xA;    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)&#xA;    at java.lang.reflect.Method.invoke(Method.java:498)&#xA;    at com.simontuffs.onejar.Boot.run(Boot.java:340)&#xA;    at com.simontuffs.onejar.Boot.main(Boot.java:166)&#xA;Caused by: java.lang.OutOfMemoryError: Java heap space&#xA;    at org.broadinstitute.pilon.GenomeRegion.&amp;lt;init&amp;gt;(GenomeRegion.scala:55)&#xA;    at org.broadinstitute.pilon.GenomeFile$$anonfun$contigRegions$1.apply(GenomeFile.scala:72)&#xA;	at org.broadinstitute.pilon.GenomeFile$$anonfun$contigRegions$1.apply(GenomeFile.scala:72)&#xA;    at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)&#xA;	at scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:234)&#xA;    at scala.collection.immutable.Range.foreach(Range.scala:160)&#xA;    at scala.collection.TraversableLike$class.map(TraversableLike.scala:234)&#xA;	at scala.collection.AbstractTraversable.map(Traversable.scala:104)&#xA;	at org.broadinstitute.pilon.GenomeFile.contigRegions(GenomeFile.scala:72)&#xA;	at org.broadinstitute.pilon.GenomeFile$$anonfun$2.apply(GenomeFile.scala:52)&#xA;    at org.broadinstitute.pilon.GenomeFile$$anonfun$2.apply(GenomeFile.scala:52)&#xA;    at scala.collection.immutable.List.map(List.scala:277)&#xA;    at org.broadinstitute.pilon.GenomeFile.&amp;lt;init&amp;gt;(GenomeFile.scala:52)&#xA;    at org.broadinstitute.pilon.Pilon$.main(Pilon.scala:98)&#xA;    at org.broadinstitute.pilon.Pilon.main(Pilon.scala)&#xA;    ... 6 more&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="48" LastEditDate="2017-11-17T08:03:46.230" LastActivityDate="2020-04-18T10:19:54.737" Title="Java error when launching Pilon" Tags="|assembly|reference-genome|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2872" PostTypeId="1" AcceptedAnswerId="2874" CreationDate="2017-10-11T20:38:10.763" Score="1" ViewCount="144" Body="&lt;p&gt;I have to run an analysis using Whole Exome Sequencing (WXS) data prepared with the TruSeq Exome Kit (2012).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need the &lt;code&gt;.bed&lt;/code&gt; file for the kit and I found it &lt;a href=&quot;https://support.illumina.com/sequencing/sequencing_kits/truseq-exome-kit/downloads.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this file was posted on Nov. 16, 2015, while the kit used in my case is from 2012. I searched but I didn't find the bedfile for that specific version online.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Will I run into problems if I use this version, and in this case, where can I retrieve the bedfile for 2012's version?&lt;/p&gt;&#xA;" OwnerUserId="1771" OwnerDisplayName="gc5" LastEditorUserId="1771" LastEditDate="2017-11-17T15:29:44.043" LastActivityDate="2017-11-17T15:37:00.930" Title="TruSeq Exome Kit bed file old version" Tags="|bed|exome|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2873" PostTypeId="1" AcceptedAnswerId="2878" CreationDate="2017-09-27T19:23:24.117" Score="1" ViewCount="124" Body="&lt;p&gt;I am running an analysis on TCGA bam files (GCRh37/hg19) of whole exome sequencing data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I usually find one bam file for each sample. However, at least one sample has two bam files, in which the last part of the name (before the extension) has the suffix &lt;code&gt;.1&lt;/code&gt; or &lt;code&gt;.2&lt;/code&gt; or &lt;code&gt;.3&lt;/code&gt;...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, &lt;code&gt;C500.TCGA-BL-A3JM-01A-12D-A21A-08.1.bam&lt;/code&gt; and &lt;code&gt;C500.TCGA-BL-A3JM-01A-12D-A21A-08.2.bam&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I didn't find an explanation for this last suffix. Are they duplicates, or.. ? Is there some wiki about the last part of the filename?&lt;/p&gt;&#xA;" OwnerUserId="1771" OwnerDisplayName="gc5" LastEditorUserId="1771" LastEditDate="2017-11-17T20:24:52.583" LastActivityDate="2017-11-17T20:24:52.583" Title="TCGA bam file duplicates (aligned reads) and related last part of TCGA barcode" Tags="|sam|sequence-alignment|human-genome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2874" PostTypeId="2" ParentId="2872" CreationDate="2017-11-17T07:55:28.080" Score="0" Body="&lt;p&gt;Usually it is more problematic to work with older versions of data with newer protocols than the other way around.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, I think that the features annotated in the (newer) bed file should be more accurate than the older one. Unless you specifically want to compare your experiment with a previous experiment performed with a bed file from 2012 you'll be fine using the newer one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However make sure that you are correctly using the right file and that you report (or record) each version of protocol and data used in case some doubts arise. &lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-11-17T15:37:00.930" LastActivityDate="2017-11-17T15:37:00.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2876" PostTypeId="2" ParentId="2871" CreationDate="2017-11-17T08:12:13.840" Score="2" Body="&lt;p&gt;Like gringer already commented, the cause of your error is nicely given by the &lt;code&gt;java&lt;/code&gt; program. Your default &lt;code&gt;max heap&lt;/code&gt; settings are probably too low for the program. If you have more memory on your machine you can give &lt;code&gt;max heap&lt;/code&gt; a higher value. See this &lt;a href=&quot;https://stackoverflow.com/questions/37335/how-to-deal-with-java-lang-outofmemoryerror-java-heap-space-error-64mb-heap&quot;&gt;post&lt;/a&gt; and answers on Stack overflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll probably need to use the &lt;code&gt;-Xmxn&lt;/code&gt; argument in &lt;code&gt;java&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2017-11-17T08:12:13.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2877" PostTypeId="1" AcceptedAnswerId="2879" CreationDate="2017-11-17T09:11:11.527" Score="4" ViewCount="1505" Body="&lt;p&gt;Does anyone know of a program that can align a genomic sequence with introns with the corresponding amino acid sequence?&lt;/p&gt;&#xA;&lt;p&gt;I have both the genomic sequence and the correct amino acid sequence but no information on the genemodel in e.g. gff or genbank format.&lt;/p&gt;&#xA;&lt;p&gt;This is not what one would call a 'normal' alignment, but I would like it to look like this:&lt;/p&gt;&#xA;&lt;h3&gt;Hypothetical example:&lt;/h3&gt;&#xA;&lt;p&gt;nucleotide sequence with intron in italic:&lt;/p&gt;&#xA;&lt;p&gt;ATGCACGATACGACTGACGTACGT&lt;em&gt;ACGTACGTACGT&lt;/em&gt;ACGTACGTACGGAGACGTAGACTC&lt;/p&gt;&#xA;&lt;p&gt;corresponding amino acid sequence:&lt;/p&gt;&#xA;&lt;p&gt;MHDTTDVRTYVRRRRL&lt;/p&gt;&#xA;&lt;p&gt;The intron does not encode for amino acid which should thus result in a gap in the 'alignment'.&#xA;Desired 'alignment':&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATGCACGATACGACTGACGTACGTACGTACGTACGTACGTACGTACGGAGACGTAGACTC&#xA;  M  H  D  T  T  D  V  R              T  Y  V  R  R  R  R L&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1817" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-11-17T11:39:02.687" Title="How to align genomic sequence with corresponding amino acid sequence" Tags="|sequence-alignment|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2878" PostTypeId="2" ParentId="2873" CreationDate="2017-11-17T09:12:10.810" Score="3" Body="&lt;p&gt;If you check the &lt;a href=&quot;https://tcga-data.nci.nih.gov/docs/publications/blca_2013/BLCA.BAMlist.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;bam list&lt;/a&gt; of TCGA, you'll see that they use different numbers for the bam files (maybe different settings). What the difference is, I don't know, but if you want the file that they used for their paper, it is &lt;strong&gt;C500.TCGA-BL-A3JM-01A-12D-A21A-08.2.bam&lt;/strong&gt;, the other one is not in the list.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2017-11-17T09:12:10.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2879" PostTypeId="2" ParentId="2877" CreationDate="2017-11-17T09:24:13.330" Score="3" Body="&lt;p&gt;Have a look at &lt;a href=&quot;https://www.ebi.ac.uk/about/vertebrate-genomics/software/exonerate&quot; rel=&quot;nofollow noreferrer&quot;&gt;exonerate&lt;/a&gt;'s protein2genome!&#xA;From the &lt;a href=&quot;https://www.ebi.ac.uk/about/vertebrate-genomics/software/exonerate-user-guide&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Aligning a protein to genomic sequence:&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Similarly, it is possible to align a protein sequence to the genome, (similar to GeneWise, but with&#xA;  heuristics).&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;code&gt;exonerate --model protein2genome query.fasta target.fasta&lt;/code&gt; &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;This model will allow introns in the alignment, but also allow frameshifts, and&#xA;  exon phase changes when a codon is split by an intron.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="196" LastEditorUserId="48" LastEditDate="2017-11-17T09:54:29.733" LastActivityDate="2017-11-17T09:54:29.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2880" PostTypeId="2" ParentId="2793" CreationDate="2017-11-17T09:44:42.133" Score="2" Body="&lt;p&gt;I'm guessing here, but I suspect this is at least in part due to the fact that all your samples in condition 1 are not expressed at all. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Formally you have an infinitely large fold-change between condition1 and condition2. Most tools solve this by adding a small pseudocount to unexpressed samples, but this will still result in a very high fold change. So far for the big fold-change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I had to guess at the reason why this is significant I would think in terms of dispersion estimation. &lt;a href=&quot;https://support.bioconductor.org/p/100764/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This Bioconductor post&lt;/a&gt; has some discussion with the authors of DESeq2 on how this works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally: calculating differential expression on these genes in these samples will never be really informative, as they really are only expressed in one sample. The way I see it you have two options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Exclude the sample from your analysis. Straightforward, but does not solve this problem for other samples.&lt;/li&gt;&#xA;&lt;li&gt;Filter your genes before calculating differential expression, i.e. remove all genes that are not expressed in a large number of samples. This approach is standard in for instance &lt;a href=&quot;https://pachterlab.github.io/sleuth/about&quot; rel=&quot;nofollow noreferrer&quot;&gt;sleuth&lt;/a&gt;. A good starting point would be to remove genes that are not expressed in 75% of your samples.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="196" LastActivityDate="2017-11-17T09:44:42.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2881" PostTypeId="1" AcceptedAnswerId="8690" CreationDate="2017-11-17T11:36:10.077" Score="8" ViewCount="378" Body="&lt;p&gt;I have ...&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A database with patient phenotypes in, stored as HPO terms&lt;/li&gt;&#xA;&lt;li&gt;Genetic data in whatever format I need&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I want ...&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;To store drug names in a way that won't make my life difficult&lt;/li&gt;&#xA;&lt;li&gt;If it's feasible, to make them so that you &lt;em&gt;could&lt;/em&gt; draw links between HPO terms, medications, genetic changes, and outcomes.  &lt;em&gt;Note that I'm not actively trying to do that, but I don't want to exclude the possibility of it in the future&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Ideally, I'd like those drug names in some kind of ontology, and even more ideally an ontology that's available on &lt;a href=&quot;http://bioportal.bioontology.org&quot; rel=&quot;noreferrer&quot;&gt;http://bioportal.bioontology.org&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So - what's the best drug ontology to use in this case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What relationships would this drug ontology describe? Structural similarity of the compounds? Affected pathways? How would they be linked? Do you want all drugs that can cause the same side effect linked, for instance? Or all drugs that can be used for the same disease? &#xA;  Drugs by disease would be the most useful.  The information that would be most useful would be something like&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Atenolol&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Atenolol 25mg&lt;/li&gt;&#xA;&lt;li&gt;Atenolol 50mg&lt;/li&gt;&#xA;&lt;li&gt;Atenolol 100mg&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Propanolol&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Propanolol 10mg&lt;/li&gt;&#xA;&lt;li&gt;Propanolol 40mg&lt;/li&gt;&#xA;&lt;li&gt;Propanolol 80mg&lt;/li&gt;&#xA;&lt;li&gt;...&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In short, being able to get a list of dosages for a given drug.  It'd be extra-useful if I could also retrieve a set of drugs for a given condition, so in the above example to retrieve a list of beta blockers (Atenolol, Metoprolol, Nebivolol, ...)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Have you looked at any of the various drug ontologies listed in bioportal.bioontology.org? Which ones have you discarded so we don't suggest the same ones? What were they missing that you need?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;MDDB doesn't seem to be structured as above&lt;/li&gt;&#xA;&lt;li&gt;Likewise The Drug Ontology&lt;/li&gt;&#xA;&lt;li&gt;RxNorm seems to be the best available that I've seen&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1819" LastEditorUserId="1819" LastEditDate="2017-11-20T15:56:59.937" LastActivityDate="2019-05-25T21:55:07.397" Title="What's a good ontology for drug names?" Tags="|gene|drugs|ontology|human|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2882" PostTypeId="2" ParentId="2877" CreationDate="2017-11-17T11:39:02.687" Score="3" Body="&lt;p&gt;The classic tools for this are &lt;a href=&quot;https://www.ebi.ac.uk/Tools/psa/genewise/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;genewise&lt;/code&gt;&lt;/a&gt; and the &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2879/298&quot;&gt;already mentioned&lt;/a&gt; &lt;code&gt;exonerate&lt;/code&gt;. Both are capable fo aligning protein sequences to genomic DNA (or cDNA) while modelling splice sites to give you a gene model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, this is &lt;a href=&quot;https://www.ebi.ac.uk/Tools/services/web/toolresult.ebi?jobId=genewise-I20171117-113519-0691-25950205-pg&quot; rel=&quot;nofollow noreferrer&quot;&gt;the output&lt;/a&gt;&lt;sup&gt;1&lt;/sup&gt; of &lt;code&gt;genewise&lt;/code&gt; on your sequences:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;EMBOSS_001         1 MHDTTDVRTYVR----RRRL                              &#xA;                     MHDTTDVRTYVR    RRRL                              &#xA;                     MHDTTDVRTYVRTYVRRRRL                              &#xA;EMBOSS_001         1 acgaaggcatgcatgcacac                              &#xA;                     taaccatgcatgcatggggt                              &#xA;                     gctgtcatgcatgcagatac                              &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output also includes a GFF of your sequence:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;EMBOSS_001  GeneWise    match   1   60  22.88   +   .   EMBOSS_001-genewise-prediction-1&#xA;EMBOSS_001  GeneWise    cds 1   60  0.00    +   0   EMBOSS_001-genewise-prediction-1    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Back when I was using these tools daily to align proteins to genomes, I found that exonerate was &lt;em&gt;much&lt;/em&gt; faster but sometimes slightly less accurate. This was a few years ago now though, so that might have changed. I would always suggest trying exonerate first and only going for genewise if it fails.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;1&lt;/sup&gt;&lt;sub&gt;This link will only be valid for a short while, but I just used the sequences from the OP as input for the &lt;a href=&quot;https://www.ebi.ac.uk/Tools/psa/genewise/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GeneWise webserver&lt;/a&gt;.&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-11-17T11:39:02.687" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2883" PostTypeId="1" AcceptedAnswerId="2884" CreationDate="2017-11-17T16:13:49.650" Score="3" ViewCount="913" Body="&lt;p&gt;I'm trying to phase my data (whole genome resequencing SNPs) with Shapeit (from ped file) and I get this error message : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[33mERROR:[0m Wrong ordering in physical positions curr_pos=54 prev_pos=65&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I found that it's due to the fact that my .map file is like this (4 columns):  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Chromosome_number    SNP_ID   SNP_genetic_position    SNP_physical_position  &#xA;Contig100131            .              0                        65  &#xA;Contig100220            .              0                        54  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And Shapeit only looks at &lt;code&gt;SNP_physical_position&lt;/code&gt; but does not check if the SNPs are on the same contig or not and so it says it is not happy because position 54 is after 65! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could I make it understand that the positions are not in the same contig and so there is no mistake? Must I run Shapeit per contig (I have a lot of them so I hope not)?  &lt;/p&gt;&#xA;" OwnerUserId="1821" LastEditorUserId="131" LastEditDate="2017-11-21T08:33:15.873" LastActivityDate="2017-11-21T08:33:15.873" Title="problem of &quot;ordering in physical positions&quot; phasing SNPs with Shapeit" Tags="|ngs|software-recommendation|errors|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2884" PostTypeId="2" ParentId="2883" CreationDate="2017-11-17T17:24:46.010" Score="5" Body="&lt;p&gt;&lt;a href=&quot;http://www.griv.org/shapeit/pages/snps.html&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;SHAPEIT Manual&quot;&gt;The manual&lt;/a&gt; suggests that each &amp;quot;chromosome&amp;quot; needs to have its own input file.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;You have to split the dataset by chromosomes. ShapeIT can just phase one chromosome at a time. If you are working on GWA data, you can proceed as follows:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;code&gt;for chr in $(seq 1 22) ; do plink --file example --chr $chr --recode --out example_chr$chr ; done&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You will obtain 44 files; &lt;code&gt;example_chr1.ped, example_chr1.map, ..., example_chr22.ped, example_chr22.map&lt;/code&gt; that you can phase separately.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="964" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-11-20T09:50:50.233" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2885" PostTypeId="1" CreationDate="2017-11-17T19:26:31.717" Score="8" ViewCount="428" Body="&lt;p&gt;I have one-dimentional array (human genome). Also I have two annotations for it, we can think about them as different peaks (it's nucleosome and secondary structures). How can we find correlation and/or causation between this annotations? Currently we are forming hypothesis like &quot;one element is always before other&quot; and statistically testing them. Can we be smarter and use known tools or at least automate this? &lt;/p&gt;&#xA;" OwnerUserId="1822" LastActivityDate="2019-06-28T11:27:48.117" Title="Comparing two genome annotations" Tags="|genome|statistics|sequence-annotation|" AnswerCount="2" CommentCount="6" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2887" PostTypeId="2" ParentId="2117" CreationDate="2017-11-19T20:42:24.850" Score="3" Body="&lt;p&gt;If you have more than a dozen or even hundreds of regions that you want to compare you could check a companion tool of HiGlass called &lt;strong&gt;&lt;a href=&quot;http://hipiler.higlass.io&quot; rel=&quot;nofollow noreferrer&quot;&gt;HiPiler&lt;/a&gt;&lt;/strong&gt;. HiPiler lets you arrange, cluster, and pile up to thousands of genomic regions. Take a look at this &lt;a href=&quot;https://www.youtube.com/watch?v=qoLqje5OYKg&quot; rel=&quot;nofollow noreferrer&quot;&gt;5min video introduction&lt;/a&gt; to see if the tool could help you to answer your questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I am the creator of HiPiler.&lt;/p&gt;&#xA;" OwnerUserId="1653" LastActivityDate="2017-11-19T20:42:24.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2888" PostTypeId="1" AcceptedAnswerId="2895" CreationDate="2017-11-20T18:45:12.740" Score="2" ViewCount="895" Body="&lt;p&gt;I am trying to assign different window sizes to my SNPs dataset to identify regions under selection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this is the head of my data&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;head(snp_ids)&#xA;   snp_id   chr  pos&#xA;Chr01__912    1  912&#xA;Chr01__944    1  944&#xA;Chr01__1107   1 1107&#xA;Chr01__1118   1 1118&#xA;Chr01__1146   1 1146&#xA;Chr01__1160   1 1160&#xA;&#xA;class(snp_ids)&#xA;data.frame&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have chosen 4 different window sizes, &lt;code&gt;win_size &amp;lt;- c(15000, 30000, 50000, 100000)&lt;/code&gt;.&#xA;I have assigned each of these different window sizes to my snp_ids dataset to identify how many SNPs are distributed within each window by looping through each window size &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (i in 1:length(win_size)){&#xA;  windows &amp;lt;- sapply(snp_ids$pos, function(x) (ceiling(x/win_size[i])))&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1749" LastEditorUserId="1749" LastEditDate="2017-11-26T02:11:20.823" LastActivityDate="2017-11-26T02:11:20.823" Title="Use paste function with apply families" Tags="|r|" AnswerCount="2" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2889" PostTypeId="1" CreationDate="2017-11-20T20:19:29.447" Score="3" ViewCount="43" Body="&lt;p&gt;I have a model (ODE) of an enzymatic reaction, and I'd like to estimate four of its parameters to fit some available data. To obtain the model I am just using Mass-Action Kinetics.&lt;/p&gt;&#xA;&lt;p&gt;Some more details:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The model is of the form &lt;span class=&quot;math-container&quot;&gt;$S_1+2S_2+S_3+E\leftrightarrow C \leftrightarrow P_1+P_2+P_3+E$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$S_i$&lt;/span&gt; are substrates concentrations, &lt;span class=&quot;math-container&quot;&gt;$P_i$&lt;/span&gt; products concentrations, &lt;span class=&quot;math-container&quot;&gt;$E$&lt;/span&gt; enzyme concentration, and &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; the complex concentration.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$S_2$&lt;/span&gt; is Hydrogen and &lt;span class=&quot;math-container&quot;&gt;$P_3$&lt;/span&gt; is Ammonium (&lt;span class=&quot;math-container&quot;&gt;$NH_4$&lt;/span&gt;), so besides the aforementioned reaction I have one reaction for the buffer and one reaction for ammonium dissociation, however, the parameters involved are known.&lt;/li&gt;&#xA;&lt;li&gt;Substrate concentrations are in the range of &lt;span class=&quot;math-container&quot;&gt;$1-10mM$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Enzyme concentrations are in the range of &lt;span class=&quot;math-container&quot;&gt;$.1-10\mu M$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The parameters I want to estimate are the 4 reaction rates, say, &lt;span class=&quot;math-container&quot;&gt;$k_1,k_{-1},k_2,k_{-2}$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The available data (measurements) is only pH. Thus I have a simulation of the model where the final output is &lt;span class=&quot;math-container&quot;&gt;$pH=log_{10}(1/S_2)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Then, I try to estimate the parameters from the data and the (simulated) pH.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;To estimate the parameters I am using Matlab, more specifically the 'Identification Toolbox' and the 'lsqnonlin' function.&lt;/p&gt;&#xA;&lt;p&gt;I set the initial guess &amp;quot;visually&amp;quot;, that is, I move manually the parameters and whenever I assume the simulation is &amp;quot;close enough&amp;quot;, I set the initial parameters guess.&lt;/p&gt;&#xA;&lt;p&gt;The problem is that some of the parameter of he initial guess are of order &lt;span class=&quot;math-container&quot;&gt;$1E20$&lt;/span&gt;. I am thinking that because of the large value of the parameters, the algorithm cannot find any better parameter values. I have tried moving several optimisation options without any valuable result.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Q:&lt;/strong&gt; Is there some specific software, or method that is commonly used to estimate parameters of biochemical reactions? Is the large values of kinetic rates a common problem in parameter identification for these type of systems?&lt;/p&gt;&#xA;&lt;p&gt;I do not know any &lt;span class=&quot;math-container&quot;&gt;$P_i$&lt;/span&gt; though, just pH as a function of time. In principle nor E neither C are constant, but I could assume quasi-steady state conditions, no problem. My feeling is that I am not using or setting up correctly the numerical tool...&lt;/p&gt;&#xA;" OwnerUserId="1830" LastEditorUserId="73" LastEditDate="2021-07-04T04:44:06.213" LastActivityDate="2021-07-04T04:44:06.213" Title="Large range parameter identification" Tags="|software-recommendation|algorithms|matlab|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2890" PostTypeId="1" AcceptedAnswerId="2901" CreationDate="2017-11-20T20:57:57.167" Score="5" ViewCount="2342" Body="&lt;p&gt;I've got what seems like it should be a simple question, but I can't seem to figure it out from google. &lt;a href=&quot;http://www.htslib.org/doc/bcftools.html#common_options&quot; rel=&quot;noreferrer&quot;&gt;bcftools&lt;/a&gt; has 4 output options:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Output compressed BCF (b), uncompressed BCF (u), compressed VCF (z), uncompressed VCF (v).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Are any of these the same as the output from &lt;a href=&quot;http://www.htslib.org/doc/tabix.html&quot; rel=&quot;noreferrer&quot;&gt;bgzip&lt;/a&gt;? &lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2017-11-21T15:39:27.753" Title="bcftools output vs. bgzip" Tags="|file-formats|vcf|htslib|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2891" PostTypeId="2" ParentId="2890" CreationDate="2017-11-20T21:43:00.983" Score="2" Body="&lt;p&gt;Yes the compressed outputs should be the same format as bgzip.  bcftools also has an option to index in either the CSI or TBI formats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The index function mentions that it indexes bgzip compressed files.  So one can assume that it also creates files using the bgzip compression.  Also all the pipelines that I have built use bgzip and bcftools.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[~]# bcftools index &#xA;&#xA;About:   Index bgzip compressed VCF/BCF files for random access.&#xA;Usage:   bcftools index [options] &amp;lt;in.bcf&amp;gt;|&amp;lt;in.vcf.gz&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="964" LastEditorUserId="964" LastEditDate="2017-11-21T15:39:27.753" LastActivityDate="2017-11-21T15:39:27.753" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2893" PostTypeId="1" AcceptedAnswerId="2894" CreationDate="2017-11-21T07:39:20.637" Score="1" ViewCount="156" Body="&lt;p&gt;I am performing a simulation study and am curious about the parameters of my simulated metagenome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the library and insert sizes of some of the most &quot;used&quot; sequencers. Mainly, I am interested in the different Illumina platforms (MiSeq, HiSeq, HiSeqX), 454 Pyrosequencing, and Ion Torrent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not need the long read technologies like Pac-Bio's SMRT and Oxford Nanopore since they are not typically used with metagenomics, yet. &lt;/p&gt;&#xA;" OwnerUserId="1833" LastActivityDate="2017-11-21T16:23:18.743" Title="What are Approximate Read Counts (Library Sizes) and Lengths (Insert Sizes) for Next-Generation DNA Sequencers?" Tags="|ngs|metagenome|reads|illumina|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2894" PostTypeId="2" ParentId="2893" CreationDate="2017-11-21T08:34:01.280" Score="0" Body="&lt;p&gt;The library size depends usually on your budget. There are some reviews where they compare the different platforms, but these papers get outdated so quickly, that it would be better to just check &lt;a href=&quot;https://en.wikipedia.org/wiki/DNA_sequencing#Next-generation_methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki&lt;/a&gt; for the specs of different platforms. Also on the websites of Illumina or Life Tech, you'll find the specs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB. 454 sequencing is discontinued, so that is not very relevant anymore in my opinion.&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2017-11-21T08:46:45.657" LastActivityDate="2017-11-21T08:46:45.657" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2895" PostTypeId="2" ParentId="2888" CreationDate="2017-11-21T08:38:04.667" Score="4" Body="&lt;p&gt;Not clear what we are trying to do, but here is the solution using &lt;code&gt;lapply&lt;/code&gt; that should replicate your existing &lt;code&gt;forloop&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# example input data&#xA;snp_ids &amp;lt;- read.table(text = &quot;&#xA;snp_id  chr  pos&#xA;Chr01__912   1  912&#xA;Chr01__944   1  944&#xA;Chr01__1107  1 1107&#xA;Chr01__1118  1 1118&#xA;Chr01__1146  1 1146&#xA;Chr01__1160  1 1160&quot;, header = TRUE)&#xA;&#xA;# window sizes to loop through&#xA;win_size &amp;lt;- c(15000, 30000, 50000, 100000)&#xA;&#xA;res &amp;lt;- cbind(snp_ids,&#xA;             data.frame(&#xA;               lapply(setNames(win_size, paste(&quot;window&quot;, win_size, sep = &quot;_&quot;)), function(w)&#xA;                 as.numeric(paste(snp_ids$chr, ceiling(snp_ids$pos/w), sep = &quot;.&quot;))&#xA;               )))&#xA;&#xA;# result&#xA;res&#xA;#        snp_id chr  pos window_15000 window_30000 window_50000 window_1e.05&#xA;# 1  Chr01__912   1  912          1.1          1.1          1.1          1.1&#xA;# 2  Chr01__944   1  944          1.1          1.1          1.1          1.1&#xA;# 3 Chr01__1107   1 1107          1.1          1.1          1.1          1.1&#xA;# 4 Chr01__1118   1 1118          1.1          1.1          1.1          1.1&#xA;# 5 Chr01__1146   1 1146          1.1          1.1          1.1          1.1&#xA;# 6 Chr01__1160   1 1160          1.1          1.1          1.1          1.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; If we really want to use &lt;code&gt;forloop&lt;/code&gt;, then try below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (i in win_size){&#xA;  windows &amp;lt;- ceiling(snp_ids$pos/i)&#xA;  snp_ids[, paste0(&quot;window_&quot;, i)] &amp;lt;- as.numeric(paste(snp_ids$chr, windows, sep = &quot;.&quot;))&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="131" LastEditorUserId="131" LastEditDate="2017-11-21T21:42:34.017" LastActivityDate="2017-11-21T21:42:34.017" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2896" PostTypeId="2" ParentId="2888" CreationDate="2017-11-21T09:24:34.490" Score="2" Body="&lt;p&gt;This explicitly doesn't use &lt;code&gt;apply()&lt;/code&gt; or &lt;code&gt;lapply()&lt;/code&gt;, but rather the &lt;code&gt;dplyr&lt;/code&gt; package. This is probably easier to read and debug:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(&quot;dplyr&quot;)&#xA;snp_ids %&amp;gt;% mutate(window_15000=as.numeric(paste(chr, ceiling(pos/15000), sep=&quot;.&quot;)),&#xA;                   window_30000=as.numeric(paste(chr, ceiling(pos/30000), sep=&quot;.&quot;)),&#xA;                   window_50000=as.numeric(paste(chr, ceiling(pos/50000), sep=&quot;.&quot;)),&#xA;                   window_100000=as.numeric(paste(chr, ceiling(pos/100000), sep=&quot;.&quot;)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-11-21T09:24:34.490" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2897" PostTypeId="1" CreationDate="2017-11-21T10:54:43.713" Score="6" ViewCount="6410" Body="&lt;p&gt;Maybe this is a silly question, but I'm really wondering why we usually get low mapping rates if we map total RNA-seq, but not poly(A)-enriched (in particular, for human, mouse, and zebrafish datasets)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doesn't the genome fasta file contain ribosomal RNAs (which are abundantly expected in the total RNA-seq libraries) as well?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have the numbers for human right now, but I recall previously having low mapping rates for human data as well. What triggered me at the moment is actually when I mapped zebrafish data. I got around 60% mapping to the genome and 45% to the transcriptome with one dataset and 36% - 35% with another dataset (note that these are from 2 different studies and both are total RNA-seq, mapping was done using STAR and Salmon, which have their own methods).&lt;/p&gt;&#xA;" OwnerUserId="1761" LastEditorUserId="77" LastEditDate="2017-11-21T13:46:19.067" LastActivityDate="2017-11-23T10:52:12.513" Title="Why total RNA-seq usually yields low mapping rate?" Tags="|rna-seq|read-mapping|mappability|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2899" PostTypeId="2" ParentId="2897" CreationDate="2017-11-21T13:40:50.957" Score="5" Body="&lt;p&gt;A likely explanation is that total RNA-Seq contains a high fraction of reads from ribosomal RNAs. Ribosomal RNAs are present in multiple copies across the genome, hence many reads map to multiple genomic locations and get discarded by the aligner. For example, STAR with default parameters considers a read as unmapped if it maps to more than 10 genomic loci (this behavior can be changed with the &lt;code&gt;--outFilterMultimapNmax&lt;/code&gt; option).&#xA;To confirm if this is actually the case, you could check the number of multi-mapping reads in the aligner's log files. &lt;/p&gt;&#xA;" OwnerUserId="1835" LastActivityDate="2017-11-21T13:40:50.957" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="2900" PostTypeId="2" ParentId="2897" CreationDate="2017-11-21T14:05:11.907" Score="3" Body="&lt;p&gt;In particular for mouse, this can happen if the ribo-depletion wasn't terribly efficient, since there's no &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene/100861531&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rn45s sequence&lt;/a&gt; in the reference genome. That, combined with numerous copies of tRNAs, 5S rRNAs, etc. causing issues with multimappers (see the answer from @Tom) can heavily decrease alignment rates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the human reference genome contains a few copies of the 45S (e.g., on GL000220.1 and chr21), so these reads will all multimap (possibly too many times). I don't know about the zebrafish reference genome.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-11-21T14:05:11.907" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2901" PostTypeId="2" ParentId="2890" CreationDate="2017-11-21T15:11:33.213" Score="5" Body="&lt;p&gt;Yes. I haven't found it documented anywhere, but I ran a simple test:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools view -Oz --no-version foo.vcf &amp;gt; bar.vcf.gz &#xA;$ md5sum bar.vcf.gz &#xA;18a2f494710cf170ba79892936015ba8  bar.vcf.gz&#xA;&#xA;$ gunzip bar.vcf.gz &#xA;$ bgzip bar.vcf &#xA;$ md5sum bar.vcf.gz &#xA;18a2f494710cf170ba79892936015ba8  bar.vcf.gz&#xA;&#xA;$ gunzip bar.vcf.gz &#xA;$ gzip bar.vcf &#xA;$ md5sum bar.vcf.gz &#xA;9f5b1cf654966912b78a9cb7175ffc2c  bar.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I passed a vcf file through &lt;code&gt;bcftools view -Oz&lt;/code&gt;, checked the &lt;code&gt;md5sum&lt;/code&gt; of the resulting file, then uncompressed it and recompressed with &lt;code&gt;bgzip&lt;/code&gt; and confirmed that the resulting file had the same &lt;code&gt;md5sum&lt;/code&gt;. Finally, I also compressed using normal &lt;code&gt;gzip&lt;/code&gt; to be sure that the &lt;code&gt;md5sum&lt;/code&gt; would be different, as it was.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So yes, it does indeed look like &lt;code&gt;bcftools&lt;/code&gt; creates files that are the same as though you'd passed them through &lt;code&gt;bgzip&lt;/code&gt;. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-11-21T15:11:33.213" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2902" PostTypeId="2" ParentId="2893" CreationDate="2017-11-21T16:23:18.743" Score="1" Body="&lt;p&gt;Both of those depend on the application of the library.  For example whole genome assembly of a mammal would require a larger library size than targeted re-sequencing of a virus.  Also insert size varies if you are doing genome assembly a larger insert size is valuable to allow for long range linking and scaffolding.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that if you edit your question of be more specific people will be able to provided a more concrete answer other wise it is &quot;it depends&quot; &lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2017-11-21T16:23:18.743" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2903" PostTypeId="1" AcceptedAnswerId="2904" CreationDate="2017-11-21T17:16:58.223" Score="3" ViewCount="105" Body="&lt;p&gt;First I am not sure why, but it seems EMT can be used to infer the status of cancer progression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kind of databases are there for the &lt;a href=&quot;https://en.wikipedia.org/wiki/Epithelial%E2%80%93mesenchymal_transition&quot; rel=&quot;nofollow noreferrer&quot;&gt;EMT&lt;/a&gt; process in the context of cancer? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I saw this &lt;a href=&quot;https://books.google.com/books?id=_m8c76i0tqUC&amp;amp;pg=PA127&amp;amp;lpg=PA127&amp;amp;&quot; rel=&quot;nofollow noreferrer&quot;&gt;book&lt;/a&gt; which mentions &lt;a href=&quot;https://www.ebi.ac.uk/arrayexpress/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ArrayExpress&lt;/a&gt;, but I need some help to understand what kind of data it stores. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there similar/better resources? How can I use EMT in the context of cancer progression study? &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Are there other process like EMT that are being used to study cancer progression? &lt;/li&gt;&#xA;&lt;li&gt;Are there EMT like events in leukemia progression? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2018-08-26T02:12:44.397" LastActivityDate="2018-08-26T02:12:44.397" Title="EMT/EMT-like processes in bioinformatics to study cancer progression" Tags="|cancer|pathway|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="2904" PostTypeId="2" ParentId="2903" CreationDate="2017-11-21T17:45:57.183" Score="2" Body="&lt;p&gt;EMT describe cells' change in their state from being epithelial to the mesenchymal class. So, if a cancer cell line is gaining properties that allow it to move, it might reach the metastasis phase, were a cell can move to any point of the body and start a new tumor there.  As in the wikipedia page you linked:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;EMT has also been shown to occur in wound healing, in organ fibrosis and in the initiation of metastasis in cancer progression.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am not aware of any kind of database for the EMT process, the most relevant resource might be the gene ontology, which has a &lt;a href=&quot;http://amigo.geneontology.org/amigo/term/GO:0001837&quot; rel=&quot;nofollow noreferrer&quot;&gt;term for it&lt;/a&gt;, which lists 185 (human) genes as related to this biological process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ArrayExpress is a database of transcriptomics, it :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;stores data from high-throughput functional genomics experiments&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can evaluate the role of EMT in the data stored in the database.&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-12-12T07:52:09.737" LastActivityDate="2017-12-12T07:52:09.737" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2905" PostTypeId="1" AcceptedAnswerId="2906" CreationDate="2017-11-21T21:49:44.550" Score="1" ViewCount="2908" Body="&lt;p&gt;I have a list of SNPs (index SNPs). As my tagging SNPs, I want to use SNP data from &lt;a href=&quot;http://web.pasteur-lille.fr/en/recherche/u744/igap/igap_download.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The data has the following column headers: Chromosome, Position, MarkerName, Effect_allele,   Non_Effect_allele, Beta, SE, Pvalue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My ultimate aim is to identify SNPs in the tagging set that are in LD with each of index SNPs. I need to calculate r2 between index SNPs and tagging SNPs within 1MB window and get a table of index vs tagging SNPs (r2&gt;0.8). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I use PLINK to carry out this work? I'm new to PLINK and LD clumping&lt;/p&gt;&#xA;" OwnerUserId="1837" LastEditorUserId="48" LastEditDate="2017-11-22T10:12:34.560" LastActivityDate="2017-11-22T10:13:52.723" Title="Using PLINK to find SNPs in LD (Linkage Disequilibrium) with another set of SNPs" Tags="|snp|plink|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2906" PostTypeId="2" ParentId="2905" CreationDate="2017-11-22T00:36:08.450" Score="1" Body="&lt;p&gt;You can't generate LD statistics for summary data from SNPs, as the LD calculation relies on the comparison of variants at the genotype level. Plink expects data to be in a genotype matrix with either each marker for each line (probably better in your case), or each individual for each line. Plink v1.9, in particular, will accept VCF-formatted input files, when populated with 'GT' information that describes the variants for each individual.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're okay with using the 1000 genome population for calculating LD, plink v1.9-compatible datasets should be accessible from&#xA;&lt;a href=&quot;https://www.cog-genomics.org/plink2/resources&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, you'll need to discuss your project with IGAP to see if you can have access to genotype-level data for LD testing. That's unlikely, because there are personal identification issues associated with non-summary data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's comprehensive documentation about the LD command(s) on the Plink 1.9 website &lt;a href=&quot;https://www.cog-genomics.org/plink/1.9/ld&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="48" LastEditDate="2017-11-22T10:13:52.723" LastActivityDate="2017-11-22T10:13:52.723" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2908" PostTypeId="1" CreationDate="2017-11-22T10:43:48.320" Score="6" ViewCount="263" Body="&lt;p&gt;I have a eukaryotic genome for which an updated sequence for a chromosome  was recently obtained. I want to map RNAseq reads on the genome (and perform other downstream analyses) and would like to use the most up-to-date information possible (so the 'new' sequence of that chromosome). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, before performing the read mapping, I'd like to update the annotation of my genome (GFF3) to match with the 'new' coordinates of this chromosome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, I would like to transfer information from the 'old-coordinates' GFF3 to obtain a 'new-coordinates' GFF3, and if possible conserve all the information/hierarchy of the file (gene, mRNA, exon, etc.). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chrXX   source  gene    222 5942    .   -   .   ID=gene_1;Name=gene_1;length=5720&#xA;chrXX   source  mRNA    222 5942    .   -   .   ID=gene_1.1;Parent=gene_1;Name=gene_1.1;length=5720&#xA;chrXX   source  exon    222 5794    .   -   .   ID=gene_1.1.2;Parent=gene_1.1&#xA;chrXX   source  exon    5889    5942    .   -   .   ID=gene_1.1.1;Parent=gene_1.1&#xA;chrXX   source  CDS 222 5794    .   -   1   ID=CDS:gene_1.1.2;Parent=gene_1.1;Name=gene_1.1&#xA;chrXX   source  CDS 5889    5942    .   -   0   ID=CDS:gene_1.1.1;Parent=gene_1.1;Name=gene_1.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;... should be updated to ...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chrXX   source  gene    333 6053    .   -   .   ID=gene_1;Name=gene_1;length=5720&#xA;chrXX   source  mRNA    333 6053    .   -   .   ID=gene_1.1;Parent=gene_1;Name=gene_1.1;length=5720&#xA;chrXX   source  exon    333 5905    .   -   .   ID=gene_1.1.2;Parent=gene_1.1&#xA;chrXX   source  exon    6000    6053    .   -   .   ID=gene_1.1.1;Parent=gene_1.1&#xA;chrXX   source  CDS 333 5905    .   -   1   ID=CDS:gene_1.1.2;Parent=gene_1.1;Name=gene_1.1&#xA;chrXX   source  CDS 6000    6053    .   -   0   ID=CDS:gene_1.1.1;Parent=gene_1.1;Name=gene_1.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The approach I tried&lt;/strong&gt; (but I am not sure it is the way to do it): &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Extract sequence of all the features that are going to be updated to fasta (&lt;code&gt;bedtools getfasta&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Map these sequences to the genome with the updated chromosome (&lt;code&gt;gmap&lt;/code&gt; with &lt;code&gt;--nosplicing&lt;/code&gt; since sequences we are mapping correspond to genomic regions). &lt;/li&gt;&#xA;&lt;li&gt;Create the updated GFF3 file. When doing that, I apply one supplementary rule: if a feature mapped on another chromosome, but there was also another alignment on the updated chromosome, prioritize the one that correspond to the same (updated) chromosome. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What would be the proper way to do such a thing?  Any suggestion of methods/tools is welcome! &lt;/p&gt;&#xA;" OwnerUserId="1839" LastActivityDate="2017-11-30T08:07:50.947" Title="Transferring genomic features on new coordinates" Tags="|gff3|gmap|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2909" PostTypeId="1" CreationDate="2017-11-22T10:46:20.670" Score="3" ViewCount="99" Body="&lt;p&gt;I'm planning to run a GSE Analysis on some microarray datasets. Until now what I have done is to pre-processed them using frma algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I chose frma instead of gcrma or rma is that I wanted to merge these different dataset into one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is how I'm planing to proceed:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dataset1 : 3x Controls , 3x Treatment1&#xA;dataset2 : 2x Controls , 2x Treatment2&#xA;dataset3 : 2x Controls , 2x Treatment3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After merging&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;7x Controls , 3x Treatment1 , 2x Treatment2 , 2x Treatment3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I will execute the GSEA for controls , Treatmen1 , Treatment2 and Treatment3 separably.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the question is if it's more reasonable to follow the path that I described or to run the GSEA on the separate datasets (for example for each control ) and then get the intersect of the gene sets of these three repetitions.&lt;/p&gt;&#xA;" OwnerUserId="704" LastActivityDate="2017-11-22T11:56:32.383" Title="Integration of different microarray dataset to run GSEA" Tags="|microarray|gse|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2910" PostTypeId="2" ParentId="2908" CreationDate="2017-11-22T11:42:11.453" Score="4" Body="&lt;p&gt;I think the standard way of doing this is to make a &lt;a href=&quot;http://genomewiki.ucsc.edu/index.php/LiftOver_Howto&quot; rel=&quot;noreferrer&quot;&gt;chain file&lt;/a&gt;, then use it to do a &lt;a href=&quot;http://crossmap.sourceforge.net&quot; rel=&quot;noreferrer&quot;&gt;liftOver&lt;/a&gt; of the annotations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mkdir psl&#xA;for i in ../ci3/rm/masked/*.masked; do blat ../ci2.2bit $i -tileSize=12 -fastMap -minIdentity=98 psl/`basename $i .fa.masked`.psl -noHead -minScore=100; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Translate psl files to chains in the directory chain: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mkdir chain&#xA;for i in psl/*.psl; do axtChain -linearGap=medium -psl $i ../ci2.2bit ../ci3/ci3.2bit chain/`basename $i .psl`.chain; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Merge short chains into longer ones into the directory chainMerge: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mkdir chainMerge&#xA;chainMergeSort chain/*.chain | chainSplit chainMerge stdin -lump=50&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;concat and sort the chains: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat chainMerge/*.chain &amp;gt; all.chain&#xA;chainSort all.chain all.sorted.chain&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Need info about chromosome sizes for netting: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;twoBitInfo ../ci3/ci3.2bit ci3.chromInfo&#xA;twoBitInfo ../ci2.2bit ci2.chromInfo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Netting: identify alignable regions from chains: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mkdir net&#xA;chainNet all.sorted.chain ci2.chromInfo ci3.chromInfo net/all.net /dev/null&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, select the right alignable regions using the nets, creating a &quot;liftOver&quot; file: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;netChainSubset net/all.net all.chain ci2ToCi3.liftOver&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Run liftOver:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CrossMapy.py bed ci2ToCi3.liftOver test.hg18.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="581" LastActivityDate="2017-11-22T11:42:11.453" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2911" PostTypeId="2" ParentId="2897" CreationDate="2017-11-22T11:46:40.057" Score="1" Body="&lt;p&gt;You can't assume that the rRNA genes are in the genome. There many copies of them and not all are placed on chromosomes. We've had this issue with human and mouse data for total RNA preps. You need to ensure you map against the &lt;em&gt;whole&lt;/em&gt; genome not just the chromosomes.&lt;/p&gt;&#xA;" OwnerUserId="1092" LastActivityDate="2017-11-22T11:46:40.057" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2912" PostTypeId="2" ParentId="2909" CreationDate="2017-11-22T11:56:32.383" Score="2" Body="&lt;p&gt;It makes more sense to evaluate by separate the pathway or gene set you want and see if in the three datasets result in a coherent message than to merge these datasets, as you will mix batch effects and treatment effects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When in several datasets one looks for the same statistic it is usually considered meta analysis, not integration. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2017-11-22T11:56:32.383" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2913" PostTypeId="1" AcceptedAnswerId="2915" CreationDate="2017-11-22T17:20:03.933" Score="4" ViewCount="2031" Body="&lt;p&gt;I am trying to run &lt;code&gt;snakemake&lt;/code&gt; version &lt;code&gt;3.5.4&lt;/code&gt; on my cluster, where jobs are partially defined by a common template build according &lt;a href=&quot;http://snakemake.readthedocs.io/en/stable/snakefiles/configuration.html#cluster-configuration&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt; and also defined though rule-specific parameters. But for some reason cluster configuration is not expanding wildcards (and command does). When I run&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;snakemake data/genome.fa.gz.bwt -p --jobs 10 --cluster-config cluster.json --cluster &quot;bsub -J &quot;{rule}.{wildcards}&quot; -q {cluster.queue} -n {threads} -M {resources.M} -R {cluster.one_host} -o {cluster.output} -e {cluster.error}&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;-J &quot;{rule}.{wildcards}&quot; -q {cluster.queue} -n {threads} -M {resources.M} -R {cluster.one_host}&lt;/code&gt; expands as expected. But &lt;code&gt;-o {cluster.output} -e {cluster.error}&lt;/code&gt; is replaced by literally &lt;code&gt;logs/cluster/{rule}.{wildcards}.out&lt;/code&gt; and &lt;code&gt;logs/cluster/{rule}.{wildcards}.err&lt;/code&gt;, i.e. there is no expansion in config file (the config file is attached bellow).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am puzzled since exactly this lines are in the documentation. What am I doing wrong? Why &lt;code&gt;snakemake&lt;/code&gt; is not expanding wildcards in the config file?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my &lt;code&gt;cluster.json&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;    &quot;__default__&quot; :&#xA;    {&#xA;        &quot;queue&quot;    : &quot;bgee&quot;,&#xA;        &quot;one_host&quot; : &quot;\&quot;span[hosts=1]\&quot;&quot;,&#xA;        &quot;name&quot;      : &quot;{rule}.{wildcards}&quot;,&#xA;        &quot;output&quot;    : &quot;logs/cluster/{rule}.{wildcards}.out&quot;,&#xA;        &quot;error&quot;     : &quot;logs/cluster/{rule}.{wildcards}.err&quot;&#xA;    },&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is relevant part of &lt;code&gt;Snakefile&lt;/code&gt; :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule index_reference :&#xA;        threads: 1&#xA;        resources: M=20000000&#xA;        input : &quot;data/{reference}/genome.fa.gz&quot;&#xA;        output : &quot;data/{reference}/genome.fa.gz.bwt&quot;&#xA;        shell : &quot;scripts/index_genome.sh {input}&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2017-11-23T09:33:36.523" LastActivityDate="2020-03-14T14:03:20.063" Title="How to specify resources for cluster in snakemake" Tags="|snakemake|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2914" PostTypeId="1" CreationDate="2017-11-22T17:47:23.990" Score="3" ViewCount="135" Body="&lt;p&gt;Are there any algorithms already in existence for reverse protein folding/docking (e.g. taking a desired surface map/docking profile and generating DNA which will produce a protein to fold into it?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Protein folding seems straightforward (though computationally intensive) and protein docking is likewise straightforward, but thus far I haven't come across any protein unfolding algorithms, though I might just be searching for the wrong keywords.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice in this area?&lt;/p&gt;&#xA;" OwnerUserId="1842" LastEditorUserId="48" LastEditDate="2017-11-22T21:44:19.187" LastActivityDate="2018-01-12T02:16:09.503" Title="What algorithms exist for reverse protein folding?" Tags="|docking|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2915" PostTypeId="2" ParentId="2913" CreationDate="2017-11-22T22:51:55.457" Score="5" Body="&lt;p&gt;This feature was added in version 3.6.0. Look at &lt;a href=&quot;http://snakemake.readthedocs.io/en/stable/project_info/history.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;release notes&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;{wildcards}&lt;/code&gt; normally expands to a something that looks vaguely like a dictionary. In other words, something like &lt;code&gt;key1=value1,key2=value2&lt;/code&gt;. Whether you really want to use that as a log file name is up to you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's use a very contrived example to demonstrate this. Here is the &lt;code&gt;Snakefile&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule all:&#xA;    input: &quot;input1.human.txt&quot;&#xA;&#xA;rule blah:&#xA;    output: &quot;{sample}.{reference}.txt&quot;&#xA;    shell: &quot;touch {output}&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm using your &lt;code&gt;cluster.json&lt;/code&gt;, so with that let's use a contrived cluster submission program:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;snakemake --cluster-config cluster.json --cluster &quot;echo 'rule.wildcards: {rule}.{wildcards} cluster: {cluster} -o {cluster.output} -e {cluster.error} -n {cluster.name}' &amp;gt;&amp;gt; log &amp;amp;&amp;amp; &quot; --jobs 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you run that and look in &lt;code&gt;log&lt;/code&gt;, you can see that &lt;code&gt;{rule}&lt;/code&gt; is expanded as expected, &lt;code&gt;{wildcards}&lt;/code&gt; is expanded to &lt;code&gt;sample=input1,reference=human&lt;/code&gt;. The &lt;code&gt;output&lt;/code&gt; and &lt;code&gt;error&lt;/code&gt; parts from &lt;code&gt;cluster.json&lt;/code&gt; are expanded correctly for me. Below are the contents of &lt;code&gt;log&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule.wildcards: blah.sample=input1,reference=human cluster: one_host=&quot;span[hosts=1]&quot;,output=logs/cluster/blah.sample=input1,reference=human.out,name=blah.sample=input1,reference=human,queue=bgee,error=logs/cluster/blah.sample=input1,reference=human.err -o logs/cluster/blah.sample=input1,reference=human.out -e logs/cluster/blah.sample=input1,reference=human.err -n blah.sample=input1,reference=human&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm not a big fan of using &lt;code&gt;{wildcards}&lt;/code&gt; in paths unless you're doing a parameter sweep, but to each their own. Note that the above results are from snakemake-4.3.1.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="57" LastEditDate="2017-11-23T09:32:34.083" LastActivityDate="2017-11-23T09:32:34.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2916" PostTypeId="2" ParentId="2913" CreationDate="2017-11-23T09:46:16.770" Score="3" Body="&lt;p&gt;The solutions are either, upgrade &lt;code&gt;snakemake&lt;/code&gt; or just take all wildcards out of &lt;code&gt;cluster.json&lt;/code&gt; file and write them directly to snakemake execution command :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;snakemake -p --jobs 10 --cluster-config cluster.json --cluster &quot;bsub -J &quot;{rule}&quot; -q {cluster.queue} -n {threads} -M {resources.M} -R {cluster.one_host} -o {&quot;logs/cluster/{rule}.{wildcards}.out&quot; -e &quot;logs/cluster/{rule}.{wildcards}.err&quot;&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I ended up instead of using of json config file in writing a simple bash wrapper &lt;code&gt;snakemake_cluster.sh&lt;/code&gt; that remembers the execution command for me in more readable format :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env bash&#xA;&#xA;snakemake $@ -p --jobs 10 --cluster &quot;bsub \&#xA;-J {rule} \&#xA;-q bgee \&#xA;-n {threads} \&#xA;-M {resources.M} \&#xA;-R \&quot;span[hosts=1]\&quot; \&#xA;-o logs/cluster/{rule}.{wildcards}.out \&#xA;-e logs/cluster/{rule}.{wildcards}.err&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Edit : All spaces after &lt;code&gt;--cluster&lt;/code&gt; command got to be escaped, otherwise it would add arguments to &lt;code&gt;snakemake&lt;/code&gt; not to &lt;code&gt;bsub&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit 2 : &lt;code&gt;$@&lt;/code&gt; means all arguments, therefore it can be either targets, flags or combinations (targets got to be specified first). So, &lt;code&gt;snakemake_cluster.sh my_file&lt;/code&gt; builds &lt;code&gt;my_file&lt;/code&gt;, &lt;code&gt;snakemake_cluster.sh --quiet&lt;/code&gt; builds default target quietly, and &lt;code&gt;snakemake_cluster.sh my_file --quiet&lt;/code&gt; builds &lt;code&gt;my_file&lt;/code&gt; quietly.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2017-11-23T11:57:16.147" LastActivityDate="2017-11-23T11:57:16.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2917" PostTypeId="2" ParentId="2897" CreationDate="2017-11-23T10:52:12.513" Score="2" Body="&lt;p&gt;When sequencing RNA from freshly harvested tissues under proper conditions, you should generally expect &gt; 50% mapped reads. In fact, everything &amp;lt; 80% would usually raise concerns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From your description (in the question and comments) it sounds as if your samples are potentially degraded and therefore saturated with short RNA fragments, either because you’ve got old tissue, you’re looking at environmental samples, or from improper handling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And since you’re not performing any enrichment, the sequencing data will then be highly saturated with degraded RNA, too. The reason for this is that RNA-seq is a (hopefully somewhat uniform) sampling of your sample aliquot: everything in it is proportionally represented in the RNA-seq data. That’s why enrichment steps are important, to increase the proportion of whatever fraction we’re interested in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Total RNA is mostly composed of genes with many copies: mostly rRNA and (much less) tRNA. Hence the importance of ribosome depletion or poly(A) selection.&lt;/li&gt;&#xA;&lt;li&gt;RNA in general is unstable. Depending on the sample collection (non-tissue sample origin? age of the tissue before harvesting?), a large fraction of the RNA may have degraded into minuscule fragments, which are further shortened by the library preparation. Size selection would get rid of these undesired fragments.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In my experience, the second issue is exacerbated on certain sequencers (I’ve seen it on an Illumina HiSeq 1500) because a mixed library of short and long RNA doesn’t seem to cycle well through the whole fragment length. So even if you have a (small but still present) fraction of long RNA fragments, the sequencer may not be able to synthesise their whole length efficiently. The effect is that there are proportionally more short reads than there were short fragments in the sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can verify this by calculating the insert size distribution of your reads after adapter trimming (e.g. using picard, or by simply tallying read lengths&lt;sup&gt;1&lt;/sup&gt;). Small reads (&amp;lt; ~14 nt) from degradation fragments are essentially unmappable, since they are too short to have a specific identity. Even if we allowed the mapper to align them their coordinates would be essentially random.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;1&lt;/sup&gt; This works:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk 'NR % 4 == 2 {c[length($0)]++} END {for (i in c) print i, c[i]}' in.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29" LastActivityDate="2017-11-23T10:52:12.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2918" PostTypeId="1" AcceptedAnswerId="2920" CreationDate="2017-11-23T11:09:24.217" Score="2" ViewCount="62" Body="&lt;p&gt;I'm using DRIMSeq to do some analysis but I'd like to make my own figures from the results ideally using what they've done as a base.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to find how they make their ggplot objects in the function &lt;code&gt;plotProportoins&lt;/code&gt; but when I use &lt;code&gt;findMethods('plotProportoins')&lt;/code&gt; it ends up with a method &lt;code&gt;dmPlotProportoins&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It says in the manual that it's a &lt;em&gt;ggplot&lt;/em&gt; object but I can't seem to find how they make it. Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="1843" LastEditorUserId="131" LastEditDate="2018-01-23T13:56:16.643" LastActivityDate="2018-01-23T13:56:16.643" Title="How can I find out the code DRIMSeq uses to make dmplotProportions objects?" Tags="|rna-seq|r|rna-splicing|ggplot2|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2919" PostTypeId="2" ParentId="2858" CreationDate="2017-11-23T12:01:27.103" Score="5" Body="&lt;pre&gt;&lt;code&gt;grep -v &quot;&amp;gt;&quot; multi.fasta | tr -d '\n' | wc -c&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;grep -v &quot;&amp;gt;&quot;&lt;/code&gt; find all non header lines (i.e. without &quot;&gt;&quot;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;tr -d '\n'&lt;/code&gt; delete all newlines&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;wc -c&lt;/code&gt; count all characters&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="57" LastActivityDate="2017-11-23T12:01:27.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2920" PostTypeId="2" ParentId="2918" CreationDate="2017-11-23T13:40:00.857" Score="4" Body="&lt;p&gt;I was curious how they have done it as well, so here expanded @Liopis comment :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wget http://bioconductor.org/packages/release/bioc/src/contrib/DRIMSeq_1.6.0.tar.gz&#xA;tar -zxf DRIMSeq_1.6.0.tar.gz &#xA;# if you need to find it, or use grep&#xA;# find DRIMSeq -name &quot;dm_plotProportions.R&quot;&#xA;less DRIMSeq/R/dm_plotProportions.R&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="57" LastActivityDate="2017-11-23T13:40:00.857" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2921" PostTypeId="1" AcceptedAnswerId="2922" CreationDate="2017-11-23T21:21:46.533" Score="6" ViewCount="316" Body="&lt;p&gt;So a poly-A tail is a long chain of adenine nucleotides that is added to a messenger RNA (mRNA) molecule during RNA processing to increase the stability of the molecule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my project, I would like to estimate the length of poly-A tails from randomly-primed RNAseq data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas how this could be done? Are there algorithms/tools which currently do this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps I would need to model this somehow? All suggestions appreciated&lt;/p&gt;&#xA;" OwnerUserId="146" LastEditorUserId="146" LastEditDate="2017-11-23T22:50:54.693" LastActivityDate="2017-12-01T17:27:18.087" Title="Estimate the length of poly-A tails from randomly-primed RNAseq data" Tags="|rna-seq|software-recommendation|rna|algorithms|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2922" PostTypeId="2" ParentId="2921" CreationDate="2017-11-23T22:12:06.313" Score="3" Body="&lt;p&gt;Just look for polyA tracts at the end of sequences, and count them if they're larger than ~18 bases. I've done this with MinION cDNA reads by mapping the polyA adapter sequence (with an elongated polyA sequence) to the reads using LAST, and working out the length of aligned sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, because the cDNA adapter finishes with a TVN sequence, rather than TTT, binding of the adapter is not necessarily a good representation of the true polyA length. I expect that Illumina adapters will have similar problems, if not worse (due to the read length limits).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might be possible to use direct-RNA sequencing to better categorise the polyA lengths, but I've yet to look at the public direct-RNA datasets. Just in case you're interested in looking yourself, here's a direct RNA dataset for NA12878 from the wonderful nanopore-WGS consortium:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/nanopore-wgs-consortium/NA12878/blob/master/RNA.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nanopore-wgs-consortium/NA12878/blob/master/RNA.md&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the consortium's graph from that summary document showing polyA length:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Sr1Fl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Sr1Fl.png&quot; alt=&quot;direct RNA polyA lengths&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-11-30T23:05:37.057" LastActivityDate="2017-11-30T23:05:37.057" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2923" PostTypeId="1" CreationDate="2017-11-23T23:53:29.477" Score="3" ViewCount="237" Body="&lt;p&gt;I am currently running canu on an LSF Linux server using the following script called assemble.sh:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;canu -p species -d output genomeSize=100m -pacbio-raw subreads.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then submit this script with the following in a computer node:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bsub -q basement -E 'test -e /nfs/users/nfs_c/cr' -R &amp;quot;select[mem&amp;gt;25000] rusage[mem=25000]&amp;quot; -M25000 -o canu.o -e canu.e -J canu 'bash assemble.sh' -n8 -R&amp;quot;span[hosts=1]&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;On my laptop, using 8 cores and 16 GB memory, assemble.sh took about 48h. On the cluster, it's been running for 72h and it is still running. I wonder if I am allocating resources correctly or if I can do something better. On my laptop, I can see in my activity monitor that canu is using many cores at the same time. Do you know how I can check this in an LSF server?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://canu.readthedocs.io/en/latest/faq.html#faq&quot; rel=&quot;nofollow noreferrer&quot;&gt;This page&lt;/a&gt; talks about submitting jobs on LSF. In particular, it mentions this: &lt;code&gt;gridOptions=&amp;quot;your options list&amp;quot;&lt;/code&gt; but I don't know how to use this option.&lt;/p&gt;&#xA;&lt;p&gt;Any help would be appreciated.&lt;/p&gt;&#xA;&lt;p&gt;checking the queue with &lt;code&gt;bjobs -u all&lt;/code&gt; shows there are 125259 jobs running. My own job is running, it's not pending. Does the number of other jobs affect the running of my own job? I will ask IT if there are preferred writing locations. I did not consider that could change the speed of the job. Do you know a command to check how many resources my job is using?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="73" LastEditDate="2021-07-04T04:42:19.523" LastActivityDate="2022-02-02T21:47:27.733" Title="How to submit a canu job on LSF high-performance computing cluster farm?" Tags="|assembly|linux|canu|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="2924" PostTypeId="1" AcceptedAnswerId="2935" CreationDate="2017-11-24T05:25:07.933" Score="7" ViewCount="2016" Body="&lt;p&gt;I have nearly 10 million SNPs located on 10 chromosomes. I want to divide the genome into non-overlapping windows of 15, 20 and 30 kb. Here is part of my SNP table:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;head (sap_ids)&#xA;       snp_id    chr  pos &#xA;    Chr01__15043   1 15043    &#xA;    Chr01__15079   1 15079   &#xA;    Chr01__15139   1 15139    &#xA;    Chr01__15165   1 15165    &#xA;    ...&#xA;    ...&#xA;    ...&#xA;    Chr17__214708424  17 214708424        &#xA;    Chr17__214708451  17 214708451        &#xA;    Chr17__214708484  17 214708484        &#xA;    Chr17__214708508  17 214708508        &#xA;    Chr17__214708574  17 214708574        &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have been using this code, but it gives me wrong output because each chromosome starts from position 1&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;win_size&amp;lt;-c(15000,30000,50000,100000)&#xA;res&amp;lt;- cbind(snp_ids,&#xA;       data.frame(lapply(setNames(win_size, paste(&quot;window&quot;,win_size, sep = &quot;_&quot;)),&#xA;       function(x)as.numeric(ceiling(snp_ids$pos/x)))))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For instance, as I you see in the example below in window 4 I get SNPs from chromosomes 1, 3 and 5:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;snp_id     chrom poistion   window&#xA;Chr01__58332   1 58332            4&#xA;Chr01__58335   1 58335            4&#xA;Chr01__58341   1 58341            4&#xA;Chr01__58450   1 58450            4&#xA;Chr01__58471   1 58471            4&#xA;Chr01__58530   1 58530            4&#xA;Chr01__58542   1 58542            4&#xA;Chr01__58641   1 58641            4&#xA;Chr03__45457   3 45457            4&#xA;Chr03__45604   3 45604            4&#xA;Chr04__56873   4 56873            4&#xA;Chr04__57387   4 57387            4&#xA;Chr04__57399   4 57399            4&#xA;Chr04__57528   4 57528            4&#xA;Chr04__58419   4 58419            4&#xA;Chr04__59670   4 59670            4&#xA;Chr04__59704   4 59704            4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I guess the solution would be to run the code above by looping through each chromosome I tried something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (i in 1:10){ &#xA;       res &amp;lt;- cbind(cats,&#xA;             data.frame(&#xA;               lapply(setNames(win_size, paste(&quot;window&quot;, win_size, sep = &quot;_&quot;)), function(w)&#xA;                 paste(cats$chr[cats$chr == i], ceiling(cats$pos/w), sep = &quot;_&quot;))&#xA;               ))&#xA;       }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But still not working appropriately (column 4 should be 1_1, for the chr 1)!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Chr01__912    1  912          2_1&#xA;Chr01__944    1  944          2_1&#xA;Chr01__1107   1  1107         2_1&#xA;Chr01__1118   1  1118         2_1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me to figure out how can I modify the code to create these non-overlapping windows properly?&lt;/p&gt;&#xA;" OwnerUserId="1749" LastEditorUserId="298" LastEditDate="2017-11-28T11:43:20.650" LastActivityDate="2017-11-28T11:43:20.650" Title="dividing genome into non-overlapping windows using R" Tags="|r|genome|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2925" PostTypeId="1" CreationDate="2017-11-24T12:35:52.727" Score="3" ViewCount="75" Body="&lt;p&gt;I'm working on the &lt;em&gt;C. elegans&lt;/em&gt; model organism, for which there exists an online biological information database: Wormbase.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm able to manually search for a transcript (say &quot;K06C4.12&quot;) and this gets me the following URL:&#xA;&lt;a href=&quot;http://www.wormbase.org/species/c_elegans/cds/K06C4.12&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.wormbase.org/species/c_elegans/cds/K06C4.12&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say I have this transcript ID: &quot;F08G2.2&quot;. I guess that I can form a valid similar URL, as follows:&#xA;&lt;a href=&quot;http://www.wormbase.org/species/c_elegans/cds/F08G2.2&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.wormbase.org/species/c_elegans/cds/F08G2.2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed, it works as expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I would like to go further and be able to download in an automated way what can be obtained by clicking on one of the &quot;download&quot; buttons in the &quot;Sequence&quot; section of the above pages. More specifically, I'm interested in the &quot;spliced + UTR&quot; and &quot;conceptual translation&quot; fasta sequences. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I use the DOM inspector in Firefox (Shift-Ctrl-C), I can locate things like the following:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-html prettyprint-override&quot;&gt;&lt;code&gt;&amp;lt;div class=&quot;button ui-corner-all generate-file-download sequence-download&quot;&amp;gt;&#xA;    &amp;lt;span class=&quot;ui-icon ui-icon-arrowthickstop-1-s&quot;&amp;gt;&amp;lt;/span&amp;gt;&#xA;    &amp;lt;span&amp;gt;download&amp;lt;/span&amp;gt;&#xA;    &amp;lt;div id=&quot;content&quot; class=&quot;ui-helper-hidden&quot;&amp;gt;&#xA;        &amp;gt;K06C4.12 spliced + UTR&#xA;        ATGGCCCCACCAAAGCCGTCTGCTAAAGGAGCAAAGAAAGCCGCCAAGACCGTCACCAAGCCAAAGGACGGAAAGAAGAG&#xA;        ACGTCATGCCCGTAAAGAATCCTACTCCGTGTACATCTACCGTGTGCTCAAGCAGGTTCATCCAGACACTGGAGTCTCTT&#xA;        CCAAGGCTATGTCCATCATGAACTCTTTCGTCAACGACGTCTTCGAGCGTATCGCTGCTGAGGCATCTCGTCTTGCTCAC&#xA;        TACAACAAGAGATCCACCATCTCTTCTCGCGAAATTCAAACTGCCGTCCGTTTGATTCTTCCAGGAGAGCTTGCCAAGCA&#xA;        CGCCGTCTCTGAGGGAACCAAGGCTGTCACCAAGTACACCTCTAGCAAGTAA&#xA;    &amp;lt;/div&amp;gt;&#xA;    &amp;lt;div id=&quot;filename&quot; class=&quot;ui-helper-hidden&quot;&amp;gt;K06C4.12conceptual translation.fasta&amp;lt;/div&amp;gt;&#xA;    &amp;lt;div id=&quot;content&quot; class=&quot;ui-helper-hidden&quot;&amp;gt;&#xA;        &amp;gt;conceptual translation&#xA;        MAPPKPSAKGAKKAAKTVTKPKDGKKRRHARKESYSVYIYRVLKQVHPDTGVSSKAMSIMNSFVNDVFERIAAEASRLAH&#xA;        YNKRSTISSREIQTAVRLILPGELAKHAVSEGTKAVTKYTSSK*&#xA;    &amp;lt;/div&amp;gt;&#xA;&amp;lt;/div&amp;gt;&#xA;&amp;lt;div class=&quot;toggle &quot;&amp;gt;&#xA;    &amp;lt;span style=&quot;float:left&quot; class=&quot;ui-icon ui-icon-triangle-1-s&quot;&amp;gt;&amp;lt;/span&amp;gt;&#xA;    view conceptual translation (123 aa)&#xA;&amp;lt;/div&amp;gt;&#xA;&amp;lt;div class=&quot;returned sequence-container&quot; style=&quot;display:block&quot;&amp;gt;&#xA;    &amp;gt;conceptual translation&#xA;    &amp;lt;br&amp;gt;&#xA;    MAPPKPSAKGAKKAAKTVTKPKDGKKRRHARKESYSVYIYRVLKQVHPDTGVSSKAMSIMNSFVNDVFERIAAEASRLAH&#xA;    &amp;lt;br&amp;gt;&#xA;    YNKRSTISSREIQTAVRLILPGELAKHAVSEGTKAVTKYTSSK*&#xA;&amp;lt;/div&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't find these if I simply download the page with &lt;code&gt;wget&lt;/code&gt; and search in the source code, so they must be dynamically generated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How could I proceed to get the sequences?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.wormbase.org/about/userguide/for_developers/API-REST&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation about Wormbases's REST API&lt;/a&gt; is not very detailed, so I don't understand how to form an URL that would give me the desired sequences.&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2017-11-24T14:12:06.163" Title="Automating sequence download from Wormbase" Tags="|python|public-databases|data-retrieval|api|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2926" PostTypeId="2" ParentId="2925" CreationDate="2017-11-24T12:35:52.727" Score="4" Body="&lt;p&gt;Further inspecting the source code of the downloaded page, I found the following:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-html prettyprint-override&quot;&gt;&lt;code&gt;&amp;lt;li id=&quot;nav-sequences&quot;&#xA;    class=&quot;module-load transcript&quot;&#xA;    wname=&quot;sequences&quot;&#xA;    href=&quot;/rest/widget/transcript/K06C4.12/sequences&quot;&amp;gt;&#xA;    &amp;lt;span class=&quot;ui-icon ui-icon-close module-close&quot; wname=&quot;sequences&quot;&#xA;    &amp;lt;/span&amp;gt;&#xA;    Sequences&#xA;&amp;lt;/li&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This suggested me to have a look at &lt;a href=&quot;http://www.wormbase.org/rest/widget/transcript/K06C4.12/sequences&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.wormbase.org/rest/widget/transcript/K06C4.12/sequences&lt;/a&gt; and I obtained a page containing the elements found in the DOM inspector.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can be parsed, for instance using the &lt;a href=&quot;https://www.crummy.com/software/BeautifulSoup/bs4/doc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;bs4&lt;/code&gt; python module&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;# To get the page&#xA;import requests&#xA;# To parse the page content&#xA;from bs4 import BeautifulSoup&#xA;# To get this into Biopython&#xA;from io import StringIO&#xA;from Bio import SeqIO&#xA;&#xA;rest_template = &quot;http://www.wormbase.org/rest/widget/transcript/{accession}/sequences&quot;&#xA;rest = rest_template.format(accession=&quot;K06C4.12&quot;)&#xA;# This did not work with &quot;html.parser&quot;&#xA;soup = BeautifulSoup(requests.get(rest).text, &quot;html5lib&quot;)&#xA;&#xA;def generate_fasta(soup):&#xA;    name = None&#xA;    for div in soup.find_all(&quot;div&quot;):&#xA;        if &quot;ui-helper-hidden&quot; in div[&quot;class&quot;]:&#xA;            if div[&quot;id&quot;] == &quot;content&quot;:&#xA;                fasta = SeqIO.read(StringIO(div.text), &quot;fasta&quot;)&#xA;                description_fields = fasta.description.split()&#xA;                if len(description_fields) == 4:&#xA;                    fasta.id = &quot;_&quot;.join(description_fields[:2])&#xA;                    if name is None:&#xA;                        name = fasta.name&#xA;                elif len(description_fields) == 2:&#xA;                    # This assumes that the &quot;conceptual_translation&quot;&#xA;                    # is after the &quot;unspliced + UTR&quot; or &quot;spliced + UTR&quot;&#xA;                    # Otherwise, one would need to pass&#xA;                    # the accession as function parameter&#xA;                    fasta.id = f&quot;{name}_translation&quot;&#xA;                else:&#xA;                    fasta.id = &quot;_&quot;.join(description_fields)&#xA;                yield (fasta.id, fasta)&#xA;&#xA;records_dict = dict(generate_fasta(soup))&#xA;print(records_dict['K06C4.12_spliced'])&#xA;print(records_dict['K06C4.12_translation'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ID: K06C4.12_spliced&#xA;Name: K06C4.12&#xA;Description: K06C4.12 spliced + UTR&#xA;Number of features: 0&#xA;Seq('ATGGCCCCACCAAAGCCGTCTGCTAAAGGAGCAAAGAAAGCCGCCAAGACCGTC...TAA', SingleLetterAlphabet())&#xA;ID: K06C4.12_translation&#xA;Name: conceptual&#xA;Description: conceptual translation&#xA;Number of features: 0&#xA;Seq('MAPPKPSAKGAKKAAKTVTKPKDGKKRRHARKESYSVYIYRVLKQVHPDTGVSS...SK*', SingleLetterAlphabet())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2017-11-24T14:12:06.163" LastActivityDate="2017-11-24T14:12:06.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2927" PostTypeId="1" CreationDate="2017-11-24T13:53:40.863" Score="3" ViewCount="77" Body="&lt;p&gt;I have a list of all SNPs (a list of RsIDs) that fall in the coding sequence of several thousand human genes, and I'm looking for a convenient method to come up with a list of known SNPs that alter an amino acid in those genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll be glad to hear your thoughts.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="298" LastEditDate="2017-11-24T14:52:21.697" LastActivityDate="2017-11-25T10:06:38.010" Title="Identifying relevant SNPs from a list" Tags="|database|gene|snp|sequence-analysis|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2928" PostTypeId="2" ParentId="2927" CreationDate="2017-11-24T14:02:17.320" Score="5" Body="&lt;p&gt;What you are looking for is SNP annotation. If you have the chromosome:position reference and alternate alleles for your SNPs of interest, it can be as simple as uploading them to the variant effect predictor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://grch37.ensembl.org/Homo_sapiens/Tools/VEP&quot; rel=&quot;noreferrer&quot;&gt;http://grch37.ensembl.org/Homo_sapiens/Tools/VEP&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will give you the predicted protein change and novelty of the variant with respect to known variation in large scale sequencing projects like the 1000 genomes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you cannot afford to upload data on the web, there are downloadable command line software like SNPeff, Variant effect predictor etc but it would require some more bioinformatics knowledge of converting your data to a VCF format and obtaining a human reference genome.&lt;/p&gt;&#xA;" OwnerUserId="601" LastActivityDate="2017-11-24T14:02:17.320" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2929" PostTypeId="1" CreationDate="2017-11-24T23:09:34.583" Score="2" ViewCount="744" Body="&lt;p&gt;I am using Ray to assemble a about 7.2 kb picornavirus genome. I am using k-mer of 55. Following is the exact code that I used.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mpiexec -n 1 Ray -k 55 -s 2-CA-SVA-2_S2_L001_R1_001.fastq_q30_trim_LR_15.fasta  -o rayd55tr&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This worked quite well as I got a &lt;code&gt;scaffolds.fasta&lt;/code&gt; file with two scaffolds one with 5730 bp and another with 1548 bp. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I am confused how I can join those scaffolds to get one linear genome. I joined them manually and blasted just to check and I got something like below :&lt;a href=&quot;https://i.stack.imgur.com/SVhyo.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SVhyo.jpg&quot; alt=&quot;Blast of the manually joined contigs&quot;&gt;&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any tool to join these two contigs together to have one complete linear genome? I have reference genome too if needed.&lt;/p&gt;&#xA;" OwnerUserId="1684" LastEditorUserId="48" LastEditDate="2017-11-25T10:09:47.090" LastActivityDate="2017-11-30T08:01:37.027" Title="Joining scaffolds to get full genome?" Tags="|ngs|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2930" PostTypeId="1" AcceptedAnswerId="2931" CreationDate="2017-11-24T23:31:11.137" Score="5" ViewCount="2561" Body="&lt;p&gt;Is there a way to estimate the size of a BAM file will have after mapping with BWA?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input file are two mates fastq files, compressed with gzip, each one about 70G.&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2017-11-24T23:46:34.837" LastActivityDate="2017-11-25T01:42:16.123" Title="Estimating BAM file from compressed fastq file size" Tags="|sam|fastq|bwa|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2931" PostTypeId="2" ParentId="2930" CreationDate="2017-11-25T01:42:16.123" Score="4" Body="&lt;p&gt;A BAM file should have a roughly similar size to a compressed FASTQ file, because they're both compressed files that contain similar information. BAM will be slightly larger, because it contains information about mapping and differences from the reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are concerned about space and have a reference sequence available, you can store mapped reads as CRAM format, which will save a lot more space.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-11-25T01:42:16.123" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2932" PostTypeId="2" ParentId="2684" CreationDate="2017-11-25T04:46:46.627" Score="0" Body="&lt;p&gt;In case it’s of any benefit to you, I’ll often check &lt;a href=&quot;http://software.broadinstitute.org/software/igv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IGV&lt;/a&gt; when looking at tracks with that kind of zoomage. Not that it zooms any better than the UCSC browser (although the bases are colour coded!) but you can see your aligned reads individually if you want, along with directional information etc etc. Loads of stuff. &lt;/p&gt;&#xA;" OwnerUserId="1849" LastActivityDate="2017-11-25T04:46:46.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2934" PostTypeId="2" ParentId="2927" CreationDate="2017-11-25T10:06:38.010" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://annovar.openbioinformatics.org/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ANNOVAR&lt;/a&gt; is another tool that will help you functionally annotate genetic variants. It will tell you if the variant is known in COSMIC or Clinvar databases, which AA change will occur, if it's a frameshift, synonymous, non-synonymous, etc...&lt;/p&gt;&#xA;" OwnerUserId="1810" LastActivityDate="2017-11-25T10:06:38.010" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2935" PostTypeId="2" ParentId="2924" CreationDate="2017-11-25T14:47:56.310" Score="7" Body="&lt;p&gt;The question is a bit confusing to me, at it's core I understand you want non-overlapping windows across chromosomes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to achieve is to use &lt;code&gt;GenomicRanges::tileGenome&lt;/code&gt; function, which needs the chromosome lengths as input, e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Biostrings)&#xA;mygenome &amp;lt;- readDNAStringSet(list.files(mypath,&quot;mygenome.fa$&quot;,full=TRUE))&#xA;chrSizes &amp;lt;- width(mygenome)&#xA;names(chrSizes) &amp;lt;- names(mygenome)&#xA;print(chrSizes)&#xA;#     Chr1     Chr2     Chr3     Chr4     Chr5 &#xA;# 18585056 19698289 23459830 26975502 30427671 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This can than be fed into tileGenomes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(GenomicRanges)&#xA;bins   &amp;lt;- tileGenome(chrSizes, tilewidth=5e5, cut.last.tile.in.chrom=T)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This creates a &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/GenomicRanges.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GRange&lt;/a&gt; object which gives you a whole repertoire of post processing options, i.e. overlapping with SNPs or NGS reads (I use the latter for genome wide coverage plots):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print(bins)&#xA;# GRanges object with 240 ranges and 0 metadata columns:&#xA;#         seqnames               ranges strand&#xA;#            &amp;lt;Rle&amp;gt;            &amp;lt;IRanges&amp;gt;  &amp;lt;Rle&amp;gt;&#xA;#     [1]     Chr1   [      1,  500000]      *&#xA;#     [2]     Chr1   [ 500001, 1000000]      *&#xA;#     [3]     Chr1   [1000001, 1500000]      *&#xA;#     [4]     Chr1   [1500001, 2000000]      *&#xA;#     [5]     Chr1   [2000001, 2500000]      *&#xA;#     ...      ...                  ...    ...&#xA;#   [236]     Chr5 [28000001, 28500000]      *&#xA;#   [237]     Chr5 [28500001, 29000000]      *&#xA;#   [238]     Chr5 [29000001, 29500000]      *&#xA;#   [239]     Chr5 [29500001, 30000000]      *&#xA;#   [240]     Chr5 [30000001, 30427671]      *&#xA;#   -------&#xA;#   seqinfo: 5 sequences from an unspecified genome&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1461" LastEditorUserId="1461" LastEditDate="2017-11-26T08:43:10.093" LastActivityDate="2017-11-26T08:43:10.093" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2936" PostTypeId="1" CreationDate="2017-11-25T21:50:26.807" Score="1" ViewCount="503" Body="&lt;p&gt;I need a gtf annotation file. UCSC doesn't give us a proper gtf file with distinct gene_id and transcript_id. It asked us to get a genePred file to convert to gtf.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But where can we get genePred files?&lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2017-11-26T14:47:38.537" Title="How can I download genePred file from UCSC?" Tags="|ucsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2937" PostTypeId="2" ParentId="2936" CreationDate="2017-11-26T00:40:24.040" Score="1" Body="&lt;p&gt;It turned out that using table browser on UCSC we could download an output with all fields. That output actually contains a GenePred file but has one extra column on the left end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The genePred file format definition is &lt;a href=&quot;http://genome.ucsc.edu/FAQ/FAQformat.html#format9&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1852" LastEditorUserId="48" LastEditDate="2017-11-26T14:47:38.537" LastActivityDate="2017-11-26T14:47:38.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2938" PostTypeId="2" ParentId="2903" CreationDate="2017-11-26T00:55:55.560" Score="1" Body="&lt;p&gt;The epithelial-mesenchymal is a process by which cells lose some of the structures specific to their cell type. This is normal during embryonic development or wound healing for the formation of new cells. However, it is very uncommon in healthy adults and is thereby associated with the cancers in which it occurs. EMT allows cancer cells to become more like stem cells and could facilitate metastasis or immortal cell growth, key processes for cancer progression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can detect the molecular components involved in EMT, you may be able to make some inferences about the progression of the patient’s disease. This process is specific to cancers in epithelial tissues (breast, stomach, and skin cancers for example). It is just one of vastly many pathways by which cancers can grow and develop. Many of these are specific to different cancers types by tissue or molecular subtype. Cancer research is therefore very challenging and requires a deep understanding of the particular system you are focusing on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gene expression data sets are a great resource for this. You can compare many pathways to clinical properties from previous studies. Microarray datasets such as ArrayExpress and Gene Expression Omnibus (GEO) were very popular for this but are now largely out of date. Many databases now support higher quality RNASeq datasets with larger sample sizes and fewer batch effects. The Cancer Genome Atlas (TCGA) and International Cancer Genome Consortium (ICGC) have among the largest Cancer databases. These are a tool for researchers to investigate any genes or pathways. They will not specifically address EMT, unless you focus on the genes involved in a relevant cancer type.&lt;/p&gt;&#xA;" OwnerUserId="1855" LastActivityDate="2017-11-26T00:55:55.560" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2939" PostTypeId="2" ParentId="2785" CreationDate="2017-11-26T03:26:46.203" Score="1" Body="&lt;p&gt;Negative values are to be expected as you have log-transformed the data. Any data value below 2 will have a negative log2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a difference between normalising for analysis and truncating for plotting. The RMA normalisation or similar is intended for differential expression analysis. I would recommend using this normalisation first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When plotting, the normalised data may still contain outliers which skew the plotting parameters (such as the colour scale). Due to the large number of genes, there may be some far beyond 3 standard deviations and plotting these at their true value will make it hard to discern trends in the middle. Truncating the data for a heatmap is common, typically at |z| &gt; 3 but quantiles should work much the same. This is a matter of visualisation, not clustering analysis. It would not be needed unless you were plotting the data.&lt;/p&gt;&#xA;" OwnerUserId="1855" LastActivityDate="2017-11-26T03:26:46.203" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2940" PostTypeId="1" CreationDate="2017-11-26T10:49:41.467" Score="4" ViewCount="336" Body="&lt;p&gt;I'm trying to annotate a genome to find all genes with a specific function. I have a FASTA and the read FASTQs - I'd like to assign the functional group of the identified proteins (e.g. Kegg orthology) automatically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more context, I have whole genome sequences of bacteria - I'm trying to find all genes involved in membrane transport. I'd either like a tool to retrieve gene ontologies from a list of gene names (input: genA output: Membrane Transport), or a tool that outputs the function as part of the annotation pipeline (e.g. genbank output with GO function defined). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or posed in a different way, how would you identify all genes with a specific function from prokaryotic whole genome sequence data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any scripts that can do this?&lt;/p&gt;&#xA;" OwnerUserId="1857" LastEditorUserId="96" LastEditDate="2019-07-11T13:46:42.103" LastActivityDate="2019-07-11T13:46:42.103" Title="Gene function annotation - bacterial genome" Tags="|bacteria|sequence-annotation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2941" PostTypeId="2" ParentId="2940" CreationDate="2017-11-27T02:59:40.593" Score="4" Body="&lt;p&gt;The PATRICBrc.org is a free resource you can annotate your genome and compare it with publicly available genomes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Refer this &lt;a href=&quot;http://docs.patricbrc.org/tutorial/genome_annotation/annotation.html&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;tutorial&quot;&gt;tutorial&lt;/a&gt; for more detail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have annotated your genome in PATRIC, you will have PATRIC Local families (k-mer based protein family), which will help you identify all genes with a specific function.&lt;/p&gt;&#xA;" OwnerUserId="1859" LastEditorUserId="964" LastEditDate="2017-11-28T04:41:22.757" LastActivityDate="2017-11-28T04:41:22.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2942" PostTypeId="1" AcceptedAnswerId="2949" CreationDate="2017-11-27T07:16:03.753" Score="3" ViewCount="140" Body="&lt;p&gt;I am assembling a ~500MB genome, and have ~150x long reads and ~200x 150bp PE short reads, with a ~400bp insert size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've done a lot of work with minimap+miniasm, and have what I think is a good set of unitigs and a .gfa file from miniasm. Depending on the parameters, I have a few hundred to ~1K unitigs. I can easily polish these unitigs with pilon, racon, and nanopolish, so that they are as error free as possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question I have is: given access to&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Polished (or not) unitigs&lt;/li&gt;&#xA;&lt;li&gt;150x long reads (long enough to span almost all repeats in the genome)&lt;/li&gt;&#xA;&lt;li&gt;200x 150bp PE short reads (~400bp insert size)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What would you do to scaffold a genome? And what are the pros and cons of your suggested approach?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I need here is a draft genome. I would prefer a conservative approach that is more fragmented and has fewer misassemblies. So of course one option is just to keep the unitigs and make no attempt at scaffolding. But it does strike me there may be suitably conservative approaches I haven't considered. E.g. perhaps I could use ABYSS to scaffold my polished unitigs using my short-read data, thereby leveraging whatever information I can from the PE short reads?&lt;/p&gt;&#xA;" OwnerUserId="156" LastActivityDate="2017-11-27T18:08:37.770" Title="Scaffolding a genome with hybrid data" Tags="|assembly|long-reads|illumina|scaffold|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2943" PostTypeId="1" CreationDate="2017-11-27T10:24:14.587" Score="5" ViewCount="2145" Body="&lt;p&gt;I found some Nanopore MinION data on SRA, which I would like to investigate. I use &lt;code&gt;sratoolkit&lt;/code&gt; for Illumina data all the time, but I am not sure how to get the &lt;code&gt;fast5&lt;/code&gt; file from the &lt;code&gt;sra&lt;/code&gt; file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I have tried is to &lt;code&gt;prefecth&lt;/code&gt; it, but then I don't know which command to use for &lt;code&gt;.sra&lt;/code&gt; to &lt;code&gt;.fast5&lt;/code&gt; conversion. Not with fastq-dump, I assume?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;prefetch -v SRR5286963 --max-size 50000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2017-11-27T14:47:37.663" LastActivityDate="2017-11-27T18:02:06.110" Title="How to get Nanopore MinION fast5 from SRA" Tags="|nanopore|fast5|sratoolkit|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2944" PostTypeId="2" ParentId="2929" CreationDate="2017-11-27T11:49:21.643" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://fenderglass.github.io/Ragout/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ragout&lt;/a&gt; worked well for me. It is a reference-based scaffolder.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ragout.py --outdir output/ input/ragout.rcp &amp;gt; message.txt 2&amp;gt; error.txt --threads 8 --overwrite&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The ragout.rcp file contains a 'recipe' with the reference sequence and the contigs in FASTA format.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;.references = ensembl&#xA;.target = assembly&#xA;&#xA;ensembl.fasta = ref.fa&#xA;assembly.fasta = contigs.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In my case, it reduced the number of contigs 10-fold and increased the N50 10-fold. It added N's between the contigs.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="48" LastEditDate="2017-11-27T15:43:51.997" LastActivityDate="2017-11-27T15:43:51.997" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2945" PostTypeId="2" ParentId="2943" CreationDate="2017-11-27T15:11:30.387" Score="2" Body="&lt;p&gt;I would suggest trying an alternative way, as the FAST5 is a fairly new entry in SRA - 6 public datasets as of &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra/?term=fast5&quot; rel=&quot;nofollow noreferrer&quot;&gt;today&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can search the ERX/ERR entry you got from SRA in &lt;a href=&quot;https://www.ebi.ac.uk/ena&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENA&lt;/a&gt; and either (1) direct download it via ftp or (2) redirect to Galaxy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a &lt;a href=&quot;https://www.bioconductor.org/packages/devel/data/experiment/vignettes/minionSummaryData/inst/doc/creating-Styphi-Dataset.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; that might be interesting for you: basically, you first get the archive from ENA, then find the FAST5 files in the top level of the downloaded archive, all to be processed in R using &lt;code&gt;Fast5Summary&lt;/code&gt; objects from &lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/IONiseR/inst/doc/IONiseR.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;IONiseR&lt;/a&gt; package.&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2017-11-27T15:11:30.387" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2947" PostTypeId="2" ParentId="2940" CreationDate="2017-11-27T16:30:28.680" Score="1" Body="&lt;p&gt;Im not 100% on what you are looking for - you seem to jump from annotation to detection of genes in your question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you have a database of genes you want to detect (presence/ absence) you could try &lt;a href=&quot;https://github.com/katholt/srst2&quot; rel=&quot;nofollow noreferrer&quot;&gt;srst2&lt;/a&gt;&#xA;which basically takes your raw reads (fastq) and maps them to the database.  it will give you a report on any hits, and any snps/ indels in the hit. The only problem is making you own database is difficult as you must adhere to the &lt;a href=&quot;https://github.com/katholt/srst2#generating-srst2-compatible-clustered-database-from-raw-sequences&quot; rel=&quot;nofollow noreferrer&quot;&gt;fasta header format&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="982" LastActivityDate="2017-11-27T16:30:28.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2948" PostTypeId="2" ParentId="2943" CreationDate="2017-11-27T16:54:33.063" Score="3" Body="&lt;p&gt;When I was trying to upload MinION data to SRA, they wouldn't allow me to upload only the raw signal data (i.e. FAST5 files). Every file needed to contain a FASTQ sequence, and this sequence was extracted from the file and stored in the archive. Given this, I don't expect it will be possible to retrieve signal or event information from SRA. I also don't expect this will be possible in the future either, given that the SRA database expects every record to have a FASTQ-like sequence attached.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, EBI/ENA &lt;em&gt;do&lt;/em&gt; allow the upload of raw FAST5 files to the database (e.g. &lt;a href=&quot;https://www.ebi.ac.uk/ena/data/view/PRJEB20824&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), but there are no FAST5 reads for the specific single-cell B1a study you have indicated. If you're interested specifically in that study, then I recommend that you ask Miten Jain directly for access to those reads; he should have another method to give you access to them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ENA is &quot;ready&quot; for ONT reads, but researchers need to upload their reads to ENA in order for that database to be useful.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-11-27T18:02:06.110" LastActivityDate="2017-11-27T18:02:06.110" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2949" PostTypeId="2" ParentId="2942" CreationDate="2017-11-27T18:08:37.770" Score="2" Body="&lt;p&gt;Try &lt;a href=&quot;http://www.fishbrowser.org/software/LR_Gapcloser/index.php/Home/Index/Downloads.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;LR_Gapcloser&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've used L_RNA_Scaffolder for trying to scaffold a genome (which turned out to be more complex than I had expected). It looks like LR_Gapcloser (written by the same people) is similar, but designed specifically for scaffolding using long-reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That page also suggests &lt;a href=&quot;https://sourceforge.net/projects/pb-jelly/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PBJelly&lt;/a&gt; and &lt;a href=&quot;https://sourceforge.net/projects/gmcloser/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GMCloser&lt;/a&gt; as competing tools.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-11-27T18:08:37.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2950" PostTypeId="1" AcceptedAnswerId="2951" CreationDate="2017-11-27T23:07:22.403" Score="1" ViewCount="81" Body="&lt;p&gt;I was wondering if we can perform differential expression on samples from different library and read lengths. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What we intend to do is, &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;n = 6 tumors (recently sequenced using Hiseq 2500 read length 150, paired end)&lt;/li&gt;&#xA;&lt;li&gt;n = 5 normals (previously sequenced using Hiseq read length 100, paired end, these are not matched normal)&lt;/li&gt;&#xA;&lt;li&gt;n = 100 ( normals, from tcga read length 100, paired)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What we would like to do is compare n=6 Vs 100 + 5 normals. What steps should we be considering before we move ahead with this analysis?&lt;/p&gt;&#xA;" OwnerUserId="926" LastEditorUserId="48" LastEditDate="2017-11-28T08:03:57.643" LastActivityDate="2017-11-29T00:10:47.307" Title="differential expression multiple studies" Tags="|rna-seq|differential-expression|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2951" PostTypeId="2" ParentId="2950" CreationDate="2017-11-28T02:30:36.287" Score="2" Body="&lt;p&gt;Adding experiments 1 &amp;amp; 2 should be fine without modification on the computer side of things, as long as your sample preparation is consistent. To be ultra-cautious, you could add in the experiment as an additional covariate for the differential expression model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm less confident about adding in TCGA data; it's probably going to look better than your own samples, which might cause problems in calculating differential expression from raw reads. You can do a separate normalisation/correction of the TCGA data, but that will limit the inferences that can be made about differential expression -- without raw count information, the shot noise associated with read sampling cannot be included in the statistical model. I notice that TCGA has already done their own filtering / expression, and that would be preferable to use (mRNA sequencing, data subtype 'Gene'):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://tcga-data.nci.nih.gov/docs/publications/tcga/datatype.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://tcga-data.nci.nih.gov/docs/publications/tcga/datatype.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, even with data that has been properly normalised and filtered to exclude batch effects, there will still be population differences that influence results. My recommendation would be to do some subsampling to reduce the population effects, something like at least the following comparisons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;6 Vs 100 + 5 normals&lt;/li&gt;&#xA;&lt;li&gt;6 Vs 100 normals&lt;/li&gt;&#xA;&lt;li&gt;6 Vs 50 + 3 normals [randomly sampled from the two normal groups]&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;And only consider differential expression that is consistently observed in all three of these comparisons. Note that the third &quot;subsampled&quot; dataset can be generated more than once, taking different subgroups each time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are going to use separately normalised data, then DESeq2 may not be appropiate (because it expects count-level data). Something like &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/limma.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Limma&lt;/a&gt; could work for DE analysis, see chapter 15 of the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/limma/inst/doc/usersguide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;user's guide&lt;/a&gt; for RNASeq analysis. Unfortunately I'm not very familiar with using Limma, so can't provide any advice on things like what to watch out for in the input data and/or results.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-11-29T00:10:47.307" LastActivityDate="2017-11-29T00:10:47.307" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2953" PostTypeId="1" CreationDate="2017-11-28T20:15:25.433" Score="2" ViewCount="201" Body="&lt;p&gt;I am trying to reproduce locally on my computer what I get running mirbase on their website using BLAST. The search sequences is: mature miRNAs which I had downloaded on my computer and make it as a BLAST database with command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./makeblastdb -in /home/marianoavino/Downloads/mature.fa -dbtype 'nucl' -out /home/marianoavino/Downloads/mature&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then on mirbase I see they use an e-value of 10, which I leave locally.&#xA;On mirbase at the end of the analysis they give you these parameter setting:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Search parameters&#xA;&#xA;Search algorithm:   BLASTN   &#xA;Sequence database: mature  &#xA;Evalue cutoff: 10  &#xA;Max alignments: 100  &#xA;Word size: 4  &#xA;Match score: +5  &#xA;Mismatch penalty: -4  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and this is the command line I use on my computer for BLAST&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./blastn -db /home/marianoavino/Downloads/mature -evalue 10 -word_size 4 -query /home/marianoavino/Downloads/testinputblast.fasta -task &quot;blastn&quot; -out /home/marianoavino/Downloads/testBLast.out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The results of the two analysis are different, with mirbase finding much more stuff than local BLAST.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any idea on which parameters I should use on local blast command line to match those listed mirbase parameters in order to get the same answer?&lt;/p&gt;&#xA;" OwnerUserId="1829" LastEditorUserId="48" LastEditDate="2017-11-29T08:00:33.447" LastActivityDate="2017-11-29T08:00:33.447" Title="Mirbase and blast locally" Tags="|blast|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="2954" PostTypeId="1" CreationDate="2017-11-29T03:50:17.127" Score="2" ViewCount="113" Body="&lt;p&gt;I know there are existing GO slims predefined. But does anyone know how I can create my own collection of GO terms for enrichment analysis?&lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2017-11-29T03:50:17.127" Title="How can I create my own GO slim?" Tags="|gene|gene-ontology|ontology|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2955" PostTypeId="1" AcceptedAnswerId="2956" CreationDate="2017-11-28T21:50:07.647" Score="6" ViewCount="2827" Body="&lt;p&gt;I'm currently trying to run bwa-mem on Influenza substrains using the following command: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;~/bwa mem h5n1_1_cons.fa h5n1_1_read1.fq h5n1_1_read2.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;h5n1_1_cons.fa is the consensus sequence for substrain h5n1_1, and the fq files are paired end reads 1 and 2. Running the program gives me this error: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[mem_sam_pe] paired reads have different names: &quot;JQ714243-920-1&quot;, &quot;JQ714243-920-2&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Visual inspection shows that the read names are identical until the last character. I have found a package (bbmap) with a utility, bbrename.sh, that renames the sequences, producing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./bbrename.sh in=h5n1_1_read1.fq out=fixed prefix=h5n1_1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which yields the following output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-bash-4.2$ head fixh5n1_1_read1.fq&#xA;@h5n1_0&#xA;gaaaatgtctcaggagttgatgcatggtgtttaatctttggtttgcgctg&#xA;+&#xA;@4&amp;gt;C5?7FA?9@@@;1,-4B+E??4E9,DA&amp;gt;6CB)*2.)?7/.'65+75/&#xA;@h5n1_1&#xA;catcagttggattttgcctaaggatgtccaccatccttatcccaccaatt&#xA;+&#xA;7(9?)=C3AEF+@@F=C-E&amp;gt;EE57CD:;'?.*?7;&amp;gt;:3&amp;gt;2.;29..)=2'&#xA;@h5n1_2&#xA;catgcatttgttggaataatgttctcacaaatcaactgtattgaccgctg&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This fixes the problem and allows BWA-MEM to work. However, header information is discarded. Is there another way to ensure the header names are the same while retaining all header information?&lt;/p&gt;&#xA;" OwnerUserId="2008" OwnerDisplayName="sgav" LastEditorUserId="48" LastEditDate="2017-11-29T15:29:26.227" LastActivityDate="2017-11-29T15:29:26.227" Title="Is there a more elegant solution to the bwa-mem: paired reads have different names error?" Tags="|bwa|errors|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2956" PostTypeId="2" ParentId="2955" CreationDate="2017-11-29T11:15:07.257" Score="7" Body="&lt;p&gt;Assuming the read name lines only contain things like &lt;code&gt;@JQ714243-920-1&lt;/code&gt; and nothing else, then &lt;code&gt;sed -i '1~4 s/-[12]$//' file.fastq&lt;/code&gt; (N.B., example updated a bit for safety, thanks to @terdon) will remove the &lt;code&gt;-1&lt;/code&gt; and &lt;code&gt;-2&lt;/code&gt; suffixes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you should try to figure out &lt;strong&gt;why&lt;/strong&gt; the read names were munged into that form to begin with (at least Illumina machines won't produce that, I don't know about others).&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-11-29T12:22:52.360" LastActivityDate="2017-11-29T12:22:52.360" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2957" PostTypeId="2" ParentId="2929" CreationDate="2017-11-30T08:01:37.027" Score="0" Body="&lt;p&gt;In some cases you might consider reference-based scaffolding, as per &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2944/196&quot;&gt;charlesdarwin's answer&lt;/a&gt;. I would argue that in most cases this will only provide limited information, as this will only really provide you with the order of the contigs &lt;em&gt;as it is in your reference&lt;/em&gt;, as opposed to how it is in your sampled organism. In that sense it really only provides pseudo-scaffolding that makes your assembly look artificially good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In all cases you will have to check wether your raw data supports your scaffolding solution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example: you will never be able to call structural variants across contig boundaries if you use reference based scaffolding. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest you try different k-mer sized for your assembly algorithm, and look into dedicated scaffolding tools. Without knowing your type of sequencing data it is hard to suggest anything specific.&lt;/p&gt;&#xA;" OwnerUserId="196" LastActivityDate="2017-11-30T08:01:37.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2958" PostTypeId="2" ParentId="2908" CreationDate="2017-11-30T08:07:50.947" Score="2" Body="&lt;p&gt;In addition to the answer by &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2910/196&quot;&gt;heathobrien&lt;/a&gt; you might take a look at &lt;a href=&quot;http://ratt.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RATT: Rapid Annotation Transfer Tool&lt;/a&gt;. The approach is essentially the same, only using more modern approaches to for instance whole genome alignments. It uses &lt;a href=&quot;https://github.com/mummer4/mummer&quot; rel=&quot;nofollow noreferrer&quot;&gt;nucmer&lt;/a&gt;, which when using the latest version (4.0) allows you to do whole genome alignments using parallel computing.&lt;/p&gt;&#xA;" OwnerUserId="196" LastActivityDate="2017-11-30T08:07:50.947" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2959" PostTypeId="1" AcceptedAnswerId="2961" CreationDate="2017-11-30T10:09:02.067" Score="9" ViewCount="1568" Body="&lt;p&gt;I have Ensembl ids in the first column and samples with RSEM counts data in other columns. I converted Ensembl ids to gene symbols. Now I see there are three genes repeated twice.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Gene   S1       S2       S3&#xA;COG8  804.07  1475.16  323.80&#xA;COG8  249.23   867.94  113.64&#xA;SDHD 1464.84  3508.02    75&#xA;SDHD  4.16     3.97       0&#xA;SOGA3 81.65   30.76    32.69&#xA;SOGA3 0.00    10.77    25.65&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These are the Ensembl ids:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;SOGA3: ENSG00000214338, ENSG00000255330&lt;/li&gt;&#xA;&lt;li&gt;SDHD: ENSG00000204370, ENSG00000255292&lt;/li&gt;&#xA;&lt;li&gt;COG8: ENSG00000213380, ENSG00000272617.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is not for DE analysis, I want to do subtyping. Should I calculate mean or median on that? How should I go forward with this duplicate genes?&lt;/p&gt;&#xA;" OwnerUserId="1377" LastEditorUserId="298" LastEditDate="2017-11-30T12:18:41.477" LastActivityDate="2017-11-30T12:18:41.477" Title="Duplicate genes with RSEM counts: Which one to choose?" Tags="|r|rna-seq|bioconductor|rsem|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2960" PostTypeId="2" ParentId="2959" CreationDate="2017-11-30T10:54:33.210" Score="5" Body="&lt;p&gt;There is no one-to-one mapping of gene ids from one database to the other. Ensembl (who maintain Ensembl ENSG IDs), ncbi (who maintain EntrezGene IDs and RefSeq transcript ids) and HUGO (who maintain gene symbols/names) all have different ideas about what a gene is, which part of the genome belongs to which gene etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, it's worse than that because HUGO gene IDs refer to the abstract idea of a gene, rather than a particular part of any chromosome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My suggestion would be to leave the IDs as Ensembl IDs (as these actually refer to a physical part of a chromosome, which is what you are quantifying), right up until you have to report the results to someone who really NEEDS gene names. If you still have duplicates, you could sum, or pick the most interesting, but I would suggest just leaving the duplicates in there, and treat it as thought there are two genes with the same name. &lt;/p&gt;&#xA;" OwnerUserId="235" LastEditorUserId="298" LastEditDate="2017-11-30T11:31:54.137" LastActivityDate="2017-11-30T11:31:54.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2961" PostTypeId="2" ParentId="2959" CreationDate="2017-11-30T12:08:54.530" Score="5" Body="&lt;p&gt;Let's look into this a bit more deeply. For instance:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;HUGO: &lt;a href=&quot;https://www.genenames.org/cgi-bin/gene_symbol_report?hgnc_id=HGNC:21494&quot; rel=&quot;nofollow noreferrer&quot;&gt;SOGA3&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Ensembl 1: &lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Gene/Compara_Paralog?db=core;g=ENSG00000214338;r=6:127472794-127519191&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENSG00000214338&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Ensembl 2: &lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Gene/Compara_Paralog?db=core;g=ENSG00000255330;r=6:127438406-127519001&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENSG00000255330&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The Ensembl pages (linked above) for both ENSG00000214338 and ENSG00000255330 show that these genes have 3 &lt;a href=&quot;https://biology.stackexchange.com/a/4964/1306&quot;&gt;paralogs&lt;/a&gt; and each lists the other as one of the paralogs. Your other two examples are similar, they are also paralogs of each other. You can also see that the position of each gene in the genome is slightly different:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ENSG00000255330 : &lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Location/View?db=core;g=ENSG00000255330;r=6:127438406-127519001&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chromosome 6: 127,438,406-127,519,001&lt;/a&gt; reverse strand.&#xA;ENSG00000214338 : &lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Location/View?db=core;g=ENSG00000214338;r=6:127472794-127519191&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chromosome 6: 127,472,794-127,519,191&lt;/a&gt; reverse strand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So these two genes refer to two distinct regions of the genome. And, as expected, they have different transcripts. However, two of those transcripts (&lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Transcript/ProteinSummary?db=core;g=ENSG00000255330;r=6:127438406-127519001;t=ENST00000481848&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENST00000481848.6&lt;/a&gt; from ENSG00000255330 and &lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Transcript/ProteinSummary?db=core;g=ENSG00000214338;r=6:127472794-127519191;t=ENST00000525778&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENST00000525778.5&lt;/a&gt; from ENSG00000214338) code for the same protein sequence. This means that we have &lt;em&gt;one&lt;/em&gt; protein but &lt;em&gt;two&lt;/em&gt; different genes for it. To illustrate, here are the two transcripts aligned:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ENST00000525778.5       CCGTGTTCGAGCGGTTTTGGGATCCTCTGCCAAGTACGGGATAAGAGAAT&#xA;ENST00000481848.6       CTC-----------------------------------------------&#xA;                        *                                                 &#xA;&#xA;ENST00000525778.5       AGAGCAAGGAGTTTGAGCGCTGCTGCCAATCTTCCATCTCGGGCGTGGCT&#xA;ENST00000481848.6       --------------------------------------------------&#xA;&#xA;&#xA;ENST00000525778.5       CTGGCCTTTTTGTCTCTTTATCCCGCCACTCCCCACCCCCGTCCCCCGTC&#xA;ENST00000481848.6       --------------------------------------------------&#xA;&#xA;&#xA;ENST00000525778.5       CCCCTCCCCGCCTTCTTCTCTTTCCTCTTGAAAGTAACCTCTCTTTGGTT&#xA;ENST00000481848.6       -------------------------------------------TTTGGTT&#xA;                                                                   *******&#xA;&#xA;ENST00000525778.5       ACTTCTACCCTATCGTCCCTTGCAATCCCGCCTTCTCTCCTGCCCAGTTC&#xA;ENST00000481848.6       ACTTCTACCCTATCGTCCCTTGCAATCCCGCCTTCTCTCCTGCCCAGTTC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GGTGAGGTCAGCTCATGAACATCTCTGCACTTCCTGGGGAGCGTCTGGCC&#xA;ENST00000481848.6       GGTGAGGTCAGCTCATGAACATCTCTGCACTTCCTGGGGAGCGTCTGGCC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CAGCTTGGTACCTCGCGTGTTAGCCTGCTGAGAGCTGCAGCTCCAGCCCA&#xA;ENST00000481848.6       CAGCTTGGTACCTCGCGTGTTAGCCTGCTGAGAGCTGCAGCTCCAGCCCA&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GCCGCGGACCGCCGGGAAGTCACCCGCTCTGGGGGCAGCTGAATCTTGAC&#xA;ENST00000481848.6       GCCGCGGACCGCCGGGAAGTCACCCGCTCTGGGGGCAGCTGAATCTTGAC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GTTACCGCCATTGTTTGGAGAAATCGAGTGGGTGTGGAACTGCGGCGAGC&#xA;ENST00000481848.6       GTTACCGCCATTGTTTGGAGAAATCGAGTGGGTGTGGAACTGCGGCGAGC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GAGCCCAAAGTCATGCTCAGCCGGAAAGTCGCATACAATAATATGAAGTT&#xA;ENST00000481848.6       GAGCCCAAAGTCATGCTCAGCCGGAAAGTCGCATACAATAATATGAAGTT&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GAACTTTCCTCGGCGCAGAAAGGCACAATAATAAACCCTGAGATCCCCGG&#xA;ENST00000481848.6       GAACTTTCCTCGGCGCAGAAAGGCACAATAATAAACCCTGAGATCCCCGG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       AATATCAGAAGGAAAGTGCTGTTGATTTGTTCAGCTACTTGGAAACCTCG&#xA;ENST00000481848.6       AATATCAGAAGGAAAGTGCTGTTGATTTGTTCAGCTACTTGGAAACCTCG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CCTCTGCTGAGAAGTGAGCAAGTTCTGGATTTCTAAAGGGTTGGGGATGA&#xA;ENST00000481848.6       CCTCTGCTGAGAA-------------------------------------&#xA;                        *************                                     &#xA;&#xA;ENST00000525778.5       ATGATGGGCTTAGGAAGGAAGGTTGGCGGCGGTGAAGAGGGAAGAGGATG&#xA;ENST00000481848.6       --------CTTAGGAAGGAAGGTTGGCGGCGGTGAAGAGGGAAGAGGATG&#xA;                                ******************************************&#xA;&#xA;ENST00000525778.5       ATCTAGTAATGGGCTCTTCCAAATCGCATCTGGTAGGTTCATAGCCATGA&#xA;ENST00000481848.6       ATCTAGTAATGGGCTCTTCCAAATCGCATCTGGTAGGTTCATAGCCATGA&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GTCAGCCACCAATCGGGGGCGCTGCCCCTGCCACAGCAGCAGCTTCTCCC&#xA;ENST00000481848.6       GTCAGCCACCAATCGGGGGCGCTGCCCCTGCCACAGCAGCAGCTTCTCCC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GCCGCTGCAGCCACTGAGGCTCGTCTGCACCCGGAGGGCAGCAGCAGAAA&#xA;ENST00000481848.6       GCCGCTGCAGCCACTGAGGCTCGTCTGCACCCGGAGGGCAGCAGCAGAAA&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GCAGCAGCGAGCTCAGTCACCCGCTAGACCGAGGGACAGTTCGCTGCGAC&#xA;ENST00000481848.6       GCAGCAGCGAGCTCAGTCACCCGCTAGACCGAGGGACAGTTCGCTGCGAC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       AGACAATCGCGGCCACCCGGTCCCCAGTGGGGGCCGGCACTAAACTCAAT&#xA;ENST00000481848.6       AGACAATCGCGGCCACCCGGTCCCCAGTGGGGGCCGGCACTAAACTCAAT&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       TCTGTTCGGCAGCAGCAGCTGCAGCAGCAGCAACAGCAGGGTAACAAGAC&#xA;ENST00000481848.6       TCTGTTCGGCAGCAGCAGCTGCAGCAGCAGCAACAGCAGGGTAACAAGAC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CGGGAGCCGCACGGGGCCGCCCGCCAGCATCCGAGGAGGCGGAGGTGGGG&#xA;ENST00000481848.6       CGGGAGCCGCACGGGGCCGCCCGCCAGCATCCGAGGAGGCGGAGGTGGGG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CCGAGAAGGCGACGCCCCTGGCGCCCAAAGGGGCCGCTCCGGGAGCTGTC&#xA;ENST00000481848.6       CCGAGAAGGCGACGCCCCTGGCGCCCAAAGGGGCCGCTCCGGGAGCTGTC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CAGCCTGTGGCTGGTGCTGAAGCGGCCCCCGCCGCGACCCTGGCCGCTTT&#xA;ENST00000481848.6       CAGCCTGTGGCTGGTGCTGAAGCGGCCCCCGCCGCGACCCTGGCCGCTTT&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GGGCGGCAGGAGGCCCGGACCGCCCGAGGAGCCGCCTCGGGAGCTAGAGT&#xA;ENST00000481848.6       GGGCGGCAGGAGGCCCGGACCGCCCGAGGAGCCGCCTCGGGAGCTAGAGT&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CCGTGCCCTCGAAACTCGGGGAACCCCCTCCGCTCGGAGAAGGGGGAGGA&#xA;ENST00000481848.6       CCGTGCCCTCGAAACTCGGGGAACCCCCTCCGCTCGGAGAAGGGGGAGGA&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GGGGGCGGGGAAGGAGGAGGAGCCGGCGGCGGCTCCGGGGAAAGGGAGGG&#xA;ENST00000481848.6       GGGGGCGGGGAAGGAGGAGGAGCCGGCGGCGGCTCCGGGGAAAGGGAGGG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GGGCGCTCCGCAGCCGCCGCCGCCCAGGGGCTGGCGAGGGAAAGGCGTAC&#xA;ENST00000481848.6       GGGCGCTCCGCAGCCGCCGCCGCCCAGGGGCTGGCGAGGGAAAGGCGTAC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GCGCTCAGCAGAGGGGCGGCAGCGGCGGGGAGGGGGCCTCCCCTTCTCCA&#xA;ENST00000481848.6       GCGCTCAGCAGAGGGGCGGCAGCGGCGGGGAGGGGGCCTCCCCTTCTCCA&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       TCCTCCTCTTCTGCGGGCAAAACCCCAGGAACCGGCAGCAGAAACTCCGG&#xA;ENST00000481848.6       TCCTCCTCTTCTGCGGGCAAAACCCCAGGAACCGGCAGCAGAAACTCCGG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       AAGCGGCGTTGCGGGGGGCGGCAGCGGTGGTGGAGGGAGCTACTGGAAAG&#xA;ENST00000481848.6       AAGCGGCGTTGCGGGGGGCGGCAGCGGTGGTGGAGGGAGCTACTGGAAAG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       AAGGATGTCTGCAGTCTGAGCTCATCCAGTTCCATCTCAAGAAGGAGCGG&#xA;ENST00000481848.6       AAGGATGTCTGCAGTCTGAGCTCATCCAGTTCCATCTCAAGAAGGAGCGG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GCGGCAGCGGCGGCGGCCGCGGCTCAGATGCACGCTAAGAACGGCGGCGG&#xA;ENST00000481848.6       GCGGCAGCGGCGGCGGCCGCGGCTCAGATGCACGCTAAGAACGGCGGCGG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CAGCAGTAGCCGCAGCTCCCCGGTGTCTGGCCCCCCTGCCGTTTGCGAGA&#xA;ENST00000481848.6       CAGCAGTAGCCGCAGCTCCCCGGTGTCTGGCCCCCCTGCCGTTTGCGAGA&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CCCTGGCCGTCGCCTCCGCCTCCCCAATGGCGGCGGCGGCGGAGGGCCCC&#xA;ENST00000481848.6       CCCTGGCCGTCGCCTCCGCCTCCCCAATGGCGGCGGCGGCGGAGGGCCCC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CAGCAGAGCGCAGAGGGCAGCGCGAGCGGCGGGGGCATGCAGGCGGCAGC&#xA;ENST00000481848.6       CAGCAGAGCGCAGAGGGCAGCGCGAGCGGCGGGGGCATGCAGGCGGCAGC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GCCCCCTTCGTCGCAGCCGCACCCGCAGCAGCTCCAAGAGCAGGAAGAAA&#xA;ENST00000481848.6       GCCCCCTTCGTCGCAGCCGCACCCGCAGCAGCTCCAAGAGCAGGAAGAAA&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       TGCAAGAGGAGATGGAGAAGCTGCGAGAGGAAAACGAGACTCTCAAGAAC&#xA;ENST00000481848.6       TGCAAGAGGAGATGGAGAAGCTGCGAGAGGAAAACGAGACTCTCAAGAAC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GAGATCGATGAGCTGAGAACCGAGATGGACGAGATGAGGGACACTTTCTT&#xA;ENST00000481848.6       GAGATCGATGAGCTGAGAACCGAGATGGACGAGATGAGGGACACTTTCTT&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CGAGGAGGATGCCTGTCAACTGCAGGAAATGCGCCACGAGTTGGAGAGAG&#xA;ENST00000481848.6       CGAGGAGGATGCCTGTCAACTGCAGGAAATGCGCCACGAGTTGGAGAGAG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CCAACAAAAACTGCCGGATCCTGCAGTACCGCCTCCGCAAAGCCGAGCGC&#xA;ENST00000481848.6       CCAACAAAAACTGCCGGATCCTGCAGTACCGCCTCCGCAAAGCCGAGCGC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       AAAAGGCTCCGCTACGCCCAGACCGGGGAAATCGACGGGGAGCTGTTGCG&#xA;ENST00000481848.6       AAAAGGCTCCGCTACGCCCAGACCGGGGAAATCGACGGGGAGCTGTTGCG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CAGCCTGGAGCAGGACCTCAAGGTTGCAAAGGATGTATCTGTGAGACTTC&#xA;ENST00000481848.6       CAGCCTGGAGCAGGACCTCAAGGTTGCAAAGGATGTATCTGTGAGACTTC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       ACCATGAATTAGAAAATGTGGAAGAAAAGAGAACAACAACAGAAGATGAA&#xA;ENST00000481848.6       ACCATGAATTAGAAAATGTGGAAGAAAAGAGAACAACAACAGAAGATGAA&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       AATGAGAAACTGAGGCAACAGCTCATAGAAGTTGAAATTGCAAAGCAAGC&#xA;ENST00000481848.6       AATGAGAAACTGAGGCAACAGCTCATAGAAGTTGAAATTGCAAAGCAAGC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       TTTACAGAATGAACTGGAAAAAATGAAAGAGTTATCCTTAAAAAGAAGAG&#xA;ENST00000481848.6       TTTACAGAATGAACTGGAAAAAATGAAAGAGTTATCCTTAAAAAGAAGAG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GAAGCAAAGATTTGCCAAAATCTGAAAAAAAGGCTCAACAGACTCCCACA&#xA;ENST00000481848.6       GAAGCAAAGATTTGCCAAAATCTGAAAAAAAGGCTCAACAGACTCCCACA&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GAGGAGGACAATGAAGATCTGAAGTGCCAGCTGCAGTTTGTTAAGGAAGA&#xA;ENST00000481848.6       GAGGAGGACAATGAAGATCTGAAGTGCCAGCTGCAGTTTGTTAAGGAAGA&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       AGCCGCTTTGATGAGAAAGAAAATGGCCAAGATTGATAAAGAAAAGGACA&#xA;ENST00000481848.6       AGCCGCTTTGATGAGAAAGAAAATGGCCAAGATTGATAAAGAAAAGGACA&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GATTTGAACACGAGCTCCAGAAGTACAGATCCTTTTATGGGGATCTGGAC&#xA;ENST00000481848.6       GATTTGAACACGAGCTCCAGAAGTACAGATCCTTTTATGGGGATCTGGAC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       AGTCCTTTGCCCAAAGGAGAAGCCGGAGGCCCTCCCAGCACTAGGGAGGC&#xA;ENST00000481848.6       AGTCCTTTGCCCAAAGGAGAAGCCGGAGGCCCTCCCAGCACTAGGGAGGC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CGAGCTCAAGCTACGGCTAAGGCTGGTGGAGGAAGAAGCCAACATCCTGG&#xA;ENST00000481848.6       CGAGCTCAAGCTACGGCTAAGGCTGGTGGAGGAAGAAGCCAACATCCTGG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GCAGGAAAATCGTCGAACTGGAGGTGGAGAACAGAGGCCTGAAGGCGGAA&#xA;ENST00000481848.6       GCAGGAAAATCGTCGAACTGGAGGTGGAGAACAGAGGCCTGAAGGCGGAA&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CTGGACGACCTTAGGGGCGATGACTTCAACGGCTCGGCCAACCCGCTCAT&#xA;ENST00000481848.6       CTGGACGACCTTAGGGGCGATGACTTCAACGGCTCGGCCAACCCGCTCAT&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GAGGGAGCAGAGCGAATCCCTGTCGGAGCTGCGGCAGCACCTGCAGCTGG&#xA;ENST00000481848.6       GAGGGAGCAGAGCGAATCCCTGTCGGAGCTGCGGCAGCACCTGCAGCTGG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       TGGAAGACGAGACGGAGCTGCTGCGGAGGAACGTGGCCGACCTGGAGGAG&#xA;ENST00000481848.6       TGGAAGACGAGACGGAGCTGCTGCGGAGGAACGTGGCCGACCTGGAGGAG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CAGAACAAGCGCATCACGGCGGAGCTCAACAAGTACAAGTACAAGTCCGG&#xA;ENST00000481848.6       CAGAACAAGCGCATCACGGCGGAGCTCAACAAGTACAAGTACAAGTCCGG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CGGCCACGACAGCGCGCGGCACCACGACAACGCCAAGACCGAGGCCCTGC&#xA;ENST00000481848.6       CGGCCACGACAGCGCGCGGCACCACGACAACGCCAAGACCGAGGCCCTGC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       AGGAGGAGCTGAAGGCGGCGCGCCTGCAGATCAACGAGCTCAGCGGCAAG&#xA;ENST00000481848.6       AGGAGGAGCTGAAGGCGGCGCGCCTGCAGATCAACGAGCTCAGCGGCAAG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GTCATGCAGCTGCAGTACGAGAACCGCGTGCTTATGTCCAACATGCAGCG&#xA;ENST00000481848.6       GTCATGCAGCTGCAGTACGAGAACCGCGTGCTTATGTCCAACATGCAGCG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CTACGACCTGGCCTCGCACCTGGGCATCCGCGGCAGCCCCCGCGACAGCG&#xA;ENST00000481848.6       CTACGACCTGGCCTCGCACCTGGGCATCCGCGGCAGCCCCCGCGACAGCG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       ACGCCGAGAGCGACGCGGGCAAGAAGGAGAGCGACGACGACTCGCGGCCT&#xA;ENST00000481848.6       ACGCCGAGAGCGACGCGGGCAAGAAGGAGAGCGACGACGACTCGCGGCCT&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CCGCACCGCAAGCGCGAAGGGCCCATCGGCGGCGAGAGCGACTCGGAGGA&#xA;ENST00000481848.6       CCGCACCGCAAGCGCGAAGGGCCCATCGGCGGCGAGAGCGACTCGGAGGA&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GGTGCGCAACATCCGCTGCCTCACGCCCACTCGCTCCTTCTACCCGGCGC&#xA;ENST00000481848.6       GGTGCGCAACATCCGCTGCCTCACGCCCACTCGCTCCTTCTACCCGGCGC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CCGGGCCCTGGCCCAAGAGCTTCTCCGATCGGCAGCAGATGAAGGACATC&#xA;ENST00000481848.6       CCGGGCCCTGGCCCAAGAGCTTCTCCGATCGGCAGCAGATGAAGGACATC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CGCTCGGAGGCCGAGCGCCTGGGCAAGACCATCGACCGGCTCATCGCCGA&#xA;ENST00000481848.6       CGCTCGGAGGCCGAGCGCCTGGGCAAGACCATCGACCGGCTCATCGCCGA&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CACGAGCACCATCATCACCGAGGCGCGCATCTACGTGGCCAACGGGGACC&#xA;ENST00000481848.6       CACGAGCACCATCATCACCGAGGCGCGCATCTACGTGGCCAACGGGGACC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       TGTTCGGACTCATGGACGAGGAGGACGACGGCAGCCGCATCCGGGAGCAC&#xA;ENST00000481848.6       TGTTCGGACTCATGGACGAGGAGGACGACGGCAGCCGCATCCGGGAGCAC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GAGCTGCTCTACCGCATCAACGCTCAGATGAAGGCCTTCCGCAAGGAGCT&#xA;ENST00000481848.6       GAGCTGCTCTACCGCATCAACGCTCAGATGAAGGCCTTCCGCAAGGAGCT&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GCAGACCTTCATCGACCGCCTCGAGGTGCCCAAGTCTGCGGACGACCGCG&#xA;ENST00000481848.6       GCAGACCTTCATCGACCGCCTCGAGGTGCCCAAGTCTGCGGACGACCGCG&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GCGCCGAGGAGCCCATTTCCGTGAGTCAGATGTTCCAGCCTATCATTTTA&#xA;ENST00000481848.6       GCGCCGAGGAGCCCATTTCCGTGAGTCAGATGTTCCAGCCTATCATTTTA&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       CTTATTCTCATTCTTGTATTATTTTCATCACTTTCTTACACAACAATATT&#xA;ENST00000481848.6       CTTATTCTCATTCTTGTATTATTTTCATCACTTTCTTACACAACAATATT&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       TAAACTTGTCTTCCTTTTTACACTGTTTTTTGTACTGTAAATCTTTCATC&#xA;ENST00000481848.6       TAAACTTGTCTTCCTTTTTACACTGTTTTTTGTACTGTAAATCTTTCATC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       ATTTACCATTCATTGTAGTATTTTCAGTTTGTTTATTTTGTTCACCCTTC&#xA;ENST00000481848.6       ATTTACCATTCATTGTAGTATTTTCAGTTTGTTTATTTTGTTCACCCTTC&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       AAGACAAGAAGTAAAAGAAGTATAATTTCTGTAGTAACCAATGCTATAAA&#xA;ENST00000481848.6       AAGACAAGAAGTAAAAGAAGTATAATTTCTGTAGTAACCAATGCTATAAA&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       AACACTGAAGACTGCTTATTTCTTTAAAAAGATACAACTCATCTTACCAA&#xA;ENST00000481848.6       AACACTGAAGACTGCTTATTTCTTTAAAAAGATACAACTCATCTTACCAA&#xA;                        **************************************************&#xA;&#xA;ENST00000525778.5       GACCAAA-------------------------------------------&#xA;ENST00000481848.6       GACCAAATTCAATAAGAAGCCCAAACACTAAAATATTTCAGGCTTTATTT&#xA;                        *******                                           &#xA;&#xA;ENST00000525778.5       --------------------------------------------------&#xA;ENST00000481848.6       TAAAGGCAAGTGAGACTGCTTCAAATAAAACAACTTCAAGCTTCCAAGAA&#xA;&#xA;[. . .]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;The ENST00000481848.6 transcript continues quite a bit after this but there is no more similarity. &lt;/sub&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The transcripts are not identical, but are very similar. Now, look at the proteins each transcript encodes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ENST00000481848.6  MSQPPIGGAAPATAAASPAAAATEARLHPEGSSRKQQRAQSPARPRDSSL&#xA;ENST00000525778.5  MSQPPIGGAAPATAAASPAAAATEARLHPEGSSRKQQRAQSPARPRDSSL&#xA;                   **************************************************&#xA;&#xA;ENST00000481848.6  RQTIAATRSPVGAGTKLNSVRQQQLQQQQQQGNKTGSRTGPPASIRGGGG&#xA;ENST00000525778.5  RQTIAATRSPVGAGTKLNSVRQQQLQQQQQQGNKTGSRTGPPASIRGGGG&#xA;                   **************************************************&#xA;&#xA;ENST00000481848.6  GAEKATPLAPKGAAPGAVQPVAGAEAAPAATLAALGGRRPGPPEEPPREL&#xA;ENST00000525778.5  GAEKATPLAPKGAAPGAVQPVAGAEAAPAATLAALGGRRPGPPEEPPREL&#xA;                   **************************************************&#xA;&#xA;ENST00000481848.6  ESVPSKLGEPPPLGEGGGGGGEGGGAGGGSGEREGGAPQPPPPRGWRGKG&#xA;ENST00000525778.5  ESVPSKLGEPPPLGEGGGGGGEGGGAGGGSGEREGGAPQPPPPRGWRGKG&#xA;                   **************************************************&#xA;&#xA;ENST00000481848.6  VRAQQRGGSGGEGASPSPSSSSAGKTPGTGSRNSGSGVAGGGSGGGGSYW&#xA;ENST00000525778.5  VRAQQRGGSGGEGASPSPSSSSAGKTPGTGSRNSGSGVAGGGSGGGGSYW&#xA;                   **************************************************&#xA;&#xA;ENST00000481848.6  KEGCLQSELIQFHLKKERAAAAAAAAQMHAKNGGGSSSRSSPVSGPPAVC&#xA;ENST00000525778.5  KEGCLQSELIQFHLKKERAAAAAAAAQMHAKNGGGSSSRSSPVSGPPAVC&#xA;                   **************************************************&#xA;&#xA;ENST00000481848.6  ETLAVASASPMAAAAEGPQQSAEGSASGGGMQAAAPPSSQPHPQQLQEQE&#xA;ENST00000525778.5  ETLAVASASPMAAAAEGPQQSAEGSASGGGMQAAAPPSSQPHPQQLQEQE&#xA;                   **************************************************&#xA;&#xA;ENST00000481848.6  EMQEEMEKLREENETLKNEIDELRTEMDEMRDTFFEEDACQLQEMRHELE&#xA;ENST00000525778.5  EMQEEMEKLREENETLKNEIDELRTEMDEMRDTFFEEDACQLQEMRHELE&#xA;                   **************************************************&#xA;&#xA;ENST00000481848.6  RANKNCRILQYRLRKAERKRLRYAQTGEIDGELLRSLEQDLKVAKDVSVR&#xA;ENST00000525778.5  RANKNCRILQYRLRKAERKRLRYAQTGEIDGELLRSLEQDLKVAKDVSVR&#xA;                   **************************************************&#xA;&#xA;ENST00000481848.6  LHHELENVEEKRTTTEDENEKLRQQLIEVEIAKQALQNELEKMKELSLKR&#xA;ENST00000525778.5  LHHELENVEEKRTTTEDENEKLRQQLIEVEIAKQALQNELEKMKELSLKR&#xA;                   **************************************************&#xA;&#xA;ENST00000481848.6  RGSKDLPKSEKKAQQTPTEEDNEDLKCQLQFVKEEAALMRKKMAKIDKEK&#xA;ENST00000525778.5  RGSKDLPKSEKKAQQTPTEEDNEDLKCQLQFVKEEAALMRKKMAKIDKEK&#xA;                   **************************************************&#xA;&#xA;ENST00000481848.6  DRFEHELQKYRSFYGDLDSPLPKGEAGGPPSTREAELKLRLRLVEEEANI&#xA;ENST00000525778.5  DRFEHELQKYRSFYGDLDSPLPKGEAGGPPSTREAELKLRLRLVEEEANI&#xA;                   **************************************************&#xA;&#xA;ENST00000481848.6  LGRKIVELEVENRGLKAELDDLRGDDFNGSANPLMREQSESLSELRQHLQ&#xA;ENST00000525778.5  LGRKIVELEVENRGLKAELDDLRGDDFNGSANPLMREQSESLSELRQHLQ&#xA;                   **************************************************&#xA;&#xA;ENST00000481848.6  LVEDETELLRRNVADLEEQNKRITAELNKYKYKSGGHDSARHHDNAKTEA&#xA;ENST00000525778.5  LVEDETELLRRNVADLEEQNKRITAELNKYKYKSGGHDSARHHDNAKTEA&#xA;                   **************************************************&#xA;&#xA;ENST00000481848.6  LQEELKAARLQINELSGKVMQLQYENRVLMSNMQRYDLASHLGIRGSPRD&#xA;ENST00000525778.5  LQEELKAARLQINELSGKVMQLQYENRVLMSNMQRYDLASHLGIRGSPRD&#xA;                   **************************************************&#xA;&#xA;ENST00000481848.6  SDAESDAGKKESDDDSRPPHRKREGPIGGESDSEEVRNIRCLTPTRSFYP&#xA;ENST00000525778.5  SDAESDAGKKESDDDSRPPHRKREGPIGGESDSEEVRNIRCLTPTRSFYP&#xA;                   **************************************************&#xA;&#xA;ENST00000481848.6  APGPWPKSFSDRQQMKDIRSEAERLGKTIDRLIADTSTIITEARIYVANG&#xA;ENST00000525778.5  APGPWPKSFSDRQQMKDIRSEAERLGKTIDRLIADTSTIITEARIYVANG&#xA;                   **************************************************&#xA;&#xA;ENST00000481848.6  DLFGLMDEEDDGSRIREHELLYRINAQMKAFRKELQTFIDRLEVPKSADD&#xA;ENST00000525778.5  DLFGLMDEEDDGSRIREHELLYRINAQMKAFRKELQTFIDRLEVPKSADD&#xA;                   **************************************************&#xA;&#xA;ENST00000481848.6  RGAEEPISVSQMFQPIILLILILVLFSSLSYTTIFKLVFLFTLFFVL&#xA;ENST00000525778.5  RGAEEPISVSQMFQPIILLILILVLFSSLSYTTIFKLVFLFTLFFVL&#xA;                   ***********************************************&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, they are 100% identical. So, we have two genes (distinct regions of the DNA) which produce the same protein. Because the gene product is the same, the &quot;function&quot; of the genes is also the same so HUGO treats them as the same gene and gives them the same name. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether or not you want to treat them as one gene or two will depend on what you are doing. If you are comparing the relative abundance of each &lt;strong&gt;&lt;em&gt;transcript&lt;/em&gt;&lt;/strong&gt;, you must treat them separately. If you want to examine the expression level of the &lt;strong&gt;&lt;em&gt;protein product&lt;/em&gt;&lt;/strong&gt; and are using the RNASEM data as a proxy for protein information, then you should treat them as one &quot;gene&quot; and simply sum your RNASEM values.  &lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-11-30T12:14:20.760" LastActivityDate="2017-11-30T12:14:20.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2964" PostTypeId="1" AcceptedAnswerId="16300" CreationDate="2017-11-30T19:24:29.017" Score="3" ViewCount="2141" Body="&lt;p&gt;I try to perform a hybrid assembly using Unicycler. Unicycler use SPAdes to assembly the illumina sequences, and the program crashes while in SPAdes. The spades.log file says something about an invalid argument and error code 22. I have run the &lt;code&gt;spades.py --test&lt;/code&gt;, and everything seems to work fine. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Command line: /home/user/SPAdes-3.11.1-Linux/bin/spades.py    -1  /home/user/minion/organism/Illumina/BD_S3_L001_R1_001.fastq.gz -2  /home/user/minion/organism/Illumina/BD_S3_L001_R2_001.fastq.gz -o  /home/user/minion/organism/hybrid_assembly/spades_assembly/read_correction --threads   8   --only-error-correction &#xA;&#xA;System information:&#xA;  SPAdes version: 3.11.1&#xA;  Python version: 2.7.12&#xA;  OS: Linux-4.10.0-40-generic-x86_64-with-Ubuntu-16.04-xenial&#xA;&#xA;Output dir: /home/user/minion/organism/hybrid_assembly/spades_assembly/read_correction&#xA;Mode: ONLY read error correction (without assembling)&#xA;Debug mode is turned OFF&#xA;&#xA;Dataset parameters:&#xA;  Multi-cell mode (you should set '--sc' flag if input data was obtained with MDA (single-cell) technology or --meta flag if processing metagenomic dataset)&#xA;  Reads:&#xA;    Library number: 1, library type: paired-end&#xA;      orientation: fr&#xA;      left reads: ['/home/user/minion/organism/Illumina/BD_S3_L001_R1_001.fastq.gz']&#xA;      right reads: ['/home/user/minion/organism/Illumina/BD_S3_L001_R2_001.fastq.gz']&#xA;      interlaced reads: not specified&#xA;      single reads: not specified&#xA;Read error correction parameters:&#xA;  Iterations: 1&#xA;  PHRED offset will be auto-detected&#xA;  Corrected reads will be compressed (with gzip)&#xA;Other parameters:&#xA;  Dir for temp files: /home/user/minion/organism/hybrid_assembly/spades_assembly/read_correction/tmp&#xA;  Threads: 8&#xA;  Memory limit (in Gb): 87&#xA;&#xA;&#xA;======= SPAdes pipeline started. Log can be found here: /home/user/minion/organism/hybrid_assembly/spades_assembly/read_correction/spades.log&#xA;&#xA;&#xA;===== Read error correction started. &#xA;&#xA;&#xA;== Running read error correction tool: /home/user/SPAdes-3.11.1-Linux/bin/hammer /home/user/minion/organism/hybrid_assembly/spades_assembly/read_correction/corrected/configs/config.info&#xA;&#xA;  0:00:00.000     4M / 4M    INFO    General                 (main.cpp                  :  83)   Starting BayesHammer, built from N/A, git revision N/A&#xA;  0:00:00.001     4M / 4M    INFO    General                 (main.cpp                  :  84)   Loading config from /home/user/minion/organism/hybrid_assembly/spades_assembly/read_correction/corrected/configs/config.info&#xA;  0:00:00.045     4M / 4M    INFO    General                 (memory_limit.hpp          :  51)   Memory limit set to 87 Gb&#xA;  0:00:00.045     4M / 4M    INFO    General                 (main.cpp                  :  93)   Trying to determine PHRED offset&#xA;  0:00:00.071     4M / 4M    INFO    General                 (main.cpp                  :  99)   Determined value is 33&#xA;  0:00:00.071     4M / 4M    INFO    General                 (hammer_tools.cpp          :  36)   Hamming graph threshold tau=1, k=21, subkmer positions = [ 0 10 ]&#xA;  0:00:00.072     4M / 4M    INFO    General                 (main.cpp                  : 120)   Size of aux. kmer data 24 bytes&#xA;     === ITERATION 0 begins ===&#xA;  0:00:00.100     4M / 4M    INFO   K-mer Index Building     (kmer_index_builder.hpp    : 289)   Building kmer index&#xA;  0:00:00.101     4M / 4M    INFO    General                 (kmer_index_builder.hpp    : 106)   Splitting kmer instances into 128 buckets using 8 threads. This might take a while.&#xA;  0:00:00.101     4M / 4M    INFO    General                 (file_limit.hpp            :  32)   Open file limit set to 1024&#xA;  0:00:00.101     4M / 4M    INFO    General                 (kmer_splitters.hpp        :  74)   Memory available for splitting buffers: 3.62484 Gb&#xA;  0:00:00.101     4M / 4M    INFO    General                 (kmer_splitters.hpp        :  82)   Using cell size of 524288&#xA;  0:00:00.102     8G / 8G    INFO   K-mer Splitting          (kmer_data.cpp             :  96)   Processing /home/user/minion/organism/Illumina/BD_S3_L001_R1_001.fastq.gz&#xA;  0:00:57.279     8G / 8G    INFO   K-mer Splitting          (kmer_data.cpp             : 106)   Processed 1338594 reads&#xA;  0:03:30.505     8G / 8G    INFO   K-mer Splitting          (kmer_data.cpp             : 106)   Processed 2152556 reads&#xA;  0:03:30.505     8G / 8G    INFO   K-mer Splitting          (kmer_data.cpp             :  96)   Processing /home/user/minion/organism/Illumina/BD_S3_L001_R2_001.fastq.gz&#xA;  0:09:23.290     8G / 8G    INFO   K-mer Splitting          (kmer_data.cpp             : 106)   Processed 3442412 reads&#xA;  0:14:08.692     8G / 8G    INFO   K-mer Splitting          (kmer_data.cpp             : 106)   Processed 4305112 reads&#xA;  0:14:08.692     8G / 8G    INFO   K-mer Splitting          (kmer_data.cpp             : 111)   Total 4305112 reads processed&#xA;  0:14:08.726    32M / 8G    INFO    General                 (kmer_index_builder.hpp    : 109)   Starting k-mer counting.&#xA;  0:20:36.408    32M / 8G    INFO    General                 (kmer_index_builder.hpp    : 115)   K-mer counting done. There are 205776690 kmers in total.&#xA;  0:20:36.409    32M / 8G    INFO    General                 (kmer_index_builder.hpp    : 121)   Merging temporary buckets.&#xA;  0:24:43.505    32M / 8G    INFO   K-mer Index Building     (kmer_index_builder.hpp    : 298)   Building perfect hash indices&#xA;  0:25:40.125   100M / 8G    INFO    General                 (kmer_index_builder.hpp    : 137)   Merging final buckets.&#xA;  1:11:46.553   100M / 8G    INFO   K-mer Index Building     (kmer_index_builder.hpp    : 320)   Index built. Total 95426752 bytes occupied (3.70991 bits per kmer).&#xA;  1:11:46.598   100M / 8G    INFO   K-mer Counting           (kmer_data.cpp             : 357)   Arranging kmers in hash map order&#xA;  1:12:55.597     3G / 8G    INFO    General                 (main.cpp                  : 155)   Clustering Hamming graph.&#xA;  1:21:35.596     3G / 8G    INFO    General                 (main.cpp                  : 162)   Extracting clusters&#xA;=== Stack Trace ===&#xA;[0x407a6a]&#xA;[0x409561]&#xA;[0x40c79b]&#xA;[0x40129a]&#xA;[0x5442b0]&#xA;[0x40620d]&#xA;Verification of expression '(intptr_t) MappedRegion != -1L' failed in function 'void MMappedWriter::reserve(size_t)'. In file '/spades/src/common/io/kmers/mmapped_writer.hpp' on line 94. Message 'mmap(2) failed. Reason: Invalid argument. Error code: 22'.&#xA;Verification of expression '(intptr_t) MappedRegion != -1L' failed in function 'void MMappedWriter::reserve(size_t)'. In file '/spades/src/common/io/kmers/mmapped_writer.hpp' on line 94. Message 'mmap(2) failed. Reason: Invalid argument. Error code: 22'.&#xA;hammer: /spades/src/common/io/kmers/mmapped_writer.hpp:94: void MMappedWriter::reserve(size_t): Assertion `(intptr_t) MappedRegion != -1L' failed.&#xA;&#xA;&#xA;== Error ==  system call for: &quot;['/home/user/SPAdes-3.11.1-Linux/bin/hammer', '/home/user/minion/organism/hybrid_assembly/spades_assembly/read_correction/corrected/configs/config.info']&quot; finished abnormally, err code: -6&#xA;&#xA;In case you have troubles running SPAdes, you can write to spades.support@cab.spbu.ru&#xA;or report an issue on our GitHub repository github.com/ablab/spades&#xA;Please provide us with params.txt and spades.log files from the output directory.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1670" LastEditorUserId="1670" LastEditDate="2017-11-30T19:54:22.453" LastActivityDate="2021-07-04T04:40:39.627" Title="SPAdes error during assembly" Tags="|fasta|assembly|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2965" PostTypeId="1" CreationDate="2017-11-30T19:43:56.637" Score="2" ViewCount="51" Body="&lt;p&gt;I am trying to build a machine learning training set for bacterial protein sequences that form biofilm, and that cannot. I collected the positive sequences from the GO ontology website but for negative sequences I am not sure which sequences to incorporate into my training set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone point me to resources for proteins sequences that are known to be not capable of forming biofilms?&lt;/p&gt;&#xA;" OwnerUserId="1911" LastActivityDate="2017-12-01T09:14:50.883" Title="Proteins that cannot form biofilm?" Tags="|proteins|machine-learning|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2966" PostTypeId="2" ParentId="2775" CreationDate="2017-11-30T19:49:28.377" Score="1" Body="&lt;p&gt;BioGeek is correct, however I will post a full answer.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pythoncyc&#xA;# To get ec-code&#xA;db = pythoncyc.select_organism('sco')&#xA;reaction = db[&quot;RXN1A0-6128&quot;]&#xA;ec_code = reaction[&quot;ec_number&quot;]&#xA;&#xA;# To get genes&#xA;pgdb = pythoncyc.PGDB(&quot;sco&quot;)&#xA;genes = pgdb.genes_of_reaction(&quot;RXN1A0-6128&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1670" LastActivityDate="2017-11-30T19:49:28.377" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2967" PostTypeId="2" ParentId="2965" CreationDate="2017-12-01T09:14:50.883" Score="2" Body="&lt;p&gt;One option is to treat all proteins that are not explicitly known to form a biofilm as proteins that are incapable of doing so, but this will likely result in many false negatives.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could take a look at &lt;a href=&quot;https://en.wikipedia.org/wiki/One-class_classification&quot; rel=&quot;nofollow noreferrer&quot;&gt;one-class classification and positive-unlabeled (PU) learning&lt;/a&gt;. Both are techniques specifically designed for these kind of problems where only one of the two classes is known. (In this case: only biofilm forming proteins are known.)&lt;/p&gt;&#xA;" OwnerUserId="196" LastActivityDate="2017-12-01T09:14:50.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2968" PostTypeId="2" ParentId="2921" CreationDate="2017-12-01T17:27:18.087" Score="4" Body="&lt;p&gt;Can’t be done. If you already sequenced then I’m afraid the money is wasted (unless of course the data is good for something else).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The standard Illumina basecaller doesn’t deal well with homopolymers and will consequently call incorrect poly(A) lengths.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Amongst others, Narry Kim’s lab developed a dedicated method — &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/24582499&quot; rel=&quot;nofollow noreferrer&quot;&gt;TAIL-seq&lt;/a&gt; — to solve this problem. It’s a protocol for library prep using pulldown of biotin tagged tails (rather than random priming), plus dedicated basecalling software that uses spike-ins for calibration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Be warned though: the protocol is reportedly technically challenging, and in order to use it you need an Illumina machine that gives you access to the raw image files &lt;em&gt;before basecalling&lt;/em&gt;. Up-to-date Illumina machines won’t give you these files, full stop. So you’ll need to find a machine that runs an outdated firmware.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2017-12-01T17:27:18.087" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2969" PostTypeId="1" CreationDate="2017-12-01T18:28:46.733" Score="8" ViewCount="630" Body="&lt;p&gt;I am performing a gene set enrichment analysis to determine if particular gene sets are coherently up- or down-regulated. I have seen several statistics for computing a p-value of GSEA-style enrichment. I'm particularly interested in the differences between a mean-rank gene set test, like the one implemented in the geneSetTest R function, and the correlation-corrected version of the test implemented with the CAMERA method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand it, CAMERA corrects for correlation within a gene set, because a highly correlated query set violates the underlying assumptions and results in a bias toward more significant p-values. So, if I find a significant mean-rank gene set test p-value, but an insignificant CAMERA p-value, I conclude that my query set is correlated, hence the correction factor and reduction in significance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I interpret the opposite case, where I find a significant CAMERA p-value but an insignificant mean-rank gene set test p-value? Does this imply that my query set is less correlated than expected, or that non-query genes are more highly correlated with one another? I feel like there's a clear connection between CAMERA's variance inflation factor and a reduction in significance, but I struggle to see how the correction factor can increase the significance over the mean-rank test.&lt;/p&gt;&#xA;" OwnerUserId="1930" LastActivityDate="2017-12-04T21:08:23.367" Title="Comparison of gene set enrichment statistics" Tags="|gene|statistics|differential-expression|pathway|gsea|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2970" PostTypeId="1" CreationDate="2017-12-01T22:00:54.980" Score="0" ViewCount="143" Body="&lt;p&gt;I have a very, very long list of genome coordinates of areas. I need to gather a sequence in order to compare it to another database of sequences that I have. How do I produce the genome sequences from these coordinates in R?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am very new to this and need a detailed answer.&lt;/p&gt;&#xA;" OwnerUserId="1935" LastEditorUserId="48" LastEditDate="2017-12-02T12:54:35.743" LastActivityDate="2017-12-02T18:07:03.620" Title="Coordinates to Genome R Studio Script?" Tags="|r|genome|sequence-alignment|ngs|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2971" PostTypeId="1" AcceptedAnswerId="2973" CreationDate="2017-12-01T22:02:55.513" Score="3" ViewCount="568" Body="&lt;p&gt;I run the MinION MinKNOW without the live base-calling option. We know there is Metrichor and Albacore to perform base-calling after this process. However, I have not done any base-calling yet.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: Is it possible to use directly the &lt;code&gt;fast5&lt;/code&gt; files with &lt;code&gt;poretools&lt;/code&gt; to extract a fastq file without doing any previous base-calling process? I tried it and I get an empty fastq file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My &lt;code&gt;reads&lt;/code&gt; directory only contains the &lt;code&gt;fast5&lt;/code&gt; folder and I run: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;poretools fastq fast5/ &amp;gt; output.fastq&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas why do I get an empty file? What is the difference between base-called &lt;code&gt;fast5&lt;/code&gt; and non-base-called &lt;code&gt;fast5&lt;/code&gt; files? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I am trying to do base-calling with Albacore to see if I get a fastq file. &lt;/p&gt;&#xA;" OwnerUserId="1421" LastEditorUserId="1421" LastEditDate="2017-12-04T16:32:57.597" LastActivityDate="2017-12-05T10:30:27.887" Title="Can we use non base-called fast5 files in poretools?" Tags="|minion|albacore|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2972" PostTypeId="1" CreationDate="2017-12-01T22:49:15.440" Score="-1" ViewCount="452" Body="&lt;p&gt;I'm running wine v 2.0.3 using ubuntu v. 16.04.2 on a Mac. I'm trying to install a program called PGDSpider2 from &lt;a href=&quot;http://www.cmpg.unibe.ch/software/PGDSpider/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Essentially the program is supposed to convert different file formats used for bioinformatics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm running it in a 32-bit wine server and I have the Java V.9 and the Microsoft.net packages downloaded on that same server. I've been getting this error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried the installation on a clean bottle but get the same following error, any thoughts?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;parallels@ubuntu:~/.wine32/drive_c/Program Files/PGDSpider_2.1.1.3$ WINEPREFIX=~/.wine32 wine PGDSpider2.exe&#xA;fixme:heap:RtlSetHeapInformation (nil) 1 (nil) 0 stub&#xA;fixme:process:SetProcessDEPPolicy (1): stub&#xA;fixme:heap:RtlSetHeapInformation (nil) 1 (nil) 0 stub&#xA;fixme:process:SetProcessShutdownParameters (00000380, 00000000): partial stub.&#xA;fixme:process:GetProcessWorkingSetSize (0xffffffff,0x73e94c,0x73e950): stub&#xA;fixme:win:RegisterDeviceNotificationA (hwnd=0x12dd30, filter=0x93e6fc,flags=0x00000001) returns a fake device notification handle!&#xA;fixme:win:UnregisterDeviceNotification (handle=0xcafecafe), STUB!&#xA;fixme:win:RegisterDeviceNotificationA (hwnd=0x12dd30, filter=0x93e698,flags=0x00000001) returns a fake device notification handle!&#xA;fixme:win:UnregisterDeviceNotification (handle=0xcafecafe), STUB!&#xA;fixme:win:RegisterDeviceNotificationA (hwnd=0x12dd30, filter=0x93e698,flags=0x00000001) returns a fake device notification handle!&#xA;fixme:win:UnregisterDeviceNotification (handle=0xcafecafe), STUB!&#xA;fixme:win:RegisterDeviceNotificationA (hwnd=0x12dd30, filter=0x93e698,flags=0x00000001) returns a fake device notification handle!&#xA;fixme:win:UnregisterDeviceNotification (handle=0xcafecafe), STUB!&#xA;fixme:win:RegisterDeviceNotificationA (hwnd=0x12dd30, filter=0x93e698,flags=0x00000001) returns a fake device notification handle!&#xA;fixme:win:UnregisterDeviceNotification (handle=0xcafecafe), STUB!&#xA;fixme:win:RegisterDeviceNotificationA (hwnd=0x12dd30, filter=0x93e698,flags=0x00000001) returns a fake device notification handle!&#xA;fixme:win:UnregisterDeviceNotification (handle=0xcafecafe), STUB!&#xA;fixme:win:RegisterDeviceNotificationA (hwnd=0x12dd30, filter=0x93e698,flags=0x00000001) returns a fake device notification handle!&#xA;fixme:win:UnregisterDeviceNotification (handle=0xcafecafe), STUB!&#xA;fixme:win:RegisterDeviceNotificationA (hwnd=0x12dd30, filter=0x93e698,flags=0x00000001) returns a fake device notification handle!&#xA;fixme:win:UnregisterDeviceNotification (handle=0xcafecafe), STUB!&#xA;fixme:win:RegisterDeviceNotificationA (hwnd=0x12dd30, filter=0x93e698,flags=0x00000001) returns a fake device notification handle!&#xA;fixme:win:UnregisterDeviceNotification (handle=0xcafecafe), STUB!&#xA;fixme:win:RegisterDeviceNotificationA (hwnd=0x12dd30, filter=0x93e698,flags=0x00000001) returns a fake device notification handle!&#xA;fixme:win:UnregisterDeviceNotification (handle=0xcafecafe), STUB!&#xA;fixme:win:RegisterDeviceNotificationA (hwnd=0x12dd30, filter=0x93e698,flags=0x00000001) returns a fake device notification handle!&#xA;fixme:win:UnregisterDeviceNotification (handle=0xcafecafe), STUB!&#xA;fixme:win:RegisterDeviceNotificationA (hwnd=0x12dd30, filter=0x93e698,flags=0x00000001) returns a fake device notification handle!&#xA;fixme:win:UnregisterDeviceNotification (handle=0xcafecafe), STUB!&#xA;fixme:win:RegisterDeviceNotificationA (hwnd=0x12dd30, filter=0x93e698,flags=0x00000001) returns a fake device notification handle!&#xA;fixme:win:UnregisterDeviceNotification (handle=0xcafecafe), STUB!&#xA;fixme:win:RegisterDeviceNotificationA (hwnd=0x12dd30, filter=0x93e698,flags=0x00000001) returns a fake device notification handle!&#xA;fixme:win:UnregisterDeviceNotification (handle=0xcafecafe), STUB!&#xA;fixme:win:RegisterDeviceNotificationA (hwnd=0x12dd30, filter=0x93e698,flags=0x00000001) returns a fake device notification handle!&#xA;fixme:win:UnregisterDeviceNotification (handle=0xcafecafe), STUB!&#xA;fixme:win:RegisterDeviceNotificationA (hwnd=0x12dd30, filter=0x93e698,flags=0x00000001) returns a fake device notification handle!&#xA;fixme:win:UnregisterDeviceNotification (handle=0xcafecafe), STUB!&#xA;fixme:win:RegisterDeviceNotificationA (hwnd=0x12dd30, filter=0x93e698,flags=0x00000001) returns a fake device notification handle!&#xA;fixme:win:UnregisterDeviceNotification (handle=0xcafecafe), STUB!&#xA;parallels@ubuntu:~/.wine32/drive_c/Program Files/PGDSpider_2.1.1.3$ fixme:win:EnumDisplayDevicesW ((null),0,0x39edcf0,0x00000000), stub!&#xA;fixme:win:EnumDisplayDevicesW ((null),1,0x39edcf0,0x00000000), stub!&#xA;X Error of failed request:  BadMatch (invalid parameter attributes)&#xA;  Major opcode of failed request:  156 (GLX)&#xA;  Minor opcode of failed request:  5 (X_GLXMakeCurrent)&#xA;  Serial number of failed request:  241&#xA;  Current serial number in output stream:  241&#xA;k&#xA;fixme:advapi:RegisterEventSourceW ((null),L&quot;.NET Runtime Optimization Service&quot;): stub&#xA;fixme:advapi:DeregisterEventSource (0xcafe4242) stub&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1088" LastEditorUserId="73" LastEditDate="2018-01-10T06:14:28.183" LastActivityDate="2018-01-10T06:14:28.183" Title="Unable to Install PGDSpider using Ubuntu and wine" Tags="|format-conversion|linux|phylogenetics|molecular-genetics|mtdna|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="2973" PostTypeId="2" ParentId="2971" CreationDate="2017-12-02T02:56:40.563" Score="4" Body="&lt;p&gt;No, poretools does not do basecalling. The &lt;code&gt;poretools fastq&lt;/code&gt; command can be used to extract the FASTQ information from the basecalled FAST5 file (via MinKNOW live-basecalling or albacore). Alternatively, both of these basecallers can export a FASTQ file directly if desired.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference between a basecalled FAST5 and a nonbasecalled FAST5 is the presence of the FASTQ within the file. This can be directly viewed using an HDF5 viewer, like &lt;a href=&quot;https://support.hdfgroup.org/HDF5/doc/RM/Tools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;h5dump&lt;/a&gt; from the commandline, or &lt;a href=&quot;https://support.hdfgroup.org/products/java/hdfview/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HDFView&lt;/a&gt; from the GUI.&lt;/p&gt;&#xA;" OwnerUserId="92" LastEditorUserId="92" LastEditDate="2017-12-02T03:01:42.357" LastActivityDate="2017-12-02T03:01:42.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2977" PostTypeId="2" ParentId="2969" CreationDate="2017-12-02T15:46:24.577" Score="4" Body="&lt;p&gt;I assume that you are talking about the implementation of these methods in the &lt;a href=&quot;https://bioconductor.org/packages/limma/&quot; rel=&quot;nofollow noreferrer&quot;&gt;limma package&lt;/a&gt;. Otherwise this answer does not apply. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that your questions can be answered with some simulations where we can test with some &quot;genes&quot; with a known relationship:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(&quot;limma&quot;)&#xA;set.seed(123)&#xA;&#xA;# Create some genes and samples&#xA;nGenes &amp;lt;- 40&#xA;nSamples &amp;lt;- 10&#xA;expr &amp;lt;- matrix(rnorm(nGenes*nSamples), ncol = nSamples, nrow = nGenes)&#xA;genes &amp;lt;- combn(LETTERS, 2, FUN = paste, collapse = &quot;&quot;)&#xA;rownames(expr) &amp;lt;- genes[seq_len(nGenes)]&#xA;colnames(expr) &amp;lt;- letters[seq_len(nSamples)]&#xA;&#xA;# Create some other genes with slightly higher values &#xA;expr2 &amp;lt;- expr+0.5&#xA;rownames(expr2) &amp;lt;- genes[seq(41, to = 80)]&#xA;expr &amp;lt;- rbind(expr, expr2)&#xA;&#xA;# Some genes are more expressed in the last 5 samples&#xA;expr[41:80, 5:10] &amp;lt;- expr[41:80, 5:10] + 2&#xA;&#xA;# Heatmap of the simulated genes and samples&#xA;heatmap(expr, scale = &quot;row&quot;, main = &quot;Heatmap&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9doOR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9doOR.png&quot; alt=&quot;Heatmap of the genes and samples&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Create a design matrix and &#xA;design &amp;lt;- matrix(c(rep(1, 10), rep(0, 5), rep(1, 5)), &#xA;                 ncol = 2, nrow = 10)&#xA;fit &amp;lt;- lmFit(expr, design)&#xA;fit2 &amp;lt;- eBayes(fit)&#xA;tt &amp;lt;- topTable(fit2, coef = 2, number = Inf, sort.by = &quot;none&quot;)&#xA;&#xA;randomGeneSet &amp;lt;- sample(1:80, 5, replace = FALSE)&#xA;# Calculate the enrichment with geneSetTest&#xA;geneSetTest(1:40, tt$t)&#xA;## [1] 1&#xA;geneSetTest(41:80, tt$t)&#xA;## [1] 9.043842e-13&#xA;geneSetTest(randomGeneSet, tt$t)&#xA;## [1] 0.9730717&#xA;&#xA;# Explore the interGeneCorrelation and the  variance inflation factor&#xA;interGeneCorrelation(expr[1:40, ], design)&#xA;## $vif&#xA;## [1] 0.2055766&#xA;## &#xA;## $correlation&#xA;## [1] -0.02036983&#xA;## &#xA;interGeneCorrelation(expr[41:80, ], design)&#xA;## $vif&#xA;## [1] 12.79611&#xA;## &#xA;## $correlation&#xA;## [1] 0.3024643&#xA;## &#xA;interGeneCorrelation(expr[randomGeneSet, ], design)&#xA;## $vif&#xA;## [1] 0.3526011&#xA;## &#xA;## $correlation&#xA;## [1] -0.1618497&#xA;&#xA;# Calculate the enrichment with camera&#xA;camera(expr, 1:40, design, contrast = 2, inter.gene.cor = NA)&#xA;##      NGenes Correlation Direction     PValue&#xA;## set1     40 -0.02036983      Down 3.0445e-06&#xA;camera(expr, 41:80, design, contrast = 2, inter.gene.cor = NA)&#xA;##      NGenes Correlation Direction      PValue&#xA;## set1     40   0.3024643        Up 0.002402933&#xA;camera(expr, randomGeneSet, design, contrast = 2, inter.gene.cor = NA)&#xA;##      NGenes Correlation Direction     PValue&#xA;## set1      5  -0.1618497      Down 0.04272392&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this simulation we created three gene sets were the p-value of both camera and geneSetTest where significant, one where only camera was significant and another one where both were below the 0.05 threshold of significance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see we can calculate the variance inflation factor for each gene set, which also yields the intercorrelation for each gene set we can summarize it into a table:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;               camera    VIF    correlation  geneSetTest&#xA;GS1 (1:40)     3.0e-06   0.203     -0.020     1.0&#xA;GS2 (41:80)    0.024     12.8       0.30      9.0e-13&#xA;randomGeneSet  0.043     0.353     -0.16      0.97&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As GS1 and GS2 are the complete opposite we see that they have quite different inter correlation, which leads to a different VIF. We can see that higher VIF or small doesn't mean that the gene set has lower p-value for camera.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How can I interpret the opposite case, where I find a significant CAMERA p-value but an insignificant mean-rank gene set test p-value? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am unsure how to interpret that comparison. It would be better to explore this  with real data (for example from ALL library), and contact the authors, either via the &lt;a href=&quot;https://support.bioconductor.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;support site in Bioconductor&lt;/a&gt; or through personal communications. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Does this imply that my query set is less correlated than expected, or that non-query genes are more highly correlated with one another?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As you might have noticed I used &lt;code&gt;inter.gene.cor = NA&lt;/code&gt; in order to estimate the correlations and avoid guessing the correlation between the genes but still the correction of intercorrelations has lead to higher p-values. So no, a wrong estimation of the correlation of the query set doesn't explain why the p-value is lower. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I only used the unranked method of the limma 3.34.1, I didn't use the preranked camera (&lt;code&gt;cameraPR&lt;/code&gt;) because it doesn't have the same results as in &lt;code&gt;camera&lt;/code&gt; and it doesn't allow to estimate the intercorrelation.&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-12-04T21:08:23.367" LastActivityDate="2017-12-04T21:08:23.367" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="2980" PostTypeId="2" ParentId="2970" CreationDate="2017-12-02T18:07:03.620" Score="2" Body="&lt;p&gt;This doesn't use R, but a simple and quite convenient method is to use &lt;code&gt;bedtools getfasta&lt;/code&gt; with your fasta file. You'll need to make a BED file out of the RepeatMasker output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN{OFS=&quot;\t&quot;}{name=sprintf(&quot;%s_%s&quot;, $10, $11); strand=&quot;+&quot;; if($9 == &quot;C&quot;){strand=&quot;-&quot;}; print $5,$6-1,$7,name,&quot;.&quot;, strand}' repeatmasker.output &amp;gt; rmsk.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, given that and your &lt;code&gt;genome.fa&lt;/code&gt; fasta:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools getfasta -name -s -fi genome.fa -bed rmsk.bed -fo output.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will produce a multiline fasta file names like &lt;code&gt;AluSx_SINE/Alu&lt;/code&gt; (due to the &lt;code&gt;-name&lt;/code&gt; option) where the sequences are reverse complemented if appropriate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can install bedtools either via conda (&lt;code&gt;conda install -c bioconda bedtools&lt;/code&gt;) or &lt;a href=&quot;https://github.com/arq5x/bedtools2/releases/download/v2.26.0/bedtools-2.26.0.tar.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;github&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-12-02T18:07:03.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2981" PostTypeId="1" CreationDate="2017-12-02T20:35:38.263" Score="7" ViewCount="308" Body="&lt;p&gt;I've downloaded reads from &lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/sra.cgi?run=SRR3385327&quot; rel=&quot;noreferrer&quot;&gt;this BioProject&lt;/a&gt;. Using canu with default parameters (no correction), I've got 4 contigs, none of which really look like the reference plasmid &lt;a href=&quot;https://www.ebi.ac.uk/ena/data/view/KX156772&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The command I used was:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;canu -p ip40a -d ip40a_assembly -useGrid=false -maxThreads=6 -genomeSize=175k -nanopore-raw reads.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Granted, the original publication for this sequence used minimap/miniasm but my questions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A. Why do I get multiple contigs? Doesn't canu generate a single assembled sequence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B. This is a plasmid but canu says &lt;code&gt;suggestedCircular=false&lt;/code&gt;. What's going on here?&lt;/p&gt;&#xA;" OwnerUserId="1947" LastEditorUserId="1947" LastEditDate="2017-12-03T00:47:44.800" LastActivityDate="2017-12-03T05:39:02.240" Title="Canu assembly not making a single consensus?" Tags="|nanopore|fastq|canu|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2982" PostTypeId="2" ParentId="2981" CreationDate="2017-12-02T22:09:14.673" Score="3" Body="&lt;p&gt;Looking that the link you provided only 30% of the reads appear to be e. coli.  I would suggest filtering the reads for those that align to e. coli and see where that gets you.  Did the original publication describe the method they used for data cleaning? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;https://trace.ncbi.nlm.nih.gov/Traces/sra/sra.cgi?run=SRR3385327&#xA;&#xA;Unidentified reads: 68.82%&#xA;Identified reads: 31.18%&#xA;cellular organisms: 30%&#xA;Bacteria: 30%&#xA;Proteobacteria: 28.09%&#xA;Gammaproteobacteria: 19.7%&#xA;Enterobacterales: 13.3%&#xA;Enterobacteriaceae: 12.54%&#xA;Vibrionales: 0.01%&#xA;Viruses: 1.17%&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;B: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without a good assembly canu won't be able to determine if the assembled genome is circular.  I would ignore this until you have a good example.&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2017-12-02T22:09:14.673" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="2983" PostTypeId="2" ParentId="2981" CreationDate="2017-12-03T00:37:18.400" Score="1" Body="&lt;p&gt;The &lt;em&gt;E. coli&lt;/em&gt; genome is closer to 4.6 Mb in size; if there are a large quantity of host reads in there, then they'll take over the reads and there won't be enough for your target plasmid. By default, it will only try to assemble from a maximum of ~40X coverage of the specified genome size (higher values do not add any more information for assembly).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's possible that miniasm will take all reads into account, rather than just a portion of reads. You may need to filter out the &lt;em&gt;E. coli&lt;/em&gt; reads for this assembly to work properly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For what it's worth, Canu's default parameters &lt;em&gt;do&lt;/em&gt; include a correction step.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-12-03T05:39:02.240" LastActivityDate="2017-12-03T05:39:02.240" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2984" PostTypeId="1" CreationDate="2017-12-03T17:14:21.787" Score="1" ViewCount="458" Body="&lt;p&gt;Assume I want to run a simple machine learning of healthy/non healthy classification, how can I use &lt;a href=&quot;https://cancergenome.nih.gov/cancersselected&quot; rel=&quot;nofollow noreferrer&quot;&gt;TCGA&lt;/a&gt; where it has only data of people with cancer. What do usually people do for example for study of gene expressions?&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2020-11-15T03:21:23.330" LastActivityDate="2020-11-15T03:21:23.330" Title="What would be a reasonable healthy data set/control group to compare with a cell line from TCGA?" Tags="|database|cancer|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="2986" PostTypeId="2" ParentId="2984" CreationDate="2017-12-04T09:51:04.930" Score="2" Body="&lt;p&gt;In the TCGA datasets there is a variable (type) containing the information if the sample was from the tumor or from an adjacent region, which is usually considered as healthy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As this samples come from the same patient and the same tissues they are used in the analysis for the comparisons and classifications.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2017-12-04T09:51:04.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2988" PostTypeId="1" AcceptedAnswerId="2990" CreationDate="2017-12-04T14:43:56.207" Score="5" ViewCount="857" Body="&lt;p&gt;I am looking at the &lt;code&gt;bwamem.h&lt;/code&gt; code in &lt;a href=&quot;http://github.com/lh3/bwa&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://github.com/lh3/bwa&lt;/a&gt; and found that BWA-MEM will give flag 0x200 to what it calls &lt;code&gt;MEM_F_SOFTCLIP&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#define MEM_F_PE        0x2&#xA;#define MEM_F_NOPAIRING 0x4&#xA;#define MEM_F_ALL       0x8&#xA;#define MEM_F_NO_MULTI  0x10&#xA;#define MEM_F_NO_RESCUE 0x20&#xA;#define MEM_F_REF_HDR   0x100&#xA;#define MEM_F_SOFTCLIP  0x200&#xA;#define MEM_F_SMARTPE   0x400&#xA;#define MEM_F_PRIMARY5  0x800&#xA;#define MEM_F_KEEP_SUPP_MAPQ 0x1000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But looking at the explanation in this website: &lt;a href=&quot;https://broadinstitute.github.io/picard/explain-flags.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://broadinstitute.github.io/picard/explain-flags.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It labels &lt;code&gt;0x200&lt;/code&gt; as read fails platform/vendor quality checks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we have reason to believe that soft clipping can happen for some of the alignments, should we just ignore the fact that IGV calls this fail QC? Does the failQC tag come from the SAM/BAM specification or is this something IGV came up with?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The SAM/BAM specification labels &lt;code&gt;0x200&lt;/code&gt; the same way as the Picard flag explaining website:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MU8rk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MU8rk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The question remains as to why BWA-MEM uses the &lt;code&gt;0x200&lt;/code&gt; flag for softclipping notation rather than Fail QC.&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="77" LastEditDate="2017-12-04T14:50:14.493" LastActivityDate="2017-12-04T17:14:27.360" Title="Interpreting 0x200 flag in bwa-mem alignments" Tags="|bwa|sam|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2990" PostTypeId="2" ParentId="2988" CreationDate="2017-12-04T15:01:06.560" Score="3" Body="&lt;p&gt;bwa mem is using exactly the same meaning of the &lt;code&gt;0x200&lt;/code&gt; flag as every other program, including picard. Don't blindly assume that that entry in the header file relates to the output flags in the SAM output. If you &lt;a href=&quot;https://github.com/lh3/bwa/blob/b58281621136a0ce2a66837ba509716c727b9387/bwamem.c#L818&quot; rel=&quot;nofollow noreferrer&quot;&gt;search the code&lt;/a&gt;, you'll see that bwa is using this internally as a flag for &quot;this alignment is soft-clipped&quot;, which it handles differently for primary and secondary alignments (hard-clipping the latter).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For what it's worth, &lt;code&gt;bwa mem&lt;/code&gt; (at least the version I just tried) doesn't set the QC failed flag. This behavour is actually pretty common among aligners, since Illumina has changed how they represent QC failure at least once.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-12-04T17:14:27.360" LastActivityDate="2017-12-04T17:14:27.360" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="2991" PostTypeId="2" ParentId="2988" CreationDate="2017-12-04T15:08:11.167" Score="0" Body="&lt;p&gt;The connection between soft clipping and a read level flag is interesting.  Soft clipping occurs at a per base level and is denoted in the CIGAR string.  The failQC flag is applied to a read and usually comes directly from the sequencer. It is likely that BWA just named their internal var weird.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Does the failQC tag come from the SAM/BAM specification or is this something IGV came up with?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The failQC flag is part of the &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sam Spec&lt;/a&gt;.    &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If we have reason to believe that soft clipping can happen for some of the alignments, should we just ignore the fact that IGV calls this fail QC?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you expect soft clipping I would suggest trimming that sequence from the reads prior to alignment.  Also you can look at read in the input reads and see if the failQC flag is set prior to alignment to understand where it is getting set.  IGV is just displaying what is in the BAM file not making any decisions about those reads.&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2017-12-04T15:08:11.167" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="2993" PostTypeId="1" CreationDate="2017-12-04T19:41:58.707" Score="5" ViewCount="45" Body="&lt;p&gt;I have a question about CAFA1 and CAFA2 experiment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;CAFA is an experiment that compare computational methods predicting protein functions. In &lt;a href=&quot;https://arxiv.org/abs/1601.00891&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper&lt;/a&gt; discussing CAFA2, there is an image comparing 2 methods (CAFA1 versus CAFA2):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xMbXq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xMbXq.png&quot; alt=&quot;Figure 8 from paper&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I interpret the numbers and colors (dark and light)? What do they mean?&lt;/p&gt;&#xA;" OwnerUserId="1978" LastEditorUserId="48" LastEditDate="2017-12-05T08:57:23.987" LastActivityDate="2017-12-05T08:57:23.987" Title="About CAFA experiment" Tags="|figure-explanation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2994" PostTypeId="2" ParentId="2993" CreationDate="2017-12-04T22:10:15.407" Score="5" Body="&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; that in the &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-1037-6&quot; rel=&quot;noreferrer&quot;&gt;article published&lt;/a&gt; that figure is different from the preprint!&lt;br&gt;&#xA;See the figure:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/QmrVD.gif&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QmrVD.gif&quot; alt=&quot;Figure 3 of the paper equivalent to the figure 8 of the preprint&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As they aren't substantially different in display other than showing the two set of predictions that are comparable between CAFA1 and CAFA2 I explain them below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The colors are the margins of $F_{max}$ they refer to the equation found in the section &quot;Comparison between CAFA1 and CAFA2 methods&quot; under methods:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\delta(m_2, m_1) = \frac{1}{B}\sum_{b=1}^{B}F_{max}^{b}(m_2)-\frac{1}{B}\sum_{b=1}^{B}F_{max}^{b}(m_1)$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where B is the 10000 bootstraps they did and $m_1$ and $m_2$ are the CAFA1 and CAFA2 methods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where $F_{max}$ is explained in the article (and in the preprint) as:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A perfect predictor would be characterized with $F_{max} = 1 $&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, the higher the (positive) difference is, the better precision and recall (&lt;a href=&quot;https://en.wikipedia.org/wiki/F1_score&quot; rel=&quot;noreferrer&quot;&gt;See wikipedia&lt;/a&gt;) are for those methods. Note that the scale is from -0.1 to 0.1 not from -1 to 1 so the differences are quite small (One could argue that the log10 would be better to see those small differences)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One can appreciate that in the bar plots at the right, where the differences between each dataset are small in the B section, so the color is white, and in the C panels they have brighter color. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The numbers in the boxes are the percentage that a CAFA2 method did better than a method in CAFA1 for another dataset in the &quot;intersection of the two GO snapshots&quot; as noted in the methods section. (But it is also not clear to me)&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="77" LastEditDate="2017-12-04T22:12:25.767" LastActivityDate="2017-12-04T22:12:25.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2995" PostTypeId="2" ParentId="2971" CreationDate="2017-12-05T10:30:27.887" Score="1" Body="&lt;p&gt;Albacore can be used to create a fastq, directly from your fast5 files. As such there is no more reason to use poretools, expect perhaps for getting some metadata. With regards to metadata I would like to add that the sequencing_summary.txt generated by albacore also contains a lot of metadata.&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2017-12-05T10:30:27.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2996" PostTypeId="1" CreationDate="2017-12-05T11:26:09.653" Score="4" ViewCount="122" Body="&lt;p&gt;I'm running stageR as part of a pipeline based on DRIMSeq and wanted to know what it means when the transcript level padj is equal to 0, particularly when the gene is non-significant. I'm pretty sure that in some cases if the padj is 0 it means that the chance of it being different due to chance is 0. There are a lot more 0s than I would expect in sites where the gene padj is non-significant. One example, I have a gene padj that's &gt;0.9 but both of the sites I'm looking at have a padj of 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a couple of suspicions about why this is: either because it defaults to this if the comparative difference is or it uses 0 as a substitute to NA. Am I thinking along the right lines? Are the 0s real or are they likely to be a quirk of the package?&lt;/p&gt;&#xA;" OwnerUserId="1843" LastActivityDate="2017-12-05T14:05:53.473" Title="What do 0s in stageR mean?" Tags="|r|rna-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2997" PostTypeId="2" ParentId="2817" CreationDate="2017-12-05T13:14:45.593" Score="1" Body="&lt;p&gt;This can also be done quite easily with awk:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;awk '{if(NR%2){printf(&quot;%s\t&quot;,$1)}else{print}}' filename | sort -k2,2| awk '{if($2==SEQ){gsub(&quot;&amp;gt;&quot;,&quot;&quot;,$1);ID=ID&quot;|&quot;$1} else{if(SEQ!=&quot;&quot;){printf(&quot;%s\n%s\n&quot;, ID,SEQ);}SEQ=$2;ID=$1;} }END{printf(&quot;%s\n%s\n&quot;, ID,SEQ)}'&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first &lt;code&gt;awk&lt;/code&gt; converts the fasta file to a tab separated file with format &lt;code&gt;ID\tSequence&lt;/code&gt;, which is then sorted by sequence by &lt;code&gt;sort&lt;/code&gt;. The last &lt;code&gt;awk&lt;/code&gt; goes through the sorted file looking at the sequences: if the sequence in the current line is the same as that in the previous line, it appends its id to the ID variable. Otherwise (i.e. we have encountered a new sequence), it prints the previous concatenated ids and the corresponding sequence. &lt;/p&gt;&#xA;" OwnerUserId="1835" LastActivityDate="2017-12-05T13:14:45.593" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2998" PostTypeId="2" ParentId="2817" CreationDate="2017-12-05T13:44:01.903" Score="2" Body="&lt;p&gt;using &lt;strong&gt;datamash&lt;/strong&gt; ans some awk&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat input.fa | paste - -  |\&#xA;cut -c2- | sort -t $'\t' -k2,2 |\&#xA;datamash  collapse 1 -g 2  | tr &quot;,&quot; &quot;_&quot; |\&#xA;awk '{printf(&quot;&amp;gt;%s\n%s\n&quot;,$2,$1);}'&#xA;&#xA;&amp;gt;prot_id_1_prot_id_5_prot_id_7&#xA;MAWIGLDISKLFVENSRDAAA&#xA;&amp;gt;prot_id_4&#xA;MEMVLSSANPSTTFADSYVV&#xA;&amp;gt;prot_id_2_prot_id_6&#xA;MENVSRIWRYQQRRVLSRAFTHWYLMGLTKHNHPS&#xA;&amp;gt;prot_id_3&#xA;MRTRPSRDFHYIVGRITRESPEEKGIFYVFH&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastActivityDate="2017-12-05T13:44:01.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="2999" PostTypeId="2" ParentId="2996" CreationDate="2017-12-05T14:01:58.927" Score="4" Body="&lt;p&gt;In stageR, we automatically return stage-wise adjusted p-values of zero for transcripts coming from genes that were found to be significant and only have 1 transcript, when testing for differential transcript expression (DTE).&#xA;Similarly, when testing for differential transcript usage (DTU), we return p-values of zero for transcripts derived from a significant gene consisting of no more than two transcripts (based on the tx2gene object you provide).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Transcript-level p-values for non-significant genes should not occur. We provide NA's for genes with only one transcript for DTU, and by default you should get NA p-values for non-significant genes, since these genes do not proceed further than the screening stage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you provide some more details on the analysis? Which multiple testing correction have you selected?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be possible to provide a reproducible example for your analysis?&lt;/p&gt;&#xA;" OwnerUserId="1995" LastEditorUserId="77" LastEditDate="2017-12-05T14:05:53.473" LastActivityDate="2017-12-05T14:05:53.473" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3000" PostTypeId="1" AcceptedAnswerId="3002" CreationDate="2017-11-28T02:05:00.323" Score="3" ViewCount="1045" Body="&lt;p&gt;When I am trying to install the RcppArmadillo Bioconductor library in R, it is giving me the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/usr/bin/ld: cannot find -lgfortran&#xA;collect2: error: ld returned 1 exit status&#xA;make: *** [RcppArmadillo.so] Error 1&#xA;ERROR: compilation failed for package ‘RcppArmadillo’&#xA;* removing ‘/home/nikita/R/x86_64-pc-linux-gnu-library/3.2/RcppArmadillo’&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am using Ubuntu 12.04, R 3.2.5 and &lt;code&gt;g++-5.4&lt;/code&gt;. Any help would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2005" OwnerDisplayName="Nikita Vlasenko" LastEditorUserId="29" LastEditDate="2017-12-07T11:58:16.043" LastActivityDate="2017-12-07T11:58:16.043" Title="RcppArmadillo installation error" Tags="|r|software-installation|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3001" PostTypeId="2" ParentId="3000" CreationDate="2017-11-28T02:27:01.890" Score="2" Body="&lt;p&gt;On Ubuntu 14.04 there is a direct route to installing RcppArmadillo. You could perhaps try the same syntax.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sudo apt-get update&#xA;sudo apt-get install r-cran-rcpparmadillo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="Ankur Chakravarthy" LastActivityDate="2017-11-28T02:27:01.890" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3002" PostTypeId="2" ParentId="3000" CreationDate="2017-11-28T03:14:17.077" Score="1" Body="&lt;p&gt;I found a similar question on &lt;a href=&quot;https://stackoverflow.com/questions/6302209/building-r-package-getting-error-ld-cannot-find-lgfortran&quot;&gt;stackoverflow&lt;/a&gt; and the solution was to create a symlink as described by &lt;code&gt;kevin&lt;/code&gt; there:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sudo ln -s /usr/lib/x86_64-linux-gnu/libgfortran.so.3 /usr/lib/libgfortran.so&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2005" OwnerDisplayName="Nikita Vlasenko" LastEditorUserId="48" LastEditDate="2017-12-06T09:03:23.027" LastActivityDate="2017-12-06T09:03:23.027" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3003" PostTypeId="1" CreationDate="2017-12-05T21:22:30.530" Score="3" ViewCount="601" Body="&lt;p&gt;I have basecalled ONT reads and converted them to multifasta. The multifasta contains the original ONT headers in this format:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;3ebb56cd-3671-4abd-b1ac-0c759bt068d0 runid=eb6214851489c8e00eb0dcbd00d737f7dddxxxx read=1078 ch=422 start_time=2017-11-21T20:35:34Z&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By checking the sequencing output as &lt;code&gt;fasta&lt;/code&gt;, the sequences are not ordered by start_time.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;f546fab3-39b8-4efb-9bc7-cf6ab19cf6dd runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=363 start_time=2017-11-21T20:36:34Z&#xA;TTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATG&#xA;&amp;gt;fa9f83ee-ec58-419b-aff7-e40604cb7a34 runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=183 start_time=2017-11-21T20:35:09Z&#xA;TGGTATGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATG &#xA;&amp;gt;371a075a-6701-4187-affc-7aa267c41850 runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=315 start_time=2017-11-21T20:35:57Z&#xA;CTCGTTTCAGGTATGCTTCGTTTCAGTTGCCCTCAGGAGCAGATTGTTTATGGGTTACTTTCCTGTCGCTCTATCTTCAC &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I learned how to extract header IDs and &lt;code&gt;time_start&lt;/code&gt; from ONT fastq files by using &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2175/1421&quot;&gt;this answer&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to have a similar approach but with fasta. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yet, I would like to create a new fasta file where the sequences are written in an ascending order as they were sequenced (&lt;code&gt;start_time&lt;/code&gt;) as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;fa9f83ee-ec58-419b-aff7-e40604cb7a34 runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=183 start_time=2017-11-21T20:35:09Z&#xA;TGGTATGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATG &#xA;&amp;gt;371a075a-6701-4187-affc-7aa267c41850 runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=315 start_time=2017-11-21T20:35:57Z&#xA;CTCGTTTCAGGTATGCTTCGTTTCAGTTGCCCTCAGGAGCAGATTGTTTATGGGTTACTTTCCTGTCGCTCTATCTTCAC&#xA;&amp;gt;f546fab3-39b8-4efb-9bc7-cf6ab19cf6dd runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=363 start_time=2017-11-21T20:36:34Z&#xA;TTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1421" LastEditorUserId="298" LastEditDate="2017-12-07T09:05:58.453" LastActivityDate="2017-12-10T20:25:56.083" Title="sort a fasta file containing the Oxford Nanopore Technologies (ONT) header by sequencing start_time ascending" Tags="|fasta|nanopore|ngs|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3004" PostTypeId="1" AcceptedAnswerId="3017" CreationDate="2017-12-05T22:35:11.973" Score="7" ViewCount="1024" Body="&lt;p&gt;I am trying to map single-cell chromatin accessibility data from doi:10.1038/nature14590, obtained using scATAC-seq, to the reference genome. Example paired-end reads are &lt;a href=&quot;ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR178/004/SRR1780164/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reads contain various adapter sequences, which the authors &quot;trimmed from FASTQs using custom python scripts&quot; [&lt;a href=&quot;https://images.nature.com/original/nature-assets/nature/journal/v523/n7561/extref/nature14590-s1.pdf&quot; rel=&quot;noreferrer&quot;&gt;ref&lt;/a&gt;]. I am having problems reproducing this trimming. They used Nextera Transposase Adapters&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;TCGTCGGCAGCGTCAGATGTGTATAAGAGACAG&#xA;  GTCTCGTGGGCTCGGAGATGTGTATAAGAGACAG&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and custom barcode adapters which are listed [&lt;a href=&quot;https://images.nature.com/original/nature-assets/nature/journal/v523/n7561/extref/nature14590-s2.xlsx&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;]. An excerpt from this list:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Custom Barcodes Adapter 1 (index i5):&#xA;  AATGATACGGCGACCACCGAGATCTACAC&lt;strong&gt;TAGATCGC&lt;/strong&gt;TCGTCGGCAGCGTCAGATGTGTAT&#xA;  AATGATACGGCGACCACCGAGATCTACAC&lt;strong&gt;TAACCAAG&lt;/strong&gt;TCGTCGGCAGCGTCAGATGTGTAT&#xA;  AATGATACGGCGACCACCGAGATCTACAC&lt;strong&gt;AGGTTGCC&lt;/strong&gt;TCGTCGGCAGCGTCAGATGTGTAT&#xA;  Custom Barcodes Adapter 2 (index i7):&#xA;  CAAGCAGAAGACGGCATACGAGAT&lt;strong&gt;ACCCAGCA&lt;/strong&gt;GTCTCGTGGGCTCGGAGATGTG&#xA;  CAAGCAGAAGACGGCATACGAGAT&lt;strong&gt;CCCAACCT&lt;/strong&gt;GTCTCGTGGGCTCGGAGATGTG&#xA;  CAAGCAGAAGACGGCATACGAGAT&lt;strong&gt;CACCACAC&lt;/strong&gt;GTCTCGTGGGCTCGGAGATGTG&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Analyzing the FASTA-files with FastQC showed that the reverse complement of those sequences was over-represented in the data and also confirmed Nextera Transposase Adapters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to trim the sequences using &lt;code&gt;cutadapt&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;cutadapt --minimum-length=20 -a CACATCTCCGAGCCCACGAGAC -a CTGTCTCTTATA -A ATACACATCTGACGCTGCCGACGA -A CTGTCTCTTATA -o SRR1780164_1_trimmed.fastq.gz -p SRR1780164_2_trimmed.fastq.gz SRR1780164_1.fastq.gz SRR1780164_2.fastq.gz&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;CACATCTCCGAGCCCACGAGAC&lt;/code&gt; being the reverse complement of the 2nd fix part of the i7 adapters&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;ATACACATCTGACGCTGCCGACGA&lt;/code&gt; being the reverse complement of the 2nd fix part of the i5 adapters&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;CTGTCTCTTATA&lt;/code&gt; for the Nextera Transposase Adapters&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;FastQC does not indicate overrepresented sequences in the resulting FASTA-files, however still indicates Nextera Transposase Adapters in the _1 files. On the other hand, after TrimGalore removes &lt;code&gt;CTGTCTCTTATA&lt;/code&gt;, Nextra Adapters are not indicated anymore, for some reason.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, mapping the reads using &lt;code&gt;bowtie2&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;bowtie2 -x GRCh38_primary_assembly_index -1 SRR1780164_1_trimmed.fastq.gz -2 SRR1780164_2_trimmed.fastq.gz &amp;gt; SRR1780164.sam&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;results in a low alignment rate&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1352 (2.79%) aligned concordantly exactly 1 time&#xA;3420 (7.07%) aligned concordantly &amp;gt;1 times&#xA;11.25% overall alignment rate&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which gives me the impression I did something wrong during adapter trimming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be much appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;--&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: As I am not sure about the sequences to use for trimming, I also tried &lt;code&gt;-a CTGTCTCTTATACACATCTCCGAGCCCACGAGAC -A CTGTCTCTTATACACATCTGACGCTGCCGACGA&lt;/code&gt;, as found in a suggestion for Nextera Adapters, however it did not remove the sequence over-representation in the -1 file. &lt;code&gt;-a CACATCTCCGAGCCCACGAGAC -a CTGTCTCTTATACACATCTCCGAGCCCACGAGAC -A CTGTCTCTTATACACATCTGACGCTGCCGACGA&lt;/code&gt; solves adapter-issues according to FastQC, but also does not give good mapping results.&lt;/p&gt;&#xA;" OwnerUserId="1998" LastEditorUserId="1998" LastEditDate="2017-12-06T01:09:32.130" LastActivityDate="2017-12-08T04:14:38.370" Title="How to trim adapter sequences from GSE65360 in order to map the reads?" Tags="|read-mapping|single-cell|trimming|adapter|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3006" PostTypeId="2" ParentId="3003" CreationDate="2017-12-06T00:32:47.110" Score="3" Body="&lt;p&gt;Edit: After doing some reading, if it can be assumed that all the timestamps are ISO8601 and of the same representation of ISO861 (there are a few), then they can be sorted lexicographically, directly, without converting to epoch-time integers. So I have edited this answer to add an answer without conversion steps:&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here's one approach that uses the timestamps directly:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) Extract the start times from the headers and sort them to another file, in ascending order:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk '(/^&amp;gt;/){ match($0, /start_time=(.*)/, a); print a[1]; }' my.fa | sort &amp;gt; starts_iso8601.sorted.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(2) Use that file of ordered ISO8601-formatted starts to write out FASTA records in your sort order:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ while IFS='' read -r line || [[ -n &quot;$line&quot; ]]; do awk -v line=$line -v RS=&quot;&amp;gt;&quot; '{ m = match($0,line); if (m) { printf(&quot;&amp;gt;%s&quot;, $0); } }' my.fa ; done &amp;lt; starts_iso8601.sorted.txt &amp;gt; my_ordered.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Quite a bit cleaner and faster. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might separate the timestamp extraction and sort steps into two commands and inspect the intermediate result, just to ensure that you have valid strings. Or use the approach below to convert the timestamps, which will catch bad input.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here's a second way that may add unnecessary work. As an advantage, however, it adds some error checking by converting the ISO8601-formatted timestamp. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the string you pull out of the header is not a valid timestamp, the first conversion step (ISO8601-to-epoch) will fail and you will then be able to catch invalid input pretty quickly, which can add confidence that your output is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) Extract the start times from the headers:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk '(/^&amp;gt;/){ match($0, /start_time=(.*)/, a); print a[1]; }' my.fa &amp;gt; starts_iso8601.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(2) Convert the ISO8601-formatted times to time-since-epoch times:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ while IFS='' read -r line || [[ -n &quot;$line&quot; ]]; do date -d&quot;$line&quot; +%s; done &amp;lt; starts_iso8601.txt &amp;gt; starts_epoch.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(3) Sort the epoch times — which are just integers — to another file, in ascending order:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sort -n starts_epoch.txt &amp;gt; starts_epoch.sorted.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(4) Convert sorted epoch starts back to ISO8601 format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ while IFS='' read -r line || [[ -n &quot;$line&quot; ]]; do date -d&quot;@$line&quot; -u +&quot;%Y-%m-%dT%H:%M:%SZ&quot;; done &amp;lt; starts_epoch.sorted.txt &amp;gt; starts_iso8601.sorted.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(5) Use that file of ordered starts to write out FASTA records in your sort order:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ while IFS='' read -r line || [[ -n &quot;$line&quot; ]]; do awk -v line=$line -v RS=&quot;&amp;gt;&quot; '{ m = match($0,line); if (m) { printf(&quot;&amp;gt;%s&quot;, $0); } }' my.fa ; done &amp;lt; starts_iso8601.sorted.txt &amp;gt; my_ordered.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note: Use &lt;code&gt;gawk&lt;/code&gt;, &lt;code&gt;gdate&lt;/code&gt;, &lt;code&gt;gsort&lt;/code&gt; in place of &lt;code&gt;awk&lt;/code&gt;, &lt;code&gt;date&lt;/code&gt; and &lt;code&gt;sort&lt;/code&gt; if you're using OS X, Homebrew, and GNU core utilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm showing the steps separately to explain the general procedure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many or all of these steps can be glued together in a Unix pipeline to save the time, memory and disk space costs of writing intermediate files. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I'm not doing much error checking here. You might look at cases where headers do not contain &lt;code&gt;start_time&lt;/code&gt;, or times are not all in ISO8601, or other cases not anticipated here.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2017-12-06T19:52:46.337" LastActivityDate="2017-12-06T19:52:46.337" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3007" PostTypeId="2" ParentId="3003" CreationDate="2017-12-06T00:54:12.217" Score="2" Body="&lt;p&gt;I've just made a small tweak to my &lt;a href=&quot;https://github.com/gringer/bioinfscripts/blob/master/fastx-sort.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;fastx-sort.pl&lt;/a&gt; script to get this working. It works on both fasta and fastq sequences, and should even work on a random mixture of the two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This script can now do a dictionary sort by the bits of an ID string that match an arbitrary search pattern, as well as the previous ID and length-based sorting:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fastx-sort.pl -p '&amp;lt;perl regex pattern&amp;gt;' &amp;lt;fastq file&amp;gt; &amp;gt; sorted.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's an example with your fasta input, sorting on the nanopore time string:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat seq.fa | grep start_time&#xA;&amp;gt;f546fab3-39b8-4efb-9bc7-cf6ab19cf6dd runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=363 start_time=2017-11-21T20:36:34Z&#xA;&amp;gt;fa9f83ee-ec58-419b-aff7-e40604cb7a34 runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=183 start_time=2017-11-21T20:35:09Z&#xA;&amp;gt;371a075a-6701-4187-affc-7aa267c41850 runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=315 start_time=2017-11-21T20:35:57Z&#xA;$ ~/scripts/fastx-sort.pl -p 'start_time=(.*?Z)' seq.fa | grep start_time&#xA;&amp;gt;fa9f83ee-ec58-419b-aff7-e40604cb7a34 runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=183 start_time=2017-11-21T20:35:09Z&#xA;&amp;gt;371a075a-6701-4187-affc-7aa267c41850 runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=315 start_time=2017-11-21T20:35:57Z&#xA;&amp;gt;f546fab3-39b8-4efb-9bc7-cf6ab19cf6dd runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=363 start_time=2017-11-21T20:36:34Z&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's a worked example with one of my fastq files for the nanopore time string:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat called_all.fastq | grep start_time | head -n 3&#xA;@3a151247-6720-4071-945e-7a92c1856e45 runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=9896 ch=246 start_time=2017-11-27T00:58:42Z&#xA;@52e439d6-f14a-4af2-adba-132d240fa967 runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=8634 ch=511 start_time=2017-11-27T00:58:41Z&#xA;@82e3dd50-c395-4ade-ab6b-0eed530d3c3f runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=8312 ch=299 start_time=2017-11-27T00:58:39Z&#xA;$ ~/scripts/fastx-sort.pl -p 'start_time=(.*?Z)' called_all.fastq | grep start_time | head -n 10&#xA;@195ad363-1d5a-40bf-abed-820c8acce8fc runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=9 ch=144 start_time=2017-11-26T22:42:32Z&#xA;@5df2fc6e-53c8-489c-8002-c97f151104bd runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=11 ch=129 start_time=2017-11-26T22:42:37Z&#xA;@23940fe6-0eef-418f-9156-d0a26df9bd80 runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=22 ch=330 start_time=2017-11-26T22:42:39Z&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This script now does a stable sort, so you can do things like sorting by channel, then by read number:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;~/scripts/fastx-sort.pl -n -p 'read=([0-9]+)' called_all.fastq | ~/scripts/fastx-sort.pl -n -p 'ch=([0-9]+)' | grep 'ch='&#xA;...&#xA;@876327ec-ccb0-44ad-a9fb-e008744d6c15 runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=5974 ch=66 start_time=2017-11-27T00:04:52Z&#xA;@e3ec5d92-ee8a-43d6-82ed-e359e0539b75 runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=6117 ch=66 start_time=2017-11-27T00:06:55Z&#xA;@410fe169-c8d1-4686-b92f-cbeb0602b2f0 runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=6280 ch=66 start_time=2017-11-27T00:09:26Z&#xA;@d27c2827-1746-4827-9fec-3075e08d3d64 runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=6375 ch=66 start_time=2017-11-27T00:10:44Z&#xA;@43079baa-8591-4498-bbfd-5f5ad677c011 runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=32 ch=69 start_time=2017-11-26T22:43:32Z&#xA;@2bd9b317-438d-4e45-958b-7f77df1dd51a runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=170 ch=69 start_time=2017-11-26T22:47:32Z&#xA;@e3599944-849a-492c-ad6d-8cc319b7c5d7 runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=207 ch=69 start_time=2017-11-26T22:49:11Z&#xA;@8b9d80d0-b28d-46bb-a4e9-6bba95a79ff4 runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=320 ch=71 start_time=2017-11-26T22:50:33Z&#xA;@b964f164-c36c-4f8a-8a1a-1ad196c996a3 runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=780 ch=71 start_time=2017-11-26T22:59:14Z&#xA;@92497c10-a46a-4a31-bc18-cfa334491ca9 runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=937 ch=71 start_time=2017-11-26T23:02:32Z&#xA;@2a715984-6e14-4f31-bd47-d36ee5cbc3e8 runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=2271 ch=71 start_time=2017-11-26T23:24:11Z&#xA;@ff2a59ae-4909-4f3e-bea2-b695d521bf5e runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=2441 ch=71 start_time=2017-11-26T23:27:01Z&#xA;@f106792d-31a4-44f2-a4fe-83f35a42a573 runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=2559 ch=71 start_time=2017-11-26T23:28:54Z&#xA;@8de1d1ca-c6f7-43f5-bbd6-3c79187c1bbe runid=256e8ca97106d2fef2538e4c078d1780a9ddb95a read=2930 ch=71 start_time=2017-11-26T23:34:33Z&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Additional notes:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In order to do the sort, the script currently stores the entirety of the input in memory.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-12-10T20:25:56.083" LastActivityDate="2017-12-10T20:25:56.083" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3008" PostTypeId="1" CreationDate="2017-12-06T10:56:59.087" Score="2" ViewCount="435" Body="&lt;p&gt;We have RNA-seq data from libraries prepared using the Smartseq2 single-cell protocol on &lt;strong&gt;500 cells&lt;/strong&gt; (mini-bulk) / library. The complexity is much better than with single-cells (~14k genes for 1.5M reads).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;6 cell types / biological replicates were collected with flow cytometry, and there are 4 control and 5 diseased biological replicates (patients).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are mostly interested in the differences between control and disease either overall or within each cell type.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can single-cell tools (e.g. Seurat) be used on this dataset? It is neither bulk nor single-cell data. How would you approach this? &lt;/p&gt;&#xA;" OwnerUserId="208" LastEditorUserId="298" LastEditDate="2017-12-06T11:05:20.523" LastActivityDate="2017-12-13T18:51:29.993" Title="Mini-bulk RNA-seq and scRNA-seq analysis differences?" Tags="|rna-seq|sequence-analysis|single-cell|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3009" PostTypeId="2" ParentId="3003" CreationDate="2017-12-07T09:22:56.180" Score="0" Body="&lt;p&gt;My go-to tools for any type of fasta file manipulation are these two simple awk scripts:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;FastaToTbl will convert fasta to &quot;tbl&quot; formato: the ID, a tab and then the sequence, all on the same line. &lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-C prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/awk -f&#xA;{&#xA;        if (substr($1,1,1)==&quot;&amp;gt;&quot;)&#xA;    		if (NR&amp;gt;1)&#xA;                    	printf &quot;\n%s\t&quot;, substr($0,2,length($0)-1)&#xA;    		else &#xA;    			printf &quot;%s\t&quot;, substr($0,2,length($0)-1)&#xA;            else &#xA;                    printf &quot;%s&quot;, $0&#xA;}END{printf &quot;\n&quot;}&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;TblToFasta will convert tbl back to fasta:&lt;/p&gt;&#xA;&#xA;   &lt;pre class=&quot;lang-C prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/awk -f&#xA;{&#xA;  sequence=$NF&#xA;&#xA;  ls = length(sequence)&#xA;  is = 1&#xA;  fld  = 1&#xA;  while (fld &amp;lt; NF)&#xA;  {&#xA;     if (fld == 1){printf &quot;&amp;gt;&quot;}&#xA;     printf &quot;%s &quot; , $fld&#xA;&#xA;     if (fld == NF-1)&#xA;      {&#xA;        printf &quot;\n&quot;&#xA;      }&#xA;      fld = fld+1&#xA;  }&#xA;  while (is &amp;lt;= ls)&#xA;  {&#xA;    printf &quot;%s\n&quot;, substr(sequence,is,60)&#xA;    is=is+60&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Armed with those and a bit of Perl, you can do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ FastaToTbl file.fa | &#xA;    perl -lne '/start_time=(\S+)/; &#xA;               $d=`date -d $1 +%s`; &#xA;               $k{$d}=$_; &#xA;               END{&#xA;                    print &quot;$k{$_}&quot; for sort(keys(%k))&#xA;               }' | TblToFasta&#xA;&amp;gt;fa9f83ee-ec58-419b-aff7-e40604cb7a34 runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=183 start_time=2017-11-21T20:35:09Z &#xA;TGGTATGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTAT&#xA;GTGGTATGTGGTATGTGGTATGTGGTATG&#xA;&amp;gt;371a075a-6701-4187-affc-7aa267c41850 runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=315 start_time=2017-11-21T20:35:57Z &#xA;CTCGTTTCAGGTATGCTTCGTTTCAGTTGCCCTCAGGAGCAGATTGTTTATGGGTTACTT&#xA;TCCTGTCGCTCTATCTTCAC&#xA;&amp;gt;f546fab3-39b8-4efb-9bc7-cf6ab19cf6dd runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=363 start_time=2017-11-21T20:36:34Z &#xA;TTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGG&#xA;TATGTTGGTATGTTGGTATGTTGGTATG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Explanation&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The perl script will take the tbl format sequences (so everything on one line), and identify the &lt;code&gt;start_time&lt;/code&gt;: &lt;code&gt;/start_time=(\S+)/&lt;/code&gt;. The parentheses capture the matched pattern so the start time will now be &lt;code&gt;$1&lt;/code&gt;. Then, we run the system &lt;code&gt;date&lt;/code&gt; command giving it the start time as the date and getting the result in seconds since the epoch (&lt;code&gt;%s&lt;/code&gt;). note that this assumes GNU &lt;code&gt;date&lt;/code&gt; (the default on Linux systems) which can handle the &lt;code&gt;-d&lt;/code&gt; option to specify a date. The next step is using &lt;code&gt;$d&lt;/code&gt;, the date in seconds, as a key for a hash whose values are the input lines: &lt;code&gt;$k{$d}=$_&lt;/code&gt;. Then, after the whole file has been processed, we print each entry of the hash in sorted order (&lt;code&gt;print &quot;$k{$_}&quot; for sort(keys(%k))&lt;/code&gt;). Finally, TblToFasta changes back to fasta format. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-12-07T09:22:56.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3010" PostTypeId="1" AcceptedAnswerId="3024" CreationDate="2017-12-07T13:31:27.203" Score="3" ViewCount="630" Body="&lt;p&gt;I am working on phylogenetic tree generation. I used BioPython and ClustalW 2.x.x for this purpose. I have generate the tree using BioPython but when I try to generate tree using &quot;MegaSoftware GUI&quot; tool my tree does not matched with the output of MegaSoftware.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I use &quot;No. of difference&quot; model then results are matched with biopython tree Like below image shown.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bUzsf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bUzsf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But whenever I choose &quot;p-distance&quot; model my results are not matched.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BZY2n.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BZY2n.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output MegaSoftware using p-distance&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/d3M5H.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/d3M5H.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output BioPython&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PP8ds.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PP8ds.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code for generating phylogenetic tree:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def conversion_fasta_dnd(clustalw, filepath, type, clustering, output, pwmatrix):&#xA;    &quot;&quot;&quot;&#xA;        ClustalW tool to conversion the file format.&#xA;        :param clustalw: path of the executable programm.&#xA;        :param filepath: path of `fasta` extension file.&#xA;        :param type: dna, protein&#xA;        :param clustering: upgma, nj&#xA;        :param output: clustal, phylip, fasta&#xA;        :param pwmatrix: blosum, pam, id&#xA;        :return: dnd, aln (files)&#xA;        &quot;&quot;&quot;&#xA;    # esblish the full path to the clustalW program&#xA;    clustalw_exe = clustalw&#xA;&#xA;    # check to make sure the program is there&#xA;    assert os.path.isfile(clustalw_exe), &quot;Clustal W executable missing&quot;&#xA;&#xA;    # create the appropriate command line&#xA;    clustalw_cline = ClustalwCommandline(clustalw_exe,&#xA;                                         infile=filepath,&#xA;                                         type=type,&#xA;                                         clustering=clustering,&#xA;                                         output=output,&#xA;                                         pwmatrix=pwmatrix)&#xA;&#xA;    # print the command line&#xA;    print(clustalw_cline)&#xA;&#xA;    # execute the command&#xA;    stdout, stderr = clustalw_cline()&#xA;&#xA;# This function draw the tree on image file&#xA;def simple_tree(filepath):&#xA;    &quot;&quot;&quot;&#xA;    Generate the simple tree image.&#xA;    :param filepath: path of dnd file format.&#xA;    &quot;&quot;&quot;&#xA;    def get_label(leaf):&#xA;        return leaf.name&#xA;&#xA;    file = filepath&#xA;    tree = Phylo.read(file, 'newick')&#xA;    Phylo.draw(tree, label_func=get_label, do_show=False, branch_labels=lambda c: c.branch_length)&#xA;&#xA;    # axis management&#xA;    pylab.axis('on')&#xA;    pylab.savefig(OUTPUT_IMAGE, format='png', bbox_inches='tight', dpi=100, bbox_extra_artists=[])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In above code I have used two things first &lt;a href=&quot;http://biopython.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioPhython&lt;/a&gt; lib and the other is &lt;a href=&quot;http://www.clustal.org/clustal2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clustal W&lt;/a&gt; for align the sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Could you please clarify what p-distances are?&lt;/strong&gt;&lt;br&gt;&#xA;I only have these reference for p-distance:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.megasoftware.net/web_help_7/hc_p_distance_nucleotide.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;p-distance (Nucleotide)&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Class/NAWBIS/Modules/Phylogenetics/phylo12a.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Proportional (p) distances&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I need to add p-distance model in my code, someone please help me out How can I do that.&lt;/p&gt;&#xA;" OwnerUserId="2012" LastEditorUserId="2012" LastEditDate="2017-12-08T05:14:21.887" LastActivityDate="2021-09-15T09:56:08.057" Title="How can I apply proportional (p) distances (Nucleotide) using bioPython" Tags="|python|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3011" PostTypeId="1" CreationDate="2017-12-07T16:31:10.507" Score="4" ViewCount="37" Body="&lt;p&gt;My group has a complex datset and I would like to validate the results and check the methods I use in a similar dataset (better if it has been already studied).&lt;/p&gt;&#xA;&lt;p&gt;Characteristics of my dataset:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;It is related to the inflammatory bowel disease (IBD)&lt;/p&gt;&#xA;&lt;p&gt;More specifically to Crohn's disease.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It has RNA-seq&lt;/p&gt;&#xA;&lt;p&gt;I have explored the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gds/?term=Inflammatory%20bowel%20disease&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;GEO query&quot;&gt;GEO&lt;/a&gt; and there are few projects (10) with RNA-seq on the tissue of my disease, and I don't know how to find the 16S-seq of those projects if it exists (see point below).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It has 16S-seq (and not only from the stools)&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://www.ebi.ac.uk/ena&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;ENA&quot;&gt;ENA&lt;/a&gt; seems to have 16S sequencing but most of them are from stools not from biopsies and still, I don't know how to find the RNA-seq of those projects if it exists (see point above)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It has both RNA-seq and 16S-seq from the &lt;em&gt;same&lt;/em&gt; individuals at the &lt;em&gt;same&lt;/em&gt; time point.&lt;/p&gt;&#xA;&lt;p&gt;The NIH &lt;a href=&quot;https://www.hmpdacc.org/&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;HMP&quot;&gt;Human Microbiome Project&lt;/a&gt; for my disease has only one individual with that both 16S-seq and RNa-seq of the same patient at the same time.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;How can I find a dataset similar to mine?&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-12-08T11:24:25.137" Title="Finding data to validate results" Tags="|human|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3012" PostTypeId="1" AcceptedAnswerId="3019" CreationDate="2017-12-07T16:47:17.387" Score="3" ViewCount="125" Body="&lt;p&gt;Not sure if this is allowed but I cannot think of a better place to ask. I am attempting to download all genomic sequences from refseq associated with the query Peptostreptococcaceae. I have been been using the following command (based on some reading of the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db nuccore -query &quot;Peptostreptococcaceae&quot; | efilter -molecule genomic -source refseq | efetch -format fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I keep getting the following error&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Unknown option: molecule   &#xA;Unknown option: source   &#xA;Must supply -query or -days or -mindate and -maxdate arguments on command &#xA;line Db value not found in fetch input No do_post output returned from &#xA;'http://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=nuccore&amp;amp;term=Peptostreptococcaceae&amp;amp;retmax=0&amp;amp;usehistory=y&amp;amp;edirect=3.80&amp;amp;tool=edirect&amp;amp;email=lgunn@sonic15' Result of do_post http request is .............&#xA;(a lot of text which I assume is .html or .xml related)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="982" LastEditorUserId="982" LastEditDate="2017-12-07T16:52:35.087" LastActivityDate="2017-12-12T11:02:00.980" Title="NCBI - edirect suite to download all genome sequences associated with a query - troubleshooting" Tags="|data-retrieval|edirect|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3013" PostTypeId="5" CreationDate="2017-12-07T16:48:10.417" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-12-07T16:48:10.417" LastActivityDate="2017-12-07T16:48:10.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3014" PostTypeId="4" CreationDate="2017-12-07T16:48:10.417" Score="0" Body="Entrez Direct (EDirect) provides access to the NCBI's suite of interconnected databases (publication, sequence, structure, gene, variation, expression, etc.) from a UNIX terminal window. Functions take search terms from command-line arguments" OwnerUserId="982" LastEditorUserId="982" LastEditDate="2017-12-14T07:56:08.710" LastActivityDate="2017-12-14T07:56:08.710" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3015" PostTypeId="1" CreationDate="2017-12-07T17:44:37.063" Score="3" ViewCount="59" Body="&lt;p&gt;I want a box representing each sequence, positioned as they are in the alignment and with gaps shown as breaks in the each box.&lt;/p&gt;&#xA;&lt;p&gt;I've been having trouble for a while with this and have been trying to get this on my own, but I can't. I'm trying to display the gaps using &lt;code&gt;Bio::Graphics&lt;/code&gt; after retrieving my sequences from an alignment file.&lt;/p&gt;&#xA;&lt;p&gt;I removed the exit portion. I was trying to format the code for this website and accidentally put the exit there. The input fill is a custalw file which is an alignment file with sequences on it. The sequences have &amp;quot;-&amp;quot; that represent gaps.&lt;/p&gt;&#xA;&lt;p&gt;The command is &lt;code&gt;./aln.pl test-bioaln.aln &amp;gt; ll.png&lt;/code&gt;&lt;br /&gt;&#xA;My expected output would be a .png file with sequences that are aligned and show gaps as a different color.&lt;/p&gt;&#xA;&lt;p&gt;The input file is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CLUSTAL W (1.81) multiple sequence alignment&#xA;&#xA;&#xA;JD1:1:102:1601:ORFJ00027      ------------------------------atgtataaacaacaatattttatttct--c&#xA;94a:1:107:117:orf00001        ------------------------------atgtataaacaacaatattttatttct-ac&#xA;118a:1:106:158122218:orf00020 ------------------------------atgtataaacaacaatattttatttct-gc&#xA;B31:1:100:4091:ORFB0018       ------------------------------atgtataaacaacaatattttatttctggc&#xA;72a:1:105:32:orf00022         ------------------------------atgtataaacaacaatattttatttctggc&#xA;64b:1:110:473:orf00001        ------------------------------atgtataaacaacaatattttatttctggc&#xA;29805:1:108:171:orf00001      ------------------------------atgtataaacaacaatattttatttctggc&#xA;BOL26:1:111:60:orf00001       ------------------------------atgtataaacaacaatattttatttctggc&#xA;CA-11.2A:1:109:33:orf00001    ------------------------------atgtataaacaacaatattttatttctggc&#xA;WI91-23:1:112:493:orf00001    ------------------------------atgtataaacaacaatattttatttctggc&#xA;297:1:103:411:ORFB00012       ttggatagattttatacaaagaaggtaataatgtataaacaacaatattttatttctggc&#xA;N40:1:101:1716:ORFK00021      ------------------------------atgtataaacaacaatattttatttctggc&#xA;ZS7:1:113:22:orf00001         ------------------------------atgtataaacaacaatattttatttctggc&#xA;                                                            ******************************&#xA;&#xA;&#xA;JD1:1:102:1601:ORFJ00027      aaggtgcaaggtgttggttttagatttttcacagagcaaatagcaaataatatgaaacta&#xA;94a:1:107:117:orf00001        aaggtacaaggtgttggttttagattttttacagagcaaatagcaaataatatgaaacta&#xA;118a:1:106:158122218:orf00020 aaggtacaaggtgttggttttagattttttacagagcaaatagcaaataatatgaaacta&#xA;B31:1:100:4091:ORFB0018       aaggtgcaaggtgttggttttagatttttcacagagcaaatagcaaataatatgaaacta&#xA;72a:1:105:32:orf00022         aaggtacaaggtgttggttttagattttttacagagcaaatagcaaataatatgaaacta&#xA;64b:1:110:473:orf00001        aaggtgcaaggtgttggttttagatttttcacagagcaaatagcaaataatatgaaacta&#xA;29805:1:108:171:orf00001      aaggtgcaaggtgttggttttagatttttcacagagcaaatagcaaataatatgaaacta&#xA;BOL26:1:111:60:orf00001       aaggtacaaggtgttggttttagattttttacagagcaaatagcaaataatatgaaacta&#xA;CA-11.2A:1:109:33:orf00001    aaggtacaaggtgttggttttagattttttacagagcaaatagcaaataatatgaaacta&#xA;WI91-23:1:112:493:orf00001    aaggtgcaaggtgttggttttagatttttcacagagcaaatagcaaataatatgaaacta&#xA;297:1:103:411:ORFB00012       aaggtacaaggtgttggttttagattttttacagagcaaatagcaaataatatgaaacta&#xA;N40:1:101:1716:ORFK00021      aaggtgcaaggtgttggttttagatttttcacagagcaaatagcaaataatatgaaacta&#xA;ZS7:1:113:22:orf00001         aaggtacaaggtgttggttttagattttttacagagcaaatagcaaataatatgaaacta&#xA;                              ***** *********************** ******************************&#xA;&#xA;&#xA;JD1:1:102:1601:ORFJ00027      aaaggatttgtaaaaaatctcaacgatggaagggtagaaattgtagctttctttaatact&#xA;94a:1:107:117:orf00001        aaaggatttgtaaaaaatctaaacgatggaagggtagaaattgtagctttctttaatact&#xA;118a:1:106:158122218:orf00020 aaaggatttgtaaaaaatctaaacgatggaagggtagaaattgtagctttctttaatact&#xA;B31:1:100:4091:ORFB0018       aaaggatttgtaaaaaatctcaacgatggaagggtagaaattgtagctttctttaatact&#xA;72a:1:105:32:orf00022         aaaggatttgtaaaaaatctaaacgatggaagggtagaaattgtagctttctttaatact&#xA;64b:1:110:473:orf00001        aaaggatttgtaaaaaatctcaacgatggaagggtagaaattgtagctttctttaatact&#xA;29805:1:108:171:orf00001      aaaggatttgtaaaaaatctcaacgatggaagggtagaaattgtagctttctttaatact&#xA;BOL26:1:111:60:orf00001       aaaggatttgtaaaaaatctcaacgatggaagggtagaaattgtagctttctttaatact&#xA;CA-11.2A:1:109:33:orf00001    aaaggatttgtaaaaaatctaaacgatggaagggtagaaattgtagctttctttaatact&#xA;WI91-23:1:112:493:orf00001    aaaggatttgtaaaaaatctcaacgatggaagggtagaaattgtagctttctttaatact&#xA;297:1:103:411:ORFB00012       aaaggatttgtaaaaaatctaaacgatggaagggtagaaattgtagctttctttaatact&#xA;N40:1:101:1716:ORFK00021      aaaggatttgtaaaaaatctcaacgatggaagggtagaaattgtagctttctttaatact&#xA;ZS7:1:113:22:orf00001         aaaggatttgtaaaaaatctcaacgatggaagggtagaaattgtagctttctttaatact&#xA;                              ******************** ***************************************&#xA;&#xA;&#xA;JD1:1:102:1601:ORFJ00027      aaagaacaaatgaaaaaatttgaaaaattattaaatgggaataagtattcaaacattaaa&#xA;94a:1:107:117:orf00001        aaagaacaaatgaaaaaatttgaaaaattattaaatgggaataagtattcaaacattgaa&#xA;118a:1:106:158122218:orf00020 aaagaacaaatgaaaaaatttgaaaaattattaaatgggaataagtattcaaacattgaa&#xA;B31:1:100:4091:ORFB0018       aaagaacaaatgaaaaaatttgaaaaattattaaatgggaataagtattcaaacattaaa&#xA;72a:1:105:32:orf00022         aaagaacaaatgaaaaaatttgaaaaattattaaatgggaataagtattcaaacattgaa&#xA;64b:1:110:473:orf00001        aaagaacaaatgaaaaaatttgaaaaattattaaatgggaataagtattcaaacattaaa&#xA;29805:1:108:171:orf00001      aaagaacaaatgaaaaaatttgaaaaattattaaatgggaataagtattcaaacattgaa&#xA;BOL26:1:111:60:orf00001       aaagaacaaatgaaaaaatttgaaaaattattaaatgggaataagtattcaaacattaaa&#xA;CA-11.2A:1:109:33:orf00001    aaagaacaaatgaaaaaatttgaaaaattattaaatgggaataagtattcaaacattgaa&#xA;WI91-23:1:112:493:orf00001    aaagaacaaatgaaaaaatttgaaaaattattaaatgggaataagtattcaaacattgaa&#xA;297:1:103:411:ORFB00012       aaagaacaaatgaaaaaatttgaaaaattattaaatgggaataagtattcaaacattgaa&#xA;N40:1:101:1716:ORFK00021      aaagaacaaatgaaaaaatttgaaaaattattaaatgggaataagtattcaaacattgaa&#xA;ZS7:1:113:22:orf00001         aaagaacaaatgaaaaaatttgaaaaattattaaatgggaataagtattcaaacattaaa&#xA;                              ********************************************************* **&#xA;&#xA;&#xA;JD1:1:102:1601:ORFJ00027      aacattgaaaaaatagttttagatgaaaattatccttttcaatttaatgattttaaaatt&#xA;94a:1:107:117:orf00001        aacattgaaaaaatagttttagatgaaaattatccttttcaatttaatgattttaaaatt&#xA;118a:1:106:158122218:orf00020 aacattgaaaaaatagttttagatgaaaattatccttttcaatttaatgattttaaaatt&#xA;B31:1:100:4091:ORFB0018       aacattgaaaaaatagttttagatgaaaattatccttttcaatttaatgattttaaaatt&#xA;72a:1:105:32:orf00022         aacattgaaaaaatagttttagatgaaaattatccttttcaatttaatgattttaaaatt&#xA;64b:1:110:473:orf00001        aacattgaaaaaatagttttagatgaaaattatccttttcaatttaatgattttaaaatt&#xA;29805:1:108:171:orf00001      aacattgaaaaaatagctttagatgaaaattatccttttcaatttaatgattttaaaatt&#xA;BOL26:1:111:60:orf00001       aacattgaaaaaatagttttagatgaaaattatccttttcaatttaatgattttaaaatt&#xA;CA-11.2A:1:109:33:orf00001    aacattgaaaaaatagttttagatgaaaattatccttttcaatttaatgattttaaaatt&#xA;WI91-23:1:112:493:orf00001    aacattgaaaaaatagctttagatgaaaattatccttttcaatttaatgattttaaaatt&#xA;297:1:103:411:ORFB00012       aacattgaaaaaatagttttagatgaaaattatccttttcaatttaatgattttaaaatt&#xA;N40:1:101:1716:ORFK00021      aacattgaaaaaatagctttagatgaaaattatccttttcaatttaatgattttaaaatt&#xA;ZS7:1:113:22:orf00001         aacattgaaaaaatagttttagatgaaaattatccttttcaatttaatgattttaaaatt&#xA;                              **************** *******************************************&#xA;&#xA;&#xA;JD1:1:102:1601:ORFJ00027      tattat&#xA;94a:1:107:117:orf00001        tattat&#xA;118a:1:106:158122218:orf00020 tattat&#xA;B31:1:100:4091:ORFB0018       tattat&#xA;72a:1:105:32:orf00022         tattat&#xA;64b:1:110:473:orf00001        tattat&#xA;29805:1:108:171:orf00001      tattat&#xA;BOL26:1:111:60:orf00001       tattat&#xA;CA-11.2A:1:109:33:orf00001    tattat&#xA;WI91-23:1:112:493:orf00001    tattat&#xA;297:1:103:411:ORFB00012       tattat&#xA;N40:1:101:1716:ORFK00021      tattat&#xA;ZS7:1:113:22:orf00001         tattat&#xA;                              ******&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here's the code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/perl&#xA;use Bio::AlignIO;&#xA;use Bio::Graphics::Panel;&#xA;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$line = shift @ARGV;&#xA;my $&lt;/span&gt;in = Bio::AlignIO-&amp;gt;new(-file=&amp;gt;&lt;span class=&quot;math-container&quot;&gt;$line,-format=&amp;gt;&quot;clustalw&quot;);&#xA;while($&lt;/span&gt;aln = &lt;span class=&quot;math-container&quot;&gt;$in-&amp;gt;next_aln()){&#xA;    foreach $seqobj($aln-&amp;gt;each_seq()){&#xA;        my $seq = $seqobj-&amp;gt;seq;&#xA;        my $id = $seqobj-&amp;gt;id;&#xA;        my $length = $seqobj-&amp;gt;length;&#xA;        my $seqobj = Bio::SeqFeature::Generic-&amp;gt;new(-start =&amp;gt;1, -end=&amp;gt;$length,-display_name=&amp;gt;$id);&#xA;        push (@seq, $seqobj);&#xA;    }&#xA;    foreach $seq(@seq){&#xA;        my @features = $seq-&amp;gt;get_SeqFeatures;&#xA;        my %sorted_features;&#xA;        for my $f (@features) {&#xA;            my $tag = &quot;\-&quot;;&#xA;            push @{$sorted_features{$tag}},$&lt;/span&gt;f;&#xA;        }&#xA;&#xA;        my &lt;span class=&quot;math-container&quot;&gt;$panel = Bio::Graphics::Panel-&amp;gt;new(&#xA;            -length    =&amp;gt; $&lt;/span&gt;seq-&amp;gt;length,&#xA;            -key_style =&amp;gt; 'between',&#xA;            -width     =&amp;gt; 800,&#xA;            -pad_left  =&amp;gt; 10,&#xA;            -pad_right =&amp;gt; 10,&#xA;        );&#xA;        &lt;span class=&quot;math-container&quot;&gt;$panel-&amp;gt;add_track(generic =&amp;gt; Bio::SeqFeature::Generic-&amp;gt;new(-start=&amp;gt;1,&#xA;                -end=&amp;gt;$&lt;/span&gt;seq-&amp;gt;length),&#xA;            -glyph  =&amp;gt; 'generic',&#xA;            -bgcolor =&amp;gt; 'blue',&#xA;            -label  =&amp;gt; 1,&#xA;        );&#xA;&#xA;        my @colors = qw(cyan orange blue purple green chartreuse magenta yellow aqua);&#xA;        my &lt;span class=&quot;math-container&quot;&gt;$idx    = 0;&#xA;        for my $&lt;/span&gt;tag (sort keys my %sorted_features) {&#xA;            my &lt;span class=&quot;math-container&quot;&gt;$features = $&lt;/span&gt;sorted_features{&lt;span class=&quot;math-container&quot;&gt;$tag};&#xA;            $&lt;/span&gt;panel-&amp;gt;add_track(&lt;span class=&quot;math-container&quot;&gt;$features,&#xA;                -glyph    =&amp;gt;  'generic',&#xA;                -bgcolor  =&amp;gt;  $&lt;/span&gt;colors[&lt;span class=&quot;math-container&quot;&gt;$idx++ % @colors],&#xA;                -fgcolor  =&amp;gt; 'black',&#xA;                -font2color =&amp;gt; 'red',&#xA;                -key      =&amp;gt; &quot;$&lt;/span&gt;{tag}s&amp;quot;,&#xA;                -bump     =&amp;gt; +1,&#xA;                -height   =&amp;gt; 8,&#xA;                -label    =&amp;gt; 1,&#xA;                -description =&amp;gt; 1,&#xA;            );&#xA;        }&#xA;        print $panel-&amp;gt;png;&#xA;        &#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Unfortunately, &lt;code&gt;$seq-&amp;gt;get_SeqFeatures&lt;/code&gt; never returns anything. I'm not sure why it doesn't since there should be features on it coming from the Bio::SimpleAlign object. Maybe I should set some new parameters?&lt;/p&gt;&#xA;" OwnerUserId="1756" LastEditorUserId="73" LastEditDate="2021-07-04T04:38:19.357" LastActivityDate="2021-07-04T04:38:19.357" Title="Trying to show the gaps of each seq on bio::Graphics after converting clustalw" Tags="|sequence-alignment|perl|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3016" PostTypeId="1" CreationDate="2017-12-07T18:56:25.760" Score="2" ViewCount="98" Body="&lt;p&gt;Currently I'm building a big matrix (using microarray, mass spectrometry, RNAseq data) that consist by pathways (rows) and treatment/drugs (columns).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The values of that matrix are scores that describe the effect/relation of that specific treatment/drug on each pathway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, some of the drugs is known to target specific disease (target disease) and thus is expected to have a high score with that disease's pathways. &lt;strong&gt;(A)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The rest of the drugs is known to target other diseases.&lt;strong&gt;(B)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My thought now is that, if I label the known drugs as &quot;1&quot; (or whatever), could I use a prediction model to find out new relationships (links) between drugs from the &lt;strong&gt;B&lt;/strong&gt; class and the significant disease pathway from &lt;strong&gt;A&lt;/strong&gt; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The whole thing, we can represent it as a bipartite network shown below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pyNwa.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pyNwa.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, here the denoted as green, drugs are the known drugs belong to the class &lt;strong&gt;A&lt;/strong&gt; that mentioned before and they are connected highly with some pathways of the target disease.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the other hand blue and red ones are drugs taken from other cases (class &lt;strong&gt;B&lt;/strong&gt;) that are used to target other diseases, but as my plot shows, blue seems to has an &quot;affinity&quot; to some of the pathways of the target disease and thus should be an alternative candidate for that disease. The red from the other side, doesn't have any &quot;affinity&quot; to any significant pathway and thus we can not consider it as possible candidate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find a solution to that kind of problem, I looked for algorithms and methods developed for social networks (e.g recommendation etc.) and mostly I read about random walks with return.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But before to proceed, I would like to know if someone else tried such an approach here and having more experience. &lt;/p&gt;&#xA;" OwnerUserId="704" LastActivityDate="2018-02-06T09:56:59.353" Title="Link Prediction in Bipartite Networks between biological pathways and drugs" Tags="|networks|pathway|machine-learning|modelling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3017" PostTypeId="2" ParentId="3004" CreationDate="2017-12-08T04:14:38.370" Score="4" Body="&lt;p&gt;First the Nextera adapters and the custom barcode adapters overlap each other&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Nextera 1                                     TCGTCGGCAGCGTCAGATGTGTATAAGAGACAG &#xA;Barcode 1   GATACGGCGACCACCGAGATCTACACTAGATCGCTCGTCGGCAGCGTCAGATGTGTAT &#xA;&#xA;Nextera 2                                    GTCTCGTGGGCTCGGAGATGTGTATAAGAGACAG&#xA;Barcode 2    CAAGCAGAAGACGGCATACGAGATACCCAGCAGTCTCGTGGGCTCGGAGATGTG &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Looking at some read 1s and read 2s you were correct that the adapters appear at the 3' end of the read reverse complimented (The &lt;code&gt;|&amp;gt;&lt;/code&gt; points to the adapter in the sequence).  You can see that &lt;code&gt;CTGTCTCTTATACACA&lt;/code&gt; would be the ideal sequence to use for the 3' end of the reads.  Since the Nextera adapter and the barcode overlap you only need to use the one that is adjacent to the insert.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SRR1780164_2.fastq: GCCCGAGGGGACGCTGCCCCACGTGCTCCAGG |&amp;gt; CTGTCTCTTATACACATCTGACGCTGCCGACGATGCACGAAGTG&#xA;SRR1780164_1.fastq: GGGCTGGACAGCGGCGGGGACGCGGACCTGCAG |&amp;gt; CTGTCTCTTATACACATCTCCGAGCCCACGAGACACAAAGTGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Of the 64k reads there are ~11k with the Nextera adapter at the 3' end based on a rough sequence &lt;code&gt;grep&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the 5' end of the read there is some Nextera adapter present but it is not reverse complimented and there are only a handful of reads with the adapter present ~250 total between read 1 and read 2.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SRR1780164_2.fastq: TGGGCTCGGAGATGTGTATAAGAGACAG &amp;lt;| CTCCCCCGCCTCCAGCATCCGGGCGAGGTAGTGCATCGACGCGTCCAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I used this command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cutadapt --minimum-length=20 -A CTGTCTCTTATACACA -a CTGTCTCTTATACACA -G GATGTGTATAAGAGACAG &#xA;-g GATGTGTATAAGAGACAG -o SRR1780164_1_trimmed.fastq.gz -p SRR1780164_2_trimmed.fastq.gz &#xA;SRR1780164_1.fastq SRR1780164_2.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which gives a trimming result of &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   === Summary ===&#xA;&#xA;Total read pairs processed:             64,832&#xA;  Read 1 with adapter:                  16,721 (25.8%)&#xA;  Read 2 with adapter:                  16,996 (26.2%)&#xA;Pairs that were too short:                  46 (0.1%)&#xA;Pairs written (passing filters):        64,786 (99.9%)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think this best represents the library construct and the trimming of the adapter sequences&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2017-12-08T04:14:38.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3018" PostTypeId="2" ParentId="3011" CreationDate="2017-12-08T05:06:21.243" Score="2" Body="&lt;p&gt;The snarky answer is that it's better to find a matching dataset at the experimental design phase (i.e. &lt;em&gt;before&lt;/em&gt; these studies are carried out), so that the bioinformatics component on the other side of things ends up being a lot easier and more robust. Although it can be quite time consuming to do a database search at the start, it's even harder to find matching projects if the legwork hasn't been done during the design phase. Researchers should be thinking about how hypotheses will be validated &lt;em&gt;before&lt;/em&gt; the experiments are carried out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every additional characteristic that is chosen for comparison adds complications with regards to matching data, and reduces the applicability of results when there is a mismatch. You would have to be somewhat lucky to be able to find a IBD-Crohn's dataset with paired RNA-seq data and 16s-seq (presumably short-read; is 454 good enough?) from a specific non-stool tissue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apart from the places you've already looked, there's &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gap/?term=16s+Crohns&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbGaP&lt;/a&gt; and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra/?term=16s+Crohn%27s+RNA&quot; rel=&quot;nofollow noreferrer&quot;&gt;SRA&lt;/a&gt;. Given the number of characteristics that have been mentioned, I expect you'll have to make some compromises when comparing other datasets.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-12-08T11:24:25.137" LastActivityDate="2017-12-08T11:24:25.137" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3019" PostTypeId="2" ParentId="3012" CreationDate="2017-12-08T09:13:21.787" Score="1" Body="&lt;p&gt;The issue here was I was using an older version of eDirect. There is no command to check version or updates. Your best bet is if you experience problems uninstall and reinstall. &lt;/p&gt;&#xA;" OwnerUserId="982" LastActivityDate="2017-12-08T09:13:21.787" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3020" PostTypeId="2" ParentId="3016" CreationDate="2017-12-08T09:23:41.637" Score="2" Body="&lt;p&gt;I am not sure if I understood correctly how you classify  the drugs. But what you attempt to do is similar to what they do &lt;a href=&quot;http://cancerres.aacrjournals.org/content/77/2/459.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; but in that article they use the drugs targeted to a molecule/pathway to find combination of drugs that are better for the disease (not an alternative drug), see the first image:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WWNuh.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WWNuh.jpg&quot; alt=&quot;Pathway targeted by two drugs&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In that article they mention the pathway cross-talk inhibition (PCI), you might want to calculate it for your pathways to assess the relative reduction of the pathway efficiency.&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-12-08T09:23:41.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3021" PostTypeId="1" CreationDate="2017-12-08T09:45:18.613" Score="1" ViewCount="1638" Body="&lt;p&gt;I have download a batch of refseq fasta files and want to split them based on strain. this is complicated by accession numbers, project IDs etc. I downloaded the genomes &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/3012/ncbi-edirect-suite-to-download-all-genome-sequences-associated-with-a-query&quot;&gt;via eDirect from NCBI&lt;/a&gt;. I know I can split the output but I want them together for some preliminary parsing (remove ambiguous and plasmid sequences etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See example below. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;NZ_AESB01000237.1 Clostridium botulinum BKT028387 contig00237 ATGC&#xA;&amp;gt;NZ_AESB01000236.1 Clostridium botulinum BKT028387 contig00236 ATGC&#xA;&amp;gt;NZ_ALNK01000035.1 Peptoanaerobacter stomatis strain OBRC8 ctg120007290634 ATGC&#xA;&amp;gt;NZ_ALNK01000031.1 Peptoanaerobacter stomatis strain OBRC8 ctg120007290635 ATGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want two files:&#xA;Clostridium botulinum BKT028387.fas and Peptoanaerobacter stomatis OBRC8.fas&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My actual dataset is much larger (~70,000 genomes) and not too sure about the headers in each case.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Im thinking the following could work but Im not sure how to execute it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a &lt;a href=&quot;https://unix.stackexchange.com/questions/361616/extract-lines-from-large-tab-delimited-file-using-a-list&quot;&gt;grep command&lt;/a&gt; to extract the lines but split the output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I download the summary list from refseq and use the names as the query example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -Fwf refseq.list sequences.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Summary file&lt;/p&gt;&#xA;  &#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;&lt;p&gt;Peptoanaerobacter stomatis strain OBRC8, whole genome shotgun sequencing project 7,810,226 rc other DNA  NZ_AJXR00000000.1 GI:938474759&lt;/p&gt;&#xA;  &#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;Clostridium botulinum BKT028387, whole genome shotgun sequencing project 4,134,593 rc other DNA  NZ_NOJY00000000.1 GI:1231817869&lt;/li&gt;&#xA;  &lt;/ol&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;  &#xA;  &lt;p&gt;But I could edit it down to &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Peptoanaerobacter stomatis strain OBRC8&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Clostridium botulinum BKT028387&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Am I one the right track. How do I finish this?&lt;/p&gt;&#xA;" OwnerUserId="982" LastEditorUserId="982" LastEditDate="2017-12-08T10:29:35.967" LastActivityDate="2018-03-29T15:50:22.970" Title="How to split multifasta based on partial fasta header" Tags="|fasta|data-management|" AnswerCount="3" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3022" PostTypeId="1" CreationDate="2017-12-08T09:54:57.273" Score="11" ViewCount="370" Body="&lt;p&gt;There are many features affecting mutation probabilities, e.g. CpG mutations are 10-fold more likely than other types of mutations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a model (preferably with software) which can take two aligned genomic regions, &lt;strong&gt;estimate&lt;/strong&gt; parameters of the neutral mutation procces and then &lt;strong&gt;simulate&lt;/strong&gt; mutations for another region? Obviously, there are models like HKY and GTR, but they are somewhat simplistic and do not take the context into account.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One model I found is &lt;a href=&quot;https://doi.org/10.1093/bioinformatics/bth917&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;, but it seems that there is no software implementing the model. Is there something more recent?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also looking for software which can account for &lt;strong&gt;indels&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="191" LastEditorUserId="3848" LastEditDate="2020-05-01T11:07:59.800" LastActivityDate="2020-05-01T11:07:59.800" Title="State of the art mutation simulation software" Tags="|phylogenetics|phylogeny|mutations|nucleotide-models|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="3023" PostTypeId="2" ParentId="3021" CreationDate="2017-12-08T09:58:02.083" Score="0" Body="&lt;p&gt;I would convert the fasta file to a format with one line per record (I have &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2651/298&quot;&gt;already posted&lt;/a&gt; the FastaToTbl and TblToFasta scripts I use for this before), and then look for a specific pattern:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FastaToTbl file.fa | grep 'BKT028387' | TblToFasta &amp;gt; clostridium.fa&#xA;FastaToTbl file.fa | grep 'OBRC8' | TblToFasta &amp;gt; peptoanaerobacter.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have downloaded a few sequences using the same command you described in your previous question, and it looks like the headers are not standardized. For example, while you have many that are in the format of &lt;code&gt;$species strain $strain&lt;/code&gt;, like these:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Clostridioides difficile strain 7496-D/ST9&#xA;Clostridioides difficile strain 5616-DH/ST42,&#xA;Peptoanaerobacter stomatis strain OBRC8&#xA;Romboutsia maritimum strain CCRI-22766&#xA;Tepidibacter mesophilus strain JCM&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Others don't have the word &lt;code&gt;strain&lt;/code&gt; and just give the strain:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Peptostreptococcus anaerobius 653-L contig00088,&#xA;Peptoclostridium acidaminophilum DSM 3953&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, as a first step, you could deal with the ones that do have the word &lt;code&gt;strain&lt;/code&gt; as the 4th word of the header line (the 1st is the seqID which I have not included in the examples above) and the actual strain as the 5th and get those out of the way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FastaToTbl foo.fa | &#xA;    awk '($4==&quot;strain&quot;){gsub(&quot;/&quot;,&quot;_&quot;); print $0 &amp;gt;&amp;gt; $2&quot;_&quot;$3&quot;_&quot;$5&quot;.tbl&quot;}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That will look for lines whose 4th field is &lt;code&gt;strain&lt;/code&gt;, then it will replace any &lt;code&gt;/&lt;/code&gt; in the line with &lt;code&gt;_&lt;/code&gt; (there's a strain called &lt;code&gt;CF/ST37&lt;/code&gt; and that cannot be in a filename) and print into a file whose name is the 2nd, 3rd and 5th fields of the line concatenated. On my test file, this produced the following files (as you can see, some are wrong since there is a comma after the strain, but that's because they're not standardized):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ls *tbl&#xA;Clostridioides_difficile_5479-NonSp_ST229,.tbl  Clostridioides_difficile_6589-DH_ST42.tbl         Clostridioides_difficile_7457-NonSp_novelST,.tbl&#xA;Clostridioides_difficile_5479-NonSp_ST229.tbl   Clostridioides_difficile_6592-G_ST8,.tbl          Clostridioides_difficile_7457-NonSp_novelST.tbl&#xA;Clostridioides_difficile_5480-E_ST6,.tbl        Clostridioides_difficile_6592-G_ST8.tbl           Clostridioides_difficile_7458-N_ST10,.tbl&#xA;Clostridioides_difficile_5480-E_ST6.tbl         Clostridioides_difficile_6593-Y_ST110,.tbl        Clostridioides_difficile_7458-N_ST10.tbl&#xA;Clostridioides_difficile_5482-L_ST55,.tbl       Clostridioides_difficile_6593-Y_ST110.tbl         Clostridioides_difficile_7459-G_ST8,.tbl&#xA;Clostridioides_difficile_5482-L_ST55.tbl        Clostridioides_difficile_6595-AL_ST58,.tbl        Clostridioides_difficile_7459-G_ST8.tbl&#xA;Clostridioides_difficile_5485-Y_ST2,.tbl        Clostridioides_difficile_6595-AL_ST58.tbl         Clostridioides_difficile_7462-DH_ST42,.tbl&#xA;Clostridioides_difficile_5485-Y_ST2.tbl         Clostridioides_difficile_6596-D_ST9,.tbl          Clostridioides_difficile_7462-DH_ST42.tbl&#xA;Clostridioides_difficile_5488-DH_ST42,.tbl      Clostridioides_difficile_6596-D_ST9.tbl           Clostridioides_difficile_7463-E_ST6,.tbl&#xA;Clostridioides_difficile_5488-DH_ST42.tbl       Clostridioides_difficile_6599-NonSp_ST103,.tbl    Clostridioides_difficile_7463-E_ST6.tbl&#xA;Clostridioides_difficile_5489-M_ST15,.tbl       Clostridioides_difficile_6599-NonSp_ST103.tbl     Clostridioides_difficile_7464-Y_ST110,.tbl&#xA;Clostridioides_difficile_5489-M_ST15.tbl        Clostridioides_difficile_6600-G_ST8,.tbl          Clostridioides_difficile_7464-Y_ST110.tbl&#xA;Clostridioides_difficile_5490-CF_ST37,.tbl      Clostridioides_difficile_6600-G_ST8.tbl           Clostridioides_difficile_7466-DH_ST42,.tbl&#xA;Clostridioides_difficile_5490-CF_ST37.tbl       Clostridioides_difficile_6606-BK_ST11,.tbl        Clostridioides_difficile_7466-DH_ST42.tbl&#xA;Clostridioides_difficile_5493-DH_ST42,.tbl      Clostridioides_difficile_6606-BK_ST11.tbl         Clostridioides_difficile_7467-DH_ST42,.tbl&#xA;Clostridioides_difficile_5493-DH_ST42.tbl       Clostridioides_difficile_6607-DH_ST42,.tbl        Clostridioides_difficile_7467-DH_ST42.tbl&#xA;Clostridioides_difficile_5495-AF_ST41,.tbl      Clostridioides_difficile_6607-DH_ST42.tbl         Clostridioides_difficile_7468-NonSp_ST97,.tbl&#xA;Clostridioides_difficile_5495-AF_ST41.tbl       Clostridioides_difficile_6610-N_ST10,.tbl         Clostridioides_difficile_7468-NonSp_ST97.tbl&#xA;Clostridioides_difficile_5497-Y_ST2,.tbl        Clostridioides_difficile_6610-N_ST10.tbl          Clostridioides_difficile_7471-BI_ST1,.tbl&#xA;Clostridioides_difficile_5497-Y_ST2.tbl         Clostridioides_difficile_6612-DH_ST28,.tbl        Clostridioides_difficile_7471-BI_ST1.tbl&#xA;Clostridioides_difficile_5500-DH_ST42,.tbl      Clostridioides_difficile_6612-DH_ST28.tbl         Clostridioides_difficile_7472-Y_ST2,.tbl&#xA;Clostridioides_difficile_5500-DH_ST42.tbl       Clostridioides_difficile_6613-NonSp_ST3,.tbl      Clostridioides_difficile_7472-Y_ST2.tbl&#xA;Clostridioides_difficile_5504-L_ST55,.tbl       Clostridioides_difficile_6613-NonSp_ST3.tbl       Clostridioides_difficile_7473-DH_ST42,.tbl&#xA;Clostridioides_difficile_5504-L_ST55.tbl        Clostridioides_difficile_6614-AL_ST34,.tbl        Clostridioides_difficile_7473-DH_ST42.tbl&#xA;Clostridioides_difficile_5505-Y_ST2,.tbl        Clostridioides_difficile_6614-AL_ST34.tbl         Clostridioides_difficile_7476-Y_ST2,.tbl&#xA;Clostridioides_difficile_5505-Y_ST2.tbl         Clostridioides_difficile_6615-NonSp_ST11,.tbl     Clostridioides_difficile_7476-Y_ST2.tbl&#xA;Clostridioides_difficile_5537-D_ST9,.tbl        Clostridioides_difficile_6615-NonSp_ST11.tbl      Clostridioides_difficile_7478-NonSp_novelST,.tbl&#xA;Clostridioides_difficile_5537-D_ST9.tbl         Clostridioides_difficile_6616-NonSp_novelST,.tbl  Clostridioides_difficile_7478-NonSp_novelST.tbl&#xA;Clostridioides_difficile_5538-BK_ST11,.tbl      Clostridioides_difficile_6616-NonSp_novelST.tbl   Clostridioides_difficile_7479-NonSp_ST3,.tbl&#xA;Clostridioides_difficile_5538-BK_ST11.tbl       Clostridioides_difficile_6617-A_ST43,.tbl         Clostridioides_difficile_7479-NonSp_ST3.tbl&#xA;Clostridioides_difficile_5539-Y_ST2,.tbl        Clostridioides_difficile_6617-A_ST43.tbl          Clostridioides_difficile_7480-M_ST15,.tbl&#xA;Clostridioides_difficile_5539-Y_ST2.tbl         Clostridioides_difficile_6618-NonSp_ST53,.tbl     Clostridioides_difficile_7480-M_ST15.tbl&#xA;Clostridioides_difficile_5541-CF_ST37,.tbl      Clostridioides_difficile_6618-NonSp_ST53.tbl      Clostridioides_difficile_7481-NonSp_ST59,.tbl&#xA;Clostridioides_difficile_5541-CF_ST37.tbl       Clostridioides_difficile_6619-CF_ST37,.tbl        Clostridioides_difficile_7481-NonSp_ST59.tbl&#xA;Clostridioides_difficile_5542-Y_ST2,.tbl        Clostridioides_difficile_6619-CF_ST37.tbl         Clostridioides_difficile_7482-DH_ST42,.tbl&#xA;Clostridioides_difficile_5542-Y_ST2.tbl         Clostridioides_difficile_6621-NonSp_ST10,.tbl     Clostridioides_difficile_7482-DH_ST42.tbl&#xA;Clostridioides_difficile_5547-DH_ST42,.tbl      Clostridioides_difficile_6621-NonSp_ST10.tbl      Clostridioides_difficile_7486-A_ST43,.tbl&#xA;Clostridioides_difficile_5547-DH_ST42.tbl       Clostridioides_difficile_6622-CF_ST37,.tbl        Clostridioides_difficile_7486-A_ST43.tbl&#xA;Clostridioides_difficile_5548-Y_ST2,.tbl        Clostridioides_difficile_6622-CF_ST37.tbl         Clostridioides_difficile_7488-NonSp_ST103,.tbl&#xA;Clostridioides_difficile_5548-Y_ST2.tbl         Clostridioides_difficile_6623-NonSp_ST54,.tbl     Clostridioides_difficile_7488-NonSp_ST103.tbl&#xA;Clostridioides_difficile_5549-NonSp_ST21,.tbl   Clostridioides_difficile_6623-NonSp_ST54.tbl      Clostridioides_difficile_7491-DH_ST42,.tbl&#xA;Clostridioides_difficile_5549-NonSp_ST21.tbl    Clostridioides_difficile_6625-Y_ST2,.tbl          Clostridioides_difficile_7491-DH_ST42.tbl&#xA;Clostridioides_difficile_5550-AH_ST67,.tbl      Clostridioides_difficile_6625-Y_ST2.tbl           Clostridioides_difficile_7492-CF_ST37,.tbl&#xA;Clostridioides_difficile_5550-AH_ST67.tbl       Clostridioides_difficile_6626-Y_ST110,.tbl        Clostridioides_difficile_7492-CF_ST37.tbl&#xA;Clostridioides_difficile_5551-CF_ST37,.tbl      Clostridioides_difficile_6626-Y_ST110.tbl         Clostridioides_difficile_7494-E_ST6,.tbl&#xA;Clostridioides_difficile_5551-CF_ST37.tbl       Clostridioides_difficile_6628-AL_ST58,.tbl        Clostridioides_difficile_7494-E_ST6.tbl&#xA;Clostridioides_difficile_5552-DH_ST42,.tbl      Clostridioides_difficile_6628-AL_ST58.tbl         Clostridioides_difficile_7496-D_ST9,.tbl&#xA;Clostridioides_difficile_5552-DH_ST42.tbl       Clostridioides_difficile_6629-NonSp_novelST,.tbl  Clostridioides_difficile_7496-D_ST9.tbl&#xA;Clostridioides_difficile_5553-L_ST55,.tbl       Clostridioides_difficile_6629-NonSp_novelST.tbl   Clostridioides_difficile_7497-NonSp_novelST,.tbl&#xA;Clostridioides_difficile_5553-L_ST55.tbl        Clostridioides_difficile_6631-DH_ST42,.tbl        Clostridioides_difficile_7497-NonSp_novelST.tbl&#xA;Clostridioides_difficile_5555-DH_ST42,.tbl      Clostridioides_difficile_6631-DH_ST42.tbl         Clostridioides_difficile_7498-M_ST15,.tbl&#xA;Clostridioides_difficile_5555-DH_ST42.tbl       Clostridioides_difficile_6632-NonSp_ST3,.tbl      Clostridioides_difficile_7498-M_ST15.tbl&#xA;Clostridioides_difficile_5557-DH_ST42,.tbl      Clostridioides_difficile_6632-NonSp_ST3.tbl       Clostridioides_difficile_7499-CF_ST37,.tbl&#xA;Clostridioides_difficile_5557-DH_ST42.tbl       Clostridioides_difficile_6634-DH_ST42,.tbl        Clostridioides_difficile_7499-CF_ST37.tbl&#xA;Clostridioides_difficile_5558-AH_ST67,.tbl      Clostridioides_difficile_6634-DH_ST42.tbl         Clostridioides_difficile_7501-BI_ST1,.tbl&#xA;Clostridioides_difficile_5558-AH_ST67.tbl       Clostridioides_difficile_6635-BI_ST1,.tbl         Clostridioides_difficile_7501-BI_ST1.tbl&#xA;Clostridioides_difficile_5559-DH_ST42,.tbl      Clostridioides_difficile_6635-BI_ST1.tbl          Clostridioides_difficile_7503-Y_ST2,.tbl&#xA;Clostridioides_difficile_5559-DH_ST42.tbl       Clostridioides_difficile_6637-AL_ST58,.tbl        Clostridioides_difficile_7503-Y_ST2.tbl&#xA;Clostridioides_difficile_5573-CF_ST37,.tbl      Clostridioides_difficile_6637-AL_ST58.tbl         Clostridioides_difficile_7505-Y_ST2,.tbl&#xA;Clostridioides_difficile_5573-CF_ST37.tbl       Clostridioides_difficile_7450-DH_ST42,.tbl        Clostridioides_difficile_7505-Y_ST2.tbl&#xA;Clostridioides_difficile_5616-DH_ST42,.tbl      Clostridioides_difficile_7450-DH_ST42.tbl         Clostridioides_difficile_NCKUH-21,.tbl&#xA;Clostridioides_difficile_5616-DH_ST42.tbl       Clostridioides_difficile_7451-NonSp_ST2,.tbl      Clostridioides_difficile_RC10,.tbl&#xA;Clostridioides_difficile_5754,.tbl              Clostridioides_difficile_7451-NonSp_ST2.tbl       Clostridioides_difficile_RC10.tbl&#xA;Clostridioides_difficile_5754.tbl               Clostridioides_difficile_7452-Y_ST110,.tbl        Clostridioides_difficile_RF17,.tbl&#xA;Clostridioides_difficile_6583-AH_ST67,.tbl      Clostridioides_difficile_7452-Y_ST110.tbl         Clostridioides_difficile_RF17.tbl&#xA;Clostridioides_difficile_6583-AH_ST67.tbl       Clostridioides_difficile_7453-Y_ST2,.tbl          Peptoanaerobacter_stomatis_OBRC8,.tbl&#xA;Clostridioides_difficile_6584-AQ_ST26,.tbl      Clostridioides_difficile_7453-Y_ST2.tbl           Peptoanaerobacter_stomatis_OBRC8.tbl&#xA;Clostridioides_difficile_6584-AQ_ST26.tbl       Clostridioides_difficile_7455-A_ST43,.tbl         Romboutsia_maritimum_CCRI-22766,.tbl&#xA;Clostridioides_difficile_6588-G_ST8,.tbl        Clostridioides_difficile_7455-A_ST43.tbl          Romboutsia_maritimum_CCRI-22766.tbl&#xA;Clostridioides_difficile_6588-G_ST8.tbl         Clostridioides_difficile_7456-DH_ST42,.tbl        Tepidibacter_mesophilus_JCM.tbl&#xA;Clostridioides_difficile_6589-DH_ST42,.tbl      Clostridioides_difficile_7456-DH_ST42.tbl&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, you can convert these back to fasta with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for f in *tbl; do TblToFasta $f &amp;gt; ${f/.tbl/.fa} &amp;amp;&amp;amp; rm $f; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will only get those sequences whose naming scheme is as described above. But that's a start. You can then see what else you have and extract the rest manually or by finding a similar approach that matches those.     &lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-12-08T10:24:54.633" LastActivityDate="2017-12-08T10:24:54.633" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3024" PostTypeId="2" ParentId="3010" CreationDate="2017-12-08T13:16:02.073" Score="3" Body="&lt;p&gt;Based on the sites you mention, it seems that p-distances are pairwise distances between aligned sequences obtained by counting the number of differences and dividing by the length of the alignment. This is the proportion of differences between two sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure exactly what you mean by &quot;add p-distance model in my code&quot;, but here is how you could compute those distances from an existing alignment (here I use as an example an alignment I have in fasta format):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import AlignIO&#xA;&#xA;def p_dist(seq1, seq2):&#xA;    ali_len = len(seq1)&#xA;    assert ali_len == len(seq2)&#xA;    return sum(0 if nuc1 == nuc2 else 1 for (nuc1, nuc2) in zip(seq1, seq2)) / ali_len&#xA;&#xA;ali = AlignIO.read(&quot;most_targeted_histones/transcript_alignments/H4_top4_loss_spliced_NT.fa&quot;, &quot;fasta&quot;)&#xA;nb_seq = len(ali)&#xA;distances = {}&#xA;for i in range(nb_seq - 1):&#xA;    rec1 = ali[i]&#xA;    j = i + 1&#xA;    while j &amp;lt; nb_seq:&#xA;        rec2 = ali[j]&#xA;        distances[(rec1.id, rec2.id)] = p_dist(rec1.seq, rec2.seq)&#xA;        j += 1&#xA;&#xA;print(*distances.items(), sep=&quot;\n&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(('ZK131.4_spliced', 'ZK131.8_spliced'), 0.0)&#xA;(('ZK131.4_spliced', 'T10C6.14_spliced'), 0.10897435897435898)&#xA;(('ZK131.4_spliced', 'ZK131.1_spliced'), 0.0)&#xA;(('ZK131.8_spliced', 'T10C6.14_spliced'), 0.10897435897435898)&#xA;(('ZK131.8_spliced', 'ZK131.1_spliced'), 0.0)&#xA;(('T10C6.14_spliced', 'ZK131.1_spliced'), 0.10897435897435898)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want to build a tree from those distances, I'm not sure you can do this with BioPython. It seems however that there are some possibilities with &lt;a href=&quot;https://pythonhosted.org/DendroPy/primer/phylogenetic_distances.html#generating-distance-trees-from-a-phylogeneticdistancematrix-object&quot; rel=&quot;nofollow noreferrer&quot;&gt;DendroPy&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would need to write the distances in a suitable format, and to choose the method between UPGMA and neighbour-joining.&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="48" LastEditDate="2017-12-08T13:27:59.797" LastActivityDate="2017-12-08T13:27:59.797" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3025" PostTypeId="1" CreationDate="2017-12-08T20:12:25.150" Score="4" ViewCount="314" Body="&lt;p&gt;I am currently writing an R package that includes a module to run featureCounts (gene quantification tool) from Rsubread. I wanted to be able to specify the correct strandedness option to featureCounts without the user needing to specify whether their protocol is unstranded, forwardly stranded, or reversely stranded. Typically when I don't know the protocol for a given BAM file, I use RSeQC's infer_experiment module, which uses the BAM plus an annotation (GTF) file to give an output like so:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unstranded:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is PairEnd Data&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fraction of reads failed to determine: 0.0172&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fraction of reads explained by &quot;1++,1--,2+-,2-+&quot;: 0.4903&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fraction of reads explained by &quot;1+-,1-+,2++,2--&quot;: 0.4925&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Reversely stranded (R2 same strand as parent gene):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is PairEnd Data&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fraction of reads failed to determine: 0.0072&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fraction of reads explained by &quot;1++,1--,2+-,2-+&quot;: 0.0487&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fraction of reads explained by &quot;1+-,1-+,2++,2--&quot;: 0.9441&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Does anyone know of an R package that does something similar? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I really needed to I could always require that users have RSeQC installed and just run infer_experiment as a system call. But it would really be better to run it from an existing R package if available. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know I could also write something up myself (use rsamtools to downsample the BAM, check ratio of SAM flags 99/147 vs. 83/163 in regions matching positive and negative strand genes). But would rather not reinvent the wheel here if there is something already available to do what I need.&lt;/p&gt;&#xA;" OwnerUserId="2016" LastActivityDate="2021-07-08T12:43:13.627" Title="R package equivalent to RSeQC infer_experiment to get strandedness of RNA-Seq" Tags="|r|rna-seq|bioconductor|strandedness|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3026" PostTypeId="1" AcceptedAnswerId="14785" CreationDate="2017-12-09T05:20:34.550" Score="3" ViewCount="45" Body="&lt;p&gt;Are there known dimensional measurements for the classification of folded proteins given a starting chunk/domain as defined by something like the clustering functionality of &lt;a href=&quot;http://msmbuilder.org/3.8.0/cluster.html#cluster&quot; rel=&quot;nofollow noreferrer&quot;&gt;MSM Builder&lt;/a&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Examples of what I would be looking for would be dimensions such as:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;pH&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Salinity&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Dissolved Sugars&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reducing Agent Concentration&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Ambient Electronegativity&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;With corresponding scales and precisions such as:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;logorithmic vs linear&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;step size&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;distribution (e.g. dissolved/flat vs fractal/inclusions)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The above parameters would be included in folding software after a known starting conformation provided by MSM Builder or similar chunking software to derive a GUID for proteins without (or with extremely minimal and tuneable) collisions?&lt;/p&gt;&#xA;&lt;p&gt;For instance, if hundreds of GUIDs on average end up equating to the same folded protein that's fine, but if one GUID could map to two or more thermodynamically stable folded proteins at greater than some incredibly small (e.g. 0.0000001%) then it wouldn't fit what I'm looking for.  I know this is inherently NP-hard, but the complete ID would be something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;    sequence,&#xA;    chunkId,&#xA;    pH,&#xA;    salinity,&#xA;    dissolvedSugar,&#xA;    reducingAgentConcentration&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So I would expect the chunkId and dimensional parameters to play a significant role in reducing collisions for different starting parameters.&lt;/p&gt;&#xA;&lt;p&gt;It's worth noting I'm not seeking some magic bullet where you type an ID and get a fully folded protein, just a reliable system of GUIDs with enough starting information embedded in the ID to start the folding process.&lt;/p&gt;&#xA;" OwnerUserId="1842" LastEditorUserId="3848" LastEditDate="2020-11-12T11:20:35.143" LastActivityDate="2020-11-12T17:15:43.207" Title="Folded Protein Chunk Dimensional Classification?" Tags="|proteins|protein-structure|identifiers|thermodynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3027" PostTypeId="1" CreationDate="2017-12-09T16:40:41.293" Score="4" ViewCount="144" Body="&lt;p&gt;I have troubles understanding the figure 10 of &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-1037-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UCm9S.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UCm9S.gif&quot; alt=&quot;Network &quot;&gt;&lt;/a&gt;&#xA;  Similarity network of participating methods for BPO. Similarities are computed as Pearson’s correlation coefficient between methods, with a 0.75 cutoff for illustration purposes. A unique color is assigned to all methods submitted under the same principal investigator. Not evaluated (organizers’) methods are shown in triangles, while benchmark methods (Naïve and BLAST) are shown in squares. The top-ten methods are highlighted with enlarged nodes and circled in red. The edge width indicates the strength of similarity. Nodes are labeled with the name of the methods followed by “-team(model)” if multiple teams/models were submitted&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This study calculated the pairwise Pearson correlation between methods on a common set of gene-concept pairs and then visualized these similarities as networks. How is the place of the methods determined in this visualization? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the relationship between Pannzer (1) and Pannzer (2) is greater than the relationship between Pannzer (1) and Pannzer (3), but the Panzer (1) and Panzer (3) are closer to each other than Panzer (1) and Panzer (2). Is this proximity positioning random?  or is there a certain rule?  &lt;/p&gt;&#xA;" OwnerUserId="2019" LastEditorUserId="48" LastEditDate="2017-12-10T09:47:59.170" LastActivityDate="2017-12-10T16:28:41.090" Title="Correlation network representation" Tags="|figure-explanation|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3028" PostTypeId="1" CreationDate="2017-12-09T17:07:09.697" Score="3" ViewCount="9493" Body="&lt;p&gt;What are the differences between the two methods? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What advantages does one have over the other, and what are their limitations relative to one another?&lt;/p&gt;&#xA;" OwnerUserId="2020" LastActivityDate="2017-12-11T15:24:31.860" Title="What are the differences between GWAS and QTL mapping?" Tags="|genomics|gwas|" AnswerCount="1" CommentCount="1" ClosedDate="2017-12-14T07:59:23.557" ContentLicense="CC BY-SA 3.0" />
  <row Id="3029" PostTypeId="2" ParentId="2088" CreationDate="2017-12-10T02:51:36.053" Score="1" Body="&lt;p&gt;Strandedness should be taken into account at the mapping stage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are presented with only a BAM file, ask for the raw reads. If that's not possible, you can recover the reads (as fastq sequences) from the BAM file and/or remap them, but that conversion requires a lot more effort (e.g. most mappers expect the fastq files from each end to be sorted by read name and in the same order).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding your specific questions...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it common to have mapping tools with an option to &quot;correct&quot; for strandedness?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, mapping tools can be provided with strand direction information for mapping. The way that this is done depends on the program, for example bowtie2 has options &lt;code&gt;--fr&lt;/code&gt;, &lt;code&gt;--rf&lt;/code&gt;, and &lt;code&gt;--ff&lt;/code&gt; to take strandedness into account. I haven't done any of my own comparisons to determine whether these options actually make a difference in mapping for Bowtie2. I know that Salmon (which does mapping and quantification at the same time) cares a lot more about strandedness, and will complain/warn if a stranded library type is discovered and not specified.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Assuming the BAM file has been built ignoring library strandedness information, is it good practice to correct for strandedness when building bigwig files?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, but it'd be better to remap with [the correct] strandedness information included. When I generate bigwig files to demonstrate strandedness (even assuming that strandedness is incorporated into the mapping), I still need to do a fair amount of legwork to convert that into bigwig files. The data needs to first be split into four groups (e.g. with &lt;code&gt;samtools view -F 0xXX -f 0xXX&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;First read, mapped in the reverse orientation&lt;/li&gt;&#xA;&lt;li&gt;Second read, mapped in the reverse orientation&lt;/li&gt;&#xA;&lt;li&gt;First read, mapped in the forward orientation&lt;/li&gt;&#xA;&lt;li&gt;Second read, mapped in the forward orientation&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I then combine two of those groups, depending on the library strandedness scheme. For a typical stranded Illumina run, groups 1 and 4 are combined (e.g. with &lt;code&gt;samtools merge&lt;/code&gt;), as are groups 2 and 3. These combined groups are turned into separate BigWig files, representing the &quot;genome-oriented&quot; and &quot;genome-reversed&quot; read pairs.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-12-10T02:51:36.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3030" PostTypeId="2" ParentId="3027" CreationDate="2017-12-10T09:43:04.070" Score="1" Body="&lt;p&gt;When plotting networks where only the weight of an edge is known like this one. The relative position between elements of the networks is irrelevant. The only informative part of the graph is (usually) the size of the edges, where is the thicker means more weight. This isn't a PCA, so the distance between different nodes is not relevant. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For those reasons there are several methods to build these networks, some of them give more emphasis to represent better the degree of the network or other characteristics of the networks. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I consider this representations bad for informing about the networks, because the cut off is arbitrary and the distance between nodes is not informative. One could first do a PCA or MDS and then add the edges between the nodes, this way the relative distance between the nodes would be meaningful as well as the edges. Or one could also use a heatmap to represent all the correlations between the nodes. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2017-12-10T09:43:04.070" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3031" PostTypeId="1" CreationDate="2017-12-10T10:33:07.383" Score="0" ViewCount="37" Body="&lt;p&gt;I have the 16S DNA Sequence of a Microorganism. Can I confirm its identity by only running a BLAST or do I have to follow other methods. If so please mention the methods.  &lt;/p&gt;&#xA;" OwnerUserId="2023" LastEditorUserId="964" LastEditDate="2017-12-11T08:29:25.407" LastActivityDate="2017-12-11T08:29:25.407" Title="Identification of Microorganism" Tags="|ngs|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3032" PostTypeId="2" ParentId="3031" CreationDate="2017-12-10T11:14:50.203" Score="3" Body="&lt;p&gt;As always with bioinformatics, any reasonable answer will be dependendant on what you want to do with this sequence, and you haven't provided enough information in that regard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For most exploratory analysis BLAST is good enough. You don't &lt;em&gt;have to&lt;/em&gt; follow any existing methods. The benefit of using NCBI's web BLAST server is that it provides you with a very quick way to find a match from the sequence information in GenBank.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BLAST does a local alignment. If you have the full 16s sequence, and it's a novel organism, you &lt;em&gt;might&lt;/em&gt; get different match results from a program that does global alignment, but then again you might get different results from a different local alignment tool.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-12-10T11:14:50.203" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3033" PostTypeId="2" ParentId="3027" CreationDate="2017-12-10T16:28:41.090" Score="0" Body="&lt;p&gt;The description you quote only states that edge thickness has relevance, not edge length. Usually with this sort of image, the program generating the graph (most probably Cytoscape) will simply arrange the nodes to maximize legibility. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While the edge length can, sometimes, be relevant it usually isn't and instead it is the thickness of the edges and the size of the nodes that carry the information. The positioning of the nodes is most likely arranged to make the image as clear as possible. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-12-10T16:28:41.090" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3034" PostTypeId="1" AcceptedAnswerId="3051" CreationDate="2017-12-10T23:39:38.020" Score="3" ViewCount="1074" Body="&lt;p&gt;I am trying to go through the following &lt;a href=&quot;https://f1000research.com/articles/5-2122/v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;scRNA-seq tutorial&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the line &lt;code&gt;sce &amp;lt;- newSCESet(countData=all.counts)&lt;/code&gt; is not working anymore with the most up-to-date version of &lt;code&gt;scater&lt;/code&gt;. Now we should use &lt;code&gt;SingleCellExperiment&lt;/code&gt; function instead, so when trying to do the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sce &amp;lt;- SingleCellExperiment(assays = list(counts = all.counts))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am getting the error: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in seq_len(ncol(assay)) : &#xA;  argument must be coercible to non-negative integer&#xA;In addition: Warning message:&#xA;In seq_len(ncol(assay)) : first element used of 'length.out' argument&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2018-01-26T09:02:01.647" LastActivityDate="2019-03-04T16:33:02.677" Title="scater: SingleCellExperiment, Error in seq_len(ncol(assay))" Tags="|r|rna-seq|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3035" PostTypeId="2" ParentId="2808" CreationDate="2017-12-11T05:27:46.740" Score="5" Body="&lt;p&gt;Thanks for your interest in RapMap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At that time we were using flux simulator for simulating read sequence data. We used the genome and gtf file together as an input to flux. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I dug into the scripts and got hold of the gtf link &lt;code&gt;ftp://ftp.ensembl.org/pub/release-80/gtf/homo_sapiens/Homo_sapiens.GRCh38.80.gtf.gz&lt;/code&gt;, &#xA;and the genome file link &lt;code&gt;ftp://ftp.ensembl.org/pub/release-80/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.toplevel.fa&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We used &lt;code&gt;rsem-prepare-reference&lt;/code&gt; to create the reference transcriptome file later in order to run RapMap and other transcriptome based quantification tools. I hope this helps. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I am a coauthor of the RapMap paper. &lt;/p&gt;&#xA;" OwnerUserId="2025" LastEditorUserId="48" LastEditDate="2017-12-11T07:57:24.453" LastActivityDate="2017-12-11T07:57:24.453" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3036" PostTypeId="1" AcceptedAnswerId="3037" CreationDate="2017-12-11T11:12:32.387" Score="4" ViewCount="111" Body="&lt;p&gt;I am trying to recreate the experiment from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5348765/&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Convergence of retrotransposons in&#xA;oomycetes and plants&lt;/em&gt;&lt;/a&gt; by Kirill Ustyantsev, Alexandr Blinov and Georgy Smyshlyaev. In the &quot;Methods&quot; section a &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;HMM profile was constructed from the amino acid alignment of aRNH&#xA;  sequences from the Ustyantsev et al&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I was wondering where I could get this profile. I checked the cited papers but they were also referencing other papers. Could someone maybe clarify where I could get the profile?&lt;/p&gt;&#xA;" OwnerUserId="2026" LastEditorUserId="298" LastEditDate="2017-12-11T11:19:34.117" LastActivityDate="2017-12-11T11:36:49.780" Title="HMM Profile from Convergence of retrotransposons in oomycetes and plants" Tags="|hidden-markov-models|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3037" PostTypeId="2" ParentId="3036" CreationDate="2017-12-11T11:36:49.780" Score="2" Body="&lt;p&gt;The alignment they used to build the HMM is included as &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5348765/bin/13100_2017_87_MOESM8_ESM.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;supplementary file 8&lt;/a&gt; in the paper you cite:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Multiple amino acid sequence alignment of RNH genes and domains from diverse taxa constructed and used for the phylogenetic reconstruction in the present study. (TXT 45 kb)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, you can download that alignment and then use a tool like hmmbuild (part of &lt;a href=&quot;http://hmmer.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HMMER&lt;/a&gt;) to build the HMM profile:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;hmmbuild model.hmm 13100_2017_87_MOESM8_ESM.txt &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The command above will create the file &lt;code&gt;model.hmm&lt;/code&gt; which contains the HMM built from the alignment in the supplementary file &lt;code&gt;13100_2017_87_MOESM8_ESM.txt&lt;/code&gt;. I am including the first few lines here (this is &lt;em&gt;not&lt;/em&gt; the entire file, don't use this as the model—build it yourself. I only include it to illustrate the format): &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;HMMER3/f [3.1b2 | February 2015]&#xA;NAME  13100_2017_87_MOESM8_ESM&#xA;LENG  121&#xA;ALPH  amino&#xA;RF    no&#xA;MM    no&#xA;CONS  yes&#xA;CS    no&#xA;MAP   yes&#xA;DATE  Mon Dec 11 13:33:29 2017&#xA;NSEQ  165&#xA;EFFN  5.697556&#xA;CKSUM 2893352190&#xA;STATS LOCAL MSV       -9.8975  0.71332&#xA;STATS LOCAL VITERBI  -10.6450  0.71332&#xA;STATS LOCAL FORWARD   -3.9247  0.71332&#xA;HMM          A        C        D        E        F        G        H        I        K        L        M        N        P        Q        R        S        T        V        W        Y   &#xA;            m-&amp;gt;m     m-&amp;gt;i     m-&amp;gt;d     i-&amp;gt;m     i-&amp;gt;i     d-&amp;gt;m     d-&amp;gt;d&#xA;  COMPO   2.34473  4.21240  2.85435  2.69203  3.54228  2.79153  3.54812  3.01140  2.72630  2.48370  3.82225  2.94269  3.49156  3.05882  2.84632  2.63343  2.86306  2.72633  4.27012  3.56181&#xA;          2.68618  4.42225  2.77519  2.73123  3.46354  2.40513  3.72494  3.29354  2.67741  2.69355  4.24690  2.90347  2.73739  3.18146  2.89801  2.37887  2.77519  2.98518  4.58477  3.61503&#xA;          0.13773  5.45955  2.08419  0.61958  0.77255  0.00000        *&#xA;      1   2.99488  4.57852  4.44659  3.29307  1.80597  4.16203  3.88788  2.87265  3.75733  2.38594  3.30344  3.33633  4.53284  3.64324  3.69630  2.90350  3.30742  2.83413  4.36991  1.50673      1 y - - -&#xA;          2.68618  4.42225  2.77519  2.73123  3.46354  2.40513  3.72494  3.29354  2.67741  2.69355  4.24690  2.90347  2.73739  3.18146  2.89801  2.37887  2.77519  2.98518  4.58477  3.61503&#xA;          0.00723  5.32905  6.05140  0.61958  0.77255  0.45275  1.01027&#xA;      2   2.66602  2.17750  5.01715  4.40788  1.72617  4.29647  4.62737  2.89579  4.21116  2.71668  3.64495  4.43195  4.66188  4.35829  3.99238  3.10363  2.02207  1.89528  5.14692  3.02155      2 f - - -&#xA;          2.68618  4.42225  2.77519  2.73123  3.46354  2.40513  3.72494  3.29354  2.67741  2.69355  4.24690  2.90347  2.73739  3.18146  2.89801  2.37887  2.77519  2.98518  4.58477  3.61503&#xA;          0.00661  5.41881  6.14115  0.61958  0.77255  0.61842  0.77391&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-12-11T11:36:49.780" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3039" PostTypeId="1" AcceptedAnswerId="3043" CreationDate="2017-12-11T14:08:37.447" Score="3" ViewCount="130" Body="&lt;p&gt;I want to use the hmmsearch proposed in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5348765/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Convergence of retrotransposons in&#xA;oomycetes and plants&lt;/em&gt;&lt;/a&gt; by Kirill Ustyantsev, Alexandr Blinov and Georgy Smyshlyaev. After the help from &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/3037/48&quot;&gt;terdon&lt;/a&gt; I managed to create the correct profile I assumed that the command &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;hmmsearch aRNH_profile.hmm prirep.ref &amp;gt; output.out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;would be the correct command. I downloaded the compressed database &lt;code&gt;RepBase22.11.fasta.tar.gz&lt;/code&gt; from &lt;a href=&quot;http://www.girinst.org/server/RepBase/index.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; in the fasta format. &#xA;However when unpacking I get the following tree &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;.&#xA;├── angrep.ref&#xA;├── appendix&#xA;│   ├── angapp.ref&#xA;│   ├── athapp.ref&#xA;│   ├── bctapp.ref&#xA;│   ├── cbrapp.ref&#xA;│  ....&#xA;├── athrep.ref&#xA;├── bctrep.ref&#xA;├── cbrrep.ref&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which file is the correct fasta file? I assumed &lt;code&gt;prirep.ref&lt;/code&gt;  might be correct file but the output shows &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[No hits detected that satisfy reporting thresholds]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2026" LastEditorUserId="48" LastEditDate="2017-12-11T15:45:56.583" LastActivityDate="2017-12-11T16:13:57.067" Title="How to use the hmmsearch on prototypic repetitive sequences Repbase Update database?" Tags="|fasta|hidden-markov-models|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3040" PostTypeId="1" CreationDate="2017-12-11T15:13:01.383" Score="5" ViewCount="1520" Body="&lt;p&gt;I'm new to PLINK and genetics, and getting confused with two PLINK commands for LD analysis:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;plink --bfile hapmap --r2 --ld-window-r2 --ld-snp-list --ld-window&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;plink --noweb --bfile --clump hapmap --clump inputfile --clump-field P --clump-p1 --clump-p2 --clump-r2 --clump-kb&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It seems to me that for #1, the input files are a reference data and a SNP list file, while for #2, it is a reference data and a .assoc file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the output for #2, p-values are included with other results (according to PLINK manual) which is not the case with the output from #1. If I'm interested in finding proxy SNPs for the SNPS on my list in hapmap data, will it be sufficient for me to use the output from #1? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After invoking command #1 using PLINK 1.9, its output gave me 6 columns: &quot;Chromosome number&quot;, &quot;chromosomal position&quot;, &quot;SNP name&quot; for query/target SNP and 1 column of &quot;R2&quot;. I cannot carry out command #1 because I do not have the .assoc file and only have a list of SNPs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My main question is - what is the difference between these two commands and which of the two should give me the best result for finding proxy SNPs for my list of SNPs.&lt;/p&gt;&#xA;" OwnerUserId="2027" LastEditorUserId="2027" LastEditDate="2017-12-12T12:44:09.013" LastActivityDate="2018-01-02T08:57:49.917" Title="LD analysis in PLINK based on reference and a SNP list" Tags="|snp|plink|molecular-genetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3041" PostTypeId="2" ParentId="3028" CreationDate="2017-12-11T15:24:31.860" Score="2" Body="&lt;p&gt;Most &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/?term=qtl+mapping+gwas&quot; rel=&quot;nofollow noreferrer&quot;&gt;papers&lt;/a&gt; that I've looked at distinguish between looking for genotype/trait associations in mapping populations created by experimental crosses (QTL mapping) and looking for genotype/trait associations in populations of unrelated individuals (GWAS). No distinction is made between susceptibility to disease (often referred to as a case-control GWAS) and continuous traits (all of the big studies on genetic associations with &lt;a href=&quot;https://www.nature.com/articles/ng.3097&quot; rel=&quot;nofollow noreferrer&quot;&gt;height&lt;/a&gt; use the term GWAS, for example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many of the papers in my first link use both methods, so you can take a look at them to get an idea of the advantages and limitations of each.&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2017-12-11T15:24:31.860" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3042" PostTypeId="1" CreationDate="2017-12-11T15:45:11.110" Score="7" ViewCount="815" Body="&lt;p&gt;I am running Picard MarkDuplicates with the following parameters below. On the file described, it takes about 41.6Gb of RAM memory and about 20-25 minutes to compute (only uses 1 core AFAICS).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -Xmx54890m -jar picard.jar MarkDuplicates \&#xA;    I=/home/dnanexus/in/sorted_bam/TST73-86-2IC_S16_L00.bml.GRCh38.karyo.bam \&#xA;    O=./TST73-86-2IC_S16_L00.bml.GRCh38.karyo.deduplicated.bam \&#xA;    M=./TST73-86-2IC_S16_L00.bml.GRCh38.karyo.duplication_metrics \&#xA;    CREATE_INDEX=true VALIDATION_STRINGENCY=SILENT REMOVE_DUPLICATES=true&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What equivalent free/open source software (samtools?, sambamba?) can I use to obtain the same deduplicated bam file but at a lower cost, considering the instance used here has 8 cores but only one of them is being used, and has about 68Gb of RAM out of which it's using 46Gb odd at peak.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think previous versions of samtools/sambamba didn't behave the same way as Picard MarkDuplicates with the parameters above, but I would like to know which recent versions would give me the same output.&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="73" LastEditDate="2017-12-11T20:00:59.823" LastActivityDate="2017-12-11T20:05:44.793" Title="low-memory high-speed replacement for Picard MarkDuplicates" Tags="|sam|samtools|sambamba|picard|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3043" PostTypeId="2" ParentId="3039" CreationDate="2017-12-11T16:13:57.067" Score="1" Body="&lt;p&gt;RepBase explain what these files are on &lt;a href=&quot;http://www.girinst.org/repbase/update/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;their website&lt;/a&gt;. It is usually a good idea to check the documentation when downloading files so you know what you're downloading. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the list of files described on the linked page, there is no &lt;code&gt;prirep.ref&lt;/code&gt; file, so I assume you mean &lt;code&gt;plnrep.ref&lt;/code&gt; which is &quot;Other Plants&quot;. In any case, again according to that page, the files with plant and/or fungal data (which is what I assume you want; you never said) are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;athrep.ref  Arabidopsis thaliana&#xA;oryrep.ref  Oryza sativa&#xA;dcotrep.ref Other Dicot Plants&#xA;mcotrep.ref Other Monocot Plants&#xA;grasrep.ref Other Grass Plants&#xA;plnrep.ref  Other Plants&#xA;fngrep.ref  Fungi&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also &lt;code&gt;chlrep.ref&lt;/code&gt; for the green alga &lt;em&gt;Chlamydomonas reinhardtii&lt;/em&gt;, and &lt;code&gt;diarep.ref&lt;/code&gt; for the diatom &lt;em&gt;Thalassiosira pseudonana&lt;/em&gt;. I don't know whether you would want to consider either here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, since these are fasta files, you cna combine them by simply &lt;code&gt;cat&lt;/code&gt;ing them together:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat athrep.ref oryrep.ref dcotrep.ref mcotrep.ref grasrep.ref \&#xA;    plnrep.ref fngrep.ref &amp;gt; plantsAndFungi.ref&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The, run your search on the &lt;code&gt;plantsAndFungi.ref&lt;/code&gt; file.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-12-11T16:13:57.067" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3044" PostTypeId="1" CreationDate="2017-12-11T17:02:13.023" Score="2" ViewCount="48" Body="&lt;p&gt;How do I know if it is a AND or OR relationship between genes associated with the same enzyme in KEGG? E.g., for EC:1.6.1.2 (&lt;a href=&quot;http://www.genome.jp/dbget-bin/www_bget?ec:1.6.1.2&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.genome.jp/dbget-bin/www_bget?ec:1.6.1.2&lt;/a&gt;) does the bacterium &lt;em&gt;Streptomyces coelicolor&lt;/em&gt; (SCO) need SCO7622 AND SCO7623 or SCO7622 OR SCO7623? &lt;a href=&quot;https://i.stack.imgur.com/zIaRw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zIaRw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="1670" LastEditorUserId="1670" LastEditDate="2017-12-18T12:26:16.460" LastActivityDate="2017-12-18T12:26:16.460" Title="AND or OR gene - protein logical relation in KEGG" Tags="|gene|kegg|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3045" PostTypeId="1" AcceptedAnswerId="17478" CreationDate="2017-12-11T18:10:57.140" Score="3" ViewCount="61" Body="&lt;p&gt;This is probably a long-shot since it would be so dependent upon the underlying folding algorithms. Are there known algorithms for mapping the path followed in a protein folding simulation when you have the starting conformation and the solution conformation such that you can count the number of steps involved in going from one to the other? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The starting conformation would be something generated from MSM Builder while the solution conformation would be the result of hours, days, weeks, months, etc of folding.&lt;/p&gt;&#xA;" OwnerUserId="1842" LastEditorUserId="48" LastEditDate="2017-12-11T21:38:16.810" LastActivityDate="2021-08-03T09:02:41.597" Title="Protein fold pathfinding?" Tags="|protein-structure|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3047" PostTypeId="2" ParentId="3042" CreationDate="2017-12-11T20:05:44.793" Score="2" Body="&lt;p&gt;As you suggested, sambamba is faster at marking duplicates than picard (it's also multithreaded). Recent versions of samtools have a rewritten duplicate marking algorithm, though I doubt it'll be as quick as sambamba. 46GB of RAM seems excessive for marking duplicates unless you're having it store the whole file in memory.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-12-11T20:05:44.793" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3048" PostTypeId="2" ParentId="3044" CreationDate="2017-12-11T21:16:38.073" Score="1" Body="&lt;p&gt;The enzyme EC:1.6.1.2 is an NAD(P) transhydrogenase and consists of two subunits. The two subunits are encoded by two different genes &lt;a href=&quot;http://www.genome.jp/dbget-bin/www_bget?sco+SCO7623&quot; rel=&quot;nofollow noreferrer&quot;&gt;pntA&lt;/a&gt; and &lt;a href=&quot;http://www.genome.jp/dbget-bin/www_bget?sco:SCO7622&quot; rel=&quot;nofollow noreferrer&quot;&gt;pntB&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in terms of computer logic I would say AND, both subunits are necessary for a functional NAD(P) transhydrogenase enzyme (EC:1.6.1.2).&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2017-12-11T21:16:38.073" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3049" PostTypeId="1" AcceptedAnswerId="8956" CreationDate="2017-12-11T22:52:11.853" Score="5" ViewCount="127" Body="&lt;p&gt;I am searching for tools to pull CRISPR Spacers from Bacterial Genomes. I am aware of the &lt;a href=&quot;http://crispr.i2bc.paris-saclay.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CRISPRDB&lt;/a&gt; and the corresponding identification tool on the web server. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Are there other tools for finding CRISPR Spacers?&lt;/li&gt;&#xA;&lt;li&gt;Is there a way to download the spacer sequences from CRISPRDB?&lt;/li&gt;&#xA;&lt;li&gt;Is there another database that contains all validated / inferred bacterial CRISPR sequences?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1833" LastEditorUserId="96" LastEditDate="2019-07-11T13:45:51.270" LastActivityDate="2019-12-08T15:01:10.470" Title="CRISPR Sequence Finder and Database Download" Tags="|database|software-recommendation|data-retrieval|crispr|bacteria|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3050" PostTypeId="2" ParentId="3012" CreationDate="2017-12-12T11:02:00.980" Score="1" Body="&lt;p&gt;If you want to find version info you can either :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ask for another edirect tool version (it will be the same version for edirect) :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;efetch -version&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;6.80&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Check the info contained in the edirect.pl script :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -i 'version = ' edirect.pl&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;$version = &quot;6.80&quot;;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1810" LastActivityDate="2017-12-12T11:02:00.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3051" PostTypeId="2" ParentId="3034" CreationDate="2017-12-12T11:21:52.930" Score="5" Body="&lt;p&gt;I think you might want to use &lt;a href=&quot;https://www.bioconductor.org/help/workflows/simpleSingleCell/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the most recent workflow&lt;/a&gt;, which should be for the latest version on Bioconductor.&lt;/p&gt;&#xA;" OwnerUserId="306" LastEditorUserId="29" LastEditDate="2017-12-12T12:53:10.413" LastActivityDate="2017-12-12T12:53:10.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3052" PostTypeId="5" CreationDate="2017-12-12T11:23:11.760" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2017-12-12T11:23:11.760" LastActivityDate="2017-12-12T11:23:11.760" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3053" PostTypeId="4" CreationDate="2017-12-12T11:23:11.760" Score="0" Body="This is a tag for molecular pathways in an organism, their representation, relationships, functions and databases storing said information." OwnerUserId="48" LastEditorUserId="77" LastEditDate="2017-12-14T07:57:42.627" LastActivityDate="2017-12-14T07:57:42.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3054" PostTypeId="1" AcceptedAnswerId="3058" CreationDate="2017-12-12T11:38:35.863" Score="5" ViewCount="288" Body="&lt;p&gt;Is it possible to check if patient has the HLAB27 antigen by comparing his exome stored in VCF with all known HLAB27 variants?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, rs4349859 is a &lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Variation/Explore?r=6:31397510-31398510;source=dbSNP;v=rs4349859;vdb=variation;vf=2732890&quot; rel=&quot;nofollow noreferrer&quot;&gt;known&lt;/a&gt; HLAB27 SNP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I can find rs4349859 in the patient's VCF file, could I say he is HLAB27+ ?&lt;/p&gt;&#xA;" OwnerUserId="2033" LastEditorUserId="48" LastEditDate="2017-12-12T13:48:22.823" LastActivityDate="2017-12-12T15:17:48.430" Title="Is it possible to check if a patient has the HLA-B27 antigen from his exome stored in a VCF file" Tags="|vcf|exome|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3055" PostTypeId="2" ParentId="254" CreationDate="2017-12-12T12:38:49.023" Score="3" Body="&lt;p&gt;One way to group similar significant pathways, is to quantify how many genes overlap between the pathways, and then use this in clustering (heatmap). I have made a tool in R which calculates the overlap index between GO terms and subsequently clusters them in a heatmap. Overlap index is the fraction of genes that overlap (number between 0-1). Also Pearson correlation can be used for clustering instead of the overlap index. My package (&lt;a href=&quot;https://sourceforge.net/projects/gogadget/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gogadget&lt;/a&gt;) works only with  goseq analysis, but you can use &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/goseq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;goseq&lt;/a&gt; also for reactome or kegg data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used this overlap clustering approach for different data sets now, and usually we can reduce 200-300 GO terms into 10-20 functional groups.&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2017-12-12T12:48:53.227" LastActivityDate="2017-12-12T12:48:53.227" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3056" PostTypeId="1" CreationDate="2017-12-12T12:57:01.693" Score="1" ViewCount="59" Body="&lt;p&gt;I want to recreate the UGENE workflow proposed in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5348765/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Convergence of retrotransposons in oomycetes and plants&lt;/em&gt;&lt;/a&gt;. After being able to understand the data and hmmsearch &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/3039/2026&quot;&gt;Here&lt;/a&gt; and &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/3036/2026&quot;&gt;Here&lt;/a&gt; and getting most of the result I want to try to recreate the workflow in UGENE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I manged to implement the workflow for the search &lt;a href=&quot;https://i.stack.imgur.com/5VTqt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5VTqt.png&quot; alt=&quot;Ugene hmm search Workflow&quot;&gt;&lt;/a&gt;&#xA;However in the latter parts I struggle to implement the steps of the algorithm&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;sequences surrounding the regions of significant similarity to the aRNH profile were expanded, when possible, to 10,000 bp in both directions&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How can sequences expanded in UGENE?&lt;/li&gt;&#xA;&lt;li&gt;How can implement the next steps also? for step three &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Third, the enlarged sequences were screened for the presence of significant similarity to RT domains of non-LTR-RTs and LTR-RTs HMM profiles using hmmsearch.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I assume it is very simmilar to the first step in building the right profile and running hmmsearch but the following steps are still unclear to me. &lt;/p&gt;&#xA;" OwnerUserId="2026" LastActivityDate="2017-12-12T12:57:01.693" Title="How to rebuild the workflow for mining in oomynete genomes in UGENE" Tags="|sequence-analysis|hidden-markov-models|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3057" PostTypeId="1" CreationDate="2017-12-12T13:33:06.243" Score="7" ViewCount="640" Body="&lt;p&gt;I'd like to create a dataset consisting of all sequences which are either present in the PDB, or whose homolog is present in the PDB. In other words, any sequence in the PDB or any sequence related to it. The similiarity margin is to be very wide, so anything above, for instance, 25% sequence identity is accepted (perhaps there are better criteria, but you get the point). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;No further limitations, so any organism, with or without experimental data, not limited to the manually curated Swissprot set. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried the SIFTS database, but that contains, as far as I can tell, only mappings for sequence identity &gt;85%. I would like to go far below that. &lt;/p&gt;&#xA;" OwnerUserId="2034" LastEditorUserId="2034" LastEditDate="2018-07-02T23:27:52.367" LastActivityDate="2018-07-02T23:27:52.367" Title="How to get all PDB homologs from Uniprot (mapping + BLAST)?" Tags="|python|public-databases|pdb|sequence-homology|uniprot|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3058" PostTypeId="2" ParentId="3054" CreationDate="2017-12-12T15:17:48.430" Score="2" Body="&lt;p&gt;Some papers claim that you can do HLA typing from exome sequencing data. I have not validated this myself. I also guess it depends on capture. Nonetheless, I don't think you can type HLA with VCF generated with a standard pipeline, which is not specifically designed for HLA. Current variant callers won't work on HLA genes due to their high diversity. Also in general, there are thousands of HLA-B alleles. In most cases, you can't find the allele by looking at only one SNP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No, you can't check if the patient has HLA-B27 only from VCF. There might be a hope if you get raw reads and run a proper HLA typing tool designed for such data.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2017-12-12T15:17:48.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3059" PostTypeId="1" AcceptedAnswerId="3060" CreationDate="2017-12-12T16:17:37.963" Score="4" ViewCount="446" Body="&lt;p&gt;The quote below is from &lt;a href=&quot;https://www.nature.com/articles/nbt.2967#s1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We performed principal component analysis (PCA) of low-coverage sequencing data to identify genes explaining variation across cells. PCA separated the cells into groups corresponding to the source populations (Fig. 2c and Supplementary Figs. 3–5), and the genes distinguishing each group reflected the biological properties of the cell types (Supplementary Fig. 5 and Supplementary Table 3). PCA of low- and high-coverage sequencing data revealed a remarkably similar graphical distribution of analyzed cells, and the majority (78%) of the top 500 genes determined by PCA were shared between PCA performed on low- and high-coverage data (Supplementary Figs. 4 and 6 and Supplementary Table 4). &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How are they performing PCA and then finding out the components which contribute to the strongest PC and then taking out the genes for downstream analysis? Let's say there are genes involved in PC1 and PC2 which define a certain cell type and that distinguish it. I think for a biologist, like me, I just want to get the genes or gene list that are getting involved in determining lineage or cell type. How can I do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a to test what they might be doing i did this &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;test&amp;lt;- read.csv('NON_CODING.csv',header = T,row.names = 1)&#xA;&#xA;# preform PCA&#xA;pca = prcomp(t(test), center=TRUE, scale=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then after calculating PCA i do see this in my console when I type pca&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; pca&#xA;Standard deviations (1, .., p=16):&#xA; [1] 3.826851e+01 2.080405e+01 1.568739e+01 1.349256e+01 1.119348e+01 9.980547e+00 8.365034e+00&#xA; [8] 8.098841e+00 7.519507e+00 6.184505e+00 5.880260e+00 5.139609e+00 4.851091e+00 4.335606e+00&#xA;[15] 3.870918e+00 3.173450e-14&#xA;&#xA;Rotation (n x k) = (2899 x 16):&#xA;                                       PC1           PC2           PC3           PC4&#xA;5S_rRNA                      -1.090574e-02 -2.412665e-03  1.637689e-02 -3.603865e-02&#xA;AB019441.29                  -1.928250e-02  1.821083e-03 -9.713724e-03 -1.978737e-03&#xA;ABBA01017803.1               -1.823266e-02 -3.727144e-03 -9.790131e-04 -3.937062e-02&#xA;ABC14-1080714F14.1            2.438024e-02  3.816019e-03  1.657784e-04 -7.141480e-03&#xA;ABC7-481722F1.1               2.467403e-02  2.873432e-03  3.781153e-03  3.790824e-03&#xA;AC000036.4                   -1.066777e-02  2.838305e-02  2.642673e-02 -7.998608e-03&#xA;AC000089.3                   -1.249026e-02 -7.870864e-04 -2.569602e-02 -3.165073e-03&#xA;AC000120.7                   -1.313696e-02  5.923245e-03  1.633255e-02  4.077602e-02&#xA;AC000123.4                    2.415996e-02  1.029732e-02  8.930792e-03 -1.118660e-02&#xA;AC000403.4                    8.692835e-03 -3.060170e-02  3.746839e-02 -6.968971e-03&#xA;AC002064.4                   -1.799590e-02  2.147160e-02 -2.303742e-02  7.647937e-03&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now how do i find which are the list or set of genes that gives me PC1 or PC2 so on &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion or help how to get the genes that make most difference between two major component that would be very helpful &lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="298" LastEditDate="2017-12-13T13:11:02.120" LastActivityDate="2017-12-14T07:48:48.733" Title="Extracting features or gene from PCA after calculating PCA for downstream analysis" Tags="|r|rna-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3060" PostTypeId="2" ParentId="3059" CreationDate="2017-12-12T16:50:18.493" Score="7" Body="&lt;p&gt;The &lt;a href=&quot;http://stat.ethz.ch/R-manual/R-devel/library/stats/html/prcomp.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;help page&lt;/a&gt; explains that there is a component which:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;rotation&lt;br&gt;&#xA;  the matrix of variable loadings (i.e., a matrix whose columns contain the eigenvectors). The function princomp returns this in the element loadings.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So using using the rotation one can use it to calculate the importance of each feature in the input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;C &amp;lt;- chol(S &amp;lt;- toeplitz(.9 ^ (0:31))) # Cov.matrix and its root&#xA;set.seed(17)&#xA;X &amp;lt;- matrix(rnorm(32000), 1000, 32)&#xA;Z &amp;lt;- X %*% C  ## ==&amp;gt;  cov(Z) ~=  C'C = S&#xA;pZ &amp;lt;- prcomp(Z)&#xA;genesImportance &amp;lt;- Z %*% pZ$rotation&#xA;genesImportance2 &amp;lt;- pZ$x&#xA;all.equal(genesImportance, genesImportance2)&#xA;## [1] &quot;Mean relative difference: 0.03680457&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This new matrix has the loading for each gene for each component. So you can now select them according to how much do they participate in each component or whatever you want. Or you can use it to predict the position of samples you didn't set in the initial data. &lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2017-12-14T07:48:48.733" LastActivityDate="2017-12-14T07:48:48.733" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="3061" PostTypeId="1" AcceptedAnswerId="3092" CreationDate="2017-12-12T18:59:47.060" Score="0" ViewCount="638" Body="&lt;p&gt;I am trying to run salmon and it keeps giving me 2 java exceptions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java.io.EOFException&#xA;    at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:268)&#xA;    at java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:258)&#xA;    at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:164)&#xA;    at java.util.zip.GZIPInputStream.&amp;lt;init&amp;gt;(GZIPInputStream.java:79)&#xA;    at java.util.zip.GZIPInputStream.&amp;lt;init&amp;gt;(GZIPInputStream.java:91)&#xA;    at uk.ac.babraham.FastQC.Utilities.MultiMemberGZIPInputStream.&amp;lt;init&amp;gt;(MultiMemberGZIPInputStream.java:37)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java.lang.ArrayIndexOutOfBoundsException: -1&#xA;    at uk.ac.babraham.FastQC.Modules.SequenceLengthDistribution.calculateDistribution(SequenceLengthDistribution.java:100)&#xA;    at uk.ac.babraham.FastQC.Modules.SequenceLengthDistribution.raisesError(SequenceLengthDistribution.java:184)&#xA;    at uk.ac.babraham.FastQC.Report.HTMLReportArchive.startDocument(HTMLReportArchive.java:336)&#xA;    at uk.ac.babraham.FastQC.Report.HTMLReportArchive.&amp;lt;init&amp;gt;(HTMLReportArchive.java:84)&#xA;    at uk.ac.babraham.FastQC.Analysis.OfflineRunner.analysisComplete(OfflineRunner.java:155)&#xA;    at uk.ac.babraham.FastQC.Analysis.AnalysisRunner.run(AnalysisRunner.java:110)&#xA;    at java.lang.Thread.run(Thread.java:745)&#xA;&#xA;    at uk.ac.babraham.FastQC.Sequence.FastQFile.&amp;lt;init&amp;gt;(FastQFile.java:80)&#xA;    at uk.ac.babraham.FastQC.Sequence.SequenceFactory.getSequenceFile(SequenceFactory.java:106)&#xA;    at uk.ac.babraham.FastQC.Sequence.SequenceFactory.getSequenceFile(SequenceFactory.java:62)&#xA;    at uk.ac.babraham.FastQC.Analysis.OfflineRunner.processFile(OfflineRunner.java:129)&#xA;    at uk.ac.babraham.FastQC.Analysis.OfflineRunner.&amp;lt;init&amp;gt;(OfflineRunner.java:102)&#xA;    at uk.ac.babraham.FastQC.FastQCApplication.main(FastQCApplication.java:316)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I installed &lt;code&gt;salmon&lt;/code&gt; using &lt;code&gt;Anaconda&lt;/code&gt;s' &lt;code&gt;conda install -c bioconda salmon&lt;/code&gt; and all other necessary packages in the same way. I thought that it could be a problem with &lt;code&gt;fastqc&lt;/code&gt;, so I uninstalled it and then installed it manually (through the fastqc&lt;code&gt;.zip&lt;/code&gt; file), but the output remained the same.&#xA;I am using &lt;code&gt;Java8&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -version&#xA;openjdk version &quot;1.8.0_92&quot;&#xA;OpenJDK Runtime Environment (Zulu 8.15.0.1-linux64) (build 1.8.0_92-b15)&#xA;OpenJDK 64-Bit Server VM (Zulu 8.15.0.1-linux64) (build 25.92-b15, mixed mode)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am running the script on a cluster with &lt;code&gt;SLURM&lt;/code&gt;. Locally on my laptop the script is running without any issues. I am really stuck and do not know what to try, so any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Update&lt;/code&gt;&#xA;Another issue that I just found is the following (the last command that &lt;code&gt;salmon&lt;/code&gt; is trying to execute):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Exception : [Failed to read 8 bytes from input stream! Read 0]&#xA;salmon quant was invoked improperly.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I searched online for the issue and found the following thread:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/COMBINE-lab/salmon/issues/129&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/COMBINE-lab/salmon/issues/129&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, updating &lt;code&gt;salmon&lt;/code&gt; to the newest &lt;code&gt;0.9.1&lt;/code&gt; version did not solve the issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Edit&lt;/code&gt;&#xA;The command I am running is the following one:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python &quot;$path/salmon_rna_seq.py&quot; -L 'IU' -E &quot;$path/jyvo_experiment-&#xA;metadata.yaml&quot; -T 'gencode_mouse_m13' -C &quot;$path/salmon_rna_seq.yaml&quot; -I &#xA;&quot;$path_to_fastq/$name-lec_1_r1.fq.gz&quot; &quot;$path_to_fastq/$name-&#xA;lec_1_r2.fq.gz&quot; -O &quot;$path/res/$name-lec_1&quot; -S &quot;$name-lec_1&quot; -P '6' -X 'True'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The command is run by &lt;code&gt;Pypiper&lt;/code&gt;: &lt;a href=&quot;https://github.com/epigen/pypiper&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/epigen/pypiper&lt;/a&gt;&#xA;Inside of &lt;code&gt;salmon_rna_seq.py&lt;/code&gt; I am also running commands to treat the data, the &lt;code&gt;salmon&lt;/code&gt; part looks the following way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def salmon(self):&#xA;    cmd = &quot;salmon quant -i &quot; + self.indexed_transcriptome&#xA;    cmd += &quot; -l &quot; + self.lib_type&#xA;&#xA;    if self.trim:&#xA;        cmd += &quot; -1 &quot; + self.sample_files['r1']['fastq']['trimmed_paired']&#xA;        cmd += &quot; -2 &quot; + self.sample_files['r2']['fastq']['trimmed_paired']&#xA;    else:&#xA;        cmd += &quot; -1 &quot; + self.sample_files['r1']['fastq']['chastity']&#xA;        cmd += &quot; -2 &quot; + self.sample_files['r2']['fastq']['chastity']&#xA;&#xA;    cmd += &quot; -o &quot; + self.sample_dirs['salmon']['base']&#xA;    cmd += &quot; --numBootstraps=30&quot;&#xA;self.pipe_manager.run(cmd, self.sample_files['salmon'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Commands that the &lt;code&gt;Pypiper&lt;/code&gt; runs are the following ones (&lt;code&gt;_commands.sh&lt;/code&gt; &lt;code&gt;Pypiper&lt;/code&gt; file): &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zcat -c $PATH/res/old-lec_1/fastq/merged/old-lec_1_r1.fq.gz | grep -A 3 &#xA;'^@.*[^:]*:N:[^:]*:' | grep -v '^\-\-$' | gzip -c &amp;gt; $PATH/res/old-lec_1/fastq&#xA;/chastity/old-lec_1_r1.fq.gz&#xA;&#xA;zcat -c $PATH/res/old-lec_1/fastq/merged/old-lec_1_r2.fq.gz | grep -A 3 &#xA;'^@.*[^:]*:N:[^:]*:' | grep -v '^\-\-$' | gzip -c &amp;gt; $PATH/res/old-lec_1/fastq&#xA;/chastity/old-lec_1_r2.fq.gz&#xA;&#xA;java -jar $HOME/anaconda2/share/trimmomatic-0.36-5&#xA;/trimmomatic.jar PE -threads 6 -phred33 $PATH/res/old-lec_1/fastq/chastity/old-&#xA;lec_1_r1.fq.gz $PATH/res/old-lec_1/fastq/chastity/old-lec_1_r2.fq.gz $PATH/res/old-&#xA;lec_1/fastq/trimmed/old-lec_1_r1.fq.gz $PATH/res/old-lec_1/fastq/trimmed/unpaired&#xA;/old-lec_1_r1.fq.gz $PATH/res/old-lec_1/fastq/trimmed/old-lec_1_r2.fq.gz $PATH/res/old-&#xA;lec_1/fastq/trimmed/unpaired/old-lec_1_r2.fq.gz ILLUMINACLIP:$HOME/anaconda2/share/trimmomatic-0.36-5/adapters&#xA;/Small_List_Of_Adapters.fa:2:30:10 LEADING:3 TRAILING:3 SLIDINGWINDOW:4:15&#xA;MINLEN:36&#xA;&#xA;fastqc $PATH/res/old-lec_1/fastq/merged/old-lec_1_r1.fq.gz --outdir $PATH/res/old-&#xA;lec_1/fastqc/merged&#xA;&#xA;fastqc $PATH/res/old-lec_1/fastq/chastity/old-lec_1_r1.fq.gz --outdir $PATH/res/old-&#xA;lec_1/fastqc/chastity&#xA;&#xA;fastqc $PATH/res/old-lec_1/fastq/merged/old-lec_1_r2.fq.gz --outdir $PATH/res/old-&#xA;lec_1/fastqc/merged&#xA;&#xA;fastqc $PATH/res/old-lec_1/fastq/chastity/old-lec_1_r2.fq.gz --outdir $PATH/res/old-&#xA;lec_1/fastqc/chastity&#xA;&#xA;fastqc $PATH/res/old-lec_1/fastq/trimmed/old-lec_1_r1.fq.gz --outdir $PATH/res/old-&#xA;lec_1/fastqc/trimmed&#xA;&#xA;fastqc $PATH/res/old-lec_1/fastq/trimmed/old-lec_1_r2.fq.gz --outdir $PATH/res/old-&#xA;lec_1/fastqc/trimmed&#xA;&#xA;salmon quant -i $HOME/Genomes/mouse/salmon&#xA;/gencode.vM13.transcripts.fa.sidx -l IU -1 $PATH/res/old-lec_1/fastq/trimmed&#xA;/old-lec_1_r1.fq.gz -2 $PATH/res/old-lec_1/fastq/trimmed/old-&#xA;lec_1_r2.fq.gz -o $PATH/res/old-lec_1/quantification --numBootstraps=30&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Edit&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After running manually &lt;code&gt;gzip -t&lt;/code&gt; on a file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-bash-4.2$ gzip -t old-lec_1_r1.fq.gz &#xA;gzip: old-lec_1_r1.fq.gz: unexpected end of file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, I guess the file is corrupted and the issue is with some library that is generating the file. However, reinstalling manually other libraries do not help.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="2005" LastEditDate="2017-12-13T18:49:57.777" LastActivityDate="2017-12-14T22:39:22.913" Title="Exception : [Failed to read 8 bytes from input stream! Read 0] salmon quant was invoked improperly" Tags="|rna-seq|quantification|salmon|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="3062" PostTypeId="1" AcceptedAnswerId="4181" CreationDate="2017-12-13T13:23:15.630" Score="1" ViewCount="724" Body="&lt;p&gt;I'm trying to run t-sne on my coding as well as non coding genes. Even though I'm getting output I'm not confident about my code (The output it's difficult to interpret).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a data set of total 16 cell type and about 3000 genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how my data is first column is gene rest is sample:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gene    H1  H2  H3  H4  M2  M3  M4  C1  C2  C3  C4  G2  G3  G4  Gran1   Gran2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The code used:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(Rtsne)&#xA;file1&amp;lt;- read.csv('UNIQUE_WGCNA_FILE.csv',header = T,row.names = 1)&#xA;trn &amp;lt;- data.matrix(file1)&#xA;dim(trn)&#xA;&#xA;tsne &amp;lt;- Rtsne(as.matrix(trn[,1:16]), check_duplicates = FALSE, pca = FALSE, perplexity=50, theta=0.5, dims=3)&#xA;&#xA;cols &amp;lt;- rainbow(10)&#xA;plot(tsne$Y, t='n')&#xA;text(tsne$Y, labels=as.numeric(trn[,1]), col=cols[trn[,1]])&#xA;&#xA;require(rgl)&#xA;plot3d(tsne$Y, col=cols[trn[,1]])&#xA;legend3d(&quot;topright&quot;, legend = '0':'5', pch = 16, col = rainbow(10))&#xA;rgl.postscript(&quot;Merge.pdf&quot;, &quot;pdf&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm getting output but I don't know how to label so its becoming difficult.&#xA;Any suggestion or help how to label would be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uoILK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uoILK.png&quot; alt=&quot;t_sne_without_label&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="2719" LastEditDate="2018-05-03T07:06:50.583" LastActivityDate="2018-05-03T07:06:50.583" Title="t-sne code and labelling issue for rna seq data" Tags="|r|rna-seq|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="3063" PostTypeId="1" CreationDate="2017-12-13T14:57:34.397" Score="4" ViewCount="811" Body="&lt;p&gt;I'm given a Fasta file, containing a large DNA(over 115,000 long) sequence, and I am tasked with finding a &lt;strong&gt;single&lt;/strong&gt; large open reading frame contained within the DNA sequence using Biopython. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm aware this has been asked before consistently however i can't find anything for actually finding the source of my ORF, but apologies for any overlap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now from other sources and the Biopython cookbook I've translated my sequence and found six open reading frames (three for each strand) and their positions within the sequence;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def find_orfs_with_trans(seq, trans_table, min_protein_length):&#xA;    answer = []&#xA;    seq_len = len(seq)&#xA;    for strand, nuc in [(+1, seq), (-1, seq.reverse_complement())]:&#xA;        for frame in range(3):&#xA;            trans = str(nuc[frame:].translate(trans_table))&#xA;            trans_len = len(trans)&#xA;            aa_start = 0&#xA;            aa_end = 0&#xA;            while aa_start &amp;lt; trans_len:&#xA;                aa_end = trans.find(&quot;*&quot;, aa_start)&#xA;                if aa_end == -1:&#xA;                    aa_end = trans_len&#xA;                if aa_end-aa_start &amp;gt;= min_protein_length:&#xA;                    if strand == 1:&#xA;                        start = frame+aa_start*3&#xA;                        end = min(seq_len,frame+aa_end*3+3)&#xA;                    else:&#xA;                        start = seq_len-frame-aa_end*3-3&#xA;                        end = seq_len-frame-aa_start*3&#xA;                    answer.append((start, end, strand,&#xA;                                   trans[aa_start:aa_end]))&#xA;                aa_start = aa_end+1&#xA;    answer.sort()&#xA;    return answer&#xA;&#xA;orf_list = find_orfs_with_trans(record.seq, table, min_pro_len)&#xA;&#xA;for start, end, strand, pro in orf_list:&#xA;print(&quot;%s...%s - length %i, strand %i, %i:%i&quot; \&#xA;      % (pro[:30], pro[-3:], len(pro), strand, start, end))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;OUTPUT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; TNRQVYGGTLQSLRTGTGIYSRLASSPTNR...CEA - length 160, strand 1, 7950:8433&#xA; GIPPGRTEGLGRYVHGESIFLEATLVPEPQ...VQS - length 171, strand -1, 19275:19791&#xA; ISVHLRRYFSVLLRAPVALNADSRVAGPLD...ITP - length 190, strand 1, 34079:34652&#xA; GCVNGNFPDYRVAVDDPGALVVGGELGQAL...VNT - length 771, strand -1, 39335:41651&#xA; LLKLLASRLTWLLVLSAALGFLTSVSYRLG...SPG - length 235, strand -1, 39358:40066&#xA; PGDVIVSKVPADNLRPRMSEINDFTNSIII...VLV - length 826, strand 1, 39362:41843&#xA; GPGRQSSSAYERDQRLHQQHHHQRERHQVR...QRS - length 764, strand 1, 39385:41680&#xA; LLLSSTSSASCWLKKFFFRLLLLSRSVALL...LAL - length 158, strand -1, 40459:40936&#xA; TRLIIRPPSHTTLYYDPYPQPLKVSSSVPC...RIT - length 159, strand -1, 54663:55143&#xA; IGAFSRGYNQGWVLIEAQEVRTTSPLFPRS...RDQ - length 201, strand 1, 56104:56710&#xA; KARGSSLGAIYCRRLKRRSLPSWQIESFNC...TNS - length 150, strand -1, 62497:62950&#xA; RRPARAGAELNKDDIRDTCLLSCSEVDNKV...VTL - length 168, strand 1, 72339:72846&#xA; SGPTTVRTSAPQRCWRTSLKPKLSLSDSTE...PVA - length 177, strand -1, 114494:115028&#xA; ERELWLETGPPTPLWGTGPDCSRAALNRQR...PRR - length 157, strand 1, 114953:115427&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However I'm unsure if these open reading frames contain the ribosome binding site(consensus sequence of &lt;em&gt;AAGGAGGTG&lt;/em&gt; and occurring between &lt;strong&gt;6 to 9&lt;/strong&gt; nucleotides upstream from the positive 5' strand). Is there anyway to add to my code to include this? or would it already be included with Biopython?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My specification says to find &lt;strong&gt;the&lt;/strong&gt; single open reading frame however since none of these begin the the &lt;em&gt;M&lt;/em&gt; start codon i'm a bit lost how they can be so, I have no information on where my sequence is from (something I have to find out) so using the standard &lt;strong&gt;translation table 1&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1860" LastEditorUserId="73" LastEditDate="2017-12-13T18:43:55.170" LastActivityDate="2017-12-13T18:43:55.170" Title="Finding a single open reading frame with ribosomal binding site, using Biopython" Tags="|python|homework|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3064" PostTypeId="2" ParentId="3063" CreationDate="2017-12-13T16:47:48.087" Score="2" Body="&lt;p&gt;You should start by finding all instances of &lt;code&gt;AAGGAGGTG&lt;/code&gt; and then look for ORFs that begin 6 to 9 nucleotides downstream of them. By begin, I mean find an AUG codon, and then extend until the next in-frame stop codon. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also no point in translating, just look for long stretches of sequence between AUG and UGA, UAA or UAG, those will define your reading frames. You seem to be starting the reading frame at any non-stop codon instead of starting from AUGs. This may well make biological sense in some contexts (there are proteins that don't start with M) but is probably beyond the scope of what sounds like a simple assignment. &lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-12-13T18:43:16.253" LastActivityDate="2017-12-13T18:43:16.253" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3065" PostTypeId="1" AcceptedAnswerId="3069" CreationDate="2017-12-13T17:20:36.860" Score="1" ViewCount="61" Body="&lt;p&gt;I have a set of whole genome sequencing samples, some of which have matched normal while some not. I want to call somatic mutations on non-coding regions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was looking for GATK best practices (like &lt;a href=&quot;https://software.broadinstitute.org/gatk/best-practices/mutect2.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;) but it seems it is not yet ready for production. Which alternative tools do you suggest?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2017-12-13T18:49:27.003" Title="Which tools are available to call somatic mutations on non-coding regions from whole genome sequencing data?" Tags="|wgs|mutations|somatic|non-coding|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3066" PostTypeId="1" AcceptedAnswerId="3068" CreationDate="2017-12-13T17:54:06.453" Score="2" ViewCount="71" Body="&lt;p&gt;I am having &lt;code&gt;RNA-seq&lt;/code&gt; data set for &lt;code&gt;C.Elegans&lt;/code&gt;. I do not know which annotation scientists used for naming the genes of it, but I need to search somehow for the &lt;code&gt;^ERCC&lt;/code&gt; and &lt;code&gt;^mt-&lt;/code&gt; genes like specified in the tutorial:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.bioconductor.org/help/workflows/simpleSingleCell/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioconductor.org/help/workflows/simpleSingleCell/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I just implement &lt;code&gt;grepl&lt;/code&gt; search for the &lt;code&gt;character&lt;/code&gt; array of gene names that I have, I am always getting all values &lt;code&gt;FALSE&lt;/code&gt; as if there are no &lt;code&gt;spike-in&lt;/code&gt;s and mitochindrial genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sample gene name from the dataset: &lt;code&gt;WBGene00023193&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="2005" LastEditDate="2017-12-13T18:53:52.020" LastActivityDate="2017-12-13T18:53:52.020" Title="Get gene annotation type from gene name" Tags="|rna-seq|gene|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3068" PostTypeId="2" ParentId="3066" CreationDate="2017-12-13T18:27:49.483" Score="2" Body="&lt;p&gt;The tutorial you linked to does not mention ERCC or mt genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before running &lt;code&gt;grepl&lt;/code&gt; or any other search, you should first check your inputs. It looks like you have WormBase gene IDs if all your genes look like &lt;code&gt;WBGene00023193&lt;/code&gt;. Therefore, your question is actually how to convert WormBase gene IDs to gene symbols.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is one potential solution: &lt;a href=&quot;https://github.com/WormBase/website/issues/4297&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/WormBase/website/issues/4297&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After you convert, you have to make sure that you have ERCC spike-ins in your gene table (probably not) and you mitochondrial genes start with &quot;mt-&quot;.&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="35" LastEditDate="2017-12-13T18:37:47.017" LastActivityDate="2017-12-13T18:37:47.017" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3069" PostTypeId="2" ParentId="3065" CreationDate="2017-12-13T18:49:27.003" Score="1" Body="&lt;p&gt;Those will have to be treated as separate analysis.  You can use Mutect or VarScan for the matched samples and the tools listed in the &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2744/somatic-tumor-only-variant-calling&quot;&gt;other question&lt;/a&gt; for the unmatched samples however that means that you won't be able to compare the results easily.  If the inter-sample comparison is important you will have to run all samples as unmatched.  Personally I have used LoFreq for somatic variant detection with some success.  It is very sensitive so the false positive rate can be high depending on how noisy your data is.&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2017-12-13T18:49:27.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3070" PostTypeId="2" ParentId="3008" CreationDate="2017-12-13T18:51:29.993" Score="2" Body="&lt;p&gt;Single-cell tools were developed to address some specific factors unique to scRNA-seq, such as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a lot of dropouts, so hundreds or thousands of detected genes, not tens of thousands (for mammalian genomes)&lt;/li&gt;&#xA;&lt;li&gt;hundreds or thousands of replicates&lt;/li&gt;&#xA;&lt;li&gt;not necessarily well-defined groups&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thus, I would say your experiment is a lot closer to bulk RNA-seq than scRNA-seq.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2017-12-13T18:51:29.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3071" PostTypeId="1" CreationDate="2017-12-13T19:10:02.260" Score="2" ViewCount="1764" Body="&lt;p&gt;I have a text file as follows&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a   k   testis  adult   male    8 week  rRNA    https://www.encodeproject.org/files/ENCFF001JPX/@@download/ENCFF001JPX.fastq.gz&#xA;b   k   testis  adult   male    8 week  rRNA    https://www.encodeproject.org/files/ENCFF001JPZ/@@download/ENCFF001JPZ.fastq.gz&#xA;c   k   testis  adult   male    8 week  rRNA    https://www.encodeproject.org/files/ENCFF001JQB/@@download/ENCFF001JQB.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The file is tab delimited and has 8 columns with 8th column being the url link  I need to download the files one by one using wget and changing the name of the downloaded file with -O option in wget. The argument I want to give to the -O option is combination col1 and col2 separated by _ so that for first line I get the downloaded file as a_k.fastq.gz and so on. For example, the first line of the file above should give:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wget https://www.encodeproject.org/files/ENCFF001JPX/@@download/ENCFF001JPX.fastq.gz -O a_k.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Following the post from &lt;a href=&quot;https://unix.stackexchange.com/questions/410710/splitting-a-line-into-array-in-bash-with-tab-as-delimiter?noredirect=1#comment735962_410710&quot;&gt;here&lt;/a&gt; I realized looping through the file is slow and sometimes difficult. How can I download these files using wget with awk or cut as the tool for fixed fields&lt;/p&gt;&#xA;" OwnerUserId="2040" LastEditorUserId="73" LastEditorDisplayName="user2045" LastEditDate="2023-01-15T20:19:12.063" LastActivityDate="2023-01-15T20:19:12.063" Title="Downloading URL using AWK for fixed fields" Tags="|awk|data-download|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3072" PostTypeId="2" ParentId="3071" CreationDate="2017-12-13T19:28:34.387" Score="5" Body="&lt;pre&gt;&lt;code&gt;awk '{print &quot;wget &quot; $9 &quot; -O &quot; $1 &quot;_&quot; $2 &quot;.fastq.gz&quot;}' tmp &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;will give you &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wget https://www.encodeproject.org/files/ENCFF001JPX/@@download/ENCFF001JPX.fastq.gz -O a_k.fastq.gz&#xA;wget https://www.encodeproject.org/files/ENCFF001JPZ/@@download/ENCFF001JPZ.fastq.gz -O b_k.fastq.gz&#xA;wget https://www.encodeproject.org/files/ENCFF001JQB/@@download/ENCFF001JQB.fastq.gz -O c_k.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you add a &lt;code&gt;| bash&lt;/code&gt; to the command it will then execute all the &lt;code&gt;wget&lt;/code&gt; commands.&lt;/p&gt;&#xA;" OwnerUserId="964" LastEditorUserId="298" LastEditDate="2017-12-13T19:46:31.503" LastActivityDate="2017-12-13T19:46:31.503" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3073" PostTypeId="1" AcceptedAnswerId="3074" CreationDate="2017-12-13T20:19:33.790" Score="4" ViewCount="300" Body="&lt;p&gt;I want to use &lt;a href=&quot;https://software.broadinstitute.org/gatk&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATK&lt;/a&gt; to estimate cross-sample contamination for Whole Genome Sequencing data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The specific tool is &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/tooldocs/current/org_broadinstitute_gatk_tools_walkers_cancer_contamination_ContEst.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;ContEst&lt;/a&gt; and it is run with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java&#xA; -jar GenomeAnalysisTK.jar \&#xA; -T ContEst \&#xA; -R reference.fasta \&#xA; -I:eval tumor.bam \&#xA; -I:genotype normal.bam \&#xA; --popfile populationAlleleFrequencies.vcf \&#xA; -L populationSites.interval_list&#xA; [-L targets.interval_list] \&#xA; -isr INTERSECTION \&#xA; -o output.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;--popfile&lt;/code&gt; option requires a vcf file representing population allele frequencies. Where can I get it? Is it the one I can get from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/projects/SNP/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbSNP&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using the human_g1k_v37.fasta reference, a version of hg19, &quot;for the phase1 analysis we mapped to GRCh37. Our fasta file which can be found here called human_g1k_v37.fasta.gz, it contains the autosomes, X, Y and MT but no haplotype sequence or EBV&quot;.&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="96" LastEditDate="2019-03-26T19:33:22.320" LastActivityDate="2019-03-26T19:33:22.320" Title="Where can I get the population allele frequency vcf file?" Tags="|vcf|variant-calling|gatk|quality-control|wgs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3074" PostTypeId="2" ParentId="3073" CreationDate="2017-12-13T20:52:44.337" Score="5" Body="&lt;p&gt;On the &lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/8897/which-one-should-i-use-for-contest-popfile&quot; rel=&quot;noreferrer&quot;&gt;GATK forum&lt;/a&gt; they've recommended &lt;a href=&quot;ftp://gsapubftp-anonymous@ftp.broadinstitute.org/bundle/b37/hapmap_3.3_b37_pop_stratified_af.vcf.gz&quot; rel=&quot;noreferrer&quot;&gt;the population stratified VCF file&lt;/a&gt; for this purpose.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-12-13T20:52:44.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3075" PostTypeId="2" ParentId="3071" CreationDate="2017-12-13T21:53:00.753" Score="4" Body="&lt;p&gt;A variation of &lt;a href=&quot;/a/3072/2042&quot;&gt;Bioathlete’s answer&lt;/a&gt; that doesn't suffer from escaping issues:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;awk 'BEGIN{ FS=&quot;\t&quot; } { printf(&quot;%s\t%s_%s.fastq.gz\n&quot;, $8, $1, $2) }' input |&#xA;while IFS=$'\t' read -r url filename; do&#xA;    wget -O &quot;$filename&quot; -- &quot;$url&quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or entirely with shell parsing:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;while IFS=$'\t' read -r c1 c2 c3 c4 c5 c6 c7 url; do&#xA;    wget -O &quot;${c1}_${c2}.fastq.gz&quot; -- &quot;$url&quot;&#xA;done &amp;lt; input&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or entirely without shell parsing but with GNU &lt;code&gt;xargs&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;awk 'BEGIN{ FS=&quot;\t&quot; } { printf(&quot;%s_%s.fastq.gz\n%s\n&quot;, $1, $2, $8) }' input |&#xA;xargs -r -d '\n' -n 2 -- wget -O&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;… you could even run multiple &lt;code&gt;wget&lt;/code&gt; instances in parallel from GNU &lt;code&gt;xargs&lt;/code&gt; with the &lt;code&gt;-P&lt;/code&gt; option, though you should set the &lt;code&gt;-q&lt;/code&gt; option on &lt;code&gt;wget&lt;/code&gt; to disable progress bars overwriting each other on the same terminal. Then again it would be better to use a real command-line download manager like Aria or Axel for that task.&lt;/p&gt;&#xA;" OwnerUserId="2042" LastEditorUserId="298" LastEditDate="2017-12-13T23:05:17.623" LastActivityDate="2017-12-13T23:05:17.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3076" PostTypeId="1" AcceptedAnswerId="3079" CreationDate="2017-12-13T23:58:07.093" Score="4" ViewCount="227" Body="&lt;p&gt;I need to determine whether a gene is mitochondrial or not for C. Elegans automatically from its name by running a regular expression on a dozen of thousands of gene names. Currently I am having gene symbols like: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clec-190, clec-189, 21ur-8912, nduo-1, 21ur-6539, 21ur-13096, nduo-6, 21ur-14241, ndfl-4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried running &lt;code&gt;grepl(&quot;^mt-&quot;, symbols)&lt;/code&gt; and it found nothing. I am sure that there are some mitochondrial genes just from their descriptions. I would assume that first I need to convert gene symbols to some other symbols to do that but I am not sure here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, I am trying to do it in a similar way to the &lt;a href=&quot;https://www.bioconductor.org/help/workflows/simpleSingleCell/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; of single cell RNAseq. &lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2017-12-14T08:00:05.557" LastActivityDate="2017-12-14T08:00:05.557" Title="Determine if a gene is mitochondrial or not" Tags="|r|rna-seq|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3077" PostTypeId="2" ParentId="3021" CreationDate="2017-12-14T04:00:24.687" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://github.com/biologyguy/BuddySuite/wiki/SB-Pull-records&quot; rel=&quot;nofollow noreferrer&quot;&gt;SeqBuddy --pull_records&lt;/a&gt; command can grab sequences based on a regular expression against record IDs and/or metadata. Given your example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$: seqbuddy.py sequences.fa --pull_records &quot;Clostridium botulinum.*BKT028387&quot; full &amp;gt; Clostridium_botulinum_BKT028387.fas&#xA;$: seqbuddy.py sequences.fa --pull_records &quot;Peptoanaerobacter stomatis.*OBRC8&quot; full &amp;gt; Peptoanaerobacter_stomatis_OBRC8.fas&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2046" LastActivityDate="2017-12-14T04:00:24.687" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3078" PostTypeId="2" ParentId="343" CreationDate="2017-12-14T06:52:14.230" Score="3" Body="&lt;p&gt;+1 for taking issue with RepBase.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use the annotations from &lt;a href=&quot;http://hammelllab.labsites.cshl.edu/software/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Hammell Lab GTFs that they put out with TEtoolkit&lt;/a&gt;. It is similar to what you described to be using so this may be a redundant and useless answer but from the digging I’ve done they do seem to be comprehensive and well curated (for Drosophila, at least). &lt;/p&gt;&#xA;" OwnerUserId="1951" LastActivityDate="2017-12-14T06:52:14.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3079" PostTypeId="2" ParentId="3076" CreationDate="2017-12-14T07:49:24.380" Score="10" Body="&lt;p&gt;You can't determine if a gene is mitochondrial purely from its name. The simplest route would be to use the &lt;a href=&quot;http://parasite.wormbase.org/biomart/martview?VIRTUALSCHEMANAME=parasite_mart&amp;amp;ATTRIBUTES=wbps_gene.default.feature_page.production_name_1010|wbps_gene.default.feature_page.wbps_gene_id|wbps_gene.default.feature_page.external_gene_id|wbps_gene.default.feature_page.chromosome_name&amp;amp;FILTERS=wbps_gene.default.naive_filters.species_id_1010.&amp;quot;elegaprjna13758&amp;quot;&amp;amp;VISIBLEPANEL=resultspanel&quot; rel=&quot;noreferrer&quot;&gt;wormbase biomart&lt;/a&gt; and either download the gene names and their associated chromosomes or use the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/biomaRt.html&quot; rel=&quot;noreferrer&quot;&gt;biomaRt package from bioconductor&lt;/a&gt; to do the same thing. You can upload a list of the genes you're interested in querying in either case (or just download all of them, and filter).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From your list, only &lt;code&gt;nduo-1&lt;/code&gt;, &lt;code&gt;nduo-6&lt;/code&gt;, and &lt;code&gt;ndfl-4&lt;/code&gt; are mitochondrial.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-12-14T07:49:24.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3080" PostTypeId="1" AcceptedAnswerId="3081" CreationDate="2017-12-14T09:41:43.270" Score="6" ViewCount="2412" Body="&lt;p&gt;I have few BAM files which are generated from Ion Torrent Server (ampliseq) aligned to hg19 genome. I want to extract read counts from the bam files and I know that &quot;featureCounts&quot; can be used for this. But before getting the read counts, to view the bam file I gave the following command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -H IonXpress_input.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@HD VN:1.4  SO:coordinate   GO:none&#xA;@SQ SN:NR_039978    LN:984&#xA;@SQ SN:NM_130786    LN:1766&#xA;@SQ SN:NM_138932    LN:9293&#xA;@SQ SN:NM_033110    LN:2678&#xA;@SQ SN:NM_000014    LN:4678&#xA;@SQ SN:NM_144670    LN:5192&#xA;@SQ SN:NR_040112    LN:1201&#xA;@SQ SN:NM_017436    LN:2106&#xA;@SQ SN:NM_016161    LN:1771&#xA;@SQ SN:NM_015665    LN:1854&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I guess these are from header. After these I see something like following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@RG ID:JODXM.IonXpress_022.1    CN:S5TorrentServerVM/MT205524   DT:2017-12-07T12:04:52+0100 FO:TACGTACGTCTGAGCATCGATCGATGTACAGCTACGTACGTCTGAGCATCGATCGATGTACAGCTACGTACGTCTGAGCATCGATCGATGTACAGCTACGTACGTCTGAGCATCGATCGATGTACAGCTACGTACGTCTGAGCATCGATCGATGTACAGCTACGTACGTCTGAGCATCGATCGATGTACAGCTACGTACGTCTGAGCATCGATCGATGTACAGCTACGTACGTCTGAGCATCGATCGATGTACAGCTACGTACGTCTGAGCATCGATCGATGTACAGCTACGTACGTCTGAGCATCGATCGATGTACAGCTACGTACGTCTGAGCATCGATCGATGTACAGCTACGTACGTCTGAGCATCGATCGATGTACAGCTACGTACGTCTGAGCATCGATCGATGTACAGCTACGTACGTCTGAGCATCGATCGATGTACAGCTACGTACGTCTGAGCATCGATCGATGTACAGCTACGTACGTCTGAGCATCGA KS:TCAGTTCGAGACGCGAT    PG:tmap PL:IONTORRENT   PU:s5/540/IonXpress_022 SM:SNU449_TEAD4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Haven't seen a bam file like this before. Any idea what are these and how I can get read counts?&lt;/p&gt;&#xA;" OwnerUserId="1377" LastEditorUserId="1377" LastEditDate="2017-12-14T10:38:13.540" LastActivityDate="2017-12-14T11:21:09.167" Title="Read counts from BAM file" Tags="|rna-seq|sam|samtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3081" PostTypeId="2" ParentId="3080" CreationDate="2017-12-14T10:07:31.273" Score="7" Body="&lt;p&gt;Your data is not aligned to hg19, but to a bunch of RNA ref sequences. If you would align to hg19 you'll get each chromosome instead of the NR_* or NM_* accession codes with your &lt;code&gt;samtools view -H&lt;/code&gt; code. With &lt;code&gt;samtools idxstats file.bam&lt;/code&gt; you'll get the reads per chromosome (or per nucleotide sequence in your case).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For use in &lt;code&gt;featureCounts&lt;/code&gt; my advice is to get the fastq files, and align it yourself to hg19 (or hg38 to be more up to date).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your data is properly aligned to hg19 your output should be something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -H aligned2hg19.bam&#xA;&#xA;@SQ SN:chrM LN:16571&#xA;@SQ SN:chr1 LN:249250621&#xA;@SQ SN:chr2 LN:243199373&#xA;@SQ SN:chr3 LN:198022430&#xA;@SQ SN:chr4 LN:191154276&#xA;@SQ SN:chr5 LN:180915260&#xA;@SQ SN:chr6 LN:171115067&#xA;@SQ SN:chr7 LN:159138663&#xA;@SQ SN:chr8 LN:146364022&#xA;@SQ SN:chr9 LN:141213431&#xA;@SQ SN:chr10    LN:135534747&#xA;@SQ SN:chr11    LN:135006516&#xA;@SQ SN:chr12    LN:133851895&#xA;@SQ SN:chr13    LN:115169878&#xA;@SQ SN:chr14    LN:107349540&#xA;@SQ SN:chr15    LN:102531392&#xA;@SQ SN:chr16    LN:90354753&#xA;@SQ SN:chr17    LN:81195210&#xA;@SQ SN:chr18    LN:78077248&#xA;@SQ SN:chr19    LN:59128983&#xA;@SQ SN:chr20    LN:63025520&#xA;@SQ SN:chr21    LN:48129895&#xA;@SQ SN:chr22    LN:51304566&#xA;@SQ SN:chrX LN:155270560&#xA;@SQ SN:chrY LN:59373566&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2017-12-14T11:21:09.167" LastActivityDate="2017-12-14T11:21:09.167" CommentCount="14" ContentLicense="CC BY-SA 3.0" />
  <row Id="3082" PostTypeId="1" AcceptedAnswerId="3083" CreationDate="2017-12-14T10:58:32.720" Score="4" ViewCount="428" Body="&lt;p&gt;I have downloaded a gene expression data from GEO database (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE3268&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSE3268&lt;/a&gt;) in which in some of its rows there are duplicate gene symbols.&lt;/p&gt;&#xA;&lt;p&gt;For example &lt;strong&gt;TP53&lt;/strong&gt; exists in two rows with different expression values and different &lt;em&gt;GenBank Accession&lt;/em&gt; IDs.&lt;/p&gt;&#xA;&lt;p&gt;They have the following &lt;em&gt;Nucleotide Title&lt;/em&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Human p53 cellular tumor antigen mRNA, complete cds&lt;/p&gt;&#xA;&lt;p&gt;Homo sapiens tumor protein p53 (TP53), transcript variant 1, mRNA&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I want to map these data with edge list of gene symbols to make a network. Which expression value should I consider for such duplicate gene symbols for mapping?&lt;/p&gt;&#xA;" OwnerUserId="1026" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2017-12-14T12:05:04.787" Title="Duplicate gene symbol handling in GEO gene expression data" Tags="|database|gene|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3083" PostTypeId="2" ParentId="3082" CreationDate="2017-12-14T12:05:04.787" Score="5" Body="&lt;p&gt;Since this is a microarray (U133A), what you're seeing is that each probe is associated with one or more transcripts. The general strategy when dealing with microarrays is to use RMA (or fRMA) to summarize to probesets and then &lt;code&gt;collapseRows&lt;/code&gt; to summarize to gene-level information. Since this is all typically done in R, you can find some further discussion &lt;a href=&quot;https://support.bioconductor.org/p/60214/&quot; rel=&quot;noreferrer&quot;&gt;on the bioconductor site&lt;/a&gt;. I would suggest going through this process and then only using the result of that in building your network (whether you're happy with sticking to transcript-level quantifications or gene-level quantifications for building your network is up to you).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-12-14T12:05:04.787" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3084" PostTypeId="1" AcceptedAnswerId="3085" CreationDate="2017-12-14T13:54:43.873" Score="3" ViewCount="3239" Body="&lt;p&gt;I have few bam files and would like to get read counts using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools idxstats&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;[Data is aligned to hg19 transcriptome]. To use that command I need a sorted bam file. So to sort them I gave the following command.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools sort -T /tmp/input.sorted -o input.sorted.bam input.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This ended up showing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[W::bam_hdr_read] EOF marker is absent. The input is probably truncated.&#xA;[E::bgzf_read] bgzf_read_block error -1 after 0 of 4 bytes&#xA;[bam_sort_core] truncated file. Aborting.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the tmp dir has 6 input.sorted.nnnn.bam files.&lt;/p&gt;&#xA;" OwnerUserId="1377" LastEditorUserId="1377" LastEditDate="2017-12-14T14:04:49.437" LastActivityDate="2017-12-14T14:08:33.120" Title="sorting BAM file error using samtools" Tags="|rna-seq|sam|samtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3085" PostTypeId="2" ParentId="3084" CreationDate="2017-12-14T14:01:55.807" Score="4" Body="&lt;p&gt;The cause of the error is that your input file is truncated or /tmp is running out of space. If you can do &lt;code&gt;samtools view -H input.bam&lt;/code&gt; without error (reading the header also checks for the magic number at the end of the file) then it's like the that /tmp is filling up.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-12-14T14:08:33.120" LastActivityDate="2017-12-14T14:08:33.120" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="3086" PostTypeId="1" AcceptedAnswerId="3117" CreationDate="2017-12-14T14:37:11.400" Score="3" ViewCount="251" Body="&lt;p&gt;I am using prokka to annotate a bacterial genome:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;prokka ecoli.fa&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Prokka is outputting a tab-separated file (called PROKKA_12142017.tsv) with differing numbers of elements in each row:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;locus_tag    ftype    gene    EC_number    product&#xA;BOFHCHGE_00001    CDS    hypothetical protein&#xA;BOFHCHGE_00002    CDS    rdgB_1    3.6.1.9    dITP/XTP pyrophosphatase&#xA;BOFHCHGE_00003    CDS    hypothetical protein&#xA;BOFHCHGE_00004    CDS    hypothetical protein&#xA;BOFHCHGE_00005    CDS    hypothetical protein&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does anybody know how I can get round this to import into a dataframe in R?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to ask prokka to add something in the gene column?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2020-10-12T21:29:34.070" Title="Import a tab-separated file with differing numbers of elements in each row; prokka output" Tags="|r|sequence-annotation|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3087" PostTypeId="2" ParentId="3086" CreationDate="2017-12-14T14:44:28.380" Score="6" Body="&lt;p&gt;You don't really need to get around anything, your example can be loaded correctly with &lt;code&gt;read.delim()&lt;/code&gt;. The rows with missing values are filled in with blanks. You may, however, prefer the &lt;code&gt;readr&lt;/code&gt; package, which handles this a bit more elegantly (it'll tell you which values were missing and not do the annoying string-&gt;factor conversion).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-12-14T14:44:28.380" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3088" PostTypeId="2" ParentId="3086" CreationDate="2017-12-14T15:12:32.487" Score="3" Body="&lt;p&gt;As Devon said, &lt;code&gt;read.delim&lt;/code&gt; can deal with this perfectly well as long as the file is properly formatted (so missing fields still end with the field separator, like &lt;code&gt;\t\t&lt;/code&gt;). For example, with this input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Field1  Field2  Field3&#xA;Row1field1  Row1field2  Row1field3&#xA;Row2field1      Row2field3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, to show the fields more clearly:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Field1\tField2\tField3&#xA;Row1field1\tRow1field2\tRow1field3&#xA;Row2field1\t\tRow2field3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;R can import:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; df&amp;lt;-read.delim(&quot;file&quot;,header=TRUE, sep=&quot;\t&quot;)&#xA;&amp;gt; df&#xA;      Field1     Field2     Field3&#xA;1 Row1field1 Row1field2 Row1field3&#xA;2 Row2field1            Row2field3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Alternatively, if you really want to add something to the missing columns, you can use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk -F&quot;\t&quot; '($&lt;/span&gt;3==&quot;&quot;){$3=&quot;none&quot; }1;' file&#xA;locus_tag   ftype   gene    EC_number   product&#xA;BOFHCHGE_00001  CDS hypothetical    protein&#xA;BOFHCHGE_00002  CDS rdgB_1  3.6.1.9 dITP/XTP    pyrophosphatase&#xA;BOFHCHGE_00003  CDS hypothetical    protein&#xA;BOFHCHGE_00004  CDS hypothetical    protein&#xA;BOFHCHGE_00005  CDS hypothetical    protein&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, to add &lt;code&gt;none&lt;/code&gt; to any empty field, not just the 3rd one:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -F&quot;\t&quot; '{for(i=1;i&amp;lt;=NF;i++){if(&lt;span class=&quot;math-container&quot;&gt;$i==&quot;&quot;){$&lt;/span&gt;i=&quot;none&quot;; }}}1;' file &amp;gt; newfile&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-01-13T15:03:33.560" LastActivityDate="2019-01-13T15:03:33.560" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="3089" PostTypeId="1" AcceptedAnswerId="3110" CreationDate="2017-12-14T16:40:01.093" Score="5" ViewCount="3104" Body="&lt;p&gt;I have some sequences shown below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CAGGTAGCC&#xA;CCGGTCAGA&#xA;AGGGTTTGA&#xA;TTGGTGAGG&#xA;CAAGTATGA&#xA;ACTGTATGC&#xA;CTGGTAACC&#xA;TATGTACTG&#xA;GCTGTGAGA&#xA;CAGGTGGGC&#xA;TCAGTGAGA&#xA;GGGGTGAGT&#xA;TGGGTATGT&#xA;GAGGTGAGA&#xA;CAGGTGGAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Each line has 9 nucleotides.&#xA;Consider it to be 9 columns.I want to calculate the nucleotide frequency of each nucleotide for each of the 9 columns.&#xA;For example 1st column will have these bases C,C,A,T,C,A etc&#xA;Out put should be something like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A   0.25    0.34    0.56    0.43    0.00    0.90    0.45    0.34    0.31&#xA;C   0.45    0.40    0.90    0.00    0.40    0.90    0.30    0.25    0.2&#xA;G   0.00    0.00    0.34    1.00    0.30    0.30    0.35    0.90    0.1&#xA;T   0.24    0.56    0.00    0.00    1.00    0.34    0.45    0.35    0.36&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;, I just made up the numbers to show you the output file format&lt;/p&gt;&#xA;" OwnerUserId="2040" LastEditorUserId="3848" LastEditDate="2020-07-22T15:55:19.083" LastActivityDate="2020-07-22T15:55:19.083" Title="calculating nucleotide frequency per column" Tags="|sequence-analysis|nucleotide-models|" AnswerCount="7" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3090" PostTypeId="2" ParentId="3089" CreationDate="2017-12-14T21:01:48.197" Score="4" Body="&lt;p&gt;Here is one example of how to do this with a bit of python. Alternatively one could create strings of each column and using &lt;code&gt;letterFrequency()&lt;/code&gt; from the &lt;code&gt;Biostrings&lt;/code&gt; package.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;#Make a list of hashes&#xA;hl = []&#xA;for i in range(9):&#xA;    hl.append({'A': 0, 'C': 0, 'G': 0, 'T': 0})&#xA;&#xA;f = open(&quot;foo.txt&quot;)  # CHANGE ME&#xA;nLines = 0&#xA;for line in f:&#xA;    for idx, c in enumerate(line.strip()):&#xA;        hl[idx][c] += 1&#xA;    nLines += 1&#xA;f.close()&#xA;&#xA;nLines = float(nLines)&#xA;for char in ['A', 'C', 'G', 'T']:&#xA;    print(&quot;{}\t{}&quot;.format(char, &quot;\t&quot;.join([&quot;{:0.2f}&quot;.format(x[char]/nLines) for x in hl])))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output of your example is then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A   0.13    0.40    0.13    0.00    0.00    0.40    0.47    0.07    0.40&#xA;C   0.40    0.27    0.00    0.00    0.00    0.07    0.07    0.13    0.27&#xA;G   0.20    0.20    0.67    1.00    0.00    0.47    0.20    0.73    0.20&#xA;T   0.27    0.13    0.20    0.00    1.00    0.07    0.27    0.07    0.13&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastEditorUserId="298" LastEditDate="2017-12-15T10:22:28.050" LastActivityDate="2017-12-15T10:22:28.050" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3091" PostTypeId="2" ParentId="3062" CreationDate="2017-12-14T21:33:27.973" Score="1" Body="&lt;p&gt;It seems that &lt;code&gt;tsne$Y&lt;/code&gt; is the matrix and the names are just taken from there. So, you could replace the names of the columns of that matrix and then it would display the names instead of the numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, you are using the function &lt;code&gt;plot3d&lt;/code&gt; from the rgl package, which has as arguments &lt;code&gt;xlab&lt;/code&gt;, &lt;code&gt;ylab&lt;/code&gt;, and &lt;code&gt;zlab&lt;/code&gt;, so you just need to give a string to those arguments to name the axes.&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="77" LastEditDate="2017-12-14T21:36:11.533" LastActivityDate="2017-12-14T21:36:11.533" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3092" PostTypeId="2" ParentId="3061" CreationDate="2017-12-14T22:39:22.913" Score="1" Body="&lt;p&gt;The issue was caused by the corrupted input &lt;code&gt;.fastq&lt;/code&gt; files that were damaged somehow upon uploading them from the local machine. We figured it out by &lt;code&gt;md5sum&lt;/code&gt; command output comparison. We have not yet managed to actually run all of the script still, it is failing, but for another reason now. &lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2017-12-14T22:39:22.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3093" PostTypeId="1" AcceptedAnswerId="3095" CreationDate="2017-12-14T22:51:34.120" Score="2" ViewCount="81" Body="&lt;p&gt;I am running a script file on a cluster with &lt;code&gt;slurm&lt;/code&gt; through &lt;code&gt;sbatch&lt;/code&gt;. Locally it runs perfectly well on 2 different linux versions (&lt;code&gt;linux mint&lt;/code&gt;, &lt;code&gt;ubuntu 12.04&lt;/code&gt;), however, it returns &lt;code&gt;-9&lt;/code&gt; error code while running on a cluster which is &lt;code&gt;CentOS 7.3&lt;/code&gt;. When we are using &lt;code&gt;Pypiper&lt;/code&gt; and run the whole &lt;code&gt;.py&lt;/code&gt; script file, it fails when it starts running &lt;code&gt;trimmomatic&lt;/code&gt; program&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.usadellab.org/cms/?page=trimmomatic&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.usadellab.org/cms/?page=trimmomatic&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when running just &lt;code&gt;trimmomatic&lt;/code&gt; program step by itself it does not fail and finishes successfully. We have a suspicion that something may be wrong with how &lt;code&gt;Pypiper&lt;/code&gt; runs on the cluster, however, feeling rather stuck.&#xA;Any suggestions would be greatly appreciated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the error that is printed out:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ILLUMINACLIP: Using 2 prefix pairs, 22 forward/reverse sequences, 0 &#xA;forward only sequences, 0 reverse only sequences&#xA;&amp;lt;/pre&amp;gt;&#xA;Process 109391 returned: (-9). Elapsed: 0:00:51. Peak memory: (Process: &#xA;5.211GB; Pipeline: 0.0GB)&#xA;&#xA;### Pipeline failed at:  (12-14 15:50:47) elapsed:0:23:09 _TIME_&#xA;&#xA;Total time: 0:23:09&#xA;&#xA;Change status from running to failed&#xA;Traceback (most recent call last):&#xA;  File &quot;...../salmon_rna_seq.py&quot;, line 344, in &amp;lt;module&amp;gt;&#xA;sys.exit(main())&#xA;File &quot;...../salmon_rna_seq.py&quot;, line 304, in main&#xA;pipeline.process()      &#xA;File &quot;...../salmon_rna_seq.py&quot;, line 277, in process&#xA;self.trim_reads()      &#xA;File &quot;...../salmon_rna_seq.py&quot;, line 195, in trim_reads&#xA;self.pipe_manager.run(cmd, self.sample_files['r1']['fastq']['trimmed_paired'])&#xA;File &quot;...../pypiper.py&quot;, line 508, in run&#xA;process_return_code, local_maxmem = self.callprint(cmd, shell, nofail, container)  # Run command&#xA;File &quot;...../pypiper.py&quot;, line 691, in callprint&#xA;self.fail_pipeline(e)&#xA;File &quot;...../pypiper.py&quot;, line 1111, in fail_pipeline&#xA;raise e&#xA;OSError: Subprocess returned nonzero result.&#xA;&#xA;Pypiper terminating spawned child process 108475...&#xA;child process terminated&#xA;slurmstepd: error: Exceeded step memory limit at some point.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The last line may be informative. Although, we are still not sure what we could possibly do.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="2005" LastEditDate="2017-12-14T23:12:28.340" LastActivityDate="2017-12-14T23:17:52.967" Title="Pipeline (Pypiper) returns error, but individual command works fine" Tags="|rna-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3094" PostTypeId="1" AcceptedAnswerId="3096" CreationDate="2017-12-14T23:09:09.643" Score="4" ViewCount="15896" Body="&lt;p&gt;I have a tab separated text file as shown below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a   0.311   0.510   0.123   0.002   0.001   0.417   0.572   0.074   0.169&#xA;b   0.324   0.592   0.070   0.028   0.028   0.535   0.535   0.127   0.113&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am trying to use heatmap.2 function from gplots package and want to change the color key so that it ranges from 0 to 1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;d &amp;lt;- read.table(&quot;file.txt&quot;,sep=&quot;\t&quot;,header=F,row.names = 1)     &#xA;colnames(d) &amp;lt;- c(&quot;-3&quot;,&quot;-2&quot;,&quot;-1&quot;,&quot;+1&quot;,&quot;+2&quot;,&quot;+3&quot;,&quot;+4&quot;,&quot;+5&quot;,&quot;+6&quot;)&#xA;d_matrix &amp;lt;- data.matrix(d)&#xA;colors&amp;lt;-seq(0,1,length.out=100)&#xA;heatmap.2(d_matrix,scale=&quot;none&quot;,col = redblue(75),trace=&quot;none&quot;,Colv = FALSE,Rowv = FALSE,breaks=colors)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But this gives me error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in image.default(1:nc, 1:nr, x, xlim = 0.5 + c(0, nc), ylim = 0.5 +  : &#xA;  must have one more break than colour&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to change to color key such that it ranges from 0 to 1. How can I do it in the function heatmap.2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I added breaks argument to the heatmap.2 function as suggested from this post &lt;a href=&quot;https://stackoverflow.com/questions/17820143/how-to-change-heatmap-2-color-range-in-r&quot;&gt;here&lt;/a&gt; but no luck..&lt;/p&gt;&#xA;" OwnerUserId="2040" LastEditorUserId="298" LastEditDate="2017-12-14T23:27:36.247" LastActivityDate="2017-12-14T23:44:45.513" Title="changing color key range to specified range in heatmap.2 function" Tags="|r|bioconductor|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3095" PostTypeId="2" ParentId="3093" CreationDate="2017-12-14T23:17:52.967" Score="3" Body="&lt;p&gt;Slurm is killing the process because it's using more memory per core than the default allocation (that's what &lt;code&gt;slurmstepd: error: Exceeded step memory limit at some point&lt;/code&gt; means). I've never used pyPiper (like seemingly most people, I use snakemake), but presuming that you have it running either &lt;code&gt;srun&lt;/code&gt; or &lt;code&gt;sbatch&lt;/code&gt; at some point then specify something like &lt;code&gt;--mem-per-cpu 10G&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-12-14T23:17:52.967" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3096" PostTypeId="2" ParentId="3094" CreationDate="2017-12-14T23:44:45.513" Score="4" Body="&lt;p&gt;You just need to add more colors. The following, for example, will work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;heatmap.2(d_matrix, scale=&quot;none&quot;, trace=&quot;none&quot;, Rowv=F, Colv=F, breaks=seq(0,1,0.01), dendrogram=&quot;none&quot;, col=redblue(100))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;code&gt;length(seq(0, 1, 0.01))&lt;/code&gt; is 101, therefore &lt;code&gt;redblue()&lt;/code&gt; must return one fewer (100) colors (or colours if you prefer extra vowels).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-12-14T23:44:45.513" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3097" PostTypeId="1" CreationDate="2017-12-14T05:22:32.143" Score="5" ViewCount="73" Body="&lt;p&gt;I am working on Vibrio parahaemolyticus. I have a gene that it is possible might be regulated by small non-coding RNAs. How can I predict possible sRNAs that target the transcript of this gene. Most software offers the opposite of what I want, i.e., they predict targets for small RNAs!&lt;/p&gt;&#xA;&lt;p&gt;I'm trying to predict bacterial small non-coding RNA in general (Support Vector Machine) but I got more than 8 thousand sequences within intergenic regions alone. Is there a code for aligning these sequences to my transcript?&lt;/p&gt;&#xA;" OwnerUserId="2059" OwnerDisplayName="Mohammad ALKADI" LastEditorUserId="73" LastEditDate="2021-05-12T02:19:38.907" LastActivityDate="2021-10-09T05:07:05.643" Title="how do I predict bacterial small non-coding RNA for a specific mRNA?" Tags="|bacteria|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3099" PostTypeId="2" ParentId="3089" CreationDate="2017-12-15T09:07:09.480" Score="2" Body="&lt;p&gt;Here an example in R using &lt;code&gt;Biostrings&lt;/code&gt; and &lt;code&gt;letterFrequency&lt;/code&gt; (as suggested by &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/77/devon-ryan&quot;&gt;Devon Ryan&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(Biostrings)&#xA;&#xA;data &amp;lt;- read.table(&quot;DNA.txt&quot;, stringsAsFactors = F)&#xA;&#xA;new &amp;lt;- matrix(nrow = 9, ncol = 15)&#xA;&#xA;for(i in 1:9){&#xA;  for(j in 1:15){&#xA;    new[i,j] &amp;lt;- substring(data[j,], i, i)&#xA;  }&#xA;}&#xA;&#xA;countTable &amp;lt;- matrix(nrow = 9, ncol = 4)&#xA;for(i in 1:9){&#xA;  columnSeq &amp;lt;- DNAStringSet(paste0(new[i,], collapse = &quot;&quot;))&#xA;  columnCounts &amp;lt;- letterFrequency(columnSeq, letters = &quot;ACGT&quot;, OR = 0)&#xA;  countTable[i,] &amp;lt;- columnCounts&#xA;}&#xA;colnames(countTable) &amp;lt;- c(&quot;A&quot;, &quot;C&quot;, &quot;G&quot;, &quot;T&quot;)&#xA;&#xA;freqTable &amp;lt;- countTable/15&#xA;round(t(freqTable), digit = 2)&#xA;  [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9]&#xA;A 0.13 0.40 0.13    0    0 0.40 0.47 0.07 0.40&#xA;C 0.40 0.27 0.00    0    0 0.07 0.07 0.13 0.27&#xA;G 0.20 0.20 0.67    1    0 0.47 0.20 0.73 0.20&#xA;T 0.27 0.13 0.20    0    1 0.07 0.27 0.07 0.13&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2017-12-15T19:59:40.387" LastActivityDate="2017-12-15T19:59:40.387" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3100" PostTypeId="2" ParentId="2788" CreationDate="2017-12-15T09:10:32.840" Score="0" Body="&lt;p&gt;Here is an implementation in pure python. Probably not the fastest, but you should not encounter any memory issues with this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: this is python3, use as follows: &lt;code&gt;cat snp_counts.tsv | python allele_frequencies.py&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;&quot;&quot;&quot;&#xA;allele_frequencies.py&#xA;&quot;&quot;&quot;&#xA;import sys&#xA;&#xA;is_odd = True&#xA;for line in sys.stdin:&#xA;    parts = line.strip().split()&#xA;    if not parts:&#xA;        continue&#xA;    allele_counts = [ int(p) for p in parts ]&#xA;    if is_odd:&#xA;        allele1 = allele_counts&#xA;        is_odd = False&#xA;    else:&#xA;        allele2 = allele_counts&#xA;        allele_sum = [ sum([a1,a2]) for a1,a2 in zip(allele1,allele2) ]&#xA;        allele1_freq = [ a / tot if tot else 0.0 for a,tot in zip(allele1, allele_sum) ]&#xA;        allele2_freq = [ a / tot if tot else 0.0 for a,tot in zip(allele2, allele_sum) ]&#xA;        print(*allele1_freq, sep = '\t')&#xA;        print(*allele2_freq, sep = '\t')&#xA;        is_odd = True&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="196" LastActivityDate="2017-12-15T09:10:32.840" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3102" PostTypeId="2" ParentId="3089" CreationDate="2017-12-15T10:01:03.897" Score="3" Body="&lt;p&gt;Here's a Perl approach:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env perl&#xA;use strict;&#xA;my %counts;&#xA;&#xA;## Read the input file line by line&#xA;while (my $line = &amp;lt;&amp;gt;) {&#xA;  print;&#xA;  ## remove trailing '\n' characters&#xA;  chomp $line;&#xA;  ## split the line into an array at every character&#xA;  my @columns=split(/./, $line); &#xA;  ## iterate over the array from the first to the last position&#xA;  for my $i (0..$#columns){&#xA;    ## The nucleotide at this position&#xA;    my $nt = $columns[$i];    &#xA;    ## Save the count in this hash of arrays. The keys are the&#xA;    ## nucleotides, and the value at each position is increased&#xA;    ## when that nucleotide is found in that position.&#xA;    $counts{$nt}[$i]++;&#xA;  }&#xA;}&#xA;## Iterate over the counts hash&#xA;for my $nt (sort keys(%counts)){&#xA;  print &quot;$nt\t&quot;;&#xA;  ## dereference the array stored in the hash&#xA;  my @countsForThisNt = @{$counts{$nt}};&#xA;  ## iterate over the counts for each position for this nt&#xA;  for (my $l=0;$l&amp;lt;=$#countsForThisNt; $l++) {#&#xA;    ## If the value for this position isn't defined,&#xA;    ## set it to 0.&#xA;    $countsForThisNt[$l]||=0;&#xA;    ## Print all the things&#xA;    printf &quot;%.2f\t&quot;, $countsForThisNt[$l]/$., $l;&#xA;  }&#xA;  print &quot;\n&quot;;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Save the script somewhere in your PATH, make it executable and run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ foo.pl file  &#xA;A   0.13    0.40    0.13    0.00    0.00    0.40    0.47    0.07    0.40    &#xA;C   0.40    0.27    0.00    0.00    0.00    0.07    0.07    0.13    0.27    &#xA;G   0.20    0.20    0.67    1.00    0.00    0.47    0.20    0.73    0.20    &#xA;T   0.27    0.13    0.20    0.00    1.00    0.07    0.27    0.07    0.13    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Alternatively, &lt;a href=&quot;https://www.youtube.com/watch?v=Be7Og9Gc_KY&amp;amp;feature=youtu.be&amp;amp;t=18s&quot; rel=&quot;nofollow noreferrer&quot;&gt;if you're into the whole brevity thing&lt;/a&gt;, and enjoy some golfing, here's the same thing as a one-liner:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; perl -ne 'chomp;@F=split(//);for$i(0..$#F){$k{$F[$i]}[$i]++}}{for $nt(sort keys(%k)){print&quot;$nt\t&quot;;for$i(0..$#{$k{$nt}}){$g=$k{$nt}[$i]||0; printf&quot;%.2f\t&quot;,$g/$.}print&quot;\n&quot;;}' file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-12-15T10:12:18.873" LastActivityDate="2017-12-15T10:12:18.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3103" PostTypeId="2" ParentId="3089" CreationDate="2017-12-15T11:21:09.063" Score="5" Body="&lt;p&gt;awk&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{L=length($1);for(i=1;i&amp;lt;=L;i++) {B=substr($1,i,1);T[i][B]++;}} END{for(BI=0;BI&amp;lt;4;BI++) {B=(BI==0?&quot;A&quot;:(BI==1?&quot;C&quot;:(BI==2?&quot;G&quot;:&quot;T&quot;)));printf(&quot;%s&quot;,B); for(i in T) {tot=0.0;for(B2 in T[i]){tot+=T[i][B2];}printf(&quot;\t%0.2f&quot;,(T[i][B]/tot));} printf(&quot;\n&quot;);}}' input.txt&#xA;&#xA;A   0.13    0.40    0.13    0.00    0.00    0.40    0.47    0.07    0.40&#xA;C   0.40    0.27    0.00    0.00    0.00    0.07    0.07    0.13    0.27&#xA;G   0.20    0.20    0.67    1.00    0.00    0.47    0.20    0.73    0.20&#xA;T   0.27    0.13    0.20    0.00    1.00    0.07    0.27    0.07    0.13&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, expanded for clarity:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{&#xA;        L=length($1);&#xA;        for(i=1;i&amp;lt;=L;i++) {&#xA;            B=substr($1,i,1);&#xA;            T[i][B]++;&#xA;        }&#xA;      } &#xA;      END{&#xA;            for(BI=0;BI&amp;lt;4;BI++) {&#xA;                B=(BI==0?&quot;A&quot;:(BI==1?&quot;C&quot;:(BI==2?&quot;G&quot;:&quot;T&quot;)));&#xA;                printf(&quot;%s&quot;,B); &#xA;                for(i in T) {&#xA;                    tot=0.0;&#xA;                    for(B2 in T[i]){&#xA;                        tot+=T[i][B2];&#xA;                    }&#xA;                printf(&quot;\t%0.2f&quot;,(T[i][B]/tot));&#xA;                } &#xA;            printf(&quot;\n&quot;);&#xA;            }&#xA;}' input.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastEditorUserId="298" LastEditDate="2017-12-16T16:34:41.507" LastActivityDate="2017-12-16T16:34:41.507" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3106" PostTypeId="2" ParentId="3086" CreationDate="2017-12-15T15:25:39.170" Score="1" Body="&lt;p&gt;Since the missing values don't have field separators, the easiest option is to rearrange the columns so that the ones with missing values are at the end. They will then be the ones that are filled with missing values when you import them into R. You can use use &lt;code&gt;fill=TRUE&lt;/code&gt; with read.table() or use &lt;a href=&quot;https://cran.r-project.org/web/packages/readr/vignettes/readr.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;readr&lt;/a&gt;, which will complain about missing columns, but fill them in with missing values automatically:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -F'\t' -v OFS='\t' '{print($1,$2,$5,$3,$4)}' file.txt |perl -pe 's/\t+/\t/g' &amp;gt;newfile.txt &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With readr:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;read_tsv(&quot;newfile.txt&quot;)&#xA;&#xA;# A tibble: 5 x 5&#xA;       locus_tag ftype                  product   gene EC_number&#xA;           &amp;lt;chr&amp;gt; &amp;lt;chr&amp;gt;                    &amp;lt;chr&amp;gt;  &amp;lt;chr&amp;gt;     &amp;lt;chr&amp;gt;&#xA;1 BOFHCHGE_00001   CDS     hypothetical protein   &amp;lt;NA&amp;gt;      &amp;lt;NA&amp;gt;&#xA;2 BOFHCHGE_00002   CDS dITP/XTP pyrophosphatase rdgB_1   3.6.1.9&#xA;3 BOFHCHGE_00003   CDS     hypothetical protein   &amp;lt;NA&amp;gt;      &amp;lt;NA&amp;gt;&#xA;4 BOFHCHGE_00004   CDS     hypothetical protein   &amp;lt;NA&amp;gt;      &amp;lt;NA&amp;gt;&#xA;5 BOFHCHGE_00005   CDS     hypothetical protein   &amp;lt;NA&amp;gt;      &amp;lt;NA&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The perl regex is necessary because awk adds tabs after the missing values, which defeats the point of reordering the columns&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2017-12-15T15:25:39.170" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3108" PostTypeId="1" AcceptedAnswerId="3109" CreationDate="2017-12-15T19:19:44.500" Score="4" ViewCount="4250" Body="&lt;p&gt;Let's say I have a pandas dataframe with fields &lt;code&gt;CHROM&lt;/code&gt;, &lt;code&gt;POS&lt;/code&gt;, &lt;code&gt;ALT&lt;/code&gt;, &lt;code&gt;REF&lt;/code&gt;. In this special case, I also wouldn't care about &lt;code&gt;ID&lt;/code&gt;or &lt;code&gt;FILTER&lt;/code&gt;, &lt;code&gt;INFO&lt;/code&gt; could be blank (or meaningless) and we'll write &lt;code&gt;QUAL&lt;/code&gt; as each 40. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;example_dict = {'CHROM':[20, 20, 20, 20, 20], 'POS':[14370, 17330, 1110696, 1230237, 1234567], 'REF':['G', 'T', 'A', 'T', 'GTCT'], 'ALT':['A', 'A', 'G', 'C', 'G']}&#xA;&#xA;df = pd.DataFrame(example_dict)&#xA;print(df)&#xA;&#xA;   CHROM      POS   REF ALT&#xA;0     20    14370     G   A&#xA;1     20    17330     T   A&#xA;2     20  1110696     A   G&#xA;3     20  1230237     T   C&#xA;4     20  1234567  GTCT   G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this case, saving the output with the extension &lt;code&gt;.vcf&lt;/code&gt; via &lt;code&gt;df.to_csv(&quot;myfile.vcf&quot;)&lt;/code&gt; with the other required fields would created the tab-delimited format of the VCF, but it wouldn't include the mandatory header. &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.1.pdf&quot; rel=&quot;noreferrer&quot;&gt;https://samtools.github.io/hts-specs/VCFv4.1.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.1&#xA;##fileDate=20090805&#xA;##source=myImputationProgramV3.1&#xA;##reference=file:///seq/references/&#xA;...&#xA;#CHROM POS ID REF ALT QUAL FILTER INFO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The most popular package to work with VCF's appears to be &lt;code&gt;PyVCF&lt;/code&gt;: &lt;a href=&quot;https://pypi.python.org/pypi/PyVCF&quot; rel=&quot;noreferrer&quot;&gt;https://pypi.python.org/pypi/PyVCF&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, one needs to begin with a VCF as a template, and add to this (from the documentation):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;vcf_reader = vcf.Reader(filename='vcf/test/tb.vcf.gz')&#xA;vcf_writer = vcf.Writer(open('/dev/null', 'w'), vcf_reader)&#xA;for record in vcf_reader:&#xA;     vcf_writer.write_record(record)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It appears the only information necessary from &lt;code&gt;vcf.Reader(filename='vcf/test/tb.vcf.gz')&lt;/code&gt; is the metadata? &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print(vcf_reader.metadata)&#xA;OrderedDict([('fileformat', 'VCFv4.0'), ('fileDate', '20090805'), ('source', ['myImputationProgramV3.1']), ('reference', '1000GenomesPilot-NCBI36'), ('phasing', ['partial'])])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Otherwise, &lt;code&gt;tb.vcf.gz&lt;/code&gt; already has variants, which I think means I would need to feed in a VCF with no variants?:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for record in vcf_reader:&#xA;     print(record)&#xA;&#xA;Record(CHROM=20, POS=14370, REF=G, ALT=[A])&#xA;Record(CHROM=20, POS=17330, REF=T, ALT=[A])&#xA;Record(CHROM=20, POS=1110696, REF=A, ALT=[G, T])&#xA;Record(CHROM=20, POS=1230237, REF=T, ALT=[None])&#xA;Record(CHROM=20, POS=1234567, REF=GTCT, ALT=[G, GTACT])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="146" LastEditorUserId="146" LastEditDate="2017-12-15T19:26:45.227" LastActivityDate="2017-12-15T19:43:38.043" Title="Using Python, how to convert a pandas DataFrame into a VCF?" Tags="|python|vcf|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3109" PostTypeId="2" ParentId="3108" CreationDate="2017-12-15T19:43:38.043" Score="8" Body="&lt;p&gt;You will have to define the header either from an existing VCF file or hardcoded into you python script.  Then write the header the the output VCF file then write the dataframe to the same file with the mode options set to 'a' to append to the end of the file.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will need to order the dataframe columns to match the VCF spec and insert the empty/ default &lt;code&gt;ID&lt;/code&gt;, &lt;code&gt;FILTER&lt;/code&gt;, &lt;code&gt;INFO&lt;/code&gt; and &lt;code&gt;QUAL&lt;/code&gt; columns&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd &#xA;&#xA;example_dict = {'CHROM':[20, 20, 20, 20, 20], 'POS':[14370, 17330, 1110696, 1230237, 1234567], 'REF':['G', 'T', 'A', 'T', 'GTCT'], 'ALT':['A', 'A', 'G', 'C', 'G']}&#xA;&#xA;df = pd.DataFrame(example_dict)&#xA;&#xA;df['ID'] = &quot;&quot;&#xA;df['QUAL'] = 40&#xA;df['FILTER'] = &quot;&quot;&#xA;df['INFO'] = &quot;&quot;&#xA;&#xA;df = df[['CHROM', 'POS', 'ID', 'REF', 'ALT', 'QUAL', 'FILTER', 'INFO']] &#xA;&#xA;header = &quot;&quot;&quot;##fileformat=VCFv4.1&#xA;##fileDate=20090805&#xA;##source=myImputationProgramV3.1&#xA;##reference=file:///seq/references/&#xA;#CHROM POS ID REF ALT QUAL FILTER INFO&#xA;&quot;&quot;&quot;&#xA;&#xA;output_VCF = &quot;myfile.vcf&quot;&#xA;with open(output_VCF, 'w') as vcf:&#xA;    vcf.write(header)&#xA;&#xA;df.to_csv(output_VCF, sep=&quot;\t&quot;, mode='a', index=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then &lt;code&gt;myfile.vcf&lt;/code&gt; looks like &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.1&#xA;##fileDate=20090805&#xA;##source=myImputationProgramV3.1&#xA;##reference=file:///seq/references/&#xA;#CHROM POS ID REF ALT QUAL FILTER INFO&#xA;CHROM   POS ID  REF ALT QUAL    FILTER  INFO&#xA;20  14370       G   A   40      &#xA;20  17330       T   A   40      &#xA;20  1110696     A   G   40      &#xA;20  1230237     T   C   40      &#xA;20  1234567     GTCT    G   40&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="964" LastActivityDate="2017-12-15T19:43:38.043" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3110" PostTypeId="2" ParentId="3089" CreationDate="2017-12-16T17:08:29.763" Score="3" Body="&lt;p&gt;For anyone interested in doing this from any sort of alignment file, I've implemented a &lt;a href=&quot;https://github.com/biologyguy/BuddySuite/wiki/AB-Position-Frequency-Matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;position frequency matrix function in AlignBuddy&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 15 9&#xA;seq_1      CAGGTAGCC&#xA;seq_2      CCGGTCAGA&#xA;seq_3      AGGGTTTGA&#xA;seq_4      TTGGTGAGG&#xA;seq_5      CAAGTATGA&#xA;seq_6      ACTGTATGC&#xA;seq_7      CTGGTAACC&#xA;seq_8      TATGTACTG&#xA;seq_9      GCTGTGAGA&#xA;seq_10     CAGGTGGGC&#xA;seq_11     TCAGTGAGA&#xA;seq_12     GGGGTGAGT&#xA;seq_13     TGGGTATGT&#xA;seq_14     GAGGTGAGA&#xA;seq_15     CAGGTGGAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;:$ alignbuddy input.phy --pos_freq_mat&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;### Alignment 1 ###&#xA;A       0.133   0.400   0.133   0.000   0.000   0.400   0.467   0.067   0.400&#xA;C       0.400   0.267   0.000   0.000   0.000   0.067   0.067   0.133   0.267&#xA;G       0.200   0.200   0.667   1.000   0.000   0.467   0.200   0.733   0.200&#xA;T       0.267   0.133   0.200   0.000   1.000   0.067   0.267   0.067   0.133&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2046" LastEditorUserId="2046" LastEditDate="2017-12-21T15:15:59.897" LastActivityDate="2017-12-21T15:15:59.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3111" PostTypeId="1" AcceptedAnswerId="3112" CreationDate="2017-12-16T18:44:45.307" Score="2" ViewCount="378" Body="&lt;p&gt;I'm curious how a PCA for population structure is built from an alignment/BAM or SAM file. How do programs like &lt;a href=&quot;http://www.popgen.dk/angsd/index.php/ANGSD&quot; rel=&quot;nofollow noreferrer&quot;&gt;angsd&lt;/a&gt; and &lt;a href=&quot;https://github.com/mfumagalli/ngsPopGen&quot; rel=&quot;nofollow noreferrer&quot;&gt;ngsCovar&lt;/a&gt; actually build a PCA? What exactly makes things come out as different and how can this be determined? If I have 3 alignments that look like this:&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Locus1:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Indiv1&#xA;ACGT&#xA;&amp;gt;Indiv2&#xA;ACGC&#xA;&amp;gt;Indiv3&#xA;ACGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Locus2:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Indiv1&#xA;GACT&#xA;&amp;gt;Indiv2&#xA;GACA&#xA;&amp;gt;Indiv3&#xA;GAGT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Locus3:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Indiv1&#xA;GGTC&#xA;&amp;gt;Indiv2&#xA;GGCT&#xA;&amp;gt;Indiv3&#xA;GCTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;(These example alignments leave out quality scores for simplicity, assume that each has high quality across the board.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What algorithm or system would be used to graph them on a PCA? How many dimensions would that PCA have? Is it possible to model this by hand so that I can understand what's going on/how it is built?&lt;/p&gt;&#xA;" OwnerUserId="2063" LastEditorUserId="2063" LastEditDate="2017-12-17T20:20:16.670" LastActivityDate="2017-12-17T20:20:16.670" Title="How is a principal component analysis for population structure made from a bam or other alignment file?" Tags="|sam|sequence-alignment|algorithms|phylogenetics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3112" PostTypeId="2" ParentId="3111" CreationDate="2017-12-16T21:01:02.127" Score="4" Body="&lt;p&gt;In general, the process works as such:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Variants are found at a number of loci in each sample (missing loci are often imputed).&lt;/li&gt;&#xA;&lt;li&gt;Each sample is scored at each locus, often according to the number of copies of the reference allele that it has (so, 0-2).&lt;/li&gt;&#xA;&lt;li&gt;The resulting matrix is used for PCA.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="77" LastEditorUserId="298" LastEditDate="2017-12-17T14:55:48.450" LastActivityDate="2017-12-17T14:55:48.450" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3114" PostTypeId="1" AcceptedAnswerId="3128" CreationDate="2017-12-17T20:27:58.083" Score="6" ViewCount="3282" Body="&lt;p&gt;I have a list of dbSNP rsIDs for GRCh37 and I want to convert them to the equivalent IDs in GRCh38. This is using the most recent dbSNP build (150 as of the time of this post). Is there any ID mapping available? If not, are there any tools I can use?&lt;/p&gt;&#xA;" OwnerUserId="2027" LastEditorUserId="77" LastEditDate="2017-12-19T10:56:29.650" LastActivityDate="2020-08-22T17:27:17.867" Title="Convert rs ID of one hg build to rs IDs of another build" Tags="|snp|plink|phylogenetics|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3115" PostTypeId="1" CreationDate="2017-12-17T21:59:20.680" Score="2" ViewCount="483" Body="&lt;p&gt;I am new to bioinformatics and PCA. What I am trying to do is to remove bad cells from a dataset that was obtained with &lt;code&gt;scRNA-seq&lt;/code&gt; for &lt;code&gt;46 000&lt;/code&gt; of genes (rows) vs &lt;code&gt;68 000&lt;/code&gt; of cells that are of different types of &lt;code&gt;C. Elegans&lt;/code&gt; that was generated in the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/28818938&quot; rel=&quot;nofollow noreferrer&quot;&gt;following paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After running &lt;code&gt;PCA&lt;/code&gt;: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fontsize &amp;lt;- theme(axis.text=element_text(size=12), &#xA;axis.title=element_text(size=16))&#xA;plotPCA(sce, pca_data_input=&quot;pdata&quot;) + fontsize&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I see the following chart:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vRRkE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vRRkE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose that the cells that are going to the left bottom corner are the bad ones since they do not fall into some restricted region. Are there any &lt;code&gt;R&lt;/code&gt; program that can automatically remove the bad cells leaving me with just the majority of cells clustered within an ellipse? I know there is &lt;code&gt;identify&lt;/code&gt; method that allows for clicking and labelling manually cells that I think may be outliers (&lt;a href=&quot;https://stackoverflow.com/questions/19683379/identifying-removing-outliers-from-pca-qq-plots&quot;&gt;see this&lt;/a&gt;) but I do not like it. I would prefer some program that would do that for me, giving some statistical output of the reliability of the removal of the cells.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2017-12-18T09:12:21.190" LastActivityDate="2017-12-18T09:12:21.190" Title="Filter out PCA outliers automatically" Tags="|r|rna-seq|pca|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3116" PostTypeId="1" CreationDate="2017-12-17T22:11:39.267" Score="5" ViewCount="699" Body="&lt;p&gt;I am new to data science. I have a dataset of single-cell gene expression from multiple cell types in &lt;em&gt;C. Elegans&lt;/em&gt;. The dataset is from the paper &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/28818938&quot; rel=&quot;nofollow noreferrer&quot;&gt;Comprehensive single-cell transcriptional profiling of a multicellular organism&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My main question is, &lt;strong&gt;Which approaches should I use for filtering out bad cells in this case when we have multiple cell types in the dataset?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I tried to filter out genes that have too high mitochondrial genes content following the &lt;a href=&quot;https://www.bioconductor.org/help/workflows/simpleSingleCell/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioconductor “simpleSingleCell” workflow&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the tutorial specifically says that the method of filtering out based on mitochondrial genes most probably will not work when the dataset has multiple cell types:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Analyzing all cell types together would unnecessarily inflate the MAD and &#xA;      compromise the removal of low-quality cells, at best; or lead to the &#xA;      entire loss of one cell type, at worst.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Any suggestions would be greatly appreciated. &lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="29" LastEditDate="2017-12-18T09:43:05.970" LastActivityDate="2018-01-10T13:53:23.080" Title="Filter out outliers of the scRNA-seq (heterogenous cells)" Tags="|rna-seq|filtering|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3117" PostTypeId="2" ParentId="3086" CreationDate="2017-12-18T03:25:03.440" Score="5" Body="&lt;p&gt;Author here - this was a bug which was fixed in August:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://github.com/tseemann/prokka/issues/250&quot; rel=&quot;noreferrer&quot;&gt;the issue&lt;/a&gt; in Github. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will need to use the latest Github HEAD version until the next release.&lt;/p&gt;&#xA;" OwnerUserId="550" LastEditorUserId="48" LastEditDate="2017-12-18T09:15:47.397" LastActivityDate="2017-12-18T09:15:47.397" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3118" PostTypeId="1" CreationDate="2017-12-18T11:08:38.893" Score="5" ViewCount="1511" Body="&lt;p&gt;Usually the expression data is transformed to log space using either RPKM, FPKM or CPM, this is required when looking for differential expression because the data is tested against the normal distribution(limma) or the negative bionimal distribution (DESeq2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In weighted gene co-expression analysis (WGCNA) there is also the recommendation to normalize the data (see &lt;a href=&quot;https://labs.genetics.ucla.edu/horvath/CoexpressionNetwork/Rpackages/WGCNA/faq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;FAQ 4&lt;/a&gt;) but says &quot;RPKM, FPKM, or simply normalized counts doesn't make a whole lot of difference for WGCNA&quot; because it is based in correlations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For PCAs the data should be on the same scale to avoid missing the scale as a factor. See &lt;a href=&quot;https://www.biostars.org/p/122086/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; for more references. But should it be normalized too to look like a normal distribution using logarithms?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does this extend to other methods like CCA? Should the data look like a normal distribution for CCA? As it is based on correlation I don't expect much changes, so I am using raw counts (RSEM output), but the examples I've seen use normalized data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;This question comes after reading a &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/3115/filter-out-pca-outliers-automatically#comment4990_3115&quot;&gt;comment&lt;/a&gt;&lt;/sub&gt; &lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="73" LastEditDate="2017-12-18T18:25:46.943" LastActivityDate="2017-12-18T18:34:57.883" Title="Normalizing RNAseq for PCA and CCA" Tags="|rna-seq|normalization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3119" PostTypeId="1" CreationDate="2017-12-12T18:30:26.920" Score="0" ViewCount="96" Body="&lt;p&gt;I'm given a Fasta file, containing a large DNA(over 115,000 long) sequence, and I am tasked with finding a &lt;strong&gt;single&lt;/strong&gt; large open reading frame contained within the DNA sequence using Biopython. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm aware this has been asked before consistently however i can't find anything for actually finding the source of my ORF, but apologies for any overlap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now from other sources and the Biopython cookbook I've translated my sequence and found six open reading frames (three for each strand) and their positions within the sequence;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def find_orfs_with_trans(seq, trans_table, min_protein_length):&#xA;    answer = []&#xA;    seq_len = len(seq)&#xA;    for strand, nuc in [(+1, seq), (-1, seq.reverse_complement())]:&#xA;        for frame in range(3):&#xA;            trans = str(nuc[frame:].translate(trans_table))&#xA;            trans_len = len(trans)&#xA;            aa_start = 0&#xA;            aa_end = 0&#xA;            while aa_start &amp;lt; trans_len:&#xA;                aa_end = trans.find(&quot;*&quot;, aa_start)&#xA;                if aa_end == -1:&#xA;                    aa_end = trans_len&#xA;                if aa_end-aa_start &amp;gt;= min_protein_length:&#xA;                    if strand == 1:&#xA;                        start = frame+aa_start*3&#xA;                        end = min(seq_len,frame+aa_end*3+3)&#xA;                    else:&#xA;                        start = seq_len-frame-aa_end*3-3&#xA;                        end = seq_len-frame-aa_start*3&#xA;                    answer.append((start, end, strand,&#xA;                                   trans[aa_start:aa_end]))&#xA;                aa_start = aa_end+1&#xA;    answer.sort()&#xA;    return answer&#xA;&#xA;orf_list = find_orfs_with_trans(record.seq, table, min_pro_len)&#xA;&#xA;for start, end, strand, pro in orf_list:&#xA;print(&quot;%s...%s - length %i, strand %i, %i:%i&quot; \&#xA;      % (pro[:30], pro[-3:], len(pro), strand, start, end))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;OUTPUT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;TNRQVYGGTLQSLRTGTGIYSRLASSPTNR...CEA - length 160, strand 1, 7950:8433&#xA; GIPPGRTEGLGRYVHGESIFLEATLVPEPQ...VQS - length 171, strand -1, 19275:19791&#xA; ISVHLRRYFSVLLRAPVALNADSRVAGPLD...ITP - length 190, strand 1, 34079:34652&#xA; GCVNGNFPDYRVAVDDPGALVVGGELGQAL...VNT - length 771, strand -1, 39335:41651&#xA; LLKLLASRLTWLLVLSAALGFLTSVSYRLG...SPG - length 235, strand -1, 39358:40066&#xA; PGDVIVSKVPADNLRPRMSEINDFTNSIII...VLV - length 826, strand 1, 39362:41843&#xA; GPGRQSSSAYERDQRLHQQHHHQRERHQVR...QRS - length 764, strand 1, 39385:41680&#xA; LLLSSTSSASCWLKKFFFRLLLLSRSVALL...LAL - length 158, strand -1, 40459:40936&#xA; TRLIIRPPSHTTLYYDPYPQPLKVSSSVPC...RIT - length 159, strand -1, 54663:55143&#xA; IGAFSRGYNQGWVLIEAQEVRTTSPLFPRS...RDQ - length 201, strand 1, 56104:56710&#xA; KARGSSLGAIYCRRLKRRSLPSWQIESFNC...TNS - length 150, strand -1, 62497:62950&#xA; RRPARAGAELNKDDIRDTCLLSCSEVDNKV...VTL - length 168, strand 1, 72339:72846&#xA; SGPTTVRTSAPQRCWRTSLKPKLSLSDSTE...PVA - length 177, strand -1, 114494:115028&#xA; ERELWLETGPPTPLWGTGPDCSRAALNRQR...PRR - length 157, strand 1, 114953:115427&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I'm unsure if these open reading frames contain the ribosome binding site(consensus sequence of &lt;em&gt;AAGGAGGTG&lt;/em&gt; and occurring between &lt;strong&gt;6 to 9&lt;/strong&gt; nucleotides upstream from the positive 5' strand). Is there anyway to add to my code to include this? or would it already be included with Biopython?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm left with the output however I've no idea how to link this with possibly an online source like NCBI to actually find the source of my ORF (i.e. which protein it comes from). Is there anyway to use Biopython for this as well?&lt;/p&gt;&#xA;" OwnerUserId="1860" OwnerDisplayName="maltedg" LastActivityDate="2017-12-18T12:40:41.443" Title="Finding a single open reading frame with ribosomal binding site, using Biopython" Tags="|python|fasta|" AnswerCount="0" CommentCount="0" FavoriteCount="0" ClosedDate="2017-12-18T13:14:55.490" ContentLicense="CC BY-SA 3.0" />
  <row Id="3120" PostTypeId="2" ParentId="3063" CreationDate="2017-12-13T16:04:01.270" Score="2" Body="&lt;p&gt;From what I can tell, the program output does not indicate whether the open reading frames (ORFs) are preceded by ribosome binding sites (RBSs). I'm not sure what parameters are used when establishing ORFs. To verify your RBS requirement, index the input DNA (using given ORF locations from biopython output, pay attention to strands, which are designated by 1/-1) and search upstream for a sequence similar to your consensus. You'd have to pre-define your criteria for similarity to consensus in order to find it programmatically (or you can do it visually to better avoid false negatives). Perhaps try a search that permits ambiguous/non-matching third nucleotides in codons (like a regular expression such as &quot;AA.GA.GT.&quot; where &quot;.&quot; can be any nucleotide). Biologically, third nucleotide changes are least probable to drastically alter the translated residue (more likely to encode amino acid with similar chemical properties). &lt;/p&gt;&#xA;" OwnerUserId="5035" OwnerDisplayName="Ghoti" LastActivityDate="2017-12-13T16:04:01.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3121" PostTypeId="1" CreationDate="2017-12-18T16:31:36.087" Score="6" ViewCount="232" Body="&lt;p&gt;We have RNA-seq fastq data from control (WT) patients and a patient with a point mutation at a known location in one gene.&#xA;I'd like to retrieve the reads aligning to that gene and show the presence of the mutation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can think of 2 strategies:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;A)&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Alignment and visualization, based on &lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/3891/calling-variants-in-rnaseq&quot; rel=&quot;noreferrer&quot;&gt;GATK&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Run STAR 2-pass&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/48719/&quot; rel=&quot;noreferrer&quot;&gt;Extract alignments of interest&lt;/a&gt;, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;samtools view input.bam &quot;Chr10:18000-45500&quot; &amp;gt; output.sam&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Visualize:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;IGV&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://wiki.transvar.org/display/igbman/Visualizing+read+alignments#Visualizingreadalignments-Loadsequencedatatoviewmismatches&quot; rel=&quot;noreferrer&quot;&gt;IGB&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://asciigenome.readthedocs.io/en/latest/description.html&quot; rel=&quot;noreferrer&quot;&gt;ASCIIGenome&lt;/a&gt; (see also &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/176/208&quot;&gt;this&lt;/a&gt; SE answer)&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How can I count the reads showing mutation vs wild type?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;B)&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The second approach would simply look for the count numbers mapping (e.g. salmon) to the transcript ID(s) corresponding to mutation and WT, based on &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2070/208&quot;&gt;this&lt;/a&gt; SE answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there a better way to do this?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2019-09-08T18:29:00.987" Title="Show presence of known mutation in RNA-seq data" Tags="|rna-seq|sequence-alignment|snp|rna-alignment|mutations|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3122" PostTypeId="2" ParentId="3118" CreationDate="2017-12-18T18:34:57.883" Score="2" Body="&lt;p&gt;I've given an answer about using how I normalise for PCA &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2332/73&quot;&gt;here&lt;/a&gt;. PCA works best if the data are normal-ish, but as models go it is fairly robust to fairly large deviations from normal. Transforming is not essential, but if data can be easily transformed into something that looks a bit more normal, then conclusions derived from looking at the PCA should be a bit more robust.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not familiar with canonical-correlation analysis (CCA), but because I see sums and square roots in the formulas, I would assume it has similar data cleaning requirements to PCA. Unfortunately the &lt;a href=&quot;http://stat.ethz.ch/R-manual/R-devel/library/stats/html/cancor.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;R documentation&lt;/a&gt; for &lt;code&gt;cancor&lt;/code&gt; doesn't have any information about any assumptions made on the data.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-12-18T18:34:57.883" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3123" PostTypeId="1" AcceptedAnswerId="4313" CreationDate="2017-12-18T19:14:12.317" Score="3" ViewCount="338" Body="&lt;p&gt;I need to normalize a large (400Mb) dataset for doing PCA analysis. I want to use &lt;code&gt;scran&lt;/code&gt; for doing that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;biocLite(&quot;SingleCellExperiment&quot;)&#xA;biocLite(&quot;scran&quot;)&#xA;library(SingleCellExperiment)&#xA;library(scran)&#xA;list_of_sce &amp;lt;- list()&#xA;# Looping though the UMI_count 'split_factor' columns at a time&#xA;split_factor = 500&#xA;for(i in seq(1,ncols, split_factor)) {&#xA;  num_loop = floor(i / split_factor) + 1&#xA;  idx = ncols&#xA;  if (i + split_factor &amp;lt; ncols) {&#xA;  idx = i + split_factor&#xA;  } &#xA;  sce &amp;lt;- SingleCellExperiment(list(counts=UMI_count[, i : idx]))&#xA;  # Normalization of the dataset containing &#xA;  # heterogenous cell data (different cell types)&#xA;  clusters &amp;lt;- quickCluster(sce) ## &amp;lt;- ** error appears here **&#xA;  sce &amp;lt;- computeSumFactors(sce, cluster=clusters)&#xA;  list_of_sce[[num_loop]] &amp;lt;- sce&#xA;}&#xA;cbind(list_of_sce)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, when I try to run this it produces an error at the &lt;code&gt;quickCluster&lt;/code&gt; step:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error: cannot allocate vector of size 23.7 Gb&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It feels like &lt;code&gt;cbind&lt;/code&gt; at the end is not the right way to go, and I need to somehow combine &lt;code&gt;sce&lt;/code&gt; and re-normalize them again somehow. The question is how to do that in a correct way: normalizing large dataset for PCA analysis when it can not be fully loaded into memory?&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="73" LastEditDate="2018-01-03T19:10:54.337" LastActivityDate="2018-05-17T16:50:44.593" Title="Large dataset normalization for PCA" Tags="|rna-seq|r|normalization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3124" PostTypeId="2" ParentId="3121" CreationDate="2017-12-18T19:26:19.140" Score="3" Body="&lt;p&gt;For counting reads I use mpileup, e.g. &lt;code&gt;samtools mpileup --reference hg38.fa -r Chr10:18000-45500 input.bam&lt;/code&gt;, which will give base-resolution coverage for a BAM file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've written &lt;a href=&quot;https://github.com/gringer/bioinfscripts/blob/master/readstomper.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;my own script&lt;/a&gt; to process mpileup output and make it easier to understand. By default it reports read coverage as a proportion of total coverage, but this can be modified by using the &lt;code&gt;-counts&lt;/code&gt; command line argument:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;$ samtools mpileup -r tig00018708_tig00000379:210665-240664:2107-2130 --reference trimMmerged_tig00018708_other.fasta local_Sampled_50M_vs_trimMmerged_tig00018708_other.bam  | /bioinf/scripts/readstomper.pl -counts&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[mpileup] 1 samples in 1 input files&#xA;&amp;lt;mpileup&amp;gt; Set max per-file depth to 8000&#xA;Assembly,Position,Coverage,ref,cR,pR,A,C,G,T,d,i,InsMode&#xA;tig00018708_tig00000379:210665-240664,2107,10,C,10,1.00,0,0,0,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2108,11,G,11,1.00,0,0,0,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2109,11,T,11,1.00,0,0,0,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2110,11,T,11,1.00,0,0,0,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2111,12,G,12,1.00,0,0,0,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2112,14,G,14,1.00,0,0,0,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2113,13,C,13,1.00,0,0,0,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2114,13,G,13,1.00,0,0,0,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2115,13,C,2,0.15,0,0,0,11,0,0&#xA;tig00018708_tig00000379:210665-240664,2116,15,T,14,1.00,0,0,0,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2117,15,G,14,1.00,0,0,0,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2118,18,A,17,1.00,0,0,0,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2119,19,T,18,1.00,0,0,0,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2120,19,C,18,1.00,0,0,0,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2121,18,A,7,0.41,0,0,10,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2122,18,G,18,1.00,0,0,0,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2123,18,T,18,1.00,0,0,0,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2124,18,C,18,1.00,0,0,0,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2125,18,C,9,0.50,0,0,0,9,0,0&#xA;tig00018708_tig00000379:210665-240664,2126,18,C,14,0.78,2,0,1,1,0,0&#xA;tig00018708_tig00000379:210665-240664,2127,18,C,18,1.00,0,0,0,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2128,18,T,15,0.83,0,0,3,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2129,17,A,13,0.76,0,0,4,0,0,0&#xA;tig00018708_tig00000379:210665-240664,2130,16,A,9,0.56,0,6,1,0,0,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that only &lt;em&gt;non-reference&lt;/em&gt; coverage is reported in the A/C/G/T/d/i columns. I have found that this makes it easier for subsequent data analysis, but your mileage may vary.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-12-18T19:26:19.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3125" PostTypeId="1" CreationDate="2017-12-18T23:28:15.580" Score="1" ViewCount="437" Body="&lt;p&gt;While trying to run the following code in &lt;code&gt;R&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;list_of_sce &amp;lt;- list()&#xA;# Looping though the UMI_count 'split_factor' columns at a time&#xA;split_factor = 500&#xA;for(i in seq(1,ncols, split_factor)) {&#xA;  num_loop = floor(i / split_factor) + 1&#xA;  idx = ncols&#xA;  if (i + split_factor &amp;lt; ncols) {&#xA;    idx = i + split_factor&#xA;  } &#xA;sce &amp;lt;- SingleCellExperiment(list(counts=UMI_count[, i : idx]))&#xA;# Normalization of the dataset containing &#xA;# heterogenous cell data (different cell types)&#xA;clusters &amp;lt;- quickCluster(sce)&#xA;sce &amp;lt;- computeSumFactors(sce, cluster=clusters)&#xA;list_of_sce[[num_loop]] &amp;lt;- sce&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am getting the error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in .computeSumFactors(assay(x, i = assay.type), subset.row = &#xA;subset.row,  : &#xA;cells should have non-zero library sizes&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are &lt;code&gt;warnings&lt;/code&gt;, so when I run &lt;code&gt;warnings()&lt;/code&gt; command I see the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; warnings()&#xA;Warning messages:&#xA;1: In .computeSumFactors(assay(x, i = assay.type), subset.row = &#xA;subset.row,  ... :&#xA;encountered negative size factor estimates&#xA;2: In .computeSumFactors(assay(x, i = assay.type), subset.row = &#xA;subset.row,  ... :&#xA;encountered negative size factor estimates&#xA;3:...&#xA;...&#xA;39:....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I do not understand why is this happening, and how to debug it. Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="29" LastEditDate="2017-12-19T11:17:43.993" LastActivityDate="2017-12-19T11:17:43.993" Title="Error in .computeSumFactors: cells should have non-zero library sizes" Tags="|rna-seq|r|scran|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3126" PostTypeId="1" AcceptedAnswerId="3127" CreationDate="2017-12-19T00:24:42.977" Score="5" ViewCount="106" Body="&lt;p&gt;In &lt;a href=&quot;https://www.nature.com/articles/nrg3542&quot; rel=&quot;noreferrer&quot;&gt;Shapiro et al.&lt;/a&gt;, when discussing about loss of molecules as source of error in single-cell sequencing, it is written that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Another source of error is losses, which can be severe. The detection limit of published protocols is $5$–$10$ molecules of mRNA. If, as seems likely, the limit of detection is primarily determined by losses during sample preparation, this would indicate that $80$–$90\%$ of mRNA was lost. Or, to put it the other way around, a $90\%$ loss leads to an approximately $50\%$ chance of failing to detect a gene that is expressed at a level of seven mRNA molecules (from the binomial distribution).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How is this probability computed using the binomial distribution? I thought that $90\%$ loss corresponds to $5$ detected molecules, and I assume that $k=7$ for the binomial calculation, but I am unable to go further.&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2017-12-19T15:12:46.773" Title="How to compute the chance of failing to detect a gene given the detection limit of a protocol" Tags="|rna-seq|rna|single-cell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3127" PostTypeId="2" ParentId="3126" CreationDate="2017-12-19T08:10:57.617" Score="3" Body="&lt;p&gt;A 90% loss can be rephrased as a 10% chance of detecting anything. So what we want to find is the probability of detecting 0 molecules, when we start with 7 and have 10% probability of success. Once can do that in R as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; pbinom(0, 7, 0.1)&#xA;0.4782969&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So ~50%, as they stated. I suspect that part of the confusion arises from the fact that the detection limit is due to lowly-expressed genes/transcript being heavily affected by this loss. So the probability of detecting a single molecule out of 4 original molecules (assuming 90% mRNA loss) is 34%, for 3 molecules it's 27%, for 2 it's 19% and for 1 it's 10%. I think the threshold of 5 is mentioned more because it's a nice round number than there's anything particularly different in detecting a gene with 4 vs. 5 molecules (34 vs. 41% probability).&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2017-12-19T15:12:46.773" LastActivityDate="2017-12-19T15:12:46.773" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3128" PostTypeId="2" ParentId="3114" CreationDate="2017-12-19T10:10:16.333" Score="2" Body="&lt;p&gt;Did you try the &lt;a href=&quot;http://genome.ucsc.edu/cgi-bin/hgLiftOver&quot; rel=&quot;nofollow noreferrer&quot;&gt;liftOver&lt;/a&gt; tool of UCSC? You'll need a BED file with your SNPs coordinates for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also do this in &lt;code&gt;R&lt;/code&gt; with &lt;code&gt;rtracklayer&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(rtracklayer)&#xA;?liftOver&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastActivityDate="2017-12-19T10:10:16.333" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3129" PostTypeId="2" ParentId="3114" CreationDate="2017-12-19T10:26:32.193" Score="1" Body="&lt;p&gt;See also CrossMap:&#xA;&lt;a href=&quot;http://crossmap.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://crossmap.sourceforge.net/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;20, 21, 22, 23, 24, 30&lt;/p&gt;&#xA;" OwnerUserId="931" LastActivityDate="2017-12-19T10:26:32.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3130" PostTypeId="1" CreationDate="2017-12-19T10:32:47.680" Score="5" ViewCount="519" Body="&lt;p&gt;In VCF I know how to indicate that two genotypes are in the same phase by using consecutive &quot;0|1&quot; and &quot;1|0&quot; genotype fields, for example. However, how do I deal with the case that the first two genotypes are in phase and the second 2 are in phase, but there is no phase between the 2nd and 3rd?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the format I think I need to write:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1 100 . A T . . . GT 1|0&#xA;chr1 200 . A T . . . GT 0|1&#xA;chr1 300 . A T . . . GT 1|0&#xA;chr1 400 . A T . . . GT 0|1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but that implies that the SNP at 200 and the SNP at 300 are in phase, when in fact they are not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to indicate the END of a haplotype block in VCF?&lt;/p&gt;&#xA;" OwnerUserId="931" LastEditorUserId="96" LastEditDate="2017-12-19T21:20:26.970" LastActivityDate="2018-03-08T11:52:31.273" Title="How to indicate the END of a haplotype block in VCF?" Tags="|file-formats|vcf|haplotypes|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3131" PostTypeId="2" ParentId="2623" CreationDate="2017-12-19T10:38:17.053" Score="2" Body="&lt;p&gt;No, this doesn't makes sense (I think).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hydrophobicity of an AA is the hydrophobicity of the AA, in whatever context, just like it's molecular weight. It's weight is it's weight, in a structure or in isolation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know this seems physically unrealistic, but so is an abstract measure of 'hydrophobicity', it's just the best we can do based on partition coefficients.&lt;/p&gt;&#xA;" OwnerUserId="931" LastActivityDate="2017-12-19T10:38:17.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3133" PostTypeId="2" ParentId="3114" CreationDate="2017-12-19T11:13:24.153" Score="5" Body="&lt;p&gt;You can assume that the overwhelming majority of rsIDs are the same between GRCh37 and GRCh38 (they're semi-stable IDs). There are, however, a number of rsIDs that are present only in GRCh37, which &lt;a href=&quot;ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606/known_issues/b150_rs_without_GRCh38_mapping.bcp&quot; rel=&quot;noreferrer&quot;&gt;you can find here&lt;/a&gt;. Note that the format of this file is a bit strange, it's &lt;code&gt;chromosome|position|ID|weight&lt;/code&gt;, where &lt;code&gt;position&lt;/code&gt; is sometimes empty, &lt;code&gt;weight&lt;/code&gt; is typically 1, and &lt;code&gt;ID&lt;/code&gt; is the rsID without the &lt;code&gt;rs&lt;/code&gt; prefix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, &lt;a href=&quot;ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606/known_issues/b150_rs_GRCh37_to_GRCh38_orientation_flip.bcp&quot; rel=&quot;noreferrer&quot;&gt;some rsIDs have a different strand&lt;/a&gt;. This is less likely to be an issue for you, but it's there in case you need it.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-12-19T11:13:24.153" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3134" PostTypeId="2" ParentId="3130" CreationDate="2017-12-19T14:05:33.133" Score="1" Body="&lt;p&gt;Two options that immediately come to mind, depending on what you hope to accomplish down the pipeline: you might simply mark the unphased sites with a &quot;/&quot; instead of &quot;|&quot; to indicate they are not phased; or, if you have multiple haplotype blocks that are all independent of each other, you could use the chromosome field to identify each unique haplotype (e.g. chr1h1, chr1h2, chr2h1, chr2h2, etc).&lt;/p&gt;&#xA;" OwnerUserId="2068" LastActivityDate="2017-12-19T14:05:33.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3135" PostTypeId="2" ParentId="3130" CreationDate="2017-12-19T18:24:29.280" Score="3" Body="&lt;p&gt;According to the VCF specification, haplotype blocks are defined by the &quot;Phase Set&quot; [PS] tag:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;PS : phase set. A phase set is defined as a set of phased genotypes to which this genotype belongs. Phased genotypes for an individual that are on the same chromosome and have the same PS value are in the same phased set. A phase set specifies multi-marker haplotypes for the phased genotypes in the set. All phased genotypes that do not contain a PS subfield are assumed to belong to the same phased set. If the genotype in the GT field is unphased, the corresponding PS field is ignored. The recommended convention is to use the position of the first variant in the set as the PS identifier (although this is not required). (Non-negative 32-bit Integer)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A good example of this can be found on a &lt;a href=&quot;https://support.10xgenomics.com/genome-exome/software/pipelines/latest/output/vcf&quot; rel=&quot;nofollow noreferrer&quot;&gt;10X Genomics support page&lt;/a&gt; about phasing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CHROM   Pos REF ALT GT  PS&#xA;chr1    1000    A   C   0|1 1000&#xA;chr1    1010    T   G   1|0 1000&#xA;chr1    2000    C   T   0|1 2000&#xA;chr1    2005    T   G   0/1 2000&#xA;chr1    2008    G   C   0|1 2000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In this example we have two phase blocks, denoted by PS=1000 and PS=2000. PS=1000 spans position 1000-1010, and PS=2000 spans position 2000-2008. In PS=1000, haplotype 1 contains the REF A allele at position 1000, and the ALT G allele as position 1010, while haplotype 2 contains the ALT C allele at position 1000 and the REF T allele at position 1010.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Note that this phase set identifier needs to be a single numeric value, which means that sub-blocks and overlapping blocks cannot be defined by this field.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-12-22T03:36:47.963" LastActivityDate="2017-12-22T03:36:47.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3136" PostTypeId="1" AcceptedAnswerId="3137" CreationDate="2017-12-19T19:21:07.883" Score="1" ViewCount="3313" Body="&lt;p&gt;I am having a &lt;code&gt;SingleCellExperiment&lt;/code&gt; (sce) object:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;sce &amp;lt;- SingleCellExperiment(list(counts=UMI_count)) &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;which has two dimensions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dim(sce)&#xA;[1] 46761 68027&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I am trying to get column sums for all rows the following does not work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;colSums(assay(sce), na.rm = FALSE, dims = 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Giving me an error: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error in colSums(assay(sce)) : 'x' must be an array of at least two dimensions&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Apparently the issue was already described:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://support.bioconductor.org/p/100954/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.bioconductor.org/p/100954/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I can not find the solution. Is there another way of getting sums of columns of either &lt;code&gt;SingleCellExperiment&lt;/code&gt; object or just &lt;code&gt;dgcMatrix&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2017-12-20T15:28:28.540" Title="Error in colSums(assay(sce)) : 'x' must be an array of at least two dimensions" Tags="|rna-seq|r|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3137" PostTypeId="2" ParentId="3136" CreationDate="2017-12-20T15:28:28.540" Score="0" Body="&lt;p&gt;Ultimately I just implemented it manually with the precursor to &lt;code&gt;SingleCellExperiment&lt;/code&gt; matrix (&lt;code&gt;UMI_count&lt;/code&gt; which is &lt;code&gt;dgcMatrix&lt;/code&gt; type) object in the following way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Removing genes with 0 - counts in all of the columns&#xA;num_removed = 0&#xA;for(i in 1:nrow(UMI_count)) {&#xA;  idx = i - num_removed&#xA;  if(sum(UMI_count[idx,]) &amp;lt; 1) {&#xA;    UMI_count &amp;lt;- UMI_count[-idx,]&#xA;    num_removed = num_removed + 1&#xA;    print(i)&#xA;  }&#xA;}&#xA;&#xA;sce &amp;lt;- SingleCellExperiment(list(counts=UMI_count))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am getting sums of the columns for each row and then filtering out rows with the &lt;code&gt;sum &amp;lt; 1&lt;/code&gt;. So, &lt;code&gt;sum&lt;/code&gt; function for the slices - rows (&lt;code&gt;numeric&lt;/code&gt; type) is working.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2017-12-20T15:28:28.540" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3139" PostTypeId="1" AcceptedAnswerId="3152" CreationDate="2017-12-20T18:07:22.280" Score="3" ViewCount="1449" Body="&lt;p&gt;I want to save a sparse and very large &lt;code&gt;dgcMatrix&lt;/code&gt; onto disc. I read that it could be done with &lt;code&gt;writeMM&lt;/code&gt; method. So, when I am trying to do that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;writeMM(UMI_count, &quot;/home/gene_count_filtered/filtered_dataset&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Rstudio&lt;/code&gt; is giving me an error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in writeMM(UMI_count, &quot;/home/gene_count_filtered/filtered_dataset&quot;) : &#xA;could not find function &quot;writeMM&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Ok, I read that this function is in the &lt;code&gt;Matrix&lt;/code&gt; package. I installed it, but when I try loading it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Matrix)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am getting another error due to other libraries using &lt;code&gt;Matrix&lt;/code&gt; already on the lower level, as I understand:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in value[[3L]](cond) : &#xA;Package ‘Matrix’ version 1.2.11 cannot be unloaded:&#xA;Error in unloadNamespace(package) : namespace ‘Matrix’ is imported by &#xA;‘scran’, ‘scater’, ‘SummarizedExperiment’ so cannot be unloaded&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also tried just&lt;code&gt;write.matrix(UMI_count, &quot;/home/nikita/filtered_dataset&quot;)&lt;/code&gt;, but got an error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Cholmod error 'out of memory' at file ../Core/cholmod_memory.c, line 147&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What could be the solution to the problem? I just need to somehow save a very large &lt;code&gt;dgcMatrix&lt;/code&gt; on disc.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2017-12-22T20:45:10.637" Title="R: 'Matrix' can not be unloaded, but 'writeMM' method not found" Tags="|rna-seq|r|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3140" PostTypeId="1" AcceptedAnswerId="3171" CreationDate="2017-12-20T20:16:24.893" Score="5" ViewCount="611" Body="&lt;p&gt;I am running a notebook with example for the &lt;a href=&quot;https://github.com/pkathail/magic&quot; rel=&quot;noreferrer&quot;&gt;MAGIC&lt;/a&gt; algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the data preprocessing step, a filtering operation is required to filter out cells with a small count of transcripts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is twofold:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Why filtering out cells with a small count of transcripts? &lt;/li&gt;&#xA;&lt;li&gt;How to decide the cutoff, i.e. the minimum number of transcripts each cell should have to not be removed from the dataset?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I attach some plots from the original notebook, which display some distributions of the single-cell rnaseq dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UtHCB.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UtHCB.png&quot; alt=&quot;Plots MAGIC&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2017-12-28T17:39:18.547" Title="Single-cell RNA sequencing (scRNA-seq): filtering cells by transcript counts, how to choose cutoffs?" Tags="|rna-seq|filtering|single-cell|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3141" PostTypeId="2" ParentId="3089" CreationDate="2017-12-21T10:51:33.163" Score="1" Body="&lt;p&gt;Here is how you get the distributions by column using the &lt;code&gt;TraMineR&lt;/code&gt; R package.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(TraMineR)&#xA;&#xA;sts.data &amp;lt;- c(&#xA;  &quot;CAGGTAGCC&quot;,&#xA;  &quot;CCGGTCAGA&quot;,&#xA;  &quot;AGGGTTTGA&quot;,&#xA;  &quot;TTGGTGAGG&quot;,&#xA;  &quot;CAAGTATGA&quot;,&#xA;  &quot;ACTGTATGC&quot;,&#xA;  &quot;CTGGTAACC&quot;,&#xA;  &quot;TATGTACTG&quot;,&#xA;  &quot;GCTGTGAGA&quot;,&#xA;  &quot;CAGGTGGGC&quot;,&#xA;  &quot;TCAGTGAGA&quot;,&#xA;  &quot;GGGGTGAGT&quot;,&#xA;  &quot;TGGGTATGT&quot;,&#xA;  &quot;GAGGTGAGA&quot;,&#xA;  &quot;CAGGTGGAG&quot;&#xA;)&#xA;seq &amp;lt;- seqdef(seqdecomp(sts.data,sep=''))&#xA;&#xA;seqstatd(seq)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and here is the outcome of the &lt;code&gt;seqstatd&lt;/code&gt; function&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  [State frequencies]&#xA;   [1]  [2]  [3] [4] [5]   [6]   [7]   [8]  [9]&#xA;A 0.13 0.40 0.13   0   0 0.400 0.467 0.067 0.40&#xA;C 0.40 0.27 0.00   0   0 0.067 0.067 0.133 0.27&#xA;G 0.20 0.20 0.67   1   0 0.467 0.200 0.733 0.20&#xA;T 0.27 0.13 0.20   0   1 0.067 0.267 0.067 0.13&#xA;&#xA;   [Valid states]&#xA;  [1] [2] [3] [4] [5] [6] [7] [8] [9]&#xA;N  15  15  15  15  15  15  15  15  15&#xA;&#xA;   [Entropy index]&#xA;   [1]  [2]  [3] [4] [5]  [6]  [7]  [8]  [9]&#xA;H 0.94 0.94 0.62   0   0 0.78 0.87 0.62 0.94&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2082" LastActivityDate="2017-12-21T10:51:33.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3142" PostTypeId="2" ParentId="481" CreationDate="2017-12-21T12:56:43.623" Score="5" Body="&lt;p&gt;A modified implementation of Vivek's answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/brentp/peddy&quot; rel=&quot;noreferrer&quot;&gt;peddy&lt;/a&gt; is a Python package that samples an input &lt;code&gt;.vcf&lt;/code&gt; at ~25000 sites and projects onto a principal component space built on 2504 thousand genome samples. The author has extensive documentation of the tool's features and a link to the preprint. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I downloaded the &lt;code&gt;.vcf&lt;/code&gt; and &lt;code&gt;.vcf.tbi&lt;/code&gt; for the NA12878 sample from Genome in a Bottle's &lt;a href=&quot;ftp://ftp-trace.ncbi.nlm.nih.gov/giab/ftp/release/NA12878_HG001/NISTv3.3.2/GRCh37/&quot; rel=&quot;noreferrer&quot;&gt;ftp here&lt;/a&gt;. Then, created a custom &lt;code&gt;.ped&lt;/code&gt; file, &lt;code&gt;NA12878.ped&lt;/code&gt;, with the contents:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;NA12878 HG001   0       0       2       0&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the command line:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;$ peddy --plot --prefix myvcf HG001_GRCh37_GIAB_highconf_CG-IllFB-IllGATKHC-Ion-10X-SOLID_CHROM1-X_v.3.3.2_highconf_PGandRTGphasetransfer.vcf.gz NA12878.ped&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output files all have the prefix &lt;code&gt;myvcf.&lt;/code&gt;, here's &lt;code&gt;myvcf.pca_check.png&lt;/code&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/FmHmB.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FmHmB.png&quot; alt=&quot;myvcf.pca_check.png&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="84" LastActivityDate="2017-12-21T12:56:43.623" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3143" PostTypeId="1" AcceptedAnswerId="3200" CreationDate="2017-12-21T13:08:29.900" Score="4" ViewCount="280" Body="&lt;p&gt;I am trying to install &lt;code&gt;RnBeads&lt;/code&gt; from bioconductor but the installation of its dependancy &lt;code&gt;TxDb.Hsapiens.UCSC.hg19.knownGene&lt;/code&gt; fails. The error is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;biocLite(&quot;RnBeads&quot;)&#xA;BioC_mirror: https://bioconductor.org&#xA;Using Bioconductor 3.6 (BiocInstaller 1.28.0), R 3.4.3 (2017-11-30).&#xA;Installing package(s) ‘RnBeads’&#xA;also installing the dependencies ‘FDb.InfiniumMethylation.hg19’, ‘methylumi’&#xA;&#xA;trying URL 'https://bioconductor.org/packages/3.6/data/annotation/src/contrib/FDb.InfiniumMethylation.hg19_2.2.0.tar.gz'&#xA;Content type 'application/x-gzip' length 38996322 bytes (37.2 MB)&#xA;==================================================&#xA;downloaded 37.2 MB&#xA;&#xA;trying URL 'https://bioconductor.org/packages/3.6/bioc/src/contrib/methylumi_2.24.1.tar.gz'&#xA;Content type 'application/x-gzip' length 6178909 bytes (5.9 MB)&#xA;==================================================&#xA;downloaded 5.9 MB&#xA;&#xA;trying URL 'https://bioconductor.org/packages/3.6/bioc/src/contrib/RnBeads_1.10.2.tar.gz'&#xA;Content type 'application/x-gzip' length 6437974 bytes (6.1 MB)&#xA;==================================================&#xA;downloaded 6.1 MB&#xA;&#xA;* installing *source* package ‘FDb.InfiniumMethylation.hg19’ ...&#xA;** R&#xA;** data&#xA;** demo&#xA;** inst&#xA;** preparing package for lazy loading&#xA;Error: package or namespace load failed for ‘TxDb.Hsapiens.UCSC.hg19.knownGene’:&#xA; .onLoad failed in loadNamespace() for 'TxDb.Hsapiens.UCSC.hg19.knownGene', details:&#xA;  call: NULL&#xA;  error: evaluation nested too deeply: infinite recursion / options(expressions=)?&#xA;Error : package ‘TxDb.Hsapiens.UCSC.hg19.knownGene’ could not be loaded&#xA;ERROR: lazy loading failed for package ‘FDb.InfiniumMethylation.hg19’&#xA;&#xA;The downloaded source packages are in&#xA;    ‘/tmp/RtmpY8Yf4l/downloaded_packages’&#xA;installation path not writeable, unable to update packages: doMC, doSNOW,&#xA;  foreach, iterators, snow, boot, class, cluster, codetools, foreign,&#xA;  KernSmooth, lattice, MASS, Matrix, mgcv, nlme, nnet, rpart, spatial, survival&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As suggested by the error message, I tried setting &lt;code&gt;options(expressions=500000)&lt;/code&gt;, which unfortunately did not change the error message. An important note may be that I am under network mount and don't have administrative access rights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have a .Rprofile file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note about &lt;code&gt;installation path not writeable&lt;/code&gt;:&#xA;Typing .libPaths() reveals that my home R folder is at the first position (and it is writable). The other paths (&lt;code&gt;&quot;/usr/local/lib/R/site-library&quot;&lt;/code&gt;                          , &lt;code&gt;&quot;/usr/lib/R/site-library&quot;&lt;/code&gt; and &lt;code&gt;&quot;/usr/lib/R/library&quot;&lt;/code&gt;) are not writable without sudo.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried to delete my home R folder and to reinstall it. The reinstallation of all packages worked for all packages except for the three listed above (same error message). I also tried &lt;code&gt;biocLite(&quot;TxDb.Hsapiens.UCSC.hg19.knownGene&quot;, lib.loc=.libPaths()[1], lib=.libPaths()[1])&lt;/code&gt; to specifically address my home R directory, but the same error is showing up (but without the &lt;code&gt;installation path not writeable&lt;/code&gt; part).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my sessionInfo():&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R version 3.4.3 (2017-11-30)&#xA;Platform: x86_64-pc-linux-gnu (64-bit)&#xA;Running under: Ubuntu 14.04.4 LTS&#xA;&#xA;Matrix products: default&#xA;BLAS: /usr/lib/openblas-base/libopenblas.so.0&#xA;LAPACK: /usr/lib/lapack/liblapack.so.3.0&#xA;&#xA;locale:&#xA; [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              &#xA; [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    &#xA; [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   &#xA; [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 &#xA; [9] LC_ADDRESS=C               LC_TELEPHONE=C            &#xA;[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       &#xA;&#xA;attached base packages:&#xA;[1] stats     graphics  grDevices utils     datasets  methods   base     &#xA;&#xA;other attached packages:&#xA;[1] BiocInstaller_1.28.0&#xA;&#xA;loaded via a namespace (and not attached):&#xA;[1] compiler_3.4.3 tools_3.4.3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thank you very much!&lt;/p&gt;&#xA;" OwnerUserId="2047" LastEditorUserId="2047" LastEditDate="2017-12-21T15:01:32.170" LastActivityDate="2018-01-03T13:22:03.760" Title="Installing RnBeads via bioconductor - .onLoad failed in loadNamespace(), call: NULL" Tags="|r|bioconductor|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3145" PostTypeId="2" ParentId="3089" CreationDate="2017-12-21T15:08:09.717" Score="2" Body="&lt;p&gt;benn’s answer works but is very un-R-like: you wouldn’t normally iteratively assign to items in a matrix: R has powerful vectorised operations which make this manual work unnecessary. Here’s a more idiomatic solution:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;sequences = readLines('sequences.txt')&#xA;bases_matrix = do.call(rbind, strsplit(sequences, ''))&#xA;&#xA;apply(bases_matrix, 2L, function (col) {&#xA;    str = DNAString(paste(col, collapse = ''))&#xA;    letterFrequency(str, letters = 'ACGT', OR = 0L, as.prob = TRUE)&#xA;})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This uses the same Bioconductor packages. Since this is such a simple problem, it can also be written without Bioconductor:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;bases = strsplit(sequences, '')&#xA;# Use a data.frame here so we can use factors in the next step:&#xA;# R does not support matrices of factors. Ugh.&#xA;bases_by_column = setNames(do.call(rbind.data.frame, bases), seq_along(bases[[1L]]))&#xA;# Ensure that every column will be a complete set of ACGT frequencies&#xA;bases_by_column = lapply(bases_by_column, factor, c('A', 'C', 'G', 'T'))&#xA;sapply(lapply(bases_by_columns, table), prop.table)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Using modern R idioms from the ‘magrittr’ package, I’d write this as a pipeline; this very directly shows the sequence of transformations.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;do.call(rbind.data.frame, bases) %&amp;gt;%&#xA;    setNames(seq_along(bases[[1L]])) %&amp;gt;%&#xA;    lapply(factor, c('A', 'C', 'G', 'T')) %&amp;gt;%&#xA;    lapply(table) %&amp;gt;%&#xA;    sapply(prop.table)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2020-07-22T13:58:33.167" LastActivityDate="2020-07-22T13:58:33.167" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="3146" PostTypeId="1" CreationDate="2017-12-21T19:43:04.800" Score="4" ViewCount="1291" Body="&lt;p&gt;I found this interesting &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE84133&quot; rel=&quot;nofollow noreferrer&quot;&gt;Single RNA-seq data set in GEO&lt;/a&gt;, but I am not sure how to analyze it appropriately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They have deposited transcriptomic profiles of human and mouse pancreatic islets (pancreatic cells: Beta cells, Delta, etc).  The problem I see is that the different pancreatic cell types are not in equal numbers: 872 Beta cells and 214 Delta cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to see whether the expression of a set of genes are correlated between the two cell types. I don't think that I can visualize the correlation between the expression of the Beta and Delta cells on a scatter plot if they are unequal in number. Is there a workaround to include all the cells in the analysis, without losing good quality transcriptome of beta/delta cell?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would like to do is to compare the expression of two pancreatic cell types (Beta v.s. Delta cells) using a scatter plot. I have a set of genes that I am interested in, and would like to see their expression profile for beta and delta cells. As far as I know, it is only possible to visualize this using a scatterplot (X:Beta vs Y:Delta) where the number of isolated beta cells is the same as the number of delta cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; Given unequal number of isolated panacriatic cells, what would be an appropriate way to compare the expression profiles? Should I just ignore the extra ones? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea how to approch this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,  &lt;/p&gt;&#xA;" OwnerUserId="2030" LastEditorUserId="73" LastEditDate="2017-12-29T01:34:36.530" LastActivityDate="2019-06-27T03:30:13.147" Title="How to compare transcriptomic profiles of two cell types (single cell RNA-seq)?" Tags="|python|transcriptome|scrnaseq|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="3147" PostTypeId="1" AcceptedAnswerId="3148" CreationDate="2017-12-21T22:49:18.793" Score="2" ViewCount="1431" Body="&lt;p&gt;What software/tool can I use to visualise a multiple sequence alignment in the xmfa format as produced by command-line tool Mauve?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am particularly interested in seeing the single nucleotide polymorphisms on the screen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am doing a multiple sequence alignment of about a dozen whole genomes, all in a single contig. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Link to &lt;a href=&quot;http://darlinglab.org/mauve/user-guide/introduction.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mauve&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2017-12-22T11:49:24.443" LastActivityDate="2017-12-22T11:49:24.443" Title="What software/tool can I use to visualise a multiple sequence alignment in the xmfa format as produced by command-line tool Mauve?" Tags="|sequence-alignment|software-recommendation|" AnswerCount="2" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3148" PostTypeId="2" ParentId="3147" CreationDate="2017-12-22T01:00:03.297" Score="2" Body="&lt;p&gt;Typically I use the &lt;a href=&quot;http://darlinglab.org/mauve/user-guide/viewer.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mauve viewer&lt;/a&gt; to visualize the alignments.  Are there things you find lacking in that viewer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are multiple &lt;a href=&quot;https://www.biostars.org/p/119721/&quot; rel=&quot;nofollow noreferrer&quot;&gt;threads&lt;/a&gt; about converting xmfa to other formats&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2017-12-22T01:00:03.297" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3149" PostTypeId="2" ParentId="3147" CreationDate="2017-12-22T08:56:09.993" Score="1" Body="&lt;p&gt;I supervised a student a while ago, and she made a tool to focus on (i.e., visualize) SNPs in multiple alignments. The tool is called &lt;a href=&quot;https://sourceforge.net/projects/adoma/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ADOMA&lt;/a&gt;, and is quite simple. It uses clustal for alignment, so I am not sure it if it may work for your Mauve data. If your data are xmfa, you'll probably have to make &lt;a href=&quot;https://sourceforge.net/p/mauve/mailman/message/35156599/&quot; rel=&quot;nofollow noreferrer&quot;&gt;fasta&lt;/a&gt; of it first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I am not sure if this is a solution, but maybe the tool can be of use for your research.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2017-12-22T08:56:09.993" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3150" PostTypeId="2" ParentId="646" CreationDate="2017-12-22T14:42:29.727" Score="2" Body="&lt;p&gt;The post might seem old, but I wish to share my thaughts based on how I solved similar problem. Here it goes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will use an example to illustrate. Other posts have already explained how you could get the protein ID from PDB and so on. I used the database PDBSUM (www.ebi.ac.uk/pdbsum/) and with protein PDB ID: 1KJF.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By following these steps I got to the results below:&#xA;Go to --&gt; www.ebi.ac.uk/pdbsum/ --&gt;(put in PDB ID) or (others e.g UniProt ID) --&gt; Enter --&gt; Click on &quot;Ligands&quot; --&gt;(a LIGPLOT appears at the botoom of the page).&#xA;You can as well search just by pasting in the sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Results:&#xA;You can see that 1KJF is made up of 3 chains A, B &amp;amp; P and both chains have clearly anntated and interacting residues. You can now see clearly read inidividual types of interactions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another good option is &lt;a href=&quot;http://protein.bio.unipd.it/ring&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://protein.bio.unipd.it/ring&lt;/a&gt;. It shows you a graph plot of all inteacting residues and how they are linked to each other. You can adjust the default result settings to meet your needs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps. :)&lt;/p&gt;&#xA;" OwnerUserId="2081" LastEditorUserId="2081" LastEditDate="2017-12-22T14:48:11.110" LastActivityDate="2017-12-22T14:48:11.110" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3151" PostTypeId="1" CreationDate="2017-12-22T17:44:13.147" Score="5" ViewCount="270" Body="&lt;p&gt;I want to get a bed file mapping human protein domains to the human genome. UniProt actually offers such a thing &lt;a href=&quot;ftp://ftp.uniprot.org/pub/databases/uniprot/current_release/knowledgebase/genome_annotation_tracks/UP000005640_9606_beds/UP000005640_9606_domain.bed&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The problem, however, is that the file doesn't include any kind of domain accession, so I have no way of knowing exactly what domain this is. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ grep P30443 UP000005640_9606_domain.bed | sed -E 's/\t+/ /g'&#xA;chr6 29933668 29933929 P30443 0 + 29933668 29933929 153,153,255 1 261 0 . P209-T295; Ig-like C1-type&#xA;chr6 29936281 29936542 P30443 0 + 29936281 29936542 153,153,255 1 261 0 . P209-T295; Ig-like C1-type&#xA;    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This tells me there's a &amp;quot;Ig-like C1-type&amp;quot; domain at positions 209-295 of the protein. But what is this domain?&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.uniprot.org/uniprot/P30443&quot; rel=&quot;nofollow noreferrer&quot;&gt;UniProt page&lt;/a&gt; for this protein doesn't add any more information:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OcsLO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OcsLO.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I thought they might be using the official description of the domain from PFam. However, there is no such domain in PFam as far as I can tell. I downloaded the full &lt;a href=&quot;ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/swisspfam.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;PFam annotation&lt;/a&gt; of SwissProt from PFam's FTP site and searched for P30443:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ zgrep -A4 P30443 swisspfam.gz | sed -E 's/  */ /g; s/[-=_]+//g'&#xA;&amp;gt;1A01HUMAN || P30443.1 365 a.a.&#xA;C1set 1  (5782) PF07654.14 Immunoglobulin C1-set domain 210290&#xA;MHCI 1  (1847) PF00129.17 Class I Histocompatibility antigen, domains alpha 1 and 2 25203&#xA;MHCIC 1  (439) PF06623.10 MHCI Cterminus 337364&#xA;    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This tells me there is a &amp;quot;Immunoglobulin C1-set domain&amp;quot; at 210-290. So both the coordinates and the domain &amp;quot;name&amp;quot; are different.&lt;/p&gt;&#xA;&lt;p&gt;I also checked the &lt;code&gt;Pfam-A.regions.uniprot.tsv.gz&lt;/code&gt; file available &lt;a href=&quot;ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.regions.uniprot.tsv.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. This one had:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ zgrep P30443 Pfam-A.regions.uniprot.tsv.gz | sed -E 's/\t+/ /g'&#xA;P30443 1 8667AFF3F06C4932 d5f7ef1d93bd6e2e0ca1f7b3477ffb67 PF07654 210 290&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This gives me a PFam accession (PF07654) but, once more, I can't use this to go back to the bed file because the coordinates are different.&lt;/p&gt;&#xA;&lt;p&gt;I hoped I could figure this out using the SwissProt flat files, but no luck there either. These are the relevant lines from the entry for P30443:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ID   1A01_HUMAN              Reviewed;         365 AA.&#xA;AC   P30443; O77964; O78171; Q9MYA3; Q9TP25; Q9TQP5;&#xA;[. . .]&#xA;DR   InterPro; IPR021157; Cyt_c1_TM_anchor_C.&#xA;DR   InterPro; IPR007110; Ig-like_dom.&#xA;DR   InterPro; IPR036179; Ig-like_dom_sf.&#xA;DR   InterPro; IPR013783; Ig-like_fold.&#xA;DR   InterPro; IPR003006; Ig/MHC_CS.&#xA;DR   InterPro; IPR003597; Ig_C1-set.&#xA;DR   InterPro; IPR011161; MHC_I-like_Ag-recog.&#xA;DR   InterPro; IPR037055; MHC_I-like_Ag-recog_sf.&#xA;DR   InterPro; IPR011162; MHC_I/II-like_Ag-recog.&#xA;DR   InterPro; IPR001039; MHC_I_a_a1/a2.&#xA;DR   InterPro; IPR010579; MHC_I_a_C.&#xA;DR   Pfam; PF07654; C1-set; 1.&#xA;DR   Pfam; PF00129; MHC_I; 1.&#xA;DR   Pfam; PF06623; MHC_I_C; 1.&#xA;DR   PRINTS; PR01638; MHCCLASSI.&#xA;DR   SMART; SM00407; IGc1; 1.&#xA;DR   SUPFAM; SSF48726; SSF48726; 1.&#xA;DR   SUPFAM; SSF54452; SSF54452; 1.&#xA;DR   PROSITE; PS50835; IG_LIKE; 1.&#xA;DR   PROSITE; PS00290; IG_MHC; 1.&#xA;FT   DOMAIN      209    295       Ig-like C1-type.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, while it does have this domain, uses the same &amp;quot;name&amp;quot; and coordinates I had in the bed file (&amp;quot;Ig-like C1-type&amp;quot;), and it has the relevant PFam accession (PF07654), there's no way to link the &lt;code&gt;FT&lt;/code&gt; line with &amp;quot;Ig-like C1-type&amp;quot; to the relevant PFam accession.&lt;/p&gt;&#xA;&lt;p&gt;I even tried using ProSite and downloaded &lt;code&gt;prosite.dat&lt;/code&gt; from &lt;a href=&quot;ftp://ftp.expasy.org/databases/prosite/prosite.dat&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. This one has yet another &amp;quot;name&amp;quot; for the domain:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;AC   PS50835;&#xA;DT   01-MAY-2003 CREATED; 10-MAY-2017 DATA UPDATE; 20-DEC-2017 INFO UPDATE.&#xA;DE   Ig-like domain profile.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Finally, I had hoped I could use the coordinates given in the bed file to map but since those aren't the same as those I find in the various PFam files, I'm stumped.&lt;/p&gt;&#xA;&lt;p&gt;Is there any way to map the domain names given in the UniProt bed files to specific domains (either PFam, or PROSITE or anything else) so that I can know exactly what domain is annotated there?&lt;/p&gt;&#xA;&lt;p&gt;Update: After jumping through various hoops using the files mentioned above, I have managed to map only 701 domain names which leaves me with 2764 unmapped.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="73" LastEditDate="2022-06-21T09:47:35.433" LastActivityDate="2022-06-21T09:47:35.433" Title="Map domain names from UniProt bed files to domain accessions" Tags="|proteins|public-databases|uniprot|domains|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3152" PostTypeId="2" ParentId="3139" CreationDate="2017-12-22T20:45:10.637" Score="1" Body="&lt;p&gt;Ultimately I just restarted &lt;code&gt;Rstudio&lt;/code&gt; as &lt;code&gt;Konrad Rudolph&lt;/code&gt; suggested, imported &lt;code&gt;Matrix&lt;/code&gt; library, and then &lt;code&gt;writeMM&lt;/code&gt; method worked just fine. However, &lt;code&gt;rownames&lt;/code&gt; were converted into just numbers, but I needed to keep the gene names instead as it was in &lt;code&gt;dgcMatrix&lt;/code&gt;, so after some search I found a different way of doing that from here: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sthda.com/english/wiki/saving-data-into-r-data-format-rds-and-rdata&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sthda.com/english/wiki/saving-data-into-r-data-format-rds-and-rdata&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I just did &lt;code&gt;save()&lt;/code&gt; (and then, to load back - &lt;code&gt;load()&lt;/code&gt;) and the &lt;code&gt;rownames&lt;/code&gt; are maintained like I wanted it to be. So, storing it now as &lt;code&gt;.RData&lt;/code&gt; file.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2017-12-22T20:45:10.637" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3153" PostTypeId="1" CreationDate="2017-12-23T11:27:16.467" Score="4" ViewCount="168" Body="&lt;p&gt;I have file 1 like following:&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   15776220    15776240    GTGACCAGCAGGTGTCTCTG    16855676    16855696    CTGTCCAGCAGAGGGCGGTG    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And file 2 as following&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   15776231    2   5008    G:5002  A:6 1   16855677    2   5008    A:5003  C:5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am trying to write a code(but failing!) to do the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if &lt;code&gt;$2&lt;/code&gt; of file2 comes in between &lt;code&gt;$2&lt;/code&gt; and &lt;code&gt;$3&lt;/code&gt; of file1 (between the interval of &lt;code&gt;$2&lt;/code&gt; and &lt;code&gt;$3&lt;/code&gt;) and if &lt;code&gt;$8&lt;/code&gt; of file2 comes in between &lt;code&gt;$5&lt;/code&gt; and &lt;code&gt;$6&lt;/code&gt; of file1 (between the interval of &lt;code&gt;$5&lt;/code&gt; and &lt;code&gt;$6&lt;/code&gt;), then the output would be &lt;code&gt;$1,$2,$3,$4&lt;/code&gt; from file1 and &lt;code&gt;$2,$3,$4,$5,$6&lt;/code&gt; from file2 and &lt;code&gt;$5,$6,$7&lt;/code&gt; from file1 and &lt;code&gt;$8,$9,$10,$11,$12&lt;/code&gt; from file 2, all in one line. So something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   15776220    15776240    GTGACCAGCAGGTGTCTCTG    15776231    2   5008    G:5002  A:6 16855676    16855696    CTGTCCAGCAGAGGGCGGTG    16855677    2   5008    A:5003  C:5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Something like bedtools does but I am not able to use bedtools one this since I don't have two columns in file2. Is there any way to make bedtools work on this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(There could be many lines from file2 intersecting between the same line in file1,so that line could be repeated)&lt;/p&gt;&#xA;" OwnerUserId="2094" LastActivityDate="2017-12-24T03:43:11.073" Title="Using column 2 of one file to match with two columns of another file, and append" Tags="|vcf|bedtools|bioawk|text-processing|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3154" PostTypeId="1" AcceptedAnswerId="3156" CreationDate="2017-12-23T11:43:27.640" Score="7" ViewCount="1916" Body="&lt;p&gt;I have problem with one my BAMs---it appears to have invalid read groups. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normally when I have such a problem, I remove all the read groups from the BAM header as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -H your.bam | grep -v &quot;^@RG&quot; | samtools reheader - original.bam &amp;gt; new.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then I would execute &lt;code&gt;AddOrReplaceReadGroups&lt;/code&gt; from Picard:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://broadinstitute.github.io/picard/command-line-overview.html#AddOrReplaceReadGroups&quot; rel=&quot;noreferrer&quot;&gt;http://broadinstitute.github.io/picard/command-line-overview.html#AddOrReplaceReadGroups&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried this, and there must still be reads with a read group in the optional tags &lt;code&gt;RG:Z:&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to do I definitively remove all read groups from a BAM, including all of the reads?&lt;/p&gt;&#xA;" OwnerUserId="1770" LastActivityDate="2017-12-23T14:00:33.083" Title="How to remove all BAM read groups from all reads (not just the header)?" Tags="|sam|samtools|reads|picard|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3155" PostTypeId="2" ParentId="3153" CreationDate="2017-12-23T12:31:51.630" Score="1" Body="&lt;p&gt;Here's one approach in Perl. Note that I have not tested this extensively, and I am not confident it works. Please test first and let me know if there are problems. &lt;/p&gt;&#xA;&#xA; &lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env perl&#xA;use strict;&#xA;use warnings;&#xA;use 5.10.0;&#xA;my (@file1,%file2);&#xA;open(my $f1, '-|',&quot;sort -nk2 $ARGV[0]&quot;);&#xA;while (&amp;lt;$f1&amp;gt;) {&#xA;  chomp;&#xA;  my @fields = split(/\s+/);&#xA;  push @file1, [ @fields ];&#xA;}&#xA;close($f1);&#xA;&#xA;open(my $f2, &quot;sort -nk2 $ARGV[1] |&quot;);&#xA;line:while (&amp;lt;$f2&amp;gt;) {&#xA;  chomp;&#xA;  my @fields = split(/\s+/);&#xA;  foreach my $file1Line (@file1) {&#xA;    my $file1Start1 = $file1Line-&amp;gt;[1];&#xA;    my $file1End1 = $file1Line-&amp;gt;[2];&#xA;    my $file1Start2 = $file1Line-&amp;gt;[4];&#xA;    my $file1End2 = $file1Line-&amp;gt;[5];&#xA;    my $file2Start1 = $fields[1];&#xA;    my $file2End1 = $fields[2];&#xA;    my $file2Start2 = $fields[7];&#xA;    my $file2End2 = $fields[8];&#xA;    my @file1Fields = @$file1Line;&#xA;    if ($file1Start1 &amp;lt; $file2Start1 &amp;amp;&amp;amp;&#xA;        $file1End1 &amp;gt; $file2End1 &amp;amp;&amp;amp;&#xA;        $file1Start2 &amp;lt; $file2Start2 &amp;amp;&amp;amp;&#xA;        $file1End2 &amp;gt; $file2End2) {&#xA;      say join &quot;\t&quot;, @{$file1Line}[0..3], @fields[1..5], @{$file1Line}[4..6], @fields[7..11];&#xA;    }&#xA;    elsif ($file1Start1 &amp;gt; $file2Start1) {&#xA;      last;&#xA;    }&#xA;  }&#xA;}&#xA;close($f2);    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Save the script as &lt;code&gt;foo.pl&lt;/code&gt; somewhere in your &lt;code&gt;$PATH&lt;/code&gt;, make it executable and run as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ foo.pl file1.bed file2.bed &#xA;1   15776220    15776240    GTGACCAGCAGGTGTCTCTG    15776231    2   5008    G:5002  A:6 16855676    16855696    CTGTCCAGCAGAGGGCGGTG    16855677    2   5008    A:5003  C:5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2017-12-23T12:52:03.510" LastActivityDate="2017-12-23T12:52:03.510" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3156" PostTypeId="2" ParentId="3154" CreationDate="2017-12-23T14:00:33.083" Score="7" Body="&lt;p&gt;You might have to manually strip those auxiliary tags off:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h your.bam | grep -v &quot;^@RG&quot; | sed &quot;s/\tRG:Z:[^\t]*//&quot; | samtools view -bo your_fixed.bam -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;sed&lt;/code&gt; bit is searching for the aux tag and removing everything up to the next tab.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2017-12-23T14:00:33.083" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3157" PostTypeId="1" CreationDate="2017-12-24T02:33:38.473" Score="4" ViewCount="96" Body="&lt;p&gt;I have a big file containing 27 columns and nearly 6 million rows. The following is a little example of my file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;head data&#xA;0.65   0.722222   1.0      0.75     0&#xA;0.35   0.277778   0.0      0.25     0&#xA;0      0.666667   0.75     0.5      0.5625&#xA;0      0.333333   0.25     0.5      0.4375 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Rows are SNPs and columns are individuals. and I have &quot;2 rows per each SNP&quot; (one for reference allele and the other for alternative allele). In the example above I showed data for 2 SNPs (rows 1 and 2 correspond to SNP1 and rows 3 and 4 correspond to SNP2). I want to check if for each SNP, both ref and alt allele frequencies are 0, replace them with 9. this is my desired output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;head desired&#xA;0.65   0.722222   1.0      0.75     9&#xA;0.35   0.277778   0.0      0.25     9&#xA;9      0.666667   0.75     0.5      0.5625&#xA;9      0.333333   0.25     0.5      0.4375 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1749" LastEditorUserId="298" LastEditDate="2017-12-24T11:10:58.503" LastActivityDate="2018-01-26T10:36:17.010" Title="replacing SNPs with missing calls with a specific string" Tags="|python|perl|text-processing|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3158" PostTypeId="2" ParentId="3153" CreationDate="2017-12-24T03:32:28.753" Score="3" Body="&lt;p&gt;Pipe a modified form of the second file to BEDOPS &lt;code&gt;bedmap&lt;/code&gt; and the first file, then pipe &lt;em&gt;that&lt;/em&gt; result to cut out desired columns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk -v OFS='\t' '{ print $1, $2, ($2+1), $3, $4, $5, $6, $7, $8, $9, $10; }' second.txt | bedmap --echo --echo-map first.bed - | cut -f1-4,8,9,11- - &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This should run pretty quickly and use very little memory, as it uses Unix streams on (presumably) sorted BED data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll want to test the column indices in my answer by running this on your test inputs, but that's an easy adjustment for the larger dataset, in case I missed a column.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're not sure about sort order, pipe to &lt;code&gt;sort-bed&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sort-bed first.unsorted.bed &amp;gt; first.bed&#xA;$ awk -v OFS='\t' '{ print $1, $2, ($2+1), $3, $4, $5, $6, $7, $8, $9, $10; }' second.txt | sort-bed - | bedmap --echo --echo-map first.bed - | cut -f1-4,8,9,11- - &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Use streams where you can. It's a different way of thinking but can pay huge dividends.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2017-12-24T03:43:11.073" LastActivityDate="2017-12-24T03:43:11.073" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3160" PostTypeId="2" ParentId="3157" CreationDate="2017-12-27T10:11:23.767" Score="2" Body="&lt;p&gt;A possible approach in plain python:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;import sys&#xA;&#xA;def both_zero(freq1, freq2):&#xA;    return freq1 == &quot;0&quot; and freq2 == &quot;0&quot;&#xA;&#xA;with open(sys.argv[1], &quot;r&quot;) as snp_file:&#xA;    for line1 in snp_file:&#xA;        # Consume one extra line to get the frequencies&#xA;        # for the alternative allele&#xA;        line2 = snp_file.readline()&#xA;        # &quot;zip&quot; the frequencies in pairs&#xA;        freq_pairs = zip(line1.strip().split(), line2.strip().split())&#xA;        recoded_pairs = [&#xA;            (&quot;9&quot;, &quot;9&quot;) if both_zero(freq1, freq2) else (freq1, freq2)&#xA;            for (freq1, freq2) in freq_pairs]&#xA;        # &quot;unzip&quot; the recoded frequencies into separate objects&#xA;        # (see https://stackoverflow.com/a/19343/1878788)&#xA;        (recoded1, recoded2) = zip(*recoded_pairs)&#xA;        print(*recoded1, sep=&quot;\t&quot;)&#xA;        print(*recoded2, sep=&quot;\t&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Calling it with your example snp frequencies file as command-line argument:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ./recode_absent.py snps.txt&#xA;0.65    0.722222        1.0     0.75    9&#xA;0.35    0.277778        0.0     0.25    9&#xA;9       0.666667        0.75    0.5     0.5625&#xA;9       0.333333        0.25    0.5     0.4375&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="292" LastActivityDate="2017-12-27T10:11:23.767" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3161" PostTypeId="1" AcceptedAnswerId="3162" CreationDate="2017-12-27T16:49:36.950" Score="0" ViewCount="190" Body="&lt;p&gt;This is the error that I am getting:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;launch_varscan.pl: Creating a pileup /home/user/lyve-SET-1.1.4f/lambda/tmp/NC001416.fasta.wgsim.fastq.gz-reference.sorted.bam.mpileup&#xA;removed ‘/home/user/lyve-SET-1.1.4f/lambda/tmp/NC001416.fasta.wgsim.fastq.gz-reference.sorted.bam.1239.mpileup’&#xA;launch_varscan.pl: Running mpileup:&#xA;  echo &quot;NC_001416.1&#xA;&quot; | xargs -P 1 -n 1 -I {} sh -c 'echo &quot;MPileup on {}&quot; &amp;gt;&amp;amp;2; samtools mpileup -f lambda/reference/reference.fasta &#xA;-Q 0 -B  --region &quot;{}&quot; /home/user/lyve-SET-1.1.4f/lambda/bam/NC001416.fasta.wgsim.fastq.gz-reference.sorted.bam &amp;gt; &#xA;/home/user/lyve-SET-1.1.4f/lambda/tmp/NC001416.fasta.wgsim.fastq.gz-reference.sorted.bam.$$.mpileup'&#xA;MPileup on NC_001416.1&#xA;mpileup: unrecognized option '--region'&#xA;open: No such file or directory&#xA;[mpileup] failed to open NC_001416.1: No such file or directory&#xA;launch_varscan.pl: main::mpileup: ERROR with xargs and samtools mpileup&#xA;Stopped at /home/arahat/lyve-SET-1.1.4f/scripts/launch_varscan.pl line 106.&#xA;QSUB ERROR&#xA;31488&#xA;launch_set.pl: Schedule::SGELK::command: ERROR with command: Inappropriate ioctl for device&#xA;  /usr/bin/perl /home/user/lyve-SET-1.1.4f/lambda/log/SGELK/qsub.2705.pl&#xA;Stopped at /home/user/lyve-SET-1.1.4f/scripts/../lib/Schedule/SGELK.pm line 661.&#xA;set_test.pl: ERROR: died while trying to run COMMAND&#xA;  launch_set.pl lambda --numcpus 1 --numnodes 20&#xA;set_test.pl: See if you have the correct software installed for this command.&#xA;6400 at /home/arahat/lyve-SET-1.1.4f/scripts/set_test.pl line 94.&#xA;make: *** [test] Error 25&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think the main problem is the &quot;mpileup: unrecognized option --region&quot;, which seems to be blank. In the beginning of the block, it says creating pileup and then it immediately removes the file, probably to create the pipeline file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot change the code for Lyve-SET pipelining or create shell scripts for the pipeline because I am only configuring my server for use with the genome sequencing pipeline. Would this be a problem with the data set or some tool that I need to install? Any insight would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2107" LastEditorUserId="964" LastEditDate="2017-12-27T17:30:20.537" LastActivityDate="2017-12-27T17:30:20.537" Title="I am getting this error when running a test data set for Lyve-SET on RHEL 7. Is this an MPileup error or something else?" Tags="|samtools|mpileup|" AnswerCount="1" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3162" PostTypeId="2" ParentId="3161" CreationDate="2017-12-27T17:19:31.107" Score="0" Body="&lt;p&gt;The &lt;a href=&quot;http://www.htslib.org/doc/samtools-0.1.19.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools v0.1.19 manual&lt;/a&gt; confirms that there is no &lt;code&gt;--region&lt;/code&gt; option.  In that version there is only the &lt;code&gt;-r&lt;/code&gt; argument to pass in the targeted region.  If you update to a newer version of samtools this should execute correctly&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2017-12-27T17:19:31.107" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3163" PostTypeId="2" ParentId="2691" CreationDate="2017-12-27T19:12:00.960" Score="2" Body="&lt;p&gt;Unless I'm mistaken, the only variant of GFF that explicitly permits Fasta entries is GFF3, and the data you've shown above is definitely not valid GFF3. The mRNA entry seems to correctly use the &lt;code&gt;=&lt;/code&gt; symbol to separate keys from values in each attribute, but the remaining columns all use spaces and quotation marks (GTF style) instead. This is likely to cause all kinds of problems with software packages that take GFF data as input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But to more directly address your question, there is no &quot;right&quot; way to connect feature entries to sequence entries in a GFF3 file. There are some &lt;em&gt;conventions&lt;/em&gt;, but these are all very specific to a small set of tools and community of practice, and it is rare that these conventions are objectively enforced. Some options include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Placing genomic nucleotide sequences in the &lt;code&gt;##FASTA&lt;/code&gt; section, and then using feature coordinates to pull out mature mRNA sequences and translate them &lt;em&gt;in silico&lt;/em&gt; to derive the corresponding peptide sequence. In this case, the Fasta entries would be genomic sequence identifiers that correspond to the first field/column in each feature entry.&lt;/li&gt;&#xA;&lt;li&gt;Placing mature mRNA sequences in the &lt;code&gt;##FASTA&lt;/code&gt; section, and then directly translating this sequence to derive the corresponding peptide sequence. In this case, you'll want the sequence IDs in the &lt;code&gt;##FASTA&lt;/code&gt; section to match the &lt;code&gt;ID&lt;/code&gt; attribute of the corresponding mRNA feature.&lt;/li&gt;&#xA;&lt;li&gt;Placing the peptide sequences directly in the &lt;code&gt;##FASTA&lt;/code&gt; section. In this case, you would either want to have &lt;code&gt;protein&lt;/code&gt; features declared in the GFF3 file (uncommon) or have some sort of &lt;code&gt;protein_id&lt;/code&gt; attribute associated with each mRNA feature, in which case you'd want the sequence IDs in the &lt;code&gt;##FASTA&lt;/code&gt; section to match this &lt;code&gt;protein_id&lt;/code&gt; attribute.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;There are probably other ways of doing it, too. Which way is &quot;right&quot;? The GFF3 specification has nothing to say about it. It depends on the tools you are using to do the subsequent processing and analysis. If you're lucky, those tools might even document what they expect in terms of input.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Let me just say, as a post-script, that I think placing sequences and feature annotations in the same file is a horrible idea. I've never liked this about GFF3.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2017-12-27T19:12:00.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3164" PostTypeId="1" AcceptedAnswerId="4105" CreationDate="2017-12-27T21:24:16.713" Score="6" ViewCount="256" Body="&lt;p&gt;I am trying to build the preprocessing pipeline presented in &lt;a href=&quot;https://www.biorxiv.org/content/early/2017/12/20/237446&quot; rel=&quot;noreferrer&quot;&gt;The Tabula Muris Consortium et al. (pp)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a pipeline to preprocess single-cell sequencing data. There is one step that is not clear:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Counts were log-normalized (log(1 + counts per N)), then scaled by linear regression against the number of reads (or UMIs), the percent of reads mapping to Rn45s, and the percent of reads to ribosomal genes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I understand the first part (I assume that log in this context is log2), but I need help on understanding how to &lt;em&gt;scale by linear regression against the number of reads, the percent of reads mapping to Rn45s, and the percent of reads to ribosomal genes.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2017-12-27T22:27:59.660" LastActivityDate="2018-06-26T21:18:56.493" Title="Scaling by linear regression against the number of reads" Tags="|reads|single-cell|scrnaseq|linear-regression|data-preprocessing|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3165" PostTypeId="1" AcceptedAnswerId="3166" CreationDate="2017-12-27T22:13:32.887" Score="6" ViewCount="5173" Body="&lt;p&gt;I am reading &lt;a href=&quot;https://www.biorxiv.org/content/early/2017/12/20/237446&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Tabula Muris Consortium et al. (pp)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In some organs, cells with more than 2 million reads were also excluded as a conservative measure to avoid doublets.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How exactly is a “doublet” defined? For example, is doublet a set of cells sequenced as a single cell (and so, the number of transcripts is double)?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="57" LastEditDate="2018-06-03T14:25:20.517" LastActivityDate="2020-07-07T07:13:29.087" Title="What are doublets in single cell RNA-seq data?" Tags="|scrnaseq|single-cell|quality-control|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3166" PostTypeId="2" ParentId="3165" CreationDate="2017-12-27T22:28:19.283" Score="7" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Is doublet a set of cells sequenced as a single cell?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes. Depending on the method of single cell sequencing it may be more or less likely for groups of cells to be captured and barcoded with the same &amp;quot;unique&amp;quot; barcode. This is more likely in split-pool RNA sequencing (e.g. &lt;a href=&quot;https://www.biorxiv.org/content/early/2017/02/02/105163&quot; rel=&quot;nofollow noreferrer&quot;&gt;SPLiT-seq&lt;/a&gt;), and less likely in cell-capture RNA sequencing (e.g. &lt;a href=&quot;https://www.fluidigm.com/products/c1-system&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fluidigm C1&lt;/a&gt;). Doublets can also be created through physical / experimental processes (e.g. from tissues that were not completely dissociated).&lt;/p&gt;&#xA;&lt;p&gt;Bear in mind that detecting doublets is not as simple as counting for doubling of transcript expression, because the expression profiles are different in different cells. That's why single-cell sequencing is useful in the first place.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-07-07T07:13:29.087" LastActivityDate="2020-07-07T07:13:29.087" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3167" PostTypeId="1" AcceptedAnswerId="3168" CreationDate="2017-12-28T00:43:27.360" Score="3" ViewCount="166" Body="&lt;p&gt;I have assembled some genome from Sequel PacBio data both with HGAP4 on the SMRT Link interface and using canu on the command line. The HGAP4 assembler outputs a fastq file of the final assembly such that I can see how reliable the sequence is along the genome. On the other hand, canu does no such thing, which means I do not know how much I can trust each nucleotide in the assembly? Is there a way canu can output the fastq file as well or at least some depth information at each base?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2017-12-28T02:05:26.477" Title="Can the canu assembler output a fastq file of the final assembly just like HGAP4?" Tags="|genome|assembly|fastq|genome-sequencing|canu|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3168" PostTypeId="2" ParentId="3167" CreationDate="2017-12-28T02:05:26.477" Score="2" Body="&lt;p&gt;I've created an &lt;a href=&quot;https://github.com/marbl/canu/issues/744&quot; rel=&quot;nofollow noreferrer&quot;&gt;issue&lt;/a&gt; on the Canu github repository for this. I'm not aware of any existing functionality to output FASTQ files, but think that this would be a useful feature to have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be possible to create something like this by aligning the original reads to the genome, generating a pileup from that alignment, then determining the sequence likelihood based on that pileup. Unfortunately, I don't have any scripts that I've written to do that... yet.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-12-28T02:05:26.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3169" PostTypeId="1" CreationDate="2017-12-28T08:52:31.577" Score="2" ViewCount="832" Body="&lt;p&gt;I need to extract (or to calculate precisely) the query coverage from Blast output. I am using NCBIWWW.qblast to query Blast in XML format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to know the query coverage parsing the XML output? If yes, how?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;result_handle_xml = NCBIWWW.qblast(&quot;blastp&quot;,&quot;pdb&quot;,my_query.seq,format_type='XML')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK279684/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; it suggest to use '-6 qcovs', but only for BLAST command line version and only for tabular format&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the fasta file that I open:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;sp|P02489|CRYAA_HUMAN Alpha-crystallin A chain OS=Homo sapiens &#xA;GN=CRYAA PE=1 SV=2&#xA;MDVTIQHPWFKRTLGPFYPSRLFDQFFGEGLFEYDLLPFLSSTISPYYRQSLFRTVLDSG&#xA;ISEVRSDRDKFVIFLDVKHFSPEDLTVKVQDDFVEIHGKHNERQDDHGYISREFHRRYRL&#xA;PSNVDQSALSCSLSADGMLTFCGPKIQTGLDATHAERAIPVSREEKPTSAPSS&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1453" LastEditorUserId="3848" LastEditDate="2020-05-03T00:14:57.960" LastActivityDate="2023-08-30T13:04:25.540" Title="Biopython - extracting query coverage from XML Blast output" Tags="|python|blast|parsing|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="3170" PostTypeId="1" AcceptedAnswerId="3194" CreationDate="2017-12-28T17:09:42.223" Score="4" ViewCount="1335" Body="&lt;p&gt;I am running the data preprocessing pipeline for scRNA-seq data presented &lt;a href=&quot;https://hemberg-lab.github.io/scRNA.seq.course/construction-of-expression-matrix.html#gene-analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;3.8.6.1 Gene expression&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In addition to removing cells with poor quality, it is usually a good idea to exclude genes where we suspect that technical artefacts may have skewed the results. Moreover, inspection of the gene expression profiles may provide insights about how the experimental procedures could be improved.&lt;/p&gt;&#xA;&lt;p&gt;It is often instructive to consider the number of reads consumed by the top 50 expressed genes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;By analyzing my data I have the following most expressed 50 genes (evaluating the total number of transcripts across cells):&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Original&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CCLEl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CCLEl.png&quot; alt=&quot;linear&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Log-transformed&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/91QoD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/91QoD.png&quot; alt=&quot;log&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since I'll be working on log-transformed data, I assumed that the distributions of the top 50 genes were not excessively skewed. However, as you can see in the first plot, there are some samples (cells) with very high number of transcripts for the top gene. Is it something I should worry about? What other insights can I get from these two figures?&lt;/p&gt;&#xA;&lt;h2&gt;Edit&lt;/h2&gt;&#xA;&lt;p&gt;Original analysis (figure attached) was:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/k0db3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/k0db3.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The distributions are relatively flat indicating (but not guaranteeing!) good coverage of the full transcriptome of these cells. However, there are several spike-ins in the top 15 genes which suggests a greater dilution of the spike-ins may be preferrable if the experiment is to be repeated.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-01-02T23:19:43.323" Title="Inspection of gene expression in scRNA-seq data" Tags="|gene|scrnaseq|homework|data-preprocessing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3171" PostTypeId="2" ParentId="3140" CreationDate="2017-12-28T17:39:18.547" Score="1" Body="&lt;p&gt;By reading &lt;a href=&quot;https://hemberg-lab.github.io/scRNA.seq.course/construction-of-expression-matrix.html#cell-qc&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Wells with few reads/molecules are likely to have been broken or failed to capture a cell, and should thus be removed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This answers the first part of the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the second part, i.e. how to decide cutoff, it still seems a bit arbitrary to me, but some authors trim the cells on the left tail of the highest mode, in order to remove other modes (likely due to noise) on the left tail, considering the distribution of cells dependent on the sum of reads (molecules per cell, first plot).&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2017-12-28T17:39:18.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3172" PostTypeId="1" CreationDate="2017-12-28T18:02:57.107" Score="7" ViewCount="307" Body="&lt;p&gt;After many years of programming in Python, C/C++ and other mainstream languages, I am looking for a new language of pleasure, mainly as a part of mental hygiene. Since I really admire the ideas behind Lisp, including the original paper from the 1950's, I would like to learn some Lisp dialect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which Lisp dialects are currently suitable for bioinformatics (e.g., for developing simple command-line tools)? I have the following requirements:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Portability across main Unix platforms and reproducibility (this seems to be a huge issue in many Lisp-like languages)&lt;/li&gt;&#xA;&lt;li&gt;Resulting programs being installable using Bioconda (it should be technically possible at least in future if not now)&lt;/li&gt;&#xA;&lt;li&gt;Negligible starting time (this disqualifies Clojure, which requires approximately 10 seconds)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2113" LastEditorUserId="29" LastEditDate="2017-12-29T16:03:01.330" LastActivityDate="2017-12-29T16:14:01.887" Title="Lisp in bioinformatics" Tags="|reproducibility|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3173" PostTypeId="2" ParentId="3172" CreationDate="2017-12-28T21:10:59.133" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Which LISP dialects are currently suitable for bioinformatics (e.g., for developing simple command-line tools)? I have the following requirements:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have recently pushed a PR for samtools (mostly for fun) &lt;a href=&quot;https://github.com/samtools/samtools/pull/752&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/samtools/samtools/pull/752&lt;/a&gt; that uses the &lt;strong&gt;GNU Guile library&lt;/strong&gt; &lt;a href=&quot;https://www.gnu.org/software/guile/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.gnu.org/software/guile/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./samtools view   -g \&#xA;   '(define read-filter (lambda (R) (and (&amp;gt; (hts-read-length R ) 10 ) (integer? (string-contains  (hts-read-seq R) &quot;GCCTTGGCTCTTGTTCC&quot;)))))' input.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastActivityDate="2017-12-28T21:10:59.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3174" PostTypeId="2" ParentId="3146" CreationDate="2017-12-29T01:47:13.300" Score="1" Body="&lt;p&gt;Scatterplots are used to represent multiple dimensions of the same piece of data. A scatterplot would be used in this case to display &lt;em&gt;all&lt;/em&gt; genes in your set on a single plot, but I would have some difficulty interpreting such a plot. If you want to look separately at each gene, then a density curve (or histogram) would be more appropriate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your problem is similar to common problems in flow cytometry, and I don't think there's any reason why similar approaches can't be used. Here's a histogram / density plot showing two different types of cells (negative control, and treatment condition), indicating the difference in fluorescence of a particular marker (somewhat analogous to gene expression) across all cells of each type. Note that the number of cells are different between the negative control and treatment:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/v4BoC.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/v4BoC.jpg&quot; alt=&quot;Flow cytometry histogram&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This image was taken from &lt;a href=&quot;http://www.assay-protocol.com/cell-biology/flow-cytometry/flowcytometry-diagrams&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, which also demonstrates a 2D scatter plot (where each point is a &lt;em&gt;cell&lt;/em&gt;, rather than a gene).&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2017-12-29T01:53:13.823" LastActivityDate="2017-12-29T01:53:13.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3175" PostTypeId="1" CreationDate="2017-12-29T09:49:47.400" Score="4" ViewCount="178" Body="&lt;p&gt;I have a list of several thousand proteins and their UNIPROT IDs. I'm looking for an efficient method of cross-referencing it against the PDB tertiary structure database, and get a list of those proteins with a tertiary structure in the PDB database.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried to BLASTP the list of UNIPROT IDs against the PDB database, using the NCBI BLAST portal but encountered too many errors of &quot;Error: Failed to read the Blast query: Sequence ID not found&quot;, making the process of manual filtering not convenient and not efficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2017-12-29T10:42:52.390" Title="Cross-reference with PDB database" Tags="|proteins|protein-structure|pdb|uniprot|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3176" PostTypeId="2" ParentId="3175" CreationDate="2017-12-29T10:12:19.040" Score="2" Body="&lt;p&gt;The UniProt website offers an &lt;a href=&quot;http://www.uniprot.org/uploadlists/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ID mapping tool&lt;/a&gt; that tends to work pretty well:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VXN4K.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VXN4K.png&quot; alt=&quot;UniProt ID mapping tool screenshot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, hit &quot;submit&quot; and you will get the list of corresponding PDB IDs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/n7SIH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/n7SIH.png&quot; alt=&quot;UniProt ID mapping tool results screenshot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-12-29T10:12:19.040" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3177" PostTypeId="2" ParentId="2486" CreationDate="2017-12-29T10:20:37.313" Score="2" Body="&lt;p&gt;&lt;code&gt;vcf-merge&lt;/code&gt; (or better: &lt;code&gt;bcftools merge&lt;/code&gt;) does a column-based merge of VCF files, basically taking on an additional genotype column to each variant. While this sort-of works, it doesn't take into account all the information about variation. There might be low coverage at a region in one VCF file that indicates a deletion, but the deletion is only observable when considering other samples with high coverage at that same region. It's also possible that INDELs could be defined differently in different samples, requiring an additional &lt;code&gt;bcftools norm&lt;/code&gt; step for left-normalisation prior to the merge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming this column-based merge is what you want (and if not, then &lt;code&gt;bcftools concat&lt;/code&gt; is your friend for row-based merging), then it's better to regenerate the merged VCF file as a multi-column file directly. One way of doing this is feeding &lt;em&gt;all&lt;/em&gt; BAM files into &lt;code&gt;samtools mpileup&lt;/code&gt; to generate genotype likelihoods, then feeding the result of that into &lt;code&gt;bcftools call&lt;/code&gt;. Something like the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools mpileup -uf ref.fa in1.bam in2.bam in3.bam | \&#xA;  bcftools call -mv -O z -o variants_ref_1-3.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2017-12-29T10:20:37.313" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3178" PostTypeId="2" ParentId="3175" CreationDate="2017-12-29T10:42:52.390" Score="2" Body="&lt;p&gt;You can use the SIFTS mapping: &lt;a href=&quot;http://www.ebi.ac.uk/pdbe/docs/sifts/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ebi.ac.uk/pdbe/docs/sifts/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3531078/&quot; rel=&quot;nofollow noreferrer&quot;&gt;an article&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Structure Integration with Function, Taxonomy and Sequences&#xA;  resource (SIFTS; &lt;a href=&quot;http://pdbe.org/sifts&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://pdbe.org/sifts&lt;/a&gt;) is a close collaboration&#xA;  between the Protein Data Bank in Europe (PDBe) and UniProt. The two&#xA;  teams have developed a semi-automated process for maintaining&#xA;  up-to-date cross-reference information to UniProt entries, for all&#xA;  protein chains in the PDB entries present in the UniProt database.&#xA;  This process is carried out for every weekly PDB release and the&#xA;  information is stored in the SIFTS database.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It has no web interface for querying. You just download CSV (or TSV) file with all the mapping data.&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2017-12-29T10:42:52.390" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3179" PostTypeId="2" ParentId="3165" CreationDate="2017-12-29T12:35:24.080" Score="7" Body="&lt;p&gt;&quot;Doublet&quot; is commonly used to describe a droplet in droplet-based sequencing that has captured atleast 2 cells. 10x states their doublet rate to be &lt;a href=&quot;https://assets.ctfassets.net/an68im79xiti/1eX2FPdpeCgnCJtw4fj9Hx/7cb84edaa9eca04b607f9193162994de/CG000204_ChromiumNextGEMSingleCell3_v3.1_Rev_D.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;0.8% per 1000 cells&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GoNFg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GoNFg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a tradeoff between targeted cell capture and doublet rate and lab protocols are usually optimized for a certain expected doublet rate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One should expect to see close to doubling of transcripts but it is not trivial to infer this computationally.&lt;/p&gt;&#xA;" OwnerUserId="2116" LastEditorUserId="-1" LastEditDate="2020-03-24T03:11:48.360" LastActivityDate="2020-03-24T03:11:48.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3180" PostTypeId="2" ParentId="3172" CreationDate="2017-12-29T16:14:01.887" Score="3" Body="&lt;p&gt;I like the idea but I’m not aware of a strong Lisp presence anywhere in bioinformatics. I think a good starting point would be the paper &lt;a href=&quot;https://doi.org/10.1093/bib/bbw130&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;How the strengths of Lisp-family languages facilitate building complex and flexible bioinformatics applications&lt;/em&gt;&lt;/a&gt; by Khomtchouk &lt;em&gt;&amp;amp;al&lt;/em&gt; (and &lt;a href=&quot;https://news.ycombinator.com/item?id=13354035&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Hacker News discussion&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Excluding Clojure from consideration will unfortunately close off a great many modern resources (although I fundamentally agree with your technical reason for doing so). Apart from this, &lt;a href=&quot;https://en.wikipedia.org/wiki/Scheme_%28programming_language%29&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scheme&lt;/a&gt; (in particular R7RS) is probably the most-used Lisp dialect, ahead of &lt;a href=&quot;https://en.wikipedia.org/wiki/Common_Lisp&quot; rel=&quot;nofollow noreferrer&quot;&gt;Common Lisp&lt;/a&gt;. Or at least the most actively developed one. As Pierre mentioned, there’s also &lt;a href=&quot;https://en.wikipedia.org/wiki/GNU_Guile&quot; rel=&quot;nofollow noreferrer&quot;&gt;GNU Guile&lt;/a&gt; but, to be honest, despite it being the default for GNU software tools I don’t think it’s frequently used outside GDB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, Clojure’s startup time is &lt;em&gt;not&lt;/em&gt; anywhere near 10 seconds on my system. It’s not exactly instantaneous but not much worse than other dynamic languages, despite being backed by the rather sluggish JVM.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2017-12-29T16:14:01.887" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3181" PostTypeId="1" AcceptedAnswerId="3186" CreationDate="2017-12-29T17:49:40.863" Score="6" ViewCount="1117" Body="&lt;p&gt;I am working on a GFF file that is missing the 5'UTR and 3'UTR information. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  ctg123 . gene      1050  9000  .  +  .  ID=gene00001;Name=EDEN&#xA;  ctg123 . mRNA      1050  9000  .  +  .  ID=mRNA00001;Parent=gene00001;Name=EDEN.1&#xA;  ctg123 . exon      1050  1500  .  +  .  ID=exon00002;Parent=mRNA00001&#xA;  ctg123 . exon      3000  3902  .  +  .  ID=exon00003;Parent=mRNA00001&#xA;  ctg123 . exon      5000  5500  .  +  .  ID=exon00004;Parent=mRNA00001&#xA;  ctg123 . exon      7000  9000  .  +  .  ID=exon00005;Parent=mRNA00001&#xA;  ctg123 . CDS       1201  1500  .  +  0  ID=cds00001;Parent=mRNA00001;Name=edenprotein.1&#xA;  ctg123 . CDS       3000  3902  .  +  0  ID=cds00001;Parent=mRNA00001;Name=edenprotein.1&#xA;  ctg123 . CDS       5000  5300  .  +  0  ID=cds00001;Parent=mRNA00001;Name=edenprotein.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a way to add rows in the GFF with the 5'UTR and 3'UTR ranges?&lt;/p&gt;&#xA;" OwnerUserId="2118" LastEditorUserId="96" LastEditDate="2019-03-26T19:02:44.160" LastActivityDate="2019-03-28T15:56:39.460" Title="Infer missing UTR features in GFF3 file" Tags="|file-formats|gff3|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3182" PostTypeId="1" CreationDate="2017-12-29T17:51:25.163" Score="1" ViewCount="13" Body="&lt;p&gt;I'm using the &amp;quot;Drug and Drug Target Mapping&amp;quot; search engine (&lt;a href=&quot;http://www.rcsb.org/pdb/ligand/drugMapping.do&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.rcsb.org/pdb/ligand/drugMapping.do&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;I've noticed that there are some drugs (generic names) that if they appear in the query, the search results end in &amp;quot;No records to view&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I've failed to find a pattern in identifying those drugs.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The site is no longer active.&lt;/p&gt;&#xA;&lt;p&gt;One replacement is the 'anatomical therapeutic chemical' (ATC) classification. Information is found on RCSB, PDB,&#xA;&lt;a href=&quot;https://www.rcsb.org/docs/search-and-browse/browse-options/atc&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/docs/search-and-browse/browse-options/atc&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;...and WHO, &lt;a href=&quot;https://www.whocc.no/atc/structure_and_principles/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.whocc.no/atc/structure_and_principles/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The question remains pertinent.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="3848" LastEditDate="2022-03-24T14:26:11.617" LastActivityDate="2022-03-24T14:26:11.617" Title="&quot;No records to view&quot; error in the &quot;Drug and Drug Target Mapping&quot; PDB's search engine" Tags="|proteins|protein-structure|pdb|protein-protein-interaction|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3184" PostTypeId="1" CreationDate="2017-12-30T12:06:41.967" Score="1" ViewCount="179" Body="&lt;p&gt;I have aligned two whole genomes of the same species in Mauve using progressiveMauve. One of the genomes is 2 megabases longer than the other so I know it must contain some insertions. Looking at the alignment in the Mauve GUI, it seems that insertions are small and scattered throughout the genome. I have exported the SNPs from the GUI. Is there a similar option for indels?&#xA;I have also inspected the Mauve file written after the alignment. It contains the two original sequences in FASTA format but I noticed in contains gaps denoted by a dash (-). Would it be possible to create a VCF from that file?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2017-12-30T12:06:41.967" Title="Can Mauve export indels or is there an whole-genome aligner that can export indels?" Tags="|sequence-alignment|genome|variant-calling|structural-variation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3185" PostTypeId="2" ParentId="2972" CreationDate="2017-12-30T12:44:23.910" Score="1" Body="&lt;p&gt;Don't use Wine at all! You seem to be running an Ubuntu installation inside a MacOS system using parallels. And then a Windows emulation layer (wine) on top of that. This is a very complex system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the software you are trying to install is written in Java and can run perfectly well on any of the operating systems you have available. So there is absolutely no reason to even try to run it via Wine. Just run it natively on either your MacOS or your Ubuntu. As explained in the page you linked to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Execute PGDSpider GUI:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;&lt;p&gt;Windows:&lt;br&gt;&#xA;  execute the file &lt;code&gt;PGDSpider2.exe&lt;/code&gt; to start the program&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;Linux:&lt;br&gt;&#xA;  execute the command &lt;code&gt;./PGDSpider2.sh&lt;/code&gt; to start the program&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;Mac and others:&lt;br&gt;&#xA;  execute the command &lt;code&gt;java -Xmx1024m -Xms512m -jar&#xA;  PGDSpider2.jar&lt;/code&gt; to start the program&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So ignore the Windows instructions and pick either one of the others.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2017-12-30T12:44:23.910" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3186" PostTypeId="2" ParentId="3181" CreationDate="2017-12-30T13:10:19.320" Score="3" Body="&lt;p&gt;Annoyingly, you can load such a GFF3 file into R (using the GenomicFeatures package) and access the UTRs, but they don't then get saved if you use the &lt;code&gt;export&lt;/code&gt; function from &lt;code&gt;rtracklayer&lt;/code&gt; (this would have been the simplest solution).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I don't know of anything to do this off-hand, I spent a couple minutes to &lt;a href=&quot;https://github.com/dpryan79/Answers/tree/master/bioinfoSE_3181&quot; rel=&quot;nofollow noreferrer&quot;&gt;write one&lt;/a&gt;. The usage would be &lt;code&gt;addGFF3UTRs.py input.gff output.gff&lt;/code&gt;. You can modify a number of the labels to appear as you'd like. This hasn't been extensively tested, but it seems to produce the correct results in a small test I just ran.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The code is hosted on GitHub at the link I gave above and I also include it here:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python&#xA;import argparse&#xA;&#xA;&#xA;def parseAttributes(kvps):&#xA;    d = dict()&#xA;    for kvp in kvps:&#xA;        k, v = kvp.split(&quot;=&quot;)&#xA;        d[k] = v&#xA;    return d&#xA;&#xA;&#xA;def parseGFF3(fname, topLevel=&quot;gene&quot;, secondLevel=&quot;mRNA&quot;, CDS=&quot;CDS&quot;, exon=&quot;exon&quot;):&#xA;    f = open(fname)&#xA;    genes = dict()&#xA;    transcripts = dict()&#xA;    g2t = dict()  # A dictionary with gene IDs associated to their transcript IDs&#xA;&#xA;    for line in f:&#xA;        if line.startswith(&quot;#&quot;):&#xA;            continue&#xA;        cols = line.strip().split(&quot;\t&quot;)&#xA;        attribs = parseAttributes(cols[8].split(&quot;;&quot;))&#xA;        ID = attribs[&quot;ID&quot;]&#xA;&#xA;        if cols[2] == topLevel:&#xA;            genes[ID] = line&#xA;            g2t[ID] = []&#xA;        elif cols[2] == secondLevel:&#xA;            parent = attribs[&quot;Parent&quot;]&#xA;            g2t[parent].append(ID)&#xA;&#xA;            # Generate a list of [line, [exon entries], [CDS entries]]&#xA;            if ID not in transcripts:&#xA;                transcripts[ID] = [line, [], []]&#xA;            else:&#xA;                transcripts[ID][0] = line&#xA;        else:&#xA;            parent = attribs[&quot;Parent&quot;]&#xA;            if parent not in transcripts:&#xA;                transcripts[parent] = [None, [], []]&#xA;            if cols[2] == exon:&#xA;                transcripts[parent][1].append((int(cols[3]), int(cols[4]), line))&#xA;            elif cols[2] == CDS:&#xA;                transcripts[parent][2].append((int(cols[3]), int(cols[4]), line))&#xA;    f.close()&#xA;&#xA;    return genes, transcripts, g2t&#xA;&#xA;&#xA;def findUTRs(transcripts, fivePrime = True):&#xA;    for k, t in transcripts.items():&#xA;        # If there are no CDS entries then skip&#xA;        if len(t[2]) == 0:&#xA;            continue&#xA;        # Get the strand (&quot;.&quot; will be treated as &quot;+&quot;)&#xA;        strand = t[0].split(&quot;\t&quot;)[6]&#xA;&#xA;        # For 3' UTR, just swap the strand&#xA;        if not fivePrime:&#xA;            strand = &quot;+&quot; if strand == &quot;-&quot; else &quot;-&quot;&#xA;&#xA;        exons = [(s, e) for s, e, _ in t[1]]&#xA;        CDSs = [(s, e) for s, e, _ in t[2]]&#xA;        exons.sort()&#xA;        CDSs.sort()&#xA;        UTRs = []&#xA;        if strand != &quot;-&quot;:&#xA;            final = CDSs[0][0]&#xA;            for s, e in exons:&#xA;                if e &amp;lt; final:&#xA;                    UTRs.append((s,e))&#xA;                elif s &amp;lt; final:&#xA;                    UTRs.append((s, final - 1))&#xA;                else:&#xA;                    break&#xA;        else:&#xA;            final = CDSs[-1][-1]&#xA;            for s, e in exons:&#xA;                if e &amp;lt; final:&#xA;                    continue&#xA;                elif s &amp;gt; final:&#xA;                    UTRs.append((s, e))&#xA;                else:&#xA;                    UTRs.append((final + 1, e))&#xA;        t.append(UTRs)&#xA;&#xA;&#xA;def saveGFF(oname, genes, transcripts, g2t, fivePrime, threePrime):&#xA;    o = open(oname, &quot;w&quot;)&#xA;    o.write(&quot;##gff-version 3\n&quot;)&#xA;    for geneID, geneLine in genes.items():&#xA;        o.write(geneLine)&#xA;        # Go through each transcript&#xA;        for transID in g2t[geneID]:&#xA;            t = transcripts[transID]&#xA;            if t[0] is None:&#xA;                continue&#xA;            o.write(t[0])&#xA;            cols = t[0].strip().split(&quot;\t&quot;)&#xA;&#xA;            # Write the exons, then CDS, then 5'UTR, then 3'UTR&#xA;            for exon in t[1]:&#xA;                o.write(exon[2])&#xA;            for CDS in t[2]:&#xA;                o.write(CDS[2])&#xA;            for idx, UTR in enumerate(t[3]):&#xA;                o.write(&quot;{}\t{}\t{}\t{}\t{}\t.\t{}\t.\t&quot;.format(cols[0], cols[1], fivePrime, UTR[0], UTR[1], cols[6]))&#xA;                o.write(&quot;ID={}.fivePrimeUTR{};Parent={}\n&quot;.format(transID, idx, transID))&#xA;            for idx, UTR in enumerate(t[4]):&#xA;                o.write(&quot;{}\t{}\t{}\t{}\t{}\t.\t{}\t.\t&quot;.format(cols[0], cols[1], threePrime, UTR[0], UTR[1], cols[6]))&#xA;                o.write(&quot;ID={}.threePrimeUTR{};Parent={}\n&quot;.format(transID, idx, transID))&#xA;    o.close()&#xA;&#xA;&#xA;parser = argparse.ArgumentParser(description=&quot;Parse a GFF3 file lacking UTR entries and add them in. Note that this program assumes a reasonably formatted GFF3 file&quot;)&#xA;parser.add_argument(&quot;--fiveUTRname&quot;, default=&quot;five_prime_UTR&quot;, help=&quot;The label for 5' UTR entries (default: %(default)s)&quot;)&#xA;parser.add_argument(&quot;--threeUTRname&quot;, default=&quot;three_prime_UTR&quot;, help=&quot;The label for 3' UTR entries (default: %(default)s)&quot;)&#xA;parser.add_argument(&quot;--topLevelID&quot;, default=&quot;gene&quot;, help=&quot;The 'type' designating the top-level entry (default: %(default)s)&quot;)&#xA;parser.add_argument(&quot;--secondLevelID&quot;, default=&quot;mRNA&quot;, help=&quot;The 'type' designating the second-level entry, typically something like 'mRNA' or 'transcript' (default: %(default)s)&quot;)&#xA;parser.add_argument(&quot;--exonID&quot;, default=&quot;exon&quot;, help=&quot;The 'type' designating exons (default: %(default)s)&quot;)&#xA;parser.add_argument(&quot;--CDSID&quot;, default=&quot;CDS&quot;, help=&quot;The 'type' designating CDS (default: %(default)s)&quot;)&#xA;parser.add_argument(&quot;input&quot;, metavar=&quot;input.gff3&quot;, help=&quot;Input file name&quot;)&#xA;parser.add_argument(&quot;output&quot;, metavar=&quot;output.gff3&quot;, help=&quot;Output file name&quot;)&#xA;args = parser.parse_args()&#xA;&#xA;genes, transcripts, g2t = parseGFF3(args.input, topLevel=args.topLevelID, secondLevel=args.secondLevelID, exon=args.exonID, CDS=args.CDSID)&#xA;&#xA;# Add the UTRs&#xA;findUTRs(transcripts)&#xA;findUTRs(transcripts, fivePrime=False)&#xA;&#xA;# Write the output&#xA;saveGFF(args.output, genes, transcripts, g2t, args.fiveUTRname, args.threeUTRname)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastEditorUserId="298" LastEditDate="2017-12-31T11:23:34.367" LastActivityDate="2017-12-31T11:23:34.367" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3187" PostTypeId="2" ParentId="2720" CreationDate="2017-12-31T14:16:03.513" Score="2" Body="&lt;p&gt;You can use &lt;a href=&quot;https://github.com/biologyguy/BuddySuite/wiki/AlignBuddy&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlignBuddy&lt;/a&gt; to process alignments in GenBank format, which allows you to keep the metadata. From the command line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$: alignbuddy input.gb --generate_alignment mafft | \&#xA;   alignbuddy --extract_feature_sequences 'PROT_FEAT_1' | \&#xA;   alignbuddy --percent_id&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want to do a large one-to-many comparison and loop through a bunch of features, I'd recommend importing AlignBuddy into a custom python script and calling the relevant functions directly. You can then collect the relevant data and build a histogram with &lt;a href=&quot;https://matplotlib.org/devdocs/api/_as_gen/matplotlib.pyplot.hist.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;matplotlib&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2046" LastActivityDate="2017-12-31T14:16:03.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3188" PostTypeId="1" AcceptedAnswerId="3199" CreationDate="2018-01-01T06:09:58.763" Score="2" ViewCount="154" Body="&lt;p&gt;I have a bunch of files (vcf format) each containing site and sample annotations of a copy number variant (following example is part of one of my files, I am showing 11 individuals but they are nearly 900 individuals)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;16  32328771    esv3638416;esv3638417;esv3638418;esv3638419;esv3638420  A   &amp;lt;CN0&amp;gt;,&amp;lt;CN2&amp;gt;,&amp;lt;CN3&amp;gt;,&amp;lt;CN4&amp;gt;,&amp;lt;CN5&amp;gt;   100 PASS    AC=27,84,3220,20,26;AF=0.00539137,0.0167732,0.642971,0.00399361,0.00519169;AN=5008;CS=DUP_gs;END=32370968;NS=2504;SVTYPE=CNV;DP=26381;EAS_AF=0.0099,0.0188,0.5427,0.004,0.0089;AMR_AF=0.0101,0.0058,0.4063,0,0.0014;AFR_AF=0.0015,0.0371,0.8374,0.003,0.003;EUR_AF=0.003,0.0089,0.5189,0.005,0.005;SAS_AF=0.0051,0.0031,0.7791,0.0072,0.0072;VT=SV  GT  3|3 3|1 0|3 3|3 3|3 3|0 0|3 3|3 0|0 3|3 1|3     &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to count each allele of CNV in each individual(sample) if there are 0, 1 or 2 copies!&#xA;this is my desired output (rows are individuals):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CN0 CN2 CN3 CN4 CN5&#xA;0   0   2   0   0&#xA;1   0   1   0   0&#xA;0   0   1   0   0&#xA;0   0   2   0   0&#xA;0   0   2   0   0&#xA;0   0   1   0   0&#xA;0   0   1   0   0&#xA;0   0   2   0   0&#xA;0   0   0   0   0&#xA;0   0   2   0   0&#xA;1   0   1   0   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any idea how to do that?&lt;/p&gt;&#xA;" OwnerUserId="1749" LastEditorUserId="1749" LastEditDate="2018-01-03T00:21:21.733" LastActivityDate="2018-01-03T10:28:41.517" Title="how to count the number of each allele of CNVs?" Tags="|python|perl|cnv|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3189" PostTypeId="2" ParentId="2321" CreationDate="2018-01-01T09:41:16.277" Score="1" Body="&lt;p&gt;This is something that Qiagen's &lt;a href=&quot;https://www.qiagen.com/us/shop/analytics-software/biological-data-tools/ingenuity-pathway-analysis/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ingenuity Pathway Analysis&lt;/a&gt; is meant to solve. Unfortunately, the process of solving pathway dependency problems like these is difficult, and frequently leads to the discovery of super-linked proteins that are seen everywhere (and consequently fairly uninteresting from a biological perspective). The challenge is in filtering out the useless noise of &quot;statistical significance&quot;, and identifying the biologically-significant signal. That's not something that computers are particularly good at doing at the moment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My own suggestion is that you would be better off finding someone who has domain knowledge in the area of secreted proteins, and trying to present results to them in such a way that it makes it easy to interpret the results and ask further questions. If you are that person, perhaps there's a more specific question that you could ask to further investigate the problem that you have.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-01-01T09:41:16.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3190" PostTypeId="2" ParentId="2623" CreationDate="2018-01-02T00:38:24.730" Score="2" Body="&lt;p&gt;Dan is correct but it doesn't answer the question.  After much thought, I have finally figured out how to do it, only thanks to Dan's answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that a subsequence is one-dimensional and does not therefore include any of the additional information which is extractable from the three-dimensional structure (when available).  Obviously, having the three-dimensional information is much more informative and descriptive for building classifiers, predicting interactions, interaction stability, or other purposes.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Find all intramolecular contacts within a radius of N. for the Amino Acid subsequence of interest (such as an Interface).  I set N to, e.g. 4 Angstroms.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate the descriptive statistics based on the set of hydrophobicity values for all intramolecular contact atoms or amino acids (depending on the granularity).  e.g. Make an average of all of the hydrophobicity scores from the selected scale.  (I use GRAVY, although there are many others).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;To improve this further, the distance value of N can be set more intelligently than as a static value.   For example, to the same as the cavity radius.&lt;/p&gt;&#xA;" OwnerUserId="1606" LastActivityDate="2018-01-02T00:38:24.730" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3191" PostTypeId="2" ParentId="3040" CreationDate="2018-01-02T08:57:49.917" Score="2" Body="&lt;p&gt;It looks like the first command is calculating pairwise linkage disequilibrium r2 between your SNPs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second command is clumping your dataset based on a p-value over a file of association test results, which I guess is a pre-cursor to computing polygenic risk scores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clumping is the process of thinning your SNP list to only keep uncorrelated SNPs so that each representative region has only one marker. In this case you are asking PLINK to keep the marker with the highest p-value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure what your definition of proxy SNPs is but if you are looking for tag SNPs to a SNP of your interest, you should go with command 1 and choose a proper LD r2 threshold.&lt;/p&gt;&#xA;" OwnerUserId="601" LastActivityDate="2018-01-02T08:57:49.917" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3192" PostTypeId="1" AcceptedAnswerId="3193" CreationDate="2018-01-02T14:32:53.690" Score="4" ViewCount="88" Body="&lt;p&gt;I have the following sets of data:&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;file1:&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   15776220    15776240    GTGACCAGCAGGTGTCTCTG    16855676    16855696    CTGTCCAGCAGAGGGCGGTG &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;file2:&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   15776231    2   5008    G:5002  A:6 &#xA;1   15776239    3   5008    C:3358  A:14    G:1636&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;file3:&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   16855677    2   5008    A:5003  C:5&#xA;1   16855689    3   5008    G:4957  A:41    T:10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Most lines have 6 columns but a few lines also have 7 columns which have data that I want to preserve. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can a script be written which does the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if &lt;code&gt;$2&lt;/code&gt; of file2 comes in between &lt;code&gt;$2&lt;/code&gt; and &lt;code&gt;$3&lt;/code&gt; of file1 (between the interval of &lt;code&gt;$2&lt;/code&gt; and &lt;code&gt;$3&lt;/code&gt;) and &lt;code&gt;$2&lt;/code&gt; of file3 comes in between &lt;code&gt;$5&lt;/code&gt; and &lt;code&gt;$6&lt;/code&gt; of file1 (between the interval of &lt;code&gt;$5&lt;/code&gt; and &lt;code&gt;$6&lt;/code&gt;), then the output would be &lt;code&gt;$1,$2,$3,$4&lt;/code&gt; from file1 and &lt;code&gt;$2,$3,$4,$5,$6,$7&lt;/code&gt; from file2 and &lt;code&gt;$5,$6,$7&lt;/code&gt; from file1 and &lt;code&gt;$2,$3,$4,$5,$6,$7&lt;/code&gt; from file 3, all in one line. So something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   15776220    15776240    GTGACCAGCAGGTGTCTCTG    15776231    2   5008    G:5002  A:6 16855676    16855696    CTGTCCAGCAGAGGGCGGTG    16855677    2   5008    A:5003  C:5&#xA;&#xA;1   15776220    15776240    GTGACCAGCAGGTGTCTCTG    15776239    3   5008    C:3358  A:14    G:1636  16855676    16855696    CTGTCCAGCAGAGGGCGGTG    1   16855689    3   5008    G:4957  A:41    T:10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Something like bedtools does but I am not able to use bedtools one this since I don't have two columns in file2 and file3. The intersection is based on $2  in both the files, and all the other columns after it are just copied &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(There could be many lines from file2 and file3 intersecting in the same line in file1,so that line could be repeated based on the number of intersections)&lt;/p&gt;&#xA;" OwnerUserId="2094" LastActivityDate="2018-01-02T16:39:37.110" Title="Intersecting two different files with one &quot;master&quot; file based on different columns" Tags="|vcf|bioawk|text-processing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3193" PostTypeId="2" ParentId="3192" CreationDate="2018-01-02T16:39:37.110" Score="4" Body="&lt;p&gt;An awk solution&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '&#xA;    FILENAME == ARGV[1] {f2[$2] = $2 OFS $3 OFS $4 OFS $5 OFS $6 OFS $7; next}&#xA;    FILENAME == ARGV[2] {f3[$2] = $2 OFS $3 OFS $4 OFS $5 OFS $6 OFS $7; next}&#xA;    {&#xA;        for (key2 in f2) {&#xA;            for (key3 in f3) {&#xA;                if (($2 &amp;lt;= key2 &amp;amp;&amp;amp; key2 &amp;lt;= $3) &amp;amp;&amp;amp; ($5 &amp;lt;= key3 &amp;amp;&amp;amp; key3 &amp;lt;= $6)) {&#xA;                    print $1,$2,$3,$4, f2[key2], $5,$6,$7, f3[key3]&#xA;                }&#xA;            }&#xA;        }&#xA;    }&#xA;' file2 file3 file1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;outputs&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1 15776220 15776240 GTGACCAGCAGGTGTCTCTG 15776231 2 5008 G:5002 A:6  16855676 16855696 CTGTCCAGCAGAGGGCGGTG 16855677 2 5008 A:5003 C:5&#xA;1 15776220 15776240 GTGACCAGCAGGTGTCTCTG 15776231 2 5008 G:5002 A:6  16855676 16855696 CTGTCCAGCAGAGGGCGGTG 16855689 3 5008 G:4957 A:41 T:10&#xA;1 15776220 15776240 GTGACCAGCAGGTGTCTCTG 15776239 3 5008 C:3358 A:14 G:1636 16855676 16855696 CTGTCCAGCAGAGGGCGGTG 16855677 2 5008 A:5003 C:5&#xA;1 15776220 15776240 GTGACCAGCAGGTGTCTCTG 15776239 3 5008 C:3358 A:14 G:1636 16855676 16855696 CTGTCCAGCAGAGGGCGGTG 16855689 3 5008 G:4957 A:41 T:10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For your sample data, all combinations of records in file2 and file3 meet the criteria for the file1 record.&lt;/p&gt;&#xA;" OwnerUserId="2131" LastActivityDate="2018-01-02T16:39:37.110" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3194" PostTypeId="2" ParentId="3170" CreationDate="2018-01-02T20:57:57.067" Score="3" Body="&lt;p&gt;In an ideal world, ribosomal RNA (as seen in your top hit) should be excluded from samples &lt;em&gt;prior&lt;/em&gt; to sequencing. Where this is not possible (i.e. in the data that have been presented to you), it would be a good idea to exclude ribosomal genes (and any other common contaminants) prior to doing further analysis (including normalisation). I believe this is the point the course instructors are trying to get across with that message box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eyeballing the results, you should also exclude any ERCC genes as well, as they are synthetic constructs and not from the target samples (although record their counts, as it may be informative for QC/normalisation purposes); they'll have odd distributions because of this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ERCC sequences are typically added based on the concentration of DNA in a sample, rather than the number of cells. However with single-cell data (i.e. with known cell counts), it'd be possible to do a proper ERCC-based normalisation, assuming an equal amount were added for each cell. I suspect that knowing a bit more about the ERCC addition protocol would help in understanding why their distribution is so different.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-01-02T23:19:43.323" LastActivityDate="2018-01-02T23:19:43.323" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3195" PostTypeId="5" CreationDate="2018-01-02T21:00:39.053" Score="0" Body="" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-01-02T21:00:39.053" LastActivityDate="2018-01-02T21:00:39.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3196" PostTypeId="4" CreationDate="2018-01-02T21:00:39.053" Score="0" Body="Use for questions that are associated with canned problems, where either the solution is typically known in advance, or data are simulated to create a specific situation." OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-01-02T21:00:39.053" LastActivityDate="2018-01-02T21:00:39.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3197" PostTypeId="1" AcceptedAnswerId="3198" CreationDate="2018-01-02T21:49:02.993" Score="1" ViewCount="6037" Body="&lt;p&gt;I want to filter out ribosomal RNA from scRNA-seq data (downloaded from &lt;a href=&quot;https://figshare.com/projects/Tabula_Muris_Transcriptomic_characterization_of_20_organs_and_tissues_from_Mus_musculus_at_single_cell_resolution/27733&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). Is there a list of known ribosomal RNA?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only solution I found is &lt;a href=&quot;http://bioinfo.lifl.fr/RNA/sortmerna/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SortMeRNA&lt;/a&gt;, however it works with raw sequencing data afaik, while I already have a matrix with transcript counts for each cell. I searched for a comprehensive list of rRNAs but I didn't find any.&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="298" LastEditDate="2018-01-03T12:58:56.233" LastActivityDate="2018-01-05T09:27:50.420" Title="How to filter ribosomal RNA from scRNA-seq data" Tags="|rna-seq|scrnaseq|rna|ribosomal|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3198" PostTypeId="2" ParentId="3197" CreationDate="2018-01-02T22:42:57.440" Score="3" Body="&lt;p&gt;The rRNA genes in that dataset are &lt;code&gt;Rn45s&lt;/code&gt; and &lt;code&gt;Rn4.5s&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, you have gene counts, not transcript counts.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-02T22:42:57.440" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3199" PostTypeId="2" ParentId="3188" CreationDate="2018-01-03T10:28:41.517" Score="1" Body="&lt;p&gt;Here's a pure python solution to process your example. It should be straightforward to loop over all the rows of your vcf files&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;my_line='16 32328771    esv3638416;esv3638417;esv3638418;esv3638419;esv3638420  A   &amp;lt;CN0&amp;gt;,&amp;lt;CN2&amp;gt;,&amp;lt;CN3&amp;gt;,&amp;lt;CN4&amp;gt;,&amp;lt;CN5&amp;gt;   100 PASS    AC=27,84,3220,20,26;AF=0.00539137,0.0167732,0.642971,0.00399361,0.00519169;AN=5008;CS=DUP_gs;END=32370968;NS=2504;SVTYPE=CNV;DP=26381;EAS_AF=0.0099,0.0188,0.5427,0.004,0.0089;AMR_AF=0.0101,0.0058,0.4063,0,0.0014;AFR_AF=0.0015,0.0371,0.8374,0.003,0.003;EUR_AF=0.003,0.0089,0.5189,0.005,0.005;SAS_AF=0.0051,0.0031,0.7791,0.0072,0.0072;VT=SV  GT  3|3 3|1 0|3 3|3 3|3 3|0 0|3 3|3 0|0 3|3 1|3 '&#xA;my_line=my_line.strip() # remove trailing whitespace&#xA;my_fields=my_line.split()&#xA;names=my_fields[4].split(',') # extract CNV names&#xA;names=[x[1:-1] for x in names] # remove '&amp;lt;&amp;gt;' symbols&#xA;print('\t'.join(names))&#xA;for sample in my_fields[9:]:&#xA;    genotypes=[0]*(len(names)+1) # initialize array of zero counts, including reference  &#xA;    for allele in sample.split('|'):&#xA;        genotypes[int(allele)]+=1&#xA;    print('\t'.join([str(x) for x in genotypes[1:]])) # print all columns except reference&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-01-03T10:28:41.517" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3200" PostTypeId="2" ParentId="3143" CreationDate="2018-01-03T13:22:03.760" Score="2" Body="&lt;p&gt;I have reported the &lt;a href=&quot;https://support.bioconductor.org/p/104488/&quot; rel=&quot;nofollow noreferrer&quot;&gt;error on support.bioconductor.org&lt;/a&gt; but we were not able to solve the problem. It seems that some dependencies are making problems, but as reinstalling my home R folder did not resolve the issue, I have come up with the following workaround:&lt;/p&gt;&#xA;&lt;p&gt;I downloaded the latest R version from my favorite CRAN mirror, opened an R session with &lt;code&gt;pathToNewR/bin/R&lt;/code&gt; from my terminal and installed &lt;code&gt;RnBeads&lt;/code&gt; there. This works fine.&lt;/p&gt;&#xA;&lt;p&gt;Note: I had to make sure that &lt;code&gt;pathToNewR/library&lt;/code&gt; is the only entry in &lt;code&gt;.libPaths()&lt;/code&gt;, as per default, my home R folder was also contained in &lt;code&gt;.libPaths()&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As a consequence, I am independent from the (maybe broken) R version and packages on my server and can finally work with &lt;code&gt;RnBeads&lt;/code&gt; :)&lt;/p&gt;&#xA;" OwnerUserId="2047" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-01-03T13:22:03.760" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3201" PostTypeId="1" CreationDate="2018-01-03T15:25:18.933" Score="5" ViewCount="1364" Body="&lt;p&gt;I have been looking around (including read the original papers) to understand what is essentially the difference between StringTie in non-reference based mode (de novo) and Trinity de novo assembly. I understand that in the genome-guided nature of StringTie, we are supported by the genome information (i.e. taking into account the mapped reads). In addition to that, we also have an option to either run using the gene annotation file or not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simply put, am I correct if I say that Trinity is purely looking at the overlapping reads, then assemble them, whereas StringTie (non-reference based) is looking at the mapped reads that are either close or overlapping to each other? Is there any other thing I'm missing here (I understand it's not that simple, but I'm trying to make it as simple/intuitive as possible)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also if that's the case, would it be safe to say that generally non-reference based StringTie is preferred over Trinity and reference based StringTie (as gene annotation can lead to some &quot;bias&quot;, hence preventing discovery of novel TSS, exon length or even make unnecessary duplicates with small differences)?&lt;/p&gt;&#xA;" OwnerUserId="1761" LastActivityDate="2018-01-03T16:06:41.087" Title="Difference between de novo transcriptome assembly methods" Tags="|rna-seq|sequence-annotation|assembly|transcriptome|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3203" PostTypeId="2" ParentId="3201" CreationDate="2018-01-03T15:53:28.827" Score="3" Body="&lt;p&gt;Your intuition is correct. stringTie is just looking at clumps of alignments and how they might relate to each other (either due to spliced alignments or proximity). Trinity is doing the more computationally difficult task of finding parts of reads that overlap with each other and trying to link them together into longer sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whether it makes sense to use something like stringTie or not depends mostly on the quality of the reference sequence. If you're working on an organism with a high-quality reference sequence (pretty much anything with a lot of papers using it) then there's no reason to bother with Trinity. If your reference genome additionally has a high-quality annotation (human, mouse, etc.) then you should use it, as anything &quot;novel&quot; that the program is likely to output otherwise is probably just an artefact. If you either don't have an annotation or it's known to be highly problematic then it makes sense to omit it when using stringTie.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without seeing evidence to the contrary, I don't buy the argument that using a good annotation is actually biasing the output in a problematic way. For well-studied organisms, anything reported as &quot;novel&quot; these days likely doesn't exist, or is otherwise so fleeting that it's biologically irrelevant. The arguments of bias sound like something that was come up with once someone didn't get as many &quot;novel&quot; results as they were hoping.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-03T15:53:28.827" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3204" PostTypeId="2" ParentId="3201" CreationDate="2018-01-03T16:06:41.087" Score="3" Body="&lt;p&gt;It's really a misnomer to call StringTie's non-reference based mode 'de-novo.' It's still using the reference genome sequence to guide the transcript assembly, it's just not using the reference annotation. Trinity is truly de-novo in that it it assembles the transcript from the overlap of the reads without mapping them to a reference genome sequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at the precision numbers from the StringTie &lt;a href=&quot;https://www.nature.com/articles/nbt.3122&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, you'll see that it maxes out around 80% under ideal circumstances with simulated data. This means that if you're working with a well annotated genome where most transcripts have already been discovered, the vast majority of the 'novel' transcripts you find will likely turn out to be artifacts. &lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-01-03T16:06:41.087" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3205" PostTypeId="1" AcceptedAnswerId="3207" CreationDate="2018-01-03T17:17:30.047" Score="3" ViewCount="5374" Body="&lt;p&gt;I am using data from the &lt;a href=&quot;https://figshare.com/projects/Tabula_Muris_Transcriptomic_characterization_of_20_organs_and_tissues_from_Mus_musculus_at_single_cell_resolution/27733&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tabula Muris Consortium&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are gene counts from scRNA-seq of mouse cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some specific genes with name ending with suffix 'Rik' (e.g. 0610005C13Rik or 0610007C21Rik). Searching online I found that they are from &lt;a href=&quot;http://www.sciencedirect.com/science/article/pii/S1631069103002166?via%3Dihub&quot; rel=&quot;nofollow noreferrer&quot;&gt;RIKEN project&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, they are different in some respects (e.g. antisense lncRNA gene, or mouse 0610007C21RIK Adenovirus).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to exclude them from downstream analysis, but it is unclear to me the main purpose of this collection. For which kind of analysis are they used?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="73" LastEditDate="2018-01-03T18:56:44.163" LastActivityDate="2018-06-21T06:27:39.287" Title="What are RIKEN genes?" Tags="|scrnaseq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3206" PostTypeId="1" CreationDate="2018-01-03T18:59:29.750" Score="3" ViewCount="366" Body="&lt;p&gt;I need to implement a parallel version of an R script because on one core it takes forever to execute. The function that takes the most of time is &lt;code&gt;scran::quickCluster()&lt;/code&gt; applied to an integer sparse matrix (&lt;code&gt;UMI_count&lt;/code&gt;) that is &lt;code&gt;26000x66000&lt;/code&gt;. The code is presented below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(scran)&#xA;library(scater)&#xA;sce &amp;lt;- newSCESet(countData=UMI_count)&#xA;clusters &amp;lt;- quickCluster(sce)&#xA;sce &amp;lt;- computeSumFactors(sce, cluster=clusters)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I did not find much on parallel versions of &lt;code&gt;quickCluster()&lt;/code&gt; function, so will it actually execute in parallel if I set the number of cores to more than one? I am trying to do normalization of heterogeneous dataset with multiple cell types, that is why I am using &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/scran/inst/doc/scran.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;quickCluster()&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I just need to speed it up to be able to compute it.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2018-09-13T09:20:11.830" LastActivityDate="2020-01-20T21:18:30.180" Title="Normalization using parallel R script" Tags="|r|scran|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3207" PostTypeId="2" ParentId="3205" CreationDate="2018-01-03T19:08:28.117" Score="3" Body="&lt;p&gt;My understanding is that these are cDNA sequences from the RIKEN project that don't yet have another gene name. The project carried out a comprehensive sequencing of the mouse genome, and as a result found a lot of cDNA transcripts that were previously unknown. Now they have some annotation (which can be varying degrees of functional annotation), and an accepted official identifier to allow people to talk about these transcripts when discovered in other studies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It'd be interesting to see if any novel transcripts have been found from the Tabula Muris project. They would also need their own identifier, possibly leading to similar intrigue about their history.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-01-03T19:08:28.117" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3208" PostTypeId="1" CreationDate="2018-01-03T20:22:15.023" Score="6" ViewCount="1322" Body="&lt;p&gt;I am preprocessing scRNA-seq data. What is the best practice in use to run both ComBat for batch effects removal, data imputation (to mitigate dropout) and library size normalization?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought that library size should be run first, since it is per-cell normalization, then ComBat batch effects removal. On the original paper - &lt;a href=&quot;https://academic.oup.com/biostatistics/article/8/1/118/252073&quot; rel=&quot;nofollow noreferrer&quot;&gt;Johnson et al. (2007)&lt;/a&gt; - it is stated that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We assume that the data have been normalized and expression values have been estimated for all genes and samples.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;However, I want to apply it to scRNA-seq data. Does this statement still hold? Additionally, I plan to apply imputation (e.g. with &lt;a href=&quot;https://www.biorxiv.org/content/early/2017/02/25/111591&quot; rel=&quot;nofollow noreferrer&quot;&gt;MAGIC&lt;/a&gt;) in the end. Is there any problem you can spot?&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Update&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I attach the PCA regarding an example Mus Musculus dataset in which different colors represent different mice. It seems clear to me that the first two principal components are affected by batches (mouse id).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TeZGZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TeZGZ.png&quot; alt=&quot;pca&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Update 2&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I rerun the PCA on raw counts data (the first PCA was on log-transformed data) and I obtain a different description of the dataset, in which batch effects seem not to be prevalent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OfSUZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OfSUZ.png&quot; alt=&quot;pca_raw&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2018-01-10T15:48:22.630" LastActivityDate="2018-10-21T18:35:04.123" Title="Order of batch effects removal, data imputation and library size normalization in scRNA-seq data" Tags="|normalization|scrnaseq|imputation|combat|batch-effects|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="3209" PostTypeId="1" CreationDate="2018-01-03T21:33:51.167" Score="3" ViewCount="79" Body="&lt;p&gt;As the question states, I am interested in an analysis similar to Gene Set Enrichment Analysis (ranked gene sets) but focused on locus-level data instead of genes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To explain in greater detail: I have a set of genomic coordinates from DNA methylation data that are scored based on their contribution to components from a non-negative matrix factorization. I am interested in better understanding the biological underpinnings of what these NMF components represent.  I know I can use Locus Overlap Analysis tools like GREAT or LOLA to tell me what GO/Functions are enriched in a particular BED file, but this type of analysis doesn't take into account the scores/rank of the loci in the BED file as would be done in an analysis akin to GSEA. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know of any tools/methods for analyzing functional enrichment while taking advantage of feature/locus ranks/scores with locus-level data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know I probably didn't explain this very clearly, so please let me know if I can clarify anything for you.  Thanks&lt;/p&gt;&#xA;" OwnerUserId="39" LastActivityDate="2018-01-07T15:24:08.747" Title="Is there anything similar to GSEA for locus-based (instead of of gene-based) data?" Tags="|gsea|go-enrichment|functional-annotation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3210" PostTypeId="2" ParentId="3209" CreationDate="2018-01-03T23:13:31.910" Score="2" Body="&lt;p&gt;I tinkered with a program &lt;a href=&quot;https://www.nr.no/en/projects/software-genomics&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSEA-SNP&lt;/a&gt; quite a few years back, which claims that it does a similar ranking procedure with SNPs. It carries out its procedure by first linking SNPs with genes, then running an algorithm similar to GSEA. There's a bit more detail in the &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/24/23/2784/180339&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately this is a space I've got a bit further away from in recent years, so I don't know if anyone has carried out the legwork to generalise the algorithm for sets of any type where each item within each set fits a continuous (but not necessarily normal) distribution.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-01-03T23:18:48.097" LastActivityDate="2018-01-03T23:18:48.097" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3211" PostTypeId="1" CreationDate="2018-01-03T23:55:56.927" Score="12" ViewCount="3899" Body="&lt;p&gt;Doublets are a known problem with scRNA-seq experiments, where 2 or more cells are sometimes captured instead. To determine their presence, there are studies that mix multiple species (such as human and mouse) or distinct cell types and then count how many cells contain genes from the very different transcriptomes. In those experiments, you can pick any cell and determine if it's a doublet based on a few key genes. In a typical single-cell experiment on one sample, you don't have the luxury of knowing what cells to expect and if a particular cell looks abnormal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a large fraction of doublets, you should be able to see a double peak if you plot the number of genes/transcripts per cell. However, if you have something like 1% doublets, you probably wouldn't notice anything. It seems that most people simply filter out top X% cells based on the number of genes/transcripts. How do you know what X should be? Can you actually estimate the fraction of doublets in a given experiment?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found a nice figure from &lt;a href=&quot;https://www.biorxiv.org/content/early/2017/12/21/237693&quot; rel=&quot;noreferrer&quot;&gt;Stoeckius et al.&lt;/a&gt; to illustrate the issue:&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/BRnAd.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BRnAd.png&quot; alt=&quot;Distribution of RNA UMIs per cell barcode&quot;&gt;&lt;/a&gt;&lt;br&gt;&#xA;Although the number of transcripts/UMIs seems to be roughly double for doublets/multiplets (median of ~1000 vs ~500), there is still high overlap.&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="35" LastEditDate="2018-05-31T16:33:30.950" LastActivityDate="2018-06-02T14:11:01.300" Title="determining doublets in single-cell RNA-seq" Tags="|rna-seq|scrnaseq|single-cell|quality-control|10x-genomics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3212" PostTypeId="1" AcceptedAnswerId="3214" CreationDate="2018-01-04T03:28:07.553" Score="4" ViewCount="127" Body="&lt;p&gt;Below is an example of a simple GFF3 file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   T1  gene    3631    4605    .   +   .   ID=ATNG01010&#xA;1   T1  mRNA    3631    4605    .   +   .   ID=ATNG01010.1;Parent=ATNG01010&#xA;1   T1  exon    3631    3913    .   +   .   ID=ATNG01010:exon:1;Parent=ATNG01010.1&#xA;1   T1  CDS 3860    3913    .   +   0   ID=ATNG01010:CDS:1;Parent=ATNG01010.1&#xA;1   T1  exon    3996    4276    .   +   .   ID=ATNG01010:exon:2;Parent=ATNG01010.1&#xA;1   T1  CDS 3996    4260    .   +   2   ID=ATNG01010:CDS:2;Parent=ATNG01010.1&#xA;1   T1  exon    4486    4605    .   +   .   ID=ATNG01010:exon:3;Parent=ATNG01010.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question is: if we found another coding sequence (encoding a different protein) range from 3752 to 3904, how should the GFF3 be formatted? It seems to me that the GFF3 file can only allow one protein-coding gene per mRNA. If not, could anyone show me one example? Thank you!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Edit]:&#xA;Sorry, using &quot;CDS&quot; is misleading. I changed the title and contents to &quot;contain more than one protein-coding genes&quot; instead of &quot;more than one CDS&quot;. I also change the starting position of the second protein to 3752 (was 3750) to reflect the fact. My sincere apologies. &lt;/p&gt;&#xA;" OwnerUserId="2118" LastEditorUserId="96" LastEditDate="2019-02-28T19:31:13.943" LastActivityDate="2019-02-28T19:31:13.943" Title="In GFF3, annotating more than one protein-coding gene (i.e. polycistronic) contained in a eukaryotic mRNA" Tags="|sequence-annotation|gff3|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3213" PostTypeId="1" CreationDate="2018-01-04T06:11:03.873" Score="4" ViewCount="84" Body="&lt;p&gt;My question is about ROC curves used in CAFA2 experiment. &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-1037-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;In this paper&lt;/a&gt; they used the ROC analysis for the term-centric evaluation. In order to &#xA;perform ROC curve analysis we should have a continuous variable and a classifier (categorical) variable. In CAFA2 experiment, &#xA;which variables did they consider as a categorical and continuous? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="1978" LastActivityDate="2018-08-04T01:00:39.530" Title="ROC analysis, CAFA 2 experiment" Tags="|proteins|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3214" PostTypeId="2" ParentId="3212" CreationDate="2018-01-04T06:35:35.630" Score="5" Body="&lt;p&gt;The &lt;a href=&quot;https://github.com/The-Sequence-Ontology/Specifications/blob/master/gff3.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;GFF3 file format specification&lt;/a&gt; doesn't care where annotations are described in the file, although it can make some things easier if the annotations are ordered by the start position:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   T1  gene    3631    4605    .   +   .   ID=ATNG01010&#xA;1   T1  mRNA    3631    4605    .   +   .   ID=ATNG01010.1;Parent=ATNG01010&#xA;1   T1  exon    3631    3913    .   +   .   ID=ATNG01010:exon:1;Parent=ATNG01010.1&#xA;1   T1  CDS     3750    3904    .   -   0   ID=ATRV01011:CDS:1;Parent=ATRV01011.0&#xA;1   T1  CDS     3860    3913    .   +   0   ID=ATNG01010:CDS:1;Parent=ATNG01010.1&#xA;1   T1  exon    3996    4276    .   +   .   ID=ATNG01010:exon:2;Parent=ATNG01010.1&#xA;1   T1  CDS     3996    4260    .   +   2   ID=ATNG01010:CDS:2;Parent=ATNG01010.1&#xA;1   T1  exon    4486    4605    .   +   .   ID=ATNG01010:exon:3;Parent=ATNG01010.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are many examples in well-studied genomes of overlapping genes, so the GFF3 format needs to be flexible enough to handle these. Different genes can be encoded across the same region, but on different strands. But different genes can also be encoded on the same strand, sometimes sharing an exon.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-01-04T19:52:25.757" LastActivityDate="2018-01-04T19:52:25.757" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3215" PostTypeId="2" ParentId="3212" CreationDate="2018-01-04T08:30:13.557" Score="1" Body="&lt;p&gt;If you found another CDS from 3750 to 3904 then it would necessarily be on a different transcript, given that you already have a CDS from 3860 to 3913 and having overlapping CDSs on the same transcript doesn't make much biological sense (though I'm sure there's some weird organism out there that has a gene like this). So the likely GFF3 file would look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   T1  gene    3631    4605    .   +   .   ID=ATNG01010&#xA;1   T1  mRNA    3631    4605    .   +   .   ID=ATNG01010.1;Parent=ATNG01010&#xA;1   T1  exon    3631    3913    .   +   .   ID=ATNG01010:exon:1;Parent=ATNG01010.1&#xA;1   T1  CDS     3860    3913    .   +   0   ID=ATNG01010:CDS:1;Parent=ATNG01010.1&#xA;1   T1  exon    3996    4276    .   +   .   ID=ATNG01010:exon:2;Parent=ATNG01010.1&#xA;1   T1  CDS     3996    4260    .   +   2   ID=ATNG01010:CDS:2;Parent=ATNG01010.1&#xA;1   T1  exon    4486    4605    .   +   .   ID=ATNG01010:exon:3;Parent=ATNG01010.1&#xA;1   T1  mRNA    3631    4605    .   +   .   ID=ATNG01010.2;Parent=ATNG01010&#xA;1   T1  exon    3631    3913    .   +   .   ID=ATNG01010.2:exon:1;Parent=ATNG01010.2&#xA;1   T1  CDS     3759    3904    .   +   0   ID=ATNG01010.2:CDS:1;Parent=ATNG01010.2&#xA;1   T1  exon    3996    4276    .   +   .   ID=ATNG01010.2:exon:2;Parent=ATNG01010.2&#xA;1   T1  CDS     3996    4260    .   +   2   ID=ATNG01010.2:CDS:2;Parent=ATNG01010.2&#xA;1   T1  exon    4486    4605    .   +   .   ID=ATNG01010.2:exon:3;Parent=ATNG01010.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that I've just added another transcript, since that's likely what would be biologically going on.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-04T08:30:13.557" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3216" PostTypeId="2" ParentId="3213" CreationDate="2018-01-04T08:37:39.840" Score="2" Body="&lt;p&gt;The paper doesn't seem to have been written for general readers. Unfortunately I don't understand the equations enough to be able to explain what the categorical and continuous variables are. Just in case it helps, here are the equations from the paper showing the precision/recall calculations:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$\begin{array}{@{}rcl@{}} \text{pr}(\tau) &amp;amp;=&amp;amp; \frac{1}{m(\tau)}\sum\limits_{i=1}^{m(\tau)} \frac{{\sum\nolimits}_{f} \unicode{x1D7D9}\left(f \in P_{i}(\tau) \wedge f \in T_{i}\right)}{\sum_{f} \unicode{x1D7D9}\left(f \in P_{i}(\tau) \right)},\\ \text{rc}(\tau) &amp;amp;=&amp;amp; \frac{1}{n_{e}}\sum\limits_{i=1}^{n_{e}} \frac{{\sum\nolimits}_{f} \unicode{x1D7D9}\left(f \in {P}_{i}(\tau) \wedge f \in T_{i}\right)}{{\sum\nolimits}_{f} \unicode{x1D7D9}\left(f \in T_{i} \right)}, \\ F_{\max} &amp;amp;=&amp;amp; \max_{\tau} \left\{ \frac{2\cdot \text{pr}(\tau)\cdot \text{rc}(\tau)}{\text{pr}(\tau) + \text{rc}(\tau)} \right\}, \end{array} $$&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;where P i (τ) denotes the set of terms that have predicted scores greater than or equal to τ for a protein sequence i, T i denotes the corresponding ground-truth set of terms for that sequence, m(τ) is the number of sequences with at least one predicted score greater than or equal to τ, $\unicode{x1D7D9}\left (\cdot \right)$ is an indicator function, and n e is the number of targets used in a particular mode of evaluation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-01-04T08:37:39.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3217" PostTypeId="1" AcceptedAnswerId="3218" CreationDate="2018-01-04T11:40:12.733" Score="5" ViewCount="320" Body="&lt;p&gt;On this &lt;a href=&quot;http://biopython.org/wiki/AlignIO&quot; rel=&quot;noreferrer&quot;&gt;Biopython tutorial&lt;/a&gt;, they describe how to import a multiple sequence alignment in the Mauve (XMFA: extensible multi fasta format). So I imported the AlignIO module:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import AlignIO&#xA;alignment = AlignIO.read(open(&quot;alignment.xmfa&quot;), &quot;mauve&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ValueError&#xA;Traceback (most recent call last) &amp;lt;ipython-input-30-396e54791d7a&amp;gt; in &amp;lt;module&amp;gt;()&#xA;----&amp;gt; 1 alignment = AlignIO.read(open(&quot;/Users/cr517/Documents/alignment.xmfa&quot;),&#xA;&quot;mauve&quot;)&#xA;&#xA;/Users/cr517/anaconda/lib/python3.6/site-packages/Bio/AlignIO/__init__.py&#xA;in read(handle, format, seq_count, alphabet)&#xA;    422     iterator = parse(handle, format, seq_count, alphabet)&#xA;    423     try:&#xA;--&amp;gt; 424         first = next(iterator)&#xA;    425     except StopIteration:&#xA;    426         first = None&#xA;&#xA;/Users/cr517/anaconda/lib/python3.6/site-packages/Bio/AlignIO/__init__.py&#xA;in parse(handle, format, seq_count, alphabet)&#xA;    366                                                 seq_count=seq_count)&#xA;    367         else:&#xA;--&amp;gt; 368             raise ValueError(&quot;Unknown format '%s'&quot; % format)&#xA;    369 &#xA;    370         # This imposes some overhead... wait until we drop Python 2.4 to fix it&#xA;&#xA;ValueError: Unknown format 'mauve'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="77" LastEditDate="2018-01-04T11:43:35.707" LastActivityDate="2018-01-04T11:53:57.620" Title="Why doesn't Biopython AlignIO.read() recognise the 'mauve' format?" Tags="|sequence-alignment|python|file-formats|errors|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3218" PostTypeId="2" ParentId="3217" CreationDate="2018-01-04T11:53:57.620" Score="5" Body="&lt;p&gt;You're using version 1.68 or older. Mauve support was &lt;a href=&quot;https://github.com/biopython/biopython/blob/master/NEWS.rst#10-july-2017-biopython-170&quot; rel=&quot;noreferrer&quot;&gt;added in 1.70&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-04T11:53:57.620" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3219" PostTypeId="1" CreationDate="2018-01-04T16:03:54.560" Score="2" ViewCount="115" Body="&lt;p&gt;I want to identify the &lt;strong&gt;most highly expressed genes across several developmental stages of a given insect&lt;/strong&gt;, in hopes of identifying a protein that can be used to design an insecticide.&#xA;I have a few doubts :&lt;/p&gt;&#xA;&lt;p&gt;Objective&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Several studies have been performed on &lt;strong&gt;individual developmental stages&lt;/strong&gt; and their transcriptome has been published. &lt;strong&gt;Should I compare each set of data against the other manually&lt;/strong&gt;, to find genes that are expressed universally and then target the most highly expressed ones or is there any &lt;strong&gt;software / computational method that can be used instead&lt;/strong&gt;?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Question&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is it right to compare transcriptomes to find commonly expressed genes?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I did try to look at the materials and methods section of a few papers on comparative transcriptomics, but it only says that data was compared without stating the method. One paper cited &lt;code&gt;VennBLAST&lt;/code&gt;, but I wasn't able to find the tool online.&lt;/p&gt;&#xA;" OwnerUserId="2136" LastEditorUserId="3848" LastEditDate="2020-07-15T07:52:43.470" LastActivityDate="2020-07-15T07:52:43.470" Title="How do I compare transcriptomes of the same species across different developmental stages?" Tags="|transcriptome|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="3220" PostTypeId="2" ParentId="3212" CreationDate="2018-01-04T19:04:42.663" Score="1" Body="&lt;p&gt;I found one similar example in an &lt;a href=&quot;ftp://ftp.arabidopsis.org/home/tair//Maps/gbrowse_data/TAIR10/TAIR10_GFF3_genes.gff&quot; rel=&quot;nofollow noreferrer&quot;&gt;Arabidopsis annotation file&lt;/a&gt;. It is case of the cpuORF (CONSERVED PEPTIDE UPSTREAM OPEN READING FRAME ). Although the first and the third mRNA isoforms are identical, they repeated the mRNA isoform and name them differently to accommodate the additional ORF/CDS. I guess this is another way to do it.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/lqu2c.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lqu2c.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the ranges for AT5G03190.1 isoform and AT5G03190.3 isoform.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Chr5    TAIR10  gene    758374  760382  .   +   .   ID=AT5G03190;Note=protein_coding_gene;Name=AT5G03190&#xA;Chr5    TAIR10  mRNA    758374  760382  .   +   .   ID=AT5G03190.1;Parent=AT5G03190;Name=AT5G03190.1;Index=1&#xA;Chr5    TAIR10  protein 758793  760148  .   +   .   ID=AT5G03190.1-Protein;Name=AT5G03190.1;Derives_from=AT5G03190.1&#xA;Chr5    TAIR10  exon    758374  760382  .   +   .   Parent=AT5G03190.1&#xA;Chr5    TAIR10  five_prime_UTR  758374  758792  .   +   .   Parent=AT5G03190.1&#xA;Chr5    TAIR10  CDS 758793  760148  .   +   0   Parent=AT5G03190.1,AT5G03190.1-Protein;&#xA;Chr5    TAIR10  three_prime_UTR 760149  760382  .   +   .   Parent=AT5G03190.1&#xA;Chr5    TAIR10  mRNA    758374  760382  .   +   .   ID=AT5G03190.2;Parent=AT5G03190;Name=AT5G03190.2;Index=1&#xA;Chr5    TAIR10  protein 758539  760148  .   +   .   ID=AT5G03190.2-Protein;Name=AT5G03190.2;Derives_from=AT5G03190.2&#xA;Chr5    TAIR10  exon    758374  758660  .   +   .   Parent=AT5G03190.2&#xA;Chr5    TAIR10  five_prime_UTR  758374  758538  .   +   .   Parent=AT5G03190.2&#xA;Chr5    TAIR10  CDS 758539  758660  .   +   0   Parent=AT5G03190.2,AT5G03190.2-Protein;&#xA;Chr5    TAIR10  exon    758843  760382  .   +   .   Parent=AT5G03190.2&#xA;Chr5    TAIR10  CDS 758843  760148  .   +   1   Parent=AT5G03190.2,AT5G03190.2-Protein;&#xA;Chr5    TAIR10  three_prime_UTR 760149  760382  .   +   .   Parent=AT5G03190.2&#xA;Chr5    TAIR10  mRNA    758374  760382  .   +   .   ID=AT5G03190.3;Parent=AT5G03190;Name=AT5G03190.3;Index=1&#xA;Chr5    TAIR10  protein 758539  758676  .   +   .   ID=AT5G03190.3-Protein;Name=AT5G03190.3;Derives_from=AT5G03190.3&#xA;Chr5    TAIR10  exon    758374  760382  .   +   .   Parent=AT5G03190.3&#xA;Chr5    TAIR10  five_prime_UTR  758374  758538  .   +   .   Parent=AT5G03190.3&#xA;Chr5    TAIR10  CDS 758539  758676  .   +   0   Parent=AT5G03190.3,AT5G03190.3-Protein;&#xA;Chr5    TAIR10  three_prime_UTR 758677  760382  .   +   .   Parent=AT5G03190.3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2118" LastEditorUserId="2118" LastEditDate="2018-01-04T19:13:59.193" LastActivityDate="2018-01-04T19:13:59.193" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3221" PostTypeId="1" AcceptedAnswerId="3233" CreationDate="2018-01-04T21:47:02.140" Score="3" ViewCount="908" Body="&lt;p&gt;When I want to load atoms for residuum &lt;code&gt;(' ', 85, ' ')&lt;/code&gt; in chain &lt;em&gt;A&lt;/em&gt; for protein &lt;strong&gt;3tmm&lt;/strong&gt; with &lt;code&gt;structure[0][chain_id][residue]&lt;/code&gt;, but I'm getting &lt;code&gt;KeyError&lt;/code&gt; exception:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  [...]&#xA;  File &quot;/home/pok/Checkouts/bilkoviny/loader.py&quot;, line 118, in _get_residue_atoms&#xA;    return list(self.structure[0][chain_id][residue])&#xA;  File &quot;/usr/lib64/python2.7/site-packages/Bio/PDB/Chain.py&quot;, line 74, in __getitem__&#xA;    return Entity.__getitem__(self, id)&#xA;  File &quot;/usr/lib64/python2.7/site-packages/Bio/PDB/Entity.py&quot;, line 40, in __getitem__&#xA;    return self.child_dict[id]&#xA;KeyError: (' ', 85, ' ')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I assume this is because this residuum do not have &quot;ATOM&quot; lines in PDB file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[...]&#xA;ATOM    704  CD1 ILE A  84      19.348  -4.839  18.138  1.00 47.08           C  &#xA;ANISOU  704  CD1 ILE A  84     5862   6108   5917   -216  -1513    609       C  &#xA;HETATM  705  N   MSE A  85      21.619  -0.130  16.236  1.00 31.89           N  &#xA;ANISOU  705  N   MSE A  85     5143   3072   3900   -766  -1011    962       N  &#xA;HETATM  706  CA  MSE A  85      22.277   0.675  15.226  1.00 40.76           C  &#xA;ANISOU  706  CA  MSE A  85     6708   3869   4911   -972   -729   1162       C  &#xA;HETATM  707  C   MSE A  85      23.146   1.786  15.818  1.00 44.98           C  &#xA;ANISOU  707  C   MSE A  85     7191   4141   5757  -1268   -558   1109       C  &#xA;HETATM  708  O   MSE A  85      24.257   2.045  15.338  1.00 43.69           O  &#xA;ANISOU  708  O   MSE A  85     7055   3855   5692  -1611   -151   1153       O  &#xA;HETATM  709  CB  MSE A  85      21.244   1.265  14.269  1.00 46.50           C  &#xA;ANISOU  709  CB  MSE A  85     8043   4309   5316   -709   -948   1345       C  &#xA;HETATM  710  CG  MSE A  85      21.849   1.954  13.052  1.00 54.03           C  &#xA;ANISOU  710  CG  MSE A  85     9604   4929   5995   -907   -623   1638       C  &#xA;HETATM  711 SE   MSE A  85      20.503   2.755  11.877  1.00 84.35          SE  &#xA;ANISOU  711 SE   MSE A  85    14348   8336   9366   -486  -1038   1884      SE  &#xA;HETATM  712  CE  MSE A  85      21.690   3.538  10.518  1.00 83.79           C  &#xA;ANISOU  712  CE  MSE A  85    15063   7865   8907   -915   -414   2326       C  &#xA;ATOM    713  N   SER A  86      22.636   2.453  16.846  1.00 43.76           N  &#xA;ANISOU  713  N   SER A  86     5234   6599   4794  -1778   -946    195       N  &#xA;[...]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have been told &quot;MSE is same as MET, it is just with selenium instead of sulphur&quot; and that I probably need to load these atoms (I'm approximating secondary structure element position, so I want individual atom coordinates).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some way how to get atoms for this residue in this protein via BioPython?&lt;/p&gt;&#xA;" OwnerUserId="2137" LastActivityDate="2018-01-06T17:37:23.143" Title="getting KeyError on certain residues in BioPython: ATOM vs. HETATM lines" Tags="|python|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3222" PostTypeId="1" AcceptedAnswerId="3226" CreationDate="2018-01-04T21:55:04.687" Score="1" ViewCount="34" Body="&lt;p&gt;I was browsering the UCSC viewer of the genome at &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTracks?db=mm10&amp;amp;lastVirtModeType=default&amp;amp;lastVirtModeExtraState=&amp;amp;virtModeType=default&amp;amp;virtMode=0&amp;amp;nonVirtPosition=&amp;amp;position=chr11%3A32271127%2D32368934&amp;amp;hgsid=650939229_Ph2CmqBGLyApOC2IRsw66VU7fSYT&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Genome position&quot;&gt;this region&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UCSC genes label two genes as Hba-a2 while GENCODE thinks the left one is Hba-a1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which annotation is correct?&lt;/p&gt;&#xA;" OwnerUserId="1852" LastEditorUserId="48" LastEditDate="2018-01-05T09:35:25.697" LastActivityDate="2018-01-05T09:37:28.423" Title="Hba-a2 has two copies on UCSC genome browser" Tags="|ucsc|genome-browser|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3224" PostTypeId="1" CreationDate="2018-01-05T03:11:07.163" Score="5" ViewCount="1164" Body="&lt;p&gt;I have a large BAM file on my IGV. I'd like to visualize a read alignment. I have the read name, and I don't care anything else in the file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IGV has an option &quot;Select by name&quot; (see screenshot). The problem is that IGV doesn't hide the alignments not matching the name. I'd still have to &lt;strong&gt;manually navigate&lt;/strong&gt; on IGV with my mouse to find my read.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That was too cumbersome. What I'm  doing now is make a new alignment BAM file. For example,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;samtools view -h L.V.51.bam | grep '@\|HWI-D00119:228:CB08RANXX:8:2116:4095:45403' | samtools view -bS - | samtools sort - &amp;gt; new.bam&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd load the new alignment file on my IGV.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I have 20 reads, I'd have to create 20 new BAM files... Is there anything I can do on IGV that I'm not aware of?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZBqFb.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZBqFb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="174" LastActivityDate="2018-01-05T09:47:22.237" Title="How to show only my read on IGV?" Tags="|igv|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3225" PostTypeId="2" ParentId="3197" CreationDate="2018-01-05T05:49:59.280" Score="6" Body="&lt;p&gt;In the paper mentioned, we used the &lt;code&gt;ScaleData&lt;/code&gt; function in Seurat to regress out the number of reads, Rn45s abundance, and percent ribosomal gene transcripts. Ribosomal genes were found with the regular expression &lt;code&gt;^Rp[sl][[:digit:]]&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tiss &amp;lt;- ScaleData(object = tiss, vars.to.regress = c(&quot;nReads&quot;, &quot;percent.ribo&quot;,&quot;Rn45s&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/czbiohub/maca/blob/master/MACA_Plate_Notebook.Rmd&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's&lt;/a&gt; a fuller notebook, and we'll have a better organized repository soon.&lt;/p&gt;&#xA;" OwnerUserId="2139" LastEditorUserId="48" LastEditDate="2018-01-05T09:27:50.420" LastActivityDate="2018-01-05T09:27:50.420" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3226" PostTypeId="2" ParentId="3222" CreationDate="2018-01-05T09:37:28.423" Score="2" Body="&lt;p&gt;Gencode is correct (when in doubt, always assume Gencode/Ensembl is correct when it differs from UCSC). What presumably happened is that the UCSC annotation pipeline noticed that Hba-a1 and Hba-a2 are nearly identical (they produce the same protein and have minimal nucleotide difference) and just misannotated.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-05T09:37:28.423" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3227" PostTypeId="2" ParentId="3224" CreationDate="2018-01-05T09:47:22.237" Score="7" Body="&lt;p&gt;Partially a shameless plug... I've written a genome browser, &lt;a href=&quot;http://asciigenome.readthedocs.io/en/latest/&quot; rel=&quot;noreferrer&quot;&gt;ASCIIGenome&lt;/a&gt;, entirely controlled through the command line. As such, it makes relatively easy to deal with questions like yours. For example, you could do: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ASCIIGenome bam/WW00292a.bam  # Load bam file(s)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Filter for reads matching the regex passed to &lt;code&gt;-i&lt;/code&gt; option:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[h] for help: grep -i '72789|57688|70785|56299|27116|12086'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Optionally, show sam records as strings)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[h] for help: print&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Example result:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zYBiK.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zYBiK.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For command help type &lt;code&gt;grep -h&lt;/code&gt; and for more filtering options see also &lt;code&gt;awk -h&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps and feel free to post questions and open issues on github!&lt;/p&gt;&#xA;" OwnerUserId="339" LastActivityDate="2018-01-05T09:47:22.237" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3228" PostTypeId="1" AcceptedAnswerId="3229" CreationDate="2018-01-05T18:02:29.210" Score="1" ViewCount="61" Body="&lt;p&gt;After normalization the way it is described here: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioconductor.org/packages/3.7/bioc/vignettes/scran/inst/doc/scran.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/3.7/bioc/vignettes/scran/inst/doc/scran.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using &lt;code&gt;quickCluster()&lt;/code&gt; function and running &lt;code&gt;computeSumFactors()&lt;/code&gt; I am trying to run &lt;code&gt;calculateQCMetrics()&lt;/code&gt; the way it is described here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://f1000research.com/articles/5-2122/v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://f1000research.com/articles/5-2122/v2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sce &amp;lt;- calculateQCMetrics(sce, feature_controls=list(Mt=is.mito))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I am getting an error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error in .checkedCall(cxx_colsum_subset, exprs_mat, is_feature_control) : &#xA;    subset indices out of range&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Does anybody know how to correct it? What is going wrong here?&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-01-05T18:11:05.347" Title="Error in .checkedCall : subset indices out of range" Tags="|r|scrnaseq|scran|" AnswerCount="1" CommentCount="0" ClosedDate="2018-01-06T14:05:09.430" ContentLicense="CC BY-SA 3.0" />
  <row Id="3229" PostTypeId="2" ParentId="3228" CreationDate="2018-01-05T18:11:05.347" Score="2" Body="&lt;p&gt;Just posted the question and found out why the error was happening. I had two datasets: raw, and filtered one with half of the rows as raw. I did normalization on the filtered one. Then I calculated &lt;code&gt;is.mito&lt;/code&gt; using the unfiltered, raw dataset. So, it produced indexes that were outside of the range of the dataset. &lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-01-05T18:11:05.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3230" PostTypeId="1" AcceptedAnswerId="3231" CreationDate="2018-01-05T21:59:48.683" Score="6" ViewCount="922" Body="&lt;p&gt;I would like to use Upper Quartile normalization for scRNA-seq data defined as:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The upperquartile (UQ) was proposed by (Bullard et al. 2010). Here each column is divided by the 75% quantile of the counts for each library. Often the calculated quantile is scaled by the median across cells to keep the absolute level of expression relatively consistent. A drawback to this method is that for low-depth scRNASeq experiments the large number of undetected genes may result in the 75% quantile being zero (or close to it). This limitation can be overcome by generalizing the idea and using a higher quantile (eg. the 99% quantile is the default in scater) or by excluding zeros prior to calculating the 75% quantile.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In particular, I'd use the 99% quantile to avoid normalizing by small number or zero. Is there any other drawback for using it on scRNA-seq data?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Bullard, James H, Elizabeth Purdom, Kasper D Hansen, and Sandrine Dudoit. 2010. “Evaluation of Statistical Methods for Normalization and Differential Expression in mRNA-Seq Experiments.” BMC Bioinformatics 11 (1). Springer Nature: 94. doi:10.1186/1471-2105-11-94.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2018-04-23T20:20:00.620" LastActivityDate="2018-04-23T20:20:00.620" Title="Drawbacks of upper quartile normalization for scRNA-seq data" Tags="|normalization|scrnaseq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3231" PostTypeId="2" ParentId="3230" CreationDate="2018-01-05T23:06:50.010" Score="4" Body="&lt;p&gt;My primary concern with using the top ~1% or so in upper quantile normalization is that it's going to be prone to the same robustness issues that RPKM/FPKMs run in to. That is, if for whatever technical reason you have to have a fair bit of variability in a couple very highly expressed genes (typically rRNAs, but one can imagine other genes) and the set of genes going into the UQ normalization is composed primarily of those then the normalization results will simply reflect irrelevant technical variability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, this all assumes that it's even appropriate to normalize scRNAseq data with a standard method like this. In some cases it is (e.g., when comparing control vs. treatment with a largely homogeneous cell population), but if you're dealing with very different cell types then I worry that the standard method is going to wash away things like differences in bulk RNA content between cell types. Presuming there are spike-ins, they should have the same normalization applied and then checked to ensure there's nothing greatly amiss. Having said that, if you're going to be doing tSNE and want to look for cell types, then odds are good you'll still be able to see most of them (after all, it's unlikely that RNA quantity is the only difference between them, though not removing that difference would make things easier).&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;The problem of global shifts in expression&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The concept of global shifts in expression is not unique to scRNAseq, in fact the most common example of this is &lt;a href=&quot;http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0153903&quot; rel=&quot;nofollow noreferrer&quot;&gt;Myc up/down regulation in relation to cancer&lt;/a&gt;. To explain the issue, let's suppose that you have the following (very small) matrix of counts:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;       sample1 sample2&#xA;gene1        1       2&#xA;gene2       10      20&#xA;gene3       15      32&#xA;gene4      100     198&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, sample2 has roughly twice as many reads for each gene as sample1, so any normalization is going to cut all of its counts in half (or the reverse for sample1, or perhaps a mixture of the two...depending on the method). That's usually fine, since the assumption with all common normalization methods is that differences like this are simply due to differences in sequencing depth between the samples. But what happens if sample2 actually has twice as much RNA as sample1? Well, then you've just normalized away all of the differences between the samples (I encourage readers to go through the literature on Myc, since that's a very nice example of this, where a mostly-global shift in expression led to apparent down-regulation of genes with constant expression). Such a situation doesn't occur very commonly in bulk RNAseq, but when it does you need to use something like spike-ins (dosed to match cell number or DNA quantity) to maintain the global difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is only rarely an issue with bulk RNAseq, then why is it more likely to be an issue in scRNAseq? The answer is that bulk RNAseq benefits from the averaging out of many different cell-types, such that ensemble averages/medians don't tend to change much due to treatments...but this isn't going to be the case at the single-cell level, since cells can vary vastly in size (compare a sperm and an oocyte...every RNA should be differentially expressed between them). This difference in base RNA content between different cell types will be washed away by common normalization methods, which can lead to either losing differences or creating differences in genes where there shouldn't be (again, see examples of this in the Myc literature). Whether this really becomes an issue will depend on what your goals are with your data and how much of a global expression difference you actually have and the type of analysis you want to perform.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the end of the day, just remember that common RNAseq normalization methods assume that (A) there's no global expression change between samples and that (B) if there are very many DE genes that the direction of change is not heavily skewed in one direction. If either of these assumptions are violated then you can run into issues.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-01-06T12:40:29.123" LastActivityDate="2018-01-06T12:40:29.123" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3232" PostTypeId="1" AcceptedAnswerId="3258" CreationDate="2018-01-06T00:10:01.707" Score="6" ViewCount="596" Body="&lt;p&gt;I got a &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE98561&quot; rel=&quot;noreferrer&quot;&gt;dataset&lt;/a&gt; from &lt;code&gt;C.Elegans scRNA-seq&lt;/code&gt; &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/28818938&quot; rel=&quot;noreferrer&quot;&gt;paper&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;GSM2599701_Gene.count.matrix.celegans.cell.Rdata&lt;/code&gt; in &lt;code&gt;GSE98561_RAW.tar&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dataset is &lt;code&gt;40 000 x 68 000&lt;/code&gt;, where rows represent genes and columns - cells. So, I took it and tried to process myself to build an &lt;code&gt;scRNA-seq&lt;/code&gt; pipeline. Here is what I did:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I filtered out the genes that have zero counts in all of the cells and the dataset was reduced to &lt;code&gt;29 000 x 68 000&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I removed all of the cells with counts &lt;code&gt;&amp;lt; 100&lt;/code&gt; in all of the genes - the dataset became &lt;code&gt;29 000 x 66 000&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then, because the dataset was too big to run normalization even on the cluster with &lt;code&gt;120Gb&lt;/code&gt; RAM (because there are multiple distinct types of cells, first clustering needs to be done), I selected just even columns and ran normalization with &lt;code&gt;29 000 x 33 000&lt;/code&gt; dataset (&lt;code&gt;UMI_count&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(scran)     &#xA;library(scater)&#xA;sce &amp;lt;- newSCESet(countData=UMI_count)&#xA;clusters &amp;lt;- quickCluster(sce)&#xA;sce &amp;lt;- computeSumFactors(sce, clusters=clusters, positive=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;After running the code above I decided to check whether the data is fine and so I ran:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; summary(sizeFactors(sce))&#xA;Min.    1st Qu.  Median  Mean   3rd Qu.    Max. &#xA;0.0000  0.0000  0.0000  0.0717  0.0000 33.3900 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also ran &lt;code&gt;PCA&lt;/code&gt; on the normalized dataset and it looks like that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4sPJN.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4sPJN.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that the normalized dataset is terrible and I need to do some further processing before I could run further analysis. What else could I do to improve it? How to filter it? There are no &lt;code&gt;spike-ins&lt;/code&gt; and maybe &lt;code&gt;200&lt;/code&gt; mitochondrial genes. The approach described &lt;a href=&quot;https://f1000research.com/articles/5-2122/v2&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; does not work, probably because the majority of the cells have low number of genes expressed:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UDcKg.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UDcKg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried removing low-abundance genes after normalization, but it seems that most of them are going to be removed:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;ave.counts &amp;lt;- rowMeans(counts(sce))&#xA;&amp;gt;keep &amp;lt;- ave.counts &amp;gt;= 1&#xA;&amp;gt;sum(keep)&#xA;109&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Should I filter all of the cells - columns - for up to &lt;code&gt;500&lt;/code&gt; total gene expression count instead of &lt;code&gt;100&lt;/code&gt;? Is it a good idea? I can't think of anything else.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2018-01-06T10:30:13.447" LastActivityDate="2018-01-09T09:18:38.857" Title="Improve scRNA-seq dataset for further analysis" Tags="|r|scrnaseq|scran|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3233" PostTypeId="2" ParentId="3221" CreationDate="2018-01-06T17:37:23.143" Score="1" Body="&lt;p&gt;You can find the answer in &lt;a href=&quot;http://biopython.org/wiki/The_Biopython_Structural_Bioinformatics_FAQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Biopython Structural Bioinformatics FAQ&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;What is a residue id?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is a bit more complicated, due to the clumsy PDB format. A&#xA;residue id is a tuple with three elements:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;strong&gt;hetero-flag&lt;/strong&gt;: this is 'H_' plus the name of the hetero-residue (e.g. 'H_GLC' in the case of a glucose molecule), or 'W' in the case&#xA;of a water molecule.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;strong&gt;sequence identifier&lt;/strong&gt; in the chain, e.g. 100&lt;/li&gt;&#xA;&lt;li&gt;The &lt;strong&gt;insertion code&lt;/strong&gt;, e.g. 'A'. The insertion code is sometimes&#xA;used to preserve a certain desirable residue numbering scheme. A Ser 80&#xA;insertion mutant (inserted e.g. between a Thr 80 and an Asn 81&#xA;residue) could e.g. have sequence identifiers and insertion codes as&#xA;follows: Thr 80 A, Ser 80 B, Asn 81. In this way the residue numbering&#xA;scheme stays in tune with that of the wild type structure.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The id of the above glucose residue would thus be ('H_GLC', 100, 'A').&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So in your case it would be &lt;code&gt;('H_MSE', 85, ' ')&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="266" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-01-06T17:37:23.143" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3234" PostTypeId="1" CreationDate="2018-01-06T21:59:15.163" Score="2" ViewCount="491" Body="&lt;p&gt;I am using &lt;code&gt;bedtools&lt;/code&gt; to test whether a set of variant calls produced by a new method is missing any variants from a set of known validated variants.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools intersect -v -a newcalls.vcf -b goldstandard.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It's working as intended for SNV calls, but it occasionally misses indels due to a single base pair discrepancy. It's usually an alignment artifact that doesn't affect the interpretation of the variant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first thought was to make a BED file with intervals that extend a few bp beyond each variant call, and comparing &lt;em&gt;this&lt;/em&gt; BED file to the known variants. However, I was wondering whether approximate matching is possible with &lt;code&gt;bedtools intersect&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2018-01-07T10:08:44.440" Title="Approximate matches with bedtools intersect" Tags="|vcf|bedtools|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3235" PostTypeId="1" CreationDate="2018-01-06T22:22:52.033" Score="2" ViewCount="1766" Body="&lt;p&gt;I am creating a .cls file for my GSEA analysis and see the below error. Can anyone suggest a solution?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I create the file using TeXworks. The file looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;37 2 1&#xA;#CD69neg CD69pos&#xA;CD69neg CD69pos&#xA;CD69neg CD69pos CD69pos CD69pos CD69neg CD69pos CD69pos CD69pos CD69pos CD69neg CD69neg CD69neg CD69neg CD69pos CD69pos CD69neg CD69pos CD69pos CD69neg CD69neg CD69neg CD69pos CD69pos CD69pos CD69neg CD69pos CD69pos CD69neg CD69neg CD69pos CD69pos CD69pos CD69neg CD69pos CD69pos CD69pos CD69pos&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Error Details&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;---- Full Error Message ----&#xA;There were errors: ERROR(S) #:1 Parsing trouble edu.mit.broad.genome.parsers.Par ...&#xA;&#xA;---- Stack Trace ----&#xA;&#xA;of exceptions: 1&#xA;------Number of items found in cls data 10 is not equal to the number specified on the header line 37------ &#xA;&#xA;edu.mit.broad.genome.parsers.ParserException:&#xA;Number of items found in cls data 10 is not equal to the number specified on the header line 37 at &#xA;edu.mit.broad.genome.parsers.ClsParser.doSanityChecks(ClsParser.java:388) at &#xA;edu.mit.broad.genome.parsers.ClsParser._parse_genecluster_style_categorical(ClsParser.java:252) at &#xA;edu.mit.broad.genome.parsers.ClsParser.parse(ClsParser.java:226) at &#xA;edu.mit.broad.genome.parsers.ParserFactory._readTemplates(ParserFactory.java:342) at &#xA;edu.mit.broad.genome.parsers.ParserFactory.readTemplate(ParserFactory.java:293) at &#xA;edu.mit.broad.genome.parsers.ParserFactory.read(ParserFactory.java:753) at &#xA;edu.mit.broad.genome.parsers.ParserFactory.read(ParserFactory.java:726) at &#xA;edu.mit.broad.genome.parsers.ParserWorker.doInBackground(ParserWorker.java:52) at &#xA;javax.swing.SwingWorker$1.call(SwingWorker.java:295) at &#xA;java.util.concurrent.FutureTask.run(FutureTask.java:266) at &#xA;javax.swing.SwingWorker.run(SwingWorker.java:334) at &#xA;java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at &#xA;java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2146" LastEditorUserId="73" LastEditDate="2018-01-07T02:54:24.513" LastActivityDate="2018-02-06T16:02:43.460" Title="GSEA- parsing error for .cls file" Tags="|gsea|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3236" PostTypeId="2" ParentId="3234" CreationDate="2018-01-07T02:20:20.810" Score="2" Body="&lt;p&gt;You can use &lt;code&gt;bedtools intersect&lt;/code&gt; but first use &lt;code&gt;bcftools norm&lt;/code&gt;.  &lt;a href=&quot;http://www.htslib.org/doc/bcftools.html#norm&quot; rel=&quot;nofollow noreferrer&quot;&gt;bcftools norm&lt;/a&gt; will left-align and normalize indels to get both vcf files consistent with each other.  This should resolve the matching if they are truly matching variants.&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2018-01-07T02:20:20.810" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3237" PostTypeId="2" ParentId="3234" CreationDate="2018-01-07T10:08:44.440" Score="-1" Body="&lt;p&gt;You could use &lt;code&gt;bedtools closest&lt;/code&gt; with &lt;code&gt;-d&lt;/code&gt; option (&lt;em&gt;-d: In addition to the closest feature in B, report its distance to A as an extra column.&lt;/em&gt;). Indels that have no match in file A would be those where the closest indel in file B is more than &lt;em&gt;x&lt;/em&gt; bases away.&lt;/p&gt;&#xA;" OwnerUserId="339" LastActivityDate="2018-01-07T10:08:44.440" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3239" PostTypeId="1" CreationDate="2018-01-07T15:04:25.000" Score="2" ViewCount="733" Body="&lt;p&gt;This is my bed file for all the exon coordinates I want to take out all the exons of a given gene. Let say I have gene in chr 1 which starts from &lt;code&gt;chr1 11868   12227&lt;/code&gt; so I want to parse out all the exons that comes in between &lt;code&gt;11868    12227&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my small subset:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat exon.bed | head -10&#xA;chr1    11868   12227   +   exon&#xA;chr1    11871   12227   +   exon&#xA;chr1    11873   12227   +   exon&#xA;chr1    12009   12057   +   exon&#xA;chr1    12178   12227   +   exon&#xA;chr1    12594   12721   +   exon&#xA;chr1    12612   12697   +   exon&#xA;chr1    12612   12721   +   exon&#xA;chr1    12612   12721   +   exon&#xA;chr1    12974   13052   +   exon&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How do I parse out?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use mostly R and bit of shell script but I'm not sure if I can use R, may a few lines of perl or shell script can help me solve my problem.&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="48" LastEditDate="2018-01-09T11:44:41.677" LastActivityDate="2018-01-09T11:44:41.677" Title="Parse out exon coordinates from bed file for each gene" Tags="|r|bed|bedtools|perl|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3240" PostTypeId="2" ParentId="3235" CreationDate="2018-01-07T15:14:57.050" Score="3" Body="&lt;p&gt;You seem to have forgotten the space after the #. It should be as in &lt;a href=&quot;http://software.broadinstitute.org/cancer/software/gsea/wiki/index.php/Data_formats#CLS:_Categorical_.28e.g_tumor_vs_normal.29_class_file_format_.28.2A.cls.29&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (Just for future readers know where to find the specs). Also it should be a single line (Maybe this error was introduced while pasting here or it has been a duplicate because there are 39 samples in the 3 line and they should be 37).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Either adding 2 more samples to the initial count and removing the newline of the third line: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;39 2 1&#xA;# CD69neg CD69pos&#xA;CD69neg CD69pos CD69neg CD69pos CD69pos CD69pos CD69neg CD69pos CD69pos CD69pos CD69pos CD69neg CD69neg CD69neg CD69neg CD69pos CD69pos CD69neg CD69pos CD69pos CD69neg CD69neg CD69neg CD69pos CD69pos CD69pos CD69neg CD69pos CD69pos CD69neg CD69neg CD69pos CD69pos CD69pos CD69neg CD69pos CD69pos CD69pos CD69pos&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or removing the third line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;37 2 1&#xA;# CD69neg CD69pos&#xA;CD69neg CD69pos CD69pos CD69pos CD69neg CD69pos CD69pos CD69pos CD69pos CD69neg CD69neg CD69neg CD69neg CD69pos CD69pos CD69neg CD69pos CD69pos CD69neg CD69neg CD69neg CD69pos CD69pos CD69pos CD69neg CD69pos CD69pos CD69neg CD69neg CD69pos CD69pos CD69pos CD69neg CD69pos CD69pos CD69pos CD69pos&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-01-07T15:14:57.050" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3241" PostTypeId="2" ParentId="3239" CreationDate="2018-01-07T15:21:07.027" Score="5" Body="&lt;p&gt;There are multiple ways to go about it. On the command line you can make a 1 line BED file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1   11868   12227&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then &lt;code&gt;bedtools intersect&lt;/code&gt; with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In R, you could load your original BED file and use GenomicRanges:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(&quot;GenomicRanges&quot;)&#xA;bed = read.delim(&quot;foo.bed&quot;, header=F)  # Rename this&#xA;# N.B., BED files use 0-based coordinates, I've switched to 1-based here&#xA;exons = GRanges(seqnames=levels(bed$V1),&#xA;                ranges=IRanges(start=bed$V2+1, end=bed$V3),&#xA;                strand=bed$V4)&#xA;# Again, I switched to 1-based coordinates...you don't have to do that&#xA;region = GRanges(seqnames=&quot;chr1&quot;,&#xA;                 ranges=IRanges(start=11869, end=12227),&#xA;                 strand=&quot;+&quot;)&#xA;theExonsYouWant = subsetByOverlaps(exons, region, type=&quot;within&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You'll need to either specify a strand in your region or use &lt;code&gt;ignore.strand=T&lt;/code&gt; in &lt;code&gt;subsetByOverlaps()&lt;/code&gt;. I hope you don't have any genes that overlap on the same strand, since they'll prove problematic.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-01-07T15:34:31.803" LastActivityDate="2018-01-07T15:34:31.803" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3242" PostTypeId="2" ParentId="3209" CreationDate="2018-01-07T15:24:08.747" Score="0" Body="&lt;p&gt;You can use GSEA-like methods for any type of data. The basis of the test is to analyze if there is a preference for a group (a gene set) in the ranking of the other variable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you first need to find where the functions you are interested are in the genome (To associate a range/position with a given function). Then using this association you can test if the ranking of the locus is associated with a specific function that is in a given area. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't seen anything like that implemented but you could do it yourself using biomaRt to retrieve data from where a gene (or a function) is located and fgsea to test which functions are associated with your score. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-01-07T15:24:08.747" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3243" PostTypeId="2" ParentId="3239" CreationDate="2018-01-07T20:17:27.923" Score="3" Body="&lt;p&gt;Via BEDOPS &lt;code&gt;bedops -n&lt;/code&gt; and Unix I/O streams:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ echo -e &quot;chr1\t11868\t12227&quot; | bedops -n 1 exon.bed - &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, if you have your genes in a BED file called &lt;code&gt;genes.bed&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops -n 1 exon.bed genes.bed &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you have your genes in some other format, like GFF or GTF, you can use &lt;code&gt;gff2bed&lt;/code&gt; or &lt;code&gt;gtf2bed&lt;/code&gt;, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops -n 1 exon.bed &amp;lt;(gff2bed &amp;lt; genes.gff) &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops -n 1 exon.bed &amp;lt;(gtf2bed &amp;lt; genes.gtf) &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The file &lt;code&gt;answer.bed&lt;/code&gt; will contain exons that do not overlap a gene annotation.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-01-07T20:22:49.107" LastActivityDate="2018-01-07T20:22:49.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3244" PostTypeId="2" ParentId="3232" CreationDate="2018-01-07T23:28:56.953" Score="1" Body="&lt;p&gt;Given that this is single-cell sequencing results, you should be using &lt;code&gt;rowSums&lt;/code&gt; instead of &lt;code&gt;rowMeans&lt;/code&gt; for filtering genes. Most genes won't be expressed at a single cell level, but there should be a lot of genes that have some expression. The zero counts are probably better represented as &quot;missing&quot; rather than zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately that feature makes running PCAs difficult, because PCAs don't work with missing values. The missing values may need to be imputed in order to generate a better PCA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;disclaimer: I don't [typically] work with single cell data, so don't have a good handle on the methods for looking at it.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-01-07T23:28:56.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3245" PostTypeId="2" ParentId="3232" CreationDate="2018-01-08T03:45:36.337" Score="5" Body="&lt;p&gt;I don't think you can conclude that the dataset is terrible based on that PCA. Depending on the specific protocol, each scRNA-seq dataset is going to be very different. Unlike bulk RNA-seq where all the samples are going to be of very similar quality, individual cells will be highly variable. For example, the most basic QC metric is the number of reads. If some of your bulk RNA-seq samples have 10X more reads than other samples, that's a cause for concern. In scRNA-seq experiments, that's not at all surprising. The PCA exposes the major sources of variation. In scRNA-seq experiment, those are probably going to be technical rather than biological, so the output may not make much sense. Furthermore, even if all the cells are of perfect quality, PCA is not really optimal for sparse data with many dropouts like you see with scRNA-seq.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regardless of PCA, those library sizes seem very small. Based on your plot, there are barely any cells with over 10K reads. I understand this is C elegans, but that seems somewhat low. That may be why you are having trouble with normalization. It would help with understanding the data if you changed the X-axis on those plots so that you can actually see where the peak is. Right now, it looks like 0, but that can't be the case.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2018-01-08T03:45:36.337" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3246" PostTypeId="1" CreationDate="2018-01-08T05:02:48.610" Score="2" ViewCount="118" Body="&lt;p&gt;I have a bunch of CNVs which I have estimated LD between each allele of CNV with all SNPs across the entire human genome. I used my R-script to do a pairwise LD analysis.&lt;/p&gt;&#xA;&lt;p&gt;The following is the first few lines of my output for the first alternative allele of CNV and some SNPs on chromosome 2 (just as an example)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CNV_chrom  CNV_allele  SNP_chrom  SNP_position  R2&#xA;&amp;quot;1&amp;quot; &amp;quot;  CNV2&amp;quot;   &amp;quot;2&amp;quot;        &amp;quot; 1000&amp;quot;    &amp;quot;-0.0463767871013283&amp;quot; &#xA;&amp;quot;1&amp;quot; &amp;quot;  CNV2&amp;quot;   &amp;quot;2&amp;quot;        &amp;quot; 1029&amp;quot;    &amp;quot;-0.201478578143067&amp;quot; &#xA;&amp;quot;1&amp;quot; &amp;quot;  CNV2&amp;quot;   &amp;quot;2&amp;quot;        &amp;quot; 1044&amp;quot;    &amp;quot;-0.595441932439136&amp;quot;&#xA;&amp;quot;1&amp;quot; &amp;quot;  CNV2&amp;quot;   &amp;quot;2&amp;quot;        &amp;quot; 1045&amp;quot;    &amp;quot;-0.60363963711489&amp;quot;&#xA;&amp;quot;1&amp;quot; &amp;quot;  CNV2&amp;quot;   &amp;quot;2&amp;quot;        &amp;quot; 1130&amp;quot;    &amp;quot;0.377970958849221&amp;quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to plot the LD results (LD between each allele of CNV and all SNPs) ,is there any way that I could do it in R?&lt;/p&gt;&#xA;&lt;p&gt;I have not done LD plotting before so just slightly confused.&lt;/p&gt;&#xA;&lt;p&gt;I saw for instance &lt;a href=&quot;https://cran.r-project.org/web/packages/LDheatmap/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;LDheatmap package&lt;/a&gt; in R, but do not know how can I summarize the results for thousands of SNPs. Thanks for any suggestion&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Summary of LDHeatmap&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Produces a graphical display, as a heat map, of measures of pairwise&#xA;linkage disequilibria between single nucleotide polymorphisms (SNPs).&#xA;Users may optionally include the physical locations or genetic map&#xA;distances of each SNP on the plot.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Citation&lt;/p&gt;&#xA;&lt;p&gt;Shin, J.-H., Blay, S., McNeney, B., &amp;amp; Graham, J. (2006). LDheatmap: An R Function for Graphical Display of Pairwise Linkage Disequilibria Between Single Nucleotide Polymorphisms. Journal of Statistical Software, Code Snippets, 16(3), 1–9. &lt;a href=&quot;https://doi.org/10.18637/jss.v016.c03&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.18637/jss.v016.c03&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1749" LastEditorUserId="3848" LastEditDate="2022-03-26T14:45:09.880" LastActivityDate="2022-03-26T14:45:09.880" Title="how to plot LD between SNPs and CNVs" Tags="|snp|heatmap|cnv|ld|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3247" PostTypeId="1" AcceptedAnswerId="3273" CreationDate="2018-01-08T10:05:33.557" Score="6" ViewCount="1109" Body="&lt;p&gt;I downloaded gene expression data (&lt;code&gt;exp_seq&lt;/code&gt;) from the &lt;a href=&quot;https://dcc.icgc.org/releases/current/Projects/BRCA-US&quot; rel=&quot;noreferrer&quot;&gt;ICGC file browser&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each sample and gene, the file contains a &lt;code&gt;normalized_read_count&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is that value? I couldn't find any information on the ICGC website. The values are definitly too low for TPM. &lt;/p&gt;&#xA;" OwnerUserId="1656" LastActivityDate="2018-01-10T14:57:21.637" Title="What is the ICGC normalized_read_count?" Tags="|rna-seq|public-databases|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3248" PostTypeId="1" CreationDate="2018-01-08T10:30:36.000" Score="3" ViewCount="33" Body="&lt;p&gt;First, please know that I am a novice in this field and that it is thus very likely that you will regard this question as basic or flat stupid. Please be gentle and don't be afraid to criticize. I am asking this so I can learn. Also, don't feel afraid to discuss in comments if you can't give a straight answer; all input is useful.&lt;/p&gt;&#xA;&lt;p&gt;I am currently working in a project where we have fractionated samples in order to dig deeper into the proteome. The fractionated samples have been analysed with shotgun LC-MS and then searched with a contemporary software using a whole proteome database and to the reversed data base in order to estimate FDR. The search result is cut-off so that the highest scoring sequences are included with an estimated FDR of 1%. In this project we have 150 samples in which we can expect to find certain proteins in almost all samples, while others are unlikely to be found in any of them (but we don't know which yet). If we were to decrease the search space to include only relevant proteins (which as previously stated are not known to us yet), we would boost our rate of true positives included after FDR cut-off, as we decrease the search space which noise randomly might match to with a high score (right?). As we can't know which ones are relevant beforehand, would it be possible to determine relevant proteins with a preliminary search? That is, to do as follows:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Search against whole proteome with a certain FDR cut-off (possibly larger than 1%)&lt;/li&gt;&#xA;&lt;li&gt;Make a new data base including only proteins that has been found in &lt;strong&gt;any of the samples&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Search data against new data base.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;To me it seems like there is a possibility that this could increase the number of true positives that are found, but I also feel like I might be missing something crucial. Please, enlighten me: Is this methododology unbiased and valid? What risks may there be? Anything more one has to think about before attempting something like this?&lt;/p&gt;&#xA;&lt;p&gt;The LC-MS was a top 10 shotgun MSMS using a quadrupole-Orbitrap instrument. The search was done using MaxQuant where fragmentation spectra are use to identify MS1 peaks, which intensity is registered as a quantitative datum. All files were searched together but are configured to be separated in the results file in regards to both sample and fraction. We will likely not perform additional analysis of these samples.&lt;/p&gt;&#xA;" OwnerUserId="796" LastEditorUserId="73" LastEditDate="2021-07-04T04:55:32.040" LastActivityDate="2021-07-04T04:55:32.040" Title="Can second search with limited search space improve result in MS proteomics data?" Tags="|database|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3249" PostTypeId="1" AcceptedAnswerId="3250" CreationDate="2018-01-08T11:53:25.147" Score="3" ViewCount="4769" Body="&lt;p&gt;I have successfully downloaded the whole nt BLAST database into the current folder using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wget -b &quot;ftp://ftp.ncbi.nlm.nih.gov/blast/db/nt.??.tar.gz&quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I would like to download it in a specific folder, e.g. output/&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wget -b &quot;ftp://ftp.ncbi.nlm.nih.gov/blast/db/nt.??.tar.gz&quot;  -O output/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but it does not work. Nothing happens. My 'wget-log' is empty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could move to that folder and download it there but I would like to remain in the same working directory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2023-06-14T16:47:25.117" Title="How to download the whole BLAST nt database into a specific folder?" Tags="|blast|data-download|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3250" PostTypeId="2" ParentId="3249" CreationDate="2018-01-08T12:21:28.730" Score="9" Body="&lt;p&gt;&lt;code&gt;-O&lt;/code&gt; is to specify the name of the output &lt;em&gt;file.&lt;/em&gt; You want the &lt;a href=&quot;https://stackoverflow.com/questions/1078524/how-to-specify-the-location-with-wget&quot;&gt;&lt;code&gt;-P&lt;/code&gt;&lt;/a&gt; option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, from your example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wget -b &quot;ftp://ftp.ncbi.nlm.nih.gov/blast/db/nt.??.tar.gz&quot; -P output/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="581" LastEditorUserId="2135" LastEditDate="2018-01-08T12:28:50.270" LastActivityDate="2018-01-08T12:28:50.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3251" PostTypeId="1" AcceptedAnswerId="3504" CreationDate="2018-01-08T16:27:03.467" Score="2" ViewCount="190" Body="&lt;p&gt;Is it possible to find out the PacBio chemistry from the PacBio Sequel BAM's header? E.g. whether it was produced with P5-C3 or P6-C4? I checked mine but I haven't found it but maybe it's hidden in one of the tags whose meaning I don't know yet, like SEQUENCINGKIT=100-861-800?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="298" LastEditDate="2018-01-08T16:55:34.137" LastActivityDate="2018-02-09T16:15:01.133" Title="Is it possible to find out the PacBio chemistry from the PacBio Sequel BAM's header?" Tags="|sam|pacbio|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3252" PostTypeId="1" CreationDate="2018-01-08T22:33:58.890" Score="1" ViewCount="184" Body="&lt;p&gt;I have 3 biological replicates of RNA seq data for a particular condition. I want to find out intron retention events from those biological replicates for a given condition. There is no comparison I am doing here. Just from the three replicates, I am trying to find out what introns are retained in my sample. There is only one condition here. Tools like rMATS require 2 conditions. Is there any tool I can use to find out intron retention events given that I only have one condition (with replicates).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally I would like to use a method similar to that used &lt;a href=&quot;http://genesdev.cshlp.org/content/suppl/2014/12/29/29.1.63.DC1/SuppMaterial.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;in this paper&lt;/a&gt;, though I can't fully grasp their methodology. This would allow me to get p-values for enrichment of specific introns.&lt;/p&gt;&#xA;" OwnerUserId="2040" LastEditorUserId="77" LastEditDate="2018-01-09T20:12:20.173" LastActivityDate="2018-03-04T08:02:22.913" Title="Intron retention events for a condition sample with 3 replicates" Tags="|rna-seq|rna-splicing|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3253" PostTypeId="1" AcceptedAnswerId="3296" CreationDate="2018-01-08T22:36:12.993" Score="5" ViewCount="352" Body="&lt;p&gt;I have large filtered, normalized dataset of &lt;code&gt;scRNA-seq&lt;/code&gt; data of &lt;code&gt;C.Elegans&lt;/code&gt; species. Rows are genes (10 000), columns are cells (66 000). Let's say that I got 40 different groups of the cells based on their expression profiles, how could I now assign cell types to these groups? I would expect to get some reference dataset for &lt;code&gt;C.Elegans&lt;/code&gt; and compare it with what I got, but I honestly have no idea of where to get such a dataset, and whether there are any &lt;code&gt;R&lt;/code&gt; packages that do that. Any advice would be greatly appreciated.  &lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2020-07-27T20:10:17.930" Title="Assign cell types to groups of cells based on their gene expression profiles" Tags="|r|scrnaseq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3254" PostTypeId="1" AcceptedAnswerId="3255" CreationDate="2018-01-09T06:36:32.393" Score="7" ViewCount="1923" Body="&lt;p&gt;I have a screenshot where many of my reads were aligned to a region that I &lt;strong&gt;suspect&lt;/strong&gt; a low complexity region. Although you can't see, all those reads are &lt;strong&gt;clipped&lt;/strong&gt; in the cigar strings. Sample cigar strings: &lt;em&gt;64S32M29S, 74S32M18S&lt;/em&gt; etc... Consequently, the actual sequence mapped to the genome is &lt;strong&gt;less&lt;/strong&gt; than the read length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a feeling that my alignments are bogus because of the complexity and clipping, but I'm not sure if the complexity is indeed low. All I see is a few &quot;T&quot; between a bunch of &quot;A&quot;. How to define a low complexity region?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q:&lt;/strong&gt; Is this a low complexity region in the human hg38 genome? Would that be fair if I report &quot;&lt;em&gt;the alignments are likely be errors due to the low complexity and alignment clipping&lt;/em&gt;&quot;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fJC2X.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fJC2X.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="174" LastEditDate="2018-01-09T06:42:54.843" LastActivityDate="2018-01-09T07:54:54.347" Title="Is this a low complexity region in our human genome?" Tags="|genome|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3255" PostTypeId="2" ParentId="3254" CreationDate="2018-01-09T07:51:02.210" Score="7" Body="&lt;p&gt;Yes, that's a short low complexity region wedged between a SINE and an snRNA. More importantly, your alignments have MAPQ of 0 (that's why they're filled with white in IGV), which will happen if they map equally well to multiple locations. Without looking at the sequence one can use that alone to determine that these are not trustworthy mappings.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-09T07:51:02.210" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3256" PostTypeId="2" ParentId="3254" CreationDate="2018-01-09T07:54:54.347" Score="6" Body="&lt;p&gt;Yes, this is a low complexity region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regions are considered low complexity (or having a simple sequence) when they contain an abundance of a single base, or an abundance of short tandem repeats. The simplest is a tandem repeat of a single base (e.g. &lt;code&gt;AAAAAAAAAAAAAAAA&lt;/code&gt;), also called a homopolymer. In your case there is a 3-unit tandem repeat of &lt;code&gt;TAAAAA&lt;/code&gt;, and surrounding As (with a few intermingled Ts). Depending on the genetic history of this region, it might be that those other As and Ts are derived from the same &lt;code&gt;TAAAAA&lt;/code&gt; unit, with some slipping that has occurred during replication to create a cryptic (i.e. non-perfect) repeat structure.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-01-09T07:54:54.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3257" PostTypeId="1" AcceptedAnswerId="3262" CreationDate="2018-01-09T09:01:54.507" Score="0" ViewCount="83" Body="&lt;p&gt;We have a column which has following format : (1, 1, 1, 0, 1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the definition of what means two columns are &lt;strong&gt;compatible&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Using the notation Oi to denote the collection of rows possessing a 1&#xA;  in their i-th element, we conclude that one of three possibilities&#xA;  must be true: Oi ⊆ Oj , Oj ⊆ Oi , or Oi and Oj are disjoint (the first&#xA;  two cases include the possibility that Oi = Oj). If columns i and j&#xA;  satisfy this condition, then we call them compatible&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The question is : &lt;strong&gt;How many possible columns of length 5 are compatible with the column (1, 1, 1, 0, 1)?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can read more &lt;a href=&quot;http://bioinformaticsalgorithms.com/excerpt/How%20Have%20Humans%20Populated%20the%20Earth.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; where problem is described in page 14-16.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance !&lt;/p&gt;&#xA;" OwnerUserId="2088" LastEditorUserId="73" LastEditDate="2018-01-09T18:20:51.863" LastActivityDate="2018-01-09T18:20:51.863" Title="Find number of possibilities" Tags="|gene|genome|homework|" AnswerCount="1" CommentCount="4" ClosedDate="2018-01-09T21:18:39.330" ContentLicense="CC BY-SA 3.0" />
  <row Id="3258" PostTypeId="2" ParentId="3232" CreationDate="2018-01-09T09:18:38.857" Score="3" Body="&lt;p&gt;I usually use a minimum of 200 genes/cell for relaxed filters (500 if possible).&#xA;For genes I use presence in at least 10% of the cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few other filters you can try are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;High percentage of reads from only top50 genes. This means you haven't captured a lot of genes and the diversity of the cell won't be useful.&lt;/li&gt;&#xA;&lt;li&gt;High percentage of mitochondrial genes. This has been reported to be linked to bad quality cells. The idea is that mRNA is depleted because of early cell death (or other outer membrane injury) leading to having only mito mRNA left.&lt;/li&gt;&#xA;&lt;li&gt;Minimum UMI per cells. Similar to minimum genes but more dependent of the protocol used.&lt;/li&gt;&#xA;&lt;li&gt;UMI per gene/cell. This can be interesting to detect a potential bias from amplification and is also dependent on the protocol.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The best would be to compare all your samples using violin plots with those different filters and choose your thresholds from there.&#xA;A quick example (using Seurat):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pre filtering:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/67jMn.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/67jMn.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Post filtering:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/9zeEG.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9zeEG.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2156" LastActivityDate="2018-01-09T09:18:38.857" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3259" PostTypeId="1" CreationDate="2018-01-09T09:41:35.720" Score="7" ViewCount="821" Body="&lt;p&gt;What tool could I use to check the ethnicity of a human bam or vcf file? I would like to use the results as a QC check to know whether a given sample or set of samples match the ethnicity information annotated in the meta data or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seen so far:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://genomemedicine.biomedcentral.com/articles/10.1186/gm473&quot; rel=&quot;noreferrer&quot;&gt;Similarity&lt;/a&gt;:  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The examples work, but starting from a GRCh38 .vcf file (from GATK UG) gives me an error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Read    mytest.vcf   :1 Individual(s)Exception in thread &quot;main&quot; java.lang.NumberFormatException: For input string: &quot;&quot;&#xA;        at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)&#xA;        at java.lang.Integer.parseInt(Integer.java:592)&#xA;        at java.lang.Integer.valueOf(Integer.java:766)&#xA;        at Individual.get_Individual(Individual.java:73)&#xA;        at similarity.main(similarity.java:80)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.popgen.dk/software/index.php/NgsRelate&quot; rel=&quot;noreferrer&quot;&gt;NGSrelate&lt;/a&gt;: &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It won't work with GATK UG .vcf files straightaway, it needs to reprocess .bam files using the &lt;a href=&quot;http://popgen.dk/angsd/index.php/Quick_Start&quot; rel=&quot;noreferrer&quot;&gt;ANGSD pipeline&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://genome.sph.umich.edu/wiki/VerifyBamID&quot; rel=&quot;noreferrer&quot;&gt;VerifyBAMID&lt;/a&gt;:  &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This tool does not give ethnicity checks but only genotype checks. Running all-vs-all vcfs vs bams can create a semi-matrix of similarities.&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="48" LastEditDate="2018-01-12T16:00:28.817" LastActivityDate="2018-01-13T22:20:29.933" Title="ethnicity check either from bam or vcf files" Tags="|sam|vcf|genomics|human|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3260" PostTypeId="2" ParentId="2696" CreationDate="2018-01-09T09:53:30.650" Score="0" Body="&lt;p&gt;Even though ococo does not seem to handle big indels well, it's a good enough tool for a first try.&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2018-01-09T09:53:30.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3261" PostTypeId="1" AcceptedAnswerId="3263" CreationDate="2018-01-09T09:56:39.343" Score="6" ViewCount="88" Body="&lt;p&gt;When doing Illumina 2x150bp sequencing of genomic DNA, and after aligning the reads to GRCh38, does the percentage of the non-N fraction of the human genome as MAPQ=0 depend on the insert sizes of the genomic fragments?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is, for two identical samples with identical final coverage, sample A having an average insert size of 250bp and sample B having an average of 450bp, would the fraction of MAPQ=0 change between the two?&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="1775" LastEditDate="2018-01-10T08:03:13.910" LastActivityDate="2018-01-11T02:10:33.020" Title="Does the MAPQ=0 fraction of a BAM file depend on the insert sizes?" Tags="|sam|illumina|mapq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3262" PostTypeId="2" ParentId="3257" CreationDate="2018-01-09T10:17:12.033" Score="2" Body="&lt;p&gt;Let $n$ be the number of rows (in your case $n=5$) and $|O_j|$ the &lt;a href=&quot;https://en.wikipedia.org/wiki/Cardinality&quot; rel=&quot;nofollow noreferrer&quot;&gt;cardinality&lt;/a&gt; of $O_i$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can calculate the number of compatible columns as sum of the number of $O_i \subseteq O_j$, $O_j \subseteq O_i$ and $O_i, O_j$ disjoint.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;number of $O_i \subseteq O_j$:&#xA;Here we need to find the number of supersets of our $O_i$. That means we need to find the number of sets which contain the indices in $O_i$ and for each index not in $O_i$, we have two choices: It is contained, or it is not contained in $O_j$. Therefore the number of supersets is $2^{n-|O_i|}$, which is 2 to the power of elements not in our set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;number of $O_i, O_j$ disjoint:&#xA;Here all elements of $O_i$ are fixed not to be in $O_j$ and for each index not in $O_i$, it can again be in $O_j$ or not. That means that the number of possible sets is again $2^{n-|O_i|}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;number of $O_j \subseteq O_i$:&#xA;Here we need to find the number of subsets of $O_i$ and this is the number of elements in its power set, which is $2^{|O_i|}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So finally, this means that the number of compatible columns can be calculated as: $2^{n-|O_i|} + 2^{n-|O_i|} + 2^{|O_i|} - 2$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why -2? We have counted 2 sets twice: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The empty set, which is a subset of $O_i$ and also counted in the number of disjoint sets&lt;/li&gt;&#xA;&lt;li&gt;and $O_i$, which we counted in $O_i \subseteq O_j$ and in $O_j \subseteq O_i$&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This means for your examle, we have $16 + 2 + 2 - 2 = 18$ compatible sets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; The all zero vector (which corresponds to $O_i=\{\}$) is a special case for which the formula above does not work, as all disjunct sets and all supersets are counted twice. For this special case, the number of compatible columns is simply $2^n$.&lt;/p&gt;&#xA;" OwnerUserId="2047" LastEditorUserId="2047" LastEditDate="2018-01-09T10:50:38.860" LastActivityDate="2018-01-09T10:50:38.860" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3263" PostTypeId="2" ParentId="3261" CreationDate="2018-01-09T10:35:17.187" Score="4" Body="&lt;p&gt;Yes, as a general rule of thumb mappability increases with insert size (up to a limit) and read length. Whether this will actually occur in a given case will depend more on how randomly the sequencing samples from the genome to begin with (i.e., if the library prep happens to select for/against high mappability regions then the insert size won't much matter). The basic reason behind this is that if one end of a read aligns to a repetitive element that the likelihood of being able to use the other end as an anchor will increase a bit if it's further away. Now having said that, Illumina sequencers in particular like a certain fragment length for efficient sequencing, so you can't go too crazy on increasing the insert sizes and still get decent output (unless you modify your library prep).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-09T10:35:17.187" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3264" PostTypeId="2" ParentId="3252" CreationDate="2018-01-09T15:30:13.830" Score="1" Body="&lt;p&gt;All introns will have some level of retention and I guess no intron will be 100% retained. In the absence of a comparator, you will need to work out how much intron retention an intron needs to be called retained. This is a human judgement, there is no automated way of deciding this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you need to do this is some sort of PSI calculation (percent-spliced-in). MISO will provide this, but it doesn't do replicates, so I guess you'll need to run it on each sample and then average the PSIs and determine if it surpasses some sort of threshold you have set. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-01-09T15:30:13.830" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3265" PostTypeId="2" ParentId="2696" CreationDate="2018-01-09T18:42:52.043" Score="2" Body="&lt;p&gt;You should be able to use something like &lt;a href=&quot;https://github.com/broadinstitute/pilon/releases&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pilon&lt;/a&gt; to convert from one genome to another, assuming the changes are all fairly small local changes (but that's almost never the case).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find your resistance to long-read data interesting. It's important to consider the cost of bioinformatics as well. Sequencing a phage genome with long reads will take less than an hour, and probably produce a few reads that cover the entire genome. With existing short-read data, those full-length reads should be able to be fixed up within a day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I spent about half a year trying to fix a eukaryotic genome assembly that was assembled only from short reads (using Pilon, and &lt;a href=&quot;http://www.fishbrowser.org/software/L_RNA_scaffolder/&quot; rel=&quot;nofollow noreferrer&quot;&gt;L-RNA-scaffolder&lt;/a&gt;), and I was only able to reduce the number of contigs down by about half (and even at that, I wasn't convinced that contigs were being properly joined). With long reads and another couple of weeks of computer work, the contigs dropped down to about a tenth of their original number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can use a pipette and have about half an hour spare, you can do rapid long-read sequencing on the MinION.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-01-09T18:42:52.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3266" PostTypeId="2" ParentId="3252" CreationDate="2018-01-09T20:43:04.013" Score="0" Body="&lt;p&gt;The method you linked to is interesting, though I suspect that Ian's answer is both easier and generally sufficient to get what you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, the general idea behind the method you linked to is to use DESeq2 to test for differentially expressed introns. To generate the control samples, they're effectively taking the number of reads across all introns in a gene and distributing them over the introns. They weight the distribution such that longer introns receive more reads (they use an effective length for this, but I'm skeptical that this makes that much of a difference). DESeq2 is then giving the p-values you're after. Unless you can get the authors to send you there code, you'll have to write a bit of code to do all of this...but it shouldn't be too bad.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, you can use the &lt;code&gt;GenomicFeatures&lt;/code&gt; package from bioconductor, specifically the &lt;code&gt;exonsBy(..., by=&quot;gene)&lt;/code&gt; follows by &lt;code&gt;gaps()&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-09T20:43:04.013" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3269" PostTypeId="2" ParentId="2520" CreationDate="2018-01-10T06:10:00.103" Score="1" Body="&lt;p&gt;Heteroplasmy at &lt;em&gt;any&lt;/em&gt; site (as long as it's a reliable variant) suggests that there is mitochondrial heteroplasmy, it doesn't matter where it is. In that case, the &quot;mitochondrial heteroplasmy&quot; statistic would simply be the maximum value at any [reliable] location.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to the low amount of variation in the mitochondrial genome, and because it's a haploid genome, I don't think that a calculation like &quot;average heteroplasmy&quot; makes much sense for mitochondria.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-01-10T06:10:00.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3270" PostTypeId="1" CreationDate="2018-01-10T10:15:39.317" Score="5" ViewCount="87" Body="&lt;p&gt;I'm running the &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/pathifier.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;pathifier&lt;/a&gt; approach against C2 pathway curated&#xA;database for a specific microarray dataset.&lt;/p&gt;&#xA;&lt;p&gt;As I was reading the documentation of the pathifier, in order to configure it properly for my dataset, I saw that for the &lt;code&gt;min_std&lt;/code&gt; argument they suggest to use the technical noise.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;min_std:   The minimal allowed standard deviation of each gene. Genes&#xA;with lower standard deviation are divided by min_std instead of their&#xA;actual standard deviation. (Recommended: set min_std to be the&#xA;technical noise).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But how am I going to calculate it ? At first run I just used the value &lt;code&gt;0.2254005&lt;/code&gt; from the example they provide with that package, but this might not be the right for my dataset.&lt;/p&gt;&#xA;&lt;p&gt;Update:&lt;/p&gt;&#xA;&lt;p&gt;So from &lt;a href=&quot;http://www.weizmann.ac.il/complex/compphys/software/yotam/pathifier/help.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; it says: &lt;code&gt;The minimal standard error (min_std) was set as the first quartile of the standard deviation in the data&lt;/code&gt;. How am I supposed to calculate this first quartile of the std? Any idea? Should I calculate the sd() for each gene and then find the first quartile?&lt;/p&gt;&#xA;&lt;p&gt;What do you think about this &lt;a href=&quot;https://pastebin.com/ni5uKhpX&quot; rel=&quot;nofollow noreferrer&quot;&gt;function&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="704" LastEditorUserId="73" LastEditDate="2021-05-12T02:18:57.080" LastActivityDate="2021-10-09T06:01:50.220" Title="Pathifier bioconductor package recommends to use technical noise for an argument. How to determine it?" Tags="|statistics|microarray|pathway|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3271" PostTypeId="2" ParentId="3116" CreationDate="2018-01-10T13:44:39.267" Score="2" Body="&lt;p&gt;From what I known, there is no clear consensus in the field and it depends on the type of cells you are interrogating. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if mitoRNA/endogenousRNA ratio does not fit for your purposes, other option is to &lt;strong&gt;check at the total number of genes/transcripts detected&lt;/strong&gt; in each cell. In this way, you can filter out cells that have considerable less genes/transcripts detected than the rest since this may be indicative of bad quality cells for any reason (apoptosis, RNA degradation, sequencing itself, etc). For instance, to &lt;strong&gt;put a threshold&lt;/strong&gt; of &gt;1000 genes/transcripts in a cell to be considered for further analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider also to take a look at the number of mapped reads for each cell, as cells with low mapped reads respect to the rest could be potentially problematic.&lt;/p&gt;&#xA;" OwnerUserId="678" LastEditorUserId="678" LastEditDate="2018-01-10T13:53:23.080" LastActivityDate="2018-01-10T13:53:23.080" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3272" PostTypeId="1" AcceptedAnswerId="3285" CreationDate="2018-01-10T14:17:53.673" Score="5" ViewCount="201" Body="&lt;p&gt;I want to understand what the hmmer Profile used in &lt;a href=&quot;http://hmmer.org/&quot; rel=&quot;noreferrer&quot;&gt;hmmer&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;consider the profile &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;HMMER2.0  [converted from 3.1b2]&#xA;NAME  aRNH_profile&#xA;LENG  121&#xA;ALPH  Amino&#xA;RF    no&#xA;CS    no&#xA;MAP   yes&#xA;NSEQ  34&#xA;DATE  Tue Apr 26 21:24:57 2016&#xA;XT       -8455     -4  -1000  -1000  -8455     -4  -8455     -4&#xA;NULT        -4  -8455&#xA;NULE       656  -1722     98    419   -333    475  -1125    239    250    947  -1073   -271    -50   -338    114    451    113    430  -2131   -717&#xA;HMM        A      C      D      E      F      G      H      I      K      L      M      N      P      Q      R      S      T      V      W      Y    &#xA;         m-&amp;gt;m   m-&amp;gt;i   m-&amp;gt;d   i-&amp;gt;m   i-&amp;gt;i   d-&amp;gt;m   d-&amp;gt;d   b-&amp;gt;m   m-&amp;gt;e&#xA;           -6      *  -7932&#xA;     1    -214   -916   -565    694    862  -1448    512   -838    115  -1042    170   1348  -1481    369   -158   -290   -159   -625    -52   1675     1&#xA;     -    -210   -336    220    -37   -342    377     73   -669    210   -511   -732    404    423     70     27    439    205   -414   -161   -176&#xA;     -     -18  -6890  -7932   -894  -1115   -701  -1378    -18      *&#xA;     2     285    990  -2213  -1678   2305  -1819   -698    219  -1289   -140    545  -1167  -1835   -973  -1322   -424   1025    762   -559    377     2&#xA;     -    -210   -336    220    -37   -342    377     73   -669    210   -511   -732    404    423     70     27    439    205   -414   -161   -176&#xA;     -     -18  -6890  -7932   -894  -1115   -701  -1378      *      *&#xA;[....]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;what exactly does each component mean? By definition a HMM $\lambda$ consist of an alphabet $V$ in my case twenty amino acids. Different states $S$ in our case main, insertion and deletion (beginning and end). And two probability matrices of state transitions and emission in certain state. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;now if I look at the profile I can see that it has the allowed transitions&lt;br&gt;&#xA;&lt;code&gt;m-&amp;gt;m   m-&amp;gt;i   m-&amp;gt;d   i-&amp;gt;m   i-&amp;gt;i   d-&amp;gt;m   d-&amp;gt;d   b-&amp;gt;m   m-&amp;gt;e&lt;/code&gt; &#xA;and all the amino acids, but what does each row in the table mean. I checked the manual from the documentation and it says rather ironically &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If you were to look at this file (and you don’t have to – it’s&#xA;  intended for HMMER’s consumption, not yours), you’d see [part of my output]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;are these number translations of the probabilities in $[0,1]$  into another interval $[-x,+y]$? what is the meaning of the three rows under &lt;code&gt;A 1&lt;/code&gt;  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-124  &#xA;-210&#xA; -18&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is this the transition matrix? It would be great if someone could explain this  or point to the correct reference. &lt;/p&gt;&#xA;" OwnerUserId="2026" LastEditorUserId="298" LastEditDate="2018-01-10T14:21:13.433" LastActivityDate="2018-01-11T09:39:39.470" Title="What is the meaning of the Hmmer profile?" Tags="|hidden-markov-models|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3273" PostTypeId="2" ParentId="3247" CreationDate="2018-01-10T14:57:21.637" Score="5" Body="&lt;p&gt;By reading &lt;a href=&quot;http://seqanswers.com/forums/showthread.php?t=42911&quot; rel=&quot;noreferrer&quot;&gt;this thread&lt;/a&gt; on seqanswers and by comparing the data to TCGA, I figured out&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;raw_read_count&lt;/code&gt; is the read count which you use as input for e.g. &lt;code&gt;DESeq2&lt;/code&gt;. It has been estimated using &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-12-323&quot; rel=&quot;noreferrer&quot;&gt;RSEM&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;normalized_read_count&lt;/code&gt; is equivalent to the &lt;code&gt;scaled_estimate&lt;/code&gt; from TCGA. This is the &lt;em&gt;estimated fraction of transcripts made up by a given gene&lt;/em&gt;, as estimated by RSEM. Multiplying this value with 1e6 yields the TPM. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1656" LastActivityDate="2018-01-10T14:57:21.637" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3274" PostTypeId="1" CreationDate="2018-01-10T19:35:07.737" Score="3" ViewCount="84" Body="&lt;p&gt;I'm trying to retrieve the most expressed genes from a cell line from the NCI-60, lets say cell line NCI-H23, but I'm not sure where to find the gene expression data. My main goal is to create a network using that data and iRefIndex to be loaded into Cytoscape. Does anyone know where I can find the correct data or guide me in the right direction?&lt;/p&gt;&#xA;" OwnerUserId="1633" LastActivityDate="2018-03-15T20:39:29.063" Title="Where can I find gene expression data on one of the cell lines in NCI-60?" Tags="|gene|drugs|cell-line|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3275" PostTypeId="1" AcceptedAnswerId="15879" CreationDate="2018-01-10T20:20:03.787" Score="0" ViewCount="48" Body="&lt;p&gt;I was trying to reach the &lt;a href=&quot;http://binddb.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;BIND&lt;/a&gt; (&lt;strong&gt;B&lt;/strong&gt;iomolecular &lt;strong&gt;I&lt;/strong&gt;nteraction &lt;strong&gt;N&lt;/strong&gt;etwork &lt;strong&gt;D&lt;/strong&gt;atabase) but the website is down. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where can I find something similar? Has this platform moved somewhere or what?  &lt;/p&gt;&#xA;" OwnerUserId="2120" LastEditorUserId="3694" LastEditDate="2021-05-07T20:57:56.063" LastActivityDate="2021-05-07T20:57:56.063" Title="Where did the BIND website go?" Tags="|database|interactions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3276" PostTypeId="1" CreationDate="2018-01-10T20:29:40.490" Score="3" ViewCount="858" Body="&lt;p&gt;I have multiplexed pair-end fastq reads with dual barcodes. The issue is that one barcode is present in the header and one is present at the beginning of the read. I need a method to demultiplex this data, but in order to assign a read to an individual, both barcodes are required, as there is overlap between the barcodes. It seems there are packages available to demultiplex using header ID or in-line barcodes to demultiplex, but not both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;example reads:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@700819F:525:HT235BCXX:2:1101:1139:2144 1:N:0:ATCACGAT&#xA;CGAATTGCAGATTTTTTCTGAATAAAGCAGTGCAATAAAATTCCCCGCAAAAACACTTNANNNGNNNNNNNNNNNNNNNNNNNNANNNNNNGNTAATAAA&#xA;+&#xA;GGGGGIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII#&amp;lt;###.####################.######&amp;lt;#.&amp;lt;&amp;lt;GGGI&#xA;@700819F:525:HT235BCXX:2:1101:1212:2172 1:N:0:ATCACGAT&#xA;AAGGATGCAGGGCATCTCCCTCAGGCTGCGCTCTATCGAAGTCATCCCAGAATTAGATTCCGACCACAGACCAGTCTTAGTCAAACTAGGACCCGAGTGT&#xA;+&#xA;GGGGGIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIGIIIIIIIIIIIIIIIIIIIIGIIIIIIGGIIIGIIIIIIIIIIIIIGGIGIGIIIGIGI&#xA;@700819F:525:HT235BCXX:2:1101:1110:2173 1:N:0:ATCACGAT&#xA;GGTTGTGCAGAAAGAGTTGCTGATAAACTTAGCCATGCAGAACAGAATTATGAGTTAGAAGTATGTATATATATACCAATCACTATATCAACCCATTACC&#xA;+&#xA;&amp;lt;G.G&amp;lt;GGIIIG.GA.GAGAAGG&amp;lt;AA&amp;lt;A&amp;lt;.&amp;lt;&amp;lt;&amp;lt;GA&amp;lt;.&amp;lt;&amp;lt;.G&amp;lt;.G&amp;lt;&amp;lt;A&amp;lt;GGAA....&amp;lt;G.G..&amp;lt;&amp;lt;.GA..&amp;lt;A.&amp;lt;GG&amp;lt;&amp;lt;&amp;lt;.&amp;lt;..&amp;lt;&amp;lt;.GG.A..G..&amp;lt;&amp;lt;&amp;lt;.&amp;lt;&amp;lt;G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One of the barcodes is the sequence found at the end of the header line (ATCACGAT). The second barcode is found at the beginning of the sequence (CGAAT in read 1). Both are required for assignment to an individual.&lt;/p&gt;&#xA;" OwnerUserId="2167" LastEditorUserId="48" LastEditDate="2018-02-14T08:09:59.623" LastActivityDate="2018-03-16T08:39:31.650" Title="How to demultiplex pair-end fastq reads with barcode 2 in the identifier line?" Tags="|fastq|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3277" PostTypeId="1" AcceptedAnswerId="3279" CreationDate="2018-01-10T20:53:08.463" Score="0" ViewCount="120" Body="&lt;p&gt;In &lt;a href=&quot;https://www.nature.com/articles/nrg3542&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shapiro et al.&lt;/a&gt;, when discussing about loss of molecules as source of error in single-cell sequencing, it is written that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Another source of error is losses, which can be severe. The detection limit of published protocols is $5$–$10$ molecules of mRNA.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I used to think that the detection limit of a protocol was the average maximum number of molecules detected across cells in a scRNA-seq experiment. However, by working on the dataset provided by &lt;a href=&quot;https://www.biorxiv.org/content/early/2017/12/20/237446&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Tabula Muris Consortium et al. (pp)&lt;/a&gt; I noticed that RNA raw counts often spikes to more than 10 molecules.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the actual meaning of detection limit of a protocol in scRNA-seq?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-01-10T23:23:42.420" Title="What is the detection limit of protocol in scRNA-seq" Tags="|scrnaseq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3278" PostTypeId="2" ParentId="3276" CreationDate="2018-01-10T23:05:26.173" Score="1" Body="&lt;p&gt;Since you've found separate programs to demultiplex by Illumina barcode and in-line barcode (I presume this is actually just the sequence from the cut site), just run one and then the other on its output. Typically the Illumina barcode would have been used to demultiplex the files before you even received them (if not, your sequencing facility is incompetent), in which case you just need to demultiplex by the in-line barcode&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-10T23:05:26.173" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3279" PostTypeId="2" ParentId="3277" CreationDate="2018-01-10T23:23:42.420" Score="1" Body="&lt;p&gt;The detection limit refers to the ability to detect a given gene/transcript. In this case, they're saying that a given transcript requires 5-10 copies in order to be likely detected (in other words, the loss rate is 80-90%, which is directly related to &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/3126/how-to-compute-the-chance-of-failing-to-detect-a-gene-given-the-detection-limit&quot;&gt;a previous question of yours&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-10T23:23:42.420" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3280" PostTypeId="1" AcceptedAnswerId="3281" CreationDate="2018-01-11T00:18:59.803" Score="4" ViewCount="537" Body="&lt;p&gt;Given a aligned bam file (wgs/bwa-mem) what steps do i need to perform to generate a plot as seen below (from this &lt;a href=&quot;http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0066621&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DQS8b.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DQS8b.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are trying to see how much of genome was covered using pacbio and illumina for cow.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;You mean remove this left plot?&#xA;&lt;a href=&quot;https://i.stack.imgur.com/JlgrX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JlgrX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="926" LastEditorUserId="48" LastEditDate="2018-01-11T08:19:16.990" LastActivityDate="2018-01-11T10:12:32.983" Title="Plot percentage of genome covered" Tags="|wgs|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3281" PostTypeId="2" ParentId="3280" CreationDate="2018-01-11T00:24:27.057" Score="5" Body="&lt;p&gt;The general steps are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Install &lt;a href=&quot;http://deeptools.readthedocs.io/en/latest/content/installation.html&quot; rel=&quot;noreferrer&quot;&gt;deepTools&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Run &lt;a href=&quot;http://deeptools.readthedocs.io/en/latest/content/tools/plotCoverage.html&quot; rel=&quot;noreferrer&quot;&gt;plotCoverage&lt;/a&gt; on your BAM files&lt;/li&gt;&#xA;&lt;li&gt;Remove the left-most plot in photoshop/gimp/imagemagick&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-11T00:24:27.057" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3282" PostTypeId="2" ParentId="3261" CreationDate="2018-01-11T02:10:33.020" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;This is, for two identical samples with identical final coverage, sample A having an average insert size of 250bp and sample B having an average of 450bp, would the fraction of MAPQ=0 change between the two?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For human, I &lt;em&gt;expect&lt;/em&gt; sample B will have fewer MAPQ=0 reads (I don't have concrete data to prove). A large part of human genome consists of ALU repeats. The average size of an ALU is ~320bp. If your insert size is ~450bp, you can map reads into most of isolated ALUs in principle. With ~250bp insert, fragments in the middle of ALUs won't be mappable uniquely.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that to see the difference, you need to use a mapper that sufficiently takes the advantage of paired-end information.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-01-11T02:10:33.020" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3283" PostTypeId="2" ParentId="774" CreationDate="2018-01-11T04:06:30.943" Score="1" Body="&lt;p&gt;Additional insights can frequently be made in research by studying things at a lower level. By peeling back the higher layers, effects can be seen that are hidden (or reversed) when looking at aggregate data. This is generally referred to as &lt;a href=&quot;https://en.wikipedia.org/wiki/Simpson%27s_paradox&quot; rel=&quot;nofollow noreferrer&quot;&gt;Simpson's Paradox&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We know that RNA is functional at an isoform/transcript level, and in some cases the expression of different transcripts from the same gene can have different biologically-relevant outcomes. By looking at transcript-level information, even if the transcripts for a gene average out to be a similar level between different treatment conditions, variation at an individual transcript level can still be discovered. This is important when looking at differential expression of &lt;em&gt;populations&lt;/em&gt; of cells, because it can clue researchers into finding out that a particular cell sub-population might be of interest; single cells typically will only produce single isoforms for each gene at a specific point in time.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-01-11T04:06:30.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3284" PostTypeId="2" ParentId="3280" CreationDate="2018-01-11T09:01:50.710" Score="1" Body="&lt;p&gt;Devon gives a practical answer using the great deepTools package. These should be the theoretical, lower level steps behind the plots:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Compute read depth at each genomic position, using for example &lt;code&gt;samtools mpileup&lt;/code&gt; with appropriate flags to accept/reject reads. You get a table like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Pos       Depth&#xA;chr1:1    0&#xA;chr1:2    10&#xA;chr1:3    11&#xA;...&#xA;chrX:100  10&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For each unique value &lt;em&gt;x&lt;/em&gt; in the &lt;em&gt;Depth&lt;/em&gt; column count how many genomic positions have depth &gt;= &lt;em&gt;x&lt;/em&gt;. You can use Unix tools like &lt;code&gt;sort ... | uniq -c | awk ...&lt;/code&gt; for this. You get a table like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Depth   N bases with depth &amp;gt;= x   % bases with depth &amp;gt;= x&#xA;0       (all genomic positions)   100 %&#xA;1       1000                      90 %&#xA;2       995                       85 %&#xA;.&#xA;.&#xA;.&#xA;N       2                         0.1 %                         &#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now just plot column &lt;code&gt;Depth&lt;/code&gt; (x) vs &lt;code&gt;% bases with depth &amp;gt;= x&lt;/code&gt; (y) with R or you favourite plotting tool.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Did I get it right?&lt;/p&gt;&#xA;" OwnerUserId="339" LastEditorUserId="48" LastEditDate="2018-01-11T10:12:32.983" LastActivityDate="2018-01-11T10:12:32.983" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3285" PostTypeId="2" ParentId="3272" CreationDate="2018-01-11T09:39:39.470" Score="1" Body="&lt;p&gt;The tutorial in the &lt;a href=&quot;http://eddylab.org/software/hmmer3/3.1b2/Userguide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; does indeed state what you quoted but a full description of the file format is given in Section 8 of the documentation, starting on page 106.&lt;/p&gt;&#xA;&lt;p&gt;To summarise, as you noted, the number columns A-K is the alphabet size (the number of amino acids). The number of rows underneath is the number of nodes, which corresponds to the number of match states. The three numbers (-214, -210 and -18 in your example) are: the match emission, insert emission, and state transition values.&lt;/p&gt;&#xA;&lt;p&gt;Caveat: The HMMER profile HMM format changes frequently between HMMER versions (even between minor versions) so there is no guarantee this information is correct for all the older file formats.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Quoting the relevant part of the documentation on page 109:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;main model section&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;All the remaining fields are mandatory. The first&#xA;two lines in the main model section are atypical.‡ They contain&#xA;information for the core model’s BEGIN node. This is stored as model&#xA;node 0, and match state 0 is treated as the BEGIN state. The begin&#xA;state is mute, so there are no match emission probabilities. The first&#xA;line is the insert 0 emissions. The second line contains the&#xA;transitions from the begin state and insert state 0. These seven&#xA;numbers are: B → M1, B → I0, B → D1; I0 → M1, I0 → I0; then a 0.0 and&#xA;a ’&lt;em&gt;’, because by convention, nonexistent transitions from the&#xA;nonexistent delete state 0 are set to log 1 = 0 and log 0 = −∞ = ‘&lt;/em&gt;’.&lt;/p&gt;&#xA;&lt;p&gt;The remainder of the model has three lines per node, for M nodes&#xA;(where M is the number of match states, as given by the LENG line).&#xA;These three lines are (K is the alphabet size in residues):&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Match emission line&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The first field is the node number (1 . . . M). The&#xA;parser verifies this number as a consistency check (it expects the&#xA;nodes to come in order). The next K numbers for match emissions, one&#xA;per symbol, in alphabetic order. The next field is the MAP annotation&#xA;for this node. If MAP was yes in the header, then this is an integer,&#xA;representing the alignment column index for this match state&#xA;(1..alen); otherwise, this field is ‘-’. The next field is the CONS&#xA;consensus residue for this node. If CONS was yes in the header, then&#xA;this is a single character, representing the consensus residue&#xA;annotation for this match state; otherwise, this field is ‘-’. The&#xA;next field is the RF annotation for this node. If RF was yes in the&#xA;header, then this is a single character, representing the reference&#xA;annotation for this match state; otherwise, this field is ‘-’. The&#xA;next field is the MM mask value for this node. If MM was yes in the&#xA;header, then this is a single ’m’ character, indicating that the&#xA;position was identified as a masked position during model&#xA;construction; otherwise, this field is ‘-’. The next field is the CS&#xA;annotation for this node. If CS was yes, then this is a single&#xA;character, representing the consensus structure at this match state;&#xA;otherwise this field is ‘-’.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Insert emission line&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The K fields on this&#xA;line are the insert emission scores, one per symbol, in alphabetic&#xA;order.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;State transition line&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The seven fields on this line are the&#xA;transitions for node k, in the order shown by the transition header&#xA;line: Mk → Mk+1, Ik, Dk+1; Ik → Mk+1, Ik; Dk → Mk+1, Dk+1. For&#xA;transitions from the final node M, match state M + 1 is interpreted as&#xA;the END state E, and there is no delete state M + 1; therefore the&#xA;final Mk → Dk+1 and Dk → Dk+1 transitions are always * (zero&#xA;probability), and the final Dk → Mk+1 transition is always 0.0&#xA;(probability 1.0).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="104" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-01-11T09:39:39.470" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3286" PostTypeId="1" CreationDate="2018-01-11T12:08:33.913" Score="1" ViewCount="58" Body="&lt;p&gt;I'm having a hard time retrieving the most expressed genes in a cell line. One of the databases I've found has values for a lot of genes in 60 different cell lines and it seems the data is obtained using GC-RMA. The data is available &lt;a href=&quot;http://biogps.org/downloads/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; in the file &lt;em&gt;NCI60_U133A_20070815.raw.csv&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My problem is what does the values actually represent? Would it be correct to assume the genes with high values are the ones most expressed in the corresponding cell line?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've found a short explanation on why there is no unit definition on the page &lt;a href=&quot;https://groups.google.com/forum/#!searchin/biogps/units/biogps/ljMdJdfkUhM/P2q4gp1cBqMJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a snapshot of the file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-            huh-7    huh-7      HEK293  ...&#xA;1007_s_at    31.727    55.551    360.731&#xA;1053_at     402.633   441.622    557.782&#xA;117_at        9.968    10.178      9.565&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1633" LastEditorUserId="48" LastEditDate="2018-01-17T15:41:45.267" LastActivityDate="2018-03-18T16:49:44.980" Title="Gene expression data in GC-RMA format?" Tags="|database|gene|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3287" PostTypeId="2" ParentId="3274" CreationDate="2018-01-11T13:55:09.937" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://discover.nci.nih.gov/cellminer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CellMiner&lt;/a&gt; portal has plenty of data about NCI-60 cell lines, inclusive microarray gene-expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, NCI-60 is a subset of the ~1000 cell lines covered by the &lt;a href=&quot;http://www.cancerrxgene.org/downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;GDSC project&lt;/a&gt; for which you have multiple types of measurements (e.g. copy-number, mutation, gene-expression).&lt;/p&gt;&#xA;" OwnerUserId="193" LastActivityDate="2018-01-11T13:55:09.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3288" PostTypeId="2" ParentId="3276" CreationDate="2018-01-11T16:48:43.597" Score="2" Body="&lt;p&gt;You could try something like that:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;#!/bin/sh&#xA;&#xA;fq_in=&quot;${1}&quot;&#xA;&#xA;bioawk -c fastx '{print substr($comment, 7, 15)}' ${fq_in} | uniq \&#xA;    &amp;gt; &quot;barcode1.txt&quot;&#xA;bioawk -c fastx '{print substr($seq, 1, 5)}' ${fq_in} | uniq \&#xA;    &amp;gt; &quot;barcode2.txt&quot;&#xA;&#xA;for bc1 in $(cat &quot;barcode1.txt&quot;)&#xA;do&#xA;    for bc2 in $(cat &quot;barcode2.txt&quot;)&#xA;    do&#xA;        fq_out=&quot;${bc1}_${bc2}.fq&quot;&#xA;        bioawk -c fastx -v bc1=${bc1} -v bc2=${bc2} \&#xA;            'substr($comment, 7, 15) == bc1 &amp;amp;&amp;amp; substr($seq, 1, 5) == bc2 {print &quot;@&quot;$name&quot; &quot;$comment&quot;\n&quot;$seq&quot;\n+\n&quot;$qual}' \&#xA;            ${fq_in} \&#xA;            &amp;gt; ${fq_out}&#xA;    done&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This might be slow. You can a little bit of time if you directly provide lists of barcodes for the for loops.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It could be more efficient to write a custom program that can write to multiple files. It happens that I wrote something that does this in python just yesterday, but only reading the inline barcode. Modifying it to also use the in-header barcode is likely simple.&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2018-01-11T16:48:43.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3289" PostTypeId="1" CreationDate="2018-01-11T18:11:00.163" Score="4" ViewCount="1375" Body="&lt;p&gt;I have paired-end RNA-seq samples and a list of intron coordinates in bed format. I want to count the intronic reads such that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1. The reads should overlap the intron by atleast 25 bp &#xA;2. If both the reads of the pair are within the intron, count it as 1.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What tool can I use to compute it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="2040" LastActivityDate="2018-01-14T12:48:07.493" Title="Counting reads within the intron" Tags="|rna-seq|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3290" PostTypeId="2" ParentId="3289" CreationDate="2018-01-11T19:06:07.157" Score="3" Body="&lt;p&gt;You could use &lt;a href=&quot;https://github.com/bedops/bedops&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS&lt;/a&gt; &lt;code&gt;bedmap&lt;/code&gt; to map reads to introns, using 1) the &lt;code&gt;--count&lt;/code&gt; operator to do counting of reads overlapping by your criteria; and, 2) the &lt;code&gt;--indicator&lt;/code&gt; operator to do a true/false operation, where reads are contained entirely within the intron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, to count reads that overlap introns by at least 25 bases, use &lt;code&gt;--count&lt;/code&gt; to count and &lt;code&gt;--bp-over 25&lt;/code&gt; to specify the minimum overlap threshold:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --echo --count --bp-ovr 25 --delim '\t' introns.bed reads.bed &amp;gt; answer.counts.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To filter for introns where reads are contained entirely within the intron, use &lt;code&gt;--indicator&lt;/code&gt; to get introns that meet the criteria of full containment of reads, via &lt;code&gt;--fraction-map 1&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --indicator --echo --fraction-map 1 --delim '\t' introns.bed reads.bed | awk '($1==1)' | cut -f2- &amp;gt; answer.indicator.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Once you have &lt;code&gt;answer.counts.bed&lt;/code&gt; and &lt;code&gt;answer.indicator.bed&lt;/code&gt;, you can compare those two results with &lt;code&gt;bedmap --indicator --exact&lt;/code&gt; to see where there are commonalities (true/1) and differences (false/0), and increment the final count where there are identical introns, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --indicator --echo --exact --delim '\t' answer.counts.bed answer.indicator.bed | awk '{ $NF += $1; print $0; }' | cut -f2- &amp;gt; answer.final.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If your reads are in a different format, i.e. BAM, you can use &lt;code&gt;bam2bed&lt;/code&gt; in a process substitution, replacing &lt;code&gt;reads.bed&lt;/code&gt; with &lt;code&gt;&amp;lt;(bam2bed &amp;lt; reads.bam)&lt;/code&gt;, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --echo --count --bp-ovr 25 --delim '\t' introns.bed &amp;lt;(bam2bed &amp;lt; reads.bam) &amp;gt; answer.counts.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also just convert separately, so that you only need to do it once:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bam2bed &amp;lt; reads.bam &amp;gt; reads.bed&#xA;$ bedmap --echo --count --bp-ovr 25 --delim '\t' introns.bed reads.bed &amp;gt; answer.counts.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Etc.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2018-01-11T19:06:07.157" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3291" PostTypeId="2" ParentId="2742" CreationDate="2018-01-12T01:55:04.370" Score="3" Body="&lt;p&gt;You can use imputation for guessing at what the missing SNPs might be based on known LD patterns in populations. This procedure will give you an idea of whether the recombinational history of genomic regions is important, rather than specific SNPs. Given that you're looking at LD-based measures, this should be okay (although the logic behind the calculation is a bit circular).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't have a big compute cluster available, the &lt;a href=&quot;https://imputationserver.sph.umich.edu/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Michigan Imputation Server&lt;/a&gt; is a free imputation service that can be used for academic purposes. Data is fed into the service as bgzip-compressed VCF files (output from &lt;code&gt;plink&lt;/code&gt; as an uncompressed VCF file, then use &lt;code&gt;bgzip&lt;/code&gt; to compress the files), and returned as a larger dataset with imputed SNPs derived from the HRC or 1kg reference panels. There's a bit of initial legwork required: you need to make sure that the variants as defined in the VCF file have the same location and strand as defined in the reference panel, otherwise you'll wait weeks in the queue and end up empty handed. To avoid hassle, I'd recommend removing any complementary SNPs (e.g. A/T, C/G), then flipping the strand for any remaining genotypes where the alleles don't match up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've recently done this again. Here's a summary of the things required to get MIS imputation working properly from a PLINK dataset:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Retrieve site data for HRC panel&lt;/li&gt;&#xA;&lt;li&gt;Generate list of complementary variants, rename chrX to chr23 to fit PLINK expectation&lt;/li&gt;&#xA;&lt;li&gt;extract out variant names (remove duplicated rs numbers)&lt;/li&gt;&#xA;&lt;li&gt;change 'RS' to 'rs' in plink BIM file&lt;/li&gt;&#xA;&lt;li&gt;filter PLINK file against variants&lt;/li&gt;&#xA;&lt;li&gt;adjust position of variants to fit with HRC panel&lt;/li&gt;&#xA;&lt;li&gt;sort HRC variants by rs number&lt;/li&gt;&#xA;&lt;li&gt;identify variants that need flipping&lt;/li&gt;&#xA;&lt;li&gt;flip variants using PLINK, create combined VCF file&lt;/li&gt;&#xA;&lt;li&gt;create VCF index for bcftools&lt;/li&gt;&#xA;&lt;li&gt;extract per-chromosome VCF files&lt;/li&gt;&#xA;&lt;li&gt;convert chromosome '23' back to chromosome 'X'&lt;/li&gt;&#xA;&lt;li&gt;upload to MIS, remembering to separate out autosomal and X-chromosomal files&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;And a full example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;## Retrieve site data for HRC panel&#xA;wget 'ftp://ngs.sanger.ac.uk/production/hrc/HRC.r1-1/HRC.r1-1.GRCh37.wgs.mac5.sites.vcf.gz'&#xA;## Generate list of complementary variants, rename chrX to chr23 to fit PLINK expectation&#xA;zgrep -v '^#' HRC.r1-1.GRCh37.wgs.mac5.sites.vcf.gz | cut -f 1-5 | \&#xA;  awk '{if(!($4 ~ /[AT]/ &amp;amp;&amp;amp; $5 ~ /[AT]/) &amp;amp;&amp;amp; !($4 ~ /[CG]/ &amp;amp;&amp;amp; $5 ~ /[CG]/)){print}}' | perl -pe 's/^X/23/' &amp;gt; HRC_complementary_variants.tsv&#xA;## extract out variant names (remove duplicated rs numbers)&#xA;cut -f 3 HRC_complementary_variants.tsv | grep '^rs' | uniq -u &amp;gt; HRC_complementary_variants_namesOnly.txt&#xA;## change 'RS' to 'rs' in plink BIM file&#xA;perl -i -pe 's/RS/rs/' plinkFileHg19.bim&#xA;## filter PLINK file against variants&#xA;plink --bfile plinkFileHg19 --extract HRC_complementary_variants_namesOnly.txt --make-bed --out noComp_plinkFileHg19&#xA;## adjust position of variants to fit with HRC panel&#xA;plink --bfile noComp_plinkFileHg19 --update-chr HRC_complementary_variants.tsv 1 3 \&#xA;  --update-map HRC_complementary_variants.tsv 2 3 --make-bed --out repositioned_nc_plinkFileHg19&#xA;## sort HRC variants by rs number&#xA;sort -k 3,3 HRC_complementary_variants.tsv &amp;gt; rsSorted_HRC_complementary_variants.tsv&#xA;## identify variants that need flipping&#xA;join -1 3 -2 2 &amp;lt;(pv rsSorted_HRC_complementary_variants.tsv) &amp;lt;(sort -k 2,2 repositioned_nc_plinkFileHg19.bim) | \&#xA;  awk '{if(!($4 == $9) &amp;amp;&amp;amp; !($4 == $8)){print $1}}' &amp;gt; toFlip_HRC_complementary_variants.txt&#xA;## flip variants using PLINK, create combined VCF file&#xA;plink --bfile repositioned_nc_plinkFileHg19 --flip toFlip_HRC_complementary_variants.txt --recode vcf bgz --out flipped_rnc_plinkFileHg19&#xA;## create VCF index for bcftools&#xA;bcftools index flipped_rnc_plinkFileHg19.vcf.gz&#xA;## extract per-chromosome VCF files&#xA;for x in $(seq 1 23); do echo ${x};&#xA;  bcftools filter -r ${x} flipped_rnc_plinkFileHg19.vcf.gz -O z -o chr${x}_flipped_rnc_plinkFileHg19.vcf.gz;&#xA;done&#xA;## convert chromosome '23' back to chromosome 'X'&#xA;zcat chr23_flipped_rnc_plinkFileHg19.vcf.gz | perl -pe 's/ID=23/ID=X/;s/^23\t/X\t/' &amp;gt; chrX_flipped_rnc_plinkFileHg19.vcf&#xA;bgzip chrX_flipped_rnc_plinkFileHg19.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-02-01T04:07:39.073" LastActivityDate="2018-02-01T04:07:39.073" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3292" PostTypeId="2" ParentId="2914" CreationDate="2018-01-12T02:16:09.503" Score="1" Body="&lt;p&gt;The researchers behind &lt;a href=&quot;http://fold.it/portal/blog&quot; rel=&quot;nofollow noreferrer&quot;&gt;FoldIt&lt;/a&gt; have the occasional &lt;em&gt;design challenges&lt;/em&gt; that use human game players to help design proteins to fit a structure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, even with a known protein sequence there's still a lot of labwork involved in trying to find a good working DNA sequence. Just generating the DNA sequence is a difficult task due to redundancy in the genetic code (i.e. 64 codons for 20 amino acids + stop signals). When that is combined with cell-specific tRNA frequencies that can influence the timing of amino acid incorporation (and therefore how a protein folds as it is created), creating a sequence that reliably fits a shape ends up being quite difficult.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-01-12T02:16:09.503" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3293" PostTypeId="2" ParentId="2561" CreationDate="2018-01-12T02:29:43.930" Score="3" Body="&lt;p&gt;Answers from Biostars:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/108669/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GeneSCF&lt;/a&gt; 'prepare_database' module will extract all the pathways with corresponding genes as simple table format in plain text file. Check 'Two step process' (sub-heading) from the provided link (First step will do your job).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;You can use &lt;a href=&quot;http://www.kegg.jp/kegg/docs/keggapi.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;KEGG REST API&lt;/a&gt; for extracting gene names from the pathway. &lt;a href=&quot;http://docs.python-requests.org/en/master/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python request library&lt;/a&gt; can be used to make the REST calls to KEGG REST API and query information. Please follow &lt;a href=&quot;http://www.pythonforbeginners.com/requests/using-requests-in-python&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt; if you are a newbie to the Python programming and REST web services.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-01-12T02:29:43.930" CommentCount="0" CommunityOwnedDate="2018-01-12T02:29:43.930" ContentLicense="CC BY-SA 3.0" />
  <row Id="3294" PostTypeId="1" AcceptedAnswerId="3295" CreationDate="2018-01-12T04:21:55.360" Score="1" ViewCount="542" Body="&lt;p&gt;Where can I find a list of SNP rs IDs that belong to the X chromosome? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to exclude the SNPs on X-chromosome in a data set, but I don't know where these SNPs come from. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp?term=X%5BChromosome%5D&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbSNP&lt;/a&gt; to search, it returns 26933249 SNPs, but the webpage seems to refuse me to download the information. &lt;/p&gt;&#xA;" OwnerUserId="1082" LastActivityDate="2018-01-14T11:35:57.607" Title="Where can I find a list of SNP rs IDs that belong to the X chromosome?" Tags="|snp|phylogenetics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3295" PostTypeId="2" ParentId="3294" CreationDate="2018-01-12T10:25:10.107" Score="6" Body="&lt;p&gt;You just searched for &lt;code&gt;X[Chromosome]&lt;/code&gt;, you didn't specify a species. Presumably, your dataset comes from a specific species, so you should limit your search to that species. For instance, for human, you can search dbSNP for &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/?term=X%5BChromosome%5D%20human%5Borganism%5D&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;X[Chromosome] human[organism]&lt;/code&gt;&lt;/a&gt;. Then, click on the &lt;code&gt;send to&lt;/code&gt; link at the top left and download the data: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BvgJG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BvgJG.png&quot; alt=&quot;dbSNP screenshot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is veeeeeery slow, however. I suggest that you instead download the data for chrX from dbSNP's FTP site (this assumes your species of interest is human): &lt;a href=&quot;ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606/chr_rpts/chr_X.txt.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606/chr_rpts/chr_X.txt.gz&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have this file, you can use it to filter your dataset:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Collect all rsIDs from the file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zcat chr_X.txt.gz | awk '/^[0-9]/{print &quot;rs&quot;$1}' &amp;gt; chrX.rsIDs&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Filter your dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You haven't shown the format of your dataset, so if it is just a list of rsIDs like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat dataset&#xA;rs77669712&#xA;rs199706910&#xA;rs2799940 &#xA;rs12205639&#xA;rs10958205&#xA;rs200778574&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can remove any that are on chrX with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -vxFf chrX.rsIDs dataset &#xA;rs77669712&#xA;rs199706910&#xA;rs12205639&#xA;rs10958205&#xA;rs200778574&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That will print all rsIDs from the text file &lt;code&gt;dataset&lt;/code&gt; that are not also in the text file &lt;code&gt;chrX.rsIDs&lt;/code&gt;. Note the &lt;code&gt;-x&lt;/code&gt; option for &lt;code&gt;grep&lt;/code&gt; which tells it to only consider matches that cover the entire line. This ensures that &lt;code&gt;rs1234&lt;/code&gt; is not removed when you're looking for &lt;code&gt;rs123&lt;/code&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-01-14T11:35:57.607" LastActivityDate="2018-01-14T11:35:57.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3296" PostTypeId="2" ParentId="3253" CreationDate="2018-01-12T16:18:38.037" Score="4" Body="&lt;p&gt;I think the analysis will involve two steps:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1) Compile a reference list&lt;/strong&gt; of cell types, with a list of genes expressed in each of them, based on the literature. &lt;em&gt;C. elegans&lt;/em&gt; has only ~1000 cells, so I'm sure it's been well researched. Mark which genes are just expressed in that particular cell type, and which are uniquely expressed, this latter will be more useful markers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Resources to do that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://openworm.org/science.html#other&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://openworm.org/science.html#other&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.wormatlas.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.wormatlas.org/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.wormbase.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.wormbase.org&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Pubmed.&lt;/li&gt;&#xA;&lt;li&gt;Your lab's knowledge.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This will involve mostly manual work, but maybe you can find some list.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, platelets express &lt;em&gt;VWF&lt;/em&gt;. And activated platelets express &lt;em&gt;SELP&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2) Use one of the single-cell RNA-seq packages&lt;/strong&gt; (e.g. &lt;a href=&quot;http://satijalab.org/seurat/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seurat&lt;/a&gt;) to cluster cells and to find the &quot;marker&quot; (DE) genes for each cluster (vs. all other clusters).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each cluster, look up the marker genes in the reference list and infer the cell type.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may need to infer the cell type from the functions and descriptions of the genes.&#xA;Also see &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2634/retrieve-detailed-gene-descriptions&quot;&gt;Retrieve detailed gene descriptions&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keep in mind that you will have many confusing doublets in the dataset expressing characteristics (genes) of two cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may also need to re-do t-SNE and clustering on a subset of cells, e.g. blood/immune cells, in an iterative, hierarchical way.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Do you have information on which cells are from the same worm? Once you have a reference list, you can try and identify cells of just one worm.&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2018-01-12T16:18:38.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3297" PostTypeId="1" AcceptedAnswerId="3298" CreationDate="2018-01-12T17:47:38.273" Score="1" ViewCount="1713" Body="&lt;p&gt;I have a bed file like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    12227   12612   LOC102725121    0   +&#xA;chr1    12721   13220   LOC102725121    0   +&#xA;chr1    12227   12612   DDX11L1 0   +&#xA;chr1    12721   13220   DDX11L1 0   +&#xA;chr1    14829   14969   WASH7P  0   -&#xA;chr1    15038   15795   WASH7P  0   -&#xA;chr1    15947   16606   WASH7P  0   -&#xA;chr1    16765   16857   WASH7P  0   -&#xA;chr1    17055   17232   WASH7P  0   -&#xA;chr1    17368   17605   WASH7P  0   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am trying to convert into gtf file for downstream analysis. Gtf file should look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1   myIntrons   feature 3073253   3074322   .   +   .     gene_id &quot;intron1&quot;; transcript_id &quot;intron1&quot;&#xA;chr1   myIntrons   feature 3075858   3079636   .   +   .     gene_id &quot;intron2&quot;; transcript_id &quot;intron2&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For the gene_id part in the 9th column in the double quotes I want my 4th column and in transcript_id I want combination of column1,column2,and column3 so that my output looks like this:&#xA;Showing for the first line&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    myIntrons   intron  12228   12612   .   +   .   gene_id &quot;LOC102725121&quot;; transcript_id &quot;chr1:12227-12612&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am using this code but under double quotes it is not expanding &lt;code&gt;$1 $2&lt;/code&gt; etc&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -v OFS=&quot;\t&quot; '{print $1,&quot;myIntrons&quot;,&quot;intron&quot;, $2+1, $3, &quot;.&quot;, $6, &quot;.&quot;, &quot;gene_id $4; transcript_id $1:$2-$3&quot;}' file.bed| head&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2040" LastEditorUserId="73" LastEditDate="2023-01-15T20:20:19.037" LastActivityDate="2023-01-15T20:20:19.037" Title="bed file to gtf conversion using awk" Tags="|awk|gtf|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3298" PostTypeId="2" ParentId="3297" CreationDate="2018-01-12T18:38:31.357" Score="2" Body="&lt;p&gt;Instead of the &lt;code&gt;&quot;gene_id ...&quot;&lt;/code&gt; bit, you want &lt;code&gt;sprintf(&quot;gene_id \&quot;%s\&quot;; transcript_id \&quot;%s:%s-%s\&quot;&quot;, $4, $1, $2, $3)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-12T18:38:31.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3299" PostTypeId="1" CreationDate="2018-01-13T18:06:10.087" Score="3" ViewCount="212" Body="&lt;p&gt;I have the UNIPROT IDs, PDB IDs and FASTA files of several known proteins. I am looking for a tool that can take as input the protein's amino acid sequence and display the coding nucleotides of those amino acids. Moreover, I look for a tool that would be able to display those nucleotides' location on the human genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final outcome I desire to achieve is to find SNPs in the exons of the protein.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A tool that can handle several amino acid sequences at once is preferred for I have plenty of proteins. I've tried to use UCSC's BLAT human genome search but it was too cumbersome. I've also tried BLAST, but I'd like a more &quot;straightforward&quot; method for displaying the genome locus for each protein. By the way, I've tried to use UCSC's Table Browser but couldn't query several genes simultaneously. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your assistance.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="73" LastEditDate="2018-01-14T03:53:38.497" LastActivityDate="2018-01-14T20:33:57.887" Title="Is there a tool that can take a protein's amino acid sequence and would display it's locus on the genome?" Tags="|fasta|sequence-alignment|snp|pdb|uniprot|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3300" PostTypeId="2" ParentId="3259" CreationDate="2018-01-13T18:25:30.483" Score="8" Body="&lt;p&gt;The main difficulty here is the use of GRCh38.  Unfortunately, despite the fact that it's more than four years old, the major ethnicity-labeled public datasets (1000 Genomes, gnomAD when allele frequencies are enough) still aren't available for that reference.  It is necessary to perform a liftover operation, or just use overlapping rsIDs and hope for the best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's suppose you go with overlapping rsIDs, and a list of those rsIDs, one per line is in 'rsids.txt'.  (Due to strand flips between reference builds, you may want to restrict this list so that all allele codes match, and it's also reasonable to throw out A/T and C/G SNPs.)  Then, the following process would work:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Download ADMIXTURE (&lt;a href=&quot;https://www.genetics.ucla.edu/software/admixture/download.html&quot; rel=&quot;noreferrer&quot;&gt;https://www.genetics.ucla.edu/software/admixture/download.html&lt;/a&gt; ), plink (&lt;a href=&quot;https://www.cog-genomics.org/plink/1.9/&quot; rel=&quot;noreferrer&quot;&gt;https://www.cog-genomics.org/plink/1.9/&lt;/a&gt; ), and/or chr1-chr22 of 1000 Genomes phase 3 (&lt;a href=&quot;ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/&quot; rel=&quot;noreferrer&quot;&gt;ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/&lt;/a&gt; ) if you don't already have them.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use plink to extract the overlapping higher-MAF rsIDs from each 1000 Genomes phase 3 VCF (&quot;plink --vcf ... --extract rsids.txt --maf 0.05 --make-bed --out ...&quot;), and then merge the resulting per-chromosome filesets (&quot;plink --merge-list ... --out merged_phase3_subset&quot;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;ADMIXTURE prefers a dataset with around 100k variants.  Unless you have a small list of overlapping rsIDs (in which case you had better use another method), you should have more than that.  plink's LD-pruning function is a good way to select a subset for ADMIXTURE's use:&#xA;&quot;plink --bfile merged_phase3_subset --indep-pairwise 500kb 1 0.2; plink --bfile merged_phase3_subset --extract plink.prune.in --make-bed --out admixture_data&quot;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Adjust the 0.2 threshold as necessary to keep the right number of variants.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Run ADMIXTURE in unsupervised mode (&quot;admixture admixture_data.bed 5 -j8&quot;; adjust the -j parameter depending on the number of processor cores).  This generates an admixture_data.5.P file with population allele frequencies, and an admixture_data.5.Q file with sample population assignments.  Verify that the .Q file corresponds to the 1000 Genomes phase 3 'superpopulations'.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Convert your data to plink-format if necessary, keeping just the overlapping rsIDs and 'downgrading' to GRCh37 coordinates.  Make sure your variants are sorted in GRCh37-coordinate order, and the allele order also matches that in your reference dataset (&quot;--a2-allele admixture_data.bim 6 2&quot; during your final --make-bed operation will do the trick); otherwise ADMIXTURE won't do the right thing.  Then run ADMIXTURE in projection mode (&quot;cp admixture_data.5.P my_data.5.P.in; admixture -P my_data.bed 5&quot;).  my_data.5.Q will then have the ethnicity estimates you're looking for.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="472" LastEditorUserId="472" LastEditDate="2018-01-13T22:20:29.933" LastActivityDate="2018-01-13T22:20:29.933" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3301" PostTypeId="2" ParentId="3299" CreationDate="2018-01-13T20:57:22.803" Score="1" Body="&lt;p&gt;Have you tried &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/blast/Blast.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;BLAST&lt;/a&gt;, or specifically &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/blast/Blast.cgi?PROGRAM=tblastn&amp;amp;PAGE_TYPE=BlastSearch&amp;amp;LINK_LOC=blasthome&quot; rel=&quot;nofollow noreferrer&quot;&gt;tBLASTn&lt;/a&gt;?  This will take a protein and compare it any set of reference genomes.  You can also BLAST multiple sequences at once.&lt;/p&gt;&#xA;" OwnerUserId="964" LastEditorUserId="73" LastEditDate="2018-01-14T03:51:59.763" LastActivityDate="2018-01-14T03:51:59.763" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3302" PostTypeId="2" ParentId="3289" CreationDate="2018-01-14T12:48:07.493" Score="1" Body="&lt;p&gt;You can almost do what you want using &lt;code&gt;featureCounts&lt;/code&gt; from the &lt;code&gt;subread&lt;/code&gt; package. The only problem is that featureCounts requires GTF rather than bed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you had a GTF of your introns, you could do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;featureCounts -a introns.gtf my_bam.bam -f -p --minReadOverlap=25 -o counts.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The problem with this then, is getting the &lt;code&gt;GTF&lt;/code&gt; when you currently have a &lt;code&gt;BED&lt;/code&gt;.&#xA;One way to get the &lt;code&gt;GTF&lt;/code&gt; would be to use &lt;code&gt;CGAT&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cgat gtf2gtf -I ensembl.gtf.gz --method=merge-exons -S genes.gtf.gz&#xA;cgat gtf2gtf -I genes.gtf.gz --method=exons2introns -S introns.gtf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or you could put this together into one pipeline:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cgat gtf2gtf -I ensembl.gtf.gz --method=merge-exons  | cgat gtf2gtf --method=exons2introns -S introns.gtf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will first merge all the exons of a gene together to make a single, collapsed transcript. It will that invert this to find that parts that are in the introns of this collapsed transcript. What you end up with is parts of the gene that are never exonic in any of the transcripts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;alternatively, you could convert your bed regions into GTF:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cgat bed2gff -I introns.bed -S introns.gtf --as-gtf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-01-14T12:48:07.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3303" PostTypeId="2" ParentId="3299" CreationDate="2018-01-14T20:33:57.887" Score="2" Body="&lt;p&gt;I think the mistake here is thinking that you need to use the sequence in order to find the genome co-ordinates. If these proteins come from a well annotated speices, you can easily do this with BioMart. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Go the the &lt;a href=&quot;http://www.ensembl.org/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENSEMBL&lt;/a&gt; web site.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;From the top menu, select BioMart&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Under &quot;-CHOSE DATABASE-&quot; select Ensembl Genes, and under &quot;-CHOOSE DATASET-&quot; select your species (eg. Human Genes). &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Select &quot;Filters&quot; from the lefthand menu and then expand the &quot;GENE&quot; section.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Select the &quot;Input external references ID list [Max 500 advised]&quot; and from the drop-down select &quot;PDB IDs&quot; or the correct &quot;Uniprot KB&quot; ID.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Paste your IDs into the box. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Select &quot;Attributes&quot; on the left-hand menu. Making sure you've the &quot;Fetaures&quot; radio box select, you can select Chromosome and either Gene Start/Gene End or Transcript start/Transcript end from the GENE section. These coordinates will also contain the untranslated regions unfortunately.  You may also want to play about with the various things under &quot;PROTEIN DOMAINS AND FAMILIES&quot;, particurlarly under &quot;Protein Features&quot;, although I'm not sure what they all mean. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When you are happy with the selection, select the big &quot;Results&quot; at the top to get the coords.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You can also get out Sequences for both the full transcript and the CDS and known germline and somatic variants that fall within the boundaries of the genes. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-01-14T20:33:57.887" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3304" PostTypeId="2" ParentId="3276" CreationDate="2018-01-14T20:45:36.607" Score="1" Body="&lt;p&gt;The program &lt;code&gt;extract&lt;/code&gt; from &lt;a href=&quot;https://github.com/CGATOxford/UMI-tools&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;UMI-Tools&lt;/code&gt;&lt;/a&gt; will extract the barcode from the read and add it to the ID. It will also handle pair-end data so that even if the barcode is only on one of the reads, it will be added to the ID of both reads.&lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-01-14T20:45:36.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3305" PostTypeId="1" AcceptedAnswerId="3306" CreationDate="2018-01-14T21:38:50.783" Score="6" ViewCount="11077" Body="&lt;p&gt;I have a several VCFs which are VCF which only contain information by chromosome. That is, there's a chromosome 1 VCF (with only chr1), a chromosome 2 VCF (with only chr2), etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I checked to make sure that these VCFs were valid via &lt;code&gt;VCFtools&lt;/code&gt;, i.e. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ vcf-validator chr1.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which works---these are valid VCFs I was given. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I would like to combine these VCFs into one VCF. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I naïvely tried the following &lt;code&gt;cat&lt;/code&gt; operation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat chr1.vcf chr2.vcf chr3.vcf ... chrX.vcf &amp;gt; total_chroms.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This doesn't work properly though&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ vcf-validator total_chroms.vcf&#xA;The header tag 'contig' not present for CHROM=chr1. (Not required but highly recommended.)&#xA;Could not parse the line, wrong number of columns: [##fileformat=VCFv4.2\n]&#xA; at /path/vcftools-0.1.14/perl/Vcf.pm line 172, &amp;lt;__ANONIO__&amp;gt; line 191016.&#xA;	    Vcf::throw('Vcf4_2=HASH(0x1ae7208)', 'Could not parse the line, wrong number of columns: [##filefor...') called at  /path/vcftools-0.1.14/perl/Vcf.pm line 335&#xA;	    Vcf::next_data_array('Vcf4_2=HASH(0x1ae7208)') called at  /path/vcftools-0.1.14/perl/Vcf.pm line 3457&#xA;	    Vcf4_1::next_data_array('Vcf4_2=HASH(0x1ae7208)') called at  /path/vcftools-0.1.14/perl/Vcf.pm line 2574&#xA;	    VcfReader::run_validation('Vcf4_2=HASH(0x1ae7208)') called at  /path/vcftools-0.1.14//bin/vcf-validator line 60&#xA;	    main::do_validation('HASH(0x16ada68)') called at  /path/vcftools-0.1.14//bin/vcf-validator line 14&#xA;$&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What tools are available to merge these VCF's together into a total VCF? &lt;/p&gt;&#xA;" OwnerUserId="146" LastActivityDate="2018-01-15T18:30:09.480" Title="How to concatenate &quot;by chromosome&quot;-VCFs?" Tags="|vcf|vcftools|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3306" PostTypeId="2" ParentId="3305" CreationDate="2018-01-14T23:25:10.867" Score="11" Body="&lt;p&gt;I would recommend &lt;a href=&quot;https://samtools.github.io/bcftools/bcftools.html&quot; rel=&quot;noreferrer&quot;&gt;bcftools&lt;/a&gt; concat.  You can't just &lt;code&gt;cat&lt;/code&gt; them together because each file has a header section.  The bcftools command will handle all that for you.  Each vcf file must be sorted prior to calling &lt;code&gt;concat&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;bcftools concat -o total_chroms.vcf chr1.vcf chr2.vcf chr3.vcf ... chrX.vcf&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2018-01-14T23:25:10.867" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="3307" PostTypeId="1" AcceptedAnswerId="3308" CreationDate="2018-01-15T10:13:51.617" Score="8" ViewCount="3435" Body="&lt;p&gt;I am calling variants from a human sample using bwa mem to align the reads and gatk to call the variants. I'm trying to understand why a specific variant was not called in my sample. I have checked the bam alignments in a GUI viewer and I can see that there are reads supporting the missing variant. It looks like the issue is a low allelic balance, with far more reads supporting the reference than the alternate allele but I want to get the actual numbers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, given a specific variant like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr22   425236  C T&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I count the number of reads in my &lt;code&gt;sample.bam&lt;/code&gt; file that support that variant and the number that don't on Linux?&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2020-04-24T08:02:37.627" LastActivityDate="2020-04-24T08:02:37.627" Title="How can I count the number of reads that support a variant in a bam file?" Tags="|sam|ngs|variant-calling|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3308" PostTypeId="2" ParentId="3307" CreationDate="2018-01-15T10:39:33.430" Score="8" Body="&lt;p&gt;If you don't mind a bit of manual counting, then &lt;code&gt;samtools mpileup -f reference.fa -r chr22:425236-425236 alignments.bam&lt;/code&gt; will produce output where you can count the bases for that position. You could, of course, use the command line to do most of that automatically:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools mpileup -f reference.fa -r chr22:425236-425236 alignments.bam |&#xA;    cut -f 5 | tr '[a-z]' '[A-Z]' | fold -w 1 | sort | uniq -c&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That'll give you a count of how many of each base were seen.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="298" LastEditDate="2018-01-15T10:44:20.830" LastActivityDate="2018-01-15T10:44:20.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3309" PostTypeId="2" ParentId="3305" CreationDate="2018-01-15T15:13:06.070" Score="2" Body="&lt;p&gt;The best tool for the job is probably &lt;code&gt;bcftools&lt;/code&gt; as &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/3306/298&quot;&gt;suggested&lt;/a&gt; by Bioathlete, but you can also do this manually. You just need to collect all header lines from all vcf files, remove any duplicates, and then print all the headers + the actual data into the new one:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep '^##' chr*vcf | sort | uniq &amp;gt; all.vcf&#xA;grep -m1 '^#CHR' chr1.vcf &amp;gt;&amp;gt; all.vcf  ## Get the chr header line &#xA;grep -v '^#' chr*vcf &amp;gt;&amp;gt; all.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-01-15T15:13:06.070" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3312" PostTypeId="2" ParentId="3305" CreationDate="2018-01-15T18:30:09.480" Score="2" Body="&lt;p&gt;use picard &lt;strong&gt;GatherVcf&lt;/strong&gt;: &lt;a href=&quot;http://broadinstitute.github.io/picard/command-line-overview.html#GatherVcfs&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://broadinstitute.github.io/picard/command-line-overview.html#GatherVcfs&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Gathers multiple VCF files from a scatter operation into a single VCF file. Input files must be supplied in genomic order and must not have events at overlapping positions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="71" LastActivityDate="2018-01-15T18:30:09.480" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3314" PostTypeId="2" ParentId="3307" CreationDate="2018-01-15T20:00:03.513" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/dariober/ASCIIGenome&quot; rel=&quot;nofollow noreferrer&quot;&gt;ASCIIGenome&lt;/a&gt; (I'm the author) has a command, &lt;a href=&quot;http://asciigenome.readthedocs.io/en/latest/commandHelp.html#filtervariantreads&quot; rel=&quot;nofollow noreferrer&quot;&gt;filterVariantReads&lt;/a&gt;, designed to inspect reads having a variant at a position or range. It would go along these lines:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ASCIIGenome -fa genome.fa aln.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then go to the region of interest and use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;goto chr9:4917981-4918161&#xA;filterVariantReads -r 4918011&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;From this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VigzQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VigzQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You get:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lxoW2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lxoW2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also print to screen or save to file the underlying sam records with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print &amp;gt; reads.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For several regions, the whole process can be scripted for automation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="339" LastActivityDate="2018-01-15T20:00:03.513" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3315" PostTypeId="1" CreationDate="2018-01-15T21:23:23.773" Score="0" ViewCount="459" Body="&lt;p&gt;I have a bed file where some entries have the exact same chr:start-end but change in the name and score column. I would like to apply bedtools or a similar tool to obtain a filtered file where for each set of chr:start-end entries, only the one with the highest score (column 5) is used, the others filtered out. I would like to do this with a bed format toolkit (bedtools, etc) rather than scripting it.&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2018-01-16T11:53:31.220" Title="bed entries collapse based on score column" Tags="|bed|bedtools|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3316" PostTypeId="2" ParentId="3315" CreationDate="2018-01-15T21:50:01.707" Score="2" Body="&lt;p&gt;Use &lt;a href=&quot;https://github.com/bedops/bedops&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS&lt;/a&gt; &lt;code&gt;bedmap --max-element&lt;/code&gt; or &lt;code&gt;bedmap --max-element-rand&lt;/code&gt; to grab the maximal-scoring element over mapped intervals, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sort-bed elements.unsorted.bed5 &amp;gt; elements.bed5&#xA;$ bedops --merge elements.bed5 &amp;gt; merged.bed3&#xA;$ bedmap --max-element merged.bed3 elements.bed5 &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The file &lt;code&gt;answer.bed&lt;/code&gt; will contain the maximal-scored element from &lt;code&gt;elements.bed5&lt;/code&gt; that overlaps the merged intervals in &lt;code&gt;merged.bed3&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;--max-element&lt;/code&gt; and &lt;code&gt;--max-element-rand&lt;/code&gt; options differ on handling ties. The first option grabs the first match from ties by &lt;code&gt;sort-bed&lt;/code&gt; (lexiographical) sort order; the second option grabs a random match on ties. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The option you pick would depend on what you're doing with the answer. Picking a random hit might be better if you're doing downstream statistical tests, for instance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can add &lt;code&gt;--sci&lt;/code&gt; and &lt;code&gt;--prec &amp;lt;int&amp;gt;&lt;/code&gt; options to format the score output to your desired level of precision. I think the default assumption is six digits of precision; adjust as needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also &lt;code&gt;--min-element&lt;/code&gt; and &lt;code&gt;--min-element-rand&lt;/code&gt; options for, conversely, reporting minimal-scoring elements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information: &lt;a href=&quot;http://bedops.readthedocs.io/en/latest/content/reference/statistics/bedmap.html#score-operations&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://bedops.readthedocs.io/en/latest/content/reference/statistics/bedmap.html#score-operations&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-01-15T22:45:32.870" LastActivityDate="2018-01-15T22:45:32.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3317" PostTypeId="2" ParentId="3315" CreationDate="2018-01-16T11:11:35.007" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://bedtools.readthedocs.io/en/latest/content/tools/groupby.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;groupby&lt;/a&gt; from bedtools might help you:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bedtools groupby -i variantsToRepeats.bed -grp 1-3 -c 5 -o max&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;-grp&lt;/code&gt; to select the columns to group by&lt;br /&gt;&#xA;&lt;code&gt;-c&lt;/code&gt; to select the columns to summarize&lt;br /&gt;&#xA;&lt;code&gt;-o&lt;/code&gt; to select how to summarize the column selected.&lt;/p&gt;&#xA;" OwnerUserId="391" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-01-16T11:53:31.220" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3318" PostTypeId="1" CreationDate="2018-01-16T14:48:54.837" Score="4" ViewCount="91" Body="&lt;p&gt;I want to use ipyrad on a new Ubuntu machine that has an NVidia Quadro K2000 card with 384 cores. One &lt;a href=&quot;http://ipyrad.readthedocs.io/HPC_script.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;can&lt;/a&gt; configure ipyrad to run on a linux cluster.   Do I have any options to get ipyrad to access these cores?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm assuming that I will be able to install cuda drivers on the machine. If ipyrad can't be made to do this, is there another way to do de novo assembly of GBS data that will take advantage of this resource?&lt;/p&gt;&#xA;" OwnerUserId="2196" LastEditorUserId="48" LastEditDate="2018-01-16T16:52:14.720" LastActivityDate="2018-03-17T18:47:39.250" Title="Make ipyrad use cuda-enabled NVidia card on Ubuntu" Tags="|python|assembly|genotyping|radseq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3319" PostTypeId="2" ParentId="3318" CreationDate="2018-01-16T15:46:19.960" Score="2" Body="&lt;p&gt;You would have to get &lt;a href=&quot;https://developer.nvidia.com/how-to-cuda-python&quot; rel=&quot;nofollow noreferrer&quot;&gt;CUDA Python&lt;/a&gt; set up on your system and likely modify ipyrad to actually use it (for example, by incorporating &lt;a href=&quot;http://numba.pydata.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;numba&lt;/a&gt;). So while this is possible, it will require significant effort on your part. In general, there are very few bioinformatics tools that support GPUs.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-16T15:46:19.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3320" PostTypeId="1" CreationDate="2018-01-16T17:20:37.473" Score="0" ViewCount="225" Body="&lt;p&gt;I'm using the QIIME1 scripts extract_barcodes.py to extract barcodes from a dual-indexed MiSeq library, and then demultiplex my reads using split_libraries_fastq.py.  I have been having trouble correctly using a couple of parameters relating to reverse complementing barcodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When extracting the barcodes with extract_barcodes.py, I use &lt;code&gt;--rev_comp_bc1&lt;/code&gt; (as directed by the individual in charge of sequencing). Then when I demultiplex the reads with split_libraries_fastq.py, sometimes I need to include the parameter &lt;code&gt;--rev_comp_mapping_barcodes&lt;/code&gt; to get the script to work, and sometimes I don't.  I'm not sure why, unless the barcodes I'm receiving in my mapping files are sometimes written in different orientations (I don't write them). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the parameter &lt;code&gt;--rev_comp_bc1&lt;/code&gt; is self-explanatory, but could someone explain what &lt;code&gt;--rev_comp_mapping_barcodes&lt;/code&gt; does and when to use it?&lt;br&gt;&#xA;To make things simple we can assume that the two barcodes are written from 5' to 3' in the mapping file. &lt;/p&gt;&#xA;" OwnerDisplayName="user2197" LastEditorUserId="48" LastEditDate="2018-01-16T21:03:41.507" LastActivityDate="2018-02-15T21:48:09.133" Title="What is the --rev_comp_mapping_barcode parameter in the QIIME1 script extract_barcodes.py?" Tags="|ngs|sequence-analysis|quality-control|barcode|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3321" PostTypeId="1" AcceptedAnswerId="3322" CreationDate="2018-01-16T20:43:00.503" Score="1" ViewCount="3993" Body="&lt;p&gt;I've got some data on genomic regions of interest in a bedgraph-like format from the &lt;em&gt;Mus musculus&lt;/em&gt; [mm10] genome, and would like to find out the nearest gene to these regions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the regions will reside within a gene (in which case I'd like to know that they're &lt;em&gt;within&lt;/em&gt; a gene), some regions may reside within multiple genes (in which case I'd like to know all gene names), and some regions are not within a gene (in which case I'd like to know the closest gene: the shortest sequence gap between either end of the gene and either end of the tagged genomic region. In this case I want to do an unstranded search, because it's possible that I might have regions describing suppressing transcripts. Here's some example data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Ref         Start             End      fwd     rev&#xA;chrX     73716152        73716152        0      -2&#xA;chrX    167207094       167209162        0      -3&#xA;chrY     30770844        30772724        3       0&#xA;chr1     24613189        24613641        0       3&#xA;chr1     24613971        24614812        0       4&#xA;chr1     35788729        35789387       -5       0&#xA;chr1     93151586        93160225        0      -2&#xA;chr1    107511454       107525597       -2       0&#xA;chr1    130882067       130887413        0      -3&#xA;chr1    134182466       134189942        2       0&#xA;chr1    135818954       135832956       -2       0&#xA;chr1    149520498       149521157        0      -3&#xA;chr1    166166500       166166500       -2       0&#xA;chr2      5379059         5379374        0      -2&#xA;chr2     32379110        32381837        0      -2&#xA;chr2     32381854        32381858        0      -2&#xA;chr2     32384633        32384633        0      -2&#xA;chr2     32384636        32387773        0      -3&#xA;chr2    120539390       120563799        0      -2&#xA;chr2    125946785       125975527        2       0&#xA;chr2    127633311       127633684        0      -2&#xA;chr2    127634841       127656210        0      -3&#xA;chr2    164562717       164562717       -2       0&#xA;chr2    164562719       164562720       -2       0&#xA;chr2    164568498       164568499       -2       0&#xA;chr2    181188007       181188388       -2       0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'd like to add another 'Gene' column to this table that gives the official names of the nearest gene(s) for each region defined by &lt;code&gt;&amp;lt;Ref&amp;gt;:&amp;lt;Start&amp;gt;-&amp;lt;End&amp;gt;&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-01-16T21:17:20.627" LastActivityDate="2018-01-16T23:34:53.790" Title="Finding the nearest gene to a specified gene region" Tags="|genome|sequence-annotation|bed|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3322" PostTypeId="2" ParentId="3321" CreationDate="2018-01-16T21:06:46.640" Score="4" Body="&lt;p&gt;You're looking for &lt;code&gt;bedtools closest&lt;/code&gt;, which thankfully defaults both considering overlapping genes and outputting all hits in case of ties.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  bedtools closest -a foo.txt -b genes.bed | awk '{foo = sprintf(&quot;%s:%i-%i&quot;, $6, $7, $8); OFS=&quot;\t&quot;; print $1, $2, $3, $4, $5, foo}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You'll need to strip the header and ensure the file is sorted. I've piped the output to &lt;code&gt;awk&lt;/code&gt; to reformat things. The output then is something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    24613189    24613641    0   3   chr1:24613188-24613971&#xA;chr1    24613971    24614812    0   4   chr1:24613973-24614651&#xA;chr1    35788729    35789387    -5  0   chr1:35806973-35810462&#xA;chr1    93151586    93160225    0   -2  chr1:93151348-93160948&#xA;chr1    93151586    93160225    0   -2  chr1:93151582-93160870&#xA;chr1    107511454   107525597   -2  0   chr1:107511422-107525598&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Those results are for the Gencode M15 release (the BED file I used has transcripts rather than genes, so you'll get a bit different coordinates). As an aside, the gene name would normally be output as well, just add column 9 to the output.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-16T21:06:46.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3323" PostTypeId="2" ParentId="3320" CreationDate="2018-01-16T21:30:06.667" Score="1" Body="&lt;p&gt;As the &lt;a href=&quot;http://qiime.org/scripts/split_libraries_fastq.html?highlight=split_libraries_fastq&quot; rel=&quot;nofollow noreferrer&quot;&gt;help page&lt;/a&gt; says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;--rev_comp_mapping_barcodes&lt;br&gt;&#xA;      Reverse complement barcode in mapping before lookup (useful if barcodes in mapping file are reverse complements of golay codes) [default: False]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As the help page of &lt;a href=&quot;http://qiime.org/scripts/extract_barcodes.html?highlight=rev_comp_bc1&quot; rel=&quot;nofollow noreferrer&quot;&gt;extract_barcode&lt;/a&gt; says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;--rev_comp_bc1&lt;br&gt;&#xA;  Reverse complement barcode 1 before writing [default: False]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You seem to be doing two complete different things. extract_barcode.py is designed to&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;format fastq sequence and barcode data so they are compatible with split_libraries_fastq.py&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;While the first one is for:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;demultiplexing of Fastq sequence data where barcodes and sequences are contained in two separate fastq files&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So it seems that depending on the machine used or the previous processing you are given with the barcode in another file or together. I would advice to ask the person who gave you this data. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-01-16T21:30:06.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3324" PostTypeId="2" ParentId="3321" CreationDate="2018-01-16T22:10:20.917" Score="1" Body="&lt;p&gt;With the prompting from @Devon_Ryan's answer, I came up with a solution that worked well for me. I downloaded the mm10 RefSeq gene locations from &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTables&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC&lt;/a&gt; [Fields: chrom, txStart, txEnd, name2], and sorted both input files using &lt;code&gt;bedtools sort&lt;/code&gt;. From there the &lt;code&gt;bedtools closest&lt;/code&gt; command got me almost to where I wanted to go:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(echo -e &quot;Chr\tStart\tEnd\tfwd\trev\tm_Chr\tm_Start\tm_End\tGene\tDistance&quot;; &#xA;  bedtools closest -d -a regions.bed -b NCBI_RefSeq_mm10.bed) &amp;gt; annotated.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Chr     Start   End     fwd     rev     m_Chr   m_Start m_End   Gene    Distance&#xA;chrX    73716152        73716152        0       -2      chrX    73686177        73716204        Bcap31  0&#xA;chrX    73716152        73716152        0       -2      chrX    73686177        73717815        Bcap31  0&#xA;chrX    73716152        73716152        0       -2      chrX    73686177        73716401        Bcap31  0&#xA;chrX    73716152        73716152        0       -2      chrX    73686177        73716502        Bcap31  0&#xA;chrX    167207094       167209162       0       -3      chrX    167207092       167213479       Tmsb4x  0&#xA;chrX    167207094       167209162       0       -3      chrX    167207093       167209218       Tmsb4x  0&#xA;chrY    30770844        30772724        3       0       chrY    30725674        30738546        Gm29582 32299&#xA;chr1    24613189        24613641        0       3       chr1    24257682        24587535        Col19a1 25655&#xA;chr1    24613189        24613641        0       3       chr1    24257682        24587535        Col19a1 25655&#xA;chr1    24613189        24613641        0       3       chr1    24257682        24587535        Col19a1 25655&#xA;chr1    24613189        24613641        0       3       chr1    24257682        24587535        Col19a1 25655&#xA;chr1    24613971        24614812        0       4       chr1    24257682        24587535        Col19a1 26437&#xA;chr1    24613971        24614812        0       4       chr1    24257682        24587535        Col19a1 26437&#xA;chr1    24613971        24614812        0       4       chr1    24257682        24587535        Col19a1 26437&#xA;chr1    24613971        24614812        0       4       chr1    24257682        24587535        Col19a1 26437&#xA;chr1    35788729        35789387        -5      0       chr1    35869594        35881164        1110002O04Rik   80208&#xA;chr1    93151586        93160225        0       -2      chr1    93151353        93160935        2310007B03Rik   0&#xA;chr1    93151586        93160225        0       -2      chr1    93151353        93160957        2310007B03Rik   0&#xA;chr1    93151586        93160225        0       -2      chr1    93151354        93160948        2310007B03Rik   0&#xA;chr1    93151586        93160225        0       -2      chr1    93151354        93160870        2310007B03Rik   0&#xA;chr1    107511454       107525597       -2      0       chr1    107500891       107525600       Serpinb2        0&#xA;chr1    107511454       107525597       -2      0       chr1    107511422       107525600       Serpinb2        0&#xA;chr1    107511454       107525597       -2      0       chr1    107511509       107525600       Serpinb2        0&#xA;chr1    130882067       130887413       0       -3      chr1    130882073       130887411       Il24    0&#xA;chr1    134182466       134189942       2       0       chr1    134182169       134190031       Chil1   0&#xA;chr1    134182466       134189942       2       0       chr1    134182403       134190031       Chil1   0&#xA;chr1    134182466       134189942       2       0       chr1    134182408       134190031       Chil1   0&#xA;chr1    135818954       135832956       -2      0       chr1    135818597       135833341       Lad1    0&#xA;chr1    149520498       149521157       0       -3      chr1    149410728       149449931       Gm29398 70568&#xA;chr1    166166500       166166500       -2      0       chr1    166130459       166166510       Gpa33   0&#xA;chr2    5379059         5379374         0       -2      chr2    5293456         5676046         Camk1d  0&#xA;chr2    5379059         5379374         0       -2      chr2    5293456         5714762         Camk1d  0&#xA;chr2    5379059         5379374         0       -2      chr2    5293456         5714762         Camk1d  0&#xA;chr2    5379059         5379374         0       -2      chr2    5293456         5714762         Camk1d  0&#xA;chr2    5379059         5379374         0       -2      chr2    5293458         5383533         Camk1d  0&#xA;chr2    5379059         5379374         0       -2      chr2    5293458         5569754         Camk1d  0&#xA;chr2    5379059         5379374         0       -2      chr2    5293458         5601302         Camk1d  0&#xA;chr2    5379059         5379374         0       -2      chr2    5293458         5715350         Camk1d  0&#xA;chr2    5379059         5379374         0       -2      chr2    5298992         5715357         Camk1d  0&#xA;chr2    32379110        32381837        0       -2      chr2    32379100        32381915        1110008P14Rik   0&#xA;chr2    32381854        32381858        0       -2      chr2    32379100        32381915        1110008P14Rik   0&#xA;chr2    32384633        32384633        0       -2      chr2    32384636        32387739        Lcn2    3&#xA;chr2    32384636        32387773        0       -3      chr2    32384636        32387739        Lcn2    0&#xA;chr2    120539390       120563799       0       -2      chr2    120506829       120563831       Zfp106  0&#xA;chr2    120539390       120563799       0       -2      chr2    120507433       120563851       Zfp106  0&#xA;chr2    120539390       120563799       0       -2      chr2    120507433       120563853       Zfp106  0&#xA;chr2    125946785       125975527       2       0       chr2    125859108       125984298       Galk2   0&#xA;chr2    125946785       125975527       2       0       chr2    125859228       125983587       Galk2   0&#xA;chr2    125946785       125975527       2       0       chr2    125866111       125983587       Galk2   0&#xA;chr2    125946785       125975527       2       0       chr2    125866218       125983587       Galk2   0&#xA;chr2    125946785       125975527       2       0       chr2    125866219       125983587       Galk2   0&#xA;chr2    125946785       125975527       2       0       chr2    125866222       125984298       Galk2   0&#xA;chr2    125946785       125975527       2       0       chr2    125920565       125983587       Galk2   0&#xA;chr2    127633311       127633684       0       -2      chr2    127633225       127656695       Mal     0&#xA;chr2    127633311       127633684       0       -2      chr2    127633225       127656695       Mal     0&#xA;chr2    127634841       127656210       0       -3      chr2    127633225       127656695       Mal     0&#xA;chr2    127634841       127656210       0       -3      chr2    127633225       127656695       Mal     0&#xA;chr2    164562717       164562717       -2      0       chr2    164562560       164568510       Wfdc2   0&#xA;chr2    164562717       164562717       -2      0       chr2    164562715       164568506       Wfdc2   0&#xA;chr2    164562719       164562720       -2      0       chr2    164562560       164568510       Wfdc2   0&#xA;chr2    164562719       164562720       -2      0       chr2    164562715       164568506       Wfdc2   0&#xA;chr2    164568498       164568499       -2      0       chr2    164562560       164568510       Wfdc2   0&#xA;chr2    164568498       164568499       -2      0       chr2    164562715       164568506       Wfdc2   0&#xA;chr2    181188007       181188388       -2      0       chr2    181187221       181188506       Ppdpf   0&#xA;chr2    181188007       181188388       -2      0       chr2    181187342       181188504       Ppdpf   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After that it was just a matter of aggregating the gene names for each unique region with a quick R script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(dplyr);&#xA;data.tbl &amp;lt;-&#xA;    read.delim(&quot;annotated.tsv&quot;) %&amp;gt;%&#xA;    group_by(Chr, Start, End, fwd, rev) %&amp;gt;%&#xA;    summarise(Genes = paste(unique(Gene), collapse=&quot;,&quot;), Distance=min(Distance));&#xA;&#xA;write.table(data.tbl, &quot;collapsed_annotated.tsv&quot;,&#xA;            sep=&quot;\t&quot;, row.names=FALSE, quote=FALSE);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Collapsed output example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Chr     Start   End     fwd     rev     Genes   Distance&#xA;chr1    24613189        24613641        0       3       Col19a1 25655&#xA;chr1    24613971        24614812        0       4       Col19a1 26437&#xA;chr1    35788729        35789387        -5      0       1110002O04Rik   80208&#xA;chr1    93151586        93160225        0       -2      2310007B03Rik   0&#xA;chr1    107511454       107525597       -2      0       Serpinb2        0&#xA;chr1    130882067       130887413       0       -3      Il24    0&#xA;chr1    134182466       134189942       2       0       Chil1   0&#xA;chr1    135818954       135832956       -2      0       Lad1    0&#xA;chr1    149520498       149521157       0       -3      Gm29398 70568&#xA;chr1    166166500       166166500       -2      0       Gpa33   0&#xA;chr2    5379059         5379374         0       -2      Camk1d  0&#xA;chr2    32379110        32381837        0       -2      1110008P14Rik   0&#xA;chr2    32381854        32381858        0       -2      1110008P14Rik   0&#xA;chr2    32384633        32384633        0       -2      Lcn2    3&#xA;chr2    32384636        32387773        0       -3      Lcn2    0&#xA;chr2    120539390       120563799       0       -2      Zfp106  0&#xA;chr2    125946785       125975527       2       0       Galk2   0&#xA;chr2    127633311       127633684       0       -2      Mal     0&#xA;chr2    127634841       127656210       0       -3      Mal     0&#xA;chr2    164562717       164562717       -2      0       Wfdc2   0&#xA;chr2    164562719       164562720       -2      0       Wfdc2   0&#xA;chr2    164568498       164568499       -2      0       Wfdc2   0&#xA;chr2    181188007       181188388       -2      0       Ppdpf   0&#xA;chrX    73716152        73716152        0       -2      Bcap31  0&#xA;chrX    167207094       167209162       0       -3      Tmsb4x  0&#xA;chrY    30770844        30772724        3       0       Gm29582 32299&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-01-16T22:10:20.917" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3325" PostTypeId="2" ParentId="3321" CreationDate="2018-01-16T23:34:53.790" Score="1" Body="&lt;p&gt;Another option is &lt;a href=&quot;http://bedops.readthedocs.io/en/latest/content/reference/set-operations/closest-features.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS &lt;code&gt;closest-features&lt;/code&gt;&lt;/a&gt;, which efficiently locates nearest annotations, overlapping or otherwise.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ closest-features --closest --delim '\t' intervals.bed genes.bed &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It's easy to reformat output by piping to &lt;code&gt;awk&lt;/code&gt;, say.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If files are very large, you can parallelize work into &quot;map-reduce&quot; form by specifying the chromosome name as an option:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ closest-features --chrom chr1 --closest --delim '\t' intervals.bed genes.bed &amp;gt; answer.chr1.bed&#xA;$ closest-features --chrom chr2 --closest --delim '\t' intervals.bed genes.bed &amp;gt; answer.chr2.bed&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To reduce, just &lt;code&gt;bedops -u&lt;/code&gt; all the &lt;code&gt;answer.*.bed&lt;/code&gt; files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Running these per-chromosome jobs in parallel makes the work take about as long as searching &lt;code&gt;chr1&lt;/code&gt;, which tends to be the largest chromosome.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2018-01-16T23:34:53.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3326" PostTypeId="2" ParentId="2418" CreationDate="2018-01-17T05:00:30.263" Score="5" Body="&lt;p&gt;I've had great results using &lt;a href=&quot;https://github.com/lh3/minimap2&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimap2&lt;/a&gt;, particularly when combined with a pre-treatment of Canu for error correction (using &lt;em&gt;minimap2&lt;/em&gt; for the read-to-read mapping):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# correct reads&#xA;~/install/canu/canu-1.6/Linux-amd64/bin/canu overlapper=minimap \&#xA;   genomeSize=100M minReadLength=100 minOverlapLength=30 -correct \&#xA;   -p 4T1_BC06 -d 4T1_BC06 \&#xA;   -nanopore-raw workspace/pass/barcode06/fastq_runid_*.fastq&#xA;# align reads&#xA;~/install/minimap2/minimap2 -p 10 \&#xA;   -a ~/db/fasta/mmus/ucsc/mmus_ucsc_all_cdna.idx \&#xA;   -x splice &amp;lt;(pv all/4T1_BC06/4T1_BC06.correctedReads.fasta.gz) | \&#xA;  samtools sort &amp;gt; 4T1_BC06_all_vs_mmusAll.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Update: the most recent nanopore basecaller, combined with the most recent minimap2, seems to now do a decent job with mapping, and pre-correction of reads no longer seems necessary. More recently I've been using LAST to map to the transcriptome, and minimap2 to map to the genome. Minimap2 has the ability to use a homopolymer-compressed genome index, which means that the most common consistent error for nanopore reads (i.e. misjudgement of homopolymer length) will not influence the mapping rate.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2019-02-22T22:54:39.600" LastActivityDate="2019-02-22T22:54:39.600" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3327" PostTypeId="1" CreationDate="2018-01-17T10:16:34.240" Score="3" ViewCount="75" Body="&lt;p&gt;I have a list of regions of the human genome and I want to predict if single-stranded molecules in a buffer would tend to fold and create pin structures by sequence self-complementarity. What's the most precise software/parameter set I can use to such effect?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TRIED SO FAR:&#xA;&lt;a href=&quot;https://www.tbi.univie.ac.at/RNA/RNALfold.1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vienna RNALFold&lt;/a&gt; (unsure if applying an RNA software to DNA is adequate):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Compute locally stable RNA secondary structure with a maximal base pair span. &#xA;For a sequence of length n and a base pair span of L the algorithm uses only &#xA;O(n+L*L) memory and O(n*L*L) CPU time. Thus it is practical to &quot;scan&quot; very &#xA;large genomes for short RNA structures.&#xA;Output consists of a list of secondary structure components of size &amp;lt;= L, one &#xA;entry per line. Each output line contains the predicted local structure its &#xA;energy in kcal/mol and the starting position of the local structure.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools faidx $ref $region | seqtk seq -A - | $HOME/viennarna/bin/RNALfold --noconv&#xA;&amp;gt;chr1:100000-100150&#xA;.((((.(......).)))). ( -2.70)  127&#xA;.(((.((........)).))). ( -5.70)  123&#xA;.(((((((.((........)).)))..)))). ( -6.30)  119&#xA;.(((.(.(((.((.....))))).)..))). ( -7.00)  117&#xA;.((((.((........)).)))). ( -8.30)  109&#xA;.((((..((.((..(..(((.....)))..)..)))))))). (-10.10)  103&#xA;.((((.((..((..(((...((((.((........)).)))).....)))))..)))))). (-10.50)   90&#xA;.((.((.((((....................)))).)).)). ( -4.10)   77&#xA;.((((.((.((((....................)))).)).)))). ( -5.70)   75&#xA;.((((..((((.((....)).)))).)))). ( -2.40)   72&#xA;.((.((..((((..((((.((....)).)))).))))..)))). ( -5.50)   65&#xA;.(.((..(..................((.((..((((..((((.((....)).)))).))))..))))..((((.((........)).)))).)..)).). (-16.20)   40&#xA;.((((......(.((..(..................((.((..((((..((((.((....)).)))).))))..))))..((((.((........)).)))).)..)).).....)))). (-18.10)   30&#xA;.(((....(((((((((...(.........)..)))).)))))((.((..((((..((((.((....)).)))).))))..))))..((((.((........)).))))......))). (-18.50)   23&#xA;.((....(((....(((((((((...(.........)..)))).)))))((.((..((((..((((.((....)).)))).))))..))))..((((.((........)).))))......)))....)). (-18.70)   17&#xA;.(((((((((......((...(((((..((((...((...))....))))..)))))...))......))))))))). ( -9.60)    7&#xA;.((.(((((((((......((...(((((..((((...((...))....))))..)))))...))......))))))))).)). (-10.90)    4&#xA;.(((..(((((........)))))..))). ( -3.60)    1&#xA;ACTTAAGTTGTAGAAGGGAATAACGCAAGAGTGAATTTAGGGCGGGGCAAAAGGATAAATTTTACGGTACAAAGTTTCTACGGGTTTATATATGTATAACTAAGTCCAAGCGCGGGGGATATGGCCAGTGCACAACGGCGGGCATCATAAT&#xA; (-21.70)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="180" LastEditorUserId="180" LastEditDate="2018-01-17T10:48:05.137" LastActivityDate="2018-01-18T12:32:09.620" Title="predict the foldability of single-stranded DNA molecules" Tags="|phylogenetics|genomics|structural-biology|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3328" PostTypeId="1" CreationDate="2018-01-17T12:51:57.140" Score="1" ViewCount="81" Body="&lt;p&gt;I'm struggling to figure out how to go from gene expression data to  be able to create a network out of the most expressed genes in a cell line. Ok, I have developed an application for Cytoscape where I want to compare my analysis with research on the NCI-60 cell lines. My goal is to do these steps:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1: Use one of the cell lines in the NCI-60 cell lines, in my case I will use the NCI_H23 where there is 3 samples in this &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE32474&quot; rel=&quot;nofollow noreferrer&quot;&gt;dataset&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2: Use the most expressed genes in the NCI_H23 cell line to create a subnetwork using a protein interaction network database like iRefIndex, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After these two steps I should finally be able to upload the network to Cytoscape and perform my analysis, but being a programmer  I'm kind of stuck. I have tried using &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM803712&quot; rel=&quot;nofollow noreferrer&quot;&gt;one&lt;/a&gt; of the 3 samples where I mapped the Affymatrix IDs to gene symbol and then selecting the top 20% genes with the highest values using a cutoff value, but I'm not sure if this is the correct way to do it and I'm not sure how to do step nr.2 above if my list of genes is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My application investigates drug combinations based on graph theory so I want to create a network that represents a cell-line so I can compare my scoring of combinations with the ones from the NCI Almanac study. I don't want to compare it with other cell-lines, but rather find the most expressed genes in that cell-line to then retrieve interaction data from iRefIndex to get edges for the network. I guess I should average the 3 samples first before I repeat what I listed above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would really appreciate help on this as I've been stuck on this for over a week now. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm stuck at now is trying to create a network from iRefIndex (using iRefR) using the set of gene symbols that I have extracted from the 3 samples. To be honest, I'm finding it hard to find the right tools and assessing what kind of interaction data i should use from iRefIndex etc, so I guess I am looking for confirmation that what I'm trying makes sense.&lt;/p&gt;&#xA;" OwnerUserId="1633" LastEditorUserId="73" LastEditDate="2018-01-17T21:22:01.083" LastActivityDate="2018-01-17T21:22:01.083" Title="Help with generating network from gene expression data" Tags="|gene|networks|cell-line|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3329" PostTypeId="2" ParentId="3286" CreationDate="2018-01-17T15:46:56.790" Score="2" Body="&lt;p&gt;These values come from a microarray so they represent the number of sequences that where attached to the microarray's probes (1007_s_at, ...). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general yes, a higher value means more expression. The normalizations (GC-RMA) applied are to correct for several effects. They help for testing the probability of the given probe to be a false positive. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-01-17T15:46:56.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3330" PostTypeId="1" AcceptedAnswerId="3332" CreationDate="2018-01-17T16:58:33.323" Score="1" ViewCount="745" Body="&lt;p&gt;I mapped raw illumina reads to longer pacbio reads and I would like to know the following information from my mapping file (SAM/BAM)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;How many PacBio reads are mapped to at least one illumina reads &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A Table showing each pacbio read, the number of illumina reads that&#xA;mapped, and total coverage (i.e. percentage of mapped regions)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2199" LastEditorUserId="73" LastEditDate="2018-01-17T21:24:25.553" LastActivityDate="2018-01-18T08:01:05.607" Title="Mapping Information from SAM/BAM file" Tags="|read-mapping|samtools|sam|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3331" PostTypeId="2" ParentId="3330" CreationDate="2018-01-17T18:00:32.357" Score="0" Body="&lt;p&gt;This infomation can be obtained with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools index mybam.bam&#xA;samtools idxstats mybam.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You'll get a table with one row per PacBio read, the length, the number of mapped reads aligned to it and the number of unmapped reads aligned to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The number of PacBio reads mapped to at least one illumina read is the number of rows where the 3rd column is greater than 0&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools idxstats mybam.bam | awk '$3&amp;gt;0' | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-01-17T18:00:32.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3332" PostTypeId="2" ParentId="3330" CreationDate="2018-01-17T18:48:45.747" Score="1" Body="&lt;p&gt;If you sort and index your BAM file:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;samtools idxstats&lt;/code&gt;. The third column is the number of Illumina reads mapped to each PacBio read. You can then pipe that to awk: &lt;code&gt;samtools idxstats foo.bam | awk '{if($1 != &quot;*&quot; &amp;amp;&amp;amp; $3 &amp;gt; 0) tot += 1}END{print tot}&lt;/code&gt;. That will print the number of PacBio reads with at least 1 read mapped to them.&lt;/li&gt;&#xA;&lt;li&gt;You'll want &lt;code&gt;samtools depth&lt;/code&gt; for that&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;samtools depth -aa foo.bam | awk '{if($1 != lread) {if(lread != &quot;&quot;) {print lread, 100*lcov/llen}; lread = $1; lcov=0; llen=0}; llen += 1; if($3&amp;gt;0) {lcov +=1}}END{print lread, 100*lcov/llen}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That won't give you the number of illumina reads per pacbio read, but you can get that from &lt;code&gt;samtools idxstats&lt;/code&gt; (column 3) and &lt;code&gt;join&lt;/code&gt; the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, your usage of &quot;coverage&quot; is non-standard. You want &quot;percentage coverage&quot; rather than &quot;coverage&quot;, which actually means &quot;fold-coverage&quot; (i.e., number of reads times their average length divided by the total genome/chromosome/etc. length).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you instead want &quot;fold-coverage&quot; then the easiest way is to first know the length of your reads. Presuming you have 100 base reads (for paired-end reads, this would be the same, presuming the ends don't overlap):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools idxstats foo.bam | awk '{print $1, $3, 100*$3/$2}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That will give you a table of PacBio read name, number of aligned reads, fold-coverage.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-01-18T08:01:05.607" LastActivityDate="2018-01-18T08:01:05.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3333" PostTypeId="1" AcceptedAnswerId="3334" CreationDate="2018-01-17T23:04:07.573" Score="3" ViewCount="406" Body="&lt;p&gt;&lt;em&gt;This is summary of discussions and a question that was posted by &lt;a href=&quot;https://twitter.com/devonorourke&quot; rel=&quot;nofollow noreferrer&quot;&gt;Devon O'Rourke&lt;/a&gt; on &lt;a href=&quot;https://twitter.com/devonorourke/status/951825589048168448&quot; rel=&quot;nofollow noreferrer&quot;&gt;Twitter&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following Albacore basecalls on a 1D² library I get two sets of .fq files, summary stats, etc.: one for the 1D basecalling script, and one for the 1D² script. I'd like to use the reads generated from these scripts in a genome assembly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After a little bit of &lt;code&gt;grep&lt;/code&gt; searching, it seems like there are overlapping reads among the 1D and 1D² directories - namely, those 1D reads that generated 1D² reads!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do people approach the choice of selecting which directory to pull the unique read from if you're planning on assembling a genome? I'd think that the 1D² reads are of higher quality and be preferred, but perhaps there are 1D² reads which are lower in quality than one of the two 1D pairs? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just looking for advice on how folks approach 1D² libraries, fastq outputs, and subsequent mapping. For what it's worth I'm hoping to use Minimap2 to proceed with alignments but am happy to hear others thoughts on alternatives.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Update&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I tried the first solution proposed by @gringer, but my filtered_1d_reads.fq file contains same number of reads as raw_1d_reads.fq file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm pretty sure the problem is the naming convention difference between the linear and paired reads. linear reads always follow a read_id of alphanumeric (8)-(4)-(4)-(4)-(12), while paired reads are (8)-(4)-(4)-(4)-(20)-(4)-(4)-(4)-(12). When I looked at it in more detail, the paired read &quot;read_id&quot; naming is just a concatenation of the two linear &quot;read_id&quot;'s that are partnered. So that (20) value is really the last (12) of the first linear read, and the first (8) of the second linear pair.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-01-17T23:12:56.537" LastActivityDate="2018-02-17T09:02:23.717" Title="Removing repeated reads from nanopore 1D² reads" Tags="|nanopore|1d2-reads|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3334" PostTypeId="2" ParentId="3333" CreationDate="2018-01-17T23:06:36.483" Score="1" Body="&lt;p&gt;You can exclude the 1D² reads by generate a list of the ids from the 1D² reads, then filtering those reads out from the 1D fastq files. Here's one way to do that using some &lt;a href=&quot;https://github.com/gringer/bioinfscripts&quot; rel=&quot;nofollow noreferrer&quot;&gt;accessory scripts&lt;/a&gt; that I have written:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat 1d2_reads.fq | ~/scripts/fastx-length.pl | awk '{print $2}' &amp;gt; readNames.txt&#xA;$ cat 1d_reads.fq | ~/scripts/fastx-fetch.pl -v -i readNames.txt &amp;gt; filtered_1d_reads.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In order to deal with the concatenated read IDs, it's necessary to switch to a language like Perl that supports the use of backreferences for the splitting. The following perl command (replacing the first line in the above procedure) inserts a line break after the 36th character in the 2nd value, then prints that value:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat 1d2_reads.fq | ~/scripts/fastx-length.pl | perl -lane '$F[1] =~ s/(.{36})/$1\n/; print $F[1];' &amp;gt; readNames.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So a full sequence of operations would look something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat 1d2_reads.fq | ~/scripts/fastx-length.pl | perl -lane '$F[1] =~ s/(.{36})/$1\n/; print $F[1];' &amp;gt; readNames.txt&#xA;$ cat 1d_reads.fq | ~/scripts/fastx-fetch.pl -v -i readNames.txt &amp;gt; filtered_1d_reads.fq&#xA;$ cat 1d2_reads.fq filtered_1d_reads.fq &amp;gt; unique_reads.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-01-17T23:23:54.350" LastActivityDate="2018-01-17T23:23:54.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3335" PostTypeId="1" CreationDate="2018-01-18T00:01:01.387" Score="0" ViewCount="235" Body="&lt;p&gt;I want to use a script to evaluate a protein structure with QMEAN scoring.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't understand why the script suggested by SWISS-MODEL (&lt;a href=&quot;https://swissmodel.expasy.org/qmean/help#help_programmatic_access&quot; rel=&quot;nofollow noreferrer&quot;&gt;access to QMEAN by json object&lt;/a&gt;) runs only if I put the commands on the shell. If I build a module, the result of the second request (request.get), return a QUEUEING status, not COMPLETE status.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code that I have been using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import json&#xA;import requests&#xA;&#xA;def qmean_page(structure):&#xA;&quot;&quot;&quot; return the Qmean4 score from SWISSMODEL QMEAN tool&quot;&quot;&quot;&#xA;&#xA;    url = &quot;https://swissmodel.expasy.org/qmean/submit/&quot;&#xA;&#xA;    # Request the resource&#xA;    response = requests.post(url=url, data={&quot;email&quot;: &quot;XXX@XXX.XX&quot;} ,files={&quot;structure&quot;: open(structure,'r'),},timeout=3,verify=True)&#xA;&#xA;    json.dumps(response.json(), indent=4, sort_keys=True)&#xA;    download_json = str(response.json()['download_url'])&#xA;&#xA;    return download_json&#xA;&#xA;def qmean_score(download_url):&#xA;# Request the resource&#xA;    r = requests.get(download_url, verify=True)&#xA;&#xA;    scores = json.dumps(r.json(), indent=4, sort_keys=True)&#xA;    print scores&#xA;    jdata = json.loads(scores)&#xA;    zscore = jdata['input_data']['models'][0]['global_scores']['qmean4']['zscore']&#xA;    return zscore&#xA;&#xA;&#xA;s = 'P02489.B99990005.pdb'&#xA;url_d = qmean_page(s)&#xA;print url_d&#xA;score  = qmean_score(url_d)&#xA;print score&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;CASE A:&lt;/strong&gt; If I comment the last 5 line, I can import qmean.py (the function file) in python terminal. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; import qmean&#xA;&amp;gt;&amp;gt;&amp;gt; s = 'P02489.B99990005.pdb'&#xA;&amp;gt;&amp;gt;&amp;gt; url_d = qmean.qmean_page(s)&#xA;&amp;gt;&amp;gt;&amp;gt; print url_d&#xA;https://swissmodel.expasy.org/qmean/project/TgEjha.json&#xA;&amp;gt;&amp;gt;&amp;gt; score  = qmean.qmean_score(url_d)&#xA;{&#xA;&quot;input_data&quot;: {&#xA;    &quot;info&quot;: {&#xA;        &quot;model_001&quot;: {&#xA;            &quot;chain_mapping&quot;: {&#xA;                &quot; &quot;: &quot;A&quot;&#xA;            }&#xA;        }&#xA;    }, &#xA;    &quot;meta&quot;: {&#xA;        &quot;created&quot;: &quot;2018-01-18T10:04:29.978&quot;, &#xA;        &quot;email&quot;: &quot;XXX@XXX.XX&quot;, &#xA;        &quot;project_name&quot;: &quot;Project&quot;, &#xA;        &quot;seqres_uploaded&quot;: false&#xA;    }, &#xA;    &quot;method&quot;: &quot;QMEANDisCo&quot;, &#xA;    &quot;models&quot;: [&#xA;        {&#xA;            &quot;error_status&quot;: null, &#xA;            &quot;global_scores&quot;: {&#xA;                &quot;acc_agreement&quot;: {&#xA;                    &quot;norm&quot;: 0.5606936416184971, &#xA;                    &quot;zscore&quot;: -1.0459143647477258&#xA;                }, &#xA;                &quot;all_atom&quot;: {&#xA;                    &quot;norm&quot;: -0.012626634534455788, &#xA;                    &quot;zscore&quot;: -2.1825427852787604&#xA;                }, &#xA;                &quot;cbeta&quot;: {&#xA;                    &quot;norm&quot;: -0.004260555683274807, &#xA;                    &quot;zscore&quot;: -1.9157144117276892&#xA;                }, &#xA;                &quot;qmean4&quot;: {&#xA;                    &quot;norm&quot;: 0.6275190794418422, &#xA;                    &quot;zscore&quot;: -3.5006904416158258&#xA;                }, &#xA;                &quot;qmean6&quot;: {&#xA;                    &quot;norm&quot;: 0.6339071011931049, &#xA;                    &quot;zscore&quot;: -3.0507104571131918&#xA;                }, &#xA;                &quot;solvation&quot;: {&#xA;                    &quot;norm&quot;: -0.609099435253941, &#xA;                    &quot;zscore&quot;: -1.2499924754236265&#xA;                }, &#xA;                &quot;ss_agreement&quot;: {&#xA;                    &quot;norm&quot;: 0.6706229586994028, &#xA;                    &quot;zscore&quot;: 0.05371252044245716&#xA;                }, &#xA;                &quot;torsion&quot;: {&#xA;                    &quot;norm&quot;: 0.0033300496230023145, &#xA;                    &quot;zscore&quot;: -2.7188811736675764&#xA;                }&#xA;            }, &#xA;            &quot;model_pdb&quot;: &quot;https://swissmodel.expasy.org/qmean/project/TgEjha/model_001.pdb&quot;, &#xA;            &quot;modelid&quot;: &quot;model_001&quot;, &#xA;            &quot;name&quot;: &quot;P02489.B99990005.pdb&quot;, &#xA;            &quot;seqres&quot;: [&#xA;                {&#xA;                    &quot;atomseq&quot;: &quot;MDVTIQHPWFKRTLGPFYPSRLFDQFFGEGLFEYDLLPFLSSTISPYYRQSLFRTVLDSGISEVRSDRDKFVIFLDVKHFSPEDLTVKVQDDFVEIHGKHNERQDDHGYISREFHRRYRLPSNVDQSALSCSLSADGMLTFCGPKIQTGLDATHAERAIPVSREEKPTSAPSS&quot;, &#xA;                    &quot;chain_name&quot;: &quot; &quot;, &#xA;                    &quot;name&quot;: &quot;seq_chain_0&quot;, &#xA;                    &quot;sequence&quot;: &quot;MDVTIQHPWFKRTLGPFYPSRLFDQFFGEGLFEYDLLPFLSSTISPYYRQSLFRTVLDSGISEVRSDRDKFVIFLDVKHFSPEDLTVKVQDDFVEIHGKHNERQDDHGYISREFHRRYRLPSNVDQSALSCSLSADGMLTFCGPKIQTGLDATHAERAIPVSREEKPTSAPSS&quot;&#xA;                }&#xA;            ]&#xA;        }&#xA;    ], &#xA;    &quot;options&quot;: {&#xA;        &quot;qmeanbrane&quot;: false, &#xA;        &quot;qmeandisco&quot;: true&#xA;    }, &#xA;    &quot;project_type&quot;: &quot;default&quot;, &#xA;    &quot;sequences&quot;: []&#xA;}, &#xA;&quot;status&quot;: &quot;COMPLETED&quot;&#xA;}&#xA;&amp;gt;&amp;gt;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;CASE B:&lt;/strong&gt; If I uncomment the last 5 lines and run 'python qmean.py', it doesn't work.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  andrea$ python qmean.py&#xA;  https://swissmodel.expasy.org/qmean/project/zrcLCK.json&#xA;  {&#xA;  &quot;status&quot;: &quot;QUEUEING&quot;&#xA;  }&#xA;  Traceback (most recent call last):&#xA;  File &quot;qmean.py&quot;, line 31, in &amp;lt;module&amp;gt;&#xA;  score  = qmean_score(url_d)&#xA;  File &quot;qmean.py&quot;, line 24, in qmean_score&#xA;  zscore = jdata['input_data']['models'][0]['global_scores']['qmean4']['zscore']&#xA;  KeyError: 'input_data'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I print 'r' variable of 'qmean_score' function, It returns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  &amp;lt;Response [200]&amp;gt;&#xA;  {&#xA;  &quot;status&quot;: &quot;QUEUEING&quot;&#xA;  }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1453" LastEditorUserId="1453" LastEditDate="2018-01-18T13:34:55.227" LastActivityDate="2018-06-28T13:20:43.110" Title="QMean standalone evaluation" Tags="|python|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3336" PostTypeId="1" AcceptedAnswerId="3337" CreationDate="2018-01-18T00:12:34.603" Score="1" ViewCount="25" Body="&lt;p&gt;I am using &lt;a href=&quot;https://github.com/pepkit/looper&quot; rel=&quot;nofollow noreferrer&quot;&gt;Looper&lt;/a&gt;. While running &lt;code&gt;looper run project_config.yaml&lt;/code&gt; I am getting &lt;code&gt;duplicate sample name&lt;/code&gt; error. I have two &lt;code&gt;fastq&lt;/code&gt; files for each sample, so in the &lt;code&gt;annotation.csv&lt;/code&gt; file I have:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sample_name,library,file&#xA;abeta-24-1,RNA-seq,data/merged/abeta_24h_1_r1.fq.gz&#xA;abeta-24-1,RNA-seq,data/merged/abeta_24h_1_r2.fq.gz&#xA;abeta-24-2,RNA-seq,data/merged/abeta_24h_2_r1.fq.gz&#xA;abeta-24-2,RNA-seq,data/merged/abeta_24h_2_r2.fq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My &lt;code&gt;pipeline_interface.yaml&lt;/code&gt; file looks the following way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;protocol_mapping:&#xA;  RNA-seq: sandro_rna_seq&#xA;&#xA;pipelines:&#xA;  sandro_rna_seq:&#xA;    name: sandro_rna_seq&#xA;    path: sandro_rna_seq.py  # relative to this &#xA;pipeline_interface.yaml file&#xA;    looper_args: True&#xA;    arguments:&#xA;      &quot;-L&quot;: &quot;IU&quot;&#xA;      &quot;-T&quot;: &quot;gencode_mouse_m13&quot;&#xA;      &quot;-E&quot;: &quot;/scratch/nv4e/binf-&#xA;pipelines/bulk_rna_seq/salmon/pypiper/jyvo_experiment-metadata.yaml&quot;&#xA;      &quot;-C&quot;: &quot;/scratch/nv4e/binf-&#xA;pipelines/bulk_rna_seq/salmon/pypiper/salmon_rna_seq.yaml&quot;&#xA;      &quot;-X&quot;: &quot;True&quot;&#xA;      &quot;-P&quot;: &quot;6&quot;&#xA;      &quot;-I&quot;: sample_name&#xA;    resources:&#xA;      default:&#xA;        file_size: &quot;0&quot;&#xA;        cores: &quot;20&quot;&#xA;        mem: &quot;120000&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Why am I getting this error? How could I fix it?&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2018-01-18T08:02:59.970" LastActivityDate="2018-01-18T08:02:59.970" Title="Looper: Duplicate sample name error" Tags="|scrnaseq|looper|yaml|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3337" PostTypeId="2" ParentId="3336" CreationDate="2018-01-18T01:16:24.287" Score="2" Body="&lt;p&gt;As explained in the &lt;a href=&quot;https://pepkit.github.io/docs/sample_annotation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pep docs&lt;/a&gt;, the &lt;code&gt;sample_name&lt;/code&gt; column should be considered a unique identifier for a sample. The PEP requires that &lt;strong&gt;One row corresponds to one sample&lt;/strong&gt; You have multiple rows with the same name; this is not allowed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You simply need to recast your annotation sheet such that it has &lt;code&gt;file1&lt;/code&gt; and &lt;code&gt;file2&lt;/code&gt; columns, but only 2 rows.&lt;/p&gt;&#xA;" OwnerUserId="829" LastEditorUserId="48" LastEditDate="2018-01-18T08:00:34.270" LastActivityDate="2018-01-18T08:00:34.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3338" PostTypeId="1" CreationDate="2018-01-18T02:36:33.407" Score="2" ViewCount="92" Body="&lt;p&gt;How can I retrieve only SNPs from the dbSNP's &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/snp/.redesign/latest_release/JSON/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FTP site&lt;/a&gt;. The files are gzipped and organized by human chromosome (i.e, chr[1-22,X,Y]), are these for one human? I need only SNPs data for various humans.&lt;/p&gt;&#xA;" OwnerUserId="2210" LastEditorUserId="48" LastEditDate="2018-01-18T15:12:53.357" LastActivityDate="2018-01-19T10:18:53.457" Title="How to retrieve SNPs data of different humans?" Tags="|genome|snp|phylogenetics|human|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3339" PostTypeId="1" AcceptedAnswerId="3343" CreationDate="2018-01-18T03:59:56.300" Score="1" ViewCount="38" Body="&lt;p&gt;While trying to run &lt;a href=&quot;https://github.com/pepkit/looper&quot; rel=&quot;nofollow noreferrer&quot;&gt;looper&lt;/a&gt; I am getting the following error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error (missing attribute): 'sandro_rna_seq' requires sample attribute '/scratch/nv4e/sandro_looper/pipeline/jyvo_experiment-metadata.yaml' for option/argument '-E'&lt;br&gt;&#xA;  Not submitted: Required attribute(s) missing for pipeline arguments string&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In my &lt;code&gt;pipeline_interface.yaml&lt;/code&gt; file I have the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;arguments:&#xA;  &quot;-L&quot;: &quot;IU&quot;&#xA;  &quot;-T&quot;: &quot;gencode_mouse_m13&quot;&#xA;  &quot;-E&quot;: &quot;/scratch/nv4e/sandro_looper/pipeline/jyvo_experiment-&#xA;metadata.yaml&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The file &lt;code&gt;&quot;/scratch/nv4e/sandro_looper/pipeline/jyvo_experiment-metadata.yaml&quot;&lt;/code&gt; is definitely present but it still gives an error...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2018-01-18T07:59:35.500" LastActivityDate="2018-01-18T12:36:06.103" Title="Looper: missing argument error" Tags="|scrnaseq|yaml|looper|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3341" PostTypeId="1" AcceptedAnswerId="3344" CreationDate="2018-01-18T11:20:59.670" Score="2" ViewCount="98" Body="&lt;p&gt;I have a file with sample data like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   90042011    90042031    AGTGCCACCAGGTGGGGCCG    90042017    2   5008    T:5006  C:2     90184334    90184354    GAGGCCACAAGAGGGCACAA    90184337    2   5008    C:5007  T:1     ß@  5&#xA;1   90042011    90042031    AGTGCCACCAGGTGGGGCCG    90042017    2   5008    T:5006  C:2     90184334    90184354    GAGGCCACAAGAGGGCACAA    90184336    2   5008    A:5007  G:1     ßß  2&#xA;1   90042011    90042031    AGTGCCACCAGGTGGGGCCG    90042030    2   5008    C:5007  A:1     90184334    90184354    GAGGCCACAAGAGGGCACAA    90184337    2   5008    C:5007  T:1     @@  1&#xA;1   90042011    90042031    AGTGCCACCAGGTGGGGCCG    90042030    2   5008    C:5007  A:1     90184334    90184354    GAGGCCACAAGAGGGCACAA    90184336    2   5008    A:5007  G:1     @ß  3&#xA;1   90042011    90042031    AGTGCCACCAGGTGGGGCCG    90042029    2   5008    G:5006  T:2     90184334    90184354    GAGGCCACAAGAGGGCACAA    90184337    2   5008    C:5007  T:1     @@  1&#xA;1   90042011    90042031    AGTGCCACCAGGTGGGGCCG    90042029    2   5008    G:5006  T:2     90184334    90184354    GAGGCCACAAGAGGGCACAA    90184336    2   5008    A:5007  G:1     @ß  3&#xA;1   94853396    94853416    GCCGCCACTAGATGGTGCTA    94853398    2   5008    T:5007  C:1     94969254    94969274    CAGTCCTCTAGAGGGAGCCC    94969266    2   5008    G:5006  A:2     ß@  5&#xA;2   103584283   103584303   TGGGCCACTAGGAGGCACTG    103584300   2   5008    C:5006  T:2     103841436   103841456   CATGCCACAAGAGGGCATCA    103841456   2   5008    A:5006  G:2     @ß  3&#xA;2   103584283   103584303   TGGGCCACTAGGAGGCACTG    103584300   2   5008    C:5006  T:2     103841436   103841456   CATGCCACAAGAGGGCATCA    103841445   2   5008    C:5006  T:2     @@  1&#xA;2   103584283   103584303   TGGGCCACTAGGAGGCACTG    103584284   2   5008    A:5007  G:1     103841436   103841456   CATGCCACAAGAGGGCATCA    103841456   2   5008    A:5006  G:2     ßß  2&#xA;2   103584283   103584303   TGGGCCACTAGGAGGCACTG    103584284   2   5008    A:5007  G:1     103841436   103841456   CATGCCACAAGAGGGCATCA    103841445   2   5008    C:5006  T:2     ß@  5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As seen, some lines have repeating values for &lt;code&gt;$2&lt;/code&gt; and &lt;code&gt;$3&lt;/code&gt;, line 1-6 and line 8-11 in this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;$20&lt;/code&gt; of my file contains codes of two characters representing each line, in the format &quot;@@&quot;,&quot;ßß&quot;, &quot;@ß&quot;,&quot;@N&quot;,&quot;ß@&quot;,&quot;ßN&quot;,&quot;N@&quot;,&quot;Nß&quot; or &quot;NN&quot;. Each line can have one of these 9 possible codes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to write a script, which splits the characters of &lt;code&gt;$20&lt;/code&gt; (@,ß or N) of the lines which are repeated (1-6 &amp;amp; 8-11 here), and reduces them into one line in the following manner:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Count the number of @, ß or N in the first and second positions respectively of the repeated lines. For the first and second characters of each repeated line, if:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;@ occurs most, print @&lt;/li&gt;&#xA;&lt;li&gt;ß occurs most, print ß&lt;/li&gt;&#xA;&lt;li&gt;N occurs most, print N&lt;/li&gt;&#xA;&lt;li&gt;@ occurs equal to ß, print N&lt;/li&gt;&#xA;&lt;li&gt;@ occurs equal to N, print @&lt;/li&gt;&#xA;&lt;li&gt;ß occurs equal to N, print ß&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In this sample dataset with repeated lines, the output file generated should be something like this, only columns &lt;code&gt;$1&lt;/code&gt;, &lt;code&gt;$2&lt;/code&gt;, &lt;code&gt;$3&lt;/code&gt;, &lt;code&gt;$4&lt;/code&gt;, &lt;code&gt;$11&lt;/code&gt;,&lt;code&gt;$12&lt;/code&gt;,&lt;code&gt;$13&lt;/code&gt;, and a new column with the new &quot;@&quot;,&quot;ß&quot;,&quot;N&quot; codes after said counting is reported:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   90042011    90042031    AGTGCCACCAGGTGGGGCCG    90184334    90184354    GAGGCCACAAGAGGGCACAA    @N  &#xA;1   94853396    94853416    GCCGCCACTAGATGGTGCTA    94969254    94969274    CAGTCCTCTAGAGGGAGCCC    ß@  &#xA;2   103584283   103584303   TGGGCCACTAGGAGGCACTG    103841436   103841456   CATGCCACAAGAGGGCATCA    NN  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the first repeated case(1-6), for the first character @ occurs four times and ß occurs twice and for the second character @ and ß occur equal number of times, so the final code is @N. In the second repeated case(8-11), @ and ß for both character positions occur equally, so the final code is NN.&lt;/p&gt;&#xA;" OwnerUserId="2094" LastEditorUserId="2094" LastEditDate="2018-01-18T12:51:17.260" LastActivityDate="2018-01-18T13:08:53.410" Title="Splitting characters of a column in repeated lines and reducing them to one line" Tags="|r|bioawk|text-processing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3342" PostTypeId="2" ParentId="3327" CreationDate="2018-01-18T12:32:09.620" Score="1" Body="&lt;p&gt;You could also take a look at &lt;a href=&quot;http://eddylab.org/infernal/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Infernal&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think your best bet is to use RNA-focused programs; I'm not aware of a DNA-specific one, which makes some sense because DNA is typically double-stranded, so the secondary structure community is primarily focused on RNA. I would predict that secondary structure potential of single-stranded DNA would be similar to that of RNA.&lt;/p&gt;&#xA;" OwnerUserId="829" LastActivityDate="2018-01-18T12:32:09.620" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3343" PostTypeId="2" ParentId="3339" CreationDate="2018-01-18T12:36:06.103" Score="2" Body="&lt;p&gt;You are using the argument values incorrectly. Argument values are sample attributes, not full character paths. Hence, &quot;IU&quot; is a sample attribute, &quot;gencode_mouse_m13&quot; is a sample attribute, so why would you think &quot;/scratch...&quot; would be treated any differently?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The pipeline interface maps sample attributes to command-line arguments. It is modulated by sample (different values per sample). What you're trying to do is pass a constant string to an argument.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this, there is a &lt;code&gt;pipeline_args&lt;/code&gt; section of your project config. Please read the docs on &lt;a href=&quot;https://pepkit.github.io/docs/project_config/&quot; rel=&quot;nofollow noreferrer&quot;&gt;project config&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="829" LastActivityDate="2018-01-18T12:36:06.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3344" PostTypeId="2" ParentId="3341" CreationDate="2018-01-18T12:52:14.460" Score="2" Body="&lt;p&gt;OK. This is a bit convoluted, but it works. There might be simpler solutions...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First we'll write a function that takes a data frame and a character position (ie; first or second position in the string), groups the data frame by the second and third columns, counts occurrences of each character in the specified position of the 18th column, and returns a vector according to your ranking scheme (this doesn't include counts of N since it doesn't occur in your example data, but it's a simple extension of the &lt;code&gt;case_when&lt;/code&gt; line to add it):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(dplyr)&#xA;library(tidyr)&#xA;library(stringr)&#xA;&#xA;summarise_characters &amp;lt;- function(df, pos) {&#xA;  df %&amp;gt;% mutate(character=str_sub(X18, pos, pos)) %&amp;gt;% &#xA;  group_by(X2, X3, character) %&amp;gt;% &#xA;  summarise(n=n()) %&amp;gt;% &#xA;  ungroup() %&amp;gt;% &#xA;  spread(character, n, fill=0) %&amp;gt;% &#xA;  mutate(pos1=case_when(`@` &amp;gt; ß ~ '@', &#xA;                        `@` == ß ~ 'N', &#xA;                        `@` &amp;lt; ß ~ 'ß')) %&amp;gt;%&#xA;  `$`(pos1)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Next we remove duplicate lines from the data frame, then create a new column, pasting together the output of our function called on the first and second character:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df %&amp;gt;% group_by(X2, X3) %&amp;gt;%  &#xA;  slice(1) %&amp;gt;%&#xA;  ungroup() %&amp;gt;%&#xA;  mutate(counts=paste0(summarise_characters(df, 1), summarise_characters(df, 2))) %&amp;gt;% &#xA;  select(X1, X2, X3, X4, X11, X12, X13, counts)&#xA;&#xA;# A tibble: 3 x 8&#xA;     X1        X2        X3                   X4       X11                  X12       X13 counts&#xA;  &amp;lt;int&amp;gt;     &amp;lt;int&amp;gt;     &amp;lt;int&amp;gt;                &amp;lt;chr&amp;gt;     &amp;lt;int&amp;gt;                &amp;lt;chr&amp;gt;     &amp;lt;int&amp;gt;  &amp;lt;chr&amp;gt;&#xA;1     1  90042011  90042031 AGTGCCACCAGGTGGGGCCG  90184354 GAGGCCACAAGAGGGCACAA  90184337     @N&#xA;2     1  94853396  94853416 GCCGCCACTAGATGGTGCTA  94969274 CAGTCCTCTAGAGGGAGCCC  94969266     ß@&#xA;3     2 103584283 103584303 TGGGCCACTAGGAGGCACTG 103841456 CATGCCACAAGAGGGCATCA 103841456     NN&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="581" LastEditorUserId="581" LastEditDate="2018-01-18T13:08:53.410" LastActivityDate="2018-01-18T13:08:53.410" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="3345" PostTypeId="2" ParentId="2470" CreationDate="2018-01-18T16:01:17.680" Score="8" Body="&lt;p&gt;Conversion using &lt;code&gt;R&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(biomaRt)&#xA;&#xA;mart &amp;lt;- useDataset(&amp;quot;hsapiens_gene_ensembl&amp;quot;, useMart(&amp;quot;ensembl&amp;quot;))&#xA;&#xA;    genes &amp;lt;- getBM(&#xA;      filters=&amp;quot;ensembl_gene_id&amp;quot;,&#xA;      attributes=c(&amp;quot;ensembl_gene_id&amp;quot;, &amp;quot;entrezgene_id&amp;quot;),&#xA;      values=ensembl.genes,&#xA;      mart=mart)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Where &lt;code&gt;ensembl.genes&lt;/code&gt; is a vector of Ensembl gene IDs.&lt;/p&gt;&#xA;" OwnerUserId="208" LastEditorUserId="10386" LastEditDate="2023-09-07T16:15:48.133" LastActivityDate="2023-09-07T16:15:48.133" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3346" PostTypeId="2" ParentId="3338" CreationDate="2018-01-19T10:18:53.457" Score="4" Body="&lt;p&gt;If you mean that you want snps from individuals, instead of all together, you can find them in &lt;a href=&quot;http://www.internationalgenome.org/data&quot; rel=&quot;nofollow noreferrer&quot;&gt;1000 genomes&lt;/a&gt;. Here different individuals from different populations are sequenced and variants are called, including snps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe you should add to your question what your goal is, what would you like to find?&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2018-01-19T10:18:53.457" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3347" PostTypeId="1" CreationDate="2018-01-19T11:56:33.463" Score="1" ViewCount="27" Body="&lt;p&gt;I would like to use &lt;a href=&quot;https://en.wikipedia.org/wiki/Direct_coupling_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;DCA&lt;/a&gt; (direct coupling analysis) to analyze the interface between 2 protein ensembles (how the 2 ensembles interacts). Do you have access to a tutorial on this matter? A source code? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found &lt;a href=&quot;https://github.com/carlobaldassi/GaussDCA.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; Julia library and &lt;a href=&quot;http://stanford.edu/~jesszhao/files/pydcaPresentation.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; library by stanford, but I am not sure if it's the right way to go and how to use these 2 for a simple test case of binding between a $\text{Ca}^{2+}$/calmodulin and a &lt;a href=&quot;https://en.wikipedia.org/wiki/Ca2%2B/calmodulin-dependent_protein_kinase_II&quot; rel=&quot;nofollow noreferrer&quot;&gt;CaMKII&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2018-01-21T21:09:17.450" LastActivityDate="2018-01-21T21:09:17.450" Title="How to use DCA to deduce the contacts between 2 proteins ensembles?" Tags="|sequence-analysis|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3348" PostTypeId="1" AcceptedAnswerId="3353" CreationDate="2018-01-19T20:50:54.970" Score="1" ViewCount="302" Body="&lt;p&gt;I have a dataframe containing missing values. I want to impute them with knnImputation as shown below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my question is: are data imputed row wise or column wise and is there a way to give this option in knnImputation?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A &amp;lt;-c(1,2,3,4,5, NA, NA)&#xA;B &amp;lt;-c(2,3,NA, 2,4, 7, 6)&#xA;C &amp;lt;-c(2,3,4, NA, 5, 1,7)&#xA;D&amp;lt;-21:27&#xA;data&amp;lt;-data.frame(A,B, C, D)&#xA;new_data&amp;lt;-knnImputation(data, k = 3, scale = T, meth = &quot;weighAvg&quot;, distData = NULL)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1622" LastEditorUserId="1622" LastEditDate="2018-12-02T09:30:50.513" LastActivityDate="2018-12-02T09:30:50.513" Title="knnImputation for missing data" Tags="|r|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3349" PostTypeId="1" AcceptedAnswerId="3350" CreationDate="2018-01-19T21:49:02.163" Score="8" ViewCount="9612" Body="&lt;p&gt;In this &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/3194/1771&quot;&gt;answer&lt;/a&gt;, it is stated that ribosomal genes should be excluded prior to normalization in scRNA-seq as contaminants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do mitochondrial genes have to be excluded as well? I plotted the top 50 expressed genes for a specific dataset and they tend to appear often (for example &lt;a href=&quot;https://ghr.nlm.nih.gov/gene/MT-ATP6&quot; rel=&quot;nofollow noreferrer&quot;&gt;MT-ATP6&lt;/a&gt;). My assumption is that, given that they work for mitochondrial function and may be highly expressed, they can dilute the signal of gene differential across cell types but expressed at lower levels. Is this biologically sound?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, in &lt;a href=&quot;https://hemberg-lab.github.io/scRNA.seq.course/construction-of-expression-matrix.html#exprs-qc&quot; rel=&quot;nofollow noreferrer&quot;&gt;this course&lt;/a&gt;, mitochondrial genes are used to filter cells when they contribute above a certain ratio of total RNA of single cells. However, I could not find an explanation for this filtering. What is the underlying process this procedure is used to account for?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2018-07-24T17:30:23.450" LastActivityDate="2018-07-26T13:24:53.497" Title="Are mitochondrial genes to exclude in scRNA-seq such as ribosomal genes?" Tags="|rna-seq|scrnaseq|quality-control|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3350" PostTypeId="2" ParentId="3349" CreationDate="2018-01-19T23:02:16.023" Score="6" Body="&lt;p&gt;According to Ilicic et al. (2016), on upregulation of mtRNA in broken cells:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;There is an extensive literature on the relationship between mtDNA, mitochondrially localized proteins, and cell death [34, 35]. However, upregulation of RNA levels of mtDNA in broken cells suggests losses in cytoplasmic content. In a situation where cell membrane is broken, cytoplasmic RNA will be lost, but RNAs enclosed in the mitochondria will be retained [...]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;According to this, I think that mtRNA levels relative to endogenous RNA can be used as a control for low quality (broken) cells, in a similar way to ERCC spike-ins.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ilicic T, Kim JK, Kolodziejczyk AA, Bagger FO, McCarthy DJ, Marioni JC, Teichmann SA. &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-0888-1&quot; rel=&quot;noreferrer&quot;&gt;Classification of low quality cells from single-cell RNA-seq data.&lt;/a&gt; Genome biology. 2016 Feb 17;17(1):29.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-01-19T23:02:16.023" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3351" PostTypeId="1" AcceptedAnswerId="3352" CreationDate="2018-01-20T02:47:17.443" Score="0" ViewCount="54" Body="&lt;p&gt;I have multiple samples that I need to run one pipeline on. I am using Looper: &lt;a href=&quot;https://github.com/pepkit/looper&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/pepkit/looper&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a real trouble specifying a separate location for each pipeline output files. The first time when I did not care about it thinking that &lt;code&gt;Looper&lt;/code&gt; would just automatically do that, it lead to multiple files being overwritten. Then, trying to avoid it I tried (and it did not work):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Adding &lt;code&gt;{sample_name}&lt;/code&gt; to the &lt;code&gt;output_dir&lt;/code&gt; of the &lt;code&gt;metadata&lt;/code&gt; section of &lt;code&gt;project_config.yaml&lt;/code&gt; file:&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; metadata:&#xA;  sample_annotation: sandro_annotation.csv&#xA;  output_dir: /scratch/nv4e/sandro_looper_results/{sample_name}&#xA;  pipelines_dir: [../pipeline/pipeline_interface.yaml]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;2&quot;&gt;&#xA;  &lt;li&gt;Creating a &lt;code&gt;derived_column&lt;/code&gt; in the &lt;code&gt;.csv&lt;/code&gt; annotation file:&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sample_name,library,sample_name_1_read,sample_name_2_read,output_folder&#xA;abeta_24h_1,RNA-seq,read1,read2,out&#xA;abeta_24h_2,RNA-seq,read1,read2,out&#xA;abeta_24h_3,RNA-seq,read1,read2,out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;pipeline_interface.yaml&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;pipelines:&#xA;  sandro_rna_seq:&#xA;    name: sandro_rna_seq&#xA;    path: sandro_rna_seq.py  # relative to this &#xA;pipeline_interface.yaml file&#xA;    looper_args: True&#xA;    arguments:&#xA;      &quot;-S&quot;: sample_name&#xA;      &quot;--input&quot;: sample_name_1_read&#xA;      &quot;-O&quot;: output_folder&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and &lt;code&gt;project.config.yaml&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;derived_columns: [sample_name_1_read,sample_name_2_read,output_folder]&#xA;data_sources:&#xA;  read1: /scratch/nv4e/sandro_looper/data/merged/{sample_name}_r1.fq.gz&#xA;  read2: /scratch/nv4e/sandro_looper/data/merged/{sample_name}_r2.fq.gz&#xA;  out: /scratch/nv4e/sandro_looper_results/results_pipeline/{sample_name}&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this particular case the parameter &lt;code&gt;-O&lt;/code&gt; was formed correctly which I was able to see in &lt;code&gt;sandro_looper_results/submissions/&lt;/code&gt; &lt;code&gt;.sub&lt;/code&gt; file, however, it was supplied the second time, apparently from the &lt;code&gt;output_dir&lt;/code&gt; section of the &lt;code&gt;project_config.yaml&lt;/code&gt; and was overriding the first &lt;code&gt;-O&lt;/code&gt; command. So, obviously I tried to delete &lt;code&gt;output_dir&lt;/code&gt; line but it is a required setting for &lt;code&gt;Looper&lt;/code&gt;, so it can not run without it.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;3&quot;&gt;&#xA;  &lt;li&gt;I have also tried supplying &lt;code&gt;-O&lt;/code&gt; command in &lt;code&gt;project_config.yaml&lt;/code&gt; in the &lt;code&gt;pipeline_args:&lt;/code&gt; section with no success.&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I really do not know what I could do at this point. Any suggestions would be greatly appreciated,&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-01-20T03:29:58.567" Title="Looper: can't specify the individual path for one pipeline run" Tags="|scrnaseq|yaml|looper|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3352" PostTypeId="2" ParentId="3351" CreationDate="2018-01-20T03:29:58.567" Score="1" Body="&lt;p&gt;Looper does not generate any of these per-sample outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looper does not have any control over where your pipeline puts its outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is in your pipeline, not in looper. Assuming you're using a pypiper pipeline, I suggest reading the docs; for example, the &quot;your first pipeline&quot; tutorial: &lt;a href=&quot;http://pypiper.readthedocs.io/en/latest/tutorials-first.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://pypiper.readthedocs.io/en/latest/tutorials-first.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For real-life example, see &lt;a href=&quot;https://github.com/databio/dnameth_pipelines/blob/3e6abd03185f8e2d5ef1c72849afdf9e5033f269/src/rrbs.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this pipeline&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&#xA;outfolder = os.path.abspath(os.path.join(args.output_parent, args.sample_name))&#xA;pm = pypiper.PipelineManager(name=&quot;RRBS&quot;, outfolder=outfolder, args=args, version=__version__)&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="829" LastActivityDate="2018-01-20T03:29:58.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3353" PostTypeId="2" ParentId="3348" CreationDate="2018-01-20T10:39:24.100" Score="4" Body="&lt;p&gt;The data are imputed using values from the same column&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a simplified example to show what's going on:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A &amp;lt;-c(1,2,3,4,5)&#xA;B &amp;lt;-c(2,3,NA, 2,4)&#xA;C &amp;lt;- c(2,3,4, 6, 1)&#xA;data&amp;lt;-data.frame(A,B,C)&#xA;data&#xA;&#xA;#&amp;gt;  A  B C&#xA;#&amp;gt;1 1  2 2&#xA;#&amp;gt;2 2  3 3&#xA;#&amp;gt;3 3 NA 4&#xA;#&amp;gt;4 4  2 6&#xA;#&amp;gt;5 5  4 1&#xA;&#xA;# I'm using scale = F to make things simpler&#xA;knnImputation(data, k = 3, scale = F, meth = &quot;weighAvg&quot;, distData = NULL)&#xA;&#xA;#&amp;gt;  A        B C&#xA;#&amp;gt;1 1 2.000000 2&#xA;#&amp;gt;2 2 3.000000 3&#xA;#&amp;gt;3 3 2.594272 4&#xA;#&amp;gt;4 4 2.000000 6&#xA;#&amp;gt;5 5 4.000000 1&#xA;&#xA;# calculate euclidean distances, dropping column of interest&#xA;dist(data[,-2])&#xA;&#xA;#&amp;gt;1        2        3        4&#xA;#&amp;gt;2 1.414214                           &#xA;#&amp;gt;3 2.828427 1.414214                  &#xA;#&amp;gt;4 5.000000 3.605551 2.236068         &#xA;#&amp;gt;5 4.123106 3.605551 3.605551 5.099020&#xA;&#xA;# use k rows with smallest distance to row of interest to calculate weighted average&#xA;# using exp(-dist) as weights&#xA;weight_sum &amp;lt;- weight_sum = exp(-2.828427) + exp(-1.414214) + exp(-2.236068)&#xA;2*exp(-2.828427)/weight_sum + 3*exp(-1.414214)/weight_sum + 2*exp(-2.236068)/weight_sum&#xA;&#xA;#&amp;gt;2.594271&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want to impute using values from the same row, all you have to do is transpose the matrix:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;t(knnImputation(t(data), k = 2, scale = F, meth = &quot;weighAvg&quot;, distData = NULL))&#xA;&#xA;#&amp;gt;     A        B C&#xA;#&amp;gt;[1,] 1 2.000000 2&#xA;#&amp;gt;[2,] 2 3.000000 3&#xA;#&amp;gt;[3,] 3 3.086729 4&#xA;#&amp;gt;[4,] 4 2.000000 6&#xA;#&amp;gt;[5,] 5 4.000000 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(I had to lower k because there are only 2 extra columns)&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-01-20T10:39:24.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3355" PostTypeId="1" CreationDate="2018-01-21T08:34:51.230" Score="1" ViewCount="141" Body="&lt;p&gt;I am new to &lt;a href=&quot;https://www.genetics.ucla.edu/software/admixture/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Admixture&lt;/a&gt; and I would like to use fastNGSadmix to look for admixture in one individual (modern). Admixture is supposed to look for proportions of ancestral populations in a sample (either a population or an individual sample). for that I will need to prepare my reference population set to compare with my actual sample which is an individual form region X for instance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I am thinking whether I have to add the X population to my reference set or not? If yes, wouldn't it introduce bias? Like my individual will always show the X population as the first admixing population?&lt;/p&gt;&#xA;" OwnerUserId="2224" LastEditorUserId="57" LastEditDate="2018-01-23T12:35:32.283" LastActivityDate="2018-01-24T10:00:52.347" Title="population admixture" Tags="|reference-genome|phylogenetics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3356" PostTypeId="1" AcceptedAnswerId="3357" CreationDate="2018-01-21T11:47:55.590" Score="1" ViewCount="50" Body="&lt;p&gt;Is it worth mapping bisulfite reads (WGBS) to a reduced size genome?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm interested only in modifications in CpG context, thus instead of mapping to a whole genome (human genome) I would:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Extract DNA sequence around each CpG (-1000/+1000 bp)&lt;/li&gt;&#xA;&lt;li&gt;Map reads (using bismark) to this subset&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My idea is to reduce mapping time, however afraid that there might be some mapping bias or other issues.&lt;/p&gt;&#xA;" OwnerUserId="186" LastActivityDate="2018-01-21T22:36:08.867" Title="Mapping bisulfite reads to a reduced size genome" Tags="|ngs|read-mapping|sequence-alignment|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3357" PostTypeId="2" ParentId="3356" CreationDate="2018-01-21T13:29:52.173" Score="3" Body="&lt;p&gt;CG sequences are pretty common; I expect you'll hit almost all of the genome when looking at a 2kb window centred on every CG, so there wouldn't be much point in doing that for efficiency purposes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/users/339/dariober&quot;&gt;dariober&lt;/a&gt; has pointed out that reads coming from elsewhere in the genome could be incorrectly mapped to the reduced genome with high mapq score since the best, genuine genomic region is not present. So even if you are interested in few small regions, you should still map to the entire genome.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-01-21T22:36:08.867" LastActivityDate="2018-01-21T22:36:08.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3358" PostTypeId="2" ParentId="3356" CreationDate="2018-01-21T18:29:51.720" Score="4" Body="&lt;p&gt;Don't restrict the portion of the genome you're mapping to, you'll just increase false-positive alignments by doing so. If you're worried about mapping time, then (A) use more cores or (B) use a different mapper (BSMAP was the quickest out there last I looked, though in my opinion it's fine to wait a bit longer for better quality results (I normally suggest bwameth to people)).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As pointed out by &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/180/719016&quot;&gt;user 719016&lt;/a&gt; in the comments, the above does not apply to amplicon sequencing. With that, you'll get VASTLY better performance if align to a reduced-size genome...and without an appreciably increased false-positive rate (unless your target-enrichment for sequencing is poor).&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-01-21T20:58:12.260" LastActivityDate="2018-01-21T20:58:12.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3359" PostTypeId="1" AcceptedAnswerId="3366" CreationDate="2018-01-21T20:03:21.367" Score="0" ViewCount="2561" Body="&lt;p&gt;I am trying to run &lt;a href=&quot;https://cran.r-project.org/web/packages/Rtsne/Rtsne.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;{Rtsne}&lt;/a&gt;. On &lt;code&gt;pca&lt;/code&gt; dataset prepared by using &lt;code&gt;dudi.pca&lt;/code&gt; from &lt;a href=&quot;https://pbil.univ-lyon1.fr/ade4/ade4-html/dudi.pca.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;{ade4}&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For pca calculation I selected 40 components while specifying parameters for &lt;code&gt;dudi.pca&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pca = dudi.pca(counts(sce_normalized), scannf=FALSE, nf=40)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;{Rtsne} gives me an error: “perplexity is too large”, how could it be fixed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code I am trying to run is below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;name_of_file &amp;lt;- &quot;dudi_pca.rds&quot;&#xA;path_to_file &amp;lt;- paste0(&quot;/scratch/nv4e/scRNA-&#xA;seq/normalization/new_datasets/&quot;,&#xA;                   name_of_file, sep=&quot;&quot;)&#xA;dudi_pca &amp;lt;- readRDS(path_to_file)&#xA;&#xA;# run Rtsne with default parameters&#xA;rtsne_out &amp;lt;- Rtsne(as.matrix(dudi_pca), pca=FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2018-01-23T07:47:57.163" LastActivityDate="2018-01-23T07:47:57.163" Title="Perplexity is too large" Tags="|r|pca|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3360" PostTypeId="1" CreationDate="2018-01-21T20:18:29.870" Score="2" ViewCount="43" Body="&lt;p&gt;Given all the genome variation information we have accumulated from resequencing human genomes up until now (early 2018), how much variation is left that hasn't been sequenced?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: given one of the answers, and the fact that 99% of the SNPs at 1% or more are found, is there a quantitative measure of remaining variation?&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="180" LastEditDate="2018-01-22T07:47:44.967" LastActivityDate="2018-01-22T07:47:44.967" Title="remaining human genome variation that hasn't been sequenced?" Tags="|genome|snp|human|variation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3361" PostTypeId="2" ParentId="3360" CreationDate="2018-01-22T00:16:47.497" Score="5" Body="&lt;p&gt;In theory, almost any base in the human genome may mutate, so you have billions of variants to go. Ok, this is not so useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A related and potentially useful question is: given a human, what is the fraction of his/her variants seen in the the previously sequenced samples? For a Wright-Fisher population, there is an analytical answer: x% of variants on a haplotype has frequency below x%. Replace &quot;x&quot; with &quot;1&quot;: 1% of variants on a haplotype have frequency below 1%. The 1000 genomes project concluded that 99% of SNPs at frequency 1% or higher have been called by the project. If we all came from a Wright-Fisher population, 99% of of SNPs have been called in 1000g. Given that the project also calls SNPs at lower frequency, nearly all SNPs in a newly sequenced sample are present in 1000g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above is a theoretical analysis. In practice, there are two major complications. First, we are not a Wright-Fisher population. Due to recent population expansion, each haplotype harbors more new mutations than the Wright-Fisher model would predict. Given a known population history, it is actually possible to numerically compute $f(x)$, the fraction of variants with frequency below x (for Wright-Fisher, $f(x)=x$). I don't have this result, though. Second, 1000g and many other genome projects didn't call SNPs in repetitive regions and missed a significant portion of high-frequency indels and structural variations. With Illumina, you can only call ~30% of long deletions callable with PacBio data, but we only have a dozen of PacBio human genomes in public. There are still a lot we don't know.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-01-22T00:16:47.497" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3362" PostTypeId="2" ParentId="3252" CreationDate="2018-01-22T06:07:18.690" Score="0" Body="&lt;p&gt;IRFinder can be used to detect the IR events and can easily reach your goal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example. IRFinder can calculate the IR ratio to detect IR event. mimirna.centenary.org.au/irfinder/example1.html. (You just need run IRFinder for one condition and each replicate will output a folder). Two conditions can be used to detect the differential IR events. If you have only one condition, you can select the IR events from the common events of the three replicates or select IR events exist at least two of three replicates based on the main output files( named IRFinder-IR-dir.txt). The IR events can be selected by filtering the IR ratio, Intron coverage etc&lt;/p&gt;&#xA;" OwnerUserId="2229" LastEditorUserId="77" LastEditDate="2018-03-04T08:02:22.913" LastActivityDate="2018-03-04T08:02:22.913" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3363" PostTypeId="1" AcceptedAnswerId="3364" CreationDate="2018-01-22T11:48:07.763" Score="1" ViewCount="227" Body="&lt;p&gt;I'm having some problems in exporting the annotations of some variants from a VCF file, after using the &quot;&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/VariantAnnotation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;VariantAnnotation&lt;/a&gt;&quot; package from bioconductor. Below the script I've used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env Rscript&#xA;&#xA;library(VariantAnnotation)&#xA;library(BSgenome.Scerevisiae.UCSC.sacCer3)&#xA;library(TxDb.Scerevisiae.UCSC.sacCer3.sgdGene)&#xA;&#xA;args = commandArgs(trailingOnly=TRUE)&#xA;&#xA;vcf &amp;lt;- readVcf(args[1], &quot;sacCer3&quot;)&#xA;&#xA;sc &amp;lt;- BSgenome.Scerevisiae.UCSC.sacCer3&#xA;txdb &amp;lt;- TxDb.Scerevisiae.UCSC.sacCer3.sgdGene&#xA;&#xA;coding &amp;lt;- data.frame(predictCoding(vcf, txdb, seqSource=sc))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;coding&lt;/code&gt; dataframe looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  seqnames start end width strand paramRangeID REF     ALT QUAL FILTER&#xA;1     chrI   455 455     1      +         &amp;lt;NA&amp;gt;   C       T   40   PASS&#xA;2     chrI   455 455     1      +         &amp;lt;NA&amp;gt;   C           40   PASS&#xA;3     chrI   455 455     1      +         &amp;lt;NA&amp;gt;   C       G   40   PASS&#xA;4     chrI   455 455     1      +         &amp;lt;NA&amp;gt;   C       A   40   PASS&#xA;5     chrI   456 456     1      +         &amp;lt;NA&amp;gt;   T       A   40   PASS&#xA;6     chrI   456 456     1      +         &amp;lt;NA&amp;gt;   C CCTGCTC   40   PASS&#xA;  varAllele CDSLOC.start CDSLOC.end CDSLOC.width PROTEINLOC QUERYID TXID CDSID&#xA;1         T          121        121            1         41       1    1     1&#xA;2                    121        121            1         41       2    1     1&#xA;3         G          121        121            1         41       3    1     1&#xA;4         A          121        121            1         41       4    1     1&#xA;5         A          122        122            1         41       5    1     1&#xA;6   CCTGCTC          122        122            1         41       6    1     1&#xA;   GENEID    CONSEQUENCE REFCODON  VARCODON REFAA VARAA&#xA;1 YAL069W  nonsynonymous      CTC       TTC     L     F&#xA;2 YAL069W not translated      CTC        TC            &#xA;3 YAL069W  nonsynonymous      CTC       GTC     L     V&#xA;4 YAL069W  nonsynonymous      CTC       ATC     L     I&#xA;5 YAL069W  nonsynonymous      CTC       CAC     L     H&#xA;6 YAL069W  nonsynonymous      CTC CCCTGCTCC     L   PCS&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But if I try to save it to file, for instance using the following command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;write.table(coding, 'test.tsv', sep='\t', row.names=F)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output table looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;seqnames&quot;      &quot;start&quot; &quot;end&quot;   &quot;width&quot; &quot;strand&quot;        &quot;paramRangeID&quot;  &quot;REF&quot;   &quot;ALT&quot;   &quot;QUAL&quot;  &quot;FILTER&quot;        &quot;varAllele&quot;     &quot;CDSLOC.start&quot;  &quot;CDSLOC.end&quot;    &quot;CDSLOC.width&quot;  &quot;PROTEINLOC&quot;    &quot;QUERYID&quot;       &quot;TXID&quot;  &quot;CDSID&quot; &quot;GENEID&quot;        &quot;CONSEQUENCE&quot;   &quot;REFCODON&quot;      &quot;VARCODON&quot;      &quot;REFAA&quot; &quot;VARAA&quot;             &#xA;&quot;chrI&quot;  455     455     1       &quot;+&quot;     NA      &quot;C&quot;     &amp;lt;S4 object of class &quot;DNAStringSet&quot;&amp;gt;     40      &quot;PASS&quot;  &quot;T&quot;     121     121     1       41      1       &quot;1&quot;     1       &quot;YAL069W&quot;       &quot;nonsynonymous&quot; &quot;CTC&quot;   &quot;TTC&quot;   &quot;L&quot;     &quot;F&quot;                            &#xA;&quot;chrI&quot;  455     455     1       &quot;+&quot;     NA      &quot;C&quot;     &amp;lt;S4 object of class &quot;DNAStringSet&quot;&amp;gt;     40      &quot;PASS&quot;  &quot;&quot;      121     121     1       41      2       &quot;1&quot;     1       &quot;YAL069W&quot;       &quot;not translated&quot; &quot;CTC&quot;   &quot;TC&quot;    &quot;&quot;      &quot;&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Notice the &lt;code&gt;&amp;lt;S4 object of class &quot;DNAStringSet&quot;&amp;gt;&lt;/code&gt; bit. Is there any way to force R to give me the same representation as &lt;code&gt;head&lt;/code&gt;? I couldn't find anything in Biostrings' documentation.&lt;/p&gt;&#xA;" OwnerUserId="123" LastEditorUserId="96" LastEditDate="2019-01-15T14:25:53.570" LastActivityDate="2019-01-15T14:25:53.570" Title="Write the output of the VariantAnnotation package to file" Tags="|r|bioconductor|variation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3364" PostTypeId="2" ParentId="3363" CreationDate="2018-01-22T12:04:38.390" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there any way to force R to give me the same representation as &lt;code&gt;head&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;head&lt;/code&gt; just cuts off everything after the first six entries. The magic, as it were, happens in the default &lt;code&gt;print&lt;/code&gt; method for the specific type. But the representation generated by it is only useful for ad-hoc display, not for storage in a structured data file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best solution is therefore to manually convert the column types. Since it’s only one column:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;coding$ALT = sapply(coding$ALT, as.character)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I’m a bit puzzled that the &lt;code&gt;REF&lt;/code&gt; and &lt;code&gt;ALT&lt;/code&gt; column have different types, anyway: that’s probably a bug in the {VariantAnnotation} package that should be reported.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, you probably want to use &lt;code&gt;quote = FALSE&lt;/code&gt; for the &lt;code&gt;write.table&lt;/code&gt;. In general, &lt;code&gt;write.table&lt;/code&gt; has terrible defaults. &lt;code&gt;readr::write_tsv&lt;/code&gt; is almost always a better choice.&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="123" LastEditDate="2018-01-22T14:51:48.223" LastActivityDate="2018-01-22T14:51:48.223" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3365" PostTypeId="1" AcceptedAnswerId="3367" CreationDate="2018-01-22T17:01:05.837" Score="16" ViewCount="5272" Body="&lt;p&gt;I have an R package on github which uses multiple Bioconductor dependencies, 'myPackage'&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I include CRAN packages in the DESCRIPTION via &lt;code&gt;Depends:&lt;/code&gt;, the packages will automatically install upon installation via devtools, i.e. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;devtools::install_github('repoName/myPackage')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is discussed in Section 1.1.3 Package Dependencies, in &lt;a href=&quot;https://cran.r-project.org/doc/manuals/r-release/R-exts.html&quot; rel=&quot;noreferrer&quot;&gt;Writing R Extensions&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to streamline this such that packages from Bioconductor are automatically installed as well?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normally, users install Bioconductor packages via BiocLite, e.g. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;source(&quot;http://www.bioconductor.org/biocLite.R&quot;)&#xA;biocLite(&quot;edgeR&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="146" LastActivityDate="2022-01-21T10:21:49.897" Title="R package development: How does one automatically install Bioconductor packages upon package installation?" Tags="|r|bioconductor|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3366" PostTypeId="2" ParentId="3359" CreationDate="2018-01-22T18:29:40.477" Score="2" Body="&lt;p&gt;I was using &lt;code&gt;dudi_pca&lt;/code&gt; incorrectly. The supplied parameter to &lt;code&gt;as.matrix()&lt;/code&gt; should have been &lt;code&gt;dudi_pca$li&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rtsne_out &amp;lt;- Rtsne(as.matrix(dudi_pca$li), pca=FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-01-22T18:29:40.477" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3367" PostTypeId="2" ParentId="3365" CreationDate="2018-01-22T19:04:53.820" Score="10" Body="&lt;p&gt;There's a trick to this where &lt;a href=&quot;https://github.com/hadley/devtools/issues/700&quot; rel=&quot;noreferrer&quot;&gt;one needs to add &lt;code&gt;biocViews:&lt;/code&gt; to the package Description&lt;/a&gt;. That's the only solution I've ever seen to allowing automatic installation of bioconductor dependencies. If you need a couple of examples, then click through the link I posted and scroll down to pull requests referencing that issue, they will generally include the actual example.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-22T19:04:53.820" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3368" PostTypeId="1" AcceptedAnswerId="3369" CreationDate="2018-01-23T09:16:50.820" Score="2" ViewCount="184" Body="&lt;p&gt;What's the easiest way to run Picard GCBias ignoring certain chromosomes/sequences in the reference? Looking at the CollectGcBiasMetrics, it seems there isn't a bed file option that can be passed.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar ~/picard-tools-1.134/picard.jar CollectGcBiasMetrics --help&#xA;USAGE: CollectGcBiasMetrics [options]&#xA;&#xA;Documentation: http://broadinstitute.github.io/picard/command-line-overview.html#CollectGcBiasMetrics&#xA;&#xA;Tool to collect information about GC bias in the reads in a given BAM file. Computes the number of windows (of size specified by WINDOW_SIZE) in the genome at each GC% and counts the number of read starts in each GC bin.  What is output and plotted is the &quot;normalized coverage&quot; in each bin - i.e. the number of reads per window normalized to the average number of reads per window across the whole genome..&#xA;&#xA;Version: 1.134(a7a08c474e4d99346eec7a9956a8fe71943b5d80_1434033355)&#xA;&#xA;&#xA;Options:&#xA;&#xA;--help&#xA;-h                            Displays options specific to this tool.&#xA;&#xA;--stdhelp&#xA;-H                            Displays options specific to this tool AND options common to all Picard command line &#xA;                              tools.&#xA;&#xA;--version                     Displays program version.&#xA;&#xA;CHART_OUTPUT=File&#xA;CHART=File                    The PDF file to render the chart to.  Required. &#xA;&#xA;SUMMARY_OUTPUT=File&#xA;S=File                        The text file to write summary metrics to.  Default value: null. &#xA;&#xA;WINDOW_SIZE=Integer           The size of windows on the genome that are used to bin reads.  Default value: 100. This &#xA;                              option can be set to 'null' to clear the default value. &#xA;&#xA;MINIMUM_GENOME_FRACTION=DoubleFor summary metrics, exclude GC windows that include less than this fraction of the &#xA;                              genome.  Default value: 1.0E-5. This option can be set to 'null' to clear the default &#xA;                              value. &#xA;&#xA;IS_BISULFITE_SEQUENCED=Boolean&#xA;BS=Boolean                    Whether the SAM or BAM file consists of bisulfite sequenced reads.  Default value: false. &#xA;                              This option can be set to 'null' to clear the default value. Possible values: {true, &#xA;                              false} &#xA;&#xA;METRIC_ACCUMULATION_LEVEL=MetricAccumulationLevel&#xA;LEVEL=MetricAccumulationLevel The level(s) at which to accumulate metrics.  Default value: [ALL_READS]. This option can &#xA;                              be set to 'null' to clear the default value. Possible values: {ALL_READS, SAMPLE, &#xA;                              LIBRARY, READ_GROUP} This option may be specified 0 or more times. This option can be set &#xA;                              to 'null' to clear the default list. &#xA;&#xA;INPUT=File&#xA;I=File                        Input SAM or BAM file.  Required. &#xA;&#xA;OUTPUT=File&#xA;O=File                        File to write the output to.  Required. &#xA;&#xA;ASSUME_SORTED=Boolean&#xA;AS=Boolean                    If true (default), then the sort order in the header file will be ignored.  Default &#xA;                              value: true. This option can be set to 'null' to clear the default value. Possible &#xA;                              values: {true, false} &#xA;&#xA;STOP_AFTER=Long               Stop after processing N reads, mainly for debugging.  Default value: 0. This option can &#xA;                              be set to 'null' to clear the default value. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Given this, and wanting to calculate GCBias the using Picard (not other software), what's the best option?&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2018-01-23T09:59:33.023" Title="Picard CollectGcBiasMetrics ignoring certain chromosomes/sequences" Tags="|sam|picard|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3369" PostTypeId="2" ParentId="3368" CreationDate="2018-01-23T09:59:33.023" Score="1" Body="&lt;p&gt;The proper solution is to use a different tool (some of this you could do with &lt;code&gt;computeGCBias&lt;/code&gt; in deepTools). But since you don't want to do that, you'll have to manually remove unwanted chromosomes from the BAM file(s):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat foo.awk&#xA;BEGIN{split(excludes, excludeList, &quot; &quot;)}&#xA;{exclude=0&#xA;if($1 == &quot;@SQ&quot;) {&#xA;    for(ex in excludeList) {&#xA;        if(ex == substr($2, 4)) {&#xA;            exclude=1&#xA;            break&#xA;        }&#xA;    }&#xA;} else { \&#xA;    for(ex in excludeList) {&#xA;        if(ex == $3) {&#xA;            exclude=1&#xA;            break&#xA;        }&#xA;    }&#xA;}&#xA;if(!exclude) print}&#xA;&#xA;$ excludes=&quot;chr1 chr2&quot;&#xA;$ samtools view -h alignments.bam | awk -v excludes=&quot;$excludes&quot; -f foo.awk | samtools -bo alignments.filtered.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The awk script is stripping unwanted chromosomes from the header and the body. You can then run the result through picard.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-23T09:59:33.023" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3370" PostTypeId="1" AcceptedAnswerId="3371" CreationDate="2018-01-23T10:53:22.950" Score="1" ViewCount="413" Body="&lt;p&gt;I have downloaded harmonised the TCGA-HNSC dataset and extracted it. It has 546 samples in total, so what I see in it are FPKM normalised values. The first column remains the same with 60000 rows and 2 columns that is first column ENSEMBL ID and second column FPKM value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now the issue is I want to join all the 546 txt file into one where the first column remains the same and rest 546 column would be joined. I'm using this command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;paste -d &quot; &quot; *txt &amp;gt; merged_file.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is joining the all the data sets as ENSEMBL ID and FPKM value..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I want 1st column as the ENSEMBLE ID and rest the sample FPKM any suggestion or help would be appreciated &lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="77" LastEditDate="2018-01-23T11:05:14.810" LastActivityDate="2018-01-23T11:11:15.057" Title="TCGA data sets joining multiple files" Tags="|rna-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3371" PostTypeId="2" ParentId="3370" CreationDate="2018-01-23T11:11:15.057" Score="2" Body="&lt;p&gt;In R:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;flist = list.files(&quot;*.txt&quot;, path=&quot;.&quot;)&#xA;fl = lapply(flist, function(x) read.delim(x, header=F))&#xA;m = sapply(fl, function(x) x$V2)&#xA;row.names(m) = fl[[1]]$V1&#xA;colnames(m) = flist&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That will give you an appropriately formatted matrix named &lt;code&gt;m&lt;/code&gt; in R, which is probably where you want to continue processing things anyway.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-23T11:11:15.057" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3372" PostTypeId="2" ParentId="3355" CreationDate="2018-01-23T12:49:31.763" Score="2" Body="&lt;p&gt;Disclaimer, I am not an expert on admixture, answer is provided without any warranty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It depends what you want to figure out. Why do you want to categorize the ancestry of the individual? If you analyze this origin of population X you should make a reference without population X and analyze all the individuals from the population with all the surrounding populations in the reference. However if you are interested in origin of a single individual, than the population X got to be in reference as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your result will be that 90% of SNPs of your individual correspond to your population X, then it means that's a normal X fellow. Why it should be a problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- edit --&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just found &lt;a href=&quot;https://www.biorxiv.org/content/early/2017/11/27/066431&quot; rel=&quot;nofollow noreferrer&quot;&gt;this preprint&lt;/a&gt; that highlights potential pitfalls in interpretation of STRUCTURE/ADMIXTURE results.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2018-01-24T10:00:52.347" LastActivityDate="2018-01-24T10:00:52.347" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3373" PostTypeId="1" AcceptedAnswerId="3374" CreationDate="2018-01-23T13:26:30.257" Score="1" ViewCount="431" Body="&lt;p&gt;I have a basic ggplot2 R question. I have a dataframe in which data looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AA 4&#xA;AA 6&#xA;BB 6&#xA;AB 5&#xA;BA 4&#xA;AA 3&#xA;NN 2&#xA;AN 6&#xA;NN 5&#xA;AN 4&#xA;NA 3&#xA;BB 6&#xA;BN 5&#xA;NB 1&#xA;BN 7&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The file is of multiple lines. I want to make a histogram using gplot2 that will plot the 9 kinds of variables AA,BB,AB,BA,AN,BN,NN,NB,NA on the x-axis and the sum of it's corresponding values in the next column on the y-axis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for example, here the y-values should be &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AA= 13&#xA;BB= 12&#xA;AB= 5&#xA;BA= 4&#xA;NN= 7&#xA;AN= 10&#xA;NA= 3&#xA;BN= 12&#xA;NB= 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2094" LastEditorUserId="131" LastEditDate="2018-01-23T16:04:52.393" LastActivityDate="2018-01-23T16:04:52.393" Title="Histogram of different classes of variables in one column" Tags="|r|visualization|ggplot2|" AnswerCount="1" CommentCount="3" ClosedDate="2018-01-24T15:33:50.973" ContentLicense="CC BY-SA 3.0" />
  <row Id="3374" PostTypeId="2" ParentId="3373" CreationDate="2018-01-23T13:31:08.680" Score="2" Body="&lt;p&gt;This isn't a bioinformatics question, but it's quicker to answer than to close.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df1 &amp;lt;- read.table(text = &quot;&#xA;AA 4&#xA;AA 6&#xA;BB 6&#xA;AB 5&#xA;BA 4&#xA;AA 3&#xA;NN 2&#xA;AN 6&#xA;NN 5&#xA;AN 4&#xA;NA 3&#xA;BB 6&#xA;BN 5&#xA;NB 1&#xA;BN 7&quot;, na.strings = &quot;&quot;)&#xA;&#xA;library(ggplot2)&#xA;&#xA;ggplot(df1, aes(x = V1, y = V2)) +&#xA;  geom_bar(stat = &quot;identity&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2srXF.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2srXF.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="131" LastEditDate="2018-01-23T13:56:01.013" LastActivityDate="2018-01-23T13:56:01.013" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3375" PostTypeId="2" ParentId="3365" CreationDate="2018-01-23T13:57:17.857" Score="18" Body="&lt;p&gt;As suggested, here’s an example showing the relevant lines &lt;a href=&quot;https://github.com/HannahVMeyer/PhenotypeSimulator/blob/6ba850c74d641278f75e30c4750cb2d545522689/DESCRIPTION#L28&quot; rel=&quot;noreferrer&quot;&gt;from a &lt;code&gt;DESCRIPTION&lt;/code&gt; file&lt;/a&gt; from a &lt;a href=&quot;https://github.com/HannahVMeyer/PhenotypeSimulator&quot; rel=&quot;noreferrer&quot;&gt;CRAN/GitHub hosted project&lt;/a&gt; that has Bioconductor dependencies (truncated):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Depends:&#xA;    R (&amp;gt;= 3.3.0)&#xA;biocViews:&#xA;Imports:&#xA;    methods,&#xA;    snpStats,&#xA;    dplyr&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The relevant bit is the empty &lt;code&gt;biocViews:&lt;/code&gt; declaration, which allows the Bioconductor dependency {snpStats} to be automatically installed.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2018-01-23T13:57:17.857" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="3376" PostTypeId="1" CreationDate="2018-01-23T14:15:28.807" Score="1" ViewCount="108" Body="&lt;p&gt;I want to identify the position and amino acid change in a specific region of interest in my sequences, and store that information in a table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to do something like this in R, by maybe using the bioconductor package? I have managed to do a simple alignment of the sequences with AlignSeqs() from the DECIPHER package, but I am unable to pull out the differences in the sequences automatically. I start with FASTA files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to end up with something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Isolate ID    Reference_AA   Sample_AA   Pos&#xA;1             S              T           254&#xA;2             T              D           200&#xA;3             L              A           230&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have a reference sequence which is 74 AA long, and I want to look at differences in the query sequences (which are much longer than the reference) compared to the reference, and list the results in a table. The position column is related to the position in the reference sequence and not in the query sequence. I want the first AA in the reference sequence to start at 68, not 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I find it difficult to add example sequences for this as they tend to be very long, but here is something shorter to work with (not related to the table above):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;reference&#xA;VGRALPDVR&#xA;&#xA;&amp;gt;query1&#xA;KSSYLDYAMSVIVGTALPDVRDGLKPVHRRVLY&#xA;&#xA;&amp;gt;query2&#xA;ELKSSYLDYAMSVIVGRAAPDVRDGLKPV&#xA;&#xA;Expected output:&#xA;&#xA;ID       Reference_AA    Sample_AA   Pos&#xA;query1   R               T           70&#xA;query2   L               A           72&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2239" LastEditorUserId="2239" LastEditDate="2018-01-25T10:17:43.047" LastActivityDate="2018-01-25T10:17:43.047" Title="Identifying amino acid substitutions from local alignments in R" Tags="|r|bioconductor|" AnswerCount="0" CommentCount="4" ClosedDate="2018-01-26T13:19:37.450" ContentLicense="CC BY-SA 3.0" />
  <row Id="3377" PostTypeId="1" CreationDate="2018-01-23T15:32:59.810" Score="3" ViewCount="341" Body="&lt;p&gt;I'd like to build the &lt;a href=&quot;https://embnet.vital-it.ch/CoursEMBnet/PHYL03/Slides/Protein_II_Phylogeny_2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;evolutionary history of a protein&lt;/a&gt;, given its sequence. Namely, given a FASTA entry how can I build an evolutionary tree? Here is the &lt;a href=&quot;https://www.rcsb.org/structure/5WXY&quot; rel=&quot;nofollow noreferrer&quot;&gt;5wxy&lt;/a&gt; protein as an example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;5WXY:A|PDBID|CHAIN|SEQUENCE&#xA;MGHHHHHHMMKTKLPILGVLGGMGPVVTAEFLKSIYEYNPFIDKEQESPNVIVFSFPSAPDRTGSIDSGKEREFIDFIQV&#xA;NLEHLNKLADCIVIGSCTAHYALPQIPENLKDKLISLIKIADQELQEYNEPTLLLASTGTYQKKLFQEGCTTADLIISLS&#xA;ESDQKLIHEMIYKVLKRGHDPLSILRDIEALLEKYNTRSYISGSTEFHLLTKSLKLKGIDSIKAIDPLSTIAQNFSQLII&#xA;KQAQVDLVTDCHQPSNPKSP&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Do I need to do some queries to &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2128/getting-protein-fasta-sequence-based-on-keyword-with-python&quot;&gt;Entrez&lt;/a&gt; in order to build the evolution tree?&lt;/p&gt;&#xA;&lt;p&gt;I saw &lt;a href=&quot;http://nature.com/articles/srep06504&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper, but I am not sure how I can use it with a single mammal's protein primary sequence.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; given a protein name, is it possible to look for its evolutionary history or build evolution tree using other databases beside &lt;a href=&quot;https://www.rcsb.org/search&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB&lt;/a&gt; on top of bioinformatics algorithms.&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="3848" LastEditDate="2023-04-28T20:12:10.913" LastActivityDate="2023-04-28T20:12:10.913" Title="How to find/build the evolutionary history of a protein from its sequence?" Tags="|phylogenetics|sequence-alignment|proteins|phylogeny|pdb|" AnswerCount="2" CommentCount="18" ContentLicense="CC BY-SA 4.0" />
  <row Id="3380" PostTypeId="1" AcceptedAnswerId="3381" CreationDate="2018-01-23T18:41:26.513" Score="8" ViewCount="11406" Body="&lt;p&gt;I have a text file 'qnames.txt' with QNAMEs in the following format:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;EXAMPLE:QNAME1&#xA;EXAMPLE:QNAME2&#xA;EXAMPLE:QNAME3&#xA;EXAMPLE:QNAME4&#xA;EXAMPLE:QNAME5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to subset my BAM &lt;code&gt;file.bam&lt;/code&gt; via all of these QNAMEs into a new SAM.&lt;/p&gt;&#xA;&lt;p&gt;Naturally, I can do this individually, e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view file.bam | grep 'EXAMPLE:QNAME1' &amp;gt; subset.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I'm unsure how to do this for a list of QNAMES:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How do I write a for loop that will do all of these queries, outputting the correct SAM needed?&lt;/p&gt;&#xA;&lt;p&gt;I could write a &lt;code&gt;for&lt;/code&gt; loop that creates &lt;em&gt;n&lt;/em&gt; SAM files, and then &lt;code&gt;cat&lt;/code&gt;s them…&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a way to specifically &lt;code&gt;grep&lt;/code&gt; by QNAME? The above may &lt;code&gt;grep&lt;/code&gt; reads that may not be associated with the correct QNAME?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How do I keep the BAM header?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1770" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2021-10-22T15:06:13.123" Title="How to subset a BAM by a list of QNAMEs?" Tags="|sam|samtools|" AnswerCount="7" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3381" PostTypeId="2" ParentId="3380" CreationDate="2018-01-23T18:49:22.583" Score="4" Body="&lt;p&gt;(1)&#xA;It won't be super fast but you can provide grep with a file of QNAMES.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view file.bam | grep -f 'qnames.txt &amp;gt; subset.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where qnames.txt has &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;EXAMPLE:QNAME1&#xA;EXAMPLE:QNAME2&#xA;EXAMPLE:QNAME3&#xA;EXAMPLE:QNAME4&#xA;EXAMPLE:QNAME5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(2)&#xA;This would be a little more complicated but can you give an example where the grep might be have the correct QNAME?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(3) &#xA;To keep the BAM header I would use &lt;code&gt;samtools -H file.bam &amp;gt; header.txt&lt;/code&gt; to get the header and then cat the header file with the grep'ed file &lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2018-01-23T18:49:22.583" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3383" PostTypeId="2" ParentId="3380" CreationDate="2018-01-23T20:21:35.443" Score="4" Body="&lt;p&gt;Below is a small bit of python code showing one naive but manageable way of doing this (N.B., I expect &lt;code&gt;grep&lt;/code&gt; to be faster, though getting it to output the header will be annoying):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pysam&#xA;qnames = set(...)  # read names go here&#xA;bam = pysam.AlignmentFile(&quot;some input file.bam&quot;)&#xA;obam = pysam.AlignmentFile(&quot;some output file.bam&quot;, &quot;w&quot;, template=bam)&#xA;&#xA;for b in bam.fetch(until_eof=True):&#xA;    if b.query_name in qnames:&#xA;        obam.write(b)&#xA;bam.close()&#xA;obam.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now that'll work, but if you have a lot of names to look through then it'll end up being quite slow (as will &lt;code&gt;grep -f ...&lt;/code&gt;). If you need to select a large list of reads by name, then a more efficient strategy is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Name sort the BAM file (&lt;code&gt;samtools sort -n&lt;/code&gt; or picard), which can be done with multiple threads.&lt;/li&gt;&#xA;&lt;li&gt;Sort the list of read names to match (this will be more complicated than one would naively think, since samtools and picard will name sort differently, so be sure to put some thought into this).&lt;/li&gt;&#xA;&lt;li&gt;Perform the same sort of iteration as above, but only compare to the top read name in your list from step 2 (removing that element from the stack after a match).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-23T20:21:35.443" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3384" PostTypeId="1" AcceptedAnswerId="3400" CreationDate="2018-01-24T00:40:22.840" Score="1" ViewCount="291" Body="&lt;p&gt;I am trying to install blasr on mac osx.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I followed the steps &lt;a href=&quot;https://github.com/PacificBiosciences/pitchfork/wiki&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get the error below after using the 'make' command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;make PREFIX=/tmp/pitchfork blasr&#xA;Cloning into 'pitchfork'...&#xA;remote: Counting objects: 10203, done.&#xA;remote: Compressing objects: 100% (51/51), done.&#xA;remote: Total 10203 (delta 25), reused 51 (delta 18), pack-reused 10128&#xA;Receiving objects: 100% (10203/10203), 1.16 MiB | 207.00 KiB/s, done.&#xA;Resolving deltas: 100% (5413/5413), done.&#xA;/Applications/Xcode.app/Contents/Developer/usr/bin/make -f mk/init.mk init&#xA;/Users/cr517/Documents/pitchfork/bin/checkSystem&#xA;/Users/cr517/Documents/pitchfork/bin/checkCC gcc&#xA;mkdir -p &quot;/Users/cr517/Documents/pitchfork/workspace&quot;&#xA;mkdir -p &quot;/Users/cr517/Documents/pitchfork/distfiles&quot;&#xA;mkdir -p &quot;/Users/cr517/Documents/pitchfork/staging&quot;&#xA;mkdir -p &quot;/tmp/pitchfork/bin&quot;&#xA;mkdir -p &quot;/tmp/pitchfork/etc&quot;&#xA;mkdir -p &quot;/tmp/pitchfork/include&quot;&#xA;mkdir -p &quot;/tmp/pitchfork/lib&quot;&#xA;mkdir -p &quot;/tmp/pitchfork/lib/pkgconfig&quot;&#xA;mkdir -p &quot;/tmp/pitchfork/share&quot;&#xA;mkdir -p &quot;/tmp/pitchfork/var/pkg&quot;&#xA;echo &quot;export DYLD_LIBRARY_PATH=/tmp/pitchfork/lib:\$DYLD_LIBRARY_PATH&quot;|sed -e 's/::*/:/g' &amp;gt; &quot;/tmp/pitchfork/setup-env.sh&quot;&#xA;echo &quot;export PATH=/tmp/pitchfork/bin:\$PATH&quot;|sed -e 's/::*/:/g' &amp;gt;&amp;gt; &quot;/tmp/pitchfork/setup-env.sh&quot;&#xA;echo &quot;OLDPS1=\$PS1&quot; &amp;gt;&amp;gt; &quot;/tmp/pitchfork/setup-env.sh&quot;&#xA;echo &quot;[[ \&quot;\$PS1\&quot; ]] &amp;amp;&amp;amp; PS1=\&quot;pitchfork($(/Users/cr517/Documents/pitchfork/bin/pitchfork short-branch)) \$PS1\&quot; || PS1=\($(/Users/cr517/Documents/pitchfork/bin/pitchfork short-branch)\)&quot; &amp;gt;&amp;gt; &quot;/tmp/pitchfork/setup-env.sh&quot;&#xA;touch mk//../initialized.o&#xA;/Applications/Xcode.app/Contents/Developer/usr/bin/make -C ports/thirdparty/zlib do-install&#xA;make[1]: Nothing to be done for `do-install'.&#xA;/Applications/Xcode.app/Contents/Developer/usr/bin/make -C ports/thirdparty/libbzip2 do-install&#xA;rm -rf /Users/cr517/Documents/pitchfork/staging/libbzip2-1.0.6&#xA;mkdir -p /Users/cr517/Documents/pitchfork/staging/libbzip2-1.0.6/lib/pkgconfig /Users/cr517/Documents/pitchfork/staging/libbzip2-1.0.6/include /Users/cr517/Documents/pitchfork/staging/libbzip2-1.0.6/bin&#xA;cp -aL /usr/bin/bzip2 /Users/cr517/Documents/pitchfork/staging/libbzip2-1.0.6/bin/&#xA;cp -aL /usr/include/bzlib.h /Users/cr517/Documents/pitchfork/staging/libbzip2-1.0.6/include&#xA;cp -aL /usr/lib/libbz2.* /Users/cr517/Documents/pitchfork/staging/libbzip2-1.0.6/lib&#xA;ln -sfn libbz2.dylib.1.0 /Users/cr517/Documents/pitchfork/staging/libbzip2-1.0.6/lib/libbz2.dylib.1&#xA;ln -sfn libbz2.dylib.1   /Users/cr517/Documents/pitchfork/staging/libbzip2-1.0.6/lib/libbz2.dylib&#xA;sed -e &quot;s@^prefix=.*@prefix=/tmp/pitchfork@&quot; bzip2.pc &amp;gt; /Users/cr517/Documents/pitchfork/staging/libbzip2-1.0.6/lib/pkgconfig/bzip2.pc&#xA;cd /Users/cr517/Documents/pitchfork/staging/libbzip2-1.0.6 &amp;amp;&amp;amp; tar cf - * | tar xf - -C /tmp/pitchfork&#xA;find /Users/cr517/Documents/pitchfork/staging/libbzip2-1.0.6 ! -type d|awk -F '/Users/cr517/Documents/pitchfork/staging/libbzip2-1.0.6/' '{print $2}' &amp;gt; /tmp/pitchfork/var/pkg/libbzip2-1.0.6&#xA;/Applications/Xcode.app/Contents/Developer/usr/bin/make -C ports/thirdparty/boost do-install&#xA;/Users/cr517/Documents/pitchfork/bin/pitchfork fetch --url https://prdownloads.sourceforge.net/boost/boost_1_60_0.tar.gz&#xA;fetching https://prdownloads.sourceforge.net/boost/boost_1_60_0.tar.gz&#xA;{'url': 'https://prdownloads.sourceforge.net/boost/boost_1_60_0.tar.gz', 'subcommand': 'fetch'}&#xA;Traceback (most recent call last):&#xA;  File &quot;/Users/cr517/Documents/pitchfork/bin/pitchfork&quot;, line 525, in &amp;lt;module&amp;gt;&#xA;    sys.exit(main())&#xA;  File &quot;/Users/cr517/Documents/pitchfork/bin/pitchfork&quot;, line 518, in main&#xA;    pitchfork(pargs)&#xA;  File &quot;/Users/cr517/Documents/pitchfork/bin/pitchfork&quot;, line 444, in pitchfork&#xA;    raise Exception(&quot;[ERROR] %s&quot; % ''.join(_err))&#xA;Exception: [ERROR] --2018-01-24 00:35:30--  https://prdownloads.sourceforge.net/boost/boost_1_60_0.tar.gz&#xA;Resolving prdownloads.sourceforge.net... 216.34.181.59&#xA;Connecting to prdownloads.sourceforge.net|216.34.181.59|:443... connected.&#xA;ERROR: cannot verify prdownloads.sourceforge.net's certificate, issued by ‘CN=COMODO RSA Domain Validation Secure Server CA,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB’:&#xA;  Unable to locally verify the issuer's authority.&#xA;To connect to prdownloads.sourceforge.net insecurely, use `--no-check-certificate'.&#xA;make[1]: *** [boost_1_60_0.tar.gz] Error 1&#xA;make: *** [boost] Error 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think the important info is here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;raise Exception(&quot;[ERROR] %s&quot; % ''.join(_err))&#xA;    Exception: [ERROR] --2018-01-24 00:35:30--  https://prdownloads.sourceforge.net/boost/boost_1_60_0.tar.gz&#xA;    Resolving prdownloads.sourceforge.net... 216.34.181.59&#xA;    Connecting to prdownloads.sourceforge.net|216.34.181.59|:443... connected.&#xA;    ERROR: cannot verify prdownloads.sourceforge.net's certificate, issued by ‘CN=COMODO RSA Domain Validation Secure Server CA,O=COMODO CA Limited,L=Salford,ST=Greater Manchester,C=GB’:&#xA;      Unable to locally verify the issuer's authority.&#xA;    To connect to prdownloads.sourceforge.net insecurely, use `--no-check-certificate'.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but how can I specify the --no-check-certificate option? That's not a make option, I've tried it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried homebrew but got this message:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Updating Homebrew...&#xA;==&amp;gt; Auto-updated Homebrew!&#xA;Updated 2 taps (caskroom/cask, caskroom/versions).&#xA;Warning: Calling 'depends_on :mysql' is deprecated!&#xA;Use 'depends_on &quot;mysql&quot;' instead.&#xA;/usr/local/Homebrew/Library/Taps/brewsci/homebrew-science/Formula/blat.rb:19:in `&amp;lt;class:Blat&amp;gt;'&#xA;Please report this to the brewsci/science tap!&#xA;&#xA;Warning: Calling 'depends_on :python3' is deprecated!&#xA;Use 'depends_on &quot;python3&quot;' instead.&#xA;/usr/local/Homebrew/Library/Taps/brewsci/homebrew-science/Formula/matplotlib.rb:56:in `&amp;lt;class:Matplotlib&amp;gt;'&#xA;Please report this to the brewsci/science tap!&#xA;&#xA;Warning: Calling &amp;lt;&amp;lt;-EOS.undent is deprecated!&#xA;Use &amp;lt;&amp;lt;~EOS instead.&#xA;/usr/local/Homebrew/Library/Taps/brewsci/homebrew-science/Formula/plink.rb:23:in `block in &amp;lt;class:Plink&amp;gt;'&#xA;Please report this to the brewsci/science tap!&#xA;&#xA;Warning: Calling &amp;lt;&amp;lt;-EOS.undent is deprecated!&#xA;Use &amp;lt;&amp;lt;~EOS instead.&#xA;/usr/local/Homebrew/Library/Taps/brewsci/homebrew-science/Formula/plink.rb:23:in `block in &amp;lt;class:Plink&amp;gt;'&#xA;Please report this to the brewsci/science tap!&#xA;&#xA;Warning: Calling &amp;lt;&amp;lt;-EOS.undent is deprecated!&#xA;Use &amp;lt;&amp;lt;~EOS instead.&#xA;/usr/local/Homebrew/Library/Taps/brewsci/homebrew-science/Formula/plink.rb:23:in `block in &amp;lt;class:Plink&amp;gt;'&#xA;Please report this to the brewsci/science tap!&#xA;&#xA;&#xA;Warning: Calling &amp;lt;&amp;lt;-EOS.undent is deprecated!&#xA;Use &amp;lt;&amp;lt;~EOS instead.&#xA;/usr/local/Homebrew/Library/Taps/brewsci/homebrew-science/Formula/blasr.rb:27:in `block in &amp;lt;class:Blasr&amp;gt;'&#xA;Please report this to the brewsci/science tap!&#xA;&#xA;Warning: Calling &amp;lt;&amp;lt;-EOS.undent is deprecated!&#xA;Use &amp;lt;&amp;lt;~EOS instead.&#xA;/usr/local/Homebrew/Library/Taps/brewsci/homebrew-science/Formula/blasr.rb:27:in `block in &amp;lt;class:Blasr&amp;gt;'&#xA;Please report this to the brewsci/science tap!&#xA;&#xA;Warning: Calling &amp;lt;&amp;lt;-EOS.undent is deprecated!&#xA;Use &amp;lt;&amp;lt;~EOS instead.&#xA;/usr/local/Homebrew/Library/Taps/brewsci/homebrew-science/Formula/blasr.rb:27:in `block in &amp;lt;class:Blasr&amp;gt;'&#xA;Please report this to the brewsci/science tap!&#xA;&#xA;Warning: Calling &amp;lt;&amp;lt;-EOS.undent is deprecated!&#xA;Use &amp;lt;&amp;lt;~EOS instead.&#xA;/usr/local/Homebrew/Library/Taps/brewsci/homebrew-science/Formula/blasr.rb:36:in `block in &amp;lt;class:Blasr&amp;gt;'&#xA;Please report this to the brewsci/science tap!&#xA;&#xA;Warning: Calling &amp;lt;&amp;lt;-EOS.undent is deprecated!&#xA;Use &amp;lt;&amp;lt;~EOS instead.&#xA;/usr/local/Homebrew/Library/Taps/brewsci/homebrew-science/Formula/blasr.rb:36:in `block in &amp;lt;class:Blasr&amp;gt;'&#xA;Please report this to the brewsci/science tap!&#xA;&#xA;Warning: Calling &amp;lt;&amp;lt;-EOS.undent is deprecated!&#xA;Use &amp;lt;&amp;lt;~EOS instead.&#xA;/usr/local/Homebrew/Library/Taps/brewsci/homebrew-science/Formula/blasr.rb:36:in `block in &amp;lt;class:Blasr&amp;gt;'&#xA;Please report this to the brewsci/science tap!&#xA;&#xA;==&amp;gt; Installing blasr from brewsci/science&#xA;Warning: Calling &amp;lt;&amp;lt;-EOS.undent is deprecated!&#xA;Use &amp;lt;&amp;lt;~EOS instead.&#xA;/usr/local/Homebrew/Library/Taps/brewsci/homebrew-science/Formula/blasr.rb:27:in `block in &amp;lt;class:Blasr&amp;gt;'&#xA;Please report this to the brewsci/science tap!&#xA;&#xA;Warning: Calling &amp;lt;&amp;lt;-EOS.undent is deprecated!&#xA;Use &amp;lt;&amp;lt;~EOS instead.&#xA;/usr/local/Homebrew/Library/Taps/brewsci/homebrew-science/Formula/blasr.rb:27:in `block in &amp;lt;class:Blasr&amp;gt;'&#xA;Please report this to the brewsci/science tap!&#xA;&#xA;Warning: Calling &amp;lt;&amp;lt;-EOS.undent is deprecated!&#xA;Use &amp;lt;&amp;lt;~EOS instead.&#xA;/usr/local/Homebrew/Library/Taps/brewsci/homebrew-science/Formula/blasr.rb:27:in `block in &amp;lt;class:Blasr&amp;gt;'&#xA;Please report this to the brewsci/science tap!&#xA;&#xA;Warning: Calling &amp;lt;&amp;lt;-EOS.undent is deprecated!&#xA;Use &amp;lt;&amp;lt;~EOS instead.&#xA;/usr/local/Homebrew/Library/Taps/brewsci/homebrew-science/Formula/blasr.rb:36:in `block in &amp;lt;class:Blasr&amp;gt;'&#xA;Please report this to the brewsci/science tap!&#xA;&#xA;Warning: Calling &amp;lt;&amp;lt;-EOS.undent is deprecated!&#xA;Use &amp;lt;&amp;lt;~EOS instead.&#xA;/usr/local/Homebrew/Library/Taps/brewsci/homebrew-science/Formula/blasr.rb:36:in `block in &amp;lt;class:Blasr&amp;gt;'&#xA;Please report this to the brewsci/science tap!&#xA;&#xA;Warning: Calling &amp;lt;&amp;lt;-EOS.undent is deprecated!&#xA;Use &amp;lt;&amp;lt;~EOS instead.&#xA;/usr/local/Homebrew/Library/Taps/brewsci/homebrew-science/Formula/blasr.rb:36:in `block in &amp;lt;class:Blasr&amp;gt;'&#xA;Please report this to the brewsci/science tap!&#xA;&#xA;==&amp;gt; Downloading https://github.com/PacificBiosciences/blasr/archive/smrtanalysis&#xA;Already downloaded: /Users/cr517/Library/Caches/Homebrew/blasr-2.2.tar.gz&#xA;==&amp;gt; make STATIC= HDF5INCLUDEDIR=/usr/local/opt/hdf5/include HDF5LIBDIR=/usr/loca&#xA;Last 15 lines from /Users/cr517/Library/Logs/Homebrew/blasr/01.make:&#xA;/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/memory:4912:1: note: candidate template ignored: could not match 'shared_ptr&amp;lt;type-parameter-0-0&amp;gt;' against 'const RegionAnnotation'&#xA;operator&amp;lt;(const shared_ptr&amp;lt;_Tp&amp;gt;&amp;amp; __x, const shared_ptr&amp;lt;_Up&amp;gt;&amp;amp; __y) _NOEXCEPT&#xA;^&#xA;/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/memory:4977:1: note: candidate template ignored: could not match 'shared_ptr&amp;lt;type-parameter-0-0&amp;gt;' against 'const RegionAnnotation'&#xA;operator&amp;lt;(const shared_ptr&amp;lt;_Tp&amp;gt;&amp;amp; __x, nullptr_t) _NOEXCEPT&#xA;^&#xA;/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/include/c++/v1/memory:4985:1: note: candidate template ignored: could not match 'shared_ptr&amp;lt;type-parameter-0-0&amp;gt;' against 'const int'&#xA;operator&amp;lt;(nullptr_t, const shared_ptr&amp;lt;_Tp&amp;gt;&amp;amp; __x) _NOEXCEPT&#xA;^&#xA;1 error generated.&#xA;make[1]: *** [bin/KBandMatcher.o] Error 1&#xA;make: *** [alignment-build] Error 2&#xA;make: *** [sequtils-build] Error 2&#xA;clang++  -O3 -Wno-div-by-zero -I ..//common -I /usr/local/opt/hdf5/include bin/PrintPBBWT.o -o bin/bwtpbPrint -L/usr/local/opt/hdf5/lib -lz -lpthread&#xA;clang++  -O3 -Wno-div-by-zero -I ..//common -I /usr/local/opt/hdf5/include bin/BWTQuery.o -o bin/bwtquery -L/usr/local/opt/hdf5/lib -lz -lpthread&#xA;&#xA;If reporting this issue please do so at (not Homebrew/brew or Homebrew/core):&#xA;https://github.com/brewsci/homebrew-science/issues&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I still need to report the issue at the provided address.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried Anaconda 'conda install -c bioconda blasr ':&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PackageNotFoundError: Packages missing in current channels:&#xA;&#xA;  - blasr&#xA;&#xA;We have searched for the packages in the following channels:&#xA;&#xA;  - https://conda.anaconda.org/bioconda/osx-64&#xA;  - https://conda.anaconda.org/bioconda/noarch&#xA;  - https://conda.anaconda.org/conda-forge/osx-64&#xA;  - https://conda.anaconda.org/conda-forge/noarch&#xA;  - https://conda.anaconda.org/r/osx-64&#xA;  - https://conda.anaconda.org/r/noarch&#xA;  - https://repo.continuum.io/pkgs/main/osx-64&#xA;  - https://repo.continuum.io/pkgs/main/noarch&#xA;  - https://repo.continuum.io/pkgs/free/osx-64&#xA;  - https://repo.continuum.io/pkgs/free/noarch&#xA;  - https://repo.continuum.io/pkgs/r/osx-64&#xA;  - https://repo.continuum.io/pkgs/r/noarch&#xA;  - https://repo.continuum.io/pkgs/pro/osx-64&#xA;  - https://repo.continuum.io/pkgs/pro/noarch&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-01-25T15:41:29.643" LastActivityDate="2018-01-25T16:27:52.093" Title="How to install blasr on Mac OS X?" Tags="|errors|pacbio|software-installation|blasr|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3386" PostTypeId="1" AcceptedAnswerId="3387" CreationDate="2018-01-24T10:23:54.207" Score="2" ViewCount="302" Body="&lt;p&gt;I have a large aligned amino acid sequences file (fasta file) from next generation sequencing data. &#xA;The unique sequences have been aligned but these all have certain read counts (frequencies associated with them).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to generate a weblogo image that has the true probability of amino acids at each position. &#xA;Is there a way to input frequencies/read counts with the aligned files of sequences using weblogo?&#xA;For example, in the title of the fasta file or as a csv/tsv file?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So that when generating the weblogo image it takes into account the abundances of each sequence?&lt;/p&gt;&#xA;" OwnerUserId="2242" LastEditorUserId="57" LastEditDate="2018-01-25T12:40:49.913" LastActivityDate="2018-01-25T12:40:49.913" Title="How to generate a weblogo image that takes into account sequence frequencies?" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3387" PostTypeId="2" ParentId="3386" CreationDate="2018-01-24T11:43:41.427" Score="3" Body="&lt;p&gt;Don't use the unique sequences. The whole point of sequence logos is that the height of the letter represents its frequency at a given position. You remove this information when you make your sequences unique. So, while it makes sense to use unique sequences when aligning, you shouldn't when building your logo.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generate a simple mapping file (&lt;code&gt;counts&lt;/code&gt;) with the number of times each sequence was found:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seq1 3&#xA;seq2 10&#xA;seq3 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, given an aligned protein fasta file (&lt;code&gt;foo.aln&lt;/code&gt;) like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;seq1&#xA;MEEPQS---DPSVEPP-LSQETFSDLWKLCFLPENNVLSPLPSQAM-DDL&#xA;&amp;gt;seq2&#xA;MEEPQSCFGDPSVEPPPLSQETFKDLWKL--LCENNVLS---SQAM-DDL&#xA;&amp;gt;seq3&#xA;MEEPCS---DPSVEPPQLSQETFSDLRKL--LPENNFLSPLPSQAMCDDL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can simply repeat the sequences as needed (see my answer &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2651/298&quot;&gt;here&lt;/a&gt; for the &lt;code&gt;FastaToTbl&lt;/code&gt; and &lt;code&gt;TblToFasta&lt;/code&gt; commands):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk '&#xA;        (NR==FNR){&#xA;            count[$1]=$2; &#xA;            next&#xA;        } &#xA;        {&#xA;            for(k=1;k&amp;lt;=count[$1];k++){&#xA;                print $1&quot;.&quot;k&quot;\t&quot;$2&#xA;            }&#xA;        }' counts &amp;lt;(FastaToTbl foo.aln) | TblToFasta&#xA;&amp;gt;seq1.1 &#xA;MEEPQS---DPSVEPP-LSQETFSDLWKLCFLPENNVLSPLPSQAM-DDL&#xA;&amp;gt;seq1.2 &#xA;MEEPQS---DPSVEPP-LSQETFSDLWKLCFLPENNVLSPLPSQAM-DDL&#xA;&amp;gt;seq1.3 &#xA;MEEPQS---DPSVEPP-LSQETFSDLWKLCFLPENNVLSPLPSQAM-DDL&#xA;&amp;gt;seq2.1 &#xA;MEEPQSCFGDPSVEPPPLSQETFKDLWKL--LCENNVLS---SQAM-DDL&#xA;&amp;gt;seq2.2 &#xA;MEEPQSCFGDPSVEPPPLSQETFKDLWKL--LCENNVLS---SQAM-DDL&#xA;&amp;gt;seq2.3 &#xA;MEEPQSCFGDPSVEPPPLSQETFKDLWKL--LCENNVLS---SQAM-DDL&#xA;&amp;gt;seq2.4 &#xA;MEEPQSCFGDPSVEPPPLSQETFKDLWKL--LCENNVLS---SQAM-DDL&#xA;&amp;gt;seq2.5 &#xA;MEEPQSCFGDPSVEPPPLSQETFKDLWKL--LCENNVLS---SQAM-DDL&#xA;&amp;gt;seq2.6 &#xA;MEEPQSCFGDPSVEPPPLSQETFKDLWKL--LCENNVLS---SQAM-DDL&#xA;&amp;gt;seq2.7 &#xA;MEEPQSCFGDPSVEPPPLSQETFKDLWKL--LCENNVLS---SQAM-DDL&#xA;&amp;gt;seq2.8 &#xA;MEEPQSCFGDPSVEPPPLSQETFKDLWKL--LCENNVLS---SQAM-DDL&#xA;&amp;gt;seq2.9 &#xA;MEEPQSCFGDPSVEPPPLSQETFKDLWKL--LCENNVLS---SQAM-DDL&#xA;&amp;gt;seq2.10 &#xA;MEEPQSCFGDPSVEPPPLSQETFKDLWKL--LCENNVLS---SQAM-DDL&#xA;&amp;gt;seq3.1 &#xA;MEEPCS---DPSVEPPQLSQETFSDLRKL--LPENNFLSPLPSQAMCDDL&#xA;&amp;gt;seq3.2 &#xA;MEEPCS---DPSVEPPQLSQETFSDLRKL--LPENNFLSPLPSQAMCDDL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that the above assumes your shell supports the &lt;code&gt;&amp;lt;(command)&lt;/code&gt; syntax for &lt;a href=&quot;http://www.tldp.org/LDP/abs/html/process-sub.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;process substitution&lt;/a&gt;. Bash, the default shell on most Linux and macOSX installations does, but if you are using a shell that doesn't, you might need to run &lt;code&gt;FastaToTbl foo.aln &amp;gt; foo.tbl&lt;/code&gt; first, and then use &lt;code&gt;foo.tbl&lt;/code&gt; as input.  &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-01-24T11:43:41.427" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3388" PostTypeId="2" ParentId="3384" CreationDate="2018-01-24T13:34:56.503" Score="1" Body="&lt;p&gt;is there reason why you want to compile it by your own? You can install blasr though &lt;a href=&quot;https://brew.sh/&quot; rel=&quot;nofollow noreferrer&quot;&gt;homebrew&lt;/a&gt; and &lt;a href=&quot;https://github.com/brewsci/homebrew-science&quot; rel=&quot;nofollow noreferrer&quot;&gt;brewsci/science&lt;/a&gt; tap :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;brew tap brewsci/science&#xA;brew install blasr&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="57" LastEditorUserId="48" LastEditDate="2018-01-24T13:49:13.163" LastActivityDate="2018-01-24T13:49:13.163" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3389" PostTypeId="1" CreationDate="2018-01-24T15:09:55.570" Score="3" ViewCount="791" Body="&lt;p&gt;This is a brief question regarding the above. I have previously used bowtie to map reads from paired-end ChIP-Seq sequencing, then used the positions for peak-calling. However, I'm trying to do similar with a dataset which may consist of repeats (I have previously aligned the dataset using bowtie, and a lot of reads aligned to repeats). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For peak-calling purposes, however, I need to know reasonably confidently where precisely those repeats are located, so that MACS2 treats the coverage pileup sensibly. So my question is this: what's the best way of mapping these (short) paired-end reads from a ChIP-Seq experiment so I can do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I want to be able to run a peak-caller after aligning the data, so I want a good approximation to the coverage at repeating elements after mapping for instance - I don't want to throw away all reads which multimap, as may mean that some reads could lose their mates, and so I won't be able to calculate coverage or call peaks using the data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want an aligner to keep both ends of the paired-end pair within about 180bp of each other (as that is the expected length of the DNA fragment that is sequenced). The immediate step after aligning the pairs is to flatten the pair into the DNA fragment that has been sequenced. With multimapped reads this step could be done by seeing which of the pair is uniquely mapped and choosing the mate which maps closest to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried bowtie (and bowtie2) for this, which is where I understood that the dataset maps to repeats.&lt;/p&gt;&#xA;" OwnerUserId="2244" LastEditorUserId="2244" LastEditDate="2018-01-24T15:47:44.330" LastActivityDate="2018-01-30T11:08:25.810" Title="Aligning ChIP-Seq reads to repeats for downstream peak analysis" Tags="|sequence-alignment|chip-seq|repeat-elements|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3390" PostTypeId="2" ParentId="3380" CreationDate="2018-01-24T22:05:01.120" Score="9" Body="&lt;p&gt;use picard FilterSamReads &lt;a href=&quot;http://broadinstitute.github.io/picard/command-line-overview.html#FilterSamReads&quot; rel=&quot;noreferrer&quot;&gt;http://broadinstitute.github.io/picard/command-line-overview.html#FilterSamReads&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;READ_LIST_FILE (File) Read List File containing reads that will be included or excluded from the OUTPUT SAM or BAM file. Default value: null. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="71" LastActivityDate="2018-01-24T22:05:01.120" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3393" PostTypeId="2" ParentId="3384" CreationDate="2018-01-25T11:34:43.123" Score="2" Body="&lt;p&gt;I've had many issues with installing blasr on my own, but it's a breeze using (bio)conda, see &lt;a href=&quot;https://anaconda.org/bioconda/blasr&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://anaconda.org/bioconda/blasr&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c bioconda blasr &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="681" LastActivityDate="2018-01-25T11:34:43.123" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3394" PostTypeId="2" ParentId="901" CreationDate="2018-01-25T12:41:34.407" Score="1" Body="&lt;p&gt;Since nobody has tackled you second question, I'll give it a go:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The complementary strand to a genome sequence is the  5'-3' &lt;em&gt;reverse&lt;/em&gt; complement, not the 3'-5' complement. This sequence is not captured in standard reference genomes, but the information is. Given a sequence, it's trivial to calculate it's reverse complement, and any tool that's designed to work with sequence data takes this feature into account.&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-01-25T12:41:34.407" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3395" PostTypeId="1" CreationDate="2018-01-25T13:23:04.917" Score="3" ViewCount="261" Body="&lt;p&gt;I am trying to identify proteins that carry a consensus target sequence for a kinase. Usually if I am working with, for e.g. transcription factors binding to DNA I would use position weight matrices (PWMs) from one of a number of databases of PWMs to search the sequences with something like FIMO.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't know of any databases that contain protein motif PWMs. Specially I am looking for one for yeast Cdc28, but it seems like it would be a generally useful resource that I can't believe doesn't exist. For example, while the MEME suite was originally developed to identify protein motifs, and ouputs PWMs, all of its databases refer to DNA or RNA. &lt;/p&gt;&#xA;" OwnerUserId="235" LastEditorUserId="-1" LastEditDate="2018-01-25T21:13:29.303" LastActivityDate="2018-01-26T14:27:45.727" Title="Database of position weight matrices for protein motifs?" Tags="|public-databases|motifs|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3396" PostTypeId="2" ParentId="3377" CreationDate="2018-01-25T13:46:14.123" Score="3" Body="&lt;p&gt;The general procedure is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Find the sequence for the same (or as similar as you can find (see mentions of blast in the comments above)) gene in other species. You can use uniprot or any other large sequence database for this.&lt;/li&gt;&#xA;&lt;li&gt;You now have a large set of sequences related to yours, which you'll need to compare. Search the methods section of your favorite paper to see how they reconstructed likely evolutionary history from this, there are a number of packages out there.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-25T13:46:14.123" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3397" PostTypeId="1" AcceptedAnswerId="3398" CreationDate="2018-01-25T14:59:41.390" Score="1" ViewCount="2658" Body="&lt;p&gt;I have the GATK FASTA reference for hg19. I would like to subset this based on the first chromosome. Normally what I do is open the FASTA as a text file, and then delete all other chromosomes except chromosome 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's awful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Picard tools has an &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/tooldocs/3.8-0/org_broadinstitute_gatk_tools_walkers_fasta_FastaReferenceMaker.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;option&lt;/a&gt;. But this doesn't appear to subset only by chromosome. How could this be done efficiently? &lt;/p&gt;&#xA;" OwnerUserId="1770" LastEditorUserId="48" LastEditDate="2018-01-25T15:30:28.450" LastActivityDate="2018-01-25T15:49:52.400" Title="How can I subset a reference based on only the first chromosome?" Tags="|fasta|reference-genome|picard|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3398" PostTypeId="2" ParentId="3397" CreationDate="2018-01-25T15:24:06.563" Score="3" Body="&lt;p&gt;It sounds like you want &lt;code&gt;samtools faidx foo.fa&lt;/code&gt; followed by &lt;code&gt;samtools faidx foo.fa chr1 &amp;gt; your_subset_file.fa&lt;/code&gt; (or whatever the first chromosome is). The output file is then a regular fasta file subset as you like. You can get any chromosome you want with that. In fact, you can also do regions (e.g., &lt;code&gt;chr1:100-1000&lt;/code&gt;, though note that the sequence name in the resulting fasta output will be &lt;code&gt;chr1:100-1000&lt;/code&gt;, which may break some downstream tools).&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-01-25T15:49:52.400" LastActivityDate="2018-01-25T15:49:52.400" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3399" PostTypeId="1" CreationDate="2018-01-25T16:12:02.067" Score="1" ViewCount="121" Body="&lt;p&gt;I am using the &lt;a href=&quot;https://www-s.ks.uiuc.edu/Research/vmd/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VMD software&lt;/a&gt;, and use &quot;Extension - Analysis - Timeline - Calculate - Calc. Sec. Structure&quot; to visualise the secondary structure of my trajectories.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I need to screenshot to get the graph. Can I output it using a build-in command instead?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I have hundreds of trajectories to analyse. Each time, I need &quot;File - Write data file&quot; to output the raw STRIDE code. Can I use some script to enable batch output?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1806" LastEditorUserId="298" LastEditDate="2018-01-26T11:40:10.137" LastActivityDate="2018-01-26T11:40:10.137" Title="Can I batch output the graph of Secondary Structure in VMD Timeline analysis?" Tags="|protein-structure|visualization|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3400" PostTypeId="2" ParentId="3384" CreationDate="2018-01-25T16:27:52.093" Score="3" Body="&lt;p&gt;The path of least resistance is probably to run the software inside a container. If you install &lt;a href=&quot;https://store.docker.com/editions/community/docker-ce-desktop-mac&quot; rel=&quot;nofollow noreferrer&quot;&gt;docker&lt;/a&gt;, you can just do the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;git clone https://github.com/muccg/docker-blasr.git&#xA;docker-blasr/bin/blasr&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(this worked for me on OSX 10.11.6)&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-01-25T16:27:52.093" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3401" PostTypeId="1" AcceptedAnswerId="3405" CreationDate="2018-01-25T18:30:26.093" Score="7" ViewCount="15554" Body="&lt;p&gt;I would like to subset a VCF which only has chromosome 2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with using various &lt;code&gt;grep&lt;/code&gt; commands, e.g. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -w '^#\|^2' my.vcf &amp;gt; my_new.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or if there's a 'chr' prefix&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -w '^#\|^chr2' my.vcf &amp;gt; my_new.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is that this will remove the header. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been trying to use &lt;code&gt;vcftools&lt;/code&gt; with the following command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;vcftools --vcf input.vcf  --chr 2 --out subset&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which does not output a VCF &lt;code&gt;subset.vcf&lt;/code&gt; as expected, but rather &lt;code&gt;subset.log&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;VCFtools - 0.1.15&#xA;(C) Adam Auton and Anthony Marcketta 2009&#xA;&#xA;Parameters as interpreted:&#xA;    --vcf input.vcf&#xA;    --chr 2&#xA;    --out subset&#xA;&#xA;After filtering, kept 2 out of 2 Individuals&#xA;After filtering, kept 80 out of a possible 720 Sites&#xA;Run Time = 0.00 seconds&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Given that the run time is &lt;code&gt;Run Time = 0.00 seconds&lt;/code&gt;, I guess there's an error here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I subset a VCF and keep the header? Is there an option with bcftools instead maybe?&lt;/p&gt;&#xA;" OwnerUserId="1770" LastActivityDate="2020-04-23T15:56:08.440" Title="How to subset a VCF by chromosome and keep the header?" Tags="|vcf|vcftools|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3402" PostTypeId="1" AcceptedAnswerId="3404" CreationDate="2018-01-25T19:08:58.473" Score="2" ViewCount="108" Body="&lt;p&gt;I have followed the steps &lt;a href=&quot;http://ratt.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; to install RATT, i.e.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;svn co &quot;https://svn.code.sf.net/p/ratt/code/&quot; ratt-code&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, when I check my installation by running '$RATT_HOME/start.ratt.sh', it says:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zsh: no such file or directory: /start.ratt.sh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which makes sense once I check that the RATT_HOME variable is empty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I downloaded their tuberculosis example, went into that directory and ran: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;~/path/to/my/ratt/ratt-code/start.ratt.sh embl F11.fasta F11 SimStrain&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and got the following output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Please set the RATT_HOME variable.&#xA;At Sanger for bash it is RATT_HOME=/nfs/users/nfs_t/tdo/Bin/ratt; export RATT_HOME&#xA;At Sanger for tcsh setenv RATT_HOME /nfs/users/nfs_t/tdo/Bin/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What should I set my RATT_HOME variable to? The message only says what it should be set to at the Sanger Institute.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-01-25T19:26:10.780" Title="Problem installing RATT" Tags="|software-installation|sequence-annotation|functional-annotation|ratt|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3403" PostTypeId="2" ParentId="3401" CreationDate="2018-01-25T19:13:22.367" Score="4" Body="&lt;p&gt;You can use &lt;code&gt;grep&lt;/code&gt; for this. The problem is the &lt;code&gt;-w&lt;/code&gt; flag which means &quot;match entire words&quot;, but VCF header lines start with &lt;code&gt;##&lt;/code&gt;, not &lt;code&gt;#&lt;/code&gt;. Here are some alternative &lt;code&gt;grep&lt;/code&gt; commands:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -E '^(#|2[[:space:]])' my.vcf &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, if you have GNU &lt;code&gt;grep&lt;/code&gt; (default on Linux):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -P '^(#|2\t)' my.vcf &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To match either &lt;code&gt;2&lt;/code&gt; or &lt;code&gt;chr2&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -E '^(#|(chr2|2)[[:space:]])' my.vcf &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -P '^(#|(chr){0,1}2\t)' my.vcf &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you can use &lt;code&gt;awk&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '/^#/ || $1==&quot;2&quot; || $2==&quot;chr2&quot;' my.vcf &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, if you want to use &lt;code&gt;vcftools&lt;/code&gt;, you need to also use the &lt;code&gt;--recode&lt;/code&gt; option:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;--recode&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;--recode-bcf&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;These  options  are used to generate a new file in either VCF or BCF from the input VCF or BCF file after applying   the&#xA;  filtering options specified by the user. The output file has the&#xA;  suffix  &quot;.recode.vcf&quot;  or  &quot;.recode.bcf&quot;.  By   default,  the  INFO&#xA;  fields are removed from the output file, as the INFO values may be&#xA;  invalidated by the recoding&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(e.g. the total depth may need to be recalculated if individuals are removed). This behavior may be  overriden  by&#xA;  the following options. By default, BCF files are written out as BGZF compressed files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But that will remove the INFO fields as explained above, so just use normal text-parsing tools like the ones I suggested.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-01-25T19:13:22.367" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3404" PostTypeId="2" ParentId="3402" CreationDate="2018-01-25T19:13:27.113" Score="3" Body="&lt;p&gt;Usually the home of a program is where the executable is. In order to make the system aware you need to include it as a variable when the terminal is run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you need to add it to the file at &lt;code&gt;~/.bashrc&lt;/code&gt; of your computer (if you are using bash as shell [you can check it with &lt;code&gt;echo $SHELL&lt;/code&gt;]). You need to add :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;export RATT_HOME=path/where/RATT/is/installed/or/downloaded&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As terdon says in a comment below:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Note that echo &lt;code&gt;$SHELL&lt;/code&gt; just prints the default login shell of the&#xA;  user, and not necessarily the shell that is currently running.  Also,&#xA;  &lt;code&gt;~/.bashrc&lt;/code&gt; is not read by macOS bash which starts login shells by&#xA;  default. In any case, this sort of global variable should be in&#xA;  &lt;code&gt;~/.profile&lt;/code&gt;, not &lt;code&gt;.bashrc&lt;/code&gt;. For more of this sort of detail see&#xA;  &lt;a href=&quot;https://askubuntu.com/questions/463462/sequence-of-scripts-sourced-upon-login/463479#463479&quot;&gt;Sequence of scripts sourced upon login&lt;/a&gt; and &lt;a href=&quot;https://unix.stackexchange.com/questions/26047/how-to-correctly-add-a-path-to-path/26059#26059&quot;&gt;How to correctly add&#xA;  a path to PATH&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2018-01-25T19:26:10.780" LastActivityDate="2018-01-25T19:26:10.780" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3405" PostTypeId="2" ParentId="3401" CreationDate="2018-01-25T20:23:28.003" Score="11" Body="&lt;p&gt;This seems to work for me with &lt;code&gt;bcftools filter&lt;/code&gt; and the &lt;code&gt;-r&lt;/code&gt; or &lt;code&gt;-R&lt;/code&gt; argument.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;   -r, --regions chr|chr:pos|chr:from-to|chr:from-[,...]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;Comma-separated list of regions, see also &lt;strong&gt;-R&lt;/strong&gt;, &lt;strong&gt;--regions-file&lt;/strong&gt;. Note that &lt;strong&gt;-r&lt;/strong&gt; cannot be used in combination with &lt;strong&gt;-R&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   -R, --regions-file FILE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;Regions can be specified either on command line or in a VCF, BED, or tab-delimited file (the default). The columns of the&#xA;             tab-delimited file are:&#xA;             CHROM, POS, and, optionally, POS_TO, where positions are 1-based and inclusive. Uncompressed files are stored in memory, while&#xA;  bgzip-compressed and&#xA;             tabix-indexed region files are streamed. Note that sequence names must match exactly, &quot;chr20&quot; is not the same as &quot;20&quot;. Also note&#xA;  that chromosome&#xA;             ordering in FILE will be respected, the VCF will be processed in the order in which chromosomes first appear in FILE.&#xA;  However, within chromosomes, the&#xA;             VCF will always be processed in ascending genomic coordinate order no matter what order they appear in FILE. Note that&#xA;  overlapping regions in FILE can&#xA;             result in duplicated out of order positions in the output. This option requires indexed VCF/BCF files. Note that &lt;strong&gt;-R&lt;/strong&gt; cannot be&#xA;  used in combination with &lt;strong&gt;-r&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here's an example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools filter vcf_nocomp_merge_geno98.vcf.gz -r 4 | head -n 38 | colrm 100 1000000&#xA;##fileformat=VCFv4.2&#xA;##FILTER=&amp;lt;ID=PASS,Description=&quot;All filters passed&quot;&amp;gt;&#xA;##fileDate=20171222&#xA;##source=PLINKv1.90&#xA;##contig=&amp;lt;ID=1,length=249167691&amp;gt;&#xA;##contig=&amp;lt;ID=2,length=242695901&amp;gt;&#xA;##contig=&amp;lt;ID=3,length=197800245&amp;gt;&#xA;##contig=&amp;lt;ID=4,length=190915651&amp;gt;&#xA;##contig=&amp;lt;ID=5,length=180666277&amp;gt;&#xA;##contig=&amp;lt;ID=6,length=170877445&amp;gt;&#xA;##contig=&amp;lt;ID=7,length=159086805&amp;gt;&#xA;##contig=&amp;lt;ID=8,length=146293415&amp;gt;&#xA;##contig=&amp;lt;ID=9,length=141018424&amp;gt;&#xA;##contig=&amp;lt;ID=10,length=135434552&amp;gt;&#xA;##contig=&amp;lt;ID=11,length=134938471&amp;gt;&#xA;##contig=&amp;lt;ID=12,length=133763353&amp;gt;&#xA;##contig=&amp;lt;ID=13,length=115045730&amp;gt;&#xA;##contig=&amp;lt;ID=14,length=107285438&amp;gt;&#xA;##contig=&amp;lt;ID=15,length=102369712&amp;gt;&#xA;##contig=&amp;lt;ID=16,length=90141356&amp;gt;&#xA;##contig=&amp;lt;ID=17,length=81006630&amp;gt;&#xA;##contig=&amp;lt;ID=18,length=78014583&amp;gt;&#xA;##contig=&amp;lt;ID=19,length=59071322&amp;gt;&#xA;##contig=&amp;lt;ID=20,length=62906515&amp;gt;&#xA;##contig=&amp;lt;ID=21,length=48077813&amp;gt;&#xA;##contig=&amp;lt;ID=22,length=51156934&amp;gt;&#xA;##contig=&amp;lt;ID=23,length=154847490&amp;gt;&#xA;##INFO=&amp;lt;ID=PR,Number=0,Type=Flag,Description=&quot;Provisional reference allele, may not be based on rea&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&quot;Genotype&quot;&amp;gt;&#xA;##bcftools_filterVersion=1.2-187-g1a55e45+htslib-1.2.1-256-ga356746&#xA;##bcftools_filterCommand=filter -r 4 vcf_nocomp_merge_geno98.vcf.gz&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  1_125   3_5555  4_7565  5_9&#xA;4   71566   rs13125929  T   C   .   PASS    PR  GT  0/1 0/1 0/0&#xA;4   87113   rs2006748   T   C   .   PASS    PR  GT  0/1 0/0 0/0&#xA;4   110646  rs11727494  C   T   .   PASS    PR  GT  0/1 0/0 1/1&#xA;4   142550  rs11735742  T   C   .   PASS    PR  GT  0/1 0/0 0/0&#xA;4   200631  rs6826124   A   G   .   PASS    PR  GT  0/1 1/1 0/1&#xA;4   221623  rs7695945   G   T   .   PASS    PR  GT  0/0 0/0 0/0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-01-25T20:23:28.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3407" PostTypeId="1" AcceptedAnswerId="3408" CreationDate="2018-01-26T10:57:42.453" Score="2" ViewCount="38" Body="&lt;p&gt;I want to read out the taxonomy id of the source organism of a pdb entry using the xml-file of the entry that is provided under &lt;code&gt;http://files.rcsb.org/view/{pdb_accession}-noatom.xml&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have so far found the taxonomy id in the tag &lt;strong&gt;PDBx:entity_src_genCategory&lt;/strong&gt; (example entry &lt;a href=&quot;http://files.rcsb.org/view/1ukv-noatom.xml&quot; rel=&quot;nofollow noreferrer&quot;&gt;1UKV&lt;/a&gt;) but also in the tag &lt;strong&gt;PDBx:entity_src_natCategory&lt;/strong&gt; (example entry &lt;a href=&quot;http://files.rcsb.org/view/2bwi-noatom.xml&quot; rel=&quot;nofollow noreferrer&quot;&gt;2BWI&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;XML for 1UKV with PDBx:entity_src_genCategory&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;PDBx:entity_src_genCategory&amp;gt;&#xA;  &amp;lt;PDBx:entity_src_gen entity_id=&quot;1&quot; pdbx_src_id=&quot;1&quot;&amp;gt;&#xA;     &amp;lt;PDBx:gene_src_common_name&amp;gt;baker&amp;amp;apos;s yeast&amp;lt;/PDBx:gene_src_common_name&amp;gt;&#xA;     &amp;lt;PDBx:gene_src_genus&amp;gt;Saccharomyces&amp;lt;/PDBx:gene_src_genus&amp;gt;&#xA;     &amp;lt;PDBx:host_org_genus&amp;gt;Escherichia&amp;lt;/PDBx:host_org_genus&amp;gt;&#xA;     &amp;lt;PDBx:host_org_species&amp;gt;Escherichia coli&amp;lt;/PDBx:host_org_species&amp;gt;&#xA;     &amp;lt;PDBx:pdbx_alt_source_flag&amp;gt;sample&amp;lt;/PDBx:pdbx_alt_source_flag&amp;gt;&#xA;     &amp;lt;PDBx:pdbx_gene_src_ncbi_taxonomy_id&amp;gt;4932&amp;lt;/PDBx:pdbx_gene_src_ncbi_taxonomy_id&amp;gt;&#xA;     &amp;lt;PDBx:pdbx_gene_src_scientific_name&amp;gt;Saccharomyces cerevisiae&amp;lt;/PDBx:pdbx_gene_src_scientific_name&amp;gt;&#xA;     &amp;lt;PDBx:pdbx_host_org_ncbi_taxonomy_id&amp;gt;469008&amp;lt;/PDBx:pdbx_host_org_ncbi_taxonomy_id&amp;gt;&#xA;     &amp;lt;PDBx:pdbx_host_org_scientific_name&amp;gt;Escherichia coli BL21(DE3)&amp;lt;/PDBx:pdbx_host_org_scientific_name&amp;gt;&#xA;     &amp;lt;PDBx:pdbx_host_org_strain&amp;gt;BL21(DE3)&amp;lt;/PDBx:pdbx_host_org_strain&amp;gt;&#xA;     &amp;lt;PDBx:pdbx_host_org_vector_type&amp;gt;PLASMID&amp;lt;/PDBx:pdbx_host_org_vector_type&amp;gt;&#xA;     &amp;lt;PDBx:plasmid_name&amp;gt;pET19tev&amp;lt;/PDBx:plasmid_name&amp;gt;&#xA;  &amp;lt;/PDBx:entity_src_gen&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;XML for 2BWI with PDBx:entity_src_natCategory&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;PDBx:entity_src_natCategory&amp;gt;&#xA;  &amp;lt;PDBx:entity_src_nat entity_id=&quot;1&quot; pdbx_src_id=&quot;1&quot;&amp;gt;&#xA;     &amp;lt;PDBx:pdbx_alt_source_flag&amp;gt;sample&amp;lt;/PDBx:pdbx_alt_source_flag&amp;gt;&#xA;     &amp;lt;PDBx:pdbx_ncbi_taxonomy_id&amp;gt;223&amp;lt;/PDBx:pdbx_ncbi_taxonomy_id&amp;gt;&#xA;     &amp;lt;PDBx:pdbx_organism_scientific&amp;gt;ACHROMOBACTER CYCLOCLASTES&amp;lt;/PDBx:pdbx_organism_scientific&amp;gt;&#xA;  &amp;lt;/PDBx:entity_src_nat&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference between them? Are there other tags that contain information about the source organism of the entry? Can they occur together in one file? If so, is one of them more valid? &lt;/p&gt;&#xA;" OwnerUserId="2255" LastEditorUserId="57" LastEditDate="2018-01-26T12:08:05.400" LastActivityDate="2018-01-26T12:08:05.400" Title="What is the difference between the entity_src_genCategory tag and the entity_src_natCategory tag in pdb-xmls?" Tags="|pdb|taxonomy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3408" PostTypeId="2" ParentId="3407" CreationDate="2018-01-26T11:31:30.107" Score="1" Body="&lt;p&gt;&lt;code&gt;src_nat&lt;/code&gt; = isolated directly from a natural tissue&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;src_gen&lt;/code&gt; = the source was genetically manipulated&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The files that you use (PDBML/XML) are derived from the primary PDB data format which is PDBx/mmCIF.&#xA;The names in XML seem to correspond to mmCIF names, so you can read documentation on &lt;a href=&quot;http://mmcif.wwpdb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mmcif.wwpdb.org/&lt;/a&gt;, and here are direct links:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;category &lt;a href=&quot;http://mmcif.wwpdb.org/dictionaries/mmcif_pdbx_v50.dic/Categories/entity_src_nat.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;entity_src_nat&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;category &lt;a href=&quot;http://mmcif.wwpdb.org/dictionaries/mmcif_pdbx_v50.dic/Categories/entity_src_gen.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;entity_src_gen&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Can they occur together in one file? I suppose they can for complexes: one &quot;entity&quot; could be natural and another genetically manipulated.&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2018-01-26T11:31:30.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3409" PostTypeId="1" AcceptedAnswerId="4146" CreationDate="2018-01-26T13:51:33.940" Score="6" ViewCount="555" Body="&lt;p&gt;I was teaching an introduction to bioinformatics when the students and I noticed strange Blast behavior that we couldn't explain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the default &lt;code&gt;evalue&lt;/code&gt; parameter, our best hit shows an evalue of &lt;code&gt;3e-06&lt;/code&gt;, while that hit disappears with a evalue threshold of &lt;code&gt;1e-5&lt;/code&gt; (which is bigger!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(after a few tries it seems that the evalue of the top hit starts changing with an evalue threshold of 2)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Input proteins:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wget ftp://ftp.ncbi.nih.gov/refseq/B_taurus/mRNA_Prot/cow.1.protein.faa.gz&#xA;wget ftp://ftp.ncbi.nih.gov/refseq/H_sapiens/mRNA_Prot/human.1.protein.faa.gz &#xA;gzip -d *.gz&#xA;head -199 cow.1.protein.faa &amp;gt; medium_cow.faa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Database building:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;makeblastdb -in human.1.protein.faa -dbtype prot&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Blast command 1 (default):&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blastp -query medium_cow.faa -db human.1.protein.faa -outfmt 7 -out test_blast&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# BLASTP 2.2.31+&#xA;# Query: NP_001180091.1 SH2 domain-containing protein 1B [Bos taurus]&#xA;# Database: human.1.protein.faa&#xA;# Fields: query id, subject id, % identity, alignment length, mismatches, gap opens, q. start, q. end, s. start, s. end, evalue, bit score&#xA;# 13 hits found&#xA;NP_001180091.1  XP_011539315.1  29.41   102     61      3       5       100     144     240     3e-06   42.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Blast command 2 (-evalue 1e-05):&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blastp -query medium_cow.faa -db human.1.protein.faa -outfmt 7 -out test_blast -evalue 1e-05&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# BLASTP 2.2.31+&#xA;# Query: NP_001180091.1 SH2 domain-containing protein 1B [Bos taurus]&#xA;# Database: human.1.protein.faa&#xA;# 0 hits found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Blast command 3 (-evalue 2):&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blastp -query medium_cow.faa -db human.1.protein.faa -outfmt 7 -out test_blast -evalue 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# BLASTP 2.2.31+&#xA;# Query: NP_001180091.1 SH2 domain-containing protein 1B [Bos taurus]&#xA;# Database: human.1.protein.faa&#xA;# Fields: query id, subject id, % identity, alignment length, mismatches, gap opens, q. start, q. end, s. start, s. end, evalue, bit score&#xA;# 6 hits found&#xA;NP_001180091.1  XP_005244948.1  40.62   32      18      1       4       35      394     424     0.076   29.3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I reproduced this behavior with Blast 2.2.31+, 2.7.1+ and the web blast at &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov&quot; rel=&quot;noreferrer&quot;&gt;https://blast.ncbi.nlm.nih.gov&lt;/a&gt; (screenshots attached)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RMVqa.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RMVqa.png&quot; alt=&quot;hits&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GPxBG.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GPxBG.png&quot; alt=&quot;no hits&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anything obvious that my students and I are missing? To my knowledge changing the evalue threshold should no change the values themselves, it should just change the reporting.&lt;/p&gt;&#xA;" OwnerUserId="179" LastEditorUserId="298" LastEditDate="2018-01-26T16:37:10.083" LastActivityDate="2018-04-25T13:36:41.993" Title="Blast hits disappearing after changing -evalue" Tags="|blast|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3410" PostTypeId="2" ParentId="3395" CreationDate="2018-01-26T14:27:45.727" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://pfam.xfam.org/f&quot; rel=&quot;nofollow noreferrer&quot;&gt;PFam&lt;/a&gt; has HMM models for protein domains, this is almost certainly what you're after. You can download all PFam-A models from &lt;a href=&quot;ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.hmm.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/Pfam-A.hmm.gz&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, you're looking for the &quot;Pkinase&quot;, PF00069 family (I found this simply by searching the PFam website for &quot;kinase&quot;), so extract it from the file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zcat Pfam-A.hmm.gz | awk -vRS='//' -vORS='//' '/PF00069/' &amp;gt; PF00069.pfam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, format it for HMMER format (you will need to install &lt;a href=&quot;http://hmmer.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HMMER&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;hmmpress PF00069.hmm&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And now you can use it to search your sequences (fasta or flat file, probably others; see the &lt;a href=&quot;http://www.csb.yale.edu/userguides/seq/hmmer/docs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; for details):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;hmmscan PF00069.hmm prots.pep &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried this on a fasta file with the sequences of &lt;a href=&quot;http://www.uniprot.org/uniprot/P53894.fasta&quot; rel=&quot;nofollow noreferrer&quot;&gt;CBK1_YEAST&lt;/a&gt;, &lt;a href=&quot;http://www.uniprot.org/uniprot/P15790.fasta&quot; rel=&quot;nofollow noreferrer&quot;&gt;CSK21_YEAST&lt;/a&gt; (kinases) and &lt;a href=&quot;http://www.uniprot.org/uniprot/P21333.fasta&quot; rel=&quot;nofollow noreferrer&quot;&gt;FLNA_HUMAN&lt;/a&gt; (non-kinase control) and got:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# hmmscan :: search sequence(s) against a profile database&#xA;# HMMER 3.1b2 (February 2015); http://hmmer.org/&#xA;# Copyright (C) 2015 Howard Hughes Medical Institute.&#xA;# Freely distributed under the GNU General Public License (GPLv3).&#xA;# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -&#xA;# query sequence file:             prots.pep&#xA;# target HMM database:             PF00069.hmm&#xA;# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -&#xA;&#xA;Query:       sp|P53894|CBK1_YEAST  [L=756]&#xA;Description: Serine/threonine-protein kinase CBK1 OS=Saccharomyces cerevisiae (strain ATCC 204508 / S288c) GN=CBK1 PE=1 SV=1&#xA;Scores for complete sequence (score includes all domains):&#xA;   --- full sequence ---   --- best 1 domain ---    -#dom-&#xA;    E-value  score  bias    E-value  score  bias    exp  N  Model    Description&#xA;    ------- ------ -----    ------- ------ -----   ---- --  -------- -----------&#xA;    2.7e-64  203.2   9.0    3.5e-46  143.8   0.2    3.1  3  Pkinase   Protein kinase domain&#xA;&#xA;&#xA;Domain annotation for each model (and alignments):&#xA;&amp;gt;&amp;gt; Pkinase  Protein kinase domain&#xA;   #    score  bias  c-Evalue  i-Evalue hmmfrom  hmm to    alifrom  ali to    envfrom  env to     acc&#xA; ---   ------ ----- --------- --------- ------- -------    ------- -------    ------- -------    ----&#xA;   1 ?   -2.9   3.2      0.19      0.19     198     226 ..     222     243 ..     191     277 .. 0.57&#xA;   2 !  143.8   0.2   3.5e-46   3.5e-46       1     153 [.     352     505 ..     352     522 .. 0.95&#xA;   3 !   63.6   0.0   9.8e-22   9.8e-22     154     264 .]     568     672 ..     551     672 .. 0.83&#xA;&#xA;  Alignments for each domain:&#xA;  == domain 1  score: -2.9 bits;  conditional E-value: 0.19&#xA;                           TTSSCHHHHHHHHHTHHHHHHHCTSHH.. CS&#xA;               Pkinase 198 sgekgkekvekeldqlekilkilgetkek 226&#xA;                           + +++++++++      + ++i ++++++&#xA;  sp|P53894|CBK1_YEAST 222 Q-QQQQQQQQQ------QHMQIQQQQQQQ 243&#xA;                           1.111111111......122222221222 PP&#xA;&#xA;  == domain 2  score: 143.8 bits;  conditional E-value: 3.5e-46&#xA;                           EEEEEEEEEESSEEEEEEEETTTTEEEEEEEEEHHCCCHHHH.HHHHHHHHHHHHHSSTTB--EEEEEEESSEEEEEEE--TTEBHHHH CS&#xA;               Pkinase   1 yekleklGeGsfGkVykaveketgkivAvKkikkekakkkke.kkvlrEikilkklkhpnivrlyevfeekdelylvleyveggeladl 88 &#xA;                           +++++ +G+G+fG+V  + +k+tgki+A+K++ k+++ kk++  +v +E  +l+ +++p +v ly++f++ ++lyl++e+++gg+l+++&#xA;  sp|P53894|CBK1_YEAST 352 FHTVKVIGKGAFGEVRLVQKKDTGKIYAMKTLLKSEMYKKDQlAHVKAERDVLAGSDSPWVVSLYYSFQDAQYLYLIMEFLPGGDLMTM 440&#xA;                           67899********************************999999********************************************** PP&#xA;&#xA;                           HHHHSS--HHHHHHHHHHHHHHHHHHHHTTEE-SS-SGGGEEEETTTEEEE-SGTTSEECCSSCS CS&#xA;               Pkinase  89 lsrkgslseeeakkiakqilegleylHskgiiHrDlKpeNiLidekgelKitDFGlakelesssk 153&#xA;                           l r + ++e+ +++++++ + ++e +H+ g+iHrD+Kp+NiLid +g++K++DFGl++ ++++++&#xA;  sp|P53894|CBK1_YEAST 441 LIRWQLFTEDVTRFYMAECILAIETIHKLGFIHRDIKPDNILIDIRGHIKLSDFGLSTGFHKTHD 505&#xA;                           **999*****************************************************9999876 PP&#xA;&#xA;  == domain 3  score: 63.6 bits;  conditional E-value: 9.8e-22&#xA;                           BCTCCSCGGGS-HHHHTTSTCSHHHHHHHHHHHHHHHHHSS-STTTSSCHHHHHHHHHTHHHHHHHCTSHH......TTS-HHHHHHHH CS&#xA;               Pkinase 154 ltsfvgtreYlAPEvlkeneyskkvDvWslGvilyelltgkppfsgekgkekvekeldqlekilkilgetkeklpeaselseeakdllk 242&#xA;                           + s vgt++Y+APE+  +++y++++D+WslG i+ye l g ppf +e+ +e+++k+++          e+++++p + ++s ea dl++&#xA;  sp|P53894|CBK1_YEAST 568 AYSTVGTPDYIAPEIFLYQGYGQECDWWSLGAIMYECLIGWPPFCSETPQETYRKIMNF---------EQTLQFPDDIHISYEAEDLIR 647&#xA;                           6789*****************************************66665533333333.........557799*************** PP&#xA;&#xA;                           HHT-SSGCCSTT....HHHHHTSGGG CS&#xA;               Pkinase 243 kllkkdpkkRlt....aeellqhpyl 264&#xA;                           +ll+    +Rl+    a+e+++hp++&#xA;  sp|P53894|CBK1_YEAST 648 RLLTHA-DQRLGrhggADEIKSHPFF 672&#xA;                           ***875.678877779********98 PP&#xA;&#xA;&#xA;&#xA;Internal pipeline statistics summary:&#xA;-------------------------------------&#xA;Query sequence(s):                         1  (756 residues searched)&#xA;Target model(s):                           1  (264 nodes)&#xA;Passed MSV filter:                         1  (1); expected 0.0 (0.02)&#xA;Passed bias filter:                        1  (1); expected 0.0 (0.02)&#xA;Passed Vit filter:                         1  (1); expected 0.0 (0.001)&#xA;Passed Fwd filter:                         1  (1); expected 0.0 (1e-05)&#xA;Initial search space (Z):                  1  [actual number of targets]&#xA;Domain search space  (domZ):               1  [number of targets reported over threshold]&#xA;# CPU time: 0.01u 0.00s 00:00:00.01 Elapsed: 00:00:00.00&#xA;# Mc/sec: inf&#xA;//&#xA;Query:       sp|P15790|CSK21_YEAST  [L=372]&#xA;Description: Casein kinase II subunit alpha OS=Saccharomyces cerevisiae (strain ATCC 204508 / S288c) GN=CKA1 PE=1 SV=1&#xA;Scores for complete sequence (score includes all domains):&#xA;   --- full sequence ---   --- best 1 domain ---    -#dom-&#xA;    E-value  score  bias    E-value  score  bias    exp  N  Model    Description&#xA;    ------- ------ -----    ------- ------ -----   ---- --  -------- -----------&#xA;    7.5e-62  195.1   7.2    3.2e-55  173.4   0.7    2.1  2  Pkinase   Protein kinase domain&#xA;&#xA;&#xA;Domain annotation for each model (and alignments):&#xA;&amp;gt;&amp;gt; Pkinase  Protein kinase domain&#xA;   #    score  bias  c-Evalue  i-Evalue hmmfrom  hmm to    alifrom  ali to    envfrom  env to     acc&#xA; ---   ------ ----- --------- --------- ------- -------    ------- -------    ------- -------    ----&#xA;   1 !   25.5   1.1   4.3e-10   4.3e-10       1      60 [.      40      94 ..      40     115 .. 0.87&#xA;   2 !  173.4   0.7   3.2e-55   3.2e-55      56     264 .]     128     363 ..     101     363 .. 0.92&#xA;&#xA;  Alignments for each domain:&#xA;  == domain 1  score: 25.5 bits;  conditional E-value: 4.3e-10&#xA;                           EEEEEEEEEESSEEEEEEEETTTTEEEEEEEEEHHCCCHHHHHHHHHHHHHHHHHSSTTB CS&#xA;                Pkinase  1 yekleklGeGsfGkVykaveketgkivAvKkikkekakkkkekkvlrEikilkklkhpni 60&#xA;                           ye ++k+G+G++++V+++v+ +++ ++++K++k  k+kk k     rEikil  l+ +++&#xA;  sp|P15790|CSK21_YEAST 40 YEIENKVGRGKYSEVFQGVKLDSKVKIVIKMLKPVKKKKIK-----REIKILTDLSNEKV 94&#xA;                           78999****************************99888855.....7******9998877 PP&#xA;&#xA;  == domain 2  score: 173.4 bits;  conditional E-value: 3.2e-55&#xA;                            S.STTB--EEEEEEE..SSEEEEEEE--TTEBHHHHHHHHSS--HHHHHHHHHHHHHHHHHHHHTTEE-SS-SGGGEEEE.TTTEEEE CS&#xA;                Pkinase  56 k.hpnivrlyevfee..kdelylvleyveggeladllsrkgslseeeakkiakqilegleylHskgiiHrDlKpeNiLid.ekgelKi 139&#xA;                            + h ni++l++++++  +++  lv+eyv++ ++  l+    +l++ e+++++ ++l++l+y+Hs+gi+HrD+Kp+N++id ++++l++&#xA;  sp|P15790|CSK21_YEAST 128 NgHANIIHLFDIIKDpiSKTPALVFEYVDNVDFRILYP---KLTDLEIRFYMFELLKALDYCHSMGIMHRDVKPHNVMIDhKNKKLRL 212&#xA;                            35*************8878888*********9988887...5**************************************66677*** PP&#xA;&#xA;                            -SGTTSEECCSSCSBCTCCSCGGGS-HHHH.TTSTCSHHHHHHHHHHHHHHHHHSS-STTTSSCHHHHHHHHHTHHHHHHHCTSHH.. CS&#xA;                Pkinase 140 tDFGlakelessskltsfvgtreYlAPEvl.keneyskkvDvWslGvilyelltgkppfsgekgkekvekeldqlekilkilgetkek 226&#xA;                            +D+Gla+ ++ + +++ +v++r +  PE+l ++++y+ + D+Ws G +l++++++++pf   +g++    ++dql+ki+k+lg+++++&#xA;  sp|P15790|CSK21_YEAST 213 IDWGLAEFYHVNMEYNVRVASRFFKGPELLvDYRMYDYSLDLWSFGTMLASMIFKREPFF--HGTS----NTDQLVKIVKVLGTSDFE 294&#xA;                            ******************************999***************************..3443....789**********77766 PP&#xA;&#xA;                            ....TTS...............................-HHHHHHHHHHT-SSGCCSTTHHHHHTSGGG CS&#xA;                Pkinase 227 lpeasel...............................seeakdllkkllkkdpkkRltaeellqhpyl 264&#xA;                                +                                 ++e++dl+++ll++d+++Rlta+e++ hp++&#xA;  sp|P15790|CSK21_YEAST 295 KYLLKYEitlprefydmdqyirkpwhrfindgnkhlsgNDEIIDLIDNLLRYDHQERLTAKEAMGHPWF 363&#xA;                            5554444588999999****************************************************9 PP&#xA;&#xA;&#xA;&#xA;Internal pipeline statistics summary:&#xA;-------------------------------------&#xA;Query sequence(s):                         1  (372 residues searched)&#xA;Target model(s):                           1  (264 nodes)&#xA;Passed MSV filter:                         1  (1); expected 0.0 (0.02)&#xA;Passed bias filter:                        1  (1); expected 0.0 (0.02)&#xA;Passed Vit filter:                         1  (1); expected 0.0 (0.001)&#xA;Passed Fwd filter:                         1  (1); expected 0.0 (1e-05)&#xA;Initial search space (Z):                  1  [actual number of targets]&#xA;Domain search space  (domZ):               1  [number of targets reported over threshold]&#xA;# CPU time: 0.01u 0.00s 00:00:00.01 Elapsed: 00:00:00.02&#xA;# Mc/sec: 4.91&#xA;//&#xA;Query:       sp|P21333|FLNA_HUMAN  [L=2647]&#xA;Description: Filamin-A OS=Homo sapiens GN=FLNA PE=1 SV=4&#xA;Scores for complete sequence (score includes all domains):&#xA;   --- full sequence ---   --- best 1 domain ---    -#dom-&#xA;    E-value  score  bias    E-value  score  bias    exp  N  Model    Description&#xA;    ------- ------ -----    ------- ------ -----   ---- --  -------- -----------&#xA;&#xA;   [No hits detected that satisfy reporting thresholds]&#xA;&#xA;&#xA;Domain annotation for each model (and alignments):&#xA;&#xA;   [No targets detected that satisfy reporting thresholds]&#xA;&#xA;&#xA;Internal pipeline statistics summary:&#xA;-------------------------------------&#xA;Query sequence(s):                         1  (2647 residues searched)&#xA;Target model(s):                           1  (264 nodes)&#xA;Passed MSV filter:                         0  (0); expected 0.0 (0.02)&#xA;Passed bias filter:                        0  (0); expected 0.0 (0.02)&#xA;Passed Vit filter:                         0  (0); expected 0.0 (0.001)&#xA;Passed Fwd filter:                         0  (0); expected 0.0 (1e-05)&#xA;Initial search space (Z):                  1  [actual number of targets]&#xA;Domain search space  (domZ):               0  [number of targets reported over threshold]&#xA;# CPU time: 0.00u 0.00s 00:00:00.00 Elapsed: 00:00:00.00&#xA;# Mc/sec: inf&#xA;//&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;HMMER is a great tool and can also analyse multiple sequence alignments to build the hmms if that's what you need. I strongly urge you to reads &lt;a href=&quot;http://www.csb.yale.edu/userguides/seq/hmmer/docs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;its documentation&lt;/a&gt; which is both informative and a surpisingly fun read (it includes such phrases as &quot;&lt;em&gt;Their format is “proprietary”, which is an open source term of art that means both “I haven’t found time&#xA;to document them yet” and “I still might decide to change them arbitrarily without telling you”&lt;/em&gt;&quot;).&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-01-26T14:27:45.727" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3411" PostTypeId="1" AcceptedAnswerId="3414" CreationDate="2018-01-26T16:55:49.837" Score="2" ViewCount="772" Body="&lt;p&gt;I'm writing an R package that manipulates MAF files. Normally, I generate a small example of a file and use this for testing. (e.g. a subsetted BAM or MAF). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, I would need a MAF and a tumor/normal BAM pair. My problem is I don't know where I could download an open-source MAF with said BAMs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where could researchers download files like this, and not violate privacy laws (i.e. they &lt;em&gt;must&lt;/em&gt; be publicly-available BAMs)?&lt;/p&gt;&#xA;" OwnerUserId="146" LastActivityDate="2018-01-26T19:09:42.617" Title="Downloading an open-source MAF with tumor and normal BAMs?" Tags="|sam|public-databases|maf|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3412" PostTypeId="1" CreationDate="2018-01-26T18:24:23.087" Score="2" ViewCount="229" Body="&lt;p&gt;I'd like to output a figure in +/-1kb around a center using &lt;a href=&quot;http://deeptools.readthedocs.io/en/latest/content/tools/computeMatrix.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;computeMatrix&lt;/a&gt; as following command:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;computeMatrix reference-point -S data.bw -R centers.bed -a 1000 -b 1000 -bs=10 -out newdata10.gz --referencePoint center&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The bed file is like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1 100 101&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I change &lt;code&gt;-bs=1&lt;/code&gt;, two outputs of the same input newdata10.gz are quite different. Which's correct with my commands? Maybe I don't understand the meaning of binsize option. &lt;/p&gt;&#xA;" OwnerUserId="2257" LastEditorUserId="48" LastEditDate="2018-01-27T10:43:43.300" LastActivityDate="2018-10-24T17:01:16.717" Title="Different outputs of computeMatrix depending on binsize" Tags="|deeptools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3413" PostTypeId="2" ParentId="3412" CreationDate="2018-01-26T18:42:38.913" Score="1" Body="&lt;p&gt;Please don't send things like this to the deepTools mailing list and post them here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I mentioned on the list, what you're likely being surprised by is that computeMatrix is not summing values, but taking the average in each bin. You can change this with the &lt;code&gt;--averageTypeBins&lt;/code&gt; option (the default is &lt;code&gt;mean&lt;/code&gt;, you presumably want &lt;code&gt;sum&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;: For those wondering the cause of the confusion was due to not using &lt;code&gt;--missingDataAsZero&lt;/code&gt;. In other words, including or not including NAs will lead to different results, since different numbers of values will be included.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-01-27T11:54:34.637" LastActivityDate="2018-01-27T11:54:34.637" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3414" PostTypeId="2" ParentId="3411" CreationDate="2018-01-26T19:09:42.617" Score="1" Body="&lt;p&gt;There's the NCI &lt;a href=&quot;https://gdc.cancer.gov/node/216/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genomic Data Commons&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For all submitted sequence data, including BAM alignment files, the GDC generates new alignments in BAM format using the latest human reference genome GRCh38 with standard alignment pipelines. Using these standard alignments, the GDC generates high level derived data, including normal and tumor variant and mutation calls in VCF and MAF formats, and gene and miRNA expression and splice junction quantification data in TSV formats.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But that needs &lt;a href=&quot;https://wiki.nci.nih.gov/display/TCGA/Application+Process&quot; rel=&quot;nofollow noreferrer&quot;&gt;an NIH account&lt;/a&gt; for individual-level data, which might be too restrictive for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's also the &lt;a href=&quot;http://txcrb.org/data.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Texas Cancer Research Biobank&lt;/a&gt; (7 human cancer patients with matched normal):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;These tumor and normal specimen sequence files (FASTQ and BAM), somatic variant calls (VCF and MAF) and germline MAF files are available for each consented patient as described on the TCRB Open Access Privacy page.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That also requires account &lt;a href=&quot;https://hgsc-sftp1.hgsc.bcm.tmc.edu/webclient/Register.xhtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;registration&lt;/a&gt;, but the registration process is automated and much simpler (requires just a name, and acceptance of usage conditions). There's a &lt;a href=&quot;http://www.nature.com/articles/sdata201610&quot; rel=&quot;nofollow noreferrer&quot;&gt;published paper&lt;/a&gt; about that dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had some vague feeling that another repository existed (created within the last year), but I can't seem to find it from quick web searches.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-01-26T19:09:42.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3415" PostTypeId="1" AcceptedAnswerId="3416" CreationDate="2018-01-26T21:53:09.050" Score="5" ViewCount="621" Body="&lt;p&gt;This is a pretty silly/simple question, I suppose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What units are DESeq2 counts? Or are the units arbitrary but internally consistent estimate of actual reads?&lt;/p&gt;&#xA;" OwnerUserId="1226" LastActivityDate="2018-01-26T22:00:05.173" Title="What are the count units in DEseq2?" Tags="|rna-seq|deseq2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3416" PostTypeId="2" ParentId="3415" CreationDate="2018-01-26T22:00:05.173" Score="7" Body="&lt;p&gt;The counts are &quot;reads&quot; for single-end datasets and &quot;fragments&quot; for paired-end datasets. In other words, they're what featureCounts and htseq-count produce. The &quot;normalized counts&quot; that you'll be able to access in DESeq2 are, as aptly named, simply normalized versions such that values are comparable across samples.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-26T22:00:05.173" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3417" PostTypeId="1" CreationDate="2018-01-27T12:48:40.680" Score="4" ViewCount="146" Body="&lt;p&gt;I am trying to count the number of paralogues for the mouse homologues of the human protein-coding genes using BioMart. But for example in the 'PLIN4' gene its counting 35,000 paralogues instead of 4. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We think it is because some genes have one to many paralogues which causes repeats. When I run a single gene its gives me back the correct number of paralogues. Is there a way to either remove these repeats from the results or a way around this so that BioMart doesn't output these repeats. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also thought of maybe running one gene at a time, then counting it by setting up some sort of loop so that it does all of the genes from the list automatically. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code I have written so far is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Load the biomaRt package:&#xA;&#xA;library(biomaRt)&#xA;ensembl_hsapiens &amp;lt;- useMart(&quot;ensembl&quot;, &#xA;                          dataset = &quot;hsapiens_gene_ensembl&quot;)&#xA;ensembl_mouse &amp;lt;- useMart(&quot;ensembl&quot;, &#xA;                       dataset = &quot;mmusculus_gene_ensembl&quot;)&#xA;&#xA;# Get all human protein coding genes:&#xA;&#xA;hsapien_PC_genes &amp;lt;- getBM(attributes = c(&quot;ensembl_gene_id&quot;, &#xA;                                         &quot;external_gene_name&quot;), &#xA;                          filters = &quot;biotype&quot;, &#xA;                          values = &quot;protein_coding&quot;, &#xA;                          mart = ensembl_hsapiens)&#xA;&#xA;&#xA;ensembl_gene_ID &amp;lt;- hsapien_PC_genes$ensembl_gene_id&#xA;&#xA;# Get mouse homologues&#xA;&#xA;mouse_homologues &amp;lt;- getBM(attributes = c(&quot;ensembl_gene_id&quot;, &quot;external_gene_name&quot;, &#xA;                                       &quot;mmusculus_homolog_associated_gene_name&quot;), &#xA;                        filters = &quot;ensembl_gene_id&quot;, &#xA;                        values = c(ensembl_gene_ID), &#xA;                        mart = ensembl_hsapiens)&#xA;&#xA;# Get mouse external gene name &#xA;&#xA;mouse_homologues_external_gene_names &amp;lt;- mouse_homologues$mmusculus_homolog_associated_gene_name&#xA;&#xA;&#xA;mouse_paralogues &amp;lt;- getBM(attributes = c(&quot;hsapiens_homolog_associated_gene_name&quot;,&#xA;                                       &quot;external_gene_name&quot;,&#xA;                                       &quot;mmusculus_paralog_associated_gene_name&quot;), &#xA;                        filters = &quot;external_gene_name&quot;, &#xA;                        values = c(mouse_homologues_external_gene_names) , mart = ensembl_mouse)&#xA;&#xA;# Remove genes with no paralogues &#xA;mouse_paralogs_data &amp;lt;- mouse_paralogues[!(is.na(mouse_paralogues$mmusculus_paralog_associated_gene_name)&#xA;                                          | &#xA;mouse_paralogues$mmusculus_paralog_associated_gene_name==&quot;&quot;), ]&#xA;&#xA;# Count paralogues per gene&#xA;&#xA;library(plyr)&#xA;count_mouse_paralogues &amp;lt;- count(mouse_paralogs_data, &quot;external_gene_name&quot;)&#xA;View(count_mouse_paralogues)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2261" LastEditorUserId="48" LastEditDate="2018-01-27T14:13:24.957" LastActivityDate="2018-12-27T12:02:50.577" Title="Counting the number of paralogues for mouse genes gives me the wrong frequency in R" Tags="|r|bioconductor|biomart|ensembl|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3418" PostTypeId="1" AcceptedAnswerId="3419" CreationDate="2018-01-27T15:40:46.113" Score="2" ViewCount="945" Body="&lt;p&gt;I am interested in downloading all the files at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;ftp://ftp.ensembl.org/pub/release-91/embl/bos_taurus/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ensembl.org/pub/release-91/embl/bos_taurus/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know how to download a single file from that page using wget (Unix) or download.file() (R) but is there an easy way of downloading all of them?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-01-27T16:46:26.133" Title="How to download the whole directory of an ensembl FTP page?" Tags="|ensembl|data-download|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3419" PostTypeId="2" ParentId="3418" CreationDate="2018-01-27T16:46:26.133" Score="5" Body="&lt;p&gt;You can use the mirroring function of wget:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wget -m ftp://ftp.ensembl.org/pub/release-91/embl/bos_taurus/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="112" LastActivityDate="2018-01-27T16:46:26.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3420" PostTypeId="1" CreationDate="2018-01-27T19:38:32.387" Score="2" ViewCount="672" Body="&lt;p&gt;I'm using R and its Rpdb package to work with PDB files. I use the &lt;code&gt;read.pdb&lt;/code&gt; function but, nevertheless, I couldn't find the argument which will allow me to view the &lt;code&gt;SEQRES&lt;/code&gt; section and the full amino-acid sequence of the protein.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I want to find which amino-acid is in the 17th position of the sequence and I want to extract it from the 'SRQRES' section in the PDB file (the original length of the sequence was reduced for the sake of the example):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SEQRES   1 A  390  GLU PRO GLU TRP THR TYR PRO ARG LEU SER CYS GLN GLY          &#xA;SEQRES   2 A  390  SER THR PHE GLN LYS ALA LEU LEU ILE SER PRO HIS ARG          &#xA;SEQRES   3 A  390  PHE GLY GLU ALA ARG GLY ASN SER ALA PRO LEU ILE ILE          &#xA;SEQRES   4 A  390  ARG GLU PRO PHE ILE ALA CYS GLY PRO LYS GLU CYS LYS          &#xA;SEQRES   5 A  390  HIS PHE ALA LEU THR HIS TYR ALA ALA GLN PRO GLY GLY          &#xA;SEQRES   6 A  390  TYR TYR ASN GLY THR ARG GLU ASP ARG ASN LYS LEU ARG          &#xA;SEQRES   7 A  390  HIS LEU ILE SER VAL LYS LEU GLY LYS ILE PRO THR VAL          &#xA;SEQRES   8 A  390  GLU ASN SER ILE PHE HIS MET ALA ALA TRP SER GLY SER          &#xA;SEQRES   9 A  390  ALA CYS HIS ASP GLY ARG GLU TRP THR TYR ILE GLY VAL          &#xA;SEQRES  10 A  390  ASP GLY PRO ASP SER ASN ALA LEU ILE LYS ILE LYS TYR          &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;...   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original file can be found here &lt;a href=&quot;https://files.rcsb.org/download/1A4G.pdb&quot; rel=&quot;nofollow noreferrer&quot;&gt;1a4g.pdb&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas on how to accomplish that?&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="48" LastEditDate="2018-01-28T13:13:00.450" LastActivityDate="2018-01-28T21:14:43.377" Title="How to read the &quot;SEQRES&quot; section from a PDB file, using R" Tags="|r|pdb|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3421" PostTypeId="1" AcceptedAnswerId="3435" CreationDate="2018-01-27T22:01:25.447" Score="9" ViewCount="6847" Body="&lt;p&gt;Normally, I would use the &lt;a href=&quot;https://github.com/mskcc/vcf2maf&quot; rel=&quot;noreferrer&quot;&gt;vcf2maf&lt;/a&gt; scripts to convert a VCF to a MAF (or vice versa).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is great software, but on my system, perl scripts with dependencies are easy to break. (Here it uses VEP.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any other alternatives to this? &lt;/p&gt;&#xA;" OwnerUserId="146" LastEditorUserId="96" LastEditDate="2018-02-01T04:22:32.267" LastActivityDate="2020-05-27T08:34:48.517" Title="Tools to do VCF to MAF and MAF to VCF conversion?" Tags="|vcf|variant-calling|maf|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3422" PostTypeId="1" AcceptedAnswerId="3426" CreationDate="2018-01-28T18:08:22.293" Score="3" ViewCount="170" Body="&lt;p&gt;I want to detect the count of isoform transcripts for a specific gene in scRNA-seq data. Data is coming from cells of Mus Musculus.&#xA;For transcript isoforms I mean the different alternatives provided by known annotation. For each one I need the total transcript count in each cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By looking at the count matrix, alternative transcripts are not defined (see for instance &lt;a href=&quot;https://www.ebi.ac.uk/arrayexpress/experiments/E-MTAB-5728/files/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this dataset&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I come back to the fastq? Which tool may I use to produce a count matrix with counts per each alternative transcript?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2018-01-28T19:32:27.157" LastActivityDate="2018-01-30T01:25:15.740" Title="Detect transcript isoform abundance for a specific gene in scRNA-seq" Tags="|rna-seq|scrnaseq|rna-splicing|isoform|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3423" PostTypeId="2" ParentId="3146" CreationDate="2018-01-28T20:14:39.997" Score="3" Body="&lt;p&gt;You are looking for a way to compare expression profiles between cell types, and your data is counts of genes expressed per cell. Your issue is that the data is highly dimensional. It has as many dimensions as you have genes. To be able find useful information, you must reduce these dimensions using a statistical technique such as Principal COmponents Analysis (PCA), t-Distributed Stochastic Neighbor Embedding (t-SNE), or Multidimensional Scaling (MDS). The result of these analyses is a reduction of your thousands of dimensions to vectors (eigenvectors in PCA) of variance which you can plot against each other in a scatter plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been working with Single Cell RNA-seq data and have been exploring the difference in gene expression in a certain type of cells between timepoints. &lt;a href=&quot;http://satijalab.org/seurat/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seurat&lt;/a&gt; (R package) is a very powerful tool for scRNA-Seq data analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a a tSNE of my data. I have added the timepoint labels, but you can see that after performing the dimensionality reduction the two different timepoints cluster by their gene expression profiles. I can then use Seurat to see which specific genes are differentially expressed between the clusters and a whole bunch of other fun things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wFePQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wFePQ.png&quot; alt=&quot;tSNE plot of scRNA-Seq data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2018-01-28T20:14:39.997" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3424" PostTypeId="2" ParentId="3420" CreationDate="2018-01-28T21:14:43.377" Score="4" Body="&lt;p&gt;I have not used &lt;code&gt;Rpdb&lt;/code&gt; but I have used, and would recommend, &lt;a href=&quot;http://thegrantlab.org/bio3d/index.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;bio3d&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To read the PDB file 1A4G:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(bio3d)&#xA;pdb1a4g &amp;lt;- read.pdb(&quot;1a4g&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To get the SEQRES information, there are two options. Get it from the &lt;code&gt;pdb&lt;/code&gt; object directly:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seq1a4g &amp;lt;- pdb1a4g$seqres&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or call &lt;code&gt;pdbseq&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seq1a4g &amp;lt;- pdbseq(pdb1a4g)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using the second method, the 17th position (note this is not the same as the 17th position of the original protein sequence):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seq1a4g[17]&#xA; 92 &#xA;&quot;Q&quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="150" LastActivityDate="2018-01-28T21:14:43.377" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3425" PostTypeId="1" AcceptedAnswerId="3433" CreationDate="2018-01-29T00:16:57.470" Score="3" ViewCount="338" Body="&lt;p&gt;I would like to simulate a biological pathway in Matlab, Mathematica or Python. To study the response of the pathway using &lt;a href=&quot;https://en.wikipedia.org/wiki/Control_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;Control Theory&lt;/a&gt;. Is there a standard way to simulate pathways by using differential equations or any other formalism. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example I would like to try to simulate &lt;a href=&quot;http://www.kegg.jp/kegg-bin/highlight_pathway?scale=1.0&amp;amp;map=map01503&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; KEGG pathway map. Is there something more recent than &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.128.7480&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; work from 2005?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example to simulate &lt;a href=&quot;https://academic.oup.com/bib/article/8/4/210/221500&quot; rel=&quot;nofollow noreferrer&quot;&gt;Petri networks&lt;/a&gt;? &lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2020-01-14T22:31:02.417" LastActivityDate="2020-01-14T22:31:02.417" Title="How to simulate biological pathways?" Tags="|networks|pathway|systems-biology|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="3426" PostTypeId="2" ParentId="3422" CreationDate="2018-01-29T12:30:02.133" Score="4" Body="&lt;p&gt;Transcript quantification is a difficult enough problem as it is, when you add the extra difficulty of going from the low read numbers available in scRNAseq if gets even more difficult.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Added to this, many scRNAseq techniques (including 10X, which is what the linked dataset is) only capture the 3' end of the gene, so you can distinguish transcript with different 3' end, but the data simply doesn't exist for doing much more with standard tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lior Patchner does claims that there is benefit in looking at &quot;transcript compatibility counts&quot; rather than just gene counts, &lt;a href=&quot;https://pachterlab.github.io/kallisto/singlecell.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;and that you can use his Kallisto tool for this&lt;/a&gt;, and the these 3' end tagging tools do give you some data for the rest of the transcript.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also isolate transcripts with differing three 3' ends and map reads to these using the something like the &lt;a href=&quot;https://github.com/CGATOxford/UMI-tools/blob/master/doc/Single_cell_tutorial.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;UMI-Tools pipeline&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively you could look at things on a junction level - you won't capture all junctions, but you may be able to look at differential junction usage for those you do capture using something like SUPPA or MISO. For this I would use UMI-Tools to get de-duplicated BAM files, split them on cell identity and then pass them into your tool of choice. I don't really how much you will capture. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last thing to say is that this might be a space worth watching for future developments (&lt;em&gt;cough&lt;/em&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Data from some other single cell methods, like SMART-seq2 or C1 will sequence the whole transcript and you may be able to apply standard tools such as Salmon, Kalisto or RSEM to get transcript usage for them, although as I said, transcript level quantification is noisey, single cell data is noisey, single-cell level transcript data will be very noisy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whatever, you are going to have to go back to the raw sequencing data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclosure: I am the author of UMI-Tools. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-01-29T12:30:02.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3427" PostTypeId="1" AcceptedAnswerId="3428" CreationDate="2018-01-29T18:23:49.970" Score="3" ViewCount="950" Body="&lt;p&gt;I just followed the &lt;a href=&quot;http://www.sthda.com/english/articles/30-advanced-clustering/105-dbscan-density-based-clustering-essentials/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; trying to do clustering on my own data, but I can't visualize it because &lt;code&gt;fviz_cluster&lt;/code&gt; function is giving me an error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error in if (xlab == &quot;x&quot;) xlab &amp;lt;- &quot;x value&quot; : argument is of length zero&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here is the code I am running:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(&quot;fpc&quot;)&#xA;db &amp;lt;- fpc::dbscan(rtsne$Y, eps = 0.15, MinPts = 5)&#xA;# Plot DBSCAN results&#xA;library(&quot;factoextra&quot;)&#xA;fviz_cluster(db, data = rtsne$Y, stand = FALSE,&#xA;        ellipse = FALSE, show.clust.cent = FALSE,&#xA;         geom = &quot;point&quot;,palette = &quot;jco&quot;, ggtheme = theme_classic())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I make &lt;code&gt;fviz_cluster&lt;/code&gt; function working?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Update&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Filed a bug: &lt;a href=&quot;https://github.com/kassambara/factoextra/issues/62&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/kassambara/factoextra/issues/62&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="2005" LastEditDate="2018-01-30T01:00:39.007" LastActivityDate="2018-01-30T01:00:39.007" Title="factoextra: Error in if (xlab == &quot;x&quot;) xlab &lt;- &quot;x value&quot; : argument is of length zero" Tags="|clustering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3428" PostTypeId="2" ParentId="3427" CreationDate="2018-01-29T18:36:45.140" Score="3" Body="&lt;p&gt;I actually found an answer just accidentally. It is very unfortunate that &lt;code&gt;factoextra&lt;/code&gt; documentation does not explicitly say that &lt;code&gt;data&lt;/code&gt; parameter should be of &lt;code&gt;data.frame&lt;/code&gt; type. In my case &lt;code&gt;rtsne$Y&lt;/code&gt; was a &lt;code&gt;matrix&lt;/code&gt;. The error that &lt;code&gt;factoextra&lt;/code&gt; is giving is very uninformative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code that ultimately worked:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fviz_cluster(db, &#xA;-----&amp;gt;  data = as.data.frame(rtsne$Y), &#xA;        stand = FALSE,&#xA;        ellipse = FALSE, show.clust.cent = FALSE,&#xA;        geom = &quot;point&quot;,palette = &quot;jco&quot;, ggtheme = theme_classic())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-01-29T18:36:45.140" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3430" PostTypeId="2" ParentId="3422" CreationDate="2018-01-30T01:25:15.740" Score="3" Body="&lt;p&gt;Looking for alternative splicing in scRNA-seq is difficult, but there is at least one tool that is specifically designed for it:&#xA;&lt;a href=&quot;https://pbtech-vc.med.cornell.edu/git/mason-lab/disco/tree/master&quot; rel=&quot;nofollow noreferrer&quot;&gt;DISCO&lt;/a&gt;. The site does not have too much info, but there is a &lt;a href=&quot;http://rna-seqsummit.com/wp-content/uploads/sites/32/2015/12/Chris-Mason_RNA-seq_Summit_2016_edit.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;presentation about it&lt;/a&gt;. It uses MISO output as the input, which in turn uses SAM.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2018-01-30T01:25:15.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3431" PostTypeId="2" ParentId="3389" CreationDate="2018-01-30T11:08:25.810" Score="2" Body="&lt;p&gt;This is very good question, but as far as I know there really isn't a good answer. I will attempt to provide some comments and tips. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, I'm trying to do similar with a dataset which may consist of repeats (I have previously aligned the dataset using bowtie, and a lot of reads aligned to repeats). &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If this dataset is also PE and the alignment is outputting either uniquely mapped reads or a single (random) position for multimappers, it is pretty much as good as it gets. A lot of repeat regions are not &lt;strong&gt;exact&lt;/strong&gt; copies, so using PE there is a good change that one read of the pair mapping to a single position is helping with mapping of the other read.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I need to know reasonably confidently where precisely those repeats are located&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is probably a matter of phrasing, but what one needs is to know the precise genomic location of the &lt;strong&gt;reads&lt;/strong&gt;. The location of the repeat elements can be obtained from the Repeat Masker track of UCSC, but you probably know this already. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;With multimapped reads this step could be done by seeing which of the pair is uniquely mapped and choosing the mate which maps closest to it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I &lt;strong&gt;think&lt;/strong&gt; &lt;code&gt;bowtie&lt;/code&gt; already does this. Did you map the read pairs together or separately? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There will always be a number of multimapping reads (fragments) that will be discarded. The information of how many are being discarded for your samples would be helpful.  &lt;/li&gt;&#xA;&lt;li&gt;Mapping can be improved using both reads and tunning &lt;code&gt;bowtie&lt;/code&gt; settings (see bellow)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Finding a valid paired-end alignment where both mates align to&#xA;  repetitive regions of the reference can be very time-consuming. By&#xA;  default, Bowtie avoids much of this cost by imposing a limit on the&#xA;  number of &quot;tries&quot; it makes to match an alignment for one mate with a&#xA;  nearby alignment for the other. The default limit is 100. This causes&#xA;  bowtie to miss some valid paired-end alignments where both mates lie&#xA;  in repetitive regions, but the user may use the --pairtries or -y&#xA;  options to increase Bowtie's sensitivity as desired. &lt;a href=&quot;http://bowtie-bio.sourceforge.net/manual.shtml#paired-end-alignment&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The immediate step after aligning the pairs is to flatten the pair into the DNA fragment that has been sequenced. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As indicated above, the aligner will take care of that sort of. It will gave the mapping information of each read pair, which is then used by &lt;code&gt;MACS&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Other tools&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I &lt;a href=&quot;https://github.com/MikeAxtell/ShortStack/issues/36#issuecomment-240542937&quot; rel=&quot;nofollow noreferrer&quot;&gt;tested&lt;/a&gt; &lt;code&gt;ShortStack&lt;/code&gt; which uses the coverage of uniquely mapped reads to place multimapping reads for ChIP-seq, and the results were mixed. That said, it was SE data and the code has been through some changes, so it be worth a try.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When &lt;a href=&quot;https://github.com/nskvir/RepEnrich/issues/11&quot; rel=&quot;nofollow noreferrer&quot;&gt;requested&lt;/a&gt;, the authors of &lt;code&gt;RepEnrich&lt;/code&gt; provided me with the scripts for an utility called &lt;code&gt;RepConsensus&lt;/code&gt; which maps reads to a &lt;strong&gt;consensus&lt;/strong&gt; reference of repeat regions. The caveats are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;mappings are better for elements whose sequences have not diverged a lot yet&lt;/li&gt;&#xA;&lt;li&gt;it will tell the position of the protein in the repeat sequence, but no information about the surrounding genomic regions because there isn't any. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Whatever you do, do not use all alignments, that is double counting of reads, and think very carefully before using custom methods. A blog post detailing some of the potential issues can be found &lt;a href=&quot;http://adomingues.github.io/2016/04/21/Repeat-mapping/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (I wrote it). &lt;/p&gt;&#xA;" OwnerUserId="108" LastActivityDate="2018-01-30T11:08:25.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3432" PostTypeId="1" CreationDate="2018-01-30T11:57:46.357" Score="3" ViewCount="126" Body="&lt;p&gt;I am using &lt;code&gt;VMD&lt;/code&gt; to visualise the secondary structure of protein &lt;a href=&quot;https://www.ks.uiuc.edu/Research/vmd/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The trajectories are from my Gromacs simulation. Firstly I use &lt;code&gt;File - New Molecule...&lt;/code&gt; to load the &lt;code&gt;protein.gro&lt;/code&gt; file. Secondly I use &lt;code&gt;File - Load Data into Molecule...&lt;/code&gt; to load my &lt;code&gt;protein.xtc&lt;/code&gt; trajectories.&lt;/p&gt;&#xA;&lt;p&gt;Then I use &lt;code&gt;Cartoon&lt;/code&gt; to visulise the structure. As I drag the time frames, all the secondary structures (e.g. alpha-helix, beta-sheet) stay the same. But I am pretty sure some of them are changing, as I look into the &lt;code&gt;Timeline&lt;/code&gt; analysis for secondary structure.&lt;/p&gt;&#xA;&lt;p&gt;So&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;how do I use &lt;code&gt;VMD&lt;/code&gt; to see the secondary structures at different time frames?&lt;/li&gt;&#xA;&lt;li&gt;Is there an alternative software that can do this?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1806" LastEditorUserId="3848" LastEditDate="2020-11-17T11:57:55.060" LastActivityDate="2020-11-17T11:57:55.060" Title="Can VMD change its cartoon representation for secondary structure according to trajectories?" Tags="|proteins|protein-structure|visualization|" AnswerCount="0" CommentCount="4" FavoriteCount="0" ClosedDate="2021-07-04T04:53:57.100" ContentLicense="CC BY-SA 4.0" />
  <row Id="3433" PostTypeId="2" ParentId="3425" CreationDate="2018-01-30T13:39:25.197" Score="7" Body="&lt;p&gt;In matlab this can be done using the &lt;a href=&quot;https://se.mathworks.com/products/simbiology.html&quot; rel=&quot;noreferrer&quot;&gt;SimBiology toolbox&lt;/a&gt;.&lt;br&gt;&#xA;In mathematica this can be done using the &lt;a href=&quot;https://www.wolfram.com/system-modeler/examples/life-sciences/&quot; rel=&quot;noreferrer&quot;&gt;SystemModeler&lt;/a&gt;.&lt;br&gt;&#xA;In python there are multiple packages, e.g. (in no particular order) &lt;a href=&quot;http://pysb.org/&quot; rel=&quot;noreferrer&quot;&gt;PySB&lt;/a&gt;, &lt;a href=&quot;http://tellurium.analogmachine.org/&quot; rel=&quot;noreferrer&quot;&gt;Tellurium&lt;/a&gt;, &lt;a href=&quot;http://www2.gsu.edu/~matrhc/PyDSTool.htm&quot; rel=&quot;noreferrer&quot;&gt;PySD&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2276" LastActivityDate="2018-01-30T13:39:25.197" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3434" PostTypeId="2" ParentId="3417" CreationDate="2018-01-31T05:57:17.057" Score="1" Body="&lt;p&gt;I'd use &lt;code&gt;dplyr&lt;/code&gt; for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, &lt;code&gt;mouse paralogues&lt;/code&gt; contains no values for &lt;code&gt;mmusculus_paralog_associated_gene_name&lt;/code&gt; that are NA:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(dplyr)&#xA;mouse_paralogues %&amp;gt;% &#xA;  filter(is.na(mmusculus_paralog_associated_gene_name)) %&amp;gt;% &#xA;  nrow()&#xA;&#xA;[1] 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But there are values which are empty strings, so you can filter for those if you wish.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;count_mouse_paralogues &amp;lt;- mouse_paralogues %&amp;gt;%&#xA;  filter(hsapiens_homolog_associated_gene_name != &quot;&quot;) %&amp;gt;%&#xA;  count(external_gene_name)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Confirm that this gives the expected result (n = 4) for Plin4:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;count_mouse_paralogues %&amp;gt;%&#xA;  filter(external_gene_name == &quot;Plin4&quot;)&#xA;&#xA;# A tibble: 1 x 2&#xA;  hsapiens_homolog_associated_gene_name     n&#xA;  &amp;lt;chr&amp;gt;                                 &amp;lt;int&amp;gt;&#xA;1 Plin4                                     4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And check &lt;code&gt;mouse_paralogues&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mouse_paralogues %&amp;gt;% &#xA;  filter(hsapiens_homolog_associated_gene_name == &quot;PLIN4&quot;)&#xA;&#xA;  hsapiens_homolog_associated_gene_name external_gene_name mmusculus_paralog_associated_gene_name&#xA;1                                 PLIN4              Plin4                                  Plin3&#xA;2                                 PLIN4              Plin4                                  Plin5&#xA;3                                 PLIN4              Plin4                                  Plin2&#xA;4                                 PLIN4              Plin4                                  Plin1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="150" LastEditorUserId="150" LastEditDate="2018-01-31T06:04:45.530" LastActivityDate="2018-01-31T06:04:45.530" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3435" PostTypeId="2" ParentId="3421" CreationDate="2018-01-31T11:19:26.433" Score="5" Body="&lt;p&gt;Check this python script &lt;a href=&quot;https://github.com/cbare/vcf2maf&quot; rel=&quot;noreferrer&quot;&gt;vcf2maf.py&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1377" LastActivityDate="2018-01-31T11:19:26.433" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3436" PostTypeId="1" AcceptedAnswerId="3437" CreationDate="2018-01-31T18:55:33.103" Score="1" ViewCount="603" Body="&lt;p&gt;My task is to count the mutations  occurred in several proteins after a treatment. The sequences are all present in the two files in the same order. I opened both files with the FASTA parser (&lt;code&gt;SeqIO.parse&lt;/code&gt;) in Biopython and I got all the proteins listed (separated before and after the treatment).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I zip the parsers together to count the mutations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I count the mutations that occured after the treatment?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;for normal_samples in SeqIO.parse(&quot;/data/statistic/normal_samples&quot;, &quot;fasta&quot;):&#xA;    print(normal_samples.id)&#xA;    print(repr(normal_samples.seq))&#xA;    print(len(normal_samples))&#xA;&#xA;for treated_samples in SeqIO.parse(&quot;/data/statistic/with_treatment&quot;, &quot;fasta&quot;):&#xA;    print(normal_samples.id)&#xA;    print(repr(normal_samples.seq))&#xA;    print(len(normal_samples))&#xA;&#xA;dict_n_t = dict(zip(normal_samples &amp;amp; treated_samples))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2286" LastEditorUserId="2286" LastEditDate="2018-02-01T15:42:40.023" LastActivityDate="2018-02-01T15:42:40.023" Title="Comparing two multi-fasta files of the same set of proteins with parser - to find and count mutations after treatment" Tags="|python|fasta|proteins|statistics|mutations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3437" PostTypeId="2" ParentId="3436" CreationDate="2018-01-31T20:42:09.413" Score="3" Body="&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;for normal, treated in zip(SeqIO.parse(&quot;/data/statistic/normal_samples&quot;, &quot;fasta&quot;),&#xA;                           SeqIO.parse(&quot;/data/statistic/with_treatment&quot;, &quot;fasta&quot;)):&#xA;    ... do stuff...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That's generally how you &lt;code&gt;zip&lt;/code&gt; iterators together in python.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-01-31T20:42:09.413" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3438" PostTypeId="1" CreationDate="2018-01-31T21:06:13.600" Score="2" ViewCount="88" Body="&lt;p&gt;I'm trying to find files containing genotypes and phenotypes for cell lines in COSMIC.  On their &lt;a href=&quot;https://cancer.sanger.ac.uk/cosmic/download&quot; rel=&quot;nofollow noreferrer&quot;&gt;downloads page&lt;/a&gt; I found links for mutation data, which I guess could be converted to genotypes, but I haven't been able to find any phenotype information.  I'm specifically interested in drug sensitivity information.  Does anyone know if it's possible to get more user friendly genotype files and whether phenotype data exists publicly?&lt;/p&gt;&#xA;" OwnerUserId="59" LastActivityDate="2018-02-01T16:56:23.657" Title="COSMIC Genotypes and Phenotypes" Tags="|ngs|cancer|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3439" PostTypeId="1" AcceptedAnswerId="3442" CreationDate="2018-01-31T23:29:32.880" Score="1" ViewCount="29" Body="&lt;p&gt;I have run &lt;a href=&quot;http://ratt.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RATT&lt;/a&gt; to transfer the annotation from a reference strain to another strain. For this, I have run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$RATT_HOME/start.ratt.sh embl ref.fa output/annotation/ Strain&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This creates several things in the working directory that I do not wish: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a Sequences folder with the sequence in FASTA format of each chromosome&lt;/li&gt;&#xA;&lt;li&gt;a copy of the reference sequence in FASTA format (followed by a mysterious number)&lt;/li&gt;&#xA;&lt;li&gt;a copy of the query sequence in FASTA format followed by a mysterious number&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I could not find any options in the RATT help to disable the creation of these files. Also, does anybody know why these files are created?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="29" LastEditDate="2018-02-01T11:11:20.417" LastActivityDate="2018-02-01T11:20:35.130" Title="Why does RATT create a directory with chromosome sequences and how can I stop it?" Tags="|sequence-alignment|sequence-annotation|functional-annotation|ratt|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3440" PostTypeId="1" AcceptedAnswerId="3461" CreationDate="2018-02-01T01:00:33.793" Score="3" ViewCount="939" Body="&lt;p&gt;I have done a compatibility check as described &lt;a href=&quot;https://nanoporetech.com/community/lab-it-requirements&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; on my computer to know if I can use MinKNOW on my mac laptop. It says that my OS is incompatible and that the OSX must be Yosemite or El Capitan. Did anyone manage to make MinKNOW work on High Sierra? I find it odd that they would not support more recent version of mac.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-02-04T22:57:03.450" Title="Does MinKNOW work with Mac OSX high sierra 10.13.1?" Tags="|nanopore|long-reads|minion|software-installation|minknow|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3442" PostTypeId="2" ParentId="3439" CreationDate="2018-02-01T11:20:35.130" Score="1" Body="&lt;p&gt;RATT seems to be a very simple wrapper script that unfortunately does not clean up its temporary files properly. Short of modifying the code yourself, your only option is to perform the cleanup manually after running RATT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The temporary copies are made because (a) the query is not an exact copy: RATT performs some cleaning. And (b) it needs to extract a FASTA file from the EMBL input for use in downstream tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The “mysterious” number behind the file names is simply the process ID of the RATT process. This is appended to make these intermediate files more or less unique (presumably to prevent overwriting existing files).&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2018-02-01T11:20:35.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3443" PostTypeId="2" ParentId="3438" CreationDate="2018-02-01T16:56:23.657" Score="3" Body="&lt;p&gt;&lt;sup&gt;&lt;sub&gt;It seems like I was wrong and COSMIC does store information about cell lines.&lt;/sub&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need to download the files from Genotypes:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Genotypes&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Files listing the SNP calls for each cell line identified by PICNIC&#xA;  analysis of Affymetrix SNP6.0 array data. Both a simple genotype (AA,&#xA;  BB - homozygous or AB - heterozygous) and a complex interpretation of&#xA;  the genotype are given (for example, in a triploid region of the&#xA;  genome the genotype maybe AAB). 'genotypes/.'&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You need to register (and get a license) to be able to download the data. Presumably the cell lines are described in the heading of the file. Then you can sort them out by cell line. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then once you know the germ line identifier look for the paper describing it to know more about their phenotype. (But maybe there is a database with phenotype of known cell lines)&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-02-01T16:56:23.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3444" PostTypeId="1" CreationDate="2018-02-01T17:05:43.680" Score="9" ViewCount="1886" Body="&lt;p&gt;Is there a pre-existing tool which will tidy up the numbering of a PDB file? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Firstly, I would like to re-number the residues on inserts to make the icode an actual residue in the chain (by that I mean it's own number, shifting everything after it, so 45A becomes 46). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, I would also like to re-number so that each residue is one after another, as there are skips in the numbering, in spite of the two residues being physically next to each other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could do this in-house with a python script, but I thought I would ask specifically if there is a library or pre-existing tool?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Worst case scenario I have found &lt;a href=&quot;http://www.canoz.com/sdh/renumberpdbchain_pl.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; Perl script that does similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="252" LastEditorUserId="3848" LastEditDate="2018-12-16T05:46:52.197" LastActivityDate="2019-09-17T11:29:50.523" Title="Is there a standard way to clean a PDB file and re-number its residues?" Tags="|python|protein-structure|pdb|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="3445" PostTypeId="1" AcceptedAnswerId="3448" CreationDate="2018-02-01T21:21:25.590" Score="2" ViewCount="133" Body="&lt;p&gt;I am a PhD student working on developing ideas for my dissertation papers. One of my planned papers will be working with some (human) gene expression data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have had a class on working with gene expression data, but no practical experience. From what I remember from my project in that class, you do a bunch of pre-checks and QC, then you remove batch effects, then you summarize expression across probes and normalize. I thank that part of my class project took me about three days.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when I talk to other people about it, they say that cleaning/QCing/normalizing gene expression data is really hard, that it will take forever to get it working, and that if I do this dissertation paper I am going to end up taking extra years to graduate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missing something here? Why do the people I talk to seem to think this is so hard and time consuming -- is it really that bad? Should I avoid having a dissertation paper that uses gene expression data if I want to graduate on time?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(The data will be from Affymetrix microarrays. There are enough samples to use two 96-well plates, so batch effects will be more of a concern than they were in my class project.)&lt;/p&gt;&#xA;" OwnerUserId="2296" LastActivityDate="2018-02-02T12:23:44.583" Title="How hard is it to clean and QC gene expression microarray data?" Tags="|microarray|quality-control|data-preprocessing|batch-effects|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="3446" PostTypeId="1" CreationDate="2018-02-02T00:13:22.643" Score="6" ViewCount="61" Body="&lt;p&gt;I have a couple of SNP identifiers such as MARC0073381 or ALGA0066960. The corresponding platform is Illumina Porcine SNP60 BeadChip (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GPL11379&quot; rel=&quot;nofollow noreferrer&quot;&gt;WG-410&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know if these SNP are located within a promoter region of a gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bioconductor offers &lt;a href=&quot;https://bioconductor.org/packages/release/data/annotation/html/porcine.db.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;information&lt;/a&gt; for Affymetrix chips, but I am not able to map the identifiers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I map SNPs from the beadchip to the porcine genome?&lt;/p&gt;&#xA;" OwnerUserId="2298" LastEditorUserId="131" LastEditDate="2018-02-06T18:07:17.913" LastActivityDate="2018-02-06T18:07:17.913" Title="SNP located within a promoter region (pig)" Tags="|r|bioconductor|snp|illumina|porcine|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3447" PostTypeId="2" ParentId="3446" CreationDate="2018-02-02T05:49:26.387" Score="5" Body="&lt;p&gt;As far as I'm aware, Illumina provide CSV annotation files for all their sequencing chips, which can be used when they can't be found in Bioconductor. You can find annotation information for the PorcineSNP60 &lt;a href=&quot;https://support.illumina.com/downloads/porcinesnp60-v2_product_files.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, in particular the Manifest file (CSV format). The format is Illumina's weird &quot;we say it's a CSV because there are commas in it&quot; format, so if there are only a few locations of interest it would be easier to use grep than try to work out the right way to load it into R (or similar).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For completeness, you want the lines between &lt;code&gt;[Assay]&lt;/code&gt; and &lt;code&gt;[Controls]&lt;/code&gt;. Here's an example way to pre-process the &quot;CSV&quot; file into a more standard format that just has one data table in it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -n '^\[' PorcineSNP60v2_15031945_C1.csv &#xA;2:[Heading]&#xA;7:[Assay]&#xA;61574:[Controls]&#xA;### See that the table starts at line 8 (after [Assay]),&#xA;###   and finishes at line 61753&#xA;$ tail -n +8 PorcineSNP60v2_15031945_C1.csv | head -n $((61574-8)) &amp;gt; cleaned_PorcineSNP60v2_15031945_C1.csv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But for a few markers and grep, it doesn't matter if you're working with the clean or dirty CSV file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -e '^IlmnID' -e '^MARC0073381' -e '^ALGA0066960' PorcineSNP60v2_15031945_C1.csv&#xA;IlmnID,Name,IlmnStrand,SNP,AddressA_ID,AlleleA_ProbeSeq,AddressB_ID,AlleleB_ProbeSeq,GenomeBuild,Chr,MapInfo,Ploidy,Species,Source,SourceVersion,SourceStrand,SourceSeq,TopGenomicSeq,BeadSetID,Exp_Clusters,Intensity_Only&#xA;ALGA0066960-2_B_F_2199219285,ALGA0066960,BOT,[T/G],0054749354,TGCCACAGGTCTGCTCAGCTCAAGCCCAACACTCGCAAGATACAGGTCTA,,,10.2,12,55218591,diploid,Sus scrofa,PorcineSNP60v2,2,BOT,GGCTCACCTCTGCCACAGGTCTGCTCAGCTCAAGCCCAACACTCGCAAGATACAGGTCTA[T/G]CCAGGCTTCTCTCTCCTACTCCTAGGGCCCCTGATGGTTCCTGCATCCTGACCAATAGTG,CACTATTGGTCAGGATGCAGGAACCATCAGGGGCCCTAGGAGTAGGAGAGAGAAGCCTGG[A/C]TAGACCTGTATCTTGCGAGTGTTGGGCTTGAGCTGAGCAGACCTGTGGCAGAGGTGAGCC,670,3,0&#xA;MARC0073381-2_T_F_2199252059,MARC0073381,TOP,[A/G],0047761415,TGGAACGGATGGTGGAGACATTCTGGAGACAGAAGACAAACTGCTTCAGA,,,10.2,7,61808556,diploid,Sus scrofa,PorcineSNP60v2,2,TOP,CAACTGTGGTTGGAACGGATGGTGGAGACATTCTGGAGACAGAAGACAAACTGCTTCAGA[A/G]CAAAGCTCAGGAAGGCAA,CAACTGTGGTTGGAACGGATGGTGGAGACATTCTGGAGACAGAAGACAAACTGCTTCAGA[A/G]CAAAGCTCAGGAAGGCAA,660,3,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can see from this that the first marker is Chromosome 12, position 55218591, and the second marker is Chromosome 7, position 61808556. The Illumina annotation doesn't include the nearest gene, so you'll need to hunt for those locations on a genome browser (or using mapping with Bioconductor).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-02-02T05:49:26.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3448" PostTypeId="2" ParentId="3445" CreationDate="2018-02-02T12:06:40.227" Score="2" Body="&lt;p&gt;I am not sure how much you know about bioinformatics already, can you use R? For a bioinformatician looking at QC for microarrays should not be a big deal, at least for me it would take maybe a day (or two) to get this done. However, if you never used R and want to start from scratch, it depends on how quickly you learn how to deal with arrays and QC. There are many tutorials and review papers about normalization and QC of arrays. Try there and see if you can easily understand what these papers are telling you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Read the limma user's guide, and try to follow the tutorials therein. If you understand that quickly you can try a real experiment of your own. If you don't understand what's described in there, you might want to look for another project for your thesis.&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2018-02-02T12:23:44.583" LastActivityDate="2018-02-02T12:23:44.583" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3449" PostTypeId="1" AcceptedAnswerId="4016" CreationDate="2018-02-02T16:01:59.903" Score="2" ViewCount="2369" Body="&lt;p&gt;Is it possible to use &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/sva.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sva's ComBat&lt;/a&gt; for batch effects removal on scRNA-seq data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any difference between RNA-seq and scRNA-seq data which doesn't allow to use ComBat on single-cell data? (I am thinking about extreme sparsity, &gt; 80%). If so, are there alternatives?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2018-02-02T16:52:29.197" LastActivityDate="2018-04-10T22:41:03.493" Title="ComBat for batch effects removal on scRNA-seq data" Tags="|rna-seq|scrnaseq|combat|batch-effects|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3451" PostTypeId="1" CreationDate="2018-02-02T20:18:14.880" Score="3" ViewCount="303" Body="&lt;p&gt;I used BWA-MEM to alignment and I would like to gather the some informations like total % of match, mismatch, insert/delete etc. I am wondering if there is any existing tools that produces this information?&lt;/p&gt;&#xA;" OwnerUserId="2199" LastActivityDate="2018-02-03T15:37:25.633" Title="Parsing SAM/BAM files for additional information" Tags="|sequence-alignment|samtools|sam|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3453" PostTypeId="2" ParentId="3451" CreationDate="2018-02-02T21:29:17.937" Score="0" Body="&lt;p&gt;I've written my own script, &lt;a href=&quot;https://github.com/gringer/bioinfscripts/blob/master/readstomper.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;readstomper&lt;/a&gt;, which gives per-site statistics from mpileup results. Here's an example usage:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ samtools mpileup -B -Q 0 -f circ-Nb-ec3-mtDNA.fasta LAST_OL_132394_vs_mtDNA.bam | \&#xA;    /bioinf/scripts/readstomper.pl &amp;gt; LAST_OLmpileupProp_132394_vs_mtDNA.csv&#xA;&#xA;Assembly,Position,Coverage,ref,cR,pR,A,C,G,T,d,i,InsMode&#xA;Nb_mtDNA,11456,86,T,79.00,0.92,0.0000,0.0465,0.0000,0.0000,0.03,0.01,A;1.00&#xA;Nb_mtDNA,11457,86,A,81.00,0.94,0.0000,0.0000,0.0233,0.0000,0.03,0.02,TA;0.50&#xA;Nb_mtDNA,11458,86,G,79.00,0.92,0.0349,0.0233,0.0000,0.0116,0.01,0.00&#xA;Nb_mtDNA,11459,86,A,78.00,0.92,0.0000,0.0000,0.0235,0.0471,0.01,0.00&#xA;Nb_mtDNA,11460,86,T,5.00,0.06,0.0233,0.8140,0.0233,0.0000,0.08,0.00&#xA;Nb_mtDNA,11461,86,G,57.00,0.66,0.1512,0.1163,0.0000,0.0465,0.02,0.01,AATAACAACACGTAACCGA;1.00&#xA;Nb_mtDNA,11462,86,G,78.00,0.91,0.0349,0.0349,0.0000,0.0116,0.01,0.00&#xA;Nb_mtDNA,11463,86,G,66.00,0.77,0.0465,0.0698,0.0000,0.0116,0.10,0.01,GCAATA;1.00&#xA;Nb_mtDNA,11464,86,T,74.00,0.86,0.0000,0.0814,0.0233,0.0000,0.03,0.00&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It defaults to reporting variants as proportions of the total coverage, but this can be changed to reporting the individual counts:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ samtools mpileup -B -Q 0 -f mmus_chrM.fa mm2_1.2M_vs_Mmus_mtDNA.bam | \&#xA;    /bioinf/scripts/readstomper.pl --counts&#xA;&#xA;Assembly,Position,Coverage,ref,skip,cR,pR,A,C,G,T,d,i,InsMode&#xA;chrM,1528,491,C,0,409,0.87,17,0,2,15,25,65,A;52&#xA;chrM,1529,492,A,0,366,0.75,0,2,2,9,112,12,A;5&#xA;chrM,1530,493,A,0,409,0.83,0,2,1,13,68,7&#xA;chrM,1531,495,A,0,463,0.94,0,1,10,0,21,8,G;5&#xA;chrM,1532,496,A,0,463,0.94,0,2,5,8,16,4,G;3&#xA;chrM,1533,495,G,0,459,0.93,3,1,0,2,30,4&#xA;chrM,1534,496,A,0,451,0.91,0,0,4,1,39,23,G;21&#xA;chrM,1535,503,G,0,391,0.78,4,1,0,4,103,9,A;6&#xA;chrM,1536,506,G,0,496,0.98,6,0,0,0,4,10,A;3&#xA;chrM,1537,506,G,0,488,0.96,8,1,0,4,5,5,A;3&#xA;chrM,1538,506,A,0,483,0.95,0,4,3,4,12,12&#xA;chrM,1539,506,C,0,466,0.92,9,0,2,14,14,27,A;6&#xA;chrM,1540,506,A,0,469,0.93,0,0,17,6,14,11,T;3&#xA;chrM,1541,506,G,0,453,0.90,12,1,0,5,31,21,A;4&#xA;chrM,1542,505,C,0,421,0.83,6,0,2,14,62,8,CT;3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-02-02T21:29:17.937" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3454" PostTypeId="2" ParentId="3451" CreationDate="2018-02-02T21:34:06.243" Score="0" Body="&lt;p&gt;using BioAlcidaeJdk: &lt;a href=&quot;http://lindenb.github.io/jvarkit/BioAlcidaeJdk.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://lindenb.github.io/jvarkit/BioAlcidaeJdk.html&lt;/a&gt; , you can loop over the cigar strings of each mapped read and extract the number of inserted bases, clipped bases, etc...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar dist/bioalcidaejdk.jar -e 'stream().filter(R-&amp;gt;!R.getReadUnmappedFlag() &amp;amp;&amp;amp; R.getCigar()!=null).flatMap(R-&amp;gt;R.getCigar().getCigarElements().stream()).collect(Collectors.toMap(C-&amp;gt;C.getOperator(),C-&amp;gt;(long)C.getLength(),(L1,L2)-&amp;gt;L1+L2)).forEach((O,N)-&amp;gt;println(O.name()+&quot;\t&quot;+N));' toy.bam&#xA;I   15&#xA;H   11&#xA;S   1&#xA;D   1&#xA;N   14&#xA;P   2&#xA;M   191&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to get the number of mismatches, you can preprocess the bam with &lt;a href=&quot;http://lindenb.github.io/jvarkit/SamFixCigar.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://lindenb.github.io/jvarkit/SamFixCigar.html&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="71" LastActivityDate="2018-02-02T21:34:06.243" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3455" PostTypeId="1" AcceptedAnswerId="3457" CreationDate="2018-02-02T23:42:57.897" Score="0" ViewCount="998" Body="&lt;p&gt;I have a bigWig file of extension &lt;code&gt;*.bw&lt;/code&gt; (which is standard), but I am suspicious that this file may be of an incorrect format. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a standard method in the field to validate a *.bigWig file? &lt;/p&gt;&#xA;" OwnerUserId="146" LastActivityDate="2018-02-03T16:59:12.200" Title="Is there a standard method to validate whether a bigWig file is correct?" Tags="|bigwig|validation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3456" PostTypeId="2" ParentId="3455" CreationDate="2018-02-03T07:50:42.860" Score="2" Body="&lt;p&gt;There's no standard validation method, but a simple one would be &lt;code&gt;python -c &quot;import pyBigWig; bw = pyBigWig.open('your_file.bw'); print(bw.IsBigWig());&quot;&lt;/code&gt;. If you get &lt;code&gt;True&lt;/code&gt; then it's a bigWig file. If you get &lt;code&gt;False&lt;/code&gt; or an error then it's not (&lt;code&gt;False&lt;/code&gt; indicates a bigBed file).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, &lt;code&gt;file your_file.bw&lt;/code&gt; should return &lt;code&gt;data&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-02-03T07:50:42.860" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3457" PostTypeId="2" ParentId="3455" CreationDate="2018-02-03T13:13:31.710" Score="1" Body="&lt;p&gt;As noted in another answer, it's relatively easy to tell if a file is bigWig or not by opening it up with, say, standard &lt;a href=&quot;https://github.com/ENCODE-DCC/kentUtils&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC Kent utilities&lt;/a&gt; like &lt;code&gt;bigWigToBedGraph&lt;/code&gt; or &lt;code&gt;bigWigToWig&lt;/code&gt;, or other bigWig readers, and seeing if you get data back.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on what you're doing with the bigWig file, the trickier part can be how the original data was indexed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the bigWig file was originally made from a bedGraph or BAM file, its coordinates will be 0-indexed when converted to human-readable form. If the bigWig file was originally made from a wig file, its coordinates will be 1-indexed when converted to a human readable form. How the file's coordinates were originally indexed may change how you work with them downstream for set operations, for instance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the only way to know about the source is to look at the header contained within the bigWig file, after extraction to human-readable form with UCSC tools. A bigWig file made from a bedGraph file will generally contain the header &lt;code&gt;#bedGraph section &amp;lt;coords&amp;gt;&lt;/code&gt; in it somewhere (after conversion), for example.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-02-03T16:59:12.200" LastActivityDate="2018-02-03T16:59:12.200" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3458" PostTypeId="2" ParentId="3451" CreationDate="2018-02-03T15:37:25.633" Score="2" Body="&lt;p&gt;Other options... &lt;a href=&quot;http://www.htslib.org/doc/samtools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools stats&lt;/a&gt; maybe gives you some of the stats you are after. Chances are you already have samtools so nothing additional to install. Also useful may be &lt;a href=&quot;https://github.com/alimanfoo/pysamstats&quot; rel=&quot;nofollow noreferrer&quot;&gt;pysamstats&lt;/a&gt;, in particular the &lt;code&gt;variation&lt;/code&gt; sub-command.&lt;/p&gt;&#xA;" OwnerUserId="339" LastActivityDate="2018-02-03T15:37:25.633" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3459" PostTypeId="1" CreationDate="2018-02-03T21:07:41.863" Score="-1" ViewCount="374" Body="&lt;p&gt;I have ~130 gene names of different human proteins.&#xA;I'm looking for a convenient and systematic way to locate each gene's location in the genome, extract its CDS sequence and find whether there are SNPs in the CDS (in any context) and what's their position.&#xA;Moreover, I've chosen for each of those proteins, several amino acids for whom I'm trying to find if the SNP fall in their exact coding nucleotide region (the amino acid triad of nucleotides).&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, I'm looking for an algorithm that would take the gene name out of my list, find its location in the genome and look for SNPs in the exons - if it would found any SNPs, then it should translate the exon to an amino acid sequence and find whether one of my pre-defined amino acids from each protein is translated by the nucleotide site (triad of nucleotides) with the SNP.&lt;/p&gt;&#xA;&lt;p&gt;I'm using R, and I thought to use web scraping (should I use the &lt;code&gt;rvest&lt;/code&gt; package?) but I'm not quite familiar with the technique. Is it possible?&#xA;What websites should I use? Genome Browser? EMBOSS Transeq? Other websites?&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="73" LastEditDate="2021-06-07T02:31:54.310" LastActivityDate="2021-06-07T02:31:54.310" Title="Web scraping in R" Tags="|r|proteins|ngs|pdb|uniprot|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3460" PostTypeId="2" ParentId="3459" CreationDate="2018-02-04T09:10:49.357" Score="1" Body="&lt;p&gt;You can search databases like Cosmic(Cancer) with the gene names to get frequently occurring mutations in cancer. I'm not sure about any specific tool/pipeline available for this exact scenario.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an alternative approach literature mining for gene associated mutation discovery, you can use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4926749/&quot; rel=&quot;nofollow noreferrer&quot;&gt;text mining&lt;/a&gt; with machine learning to automate the whole process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With no specific context in mind, you will end with several combinations of mutation per gene. Then for each variation translate the DNA sequence and match the result with reference peptide sequence you have.&lt;/p&gt;&#xA;" OwnerUserId="206" LastActivityDate="2018-02-04T09:10:49.357" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3461" PostTypeId="2" ParentId="3440" CreationDate="2018-02-04T13:17:22.220" Score="2" Body="&lt;p&gt;Some replies from ONT is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;OS X is supported and you can view the installation instructions on&#xA;  our Downloads page linked below. At the moment High Sierra is not&#xA;  officially supported as we have not yet fully tested it, but other&#xA;  users in the community have reported no issues with it so far.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The compatibility test only performs a simplistic check to see if you&#xA;  are using one of our officially supported OS X versions. Since you are&#xA;  on High Sierra, it will show a failure. That does not mean the&#xA;  software can't be installed and used. As I said, other users have&#xA;  reported no problems, but we can't officially support it if you&#xA;  encounter an issue.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, I am going ahead and installing MinKNOW on my High Sierra.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="48" LastEditDate="2018-02-04T22:57:03.450" LastActivityDate="2018-02-04T22:57:03.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3462" PostTypeId="1" AcceptedAnswerId="3467" CreationDate="2018-02-04T16:24:35.793" Score="8" ViewCount="2546" Body="&lt;p&gt;When counting canonical kmers, ie kmers in which both the forward and reverse complement of a sequence are treated as identical, how do kmer counting programs decide which kmer to use as the canonical sequence? Do they all work the same way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To investigate I made a string with &lt;code&gt;GAGTGCGGAATACCACTCTT&lt;/code&gt; which contains all 16 possible 2mers. I then used &lt;a href=&quot;https://github.com/refresh-bio/KMC&quot; rel=&quot;noreferrer&quot;&gt;kmc&lt;/a&gt; to figure out how they determine which kmer is used. Only the kmers in the &lt;code&gt;filtered&lt;/code&gt; column below appeared. So, it looks like KMCs' 'canonical' kmers are the ones that first occur alphabetically.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;╔════════════════╦═════╦════════════════════╦══════════╗&#xA;║ Possible Kmers ║ RCs ║ RC occurs earlier? ║ filtered ║&#xA;╠════════════════╬═════╬════════════════════╬══════════╣&#xA;║ TT             ║ AA  ║ YES                ║ TA       ║&#xA;║ TG             ║ CA  ║ YES                ║ GC       ║&#xA;║ TC             ║ GA  ║ YES                ║ GA       ║&#xA;║ TA             ║ TA  ║                    ║ CG       ║&#xA;║ GT             ║ AC  ║ YES                ║ CC       ║&#xA;║ GG             ║ CC  ║ YES                ║ CA       ║&#xA;║ GC             ║ GC  ║                    ║ AT       ║&#xA;║ GA             ║ TC  ║                    ║ AG       ║&#xA;║ CT             ║ AG  ║ YES                ║ AC       ║&#xA;║ CG             ║ CG  ║                    ║ AA       ║&#xA;║ CC             ║ GG  ║                    ║          ║&#xA;║ CA             ║ TG  ║                    ║          ║&#xA;║ AT             ║ AT  ║                    ║          ║&#xA;║ AG             ║ CT  ║                    ║          ║&#xA;║ AC             ║ GT  ║                    ║          ║&#xA;║ AA             ║ TT  ║                    ║          ║&#xA;╚════════════════╩═════╩════════════════════╩══════════╝&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Do all kmer counting programs use the same canonical kmers, and if so do you have documentation explaining this? I wasn't able to find anything in the papers for &lt;code&gt;jellyfish&lt;/code&gt; or &lt;code&gt;kmc&lt;/code&gt;. &lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-02-05T14:04:07.793" Title="How do kmer counters determine which kmer is 'canonical'?" Tags="|k-mer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3463" PostTypeId="1" AcceptedAnswerId="3465" CreationDate="2018-02-04T18:02:28.213" Score="11" ViewCount="6960" Body="&lt;p&gt;What are the pros and cons of the different basecallers in Oxford Nanopore Technology Sequencing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am about to start a MinION run on my laptop. What should I consider when choosing my basecaller? Can I let MinION do its sequencing and generate fast5 files and call the bases later? I think I might use Albacore. Metrichor and MinKNOW can also do basecalling. Are there reasons to choose one over another?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-12-17T18:51:37.453" Title="What are the pros and cons of the different basecallers in Oxford Nanopore Technology Sequencing?" Tags="|nanopore|ngs|minion|genome-sequencing|base-calling|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3465" PostTypeId="2" ParentId="3463" CreationDate="2018-02-04T21:18:39.830" Score="6" Body="&lt;p&gt;First of all - yes, you can generate FAST5 files and basecall later. Basecalling during the sequencing run is useful if you want results more quickly. You can also recall your FAST5 files with multiple basecallers, if desired.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several ways to basecall currently:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;MinKNOW&lt;/li&gt;&#xA;&lt;li&gt;Albacore&lt;/li&gt;&#xA;&lt;li&gt;Guppy&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;MinKNOW uses an embedded version of Albacore to perform its basecalling. However, the version in MinKNOW occasionally lags behind the released Albacore version.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Guppy is a GPU enhanced basecaller, and has not yet been released by ONT. It is used by default on the GridION. Results are similar, but not identical to Albacore, but Guppy is much faster if you have a supported GPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There used to be a cloud version of basecalling in Metrichor, but this is no longer available, and all basecalling must now be performed locally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd suggest looking over &lt;a href=&quot;https://github.com/rrwick/Basecalling-comparison&quot; rel=&quot;noreferrer&quot;&gt;Ryan Wick's comparison&lt;/a&gt; of the different available base callers, as he has done a comprehensive look (mainly from a qualitative perspective) at the different basecallers.&lt;/p&gt;&#xA;" OwnerUserId="92" LastEditorUserId="92" LastEditDate="2018-02-04T21:33:54.370" LastActivityDate="2018-02-04T21:33:54.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3466" PostTypeId="2" ParentId="3463" CreationDate="2018-02-05T00:12:47.407" Score="7" Body="&lt;p&gt;There are also third party free and open source basecallers that haven't been developed by Oxford Nanopore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of particular note is &lt;a href=&quot;https://github.com/haotianteng/chiron&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chiron&lt;/a&gt;, which gave the best uncorrected assembly identity among the base callers that &lt;a href=&quot;https://github.com/rrwick/Basecalling-comparison&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ryan Wick tested&lt;/a&gt;. It's slower than Albacore, but appears to be more customisable, and could theoretically be used to model any DNA feature that is detectable in the electrical signal, rather than just the A/C/G/T base sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a paper about Chiron &lt;a href=&quot;https://www.biorxiv.org/content/early/2017/08/23/179531&quot; rel=&quot;nofollow noreferrer&quot;&gt;on BiorXiv&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-03-02T04:25:29.600" LastActivityDate="2018-03-02T04:25:29.600" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3467" PostTypeId="2" ParentId="3462" CreationDate="2018-02-05T04:49:04.590" Score="9" Body="&lt;p&gt;When a k-mer is identical to its reverse complement, both are canonical. Note that a canonical k-mer is a sequence, irrelevant to its position(s) in the input string. More precisely, give a string $s$, its &lt;em&gt;canonical string&lt;/em&gt; is&#xA;$$&#xA;{\rm canonical}(s|h)=\left\{\begin{array}{ll}&#xA;s &amp;amp; \mbox{if $h(s)&amp;lt;h(\overline{s})$}\\&#xA;\overline{s} &amp;amp; \mbox{otherwise}\\&#xA;\end{array}\right.&#xA;$$&#xA;where $\overline{s}$ is the Watson-Crick reverse complement of $s$ and $h$ is an arbitrary string hash function. In practice, we most often use 2-bit encoding as $h$. Order under such a hash function is equivalent to the lexicographical order.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2018-02-05T14:04:07.793" LastActivityDate="2018-02-05T14:04:07.793" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3468" PostTypeId="2" ParentId="2698" CreationDate="2018-02-05T11:04:18.407" Score="4" Body="&lt;p&gt;You can use the functions &lt;code&gt;to_hdf5&lt;/code&gt; and &lt;code&gt;to_json&lt;/code&gt; from the &lt;a href=&quot;http://biom-format.org/documentation/generated/biom.table.Table.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;biom-format package&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;to_hdf5&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;with biom.util.biom_open('output.biom', 'w') as f:&#xA;    merged_table.to_hdf5(f, &quot;example&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;to_json&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;with open('output.txt', 'w') as f:&#xA;    f.write(merged_table.to_json(&quot;example&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2307" LastEditorUserId="2307" LastEditDate="2018-08-02T20:50:44.240" LastActivityDate="2018-08-02T20:50:44.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3469" PostTypeId="2" ParentId="3463" CreationDate="2018-02-05T11:23:43.163" Score="1" Body="&lt;p&gt;Depending on your local system I'd suggest to run the MinION without local basecalling, and perform basecalling later using Albacore on server/cluster. That will keep storage requirements on your nanopore-computer low. Have albacore output fastq files which will be okay for most typical applications. Keep the fast5 files to repeat basecalling with newer versions of basecallers and to call nucleotide modifications.&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2018-02-05T11:23:43.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3470" PostTypeId="2" ParentId="3459" CreationDate="2018-02-05T23:45:24.137" Score="1" Body="&lt;p&gt;I don't have a complete answer to your question, but as a primer I can tell you that you should look into the Bioconductor packages. The reason is that biological sequence data are deposited in online databases (like GenBank, and others), and exposed by these databases in such a way that you precisely don't need web scrapping techniques to retrieve the data you're interested in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;rvest&lt;/code&gt; is a great package for web scrapping, but definitely not the appropriate tool to solve your problem.&lt;/p&gt;&#xA;" OwnerUserId="2189" LastActivityDate="2018-02-05T23:45:24.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3471" PostTypeId="1" AcceptedAnswerId="3472" CreationDate="2018-02-06T11:13:53.777" Score="3" ViewCount="481" Body="&lt;p&gt;What is the standard way to measure contig sequence lengths in a BAM? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding is that the community would use &lt;code&gt;samtools idxstats&lt;/code&gt; to compute this information from the corresponding index file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there more precise/standard approaches? &lt;/p&gt;&#xA;" OwnerUserId="1770" LastActivityDate="2018-02-06T11:18:04.657" Title="What is the standard way to measure contig sequence lengths in a BAM?" Tags="|samtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3472" PostTypeId="2" ParentId="3471" CreationDate="2018-02-06T11:18:04.657" Score="4" Body="&lt;p&gt;Standard approaches would be either &lt;code&gt;samtools idxstats&lt;/code&gt; or &lt;code&gt;samtools view -H&lt;/code&gt;, both of which will produce the exact same results. In fact, all methods will produce the exact same result, since contig length is set by the reference you align against.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-02-06T11:18:04.657" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3473" PostTypeId="1" CreationDate="2018-02-06T18:05:24.293" Score="-1" ViewCount="213" Body="&lt;p&gt;I have a list of gene names which I know are present in a genbank file. I want to order these genes based on the order of their presence in the genbank. This should also account for whether they are on the positive or negative strand. So for example I have four genes; genes_1? are on the positive strand and genes_2? are negative &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;nano Gene_names.txt&#xA;gene_1A  #assume this is the first gene for positive strand&#xA;gene_1B  &#xA;gene_2A #assume this is first gene for negative strand&#xA;gene_2B &#xA;&#xA;&#xA;&amp;gt;nano Desired_out.txt&#xA;gene_1A&#xA;gene_2B&#xA;gene_1B&#xA;gene_2A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8JEQY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8JEQY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edits&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The genbank would be the standard flat file as &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Sitemap/samplerecord.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;linked below&lt;/a&gt; by terdon. &#xA;The gene names are from a text list file and I have edited them now to reflect this.Im not exactly sure how to approach the strandness issue but I would imagine the complement would be helpful. I am unsure as to the strandness of the genes as is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A working example of my work is to get the order of SpoI-V and other sporulation genes from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NZ_CP010905.2&quot; rel=&quot;nofollow noreferrer&quot;&gt;C. difficile 630&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g. sigE, sigG, spoIIE, sleC &lt;/p&gt;&#xA;" OwnerUserId="982" LastEditorUserId="982" LastEditDate="2018-02-06T18:21:10.283" LastActivityDate="2018-02-06T19:16:20.550" Title="How do I get the gene order from genbank file using gene names?" Tags="|data-mining|genbank|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3474" PostTypeId="5" CreationDate="2018-02-06T18:08:25.207" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-02-06T18:08:25.207" LastActivityDate="2018-02-06T18:08:25.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3475" PostTypeId="4" CreationDate="2018-02-06T18:08:25.207" Score="0" Body="NCBI sequence annotation flat file. " OwnerUserId="982" LastEditorUserId="982" LastEditDate="2018-02-07T21:42:40.207" LastActivityDate="2018-02-07T21:42:40.207" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3476" PostTypeId="2" ParentId="3473" CreationDate="2018-02-06T19:16:20.550" Score="1" Body="&lt;p&gt;I can't quite grok your sorting scheme, but you can use &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/genbankr/inst/doc/genbankr.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;genbankr&lt;/a&gt; to read your genbank file into a genomic ranges object in R, and filter/sort it however you like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;source(&quot;https://bioconductor.org/biocLite.R&quot;)&#xA;biocLite(&quot;genbankr&quot;)&#xA;library(genbankr)&#xA;gb = readGenBank(&quot;sequence.gb&quot;)&#xA;subset(cds(gb), gene %in% c('sigE', 'sigG', 'spoIIE', 'sleC', 'REV7'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GRanges object with 1 range and 10 metadata columns:&#xA;      seqnames       ranges strand |        type        gene        note&#xA;         &amp;lt;Rle&amp;gt;    &amp;lt;IRanges&amp;gt;  &amp;lt;Rle&amp;gt; | &amp;lt;character&amp;gt; &amp;lt;character&amp;gt; &amp;lt;character&amp;gt;&#xA;  [1]       IX [3300, 4037]      - |         CDS        REV7        &amp;lt;NA&amp;gt;&#xA;      codon_start     product  protein_id   translation     loctype&#xA;        &amp;lt;numeric&amp;gt; &amp;lt;character&amp;gt; &amp;lt;character&amp;gt; &amp;lt;AAStringSet&amp;gt; &amp;lt;character&amp;gt;&#xA;  [1]           1       Rev7p  AAA98667.1     MNR...SLF      normal&#xA;          gene_id transcript_id&#xA;      &amp;lt;character&amp;gt;   &amp;lt;character&amp;gt;&#xA;  [1]        REV7        REV7.1&#xA;  -------&#xA;  seqinfo: 1 sequence from U49845.1 genome&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-02-06T19:16:20.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3477" PostTypeId="1" CreationDate="2018-02-07T00:39:45.323" Score="15" ViewCount="34840" Body="&lt;p&gt;I have VCF files (SNPs &amp;amp; indels) for WGS on 100 samples, but I want to only use a specific subset of 10 of the samples. Is there a relatively easy way to pull out only the 10 samples, while still keeping all of the data for the entire genome?&lt;/p&gt;&#xA;&lt;p&gt;I have a script that allows me to pull out regions of the whole genome for all 100 samples, so if I could do something similar but only put regions for the 10 samples that I want that would be ideal.&lt;/p&gt;&#xA;" OwnerUserId="2312" LastEditorUserId="3694" LastEditDate="2023-09-06T09:37:37.197" LastActivityDate="2024-01-30T10:34:04.933" Title="How to subset samples from a VCF file?" Tags="|vcf|wgs|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3478" PostTypeId="2" ParentId="3477" CreationDate="2018-02-07T02:11:01.323" Score="16" Body="&lt;p&gt;Bcftools has sample/individual filtering as an option for most of the commands. You can subset individuals by using the &lt;code&gt;-s&lt;/code&gt; or &lt;code&gt;-S&lt;/code&gt; option:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;-s&lt;/code&gt;, &lt;code&gt;--samples [^]LIST&lt;/code&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Comma-separated list of samples to include or exclude if prefixed with &quot;^&quot;. Note that in general tags such as INFO/AC, INFO/AN, etc are not updated to correspond to the subset samples. &lt;code&gt;bcftools view&lt;/code&gt; is the exception where some&#xA;  tags will be updated (unless the &lt;code&gt;-I&lt;/code&gt;, &lt;code&gt;--no-update&lt;/code&gt; option is used; see &lt;code&gt;bcftools view&lt;/code&gt; documentation). To use updated tags for the subset in another command one can pipe from view into that command. For example:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;code&gt;-S&lt;/code&gt;, &lt;code&gt;--samples-file FILE&lt;/code&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;File of sample names to include or exclude if prefixed with &quot;^&quot;. One sample per line. See also the note above for the &lt;code&gt;-s&lt;/code&gt;, &lt;code&gt;--samples&lt;/code&gt; option. The command &lt;code&gt;bcftools call&lt;/code&gt; accepts an optional second column indicating ploidy (0, 1 or 2) or sex (as defined by &lt;code&gt;--ploidy&lt;/code&gt;, for example &quot;F&quot; or &quot;M&quot;), and can parse also PED files. If the second column is not present, the sex &quot;F&quot; is assumed. With &lt;code&gt;bcftools call -C trio&lt;/code&gt;, PED file is expected. File formats examples:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sample1    1&#xA;sample2    2&#xA;sample3    2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sample1    M&#xA;sample2    F&#xA;sample3    F&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;or a .ped file (here is shown a minimum working example, the first column is ignored and the last indicates sex: 1=male, 2=female):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ignored daughterA fatherA motherA 2&#xA;ignored sonB fatherB motherB 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Example usage:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view -s sample1,sample2 file.vcf &amp;gt; filtered.vcf&#xA;bcftools view -S sample_file.txt file.vcf &amp;gt; filtered.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;See the bcftools manpage for more information.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-02-07T02:16:10.953" LastActivityDate="2018-02-07T02:16:10.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3479" PostTypeId="1" AcceptedAnswerId="3481" CreationDate="2018-02-07T05:29:25.317" Score="4" ViewCount="116" Body="&lt;p&gt;Li and Durbin's &quot;Fast and accurate short read alignment with burrows-whleeler transform&quot; found &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2705234/#!po=74.5283&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We evaluate the performance of BWA on ... real paired-end data by&#xA;  checking the fraction of reads mapped in consistent pairs and by&#xA;  counting misaligned reads mapped against a hybrid genome. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The data for the real genome aligned to is in table 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which pairs are referred to here? Are these the different software tools aligning the same corresponding reads meaning that they aligned the same sequences to similar locations? Are these paired ends? Or is this something different? &lt;/p&gt;&#xA;" OwnerUserId="2063" LastEditorUserId="298" LastEditDate="2018-02-07T15:23:07.287" LastActivityDate="2018-02-07T15:23:07.287" Title="How does Li and Durbin's BWA paper compare alignment programs on real data?" Tags="|sequence-alignment|bwa|algorithms|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3480" PostTypeId="2" ParentId="3479" CreationDate="2018-02-07T09:18:49.103" Score="2" Body="&lt;p&gt;What they are pairing is the RNAseq data with the reference genome: as the legend of the table 2 says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The 12.2 million read pairs were mapped to the human genome&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;These softwares align the same read to the same genome. They aren't comparing if it is mapped to the same location, only how many mismatches has each program.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-02-07T09:18:49.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3481" PostTypeId="2" ParentId="3479" CreationDate="2018-02-07T09:20:25.457" Score="2" Body="&lt;p&gt;For the simulated reads, they know &lt;em&gt;exactly&lt;/em&gt; where the reads came from, because they are able to create that link at the time paired-end reads are generated from the genome. Those simulated reads would be created through a process similar to the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Pick a single position in the genome at random&lt;/li&gt;&#xA;&lt;li&gt;Pick another region in the same chromosome that is &lt;em&gt;&amp;lt;fragment length&amp;gt;&lt;/em&gt;+&lt;em&gt;error&lt;/em&gt; bases away from the first location&lt;/li&gt;&#xA;&lt;li&gt;Extend these positions by &lt;em&gt;&amp;lt;read length&amp;gt;&lt;/em&gt; (one read in the forward direction, and the other read in the reverse complement direction) to generate a read pair&lt;/li&gt;&#xA;&lt;li&gt;Change at random some bases (or none) within the reads to simulate sequencing error&lt;/li&gt;&#xA;&lt;li&gt;store the read pair in a FASTA file, together with the original position&lt;/li&gt;&#xA;&lt;li&gt;repeat steps 1-5 a few hundred million times&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The effectiveness of the algorithm would be based on how good the algorithm was at picking the correct genomic location.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the real paired-end data, it's a classical mapping problem: there's no ground truth, so all that can be done is starting at step 5, and working back to all the possibilities for step 1. If the two sequences from a read pair are derived from the same fragment, they should map to approximately the same genomic location and [usually] in a reverse-complement orientation to each other. Such an alignment would be called &quot;consistent&quot;, or &quot;properly mapped&quot;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-02-07T09:20:25.457" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3482" PostTypeId="1" AcceptedAnswerId="3484" CreationDate="2018-02-07T17:51:53.947" Score="0" ViewCount="548" Body="&lt;p&gt;I have bam files. I want to find the total read counts associated with all the biotypes, eg snRNA,rRNA,tRNA mRNA,scRNA,snoRNA etc. I can use ht-seq count to get read  counts for the genes, but is there a tool which can directly sum up counts for each of the above category. I have a gencode hg38 gtf file as my annotation file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="2040" LastActivityDate="2022-01-13T03:54:04.760" Title="Counting reads for each biotype" Tags="|sam|reads|gtf|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3483" PostTypeId="1" AcceptedAnswerId="3485" CreationDate="2018-02-07T18:46:50.190" Score="2" ViewCount="1247" Body="&lt;p&gt;I subset a BAM to only include primary reads using the following samtools commands:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -F 256 input.bam &amp;gt; input.primaryOnly.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, in order to convert this SAM into a new subsetted BAM, I would need to add a header. The way I normally do this is with Picard:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar picard.jar ReplaceSamHeader \&#xA;      I=input.primaryOnly.sam \&#xA;      HEADER=input.header.sam \&#xA;      O=new.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;whereby the header &lt;code&gt;input.header.sam&lt;/code&gt; is from the original BAM: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -H input.bam &amp;gt; input.header.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I could convert the SAM into a BAM with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -S -b i new.sam &amp;gt; new.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My problem is, I'm using the original BAM header. If I look back at the BAM, it now contains wrong information, e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools flagstat new.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;might save that secondary alignments exist, when they don't. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I get a BAM header that makes sense given my new BAM (with only primary alignments)? &lt;/p&gt;&#xA;" OwnerUserId="1770" LastActivityDate="2018-02-07T21:41:46.003" Title="Modifying a SAM header after removing all non-primary reads" Tags="|sam|samtools|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3484" PostTypeId="2" ParentId="3482" CreationDate="2018-02-07T19:19:10.087" Score="1" Body="&lt;p&gt;Maybe there's one tool that does everything, but here's one approach that uses a mix of Unix tools and BEDOPS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the start, one thing that may be necessary is to fix GTF files that lack an attribute required by specification:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk '{ if ($0 ~ &quot;transcript_id&quot;) print $0; else print $0&quot; transcript_id \&quot;\&quot;;&quot;; }' annotations.gtf &amp;gt; annotations.fixed.gtf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This may not be an issue for your GTF file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a correctly-formatted GTF, we can filter it by &lt;code&gt;gene_biotype&lt;/code&gt; attribute, e.g., &lt;code&gt;snoRNA&lt;/code&gt;, and convert those filtered annotations to a sorted BED file with &lt;code&gt;gtf2bed&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ NEEDLE=&quot;snoRNA&quot;&#xA;$ grep -v '^#' annotations.fixed.gtf | awk -vneedle=${NEEDLE} '{ match($0, /gene_biotype &quot;([a-zA-Z_]+)&quot;/, a); if ((needle==a[1]) &amp;amp;&amp;amp; ($8==&quot;gene&quot;)) { print $0; } }' | gtf2bed - &amp;gt; ${NEEDLE}.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Secondly, we can convert the BAM files to sorted BED files via &lt;code&gt;bam2bed&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bam2bed &amp;lt; reads.bam &amp;gt; reads.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Once we have &lt;code&gt;snoRNA.bed&lt;/code&gt; and &lt;code&gt;reads.bed&lt;/code&gt;, we can use &lt;code&gt;bedmap --count&lt;/code&gt; to count the number of reads which overlap each snoRNA-annotated gene from the original annotations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --count snoRNA.bed reads.bed &amp;gt; answer.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One complication is that the reads should have a chromosome name scheme identical to that of your GTF file, so that mapping can be done between elements on the same chromosome. Mixing Ensembl and UCSC sourced files, for example, could require an extra step to add/remove &lt;code&gt;chr&lt;/code&gt; prefix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, each line of &lt;code&gt;answer.txt&lt;/code&gt; is the number of reads that overlap a snoRNA-typed annotation, not the overall sum.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To sum all values, you can add to the &lt;code&gt;bedmap&lt;/code&gt; statement an &lt;code&gt;awk&lt;/code&gt; command that runs a simple accumulator:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --count snoRNA.bed reads.bed | awk 'START{s=0;}{s+=$0;}END{print s;}' &amp;gt; sum.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The file &lt;code&gt;sum.txt&lt;/code&gt; will have the sum of reads overlapping annotations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This procedure could be repeated for other gene biotype categories, &lt;code&gt;snRNA&lt;/code&gt;, &lt;code&gt;rRNA&lt;/code&gt;, etc. by making a script that swaps in a category name from a list of such, rerunning the GTF conversion and filtering step, and re-running the &lt;code&gt;bedmap&lt;/code&gt; step on the new annotation subset.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2018-02-07T19:19:10.087" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3485" PostTypeId="2" ParentId="3483" CreationDate="2018-02-07T21:41:46.003" Score="5" Body="&lt;p&gt;Your first command only needs a slight modification to add in &lt;code&gt;-h&lt;/code&gt;. This will create a SAM file with a header.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h -F 256 input.bam &amp;gt; headered.input.primaryOnly.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;-h&lt;/strong&gt;        Include the header in the output.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If your primary end goal is to create a BAM, then as Pierre has pointed out you can create BAM files directly from &lt;code&gt;samtools view&lt;/code&gt; with the &lt;code&gt;-b&lt;/code&gt; option:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -b -F 256 input.bam &amp;gt; input.primaryOnly.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And if your end-goal is a &lt;em&gt;sorted&lt;/em&gt; BAM file, then pipe the BAM output through samtools sort:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -b -F 256 input.bam | samtools sort &amp;gt; sorted.input.primaryOnly.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;So that's fixing what &lt;em&gt;I&lt;/em&gt; think your problem is, but what about the specific question that you have asked, i.e. how can a header be added to an existing headerless SAM file?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have an existing BAM file, you can use Picard (as you have mentioned), or manually piece the files together with &lt;code&gt;cat&lt;/code&gt;, which will have a similar function (although Picard probably includes a few checks to make sure that the header is consistent with the input file):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(samtools view -H input.bam; cat input.primaryOnly.sam) &amp;gt; headered.input.primaryOnly.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If the only thing that matters is that the BAM file has correct reference information (which is fairly common), then the &lt;code&gt;-T&lt;/code&gt; option can be used with &lt;code&gt;samtools view&lt;/code&gt;. Again, &lt;code&gt;-h&lt;/code&gt; or &lt;code&gt;-b&lt;/code&gt; is required to make sure both the header and the usual SAM fields are displayed:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -T reference.fa -h input.primaryOnly.sam &amp;gt; headered.input.primaryOnly.sam&#xA;samtools view -T reference.fa -b input.primaryOnly.sam &amp;gt; headered.input.primaryOnly.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-02-07T21:41:46.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3486" PostTypeId="1" AcceptedAnswerId="3494" CreationDate="2018-02-08T00:06:19.030" Score="7" ViewCount="868" Body="&lt;p&gt;This question is a follow up from &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/3462&quot;&gt;How do kmer counters determine which kmer is 'canonical'?&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that question we learned that kmer counting programs use a 2-bit hash function to internally represent canonical kmers as they are being counted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I am wondering, how can we implement such a function in C/C++ or in python? More arbitrarily, how could I calculate the canonical kmer hash value using a mathematical function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, how would we transform the 3-mer &lt;code&gt;GAT&lt;/code&gt; or the 21-mer &lt;code&gt;GAATACCATAGGATA&lt;/code&gt; to 1s and 0s such that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;hash(GAT) == hash(ATC)&#xA;hash(GAATACCATAGGATA) == hash(TATCCTATGGTATTC)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2019-02-02T10:31:19.663" Title="How to write a hash function for canonical kmers?" Tags="|python|k-mer|c++|hashing|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3487" PostTypeId="1" CreationDate="2018-02-08T02:36:12.927" Score="2" ViewCount="56" Body="&lt;p&gt;I'd like to analyze some Copy Number Variation data produced with SNP chip technologies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Data is provided using contingency tables, i.e. for each gene it is provided:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;         Test_Positive  Test_Negative&#xA;Cases           [0,9]+            ...&#xA;Controls           ...            ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In example, test positive may indicate detected heterozygous deletion. In each cell there will be the count of cases/controls tested positive/negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which methodologies may be used other than Fisher's exact test?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-02-11T18:40:02.053" Title="Methodologies for CNV analysis on SNP chip data" Tags="|cnv|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3488" PostTypeId="2" ParentId="3486" CreationDate="2018-02-08T04:40:20.200" Score="5" Body="&lt;p&gt;However you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to do this is to generate both the forward and reverse complement kmers, then choose the lexicographically-least kmer for the storage key. To delve further into this requires discussion about things like the size of the underlying array, the expected distribution of keys throughout the array, and what type of key clustering is desirable/undesirable. What is actually done doesn't matter, as long as there is a deterministic algorithm that can be applied to every single possible sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The nice thing about most modern languages is that they've already got their own hash implementations that map keys to values, which work fine for quick experiments: C++ has &lt;a href=&quot;http://www.cplusplus.com/reference/unordered_map/unordered_map/&quot; rel=&quot;nofollow noreferrer&quot;&gt;unordered map&lt;/a&gt;, perl has &lt;a href=&quot;http://perldoc.perl.org/perlintro.html#Perl-variable-types&quot; rel=&quot;nofollow noreferrer&quot;&gt;hashes&lt;/a&gt;, python has &lt;a href=&quot;https://docs.python.org/3/tutorial/datastructures.html#dictionaries&quot; rel=&quot;nofollow noreferrer&quot;&gt;dictionaries&lt;/a&gt;, and R has &lt;a href=&quot;https://www.rdocumentation.org/packages/base/versions/3.4.3/topics/name&quot; rel=&quot;nofollow noreferrer&quot;&gt;names&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The actual bare-metal implementation of the thing that converts a string of text into a number is complicated, particularly if you want code to be cryptographically secure, and the hash array to have an access speed that is very close to the amount of time it takes to retrieve a single value from memory. A useful starting point to explore this in more depth might be the Wikipedia page on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Java_hashCode%28%29&quot; rel=&quot;nofollow noreferrer&quot;&gt;Java hashcode function&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2019-02-02T10:31:19.663" LastActivityDate="2019-02-02T10:31:19.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3490" PostTypeId="2" ParentId="3477" CreationDate="2018-02-08T10:20:01.993" Score="2" Body="&lt;p&gt;You can use the GATK's &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/tooldocs/current/org_broadinstitute_hellbender_tools_walkers_variantutils_SelectVariants.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;SelectVariants&lt;/a&gt; tool with the &lt;code&gt;-sn&lt;/code&gt; flag. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&#xA;gatk SelectVariants -V input.vcf -R reference.fasta -sn Sample_01 -out sample.vcf&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may use the &lt;code&gt;-sn&lt;/code&gt; flag several times so as to select several samples, or use it to point to a file containing a sample name on every line. &lt;/p&gt;&#xA;" OwnerUserId="1394" LastActivityDate="2018-02-08T10:20:01.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3491" PostTypeId="1" CreationDate="2018-02-08T12:32:21.160" Score="2" ViewCount="94" Body="&lt;p&gt;I am working on big networks of genes. I want to locate KEGG/Reactome pathways in my network. I have already tried cytoKegg and ReactomeFI (Plugins of Cytoscape), they only give me a list of Pathways with the genes in it. But I want to visually see the grouped genes from my network which belong to one pathway in either KEGG or Reactome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My network file looks like this &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ADAM17 NOTCH1 -&amp;gt;&#xA;ADAM17 TNF -&amp;gt;&#xA;ADAM17 TRAF6 -&amp;gt;&#xA;ADRB2 GNAI1 -&amp;gt;&#xA;ADRB2 GNAI2 -&amp;gt;&#xA;AJUBA CDH1 &amp;lt;-&#xA;AJUBA CTNNB1 &amp;lt;-&#xA;AJUBA LATS2 -|&#xA;AMOTL2 LATS2 -&amp;gt;&#xA;AMPH EGF -&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to see the genes in a pathway in KEGG or Reactome from my network to be either grouped together or colored. Is there any tool that does this?&lt;/p&gt;&#xA;" OwnerUserId="2319" LastEditorUserId="2319" LastEditDate="2018-02-08T13:01:39.660" LastActivityDate="2018-02-08T13:01:39.660" Title="Locate KEGG/Reactome pathways in user defined network" Tags="|gene|networks|kegg|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3492" PostTypeId="1" AcceptedAnswerId="3493" CreationDate="2018-02-08T19:47:59.787" Score="3" ViewCount="552" Body="&lt;p&gt;I would like to find out where (in genomic coordinates) large insertions are found within a given BAM file, &lt;code&gt;file.bam&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(In terms of genomic coordinates, I just mean I would like to have a rough idea where to look at the BAM in IGV, chromosome and position.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the BAM CIGAR, I should be able to &lt;code&gt;grep&lt;/code&gt; the BAM based on some cut-off of &quot;large inserted&quot; bases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's unclear to me how to best do this via samtools; one could parse the cigar string via &lt;code&gt;samtools&lt;/code&gt; for insertions, but I don't understand how to translate this into the genomic position in the alignment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This goes with various samtools wrappers, e.g. &lt;code&gt;pysam&lt;/code&gt; in python:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pysam&#xA;&#xA;bamfile = pysam.Samfile(bamFile, &quot;rb&quot;);&#xA;&#xA;for read in bamfile:&#xA;    if not read.is_unmapped:   ## only get mapped reads&#xA;        cigarLine = read.cigar&#xA;        for cigarType, cigarLength in cigarLine:&#xA;            if cigarType == 1: ## this means it's an insertion...&#xA;                  ## somehow filter only large insertions, and then get coordinate information&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm not sure how to do this in a non-clumsy manner. Any help appreciated.&lt;/p&gt;&#xA;" OwnerUserId="146" LastEditorUserId="77" LastEditDate="2018-02-08T20:00:08.060" LastActivityDate="2018-02-09T14:13:31.867" Title="Parse BAM by insertion size and get genomic coordinates" Tags="|sam|samtools|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3493" PostTypeId="2" ParentId="3492" CreationDate="2018-02-08T20:06:06.460" Score="2" Body="&lt;p&gt;You basically have the right framework already in your pysam code, which I took the liberty of tidying up a bit. You just need to add a bit:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;for read in bamfile:&#xA;    if not read.is_unmapped:&#xA;        cigarLine = read.cigar&#xA;        interesting = False&#xA;        for cigarType, cigarLength in cigarLine:&#xA;            if cigarType == 1:&#xA;                if cigarLength &amp;gt; 100:  ## Arbitrary value&#xA;                    interesting = True&#xA;                    break&#xA;        if interesting:&#xA;            print(&quot;{}:{}-{}&quot;.format(read.reference_name, read.reference_start, read.reference_end))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unless you have long reads, this will be close enough for IGV.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-02-08T20:06:06.460" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3494" PostTypeId="2" ParentId="3486" CreationDate="2018-02-08T21:09:41.050" Score="7" Body="&lt;p&gt;You can convert any string hash function to a &quot;canonical&quot; DNA string hash function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a DNA string $s$, let $\overline{s}$ be its Watson-Crick reverse complement. Suppose $h:\Sigma^*\to\mathbb{Z}$ is an arbitrary string hash function. You can define&#xA;$$&#xA;\tilde{h}(s)\triangleq \min\{h(s),h(\overline{s})\}&#xA;$$&#xA;Then for any DNA string $s$&#xA;$$&#xA;\tilde{h}(s)=\tilde{h}(\overline{s})=\tilde{h}({\rm canonical}(s|h))&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: There are more than one ways to define &quot;canonical&quot; hash function. For example, let $h_0(\cdot)$ be the 2-bit encoding hash function. Given any &lt;em&gt;integer&lt;/em&gt; hash function $g(\cdot)$, we can define:&#xA;$$&#xA;h'(s)\triangleq g\big[\min\{h_0(s),h_0(\overline{s})\}\big]&#xA;$$&#xA;Then we still have&#xA;$$&#xA;h'(s)=h'(\overline{s})&#xA;$$&#xA;You can also replace $\min\{\}$ with XOR or plus to get new definitions.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2018-02-08T21:21:39.330" LastActivityDate="2018-02-08T21:21:39.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3495" PostTypeId="2" ParentId="3380" CreationDate="2018-02-08T21:30:12.453" Score="1" Body="&lt;p&gt;You can use &lt;a href=&quot;https://github.com/karel-brinda/samsift&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAMsift&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;samsift -i file.bam -0 'q=open(&quot;qnames.txt&quot;).read().splitlines()' -f 'QNAME in q'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;-i file.bam&lt;/code&gt; specifies the input BAM file. &lt;code&gt;-0 'q=open(&quot;qnames.txt&quot;).read().splitlines()'&lt;/code&gt; loads your list of QNAMEs. &lt;code&gt;-f 'QNAME in q'&lt;/code&gt; specifies the filter for alignments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With SAMsift one can use any Python expression for filtering. The downside is that it's slower than the other tools.&lt;/p&gt;&#xA;" OwnerUserId="425" LastEditorUserId="425" LastEditDate="2018-02-08T21:58:41.100" LastActivityDate="2018-02-08T21:58:41.100" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3496" PostTypeId="1" AcceptedAnswerId="3497" CreationDate="2018-02-08T22:34:55.673" Score="2" ViewCount="6659" Body="&lt;p&gt;Trying to use deseq2 for differential expression analysis (rna-seq) between three groups and also account for batch effect as the control were sequenced at a different time point.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;control: con &lt;/li&gt;&#xA;&lt;li&gt;sample with mutation A: mutA &lt;/li&gt;&#xA;&lt;li&gt;sample with mutation B: mutB&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;here is my design &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   sample condition batch&#xA;1     C_1         con     1&#xA;2     C_2         con     1&#xA;3     C_5         con     1&#xA;4     C_3         con     1&#xA;5     C_4         con     1&#xA;6     M_6        mutA     2&#xA;7     M_2        mutA     2&#xA;8     M_5        mutA     2&#xA;9     M_1        mutA     2&#xA;10    M_4        mutA     2&#xA;11    M_3        mutA     2&#xA;12   MA_2        mutB     2&#xA;13   MA_6        mutB     2&#xA;14   MA_5        mutB     2&#xA;15   MA_4        mutB     2&#xA;16   MA_3        mutB     2&#xA;17   MA_1        mutB     2 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;when i create matrix:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData = data, colData = coldata, design = ~batch + condition)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I see this error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in checkFullRank(modelMatrix) : &#xA;  the model matrix is not full rank, so the model cannot be fit as specified.&#xA;  One or more variables or interaction terms in the design formula are linear&#xA;  combinations of the others and must be removed.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also tried to compare only two conditions subsetting the original data, i still see the same error what am i missing?&lt;/p&gt;&#xA;" OwnerUserId="926" LastActivityDate="2023-03-11T07:44:46.800" Title="Error in checkFullRank(modelMatrix) deseq2" Tags="|rna-seq|deseq2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3497" PostTypeId="2" ParentId="3496" CreationDate="2018-02-08T22:56:08.313" Score="4" Body="&lt;p&gt;See the second example from &lt;a href=&quot;https://bioconductor.org/packages/3.7/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#model-matrix-not-full-rank&quot; rel=&quot;nofollow noreferrer&quot;&gt;the vignette&lt;/a&gt;, which is exactly your situation. The software cannot differentiate between the batch effects vs. condition effects because the groups are exactly the same.&lt;/p&gt;&#xA;" OwnerUserId="2320" LastEditorUserId="48" LastEditDate="2018-02-09T08:19:05.103" LastActivityDate="2018-02-09T08:19:05.103" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3498" PostTypeId="1" CreationDate="2018-02-09T10:16:18.103" Score="2" ViewCount="1315" Body="&lt;p&gt;The &lt;code&gt;CONECT&lt;/code&gt; section in a PDB file describes &lt;a href=&quot;http://www.wwpdb.org/documentation/file-format-content/format33/sect10.html#CONECT&quot; rel=&quot;nofollow noreferrer&quot;&gt;atomic connectivity&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;SITE&lt;/code&gt; section describes (among other characteristics), &lt;a href=&quot;http://www.wwpdb.org/documentation/file-format-content/format33/sect7.html#SSBOND&quot; rel=&quot;nofollow noreferrer&quot;&gt;environments surrounding ligands present in the structure&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, how come not every atom in the &lt;code&gt;CONECT&lt;/code&gt; section is described in the &lt;code&gt;SITE&lt;/code&gt; section? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example: &lt;a href=&quot;https://files.rcsb.org/view/1A28.pdb&quot; rel=&quot;nofollow noreferrer&quot;&gt;1a28.pdb&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="48" LastEditDate="2018-02-09T11:40:28.910" LastActivityDate="2018-02-09T11:40:28.910" Title="What are the differences between &quot;Site&quot; and &quot;CONECT&quot; in a PDB file" Tags="|protein-structure|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3499" PostTypeId="2" ParentId="3498" CreationDate="2018-02-09T11:33:21.543" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;how come not every atom in the CONECT section is described in the SITE section?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Because it is not required and not needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;CONECT&lt;/code&gt; is for listing bonds, so that molecular graphics programs know how to draw the structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;SITE&lt;/code&gt; is for annotation of interesting sites. Each listed SITE has a corresponding REMARK 800 that &quot;details its significance&quot;. A graphical program may provide a menu with a list of all interesting sites for easy navigation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: PDB is a legacy file format. The primary file format used nowadays by the PDB (organization) is called PDBx/mmCIF. So when looking into such details it is often better to look directly into mmCIF files.&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2018-02-09T11:33:21.543" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3500" PostTypeId="1" CreationDate="2018-02-09T11:54:18.113" Score="5" ViewCount="433" Body="&lt;p&gt;I am trying to analyse the hydrophobicity of a sequence using BioPython's &quot;SeqUtil&quot;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;analysed_seq = ProteinAnalysis(&quot;AKVLGSLEALEPVHYEEKNWCE&quot;)&#xA;analysed_seq.protein_scale(Scale, WindowSize, Edge)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Known parameters&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;WindowSize&lt;/code&gt; refers to the total number of amino acids to be considered either side of the residue (an &lt;code&gt;int&lt;/code&gt; variable). &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;Edge&lt;/code&gt; refers to the weighting of the scale further from the centre of the window (0-1). `&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Unknown parameter&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Scale&lt;code&gt;refers to which scale being used. I am interested in how to give the&lt;/code&gt;Scale` variable a valid variable.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://biopython.org/wiki/ProtParam&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biopython docs&lt;/a&gt; simply point out that there are many &lt;code&gt;scales&lt;/code&gt; to choose from and the &lt;a href=&quot;http://biopython.org/DIST/docs/api/Bio.SeqUtils.ProtParam-pysrc.html#ProteinAnalysis.protein_scale&quot; rel=&quot;nofollow noreferrer&quot;&gt;source code&lt;/a&gt; doesn't seem to give any clues about scale options.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where can I find a list of the &lt;code&gt;scale&lt;/code&gt; are available and how to call them?&lt;/p&gt;&#xA;" OwnerUserId="747" LastEditorUserId="3848" LastEditDate="2020-04-28T20:33:36.503" LastActivityDate="2020-04-28T20:33:36.503" Title="How to find which scales are available in SeqUtil of Biopython?" Tags="|phylogenetics|python|phylogeny|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3501" PostTypeId="1" AcceptedAnswerId="3508" CreationDate="2018-02-09T13:40:15.697" Score="4" ViewCount="289" Body="&lt;p&gt;My aim is to parse an experimental transcript set (obtained by RNAseq) to check which splice junctions are already reported in a reference annotation and which ones are new.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried &lt;code&gt;Regtools&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;feed the experimental alignment file (BAM: aligned transcripts to reference genome) to &lt;a href=&quot;https://regtools.readthedocs.io/en/latest/commands/junctions-extract/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Regtools junctions extract&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;use &lt;a href=&quot;https://regtools.readthedocs.io/en/latest/commands/junctions-annotate/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Regtools junctions annotate&lt;/a&gt; to compare with the reference annotation (GTF)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Exact commands I used for this are: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;[..]/regtools/build/regtools junctions extract aln_sorted.bam -o regtools.bed&lt;/code&gt; - where the sorted BAM file has index in the same folder&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;[..]/regtools/build/regtools junctions annotate regtools.bed [..]/ref/genome.fa [..]/ref/annotation.gtf -o regtools_annotate_out&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The alignments have been generated with &lt;a href=&quot;https://github.com/lh3/minimap2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Minimap2&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It did not behave as expected: this gives me as output a file with only junctions annotated as 'new' (&lt;code&gt;$11==N&lt;/code&gt;), while other annotations should be possible according to the &lt;a href=&quot;https://regtools.readthedocs.io/en/latest/commands/junctions-annotate/&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;DA - This exon-exon junction is present in the transcriptome provided by the user(GTF) The ends of this junction are hence known donor and known acceptor sites according to the GTF file.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;NDA - This exon-exon junction is not present(novel) in the transcriptome provided by the user(GTF) The ends of this junction are&#xA;   known donor and known acceptor sites according to the GTF file.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;D - This exon-exon junction is not present(novel) in the transcriptome provided by the user(GTF) The donor of this junction is a known donor but the acceptor is novel.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;A - This exon-exon junction is not present(novel) in the transcriptome provided by the user(GTF) The acceptor of this junction is a known acceptor but the donor is novel.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;N - This exon-exon junction is not present(novel) in the transcriptome provided by the user(GTF) The ends of this junction are hence not known donor/acceptor sites according to the GTF file.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My questions are: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;am I doing something wrong?&lt;/li&gt;&#xA;&lt;li&gt;do you have any experience with &lt;code&gt;Regtools&lt;/code&gt; and can help debugging this behavior?&lt;/li&gt;&#xA;&lt;li&gt;do you have other suggestion to achieve the output I am seeking?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="294" LastEditorUserId="294" LastEditDate="2018-02-09T15:56:02.433" LastActivityDate="2018-02-12T14:52:21.027" Title="Exon-exon junctions: compare experimental transcripts to reference annotation" Tags="|rna-seq|software-recommendation|rna-splicing|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3502" PostTypeId="2" ParentId="3492" CreationDate="2018-02-09T14:13:31.867" Score="1" Body="&lt;p&gt;a one-liner using &lt;strong&gt;bioalcidaejdk&lt;/strong&gt;: &lt;a href=&quot;http://lindenb.github.io/jvarkit/BioAlcidaeJdk.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://lindenb.github.io/jvarkit/BioAlcidaeJdk.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ java -jar dist/bioalcidaejdk.jar -e 'stream().filter(R-&amp;gt;!R.getReadUnmappedFlag() &amp;amp;&amp;amp; R.getCigar().getCigarElements().stream().anyMatch(C-&amp;gt;C.getLength()&amp;gt;100 &amp;amp;&amp;amp; (C.getOperator().equals(CigarOperator.N) || C.getOperator().equals(CigarOperator.D)))).forEach(R-&amp;gt;println(R.getContig()+&quot;\t&quot;+R.getStart()+&quot;\t&quot;+R.getEnd()));' input.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastActivityDate="2018-02-09T14:13:31.867" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3503" PostTypeId="1" AcceptedAnswerId="3505" CreationDate="2018-02-09T15:02:47.843" Score="2" ViewCount="1295" Body="&lt;p&gt;What to do with the configuration test cell after configuring the MinION with it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I received a MinION with a configuration test cell in it. After running a configuration in the MinKNOW software while the CTC was in the MinION, I wonder what to do with the CTC? Am I going to need it later or can I throw it away? If I should keep it, in what conditions should I keep it (temperature, light, etc.)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, do you know why we need to do a configuration of the MinION the first time?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I posted this question on &lt;a href=&quot;https://biology.stackexchange.com/questions/70319/what-to-do-with-the-configuration-test-cell-after-configuring-the-minion-with-it&quot;&gt;Biology Stackexchange&lt;/a&gt; as well but maybe the community here will be more receptive?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-02-09T16:42:37.493" Title="What to do with the configuration test cell after configuring the MinION with it?" Tags="|nanopore|ngs|long-reads|minion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3504" PostTypeId="2" ParentId="3251" CreationDate="2018-02-09T16:15:01.133" Score="1" Body="&lt;p&gt;According to &lt;a href=&quot;http://pacbiofileformats.readthedocs.io/en/5.0/BAM.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PacBio Sequel BAM format manual&lt;/a&gt;, you can find information on the &lt;a href=&quot;https://www.pacb.com/blog/new-chemistry-software-sequel-system-improve-read-length-lower-project-costs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;chemistry version&lt;/a&gt; in the header entries &lt;code&gt;@RG&lt;/code&gt; (read group), tag &lt;code&gt;DS&lt;/code&gt; (description), entry &lt;code&gt;BASECALLERVERSION&lt;/code&gt; . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case of future very PacBio-specific questions, you can contact Liz Tseng on Twitter (@magdoll), one of the most responsive PacBio developers. They also have a &lt;a href=&quot;http://seqanswers.com/forums/forumdisplay.php?f=39&quot; rel=&quot;nofollow noreferrer&quot;&gt;dedicated SeqAnswers forum&lt;/a&gt;, which is directly monitored by them and you are very likely to get a fast and accurate answer. &lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2018-02-09T16:15:01.133" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3505" PostTypeId="2" ParentId="3503" CreationDate="2018-02-09T16:42:37.493" Score="2" Body="&lt;p&gt;I would keep it, in case you need to do another configuration later on. I have used mine for that purpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think storage doesn't really matter, I just put it in a drawer somewhere. Keep it away from liquids and extreme temperatures :)&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2018-02-09T16:42:37.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3506" PostTypeId="1" AcceptedAnswerId="3507" CreationDate="2018-02-09T17:52:45.343" Score="2" ViewCount="320" Body="&lt;p&gt;Also posted &lt;a href=&quot;https://www.biostars.org/p/298033/&quot; rel=&quot;nofollow noreferrer&quot;&gt;on biostars&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After indexing&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bwa index -a bwtsw reference.fa &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I got files. like .bwt file and .sa file. The naive way of generating these file is:&#xA;  &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Create all rotations of T&lt;/li&gt;&#xA;&lt;li&gt;Sort the rotations lexicographically  &lt;/li&gt;&#xA;&lt;li&gt;Concatenate the last character of each rotation to form &#xA; BWT(T)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I used this naive way in my algorithm. The problem is it took too much memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: is there an effective memory way to generate .bwt file and .sa file ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help will be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2322" LastEditorUserId="2322" LastEditDate="2018-02-10T09:27:21.707" LastActivityDate="2018-02-10T09:27:21.707" Title="Generating burrows-wheeler-transform and a suffix array of a DNA sequence with less memory" Tags="|sequence-alignment|bwa|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3507" PostTypeId="2" ParentId="3506" CreationDate="2018-02-09T19:53:46.443" Score="4" Body="&lt;p&gt;No, there is no way to decompress .bwt or .sa files, because they are not compressed. After indexing you'll get these files, which some programs (bwa) use. They should be in the same folder as your reference fasta file. What are you trying to do with these files? To get proper help here, please explain why you need these files, and show how you made the files.&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2018-02-09T20:05:30.587" LastActivityDate="2018-02-09T20:05:30.587" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3508" PostTypeId="2" ParentId="3501" CreationDate="2018-02-10T00:10:54.673" Score="4" Body="&lt;p&gt;You can use &lt;a href=&quot;https://github.com/lh3/minimap2/blob/master/misc/paftools.js&quot; rel=&quot;nofollow noreferrer&quot;&gt;the script&lt;/a&gt; that comes with minimap2:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;# install the k8 javascript shell&#xA;curl -L https://github.com/attractivechaos/k8/releases/download/v0.2.4/k8-0.2.4.tar.bz2 | tar -jxf -&#xA;cp k8-0.2.4/k8-`uname -s` $HOME/bin/k8    # assuming $HOME/bin in your $PATH&#xA;&#xA;# acquire the latest minimap2&#xA;git clone https://github.com/lh3/minimap2.git&#xA;&#xA;# print novel/erroneous junctions&#xA;minimap2/misc/paftools.js junceval -e anno.gtf rna-seq.sam &amp;gt; output.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Some example output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;P       read1    4       chr2    227332588       227340301       [(227332588,227340291)]&#xA;P       read2    1       chr9    19376379        19376489        [(19376388,19376493)]&#xA;P       read2    3       chr9    19378506        19378707        [(19378514,19378707)]&#xA;N       read3    1       chr1    160647335       160647472&#xA;N       read3    2       chr1    160647581       160659672&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here column 4–6 gives junctions not in the annotation. Column 7, if not empty, gives the list of annotated junctions/introns that overlap the junction in read alignment. Note that a big fraction of outputted records are errors in read alignment.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2018-02-12T14:52:21.027" LastActivityDate="2018-02-12T14:52:21.027" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3509" PostTypeId="1" AcceptedAnswerId="3531" CreationDate="2018-02-10T05:29:46.147" Score="2" ViewCount="922" Body="&lt;p&gt;I'm trying to normalize the coverage signals of ATAC-seq reads against its own background using normal distribution, described in this &lt;a href=&quot;https://elifesciences.org/articles/20148&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It says: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Finally, all open chromatin coverage measurements were normalized by&#xA;  standardization to the mean and standard deviation of coverage over a&#xA;  set of 25,000 randomly selected background regions. To select&#xA;  background regions, the set of peak open regions were widened to&#xA;  20,000 bp, reduced, and subtracted from the genome assembly.&#xA;  Thereafter, 25,000 random positions were selected and widened to&#xA;  reflect the distribution of widths in the set of open peaks. Coverage&#xA;  within these background regions was then calculated, and regions with&#xA;  zero coverage were discarded (~5%). The distribution of counts within&#xA;  background regions approximated a log-normal distribution. Mean and&#xA;  standard deviation of these background regions was calculated and used&#xA;  to transform the coverage measurements for the entire genome.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Does anyone know any existing codes doing that? I know Anshul's &lt;a href=&quot;https://github.com/kundajelab/atac_dnase_pipelines&quot; rel=&quot;nofollow noreferrer&quot;&gt;pipeline&lt;/a&gt; does that but can't find the specific script for that step. &lt;/p&gt;&#xA;" OwnerUserId="1852" LastEditorUserId="939" LastEditDate="2018-02-11T15:50:10.650" LastActivityDate="2018-02-13T17:40:22.193" Title="Normalize ATAC-seq/Dnase-seq sequencing reads coverage signals over estimated background" Tags="|normalization|atac-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3510" PostTypeId="1" AcceptedAnswerId="3512" CreationDate="2018-02-10T10:56:33.993" Score="1" ViewCount="598" Body="&lt;p&gt;I need to know how BWA generate bw and sa in less memory usage. I didn't find details in the &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/25/14/1754/225615&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt;. Do they save all rotations temporary ? I need to know do BWA stores all rotations so that it can sort the rotations alphabetically ?&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="2322" LastEditorUserId="73" LastEditDate="2022-07-21T22:26:40.067" LastActivityDate="2022-07-21T22:26:40.067" Title="How BWA generate index files?" Tags="|sequence-alignment|bwa|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3512" PostTypeId="2" ParentId="3510" CreationDate="2018-02-11T18:19:18.043" Score="4" Body="&lt;h2&gt;Papers&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The space-efficient BWT construction algorithm is in &lt;a href=&quot;https://github.com/lh3/bwa/blob/master/bwt_gen.c&quot; rel=&quot;nofollow noreferrer&quot;&gt;bwt_gen.c&lt;/a&gt;. One of the first things you see there is the copyright line:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Copyright (C) 2004, Wong Chi Kwong.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;With that name, you can find the relevant papers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Lam et al: &lt;a href=&quot;https://doi.org/10.1093/bioinformatics/btn032&quot; rel=&quot;nofollow noreferrer&quot;&gt;Compressed indexing and local alignment of DNA&lt;/a&gt; (Bioinformatics, 2008). This paper describes the BWT-SW alignment algorithm.&lt;/li&gt;&#xA;&lt;li&gt;Hon et al: &lt;a href=&quot;http://www.siam.org/meetings/alenex04/abstacts/WHon.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Practical Aspects of Compressed Suffix Arrays and FM-index in Searching DNA Sequences&lt;/a&gt; (ALENEX 2004). This was the first paper to demonstrate that it's feasible to build the FM-index for a human genome.&lt;/li&gt;&#xA;&lt;li&gt;Hon et al: &lt;a href=&quot;https://doi.org/10.1007/s00453-006-1228-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Space and Time Efficient Algorithm for Constructing Compressed Suffix Arrays&lt;/a&gt; (Algorithmica, 2007). This paper contains the theoretical description of the construction algorithm.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The third paper only mentions BWT briefly in the last section. It deals with compressed suffix arrays, which are space-efficient representations of the inverse function of LF-mapping. But if you have a representation of (the inverse of) the LF-mapping, you have a representation of the BWT. Hence the algorithm can be adapted for BWT construction.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Algorithm&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;When building BWT, you usually append a unique character &lt;code&gt;$&lt;/code&gt; to the end of the text, either implicitly or explicitly. This way you can reduce sorting rotations to sorting suffixes. You sort the suffixes in lexicographic order, and then you output the preceding character for each suffix in that order to form the BWT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that you already have the BWT for string &lt;code&gt;X&lt;/code&gt;, and you want to transform it into the BWT of string &lt;code&gt;cX&lt;/code&gt;, where &lt;code&gt;c&lt;/code&gt; is a single character. You can do that in three steps (this assumes 0-based array indexing):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Find the position &lt;code&gt;i&lt;/code&gt; of the full suffix &lt;code&gt;X&lt;/code&gt;, and replace the &lt;code&gt;$&lt;/code&gt; in the BWT with the inserted character &lt;code&gt;c&lt;/code&gt; (set &lt;code&gt;BWT[i] = c&lt;/code&gt;).&lt;/li&gt;&#xA;&lt;li&gt;Determine the number of suffixes &lt;code&gt;j&lt;/code&gt; that are lexicographically smaller than &lt;code&gt;cX&lt;/code&gt; with &lt;code&gt;j = LF(i, c) = C[c] + BWT.rank(i, c)&lt;/code&gt;. Here &lt;code&gt;C[c]&lt;/code&gt; is the number of suffixes that start with a character smaller than &lt;code&gt;c&lt;/code&gt;, and &lt;code&gt;BWT.rank(i, c)&lt;/code&gt; is the number of occurrences of &lt;code&gt;c&lt;/code&gt; in the prefix &lt;code&gt;BWT[0..i-1]&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Insert &lt;code&gt;$&lt;/code&gt; between &lt;code&gt;BWT[j-1]&lt;/code&gt; and &lt;code&gt;BWT[j]&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;By iterating this, you can build the BWT for any string slowly but space-efficiently. The algorithm of Hon et al. is based on the same idea, but they make it faster by inserting multiple characters at the same time. Given the BWT of string &lt;code&gt;X&lt;/code&gt; and another string &lt;code&gt;Y&lt;/code&gt;, they produce the BWT of the concatenation &lt;code&gt;YX&lt;/code&gt;. The details are quite complicated, but you can find them in the third paper and in the source code.&lt;/p&gt;&#xA;" OwnerUserId="2330" LastActivityDate="2018-02-11T18:19:18.043" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3513" PostTypeId="1" AcceptedAnswerId="3518" CreationDate="2018-02-11T23:15:17.050" Score="3" ViewCount="87" Body="&lt;p&gt;So if I attempt this question using their method I put the data of genotype and phenotype into a data frame in R then use &lt;code&gt;lm()&lt;/code&gt; to do linear regression and &lt;code&gt;summary()&lt;/code&gt; to get an F statistic with p value 0.54. I'm not connecting what the F statistic means in this case to the answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question 2: Assuming that we have the following SNP and phenotype data, is the SNP significantly associated with the phenotype? Here, we represent each SNP site as the number of minor alleles on that locus, so 0 and 2 are for major and minor homozygous sites, respectively, and 1 is for the heterozygous sites. We also assume that minor alleles contribute to the phenotype and the effect is additive. In other words, the effect from a minor homozygous site should be twice as large as that from a heterozygous site. You may use any test methods introduced in the chapter. How about permutation tests?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;GENOTYPE: 1, 2, 2, 1, 1, 0, 2, 0, 1, 0&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;PHENOTYPE: 0.53, 0.78, 0.81, -0.23, -0.73, 0.81, 0.27, 2.59, 1.84, 0.03&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Answer: Take the linear regression with F-test as an example, the p-value without the permutation test is 0.54 and is 0.55 after 1000 permutation (this number could be different with your answer because of the randomness of the permutation test).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Link to where I'm getting this: &lt;a href=&quot;http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002828#pcbi.1002828-Balding1&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002828#pcbi.1002828-Balding1&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="2332" LastEditorUserId="57" LastEditDate="2018-02-12T11:43:28.693" LastActivityDate="2018-02-12T11:43:28.693" Title="Assuming that we have the following SNP and phenotype data, is the SNP significantly associated with the phenotype?" Tags="|snp|gwas|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3514" PostTypeId="1" AcceptedAnswerId="3520" CreationDate="2018-02-12T04:03:23.460" Score="2" ViewCount="96" Body="&lt;p&gt;I have file called pangenome.txt which has only one column containing the names of the genes&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;head pangenome.txt&#xA;    Ha1_00044161&#xA;    Ha11_00031248&#xA;    Ha16_00021664&#xA;    Ha8_00042762&#xA;    Ha4_00025807&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have another bunch of files in a tabular blast format (330 in total: individual1.blast.txt,individual2.blast.txt, ...,individual330.blast.txt ) which are the outputs of Diamond protein aligner. Below is part of individual1.blast.txt file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;head blast.individual1.txt&#xA;    HWI-ST913:300:C5W5DACXX:7:1101:1477:2147    Ha16_00021664   77.4    31  7   0   94  2   49  79  7.3e-159.3&#xA;    HWI-ST913:300:C5W5DACXX:7:1101:1477:2147    Ha8_00042762    74.2    31  8   0   94  2   1   31  9.6e-158.9&#xA;    HWI-ST913:300:C5W5DACXX:7:1101:1477:2147    Ha4_00025807    77.4    31  7   0   94  2   1   31  1.3e-058.5&#xA;    HWI-ST913:300:C5W5DACXX:7:1101:1477:2147    Ha16_00022939   77.4    31  7   0   94  2   295 325 2.1e-057.8&#xA;    HWI-ST913:300:C5W5DACXX:7:1101:1477:2147    Ha12_00032339   71.0    31  9   0   94  2   2   32  2.8e-057.4&#xA;    HWI-ST913:300:C5W5DACXX:7:1101:1477:2147    Ha16_00021553   71.0    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to know which genes from the pangemome.txt is present in my individuals file and which are absent.&#xA;for the example above this is my desired output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;head output.individual1.txt&#xA;    Ha1_00044161   0&#xA;    Ha11_00031248  0&#xA;    Ha16_00021664  1&#xA;    Ha8_00042762   1&#xA;    Ha4_00025807   1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My desired out put is a file with 2 columns (tabular format), the first column the list of the all genes form pangemome.txt pan, and the second column is the presence and absence of these genes in the individuals files if the gene is present present in the blast.individual1.txt, I want to assign 1 and if it is absent I want to assign 0. &#xA;Can anyone help me to do this?&#xA;Thanks a lot&lt;/p&gt;&#xA;" OwnerUserId="1749" LastEditorUserId="1749" LastEditDate="2018-02-12T15:11:16.453" LastActivityDate="2018-02-12T16:08:52.723" Title="assigning different characters to present and absent genes from blast output using perl or python" Tags="|python|blast|perl|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3515" PostTypeId="2" ParentId="3510" CreationDate="2018-02-12T04:24:49.027" Score="3" Body="&lt;p&gt;The following is only a comment to @JouniS' answer, but it is too long to fit in 512 characters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The algorithm @JouniS was describing has been rediscovered multiple times. The C program below constructs BWT with this algorithm. It is proof-of-concept only, not intended for long strings.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;string.h&amp;gt;&#xA;#include &amp;lt;stdio.h&amp;gt;&#xA;int str2bwt(char *s)&#xA;{&#xA;    int i, l, c = 0, k = 0;&#xA;    l = strlen(s); // length of input string&#xA;    for (i = l - 1; i &amp;gt;= 0; --i) {&#xA;        int j, r = 0, a = s[i];&#xA;        memmove(&amp;amp;s[i], &amp;amp;s[i+1], k); // make room for insertion&#xA;        s[i+k] = a; // insert the char&#xA;        for (j = i; j &amp;lt; i + k; ++j) r += (s[j] &amp;lt;= a);&#xA;        for (; j &amp;lt; l; ++j) r += (s[j] &amp;lt; a);&#xA;        k = r + 1, c = a;&#xA;    }&#xA;    return k; // the position of sentinel '$'&#xA;}&#xA;int main(int argc, char *argv[])&#xA;{&#xA;    int k;&#xA;    if (argc == 1) return 1;&#xA;    k = str2bwt(argv[1]);&#xA;    fwrite(argv[1], 1, k, stdout); putchar('$'); puts(&amp;amp;argv[1][k]);&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;PS: this is one of the simplest ways to construct BWT and does not require any additional dependencies. It is slow for long strings, though.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-02-12T04:24:49.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3516" PostTypeId="1" AcceptedAnswerId="3537" CreationDate="2018-02-12T04:44:03.710" Score="3" ViewCount="458" Body="&lt;p&gt;In BWA project in bwt.c file I found bwt_dump_bwt method and bwt_dump_sa method. I want to use them to read .bwt file and .sa file. I wrote the following program:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int main(void)&#xA;{&#xA;  char *tmp, *prefix;&#xA;    bwt_t *bwt;&#xA;    prefix = bwa_idx_infer_prefix(&quot;hg19bwaidx&quot;);&#xA;&#xA;    tmp = calloc(strlen(prefix) + 5, 1);&#xA;    strcat(strcpy(tmp, prefix), &quot;.bwt&quot;);&#xA;    bwt = bwt_restore_bwt(tmp);&#xA;&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I got the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Undefined symbols for architecture x86_64:&#xA;  &quot;_bwa_idx_infer_prefix&quot;, referenced from:&#xA;      _main in readbwt-3022ec.o&#xA;  &quot;_bwt_restore_bwt&quot;, referenced from:&#xA;      _main in readbwt-3022ec.o&#xA;ld: symbol(s) not found for architecture x86_64&#xA;clang: error: linker command failed with exit code 1 (use -v to see invocation)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any help ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="2322" LastEditorUserId="77" LastEditDate="2018-02-12T13:07:48.053" LastActivityDate="2018-02-14T02:12:31.080" Title="How to read .bwt file .sa file?" Tags="|sequence-alignment|bwa|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3517" PostTypeId="2" ParentId="3516" CreationDate="2018-02-12T08:07:16.667" Score="4" Body="&lt;p&gt;You need to include &lt;code&gt;bwa.o&lt;/code&gt; in your call to &lt;code&gt;clang&lt;/code&gt;, since it holds the definitions of those functions. This will cause a good number of object files, so you might as well just &lt;code&gt;clang -o foo *.o foo.c -lz -lpthread -lm -lrt&lt;/code&gt; and get it over with. Since this was quite obviously never meant to be done, you'll have to remove at least one &lt;code&gt;main()&lt;/code&gt; function from &lt;code&gt;bwa.c&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that further questions regarding this will be considered off-topic and closed, since it's purely about compiling something that you want to write yourself.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-02-12T08:07:16.667" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3518" PostTypeId="2" ParentId="3513" CreationDate="2018-02-12T09:23:59.950" Score="2" Body="&lt;p&gt;Your approach is correct - it's not the only one permitted under their conditions, but it would be the standard way given only that information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;F-test compares a &quot;full model&quot; (predicting the phenotype using genotype and an intercept) versus a &quot;reduced model&quot; (predicting the phenotype using only the intercept, i.e. the mean value of phenotype). The p-value tells whether the full model predicts better, i.e. whether adding the genotype variable significantly improves the prediction. In this single-variable case, it should be approximately equal to the p-value for the beta coefficient of the genotype, if that's more familiar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, this isn't specific to genetics, so you can check out other online sources - e.g. &lt;a href=&quot;https://onlinecourses.science.psu.edu/stat501/node/295&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://onlinecourses.science.psu.edu/stat501/node/295&lt;/a&gt; for a longer, but accessible explanation of the F-test, and &lt;a href=&quot;https://stats.stackexchange.com/&quot;&gt;https://stats.stackexchange.com/&lt;/a&gt; for many answers to similar questions.&lt;/p&gt;&#xA;" OwnerUserId="920" LastActivityDate="2018-02-12T09:23:59.950" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3519" PostTypeId="2" ParentId="3514" CreationDate="2018-02-12T10:41:18.547" Score="2" Body="&lt;p&gt;Using &lt;code&gt;python&lt;/code&gt; it should be relatively trivial, using &lt;code&gt;sets&lt;/code&gt;. The following snippet works with your pangenome file and one blast file; adding more blast files should be trivial, making use of the &lt;a href=&quot;https://docs.python.org/3.6/library/stdtypes.html#frozenset.union&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;union&lt;/code&gt;&lt;/a&gt; method of &lt;code&gt;python&lt;/code&gt;'s &lt;code&gt;sets&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pangenome = {x.rstrip() for x in open('pangenome.txt')}&#xA;blast = {x.rstrip().split()[1] for x in open('blast.txt')}&#xA;for x in pangenome:&#xA;    if x in blast:&#xA;        print('%s\t1' % x)&#xA;    else:&#xA;        print('%s\t0' % x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="123" LastActivityDate="2018-02-12T10:41:18.547" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3520" PostTypeId="2" ParentId="3514" CreationDate="2018-02-12T12:45:14.613" Score="3" Body="&lt;p&gt;I wouldn't do this in Python, myself. This is a very simple text parsing problem and the standard *nix tools will be able to do it very easily. For example with &lt;code&gt;awk&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk 'NR==FNR{a[$1]=0; next}&#xA;       { &#xA;        if($2 in a){&#xA;            a[$2]=1&#xA;        }&#xA;       }END{&#xA;        for(i in a){&#xA;            print i,a[i]&#xA;        }&#xA;       }' pangenome.txt blast.individual1.txt &#xA;Ha1_00044161 0&#xA;Ha4_00025807 1&#xA;Ha11_00031248 0&#xA;Ha16_00021664 1&#xA;Ha8_00042762 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above is just a one-liner, I expanded it for clarity. You can run it on as many files as you like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk 'NR==FNR{a[$1]=0; next}{if($2 in a){a[$2]=1}}END{for(i in a){print i,a[i]}}' pangenome.txt blast.individual*.txt &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For many large files, the fastest approach would probably be to just grep the names (unlike the awk, this will also match if the target gene is in the 1st field, but that's probably not an issue for you):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -wof pangenome.txt blast.individual1.txt.orig &#xA;Ha16_00021664&#xA;Ha8_00042762&#xA;Ha4_00025807&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That will give you the list of genes found in the target blast files. To get the 1 and 0, you can do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sort &amp;lt;(grep -owf pangenome.txt blast.individual1.txt) pangenome.txt | &#xA;    uniq -c | awk '{print $2,$1-1}'&#xA;Ha1_00044161 0&#xA;Ha11_00031248 0&#xA;Ha16_00021664 1&#xA;Ha4_00025807 1&#xA;Ha8_00042762 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For multiple files, just run it on all of them:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sort &amp;lt;(grep -owf pangenome.txt blast.individual*.txt) pangenome.txt | &#xA;    uniq -c | awk '{print $2,$1-1}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-02-12T12:45:14.613" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3521" PostTypeId="2" ParentId="3501" CreationDate="2018-02-12T13:39:25.533" Score="1" Body="&lt;p&gt;For future googlers, Ding et al., 2017 published a &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5476949/&quot; rel=&quot;nofollow noreferrer&quot;&gt;review&lt;/a&gt;: &quot;Comparison of Alternative Splicing Junction Detection Tools Using RNA-Seq Data&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5476949/table/T1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Table 1&lt;/a&gt; has the list of software in this review. &lt;/p&gt;&#xA;" OwnerUserId="294" LastEditorUserId="294" LastEditDate="2018-02-12T14:28:27.553" LastActivityDate="2018-02-12T14:28:27.553" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3522" PostTypeId="2" ParentId="3514" CreationDate="2018-02-12T16:08:52.723" Score="2" Body="&lt;p&gt;Here is a way to do it using a python dictionary:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;import sys&#xA;&#xA;gene_filename = sys.argv[1]&#xA;blast_filename = sys.argv[2]&#xA;&#xA;# Set to 0 all genes in the gene file&#xA;genes = {}&#xA;with open(gene_filename, &quot;r&quot;) as gene_file:&#xA;    for line in gene_file:&#xA;        genes[line.strip()] = 0&#xA;&#xA;# Set to 1 those genes found in the blast file&#xA;with open(blast_filename, &quot;r&quot;) as blast_file:&#xA;    for line in blast_file:&#xA;        gene = line.strip().split()[1]&#xA;        if gene in genes:&#xA;            genes[gene] = 1&#xA;&#xA;# Print the results&#xA;for (gene, presence) in genes.items():&#xA;    print(gene, presence, sep=&quot;\t&quot;)&#xA;&#xA;exit(0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Executing it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ./find_genes_in_blast.py pangenome.txt blast.individual1.txt &#xA;Ha1_00044161    0&#xA;Ha11_00031248   0&#xA;Ha16_00021664   1&#xA;Ha8_00042762    1&#xA;Ha4_00025807    1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="292" LastActivityDate="2018-02-12T16:08:52.723" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3523" PostTypeId="1" AcceptedAnswerId="3524" CreationDate="2018-02-12T16:41:54.893" Score="2" ViewCount="559" Body="&lt;p&gt;I have a bed file where some entries have the same name in column 4 but are book-ended in coordinates, e.g. see below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    0       10000   LOC101929784::chr1:13201-13800::E1&#xA;chr1    10000   10600   LOC101929784::chr1:13201-13800::XYZ&#xA;chr1    10600   10800   LOC101929784::chr1:13201-13800::E1&#xA;chr1    10800   12000   DDX11L1::chr1:11868-14409.E1&#xA;chr1    12000   12200   DDX11L1::chr1:11868-14409.E1&#xA;chr1    12200   13000   DDX11L1::chr1:11868-14409.E1&#xA;chr1    13000   13200   DDX11L1::chr1:11868-14409.E1&#xA;chr1    13200   15800   WASH7P::chr1:14403-29570.E1&#xA;chr1    15800   16000   WASH7P::chr1:14403-29570.E1&#xA;chr1    16000   18200   WASH7P::chr1:14403-29570.E1&#xA;chr1    18200   18400   WASH7P::chr1:14403-29570.E1&#xA;chr1    18400   23000   WASH7P::chr1:14403-29570.E1&#xA;chr1    23000   23400   WASH7P::chr1:14403-29570.E1&#xA;chr1    23400   24000   WASH7P::chr1:14403-29570.E1&#xA;chr1    24000   24200   WASH7P::chr1:14403-29570.E1&#xA;chr1    24200   25200   WASH7P::chr1:14403-29570.E1&#xA;chr1    25200   25800   WASH7P::chr1:14403-29570.E1&#xA;chr1    25800   26200   WASH7P::chr1:14403-29570.E1&#xA;chr1    26200   26400   WASH7P::chr1:14403-29570.E1&#xA;chr1    26400   27000   WASH7P::chr1:14403-29570.E1&#xA;chr1    27000   27600   WASH7P::chr1:14403-29570.E1&#xA;chr1    27600   27800   WASH7P::chr1:14403-29570.E1&#xA;chr1    27800   29600   WASH7P::chr1:14403-29570.E1&#xA;chr1    29600   30800   MIR1302-2HG::chr1:29553-31109.E1&#xA;chr1    30800   31400   MIR1302-2HG::chr1:29553-31109.E1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to apply an operation that seems to me is a mix of &lt;code&gt;bedtools merge -d 0&lt;/code&gt; and &lt;code&gt;bedtools groupby&lt;/code&gt;, but I can't figure out what's the combination.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the 4 entries with &lt;code&gt;DDX11L1::chr1:11868-14409.E1&lt;/code&gt; I would like merged as they are book-ended, same with the &lt;code&gt;WASH7P::chr1:14403-29570.E1&lt;/code&gt; entries and the &lt;code&gt;MIR1302-2HG::chr1:29553-31109.E1&lt;/code&gt; entries. There are two &lt;code&gt;LOC101929784::chr1:13201-13800::E1&lt;/code&gt; entries, but I don't want to merge them because they are not book-ended, since they are separated by &lt;code&gt;LOC101929784::chr1:13201-13800::XYZ&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2018-04-20T21:06:16.953" Title="bed file merge book-end features only if same name in column 4" Tags="|bed|bedtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3524" PostTypeId="2" ParentId="3523" CreationDate="2018-02-12T17:04:57.130" Score="5" Body="&lt;pre&gt;&lt;code&gt;awk 'BEGIN{OFS=&quot;\t&quot;}{if($1==lchrom &amp;amp;&amp;amp; $4==lname &amp;amp;&amp;amp; $2 == lend) {lend = $3}else{if(lchrom) {print lchrom, lstart, lend, lname;}; lchrom=$1; lstart=$2; lend=$3; lname=$4}}END{print lchrom, lstart, lend, lname}' foo.bed &amp;gt; new.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That produces:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    0   10000   LOC101929784::chr1:13201-13800::E1&#xA;chr1    10000   10600   LOC101929784::chr1:13201-13800::XYZ&#xA;chr1    10600   10800   LOC101929784::chr1:13201-13800::E1&#xA;chr1    10800   13200   DDX11L1::chr1:11868-14409.E1&#xA;chr1    13200   29600   WASH7P::chr1:14403-29570.E1&#xA;chr1    29600   31400   MIR1302-2HG::chr1:29553-31109.E1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This explicitly checks for bookends and nothing else and assumes that everything is sorted.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-02-12T17:04:57.130" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3525" PostTypeId="1" CreationDate="2018-02-12T20:34:18.410" Score="3" ViewCount="210" Body="&lt;p&gt;I have five species trees run using starBeast and Beauti. Here is a link to the paper discussing both, &#xA;&lt;a href=&quot;http://journals.plos.org/ploscompbiol/article/file?type=supplementary&amp;amp;id=info:doi/10.1371/journal.pcbi.1003537.s006&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://journals.plos.org/ploscompbiol/article/file?type=supplementary&amp;amp;id=info:doi/10.1371/journal.pcbi.1003537.s006&lt;/a&gt;&#xA; Each of these trees has the same individuals, and each locus used (10 loci per tree) has individual models set to it. All the trees used a relaxed log clock normal. I would like to be able to combine these trees into one. I have attempted to find a way to do this within Densitree and starBeast, but to no avail. When all loci are run with all trees, the program simply dies, leading me to believe that running the trees together will all parameters is computationally impossible for my hardware. Is there a way to combine these trees?&lt;/p&gt;&#xA;" OwnerUserId="2338" LastEditorUserId="2338" LastEditDate="2018-02-12T21:46:46.343" LastActivityDate="2018-02-14T10:41:15.887" Title="Combine multiple species trees" Tags="|gene|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3526" PostTypeId="1" AcceptedAnswerId="3529" CreationDate="2018-02-13T09:45:04.603" Score="7" ViewCount="1740" Body="&lt;p&gt;How do I obtain a bed file with the list of N-regions of GRCh38 reference? This is, the regions where the sequence is a stretch of Ns.&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2019-11-01T07:46:25.240" Title="bed file with N regions of GRCh38 reference?" Tags="|genome|bed|" AnswerCount="6" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3527" PostTypeId="2" ParentId="3526" CreationDate="2018-02-13T10:53:30.140" Score="1" Body="&lt;p&gt;Here's a way to generate it yourself from the genome sequence. First, convert the fasta file of the genome to tbl format (&lt;code&gt;&amp;lt;seq id&amp;gt;\t&amp;lt;sequence&amp;gt;)&lt;/code&gt;, then use perl to find the start and end positions of all stretches of consecutive of &lt;code&gt;N&lt;/code&gt; or &lt;code&gt;n&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FastaToTbl hg38.fa | &#xA;    perl -F&quot;\t&quot; -ane 'while(/N+/ig){&#xA;                            for(0..$#-){&#xA;                                print &quot;$F[0]\t$-[$_]\t$+[$_]\n&quot;&#xA;                            }&#xA;                      }' &amp;gt; hg38.n.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Explanation&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;FastaToTbl&lt;/code&gt; : this is a very simple script that converts fasta to tbl. Just save the lines below somewhere in your &lt;code&gt;$PATH&lt;/code&gt; (e.g. &lt;code&gt;~/bin&lt;/code&gt;) as &lt;code&gt;FastaToTbl&lt;/code&gt; and make it executable. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/awk -f&#xA;{&#xA;    if (substr($1,1,1)==&quot;&amp;gt;&quot;)&#xA;            if (NR&amp;gt;1)&#xA;                printf &quot;\n%s\t&quot;, substr($0,2,length($0)-1)&#xA;            else &#xA;                printf &quot;%s\t&quot;, substr($0,2,length($0)-1)&#xA;        else &#xA;            printf &quot;%s&quot;, $0&#xA;}&#xA;END{printf &quot;\n&quot;}&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The Perl magic. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;code&gt;-a&lt;/code&gt; makes &lt;code&gt;perl&lt;/code&gt; behave like &lt;code&gt;awk&lt;/code&gt;, splitting each input line on the value given by &lt;code&gt;-F&lt;/code&gt; (a tab in this case) and saving the result in the aray &lt;code&gt;@F&lt;/code&gt;. So, &lt;code&gt;$F[0]&lt;/code&gt; will be the sequence id. &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;while(/N+/ig){&lt;/code&gt; : this will match all cases of consecutive &lt;code&gt;N&lt;/code&gt; or &lt;code&gt;n&lt;/code&gt; (the &lt;code&gt;i&lt;/code&gt; flag makes it case-insensitive). Perl will store the start positions of all matches in the special array &lt;code&gt;@-&lt;/code&gt; and the end positions in &lt;code&gt;@+&lt;/code&gt;. &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;for(0..$#-){&lt;/code&gt; : iterate over all numbers from 0 to the final index (&lt;code&gt;$#-&lt;/code&gt;) in the array &lt;code&gt;@-&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;print &quot;$F[0]\t$-[$_]\t$+[$_]\n&quot;&lt;/code&gt; : print minimal bed format data. The name of the current sequence (&lt;code&gt;$F[0]&lt;/code&gt;), the start position of the current match (&lt;code&gt;$-[$_]&lt;/code&gt;) and the corresponding end position (&lt;code&gt;$+[$_]&lt;/code&gt;).&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I just ran the above on my system and it generated &lt;a href=&quot;https://pastebin.com/UbakGp9b&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-02-13T10:53:30.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3528" PostTypeId="2" ParentId="3526" CreationDate="2018-02-13T12:16:07.913" Score="4" Body="&lt;p&gt;This information is stored in the 2bit file representation of the sequence, so if you happen to have a 2bit file locally (or want to download one from UCSC) and have py2bit installed (you'll need version 3.0, since I literally just added support for this):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import py2bit&#xA;tb = py2bit.open(&quot;genome.2bit&quot;)&#xA;of = open(&quot;NNNNN.bed&quot;, &quot;w&quot;)  # Do change the input and output file names&#xA;for chrom in tb.chroms().keys():&#xA;    blocks = tb.hardMaskedBlocks(chrom)&#xA;    for block in blocks:&#xA;        of.write(&quot;{}\t{}\t{}\n&quot;.format(chrom, block[0], block[1]))&#xA;of.close()&#xA;tb.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In case it's helpful, you can also use this to get all soft-masked regions. Just replace &lt;code&gt;hardMaskedBlocks()&lt;/code&gt; with &lt;code&gt;softMaskedBlocks()&lt;/code&gt; and ensure to specify &lt;code&gt;storeMasked=True&lt;/code&gt; when you open the file.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-02-13T12:48:30.077" LastActivityDate="2018-02-13T12:48:30.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3529" PostTypeId="2" ParentId="3526" CreationDate="2018-02-13T14:25:04.800" Score="5" Body="&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;# if you have seqtk installed, skip the following two lines&#xA;git clone https://github.com/lh3/seqtk&#xA;cd seqtk &amp;amp;&amp;amp; make&#xA;# the main command line&#xA;./seqtk cutN -gp10000000 -n1 hg38.fa &amp;gt; hg38-N.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Option &lt;code&gt;-n&lt;/code&gt; sets the min stretch length. Just use &lt;code&gt;-p&lt;/code&gt; as is. It is a bit complicated to explain what it is doing.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-02-13T14:25:04.800" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3531" PostTypeId="2" ParentId="3509" CreationDate="2018-02-13T17:40:22.193" Score="2" Body="&lt;p&gt;Supplementary File &lt;a href=&quot;https://doi.org/10.7554/eLife.20148.041&quot; rel=&quot;nofollow noreferrer&quot;&gt;5&lt;/a&gt; and &lt;a href=&quot;https://doi.org/10.7554/eLife.20148.042&quot; rel=&quot;nofollow noreferrer&quot;&gt;6&lt;/a&gt; contain the code you're looking for. &lt;/p&gt;&#xA;" OwnerUserId="2346" LastActivityDate="2018-02-13T17:40:22.193" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3532" PostTypeId="2" ParentId="3211" CreationDate="2018-02-13T17:50:12.857" Score="10" Body="&lt;h1&gt;Expected rates of doublets / duplets / multiplets&lt;/h1&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://genomics.cruk.cam.ac.uk/services/Single-cell#section-2&quot; rel=&quot;noreferrer&quot;&gt;Fluidigm C1 doublet rate&lt;/a&gt;: around 1-5% depending on chip type used.&#xA;More information: &lt;em&gt;&lt;a href=&quot;http://info.fluidigm.com/rs/673-MRG-416/images/C1-Med-96-IFC-Redesign_wp_101-3328B1_FINAL.pdf&quot; rel=&quot;noreferrer&quot;&gt;Fluidigm white paper: Redesign of C1 Medium-Cell 96 IFCs Improves Single-Cell Capture Efficiency (PN 101-3328 B1)&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://genomics.cruk.cam.ac.uk/services/Single-cell#section-1&quot; rel=&quot;noreferrer&quot;&gt;10x Genomics Chromium 3'mRNA-Seq doublet rate&lt;/a&gt;: around 1% per 1000 cells captured&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Rates in the &lt;em&gt;'Chromium Single Cell 3’ Reagent Kits v2 User Guide':&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Multiplet Rate (%)  # of Cells Loaded   # of Cells Recovered&#xA;0.40%   870 500&#xA;0.80%   1700    1000&#xA;1.60%   3500    2000&#xA;2.30%   5300    3000&#xA;3.10%   7000    4000&#xA;3.90%   8700    5000&#xA;4.60%   10500   6000&#xA;5.40%   12200   7000&#xA;6.10%   14000   8000&#xA;6.90%   15700   9000&#xA;7.60%   17400   10000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h1&gt;Detection and identification of doublets and multiplets&lt;/h1&gt;&#xA;&#xA;&lt;h2&gt;Experimental&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Visual inspection&lt;/strong&gt; (e.g. Fluidigm C1)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;FACS index sorting data&lt;/strong&gt; can be used to identify doublets.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Mixed species experiment&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.illumina.com/content/dam/illumina-marketing/documents/products/technotes/single-cell-rna-data-analysis-tech-note-1070-2017-001.pdf&quot; rel=&quot;noreferrer&quot;&gt;Illumina | BioRad White paper&lt;/a&gt; Single-Cell RNA Data Analysis Workflow  &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://dx.doi.org/10.1038/ncomms14049&quot; rel=&quot;noreferrer&quot;&gt;Zheng et al., 2017&lt;/a&gt;: Massively parallel digital transcriptional profiling of single cells&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Cell &quot;hashing&quot; with oligo-tagged antibodies&lt;/strong&gt; (&lt;a href=&quot;http://dx.doi.org/10.1101/237693&quot; rel=&quot;noreferrer&quot;&gt;Stoeckius et al., 2017&lt;/a&gt;): cells (barcodes) positive for two or more oligos are multiplets.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Computational&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Mixed gene expression (e.g. lymphoid and myeloid, or T and B cell)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1186/s13059-016-0888-1&quot; rel=&quot;noreferrer&quot;&gt;Ilicic et al.&lt;/a&gt; describe an approach for detecting low quality cells (incl. multiplets). They train a support vector machine (SVM) model on a dataset annotated by microscopy inspection. They find, among others, that&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;&quot;wells containing multiple cells (multiples) show similar expression and noise patterns to broken cells&quot;&lt;/em&gt;, suggesting &lt;em&gt;&quot;that multiple cells contain a mixture of broken and high quality cells.&quot;&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;difference in proportion of duplicated reads&lt;/li&gt;&#xA;&lt;li&gt;higher ERCC spike-in ratios&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/statgen/demuxlet&quot; rel=&quot;noreferrer&quot;&gt;demuxlet&lt;/a&gt; (Kang et al., 2017, Nature Biotechnology)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/JonathanShor/DoubletDetection/blob/master/docs/DoubletDetection.pdf&quot; rel=&quot;noreferrer&quot;&gt;DoubletDetection&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="208" LastActivityDate="2018-02-13T17:50:12.857" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3533" PostTypeId="2" ParentId="3526" CreationDate="2018-02-13T19:17:43.693" Score="2" Body="&lt;p&gt;I might as well jump on as well. Here's a Perl script that I wrote a while ago to split a fasta sequence at stretches of Ns:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/gringer/bioinfscripts/blob/master/fasta-nsplit.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/gringer/bioinfscripts/blob/master/fasta-nsplit.pl&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've just modified it to spit out a BED formatted file showing the N locations on standard error. Use it as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./fasta-nsplit.pl tig87_withN.fa 2&amp;gt;out.bed &amp;gt; out.split.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output (BED file):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tig00000087 0   60&#xA;tig00000087 900 960&#xA;tig00000087 3840    3960&#xA;tig00000087 14880   14940&#xA;tig00000087 59520   59700&#xA;tig00000087 93000   93120&#xA;tig00000087 107880  107940&#xA;tig00000087 109135  109140&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's the full script, for completeness / fixing / comment:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/perl&#xA;use warnings;&#xA;use strict;&#xA;&#xA;my $seq = &quot;&quot;;&#xA;my $shortSeqID = &quot;&quot;;&#xA;my $seqID = &quot;&quot;;&#xA;my $keep = 0;&#xA;my $cumLength = 0;&#xA;while(&amp;lt;&amp;gt;){&#xA;  chomp;&#xA;  if(/^&amp;gt;((.+?)( .*?\s*)?)$/){&#xA;    my $newID = $1;&#xA;    my $newShortID = $2;&#xA;    if($seq){&#xA;      my $inc = 0;&#xA;      while($seq =~ s/(NNNN+)(.*)//){&#xA;	my $nStretch = $1;&#xA;        my $newSeq = $2;&#xA;        printf(&quot;&amp;gt;%s.%s\n%s\n&quot;, $seqID, $inc++, $seq) if ($seq);&#xA;	$cumLength += length($seq);&#xA;	printf(STDERR &quot;%s\t%d\t%d\n&quot;, $shortSeqID, $cumLength, &#xA;	       $cumLength + length($nStretch));&#xA;	$cumLength += length($nStretch);&#xA;        $seq = $newSeq;&#xA;      }&#xA;      printf(&quot;&amp;gt;%s\n%s\n&quot;, $seqID, $seq) if ($seq);&#xA;    }&#xA;    $seq = &quot;&quot;;&#xA;    $shortSeqID = $newShortID;&#xA;    $seqID = $newID;&#xA;    $cumLength = 0;&#xA;  } else {&#xA;    $seq .= $_;&#xA;  }&#xA;}&#xA;if($seq){&#xA;  my $inc = 0;&#xA;  while($seq =~ s/(NNNN+)(.*)//){&#xA;    my $nStretch = $1;&#xA;    my $newSeq = $2;&#xA;    printf(&quot;&amp;gt;%s.%s\n%s\n&quot;, $seqID, $inc++, $seq) if ($seq);&#xA;    $cumLength += length($seq);&#xA;    printf(STDERR &quot;%s\t%d\t%d\n&quot;, $shortSeqID, $cumLength, &#xA;       $cumLength + length($nStretch));&#xA;    $cumLength += length($nStretch);&#xA;    $seq = $newSeq;&#xA;  }&#xA;  printf(&quot;&amp;gt;%s\n%s\n&quot;, $seqID, $seq) if ($seq);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-02-14T03:51:29.297" LastActivityDate="2018-02-14T03:51:29.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3534" PostTypeId="2" ParentId="3274" CreationDate="2018-02-13T19:32:27.483" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://cancer.sanger.ac.uk/cell_lines/download&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cosmic Cell Lines Project&lt;/a&gt; provides gene expression data for 1000+ cell lines (including NCI-60 panel together with NCI-H23 cell line). For each gene in a cell line, you can get Z-score and over/under-expression status calculated across all tumor samples diploid for that gene. &lt;/p&gt;&#xA;" OwnerUserId="2346" LastEditorUserId="2346" LastEditDate="2018-02-13T19:40:57.117" LastActivityDate="2018-02-13T19:40:57.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3536" PostTypeId="1" CreationDate="2018-02-14T01:49:30.387" Score="1" ViewCount="44" Body="&lt;p&gt;Below are the two files I am working with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat mys.pep&#xA;&amp;gt;00001 16 324  len=309&#xA;LDRFYTKKVIMYKQQYFISGKVQGVGFRFFTEQIANNMKLKGFVKNLNDGRVEIVAFFNT&#xA;KEQMKKFEKLLNGNKYSNIKNIEKIVLDENYPFQFNDFKIYY*&#xA;&amp;gt;00002 751 308  len=398&#xA;YKLYNILLIKYFLIFPKYIDNA*I*KNKNYFKTFLLDRFYTKKVIMYKQQYFISGKVQGV&#xA;GFRFFTEQIANNMKLKGFVKNLNDGRVEIVAFFNTKEQMKKFEKLLNGNKYSNIKNIEKI&#xA;&#xA;$cat mys.nuc&#xA;&amp;gt;00001  16 324  len=309&#xA;TTGGATAGATTTTATACAAAGAAGGTAATAATGTATAAACAACAATATTTTATTTCTGGC&#xA;AAGGTGCAAGGTGTTGGTTTTAGATTTTTCACAGAGCAAATAGCAAATAATATGAAACTA&#xA;AAAGGATTTGTAAAAAATCTCAACGATGGAAGGGTAGAAATTGTAGCTTTCTTTAATACT&#xA;AAAGAACAAATGAAAAAATTTGAAAAATTATTAAATGGGAATAAGTATTCAAACATTAAA&#xA;AACATTGAAAAAATAGTTTTAGATGAAAATTATCCTTTTCAATTTAATGATTTTAAAATT&#xA;TATTATTAG&#xA;&amp;gt;00002  751 308  len=398&#xA;TACAAATTATATAATATCTTATTAATAAAATATTTCCTAATATTTCCCAAATATATTGAT&#xA;AATGCCTGAATTTAAAAAAACAAAAACTATTTTAAAACTTTTCTATTGGATAGATTTTAT&#xA;ACAAAGAAGGTAATAATGTATAAACAACAATATTTTATTTCTGGCAAGGTGCAAGGTGTT&#xA;GGTTTTAGATTTTTCACAGAGCAAATAGCAAATAATATGAAACTAAAAGGATTTGTAAAA&#xA;AATCTCAACGATGGAAGGGTAGAAATTGTAGCTTTCTTTAATACTAAAGAACAAATGAAA&#xA;AAATTTGAAAAATTATTAAATGGGAATAAGTATTCAAACATTAAAAACATTGAAAAAATA&#xA;GTTTTAGATGAAAATTATCCTTTTCAATTTAATGATTT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What I have done so far is try to run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bioseq -B mys.nuc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;00001&#xA;00002&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Ultimately I am mean to extract the ORF sequence and save as &lt;code&gt;mys.nuc2&lt;/code&gt; and then translate &lt;code&gt;mys.nuc2&lt;/code&gt; with &lt;code&gt;bioseq -t1&lt;/code&gt; where that file should NOT contain any internal stop codons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not understand why I get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;00001&#xA;00002&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or what I am meant to do with it, if that indeed is the correct output.&lt;/p&gt;&#xA;" OwnerUserId="2348" LastEditorUserId="48" LastEditDate="2018-02-14T07:57:23.963" LastActivityDate="2018-02-14T07:57:23.963" Title="I have two files: mys.nuc and ms.prep and need to use bioseq to extract the ORF sequences and save as “mys.nuc2”" Tags="|fasta|ngs|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3537" PostTypeId="2" ParentId="3516" CreationDate="2018-02-14T02:12:31.080" Score="3" Body="&lt;p&gt;In the bwa source code, file bwt.{c,h} implement basic operations on .bwt and .sa files. They depend on utils.{c,h}, but you can create a fake utils.h to cut this dependency as is shown below. Then you only need three files: kvec.h, bwt.h and bwt.c to access .bwt and .sa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following example demonstrates how to find super-maximal exact matches (SMEMs). Note that internally bwa concatenates all database sequences into one long string. If you use bwt.{c,h} only, you get coordinates on this long string but not on each chromosome. You have to call APIs in bntseq.{c,h}, which are fairly independent, too. You can go on to implement a mini bwa-mem with &lt;a href=&quot;https://github.com/lh3/bwa/blob/master/example.c&quot; rel=&quot;nofollow noreferrer&quot;&gt;this example&lt;/a&gt;, but you will need all the bwa-mem source code, except the main function.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Compiling and running&lt;/h2&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;# compile the example&#xA;git clone https://github.com/lh3/bwa&#xA;mkdir -p tiny-smem &amp;amp;&amp;amp; cd tiny-smem&#xA;cp ../bwa/{bwt.c,bwt.h,kvec.h} .              # you need at least these three files&#xA;### then create utils.h and test.c shown below ###&#xA;gcc -g -O2 -Wall bwt.c test.c -o tiny-smem    # compile tiny-smem&#xA;&#xA;# create a bwa index if you don't have it. We use chr20 from GRCh38 as an example:&#xA;(cd ../bwa &amp;amp;&amp;amp; make)                           # compile bwa for indexing&#xA;../bwa/bwa index -p chr20 path/to/human/chr20.fa&#xA;&#xA;# run the example&#xA;./tiny-smem chr20.bwt chr20.sa TTTTCTTCTTTTGTAATATAAGCATTgGTTGTTCTAAATTTCTCTCTTGGCACTGCTTTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The last command line above outputs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0       0       26      1       -125734&#xA;0       14      27      4       -3657204        +16021887       +11802682       +54692983&#xA;0       15      29      1       +8344885&#xA;0       17      31      1       -39545379&#xA;0       19      37      1       +55060482&#xA;0       25      39      2       +1198672        -26409692&#xA;0       27      60      1       -125700&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where column 2 and 3 give the coordinate of SMEM on the query, col4 the number of exact hits and the following columns are the list of hits in the concatenated coordinate.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;File: utils.h&lt;/h2&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;#ifndef FAKE_UTILS_H&#xA;#define FAKE_UTILS_H&#xA;&#xA;#define xassert(cond, msg) assert(cond)&#xA;#define xopen(fn, mode) fopen((fn), (mode))&#xA;#define err_fclose(fp) fclose(fp)&#xA;#define err_fread_noeof(ptr, size, n, fp) fread((ptr), (size), (n), (fp))&#xA;#define err_fwrite(ptr, size, n, fp) fwrite((ptr), (size), (n), (fp))&#xA;#define err_fflush(fp) fflush(fp)&#xA;#define err_fseek(fp, off, whence) fseek((fp), (off), (whence))&#xA;#define err_ftell(fp) ftell(fp)&#xA;&#xA;#endif // ~FAKE_UTILS_H&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;File: test.c&lt;/h2&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;stdlib.h&amp;gt;&#xA;#include &amp;lt;string.h&amp;gt;&#xA;#include &amp;lt;ctype.h&amp;gt;&#xA;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &quot;bwt.h&quot;&#xA;&#xA;int main(int argc, char *argv[]) {&#xA;  bwt_t *bwt;&#xA;  int j, max_intv = 100; // ignore SA intervals larger than this value&#xA;  bwtintv_v r = {0,0,0};&#xA;&#xA;  if (argc &amp;lt; 4) { fprintf(stderr, &quot;Usage: ./test &amp;lt;.bwt&amp;gt; &amp;lt;.sa&amp;gt; &amp;lt;seq&amp;gt;\n&quot;); return 1; }&#xA;  if ((bwt = bwt_restore_bwt(argv[1])) == NULL) abort(); // read .bwt&#xA;  bwt_restore_sa(argv[2], bwt);                          // read .sa&#xA;  for (j = 3; j &amp;lt; argc; ++j) {&#xA;    int i, k, x, len = strlen(argv[j]);&#xA;    uint8_t *seq = (uint8_t*)malloc(len);&#xA;    for (i = 0; i &amp;lt; len; ++i) { // convert A/C/G/T/N =&amp;gt; 0/1/2/3/4&#xA;      int c = tolower(argv[j][i]);&#xA;      if (c == 'a') seq[i] = 0;&#xA;      else if (c == 'c') seq[i] = 1;&#xA;      else if (c == 'g') seq[i] = 2;&#xA;      else if (c == 't') seq[i] = 3;&#xA;      else seq[i] = 4;&#xA;    }&#xA;    x = 0, r.n = 0;&#xA;    while (x &amp;lt; len) {&#xA;      x = bwt_smem1(bwt, len, seq, x, 1, &amp;amp;r, 0); // find SMEMs&#xA;      for (k = 0; k &amp;lt; r.n; ++k) { // traverse each SMEM&#xA;        bwtintv_t *p = &amp;amp;r.a[k]; // p-&amp;gt;info encodes start and end on the query&#xA;        printf(&quot;%d\t%d\t%d\t%d&quot;, j-3, (int)(p-&amp;gt;info&amp;gt;&amp;gt;32), (int)p-&amp;gt;info, (int)p-&amp;gt;x[2]);&#xA;        if (r.a[k].x[2] &amp;lt;= max_intv) { // don't retrieve coordinates for huge intervals&#xA;          int l_mem = (int)p-&amp;gt;info - (int)(p-&amp;gt;info&amp;gt;&amp;gt;32);&#xA;          for (i = 0; i &amp;lt; p-&amp;gt;x[2]; ++i) { // traverse each hit&#xA;            uint64_t y = bwt_sa(bwt, p-&amp;gt;x[0] + i); // get suffix array coordinate&#xA;            if (y &amp;lt; bwt-&amp;gt;seq_len&amp;gt;&amp;gt;1) printf(&quot;\t+%ld&quot;, (long)y);&#xA;            else printf(&quot;\t-%ld&quot;, (long)(bwt-&amp;gt;seq_len - y - l_mem));&#xA;          }&#xA;        }&#xA;        putchar('\n');&#xA;      }&#xA;    }&#xA;    free(seq);&#xA;  }&#xA;  free(r.a);&#xA;  bwt_destroy(bwt);&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-02-14T02:12:31.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3538" PostTypeId="1" CreationDate="2018-02-14T04:49:42.393" Score="2" ViewCount="809" Body="&lt;p&gt;Currently I have 4 fastq file for each patient as WES data. For WES pipeline, I want to combine those fastq files. I am using writeFastq from ShortRead package in Bioconductor.But I think this function is not working properly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason of my doubt is the the size the next step is quality check using FastQC tool. This tool is working on individual file but not on combined file and shows the error that ID line doesn't start with &quot;@&quot; . So I want to know what are the other effective tool for combining multiple fastq files without any loss of information. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me know if I am doing something wrong. Thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;[EDIT-1]&lt;/strong&gt; : The codes for combining multiple fastq files into one is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;reference_fastq = file.path(paste(getwd(),&quot;refs&quot;,sep = &quot;/&quot;, collapse = &quot; &quot;), fsep = &quot;.fasta&quot;)&#xA;fastqDir &amp;lt;- file.path(dataDir, fsep = &quot;.fastq&quot;)&#xA;outfile_temp &amp;lt;- paste(dirname(dataDir),basename(fastqDir),    paste(basename(fastqDir),&quot;combined&quot;,&quot;fastq&quot;, sep = &quot;.&quot;),sep = &quot;/&quot;, collapse = &quot; &quot;)&#xA;fls &amp;lt;- list.files(fastqDir, &quot;fastq$&quot;, full=TRUE)&#xA;for (fl in fls) {&#xA;  fq &amp;lt;- readFastq(fl)&#xA;  writeFastq(fq,outfile_temp, mode=&quot;a&quot;)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;First four lines of my fastq file (one of 4 indivual file) are as follows :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@D00239:185:C9KELANXX:2:2201:1088:2107 1:N:0:TCCTGAGC&#xA;GTACAAGAAGCATAACATCAGCATGTGTTTTGGTGAGAACCTTAGGAAGTTTGCAATCATGACAGAAGGTGAAGG&#xA;+&#xA;/&amp;lt;BBBFFFBFFFFFFFFFFFBF&amp;lt;FFBF&amp;lt;B&amp;lt;FFFFFFFFFFB/B/&amp;lt;////&amp;lt;&amp;lt;&amp;lt;FF&amp;lt;B&amp;lt;FF&amp;lt;FFFFFFFFF/FFFFF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;First four lines of combines fastq file are as follows: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@D00239:185:C9KELANXX:2:2201:1088:2107 1:N:0:TCCTGAGC&#xA;GTACAAGAAGCATAACATCAGCATGTGTTTTGGTGAGAACCTTAGGAAGTTTGCAATCATGACAGAAGGTGAAGG&#xA;+&#xA;/&amp;lt;BBBFFFBFFFFFFFFFFFBF&amp;lt;FFBF&amp;lt;B&amp;lt;FFFFFFFFFFB/B/&amp;lt;////&amp;lt;&amp;lt;&amp;lt;FF&amp;lt;B&amp;lt;FF&amp;lt;FFFFFFFFF/FFFFF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is pretty clear that combined fastq file is also starting wiht proper identifier but why FastQC tool is saying that identifier '@' is missing. Here is the screen shot of the error message:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aZ3E2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aZ3E2.png&quot; alt=&quot;Screen shot of error message&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="2141" LastEditorUserId="2141" LastEditDate="2018-02-14T08:48:02.443" LastActivityDate="2018-02-14T10:20:59.917" Title="Combine Fastq by writeFastq is not working properly" Tags="|r|bioconductor|sequence-analysis|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3539" PostTypeId="2" ParentId="3538" CreationDate="2018-02-14T10:09:54.423" Score="4" Body="&lt;p&gt;You can drastically simplify your effort because FASTQ files are simple text files; therefore, standard UNIX test file tools work:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;cat refs/*.fastq &amp;gt; combined.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this even works with &lt;code&gt;.fastq.gz&lt;/code&gt; files — no unzipping required.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Apart from this, your R code contains several errors. For instance, your variable names change throughout, and you’re using the &lt;code&gt;fsep&lt;/code&gt; parameter wrongly. In fact it’s unclear to me what the code is supposed to do. At an approximation (and untested), the following should work:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;fastq_dir = file.path(getwd(), &quot;refs&quot;)&#xA;fastq_files = dir(fastq_dir, pattern = &quot;\\.fastq$&quot;, full.names = TRUE)&#xA;outfile = file.path(dirname(fastq_dir), &quot;combined.fastq&quot;)&#xA;&#xA;for (file in fastq_files) {&#xA;    fq = readFastq(file)&#xA;    writeFastq(fq, outfile, mode = &quot;a&quot;)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is equivalent to running the shell command shown above, just in a more convoluted way (parsing and re-generating the FASTQ files).&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2018-02-14T10:20:59.917" LastActivityDate="2018-02-14T10:20:59.917" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3540" PostTypeId="2" ParentId="3525" CreationDate="2018-02-14T10:41:15.887" Score="2" Body="&lt;p&gt;Although I've never tried it myself, I think what you need can be achieved through the &lt;a href=&quot;http://beast.community/logcombiner&quot; rel=&quot;nofollow noreferrer&quot;&gt;LogCombiner program&lt;/a&gt; from the BEAST package -- assuming the species trees files are comparable (ran with same parameters, same species names, etc.) I am also assuming that you meant 'five species tree files', each file with the posterior sample of species trees.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I do believe you should try to run all loci within the same analysis, or at least do some prior check to see if the distributions between data sets are comparable. LogCombiner assumes &lt;strong&gt;independent runs from the same data&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also recommend taking a look (and following) the &lt;a href=&quot;https://groups.google.com/forum/#!forum/beast-users&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEAST google group&lt;/a&gt; (which includes starBEAST discussions).&lt;/p&gt;&#xA;" OwnerUserId="45" LastActivityDate="2018-02-14T10:41:15.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3541" PostTypeId="1" AcceptedAnswerId="3625" CreationDate="2018-02-14T11:16:35.723" Score="6" ViewCount="240" Body="&lt;p&gt;A news item from 2017-10-18 on the website of the &lt;a href=&quot;https://www.humancellatlas.org/&quot; rel=&quot;noreferrer&quot;&gt;Human Cell Atlas&lt;/a&gt; states:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In addition, the consortium today also announced the impending release of gene expression profiles from the first one million immune cells collected under the HCA, toward an initial milestone of collecting at least 30 million cells representing several tissues and organs for the atlas’ first draft. &lt;strong&gt;These data, to be posted on an online repository by early November, will be freely available for researchers’ use&lt;/strong&gt;. (&lt;a href=&quot;https://www.humancellatlas.org/news/14&quot; rel=&quot;noreferrer&quot;&gt;https://www.humancellatlas.org/news/14&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As of now, Feb 2018, I cannot find these data anywhere. &#xA;Does anybody know where I can download the gene expression data of these 1 million immune cells or when it will be eventually published?&lt;/p&gt;&#xA;" OwnerUserId="1656" LastActivityDate="2018-02-26T21:35:53.963" Title="Human Cell Atlas - Data availability" Tags="|public-databases|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3542" PostTypeId="2" ParentId="3526" CreationDate="2018-02-14T19:58:31.830" Score="3" Body="&lt;p&gt;Use twoBitInfo:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ twoBitInfo file.fa -nBed output.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For example, to get all the N-masked regions on chromosome Y (also note you can use stdout as a filename to write directly to stdout, and use of the url as the input, no need to download the 2bit file):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ twoBitInfo http://hgdownload.soe.ucsc.edu/goldenPath/hg38/bigZips/hg38.2bit:chrY -nBed stdout | head -3&#xA;chrY    0   10000&#xA;chrY    44821   94821&#xA;chrY    133871  222346&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can download twoBitInfo from the directory appropriate to your operating system here:&#xA;&lt;a href=&quot;http://hgdownload.soe.ucsc.edu/admin/exe/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://hgdownload.soe.ucsc.edu/admin/exe/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1000" LastActivityDate="2018-02-14T19:58:31.830" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3543" PostTypeId="1" AcceptedAnswerId="3544" CreationDate="2018-02-14T23:47:41.607" Score="1" ViewCount="89" Body="&lt;p&gt;If I were to run BAM/SAM files through a program like STRUCTURE, or ADMIXTURE described  &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3665925/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; how would they determine which individuals belong to what groups?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What part of the files would these programs use to determine which individuals belonged to different populations? I believe they do this via determining allele frequencies for each individual and then putting them into a population based on allele frequency, is the only criteria that they use comparison of allele frequencies between individuals?&lt;/p&gt;&#xA;" OwnerUserId="2063" LastEditorUserId="73" LastEditDate="2018-02-15T06:27:22.147" LastActivityDate="2018-02-15T11:44:44.920" Title="How do programs like STRUCTURE determine populations from alignments?" Tags="|sam|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3544" PostTypeId="2" ParentId="3543" CreationDate="2018-02-15T02:39:09.967" Score="2" Body="&lt;p&gt;&lt;em&gt;structure&lt;/em&gt; works with SNP/genotype information, so won't work directly off BAM/SAM files. You would need to convert to VCF (e.g. using mpileup/vcftools), then process using something like PLINK to convert it into a format that STRUCTURE recognises.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the description I have about &lt;em&gt;structure&lt;/em&gt; from &lt;a href=&quot;http://researcharchive.vuw.ac.nz/handle/10063/1987&quot; rel=&quot;nofollow noreferrer&quot;&gt;my PhD thesis&lt;/a&gt;. It might be a bit dated now, but I expect most of it is still applicable:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A clustering program called &lt;em&gt;structure&lt;/em&gt; was created by &lt;a href=&quot;http://www.genetics.org/content/164/4/1567.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;Falush et al. (2003)&lt;/a&gt; that is used to determine population structure using a set of unlinked genotypes. The expected use case for &lt;em&gt;structure&lt;/em&gt; is in the analysis of population structure, but it can also be useful for determining the utility of a given set of markers for categorising (or quantifying) a given trait.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main benefit of using &lt;em&gt;structure&lt;/em&gt; over other methods is that it is able to derive estimated ancestry coefficients from the data, rather than a discrete yes/no for each individual. This allows for the use of cutoff values for the assignment of individuals (useful for tweaking false positive and false negative rates in diagnostic tests), and a quantifiable value where a classification is not useful or does not make sense (as in the case of a continuous trait with high heritability such as height).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The program begins by assigning each individual to a given population. This can be either given in the input data as a ‘popinfo’ flag, or determined randomly by structure. After this, the program constructs a probability distribution for the data, considering the population assignments given, and uses that to estimate population allele frequencies. The program then constructs another probability distribution for these data, and uses that to estimate the population that each individual is likely to have originated from. This process is repeated many times (such a process is known as a random walk), and the underlying theory suggests that the progression of population assignments through the random walk process can be used as an approximation of the actual populations of origin for each individual.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The documentation included with &lt;em&gt;structure&lt;/em&gt; describes the general algorithm for each iteration (of maybe hundreds of thousands of iterations) as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Sample (population allele frequencies for iteration m) from the probability of (allele frequency) given (genotypes of sampled individuals) and (population of origin of individuals for iteration m − 1).&lt;/li&gt;&#xA;&lt;li&gt;Sample (population of origin of individuals for iteration m) from the&#xA;probability of (population of origin) given (genotypes of sampled individuals) and (allele frequencies for iteration m).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In more simple terms, the steps are similar to the following statements:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;these populations probably have &lt;em&gt;this&lt;/em&gt; allele frequency distribution&#xA;therefore...&lt;/li&gt;&#xA;&lt;li&gt;these individuals probably came from &lt;em&gt;this&lt;/em&gt; population&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The &lt;em&gt;structure&lt;/em&gt; program is able to accommodate admixture into its model&#xA;by considering the population of origin as a probability, rather than a single&#xA;discrete value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The file that is output from &lt;em&gt;structure&lt;/em&gt; is a text file with individual identifiers and predicted ancestry coefficients (Q values). While the program does provide means to display these data graphically, the visual representation of data (including sorting) is better controlled using an external program. The most commonly used program is Noah Rosenberg’s distruct &lt;a href=&quot;http://onlinelibrary.wiley.com/doi/10.1046/j.1471-8286.2003.00566.x/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;(Rosenberg, 2004)&lt;/a&gt;, a program that allows colour customisation and group re-ordering. For the purpose of this thesis, a custom R script, &lt;code&gt;snpchip2structure.R&lt;/code&gt;, has been designed which also allows selection of different sorting methods, a scatter plot for two populations, error bars, and a few additional features (see figure 1.7). More details about this script can be found in the programs directory of the supplementary CD for this thesis.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-02-15T11:44:44.920" LastActivityDate="2018-02-15T11:44:44.920" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3545" PostTypeId="1" CreationDate="2018-02-15T09:32:26.967" Score="1" ViewCount="74" Body="&lt;p&gt;NPDock service &lt;a href=&quot;http://genesilico.pl/NPDock/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genesilico.pl/NPDock/&lt;/a&gt; is unavailable, at least that's the message that shows after you try to submit a job. It's been like this for the last few weeks. Contact email address provided on the page also doesn't work. Does NPDock work for anyone else? &lt;/p&gt;&#xA;" OwnerUserId="2362" LastEditorUserId="2362" LastEditDate="2018-02-15T09:42:00.247" LastActivityDate="2018-02-15T09:42:00.247" Title="NPDock - service unavailable" Tags="|rna|docking|webservice|" AnswerCount="0" CommentCount="4" ClosedDate="2018-02-15T13:54:30.993" ContentLicense="CC BY-SA 3.0" />
  <row Id="3546" PostTypeId="1" AcceptedAnswerId="3554" CreationDate="2018-02-15T19:10:57.857" Score="5" ViewCount="95" Body="&lt;p&gt;I am experimenting with adding additional reads to the input files I'm giving SOAPdenovo2, and there comes a point where a good contig I've been watching actually stops showing up. Does anyone have a quick answer to why that might happen?&lt;/p&gt;&#xA;" OwnerUserId="83" LastActivityDate="2018-03-01T06:10:34.227" Title="Why can using more reads lead to a lower quality assembly?" Tags="|assembly|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3547" PostTypeId="1" CreationDate="2018-02-15T19:29:23.787" Score="1" ViewCount="545" Body="&lt;p&gt;I used hisat2 for aligning reads to to the genome. I have an alignment summary for sample1 as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;89038751 reads; of these:&#xA;  89038751 (100.00%) were paired; of these:&#xA;    5641019 (6.34%) aligned concordantly 0 times&#xA;    67665552 (76.00%) aligned concordantly exactly 1 time&#xA;    15732180 (17.67%) aligned concordantly &amp;gt;1 times&#xA;    ----&#xA;    5641019 pairs aligned concordantly 0 times; of these:&#xA;      651415 (11.55%) aligned discordantly 1 time&#xA;    ----&#xA;    4989604 pairs aligned 0 times concordantly or discordantly; of these:&#xA;      9979208 mates make up the pairs; of these:&#xA;        5887822 (59.00%) aligned 0 times&#xA;        2656902 (26.62%) aligned exactly 1 time&#xA;        1434484 (14.37%) aligned &amp;gt;1 times&#xA;96.69% overall alignment rate&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To check the alignment quality I used Qualimap. In the report I see the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Reads alignment&#xA;&#xA;Number of mapped reads: 172,189,680&#xA;Total number of alignments: 244,302,519&#xA;Number of secondary alignments: 72,112,839&#xA;Number of non-unique alignments:    22,973,098&#xA;Aligned to genes:   2,081&#xA;Ambiguous alignments:   173&#xA;No feature assigned:    178,975&#xA;Not aligned:    5,887,822&#xA;&#xA;Reads genomic origin&#xA;&#xA;Exonic: 2,081 / 1.15%&#xA;Intronic:   6,866 / 3.79%&#xA;Intergenic: 172,109 / 95.06%&#xA;Intronic/intergenic overlapping exon:   302 / 0.17%&#xA;&#xA;Transcript coverage profile&#xA;&#xA;5' bias:    0.69&#xA;3' bias:    2.85&#xA;5'-3' bias: 0.39&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can anyone tell me why the numbers look different? Did I go wrong somewhere? Help me in understanding the results.&lt;/p&gt;&#xA;" OwnerUserId="1369" LastEditorUserId="73" LastEditDate="2018-02-15T22:10:28.970" LastActivityDate="2018-02-15T23:19:37.523" Title="Alignment QC differences between HISAT2 and Qualimap" Tags="|rna-seq|sam|sequence-alignment|rna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3548" PostTypeId="1" AcceptedAnswerId="3553" CreationDate="2018-02-15T19:42:45.247" Score="2" ViewCount="35" Body="&lt;p&gt;I am looking at some &lt;a href=&quot;https://www.ebi.ac.uk/ena/data/view/SAMEA2757769&quot; rel=&quot;nofollow noreferrer&quot;&gt;yeast ONT data&lt;/a&gt; from &lt;a href=&quot;https://www.nature.com/articles/s41598-017-03996-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;De novo yeast genome assemblies from MinION, PacBio and MiSeq platforms, Giordano et al 2017&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I noticed that the Minion fastq files have both a forward and reverse file. The methods paper says that these reads are from the SQK-MAP006 and SQK-MAP007 protocols which both appear to be 2D. Are these just the 2D forward and reverse files?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Still being unsure, I checked if this was true by counting the number of characters in each read. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt; zcat ERR1883385_1.fastq.gz | head -1&#xA;@ERR1883385.1 dc5b9de2-ba00-47ab-b698-8ad470704c0a_Basecall_2D_000_template EVO107591_Sanger_FAA52213_CBS432_Phenol_3a_4517_1_ch251_file44_strand/1&#xA;&#xA;&amp;gt;&amp;gt; zcat ERR1883385_2.fastq.gz | head -1&#xA;@ERR1883385.1 dc5b9de2-ba00-47ab-b698-8ad470704c0a_Basecall_2D_000_template EVO107591_Sanger_FAA52213_CBS432_Phenol_3a_4517_1_ch251_file44_strand/2&#xA;&#xA;&#xA;&amp;gt;&amp;gt; zcat ERR1883385_1.fastq.gz | head -2 | awk '{ print length($0); }' -&#xA;147&#xA;1675&#xA;&#xA;&amp;gt;&amp;gt; zcat ERR1883385_2.fastq.gz | head -2 | awk '{ print length($0); }' -&#xA;147&#xA;1693&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, the read is the same but the forward and reverse strands are different lengths. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some way to just download the &lt;code&gt;poretools&lt;/code&gt; equivalent to  &lt;code&gt;2D&lt;/code&gt; from ENA without having to download the fast5 files and extract them myself?&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-02-15T22:29:59.420" Title="ENA minion 2D data doesn't actually have 2D data" Tags="|minion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3549" PostTypeId="1" CreationDate="2018-02-15T19:50:52.170" Score="4" ViewCount="321" Body="&lt;p&gt;I have a bam file and a bed file that defines a list of SNPs. I would like to filter the bam file to contain only those reads with a minimum mapping quality that overlap at least one SNP with a minimum base quality. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Samtools seems to almost solve this problem, but not quite:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;samtools depth -b snp_list -q 20 -Q 30 example.bam&lt;/code&gt; allows me to count the number of alignments meeting my desired criteria, but not to produce a bam with these alignments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;samtools view -L snp_bed -q 30 example.bam&lt;/code&gt; allows me to filter the alignments overlapping any SNP with minimum mapping quality, but not with a minimum base quality at the SNP sites. There is no &lt;code&gt;-Q&lt;/code&gt; option for samtools view, and the &lt;code&gt;-q&lt;/code&gt; option is for mapping quality&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can this be achieved in samtools? Or is there another preexisting tool that can? &lt;/p&gt;&#xA;" OwnerUserId="272" LastActivityDate="2018-02-15T21:55:06.853" Title="Filter bam using SNP list in bed format with minimum mapping quality and base quality" Tags="|sam|samtools|bed|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3550" PostTypeId="2" ParentId="3549" CreationDate="2018-02-15T20:11:20.680" Score="1" Body="&lt;p&gt;You're not going to get samtools to do that, though I wouldn't be surprised if something in &lt;a href=&quot;https://github.com/lindenb/jvarkit&quot; rel=&quot;nofollow noreferrer&quot;&gt;jvarkit&lt;/a&gt; can do it. Baring that, here's a bit of python.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pysam&#xA;&#xA;bed = open(&quot;foo.bed&quot;)&#xA;bam = pysam.AlignmentFile(&quot;foo.bam&quot;)&#xA;of = pysam.AlignmentFile(&quot;filtered.bam&quot;, mode=&quot;wb&quot;, template=bam)&#xA;seen = set()  # memoize reads seen to not double print them&#xA;for line in bed:&#xA;    cols = line.strip().split(&quot;\t&quot;)&#xA;    for col in bam.pileup(cols[0], int(cols[1]), int(cols[2])):&#xA;        for b in col.pileups:&#xA;            if b.alignment.mapping_quality &amp;lt; MIN_MAPQ:&#xA;                continue&#xA;            pos = b.query_position&#xA;            if not pos:&#xA;                continue&#xA;            if b.alignment.query_qualities[pos] &amp;lt; MIN_PHRED:&#xA;                continue&#xA;            if b.alignment.query_name not in seen:&#xA;                seen.add(b.alignment.query_name)&#xA;                of.write(b.alignment)&#xA;of.close()&#xA;bam.close()&#xA;bed.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or something along those lines, I haven't debugged that.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-02-15T20:11:20.680" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3551" PostTypeId="2" ParentId="3547" CreationDate="2018-02-15T20:34:33.810" Score="2" Body="&lt;p&gt;89038751 pairs times 2 is 178 million reads. That times the approximately 97% alignment rate is around 172 million reads, which is what qualimap is reporting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &quot;non-unique&quot; alignment is one that can map to multiple places (either equally well, or where all of the alignments are of similar enough quality). Such alignments have a single &quot;primary&quot; alignment and one or more &quot;secondary&quot; alignments, which is where the &quot;secondary&quot; metrics come from.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the qualimap output, there are additional metrics describing how reads overlap genes (or not). Of note are the &quot;ambiguous&quot; alignments, which are those overlapping multiple genes. &quot;No feature&quot; metrics, then, refer to alignments not overlapping any gene (see &lt;a href=&quot;http://qualimap.bioinfo.cipf.es/doc_html/analysis.html#id7&quot; rel=&quot;nofollow noreferrer&quot;&gt;the qualimap documentation&lt;/a&gt; for further details).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guess is that you have such a high &quot;intergenic&quot; rate because you've either used the wrong GTF file or have told qualimap the wrong strand setting. For unusual organisms one expects many reads to arise from unannotated genes. This is not the case for common model organisms, such as mice and humans, where there are very few remaining commonly expressed unannotated genes.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-02-15T23:19:37.523" LastActivityDate="2018-02-15T23:19:37.523" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3552" PostTypeId="2" ParentId="3549" CreationDate="2018-02-15T21:55:06.853" Score="2" Body="&lt;p&gt;Thanks Devon ;-) &#xA;using samjdk : &lt;a href=&quot;http://lindenb.github.io/jvarkit/SamJdk.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://lindenb.github.io/jvarkit/SamJdk.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a bed file 'input.bed'&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the following snippet:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;private IntervalTreeMap&amp;lt;Boolean&amp;gt; treeMap=null;&#xA;private final int BASE_QUALITY= 20;&#xA;@Override&#xA;public Object apply(final SAMRecord record) {&#xA;if(record.getReadUnmappedFlag() || record.getCigar()==null) return null;&#xA;if(treeMap==null) {&#xA;    treeMap = new  IntervalTreeMap&amp;lt;Boolean&amp;gt;();&#xA;    final String bedStr;&#xA;    try {&#xA;        bedStr = IOUtil.slurp(new java.io.File(&quot;input.bed&quot;));&#xA;        }&#xA;    catch(java.io.IOException err)&#xA;        {&#xA;        throw new RuntimeIOException(err);&#xA;        }&#xA;    Arrays.stream(bedStr.split(&quot;[\n]&quot;)).&#xA;        filter(L-&amp;gt;!(L.startsWith(&quot;track&quot;) || L.isEmpty() || L.startsWith(&quot;browser&quot;))).&#xA;        map(L-&amp;gt;L.split(&quot;\t&quot;)).&#xA;        map(A-&amp;gt;new Interval(A[0],Integer.parseInt(A[1])+1,Integer.parseInt(A[2]))).&#xA;        forEach(I-&amp;gt;treeMap.put(I,true));&#xA;    }&#xA;&#xA;&#xA;byte quals[]= record.getBaseQualities();&#xA;int refpos= record.getStart();&#xA;for(CigarElement ce:record.getCigar()) {&#xA;    CigarOperator op= ce.getOperator();&#xA;    if(op.consumesReferenceBases())&#xA;        {&#xA;        if(op.consumesReadBases()) {&#xA;            for(int i=0;i&amp;lt; ce.getLength();i++)&#xA;                {&#xA;                Interval t = new Interval(record.getContig(),refpos+i,refpos+i);&#xA;                if(!treeMap.containsOverlapping(t)) continue;&#xA;                int readpos = record.getReadPositionAtReferencePosition(refpos+i);&#xA;                if(readpos==0 ) continue;&#xA;                readpos--;&#xA;                if(readpos&amp;gt;=quals.length)  continue;&#xA;                if(quals[readpos]&amp;gt;BASE_QUALITY) return true;&#xA;                }&#xA;            }&#xA;&#xA;        refpos+=ce.getLength();&#xA;        }&#xA;&#xA;    }&#xA;return null;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;usage:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -bu -L input.bed input.bam |\&#xA;   java -jar dist/samjdk.jar --body -f snippet.code &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastActivityDate="2018-02-15T21:55:06.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3553" PostTypeId="2" ParentId="3548" CreationDate="2018-02-15T22:29:59.420" Score="2" Body="&lt;p&gt;Probably not. ENA doesn't have any ability to specifically flag a file as 2D, and the nanopore file formats have changed a lot, so you pretty much need to rely on the submitter to separate out the reads in a meaningful way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, there wasn't really any consistent convention for what ONT used in their read headers. Looking at some of my old files they had a plain sequence ID (i.e. without anything suggesting temp/comp) in different template/complement folders in the FAST5 files for the components of a 2D read and the consensus sequence, but later changed it to _2D_template and _2D_complement as in the example you have (possibly with an intermediate _1D_template and _1D_complement format). Parsing/understanding those FAST5 files can be difficult, which is why poretools was created in the first place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's still a bit of a mess, with the 1D$^2$ reads ending up in a Frankenstein-like state that's partially 1D and partially 2D.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-02-15T22:29:59.420" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3554" PostTypeId="2" ParentId="3546" CreationDate="2018-02-15T23:39:58.220" Score="4" Body="&lt;p&gt;The more reads you add the more errors you add into the assembly. This is because additional duplicate reads don't add nodes/edges to the de Bruijn graph, but those with errors do. By preselecting those reads aligning to your desired contigs, you may be further exacerbating this effect and further hindering SOAPdenovo2's ability to do k-mer correction (after all, the observed k-mer frequency of errors would be higher). This then results in additional incorrect paths through the graph that the assembler has to traverse.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-02-15T23:49:21.790" LastActivityDate="2018-02-15T23:49:21.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3555" PostTypeId="2" ParentId="3546" CreationDate="2018-02-15T23:44:05.020" Score="0" Body="&lt;p&gt;I think the reason that no one has submitted an actual answer yet is because this question is really dependent on the size of the genome you are sequencing, its repetitive content, the level of heterozygosity, the amount of contamination, the type of data (illumina, long illumina, long reads), the library type, library peculariaties, PCR error, the heterozygosity of the contig in question, the complexity of the contig in question (high information content? repeats? AT/GC%?), and probably some other things that I haven't thought of. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like @user172818 said, all of those things ON TOP OF all of the quirks of different assemblers make this a hard question to answer for you without a full diagnosis of your data and the genome of the thing you are sequencing/assembling.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-02-15T23:44:05.020" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3557" PostTypeId="1" CreationDate="2018-02-16T14:27:39.633" Score="3" ViewCount="1722" Body="&lt;p&gt;I have data in Plink format. I removed Indels using --snps-only command in Plink, but still some snps are as below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rs58119544;rs5746975    0   16915280    T   A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I wonder, how i can get rid of these snps? the correct should be single snps without this (;) sign. Any idea to fix it please?&lt;/p&gt;&#xA;" OwnerUserId="2365" LastActivityDate="2018-02-16T15:43:49.977" Title="removing indels in plink" Tags="|plink|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3558" PostTypeId="2" ParentId="3557" CreationDate="2018-02-16T15:43:49.977" Score="1" Body="&lt;p&gt;Use &lt;em&gt;grep&lt;/em&gt; to get SNPs that have &lt;code&gt;&quot;;&quot;&lt;/code&gt; in the name:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;$ grep &quot;;&quot; myMap.txt | cut -f1 -d&quot; &quot; &amp;gt; myBadSNPs.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then use plink with &lt;a href=&quot;https://www.cog-genomics.org/plink/1.9/filter#snp&quot; rel=&quot;nofollow noreferrer&quot;&gt;--exclude&lt;/a&gt; :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plink --file myFile&#xA;--exclude myBadSNPs.txt&#xA;--out myFileFiltered&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="131" LastActivityDate="2018-02-16T15:43:49.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3559" PostTypeId="1" AcceptedAnswerId="3560" CreationDate="2018-02-16T16:56:20.233" Score="1" ViewCount="257" Body="&lt;p&gt;I am planning an experiment on gene expression. I have 6 conditions and 3 replicates in each condition. I would like to use &lt;a href=&quot;https://www.thermofisher.com/order/catalog/product/4456740&quot; rel=&quot;nofollow noreferrer&quot;&gt;ERCC spike-in mixes&lt;/a&gt; for quality check: lower limit of detection, dynamic range and also check that the fold-changes are accurate. For example, by knowing that spike-in sequence xyz in condition 2 should be a 4-fold higher than in condition 1, does it actually get detected as a 4-fold change in my analysis? To assess the fold-changes, I need to use 2 different mixes. Only one mix should be used per sample. The manual suggests either adding mix 1 in a reference and mix 2 in all other samples or randomly assigning mixes 1 and 2 among samples. The latter is not very clear to me. Should I use the same mix for all replicates within a condition or should I randomise really among all the samples? Or can I randomise among conditions (for example 1, 3, 4 get mix 1 and 2, 5, 6 get mix 2)? So far, I am intending to do the latter, so using mix 1 in conditions 1, 3, 4 and mix 2 in conditions 2, 5, 6. This would also be cheaper because I can only buy equal volumes of mix 1 and 2. Do people have any advice on this?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-02-17T00:22:28.237" Title="Optimal design for an RNA-seq experiment with ERCC RNA Spike-In Control Mixes" Tags="|rna-seq|rna|quality-control|spike-in|ercc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3560" PostTypeId="2" ParentId="3559" CreationDate="2018-02-16T19:44:47.647" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I have 6 conditions and 3 replicates in each condition.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If including more biological replicates has a low experimental cost, you should be using 6 replicates per condition (and fewer reads per condition, i.e. maintaining the total number of reads) in order to pick up the majority of differentially-expressed genes. For that, I'd recommend using 3 lanes of a flow cell, with 12 dual-indexed barcodes (with the same index on each end) for each lane. Doing dual indexing will protect a bit against index switching and make the results more robust.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;randomly assigning mixes 1 and 2 among samples. The latter is not very clear to me. Should I use the same mix for all replicates within a condition or should I randomise really among all the samples?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Randomly assign mixes among and within the conditions. Make sure that the mixes are equally represented (as much as possible) across the conditions, but that each condition has at least one of each mix. This can be done using something like the following in R:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;numReplicates &amp;lt;- 7;&#xA;numEachMix &amp;lt;- ceiling(numReplicates / 2);&#xA;assignments &amp;lt;- sample(rep(c(&quot;mix1&quot;,&quot;mix2&quot;),numEachMix))[1:numReplicates]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For working out the level of fold change this is reliably replicated (or &quot;statistically significant&quot;), you can split the samples into mix 1 and mix 2, then apply the same algorithms for ERCC differential expression as with the non-ERCC genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, as ERCC is typically added in proportion to the amount of RNA (or, if you're lucky and have cell-sorted populations, possibly the number of cells), all it's really testing is the accuracy of sample prep.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-02-17T00:22:28.237" LastActivityDate="2018-02-17T00:22:28.237" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3561" PostTypeId="1" CreationDate="2018-02-16T20:03:26.460" Score="3" ViewCount="2205" Body="&lt;p&gt;this is a very basic question, but I cannot find it explicitly stated anywhere: When exactly should I filter the info scores after imputation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did imputation in Impute2 and am planning on using SNPtest to analyze, but cannot tell if I should filter the scores in Impute2, SNPtest, Gtool, QCtool, or through python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm hoping to get rid of the poorly imputed results (with an info score cutoff &amp;lt;0.5) in a population of ~4000 (half cases, half controls). After this, I'm running an association analysis -- probably through PLINK, but I'm still figuring out the best way to go from Impute2 to PLINK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be greatly appreciated!!&lt;/p&gt;&#xA;" OwnerUserId="2368" LastEditorUserId="73" LastEditDate="2018-02-23T23:41:34.767" LastActivityDate="2018-02-25T21:32:18.797" Title="How to filter info score post-imputation?" Tags="|gwas|imputation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3562" PostTypeId="2" ParentId="3561" CreationDate="2018-02-16T21:43:29.387" Score="2" Body="&lt;p&gt;In simple GWAS setups, each SNP is analyzed independently.  In those cases you can filter out SNPs with poor INFO scores at any point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For analyses that combine information across SNPs (for example FaSTLMM), I would recommend filtering out SNPs before running the step that combines information across SNPs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if you filter right after running Impute2, you should be safe.&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2018-02-16T21:43:29.387" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3563" PostTypeId="1" AcceptedAnswerId="3564" CreationDate="2018-02-16T22:34:45.990" Score="4" ViewCount="225" Body="&lt;p&gt;In &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/3194/1771&quot;&gt;this answer&lt;/a&gt; (actually in a comment), it is stated that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;As you've noticed from your own analysis, the ribosomal genes have quite variable expression across cells. They're expressed everywhere, and quite &lt;strong&gt;difficult to completely remove from a sample (even with polyA selection)&lt;/strong&gt;, with success of removal depending on things like the amount of mechanical damage that the RNA has been exposed to.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Which are the most relevant issues that make Poly(A) selection fail? Looking online, Poly(A) &lt;a href=&quot;https://www.researchgate.net/post/For_RNAseq_analysis_what_would_you_advise_poly-A_selection_or_rRNA_depletion2&quot; rel=&quot;nofollow noreferrer&quot;&gt;is suggested&lt;/a&gt; often but I didn't find an explanation for the different ways in which it can fail.&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2018-02-17T02:06:20.613" LastActivityDate="2018-02-18T18:04:16.227" Title="Why is ribosomal RNA difficult to remove even with Poly(A) selection?" Tags="|rna-seq|scrnaseq|rna|ribosomal|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3564" PostTypeId="2" ParentId="3563" CreationDate="2018-02-17T02:15:21.587" Score="5" Body="&lt;p&gt;We've found ribosomal RNA to be less of a problem with sequencing that depends on polyA, which suggests the issue might be in the library preparation, rather than the selection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many polyA RNA library preparation methods involve amplification, rather than selection, which means that existing transcripts that are present in very high abundance (such as rRNA) will still show up in the mixture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But polyA-tailed RNA isn't the only RNA present in a sample. For a comprehensive look at RNA, the whole-RNA sample preparation methods can't use &lt;em&gt;any&lt;/em&gt; form of selection to identify target molecules. In this case, it's typically recommended to use a form of ribosomal RNA removal, where a probe to ribosomal RNA is attached to magnetic beads (or a column, or similar), and used to fish out out ribosomal RNA, leaving the remainder behind. Unfortunately those probes don't work perfectly, either due to the ribosomal RNA being damaged at the probe site, or the probe not matching the ribosomal RNA molecule, or the RNA just not getting into the right physical state to bind with the probe. All these things mean incomplete removal of ribosomal RNA, ending up with a situation where the highly-abundant ribosomal RNA still pokes through in the reads.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-02-18T18:04:16.227" LastActivityDate="2018-02-18T18:04:16.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3565" PostTypeId="1" AcceptedAnswerId="3566" CreationDate="2018-02-17T12:00:17.873" Score="6" ViewCount="4152" Body="&lt;p&gt;There are many cases whereby I would like to subset a BAM to create a small file in order to work with (e.g. algorithmic testing, debugging, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normally I do the following, which will subset the BAM &lt;code&gt;file.bam&lt;/code&gt; and keep the header&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -H file.bam &amp;gt; header.sam&#xA;samtools view file.bam | head -n 5000 | cat header.sam - | samtools view -Sb - &amp;gt; file.unique.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this case, I would like 5000 rows in chromosome 1 and 5000 in chromosome 2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could first try grepping by individual chromosome, and then combining the two SAMs&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g. here's complete BAM with grepped chr1 and (incorrect but complete) header&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -H file.bam &amp;gt; header.sam&#xA;samtools view file.bam | grep &quot;chr1&quot; | cat header.sam - | samtools view -Sb - &amp;gt; file.unique.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but then I have two problems: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) I may not be grepping the alignments to chromsome 2---there may exist BAM rows which contain 'chr2' but are not alignments. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) I think one must manually edit the header. There's probably no way around this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an easy way, Bioinformatics SO? &lt;/p&gt;&#xA;" OwnerUserId="1770" LastActivityDate="2018-02-26T23:22:00.417" Title="Subset smaller BAM to contain several thousand rows from multiple chromosomes" Tags="|sam|samtools|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3566" PostTypeId="2" ParentId="3565" CreationDate="2018-02-17T12:29:12.107" Score="3" Body="&lt;p&gt;If you're not too hung up on exact numbers like 5000 reads then you can do that with a single samtools command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -bo subset.bam -s 123.4 alignments.bam chr1 chr2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That will select 40% (the &lt;code&gt;.4&lt;/code&gt; part) of the reads (&lt;code&gt;123&lt;/code&gt; is a seed, which is convenient for reproducibility). The convenient part of this is that it'll keep mates paired if you have paired-end reads. For 5000 reads per chromosome just change the &lt;code&gt;.4&lt;/code&gt; part to a sufficiently small number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general you don't really need to subset the header. Some tools will perform a bit better if you do, but you'll generally get the same results regardless.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-02-17T12:29:12.107" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3567" PostTypeId="2" ParentId="3565" CreationDate="2018-02-17T15:20:09.120" Score="2" Body="&lt;p&gt;I'd generally recommend Devon Ryan's answer. However, if you do care about having the same number from each Chr, you could use the following python/pysam code (this will output approx 5000 from every Chr):&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;from pysam import AlignmentFile&#xA;from random import random&#xA;&#xA;nreads = 5000&#xA;&#xA;infile = AlignmentFile(&quot;mybam.bam&quot;)&#xA;outfile = AlignmentFile(&quot;outbam.bam&quot;, &quot;wb&quot;, template=infile)&#xA;&#xA;for chr in infile.reference_names:&#xA;     reads_in_chr = infile.count(chr)&#xA;     frac = float(nreads)/reads_in_chr&#xA;     count = 0&#xA;     #-- Replace this loop for the exactly 5000 first reads --#&#xA;     for read in infile.fetch(chr, multiple_iterators=True):&#xA;        if random() &amp;lt; frac:&#xA;            count += 1&#xA;            outfile.write(read)&#xA;     print(&quot;ouputted %i reads from %s&quot; %(count, chr))&#xA;outfile.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This uses all chromosomes. If you only want to use, say chr1 and chr2 replace &lt;code&gt;infile.reference_names&lt;/code&gt; with &lt;code&gt;[&quot;chr1&quot;,&quot;chr2&quot;]&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want exactly 5000 reads from each chr, but don't care if they are the first ones or not, then you could replace the inner for loop with:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;for read in infile.fetch(chr, multiple_iterators=True):&#xA;    count += 1&#xA;    if count &amp;gt;= nreads:&#xA;        break&#xA;    outfile.write(read)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;if you wanted the mates of those reads as well, you could add the following after &lt;code&gt;outfile.write(read)&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;mate_read = infile.mate(read)&#xA;outfile.write(mate_read)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that this is slow.&lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-02-17T15:20:09.120" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3568" PostTypeId="1" AcceptedAnswerId="3569" CreationDate="2018-02-18T10:36:18.270" Score="6" ViewCount="4896" Body="&lt;p&gt;I'm working with rna-seq samples. I see 5' bias and also 3' bias in the per-base sequence content plot. From this &lt;a href=&quot;https://sequencing.qcfail.com/articles/positional-sequence-bias-in-random-primed-libraries/&quot; rel=&quot;noreferrer&quot;&gt;link&lt;/a&gt; I see that the bias at the start of the sequences appears to be the result of biased selection of fragments from the library. And this won't effect any downstream analysis. But In all of my samples I also see bias at the end of the sequences. What can be the problem? Samples are strand-specific sequenced with poly-A selection protocol.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Transcript coverage profile I see 5'-3' bias for all my samples is: 0.9&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Showing the per-base sequence plot for one of the sample: For all the samples I see that 5' and 3' bias.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I looked at Adapter content this is what I see:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No samples found with any adapter contamination &gt; 0.1%&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OlEp0.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OlEp0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1377" LastActivityDate="2020-05-20T09:22:48.993" Title="5' and 3' bias in Rna-seq data" Tags="|rna-seq|r|fastq|rna|rna-alignment|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3569" PostTypeId="2" ParentId="3568" CreationDate="2018-02-18T14:34:53.030" Score="7" Body="&lt;p&gt;I expect there was a sequencing problem during the last base, where some of the reagents were running low on the sequencer. This won't pose any real problem, RNAseq aligners like STAR will just soft-clip the last base or two if they're mismatches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's common to see a bit of bias toward the 5' or 3' ends in RNAseq, mostly due to whether poly-A selection was done or there was a bit of degradation. The primary thing to ensure is that the bias is similar across samples/groups. If that's the case then you don't have to worry that your downstream analyses are going be affected by some sort of per-group bias.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-02-18T14:34:53.030" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3570" PostTypeId="1" AcceptedAnswerId="3571" CreationDate="2018-02-18T21:19:59.350" Score="1" ViewCount="88" Body="&lt;p&gt;I have a set of gene names in HGNC standard and I want to get all their exons (in BED format) from the UCSC browser but it seems like the browser doesn't index genes by their HGNC name. How can I do that?&lt;/p&gt;&#xA;" OwnerUserId="122" LastActivityDate="2018-02-19T14:28:25.703" Title="Get gene exons from UCSC browser by HGNC name" Tags="|ucsc|genome-browser|exon|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3571" PostTypeId="2" ParentId="3570" CreationDate="2018-02-18T21:31:43.607" Score="2" Body="&lt;p&gt;You could grab exons via Gencode v27 (hg38) GFF annotations and convert them to BED via &lt;a href=&quot;https://github.com/bedops/bedops&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS&lt;/a&gt; &lt;code&gt;convert2bed&lt;/code&gt;/&lt;code&gt;gff2bed&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ wget -qO- ftp://ftp.sanger.ac.uk/pub/gencode/Gencode_human/release_27/gencode.v27.annotation.gff3.gz \&#xA;    | gunzip --stdout - \&#xA;    | awk '$3 == &quot;exon&quot;' \&#xA;    | convert2bed -i gff - \&#xA;    &amp;gt; exons.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then use &lt;code&gt;grep&lt;/code&gt; to filter exons against your list of gene names:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -w -f hgnc.txt exons.bed &amp;gt; exons.filtered.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="776" LastActivityDate="2018-02-18T21:31:43.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3572" PostTypeId="1" CreationDate="2018-02-18T21:47:27.917" Score="3" ViewCount="2708" Body="&lt;p&gt;I would be grateful if someone could take a quick look at these FASTQC results. This is rna-seq paired-end data. From the FASTQC manual, an unusual distribution seems to be suggestive of contamination and a shift in the curve is suggestive of a systematic bias. GC content distribution both pre-alignment and post-alignment are strange. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Samples are paired end, strand specific and % of mapped reads is above 95% for all the samples. There is no adapter content also. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could be the problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZAO4g.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZAO4g.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ZwwdL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZwwdL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1369" LastEditorUserId="1369" LastEditDate="2018-04-19T08:56:50.807" LastActivityDate="2018-04-19T08:56:50.807" Title="Strange per sequence GC content results" Tags="|rna-seq|sequence-alignment|read-mapping|rna|" AnswerCount="1" CommentCount="13" ContentLicense="CC BY-SA 3.0" />
  <row Id="3573" PostTypeId="2" ParentId="3572" CreationDate="2018-02-18T23:15:08.713" Score="3" Body="&lt;p&gt;Although generally I recommend people to be wary about the warnings that FastQC gives (it tends to be overly paranoid), the GC content graphs here do look odd. It's good that you've had a look at it with another program to confirm the observation. They should have a distribution that is fairly close to normal, a bit like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Hwnvx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Hwnvx.png&quot; alt=&quot;GC content from mouse cDNA&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without more information about your sample, I can't tell whether or not this is expected from your input. This looks like fairly substantial contamination, but could also be caused by biased selection during sample preparation (e.g. low quality sample, low yield, PCR amplification bias), or just looking at a multi-organism sample rather than a single-organism sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a contamination check, I'd recommend running your RNASeq reads through a kmer-based classification tool (e.g. &lt;a href=&quot;https://www.onecodex.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OneCodex&lt;/a&gt;, &lt;a href=&quot;https://ccb.jhu.edu/software/centrifuge/manual.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;Centrifuge&lt;/a&gt;, &lt;a href=&quot;https://www.biostars.org/p/234837/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BBSketch&lt;/a&gt;, or &lt;a href=&quot;https://github.com/dib-lab/sourmash&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sourmash&lt;/a&gt;). This should tell you the most likely place that contaminant sequences came from.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-02-18T23:15:08.713" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3576" PostTypeId="1" CreationDate="2018-02-19T11:40:17.363" Score="2" ViewCount="254" Body="&lt;p&gt;I have whole genome sequence (human) data (SNP data), but don't have phenotype data ready for any traits yet. Hopefully, I will get real phenotype data soon. To run GWAS (genome-wide-association study) or any other analysis, i want to create artificial/dummy data for any trait, let's say human height. Could anyone please recommend what is best option? Does plink generate any dummy phenotype data? or any other programme?  &lt;/p&gt;&#xA;" OwnerUserId="2365" LastActivityDate="2018-02-19T16:40:12.030" Title="How to create artificial or dummy phenotype data" Tags="|r|plink|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3577" PostTypeId="1" CreationDate="2018-02-19T13:04:08.697" Score="1" ViewCount="206" Body="&lt;p&gt;I am using the seqret tool from the Emboss suite to transform produce an EMBL annotation file from a GTF annotation file and the corresponding FASTA sequence:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seqret -sequence ecoli.fa -feature -fformat gff -fopenfile ecoli.gtf -osformat embl -auto -outseq ecoli.embl&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It produces my desired EMBL file but it warns against a 'bad protein_id':&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;Warning: bad /protein_id value '&quot;CCE57732&quot;'&#xA;Warning: bad /protein_id value '&quot;CCE57733&quot;'&#xA;Warning: bad /protein_id value '&quot;CCE57734&quot;'&#xA;Warning: bad /protein_id value '&quot;CCE57735&quot;'&#xA;Warning: bad /protein_id value '&quot;CCE57754&quot;'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What's wrong with these protein IDs? The organism I am working with is &lt;em&gt;Escherichia coli&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are a few lines from my GTF file. The first line has a protein_id.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Chromosome      ena     CDS     349     2808    .       +       0       gene_id &quot;HUS2011_0001&quot;; transcript_id &quot;CCQ26880&quot;; exon_number &quot;1&quot;; gene_name &quot;thrA&quot;; gene_source &quot;ena&quot;; gene_biotype &quot;protein_coding&quot;; transcript_name &quot;thrA-1&quot;; transcript_source &quot;ena&quot;; transcript_biotype &quot;protein_coding&quot;; protein_id &quot;CCQ26880&quot;;&#xA;Chromosome      ena     start_codon     349     351     .       +       0       gene_id &quot;HUS2011_0001&quot;; transcript_id &quot;CCQ26880&quot;; exon_number &quot;1&quot;; gene_name &quot;thrA&quot;; gene_source &quot;ena&quot;; gene_biotype &quot;protein_coding&quot;; transcript_name &quot;thrA-1&quot;; transcript_source &quot;ena&quot;; transcript_biotype &quot;protein_coding&quot;;&#xA;Chromosome      ena     stop_codon      2809    2811    .       +       0       gene_id &quot;HUS2011_0001&quot;; transcript_id &quot;CCQ26880&quot;; exon_number &quot;1&quot;; gene_name &quot;thrA&quot;; gene_source &quot;ena&quot;; gene_biotype &quot;protein_coding&quot;; transcript_name &quot;thrA-1&quot;; transcript_source &quot;ena&quot;; transcript_biotype &quot;protein_coding&quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-02-20T16:11:49.673" LastActivityDate="2018-02-20T16:52:38.300" Title="seqret: Warning: bad /protein_id value" Tags="|file-formats|format-conversion|gtf|emboss|seqret|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3578" PostTypeId="2" ParentId="3565" CreationDate="2018-02-19T13:49:23.270" Score="1" Body="&lt;p&gt;using &lt;strong&gt;samjdk&lt;/strong&gt;: &lt;a href=&quot;http://lindenb.github.io/jvarkit/SamJdk.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://lindenb.github.io/jvarkit/SamJdk.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ java -jar dist/samjdk.jar --body -e \&#xA;  'Map&amp;lt;String,Integer&amp;gt; c=new HashMap&amp;lt;&amp;gt;(); public Object apply(SAMRecord r) {int n=c.getOrDefault(r.getContig(),0);if(n&amp;gt;=5000) return false; c.put(r.getContig(),n+1); return r;}' \&#xA; input.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastActivityDate="2018-02-19T13:49:23.270" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3579" PostTypeId="2" ParentId="3570" CreationDate="2018-02-19T14:28:25.703" Score="2" Body="&lt;p&gt;If you need a 3-column BED, you can just use standard Unix tools:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;curl ftp://ftp.sanger.ac.uk/pub/gencode/Gencode_human/release_27/gencode.v27.annotation.gtf.gz \&#xA;  | gzip -dc \&#xA;  | grep -wFf list.txt \&#xA;  | awk -vFS=&quot;\t&quot; '$3==&quot;exon&quot;{print $1&quot;\t&quot;($4-1)&quot;\t&quot;$5&quot;\t&quot;$9}' &amp;gt; out-3.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you need a 12-column BED:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;# install k8 and paftools.js&#xA;curl -L http://bit.ly/k8-024 | tar -jxf -&#xA;cp k8-0.2.4/k8-`uname -s` k8&#xA;git clone https://github.com/lh3/minimap2&#xA;# actual command line:&#xA;curl ftp://ftp.sanger.ac.uk/pub/gencode/Gencode_human/release_27/gencode.v27.annotation.gtf.gz \&#xA;  | ./k8 minimap2/misc/paftools.js gff2bed - \&#xA;  | grep -wFf list.txt &amp;gt; out-12.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-02-19T14:28:25.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3580" PostTypeId="2" ParentId="3576" CreationDate="2018-02-19T16:40:12.030" Score="3" Body="&lt;p&gt;Using &lt;a href=&quot;https://CRAN.R-project.org/package=wakefield&quot; rel=&quot;nofollow noreferrer&quot;&gt;wakefield package&lt;/a&gt; it is pretty easy, here is an example from &lt;a href=&quot;https://github.com/trinker/wakefield&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub page&lt;/a&gt;. This will create 500 dummy variables for id, race, age, etc. :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(wakefield)&#xA;&#xA;myDummyPheno &amp;lt;- r_data_frame(&#xA;  n = 500,&#xA;  id,&#xA;  race,&#xA;  age,&#xA;  sex,&#xA;  hour,&#xA;  iq,&#xA;  height,&#xA;  died&#xA;)&#xA;&#xA;# result&#xA;myDummyPheno&#xA;&#xA;# # A tibble: 500 x 8&#xA;#    ID    Race        Age Sex    Hour           IQ Height Died &#xA;#   &amp;lt;chr&amp;gt;  &amp;lt;fct&amp;gt;     &amp;lt;int&amp;gt; &amp;lt;fct&amp;gt;  &amp;lt;S3: times&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt; &amp;lt;lgl&amp;gt;&#xA;#  1 001   Bi-Racial    27 Male   00:00:00     93.0   71.0 F    &#xA;#  2 002   White        28 Female 00:00:00    104     64.0 T    &#xA;#  3 003   Black        28 Male   00:00:00    111     67.0 T    &#xA;#  4 004   Hispanic     35 Male   00:00:00     90.0   67.0 T    &#xA;#  5 005   Hispanic     21 Female 00:00:00     94.0   70.0 T    &#xA;#  6 006   Black        35 Male   00:00:00     74.0   73.0 F    &#xA;#  7 007   Black        27 Male   00:00:00    102     68.0 T    &#xA;#  8 008   White        28 Female 00:00:00     95.0   70.0 F    &#xA;#  9 009   White        30 Female 00:00:00     80.0   71.0 F    &#xA;# 10 010   White        30 Female 00:00:00    114     66.0 T    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="131" LastActivityDate="2018-02-19T16:40:12.030" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3581" PostTypeId="1" AcceptedAnswerId="4446" CreationDate="2018-02-19T17:25:02.647" Score="5" ViewCount="3811" Body="&lt;p&gt;I have 4 different single-cell RNAseq experiments, each one representing a different sample of cell type population. I'd like to merge them to a single dataset. However, different cell types are enriched in each sample, and different samples can represent also different conditions (control and disease).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I run a PCA on the merged dataset (without any processing of data) I find that the different sample's cells cluster separately. This could be due to biological and technical (i.e. batch effects) variation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to merge these different samples while retaining a good amount of information or should they be analyzed separately?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2019-06-27T03:11:00.333" Title="Is it possible to merge scRNAseq data from experiments with different design?" Tags="|rna-seq|scrnaseq|batch-effects|merge|" AnswerCount="3" CommentCount="9" ContentLicense="CC BY-SA 3.0" />
  <row Id="3582" PostTypeId="1" CreationDate="2018-02-20T04:17:28.313" Score="2" ViewCount="77" Body="&lt;p&gt;I am wondering how to interpret values inside the gene expression levels contained within Bioconductor. For example, if we have the following commands via &lt;code&gt;R&lt;/code&gt;, we get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;source(&quot;https://bioconductor.org/biocLite.R&quot;) &#xA;biocLite(&quot;curatedOvarianData&quot;)&#xA;library(curatedOvarianData)&#xA;data(TCGA_eset)&#xA;head(exprs(TCGA_eset)[&quot;CXCL12&quot;,])&#xA;&#xA;TCGA.20.0987 TCGA.23.1031 TCGA.24.0979 TCGA.23.1117 TCGA.23.1021 TCGA.04.1337 &#xA;    5.437923     6.307250     4.193748     9.071293     5.101532     6.697770 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above calls out gene expression levels for the gene &lt;code&gt;CXCL12&lt;/code&gt;. However, I am wondering how to interpret the values &lt;code&gt;5.437923&lt;/code&gt;, &lt;code&gt;6.307250&lt;/code&gt;, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know they are expression levels, but do they come with units? Do larger values mean greater cancer possibility?&lt;/p&gt;&#xA;" OwnerUserId="2374" LastActivityDate="2018-02-20T20:22:05.553" Title="When working with Bioconductor and the curatedOvarianData dataset, what are the unit values of the expression data?" Tags="|r|bioconductor|gene|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3583" PostTypeId="1" AcceptedAnswerId="3585" CreationDate="2018-02-20T05:10:43.110" Score="21" ViewCount="9498" Body="&lt;p&gt;I am writing a python script that requires a reverse complement function to be called on DNA strings of length 1 through around length 30. Line profiling programs indicate that my functions spend a lot of time getting the reverse complements, so I am looking to optimize.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the fastest way to get the reverse complement of a sequence in python? I am posting my skeleton program to test different implementations below with DNA string size 17 as an example.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;import random&#xA;import timeit&#xA;&#xA;global complement&#xA;complement = {'A': 'T', 'C': 'G', 'G': 'C', 'T': 'A'}&#xA;&#xA;DNAlength = 17&#xA;&#xA;#randomly generate 100k bases&#xA;int_to_basemap = {1: 'A', 2: 'C', 3: 'G', 4: 'T'}&#xA;num_strings = 500000&#xA;random.seed(90210)&#xA;DNAstrings = [&quot;&quot;.join([int_to_basemap[random.randint(1,4)] for i in range(DNAlength)])&#xA;              for j in range(num_strings)]&#xA;#get an idea of what the DNAstrings look like&#xA;print(DNAstrings[0:5])&#xA;&#xA;def reverse_complement_naive(seq):&#xA;    this_complement = {'A': 'T', 'C': 'G', 'G': 'C', 'T': 'A'}&#xA;    return &quot;&quot;.join(this_complement.get(base, base) for base in reversed(seq))&#xA;&#xA;def reverse_complement(seq):&#xA;    return &quot;&quot;.join(complement.get(base, base) for base in reversed(seq))&#xA;&#xA;&#xA;tic=timeit.default_timer()&#xA;rcs = [reverse_complement_naive(seq) for seq in DNAstrings]&#xA;toc=timeit.default_timer()&#xA;baseline = toc - tic&#xA;namefunc = {&quot;naive implementation&quot;: reverse_complement_naive,&#xA;            &quot;global dict implementation&quot;: reverse_complement}&#xA;&#xA;for function_name in namefunc:&#xA;    func = namefunc[function_name]&#xA;    tic=timeit.default_timer()&#xA;    rcs = [func(seq) for seq in DNAstrings]&#xA;    toc=timeit.default_timer()&#xA;    walltime = toc-tic&#xA;    print(&quot;&quot;&quot;{}&#xA;       {:.5f}s total,&#xA;       {:.1f} strings per second&#xA;       {:.1f}% increase over baseline&quot;&quot;&quot;.format(&#xA;           function_name,&#xA;           walltime,&#xA;           num_strings/walltime,&#xA;           100- ((walltime/baseline)*100)   ))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;By the way, I get output like this. It varies by the call, of course!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;naive implementation&#xA;       1.83880s total,&#xA;       271916.7 strings per second&#xA;       -0.7% increase over baseline&#xA;global dict implementation&#xA;       1.74645s total,&#xA;       286294.3 strings per second&#xA;       4.3% increase over baseline&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Edit: Great answers, everyone! When I get a chance in a day or two I will add all of these to a test file for the final run. When I asked the question, I had not considered whether I would allow for cython or c extensions when selecting the final answer. What do you all think?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit 2: Here are the results of the final simulation with everyone's implementations. I am going to accept the highest scoring pure python code with no Cython/C. For my own sake I ended up using user172818's c implementation. If you feel like contributing to this in the future, &lt;a href=&quot;https://github.com/conchoecia/fastest_rc_python&quot; rel=&quot;noreferrer&quot;&gt;check out the github page I made for this question&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;the runtime of reverse complement implementations.&#xA;10000 strings and 250 repetitions&#xA;╔══════════════════════════════════════════════════════╗&#xA;║ name                       %inc   s total  str per s ║&#xA;╠══════════════════════════════════════════════════════╣&#xA;║ user172818 seqpy.c        93.7%  0.002344  4266961.4 ║&#xA;║ alexreynolds Cython (v2)  93.4%  0.002468  4051583.1 ║&#xA;║ alexreynolds Cython (v1)  90.4%  0.003596  2780512.1 ║&#xA;║ devonryan string          86.1%  0.005204  1921515.6 ║&#xA;║ jackaidley bytes          84.7%  0.005716  1749622.2 ║&#xA;║ jackaidley bytesstring    83.0%  0.006352  1574240.6 ║&#xA;║ global dict                5.4%  0.035330   283046.7 ║&#xA;║ revcomp_translateSO       45.9%  0.020202   494999.4 ║&#xA;║ string_replace            37.5%  0.023345   428364.9 ║&#xA;║ revcom from SO            28.0%  0.026904   371694.5 ║&#xA;║ naive (baseline)           1.5%  0.036804   271711.5 ║&#xA;║ lambda from SO           -39.9%  0.052246   191401.3 ║&#xA;║ biopython seq then rc    -32.0%  0.049293   202869.7 ║&#xA;╚══════════════════════════════════════════════════════╝&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-02-22T05:35:59.163" LastActivityDate="2020-11-08T15:01:38.253" Title="What is the fastest way to get the reverse complement of a DNA sequence in python?" Tags="|python|phylogenetics|benchmarking|" AnswerCount="8" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3584" PostTypeId="2" ParentId="3582" CreationDate="2018-02-20T05:29:25.307" Score="2" Body="&lt;p&gt;These values represent the expression of a particular gene in each sample. While expression of an immune system gene like &lt;code&gt;CXLC12&lt;/code&gt; might be associated with the aggressiveness of a cancer, that's not going to be the case for the majority of genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at the numbers, these look like they are probably log$_2$-transformed values. In order to know for sure, plotting a density distribution of the dataset is a good idea:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plot(density(exprs(TCGA_eset)));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If the values are in the range of around 0..15, then it'll be log$_2$-transformed values, whereas if they're in the range of 0..100000, it'll be &quot;raw&quot; expression values. The log-transformed values are preferred for data analysis because gene expression tends to be close to a normal distribution in log-space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These numbers are best treated as relative counts (i.e. the most appropriate &lt;em&gt;unit&lt;/em&gt; is &quot;expression&quot;), as variation in the preparation of the sample (or the sequencer, or the microarray) can influence the results that come out the other end. Converting them to something more meaningful (e.g. &quot;transcripts per cell&quot;) requires normalising against a known reference sample.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-02-20T20:04:29.387" LastActivityDate="2018-02-20T20:04:29.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3585" PostTypeId="2" ParentId="3583" CreationDate="2018-02-20T07:34:17.150" Score="23" Body="&lt;p&gt;I don't know if it's the fastest, but the following provides an approximately 10x speed up over your functions:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import string&#xA;tab = string.maketrans(&quot;ACTG&quot;, &quot;TGAC&quot;)&#xA;&#xA;def reverse_complement_table(seq):&#xA;    return seq.translate(tab)[::-1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The thing with hashing is that it adds a good bit of overhead for a replacement set this small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For what it's worth, I added that to your code as &quot;with a translation table&quot; and here is what I got on my workstation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;global dict implementation&#xA;       1.37599s total,&#xA;       363374.8 strings per second&#xA;       3.3% increase over baseline&#xA;naive implementation&#xA;       1.44126s total,&#xA;       346919.4 strings per second&#xA;       -1.3% increase over baseline&#xA;with a translation table&#xA;       0.16780s total,&#xA;       2979755.6 strings per second&#xA;       88.2% increase over baseline&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you need python 3 rather than python 2, then substitute &lt;code&gt;tab = str.maketrans(&quot;ACTG&quot;, &quot;TGAC&quot;)&lt;/code&gt; for &lt;code&gt;tab = string.maketrans(&quot;ACTG&quot;, &quot;TGAC&quot;)&lt;/code&gt;, since &lt;code&gt;maketrans&lt;/code&gt; is now a static method on the &lt;code&gt;str&lt;/code&gt; type.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For those wondering, using biopython is slower for this (~50% slower than the naive implementation), presumably due to the overhead of converting the strings to &lt;code&gt;Seq&lt;/code&gt; objects. If one were already reading sequences in using biopython, though, I wouldn't be surprised if the performance was much different.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="2385" LastEditDate="2018-02-20T21:36:01.947" LastActivityDate="2018-02-20T21:36:01.947" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3586" PostTypeId="2" ParentId="3583" CreationDate="2018-02-20T10:04:05.030" Score="7" Body="&lt;p&gt;The most reliable and simplest way is probably using &lt;code&gt;Biopython&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.Seq import Seq&#xA;&#xA;def my_reverse_complement(seq):&#xA;    return Seq(seq).reverse_complement()&#xA;&#xA;print(my_reverse_complement('ATGCGTA'))&#xA;# TACGCAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As Devon has already &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/3585/104&quot;&gt;said here&lt;/a&gt; using &lt;code&gt;Biopython&lt;/code&gt; isn't as fast as the naive Python solution, and I also tested that &lt;a href=&quot;https://pastebin.com/Vupe6aQn&quot; rel=&quot;nofollow noreferrer&quot;&gt;shown here&lt;/a&gt; with ipython. However, this is because &lt;code&gt;Biopython&lt;/code&gt;'s implementation, although similar to the naive approach, includes other features; it can reverse complement RNA as well as DNA and it will tell you if you're mixing DNA and RNA&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that if you really want a fast way you could look at &lt;code&gt;Cython&lt;/code&gt; or another python extension. As I edit this now, there are several nice answers taking this approach from user172818 and Alex Reynolds. &lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="104" LastEditDate="2018-02-21T22:36:42.120" LastActivityDate="2018-02-21T22:36:42.120" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3587" PostTypeId="2" ParentId="3583" CreationDate="2018-02-20T10:07:46.673" Score="9" Body="&lt;p&gt;Here's a Cython approach that might suggest a generic approach to speeding up Python work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're manipulating (ASCII) character strings and performance is a design consideration, then C or Perl are probably preferred options to Python. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, this Cython test uses Python 3.6.3:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ python --version&#xA;Python 3.6.3 :: Anaconda custom (64-bit)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To install Cython, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ conda install -c anaconda cython&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The Cython code below seems to offer about the same speed bump as the translation table — perhaps similar code is run under the hood of that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two files are needed, starting with &lt;code&gt;setup.py&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from distutils.core import setup&#xA;from Cython.Build import cythonize&#xA;&#xA;setup(&#xA;  name = 'Reverse complement C test',&#xA;  ext_modules = cythonize(&quot;revcomp_c.pyx&quot;),&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then a second file called &lt;code&gt;revcomp_c.pyx&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from libc.stdlib cimport malloc&#xA;&#xA;cdef int seq_len = 17&#xA;cdef char *seq_dest = &amp;lt;char *&amp;gt;malloc(seq_len + 1)&#xA;seq_dest[seq_len] = '\0'&#xA;&#xA;def reverse_complement_c_v1(str seq):&#xA;    cdef bytes py_bytes = seq.encode('ascii')&#xA;    cdef char *seq_src = py_bytes&#xA;    cdef int i = 0&#xA;    cdef int d = 0&#xA;    for i in range(seq_len):&#xA;        d = seq_len - i - 1&#xA;        if seq_src[i] == 'A':&#xA;            seq_dest[d] = 'T'&#xA;        elif seq_src[i] == 'G':&#xA;            seq_dest[d] = 'C'&#xA;        elif seq_src[i] == 'C':&#xA;            seq_dest[d] = 'G'&#xA;        elif seq_src[i] == 'T':&#xA;            seq_dest[d] = 'A'&#xA;    return seq_dest[:seq_len].decode('ascii')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This can be compiled into a Python module like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ python setup.py build_ext --inplace&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then we can modify the test bench to include this Cython module and the relevant test method:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;import random&#xA;import timeit&#xA;from revcomp_c import reverse_complement_c_v1&#xA;&#xA;global complement&#xA;complement = {'A': 'T', 'C': 'G', 'G': 'C', 'T': 'A'}&#xA;&#xA;DNAlength = 17&#xA;&#xA;#randomly generate 100k bases&#xA;int_to_basemap = {1: 'A', 2: 'C', 3: 'G', 4: 'T'}&#xA;num_strings = 500000&#xA;random.seed(90210)&#xA;DNAstrings = [&quot;&quot;.join([int_to_basemap[random.randint(1,4)] for i in range(DNAlength)])&#xA;              for j in range(num_strings)]&#xA;#get an idea of what the DNAstrings look like&#xA;print(DNAstrings[0:5])&#xA;&#xA;def reverse_complement_naive(seq):&#xA;    this_complement = {'A': 'T', 'C': 'G', 'G': 'C', 'T': 'A'}&#xA;    return &quot;&quot;.join(this_complement.get(base, base) for base in reversed(seq))&#xA;&#xA;def reverse_complement(seq):&#xA;    return &quot;&quot;.join(complement.get(base, base) for base in reversed(seq))&#xA;&#xA;tic=timeit.default_timer()&#xA;rcs = [reverse_complement_naive(seq) for seq in DNAstrings]&#xA;toc=timeit.default_timer()&#xA;baseline = toc - tic&#xA;namefunc = {&quot;naive implementation&quot;: reverse_complement_naive,&#xA;            &quot;global dict implementation&quot;: reverse_complement,&#xA;            &quot;Cython implementation (v1)&quot;: reverse_complement_c_v1}&#xA;&#xA;for function_name in namefunc:&#xA;    func = namefunc[function_name]&#xA;    tic=timeit.default_timer()&#xA;    rcs = [func(seq) for seq in DNAstrings]&#xA;    toc=timeit.default_timer()&#xA;    walltime = toc-tic&#xA;    print(&quot;&quot;&quot;{}&#xA;       {:.5f}s total,&#xA;       {:.1f} strings per second&#xA;       {:.1f}% increase over baseline&quot;&quot;&quot;.format(&#xA;           function_name,&#xA;           walltime,&#xA;           num_strings/walltime,&#xA;           100- ((walltime/baseline)*100)   ))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A sample run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ python ./revcomp_test.py&#xA;['ACTGCAATAGACCACGC', 'CAGCTTGAGCCATTAAC', 'GGCCCAAGAGTTCGAAC', 'CGACTGTCTCGAATTGT', 'ATCCGCTATGCGCCGTC']&#xA;naive implementation&#xA;       1.73295s total,&#xA;       288525.0 strings per second&#xA;       -2.5% increase over baseline&#xA;global dict implementation&#xA;       1.70896s total,&#xA;       292575.5 strings per second&#xA;       -1.0% increase over baseline&#xA;Cython implementation (v1)&#xA;       0.21458s total,&#xA;       2330158.7 strings per second&#xA;       87.3% increase over baseline&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One easy way to speed this up is to use a &lt;code&gt;static const unsigned char&lt;/code&gt; array as an ASCII lookup table, which maps a residue directly to its complement. This would replace the nest of &lt;code&gt;if&lt;/code&gt; statements and probably give a nice little boost (&lt;a href=&quot;https://bioinformatics.stackexchange.com/a/3595/776&quot;&gt;and it appears it does, making it among the best performers so far!&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my fast implementation of a reverse complement function in C:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gist.github.com/alexpreynolds/4f75cab4350e9d937f4a&quot; rel=&quot;noreferrer&quot;&gt;https://gist.github.com/alexpreynolds/4f75cab4350e9d937f4a&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might be able to use this directly in Python via the &lt;code&gt;subprocess&lt;/code&gt; library. Outsourcing the reverse complement step to a utility written in C will almost always beat the best that Python can do, and you can do nice and important things like bounds checking etc. without losing much speed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's unclear how &quot;pure&quot; the answer needs to be, but making a system call from Python seems fair if you're processing strings and your goal is performance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another direction to take may be to look at multithreading, if you don't need ordered output. If you have many thousands of sequences stored in memory, you could split an array of sequences up into smaller arrays by use of offsets or array indices. Each thread would work on &quot;rc&quot;-ing sequences in its own piece of the array.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-02-20T22:55:02.400" LastActivityDate="2018-02-20T22:55:02.400" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3588" PostTypeId="2" ParentId="3583" CreationDate="2018-02-20T13:58:48.867" Score="3" Body="&lt;p&gt;&lt;strong&gt;Use a &lt;code&gt;bytearray&lt;/code&gt; instead of a &lt;code&gt;string&lt;/code&gt; and then employ &lt;code&gt;maketrans&lt;/code&gt; to translate&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You do not need the more advanced string encoding capabilities of &lt;code&gt;string&lt;/code&gt; to store a string of bases, but you're still paying for it in performance. Devon Ryan's suggestion of &lt;code&gt;maketrans&lt;/code&gt; is the huge improvement, 10x faster than your naive implementation. Using the same approach, but swapping everything out for &lt;code&gt;byte&lt;/code&gt;s allows a further 40% speed improvement, however:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# DNA sequence generation also needs to be swapped to bytes&#xA;tab_b = bytes.maketrans(b&quot;ACTG&quot;, b&quot;TGAC&quot;)&#xA;&#xA;def reverse_complement_bytes(seq):&#xA;    return seq.translate(tab_b)[::-1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In my comparison:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Table implementation&#xA;       0.24339s total,&#xA;       2054315.7 strings per second&#xA;       82.5% increase over baseline&#xA;Bytes table implementation&#xA;       0.14985s total,&#xA;       3336755.9 strings per second&#xA;       89.2% increase over table&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2379" LastEditorUserId="2379" LastEditDate="2018-02-22T11:44:06.983" LastActivityDate="2018-02-22T11:44:06.983" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3589" PostTypeId="1" CreationDate="2018-02-20T14:45:54.137" Score="2" ViewCount="164" Body="&lt;p&gt;I want to use SNPs produced by Ipyrad, which is a python script for RADseq, using maize genome data via &lt;a href=&quot;https://sco.h-its.org/exelixis/web/software/raxml/index.html#documentation&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;A maximum likelihood approach to tree building&quot;&gt;RAxML&lt;/a&gt; to examine the monophyly of a highly-variable focal species and its phylogenetic relationships with about 40 congeners.&lt;/p&gt;&#xA;&lt;p&gt;Essentially phylogenomics using maximum likelihood trees.&lt;/p&gt;&#xA;&lt;p&gt;An initial analysis suggested that samples from two outgroup species might share few SNPs with other samples, so the threshold for shared loci was set to two samples.  As a result, the .u.snps.phy alignment has over 600,000 SNPs.&lt;/p&gt;&#xA;&lt;p&gt;For subsequent use in RAxML, I would like to implement two filters to construct alignments with subsets of SNPs:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;SNPs that occur in at least one outgroup sample and in some (potentially maximal) number of other samples,&lt;/li&gt;&#xA;&lt;li&gt;SNPs constituting a random sample from an arbitrary subset of samples, i.e. potentially excluding the outgroup samples or other samples,&lt;/li&gt;&#xA;&lt;li&gt;merge the alignments into one&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;How can I do this efficiently, maximizing use of existing functionality in R or Python? Either code or a (likely successful) strategy would be helpful. I have access to all output formats from ipyrad as potential starting points.&lt;/p&gt;&#xA;" OwnerUserId="2196" LastEditorUserId="3848" LastEditDate="2020-07-15T20:11:36.853" LastActivityDate="2020-07-15T20:11:36.853" Title="Post ipyrad filtering of SNP loci from GBS, based on SNP quality" Tags="|r|python|vcf|snp|radseq|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="3590" PostTypeId="1" AcceptedAnswerId="3597" CreationDate="2018-02-20T15:03:44.777" Score="3" ViewCount="2679" Body="&lt;p&gt;I am working on ONT data. Initially, I have worked on data from 1D using Minimap2 aligner. I came across 1D^2 from ONT website. I was wondering how different is the data from two techniques? What is the difference between 1D and 2D? and do we get 2  fastq files (one per strand) for each sample? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;TIA&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would greatly appreciate any help regarding my question. &lt;/p&gt;&#xA;" OwnerUserId="2381" LastActivityDate="2018-02-20T20:22:49.227" Title="Difference between 1D and 1D^2 data" Tags="|nanopore|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3591" PostTypeId="2" ParentId="3565" CreationDate="2018-02-20T15:08:42.420" Score="1" Body="&lt;p&gt;If you just want to create a truncated BAM file with a header then you can significantly simplify your original code:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;(samtools view -H input.bam; samtools view input.bam | head -5000) | samtools -bo output.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This command avoids the intermediate file and few gratuitous intermediate command invocations, at the cost of a subshell (invoked by &lt;code&gt;(…)&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As above, but with formatting:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;(&#xA;    samtools view -H input.bam&#xA;    samtools view input.bam | head -5000 # (*)&#xA;) \&#xA;| samtools -bo output.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;… this can of course be extended to filter by multiple chromosomes by replacing the line marked with &lt;code&gt;(*)&lt;/code&gt; above by one or multiple lines that subset by chromosome name (&lt;code&gt;samtools view input.bam chr&lt;em&gt;x&lt;/em&gt;&lt;/code&gt;, no need for &lt;code&gt;grep&lt;/code&gt; if you have indexed the original BAM file!).&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2018-02-20T15:24:00.137" LastActivityDate="2018-02-20T15:24:00.137" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3593" PostTypeId="2" ParentId="3577" CreationDate="2018-02-20T16:52:38.300" Score="1" Body="&lt;p&gt;You're supplying a GTF feature file, but telling it that the features are in GFF3 format. I think you need &lt;code&gt;-fformat gff2&lt;/code&gt;. See &lt;a href=&quot;https://www.ebi.ac.uk/Tools/sfc/emboss_seqret/help/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/Tools/sfc/emboss_seqret/help/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-02-20T16:52:38.300" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3594" PostTypeId="2" ParentId="3583" CreationDate="2018-02-20T17:51:22.480" Score="5" Body="&lt;p&gt;Another python extension but without cython. The source code is available at the bottom of this answer or &lt;a href=&quot;https://gist.github.com/attractivechaos/15a61b2517c253bb31bffcde3e45c6e8&quot; rel=&quot;noreferrer&quot;&gt;from this gist&lt;/a&gt;. On Mac with Python3:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;string&#xA;   0.38166s total,&#xA;   1310067.4 strings per second&#xA;   84.4% increase over baseline&#xA;seqpy&#xA;   0.20524s total,&#xA;   2436182.7 strings per second&#xA;   91.6% increase over baseline&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;On Linux with Python2 (seqpy is the first):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seqpy  &#xA;   0.10960s total,&#xA;   4561842.5 strings per second&#xA;   93.3% increase over baseline&#xA;string &#xA;   0.17811s total,&#xA;   2807330.9 strings per second&#xA;   89.1% increase over baseline&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This is file &lt;code&gt;setup.py&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;from setuptools import setup, Extension&#xA;&#xA;seqpy_module = Extension('seqpy', sources = [&quot;seqpy.c&quot;])&#xA;&#xA;setup(&#xA;    name = &quot;seqpy&quot;,&#xA;    description = &quot;seqpy&quot;,&#xA;    ext_modules = [seqpy_module])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is file &lt;code&gt;seqpy.c&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;stdlib.h&amp;gt;&#xA;#include &amp;lt;stdint.h&amp;gt;&#xA;#include &amp;lt;Python.h&amp;gt;&#xA;&#xA;static unsigned char seq_comp_table[256] = {&#xA;      0,   1,   2,   3,   4,   5,   6,   7,   8,   9,  10,  11,  12,  13,  14,  15,&#xA;     16,  17,  18,  19,  20,  21,  22,  23,  24,  25,  26,  27,  28,  29,  30,  31,&#xA;     32,  33,  34,  35,  36,  37,  38,  39,  40,  41,  42,  43,  44,  45,  46,  47,&#xA;     48,  49,  50,  51,  52,  53,  54,  55,  56,  57,  58,  59,  60,  61,  62,  63,&#xA;     64, 'T', 'V', 'G', 'H', 'E', 'F', 'C', 'D', 'I', 'J', 'M', 'L', 'K', 'N', 'O',&#xA;    'P', 'Q', 'Y', 'S', 'A', 'A', 'B', 'W', 'X', 'R', 'Z',  91,  92,  93,  94,  95,&#xA;     64, 't', 'v', 'g', 'h', 'e', 'f', 'c', 'd', 'i', 'j', 'm', 'l', 'k', 'n', 'o',&#xA;    'p', 'q', 'y', 's', 'a', 'a', 'b', 'w', 'x', 'r', 'z', 123, 124, 125, 126, 127,&#xA;    128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143,&#xA;    144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159,&#xA;    160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175,&#xA;    176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191,&#xA;    192, 193, 194, 195, 196, 197, 198, 199, 200, 201, 202, 203, 204, 205, 206, 207,&#xA;    208, 209, 210, 211, 212, 213, 214, 215, 216, 217, 218, 219, 220, 221, 222, 223,&#xA;    224, 225, 226, 227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239,&#xA;    240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, 253, 254, 255&#xA;};&#xA;&#xA;static PyObject *seqpy_revcomp(PyObject *self, PyObject *args)&#xA;{&#xA;    PyObject *r;&#xA;    char *seq, *rev;&#xA;    int i, len;&#xA;    PyArg_ParseTuple(args, &quot;s#&quot;, &amp;amp;seq, &amp;amp;len);&#xA;    rev = (char*)malloc(len);&#xA;    for (i = 0; i &amp;lt; len; ++i)&#xA;        rev[len - i - 1] = seq_comp_table[(uint8_t)seq[i]];&#xA;    r = Py_BuildValue(&quot;s#&quot;, rev, len);&#xA;    free(rev);&#xA;    return r;&#xA;}&#xA;&#xA;static PyMethodDef seqpy_methods[] = {&#xA;    {&quot;revcomp&quot;, seqpy_revcomp, METH_VARARGS, &quot;Reverse complement a DNA sequence&quot;},&#xA;    {NULL, NULL, 0, NULL}&#xA;};&#xA;&#xA;#if PY_MAJOR_VERSION &amp;gt;= 3&#xA;static struct PyModuleDef seqpy_module = { PyModuleDef_HEAD_INIT, &quot;seqpy&quot;, NULL, -1, seqpy_methods };&#xA;PyMODINIT_FUNC PyInit_seqpy(void) { return PyModule_Create(&amp;amp;seqpy_module); }&#xA;#else&#xA;PyMODINIT_FUNC initseqpy(void) { Py_InitModule3(&quot;seqpy&quot;, seqpy_methods, NULL); }&#xA;#endif&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To compile and test:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python setup.py build_ext -i&#xA;python -c 'import seqpy; print(seqpy.revcomp(&quot;GGGTT&quot;))'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2018-02-20T17:59:43.647" LastActivityDate="2018-02-20T17:59:43.647" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3595" PostTypeId="2" ParentId="3583" CreationDate="2018-02-20T18:55:16.333" Score="3" Body="&lt;p&gt;Here is a revision of &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/3587/776&quot;&gt;my original Cython answer&lt;/a&gt; which incorporates my suggestion to use a &lt;code&gt;char&lt;/code&gt; lookup array:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from libc.stdlib cimport malloc&#xA;&#xA;cdef int seq_len = 17&#xA;cdef char *seq_dest = &amp;lt;char *&amp;gt;malloc(seq_len + 1)&#xA;seq_dest[seq_len] = '\0'&#xA;&#xA;cdef char *basemap = [ '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0',   '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0',  'T', '\0',  'G', '\0', '\0', '\0',  'C', '\0', '\0', '\0', '\0', '\0', '\0',  'N', '\0', '\0', '\0', '\0', '\0',  'A',  'A', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0',  't', '\0',  'g', '\0', '\0', '\0',  'c', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0', '\0',  'a',  'a' ]&#xA;&#xA;def reverse_complement_c_v2(str seq):&#xA;    cdef bytes py_bytes = seq.encode('UTF-8')&#xA;    cdef char *seq_src = py_bytes&#xA;    cdef int i = 0&#xA;    for i in range(seq_len):&#xA;        seq_dest[seq_len - i - 1] = basemap[&amp;lt;int&amp;gt;seq_src[i]]&#xA;    return seq_dest[:seq_len].decode('UTF-8')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In testing under OS X, Python 3.6.3:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ python ./revcomp.py&#xA;['ACTGCAATAGACCACGC', 'CAGCTTGAGCCATTAAC', 'GGCCCAAGAGTTCGAAC', 'CGACTGTCTCGAATTGT', 'ATCCGCTATGCGCCGTC']&#xA;...&#xA;Cython implementation (v1)&#xA;       0.20664s total,&#xA;       2419626.4 strings per second&#xA;       88.3% increase over baseline&#xA;Cython implementation (v2)&#xA;       0.14170s total,&#xA;       3528668.5 strings per second&#xA;       92.0% increase over baseline&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using my lookup array approach (&quot;v2&quot;) adds a very decent performance bump over using &lt;code&gt;if&lt;/code&gt; blocks (&quot;v1&quot;), and you can keep everything as a Python string.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-02-20T19:16:22.180" LastActivityDate="2018-02-20T19:16:22.180" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3596" PostTypeId="2" ParentId="3582" CreationDate="2018-02-20T20:22:05.553" Score="2" Body="&lt;p&gt;The values are &lt;code&gt;rma&lt;/code&gt; normalized expression values from affy microarrays. They are indeed log2 transformed (as @gringer already expected). You can find more info about the data &lt;a href=&quot;https://bioconductor.org/packages/release/data/experiment/manuals/curatedOvarianData/man/curatedOvarianData.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, under heading &lt;code&gt;TCGA_eset&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get more info about &lt;code&gt;rma&lt;/code&gt;, check &lt;a href=&quot;https://www.rdocumentation.org/packages/affy/versions/1.50.0/topics/rma&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Larger values are not necessarily greater cancer possibility, I wish biology was that simple. &lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2018-02-20T20:22:05.553" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3597" PostTypeId="2" ParentId="3590" CreationDate="2018-02-20T20:22:49.227" Score="2" Body="&lt;p&gt;1D² data that is called by Albacore generates two overlapping sets of files. Here's what Devon O'Rourke mentioned on the ONT community forums and on Twitter:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Following Albacore basecalls on a 1d2 library I get two sets of .fq files, summary stats, etc.: one for the 1D basecalling script, and one for the 1D2 script. I'd like to use the reads generated from these scripts in a genome assembly.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;After a little bit of &lt;code&gt;grep&lt;/code&gt; searching, it seems like there are overlapping reads among the 1D and 1D2 directories - namely, those 1d reads that generated 1d2 reads!&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Link &lt;a href=&quot;https://community.nanoporetech.com/posts/albacore-1d2-output-direct&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, for those who have ONT community access.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've recommended generating a list of the 1D^2 reads, then excluding those reads from the 1D fastq files. I have a &lt;a href=&quot;https://github.com/gringer/bioinfscripts/blob/master/fastx-length.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;couple&lt;/a&gt; of &lt;a href=&quot;https://github.com/gringer/bioinfscripts/blob/master/fastx-fetch.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;scripts&lt;/a&gt; that can help for this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat 1d2_reads.fq | ~/scripts/fastx-length.pl | perl -lane '$F[1] =~ s/(.{36})/$1\n/; print $F[1];' &amp;gt; readNames.txt&#xA;$ cat 1d_reads.fq | ~/scripts/fastx-fetch.pl -v -i readNames.txt &amp;gt; filtered_1d_reads.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;More information on this can also be found in &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/3333/removing-repeated-reads-from-nanopore-1d%C2%B2-reads&quot;&gt;this StackExchange answer&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-02-20T20:22:49.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3598" PostTypeId="1" AcceptedAnswerId="3606" CreationDate="2018-02-21T10:37:43.777" Score="2" ViewCount="35" Body="&lt;p&gt;I have this link that comes up when I am in the browser window and click on the gene name, which leads me to description page about the gene (in this example it is BOK gene).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I generate that link? Below are some of my failed attempts. There is a &lt;a href=&quot;https://genome.ucsc.edu/FAQ/FAQlink.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC FAQ page about the links&lt;/a&gt;, didn't help much, maybe I am reading it wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Link that works, generated when clicked on gene on the website:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://genome-euro.ucsc.edu/cgi-bin/hgc?hgsid=226968178_C6XmUjsjtt3Mm9fOmePo6VqRN6TV&amp;amp;c=chr2&amp;amp;l=241667282&amp;amp;r=242920780&amp;amp;o=242498135&amp;amp;t=242513546&amp;amp;g=gtexGene&amp;amp;i=BOK&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genome-euro.ucsc.edu/cgi-bin/hgc?hgsid=226968178_C6XmUjsjtt3Mm9fOmePo6VqRN6TV&amp;amp;c=chr2&amp;amp;l=241667282&amp;amp;r=242920780&amp;amp;o=242498135&amp;amp;t=242513546&amp;amp;g=gtexGene&amp;amp;i=BOK&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which gives me this page:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/OfI7M.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OfI7M.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My failed attempts to generated custom link:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://genome-euro.ucsc.edu/cgi-bin/hgTracks?db=hg19&amp;amp;position=chr2%3A241667283-242920780&amp;amp;gtexGene&amp;amp;i=BOK&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genome-euro.ucsc.edu/cgi-bin/hgTracks?db=hg19&amp;amp;position=chr2%3A241667283-242920780&amp;amp;gtexGene&amp;amp;i=BOK&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://genome.ucsc.edu/cgi-bin/org=human&amp;amp;db=hg19&amp;amp;gtexGene&amp;amp;i=BOK&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genome.ucsc.edu/cgi-bin/org=human&amp;amp;db=hg19&amp;amp;gtexGene&amp;amp;i=BOK&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://genome.ucsc.edu/cgi-bin/gtexGene?org=human&amp;amp;db=hg19&amp;amp;i=BOK&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genome.ucsc.edu/cgi-bin/gtexGene?org=human&amp;amp;db=hg19&amp;amp;i=BOK&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://genome.ucsc.edu/cgi-bin/gtexGene?org=human&amp;amp;db=hg19&amp;amp;hgg_gene=BOK&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genome.ucsc.edu/cgi-bin/gtexGene?org=human&amp;amp;db=hg19&amp;amp;hgg_gene=BOK&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://genome.ucsc.edu/cgi-bin/gtexGene?org=human&amp;amp;db=hg19&amp;amp;i=BOK&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genome.ucsc.edu/cgi-bin/gtexGene?org=human&amp;amp;db=hg19&amp;amp;i=BOK&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://genome.ucsc.edu/cgi-bin/gtexGene?org=human&amp;amp;db=hg19&amp;amp;g=gtexGene&amp;amp;i=BOK&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genome.ucsc.edu/cgi-bin/gtexGene?org=human&amp;amp;db=hg19&amp;amp;g=gtexGene&amp;amp;i=BOK&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://genome.ucsc.edu/cgi-bin/org=human&amp;amp;db=hg19&amp;amp;g=gtexGene&amp;amp;i=BOK&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genome.ucsc.edu/cgi-bin/org=human&amp;amp;db=hg19&amp;amp;g=gtexGene&amp;amp;i=BOK&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://genome.ucsc.edu/cgi-bin/hgTracks?org=human&amp;amp;db=hg19&amp;amp;g=gtexGene&amp;amp;i=BOK&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genome.ucsc.edu/cgi-bin/hgTracks?org=human&amp;amp;db=hg19&amp;amp;g=gtexGene&amp;amp;i=BOK&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="131" LastEditorUserId="131" LastEditDate="2018-02-21T13:51:54.080" LastActivityDate="2018-02-21T17:08:45.087" Title="Link to UCSC gtexGene track description page for selected gene symbol" Tags="|gene|ucsc|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3600" PostTypeId="1" AcceptedAnswerId="3666" CreationDate="2018-02-21T11:05:07.590" Score="6" ViewCount="2179" Body="&lt;p&gt;I have genomic .bed file data of 4 different types; type A,B,C,D. These are some examples-&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Type A: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   101380000   101710000   A&#xA;1   110085000   110320000   A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Type B:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   100930000   101335000   B&#xA;1   10430000    10560000    B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Type C: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   101775000   101825000   C&#xA;1   103570000   104070000   C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Type D:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   153325000   153475000   D&#xA;1   156355000   156415000   D&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This data is for chr1, but I such data for all the chromosomes. I want to draw a chromosome map (possibly using the Gviz package?) which can assign different colors to the 4 types of genomic ranges and make me an idiogram like plot for the chromosomes I wish to see, so I know where my ranges physically are, on the chromosome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can this be done using the &lt;strong&gt;Gviz&lt;/strong&gt; package in R? If not, what is the best way and code to do this?  &lt;/p&gt;&#xA;" OwnerUserId="2094" LastActivityDate="2020-06-20T07:55:44.893" Title="How to make chromosome color maps for bed ranges" Tags="|r|gviz|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3601" PostTypeId="1" AcceptedAnswerId="3602" CreationDate="2018-02-21T15:09:38.657" Score="4" ViewCount="445" Body="&lt;p&gt;Per request from meta &lt;a href=&quot;https://bioinformatics.meta.stackexchange.com/questions/165/would-questions-on-whole-cell-modeling-get-answers-at-this-site?noredirect=1#comment1383_165&quot;&gt;comment&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am self-learning about whole cell modeling, specifically &lt;a href=&quot;http://intro-to-wc-modeling.readthedocs.io/en/latest/index.html#&quot; rel=&quot;nofollow noreferrer&quot;&gt;An introduction to whole-cell modeling&lt;/a&gt; and &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/1420084100&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fundamentals of Systems Biology: From Synthetic Circuits to Whole-cell Models&lt;/a&gt;. While my background is decades of programming, for cellular biology it is only skimming through a few books. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the process of learning I soon discovered &lt;a href=&quot;http://sbml.org/SBML_Software_Guide/SBML_Software_Matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;software/apps&lt;/a&gt; that use &lt;a href=&quot;http://sbml.org/Main_Page&quot; rel=&quot;nofollow noreferrer&quot;&gt;SBML&lt;/a&gt; (System Biology Markup language) but in &lt;a href=&quot;http://intro-to-wc-modeling.readthedocs.io/en/latest/cell_modeling/bio_net_gen/bio_net_gen.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;referring&lt;/a&gt; back to &quot;An introduction to whole-cell modeling&quot; see that they use &lt;a href=&quot;http://www.csb.pitt.edu/Faculty/Faeder/?page_id=409&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioNetGen&lt;/a&gt; (&lt;a href=&quot;https://github.com/RuleWorld/bionetgen&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt;) which has its own language BioNetGen language (BNGL)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the differences between BNGL (BioNetGen Language) and SBML (System Biology Markup Langague) formats?&lt;/p&gt;&#xA;" OwnerUserId="1586" LastEditorUserId="48" LastEditDate="2018-02-21T15:47:08.623" LastActivityDate="2018-02-21T15:50:10.623" Title="What are the differences between BNGL (BioNetGen Language) and SBML (System Biology Markup Langague) formats?" Tags="|file-formats|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3602" PostTypeId="2" ParentId="3601" CreationDate="2018-02-21T15:45:42.517" Score="3" Body="&lt;p&gt;Both formats have &lt;a href=&quot;https://en.wikipedia.org/wiki/Formal_grammar&quot; rel=&quot;nofollow noreferrer&quot;&gt;formal grammars&lt;/a&gt; for describing similar concepts, however they are different and to my knowledge one is not a proper subset of the other. While one could do a comparison of a syntactic nature, a simple Google search of just both acronyms reveals &lt;a href=&quot;https://arxiv.org/pdf/1507.03572.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioNetGen 2.2: Advances in Rule-Based Modeling&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In here can be found:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;2.4 SBML-to-BNGL translation&lt;br&gt;&#xA;  SBML is a widely-used model exchange format in systems biology (Hucka et al., 2003). Models encoded in SBML&#xA;  are ﬂat, i.e., their species do not have internal structure, which&#xA;  limits their utility for rule-based modeling. BioNetGen 2.2 includes&#xA;  an SBML-to-BNGL translator, called Atomizer (also available as a web&#xA;  tool at ratomizer.appspot.com), that can extract implicit molecular&#xA;  structure from ﬂat species (Tapia and Faeder, 2013). A full report on&#xA;  Atomizer and its application to the BioModels database (Li et al.,&#xA;  2010) is currently in preparation. However, Tapia and Faeder (2013)&#xA;  reported that an early version of the tool could recover implicit&#xA;  structure for about 60% of species in models within the database that&#xA;  contain ≥20 species. Thus, Atomizer makes available a large set of&#xA;  pre-existing models in a rule-based format, facilitating their&#xA;  visualization (Wenskovitch et al., 2014) and extension (Chylek et al.,&#xA;  2015).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;While not directly related to the question this also helped in understanding more about BioNetGen&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wikipedia: &lt;a href=&quot;https://en.wikipedia.org/wiki/Multi-state_modeling_of_biomolecules&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multi-state modeling of biomolecules&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Biological signaling systems often rely on complexes of biological&#xA;  macromolecules that can undergo several functionally significant&#xA;  modifications that are mutually compatible. Thus, they can exist in a&#xA;  very large number of functionally different states. Modeling such&#xA;  multi-state systems poses two problems: The problem of how to describe&#xA;  and specify a multi-state system (the &quot;specification problem&quot;) and the&#xA;  problem of how to use a computer to simulate the progress of the&#xA;  system over time (the &quot;computation problem&quot;). To address the&#xA;  specification problem, modelers have in recent years moved away from&#xA;  explicit specification of all possible states, and towards rule-based&#xA;  formalisms that allow for implicit model specification, including the&#xA;  κ-calculus, BioNetGen, the Allosteric Network Compiler and others. To&#xA;  tackle the computation problem, they have turned to particle-based&#xA;  methods that have in many cases proved more computationally efficient&#xA;  than population-based methods based on ordinary differential&#xA;  equations, partial differential equations, or the Gillespie stochastic&#xA;  simulation algorithm. Given current computing technology,&#xA;  particle-based methods are sometimes the only possible option.&#xA;  Particle-based simulators further fall into two categories:&#xA;  Non-spatial simulators such as StochSim, DYNSTOC, RuleMonkey, and&#xA;  NFSim and spatial simulators, including Meredys, SRSim and MCell.&#xA;  Modelers can thus choose from a variety of tools; the best choice&#xA;  depending on the particular problem. Development of faster and more&#xA;  powerful methods is ongoing, promising the ability to simulate ever&#xA;  more complex signaling processes in the future.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1586" LastEditorUserId="48" LastEditDate="2018-02-21T15:50:10.623" LastActivityDate="2018-02-21T15:50:10.623" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3603" PostTypeId="1" CreationDate="2018-02-21T16:16:09.500" Score="1" ViewCount="24" Body="&lt;p&gt;Lets say I am using reads downloaded from &lt;a href=&quot;https://www.ebi.ac.uk/ena/data/view/PRJEB6603&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENA&lt;/a&gt;, assembling them and doing further downstream analyses. Would it be possible to then upload the results of the co-assemblies and other analyses back to ENA, and just linking them up with the reads I originally downloaded, without having to re-upload the original reads? &lt;/p&gt;&#xA;" OwnerUserId="369" LastActivityDate="2018-02-21T16:16:09.500" Title="Can I upload only co-assemblies to ENA, using existing ENA samples?" Tags="|public-databases|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3604" PostTypeId="1" AcceptedAnswerId="3608" CreationDate="2018-02-21T16:29:22.067" Score="5" ViewCount="619" Body="&lt;p&gt;I have a Mac Book Pro and I am about to start a sequencing run on the MinION. MinKNOW is going to be running for 2 days. &#xA;Can I use my computer during sequencing? Can I browse the web and use Excel, etc? I am not speaking about running a parallelised genome assembly on all my cores using my entire RAM but more mundane activities, like checking my emails or taking notes. Furthermore, can I lock my screen using Control + Shift + OFF without negatively affecting the sequencing? &lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-03-01T06:16:59.823" Title="Can I use my computer while MinION is sequencing without negatively affecting the run?" Tags="|nanopore|ngs|minion|minknow|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3605" PostTypeId="1" AcceptedAnswerId="3870" CreationDate="2018-02-21T16:47:17.923" Score="2" ViewCount="5971" Body="&lt;p&gt;I would like to set the folder in which MinKNOW writes the raw data. How can I do this? I do not currently know where MinKNOW will output my data. What is the default directory where MinKNOW outputs the data? My OS is Mac OS X.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-02-22T10:05:26.550" LastActivityDate="2018-03-12T16:49:12.607" Title="Set directory where MinKNOW writes FAST5 files" Tags="|nanopore|ngs|minion|minknow|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3606" PostTypeId="2" ParentId="3598" CreationDate="2018-02-21T17:08:45.087" Score="2" Body="&lt;p&gt;You never want to include the hgsid parameter in your url because that is the link to your current browsing session and all its specific contents. So if you send that link to someone else and then they start browsing around that link may not work correctly again in the future if you or the person you sent the link to modify your browsing session.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to how to create links to hgc pages, the usually follow a somewhat general format:&#xA;&lt;a href=&quot;http://genome-euro.ucsc.edu/cgi-bin/hgc?db=hg19&amp;amp;c=chr2&amp;amp;o=242498135&amp;amp;t=242513546&amp;amp;g=gtexGene&amp;amp;i=BOK&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genome-euro.ucsc.edu/cgi-bin/hgc?db=hg19&amp;amp;c=chr2&amp;amp;o=242498135&amp;amp;t=242513546&amp;amp;g=gtexGene&amp;amp;i=BOK&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need the db and g parameters to specify the assembly build and track, and then you need to provide the position in the form of the c,o and t parameters. The last thing you want is the i paramter to specify the item name. The reason the i parameter alone is not enough is because we do not require uniqueness on item names (there may be many copies of an item across the genome for instance).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can obtain the c, o, t and i parameters by selecting the chrom, chromStart, chromEnd and name fields from the gtexGene table:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ mysql --user=genome --host=genome-mysql.soe.ucsc.edu -A -P 3306 -e &quot;select chrom, chromStart, chromEnd, name from gtexGene limit 1&quot; hg38&#xA;+-------+------------+----------+---------+&#xA;| chrom | chromStart | chromEnd | name    |&#xA;+-------+------------+----------+---------+&#xA;| chr1  |      11868 |    13052 | DDX11L1 |&#xA;+-------+------------+----------+---------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1000" LastActivityDate="2018-02-21T17:08:45.087" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3607" PostTypeId="1" AcceptedAnswerId="3617" CreationDate="2018-02-21T17:30:13.887" Score="1" ViewCount="176" Body="&lt;p&gt;I am unable to understand what exactly the full_1dsq_basecaller.py/.exe script does. Is it giving the consensus sequence for linear and complementary strand, or it is just giving the reads that came from the complimentary strand? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we are getting two fastq files as output which one we should use for alignment? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance. &#xA;I greatly appreciate any help&lt;/p&gt;&#xA;" OwnerUserId="2381" LastEditorUserId="73" LastEditDate="2018-02-22T20:26:58.037" LastActivityDate="2018-02-22T21:19:52.587" Title="Basecalling process carried out by full_1dsq_basecaller.py/.exe" Tags="|nanopore|albacore|1d2-reads|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3608" PostTypeId="2" ParentId="3604" CreationDate="2018-02-21T19:04:16.137" Score="4" Body="&lt;p&gt;Yes, that should be fine, especially if basecalling is disabled. Locking a screen should be fine as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I prefer to do basecalling after a run anyway, because there's a chance that it could chew up too much processor time and result in lots of skipped reads.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-02-21T19:04:16.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3609" PostTypeId="2" ParentId="2111" CreationDate="2018-02-21T20:17:52.403" Score="6" Body="&lt;p&gt;Your approach in general seems to be OK, but I have a few suggestions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You are basing all your calculations on whether the pH is above or below 7. However the N and C termini have pKa values that determine whether these groups will be ionized (8 and 3, respectively). You should check whether the pH is above/below these values instead of 7.&lt;/li&gt;&#xA;&lt;li&gt;Different amino acid side chains can also be ionized and should be accounted for when calculating the MW. Amino acid pKas can be found &lt;a href=&quot;http://www.chem.ucla.edu/~rebecca/153A/amino_acids.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Code wise: you pass a variable unimod_location in your function but it's never used. If it's not being used then take it out.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="388" LastActivityDate="2018-02-21T20:17:52.403" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3610" PostTypeId="2" ParentId="3057" CreationDate="2018-02-21T22:17:12.973" Score="1" Body="&lt;p&gt;You can download a BLAST db containing all sequences of proteins in Uniprot and in the PDB. The way I would go about this is first download the databases for uniprot and PDB, then query the PDB database for each sequence from Uniprot. If you get a BLAST hit above a certain threshold (whatever you define as homolog) then add that sequence to a file, and voila, you have a full set of sequences that fulfill your criteria. The only tools you would need for this are the BLAST executables (&lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PAGE_TYPE=BlastDocs&amp;amp;DOC_TYPE=Download&quot; rel=&quot;nofollow noreferrer&quot;&gt;found here&lt;/a&gt;) and the databases (info on how to download can be found &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK62345/#_blast_ftp_site_The_blastdb_subdirectory_&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should note that you don't really need any Python to get this to work, only some shell scripting. Although you can run these commands through a Python wrapper if you really want to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="388" LastActivityDate="2018-02-21T22:17:12.973" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3611" PostTypeId="1" AcceptedAnswerId="3612" CreationDate="2018-02-22T10:18:24.313" Score="2" ViewCount="105" Body="&lt;p&gt;What's an efficient way of counting of dinucleotides/trinucleotide pattern on fastq.gz file reads?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know there are tools like seqtk that will be very efficient at reading through the .fastq.gz files, and tools like cutadapt that are good at finding long(ish) adapter patterns in the fastq.gz PE of files, but, what would be the best tool to count the number of reads where a di-/trinucleotide is present N-number of times in the read? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;countingtool --pattern CGC file_R1.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Result: column two is number of times the pattern is seen (exclusive numbers, no overlap)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CGC    0     51341234&#xA;CGC    1     13948130&#xA;CGC    2      1234344&#xA;CGC    3        51344&#xA;CGC    4         1343&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="180" LastEditorUserId="73" LastEditDate="2018-02-22T20:20:53.610" LastActivityDate="2018-02-22T20:20:53.610" Title="efficient counting of dinucleotides/trinucleotides on fastq reads?" Tags="|fastq|benchmarking|seqtk|cutadapt|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3612" PostTypeId="2" ParentId="3611" CreationDate="2018-02-22T12:36:04.043" Score="1" Body="&lt;p&gt;Well, I'm sure you can find a more efficient approach, but here's a simplistic solution using standard UNIX tools:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zcat file.fastq.gz | grep CGC | &#xA;    awk -F'CGC' 'NR%4==2{a[NF-1]++}END{for(i in a){print i,a[i]}}' |&#xA;         sort -rnk2   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tested this on a 2.6G fastq file (exome data) and it took:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;real    1m18.987s&#xA;user    1m41.392s&#xA;sys     0m10.684s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;On a 66G WGS sequence file, it took:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;real    34m6.771s&#xA;user    40m16.452s&#xA;sys     4m10.176s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The extra &lt;code&gt;grep CGC&lt;/code&gt; isn't needed, you can do the match in awk, but it does speed things up a little bit. You can always get the reads with 0 matches by counting the number of reads and subtracting the ones with 1. Alternatively, remove the &lt;code&gt;grep&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zcat file.fq.gz | awk -F'CGC' 'NR%4==2{a[NF-1]++}END{for(i in a){print i,a[i]}}' |&#xA;    sort -rnk2   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Following the OP's suggestion, I also tested with &lt;code&gt;pigz&lt;/code&gt; and, it turns out, pigz can fly!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time unpigz -c exome.fastq.gz | grep CGC | &#xA;    awk -F'CGC' 'NR%4==2{a[NF-1]++}END{for(i in a){print i,a[i]}}' |&#xA;         sort -rnk2&#xA;&#xA;&#xA;real    0m44.012s&#xA;user    1m19.544s&#xA;sys     0m14.900s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And for the 66G WGS:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;real    17m29.935s&#xA;user    29m23.816s&#xA;sys     5m17.856s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-02-22T14:28:07.530" LastActivityDate="2018-02-22T14:28:07.530" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3613" PostTypeId="1" CreationDate="2018-02-22T14:40:31.730" Score="2" ViewCount="1028" Body="&lt;p&gt;I download the &lt;a href=&quot;https://mirror.oxfordnanoportal.com/software/analysis/ont_albacore-2.1.10-cp36-cp36m-macosx_10_11_x86_64.whl&quot; rel=&quot;nofollow noreferrer&quot;&gt;Albacore wheel&lt;/a&gt; from Oxford Nanopore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I activated my conda environment that has Python 3 as the default python version.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to install Albacore with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pip3 install ont_albacore-2.1.10-cp36-cp36m-macosx_10_11_x86_64.whl&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but got the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ont_albacore-2.1.10-cp36-cp36m-macosx_10_11_x86_64.whl is not a supported wheel on this platform.&#xA;You are using pip version 8.1.2, however version 9.0.1 is available.&#xA;You should consider upgrading via the 'pip install --upgrade pip' command.`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I then upgraded my &lt;code&gt;pip&lt;/code&gt; with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pip install --upgrade pip&#xA;Requirement already up-to-date: pip in /Users/.../anaconda3/envs/python3env/lib/python3.6/site-packages`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So it seems that my pip was already up to date. I do not understand why it complains about my &lt;code&gt;pip&lt;/code&gt; if I am using &lt;code&gt;pip3&lt;/code&gt; anyways. I tried to install Albacore again but I got the same message as before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me install Albacore on my Mac El Capitan laptop, please?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update based on Wouter's suggestion: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python -m pip install ont_albacore-2.1.10-cp36-cp36m-macosx_10_11_x86_64.whl&#xA;Processing ./ont_albacore-2.1.10-cp36-cp36m-macosx_10_11_x86_64.whl&#xA;Requirement already satisfied: python-dateutil in /Users/userName/anaconda3/lib/python3.6/site-packages (from ont-albacore==2.1.10)&#xA;Requirement already satisfied: setuptools in /Users/userName/anaconda3/lib/python3.6/site-packages (from ont-albacore==2.1.10)&#xA;Requirement already satisfied: h5py in /Users/userName/anaconda3/lib/python3.6/site-packages (from ont-albacore==2.1.10)&#xA;Collecting progressbar33 (from ont-albacore==2.1.10)&#xA;Requirement already satisfied: numpy&amp;gt;=1.12.1 in /Users/userName/anaconda3/lib/python3.6/site-packages (from ont-albacore==2.1.10)&#xA;Collecting ont-fast5-api&amp;gt;=0.4.1 (from ont-albacore==2.1.10)&#xA;  Using cached ont_fast5_api-0.4.1-py2.py3-none-any.whl&#xA;Requirement already satisfied: six&amp;gt;=1.5 in /Users/userName/anaconda3/lib/python3.6/site-packages (from python-dateutil-&amp;gt;ont-albacore==2.1.10)&#xA;Installing collected packages: progressbar33, ont-fast5-api, ont-albacore&#xA;Successfully installed ont-albacore-2.1.10 ont-fast5-api-0.4.1 progressbar33-2.4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;At first sight, it looks like Albacore was successfully installed but if I run: &lt;code&gt;read_fast5_basecaller.py --help&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &quot;/Users/userName/anaconda3/bin/read_fast5_basecaller.py&quot;, line 69, in &amp;lt;module&amp;gt;&#xA;    from albacore.input_utils import list_input_files&#xA;  File &quot;/Users/userName/anaconda3/lib/python3.6/site-packages/albacore/input_utils.py&quot;, line 7, in &amp;lt;module&amp;gt;&#xA;    from albacore import log_utils&#xA;  File &quot;/Users/userName/anaconda3/lib/python3.6/site-packages/albacore/log_utils.py&quot;, line 5, in &amp;lt;module&amp;gt;&#xA;    from albacore.pipeline_core import get_debug_level&#xA;ImportError: dlopen(/Users/userName/anaconda3/lib/python3.6/site-packages/albacore/pipeline_core.cpython-36m-darwin.so, 2): Library not loaded: /usr/local/opt/python3/Frameworks/Python.framework/Versions/3.6/Python&#xA;  Referenced from: /Users/userName/anaconda3/lib/python3.6/site-packages/albacore/pipeline_core.cpython-36m-darwin.so&#xA;  Reason: image not found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a library missing?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="48" LastEditDate="2018-02-26T11:20:13.763" LastActivityDate="2018-11-14T21:45:44.843" Title="Albacore wheel not supported on this platform (Mac OS X El Capitan)" Tags="|nanopore|minion|software-installation|albacore|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3614" PostTypeId="1" AcceptedAnswerId="3615" CreationDate="2018-02-22T15:02:31.390" Score="5" ViewCount="781" Body="&lt;p&gt;One of my studies (A) involve sequencing the microbiome. After selecting the variable region and the primers for the targeted region of the 16S sequence. The samples were sent to a platform.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For another study (B) we used the same strategy (same primers, region, machine...) but another platform. However we obtained from this platform a lot more sequences:&lt;br&gt;&#xA;In study A the mean sequences per sample are 10k, while in the study B the mean sequences per sample are 80k&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I consulted with others and it seems that the study A we under-sampled and in study B we over-sampled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can understand that by under-sampling we didn't obtain enough reads to be sure about the microbiome present in the sample. But what does it mean to over-sampling?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That expert I consulted suggested normalize the data to fewer sequences, discarding information we already have. I can understand that the sequencing has errors and can create chimeras but the resulting consensus sequences of the OTUs have found a match with databases of microorganisms. What or how should I take into account that I over-sampled?&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2018-02-22T16:27:36.743" LastActivityDate="2018-02-22T17:30:18.467" Title="What does it mean to over sample?" Tags="|ngs|microbiology|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3615" PostTypeId="2" ParentId="3614" CreationDate="2018-02-22T15:38:15.573" Score="4" Body="&lt;p&gt;As you sequence a library you are sampling from the pool of all possible oligonucleotides found in it. This is a stochastic process, meaning that you are more likely to sequence a piece of DNA that is more prevalent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you increase the number of reads that you sequence for a library, you decrease the chances of seeing a new read that you have not yet seen before. If you plot the number of unique reads per total reads sequenced, it will look something like the &lt;a href=&quot;https://en.wikipedia.org/wiki/Exponential_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;cumulative density function of the exponential distribution&lt;/a&gt;, where you have diminishing returns on unique sequences as you continue to sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you sequence a library and quickly stop seeing unique reads, we say that this library has &lt;strong&gt;low complexity&lt;/strong&gt;. Therefore, you have &quot;oversampled&quot; it if you sequence too deeply. I am guessing that your metagenomics library actually has a lot fewer species/unique sequences than they anticipated, and that you sequenced deeply anticipating more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: In response to your comments, the bigger problem in 16S analyses is not sequencing too deeply, it is over-amplifying your samples such that they no longer represent what is actually in your gut microbiome samples. In this case if you used read normalization you would not accurately represent what was in the library that you sequenced since the amplified molecules (16S) are so small that you only have len(16S molecule)-(read len) + 1 possible reads in each direction. You might consider removing PCR duplicates with GATK but this will still have the same effect of not representing what was in the sequencing pool. It is very possible that you have many PCR duplicates after 16S amplification. I generally don't trust amplicon sequencing for quantitive analyses, and don't really trust for low coverage &quot;is-this-species-there-or-not&quot; either, especially when dealing with the 'dark matter' of uncharacterized species. You might be well off just asking &quot;is this species present in high coverage at this time point?&quot;&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-02-22T17:30:18.467" LastActivityDate="2018-02-22T17:30:18.467" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3616" PostTypeId="2" ParentId="3614" CreationDate="2018-02-22T16:48:46.530" Score="1" Body="&lt;p&gt;I don't think there's anything inherently wrong with sequencing to high depth (though naive approaches to dealing with sequencing errors such as singleton removal may fail if the depth is so high that the same errors may be encountered repeatedly by chance). The bigger issue is that they are over sampled &lt;em&gt;relative&lt;/em&gt; to the undersampled microbiomes. This does complicate comparisons, particularly for things like OTU richness. The obvious solution is to down-sample the oversampled microbiomes so you have the same number of reads for each. This is probably unnecessary if your analyzing the oversampled microbiomes on their own.&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-02-22T16:48:46.530" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3617" PostTypeId="2" ParentId="3607" CreationDate="2018-02-22T21:19:52.587" Score="3" Body="&lt;p&gt;My understanding is that the 1D$^2$ process is carried out in three stages:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Carry out a 1D calling of reads (i.e. what the standard Albacore does at the moment)&lt;/li&gt;&#xA;&lt;li&gt;Link reads that are similar to each other (in reverse-complement orientation), from the same channel, and have a very short pore transition time&lt;/li&gt;&#xA;&lt;li&gt;Create a 2D consensus base call, using information from the two linked reads&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The two outputs from the basecaller are the 1D calls from step 1, and the consensus calls from step 3. Ideally just the 2D consensus calls should be used for assembly (assuming it represents a random distribution of reads), but  if that gives too low coverage (or is not random enough), then the non-converted reads from the 1D calls could be included as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because the linking is not a physical link, it's not a good idea to use 1D$^2$ for amplicon reads; the likelihood of two complementary reads from different templates appearing one after another is too high.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-02-22T21:19:52.587" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3619" PostTypeId="1" AcceptedAnswerId="3622" CreationDate="2018-02-23T05:09:25.510" Score="10" ViewCount="640" Body="&lt;p&gt;I want to write a program in C++ that stores kmers in a hash or in a trie. How can I calculate how much RAM I would need for each type of data structure? For this application my kmers are strand-specific, so I cannot reduce the size complexity by considering canonical kmers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a hash, for example, I know that there are $4^k$ possible kmers of length $k$. I also am assuming that DNA character with an alphabet $\in \{G,A,T,C\}$ can be stored in two bits ($2^2 = 4$). Using this logic the kmer hash should occupy $(4^k) \cdot 2 \cdot k$ bits. Plugging this into Wolfram Alpha (&lt;code&gt;(4^13) * 2 * 13 bits to gigabytes&lt;/code&gt;), I calculated the amount of RAM that some small kmer sizes would require to store the entire hash in memory in GB.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;k = 13 requires    0.2181 GB&#xA;k = 15 requires    4.027  GB&#xA;k = 17 requires   73.01  GB&#xA;k = 19 requires 1306   GB&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This escalated rather quickly, and I think that my calculations must be wrong. Otherwise, how could programs count kmers in RAM?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The kmers that I want to store are sometimes sparse compared to the entire space (for k=13, only 29 million or so out of 67 million possibilities).&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Constraints&lt;/h1&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I do not need to store the number of occurences of a kmer, just its existence&lt;/li&gt;&#xA;&lt;li&gt;I cannot risk false positives. Bloom filters won't work in this case. :^/&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-02-23T05:46:38.257" LastActivityDate="2018-02-26T07:07:02.470" Title="How to calculate the memory usage of storing kmers in RAM" Tags="|genome|k-mer|c++|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3621" PostTypeId="2" ParentId="3619" CreationDate="2018-02-23T08:10:58.330" Score="5" Body="&lt;p&gt;If you simply need a set of present/absent k-mers and your kmer length is short, then convert the sequence length to an integer (a 13-mer would require 26 bits or 4 bytes) and use that as an index into a bit array. Initialize the array to 0 and use 1 for &quot;k-mer present&quot;. If you have ~67 million k-mers then your array is 67 million bits, which is only about 8.5 megs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will scale &quot;OK&quot;, but eventually balloon to a huge size if you use larger k-mers. At that point you simply have to use other data structures. The reality is that most applications are fairly tolerant to small amounts of error.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-02-23T08:37:42.757" LastActivityDate="2018-02-23T08:37:42.757" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3622" PostTypeId="2" ParentId="3619" CreationDate="2018-02-23T10:58:33.337" Score="7" Body="&lt;p&gt;To follow up on Devon Ryan's answer, I thought it would be a little fun to write a Python script that demonstrates using a bit array to maintain a presence/absence table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I wrote a C++ port that includes a custom bitset implementation that can be sized at runtime. This and the Python script are available on Github: &lt;a href=&quot;https://github.com/alexpreynolds/kmer-boolean&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/alexpreynolds/kmer-boolean&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;&#xA;import sys&#xA;import os&#xA;import bitarray&#xA;&#xA;# read FASTA&#xA;def read_sequences():&#xA;    global seqs&#xA;    seqs = []&#xA;    seq = &quot;&quot;&#xA;    for line in sys.stdin:&#xA;        if line.startswith('&amp;gt;'):&#xA;            if len(seq) &amp;gt; 0:&#xA;                seqs.append(seq)&#xA;                seq = &quot;&quot;&#xA;        else:&#xA;            seq += line.strip()&#xA;    seqs.append(seq)&#xA;&#xA;# build and initialize bit array&#xA;def initialize_bitarray():&#xA;    global ba &#xA;    ba = bitarray.bitarray(4**k)&#xA;    ba.setall(False)&#xA;    sys.stderr.write(&quot;Memory usage of bitarray.bitarray instance is [%ld] bytes\n&quot; % (ba.buffer_info()[1]))&#xA;&#xA;# process sequences&#xA;def process_sequences():&#xA;    global observed_kmers&#xA;    observed_kmers = {}&#xA;    for seq in seqs:&#xA;        for i in range(0, len(seq)):&#xA;            kmer = seq[i:i+k]&#xA;            if len(kmer) == k:&#xA;                observed_kmers[kmer] = None&#xA;                idx = 0&#xA;                for j in range(k-1, -1, -1):&#xA;                    idx += 4**(k-j-1) * bm[kmer[j]]&#xA;                ba[idx] = True&#xA;&#xA;def test_bitarray():&#xA;    test_idx = 0&#xA;    for j in range(k-1, -1, -1):&#xA;        test_idx += 4**(k-j-1) * bm[test_kmer[j]]&#xA;    test_result = ba[test_idx]&#xA;    if test_result:&#xA;        sys.stdout.write(&quot;%s found\n&quot; % (test_kmer))&#xA;        sys.exit(os.EX_OK)&#xA;    else: &#xA;        sys.stdout.write(&quot;%s not found\n&quot; % (test_kmer))&#xA;        sys.exit(os.EX_DATAERR)&#xA;&#xA;def main():&#xA;    global k&#xA;    k = int(sys.argv[1])&#xA;&#xA;    global bm &#xA;    bm = { 'A' : 0, 'C' : 1, 'T' : 2, 'G' : 3 }&#xA;&#xA;    read_sequences()&#xA;    initialize_bitarray()&#xA;    process_sequences()&#xA;&#xA;    try:&#xA;        global test_kmer&#xA;        test_kmer = sys.argv[2]&#xA;        if len(test_kmer) == k:&#xA;            test_bitarray()&#xA;        else:&#xA;            raise ValueError(&quot;test kmer (%s) should be of length k (%d)&quot; % (test_kmer, k))&#xA;    except IndexError as err:&#xA;        keys = list(observed_kmers.keys())&#xA;        for i in range(0, len(keys)):&#xA;            sys.stdout.write(&quot;%s found\n&quot; % (keys[i]))&#xA;&#xA;    sys.exit(os.EX_OK)&#xA;&#xA;if __name__== &quot;__main__&quot;:&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that this doesn't look at canonical kmers, &lt;em&gt;e.g.&lt;/em&gt;, &lt;code&gt;AG&lt;/code&gt; is considered a distinct 2mer from its reverse complement &lt;code&gt;CT&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To use this script, you pipe in your FASTA, specify the &lt;em&gt;k&lt;/em&gt;, and an optional kmer that you want to test for presence/absence, &lt;em&gt;e.g.&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ echo -e &quot;&amp;gt;foo\nCATTCTC\nGGGAC\n&amp;gt;bar\nTTATAT\n&amp;gt;baz\nTTTATTAG\nACCTCT&quot; | ./kmer-bool.py 2 CG&#xA;Memory usage of bitarray.bitarray instance is [2] bytes&#xA;CG found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ echo -e &quot;&amp;gt;foo\nCATTCTC\nGGGAC\n&amp;gt;bar\nTTATAT\n&amp;gt;baz\nTTTATTAG\nACCTCT&quot; | ./kmer-bool.py 3 AAA&#xA;Memory usage of bitarray.bitarray instance is [8] bytes&#xA;AAA not found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or if the optional test kmer is left out:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ echo -e &quot;&amp;gt;foo\nCATTCTC\nGGGAC\n&amp;gt;bar\nTTATAT\n&amp;gt;baz\nTTTATTAG\nACCTCT&quot; | ./kmer-bool.py 5&#xA;Memory usage of bitarray.bitarray instance is [128] bytes&#xA;CATTC found&#xA;ATTCT found&#xA;TTCTC found&#xA;TCTCG found&#xA;CTCGG found&#xA;TCGGG found&#xA;CGGGA found&#xA;GGGAC found&#xA;TTATA found&#xA;TATAT found&#xA;TTTAT found&#xA;TTATT found&#xA;TATTA found&#xA;ATTAG found&#xA;TTAGA found&#xA;TAGAC found&#xA;AGACC found&#xA;GACCT found&#xA;ACCTC found&#xA;CCTCT found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or for the ~67M kmers in a 13-mer set, for which a roughly 8.4MB bit array is reserved:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ echo -e &quot;&amp;gt;foo\nCATTCTC\nGGGAC\n&amp;gt;bar\nTTATAT\n&amp;gt;baz\nTTTATTAG\nACCTCT&quot; | ./kmer-bool.py 13&#xA;Memory usage of bitarray.bitarray instance is [8388608] bytes&#xA;TTTATTAGACCTC found&#xA;TTATTAGACCTCT found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-02-26T07:07:02.470" LastActivityDate="2018-02-26T07:07:02.470" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3623" PostTypeId="1" AcceptedAnswerId="3624" CreationDate="2018-02-23T11:44:05.260" Score="3" ViewCount="1361" Body="&lt;p&gt;I am running a 48 h sequencing protocol on a FLO-MIN107, on MinION, with DNA library-prepped with SQK-RAD004. MinKNOW is installed on my Mac and controlling the MinION. I have found the following histogram in MinKNOW. What do the numbers mean? In particular, how many reads have been produced? I understood that the events are consecutive electrical signals and are then transformed into reads? Do I have 4029 reads? The run has started 24 hours ago. 4029 reads surely is really low, isn't it? What does 4,120,722 mean?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FHYiV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FHYiV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Platform QC of the flow cell was as follows: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;group 1: 480&lt;/li&gt;&#xA;&lt;li&gt;group 2: 374&lt;/li&gt;&#xA;&lt;li&gt;group 3: 201&lt;/li&gt;&#xA;&lt;li&gt;group 4: 59&lt;/li&gt;&#xA;&lt;li&gt;total = 1114&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-02-23T13:52:10.793" LastActivityDate="2018-02-23T13:52:10.793" Title="How many reads has my sequencing run produced on minion?" Tags="|nanopore|ngs|minion|long-reads|minknow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3624" PostTypeId="2" ParentId="3623" CreationDate="2018-02-23T13:45:24.410" Score="3" Body="&lt;p&gt;That's indeed the number of reads and that's quite low. How was your pore occupancy (number of pores sequencing) and your flow cell QC (number of pores good enough for sequencing)? How was your library concentration?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Events is an approximation of nucleotides: it's a guess based on the current signal. As far as I know if you multiply this by 1.8 you get approximately your throughput in nucleotides.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 4,120,722 is the number of events in the currently selected (mouse hovering?) bin of the histogram.&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2018-02-23T13:45:24.410" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3625" PostTypeId="2" ParentId="3541" CreationDate="2018-02-23T16:10:28.667" Score="6" Body="&lt;p&gt;There is now &lt;a href=&quot;https://www.biostars.org/p/300431/&quot; rel=&quot;nofollow noreferrer&quot;&gt;a post on Biostars&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We are starting to define the release process for the Human Cell&#xA;  Atlas.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The first couple of HCA releases will be primarily single-cell RNA-Seq&#xA;  and we are interested to know what you would expect our releases to&#xA;  contain&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Please fill out our survey if you would like to help us define our&#xA;  releases&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;a href=&quot;https://www.surveymonkey.com/r/S6Y753P&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.surveymonkey.com/r/S6Y753P&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And I suppose that answers the original question: the data has not been released yet and probably won't be for a while considering they are still discussing how to make it available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update (from the same post):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The first production release of the DCP is happening in June.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The first data release of HCA is targetting Q4 2018.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="35" LastEditorUserId="35" LastEditDate="2018-02-26T21:35:53.963" LastActivityDate="2018-02-26T21:35:53.963" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3626" PostTypeId="2" ParentId="3619" CreationDate="2018-02-23T20:04:57.017" Score="6" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I think that my calculations must be wrong. Otherwise, how could programs count kmers in RAM?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Hash table based k-mer counters only keep k-mers seen in data. For $16&amp;lt;k\le32$, you need 64-bit integers to store a k-mer. Given $n$ distinct k-mers, a naive implementation with open addressing hash table would roughly take $2n\cdot 64/8=16n$ bytes. We assume the table is half full – this is where factor 2 comes from. There are various tricks to reduce the memory footprint. You'd better read papers. On a side note, don't use std::unordered_map. It wastes extra memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A bitmap described by other answers takes $4^k/8=2^{2k-3}$ bytes, which is a better choice sometimes, but rarely used by k-mer counters as it does not work with long k, either. Trie or full-text indices are overkilling for fixed k-mers and are usually slower.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, this question reminds me of your question on efficient reverse complement. Note that for k-mer counting, you traverse both strands at the same time. You should not reverse complement the whole k-mer.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-02-23T20:04:57.017" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3627" PostTypeId="2" ParentId="3613" CreationDate="2018-02-24T11:42:26.237" Score="2" Body="&lt;p&gt;Not sure about &lt;code&gt;conda&lt;/code&gt; as I don't use it, but I can highly recommend &lt;a href=&quot;https://pipenv.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;pipenv&lt;/code&gt;&lt;/a&gt; which is the python.org recommended package management system for python.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tested the following on my Mac and it worked like a dream  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mkdir albacore &amp;amp;&amp;amp; cd albacore&#xA;mv ../ont_albacore-2.1.10-cp36-cp36m-macosx_10_11_x86_64.whl .&#xA;&#xA;# install pipenv if you don't have it&#xA;brew install pipenv  # or pip3 install pipenv&#xA;&#xA;# the following will create a virtual environment and install albacore into it&#xA;pipenv install ont_albacore-2.1.10-cp36-cp36m-macosx_10_11_x86_64.whl&#xA;&#xA;# activate the virtual environment&#xA;pipenv shell&#xA;&#xA;read_fast5_basecaller.py -h&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And that should bring up the albacore help menu and you're ready to go.&lt;/p&gt;&#xA;" OwnerUserId="149" LastEditorUserId="149" LastEditDate="2018-02-26T08:45:44.387" LastActivityDate="2018-02-26T08:45:44.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3629" PostTypeId="2" ParentId="3561" CreationDate="2018-02-25T21:32:18.797" Score="1" Body="&lt;p&gt;Depending on the size of your samples (1K, 10K, 100K?) or the size of imputation region (1 gene, 1 chromosome, whole genome?) imputation files even after zipping are usually huge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of filtering SNPs and creating yet more gazillions of subset files. I prefer to have &lt;code&gt;excludeSNP.txt&lt;/code&gt; file list of SNPs. This list then used as input for further analysis using &lt;em&gt;SNPTEST&lt;/em&gt; flag &lt;code&gt;-exclude_snps_g&lt;/code&gt;. This list is part of &lt;a href=&quot;https://mathgen.stats.ox.ac.uk/impute/output_file_options.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;IMPUTE2 output&lt;/a&gt; or could be additional list of SNPs that we wish to exclude for other reasons. In short, filter at the point of analysis not the imputated files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;em&gt;info&lt;/em&gt; file is missing we can run &lt;em&gt;SNPTEST&lt;/em&gt; with &lt;code&gt;-summary_stats_only&lt;/code&gt; flag, which gives you the info score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an old still relevant post at BioStars &lt;a href=&quot;https://www.biostars.org/p/6476/&quot; rel=&quot;nofollow noreferrer&quot;&gt;post-imputation QC&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="131" LastActivityDate="2018-02-25T21:32:18.797" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3630" PostTypeId="1" AcceptedAnswerId="3632" CreationDate="2018-02-26T00:16:32.437" Score="14" ViewCount="943" Body="&lt;p&gt;I had this question from a graduate student yesterday, and I was stuck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should I say? Why use a CRAM instead of a BAM? &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;When is it a good idea to use a CRAM instead of a BAM? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When is it a bad idea?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1770" LastEditorUserId="29" LastEditDate="2018-08-03T11:17:04.260" LastActivityDate="2018-08-03T11:17:04.260" Title="Why would someone use a CRAM instead of a BAM?" Tags="|sam|file-formats|cram|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3631" PostTypeId="1" CreationDate="2018-02-26T01:00:02.213" Score="1" ViewCount="368" Body="&lt;p&gt;I am studying the transcriptome of Arabidopsis. Interestingly, the 5' UTR of the latest annotation is usually too long. &#xA;Here is an example (this gene is right to left).&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Ig3t6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ig3t6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;You can see both EST (orange bars) and the RNA-seq reads cover a much smaller region of the annotated 5' UTR. Unfortunately, many 5'UTR of the genes have the same problem. Is there a way to fix that? I was wondering if any assembly tools can help me to get the correct 5'UTR ranges. Thanks!!!&lt;/p&gt;&#xA;" OwnerUserId="2118" LastEditorUserId="2118" LastEditDate="2018-02-26T14:22:21.080" LastActivityDate="2018-03-01T05:26:09.037" Title="How to fix 5'UTR annotation with RNA-seq data?" Tags="|rna-seq|sequence-annotation|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="3632" PostTypeId="2" ParentId="3630" CreationDate="2018-02-26T01:10:37.510" Score="17" Body="&lt;ol&gt;&#xA;&lt;li&gt;Whenever you want to save space (this can be a substantial savings). Until quite recently (samtools/htslib 1.7), only CRAM supported long CIGAR strings.&lt;/li&gt;&#xA;&lt;li&gt;If you need to guarantee that any random obscure downstream program will be able to handle it.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Uptake of CRAM has been pretty slow. Java programs using htsjdk (e.g., picard, IGV and GATK) have only relatively recently added support for CRAM. If you need to use an old version of those for some very odd reason then CRAM may not be supported.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a lot of programs written in python that use pysam to open BAM files and these should, theoretically, support CRAM. The issue is that some of the functions may fail and one can't assume that authors will have always written the code needed to handle this. I'll use deepTools as an example, since I'm one of its developers. One of the things about CRAM files is that they (by default) are made such that you require a reference genome in order to construct the sequence field in each alignment. This works fine if you're using a standard genome (htslib, via pysam, can fetch many standard genomes from the web automatically), but if you're not, then you need to specify a fasta file to use for decompression. Every tool, then, needs to add an option for this. With pysam 0.14 and htslib 1.7 this can be circumvented by not decompressing the sequence, but behavior has to be explicitly requested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another issue is that many tools will use features from the file index, such as the &lt;code&gt;.mapped&lt;/code&gt; accessor, to get the number of mapped reads in a file. CRAM files contain very very little information, so this then fails. Consequently, tool authors need to check for CRAM files and both derive and propagate this information through their functions if it's needed. This can be a time-consuming task (e.g., it took me a couple days to get this implemented in deepTools). Relatedly, &lt;code&gt;samtools idxstats&lt;/code&gt; is useless on CRAM files, since there are no statistics stored in the index.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That having been said, it's likely that CRAMs slowly gaining acceptance will eventually make it the standard. It's already a convenient archival format, it's just a matter of time before users can assume that most analysis programs are written to handle it.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-02-26T01:10:37.510" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3634" PostTypeId="1" CreationDate="2018-02-26T05:21:58.930" Score="2" ViewCount="299" Body="&lt;p&gt;A common way to model RNA-seq data is using a negative binomial distribution, where each sample-gene pair is modeled by a different negative binomial distribution with mean $\mu_{ij}$ where $i$ and $j$ are indices for genes and samples, respectively (see &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/gb-2010-11-10-r106&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;article&quot;&gt;article&lt;/a&gt;). My question is, given we only have a single observation ($X_{ij}$) from each sample-gene pair, how come we can learn the $\mu_{ij}$ from that single observation? We need multiple observations to estimate a mean, right? Maybe I am missing something critical.&lt;/p&gt;&#xA;" OwnerUserId="1331" LastEditorUserId="48" LastEditDate="2018-03-28T11:09:13.827" LastActivityDate="2023-05-20T22:51:40.610" Title="Negative binomial modeling of RNA-Seq data" Tags="|rna-seq|deseq2|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3635" PostTypeId="2" ParentId="3634" CreationDate="2018-02-26T07:48:14.643" Score="1" Body="&lt;p&gt;The $\mu_{ij}$ in that section of the DESeq manuscript is the expected value of sample $j$ in gene $i$ given its group association (with expected value $\mu_{ij}$). This is why $K_{ij}$ (this is presumably what you meant by $X_{ij}$) is a function of $\mu_{ij}$, rather than $\mu_{ij}$ itself. I think you're just swapping $K_{ij}$ and $\mu_{ij}$ in your head.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-02-26T07:48:14.643" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3636" PostTypeId="1" AcceptedAnswerId="3638" CreationDate="2018-02-26T12:07:33.067" Score="8" ViewCount="1553" Body="&lt;p&gt;I use &lt;a href=&quot;http://cutadapt.readthedocs.io/en/stable/guide.html#quality-trimming&quot; rel=&quot;noreferrer&quot;&gt;cutadapt&lt;/a&gt; to remove low quality bases from my Illumina reads. The algorithm only removes low quality bases from the end until it reaches a good quality base. If there is a bad quality base beyond that, it is not trimmed. Why? Why doesn't the algorithm remove all low quality bases? It could replace low quality bases in the middle of a read by an N, for example. Or is it that the low quality base is still most probably correct and hence one does not want to lose the base information for mapping?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-03-01T06:58:44.520" Title="Why does cutadapt remove low quality bases from the ends of reads only?" Tags="|reads|quality-control|trimming|qc|cutadapt|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3637" PostTypeId="2" ParentId="3636" CreationDate="2018-02-26T12:45:39.837" Score="9" Body="&lt;p&gt;If you check the read QC statistics of an Illumina run in e.g. fastQC, you will see that at the end of the read the quality decreases. This is because of exhaustion of chemicals at the end of the run. This is a general trend seen in all runs, therefore you can remove these low quality bases from the end of your run. If you have incidentally a bad quality base in the middle of a read this is not the same general trend but is sporadic. Which is hard to automatically remove with tools such as cutadapt. If you really want an N for each low quality bases within the read, you can make a simple program for that, but why would you need that? Most aligners can handle a few bad quality bases within a read, and for variant calling you'll need many reads (so the incidental error base will be overruled by the good ones).&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2018-02-26T12:45:39.837" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3638" PostTypeId="2" ParentId="3636" CreationDate="2018-02-26T16:13:06.523" Score="7" Body="&lt;p&gt;I am commenting on this part:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The algorithm only removes low quality bases from the end until it reaches a good quality base. If there is a bad quality base beyond that, it is not trimmed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;According to its user guide, cutadap is designed this way: it trims off bases from the 3'-end until it sees a base with quality higher than a threshold. This is not a good algorithm. For example, when we see a quality string &lt;code&gt;30,30,30,3,3,3,3,3,20,3,3&lt;/code&gt;, we would prefer to trim it down to &lt;code&gt;30,30,30&lt;/code&gt; because a &lt;code&gt;3,3,3,3,3,20&lt;/code&gt; tail is still not useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some tools such as trimmomatic uses a sliding window to avoid this issue. In my view, a better algorithm is the so-called the &quot;&lt;a href=&quot;http://www.phrap.org/phrap.docs/phred.html&quot; rel=&quot;noreferrer&quot;&gt;modified Mott algorithm&lt;/a&gt;&quot; used by phred 20 years ago. seqtk among others implements this algorithm. Note that the Mott algorithm always trims from both ends, which is sometimes not preferred. BWA implements a variant of this algorithm, trimming from the 3'-end only.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, though, different quality trimming algorithms probably work equally well because it is relatively rare to see a high-quality base in the middle of a low-quality tail. In addition, modern aligners can well handle low-quality tails; assemblers and error correctors can correct through such tails, too. It is usually not necessary to apply quality trimming.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It could replace low quality bases in the middle of a read by an N, for example. Or is it that the low quality base is still most probably correct and hence one does not want to lose the base information for mapping?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;With a low-quality base, you might have an error/mismatch, but with an N, you always have a mismatch. Masking low-quality bases to N is not as good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: responding to the following comment from OP:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What if the base is low-quality and then it matches the reference by mistake? Wouldn't it be better if it did not match and was penalized? I guess this should be modeled mathematically.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If a read could be mismapped due to a sequencing error, its true location is often in the genome. In this case, a competent mapper will give the alignment a low mapping quality (mapQ). A quality-aware mapper such as novoalign will penalize mapQ even more. In comparison, if you hard mask this sequencing error to &quot;N&quot;, you will get a mapQ=0. You can see the difference between the two approaches mostly comes from mapQ.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In modern Illumina data, it is quite often to see a Q8 base in the middle of high-quality bases. &gt;80% of them (in theory) are still correct. My &lt;em&gt;hunch&lt;/em&gt; is that hard masking all of them would lead to considerable data loss.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2018-02-26T17:14:48.023" LastActivityDate="2018-02-26T17:14:48.023" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3640" PostTypeId="1" AcceptedAnswerId="3641" CreationDate="2018-02-26T19:58:58.640" Score="5" ViewCount="3860" Body="&lt;p&gt;I am using &lt;a href=&quot;https://github.com/alexdobin/STAR&quot; rel=&quot;noreferrer&quot;&gt;STAR&lt;/a&gt; to align RNA-seq reads to a reference genome. Before the alignment, I need to generate an index of the reference genome. I use the following code to generate the index successfully:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;STAR --runThreadN 8 --runMode genomeGenerate --genomeDir output/index/star --genomeFastaFiles ref.fa --sjdbGTFfile ref.gtf --sjdbOverhang 100&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This works fine. However, I would like to keep my reference genome compressed to save disk space. So I am trying the following command:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;STAR --runThreadN 8 --runMode genomeGenerate --genomeDir output/index/star --genomeFastaFiles ref.fa.gz --readFilesCommand &quot;gunzip -c&quot; --sjdbGTFfile ref.gtf --sjdbOverhang 100&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;EXITING because of INPUT ERROR: the file format of the genomeFastaFile: ref.fa.gz is not fasta: the first character is '' (31), not '&amp;gt;'.&#xA; Solution: check formatting of the fasta file. Make sure the file is uncompressed (unzipped).&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the readFilesCommand successfully with compressed RNA-seq fastq files. Does anybody know if there is a similar option to use compressed references? Is there a workaround using Unix commands (maybe piping?) or do I need to decompress the reference, index it and then compress it again?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-02-26T20:11:11.703" Title="Can I index a compressed FASTA file using STAR?" Tags="|sequence-alignment|genome|star|genome-index|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3641" PostTypeId="2" ParentId="3640" CreationDate="2018-02-26T20:11:11.703" Score="5" Body="&lt;p&gt;I can think of two possible workarounds. The simplest would be to try using &lt;code&gt;zcat&lt;/code&gt; instead of &lt;code&gt;gunzip -c&lt;/code&gt;, in case that works. Assuming it doesn't, you can use &lt;a href=&quot;http://www.gnu.org/software/bash/manual/bash.html#Process-Substitution&quot; rel=&quot;noreferrer&quot;&gt;process substitution&lt;/a&gt;, if your shell supports it. Something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;STAR --runThreadN 8 --runMode genomeGenerate --genomeDir output/index/star \&#xA;    --genomeFastaFiles &amp;lt;(zcat ref.fa.gz) --sjdbGTFfile ref.gtf --sjdbOverhang 100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;&amp;lt;()&lt;/code&gt; creates a file descriptor and passes that to the command. It is very useful for passing streams to commands that expect files. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your shell doesn't support this construct, assuming you are on a system that supports &lt;a href=&quot;https://en.wikipedia.org/wiki/Named_pipe&quot; rel=&quot;noreferrer&quot;&gt;named pipes&lt;/a&gt; (Linux does), you can do the same thing a bit more manually:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Create a named pipe (FIFO)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mkfifo foo&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Set it to read the uncompressed genome&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zcat ref.fa.gz &amp;gt; foo &amp;amp;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &amp;amp; is important since the command above will hang until something starts reading from &lt;code&gt;foo&lt;/code&gt;. Alternatively, you can just run this in one terminal and the next command in another. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Read the input from the pipe&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;STAR --runThreadN 8 --runMode genomeGenerate --genomeDir output/index/star \&#xA;    --genomeFastaFiles foo --sjdbGTFfile ref.gtf --sjdbOverhang 100&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-02-26T20:11:11.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3642" PostTypeId="2" ParentId="3565" CreationDate="2018-02-26T22:30:14.907" Score="3" Body="&lt;p&gt;You can use &lt;a href=&quot;https://github.com/karel-brinda/samsift&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAMsift&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samsift \&#xA;   -i file.bam \&#xA;   -0 'c={&quot;chr1&quot;:5000,&quot;chr2&quot;:5000}' \&#xA;   -f 'c[RNAME]&amp;gt;0' \&#xA;   -c 'c[RNAME]-=1' \&#xA;   -m nonstop-remove&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Explanation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;-i file.bam&lt;/code&gt; – input file&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;-0 'c={&quot;chr1&quot;:5000,&quot;chr2&quot;:5000}'&lt;/code&gt; – initialization (create a count-down dictionary for the chromosomes of interest)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;-f 'c[RNAME]&amp;gt;0'&lt;/code&gt; – filtering criterion (is the counter for the current chromosome still &gt;0?)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;-c 'c[RNAME]-=1'&lt;/code&gt; – code decrementing the counter of the current chromosome (5000 -&gt; 4999 -&gt; ... -&gt; 1 -&gt; 0 -&gt; -1 -&gt; ...)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;-m nonstop-remove&lt;/code&gt; – remove lines causing Python errors and don't stop (in this case, an error can be caused by accessing a non-existing counter for another chromosome, e.g., for chr3)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;See the &lt;a href=&quot;https://github.com/karel-brinda/samsift&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAMsift readme&lt;/a&gt; for more info.&lt;/p&gt;&#xA;" OwnerUserId="425" LastEditorUserId="425" LastEditDate="2018-02-26T23:22:00.417" LastActivityDate="2018-02-26T23:22:00.417" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3643" PostTypeId="1" CreationDate="2018-02-27T00:19:39.680" Score="1" ViewCount="724" Body="&lt;p&gt;I have a bam file aligned to a reference sequence. I am interested in finding the number of reads for read length 18 to 34, where the last base is mis-matched. I got this bam file from bowtie alignment allowing 3 mismatches (-v 0) option, so I am expecting some of the reads to have terminal bases mis-matched. How can we count the number of sequences of a specific length with terminal mismatch in bam file?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;**UPDATE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have created a bowtie aligned .sam file with this command: &#xA;&lt;code&gt;bowtie -S -p 18 /media/owner/ref.fa test1.fastq test1.sam&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which I then converted to .bam file like this: &#xA;&lt;code&gt;samtools view -bS -o test1.bam test1.sam&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this test1.bam, I was able to get the total number of plus and minus strand of reads aligned to the reference loci using bedtools coverage maping (for positives reads and negative reads) like this: &#xA;&lt;code&gt;bedtools coverage -a my_region.bed -b test1.bam -bed -d -s | gzip &amp;gt; test1_region_pos.tsv.gz&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I used default bowtie option which allows for three mismatches, I want to know how many reads are aligned with terminal mismatches (i.e,last three, last two and last one base mismatched at 3' region of the aligned reads at their loci position). &lt;/p&gt;&#xA;" OwnerUserId="2425" LastEditorUserId="2425" LastEditDate="2018-02-27T01:58:50.583" LastActivityDate="2018-02-27T02:09:14.463" Title="How to you find read statistics for bam files with terminal mismatch?" Tags="|sam|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3644" PostTypeId="2" ParentId="3643" CreationDate="2018-02-27T01:53:06.107" Score="2" Body="&lt;p&gt;Could you provide some examples of lines that should be counted? I'm not sure how mismatches are encoded in the version of Bowtie you're using. As far as I remember, Bowtie doesn't distinguish between &lt;code&gt;=&lt;/code&gt; and &lt;code&gt;X&lt;/code&gt; in CIGARs (and uses &lt;code&gt;M&lt;/code&gt; for both), therefore, mismatches must be detected from the MD tag. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use &lt;a href=&quot;http://github.com/karel-brinda/samsift&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAMsift&lt;/a&gt; for filtering the alignments. The following filtering criterion should keep only reads with  read length from 18 to 34 that have an MD tag with &lt;code&gt;0&lt;/code&gt; as the last character, which must be preceded by a non-number (this should signalise a mismatch at the last position).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samsift \&#xA;  -i test1.bam \&#xA;  -f '18 &amp;lt;= len(SEQ) &amp;lt;= 34 and MD[-1]==&quot;0&quot; and not MD[-2].isdigit()' \&#xA;  -m nonstop-remove&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The obtained alignments can be counted by adding &lt;code&gt;| samtools view | wc -l&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Explanation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;-f ...&lt;/code&gt; – filtering criterion&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;18 &amp;lt;= len(SEQ) &amp;lt;= 34&lt;/code&gt; – the allowed read lengths&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;MD[-1]==&quot;0&quot;&lt;/code&gt; – the last character of the &lt;code&gt;MD&lt;/code&gt; tag is &lt;code&gt;0&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;not MD[-2].isdigit()&lt;/code&gt; – the penultimate character of the &lt;code&gt;MD&lt;/code&gt; tag is not a digit&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;-m nonstop-remove&lt;/code&gt; – remove alignments without any &lt;code&gt;MD&lt;/code&gt; tag (probably unaligned, they would cause an error).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="425" LastEditorUserId="425" LastEditDate="2018-02-27T02:09:14.463" LastActivityDate="2018-02-27T02:09:14.463" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="3645" PostTypeId="1" AcceptedAnswerId="3650" CreationDate="2018-02-27T10:31:08.127" Score="6" ViewCount="141" Body="&lt;p&gt;I was wondering if there is any way to apply classification algorithms (e.g random forest) on microarray data but not using the genes as predictors/features but the pathway they belong to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing is that the expressions of the genes that belong to the same pathway should somehow, grouped together and transformed into a new value that will describe that pathway for each specific sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me give an example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that we have &lt;code&gt;g1, g2, g3, g4, g5, g6, g7, g8, g9, g10&lt;/code&gt; as genes&#xA;and &lt;code&gt;p1, p2, p3&lt;/code&gt; as pathways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And &lt;code&gt;g1, g2, g3&lt;/code&gt; belong to &lt;code&gt;p1&lt;/code&gt; while &lt;code&gt;g3, g4, g5, g6&lt;/code&gt;belong to &lt;code&gt;p2&lt;/code&gt; and &lt;code&gt;g2, g6, g7, g8, g9, g10&lt;/code&gt; belong to &lt;code&gt;p3&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thought is to calculate the mean expression of the above gene groups per sample and then run the classification algorithm on top of these values. But I would like to hear/read your opinion on such an approach and also your suggestions on using a different method to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final goal would be the extraction of the most significant classification predictors (pathways). &lt;/p&gt;&#xA;" OwnerUserId="704" LastActivityDate="2018-02-27T14:28:54.310" Title="Classification (supervised learning) of expression data on pathway level" Tags="|microarray|pathway|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3646" PostTypeId="1" AcceptedAnswerId="3648" CreationDate="2018-02-27T10:51:52.030" Score="1" ViewCount="538" Body="&lt;p&gt;I am running the following command to index my genome:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;STAR --runThreadN 8 --runMode genomeGenerate --genomeDir output/index/star --genomeFastaFiles ref.fa --sjdbGTFfile ref.gff3 --sjdbGTFtagExonParentTranscript Parent --sjdbOverhang 100&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I get the following output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Feb 27 10:27:37 ..... started STAR run&#xA;Feb 27 10:27:37 ... starting to generate Genome files&#xA;Feb 27 10:27:40 ... starting to sort Suffix Array. This may take a long time...&#xA;Feb 27 10:27:42 ... sorting Suffix Array chunks and saving them to disk...&#xA;Feb 27 10:28:26 ... loading chunks from disk, packing SA...&#xA;Feb 27 10:28:30 ... finished generating suffix array&#xA;Feb 27 10:28:30 ... generating Suffix Array index&#xA;terminate called after throwing an instance of 'std::bad_alloc'&#xA;  what():  std::bad_alloc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Searches for the GNU/Linux error lead to some pages about C++ that I don't understand. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Submitting it as a batch job works fine but I would like to know why it does not work on a node. It is quite practical for me to test small code like this without submitting a batch job.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The command above works fine on my laptop running Mac OSX.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The STAR version is STAR_2.5.3a on both computers: STAR_2.5.3a&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The memory usage of the node is the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;total            used       free     shared    buffers       cached&#xA;&#xA;251              251         0         0       0             232&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;so, is there 0 memory for me to use on this node?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is my indexing aborting? &lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-02-27T13:23:35.987" LastActivityDate="2018-02-27T13:23:35.987" Title="Why is my STAR reference genome indexing aborting on my GNU/Linux server but not on my Mac OS X laptop?" Tags="|sequence-alignment|rna-alignment|linux|star|genome-index|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3647" PostTypeId="1" CreationDate="2018-02-27T10:57:44.653" Score="1" ViewCount="146" Body="&lt;p&gt;I tried to create &lt;a href=&quot;https://galaxyproject.org/toolshed/&quot; rel=&quot;nofollow noreferrer&quot;&gt;galaxy local toolshed&lt;/a&gt; using &lt;a href=&quot;https://github.com/galaxyproject/planemo&quot; rel=&quot;nofollow noreferrer&quot;&gt;planemo&lt;/a&gt;. I used command as follows,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;create virtual env&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;virtualenv .venv; .venv/bin/activate&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Install planemo&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pip install planemo&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To configuring a shed account&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;planemo config_init&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Create a repository &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;planemo shed_init --name=seqtk_seq \&#xA;                  --owner=planemo \&#xA;                  --description=seqtk_seq \&#xA;                  --long_description=&quot;Tool that converts FASTQ &#xA;                                      to FASTA files using seqtk&quot; \&#xA;                  --category=&quot;Fastq Manipulation&quot;&#xA;planemo shed_create --shed_target local&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This produces the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Failed to include files for [{'source': '**'}]&#xA;Problem encountered executing action for one or more repositories.&#xA;(.venv) varuna@varuna-Inspiron-15-3567:~/git/galaxy_stable/planemo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2375" LastEditorUserId="77" LastEditDate="2018-03-06T09:36:25.670" LastActivityDate="2018-08-03T13:02:01.960" Title="Create galaxy local toolshed using Planemo" Tags="|software-installation|galaxy|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3648" PostTypeId="2" ParentId="3646" CreationDate="2018-02-27T12:41:15.767" Score="1" Body="&lt;p&gt;A &lt;code&gt;std::bad_alloc&lt;/code&gt; error tends to mean that you've run out of memory. It's not unusual for head nodes to be fairly limited in capacity, so presumably that's the issue.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-02-27T12:41:15.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3649" PostTypeId="2" ParentId="3647" CreationDate="2018-02-27T12:49:06.913" Score="1" Body="&lt;p&gt;&lt;code&gt;--shed_target local&lt;/code&gt; will only work if you're running a Galaxy toolshed on your local computer, since it's reading the &lt;code&gt;.shed.yaml&lt;/code&gt; file you made with &lt;code&gt;planemo shed_init&lt;/code&gt; and trying to create a repository with that name or a local (in this case) server. Planemo itself won't start a Galaxy toolshed, since it's fairly unusual for people to want to do testing involving creating local repositories. What you probably want to do instead is use &lt;code&gt;planemo test&lt;/code&gt; instead. Once the planemo testing passes then use the various &lt;code&gt;shed*&lt;/code&gt; commands to upload your wrapper to the toolshed or testtoolshed.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-02-27T12:49:06.913" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3650" PostTypeId="2" ParentId="3645" CreationDate="2018-02-27T14:28:54.310" Score="6" Body="&lt;p&gt;Have a look at the &lt;a href=&quot;http://bioconductor.org/packages/GSVA&quot; rel=&quot;noreferrer&quot;&gt;GSVA&lt;/a&gt; package. It allows to convert a matrix with &lt;code&gt;genes x Samples&lt;/code&gt; to a &lt;code&gt;pathways x Samples&lt;/code&gt; using several methods ssgsea, gage, gsva...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Afterwards you can use that matrix as input for differential expression of pathways or classification algorithms or whatever. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;However it depends on the input of the &quot;pathways&quot; you give it. Make sure that the pathways are correct. Also consider that the pathway topology might change with conditions.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-02-27T14:28:54.310" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3651" PostTypeId="1" AcceptedAnswerId="3652" CreationDate="2018-02-27T14:55:38.870" Score="3" ViewCount="896" Body="&lt;p&gt;&lt;strong&gt;&lt;em&gt;Disclaimer&lt;/strong&gt;: I had this issue posted on &lt;a href=&quot;https://groups.google.com/forum/#!topic/tuxedo-tools-users/i79gfggEBAQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tuxedo Tools users group&lt;/a&gt; and shared it on &lt;a href=&quot;https://twitter.com/aechchiki/status/953632647074516992&quot; rel=&quot;nofollow noreferrer&quot;&gt;Twitter&lt;/a&gt;, but could not get an answer to this from the developers, nor find documentation of this issue online. So, I'll share my solution here below and see if some of you have a better solution to add.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using &lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/file_formats/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gffread&lt;/a&gt; utility to convert transcripts from GTF as output of &lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/cufflinks/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cufflinks&lt;/a&gt; to FASTA, I get some &lt;code&gt;GFaSeqGet&lt;/code&gt; errors, as follows (here is the full log file):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ gffread transcripts.gtf -g ../genome.fa -w transcripts.fasta&#xA;Error (GFaSeqGet): end coordinate (32519) cannot be larger than sequence length 32512&#xA;Error (GFaSeqGet): end coordinate (7812) cannot be larger than sequence length 7800&#xA;Error (GFaSeqGet): end coordinate (7812) cannot be larger than sequence length 7800&#xA;Error (GFaSeqGet): end coordinate (6327) cannot be larger than sequence length 6326&#xA;Error (GFaSeqGet): end coordinate (4651) cannot be larger than sequence length 4648&#xA;Error (GFaSeqGet): subsequence cannot be larger than 3551&#xA;Error getting subseq for CUFF.29386.2 (1..3568)!&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That's a bit weird since I used the same genome file as the one used to generate the alignment file required by Cufflinks, and the coordinates overhang does not make a lot of sense to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The alignment file (SAM) was generated using &lt;a href=&quot;https://ccb.jhu.edu/software/hisat2/index.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hisat2&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ hisat2&#xA;    --rna-strandness RF&#xA;    --dta-cufflinks &#xA;    -x ./hisat_idx&#xA;    -q  &#xA;    -1 ../1A_1.fq.gz,../1B_1.fq.gz,../2A_1.fq.gz,../2B_1.fq.gz,../3A_1.fq.gz,../3B_1.fq.gz &#xA;    -2 ../1A_2.fq.gz,../1B_2.fq.gz,../2A_2.fq.gz,../2B_2.fq.gz,../3A_2.fq.gz,../3B_2.fq.gz &#xA;    -S out.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then sorted:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat out.sam | sort -k 3,3 -k 4,4n -T . &amp;gt; out_sorted.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question is: are only the few sequences reported in the log affected? Or is it a more general problem? The answer seem to be the second one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Out of gffread, alongside the log file, I actually got a FASTA file with no additional error messages during its generation (537M out of 404M GTF file) - but there are around 13k transcripts which are present in the GTF but seem to be missing from the FASTA:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat transcripts.gtf | awk '$3==&quot;transcript&quot;{print $0}' | wc -l&#xA;231139&#xA;$ cat transcripts.fasta | grep '^&amp;gt;' | wc -l&#xA;218075&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Do you know what would be a cause of this problem?&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2018-02-27T14:55:38.870" Title="gffread: GFaSeqGet errors on coordinate overhang" Tags="|rna-seq|sequence-alignment|software-recommendation|base-clipping|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3652" PostTypeId="2" ParentId="3651" CreationDate="2018-02-27T14:55:38.870" Score="2" Body="&lt;p&gt;Basically, the source of the problem is issues with hard/soft clipped bases in short-reads RNAseq reads alignments to reference genome in post-transcriptome assembly steps. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Truncating the coordinates in the gtf file to match the reference is a solution. For this, I will share a perl solution (non-official) that will do the job. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl cufftrim.pl genome.fa.fai transcripts_to_fix.gtf &amp;gt; transcripts_fixed.gtf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;genome.fa.fai&lt;/code&gt; is the fasta index of the genome file&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;transcripts_to_fix.gtf&lt;/code&gt; is the transcript set out of the assembler with issues&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;transcripts_fixed.gtf&lt;/code&gt; is the new transcript file with the repaired end coordinates&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Content of &lt;code&gt;cufftrim.pl&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/perl&#xA;use strict;&#xA;use Getopt::Std;&#xA;&#xA;my $usage = q/Usage:&#xA; cufftrim.pl genome.fa.fai transcripts.gtf &amp;gt; transcripts.trim.gtf&#xA;/;&#xA;umask 0002;&#xA;getopts('o:') || die($usage.&quot;\n&quot;);&#xA;my $outfile=$Getopt::Std::opt_o;&#xA;if ($outfile) {&#xA;  open(OUTF, '&amp;gt;'.$outfile) || die(&quot;Error creating output file $outfile\n&quot;);&#xA;  select(OUTF);&#xA;  }&#xA;# --&#xA;my $fai=shift(@ARGV) || die (&quot;$usage\nSamtools fasta index required!\n&quot;);&#xA;my $gtf=shift(@ARGV) || die (&quot;$usage\nCufflinks' transcripts file required!\n&quot;);&#xA;my %slen;&#xA;&#xA;open(FAI, $fai) || die(&quot;Error opening $fai\n&quot;);&#xA;open(GTF, $gtf) || die(&quot;Error opening $gtf\n&quot;);&#xA;while (&amp;lt;FAI&amp;gt;) {&#xA; chomp;&#xA; next if m/^#/;&#xA; my @t=split(/\t/);&#xA; die(&quot;Parsing error: $t[1] length for sequence $t[0] ?\n&quot;) &#xA;    unless $t[1]&amp;gt;10;&#xA; $slen{$t[0]}=int($t[1]);&#xA;}&#xA;close(FAI);&#xA;&#xA;# now process the GTF file&#xA;my ($tcount, $ecount); #trim counts&#xA;while (&amp;lt;GTF&amp;gt;) {&#xA; my @t=split(/\t/);&#xA; my $l=$slen{$t[0]} || die(&quot;Error: contig $t[0] not found in the fasta index!\n&quot;);&#xA; if ($t[4]&amp;gt;$l) {&#xA;   $t[4]=$l;&#xA;   $t[2] eq 'exon' ? ++$ecount : ++$tcount;&#xA; }&#xA; print join(&quot;\t&quot;, @t);&#xA;}&#xA;close(GTF);&#xA;print STDERR &quot;$tcount transcripts, $ecount exons adjusted.\n&quot;;&#xA;# --&#xA;if ($outfile) {&#xA; select(STDOUT);&#xA; close(OUTF);&#xA; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Filtering the SAM file prior to assembly to exclude clipping might also be an alternative solution. &lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2018-02-27T14:55:38.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3653" PostTypeId="1" CreationDate="2018-02-27T15:01:29.343" Score="5" ViewCount="527" Body="&lt;p&gt;I'm considering purchasing the 'MyGenome' product by Veritas Genetics to analyze my genome for a project. I'd like the data to be in FASTA format, but Veritas only provides VCF data. Is it possible to convert this VCF data into FASTA format?&lt;/p&gt;&#xA;" OwnerUserId="2429" LastEditorUserId="96" LastEditDate="2019-03-26T19:28:38.780" LastActivityDate="2019-03-26T19:28:38.780" Title="Can a data file in VCF format be converted into FASTA?" Tags="|fasta|vcf|ngs|file-formats|variant-calling|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="3654" PostTypeId="2" ParentId="3653" CreationDate="2018-02-27T15:15:03.227" Score="5" Body="&lt;p&gt;You can try &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/tooldocs/3.8-0/org_broadinstitute_gatk_tools_walkers_fasta_FastaAlternateReferenceMaker.php&quot; rel=&quot;noreferrer&quot;&gt;gatk&lt;/a&gt; function &lt;code&gt;FastaAlternateReferenceMaker&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar \&#xA;   -T FastaAlternateReferenceMaker \&#xA;   -R reference.fasta \&#xA;   -o output.fasta \&#xA;   -L input.intervals \&#xA;   -V input.vcf \&#xA;   [--snpmask mask.vcf]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastActivityDate="2018-02-27T15:15:03.227" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3655" PostTypeId="1" AcceptedAnswerId="3656" CreationDate="2018-02-27T16:27:53.620" Score="2" ViewCount="76" Body="&lt;p&gt;On my way to create a co-expression matrix for a microarray experiment, using the &lt;code&gt;coexnet&lt;/code&gt; package I had to use a function called &lt;code&gt;getAffy()&lt;/code&gt; which according to the packages' &lt;a href=&quot;https://www.bioconductor.org/packages/3.7/bioc/vignettes/coexnet/inst/doc/coexnet.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt; it :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;searches in your current or designated path file the folder with the&#xA;  GSE accession number and reads the filelist.txt file that contains the&#xA;  name of each GSM sample, in order to recognize them and join them in&#xA;  an unified AffyBatch object.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So in some way, this file is needed to create the AffyBatch object. The thing here is that there is no such a file downloaded from the &lt;code&gt;getInfo()&lt;/code&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Googling out the &lt;code&gt;filelist.txt&lt;/code&gt; I reached only this &lt;a href=&quot;https://kasperdanielhansen.github.io/genbioconductor/html/GEOquery.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;result&lt;/a&gt; from GEOQuery manual, referring to that file. And as you can see there is a command in it &lt;code&gt;eList2 &amp;lt;- getGEOSuppFiles(&quot;GSE11675&quot;)&lt;/code&gt; that is believed that it returns also the filelist.txt. I tried even with the example of the previously mentioned command, but no filelist.txt file ever fetched.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way to setup my own filelist.txt file? Does anyone know what it contains and in what form?&lt;/p&gt;&#xA;" OwnerUserId="704" LastEditorUserId="48" LastEditDate="2018-02-27T20:53:59.387" LastActivityDate="2018-02-27T20:53:59.387" Title="coexnet package for network co-expression analysis, is looking for a filelist.txt that doesn't exist" Tags="|microarray|geoquery|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3656" PostTypeId="2" ParentId="3655" CreationDate="2018-02-27T19:31:51.410" Score="1" Body="&lt;p&gt;Finally, I found it. &lt;code&gt;filelist.txt&lt;/code&gt; file is accessible from the FTP server of the NCBI. For example for that specific microarray is stored &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/geo/series/GSE47nnn/GSE47010/suppl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So there might be a problem at software side and they don't get downloaded.&lt;/p&gt;&#xA;" OwnerUserId="704" LastActivityDate="2018-02-27T19:31:51.410" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3657" PostTypeId="1" AcceptedAnswerId="3663" CreationDate="2018-02-27T23:14:41.577" Score="0" ViewCount="66" Body="&lt;p&gt;I have done some genome scan analysis and I want to identify the outlier loci. For an expected value of the FRD equal to q = 10, a list of candidate loci can be obtained by using the Benjamin-Hochberg procedure as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;q=0.1&#xA;L = length(adjusted.p.values)&#xA;W = which(sort(adjusted.p.values) &amp;lt; q * (1:L) / L)&#xA;candidates = order (adjusted.p.values)[w]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and this is part of the outliers and my output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;head(candidates)&#xA; [1]  3  4  9   10  13 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I need the output in a different type.&#xA;I want to assign TRUE if a locus has been chosen as outlier and FALSE if it has not &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  head (desired_output)&#xA;&#xA;  [1] FALSE FALSE TRUE FALSE FALSE FALSE FALSE FALSE TRUE TRUE FALSE FALSE TRUE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any idea how can I modify the Code?&lt;/p&gt;&#xA;" OwnerUserId="1749" LastEditorUserId="29" LastEditDate="2018-02-28T10:39:21.047" LastActivityDate="2018-02-28T10:45:20.370" Title="Identifying outlier loci in R" Tags="|r|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3658" PostTypeId="1" AcceptedAnswerId="3664" CreationDate="2018-02-28T00:34:42.917" Score="2" ViewCount="545" Body="&lt;p&gt;I am using &lt;a href=&quot;http://darlinglab.org/mauve/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mauve&lt;/a&gt; to align two whole genomes. I use the following command to get the alignment in xmfa format:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;/Applications/Mauve.app/Contents/MacOS/progressiveMauve ref1.fa ref2.fa --output alignment.xmfa&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then open the xmfa file in the Mauve GUI and export the SNPs. Is there a way of doing this using a command? I have tried to follow the procedure described &lt;a href=&quot;https://sourceforge.net/p/mauve/mailman/message/31871278/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; but the &lt;a href=&quot;http://gel.ahabs.wisc.edu/mauve/snapshots&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; to snapshot releases of Mauve does not work anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could not find anything in the Mauve command line help:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;/Applications/Mauve.app/Contents/MacOS/progressiveMauve --help&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;nor on the &lt;a href=&quot;http://darlinglab.org/mauve/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mauve website&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried the procedure described by Konrad Rudolph. I wrote:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MAUVE_DIR=/Applications/Mauve.app/Contents/Java&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in the terminal. Is this correct? Don't I need an export and quotation marks to create a bash variable? I tried adding an export and using quotation marks for the file but the error below remained the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;followed by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;export CLASSPATH=&quot;$(find &quot;$MAUVE_DIR&quot; -name \*.jar -print0 | tr '\0' :)$CLASSPATH&quot;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;followed by:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;java org.gel.mauve.analysis.SnpExporter -f alignment.xmfa -o aln.snps&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and it seems to work.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="96" LastEditDate="2019-01-15T14:25:39.610" LastActivityDate="2019-01-15T14:25:39.610" Title="Can I export the SNPs between whole genome alignments using the command line?" Tags="|sequence-alignment|snp|variation|mauve|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3659" PostTypeId="5" CreationDate="2018-02-28T00:35:48.333" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-02-28T00:35:48.333" LastActivityDate="2018-02-28T00:35:48.333" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3660" PostTypeId="4" CreationDate="2018-02-28T00:35:48.333" Score="0" Body="Questions regarding the Multiple Genome Alignment software Mauve developed by the Darling lab." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-01T08:58:06.303" LastActivityDate="2018-03-01T08:58:06.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3661" PostTypeId="2" ParentId="3657" CreationDate="2018-02-28T07:57:37.930" Score="-1" Body="&lt;p&gt;Remove the &lt;code&gt;which()&lt;/code&gt;, which is converting the boolean vector to a vector if indices to &lt;code&gt;TRUE&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-02-28T07:57:37.930" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3662" PostTypeId="1" AcceptedAnswerId="3665" CreationDate="2018-02-28T09:57:07.297" Score="3" ViewCount="804" Body="&lt;p&gt;A typical VCF file has:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##contig=&amp;lt;ID=chr1,length=248956422&amp;gt;&#xA;##contig=&amp;lt;ID=chr2,length=242193529&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to use &lt;code&gt;htslib&lt;/code&gt; in C++ to read it. My attempt:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;htsFile *fp = bcf_open(&quot;my.vcf&quot;, &quot;r&quot;);&#xA;bcf_hdr_t *hdr = bcf_hdr_read(fp); &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://github.com/samtools/htslib/blob/develop/htslib/vcf.h&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/samtools/htslib/blob/develop/htslib/vcf.h&lt;/a&gt;, I'm not able to find a function that can do that for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to read &lt;code&gt;chr1&lt;/code&gt; and &lt;code&gt;248956422&lt;/code&gt; in C++?&lt;/p&gt;&#xA;" OwnerUserId="174" LastActivityDate="2018-02-28T14:10:01.030" Title="How to use C++ htslib to read VCF contig name and size?" Tags="|c++|htslib|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3663" PostTypeId="2" ParentId="3657" CreationDate="2018-02-28T10:45:20.370" Score="3" Body="&lt;p&gt;As Devon said, removing the &lt;code&gt;which&lt;/code&gt; will give you &lt;code&gt;W&lt;/code&gt; as a logical vector that can be used to index the &lt;em&gt;sorted&lt;/em&gt; p-values. To get a logical vector that can be used to subset the original (unsorted) values you’ll need to go a step further:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;candidates = is.element(1 : L, order(adjusted.p.values)[W])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I’m not sure this will work with your data: the name &lt;code&gt;adjusted.p.values&lt;/code&gt; suggests that these values are &lt;em&gt;already&lt;/em&gt; FDR-adjusted. So you cannot/should not perform additional correction on them. If they are already adjusted, then you can get candidate loci simply by&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;candidates = adjusted.p.values &amp;lt; q&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29" LastActivityDate="2018-02-28T10:45:20.370" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3664" PostTypeId="2" ParentId="3658" CreationDate="2018-02-28T11:11:23.110" Score="2" Body="&lt;p&gt;It works for me with the &lt;a href=&quot;http://darlinglab.org/mauve/download.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;download from download page&lt;/a&gt;. The download archive contains &lt;code&gt;Mauve.jar&lt;/code&gt; so the command line in the answer you link to should work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you might need to include all the jar files (not just &lt;code&gt;Mauve.jar&lt;/code&gt;) in the classpath for this to command work, though. For instance, the following works for me:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;MAUVE_DIR=‹the-dir-containing-Mauve.jar›&#xA;export CLASSPATH=&quot;$(find &quot;$MAUVE_DIR&quot; -name \*.jar -print0 | tr '\0' :)$CLASSPATH&quot;&#xA;java org.gel.mauve.analysis.SnpExporter -f aln.xmfa -o aln.snps&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;… there’s probably a smarter way to make this work but I don’t use Java enough to know how.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2018-02-28T11:11:23.110" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3665" PostTypeId="2" ParentId="3662" CreationDate="2018-02-28T14:10:01.030" Score="4" Body="&lt;pre class=&quot;lang-c prettyprint-override&quot;&gt;&lt;code&gt;#include &quot;htslib/vcf.h&quot;&#xA;&#xA;int main(int argc, char *argv[])&#xA;{&#xA;    htsFile *fp;&#xA;    bcf_hdr_t *hdr;&#xA;    bcf_idpair_t *ctg;&#xA;    int i;&#xA;    if (argc == 1) {&#xA;        fprintf(stderr, &quot;Usage: print-ctg &amp;lt;in.vcf&amp;gt;\n&quot;);&#xA;        return 1;&#xA;    }&#xA;    fp = vcf_open(argv[1], &quot;r&quot;);&#xA;    hdr = vcf_hdr_read(fp);&#xA;    ctg = hdr-&amp;gt;id[BCF_DT_CTG];&#xA;    for (i = 0; i &amp;lt; hdr-&amp;gt;n[BCF_DT_CTG]; ++i)&#xA;        printf(&quot;%s\t%d\n&quot;, ctg[i].key, ctg[i].val-&amp;gt;info[0]);&#xA;    bcf_hdr_destroy(hdr);&#xA;    vcf_close(fp);&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;On stability: this use has been in htslib forever. In general, functions/structs/variables in the public headers are meant to be stable. However, there is no guarantee that future versions will always keep the same APIs.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-02-28T14:10:01.030" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3666" PostTypeId="2" ParentId="3600" CreationDate="2018-02-28T16:21:13.677" Score="12" Body="&lt;p&gt;You can use &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/karyoploteR.html&quot; rel=&quot;noreferrer&quot;&gt;karyoploteR&lt;/a&gt; to plot your regions on an ideogram quite easily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Disclosure:&lt;/strong&gt; I'm one of the authors of the tool.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this example I'll start creating 4 GRanges with random regions, but with your data you should load your bed into R and split it into four GRanges.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(karyoploteR)&#xA;&#xA;&#xA;typeA &amp;lt;- createRandomRegions(nregions = 100, length.mean = 5e6, length.sd = 3e6)&#xA;typeB &amp;lt;- createRandomRegions(nregions = 20, length.mean = 15e6, length.sd = 10e6)&#xA;typeC &amp;lt;- createRandomRegions(nregions = 1000, length.mean = 5e5, length.sd = 2e5)&#xA;typeD &amp;lt;- createRandomRegions(nregions = 100, length.mean = 2e6, length.sd = 5e6)&#xA;&#xA;colors &amp;lt;- c(A=&quot;#FFD700&quot;, B=&quot;#00BFFF&quot;, C=&quot;#FF7F50&quot;, D=&quot;#90EE90&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And we can plot them. We have to start creating an empty ideogram with &lt;code&gt;plotKaryotype&lt;/code&gt; (we are working with human here, but you can use &lt;a href=&quot;https://bernatgel.github.io/karyoploter_tutorial//Tutorial/CustomGenomes/CustomGenomes.html&quot; rel=&quot;noreferrer&quot;&gt;any genome you want&lt;/a&gt;) and then plot the regions in the GRanges with &lt;a href=&quot;https://bernatgel.github.io/karyoploter_tutorial//Tutorial/PlotRegions/PlotRegions.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;kpPlotRegions&lt;/code&gt;&lt;/a&gt;. As a bonus, if two regions overlap it will plot them one above the other.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;kp &amp;lt;- plotKaryotype(genome = &quot;hg19&quot;)&#xA;&#xA;kpPlotRegions(kp, data=typeA, r0=0, r1=0.2, col=colors[&quot;A&quot;])&#xA;kpPlotRegions(kp, data=typeB, r0=0.25, r1=0.45, col=colors[&quot;B&quot;])&#xA;kpPlotRegions(kp, data=typeC, r0=0.5, r1=0.7, col=colors[&quot;C&quot;])&#xA;kpPlotRegions(kp, data=typeD, r0=0.75, r1=0.95, col=colors[&quot;D&quot;])&#xA;legend(x = &quot;right&quot;, legend = names(colors), fill = colors, border=NA)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And you'll get something similar to this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/u9HKe.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/u9HKe.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this example we have fairly large regions and not a lot of them. As @heathobrien pointed out, plotting the actual regions in the whole genome plot might not be the best visualization option. You can either use &lt;a href=&quot;https://bernatgel.github.io/karyoploter_tutorial//Tutorial/Zoom/Zoom.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;zoom&lt;/code&gt;&lt;/a&gt; to create different plots of smaller parts of the genome or change your visualization to show the density of regions along the genome with &lt;a href=&quot;https://bernatgel.github.io/karyoploter_tutorial//Tutorial/PlotDensity/PlotDensity.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;kpPlotDensity&lt;/code&gt;&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plotting the density and tweaking a bit the visualization (and changing the example to have 10 times more regions) with this codei&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;typeA &amp;lt;- createRandomRegions(nregions = 1000, length.mean = 5e6, length.sd = 3e6)&#xA;typeB &amp;lt;- createRandomRegions(nregions = 200, length.mean = 15e6, length.sd = 10e6)&#xA;typeC &amp;lt;- createRandomRegions(nregions = 10000, length.mean = 5e5, length.sd = 2e5)&#xA;typeD &amp;lt;- createRandomRegions(nregions = 1000, length.mean = 2e6, length.sd = 5e6)&#xA;&#xA;kp &amp;lt;- plotKaryotype(genome = &quot;hg19&quot;, plot.type = 4, &#xA;                    ideogram.plotter = NULL, labels.plotter = NULL, main=&quot;Density&quot;)&#xA;kpAddCytobandsAsLine(kp)&#xA;kpAddChromosomeNames(kp, srt=45)&#xA;&#xA;kpPlotDensity(kp, data=typeA, r0=0, r1=0.2, col=colors[&quot;A&quot;], window.size     = 10e6)&#xA;kpAddLabels(kp, labels = &quot;Type A&quot;, r0=0, r1=0.2)&#xA;kpPlotDensity(kp, data=typeB, r0=0.25, r1=0.45, col=colors[&quot;B&quot;], window.size = 10e6)&#xA;kpAddLabels(kp, labels = &quot;Type B&quot;, r0=0.25, r1=0.45)&#xA;kpPlotDensity(kp, data=typeC, r0=0.5, r1=0.7, col=colors[&quot;C&quot;], window.size = 10e6)&#xA;kpAddLabels(kp, labels = &quot;Type C&quot;, r0=0.5, r1=0.7)&#xA;kpPlotDensity(kp, data=typeD, r0=0.75, r1=0.95, col=colors[&quot;D&quot;], window.size = 10e6)&#xA;kpAddLabels(kp, labels = &quot;Type D&quot;, r0=0.75, r1=0.95)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;it would look something like this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/e44ez.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/e44ez.png&quot; alt=&quot;Whole Genome Plot of density of regions using karyoploteR&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plots could be tweaked a lot changing various parameters and you could add other data on top of it if needed. You can find more information on the Bioconductor landing page or at the &lt;a href=&quot;https://bernatgel.github.io/karyoploter_tutorial/&quot; rel=&quot;noreferrer&quot;&gt;karyoploteR's Tutorial and Examples page&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="936" LastEditorUserId="936" LastEditDate="2018-03-01T07:53:18.620" LastActivityDate="2018-03-01T07:53:18.620" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3667" PostTypeId="1" AcceptedAnswerId="3668" CreationDate="2018-02-28T18:55:00.440" Score="12" ViewCount="17061" Body="&lt;p&gt;I am trying to find the best way to convert VCF files to PLINK binary bed/bim/fam files, but it seems like there are many varied ways to do this. (For example, using &lt;a href=&quot;https://www.cog-genomics.org/plink2/input&quot; rel=&quot;noreferrer&quot;&gt;Plink 1.9 --vcf tag&lt;/a&gt;, &lt;a href=&quot;http://apol1.blogspot.com/2014/11/best-practice-for-converting-vcf-files.html&quot; rel=&quot;noreferrer&quot;&gt;bcftools&lt;/a&gt;, &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/tooldocs/3.8-0/org_broadinstitute_gatk_tools_walkers_variantutils_VariantsToBinaryPed.php&quot; rel=&quot;noreferrer&quot;&gt;GATK&lt;/a&gt;, and &lt;a href=&quot;https://vcftools.github.io/documentation.html#plink&quot; rel=&quot;noreferrer&quot;&gt;vcftools&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously they all probably have their uses and you could likely spend a while discussing this, but can anyone give a short, simple explanation as to whether the methods are generally the same, or whether one or another is generally best for certain situations? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1574" LastEditorUserId="2787" LastEditDate="2018-05-10T05:08:19.180" LastActivityDate="2022-02-22T20:59:17.407" Title="Converting VCF file to PLINK bed/bim/fam files" Tags="|vcf|plink|gatk|vcftools|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3668" PostTypeId="2" ParentId="3667" CreationDate="2018-02-28T21:38:45.087" Score="8" Body="&lt;p&gt;Given that Plink reads in VCF files natively, and Plink is your desired output format, I expect that &lt;code&gt;plink --vcf &amp;lt;file&amp;gt;&lt;/code&gt; would be the best option.&lt;/p&gt;&#xA;&lt;p&gt;As mentioned by @sarah, Plink uses the sample IDs to create family/individual IDs, see  &amp;quot;Variant Call Format&amp;quot; section on this &lt;a href=&quot;https://www.cog-genomics.org/plink2/input&quot; rel=&quot;nofollow noreferrer&quot;&gt;page&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;VCF files just contain sample IDs, instead of the distinct family and&#xA;within-family IDs tracked by PLINK. We offer three ways to convert&#xA;these IDs:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;--double-id causes both family and within-family IDs to be set to the sample ID.&#xA;--const-fid converts sample IDs to within-family IDs while setting all family IDs to a single value (default '0').&#xA;--id-delim causes sample IDs to be parsed as &amp;lt;FID&amp;gt;&amp;lt;delimiter&amp;gt;&amp;lt;IID&amp;gt;; the default delimiter is '_'. If any sample ID&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;does not contain exactly one instance of the delimiter, an error is&#xA;normally reported; however, if you have simultaneously specified&#xA;--double-id or --const-fid, PLINK will fall back on that approach to handle zero-delimiter IDs.&lt;/p&gt;&#xA;&lt;p&gt;If none of these three flags is present, the loader defaults to&#xA;--double-id + --id-delim.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There are caveats associated with the types of variants that Plink will pull in from the VCF files, mostly related to limitations of Plink's own file formats. But if your end-goal is Plink, then that should be fine.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-02-22T20:59:17.407" LastActivityDate="2022-02-22T20:59:17.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3670" PostTypeId="1" AcceptedAnswerId="3681" CreationDate="2018-03-01T04:07:12.210" Score="6" ViewCount="884" Body="&lt;p&gt;I have a (small) BAM file with CIGAR and MD fields. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question 1: What tools exists in Python and/or R to reconstruct the alignment between the reference and the read in a BAM? Given that this is a very standard analysis, I feel that there should be R packages or Python libraries which have functions which do this task given a BAM input...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal was to create a *tsv file with two columns, one with the reconstructed alignment and the other column with the corresponding sequence. This would be useful in my analysis. (I would either use R data.table, or pandas in Python.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is read and reference:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CGGGCCGGTCCCCCCCCGCCGGGTCCGCCCCCGGC&#xA;|||||||||||||||| ||||||||||||||||| &#xA;CGGGCCGGTCCCCCCC-GCCGGGTCCGCCCCCGGG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the desired output dataframe (with only the one example row). We could use an R data.table/data.frame, or a pandas DataFrame. Column1 are the alignments (i.e. the reconstruction of the alignment between the reference and the read based on the CIGAR and MD strings), Column 2 are the corresponding sequences. :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ReadAligned    Reference   &#xA;CGGGCCGGTCCCCCCC-GCCGGGTCCGCCCCCGGG    CGGGCCGGTCCCCCCCCGCCGGGTCCGCCCCCGGC&#xA;...    ....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Question 2:&#xA;Using the CIGAR/MD tags, one could then create columns with the number of insertions and the number of deletions. This problem feels much more straightforward, e.g. the python library &lt;code&gt;cigar&lt;/code&gt; or the cigar parsing functions of &lt;code&gt;pysam&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="146" LastEditorUserId="29" LastEditDate="2018-04-24T11:21:21.107" LastActivityDate="2018-11-26T01:59:37.483" Title="Generating the reconstructed alignment from BAM" Tags="|r|python|sam|sequence-alignment|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3671" PostTypeId="2" ParentId="3631" CreationDate="2018-03-01T05:26:09.037" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;Genes, particularly transcribed genes in Eukaryotes, can have alternate start and stop sites (and splicing).  Alternate transcriptional start sites may be found in different tissues or at different developmental periods due to alternate transcription factor binding sites being used.  The only reliable way to determine the transcriptional start and stop sites is by read coverage.&lt;/li&gt;&#xA;&lt;li&gt;Genome annotations are generally the product of bioinformatic pipelines and are mostly predictions based upon computational models.  Different gene-finders may call different boundaries; for example, in eukaryotic genomes, an exon may be assigned to the wrong up/down-stream gene.  Arabadopsis, as model organism, has had it's gene boundaries considerably refined/corrected/validated, but even so, there could still exist errors in the annotation or, more likely, the boundaries indicated are the most common start/stop sites (see point 1 above).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In short, your UTRs are whatever your experimental data (reads) show it to be.  Don't worry too much about it not matching up to the standard annotation boundaries.&lt;/p&gt;&#xA;" OwnerUserId="2436" LastActivityDate="2018-03-01T05:26:09.037" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3672" PostTypeId="2" ParentId="807" CreationDate="2018-03-01T05:41:23.087" Score="1" Body="&lt;p&gt;I believe some of these answers misinterpret the original question.  The OP (I believe) doesn't wish to find particular sequences, so indexing is unnecessary.  I believe the OP just wants a random sampling of the dataset.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If the number of reads you need in small, sure you can seek on the file and return the next complete record.  Remember the read IDs to sample without replacement.  Jumping around on disk is slow, however, and you'll be reading 4k or whatever your disk buffer size is every time.&lt;/li&gt;&#xA;&lt;li&gt;If the number of reads you need is large, read the entire file and decide for each read whether or not to sample it.&lt;/li&gt;&#xA;&lt;li&gt;The way I do it (for Illumina): estimate the number of reads using the file size and read length and header length, decide how many reads I want, skip the first bunch of reads (due to edge effect, e.g. 20-100k; you can use seek or just read from start and discard), and read the next batch sequentially.  I suppose it depends on what you're using the reads for; i.e. if you want random reads from your sample for biological analysis, or you want random reads from your run for QC analysis.  This method is more appropriate for the former, but works well enough for the latter too.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2436" LastActivityDate="2018-03-01T05:41:23.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3673" PostTypeId="2" ParentId="361" CreationDate="2018-03-01T05:55:22.123" Score="0" Body="&lt;p&gt;Here are perl 1-liners for fasta and fastq files that are fast enough if you just want totals:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;$ perl -ne 'BEGIN { my $n=0 }; /^&amp;gt;/ or $n += s/N//gi; END { print &quot;N=$n\n&quot; }' contigs.fasta&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each line of a FASTA file, if not header (/^&gt;/) then it's a sequence line. &#xA; Substitute Ns, which returns the number of substitutions made, which is added to the total.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;$ zcat reads.fastq | perl -ne 'BEGIN { my $n=0 }; /^[ATCGN]+$/i and $n += s/N//gi; END { print &quot;N=$n\n&quot; }'&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each sequence line of a FASTQ file, matched by /^[ATCGN]+$/i (i.e. all NTs), then count substitutions made in the same way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may use pigz instead of zcat (or gunzip -c) if you have pigz installed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An uncompressed fastq file of 14M 250bp Illumina reads completed in 40sec.&lt;/p&gt;&#xA;" OwnerUserId="2436" LastEditorUserId="2436" LastEditDate="2018-03-01T23:15:30.270" LastActivityDate="2018-03-01T23:15:30.270" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3674" PostTypeId="2" ParentId="3546" CreationDate="2018-03-01T06:10:34.227" Score="0" Body="&lt;p&gt;Yes, this is commonly observed and both the previous answers (Devon Ryan and conchoecia) are spot on so I won't reiterate their explanations on why.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try bfc (bloom-filter correction) to &quot;fix&quot; likely erroneous base-calls (i.e. uncharacteristically rare kmers) in the reads prior to assembly.  This will not only help your assembly, but it shall require less RAM and time.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also recommend trying running your assembler with multiple kmer parameters, as some values will work better than others for a particular dataset.  You can simply choose the best assembly or go through a more complicated effort to merge them, depending on your circumstances.&lt;/p&gt;&#xA;" OwnerUserId="2436" LastActivityDate="2018-03-01T06:10:34.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3675" PostTypeId="2" ParentId="3604" CreationDate="2018-03-01T06:16:59.823" Score="1" Body="&lt;p&gt;Since biological samples and reagents cost time and money, I wouldn't push it too much.  Checking your email would be fine, but I wouldn't be browsing the web (besides gmail/calendar) in case a plug-in (e.g. flash), bad javascript, etc. causes any trouble.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd disable any auto-updates too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Locking the screen would be fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally, I'd buy a cheap chromebook and let the Mac work.&lt;/p&gt;&#xA;" OwnerUserId="2436" LastActivityDate="2018-03-01T06:16:59.823" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3676" PostTypeId="2" ParentId="3636" CreationDate="2018-03-01T06:58:44.520" Score="1" Body="&lt;p&gt;For Illumina (and 454) reads, the quality decreases with read length.  It's not linear and is run/library-dependent.  It has less to do with exhaustion of reagents and more to do with the strands on a spot being out of phase due to incomplete/missed base incorporation during sequencing.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is common practice to trim off the low quality 3' ends as the first step in QC.  Most analyses and most can deal with a few low errors in the middle of the read and what you do about such sequencing errors may vary depending on what type and use of the data.  But in general, nobody wants the garbage at the end of the reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. Even for the same library prep, the fwd/rev runs will have different quality curves.&#xA;P.P.S. Sometimes the first couple bases are also poor too.&lt;/p&gt;&#xA;" OwnerUserId="2436" LastActivityDate="2018-03-01T06:58:44.520" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3677" PostTypeId="2" ParentId="951" CreationDate="2018-03-01T07:24:44.267" Score="2" Body="&lt;p&gt;With metagenomes, there's several strategies and no one-size-fits-all. &#xA; You can spend a lot of time playing with the data to try to get the &quot;best assembly.&quot;  But it depends on what you're trying to do, e.g. whether or not you are mostly interested in the highest abundant species or are interested in the less abundant ones too.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But to propose a strategy that seems reasonable and hasn't been mentioned yet, my initial thoughts are that since the gut microbiome isn't complex, you can likely get decent assemblies from each run separately.  Then you can run a contig clustering algorithm (e.g. MetaBAT) to bin the contigs into genomes.  Next compare the bins from the various assemblies (i.e. cluster the clusters).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It may or may not be worth the trouble to try to combine the related bins' contigs into a consensus (pan)genome sequence.  If you want to, methods that have been used successfully include shredding the contigs into overlapping chunks and assembling with an OLC assembler.  But if you're interested in KO:read-counts (for example), a consensus genome sequence isn't really required.  Cross-mapping the unassembled reads can be useful too (i.e. a sample may have a low-abundance OTU that doesn't produce good contigs, but it can be associated with an OTU that is better assembled in another sample).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you provide more detail on your goal, I can comment further.&lt;/p&gt;&#xA;" OwnerUserId="2436" LastActivityDate="2018-03-01T07:24:44.267" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3678" PostTypeId="1" AcceptedAnswerId="3819" CreationDate="2018-03-01T12:25:31.853" Score="2" ViewCount="282" Body="&lt;p&gt;I have got PacBio data from a Sequel machine at a coverage &gt; 50X. I have run it in SMRT Link with their Base Modification Detection algorithm. I am now trying to figure out how to interpret the output and where to find the information about the modified bases. One of the outputs is a GFF3 file. The top of this file looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; ##gff-version 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; I kinModCall  modified_base   1168    1168    31  -   .   coverage=48;context=TTAAACTAATTTCCAAAAATTTTGAACAGTTTTTAATACAA;IPDRatio=2.96&#xA;&amp;gt; I kinModCall  modified_base   1497    1497    32  -   .   coverage=50;context=CTTGAAAACACGAATATCTGTTTAGACAATAGTTTTGCCAA;IPDRatio=3.19&#xA;&amp;gt; I kinModCall  modified_base   1509    1509    33  -   .   coverage=48;context=CATGAAAAATTTCTTGAAAACACGAATATCTGTTTAGACAA;IPDRatio=2.84&#xA;&amp;gt; I kinModCall  modified_base   1529    1529    39  -   .   coverage=46;context=AAGAAAAGACCAAGAAAAACCATGAAAAATTTCTTGAAAAC;IPDRatio=3.84&#xA;&amp;gt; I kinModCall  modified_base   1904    1904    36  -   .   coverage=54;context=TAAAACCTGACAGAAAGAATTGGTCCCGTCATCTACTTTCT;IPDRatio=2.86&#xA;&amp;gt; I kinModCall  modified_base   2152    2152    31  -   .   coverage=59;context=TTAAAAATTAATATAAAATCAGGGAAATTTTTTTAAATTTT;IPDRatio=2.98&#xA;&amp;gt; I kinModCall  modified_base   2226    2226    36  -   .   coverage=63;context=TGGGCGGGGCGTCTCTGCCGGGAAAATTTTGCGTTTAAAAA;IPDRatio=2.59&#xA;&amp;gt; I kinModCall  modified_base   2228    2228    34  -   .   coverage=52;context=CGTGGGCGGGGCGTCTCTGCCGGGAAAATTTTGCGTTTAAA;IPDRatio=2.98&#xA;&amp;gt; I kinModCall  modified_base   2237    2237    33  -   .   coverage=60;context=GCACGGTTTCGTGGGCGGGGCGTCTCTGCCGGGAAAATTTT;IPDRatio=3.07&#xA;&amp;gt; I kinModCall  modified_base   2240    2240    44  -   .   coverage=63;context=GCGGCACGGTTTCGTGGGCGGGGCGTCTCTGCCGGGAAAAT;IPDRatio=2.89&#xA;&amp;gt; I kinModCall  modified_base   2311    2311    31  -   .   coverage=63;context=AAAATCGACGAAAATCGGTATAAAATCAAATAAAAATAGAA;IPDRatio=4.35&#xA;&amp;gt; I kinModCall  modified_base   2313    2313    39  -   .   coverage=62;context=GAAAAATCGACGAAAATCGGTATAAAATCAAATAAAAATAG;IPDRatio=4.00&#xA;&amp;gt; I kinModCall  modified_base   2314    2314    34  -   .   coverage=63;context=AGAAAAATCGACGAAAATCGGTATAAAATCAAATAAAAATA;IPDRatio=2.63&#xA;&amp;gt; I kinModCall  modified_base   2333    2333    35  -   .   coverage=62;context=AAAAAAAAGAGAAAAAATGAGAAAAATCGACGAAAATCGGT;IPDRatio=3.77&#xA;&amp;gt; I kinModCall  modified_base   2334    2334    35  -   .   coverage=63;context=CAAAAAAAAGAGAAAAAATGAGAAAAATCGACGAAAATCGG;IPDRatio=2.77&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I expected something along the lines of 'chromosome I position 1497 methylcytosine', etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it seems that the algorithm has detected modified bases at the following positions but it cannot call them, maybe because of a lack of coverage?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: the screenshot below shows there is no CSV file in the Data tab of the completed analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EIdjU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EIdjU.png&quot; alt=&quot;Screenshot from SMRT Link&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-09T01:44:57.557" LastActivityDate="2018-03-12T16:02:54.507" Title="How to interpret the SMRT Link base modification algorithm output?" Tags="|long-reads|pacbio|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3679" PostTypeId="1" CreationDate="2018-03-01T13:19:01.197" Score="6" ViewCount="244" Body="&lt;p&gt;I have a number of single-cell projects processed with &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/what-is-cell-ranger&quot; rel=&quot;noreferrer&quot;&gt;cellranger from 10x Genomics&lt;/a&gt;. The pipeline produces a number of handy metrics that are summarized for each Illumina sample in &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/output/summary&quot; rel=&quot;noreferrer&quot;&gt;web page&lt;/a&gt;. These include things like the estimated number of cells, the fraction of reads in cells, and mapping rates. What I would like is to aggregate this information from multiple Illumina samples into a single file. &lt;a href=&quot;http://multiqc.info/&quot; rel=&quot;noreferrer&quot;&gt;MultiQC&lt;/a&gt; is the obvious choice for such a task, but it appears that no one has written a module to process cellranger data for it. Are there any other prepackaged ways to perform this aggregation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there's no convenient tool for this I'll just write a MultiQC module, but I'd obviously like to avoid the extra effort unless it's really needed.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-04-22T00:08:25.960" Title="Aggregate sequencing/mapping/etc. metrics from cellranger across Illumina samples" Tags="|scrnaseq|cellranger|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3680" PostTypeId="1" CreationDate="2018-03-01T14:59:40.407" Score="2" ViewCount="158" Body="&lt;p&gt;I am wondering if anyone has tried aligning 1d^2 RNA-seq data. I am trying that on minimap2. It is giving me very low ~50% of mapped reads. it gives ~95% mapped reads for 1D.&lt;/p&gt;&#xA;&lt;p&gt;I am using the following command for 1d and 1d^2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 -ax splice -uf -k14 ref.fa direct-rna.fq &amp;gt; aln.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Update:&lt;/p&gt;&#xA;&lt;p&gt;I also tried to play around the command using :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 -ax splice -k14 -a ref.fa direct-rna.fq &amp;gt; aln.sam&#xA;minimap2 -ax splice -ur -k14 ref.fa direct-rna.fq &amp;gt; aln.sam&#xA;minimap2 -ax splice -ub -k14 ref.fa direct-rna.fq &amp;gt; aln.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It did not work better.&lt;/p&gt;&#xA;&lt;p&gt;FYI: I am trying to find answer for this from few days. I have posted this question on &lt;a href=&quot;https://www.biostars.org/p/300946/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt; before. I did not get any response. Hence, I am trying to find the answer here.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance. I really appreciate any help.&lt;/p&gt;&#xA;" OwnerUserId="2381" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2023-09-14T07:03:25.360" Title="aligner for 1D^2 oxford nanopore data" Tags="|sequence-alignment|nanopore|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3681" PostTypeId="2" ParentId="3670" CreationDate="2018-03-01T18:42:45.407" Score="5" Body="&lt;p&gt;If all you want are the number of insertions/deletions, you can use &lt;a href=&quot;https://pypi.python.org/pypi/cigar/0.1&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt; to extract that info:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from cigar import Cigar&#xA;c = c=Cigar('20M5I10M3I5M')&#xA;num_insertions = len(list(filter(lambda x: x[1] == 'I', c.items())))&#xA;insertion_length = sum([x[0] for x in list(filter(lambda x: x[1] == 'I', c.items()))])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Deletions would be the same thing with &lt;code&gt;'D'&lt;/code&gt; in place of &lt;code&gt;'I'&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-03-01T18:42:45.407" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3682" PostTypeId="1" AcceptedAnswerId="3779" CreationDate="2018-03-02T02:29:07.333" Score="5" ViewCount="106" Body="&lt;p&gt;I have used canu (correct) and smartdenovo to assembly PacBio reads. Next I am going to polish my assembly using illumina pair end reads by Pilon. However, I found there is high heterozygosity in my pair end reads when I used jellyfish to do k-mer analysis. There are two peaks in the k-mer frequencies distribution plot. The size of my genome is about 190M, and it is a diploid organism.&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/y93rV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/y93rV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;What I am doubting is that can I use this pair end data to polish my assembly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, I have to do sequencing again. &lt;/p&gt;&#xA;" OwnerUserId="2442" LastEditorUserId="2442" LastEditDate="2018-03-07T01:13:22.440" LastActivityDate="2018-03-07T01:13:22.440" Title="Can pair end reads with high heterozygosity be used to polish PacBio assembly?" Tags="|pacbio|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3683" PostTypeId="2" ParentId="3670" CreationDate="2018-03-02T06:29:51.457" Score="5" Body="&lt;p&gt;I am answering question 1. However, as I am unfamiliar with python, I am providing a javascript solution only to show the basic logic. To try the code at the end, you need to install node.js and run it with &lt;code&gt;node this-script.js&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Given that this is a very standard analysis&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I tend to believe this is a very rare analysis. I have only seen it done twice. Converting CIGAR+MD to padded alignment is complicated in coding, as you have to keep sequence, cigar and MD in sync all the time. The code below generates two strings in one go. It may be logically easier to generate the reference and the query strings separately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, MD is ill defined. It is too difficult to work with. If possible, avoid this tag. You can do a lot of things with the combination of CIGAR and NM.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-js prettyprint-override&quot;&gt;&lt;code&gt;function md2pad(CIGAR, MD, SEQ) {&#xA;    var m, re_cigar = /(\d+)([MIDSHNX=])/g, re_MD = /(\d+)|(\^[A-Za-z]+)|([A-Za-z])/g;&#xA;    var cigar = [];&#xA;    while ((m = re_cigar.exec(CIGAR)) != null) // parse CIGAR into an array&#xA;        if (m[2] != 'H') // do nothing for hard clipping, as it has no effect&#xA;            cigar.push([parseInt(m[1]), m[2]]);&#xA;    var k = 0, sx = '', sy = ''; // k: k-th cigar operator; x: ref; y: query&#xA;    // cx/cy: start coordinate of cigar[k] on ref/query; mx/my: start of current MD op&#xA;    var cx = 0, cy = 0, mx = 0, my = 0;&#xA;    while ((m = re_MD.exec(MD)) != null) {&#xA;        if (m[2] != null) { // deletion from the reference&#xA;            var len = m[2].length - 1; // length of deletion&#xA;            sx += m[2].substr(1), sy += Array(len+1).join(&quot;-&quot;);&#xA;            mx += len, cx += len, ++k; // consume a deletion D from ref&#xA;        } else { // copy or mismatch&#xA;            var ml = m[1] != null? parseInt(m[1]) : 1; // length of this MD operation&#xA;            while (k &amp;lt; cigar.length &amp;amp;&amp;amp; cigar[k][1] != 'D') {&#xA;                var cl = cigar[k][0], op = cigar[k][1];&#xA;                if (op == 'M' || op == 'X' || op == '=') {&#xA;                    if (my + ml &amp;lt; cy + cl) { // an MD ends in the middle of cigar M&#xA;                        if (ml &amp;gt; 0) {&#xA;                            if (m[3] != null) sx += m[3], sy += SEQ[my];&#xA;                            else sx += SEQ.substr(my, ml), sy += SEQ.substr(my, ml);&#xA;                        }&#xA;                        mx += ml, my += ml, ml = 0;&#xA;                        break;&#xA;                    } else {&#xA;                        var dl = cy + cl - my;&#xA;                        sx += SEQ.substr(my, dl), sy += SEQ.substr(my, dl);&#xA;                        cx += cl, cy += cl, ++k; // consume an M operator&#xA;                        mx += dl, my += dl, ml -= dl;&#xA;                    }&#xA;                } else if (op == 'I') {&#xA;                    sy += SEQ.substr(cy, cl), sx += Array(cl+1).join(&quot;-&quot;);&#xA;                    cy += cl, my += cl, ++k; // consume an insertion I to ref&#xA;                } else if (op == 'S') {&#xA;                    cy += cl, my += cl, ++k; // consume a soft-clipping S&#xA;                } else throw Error(&quot;inconsistent MD&quot;); // not working with N&#xA;            }&#xA;            if (ml != 0) throw Error(&quot;inconsistent MD&quot;);&#xA;        }&#xA;    }&#xA;    if (cx != mx || cy != my) throw Error(&quot;inconsistent MD&quot;);&#xA;    return [sx, sy];&#xA;}&#xA;&#xA;var cigar = '16025H3M2I55M1D108M306H';&#xA;var MD = '7T15C24T9^C0G2T6T16T3A1A7G21T17G6C1T6T5A4';&#xA;var seq = 'GATCACAGGCCTATCACCCTATTAATCACTCACGGGAGCTCTCCATGCATCTGGTATTTTTTCGGGGGGGGATGCACGCGATAGCATCGCGGGCCGCTGGAACCGGAGCACCCTATGTCGCAGGATCTGTCTTTGATTCCTACCTCATGCCATTATTAATCGCGCCTA';&#xA;var s = md2pad(cigar, MD, seq);&#xA;console.log(s[0]);&#xA;console.log(s[1]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-03-02T06:29:51.457" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3685" PostTypeId="5" CreationDate="2018-03-03T14:29:11.733" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T14:29:11.733" LastActivityDate="2018-03-03T14:29:11.733" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3686" PostTypeId="4" CreationDate="2018-03-03T14:29:11.733" Score="0" Body="Basic Local Alignment Search Tool: algorithm that finds regions of similarity between a pair of sequences, can be used to query an unknown sequence against a database of known sequences." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T07:59:37.007" LastActivityDate="2018-03-04T07:59:37.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3687" PostTypeId="5" CreationDate="2018-03-03T14:32:43.957" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T14:32:43.957" LastActivityDate="2018-03-03T14:32:43.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3688" PostTypeId="4" CreationDate="2018-03-03T14:32:43.957" Score="0" Body="Process of creating the original sequence from the read sequences that it generated during a sequencing experiment. Can refer to genome assembly, in which case the original sequence is a genome, or transcripts assembly, in which case the original sequences are RNA transcripts. " OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-03T14:50:07.147" LastActivityDate="2018-03-03T14:50:07.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3689" PostTypeId="5" CreationDate="2018-03-03T14:34:12.840" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T14:34:12.840" LastActivityDate="2018-03-03T14:34:12.840" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3690" PostTypeId="4" CreationDate="2018-03-03T14:34:12.840" Score="0" Body="The 3-D structure of a protein in space. It can refer to secondary structures, such as alpha helices or beta sheets, or the 3-D coordinates of the atoms making up a protein." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T07:58:56.813" LastActivityDate="2018-03-04T07:58:56.813" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3691" PostTypeId="5" CreationDate="2018-03-03T14:39:02.793" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T14:39:02.793" LastActivityDate="2018-03-03T14:39:02.793" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3692" PostTypeId="4" CreationDate="2018-03-03T14:39:02.793" Score="0" Body="An abnormal growth of cells that can threaten an individual's life. Cancerous cells are very proliferative and can spread to other parts of the body, creating metastases." OwnerUserId="1075" LastEditorUserId="77" LastEditDate="2018-03-04T08:00:39.623" LastActivityDate="2018-03-04T08:00:39.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3693" PostTypeId="5" CreationDate="2018-03-03T14:39:49.240" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T14:39:49.240" LastActivityDate="2018-03-03T14:39:49.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3694" PostTypeId="4" CreationDate="2018-03-03T14:39:49.240" Score="0" Body="The process of downloading data from the internet." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T07:58:37.953" LastActivityDate="2018-03-04T07:58:37.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3695" PostTypeId="5" CreationDate="2018-03-03T14:41:42.347" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T14:41:42.347" LastActivityDate="2018-03-03T14:41:42.347" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3696" PostTypeId="4" CreationDate="2018-03-03T14:41:42.347" Score="0" Body="The long-read genome assembler suitable for long reads having a relatively high error rate, such as PacBio and Nanopore data." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T07:50:26.570" LastActivityDate="2018-03-04T07:50:26.570" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3697" PostTypeId="5" CreationDate="2018-03-03T14:43:15.450" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T14:43:15.450" LastActivityDate="2018-03-03T14:43:15.450" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3698" PostTypeId="4" CreationDate="2018-03-03T14:43:15.450" Score="0" Body="The process of discovering variants in individuals starting from sequencing reads. Can be subdivided into germline variant calling and somatic variant calling. Can also be used for the more specific process of finding variants starting with reads aligned to a reference genome (SAM/BAM files)." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T07:59:00.947" LastActivityDate="2018-03-04T07:59:00.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3699" PostTypeId="5" CreationDate="2018-03-03T14:47:49.350" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T14:47:49.350" LastActivityDate="2018-03-03T14:47:49.350" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3700" PostTypeId="4" CreationDate="2018-03-03T14:47:49.350" Score="0" Body="When referring to the coverage of a base in a reference genome, the number of reads covering that base in an alignment. When referring to the coverage of a genome in a sequencing experiment, the average number of times that a base was sequenced. = total number of sequenced bases / genome size. For example, a 100x coverage means that each base was sequenced on average 100 times. For the human genome, that would be ~ 300 Gigabases." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T07:59:34.653" LastActivityDate="2018-03-04T07:59:34.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3701" PostTypeId="5" CreationDate="2018-03-03T15:54:51.967" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T15:54:51.967" LastActivityDate="2018-03-03T15:54:51.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3702" PostTypeId="4" CreationDate="2018-03-03T15:54:51.967" Score="0" Body="The software developed by Oxford Nanopore Technologies used to control the MinION sequencing device. MinKNOW is used for Flow Cell Platform QC and sequencing. MinKNOW can also be used for live basecalling." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T08:01:07.567" LastActivityDate="2018-03-04T08:01:07.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3703" PostTypeId="5" CreationDate="2018-03-03T15:57:18.790" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T15:57:18.790" LastActivityDate="2018-03-03T15:57:18.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3704" PostTypeId="4" CreationDate="2018-03-03T15:57:18.790" Score="0" Body="A sequence of 3 DNA or RNA bases coding for an amino acid or a stop signal. Codons are read in the ribosome to synthesise proteins using the tRNA, which are the link between each codon and the corresponding amino acid/stop signal. Codons can have different meanings depending on the organism." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T07:59:10.343" LastActivityDate="2018-03-04T07:59:10.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3705" PostTypeId="5" CreationDate="2018-03-03T15:59:06.307" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T15:59:06.307" LastActivityDate="2018-03-03T15:59:06.307" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3706" PostTypeId="4" CreationDate="2018-03-03T15:59:06.307" Score="0" Body="Reads Per Kilobase per Million. The number of reads that align to a specific gene, divided by the length of the gene in kilobases divided by the number of reads in million. It is the number of reads per gene normalised for the gene length and the library size." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T08:00:55.490" LastActivityDate="2018-03-04T08:00:55.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3707" PostTypeId="5" CreationDate="2018-03-03T16:01:13.703" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T16:01:13.703" LastActivityDate="2018-03-03T16:01:13.703" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3708" PostTypeId="4" CreationDate="2018-03-03T16:01:13.703" Score="0" Body="One of the three domains of life, with Archaea and eukaryotes. Bacteria, along with Archaea, are prokaryotes, i.e. lack a nucleus. " OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T07:59:19.387" LastActivityDate="2018-03-04T07:59:19.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3709" PostTypeId="5" CreationDate="2018-03-03T16:01:58.483" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T16:01:58.483" LastActivityDate="2018-03-03T16:01:58.483" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3710" PostTypeId="4" CreationDate="2018-03-03T16:01:58.483" Score="0" Body="The variant of BLAST (Basic Local Alignment Search Tool) used for protein sequence alignment." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T07:58:42.753" LastActivityDate="2018-03-04T07:58:42.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3711" PostTypeId="5" CreationDate="2018-03-03T16:03:21.873" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T16:03:21.873" LastActivityDate="2018-03-03T16:03:21.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3712" PostTypeId="4" CreationDate="2018-03-03T16:03:21.873" Score="0" Body="The phenomenon of having mitochondria or chloroplasts in a cell that have a different genome. For example, some mitochondria in a cell might have a deleterious mutation while the others are healthy." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T07:58:22.487" LastActivityDate="2018-03-04T07:58:22.487" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3713" PostTypeId="5" CreationDate="2018-03-03T16:43:05.297" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T16:43:05.297" LastActivityDate="2018-03-03T16:43:05.297" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3714" PostTypeId="4" CreationDate="2018-03-03T16:43:05.297" Score="0" Body="Quality Check or Quality Control. The assessment of the quality of a product, data. For example, in sequencing data, the assessment of the quality of the bases. QC can also include the pre-processing of data to make it attain minimum quality requirements." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T08:01:02.473" LastActivityDate="2018-03-04T08:01:02.473" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3715" PostTypeId="5" CreationDate="2018-03-03T16:45:13.400" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T16:45:13.400" LastActivityDate="2018-03-03T16:45:13.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3716" PostTypeId="4" CreationDate="2018-03-03T16:45:13.400" Score="0" Body="Each of several mRNAs encoded by different transcripts of the same gene. Isoforms can arise by different splicings of the same pre-mRNA. Each isoform gives rise to a different protein sequence." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T07:59:14.490" LastActivityDate="2018-03-04T07:59:14.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3717" PostTypeId="5" CreationDate="2018-03-03T16:46:37.500" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T16:46:37.500" LastActivityDate="2018-03-03T16:46:37.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3718" PostTypeId="4" CreationDate="2018-03-03T16:46:37.500" Score="0" Body="A discrete region of a protein with a defined structure. A domain usually has a specific function. A protein is usually made up of one or several domains. The same domain can occur in different proteins. " OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T07:58:33.857" LastActivityDate="2018-03-04T07:58:33.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3719" PostTypeId="5" CreationDate="2018-03-03T16:47:40.457" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T16:47:40.457" LastActivityDate="2018-03-03T16:47:40.457" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3720" PostTypeId="4" CreationDate="2018-03-03T16:47:40.457" Score="0" Body="A command-line tool used to remove adapter sequences from raw sequencing reads. Can also be used to trim low-quality bases from the ends of reads. " OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T08:01:11.140" LastActivityDate="2018-03-04T08:01:11.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3721" PostTypeId="5" CreationDate="2018-03-03T16:50:39.580" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T16:50:39.580" LastActivityDate="2018-03-03T16:50:39.580" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3722" PostTypeId="4" CreationDate="2018-03-03T16:50:39.580" Score="0" Body="A string present in each line of a SAM/BAM alignment file used to describe the similarity and differences between a read and the region it aligns to in the reference genome. The CIGAR string records the matching, mismatching and indels in the read compared to the reference. For example, a CIGAR string could encode a read that has 50 matches, followed by a deletion, followed by another 50 matching bases." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T07:59:28.457" LastActivityDate="2018-03-04T07:59:28.457" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3723" PostTypeId="5" CreationDate="2018-03-03T16:53:04.663" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-03T16:53:04.663" LastActivityDate="2018-03-03T16:53:04.663" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3724" PostTypeId="4" CreationDate="2018-03-03T16:53:04.663" Score="0" Body="The process of transforming the raw signal of a sequencer (light, images, electrical signal) into DNA or RNA bases. For Illumina, the basecalling consists of transforming the pictures with lights of different colours into bases. For Nanopore, it consists in transforming the electrical signal into bases. Basecalling can also include the calculation of the probability of the called base being correct (PHRED score). Albacore is a basecaller for Nanopore data." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T07:58:06.127" LastActivityDate="2018-03-04T07:58:06.127" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3725" PostTypeId="1" AcceptedAnswerId="3727" CreationDate="2018-03-03T19:48:46.877" Score="4" ViewCount="3943" Body="&lt;p&gt;I am sequencing a whole genome on MinION. I have used this flow cell for 6 hours for the phage lambda control. That gave me around 300,000 reads. I started a 48h sequencing run on the same flow cell after washing it. The sequencing has been going on for 1 day and 21 hours. It is not producing any reads at the moment (Strand = 0). The number of single pores is also 0. Can I &quot;Stop acquisition&quot; without negatively affecting the results?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The image below is a screenshot taken from MinKNOW, the software used to control the MinION.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/D5FuH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D5FuH.png&quot; alt=&quot;Pore description and read summary&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-10-26T20:08:33.717" Title="Can I stop my nanopore sequencing run if there are no more reads being produced?" Tags="|nanopore|ngs|minion|long-reads|minknow|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3726" PostTypeId="1" AcceptedAnswerId="3728" CreationDate="2018-03-04T00:43:01.420" Score="6" ViewCount="112" Body="&lt;p&gt;I believe that the human genome reference took DNA samples from different people and that some natural variation is included in extra contigs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the human reference genome comes from a limited number of people. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can the limited number of donors and the linear representation of the genome (as opposed to a graph representation that could potentially include variation) bias an analysis in favour of one ethnic group over another? Are there studies that have compared the alignment rates/mappabilities of reads of people coming from different geographic regions? Do some ethnic groups' reads align better to the reference?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-03-04T13:56:48.697" Title="Does the DNA or RNA of some ethnic groups map better than others' to the human reference genome sequence?" Tags="|sequence-alignment|genome|human-genome|mappability|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3727" PostTypeId="2" ParentId="3725" CreationDate="2018-03-04T03:00:14.297" Score="6" Body="&lt;p&gt;Yes, you can click &quot;stop acquisition&quot; and your run won't be negatively affected. All of the reads are saved as they are generated. I am not sure how this will impact live basecalling though if that is something that you do.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-03-04T03:00:14.297" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3728" PostTypeId="2" ParentId="3726" CreationDate="2018-03-04T08:12:35.767" Score="6" Body="&lt;p&gt;For simple variants like SNPs it would not really be a problem to use the current genome assembly for other ethnic groups. But for more complex variants this could be indeed problematic, however not only for ethnic groups but also for individuals within the same population. Think of very complex regions such as HLA or KIR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In studies where they compare different ethnic groups (e.g., 1000 G) they rather not discuss such matters, but act like the analysis is 100% perfect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a nice comment in &lt;a href=&quot;https://doi.org/10.1186/s13059-015-0587-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genome Biology&lt;/a&gt; about this issue.&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="298" LastEditDate="2018-03-04T13:56:48.697" LastActivityDate="2018-03-04T13:56:48.697" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3729" PostTypeId="5" CreationDate="2018-03-04T13:05:31.463" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-04T13:05:31.463" LastActivityDate="2018-03-04T13:05:31.463" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3730" PostTypeId="4" CreationDate="2018-03-04T13:05:31.463" Score="0" Body="When speaking of a sequencing read and a reference genome, the ability of an algorithm to determine the origin of the read in the genome. If the read can be unequivocally located in the reference genome, it has high mappability. If the read can be mapped to multiple loci, it has intermediate mappability. If the read cannot be mapped to any locus, it has low or no mappability." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T17:39:26.803" LastActivityDate="2018-03-04T17:39:26.803" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3731" PostTypeId="5" CreationDate="2018-03-04T13:06:46.147" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-04T13:06:46.147" LastActivityDate="2018-03-04T13:06:46.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3732" PostTypeId="4" CreationDate="2018-03-04T13:06:46.147" Score="0" Body="Mitochondrial DNA. Questions specific to mitochondrial DNA, such as annotation, inheritance, heteroplasmy." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T17:39:44.917" LastActivityDate="2018-03-04T17:39:44.917" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3733" PostTypeId="5" CreationDate="2018-03-04T13:10:00.053" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-04T13:10:00.053" LastActivityDate="2018-03-04T13:10:00.053" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3734" PostTypeId="4" CreationDate="2018-03-04T13:10:00.053" Score="0" Body="Short nucleotide sequence used by many sequencing methods in order to amplify DNA/RNA fragments, identify DNA/RNA fragments uniquely (Unique Molecular Identifier), anchor DNA/RNA fragments to the flow cell (Illumina). Adapters are useful during the sequencing process and sometimes during data analysis (e. g. indices for demultiplexing, UMIs for removing duplicates) but can be a nuisance and need to be removed prior to downstream analysis, e.g. alignment." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T17:39:21.230" LastActivityDate="2018-03-04T17:39:21.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3735" PostTypeId="5" CreationDate="2018-03-04T13:13:28.727" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-04T13:13:28.727" LastActivityDate="2018-03-04T13:13:28.727" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3736" PostTypeId="4" CreationDate="2018-03-04T13:13:28.727" Score="0" Body="A repository for data that is organised according to specific rules. Important biological databases include PDB (Protein Data Bank), Ensembl, Uniprot." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T17:39:31.240" LastActivityDate="2018-03-04T17:39:31.240" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3737" PostTypeId="5" CreationDate="2018-03-04T13:16:28.257" Score="0" Body="&lt;p&gt;The entire set of genetic information of a cell, tissue, organ, organism, etc. For eukaryotes, the genome includes the nuclear genome and the organellar genome. The genome is usually represented as a linear text or string in the FASTA format using the 4 nucleotide bases or the extended nucleotide IUPAC alphabet.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2018-05-08T08:45:53.173" LastActivityDate="2018-05-08T08:45:53.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3738" PostTypeId="4" CreationDate="2018-03-04T13:16:28.257" Score="0" Body="for questions related to whole genome analyses. For questions related to individual genes, use tag gene instead. Questions specific to format (like fasta) should be tagged with a format-specific tag." OwnerUserId="1075" LastEditorUserId="57" LastEditDate="2018-05-03T12:57:22.807" LastActivityDate="2018-05-03T12:57:22.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3739" PostTypeId="5" CreationDate="2018-03-04T13:17:43.343" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-04T13:17:43.343" LastActivityDate="2018-03-04T13:17:43.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3740" PostTypeId="4" CreationDate="2018-03-04T13:17:43.343" Score="0" Body="File format are sets of defined rules to organise a specific type of data. Common file format in bioinformatics include FASTA, FASTQ, SAM, BAM, VCF." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T17:39:52.627" LastActivityDate="2018-03-04T17:39:52.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3741" PostTypeId="5" CreationDate="2018-03-04T13:19:58.530" Score="0" Body="&lt;p&gt;Annotations may include the location of particular features (a gene on a segment of chromosomal DNA, or a conserved domain on a protein sequence), the function of the sequence or one of its features, or other descriptive contextual information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most common sequence annotations describe the location and function of protein-coding genes, tRNA genes, and rRNA genes on a genomic sequence. Tools such as Prokka and Maker were designed for this kind of genome annotation in prokaryotes and eukaryotes. Other features that may be annotated include the location of transposable and other repetitive elements, binding sites and other regulatory sequences, restriction sites, and many other features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Attaching functional information to an entire sequence or to one of its annotated features often involves adopting a functional description from a highly similar sequence whose function has been determined previously by experiment. Due to the rapid growth of sequence data in public databases, however, most functional annotation is several degrees removed from any sequence with direct experimental confirmation.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2019-07-12T19:50:22.390" LastActivityDate="2019-07-12T19:50:22.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3742" PostTypeId="4" CreationDate="2018-03-04T13:19:58.530" Score="0" Body="The process of attaching contextual information to a biomolecular (DNA, RNA, or protein) sequence, such as the location of genes, promoters, or repeats in genomic DNA, or the function of a protein." OwnerUserId="1075" LastEditorUserId="96" LastEditDate="2019-07-12T19:50:28.260" LastActivityDate="2019-07-12T19:50:28.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3744" PostTypeId="1" CreationDate="2018-03-04T19:25:09.647" Score="1" ViewCount="296" Body="&lt;p&gt;I'm attempting to query from BioMart in R using the biomaRt package with a script I've used many times before. I connect to the service with this line of code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mart &amp;lt;- useMart(biomart = &quot;ENSEMBL_MART_ENSEMBL&quot;, dataset = &quot;mmusculus_gene_ensembl&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I query from the service with this line of code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;getBM(attributes = c(&quot;ensembl_gene_id&quot;, &quot;mgi_symbol&quot;, &quot;gene_biotype&quot;, &quot;description&quot;), filters = &quot;ensembl_gene_id&quot;, values = function.df[, 1], mart = mart)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, when attempting to rerun the code this morning I was met with the following error message:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in getBM(attributes = c(&quot;ensembl_gene_id&quot;, &quot;mgi_symbol&quot;, &quot;gene_biotype&quot;,  : &#xA;  The query to the BioMart webservice returned an invalid result: biomaRt expected a character string of length 1. Please report this to the mailing list. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When trying to troubleshoot the problem, I discovered that useast.ensembl.org is down, which I believe is the mirror I connect to in the script. From &lt;a href=&quot;http://www.ensembl.org/info/data/biomart/biomart_r_package.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt; I read that you can manually edit the mirror you connect to with the host option in useMart(), so I edited my connection line like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mart &amp;lt;- useMart(biomart = &quot;ENSEMBL_MART_ENSEMBL&quot;, dataset = &quot;mmusculus_gene_ensembl&quot;, host = &quot;uswest.ensembl.org&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Looking at uswest.ensembl.org, it appears to be up and running, so I expected this fix to work, however, I get the same error. Am I doing something wrong with this fix so that biomaRt still connects to the useast mirror? Is there a way to get the connection to work currently, or am I forced to wait until useast comes back online?&lt;/p&gt;&#xA;" OwnerUserId="506" LastEditorUserId="48" LastEditDate="2018-03-05T22:10:22.530" LastActivityDate="2018-03-05T22:10:22.530" Title="Cannot connect to BioMart webservice in R script, how to troubleshoot connection?" Tags="|r|biomart|ensembl|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3745" PostTypeId="5" CreationDate="2018-03-04T21:21:13.370" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-04T21:21:13.370" LastActivityDate="2018-03-04T21:21:13.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3746" PostTypeId="4" CreationDate="2018-03-04T21:21:13.370" Score="0" Body="Questions asking whether a software exists for a certain purpose; questions asking which software is best among a selection for a certain purpose." OwnerUserId="1075" LastEditorUserId="73" LastEditDate="2018-03-05T02:53:27.547" LastActivityDate="2018-03-05T02:53:27.547" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3747" PostTypeId="5" CreationDate="2018-03-04T21:22:15.557" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-04T21:22:15.557" LastActivityDate="2018-03-04T21:22:15.557" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3748" PostTypeId="4" CreationDate="2018-03-04T21:22:15.557" Score="0" Body="Single Nucleotide Polymorphisms. How to find them? How to correlate them with other data? How to represent them? What is their distribution?" OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T21:36:31.897" LastActivityDate="2018-03-04T21:36:31.897" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3749" PostTypeId="5" CreationDate="2018-03-04T21:23:12.590" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-04T21:23:12.590" LastActivityDate="2018-03-04T21:23:12.590" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3750" PostTypeId="4" CreationDate="2018-03-04T21:23:12.590" Score="0" Body="Questions relating to the conversion of one format (e.g. FASTQ) into another (FASTA). " OwnerUserId="1075" LastEditorUserId="57" LastEditDate="2018-03-09T10:14:12.303" LastActivityDate="2018-03-09T10:14:12.303" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3751" PostTypeId="5" CreationDate="2018-03-04T21:24:17.873" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-04T21:24:17.873" LastActivityDate="2018-03-04T21:24:17.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3752" PostTypeId="4" CreationDate="2018-03-04T21:24:17.873" Score="0" Body="Reads are the sequences output by a sequencing machine after the raw signal (e.g. light, electricity) is converted into bases by a basecaller." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-05T01:07:59.280" LastActivityDate="2018-03-05T01:07:59.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3753" PostTypeId="5" CreationDate="2018-03-04T21:25:13.163" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-04T21:25:13.163" LastActivityDate="2018-03-04T21:25:13.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3754" PostTypeId="4" CreationDate="2018-03-04T21:25:13.163" Score="0" Body="The three-dimensional structure of RNA. How to visualise it, determine it from data, analyse its implications." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-04T21:35:34.077" LastActivityDate="2018-03-04T21:35:34.077" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3755" PostTypeId="5" CreationDate="2018-03-04T23:05:45.640" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-04T23:05:45.640" LastActivityDate="2018-03-04T23:05:45.640" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3756" PostTypeId="4" CreationDate="2018-03-04T23:05:45.640" Score="0" Body="The set of all exons of a genome. Exons, as opposed to introns, are the sequences in a gene that code for a protein." OwnerUserId="1075" LastEditorUserId="73" LastEditDate="2018-03-05T02:54:40.433" LastActivityDate="2018-03-05T02:54:40.433" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3757" PostTypeId="5" CreationDate="2018-03-04T23:09:07.863" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-04T23:09:07.863" LastActivityDate="2018-03-04T23:09:07.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3758" PostTypeId="4" CreationDate="2018-03-04T23:09:07.863" Score="0" Body="Use for questions involving the assignment of a taxonomic groups to each species, the ordering of species into categories of different hierarchical value, or the classification of organisms into species, genus, family, order, class and phylum." OwnerUserId="1075" LastEditorUserId="73" LastEditDate="2018-03-05T02:51:20.370" LastActivityDate="2018-03-05T02:51:20.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3759" PostTypeId="5" CreationDate="2018-03-04T23:10:18.817" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-04T23:10:18.817" LastActivityDate="2018-03-04T23:10:18.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3760" PostTypeId="4" CreationDate="2018-03-04T23:10:18.817" Score="0" Body="MATrix LABoratory. A scripting language similar that is popular in modelling and image analysis, as well as bioinformatics and other fields. The language is particularly well suited for common matrix and array operations." OwnerUserId="1075" LastEditorUserId="73" LastEditDate="2018-03-05T02:49:05.117" LastActivityDate="2018-03-05T02:49:05.117" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3761" PostTypeId="5" CreationDate="2018-03-04T23:12:40.817" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-04T23:12:40.817" LastActivityDate="2018-03-04T23:12:40.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3762" PostTypeId="4" CreationDate="2018-03-04T23:12:40.817" Score="0" Body="Principal Components Analysis. A statistical method used to reduce the dimensionality of a dataset while keeping as much variance in the first principal components as possible. It can be used to visualise samples with many variables in 2-D or 3-D, thus allowing for a visual non-supervised grouping of points." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-05T08:10:56.630" LastActivityDate="2018-03-05T08:10:56.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3763" PostTypeId="1" AcceptedAnswerId="3764" CreationDate="2018-03-04T23:34:46.027" Score="2" ViewCount="226" Body="&lt;p&gt;I am trying to look for motifs in a set of proteins returned by a mass spec experiment using MEME. The experiment was carried out in the yeast Candida albicans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know if it is worth calculating either a custom background model or a position specific prior using the non-hit proteins? I don't generally bother for nucleotide motif searching, but I've never done protein motif searching before. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also advice of whether to use the ZOOPS or ANR models appreciated: is there any reason not to use ANR other than run time?&lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-03-05T07:48:06.967" Title="Searching for protein motifs with MEME" Tags="|proteins|motifs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3764" PostTypeId="2" ParentId="3763" CreationDate="2018-03-05T06:23:59.613" Score="2" Body="&lt;p&gt;Some answers by the author Charles Grant via the MEME Suite Google group may be useful to you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Via: &lt;a href=&quot;https://groups.google.com/forum/#!topic/meme-suite/yNascbE8Tig/discussion&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://groups.google.com/forum/#!topic/meme-suite/yNascbE8Tig/discussion&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Providing your own higher-order background model can greatly improve MEME's ability to discover motifs...&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The command-line utility &lt;code&gt;fasta-get-markov&lt;/code&gt;, included in the MEME Suite download, is used to generate custom background Markov models. The input to &lt;code&gt;fasta-get-markov&lt;/code&gt; is a FASTA file containing &quot;background&quot; sequences. Ideally, these &quot;background&quot; sequences will be different from the sequences you are analyzing with MEME, but as similar in nature as possible. For example, if you wanted to discover motifs in certain intergenic regions, you might use other sequence data from other intergenic regions to generate the background. The larger the set of &quot;background&quot; sequences is, the better the results will be...&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Typically, you should not specify an order larger than 3 for DNA sequences, or larger than 2 for protein sequences. However, if your input sequences contain higher-order non-random effects that are getting in the way of motif finding, you can follow the following &quot;rules of thumb&quot;:&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Use a background model at least four orders less than the shortest motifs you are looking for. So, if you want to find motifs as short as six, I wouldn't use a model higher than order two. &lt;/li&gt;&#xA;  &lt;li&gt;For an accurate model of order N, you need to use a FASTA file as input to &lt;code&gt;fasta-get-markov&lt;/code&gt; with at least 10 times 4**(N+1) DNA characters in it. &lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;  &#xA;  &lt;p&gt;So,&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;order-3 requires 2560 characters &lt;/li&gt;&#xA;  &lt;li&gt;order 4 requires 10240 characters &lt;/li&gt;&#xA;  &lt;li&gt;order 5 requires 40960 characters etc.&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not certain if that input requirement scales differently for proteins (i.e., 20^(N+1)). That may be worth following up on the Google group.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And via: &lt;a href=&quot;https://groups.google.com/forum/#!topic/meme-suite/UiDhdY6WlK4/discussion&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://groups.google.com/forum/#!topic/meme-suite/UiDhdY6WlK4/discussion&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;You should choose the motif occurrence model based on your understanding of the experimental protocol that generated the sequence data and the underlying biology. MEME is trying to find short sequences that are statistically over-represented in your sequence data. To do this, it has to assume a model for how many occurrences of a motif there will be in each sequence. The nature of your experiment should be the basis for the model you choose. For example, data from a ChIP-Seq experiment will typically contain exactly one occurrence of a motif per sequence. On the other hand, sequences drawn from a database of upstream regions of co-regulated genes might contain any number of motif occurrences, including 0.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Via: &lt;a href=&quot;https://groups.google.com/d/topic/meme-suite/QJiJsy1QxYk/discussion&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://groups.google.com/d/topic/meme-suite/QJiJsy1QxYk/discussion&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The MEME algorithm run time is cubic with respect to the number of input sequences, therefore, it is unsuitable for OOPS (only one per sequence) analyses that have more than 1000 sequences. For ZOOPS (zero or one per sequence) and ANR (any number of repeats) analyses, having more than 1000 sites are also intractable in terms of run time. For example, on a 3 GHz CPU, initialization of the p-value table takes about 10 min for 1000 sites/sequences in OOPS, about a day for 5000 sites, and about 1 week for 10,000 sites. Then the quadratic run time with respect to the total number of characters kicks in next.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Thus it seems like the ZOOPS and ANR models have similar runtime complexity, and use of the MPI-based parallel &lt;code&gt;meme_p&lt;/code&gt; binary may be helpful, depending on your analysis:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The parallel version of MEME scales up to about 128 processors. Please see &lt;a href=&quot;http://www.sdsc.edu/~tbailey/papers/cabios96.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sdsc.edu/~tbailey/papers/cabios96.pdf&lt;/a&gt; for a discussion of the parallel program. You must bear in mind however, that doubling the total number of sequences input to MEME means that you will need 8 times more processors for the job to finish in the same amount of time.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-03-05T07:48:06.967" LastActivityDate="2018-03-05T07:48:06.967" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3766" PostTypeId="1" CreationDate="2018-03-05T14:27:58.683" Score="1" ViewCount="241" Body="&lt;p&gt;I'm using &lt;code&gt;adonis&lt;/code&gt;, an implementation of MANOVA in R (&lt;code&gt;vegan&lt;/code&gt; package) to understand how continuous (temperature, pH) and categorical (Site ID, Month) variables affect my 16S dataset. Applying it using my metadata 'as-is', I see that ~80% of the variation (R2) is significantly explained by one variable - Site ID.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, after suggestions from colleagues to normalize my data (as shown &lt;a href=&quot;http://vitalflux.com/data-science-scale-normalize-numeric-data-using-r&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for 'min-max' normalization), the same test not only shows Site ID as the third most important variable (in explaining the most variation significantly), it only shows ~31% variation being explained from the first three variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any opinions on best practices for this kind of analysis?&lt;/p&gt;&#xA;" OwnerUserId="445" LastEditorUserId="48" LastEditDate="2018-03-05T16:41:39.020" LastActivityDate="2018-03-05T16:41:39.020" Title="Normalize before using adonis?" Tags="|r|statistics|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3767" PostTypeId="5" CreationDate="2018-03-05T15:24:27.137" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-05T15:24:27.137" LastActivityDate="2018-03-05T15:24:27.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3768" PostTypeId="4" CreationDate="2018-03-05T15:24:27.137" Score="0" Body="A set of contigs, the order of which corresponds to consecutive genomic locations. Contigs can be scaffolded using a reference genome or optical mapping." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-05T17:06:14.887" LastActivityDate="2018-03-05T17:06:14.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3769" PostTypeId="5" CreationDate="2018-03-05T15:25:38.007" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-05T15:25:38.007" LastActivityDate="2018-03-05T15:25:38.007" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3770" PostTypeId="4" CreationDate="2018-03-05T15:25:38.007" Score="0" Body="Whole genome sequencing. Can refer to sequencing that is done on the entire genomic DNA, as opposed to techniques that only sequence a subset of the genome, like ChIP, RAD-seq or exome sequencing." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-05T17:06:08.850" LastActivityDate="2018-03-05T17:06:08.850" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3771" PostTypeId="5" CreationDate="2018-03-05T15:29:10.387" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-05T15:29:10.387" LastActivityDate="2018-03-05T15:29:10.387" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3772" PostTypeId="4" CreationDate="2018-03-05T15:29:10.387" Score="0" Body="External sequences that are added to DNA or RNA at one of several stages of the sequencing protocol in order to assess the functioning of the sequencing and accuracy of data analysis. Spike-ins include phage DNA or RNA ERCC sequences that have a known sequence and concentration." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-05T17:05:59.900" LastActivityDate="2018-03-05T17:05:59.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3773" PostTypeId="5" CreationDate="2018-03-05T15:51:23.630" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-05T15:51:23.630" LastActivityDate="2018-03-05T15:51:23.630" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3774" PostTypeId="4" CreationDate="2018-03-05T15:51:23.630" Score="0" Body="Nomenclature. Questions relating to conventions, definitions, word usage." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-05T17:05:55.770" LastActivityDate="2018-03-05T17:05:55.770" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3775" PostTypeId="5" CreationDate="2018-03-05T15:52:36.550" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-05T15:52:36.550" LastActivityDate="2018-03-05T15:52:36.550" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3776" PostTypeId="4" CreationDate="2018-03-05T15:52:36.550" Score="0" Body="High-throughput sequencing. " OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-05T17:06:03.340" LastActivityDate="2018-03-05T17:06:03.340" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3777" PostTypeId="5" CreationDate="2018-03-05T15:53:38.853" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-05T15:53:38.853" LastActivityDate="2018-03-05T15:53:38.853" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3778" PostTypeId="4" CreationDate="2018-03-05T15:53:38.853" Score="0" Body="Refers to genetic variation such as SNPs, indels, and structural variants within a population or between individuals." OwnerUserId="1075" LastEditorUserId="96" LastEditDate="2019-01-15T21:51:34.650" LastActivityDate="2019-01-15T21:51:34.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3779" PostTypeId="2" ParentId="3682" CreationDate="2018-03-05T17:40:09.607" Score="3" Body="&lt;p&gt;Resequencing your sample is not going to make it any less heterozygous. I am assuming that you are working with a diploid organism? Please update your question and specify the genome size and ploidy of your organism (the type of organism will help us update our answers, too).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer, yes, you can use Pilon + short reads to correct the PacBio assembly even with a highly heterozygous sample. To do so you should use the &lt;code&gt;--diploid&lt;/code&gt; option in Pilon to let it know that there will be heterozygous sites. The way that it corrects heterozygous SNPs and heterozygous indels is different, and it seems to perform best with SNPs and sometimes does not correctly identify heterozygous indels (&lt;a href=&quot;http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0112963&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pilon itself doesn't handle phasing, but just corrects with whatever reads are in the &lt;code&gt;bam&lt;/code&gt; file that it is provided with. The end result of providing a &lt;code&gt;bam&lt;/code&gt; with reads from both haplotypes is that the corrected sequence will be a mosaic of the dominant allele at the heterozygous site as was observed in the short read data. This may or may not select for some alleles based on sequencing coverage bias of Illumina sequencers.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-03-05T17:40:09.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3781" PostTypeId="1" CreationDate="2018-03-05T23:24:03.553" Score="5" ViewCount="68" Body="&lt;p&gt;I want to create my own pathway with a custom knockout function using &lt;a href=&quot;https://escher.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ESCHER&lt;/a&gt;. Consider the pathway knockout &lt;a href=&quot;https://escher.github.io/escher-demo/knockout/&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see by clicking the reaction the color changes and the text with the html/css id &lt;code&gt;knockout-status&lt;/code&gt; adds the reactions that was clicked and the growth rate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to work out how the &lt;code&gt;knockout.js&lt;/code&gt;  handles the clicks and changes the colors. so far I would say that this block - as commented -  handles the clicks&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-javascript prettyprint-override&quot;&gt;&lt;code&gt;// initialize event listeners&#xA;var sel = builder.selection;&#xA;sel.selectAll('.reaction,.reaction-label')&#xA;    .style('cursor', function(d) {&#xA;        if (knockable(d.bigg_id)) return 'pointer';&#xA;        else return null;&#xA;    })&#xA;    .on('click', function(d) {&#xA;        if (knockable(d.bigg_id)) {&#xA;            if (!(d.bigg_id in knockouts))&#xA;                knockouts[d.bigg_id] = true;&#xA;            model = knock_out_reaction(model, d.bigg_id);&#xA;            solve_and_display(model, knockouts);&#xA;        }&#xA;    });&#xA;// grey for reactions that cannot be knocked out&#xA;sel.selectAll('.reaction-label')&#xA;    .style('fill', function(d) {&#xA;        if (!knockable(d.bigg_id)) return '#888';&#xA;        else return null;&#xA;    });&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the &lt;code&gt;'.reaction,.reaction-label'&lt;/code&gt; finds all css class with the reactions. What I am missing is how to create my own maps with my own click listener? I checked the &lt;a href=&quot;https://escher.readthedocs.io/en/stable/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; but it handles more the creation of the maps, but I am also want to build the js knockout functionality. Could someone maybe show on a small example with 4-5 reactions how I could create my own maps. &lt;/p&gt;&#xA;" OwnerUserId="2026" LastEditorUserId="2719" LastEditDate="2018-05-02T07:01:40.257" LastActivityDate="2021-07-08T13:45:30.990" Title="How to create a pathway knockout using the JavaScript library ESCHER?" Tags="|pathway|" AnswerCount="0" CommentCount="1" ClosedDate="2021-07-08T13:45:47.687" ContentLicense="CC BY-SA 4.0" />
  <row Id="3782" PostTypeId="5" CreationDate="2018-03-06T10:57:11.663" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-06T10:57:11.663" LastActivityDate="2018-03-06T10:57:11.663" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3783" PostTypeId="4" CreationDate="2018-03-06T10:57:11.663" Score="0" Body="To cut from the end. Trimming is often done on raw Illumina reads to discard low-quality bases or adapter sequences. Questions related to trimming algorithms, when, how to trim, trimming software." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-06T13:37:53.740" LastActivityDate="2018-03-06T13:37:53.740" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3784" PostTypeId="5" CreationDate="2018-03-06T10:58:05.310" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-06T10:58:05.310" LastActivityDate="2018-03-06T10:58:05.310" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3785" PostTypeId="4" CreationDate="2018-03-06T10:58:05.310" Score="0" Body="The study of microbes, including bacteria, archaea and microscopic eukaryotes." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-06T13:37:45.723" LastActivityDate="2018-03-06T13:37:45.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3786" PostTypeId="5" CreationDate="2018-03-06T11:00:05.047" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-06T11:00:05.047" LastActivityDate="2018-03-06T11:00:05.047" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3787" PostTypeId="4" CreationDate="2018-03-06T11:00:05.047" Score="0" Body="This refers to parts of or whole transcripts (or occasionally genes) that are not translated into proteins." OwnerUserId="1075" LastEditorUserId="77" LastEditDate="2018-03-06T13:40:07.903" LastActivityDate="2018-03-06T13:40:07.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3788" PostTypeId="1" AcceptedAnswerId="3789" CreationDate="2018-03-06T11:01:48.380" Score="1" ViewCount="48" Body="&lt;p&gt;I am trying to install Rostlab's PredictProtein in a Virtual Machine as detailed here &lt;a href=&quot;https://rostlab.org/owiki/index.php/How_to_Run_The_PredictProtein_Machine_Image_with_VirtualBox&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rostlab.org/owiki/index.php/How_to_Run_The_PredictProtein_Machine_Image_with_VirtualBox&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;&lt;a href=&quot;http://rostlab.org/services/ppmi/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Download PredictProtein Machine Image&lt;/a&gt; to your local drive. You will need the &lt;a href=&quot;http://nebc.nerc.ac.uk/tools/bio-linux/bio-linux-6.0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bio-Linux&lt;/a&gt;-based 'VMWARE/VMDK' format version.  &#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Choose the Debian-based version in case you do not need a graphical&#xA;  user interface. This image is only half the size of the&#xA;  Bio-Linux-based image.&lt;/li&gt;&#xA;  &lt;/ul&gt;&lt;/li&gt;&#xA;  &lt;li&gt;Download a &lt;a href=&quot;http://rostlab.org/services/ppmi/&quot; rel=&quot;nofollow noreferrer&quot;&gt;database image&lt;/a&gt; to your local drive. We recommend the 'raw' format version.&lt;/li&gt;&#xA;  &lt;li&gt;Create New Virtual Machine: Machine -&gt; New&lt;/li&gt;&#xA;  &lt;li&gt;Name your virtual machine as 'PredictProtein'&lt;/li&gt;&#xA;  &lt;li&gt;Choose Linux and Debian as operation system&lt;/li&gt;&#xA;  &lt;li&gt;Locate and select the VMware .vmdk file as your boot disk&lt;/li&gt;&#xA;  &lt;li&gt;Give at least 2GB of RAM to your virtual machine&lt;/li&gt;&#xA;  &lt;li&gt;Click 'Finish'&lt;/li&gt;&#xA;  &lt;li&gt;Boot your new virtual machine.&lt;/li&gt;&#xA;  &lt;li&gt;If the message &quot;The disk drive for /mnt/local-storage is not ready yet or not present&quot; appears, press 's'.&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The problem comes at this step:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;11&quot;&gt;&#xA;  &lt;li&gt;&lt;p&gt;Open a terminal and enter:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sudo mkfs.ext4 -F -m 0 -L local-storage /dev/sdb&#xA;sudo mount -a&#xA;sudo mkdir /mnt/local-storage/rostlab-data &amp;amp;&amp;amp; cd /mnt/local-storage/rostlab-data&#xA;sudo mount -a&#xA;sudo mount /mnt/rostlab-data&#xA;sudo tar -xvJf /mnt/rostlab-data/rostlab-data.txz&#xA;sudo umount /mnt/rostlab-data &amp;amp;&amp;amp; cd&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;When I run &lt;code&gt;sudo mount /mnt/rostlab-data&lt;/code&gt;, I get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mount: special device LABEL=rostlab-data does not exist&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How should I proceed? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional notes - &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output for &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat /etc/fstab&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# /etc/fstab: static file system information.&#xA;#&#xA;# Use 'blkid -o value -s UUID' to print the universally unique identifier&#xA;# for a device; this may be used with UUID= as a more robust way to name&#xA;# devices that works even if disks are added and removed. See fstab(5).&#xA;#&#xA;# &amp;lt;file system&amp;gt; &amp;lt;mount point&amp;gt;   &amp;lt;type&amp;gt;  &amp;lt;options&amp;gt;       &amp;lt;dump&amp;gt;  &amp;lt;pass&amp;gt;&#xA;proc            /proc           proc    nodev,noexec,nosuid 0   0&#xA;# / was on /dev/sda1 during installation&#xA;UUID=9a9418ff-48f9-421a-b25a-82161839351d / ext4 errors=remount-ro 0 1&#xA;# swap was on /dev/sda5 during installation&#xA;UUID=a96757f4-8cc7-410f-95ee-b5ae642eb9ed none swap sw  0       0&#xA;&#xA;LABEL=local-storage /mnt/local-storage auto defaults,nofail 0   2&#xA;LABEL=rostlab-data /mnt/rostlab-data auto defaults,noauto,ro 0  0&#xA;&#xA;/mnt/local-storage/rostlab-data /usr/share/rostlab-data none bind,nofail 0 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output for &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blkid &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/dev/ramzswap0: TYPE=&quot;swap&quot;&#xA;/dev/sda1: UUID=&quot;9a9418ff-48f9-421a-b25a-82161839351d&quot; TYPE=&quot;ext4&quot;&#xA;/dev/sda5: UUID=&quot;a96757f4-8cc7-410f-95ee-b5ae642eb9ed&quot; TYPE=&quot;swap&quot;&#xA;/dev/sdb: LABEL=&quot;local-storage&quot; UUID=&quot;df841028-8dd1-4cd2-b1bb-6ae1f6887510&quot; TYPE=&quot;ext4&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2458" LastEditorUserId="298" LastEditDate="2018-03-06T12:54:30.073" LastActivityDate="2018-03-06T12:57:30.767" Title="Installing Rostlab's PredictProtein" Tags="|proteins|software-installation|linux|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3789" PostTypeId="2" ParentId="3788" CreationDate="2018-03-06T12:57:30.767" Score="0" Body="&lt;p&gt;It looks like the image you downloaded is missing the partition with the label &lt;code&gt;rostlab-data&lt;/code&gt;. Your &lt;code&gt;blkid&lt;/code&gt; output shows that you have the &lt;code&gt;local-storage&lt;/code&gt; partition, but not the &lt;code&gt;rostlab-data&lt;/code&gt; which is automatically added to your &lt;code&gt;/etc/fstab&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am afraid you'll have to contact the developers and let them know. Sounds like a bug.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-03-06T12:57:30.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3790" PostTypeId="2" ParentId="3213" CreationDate="2018-03-06T21:18:20.577" Score="2" Body="&lt;p&gt;From a quick reading of the paper it looks like they are predicting biological function of proteins. They frame this as a classification problem, where they try to classify a protein as pertaining to a set of terms (Abnormality of the lymph nodes, Abnormality of skin physiology, etc). It looks like they predict for each protein whether it falls into this category and create a ROC curve to plot the false positive and true positive rates of prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, the continuous variable would be your predictions (Abnormality of the lymph nodes: predicted 0.65 probability, Abnormality of skin physiology: predicted 0.25 probability). The categorical variable would then be the ground truth that you are trying to predict (Abnormality of the lymph nodes: known 1, Abnormality of skin physiology: known 0). Based on these two variables you can make your ROC curve.&lt;/p&gt;&#xA;" OwnerUserId="388" LastActivityDate="2018-03-06T21:18:20.577" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3791" PostTypeId="2" ParentId="288" CreationDate="2018-03-07T00:10:38.133" Score="1" Body="&lt;p&gt;Where transcription factor binding sites are concerned, in our papers we have used per-base &lt;a href=&quot;http://compgen.cshl.edu/phast/faq.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;phyloP&lt;/a&gt; conservation data instead of smoothed phastCons scores. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We use BEDOPS &lt;code&gt;bedmap&lt;/code&gt; to map scores over multiple, padded binding sites (BED-formatted) for a given motif model. The phyloP scores are WIG files obtained from UCSC goldenpath and converted to BED via &lt;code&gt;wig2bed&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resulting matrix of binding site intervals and their per-base scores can be turned into a ranked heatmap or aggregated to determine average per-base conservation for a motif model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While TF binding sites innately show high information content and therefore high conservation, matrices can be further sorted by score-maps of ChIP-seq or DNaseI-seq tag density. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This can help filter for interesting low-noise, high-signal patterns and is useful when the matrix is windowed and the window includes low-information regions with a few high-information residues outside of the main motif — as is the case with CTCF, for instance.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2018-03-07T00:10:38.133" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3792" PostTypeId="1" AcceptedAnswerId="3795" CreationDate="2018-03-07T03:57:10.430" Score="7" ViewCount="1536" Body="&lt;p&gt;I've been reading this paper lately:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/nbt.2862&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sailfish Enables Alignment-Free Isoform Quantification from RNA-Seq Reads Using Lightweight Algorithms&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't really understand the second paragraph under Quantification in the Online Methods section. I quote the author:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In an unstranded protocol, sequencing reads, and hence the k-mers they contain, may originate from transcripts in either the forward or reverse direction.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As far as I understand we are sequencing cDNA here and it is a single stranded molecule so all the reads will have the same direction (unlike the DNA where the reverse complement strand is also there), so I don't see the author's point. Can someone clarify please? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="122" LastEditorUserId="77" LastEditDate="2018-03-07T13:29:41.897" LastActivityDate="2018-04-10T09:27:42.143" Title="Stranded vs. unstranded library preparation protocols in RNAseq" Tags="|rna-seq|k-mer|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3793" PostTypeId="2" ParentId="3792" CreationDate="2018-03-07T07:27:38.693" Score="5" Body="&lt;p&gt;There are a number of library preparation methods out there. Some, like the popular dUTP-based methods, result in libraries where the orientation of a read's alignment confers information regarding the strand of the transcript from whence it came. Other protocols do more PCR cycles and don't use dUTP to allow distinguishing strands. These are then termed &quot;unstranded&quot; libraries and one can no longer determine the strand/direction of the originating transcript from the orientation of a read's alignment. In general, a transcript will then have reads pointing in both directions in more or less equal proportions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You tend to not see unstranded libraries much any more. Early on they were significantly more expensive, but that's not really the case any more. There also used to be two types of stranded protocols, whereas you mostly see dUTP-based protocols these days.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, you'll see these same settings in most RNAseq programs. The equivalent to an unstranded protocol setting in featureCounts is &lt;code&gt;-s 0&lt;/code&gt;, in tophat2 it was &lt;code&gt;--library-type fr-unstranded&lt;/code&gt; (no one should use tophat2 any more, it's just an example) and so on in other programs.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-03-07T07:27:38.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3794" PostTypeId="1" AcceptedAnswerId="3797" CreationDate="2018-03-07T09:34:50.020" Score="9" ViewCount="399" Body="&lt;p&gt;I have a Fastq file and I want to get the exact count of each possible kmer from this file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On a previous post called &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/562/48&quot;&gt;How to use khmer to count k-mers?&lt;/a&gt; Daniel Standage proposed a custom script based on khmer methods that you can see down below :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; outfile = open('outfile.txt', 'w') &#xA;&amp;gt;&amp;gt;&amp;gt; seenkmers = set()  # Consumes a lot of memory for large input!!!&#xA;&amp;gt;&amp;gt;&amp;gt; for read in khmer.ReadParser('reads.fq.gz'):&#xA;...     for kmer in counts.get_kmers(read.sequence):&#xA;...         if kmer not in seenkmers:&#xA;...             print(kmer, counttable.get(kmer), file=outfile)&#xA;...             seenkmers.add(kmer)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The code works if I switch &lt;code&gt;counttable.get(kmer)&lt;/code&gt; with &lt;code&gt;counts.get(kmer)&lt;/code&gt; but the main problem is that when I try on a customized fastq file with 300 occurrences of one kmer, the count will still be stuck at 255 and can't go further, and for some reason I can't pass through this limit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this coming from the minimum table size ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know if there is a parameter that can work on a custom script to disable this limit ? &lt;/p&gt;&#xA;" OwnerUserId="2461" LastEditorUserId="2461" LastEditDate="2018-03-09T08:40:06.703" LastActivityDate="2018-03-09T08:40:06.703" Title="How to get the count of each kmer past 255 using khmer" Tags="|python|fastq|k-mer|khmer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3795" PostTypeId="2" ParentId="3792" CreationDate="2018-03-07T10:21:36.013" Score="8" Body="&lt;p&gt;Short answer: cDNA is double-stranded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll try to explain the concepts surrounding this with a nanopore read, where the cDNA component of the read (from the mouse genome) is indicated in a light grey, and the adapter components are either coloured or dark grey:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/z7gA5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/z7gA5.png&quot; alt=&quot;klk10 transcript (corrected read) from a nanopore cDNA sequencing run&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cDNA library from this RNA was prepared by adding a polyT adapter (binding to the polyA sequence of the RNA), and carrying out reverse transcription to the start of the transcript. A strand-switching process was then used to prime and extend along the opposite strand of the DNA, creating a &lt;strong&gt;double-stranded&lt;/strong&gt; full-length cDNA template. This template was then amplified using PCR to increase cDNA concentration and add in barcoding sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may notice that this particular read has a polyT head, rather than a polyA tail, attached to the &lt;code&gt;VNP&lt;/code&gt; primer in the forward orientation. I can tell by this feature that this read comes from the reverse-complement strand of a DNA transcript, and so I know that the actual RNA transcript sequence is the reverse-complement of this read. Both the forward and reverse-complement sequences could potentially come through in the sequencer, because the adapters for sequencing are attached to both sides of the cDNA template, but the transcript direction can still be determined by looking at the orientation of the adapter sequences. When mapping this sequence to the genome, only a single strand (and its associated set of kmers) needs to be considered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When considering what happens after mapping to the genome, there are four possible match situations:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Adapter is in reverse orientation, genome is in reverse orientation&lt;/li&gt;&#xA;&lt;li&gt;Adapter is in forward orientation, genome is in reverse orientation&lt;/li&gt;&#xA;&lt;li&gt;Adapter is in reverse orientation, genome is in forward orientation&lt;/li&gt;&#xA;&lt;li&gt;Adapter is in forward orientation, genome is in forward orientation&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In the case of the library preparation above, situations 2 and 3 correspond to a transcript that is encoded on the forward strand of the genome, whereas situations 1 and 4 correspond to a reverse-encoded transcript. Different library preparation methods could have a different way of encoding transcript direction. See &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/3922/73&quot;&gt;here&lt;/a&gt; for more details on how these match situations can be used to produce strand-specific BAM files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I were to use an unstranded protocol (e.g. generating a cDNA library through other means, and attaching adapters randomly to the forward or reverse transcript strand), then the adapter direction wouldn't give me information about which strand this sequence came from. I would need to consider both strands when mapping (or more correctly, when identifying the native direction of the originating transcript), because the adapter sequence has been decoupled from the transcript direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On Illumina systems, this strand direction calculation is performed via the chemistry on the flow cell, and the adapters are [usually] not sequenced by the sequencer at all. Adapters from one direction bind to complementary sequences on the flow cell and are extended to produce the first read, and adapters from the other direction bind to produce the second read. Even with a stranded protocol, adapters from both directions are bound (because the cDNA template is double-stranded), but the binding order (as first or second read) can be used to determine the native direction of the originating transcript.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-04-10T09:27:42.143" LastActivityDate="2018-04-10T09:27:42.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3796" PostTypeId="2" ParentId="143" CreationDate="2018-03-07T10:47:55.103" Score="3" Body="&lt;p&gt;I don't think it will be possible to do what you ask, right now with current knowledge. Selecting variants relevant to cancer risk is still an open problem and usually requires quite a lot of human intervention.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use different measures of population frequency to filter common variants with the assumption that frequent variants won't be pathogenic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use multiple in-silico predictors of pathogenicity to try to remove &quot;benign&quot; or &quot;tolerated&quot; variants, but predictors are far from perfect and you have to take into account that many of them are essentially measuring the same, so when combining their outputs you may be biasing your analysis towards nucleotide conservation, etc...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use in-silico predictors of splicing effects. These are even farther from perfect and return very noisy data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can take into account where the variants lie in the gene. In many genes, variants in the last few nucleotides won't be pathogenic, but may be important in others. In some genes only variants in some domains have an effect, while not in others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have to take into account if only one or both alleles of certain genes are affected. For that you may need phasing information. And obviously the list of genes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some genes have an impact on cancer risk when they are &quot;deactivated&quot; (or broken somehow) (tumor supressors) while others have an effect when their function is improved (oncogenes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And for many many genes you'll simply not know if they have any effect in cancer risk at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can get data from different databases and annotate your genomes to remove many variants but in the end you won't have very high quality data to enter to your ML algorithms. Maybe this will work for your project, but take into account that this &quot;first step&quot; of data cleaning is not easy or obvious with current biological knowledge.&lt;/p&gt;&#xA;" OwnerUserId="936" LastActivityDate="2018-03-07T10:47:55.103" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3797" PostTypeId="2" ParentId="3794" CreationDate="2018-03-07T19:57:04.743" Score="6" Body="&lt;p&gt;255 is the default maximum size of a &lt;code&gt;Counttable&lt;/code&gt; in khmer. You want to do the following:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import khmer&#xA;counts = khmer.Counttable(31, 1e7, 1)  # Feel free to change the parameters&#xA;counts.set_use_bigcount(True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The last line increases the maximum value from 255 to 65535. I don't think there's any way to currently go above that without changing the khmer code (at some point you have to increase the size of the number used in the hash table).&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-03-07T21:16:08.987" LastActivityDate="2018-03-07T21:16:08.987" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3798" PostTypeId="2" ParentId="3130" CreationDate="2018-03-07T21:56:35.440" Score="1" Body="&lt;p&gt;There is a new idea of reporting the &quot;phased haplotype/genotype block&quot; with unique block ID.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/secastel/phaser/tree/master/phaser&quot; rel=&quot;nofollow noreferrer&quot;&gt;phaser&lt;/a&gt; uses &quot;PI&quot; (unique phase-index) and &quot;PG&quot; (phased-genotype). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;GATK is also using this idea of &quot;unique-block&quot; and &quot;phased-genotype&quot;; See &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/tooldocs/3.8-0/org_broadinstitute_gatk_tools_walkers_phasing_ReadBackedPhasing.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;the &quot;HP&quot; tag&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally, I feel the representation of &quot;phaser&quot; is way better and comprehensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would represent your data as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1 100 . A T . . . GT:PI:PG    1/0:3:1|0&#xA;chr1 200 . A T . . . GT:PI:PG    0/1:3:0|1&#xA;chr1 300 . A T . . . GT:PI:PG    1/0:4:1|0&#xA;chr1 400 . A T . . . GT:PI:PG    0/1:4:1|0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where first two lines have same &quot;PI&quot; (i.e 3) stating they are phased with each other. Same with the later two lines which have another &quot;PI&quot; (i.e 4).&lt;/p&gt;&#xA;" OwnerUserId="1099" LastEditorUserId="57" LastEditDate="2018-03-08T11:52:31.273" LastActivityDate="2018-03-08T11:52:31.273" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3800" PostTypeId="2" ParentId="333" CreationDate="2018-03-08T12:08:47.270" Score="1" Body="&lt;p&gt;As Iakov Davydov points out above mathematical calculation is very difficult. Particularly if you do not have uniform nucleotide, dinucleotide etc. frequencies. Thus I'd recommend using a method based on randomisation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'd want to randomise only within genes since the likelihood of getting an editing island is likely to be dependent on expression level of the RNA. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might like to look at what people analysing CLIP data do for these sorts of analyses. See for example &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/21048981&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wang et al&lt;/a&gt;. Or I have code for this sort of thing in my &lt;a href=&quot;https://github.com/sudlab/iCLIPlib&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;iCLIPlib&lt;/code&gt;&lt;/a&gt;. See the module &lt;a href=&quot;https://github.com/sudlab/iCLIPlib/blob/master/iCLIP/kmers.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;kmer.py&lt;/code&gt;&lt;/a&gt;. (WARNING: this code is still in development, and is not &quot;released&quot; yet, so I offer no promises about bugs or how easy it would be for your to get working). However, these approaches generally only &lt;em&gt;rank&lt;/em&gt; the significance of kmers rather than giving absolute significance values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://gat.readthedocs.io/en/latest/contents.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genome Annotation Tester (GAT)&lt;/a&gt; might be of use here. You would define genes as your workspace, islands as your annotations, and motif matches as your segments of interest. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-03-08T12:08:47.270" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3802" PostTypeId="5" CreationDate="2018-03-08T14:43:24.017" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-08T14:43:24.017" LastActivityDate="2018-03-08T14:43:24.017" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3803" PostTypeId="4" CreationDate="2018-03-08T14:43:24.017" Score="0" Body="Pacific Biosciences is a biotech company selling a third-generation sequencing technology often denoted by the same name, also known as Single-Molecule Real-Time (SMRT) sequencing. Questions related to data generated by PacBio/SMRT sequencing." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-08T19:36:39.870" LastActivityDate="2018-03-08T19:36:39.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3805" PostTypeId="1" AcceptedAnswerId="3806" CreationDate="2018-03-08T17:24:03.710" Score="1" ViewCount="110" Body="&lt;p&gt;I made some NEB ultra II whole-genome shotgun libraries with around ~200ng of template going into the indexing PCR step. The PCR was run for 13 cycles and I ended up having high library concentrations between 60-90 ng/µL. Should I be worried about complexity or PCR duplicates? How would all of you QC to make sure the library is complex enough and not overamplified too badly?&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-03-08T17:56:22.650" Title="How to QC overamplified shotgun library?" Tags="|ngs|illumina|genome-sequencing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3806" PostTypeId="2" ParentId="3805" CreationDate="2018-03-08T17:56:22.650" Score="1" Body="&lt;p&gt;FastQC (&lt;a href=&quot;https://www.bioinformatics.babraham.ac.uk/projects/fastqc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioinformatics.babraham.ac.uk/projects/fastqc/&lt;/a&gt;) is a good start: it will show you observed/expected ratios for the most overrepresented sequences, and running it on a few datasets you know to be high quality should give you intuition for what looks good and what doesn't.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My labmate also produced something called preseq (&lt;a href=&quot;http://smithlabresearch.org/software/preseq/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://smithlabresearch.org/software/preseq/&lt;/a&gt;) that uses the library complexity of an initial sequencing experiment to predict how many more unique reads you would get with additional sequencing of the same library. It could give you some intuition for whether or not it is worth sequencing your library deeper to get the information you need.&lt;/p&gt;&#xA;" OwnerUserId="643" LastActivityDate="2018-03-08T17:56:22.650" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3807" PostTypeId="5" CreationDate="2018-03-08T18:27:17.343" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-08T18:27:17.343" LastActivityDate="2018-03-08T18:27:17.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3808" PostTypeId="4" CreationDate="2018-03-08T18:27:17.343" Score="0" Body="Rapid Annotation Transfer Tool. Bioinformatics software to transfer the annotation from one strain or species to another, relying on similarity of sequences. " OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-08T21:53:13.597" LastActivityDate="2018-03-08T21:53:13.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3809" PostTypeId="5" CreationDate="2018-03-08T18:27:51.230" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-08T18:27:51.230" LastActivityDate="2018-03-08T18:27:51.230" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3810" PostTypeId="4" CreationDate="2018-03-08T18:27:51.230" Score="0" Body="Often repetitive sequences linking a pair of sister chromatids." OwnerUserId="-1" LastEditorUserId="77" LastEditDate="2018-03-08T22:04:04.737" LastActivityDate="2018-03-08T22:04:04.737" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3811" PostTypeId="5" CreationDate="2018-03-08T21:30:54.263" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-08T21:30:54.263" LastActivityDate="2018-03-08T21:30:54.263" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3812" PostTypeId="4" CreationDate="2018-03-08T21:30:54.263" Score="0" Body="Sequence units in the genome that have the ability to move in the genome, either through a copy-and-paste mechanism or a cut-and-paste mechanism. Transposable elements create interspersed repeats in genomes and can make up a significant proportion of an eukaryotic genome." OwnerUserId="1075" LastEditorUserId="1157" LastEditDate="2018-10-18T08:20:53.743" LastActivityDate="2018-10-18T08:20:53.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3813" PostTypeId="5" CreationDate="2018-03-08T22:38:25.923" Score="0" Body="&lt;p&gt;ggplot2 is an advanced and widely used graphics plotting package in R. Its beauty is the ability to continually stack features onto a basic graphic and multitude of options therein. It is arguably the most advanced code based graphics package, considering the clarity of the code and wide range of feature.&lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="3848" LastEditDate="2020-06-29T13:56:11.430" LastActivityDate="2020-06-29T13:56:11.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3814" PostTypeId="4" CreationDate="2018-03-08T22:38:25.923" Score="0" Body="Advanced R visualisation and plotting package based on the grammar of graphics. ggplot2 is also available in Python via 'plotnine' which uses ggplot2 code " OwnerUserId="1075" LastEditorUserId="3848" LastEditDate="2020-06-29T13:53:54.450" LastActivityDate="2020-06-29T13:53:54.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3815" PostTypeId="5" CreationDate="2018-03-08T22:39:12.143" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-08T22:39:12.143" LastActivityDate="2018-03-08T22:39:12.143" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3816" PostTypeId="4" CreationDate="2018-03-08T22:39:12.143" Score="0" Body="A python-based scripting language to visualise 3-D structures, especially of proteins." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-09T10:15:11.140" LastActivityDate="2018-03-09T10:15:11.140" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3817" PostTypeId="5" CreationDate="2018-03-08T23:40:05.687" Score="0" Body="&lt;p&gt;Resources maintained by the GATK development team at the Broad Institute:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Documentation website: &lt;a href=&quot;https://gatk.broadinstitute.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Support forum: &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/community/topics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/community/topics&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Codebase: &lt;a href=&quot;https://github.com/broadinstitute/gatk&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/broadinstitute/gatk&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2776" LastEditorUserId="2776" LastEditDate="2020-04-23T21:18:25.473" LastActivityDate="2020-04-23T21:18:25.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3818" PostTypeId="4" CreationDate="2018-03-08T23:40:05.687" Score="0" Body="Genome Analysis Toolkit. Popular open-source bioinformatics suite to call variants from second-generation sequencing data." OwnerUserId="1075" LastEditorUserId="2776" LastEditDate="2020-04-23T21:18:18.137" LastActivityDate="2020-04-23T21:18:18.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3819" PostTypeId="2" ParentId="3678" CreationDate="2018-03-09T00:10:52.220" Score="2" Body="&lt;p&gt;It sounds like you're looking for the &lt;code&gt;modifications.csv&lt;/code&gt; file, rather than the &lt;code&gt;modifications.gff&lt;/code&gt; file. The &lt;code&gt;.gff&lt;/code&gt; file is for viewing in &lt;code&gt;SMRT View&lt;/code&gt; (or other viewers). You can see schema for both files at PacBio's &lt;a href=&quot;https://github.com/PacificBiosciences/Bioinformatics-Training/wiki/Methylome-Analysis-Technical-Note&quot; rel=&quot;nofollow noreferrer&quot;&gt;Methylome Analysis Technical Note&lt;/a&gt;. Specifically, &lt;code&gt;modified_base&lt;/code&gt; means the algorithm could not determine the exact base and modification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also found a video tutorial for performing the analysis in SMRTLink on &lt;a href=&quot;https://youtu.be/BcpDXqcMiuA&quot; rel=&quot;nofollow noreferrer&quot;&gt;YouTube&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: Per PacBio's &lt;a href=&quot;https://github.com/PacificBiosciences/Bioinformatics-Training/wiki/Methylome-Analysis-Technical-Note&quot; rel=&quot;nofollow noreferrer&quot;&gt;Methylome Analysis Technical Note&lt;/a&gt; documentation, the &lt;code&gt;.csv&lt;/code&gt; &quot;...is also produced when motif analysis is not active, such as with the RS_Modification_Detection analysis protocol.&quot; Not sure why it's missing, perhaps because the algorithm couldn't clearly identify any of the modifications. You'll probably need to contact tech support on this issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In either case, the &lt;code&gt;.gff&lt;/code&gt; you posted includes the significant hits (p &amp;lt; 0.01), but the algorithm was unable to classify them, likely because of insufficient coverage. PacBio's &lt;a href=&quot;https://github.com/PacificBiosciences/Bioinformatics-Training/wiki/Methylome-Analysis-Technical-Note&quot; rel=&quot;nofollow noreferrer&quot;&gt;Methylome Analysis Technical Note&lt;/a&gt; documentation gives specific coverage recommendations for each event type, but 100x is the lower bound. Lots of good information there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Probably not what you wanted to hear, but I hope that gets you going. &lt;/p&gt;&#xA;" OwnerUserId="136" LastEditorUserId="136" LastEditDate="2018-03-12T16:02:54.507" LastActivityDate="2018-03-12T16:02:54.507" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3820" PostTypeId="5" CreationDate="2018-03-09T01:53:15.560" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-09T01:53:15.560" LastActivityDate="2018-03-09T01:53:15.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3821" PostTypeId="4" CreationDate="2018-03-09T01:53:15.560" Score="0" Body="ERCC (External RNA Controls Consortium) RNA spike-in mix: set of 92 RNA molecules of known sequence and concentration. They are added to RNA before sequencing in order to assess the quality of the analysis. They can be used to assess the dynamic range of the sequencing, the lower detection limit, the linearity between RNA concentration and RPKM and the accuracy of fold-changes between two conditions if two spike-in mixes are used." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-09T10:14:25.527" LastActivityDate="2018-03-09T10:14:25.527" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3822" PostTypeId="5" CreationDate="2018-03-09T01:54:01.857" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-09T01:54:01.857" LastActivityDate="2018-03-09T01:54:01.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3823" PostTypeId="4" CreationDate="2018-03-09T01:54:01.857" Score="0" Body="Long-read aligner suitable for pacbio data." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-09T10:14:29.520" LastActivityDate="2018-03-09T10:14:29.520" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3824" PostTypeId="5" CreationDate="2018-03-09T01:55:02.003" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-09T01:55:02.003" LastActivityDate="2018-03-09T01:55:02.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3825" PostTypeId="4" CreationDate="2018-03-09T01:55:02.003" Score="0" Body="Toolkit for processing sequence data in FASTA and FASTQ format." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-09T10:13:52.923" LastActivityDate="2018-03-09T10:13:52.923" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3826" PostTypeId="1" AcceptedAnswerId="3828" CreationDate="2018-03-09T03:23:16.983" Score="1" ViewCount="215" Body="&lt;p&gt;Hello I am trying to create frequency plot for a motif with frequency unit on the Y axis from 0 -100 nt as the unit as represented in this paper.(&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/29457796&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seiler et al.,&lt;/a&gt; 2018)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/h0Cq4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/h0Cq4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2478" LastActivityDate="2018-03-09T07:27:20.490" Title="Creating frequency plot from weblogo with frequency unit on the Y axis" Tags="|sequence-alignment|motifs|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3827" PostTypeId="1" CreationDate="2018-03-09T05:43:34.967" Score="3" ViewCount="52" Body="&lt;p&gt;I found that there is a Syntenic gene browser at &lt;a href=&quot;https://genomevolution.org/wiki/index.php/Syntenic_gene_sets&quot; rel=&quot;nofollow noreferrer&quot;&gt;GEvo&lt;/a&gt;. Do anyone know where could I find a similar browser for a local installation?&lt;/p&gt;&#xA;" OwnerUserId="2477" LastEditorUserId="48" LastEditDate="2018-03-09T08:05:48.873" LastActivityDate="2018-05-12T17:12:44.900" Title="Syntenic gene browser" Tags="|sequence-annotation|genome-browser|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3828" PostTypeId="2" ParentId="3826" CreationDate="2018-03-09T06:23:52.737" Score="4" Body="&lt;p&gt;Within weblogo, add the &lt;code&gt;--units probability&lt;/code&gt; argument and edit the figure with Illustrator (PDF/SVG) or Inkscape (SVG) to modify the y-axis, changing fractional probability (&lt;code&gt;[0,1]&lt;/code&gt;) to a percentage (&lt;code&gt;[0,100]&lt;/code&gt;) by multiplying by 100. Refer to the &lt;a href=&quot;http://weblogo.threeplusone.com/manual.html#CLI&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt; for more detail.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-03-09T07:27:20.490" LastActivityDate="2018-03-09T07:27:20.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3829" PostTypeId="1" AcceptedAnswerId="3832" CreationDate="2018-03-09T12:06:58.260" Score="3" ViewCount="152" Body="&lt;p&gt;Let's say I want to retrieve the yeast-specific identifier for a certain protein, in the example below it will be &lt;a href=&quot;http://www.uniprot.org/uniprot/P00330&quot; rel=&quot;nofollow noreferrer&quot;&gt;P00330&lt;/a&gt; which I would like to link to &lt;a href=&quot;https://www.yeastgenome.org/locus/S000005446&quot; rel=&quot;nofollow noreferrer&quot;&gt;YOL086C&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I go to &lt;a href=&quot;https://sparql.uniprot.org/sparql&quot; rel=&quot;nofollow noreferrer&quot;&gt;uniprot's sparql UI&lt;/a&gt; and type &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PREFIX up:&amp;lt;http://purl.uniprot.org/core/&amp;gt; &#xA;PREFIX rdfs:&amp;lt;http://www.w3.org/2000/01/rdf-schema#&amp;gt; &#xA;&#xA;SELECT *&#xA;WHERE&#xA;{&#xA;    Bind(&amp;lt;http://purl.uniprot.org/uniprot/P00330&amp;gt; as ?protein).&#xA;    ?protein rdfs:seeAlso ?db .&#xA;    ?db up:database &amp;lt;http://purl.uniprot.org/database/SGD&amp;gt; .&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;it returns&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;protein db&#xA;http://purl.uniprot.org/uniprot/P00330  http://purl.uniprot.org/sgd/S000005446&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How would I now go on and find the &lt;code&gt;YOL086C&lt;/code&gt;? When I add the line&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;?db ?p ?o&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to the query above, I don't see anything that would directly lead me to the identifier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried to access another database&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;?db2 up:database &amp;lt;http://purl.uniprot.org/database/EuPathDB&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but this returns all kind of databases while I am interested only in the &lt;code&gt;FungiDB&lt;/code&gt;. How could I select this one?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could I link the two IDs to each other?&lt;/p&gt;&#xA;" OwnerUserId="351" LastEditorUserId="351" LastEditDate="2018-03-10T19:53:19.803" LastActivityDate="2018-03-15T07:33:05.427" Title="How to retrieve the yeast locus tag from uniprot via sparql?" Tags="|uniprot|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3830" PostTypeId="1" AcceptedAnswerId="3831" CreationDate="2018-03-09T12:23:05.450" Score="4" ViewCount="286" Body="&lt;p&gt;I'm trying to make a Biopython script that reproduces a manual search on the NCBI website.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My manual search gives me the following URL:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore?term=ANOS1[gene%20name]%20AND%20refseq[filter]&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore?term=ANOS1[gene%20name]%20AND%20refseq[filter]&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the &quot;Search details&quot; box indicates:&lt;code&gt;ANOS1[gene name] AND refseq[filter]&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get 338 results that I &quot;Send to&quot; &quot;File&quot; as &quot;GI List&quot;, save as &lt;code&gt;ANOS1_orthologs_manual.txt&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How can I do the same using Biopython?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2018-03-09T13:33:49.830" LastActivityDate="2018-03-09T14:49:23.877" Title="How can I reproduce a manual NCBI search with Biopython Entrez module?" Tags="|python|public-databases|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3831" PostTypeId="2" ParentId="3830" CreationDate="2018-03-09T12:34:19.233" Score="6" Body="&lt;p&gt;This can be done by using the &quot;Search details&quot; as a search term in &lt;code&gt;Entrez.esearch&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;import os&#xA;OPJ = os.path.join&#xA;base_dir = os.getcwd()&#xA;&#xA;from Bio import Entrez&#xA;Entrez.email = &quot;my_email_here&quot;&#xA;query = &quot;ANOS1[gene name] AND refseq[filter]&quot;&#xA;handle = Entrez.esearch(term=query, db=&quot;nuccore&quot;, retmax=400)&#xA;ids = Entrez.read(handle)[&quot;IdList&quot;]&#xA;with open(OPJ(base_dir, &quot;ANOS1_orthologs_max400_ids.txt&quot;), &quot;w&quot;) as ids_file:&#xA;    ids_file.write(&quot;\n&quot;.join(ids))&#xA;    ids_file.write(&quot;\n&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2018-03-09T13:37:00.753" LastActivityDate="2018-03-09T13:37:00.753" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3832" PostTypeId="2" ParentId="3829" CreationDate="2018-03-09T12:59:37.797" Score="4" Body="&lt;p&gt;Ok, figured it out by looking at the &lt;a href=&quot;http://www.uniprot.org/format/uniprot_rdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;scheme&lt;/a&gt; uniprot provides:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PREFIX up:&amp;lt;http://purl.uniprot.org/core/&amp;gt; &#xA;&#xA;SELECT ?protein ?yeast_id&#xA;WHERE&#xA;{&#xA;    Bind(&amp;lt;http://purl.uniprot.org/uniprot/P00330&amp;gt; as ?protein).&#xA;    ?protein up:encodedBy ?gene .&#xA;    ?gene up:locusName ?yeast_id    &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So one starts at the accession number in the middle and then uses the path to the actual &lt;code&gt;locusName&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also the path via the database works fine, I just did not use the &lt;code&gt;seeAlso&lt;/code&gt; correctly (&lt;code&gt;protein&lt;/code&gt; was only bound to &lt;code&gt;db&lt;/code&gt; but not to &lt;code&gt;db2&lt;/code&gt;, that's why it returned all entries):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PREFIX up:&amp;lt;http://purl.uniprot.org/core/&amp;gt; &#xA;PREFIX rdfs:&amp;lt;http://www.w3.org/2000/01/rdf-schema#&amp;gt; &#xA;&#xA;SELECT ?protein ?db ?db2&#xA;WHERE&#xA;{&#xA;    Bind(&amp;lt;http://purl.uniprot.org/uniprot/P00330&amp;gt; as ?protein).&#xA;    ?protein rdfs:seeAlso ?db .&#xA;    ?protein rdfs:seeAlso ?db2 .&#xA;    ?db up:database &amp;lt;http://purl.uniprot.org/database/SGD&amp;gt; .&#xA;    ?db2 up:database &amp;lt;http://purl.uniprot.org/database/EuPathDB&amp;gt;&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will return the desired output (one would still have to process the &lt;code&gt;db2&lt;/code&gt; entry, of course):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;protein db  db2&#xA;http://purl.uniprot.org/uniprot/P00330  http://purl.uniprot.org/sgd/S000005446  http://purl.uniprot.org/eupathdb/FungiDB:YOL086C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="351" LastEditorUserId="351" LastEditDate="2018-03-15T07:33:05.427" LastActivityDate="2018-03-15T07:33:05.427" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3833" PostTypeId="1" CreationDate="2018-03-09T13:03:46.343" Score="3" ViewCount="47" Body="&lt;p&gt;Errors happen frequently in the lab and I got one sample that we mixed one important information (the region of the sample) between two samples. Now we don't know which sample is from which region, but we know all the other variables (Date of the sample, patient, and other variables).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, my solution has been to make that values as &lt;code&gt;NA&lt;/code&gt; to be able to analyze it, but I would like to know if there is any solution. If I can convey to test or other analysis downstream that sample A and sample B are from Location1 or Location2 and if A is from Location1, B is from Location2 and otherwise round.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several packages for imputing quantitative data, but I couldn't find any to store this kind of information or to impute qualitative data. I can recall a recent paper with single-cell sequencing that estimated phenotype and some other variables but I can't find it neither I think it is possible with RNAseq.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although maybe I could build a classifier to actually do that (but that would be too long and I'm not sure if it is worth it)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;My samples are from more than two locations.&lt;/li&gt;&#xA;&lt;li&gt;My samples consist of 16S microbiome DNA-seq and RNA-seq&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="48" LastEditorUserId="492" LastEditDate="2018-03-10T18:37:09.267" LastActivityDate="2018-08-15T11:05:24.777" Title="Impute phenotype under some constraints" Tags="|data-management|imputation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3835" PostTypeId="1" AcceptedAnswerId="3902" CreationDate="2018-03-09T18:03:10.157" Score="2" ViewCount="701" Body="&lt;p&gt;I have pacbio Sequel data at 50 x coverage for a strain of animal. I would like to find structural variants compared to the reference genome sequence. At the moment, I align my reads to the reference genome using &lt;a href=&quot;https://github.com/philres/ngmlr&quot; rel=&quot;nofollow noreferrer&quot;&gt;NGM-LR&lt;/a&gt; and call the variants using &lt;a href=&quot;https://github.com/fritzsedlazeck/Sniffles&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sniffles&lt;/a&gt;. The resulting VCF file contains variants that are well supported in a visual inspection in IGV as well as some more questionable variants (especially larger events). I have used the reads corrected by &lt;a href=&quot;http://canu.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;canu&lt;/a&gt; for alignment. I think it splits my long reads into very long reads and not-so-long reads and uses the shorter reads to correct the longer reads. However, in IGV, my reads still have a considerable number of mistakes compared to the reference. I was wondering if it is OK to use &lt;a href=&quot;https://github.com/broadinstitute/pilon&quot; rel=&quot;nofollow noreferrer&quot;&gt;pilon&lt;/a&gt; to polish my long reads with some Illumina data from the same sample? The main purpose of pilon is to correct assemblies but I was wondering if it can be used to correct subreads from PacBio sequencing?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="73" LastEditDate="2018-03-14T21:20:17.777" LastActivityDate="2018-05-13T22:14:33.227" Title="Can I pilon-polish long reads with Illumina short reads to improve structural variant detection?" Tags="|variant-calling|long-reads|structural-variation|pacbio|read-correction|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3836" PostTypeId="5" CreationDate="2018-03-09T18:05:37.490" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-09T18:05:37.490" LastActivityDate="2018-03-09T18:05:37.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3837" PostTypeId="4" CreationDate="2018-03-09T18:05:37.490" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-09T18:05:37.490" LastActivityDate="2018-03-09T18:05:37.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3838" PostTypeId="1" AcceptedAnswerId="3841" CreationDate="2018-03-09T18:09:08.083" Score="7" ViewCount="3089" Body="&lt;p&gt;Is there a standard method in Python to extract a CIGAR string from the BAM?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are great libraries which parse the CIGAR, e.g. &lt;a href=&quot;https://pypi.python.org/pypi/cigar/0.1&quot; rel=&quot;noreferrer&quot;&gt;https://pypi.python.org/pypi/cigar/0.1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; c = Cigar('10M20S10M')&#xA;&amp;gt;&amp;gt;&amp;gt; c.mask_left(10).cigar&#xA;'10S20S10M'&#xA;&amp;gt;&amp;gt;&amp;gt; c.mask_left(9).cigar&#xA;'9S1M20S10M'&#xA;&amp;gt;&amp;gt;&amp;gt; Cigar('10S').mask_left(10).cigar&#xA;'10S'&#xA;&amp;gt;&amp;gt;&amp;gt; Cigar('10H').mask_left(10).cigar&#xA;'10H'&#xA;&amp;gt;&amp;gt;&amp;gt; Cigar('10H').mask_left(11).cigar&#xA;'10H'&#xA;&amp;gt;&amp;gt;&amp;gt; Cigar('10H').mask_left(9).cigar&#xA;'10H'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It looks like pysam already parses the cigar string, if I'm not mistaken:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pysam&#xA;&#xA;bam = 'myfile.bam'&#xA;bamfile = pysam.AlignmentFile(bam, 'rb')&#xA;&#xA;for read in bamfile:&#xA;    if not read.is_unmapped:&#xA;        cigar = read.cigar&#xA;        print(cigar)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This outputs lists of tuples of the parse output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[(5, 61), (0, 30), (5, 198)]&#xA;[(4, 11), (0, 30), (4, 248)]&#xA;[(4, 11), (0, 30), (4, 248)]&#xA;....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How could I simply output the CIGAR from &lt;code&gt;read&lt;/code&gt;? I would prefer to create a column in a pandas DataFrame&lt;/p&gt;&#xA;" OwnerUserId="146" LastActivityDate="2021-03-04T15:05:14.110" Title="Extracting the CIGAR string from a BAM via Python?" Tags="|python|sam|pysam|cigar|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3839" PostTypeId="5" CreationDate="2018-03-09T18:47:58.953" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-09T18:47:58.953" LastActivityDate="2018-03-09T18:47:58.953" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3840" PostTypeId="4" CreationDate="2018-03-09T18:47:58.953" Score="0" Body="Questions relating to long reads, such as pacbio or nanopore reads. " OwnerUserId="1075" LastEditorUserId="298" LastEditDate="2018-03-10T23:25:43.080" LastActivityDate="2018-03-10T23:25:43.080" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3841" PostTypeId="2" ParentId="3838" CreationDate="2018-03-09T19:20:20.573" Score="4" Body="&lt;p&gt;If you really do just want the cigar string then it's &lt;code&gt;read.cigarstring&lt;/code&gt;. However, I'm not sure what you're trying to gain with the cigar package from Brent. Unless you want to get the string with the masking changed then the tuple you get from pysam is the same as what you get from cigar (with the exception of the numeric operations instead of character operations).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-03-09T19:20:20.573" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3842" PostTypeId="5" CreationDate="2018-03-09T22:22:03.110" Score="0" Body="&lt;p&gt;Poricine simply means relating to pigs. The pig genome was deciphered to reveal 18 autosomes, with X and Y sex chromosomes and is about the same size as the human genome. Pigs have resulted in several established cell lines. For example, porcine sus (PS) cell lines are used for flavvivirus isolation and maintenance, albeit are dropping out of favour because of variation in glycosylation patterns between mammalian cell lines. African swine fever is an emergent infection with a devasting impact on the global pig population and is unusual because it is a large DNA virus. Most emergent viruses are RNA viruses.&lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="3848" LastEditDate="2020-07-06T19:23:43.153" LastActivityDate="2020-07-06T19:23:43.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3843" PostTypeId="4" CreationDate="2018-03-09T22:22:03.110" Score="0" Body="Relating to pigs, albeit  Sus scrofa domesticus is the scientific name of the domestic pig." OwnerUserId="-1" LastEditorUserId="3848" LastEditDate="2020-07-06T19:23:43.153" LastActivityDate="2020-07-06T19:23:43.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3844" PostTypeId="1" AcceptedAnswerId="3847" CreationDate="2018-03-09T23:11:04.390" Score="2" ViewCount="425" Body="&lt;p&gt;I am using the tool &lt;a href=&quot;http://emboss.sourceforge.net/apps/release/6.6/emboss/apps/seqret.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqret&lt;/a&gt; from emboss to transform an annotation file in GFF3 format and a FASTA file into an EMBL file because Wormbase does not supply an EMBL file with annotation and sequence together. I am running it on Mac OS X. I got the gff3 and fasta files from &lt;a href=&quot;ftp://ftp.wormbase.org/pub/wormbase/releases/current-production-release/species/c_elegans/PRJNA13758/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wormbase&lt;/a&gt;. I installed emboss using conda from the bioconda channel. I use the following command:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;seqret -sequence c_elegans.PRJNA13758.WS263.genomic.fa -feature -fformat gff -fopenfile c_elegans.PRJNA13758.WS263.annotations.gff3 -osformat embl -auto -outseq c_elegans.PRJNA13758.WS263.embl&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It fails with the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/Users/me/anaconda/bin/seqret`: line 7: 54774 Killed: 9               _seqret &quot;$@&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;/Users/me/anaconda/bin/seqret&lt;/code&gt; is a short file that indeed contains &lt;code&gt;_seqret &quot;$@&quot;&lt;/code&gt; on line 7:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/sh&#xA;BIN_DIR=$(dirname $(which seqret))&#xA;export EMBOSS_ACDROOT=$BIN_DIR/../share/EMBOSS/acd/&#xA;export EMBOSS_DATA=$BIN_DIR/../share/EMBOSS/data/&#xA;export PLPLOT_LIB=$BIN_DIR/../share/EMBOSS/&#xA;_seqret &quot;$@&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried to erase it but that did not help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried &lt;a href=&quot;https://github.com/NBISweden/EMBLmyGFF3&quot; rel=&quot;nofollow noreferrer&quot;&gt;EMBLmyGFF3&lt;/a&gt; in a Python 2 environment:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;EMBLmyGFF3 c_elegans.PRJNA13758.WS263.annotations.gff3 c_elegans.PRJNA13758.WS263.genomic.fa --topology linear --molecule_type 'genomic DNA' --transl_table 1 --species 'Caenorhabditis elegans' --locus_tag CELE --project_id PRJNA13758 -o c_elegans.PRJNA13758.WS263.annotations.embl&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I just had this message on display for hours:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#############################################################################&#xA;# NBIS 2016 - Sweden                                                        #&#xA;# Authors: Martin Norling, Niclas Jareborg, Jacques Dainat                  #&#xA;# Please visit https://github.com/NBISweden/EMBLmyGFF3 for more information #&#xA;#############################################################################&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Update 11th March:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I started EMBLmyGFF3 yesterday at 5 pm and this morning it started outputting messages such as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;08:22:08 ERROR feature: &amp;gt;&amp;gt;inverted_repeat&amp;lt;&amp;lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.&#xA;Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.&#xA;08:22:08 ERROR feature: &amp;gt;&amp;gt;assembly_component&amp;lt;&amp;lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.&#xA;Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.&#xA;08:22:08 WARNING feature: Unknown qualifier 'genbank' - skipped&#xA;08:22:08 ERROR feature: &amp;gt;&amp;gt;tandem_duplication&amp;lt;&amp;lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.&#xA;Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.&#xA;08:22:08 WARNING feature: Unknown qualifier 'public_name' - skipped&#xA;08:22:08 WARNING feature: Unknown qualifier 'polymorphism' - skipped&#xA;08:22:08 WARNING feature: Unknown qualifier 'other_name' - skipped&#xA;08:22:08 WARNING feature: Unknown qualifier 'consequence' - skipped&#xA;08:22:08 WARNING feature: Unknown qualifier 'variation' - skipped&#xA;08:22:08 ERROR feature: &amp;gt;&amp;gt;biological_region&amp;lt;&amp;lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.&#xA;Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.&#xA;08:22:08 WARNING feature: Unknown qualifier 'balancer_type' - skipped&#xA;08:22:08 WARNING feature: Unknown qualifier 'balancer' - skipped&#xA;08:22:08 ERROR feature: &amp;gt;&amp;gt;protein_match&amp;lt;&amp;lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.&#xA;Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.&#xA;08:22:08 ERROR feature: &amp;gt;&amp;gt;transcript_region&amp;lt;&amp;lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.&#xA;Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.&#xA;08:22:08 ERROR feature: &amp;gt;&amp;gt;transcribed_fragment&amp;lt;&amp;lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.&#xA;08:45:36 ERROR feature: &amp;gt;&amp;gt;piRNA&amp;lt;&amp;lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.&#xA;Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.&#xA;08:46:00 ERROR feature: &amp;gt;&amp;gt;pseudogenic_tRNA&amp;lt;&amp;lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.&#xA;Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.&#xA;08:47:59 ERROR feature: &amp;gt;&amp;gt;antisense_RNA&amp;lt;&amp;lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.&#xA;Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.&#xA;08:47:59 ERROR feature: &amp;gt;&amp;gt;lincRNA&amp;lt;&amp;lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.&#xA;Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.&#xA;08:48:03 ERROR feature: &amp;gt;&amp;gt;pre_miRNA&amp;lt;&amp;lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.&#xA;Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.&#xA;08:48:03 ERROR feature: &amp;gt;&amp;gt;miRNA&amp;lt;&amp;lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.&#xA;Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.&#xA;08:50:37 ERROR feature: &amp;gt;&amp;gt;snRNA&amp;lt;&amp;lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.&#xA;Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.&#xA;09:04:00 ERROR feature: &amp;gt;&amp;gt;miRNA_primary_transcript&amp;lt;&amp;lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For some of these features, it seems OK to not report them. But for others, such as piRNA, I need to find a solution. &lt;em&gt;C. elegans&lt;/em&gt; has got 1000s of piRNAs, it would be nice to have them in the EMBL file. I need to find a list of valid EMBL feature types.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update + ~ 24h computation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.&#xA;12:21:19 ERROR feature: &amp;gt;&amp;gt;scRNA&amp;lt;&amp;lt; is not a valid EMBL feature type. You can ignore this message if you don't need it.&#xA;Otherwise tell me which EMBL feature it corresponds to by adding the information within the json mapping file.&#xA;Conversion done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My EMBL file is only 1.19 GB compared to the GFF3 3.34 GB, so I probably lost a lot of annotation that is not supported by EMBL.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-12T01:08:04.397" LastActivityDate="2018-03-12T01:08:04.397" Title="seqret error when trying to transform GFF3 and FASTA files into EMBL format" Tags="|file-formats|format-conversion|sequence-annotation|emboss|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3845" PostTypeId="5" CreationDate="2018-03-09T23:13:22.363" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-09T23:13:22.363" LastActivityDate="2018-03-09T23:13:22.363" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3846" PostTypeId="4" CreationDate="2018-03-09T23:13:22.363" Score="0" Body="Use this tag for questions about the EMBOSS suite of tools. Please also mention your operating system when asking about such tools." OwnerUserId="1075" LastEditorUserId="298" LastEditDate="2018-03-10T23:24:23.887" LastActivityDate="2018-03-10T23:24:23.887" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3847" PostTypeId="2" ParentId="3844" CreationDate="2018-03-10T00:48:59.633" Score="2" Body="&lt;p&gt;That error is telling you that the script you were running was killed by signal &lt;code&gt;9&lt;/code&gt;, also known as &lt;code&gt;SIGKILL&lt;/code&gt;, while executing the command &lt;code&gt;_seqret &quot;$@&quot;&lt;/code&gt;. This is the way the operating system will kill processes that are asking for more memory than it can provide. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that removing &lt;code&gt;_secret &quot;$@&quot;&lt;/code&gt; is not the solution since that's what's actually doing the job. &lt;code&gt;$@&lt;/code&gt; is a bash array that contains all the arguments passed to a bash script. The &lt;code&gt;/Users/me/anaconda/bin/seqret&lt;/code&gt; file is just a simple script that sets up a few variables and then launches &lt;code&gt;_seqret&lt;/code&gt;, passing it the arguments you gave to &lt;code&gt;/Users/me/anaconda/bin/seqret&lt;/code&gt; as &lt;code&gt;$@&lt;/code&gt;. So deleting that lines just makes the script useless.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on your question, I am guessing you're running macOS on a laptop, so it isn't unreasonable that something that needs to read an entire genome and what looks like a full set of annotations will run out of memory on a laptop. Of course, this is a pretty small genome, but the annotations are still 396MB compressed, so depending on your machine's specs and what kind of internal processing the command is doing, it isn't too surprising that it fails. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, either run this on a more powerful machine, or try splitting the annotations file into smaller files and running it sequentially on each of those.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-03-10T14:17:43.210" LastActivityDate="2018-03-10T14:17:43.210" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3848" PostTypeId="2" ParentId="3844" CreationDate="2018-03-10T16:29:53.673" Score="3" Body="&lt;p&gt;I'm one of the EMBLmyGFF3 developers. If the compressed annotation is 396MB as mentioned by @terdon it is not a small annotation... In the worse case we have tested we never waited more than few minutes. We haven't done any particular speed test. It would be valuable if you could tell us how long it has taken in your case (If you didn't kill the process before the end...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At least we could try to implement some nice features, like to see that the tool is working , or/and see which percentage of the file has been already processed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is really slow we could think to try to improve its speed.&lt;/p&gt;&#xA;" OwnerUserId="2486" LastActivityDate="2018-03-10T16:29:53.673" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3849" PostTypeId="5" CreationDate="2018-03-10T17:08:27.227" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-10T17:08:27.227" LastActivityDate="2018-03-10T17:08:27.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3850" PostTypeId="4" CreationDate="2018-03-10T17:08:27.227" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-10T17:08:27.227" LastActivityDate="2018-03-10T17:08:27.227" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3851" PostTypeId="2" ParentId="3833" CreationDate="2018-03-10T17:38:38.430" Score="2" Body="&lt;p&gt;&lt;strong&gt;Question 1: Is there a tool for imputing phenotypes while also using genetic data?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yes, try Andy Dahl's &lt;a href=&quot;http://mathgen.stats.ox.ac.uk/genetics_software/phenix/phenix.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Phenix&lt;/a&gt; from the Marchini lab.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclosure: I did my PhD in the Marchini lab, although I was not directly involved in this research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 2: Is there a way to impute categorical data using methods intended for continuous data?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the variable of interest has only two categories, then you should be able to simply recode the categories as arbitrary (differing) numbers and get sensible results. But do make sure you understand the tool you are using to see if this will work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more than two categories, if your imputation approach allows the simultaneous imputation of multiple variables, you could try recoding your categories as &lt;a href=&quot;https://en.wikipedia.org/wiki/Contrast_(statistics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;contrasts&lt;/a&gt; or &lt;a href=&quot;http://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OneHotEncoder.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;one-hot&lt;/a&gt;.  After recoding, you could impute the variables resulting from recoding together.&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="492" LastEditDate="2018-08-15T11:05:24.777" LastActivityDate="2018-08-15T11:05:24.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3852" PostTypeId="5" CreationDate="2018-03-10T19:54:11.960" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-10T19:54:11.960" LastActivityDate="2018-03-10T19:54:11.960" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3853" PostTypeId="4" CreationDate="2018-03-10T19:54:11.960" Score="0" Body="Questions relating to the structure (3-D conformation) of biological molecules, such as protein, RNA and DNA." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-10T23:25:15.690" LastActivityDate="2018-03-10T23:25:15.690" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3854" PostTypeId="1" CreationDate="2018-03-10T20:32:59.913" Score="7" ViewCount="1017" Body="&lt;p&gt;I'm trying to understand the way the GSVA analysis is working behind the scenes.And I was wondering if there is any way to understand it more intuitively the whole process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So at first according to &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-14-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; it &lt;em&gt;starts by evaluating whether a gene i is highly or lowly expressed in sample j in the context of the sample population distribution.&lt;/em&gt; They use these kernel estimations of the cumulative density functions to transform the initial values so not to be affected by the problematic intensities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After this &quot;transformation&quot; and a following normalization, GSVA calculates the enrichment scores &lt;em&gt;using the Kolmogorov-Smirnov (KS) like random walk statistic&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I know, Kolmogorov-Smirnov checks for differences in distributions. Which distributions does it check? Gene-set's against all the others genes? And what is the role of the random walk?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So is there any intuitive way to understand this kind of Kolmogorov-Smirnov (KS) like random walk statistic? How does it actually work? Which one is the null and which the alternative hypothesis in that case? &lt;/p&gt;&#xA;" OwnerUserId="704" LastEditorUserId="48" LastEditDate="2018-03-11T09:56:48.597" LastActivityDate="2019-01-08T20:46:18.663" Title="Intuitive explanation of GSVA analysis" Tags="|statistics|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3855" PostTypeId="5" CreationDate="2018-03-11T01:27:03.683" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-11T01:27:03.683" LastActivityDate="2018-03-11T01:27:03.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3856" PostTypeId="4" CreationDate="2018-03-11T01:27:03.683" Score="0" Body="Bioinformatics tool from the emboss suite used to RETurn SEQuences (hence seqret). I can be used for file conversions between FASTA and FASTQ as well as between different annotation format, and more." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-11T18:17:51.500" LastActivityDate="2018-03-11T18:17:51.500" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3857" PostTypeId="5" CreationDate="2018-03-11T01:29:05.220" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-11T01:29:05.220" LastActivityDate="2018-03-11T01:29:05.220" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3858" PostTypeId="4" CreationDate="2018-03-11T01:29:05.220" Score="0" Body="Referring to the soma, i.e. non-germline cells in the body. The somatic genome is not inherited. Somatic mutations can lead to cancer. Somatic variant calling differs from germline variant calling because one often expects a mixture of mutated cells and non-mutated cells." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-11T18:27:07.163" LastActivityDate="2018-03-11T18:27:07.163" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3859" PostTypeId="1" CreationDate="2018-03-11T11:41:38.640" Score="5" ViewCount="170" Body="&lt;p&gt;For a dataset of several hundred individuals, I applied a hierarchical clustering to generate clusters based on a functional trait that sets them apart.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My task is now to evaluate if these clusters can be supported by the nucleotide  sequence data of the corresponding gene, i.e. is there more genetic similarity within each cluster than between the clusters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a first approach, I created a multiple sequence alignment for every cluster and calculated the the % identical sites value. There are more identical sites within each cluster than in an alignment of all sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know any good tools (preferably python) that can perform a more sophisticated evaluation of existing clusters based on sequence data?&#xA;Are there other scoring methods with which I can evaluate the sequence similarity within the clusters (again preferably python)?&lt;/p&gt;&#xA;" OwnerUserId="2489" LastActivityDate="2018-07-13T19:46:01.440" Title="Evaluate clusters of individuals by using their sequence data" Tags="|python|sequence-alignment|clustering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3860" PostTypeId="5" CreationDate="2018-03-11T17:16:01.287" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-11T17:16:01.287" LastActivityDate="2018-03-11T17:16:01.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3861" PostTypeId="4" CreationDate="2018-03-11T17:16:01.287" Score="0" Body="A framework for doing bioinformatics in a GUI, on the cloud. Free." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-11T18:27:12.617" LastActivityDate="2018-03-11T18:27:12.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3862" PostTypeId="5" CreationDate="2018-03-11T23:57:13.687" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-11T23:57:13.687" LastActivityDate="2018-03-11T23:57:13.687" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3863" PostTypeId="4" CreationDate="2018-03-11T23:57:13.687" Score="0" Body="Integrated Genome Viewer. GUI intended for the visualisation of genomic and transcriptomic data, along with annotations and other things." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-13T08:14:00.087" LastActivityDate="2018-03-13T08:14:00.087" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3864" PostTypeId="1" CreationDate="2018-03-12T06:02:21.227" Score="8" ViewCount="452" Body="&lt;p&gt;I have a large collection of bam files and I want to post-process each of them into another bam where I can make queries about:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the reads position and pair-endness, &lt;/li&gt;&#xA;&lt;li&gt;insert sizes, &lt;/li&gt;&#xA;&lt;li&gt;MAPQ and other flags, etc. of the reads &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;but where I don't need to make use of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the sequence string of the reads, &lt;/li&gt;&#xA;&lt;li&gt;the Q-scores qualities or,&lt;/li&gt;&#xA;&lt;li&gt;the read ids, apart from keeping pair-endness in the read id, &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So I would like to know what's a good tool/parameters to process each bam and turn them into a small size bam that still retains the information needed above. CRAM format is not an option at this point as several of the tools I use are not CRAM-compatible yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="131" LastEditDate="2018-03-15T09:42:00.043" LastActivityDate="2018-03-15T09:42:00.043" Title="Convert bam file to highly compressible bam" Tags="|sam|samtools|format-conversion|sambamba|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3865" PostTypeId="2" ParentId="3864" CreationDate="2018-03-12T08:00:42.943" Score="12" Body="&lt;p&gt;You can just set the fields you don't need to &lt;code&gt;*&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h foo.bam | awk 'BEGIN{FS=&quot;\t&quot;; OFS=&quot;\t&quot;}{if($1~/^@/) {print $0} else {print &quot;*&quot;, $2, $3, $4, $5, $6, $7, $8, $9, &quot;*&quot;, &quot;*&quot;}}' | samtools view -bo smaller.bam -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will set each read's name to &lt;code&gt;*&lt;/code&gt;, but you can still see where its mate maps with the PNEXT and RNEXT fields.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resulting BAM file looks like: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;*   83  chr1    3003308 37  51M =   3003157 -202    *   *&#xA;*   99  chr1    3003364 33  51M =   3003547 234 *   *&#xA;*   99  chr1    3003410 37  50M =   3003636 269 *   *&#xA;*   99  chr1    3003438 255 50M =   3003760 373 *   *&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-03-12T08:00:42.943" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3866" PostTypeId="2" ParentId="3864" CreationDate="2018-03-12T10:21:45.327" Score="3" Body="&lt;p&gt;I wrote  &lt;a href=&quot;http://lindenb.github.io/jvarkit/Biostar173114.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://lindenb.github.io/jvarkit/Biostar173114.html&lt;/a&gt; for biostars &lt;a href=&quot;https://www.biostars.org/p/173114/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;What is the best way to make a bam file smaller by removing unwanted information?&quot;&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Usage: biostar173114 [options] Files&#xA;&#xA;    -keepAtt, --keepAttributes&#xA;      keep Attributes&#xA;      Default: false&#xA;    -keepCigar, --keepCigar&#xA;      keep cigar : don't remove hard clip&#xA;      Default: false&#xA;    -keepName, --keepName, --name&#xA;      keep Read Name, do not try to create a shorter name&#xA;      Default: false&#xA;    -keepQuals, --keepQualities&#xA;      keep base qualities&#xA;      Default: false&#xA;    -keepRG, --keepReadGroup&#xA;      if attributes are removed, keep the RG&#xA;      Default: false&#xA;    -keepSeq, --keepSequence&#xA;      keep read sequence&#xA;      Default: false&#xA;    -mate, --mate&#xA;      keep Mate/Paired Information&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;e.g:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ java -jar dist/biostar173114.jar src/test/resources/S1.bam --keepSequence | head -n 20&#xA;@HD VN:1.5  SO:coordinate&#xA;(...)&#xA;@SQ SN:RF11 LN:666&#xA;@RG ID:S1   SM:S1   LB:L1   CN:Nantes&#xA;R0  128 RF01    1   60  70M *   0   0   GGCTATTAAAGCTATACAATGGGGCCGTATAATCTAATCTTGTCAGAATATTTATCATTTATATATAACT  *&#xA;R1  64  RF01    8   60  70M *   0   0   AAAGCTATACAATGGGGAAGTATAATCTAATCTTGTCAGAATATTTATCATTTATATATAACTCACAATG  *&#xA;R2  64  RF01    11  60  70M *   0   0   GCTATACAATGGGGAAGTATAATCTAATCTTGTCAGAATATTTATCATTTATATATAACTCACAATCCGC  *&#xA;R3  64  RF01    12  60  70M *   0   0   CTATACAATGGGGAAGTATAATCTAATCTTGTCAGAATATTTATCATTTATATATAACTCACAATCCGCA  *&#xA;R4  128 RF01    27  60  70M *   0   0   GTATCATCTAATCTTGTCATAATATTTATCATATATATATAACTCACAATCCGCAGTTCAAATTCCAATA  *&#xA;R5  64  RF01    44  60  70M *   0   0   CAGAATATTTATCATTTATATATAACTCAGAATCCGCAGTTCAAATTCCAATATACTATTCTTCCAATAG  *&#xA;R6  128 RF01    44  60  70M *   0   0   CAGAATATTTATCATTTATATATAACTCACAATCCGCAGTTCAAATTCCAATATACTATTCTTCCAATAG  *&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastActivityDate="2018-03-12T10:21:45.327" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3867" PostTypeId="1" AcceptedAnswerId="3868" CreationDate="2018-03-12T13:51:53.923" Score="6" ViewCount="333" Body="&lt;p&gt;I quantified some samples using &lt;a href=&quot;http://salmon.readthedocs.io/en/latest/salmon.html&quot; rel=&quot;noreferrer&quot;&gt;Salmon&lt;/a&gt;. According to the documentation of &lt;a href=&quot;http://salmon.readthedocs.io/en/latest/file_formats.html#fileformats&quot; rel=&quot;noreferrer&quot;&gt;output format&lt;/a&gt;, the last column of the 'Quantification File' represents the number of reads that supported the given transcript, namely:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;NumReads — This is salmon’s estimate of the number of reads mapping to each transcript that was quantified. It is an “estimate” insofar as it is the expected number of reads that have originated from each transcript given the structure of the uniquely mapping and multi-mapping reads and the relative abundance estimates for each transcript.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In some case, I had in the output some transcripts that had &lt;code&gt;NumReads=0.000000&lt;/code&gt;. Does this mean that I should discard them from the list of transcripts obtained in that sample? How comes that those transcripts end up in the output file in first place, since they don't have any reads support and the quantification was done using the reads themselves?&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2018-03-12T14:06:41.533" Title="Salmon output: transcripts quantified with zero reads support" Tags="|rna-seq|quantification|salmon|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3868" PostTypeId="2" ParentId="3867" CreationDate="2018-03-12T14:06:41.533" Score="4" Body="&lt;p&gt;All transcripts should always be returned in the output, whether there's evidence supporting their expression or not. This is for the sake of simple convenience, since typically one would be merging these values across samples into a matrix. Missing values will then tend to become either &lt;code&gt;NA&lt;/code&gt; or cause an error, when what would be appropriate would be to have values of 0. This also facilitates calculations such as &quot;what's the median number of reads per transcript?&quot; or &quot;what percent of transcripts have an expected reads count of at least X&quot;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-03-12T14:06:41.533" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3870" PostTypeId="2" ParentId="3605" CreationDate="2018-03-12T16:49:12.607" Score="3" Body="&lt;p&gt;I found a page explaining where this is set up in a &lt;code&gt;user_conf&lt;/code&gt; file here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://community.nanoporetech.com/support/faq/test1/minknow/minknow/how-do-i-change-the-directory-for-the-reads-folder?search_term=user_conf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://community.nanoporetech.com/support/faq/test1/minknow/minknow/how-do-i-change-the-directory-for-the-reads-folder?search_term=user_conf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Windows&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Open Notepad with administrative rights (search for notepad, right click and then select run as administrator). Select open file and navigate to C:\ProgramFiles\OxfordNanopore\MinKNOW\conf and then open user_conf to view the script. Under output_dirs change the two instances of &quot;C:\data&quot; (default read location) to the directory required. Please note the &quot;\&quot; is important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mac OS&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Go to /Applications/MinKNOW.app/Contents/Resources/conf/user_conf and open user_conf to view the script. Under output_dirs replace the default path &quot;/Library/MinKNOW/data&quot; with your path of choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ubuntu&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Go to /opt/ONT/MinKNOW/conf and open user_conf to view the script. Under output_dirs replace the default path &quot;/var/lib/MinKNOW/data/reads&quot; with your path of choice.&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2018-03-12T16:49:12.607" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3871" PostTypeId="1" CreationDate="2018-03-12T17:10:06.540" Score="0" ViewCount="165" Body="&lt;p&gt;When I try to open a simple bam file with the SeqAn library using the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CharString bamFileInName = getAbsolutePath(&quot;/home/omair/TFG/Files/9827_2#49.bam&quot;)&#xA;BamFileIn bamFileIn(toCString(bamFileInName));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;terminate called after throwing an instance of 'seqan::FileOpenError'&#xA;  what():  Could not open file /home/omair/Programs/seqan//home/omair/TFG/Files/9827_2#49.bam: iostream error&#xA;&#xA;Process finished with exit code 134 (interrupted by signal 6: SIGABRT)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The file exists and the path is correct.&lt;/p&gt;&#xA;" OwnerUserId="2472" LastEditorUserId="29" LastEditDate="2018-04-12T16:33:53.073" LastActivityDate="2018-04-12T16:33:53.073" Title="While opening a Bam file with the SeqAn library I get 'seqan::FileOpenError'" Tags="|sam|reads|c++|" AnswerCount="1" CommentCount="3" ClosedDate="2018-03-15T12:25:01.140" ContentLicense="CC BY-SA 3.0" />
  <row Id="3872" PostTypeId="5" CreationDate="2018-03-12T19:15:27.647" Score="0" Body="&lt;p&gt;Tool documentation:&lt;br&gt;&#xA;&lt;a href=&quot;https://broadinstitute.github.io/picard/command-line-overview.html#MarkDuplicates&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://broadinstitute.github.io/picard/command-line-overview.html#MarkDuplicates&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related tool:&lt;br&gt;&#xA;&lt;a href=&quot;https://broadinstitute.github.io/picard/command-line-overview.html#MarkDuplicatesWithMateCigar&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://broadinstitute.github.io/picard/command-line-overview.html#MarkDuplicatesWithMateCigar&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2776" LastEditorUserId="2776" LastEditDate="2020-04-24T18:30:39.410" LastActivityDate="2020-04-24T18:30:39.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3873" PostTypeId="4" CreationDate="2018-03-12T19:15:27.647" Score="0" Body="A tool from the Picard toolkit to mark or remove optical or PCR duplicates in sequencing data." OwnerUserId="1075" LastEditorUserId="2776" LastEditDate="2020-04-24T18:30:29.623" LastActivityDate="2020-04-24T18:30:29.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3874" PostTypeId="5" CreationDate="2018-03-12T19:16:19.870" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-12T19:16:19.870" LastActivityDate="2018-03-12T19:16:19.870" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3875" PostTypeId="4" CreationDate="2018-03-12T19:16:19.870" Score="0" Body="The removal of bases at the end of reads to make an alignment better." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-13T08:13:37.653" LastActivityDate="2018-03-13T08:13:37.653" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3876" PostTypeId="5" CreationDate="2018-03-12T19:17:02.903" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-12T19:17:02.903" LastActivityDate="2018-03-12T19:17:02.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3877" PostTypeId="4" CreationDate="2018-03-12T19:17:02.903" Score="0" Body="Determination of the genotype (for example SNP, allele, heterozygote mutant, etc.) of a sample." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-13T08:13:56.260" LastActivityDate="2018-03-13T08:13:56.260" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3878" PostTypeId="1" CreationDate="2018-03-12T22:40:16.527" Score="-3" ViewCount="56" Body="&lt;p&gt;I have a big df like following (pseudo) df and I want to retrieve values in different col. according to the rows in the first col.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A B C D&#xA;a 1 1 1&#xA;a 2 3 1 &#xA;a 3 3 3 &#xA;a 4 5 6 &#xA;b 5 6 5 &#xA;b 7 6 5 &#xA;c 7 6 5 &#xA;c 5 4 3 &#xA;c 5 5 4 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to retrieve the values in col. B, C, and D according to the rows in col A and perform further analysis of them. For example, multiply the values in col B and C for all the rows of &quot;a&quot; in col A. Can someone suggest a commend for such a naive question. Thanks. &lt;/p&gt;&#xA;" OwnerUserId="1513" LastEditorUserId="150" LastEditDate="2018-03-12T23:07:00.833" LastActivityDate="2018-03-12T23:45:17.340" Title="simple data subsetting on r" Tags="|r|subset|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3879" PostTypeId="2" ParentId="3878" CreationDate="2018-03-12T22:59:28.873" Score="2" Body="&lt;p&gt;Here's an approach that uses native R, using the vectorized (fast) family of &lt;code&gt;apply()&lt;/code&gt; functions. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; df2 &amp;lt;- df[df$A == 'a', which(names(df) %in% c(&quot;B&quot;,&quot;C&quot;))]&#xA;&amp;gt; apply(df2, 1, prod)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Use apply/lapply/mapply/etc. where you can in place of alternatives, where you don't know their run characteristics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a more concrete example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; df&#xA;  A B C D&#xA;1 a 1 1 1&#xA;2 a 2 3 1&#xA;3 a 3 3 3&#xA;4 a 4 5 6&#xA;5 b 5 6 5&#xA;...&#xA;&amp;gt; df2 &amp;lt;- df[df$A == 'a', which(names(df) %in% c(&quot;B&quot;,&quot;C&quot;))]&#xA;&amp;gt; df2&#xA;  B C&#xA;1 1 1&#xA;2 2 3&#xA;3 3 3&#xA;4 4 5&#xA;&amp;gt; apply(df2, 1, prod)&#xA; 1  2  3  4 &#xA; 1  6  9 20&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: If you need to do this on each unique value of &lt;code&gt;df$A&lt;/code&gt;, you can nest &lt;code&gt;apply&lt;/code&gt; calls to make this scale:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; perA.list &amp;lt;- apply(data.frame(unique(df$A)), 1, function(x) apply(df[df$A == x, which(names(df) %in% c(&quot;B&quot;,&quot;C&quot;))], 1, prod))&#xA;&amp;gt; perA.list&#xA;[[1]]&#xA; 1  2  3  4 &#xA; 1  6  9 20&#xA;&#xA;[[2]]&#xA; 5  6&#xA;30 35&#xA;&#xA;[[3]]&#xA; 7  8  9&#xA;35 20 25&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Each index in this list correspondents to the index of the value from &lt;code&gt;unique(df$A)&lt;/code&gt;, i.e. &lt;code&gt;a&lt;/code&gt;, &lt;code&gt;b&lt;/code&gt;, and &lt;code&gt;c&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't want to do this for all unique values in &lt;code&gt;df$A&lt;/code&gt;, but for some specific element in &lt;code&gt;df$A&lt;/code&gt;, then you would just need to write a function. 95% of the code is in this answer to write such a function, so I'll leave that as a homework exercise for you to implement.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-03-12T23:45:17.340" LastActivityDate="2018-03-12T23:45:17.340" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3880" PostTypeId="1" AcceptedAnswerId="3882" CreationDate="2018-03-12T23:54:40.600" Score="-1" ViewCount="996" Body="&lt;p&gt;I have been attempting to map reads to a reference genome using HISAT2 using the Pertea, et al 2016 Nature Protocols paper. I am able to successfully generate an index (using the hisat2-build command), but when it comes to the actual mapping, I run into problems. The code I'm using (from the paper):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$hisat2 -p 8 --dta -x daphnia_tran -1 C2*R1*.fastq -2 C2*R2*.fastq -S C2paired.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I get the error message:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./HISAT2_Basic_14.sh: line 26: -p: command not found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I remove the -p argument, the error message is similar, but says &quot;--dta: command not found&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas on what the issue is?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="2497" LastActivityDate="2018-03-13T07:59:38.627" Title="Command not found error on HISAT2" Tags="|sequence-alignment|transcriptome|genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3881" PostTypeId="1" AcceptedAnswerId="3890" CreationDate="2018-03-13T01:18:02.283" Score="4" ViewCount="1696" Body="&lt;p&gt;I'm trying to play around with my family's raw genetic data from Ancestry.com so that I can do some genetic analyses, and I noticed in the AncestryDNA raw data files that some regions of the genome have more than one variant associated with them and I would like to know why this is. The end goal is get this into a regularly formatted VCF, but I feel that these double variants will probably mess things up. So for example one might see something similar to this in the tab-delimited file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rs6010164      22    50515270    T    T&#xA;rs267607236    22    50515270    I    I&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(&lt;em&gt;Header line&lt;/em&gt;: rsid    chromosome    position    allele1    allele2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that when rsid numbers were created, sometimes multiple rsid numbers were accidentally assigned to the same region of the genome, but  I do not understand how two separate variants can be assigned to one single region in the genome. I could be wrong but as I understand it, I should only see one line here (either a &quot;T    T&quot; or an &quot;I    I&quot;) because genotypes are reported in comparison to the human reference genome and so each position should uniquely refer to a single nucleotide in the said human reference genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in this example case one line says there is a &quot;T    T&quot; genotype at this position, and the other line says there is an insertion(see &quot;How do I read my raw DNA data?&quot; &lt;a href=&quot;https://www.ancestry.com/dna/en/legal/us/faq#raw-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).  One possible explanation I thought of is if this would just be saying that a T was inserted, but if this is the case, then as I understand it the reference allele present in the individual should still reported, just in conjunction with the allele at the index. (So assuming an extra &quot;A&quot; was inserted after the T, which could be the reference or alternative allele, something like this should be recorded: rs6010164  22  50515270    TA  TA).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me understand why there are two separate variants associated with chromosomal position? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1574" LastEditorUserId="1574" LastEditDate="2018-03-13T19:59:30.490" LastActivityDate="2023-08-21T04:56:37.607" Title="Two variants associated with same chromosomal position in genotype data for one individual" Tags="|format-conversion|reference-genome|genotyping|" AnswerCount="4" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3882" PostTypeId="2" ParentId="3880" CreationDate="2018-03-13T07:59:38.627" Score="3" Body="&lt;p&gt;tldr: Remove the &lt;code&gt;$&lt;/code&gt; from the command.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I imagine you're literally typing &lt;code&gt;$hisat2&lt;/code&gt;, where you mean to instead type &lt;code&gt;hisat2&lt;/code&gt;. The &lt;code&gt;$&lt;/code&gt; is just meant to show the end of the command prompt. If you actually type &lt;code&gt;$hisat2&lt;/code&gt;, you're not executing the hisat2 program, but whatever the &lt;code&gt;$hisat2&lt;/code&gt; variable is set to. Since it's likely not set to anything, it's just ignored. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ryan@maximus:~$ $hisat2 ls&#xA;bin              convertChromosomesChain.py  ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since there's no &lt;code&gt;$hisat2&lt;/code&gt; variable, I'm actually just running &lt;code&gt;ls&lt;/code&gt;. Likewise, your command is just reducing to &lt;code&gt;-p 8 --dta -x daphnia_tran -1 C2*R1*.fastq -2 C2*R2*.fastq -S C2paired.sam&lt;/code&gt;, which is the source of the error.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-03-13T07:59:38.627" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3883" PostTypeId="1" CreationDate="2018-03-13T11:40:35.167" Score="1" ViewCount="211" Body="&lt;p&gt;I am trying to get screenshots from alignments of reads to a genome in &lt;a href=&quot;http://software.broadinstitute.org/software/igv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IGV&lt;/a&gt; (Integrated Genome Viewer) using &lt;a href=&quot;https://github.com/macarthur-lab/igv_utils/tree/master/igv_plotter&quot; rel=&quot;nofollow noreferrer&quot;&gt;igv_plotter&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;igv_plotter --genome ref.fa readsSorted.bam --locus-file locus_file.txt --max-memory 10G&#xA;&#xA;Traceback (most recent call last):&#xA;  File &quot;/Users/user/anaconda/envs/python3env/bin/igv_plotter&quot;, line 140, in &amp;lt;module&amp;gt;&#xA;    igv_robot.execute()&#xA;  File &quot;/Users/user/anaconda/envs/python3env/lib/python3.6/site-packages/igv_api.py&quot;, line 256, in execute&#xA;    self._execute_impl(self.__command_queue)&#xA;  File &quot;/Users/user/anaconda/envs/python3env/lib/python3.6/site-packages/igv_api.py&quot;, line 387, in _execute_impl&#xA;    fake_display.start()&#xA;  File &quot;/Users/user/anaconda/envs/python3env/lib/python3.6/site-packages/xvfbwrapper.py&quot;, line 79, in start&#xA;    raise RuntimeError('Xvfb did not start')&#xA;RuntimeError: Xvfb did not start&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;locus_file.txt only contains this: &lt;code&gt;I:10,000-20,000&lt;/code&gt;. From reading online, it seems that this Xvfb is what I need to be able to use IGV without a GUI. I am working on Mac High Sierra and I don't know if I can install Xvfb on Mac OS X or how I can do it. I was quickly out of my depths on the online posts about Xvfb on Mac. I checked that I installed igv_plotter correctly by successfully pulling out the help with &lt;code&gt;igv_plotter -h&lt;/code&gt;. I could not find Xvfb using brew.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, is there another programmatic way of taking images of aligned reads to a reference, similar to the one below?&lt;a href=&quot;https://i.stack.imgur.com/Ie14x.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ie14x.png&quot; alt=&quot;reads aligned to a reference&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Opening IGV and making a screenshot is not a viable option for me, as I would like to export hundreds or thousands or regions.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-03-13T12:58:39.210" LastActivityDate="2018-03-13T12:58:39.210" Title="igv_plotter error: Xvfb did not start" Tags="|sequence-alignment|visualization|igv|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3884" PostTypeId="2" ParentId="3871" CreationDate="2018-03-13T12:28:25.073" Score="3" Body="&lt;p&gt;Remove the &lt;code&gt;getAbsolutePath&lt;/code&gt; call — your path is already absolute. &lt;code&gt;getAbsolutePath&lt;/code&gt; garbles it, as you can see in the error message.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2018-03-13T12:28:25.073" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3885" PostTypeId="2" ParentId="3881" CreationDate="2018-03-13T14:10:04.313" Score="2" Body="&lt;p&gt;It's pretty common for SNPs and indels that map to the same position in the genome to be assigned different rsIDs, particularly when the indel spans multiple base pairs in the reference. You can get more info about these particular variants &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/projects/SNP/snp_ref.cgi?rs=6010164&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/projects/SNP/snp_ref.cgi?rs=267607236&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Or you can see how both of these variants relate to the reference sequence &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/variation/view/?q=rs267607236&amp;amp;filters=source%3Adbsnp&amp;amp;assm=GCF_000001405.25&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without info about the reference allele it's pretty confusing (particularly because they appear to be reporting the opposite strand from the database (G vs. C/T), but I would interpret this to mean that both alleles match the reference for rs267607236 (they have the insertion), but do not match the reference for rs6010164 (G-C pair rather than T-A). This is also the most common genotype at this position, being found in almost 80% of people in most populations. &lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-03-13T14:10:04.313" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3886" PostTypeId="1" AcceptedAnswerId="3887" CreationDate="2018-03-13T14:22:26.153" Score="0" ViewCount="651" Body="&lt;pre&gt;&lt;code&gt;ggplot(dataset, aes( x = Norm, fill = Condition, y=log2(measure))) + geom_boxplot() +&#xA;        scale_fill_manual(values=c(&quot;brown1&quot;,&quot;darkolivegreen4&quot;,&quot;burlywood3&quot;)) + theme_classic() + &#xA;        labs(title=&quot;Comparison&quot;,x=&quot;Norm&quot;, y = &quot;counts&quot;) +&#xA;        scale_fill_continuous(labels = paste(&quot;condition1&quot;, &quot;condition2&quot;, &quot;condition3&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am trying to change the labels of the legend. Currently, it shows the labels that are present in my dataset. However, I want to change it to &quot;condition1&quot;, &quot;condition2&quot;, &quot;condition3&quot;. Above is the code I am trying. It gives me following error.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Scale for 'fill' is already present. Adding another scale for 'fill', which will replace the existing scale. Error: Discrete value supplied to continuous scale In addition: Warning message: Removed 18 rows containing non-finite values (stat_boxplot).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Could someone help me here?&lt;/p&gt;&#xA;" OwnerUserId="2381" LastEditorUserId="29" LastEditDate="2018-03-13T17:17:40.140" LastActivityDate="2018-03-13T17:17:40.140" Title="Changing the legend names" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3887" PostTypeId="2" ParentId="3886" CreationDate="2018-03-13T14:27:52.213" Score="5" Body="&lt;p&gt;You don't have any continuous fill scales, since you're filling according to &lt;code&gt;Condition&lt;/code&gt;, which is a factor. This is leading to the &lt;code&gt;Error: Discrete value supplied to continuous scale&lt;/code&gt; error. You just need to combine your previous &lt;code&gt;scale_fill_manual(values=...)&lt;/code&gt; command with the &lt;code&gt;labels=&lt;/code&gt; part of your incorrect &lt;code&gt;scale_fill_continuous()&lt;/code&gt; command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(dataset, aes( x = Norm, fill = Condition, y=log2(measure))) +&#xA;    geom_boxplot() +&#xA;    scale_fill_manual(values=c(&quot;brown1&quot;,&quot;darkolivegreen4&quot;,&quot;burlywood3&quot;),&#xA;                      labels = paste(&quot;condition1&quot;, &quot;condition2&quot;, &quot;condition3&quot;)) +&#xA;    theme_classic() + &#xA;    labs(title=&quot;Comparison&quot;,x=&quot;Norm&quot;, y = &quot;counts&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Something along those lines should work.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-03-13T14:27:52.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3888" PostTypeId="2" ParentId="3827" CreationDate="2018-03-13T16:59:13.567" Score="1" Body="&lt;p&gt;The UCSC genome browser supports local mirrors. You could make a local mirror that has the &quot;TransMap Alignments&quot; cross-species gene track. From the &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTrackUi?hgsid=660022369_AUzllaH1GkM6HAQrM1t7y25Cdpme&amp;amp;c=chr5&amp;amp;g=transMapV4&quot; rel=&quot;nofollow noreferrer&quot;&gt;track description page&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;These tracks contain cDNA and gene alignments produced by the TransMap cross-species alignment algorithm from other vertebrate species in the UCSC Genome Browser. For closer evolutionary distances, the alignments are created using syntenically filtered LASTZ or BLASTZ alignment chains, resulting in a prediction of the orthologous genes in human. For more distant organisms, reciprocal best alignments are used.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;They provide a helpful tutorial on setting up a local installation &lt;a href=&quot;https://genome.ucsc.edu/goldenpath/help/mirror.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="643" LastActivityDate="2018-03-13T16:59:13.567" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3889" PostTypeId="1" CreationDate="2018-03-13T19:01:31.677" Score="13" ViewCount="186" Body="&lt;p&gt;I have estimated genome-wide heterozygosity levels using maximum likelihood and classical substitution model implemented in package &lt;a href=&quot;https://bitbucket.org/WegmannLab/atlas/wiki/Home&quot; rel=&quot;nofollow noreferrer&quot;&gt;atlas&lt;/a&gt;. These estimates are way more robust than classical SNP calling and simply counting number of heterogeneous locations because they integrate information from a big window into one estimate. I.e. I am quite sure about heterozygosity levels of SNPs of my genomes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, now I would like to know also about heterozygosity of indels and call individual snps using my robust estimates as a prior. GATK allows to specify priors on heterozygosity both of SNPs and indels, which might improve variant calls a lot, because default GATK is parametrised for human (Default priors for heterozygous SNPs is 0.001 and for indel 1.25E-4.). But, I work with insects I know that my SNPs heterozygosity is somewhere between 0.01 and 0.03 (1% - 3%).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So here comes the question, if I know the heterozygosity of SNPs, can I simply guess a reasonable prior for the heterozygosity of indels? GATK defaults for priors are 10x different, should I assume the same relation? Is there a reasonable way how to guess a prior for heterozygosity of indels?&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2018-11-12T23:14:40.213" LastActivityDate="2018-11-15T14:23:05.497" Title="Is it possible to use SNP heterozygosity as a proxy for Indel heterozygosity?" Tags="|snp|genomics|indel|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3890" PostTypeId="2" ParentId="3881" CreationDate="2018-03-13T20:10:37.393" Score="2" Body="&lt;p&gt;First of all, &lt;strong&gt;take these data with a very large pinch of salt&lt;/strong&gt;. This sort of targeted analysis is not designed to produce high quality genetic data but, to give an idea of a sample's ancestry. Given a file like the one you describe with no quality information, you have no way of knowing how reliable any of the variants called actually is. From personal experience with another, similar company, I expect that they aren't reliable at all. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tested four individuals using a commercial tool like the one you describe and while I don't doubt the validity of their main results (those displayed on their website), their raw data showed combinations of variants that were extremely unlikely to be real given that the individuals were still breathing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This isn't to say that such companies don't do their job well, only that their job is limited to what they actually display. When you look into their raw data, you're on your own. Presumably, they apply various filters before displaying the main results, but these are most likely not applied to the raw data. So if you want to do your own analyses, this is unlikely to be a reliable dataset. Also, I would strongly urge you to seek the help of an expert when interpreting any data you do have. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;That said, the rsIDs are simply identifiers of a specific genetic variation, not of a location. Any genomic position can have many (essentially infinite) possible variations. For example, a C residue at position 100 of chromosome 1 could become a G, a T or an A. Each of these will have its own, separate rsID. Now, if you also consider insertions and deletions, you can see why you can have essentially limitless possible variations at that specific position. So no, having more than one rsID describing the same position is not strange at all. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way these tests work is they look at a list of possible variants, each of which has its own rsID, and they report what they find for that rsID in the sample. Then, they will report what they find at the rsID's position for each allele. This is explained in &lt;a href=&quot;https://www.ancestry.com/dna/en/legal/us/faq#raw-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;the FAQ article&lt;/a&gt; you linked to:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The possible observations are A for adenine, C for cytosine, G for guanine, T for thymine, I for insertion’ and ‘D for deletion, or 0 for missing data. Column one provides the identifier (including the #rsID where possible). Columns two and three contain the chromosome and basepair position of the variant using human reference 37.1 coordinates. Columns four and five contain the two alleles observed at this variant (genotype)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Let's have a look at the two rsIds you mention.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://varsome.com/variant/hg38/rs6010164&quot; rel=&quot;nofollow noreferrer&quot;&gt;rs6010164&lt;/a&gt;: This is a single nucleotide variant (SNV) which describes a C residue where the reference genome has a T. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://varsome.com/variant/hg38/rs267607236&quot; rel=&quot;nofollow noreferrer&quot;&gt;rs267607236&lt;/a&gt;: this is a deletion at position 50076841, so a very different variant to the above. In this one, the sequence TGAG is deleted. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Your results are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rs6010164      22    50515270    T    T&#xA;rs267607236    22    50515270    I    I&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, your sample doesn't have rs6010164 since it has the same sequence at that position as the reference. However, it does have the insertion as indicated by the two Is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Is and Ds of the insertions and deletions (indels) are going to be the easy bit here. The only way you can parse this sort of file and convert it to VCF is to get the list of target SNV rsIDs, check what they describe as the reference and as the variant and then, for every one, check which of the two is shown in your file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're not an expert in this field, you will probably be better off hiring someone to do this for you. There are actually companies that offer this sort of service and can take these tab-delimited files as input. I even work for one! &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-03-13T20:10:37.393" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3891" PostTypeId="1" AcceptedAnswerId="3893" CreationDate="2018-03-14T04:28:04.560" Score="4" ViewCount="158" Body="&lt;p&gt;Basically I have a perl script in which I have an array (where each element of the array references a hash) and need to be able to print the array with a dumper function. &#xA;Thus I need to be able to split the $line on white spaces and save into variables and then construct anonymous hash and push into @genes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/perl&#xA;use strict;&#xA;use warnings;&#xA;use Data::Dumper; # print complex data structure&#xA;my @genes; # declare the array&#xA;while(&amp;lt;&amp;gt;) { # this means that as long as lines come from the pipe we keep going&#xA;  my $line = $_; # a line that come from the pipe (we go line by line)&#xA;  next unless $line =~ /^\d+/; # skip lines except those reporting genes &#xA;  &amp;lt;split the $line on white spaces and save into variables&amp;gt;&#xA;  &amp;lt;anonymous hash and push into @genes&amp;gt;&#xA;}&#xA;print Dumper(\@genes);&#xA;exit;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't  even know how to approach this, this is my first time perl scripting and I am really confused. Any insight would be extremely helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;my $gene = {'id' =&amp;gt; $id, 'start' =&amp;gt; $start, 'end' =&amp;gt; $end, 'frame' = $frame, 'score' =&amp;gt; $score};&#xA;my @line_split = split(' ', $line);&#xA;my $id = $line_split [0], $start = $line_split [1], $end = $line_split [2], $frame = $line_split [3], $score = $line_split [4]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2348" LastEditorUserId="73" LastEditDate="2018-03-15T21:20:28.507" LastActivityDate="2018-03-16T23:47:24.860" Title="Writing a perl script to holding information for two genes" Tags="|gene|perl|homework|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3892" PostTypeId="2" ParentId="3891" CreationDate="2018-03-14T05:52:13.277" Score="2" Body="&lt;p&gt;The following Perl documentation pages should be informative:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://perldoc.perl.org/functions/split.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;split&lt;/a&gt; - for splitting a scalar at all matches of a defined pattern&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://perldoc.perl.org/perlreftut.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;perlreftut&lt;/a&gt; - discusses the approach of anonymous variables and how to combine them&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-03-14T05:52:13.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3893" PostTypeId="2" ParentId="3891" CreationDate="2018-03-14T06:31:33.300" Score="6" Body="&lt;p&gt;Perhaps this small test script will help demonstrate some of the principles:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env perl                                                                                                                                              &#xA;&#xA;use strict;&#xA;use warnings;&#xA;use Data::Dumper;&#xA;&#xA;my @arr;&#xA;&#xA;# define the current line    &#xA;my $line = &quot;foo\tbar\tbaz\n&quot;;&#xA;&#xA;# remove the trailing newline character&#xA;chomp $line; &#xA;&#xA;# put the line's elements into an anonymous array&#xA;my ($a, $b, $c) = split('\t', $line);&#xA;&#xA;# insert key-value pairs into anonymous hash    &#xA;my $hash = {a =&amp;gt; $a, b =&amp;gt; $b, c =&amp;gt; $c};&#xA;&#xA;# add the hash table to the array&#xA;push @arr, $hash;&#xA;&#xA;# print the array&#xA;print Dumper \@arr;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Run this script and see what it prints out.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-03-14T06:48:49.310" LastActivityDate="2018-03-14T06:48:49.310" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3894" PostTypeId="1" CreationDate="2018-03-14T08:44:03.440" Score="6" ViewCount="105" Body="&lt;p&gt;I'm looking for some software that can &quot;efficiently&quot; (time and memory) compute the exact percentage of non-unique k-mers in a genome for given k. I don't need the k-mers or the abundances itself, I just need the percentage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, the result could be:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the number of different k-mers observed in the genome&lt;/li&gt;&#xA;&lt;li&gt;the number of unique k-mers, i.e. k-mers that occur only once in the genome, &lt;strong&gt;or&lt;/strong&gt; the number of non-unique k-mers, i.e., k-mers that occur more than once in the genome.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;which can be easily used to compute the requested percentage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Requested features:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;don't count k-mers with ambiguous nucleotides&lt;/li&gt;&#xA;&lt;li&gt;count a k-mer and its reverse complement together&lt;/li&gt;&#xA;&lt;li&gt;k upto 50 or 100&lt;/li&gt;&#xA;&lt;li&gt;for large genomes (upto some GB)  &lt;/li&gt;&#xA;&lt;li&gt;efficiency&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For the study in mind, I like to vary k. Hence, I will run the software several times with different k.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is welcome.&lt;/p&gt;&#xA;" OwnerUserId="2504" LastActivityDate="2018-03-14T12:04:59.950" Title="How to efficiently compute the exact percentage of non-unique k-mers in a genome for given k?" Tags="|genome|software-recommendation|k-mer|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3895" PostTypeId="1" CreationDate="2018-03-14T11:45:07.760" Score="8" ViewCount="489" Body="&lt;p&gt;Working on a fastQ file, I would like to get the occurrences of repeated sequences for all possible kmers of a given length that cover at least 90% of the read's length for the whole data set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;example : &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;for a length 6 with the kmer &quot;ATTGGG&quot; and a data set  containing reads of 300 bases each&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I want to obtain the occurrences of the following kmers :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ATTGGGATTGGGATTGGGATTGGGATTGGGATTGGG......  (90 % of the read's length without  gaps) &#xA;ATTGGGATTGGGATTGGGATTGGGATTGGGATTGGGA.....  (91%)&#xA;ATTGGGATTGGGATTGGGATTGGGATTGGGATTGGGAT....  (92%)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;this for all possible combinations of kmers of length = 6.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have done some research and tested many softwares such as (KMC2,Jellyfish,Khmer,Dsk,Scturtle) but none matched what I would like to do, because I want to obtain the occurrences of the repeated kmer without any possible gaps in-between with a certain % of read coverage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've done a naive algorithm that works but the optimization isn't the best obviously. Is there a tool that does the trick ?&lt;/p&gt;&#xA;" OwnerUserId="2461" LastEditorUserId="2461" LastEditDate="2018-03-14T12:06:27.773" LastActivityDate="2018-03-15T15:08:19.987" Title="Counting repeated kmers sequences that match at least x % of reads sequence" Tags="|fastq|k-mer|reads|repeat-elements|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3896" PostTypeId="2" ParentId="3894" CreationDate="2018-03-14T12:04:59.950" Score="5" Body="&lt;p&gt;You can do all of that with &lt;a href=&quot;http://khmer.readthedocs.io/en/v2.1.1/&quot; rel=&quot;noreferrer&quot;&gt;khmer&lt;/a&gt;. For example, &lt;code&gt;abundance-dist-single.py&lt;/code&gt; produces a file with columns: &lt;code&gt;k-mer abundance&lt;/code&gt;, &lt;code&gt;k-mer count&lt;/code&gt;, &lt;code&gt;cumulative count&lt;/code&gt;, and fraction of total distinct k-mers&lt;code&gt;. So for question 1 you would sum column 2. For question 2 you would just get the&lt;/code&gt;k-mer count&lt;code&gt;associated with a&lt;/code&gt;k-mer abundance` of 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That package also provides a python API (see the read the docs link above) if you need to customize things more.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-03-14T12:04:59.950" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3897" PostTypeId="1" AcceptedAnswerId="3904" CreationDate="2018-03-14T14:21:08.887" Score="6" ViewCount="366" Body="&lt;p&gt;Is there any possibility to run &lt;a href=&quot;http://ccb.jhu.edu/software/stringtie/gff.shtml&quot; rel=&quot;noreferrer&quot;&gt;gffread&lt;/a&gt; in multi-thread mode? The answer seems to be 'no' from the &lt;a href=&quot;http://manpages.ubuntu.com/manpages/trusty/man1/gffread.1.html&quot; rel=&quot;noreferrer&quot;&gt;manual&lt;/a&gt; (or &lt;code&gt;gffread -h&lt;/code&gt;), as no multi-thread option is mentioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm mostly using this utility to extract transcript sequences (FASTA) from annotation files (GTF). In single-thread mode, runtime is acceptable in most cases, but would like to speed up a bit in some cases (especially for big genomes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I already tried, as mentioned in the manual, to index the genome first (which, again, cuts the runtime in most but not all cases): &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Note that the retrieval of the transcript sequences this way is going to be much faster if a fasta index file (genome.fa.fai in this example) is found in the same directory with the genomic fasta file. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Such an index file can be created with the samtools utility prior to running gffread, like this: &lt;code&gt;samtools faidx genome.fa&lt;/code&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Then in subsequent runs using the -g option gffread will find this&#xA;  FASTA index and use it to speed up the extraction of transcript&#xA;  sequences.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm also open to different alternatives than &lt;code&gt;gffread&lt;/code&gt; to convert GTF -&gt; FASTA. &lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2018-03-14T23:30:11.627" Title="Run gffread in multi-thread mode" Tags="|rna-seq|fasta|file-formats|gtf|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3898" PostTypeId="2" ParentId="3895" CreationDate="2018-03-14T15:03:22.837" Score="6" Body="&lt;p&gt;The following javascript does what you want. You need node.js to run it. It should be easy to translate the code to Python. I have not carefully tested it. Use with caution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT (response to new comments): the program has been changed to compute the sum of lengths. Note that only a tandem repeat of length k*2 or longer is counted. For example in sequence &lt;code&gt;ATTGGGATTGGGATTcGGGATTGGG&lt;/code&gt;, the script returns 15, because the second is not long enough. Counting length smaller than k*2 will be slower and more complex, but is often not the right thing to do. For example, in &lt;code&gt;ATTGGGATTGGGcccccccgaaatcgatagcatcgaGGGATTGcgatc&lt;/code&gt;, we would not count the second &lt;code&gt;GGGATTG&lt;/code&gt; as a tandem repeat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On performance, for an input string of length $l$, the time complexity of this script is $O(l)$ associated with a small constant. It will be efficient and adequate to practical uses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use &lt;a href=&quot;https://tandem.bu.edu/trf/trf.html&quot; rel=&quot;noreferrer&quot;&gt;TRF&lt;/a&gt;, you should run it on your sequences and then test if any tandem repeats it finds match &lt;code&gt;GGGAAT&lt;/code&gt; – the procedure will be similar to my script, except that TRF will also find impure repeats.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-js prettyprint-override&quot;&gt;&lt;code&gt;// given the repeat unit length _k_, find all tandem repeats of 2*k or longer&#xA;function trf_k(k, str)&#xA;{&#xA;    var streak = 0, a = [];&#xA;    for (var i = k; i &amp;lt;= str.length; ++i) {&#xA;        if (i &amp;lt; str.length &amp;amp;&amp;amp; str[i] == str[i - k]) {&#xA;            ++streak;&#xA;        } else {&#xA;            if (streak &amp;gt;= k)&#xA;                a.push([i - streak - k, i]);&#xA;            streak = 0;&#xA;        }&#xA;    }&#xA;    return a;&#xA;}&#xA;// find the sum of tandem repeat length with _kmer_ being the repeat unit&#xA;function trf_kmer(kmer, str)&#xA;{&#xA;    var a = trf_k(kmer.length, str), sum = 0;&#xA;    for (var i = 0; i &amp;lt; a.length; ++i)&#xA;        if (str.substring(a[i][0], a[i][1]).indexOf(kmer) &amp;gt;= 0)&#xA;            sum += a[i][1] - a[i][0];&#xA;    return sum;&#xA;}&#xA;&#xA;var str = 'xATTGGGATTGGGATTGxGATTGGGATTGGGATTGGGATTGxx';&#xA;console.log(trf_kmer('GGGATT', str));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2018-03-15T15:08:19.987" LastActivityDate="2018-03-15T15:08:19.987" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3899" PostTypeId="5" CreationDate="2018-03-14T15:26:58.443" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-14T15:26:58.443" LastActivityDate="2018-03-14T15:26:58.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3900" PostTypeId="4" CreationDate="2018-03-14T15:26:58.443" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-03-14T15:26:58.443" LastActivityDate="2018-03-14T15:26:58.443" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3901" PostTypeId="2" ParentId="3854" CreationDate="2018-03-14T17:08:33.247" Score="4" Body="&lt;p&gt;The most intuitive explanation is  also explained in the background section:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Conceptually, this methodology can be understood as a change in coordinate systems for gene expression data, from genes to gene sets.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Which I think it is explained in Figure 1 of the paper, which is also one of the most informative I found about the methods of an algorithm&lt;sup&gt;1&lt;/sup&gt;.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/cRPLI.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cRPLI.jpg&quot; alt=&quot;GSVA figure 1&quot;&gt;&lt;/a&gt;&#xA;I'll will use it to explain what's going on.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Fit a distribution function per gene&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For each gene expression profile ... a non-parametric kernel estimation ... function is performed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I can't give more detail because I haven't understood it fully&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Then they are normalized to make the ranks symmetric around zero&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is an inline formula between equations 2 and 3.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The Kolmogorov-Smirnov like random walk statistic is applied to those kernel estimations normalized distributions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which two distributions does it compare? The genes in a gene set and all the others (but I'm not totally sure)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The role of random walk is explained a bit later from your quote:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[the random walk] produces a distribution over the genes to assess if the genes in the gene set are more likely to be found at either tail of the rank distribution&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Which is later used to calculate the enrichment score by looking the maximum deviation from zero, or by summing the largest and the lowest deviations. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;To address some other questions:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;GSVA calculates sample-wise gene set enrichment scores as a function of genes inside and outside the gene set, analogously to a competitive gene set test&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;1: Try finding which fittings and approximations are done inside limma&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2019-01-08T20:46:18.663" LastActivityDate="2019-01-08T20:46:18.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="3902" PostTypeId="2" ParentId="3835" CreationDate="2018-03-14T21:30:02.287" Score="2" Body="&lt;p&gt;I'd recommend using Canu for correction rather than Pilon, because it has a component that is specifically designed for read-level correction. The newest version of Canu (v1.7) will track &lt;em&gt;all&lt;/em&gt; reads through the correction and assembly process:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Ensure that every raw read is either corrected or used as evidence for correcting some other raw read. This serves to rescue short plasmids in high coverage datasets, and it is no longer necessary to set corOutCoverage to achieve the same result.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;More details &lt;a href=&quot;https://github.com/marbl/canu/releases/tag/v1.7&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-03-14T21:30:02.287" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3903" PostTypeId="1" AcceptedAnswerId="4780" CreationDate="2018-03-14T23:25:23.123" Score="8" ViewCount="682" Body="&lt;p&gt;I've been searching TaxIDs against NCBI's Taxonomy DB to get taxonomic lineages for species.  I have successfully done this for 1,000's of TaxIDs that were returned to me by Blast+ blasts in a CSV.  (The first column in the CSV was the TaxID for the top hit of a sequence, and the other columns were for GI, Scientific Name, e-value, etc.).  I simply wrote a script that would take the first field, which is always the TaxID, and search it against Taxonomy to get lineages for the species those TaxIDs represented.  Everything worked fine!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, some of the records returned from my original blast (that gave me the TaxIDs in the first place) would return two or more TaxIDs per record.  (I suspect because a sequence matched equally well with two or more different species?)  That's fine, but I had to remove these few exceptions before I searched the rest of the regular results against Taxonomy.  So I've got most of my data/lineages, but for about 1:10,0000 records that gave multiple TaxIDs, I would like to know the lineages of the multiple species that the blast suggested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I now took all of those &quot;more-than-one-taxid&quot; hits, separated them out into columns in an excel sheet and saved it as a CSV, and am trying to search these against Taxonomy with essentially the same script I used for the rest of the data, but am getting an error for &lt;em&gt;every&lt;/em&gt; taxID I search (see below).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Complete Error&lt;/strong&gt; (for each taxid searched against Taxonomy):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;400 Bad Request&#xA;&amp;amp;rettype=full&amp;amp;retmode=xml&amp;amp;edirect_os=linux&amp;amp;edirect=7.90&amp;amp;tool=edirect&amp;amp;email=x@xxx.localdomain'my&amp;amp;id=464126&#xA;Result of do_post http request is&#xA;$VAR1 = bless( {&#xA;                 '_protocol' =&amp;gt; 'HTTP/1.1',&#xA;                 '_content' =&amp;gt; '&amp;lt;?xml version=&quot;1.0&quot; encoding=&quot;UTF-8&quot; ?&amp;gt;&#xA;&amp;lt;!DOCTYPE eEfetchResult PUBLIC &quot;-//NLM//DTD efetch 20131226//EN&quot; &quot;https://eutils.ncbi.nlm.nih.gov/eutils/dtd/20131226/efetch.dtd&quot;&amp;gt;&#xA;&amp;lt;eFetchResult&amp;gt;&#xA;	&amp;lt;ERROR&amp;gt;WebEnv parameter is required&amp;lt;/ERROR&amp;gt;&#xA;&amp;lt;/eFetchResult&amp;gt;&#xA;',&#xA;                 '_rc' =&amp;gt; 400,&#xA;                 '_headers' =&amp;gt; bless( {&#xA;                                        'connection' =&amp;gt; 'close',&#xA;                                        'set-cookie' =&amp;gt; 'ncbi_sid=9FF573148F458940_CEC6SID; domain=.nih.gov; path=/; expires=Thu, 14 Mar 2019 22:36:03 GMT',&#xA;                                        'cache-control' =&amp;gt; 'private',&#xA;                                        'access-control-allow-origin' =&amp;gt; '*',&#xA;                                        'date' =&amp;gt; 'Wed, 14 Mar 2018 22:36:03 GMT',&#xA;                                        'vary' =&amp;gt; 'Accept-Encoding',&#xA;                                        'client-ssl-cert-issuer' =&amp;gt; '/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA',&#xA;                                        'client-ssl-cipher' =&amp;gt; 'ECDHE-RSA-AES256-GCM-SHA384',&#xA;                                        'client-peer' =&amp;gt; '130.14.29.110:443',&#xA;                                        'strict-transport-security' =&amp;gt; 'max-age=31536000; includeSubDomains; preload',&#xA;                                        'client-date' =&amp;gt; 'Wed, 14 Mar 2018 22:36:03 GMT',&#xA;                                        'content-type' =&amp;gt; 'text/xml; charset=UTF-8',&#xA;                                        'x-xss-protection' =&amp;gt; '1; mode=block',&#xA;                                        'client-transfer-encoding' =&amp;gt; [&#xA;                                                                        'chunked'&#xA;                                                                      ],&#xA;                                        'server' =&amp;gt; 'Finatra',&#xA;                                        'client-ssl-socket-class' =&amp;gt; 'IO::Socket::SSL',&#xA;                                        'client-response-num' =&amp;gt; 1,&#xA;                                        'ncbi-phid' =&amp;gt; 'DAF6EB0134DF5AEC039F11D248910BE7.1',&#xA;                                        'ncbi-sid' =&amp;gt; '9FF573148F458940_CEC6SID',&#xA;                                        '::std_case' =&amp;gt; {&#xA;                                                          'set-cookie' =&amp;gt; 'Set-Cookie',&#xA;                                                          'access-control-allow-origin' =&amp;gt; 'Access-Control-Allow-Origin',&#xA;                                                          'client-ssl-cert-issuer' =&amp;gt; 'Client-SSL-Cert-Issuer',&#xA;                                                          'client-ssl-cipher' =&amp;gt; 'Client-SSL-Cipher',&#xA;                                                          'client-peer' =&amp;gt; 'Client-Peer',&#xA;                                                          'strict-transport-security' =&amp;gt; 'Strict-Transport-Security',&#xA;                                                          'client-date' =&amp;gt; 'Client-Date',&#xA;                                                          'x-xss-protection' =&amp;gt; 'X-XSS-Protection',&#xA;                                                          'client-transfer-encoding' =&amp;gt; 'Client-Transfer-Encoding',&#xA;                                                          'client-ssl-socket-class' =&amp;gt; 'Client-SSL-Socket-Class',&#xA;                                                          'client-response-num' =&amp;gt; 'Client-Response-Num',&#xA;                                                          'ncbi-phid' =&amp;gt; 'NCBI-PHID',&#xA;                                                          'ncbi-sid' =&amp;gt; 'NCBI-SID',&#xA;                                                          'x-ua-compatible' =&amp;gt; 'X-UA-Compatible',&#xA;                                                          'client-ssl-cert-subject' =&amp;gt; 'Client-SSL-Cert-Subject',&#xA;                                                          'content-security-policy' =&amp;gt; 'Content-Security-Policy'&#xA;                                                        },&#xA;                                        'x-ua-compatible' =&amp;gt; 'IE=Edge',&#xA;                                        'client-ssl-cert-subject' =&amp;gt; '/C=US/ST=Maryland/L=Bethesda/O=National Library of Medicine/CN=*.ncbi.nlm.nih.gov',&#xA;                                        'content-security-policy' =&amp;gt; 'upgrade-insecure-requests'&#xA;                                      }, 'HTTP::Headers' ),&#xA;                 '_msg' =&amp;gt; 'Bad Request',&#xA;                 '_request' =&amp;gt; bless( {&#xA;&amp;amp;rettype=full&amp;amp;retmode=xml&amp;amp;edirect_os=linux&amp;amp;edirect=7.90&amp;amp;tool=edirect&amp;amp;email=x@xxx.localdomain',&#xA;                                        '_uri' =&amp;gt; bless( do{\(my $o = 'https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi')}, 'URI::https' ),&#xA;                                        '_headers' =&amp;gt; bless( {&#xA;                                                               'user-agent' =&amp;gt; 'libwww-perl/6.05',&#xA;                                                               'content-type' =&amp;gt; 'application/x-www-form-urlencoded',&#xA;                                                               '::std_case' =&amp;gt; {&#xA;                                                                                 'if-ssl-cert-subject' =&amp;gt; 'If-SSL-Cert-Subject'&#xA;                                                                               }&#xA;                                                             }, 'HTTP::Headers' ),&#xA;                                        '_method' =&amp;gt; 'POST',&#xA;                                        '_uri_canonical' =&amp;gt; $VAR1-&amp;gt;{'_request'}{'_uri'}&#xA;                                      }, 'HTTP::Request' )&#xA;               }, 'HTTP::Response' );&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I've done:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a sample of the data I used that worked correctly and yielded lineages when searching the taxIDs (taxIDs in 1st column):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;549298,Contig1,Francisella halioticida,5.44e-126,gi|1213460577|gb|CP022132.1|,CP022132&#xA;240395,Contig2,Glandirana emeljanovi,4.74e-131,gi|1025812536|gb|KU641020.1|,KU641020&#xA;1069680,Contig3,Pneumocystis murina B123,2.34e-85,gi|1124120202|ref|XM_019757798.1|,XM_019757798&#xA;3871,Contig4,Lupinus angustifolius,1.89e-93,gi|1239725835|gb|CP023132.1|,CP023132&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is a sample of the &lt;strong&gt;data that I am trying to get to work&lt;/strong&gt; (when I search these manually against Taxonomy from their web page they seem to work and give lineages; it's just that I have just enough of them to make a non-programmatic solution really time consuming and tedious):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;258234&#xA;1316279&#xA;1316315&#xA;710996&#xA;711000&#xA;225338&#xA;985895&#xA;710996&#xA;1382731&#xA;77133&#xA;344338&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the business end of &lt;strong&gt;the script that I used that worked for the  working data&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;while IFS=',' read -r f1 f2 f3 f4 f5 f6&#xA;do      &#xA;        #Firstly echo the Contig# (Query Seq. ID) followed by lineage information.&#xA;        echo -n &quot;$f2,&quot; &amp;gt;&amp;gt; $2&#xA;&#xA;        #Now get the lineage, pipe to xtract and parse XML for readability in output file&#xA;        efetch -db taxonomy \&#xA;        -id $f1 \&#xA;        -format xml \&#xA;        | xtract -pattern Taxon \&#xA;                -tab ',' -sep ',' \&#xA;                -def &quot;NONE&quot; \&#xA;                -element TaxId,ScientificName \&#xA;                -division LineageEx \&#xA;                -group Taxon \&#xA;                -if Rank -equals superkingdom \&#xA;                -or Rank -equals kingdom \&#xA;                ...&#xA;                -or Rank -equals species \&#xA;                -tab ',' -sep ',' \&#xA;                -def &quot;NOT_AVAILABLE&quot; \&#xA;                -element Rank,ScientificName \&#xA;                &amp;gt;&amp;gt; $2&#xA;done &amp;lt; &quot;$1&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the slightly altered script that I am trying to use to make the not-working set of data yield results (and that produces the error):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;while IFS='' read -r line || [[ -n &quot;$line&quot; ]]; do&#xA;        efetch -db taxonomy \&#xA;        -id $line \&#xA;        -format xml \&#xA;        | xtract -pattern Taxon \&#xA;                -tab ',' -sep ',' \&#xA;                -def &quot;NONE&quot; \&#xA;                -element TaxId,ScientificName \&#xA;                -division LineageEx \&#xA;                -group Taxon \&#xA;                -if Rank -equals superkingdom \&#xA;                -or Rank -equals kingdom \&#xA;                ...&#xA;                -or Rank -equals species \&#xA;                -or Rank -equals subspecies \&#xA;                -tab ',' -sep ',' \&#xA;                -def &quot;NOT_AVAILABLE&quot; \&#xA;                -element Rank,ScientificName \&#xA;                &amp;gt;&amp;gt; $2&#xA;done &amp;lt; &quot;$1&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, &lt;strong&gt;the only difference&lt;/strong&gt; is that I am reading in entire lines for the TaxIDs list because it is simply a list with line breaks between the IDs as opposed to a true CSV with other values as in the example that works.  I've saved the taxIDs list as both a CSV and TXT and it doesn't make a difference.  Also: I can echo what $line is in the second script and it is exactly I'd expect--just the numbers of the taxID, &lt;strong&gt;so why would I be getting a 400 error for bad syntax on my query if it looks exactly like what I am sending in the first, working script?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="2446" LastActivityDate="2018-07-27T03:54:22.750" Title="Using a Bash Script to search TaxIDs against NCBI's Taxonomy yields &quot;400 Bad Request&quot; error?" Tags="|taxonomy|shell|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3904" PostTypeId="2" ParentId="3897" CreationDate="2018-03-14T23:30:11.627" Score="3" Body="&lt;p&gt;Since there doesn't seem to be an easy way to run this in parallel, you could instead break the job into sections. For example, separate each chromosome into its own gtf file, extract the sequences using that file and then cat them all together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The commands would be something like this (using the human hg38 genome and the gff GENCODE annotations downloaded from &lt;a href=&quot;https://www.gencodegenes.org/releases/current.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Split the file into one file per chromosome (or whatever else you have as the first field of your gff/gtf file):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tmp=$(mktemp -d); ## create a temp directory&#xA;awk -vtmp=&quot;$tmp&quot; '($1!~/^#/){print &amp;gt; tmp&quot;/&quot;$1&quot;.gff&quot;}' gencode.v27.annotation.gtf &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Extract the sequences described in each file, running each command in the background (&lt;code&gt;&amp;amp;&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for f in $tmp/*; do &#xA;    gffread -w &quot;$f.fa&quot; -g hg38.fa &quot;$f&quot; &amp;amp; &#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Concatenate them into one file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat &quot;$tmp/*fa&quot; &amp;gt; all.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can combine all this into a single command, using &lt;code&gt;wait&lt;/code&gt; to make sure you don't concatenate until all of the sub-commands have finished:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tmp=$(mktemp -d); &#xA;awk -vtmp=&quot;$tmp&quot; '($1!~/^#/){print &amp;gt; tmp&quot;/&quot;$1&quot;.gff&quot;}' gencode.v27.annotation.gtf &amp;amp;&amp;amp; &#xA;    for f in &quot;$tmp&quot;/*; do &#xA;        gffread-0.9.12.Linux_x86_64/gffread -w $f.fa -g hg38.fa &quot;$f&quot; &amp;amp; &#xA;    done &#xA;    wait &#xA;    cat &quot;$tmp&quot;/*fa &amp;gt; all.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, just save it as a script called pgffread.sh (or whatever):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;#!/bin/sh&#xA;gtf=&quot;$1&quot;&#xA;genome=&quot;$2&quot;&#xA;&#xA;tmp=$(mktemp -d);&#xA;awk -vtmp=&quot;$tmp&quot; '($1!~/^#/){print &amp;gt; tmp&quot;/&quot;$1&quot;.gff&quot;}' &quot;$gtf&quot; &amp;amp;&amp;amp;&#xA;    for f in &quot;$tmp&quot;/*; do&#xA;      gffread -w &quot;$f&quot;.fa -g &quot;$genome&quot; &quot;$f&quot; &amp;amp;&#xA;    done&#xA;wait&#xA;cat &quot;$tmp&quot;/*fa &#xA;rm -rf &quot;$tmp&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then run it giving the genome and annotation files as parameters:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pgffread.sh gencode.v27.annotation.gtf hg38.fa &amp;gt; all.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;On the system I ran it on, which has 48 cores, the time difference was significant (although it's so fast anyway, I really don't know if it is worth the effort unless you do this very often):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ time gffread -w out.fa -g hg38.fa gencode.v27.annotation.gtf&#xA;real    1m52.072s&#xA;user    1m3.160s&#xA;sys     0m47.257s&#xA;&#xA;$ time ./pgffread.sh gencode.v27.annotation.gtf hg38.fa &amp;gt; all.fa&#xA;real    0m17.533s&#xA;user    1m5.228s&#xA;sys     0m43.039s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-03-14T23:30:11.627" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3907" PostTypeId="1" CreationDate="2018-03-15T18:02:38.800" Score="5" ViewCount="181" Body="&lt;p&gt;I am trying to use DRIMseq for DTU with 2 treatments on two different strains of mice. In my previous analysis with genes only, I had found a strong interaction effect  of strain and treatments. Is it possible to model that with DRIMseq? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also when I try to add strain term and use it as the batch effect as described in the tutorial, I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;! Using a subset of 0.1 genes to estimate common precision !&#xA;Error in optimHess(par = par, fn = dm_lik_regG, gr = dm_score_regG, x = x,  : &#xA;  non-finite value supplied by optim&#xA;In addition: There were 50 or more warnings (use warnings() to see the first 50)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Please see the sessionInfo() below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R version 3.4.0 (2017-04-21)&#xA;Platform: x86_64-redhat-linux-gnu (64-bit)&#xA;Running under: CentOS Linux 7 (Core)&#xA;&#xA;Matrix products: default&#xA;BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so&#xA;&#xA;locale:&#xA; [1] LC_CTYPE=en_GB.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_GB.UTF-8     LC_MONETARY=en_US.UTF-8   &#xA; [6] LC_MESSAGES=en_GB.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            &#xA;[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       &#xA;&#xA;attached base packages:&#xA;[1] stats4    parallel  stats     graphics  grDevices utils     datasets  methods   base     &#xA;&#xA;other attached packages:&#xA; [1] DRIMSeq_1.6.0                            pheatmap_1.0.8                           ggplot2_2.2.1                           &#xA; [4] edgeR_3.16.5                             limma_3.30.13                            TxDb.Mmusculus.UCSC.mm10.knownGene_3.4.0&#xA; [7] GenomicFeatures_1.26.4                   AnnotationDbi_1.36.2                     Biobase_2.34.0                          &#xA;[10] GenomicRanges_1.26.4                     GenomeInfoDb_1.10.3                      IRanges_2.8.2                           &#xA;[13] S4Vectors_0.12.2                         BiocGenerics_0.20.0                      tximport_1.2.0                          &#xA;&#xA;loaded via a namespace (and not attached):&#xA; [1] Rcpp_0.12.12               RColorBrewer_1.1-2         plyr_1.8.4                 pillar_1.2.1               compiler_3.4.0            &#xA; [6] XVector_0.14.1             bitops_1.0-6               tools_3.4.0                zlibbioc_1.20.0            biomaRt_2.30.0            &#xA;[11] digest_0.6.12              bit_1.1-12                 gtable_0.2.0               RSQLite_2.0                memoise_1.1.0             &#xA;[16] tibble_1.4.2               lattice_0.20-35            pkgconfig_2.0.1            rlang_0.2.0                Matrix_1.2-10             &#xA;[21] DBI_0.7                    stringr_1.3.0              rtracklayer_1.34.2         Biostrings_2.42.1          locfit_1.5-9.1            &#xA;[26] bit64_0.9-7                grid_3.4.0                 XML_3.98-1.10              BiocParallel_1.8.2         magrittr_1.5              &#xA;[31] reshape2_1.4.3             blob_1.1.0                 MASS_7.3-47                scales_0.5.0               Rsamtools_1.26.2          &#xA;[36] GenomicAlignments_1.10.1   SummarizedExperiment_1.4.0 colorspace_1.3-2           labeling_0.3               stringi_1.1.7             &#xA;[41] lazyeval_0.2.1             munsell_0.4.3              RCurl_1.95-4.10           &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2514" LastEditorUserId="48" LastEditDate="2018-03-15T21:42:14.063" LastActivityDate="2018-03-15T22:19:01.873" Title="Can I use DRIMseq to model interaction effect of variables?" Tags="|differential-expression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3908" PostTypeId="2" ParentId="3907" CreationDate="2018-03-15T22:19:01.873" Score="2" Body="&lt;p&gt;One way to see the interaction effect would be including it in the design matrix as explained in the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/limma/inst/doc/usersguide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;limma tutorial&lt;/a&gt;. Something on the line of:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;design_full &amp;lt;- model.matrix(~ treatment*strain, data = samples(d))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or you could build it yourself in a matrix like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; (Intercept) strainA_treatmentA strainA_treatmentB  strainB_treatmentA strainB_treatmentB&#xA;       1                      0                  1                  0                   0&#xA;       1                      1                  0                  0                   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;About the error, it seems like the data is not in the right format, you have either &lt;code&gt;NA&lt;/code&gt; or &lt;code&gt;Inf&lt;/code&gt; somewhere in the matrix. Check it and rethink the processing you made. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-03-15T22:19:01.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3909" PostTypeId="1" CreationDate="2018-03-16T15:41:59.020" Score="4" ViewCount="82" Body="&lt;p&gt;I'm mathematically modeling yellow fever, and yet, after assembling the model, I need to find the stationary states of the model. I had to do it by hand because wolfram software could not get the necessary values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, how do I check if my stationary states are in line with my biological model? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was told that there is a way to verify this by performing a substitution of the s values of the points within my own template code. But I have no idea how to do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code used is taking forever to get back to me as this data behaves in my model&#xA;    &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function ypsir = ypsirtry(t,y)&#xA;betah = .01;&#xA;muh = 0.01;&#xA;sigmah = 0.05;&#xA;alphah = 0.06;&#xA;deltah = 0.05; &#xA;mui = 0.07;&#xA;mum = 0.07;&#xA;betai = 0.03;&#xA;lambdai = 0.06;&#xA;lambdah = 0.5;&#xA;k = 500;&#xA;&#xA;ypsir = zeros(5,1);&#xA;ypsir(1) = sigmah*y(3) - muh*y(1) - betah*y(1)*y(5) + lambdah*(y(2)+ y(3) + &#xA;y(1));&#xA;ypsir(2) = -deltah*y(2) - y(2)*(alphah + muh) + betah*y(5)*y(1);&#xA;ypsir(3) = deltah*y(2) - sigmah*y(3);&#xA;ypsir(4) = -mui*y(4) - mum*y(4) - betai*y(2)*y(4) + lambdai*(y(4) + y(5))*(1 &#xA;- (y(4) + y(5))/k);&#xA;ypsir(5) = betai*y(2)*y(4) - mui*y(5) - mum*y(5);&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Along with this calling program&#xA;    &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clear;&#xA;to = 0;&#xA;tf =50;&#xA;yo = [99 1 0 1 3];&#xA;[t y] = ode45(@(t,y)ypsirtry(t,y),[to tf],yo); &#xA;plot(t,y(:,1),t,y(:,2),t,y(:,3),t,y(:,4),t,y(:,5))&#xA;title('First Model of Yellow Fever')&#xA;xlabel('time')&#xA;ylabel('susceptible, infected, recovered,Mosquito Non-carrier, Carrier &#xA;mosquito') &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My stationary states are:&#xA;    &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;S = (1/betai)*[(muh+alphah)+(sigmah/deltah)*deltah]*(mui+mum)/betai*N*I);&#xA;&#xA;I = sigmah*R/deltah;&#xA;&#xA;R = ((muh/betah + lambdah)*(muh+alphah))/(sigmah+lamdah)-(muh+alphah)* &#xA;(deltah/sigmah*deltah)-(betah/muh - lambdah)*(betai*N*I)* &#xA;(delta/sigmah*deltah));&#xA;&#xA;N = -k/lambdai((-lamdai+(lambdai/k))*P/-betai*I + lambdai - (2k/lambdai*P)- &#xA;mui-mum);&#xA;&#xA;P = betai*N*I/(mui+mum); &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;At where&#xA;S = susceptible&#xA;I = infected&#xA;R = retweeted&#xA;N = non-carrier mosquito&#xA;P = carrier mosquito&lt;/p&gt;&#xA;" OwnerUserId="2518" LastEditorUserId="48" LastEditDate="2018-03-17T10:20:50.740" LastActivityDate="2018-05-23T12:30:10.377" Title="How to verify if the stationary states of a biological model are correct?" Tags="|modelling|matlab|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3910" PostTypeId="1" AcceptedAnswerId="3919" CreationDate="2018-03-16T17:59:39.200" Score="3" ViewCount="33" Body="&lt;p&gt;I scoured and searched the web for some implementation of digisome creation, but I had a hard time finding any good source code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By digisomes, what I mean is images with chromosomal masks/borders that have been simulated. I am seeking this because I am trying to do a pixel level prediction on chromosomes using a deep network. And, I need ground truths with the chromosomes being illuminated. So ideally, I would like a binary image with just the chromosomes being illuminated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How exactly are digisomes generated? Is a single actual chromosome extracted from some metaphase image, then morphed in interesting ways, and finally, all the different morphs are stitched together in one big collage format so that it there are numerous chromosomes all seemingly different? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if someone could point me to some good source code, that'd be helpful.&lt;/p&gt;&#xA;" OwnerUserId="2519" LastEditorUserId="29" LastEditDate="2018-03-19T19:35:38.993" LastActivityDate="2018-03-19T19:35:38.993" Title="Digisome generation" Tags="|database|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3911" PostTypeId="1" AcceptedAnswerId="3912" CreationDate="2018-03-17T12:54:40.727" Score="5" ViewCount="76" Body="&lt;p&gt;I'm a little confused about how the position indices work in the browser. Based on the picture below, is base number 755440 an &lt;code&gt;A&lt;/code&gt; or a &lt;code&gt;G&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general question is how do the indexing guides work? Is the base right under the &lt;code&gt;|&lt;/code&gt; the one corresponding to the index or is it the one just before the &lt;code&gt;|&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/j4i86.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/j4i86.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="122" LastEditorUserId="29" LastEditDate="2018-03-19T19:36:38.840" LastActivityDate="2018-03-19T19:36:38.840" Title="How are positions visually indicated in the UCSC Genome Browser?" Tags="|ucsc|genome-browser|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3912" PostTypeId="2" ParentId="3911" CreationDate="2018-03-17T13:54:59.637" Score="5" Body="&lt;p&gt;It's an A. The number refers to the position just before the &lt;code&gt;|&lt;/code&gt;. This is clearer on different zoom levels, but I double checked by going to the &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTracks?db=hg18&amp;amp;lastVirtModeType=default&amp;amp;lastVirtModeExtraState=&amp;amp;virtModeType=default&amp;amp;virtMode=0&amp;amp;nonVirtPosition=&amp;amp;position=chr7%3A1-33&amp;amp;hgsid=660570179_mIq0EMY1BriaU5C7xNPQAELZSbzv&quot; rel=&quot;nofollow noreferrer&quot;&gt;very beginning of a chromosome&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JktnE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JktnE.png&quot; alt=&quot;chr7 position 1-15&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see in the image above, the &lt;code&gt;|&lt;/code&gt; is always after the base it refers to. I also confirmed by checking a few random positions using another genome browser which places the positions unambiguously above each base. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more details, see &lt;a href=&quot;http://genome.ucsc.edu/blog/the-ucsc-genome-browser-coordinate-counting-systems/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this UCSC blog post&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-03-19T16:53:54.747" LastActivityDate="2018-03-19T16:53:54.747" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3913" PostTypeId="1" CreationDate="2018-03-18T20:29:34.830" Score="3" ViewCount="244" Body="&lt;p&gt;Aim is to code the theoretical peptic cleavage of protein sequences in Python. The cleavage (cutting) rule for pepsin is: &lt;code&gt;1234^567&lt;/code&gt;. This is a position, &lt;code&gt;^&lt;/code&gt; stands for the cleavage point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are rules that it mustn't have at all and rules where it must have at least one. The rule for mustn't is in position 2, it mustn't be R, K or H. In position 3 and 7, there can't be a P and in position 4, there mustn't be an R. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it follows &quot;the must not&quot; rules, then it will check &quot;the must&quot; rules so that if any of one them is met, it will cleave. So if there is a P in position 1 or 2, or if there's an L, W, Y or F in position 4 or 5 then, it will be cleaved (use newline).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the code I have so far, but it doesn't work yet:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;print('INSERT YOUR PEPTIDE') &#xA;X= input(); &#xA;r=re.findall('(?&amp;lt;=[P])(?&amp;lt;=[P,^R,^K,^H])(?&amp;lt;=([^P])(?&amp;lt;=[L,W,F,Y,^R])(?&amp;lt;=[L,W,Y,F])/D(?&amp;lt;=[^P])',X) &#xA;print(r)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My input is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TTGWMFKGLYQLKSDEARMHNLQRWYVFKVIEFWGINRKDQHDSRYWFSHFCWKLEHGMVPKCFNELLMPPMSTHRSMFNAYSMSFCVQVKEWRGSISRMCDHHQQQCQYWRLMAGGTLI &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I make this work?&lt;/p&gt;&#xA;" OwnerUserId="2523" LastEditorUserId="298" LastEditDate="2018-03-18T22:05:43.590" LastActivityDate="2018-03-19T03:08:22.373" Title="Pepsin digest (cleavage) does not work using RE?" Tags="|python|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3914" PostTypeId="2" ParentId="3913" CreationDate="2018-03-18T22:59:37.990" Score="2" Body="&lt;p&gt;I am really not sure if I've understood your rules correctly, but I think you want this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If all of these are true:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;position 2: not R, K or H&lt;/li&gt;&#xA;&lt;li&gt;position 3: not P&lt;/li&gt;&#xA;&lt;li&gt;position 4: not R&lt;/li&gt;&#xA;&lt;li&gt;position 7: not P&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;And, if at least one of these is true:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;position 1: P&lt;/li&gt;&#xA;&lt;li&gt;position 1: P&lt;/li&gt;&#xA;&lt;li&gt;position 4: L or W or Y or F&lt;/li&gt;&#xA;&lt;li&gt;position 5: L or W or Y or F&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Then cleave.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, then you probably don't want to do it using regular expressions. First because it will be hard, and second because it will be absolutely awful to maintain if you need to come back to it. Instead, try something like this:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;# X = input(); &#xA;X = &quot;TTGWMFKGLYQLKSDEARMHNLQRWYVFKVIEFWGINRKDQHDSRYWFSHFCWKLEHGMVPKCFNELLMPPMSTHRSMFNAYSMSFCVQVKEWRGSISRMCDHHQQQCQYWRLMAGGTLI&quot;&#xA;seq = list(X)&#xA;for i, res in enumerate(seq):&#xA;    if i &amp;gt; len(seq) - 7:&#xA;        break&#xA;    if (&#xA;            seq[i+1] in ('R','K','H') or&#xA;            seq[i+2] == 'P' or&#xA;            seq[i+3] == 'R' or&#xA;            seq[i+6] == 'P'&#xA;    ):&#xA;        continue&#xA;    if res == 'P':&#xA;        start = i&#xA;    elif seq[i+1] == 'P':&#xA;        start = i+1&#xA;    else:&#xA;        continue&#xA;    if (&#xA;            seq[i+4] in ('L', 'W', 'Y', 'F') or&#xA;            seq[i+5] in ('L', 'W', 'Y', 'F')&#xA;    ):&#xA;        print(&quot;%s\n%s\n%s&quot; %&#xA;            (&quot;&quot;.join(seq[0:start]),&#xA;             &quot;&quot;.join(seq[start:start+7]),&#xA;             &quot;&quot;.join(seq[start+7:])&#xA;            )&#xA;        )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will produce the following output when run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ foo.py&#xA;TTGWMFKGLYQLKSDEARMHNLQRWYVFKVIEFWGINRKDQHDSRYWFSHFCWKLEHGMV&#xA;PKCFNEL&#xA;LMPPMSTHRSMFNAYSMSFCVQVKEWRGSISRMCDHHQQQCQYWRLMAGGTLI&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If you insist on using regular expressions, you could separate it into two expressions and use something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;X = &quot;TTGWMFKGLYQLKSDEARMHNLQRWYVFKVIEFWGINRKDQHDSRYWFSHFCWKLEHGMVPKCFNELLMPPMSTHRSMFNAYSMSFCVQVKEWRGSISRMCDHHQQQCQYWRLMAGGTLI&quot;&#xA;re1='P[^RKH][^P][^R][LWYF].[^P]'&#xA;re2='P[^P][^R][LWYF].[^P]'&#xA;r=re.findall('(?:P[^RKH][^P][^R][LWYF].[^P])|(?:P[^P][^R][LWYF].[^P])',X) &#xA;print(r)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Before you do, however, please consider the wise words of &lt;a href=&quot;https://www.xkcd.com/1171/&quot; rel=&quot;nofollow noreferrer&quot;&gt;XKCD 208&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lIrt1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lIrt1.png&quot; alt=&quot;XKCD 1171&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-03-18T23:22:13.213" LastActivityDate="2018-03-18T23:22:13.213" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3915" PostTypeId="2" ParentId="3913" CreationDate="2018-03-19T03:08:22.373" Score="2" Body="&lt;p&gt;Ok I think I almost solve this problem so I still using RE &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;myprotein = &quot;TTGWMFKGLYQLKSDEARMHNLQRWYVFKVIEFWGINRKDQHDSRYWFSHFCWKLEHGMVPKCFNELLMPPMSTHRSMFNAYSMSFCVQVKEWRGSISRMCDHHQQQCQYWRLMAGGTLI&quot;&#xA;my_protein = []&#xA;for protein in myprotein:&#xA;    myprotein = protein.rstrip('\n')&#xA;    my_protein.append(myprotein)&#xA;    my_pro = (''.join(my_protein))&#xA;&#xA;   #cleaves sequence    &#xA;peptides = re.sub(r'(?&amp;lt;=[^HRK][^P][^R])(?=[FLWY][^P])|(?&amp;lt;=[^HRK][^P][FLWY])(?=.[^P])','\n', my_pro)&#xA;&#xA;print (peptides)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and it produce to &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TTG&#xA;W&#xA;M&#xA;F&#xA;KG&#xA;LY&#xA;Q&#xA;L&#xA;KSDEARMHN&#xA;LQRW&#xA;YV&#xA;F&#xA;KVIE&#xA;F&#xA;W&#xA;GINRKDQHDSRY&#xA;WF&#xA;SH&#xA;F&#xA;CW&#xA;K&#xA;L&#xA;EHGMVPKC&#xA;FNE&#xA;L&#xA;LMPPMSTHRSMF&#xA;NA&#xA;Y&#xA;SMS&#xA;F&#xA;CVQVKE&#xA;WRGSISRMCDHHQQQCQ&#xA;Y&#xA;W&#xA;RL&#xA;MAGGT&#xA;LI&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but the true answer should be like (checking from &lt;a href=&quot;https://web.expasy.org/peptide_cutter/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://web.expasy.org/peptide_cutter/&lt;/a&gt; )&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TTG&#xA;W&#xA;M&#xA;F&#xA;KG&#xA;LY&#xA;Q&#xA;L&#xA;KSDEARMHN&#xA;LQRW&#xA;YV&#xA;F&#xA;KVIE&#xA;F&#xA;W&#xA;GINRKDQHDSRY&#xA;WF&#xA;SH&#xA;F&#xA;CW&#xA;K&#xA;L&#xA;EHGMVPKC&#xA;FNE&#xA;L&#xA;LMPPMSTHRSMF&#xA;NA&#xA;Y&#xA;SMS&#xA;F&#xA;CVQVKE&#xA;WRGSISRMCDHHQQQCQ&#xA;Y&#xA;W&#xA;RL&#xA;MAGGT&#xA;L&#xA;I &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;so how can I make this work?&lt;/p&gt;&#xA;" OwnerUserId="2523" LastActivityDate="2018-03-19T03:08:22.373" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3917" PostTypeId="1" CreationDate="2018-03-19T09:42:14.063" Score="1" ViewCount="760" Body="&lt;p&gt;I have a matrix object in R of 1500 rows and 20 columns. Based on my own algorithm of analysis, I know there are roughly 7 clusters. In my matrix object, each row is one data point, and is labeled to which cluster it belongs by its row name. Thus, there are duplicated row names. I wanted to see if PCA would give me the approximate same clustering of my data.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Segment1 Segment2 Segment3 Segment4 Segment5 Segment6 Segment7 Segment8 Segment9 Segment10 Segment11 Segment12 Segment13 Segment14 Segment15 Segment16 Segment17 Segment18 Segment19 Segment20&#xA;B  38.40000 41.75200 44.38400 44.18400 45.37600 37.49600 41.36800 33.93600 38.00800  42.51200 46.49600  40.48000  45.40800  46.32800  43.78400  39.88800  38.84000  40.56800  42.03200 38.89185&#xA;C  45.53846 50.08462 39.91538 36.95385 34.96154 39.74615 38.01538 35.75385 35.54615  36.69231 35.20769  38.05385  39.29231  37.96923  37.30000  36.86923  39.19231  38.81538  43.69231 38.06400 &#xA;C  46.05176 41.69412 38.80000 37.75529 39.67529 39.07765 39.17647 38.24941 39.58588  38.63529 38.30588  41.87765  38.97412  40.13647  42.27294  38.24471  35.41647  40.80000  38.07059 42.11294&#xA;B  44.20000 43.42857 44.80000 35.20000 35.91429 37.82857 51.45714 44.68571 46.68571  48.74286 41.25091  39.45455  38.17091  40.70182  40.39273  41.28727  40.63636  41.50909  41.68364 41.29455&#xA;E1 45.06909 41.09818 40.02909 42.50182 42.34909 39.84727 41.42909 40.47273 40.28000  40.51636 41.25091  39.45455  38.17091  40.70182  40.39273  41.28727  40.63636  41.50909  41.68364 41.29455&#xA;E3 40.87407 39.27704 44.13630 43.25037 35.86667 37.30667 38.76148 40.74667 38.93333  43.16148 37.47259  37.73630  38.34370  39.00148  36.96889  37.76593  39.14667  37.92593  37.62963 38.89185&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(The first column are the row names, either being A,B,C,D,E1,E2 or E3 according to my clustering)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used various packages and functions like &lt;strong&gt;pcomp()&lt;/strong&gt;, &lt;strong&gt;princomp()&lt;/strong&gt; from the &lt;strong&gt;stats&lt;/strong&gt; package, and &lt;strong&gt;PCA()&lt;/strong&gt; from the &lt;strong&gt;FactoMineR&lt;/strong&gt; package. If I wanted to do hierarchical clustering with the &lt;strong&gt;HCPC()&lt;/strong&gt; function from the same package, it was not possible because of the duplicated row names. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fviz_pca_ind(pca1, geom.ind = &quot;point&quot;, axes = c(1,3) , col.ind = new$Class, palette = c(&quot;#66C2A5&quot; ,&quot;#FC8D62&quot; ,&quot;#8DA0CB&quot;, &quot;#E78AC3&quot; ,&quot;#A6D854&quot;, &quot;#FFD92F&quot;, &quot;#E5C494&quot;), addEllipses = T, legend.title=&quot;Classes&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I got the individual datapoints on my two PC axes with ellipses around them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But for clustering: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PCA_FR &amp;lt;- PCA(mat_data, ncp = 4, graph = FALSE)&#xA;Hier_PCA&amp;lt;- HCPC(PCA_FR, graph = F)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I couldn't compute Hier_PCA because of duplicated row names. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; Hier_PCA&amp;lt;- HCPC(PCA_FR, graph = F)&#xA;Error in `contrasts&amp;lt;-`(`*tmp*`, value = contr.funs[1 + isOF[nn]]) : &#xA;contrasts can be applied only to factors with 2 or more levels&#xA;In addition: Warning messages:&#xA;1: In data.row.names(row.names, rowsi, i) :&#xA;some row.names duplicated:  3,4,7,8,10,11,12,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52,53,54,55,56,57,58,59,60,61,62,63,64,65,66,67,68,69,70,71,72,73,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I best cluster my data using the first three PCs, and visualize the 1500 datapoints, but labeled only in 7 ways? &lt;/p&gt;&#xA;" OwnerUserId="2094" LastEditorUserId="2094" LastEditDate="2018-03-19T10:13:49.163" LastActivityDate="2018-03-19T20:30:27.513" Title="Hierarchial PCA Clustering with duplicated row names" Tags="|r|pca|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3919" PostTypeId="2" ParentId="3910" CreationDate="2018-03-19T17:09:14.947" Score="2" Body="&lt;p&gt;I found a very nice &lt;a href=&quot;https://github.com/jeanpat/DeepFISH&quot; rel=&quot;nofollow noreferrer&quot;&gt;github repo&lt;/a&gt; that simulates chromosomes and their labels for overlapping chromosomes - and with a little bit of tweaking, it can be amended for a variety of different simulations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This was also used in a Kaggle post; so right up the alley for deep learning. &lt;/p&gt;&#xA;" OwnerUserId="2519" LastActivityDate="2018-03-19T17:09:14.947" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3920" PostTypeId="2" ParentId="3917" CreationDate="2018-03-19T20:30:27.513" Score="2" Body="&lt;p&gt;Try to make your rownames unique first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you have this matrix with duplicate rownames:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;m &amp;lt;- replicate(20, rnorm(6)) &#xA;rownames(m) &amp;lt;- c(&quot;B&quot;, &quot;C&quot;, &quot;C&quot;, &quot;B&quot;, &quot;E1&quot;, &quot;E3&quot;)&#xA;m&#xA;         [,1]        [,2]      [,3]      [,4]       [,5]        [,6]       [,7]&#xA;B   2.4071997  1.30040528 1.0765500 0.2005440  1.1732384  2.34462132  0.3702172&#xA;C  -0.2683534 -0.74607893 0.1611073 0.3145887  0.8208741  1.26575765  0.4205310&#xA;C   0.1997264  0.03318396 0.1967151 0.7785350  0.9382146 -0.32475195 -0.9832998&#xA;B  -1.2726538  0.22519992 1.1556121 0.7161238  0.5601368  1.02900724 -0.2726709&#xA;E1  0.8062722 -1.80899128 0.3936155 0.5724118 -0.8163454  0.04306317 -0.4074828&#xA;E3  0.3320706  0.21218846 0.9210126 0.9151837  0.4588621 -1.27784583  1.3292399&#xA;         [,8]       [,9]       [,10]      [,11]       [,12]       [,13]&#xA;B   0.2305559  0.2964808 -0.01035426 -2.6062776 -0.03143381 -1.48828554&#xA;C   0.1622548 -0.6244459 -0.66571655 -0.3529994  1.07540375 -1.58949697&#xA;C   0.5984752 -0.6423845 -1.00001700  0.2212160  0.05499773  1.36182121&#xA;B  -0.5594747 -0.3570748  1.44098577 -0.4939490 -0.93386684  0.54919399&#xA;E1  1.6378047  0.8702920  3.06641490  0.1948452  0.36339348 -1.04694826&#xA;E3 -2.2422997  1.0682864  0.14326146 -0.4513848 -0.25819231  0.01364221&#xA;        [,14]      [,15]      [,16]      [,17]       [,18]      [,19]&#xA;B  -2.0410367  2.4858879  1.0564085 -1.7353230 -0.63024160 -1.2642884&#xA;C  -1.2216387  0.5533529 -0.2757584 -0.2119105 -0.68899706  1.0587382&#xA;C   0.6401224 -1.5998479  1.4072007 -1.5062900 -0.60161624  0.9781313&#xA;B   0.4309208  1.4310142 -0.1702066 -0.7195977  0.37466185  0.2605195&#xA;E1  1.2538412  0.3413762 -1.9346107  0.7314531 -0.03105008  0.7341104&#xA;E3 -0.3146694 -1.3028086  0.5043235 -0.2568268  0.08336845  1.3097557&#xA;        [,20]&#xA;B  -1.1245496&#xA;C  -0.4552617&#xA;C  -1.5498408&#xA;B   0.1261818&#xA;E1 -0.1180164&#xA;E3  2.1035368&#xA;&#xA;# Make them unique with make.names function&#xA;&#xA;rownames(m) &amp;lt;- make.names(rownames(m), unique=T)&#xA;&#xA;m&#xA;          [,1]        [,2]      [,3]      [,4]       [,5]        [,6]&#xA;B    2.4071997  1.30040528 1.0765500 0.2005440  1.1732384  2.34462132&#xA;C   -0.2683534 -0.74607893 0.1611073 0.3145887  0.8208741  1.26575765&#xA;C.1  0.1997264  0.03318396 0.1967151 0.7785350  0.9382146 -0.32475195&#xA;B.1 -1.2726538  0.22519992 1.1556121 0.7161238  0.5601368  1.02900724&#xA;E1   0.8062722 -1.80899128 0.3936155 0.5724118 -0.8163454  0.04306317&#xA;E3   0.3320706  0.21218846 0.9210126 0.9151837  0.4588621 -1.27784583&#xA;          [,7]       [,8]       [,9]       [,10]      [,11]       [,12]&#xA;B    0.3702172  0.2305559  0.2964808 -0.01035426 -2.6062776 -0.03143381&#xA;C    0.4205310  0.1622548 -0.6244459 -0.66571655 -0.3529994  1.07540375&#xA;C.1 -0.9832998  0.5984752 -0.6423845 -1.00001700  0.2212160  0.05499773&#xA;B.1 -0.2726709 -0.5594747 -0.3570748  1.44098577 -0.4939490 -0.93386684&#xA;E1  -0.4074828  1.6378047  0.8702920  3.06641490  0.1948452  0.36339348&#xA;E3   1.3292399 -2.2422997  1.0682864  0.14326146 -0.4513848 -0.25819231&#xA;          [,13]      [,14]      [,15]      [,16]      [,17]       [,18]&#xA;B   -1.48828554 -2.0410367  2.4858879  1.0564085 -1.7353230 -0.63024160&#xA;C   -1.58949697 -1.2216387  0.5533529 -0.2757584 -0.2119105 -0.68899706&#xA;C.1  1.36182121  0.6401224 -1.5998479  1.4072007 -1.5062900 -0.60161624&#xA;B.1  0.54919399  0.4309208  1.4310142 -0.1702066 -0.7195977  0.37466185&#xA;E1  -1.04694826  1.2538412  0.3413762 -1.9346107  0.7314531 -0.03105008&#xA;E3   0.01364221 -0.3146694 -1.3028086  0.5043235 -0.2568268  0.08336845&#xA;         [,19]      [,20]&#xA;B   -1.2642884 -1.1245496&#xA;C    1.0587382 -0.4552617&#xA;C.1  0.9781313 -1.5498408&#xA;B.1  0.2605195  0.1261818&#xA;E1   0.7341104 -0.1180164&#xA;E3   1.3097557  2.1035368&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastActivityDate="2018-03-19T20:30:27.513" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3921" PostTypeId="1" CreationDate="2018-03-20T00:26:56.960" Score="7" ViewCount="747" Body="&lt;p&gt;I recently performed my first MinION run, and now I'm trying to analyze the data. Being pretty new to the bioinformatics field, I was hoping some of you could help me out. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a bit of background, I'm trying to define transcript isoforms, and down the line, I'm hoping to quantify isoform abundances. As my first attempt, I used a 1D kit that first performs RT starting at the polyA tail and then template switching to enrich for full length transcripts. To initiate RT, a poly-dT primer is added and after template switching a second primer is added. Using those added primer sequences, I then performed PCR to amplify the full-length cDNAs. This seemed to work well, so I went ahead and sequenced the library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, on to my problem. I have aligned my fastq file to my genome with both bwa-mem and minimap2. In both cases, I get alignment of reads to the genome where I expect transcription to be occurring, but half of the reads are in the sense direction and half are in the antisense direction. This is presumably due to the fact that the cDNA is double stranded and either strand can be recruited to pore to be sequenced. However, because I'm interested in mapping transcript isoforms which could include anti-sense transcripts, this isn't exactly what I hoped for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to figure out a way to extract only the reads that correspond to the actual mRNA transcript's sequence and not the reverse complement. Does anyone have ideas as far as programs that would help with this?&lt;br&gt;&#xA;I'm thinking that I should be able to extract out only the molecules that have the polyA or have the strand switching primer sequence or something like that, but I'm not sure how to properly implement this.&lt;/p&gt;&#xA;" OwnerUserId="2528" LastEditorUserId="48" LastEditDate="2018-03-20T08:10:27.753" LastActivityDate="2018-12-24T08:58:25.823" Title="Extracting strand specific reads from MinION cDNA-PCR protocol" Tags="|sequence-alignment|nanopore|minion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3922" PostTypeId="2" ParentId="3921" CreationDate="2018-03-20T04:20:02.017" Score="3" Body="&lt;p&gt;This has been set up as a bioinformatics protocol on &lt;a href=&quot;https://www.protocols.io/view/stranded-mapping-from-long-reads-n8cdhsw&quot; rel=&quot;nofollow noreferrer&quot;&gt;protocols.io&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been using LAST to identify the adapter orientation relative to the genome, and then using that information to split BAM files up and recombine them to create two strand-specific files that are displayable in a genome browser.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have written &lt;a href=&quot;https://github.com/gringer/bioinfscripts/blob/master/maf_bcsplit.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;my own script&lt;/a&gt; to process LAST results into a CSV format, which makes it easier to do line-by-line data filtering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note:&lt;/em&gt; these scripts have been slightly modified from scripts that I have run. Consider them demonstrative: pay attention to the words rather than the script.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Read correction&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I prefer starting off my data analysis with a read correction with Canu (ideally v1.7 now that it's out, because that attempts correction of &lt;em&gt;all&lt;/em&gt; reads, but here I use canu v1.6). I use &lt;code&gt;minimap&lt;/code&gt; as the mapper to speed this up. The &lt;code&gt;genomeSize&lt;/code&gt; parameter should be approximately a tenth to a fortieth of the number of bases in your dataset to make sure that no sequences are excluded (bigger is better, as long as Canu doesn't freak out about memory consumption):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;~/install/canu/canu-1.6/Linux-amd64/bin/canu overlapper=minimap \&#xA;  genomeSize=100M minReadLength=100 minOverlapLength=30 -correct \&#xA;  -p 4T1_BC06 -d 4T1_BC06 -nanopore-raw \&#xA;  workspace/pass/barcode06/fastq_runid_*.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This creates a file &lt;code&gt;4T1_BC06/4T1_BC06.correctedReads.fasta.gz&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Chimeric read filtering&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The next step I carry out is a basic read-level QC to exclude &lt;a href=&quot;https://f1000research.com/articles/6-631/v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;chimeric reads&lt;/a&gt;. &lt;a href=&quot;https://github.com/rrwick/Porechop&quot; rel=&quot;nofollow noreferrer&quot;&gt;Porechop&lt;/a&gt; can be used for this, although that removes adapters by default, which is not particularly useful in this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use LAST to search for adapter sequences within the corrected reads, pass it through my conversion script, and extract out duplicated mappings (i.e. where the same read/adapter pair appears more than once in the mapping results):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;lastal -P 10 ONT_barcodes_adapters.fa &amp;lt;(zcat 4T1_BC06/4T1_BC06.correctedReads.fasta.gz) | \&#xA;  ~/scripts/maf_bcsplit.pl | awk -F',' '{print $1,$2}' | sort | \&#xA;  uniq -d | awk '{print $1}' | uniq &amp;gt; reads_with_duplicated_adapters.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I use this file to filter out chimeric reads from the corrected dataset using another &lt;a href=&quot;https://github.com/gringer/bioinfscripts/blob/master/fastx-fetch.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;fastq filtering script&lt;/a&gt; I've created:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;~/scripts/fastx-fetch.pl -v -i reads_with_duplicated_adapters.txt 4T1_BC06/4T1_BC06.correctedReads.fasta.gz | \&#xA;  gzip &amp;gt; 4T1_BC06.correctedReads.uniqueOnly.fasta.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;Adapter filtering&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Along roughly similar lines to the chimeric read filtering, I then look for the strand switch and VNT adapters in the sequences. For a forward-orientated query, I expect the strand switch primer to be in the forward direction, and the VNP primer to be in the reverse direction:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;lastal -P 10 ONT_barcodes_adapters.fa &amp;lt;(zcat 4T1_BC06.correctedReads.uniqueOnly.fasta.gz) | \&#xA;  ~/scripts/maf_bcsplit.pl | grep -e 'ONT_SSP,+' -e 'ONT_VNP,-' | \&#xA;  awk -F',' '{print $1}' | sort | uniq &amp;gt; fwdQry_seqs_BC06.txt&#xA;lastal -P 10 barcodes_primerSeqs.fa &amp;lt;(zcat 4T1_BC06.correctedReads.uniqueOnly.fasta.gz) | \&#xA;  ~/scripts/maf_bcsplit.pl | grep -e 'ONT_SSP,-' -e 'ONT_VNP,+' | \&#xA;  awk -F',' '{print $1}' | sort | uniq &amp;gt; revQry_seqs_BC06.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then more filtering to split the reads up into forward and reverse [genome-relative] subsets:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;~/scripts/fastx-fetch.pl -i fwdQry_seqs_BC06.txt 4T1_BC06.correctedReads.uniqueOnly.fasta.gz | \&#xA;  gzip &amp;gt; fwd_4T1_BC06.correctedReads.uniqueOnly.fasta.gz&#xA;~/scripts/fastx-fetch.pl -i revQry_seqs_BC06.txt 4T1_BC06.correctedReads.uniqueOnly.fasta.gz | \&#xA;  gzip &amp;gt; rev_4T1_BC06.correctedReads.uniqueOnly.fasta.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;Mapping&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Now that the reads have been oriented, the mapping can be done:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;~/install/minimap2/minimap2 -t 10 -a mmus_ucsc_all_cdna.idx -x splice \&#xA;  fwd_4T1_BC06.correctedReads.uniqueOnly.fasta.gz \&#xA;  &amp;gt; fwd_4T1_BC06.CU_vs_mmus.bam&#xA;~/install/minimap2/minimap2 -t 10 -a mmus_ucsc_all_cdna.idx -x splice \&#xA;  rev_4T1_BC06.correctedReads.uniqueOnly.fasta.gz \&#xA;  &amp;gt; rev_4T1_BC06.CU_vs_mmus.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;Splitting and recombining&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;The mapped files are then split based on their mapping direction using &lt;code&gt;samtools view&lt;/code&gt; flag filtering to exclude or include reverse-mapped reads:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -F 0x10 -b fwd_4T1_BC06.CU_vs_mmus.bam &amp;gt; fwd_fwd_4T1_BC06.CU_vs_mmus.bam&#xA;samtools view -f 0x10 -b fwd_4T1_BC06.CU_vs_mmus.bam &amp;gt; rev_fwd_4T1_BC06.CU_vs_mmus.bam&#xA;samtools view -F 0x10 -b rev_4T1_BC06.CU_vs_mmus.bam &amp;gt; fwd_rev_4T1_BC06.CU_vs_mmus.bam&#xA;samtools view -f 0x10 -b rev_4T1_BC06.CU_vs_mmus.bam &amp;gt; rev_rev_4T1_BC06.CU_vs_mmus.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And finally the files are recombined to identify forward and reverse-encoded transcripts via &lt;code&gt;samtools merge&lt;/code&gt;. I'm calling the &quot;pos&quot; strand the one that is encoded in the same direction as the genome, and the &quot;neg&quot; strand the reverse-complement direction:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools merge pos_4T1_BC06.CU_vs_mmus.bam \&#xA;  rev_fwd_4T1_BC06.CU_vs_mmus.bam fwd_rev_4T1_BC06.CU_vs_mmus.bam&#xA;samtools merge neg_4T1_BC06.CU_vs_mmus.bam \&#xA;  fwd_fwd_4T1_BC06.CU_vs_mmus.bam rev_rev_4T1_BC06.CU_vs_mmus.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Alternatively, it's possible to set the BAM flags for read 1 and read 2 for the reverse and forward-encoded adapters respectively, which allows everything to then be combined into a single BAM file (and treated in the same way as a strand-specific Illumina BAM file).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this has worked properly, then mapping human or mouse to the mitochondrial genome should show most expression appearing on the positive strand, with a small scattering of negative-strand expression, a bit like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YU56W.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YU56W.png&quot; alt=&quot;mtDNA transcript expression (nanopore)&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a bit of a background on how/why this works, see &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/3792/stranded-vs-unstranded-library-preparation-protocols-in-rnaseq/3795#3795&quot;&gt;this answer&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-04-04T05:01:57.430" LastActivityDate="2018-04-04T05:01:57.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3923" PostTypeId="1" AcceptedAnswerId="3924" CreationDate="2018-03-20T09:02:59.697" Score="6" ViewCount="742" Body="&lt;p&gt;I have a huge file (20 &lt;code&gt;GB&lt;/code&gt;) which has a range of genomic locations, and for each location there is an identifier(&lt;code&gt;4th column&lt;/code&gt;), which is sometimes the same.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;file1.txt&#xA;&#xA;chr1  10  20  ABC&#xA;chr1  13  20  ABC&#xA;chr1  14  21  ABC&#xA;chr1  22  27  ABC&#xA;chr1  29  37  ABC&#xA;chr2  15  21  JJJ&#xA;chr2  21  31  JJJ&#xA;chr2  23  27  JJJ&#xA;chr2  35  56  JJJ&#xA;chr2  25  26  MMM&#xA;chr3  32  42  MMM&#xA;chr3  45  76  MMM&#xA;chr3  88  101 MMM&#xA;chr3  101 105 MMM&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am trying to merge the ranges of column &lt;code&gt;2&lt;/code&gt; and &lt;code&gt;3&lt;/code&gt;, if they are in the same chromosome and having the same identifier ( having the &lt;code&gt;1st&lt;/code&gt; and &lt;code&gt;4th&lt;/code&gt; columns the same).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to use &lt;code&gt;bedtools&lt;/code&gt;, as shown in &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2265/merging-bed-records-based-on-name&quot; title=&quot;this&quot;&gt;this&lt;/a&gt; post, which looks very similar but not the same with mine, as they are looking for different outputs ( range vs merge )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the answers in &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2265/merging-bed-records-based-on-name&quot;&gt;that&lt;/a&gt; post was with using &lt;code&gt;groupby&lt;/code&gt; in &lt;a href=&quot;https://github.com/arq5x/bedtools2&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools2&lt;/a&gt;,source code from &lt;code&gt;github&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have applied the same as the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./groupBy -i ../file1.txt -g 1,4 -c 2,3 -o min,max | awk -v OFS='\t' '{print $1, $3, $4, $2}'&#xA;&#xA;chr1    10      37      ABC&#xA;chr2    15      56      JJJ&#xA;chr2    25      26      MMM&#xA;chr3    32      105     MMM&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But it only connects the ranges of identifiers.. It groups by, and does not merge. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And, if I use &lt;code&gt;bedtools v2.26 merge&lt;/code&gt; function :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;sort -k1,1 -k2,2n file1.txt &amp;gt; file2.txt&#xA;&amp;gt;cat file2.txt&#xA;chr1    10      20      ABC&#xA;chr1    13      20      ABC&#xA;chr1    14      21      ABC&#xA;chr1    22      27      ABC&#xA;chr1    29      37      ABC&#xA;chr2    15      21      JJJ&#xA;chr2    21      31      JJJ&#xA;chr2    23      27      JJJ&#xA;chr2    25      26      MMM&#xA;chr2    35      56      JJJ&#xA;chr3    32      42      MMM&#xA;chr3    45      76      MMM&#xA;chr3    88      101     MMM&#xA;chr3    101     105     MMM&#xA;&#xA;&#xA;bedtools merge -i file2.txt &#xA;chr1    10      21&#xA;chr1    22      27&#xA;chr1    29      37&#xA;chr2    15      31&#xA;chr2    35      56&#xA;chr3    32      42&#xA;chr3    45      76&#xA;chr3    88      105&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which, merges without seeing the identifiers as it also gives me this output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want is to &lt;code&gt;merge&lt;/code&gt; the ranges if they &lt;code&gt;overlap&lt;/code&gt;, if having the &lt;code&gt;same 1st and 4th column&lt;/code&gt; as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;expected_output.txt&#xA;&#xA;&#xA;chr1  10  21  ABC&#xA;chr1  22  27  ABC&#xA;chr1  29  37  ABC&#xA;chr2  15  31  JJJ&#xA;chr2  35  56  JJJ&#xA;chr2  25  26  MMM&#xA;chr3  32  42  MMM&#xA;chr3  45  76  MMM&#xA;chr3  88  105 MMM&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I only have access to 15GB of RAM, so I can't store the entire file in memory.&lt;/p&gt;&#xA;" OwnerUserId="2530" LastEditorUserId="2530" LastEditDate="2018-03-21T12:28:29.500" LastActivityDate="2018-03-21T12:28:29.500" Title="Merging regions according to their identifier" Tags="|bedtools|merge|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3924" PostTypeId="2" ParentId="3923" CreationDate="2018-03-20T09:42:11.720" Score="3" Body="&lt;p&gt;The following python script will do what you want and should be relatively memory efficient. It processes a single chromosome at a time, so either sort the input or ensure that at least entries in a group are next to each other.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python&#xA;import sys&#xA;&#xA;# Process a sorted list of tuples (e.g., [(0, 10), (10, 20), ...])&#xA;# Overlapping items will be merged and a list of the merged regions is&#xA;# returned (e.g., [(0, 20), ...])&#xA;def merge(values):&#xA;    start, end = values[0]&#xA;    o = []&#xA;    for v in values[1:]:&#xA;        if v[0] &amp;lt;= end:&#xA;            if v[1] &amp;gt; end:&#xA;                end = v[1]&#xA;        else:&#xA;            o.append((start, end))&#xA;            start, end = v&#xA;    o.append((start, end))&#xA;    return o&#xA;&#xA;&#xA;# Process a single chromosome, one group at a time&#xA;def processChrom(d, chrom):&#xA;    for k, v in d.items():&#xA;        v.sort()&#xA;        tuples = merge(v)&#xA;        for t in tuples:&#xA;            print(&quot;{}\t{}\t{}\t{}&quot;.format(chrom, t[0], t[1], k))&#xA;&#xA;&#xA;def main():&#xA;    currentChrom = None&#xA;    d = dict()&#xA;    for line in sys.stdin:&#xA;        cols = line.split()&#xA;        if cols[0] == currentChrom:&#xA;            if cols[3] not in d:&#xA;                d[cols[3]] = []&#xA;            d[cols[3]].append((int(cols[1]), int(cols[2])))&#xA;        else:&#xA;            if currentChrom:&#xA;                processChrom(d, currentChrom)&#xA;            d = dict()&#xA;            d[cols[3]] = []&#xA;            d[cols[3]].append((int(cols[1]), int(cols[2])))&#xA;            currentChrom = cols[0]&#xA;&#xA;    if currentChrom:&#xA;        processChrom(d, currentChrom)&#xA;&#xA;if __name__ == &quot;__main__&quot;:&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Usage is &lt;code&gt;python foo.py &amp;lt; foo.txt&lt;/code&gt; and the output is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    10  21  ABC&#xA;chr1    22  27  ABC&#xA;chr1    29  37  ABC&#xA;chr2    25  26  MMM&#xA;chr2    15  31  JJJ&#xA;chr2    35  56  JJJ&#xA;chr3    32  42  MMM&#xA;chr3    45  76  MMM&#xA;chr3    88  105 MMM&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You'll notice that this isn't necessarily sorted, so you may want to pass the output through &lt;code&gt;bedtools sort&lt;/code&gt; or &lt;code&gt;sort&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-03-20T09:42:11.720" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3925" PostTypeId="2" ParentId="3923" CreationDate="2018-03-20T10:38:19.300" Score="6" Body="&lt;p&gt;If I understand correctly, you could create dummy chromosomes made by merging chromosome and identifier, merge with bedtools, split back chromosomes and identifiers. E.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -v OFS='\t' '{print $1&quot;_&quot;$4, $2, $3}' file1.txt \&#xA;| mergeBed \&#xA;| sed 's/_/\t/' \&#xA;| awk -v OFS='\t' '{print $1, $3, $4, $2}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Assuming &quot;_&quot; is not part of chromosome or identifier name, adjust accordingly if so).&lt;/p&gt;&#xA;" OwnerUserId="339" LastActivityDate="2018-03-20T10:38:19.300" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3927" PostTypeId="1" CreationDate="2018-03-20T13:58:00.943" Score="3" ViewCount="54" Body="&lt;p&gt;The following is an excerpt from an edge list file from the Gene Regulatory Database, &lt;a href=&quot;https://www.inetbio.org/yeastnet/&quot; rel=&quot;nofollow noreferrer&quot;&gt;YeastNet v3&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;YML100W YMR261C 5.73968189451551&#xA;YDR074W YML100W 5.73143623470086&#xA;YDR074W YMR261C 5.72385204041689&#xA;YML106W YMR271C 5.64698433340441&#xA;YGR240C YMR205C 5.64063955646679&#xA;YDL131W YDL182W 5.63907349642605&#xA;YNL037C YOR136W 5.62782704009976&#xA;YHR208W YJR148W 5.61712512462848&#xA;YAL062W YOR375C 5.59956647235823&#xA;YNL104C YOR108W 5.59046546303091&#xA;YGR132C YGR231C 5.58996626839351&#xA;YBR140C YOL081W 5.58738025271142&#xA;YDR453C YML028W 5.58281825879471&#xA;YNL096C YOR096W 5.58060076040965&#xA;YGR180C YJL026W 5.57785446902287&#xA;YGR056W YLR357W 5.57582611528462&#xA;YBR009C YNL030W 5.57261292043205&#xA;YMR169C YMR170C 5.55866364259678&#xA;YAR071W YBR093C 5.55668852247789&#xA;YER081W YIL074C 5.55252000887001&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What do these rows mean?&lt;/p&gt;&#xA;" OwnerUserId="2533" LastEditorUserId="57" LastEditDate="2018-03-21T09:45:58.213" LastActivityDate="2018-03-21T09:45:58.213" Title="How to read gene regulatory network edge list files?" Tags="|gene|networks|transcriptome-regulation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3928" PostTypeId="2" ParentId="3923" CreationDate="2018-03-20T14:06:23.507" Score="4" Body="&lt;p&gt;Just to add this can be done with &lt;code&gt;bedtools&lt;/code&gt; by processing each identifier separately which may or may not be practical given your large file. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for ID in $(perl -ane 'print $F[3] . &quot;\n&quot;;' file1.txt | sort | uniq);do &#xA;    grep $ID file1.txt | \&#xA;    perl -ane 'chomp;print join(&quot;\t&quot;,@F[0..$#F]) . &quot;\n&quot;;' | \&#xA;    sort -V | \&#xA;    bedtools merge -i stdin -c 4 -o distinct&#xA;done | sort -V&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user1141" LastActivityDate="2018-03-20T14:06:23.507" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3929" PostTypeId="2" ParentId="3927" CreationDate="2018-03-20T14:12:26.913" Score="3" Body="&lt;p&gt;You can read about it on the &lt;a href=&quot;https://www.inetbio.org/yeastnet/&quot; rel=&quot;nofollow noreferrer&quot;&gt;website&lt;/a&gt;, the last column is the Log likelihood scores (LLS) between the genes in the first two columns. An LLS of 0 means not better than random link.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the website they show papers with fuller details about the statistics.&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2018-03-20T14:21:57.003" LastActivityDate="2018-03-20T14:21:57.003" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3930" PostTypeId="2" ParentId="3923" CreationDate="2018-03-20T15:04:13.410" Score="3" Body="&lt;p&gt;Here is a fairly simple (and hopefully readable) native Python solution. It assumes that the bed file is sorted prior to parsing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from itertools import groupby&#xA;&#xA;class BedRecord(object):&#xA;    def __init__(self, chr, start, end, feature):&#xA;        self.chr = chr&#xA;        self.start = start&#xA;        self.end = end&#xA;        self.feature = feature&#xA;&#xA;def parse_records(in_file):&#xA;    with open(in_file) as f:&#xA;        for line in f:&#xA;            yield BedRecord(*line.strip().split())&#xA;&#xA;def merge_groups(group):&#xA;    output = [next(group)]&#xA;    for record in group:&#xA;        if record.start &amp;lt;= output[-1].end:&#xA;            output[-1].end = max(output[-1].end, record.end)&#xA;        else:&#xA;            output.append(record)&#xA;    return output&#xA;&#xA;def main(in_file):&#xA;    for _, group in groupby(parse_records(in_file), lambda x: (x.chr, x.feature)):&#xA;        for r in merge_groups(group):&#xA;            print('\t'.join([r.chr, r.start, r.end, r.feature]))&#xA;&#xA;if __name__ == '__main__':&#xA;    main('test.bed')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastEditorUserId="104" LastEditDate="2018-03-20T15:12:04.870" LastActivityDate="2018-03-20T15:12:04.870" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3931" PostTypeId="1" CreationDate="2018-03-20T18:08:43.617" Score="9" ViewCount="23595" Body="&lt;p&gt;I have a fasta file like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Id1&#xA;ATCCTT&#xA;&amp;gt;Id2&#xA;ATTTTCCC&#xA;&amp;gt;Id3&#xA;TTTCCCCAAAA&#xA;&amp;gt;Id4&#xA;CCCTTTAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to delete sequences that have the following IDs.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Id2&#xA;Id3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The IDs are in a .txt file, and the text file will be used to match and delete those sequences. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My output should be a &lt;code&gt;fasta&lt;/code&gt; file like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Id1&#xA;ATCCTT&#xA;&amp;gt;Id4&#xA;CCCTTTAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I want it with &lt;code&gt;awk&lt;/code&gt; and/or &lt;code&gt;sed&lt;/code&gt; and/or &lt;code&gt;bash&lt;/code&gt; (no python or perl).&lt;br&gt;&#xA;How can I do it?&lt;/p&gt;&#xA;" OwnerUserId="1421" LastEditorUserId="73" LastEditDate="2023-01-15T20:29:16.397" LastActivityDate="2024-01-05T01:22:45.260" Title="Remove/delete sequences by ID from multifasta" Tags="|fasta|awk|shell|benchmarking|" AnswerCount="10" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3932" PostTypeId="2" ParentId="3931" CreationDate="2018-03-20T18:41:16.493" Score="2" Body="&lt;p&gt;Presume grep is OK if sed, awk are. The -A(n) and -B(n) flags give (n) lines post match. Presuming all your fasta sequence will be one line is a bit dangerous, but it works for your example. First get those IDs to be removed (in rmid.txt), then inverse grep against the initial fasta.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -A1 -f rmid.txt fasta.fa &amp;gt; rmfile.fa&#xA;grep -v -f rmfile.fa fasta.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The real answer is to use a script that defines a delimiter other than newline &quot;\n&quot;, then parse for IDs, so better to use one of the languages you don't want to use... &lt;/p&gt;&#xA;" OwnerDisplayName="user1141" LastEditorDisplayName="user1141" LastEditDate="2018-03-20T19:17:13.720" LastActivityDate="2018-03-20T19:17:13.720" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3933" PostTypeId="1" AcceptedAnswerId="3945" CreationDate="2018-03-20T18:53:31.390" Score="5" ViewCount="89" Body="&lt;p&gt;I have a set of experiments which should result in WGS reads with a high density of indels. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question: Are there certain alignment tools/methods which perform respectively &quot;better&quot; with these reads? I'm not very interested in performance, but rather &quot;better&quot; in terms of dealing with the presence of many deletions/insertions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Options I know of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/lh3/bwa&quot; rel=&quot;nofollow noreferrer&quot;&gt;BWA&lt;/a&gt;  (I think there are special settings whereby I could run bwa&#xA;for this case, e.g. see &lt;a href=&quot;http://blog.liheng.org/2014/12/10/bwa-mem-for-long-error-prone-reads&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/BioInfoTools/BBMap&quot; rel=&quot;nofollow noreferrer&quot;&gt;BBMap&lt;/a&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1770" LastEditorUserId="29" LastEditDate="2018-03-22T10:13:22.503" LastActivityDate="2018-03-22T10:13:22.503" Title="Are there certain alignment methods/tools which perform better with a high density of indels?" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3934" PostTypeId="2" ParentId="3931" CreationDate="2018-03-20T19:02:00.587" Score="3" Body="&lt;p&gt;Using the &lt;code&gt;FastaToTbl&lt;/code&gt; and &lt;code&gt;TblToFasta&lt;/code&gt; scripts I have &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2651/298&quot;&gt;posted before&lt;/a&gt;, you can do:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;FastaToTbl file.fa | grep -vwf ids.txt | TblToFasta &amp;gt; file.2.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-03-23T08:58:29.123" LastActivityDate="2018-03-23T08:58:29.123" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3935" PostTypeId="2" ParentId="3931" CreationDate="2018-03-20T20:03:21.150" Score="6" Body="&lt;p&gt;If you want to learn how to do things with command-line tools, you can linearize the FASTA with &lt;code&gt;awk&lt;/code&gt;, pipe to &lt;code&gt;grep&lt;/code&gt; to filter for items of interest named in &lt;code&gt;patterns.txt&lt;/code&gt;, then pipe to &lt;code&gt;tr&lt;/code&gt; to delinearize:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk '{ if ((NR&amp;gt;1)&amp;amp;&amp;amp;($0~/^&amp;gt;/)) { printf(&quot;\n%s&quot;, $0); } else if (NR==1) { printf(&quot;%s&quot;, $0); } else { printf(&quot;\t%s&quot;, $0); } }' in.fa | grep -Ff patterns.txt - | tr &quot;\t&quot; &quot;\n&quot; &amp;gt; out.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will work on multiline FASTA.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-03-20T21:49:46.863" LastActivityDate="2018-03-20T21:49:46.863" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3936" PostTypeId="1" AcceptedAnswerId="4256" CreationDate="2018-03-20T20:52:16.840" Score="5" ViewCount="432" Body="&lt;p&gt;I am starting to run analysis for differential expression in scRNA-Seq. Which tools are available for this kind of analysis?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can tools for bulk RNA-Seq like &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/DESeq/inst/doc/DESeq.pdf&quot; rel=&quot;noreferrer&quot;&gt;DESeq&lt;/a&gt; be used for scRNA-Seq?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-05-08T19:42:09.730" Title="Which tools for differential expression analysis in scRNA-Seq?" Tags="|rna-seq|scrnaseq|differential-expression|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3937" PostTypeId="1" AcceptedAnswerId="3939" CreationDate="2018-03-21T01:44:16.600" Score="4" ViewCount="126" Body="&lt;p&gt;I have RNA-Seq data from 4 time points (3 hours awake, 9 hours awake, 3 hours asleep, 9 hours asleep).  I'm interested in doing something similar to a LRT where genes are found to be significant if they changed at any point in the time course.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm not sure how to do this considering this is cyclical.  Running a LRT test in DESeq2 gives different results depending on which timepoint I choose as the beginning of the time course since I assume it doesn't calculate the last transition back to being awake (Or asleep depending on your reference).&lt;br&gt;&#xA;Any ideas on an appropriate way to do this? &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time = c(3,9,15,21)&#xA;ddsLRT &amp;lt;- DESeqDataSetFromTximport(txi.rsem.lrt, sampleTable, ~time)  &#xA;ddsLRT&amp;lt;- DESeq(ddsLRT, test=&quot;LRT&quot;, reduced = ~1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2537" LastEditorUserId="48" LastEditDate="2018-03-21T08:30:13.613" LastActivityDate="2019-01-02T12:56:20.943" Title="LRT or LRT-like test on cyclical (Sleep) data" Tags="|rna-seq|r|statistics|deseq2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3938" PostTypeId="2" ParentId="3931" CreationDate="2018-03-21T02:08:59.863" Score="7" Body="&lt;p&gt;Suppose you keep sequence names in &lt;code&gt;ids.txt&lt;/code&gt; and sequences in &lt;code&gt;seq.fa&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN{while((getline&amp;lt;&quot;ids.txt&quot;)&amp;gt;0)l[&quot;&amp;gt;&quot;$1]=1}/^&amp;gt;/{f=!l[$1]}f' seq.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-03-21T02:08:59.863" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3939" PostTypeId="2" ParentId="3937" CreationDate="2018-03-21T07:31:59.637" Score="5" Body="&lt;p&gt;I think the problem is that you have &lt;code&gt;time&lt;/code&gt; as a linear value rather than a factor. While this naively makes sense (after all, you have to go through 3 hours awake before you can get to 9 hours awake), what you end up doing is fitting a single coefficient to each gene of &quot;linearly changes with time&quot;. What you want instead is factor, so you can have a system where things like &lt;code&gt;per&lt;/code&gt; and &lt;code&gt;clk&lt;/code&gt; can cycle:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time = factor(c(3,9,15,21))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The rest is then correct. Note that LRTs are not asking, &quot;is there a change at any time point?&quot;, since there will likely be DE genes according to this that are not DE at any single time point. Rather an LRT such as this is asking, &quot;What are genes that vary significantly over time?&quot; There's a subtle different between those two questions. The former question is actually answered by a bunch of pairwise comparisons (taking the union of the DE genes).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-03-21T07:31:59.637" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3940" PostTypeId="2" ParentId="3931" CreationDate="2018-03-21T12:28:47.583" Score="7" Body="&lt;p&gt;Just today I wrote a script to do exactly this using &lt;code&gt;Biopython&lt;/code&gt;. I also add a warning If any the headers I wanted to filter was not present in the original fasta. So the python script &lt;code&gt;filter_fasta_by_list_of_headers.py&lt;/code&gt; is :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;from Bio import SeqIO&#xA;import sys&#xA;&#xA;ffile = SeqIO.parse(sys.argv[1], &amp;quot;fasta&amp;quot;)&#xA;header_set = set(line.strip() for line in open(sys.argv[2]))&#xA;&#xA;for seq_record in ffile:&#xA;    try:&#xA;        header_set.remove(seq_record.name)&#xA;    except KeyError:&#xA;        print(seq_record.format(&amp;quot;fasta&amp;quot;))&#xA;        continue&#xA;if len(header_set) != 0:&#xA;    print(len(header_set),'of the headers from list were not identified in the input fasta file.', file=sys.stderr)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and usage :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;filter_fasta_by_list_of_headers.py input.fasta list_of_scf_to_filter &amp;gt; filtered.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;P.S. it's quite easy to turn over the script to extract the sequences from the list (just the print line would have to move after line &lt;code&gt;header_set.remove(seq_record.name&lt;/code&gt;)&lt;/p&gt;&#xA;&lt;p&gt;I made a modification to use &lt;code&gt;in&lt;/code&gt; and benchmarked it on normal insect genome assembly - 1Gbp, 300,000 scaffolds and I gave it lists of 5,000 and 100,000 scaffolds to filter. Regardless of the size of the list or algorithm used it took 29-30s. In other words, if LBYL is faster than EAFP, it's on a scale that totally does not matter.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="73" LastEditDate="2024-01-04T21:22:20.490" LastActivityDate="2024-01-04T21:22:20.490" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="3941" PostTypeId="1" AcceptedAnswerId="3942" CreationDate="2018-03-21T15:22:24.297" Score="5" ViewCount="2904" Body="&lt;p&gt;Suppose I have a BAM file indicating where reads in a library have mapped, and a bed file describing a set of genomic regions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to easily get the size distribution of the reads mapping on this set of regions?&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2022-11-08T16:55:17.893" Title="How can I easily get the read size distribution of reads mapping on a certain set of regions?" Tags="|bed|sam|text-processing|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3942" PostTypeId="2" ParentId="3941" CreationDate="2018-03-21T15:22:24.297" Score="4" Body="&lt;p&gt;This can be done rather simply by combining &lt;a href=&quot;https://www.biostars.org/p/98407/#98408&quot; rel=&quot;nofollow noreferrer&quot;&gt;this samtools based answer&lt;/a&gt; and bioawk:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -L regions.bed aligned_reads.bam \&#xA;    | bioawk -c sam '{hist[length($seq)]++} END {for (l in hist) print l, hist[l]}' \&#xA;    | sort -n -k1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;samtools&lt;/code&gt; command extracts the alignements in the region given with the &lt;code&gt;-L&lt;/code&gt; option, in sam format. The &lt;code&gt;-c sam&lt;/code&gt; option of &lt;code&gt;bioawk&lt;/code&gt; makes the &lt;code&gt;$seq&lt;/code&gt; variable (corresponding to the read sequence field) available in the awk command.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final &lt;code&gt;sort&lt;/code&gt; ensures the size classes are ordered numerically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We obtain a two-column &lt;code&gt;size    count&lt;/code&gt; table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Examples with timings:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ time samtools view -L /Genomes/C_elegans/Caenorhabditis_elegans/Ensembl/WBcel235/Annotation/Genes/protein_coding.bed results_Ribo-seq_test_1/bowtie2/mapped_C_elegans/WT_1/18-30_on_C_elegans_sorted.bam | bioawk -c sam '{histo[length($seq)]++} END {for (l in histo) print l, histo[l]}' | sort -n -k1&#xA;18  201&#xA;19  560&#xA;20  1017&#xA;21  2568&#xA;22  5554&#xA;23  11221&#xA;24  27637&#xA;25  38797&#xA;26  17042&#xA;27  9853&#xA;28  15181&#xA;29  5888&#xA;30  528&#xA;&#xA;real    0m0.592s&#xA;user    0m0.816s&#xA;sys     0m0.036s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since sam format is actually simple to parse, one can try to use &lt;code&gt;mawk&lt;/code&gt; instead for slightly higher speed (the sequence being in the 10th field):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -L regions.bed aligned_reads.bam \&#xA;    | mawk '{hist[length($10)]++} END {for (l in hist) print l&quot;\t&quot;hist[l]}' \&#xA;    | sort -n -k1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Contrary to bioawk, the default output field separator is not a tabulation.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ time samtools view -L /Genomes/C_elegans/Caenorhabditis_elegans/Ensembl/WBcel235/Annotation/Genes/protein_coding.bed results_Ribo-seq_test_1/bowtie2/mapped_C_elegans/WT_1/18-30_on_C_elegans_sorted.bam | mawk '{histo[length($10)]++} END {for (l in histo) print l&quot;\t&quot;histo[l]}' | sort -n -k1&#xA;18  201&#xA;19  560&#xA;20  1017&#xA;21  2568&#xA;22  5554&#xA;23  11221&#xA;24  27637&#xA;25  38797&#xA;26  17042&#xA;27  9853&#xA;28  15181&#xA;29  5888&#xA;30  528&#xA;&#xA;real    0m0.362s&#xA;user    0m0.500s&#xA;sys     0m0.004s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2018-03-21T16:24:05.467" LastActivityDate="2018-03-21T16:24:05.467" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3943" PostTypeId="1" AcceptedAnswerId="3944" CreationDate="2018-03-21T16:27:53.443" Score="9" ViewCount="4396" Body="&lt;p&gt;I have done some work in R and would like to try a Python tool.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a good way to &lt;strong&gt;import&lt;/strong&gt; the data (and its annotations etc) &lt;strong&gt;as a Python object?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am particularly interested in converting a &lt;code&gt;Seurat&lt;/code&gt; object into an &lt;a href=&quot;https://scanpy.readthedocs.io/en/latest/api/scanpy.api.AnnData.html#scanpy.api.AnnData&quot; rel=&quot;noreferrer&quot;&gt;AnnData&lt;/a&gt; object. (Either directly or as a Python object which can be converted into an &lt;code&gt;AnnData&lt;/code&gt;.)&lt;/p&gt;&#xA;" OwnerUserId="208" LastEditorUserId="29" LastEditDate="2018-03-22T10:11:47.350" LastActivityDate="2021-10-20T10:37:24.413" Title="Convert R RNA-seq data object to a Python object" Tags="|rna-seq|r|python|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3944" PostTypeId="2" ParentId="3943" CreationDate="2018-03-21T17:45:49.547" Score="10" Body="&lt;p&gt;A simple solution for converting Seurat objects into AnnData, &lt;a href=&quot;https://satijalab.org/seurat/conversion_vignette.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;as described in this vignette&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(reticulate)&#xA;seuratobject_ad &amp;lt;- Convert(from=seuratobject, to=&quot;anndata&quot;, filename=&quot;seuratobject.h5ad&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Alternatively, one can use &lt;a href=&quot;http://loompy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Loom&lt;/a&gt;, &lt;em&gt;&quot;a file format designed to store and work with single-cell RNA-seq data&quot;.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In R, install &lt;a href=&quot;http://satijalab.org/loomR/loomR_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;LoomR&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;devtools::install_github(repo = &quot;mojaveazure/loomR&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Convert from Seurat object to loom:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pfile &amp;lt;- Convert(from = pbmc_small, to = &quot;loom&quot;, filename = &quot;pbmc_small.loom&quot;)&#xA;pfile$close() # close loom objects when finished using them.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then import loom object in Python using &lt;a href=&quot;http://linnarssonlab.org/loompy/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;loompy&lt;/a&gt;, or directly as AnnData:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://scanpy.readthedocs.io/en/latest/api/scanpy.api.read_loom.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;scanpy.api.read_loom&lt;/code&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Alternatively, see &lt;a href=&quot;https://github.com/wesm/feather&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;feather&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/46390461/the-best-way-to-import-r-object-into-python&quot;&gt;Or export&lt;/a&gt; as some text format (csv, json) then import into Python.&lt;/p&gt;&#xA;" OwnerUserId="208" LastEditorUserId="208" LastEditDate="2018-08-29T16:58:47.240" LastActivityDate="2018-08-29T16:58:47.240" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="3945" PostTypeId="2" ParentId="3933" CreationDate="2018-03-21T23:41:06.860" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/lh3/minimap2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Minimap2&lt;/a&gt; attempts to solve a lot of problems that are not properly dealt with by other mapping software. It's written by the same author as BWA, but designed to deal with situations where the reference and query sequences are quite different (e.g. for noisy long reads).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-03-21T23:41:06.860" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3946" PostTypeId="1" CreationDate="2018-03-22T01:09:28.193" Score="3" ViewCount="277" Body="&lt;p&gt;In the NCBI gene database, I can add the expression tracks (circled in picture blow) through 'Tracks' button, but &lt;strong&gt;How I can download the expression data directly, not just look the picture?&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/yVA61.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yVA61.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2547" LastActivityDate="2018-03-22T17:24:41.810" Title="How to download gene expression data from NCBI gene database" Tags="|rna-seq|database|gene|data-download|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3947" PostTypeId="1" AcceptedAnswerId="4027" CreationDate="2018-03-22T11:31:53.393" Score="3" ViewCount="168" Body="&lt;p&gt;My first data frame whose first column i intend to use which is PATIENT_ID&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PATIENT_ID  SEX RACE    FAB AGE BM_BLAST_PERCENTAGE WBC&#xA;TCGA-AB-2805    Male    WHITE   M0  77  67  92&#xA;TCGA-AB-2822    Male    WHITE   M0  65  99  2.9&#xA;TCGA-AB-2831    Male    WHITE   M0  59  61  1.4&#xA;TCGA-AB-2890    Male    NH/C    M0  69  60  6.1&#xA;TCGA-AB-2907    Male    WHITE   M0  69  82  1.6&#xA;TCGA-AB-2913    Male    WHITE   M0  61  95  61.6&#xA;TCGA-AB-2922    Male    WHITE   M0  83  75  77.9&#xA;TCGA-AB-2936    Female  NH/C    M0  60  50  2.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My second data frame where i have the respective patient ID as individual column with expression values &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Hugo_Symbol TCGA-AB-2803    TCGA-AB-2805    TCGA-AB-2806    TCGA-AB-2807    TCGA-AB-2808&#xA;A1BG    0   0   0   0   0&#xA;A1CF    0   0   0   0   0&#xA;A2BP1   71.4095 48.5475 50.8339 45.7498 47.1328&#xA;A2LD1   2.0576  3.3467  5.7288  0.993   5.4988&#xA;A2ML1   467.0782    16.0643 7.0019  343.5948    255.9152&#xA;A2M 11.3169 0.6693  17.823  116.1867    182.2467&#xA;A4GALT  0   2.6774  1.2731  4.9652  0.7855&#xA;A4GNT   0   0   0   0   0&#xA;AAA1    514.4033    532.7979    504.1375    697.1202    443.8335&#xA;AAAS    0   0   0   0   0&#xA;AACSL   190.3292    74.2972 84.6595 559.0864    317.3606&#xA;AACS    0   0   0   1.9861  0&#xA;AADACL2 0   0   0   0   0&#xA;AADACL3 0   0   0   0   0&#xA;AADACL4 1.0288  0   0.6365  0   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What I tried so far transposing the second data frame so that i can get the individual column as my first column ,and subset by using dplyr library but so far im not able to ..any help or suggestion how to do it would be highly appreciable &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to have my expected data frame as such &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PATIENT_ID  Gene B  GeneA   GeneC   GeneA   GeneA &#xA;TCGA-AB-2803-03 0   0   0   92.5926 876.5432&#xA;TCGA-AB-2805-03 0.6693  1.3387  0   92.6305 421.6867&#xA;TCGA-AB-2806-03 0   14.1884 16.3654 119.599 625.0796&#xA;TCGA-AB-2807-03 2.006   9.3247  13.5154 178.4211    537.2393&#xA;TCGA-AB-2808-03 0   42.3016 28.3975 170.2042    653.5742&#xA;TCGA-AB-2810-03 0   1.5668  8.9665  152.0747    1087.5648&#xA;TCGA-AB-2811-03 0   0   6.9556  30.3103 1414.6602&#xA;TCGA-AB-2812-03 0   14.3289 11.9466 120.3591    962.5575&#xA;TCGA-AB-2813-03 0   3.4938  1.3813  51.1883 594.7593&#xA;TCGA-AB-2814-03 0   56.8659 118.7051    87.1817 472.6912&#xA;TCGA-AB-2815-03 0   2.0692  3.6986  104.6575    502.5234&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="285" LastEditorUserId="964" LastEditDate="2018-04-01T23:33:09.203" LastActivityDate="2018-04-09T04:18:06.873" Title="Subset data frame by similar elements from rows and column between two data frames" Tags="|r|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3948" PostTypeId="2" ParentId="3947" CreationDate="2018-03-22T13:13:36.677" Score="4" Body="&lt;p&gt;You can take the PATIENT_ID from the first dataframe and use them to select from your second dataframe. Than transpose, so something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ID_Names &amp;lt;- First_df$PATIENT_ID&#xA;&#xA;rownames(Second_df) &amp;lt;- Second_df$Hugo_Symbol&#xA;&#xA;Selected_df &amp;lt;- Second_df[, ID_Names]&#xA;&#xA;Expected_df &amp;lt;- t(Selected_df)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2018-03-25T10:24:32.757" LastActivityDate="2018-03-25T10:24:32.757" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="3949" PostTypeId="2" ParentId="3946" CreationDate="2018-03-22T17:24:41.810" Score="3" Body="&lt;p&gt;You can download raw data from SRA (NCBI). On this &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genome/annotation_euk/Capra_hircus/102/&quot; rel=&quot;nofollow noreferrer&quot;&gt;website&lt;/a&gt; under RNA-Seq alignments, you'll find the samples. Click on them, for example &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/biosample/?term=SAMN05231885&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAMN05231885&lt;/a&gt;, and then click on &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/bioproject/325427&quot; rel=&quot;nofollow noreferrer&quot;&gt;PRJNA325427&lt;/a&gt; to see the link to the SRA data (click on &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra?linkname=bioproject_sra_all&amp;amp;from_uid=325427&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;). Here you can download the raw fastq files of the experiment.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2018-03-22T17:24:41.810" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3950" PostTypeId="1" AcceptedAnswerId="3951" CreationDate="2018-03-22T19:24:52.773" Score="5" ViewCount="712" Body="&lt;p&gt;I want to use the &lt;a href=&quot;https://www.bioconductor.org/packages/release/data/annotation/manuals/Homo.sapiens/man/Homo.sapiens.pdf&quot; rel=&quot;noreferrer&quot;&gt;Homo.sapiens&lt;/a&gt; package in Bioconductor to retrieve the chromosome location start and end for each gene symbol in a specific reference (e.g. hg19 or hg38).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Right now, I am using the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; select(Homo.sapiens, keys=c('KRAS'), cols &amp;lt;-c('SYMBOL', 'TXCHROM', 'TXSTART', 'TXEND'), keytype='SYMBOL')&#xA;&#xA;  SYMBOL TXCHROM  TXSTART    TXEND&#xA;1   KRAS   chr12 25358180 25403854&#xA;2   KRAS   chr12 25386768 25403863&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question is two-fold:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How can I specify the human genome reference I want to retrieve the data from? (e.g. hg19 or hg38)&lt;/li&gt;&#xA;&lt;li&gt;Why do I receive multiple and different mappings through &lt;code&gt;Homo.sapiens&lt;/code&gt; for some genes? If I check the locations through &lt;a href=&quot;https://www.genecards.org/cgi-bin/carddisp.pl?gene=KRAS&quot; rel=&quot;noreferrer&quot;&gt;GeneCards&lt;/a&gt; I get:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;chr12:25,204,789-25,250,936 (GRCh38/hg38)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;chr12:25,357,723-25,403,870 (GRCh37/hg19)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-03-26T15:38:14.303" Title="How to get results from Homo.sapiens package in bioconductor for a specific reference" Tags="|gene|reference-genome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3951" PostTypeId="2" ParentId="3950" CreationDate="2018-03-22T22:08:07.110" Score="3" Body="&lt;p&gt;I couldn't find how is this package build, my guess is that is as in the example of the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/OrganismDbi/inst/doc/OrganismDbi.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;OrganismDBI&lt;/a&gt; package. See that vignette section to build your own data package with the genome you want. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The locations are different because you ask for the transcript start and end, if you add the &quot;TXNAME&quot;, you will (probably) see there are different transcripts, so different mappings are reported (untested). &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Update&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The example of the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/OrganismDbi/inst/doc/OrganismDbi.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;OrganismDBI&lt;/a&gt; package builds correctly the Homo.sapiens package, which appears to be linked with hg19. I provide here the relevant code to build the package for hg19 and hg38.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; source(&quot;https://bioconductor.org/biocLite.R&quot;)&#xA;&amp;gt; biocLite(&quot;TxDb.Hsapiens.UCSC.hg19.knownGene&quot;)&#xA;&amp;gt; biocLite(&quot;TxDb.Hsapiens.UCSC.hg38.knownGene&quot;)&#xA;&#xA;&amp;gt; library(OrganismDbi)&#xA;&#xA;# instructions for hg19, for hg38 change accordingly&#xA;&amp;gt; gd &amp;lt;- list(join1 = c(GO.db=&quot;GOID&quot;, org.Hs.eg.db=&quot;GO&quot;), &#xA;             join2 = c(org.Hs.eg.db = &quot;ENTREZID&quot;, &#xA;                       TxDb.Hsapiens.UCSC.hg19.knownGene = &quot;GENEID&quot;))&#xA;&amp;gt; destination &amp;lt;- tempfile()&#xA;&amp;gt; dir.create(destination)&#xA;&amp;gt; makeOrganismPackage(pkgname = &quot;Homo.sapiens.hg19&quot;, graphData = gd, &#xA;                      organism = &quot;Homo sapiens&quot;, version = &quot;1.0.0&quot;, &#xA;                      maintainer = &quot;Maintainer&amp;lt;maintainer@email&amp;gt;&quot;, &#xA;                      author = &quot;Author Name&quot;, destDir = destination, &#xA;                      license = &quot;Artistic-2.0&quot;)&#xA;&#xA;Creating package in /var/folders/sf/tfrswpc96jl8b4520mn784440000gn/T//RtmpzR4YnY/file111b490af70f/Homo.sapiens.hg19&#xA;&#xA;# installing package&#xA;&amp;gt; install.packages(destination, repos = NULL, type=&quot;source&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, you can load and use the packages with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; library(Homo.sapiens.hg19)&#xA;&amp;gt; library(Homo.sapiens.hg38)&#xA;&#xA;&amp;gt; select(Homo.sapiens.hg19, keys=c('KRAS'), &#xA;     cols = c('SYMBOL', 'TXCHROM', 'TXSTART', 'TXEND'), &#xA;     keytype = 'SYMBOL')&#xA;'select()' returned 1:many mapping between keys and columns&#xA;  SYMBOL TXCHROM  TXSTART    TXEND&#xA;1   KRAS   chr12 25358180 25403854&#xA;2   KRAS   chr12 25386768 25403863&#xA;&amp;gt; select(Homo.sapiens.hg38, keys = c('KRAS'), cols = &#xA;       c('SYMBOL', 'TXCHROM', 'TXSTART', 'TXEND'), keytype = 'SYMBOL')&#xA;'select()' returned 1:many mapping between keys and columns&#xA;  SYMBOL TXCHROM  TXSTART    TXEND&#xA;1   KRAS   chr12 25204789 25250931&#xA;2   KRAS   chr12 25209168 25250936&#xA;3   KRAS   chr12 25209431 25250803&#xA;4   KRAS   chr12 25233819 25250929&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2018-03-26T15:38:14.303" LastActivityDate="2018-03-26T15:38:14.303" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3954" PostTypeId="2" ParentId="3909" CreationDate="2018-03-23T22:45:53.823" Score="1" Body="&lt;p&gt;After much analysis of the model, I came to the conclusion that I will not be able to verify how the equilibrium points behave graphically. Since you will have to create a phase diagram in R ^ 5, there are five equations, and I can not do two-to-two comparison in these cases, because I will always have a function that depends on a variable that will not be placed, but will be in the equation plotted anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in any case, if it were in three dimensions, for the purpose of information for those who are interested, the verification of the points would be to place the points (which in calculating have both numbers and not functions in function of other functions), within the model created, hence plot the phase diagram and check whether the arrows converge or not to the point. If they are converging, they are correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the model is taking forever, I took the logistics part and ran well!&lt;/p&gt;&#xA;" OwnerUserId="2518" LastEditorUserId="48" LastEditDate="2018-03-24T11:54:36.767" LastActivityDate="2018-03-24T11:54:36.767" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3955" PostTypeId="2" ParentId="112" CreationDate="2018-03-27T03:17:13.147" Score="3" Body="&lt;p&gt;This answer will sort of only cover the big data parts, i.e. things &gt; 100MB ish, and only if your analysis pipeline ties in with the Python ecosystem. It will require a bit of learning&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try using &lt;a href=&quot;https://quiltdata.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;quilt&lt;/a&gt; which is sort of like 'docker for data' (&lt;a href=&quot;https://github.com/quiltdata/quilt&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github page&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ pip install quilt&#xA;$ quilt install uciml/iris -x da2b6f5  #note the short hash&#xA;$ python&#xA;&amp;gt;&amp;gt;&amp;gt; from quilt.data.uciml import iris&#xA;# you've got data&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Pros:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There doesn't seem to be a filesize limit for public packages (haven't stress tested this though)&lt;/li&gt;&#xA;&lt;li&gt;Hashes your data to ensure reproducibility&lt;/li&gt;&#xA;&lt;li&gt;Support for versioning and tags&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Cons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Storing private data starts from $7 per user / month up to 1TB of data&lt;/li&gt;&#xA;&lt;li&gt;Pretty much Python only at this point, with some community &lt;a href=&quot;https://github.com/stillmatic/quiltr&quot; rel=&quot;nofollow noreferrer&quot;&gt;support for R&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;More information &lt;a href=&quot;https://blog.jupyter.org/reproducible-data-dependencies-for-python-guest-post-d0f68293a99&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2569" LastEditorUserId="48" LastEditDate="2018-03-27T07:43:04.027" LastActivityDate="2018-03-27T07:43:04.027" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3956" PostTypeId="1" CreationDate="2018-03-27T12:48:09.653" Score="8" ViewCount="500" Body="&lt;p&gt;I'm looking for a book about bioinformatics algorithms, such as alignment, BLAST search, and variant calling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm hoping reading about this subject will give me a deeper understanding of the foundations of bioinformatics, and I'm also interested in developing my own tools and algorithms for sequence and variant analysis. I studied applied mathematics and computer science at university, so I don't shy away from math-heavy books.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've listed some candidates below. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0521629713&quot; rel=&quot;noreferrer&quot;&gt;Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleid Acids, 1st edition, Durbin et al. 1998&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0262101068&quot; rel=&quot;noreferrer&quot;&gt;An Introduction to Bioinformatics Algorithms, 1st edition, Jones et al. 2004&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.amazon.co.uk/Algorithms-Bioinformatics-Introduction-Mathematical-Computational/dp/1420070339/ref=sr_1_3?ie=UTF8&amp;amp;qid=1517212543&amp;amp;sr=8-3&amp;amp;keywords=bioinformatics+algorithms&quot; rel=&quot;noreferrer&quot;&gt;Algorithms in Bioinformatics: A Practical Introduction, Sung 2010&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Durbin et al. is quite old, Sung is the newest. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for books with:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;material (algorithms) on alignment, assembly, and variant calling if possible. &lt;/li&gt;&#xA;&lt;li&gt;New books are preferred&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1296" LastEditorUserId="1296" LastEditDate="2018-04-03T07:56:46.993" LastActivityDate="2020-04-22T20:10:31.257" Title="Books on bioinformatics algorithms" Tags="|sequence-alignment|variant-calling|sequence-analysis|algorithms|literature-search|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3957" PostTypeId="2" ParentId="3956" CreationDate="2018-03-27T16:33:15.423" Score="4" Body="&lt;p&gt;Here are a couple books I'd recommend:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dan Gusfield's &lt;a href=&quot;https://www.cambridge.org/core/books/algorithms-on-strings-trees-and-sequences/F0B095049C7E6EF5356F0A26686C20D3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Algorithms on Strings, Trees, and Sequences&lt;/a&gt; is a deep and wide treatment of aligning, searching, and processing strings, trees, and sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Warren Ewens' &lt;a href=&quot;http://www.springer.com/gp/book/9780387400822&quot; rel=&quot;nofollow noreferrer&quot;&gt;Statistical Methods in Bioinformatics&lt;/a&gt; devotes a chapter to BLAST and the math underneath it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit&lt;/em&gt; - Another book that may be useful for some mathematical underpinnings would be Michael S. Waterman's &lt;a href=&quot;https://doi.org/10.1002/(SICI)1097-0258(19961030)15:20%3C2264::AID-SIM386%3E3.0.CO;2-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to Computational Biology&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-03-27T19:18:55.387" LastActivityDate="2018-03-27T19:18:55.387" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3958" PostTypeId="1" AcceptedAnswerId="4492" CreationDate="2018-03-27T20:02:19.267" Score="7" ViewCount="987" Body="&lt;p&gt;Does anyone know of a good tool for visualizing gene loci? Basically, I'd like to show how a group of genes look in different strains of a bacteria. I have annotation of course, I just need a tool to plug it into and get an image of the locus to compare.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm talking something basic like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CPGkw.gif&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CPGkw.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2572" LastEditorUserId="57" LastEditDate="2018-03-29T09:45:08.603" LastActivityDate="2021-02-05T13:11:45.763" Title="A good tool for gene locus visualization" Tags="|gene|visualization|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3959" PostTypeId="2" ParentId="3958" CreationDate="2018-03-27T20:15:46.607" Score="5" Body="&lt;p&gt;I think in R, &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/Gviz.html&quot; rel=&quot;noreferrer&quot;&gt;Gviz&lt;/a&gt; is the best choice. I have used it for human sequences but it will probably also be suitable for Prokaryotes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option is MS powerpoint, if you only need the arrows...&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2018-03-27T20:15:46.607" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3960" PostTypeId="2" ParentId="3941" CreationDate="2018-03-27T22:23:26.280" Score="1" Body="&lt;p&gt;Via &lt;a href=&quot;https://github.com/bedops/bedops&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS&lt;/a&gt; &lt;code&gt;bedmap&lt;/code&gt; and &lt;code&gt;bam2bed&lt;/code&gt;, along with the &lt;code&gt;hist&lt;/code&gt; functionality in &lt;a href=&quot;https://github.com/glamp/bashplotlib&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;bashplotlib&lt;/code&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --echo-map-size --multidelim '\n' intervals.bed &amp;lt;(bam2bed &amp;lt; reads.bam) | hist&#xA;&#xA; 77|           o&#xA; 73|           o&#xA; 69|           o&#xA; 65|           o&#xA; 61|           o&#xA; 57|           o&#xA; 53|           o&#xA; 49|           o&#xA; 45|           o&#xA; 41|           o&#xA; 37|           o&#xA; 33|           o&#xA; 29|           o&#xA; 25|           o&#xA; 21|           o&#xA; 17|          oo&#xA; 13|         ooo&#xA;  9|         ooo&#xA;  5|         ooo&#xA;  1| o    o oooo&#xA;    -----------&#xA;&#xA;------------------------------&#xA;|          Summary           |&#xA;------------------------------&#xA;|     observations: 118      |&#xA;|    min value: 16.000000    |&#xA;|      mean : 34.415254      |&#xA;|    max value: 36.000000    |&#xA;------------------------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="776" LastActivityDate="2018-03-27T22:23:26.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3961" PostTypeId="1" CreationDate="2018-03-28T07:08:51.823" Score="2" ViewCount="49" Body="&lt;p&gt;What is the state-of-the-art to obtain SNVs from circulating tumor DNA after Illumina sequencing? This is, finding a list of nucleotide locations that show tumour SNVs filtered out of any germline polymorphisms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Found so far:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/27531099&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;article&quot;&gt;SiNVICT: ultra-sensitive detection of single nucleotide variants and&#xA;indels in circulating tumour DNA.&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/OpenGene/ctdna-pipeline&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;code&quot;&gt;OpenGene/ctdna-pipeline&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/mikessh/mageri&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;code&quot;&gt;MAGERI&lt;/a&gt;: &lt;a href=&quot;http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005480&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;article&quot;&gt;article&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/broadinstitute/mutect&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;code&quot;&gt;MuTect&lt;/a&gt;: running it against a synthetic normal for unmatched cfDNA samples that don't have normal, then filtering for 1000G germline mutations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/cancerit/CaVEMan&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;code&quot;&gt;CaVEMan&lt;/a&gt;: running it against a reference normal for unmatched cfDNA samples that don't have equivalent normal, then filtering for 1000G germline mutations.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://bioconductor.org/packages/ShearWater&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Bioconductor&quot;&gt;ShearWater&lt;/a&gt;: &lt;a href=&quot;https://bioconductor.org/packages/3.7/bioc/vignettes/deepSNV/inst/doc/shearwaterML.html&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;vignette&quot;&gt;corrects against germline mutations by training against a set of healthy normal samples.&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="180" LastEditorUserId="48" LastEditDate="2018-03-28T08:28:12.600" LastActivityDate="2018-03-28T08:28:12.600" Title="state-of-the-art methods to obtain SNVs from circulating tumor DNA?" Tags="|snv|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3962" PostTypeId="1" CreationDate="2018-03-28T08:58:54.607" Score="4" ViewCount="214" Body="&lt;p&gt;When using GOseq analysis on RNA-seq data, I often find many 'false positives'. What I mean with that is that some genes in certain categories are not really involved in the process, but are only involved due to 'Electronic Annotation' (IEA). Most of the genes are IEA. A list of all evidence codes is &lt;a href=&quot;http://www.geneontology.org/page/guide-go-evidence-codes&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to (easily) do GOseq analysis without this evidence group?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mean, I know how to do it the hard way (make a custom GO-genes annotation in e.g., biomaRt removing all IEA evidence code genes, and perform GOseq with custom genome). But is there also a easy way in GOseq available? &lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="298" LastEditDate="2018-03-28T10:47:19.203" LastActivityDate="2018-03-29T09:22:05.343" Title="GOseq analysis with evidence code filter" Tags="|rna-seq|go-enrichment|go|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="3963" PostTypeId="2" ParentId="3958" CreationDate="2018-03-28T09:05:21.113" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://big.crg.cat/services/gff2ps&quot; rel=&quot;nofollow noreferrer&quot;&gt;GFF2PS&lt;/a&gt; was developed for exactly this sort of thing. It takes a GFF file as input, and will produce a pretty PS output showing the annotations. It was used to create &lt;a href=&quot;http://science.sciencemag.org/content/suppl/2001/02/10/291.5507.1304.DC2&quot; rel=&quot;nofollow noreferrer&quot;&gt;the poster&lt;/a&gt; for Celera's human genome draft &lt;a href=&quot;http://www.sciencemag.org/cgi/content/abstract/291/5507/1304&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1]&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a very simple example of what it can do:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DrQxG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DrQxG.png&quot; alt=&quot;gff2ps example&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And a more complex one from the mosquito genome paper &lt;a href=&quot;http://www.sciencemag.org/cgi/content/abstract/298/5591/129&quot; rel=&quot;nofollow noreferrer&quot;&gt;[2]&lt;/a&gt; (click &lt;a href=&quot;http://genome.imim.es/references/genome_maps/2002_Science_v298_i5591_p129_fig1_MosquitoGenome.ps.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for a full size, 1.5M gzipped PostScript file):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/twKYV.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/twKYV.jpg&quot; alt=&quot;anopheles gambiae genome&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are happy with the defaults, it is very easy to use, but otherwise it is a bit of a pain to configure. It is absurdly powerful though. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://science.sciencemag.org/content/suppl/2001/02/10/291.5507.1304.DC2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Venter et al. Science 291(5507):1304-1351 (2001)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.sciencemag.org/cgi/content/abstract/298/5591/129&quot; rel=&quot;nofollow noreferrer&quot;&gt;Holt et al. Science 298(5591):129-149 (2002)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-03-28T09:05:21.113" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3964" PostTypeId="1" CreationDate="2018-03-28T18:10:11.693" Score="4" ViewCount="144" Body="&lt;p&gt;I have a list of several protein names, their primary gene name and their amino acid sequence (as extracted from the 'SEQRES' section in their corresponding PDB files).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for the genomic position (triad of translated nucleotides) of a specific amino acid in the sequence (i.e. residue number 64 in a protein sequence of 344 amino acids).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to BLAT an amino acid sequence by using biomaRt in R? &#xA;Do you suggest other methods to achieve this task?&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="48" LastEditDate="2018-03-29T09:30:30.900" LastActivityDate="2018-03-29T09:30:30.900" Title="Retrieval of genomic position by using biomaRt package" Tags="|r|proteins|ngs|pdb|biomart|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3965" PostTypeId="2" ParentId="3964" CreationDate="2018-03-29T08:49:38.330" Score="4" Body="&lt;p&gt;BiomaRt cannot do that I'm afraid. There is an &lt;a href=&quot;http://rest.ensembl.org/documentation/info/assembly_translation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl REST API endpoint&lt;/a&gt; that will get you the genomic location of protein coordinates. It needs an Ensembl peptide ID an input though, so you could use the &lt;a href=&quot;http://rest.ensembl.org/documentation/info/xref_external&quot; rel=&quot;nofollow noreferrer&quot;&gt;xref endpoint&lt;/a&gt; first to get that. There's a bit of sample code in R on those pages, you could use that to script together if you've got a list of these.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2018-03-29T08:49:38.330" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3966" PostTypeId="2" ParentId="3962" CreationDate="2018-03-29T09:22:05.343" Score="4" Body="&lt;p&gt;A bit weird to answer my own question, but for who's interested I can show the 'hard' way using &lt;code&gt;biomaRt&lt;/code&gt; and then use &lt;code&gt;goseq&lt;/code&gt; with &lt;code&gt;gene2cat&lt;/code&gt; defined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So first you need biomaRt to get annotation of the genes. I have ensembl genes from mm10.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(biomaRt)&#xA;&#xA;table &amp;lt;- read.table(file, header = T, stringsAsFactors = F)&#xA;&#xA;ensembl &amp;lt;- useMart(&quot;ENSEMBL_MART_ENSEMBL&quot;)&#xA;&#xA;ensembl &amp;lt;- useDataset(&quot;mmusculus_gene_ensembl&quot;, mart = ensembl)&#xA;&#xA;allGenes &amp;lt;- table$Geneid&#xA;&#xA;GOs &amp;lt;- getBM(attributes=c('ensembl_gene_id', 'go_id', 'name_1006', 'go_linkage_type', 'namespace_1003'), &#xA;      filters = 'ensembl_gene_id', &#xA;      values = allGenes, &#xA;      mart = ensembl)&#xA;&#xA;dim(GOs)&#xA;[1] 477082      5&#xA;&#xA;table(GOs$go_linkage_type)&#xA;          EXP    IBA     IC    IDA    IEA    IEP    IGI    IMP    IPI    ISA    ISM    ISO    ISS    NAS     ND    RCA    TAS &#xA; 36931     36  27918    525  34038 210114    630   6891  30219   6205   4234     20  77365  24673    609  12821     99   3754&#xA;&#xA;# I only want BP and no IEA evidence code&#xA;GOs_no_IEA &amp;lt;- GOs[GOs$go_linkage_type != &quot;IEA&quot; &amp;amp; GOs$namespace_1003 == &quot;biological_process&quot;,]&#xA;&#xA;dim(GOs_no_IEA)&#xA;[1] 111944      5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you can continue with goseq.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(goseq)&#xA;&#xA;All &amp;lt;- read.table(input_All, sep = &quot;\t&quot;, header = T)&#xA;&#xA;All_genes &amp;lt;- as.character(All$Geneid)&#xA;&#xA;TopTable &amp;lt;- read.table(input, sep = &quot;\t&quot;, header = T)&#xA;&#xA;sig_genes &amp;lt;- as.character(TopTable$Ensembl)&#xA;&#xA;# Make gene vector&#xA;&#xA;gene.vector &amp;lt;- as.integer(All_genes %in% sig_genes)&#xA;&#xA;names(gene.vector) &amp;lt;- All_genes&#xA;&#xA;gene.lengths &amp;lt;- All$Length&#xA;&#xA;pwf &amp;lt;- nullp(gene.vector, &quot;mm10&quot;, &quot;ensGene&quot;, bias.data = gene.lengths)&#xA;head(pwf)&#xA;&#xA;GO.wall &amp;lt;- goseq(pwf, &quot;mm10&quot;, &quot;ensGene&quot;, gene2cat = GOs_no_IEA[, c(1:2)], use_genes_without_cat = F, test.cats = &quot;GO:BP&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastActivityDate="2018-03-29T09:22:05.343" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3967" PostTypeId="2" ParentId="3021" CreationDate="2018-03-29T15:50:22.970" Score="1" Body="&lt;p&gt;I had a similar problem and found a solution using awk. Since this is the first time I use awk, the syntax might not be perfect, but you should get an idea of what is possible.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -F' ' '{if(/^&amp;gt;/){sub(&quot;strain &quot;, &quot;&quot;); name=$2&quot; &quot;$3&quot; &quot;$4; print &amp;gt; name&quot;.fa&quot;}else{print &amp;gt; name&quot;.fa&quot;}}' yourfile.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(more information below)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This should transform a file like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;yourfile.fa&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;NZ_AESB01000237.1 Clostridium botulinum BKT028387 contig00237 ATGC&#xA;ATGCATGC&#xA;ATGCATGC&#xA;&amp;gt;NZ_AESB01000236.1 Clostridium botulinum BKT028387 contig00236 ATGC&#xA;ATGCATGC&#xA;&amp;gt;NZ_ALNK01000035.1 Peptoanaerobacter stomatis strain OBRC8 ctg120007290634 ATGC&#xA;ATGCATGC&#xA;ATGCATGC&#xA;&amp;gt;NZ_ALNK01000031.1 Peptoanaerobacter stomatis strain OBRC8 ctg120007290635 ATGC&#xA;ATGCATGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Into two files like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;'Clostridium botulinum BKT028387.fa'&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;NZ_AESB01000237.1 Clostridium botulinum BKT028387 contig00237 ATGC&#xA;ATGCATGC&#xA;ATGCATGC&#xA;&amp;gt;NZ_AESB01000236.1 Clostridium botulinum BKT028387 contig00236 ATGC&#xA;ATGCATGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;'Peptoanaerobacter stomatis OBRC8.fa'&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;NZ_ALNK01000035.1 Peptoanaerobacter stomatis OBRC8 ctg120007290634 ATGC&#xA;ATGCATGC&#xA;ATGCATGC&#xA;&amp;gt;NZ_ALNK01000031.1 Peptoanaerobacter stomatis OBRC8 ctg120007290635 ATGC&#xA;ATGCATGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How it works&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;-F' '&lt;/code&gt; tells awk to split the line by space (accessible with &lt;code&gt;$1&lt;/code&gt;, &lt;code&gt;$2&lt;/code&gt;, ...)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;if(/^&amp;gt;/)&lt;/code&gt; looks for line starting with the &lt;code&gt;&amp;gt;&lt;/code&gt; character, then &lt;code&gt;sub(&quot;strain &quot;, &quot;&quot;)&lt;/code&gt; remove the word &lt;code&gt;strain&lt;/code&gt; and &lt;code&gt;name=$2&quot; &quot;$3&quot; &quot;$4; print &amp;gt; name&quot;.fa&quot;&lt;/code&gt; print the line into a new file with a name based on what you asked for.&lt;/li&gt;&#xA;&lt;li&gt;else if the line does not start with &lt;code&gt;&amp;gt;&lt;/code&gt;, &lt;code&gt;{print &amp;gt; name&quot;.fa&quot;}&lt;/code&gt; appends it to a file using the current value of &lt;code&gt;name&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This solution will work regardless of the number of lines between the fasta headers.&#xA;Note that I won't recommend using spaces in filenames, so I would replace &lt;code&gt;name=$2&quot; &quot;$3&quot; &quot;$4&lt;/code&gt; with something like &lt;code&gt;name=$2&quot;_&quot;$3&quot;_&quot;$4&lt;/code&gt;.&#xA;I used this solution to split a large fasta file (&gt;5 GB) containing &gt;5k genomes with &gt;25k scaffolds and I found it fast and rather memory efficient.&lt;/p&gt;&#xA;" OwnerUserId="2580" LastActivityDate="2018-03-29T15:50:22.970" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3968" PostTypeId="1" AcceptedAnswerId="3969" CreationDate="2018-03-29T15:51:42.627" Score="1" ViewCount="481" Body="&lt;p&gt;I need some help. I have &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/3913/298&quot;&gt;written a program&lt;/a&gt; that simulates protein cleavage but I need to modify it so that it produces fasta formatted output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here is all of my sequence in .fas file &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;jgi|Cormi1|6|CCM_00006m.01&#xA;MSSFTRAIRPALQGGRRVLLAQRLSTPRIAHPIAATALRSLSTTAPRRSVDLSELQPTPI&#xA;THLSEVETAMADAVSKFATEVVLPKAREMDEAEAMDPAVVEQLFEQGLMGIEIPEEYGGA&#xA;GMNFTAAIIAIEELARADPSVSVLVDVHNTLCNTALMKWGSDALKKKYLPRLATNTVASF&#xA;CLSEPVSGSDAFALATRATETPDGYTITGSKMWITNSMEAGFFIVFANLDPAKGYKGITA&#xA;FVVDKDTPGFSIAKKEKKLGIRASSTCVLNFDDVAVPKENLLGKPGEGYKYAIGILNEGR&#xA;IGIAAQMTGLALGAFENAVRYVWNDRKQFGTLVGEFQGMQHQIAQSYTEIAAARALVYNA&#xA;ARKKEAGEDFVRDAAMAKLYASQVAGRVAGLAVEWMGGMGFVREGLAEKYWRDSKIGAIY&#xA;EGTSNIQLNTIAKLLQKEYTA*&#xA;&amp;gt;jgi|Cormi1|8|CCM_00008m.01&#xA;MLAGSNARILRYIVVAGIILIFFYHFSGSRYDTLPRPVPAPDHAQAASQPSPPNDEATTA&#xA;GRPVDSTQKSIPADRDRPNSGFPKSVSASTDAAASTLADIDPGPRANATFVSLARNADVW&#xA;DIARSIRQVEDRFNRRFNYDWVFLNDKPFDDNFRNITSALVSGTARFGQIPVEHWSVPDW&#xA;IDEDKATKVREDMKERKIIYGDSMSYRHMCRFESGFFFQHPIMKNYDFYWRVEPSIELFC&#xA;DIAQDPFRFMKENGKKYSFTISLLEYVDTIPTLWDSAVKFIENYPQHVAADNSMEFLSDD&#xA;GGQTYNMCHFWSNFEIGSLEWMRSDAYVDFFNSLDQDGGFFYERWGDAPVHSIAAGLLLN&#xA;KTELHFFDEIAYFHMPFTHCPNDVQSRIDLKCHCNPKDNFDWSRYSCTSRYFEMMGMQKP&#xA;DGVD*&#xA;&amp;gt;jgi|Cormi1|10|CCM_00010m.01&#xA;MSTSPSDGTVDVDLPDAPTAPFPLHSDTHTANHVENDEDDDPITASYPVFLNPALPLGRR&#xA;LLVLQHPNRTDDHGARAPPSELRLKAESGMVEVDMPLDYNEAYDRDKGLRWGRTLQTSMA&#xA;AKNGGSHGLAGGFGFGAVQQRGAGAGAGGARKRNEVDAEDLIDWNEAVRQDKVLRTQTLG&#xA;GQYPDADEVQYMVGVFQGNDLHLTPVSSLVHLRPQLHHLDAATQQERQAAAKEAGAAAAS&#xA;SSGGATAGPGATGNPAARAIHMSIKTTADGETVTTETIADRLRFVQGEAWRKLRYTDENE&#xA;EAAWEVYNESLFLSASEARAPPPGSEKKKKTGSNEEELPYRSLQGAVATFASKWESGQLL&#xA;EAVSGMKKPPPVVVVPAAVKKAKAAAEAEAKAKDVEEKRKAAEAQMAATATAAAAAEETR&#xA;RPKRAPRGGGGLGTAKRPSRAK*&#xA;&amp;gt;jgi|Cormi1|11|CCM_00011m.01&#xA;MASATTPILPAEGKRNVLITSALPYSNNLPHLGNVIGSVLSADLYARYCRARGYQTLYVC&#xA;GTDQYGTATETKALEEGTDPASLCAKYHALHKSIYDWFRIDFDIFDKTPTQHQTDIVQDI&#xA;FTKLWKNGYIYEDETIQPFCPVESHNTFLADRFVEGECSICGDKGARGDQCDACGSILDP&#xA;LKPEETGAVDGDAAIEVKATGWLVNPRCKVDGTAPEARKTKHLYLKLDDISEELKAWYKT&#xA;TADNGWSTNAKAITQSWIDKGLKPRAISRDLKWGVPIPAVEGLNVEDYARKVFYVWFDAC&#xA;IGYISITKTYTDGDNRDGKNWEKWWKNPENVELMQFQGKDNVGFHSIVFPSSLLGTKENW&#xA;TKVTRISATEYLNYEQGKFSKSRGIGVFGNNAKDTGIDPDIWRFYLLSRRPELADSEFKW&#xA;EEFIDVNNNELLKNVGNFIQRTLKFCVAKMDSTVPAATVSNDVIESHKQSANQILASYIE&#xA;HQEANKMRIGISDILTFSALGNKFLQDNGLNNKLLAEEPELCNTVINVAINHVHLLAGLL&#xA;YPYMPTTAESIFQQLGQEPNPFIPDVFGFTAVPVGHKIGTPGPLFSVIPAAKLDEWRDAY&#xA;GGAELKKQKAIEAEKAAAKKAKKLADKEKKKAKKAETADLKTEGASTAADISKLSLEEKK&#xA;*&#xA;&amp;gt;jgi|Cormi1|15|CCM_00015m.01&#xA;MAEEEVRSTVALVVGALLPALMTVFFNGLPGAVGGSNLPRWGVRLPILLTCAAQLWFMLV&#xA;SRVVPEPYLDEIFHIPQAQKYCEGRFFDWDDKITTPPGLYFFSVAVHRIASALRIPQLSV&#xA;CDPFSLRAVNFAGVLVVSYLALWCRQALEARQHEAISSPTPARVRAFSQYAVHTAVNIGL&#xA;FPLLFFFGGLYYTDVLSTGVVLAAFLNHLRRVGVPRSSVWSDLGTVALGVAALGMRQTNV&#xA;FWIVVFMGGLEAVHAVKTLRPARVDQPVMTTLAAQCRYFAWRYSLGDIHDLPVHKAYPDD&#xA;MIFTALSIGIAALCNPLRVVRQIWPYLAVLASFAAFVQWNGSVVLGDKSNHVATIHLAQM&#xA;LYIWPFFAFFSLPLLLPSGLSFLNMVVSFFRTHARASPTPDAAADGLLRATLRFFHSKVL&#xA;WPAYLAATAVLSLAVVRFNTIIHPFTLADNRHYMFYVFRYTIRRGGAVRLALALPYTVCR&#xA;WVVWDALGGHNSWSLGNDKPTAFVSHPFWTPPSAKVRRQTSTSYPRPPERASTAESRQAS&#xA;EAALKARLEQDSLYLSQQSVSTSTGLIFLLATALSLVTAPLVEPRYFILPWVVWRLMVPA&#xA;WRLQDHLRGGILEGSPAAKAVAHWSASYDLRLFLESAWHVVINLATGYVFLYKPYVWRAE&#xA;DGTVLEGGNLQRFMW*&#xA;&amp;gt;jgi|Cormi1|20|CCM_00020m.01&#xA;MTNTIVIVGQVLFPLLTRTTNCIANERRPPSSGGVSGLTSAYLLSKSKGNTVTLVAKHMP&#xA;GDYDIEYASPIAGANVLPMNPDADSRWERRTWPEFVWLCRNVPEAGFHFKNALVLRQHKD&#xA;QEADPSGGPIDPLFDRDPWYRHLTPDYRELAASELPPGQDSGCSFTSMCLNTPMYLFWLV&#xA;GQCLRQGVVFRRAVLHDLTEARRLAYSGRPADVLINATGLGARTLGGVRDAGMLPARGQI&#xA;VLVRNDGSSMLNTSAVDDGSGEILYTMQRAAGGGTILGGTYDLGNWESQPCPNVATRIMA&#xA;RAVRLDPALAGGKGVAGLDVIRHAVGLRPYRKGGVRVERETLRDEVDGASVHVVHNYGHA&#xA;GWGFQGSFGCAEKVVELVNEIRAANGEDLSKEPTLFNFEAAAASKL*&#xA;&amp;gt;jgi|Cormi1|21|CCM_00021m.01&#xA;MEATNSPARAAREPAVHVRALTFVVDVACGRHALSPLIPVALWLADAVLTCLIIWKIPYT&#xA;EIDWVAYMEQITQFVHGERDYTKIKGGTGPLVYPAAHVYTYTGLYYLTGEGENVLLAQQL&#xA;FAVLYMVTLGLVMLCYWKAKVPPYIFPLLILSKRLHSVFVLRCFNDCFAVFFLWLAIYSF&#xA;QRRLWALGSMAYTWGLGIKMSLLLVLPAVGAVLLHACGFSGALSLAWLMAQVQIVIALPF&#xA;IATNTNGYLARAFELSRQFQFEWTVNWRMLGEATFRRREWALLLLGLHAATLALFLATRW&#xA;MRPAGKTLRRMVPAWLRGRAALTPDEEIRVSRRVTPDFVLTTMLSANLVGLLFARSLHYQ&#xA;FYAYLAWSTPFLLWRALPGGAAVLTYPLWLLQEWAWNVFPSTSESSTAAVAVMATTVLVV&#xA;YFGTRNDGLTIKKPAAKKQ*&#xA;&amp;gt;jgi|Cormi1|32|CCM_00032m.01&#xA;MRLIAAQMLRGTAQRRGFATTPRRLDNYAFVGLGQMGYQMAKNLQSKLQPSDKISLYDIN&#xA;TDAMKRLQVEMKAASNGAAVELAASAHDAAKEADTVITVLPEPQHVQGVYKSILTPSLPR&#xA;RDRVFIDCSTIDPSTSRAVAASVAAAQQGAFVDAPMSGGVVGATAGTLTFMLGAPASLLP&#xA;RLEPALLTMGRKVLHCGDQGAGLSAKLANNYLLALNNIATAEAMNLGIRWGLDPKKLAGV&#xA;LNVSTGRCWPSEVNNPVKGVVETAPAGRDYSGGFGIALMKKDLRLAMVAAAEAGADMKLA&#xA;DAAFKVYEEASEREDCKGRDFSVVYRYLGGKE*&#xA;&amp;gt;jgi|Cormi1|35|CCM_00035m.01&#xA;MPGLPTSVDLDECISRLYKKELLAESVIEAICAKTKELLMRESNVVHVQAPVTVVGDIHG&#xA;QFYDLIEIFRIGGYCPDTNYLFLGDYVDRGMFSVETISLLVCLKLRYPERVHLIRGNHES&#xA;RGVTQSYGFYTECSRKYGNANVWHYFTDMFDFMTLSVVIDNQIFCVHGGLSPSIHSIDQI&#xA;KIIDRFREIPHEGPMADLVWSDPDPDRDEFSLSPRGAGYTFGAQVVKKFLAVNNMSHILR&#xA;AHQLCQEGFQVLYEDRLSTVWSAPNYCYRCGNMASVLEVNSDGERFFNVFAAAPENDQVK&#xA;DMQPGIDKAGDGNALPDYFL*&#xA;&amp;gt;jgi|Cormi1|37|CCM_00037m.01&#xA;MRYAFAYAALVSTASAHGLVTMIKGDNGVMMPGLGVTDGTPRDCSSNGCGSQADTSIIRN&#xA;RDMQQNPLGKTQGTGPVDAAVNVAAFMGTSKNAPKNNGASAGVGQEDDLSGLPAGAAKRH&#xA;EERKNMVRGLFGDLLGGAAGGAGAAGGAAGGAAGGLGGLLGGAGGAGGAAGGAAGGLGGL&#xA;LGGLGGGGGGTKSNLAAESMVADTAGMGSTQGMPTSDSTGTVSLVYRQINQDGAGPLTAA&#xA;VDGTSGGTDASAFQQAEVTKDVPGLGIGGLSLATNTDFPLQVKMPAGMTCEGTVGGVKNV&#xA;CIVRVRNSAAAGPFGGSAAFTQNATARKRAIAYRLRKRMEIAREAVPQIDYTKADAAEEL&#xA;EDEEDE*&#xA;&amp;gt;jgi|Cormi1|65|CCM_00065m.01&#xA;MRWHVYWGVVQLVELSNVDKRVAVACHVHTLALPDISLSHHPASSATVTPVIIVRLNRPE&#xA;SRHAYTDTMGASLVTAYGLLSADPRVRAIVLTSADASNKFFCAGMDFNVRAQRVEDPLQY&#xA;RDSGGTVSIAMHRCQKPIVAAVNGAAVGIGMTMTLPATVRVASTKAKCGFVFAQRGFCME&#xA;ACSSFFLPRLIGTAKAMHLVSTGAVYPATHRLFDDLFSEVVEPEDVLPTALKIAEGMAQN&#xA;CSIVSTQVMKDMMYSGPKSPEEAHLLESKIFYSLTCPDTDSAEGVQSFLQKRPPKFTGTM&#xA;ENQRPFGWPWWSNLDVRPKI*&#xA;&amp;gt;jgi|Cormi1|71|CCM_00071m.01&#xA;MSRIEKLSISGVRSFSPACREAIQFNTPLTLIVGYNGSGKTTIIECLKYATTGELPPNSK&#xA;GGAFIHDPKLCGEKEVLAQVKLQFRSINDRQHVATRSLQLTVKKTTRSQKTLDCSLVVLN&#xA;NGEKTTTSTRQAQLDEMIPERLGVSPAILDAVIFCHQDESLWPLSEPSVLKKRFDEIFEA&#xA;LKYTKAIENLKVLRKKQVEELGKLQNDEFHNKGNKDRGDRAEKRMTGLQAEIEAAREKCE&#xA;AITAHMQETQDKIKQKHEQANSFLQIVQNLGNKREQLDYRQDAVDELRQTIDELPEDDRS&#xA;LERQLVQYEESMRRQSEEAQRNKTQYGDLQQDLVKSRKSLSTKLAEQGKHQSDKEKHERQ&#xA;LKMRMEMVQEAAQHHGFRGYEDDLTDALIKQFNDKIQKLFAEKKRELERLQKENAAELDK&#xA;ASATISEFEGQKAARTQDRVSAKQRQGAIEKRTIVLQNEASSIDVDEGLKAILDGQLGEL&#xA;ESKFSQVHREFETAAWDNQISDENAKLWQLESESEKLGRELIECTRLASERAQLDFRRKE&#xA;LSDRKLKLTSLTETWRSKLDKVIGGDWSPEDLEAKFQALLAAQNKALQEARQQQEQTRQK&#xA;QARVDFKLKTVKETCEQKQTDQIECKKKVLQALQNVRDSAVIEDYTEEVSATEQQIETLR&#xA;DDLSLIDALKDYYTKCKRALDAKKTCLLCDRHFDDSQAASIDRLRVKIDRHLNSKTKIEA&#xA;EQDLSEASSTLQSLVAVRSQYDTYERLGKELPGLRDERKSIEADHESTERLLEEHDAVVA&#xA;AEEDKKRDMNDMSKTVSNITQTLREIKDSEGQVERIASQQQSVGTGRSADEIHDLQGTVS&#xA;EQMRASKNRVAKLSNDRQRARDQISTLELEKSELRNKISLTLGQLDKKQDLLGQIQSLKE&#xA;EQTHQREAIQRADEELEKVEPRITEARSIRDETLKRGRAKEQVVTESRDSVATSINEIKM&#xA;IESDIRDYEDRGGASNLASNQRAIATLEQNITTTETEISELTQRTNKLKQDIDNGDRTKK&#xA;NINDNLNYRKHLRMLETLRGEINELEDRNAHEDYERLQGEARALENQSNRLFAERGSVMG&#xA;SMKTKDEDLDRLLKEWEMDYKDAKQKYRESHIRVETTKAAIEDLAQCGAAVDKAVMQFHA&#xA;MKMAEVNRIAGELWQSTYQGTDIDTILIRSDNEGTTGRRNYNYRLCMVKQDAEMDMRGRC&#xA;SAGQKVLASIIVRLALAESFGVNCGLIALDEPTTNLDRDNIKSLAESLHNIIRTRQAQSN&#xA;FQLIVITHDEDFLRHMRCSDFCDSFFRVKRDDKQNSVISRESITKIF*&#xA;&amp;gt;jgi|Cormi1|75|CCM_00075m.01&#xA;MPSSTPLPPPKGFLSYPDVEEIISPRYEPRKDANPVLSGLPLVIASSLLTHSSFLAKIFY&#xA;NNAGLQKIKDLPFLDDVPYTFHPLVIPLGDNGPMVELGEAQLNPKNRGDTHHYTASDYHA&#xA;MYKSGQVTPLQVTETLLSITSKYAKEASIYQDAWADSHGADQLALDAAKASTERWAAGKA&#xA;LGVLDGVPIGVKDDLSVKGYIDHDGMAYKKGVPFFKSAEETVWPVQALQAAGAVVIGKNR&#xA;MHELGSDTNGLNPHQGTPTNHLNNQYYPGGSSSGAGSSLGAGVIPIAVGTDAGGSVRFPA&#xA;VYNGVYGLKPTHHRTLEMNSTMCVAGPMAANVADLTIAYRIMSQPDPTTSIQSRFAVSQP&#xA;PAAGTNKYIGVDRAWWGASDPRVIGACNKALEYFTSQGYELVDITIPYIDEAQTAHGGVT&#xA;IAEMTEAARRRGGDKTSWTTLIGPVNKLVLALGLETNAADLLKFNSMRTLIMRHMAWLFQ&#xA;KYPGLLILTPTAPFLGWEKKPAHAKYGVSDGDKTFLSMIYVFLANIAGLPALTAPIAYVD&#xA;PDQGEGKVCIGMMATGEWGAEEALLAWAADAETYLHEVYPEGRRRPKTWVDVLGLAAKQG&#xA;TV*&#xA;&amp;gt;jgi|Cormi1|77|CCM_00077m.01&#xA;MLSSTITRSRPTATMAKLAGGMLVAGLATKYTIGVAHAESPAAPPRSQVFGRGPALVSLV&#xA;LEDAEMVNHNTRRLRFRFQDKEAASGLPLTSALLTYSWPKDSWLPVIRPYTPITTADETG&#xA;HLDLLVKQYPNGKSSTHLHSLKPGHTLFFVAPLSGHQWQPNSYKHVTLIAGGAGITPIYQ&#xA;LAQGILNNPDDKTAVTVVFGVNSDRDVLFKKEFDEYEQRFPGRFRAVYTVSSPVPGSPHR&#xA;KGHVNEALLKEVTPAPGPGNMVFVSGPPPMETALVGSRREPGILEKYGYRKEQVYKF*&#xA;&amp;gt;jgi|Cormi1|78|CCM_00078m.01&#xA;MAADASFDNLARQFSHLPLIDKLPNCFPDINPVDIYRAHITSILSHITGADKSLIYPVLQ&#xA;WTATLDKGDLMLAIPALRIKGKPDDLGKQWVEKWPESPLVEKPVHNGAFISFFFKPEPMA&#xA;KTVIPMALQHKQDFGSNPANGLRDQNDPSKGRQKIVIEFSSPNIAKPFHAGHLRSTIIGG&#xA;FLANLYRSAGWDVATVNYLGDWGKQYGVLALGFEKYGSEEELVKDPINHLFTVYVKISKE&#xA;VAEEKEQADKLKAEGKEEEAQKILDNGVDEKARSYFKKMTEGDEQAIALWKRFRDYSIER&#xA;YKQTYARLNIHFDEYSGESKVSEDEMQQAAAKLKEMGLAHEDRGALLMDFTKSVPGKEGK&#xA;TLGKAILRKKDGTALYLTRDISELINRHERYKFDKMIYVIASEQDLHVKQFFKIVRMMGR&#xA;DDIADKCQHVNFGLVLGMSTRKGTVKFLNDILRDVGDHMHDVMKKNEDKYSQVEDPEQIA&#xA;DKLGISSVMVQDMNGKRINNYTFDLSRMTSFEGDTGPYLQYAHARLCSIYRKAELSDEAI&#xA;AAADLSLLTEPMAVNLLRVIAQWPDVVQNVLKTLEPTTILTYLFKLTHTLSTSYTHLRIV&#xA;GSEPEVMKARLALYNSTRVVLYNGMAILGLTPIERM*&#xA;&amp;gt;jgi|Cormi1|79|CCM_00079m.01&#xA;MRSRNASPHIGPWRNPARRVRHGTTRNYFDSGCSVWMSSPSRLSAPVPPASPLQKNKFSH&#xA;IAIYEQRAATGGVWNTSHVSQEPGFSIPRTTPTSEHEYVVVEDGHDAGARVELVSPVYDN&#xA;LETNIPHGLMSYTDLEFPAGTPLFPEHGTVLAYIQQYGRDVEHLVTFETQVRDVRKTTAE&#xA;DGRLGWTVETKDLKSGVVSAKSYDAVVAASGHYSDPFVPDIPGIAEFEAVQPGAILHSKF&#xA;YRRPEQFTGKKVVVVGNAASGIDISHQIATSAQLPVLISEKDVPGAPPAASNNWSRHVGQ&#xA;IVELLPVTRSVRFSSGRIEDGVDAVIFCTGYHYSFPFLGSLAPGITAPNGTYADHLWEHM&#xA;LYAPDPTLALLVIPKRIVPFPFAEAQMAVIARIWAGRLNVPNQSERDAWVRRCGAESAPG&#xA;ARHTFGYPQDVDYINRLHALSGQARPDAERGLENDGAGKRSPYWGEELSWVRSQIFAIKV&#xA;ASRALGERRKECKTLADLGFDFQAAK*&#xA;&amp;gt;jgi|Cormi1|87|CCM_00087m.01&#xA;MFLLATLFALSALPWTAPAMRSDHLAKLRQETVDMFYHGYNNYMLHAFPEDELRPVSCQP&#xA;LTRNPNKPADIGLNDVLGNYSLTLIDSLSTLAILAGGPANEKYTGSQALSDFQDSVAQFV&#xA;SYYGDGRQGSSGQGLRSRGFDLDSKVQVFETVIRGVGGLLSAHLFAVGDLPIRGYDPKPD&#xA;SKWASEDPLETAPISWHGGFRYDGQLLRLALDLSERLLPAFYTKTGIPYPRVNLRSGIPF&#xA;YINSPLHQDSEDTEPVVGNAEITETCSAGAGSLTLEFTVLSRLSGDPRFEQAAKRAFWEV&#xA;WERRSDIGLIGNGIDAERGVWIGPHSGIGAGMDSFFEYALKSHILLSGQETPNMTRSHRQ&#xA;STTGWLDPNSLHPPLPPEMHSSDAFLEAWHQAHASVKRYIYTDRNHFPYYSNNHRATGQP&#xA;YTMWIDSLGAFYPGLLALAGEVDEAVEANLVYTALWTRYGAIPERWSVRESNVEPGIAWW&#xA;PGRPEFIESTYHIYRATADPWYLRVGEMVLHDIQRRCWVACGWAGLENVLSGEKQDRMES&#xA;FFLGETTKYLYLLFDPEHPLNNLDAAYVFTTEGHPLIIPRRARGSSTPRRRASRTSRQEE&#xA;KDVAVYSYYDTRFTNSCPAARTPTEPLAGSNTAARRNLFDVSRFTNLYNTPNIHGPLESF&#xA;KTRDEHKGIITQYRATSNYSVFPWTLPPTMLPHNGTCPVPSNRILSWIEFPTVDPAPSLF&#xA;ATRGGGGDSNTPLRWYHNVGPTVERVDGLKLQLEQEFNEAAGTDVWMVTHVAGKTVGRHE&#xA;NVIFHADHVRHFRDDAFSVLRRKDMVEVVLLVQPEPTPQASRELPPAVMPLPVPGALDEV&#xA;SSLDAAPDNTSSTRRPSVRASGQPPATDPDSLFKSILRAVSSALEGSATASTPSPSPLRG&#xA;SGSGSGSRDDSSNSDDDVGVASIHSWLANTATGAGAFPLPSVRDALLRNSPDFSAADPAA&#xA;SLPWRHVYMAGSACDGPLPEDAPRRHQVIVMRRGGCPFGDKLAHVPNFVPDEASLQLVVV&#xA;VDEEGGGHVDDAPRPLLATEQRTPRGTPRLHGIPMVLLPASKGTYELFGEAAAVGLRRKY&#xA;VVKSQGYVIENAIVM*&#xA;&amp;gt;jgi|Cormi1|88|CCM_00088m.01&#xA;MSAGHRVARVHLKEKNTAQDASGRQPSSSSSATAQLASAKVVLGNNLRRFPDFPIPGINF&#xA;VDIMPLFANPEAHATLLNALELQITQTFVNTKPDVIVGLDARGFLFGPGLALRLGVPFAA&#xA;VRKQGKLPGPCVTAEYVKEYGKDFFQMQEDAIREGQKVLVVDDIIATGGSAKAASDLITQ&#xA;LKGHVMGYLFILEIPGLNGRDKLGAAPTTILLQDE*&#xA;&amp;gt;jgi|Cormi1|96|CCM_00096m.01&#xA;MHLVPKELDKLVITQLGLLAQRRLARGVKLNHAEATALIANNIHELIRDGNHTVADLMAL&#xA;GATMLGRRHVIPSVCHTLKEIQVEGTFPTGTYLVTVHNPISTNDGDLARALYGSFLPVPR&#xA;DADRLFPAALDPADFAPARQPGAVVAVKGDKITLNADRRRIALRVVSEGDRPIQVGSHYH&#xA;FIETNPQLVFDRARAYGFRLDIPAGTSYRFEPGDAKTVTLVEIAGHQVIRGGNGLATGGV&#xA;ERWRVDAIVERLQKAGYAHAPEPEVNGVVAGRRRAAAARVPRPYRMDRTAYTVMFGPTTG&#xA;DRVRLSSTDLWVKVERDYTTYGDECKFGGGKTLREGMGQASGRPDADSLDLVITNALIID&#xA;YTGIVKADIGVKNGIIVGIGKAGNPDVMEGVSPNMIVGSCTDVIAGENKIITAGAIDTHI&#xA;HLICPQQAQEALASGVTTFLGGGTGPSAGSNATTCTPGKHLMKQMLQACDALPVNVGITA&#xA;KGSDSAPEALREQVAAGACGLKIHEDWGATPSAIDTCLSVCDEMDVQCLIHTDTLNESGF&#xA;VESTIASFKNRTIHTYHTEGAGGGHAPDIISVVERANVLPSSTNPTRPFTRNTLDEHLDM&#xA;LMVCHHLSKDIPEDVAFAESRIRAETIAAEDVLHDTGAISMMSSDSQAMGRCGEVVLRTW&#xA;NTAHKNKLQRGCLPEDDGTGADNFRVKRYVSKYTINPALAQGFSHLVGSVEVGKLADLVV&#xA;WDPAWFGTKPHQVIKSGLIAWSQMGDPNASIPTVQPVIGRPMFASFVPAASVVFVSQASV&#xA;ASGAVASYGLRKRVEAVRGCRTVGKADMRFNDAMPKMRVDPESYTVEADGEVCTAEPAET&#xA;LPLTQSWYGILDNQNAPA*&#xA;&amp;gt;jgi|Cormi1|100|CCM_00100m.01&#xA;MAARLSAAALAPLRPISSSVCAARQLSQSLLQSQQTRGVAQTYLKKVAEGEERWKARAER&#xA;IQNGEQRHVWDILDERGYIKDVAGNTDRIKEVMRIKRIGAYVGIDPTADSLHVGHMLPLM&#xA;PLFWLWFHGYPAVTLIGGSTARIGDPTGRLQSREHITNSDISKNITKIHYQLSKLWHNVV&#xA;QMRIKYGYEDDWAAKRSLLNNNMWLQGLTLYDFIKRLARDTRIGPMLSRDTAKRKLTDGD&#xA;GMSLGEFMYPLLQGWDFWHMYNKIGVQMQIGGSDQYGNIVSGIESLKTIRETEEAPYARM&#xA;ESGWQHDPIGFTVPLLTDSAGVKFGKSAGNAIWLDEFKTSAFELYGYFMRRTDDEVEKLL&#xA;KLFTFLPMPKIQSIMEEHNADPAKRIAQHNLAFEFVSLIHGSQKALEEAQQHSFRFGGEL&#xA;PSIVKEPAASSGIINANTAPRSDIKLPRSVMKLSPAKLLFATGLAPSAAEGQRLVKMQGA&#xA;YVAAQPGQKRGLVPGNLNWTPMKMWYPEETSKFLLDDRMLILRKGKHNVRIVELVDDEEW&#xA;KASGEFYPGEPYTGEVRRRKQEIIAQAAEDGKTLTMTQVNKMLKKKGRQEGLRVANNPDI&#xA;ELPTKTEVRERVRLTHMHKKEERRTRGDEHEW*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My current code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;&#xA;myprotein = &quot;&quot;&#xA;my_protein = []&#xA;for protein in myprotein:&#xA;    myprotein = protein.rstrip('\n')&#xA;    my_protein.append(myprotein)&#xA;    my_pro = (''.join(my_protein))&#xA;&#xA;   #cleaves sequence    &#xA;peptides = re.sub(r'(?&amp;lt;=[^HRK][^P][^R])(?=[FLWY][^P])|(?&amp;lt;=[^HRK][^P][FLWY])(?=.[^P]?)','\n', my_pro)&#xA;&#xA;print (peptides)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the output is like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MSS&#xA;F&#xA;TRAIRPAL&#xA;QGGRRVLL&#xA;AQRL&#xA;STPRIAHPIAATA&#xA;L&#xA;RS&#xA;LSTTAPRRSVD&#xA;L&#xA;SE&#xA;L&#xA;QPTPITH&#xA;L&#xA;SEVETAMADAVSK&#xA;F&#xA;ATEVVL&#xA;PKAREMDEAEAMDPAVVEQ&#xA;L&#xA;F&#xA;EQG&#xA;L&#xA;MGIEIPEE&#xA;Y&#xA;GGAGMN&#xA;F&#xA;TAAIIAIEE&#xA;L&#xA;ARADPSVSV&#xA;L&#xA;VDVHNTL&#xA;CNTA&#xA;L&#xA;MK&#xA;W&#xA;GSDA&#xA;L&#xA;KKKYLPRL&#xA;ATNTVAS&#xA;F&#xA;C&#xA;L&#xA;SEPVSGSDA&#xA;F&#xA;A&#xA;L&#xA;ATRATETPDG&#xA;Y&#xA;TITGSKM&#xA;WITNSMEAG&#xA;F&#xA;F&#xA;IV&#xA;F&#xA;AN&#xA;L&#xA;DPAKG&#xA;YKGITA&#xA;F&#xA;VVDKDTPGF&#xA;SIAKKEKK&#xA;LGIRASSTCV&#xA;L&#xA;N&#xA;F&#xA;DDVAVPKENL&#xA;L&#xA;GKPGEG&#xA;Y&#xA;K&#xA;Y&#xA;AIGI&#xA;L&#xA;NEGRIGIAAQMTG&#xA;L&#xA;A&#xA;L&#xA;GA&#xA;F&#xA;ENAVRY&#xA;VW&#xA;NDRKQFGT&#xA;L&#xA;VGE&#xA;F&#xA;QGMQHQIAQS&#xA;Y&#xA;TEIAAARA&#xA;LV&#xA;Y&#xA;NAARKKEAGED&#xA;F&#xA;VRDAAMAK&#xA;L&#xA;YASQVAGRVAG&#xA;L&#xA;AVE&#xA;W&#xA;MGGMG&#xA;F&#xA;VREGL&#xA;AEK&#xA;Y&#xA;WRDSKIGAI&#xA;Y&#xA;EGTSNIQ&#xA;L&#xA;NTIAK&#xA;L&#xA;LQKE&#xA;YTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I need every sequence on a separate line, starting with the sequence header like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;jgi|Cormi1|6|CCM_00006m.01 frag.1&#xA;MSS&#xA;&amp;gt;jgi|Cormi1|6|CCM_00006m.01 frag.2&#xA;F&#xA;&amp;gt;jgi|Cormi1|6|CCM_00006m.01 frag.3&#xA;TRAIRPAL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7Z3tv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7Z3tv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;How could I make this work?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import sys&#xA;import re&#xA;&#xA;sequence = open(input('Enter input filename: '),'r')&#xA;def digest(sequence):&#xA;    return re.sub(r'(?&amp;lt;=[^HRK][^P][^R])(?=[FLWY][^P])|(?&amp;lt;=[^HRK][^P][FLWY])(?=.[^P]?)','\n', sequence)&#xA;&#xA;seq = &quot;&quot;&#xA;header = None&#xA;for line in sys.stdin:&#xA;    if not line.startswith('&amp;gt;'):&#xA;        seq = seq + line.rstrip()&#xA;    else:&#xA;        if header:&#xA;            for idx, fragment in enumerate(digest(seq).split('\n')):&#xA;                sys.stdout.write(&quot;%s frag.%d\n%s\n&quot; % (header, idx + 1, fragment))&#xA;            seq = &quot;&quot;&#xA;        header = line.rstrip()&#xA;&#xA;# final pass&#xA;for idx, fragment in enumerate(digest(seq).split('\n')):&#xA;    sys.stdout.write(&quot;%s frag.%d\n%s\n&quot; % (header, idx + 1, fragment))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2523" LastEditorUserId="2523" LastEditDate="2018-04-02T12:06:05.417" LastActivityDate="2018-04-02T12:06:05.417" Title="I need to use my code with fasta format. Can anybody help?" Tags="|python|fasta|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3969" PostTypeId="2" ParentId="3968" CreationDate="2018-03-29T16:32:52.570" Score="5" Body="&lt;p&gt;One way is to store the header as a separate string from your protein residues. Use a list comprehension to build a list of formatted header-fragment pairs, and then print them to standard output.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python                                                                                                                                                                                                                                                    &#xA;&#xA;import sys&#xA;import re&#xA;&#xA;myprotein = &quot;&amp;gt;jgi|Cormi1|6|CCM_00006m.01\nMSSFTRAIRPALQGGRRVLLAQRLSTPRIAHPIAATALRSLSTTAPRRSVDLSELQPTPI\nTHLSEVETAMADAVSKFATEVVLPKAREMDEAEAMDPAVVEQLFEQGLMGIEIPEEYGGA\nGMNFTAAIIAIEELARADPSVSVLVDVHNTLCNTALMKWGSDALKKKYLPRLATNTVASF\nCLSEPVSGSDAFALATRATETPDGYTITGSKMWITNSMEAGFFIVFANLDPAKGYKGITA\nFVVDKDTPGFSIAKKEKKLGIRASSTCVLNFDDVAVPKENLLGKPGEGYKYAIGILNEGR\nIGIAAQMTGLALGAFENAVRYVWNDRKQFGTLVGEFQGMQHQIAQSYTEIAAARALVYNA\n&quot;&#xA;&#xA;# parse record&#xA;my_protein = []&#xA;for line in myprotein.split('\n'):&#xA;    if not line.startswith('&amp;gt;'):&#xA;        my_protein.append(line)&#xA;    else:&#xA;        header = line&#xA;&#xA;# build sequence&#xA;my_pro = ''.join(my_protein)&#xA;&#xA;# cleave sequence                                                                                                                                                                                                                                                        &#xA;peptides = re.sub(r'(?&amp;lt;=[^HRK][^P][^R])(?=[FLWY][^P])|(?&amp;lt;=[^HRK][^P][FLWY])(?=.[^P]?)','\n', my_pro)&#xA;&#xA;# build a formatted list&#xA;records = [&quot;%s frag.%d\n%s\n&quot; % (header, idx + 1, fragment) for idx, fragment in enumerate(peptides.split('\n'))]&#xA;&#xA;# print list&#xA;for record in records:&#xA;    sys.stdout.write(&quot;%s&quot; % (record))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This gives:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;jgi|Cormi1|6|CCM_00006m.01 frag.1&#xA;MSS&#xA;&amp;gt;jgi|Cormi1|6|CCM_00006m.01 frag.2&#xA;F&#xA;&amp;gt;jgi|Cormi1|6|CCM_00006m.01 frag.3&#xA;TRAIRPAL&#xA;...&#xA;&amp;gt;jgi|Cormi1|6|CCM_00006m.01 frag.89&#xA;LV&#xA;&amp;gt;jgi|Cormi1|6|CCM_00006m.01 frag.90&#xA;Y&#xA;&amp;gt;jgi|Cormi1|6|CCM_00006m.01 frag.91&#xA;NA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If you have multiple records in one file called &lt;code&gt;seqs.fa&lt;/code&gt;, then you could use standard input to this script (&lt;em&gt;e.g.&lt;/em&gt;, named &lt;code&gt;digest.py&lt;/code&gt;) to process them all at once:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python                                                                                                                                                                                                                                                    &#xA;&#xA;import sys&#xA;import re&#xA;&#xA;def digest(sequence):&#xA;    return re.sub(r'(?&amp;lt;=[^HRK][^P][^R])(?=[FLWY][^P])|(?&amp;lt;=[^HRK][^P][FLWY])(?=.[^P]?)','\n', sequence)&#xA;&#xA;seq = &quot;&quot;&#xA;header = None&#xA;for line in sys.stdin:&#xA;    if not line.startswith('&amp;gt;'):&#xA;        seq = seq + line.rstrip()&#xA;    else:&#xA;        if header:&#xA;            for idx, fragment in enumerate(digest(seq).split('\n')):&#xA;                sys.stdout.write(&quot;%s frag.%d\n%s\n&quot; % (header, idx + 1, fragment))&#xA;            seq = &quot;&quot;&#xA;        header = line.rstrip()&#xA;&#xA;# final pass&#xA;for idx, fragment in enumerate(digest(seq).split('\n')):&#xA;    sys.stdout.write(&quot;%s frag.%d\n%s\n&quot; % (header, idx + 1, fragment))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Creating a function called &lt;code&gt;digest&lt;/code&gt; lets you change the digestion rules in one place. It reduces code bloat and also reduces errors from changing things in one place and not another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could use this script like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ./digest.py &amp;lt; seqs.fa &amp;gt; seqs.digested.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The file &lt;code&gt;seqs.digested.fa&lt;/code&gt; would look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat seqs.digested.fa&#xA;&amp;gt;jgi|Cormi1|6|CCM_00006m.01 frag.1&#xA;MSS&#xA;&amp;gt;jgi|Cormi1|6|CCM_00006m.01 frag.2&#xA;F&#xA;&amp;gt;jgi|Cormi1|6|CCM_00006m.01 frag.3&#xA;TRAIRPAL&#xA;...&#xA;&amp;gt;jgi|Cormi1|100|CCM_00100m.01 frag.143&#xA;PTKTEVRERVRL&#xA;&amp;gt;jgi|Cormi1|100|CCM_00100m.01 frag.144&#xA;THMHKKEERRTRGDEHE&#xA;&amp;gt;jgi|Cormi1|100|CCM_00100m.01 frag.145&#xA;W*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-03-30T19:44:45.500" LastActivityDate="2018-03-30T19:44:45.500" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="3971" PostTypeId="1" CreationDate="2018-03-30T11:20:03.800" Score="2" ViewCount="120" Body="&lt;p&gt;I need tools I am writing to output data in terms of score for each base in the genome. The format needs to be indexed for fast random access. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The obvious choices are bigWig and tabix-index bedgraph (indexed bedGraph seems quicker for computation, bigWig better for visualization), but the problem is that I need the data to be stranded - i.e. I need to record which strand the data belongs to (and perhaps even have a score on the positive and negative strands). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my choices are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Invent a new format&lt;/li&gt;&#xA;&lt;li&gt;Bodge one of the existing formats (this is almost as bad as inventing a new format, if not worse). For example: bedGraph, except with two score columns - one for + and one for -, or have a bedGraph with with a strand column. This doesn't seem ideal because other downstream tools won't be able to process it. &lt;/li&gt;&#xA;&lt;li&gt;Have seperate files for + and - strand scores. This is unweidly and not great if you sometimes don't care about strand, or want to visualise on a single track.&lt;/li&gt;&#xA;&lt;li&gt;Use a magic format I've not heard of but that someone is going to point out in the answers...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;None of these are ideal. As they either make tool output files incompatible with other tools, or don't encode all the information I want to. Is there a an established format I can use for this?&lt;/p&gt;&#xA;" OwnerUserId="235" LastEditorUserId="298" LastEditDate="2018-03-30T13:24:16.803" LastActivityDate="2018-03-30T13:24:16.803" Title="Which data format for stranded coverage data" Tags="|file-formats|bigwig|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3972" PostTypeId="2" ParentId="2628" CreationDate="2018-03-30T16:08:15.067" Score="4" Body="&lt;p&gt;In this answer I want to show you some benchmarks that compares three different serial ways of reading data in C++, the third one being the fastest. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the first method I use an &lt;code&gt;std::istreambuf_iterator&lt;/code&gt;, in the second I read the file line by line into a &lt;code&gt;std::vector&lt;/code&gt; and in the third one I use C flavoured operations.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/*&#xA;compile with g++ -std=c++11 -O2 reading_files.cpp -o reading_files&#xA;test the program with the following&#xA;dd if=/dev/random of=./dummy5GiB bs=1024 count=$[1024*1024*5]&#xA;	for i in 0 1 2; do time ./reading_files dummy5GiB $i; done&#xA;*/&#xA;&#xA;#include &amp;lt;fstream&amp;gt;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;vector&amp;gt;&#xA;#include &amp;lt;cstring&amp;gt; //for strcmp&#xA;#include &amp;lt;string&amp;gt;&#xA;#include &amp;lt;algorithm&amp;gt; //for min&#xA;&#xA;using namespace std;&#xA;&#xA;int main(int argc, char * argv[])&#xA;{&#xA;    if(argc != 3) {&#xA;        cerr &amp;lt;&amp;lt; &quot;usage: reading_files &amp;lt;path&amp;gt; &amp;lt;number 0 or 1 or 2&amp;gt;\n&quot;;&#xA;        return 1;&#xA;    }&#xA;&#xA;    ifstream file(argv[1], ios::in|ios::binary); &#xA;    if(!file.is_open()) {&#xA;        cerr &amp;lt;&amp;lt; argv[1] &amp;lt;&amp;lt; &quot; file not opened!\n&quot;;&#xA;        return 1;&#xA;    }&#xA;    cout &amp;lt;&amp;lt; &quot;---------------------------\n&quot;;&#xA;    if(!strcmp(argv[2],&quot;0&quot;)) {&#xA;        cout &amp;lt;&amp;lt; &quot;method 0\n&quot;;&#xA;        vector&amp;lt;char&amp;gt; content((std::istreambuf_iterator&amp;lt;char&amp;gt;(file)), std::istreambuf_iterator&amp;lt;char&amp;gt;());&#xA;        cout &amp;lt;&amp;lt; content.size() &amp;lt;&amp;lt; &quot; bytes read\n&quot;;&#xA;    } else if(!strcmp(argv[2],&quot;1&quot;)) {&#xA;        cout &amp;lt;&amp;lt; &quot;method 1\n&quot;;&#xA;        string line;&#xA;        vector&amp;lt;string&amp;gt; content;&#xA;        while(getline(file,line)) {&#xA;            content.push_back(line);&#xA;        }&#xA;        cout &amp;lt;&amp;lt; content.size() &amp;lt;&amp;lt; &quot; lines read\n&quot;;&#xA;    } else if(!strcmp(argv[2],&quot;2&quot;)) {&#xA;        cout &amp;lt;&amp;lt; &quot;method 2\n&quot;;&#xA;        file.seekg(0, std::ios::end);&#xA;        unsigned long long file_size = file.tellg();&#xA;        cout &amp;lt;&amp;lt; &quot;file size is &quot; &amp;lt;&amp;lt; file_size &amp;lt;&amp;lt; &quot;\n&quot;;&#xA;        //maximum 5 GiB of memory, so to fit my RAM&#xA;        unsigned long long max_block_size = 1024ULL * 1024ULL * 1024ULL * 5; &#xA;        file.clear();&#xA;        file.seekg(0, std::ios::beg);&#xA;&#xA;        unsigned long long block_size = min(max_block_size,file_size);&#xA;        char * buffer = (char*)malloc(block_size*sizeof(char));&#xA;        unsigned long long processed = 0;&#xA;&#xA;        while(!file.eof()) {&#xA;            file.read(buffer, block_size);&#xA;            cout &amp;lt;&amp;lt; &quot;read a block of &quot; &amp;lt;&amp;lt; min(file_size - processed, block_size) &amp;lt;&amp;lt; &quot; bytes\n&quot;;&#xA;            //do work here with the block in memory (if the block size is &amp;gt; 0)&#xA;            processed += block_size;&#xA;        }&#xA;&#xA;        free(buffer);&#xA;    }&#xA;    file.close();&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the third method I read the file in blocks of 5GiB, so to fit the free RAM of my machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the benchmark of the program on a 5 GiB randomly generated file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bash-3.2$ for i in 0 1 2; do time ./reading_files dummy5GiB $i; done&#xA;---------------------------&#xA;method 0&#xA;5368709120 bytes read&#xA;&#xA;real    0m35.194s&#xA;user    0m19.665s&#xA;sys 0m12.999s&#xA;---------------------------&#xA;method 1&#xA;20973135 lines read&#xA;&#xA;real    0m40.478s&#xA;user    0m34.046s&#xA;sys 0m5.870s&#xA;---------------------------&#xA;method 2&#xA;file size is 5368709120&#xA;read a block of 5368709120 bytes&#xA;read a block of 0 bytes&#xA;&#xA;real    0m4.413s&#xA;user    0m1.757s&#xA;sys 0m2.617s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the benchmark on a real 32GiB .bam file using only the third method, here the splitting of the file read plays a fundamental role. I can't tell you how big because my hard disk memory gets saturated before the end of the execution and so I have to kill the program.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time ./reading_files ./HG00252.mapped.ILLUMINA.bwa.GBR.low_coverage.20130415.bam 2&#xA;---------------------------&#xA;method 2&#xA;file size is 34316054058&#xA;read a block of 5368709120 bytes&#xA;read a block of 5368709120 bytes&#xA;read a block of 5368709120 bytes&#xA;read a block of 5368709120 bytes&#xA;read a block of 5368709120 bytes&#xA;read a block of 5368709120 bytes&#xA;read a block of 2103799338 bytes&#xA;&#xA;real    0m50.656s&#xA;user    0m9.135s&#xA;sys 0m35.728s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The key point is to minimise the file operations reading large block of data at once (as long as they fit the memory) and operate on them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using &lt;code&gt;mmap&lt;/code&gt; could lead to an improvement, but I don't think it would be as large as using the third method instead of the first two.&lt;/p&gt;&#xA;" OwnerUserId="1617" LastEditorUserId="1617" LastEditDate="2018-03-30T20:06:50.703" LastActivityDate="2018-03-30T20:06:50.703" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3973" PostTypeId="2" ParentId="2628" CreationDate="2018-03-30T16:47:21.160" Score="3" Body="&lt;p&gt;If you need to iterate through the entire BAM file, decompression is usually the main bottleneck.  Three things you can do:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Install libdeflate (&lt;a href=&quot;https://github.com/ebiggers/libdeflate&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ebiggers/libdeflate&lt;/a&gt; ) and then the development version of htslib, and make sure the latter is compiled to use libdeflate.  This roughly doubles decompression efficiency in my experience.&lt;/li&gt;&#xA;&lt;li&gt;Call htslib's bgzf_mt() function right after opening the file for reading, to request multiple decompressor threads.  I generally go with 2-6, depending on what I'm actually doing with the reads.&lt;/li&gt;&#xA;&lt;li&gt;Have a separate read/decompress-ahead thread perform the bam_read1() calls.  Ideally, when your main thread is processing read #X, background threads are already working on decompressing reads e.g. (X+100) through (X+200).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Also, if your workload is &quot;embarrassingly parallel&quot;, you may also benefit from launching multiple processes simultaneously, configuring each to handle a similarly-sized chunk of the BAM.&lt;/p&gt;&#xA;" OwnerUserId="472" LastActivityDate="2018-03-30T16:47:21.160" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3975" PostTypeId="1" CreationDate="2018-03-31T17:16:31.390" Score="5" ViewCount="118" Body="&lt;p&gt;I have crossposted this question in &lt;a href=&quot;https://math.stackexchange.com/q/2716192&quot;&gt;maths StackExchange&lt;/a&gt;. The problem is dominantly mathematical (this question) but the application of the problem is mainly biological. Hoping that people in this forum have faced similar problems, I am posting this in bioinformatics forum to get some ideas and probably solutions to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working on a problem that allows haplotype phasing. I have developed this computation method (expressed in detail below) and have developed a python code (not shown here) to solve the issue. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I am having a hard time trying to translate this computation in mathematical and/or statistical language. I have revisited maths and I am able to comprehend the process at several places, but I cannot translate those ideas onto my computation (or algorithm).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here is my computation (step by step) with a very workable example. I have tried to make it as comprehensive as possible, let me know if there is any confustion:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fig 01: Example of the haplotype file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;contig   pos     ref   all-alleles   ms01e_PI   ms01e_PG_al   ms02g_PI   ms02g_PG_al   ms03g_PI   ms03g_PG_al   ms04h_PI   ms04h_PG_al   ms05h_PI   ms05h_PG_al   ms06h_PI   ms06h_PG_al&#xA;2      15881764   .      .           4           C|T           6           C|T           7           T|T           7           T|T           7           C|T           7           C|T&#xA;2      15881767   .      .           4           C|C           6           T|C           7           C|C           7           C|C           7           T|C           7           C|C&#xA;2      15881989   .      .           4           C|C           6           A|C           7           C|C           7           C|C           7           A|T           7           A|C&#xA;2      15882091   .      .           4           G|T           6           G|T           7           T|A           7           A|A           7           A|T           7           A|C&#xA;2      15882451   .      .           4           C|T           4           T|C           7           T|T           7           T|T           7           C|T           7           C|A&#xA;2      15882454   .      .           4           C|T           4           T|C           7           T|T           7           T|T           7           C|T           7           C|T&#xA;2      15882493   .      .           4           C|T           4           T|C           7           T|T           7           T|T           7           C|T           7           C|T&#xA;2      15882505   .      .           4           A|T           4           T|A           7           T|T           7           T|T           7           A|C           7           A|T &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This (fig 01) is my main data file. The main idea is to take a pool of several samples which have short phased haplotype blocks represented as &lt;code&gt;PI (phased index)&lt;/code&gt; and  &lt;code&gt;PG_al  (phased genotype)&lt;/code&gt;. For any sample, the site where two consecutive haplotype blocks are not joined represents the break point (and has different &lt;code&gt;PI&lt;/code&gt; values). In the above example only sample &lt;code&gt;ms02g&lt;/code&gt; has break point and needs to be phased. All other samples have haplotype block that bridges these two consecutive haplotype blocks and contains the data required to extend the phase state of &lt;code&gt;sample ms02g&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fig 02: Representing a break-point in the sample: ms02g&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;contig    pos    ms02g_PI    ms02g_PG_al&#xA;2     15881764     6         C|T&#xA;2     15881767     6         T|C&#xA;2     15881989     6         A|C&#xA;2     15882091     6         G|T&#xA;                   ×——————————×—————&amp;gt; Break Point&#xA;2     15882451     4         T|C&#xA;2     15882454     4         T|C &#xA;2     15882493     4         T|C&#xA;2     15882505     4         T|A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, in the above haplotype file there is a breakpoint in &lt;code&gt;sample ms02g&lt;/code&gt; at the position &lt;em&gt;15882091-15882451&lt;/em&gt;. For sample  ms02g(PI-6) the haplotypes are  &lt;code&gt;C-T-A-G&lt;/code&gt; and  &lt;code&gt;T-C-C-T&lt;/code&gt;. Similarly, at PI-4 the haplotypes are &lt;code&gt;T-T-T-T&lt;/code&gt; and &lt;code&gt;C-C-C-A&lt;/code&gt;. Since the haplotype is broken in two levels, we don’t know which phase from level-6 goes with which phase of level-4. But, all other samples have full haplotype intact that bridges this position. We can therefore use this information from other samples to join the two consecutive haplotype in sample ms02g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using markov-chain transition to extend the haplotype block: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since, all other samples are completely phased bridging that breakpoint, I can run a markov-chain transition probabilities to solve the phase state in the &lt;code&gt;sample ms02g&lt;/code&gt; . To the human eye/mind you can clearly see and say that left part of ms02g (PI-6, i.e C-T-A-G is more likely to go with right block of PI-4 C-C-C-A), thereby creating the extended haplotype block as  &lt;code&gt;C-T-A-G-C-C-C-A&lt;/code&gt; and &lt;code&gt;T-C-C-T-T-T-T-T&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below, I show how I can apply the first order markov transition matrix  to compute the likelyhood estimates, calculate the log2Odds and then assign and extend the haplotype in proper configuration. And, I feed this logic to the computer using python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Calculation of likelyhood estimates using markov transition:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Step 01:   prepare required haplotype configuration&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The top &lt;code&gt;PI-6 is Block 01&lt;/code&gt; and the bottom &lt;code&gt;PI-4 is Block 02&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The phased haplotype in the left within each block is &lt;code&gt;Hap-A&lt;/code&gt; and on the right is &lt;code&gt;Hap-B&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Fig 03: representation of the haplotype breakpoint and block assignment&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ms02g_PI    ms02g_PG_al&#xA;  6         C|T \&#xA;  6         T|C | Block 1&#xA;  6         A|C |&#xA;  6         G|T /&#xA;  ×——————————×—————&amp;gt; Break Point&#xA;  4         T|C \&#xA;  4         T|C |&#xA;  4         T|C | Block 2&#xA;  4         T|A /&#xA;           ↓   ↓&#xA;       Hap-A   Hap-B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, the two consecutive blocks can be extended in one of the two possible haplotype configurations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Parallel Configuration:&#xA; Block01-HapA with Block02-HapA, so B01-HapB with B02-HapB &#xA;Vs. Alternate Configuration:&#xA; Block01-HapA with Block02-HapB, so B01-HapB with B02-HapA &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Step 02: Compute transition matrix and estimate likelihood for each configuration.&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Fig 04: Representation of the allele transition matrix (from alleles of former block-01 to alleles of later block-02).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Possible&#xA;transitions     ms02g_PG_al&#xA; │              ┌┬┬┬  C|T \&#xA; │              ││││  T|C | Block 1&#xA; │              ││││  A|C |&#xA; │              ││││  G|T /&#xA; └────────────&amp;gt; ││││   ×—————&amp;gt; Break Point&#xA;                │││└&amp;gt; T|C \&#xA;                ││└─&amp;gt; T|C | Block 2&#xA;                │└──&amp;gt; T|C |&#xA;                └───&amp;gt; T|A /&#xA;                     ↓   ↓ &#xA;                 Hap-A   Hap-B&#xA;&#xA;**note: In this example I am showing transition only from first &#xA;  nucleotide of Block01 to first nucleotide of Block02. Actually we &#xA;  prepare transition from all nucleotides of B01 to all nucleotide of &#xA;  B02 in both parallel and alternate configuration. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I count the number of transitions from each nucleotide of PI-6 to each nucleotide of PI-4 for each haplotype configuration across all the samples and convert them to transition probabilities. &lt;/li&gt;&#xA;&lt;li&gt;And multiply the transition probabilities from the first nucleotide in PI-6 to all nucleotides of PI-4. Then similarly multiply the transition probability from 2nd nucleotide of PI-6 to all nucleotides in PI-4, and so on.&lt;/li&gt;&#xA;&lt;li&gt;When transition probabilities are calculated for all possible combination (from each position of PI-6 to each position of PI-4), then I compute the cumulative transition probabilities for each possible haplotype configuration.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Fig 05 : Representation of nucleotide counts (emission counts) at positions 15881764 and 15882451.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pos\allele     A    T    G    C&#xA;15881764       0    8    0    4&#xA;15882451       1    7    0    4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Fig 06 : Representation of transition matrix counts (from pos 15881764) to (pos 15882451).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This transition matrix is computed from nucleotides (A,T,G,C) at block01 to nucleotides (A,T,G,C) at block02 for all the positions. Transition counts are then converted to transition probabilities.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from     to&#xA;          A    T    G    C&#xA;  A       0    0    0    0&#xA;  T       1    6.5  0    0.5&#xA;  G       0    0    0    0&#xA;  C       0    0.5  0    3.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note: if the PI matches between two blocks the transition are counted &#xA;      as 1, else 0.5. Sample ms02g itself is also taken as an observation.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Step 03: Compute the maximul likelihood for each configuration&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Fig 06 : Likelihood estimate for parallel configuration using transition counts (probabilities)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Parallel configuration:&#xA;Block-1-Hap-A (C-T-A-G) with Block-2-Hap-A (T-T-T-T)&#xA;  CtT × CtT × CtT × CtT = (0.5/4)*(0.5/4)*(0.5/4)*(0.5/4) = 0.000244&#xA;+ TtT × TtT × TtT × TtT = (0.5/2)*(0.5/2)*(0.5/2)*(0.5/2) = 0.003906&#xA;+ AtT × AtT × AtT × AtT = (0.5/3)*(0.5/3)*(0.5/3)*(0.5/3) = 0.0007716&#xA;+ GtT × GtT × GtT × GtT = (0.5/2)*(0.5/2)*(0.5/2)*(0.5/2) = 0.003906&#xA;——————— ————————— ——————— ————————— Max Sum (likelihoods) = 0.008828&#xA;                                    Average (likelihoods) = 0.002207&#xA;&#xA;Block-1-Hap-B (T-C-C-T) with Block-2-Hap-B (C-C-C-A)&#xA;  TtC × TtC × TtC × TtA = (0.5/8)*(0.5/8)*(0.5/8)*(0.5/8) = 0.00001526&#xA;+ CtC × CtC × CtC × CtA = (2.5/10)*(2.5/10)*(2.5/10)*(2.5/10) = 0.003906&#xA;+ CtC × CtC × CtC × CtA = (1.5/8)*(1.5/8)*(1.5/8)*(1.5/8) = 0.001236&#xA;+ TtC × TtC × TtC × TtA = (0.5/4)*(0.5/4)*(0.5/4)*(0.5/4) = 0.000244&#xA;——————— ————————— ——————— ————————— Max Sum (likelihoods) = 0.0054016 &#xA;                                    Average (likelihoods) = 0.0013504&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;note: &#xA;      - &quot;AtC&quot; -&gt; represent &quot;A&quot; to &quot;C&quot; transition&#xA;      - &quot;+&quot; represents the summation of the likelyhoods&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fig 07 : Likelihood estimate for alternate configuration using transition counts (probabilities)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Alternate configuration:&#xA;Block-1-Hap-A (C-T-A-G) with Block-2-Hap-B (C-C-C-A)&#xA;  CtC × CtC × CtC × CtA = (3.5/4)*(3.5/4)*(3.5/4)*(3.5/4) = 0.5861 &#xA;+ TtC × TtC × TtC × TtA = (1.5/2)*(1.5/2)*(1.5/2)*(1.5/2) = 0.3164&#xA;+ AtC × AtC × AtC × AtA = (2.5/3)*(2.5/3)*(2.5/3)*(2.5/3) = 0.4823&#xA;+ GtC × GtC × GtC × GtA = (1.5/2)*(1.5/2)*(1.5/2)*(1.5/2) = 0.3164&#xA;——————— ————————— ——————— ————————— Max Sum (likelyhoods) = 1.7012 &#xA;                                    Average (likelihoods) = 0.425311&#xA;&#xA;Block-1-Hap-B (T-C-C-T) with Block-2-Hap-A (T-T-T-T) &#xA;  TtC × TtC × TtC × TtA = (6.5/8)*(7.5/8)*(7.5/8)*(6.5/8) = 0.5802&#xA;+ CtC × CtC × CtC × CtA = (6.5/10)*(7.5/10)*(7.5/10)*(6.5/10) = 0.237&#xA;+ CtC × CtC × CtC × CtA = (5.5/8)*(6.5/8)*(6.5/8)*(6.5/8) = 0.36875&#xA;+ TtC × TtC × TtC × TtA = (3.5/4)*(3.5/4)*(3.5/4)*(2.5/4) = 0.4187&#xA;——————— ————————— ——————— ————————— Max Sum (likelyhoods) = 1.60465&#xA;                                    Average (likelihoods) = 0.4011625&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;note: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  - the sum of the likelihoods &amp;gt; 1, in the above example.&#xA;  - So, we can rather use the product of the likelyhoods.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Fig 08 : Likelhood estimate of Parallel vs. Alternate configuration.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Likelihood of Parallel vs. Alternate configuration&#xA;&#xA;= likelihood of Parallel config / likelihood of Alternate config&#xA;&#xA;= (0.002207 + 0.0013504)/ (0.425311 + 0.4011625)&#xA;= 0.0043043  (i.e 1/232)&#xA;Therefore, haplotype is 232 times more likely to be phased &#xA;in &quot;alternate configuration&quot;.&#xA;&#xA;log2Odds = log2(0.0043043) = -7.860006&#xA;So, with our default cutoff threshold of &quot;|5|&quot;, we will be extending &#xA;the haplotype in &quot;alternate configuration&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Final Output data:&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;contig   pos   ref   all-alleles   ms02g_PI   ms02g_PG_al&#xA;2   15881764   .    .                  6      C|T&#xA;2   15881767   .    .                  6      T|C&#xA;2   15881989   .    .                  6      A|C&#xA;2   15882091   .    .                  6      G|T&#xA;2   15882451   .    .                  6      C|T&#xA;2   15882454   .    .                  6      C|T&#xA;2   15882493   .    .                  6      C|T&#xA;2   15882505   .    .                  6      A|T&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, how do I translate my overall computation process in mathematical and statistical language. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking to represent computation of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;emission counts (and probabilities)&lt;/li&gt;&#xA;&lt;li&gt;transition counts (and probabilities)&lt;/li&gt;&#xA;&lt;li&gt;summation of the transition counts for each configuration&lt;/li&gt;&#xA;&lt;li&gt;computation of likelihood ratio and it's conversion to log2Odds&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am trying to represent my computation (algorithm) in the way it's shown in these links, but have a hard time doing it. I've read, &lt;a href=&quot;https://www.probabilitycourse.com/chapter11/11_2_3_probability_distributions.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;, and &lt;a href=&quot;https://stats.stackexchange.com/questions/25540/summation-of-conditional-probability-in-bayesian-network&quot;&gt;this&lt;/a&gt;, and &lt;a href=&quot;https://math.stackexchange.com/questions/389062/how-to-write-an-algorithm-as-an-equation&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FYI - I am biologist but have managed to learn python and write a python program to &lt;a href=&quot;https://codereview.stackexchange.com/questions/186396/solve-the-phase-state-between-two-haplotype-blocks-using-markov-transition-proba&quot;&gt;work it out&lt;/a&gt;. The code isn't relevant in this question though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, it's my time to work out some maths and represent my algorithm. &lt;/p&gt;&#xA;" OwnerUserId="1099" LastEditorUserId="939" LastEditDate="2018-04-04T09:49:00.397" LastActivityDate="2018-04-04T09:49:00.397" Title="How to convert the given mathematical computation (on biological problem) to mathematical fomula, equation?" Tags="|python|bioconductor|hidden-markov-models|haplotypes|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3976" PostTypeId="1" CreationDate="2018-04-02T19:00:21.097" Score="2" ViewCount="92" Body="&lt;p&gt;I have a folder containing 500 files from different individuals:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$ ls&#xA;HI.3577.005.Index_25.QPG4_KS2W-35_all_PA&#xA;HI.3577.005.Index_27.QPH4_SD1W-13_all_PA&#xA;HI.3577.005.Index_9.QPA2_KS2W-8_all_PA&#xA;HI.3577.007.Index_1.Q004B_SD1A-1_all_PA&#xA;HI.3577.007.Index_21.Q076C_SD2W-23_all_PA&#xA;HI.3577.007.Index_23.Q079_KS1A-15_all_PA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Each of these individuals belongs to a population (in the example above individuals belong to KS2W, SD1W, KS2W, SD1A, SD2W, and KS1A respectively). Each of my files looks as follows (this sample belongs to from &quot;MK4&quot; population):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$ head HI.3579.001.Index_9.Q089_MK4-4R_all_PA&#xA;Ha8_00040788 1&#xA;Ha4_00024045 1&#xA;Ha4_00025366 1&#xA;Ha16_00022130 0&#xA;Ha16_00023451 0&#xA;Ha8_00040789 1&#xA;Ha4_00025367 1&#xA;Ha4_00024046 0&#xA;Ha16_00022131 1&#xA;Ha16_00023452 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or another sample from MK4 population&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$ head HI.221.009.Index_9.P089_MK4-4R_all_PA&#xA;Ha8_00040788 1&#xA;Ha4_00024045 1&#xA;Ha4_00025366 1&#xA;Ha16_00022130 1&#xA;Ha16_00023451 1&#xA;Ha8_00040789 1&#xA;Ha4_00025367 1&#xA;Ha4_00024046 0&#xA;Ha16_00022131 1&#xA;Ha16_00023452 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the first column is genes and the second columns refers to the presence of that gene (coded as 1) or absence of gene (coded as 0).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For all individuals belonging to each of the populations, I would like to count the total number individuals that have the gene and individuals that do not have the gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, for the 2 examples above, as they both belong to MK4 population my desired output is going to looke like this:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$ head MK4&#xA;Ha8_00040788   2  0&#xA;Ha4_00024045   2  0&#xA;Ha4_00025366   2  0&#xA;Ha16_00022130  1  1 &#xA;Ha16_00023451  1  1&#xA;Ha8_00040789   2  0&#xA;Ha4_00025367   2  0&#xA;Ha4_00024046   0  2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first column is the gene, the second column is the number of individuals in the MK4 population that have the gene and the third column is the number of individuals that do not have that gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have made a file containing population IDs&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$ head pop.txt&#xA;KS2W&#xA;SD1W&#xA;SD1A&#xA;KS1A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest me how I could do this? thanks&lt;/p&gt;&#xA;" OwnerUserId="1749" LastEditorUserId="292" LastEditDate="2018-04-04T16:41:35.550" LastActivityDate="2018-04-04T17:10:01.943" Title="counting individuals with the present gene and absent gene for each population" Tags="|python|perl|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3977" PostTypeId="1" CreationDate="2018-04-02T19:01:38.597" Score="1" ViewCount="88" Body="&lt;p&gt;I have compiled a list of genes that are distinct from 2 different conditions. &#xA;From this list (of about 600 human genes), I want to find out if any of the genes in this list have acetyl transferase function i.e. are acetylases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an online platform where I could say put in this list of genes and the output I can select genes which have this property/function? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I.e some sort of gene annotation for functions for a list of human genes?&lt;/p&gt;&#xA;" OwnerUserId="2593" LastEditorUserId="48" LastEditDate="2018-04-02T21:45:58.120" LastActivityDate="2018-04-03T08:33:56.983" Title="Find specific functional genes from a list?" Tags="|sequence-annotation|functional-annotation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3978" PostTypeId="1" AcceptedAnswerId="3986" CreationDate="2018-04-02T21:53:02.180" Score="3" ViewCount="475" Body="&lt;p&gt;I am running &lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cufflinks&lt;/a&gt; for transcriptome assembly using the .bam file generated by &lt;a href=&quot;https://ccb.jhu.edu/software/hisat2/index.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hisat2&lt;/a&gt;. I tried both bam and sorted bam files&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cufflinks --no-update-check -o ./ -p 15 accepted_hits.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;it gives the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error: sort order of reads in BAMs must be the same&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;UPDATE: It is working fine now. It seems there were some issues with the reference file. The BAM files generated from &lt;a href=&quot;https://ccb.jhu.edu/software/tophat/index.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tophat2&lt;/a&gt; also gave the same error. I downloaded the updated reference file and it worked great.&lt;/p&gt;&#xA;" OwnerUserId="2594" LastEditorUserId="2594" LastEditDate="2018-04-05T19:24:14.973" LastActivityDate="2018-04-05T19:24:14.973" Title="Cufflinks Error: sort order of reads in BAMs must be the same" Tags="|rna-seq|assembly|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3979" PostTypeId="1" AcceptedAnswerId="3984" CreationDate="2018-04-03T03:49:40.190" Score="7" ViewCount="665" Body="&lt;p&gt;I am confused regarding how various software libraries deal with optional fields in a BAM:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based upon the &lt;a href=&quot;https://github.com/samtools/hts-specs&quot; rel=&quot;noreferrer&quot;&gt;BAM specification&lt;/a&gt;, there are 11 mandatory fields to a BAM:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;QNAME, FLAG, RNAME, POS, MAPQ, CIGAR, RNEXT, PNEXT, TLEN, SEQ, QUAL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The BAM files of today normally have many optional fields as well, which must be of the format &lt;code&gt;TAG:TYPE:VALUE&lt;/code&gt;, whereby &lt;code&gt;TAG&lt;/code&gt; is a two-character string that matches &lt;code&gt;/[A-Za-z][A-Za-z0-9]/&lt;/code&gt; and &lt;code&gt;TYPE&lt;/code&gt; is a single case-sensitive letter which defines the format of &lt;code&gt;VALUE&lt;/code&gt;. Each &lt;code&gt;TAG&lt;/code&gt; can only appear once in one alignment line.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Certain BAMs may have no optional fields, while others have several. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding is that optional tags may be in a different order from row to row, and then sometimes they simply do not exist. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How does &lt;a href=&quot;https://github.com/samtools/htslib&quot; rel=&quot;noreferrer&quot;&gt;htslib&lt;/a&gt;/&lt;a href=&quot;https://github.com/samtools/samtools&quot; rel=&quot;noreferrer&quot;&gt;samtools&lt;/a&gt; (e.g. &lt;code&gt;samtools view&lt;/code&gt;) access these optional fields, which differ in number from BAM to BAM (possible row to row)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;code&gt;htslib/sam.h&lt;/code&gt;, a BAM alignment struct is defined as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/*! @typedef&#xA; @abstract Structure for one alignment.&#xA; @field  core       core information about the alignment&#xA; @field  l_data     current length of bam1_t::data&#xA; @field  m_data     maximum length of bam1_t::data&#xA; @field  data       all variable-length data, concatenated; structure: qname-cigar-seq-qual-aux&#xA;typedef struct {&#xA;    bam1_core_t core;&#xA;    int l_data;&#xA;    uint32_t m_data;&#xA;    uint8_t *data;&#xA;#ifndef BAM_NO_ID&#xA;    uint64_t id;&#xA;#endif&#xA;} bam1_t;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and followed by several methods which access mandatory BAM fields, e.g. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/*! @function&#xA; @abstract  Get the name of the query&#xA; @param  b  pointer to an alignment&#xA; @return    pointer to the name string, null terminated&#xA; */&#xA;#define bam_get_qname(b) ((char*)(b)-&amp;gt;data)&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How does htslib access each of the optional fields, given this &lt;code&gt;structure&lt;/code&gt;? Does it keep the order of the optional fields, or is this information unavailable? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When a user runs htslib or even &lt;code&gt;samtools view&lt;/code&gt;, how does the BAM row &quot;end&quot;, i.e. what data structure exists to show that the BAM row has no more optional fields, and that the algorithm should &quot;move on&quot; to the next BAM row. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Any clarification for how the source code works in this case is much appreciated. &lt;/p&gt;&#xA;" OwnerUserId="1770" LastEditorUserId="29" LastEditDate="2018-04-03T10:55:56.833" LastActivityDate="2018-04-03T18:04:13.697" Title="How does htslib/samtools access optional BAM fields?" Tags="|sam|samtools|algorithms|htslib|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3980" PostTypeId="1" CreationDate="2018-04-03T06:18:25.510" Score="0" ViewCount="73" Body="&lt;p&gt;I am trying to analyze my .fastq files (MiSeq exon reads) with the jMHC software, but output files I have obtained have no results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think I might be doing something wrong during the first step: &quot;connect to database&quot;. &#xA;Because on the &quot;import/extracting&quot; step, I open all my .fastq files, click &quot;Start&quot;, and it shows this message: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Success! Processed: 139, Imported: 0&quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So I assume it is reading the files OK, but not importing them to the &quot;database&quot;? Could it be the extension of my files? The software says it operates on FASTA files. Are .fastq files not compatible?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anybody that has successfully run this program could give me a hand?&lt;/p&gt;&#xA;" OwnerUserId="2596" LastEditorUserId="57" LastEditDate="2018-04-03T12:45:13.717" LastActivityDate="2024-01-23T07:05:00.717" Title="Error while Importing fastq sequences to jMHC software" Tags="|sequence-analysis|errors|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3981" PostTypeId="1" CreationDate="2018-04-03T08:25:02.143" Score="2" ViewCount="1728" Body="&lt;p&gt;I'm using hisat2, stringtie tools for the RNA-Seq analysis. After stringtie using ballgown I get FPKM and TPM values for every gene. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen that edgeR, Deseq2 can be used for Counts data. I would like to know which R package needs to be used for differential analysis with TPM values?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which one is better for differential analysis FPKM or TPM?&lt;/p&gt;&#xA;" OwnerUserId="1369" LastEditorUserId="73" LastEditDate="2018-04-06T20:28:23.243" LastActivityDate="2018-04-06T20:28:23.243" Title="Which R package to use for differential analysis with TPM values?" Tags="|rna-seq|sam|samtools|differential-expression|edger|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3982" PostTypeId="2" ParentId="3977" CreationDate="2018-04-03T08:33:56.983" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=QvGT2G0-hYA&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioMart&lt;/a&gt;. Filter by your list of genes and a &lt;a href=&quot;http://amigo.geneontology.org/amigo/search/ontology?q=acetyl%20transferase&quot; rel=&quot;nofollow noreferrer&quot;&gt;GO term&lt;/a&gt; linked to acetyl transferase activity.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2018-04-03T08:33:56.983" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3983" PostTypeId="2" ParentId="3979" CreationDate="2018-04-03T09:42:05.730" Score="3" Body="&lt;ol&gt;&#xA;&lt;li&gt;In a BAM file, all of the auxiliary tags (the optional fields you refer to) are held what is effectively a &lt;code&gt;void *&lt;/code&gt; field, one after the other. Each entry within this field is structured as a two &lt;code&gt;char&lt;/code&gt;s storing the tag name (e.g., &lt;code&gt;NM&lt;/code&gt;, followed by a &lt;code&gt;char&lt;/code&gt; storing the type (e.g., &lt;code&gt;Z&lt;/code&gt; or &lt;code&gt;i&lt;/code&gt;), followed by the data in whatever format is appropriate (&lt;code&gt;char&lt;/code&gt;, &lt;code&gt;float&lt;/code&gt;, etc). Strings in the aux tags are &lt;code&gt;NULL&lt;/code&gt; terminated, so their lengths aren't explicitly stored.&lt;/li&gt;&#xA;&lt;li&gt;To access a given field, htslib has to iterate over all of them, in whatever order they happen to be stored, until it finds the correct one. The output order with &lt;code&gt;samtools view&lt;/code&gt; is the order in which they're stored.&lt;/li&gt;&#xA;&lt;li&gt;The size of &lt;code&gt;*data&lt;/code&gt; is stored as &lt;code&gt;l_data&lt;/code&gt;, so once it hits that then it knows that the record is finished.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Regarding point 1, this is a standard C trick of casting between data types. So, while &lt;code&gt;*data&lt;/code&gt; is actually a &lt;code&gt;uint8_t *&lt;/code&gt; (aka, a byte array), the first two bytes of each tag are &lt;code&gt;char&lt;/code&gt;s and whatever is after them (well, after the byte specifying the type) is then cast to the appropriate type (be it &lt;code&gt;float&lt;/code&gt; or &lt;code&gt;char*&lt;/code&gt; or what not). This is all probably clearer if you &lt;a href=&quot;https://github.com/samtools/htslib/blob/develop/sam.c#L1727-L1758&quot; rel=&quot;nofollow noreferrer&quot;&gt;look at the code&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-04-03T18:04:13.697" LastActivityDate="2018-04-03T18:04:13.697" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3984" PostTypeId="2" ParentId="3979" CreationDate="2018-04-03T10:53:37.280" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;How does htslib access each of the optional fields, given this structure?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;htslib uses a macro, &lt;code&gt;bam_get_aux&lt;/code&gt;, for this purpose. You can check the source code of this macro — it essentially just advances the data pointer of the current record to the start of the optional data, which is after all other data. But you shouldn’t care about this, and just use the relevant exported functions for reading optional (“auxiliary”) data:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;bam_aux_get&lt;/code&gt; returns a pointer to an optional record, given by its two-character tag name.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;bam_aux2‹C›&lt;/code&gt; converts the pointer returned by the previous function to the actual type of the optional tag; here, &lt;code&gt;‹C›&lt;/code&gt; is a type identifier: &lt;code&gt;i&lt;/code&gt; for an integer, &lt;code&gt;Z&lt;/code&gt; for a zero-terminated character string, etc. Refer to the documentation and the &lt;code&gt;htslib/sam.h&lt;/code&gt; file for a complete list.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Does it keep the order of the optional fields, or is this information unavailable?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As long as you don’t modify optional fields, the order is kept, but unspecified. &lt;em&gt;Do not rely on the order&lt;/em&gt;!&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;When a user runs htslib or even &lt;code&gt;samtools view&lt;/code&gt;, how does the BAM row &quot;end&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;BAM records store a header that contains the record size.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2018-04-03T10:53:37.280" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="3985" PostTypeId="2" ParentId="3956" CreationDate="2018-04-03T11:05:40.010" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/1118581784&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioinformatics and Functional Genomics&lt;/a&gt; by Jonathan Pevsner is a good one, he goes over the biology as well as how the algorithms work, and provides real-world examples.&lt;/p&gt;&#xA;" OwnerUserId="369" LastActivityDate="2018-04-03T11:05:40.010" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3986" PostTypeId="2" ParentId="3978" CreationDate="2018-04-03T12:18:12.840" Score="6" Body="&lt;p&gt;Just add &lt;code&gt;--dta-cufflinks&lt;/code&gt; to your Hisat2 command, so that the output alignment file provides the attributes needed by Cufflinks (XS flags). This should do the job.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;https://ccb.jhu.edu/software/hisat2/manual.shtml&quot; rel=&quot;noreferrer&quot;&gt;manual&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Report alignments tailored specifically for Cufflinks. In addition to&#xA;  what HISAT2 does with the above option (--dta), With this option,&#xA;  HISAT2 looks for novel splice sites with three signals (GT/AG, GC/AG,&#xA;  AT/AC), but all user-provided splice sites are used irrespective of&#xA;  their signals. HISAT2 produces an optional field, XS:A:[+-], for every&#xA;  spliced alignment.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="294" LastEditorUserId="294" LastEditDate="2018-04-03T12:26:01.440" LastActivityDate="2018-04-03T12:26:01.440" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3987" PostTypeId="2" ParentId="3976" CreationDate="2018-04-03T14:37:46.340" Score="1" Body="&lt;p&gt;I can show you how I would approach it in &lt;code&gt;R&lt;/code&gt;. You'll have to adjust and customize it to your own data of course. I only had two files to play with, from your example. Especially for the &lt;code&gt;merge&lt;/code&gt; step you should build a &lt;code&gt;for&lt;/code&gt; loop around it so that it will merge every next file automatically, since you have 500 files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here an example for the KS2W files:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;myFiles &amp;lt;- dir(pattern=&quot;KS2W&quot;)&#xA;&#xA;myFiles&#xA;[1] &quot;HI.3577.005.Index_25.QPG4_KS2W-35_all_PA.txt&quot;&#xA;[2] &quot;HI.3577.005.Index_9.QPA2_KS2W-8_all_PA.txt&quot;  &#xA;&#xA;fileNames &amp;lt;- list()&#xA;&#xA;for(i in 1:2){&#xA;    fileNames[[i]] &amp;lt;- read.table(myFiles[i], sep = &quot; &quot;)&#xA;}  &#xA;&#xA;merged &amp;lt;- merge(fileNames[[1]], fileNames[[2]], by.x = &quot;V1&quot;, by.y = &quot;V1&quot;, all = F, sort = F)&#xA;&#xA;present &amp;lt;- rowSums(merged[,2:3])&#xA;&#xA;zeros &amp;lt;- rowSums(merged[,2:3] == 0)&#xA;&#xA;output &amp;lt;- cbind(present, zeros)&#xA;&#xA;rownames(output) &amp;lt;- merged[,1]&#xA;&#xA;output&#xA;              present zeros&#xA;Ha8_00040788        2     0&#xA;Ha4_00024045        2     0&#xA;Ha4_00025366        2     0&#xA;Ha16_00022130       1     1&#xA;Ha16_00023451       1     1&#xA;Ha8_00040789        2     0&#xA;Ha4_00025367        2     0&#xA;Ha4_00024046        0     2&#xA;Ha16_00022131       2     0&#xA;Ha16_00023452       2     0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To add more files with &lt;code&gt;merge&lt;/code&gt;, you could add something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;merged &amp;lt;- merge(fileNames[[1]], fileNames[[2]], by.x = &quot;V1&quot;, by.y = &quot;V1&quot;, all = F, sort = F)&#xA;&#xA;for(i in 3:30){&#xA;  merged &amp;lt;- merge(merged, fileNames[[i]], by.x = &quot;V1&quot;, by.y = &quot;V1&quot;, all = F, sort = F)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You'll have to adjust all the numbers and coordinates to fit your data.&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2018-04-03T15:37:16.977" LastActivityDate="2018-04-03T15:37:16.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3989" PostTypeId="2" ParentId="2111" CreationDate="2018-04-03T16:24:46.723" Score="3" Body="&lt;p&gt;Amino-acid content (eg Lys, Asp) can have drastic effect on observance in a particular MS-mode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should consider including &lt;code&gt;Peptides::pI()&lt;/code&gt; or &lt;code&gt;Peptides::charge()&lt;/code&gt; to determine the likelihood of (de-)protonation in your calculations.&lt;/p&gt;&#xA;" OwnerUserId="2597" LastActivityDate="2018-04-03T16:24:46.723" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="3990" PostTypeId="1" CreationDate="2018-04-03T16:50:51.937" Score="1" ViewCount="40" Body="&lt;p&gt;I have downloaded the core version of SIFTER. I am not able to run it because I get the following error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Functions gene ontology database does not exist. Get function.ontology from www.geneontology.org&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have put the function.ontology into the folder &lt;code&gt;data&lt;/code&gt;, but I still get the same error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The full error message is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Family set to test&#xA;ERROR: File not found.&#xA;   Functions gene ontology database does not exist. Get function.ontology from www.geneontology.org..&#xA;   To specify files, please use the options listed under Sifter's help by entering:&#xA;    java -jar sifter.jar --help&#xA;&#xA;&#xA;JOB SUMMARY&#xA;FILE ERROR: &#xA;   Protein family file (.pli) does not exist. To obtain a .pli file, use 'pfam2pli' script. See README.txt..&#xA;   To specify files, please use the options listed under Sifter's help by entering:&#xA;    java -jar sifter.jar --help&#xA;Exception in thread &quot;main&quot; java.lang.NullPointerException&#xA;    at java.io.File.&amp;lt;init&amp;gt;(File.java:277)&#xA;    at SifterPipelineObject.initDefaultSettings(SifterPipelineObject.java:85)&#xA;    at SifterPipelineObject.&amp;lt;init&amp;gt;(SifterPipelineObject.java:93)&#xA;    at Sifter.main(Sifter.java:36)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2218" LastEditorUserId="48" LastEditDate="2018-04-04T20:40:46.247" LastActivityDate="2018-04-04T20:40:46.247" Title="How to run SIFTER stand-alone version" Tags="|functional-annotation|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="3991" PostTypeId="1" CreationDate="2018-04-03T17:19:47.600" Score="3" ViewCount="166" Body="&lt;p&gt;One of my dissertation papers is going to involve a GWAS. I have never actually run a GWAS before, and do not know how to estimate the computing resources I need for it. I asked someone on my committee who said &quot;it's all guesswork&quot; but I don't even know how to come up with a reasonable guess!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to put some estimate of how much cluster time I will need (XX cores for YY hours) in order to do a dissertation grant. I've tried Googling and searching literature for how to do an estimate of this, but I'm not finding much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Details:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;GWAS, total n=350,000, 10-15% cases&lt;/li&gt;&#xA;&lt;li&gt;Already imputed by the time I get it, will have about 96 million variants&lt;/li&gt;&#xA;&lt;li&gt;Will be using Plink&lt;/li&gt;&#xA;&lt;li&gt;Cluster nodes have 24 cores, and jobs can use up to 4 gigs of RAM per core. If Plink can handle being spread across nodes, I can use up to 100 cores at once.&lt;/li&gt;&#xA;&lt;li&gt;Time is billed in core-hours, using 20 cores for one hour is equal to using one core for 20 hours&lt;/li&gt;&#xA;&lt;li&gt;We may want to do a few smaller GWAS where we restrict by age (eg. one of all cases and controls under 25, and one of all cases and controls who are 65+)&lt;/li&gt;&#xA;&lt;li&gt;Need to come up with an estimate of cluster time needed for each GWAS that we might run (XX cores for YY hours)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Can you please advise me on how to estimate something like this?&lt;/p&gt;&#xA;" OwnerUserId="2296" LastActivityDate="2024-03-20T14:07:25.943" Title="Estimating computing resources needed for a GWAS?" Tags="|gwas|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="3992" PostTypeId="2" ParentId="3975" CreationDate="2018-04-03T20:28:57.417" Score="1" Body="&lt;p&gt;I am not sure I understood all, but what you do is like doing a Markov chain but instead of saying that the previous position(s) decides the fate of the next one(s) is the block what determines which other block follows it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mathematically I think that could be explained in terms of &lt;a href=&quot;https://www.nature.com/articles/nrg1123&quot; rel=&quot;nofollow noreferrer&quot;&gt;linkage disequilibrium&lt;/a&gt; (and &lt;a href=&quot;https://en.wikipedia.org/wiki/Linkage_disequilibrium&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia page&lt;/a&gt;), but instead of using a single position you use a block. I am not familiarly enough with the field to see if a similar (or this method) has been previously described, but I think you could find inspiration about how to describe it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A related field of work that also deals with this kind of data (and problems) is structural chromatin, where they work with TADs (their blocks of &lt;a href=&quot;https://en.wikipedia.org/wiki/Topologically_associating_domain&quot; rel=&quot;nofollow noreferrer&quot;&gt;Topologically Associating Domains&lt;/a&gt;) and study which ones are near (spatially instead of probabilistic) with others regions, that field could be also interesting to look for formulas and how do they present this kind of analysis.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-04-03T20:28:57.417" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3993" PostTypeId="1" AcceptedAnswerId="3996" CreationDate="2018-04-04T05:03:59.273" Score="8" ViewCount="863" Body="&lt;p&gt;Note this is a question from a lay reader.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've read in the popular press that ~1-4% of the genome of non-African &lt;em&gt;Homo sapiens&lt;/em&gt; is inherited from Neanderthals, or that Melanesians derive a similar amount from the Denisovans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seems like a lot, until you read (elsewhere!) that Neanderthals are 99.7% identical to humans, so most of the fraction inherited is identical in any case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or taking it another way, reading that two individuals will typically have 99.9% identical DNA seems pretty similar, until you realise you are only 0.3% 'different' from a Neanderthal, or 15% 'different' from your dog.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me this use of percentages is confusing, misleading, and open to abuse. When comparing species (for example) I would think that they should speak of differences in terms of  base pairs that we could not otherwise have had. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How should the lay (or popular science) press best communicate similarities or differences in genomes when talking about individuals, races, or species?&lt;/p&gt;&#xA;" OwnerUserId="2603" LastEditorUserId="939" LastEditDate="2018-04-04T08:30:06.570" LastActivityDate="2018-04-04T16:18:52.820" Title="How should the popular press compare similarity of genomes?" Tags="|genome|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="3994" PostTypeId="1" AcceptedAnswerId="4001" CreationDate="2018-04-04T08:45:54.470" Score="9" ViewCount="2500" Body="&lt;p&gt;I have TCGA gene expression data. I'm interested in identifying gene expression signatures using the data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know whether there are any tools or R packages for identifying gene signatures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How are gene signatures different from GSEA?&lt;/p&gt;&#xA;" OwnerUserId="1377" LastEditorUserId="77" LastEditDate="2018-04-04T14:35:48.457" LastActivityDate="2018-06-02T16:05:23.103" Title="How to identify gene expression signatures from gene expression data?" Tags="|rna-seq|r|gene|bioconductor|" AnswerCount="3" CommentCount="11" ContentLicense="CC BY-SA 3.0" />
  <row Id="3995" PostTypeId="2" ParentId="3993" CreationDate="2018-04-04T09:22:14.970" Score="5" Body="&lt;p&gt;I don’t think it’s misleading. Your question seems to be based on the mistaken assumption that the percentages you mention — 1%–4% Neanderthal origin, 99.7% similarity between individuals, etc — should somehow add up to 100%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you note yourself, that’s obviously not the case. There was a similar question to the effect, “&lt;a href=&quot;https://biology.stackexchange.com/q/1981/166&quot;&gt;how can we share 99% of our genes with chimpanzees if we only share 50% of our genes with our mother and 50% with our father?&lt;/a&gt;” and the answer is fundamentally the same. Namely, that these numbers refer to different things. And the DNA or gene content we share with these individuals and groups of individuals isn’t additive, or mutually exclusive. Think of it as a Venn diagram:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RaHa0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RaHa0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Ratios obviously exaggerated, since showing a diagram where mother and father share 99% wouldn’t be very readable.)&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2018-04-04T14:47:36.193" LastActivityDate="2018-04-04T14:47:36.193" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3996" PostTypeId="2" ParentId="3993" CreationDate="2018-04-04T13:41:21.000" Score="7" Body="&lt;p&gt;1–4% is from an evolution point of view. 99.7% is from a sequence point of view. Because they are measuring different things, they can be compatible with each other. The correct interpretation is: 1–4% of a non-African genome is inherited from Neanderthals and the sequence of this 1–4% differs from modern human sequence by 0.3%. PS: so, for each non-African, Neanderthals contributed 1–4% * 0.3% of base pairs not present in modern human. This sounds tiny, but even a single SNP may lead to severe disease. This tiny fraction has known functional impact to diabetes and high-altitude adaption.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Others pointed out two related questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://biology.stackexchange.com/q/1981/166&quot;&gt;How many genes do we share with our mother?&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://biology.stackexchange.com/questions/9172/do-apes-and-humans-share-99-of-dna-or-99-of-genes-what-is-the-difference&quot;&gt;Do apes and humans share 99% of DNA or 99% of genes? What is the difference?&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These questions are really about the ambiguity in the definition of &quot;gene&quot;. Importantly, traditionally in popular science and population genetics, a gene could mean an &lt;a href=&quot;https://en.wikipedia.org/wiki/Allele&quot; rel=&quot;noreferrer&quot;&gt;allele&lt;/a&gt;. This makes differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, by common uses of &quot;gene&quot; in biology, the answer to &quot;how many genes do we share with our mother&quot; would be nearly 100% – every human has nearly the same set of genes. However, clearly this is not what the asker wants. By gene, he/she really means allele. The expected answer is 50% – we inherit 50% of alleles from our mother.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end, what is a &quot;gene&quot; exactly? I don't know. Biologists can often figure out the meaning of word &quot;gene&quot; in the right context, but non-biologists will sometimes have troubles with the ambiguity.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2018-04-04T13:56:45.350" LastActivityDate="2018-04-04T13:56:45.350" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="3997" PostTypeId="2" ParentId="3994" CreationDate="2018-04-04T14:25:13.167" Score="4" Body="&lt;p&gt;If you're interested in an unbiased approach you could have a look at &lt;a href=&quot;https://labs.genetics.ucla.edu/horvath/htdocs/CoexpressionNetwork/Rpackages/WGCNA/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gene Correlation Network Analysis&lt;/a&gt;. WGCNA is a fairly old R package, with a somewhat confusing workflow and results that can be difficult to interpret. It also needs a large sample set, preferably from fairly different biological conditions, to produce meaningful results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Asking how &quot;gene expression signatures&quot; differ from GSEA is like asking how surface textures differ from trademarks. Gene set enrichment analysis is for looking at gene rankings and seeing how they compare to a known / existing gene set. A gene set is considered &quot;enriched&quot; if the majority of the gene set appears in high-ranked genes (or alternatively in low-ranked genes), rather than a more even distribution of the set across the ranked gene list. Most commonly people will rank genes based on expression (or fold change), but the method is agnostic to the nature of the ranking.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-04-04T14:25:13.167" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="3998" PostTypeId="2" ParentId="3993" CreationDate="2018-04-04T16:18:52.820" Score="1" Body="&lt;p&gt;I don't think there's any way to summarize data like this in a way that is both comprehensive in the &quot;covering all relevant facts and concepts&quot; meaning and in the &quot;able to be understood&quot; meaning. Relatedness is not a simple quantity that can be summarized in one statistic. There's what percentage of total genome is shared. There's percentage of &lt;em&gt;coding&lt;/em&gt; genome shared (I would expect the non-coding portion of the genome to be disproportionately represented in similarities between species). There's in-group versus out-group variation (i.e. how does the variation between Neanderthals and modern humans compare to the variation within modern humans and within Neanderthals). Then there's more abstract concepts such as consanguinity index and metrics of genetic drift (i.e. species that separated more recently have less drift between them). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, a journalist would pick the statistic that is most relevant to the issue being discussed in the article, give an overview of how the statistic is calculated and what it represents, and explain why they think it is the most relevant statistic, and perhaps explain what some other statistics are and whey the journalist chose not to focus on those. But this would be a lot to expect from a journalist, and for a reader to go through. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fact is, 87.6% of percentages are largely or completely meaningless. If someone claims that teenagers get into accidents 20% more often than the general public, does that mean that the number of accidents per teenagers is 20% more than accidents per general person? Accidents per mile driven is 20% more? Accidents per hour? Is it comparing all teenagers to the general public? Teenagers with driver's licenses to the general public with driver's licenses? Is number of accidents, or dollar amount of damages? Then you have things like Simpson's Paradox and selection bias. A percentage is supposed to be one number divided by another, and giving just that number is implicitly asserting that it summarizes the data. But generally speaking, knowing those two original numbers, or at the very least where those numbers come from, is an important part of interpreting the data.&lt;/p&gt;&#xA;" OwnerUserId="2609" LastActivityDate="2018-04-04T16:18:52.820" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="3999" PostTypeId="1" AcceptedAnswerId="4005" CreationDate="2018-04-04T17:05:03.403" Score="5" ViewCount="413" Body="&lt;p&gt;I'm writing a python function from scratch to do this, but I feel like this must exist in some standard bioinformatics library already. In principle, this is a simply regex operation which many must have written previously. With the goal of having a centralized reference as opposed to everyone writing their own scripts to accomplish the same task (which is time-consuming and not necessarily reproducible), here is the &quot;construct a CIGAR&quot; function:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an alignment: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ACGT-TGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is a reference:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ACGTATGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The reference will be required in order to record indels. One could in principle have a series of &quot;queries&quot;, e.g. an R vector or a Python list. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would generate the following CIGAR: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;4=1D3=&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which R packages/ python library exists to do this operation? I suspect there is something related to Bioconductor as well.&lt;/p&gt;&#xA;" OwnerUserId="1770" LastEditorUserId="1770" LastEditDate="2018-04-05T00:04:21.430" LastActivityDate="2018-04-09T19:24:03.817" Title="Is there a Python/R package with the ability to convert an alignment and reference into a CIGAR?" Tags="|r|python|sequence-alignment|bioconductor|cigar|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4000" PostTypeId="2" ParentId="3976" CreationDate="2018-04-04T17:10:01.943" Score="1" Body="&lt;p&gt;The following python script takes a population name as argument, searches for all files for that population (assuming some regularity in the file names), and records the counts of &quot;0&quot; and &quot;1&quot; for all the genes encountered:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;from sys import argv&#xA;from glob import glob&#xA;from collections import defaultdict&#xA;&#xA;filenames = glob(&quot;HI*Index_*_{pop}-*_all_PA&quot;.format(pop=argv[1]))&#xA;&#xA;gene_counts = defaultdict(lambda : defaultdict(int))&#xA;&#xA;for filename in filenames:&#xA;    with open(filename, &quot;r&quot;) as genes_file:&#xA;        for line in genes_file:&#xA;            (gene, num) = line.strip().split()&#xA;            gene_counts[gene][num] += 1&#xA;&#xA;for (gene, counts) in gene_counts.items():&#xA;    print(gene, counts[&quot;1&quot;], counts[&quot;0&quot;], sep=&quot;\t&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With the two example files you give being present in the current directory:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ./count_gene.py MK4&#xA;Ha8_00040788    2   0&#xA;Ha4_00024045    2   0&#xA;Ha4_00025366    2   0&#xA;Ha16_00022130   1   1&#xA;Ha16_00023451   1   1&#xA;Ha8_00040789    2   0&#xA;Ha4_00025367    2   0&#xA;Ha4_00024046    0   2&#xA;Ha16_00022131   2   0&#xA;Ha16_00023452   2   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I haven't tested with python 2. If you use python 2, you should at least add &lt;code&gt;from __future__ import print_function&lt;/code&gt; as first import, or adapt the &lt;code&gt;print&lt;/code&gt; part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, to run on all populations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for pop in $(cat pop.txt)&#xA;do&#xA;    ./count_gene.py ${pop} &amp;gt; ${pop}&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="292" LastActivityDate="2018-04-04T17:10:01.943" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4001" PostTypeId="2" ParentId="3994" CreationDate="2018-04-04T21:11:21.013" Score="7" Body="&lt;p&gt;There is no golden/standard way to define a gene signature but they are completely different from a gene set enrichment analysis (GSEA). I will start with how to obtain a &lt;strong&gt;gene signature&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually this kind of signatures are defined by comparing a group against another ie doing a differential gene expression analysis. One then selects the most up or down regulated genes and defines them as the signature of the group of cells. That's how the gene sets of the &lt;a href=&quot;http://software.broadinstitute.org/gsea/msigdb/index.jsp&quot; rel=&quot;noreferrer&quot;&gt;MSigDB&lt;/a&gt; which are not from (or derived from) pathway databases or gene ontologies are defined. There is no gold threshold of a fold change or a specific number of genes, it (highly) depends on the experiment you run and how good are the isolation of the cells.&lt;br&gt;&#xA;Tools: DESeq2, limma, edgeR,...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As suggested by gringer, another way is to find which genes are expressed together (independently of how do they relate to other groups). This method relays on having a good enough number of samples from the same condition to be able to identify which genes are expressed the same way in those cells. If these genes are not expressed that way in other cell lines, then you have your signature for this group. Again, there isn't a golden method, different types of co-expression networks can be build and depends on how specific do you want to be or which is the data provided.&lt;br&gt;&#xA;Tools: WGCNA, diffcoexp, TReNA,...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is different from a &lt;strong&gt;gene &lt;em&gt;set&lt;/em&gt; enrichment analysis&lt;/strong&gt;&lt;sup&gt;1&lt;/sup&gt;, which consists on evaluating if a group of genes (usually a gene set, a pathway or genes from a specific gene ontology) are not randomly sorted in a sorted list of genes (usually this sorted list is the fold-change of a differential analysis).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several methods developed depending on the null hypothesis, the type of sub-method and what do they take into account...&lt;br&gt;&#xA;Tools: GSEA, PLAGUE, GSVA, limma, fgsea,... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;TLDR: To perform a gene set enrichment you don't need a gene signature but a gene set (that &lt;em&gt;can&lt;/em&gt; be a gene signature or a pathway or a random group).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1: Note that gene set enrichment analyisis is different from gene enrichment analysis. For a good review on the methods I recommend this &lt;a href=&quot;http://dx.plos.org/10.1371/journal.pcbi.1002375&quot; rel=&quot;noreferrer&quot;&gt;article&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-04-04T21:11:21.013" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4002" PostTypeId="1" AcceptedAnswerId="4009" CreationDate="2018-04-05T05:23:40.733" Score="2" ViewCount="162" Body="&lt;p&gt;I'm a bit confused and would appreciate your assistance, please.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to use the &lt;code&gt;pairwiseAlignment&lt;/code&gt; function from the &lt;code&gt;Biostrings&lt;/code&gt; package to align two protein sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not quite sure where to put my reference sequence, the sequence I want to compare to. Is it under &quot;subject&quot; or &quot;pattern&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, is the score I get the edit distance? Is it the alignment score? Or is it the the alignment length? When I use local alignment, will the score be higher the more the sequences resemble each other? And when I use global alignment, will the score be lower the more the sequences resemble each other?&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="298" LastEditDate="2018-04-05T12:23:07.120" LastActivityDate="2018-04-05T23:18:50.167" Title="Question regarding the function pairwiseAlignment in R" Tags="|r|sequence-alignment|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4003" PostTypeId="2" ParentId="4002" CreationDate="2018-04-05T07:53:20.900" Score="2" Body="&lt;p&gt;According to the &lt;a href=&quot;https://www.rdocumentation.org/packages/Biostrings/versions/2.40.2/topics/pairwiseAlignment&quot; rel=&quot;nofollow noreferrer&quot;&gt;help page&lt;/a&gt;, it seems more logic to put the reference sequence in &lt;code&gt;subject&lt;/code&gt;, and your query sequence in &lt;code&gt;pattern&lt;/code&gt;, but it won't really matter for the outcome (except that the pattern sequence will be on top of the alignment).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your second question: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the scores for the optimal pairwise alignments is returned&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="939" LastActivityDate="2018-04-05T07:53:20.900" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4005" PostTypeId="2" ParentId="3999" CreationDate="2018-04-05T14:50:04.103" Score="5" Body="&lt;p&gt;Your request is easy to implement. I wouldn't use any libraries in this case. As you haven't showed your implementation, I will provide one:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;def gen_cigar(ref, qry):&#xA;    if len(ref) != len(qry):&#xA;        raise Exception('unequal length')&#xA;    cigar = []&#xA;    for i in range(len(ref)):&#xA;        r, q = ref[i], qry[i];&#xA;        if r is '-' and q is '-':&#xA;            raise Exception('both gaps')&#xA;        op = '=' if r is q else 'I' if r is '-' else 'D' if q is '-' else 'X';&#xA;        if len(cigar) &amp;gt; 0 and cigar[-1][1] is op: # add to the last operation&#xA;            cigar[-1][0] += 1&#xA;        else: cigar.append([1, op]);              # a new operation&#xA;    return &quot;&quot;.join(map(lambda x: str(x[0]) + x[1], cigar)); # turn to string&#xA;&#xA;print(gen_cigar('ACGTAT-CT', 'ACGT-TGGA'));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;4=1D1=1I2X&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;PS: It is interesting that you often approach problems from a different angle. The solutions that you think &quot;must exist&quot; actually rarely exist because those are not the ways we (me at least) work with data. This is why I prefer an asker to explain the bigger picture in addition to isolated intermediate problems – there could be simpler and more straightforward solutions without the need of solving the contrived problems.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-04-05T14:50:04.103" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4006" PostTypeId="1" CreationDate="2018-04-05T19:23:45.073" Score="1" ViewCount="68" Body="&lt;p&gt;By reading Bacher &amp;amp; Kendziorski (2016), I found the following statement:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;recent droplet technologies are not yet able to accommodate spike-ins&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What's the reason why current droplet technologies cannot use spike-ins? By searching through the available methodologies I found &lt;a href=&quot;https://community.10xgenomics.com/t5/Single-Cell-Forum/Thoughts-on-ERCC-Spike-Ins/td-p/207&quot; rel=&quot;nofollow noreferrer&quot;&gt;this question&lt;/a&gt; on 10X community, but still I cannot understand what is the problem created by using spike-ins with droplet technologies.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Bacher, R. &amp;amp; Kendziorski, C. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4823857/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Design and computational analysis of single-cell RNA-sequencing experiments&lt;/a&gt;. Genome Biol. 17, 63 (2016).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-11-12T11:11:19.027" Title="Is it possible to use spike-ins with droplet technologies?" Tags="|scrnaseq|spike-in|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4007" PostTypeId="1" AcceptedAnswerId="4008" CreationDate="2018-04-05T20:08:04.580" Score="2" ViewCount="3152" Body="&lt;p&gt;I am reading Tang et al. (2009).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the method description it is stated that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A single cell is manually picked under a microscope and lysed. Then mRNAs are reverse-transcribed into cDNAs using a poly(T) primer &lt;strong&gt;with anchor sequence&lt;/strong&gt; (UP1) and unused primers are digested.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What is an anchor sequence in this case?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Tang, F. et al. &lt;a href=&quot;https://www.nature.com/articles/nmeth.1315&quot; rel=&quot;nofollow noreferrer&quot;&gt;mRNA-Seq whole-transcriptome analysis of a single cell&lt;/a&gt;. Nat. Methods 6, 377–382 (2009).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-04-05T21:50:24.363" Title="What is a poly(T) primer anchor sequence?" Tags="|scrnaseq|primer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4008" PostTypeId="2" ParentId="4007" CreationDate="2018-04-05T21:50:24.363" Score="4" Body="&lt;p&gt;As shown in figure 1 of that paper, the anchor sequence is additional sequence that is added to the end of the polyT primer that provides a known and non-target binding site for additional PCR reactions. I gave &lt;a href=&quot;https://f1000research.com/slides/7-251&quot; rel=&quot;nofollow noreferrer&quot;&gt;a presentation&lt;/a&gt; about a month ago about the process that Oxford Nanopore Technology use in their strand-switching cDNA kit, which may be useful for understanding the process used by Tang &lt;em&gt;et al.&lt;/em&gt;. Here are my strand-switch/PCR images from that presentation:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;PolyT primer addition, with mismatched anchor sequence&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rpSO9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rpSO9.png&quot; alt=&quot;PolyT primer addition, with mismatched anchor sequence&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Strand switch primer addition, with overhanging anchor sequence&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lK3qh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lK3qh.png&quot; alt=&quot;Strand switch primer addition, with overhanging anchor sequence&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;PCR amplification, using primers that are complementary to the anchor sequences&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QnIow.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QnIow.png&quot; alt=&quot;PCR amplification using complementary primers&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of the Tang &lt;em&gt;et al.&lt;/em&gt; paper, they use a polyA-tailing method for strand-switching, but the process is essentially the same.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-04-05T21:50:24.363" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4009" PostTypeId="2" ParentId="4002" CreationDate="2018-04-05T23:18:50.167" Score="2" Body="&lt;p&gt;For semantic purposes, it's better to use the reference sequence as the pattern. Note that only the pattern argument can be used as a multi-element vector, i.e. you can match multiple patterns against a single subject with this function, but not a single pattern against multiple subjects. Knowing that might alter what is defined as a &quot;pattern&quot; or a &quot;subject&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The scores are associated with the alignment method that is used. Alignment scores are higher for more similar matches and lower for poor matches, but the weightings associated with matches and mismatches dictate what that actually means. It might be that a score of -2 happens to be the optimal alignment score, because all other possible alignments are more negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's possible to use &lt;code&gt;pairwiseAlignment&lt;/code&gt; to generate the levenshtein distance (most commonly-used edit distance) between two sequences by setting the match score to 0, the mismatch score to -1, gap opening penalty to 0, gap extension penalty to 1, and then negating the resultant &lt;code&gt;pairwiseAlignment&lt;/code&gt; score.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-04-05T23:18:50.167" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4010" PostTypeId="1" CreationDate="2018-04-06T05:02:53.613" Score="0" ViewCount="106" Body="&lt;p&gt;I am putting this question because I did not find any useful information from internet because of limited access. My question is related to control (or normal) data that we use for somatic mutation detection (with the help of mutect or SomaticSnipper etc) with tumor data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have control data and my aim is to find somatic mutations in Multiple myeloma WES caner data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we use Hg19 as a control data? If not, what are the technical reasons of not using hg19 as a control data because it hg19 is consensus call.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way to infer (or predict) somatic mutations (either statistically or by something else) out of all possible mutations? For this reason, I want to know what information is given in control data.&lt;/p&gt;&#xA;" OwnerUserId="2141" LastEditorUserId="48" LastEditDate="2018-11-30T10:45:32.190" LastActivityDate="2018-11-30T10:45:32.190" Title="Information about control data" Tags="|database|cancer|mutations|somatic|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4011" PostTypeId="1" AcceptedAnswerId="4041" CreationDate="2018-04-06T06:02:17.120" Score="2" ViewCount="246" Body="&lt;p&gt;I have exome data 32 patients. My aim is to identify unique somatic mutations in each patients. For this I have completed my pipeline for Whole Exome Data analysis using various tools like BWA mem, picard, gatk etc. Now I have VCF and gVCF file of each patient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For Variant Quality Score recalibration (VQSR), as per the doccumentation, we should have at least 30 patient data for good recalibration. So in order to perform VQSR, I combined gVCF of all 32 patients using genotypeGVCFs tool of gatk and done with VQSR. But now I have cohort of all patient data (a large combined vcf file). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know how variant sub-class tool or other extraction tool work to extract all of the unique and related features to each patient. I know how to get significant mutations using mutect but here it is combined file but I want to get it for each one individually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I get a vcf file for each patient (without missing a single variant associated to that patient) ?&lt;/p&gt;&#xA;" OwnerUserId="2141" LastEditorUserId="96" LastEditDate="2019-01-15T14:25:06.960" LastActivityDate="2019-01-15T14:25:06.960" Title="How to get unique somatic mutations for each individual patients" Tags="|vcf|mutations|exome|variation|somatic|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4013" PostTypeId="1" AcceptedAnswerId="4036" CreationDate="2018-04-06T19:07:18.433" Score="7" ViewCount="2671" Body="&lt;p&gt;Reading Islam et al. (2011):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;From each transcript, a single read is obtained, corresponding to a template-switching site located preferentially at the 59 end of the mRNA.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;By reading &lt;a href=&quot;https://www.idtdna.com/pages/education/decoded/article/use-of-template-switching-oligos-(ts-oligos-tsos)-for-efficient-cdna-library-construction&quot; rel=&quot;noreferrer&quot;&gt;this page&lt;/a&gt; I think that a template switching site is an additional set of nucleotides added at the 3' end of the newly synthesized cDNA, that serve to anchor a oligo-nucleotide used by the reverse transcriptase to synthesize the second cDNA strand. It is called template switching because reverse transcriptase switches strand upon reaching this set of nucleotides. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it correct?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Islam, S. et al. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3129258/&quot; rel=&quot;noreferrer&quot;&gt;Characterization of the single-cell transcriptional landscape by highly multiplex RNA-seq&lt;/a&gt;. genome.cshlp.org&#xA;21, 1160–1167 (2011).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-04-10T13:13:58.147" Title="What's a template switching site?" Tags="|scrnaseq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4014" PostTypeId="2" ParentId="3583" CreationDate="2018-04-06T20:37:38.893" Score="2" Body="&lt;p&gt;Since at least version &lt;code&gt;1.71&lt;/code&gt; of biopython you can use &lt;code&gt;Bio.Seq.reverse_complement&lt;/code&gt;, which also works on plain strings natively (no conversion to Seq objects).  On my mac I get 800k strings converted with that implementation (&quot;biopython just rc&quot;) when using &lt;a href=&quot;https://github.com/conchoecia/fastest_rc_python&quot; rel=&quot;nofollow noreferrer&quot;&gt;the benchmark&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;10000 strings and 250 repetitions&#xA;                         percent increase over baseline  seconds total strings per second&#xA;name                                                                                     &#xA;alexreynolds Cython (v2)                          94.0%       0.002491          4015188.7&#xA;user172818 seqpy.c                                93.2%       0.002833          3529944.4&#xA;alexreynolds Cython (v1)                          90.7%       0.003885          2573863.7&#xA;devonryan string                                  86.0%       0.005804          1722983.5&#xA;jackaidley bytes                                  85.7%       0.005959          1678085.3&#xA;jackaidley bytesstring                            82.1%       0.007459          1340693.2&#xA;biopython just rc                                 70.5%       0.012275           814638.6&#xA;revcomp_translateSO                               49.9%       0.020822           480254.6&#xA;revcom from SO                                    41.7%       0.024253           412320.2&#xA;string_replace                                    35.2%       0.026946           371107.3&#xA;lambda from SO                                    10.7%       0.037111           269464.0&#xA;global dict                                        0.0%       0.041556           240641.7&#xA;biopython seq then rc                            -30.9%       0.054430           183721.9&#xA;naive (baseline)                                  -0.9%       0.041962           238313.6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="492" LastEditorUserId="492" LastEditDate="2018-04-09T11:04:41.833" LastActivityDate="2018-04-09T11:04:41.833" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4015" PostTypeId="1" AcceptedAnswerId="4028" CreationDate="2018-04-07T04:41:04.070" Score="0" ViewCount="175" Body="&lt;p&gt;I have 16S DNA sequences from diversity sequencing, and want to query these against NCBI's 16S rRNA database.  If I ultimately want to get taxids for the species that my sequences have, does it make a difference that my sequences are 16S DNA and not RNA?  I am using Blast+.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I imagine this is how analysis of 16S sequences normally goes (get 16S DNA sequences, query them against a 16S DB), but does anyone know if Blast+ or the 16S rRNA DB itself allow for DNA queries?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, if I were to blast a sequence of 16S DNA and its exact transcript RNA, would I get the same results?&lt;/p&gt;&#xA;" OwnerUserId="2446" LastActivityDate="2018-04-09T09:21:13.040" Title="Is it OK to use Blast+ to query NCBI's 16s rRNA database with 16s DNA sequences?" Tags="|ngs|blast|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4016" PostTypeId="2" ParentId="3449" CreationDate="2018-04-07T10:10:49.977" Score="5" Body="&lt;p&gt;Look at &lt;a href=&quot;https://www.nature.com/articles/s41467-017-02554-5&quot; rel=&quot;noreferrer&quot;&gt;this recent paper&lt;/a&gt; that uses ComBat on scRNA-seq data for batch effect removal and states that it &quot;successfully does so&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also suggest that you check out &lt;a href=&quot;https://arxiv.org/pdf/1610.04181.pdf&quot; rel=&quot;noreferrer&quot;&gt;this publication&lt;/a&gt; on Distribution Matching Residual-Nets. Authors evaluated their method also on scRNA-seq data and thus it may be something you are looking for. I personally played a bit with their code and found it straightforward to use. &lt;/p&gt;&#xA;" OwnerUserId="2440" LastActivityDate="2018-04-07T10:10:49.977" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4017" PostTypeId="1" CreationDate="2018-04-07T13:51:59.637" Score="2" ViewCount="154" Body="&lt;p&gt;I am new to the field of Bioinformatics. Single Nucleotide Polymorphism or SNP data are available on the the Internet to download.SNP related data are valuable for variety of reasons ranging from  disease related charatertcics and  precision medicine to name the few.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can somebody explain me the steps of downloading this kind of data from NCBI or other resource.&#xA;For example if I want to download the the SNP in bunch of humans on a particular Gene from the specific start to end. What are the sequence of steps I would do online to download such data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want SNP data for example for a particular gene say X from positions in range say base pairs 100,000 to 200,000 for individuals say 100 in txt or csv format or standard format that is available?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like if I choose Gene X and position ranging from 100 to 109 base pairs in 5 persons I want following kind of data in txt or csv format with say two SNV at position 101 and 109&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A &lt;strong&gt;A&lt;/strong&gt; A A A G G G G &lt;strong&gt;G&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;A &lt;strong&gt;G&lt;/strong&gt; A A A G G G G &lt;strong&gt;A&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;A &lt;strong&gt;A&lt;/strong&gt; A A A G G G G &lt;strong&gt;A&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;A &lt;strong&gt;A&lt;/strong&gt; A A A G G G G &lt;strong&gt;A&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;A &lt;strong&gt;G&lt;/strong&gt; A A A G G G G &lt;strong&gt;G&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;regards &lt;/p&gt;&#xA;" OwnerUserId="2622" LastEditorUserId="2622" LastEditDate="2018-04-08T07:34:13.113" LastActivityDate="2018-04-08T07:34:13.113" Title="SNP data Download Help" Tags="|snp|sequence-analysis|data-download|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4018" PostTypeId="2" ParentId="4017" CreationDate="2018-04-07T16:21:22.280" Score="2" Body="&lt;p&gt;I would recommend to do this in R, using biomaRt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example if you want all the SNPs from IDH1 (ENSG00000138413) gene:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(biomaRt)&#xA;&#xA;ensembl&amp;lt;- useMart(&quot;ensembl&quot;)&#xA;ensembl &amp;lt;- useDataset(&quot;hsapiens_gene_ensembl&quot;,mart=ensembl)&#xA;&#xA;IDH1 &amp;lt;- getBM(attributes=c('ensembl_gene_id','variation_name'), filters = 'ensembl_gene_id', values =&quot;ENSG00000138413&quot;, mart = ensembl)&#xA;&#xA;IDH1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can add much more information, take a look at the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/biomaRt/inst/doc/biomaRt.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;help page&lt;/a&gt; of biomaRt.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2018-04-07T16:21:22.280" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4019" PostTypeId="1" AcceptedAnswerId="4022" CreationDate="2018-04-07T18:03:32.077" Score="9" ViewCount="11906" Body="&lt;p&gt;I have NGS illumina RNA-seq reads from &lt;em&gt;M. musculus&lt;/em&gt; (mm10). I am trying to find variants along the strand portion of the reads in the refseq (mm10).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mapped a pair of sequence files and generated a BAM file. Now I need to generate a VCF file from this BAM file. I’ve checked Galaxy for direct tools, but I couldn’t identify one. Any suggestions on how to turn a BAM file into VCF? Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="2621" LastEditorUserId="96" LastEditDate="2019-02-27T20:18:23.123" LastActivityDate="2019-02-27T20:18:23.123" Title="Converting a BAM file into VCF" Tags="|sam|vcf|variant-calling|variation|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4020" PostTypeId="2" ParentId="2663" CreationDate="2018-04-08T06:07:19.657" Score="0" Body="&lt;p&gt;I wrote a tool that will rerun an experiment. So all you would need to do is just download an ONT dataset from ENA or something and you can just rerun it as many times as you want.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/mbhall88/taeper&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mbhall88/taeper&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="149" LastActivityDate="2018-04-08T06:07:19.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4021" PostTypeId="2" ParentId="4019" CreationDate="2018-04-08T14:21:47.900" Score="5" Body="&lt;p&gt;You can use &lt;a href=&quot;https://github.com/ekg/freebayes&quot; rel=&quot;noreferrer&quot;&gt;Freebayes&lt;/a&gt;, provided you have your BAM file and the reference genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;freebayes -f genome.fa aln.bam &amp;gt; var.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'd suggest you go through the whole github documentation, and choose the option(s) more suitable for your needs (ploidy, read support, etc.)&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2018-04-08T14:21:47.900" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4022" PostTypeId="2" ParentId="4019" CreationDate="2018-04-08T15:53:07.300" Score="19" Body="&lt;p&gt;It's not really possible to &lt;strong&gt;convert&lt;/strong&gt; &lt;code&gt;bam&lt;/code&gt; to &lt;code&gt;vcf&lt;/code&gt;. &lt;code&gt;bam&lt;/code&gt; is a mapping file, it does not contain the information about variants, this information needs to be inferred in process called variant calling. I find important to mention that it's not just a different format of the same thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to &lt;strong&gt;call variants&lt;/strong&gt; (a &lt;code&gt;vcf&lt;/code&gt; file) from mapped reads (a &lt;code&gt;bam&lt;/code&gt; file) is very broad question, it depends on the sequencing technology (Illumina, PacBio, ...), the type of sequencing (whole genome sequencing, RAD, RNA-seq, ...), the sampling (calling of individuals vs population variability), the quality of a reference (human vs denovo assembly made by a postdoc in your lab)... For a lot of cases the answer is simply &lt;a href=&quot;https://software.broadinstitute.org/gatk/&quot; rel=&quot;noreferrer&quot;&gt;GATK&lt;/a&gt;, partially because it has well written manual, but mainly just because many people use it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some more detailed cases are well answered in more specific questions :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/654/how-do-i-generate-a-variant-list-i-e-vcf-file-using-illumina-reads-from-a-hum&quot;&gt;How do I generate a variant list (i.e. VCF file) using Illumina reads from a human genome?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2744/somatic-tumor-only-variant-calling&quot;&gt;Somatic tumor only variant calling?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/672/why-are-there-missing-calls-in-a-vcf-file-from-exome-sequencing&quot;&gt;Why are there missing calls in a VCF file from exome sequencing?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2018-10-07T21:40:47.400" LastActivityDate="2018-10-07T21:40:47.400" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4024" PostTypeId="1" AcceptedAnswerId="4046" CreationDate="2018-04-08T20:51:50.223" Score="3" ViewCount="2613" Body="&lt;p&gt;Also posted &lt;a href=&quot;https://www.biostars.org/p/308123/&quot; rel=&quot;nofollow noreferrer&quot;&gt;on biostars&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to use &lt;code&gt;cellranger&lt;/code&gt; or &lt;code&gt;bcl2fastq&lt;/code&gt; to convert the &lt;code&gt;.bcl&lt;/code&gt; files that I got from single cell analysis run into &lt;code&gt;fastq&lt;/code&gt; files for further analysis. I needed to generate &lt;code&gt;sample_sheet.csv&lt;/code&gt; and so I used the following tool:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/bcl2fastq-direct&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/bcl2fastq-direct&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which I copy-pasted into a text file using &lt;code&gt;vim&lt;/code&gt;. &lt;code&gt;sample_sheet.csv&lt;/code&gt; looks like that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[Header]&#xA;EMFileVersion,4&#xA;&#xA;&#xA;[Reads]&#xA;26&#xA;8&#xA;98&#xA;&#xA;[Data]&#xA;Lane,Sample_ID,Sample_Name,index,Sample_Project&#xA;1,SI-GA-B4_1,17R,ACTTCATA,Chromium_20180406&#xA;1,SI-GA-B4_2,17R,GAGATGAC,Chromium_20180406&#xA;1,SI-GA-B4_3,17R,TGCCGTGG,Chromium_20180406&#xA;1,SI-GA-B4_4,17R,CTAGACCT,Chromium_20180406&#xA;1,SI-GA-B5_1,19RL,AATAATGG,Chromium_20180406 &#xA;1,SI-GA-B5_2,19RL,CCAGGGCA,Chromium_20180406&#xA;1,SI-GA-B5_3,19RL,TGCCTCAT,Chromium_20180406&#xA;1,SI-GA-B5_4,19RL,GTGTCATC,Chromium_20180406&#xA;2,SI-GA-B4_1,17R,ACTTCATA,Chromium_20180406&#xA;2,SI-GA-B4_2,17R,GAGATGAC,Chromium_20180406&#xA;2,SI-GA-B4_3,17R,TGCCGTGG,Chromium_20180406&#xA;2,SI-GA-B4_4,17R,CTAGACCT,Chromium_20180406&#xA;2,SI-GA-B5_1,19RL,AATAATGG,Chromium_20180406&#xA;2,SI-GA-B5_2,19RL,CCAGGGCA,Chromium_20180406&#xA;2,SI-GA-B5_3,19RL,TGCCTCAT,Chromium_20180406&#xA;2,SI-GA-B5_4,19RL,GTGTCATC,Chromium_20180406&#xA;3,SI-GA-B4_1,17R,ACTTCATA,Chromium_20180406&#xA;3,SI-GA-B4_2,17R,GAGATGAC,Chromium_20180406&#xA;3,SI-GA-B4_3,17R,TGCCGTGG,Chromium_20180406&#xA;3,SI-GA-B4_4,17R,CTAGACCT,Chromium_20180406&#xA;3,SI-GA-B5_1,19RL,AATAATGG,Chromium_20180406&#xA;3,SI-GA-B5_2,19RL,CCAGGGCA,Chromium_20180406&#xA;3,SI-GA-B5_3,19RL,TGCCTCAT,Chromium_20180406&#xA;3,SI-GA-B5_4,19RL,GTGTCATC,Chromium_20180406&#xA;4,SI-GA-B4_1,17R,ACTTCATA,Chromium_20180406&#xA;4,SI-GA-B4_2,17R,GAGATGAC,Chromium_20180406&#xA;4,SI-GA-B4_3,17R,TGCCGTGG,Chromium_20180406&#xA;4,SI-GA-B4_4,17R,CTAGACCT,Chromium_20180406&#xA;4,SI-GA-B5_1,19RL,AATAATGG,Chromium_20180406&#xA;4,SI-GA-B5_2,19RL,CCAGGGCA,Chromium_20180406&#xA;4,SI-GA-B5_3,19RL,TGCCTCAT,Chromium_20180406&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I ran &lt;code&gt;cellranger&lt;/code&gt; using &lt;code&gt;bash&lt;/code&gt; script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA; FLOWCELL_DIR=&quot;/scratch/nv4e/kipnis/180403_NB501830_0158_AHN3LLBGX5&quot;&#xA; OUTPUT_DIR=&quot;/scratch/nv4e/kipnis/fastq&quot;&#xA; SAMPLE_SHEET_PATH=&quot;/scratch/nv4e/kipnis/sample_sheet_2.csv&quot;&#xA;&#xA;cellranger mkfastq --id=&quot;AHN3LLBGX5&quot; --run=${FLOWCELL_DIR} --csv=${SAMPLE_SHEET_PATH} --output-dir=${OUTPUT_DIR}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It fails with the following output:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/coXGs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/coXGs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;_stderr&lt;/code&gt; file shows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ECDVI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ECDVI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and that my &lt;code&gt;sample_sheet.csv&lt;/code&gt; can not be parsed:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NVNW6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NVNW6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It could be because &lt;code&gt;CLRF&lt;/code&gt; needs to be converted into &lt;code&gt;LF&lt;/code&gt;, but I tried two commands to do this unsuccessfully:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;tr -d '\r' &amp;lt; input &amp;gt; output&lt;/code&gt; and &lt;code&gt;perl -pi -e 's/\r\n/\n/g' input&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the following thread:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://unix.stackexchange.com/questions/277217/how-to-install-dos2unix-on-linux-without-root-access&quot;&gt;https://unix.stackexchange.com/questions/277217/how-to-install-dos2unix-on-linux-without-root-access&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above commands successfully strip the &lt;code&gt;\r&lt;/code&gt; from the line endings, but the error from bcl2fastq remains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I doing wrong? Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Update&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I looked into the folder that is generated when the &lt;code&gt;cellranger&lt;/code&gt; runs and found two &lt;code&gt;.csv&lt;/code&gt; files generated by the &lt;code&gt;cellranger&lt;/code&gt;. One of them, &lt;code&gt;samplesheet.csv&lt;/code&gt;, contains these &lt;code&gt;^M&lt;/code&gt; characters that are shown in the error:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1FYBP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1FYBP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Update&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Maybe one of the issues was that I was submitting regular &lt;code&gt;.sh&lt;/code&gt; file on the &lt;code&gt;SLURM&lt;/code&gt; cluster. However, I wrote the right &lt;code&gt;.slurm&lt;/code&gt; file and submitted it yesterday with &lt;code&gt;sbatch&lt;/code&gt; command:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zjDEy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zjDEy.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, today I got the message in my email:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/npZzE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/npZzE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And neither &lt;code&gt;_stderr&lt;/code&gt;, nor &lt;code&gt;generate_fastq_id.err&lt;/code&gt; or &lt;code&gt;generate_fastq_id.out&lt;/code&gt; files contain anything. I thought initially that I set not enough time and it timed out, however, it is not true: I set the time to &lt;code&gt;24h&lt;/code&gt; and it has been running for just &lt;code&gt;13h&lt;/code&gt; before failing. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Update&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I searched over all of the files in the generated folder with the &lt;code&gt;find&lt;/code&gt; linux command and actually found the error files that show the same error as before: issue with the &lt;code&gt;sample_sheet.csv&lt;/code&gt; file. This error file is &lt;code&gt;AHN3LLBGX5/MAKE_FASTQS_CS/MAKE_FASTQS/BCL2FASTQ_WITH_SAMPLESHEET/fork0/chnk0-u179acba3d8/_stderr&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, right now the issue seems to be the following: I am feeding in the right &lt;code&gt;sample_sheet.csv&lt;/code&gt; without the &lt;code&gt;^M&lt;/code&gt; symbols, but &lt;code&gt;cellranger&lt;/code&gt; transforms it into one that contains &lt;code&gt;^M&lt;/code&gt; and then uses it giving the error. Above I see also &lt;code&gt;which: no configureBclToFastq.pl&lt;/code&gt; and &lt;code&gt;ERROR: bcl2fastq::common::Exception: 2018-Apr-09 13:33:14: Inappropriate ioctl for device&lt;/code&gt;, so maybe my input parameters are just wrong to &lt;code&gt;cellranger&lt;/code&gt; and it has nothing to do with the &lt;code&gt;sample_sheet.csv&lt;/code&gt;? &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Update&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I tried to use &lt;code&gt;--sample-sheet&lt;/code&gt; instead of &lt;code&gt;--csv&lt;/code&gt; since I am feeding in complex sample sheet. No luck.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Update&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here you can find all the error document in text:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;which: no configureBclToFastq.pl in (/sfs/lustre/scratch/nv4e/cellranger/samtools_new/1.6:/sfs/lustre/scratch/nv4e/cellranger/cellranger-tiny-fastq/1.2.0:/sfs/lustre/scratch/nv4e/cellranger/cellranger-tiny-ref/1.2.0:/sfs/lustre/scratch/nv4e/cellranger/miniconda-cr-cs/4.3.21-miniconda-cr-cs-c9/bin:/sfs/lustre/scratch/nv4e/cellranger/cellranger-cs/2.1.1/lib/bin:/sfs/lustre/scratch/nv4e/cellranger/cellranger-cs/2.1.1/tenkit/lib/bin:/sfs/lustre/scratch/nv4e/cellranger/cellranger-cs/2.1.1/tenkit/bin:/sfs/lustre/scratch/nv4e/cellranger/cellranger-cs/2.1.1/bin:/sfs/lustre/scratch/nv4e/cellranger/lz4/v1.8.0:/sfs/lustre/scratch/nv4e/cellranger/martian-cs/2.3.2/bin:/sfs/lustre/scratch/nv4e/cellranger/STAR/5dda596:/sfs/lustre/scratch/nv4e/cellranger/samtools_new/1.6:/sfs/lustre/scratch/nv4e/cellranger/cellranger-tiny-fastq/1.2.0:/sfs/lustre/scratch/nv4e/cellranger/cellranger-tiny-ref/1.2.0:/sfs/lustre/scratch/nv4e/cellranger/miniconda-cr-cs/4.3.21-miniconda-cr-cs-c9/bin:/sfs/lustre/scratch/nv4e/cellranger/cellranger-cs/2.1.1/lib/bin:/sfs/lustre/scratch/nv4e/cellranger/cellranger-cs/2.1.1/tenkit/lib/bin:/sfs/lustre/scratch/nv4e/cellranger/cellranger-cs/2.1.1/tenkit/bin:/sfs/lustre/scratch/nv4e/cellranger/cellranger-cs/2.1.1/bin:/sfs/lustre/scratch/nv4e/cellranger/lz4/v1.8.0:/sfs/lustre/scratch/nv4e/cellranger/martian-cs/2.3.2/bin:/sfs/lustre/scratch/nv4e/cellranger/STAR/5dda596:/scratch/nv4e/cellranger:/scratch/nv4e/bcl2fastq/build/bin:/scratch/nv4e/spark/bin:/scratch/nv4e/scala/bin:/sfs/nfs/blue/nv4e/private/bin:/sfs/nfs/blue/nv4e/anaconda2/bin:/sfs/nfs/blue/nv4e/.local/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/slurm/current/bin:/opt/slurm/current/sbin:/opt/singularity/current/bin:/opt/rci/bin:/opt/rci/sbin:/opt/nhc/current/sbin:/share/rci_apps/common/bin:/share/resources/HPCtools/)&#xA;  BCL to FASTQ file converter&#xA;  bcl2fastq v2.20.0.422&#xA;  Copyright (c) 2007-2017 Illumina, Inc.&#xA;  2018-04-09 18:31:29 [7fcdb39f37c0] Command-line invocation: bcl2fastq --minimum-trimmed-read-length 8 --mask-short-adapter-reads 8 --create-fastq-for-index-reads --ignore-missing-positions --ignore-missing-filter --ignore-missing-bcls --use-bases-mask=Y26,I8,Y98 -R /scratch/nv4e/kipnis/180403_NB501830_0158_AHN3LLBGX5 --output-dir=/scratch/nv4e/kipnis/AHN3LLBGX5/MAKE_FASTQS_CS/MAKE_FASTQS/BCL2FASTQ_WITH_SAMPLESHEET/fork0/chnk0-u21f1cbe9bf/files/fastq_path --interop-dir=/scratch/nv4e/kipnis/AHN3LLBGX5/MAKE_FASTQS_CS/MAKE_FASTQS/BCL2FASTQ_WITH_SAMPLESHEET/fork0/chnk0-u21f1cbe9bf/files/interop_path --sample-sheet=/scratch/nv4e/kipnis/AHN3LLBGX5/MAKE_FASTQS_CS/MAKE_FASTQS/PREPARE_SAMPLESHEET/fork0/chnk0-u09e7cbe891/files/samplesheet.csv -p 6 -r 6 -w 6&#xA;  2018-04-09 18:31:29 [7fcdb39f37c0] INFO: Minimum log level: INFO&#xA;  2018-04-09 18:31:29 [7fcdb39f37c0] INFO: Sample sheet: '/scratch/nv4e/kipnis/AHN3LLBGX5/MAKE_FASTQS_CS/MAKE_FASTQS/PREPARE_SAMPLESHEET/fork0/chnk0-u09e7cbe891/files/samplesheet.csv'&#xA;  2018-04-09 18:31:29 [7fcdb39f37c0] ERROR: bcl2fastq::common::Exception: 2018-Apr-09 18:31:29: Inappropriate ioctl for device (25): /scratch/nv4e/bcl2fastq/src/cxx/include/common/CsvGrammar.hpp(92): Throw in function bcl2fastq::common::CsvGrammarAttribute bcl2fastq::common::parseCsvData(Iterator, Iterator) [with Iterator = __gnu_cxx::__normal_iterator &gt;; bcl2fastq::common::CsvGrammarAttribute = std::vector &gt; &gt;]&#xA;  Dynamic exception type: boost::exception_detail::clone_impl&#xA;  std::exception::what: Could not parse the CSV stream text:&#xA;   ^M&#xA;  ^M&#xA;  [Reads]^M&#xA;  26^M&#xA;  98^M&#xA;   ^M&#xA;  ^M&#xA;  [Data]^M&#xA;  Lane,Sample_ID,Sample_Name,index,Sample_Project,Original_Sample_ID^M&#xA;  1,SI-GA-B4_1,17R,ACTTCATA,Chromium_20180409,SI-GA-B4_1^M&#xA;  1,SI-GA-B4_2,17R,GAGATGAC,Chromium_20180409,SI-GA-B4_2^M&#xA;  1,SI-GA-B4_3,17R,TGCCGTGG,Chromium_20180409,SI-GA-B4_3^M&#xA;  1,SI-GA-B4_4,17R,CTAGACCT,Chromium_20180409,SI-GA-B4_4^M&#xA;  1,SI-GA-B5_1,19RL,AATAATGG,Chromium_20180409,SI-GA-B5_1^M&#xA;  1,SI-GA-B5_2,19RL,CCAGGGCA,Chromium_20180409,SI-GA-B5_2^M&#xA;  1,SI-GA-B5_3,19RL,TGCCTCAT,Chromium_20180409,SI-GA-B5_3^M&#xA;  1,SI-GA-B5_4,19RL,GTGTCATC,Chromium_20180409,SI-GA-B5_4^M&#xA;  2,SI-GA-B4_1,17R,ACTTCATA,Chromium_20180409,SI-GA-B4_1^M&#xA;  2,SI-GA-B4_2,17R,GAGATGAC,Chromium_20180409,SI-GA-B4_2^M&#xA;  2,SI-GA-B4_3,17R,TGCCGTGG,Chromium_20180409,SI-GA-B4_3^M&#xA;  2,SI-GA-B4_4,17R,CTAGACCT,Chromium_20180409,SI-GA-B4_4^M&#xA;  2,SI-GA-B5_1,19RL,AATAATGG,Chromium_20180409,SI-GA-B5_1^M&#xA;  2,SI-GA-B5_2,19RL,CCAGGGCA,Chromium_20180409,SI-GA-B5_2^M&#xA;  2,SI-GA-B5_3,19RL,TGCCTCAT,Chromium_20180409,SI-GA-B5_3^M&#xA;  2,SI-GA-B5_4,19RL,GTGTCATC,Chromium_20180409,SI-GA-B5_4^M&#xA;  3,SI-GA-B4_1,17R,ACTTCATA,Chromium_20180409,SI-GA-B4_1^M&#xA;  3,SI-GA-B4_2,17R,GAGATGAC,Chromium_20180409,SI-GA-B4_2^M&#xA;  3,SI-GA-B4_3,17R,TGCCGTGG,Chromium_20180409,SI-GA-B4_3^M&#xA;  3,SI-GA-B4_4,17R,CTAGACCT,Chromium_20180409,SI-GA-B4_4^M&#xA;  3,SI-GA-B5_1,19RL,AATAATGG,Chromium_20180409,SI-GA-B5_1^M&#xA;  3,SI-GA-B5_2,19RL,CCAGGGCA,Chromium_20180409,SI-GA-B5_2^M&#xA;  3,SI-GA-B5_3,19RL,TGCCTCAT,Chromium_20180409,SI-GA-B5_3^M&#xA;  3,SI-GA-B5_4,19RL,GTGTCATC,Chromium_20180409,SI-GA-B5_4^M&#xA;  4,SI-GA-B4_1,17R,ACTTCATA,Chromium_20180409,SI-GA-B4_1^M&#xA;  4,SI-GA-B4_2,17R,GAGATGAC,Chromium_20180409,SI-GA-B4_2^M&#xA;  4,SI-GA-B4_3,17R,TGCCGTGG,Chromium_20180409,SI-GA-B4_3^M&#xA;  4,SI-GA-B4_4,17R,CTAGACCT,Chromium_20180409,SI-GA-B4_4^M&#xA;  4,SI-GA-B5_1,19RL,AATAATGG,Chromium_20180409,SI-GA-B5_1^M&#xA;  4,SI-GA-B5_2,19RL,CCAGGGCA,Chromium_20180409,SI-GA-B5_2^M&#xA;  4,SI-GA-B5_3,19RL,TGCCTCAT,Chromium_20180409,SI-GA-B5_3^M&#xA;  4,SI-GA-B5_4,19RL,GTGTCATC,Chromium_20180409,SI-GA-B5_4^M&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="2005" LastEditDate="2018-04-10T17:43:00.260" LastActivityDate="2020-08-26T21:45:26.303" Title="10X Illumina demultiplexing sample sheet issue" Tags="|scrnaseq|10x-genomics|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="4025" PostTypeId="1" AcceptedAnswerId="4029" CreationDate="2018-04-09T00:35:50.737" Score="2" ViewCount="519" Body="&lt;p&gt;I ran bulk &lt;code&gt;RNA-seq&lt;/code&gt; experiment and got &lt;code&gt;quant.sf&lt;/code&gt; file. Now, I am struggling with understanding what &lt;code&gt;tximport&lt;/code&gt; package does and how to use it correctly. My ultimate goal is to feed the data into &lt;code&gt;DESeq2&lt;/code&gt; for differential expression analysis. In the documentation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://bioconductor.org/packages/release/bioc/manuals/tximport/man/tximport.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://bioconductor.org/packages/release/bioc/manuals/tximport/man/tximport.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samples &amp;lt;- read.table(file.path(dir,&quot;samples.txt&quot;), header=TRUE)&#xA;files &amp;lt;- file.path(dir,&quot;salmon&quot;, samples$run, &quot;quant.sf&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I do not understand what are these &lt;code&gt;samples.txt&lt;/code&gt;. It is not generated by &lt;code&gt;salmon&lt;/code&gt;, so I suppose that I need to write it myself. How then? What is the correct format for the file? And what if I want to feed just one sample in? That is actually what I want initially: process each sample separately. Could anyone explain step by step how we could utilize &lt;code&gt;tximport&lt;/code&gt; when we have just one &lt;code&gt;quant.sf&lt;/code&gt; file to get to raw count matrix to feed it into &lt;code&gt;DeSeq2&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-04-09T09:29:20.213" Title="Salmon tximport" Tags="|rna-seq|deseq2|tximport|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4026" PostTypeId="2" ParentId="2036" CreationDate="2018-04-09T01:45:00.267" Score="1" Body="&lt;p&gt;Thanks to @Devon_Ryan's suggestion, I had a go at using GMAP with our own &lt;em&gt;de-novo&lt;/em&gt; assembly. Here's what I started with:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Genome assembled from nanopore long reads, corrected using Illumina cDNA reads&lt;/li&gt;&#xA;&lt;li&gt;genome-guided Trinity transcriptome assembly, with reads grouped based on mapping to the assembled genome, then assembled by Trinity&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The transcriptome assembly was filtered to only include transcripts that had reasonable coverage from cDNA reads, and further filtered to cluster similar transcripts and only use the longest isoform. This process is detailed in &lt;a href=&quot;https://bmcbiol.biomedcentral.com/articles/10.1186/s12915-017-0473-4#Sec9&quot; rel=&quot;nofollow noreferrer&quot;&gt;our paper&lt;/a&gt;. If I were to do that again, I'd probably create a supertranscript containing all exons, rather than the longest isoform.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I first created an index for the genome using &lt;code&gt;gmap_build&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gmap_build -k 15 -j 8 -d 'MINBRA' \&#xA;  Nb_ONTCFED_65bpTrim_t1.contigs.hpcleaned.fasta -D MINBRA_GMAP&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I then used gmap to map the transcriptome to the genome, generating gene models as a GFF3 file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gmap -t 10 -d MINBRA -D ./MINBRA_GMAP \&#xA;  -A locusTagged_CDLI_CD98LMOHC50_TBNOCFED.tran.fasta \&#xA;  -f 2 &amp;gt; Nb_TrinTran_vs_Canu.gff3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's an example from the annotation file that was produced by GMAP:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;tig00022147 MINBRA  gene    413327  414778  .   +   .   ID=MINBRA_TGG_483_c1_g1_i1.path1;Name=MINBRA_TGG_483_c1_g1_i1&#xA;tig00022147 MINBRA  mRNA    413327  414778  .   +   .   ID=MINBRA_TGG_483_c1_g1_i1.mrna1;Name=MINBRA_TGG_483_c1_g1_i1;Parent=MINBRA_TGG_483_c1_g1_i1.path1;coverage=100.0;identity=99.9;matches=1452;mismatches=0;indels=2;unknowns=0&#xA;tig00022147 MINBRA  exon    413327  414778  99  +   .   ID=MINBRA_TGG_483_c1_g1_i1.mrna1.exon1;Name=MINBRA_TGG_483_c1_g1_i1;Parent=MINBRA_TGG_483_c1_g1_i1.mrna1;Target=MINBRA_TGG_483_c1_g1_i1 1454 1 .&#xA;tig00022147 MINBRA  CDS 413417  413557  100 +   0   ID=MINBRA_TGG_483_c1_g1_i1.mrna1.cds1;Name=MINBRA_TGG_483_c1_g1_i1;Parent=MINBRA_TGG_483_c1_g1_i1.mrna1;Target=MINBRA_TGG_483_c1_g1_i1 231 91 .&#xA;###&#xA;tig00001947 MINBRA  gene    37526   38961   .   +   .   ID=MINBRA_TGG_483_c1_g1_i1.path2;Name=MINBRA_TGG_483_c1_g1_i1&#xA;tig00001947 MINBRA  mRNA    37526   38961   .   +   .   ID=MINBRA_TGG_483_c1_g1_i1.mrna2;Name=MINBRA_TGG_483_c1_g1_i1;Parent=MINBRA_TGG_483_c1_g1_i1.path2;coverage=100.0;identity=98.7;matches=1435;mismatches=1;indels=18;unknowns=0&#xA;tig00001947 MINBRA  exon    37526   38961   98  +   .   ID=MINBRA_TGG_483_c1_g1_i1.mrna2.exon1;Name=MINBRA_TGG_483_c1_g1_i1;Parent=MINBRA_TGG_483_c1_g1_i1.mrna2;Target=MINBRA_TGG_483_c1_g1_i1 1454 1 .&#xA;tig00001947 MINBRA  CDS 37614   37750   97  +   0   ID=MINBRA_TGG_483_c1_g1_i1.mrna2.cds1;Name=MINBRA_TGG_483_c1_g1_i1;Parent=MINBRA_TGG_483_c1_g1_i1.mrna2;Target=MINBRA_TGG_483_c1_g1_i1 231 91 .&#xA;###&#xA;tig00022147 MINBRA  gene    464550  465456  .   +   .   ID=MINBRA_TGG_489_c13_g1_i2.path1;Name=MINBRA_TGG_489_c13_g1_i2&#xA;tig00022147 MINBRA  mRNA    464550  465456  .   +   .   ID=MINBRA_TGG_489_c13_g1_i2.mrna1;Name=MINBRA_TGG_489_c13_g1_i2;Parent=MINBRA_TGG_489_c13_g1_i2.path1;coverage=100.0;identity=99.3;matches=907;mismatches=0;indels=6;unknowns=0&#xA;tig00022147 MINBRA  exon    464550  465456  99  +   .   ID=MINBRA_TGG_489_c13_g1_i2.mrna1.exon1;Name=MINBRA_TGG_489_c13_g1_i2;Parent=MINBRA_TGG_489_c13_g1_i2.mrna1;Target=MINBRA_TGG_489_c13_g1_i2 913 1 .&#xA;tig00022147 MINBRA  CDS 464909  465053  98  +   0   ID=MINBRA_TGG_489_c13_g1_i2.mrna1.cds1;Name=MINBRA_TGG_489_c13_g1_i2;Parent=MINBRA_TGG_489_c13_g1_i2.mrna1;Target=MINBRA_TGG_489_c13_g1_i2 507 361 .&#xA;###&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;GMAP told me on standard output what transcripts were not mappable to the genome (136 of 36830, or about 0.4%).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've now used STAR to map these cDNA reads to the transcripts (using the GFF3 annotation file for initial evidence) in order to confirm that the mapping carried out by GMAP is reasonable. I ended up throwing out another 7.4% of transcripts. First, I generated a genome index using the generated gff3 file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;~/install/star/STAR-2.5.4b/bin/Linux_x86_64/STAR --runThreadN 10 \&#xA;   --runMode genomeGenerate --genomeDir ./star_genome \&#xA;   --genomeFastaFiles Nb_ONTCFED_65bpTrim_t1.contigs.hpcleaned.fasta \ &#xA;   --sjdbGTFfile Nb_TrinTran_vs_Canu.gff3 \&#xA;   --sjdbGTFtagExonParentTranscript Parent&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Followed by mapping my RNASeq experiments using STAR. STAR can sort the genome mapping, but not the transcriptome mapping, so an additional sort/index was done on that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for x in $(seq 1 9); do&#xA;  ~/install/star/STAR-2.5.4b/bin/Linux_x86_64/STAR --runThreadN 10 \&#xA;     --readFilesCommand zcat --genomeDir ./star_genome \&#xA;     --readFilesIn *-1563-0${x}-*.fastq.gz \&#xA;     --outFileNamePrefix mapped/1563-0${x}_vs_BNOCFED \&#xA;     --outSAMtype BAM SortedByCoordinate --quantMode TranscriptomeSAM&#xA;  samtools sort mapped/1563-0${x}.toTranscriptome.*.bam \&#xA;    &amp;gt; mapped/1563-0${x}.toTranscriptome.sortedByCoord.out.bam&#xA;  samtools index mapped/1563-0${x}.toTranscriptome.sortedByCoord.out.bam&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;BAM hits were collated into a single file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(for x in $(seq 1 9); &#xA;    do samtools idxstats 1563-0${x}.toTranscriptome.sortedByCoord.out.bam | \&#xA;      perl -pe &quot;s/^/1563-0${x}\t/&quot;; &#xA; done) | gzip  &amp;gt; readCounts_all_vs_transcriptome.tsv.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which was then processed using R to identify unmapped transcript annotations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/Rscript&#xA;data.df &amp;lt;-&#xA;    read.delim(&quot;readCounts_all_vs_transcriptome.tsv.gz&quot;, header=FALSE,&#xA;               col.names=c(&quot;Sample&quot;,&quot;Transcript&quot;,&#xA;                           &quot;Length&quot;,&quot;Mapped&quot;,&quot;Unmapped&quot;));&#xA;gene.counts &amp;lt;- tapply(data.df$Mapped + data.df$Unmapped,&#xA;                      list(data.df$Transcript, data.df$Sample), sum);&#xA;sum(rowSums(gene.counts)==0) / nrow(gene.counts);&#xA;TranscriptsToRemove &amp;lt;- rownames(gene.counts)[rowSums(gene.counts) == 0];&#xA;cat(TranscriptsToRemove, file=&quot;transcripts_to_remove.txt&quot;, sep=&quot;\n&quot;);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The unmapped annotations were then excluded from the gff3 file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -v -F -f mapped/transcripts_to_remove.txt Nb_TrinTran_vs_Canu.gff3 \&#xA;  &amp;gt; mappable_Nb_TrinTran_vs_Canu.gff3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-04-10T23:42:16.903" LastActivityDate="2018-04-10T23:42:16.903" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4027" PostTypeId="2" ParentId="3947" CreationDate="2018-04-09T04:18:06.873" Score="1" Body="&lt;p&gt;If you're still looking for an answer on this, I would use the &lt;code&gt;gather&lt;/code&gt; and &lt;code&gt;spread&lt;/code&gt; functions in the &lt;code&gt;tidyr&lt;/code&gt; package (part of the &lt;code&gt;tidyverse&lt;/code&gt; suite of packages - which also includes &lt;code&gt;dplyr&lt;/code&gt; and &lt;code&gt;ggplot2&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse) &#xA;df2_long &amp;lt;- gather(PATIENT_ID, value, -Hugo_Symbol)&#xA;df1 %&amp;gt;% &#xA;  left_join(df2_long) %&amp;gt;%   # merges the 2 data frames together&#xA;  spread(Hugo_Symbol, value)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2632" LastActivityDate="2018-04-09T04:18:06.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4028" PostTypeId="2" ParentId="4015" CreationDate="2018-04-09T09:21:13.040" Score="2" Body="&lt;p&gt;Your confusion is understandable, since it isn't explained in the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/refseq/targetedloci/16S_process&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI description&lt;/a&gt; of the database, but most of the sequences in the database are derived from DNA that encodes the 16S rRNA &lt;em&gt;gene&lt;/em&gt;, not from the rRNA itself. In some cases, sequences are derived from cDNA that is synthesized using the actual rRNA as a template, but it's still DNA sequencing. Direct RNA sequencing is &lt;a href=&quot;https://www.nature.com/articles/nmeth.4577&quot; rel=&quot;nofollow noreferrer&quot;&gt;extremely uncommon&lt;/a&gt;. Depending on the cDNA sequencing protocol (first strand vs. second strand sequencing), the transcript sequence might be the reverse-complement of the 16S rDNA, but the only difference that you would see is that the numbering of the bases would be reversed. If you are sequencing rDNA from organelles, there may also be a few bases that are different, due to &lt;a href=&quot;https://en.wikipedia.org/wiki/RNA_editing&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNA editing&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-04-09T09:21:13.040" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4029" PostTypeId="2" ParentId="4025" CreationDate="2018-04-09T09:29:20.213" Score="5" Body="&lt;p&gt;You are right, &lt;code&gt;samples.txt&lt;/code&gt; is not generated by Salmon (or any other transcript abundance quantifiers). From &lt;a href=&quot;https://bioconductor.org/packages/devel/data/experiment/vignettes/tximportData/inst/doc/tximportData.html&quot; rel=&quot;noreferrer&quot;&gt;documentation&lt;/a&gt;, you can find a &lt;a href=&quot;http://www.geuvadis.org/web/geuvadis&quot; rel=&quot;noreferrer&quot;&gt;link&lt;/a&gt; to an example of how &lt;code&gt;samples.txt&lt;/code&gt; should look like. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: At the moment, the server is unavailable (I got an &lt;code&gt;Error 503: Service Temporarily Unavailable&lt;/code&gt;) - I'd suggest you get directly in contact with the owners / &lt;a href=&quot;https://bioconductor.org/packages/devel/data/experiment/vignettes/tximportData/inst/doc/tximportData.html&quot; rel=&quot;noreferrer&quot;&gt;authors&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Luckily, there is a call of the function importing &lt;code&gt;samples.txt&lt;/code&gt; file in the &lt;a href=&quot;https://bioconductor.org/packages/devel/data/experiment/vignettes/tximportData/inst/doc/tximportData.html&quot; rel=&quot;noreferrer&quot;&gt;doc&lt;/a&gt; itself (variable &lt;code&gt;samples&lt;/code&gt;), so you can get an idea of ow you have to structure it: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; samples&#xA;&#xA;##   pop center                assay    sample experiment       run&#xA;## 1 TSI  UNIGE NA20503.1.M_111124_5 ERS185497  ERX163094 ERR188297&#xA;## 2 TSI  UNIGE NA20504.1.M_111124_7 ERS185242  ERX162972 ERR188088&#xA;## 3 TSI  UNIGE NA20505.1.M_111124_6 ERS185048  ERX163009 ERR188329&#xA;## 4 TSI  UNIGE NA20507.1.M_111124_7 ERS185412  ERX163158 ERR188288&#xA;## 5 TSI  UNIGE NA20508.1.M_111124_2 ERS185362  ERX163159 ERR188021&#xA;## 6 TSI  UNIGE NA20514.1.M_111124_4 ERS185217  ERX163062 ERR188356&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As an alternate workaround, find the &lt;code&gt;samples.txt&lt;/code&gt; format directly from &lt;a href=&quot;https://bioconductor.org/packages/release/data/experiment/html/tximportData.html&quot; rel=&quot;noreferrer&quot;&gt;tximportData Bioconductor page&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;download &lt;a href=&quot;https://bioconductor.org/packages/release/data/experiment/src/contrib/tximportData_1.6.0.tar.gz&quot; rel=&quot;noreferrer&quot;&gt;source code&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;extract&lt;/li&gt;&#xA;&lt;li&gt;navigate to: &lt;code&gt;[...]/tximportData 2/inst/extdata&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;read content of &lt;code&gt;samples.txt&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="294" LastActivityDate="2018-04-09T09:29:20.213" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4030" PostTypeId="1" AcceptedAnswerId="4031" CreationDate="2018-04-09T12:56:44.967" Score="4" ViewCount="539" Body="&lt;p&gt;Consider an &lt;em&gt;m x n&lt;/em&gt; genotype matrix of &lt;em&gt;m&lt;/em&gt; haploid samples and &lt;em&gt;n&lt;/em&gt; SNPs where each value is an allele encoded by an integer (0,1,2,3). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a good/standard way to encode the alleles in order to perform a PCA on this matrix to investigate population structure ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen this with matrices of diploid samples where each SNP is encoded as 0/1/2 to represent the number of non-reference allele, but is there a way to consider more than 2 alleles ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g. difference between alleles 1 and 3 should be equal to difference between alleles 1 and 2&lt;/p&gt;&#xA;" OwnerUserId="439" LastEditorUserId="439" LastEditDate="2018-04-11T10:30:24.170" LastActivityDate="2018-04-11T14:54:58.677" Title="PCA on genotype matrix with multiple alleles" Tags="|snp|pca|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4031" PostTypeId="2" ParentId="4030" CreationDate="2018-04-09T13:01:49.597" Score="3" Body="&lt;p&gt;You can make a 'dummy variable' for each allele. That means that you don't have info per SNP, but for SNPs with more alleles, the allele is present (1) or not (0).&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2018-04-09T13:01:49.597" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4032" PostTypeId="2" ParentId="3999" CreationDate="2018-04-09T19:24:03.817" Score="3" Body="&lt;p&gt;Here is what I would do in Python. There are a few extra checks I've included, and my definitions of a few cases are different than Heng Li's above, @user172818. The performance is fairly similar. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def create_cigar(reference, alignment):&#xA;    '''Construct CIGAR from reference and alignment'''&#xA;    ## if reference and alignment have unequal lengths, stop&#xA;    if len(reference) != len(alignment):&#xA;        raise Exception('Error: reference and alignment are of unequal length')&#xA;    ## replace any '-' and '*' to '_'&#xA;    reference = reference.translate(bytes.maketrans(b&quot;*-&quot;, b&quot;__&quot;))&#xA;    alignment = alignment.translate(bytes.maketrans(b&quot;*-&quot;, b&quot;__&quot;))&#xA;    ### creates string with a CIGAR symbol for each position using zip()&#xA;    compare_strings = ''.join(['=', 'X'][ref == '_'] if ref == algn else 'X' if ref != '_' and algn != '_' else ['I', 'D'][algn == '_' and ref != '_'] for ref, algn in zip(reference, alignment))&#xA;    ## now, parse string into CIGAR format; faster than itertools&#xA;    total_match = 0         # total number of adjacent matches&#xA;    current_base = compare_strings[0]     # current character being processed&#xA;    result = ''     # result of function&#xA;    for i in range(len(compare_strings)):&#xA;        if compare_strings[i] == current_base:&#xA;            total_match += 1&#xA;        else:&#xA;            result += str(total_match) + current_base&#xA;            current_base = compare_strings[i]&#xA;            total_match = 1&#xA;    result += str(total_match) + current_base&#xA;    return result&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Following the example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print(create_cigar('ACGTAT-CT', 'ACGT-TGGA'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which outputs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;4=1D1=1I2X&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(1) I standard the inputs, such that gaps can be either &lt;code&gt;-&lt;/code&gt;, &lt;code&gt;_&lt;/code&gt;, or &lt;code&gt;*&lt;/code&gt;. (Users could add their own.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) There's no &lt;code&gt;N&lt;/code&gt;, skipped regions, though based on the use case, I could see users writing this, e.g. a deletion &gt;= size some number could work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(3) Hard and soft padding do not make sense in this context. These will be classified as &lt;code&gt;X&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(4) By my definition above, if there's a gap in the reference and the alignment, this is a padding &lt;code&gt;P&lt;/code&gt;. &lt;/p&gt;&#xA;" OwnerUserId="1770" LastActivityDate="2018-04-09T19:24:03.817" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4033" PostTypeId="1" CreationDate="2018-04-10T00:30:48.497" Score="0" ViewCount="443" Body="&lt;p&gt;I am using gene expression data from a microarray dataset for the purpose of learning the network structure of genes in the dataset. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now to do this I need to binarize my data, i.e convert the data for each gene to 0s and 1s. I am new to bioinformatics and am not sure what approach is considered standard for microarray data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I appreciate your inputs regarding this and I am implementing this in R. So if you know any package that does this please let me know.&lt;/p&gt;&#xA;" OwnerUserId="1782" LastEditorUserId="48" LastEditDate="2018-04-10T09:11:16.790" LastActivityDate="2018-06-09T10:32:25.227" Title="What is a standard approach to binarize microarray gene expression data?" Tags="|r|microarray|data-preprocessing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4035" PostTypeId="2" ParentId="4033" CreationDate="2018-04-10T06:20:44.623" Score="2" Body="&lt;p&gt;Based on the info you provide, &lt;a href=&quot;https://cran.r-project.org/web/packages/ArrayBin/ArrayBin.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ArrayBin&lt;/a&gt; R package provides you the necessary tools:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;binarize.array()&lt;/code&gt; from &lt;a href=&quot;https://cran.r-project.org/web/packages/ArrayBin/ArrayBin.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ArrayBin&lt;/a&gt;, allowing:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Implementation of an adaptive method for binarizing gene expression data on a per-probe basis and demonstrate the superior effectiveness of our method when compared with other, commonly used approaches. This adaptive binarization method can be applied to DNA methylation microarray data,&#xA;  which has implications for cross-platform integration, and can reduce batch effects in the data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In complement, I'd advise you go through some literature and choose the approach which fits you best. Some suggestions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.nature.com/articles/35076576&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational analysis of microarray data&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://link.springer.com/article/10.1007/s10618-009-0145-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Binary matrix factorization for analyzing gene expression data&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;and probably most relevant for you given your application: &lt;a href=&quot;http://sci-hub.tw/10.1109/tcbb.2011.62&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multiscale Binarization of Gene Expression Data for Reconstructing Boolean Networks&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Maybe you also want to look into more general concepts of transformation prior to microarray data analysis:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.cs.cmu.edu/~zivbj/class05/reading/norm.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Microarray data normalization and transformation&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.ub.edu/stat/docencia/bioinformatica/microarrays/ADM/slides/A_Tutorial_Review_of_Microarray_data_Analysis_17-06-08.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;A Tutorial Review of Microarray Data Analysis&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="294" LastActivityDate="2018-04-10T06:20:44.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4036" PostTypeId="2" ParentId="4013" CreationDate="2018-04-10T12:17:02.027" Score="5" Body="&lt;p&gt;From the page &lt;a href=&quot;https://www.idtdna.com/pages/education/decoded/article/use-of-template-switching-oligos-(ts-oligos-tsos)-for-efficient-cdna-library-construction&quot; rel=&quot;nofollow noreferrer&quot;&gt;you cited&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;During first-strand synthesis, upon reaching the 5’ end of the RNA template, the terminal transferase activity of the MMLV reverse transcriptase adds a few additional nucleotide&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That usually are CCC (1st is almost always C and the other two most of the times), see: &lt;a href=&quot;http://journals.plos.org/plosone/article?id=10.1371/journal.pone.0085270&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zajac et al., 2013&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These CCCs are used for priming the Template Switching Oligo (TSO), that has GGG and whatever you want to add to the sequence (for instance UMIs and adapters). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, answering to your question the &lt;strong&gt;polimerase does not switch strands&lt;/strong&gt;, it &lt;strong&gt;switches from mRNA as a template to TSO as template&lt;/strong&gt;. Because of that is called template switching.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/w1D7z.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/w1D7z.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see it in the graph from the page you cited. In the template switching step polimerase switch from the mRNA to the TSO and continue with the synthesis of the first cDNA strand using TSO as template. &lt;/p&gt;&#xA;" OwnerUserId="678" LastEditorUserId="678" LastEditDate="2018-04-10T13:13:58.147" LastActivityDate="2018-04-10T13:13:58.147" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4038" PostTypeId="1" AcceptedAnswerId="4047" CreationDate="2018-04-10T21:00:22.507" Score="0" ViewCount="1688" Body="&lt;p&gt;I am trying to use &lt;code&gt;tximport&lt;/code&gt; to get raw count matrices of a bulk RNA-seq sequencing experiment. I followed the tutorial:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioconductor.org/packages/3.7/bioc/vignettes/tximport/inst/doc/tximport.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/3.7/bioc/vignettes/tximport/inst/doc/tximport.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where they generate &lt;code&gt;tx2gene&lt;/code&gt; in the following way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(TxDb.Hsapiens.UCSC.hg19.knownGene)&#xA;txdb &amp;lt;- TxDb.Hsapiens.UCSC.hg19.knownGene&#xA;k &amp;lt;- keys(txdb, keytype = &quot;TXNAME&quot;)&#xA;tx2gene &amp;lt;- select(txdb, k, &quot;GENEID&quot;, &quot;TXNAME&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, I reasoned to do it the same way. But since my data is mouse one, I used a different gene annotation file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(TxDb.Mmusculus.UCSC.mm10.knownGene)&#xA;txdb &amp;lt;- TxDb.Mmusculus.UCSC.mm10.knownGene&#xA;k &amp;lt;- keys(txdb, keytype = &quot;TXNAME&quot;)&#xA;tx2gene &amp;lt;- select(txdb, k, &quot;GENEID&quot;, &quot;TXNAME&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The transcript annotation file that I used when running &lt;code&gt;salmon&lt;/code&gt; is &lt;code&gt;gencode.vM13.transcripts.fa.sidx&lt;/code&gt;. Then when I try getting &lt;code&gt;txi&lt;/code&gt; object:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;txi &amp;lt;- tximport(files, type = &quot;salmon&quot;, tx2gene = tx2gene)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It raises the error:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vCt2U.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vCt2U.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seems like I am creating &lt;code&gt;tx2gene&lt;/code&gt; in the wrong way. Maybe I need to use a different library, not &lt;code&gt;TxDb.Mmusculus.UCSC.mm10.knownGene&lt;/code&gt;. Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Update&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I looked into the &lt;code&gt;quant.sf&lt;/code&gt; files that I got from &lt;code&gt;salmon&lt;/code&gt; and it looked the following way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LmbuM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LmbuM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A colleague adviced me to strip away everything that follows the dot in the &lt;code&gt;Name&lt;/code&gt; column, so it started to look:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LaSa2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LaSa2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, still no luck. The identifiers in the &lt;code&gt;k &amp;lt;- keys(txdb, keytype = &quot;TXNAME&quot;)&lt;/code&gt; look like that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;uc007afg.1&quot; &quot;uc007afh.1&quot; &quot;uc007afi.2&quot; &quot;uc011wht.1&quot; &quot;uc011whu.1&quot; &#xA;&quot;uc057aty.1&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I suppose I need to convert these identifiers first to &lt;code&gt;ENS&lt;/code&gt;-like ones, but do not know how to.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="2005" LastEditDate="2018-04-11T19:13:46.380" LastActivityDate="2018-04-11T21:09:08.857" Title="tx2gene file for tximport issue" Tags="|rna-seq|salmon|tximport|tx2gene|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4039" PostTypeId="2" ParentId="4038" CreationDate="2018-04-10T21:30:54.183" Score="1" Body="&lt;p&gt;Do your ids in the expression file (from gencode) match up with the gene IDs in the transcript file (from UCSC?) I'm willing to bet that they don't. Stay consistent throughout your process. Either pull down the gencode transcript to gene translations, or do your pseudoalignments to UCSC transcripts.&lt;/p&gt;&#xA;" OwnerUserId="74" LastActivityDate="2018-04-10T21:30:54.183" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4040" PostTypeId="2" ParentId="3449" CreationDate="2018-04-10T22:41:03.493" Score="4" Body="&lt;p&gt;There are competing claims regarding how well ComBat works on scRNA-seq datasets. A &lt;a href=&quot;https://www.nature.com/articles/nbt.4091&quot; rel=&quot;nofollow noreferrer&quot;&gt;recent paper&lt;/a&gt; from John Marioni's introduces a mutual nearest neighbors method that seems to outperform ComBat in at least some relevant scenarios. In general, it's best to look at tSNE or other diagnostic plots after batch correction to see if the results seem reasonable (this advice holds generally).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-10T22:41:03.493" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4041" PostTypeId="2" ParentId="4011" CreationDate="2018-04-11T09:17:01.823" Score="4" Body="&lt;p&gt;If what you want is to split the main VCF file into 1 file per sample, you could use bcftools &lt;a href=&quot;http://www.htslib.org/doc/bcftools.html#query&quot; rel=&quot;nofollow noreferrer&quot;&gt;query&lt;/a&gt; and &lt;a href=&quot;http://www.htslib.org/doc/bcftools.html#view&quot; rel=&quot;nofollow noreferrer&quot;&gt;view&lt;/a&gt; commands.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A similar question was asked on &lt;a href=&quot;https://www.biostars.org/p/130456/&quot; rel=&quot;nofollow noreferrer&quot;&gt;biostars&lt;/a&gt;, adapting Jorge Amigo's solution to your situation, you might use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for sample in $(bcftools query -l $combined_vcf)&#xA;do&#xA;    bcftools view -c1 -Ov -s $sample -o ${$sample.vcf} $file&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="439" LastActivityDate="2018-04-11T09:17:01.823" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4042" PostTypeId="2" ParentId="4030" CreationDate="2018-04-11T09:59:17.500" Score="2" Body="&lt;p&gt;I recommend the &lt;a href=&quot;https://en.wikipedia.org/wiki/Hamming_distance&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hamming distance&lt;/a&gt; and doing a &lt;a href=&quot;https://en.wikipedia.org/wiki/Multidimensional_scaling&quot; rel=&quot;nofollow noreferrer&quot;&gt;multidimensional scaling&lt;/a&gt;  (a procedure &lt;a href=&quot;https://stats.stackexchange.com/q/14002/105234&quot;&gt;similar&lt;/a&gt; to PCA but for distances), that way you don't create new variables for the same position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distance function can be defined as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;hamming_dist &amp;lt;- function(x, y) {&#xA;    if (x != y) {&#xA;        1&#xA;    } else {&#xA;        0&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or you can use a package:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(e1071)&#xA;H &amp;lt;- hamming.distance(as.matrix(X))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or as you yourself pointed &lt;a href=&quot;https://johanndejong.wordpress.com/2015/10/02/faster-hamming-distance-in-r-2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;other implementations&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use them to calculate the distance between all samples and then use it for the multidimensional analysis (&lt;code&gt;cmdscale&lt;/code&gt; in R)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plot(cmdscale(H))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2018-04-11T14:54:58.677" LastActivityDate="2018-04-11T14:54:58.677" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4043" PostTypeId="1" AcceptedAnswerId="4056" CreationDate="2018-04-11T14:43:55.777" Score="-2" ViewCount="108" Body="&lt;p&gt;I would like to do a differential gene expression analysis on a microarray data.  From the literature, I understood, &lt;strong&gt;call detection (Present, Absent, Marginal)&lt;/strong&gt; has to be done to minimize false positives. I found this &lt;a href=&quot;http://faculty.ucr.edu/~tgirke/Documents/R_BioCond/ProtocolSimpleaffy.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; for the call detection and also to find the DEGs, &lt;strong&gt;from .cel files.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this particular &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE837&quot; rel=&quot;nofollow noreferrer&quot;&gt;dataset&lt;/a&gt; used for my study, &lt;strong&gt;.cel file is unavailable&lt;/strong&gt; and only .txt file/ SOFT file is available for download. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Can call detection be done from these files? &lt;/li&gt;&#xA;&lt;li&gt;Or is there any other method/ package that can be used for the same?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2645" LastEditorUserId="2645" LastEditDate="2018-04-13T05:27:04.780" LastActivityDate="2018-04-13T07:47:43.137" Title="How to calculate call detection values in microarray datase if .cel file unavailable" Tags="|bioconductor|file-formats|differential-expression|microarray|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4044" PostTypeId="2" ParentId="2073" CreationDate="2018-04-11T15:08:30.363" Score="1" Body="&lt;p&gt;GSEAPreranked is a limited version of GSEA. &#xA;If your sample number is enough(more than 5,6 samples in each condition), you can use GSEA version. &#xA;Moreover, you can ask this question to the gsea-help google group. &lt;/p&gt;&#xA;" OwnerUserId="2647" LastActivityDate="2018-04-11T15:08:30.363" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4045" PostTypeId="1" CreationDate="2018-04-11T19:27:37.310" Score="1" ViewCount="37" Body="&lt;p&gt;I have a list of primers for sequencing fungal ITS1/2 regions. I also have access to different fungal databases. I would like to perform &lt;em&gt;in silico&lt;/em&gt; PCR analysis for comparing different primer pairs and choose the best one, which means, greater taxa amplification and the greater quantity of unique amplicons.&lt;/p&gt;&#xA;" OwnerUserId="2648" LastActivityDate="2018-04-11T19:27:37.310" Title="Which software is recommended for sequencing primers selection?" Tags="|ngs|primer|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4046" PostTypeId="2" ParentId="4024" CreationDate="2018-04-11T20:38:44.463" Score="2" Body="&lt;p&gt;So, we figured it out. If you delete everything until the &lt;code&gt;[Data]&lt;/code&gt; tag in the sample sheet, leaving the &lt;code&gt;[Data]&lt;/code&gt; tag, it starts working. Here is the full &lt;code&gt;sample_sheet.csv&lt;/code&gt; that is working:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[Data]&#xA;Lane,Sample_ID,Sample_Name,index,Sample_Project&#xA;1,SI-GA-B4_1,17R,ACTTCATA,Chromium_20180409&#xA;1,SI-GA-B4_2,17R,GAGATGAC,Chromium_20180409&#xA;1,SI-GA-B4_3,17R,TGCCGTGG,Chromium_20180409&#xA;1,SI-GA-B4_4,17R,CTAGACCT,Chromium_20180409&#xA;1,SI-GA-B5_1,19RL,AATAATGG,Chromium_20180409&#xA;1,SI-GA-B5_2,19RL,CCAGGGCA,Chromium_20180409&#xA;1,SI-GA-B5_3,19RL,TGCCTCAT,Chromium_20180409&#xA;1,SI-GA-B5_4,19RL,GTGTCATC,Chromium_20180409&#xA;2,SI-GA-B4_1,17R,ACTTCATA,Chromium_20180409&#xA;2,SI-GA-B4_2,17R,GAGATGAC,Chromium_20180409&#xA;2,SI-GA-B4_3,17R,TGCCGTGG,Chromium_20180409&#xA;2,SI-GA-B4_4,17R,CTAGACCT,Chromium_20180409&#xA;2,SI-GA-B5_1,19RL,AATAATGG,Chromium_20180409&#xA;2,SI-GA-B5_2,19RL,CCAGGGCA,Chromium_20180409&#xA;2,SI-GA-B5_3,19RL,TGCCTCAT,Chromium_20180409&#xA;2,SI-GA-B5_4,19RL,GTGTCATC,Chromium_20180409&#xA;3,SI-GA-B4_1,17R,ACTTCATA,Chromium_20180409&#xA;3,SI-GA-B4_2,17R,GAGATGAC,Chromium_20180409&#xA;3,SI-GA-B4_3,17R,TGCCGTGG,Chromium_20180409&#xA;3,SI-GA-B4_4,17R,CTAGACCT,Chromium_20180409&#xA;3,SI-GA-B5_1,19RL,AATAATGG,Chromium_20180409&#xA;3,SI-GA-B5_2,19RL,CCAGGGCA,Chromium_20180409&#xA;3,SI-GA-B5_3,19RL,TGCCTCAT,Chromium_20180409&#xA;3,SI-GA-B5_4,19RL,GTGTCATC,Chromium_20180409&#xA;4,SI-GA-B4_1,17R,ACTTCATA,Chromium_20180409&#xA;4,SI-GA-B4_2,17R,GAGATGAC,Chromium_20180409&#xA;4,SI-GA-B4_3,17R,TGCCGTGG,Chromium_20180409&#xA;4,SI-GA-B4_4,17R,CTAGACCT,Chromium_20180409&#xA;4,SI-GA-B5_1,19RL,AATAATGG,Chromium_20180409&#xA;4,SI-GA-B5_2,19RL,CCAGGGCA,Chromium_20180409&#xA;4,SI-GA-B5_3,19RL,TGCCTCAT,Chromium_20180409&#xA;4,SI-GA-B5_4,19RL,GTGTCATC,Chromium_20180409&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is very unfortunate and disappointing that &lt;code&gt;10X&lt;/code&gt; website is generating wrong sample sheet. Still we got very low mapping rates and trying to figure out what went wrong.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Update&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I was supplying &lt;code&gt;--sample-sheet=&lt;/code&gt; property, but it should be &lt;code&gt;--samplesheet=&lt;/code&gt;. Now it is running fine (except for very low mapping rates) with all the headers on.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Update&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It turned out that we were given the wrong indexes, so we made a sample sheet using sample sheet generator again and even with &lt;code&gt;--samplesheet=&lt;/code&gt; correct parameter it was not running. Convertion of &lt;code&gt;CLRF&lt;/code&gt; with &lt;code&gt;perl&lt;/code&gt; or &lt;code&gt;tr&lt;/code&gt; did not help. However, concise format of the sample sheet worked out well which looks like the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Lane,Sample,Index&#xA;1-4,17R,SI-GA-B8&#xA;1-4,19RL,SI-GA-B9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It needs to be supplied with &lt;code&gt;--csv=&lt;/code&gt; parameter.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="2005" LastEditDate="2018-04-18T21:41:18.510" LastActivityDate="2018-04-18T21:41:18.510" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4047" PostTypeId="2" ParentId="4038" CreationDate="2018-04-11T21:09:08.857" Score="2" Body="&lt;p&gt;The solution was to use a different library as I expected. It should be &lt;code&gt;library(TxDb.Hsapiens.UCSC.hg19.ensGene)&lt;/code&gt;. And also, as specified in the question, to remove everything after the dot in the &lt;code&gt;Name&lt;/code&gt; column of the &lt;code&gt;quant.sf&lt;/code&gt; files.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-04-11T21:09:08.857" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4048" PostTypeId="1" AcceptedAnswerId="4049" CreationDate="2018-04-11T23:14:11.773" Score="0" ViewCount="75" Body="&lt;p&gt;I am creating &lt;code&gt;DeSeqDataSet&lt;/code&gt; and I am unsure whether I need to include the number of mice into the design formula. Basically, the samples look like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ayyrm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ayyrm.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have 4 different groups: &lt;code&gt;GF&lt;/code&gt;, &lt;code&gt;B6&lt;/code&gt;, &lt;code&gt;ET&lt;/code&gt; and &lt;code&gt;Wld&lt;/code&gt;. It is obvious that these should go into the design formula, but I am not sure about the mice number. Basically for each group, e.g. &lt;code&gt;GF&lt;/code&gt;, there are 4 experiments with &lt;code&gt;1&lt;/code&gt;, &lt;code&gt;2&lt;/code&gt;, &lt;code&gt;3&lt;/code&gt; and &lt;code&gt;4&lt;/code&gt; mice each. When I am trying to use both &lt;code&gt;Groups&lt;/code&gt; and &lt;code&gt;Mouse&lt;/code&gt; columns as a design formula, &lt;code&gt;R&lt;/code&gt; is telling that I need to convert &lt;code&gt;Mouse&lt;/code&gt; to &lt;code&gt;factor&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3qc1w.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3qc1w.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am a bit confused about two things here: first, as I told already, should I include this &lt;code&gt;Mouse&lt;/code&gt; column at all in the design formula of &lt;code&gt;DeSeqDataSet&lt;/code&gt; and second, if I do need it, how to convert it to factor? Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It turned out that I was wrong in my initial assumptions: &lt;code&gt;Mouse#&lt;/code&gt; really was an ‘id’ which I did not include in the design formula. There were 4 groups of 16 mice in each for 4 trials.&lt;/p&gt;&#xA;&lt;p&gt;What concerns me also is that if I specify design formula as &lt;code&gt;~ Groups + Mouse&lt;/code&gt;, the DeSeq() function starts comparing the &lt;code&gt;Mouse&lt;/code&gt; but not the &lt;code&gt;Groups&lt;/code&gt; which is the most interesting for me.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="73" LastEditDate="2022-08-27T09:33:16.297" LastActivityDate="2022-08-27T09:33:16.297" Title="DeSeqDataSet experimental design: column with integer values" Tags="|rna-seq|deseq2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4049" PostTypeId="2" ParentId="4048" CreationDate="2018-04-12T00:24:24.813" Score="5" Body="&lt;p&gt;Any IDs (e.g. replicate number) don't need to be included in the design formula if it's not a repeated factor in multiple samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, you have said that mouse number is related to the number of mice used in the pool (and is a variable factor that is consistent across multiple samples), in which case it should be included in the design matrix. This would also be the case if the &lt;em&gt;same&lt;/em&gt; mouse were being used for multiple RNASeq experiments. Converting an integer value to a factor is done by the &lt;code&gt;factor&lt;/code&gt; method:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sampleTable$MiceUsed &amp;lt;- factor(sampleTable$MiceUsed);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Other examples of things that might be useful in the design formula would be sex, age, and sequencing run batch (assuming these are not consistent across samples).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a design formula of something like &lt;code&gt;~Groups + Mouse&lt;/code&gt;, it will also be comparing Groups, it's just hidden in the results. The &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#interactions&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2 vignette&lt;/a&gt; is really great at explaining what's going on, but as a quick answer you can pull out the &lt;code&gt;Groups&lt;/code&gt; results with the &lt;code&gt;contrast&lt;/code&gt; function. This will pull out group differences that are independent of mouse number:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;results(dds, contrast=c(&quot;Groups&quot;, &quot;GF&quot;, &quot;Wld&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;[first value is the variable name, second value is the numerator condition, third is the denominator condition]&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-04-12T22:35:44.753" LastActivityDate="2018-04-12T22:35:44.753" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4050" PostTypeId="1" AcceptedAnswerId="4058" CreationDate="2018-04-12T15:52:09.170" Score="0" ViewCount="342" Body="&lt;p&gt;I have a small sample of records from a bam file. I obtained them by intersecting with a bed file not enforcing strand-specificity. So I get them all back upon performing the same intersection again:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedtools intersect -a sample.bam -b features.bed -wa -u -f 1.0 | samtools view&#xA;MN00481:17:000H2FH5N:1:11104:12480:15984    16  III 4499483 42  28M *   0   0   CAGGAACACGCACATCGTCGATTTTCAG    FFFFFAFFFFFFFFFFFFFFFFFFFFFF    AS:i:0  XN:i:0  XM:i:0  XO:i:0  XG:i:0  NM:i:0  MD:Z:28 YT:Z:UU&#xA;MN00481:17:000H2FH5N:1:11103:9925:11522 16  IV  2388448 42  30M *   0   0   CACTGGAACTGGTCCGATCTTCAAGCCGGT  FFFFFFFFFFA/FFFFAFFFFFF/FFAFAF  AS:i:0  XN:i:0  XM:i:0  XO:i:0  XG:i:0  NM:i:0  MD:Z:30 YT:Z:UU&#xA;MN00481:17:000H2FH5N:1:22109:11939:5243 16  IV  4749336 42  29M *   0   0   ATGTGGATGCGGTCAGCTTGGGCACGATG   FFFFFFFFFFFF/FFFF6FFFFFFFF/FF   AS:i:0  XN:i:0  XM:i:0  XO:i:0  XG:i:0  NM:i:0  MD:Z:29 YT:Z:UU&#xA;MN00481:17:000H2FH5N:1:22104:17385:19431    16  V   4443064 42  29M *   0   0   CAAAATGTGAACAGGAACGTTTTCATGAG   F/FFFFFFFFFFFFFFFFFFFFFFFFFFF   AS:i:0  XN:i:0  XM:i:0  XO:i:0  XG:i:0  NM:i:0  MD:Z:29 YT:Z:UU&#xA;MN00481:17:000H2FH5N:1:23105:22058:12968    0   X   981524  42  29M *   0   0   CCATCGATTCTCATTTTAATGCTGGTGCT   FFFFFFFFFFFFFFFFFFFFFFFFFFFFF   AS:i:0  XN:i:0  XM:i:0  XO:i:0  XG:i:0  NM:i:0  MD:Z:29 YT:Z:UU&#xA;MN00481:17:000H2FH5N:1:12104:7867:7500  16  X   4214381 42  29M *   0   0   GCTGCAGGAATGACGGCTCTAGCAAACGT   FFFFFFFFFFFFFFFFFFFFFFFFFFFFF   AS:i:0  XN:i:0  XM:i:0  XO:i:0  XG:i:0  NM:i:0  MD:Z:29 YT:Z:UU&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now I would like to enforce strand-specificity, but in either case, none of the records pass the filter:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Same strand (&lt;code&gt;-s&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedtools intersect -a sample.bam -b features.bed -wa -u -f 1.0 -s | samtools view&#xA;$ # nothing here&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Opposite strand (&lt;code&gt;-S&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedtools intersect -a sample.bam -b features.bed -wa -u -f 1.0 -S | samtools view&#xA;$ # nothing here&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is the correct way of doing this?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Edit&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Bedtools version:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedtools --version&#xA;bedtools v2.27.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My bed file has strand information:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ tail features.bed &#xA;X   17617826    17619458    &quot;WBGene00015867&quot;    +   .&#xA;X   17622777    17640743    &quot;WBGene00019211&quot;    -   .&#xA;X   17650177    17651594    &quot;WBGene00044771&quot;    +   .&#xA;X   17662457    17665357    &quot;WBGene00003744&quot;    +   .&#xA;X   17671028    17680790    &quot;WBGene00020732&quot;    +   .&#xA;X   17680910    17682294    &quot;WBGene00044772&quot;    -   .&#xA;X   17695231    17701744    &quot;WBGene00017625&quot;    +   .&#xA;X   17710485    17711674    &quot;WBGene00019189&quot;    -   .&#xA;X   17714455    17714832    &quot;WBGene00007069&quot;    +   .&#xA;X   17714967    17718717    &quot;WBGene00007068&quot;    +   .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2018-04-13T11:07:32.353" LastActivityDate="2018-04-13T13:09:11.667" Title="Filtering bam file with bedtools intersect using strand specificity" Tags="|sam|bedtools|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4052" PostTypeId="1" AcceptedAnswerId="4063" CreationDate="2018-04-12T23:27:31.850" Score="4" ViewCount="177" Body="&lt;p&gt;I am trying to make &lt;code&gt;MA&lt;/code&gt;-plot for my &lt;code&gt;bulk-RNA-seq&lt;/code&gt; dataset and experiencing issues with it. I just copy-pasted the code that can be found here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://f1000research.com/articles/4-1070/v2&quot; rel=&quot;noreferrer&quot;&gt;https://f1000research.com/articles/4-1070/v2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(&quot;DESeq2&quot;)&#xA;plotMA(resLFC1, ylim=c(-5,5))&#xA;topGene &amp;lt;- rownames(resLFC1)[which.min(resLFC1$padj)]&#xA;with(resLFC1[topGene, ], {&#xA;  points(baseMean, log2FoldChange, col=&quot;dodgerblue&quot;, cex=2, lwd=2)&#xA;  text(baseMean, log2FoldChange, topGene, pos=2, col=&quot;dodgerblue&quot;)&#xA;})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I am not getting the right plot with the top gene circled, but the following one:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SwZ1R.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SwZ1R.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe it is a bug in the &lt;code&gt;plotMA&lt;/code&gt; function, not sure. How can I reproduce the figure from the publication?&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2018-09-04T10:07:07.053" LastActivityDate="2018-09-04T10:07:07.053" Title="plotMA function issue: can not circle genes" Tags="|rna-seq|deseq2|figure-reproduction|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4053" PostTypeId="1" AcceptedAnswerId="4054" CreationDate="2018-04-12T23:59:05.080" Score="0" ViewCount="2822" Body="&lt;p&gt;In the following &lt;a href=&quot;https://f1000research.com/articles/4-1070/v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;. There are two ways of getting highly variable genes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;resSig &amp;lt;- subset(res, padj &amp;lt; 0.1)&#xA;head(resSig[ order(resSig$log2FoldChange, decreasing=TRUE), ])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Getting genes that are statistically significant, then ordering them by the &lt;code&gt;log2FoldChange&lt;/code&gt; which is &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The column &lt;em&gt;log2FoldChange&lt;/em&gt; is the effect size estimate. It tells us how much the gene’s expression seems to have changed due to treatment with dexamethasone in comparison to untreated samples.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And the second one:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;topVarGenes &amp;lt;- head(order(rowVars(assay(rld)),decreasing=TRUE),20)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am struggling to understand what are the differences between the two ways of getting the most variable genes. What is the difference?&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2018-04-13T10:01:41.180" LastActivityDate="2018-04-13T10:01:41.180" Title="Highly variable genes analysis by DESeq2" Tags="|rna-seq|deseq2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4054" PostTypeId="2" ParentId="4053" CreationDate="2018-04-13T00:38:42.683" Score="4" Body="&lt;p&gt;The first method (&lt;code&gt;resSig$log2FoldChange&lt;/code&gt;) is for getting the most differentially expressed genes. These are the genes with the biggest differences between specific groups you pre-define.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second method (&lt;code&gt;rowVars(assay(rld))&lt;/code&gt;) is for getting the highly variable genes. These are the genes most variable across all samples regardless of which samples they are.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2018-04-13T00:38:42.683" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4055" PostTypeId="2" ParentId="2073" CreationDate="2018-04-13T00:42:26.610" Score="1" Body="&lt;p&gt;According to the &lt;a href=&quot;https://software.broadinstitute.org/cancer/software/gsea/wiki/index.php/FAQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSEA documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;we are recommending use of the GSEAPreranked tool for conducting gene&#xA;  set enrichment analysis of data derived from RNA-seq experiments&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Then more specifically to your question:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Based on your differential expression analysis, rank your features and&#xA;  capture your ranking in an RNK-formatted file. The ranking metric can&#xA;  be whatever measure of differential expression you choose from the&#xA;  output of your selected DE tool. For example, cuffdiff provides the&#xA;  (base 2) log of the fold change.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="35" LastActivityDate="2018-04-13T00:42:26.610" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4056" PostTypeId="2" ParentId="4043" CreationDate="2018-04-13T07:47:43.137" Score="0" Body="&lt;p&gt;With things like MAS5, presence/absence calls were done using the P/M/A probe intensities and corresponded more to a &quot;signal is reliable&quot; p-value than literal presence/absence. This data is absent from the SOFT files, so you can't use the standard algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only method left is to use &quot;above X percentile of negative probes&quot; as a cut-off for presence, which has been mentioned in a number of papersAFFX-BioB-3_at (e.g., &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3122888/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). While there are a handful of control probes on that chip, I'm not aware of any of them being negative probes (or if there are any, there don't seem to be enough to be useful for this purpose). Thus I don't think you can even use this method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course you can still do standard differential expression, even with GEO2R in the browser.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-13T07:47:43.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4057" PostTypeId="2" ParentId="4050" CreationDate="2018-04-13T12:55:29.990" Score="1" Body="&lt;p&gt;Your score and strand columns are swapped. If you correct that then you'll get stranded results.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-13T12:55:29.990" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4058" PostTypeId="2" ParentId="4050" CreationDate="2018-04-13T13:02:51.327" Score="2" Body="&lt;p&gt;Your bed file switch the score and strand columns. Strand should be column 6, score column 5. For example take this bed file with the 5th and 6th columns switched - zero results for strand specific intersection b/c strand is actually '.'&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat wrong.bed&#xA;chr1    30000000        31000000        noname  +       .&#xA;$ bedtools intersect -a mm10_500k_pe.bam -b wrong.bed -wa -u -f 1.0 \&#xA;    | samtools view | wc -l&#xA;166&#xA;$ bedtools intersect -a mm10_500k_pe.bam -b wrong.bed -wa -u -f 1.0 -s \&#xA;   | samtools view | wc -l                              &#xA;0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After fixing the bed file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat right.bed&#xA;chr1    30000000        31000000        noname  .       +&#xA;$ bedtools intersect -a mm10_500k_pe.bam -b right.bed -wa -u -f 1.0 \&#xA;      | samtools view | wc -l&#xA;166&#xA;$ bedtools intersect -a mm10_500k_pe.bam -b right.bed -wa -u -f 1.0 -s \&#xA;      | samtools view | wc -l&#xA;83&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2656" LastEditorUserId="2656" LastEditDate="2018-04-13T13:09:11.667" LastActivityDate="2018-04-13T13:09:11.667" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4061" PostTypeId="1" AcceptedAnswerId="4127" CreationDate="2018-04-13T16:50:39.407" Score="1" ViewCount="2590" Body="&lt;p&gt;I am making some infographics of different library prep types and noticed something weird about NEB's Ultra II adapters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Molecule 1&lt;/code&gt; is the desired product of the adapter ligation step. &lt;code&gt;*&lt;/code&gt; indicates a phosphorothioate bond and &lt;code&gt;-&lt;/code&gt; indicates a normal bond. &lt;code&gt;|&lt;/code&gt; indicates complementary sequence through hydrogen bonding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Molecule 2&lt;/code&gt; is the product after treatment with the USER enzyme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Molecule 3&lt;/code&gt; is the result of melting molecule &lt;code&gt;2&lt;/code&gt; and hybridizing the NEBNext Universal PCR Primer for Illumina.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: Why does &lt;code&gt;Molecule 3&lt;/code&gt; have a large non-complementary region on the 3' end of the original molecule? Is that just to force &lt;code&gt;Molecule 1&lt;/code&gt; to have an open loop conformation so that the USER enzyme can access it better? Or maybe it is something to do with PCR kinetics?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that PCR synthesizes in the &lt;code&gt;5'-&amp;gt;3'&lt;/code&gt; direction, so &lt;code&gt;Molecule 3&lt;/code&gt; only forms with the original adapter-ligated molecules before PCR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eSLst.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eSLst.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-04-24T03:33:00.627" Title="Why do NEB adapters have non-complementary sequence?" Tags="|phylogenetics|adapter|pcr|library|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4062" PostTypeId="2" ParentId="4061" CreationDate="2018-04-13T19:22:51.370" Score="1" Body="&lt;p&gt;The original point of the non-complementary sequence is to allow formation of a Y-adapter after USER treatment, since that's what's needed for the Illumina chemistry to work. In reality the bottom strand may not be there in step 3, since USER has already digested one strand (creating the hairpin in step 2) and size selection has been performed. Regardless, it'll get removed once cluster formation is performed.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-13T19:22:51.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4063" PostTypeId="2" ParentId="4052" CreationDate="2018-04-13T21:07:54.417" Score="4" Body="&lt;p&gt;It's likely that the fold-change of the &lt;code&gt;topGene&lt;/code&gt; is greater than 5. Consequently, the circle is being drawn and the text produced, but they're outside the bounds of the plot (the bounds don't update if you add a circle and text label). Look at the &lt;code&gt;log2FoldChange&lt;/code&gt; column of &lt;code&gt;resLFC1[topGene, ]&lt;/code&gt; and increase the bounds to encompass that.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-13T21:07:54.417" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4064" PostTypeId="1" CreationDate="2018-04-13T21:21:01.820" Score="1" ViewCount="124" Body="&lt;p&gt;I'm running the script &lt;code&gt;run_spp_nodups.R&lt;/code&gt; from &lt;a href=&quot;https://github.com/crazyhottommy/phantompeakqualtools&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;phantompeakqualtools&lt;/code&gt;&lt;/a&gt;. The input is a pair of BED 3+3 (&quot;tagalign&quot;) files that are roughly 300M, and I'm using 8 threads. How long would you expect it to take? I am happy to provide other info to the extent that I can manage -- post me a comment!&lt;/p&gt;&#xA;" OwnerUserId="1708" LastActivityDate="2018-04-13T21:21:01.820" Title="How long does it typically take to call peaks via `phantompeakqualtools`?" Tags="|chip-seq|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4065" PostTypeId="1" CreationDate="2018-04-13T23:04:16.940" Score="1" ViewCount="4297" Body="&lt;p&gt;I have a &lt;code&gt;data.frame&lt;/code&gt; of &lt;code&gt;27 000 x 17&lt;/code&gt; and I want to feed it into &lt;code&gt;UpSetR&lt;/code&gt; program, but getting the error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error in start_col:end_col : argument of length 0&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My &lt;code&gt;data.frame&lt;/code&gt; looks like that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dzQ7h.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dzQ7h.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the &lt;code&gt;upSetR&lt;/code&gt; code that I am using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;upset(df, nsets = 4, number.angles = 30, point.size = 3.5, line.size = 2, &#xA;mainbar.y.label = &quot;Gene Intersections&quot;, sets.x.label = &quot;Genes Per Group&quot;, &#xA;text.scale = c(1.3, 1.3, 1, 1, 2, 0.75))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What am I doing wrong here? Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Update&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I found here a similar issue: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://plus.google.com/109635504520741246387/posts/Etujrp6sU1Q&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://plus.google.com/109635504520741246387/posts/Etujrp6sU1Q&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They say that the input should be binary. But then, how can I calculate this binary matrix from the &lt;code&gt;rlog&lt;/code&gt;-data? I am not sure...&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="2005" LastEditDate="2018-04-16T02:06:41.743" LastActivityDate="2019-10-17T15:38:46.100" Title="How can I use UpSet plot for expression/RNA-seq data" Tags="|r|upsetr|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4066" PostTypeId="1" AcceptedAnswerId="4067" CreationDate="2018-04-15T06:40:29.983" Score="2" ViewCount="2558" Body="&lt;p&gt;I have FPKM values for Normal(80 cases), Group1(135 cases), Group2(147 cases), and Group3(102 cases) for 160 genes (Groups were obtained by clustering tumor data). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to perform t-test (two-sided, unpaired, unequal variance) between Normal and Group1, Normal and Group2, and Normal and Group3 such that I get p.value for each gene.&lt;br&gt;&#xA;The t.test function in R returns single p.value and not gene-wise (I don't want to do t-test in Excel). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which package/function should I use to do this in R?&lt;/p&gt;&#xA;" OwnerUserId="2247" LastEditorUserId="48" LastEditDate="2018-04-16T13:32:04.370" LastActivityDate="2018-04-16T13:32:04.370" Title="Performing t.test to obtain p.values for each gene" Tags="|r|differential-expression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4067" PostTypeId="2" ParentId="4066" CreationDate="2018-04-15T07:39:20.590" Score="4" Body="&lt;p&gt;You can simply &lt;code&gt;apply()&lt;/code&gt; &lt;code&gt;t.test()&lt;/code&gt; to your matrix. In general, though, I expect &lt;code&gt;lmFit()&lt;/code&gt; from the &lt;code&gt;limma&lt;/code&gt; package will be simpler to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example with &lt;code&gt;t.test()&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;m = matrix(runif(100, max=c(rep(1, 50), rep(1.5, 50))), nrow=10) # make some data&#xA;apply(m, 1, function(x) t.test(x[1:5], x[6:10])$p.value)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Neither &lt;code&gt;t.test()&lt;/code&gt; nor &lt;code&gt;lmFit()&lt;/code&gt; should be expected to produce the same results as excel, since excel is inferior in every way and should never be relied upon for any statistical analysis (plus &lt;code&gt;lmFit&lt;/code&gt; is doing something much much more complicated than a simple t-test).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an aside, FPKMs are highly problematic for use in statistics. If possible, I strongly encourage you to get either per-gene counts or per-gene estimated counts and to use those in &lt;code&gt;limma&lt;/code&gt; or another differential expression package instead, as the results will be much more reliable. I presume, though, that you're using TCGA data, so you may be stuck with FPKMs.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-04-15T18:23:32.837" LastActivityDate="2018-04-15T18:23:32.837" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="4068" PostTypeId="1" AcceptedAnswerId="4069" CreationDate="2018-04-16T07:57:45.697" Score="2" ViewCount="124" Body="&lt;p&gt;I'm exploring gene expression on public available databases. &lt;a href=&quot;http://sbmdb.genome.rcast.u-tokyo.ac.jp/refexa/main_search.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefExa&lt;/a&gt; is one of the databases I've been studying.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I searched something there, the website reported the expression value unit as &quot;signal&quot;. See screenshot where I highlighted:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/znUEJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/znUEJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not able to find anywhere what that really mean. TPM? RPKM? Raw counts? Normalizated for gene length?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is that how to interpret gene expression &quot;signal&quot;? What does signal 100 mean?&lt;/p&gt;&#xA;" OwnerUserId="174" LastActivityDate="2018-04-16T09:01:19.277" Title="What's &quot;signal&quot; in gene expression?" Tags="|gene|database|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4069" PostTypeId="2" ParentId="4068" CreationDate="2018-04-16T09:01:19.277" Score="4" Body="&lt;p&gt;Looking at this resource, these data are microarray data rather than RNAseq data. Thus the &quot;signal&quot; in question is the intensity of florescent light being emitted from the the probe associated with your gene.  The relationship between intensity and quantity of RNA is different for every probe because each probe will have a different affinity for complementary fragments dependent on the sequence in question (not least the melting temperature). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Affy intensities are usually presented as log values going up to around 12, but the raw number is next to meaningless and usually only means something as a normalized (e.g. RMA) log fold change. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've never come across RefExA before, and it appears to be undocumented and unmaintained. I wouldn't rely on it for usable data if I were you. A more modern database of expression data is the &lt;a href=&quot;https://www.ebi.ac.uk/gxa/home/&quot; rel=&quot;noreferrer&quot;&gt;EBI Expression Atlas&lt;/a&gt;. It presents data from a wide range of different studies, processed and normalised on a unified pipeline and expressed as TPMs. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-04-16T09:01:19.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4070" PostTypeId="2" ParentId="4010" CreationDate="2018-04-16T10:28:14.327" Score="2" Body="&lt;p&gt;Without control data from your subjects, I don't think there's really no way to distinguish somatic mutations from germ-line mutations. The best you can do is to screen out common variants, which are germ-line mutations that are shared by large numbers of individuals using the population frequencies from something like the  &lt;a href=&quot;http://annovar.openbioinformatics.org/en/latest/user-guide/filter/#exac-annotations&quot; rel=&quot;nofollow noreferrer&quot;&gt;Exome Aggregation Consortium&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;annotate_variation.pl -downdb -webfrom annovar -build hg19 exac03 humandb/&#xA;annotate_variation.pl -filter -build hg19 -dbtype exac03 example/ex1.avinput humandb/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="581" LastEditorUserId="581" LastEditDate="2018-04-17T09:44:25.207" LastActivityDate="2018-04-17T09:44:25.207" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="4071" PostTypeId="1" AcceptedAnswerId="4072" CreationDate="2018-04-16T20:13:10.163" Score="3" ViewCount="47" Body="&lt;p&gt;I am supplying 30 ensembl ids into BioMart, but somehow it returns 31 result. What is going wrong here?&#xA;Here you can see the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library('biomaRt')&#xA;mart &amp;lt;- useDataset(&quot;mmusculus_gene_ensembl&quot;, useMart(&quot;ensembl&quot;))&#xA;genes &amp;lt;- rownames(mat)&#xA;G_list &amp;lt;- getBM(filters= &quot;ensembl_gene_id&quot;, attributes= &#xA;c(&quot;ensembl_gene_id&quot;,&quot;entrezgene&quot;, &#xA;&quot;mgi_symbol&quot;,&quot;description&quot;),values=genes,mart= mart)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;genes list is the following one: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[1] &quot;ENSMUSG00000097971&quot; &quot;ENSMUSG00000097582&quot; &quot;ENSMUSG00000061808&quot; &quot;ENSMUSG00000097206&quot;&#xA;[5] &quot;ENSMUSG00000022425&quot; &quot;ENSMUSG00000094708&quot; &quot;ENSMUSG00000095937&quot; &quot;ENSMUSG00000032484&quot;&#xA;[9] &quot;ENSMUSG00000060550&quot; &quot;ENSMUSG00000073409&quot; &quot;ENSMUSG00000026822&quot; &quot;ENSMUSG00000056071&quot;&#xA;[13] &quot;ENSMUSG00000035299&quot; &quot;ENSMUSG00000045932&quot; &quot;ENSMUSG00000006154&quot; &quot;ENSMUSG00000032496&quot;&#xA;[17] &quot;ENSMUSG00000040809&quot; &quot;ENSMUSG00000032690&quot; &quot;ENSMUSG00000055322&quot; &quot;ENSMUSG00000066443&quot;&#xA;[21] &quot;ENSMUSG00000005089&quot; &quot;ENSMUSG00000073489&quot; &quot;ENSMUSG00000005800&quot; &quot;ENSMUSG00000034855&quot;&#xA;[25] &quot;ENSMUSG00000029417&quot; &quot;ENSMUSG00000029561&quot; &quot;ENSMUSG00000021843&quot; &quot;ENSMUSG00000020108&quot;&#xA;[29] &quot;ENSMUSG00000037849&quot; &quot;ENSMUSG00000066861&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see I have here 30 genes, but BioMart returns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; G_list$mgi_symbol&#xA;[1] &quot;Slc1a2&quot;  &quot;Mmp8&quot;    &quot;Eps8l1&quot;  &quot;Ddit4&quot;   &quot;Ktn1&quot;    &quot;Enpp2&quot;   &quot;Lcn2&quot;    &quot;Cxcl9&quot;  &#xA;[9] &quot;Oasl2&quot;   &quot;Ngp&quot;     &quot;Ltf&quot;     &quot;Oas2&quot;    &quot;Cxcl10&quot;  &quot;Mid1&quot;    &quot;Ifi206&quot;  &quot;Chil3&quot;  &#xA;[17] &quot;Ifit2&quot;   &quot;Tns1&quot;    &quot;S100a9&quot;  &quot;H2-Q7&quot;   &quot;H2-Q7&quot;   &quot;Ttr&quot;     &quot;Gm10163&quot; &quot;Oas1g&quot;  &#xA;[25] &quot;H2-Q6&quot;   &quot;Ifi204&quot;  &quot;Gm10359&quot; &quot;Gm12671&quot; &quot;Gm26699&quot; &quot;Gm26527&quot; &quot;Gm26917&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2018-04-17T07:16:50.203" LastActivityDate="2018-04-17T07:16:50.203" Title="BioMart returns one more item" Tags="|rna-seq|biomart|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4072" PostTypeId="2" ParentId="4071" CreationDate="2018-04-16T20:28:52.957" Score="6" Body="&lt;p&gt;If you print &lt;code&gt;G_list&lt;/code&gt;, you'll see this half way down:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1 ENSMUSG00000060550      15018      H2-Q7      Mus musculus histocompatibility 2, Q region locus 9 (H2-Q9), mRNA. [Source:RefSeq mRNA;Acc:NM_001201460]&#xA;2 ENSMUSG00000060550     110558      H2-Q7      Mus musculus histocompatibility 2, Q region locus 9 (H2-Q9), mRNA. [Source:RefSeq mRNA;Acc:NM_001201460]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So there are two &lt;code&gt;entrez&lt;/code&gt; ids that map to the same &lt;code&gt;Ensembl&lt;/code&gt; id. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene/?term=110558&quot; rel=&quot;noreferrer&quot;&gt;One&lt;/a&gt; of them maps to an alt locus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you exclude &lt;code&gt;&quot;entrezgene&quot;&lt;/code&gt; from the query, you'll get 30 results.&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-04-16T20:28:52.957" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4073" PostTypeId="1" CreationDate="2018-04-16T20:48:35.677" Score="2" ViewCount="575" Body="&lt;p&gt;Using starBeast I ran a 50 locus tree, with 34 individuals, using evolutionary models indicated by AICc in jModel test. I ran this tree with a MCMC chain length of 500 million, storing every 10,000 generations. I used a relaxed clock log normal. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get low ESS values for posterior, prior, birthRate.t:Species, YuleModel.t:Species, popMean, and TreeHeight.Species. &#xA;ucldMean.c:Locus_name is low for each loci. &#xA;The TreeHeight statisitcs also have low ESS values.&#xA;Also rate.c:Locus_name.mean for each loci is low as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to change my parameters in Beauti to help increase the ESS values of these statistics? Is there a parameter that would likely cause the ESS values of these particular statistics to be low? Would increasing the MCMC chain length increase the ESS values, given that I've already run half a billion? &lt;/p&gt;&#xA;" OwnerUserId="2338" LastEditorUserId="2338" LastEditDate="2018-04-16T22:30:20.967" LastActivityDate="2018-04-25T12:40:18.560" Title="How can I improve ESS values for ucldMean in beast?" Tags="|genomics|phylogenetics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4074" PostTypeId="1" AcceptedAnswerId="4075" CreationDate="2018-04-17T09:50:01.360" Score="4" ViewCount="5054" Body="&lt;p&gt;I was trying hisat2 I get confused by the strand options...&#xA;The question has been already asked on github &lt;a href=&quot;https://github.com/infphilo/hisat2/issues/109&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; but didn't get any satisfactory answer.&#xA;I'm surprise how much this strand information could be confusing in different tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at the &lt;a href=&quot;https://ccb.jhu.edu/software/hisat2/manual.shtml#indexing-a-reference-genome&quot; rel=&quot;nofollow noreferrer&quot;&gt;hisat2 manual&lt;/a&gt; I get confuse by the two different options:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;--rna-strandness&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Specify strand-specific information: the default is unstranded.&#xA;  For single-end reads, use F or R. 'F' means a read corresponds to a transcript. 'R' means a read corresponds to the reverse complemented counterpart of a transcript. For paired-end reads, use either FR or RF.&#xA;  With this option being used, every read alignment will have an XS attribute tag: '+' means a read belongs to a transcript on '+' strand of genome. '-' means a read belongs to a transcript on '-' strand of genome.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and &lt;code&gt;--fr&lt;/code&gt;/&lt;code&gt;--rf&lt;/code&gt;/&lt;code&gt;--ff&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The upstream/downstream mate orientations for a valid paired-end alignment against the forward reference strand. E.g., if --fr is specified and there is a candidate paired-end alignment where mate 1 appears upstream of the reverse complement of mate 2 and the fragment length constraints (-I and -X) are met, that alignment is valid. Also, if mate 2 appears upstream of the reverse complement of mate 1 and all other constraints are met, that too is valid. --rf likewise requires that an upstream mate1 be reverse-complemented and a downstream mate2 be forward-oriented. --ff requires both an upstream mate 1 and a downstream mate 2 to be forward-oriented. Default: --fr (appropriate for Illumina's Paired-end Sequencing Assay).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Should both need to be specified ? Are they redundant ?&#xA;I didn't find so many person struggling with that, so I guess there is something obvious that I don't get. I hope someone may enlighten me about it !&lt;/p&gt;&#xA;" OwnerUserId="2486" LastEditorUserId="294" LastEditDate="2018-04-17T12:11:36.133" LastActivityDate="2018-04-17T12:11:36.133" Title="Hisat2 : which option should mention for strand specific library read" Tags="|read-mapping|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4075" PostTypeId="2" ParentId="4074" CreationDate="2018-04-17T09:57:29.003" Score="4" Body="&lt;p&gt;&lt;code&gt;--fr&lt;/code&gt;/&lt;code&gt;--rf&lt;/code&gt;/&lt;code&gt;--ff&lt;/code&gt; should rarely be set, since they refer to the relative orientation of reads and basically everything these days is &lt;code&gt;--fr&lt;/code&gt;. Mate-pairs were &lt;code&gt;--rf&lt;/code&gt;, but you don't see those much any more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;--rna-strandedness&lt;/code&gt; is a very different option, since it sets how reads are expected to align against genes. Most stranded protocols in use these days follow the dUTP-method, where read #2 in a pair has the same orientation as the transcript from which it arose. So either &lt;code&gt;R&lt;/code&gt; or &lt;code&gt;RF&lt;/code&gt; would typically be appropriate, unless the library is unstranded. In practice, I expect this is more useful if you plan to run stringTie downstream, since then the &lt;code&gt;XS&lt;/code&gt; auxiliary tag is set appropriately.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-17T09:57:29.003" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4076" PostTypeId="1" CreationDate="2018-04-17T17:47:35.077" Score="3" ViewCount="315" Body="&lt;p&gt;I'm currently performing differential expression analysis using alignment-free quantification using Kallisto. To do this, I need to create a Kallisto index using Ensembl's cDNA and ncRNA annotations available at the following two links:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;ftp://ftp.ensembl.org/pub/current_fasta/mus_musculus/cdna/Mus_musculus.GRCm38.cdna.all.fa.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ensembl.org/pub/current_fasta/mus_musculus/cdna/Mus_musculus.GRCm38.cdna.all.fa.gz&lt;/a&gt; &lt;a href=&quot;ftp://ftp.ensembl.org/pub/current_fasta/mus_musculus/ncrna/Mus_musculus.GRCm38.ncrna.fa.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ensembl.org/pub/current_fasta/mus_musculus/ncrna/Mus_musculus.GRCm38.ncrna.fa.gz&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, later on in my analysis I realized that I had multiple Ensembl gene IDs pointing to the same gene. For example, I was gettign results for both ENSMUSG00000026012 and ENSMUSG00000102412 which both map to Cd28. I realized that the reason for this error was due to the fact that the annotations I was using contained haplotypes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to remove these haplotypes, and to do this I believe that I need to rerun my analysis using a primary assembly. However, Ensembl doesn't have annotated cDNA and ncRNA FASTA files of the primary assembly. How can I filter these files to contain only genes present in the primary assembly? Is there a better way for me to solve this issue?&lt;/p&gt;&#xA;" OwnerUserId="2679" LastEditorUserId="73" LastEditDate="2018-04-17T23:50:04.630" LastActivityDate="2018-05-18T00:43:11.263" Title="How to filter Ensembl cDNA and ncRNA FASTA files by primary assembly?" Tags="|ensembl|filtering|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4077" PostTypeId="1" CreationDate="2018-04-17T17:49:29.030" Score="2" ViewCount="634" Body="&lt;p&gt;Currently my lab manually uses pop art to make the network after using PHASE to id an individual's alleles. I would like to know how to generate the graph structure (i.e. the links between vertices), as well as how to draw the graph.&lt;/p&gt;&#xA;&lt;p&gt;Input data are formatted as FASTA files with the sequence in nucleotide formatting.&lt;/p&gt;&#xA;&lt;p&gt;Example:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/nnRTj.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nnRTj.jpg&quot; alt=&quot;network of alleles&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/CR8Dh.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CR8Dh.jpg&quot; alt=&quot;Network  in black and white of groups of alleles&quot; /&gt;&lt;/a&gt;&#xA;I just feel like there's got to be a better, more time effective way to do this.&lt;/p&gt;&#xA;" OwnerUserId="2680" LastEditorUserId="73" LastEditDate="2022-03-19T11:32:14.543" LastActivityDate="2022-03-19T11:32:14.543" Title="Program to make a haplotype network for a specific gene" Tags="|networks|phylogeny|phylogenetics|haplotypes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4079" PostTypeId="2" ParentId="4076" CreationDate="2018-04-17T22:25:56.617" Score="1" Body="&lt;p&gt;The simplest method is to download the GTF file for GRCm38 and filter that. You can then use one of the many tools out there (&lt;a href=&quot;http://bedtools.readthedocs.io/en/latest/content/tools/getfasta.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools getfasta&lt;/a&gt; (make sure to enforce strandedness and possibly use the &lt;code&gt;--split&lt;/code&gt; option, if you happen to use a BED12 file), &lt;a href=&quot;https://ccb.jhu.edu/software/tophat/index.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;gtf_to_fasta&lt;/a&gt; from tophat2 (it's the only reason to keep tophat2 installed), etc.) to make a transcriptomic fasta file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, &lt;a href=&quot;https://www.ensembl.org/biomart/martview&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl BioMart&lt;/a&gt; can be used to directly download a transcriptomic fasta file using a filter to select only those transcripts on pre-defined chromosomes. Other filters can be applied as well. If you don't already have a GTF file downloaded and one of the previously mentioned packages installed, then this may be the simplest way to get what you want.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-17T22:25:56.617" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4080" PostTypeId="1" AcceptedAnswerId="4088" CreationDate="2018-04-18T08:45:47.317" Score="6" ViewCount="8429" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/309962/&quot; rel=&quot;nofollow noreferrer&quot;&gt;biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I reproduce this volcano plot?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KD9Eh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KD9Eh.png&quot; alt=&quot;Volcano plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm only able to do the traditional one, I'm kind knew too these field. &lt;/p&gt;&#xA;" OwnerUserId="2686" LastEditorUserId="57" LastEditDate="2018-05-10T09:45:12.290" LastActivityDate="2018-12-07T15:13:09.043" Title="Volcano plot in R" Tags="|rna-seq|r|visualization|" AnswerCount="3" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4081" PostTypeId="1" CreationDate="2018-04-18T10:44:30.667" Score="0" ViewCount="62" Body="&lt;p&gt;3 Control replicate and 2 test sample ,so Im comparing my Control which is my Progenitor cells to Monocyte which is mature stage&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this is my code im using&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;condition &amp;lt;- factor(c(rep(&quot;Control&quot;, 3),rep(&quot;Test&quot;, 2)),levels=c(&quot;Control&quot;, &quot;Test&quot;))&#xA;&#xA;&#xA;(coldata &amp;lt;- data.frame(row.names=colnames(countdata), condition))&#xA;dds &amp;lt;- DESeqDataSetFromMatrix(countData=countdata, colData=coldata, design=~condition)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is my deseq2 output &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gene baseMean log2FoldChange lfcSE  stat pvalue padj CB1 CB2 CB3 GD5_1  GD5_2&#xA;ENSG00000144655 2030.6146124999 -5.0812043868   0.1183868766    -42.9203348727  0   0   3327.4890036209 3267.1877800456 3362.7407570783 98.5079578403   97.1475639145&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Im not sure how come the pvalue and  padj is 0? even though the sample value there is a difference and there is also fold change ,something wrong with the result ? &lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2018-04-18T10:59:58.753" LastActivityDate="2018-04-18T11:04:21.640" Title="Deseq2 output issue" Tags="|rna-seq|r|deseq2|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4082" PostTypeId="2" ParentId="4081" CreationDate="2018-04-18T11:04:21.640" Score="2" Body="&lt;p&gt;A p-value of 0 is expected in this case. The p-values are calculated by comparing the Wald stat to a standard normal, so you can confirm that this is correct like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; pnorm(-42.9203348727)&#xA;[1] 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It really means p &amp;lt; 2.225074e-308 See &lt;a href=&quot;https://support.bioconductor.org/p/64787&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for a more details&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-04-18T11:04:21.640" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4083" PostTypeId="1" CreationDate="2018-04-18T11:34:09.150" Score="1" ViewCount="1400" Body="&lt;p&gt;I'm performing RNA-seq analysis. I have used Hisat2 for aligning reads to the genome and stringtie for quantification and extracted read count information directly from the files generated by StringTie using &quot;prepDE.py&quot; mentioned in &lt;a href=&quot;http://ccb.jhu.edu/software/stringtie/index.shtml?t=manual&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stringtie manual&lt;/a&gt;. With the counts data I have used edgeR for Differential analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before that I have seen MDS plot showing in the following way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/peGiD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/peGiD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see that three samples are not clustered and thought that they are not sequenced well. So I removed the three samples and again made an MDS plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2Dujc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2Dujc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone tell me what could be the reason for this type of clustering? Is that because of sequencing? or using count data directly from stringtie or anything else?&lt;/p&gt;&#xA;" OwnerUserId="1377" LastActivityDate="2018-04-18T22:46:06.663" Title="What could be the reason for the samples not clustering?" Tags="|rna-seq|r|rna|clustering|edger|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4084" PostTypeId="2" ParentId="4083" CreationDate="2018-04-18T11:54:43.307" Score="2" Body="&lt;p&gt;Samples will only cluster by experimental group if the experimental effect is large enough that it's the primary source of variance between your samples. If that's not the case then you'll get results like you're seeing. Whether that's a problem or not ends up depending on how large of an effect you expect to see. If your groups are different stages in differentiation or another large effect, then you expect that samples will cluster into groups. If you're looking at something like some large unaffected tissue in a disease vs. normal state then seeing a cloud of points is quit reasonable.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-18T11:54:43.307" CommentCount="10" ContentLicense="CC BY-SA 3.0" />
  <row Id="4085" PostTypeId="1" CreationDate="2018-04-18T14:58:58.183" Score="5" ViewCount="2226" Body="&lt;p&gt;I have a dataset of single cell count data and I want to regress out the variation caused by the number of UMI's and the percentage of mitochondrial genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that count data is discrete data, and commonly follows a negative binomial distribution. To regress out these two confounders, should I use a linear model or GLM (poisson/negative binomial)? And how can I determine the optimal choice?&lt;/p&gt;&#xA;" OwnerUserId="973" LastEditorUserId="1771" LastEditDate="2018-04-18T15:45:30.687" LastActivityDate="2018-04-24T16:16:37.890" Title="Regressing out unwanted sources of variation in single cell RNA-seq data" Tags="|rna-seq|scrnaseq|single-cell|linear-regression|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4087" PostTypeId="1" AcceptedAnswerId="4094" CreationDate="2018-04-18T19:46:49.773" Score="2" ViewCount="528" Body="&lt;p&gt;I am supplying to &lt;code&gt;BioMart&lt;/code&gt; &lt;code&gt;81&lt;/code&gt; &lt;code&gt;ensemblID&lt;/code&gt;s, but getting &lt;code&gt;80&lt;/code&gt; results where &lt;code&gt;1&lt;/code&gt; &lt;code&gt;ensembleID&lt;/code&gt;: &lt;code&gt;ENSMUSG00000083840&lt;/code&gt; is absent. What does it mean if &lt;code&gt;BioMart&lt;/code&gt; can not find the gene? Does it mean that it is a non-coding one or that there is just no symbol attached to it? Here is my &lt;code&gt;BioMart&lt;/code&gt; query:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library('biomaRt')&#xA;mart &amp;lt;- useDataset(&quot;mmusculus_gene_ensembl&quot;, useMart(&quot;ensembl&quot;))&#xA;genes &amp;lt;- rownames(mat)&#xA;G_list &amp;lt;- getBM(filters= &quot;ensembl_gene_id&quot;, attributes= &#xA;c(&quot;ensembl_gene_id&quot;,&quot;mgi_symbol&quot;,&quot;description&quot;),values=genes,mart= mart)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-04-20T07:35:29.153" Title="BioMart does not return a result" Tags="|r|biomart|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4088" PostTypeId="2" ParentId="4080" CreationDate="2018-04-18T19:48:47.460" Score="13" Body="&lt;p&gt;This plot is clearly done using core R functions. There are smoother alternatives how to make a pretty volcano plot (like &lt;code&gt;ggplot&lt;/code&gt; with example &lt;a href=&quot;https://www.biostars.org/p/309962/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;), but if you really wish to, here is my attempt to reproduce it :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Key5z.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Key5z.png&quot; alt=&quot;volcano plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I obviously had to generate data since I do not have the expression data from the figure, but the procedure will be about the same with the real data. Here is the R script :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;### define interesting genes&#xA;interesting_genes &amp;lt;- c('Il17s', 'Casz1', 'Rorc', 'Il22')&#xA;fc_interesting_genes &amp;lt;- c(-1.8, -1.3, -0.9, -0.85)&#xA;pval_interesting_genes &amp;lt;- log10(c(0.0005, 0.01, 1e-4, 4e-05))&#xA;&#xA;### define colours (grey, blue, red)&#xA;palette &amp;lt;- c(rgb(0, 0, 0, max = 255, alpha = 65),&#xA;    rgb(0, 0, 255, max = 255, alpha = 125),&#xA;    rgb(255, 0, 0, max = 255, alpha = 125))&#xA;&#xA;### generate some expression like data, fold change using normal distr, and tranformed uniform for pvalues&#xA;fold_changes &amp;lt;- c(rnorm(1000, 0, 2), fc_interesting_genes)&#xA;pvalues &amp;lt;- c(log10(runif(1000) / 3^(1 + abs(fold_changes[1:1000]))), pval_interesting_genes)&#xA;&#xA;# find deferentially expressed genes to colour&#xA;left_biased &amp;lt;- fold_changes &amp;lt; -1&#xA;right_biased &amp;lt;- fold_changes &amp;gt; 1&#xA;&#xA;point_type = 20&#xA;point_size = 1.5&#xA;&#xA;### plot corpus&#xA;plot(fold_changes, pvalues,&#xA;     pch = point_type, cex = point_size, col = palette[1],&#xA;     xlab = &quot;Fold Change (log2)&quot;, ylab = &quot;P value&quot;,&#xA;     xlim = c(-6,6), ylim = c(-5, 0), yaxt=&quot;n&quot;)&#xA;aty &amp;lt;- axTicks(2)&#xA;labels &amp;lt;- sapply(aty,function(i)&#xA;            as.expression(bquote(10^ .(i)))&#xA;          )&#xA;labels[6] &amp;lt;- 1&#xA;axis(2,at=aty,labels=labels)&#xA;&#xA;### plot deferentially expressed genes in colour + genes on interest with black&#xA;points(fold_changes[left_biased], pvalues[left_biased],&#xA;       pch = point_type, cex = point_size, col = palette[2])&#xA;points(fold_changes[right_biased], pvalues[right_biased],&#xA;       pch = point_type, cex = point_size, col = palette[3])&#xA;points(fc_interesting_genes, pval_interesting_genes, pch = point_type, cex = point_size)&#xA;&#xA;### add text&#xA;labelx &amp;lt;- c(-4, -4, 4, 4)&#xA;labely &amp;lt;- c(-1.5, -0.5, -0.5, -1.5)&#xA;labels &amp;lt;- c('Il17s', 'Casz1', 'Rorc', 'Il22')&#xA;text(labelx, labely, labels)&#xA;text(-4, -4, &quot;down\n194 genes&quot;)&#xA;text(4, -4, &quot;up\n156 genes&quot;)&#xA;&#xA;### add lines between text and data points&#xA;for(i in 1:4){&#xA;    lines(c(labelx[i], fc_interesting_genes[i]), c(labely[i], pval_interesting_genes[i]))&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="57" LastActivityDate="2018-04-18T19:48:47.460" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4089" PostTypeId="2" ParentId="4085" CreationDate="2018-04-18T22:34:11.617" Score="2" Body="&lt;p&gt;Cleaning data before doing analysis can be more important than the method of data analysis. For UMIs, there's an obvious cleaning step that can be done: it would be better to filter out duplicates (prior to generating counts) than to try to incorporate that information in the count analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not familiar with single cell analysis, but I am aware that DESeq2 can be &lt;a href=&quot;https://t.co/KsUpczFn21&quot; rel=&quot;nofollow noreferrer&quot;&gt;adapted&lt;/a&gt; to work with single cells, and incorporating additional variation is just a matter of adding an &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#multi-factor-designs&quot; rel=&quot;nofollow noreferrer&quot;&gt;additional covariate&lt;/a&gt; into the DESeq2 formula model.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-04-18T22:34:11.617" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4090" PostTypeId="2" ParentId="4083" CreationDate="2018-04-18T22:46:06.663" Score="0" Body="&lt;p&gt;The largest variation differences in total expression will be seen in cell population differences. It's a good thing that your conditions aren't clustering on the first and second principal components when all genes are considered, because it means that cell population is probably not a factor in the variation of expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There may be some other principal component that &lt;em&gt;does&lt;/em&gt; separate the conditions, but you might need to drill down fairly deep into the data. This will depend on how much the treatment conditions contribute to the total expression variation. We got lucky with our &lt;a href=&quot;http://jem.rupress.org/content/early/2016/12/01/jem.20160470#F3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Th2 RNASeq study&lt;/a&gt; in that cell population was mostly on the first and second principal components, and treatment condition appeared on the third principal component.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to force condition clustering, identify the genes that are differentially expressed, and only use those genes for the principal component analysis.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-04-18T22:46:06.663" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4091" PostTypeId="1" AcceptedAnswerId="11002" CreationDate="2018-04-18T23:56:03.977" Score="3" ViewCount="178" Body="&lt;p&gt;I have a genome-wide list of germline SNPs and short indels for Arabidopsis thaliana, which I generated with Varscan. Regardless of the tool used to generate them, I would like to annotate them, i.e. knowing which ones can cause an aminoacid change or an early stop, using the default Arabidopsis thaliana Columbia 0 cultivar, for which I have both the sequence (FASTA from TAIR10) and the updated annotation (GFF from Araport).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Chrom   Position    Ref Var&#xA;Chr1    626503  G   T&#xA;Chr1    926694  C   T&#xA;Chr1    5280350 C   A&#xA;Chr1    5699993 C   A&#xA;Chr1    7004559 G   A&#xA;Chr1    8325810 C   T&#xA;Chr1    9371723 T   G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What I want to do is similar to what Annovar does, but unfortunately Annovar does not support Arabidopsis. I was thinking of an already existing R pipeline that takes in a genome, an annotation, a SNP/indel list and &lt;em&gt;boom&lt;/em&gt;, annotation. But I couldn't find any, except maybe &lt;a href=&quot;https://github.com/douglasgscofield/snpEffect&quot; rel=&quot;nofollow noreferrer&quot;&gt;snpEffect&lt;/a&gt;. Any tips?&lt;/p&gt;&#xA;" OwnerUserId="2693" LastEditorUserId="48" LastEditDate="2018-04-24T11:03:05.727" LastActivityDate="2019-12-18T17:03:40.273" Title="Generate SNP/indel annotation in Arabidopsis" Tags="|sequence-annotation|snp|indel|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4092" PostTypeId="1" AcceptedAnswerId="4093" CreationDate="2018-04-19T03:32:18.087" Score="5" ViewCount="1107" Body="&lt;p&gt;How are Principal Component analyses and Admixture analyses from a genetic alignment different? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding is that a PCA will take raw genetic differences across the entire alignment and plot them using dimensionality reduction techniques (single value decomposition for instance) whereas the Admixture analysis will plot likelihoods of sharing certain amounts of ancestry, based on allele frequency data. What I don't understand is how the two are fundamentally different. That is, isn't every variable position found in a PCA going to represent an allele used in Admixture?&lt;/p&gt;&#xA;" OwnerUserId="2338" LastActivityDate="2018-04-19T12:38:47.137" Title="How are Principal Component analyses and Admixture analyses from a genetic alignment different?" Tags="|sequence-alignment|pca|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4093" PostTypeId="2" ParentId="4092" CreationDate="2018-04-19T07:05:14.083" Score="3" Body="&lt;p&gt;The fundamental difference lies mostly in the math.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An admixture analysis involves assuming that genotypes (or more likely, genotype likelihoods) in an unknown sample can be modeled with Hardy-Weinburg equilibrium as arising from the combination from two or more source pools of background variants. The output is then an admixture proportion that dictates the mostly percent contribution from each of the source populations (e.g., 23% Irish, 15% Norwegian, 30% Northern African, ...). There are versions of this specifically for handling pecularities of NGS datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A PCA is a very very generic method that uses the same input allelic information, but simply finds &lt;strong&gt;orthogonal&lt;/strong&gt; projections of that onto lower dimensions that maximize the variance on the first principal component. You've probably seen plots like the following (from &lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fgene.2013.00127/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper about 1000 genomes projects data&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://www.frontiersin.org/files/Articles/52177/fgene-04-00127-r2/image_m/fgene-04-00127-g002.jpg&quot; alt=&quot;A figure from the paper&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That allows you to get an idea about how populations are structured, but they're not so useful for determining how individual samples are most likely related to the various populations.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-19T07:05:14.083" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4094" PostTypeId="2" ParentId="4087" CreationDate="2018-04-19T08:54:34.273" Score="9" Body="&lt;p&gt;Whenever you are wondering about things like this, just look up the identifier on the Ensembl web page. If you &lt;a href=&quot;http://www.ensembl.org/Mus_musculus/Gene/Idhistory?g=ENSMUSG00000083840&quot; rel=&quot;nofollow noreferrer&quot;&gt;look up&lt;/a&gt; &lt;code&gt;ENSMUSG00000083840&lt;/code&gt;, you will see:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This identifier is not in the current EnsEMBL database&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In this case, it doesn't seem to have been replaced by anything and it wasn't a protein coding gene, so it looks like it was simply a wrong annotation which has been removed. &lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-04-20T07:35:29.153" LastActivityDate="2018-04-20T07:35:29.153" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4095" PostTypeId="2" ParentId="4085" CreationDate="2018-04-19T09:36:20.867" Score="0" Body="&lt;p&gt;If you just want to filter UMI related problems and cells with high mitochondrial to nuclear ratio, maybe there is no need to use modeling.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;For UMIs&lt;/strong&gt;: You can &lt;strong&gt;filter UMIs&lt;/strong&gt; that are &lt;strong&gt;not supported by at least x reads&lt;/strong&gt; (for instance 3 reads) and that probably arise from sequencing errors.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;For high mitochondrial to nuclear ratio&lt;/strong&gt;: You can compute (number of molecules of mitochondrial genes / number of molecules of nuclear genes) and plot this ratio for all the cells and try to put a &lt;strong&gt;threshold value for cells that have high values for this ratio&lt;/strong&gt;. Thus, filter these cells that behave strange compared to the rest.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="678" LastActivityDate="2018-04-19T09:36:20.867" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4096" PostTypeId="2" ParentId="4092" CreationDate="2018-04-19T12:38:47.137" Score="4" Body="&lt;p&gt;Just to add to the great summary by Devon Ryan: admixture analysis tools are much more flexible than PCA (which is just a fixed mathematical operation), so they can be designed to incorporate LD patterns, phase information, different population evolution models etc.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a paper from the creators of fineSTRUCTURE and Chromopainter admixture tools detailing the action of both - it seems they provide some comparisons with PCA as well:&#xA;&lt;a href=&quot;http://journals.plos.org/plosgenetics/article?id=10.1371/journal.pgen.1002453&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://journals.plos.org/plosgenetics/article?id=10.1371/journal.pgen.1002453&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="920" LastActivityDate="2018-04-19T12:38:47.137" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4098" PostTypeId="1" AcceptedAnswerId="4100" CreationDate="2018-04-19T14:21:39.307" Score="6" ViewCount="195" Body="&lt;p&gt;Google released their variant caller &lt;a href=&quot;https://github.com/google/deepvariant&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepVariant&lt;/a&gt; which won the highest SNP performance award in the &lt;a href=&quot;https://precision.fda.gov/challenges/truth/results&quot; rel=&quot;nofollow noreferrer&quot;&gt;Precision FDA Truth challenge&lt;/a&gt; (99.999% accuracy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the linked github repo, we see that DeepVariant is a CNN, we provide images of the reads as an input and train the neural network with a stochastic gradient descent algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To me this is unintuitive. Why use images of reads when you already have the reads data?&lt;/p&gt;&#xA;" OwnerUserId="2246" LastEditorUserId="57" LastEditDate="2018-04-21T11:40:01.430" LastActivityDate="2018-04-21T11:40:01.430" Title="Why chose CNN for a variant caller" Tags="|variant-calling|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4099" PostTypeId="1" CreationDate="2018-04-19T14:25:24.777" Score="1" ViewCount="116" Body="&lt;p&gt;I have installed &lt;a href=&quot;http://mugsy.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;mugsy&lt;/a&gt; in order to create a multiple genome alignment and a phylogenetic tree of several species of nematodes. The following command successfully pulls out the help:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;output/software/mugsy_x86-64-v1r2.3/mugsy -h&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To test that the software is installed correctly and that I am using it correctly, I ran the following alignment between the Wormbase sequence and the Ensembl sequence of the &lt;em&gt;C. elegans&lt;/em&gt; genome:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;output/software/mugsy_x86-64-v1r2.3/mugsy  --directory /.../project/output/software/mugsy_x86-64-v1r2.3/ --prefix mygenomes output/genome/celegans/ref/wormbase/seq/c_elegans.PRJNA13758.WS263.genomic.fa output/genome/celegans/ref/ensembl/seq/celegans.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get the following output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Parsing sequences for c_elegans  num_seqs:7&#xA;Parsing sequences for celegans  num_seqs:1&#xA;Can't find species II dna at output/software/mugsy_x86-64-v1r2.3/mugsy line 501.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The headers of the wormbase sequences are: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;I&#xA;&amp;gt;II&#xA;&amp;gt;III&#xA;&amp;gt;IV&#xA;&amp;gt;V&#xA;&amp;gt;X&#xA;&amp;gt;MtDNA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the headers of the ensembl sequences are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;I dna:chromosome chromosome:WBcel235:I:1:15072434:1 REF&#xA;&amp;gt;II dna:chromosome chromosome:WBcel235:II:1:15279421:1 REF&#xA;&amp;gt;III dna:chromosome chromosome:WBcel235:III:1:13783801:1 REF&#xA;&amp;gt;IV dna:chromosome chromosome:WBcel235:IV:1:17493829:1 REF&#xA;&amp;gt;MtDNA dna:chromosome chromosome:WBcel235:MtDNA:1:13794:1 REF&#xA;&amp;gt;V dna:chromosome chromosome:WBcel235:V:1:20924180:1 REF&#xA;&amp;gt;X dna:chromosome chromosome:WBcel235:X:1:17718942:1 REF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Maybe this line:&#xA;    &lt;code&gt;Parsing sequences for celegans  num_seqs:1&lt;/code&gt; should make me worry. Why does mugsy think there is only one sequence in that genome? Why do I get the following error and how can I fix it?:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Can't find species II dna at output/software/mugsy_x86-64-v1r2.3/mugsy line 501.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am working on a GNU/Linux platform.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-04-19T14:25:24.777" Title="Mugsy error: Can't find species II dna at output/software/mugsy_x86-64-v1r2.3/mugsy line 501" Tags="|sequence-alignment|software-usage|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4100" PostTypeId="2" ParentId="4098" CreationDate="2018-04-19T14:37:20.753" Score="4" Body="&lt;p&gt;It's more &quot;images of reads&quot; rather than actual images of reads. In reality, they're feeding in a pileup, which is just a matrix of letters or numbers (you can visualize this as an image, of course). This is more apparent if you look at &lt;a href=&quot;https://github.com/google/deepvariant/blob/r0.6/docs/visualizing_examples.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;one of their example python notebooks&lt;/a&gt;, CNNs are popular in the image processing field, but at the end of the day images are just large matrices of values so any time you can convert your input into that sort of format then a CNN can be useful (provided you have sufficient training data and some GPUs to throw at things).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-19T14:37:20.753" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4101" PostTypeId="1" AcceptedAnswerId="4102" CreationDate="2018-04-19T16:17:41.203" Score="2" ViewCount="123" Body="&lt;p&gt;I am reading Elowitz et al. (2002). When talking about intrinsic noise, i.e. noise due to microscopic events that govern which reactions occur and in what order, it is stated that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Models of stochastic gene expression predict that intrinsic noise should increase as the amount of transcript decreases.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Noise, defined in the article as the standard deviation divided by the mean, can be divided in intrinsic (due to microscopic events during transcription) and extrinsic (due to localization, state and concentration of other cellular components, e.g. polymerases).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the intrinsic noise should increase at low level of transcript amount because the average (mean) amount will be lower but deviation decrease is smaller than mean decrease. I don't know, however, if this is the actual reason (and why deviation decrease should be smaller than mean decrease) or if deviation decreases with the mean and there's another more likely explanation.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Elowitz, M. B., Levine, A. J., Siggia, E. D. &amp;amp; Swain, P. S. &lt;a href=&quot;http://science.sciencemag.org/content/297/5584/1183&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stochastic gene expression in a single cell&lt;/a&gt;. Science 297, 1183–1186 (2002).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2021-04-23T16:17:35.080" Title="Why models of stochastic gene expression predict that intrinsic noise should increase as the amount of transcript decrease" Tags="|noise|gene-expression|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4102" PostTypeId="2" ParentId="4101" CreationDate="2018-04-19T19:22:18.810" Score="2" Body="&lt;p&gt;Reading Kaufmann and van Oudenaarden (2007), it seems to validate the first alternative (using results from the Central Limit Theorem):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Although biochemical fluctuations influence all stages of gene expression, those involving molecules in extremely low abundance are expected from a statistical standpoint to be larger in magnitude and therefore to contribute disproportionately to the overall variation between cells.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Kaufmann, B. B. &amp;amp; van Oudenaarden, A. &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0959437X07000330&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stochastic gene expression: from single molecules to the proteome.&lt;/a&gt; Curr. Opin. Genet. Dev. 17, 107–112 (2007).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2018-04-19T19:33:26.400" LastActivityDate="2018-04-19T19:33:26.400" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4103" PostTypeId="2" ParentId="4098" CreationDate="2018-04-19T22:47:29.980" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why use images of reads when you already have the reads data?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Neural networks take one or multiple &lt;a href=&quot;https://docs.scipy.org/doc/numpy-1.13.0/reference/arrays.ndarray.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;N-dimension arrays&lt;/a&gt; (n-d arrays) as input; they can't take arbitrary forms of data. To feed reads and their alignment into the network, you have to encode them as n-d arrays. If you ignore &quot;colors&quot; for now, a straightforward choice is a (windowLength, readDepth) 2D-array as you see in an alignment browser – it is an image. The &quot;colors&quot; of this image are often called &quot;channels&quot; or &quot;features&quot; in deep learning (DL). They represent base types and base quality in deepVariant. In the end, you get a (windowLength, readDepth, nFeatures) 3D array as input. This is the most natural encoding to DL researchers.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why chose CNN for a variant caller&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That is because CNN has been repeatedly proved to be the best architecture for such (height, width, nChannels) 3D arrays as input. Read alignments are exactly that.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;On a historical note, deepVariant was first built on top of google's now-deprecated &lt;a href=&quot;http://www.techtimes.com/articles/104696/20151109/google-announces-machine-learning-system-tensorflow-now-open-sourced.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;DisBelief framework&lt;/a&gt; that only takes RGB images as input. They had to squeeze all types of information into three color channels. Now with TensorFlow, an image can have an arbitrary number of color channels. This simplifies implementation and improves accuracy. In addition to this image, deepVariant also takes other information as input.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-04-19T22:47:29.980" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4104" PostTypeId="1" AcceptedAnswerId="4108" CreationDate="2018-04-20T04:23:49.947" Score="1" ViewCount="204" Body="&lt;p&gt;I was looking at this script from &lt;a href=&quot;https://biopython.org/DIST/docs/tutorial/Tutorial.html#htoc297&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biopython.org/DIST/docs/tutorial/Tutorial.html#htoc297&lt;/a&gt; and wanted to use it to determine outliers for tentative Horizontal gene transfer.&lt;/p&gt;&#xA;&lt;p&gt;The data set is provided in the example, under the section &lt;strong&gt;20.2.2 Plot of sequence GC%&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I want to yield a identifier for the x axis instead of just an arbitrary number of genes or cds' so I can actually determine which sequences have the outlying values, as I can not with the current graph.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TjoD4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TjoD4.png&quot; alt=&quot;image taken from biopython&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am not great at python or understanding list comprehensions, so if anyone could augment the code provided in the example to do so, it would be appreciated.&lt;/p&gt;&#xA;&lt;p&gt;I provided the code below also.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#from : https://biopython.org/DIST/docs/tutorial/Tutorial.html#htoc297, data set under &#xA;#section 20.2.2 Plot of sequence GC%, ls_orchid.fasta&#xA;&#xA;from Bio import SeqIO&#xA;from Bio.SeqUtils import GC&#xA;&#xA;gc_values = sorted(GC(rec.seq) for rec in SeqIO.parse(&amp;quot;ls_orchid.fasta&amp;quot;, &amp;quot;fasta&amp;quot;))&#xA;import pylab&#xA;pylab.plot(gc_values)&#xA;pylab.title(&amp;quot;%i orchid sequences\nGC%% %0.1f to %0.1f&amp;quot; \&#xA;            % (len(gc_values),min(gc_values),max(gc_values)))&#xA;pylab.xlabel(&amp;quot;Genes&amp;quot;)&#xA;pylab.ylabel(&amp;quot;GC%&amp;quot;)&#xA;pylab.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2701" LastEditorUserId="937" LastEditDate="2020-08-07T15:12:51.840" LastActivityDate="2020-08-07T15:12:51.840" Title="Obtaining identifier from plot of sequence GC%" Tags="|gene|genome|python|identifiers|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4105" PostTypeId="2" ParentId="3164" CreationDate="2018-04-20T07:52:05.410" Score="3" Body="&lt;p&gt;I don't know if this question has been solved already, but what they try to do is equalize the depth of sequencing for each cell. Therefore, they scale for the total number of reads. If you regress out (via linear or negative binomial regression) the differences in the number of reads per cell, you end up with cells that have been sequenced with the same depth. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my opinion they use the ribosomal genes in the same way. They are considered to be a kind of housekeeping genes, which you can use to equalize the sequencing depth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And yes, the log in this context is log2, used to get fold-change values instead of counts.&lt;/p&gt;&#xA;" OwnerUserId="973" LastActivityDate="2018-04-20T07:52:05.410" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4106" PostTypeId="5" CreationDate="2018-04-20T07:56:07.790" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-04-20T07:56:07.790" LastActivityDate="2018-04-20T07:56:07.790" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4107" PostTypeId="4" CreationDate="2018-04-20T07:56:07.790" Score="0" Body="Questions about techniques in which the genetic material of a single cell is analysed individually" OwnerUserId="973" LastEditorUserId="73" LastEditDate="2018-04-24T19:17:07.537" LastActivityDate="2018-04-24T19:17:07.537" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4108" PostTypeId="2" ParentId="4104" CreationDate="2018-04-20T08:05:41.267" Score="2" Body="&lt;p&gt;The tricky bit here is that you need a list of sequence records sorted in the same order as the CG values. You can accomplish this by making tuples of the seq records and GC values, then sorting the tuples specifying the GC values as the sorting key:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gc_w_records = sorted(((GC(rec.seq), rec) for rec in SeqIO.parse(&quot;ls_orchid.fasta&quot;, &quot;fasta&quot;)), key=lambda tup: tup[0])&#xA;gc_values = [x[0] for x in gc_w_records]&#xA;records = [x[1] for x in gc_w_records] &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't know much about &lt;code&gt;matplotlib&lt;/code&gt; and it's going to be challenging to display 100 IDs, but you'll probably be interested in the extremes of the GC distribution, which you can obtain directly from the sorted list of records:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;records[0:2]&#xA;# [SeqRecord(seq=Seq('CGTCACGAGGTCTCCGGATGTGACCCTGCGGAAGGATCATTGTTGAGATCACAT...CAT', SingleLetterAlphabet()), id='gi|2765587|emb|Z78462.1|PSZ78462', name='gi|2765587|emb|Z78462.1|PSZ78462', description='gi|2765587|emb|Z78462.1|PSZ78462 P.sukhakulii 5.8S rRNA gene and ITS1 and ITS2 DNA', dbxrefs=[]),&#xA;# SeqRecord(seq=Seq('CGTAACAAGGTTTCCGTAGGTGAACCTGCGGAAGGATCATTGTTGAGATCACAT...AGG', SingleLetterAlphabet()), id='gi|2765568|emb|Z78443.1|PLZ78443', name='gi|2765568|emb|Z78443.1|PLZ78443', description='gi|2765568|emb|Z78443.1|PLZ78443 P.lawrenceanum 5.8S rRNA gene and ITS1 and ITS2 DNA', dbxrefs=[])]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;records[-3:-1]&#xA;# [SeqRecord(seq=Seq('CGTAACAAGGTTTCCGTAGGTGAACCTGCGGAAGGATCATTGTTGAGACAGCAG...TAA', SingleLetterAlphabet()), id='gi|2765656|emb|Z78531.1|CFZ78531', name='gi|2765656|emb|Z78531.1|CFZ78531', description='gi|2765656|emb|Z78531.1|CFZ78531 C.fasciculatum 5.8S rRNA gene and ITS1 and ITS2 DNA', dbxrefs=[]),&#xA;# SeqRecord(seq=Seq('CGTAACAAGGTTTCCGTAGGTGAACCTGCGGAAGGCTCATTGTTGAGACCGCAA...AAG', SingleLetterAlphabet()), id='gi|2765625|emb|Z78500.1|PWZ78500', name='gi|2765625|emb|Z78500.1|PWZ78500', description='gi|2765625|emb|Z78500.1|PWZ78500 P.warszewiczianum 5.8S rRNA gene and ITS1 and ITS2 DNA', dbxrefs=[])]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-04-20T08:05:41.267" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4111" PostTypeId="1" AcceptedAnswerId="4141" CreationDate="2018-04-20T14:52:26.637" Score="3" ViewCount="587" Body="&lt;p&gt;I am looking for a publicly available matched tumor-normal sample. I need Illumina fastq reads (or an aligned bam file, since I could extract the reads from it) from a tumor and a matching, non-tumor control set from the same patient. Ideally, I would also have a truth set of known variants in the sample, but I can live without that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I checked the &lt;a href=&quot;http://gdc.cancer.gov&quot; rel=&quot;nofollow noreferrer&quot;&gt;Next Generation Cancer Knowledge Network&lt;/a&gt; but although they do seem to have appropriate bam files, which I could convert to fastq, all of the ones I found were &quot;controlled access&quot; and getting access seems very complicated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there something like the public NA12878 sample for tumor-normal analyses? &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-04-25T10:07:55.363" Title="Is there a publicly available tumor-normal sample?" Tags="|ngs|cancer|public-dataset|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="4114" PostTypeId="1" CreationDate="2018-04-21T23:48:23.603" Score="1" ViewCount="672" Body="&lt;p&gt;I am doing RNA-Seq data analysis using data available on SRA (SRR6047326). Samples are from the pig and they are paired. So, I did alignment using HISAT2.I used pig reference genome available on ENSEMBL. Then, I sorted .sam output files obtained from HISAT2. Now I am trying to get the count file using sorted sam file and it is not working. I am using annotated GTF file for pig which I downloaded from Ensembl.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get count files using following commands, but all the counts are 0.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;htseq-counts -i no myfile.sam genes.gtf&amp;gt;output.count&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is wrong here ? Are there any alternatives to htseq-count? &lt;/p&gt;&#xA;" OwnerUserId="1684" LastEditorUserId="48" LastEditDate="2018-04-22T11:02:53.167" LastActivityDate="2018-04-22T16:37:56.963" Title="Count files using htseq-count?" Tags="|rna-seq|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 3.0" />
  <row Id="4115" PostTypeId="2" ParentId="4114" CreationDate="2018-04-22T03:46:08.070" Score="2" Body="&lt;p&gt;I m not sure about htseq-counts but I would suggest go for featureCounts, I will update my answer and share the script/command. Give it a try &lt;/p&gt;&#xA;&#xA;&lt;p&gt;the command i use &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;featureCounts -T 50 -C -p -t exon -g gene_id -a /run/media/data2/gencode.v21.annotation.gtf  -o PRIMARY_CELL.txt $(ls *.bam | sort )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2018-04-22T16:37:56.963" LastActivityDate="2018-04-22T16:37:56.963" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="4116" PostTypeId="1" CreationDate="2018-04-23T07:05:10.657" Score="4" ViewCount="831" Body="&lt;p&gt;There's a commonly accepted convention for naming the wells in a 384-well-plate: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A01..A24&#xA;B01..B24&#xA;..&#xA;P01..P24&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It's concise and unambiguous and thus very useful.&#xA;For 1536-well plates (32x48) it's more difficult because there's more rows than letters in the alphabet. It seems there's no standard.&#xA;Which nomenclatures are out in the field and which is the best?&lt;/p&gt;&#xA;" OwnerUserId="2711" LastEditorUserId="2711" LastEditDate="2018-04-23T08:29:05.807" LastActivityDate="2018-04-23T08:29:05.807" Title="well identifiers in a 1536-well mtp plate" Tags="|hts|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4117" PostTypeId="2" ParentId="4116" CreationDate="2018-04-23T07:50:05.783" Score="2" Body="&lt;p&gt;Here's one way I &lt;a href=&quot;http://mscreen.lsi.umich.edu/mscreenwiki/index.php/COORDINATE&quot; rel=&quot;nofollow noreferrer&quot;&gt;found&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A01..A48&#xA;..&#xA;Z01..Z48&#xA;AA01..AA48&#xA;..&#xA;AF01..AF48&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Pros:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Excel does it like that with columns&lt;/li&gt;&#xA;&lt;li&gt;it's clear what's the row and what's the column&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Cons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;alphabetic sorting of well ids does not what you want&lt;/li&gt;&#xA;&lt;li&gt;when reading A01 it's not clear whether the plate size is 384 or 1536&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2711" LastActivityDate="2018-04-23T07:50:05.783" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4118" PostTypeId="2" ParentId="4116" CreationDate="2018-04-23T07:54:09.660" Score="2" Body="&lt;p&gt;and here's a scheme I made up myself:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AA01..AA48&#xA;..&#xA;AZ01..AZ48&#xA;BA01..BA48&#xA;..&#xA;BF01..BF48&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Pros:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;alphabetic sorting of well ids does what you expect&lt;/li&gt;&#xA;&lt;li&gt;it's clear what's the row and what's the column&lt;/li&gt;&#xA;&lt;li&gt;it's easy to distinguish well ids of 384-well plate from those of a 1536-well plate&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Cons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it's not used anywhere AFAIK&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2711" LastActivityDate="2018-04-23T07:54:09.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4119" PostTypeId="1" AcceptedAnswerId="4122" CreationDate="2018-04-23T08:06:29.797" Score="-1" ViewCount="838" Body="&lt;p&gt;Original tar.gz files look like below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;UNCID_2668056.27d4c881-e906-437f-921d-467eb107798a.150108_UNC15-SN850_0402_AC5FTHACXX_2_GTTTCG.tar.gz&#xA;UNCID_2206882.58e94be5-2da2-4a96-944a-060f20216572.150108_UNC15-SN850_0402_AC5FTHACXX_3_CTTGTA.tar.gz&#xA;UNCID_2205121.2b3d596a-68ca-4c60-bb16-8c0cbd8fb190.140828_UNC11-SN627_0378_AC5FAWACXX_4_TAGCTT.tar.gz&#xA;UNCID_2209082.76a625e9-22b3-42b8-8813-27c133d0b248.110608_UNC11-SN627_0103_BD0CVVABXX_7.tar.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Before extracting the tar.gz files, I changed their names based on names.text file as below and using the command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;xargs -n2 mv -i &amp;lt;&amp;lt;&amp;lt;&quot;$(sed '1d; s/$/.tar.gz/' names.txt)&quot;&#xA;&#xA;Tar.gz_filenames       Sample_Names&#xA;UNCID_2668056.27d4c881-e906-437f-921d-467eb107798a.150108_UNC15-SN850_0402_AC5FTHACXX_2_GTTTCG.tar.gz                Sample2&#xA;UNCID_2206882.58e94be5-2da2-4a96-944a-060f20216572.150108_UNC15-SN850_0402_AC5FTHACXX_3_CTTGTA.tar.gz                Sample1&#xA;UNCID_2205121.2b3d596a-68ca-4c60-bb16-8c0cbd8fb190.140828_UNC11-SN627_0378_AC5FAWACXX_4_TAGCTT.tar.gz                Sample3&#xA;UNCID_2209082.76a625e9-22b3-42b8-8813-27c133d0b248.110608_UNC11-SN627_0103_BD0CVVABXX_7.tar.gz                       Sample4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After changing the names, looks like below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sample1.tar.gz&#xA;Sample2.tar.gz&#xA;Sample3.tar.gz&#xA;Sample4.tar.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using the tar.gz files after changing names I used the below code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in /doc/*.tar.gz&#xA;do&#xA;    tar xvzf $i -C $TMPDIR&#xA;    for sample in $TMPDIR/*1.fastq&#xA;    do&#xA;        dir2=&quot;/destination/directory&quot;&#xA;        base=$(basename $sample &quot;.1.fastq&quot;)&#xA;        module load HISAT2/2.0.4-goolf-1.7.20; module load SAMtools/1.3.1-goolf-1.7.20; hisat2 -p 8 --dta --rna-strandness RF -x /genome_index -1 $TMPDIR/${base}.1.fastq -2 $TMPDIR/${base}.2.fastq | samtools view -Sb - &amp;gt; ${dir2}/${base}.bam&#xA;    done&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I run the above code, the output is like below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;150108_UNC15-SN850_0402_AC5FTHACXX_GTTTCG_L002_1.fastq&#xA;150108_UNC15-SN850_0402_AC5FTHACXX_GTTTCG_L002_2.fastq&#xA;150108_UNC15-SN850_0402_AC5FTHACXX_CTTGTA_L003_1.fastq&#xA;150108_UNC15-SN850_0402_AC5FTHACXX_CTTGTA_L003_2.fastq&#xA;140828_UNC11-SN627_0378_AC5FAWACXX_TAGCTT_L004_2.fastq&#xA;140828_UNC11-SN627_0378_AC5FAWACXX_TAGCTT_L004_1.fastq&#xA;110608_UNC11-SN627_0103_BD0CVVABXX.7_1.fastq&#xA;110608_UNC11-SN627_0103_BD0CVVABXX.7_2.fastq&#xA;Warning: Could not open read file &quot;/scratch/username/slurm-job.36320423/140828_UNC11-SN627_0378_AC5FAWACXX_TAGCTT_L004_1.fastq.1.fastq&quot; for reading; skipping...&#xA;Error: No input read files were valid&#xA;(ERR): hisat2-align exited with value 1&#xA;Warning: Could not open read file &quot;/scratch/username/slurm-job.36320423/150108_UNC15-SN850_0402_AC5FTHACXX_CTTGTA_L003_1.fastq.1.fastq&quot; for reading; skipping...&#xA;Error: No input read files were valid&#xA;(ERR): hisat2-align exited with value 1&#xA;Warning: Could not open read file &quot;/scratch/username/slurm-job.36320423/150108_UNC15-SN850_0402_AC5FTHACXX_GTTTCG_L002_1.fastq.1.fastq&quot; for reading; skipping...&#xA;Error: No input read files were valid&#xA;(ERR): hisat2-align exited with value 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I dont mins whatever the name of fastq's in &lt;code&gt;$TMPDIR&lt;/code&gt; but from &lt;code&gt;$TMPDIR&lt;/code&gt; hisat2 should use those fastq's and give output bam files in /destination like below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sample1.bam&#xA;Sample2.bam&#xA;Sample3.bam&#xA;Sample4.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to run a script doing tar.gz files -&gt; fastq -&gt; bam (everything in loop)&lt;/p&gt;&#xA;" OwnerUserId="1377" LastEditorUserId="1377" LastEditDate="2018-04-23T09:06:45.370" LastActivityDate="2018-04-30T07:12:16.343" Title="Script to run everything in a loop for extracting tar.gz files into fastq and to bam with alignment?" Tags="|rna-seq|file-formats|fastq|linux|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4121" PostTypeId="2" ParentId="4119" CreationDate="2018-04-23T09:31:54.723" Score="1" Body="&lt;p&gt;Your samples end with &lt;code&gt;_1.fastq&lt;/code&gt; and &lt;code&gt;_2.fastq&lt;/code&gt;, so you need to change this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-1 $TMPDIR/${base}.1.fastq -2 $TMPDIR/${base}.2.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-1 $TMPDIR/${base}_1.fastq -2 $TMPDIR/${base}_2.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-23T09:31:54.723" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="4122" PostTypeId="2" ParentId="4119" CreationDate="2018-04-23T09:33:31.583" Score="7" Body="&lt;p&gt;This should work for you:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in *.tar.gz&#xA;do&#xA;    dir2=&quot;/destination/directory&quot;&#xA;    tar xvzf $i -C $TMPDIR&#xA;    files=($TMPDIR/*)&#xA;    module load HISAT2/2.0.4-goolf-1.7.20; module load SAMtools/1.3.1-goolf-1.7.20; hisat2 -p 8 --dta --rna-strandness RF -x /genome_index -1 $files[0] -2 $files[1]| samtools view -Sb - &amp;gt; ${dir2}/$i.bam&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(If this is the sort of thing you'll be doing often, it would be worth investing the time to learn a more robust approach such as Snakemake or Nextflow).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to run on all files at the same time, the easiest way is &lt;code&gt;find&lt;/code&gt; / &lt;code&gt;xargs&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Change you script to this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in $@&#xA;do&#xA;    dir2=&quot;/destination/directory&quot;&#xA;    tar xvzf $i -C $TMPDIR&#xA;    files=($TMPDIR/*)&#xA;    module load HISAT2/2.0.4-goolf-1.7.20; module load SAMtools/1.3.1-goolf-1.7.20; hisat2 -p 8 --dta --rna-strandness RF -x /genome_index -1 $files[0] -2 $files[1]| samtools view -Sb - &amp;gt; ${dir2}/$i.bam&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then run it like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;find . -d 1 -name '*.tar.gz' | xargs -n 1 bash NAMEOFSCRIPT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This works for SGE. I'm not sure about the details of SLURM. You might have to figure out how to run each job in the background or use &lt;code&gt;gnu parallel&lt;/code&gt; or something. As &lt;code&gt;@Ian Sudbury&lt;/code&gt; says, pipeline software can take care of all these details for you.&lt;/p&gt;&#xA;" OwnerUserId="581" LastEditorUserId="581" LastEditDate="2018-04-30T07:12:16.343" LastActivityDate="2018-04-30T07:12:16.343" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4123" PostTypeId="2" ParentId="4091" CreationDate="2018-04-23T14:26:13.680" Score="1" Body="&lt;p&gt;I recently used &lt;a href=&quot;http://sift.bii.a-star.edu.sg/sift4g/AnnotateVariants.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SIFT4G&lt;/a&gt; to predict the deleteriousness for a SNP dataset in Arabidopsis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I noticed that besides this, the tool also provides annotation for the SNPs (like gene region, amino acid substitution, start/stop etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They have an Arabidopsis database readily available so you don't have to generate your own. Your input should be enough to run the tool, although you might have to do some reformatting (the tool requires VCF format files).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing though, not all SNPs are considered by the tool. Intronic SNPs and SNPs outside gene regions are ignored afaik.&lt;/p&gt;&#xA;" OwnerUserId="2714" LastEditorUserId="48" LastEditDate="2018-04-24T11:05:12.380" LastActivityDate="2018-04-24T11:05:12.380" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4124" PostTypeId="1" CreationDate="2018-04-23T20:26:54.083" Score="1" ViewCount="581" Body="&lt;p&gt;I want to compute the depth of coverage only for specific intervals in phase 3, 1000 genomes project.I have not worked with 1000 genomes project before, so a bit unfamiliar with it. I do not want to download all of the bam files for the entire genomic region, just those intervals which I am interested in them.&#xA;I guess I can use this (I found from biostars forum)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools -bu view 'http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/phase3/data/HG01375/alignment/HG01375.mapped.ILLUMINA.bwa.CLM.low_coverage.20120522.bam' &quot;2:1000000-2000000&quot; | (...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I am just a bit confused. First of all, when I try the code above directly from the command line, I get an error message &quot;No such file or directory&quot;! Am I doing something wrong? or should I provide a specific path?&#xA;Second, I want to run  it for the mapped files for all individuals: &lt;code&gt;*.bam”.&lt;/code&gt; through a loop. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me to solve my problem?&lt;/p&gt;&#xA;" OwnerUserId="1749" LastActivityDate="2018-04-23T20:36:53.930" Title="running a samtools command for multiple bam files from 1000 genomes project" Tags="|sam|samtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4125" PostTypeId="2" ParentId="4124" CreationDate="2018-04-23T20:36:53.930" Score="1" Body="&lt;p&gt;The command you're looking for is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -c http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/phase3/data/HG01375/alignment/HG01375.mapped.ILLUMINA.bwa.CLM.low_coverage.20120522.bam 2:1000000-2000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For a series of intervals, you'd be better off scripting something with pysam:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pysam&#xA;bam = pysam.AlignmentFile(&quot;some file.bam&quot;)&#xA;regions = open(&quot;your regions.bed&quot;)&#xA;for region in regions:&#xA;    chrom, start, end = region.split(&quot;\t&quot;)&#xA;     bam.count(chrom, int(start), int(end))  # Do something with the result&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You might see if there are bigWig files available, they'd be faster to process.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-23T20:36:53.930" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4126" PostTypeId="2" ParentId="4104" CreationDate="2018-04-23T23:44:32.183" Score="2" Body="&lt;p&gt;There are various attempts to guess putative horizontal gene transfer using sequence composition (kmer-frequencies or GC content), you do not have to implement a new method by your own. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I co-authored one of them called &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/30/8/1081/254636&quot; rel=&quot;nofollow noreferrer&quot;&gt;SigHunt&lt;/a&gt;. It's kind of working, but the code is really messy and there are now bit more precise methods out there. The advantage is SigHunt is that it provides access to all the data calculated on the way (the tetranucleotide frequencies), so if you want to play with it, it's fun.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method I would probably go for now if I would have searched for HGT is &lt;a href=&quot;https://academic.oup.com/bib/advance-article-abstract/doi/10.1093/bib/bbw118/2726041?redirectedFrom=fulltext&quot; rel=&quot;nofollow noreferrer&quot;&gt;MTGIpick&lt;/a&gt;. The most of the tools detects HGT within a genome by a sliding window, but MTGIpick is refining boundaries where the change of kmer frequencies is rapid. I never tried it personally but the approach is sound to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively there are other paradigms for detection of HGT. Phylogeny based methods (there are many, but &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-11-324&quot; rel=&quot;nofollow noreferrer&quot;&gt;for instance&lt;/a&gt;) or by &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/32/17/i595/2450753&quot; rel=&quot;nofollow noreferrer&quot;&gt;mapping reads to species boundaries&lt;/a&gt;. Phylogeny based methods usually provide stronger evident, but have a disadvantage to be database dependent (i.e. you can not build a tree if you don't have sufficient coverage of homologs across tree of life).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also like to mention that right now I do not follow the HGT literature closely, there might be some big new I just missed.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="73" LastEditDate="2018-04-30T20:16:44.190" LastActivityDate="2018-04-30T20:16:44.190" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4127" PostTypeId="2" ParentId="4061" CreationDate="2018-04-24T03:33:00.627" Score="1" Body="&lt;p&gt;Module 3 does not exist during PCR amplification. Here is how NEBNext adaptor and multiplex PCR primers work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;During the 1st PCR cycle, only the P7 primers are used to bind and amplify ligated DNA, generating the complementary strands that contain the binding sites for P5 primers (Universal in this case). During the second cycle of amplification, the P5  primer hybridizes to the DNA strand generated during the first cycle and start to amplify. Please refer to the following figure for more details.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fY6K8.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fY6K8.jpg&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2715" LastActivityDate="2018-04-24T03:33:00.627" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4128" PostTypeId="1" AcceptedAnswerId="4129" CreationDate="2018-04-24T08:26:04.937" Score="2" ViewCount="1369" Body="&lt;p&gt;I have a gene expression data (read counts) table with many samples. Showing some of them below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Ensembl_ID          A   B   C   D   E   F   G&#xA;ENSG00000236601.2   0   0   0   1   0   0   1&#xA;ENSG00000237094.12  2   187 442 109 88  144 486&#xA;ENSG00000269732.1   0   0   0   0   0   0   0&#xA;ENSG00000284733.1   0   0   0   0   0   0   0&#xA;ENSG00000233653.3   0   0   0   0   0   0   0&#xA;ENSG00000250575.1   1   11  17  5   5   8   17&#xA;ENSG00000278757.1   0   0   2   0   1   0   7&#xA;ENSG00000230021.9   0   25  30  27  9   8   11&#xA;ENSG00000235146.2   0   0   0   0   0   0   0&#xA;ENSG00000225972.1   0   7   3016    11  5   14  5&#xA;ENSG00000225630.1   1   113 194 76  91  47  94&#xA;ENSG00000237973.1   25  1037    9767    1160    886 1321    5220&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to select top 100 genes with reverse sort of variance. How to do that?&lt;/p&gt;&#xA;" OwnerUserId="1369" LastEditorUserId="96" LastEditDate="2019-01-30T17:35:58.177" LastActivityDate="2019-01-30T17:35:58.177" Title="Select top 100 genes ranked by variance in read counts" Tags="|rna-seq|r|bioconductor|reads|filtering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4129" PostTypeId="2" ParentId="4128" CreationDate="2018-04-24T08:39:45.477" Score="4" Body="&lt;p&gt;I assume by &quot;reverse sort of variance&quot; you mean &quot;highest variance&quot;. Assuming you made a matrix out of that (set the row names to the first column and then remove it) and called it &lt;code&gt;m&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sel = order(apply(m, 1, var), decreasing=TRUE)[1:100]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;sel&lt;/code&gt; then contains the indices into your matrix (or the original dataframe).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, I hope you're not filtering by variance before performing differential expression. That would be incredibly incorrect. Doing this sort of filtering for performing PCA or something like that can be useful, though you should really be using normalized counts (or a transformation of them).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-24T08:39:45.477" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="4130" PostTypeId="2" ParentId="4085" CreationDate="2018-04-24T16:16:37.890" Score="2" Body="&lt;p&gt;Seurat has as part of its protocol a step where you filter based on UMI counts and percent mitochondrial&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://satijalab.org/seurat/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://satijalab.org/seurat/pbmc3k_tutorial.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2018-04-24T16:16:37.890" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4131" PostTypeId="1" CreationDate="2018-04-24T18:10:19.913" Score="0" ViewCount="82" Body="&lt;p&gt;I have been asked to write a script in R to edit, using data.frame, this .fa file that is the result from extracting gene sequences from a repeat masker coordinate file on the genome I am working with. Each input looks as such:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;rnd-5_family-5445_Unspecified(-)&#xA;TTTCACCGTAAATTAACTTTGAGAGGAGCTAATTCCTAAAAGAATTATACCGGCCTATTTG&#xA;&amp;gt;rnd-5_family-2614_Unspecified(-)&#xA;AATCTGAGATTAGATTTATTTATCTGTTATCCGCTCAGGTTGAAAAGTTTGCGCAATGTATAACTATA&#xA;AAAATCTTTCTACCACTTTTGATTCATTTTTTAAATCTGGGGTCACATTTTATTCACGGTAGAAATTG&#xA;TAATTTACAAAATGAATTACTTGAAGGCAACACGAATCCAGAGTGATGCTTTACATAAATCTGCTTCT&#xA;ACCGATGCCAAAAATTGACGATATTCTATTATTTAATCTAAATGTTAGTCTTTACATACCCTCCCCTA&#xA;ATTGTTAGAATTTTATGAAATTTGATTTCAGGGGTCAGTTTAGCATGCTAAATCTAATTCAATAGATT&#xA;GATATTTTTCTTCAGGTAAAGAAAATTTTTGCGTCAAAGTAATCATATTCCTCCACGATTGCATATAA&#xA;CTATGGTATATAATTTAAAAGATTACACTTTACGTAATGAAAAATCGGCCAATCATTCAAAAGTTATG&#xA;AAAGTGATC &#xA;&amp;gt;rnd-5_family-2614_Unspecified(-)&#xA;TTTCTAGTTTGTATTTGAGGAAAGAAAATCCTACTATCACTTTTTAAAAAATTCACCAATCTGAGATT&#xA;AGATTTATTTATCTGTTATCCGCTCAGGTTGAAAAGTTTGCGCAATGTATAACTATAAAAATCTTTCT&#xA;ACCACTTTTGATTCATTTTTTAAATCTGGGGTCACATTTTATTCACGGTAGAAATTGTAATTTACAAA&#xA;ATGAATTACTT &#xA;&amp;gt;rnd-4_family-798_Unspecified(-)&#xA;ATGCTTAATTTGCTAAGCATTTAATAACCTTTTTGGGTTTTGTGATAAAGGATGCTGTAGACATTAAA&#xA;ATAAACCTTATACTGCTAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the repeat masker coordinates that the above were derived from:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;KB824701.1 417 478 rnd-5_family-5445_Unspecified . -&#xA;&amp;gt;KB824701.1 587 1072 rnd-5_family-2614_Unspecified . -&#xA;&amp;gt;KB824701.1 914 1129 rnd-5_family-2614_Unspecified . -&#xA;&amp;gt;KB824701.1 1138 1225 rnd-4_family-798_Unspecified . -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and ideally I would like it to be in the format of this, for example using the first one:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;rnd-5_family-5445_KB824701.1_417_478&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I was told to use data.frame in R but I am not sure exactly how this will work? Can someone help please&lt;/p&gt;&#xA;" OwnerUserId="1935" LastEditorUserId="77" LastEditDate="2018-04-24T19:48:29.710" LastActivityDate="2020-10-03T20:00:15.053" Title="Help Adjusting Bedtools Output in R" Tags="|r|ngs|phylogenetics|sequence-annotation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4132" PostTypeId="1" AcceptedAnswerId="4134" CreationDate="2018-04-24T18:24:55.897" Score="2" ViewCount="150" Body="&lt;p&gt;The SAM file format specification makes clear that optional fields must follow the format &lt;code&gt;TAG:TYPE:VALUE&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Type   Regexp matching                                       VALUE Description&#xA;A     [!-~]                                                  Printable character&#xA;i     [-+]?[0-9]+                                            Signed integer&#xA;f     [-+]?[0-9]*\.?[0-9]+([eE][-+]?[0-9]+)?                 Single-precision floating number&#xA;Z     [ !-~]*                                                Printable string, including space&#xA;H     ([0-9A-F][0-9A-F])*                                    Byte array in the Hex format10&#xA;B     [cCsSiIf](,[-+]?[0-9]*\.?[0-9]+([eE][-+]?[0-9]+)?)+    Integer or numeric array&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, looking at the samtools source code, I noticed this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/samtools/samtools/blob/develop/sam_view.c#L920-L931&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/samtools/samtools/blob/develop/sam_view.c#L920-L931&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;switch (aux_type) {&#xA;    case 'H':&#xA;    case 'Z':&#xA;        if (kputs(bam_aux2Z(s), linebuf) &amp;lt; 0) return false;&#xA;        break;&#xA;    case 'i': kputw(bam_aux2i(s), linebuf); break;&#xA;    case 'I': kputuw(bam_aux2i(s), linebuf); break;&#xA;    case 'A': kputc(bam_aux2A(s), linebuf); break;&#xA;    case 'f': kputd(bam_aux2f(s), linebuf); break;&#xA;    case 'B': kputs(&quot;*** Unhandled aux type ***&quot;, linebuf); return false;&#xA;    default:  kputs(&quot;*** Unknown aux type ***&quot;, linebuf); return false;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) I have never seen an &quot;Integer or numeric array&quot; of &lt;code&gt;case &quot;B&quot;&lt;/code&gt;. What does such an tag look like? Is there an example? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) Why is this currently &lt;code&gt;&quot;*** Unhandled aux type ***&quot;&lt;/code&gt;? Was this the case with previous versions of samtools?&lt;/p&gt;&#xA;" OwnerUserId="1770" LastActivityDate="2018-04-24T19:43:54.937" Title="SAM format: Does the BAM &quot;Integer or numeric array&quot; field no longer exist? Why?" Tags="|sam|samtools|htslib|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4133" PostTypeId="1" CreationDate="2018-04-24T18:49:25.273" Score="3" ViewCount="947" Body="&lt;p&gt;I have a list of genes nearly 20000:&#xA;gene name (column1) and coordinates (columns 2 and 3)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head genes&#xA;    Ma0_73Ns    2359    2923        &#xA;    Ma0_73Ns    4727    4868&#xA;        &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to find the chromosomal location of each gene, on which chromosome each gene located? my study organism is &lt;em&gt;Helianthus annuus&lt;/em&gt; (sunflower)&lt;/p&gt;&#xA;" OwnerUserId="1749" LastEditorUserId="475" LastEditDate="2022-11-16T14:15:37.167" LastActivityDate="2022-11-20T16:42:30.837" Title="How can I find the chromosomal location of a list of genes?" Tags="|gene|sequence-annotation|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4134" PostTypeId="2" ParentId="4132" CreationDate="2018-04-24T19:43:54.937" Score="3" Body="&lt;ol&gt;&#xA;&lt;li&gt;An example would be &lt;code&gt;ZC:B:i,271,271,1,0&lt;/code&gt;, which is from TMAP. The &lt;code&gt;i&lt;/code&gt; part denotes the values held in the array, which are comma-separated.&lt;/li&gt;&#xA;&lt;li&gt;That code is only used in &lt;code&gt;samtools fastq&lt;/code&gt;, where that tag type isn't output into fastq files.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-24T19:43:54.937" CommentCount="8" ContentLicense="CC BY-SA 3.0" />
  <row Id="4135" PostTypeId="2" ParentId="4133" CreationDate="2018-04-24T20:24:52.973" Score="2" Body="&lt;p&gt;Ok so I found two websites that could help. The one I found first is more intuitive to go through, but it looks like it could be outdated as the second website I found seems more recent so it probably has a more complete genome to use as the reference. (Sorry, I haven't worked personally with sunflowers and don't know for sure where you would go for the &quot;official&quot; reference genome, or if there even is an &quot;official&quot; one, but hopefully one of these websites will help.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) This is a website I found that looks like it is a website for the sunflower genome and includes gene annotation information. (The recent publications list only goes up till 2015 though so it may not be the most recent data)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.sunflowergenome.org/annotations-data/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sunflowergenome.org/annotations-data/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you scroll down to the bottom of this page there is a text &lt;a href=&quot;https://www.sunflowergenome.org/annotations-data/assets/data/annotations/FILE_SPECIFICATIONS.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;file&lt;/a&gt; detailing the descriptions of how the files are formatted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the linked to file gives the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;&lt;p&gt;Ha412v1r1_CDS_v1.0.fasta.gz&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Description: Nucleotide CDS for each gene&lt;/li&gt;&#xA;  &lt;li&gt;Header format: &gt;TranscriptID GeneID Chromosome_start_end_strand&lt;/li&gt;&#xA;  &lt;li&gt;Example: &gt;Ha1_00044280-RA Ha1_00044280 Ha1_94590914-94591860_-&lt;/li&gt;&#xA;  &lt;/ul&gt;&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you download this fasta file you could then create a python script that will read in the file and extract the headers. If each header is a string variable called &quot;mytext&quot;, you could then extract the TranscriptID, GeneID, and chromosomal info and using mytext.split(&quot; &quot;). You could then split on &quot;_&quot; to further parse the chromosomal location info.  You could save the extracted information in a dictionary using the gene name as a key and parse through the original file you mentioned containing the list of genes, in order to get the chromosomal location information you desire.  The one caveat is that this is assuming that your gene names are the same as the geneIDs the above website uses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did not exhaustively examine the list of files and perhaps there is another one that would be a better one to parse through for your purpose, but the linked to text file gives more details on the files and their formats and presumably they could be parsed in a similar manner. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Double checking my first answer I found this &lt;a href=&quot;https://www.nature.com/articles/nature22380&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Badouin H, Gouzy J, Grassa CJ, Murat F, Staton SE, Cottret L,&#xA;  Lelandais-Brière C, Owens GL, Carrère S, Mayjonade B, Legrand L. The&#xA;  sunflower genome provides insights into oil metabolism, flowering and&#xA;  Asterid evolution. Nature. 2017 Jun;546(7656):148.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Which links to a website as follows:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;The integrative web interface Heliagene provides visualization, querying tools for data mining and network exploration for the community (&lt;a href=&quot;https://www.heliagene.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.heliagene.org&lt;/a&gt;).&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So it looks like this website also has reference genomes for sunflowers, although a lot of the annotation information seems to be locked unless you have a login. Like I mentioned previously, this website is a little less intuitive to go through but you might be able to find something helpful here if you think the other website isn't official or recent enough.&lt;/p&gt;&#xA;" OwnerUserId="1574" LastEditorUserId="1574" LastEditDate="2018-04-24T22:28:03.560" LastActivityDate="2018-04-24T22:28:03.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4136" PostTypeId="1" AcceptedAnswerId="4137" CreationDate="2018-04-24T20:33:44.987" Score="3" ViewCount="62" Body="&lt;p&gt;This is pretty vexing: I got asked to explain the meaning of the match line in a blastp output, especially the subtle difference between a blank and plus. E.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; Query  181  NHIEEL...&#xA;             N I+EL...&#xA; Sbjct  181  NQIKEL...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For me this is under the &quot;everyone knows this&quot; category, but I wanted to have an authoritative reference I could cite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alas, none of the documents on the &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&amp;amp;PAGE_TYPE=BlastDocs%20BLAST%20help%20page&quot; rel=&quot;nofollow noreferrer&quot;&gt;BLAST help page&lt;/a&gt; seems to have this information. My Google fu also failed as I was not able to find an authoritative source which I could cite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I'm not sure whether BLAST introduced this notation or it simply picked it up somewhere else.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any pointers appreciated.&lt;/p&gt;&#xA;" OwnerUserId="44" LastActivityDate="2018-04-25T10:11:31.873" Title="Official documentation for textual BLAST output?" Tags="|blast|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4137" PostTypeId="2" ParentId="4136" CreationDate="2018-04-24T23:27:38.677" Score="1" Body="&lt;p&gt;The reference that is mentioned when I run a BLASTP search using NCBI's web service is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC146917/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;. Here's some relevant text from the caption of Figure 3 of that paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;On the central lines of the alignment, identities are echoed and substitutions to which the BLOSUM-62 matrix (18) gives a positive score are indicated by a ‘+’ symbol.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-04-24T23:27:38.677" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4138" PostTypeId="1" CreationDate="2018-04-25T02:01:56.937" Score="4" ViewCount="1085" Body="&lt;p&gt;I am searching for a plant metabolome or secondary metabolite database for a college bioinformatics project. Up until now, I have found these two databases, as I am giving their links below;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://scbt.sastra.edu/pmdb/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://scbt.sastra.edu/pmdb/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.plantcyc.org/https://www.plantcyc.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.plantcyc.org/https://www.plantcyc.org/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and searched for different scientific articles and reviews;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fpls.2013.00073/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.frontiersin.org/articles/10.3389/fpls.2013.00073/full&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/27258266&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pubmed/27258266&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But only one and updated database which could contain all known plants and all of their known metabolites would be perfect. Could you suggest me any database or any other source with a high coverage for my project?&lt;/p&gt;&#xA;" OwnerUserId="2722" LastEditorUserId="2722" LastEditDate="2018-04-25T19:27:14.480" LastActivityDate="2018-05-01T23:15:13.657" Title="A Database for Plant Metabolome or Secondary Metabolites?" Tags="|database|public-databases|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4139" PostTypeId="1" AcceptedAnswerId="4155" CreationDate="2018-04-25T08:34:18.413" Score="7" ViewCount="2465" Body="&lt;p&gt;I was asked to check the synteny of some genes in two genome assemblies of the same species (PacBio &amp;amp; Illumina). I was given two scaffolds couples (so 4 scaffolds total) in the Illumina genome and asked if they were assembled together in the PacBio assembly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I proceeded with a DNA-DNA alignment with &lt;a href=&quot;https://github.com/lh3/minimap2&quot; rel=&quot;noreferrer&quot;&gt;minimap2&lt;/a&gt; of the two Illumina scaffolds on the PacBio whole genome. But I am not sure on how to directly use the sam file for the question I was asked. So I tried to visualize this alignment using &lt;a href=&quot;http://darlinglab.org/mauve/mauve.html&quot; rel=&quot;noreferrer&quot;&gt;Mauve&lt;/a&gt;. It worked for the first couple, but it failed (blank page) for the second couple of scaffolds. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Surely, Mauve is not the best option out, so can you suggest an alternative? This is a chordata genome. &lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2018-04-26T15:24:40.913" Title="visualisation of genome alignment" Tags="|sequence-alignment|software-recommendation|mauve|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4140" PostTypeId="1" CreationDate="2018-04-25T10:02:26.403" Score="1" ViewCount="66" Body="&lt;p&gt;At present, I have vcf file for each patient generated from tumor data. I am using virtual normal correction method for somatic mutation detection because I don't have normal data. For SNP or small variants detection, I have already converted vcf into tsv format but for SV, I need complete genomics junction file or high confidence junction beta file. Basically this file is used to represent SVs in the subject sequence. Look at the following pics for better idea :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FgEj0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FgEj0.png&quot; alt=&quot;SV File Structure. Junction file is nothing but alljunctionbeta-[ASM-ID].tsv in the above image.&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and the header of the file is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ID    LeftChr    LeftPosition    LeftStrand    LeftLength    RightChr    RightPosition    RightStrand    RightLength StrandConsistent    Interchromosomal    Distance    DiscordantMatePairAlignment    ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This file is specially generated and used by the company &lt;a href=&quot;http://www.completegenomics.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Complete Genomics&amp;quot;&lt;/a&gt;. This file format is basically used in SV representation. I have no idea about the meaning of junction. For more information, look at Data type [http://www.bx.psu.edu/~giardine/tests/tmp/DataFileFormats112.pdf].&lt;/p&gt;&#xA;&lt;p&gt;My question is how can I generate cg junction file from the given vcf and bam file.Can anybody tell me how to get it?&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;&lt;p&gt;p.s. I'm aware that Complete Genomics was acquired by BGI in $117.6M. Now the paper I am trying to understand can be obtained from &lt;a href=&quot;https://github.com/shiltemann/Virtual-Normal-Correction&quot; rel=&quot;nofollow noreferrer&quot;&gt;this github link&lt;/a&gt; because I don't have matched normal data. In order to identify SVs, I need cg junction file. I don't know how to get it. That's why I am looking here for some help. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="2141" LastEditorUserId="73" LastEditDate="2021-05-07T09:40:33.777" LastActivityDate="2021-05-07T09:40:33.777" Title="Complete Genomics Junction file for SV Detection" Tags="|vcf|genomics|structural-variation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4141" PostTypeId="2" ParentId="4111" CreationDate="2018-04-25T10:07:55.363" Score="3" Body="&lt;p&gt;Please take a look at &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/sdata201610&quot; rel=&quot;nofollow noreferrer&quot;&gt;An open access pilot freely sharing cancer genomic data from participants in Texas&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although I have not worked on this paper but author claimed that both tumor and normal data is publicly available. In case if you still need something more, than you can work on virtual normal data. And there are several papers which provide machnine learning techniques to overcome the deficiency of normal data (not much efficient but still they give considerable results). Hope this information will help you. Thanks&lt;/p&gt;&#xA;" OwnerUserId="2141" LastActivityDate="2018-04-25T10:07:55.363" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4142" PostTypeId="2" ParentId="4136" CreationDate="2018-04-25T10:11:31.873" Score="2" Body="&lt;p&gt;I think &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/4137/104&quot;&gt;gringer&lt;/a&gt; is correct to say this is because of the BLOSUM62 matrix and a ‘+’ symbol inidcating a positive score for a mismatch but here is a little more explanation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BLASTP looks for (nearly) identical short protein matches pairwise, called seeds, and then extends the seeds in each direction one residue at a time using arbitrary gap penalties for indels and a matrix for substitutions. Based on some thresholds and more, the highest scoring alignments are kept and ultimately reported.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The default choice of matrix for BLASTP is BLOSUM62, which looks like &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Class/FieldGuide/BLOSUM62.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. Obviously matches have high positive scores, but also mismatches have varying positive or negative scores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look up your changes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;H -&amp;gt; Q&lt;/code&gt;; &lt;code&gt;&lt;/code&gt;; score 0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;E -&amp;gt; K&lt;/code&gt;; &lt;code&gt;+&lt;/code&gt;; score 1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that this matrix is symmetric so the scores for &lt;code&gt;H -&amp;gt; Q == Q -&amp;gt; H&lt;/code&gt; and &lt;code&gt;E -&amp;gt; K == K -&amp;gt; E&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of why &lt;code&gt;E -&amp;gt; K&lt;/code&gt; has a positive score, well not all amino acid changes are viewed equally. BLOSUM matrices are not based on any biological theory but instead use the log-odds ratio, a likelihood approach, that estimates from real sequence alignments how frequently different amino acid changes occur. The more often &lt;code&gt;E -&amp;gt; K&lt;/code&gt; substitutions are observed in these alignments, the higher the substitution will score in the derived matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully these scores are intupretable based on the biochemical properties of the amino acids. For example, &lt;code&gt;R -&amp;gt; K&lt;/code&gt; scores 2, a  high score for a mismatch, but both amino acids are basic and so share a &lt;a href=&quot;https://www.mun.ca/biology/scarr/iGen3_06-02_basic.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;similar structure&lt;/a&gt;, presumably making a change more likely to occur and/or more likely to maintain the protein function and so not be harmful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your case, I don't have a good biological explanation for why &lt;code&gt;E -&amp;gt; K&lt;/code&gt; scores positive, another author &lt;a href=&quot;http://www.uky.edu/Classes/BIO/520/BIO520WWW/blosum62.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;notes&lt;/a&gt; this is surprising, but ultimately the reason it is given a positive score is just because regardless of why, &lt;code&gt;E -&amp;gt; K&lt;/code&gt; does apparently frequently occur, at least in the alignment data used to derive this scoring matrix. Incidentally, BLOSUM matrices are pretty ancient and based on few alignments by modern standards, but nonetheless appear to work quite well.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2018-04-25T10:11:31.873" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4143" PostTypeId="2" ParentId="4139" CreationDate="2018-04-25T11:30:14.077" Score="3" Body="&lt;p&gt;I think you could use &lt;a href=&quot;http://circos.ca/&quot; rel=&quot;nofollow noreferrer&quot;&gt;circos&lt;/a&gt; for this. The configuration is kind of tedious but it gives nice results. You would basically need to retrieve alignments coordinates and parse them into this format.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;scaf1 19720 31917 scaf2 28307 34227&#xA;scaf1 28307 34227 scaf2 19720 31917&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Example &lt;a href=&quot;https://link.springer.com/article/10.1186%2Fs12915-017-0439-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;figure&lt;/a&gt;: &#xA;&lt;a href=&quot;https://i.stack.imgur.com/SrWwL.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SrWwL.gif&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="439" LastActivityDate="2018-04-25T11:30:14.077" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4144" PostTypeId="2" ParentId="3980" CreationDate="2018-04-25T11:43:19.423" Score="0" Body="&lt;p&gt;Yes, use .fasta , you can try to convert your fastq files with NextGen Workbench for example.&lt;/p&gt;&#xA;" OwnerUserId="2724" LastActivityDate="2018-04-25T11:43:19.423" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4145" PostTypeId="2" ParentId="4073" CreationDate="2018-04-25T12:40:18.560" Score="1" Body="&lt;p&gt;The official &lt;a href=&quot;http://www.beast2.org/increasing-esss/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEAST&lt;/a&gt; website suggests a number of solutions to improve the ESS of a parameter&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not uncommon to have &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/27539485&quot; rel=&quot;nofollow noreferrer&quot;&gt;several billions of generations&lt;/a&gt; so I don't imagine increasing your number of generations would hurt.&lt;/p&gt;&#xA;" OwnerUserId="439" LastActivityDate="2018-04-25T12:40:18.560" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4146" PostTypeId="2" ParentId="3409" CreationDate="2018-04-25T13:36:41.993" Score="3" Body="&lt;p&gt;I am the author of the question / answer here &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2846/why-does-a-very-strong-blast-hit-get-lost-when-i-change-num-alignments-num-desc?noredirect=1&amp;amp;lq=1&quot;&gt;Why does a very strong BLAST hit get lost when I change num_alignments, num_descriptions or max_target_seqs parameter?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To my knowledge changing the evalue threshold should no change the&#xA;  values themselves, it should just change the reporting.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That is sadly not the case, though thats what I would say is typically expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yours was the exact situation that lead me to question the implementation of NCBI blast. I got results with an evalue above a certain value with a relaxed threshold, but with a stricter threshold (still lower than the value of the results I got) those results disappeared.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I found all parameters imposing some threshold show this behavior and are applied in an early step of the program leading to prematurely removed results that later on might score higher than those thresholds. (Say your hit first has an evalue of 5: threshold 10 = ok and kept, threshold 1e-5 = gets removed, but that same hit gets improved further along in the program and then gets an evalue of 1e-6. Now this is only shown when you use an evalue threshold of 5 or higher, though 1e-6 is clearly also a result you would want to see when using a threshold of 3, 1, 0.01, ... etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was relatively shocked by this and the fact that its &lt;strong&gt;basically nowhere officially documented and generally a not known fact&lt;/strong&gt; what influence these options have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sadly for now the stance of the developers is basically all is working as intended and the only feedback I got amounts to &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The safest approach is to relax (increase) the threshold and deal with a larger result set.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;aka Suck it up and deal with it.&lt;/p&gt;&#xA;" OwnerUserId="1786" LastActivityDate="2018-04-25T13:36:41.993" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4147" PostTypeId="2" ParentId="3057" CreationDate="2018-04-25T15:06:40.590" Score="4" Body="&lt;p&gt;For the correspondence between PDB and Uniprot entries you can use &lt;a href=&quot;https://www.ebi.ac.uk/pdbe/docs/sifts/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SIFTS&lt;/a&gt; -- a semi-automated mapping between PDB and UniProt maintained by PDBe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The pipeline that creates the mappings uses BLAST and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3531078/&quot; rel=&quot;nofollow noreferrer&quot;&gt;a few other criteria&lt;/a&gt; to decide which UniProt entry should be assigned to each PDB entry. The SIFTS website has all the data in CSV files.&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2018-04-25T15:06:40.590" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4148" PostTypeId="1" AcceptedAnswerId="4149" CreationDate="2018-04-25T18:22:17.867" Score="2" ViewCount="1684" Body="&lt;p&gt;When I am trying to install &lt;code&gt;Seurat&lt;/code&gt; in &lt;code&gt;R markdown file&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;install.packages(&quot;Seurat&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;it fails with the following errors:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Installing package into ‘/sfs/qumulo/qhome/nv4e/R/x86_64-pc-linux-gnu-library/3.3’&#xA;(as ‘lib’ is unspecified)&#xA;Warning in install.packages :&#xA;  dependency ‘proxy’ is not available&#xA;also installing the dependencies ‘igraph’, ‘dtw’, ‘diffusionMap’&#xA;&#xA;trying URL 'https://cran.rstudio.com/src/contrib/igraph_1.2.1.tar.gz'&#xA;&#xA;Content type 'unknown' length 2703426 bytes (2.6 MB)&#xA;&#xA;downloaded 2.6 MB&#xA;&#xA;trying URL 'https://cran.rstudio.com/src/contrib/dtw_1.18-1.tar.gz'&#xA;&#xA;Content type 'unknown' length 849256 bytes (829 KB) downloaded 829 KB trying URL 'https://cran.rstudio.com/src/contrib/diffusionMap_1.1-0.tar.gz'&#xA;&#xA;Content type 'unknown' length 39130 bytes (38 KB)&#xA;downloaded 38 KB&#xA;&#xA;trying URL 'https://cran.rstudio.com/src/contrib/Seurat_2.3.0.tar.gz'&#xA;&#xA;Content type 'unknown' length 846065 bytes (826 KB)&#xA;&#xA;downloaded 826 KB&#xA;* installing *source* package ‘igraph’ ...&#xA;** package ‘igraph’ successfully unpacked and MD5 sums checked&#xA;installing to /sfs/qumulo/qhome/nv4e/R/x86_64-pc-linux-gnu-library/3.3/igraph/libs&#xA;** R&#xA;** demo&#xA;** inst&#xA;** preparing package for lazy loading&#xA;** help&#xA;*** installing help indices&#xA;** building package indices&#xA;** testing if installed package can be loaded&#xA;&#xA;Error in dyn.load(file, DLLpath = DLLpath, ...) :    unable to load shared object '/sfs/qumulo/qhome/nv4e/R/x86_64-pc-linux-gnu-library/3.3/igraph/libs/igraph.so': libicui18n.so.58: cannot open shared object file: No such file or directory&#xA;&#xA;Error: loading failed Execution halted ERROR: loading failed&#xA;* removing ‘/sfs/qumulo/qhome/nv4e/R/x86_64-pc-linux-gnu-library/3.3/igraph’ Warning in install.packages :   installation of package ‘igraph’ had non-zero exit status&#xA;&#xA;ERROR: dependency ‘proxy’ is not available for package ‘dtw’&#xA;* removing ‘/sfs/qumulo/qhome/nv4e/R/x86_64-pc-linux-gnu-library/3.3/dtw’ Warning in install.packages :   installation of package ‘dtw’ had non-zero exit status&#xA;&#xA;ERROR: dependency ‘igraph’ is not available for package ‘diffusionMap’&#xA;* removing ‘/sfs/qumulo/qhome/nv4e/R/x86_64-pc-linux-gnu-library/3.3/diffusionMap’ Warning in install.packages :   installation of package ‘diffusionMap’ had non-zero exit status&#xA;&#xA;ERROR: dependencies ‘igraph’, ‘dtw’, ‘diffusionMap’ are not available for package ‘Seurat’&#xA;* removing ‘/sfs/qumulo/qhome/nv4e/R/x86_64-pc-linux-gnu-library/3.3/Seurat’ Warning in install.packages :   installation of package ‘Seurat’ had non-zero exit status&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How could it be fixed? What to try?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Operating system is &lt;code&gt;CentOS Linux release 7.3.1611 (Core)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Update&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I also tried development version of &lt;code&gt;Seurat&lt;/code&gt; but it failed too:&#xA;&lt;a href=&quot;http://satijalab.org/seurat/devinstall.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://satijalab.org/seurat/devinstall.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Update&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I tried installing &lt;code&gt;Seurat&lt;/code&gt; using &lt;code&gt;conda install -c bioconda r-seurat&lt;/code&gt; and it is not working. It is giving the error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;-bash-4.2$ conda install -c bioconda r-seurat &#xA;  Fetching package metadata .............&#xA;  Solving package specifications: &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;PackageNotFoundError: Packages missing in current channels:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;r-seurat -&gt; r-cowplot&lt;/li&gt;&#xA;&lt;li&gt;r-seurat -&gt; r-diffusionmap&lt;/li&gt;&#xA;&lt;li&gt;r-seurat -&gt; r-dtw&lt;/li&gt;&#xA;&lt;li&gt;r-seurat -&gt; r-ggridges&lt;/li&gt;&#xA;&lt;li&gt;r-seurat -&gt; r-ica&lt;/li&gt;&#xA;&lt;li&gt;r-seurat -&gt; r-pbapply&lt;/li&gt;&#xA;&lt;li&gt;r-seurat -&gt; r-ranger&lt;/li&gt;&#xA;&lt;li&gt;r-seurat -&gt; r-rcppprogress&lt;/li&gt;&#xA;&lt;li&gt;r-seurat -&gt; r-sdmtools&lt;/li&gt;&#xA;&lt;li&gt;r-seurat -&gt; r-tclust&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We have searched for the packages in the following channels:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://conda.anaconda.org/bioconda/linux-64&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://conda.anaconda.org/bioconda/linux-64&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://conda.anaconda.org/bioconda/noarch&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://conda.anaconda.org/bioconda/noarch&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://repo.continuum.io/pkgs/main/linux-64&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://repo.continuum.io/pkgs/main/linux-64&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://repo.continuum.io/pkgs/main/noarch&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://repo.continuum.io/pkgs/main/noarch&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://repo.continuum.io/pkgs/free/linux-64&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://repo.continuum.io/pkgs/free/linux-64&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://repo.continuum.io/pkgs/free/noarch&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://repo.continuum.io/pkgs/free/noarch&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://repo.continuum.io/pkgs/r/linux-64&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://repo.continuum.io/pkgs/r/linux-64&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://repo.continuum.io/pkgs/r/noarch&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://repo.continuum.io/pkgs/r/noarch&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://repo.continuum.io/pkgs/pro/linux-64&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://repo.continuum.io/pkgs/pro/linux-64&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://repo.continuum.io/pkgs/pro/noarch&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://repo.continuum.io/pkgs/pro/noarch&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="2005" LastEditDate="2018-04-25T22:19:18.813" LastActivityDate="2018-04-25T22:19:18.813" Title="Seurat installation fails" Tags="|r|seurat|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 3.0" />
  <row Id="4149" PostTypeId="2" ParentId="4148" CreationDate="2018-04-25T22:03:25.490" Score="2" Body="&lt;p&gt;As @terdon mentioned in the comments, you need to &lt;code&gt;sudo yum install libicu&lt;/code&gt; on the relevant nodes (or have your admin do that for you). Alternatively, seurat can be installed via conda, which means you don't need root access.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-25T22:03:25.490" CommentCount="3" ContentLicense="CC BY-SA 3.0" />
  <row Id="4150" PostTypeId="1" CreationDate="2018-04-26T04:49:14.583" Score="2" ViewCount="475" Body="&lt;p&gt;I'm starting for protein expression analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In gene expression, we have TPM, RPKM etc. However, all I can see for protein expression is: &quot;High&quot;, &quot;Medium&quot;, &quot;Low&quot; and &quot;Not detected&quot;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://www.proteinatlas.org/ENSG00000140326-CDAN1/tissue&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.proteinatlas.org/ENSG00000140326-CDAN1/tissue&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TOng7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TOng7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q: Is there a unit for measuring protein expression like gene expression? I've seen people talking about PE (protein existence). Can we use it for protein expression?&lt;/p&gt;&#xA;" OwnerUserId="174" LastActivityDate="2018-04-26T09:41:00.430" Title="Protein expression units?" Tags="|proteins|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4151" PostTypeId="2" ParentId="4150" CreationDate="2018-04-26T06:53:50.623" Score="0" Body="&lt;p&gt;The unit of measurement in protein expression is more or less &quot;abundance&quot;. In mass spec this is derived from the area under the curve in the spectrum. In things like an ELISA this is derived from a color intensity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Protein existence is a vague summarization of multiple data sources to try to quantify the likelihood of a protein existing. You can't use that for expression, though expression would influence the likelihood of existence.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-26T06:53:50.623" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4152" PostTypeId="2" ParentId="4150" CreationDate="2018-04-26T09:41:00.430" Score="1" Body="&lt;p&gt;I would guess that the real unit for measuring protein expression would be pM (pico-moles) or a count of protein molecules.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure we have any methods to calculate such quantities though. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-04-26T09:41:00.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4153" PostTypeId="2" ParentId="4139" CreationDate="2018-04-26T14:40:45.757" Score="2" Body="&lt;p&gt;I remember a few options for pairwise genome visualisation (which may be outdated BTW):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.sanger.ac.uk/science/tools/artemis-comparison-tool-act&quot; rel=&quot;nofollow noreferrer&quot;&gt;ACT&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://bioinfo.lifl.fr/yass/index.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;YASS&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://alitvteam.github.io/AliTV/&quot; rel=&quot;nofollow noreferrer&quot;&gt;aliTV&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.bx.psu.edu/miller_lab/dist/README.lastz-1.02.00/README.lastz-1.02.00a.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;LASTZ&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="45" LastActivityDate="2018-04-26T14:40:45.757" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4154" PostTypeId="2" ParentId="4133" CreationDate="2018-04-26T14:47:40.373" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://plants.ensembl.org/Helianthus_annuus/Info/Index?db=core&quot; rel=&quot;nofollow noreferrer&quot;&gt;Helianthus annuus is an Ensembl plants genome&lt;/a&gt;. You should be able to use the &lt;a href=&quot;https://plants.ensembl.org/biomart/martview/&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioMart&lt;/a&gt; there to pull out genomic co-ordinates (chromosomes etc) for genes. However, searching for the gene names you posted does not return any results. The same is true of the two websites posted above. Are they real gene names or just examples you made up for the post?&lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-04-26T14:47:40.373" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4155" PostTypeId="2" ParentId="4139" CreationDate="2018-04-26T15:24:40.913" Score="4" Body="&lt;p&gt;After a bit of digging, finally found exactly what I was looking for. I'll share here in case someone else was looking for the same thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used a standard Nucleotide BLAST (blastn), coupled with &lt;a href=&quot;http://mummer.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MUMmer&lt;/a&gt;. Short tutorial. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that &lt;code&gt;xn&lt;/code&gt; &amp;amp; &lt;code&gt;yn&lt;/code&gt; are the couple of scaffolds I want to check if they are assembled together in the Pacbio assembly (a multi-fasta file):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) first, I blasted the pair of Illumina scaffolds (as separate queries) to the PacBio genome (as subject): &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;blastn -query ../illumina_asm/scaffold_x1.fa -subject ../pacbio_asm/genome.fa &amp;gt; out_scaffold_x1; blastn -query ../illumina_asm/scaffold_y1.fa -subject ../pacbio_asm/genome.fa &amp;gt; out_scaffold_y1&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The candidates PacBio scaffolds shall appear in the top blastn hits in both &lt;code&gt;out_scaffold_x1&lt;/code&gt; &amp;amp; &lt;code&gt;out_scaffold_y1&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume we found one common scaffold only - corresponding to the fasta entry &lt;code&gt;pacbio_scaffold&lt;/code&gt; in the PacBio genome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) extract the sequence of the target scaffold from the PacBio assembly, using &lt;a href=&quot;http://www.htslib.org/doc/faidx.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools faidx&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;samtools faidx ../pacbio_asm/genome.fa; samtools faidx ../pacbio_asm/genome.fa 'pacbio_scaffold' &amp;gt; pacbio_scaffold.fa&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) concatenate the Illumina scaffolds to analyze:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;cat ../illumina_asm/scaffold_x1.fa ../illumina_asm/scaffold_y1.fa &amp;gt; illumina_scaffolds.fa&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) compare the Illumina scaffolds to the PacBio assembly using &lt;a href=&quot;https://github.com/garviz/MUMmer/blob/master/nucmer&quot; rel=&quot;nofollow noreferrer&quot;&gt;nucmer&lt;/a&gt; from MUMmer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;nucmer --prefix=pacbio_scaffold pacbio_scaffold.fa illumina_scaffolds.fa&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;this will generate a &lt;code&gt;pacbio_scaffold.delta&lt;/code&gt; file, used in the next step&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5) filter the delta file to keep only the most meaningful alignments file using &lt;a href=&quot;https://github.com/garviz/MUMmer/blob/master/delta-filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;delta-filter&lt;/a&gt; from MUMmer (avoiding confusing messy plots later):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;delta-filter -m pacbio_scaffold.delta &amp;gt; pacbio_scaffold.delta.m&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;6) use &lt;a href=&quot;https://github.com/garviz/MUMmer/blob/master/mummerplot&quot; rel=&quot;nofollow noreferrer&quot;&gt;mummerplot&lt;/a&gt; from MUMmer to achieve a graph:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;mummerplot --png pacbio_scaffold.delta.m -R pacbio_scaffold.fa -Q illumina_scaffolds.fa --prefix=pacbio_scaffold -large -layout&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the x-axis, the PacBio scaffold they have in common. On the y-axis, the two Illumina scaffolds (separated by the horizontal grey abline). Colors correspond to the orientation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example of Illumina scaffolds linked in the PacBio assembly: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7Oinb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7Oinb.png&quot; alt=&quot;Example of Illumina scaffolds linked in the PacBio assembly&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2018-04-26T15:24:40.913" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4156" PostTypeId="1" AcceptedAnswerId="4158" CreationDate="2018-04-26T18:55:00.530" Score="1" ViewCount="635" Body="&lt;p&gt;I need to use &lt;code&gt;groupGO&lt;/code&gt; in &lt;code&gt;clusterProfiler&lt;/code&gt; to find functional profile of a list of genes, and I am having trouble finding out what some parameters of the function mean and which I should select for my specific case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The function is defined as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;groupGO(gene, OrgDb, keyType = &quot;ENTREZID&quot;, ont = &quot;CC&quot;, level = 2,&#xA;  readable = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have mouse genes dataset. What I should select for &lt;code&gt;ont&lt;/code&gt; and &lt;code&gt;level&lt;/code&gt; parameters? What does &lt;code&gt;ont&lt;/code&gt; types: &lt;code&gt;MF&lt;/code&gt;, &lt;code&gt;BP&lt;/code&gt;, &lt;code&gt;CC&lt;/code&gt; mean? What is &lt;code&gt;level&lt;/code&gt;? Where I could find this info? It is definitely not in the &lt;code&gt;vignette&lt;/code&gt;...&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-04-26T20:49:29.370" Title="groupGO parameters explanation (ont and level)" Tags="|rna-seq|groupgo|clusterprofiler|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4157" PostTypeId="1" AcceptedAnswerId="4159" CreationDate="2018-04-26T20:17:00.997" Score="2" ViewCount="3810" Body="&lt;p&gt;I am running &lt;code&gt;groupGO&lt;/code&gt; function from &lt;code&gt;clusterProfiler&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggo &amp;lt;- groupGO(gene     = head(gene, 200),&#xA;           OrgDb    = org.Mm.eg.db,&#xA;           keyType = &quot;ENSEMBL&quot;,&#xA;           ont      = &quot;BP&quot;,&#xA;           level    = 2,&#xA;           readable = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And getting the following output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(ggo)&#xA;                   ID            Description Count GeneRatio&#xA;GO:0000003 GO:0000003           reproduction     5     5/200&#xA;GO:0008152 GO:0008152      metabolic process    40    40/200&#xA;GO:0001906 GO:0001906           cell killing     0     0/200&#xA;GO:0002376 GO:0002376  immune system process    20    20/200&#xA;GO:0006791 GO:0006791     sulfur utilization     0     0/200&#xA;GO:0006794 GO:0006794 phosphorus utilization     0     0/200&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As can be seen &lt;code&gt;cell killing&lt;/code&gt;, &lt;code&gt;sulfur utilization&lt;/code&gt; and &lt;code&gt;phosphorus utilization&lt;/code&gt; have &lt;code&gt;zero&lt;/code&gt; genes. Then, why are they here at all? Should not the &lt;code&gt;groupGO&lt;/code&gt; show only &lt;code&gt;biological processes&lt;/code&gt; the fed in genes are related to? This seems very weird to me. What is the meaning of the &lt;code&gt;GeneRatio&lt;/code&gt; column?&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-04-26T21:05:52.277" Title="clusterProfiler, groupGO: meaning of GeneRatio" Tags="|rna-seq|groupgo|clusterprofiler|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4158" PostTypeId="2" ParentId="4156" CreationDate="2018-04-26T20:49:29.370" Score="1" Body="&lt;p&gt;As mentioned in the comments &lt;code&gt;MF&lt;/code&gt; is &lt;code&gt;molecular function&lt;/code&gt;, &lt;code&gt;BP&lt;/code&gt; is &lt;code&gt;biological process&lt;/code&gt; and &lt;code&gt;CC&lt;/code&gt; is &lt;code&gt;cellular compartment&lt;/code&gt;. These are the 3 domains of the ontologies. The &lt;code&gt;level&lt;/code&gt; refers to the level in the &lt;a href=&quot;http://geneontology.org/page/ontology-structure#graph&quot; rel=&quot;nofollow noreferrer&quot;&gt;ontology graph&lt;/a&gt;. In the example in that link, &lt;code&gt;pigmentation&lt;/code&gt; would be level 1, &lt;code&gt;pigmentation during development&lt;/code&gt; level 2, &lt;code&gt;regulation of pigmentation during development&lt;/code&gt; is level 3 and so on. You can find more discussion of this in &lt;a href=&quot;https://www.biostars.org/p/237816/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt; on biostars from a year ago.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the exact settings you should use, I don't personally find &lt;code&gt;CC&lt;/code&gt; to be an informative domain. The first couple levels are typically too generic to be useful and they'll include too many genes anyway for you to likely pick up any changes. I'd start around level 3 and see how things go (in other programs you would get all of the levels and domains at once).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-26T20:49:29.370" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4159" PostTypeId="2" ParentId="4157" CreationDate="2018-04-26T21:05:52.277" Score="2" Body="&lt;p&gt;As you seem to be learning, the clusterProfiler package is a bit &quot;special&quot;. It doesn't filter the results you're seeing for relevance, it returns everything. The &lt;code&gt;GeneRatio&lt;/code&gt; is the number of genes in your input list associated with the given GO term (e.g., &lt;code&gt;5&lt;/code&gt;) and then a &lt;code&gt;/&lt;/code&gt; and the total number of input genes. This is generally not that useful. Since many things aren't documented, you may find &lt;a href=&quot;https://github.com/GuangchuangYu/clusterProfiler/blob/6352683788121b8ff1a3dde3312fbe096e3f9f52/R/groupGO.R#L43-L55&quot; rel=&quot;nofollow noreferrer&quot;&gt;the source code&lt;/a&gt; helpful.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-04-26T21:05:52.277" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4160" PostTypeId="1" AcceptedAnswerId="4170" CreationDate="2018-04-27T09:23:13.497" Score="0" ViewCount="137" Body="&lt;p&gt;I would like to create a URL that takes input from different arrays to generate a search. The data I would like to retrieve is from NCBI. Below you can find the code I wrote. But I can't get it to work; I am stuck. Thank you in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should have something like this as result but i got nothing. &lt;a href=&quot;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=gene&amp;amp;retmode=xml&amp;amp;retmax=10&amp;amp;sort=relevance&amp;amp;term=A20%20AND%20Homo%20sapiens%20%5borgn%5d%20AND%20alive%5bprop%5d&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=gene&amp;amp;retmode=xml&amp;amp;retmax=10&amp;amp;sort=relevance&amp;amp;term=A20%20AND%20Homo%20sapiens%20%5borgn%5d%20AND%20alive%5bprop%5d&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-php prettyprint-override&quot;&gt;&lt;code&gt;$eutils = &quot;http://eutils.ncbi.nlm.nih.gov/entrez/eutils&quot;;&#xA;$db = &quot;gene&quot;;&#xA;$gene_symbols = array(&quot;A20&quot;,&quot;HBB&quot;,&quot;ABO&quot;);&#xA;$species = array(&quot;homo sapiens&quot;, &quot;Escherichia coli&quot;,&quot;Saccharomyces cerevisiae&quot;);&#xA;$links = array();&#xA;foreach ($gene_symbols as $key =&amp;gt; $GS){&#xA;    $SP = $species[$key];&#xA;    $query = &quot;$GS AND $SP [orgn] AND alive&quot;;&#xA;    $esearch = &quot;$eutils/esearch.fcgi?db=$db&amp;amp;retmode=xml&amp;amp;retmax=10&amp;amp;sort=relevance&amp;amp;term=&quot;;&#xA;    #echo $esearch;&#xA;    $links[] = get($esearch.$query);&#xA;}&#xA;foreach($links as $lk){&#xA;    xml = simplexml_load_file($lk) or die(&quot;feed not loading&quot;);&#xA;}&#xA;var_dump(xml);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As output I got nothing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I would like to have this: &lt;a href=&quot;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=gene&amp;amp;retmode=xml&amp;amp;retmax=10&amp;amp;sort=relevance&amp;amp;term=A20%20AND%20Homo%20sapiens%20%5borgn%5d%20AND%20alive%5bprop%5d&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=gene&amp;amp;retmode=xml&amp;amp;retmax=10&amp;amp;sort=relevance&amp;amp;term=A20%20AND%20Homo%20sapiens%20%5borgn%5d%20AND%20alive%5bprop%5d&lt;/a&gt;&#xA;And this for every gene. For some reason it put them all after each other and the foreach loop doesn't seem to work. I got no errors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I run this code in php using visual studio code. The result should be visualised in the webbrowser (&lt;a href=&quot;http://localhost/test.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://localhost/test.php&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="2736" OwnerDisplayName="nemo" LastEditorUserId="2719" LastEditDate="2018-05-01T08:49:17.097" LastActivityDate="2018-05-14T14:12:06.127" Title="Generating multiple NCBI URLs with eutils" Tags="|phylogenetics|" AnswerCount="3" CommentCount="7" ContentLicense="CC BY-SA 3.0" />
  <row Id="4161" PostTypeId="2" ParentId="4160" CreationDate="2018-04-27T11:35:25.430" Score="0" Body="&lt;p&gt;I don't know anything about php, but when I copy your code into &lt;a href=&quot;http://phptester.net&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; it isn't happy about the variable name &lt;code&gt;$query.&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This code appears to produce valid urls, but I can't make the &lt;code&gt;get&lt;/code&gt; part work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$eutils = &quot;http://eutils.ncbi.nlm.nih.gov/entrez/eutils&quot;; $db = &quot;gene&quot;;$gene_symbols = array(&quot;A20&quot;,&quot;HBB&quot;,&quot;ABO&quot;);&#xA;$species = array(&quot;homo sapiens&quot;, &quot;Escherichia coli&quot;,&quot;Saccharomyces cerevisiae&quot;); $links = array();&#xA;&#xA;foreach ($gene_symbols as $key =&amp;gt; $GS){ &#xA;   $SP = $species[$key];&#xA;   $SearchTerm = &quot;$GS AND $SP [orgn] AND alive&quot;;&#xA;&#xA;   $esearch = &quot;$eutils/esearch.fcgi?db=$db&amp;amp;retmode=xml&amp;amp;retmax=10&amp;amp;sort=relevance&amp;amp;term=&quot;;&#xA;&#xA;   echo $esearch.$SearchTerm;&#xA;}&#xA;&#xA;http://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=gene&amp;amp;retmode=xml&amp;amp;retmax=10&amp;amp;sort=relevance&amp;amp;term=A20 AND homo sapiens [orgn] AND alivehttp://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=gene&amp;amp;retmode=xml&amp;amp;retmax=10&amp;amp;sort=relevance&amp;amp;term=HBB AND Escherichia coli [orgn] AND alivehttp://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=gene&amp;amp;retmode=xml&amp;amp;retmax=10&amp;amp;sort=relevance&amp;amp;term=ABO AND Saccharomyces cerevisiae [orgn] AND alive&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-04-27T11:35:25.430" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4162" PostTypeId="1" AcceptedAnswerId="4164" CreationDate="2018-04-27T12:49:20.100" Score="3" ViewCount="1083" Body="&lt;p&gt;I did flow cytometry analysis of &lt;em&gt;E. coli&lt;/em&gt; cells 7 hours post inoculation (1% overnight culture in 3ml).&lt;/p&gt;&#xA;&lt;p&gt;I am analysing the FCS3.0 files using the flowCore package in R. I observe that FSC-A has negative values in some events, although the corresponding FSC-H values were all positive. These points comprised just a minor fraction of the data (~250/10000 in this case). The FSC voltage was set at 788V and the threshold flow rate was 5000-10000 events/sec.&lt;/p&gt;&#xA;&lt;p&gt;What do these negative values mean?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZbLyz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZbLyz.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;strong&gt;FSC-A vs FSC-H&lt;/strong&gt;: plotted only negative FSC-A values&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;br&gt;&#xA;&lt;hr&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dTPlk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dTPlk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;strong&gt;SSC-H vs FSC-A&lt;/strong&gt;: red points indicate negative FSC-A values&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="345" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-04-27T13:59:06.860" Title="What are negative FSC-A values in flow cytometry data?" Tags="|flow-cytometry|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4163" PostTypeId="1" CreationDate="2018-04-27T12:54:49.830" Score="1" ViewCount="1424" Body="&lt;p&gt;I've seen two uses of this term that seem to refer to completely different phenomena. Is my understanding correct?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In Jain et al. &quot;Active promoters give rise to false positive ‘Phantom Peaks’ in ChIP-seq experiments&quot; (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4538825/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PMC4538825&lt;/a&gt;), a phantom peak is a region on the genome where lots of reads align in a control experiment where there is none of the protein being IP'd, but not in a control experiment without any antibody. The authors (speculatively) attribute this to general stickiness of transcription factors and the transcriptional machinery, which can bind with potentially any antibody and get IP'd.&lt;/li&gt;&#xA;&lt;li&gt;The ENCODE quality metrics (&lt;a href=&quot;https://genome.ucsc.edu/ENCODE/qualityMetrics.html#chipSeq&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;) say &quot;a cross-correlation phantom-peak is ... observed at a strand-shift equal to the read length.&quot; They are referring to a high correlation of x[i] and y[i + r] where r is the read length, x[i] contains counts of tags aligning to the Watson strand at coordinate i, and y[i+r] contains counts of tags aligning to the Crick strand at coordinate i+r. They attribute this phenomenon to variations in mappability. (I don't understand their explanation at all, but that's a separate question.)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1708" LastActivityDate="2021-05-28T14:06:00.697" Title="What are phantom peaks in ChIP-seq?" Tags="|chip-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4164" PostTypeId="2" ParentId="4162" CreationDate="2018-04-27T13:19:26.303" Score="3" Body="&lt;p&gt;This is probably a matter of calibration. If this is not the raw data (actual voltage curves), look at that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an area below 0 is not possible, it has to be a calibration issue, either in the machine, or in the software. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;the -H plots are about the maximum height of the voltage spike in the side or forward light detector, while the -W plots are about the width (using some definition of flank-detection, so variation prone).&#xA;The area is also computed in various ways: using something like (H*W)/c (with c something like 2) or with an actual computation of the area of the spike, but again this has all kinds of predefinitions in it (what's the beginning/end of a  spike, what's the background-level, ...) Some of these predefinitions you can alter in software, some are hardcoded or even part of the functionality of the hardware, so it might be easy to impossible to get rid of the negative values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The FSC-H values corresponding to the FSC-A values are in your plot, and positive, so the most obvious solution (a bad algorithm that calculated Area as Height times Width times Magic and did not account for negative Heights (which depending on spike definition, you might get)) is out.&lt;/p&gt;&#xA;" OwnerUserId="2738" LastEditorUserId="2738" LastEditDate="2018-04-27T13:36:15.220" LastActivityDate="2018-04-27T13:36:15.220" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4166" PostTypeId="1" CreationDate="2018-04-27T14:58:11.940" Score="7" ViewCount="1924" Body="&lt;p&gt;I am working with WES data for detection of somatic variants and I have used two variant callers because no variant caller is complete in itself. I have used GATK Haplotypecaller for small variants like snp,ins,del etc. And for large variants I have used pindel (specially for SVs detection). Now I have two vcf file for each patients, one from gatk and another from pindel. Although I know various vcf-merging tools but is it right to merge two different vcf file from different callers. Do they give right and proper merged vcf file. Are those vcf-merging tools SV-aware and consider only trivial call for the same variant represented differently in different callers. I know these are too many questions for a single thread but all these are related to each other. I am in dilemma whether I should merge them or use them separately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="2141" LastEditorUserId="96" LastEditDate="2019-01-15T14:24:52.133" LastActivityDate="2022-06-21T09:13:18.043" Title="Merge 2 VCFs from different variant callers" Tags="|vcf|variant-calling|gatk|variation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4167" PostTypeId="2" ParentId="4166" CreationDate="2018-04-27T15:18:51.643" Score="4" Body="&lt;p&gt;The merge utility from &lt;a href=&quot;https://github.com/fritzsedlazeck/SURVIVOR&quot; rel=&quot;nofollow noreferrer&quot;&gt;SURVIVOR&lt;/a&gt; seems to be what you are looking for. I did not try it yet, but it seems to be especially designed to handle SV.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest thing to do would probably to first merge your patients files together for each tool and have one multi-patient file per variant caller. Then I would merge the multi-sample files of the different tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a detailed &lt;a href=&quot;http://simpsonlab.github.io/2015/06/15/merging-sv-calls/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt; describing the different challenges associated with merging structural variants from different callers and explaining the steps they took to address it. All the code to reproduce the pipeline is available on a &lt;a href=&quot;https://github.com/ljdursi/mergevcf/tree/master/mergevcf&quot; rel=&quot;nofollow noreferrer&quot;&gt;github repo&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="439" LastEditorUserId="439" LastEditDate="2018-04-28T09:42:52.040" LastActivityDate="2018-04-28T09:42:52.040" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4168" PostTypeId="1" AcceptedAnswerId="4171" CreationDate="2018-04-27T16:43:26.237" Score="0" ViewCount="447" Body="&lt;p&gt;I have downloaded KIRC Methylation data (450K) using TCGA toolbox. The beta values given are with reference to CPG probes. Is there any package or function in R through which I can get the gene names corresponding to these probes? I was able to get the gene I.Ds using &lt;code&gt;getMappedEntrezIDs&lt;/code&gt;. Is there any function to map these I.Ds to respective gene names?&lt;/p&gt;&#xA;" OwnerUserId="2247" LastEditorUserId="2247" LastEditDate="2018-04-27T17:06:38.213" LastActivityDate="2018-04-29T23:12:11.423" Title="Get gene names corresponding to CPG probes" Tags="|r|illumina|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4169" PostTypeId="1" AcceptedAnswerId="4183" CreationDate="2018-04-27T19:50:18.500" Score="2" ViewCount="86" Body="&lt;p&gt;I recently started using the Bioconductor AnnotationHub package for accessing lots of common datasets, but I've noticed that there are a lot of newer databases that have not made it into AnnotationHub's collection.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone had any luck getting databases added to AnnotationHub or adding them manually yourself (I guess via the package process?)?  Is it painful or relatively straightforward? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I love the annotationhub interface, but I'm wondering if it is way too inefficient to add new resources rather than just manually downloading and curating myself.  Thanks!&lt;/p&gt;&#xA;" OwnerUserId="39" LastActivityDate="2018-04-30T08:29:53.310" Title="How to add data to AnnotationHub" Tags="|bioconductor|sequence-annotation|public-databases|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4170" PostTypeId="2" ParentId="4160" CreationDate="2018-04-27T20:07:29.837" Score="0" Body="&lt;p&gt;I don't know PHP, so I'm sure there are better ways of doing this, but you had various issues in your code. This seems to be a working version of what you're after:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-php prettyprint-override&quot;&gt;&lt;code&gt;&amp;lt;?php&#xA;$eutils = &quot;http://eutils.ncbi.nlm.nih.gov/entrez/eutils&quot;; $db = &quot;gene&quot;; $gene_symbols = array(&quot;A20&quot;,&quot;HBB&quot;,&quot;ABO&quot;);&#xA;&#xA;$species = array(&quot;homo%20sapiens&quot;, &quot;Escherichia%20coli&quot;,&quot;Saccharomyces%20cerevisiae&quot;);&#xA;$links = array();&#xA;&#xA;foreach ($gene_symbols as $key =&amp;gt; $GS){ &#xA;  $SP = $species[$key];&#xA;  $query = &quot;$GS%20AND%20$SP%20[orgn]%20AND%20alive&quot;;&#xA;&#xA;  $esearch = &quot;$eutils/esearch.fcgi?db=$db&amp;amp;retmode=xml&amp;amp;retmax=10&amp;amp;sort=relevance&amp;amp;term=&quot;;&#xA;&#xA;  echo &quot;$esearch$query\n&quot;;&#xA;  $links[] = file_get_contents($esearch.$query);&#xA;  foreach($links as $lk){&#xA;    echo &quot;|------ $lk ----|&quot;;&#xA;    $xml = simplexml_load_string($lk) or die(&quot;feed not loading&quot;);&#xA;    var_dump($xml);&#xA;  }&#xA;}&#xA;?&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There doesn't seem to be a &lt;code&gt;get()&lt;/code&gt; function in PHP, you want &lt;code&gt;file_get_contents&lt;/code&gt;. Also, your loops were nested incorrectly and you needed &lt;code&gt;var_dump($xml)&lt;/code&gt; to dump the variable &lt;code&gt;$xml&lt;/code&gt; not &lt;code&gt;var_dump(xml);&lt;/code&gt; which tries to dump the string &lt;code&gt;xml&lt;/code&gt;. &lt;a href=&quot;https://stackoverflow.com/a/21661663/1081936&quot;&gt;You're not getting a file here but a string&lt;/a&gt;, so you need &lt;code&gt;simplexml_load_string&lt;/code&gt;. Finally, you can't have unescaped whitespace in URLs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next time, the way to debug this sort of thing is to have it print the URL it is generating and try to visit it manually. You would have seen the spaces and noticed that the working example you show uses &lt;code&gt;%20&lt;/code&gt; instead of space.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-04-27T20:07:29.837" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4171" PostTypeId="2" ParentId="4168" CreationDate="2018-04-28T00:45:17.663" Score="3" Body="&lt;p&gt;Here's a Python script (e.g., called &lt;code&gt;map_entrez_to_ensembl.py&lt;/code&gt; that uses &lt;a href=&quot;https://github.com/biothings/mygene.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;mygene.info&lt;/a&gt; to convert a list of Entrez IDs to Ensembl gene names:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;&#xA;import sys&#xA;import mygene&#xA;&#xA;mg = mygene.MyGeneInfo()&#xA;&#xA;genes = ['3815',&#xA;         '3816',&#xA;         '2341']&#xA;&#xA;results = mg.querymany(genes, scopes=['entrezgene'], fields=['ensembl'], species='human', verbose=False)&#xA;&#xA;for res in results:&#xA;    entrez_id = res['query']&#xA;    if 'ensembl' in res and 'gene' in res['ensembl']:&#xA;        sys.stdout.write('%s\t%s\n' % (entrez_id, res['ensembl']['gene']))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ./map_entrez_to_ensembl.py &#xA;3815    ENSG00000157404&#xA;3816    ENSG00000167748&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Looks like there's an &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/mygene.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;R-based interface to mygene.info&lt;/a&gt;, if you really need to use R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could use their &lt;a href=&quot;http://mygene.info/tryapi/&quot; rel=&quot;nofollow noreferrer&quot;&gt;API test page&lt;/a&gt; to see how data are structured and what keywords are available for different kinds of searches. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, if you wanted aliases, in addition to Ensembl names:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;&#xA;import sys&#xA;import mygene&#xA;&#xA;mg = mygene.MyGeneInfo()&#xA;&#xA;genes = ['3815',&#xA;         '3816',&#xA;         '2341']&#xA;&#xA;results = mg.querymany(genes, scopes=['entrezgene'], fields=['ensembl', 'alias'], species='human', verbose=False)&#xA;&#xA;for res in results:&#xA;    entrez_id = res['query']&#xA;    if 'ensembl' in res and 'gene' in res['ensembl'] and 'alias' in res:&#xA;        for alias in res['alias']:&#xA;            sys.stdout.write('%s\t%s\t%s\n' % (entrez_id, alias, res['ensembl']['gene']))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ./map_entrez_to_alias_and_ensembl.py&#xA;3815    C-Kit   ENSG00000157404&#xA;3815    CD117   ENSG00000157404&#xA;3815    PBT     ENSG00000157404&#xA;3815    SCFR    ENSG00000157404&#xA;3816    KLKR    ENSG00000167748&#xA;3816    Klk6    ENSG00000167748&#xA;3816    hK1     ENSG00000167748&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-04-29T23:12:11.423" LastActivityDate="2018-04-29T23:12:11.423" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4172" PostTypeId="1" AcceptedAnswerId="8897" CreationDate="2018-04-28T10:07:40.070" Score="1" ViewCount="240" Body="&lt;p&gt;I have some protein sequences and I need to predict the physical distance between each two residues in the protein when folded correctly in 3 dimensions; I need to predict this distances just by having the protein sequence and not the structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a group of pdb IDs and I have got homologs for them and made a multiple sequence alignment for each query sequence; But later I need to imagine that I don't have any structure for being able to predict the distance between residues in query sequences without any known structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some articles for solving this issue using machine learning methods but I don't want to train with a large numbers of structures. Any one knows any way to solve this problem by using mathematical or statistical methods?&lt;/p&gt;&#xA;" OwnerUserId="818" LastEditorUserId="818" LastEditDate="2018-04-29T16:30:48.873" LastActivityDate="2019-07-02T18:13:53.690" Title="How to predict the distance between two residues in a protein sequence" Tags="|proteins|sequence-analysis|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 3.0" />
  <row Id="4174" PostTypeId="1" CreationDate="2018-04-29T00:14:28.397" Score="1" ViewCount="2654" Body="&lt;p&gt;While running &lt;code&gt;cellranger count&lt;/code&gt; using the following &lt;code&gt;.slurm&lt;/code&gt; file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#SBATCH -p standard&#xA;#SBATCH -A overall &#xA;#SBATCH --time=48:00:00&#xA;#SBATCH --mail-type=end&#xA;#SBATCH --mail-user=nv4e@virginia.edu&#xA;#SBATCH --output=generate_count_%j.out&#xA;#SBATCH --error=generate_count_%j.err&#xA;#SBATCH --nodes=1&#xA;#SBATCH --ntasks=1&#xA;#SBATCH --cpus-per-task=20&#xA;#SBATCH --mem=120000&#xA;&#xA;&#xA;# Run driver program&#xA;FASTQS=&quot;/scratch/nv4e/kipnis/AHN3LLBGX5/outs/fastq_path/HN3LLBGX5&quot;&#xA;TR=&quot;/scratch/nv4e/kipnis/refdata-cellranger-mm10-1.2.0&quot;&#xA;&#xA;cellranger count --id=17R_1 --transcriptome=${TR} --fastqs=${FASTQS} --sample=17R --expect-cells=2000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I got the error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;2018-04-28 20:04:37 [runtime] (progress)        &#xA;ID.17R_1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.CHEMISTRY_DETECTOR.DETECT_CHEMI&#xA;STRY.fork0.chnk0: Indexing genome...&#xA;2018-04-28 20:04:49 [runtime] (progress)        ID.17R_1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.CHEMISTRY_DETECTOR.DETECT_CHEMISTRY.fork0.chnk0: Building transcriptome...&#xA;2018-04-28 20:04:58 [runtime] (progress)        ID.17R_1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.CHEMISTRY_DETECTOR.DETECT_CHEMISTRY.fork0.chnk0: Building kmer index...&#xA;2018-04-28 20:05:25 [runtime] (progress)        ID.17R_1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.CHEMISTRY_DETECTOR.DETECT_CHEMISTRY.fork0.chnk0: Mapping reads...&#xA;2018-04-28 20:05:28 [runtime] (chunks_complete) ID.17R_1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.CHEMISTRY_DETECTOR.DETECT_CHEMISTRY&#xA;2018-04-28 20:05:28 [runtime] (run:local)       ID.17R_1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.CHEMISTRY_DETECTOR.DETECT_CHEMISTRY.fork0.join&#xA;2018-04-28 20:05:31 [runtime] (join_complete)   ID.17R_1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.CHEMISTRY_DETECTOR.DETECT_CHEMISTRY&#xA;&#xA;2018-04-28 20:05:34 [runtime] (ready)           ID.17R_1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.SETUP_CHUNKS&#xA;2018-04-28 20:05:37 [runtime] (split_complete)  ID.17R_1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.SETUP_CHUNKS&#xA;2018-04-28 20:05:37 [runtime] (run:local)       ID.17R_1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.SETUP_CHUNKS.fork0.chnk0.main&#xA;2018-04-28 20:05:40 [runtime] (failed)          &#xA;ID.17R_1.SC_RNA_COUNTER_CS.SC_RNA_COUNTER.SETUP_CHUNKS&#xA;&#xA;&amp;gt;[error] The chemistry was unable to be automatically determined. This can happen if not enough reads originate from the given reference. Please verify your choice of reference or explicitly specify the chemistry via the --chemistry argument&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is going wrong here? How could I correct it?&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="77" LastEditDate="2018-04-29T06:06:11.000" LastActivityDate="2018-05-03T18:10:42.313" Title="10X cellranger count, [error] The chemistry was unable to be automatically determined" Tags="|10x-genomics|cellranger|slurm|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4176" PostTypeId="5" CreationDate="2018-04-29T16:04:04.953" Score="0" Body="&lt;p&gt;A gene is a sequence of DNA or RNA that codes for a molecule that has a function. During gene expression, the DNA is first copied into RNA. The RNA can be directly functional or be the intermediate template for a protein that performs a function.&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-07-31T11:07:38.440" LastActivityDate="2018-07-31T11:07:38.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4177" PostTypeId="4" CreationDate="2018-04-29T16:04:04.953" Score="0" Body="Use this tag to refer to the gene sequence/coding part of the sequence. " OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-07-31T14:21:54.727" LastActivityDate="2018-07-31T14:21:54.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4178" PostTypeId="5" CreationDate="2018-04-29T16:07:37.397" Score="0" Body="&lt;p&gt;DNA sequencing includes any method or technology that is used to determine the order of the four bases—adenine, guanine, cytosine, and thymine in a strand of DNA. Popular sequencing methods include sanger sequencing and sequencing-by-synthesis (SBS).&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="73" LastEditDate="2018-07-31T11:09:54.293" LastActivityDate="2018-07-31T11:09:54.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4179" PostTypeId="4" CreationDate="2018-04-29T16:07:37.397" Score="0" Body="DNA sequencing is the process of determining the precise order of nucleotides within a DNA molecule." OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-08-02T09:07:59.317" LastActivityDate="2018-08-02T09:07:59.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4180" PostTypeId="1" AcceptedAnswerId="4198" CreationDate="2018-04-30T00:39:49.513" Score="2" ViewCount="379" Body="&lt;p&gt;I am running &lt;code&gt;gseGO&lt;/code&gt; function from &lt;code&gt;clusterProfiler&lt;/code&gt; and getting an output where terms are ordered by the &lt;code&gt;setSize&lt;/code&gt; parameter. What does this ranking mean? Are terms ordered by the sizes of the gene sets? If yes, then what is the ultimate value of such ranking? My goal is to get the most relevant terms, but I am not completely sure that such an ordering based on the &lt;code&gt;setSize&lt;/code&gt; is actually the right one.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="29" LastEditDate="2018-05-03T12:31:38.300" LastActivityDate="2018-05-03T12:31:38.300" Title="What does the setSize ordering mean?" Tags="|r|clusterprofiler|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4181" PostTypeId="2" ParentId="3062" CreationDate="2018-04-30T05:55:29.227" Score="3" Body="&lt;p&gt;Although a bit late, here is the solution for this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Create color/column levels:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;levels = unlist(lapply(colnames(trn[,1:16]), function(x)&#xA;    {&#xA;&#xA;        if(x == &quot;H1&quot;) return(&quot;H1&quot;)&#xA;        else if(x == &quot;H2&quot;) return(&quot;H2&quot;)&#xA;&#xA;&#xA;}))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;2) &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plot(tsne$Y, col=factor(levels))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This should color columns H1 and H2 differently&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a sample complete code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Rtsne)&#xA;file1&amp;lt;- read.csv('filename.csv',header = T, row.names=1)&#xA;trn &amp;lt;- data.matrix(file1)&#xA;dim(trn)&#xA;&#xA;tsne &amp;lt;- Rtsne(as.matrix(trn[,1:16]), check_duplicates = FALSE, pca = FALSE, perplexity=5, theta=0.5, dims=3)&#xA;&#xA;collevels = unlist(lapply(colnames(trn), function(x)&#xA;{&#xA;  if(grepl(&quot;H1&quot;,x))&#xA;    return(&quot;blue&quot;)&#xA;  else if(grepl(&quot;H2&quot;,x))&#xA;    return(&quot;red&quot;)&#xA;&#xA;  # etc ... for all the columns you want to color&#xA;}&#xA;))&#xA;&#xA;colorColumns = factor(collevels)&#xA;require(rgl)&#xA;plot3d(tsne$Y[,1], tsne$Y[,2], tsne$Y[,3], col=colorColumns)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2743" LastEditorUserId="2743" LastEditDate="2018-05-02T15:19:26.797" LastActivityDate="2018-05-02T15:19:26.797" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4182" PostTypeId="1" CreationDate="2018-04-30T07:02:49.857" Score="1" ViewCount="39" Body="&lt;p&gt;Given a known interaction of a molecule with some proteases (having X-rays images too) is it possible to predict interactions with other proteins either using their 3D structure or not? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone has already tried such techniques I would like to hear a little brief on that subject. Which are the possible techniques, computational power needed and so on.&lt;/p&gt;&#xA;" OwnerUserId="704" LastActivityDate="2019-03-25T11:29:04.333" Title="Finding new drug-protein targets/interactions using a known one as base" Tags="|protein-structure|interactions|drugs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4183" PostTypeId="2" ParentId="4169" CreationDate="2018-04-30T08:29:53.310" Score="2" Body="&lt;p&gt;As the description of the package says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The AnnotationHub web resource provides a central location where&#xA;  genomic files (e.g., VCF, bed, wig) and other resources from standard&#xA;  locations (e.g., UCSC, Ensembl) can be discovered.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If the new resources aren't in the standard locations it won't be listed by AnnotationHub. The easy/common way to have the data accessible through AnnotationHub is make it available through these standard locations (before publications if you want to give early access).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another method would be to follow this steps of &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/AnnotationHub/inst/doc/CreateAnAnnotationPackage.html#new-resources&quot; rel=&quot;nofollow noreferrer&quot;&gt;the vignette&lt;/a&gt; to add new resources, but it might involve creating a new package.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-04-30T08:29:53.310" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4185" PostTypeId="1" AcceptedAnswerId="4186" CreationDate="2018-04-30T13:16:19.487" Score="3" ViewCount="186" Body="&lt;p&gt;I'm using this command (excuse the duplicate naming, I know it's bad form):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blastn -query cm-seqs/combined_seqs.fna -db combined_seqs.fna -out cm-matched.txt -num_alignments 1 -outfmt 10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to take a set of fasta sequences to blast against another data base I built with blast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My query file has 4,364,417 sequences. The files resulting from the above command spat out 4,362,639 results. The difference is about 1800. Is this due to poor alignments for some sequences that didn't pass any of blasts default parameters? I assumed the result would be 1 &quot;best&quot; match in the database for each query sequence.&lt;/p&gt;&#xA;" OwnerUserId="2747" LastEditorUserId="73" LastEditDate="2018-04-30T20:27:40.713" LastActivityDate="2018-04-30T20:27:40.713" Title="Will blastn remove sequences from a search with low identity?" Tags="|blast|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 3.0" />
  <row Id="4186" PostTypeId="2" ParentId="4185" CreationDate="2018-04-30T13:20:12.097" Score="1" Body="&lt;p&gt;Yes, if a query sequence has no good alignment, blast will not return a result for it. Your command will indeed output only the best hit for each query sequence, but there is no reason to assume &lt;em&gt;all&lt;/em&gt; of your sequences will have a match. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those that don't have a match won't be included in the output. This sounds perfectly normal unless you are blasting the same input file against a DB created from that file. Otherwise, you don't expect all of your sequences to necessarily have a match. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note that since the default blastn settings are quite permissive (usually an e-value of 10, for example), don't make the mistake of assuming that a blast hit means anything. The details will always depend on the specific search you are running (I have discussed this a little in my answer &lt;a href=&quot;https://biology.stackexchange.com/a/19356/1306&quot;&gt;here&lt;/a&gt;), but in the vast majority of cases, any hit with an e-value &gt; 1 can safely be ignored. In most cases, you are only interested in e-values well under 1, actually. So you probably already have all sorts of spurious results in there anyway. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more details, please edit your question and explain what the query and DB files are.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-04-30T14:49:03.437" LastActivityDate="2018-04-30T14:49:03.437" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4187" PostTypeId="1" AcceptedAnswerId="4188" CreationDate="2018-04-30T16:05:33.093" Score="4" ViewCount="105" Body="&lt;p&gt;I'm working on a project to identify variants as being either germline or somatic based on nearby SNPs, and I was wondering if there were any databases of SNPs for Homo sapiens other than dbSNP? I'm not looking for databases that contain additional information (pathogenicity etc.) on SNPs that are already in dbSNP, instead I want to know if there are any other databases that contain SNPs that aren't found in dbSNP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been searching online and so far the only databases I can find are either deprecated or integrated into dbSNP, so any suggestions would be great.&lt;/p&gt;&#xA;" OwnerUserId="885" LastActivityDate="2018-04-30T16:30:56.470" Title="Human SNP databases other than dbSNP" Tags="|database|public-databases|snp|variant-calling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4188" PostTypeId="2" ParentId="4187" CreationDate="2018-04-30T16:30:56.470" Score="3" Body="&lt;p&gt;I'm pretty sure there are no public databases of variants that aren't included in dbSNP. That is &lt;em&gt;the&lt;/em&gt; variant database so that's the one you want to use. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While there are various other databases around, they tend to use dbSNP rsIDs as well so any variants in those will also be in dbSNP. So while dbSNP is nowhere near complete, of course, it's still the most complete one you're likely to find.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-04-30T16:30:56.470" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4189" PostTypeId="1" AcceptedAnswerId="4194" CreationDate="2018-04-30T16:34:56.247" Score="2" ViewCount="201" Body="&lt;p&gt;I have two csv file - one is a list of strings and other has both strings and floating values. If the strings in both the files match I need to copy the complete row from the second file to a temporary array. I have declared a matrix with n rows and m columns and saved is as dataframe &lt;code&gt;temp &amp;lt;- as.data.frame(matrix(nrow = n, ncol = m))&lt;/code&gt;. I am using &lt;code&gt;for&lt;/code&gt; loop to iterate over all rows and &lt;code&gt;strcmp&lt;/code&gt; to compare the strings. But when the data is written to &lt;code&gt;temp&lt;/code&gt; the strings are stored as integers. How should I declare this array? Is it possible to write this loop without declaring an array in R?&lt;/p&gt;&#xA;" OwnerUserId="2247" LastActivityDate="2018-04-30T20:13:12.490" Title="Declaring an array of strings and integers" Tags="|r|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4190" PostTypeId="1" CreationDate="2018-04-30T17:44:53.723" Score="2" ViewCount="281" Body="&lt;p&gt;I want to convert names according to the &quot;cluster of differentiation&quot; nomenclature into accepted gene symbols. For example, CD3 should map to &quot;CD3a;CD3b;CD3c;CD247&quot; and &quot;CD45RO&quot; should map to &quot;PTPRC&quot;. If the surface antigen isn't coded for by a gene, it should map to &quot;&quot; or something similar. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've seen &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2655/obtain-cd-cluster-of-differentiation-marker-expression-information&quot;&gt;Obtain CD (cluster of differentiation) marker expression information&lt;/a&gt;, but the Abcam table used there does not give the gene symbol -- it gives a list of alternate names. I've seen table 1 of &lt;a href=&quot;http://www.jimmunol.org/content/195/10/4555&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, but it contains errors -- for example, TNFRSF11 should be TNFRSF11A. I could copy &lt;a href=&quot;http://www.hcdm.org/index.php/molecule-information&quot; rel=&quot;nofollow noreferrer&quot;&gt;this table&lt;/a&gt; by hand, or I could download &lt;a href=&quot;https://www.genenames.org/cgi-bin/genefamilies/set/471&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;, though that's annoying because it doesn't have one row per CD. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a canonical (or just a convenient and reliable) way to do this?&lt;/p&gt;&#xA;" OwnerUserId="1708" LastActivityDate="2018-04-30T18:07:31.693" Title="Convert CD (cluster of differentiation) into gene symbol" Tags="|public-databases|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4191" PostTypeId="2" ParentId="4190" CreationDate="2018-04-30T18:07:31.693" Score="2" Body="&lt;p&gt;I ended up using option 4 and parsing the columns for &quot;Synonym&quot; and &quot;Previous Symbol&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.genenames.org/cgi-bin/genefamilies/set/471&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.genenames.org/cgi-bin/genefamilies/set/471&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1708" LastActivityDate="2018-04-30T18:07:31.693" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4192" PostTypeId="1" AcceptedAnswerId="4195" CreationDate="2018-04-30T18:29:08.397" Score="5" ViewCount="405" Body="&lt;p&gt;I'm currently updating my Variant Calling Pipeline by switching the VCF annotating software from Annovar to VEP for a variety of regions, not least how easy it is to annotate with HGVS notation and keep datasets up to date in VEP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the most part everything is running smoothly, with the exception that some of the data in the VCFs is lost during annotation (and conversion to tsv). The VCFs are created with GATK's UnifiedGenotyper and include a 'Format' column where each value is 'GT:AD:DP:GQ:PL' and a column named after the Individual, which contains semicolon-separated data that corresponds to the Format column (i.e. Genotype;Allele Depth;Depth;Genotype Quality;Phred-likelihood). When I annotate with VEP none of this data is carried over to the output file as it would be in Annovar, leaving me with an annotated file that has no information on read depth, genotype or any of the other data in the two lost columns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've included the command I'm currently using for annotation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./vep -i RM0108.vcf --cache --force_overwrite --tab --merged --variant_class --sift b --polyphen b --hgvs --symbol --canonical --check_existing --af_1kg --af_gnomad --humdiv --pick -o RM0108.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can't find information on this in the VEP documentation or elsewhere online. I could write something to take the relevant information from the VCF and add it to the tsv after VEP has finished running, but it seems like there may be an easier solution that I'm missing, so any help would be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also posted this question on BioStars &lt;a href=&quot;https://www.biostars.org/p/312290/&quot; rel=&quot;noreferrer&quot;&gt;Link Here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="885" LastActivityDate="2018-05-01T07:49:32.660" Title="Keep Format and Individual fields when annotating VCF with VEP" Tags="|vcf|sequence-annotation|variant-calling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4193" PostTypeId="2" ParentId="4189" CreationDate="2018-04-30T19:00:10.517" Score="1" Body="&lt;p&gt;When you read in your two CSV files, you can explicitly declare the variable types for each column:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; a &amp;lt;- read.csv(..., colClasses=c(&quot;character&quot;), ...)&#xA;&amp;gt; b &amp;lt;- read.csv(..., colClasses=c(&quot;character&quot;, &quot;numeric&quot;, &quot;numeric&quot;, ...), ...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You could also investigate &lt;code&gt;as.is&lt;/code&gt; or &lt;code&gt;stringsAsFactors&lt;/code&gt;, although this can cause problems where there are duplicate strings. Using &lt;code&gt;colClasses&lt;/code&gt; is more explicit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Read the help documentation in &lt;code&gt;?read.csv&lt;/code&gt; for a more complete explanation of these operators.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can filter with the &lt;code&gt;%in%&lt;/code&gt; operator in &lt;code&gt;match&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; strings_of_interest &amp;lt;- as.vector(a$V1)&#xA;&amp;gt; b.filtered &amp;lt;- b[b$V1 %in% strings_of_interest,]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I am reading your question correctly, perhaps you can then take &lt;code&gt;b.filtered&lt;/code&gt; and turn that into the matrix you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Avoid use of &lt;code&gt;for&lt;/code&gt; loops where you can. They can slow R down considerably. A good read on this subject is in the third chapter of Patrick Burns' &lt;a href=&quot;https://www.burns-stat.com/pages/Tutor/R_inferno.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The R Inferno&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-04-30T19:05:50.140" LastActivityDate="2018-04-30T19:05:50.140" CommentCount="2" ContentLicense="CC BY-SA 3.0" />
  <row Id="4194" PostTypeId="2" ParentId="4189" CreationDate="2018-04-30T20:13:12.490" Score="1" Body="&lt;p&gt;The &lt;code&gt;merge&lt;/code&gt; function of R might be useful. The function expects two data frames, with [optionally] columns to compare, and it will output all fields from both data frames where the comparison columns match.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; a &amp;lt;- data.frame(test=1:10);&#xA;&amp;gt; b &amp;lt;- data.frame(test=c(sample(1:10,5),sample(c(&quot;A&quot;,&quot;B&quot;,&quot;C&quot;,&quot;D&quot;,&quot;E&quot;))),&#xA;                  output=rnorm(10));&#xA;&amp;gt; merge(a, b, by=&quot;test&quot;);&#xA;  test     output&#xA;1    1 -1.3800443&#xA;2    2  0.2934274&#xA;3    3  0.0868700&#xA;4    9  1.1766914&#xA;5   10  0.3793448&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;merge&lt;/code&gt; function can be customised quite a bit (e.g. if you want NA set for non-matching rows), see &lt;code&gt;?merge&lt;/code&gt; for more details.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-04-30T20:13:12.490" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4195" PostTypeId="2" ParentId="4192" CreationDate="2018-05-01T07:49:32.660" Score="4" Body="&lt;p&gt;The VEP TSV format can only keep in its own specified columns. If you want to maintain the data from your original input, get your output in VCF. It will add the VEP annotation to the INFO column, and keep all the stuff you already have there.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2018-05-01T07:49:32.660" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4196" PostTypeId="1" AcceptedAnswerId="15891" CreationDate="2018-05-01T09:04:28.880" Score="5" ViewCount="315" Body="&lt;p&gt;I have very noisy nanopore reads and am trying to call SNPs/indels.&lt;/p&gt;&#xA;&lt;p&gt;I'm runinig into some trouble when truing to use the &lt;code&gt;samtools mpileup | bcftools call&lt;/code&gt; combination.&lt;/p&gt;&#xA;&lt;p&gt;It seems to incorrectly be calling very long indels where it looks like there is no support. For example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;samtools mpileup -f ref.fa sample.bam -r Chromosome:198940-198940&lt;/code&gt; produces:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Chromosome    198940    C    37    .,.-1A.-1A...-1A.-1A.-1A.,-1a.-1A.,-1a,-1a.,.-1A,,.-1A,,+1a,+1a,.-1A,,-1a.,-1a,-1a,,-1a,-1a.-1A,-1a.    06?5/&amp;lt;:8&amp;gt;/4&amp;lt;?3691465&amp;lt;3354.08:23:11@3C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This looks fine... however when I try to call the genotype with bcftools I get a very long indel.&#xA;&lt;code&gt;samtools mpileup -ugf ref.fa sample.bam -r Chromosome:198940-198940 -t AD | bcftools call -mv&lt;/code&gt; produces:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Chromosome    198940    .    CAAAAGACGTCATCGACGTACGGTCGGTGACCACCGAGATCAACACGTTGTTCCAGACGCTCACCTCGATCGCCGA    C    225    .    INDEL;IDV=32;IMF=0.5;DP=62;VDB=0.145385;SGB=-0.662043;MQSB=1;MQ0F=0;AC=2;AN=2;DP4=0,0,7,2;MQ=60    GT:PL:AD    1/1:255,27,0:0,9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Using &lt;code&gt;htsbox pileup -f ref.fa sample.bam | grep 198940&lt;/code&gt; it shows there is one read with this indels together with many other reads with smaller indels.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Chromosome  198940  C   C,C-75AAAAGACGTCATCGACGTACGGTCGGTGACCACCGAGATCAACACGTTGTTCCAGACGCTCACCTCGATCGCCGA,C-4AAAA,C-2AA,C-1A,T,C+1A,C+1T,C+3GGT 0/4:25,1,1,2,24,5,2,1,1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there any reason why it would be grouping these reads together into the big indel?&lt;/p&gt;&#xA;&lt;p&gt;Any idea what is going wrong here?&lt;/p&gt;&#xA;" OwnerUserId="2750" LastEditorUserId="73" LastEditDate="2021-05-10T05:45:48.877" LastActivityDate="2021-06-09T06:00:25.113" Title="Samtools/bcftools calling indels in noisy reads" Tags="|samtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4197" PostTypeId="1" AcceptedAnswerId="4200" CreationDate="2018-05-01T14:41:29.730" Score="4" ViewCount="372" Body="&lt;p&gt;I have a file with protein sequences, all on a single line, like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Contig_1 1FFPMPQ*CCLFSSSYLPA*AIMVTGSCQTLVQDKV*CLASCHVMIPPCVFHPIYPLFTWSPLHPIISGCF*AVCMCSWIHALFVEIILSWSLSLPFTFEISSD&amp;gt;Contig_2 1FFPCHNSVVYFLLLICLLKPLWSQGVVRHWYRTRCDAWLAVML*YLHVSFIPSIHFLLGRLCTLLYLDVFELYACAAGFMPCLLR*FSLGHSLFLLPLKFPATVKFFSLLGQIMNNISLMNDLQTGEKLQETH*EVEDKSALLVSIIESRCCVTLQPQDFSN*HLGKL*KSVGQKLK*VNDQITY*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to print all sequences between an &lt;code&gt;M&lt;/code&gt; residue and a &lt;code&gt;*&lt;/code&gt; that are at least 6 residues long, and I would like the output like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;contig_1.1.1 &#xA;M.......*&#xA;&amp;gt;contig_1.1.2 &#xA;M........*&#xA;&amp;gt;contig_2.1.1 &#xA;M............*&#xA;&amp;gt;contig_2.1.2 &#xA;M......*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where the first number after the underscore is the number found in as the first character of the sequence (it is the reading frame) and the second number is the sequence number. and the last number is the fragment number. So, since the 1st sequence starts with;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Contig_1 1FFPMPQ&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The name will be &lt;code&gt;&amp;gt;Contig_1.1&lt;/code&gt; and each fragment found will be &lt;code&gt;&amp;gt;Contig_1.1.1&lt;/code&gt;, &lt;code&gt;&amp;gt;Contig_1.1.2&lt;/code&gt;, ... , &lt;code&gt;&amp;gt;Contig_1.1.N&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After implementing the suggestion in the answer below and escaping the &lt;code&gt;*&lt;/code&gt; in my regular expression, my current code is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import sys&#xA;import re&#xA;&#xA;myprotein = &quot;&amp;gt;Contig_1 1FFPMPQ*CCLFSSSYLPA*AIMVTGSCQTLVQDKV*CLASCHVMIPPCVFHPIYPLFTWSPLHPIISGCF*AVCMCSWIHALFVEIILSWSLSLPFTFEISSD&amp;gt;Contig_2 1FFPCHNSVVYFLLLICLLKPLWSQGVVRHWYRTRCDAWLAVML*YLHVSFIPSIHFLLGRLCTLLYLDVFELYACAAGFMPCLLR*FSLGHSLFLLPLKFPATVKFFSLLGQIMNNISLMNDLQTGEKLQETH*EVEDKSALLVSIIESRCCVTLQPQDFSN*HLGKL*KSVGQKLK*VNDQITY*&quot;&#xA;# parse record&#xA;my_protein = []&#xA;for line in myprotein:&#xA;    if not line.startswith('&amp;gt;'):&#xA;        my_protein.append(line)&#xA;    else:&#xA;        header = line&#xA;&#xA;# build sequence&#xA;my_pro = ''.join(my_protein)&#xA;&#xA;# cleave sequence                                                                                                                                                                                                                                                        &#xA;peptides = re.findall(r'(M[A-Z]{6,}\*)', my_pro)&#xA;&#xA;# build a formatted list&#xA;records = [&quot;%sNo.%d\n%s\n&quot; % (header, idx + 1, fragment) for idx, fragment in     enumerate(peptides)]&#xA;&#xA;# print list&#xA;for record in records:&#xA;    sys.stdout.write(&quot;%s&quot; % (record))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And it prints:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; &amp;gt;No.1&#xA; MVTGSCQTLVQDKV*&#xA; &amp;gt;No.2&#xA; MIPPCVFHPIYPLFTWSPLHPIISGCF*&#xA; &amp;gt;No.3&#xA; MNNISLMNDLQTGEKLQETH*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I change this to get my desired output format?&lt;/p&gt;&#xA;" OwnerUserId="2523" LastEditorUserId="29" LastEditDate="2018-05-03T11:39:10.880" LastActivityDate="2018-05-03T11:39:10.880" Title="Finding proteins in a sequence" Tags="|fasta|python|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4198" PostTypeId="2" ParentId="4180" CreationDate="2018-05-01T14:54:55.613" Score="0" Body="&lt;p&gt;As you can see on &lt;a href=&quot;https://guangchuangyu.github.io/2016/01/go-analysis-using-clusterprofiler/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog&lt;/a&gt; post of the author, the result is ordered by &lt;code&gt;pvalue&lt;/code&gt;, but it can coincidentally sort by &lt;code&gt;setSize&lt;/code&gt;. The &lt;code&gt;setSize&lt;/code&gt; means that that term has so many genes on under that category. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-05-01T14:54:55.613" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4200" PostTypeId="2" ParentId="4197" CreationDate="2018-05-01T15:45:54.247" Score="2" Body="&lt;p&gt;The &lt;code&gt;*&lt;/code&gt; character has a special meaning in regular expressions, it means &quot;0 or more&quot;. So when you write &lt;code&gt;{6,}*&lt;/code&gt;, the regex engine tries to apply the &quot;0 or more&quot; to the &lt;code&gt;{6,*}&lt;/code&gt; and gets confused:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &quot;&amp;lt;stdin&amp;gt;&quot;, line 1, in &amp;lt;module&amp;gt;&#xA;  File &quot;/usr/lib/python3.6/re.py&quot;, line 222, in findall&#xA;    return _compile(pattern, flags).findall(string)&#xA;  File &quot;/usr/lib/python3.6/re.py&quot;, line 301, in _compile&#xA;    p = sre_compile.compile(pattern, flags)&#xA;  File &quot;/usr/lib/python3.6/sre_compile.py&quot;, line 562, in compile&#xA;    p = sre_parse.parse(p, flags)&#xA;  File &quot;/usr/lib/python3.6/sre_parse.py&quot;, line 855, in parse&#xA;    p = _parse_sub(source, pattern, flags &amp;amp; SRE_FLAG_VERBOSE, 0)&#xA;  File &quot;/usr/lib/python3.6/sre_parse.py&quot;, line 416, in _parse_sub&#xA;    not nested and not items))&#xA;  File &quot;/usr/lib/python3.6/sre_parse.py&quot;, line 619, in _parse&#xA;    source.tell() - here + len(this))&#xA;sre_constants.error: multiple repeat at position 10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In order to use special characters literally, as simple strings, you need to escape them by adding a &lt;code&gt;\&lt;/code&gt; before them. So try:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;import re&#xA;myprotein = &quot;&amp;gt;Contig_1 1FFPMPQ*CCLFSSSYLPA*AIMVTGSCQTLVQDKV*CLASCHVMIPPCVFHPIYPLFTWSPLHPIISGCF*AVCMCSWIHALFVEIILSWSLSLPFTFEISSD&quot;&#xA;protein = re.findall('M[A-Z]{6,}\*',myprotein)&#xA;print(protein)&#xA;['MVTGSCQTLVQDKV*', 'MIPPCVFHPIYPLFTWSPLHPIISGCF*']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To get the output you asked for while reading from a file, you can do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import sys&#xA;import re&#xA;&#xA;proteinFile = sys.argv[1]&#xA;&#xA;with open(proteinFile, 'r') as f:&#xA;    for line in f:&#xA;        ## Split the long input line into the list 'sequences'&#xA;        sequences = line.rstrip('\n').split('&amp;gt;')&#xA;&#xA;        for i, k in list(enumerate(sequences)):&#xA;            ## Skip the first element which will be empty.&#xA;            ## there must be a pythonic way of doing this, but I'm&#xA;            ## a Perl hacker, so I don't know it.             &#xA;            if len(k) == 0:&#xA;                continue&#xA;&#xA;            seqName, seq = k.split(' ')&#xA;            ## Get the frame&#xA;            frame = seq[0]&#xA;            ## Remove the frame from the sequence&#xA;            seq = seq[1:]&#xA;            # cleave sequence&#xA;            peptides = re.findall(r'(M[A-Z]{6,}\*)', seq)&#xA;&#xA;            for idx, peptide in enumerate(peptides):&#xA;                print(&quot;&amp;gt;%s.%s.%d\n%s&quot; % (seqName, frame, idx+1, peptide) )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-05-02T14:38:57.300" LastActivityDate="2018-05-02T14:38:57.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4201" PostTypeId="1" AcceptedAnswerId="4209" CreationDate="2018-05-01T17:04:54.327" Score="5" ViewCount="518" Body="&lt;p&gt;I'm writing a function in R for an R package which takes as input a BAM. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;my_func = function(my_bam){&#xA;    if (!file.exists(my_bam)){&#xA;        stop(&quot;Input a valid BAM file.&quot;)&#xA;    }&#xA;&#xA;    ## do awesome stuff with BAM&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Trying to write robust software, I would like to do more of a check whether this BAM is actually a BAM. Above, I'm just using &lt;code&gt;file.exists()&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could also do something simple like &quot;check whether the file has the extension &lt;code&gt;*.bam&lt;/code&gt;&quot; with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grepl('*.bam', my_bam)  ## TRUE for file.bam, FALSE for file.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, is there a quick way to check that the input &lt;em&gt;is&lt;/em&gt; actually a BAM? I would prefer to throw an error here, if the user inputs a *txt or *bedpe. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think it's necessary to &lt;em&gt;validate&lt;/em&gt; the BAM per se, whereby I believe most users rely upon Picard:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://broadinstitute.github.io/picard/command-line-overview.html#ValidateSamFile&quot; rel=&quot;noreferrer&quot;&gt;https://broadinstitute.github.io/picard/command-line-overview.html#ValidateSamFile&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first thought was Rsamtools for this operation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/Rsamtools.html&quot; rel=&quot;noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/html/Rsamtools.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it doesn't appear that there is a function here which checks the user inputs a BAM, e.g. &lt;code&gt;BamFile()&lt;/code&gt; doesn't appear to throw an error&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; library(Rsamtools)&#xA;&amp;gt; fake_bam = 'nonsense_string.bam'   ## a string I made up, not a real BAM&#xA;&amp;gt; BamFile(fake_bam)&#xA;class: BamFile &#xA;path: nonsense_string.bam&#xA;index: NA&#xA;isOpen: FALSE &#xA;yieldSize: NA &#xA;obeyQname: FALSE &#xA;asMates: FALSE &#xA;qnamePrefixEnd: NA &#xA;qnameSuffixStart: NA &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How could I &quot;check&quot; that the user is inputting a BAM file? Perhaps pipe to samtools (which doesn't feel very stable in terms of software)? &lt;/p&gt;&#xA;" OwnerUserId="1770" LastActivityDate="2021-12-17T12:17:09.147" Title="Is there an efficient way to check an input BAM in R?" Tags="|r|sam|bioconductor|samtools|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4202" PostTypeId="1" AcceptedAnswerId="4208" CreationDate="2018-05-01T22:02:06.143" Score="7" ViewCount="12210" Body="&lt;p&gt;&lt;code&gt;FeaturePlot&lt;/code&gt; is a function in &lt;a href=&quot;http://satijalab.org/seurat/pbmc3k_tutorial.html&quot; rel=&quot;noreferrer&quot;&gt;Seurat&lt;/a&gt; package.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And in the vignette it is written that if we specify parameter &lt;code&gt;do.return = TRUE&lt;/code&gt; it should return ggplot2 object. It is not working. My goal here is just to change the title of the plot. In case of violin plot I can do the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;VlnPlot(object = seurat_object, features.plot = id, do.return = TRUE) + labs(title = endothelial_symbols[1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But with &lt;code&gt;FeaturePlot&lt;/code&gt; similar code fails to work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FeaturePlot(object = seurat_object, features.plot = id, cols.use = c(&quot;grey&quot;, &quot;blue&quot;),&#xA;                          reduction.use = &quot;tsne&quot;, do.return = TRUE) + labs(title = endothelial_symbols[1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Giving the error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error in FeaturePlot(object = seurat_object, features.plot = id, cols.use = c(&quot;grey&quot;,  : &#xA;    non-numeric argument to binary operator&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2018-05-02T08:04:24.033" LastActivityDate="2018-06-12T20:05:40.753" Title="FeaturePlot from Seurat: change its title" Tags="|r|ggplot2|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4203" PostTypeId="1" AcceptedAnswerId="4210" CreationDate="2018-05-01T22:22:57.147" Score="4" ViewCount="1120" Body="&lt;p&gt;I have a dataset with (microarray) gene expression data that was sampled from the same individuals at multiple timepoints. Our exposure is a continuous variable, and because this was an observational study there is no consistent pattern of change in exposure over time (eg. one person's exposure may continuously increase, another's may go up and down, another may have no changes, etc.). The exposure is not expected to have any cumulative effect, so different timepoints within a person do not influence each-other. Our goal is to find genes where expression for the gene is associated with the level of exposure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are thinking of analyzing this as a hierarchical model, with timepoints nested within person (so a mixed model with a random effect for person). We will also need to control for a few covariates, some of which will be the same at all timepoints (race, sex) and some of which may change between timepoints (use of particular medications).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this something that it is possible to use limma for? I found that limma has a function called &lt;code&gt;duplicateCorrelation()&lt;/code&gt; intended for block designs that some people have tried to use to fit mixed models, but I am not sure whether it can be used for this. Or if not limma, then what would you suggest?&lt;/p&gt;&#xA;" OwnerUserId="2296" LastEditorUserId="48" LastEditDate="2018-05-02T10:03:16.113" LastActivityDate="2018-05-02T10:03:16.113" Title="Hierarchical models with limma?" Tags="|bioconductor|microarray|transcriptome|modelling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4204" PostTypeId="2" ParentId="4138" CreationDate="2018-05-01T23:15:13.657" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://phytozome.jgi.doe.gov/phytomine/begin.do&quot; rel=&quot;nofollow noreferrer&quot;&gt;PhytoMine&lt;/a&gt; is an InterMine interface to data from Phytozome, the Plant Comparative Genomics portal of the Department of Energy's Joint Genome Institute, which hosts 81 assembled and annotation genomes, from 78 viridiplantae species.&lt;/p&gt;&#xA;" OwnerUserId="2719" LastActivityDate="2018-05-01T23:15:13.657" CommentCount="0" ContentLicense="CC BY-SA 3.0" />
  <row Id="4205" PostTypeId="1" AcceptedAnswerId="4224" CreationDate="2018-05-01T23:35:10.913" Score="2" ViewCount="140" Body="&lt;p&gt;I have a matrix with &lt;code&gt;ensemblIds&lt;/code&gt; which I am trying to convert to gene symbols using &lt;code&gt;BioMart&lt;/code&gt;. Some of the &lt;code&gt;ensemblIds&lt;/code&gt; will not be mapped, so I need to subset the rows of the matrix. It seems like &lt;code&gt;BioMart&lt;/code&gt; adds one &lt;code&gt;ensemblId&lt;/code&gt; or at least I see one more result ultimately than should be. Here is what I am trying to do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(dplyr)&#xA;library(tibble)&#xA;ids &amp;lt;- rownames(mtx_combined)&#xA;mart &amp;lt;- useDataset(&quot;mmusculus_gene_ensembl&quot;, useMart(&quot;ensembl&quot;))&#xA;list &amp;lt;- getBM(filters= &quot;ensembl_gene_id&quot;, attributes= c(&quot;ensembl_gene_id&quot;,&#xA;                           &quot;mgi_symbol&quot;,&quot;description&quot;),values=ids,mart= mart)&#xA;mtx_mapped &amp;lt;- as.matrix(as.data.frame(mtx_combined) %&amp;gt;% &#xA;rownames_to_column(&quot;gene&quot;) %&amp;gt;%&#xA;filter(rownames(mtx_combined) %in% list$ensembl_gene_id) %&amp;gt;%&#xA;column_to_rownames(&quot;gene&quot;))&#xA;&#xA;rownames(mtx_mapped) &amp;lt;- list$mgi_symbol   //Fails &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Length of the returned &lt;code&gt;list&lt;/code&gt; is &lt;code&gt;18337&lt;/code&gt;, number of rows in the initial matrix is &lt;code&gt;18376&lt;/code&gt;. After subsetting I expect to get &lt;code&gt;18337&lt;/code&gt; rows, but getting one less - &lt;code&gt;18336&lt;/code&gt;. What am I doing wrong here?&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="29" LastEditDate="2018-05-03T11:40:14.727" LastActivityDate="2018-05-03T12:31:05.400" Title="Off-by-one error when mapping Ensembl IDs to gene symbols" Tags="|r|biomart|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4206" PostTypeId="1" AcceptedAnswerId="4472" CreationDate="2018-05-02T01:10:40.813" Score="6" ViewCount="714" Body="&lt;p&gt;I have a series of proteins that are all phylogenetically related. Only one of this proteins is currently X-ray crystallized. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it valid to modify the reference PDB file (I mean by hand) to match the mutated sequence? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And if so, is there any way to minimize the protein energy to find the most likely structure? (Trying to match as closely as possible the reference structure)&lt;/p&gt;&#xA;" OwnerUserId="2754" LastEditorUserId="48" LastEditDate="2018-05-02T07:12:50.890" LastActivityDate="2023-11-13T20:18:23.347" Title="If I modify a PDB file with a specific mutation, how to minimize energy?" Tags="|proteins|protein-structure|pdb|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4207" PostTypeId="2" ParentId="4201" CreationDate="2018-05-02T05:33:28.687" Score="3" Body="&lt;p&gt;Take your &lt;code&gt;BamFile(fake_bam)&lt;/code&gt; and pass it through &lt;code&gt;open()&lt;/code&gt;. That will return a value if &lt;code&gt;fake_bam&lt;/code&gt; is a real BAM (or CRAM or maybe SAM) file and through an error otherwise. You can then wrap all of that in a &lt;code&gt;trycatch()&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;trycatch(open(BamFile(fake_bam)), do something, do something else)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Alternatively, ditch the &lt;code&gt;trycatch()&lt;/code&gt; and let &lt;code&gt;open()&lt;/code&gt; return the error if an invalid file in entered.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-05-02T05:33:28.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4208" PostTypeId="2" ParentId="4202" CreationDate="2018-05-02T07:58:14.953" Score="3" Body="&lt;p&gt;I tried with some data that I have and this is working for me: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p &amp;lt;- FeaturePlot(object = seurat_object, features.plot = id, cols.use = c(&quot;grey&quot;, &quot;blue&quot;),&#xA;                      reduction.use = &quot;tsne&quot;, do.return = TRUE)&#xA;&#xA;&#xA;lapply(p, function(x){x + labs(title = endothelial_symbols[1])})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think it is because FeaturePlot returns several ggplot objects according to the ids you put. This is returned as a list of plots so you have to iterate each one for labeling. &lt;/p&gt;&#xA;" OwnerUserId="678" LastActivityDate="2018-05-02T07:58:14.953" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4209" PostTypeId="2" ParentId="4201" CreationDate="2018-05-02T08:38:10.800" Score="6" Body="&lt;p&gt;You won’t know whether it’s a valid BAM file until you attempt reading it in full; even Devon’s method (which I recommend) only does a superficial check — i.e. it checks whether (parts of) the BAM &lt;em&gt;header&lt;/em&gt; can be read correctly. However, this doesn’t guarantee that the file isn’t truncated or contains invalid data.&lt;/p&gt;&#xA;&lt;p&gt;This is of course a more general problem, but it’s perhaps a bit stranger for BAMs (and other biological formats), as it’s relatively frequent to end up with technically invalid files that were produced by existing software. The only recourse is actually trying to use the file, and to handle errors appropriately.&lt;/p&gt;&#xA;&lt;p&gt;For a &lt;em&gt;very quick check&lt;/em&gt;™ you could also pass the file through &lt;code&gt;gunzip&lt;/code&gt; and test whether the “magic bytes” at the beginning of the file equal &lt;code&gt;'BAM\1'&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bgzf_stream = gzfile(filename, 'r')&#xA;on.exit(close(bgzf_stream))&#xA;magic = readChar(bgzf_stream, 4)&#xA;stopifnot(identical(magic, 'BAM\1'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2021-12-17T12:17:09.147" LastActivityDate="2021-12-17T12:17:09.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4210" PostTypeId="2" ParentId="4203" CreationDate="2018-05-02T09:31:21.310" Score="3" Body="&lt;p&gt;Yes, you can use limma for this mixed model approach. Like you suggest, the random effect (persons) can be put in duplicateCorrelation().&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://support.bioconductor.org/p/87931/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a similar example with RNAseq data, on bioconductor support site.  &lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2018-05-02T09:47:52.347" LastActivityDate="2018-05-02T09:47:52.347" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="4211" PostTypeId="1" CreationDate="2018-05-02T12:26:01.973" Score="0" ViewCount="107" Body="&lt;p&gt;I have used Deseq for estimating the differentially expressed introns for the RNA seq data using the UCSC intron bed file. can Deseq be used for differentially expressed intron analysis?&lt;/p&gt;&#xA;" OwnerUserId="2757" LastActivityDate="2018-05-02T12:33:42.243" Title="differentially expressed intron analysis?" Tags="|rna-seq|deseq2|ucsc|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4212" PostTypeId="2" ParentId="4211" CreationDate="2018-05-02T12:33:42.243" Score="2" Body="&lt;p&gt;Typical tools to use for this would be the DEXseq packages (&lt;a href=&quot;https://currentprotocols.onlinelibrary.wiley.com/doi/abs/10.1002/0471142905.hg1116s87&quot; rel=&quot;nofollow noreferrer&quot;&gt;as done here&lt;/a&gt;) or &lt;a href=&quot;http://rnaseq-mats.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;rMATS&lt;/a&gt;, which will calculate intron retention rates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would generally suggest not summing intronic counts by gene and using that in DESeq2 (I assume that's what you meant by DESeq, since there's no reason to use the original DESeq for anything). What you'd be getting if you did this is less expressed introns and more sample-specific bias in enriching for mature versus pre-mRNA.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-05-02T12:33:42.243" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4213" PostTypeId="1" AcceptedAnswerId="4219" CreationDate="2018-05-02T21:49:22.110" Score="3" ViewCount="117" Body="&lt;p&gt;I'm trying to run a differential expression analysis of 2 samples of the same species, but with different treatments using DESeq2. I'm using Ensembldb to create a tx2gene dataframe for tximport, however tximport does not select the columns I specify, and instead always selects the tx_biotype leading to the following output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;.                    25_quant    28_quant&#xA;nontranslating_CDS    10422.41    10022.74&#xA;protein_coding     22196722.59 24002269.26&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm running tximport using the following params:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;txi_tx &amp;lt;- tximport(files, type = &quot;salmon&quot;, tx2gene = tx2gene,&#xA;                   txIdCol = &quot;tx_id&quot;, geneIdCol = &quot;gene_id&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've gotten around this by deleting all columns from my tx2gene DF apart from tx_id and gene_id. However, I think this might be causing problems in the DESeq2 analysis somehow, and I would like to rule out that possibility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what my &lt;code&gt;names(tx2gene)&lt;/code&gt; looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;tx_id&quot;            &quot;tx_biotype&quot;       &quot;tx_seq_start&quot;     &quot;tx_seq_end&quot;&#xA;&quot;tx_cds_seq_start&quot; &quot;tx_cds_seq_end&quot;  &#xA;&quot;gene_id&quot;          &quot;tx_name&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What am I doing wrong?&lt;/p&gt;&#xA;" OwnerUserId="2721" LastEditorUserId="48" LastEditDate="2018-05-03T07:07:54.093" LastActivityDate="2018-05-03T07:07:54.093" Title="tximport txidcol and geneidcol selecting wrong column" Tags="|r|ensembl|tximport|tx2gene|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4214" PostTypeId="1" AcceptedAnswerId="4218" CreationDate="2018-05-02T22:25:33.417" Score="10" ViewCount="240" Body="&lt;p&gt;I'm about to run samtools sort (version 1.6) and I'm reviewing the available configuration options.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ samtools sort&#xA;Usage: samtools sort [options...] [in.bam]&#xA;Options:&#xA;  -l INT     Set compression level, from 0 (uncompressed) to 9 (best)&#xA;  -m INT     Set maximum memory per thread; suffix K/M/G recognized [768M]&#xA;  -n         Sort by read name&#xA;  -t TAG     Sort by value of TAG. Uses position as secondary index (or read name if -n is set)&#xA;  -o FILE    Write final output to FILE rather than standard output&#xA;  -T PREFIX  Write temporary files to PREFIX.nnnn.bam&#xA;      --input-fmt-option OPT[=VAL]&#xA;               Specify a single input file format option in the form&#xA;               of OPTION or OPTION=VALUE&#xA;  -O, --output-fmt FORMAT[,OPT[=VAL]]...&#xA;               Specify output format (SAM, BAM, CRAM)&#xA;      --output-fmt-option OPT[=VAL]&#xA;               Specify a single output file format option in the form&#xA;               of OPTION or OPTION=VALUE&#xA;      --reference FILE&#xA;               Reference sequence FASTA FILE [null]&#xA;  -@, --threads INT&#xA;               Number of additional threads to use [0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm curious about the &lt;code&gt;--reference&lt;/code&gt; flag. I've never used it before with &lt;code&gt;samtools sort&lt;/code&gt; and I'm wondering what behavior it invokes. The &lt;a href=&quot;http://www.htslib.org/doc/samtools.html&quot; rel=&quot;noreferrer&quot;&gt;online documentation&lt;/a&gt; doesn't even mention the flag. &lt;strong&gt;Under what circumstances would I want/need to specify the reference genome?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2018-05-03T06:51:14.930" Title="Behavior of `--reference` flag with samtools sort" Tags="|samtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4216" PostTypeId="1" AcceptedAnswerId="4286" CreationDate="2018-05-03T00:23:12.150" Score="4" ViewCount="852" Body="&lt;p&gt;I have called variants on two strains of &lt;em&gt;C. elegans&lt;/em&gt; separately. I now want to merge the VCF files into one using the following code:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Create a sequence dictionary of the reference sequence&lt;/li&gt;&#xA;&lt;li&gt;Sort the VCF files with Picard&lt;/li&gt;&#xA;&lt;li&gt;Merge the sorted VCF files using GATK&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&amp;nbsp;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;picard CreateSequenceDictionary \&#xA;    REFERENCE=c_elegans.PRJNA13758.WS263.genomic.fa \&#xA;    OUTPUT=c_elegans.PRJNA13758.WS263.genomic.dict&#xA;&#xA;picard SortVcf INPUT=strain1.vcf \&#xA;    OUTPUT=strain1sorted.vcf \&#xA;    SEQUENCE_DICTIONARY=c_elegans.PRJNA13758.WS263.genomic.dict&#xA;&#xA;picard SortVcf INPUT=strain2.vcf \&#xA;    OUTPUT=strain2sorted.vcf  \     &#xA;    SEQUENCE_DICTIONARY=c_elegans.PRJNA13758.WS263.genomic.dict&#xA;&#xA;GATK --analysis_type CombineVariants \&#xA;    -R c_elegans.PRJNA13758.WS263.genomic.fa \&#xA;    --variant strain1sorted.vcf \&#xA;    --variant strain2sorted.vcf \&#xA;    -o all.vcf \&#xA;    -genotypeMergeOptions UNIQUIFY&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The last command gives me the following error message:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ERROR MESSAGE: Input files variant and reference have incompatible contigs. Please see https://www.broadinstitute.org/gatk/guide/article?id=63 for more information. Error details: The contig order in variant and reference is not the same; to fix this please see: (https://www.broadinstitute.org/gatk/guide/article?id=1328),  which describes reordering contigs in BAM and VCF files..&#xA;##### ERROR   variant contigs = [I, II, III, IV, MtDNA, V, X]&#xA;##### ERROR   reference contigs = [I, II, III, IV, V, X, MtDNA]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I have sorted the VCF files using Picard, so I don't know what else to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your help is appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: I asked this question on the &lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/11918/gatk-combinevariants-complains-the-contig-order-in-the-vcf-files/p1?new=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATK&lt;/a&gt; forum on 4th May 2018 (today).&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="96" LastEditDate="2019-01-15T14:24:35.427" LastActivityDate="2019-01-15T14:24:35.427" Title="GATK CombineVariants complains the contig order in the VCF files" Tags="|vcf|variant-calling|gatk|picard|variation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4218" PostTypeId="2" ParentId="4214" CreationDate="2018-05-03T06:51:14.930" Score="4" Body="&lt;p&gt;You would only need to supply a reference when using CRAM output encoding. The CRAM format allows storing entries without explicitly encoding the base calls, since one can instead encode the differences to the reference genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With BAM output the &lt;code&gt;--reference&lt;/code&gt; option has no effect.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-05-03T06:51:14.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4219" PostTypeId="2" ParentId="4213" CreationDate="2018-05-03T06:57:58.867" Score="2" Body="&lt;p&gt;By setting &lt;code&gt;type=&quot;salmon&quot;&lt;/code&gt; you're overriding your settings for things like &lt;code&gt;txIdCol&lt;/code&gt;. The various &lt;code&gt;type&lt;/code&gt; options (except &lt;code&gt;none&lt;/code&gt;) are presets to set things correctly for the output from various programs like &lt;code&gt;salmon&lt;/code&gt; or &lt;code&gt;kallisto&lt;/code&gt;. If you need to specify custom column names, set &lt;code&gt;type=&quot;none&quot;&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an aside, if I'm interpreting your question correctly you have only two samples in total. You can not perform a meaningful analysis with an unreplicated experiment, so if that is indeed your case then don't waste your time further with this dataset.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-05-03T06:57:58.867" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4220" PostTypeId="5" CreationDate="2018-05-03T12:07:26.143" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-05-03T12:07:26.143" LastActivityDate="2018-05-03T12:07:26.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4221" PostTypeId="4" CreationDate="2018-05-03T12:07:26.143" Score="0" Body="For statistical questions related to bioinformatics. If the nature of question is purely statistical, consider stats.stackexchange.com instead." OwnerUserId="57" LastEditorUserId="77" LastEditDate="2018-05-03T12:59:11.750" LastActivityDate="2018-05-03T12:59:11.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4222" PostTypeId="5" CreationDate="2018-05-03T12:11:52.143" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-05-03T12:11:52.143" LastActivityDate="2018-05-03T12:11:52.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4223" PostTypeId="4" CreationDate="2018-05-03T12:11:52.143" Score="0" Body="Use this tag for questions specifically related to the text SAM format. For general questions about the SAM/BAM formats use the tag BAM." OwnerUserId="57" LastEditorUserId="77" LastEditDate="2018-05-03T12:57:17.567" LastActivityDate="2018-05-03T12:57:17.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4224" PostTypeId="2" ParentId="4205" CreationDate="2018-05-03T12:31:05.400" Score="6" Body="&lt;p&gt;There’s no 1:1 mapping between the IDs of different database schemas. As a consequence, some Ensembl gene IDs map to multiple MGI symbols, or to none (and vice versa).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, you can’t assume that the number of result rows will be equal to the number of query IDs.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2018-05-03T12:31:05.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4225" PostTypeId="1" CreationDate="2018-05-03T14:18:16.577" Score="1" ViewCount="352" Body="&lt;p&gt;&lt;em&gt;This has also &lt;a href=&quot;https://www.biostars.org/p/312943/&quot; rel=&quot;nofollow noreferrer&quot;&gt;been posted on biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I had an initial bipartite transcriptional regulatory network for bees. My idea is to create a similar network for termites so as to check if similar genes are hubs in both the networks (being that both are eusocial). For this purpose, I took the sequences of the genes in the honey bee GRN and did a BLAST to identify similar genes in the species of eusocial termite that I am interested in. The problem is that I am out of ideas on how to infer a network from the set of genes that I have.&lt;/p&gt;&#xA;&lt;p&gt;I cannot use Genemania which can only be used for genes from specific species like humans, flies etc. I cannot use STRING because that uses protein sequences and the species I work with is also not on it. Most tools I found can be used for network creation only if the genes I have is from well studies species like mus musculus, drosophila etc.&lt;/p&gt;&#xA;&lt;p&gt;Is there anyway in which I can create a network from the set of genes that I have?&lt;/p&gt;&#xA;&lt;p&gt;I do not have interaction information. I have a network of genes from honey bees. I did a BLAST of the genes in that network and retrieved the gene names for the related genes that I got from a species of termites. I would like to create a gene network of that to check if the same genes form the hubs in both the networks.&lt;/p&gt;&#xA;&lt;p&gt;The gene network in bees is a bipartite transcription regulatory network. I haven't checked the gene ontology of the termite genes but I would think that the genes should be having similar functions because they are BLAST results. There are around 2200 genes in the honey bee GRN and I could find approx 1000 genes from the BLAST and UNIPROT for termites. The honey bee GRN was generated experimentally by a different lab so I could not replicate what they did with ARACNE.&lt;/p&gt;&#xA;&lt;p&gt;What do you suggest that I do to find similar genes then if a BLASTN is not sufficient information?&lt;/p&gt;&#xA;" OwnerUserId="976" LastEditorUserId="73" LastEditDate="2021-05-10T02:51:59.297" LastActivityDate="2021-05-10T02:51:59.297" Title="Create gene network from list of genes" Tags="|networks|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4226" PostTypeId="1" AcceptedAnswerId="4227" CreationDate="2018-05-03T15:29:46.017" Score="12" ViewCount="4241" Body="&lt;p&gt;I am currently exploring the BLAST program, just for testing purpose i generated two FASTA files, containing two genes A and B, such that B is just a motif of repeated 'G's that occurs in A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;file A.fna:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;A&#xA;ACCAACAGACGGGGGGGGGGGGGAGAAAACCGGCCCCCGCAGGGACGCCACCCAGAGGGAAGGG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;file B.fna:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;B&#xA;GGGGGGGGGGGG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So by my understanding BLAST should find the local alignment of B in A easily? But the program returns 'No hits found'? Maybe I am doing something wrong here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I executed the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;makeblastdb -in A.fna -dbtype 'nucl'&#xA;makeblastdb -in B.fna -dbtype 'nucl'&#xA;blastn -db A.fna -query B.fna&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;output of blastn:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BLASTN 2.7.1+&#xA;&#xA;&#xA;Reference: Zheng Zhang, Scott Schwartz, Lukas Wagner, and Webb Miller (2000), &quot;A greedy algorithm for aligning DNA sequences&quot;, J Comput Biol 2000; 7(1-2):203-14.&#xA;&#xA;&#xA;&#xA;Database: A.fna&#xA;           1 sequences; 64 total letters&#xA;&#xA;&#xA;&#xA;Query= B&#xA;&#xA;Length=12&#xA;&#xA;&#xA;***** No hits found *****&#xA;&#xA;&#xA;&#xA;Lambda      K        H&#xA;    1.33    0.621     1.12 &#xA;&#xA;Gapped Lambda      K        H&#xA;    1.28    0.460    0.850 &#xA;&#xA;Effective search space used: 480&#xA;&#xA;&#xA;  Database: A.fna&#xA;    Posted date:  May 3, 2018  5:17 PM   Number of letters in database: 64   Number of sequences in database:  1&#xA;&#xA;&#xA;&#xA;Matrix: blastn matrix 1 -2 Gap Penalties: Existence: 0, Extension: 2.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thanks for any suggestions (:&lt;/p&gt;&#xA;" OwnerUserId="2766" LastEditorUserId="298" LastEditDate="2018-05-03T16:27:31.107" LastActivityDate="2023-01-26T21:43:04.243" Title="BLAST(n): No hits found" Tags="|sequence-alignment|blast|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4227" PostTypeId="2" ParentId="4226" CreationDate="2018-05-03T17:06:31.820" Score="13" Body="&lt;p&gt;There are three possible problems that come to mind.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Masking&lt;/p&gt;&#xA;&lt;p&gt;Blast will mask low complexity regions by default. Since your sequence is nothing but Gs, it is a safe bet that it is being masked, so no hits will be found for it.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Score/e-value thresholds&lt;/p&gt;&#xA;&lt;p&gt;Another source of complication is that even if a match is found, that match will have very bad scores. Both the actual score of the alignment and the e-value will be very bad. Since this is such a simple sequence, it will always score badly.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Word-size&lt;/p&gt;&#xA;&lt;p&gt;The way blast works is (simplifying a bit) by finding a match for N residues (the word size) and then extending that match if extending increases the score. If your query sequence is shorter than the word size, no match will ever be found.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;With all that said, the specific problem in your case seems to be a combination of the word-size and masking. If you run your query setting the word-size to 11 and disabling masking, as shown below, you should get results.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastn -db A.fna -query B.fna -word_size 11 -dust no&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That should return two hits (two because the word size is one less than the query size, so it can align perfectly at two locations):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; A&#xA;Length=64&#xA;&#xA; Score = 23.3 bits (12),  Expect = 5e-05&#xA; Identities = 12/12 (100%), Gaps = 0/12 (0%)&#xA; Strand=Plus/Plus&#xA;&#xA;Query  1   GGGGGGGGGGGG  12&#xA;           ||||||||||||&#xA;Sbjct  11  GGGGGGGGGGGG  22&#xA;&#xA; Score = 23.3 bits (12),  Expect = 5e-05&#xA; Identities = 12/12 (100%), Gaps = 0/12 (0%)&#xA; Strand=Plus/Plus&#xA;&#xA;Query  1   GGGGGGGGGGGG  12&#xA;           ||||||||||||&#xA;Sbjct  12  GGGGGGGGGGGG  23&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that the default word size for BLASTn searches is usually 11. I don't know why the blast+ version (2.2.31+) I have on my Ubuntu machine didn't use that as a default. The manual says it should. But I got no hits when I didn't explicitly set the word size to 11.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-05-03T17:06:31.820" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4229" PostTypeId="2" ParentId="4201" CreationDate="2018-05-03T17:13:50.197" Score="1" Body="&lt;p&gt;Just expanding upon the answers by @DevonRyan and @KonradRudolph, here's one way to put this in an R package:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Either using &lt;code&gt;open(BamFile()&lt;/code&gt; from Devon's answer, which needs &lt;code&gt;Rsamtools&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;my_func = function(my_bam){&#xA;&#xA;    out = tryCatch(open(Rsamtools::BamFile(my_bam)), error = function(e) e)&#xA;    if (inherits(out, &quot;error&quot;)){&#xA;        stop(&quot;Input a valid BAM file.&quot;)&#xA;    }&#xA;&#xA;    ## do awesome stuff with BAM&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or from Konrad's answer&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;my_func = function(my_bam){&#xA;&#xA;    magic = readChar(gzfile(filename, 'r'), 4)&#xA;    if (!identical(magic, 'BAM\1')){&#xA;        stop(&quot;Input a valid BAM file.&quot;)&#xA;    }&#xA;&#xA;    ## do awesome stuff with BAM&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1770" LastEditorUserId="29" LastEditDate="2018-05-04T09:10:03.657" LastActivityDate="2018-05-04T09:10:03.657" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4230" PostTypeId="2" ParentId="4174" CreationDate="2018-05-03T18:10:42.313" Score="2" Body="&lt;p&gt;The issue was that I supplied wrong indexes during &lt;code&gt;fastq&lt;/code&gt; files generation, however, they were generated successfully, and only the next step failed with not that informative error.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-05-03T18:10:42.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4231" PostTypeId="2" ParentId="664" CreationDate="2018-05-03T22:28:29.710" Score="1" Body="&lt;p&gt;I would go for the topGO package. The logical steps in your pipeline would be:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Build a matrix with the following variables:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;|gene_ID|GO_ID|GO_term_description|&#xA;You can enter a list of uniprot IDs to uniprotkb and retrieve the GO annotations for each gene in your list/file&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) filter that list for the genes that were captured in your differential expression analysis even tho they are not differentially expressed. This filter is defined by DESeq/Sleuth by &quot;at least 4 read counts in at least 49% of the libraries&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3)Define this gene list as gene_universe&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) define your gene set and get the gene_IDs and if possible, also a ranked score (p-val/FDR) of the genes you are interested in&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5) plot the graphs and get tables of enriched GO terms (these would be ranked by p-value without need of performing correction analysis&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: you are going to need some data manipulation functions; I would read the &quot;tidyverse&quot; documentation&lt;/p&gt;&#xA;" OwnerUserId="1568" LastActivityDate="2018-05-03T22:28:29.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4232" PostTypeId="1" AcceptedAnswerId="4233" CreationDate="2018-05-04T01:00:04.657" Score="4" ViewCount="3342" Body="&lt;p&gt;I know that I can do subsetting on just one gene in &lt;code&gt;Seurat&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seurat_subset &amp;lt;- SubsetData(seurat_object, subset.name = neuron_ids[1], accept.low = 0.1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I want to subset on multiple genes. Is there a way to do that? I just do not want to do manual subsetting on 10 genes, then manually getting &lt;code&gt;@data&lt;/code&gt; matrix from each subset, and recreating seurat object afterwards.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="181" LastEditDate="2018-05-17T00:41:05.707" LastActivityDate="2018-05-17T00:41:05.707" Title="Subset on multiple genes in Seurat" Tags="|r|rna-seq|single-cell|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4233" PostTypeId="2" ParentId="4232" CreationDate="2018-05-04T01:17:43.603" Score="6" Body="&lt;p&gt;I was able to achieve this in the following way:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;require(data.table)&#xA;cell_names &amp;lt;- vector(mode=&quot;character&quot;)&#xA;for (i in 1:length(neuron_ids)) {&#xA;  seurat_subset &amp;lt;- SubsetData(seurat_object, subset.name = neuron_ids[i], accept.low = 0.1)&#xA;  genes &amp;lt;- colnames(seurat_subset@data)&#xA;  cell_names &amp;lt;- c(cell_names, genes[! genes %chin% cell_names])&#xA;}&#xA;seurat_subset &amp;lt;- SubsetData(seurat_object, cells.use = cell_names)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Would be interesting to know if &lt;code&gt;Seurat&lt;/code&gt; provides such functionality out of the box.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="77" LastEditDate="2018-05-04T07:59:00.323" LastActivityDate="2018-05-04T07:59:00.323" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4235" PostTypeId="1" CreationDate="2018-05-04T14:07:01.117" Score="1" ViewCount="289" Body="&lt;p&gt;Im trying to filter SNPs using PLINK with this command. Also, the .map and .ped files were converted to a plink format from a vcf file (&lt;code&gt;python executable.py --input allindividuals.vcf&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plink --ped allindividuals.ped --map allindividuals.map --allow-extra-chr --indep-pairwise 50 10 0.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but at the time I count the SNPs in the file plink.prune.in it has only 15 SNPs and in the file plink.prune.out it remains millions of them. In addition, when I change the parameters, the result is the same, no matter how many times I change it.&lt;/p&gt;&#xA;" OwnerUserId="2768" LastEditorUserId="48" LastEditDate="2018-05-04T14:29:19.233" LastActivityDate="2018-05-04T14:29:19.233" Title="Error filtering SNPs with Plink" Tags="|vcf|snp|filtering|plink|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4236" PostTypeId="1" AcceptedAnswerId="4237" CreationDate="2018-05-04T16:00:49.160" Score="0" ViewCount="205" Body="&lt;p&gt;The chromosome sizes at UCSC don't seem to contain chr29:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;ftp://hgdownload.soe.ucsc.edu/goldenPath/galGal5/bigZips/galGal5.chrom.sizes&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://hgdownload.soe.ucsc.edu/goldenPath/galGal5/bigZips/galGal5.chrom.sizes&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has a chr28 and a chr30. Am I missing something or is there some piece of history that led to the omission of this chromosome?&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2018-05-04T18:59:16.517" Title="Does the galGal5 chicken assembly have a chromosome 29?" Tags="|genome|assembly|reference-genome|ucsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4237" PostTypeId="2" ParentId="4236" CreationDate="2018-05-04T18:59:16.517" Score="2" Body="&lt;p&gt;The current ensembl entry doesn't have a 29 either.  The archived ensembl assembly  lacks 29 30, and 31 and 33 and LGE64.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The chromosomes after 30 are tiny, so they might not be visible in a karyotype.  They probably realized that &quot;chr 29&quot; was really attached to some other chromosome.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2018-05-04T18:59:16.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4238" PostTypeId="1" CreationDate="2018-05-04T19:13:10.230" Score="3" ViewCount="231" Body="&lt;p&gt;I have a list of gene symbols:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;c(&quot;cd45&quot;, &quot;Tmem119&quot;, &quot;CD11b&quot;, &quot;P2Yr12&quot;, &quot;Csf1r&quot;, &quot;Bst2&quot;, &quot;Cd74&quot;, &#xA;&quot;Cx3cr1&quot;, &quot;Trem2&quot;, &quot;Lyz2&quot;, &quot;GLAST&quot;, &quot;GFAP&quot;, &quot;ALDH1L1&quot;, &quot;Aldoc&quot;, &quot;Aqp4&quot;, &#xA;&quot;Glul&quot;, &quot;S100a&quot;, &quot;Olig1&quot;, &quot;Olig2&quot;, &quot;Olig3&quot;, &quot;Mbp&quot;, &quot;Pdgfra&quot;, &quot;Pecam&quot;, &#xA;&quot;Cldn5&quot;, &quot;Cldn10&quot;, &quot;Epas1&quot;, &quot;Crip1&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I feed it to &lt;code&gt;BioMart&lt;/code&gt; to get &lt;code&gt;ensembl ids&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mart &amp;lt;- useDataset(&quot;mmusculus_gene_ensembl&quot;, useMart(&quot;ensembl&quot;))&#xA;list &amp;lt;- getBM(filters= &quot;mgi_symbol&quot;, attributes= c(&quot;ensembl_gene_id&quot;,&#xA;                           &quot;mgi_symbol&quot;,&quot;description&quot;),values=symbols,mart= mart)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the following genes are missed:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;cd45&quot;   &quot;cd11b&quot;  &quot;p2yr12&quot; &quot;glast&quot;  &quot;s100a&quot;  &quot;pecam&quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;All of them are pretty well-known genes, and I can manually find their &lt;code&gt;ensembl ids&lt;/code&gt; by googling it, for instance:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.informatics.jax.org/marker/MGI:97810&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.informatics.jax.org/marker/MGI:97810&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried supplying aliases, but it does not change the output. So, from my understanding &lt;code&gt;BioMart&lt;/code&gt; is not working properly either because I am doing something wrong here or because &lt;code&gt;BioMart&lt;/code&gt; itself is not a good tool to use. Is there a better way of getting the mapping that would map all of the gene symbols?&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-05-11T12:55:43.683" Title="BioMart Ensembl id imperfect retrieval" Tags="|r|biomart|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4239" PostTypeId="2" ParentId="4238" CreationDate="2018-05-04T19:27:44.643" Score="4" Body="&lt;p&gt;The genes that are missed are probably not official mgi symbols.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might wanna look them up at mgi:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cd45 -&amp;gt; Ptprc &#xA;&#xA;cd11b -&amp;gt; Itgam &#xA;&#xA;p2ry12 -&amp;gt; P2ry12 (?) (CAPS sensitive?)    &#xA;&#xA;glast -&amp;gt; Slc1a3 &#xA;&#xA;s100a -&amp;gt; S100a1 &#xA;&#xA;pecam -&amp;gt; Pecam1 (?)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I suspect biomaRt is not the problem here, the only better way I can come up with is to download the ENSEMBL annotation (GTF) &lt;a href=&quot;ftp://ftp.ensembl.org/pub/release-92/gtf/mus_musculus&quot; rel=&quot;nofollow noreferrer&quot;&gt;file&lt;/a&gt;, and get the symbol out of this annotation with AWK or in R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here an example in R, since you don't want to use AWK:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;file &amp;lt;- read.table(&quot;Mus_musculus.GRCm38.78.gtf&quot;, sep = &quot;\t&quot;, comment.char = &quot;#&quot;)&#xA;&#xA;exons &amp;lt;- subset(file, V3 == 'exon')&#xA;&#xA;V9 &amp;lt;- file[,&quot;V9&quot;]&#xA;&#xA;ensembl_names &amp;lt;- sub(&quot;gene_id &quot;,&quot;&quot;,V9)&#xA;&#xA;split &amp;lt;- strsplit(ensembl_names, &quot;;&quot;)&#xA;&#xA;splitting &amp;lt;- sapply(split, function(x) x[1])&#xA;&#xA;split_symbol &amp;lt;- strsplit(ensembl_names, &quot;gene_name &quot;)&#xA;&#xA;splitting_symbol &amp;lt;- sapply(split_symbol, function(x) x[2])&#xA;&#xA;splitting_symbol2 &amp;lt;- strsplit(splitting_symbol, &quot;; &quot;)&#xA;&#xA;symbol_names &amp;lt;- sapply(splitting_symbol2, function(x) x[1])&#xA;&#xA;ensemble_symbol_big &amp;lt;- cbind(splitting, symbol_names)&#xA;&#xA;ensembl_symbol &amp;lt;- unique(ensemble_symbol_big)&#xA;&#xA;colnames(ensembl_symbol) &amp;lt;- c(&quot;ENSEMBL&quot;, &quot;Symbol&quot;)&#xA;&#xA;write.table(ensembl_symbol, &quot;ensembl_symbol.txt&quot;, sep = &quot;\t&quot;, row.names = F)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2018-05-11T12:55:43.683" LastActivityDate="2018-05-11T12:55:43.683" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4240" PostTypeId="1" AcceptedAnswerId="4242" CreationDate="2018-05-05T08:53:04.593" Score="0" ViewCount="1209" Body="&lt;p&gt;I am a bit in doubt how to calculate correctly the mean (average) in the logarithmic scale. I know e.g., that a division in the normal scale becomes a subtraction in the logarithmic scale, and so forth, that's why I was wondering what the correct way is to calculate the mean (since the mean is calculated by additions and divisions in the normal scale).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for example, if I have in RNAseq data a group with 3 samples, and have log2 CPM values for each sample. How can I calculate the mean of the group correctly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I first need to get CPM (without log2), calculate the mean and then apply log2? Or can I just ignore the fact that the log2 CPM values are in the log scale, and just calculate the mean as if it was the normal scale (add all up and divide by &lt;em&gt;n&lt;/em&gt;)?&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="57" LastEditDate="2018-05-06T07:14:18.857" LastActivityDate="2018-05-06T21:56:07.830" Title="Average of logCPM" Tags="|rna-seq|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4241" PostTypeId="1" CreationDate="2018-05-05T17:40:37.300" Score="4" ViewCount="1700" Body="&lt;p&gt;I am using &lt;code&gt;ggplot2&lt;/code&gt; to plot a scatter plot.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(ggforce) # required by facet_zoom&#xA;&#xA;g &amp;lt;- ggplot(data.plot, aes(x = Methyl_Average, y = Average_Gene_Expression))  &#xA;g1 &amp;lt;-  g + geom_point(color = &quot;steelblue&quot;) + theme_bw(base_family = &quot;Times&quot;) +&#xA;       ggtitle( &quot;Scatter Plot&quot;) + &#xA;       theme_bw(base_family = &quot;Times&quot;) +&#xA;       theme(plot.title = element_text(hjust = 0.5))    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uKecW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uKecW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As most of my data points are in the range of 0 - 200 on y-axis, I want to zoom this section. I tried doing it using &lt;code&gt;facet_zoom&lt;/code&gt; from &lt;code&gt;ggforce&lt;/code&gt; package :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;g1 + facet_zoom(y = Average_Gene_Expression &amp;gt; 0 &amp;amp; Average_Gene_Expression &amp;lt; 200)    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iNNjD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iNNjD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of two plots in one figure, I want one plot (like first figure), where 80% of y-axis displays the data points in the range of 0 - 200 and 20% of the y-axis displays the remaining and x-axis unaltered. Which function/package should I use do it in R? &lt;/p&gt;&#xA;" OwnerUserId="2247" LastEditorUserId="2247" LastEditDate="2018-05-10T15:39:46.513" LastActivityDate="2018-05-10T15:39:46.513" Title="How to enlarge a section of y-axis?" Tags="|r|visualization|ggplot2|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4242" PostTypeId="2" ParentId="4240" CreationDate="2018-05-06T08:34:32.710" Score="2" Body="&lt;p&gt;The mean of log-transformed values is the log-transformed geometric mean of the untransformed values (i.e., $log_{2}(geometric~mean)$). This can be useful when the values you want to summarize have vastly different ranges (so one doesn't dominate the average). I would expect that to generally &lt;strong&gt;NOT&lt;/strong&gt; be the case for CPMs of the same gene, so it's unclear to me what the point of that would be. It can also be difficult to interpret these results due to the geometric mean being dependent upon the variance:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; originalValues = rnorm(1000, mean=10, sd=c(rep(1, 500), rep(2, 500)))&#xA;&amp;gt; mean(originalValues[1:500])&#xA;[1] 9.991464&#xA;&amp;gt; mean(originalValues[501:1000])&#xA;[1] 10.04407&#xA;&amp;gt; log2(mean(originalValues[1:500]))&#xA;[1] 3.320696&#xA;&amp;gt; log2(mean(originalValues[501:1000]))&#xA;[1] 3.328272&#xA;&amp;gt; mean(log2(originalValues[1:500]))&#xA;[1] 3.313263&#xA;&amp;gt; mean(log2(originalValues[501:1000]))&#xA;[1] 3.300103&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Above I created two groups of 500 values each. Each group has approximately the same mean (10) but with different variance (1 vs 2). As you can see, the second group of 500 values randomly has a slightly higher average. The log2(mean) preserves this difference, but the log2(geometric mean) inverts the relationship such that group 2 now appears to have a lower mean value. This isn't really wrong, it's the difference between mean and geometric mean, but it's something that needs to be kept in mind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has implications if your want to compare values between genes, since your comparison metric is then not just dependent upon the mean, but also the variability of the data (such that more variable genes will have systematically lower geometric means).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, the geometric mean is useful when values are on different scales (such as CPM values of all genes in a sample, but probably not CPM values of the same gene between samples). If that's not the case, then you're mostly removing interpretability from the results.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-05-06T21:56:07.830" LastActivityDate="2018-05-06T21:56:07.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4243" PostTypeId="2" ParentId="4216" CreationDate="2018-05-06T22:12:09.937" Score="3" Body="&lt;p&gt;Answered in the &lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/11918/gatk-combinevariants-complains-the-contig-order-in-the-vcf-files&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATK forum&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Try regenerating the index files for your VCFs. Picard SortVcf doesn't do it for you iirc, so when GATK looks at the index files (before opening the VCFs themselves) it sees the old order and complains.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2776" LastEditorUserId="48" LastEditDate="2018-05-08T07:41:45.797" LastActivityDate="2018-05-08T07:41:45.797" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4244" PostTypeId="1" CreationDate="2018-05-07T08:07:25.720" Score="2" ViewCount="387" Body="&lt;p&gt;I used this command for extracting read counts for almost 60 samples using featureCounts.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;featureCounts -a /gencode.v27.primary_assembly.annotation_nochr.gtf -F GTF -p -s 2 -T 8 -o counts.txt sample.sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I see many &quot;temp-core-100373-A4BF0116E148.sam-TH00-BK004414.tmp&quot; files like this dumped in my directory. I'm able to delete them only manually. Is there a way to get rid of these tmp files from my directory while using the command?&lt;/p&gt;&#xA;" OwnerUserId="1369" LastActivityDate="2018-05-07T08:21:06.017" Title="How to get rid of the temp files while using &quot;featureCounts&quot; for extracting readcounts from bam files?" Tags="|rna-seq|sam|rna|reads|gtf|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4245" PostTypeId="2" ParentId="4244" CreationDate="2018-05-07T08:21:06.017" Score="2" Body="&lt;p&gt;You shouldn't delete them while featureCounts is running, it's creating read-name-sorted temporary files so it can more easily look at both mates together. Just wait for it to finish running and it will clean up the files itself. Doing so beforehand is liable to cause an error.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-05-07T08:21:06.017" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4246" PostTypeId="1" CreationDate="2018-05-07T15:49:42.250" Score="5" ViewCount="1191" Body="&lt;p&gt;After the alignment step I checked the rnaseq metrics of all the samples. Among 40 samples three samples show high percentage of reads mapped to intronic regions. What could be the reason?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Samples   Exonic            Intronic    Intergenic&#xA;Sample1 545479 (12.8%)  2512309 (58.8%) 1217201 (28.5%)&#xA;Sample2 372836 (8.3%)   2556934 (56.8%) 1573032 (34%)&#xA;Sample3 529618 (7.3%)   3934615 (54.5%) 2750720 (38.1%)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also see that 35-40k genes among 58k genes having &quot;zero&quot; read counts. Is this due to contamination? What could be the reason? What does reads mapping to exonic, intronic, intergenic tell?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: I used hisat2 for alignment. I used human genome &quot;grch38_snp_tran&quot; from hisat2 website. Libraries were generated using ribosome depletion kit.&lt;/p&gt;&#xA;" OwnerUserId="1377" LastEditorUserId="1377" LastEditDate="2018-05-07T16:22:29.827" LastActivityDate="2019-11-24T12:06:28.883" Title="Reads mapped to exonic, intronic and intergenic regions" Tags="|rna-seq|sequence-alignment|rna|reads|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="4248" PostTypeId="1" CreationDate="2018-05-07T20:19:22.317" Score="13" ViewCount="3307" Body="&lt;p&gt;I am using Louvain clustering (1,2) to cluster cells in scRNAseq data, as implemented by &lt;a href=&quot;https://scanpy.readthedocs.io&quot; rel=&quot;noreferrer&quot;&gt;scanpy&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the parameter required for this kind of clustering is the number of neighbors used to construct the neighborhood graph of cells (&lt;a href=&quot;https://scanpy.readthedocs.io/en/latest/api/scanpy.api.pp.neighbors.html#scanpy.api.pp.neighbors&quot; rel=&quot;noreferrer&quot;&gt;docs&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Larger values result in a more global view of the manifold, leading to lower number of clusters, while reducing the number of neighbors goes in the opposite direction. However, it is unclear how to choose this parameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resolution parameter seems to work in the opposite way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know of any methodology and/or rule-of-thumb to define these parameters? E.g. depending on the size of the dataset?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Levine, Jacob H., et al. &quot;Data-driven phenotypic dissection of AML reveals progenitor-like cells that correlate with prognosis.&quot; Cell 162.1 (2015): 184-197.&lt;/li&gt;&#xA;&lt;li&gt;Blondel, Vincent D., et al. &quot;Fast unfolding of communities in large networks.&quot; Journal of statistical mechanics: theory and experiment 2008.10 (2008): P10008.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2018-05-07T22:22:54.430" LastActivityDate="2020-03-28T22:25:58.493" Title="How to decide number of neighbors and resolution for Louvain clustering" Tags="|scrnaseq|clustering|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4249" PostTypeId="1" AcceptedAnswerId="4252" CreationDate="2018-05-07T22:20:07.457" Score="5" ViewCount="12865" Body="&lt;p&gt;I want to define two clusters of cells in my dataset and find marker genes that are specific to one and the other. Is there a way to do this in &lt;code&gt;Seurat&lt;/code&gt;? Say, if I produce two subsets by the &lt;code&gt;SubsetData&lt;/code&gt; function, is there a way to feed them into some other function that would calculate marker genes? If not, what other packages would you recommend for doing that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://satijalab.org/seurat/seurat_clustering_tutorial_part2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/seurat_clustering_tutorial_part2.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just need a way to define &lt;code&gt;ident&lt;/code&gt; myself, the number of levels (2) and assign numbers to each cell (0, 1), and then run &lt;code&gt;DE&lt;/code&gt; between &lt;code&gt;0&lt;/code&gt; and &lt;code&gt;1&lt;/code&gt; clusters which is obvious how to do afterwards.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="181" LastEditDate="2018-05-14T22:18:00.090" LastActivityDate="2018-05-15T16:54:50.990" Title="Manually define clusters in Seurat and determine marker genes" Tags="|r|single-cell|seurat|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4250" PostTypeId="1" CreationDate="2018-05-08T03:08:04.570" Score="0" ViewCount="2535" Body="&lt;p&gt;There is a &lt;code&gt;normalized&lt;/code&gt; expression matrix. I split it into two and want to do &lt;code&gt;DE&lt;/code&gt; on the two cells' subsets. I am having trouble transforming it into the format that &lt;code&gt;DESeq2&lt;/code&gt; would accept. The function that I would think I need to use is the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData = cts,&#xA;                          colData = coldata,&#xA;                          design= ~ batch + condition) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It would be perfect if I could somehow feed two matrixes separately, but I can not find it. Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Update&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I tried the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cells &amp;lt;- c(cell_names, cell_names_cyt)&#xA;groups &amp;lt;- c(rep(&quot;NE&quot;, times=length(cell_names)), rep(&quot;EX&quot;, &#xA;times=length(cell_names_cyt)))&#xA;&#xA;colData &amp;lt;- data.frame(&#xA;  Cells &amp;lt;- cells,&#xA;  Groups &amp;lt;- groups&#xA;)&#xA;&#xA;dds &amp;lt;- DESeqDataSetFromMatrix(countData = df,&#xA;                          colData = colData,&#xA;                          design= ~ Groups)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But getting the error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error in DESeqDataSet(se, design = design, ignoreRank) : &#xA;    some values in assay are not integers&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Of course, they are floats since I already normalized the matrices.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Update&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I did the following since I am using &lt;code&gt;Seurat&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df &amp;lt;- as.data.frame(as.matrix(seurat_object@raw.data))&#xA;df &amp;lt;- df[,colnames(df) %in% cells]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But then the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData = df,&#xA;                          colData = colData,&#xA;                          design= ~ Groups)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Gave me an error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error in DESeqDataSet(se, design = design, ignoreRank) : all variables in design formula must be columns in colData&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="2005" LastEditDate="2018-05-08T03:43:40.927" LastActivityDate="2018-05-09T17:04:56.207" Title="Transform and feed data into DESeq2 with DESeqDataSetFromMatrix" Tags="|r|rna-seq|deseq2|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4251" PostTypeId="1" CreationDate="2018-05-08T03:09:44.757" Score="4" ViewCount="123" Body="&lt;p&gt;I have two different stem cell types and their respective gene expressions from RNAseq. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I noticed that there is differential expression in some of the genes coding for histones as well as transcription factors?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kind of interesting downstream analysis might be available?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it feasible to survey ENCODE for downstream TF binding sites?&#xA;Can anything interesting be investigated with regard to the histone gene data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="823" LastEditorUserId="77" LastEditDate="2018-05-08T07:47:08.790" LastActivityDate="2018-06-02T15:14:39.570" Title="What kind of analysis can be done with differential expression of transcription factors?" Tags="|rna-seq|differential-expression|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4252" PostTypeId="2" ParentId="4249" CreationDate="2018-05-08T07:46:49.747" Score="3" Body="&lt;p&gt;I think you are looking to &lt;code&gt;FindAllMarkers&lt;/code&gt; function from &lt;code&gt;Seurat&lt;/code&gt;. As you said, you just have to define your &lt;code&gt;ident&lt;/code&gt;, that have to have the structure of a table (cell names as names and cluster as value):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pident=as.factor(clusters)&#xA;names(pident)=cellNames    &#xA;object1@ident=pident&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then run the &lt;code&gt;FindAllMarkers&lt;/code&gt; function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FindAllMarkers(object1, min.pct = 0.25, min.diff.pct = 0.25)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can specify several parameters in this function (type of DE to perform, thresholds of expression, etc).&lt;/p&gt;&#xA;" OwnerUserId="678" LastActivityDate="2018-05-08T07:46:49.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4253" PostTypeId="1" AcceptedAnswerId="4254" CreationDate="2018-05-08T11:13:41.853" Score="4" ViewCount="1313" Body="&lt;p&gt;I would like to retrieve gene lines from a GTF file for which I only have exons &amp;amp; transcripts lines (output from &lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cufflinks&lt;/a&gt;) and alternative splicing possible. I need gene lines for compatibility with a pipeline dealing with &lt;a href=&quot;https://www.ensembl.org/info/website/upload/gff.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GTF in Ensembl format&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, I would have gene lines representing the longest transcript - but I am open to discussion about that. The aim would be to separate the genome in genic &amp;amp; intergenic portions - boundaries being defined by start-end coordinates of the genes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sample input (e.g. gene &lt;code&gt;&quot;CUFF.105&quot;&lt;/code&gt; with 5 transcripts): &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat subset_105.test | awk '$3==&quot;transcript&quot;{print $0}'&#xA;&#xA;tig00000046 Cufflinks   transcript  26170   42766   202 +   .   gene_id &quot;CUFF.105&quot;; transcript_id &quot;CUFF.105.3&quot;; FPKM &quot;0.2320304094&quot;; frac &quot;0.197788&quot;; conf_lo &quot;0.164885&quot;; conf_hi &quot;0.299176&quot;; cov &quot;5.188855&quot;;&#xA;tig00000046 Cufflinks   transcript  26170   42766   266 +   .   gene_id &quot;CUFF.105&quot;; transcript_id &quot;CUFF.105.1&quot;; FPKM &quot;0.3061260755&quot;; frac &quot;0.249779&quot;; conf_lo &quot;0.228860&quot;; conf_hi &quot;0.383392&quot;; cov &quot;6.845843&quot;;&#xA;tig00000046 Cufflinks   transcript  26170   39469   470 +   .   gene_id &quot;CUFF.105&quot;; transcript_id &quot;CUFF.105.2&quot;; FPKM &quot;0.5403628578&quot;; frac &quot;0.161731&quot;; conf_lo &quot;0.372512&quot;; conf_hi &quot;0.708214&quot;; cov &quot;12.084038&quot;;&#xA;tig00000046 Cufflinks   transcript  28928   39469   1000    +   .   gene_id &quot;CUFF.105&quot;; transcript_id &quot;CUFF.105.4&quot;; FPKM &quot;1.1485378983&quot;; frac &quot;0.233911&quot;; conf_lo &quot;0.853387&quot;; conf_hi &quot;1.443688&quot;; cov &quot;25.684547&quot;;&#xA;tig00000046 Cufflinks   transcript  29614   42766   181 +   .   gene_id &quot;CUFF.105&quot;; transcript_id &quot;CUFF.105.5&quot;; FPKM &quot;0.2087076495&quot;; frac &quot;0.156792&quot;; conf_lo &quot;0.137112&quot;; conf_hi &quot;0.280304&quot;; cov &quot;4.667292&quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What it came to my mind is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;sort the file by &lt;code&gt;$4&lt;/code&gt; (numeric: n) and &lt;code&gt;$5&lt;/code&gt; (reverse: nr) (&lt;code&gt;sort -k4,4n -k5,5nr&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;take the 1st line&lt;/li&gt;&#xA;&lt;li&gt;remove unnecessary fields from &lt;code&gt;$9&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But this would fail for transcripts annotated in the reverse strand, and I won't be able to deal (how to fill) the &lt;code&gt;$6&lt;/code&gt;  (score - a floating point value). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas of an existing method that could help me retrieve those gene lines?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Options to retrieve those lines directly from Cufflinks would also be accepted (I didn't find a suitable parameter in the &lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/cufflinks/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt;). &lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2018-05-08T14:16:09.467" Title="get gene lines from gtf file" Tags="|sequence-annotation|file-formats|gtf|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4254" PostTypeId="2" ParentId="4253" CreationDate="2018-05-08T13:03:24.770" Score="1" Body="&lt;p&gt;Assuming none of the genes have duplicate entries across chromosomes (or strand or other biologically implausible things) in R one can:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(rtracklayer)&#xA;library(GenomicRanges)&#xA;gtf = import.gff(&quot;file.gtf&quot;)&#xA;grl = split(gtf, gtf$gene_id)  # This produces a GRangesList&#xA;grl = endoapply(grl, function(x) {&#xA;    foo = x[1]&#xA;    foo$type = 'gene'&#xA;    start(foo) = min(start(x))&#xA;    end(foo) = max(end(x))&#xA;    return(c(foo, x))&#xA;    })&#xA;gr = unlist(gr)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can then either write &lt;code&gt;gr&lt;/code&gt; out to a file (either with &lt;code&gt;export()&lt;/code&gt; or just printing the rows).&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-05-08T14:16:09.467" LastActivityDate="2018-05-08T14:16:09.467" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4255" PostTypeId="1" AcceptedAnswerId="4302" CreationDate="2018-05-08T14:39:44.183" Score="1" ViewCount="101" Body="&lt;p&gt;I'm using the Ensembl Variant Effect Predictor (VEP) to annotate VCF files, and I am trying to add some additional fields to the output, among these is &lt;code&gt;OVERLAP_BP&lt;/code&gt;, which is listed &lt;a href=&quot;http://www.ensembl.org/info/docs/tools/vep/vep_formats.html#output&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; in available fields:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;OVERLAP_BP - Number of base pairs overlapping with the corresponding variation feature&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's clear that some (many? all?) of these additional fields are only enabled by adding extra options to the VEP command (as listed on the RHS of the &lt;a href=&quot;http://www.ensembl.org/info/docs/tools/vep/script/vep_options.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;options table&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the &lt;code&gt;ZYG&lt;/code&gt; field is made available by adding the &lt;code&gt;--individual&lt;/code&gt; flag to the VEP call. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But &lt;code&gt;OVERLAP_BP&lt;/code&gt; doesn’t appear in the output fields column on that page, and when I add it to the output using &lt;code&gt;--fields&lt;/code&gt;, it is consistently empty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What options are required to make the &lt;code&gt;OVERLAP_BP&lt;/code&gt; output field available to me in VEP?&lt;/p&gt;&#xA;" OwnerUserId="194" LastEditorUserId="57" LastEditDate="2018-10-04T11:45:09.957" LastActivityDate="2018-10-04T11:45:09.957" Title="How to add OVERLAP_BP field to VEP output?" Tags="|vcf|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4256" PostTypeId="2" ParentId="3936" CreationDate="2018-05-08T19:42:09.730" Score="1" Body="&lt;p&gt;According to Jaakkola et al. (ref. 1):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Based on our comparisons, DESeq and Limma without any modifications are not suitable for scRNA-seq data analysis, and yet, they have performed well in the context of bulk RNA-seq data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;They also provided a summary of the comparison between 5 tools (2 designed for Single-Cell, 2 designed for bulk, 1 general): &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Q6Sj6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Q6Sj6.png&quot; alt=&quot;compar&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case I ended up using MAST (ref. 2, &lt;a href=&quot;https://github.com/RGLab/MAST&quot; rel=&quot;nofollow noreferrer&quot;&gt;github&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Jaakkola, M. K., Seyednasrollah, F., Mehmood, A. &amp;amp; Elo, L. L. &lt;a href=&quot;https://academic.oup.com/bib/article/18/5/735/2562772&quot; rel=&quot;nofollow noreferrer&quot;&gt;Comparison of methods to detect differentially expressed genes between single-cell populations.&lt;/a&gt; Briefings in Bioinformatics 18, 735–743 (2017).&lt;/li&gt;&#xA;&lt;li&gt;Finak, G. et al. &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-015-0844-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;MAST: A flexible statistical framework for assessing transcriptional changes and characterizing heterogeneity in single-cell RNA sequencing data.&lt;/a&gt; Genome Biol. 16, 1183 (2015).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-05-08T19:42:09.730" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4257" PostTypeId="1" AcceptedAnswerId="4258" CreationDate="2018-05-08T21:03:45.950" Score="6" ViewCount="420" Body="&lt;p&gt;I currently have several hundred BAM files which were downloaded by someone else. These have remained untouched---before working with them, I would like to double-check that these BAMs have been fully downloaded. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't MD5 Checksums to look at. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to validate the BAMs, I would use something like Picard's ValidateSamFile, &lt;a href=&quot;https://broadinstitute.github.io/picard/command-line-overview.html#ValidateSamFile&quot; rel=&quot;noreferrer&quot;&gt;https://broadinstitute.github.io/picard/command-line-overview.html#ValidateSamFile&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar picard.jar ValidateSamFile \&#xA;      I=input.bam \&#xA;      MODE=SUMMARY&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I could write a bash script, and run through these BAMs, and then read through the Picard output. That is checking whether the BAM is valid though, not whether it was downloaded fully (e.g. the BAM could be downloaded correctly, but be inherently flawed in some other way).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the BAM is downloaded fully, isn't there some &quot;end of file&quot; bytes I could quickly check? Are there other more efficient methods?&lt;/p&gt;&#xA;" OwnerUserId="1770" LastActivityDate="2018-05-08T21:05:11.887" Title="How to validate that BAMs have been downloaded correctly?" Tags="|sam|data-download|picard|validation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4258" PostTypeId="2" ParentId="4257" CreationDate="2018-05-08T21:05:11.887" Score="9" Body="&lt;p&gt;&lt;code&gt;samtools quickcheck&lt;/code&gt; is all you need. From the manual:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Quickly check that input files appear to be intact. Checks that beginning of the file contains a valid header (all formats) containing at least one target sequence and then seeks to the end of the file and checks that an end-of-file (EOF) is present and intact (BAM only).&lt;/p&gt;&#xA;&lt;p&gt;Data in the middle of the file is not read since that would be much more time consuming, so please note that this command will not detect internal corruption, but is useful for testing that files are not truncated before performing more intensive tasks on them.&lt;/p&gt;&#xA;&lt;p&gt;This command will exit with a non-zero exit code if any input files don't have a valid header or are missing an EOF block. Otherwise it will exit successfully (with a zero exit code).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="77" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-05-08T21:05:11.887" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4259" PostTypeId="1" AcceptedAnswerId="4612" CreationDate="2018-05-09T01:33:34.487" Score="4" ViewCount="5331" Body="&lt;p&gt;I compared two manually defined clusters using &lt;code&gt;Seurat&lt;/code&gt; package function &lt;code&gt;FindAllMarkers&lt;/code&gt; and got the output:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yja4Q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yja4Q.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I am confused about three things:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What are &lt;code&gt;pct.1&lt;/code&gt; and &lt;code&gt;pct.2&lt;/code&gt;?&lt;/li&gt;&#xA;&lt;li&gt;How come &lt;code&gt;p-adjusted values&lt;/code&gt; equal to &lt;code&gt;1&lt;/code&gt;? What does it mean?&lt;/li&gt;&#xA;&lt;li&gt;If we take first row, what does &lt;code&gt;avg_logFC&lt;/code&gt; value of &lt;code&gt;-1.35264&lt;/code&gt; mean when we have cluster &lt;code&gt;0&lt;/code&gt; in the &lt;code&gt;cluster&lt;/code&gt; column? Is it that in cluster &lt;code&gt;0&lt;/code&gt; the &lt;code&gt;Cttnbp2&lt;/code&gt; gene downregulated by a factor of &lt;code&gt;2^1.35264&lt;/code&gt;?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2021-11-20T06:31:08.733" Title="Output of Seurat FindAllMarkers parameters" Tags="|r|seurat|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4261" PostTypeId="2" ParentId="4241" CreationDate="2018-05-09T09:46:13.630" Score="5" Body="&lt;p&gt;I would strongly discourage you from making discontinuous axis, it's going to be very confusing for a reader. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The facet plot you proposed seems like a good solution to me. Alternatively you can use log transformation. To demonstrate I made it on simulated data that look appox like yours :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;set.seed(940401)&#xA;&#xA;data.plot &amp;lt;- data.frame(Methyl_Average = c(0.05, rbeta(699, 2, 5) + 0.15, runif(300, 0.18, 0.85)),&#xA;                        Average_Gene_Expression = c(165, exp(rnorm(999, 3))))&#xA;&#xA;library(ggplot2)&#xA;require(gridExtra)&#xA;&#xA;g &amp;lt;- ggplot(data.plot, aes(x = Methyl_Average, y = Average_Gene_Expression))&#xA;g1 &amp;lt;-  g + geom_point(color = &quot;steelblue&quot;) + theme_bw(base_family = &quot;Times&quot;) +&#xA;       ggtitle( &quot;Scatter Plot&quot;) +&#xA;       theme_bw(base_family = &quot;Times&quot;) +&#xA;       theme(plot.title = element_text(hjust = 0.5))&#xA;&#xA;sc_plot &amp;lt;- g1 +&#xA;           labs(x = &quot;Differential Methylation&quot;, y = &quot;Differential Expression&quot;)&#xA;logsc_plot &amp;lt;- g1 + &#xA;              labs(x = &quot;Differential Methylation&quot;, y = &quot;Differential Expression (log10 scale)&quot;) + &#xA;              scale_y_continuous(trans='log10')&#xA;&#xA;grid.arrange(sc_plot, logsc_plot, ncol = 2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4aHFH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4aHFH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2018-05-09T15:49:31.037" LastActivityDate="2018-05-09T15:49:31.037" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4263" PostTypeId="1" CreationDate="2018-05-09T12:30:45.780" Score="4" ViewCount="471" Body="&lt;p&gt;I would like to check the expression of a gene in different groups like Disease vs Normal samples. I want to make a plot out of that to check whether it is significant or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this paper &lt;a href=&quot;https://www.nature.com/articles/nsmb.3211.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;lncRNA&lt;/a&gt; I see that Figure 1C they used RPKM value. But, I'm using read counts data. My question is can I use log cpm to make a plot like that or should I need to use only RPKM?&lt;/p&gt;&#xA;" OwnerUserId="1369" LastEditorUserId="29" LastEditDate="2018-05-09T20:15:40.967" LastActivityDate="2018-06-02T15:34:35.197" Title="Expression of a gene in different groups" Tags="|rna-seq|gene|differential-expression|rna|fpkm|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4264" PostTypeId="1" AcceptedAnswerId="4266" CreationDate="2018-05-09T14:43:31.047" Score="0" ViewCount="235" Body="&lt;p&gt;I am trying to reproduce the preprocessing of paired-end sequencing reads in Lake et al. (ref. 1).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;First, paired-end reads were filtered out if read 1 had more than four non-T bases in the last ten bases (to remove all non-poly(T)-captured contaminated reads), or had one or more bases with a poor quality score (&amp;lt;10). Cell barcode and UMI information were then inferred from the first 12 bases and the next 8 bases of read 1, respectively. The right mate of each read pair was trimmed to remove any portion of the SMART adaptor sequence or large stretches of poly(A) tails (6 consecutive bp or larger).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I was able to filter out reads as described in the first sentence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am unsure on which tool or procedure to use to demultiplex based on cell barcode and UMI, and to trim the right mate. Do you know what I can use or something I can read to understand how to do it?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Lake, B. B. et al. &lt;a href=&quot;https://www.nature.com/articles/nbt.4038&quot; rel=&quot;nofollow noreferrer&quot;&gt;Integrative single-cell analysis of transcriptional and epigenetic states in the human adult brain&lt;/a&gt;. Nature Biotechnology 36, 70–80 (2017).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-05-09T15:37:59.153" Title="Demultiplexing and preprocessing for custom single-nucleus Drop-seq data" Tags="|scrnaseq|reads|trimming|demultiplexing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4265" PostTypeId="2" ParentId="4263" CreationDate="2018-05-09T15:35:18.507" Score="1" Body="&lt;p&gt;For visualization purposes, using log(cpm) is fine. But plot don't check if differences are significant or not, statistical tests do. You can certainly add the results of a proper statistical test to a plot - like adding asterisks to denote a significant difference, for example.&lt;/p&gt;&#xA;" OwnerUserId="1642" LastActivityDate="2018-05-09T15:35:18.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4266" PostTypeId="2" ParentId="4264" CreationDate="2018-05-09T15:37:59.153" Score="0" Body="&lt;p&gt;I didn't notice the sentence:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Paired-end sequencing reads were processed largely as described (&lt;a href=&quot;http://mccarrolllab.com/wp-content/uploads/2016/03/Drop-seqAlignmentCookbookv1.2Jan2016.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mccarrolllab.com/wp-content/uploads/2016/03/Drop-seqAlignmentCookbookv1.2Jan2016.pdf&lt;/a&gt; ), with additional correction steps.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This should provide instructions on how to lead the preprocessing and mapping of data.&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-05-09T15:37:59.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4267" PostTypeId="2" ParentId="4250" CreationDate="2018-05-09T17:04:56.207" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Error in DESeqDataSet(se, design = design, ignoreRank) : all variables&#xA;  in design formula must be columns in colData&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think that means just what it says...you don't have a &quot;Groups&quot; column in coldata.  Did you spotcheck your ColData data frame to make sure it looks the way it should?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; cells &amp;lt;- c(&quot;1&quot;,&quot;2&quot;,&quot;3&quot;)&#xA;&amp;gt; groups &amp;lt;- c(&quot;a&quot;, &quot;b&quot;, &quot;c&quot;)&#xA;&amp;gt; test &amp;lt;- data.frame(Cells &amp;lt;- cells, Groups &amp;lt;- groups)&#xA;&amp;gt; test&#xA;  Cells....cells Groups....groups&#xA;1              1                a&#xA;2              2                b&#xA;3              3                c&#xA;&amp;gt; colnames(test)&#xA;[1] &quot;Cells....cells&quot;   &quot;Groups....groups&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2018-05-09T17:04:56.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4268" PostTypeId="5" CreationDate="2018-05-09T20:16:23.413" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-05-09T20:16:23.413" LastActivityDate="2018-05-09T20:16:23.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4269" PostTypeId="4" CreationDate="2018-05-09T20:16:23.413" Score="0" Body="Fragments Per Kilobase per Million. The number of sequenced fragments that align to a specific gene, divided by the length of the gene in kilobases divided by the number of reads in million. It is the number of reads per gene normalised for the gene length and the library size.&#xD;&#xA;&#xD;&#xA;The term RPKM is an obsolete synonym of FPKM." OwnerUserId="29" LastEditorUserId="29" LastEditDate="2018-05-11T08:46:50.150" LastActivityDate="2018-05-11T08:46:50.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4270" PostTypeId="2" ParentId="4263" CreationDate="2018-05-09T20:23:01.133" Score="2" Body="&lt;p&gt;You should &lt;em&gt;never&lt;/em&gt; use RPKM. &lt;a href=&quot;https://haroldpimentel.wordpress.com/2014/05/08/what-the-fpkm-a-review-rna-seq-expression-units/&quot; rel=&quot;nofollow noreferrer&quot;&gt;It’s simply obsolete&lt;/a&gt; in the age of paired-end sequencing, and has been replaced by FPKM (which is, strictly speaking, a synonym).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The linked blog post explains more generally the problems that measures such as FPKM and CPM suffer from. A more robust measure is the TPM (transcripts per million), which scales CPM by the lengths of the transcripts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, none of these measures perform very well for comparing gene expression across replicates. Since you have read count data, &lt;a href=&quot;http://bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;you should run DESeq2&lt;/a&gt; or edgeR to perform differential expression analysis across all genes, and go from there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the comment noted, the paper you cite does not represent scientific best practice. In fact, a competent reviewer would have rejected this analysis.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2018-05-09T20:23:01.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4271" PostTypeId="1" AcceptedAnswerId="4278" CreationDate="2018-05-10T01:10:01.447" Score="4" ViewCount="2829" Body="&lt;p&gt;I've recently started working with the 10X-Genomics platform with Illumina (MiSeq and HiSeq) for single-cell RNA-Seq. I've been recommended the &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/what-is-cell-ranger&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;cell ranger&quot;&gt;&quot;cellranger&quot;&lt;/a&gt; (version 2.1.0) which I understand handles the barcoding of the platform and performs processing of the data. For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) convert raw Illumina .bcl files into .fastq (demultiplex)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cellranger mkfastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;2) perform alignment, barcode counting, and generate gene-barcode-martices&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cellranger count&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, it is unclear which underlying bioinformatics procedure (with which parameters) are being used here. For this pipeline, I wish to know how the trimming, alignment and counts are being performed. This is important to understand whether further normalisation and quality checking is necessary. What steps would I need to do to perform a similar analysis? &#xA;If for example, I wish to document in more detail what has been performed here in a reproducible manner or reproduce the analysis pipeline on a different platform (such as Nadia) to compare them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, I would like to know the analogous pipeline using open-source tools but at the moment I am first seeking to understand what is going on under the hood when I am running the &lt;code&gt;cellranger&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1855" LastActivityDate="2018-05-14T19:23:16.813" Title="What is cellranger doing in comparison to other methods?" Tags="|rna-seq|sequence-analysis|single-cell|10x-genomics|cellranger|" AnswerCount="2" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4272" PostTypeId="1" AcceptedAnswerId="4275" CreationDate="2018-05-10T09:16:52.510" Score="1" ViewCount="1206" Body="&lt;p&gt;I have used dbSNP build 138 to tag 'rs' ID in my VCF file. But later I came to know that now dbSNP build 150 has been released. I want to update the rs IDs in my VCF file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried but I found a problem for which I need suggestion. With dbSNP build 138, I am getting 450076 total variants, 248236 as known variants and the rest as novel. But when I use the latest dbSNP file, &lt;code&gt;All_20170710.vcf.gz&lt;/code&gt;, I am getting same total variants but 0 as known and all as novel which is definitely wrong. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I checked the reason for this, I found that in the &lt;code&gt;All_20170710.vcf.gz&lt;/code&gt; VCF file, all chromosomes are written as numbers like &lt;code&gt;1&lt;/code&gt;, &lt;code&gt;2&lt;/code&gt;, instead of &lt;code&gt;chr1&lt;/code&gt;, &lt;code&gt;chr2&lt;/code&gt; while in my vcf file, all chromosomes are written as &lt;code&gt;chr1&lt;/code&gt;, &lt;code&gt;chr2&lt;/code&gt;. The dbSNP build 138 file also has the chromosome names with the &lt;code&gt;chr&lt;/code&gt; prefix, and that's why 138th build was giving good results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One solution would be add the &lt;code&gt;chr&lt;/code&gt; word before chromosome number. Is that an efficient way? How do I do that? If not, what is the correct way to use  the latest dbSNP &lt;code&gt;All_20170710.vcf.gz&lt;/code&gt; to tag rs IDs in a vcf file which contains ~32 Billion SNPs instead of &lt;code&gt;dbsnp_138.vcf&lt;/code&gt; which contains only ~6 Billion SNPs? Any suggestions??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="2141" LastEditorUserId="298" LastEditDate="2018-05-10T10:55:24.867" LastActivityDate="2018-05-10T13:21:56.890" Title="update dbSNP ID" Tags="|vcf|sequence-annotation|snp|ngs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4273" PostTypeId="5" CreationDate="2018-05-10T09:54:05.140" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-05-10T09:54:05.140" LastActivityDate="2018-05-10T09:54:05.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4274" PostTypeId="4" CreationDate="2018-05-10T09:54:05.140" Score="0" Body="use for questions regarding plotting and representation of data, combine with tags specific to type of analyses (genome, protein, rna-seq) and language or library (python, R, ggplot2, igv) if applicable. You are welcome to upload a sketch or a scheme of desired outcome." OwnerUserId="57" LastEditorUserId="57" LastEditDate="2018-05-13T11:58:17.523" LastActivityDate="2018-05-13T11:58:17.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4275" PostTypeId="2" ParentId="4272" CreationDate="2018-05-10T11:24:18.290" Score="2" Body="&lt;p&gt;It sounds like you downloaded the dbSNP release for the GRCh38 or GRCh37 genome releases from the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/grc&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genome Reference Consortium&lt;/a&gt; (GRC) which name the chromosomes without the &lt;code&gt;chr&lt;/code&gt; prefix and you were previously using a file that had been built using either hg19 or hg38 which use the &lt;code&gt;chr&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know, dbSNP only offers vcf files built against the GRC genome builds, so you (or someone else) had probably already modified the old file you were using and changed the chromosome names.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also note that if you want the latest version, you should download &lt;a href=&quot;ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606/VCF/All_20180418.vcf.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606/VCF/All_20180418.vcf.gz&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, yes, you can simply change the VCF to use &lt;code&gt;chrN&lt;/code&gt; instead of &lt;code&gt;N&lt;/code&gt;. Note, however, that you will also need to change the name of the mitochondrial chromosome from &lt;code&gt;MT&lt;/code&gt; to &lt;code&gt;chrM&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do all this very easily with &lt;code&gt;sed&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zcat All_20170710.vcf.gz | &#xA;    sed -E '/^[^#]/{s/^([^#]+)/chr\1/; s/^chrMT/chrM/}' &amp;gt; All_20170710.fixed.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The command above will work on most &lt;code&gt;sed&lt;/code&gt; implementations, including Linux (GNU sed) and also macOS (BSD sed). &lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-05-10T13:21:56.890" LastActivityDate="2018-05-10T13:21:56.890" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="4276" PostTypeId="1" AcceptedAnswerId="4283" CreationDate="2018-05-10T12:52:35.713" Score="5" ViewCount="715" Body="&lt;p&gt;I am trying to build a workflow that gets data automatically from databases of sequencing reads (SRA - sequence read archive, ENA - european nucleotide archive). Till now I was pulling everything from SRA and my script have worked, but now I encountered a first entry that is only in ENA (&lt;a href=&quot;https://www.ebi.ac.uk/ena/data/view/ERR2135453&quot; rel=&quot;nofollow noreferrer&quot;&gt;ERR2135453&lt;/a&gt;), &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra/?term=ERR2135453&quot; rel=&quot;nofollow noreferrer&quot;&gt;not in SRA&lt;/a&gt;. And I found that my download script has its limitations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My bash script looks like this (requires aspera &lt;code&gt;ascp&lt;/code&gt; and &lt;code&gt;fastq-dump&lt;/code&gt;) and I wanted to make changes that are commented now&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;ACCESION=$1&#xA;OPENSSH=/my/part/to/key/asperaweb_id_dsa.openssh&#xA;&#xA;# if [ ${ACCESION:0:3} == SRA ]; then&#xA;    URL=anonftp@ftp-private.ncbi.nlm.nih.gov:/sra/sra-instant/reads/ByRun/sra/${ACCESION::3}/${ACCESION::6}/&quot;$ACCESION&quot;/&quot;$ACCESION&quot;.sra data/&#xA;# else&#xA;#    URL=ena_url&#xA;# fi    &#xA;&#xA;ascp -QT -i $OPENSSH $URL&#xA;&#xA;fastq-dump data/&quot;$ACCESION&quot;.sra --outdir data/ --split-files --gzip&#xA;rm data/&quot;$ACCESION&quot;.sra&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The problem is in getting ENA url. Unlike SRA url that I can easily get from the accession (&lt;code&gt;anonftp@ftp-private.ncbi.nlm.nih.gov:/sra/sra-instant/reads/ByRun/sra/${ACCESION::3}/${ACCESION::6}/&quot;$ACCESION&quot;/&quot;$ACCESION&quot;.sra data/&lt;/code&gt;), the ftp server of ENA is bit more confusing. The url corresponding to ERR2135453 accession is &lt;code&gt;ftp://ftp.sra.ebi.ac.uk//vol1/err/ERR213/003/ERR2135453&lt;/code&gt;. Where &lt;code&gt;003&lt;/code&gt; there comes from? Also not all reads are in &lt;code&gt;err&lt;/code&gt; subdirectory. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an easy way how to download reads from ENA knowing just the accession number?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;--- edit ---&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found on the relevant ENA &lt;a href=&quot;https://www.ebi.ac.uk/ena/browse/read-download#downloading_files_aspera&quot; rel=&quot;nofollow noreferrer&quot;&gt;site about downloading data using aspera&lt;/a&gt; that ftp urls should be possible to derive only from accession also in ENA.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Submitted read data files are organised by submission accession number&#xA;  under vol1/ directory in ftp.sra.ebi.ac.uk:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;a href=&quot;ftp://ftp.sra.ebi.ac.uk/vol1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.sra.ebi.ac.uk/vol1/&lt;/a&gt;/&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;where  contains the first 6 letters and&#xA;  numbers of the SRA Submission accession.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;For example, the files submitted in the SRA Submission ERA007448 are&#xA;  available at: &lt;a href=&quot;ftp://ftp.sra.ebi.ac.uk/vol1/ERA007/ERA007448/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.sra.ebi.ac.uk/vol1/ERA007/ERA007448/&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The reason why there is this strange &lt;code&gt;003&lt;/code&gt; will probably be that it's quite recent submission and it has not been properly categorized yet (other accessions I tried were on &quot;expected&quot; urls).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the problem still persist. Is there a way how to fetch data of new submissions using accessions only?&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2018-05-11T23:03:43.350" LastActivityDate="2018-05-25T11:52:34.537" Title="A reliable fetcher of short read using SRA/ENA accession" Tags="|data-download|sratoolkit|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4277" PostTypeId="1" CreationDate="2018-05-10T15:48:00.160" Score="1" ViewCount="22" Body="&lt;p&gt;I'd like to do a targeted screening analysis for trace contaminates.  However, due to the nature of the sample preparations, the retention times I'm getting drift a lot from run to run and XCMS isn't properly identifying my standards and thus affecting my ability to evaluate my samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some trick to getting XCMS to identify the proper retention time that I'm missing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there some other software I should be using? (We have several different instruments from different vendors so it's better if it's not tied to the particular MS)&lt;/p&gt;&#xA;" OwnerUserId="2792" LastEditorUserId="57" LastEditDate="2018-05-10T22:54:23.477" LastActivityDate="2018-05-10T22:54:23.477" Title="XCMS isn't evaluating the proper peak due to retention time drift" Tags="|software-recommendation|software-usage|mass-spectrometry|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4278" PostTypeId="2" ParentId="4271" CreationDate="2018-05-10T21:33:31.400" Score="4" Body="&lt;p&gt;cellranger mkfastq is not necessary anymore.  It used to be that the cellranger software wanted the reads to be interleaved, and you could use cellranger to do that for you if you couldn't do it yourself.  Newer versions of cellranger will take the fastq files just like Illumina's bcl2fastq makes them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cellranger count aligns the reads, filters away duplicates based on UMIs, tries to figure out which cell barcodes really captured cells, and tells you for each cell barcode how many reads hit each gene.  You can go through the .bams, and see which reads were counted for what gene and what barcode, based on the flags and other tags in the .bam&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want another pipeline to compare to, you can use the McCarroll lab's DropSeq pipeline, which is a bunch of little scripts, around the output of STAR aligner (just like 10XGenomics uses).  I have put the same data sets through both 10X and Dropseq, and they return about the same thing.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2018-05-10T21:33:31.400" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4279" PostTypeId="1" AcceptedAnswerId="4282" CreationDate="2018-05-11T00:30:22.247" Score="5" ViewCount="218" Body="&lt;p&gt;In &lt;code&gt;Seurat&lt;/code&gt; we can specify &lt;a href=&quot;https://satijalab.org/seurat/de_vignette.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;multiple methods&lt;/a&gt; for finding &lt;code&gt;DE&lt;/code&gt; genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering when should we use which one, the use cases. As I understand, whole &lt;code&gt;Seurat&lt;/code&gt; package is used for single cell sequencing analysis. I ran different methods for my dataset that was produced from brain cells, around &lt;code&gt;800&lt;/code&gt; cells in each sample, by single cell sequencing. Now I am confused which methods are the most appropriate to use, which results are the most reliable. For single cell sequencing when would you prefer to use one method over the other?&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="181" LastEditDate="2018-05-13T08:04:44.683" LastActivityDate="2018-05-13T08:04:44.683" Title="Which are the use cases for the methods for DE in Seurat" Tags="|r|single-cell|seurat|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4280" PostTypeId="1" CreationDate="2018-05-11T05:22:34.317" Score="1" ViewCount="147" Body="&lt;p&gt;I was perfectly using &lt;code&gt;bioMart&lt;/code&gt;, but it gave an error, I went ahead trying to follow the following &lt;a href=&quot;https://support.bioconductor.org/p/104194/&quot; rel=&quot;nofollow noreferrer&quot;&gt;thread&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And there were some issues with &lt;code&gt;BiocInstaller::biocLite('grimbough/biomaRt')&lt;/code&gt;, namely, I needed to install &lt;code&gt;biocLite(&quot;remotes&quot;)&lt;/code&gt; and then follow the &lt;a href=&quot;https://thecoatlessprofessor.com/programming/rcpp-rcpparmadillo-and-os-x-mavericks--lgfortran-and--lquadmath-error/&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; to solve &lt;code&gt;lgfortran error&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After doing all of that and successfully running &lt;code&gt;BiocInstaller::biocLite('grimbough/biomaRt')&lt;/code&gt; I still got the same error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error in checkDataset(dataset = dataset, mart = mart) : &#xA;    The given dataset:  mmusculus_gene_ensembl , is not valid.  Correct dataset names can be obtained with the listDatasets() function.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am using &lt;code&gt;MacOS High Sierra&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; sessionInfo()&#xA;R version 3.5.0 (2018-04-23)&#xA;Platform: x86_64-apple-darwin15.6.0 (64-bit)&#xA;Running under: macOS High Sierra 10.13.4&#xA;&#xA;Matrix products: default&#xA;BLAS: &#xA; /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib&#xA;LAPACK: /Library/Frameworks/R.framework/Versions/3.5/Resources/lib/libRlapack.dylib&#xA;&#xA;locale:&#xA;[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8&#xA;&#xA;attached base packages:&#xA;[1] parallel  stats4    stats     graphics  grDevices utils     datasets  methods   base     &#xA;&#xA;other attached packages ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2018-05-11T06:56:11.027" LastActivityDate="2018-05-11T06:56:11.027" Title="Error in checkDataset, The given dataset: mmusculus_gene_ensembl , is not valid" Tags="|r|biomart|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4281" PostTypeId="2" ParentId="4251" CreationDate="2018-05-11T07:56:12.000" Score="1" Body="&lt;p&gt;Expanding my comment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;TF usually bind in the upper region of the genes it regulates, so you could see which genes are in the region below the binding site. Perhaps there are some databases for TF. &#xA;I found &lt;a href=&quot;http://amp.pharm.mssm.edu/Harmonizome/dataset/ENCODE+Transcription+Factor+Targets&quot; rel=&quot;nofollow noreferrer&quot;&gt;a website&lt;/a&gt; base on the ENCODE project with the genes regulated by each TF.You can use them for pathway or gene set enrichment analysis, but I am not aware of any specific analysis for genes regulated by a single TF. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The histone data is harder if it was methylation I would say to look if different states of the histones are in different spatial state, but with gene expression is harder to conclude anything. Nevertheless you can check methylation studies to see if gene expression is associated with methylation changes. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-05-11T07:56:12.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4282" PostTypeId="2" ParentId="4279" CreationDate="2018-05-11T10:07:40.193" Score="5" Body="&lt;p&gt;You can take a look at the recently published article: &lt;a href=&quot;https://www.nature.com/articles/nmeth.4612&quot; rel=&quot;noreferrer&quot;&gt;Bias, robustness and scalability in single-cell differential expression analysis&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We evaluated 36 approaches using experimental and synthetic data and found considerable differences in the number and characteristics of the genes that are called differentially expressed. Prefiltering of lowly expressed genes has important effects, particularly for some of the methods developed for bulk RNA-seq data analysis. However, we found that &lt;strong&gt;bulk RNA-seq analysis methods do not generally perform worse&lt;/strong&gt; than those developed specifically for scRNA-seq. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Then you can decide which method fits better to your type of scRNA-seq protocol. Since Seurat offers DESeq2 could be a good idea to use it, but this depends a lot on your data. &lt;/p&gt;&#xA;" OwnerUserId="678" LastActivityDate="2018-05-11T10:07:40.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4283" PostTypeId="2" ParentId="4276" CreationDate="2018-05-11T14:38:27.583" Score="1" Body="&lt;p&gt;Apparently there is no &lt;strong&gt;100% reliable&lt;/strong&gt; solution, but in the most cases, you should be able to use the &lt;a href=&quot;https://github.com/enasequence/enaBrowserTools&quot; rel=&quot;nofollow noreferrer&quot;&gt;enaBrowserTools&lt;/a&gt; python scripts (enaDataGet) to quite reliably download data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The run ERR2135453 is a special case because the sample submission (ERS1939590) for it has been cancelled. The solution for these reads is to download the data manually from the ftp location.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This answer is based on kind assistance of Suran from ebi helpdesk.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2018-05-25T11:52:34.537" LastActivityDate="2018-05-25T11:52:34.537" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4284" PostTypeId="2" ParentId="4276" CreationDate="2018-05-11T15:44:23.840" Score="1" Body="&lt;p&gt;Here are two python functions I use for getting stuff from ENA based on an accession:&#xA;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import os&#xA;import itertools&#xA;from urllib.request import urlopen&#xA;&#xA;def fetch_ENA_files(accession):&#xA;    &quot;&quot;&quot;Get the names of the files matching the ENA accession&quot;&quot;&quot;&#xA;&#xA;    # Get the paths to the files&#xA;    url = &quot;http://www.ebi.ac.uk/ena/data/warehouse/filereport?accession=%s&amp;amp;result=read_run&amp;amp;fields=fastq_ftp&quot; \&#xA;          % accession&#xA;&#xA;    try:&#xA;        paths = urlopen(url).readlines()[1:]&#xA;    except:&#xA;        E.debug(&quot;couldn't access %s&quot; % url)&#xA;        raise&#xA;&#xA;    paths = list(itertools.chain.from_iterable(&#xA;        [p.strip().split(&quot;;&quot;) for p in paths]))&#xA;    filenames = [os.path.basename(x) for x in paths]&#xA;    dl_paths = [os.path.join(*x.split(&quot;/&quot;)[1:]) for x in paths]&#xA;    return filenames, dl_paths&#xA;&#xA;def fetch_ENA(dl_path, outdir, protocol=&quot;ascp&quot;):&#xA;    &quot;&quot;&quot;Fetch fastq from ENA given accession&quot;&quot;&quot;&#xA;&#xA;    if protocol == &quot;ascp&quot;:&#xA;        statement = &quot;&quot;&quot;ascp -QT -P33001 -l 100M -i $ASCP_KEY_PATH&#xA;                       era-fasp@fasp.sra.ebi.ac.uk:/%(dl_path)s %(outdir)s &quot;&quot;&quot; % locals()&#xA;    elif protocol == &quot;http&quot;:&#xA;        fn = os.path.basename(dl_path)&#xA;        outfile = os.path.join(outdir, fn)&#xA;        statement = &quot;wget -O %(outfile)s ftp://ftp.sra.ebi.ac.uk:/%(dl_path)s&quot; % locals()&#xA;&#xA;    return statement&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;fetch_ENA_files&lt;/code&gt; queries the ENA database for the names of all the files associated with a accession and returns a tuple of lists. The first entry in the tuple contains the filenames (which you might need later in your application), the second contains a list of paths on the ENA servers where those files can be found. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;fetch_ENA(dl_path, outdir, protocol)&lt;/code&gt; takes a download path and downloads it to &lt;code&gt;outdir&lt;/code&gt; using with ftp or ascp.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might use it like:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;filenames, paths = fetch_ENA_files(&quot;ERR2135453&quot;)&#xA;for dlp in paths:&#xA;    bash_statement = fetch_ENA(dlp, &quot;downloads&quot;, &quot;ascp&quot;)&#xA;    os.system(bash_statement)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(we don't we actaully use our modules to send it to the cluster)&lt;/p&gt;&#xA;" OwnerUserId="235" LastEditorUserId="581" LastEditDate="2018-05-11T17:06:35.770" LastActivityDate="2018-05-11T17:06:35.770" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4286" PostTypeId="2" ParentId="4216" CreationDate="2018-05-11T23:55:06.153" Score="2" Body="&lt;p&gt;GATK cannot combine VCF files generated by Sniffles, the variant caller that I used to call structural variants (answer from the GATK team).&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-05-11T23:55:06.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4289" PostTypeId="1" CreationDate="2018-05-14T07:58:06.030" Score="0" ViewCount="702" Body="&lt;p&gt;I would like to plot the correlation between two specific genes in my data. I have a matrix with genes in rows and samples in columns, with read counts data. If I want to look at the correlation between 2 genes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) do I need to convert the counts to some other units?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Is normalization needed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I currently have counts returned by &lt;code&gt;featureCounts&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;          Sample1  Sample2  Sample3  Sample4  Sample5&#xA;&#xA;ESAM         803    338       528       841      712&#xA;ESCO1        1594   1401      847       392      611&#xA;ESCO2         274   778       288       77       204&#xA;ESD          6192   3502      2013      1573     1808&#xA;ESF1         1356   1497      1011      372      701&#xA;ESM1         398    58        114       582      135 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried this way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I transposed the above data making genes as columns and sample names as rows. I removed the rows. Lets say the genes as columns is in dataframe &quot;df&quot;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df.mat &amp;lt;- cor(df)&#xA;colnames(df.mat) &amp;lt;- NULL&#xA;rownames(df.mat) &amp;lt;- NULL&#xA;heatmap.2(df.mat, trace=&quot;none&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have an error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in .External.graphics(C_layout, num.rows, num.cols, mat, as.integer(num.figures),  : &#xA;  invalid graphics state&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to look at the correlation between ESCO1 and ESCO2. Can anyone tell me how to do that and plot with a small example.&lt;/p&gt;&#xA;" OwnerUserId="1369" LastEditorUserId="77" LastEditDate="2018-05-14T08:49:42.300" LastActivityDate="2018-06-25T07:56:12.137" Title="How to find correlation between two specific genes in same dataset?" Tags="|rna-seq|r|gene|bioconductor|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4290" PostTypeId="2" ParentId="4289" CreationDate="2018-05-14T08:49:10.620" Score="3" Body="&lt;ol&gt;&#xA;&lt;li&gt;I suppose in an idealized world you'd use something like &lt;code&gt;salmon&lt;/code&gt; to get TPMs, which can account for differences in effective gene length by sample. However that's quite likely to be overkill.&lt;/li&gt;&#xA;&lt;li&gt;Not necessary, you're using a with-sample ratio, which wouldn't be changed by normalization.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Here is a small example in R, assuming your matrix or data frame is named &lt;code&gt;m&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;first = which(row.names(m) == &quot;ESCO1&quot;)&#xA;second = which(row.names(m) == &quot;ESCO2&quot;)&#xA;plot(m[first,], m[second,])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or &lt;code&gt;cor(m[first, ], m[second,])&lt;/code&gt;. You'll want a rank-based correlation.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-05-14T08:49:10.620" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="4291" PostTypeId="2" ParentId="4289" CreationDate="2018-05-14T10:08:10.333" Score="0" Body="&lt;p&gt;I'm answering my own question. I tried this way.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df &amp;lt;- read.csv(&quot;df.csv&quot;)&#xA;&#xA;Samples ESAM    ESCO1   ESCO2   ESD ESF1    ESM1&#xA;Sample1 803     1594    274    6192 1356    398&#xA;Sample2 338     1401    778    3502 1497    58&#xA;Sample3 528      847    288    2013 1011    114&#xA;Sample4 841      392    77     1573  372    582&#xA;Sample5 712      611    204    1808  701    135&#xA;&#xA;df2 &amp;lt;- data.frame(df[,-1],row.names = df[,1])&#xA;&#xA;cormat &amp;lt;- round(cor(df2,use=&quot;pairwise.complete.obs&quot;),2)&#xA;head(cormat)&#xA;&#xA;library(ggcorrplot)&#xA;pdf(&quot;eg.pdf&quot;)&#xA;ggcorrplot(cormat,hc.order = TRUE,&#xA;           outline.col = &quot;white&quot;,&#xA;           ggtheme = ggplot2::theme_gray,&#xA;           colors = c(&quot;#6D9EC1&quot;, &quot;white&quot;, &quot;#E46726&quot;))&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4hmEq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4hmEq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1369" LastEditorUserId="48" LastEditDate="2018-05-14T10:11:21.730" LastActivityDate="2018-05-14T10:11:21.730" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4292" PostTypeId="2" ParentId="4160" CreationDate="2018-05-14T14:12:06.127" Score="0" Body="&lt;pre&gt;&lt;code&gt;$eutils = &quot;http://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi&quot;;&#xA;&#xA;   $databank = &quot;gene&quot;;&#xA;   $gene_symbols = array(&quot;A20&quot;,&quot;HBB&quot;,&quot;ABO&quot;);&#xA;&#xA;   $species = array(&quot;homo sapiens&quot;, &quot;Escherichia coli&quot;,&quot;Saccharomyces cerevisiae&quot;);&#xA;&#xA;foreach($gene_symbols as $value =&amp;gt; $GS){&#xA;    $Species = $species[$value];&#xA;&#xA;&#xA;    $esearch = &quot;$GS%AND%alive[prop]%AND+$Species[orgn]&quot;;&#xA;    $esearch_test = &quot;$eutils?db=$databank&amp;amp;retmode=xml&amp;amp;retmax=10&amp;amp;sort=relevance&amp;amp;term=$esearch&quot;;&#xA;    $result = file_get_contents($esearch_test);&#xA;    echo &quot;Query: &quot; . $esearch_test. &quot;&amp;lt;br \&amp;gt;\n&quot;;&#xA;    echo &quot;Result: &quot; . $result . &quot;&amp;lt;br \&amp;gt;\n&quot;;&#xA;&#xA;    $xml = simplexml_load_string($result);&#xA;    echo &quot;Count:&amp;lt;br \&amp;gt;\n&quot;;&#xA;    echo $xml-&amp;gt;Count;&#xA;    $C =  $xml-&amp;gt;Count;&#xA;    echo $C;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2736" LastActivityDate="2018-05-14T14:12:06.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4293" PostTypeId="1" AcceptedAnswerId="4345" CreationDate="2018-05-14T14:34:19.500" Score="4" ViewCount="94" Body="&lt;p&gt;I have some data on Copy Number Variation (SNP chip) for a population of samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, I have a set of samples (considered as cases) which display a specific disease phenotype, and another set (considered as controls) which do not. The cases have not a matched-control. All the controls are taken as a random sample from the population that does not present the disease.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For both cases and controls I have the copy number of some regions. How can I compute the copy number change in cases, such as in &lt;a href=&quot;https://docs.gdc.cancer.gov/Data/Bioinformatics_Pipelines/CNV_Pipeline/&quot; rel=&quot;noreferrer&quot;&gt;GDC&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I use (as described in the previously linked page):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;performing tangent normalization, which subtracts variation that is found in a set of normal samples&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and do you know of any tool to perform this computation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data I have is formatted in this way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sample_ID   chrom      start           end  CN&#xA;Sample11       19   11991477      12133823   1&#xA;Sample11        2   52260564      52431658   1&#xA;Sample12        7    5721757       5896192   3&#xA;Sample13       10    2269963       2473585   3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2018-05-22T19:16:58.347" LastActivityDate="2018-05-24T14:08:47.887" Title="Compute copy number from cases and controls" Tags="|cnv|copy-number|snp-chip|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4294" PostTypeId="1" CreationDate="2018-05-14T14:40:08.313" Score="6" ViewCount="756" Body="&lt;p&gt;From a single record, I can get the reference sequence ID from the field rID, but how can I get the total number of different reference sequences stored in the whole SAM file?&#xA;It can be simple as just looping through all records, but I need some other solution.&lt;/p&gt;&#xA;" OwnerUserId="2472" LastActivityDate="2018-11-21T15:48:14.813" Title="Number of reference sequences in a SAM file" Tags="|reference-genome|sam|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4296" PostTypeId="2" ParentId="4294" CreationDate="2018-05-14T15:45:47.297" Score="3" Body="&lt;p&gt;There are multiple ways. Here are two:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;samtools idxstats ‹bamfile›&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;samtools view -H ‹bamfile› | grep '^@SQ'&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Both commands give you a list of the references in the BAM file. To get their number, simply append &lt;code&gt;| wc -l&lt;/code&gt;. But note that &lt;code&gt;idxstats&lt;/code&gt; also outputs a final entry, &lt;code&gt;*&lt;/code&gt;, which doesn’t correspond to any reference but to all unmapped reads. So subtract that from the number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, beware that while this lists all references associated with a BAM file, not every reference necessarily has records associated with it. So there might be no reads mapping to a given reference. This is the third column in the &lt;code&gt;idxstats&lt;/code&gt; output. So, to get only those references with at least one mapped read, we can filter for those:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools idxstats ‹bamfile› | awk '$4 != 0' | head -n-1 | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Oh, and these (and other) commands work on &lt;strong&gt;BAM&lt;/strong&gt; files, not SAM. The &lt;code&gt;idxstats&lt;/code&gt; command additionally needs the BAM index, since it’s actually reading information from that index file, not the BAM (this also means that it only works on sorted BAM files). Working on SAM would require looping through the whole file, which is quite inefficient. You should generally be working with BAM rather than SAM, so this is hopefully not a restriction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To do the same on a SAM file, you need to iterate over the file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{print $3}' ‹samfile› | sort -u | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{refs[$3] = 1} END {for (ref in refs) print ref}' ‹samfile› | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2018-05-15T08:38:57.963" LastActivityDate="2018-05-15T08:38:57.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4297" PostTypeId="1" AcceptedAnswerId="4303" CreationDate="2018-05-14T18:10:30.023" Score="7" ViewCount="15450" Body="&lt;p&gt;In &lt;code&gt;Seurats&lt;/code&gt;' documentation for &lt;code&gt;FindClusters()&lt;/code&gt; function it is written that for around &lt;code&gt;3000&lt;/code&gt; cells the &lt;code&gt;resolution&lt;/code&gt; parameter should be from &lt;code&gt;0.6&lt;/code&gt; and up to &lt;code&gt;1.2&lt;/code&gt;. I am wondering then what should I use if I have &lt;code&gt;60 000&lt;/code&gt; cells? How to determine that? &lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="73" LastEditDate="2024-01-04T21:20:12.490" LastActivityDate="2024-01-04T21:20:12.490" Title="Finding the optimal resolution parameter in Seurat's FindClusters function" Tags="|r|single-cell|seurat|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4298" PostTypeId="2" ParentId="4297" CreationDate="2018-05-14T19:17:48.590" Score="3" Body="&lt;p&gt;That is a very general recommendation. Depending on your experiment, you can get a very different number of clusters with the same number of cells at the same resolution.&lt;/p&gt;&#xA;&lt;p&gt;You can actually use a vector of different resolutions and see which one performs best:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pbmc_small &amp;lt;- FindClusters(&#xA;  object = pbmc_small,&#xA;  reduction.type = &amp;quot;pca&amp;quot;,&#xA;  resolution = c(0.4, 0.8, 1.2),&#xA;  dims.use = 1:10,&#xA;  save.SNN = TRUE&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There is no software tool that will tell you what is the best number of clusters. You will have to check the expression of known genes or cluster markers to determine which clusters make the most biological sense. This step is probably the most difficult part of single-cell analysis.&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="73" LastEditDate="2024-01-04T21:18:50.860" LastActivityDate="2024-01-04T21:18:50.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4299" PostTypeId="2" ParentId="4271" CreationDate="2018-05-14T19:23:16.813" Score="3" Body="&lt;p&gt;The best approach would be to contact 10x Genomics support: support@10xgenomics.com . They usually respond within a few hours.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, you can check the methods from the &lt;a href=&quot;https://www.nature.com/articles/ncomms14049&quot; rel=&quot;nofollow noreferrer&quot;&gt;first 10x Genomics paper&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Cell Ranger Single-Cell Software Suite was used to perform sample&#xA;  demultiplexing, barcode processing and single-cell 3′ gene counting&#xA;  (&lt;a href=&quot;http://software.10xgenomics.com/single-cell/overview/welcome&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://software.10xgenomics.com/single-cell/overview/welcome&lt;/a&gt;). First,&#xA;  sample demultiplexing was performed based on the 8 bp sample index&#xA;  read to generate FASTQs for the Read1 and Read2 paired-end reads, as&#xA;  well as the 14 bp GemCode barcode. Ten basepair UMI tags were&#xA;  extracted from Read2 (14 libraries were made with 5 bp UMI tags, as&#xA;  noted in Supplementary Table 1, due to an earlier iteration of the&#xA;  methods. For these samples, 5 bp UMI tags were extracted from Read2.).&#xA;  Then, Read1, which contains the cDNA insert, was aligned to an&#xA;  appropriate reference genome using STAR35. For mouse cells, mm10 was&#xA;  used and for human cells, hg19 was used. For samples with mouse and&#xA;  human cell mixtures, the union of hg19 and mm10 were used. For ERCC&#xA;  samples, ERCC reference&#xA;  (&lt;a href=&quot;https://tools.thermofisher.com/content/sfs/manuals/cms_095047.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://tools.thermofisher.com/content/sfs/manuals/cms_095047.txt&lt;/a&gt;)&#xA;  was used.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Next, GemCode barcodes and UMIs were filtered. All of the known listed&#xA;  of barcodes that are 1-Hamming-distance away from an observed barcode&#xA;  are considered. Then, the posterior probability that the observed&#xA;  barcode was produced by a sequencing error is computed, given the base&#xA;  qualities of the observed barcode and the prior probability of&#xA;  observing the candidate barcode (taken from the overall barcode count&#xA;  distribution). If the posterior probability for any candidate barcode&#xA;  is at least 0.975, then the barcode is corrected to the candidate&#xA;  barcode with the highest posterior probability. If all candidate&#xA;  sequences are equally probable, then the one appearing first by&#xA;  lexical order is picked.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;UMIs with sequencing quality score &gt;10 were considered valid if they&#xA;  were not homopolymers. Qual=10 implies 90% base call accuracy. A UMI&#xA;  that is 1-Hamming-distance away from another UMI (with more reads) for&#xA;  the same cell barcode and gene is corrected to the UMI with more&#xA;  reads. This approach is nearly identical to that in Jaitin et al.4,&#xA;  and is similar to that in Klein et al.8 (although Klein et al.8 also&#xA;  used UMIs to resolve multimapped reads, which was not implemented&#xA;  here).&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Last, PCR duplicates were marked if two sets of read pairs shared a&#xA;  barcode sequence, a UMI tag, and a gene ID (Ensembl GTFs GRCh37.82,&#xA;  &lt;a href=&quot;ftp://ftp.ensembl.org/pub/grch37/release-84/gtf/homo_sapiens/Homo_sapiens.GRCh37.82.gtf.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ensembl.org/pub/grch37/release-84/gtf/homo_sapiens/Homo_sapiens.GRCh37.82.gtf.gz&lt;/a&gt;&#xA;  and GRCm38.84,&#xA;  &lt;a href=&quot;ftp://ftp.ensembl.org/pub/release-84/gtf/mus_musculus/Mus_musculus.GRCm38.84.gtf.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ensembl.org/pub/release-84/gtf/mus_musculus/Mus_musculus.GRCm38.84.gtf.gz&lt;/a&gt;,&#xA;  were used). Only confidently mapped (MAPQ=255), non-PCR duplicates&#xA;  with valid barcodes and UMIs were used to generate gene-barcode&#xA;  matrix.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Cell barcodes were determined based on distribution of UMI counts. All&#xA;  top barcodes within the same order of magnitude (&gt;10% of the top nth&#xA;  barcode, where n is 1% of the expected recovered cell count) were&#xA;  considered cell barcodes. Number of reads that provide meaningful&#xA;  information is calculated as the product of four metrics: (1) valid&#xA;  barcodes; (2) valid UMI; (3) associated with a cell barcode; and (4)&#xA;  confidently mapped to exons.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Of course, they have since updated the library prep and the Cell Ranger software, so the exact protocol is somewhat different.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2018-05-14T19:23:16.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4300" PostTypeId="1" CreationDate="2018-05-15T09:47:08.303" Score="1" ViewCount="891" Body="&lt;p&gt;I have a RNA seq data which I am trying to identify DEGs. Dataset contains:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Untreated - 2 replicates (time point 1st day)&lt;br&gt;&#xA;Treated - 4 replicates (2 replicates at 3rd day &amp;amp; 2 replicates at 7th day ) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I merged all the samples and tried to identify DEGs untreated vs treated. The 1st day sample is considered as control (untreated) for the study.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is it correct to merge all samples and compare. Or should I compare the two time point treated samples with untreated as 1st day vs 3rd day, 1st day vs 7th day and 3rd day vs 7th day separately.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I tried LRT test in DEseq2 to understand the changes in gene expression across time between untreated and treated. The code is given below.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sampleFiles&amp;lt;-grep('SRR',list.files(directory),value=TRUE)&#xA;sampleCondition&amp;lt;-c('untreated','untreated','treated','treated','treated','treated')&#xA;sampleTable&amp;lt;-data.frame(sampleName=sampleFiles, fileName=sampleFiles, condition=sampleCondition)&#xA;ddsHTSeq&amp;lt;-DESeqDataSetFromHTSeqCount(sampleTable=sampleTable, directory=directory, design=~condition)&#xA;colData(ddsHTSeq)$condition&amp;lt;-factor(colData(ddsHTSeq)$condition, levels=c('untreated','treated'))&#xA;dds &amp;lt;- DESeq(ddsHTSeq, test=&quot;LRT&quot;, reduced=~1)&#xA;resLRT &amp;lt;- results(dds)&#xA;resOrdered &amp;lt;- resLRT[order(resLRT$padj),]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After running the code I am getting huge number of DEGs (~15000 genes) as significant DEGs after applying FDR correction cutoff &amp;lt;0.05.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there any error in mentioning the condition in the code.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can you please tell the condition I have specified for LRT test in the code is enough to get the desired output.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Whether LRT test gives huge no of DEGs after analysis as significant DEGs.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2811" LastEditorUserId="48" LastEditDate="2018-05-15T10:14:08.983" LastActivityDate="2018-05-15T10:14:08.983" Title="Likelihood Ratio Test in DEseq2" Tags="|r|rna-seq|deseq2|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4301" PostTypeId="1" AcceptedAnswerId="4312" CreationDate="2018-05-15T12:32:29.403" Score="4" ViewCount="438" Body="&lt;p&gt;Currently, I am working on NGS data and my aim is to get significance prediction of variants present in the vcf file. As we know about &lt;a href=&quot;http://sift.jcvi.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SIFT Score&lt;/a&gt; for significance score prediction, I am trying to understand how this score works. When I read its &lt;a href=&quot;http://sift.jcvi.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;reference paper&lt;/a&gt;, in the method section, the procedure to determine the sift score starts with patient protein query sequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how to get patient protein sequence from patient data (either vcf file or bam file) (which is not explained in the paper). Rest I can follow by other algorithms like PSI-BLAST &amp;amp; MOTIF.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions....&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="2141" LastEditorUserId="96" LastEditDate="2019-01-15T14:24:01.757" LastActivityDate="2019-01-15T14:24:01.757" Title="Protein sequence from patient data" Tags="|vcf|ngs|variation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4302" PostTypeId="2" ParentId="4255" CreationDate="2018-05-15T14:59:38.343" Score="1" Body="&lt;p&gt;I also opened this as a &lt;a href=&quot;https://github.com/Ensembl/ensembl-vep/issues/204&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github issue&lt;/a&gt;, and received the following response:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It seems there are some inconsistencies on our end. I'm looking into it.&#xA;  Until the fix, using &lt;code&gt;--fields OverlapBP,OverlapPC&lt;/code&gt; works with any output format, while &lt;code&gt;--fields OVERLAP_BP,OVERLAP_PC&lt;/code&gt; works only with default output format.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;A small note that this overlap data is available for structural variants only.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="194" LastActivityDate="2018-05-15T14:59:38.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4303" PostTypeId="2" ParentId="4297" CreationDate="2018-05-15T15:35:18.240" Score="7" Body="&lt;p&gt;Assuming you have an informative selection of variable genes from which you have constructed a number of &lt;strong&gt;useful&lt;/strong&gt; PCs, I'd run a number of iterations with &lt;code&gt;FindClusters()&lt;/code&gt; as described in the other answer, then choose a level which &lt;strong&gt;overclusters&lt;/strong&gt; the dataset (for example, clusters that are visibly separate on a t-SNE or other dimensionality reduction plot should definitely have their own number):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seuratobject &amp;lt;- SetAllIdent(seuratobject, id='chosen.resolution')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Seurat::BuildClusterTree()&#xA;Seurat::FindAllMarkersNode()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Assessing the cluster markers for each node will hopefully give you a good idea on which clusters should be combined. Then you can &quot;combine&quot; the clusters and re-label the cells using something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(plyr)&#xA;cell.labels &amp;lt;- seuratobject@ident&#xA;cell.labels &amp;lt;- mapvalues(cell.labels,&#xA;  from=0:16, # cluster numbers&#xA;  to=c('A', 'B', 'C', 'C', 'D', 'E', 'E', ... )) # etc&#xA;&#xA;seuratobject &amp;lt;- AddMetaData(seuratobject, cell.labels, 'Combined.clusters')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The usefulness of the clustering will very much depend on the selection of variable genes, therefore, depending on the (diversity of the) dataset, you will want to &lt;strong&gt;experiment with selection parameters&lt;/strong&gt; or &lt;strong&gt;subset the dataset&lt;/strong&gt; and repeat the above procedure.&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2018-05-15T15:35:18.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4304" PostTypeId="2" ParentId="4249" CreationDate="2018-05-15T15:49:43.123" Score="3" Body="&lt;p&gt;Seurat has functions for adding metadata and setting identities. Get unique cell names:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cell.labels &amp;lt;- seuratobject@ident&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Replace column and its name with your cluster labels (&lt;a href=&quot;https://bioinformatics.stackexchange.com/a/4303/208&quot;&gt;e.g.&lt;/a&gt;), then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seuratobject &amp;lt;- AddMetaData(seuratobject, metadata=cell.labels)&#xA;seuratobject &amp;lt;- SetAllIdent(seuratobject, id='yourclusterlabels')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Because you want to contrast two clusters against each other, I suggest using &lt;code&gt;FindMarkers()&lt;/code&gt; as opposed to &lt;code&gt;FindAllMarkers()&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FindMarkers(object, ident.1, ident.2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It can also compare combinations of clusters.&lt;/p&gt;&#xA;" OwnerUserId="208" LastEditorUserId="48" LastEditDate="2018-05-15T16:54:50.990" LastActivityDate="2018-05-15T16:54:50.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4305" PostTypeId="2" ParentId="4301" CreationDate="2018-05-15T18:26:18.370" Score="3" Body="&lt;p&gt;You can use SIFT in two different ways. One way is manually via their website. This is done by downloading the protein sequence (fasta) from a reliable repository (e.g., uniprot or NCBI), and change the amino acid of interest. Then calculate the SIFT score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other way, which is probably what you are interested in is with NGS data. Here normally SIFT is already known (precalculated) for all coding SNPs, and you can use that info directly during annotation of your variants. I don't know which variant annotation pipeline you are using, but for example annovar has SIFT data available at their &lt;a href=&quot;http://annovar.openbioinformatics.org/en/latest/user-guide/download/&quot; rel=&quot;nofollow noreferrer&quot;&gt;databases&lt;/a&gt; repository.&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2018-05-15T19:35:31.977" LastActivityDate="2018-05-15T19:35:31.977" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4306" PostTypeId="1" AcceptedAnswerId="4309" CreationDate="2018-05-15T19:19:22.300" Score="9" ViewCount="1642" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a standard definition for &quot;assembly polishing&quot; in the field? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a standard definition for what polishing algorithms do? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My understanding of &quot;polishing&quot; is strongly influenced by &lt;a href=&quot;https://github.com/broadinstitute/pilon/wiki&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pilon&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whereby users input a reference FASTA along with the BAM of aligned reads against this reference FASTA. The polishing algorithm then does alignment to discover local misassemblies and other alignment inconsistencies. These are then &quot;corrected&quot;, i.e. &quot;polished&quot;. &lt;/p&gt;&#xA;" OwnerUserId="1770" LastEditorUserId="29" LastEditDate="2018-05-17T13:32:52.093" LastActivityDate="2018-05-17T13:32:52.093" Title="Is there a standard definition for &quot;assembly polishing&quot;?" Tags="|sequence-alignment|assembly|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4307" PostTypeId="2" ParentId="4306" CreationDate="2018-05-15T21:21:13.363" Score="5" Body="&lt;p&gt;&lt;strong&gt;Disclaimer&lt;/strong&gt;, this answer is based on feeling I got from talks and papers, but I do not have any hard reference supporting it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe that genome polishing is a technique that was introduced for correction of individual bases of long noisy read assemblies - i.e. fixing SNPs and short indels. Guessing a correct base from long reads during assembly seems to be bit more complicated task in comparison to short read assemblies, therefore it is isolated into separated algorithms like &lt;a href=&quot;https://github.com/PacificBiosciences/unanimity&quot; rel=&quot;noreferrer&quot;&gt;Quiver/Arrow&lt;/a&gt;, &lt;a href=&quot;https://github.com/jts/nanopolish&quot; rel=&quot;noreferrer&quot;&gt;Nanopolish&lt;/a&gt; (for correction by long reads) or the tool you mentioned - &lt;a href=&quot;https://github.com/broadinstitute/pilon/wiki&quot; rel=&quot;noreferrer&quot;&gt;Pilon&lt;/a&gt; (correction by short reads).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Miss-assemblies could be corrected even prior long reads for instance by matching the assembly to optical mapping, or by tools like &lt;a href=&quot;https://www.sanger.ac.uk/science/tools/reapr&quot; rel=&quot;noreferrer&quot;&gt;REAPR&lt;/a&gt; or &lt;a href=&quot;https://github.com/ksanao/TGNet&quot; rel=&quot;noreferrer&quot;&gt;TGnet&lt;/a&gt;. However, I have never heard term &quot;genome polishing&quot; used for this type of genome correction.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2018-05-15T21:21:13.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4308" PostTypeId="1" CreationDate="2018-05-15T21:30:21.940" Score="3" ViewCount="1070" Body="&lt;p&gt;Is there a way to do GWAS on phenotype data that is not normally distributed?&lt;/p&gt;&#xA;&lt;p&gt;For example, if you were measuring a trait as a proportion, and the majority of the data consisted of 0.00 with a long tail of variation. Or, perhaps the data was skewed the other way where the majority of the observations were 100% with a long tail towards zero.&lt;/p&gt;&#xA;&lt;p&gt;Is there a GWAS method that is appropriate for these types of distributions?&lt;/p&gt;&#xA;" OwnerUserId="2020" LastEditorUserId="3848" LastEditDate="2020-11-14T09:13:00.543" LastActivityDate="2021-02-17T23:57:13.547" Title="Is there a way to do GWAS on phenotype data that is not normally distributed?" Tags="|genome|statistics|gwas|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4309" PostTypeId="2" ParentId="4306" CreationDate="2018-05-16T01:47:23.723" Score="6" Body="&lt;p&gt;In long-read assembly, &quot;polish&quot; refers to the step to improve the base accuracy of contig sequences. I believe the terminology was originated from the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/23644548&quot; rel=&quot;noreferrer&quot;&gt;HGAP paper&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The final consensus–calling algorithm Quiver, which takes into account all of the underlying data and the raw quality values inherent to SMRT sequencing, then &lt;strong&gt;&lt;em&gt;polishes&lt;/em&gt;&lt;/strong&gt; the assembly for final consensus accuracies in excess of 99.999% (QV of &gt;50)...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Pilon was published more than a year later and in fact, neither its paper nor README mentions word &quot;polish&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, &quot;error correction&quot; refers to the step to fix sequencing errors in raw reads, not in contigs. In Canu/HGAP/Falcon, that step is done by pbdagcon or falcon_sense.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-05-16T01:47:23.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4312" PostTypeId="2" ParentId="4301" CreationDate="2018-05-16T08:30:22.740" Score="5" Body="&lt;p&gt;Ensembl now have a tool called &lt;a href=&quot;https://github.com/Ensembl/ensembl-vep#haplo&quot; rel=&quot;nofollow noreferrer&quot;&gt;Haplosaurus&lt;/a&gt; (still in beta) which will convert your phased VCF into actual protein sequences. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;h2&gt;Haplosaurus&lt;/h2&gt;&#xA;  &#xA;  &lt;p&gt;&lt;code&gt;haplo&lt;/code&gt; is a local tool implementation of the same functionality that&#xA;  powers the Ensembl transcript haplotypes view. It takes phased&#xA;  genotypes from a VCF and constructs a pair of haplotype sequences for&#xA;  each overlapped transcript; these sequences are also translated into&#xA;  predicted protein haplotype sequences. Each variant haplotype sequence&#xA;  is aligned and compared to the reference, and an HGVS-like name is&#xA;  constructed representing its differences to the reference.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;This approach offers an advantage over VEP's analysis, which treats&#xA;  each input variant independently. By considering the combined change&#xA;  contributed by all the variant alleles across a transcript, the&#xA;  compound effects the variants may have are correctly accounted for.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;code&gt;haplo&lt;/code&gt; shares much of the same command line functionality with &lt;code&gt;vep&lt;/code&gt;, and&#xA;  can use VEP caches, Ensembl databases, GFF and GTF files as sources of&#xA;  transcript data; all &lt;code&gt;vep&lt;/code&gt; command line flags relating to this&#xA;  functionality work the same with &lt;code&gt;haplo&lt;/code&gt;.&lt;/p&gt;&#xA;  &#xA;  &lt;h2&gt;Usage&lt;/h2&gt;&#xA;  &#xA;  &lt;p&gt;Input data must be a VCF containing phased genotype data for at&#xA;  least one individual; no other formats are currently supported.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;When using a VEP cache as the source of transcript annotation, the&#xA;  first time you run haplo with a particular cache it will spend some&#xA;  time scanning transcript locations in the cache.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./haplo -i input.vcf -o out.txt -cache&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1332" LastEditorUserId="298" LastEditDate="2018-05-17T11:45:12.590" LastActivityDate="2018-05-17T11:45:12.590" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4313" PostTypeId="2" ParentId="3123" CreationDate="2018-05-16T09:06:55.647" Score="3" Body="&lt;p&gt;I normalized a high throughput dataset for a school project using &lt;code&gt;DESeq&lt;/code&gt; library using the script bellow. The code is based on a lesson I had. My goal was determine the over expressed genes, but the normalization step should be the same.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#####################################################################&#xA;### DATA PRE-PROCESSING: Normalising gene expression distributions&#xA;#####################################################################&#xA;# Genes that are not expressed at a biologically meaningful level &#xA;# in any condition should be discarded to reduce the subset of genes &#xA;# to those that are of interest, and to reduce the number of tests carried out &#xA;# downstream when looking at differential expression.&#xA;# Using a nominal CPM value of 1 (which is equivalent to a log-CPM value of 0 &#xA;# a gene is deemed to be expressed in a given sample if its transformed count &#xA;# is above this threshold, and unexpressed otherwise. &#xA;# Genes must be expressed in at least one group (or roughly any three samples &#xA;# across the entire experiment) to be kept for downstream analysis.&#xA;&#xA;library(&quot;DESeq&quot;)&#xA;&#xA;keep.exprs &amp;lt;- rowSums(cpm&amp;gt;1)&amp;gt;=3 # is filtering is ok&#xA;DGEfiltered &amp;lt;- DGEobject[keep.exprs,, keep.lib.sizes = FALSE]&#xA;&#xA;## Normalisation generally required to ensure that expression distributions&#xA;## of each sample are similar across the entire experiment.&#xA;## Any plot showing the per sample expression distributions, such as a density or boxplot,&#xA;## is useful in determining whether any samples are dissimilar to others. &#xA;## Normalisation by the method of trimmed mean of M-values (TMM)&#xA;## is performed using the calcNormFactors function in edgeR.&#xA;# cpmF = filtered subset # log.cpmF = log cpm filtered subset&#xA;# Copy unnormalized, filtered data to show in boxplot&#xA;DGEf.notnorm &amp;lt;- DGEfiltered # normaliseerde &#xA;DGEf.norm &amp;lt;- calcNormFactors(DGEfiltered, method = &quot;TMM&quot;)&#xA;DGEf.norm$samples$norm.factors&#xA;&#xA;&#xA;# --&amp;gt; scaling factors relatively close to 1 --&amp;gt; mild TMM normalisation&#xA;&#xA;## Boxplots of log-CPM values showing expression distributions &#xA;## for unnormalised data (A) and normalised data (B) for each sample&#xA;par(mfrow = c(1,1))&#xA;par(mar = c(7,4,2,1))&#xA;# Boxplot unnormalised data&#xA;DGEf.notnorm$samples$norm.factors&#xA;# [1] 1 1 1 1 1 1 1 1 1 1 1 1 1 1&#xA;log.cpmF.notnorm &amp;lt;- cpm(DGEf.notnorm, log = TRUE)&#xA;boxplot(log.cpmF.notnorm, las = 2, col = sample.col, cex = 0.9, &#xA;        main = &quot;A. Unnormalised data&quot;, ylab = &quot;Log-cpm&quot;)&#xA;# Boxplot normalised data&#xA;DGEf.norm$samples$norm.factors&#xA;&#xA;log.cpmF.norm &amp;lt;- cpm(DGEf.norm, log = TRUE)&#xA;boxplot(log.cpmF.norm, las = 2, col = sample.col, cex = 0.9, &#xA;        main = &quot;B. Normalised data&quot;, ylab = &quot;Log-cpm&quot;)&#xA;# --&amp;gt; only when scaling factors are not close to 1 you will see a big difference&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2736" LastEditorUserId="57" LastEditDate="2018-05-17T16:50:44.593" LastActivityDate="2018-05-17T16:50:44.593" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4314" PostTypeId="1" CreationDate="2018-05-16T09:48:55.963" Score="0" ViewCount="41" Body="&lt;p&gt;I would like to retrieve all Other-source_anchor on the level of DNA binding from within a Gene Ontology XML file using php. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I try this with my code, I only get Gene Ontology back and not the nodeValues in Other-source_anchor. Could someone tell me how to do this? &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Xml file were I wwant to get the nodeValues from&lt;/h1&gt;&#xA;&#xA;&lt;pre class=&quot;lang-xml prettyprint-override&quot;&gt;&lt;code&gt;&amp;lt;Gene-commentary_heading&amp;gt;GeneOntology&amp;lt;/Gene-commentary_heading&amp;gt;&#xA;      &amp;lt;Gene-commentary_source&amp;gt;&#xA;        &amp;lt;Other-source&amp;gt;&#xA;          &amp;lt;Other-source_pre-text&amp;gt;Provided by&amp;lt;/Other-source_pre-text&amp;gt;&#xA;          &amp;lt;Other-source_anchor&amp;gt;GOA&amp;lt;/Other-source_anchor&amp;gt;&#xA;          &amp;lt;Other-source_url&amp;gt;http://www.ebi.ac.uk/GOA/&amp;lt;/Other-source_url&amp;gt;&#xA;        &amp;lt;/Other-source&amp;gt;&#xA;      &amp;lt;/Gene-commentary_source&amp;gt;&#xA;      &amp;lt;Gene-commentary_comment&amp;gt;&#xA;        &amp;lt;Gene-commentary&amp;gt;&#xA;          &amp;lt;Gene-commentary_type value=&quot;comment&quot;&amp;gt;254&amp;lt;/Gene-commentary_type&amp;gt;&#xA;          &amp;lt;Gene-commentary_label&amp;gt;Function&amp;lt;/Gene-commentary_label&amp;gt;&#xA;          &amp;lt;Gene-commentary_comment&amp;gt;&#xA;            &amp;lt;Gene-commentary&amp;gt;&#xA;              &amp;lt;Gene-commentary_type value=&quot;comment&quot;&amp;gt;254&amp;lt;/Gene-commentary_type&amp;gt;&#xA;              &amp;lt;Gene-commentary_source&amp;gt;&#xA;                &amp;lt;Other-source&amp;gt;&#xA;                  &amp;lt;Other-source_src&amp;gt;&#xA;                    &amp;lt;Dbtag&amp;gt;&#xA;                      &amp;lt;Dbtag_db&amp;gt;GO&amp;lt;/Dbtag_db&amp;gt;&#xA;                      &amp;lt;Dbtag_tag&amp;gt;&#xA;                        &amp;lt;Object-id&amp;gt;&#xA;                          &amp;lt;Object-id_id&amp;gt;3677&amp;lt;/Object-id_id&amp;gt;&#xA;                        &amp;lt;/Object-id&amp;gt;&#xA;                      &amp;lt;/Dbtag_tag&amp;gt;&#xA;                    &amp;lt;/Dbtag&amp;gt;&#xA;                  &amp;lt;/Other-source_src&amp;gt;&#xA;                  &amp;lt;Other-source_anchor&amp;gt;DNA binding&amp;lt;/Other-source_anchor&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;code:&lt;/h1&gt;&#xA;&#xA;&lt;pre class=&quot;lang-php prettyprint-override&quot;&gt;&lt;code&gt;$esearch_test = &quot;https://www.ncbi.nlm.nih.gov/gene/7128?report=xml&amp;amp;format=text&quot;;&#xA;$result = file_get_contents($esearch_test);&#xA;$xml = simplexml_load_string($result);&#xA;$doc = new DOMDocument();&#xA;$doc = DOMDocument::loadXML($xml);&#xA;$c = 1;&#xA;foreach($doc-&amp;gt;getElementsByTagName('Gene-commentary_heading') as $node) {&#xA;    if ($node-&amp;gt;textContent ==&quot;GeneOntology&quot;){&#xA;        echo &quot;&amp;lt;pre&amp;gt;&quot;.&quot;$c: &quot;.$node-&amp;gt;nodeName.&quot;&amp;lt;/pre&amp;gt;&quot;;&#xA;        echo &quot;&amp;lt;pre&amp;gt;&quot;.&quot;$c: &quot;.$node-&amp;gt;nodeValue.&quot;&amp;lt;/pre&amp;gt;&quot;;&#xA;        $cn = 1;&#xA;        foreach ($node-&amp;gt;getElementsByTagName('Other-source_anchor') as $subnode){&#xA;            echo &quot;&amp;lt;pre&amp;gt;&quot;.&quot;$c: &quot;.$subnode-&amp;gt;nodeName.&quot;&amp;lt;/pre&amp;gt;&quot;;&#xA;            echo &quot;&amp;lt;pre&amp;gt;&quot;.&quot;$c: &quot;.$subnode-&amp;gt;nodeValue.&quot;&amp;lt;/pre&amp;gt;&quot;; &#xA;&#xA;            echo $cn++;&#xA;        }&#xA;    }&#xA;    $c++;&#xA;} &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2736" LastEditorUserId="298" LastEditDate="2018-05-17T00:40:55.893" LastActivityDate="2018-05-17T00:40:55.893" Title="NCBI xml-format Printing nodeValues from tags with the same names" Tags="|gene-ontology|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4315" PostTypeId="2" ParentId="4314" CreationDate="2018-05-16T15:14:03.673" Score="3" Body="&lt;p&gt;use a XSLT processor (&lt;a href=&quot;http://php.net/manual/en/class.xsltprocessor.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://php.net/manual/en/class.xsltprocessor.php&lt;/a&gt;) with the following stylesheet.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-xslt prettyprint-override&quot;&gt;&lt;code&gt;&amp;lt;?xml version='1.0' encoding=&quot;UTF-8&quot;?&amp;gt;&#xA;&amp;lt;xsl:stylesheet xmlns:xsl='http://www.w3.org/1999/XSL/Transform' version='1.0'&amp;gt;&#xA;    &amp;lt;xsl:output method=&quot;html&quot;/&amp;gt;&#xA;&#xA;    &amp;lt;xsl:template match=&quot;/&quot;&amp;gt;&#xA;        &amp;lt;div&amp;gt;&#xA;            &amp;lt;xsl:apply-templates select=&quot;//Gene-commentary_heading[ text() = 'GeneOntology']&quot;/&amp;gt;&#xA;        &amp;lt;/div&amp;gt;&#xA;    &amp;lt;/xsl:template&amp;gt;&#xA;&#xA;    &amp;lt;xsl:template match=&quot;Gene-commentary_heading&quot;&amp;gt;&#xA;        &amp;lt;pre&amp;gt;&amp;lt;xsl:value-of select=&quot;name(.)&quot;/&amp;gt;&amp;lt;/pre&amp;gt;&#xA;        &amp;lt;pre&amp;gt;&amp;lt;xsl:value-of select=&quot;.&quot;/&amp;gt;&amp;lt;/pre&amp;gt;&#xA;        &amp;lt;xsl:apply-templates select=&quot;..//Other-source_anchor&quot;/&amp;gt;&#xA;    &amp;lt;/xsl:template&amp;gt;&#xA;&#xA;    &amp;lt;xsl:template match=&quot;Other-source_anchor&quot;&amp;gt;&#xA;        &amp;lt;pre&amp;gt;&amp;lt;xsl:value-of select=&quot;name(.)&quot;/&amp;gt;&amp;lt;/pre&amp;gt;&#xA;        &amp;lt;pre&amp;gt;&amp;lt;xsl:value-of select=&quot;.&quot;/&amp;gt;&amp;lt;/pre&amp;gt;&#xA;    &amp;lt;/xsl:template&amp;gt;&#xA;&#xA;&amp;lt;/xsl:stylesheet&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastEditorUserId="2719" LastEditDate="2018-05-17T00:39:03.397" LastActivityDate="2018-05-17T00:39:03.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4316" PostTypeId="1" CreationDate="2018-05-17T03:04:30.610" Score="2" ViewCount="939" Body="&lt;p&gt;I'm analysing single-cell RNA-Seq data using the 10X Genomics &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/what-is-cell-ranger&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;cellranger&lt;/code&gt;&lt;/a&gt; platform. While they provide reference data for Mouse and Humans, other species require a reference &quot;transcriptome&quot; to be generated with the cellranger &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/advanced/references&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;mkref&lt;/code&gt;&lt;/a&gt; tool. However, I've found it to be very temperamental giving errors such as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gffread genome_GFF3_genes.gff -T -o genome_genes.gtf&#xA;&#xA;cellranger mkgtf genome_genes.gtf genome_genes.filtered.gtf \&#xA;                   --attribute=gene_biotype:protein_coding \&#xA;                   --attribute=gene_biotype:lincRNA \&#xA;                   --attribute=gene_biotype:antisense \&#xA;                   --attribute=gene_biotype:IG_LV_gene \&#xA;                   --attribute=gene_biotype:IG_V_gene \&#xA;                   --attribute=gene_biotype:IG_V_pseudogene \&#xA;                   --attribute=gene_biotype:IG_D_gene \&#xA;                   --attribute=gene_biotype:IG_J_gene \&#xA;                   --attribute=gene_biotype:IG_J_pseudogene \&#xA;                   --attribute=gene_biotype:IG_C_gene \&#xA;                   --attribute=gene_biotype:IG_C_pseudogene \&#xA;                   --attribute=gene_biotype:TR_V_gene \&#xA;                   --attribute=gene_biotype:TR_V_pseudogene \&#xA;                   --attribute=gene_biotype:TR_D_gene \&#xA;                   --attribute=gene_biotype:TR_J_gene \&#xA;                   --attribute=gene_biotype:TR_J_pseudogene \&#xA;                   --attribute=gene_biotype:TR_C_gene&#xA;&#xA;cellranger mkref --genome=&quot;reference-name&quot; \&#xA;                 --fasta=&quot;transcriptome.fasta&quot; \&#xA;                 --genes=&quot;genome_genes.filtered.gtf&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Fatal INPUT FILE error, no valid exon lines in the GTF file: reference_name/genes/genes.gtf&#xA;  Solution: check the formatting of the GTF file. Most likely cause is the difference in chromosome naming between GTF and FASTA file.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;subprocess.CalledProcessError: Command '['STAR', '--runMode', 'genomeGenerate', '--genomeDir', 'reference_name/star', '--runThreadN', '1', '--genomeFastaFiles', 'reference_name/fasta/genome.fa', '--sjdbGTFfile', 'reference_name/genes/genes.gtf', '--limitGenomeGenerateRAM', '17179869184', '--genomeSAsparseD', '1', '--genomeSAindexNbases', '12', '--genomeChrBinNbits', '10']' returned non-zero exit status 104&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;subprocess.CalledProcessError: Command '['STAR', '--runMode', 'genomeGenerate', '--genomeDir', 'reference_name/star', '--runThreadN', '1', '--genomeFastaFiles', 'reference_name/fasta/genome.fa', '--sjdbGTFfile', 'reference_name/genes/genes.gtf', '--limitGenomeGenerateRAM', '17179869184', '--genomeSAsparseD', '1', '--genomeSAindexNbases', '12', '--genomeChrBinNbits', '10']' returned non-zero exit status -11&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What input data is required for the &lt;code&gt;cellranger&lt;/code&gt; to construct a reference? What format do the fasta and gtf files need be in? Is it possible to use a &quot;transcriptome&quot; rather than a &quot;genome&quot; as a reference here? For example, I have a transcriptome reference with fasta names &lt;code&gt;&amp;gt;name&lt;/code&gt; for each transcript rather than each chromosome.&lt;/p&gt;&#xA;" OwnerUserId="1855" LastActivityDate="2018-12-15T03:01:39.747" Title="What process and input data is required for a cellranger reference transcriptome?" Tags="|scrnaseq|reference-genome|transcriptome|star|cellranger|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4317" PostTypeId="2" ParentId="4316" CreationDate="2018-05-17T06:52:57.880" Score="4" Body="&lt;p&gt;Your problem is caused by using the transcriptome fasta file rather than the genome fasta file. You've already given it transcriptome information with &lt;code&gt;genome_genes.filtered.gtf&lt;/code&gt;, it needs the genomic sequence to go along with that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an aside, your GTF filtering command is unlikely to be useful unless you have a Gencode or Ensembl GTF for human. Have a look at the &lt;code&gt;gene_biotype&lt;/code&gt;s in your file and decide for yourself what (if anything) needs to be filtered. For example, I made a cellranger reference last night for dm3 (the old fruit fly genome) with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cellranger mkgtf annotation.gtf genes.gtf \&#xA;    --attribute=gene_biotype:protein_coding \&#xA;    --attribute=gene_biotype:ncRNA \&#xA;    --attribute=gene_biotype:snRNA \&#xA;    --attribute=gene_biotype:snoRNA \&#xA;    --attribute=gene_biotype:pre_miRNA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That should be customized for relevance. It's quite possible that your GTF doesn't even have &lt;code&gt;gene_biotype&lt;/code&gt; attributes, that's specific to Ensembl I think.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-05-17T06:52:57.880" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4318" PostTypeId="2" ParentId="4080" CreationDate="2018-05-17T08:58:40.483" Score="3" Body="&lt;p&gt;Without a minimal reproducible example (MWE), I can't reproduce the plot but I would suggest using existing &lt;code&gt;volcanoplot&lt;/code&gt; functions such as the &lt;code&gt;limma&lt;/code&gt; package on Bioconductor. Here is a typical workflow for a differential expression analysis that produces a violinplot:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#install package&#xA;source(&quot;https://bioconductor.org/biocLite.R&quot;)&#xA;biocLite(&quot;limma&quot;)&#xA;&#xA;#load package&#xA;library(&quot;limma&quot;)&#xA;&#xA;#define phenotype/experimental conditions for samples&#xA;group = rep(c(&quot;Treatment&quot;,&quot;Control&quot;), &#xA;c(number_of_treated_samples,number_of_controls))&#xA;design = model.matrix(~group);&#xA;colnames(design) = c(&quot;Control&quot;,&quot;TreatmentvsControl&quot;)&#xA;&#xA;#fit empirical Bayes model&#xA;fit &amp;lt;- lmFit(NormalisedExpression, design)&#xA;fit &amp;lt;- eBayes(fit)&#xA;&#xA;#summarise significant results&#xA;tt &amp;lt;- topTable(fit, coef=&quot;TreatmentvsControl&quot;,&#xA;                adjust=&quot;BH&quot;,n=nrow(NormalisedExpression))&#xA;&#xA;#plot results&#xA;volcanoplot(fit, coef=&quot;TreatmentvsControl&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This plot can be customised in a similar manner to base R plots by passing the relevant arguments as shown in the &lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/manuals/limma/man/limma.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;limma documentation&lt;/a&gt;. This is the recommended plot format that readers in the field will be familiar with. While I do not recommend deviating from this standard, it is possible to reverse axes in R and the source code for the plot function can be retrieved from R by entering &lt;code&gt;volcanoplot&lt;/code&gt; (without any arguments) if you wish to modify it.&lt;/p&gt;&#xA;" OwnerUserId="1855" LastEditorUserId="1855" LastEditDate="2018-05-22T02:47:26.903" LastActivityDate="2018-05-22T02:47:26.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4319" PostTypeId="1" CreationDate="2018-05-17T18:28:37.097" Score="2" ViewCount="122" Body="&lt;p&gt;I ran verifyBAMID on each 200 chip-seq BAM files with input merged vcf file of Genotypes of 600 samples ( 200 of them are chip-seq individuals). I had AF,AC,AN in the info column which was calculated using these genotypes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;~80% of 200 samples are Caucasians.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am struggling to decide for the samples which have freemix &amp;lt; 2% but chipmix &gt;2%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For sample 1365, I had histone modified chip-seq sample 1365_A with two input samples 1365_Pos_IN_2, 1365_Pos_IN_1. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;histone modified chip-seq BAM 1365_A has .00669  whereas chipmix is 0.3765 where with input samples it does resonable job with correct matching of genotype column and reasonable chipmix and freemix values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is chipmix so high for histone sample?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SEQ_ID  RG  CHIP_ID #SNPS   #READS  AVG_DP  FREEMIX FREELK1 FREELK0 FREE_RH FREE_RA CHIPMIX CHIPLK1 CHIPLK0 CHIP_RH CHIP_RA DPREF   RDPHET  RDPALT&#xA;1365_A  ALL 1365_A  683278  436495  0.64    **0.00669** 160631.69   160867.97   NA  NA  **0.37645** 145839.46   194576.92   NA  NA  0.657   0.9478  0.9369&#xA;1365_Pos_IN_2   ALL 1365_A  683278  1770725 2.59    **0.00320** 676736.66   676773.49   NA  NA  **0.00298** 425582.72   426459.54   NA  NA  2.598   0.9965  0.9923&#xA;1365_Pos_IN_1   ALL 1365_A  683278  1127574 1.65    **0.00000** 446421.43   446421.43   NA  NA  **0.00193** 254020.46   254353.15   NA  NA  1.653   0.9961  0.9982&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What does it mean when your target sample matched with the intended sample genotype from a vcf file with &amp;lt; 2% free mix value, but with chipmix value &gt; 2%?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for sample 1368_A and 1213_A&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1368_A  ALL 1368_A  683278  450800  0.66    **0.01019** 167252.18   167673.56   NA  NA  **0.56004** 158591.79   248605.77   NA  NA  0.677   0.9507  0.9477&#xA;1213_A  ALL 1213_A  683278  945562  1.38    **0.01361** 345881.30   347208.39   NA  NA  **0.97533** 346470.85   647079.17   NA  NA  1.396   0.9783  0.9891&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This is clearly contaminated sample with 1287_A. then why is its chipmix not 0.9?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1215_Pos_IN ALL 1287_A  683278  3336277 4.88    **0.15412** 1360018.71  1407231.58  NA  NA  **0.18590** 1109098.83  1431053.16  NA  NA  4.900   0.9941  0.9917&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2831" LastEditorUserId="298" LastEditDate="2018-05-17T19:20:14.203" LastActivityDate="2018-05-17T19:20:14.203" Title="verifybamid results with freemix &lt; 2% and chipmix &gt; 10%" Tags="|sam|snp|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4320" PostTypeId="1" CreationDate="2018-05-17T20:13:35.170" Score="0" ViewCount="69" Body="&lt;p&gt;The &lt;a href=&quot;https://www.rcsb.org/pdb/ligand/drugMapping.do&quot; rel=&quot;nofollow noreferrer&quot;&gt;drug and drug target mapping&lt;/a&gt; tool shows the &lt;strong&gt;top 3&lt;/strong&gt; PDB IDs for each drug based on the drug target sequence similarity search and ligand id search. Is there a way to get all the PDB ids of the drug-target PDB files and not only the top 3?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know you can get this information after clicking the &quot;find&quot; button under the last column of the output table (the column named &quot;All PDB&quot;), but I have a query of several hundreds of drugs and I'm not willing to do it manually for each result.&lt;/p&gt;&#xA;" OwnerUserId="1847" LastEditorUserId="48" LastEditDate="2018-05-18T07:56:58.997" LastActivityDate="2024-02-22T07:07:05.960" Title="Automating download of results of Drug and Drug Target Mapping" Tags="|proteins|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4321" PostTypeId="1" CreationDate="2018-05-17T23:04:24.937" Score="2" ViewCount="124" Body="&lt;p&gt;Sorry, I have been trying to work this out desparately all day. &lt;a href=&quot;https://www.biostars.org/p/315458/#315686&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars couldn't find any answer&lt;/a&gt; so I am cross posting here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to get both the left and right reference genome using a sample name in phase VCF file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This works: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools consensus -f lyrata_chr01-short.fasta phasedVCF-short.vcf.gz -s ms01e -H 1 &amp;gt; ms01e-left.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I am trying to make the new Ref Sequence by imputing genotype from the right haplotype, this doesn't work.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools consensus -f lyrata_chr01-short.fasta phasedVCF-short.vcf.gz -s ms01e -H 2 &amp;gt; ms01e-right.fa&#xA;Broken VCF, too few alts at 1:141&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What is the issue ?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my VCF data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  MA605   MA611   MA622   MA625   MA629   Ncm8    Sp154   Sp164   Sp21    Sp3 Sp76    SpNor33 ms01e   ms02g   ms03g   ms04h&#xA;1   141 .   C   T   .   .   .   GT:PG:PG_al:PI  .   .   .   0|1:0|1:C|T:10763   .   .   .   .   .   .   .   .   .   .   .   .&#xA;1   3697    .   T   C   .   .   .   GT:PG:PG_al:PI  .   0|0:0|0:T|T:11398   .   0|1:0|1:T|C:8039    0|1:0|1:T|C:8202    0|1:0|1:T|C:4616    0|1:0|1:T|C:3938    .   0|1:0|1:T|C:5102    .   .   0|1:0|1:T|C:17340   1|0:1|0:C|T:1   .   .   .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;GT&lt;/code&gt; field is empty at &lt;code&gt;POS 141&lt;/code&gt; for &lt;code&gt;SAMPLE ms01e&lt;/code&gt;, but in that situation it should by default pick a &lt;code&gt;REF&lt;/code&gt; allele. Isn't it? &#xA;And, it only is the problem when doing &lt;code&gt;-H 2&lt;/code&gt; not &lt;code&gt;-H 1&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;: I have been trying to fix this issue by playing with &lt;code&gt;-i and -e&lt;/code&gt; using the BCFtools manual. But, still cannot find the fix.&lt;/p&gt;&#xA;" OwnerUserId="1099" LastEditorUserId="96" LastEditDate="2019-02-27T20:15:31.863" LastActivityDate="2019-02-27T20:15:31.863" Title="bcftools consensus not working when using -H flag" Tags="|vcf|vcftools|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4322" PostTypeId="1" CreationDate="2018-05-18T01:19:09.287" Score="1" ViewCount="24" Body="&lt;p&gt;My project is on Mathematical Modeling of the relationship between viral load and antiretroviral therapy. Its purely an academic investigation which is exploratory in nature.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; I am seeking a public-database or a published data set for drug-HIV interactions and in particular drug vs viral load, for example a time series.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Project&lt;/strong&gt; Intend using the different orthogonal polynomials to fit the curve and then test them statistically to know which fits best.&lt;/p&gt;&#xA;&lt;p&gt;Any other suggestions are welcomed.&lt;/p&gt;&#xA;" OwnerUserId="2832" LastEditorUserId="3848" LastEditDate="2022-03-08T15:52:47.160" LastActivityDate="2022-03-08T15:52:47.160" Title="Viral Load and Antiretroviral Therapy Dataset" Tags="|public-databases|covid-19|data-retrieval|virus|data-mining|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4323" PostTypeId="2" ParentId="4320" CreationDate="2018-05-18T09:17:06.460" Score="0" Body="&lt;p&gt;Simpler option would be to not use the query interface  and directly parse the CSV file found on that page.&lt;/p&gt;&#xA;&lt;p&gt;I just read @llrs comment and I guess it's very much the same spirit as my own - download the data and query it programmatically. I see now (was on mobile earlier) that the 'FIND' URL links out to a BLAST search. so it might be as easy as retrieving the relevant protein sequence and replicating the URL. Or if you want to go a different route and finding all PDB identifiers with the given ligand, cross-reference the above data with what you'd find &lt;a href=&quot;http://www.rcsb.org/pdb/rest/customReport.csv?pdbids=*&amp;amp;reportName=Ligands&amp;amp;service=wsfile&amp;amp;format=csv&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="596" LastEditorUserId="73" LastEditDate="2021-06-07T01:30:21.173" LastActivityDate="2021-06-07T01:30:21.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4324" PostTypeId="1" CreationDate="2018-05-18T09:58:18.503" Score="2" ViewCount="125" Body="&lt;p&gt;I have a table with information about SNPs, the table has a total of 55127 rows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I know the number of gene mutations in the table and if a gene is present?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The table is in xls format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MfFs4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MfFs4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2686" LastEditorUserId="939" LastEditDate="2018-05-18T14:02:52.540" LastActivityDate="2018-05-18T14:02:52.540" Title="how to do analysis of a table content SNP?" Tags="|r|snp|mutations|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4325" PostTypeId="2" ParentId="4324" CreationDate="2018-05-18T11:20:09.763" Score="1" Body="&lt;p&gt;For working with a relatively small dataset in tabular format, I would recommend &lt;a href=&quot;https://cran.r-project.org/web/packages/dplyr/vignettes/dplyr.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;dplyr&lt;/a&gt;. You can read in a &lt;code&gt;excel&lt;/code&gt; file using &lt;a href=&quot;https://readxl.tidyverse.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;readxl&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(readxl)&#xA;library(dplyr)&#xA;&#xA;df &amp;lt;- read_excel(FILENAME)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To get the number of mutations per gene:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df %&amp;gt;% group_by(`Gene name`) %&amp;gt;%&#xA;  summarise(num_mutations=n())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want to know if there are mutations in a particular gene:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df %&amp;gt;% filter(`Gene name` == QUERY)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-05-18T11:20:09.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4326" PostTypeId="1" CreationDate="2018-05-18T10:10:20.243" Score="4" ViewCount="1435" Body="&lt;p&gt;KronaTools lacks a R pipeline to convert any data.frame into a krona plot. There is a way to go via a phyloseq-class object, but coming from a data.frame I did not find any documentation on how to transform these.&lt;/p&gt;&#xA;" OwnerDisplayName="user5878028" LastActivityDate="2018-05-18T14:08:43.733" Title="Transform a R data.frame into a krona plot without krona tools?" Tags="|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4327" PostTypeId="2" ParentId="4326" CreationDate="2018-05-18T10:10:20.243" Score="3" Body="&lt;p&gt;First download any Krona plot (.html) file or make one by using the Excel sheet from the Krona Package. Open in text-editor and remove anything below &quot;datasets&quot; tag and/or the first &quot;node&quot; tag till the end of the document.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the path to go: 1) get your data.frame in hierachical order 2) transform to data.tree object 3) transform to xml file 4) gsub until all the tags match krona annotation&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;options(scipen=999)&#xA;options(stringsAsFactors=FALSE)&#xA;library(data.tree)&#xA;library(xml2)&#xA;library(magrittr)&#xA;&#xA;#get your data into a hieracial format in a data.frame   &#xA;t &amp;lt;- structure(list(freq = c(34L, 127L, 42L, 123L, 343L, 13L, 104L, &#xA;21L, 135L, 91L, 167L, 148L, 90L, 46L, 25L, 79L, 1726L), king.n = c(&quot;&quot;, &#xA;&quot;Fungi&quot;, &quot;Fungi&quot;, &quot;Fungi&quot;, &quot;Fungi&quot;, &quot;Fungi&quot;, &quot;Fungi&quot;, &quot;Plantae&quot;, &#xA;&quot;Plantae&quot;, &quot;Plantae&quot;, &quot;Plantae&quot;, &quot;Plantae&quot;, &quot;Plantae&quot;, &quot;Plantae&quot;, &#xA;&quot;Plantae&quot;, &quot;Plantae&quot;, &quot;Plantae&quot;), class.n = c(&quot;&quot;, &quot;Dothideomycetes&quot;, &#xA;&quot;Dothideomycetes&quot;, &quot;Dothideomycetes&quot;, &quot;Dothideomycetes&quot;,     &quot;Dothideomycetes&quot;, &#xA;&quot;Dothideomycetes&quot;, &quot;Liliopsida&quot;, &quot;Magnoliopsida&quot;, &quot;Magnoliopsida&quot;, &#xA;&quot;Magnoliopsida&quot;, &quot;Magnoliopsida&quot;, &quot;Magnoliopsida&quot;, &quot;Magnoliopsida&quot;, &#xA;&quot;Magnoliopsida&quot;, &quot;Magnoliopsida&quot;, &quot;Magnoliopsida&quot;), fam.n = c(&quot;&quot;, &#xA;&quot;&quot;, &quot;Cladosporiaceae&quot;, &quot;Didymellaceae&quot;, &quot;Pleosporaceae&quot;, &quot;Pleosporaceae&quot;, &#xA;&quot;Pleosporaceae&quot;, &quot;Poaceae&quot;, &quot;&quot;, &quot;Asteraceae&quot;, &quot;Asteraceae&quot;, &quot;Asteraceae&quot;, &#xA;&quot;Asteraceae&quot;, &quot;Fabaceae&quot;, &quot;Hydrophyllaceae&quot;, &quot;Hypericaceae&quot;, &#xA;&quot;Malvaceae&quot;), gen.n = c(&quot;&quot;, &quot;&quot;, &quot;Cladosporium&quot;, &quot;&quot;, &quot;Alternaria&quot;, &#xA;&quot;Pyrenophora&quot;, &quot;Pyrenophora&quot;, &quot;Phleum&quot;, &quot;&quot;, &quot;&quot;, &quot;Achillea&quot;, &quot;Centaurea&quot;, &#xA;&quot;Jurinea&quot;, &quot;Trifolium&quot;, &quot;Phacelia&quot;, &quot;Hypericum&quot;, &quot;Malva&quot;), spec.n = c(&quot;&quot;, &#xA;&quot;&quot;, &quot;cladosporioides&quot;, &quot;&quot;, &quot;&quot;, &quot;&quot;, &quot;tritici-repentis&quot;, &quot;pratense&quot;, &#xA;&quot;&quot;, &quot;&quot;, &quot;&quot;, &quot;cyanus&quot;, &quot;&quot;, &quot;&quot;, &quot;&quot;, &quot;&quot;, &quot;&quot;)), .Names = c(&quot;freq&quot;, &#xA;&quot;king.n&quot;, &quot;class.n&quot;, &quot;fam.n&quot;, &quot;gen.n&quot;, &quot;spec.n&quot;), row.names = c(1L, &#xA;3L, 4L, 5L, 8L, 9L, 11L, 15L, 16L, 17L, 18L, 19L, 21L, 22L, 25L, &#xA;26L, 28L), class = &quot;data.frame&quot;)&#xA;&#xA;#if any sequences can not be assigned to the highest hieracial level give them a random name like &quot;NA&quot;&#xA;t[t[,&quot;king.n&quot;]==&quot;&quot;,&quot;king.n&quot;] &amp;lt;- &quot;NA&quot;&#xA;#define the hieracy (do not include the counting var here), the first space is the root category&#xA;t$pathString &amp;lt;- paste(&quot; &quot;,t$king.n,t$class.n,t$fam.n,t$gen.n,t$spec.n,sep = &quot;/&quot;)&#xA;#transform to a data.tree&#xA;t &amp;lt;- as.Node(t)&#xA;#define the sequence frequency calculations&#xA;t$Do(function(x) x$freq &amp;lt;- Aggregate(x, &quot;freq&quot;, sum), traversal = &quot;post-order&quot;)&#xA;print(t,&quot;freq&quot;)&#xA;#get all frequencies for each node&#xA;repval &amp;lt;- t$Get(&quot;freq&quot;)&#xA;&#xA;#tranform into a nested list and afterwards into a xml document to get the nesting&#xA;t &amp;lt;- t %&amp;gt;% as.list(.,mode=&quot;simple&quot;,unname=T)&#xA;t &amp;lt;- as_xml_document(list(root=t))%&amp;gt;%as.character&#xA;&#xA;#from here it is just a bunch of regex&#xA;#split the xml file into one line for each &amp;lt; &#xA;temp &amp;lt;- strsplit(t,&quot;\n&quot;)[[1]][2] %&amp;gt;% strsplit(.,&quot;(?=&amp;lt;)&quot;,perl=T) %&amp;gt;% .[[1]]&#xA;temp &amp;lt;- paste0(temp[c(T,F)],temp[c(F,T)]) %&amp;gt;% gsub(&quot;[0-9]&quot;,&quot;&quot;,.)&#xA;#remove the names nodes and replace by krona annotation&#xA;#replace the closing named tags by generic closing &quot;node&quot; tags&#xA;temp &amp;lt;- gsub(&quot;&amp;lt;/(.*)?&amp;gt;&quot;,&quot;&amp;lt;/node&amp;gt;&quot;,temp) %&amp;gt;% gsub(&quot;&amp;gt;.*$&quot;,&quot;&amp;gt;&quot;,.) &#xA;#replace the named opening tags by generic &quot;node&quot; tags with the &quot;name&quot; attribute, replace all (because they are incomplete) numeric values with a placeholder&#xA;temp &amp;lt;- gsub(&quot;&amp;lt;(?=[^/])(.*)?&amp;gt;&quot;,&#xA;     '&amp;lt;node name=&quot;\\1&quot;&amp;gt;&amp;lt;magnitude&amp;gt;&amp;lt;val&amp;gt;__\\1&amp;lt;/val&amp;gt;&amp;lt;/magnitude&amp;gt;&amp;lt;score&amp;gt;&amp;lt;val&amp;gt;1&amp;lt;/val&amp;gt;&amp;lt;/score&amp;gt;',temp,perl=T)&#xA;temp %&amp;lt;&amp;gt;% gsub(&quot;root&quot;,&quot; &quot;,.)&#xA;#find the placeholders to insert the frequencies and execute&#xA;repos &amp;lt;- grep(&quot;__&quot;,temp)&#xA;for (i in repos)&#xA;  temp[i] &amp;lt;- gsub(&quot;^(.*)(__.*?)(?=&amp;lt;)(.*)&quot;,paste0(&quot;\\1&quot;,repval[which(repos==i)],&quot;\\3&quot;),temp[i],perl=T)&#xA;&#xA;#add the end of the .html file&#xA;temp &amp;lt;- c(temp,&quot;&amp;lt;/krona&amp;gt;&amp;lt;/div&amp;gt;&quot;,&quot;&amp;lt;/body&amp;gt;&quot;,&quot;&amp;lt;/html&amp;gt;&quot;)  %&amp;gt;% unlist&#xA;tout &amp;lt;- paste(temp,collapse=&quot;\n&quot;) &#xA;&#xA;#append your data to the Krona plot&#xA;write(tout,file=&quot;Krona1.html&quot;,append=TRUE) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Et voilá!&lt;/p&gt;&#xA;" OwnerDisplayName="user5878028" LastActivityDate="2018-05-18T10:34:36.707" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4328" PostTypeId="1" CreationDate="2018-05-18T16:25:48.897" Score="4" ViewCount="115" Body="&lt;p&gt;I have this table: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2rkaG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2rkaG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I want to get the rows that are equal to the first three columns, like this: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1PBVw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1PBVw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried these functions, but when I get the index of the lines, r doesn't give the output that I want:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df$obj&amp;lt;-sapply(c(&quot;sample1&quot;, &quot;sample2&quot;, &quot;sample3&quot;), function (comparison) {&#xA;    df$sample1 == df[comparison]&#xA;})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2686" LastEditorUserId="2719" LastEditDate="2018-05-19T07:12:44.153" LastActivityDate="2018-05-22T13:42:29.970" Title="How to compare the contents of a column of the same data table" Tags="|r|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4329" PostTypeId="2" ParentId="4328" CreationDate="2018-05-18T16:44:49.110" Score="3" Body="&lt;p&gt;No need to use &lt;em&gt;dplyr&lt;/em&gt;, we can do this using &lt;em&gt;base&lt;/em&gt;, see below example: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# reproducible example data&#xA;df1 &amp;lt;- data.frame(&#xA;  t(data.frame(&#xA;    one = c(&quot;A&quot;, &quot;A&quot;, &quot;A&quot;, &quot;C&quot;, &quot;C&quot;, &quot;C&quot;),&#xA;    two = c(&quot;A&quot;, &quot;A&quot;, &quot;A&quot;, &quot;A&quot;, &quot;A&quot;, &quot;C&quot;), &#xA;    thr = c(&quot;A&quot;, &quot;A&quot;, &quot;A&quot;, &quot;A&quot;, &quot;A&quot;, &quot;A&quot;), &#xA;    fou = c(&quot;C&quot;, &quot;A&quot;, &quot;A&quot;, &quot;A&quot;, &quot;A&quot;, &quot;A&quot;))), &#xA;  stringsAsFactors = FALSE)&#xA;&#xA;df1&#xA;#     X1 X2 X3 X4 X5 X6&#xA;# one  A  A  A  C  C  C&#xA;# two  A  A  A  A  A  C&#xA;# thr  A  A  A  A  A  A&#xA;# fou  C  A  A  A  A  A&#xA;&#xA;# solution, subset using logical index:&#xA;selection &amp;lt;- df1[df1[, 1] == df1[, 2] &amp;amp; df1[, 1] == df1[, 3], ]&#xA;&#xA;selection&#xA;#     X1 X2 X3 X4 X5 X6&#xA;# one  A  A  A  C  C  C&#xA;# two  A  A  A  A  A  C&#xA;# thr  A  A  A  A  A  A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastEditorUserId="131" LastEditDate="2018-05-22T13:42:29.970" LastActivityDate="2018-05-22T13:42:29.970" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4330" PostTypeId="1" AcceptedAnswerId="4338" CreationDate="2018-05-18T18:27:19.050" Score="4" ViewCount="91" Body="&lt;p&gt;I'm confused why the &lt;a href=&quot;https://github.com/GFA-spec/GFA-spec&quot; rel=&quot;nofollow noreferrer&quot;&gt;GFA format&lt;/a&gt; doesn't have a standard JSON parser. It appears to be related to a decision made several years ago, &lt;a href=&quot;https://github.com/GFA-spec/GFA-spec/issues/15&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The objection appears to be &quot;JSON isn't necessary, as it's just a simple linked list. Users could write their own&quot;. However, I would think this is a useful feature. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are applications I could think of whereby a JSON would be necessary, or at least very convenient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) Is there a standard way to convert GFA into JSON? Or do users write their own scripts? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) Is there a conceptual reason why GFA doesn't convert into JSON? &lt;/p&gt;&#xA;" OwnerUserId="1770" LastEditorUserId="48" LastEditDate="2018-05-21T17:38:24.617" LastActivityDate="2018-05-21T17:38:24.617" Title="Why isn't there a standard method to convert GFA to JSON?" Tags="|assembly|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4331" PostTypeId="1" AcceptedAnswerId="4332" CreationDate="2018-05-19T20:53:57.470" Score="1" ViewCount="234" Body="&lt;p&gt;I want to load a GFF (annotation) file into an R data frame in order to extract some information from it, e.g. the location of transposons in the genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(rtracklayer)&#xA;gff &amp;lt;- rtracklayer::import('c_elegans.PRJNA13758.WS263.annotations.gff3.gz')&#xA;Error: cannot allocate vector of size 210.2 Mb&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It seems that the GFF3 file is too big for loading into R. Do you know if this is limited by R or by my computer's memory?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran this on a GNU/Linux computer.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-05-21T08:09:36.823" Title="GFF file too big to load into R" Tags="|r|bioconductor|sequence-annotation|errors|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4332" PostTypeId="2" ParentId="4331" CreationDate="2018-05-19T22:10:49.897" Score="3" Body="&lt;p&gt;It depends of course on your available RAM on your computer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under Windows you can increase the available RAM in R with memory limit:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; memory.limit()&#xA;[1] 8070&#xA;&amp;gt; memory.limit(size=10000)&#xA;[1] 10000&#xA;&amp;gt; memory.limit()&#xA;[1] 10000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But that is not necessary on linux/unix.&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2018-05-21T08:09:36.823" LastActivityDate="2018-05-21T08:09:36.823" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4333" PostTypeId="1" AcceptedAnswerId="4334" CreationDate="2018-05-19T23:57:36.553" Score="0" ViewCount="366" Body="&lt;p&gt;Single stranded cDNA is used by race pcr and rt-pcr while next-gen sequencing (Illumina) requires double stranded cDNA. What is the reason behind this ? &lt;/p&gt;&#xA;" OwnerUserId="1684" LastActivityDate="2018-05-20T06:47:44.827" Title="single stranded cDNA vs double-stranded cDNA" Tags="|phylogenetics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4334" PostTypeId="2" ParentId="4333" CreationDate="2018-05-20T06:47:44.827" Score="1" Body="&lt;p&gt;Sequencing also requires single stranded DNA, otherwise the sequencing primers can’t bind. Sure there are steps in the protocol where double stranded material exists, but that’s transient. &lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-05-20T06:47:44.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4335" PostTypeId="1" CreationDate="2018-05-21T02:13:34.370" Score="2" ViewCount="1805" Body="&lt;p&gt;I am curious how people make genome track plots such as what can be done with the Bioconductor package &lt;code&gt;ggbio&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For example this plot was made w/ ggbio:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/E56jf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/E56jf.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I think ggbio is great for making fast plots, but it is kind of a pain to make publication quality figures because it is hard to manipulate the plots after they are constructed. I have noticed that ggbio constructed plots do not return a ggplot or gtable object. I can access the grobs from a ggbio tracks object, but they won't be aligned as they would be in the final plot. I like to make shared legends and include them in the plot, and without having a gtable or ggplot object it is hard to add the shared legends in, and I end up having to save them as a separate file which is tedious. FYI I mainly use ggbio for gene tracks + x-axis alignment.&lt;/p&gt;&#xA;&lt;p&gt;Normally I would use ggplot2 to customize the plot extensively, but it doesn't work very well after plotting with ggbio.&lt;/p&gt;&#xA;&lt;p&gt;How can I use ggbio for these types of plots? Is there a better tool instead?&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&#xA;I would get the legends for each individual plot and combine them before adding back to the main plot using something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;leg1.grob = cowplot::get_legend(plot1)&#xA;leg2.grob = cowplot::get_legend(plot2)&#xA;sharedleg.grob = gridExtra::gtable_rbind(leg1,leg2)&#xA;&#xA;mainplot.grob = ggplotGrob(main_plot)&#xA;mainplot.grob = gtable_add_cols(mainplot.grob, &#xA;                                widths = sum(sharedleg.grob$widths))&#xA;mainplot.grob = gtable_add_grobs(mainplot.grob, &#xA;                                 grobs = sharedleg.grob, &#xA;                                 t=t,b=b,l=l,r=r)&#xA;cowplot::ggdraw(mainplot.grob)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've used &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/Gviz.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;gviz&lt;/a&gt; before, but long ago. Do you know if you can extract the grid plot grobs of the gene track plots with Gviz?&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="73" LastEditDate="2021-07-06T01:39:10.323" LastActivityDate="2021-08-05T02:05:25.430" Title="Plotting gene models - alternatives to ggbio" Tags="|r|visualization|ggplot2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4336" PostTypeId="2" ParentId="4328" CreationDate="2018-05-21T09:49:25.477" Score="4" Body="&lt;p&gt;Loop through 3 columns by row using &lt;em&gt;apply&lt;/em&gt;, then convert &lt;em&gt;toupper&lt;/em&gt; case (as we want &lt;code&gt;a&lt;/code&gt; same as &lt;code&gt;A&lt;/code&gt;), get &lt;em&gt;unique&lt;/em&gt;, and get &lt;em&gt;lengths&lt;/em&gt;, if length is 1 then all 3 values are equal:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df1[ lengths(&#xA;  apply(df1[, 1:3], 1, function(i) unique(toupper(i)))&#xA;  ) == 1, ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This would scale better if we want to compare 4, 10, n(?) columns.&lt;/p&gt;&#xA;" OwnerUserId="131" LastEditorUserId="131" LastEditDate="2018-05-22T10:36:30.467" LastActivityDate="2018-05-22T10:36:30.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4337" PostTypeId="1" AcceptedAnswerId="4339" CreationDate="2018-05-21T12:41:02.080" Score="14" ViewCount="2118" Body="&lt;p&gt;Having recently constructed a lot of phylogenetic trees with the module TreeConstruction from Phylo package from Biopython, I've been asked to replace the branch tip labels by the corresponding sequence logos (which I have in the same folder). I thought that it would be more efficient to make a code to generate the logo-trees automatically, as I would have to make a lot of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first idea that I came up with was to see whether the functions used to build the tree had an argument to replace the branch tip labels or to remove them, which I couldn't find. Therefore, I removed the branch tip labels by setting their font size to 0: (following is the code to build the tree)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Modules to build the tree&#xA;from Bio.Phylo.TreeConstruction import DistanceCalculator, DistanceTreeConstructor&#xA;from Bio.Phylo import draw&#xA;from Bio import Phylo, AlignIO&#xA;import subprocess&#xA;import matplotlib&#xA;import matplotlib.pyplot as plt&#xA;&#xA;alignment = AlignIO.read('MotifSeqAligned.fasta', 'fasta') # reading the alignment file&#xA;&#xA;calculator = DistanceCalculator('ident')&#xA;dm = calculator.get_distance(alignment) # distance matrix&#xA;&#xA;constructor = DistanceTreeConstructor()&#xA;tree = constructor.nj(dm) # build with neighbour joining algorithm a tree from dm&#xA;&#xA;Phylo.write(tree, 'TreeToCutOff.nwk', 'newick')&#xA;&#xA;plt.rc('font', size=0)          # controls default text sizes #HERE IS THE SETTING FOR THAT ALLOWS ME TO HIDE THE BRANCH TIP LABELS&#xA;plt.rc('axes', titlesize=14)     # fontsize of the axes title&#xA;plt.rc('xtick', labelsize=10)    # fontsize of the tick labels&#xA;plt.rc('ytick', labelsize=10)    # fontsize of the tick labels&#xA;plt.rc('figure', titlesize=18)   # fontsize of the figure title&#xA;&#xA;draw(tree, do_show=False)&#xA;plt.savefig(&quot;TreeToCutOff.svg&quot;, format='svg', dpi=1200)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;From this code I could get the tree:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/eLc0c.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eLc0c.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I don't know how to get the y coordinates of the branches to add the logos one by one, I built a column of logos with matplotlib, that I intended then to paste on the tree in python. The code to build the column of logos is the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Extract filename from newick&#xA;newickFile = open(&quot;TreeToCutOff.nwk&quot;, 'r').read()&#xA;orderedLogos = [&quot;{}.eps&quot;.format(i) for i in re.split('(\W)', newickFile) if &quot;Profile&quot; in i]&#xA;&#xA;&#xA;#Initialize the figure&#xA;fig = plt.figure()&#xA;&#xA;&#xA;#Add each image one after the other in the right order&#xA;for i, files in enumerate(orderedLogos):&#xA;    img1 = mpimg.imread(files)&#xA;    ax1 = fig.add_subplot(len(orderedLogos), 1, 1+i)&#xA;    ax1.imshow(img1)&#xA;    ax1.set_xticks([])&#xA;    ax1.set_yticks([])&#xA;&#xA;# plt.show()&#xA;plt.savefig(&quot;RowsOfLogos.svg&quot;, format='svg', dpi=1200)&#xA;plt.clf()&#xA;plt.cla()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tamkV.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tamkV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having my tree and the column of logos in .svg or .png, I couldn't find any way to stack them properly. My first idea was to use the library svgutils which seemed to be easy to handle, with the following code: (taken from svgutils tutorials)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import svgutils.transform as sg&#xA;# Assemble&#xA;#create new SVG figure&#xA;fig = sg.SVGFigure(&quot;14cm&quot;, &quot;14cm&quot;)&#xA;&#xA;# load figures&#xA;fig1 = sg.fromfile('TreeToCutOff.svg')&#xA;fig2 = sg.fromfile('RowsOfLogos.svg')&#xA;&#xA;# get the plot objects&#xA;plot1 = fig1.getroot()&#xA;plot2 = fig2.getroot()&#xA;plot2.moveto(280, 100, scale=0.05)&#xA;&#xA;# append plots and labels to figure&#xA;fig.append([plot1, plot2])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But the issue with the output was that the background of the column of logos was white and thus, I was pasting a huge white image with a thin column of logos on the tree. And I couldn't find a way to crop the column of logos with svgutils. I tried the module Image from PIL package to build a tree of logos from .png files but couldn't see the tree used as background after pasting the column of logos.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There may be a way to do what I'm aiming for with matplotlib (which would be to stack 2 files .png, and place the logos all the time at the same distance), but I couldn't work it out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know what the best solution is to make a tree of logos (as in the following image which I could only build manually with inkscape) with python libraries, allowing to automate the process without having to adapt the code depending on the number of branches?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/M7p7a.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/M7p7a.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following is a subset of &quot;MotifSeqAligned.fasta&quot; containing the aligned sequences used to build the trees:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;ProfileCluster0.meme&#xA;---SSNDTTTCCAGGAAD-&#xA;&amp;gt;ProfileCluster1.meme&#xA;YBNRD---TTCYYGGAAT-&#xA;&amp;gt;ProfileCluster10.meme&#xA;---VDKDWTTCTYGGAAT-&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With the 3 corresponding logos:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/4GChu.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4GChu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ATI9L.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ATI9L.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/TjhTt.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TjhTt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The full &quot;MotifSeqAligned.fasta&quot; and all the logos.eps (as I used them instead of .png which is the format asked by the forum) can be found &lt;a href=&quot;https://github.com/bschnide/Tree-Logos&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2842" LastEditorUserId="2842" LastEditDate="2018-05-21T14:58:17.030" LastActivityDate="2018-05-22T10:17:08.633" Title="Biopython Phylogenetic Tree replace branch tip labels by sequence logos" Tags="|python|motifs|phylogenetics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4338" PostTypeId="2" ParentId="4330" CreationDate="2018-05-21T16:55:38.860" Score="4" Body="&lt;p&gt;Short answer: no, there isn't a &quot;standard&quot; method, and there won't be.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Long answer: like TAB-delimited and XML, JSON alone is not a specific format. You have to define a schema to give meanings to data. However, unlike TAB-delimited &quot;format&quot; and XML, there is not an official way to define JSON schema. There have been various attempts but all of them are complicated and none of them are widely adopted. As you can't write down a JSON schema for GFA in the first place, there won't be a &quot;standard&quot; way to convert GFA to JSON. The best you can do is to roll out your own in-house solution, unfortunately.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-05-21T16:55:38.860" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4339" PostTypeId="2" ParentId="4337" CreationDate="2018-05-22T09:29:00.973" Score="8" Body="&lt;p&gt;To expand on my comment from yesterday. You could do this with the &lt;a href=&quot;http://etetoolkit.org&quot; rel=&quot;noreferrer&quot;&gt;ETE Toolkit&lt;/a&gt; (I just copied one logo file rather than converting all 26 to &lt;code&gt;png&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from ete3 import Tree, TreeStyle, faces&#xA;def mylayout(node):&#xA;    if node.is_leaf():&#xA;        logo_face = faces.ImgFace(str.split(node.name, '.')[0] + &quot;.png&quot;) # this doesn't seem to work with eps files. You could try other formats&#xA;        faces.add_face_to_node(logo_face, node, column=0)&#xA;    node.img_style[&quot;size&quot;] = 0  # remove blue dots from nodes&#xA;&#xA;t = Tree(&quot;tree.nwk&quot;, format=3)&#xA;ts = TreeStyle()&#xA;ts.layout_fn = mylayout&#xA;ts.show_leaf_name=False # remove sequence labels&#xA;ts.scale =  10000 # rescale branch lengths so they are longer than the width of the logos&#xA;t.render(&quot;formatted.png&quot;, tree_style = ts, h=3000, w=3000) # you may need to fiddle with dimensions and scaling to get the look you want&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Yd8z6.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Yd8z6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want all of the logos lined up in a column add &lt;code&gt;aligned=True&lt;/code&gt; to &lt;code&gt;faces.add_face_to_node&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="581" LastEditorUserId="581" LastEditDate="2018-05-22T10:17:08.633" LastActivityDate="2018-05-22T10:17:08.633" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4340" PostTypeId="1" AcceptedAnswerId="4430" CreationDate="2018-05-23T09:46:36.980" Score="5" ViewCount="1896" Body="&lt;p&gt;I am trying to index wheat genome using STAR through following command&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;STAR --runMode genomeGenerate --genomeFastaFiles Triticum_aestivum.TGACv1.dna_sm.toplevel.fa --runThreadN 28 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But getting following error,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;terminate called after throwing an instance of 'std::bad_alloc'&#xA;what():  std::bad_alloc&#xA;Aborted (core dumped)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The size of the genome is 13.5 GB and the server capacity is 32 cores with 125 GB RAM, how this error can be resolved?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Memory usage: &lt;a href=&quot;https://i.stack.imgur.com/uKcEQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uKcEQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have decreased the number of threads to 8 but again getting the same error as shown below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; STAR   --runMode genomeGenerate   --runThreadN 8   --genomeDir wheat   --genomeFastaFiles wheat.fa                                                                      &#xA;Jun 04 02:13:31 ..... started STAR run                                                                                                                                                                                           &#xA;Jun 04 02:13:31 ... starting to generate Genome files                                                                                                                                                                            &#xA;terminate called after throwing an instance of 'std::bad_alloc'                                                                                                                                                                  &#xA;what():  std::bad_alloc                                                                                                                                                                                                        &#xA;Aborted (core dumped)                                           &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Even I have tried to use STAR without any threads but still the same error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; STAR   --runMode genomeGenerate   --genomeDir wheat   --genomeFastaFiles wheat.fa                                                                         &#xA;Jun 04 02:19:44 ..... started STAR run                                                                                                                                                                                           &#xA;Jun 04 02:19:44 ... starting to generate Genome files                                                                                                                                                                            &#xA;terminate called after throwing an instance of 'std::bad_alloc'                                                                                                                                                                  &#xA;what():  std::bad_alloc&#xA;Aborted (core dumped)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;May there can be an issue in reference of wheat that I am using, I got this reference from &lt;a href=&quot;ftp://ftp.ensemblgenomes.org/pub/plants/release-39/fasta/triticum_aestivum/dna/Triticum_aestivum.TGACv1.dna.toplevel.fa.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;here.&lt;/a&gt;&#xA;Here are the first ten lines of the reference using head command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head wheat.fa&#xA;&amp;gt;Pt dna:chromosome chromosome:TGACv1:Pt:1:135835:1 REF                                                                                                                                                                           &#xA;ACAGAAATACCCAATATCTTGTTCTAGCAAGATATTGGGTATTTCTGTCTTTTCTTTCTT                                                                                                                                                                     &#xA;CAAAAATTCTTATATGTTAGCAGAAAAACCTTATCCATTAATAGATGGAACTTCAACAGC                                                                                                                                                                     &#xA;AGCTAAGTCTAGAGGGAAGTTGTGAGCATTACGTTCGTGCATTACTTCCATACCAAGGTT                                                                                                                                                                     &#xA;AGCACGGTTGATGATATCAGCCCAAGTATTAATAACGCGACCTTGACTATCAACTACAGA                                                                                                                                                                     &#xA;TTGGTTGAAATTGAAACCATTTAGGTTGAAAGCCATAGTACTAATACCTAAAGCAGTGAA                                                                                                                                                                     &#xA;CCAGATTCCTACTACAGGCCAAGCAGCCAAGAAGAAGTGTAAAGAACGAGAGTTGTTGAA                                                                                                                                                                     &#xA;ACTAGCATATTGGAAGATTAATCGGCCAAAATAACCATGAGCAGCCACAATATTATAAGT                                                                                                                                                                     &#xA;TTCTTCCTCTTGACCAAATTTGTAACCCTCATTAGCAGATTCATTTTCAGTAGTTTCCCT                                                                                                                                                                     &#xA;GATCAAACTAGAGGTTACCAAGGAACCATGCATAGCACTGAATAGGGAACCGCCGAATAC &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How this problem can be resolved? &lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-06-04T09:25:16.297" LastActivityDate="2018-07-25T08:46:00.617" Title="Error creating indices using STAR" Tags="|rna-seq|rna-alignment|star|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="4341" PostTypeId="1" CreationDate="2018-05-23T16:03:01.603" Score="5" ViewCount="2264" Body="&lt;p&gt;I'm looking for single-cell specific methods to construct (using gene expression data) new features that express pathway &quot;level&quot; or &quot;activity&quot;, and then use these for clustering cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One example for bulk RNA-seq is &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-6-225&quot; rel=&quot;noreferrer&quot;&gt;PLAGE&lt;/a&gt;, implemented in the &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-14-7&quot; rel=&quot;noreferrer&quot;&gt;GSVA&lt;/a&gt; R package.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found only one method for single-cell RNA-seq that comes close: &lt;a href=&quot;http://hms-dbmi.github.io/scde/pagoda.html&quot; rel=&quot;noreferrer&quot;&gt;pagoda in the scde R package&lt;/a&gt;. This performs PCAs for each pathway, using its genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A very similar question is &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/3645/classification-supervised-learning-of-expression-data-on-pathway-level&quot;&gt;Classification (supervised learning) of expression data on pathway level&lt;/a&gt;, on bulk/microarray data.&lt;/p&gt;&#xA;" OwnerUserId="208" LastEditorUserId="29" LastEditDate="2018-05-24T11:26:22.330" LastActivityDate="2020-09-03T15:34:44.757" Title="Pathway level analysis of single-cell gene expression" Tags="|r|scrnaseq|pathway|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4344" PostTypeId="2" ParentId="4340" CreationDate="2018-05-24T09:32:27.990" Score="5" Body="&lt;p&gt;Try reducing the number of threads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When multithreading the index creation (and other memory bound tasks), the memory usage increases linearly with the number of threads. If you required a 10 GiB working space work with a single thread, then the requirement for 10 threads would be at 100 GiB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;STAR is particularly memory hungry for the index creation to start with, and your genome is exceptionally large. I’ve had STAR require &gt; 32 GiB of memory for the human genome, with a moderate number of threads. I’m not surprised that it fails with a 4× larger genome and many more threads: it would probably require at least 2× as much RAM as your machine has.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2018-05-24T09:32:27.990" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4345" PostTypeId="2" ParentId="4293" CreationDate="2018-05-24T14:08:47.887" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://portals.broadinstitute.org/cgi-bin/cancer/publications/pub_paper.cgi?mode=view&amp;amp;paper_id=216&amp;amp;p=t&quot; rel=&quot;nofollow noreferrer&quot;&gt;GISTIC&lt;/a&gt; does exactly this, and is very possibly the tool used in your link above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input files need to be in a different format than what you currently have, but if you're using a SNP6 array, there are guides out there that will tell you how to get the proper files from your &lt;code&gt;.cel&lt;/code&gt; files.&lt;/p&gt;&#xA;" OwnerUserId="1269" LastActivityDate="2018-05-24T14:08:47.887" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4346" PostTypeId="1" AcceptedAnswerId="4347" CreationDate="2018-05-24T14:30:02.787" Score="5" ViewCount="667" Body="&lt;p&gt;I am reading Smyth et al. (ref. 1). I want to run differential expression analysis on a bulk RNA-Seq dataset in which each group is composed by 2 samples. In the paper previously cited it is written that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Genes must be expressed in at least one group (or in at least three samples across the entire experiment, where &lt;strong&gt;three was chosen as this is the smallest group size&lt;/strong&gt;) to be kept for downstream analysis.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Is it possible to use limma DE analysis also with groups composed by only 2 samples? NB. It is possible that in this particular dataset the smallest group size is 3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, which alternative should I use?&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Update&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I have 17 samples. My idea is to test one group vs the rest of groups (all together). So at least the reference group would be composed by &gt; 10 samples. In this case, what can be a feasible analysis for DE?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Smyth, G. K., Law, C. W., Alhamdoosh, M., Su, S. &amp;amp; Ritchie, M. E. &lt;a href=&quot;https://f1000research.com/articles/5-1408/v2&quot; rel=&quot;noreferrer&quot;&gt;RNA-seq analysis is easy as 1-2-3 with limma, Glimma and edgeR&lt;/a&gt;. F1000Research 5, 1408 (2016).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2018-05-24T15:49:21.910" LastActivityDate="2018-05-24T16:07:55.560" Title="Smallest group size for differential expression in limma (bulk RNA-Seq)" Tags="|rna-seq|differential-expression|limma|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4347" PostTypeId="2" ParentId="4346" CreationDate="2018-05-24T14:43:57.423" Score="3" Body="&lt;p&gt;The problem with most of the methods is that use the gene's variance for each group, which can't be calculated (reliable) when the sample is &amp;lt;= 2. Also, statistically, it would have extremely low power, so the conclusions couldn't be trusted much. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also calculate the &quot;raw&quot; fold change by yourself (ie without the variance estimation and adjusting). If you had one more sample per group you could use &lt;a href=&quot;https://bioconductor.org/packages/DESeq2&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2&lt;/a&gt; which was &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-014-0550-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;thought&lt;/a&gt; for such cases. But from the paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;However, if there are two or fewer replicates for a condition, these samples do not contribute to outlier detection, as there are insufficient replicates to determine outlier status.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Which makes me unsure if it will work well for just two samples as it won't help to detect outliers genes. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-05-24T14:43:57.423" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4348" PostTypeId="1" AcceptedAnswerId="4349" CreationDate="2018-05-24T15:04:15.903" Score="3" ViewCount="49" Body="&lt;p&gt;I have recently tried a few tools (&lt;a href=&quot;http://mutationfinder.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MutationFinder&lt;/a&gt;, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/CBBresearch/Lu/Demo/tmTools/demo/tmVar/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tmVar&lt;/a&gt;, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/CBBresearch/Lu/Demo/PubTator/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PubTator&lt;/a&gt; etc.) that extract mutations from a given text. However, for my work I need something that will take a list of mutations as an input and return a list of pubmed id in which the mutations has been found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example input:&#xA;mut_list = [Y702F, L483Y]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example output:&#xA;pub_list = [12215417, 18537569]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone give any idea?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="2854" LastActivityDate="2018-05-24T23:33:33.963" Title="How can I get a list of papers where a particular mutation has been mentioned?" Tags="|mutations|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4349" PostTypeId="2" ParentId="4348" CreationDate="2018-05-24T15:23:38.680" Score="5" Body="&lt;p&gt;You could use the pubmed API to query it directly.  Typically I use &lt;a href=&quot;https://biopython.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biopython&lt;/a&gt; for my &lt;a href=&quot;http://biopython.org/DIST/docs/api/Bio.Entrez-module.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;pubmed queries&lt;/a&gt;.  There are some good examples in the &lt;a href=&quot;http://biopython.org/DIST/docs/tutorial/Tutorial.html#htoc129&quot; rel=&quot;nofollow noreferrer&quot;&gt;cookbook&lt;/a&gt; and on other &lt;a href=&quot;https://stackoverflow.com/questions/21665932/is-there-a-way-with-biopython-to-obtain-the-full-abstract-from-a-pubmed-article&quot;&gt;SO posts&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This example from the &lt;a href=&quot;https://biopython.org/DIST/docs/tutorial/Tutorial.html#htoc129&quot; rel=&quot;nofollow noreferrer&quot;&gt;cookbook link&lt;/a&gt; above has the guts of what you need.  You will just have to iterate over your list.  Depending on the number of queries you want to do you might have issues getting throttled by NCBI&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In this example, we will query PubMed for all articles having to do with orchids (see section 2.3 for our motivation). We first check how many of such articles there are:&#xA;&#xA;&amp;gt;&amp;gt;&amp;gt; from Bio import Entrez&#xA;&amp;gt;&amp;gt;&amp;gt; Entrez.email = &quot;A.N.Other@example.com&quot;     # Always tell NCBI who you are&#xA;&amp;gt;&amp;gt;&amp;gt; handle = Entrez.egquery(term=&quot;orchid&quot;)&#xA;&amp;gt;&amp;gt;&amp;gt; record = Entrez.read(handle)&#xA;&amp;gt;&amp;gt;&amp;gt; for row in record[&quot;eGQueryResult&quot;]: ...     &#xA;        if row[&quot;DbName&quot;]==&quot;pubmed&quot;: ...         &#xA;            print(row[&quot;Count&quot;]) 463 &#xA;&#xA;Now we use the Bio.Entrez.efetch function to download the PubMed IDs of these 463 articles:&#xA;&#xA;&amp;gt;&amp;gt;&amp;gt; from Bio import Entrez&#xA;&amp;gt;&amp;gt;&amp;gt; handle = Entrez.esearch(db=&quot;pubmed&quot;, term=&quot;orchid&quot;, retmax=463)&#xA;&amp;gt;&amp;gt;&amp;gt; record = Entrez.read(handle)&#xA;&amp;gt;&amp;gt;&amp;gt; handle.close()&#xA;&amp;gt;&amp;gt;&amp;gt; idlist = record[&quot;IdList&quot;] &#xA;&#xA;This returns a Python list containing all of the PubMed IDs of articles related to orchids&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="964" LastEditorUserId="964" LastEditDate="2018-05-24T23:33:33.963" LastActivityDate="2018-05-24T23:33:33.963" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4350" PostTypeId="1" AcceptedAnswerId="4360" CreationDate="2018-05-24T15:40:47.087" Score="1" ViewCount="75" Body="&lt;p&gt;This error appeared to me, and I have the program connected to the internet. I already installed java 8, but it still doesn't get me the gene set databases.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/n1Ehp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/n1Ehp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2686" LastEditorUserId="48" LastEditDate="2018-05-24T17:30:09.100" LastActivityDate="2018-05-25T06:08:06.667" Title="Retrieving gene matrix from internet in GSEA desktop" Tags="|rna-seq|gsea|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4351" PostTypeId="2" ParentId="4346" CreationDate="2018-05-24T16:07:55.560" Score="3" Body="&lt;p&gt;While I share Llopis' concern about estimating variance from 2 samples, the statement you quoted is about avoiding false positives from genes that are only expressed in a few samples. It's fairly common to exclude genes that expressed in fewer samples than the smallest group even if the number of samples per group is much higher than 3.&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-05-24T16:07:55.560" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4352" PostTypeId="1" AcceptedAnswerId="4354" CreationDate="2018-05-24T16:51:20.290" Score="2" ViewCount="1282" Body="&lt;p&gt;I have a bed-like file in which I have several overlapping regions with associated features. I want to split the overlapping regions into disjoint regions in order to process the file (e.g. with pandas), group by regions and compute an aggregate function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input data format is the following (overlapping are marked with &lt;code&gt;*&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr9    8855925    8986425    -1  *&#xA;chr9    8855965    8986325     0  *&#xA;chr9    8893857    9022105    -1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to split into disjoint intervals, then I want to groupby per-interval and compute an aggregation function (e.g. the mean of the last field):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr9    8855925    8855965    -1&#xA;chr9    8855965    8893857  -0.5&#xA;chr9    8893857    8986325 -0.67&#xA;chr9    8986325    8986425    -1&#xA;chr9    8986425    9022105    -1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I was thinking of using &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools&lt;/a&gt; and &lt;a href=&quot;https://pandas.pydata.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pandas&lt;/a&gt; to aggregate. However, I was unable to find the right tool to do this job in the bedtools/pybedtools documentation.&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2018-05-24T19:40:09.530" LastActivityDate="2020-12-19T00:34:22.217" Title="How to split bedfile into non-overlapping regions and compute aggregation function on duplicate segments" Tags="|bed|bedtools|pybedtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4353" PostTypeId="1" AcceptedAnswerId="4361" CreationDate="2018-05-24T18:05:26.773" Score="4" ViewCount="3827" Body="&lt;p&gt;After running &lt;code&gt;cellranger count&lt;/code&gt; I got two relevant for further analysis folders: &lt;code&gt;filtered_gene_bc_matrices&lt;/code&gt; and &lt;code&gt;raw_gene_bc_matrices&lt;/code&gt;. What is the difference between them? What does &lt;code&gt;cellranger&lt;/code&gt; filter out, and what should we preferably use for further analysis?&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-05-25T12:16:44.313" Title="Raw vs Filtered in the output of cellranger count" Tags="|scrnaseq|10x-genomics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4354" PostTypeId="2" ParentId="4352" CreationDate="2018-05-24T18:46:42.430" Score="3" Body="&lt;p&gt;To make disjoint intervals, you could use BEDOPS &lt;code&gt;bedops --partition&lt;/code&gt;, piping to &lt;code&gt;bedmap --mean&lt;/code&gt; to get the mean signal over disjoint regions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Starting with the input bedgraph file, convert it to five-column BED with GNU &lt;code&gt;awk&lt;/code&gt;, putting the signal in the fifth column per UCSC convention:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk -vOFS=&quot;\t&quot; '{ print $1, $2, $3, &quot;.&quot;, $4 }' /tmp/in.bedgraph | sort-bed - &amp;gt; /tmp/in.bed&#xA;$ cat in.bed&#xA;chr9    8855925 8986425 .   -1&#xA;chr9    8855965 8986325 .   0&#xA;chr9    8893857 9022105 .   -1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The disjoint set from these intervals would look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops --partition /tmp/in.bed&#xA;chr9    8855925 8855965&#xA;chr9    8855965 8893857&#xA;chr9    8893857 8986325&#xA;chr9    8986325 8986425&#xA;chr9    8986425 9022105&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We can pipe these intervals to &lt;code&gt;bedmap --mean&lt;/code&gt; with the original five-column BED file as a &quot;map&quot; file, calculating the mean signal over the disjoint intervals:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops --partition /tmp/in.bed | bedmap --echo --mean --delim '\t' - /tmp/in.bed &amp;gt; /tmp/answer.bedgraph&#xA;$ cat /tmp/answer.bedgraph&#xA;chr9    8855925 8855965 -1.000000&#xA;chr9    8855965 8893857 -0.500000&#xA;chr9    8893857 8986325 -0.666667&#xA;chr9    8986325 8986425 -1.000000&#xA;chr9    8986425 9022105 -1.000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;--mean&lt;/code&gt; option calculates the arithmetic mean. Other aggregation functions are available to process the signal or score (fifth) column in map BED files, like &lt;code&gt;--stdev&lt;/code&gt;, &lt;code&gt;--median&lt;/code&gt;, &lt;code&gt;--mad&lt;/code&gt;, etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When calculating statistics, the default precision is six digits. The &lt;code&gt;--prec &amp;lt;n&amp;gt;&lt;/code&gt; option can be used to specify desired precision in the score result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;code&gt;bedmap --help&lt;/code&gt; or the online documentation for a full description.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To avoid making and cleaning up intermediate files, &lt;code&gt;bash&lt;/code&gt; process substitutions could be used to generate a one-liner:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops --partition &amp;lt;(sort-bed in.bedgraph | awk -vOFS=&quot;\t&quot; '{ print $1, $2, $3, &quot;.&quot;, $4 }') | bedmap --echo --mean --delim '\t' - &amp;lt;(sort-bed in.bedgraph | awk -vOFS=&quot;\t&quot; '{ print $1, $2, $3, &quot;.&quot;, $4 }') &amp;gt; answer.bedgraph&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This can be useful for cleaner pipelines or with processing very large, whole-genome scale inputs on systems with limited disk space.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-05-24T20:52:17.630" LastActivityDate="2018-05-24T20:52:17.630" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4355" PostTypeId="1" AcceptedAnswerId="4356" CreationDate="2018-05-24T20:56:41.603" Score="3" ViewCount="58" Body="&lt;p&gt;I have two bed-like files:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;File1&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1     1     10    .    1    Sample1&#xA;chr1    20    180    .    1    Sample1&#xA;chr1     1      5    .    1    Sample2&#xA;chr1    70    100    .    1    Sample3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;File2&lt;/strong&gt; (reference)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1     1     5     .    0 &#xA;chr1     5    20     .    1&#xA;chr1    50   120     .   -1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'd like to find a way to apply a function (in my case a subtraction) between the first file and the second, using the second as a reference, and producing a results file containing also the indication of the sample name.&#xA;For example, in my case the results file would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1     1      5    .    1    Sample1&#xA;chr1     5     10    .    0    Sample1&#xA;chr1    20     50    .    1    Sample1&#xA;chr1    50    120    .    2    Sample1&#xA;chr1   120    180    .    1    Sample1&#xA;chr1     1      5    .    1    Sample2&#xA;chr1    70    100    .    2    Sample3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I was thinking about implementing it in pandas, but it would be long to do and probably not so efficient. I looked for &lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/statistics/bedmap.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedmap&lt;/a&gt; but it seems to me that this operation can be carried on (with the sum operation) only within a single file during aggregation, not between two different bed files. So maybe it is not the right tool to use in this case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, it would be important in the final file to retain the sample name.&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-05-24T22:10:32.117" Title="How to apply function on bedfile based on a reference bedfile" Tags="|bed|bedtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4356" PostTypeId="2" ParentId="4355" CreationDate="2018-05-24T21:36:56.133" Score="2" Body="&lt;p&gt;You could munge your first file into a BED5 file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk -vOFS=&quot;\t&quot; '{ print $1, $2, $3, $6, $5 }' /tmp/f1.bed | sort-bed - &amp;gt; /tmp/f1.bed5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Sort the second file, if necessary:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sort-bed /tmp/f2.unknownSort.bed5 &amp;gt; /tmp/f2.bed5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Take the union of the two sorted files:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops -u /tmp/f1.bed5 /tmp/f2.bed5 &amp;gt; /tmp/union.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's what the union looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    1       5       .       0       &#xA;chr1    1       5       Sample2 1&#xA;chr1    1       10      Sample1 1&#xA;chr1    5       20      .       1&#xA;chr1    20      180     Sample1 1&#xA;chr1    50      120     .       -1&#xA;chr1    70      100     Sample3 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We have one name scheme for reference element IDs (&lt;code&gt;.&lt;/code&gt;) and a second name scheme for sample IDs (&lt;code&gt;Sample*&lt;/code&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We will exploit this difference in name schemes when we apply operations in a certain order. For this example, we will subtract a reference element score from a sample element score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can partition the union, mapping the disjoint elements back to the unioned set:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops --partition /tmp/union.bed | bedmap --echo --echo-map-id --echo-map-score --delim '\t' - /tmp/union.bed&#xA;chr1    1   5   .;Sample2;Sample1   0.000000;1.000000;1.000000&#xA;chr1    5   10  Sample1;.   1.000000;1.000000&#xA;chr1    10  20  .   1.000000&#xA;chr1    20  50  Sample1 1.000000&#xA;chr1    50  70  Sample1;.   1.000000;-1.000000&#xA;chr1    70  100 Sample1;.;Sample3   1.000000;-1.000000;1.000000&#xA;chr1    100 120 Sample1;.   1.000000;-1.000000&#xA;chr1    120 180 Sample1 1.000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Each of the fourth and fifth columns are semi-colon-delimited strings. Each element is ordered by the order in which it is discovered in a &lt;code&gt;bedmap&lt;/code&gt; operation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both columns have the same ordering or same one-to-one correspondence of ID to signal score. This is very useful, because we can turn these into a pair of ordered arrays and apply some logic, depending on the answer we want back.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, here is some &lt;code&gt;awk&lt;/code&gt; logic to parse those two strings and subtract a reference element score from a sample element score:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops --partition /tmp/union.bed \&#xA;    | bedmap --echo --echo-map-id --echo-map-score --delim '\t' - /tmp/union.bed \&#xA;    | awk -vFS=&quot;\t&quot; -vOFS=&quot;\t&quot; \&#xA;        '{ \&#xA;             i=split($4,a,&quot;;&quot;); \&#xA;             j=split($5,b,&quot;;&quot;); \&#xA;             if ((i==1) &amp;amp;&amp;amp; (a[i]==&quot;.&quot;)) { pass; } \&#xA;             else { \&#xA;                 for (k=1; k&amp;lt;=i; k++) { \&#xA;                     if (a[k]==&quot;.&quot;) { refIdx=k; } \&#xA;                 } \&#xA;                 for (k=1; k&amp;lt;=i; k++) { \&#xA;                     if (k!=refIdx) { print $0, &quot;.&quot;, b[k]-b[refIdx], a[k]; } \&#xA;                 } \&#xA;             } \&#xA;         }' \&#xA;    | cut -f1-3,6-8 \&#xA;    | sort-bed - \&#xA;    &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The file &lt;code&gt;answer.bed&lt;/code&gt; will look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    1   5   .   1   Sample1&#xA;chr1    1   5   .   1   Sample2&#xA;chr1    5   10  .   0   Sample1&#xA;chr1    20  50  .   1   Sample1&#xA;chr1    50  70  .   2   Sample1&#xA;chr1    70  100 .   2   Sample1&#xA;chr1    70  100 .   2   Sample3&#xA;chr1    100 120 .   2   Sample1&#xA;chr1    120 180 .   1   Sample1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This partitioning of intervals is different from the one in your question. Perhaps this is due to how you partitioned your set. In any case, perhaps this would give you an idea of how to apply custom operations over a disjoint partitioning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main idea is that this part of the &lt;code&gt;awk&lt;/code&gt; script finds the score associated with the reference file (&quot;File2&quot;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (k=1; k&amp;lt;=i; k++) {&#xA;    if (a[k]==&quot;.&quot;) { refIdx=k; }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any index other than &lt;code&gt;refIdx&lt;/code&gt; corresponds to a &lt;code&gt;Sample*&lt;/code&gt; element. That means we can do operations between &lt;code&gt;Sample*&lt;/code&gt; scores and the specific reference score.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also leave out mappings where a reference score does not associate with any &lt;code&gt;Sample*&lt;/code&gt; element:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if ((i==1) &amp;amp;&amp;amp; (a[i]==&quot;.&quot;)) { pass; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thus, all the operations are always between a pairing of a sample and the mapped reference element.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;cut&lt;/code&gt; and &lt;code&gt;sort-bed&lt;/code&gt; statements at the end just clean up the result so that you get something that looks like what you are expecting at the end, which is also sorted correctly and ready to use for downstream set operations.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-05-24T22:10:32.117" LastActivityDate="2018-05-24T22:10:32.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4357" PostTypeId="1" AcceptedAnswerId="4402" CreationDate="2018-05-25T00:38:49.370" Score="3" ViewCount="4000" Body="&lt;p&gt;I am trying to run &lt;code&gt;UMAP&lt;/code&gt; in the following way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;RunUMAP(seurat_object, dims.use = 1:15, reduction.use = &quot;pca&quot;, reduction.name = &quot;umap&quot;, &#xA;    reduction.key = &quot;UMAP&quot;, n_neighbors = 30L, min_dist = 0.3, metric = &quot;correlation&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or just this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;RunUMAP(seurat_object, dims.use = 1:15, reduction.use = &quot;pca&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is giving me the error: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error in py_get_attr_impl(x, name, silent) : &#xA;    AttributeError: module 'umap' has no attribute 'UMAP'&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I already installed &lt;code&gt;umap-learn&lt;/code&gt; successfully in this way: &lt;code&gt;conda install -c conda-forge umap-learn&lt;/code&gt; and checked that &lt;code&gt;Rstudio&lt;/code&gt; is using the same &lt;code&gt;python&lt;/code&gt; version as is provided by default in the terminal: &lt;code&gt;3.6.5&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-05-31T00:52:35.393" Title="RunUMAP in Seurat not working: module 'umap' has no attribute 'UMAP'" Tags="|scrnaseq|seurat|umap|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4358" PostTypeId="5" CreationDate="2018-05-25T05:49:51.103" Score="0" Body="&lt;p&gt;For questions regarding use of the 10X Genomics' &quot;Cell Ranger&quot; analysis tool for the Chromium single-cell sequencing platform.&lt;/p&gt;&#xA;" OwnerUserId="1855" LastEditorUserId="1855" LastEditDate="2018-05-25T12:39:05.140" LastActivityDate="2018-05-25T12:39:05.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4359" PostTypeId="4" CreationDate="2018-05-25T05:49:51.103" Score="0" Body="For questions regarding use of the 10X Genomics' &quot;Cell Ranger&quot; analysis tool for the Chromium single-cell sequencing platform." OwnerUserId="1855" LastEditorUserId="1855" LastEditDate="2018-05-25T12:39:19.490" LastActivityDate="2018-05-25T12:39:19.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4360" PostTypeId="2" ParentId="4350" CreationDate="2018-05-25T06:08:06.667" Score="2" Body="&lt;p&gt;Have you checked firewall settings on your laptop?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;strong&gt;Firewall / FTP connection issues&lt;/strong&gt; at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://software.broadinstitute.org/cancer/software/gsea/wiki/index.php/Known_Issues&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSEA wiki - known issues&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2153" LastActivityDate="2018-05-25T06:08:06.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4361" PostTypeId="2" ParentId="4353" CreationDate="2018-05-25T12:16:44.313" Score="5" Body="&lt;p&gt;I'm unsure whether this is the answer you are looking for, but when looking into 10X cellranger documentation for the Matrices Output:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Unfiltered gene-barcode matrices&lt;/strong&gt;:&#xA;  &lt;em&gt;Contains every barcode from fixed list of known-good barcode sequences. This includes background and non-cellular barcodes.&lt;/em&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Filtered gene-barcode matrices&lt;/strong&gt;:&#xA;  &lt;em&gt;Contains only detected cellular barcodes.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/output/matrices&quot; rel=&quot;noreferrer&quot;&gt;Source&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2153" LastActivityDate="2018-05-25T12:16:44.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4362" PostTypeId="1" AcceptedAnswerId="4370" CreationDate="2018-05-25T12:55:03.983" Score="9" ViewCount="9752" Body="&lt;p&gt;I'm interested in the transition/transversion (Ti/Tv) ratio:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In substitution mutations, transitions are defined as the interchange of the purine-based A↔G or pryimidine-based C↔T. Transversions are defined as the interchange between two-ring purine nucleobases and one-ring pyrimidine bases.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;(&lt;a href=&quot;https://doi.org/10.1093/bioinformatics/btu668&quot; rel=&quot;noreferrer&quot;&gt;Wang &lt;em&gt;et al&lt;/em&gt;, &lt;em&gt;Bioinformatics&lt;/em&gt;, &lt;strong&gt;31&lt;/strong&gt; (3), 2015&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How exactly does this ratio imply false positives? Too high ==&gt; high false positive rates? Or too low?&lt;/li&gt;&#xA;&lt;li&gt;Why is the expected value for random substitutions for the Ti/Tiv ratio 0.5?&lt;/li&gt;&#xA;&lt;li&gt;If the ratio is expected to be 2.10 for WGS, but I get 3.00 what does that mean? What if I get 1.00?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="174" LastEditorUserId="298" LastEditDate="2018-05-25T13:31:36.637" LastActivityDate="2018-05-26T15:19:43.720" Title="Why Ti/Tv ratio?" Tags="|variant-calling|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4363" PostTypeId="2" ParentId="4362" CreationDate="2018-05-25T15:01:02.287" Score="3" Body="&lt;p&gt;I haven't done variant calling myself, but 5 seconds of googling lead me to &lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/6308/evaluating-the-quality-of-a-variant-callset&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This metric is the ratio of transition (Ti) to transversion (Tv) SNPs. If the distribution of transition and transversion mutations were random (i.e. without any biological influence) we would expect a ratio of 0.5. This is simply due to the fact that there are twice as many possible transversion mutations than there are transitions. However, in the biological context, it is very common to see a methylated cytosine undergo deamination to become thymine. As this is a transition mutation, it has been shown to increase the expected random ratio from 0.5 to ~2.01. Furthermore, CpG islands, usually found in primer regions, have higher concentrations of methylcytosines. By including these regions, whole exome sequencing shows an even stronger lean towards transition mutations, with an expected ratio of 3.0-3.3. A significant deviation from the expected values could indicate artifactual variants causing bias. If your TiTv Ratio is too low, your callset likely has more false positives.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Since methylated cytosine deamination is a fairly universal phenomenon, if you're seeing a ratio that deviates substantially from this expectation, you've either discovered some exciting new biology, or there are issues with your variant calling pipeline.&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-05-25T15:01:02.287" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4364" PostTypeId="1" AcceptedAnswerId="4368" CreationDate="2018-05-25T20:46:17.193" Score="4" ViewCount="2563" Body="&lt;p&gt;I wish to use &lt;a href=&quot;https://github.com/mourisl/Rascaf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rascaf&lt;/a&gt; to scaffold a fragmented draft genome.&lt;/p&gt;&#xA;&lt;p&gt;For this, I need to provide a BAM file of aligned RNA-seq reads and the draft genome.&lt;/p&gt;&#xA;&lt;p&gt;So, I indexed the draft genome with STAR like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;STAR --runMode genomeGenerate --genomeDir output/index --genomeFastaFiles draft.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then, I tried to aligned the reads like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;STAR --genomeDir output/index --readFilesIn reads.fastq.gz --readFilesCommand gunzip -c --outFileNamePrefix output/alignment --quantMode GeneCounts --outSAMunmapped None --outSAMtype BAM SortedByCoordinate --outSAMattrRGline ID:RG1 CN:yy \&amp;quot;DS: z z z\&amp;quot; SM:sample&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The latter commands outputs the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Transcriptome.cpp:51:Transcriptome: exiting because of *INPUT FILE* error: could not open input file exonGeTrInfo.tab&#xA;Solution: check that the file exists and you have read permission for this file&#xA;          SOLUTION: utilize --sjdbGTFfile /path/to/annotantions.gtf option at the genome generation step or mapping step&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It suggests I should use a GTF file. But this is a draft genome of an individual for which no GTF is available. I could try to adapt a GTF from a close relative but for the scaffolding, it seems superfluous. Is there a way I can map with STAR without the GTF?&lt;/p&gt;&#xA;&lt;p&gt;These commands were run on a GNU/Linux machine.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-05-26T04:04:00.897" Title="Can I run STAR without an annotation file?" Tags="|read-mapping|rna-alignment|genome-sequencing|scaffold|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4365" PostTypeId="1" AcceptedAnswerId="4366" CreationDate="2018-05-25T21:04:46.707" Score="3" ViewCount="2700" Body="&lt;p&gt;Let's say I have a genbank file, e.g. the one downloaded from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NC_000913.3&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, that contains entries as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; gene            190..255&#xA;                 /gene=&quot;thrL&quot;&#xA;                 /locus_tag=&quot;b0001&quot;&#xA;                 /gene_synonym=&quot;ECK0001; JW4367&quot;&#xA;                 /db_xref=&quot;EcoGene:EG11277&quot;&#xA;                 /db_xref=&quot;GeneID:944742&quot;&#xA; CDS             190..255&#xA;                 /gene=&quot;thrL&quot;&#xA;                 /locus_tag=&quot;b0001&quot;&#xA;                 /gene_synonym=&quot;ECK0001; JW4367&quot;&#xA;                 /function=&quot;leader; Amino acid biosynthesis: Threonine&quot;&#xA;                 /GO_process=&quot;GO:0009088 - threonine biosynthetic process&quot;&#xA;                 /codon_start=1&#xA;                 /transl_table=11&#xA;                 /product=&quot;thr operon leader peptide&quot;&#xA;                 /protein_id=&quot;NP_414542.1&quot;&#xA;                 /db_xref=&quot;ASAP:ABE-0000006&quot;&#xA;                 /db_xref=&quot;UniProtKB/Swiss-Prot:P0AD86&quot;&#xA;                 /db_xref=&quot;EcoGene:EG11277&quot;&#xA;                 /db_xref=&quot;GeneID:944742&quot;&#xA;                 /translation=&quot;MKRISTTITTTITITTGNGAG&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I want to extract the potein fasta from it, how would I do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found a code sample &lt;a href=&quot;https://warwick.ac.uk/fac/sci/moac/people/students/peter_cock/python/genbank2fasta/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. When I then run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;gbk_filename = 'NC_000913.3.gb'&#xA;faa_filename = 'NC_000913.3_warwick.fasta'&#xA;input_handle  = open(gbk_filename, &quot;r&quot;)&#xA;output_handle = open(faa_filename, &quot;w&quot;)&#xA;&#xA;for seq_record in SeqIO.parse(input_handle, &quot;genbank&quot;) :&#xA;    print &quot;Dealing with GenBank record %s&quot; % seq_record.id&#xA;    for seq_feature in seq_record.features :&#xA;        if seq_feature.type==&quot;CDS&quot; :&#xA;            assert len(seq_feature.qualifiers['translation'])==1&#xA;            output_handle.write(&quot;&amp;gt;%s from %s\n%s\n&quot; % (&#xA;                   seq_feature.qualifiers['locus_tag'][0],&#xA;                   seq_record.name,&#xA;                   seq_feature.qualifiers['translation'][0]))&#xA;&#xA;output_handle.close()&#xA;input_handle.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;it results in &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     31     for seq_feature in seq_record.features :&#xA;     32         if seq_feature.type==&quot;CDS&quot; :&#xA;---&amp;gt; 33             assert len(seq_feature.qualifiers['translation'])==1&#xA;     34             output_handle.write(&quot;&amp;gt;%s from %s\n%s\n&quot; % (&#xA;     35                    seq_feature.qualifiers['locus_tag'][0],&#xA;&#xA;KeyError: 'translation'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How could this be fixed?&lt;/p&gt;&#xA;" OwnerUserId="351" LastEditorUserId="351" LastEditDate="2018-05-25T21:29:42.250" LastActivityDate="2018-05-26T10:16:56.373" Title="How to extract the protein fasta file from a genbank file?" Tags="|fasta|python|genbank|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4366" PostTypeId="2" ParentId="4365" CreationDate="2018-05-25T21:07:06.313" Score="6" Body="&lt;p&gt;One can get it to work by using &lt;a href=&quot;http://biopython.org/DIST/docs/api/Bio.SeqIO.InsdcIO-module.html#GenBankCdsFeatureIterator&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;SeqIO.InsdcIO.GenBankCdsFeatureIterator&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;file_name = 'NC_000913.3.gb'&#xA;&#xA;# stores all the CDS entries&#xA;all_entries = []&#xA;&#xA;with open(file_name, 'r') as GBFile:&#xA;&#xA;    GBcds = SeqIO.InsdcIO.GenBankCdsFeatureIterator(GBFile)&#xA;&#xA;    for cds in GBcds:&#xA;        if cds.seq is not None:&#xA;            cds.id = cds.name&#xA;            cds.description = ''&#xA;            all_entries.append(cds)&#xA;&#xA;&#xA;# write file&#xA;SeqIO.write(all_entries, '{}.fasta'.format(file_name[:-3]), 'fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will then give the desired output (just the first two entries):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;b0001&#xA;MKRISTTITTTITITTGNGAG&#xA;&amp;gt;b0002&#xA;MRVLKFGGTSVANAERFLRVADILESNARQGQVATVLSAPAKITNHLVAMIEKTISGQDA&#xA;LPNISDAERIFAELLTGLAAAQPGFPLAQLKTFVDQEFAQIKHVLHGISLLGQCPDSINA&#xA;ALICRGEKMSIAIMAGVLEARGHNVTVIDPVEKLLAVGHYLESTVDIAESTRRIAASRIP&#xA;ADHMVLMAGFTAGNEKGELVVLGRNGSDYSAAVLAACLRADCCEIWTDVDGVYTCDPRQV&#xA;PDARLLKSMSYQEAMELSYFGAKVLHPRTITPIAQFQIPCLIKNTGNPQAPGTLIGASRD&#xA;EDELPVKGISNLNNMAMFSVSGPGMKGMVGMAARVFAAMSRARISVVLITQSSSEYSISF&#xA;CVPQSDCVRAERAMQEEFYLELKEGLLEPLAVTERLAIISVVGDGMRTLRGISAKFFAAL&#xA;ARANINIVAIAQGSSERSISVVVNNDDATTGVRVTHQMLFNTDQVIEVFVIGVGGVGGAL&#xA;LEQLKRQQSWLKNKHIDLRVCGVANSKALLTNVHGLNLENWQEELAQAKEPFNLGRLIRL&#xA;VKEYHLLNPVIVDCTSSQAVADQYADFLREGFHVVTPNKKANTSSMDYYHQLRYAAEKSR&#xA;RKFLYDTNVGAGLPVIENLQNLLNAGDELMKFSGILSGSLSYIFGKLDEGMSFSEATTLA&#xA;REMGYTEPDPRDDLSGMDVARKLLILARETGRELELADIEIEPVLPAEFNAEGDVAAFMA&#xA;NLSQLDDLFAARVAKARDEGKVLRYVGNIDEDGVCRVKIAEVDGNDPLFKVKNGENALAF&#xA;YSHYYQPLPLVLRGYGAGNDVTAAGVFADLLRTLSWKLGV&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="351" LastEditorUserId="351" LastEditDate="2018-05-26T10:16:56.373" LastActivityDate="2018-05-26T10:16:56.373" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4367" PostTypeId="1" AcceptedAnswerId="4387" CreationDate="2018-05-26T00:07:46.367" Score="5" ViewCount="653" Body="&lt;p&gt;In order to define which cell is of which type we need to identify a set of rules, for instance neurons should express one of the following: &lt;code&gt;Thy1&lt;/code&gt;, &lt;code&gt;Rbfox3&lt;/code&gt;, &lt;code&gt;MAP2&lt;/code&gt;, &lt;code&gt;Camk2b&lt;/code&gt;, &lt;code&gt;Gad1&lt;/code&gt;,&lt;code&gt;Cck&lt;/code&gt;, &lt;code&gt;Reln&lt;/code&gt;, and should not express any of the following: &lt;code&gt;cd45&lt;/code&gt;, &lt;code&gt;Tmem119&lt;/code&gt;, &lt;code&gt;CD11b&lt;/code&gt;, ... and others. Maybe &lt;code&gt;Rbfox3&lt;/code&gt; should always be expressed in any neurons. So, these rules need to be compiled manually as I understand to identify each cell type, from the literature. I would assume that many people already faced this issue, and maybe came up with software to do that? Are there any software where we would be able to supply a matrix of single cell expression data and have the types assigned with certain degree of certainty?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that maybe we can also use heatmaps for that, like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2547/cell-type-specific-genes-heatmap-using-rank-based-approach&quot;&gt;cell type specific genes - heatmap using rank based approach&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But again, this seems too manual work. Besides, I do not think that we can assign different neuronal types based on just a heatmap.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2020-07-27T20:18:54.310" Title="Specific cell type identification in Single Cell Sequencing" Tags="|scrnaseq|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4368" PostTypeId="2" ParentId="4364" CreationDate="2018-05-26T04:04:00.897" Score="3" Body="&lt;p&gt;I don't think you can use the &lt;code&gt;--quantMode GeneCounts&lt;/code&gt; option with no annotations. I think the error is trying to look for an exon file generated from the annotations to do the quantitation on. Remove that and I think it should work, as the manual specifically states that annotations are optional but highly recommended.&lt;/p&gt;&#xA;" OwnerUserId="1269" LastActivityDate="2018-05-26T04:04:00.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4370" PostTypeId="2" ParentId="4362" CreationDate="2018-05-26T15:19:43.720" Score="8" Body="&lt;p&gt;I more like to use &quot;ts/tv&quot; for transition-to-transversion ratio. This abbreviation had been used in phylogenetics. When NGS came along, some important developers started to use &quot;ti/tv&quot;, but I am still used to the old convention.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why is the expected value for random substitutions for the Ti/Tv ratio 0.5?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are six types of base changes. Two of them are transitions: &lt;code&gt;A&amp;lt;-&amp;gt;G&lt;/code&gt; and &lt;code&gt;C&amp;lt;-&amp;gt;T&lt;/code&gt; and the other four types are transversions. If everything were random, you would expect to see twice as many transversions – ts:tv=2:4=0.5.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If the ratio is expected to be 2.10 for WGS&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The expected ratio is not &quot;2.10 for WGS&quot;. It is 2–2.10 for human across the whole genome. You see this number when you align the human genome to chimpanzee or when you focus on an accurate subset of human variant calls. However, in other species, the expected ts/tv may be very different. Also, this number is correlated with GC content. You get a higher ts/tv in high-GC regions, or in coding regions which tend to have higher GC, too. Partly as a result, it is hard to say what is expected ts/tv in exact.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;but I get 3.00 what does that mean? What if I get 1.00?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you get 3.00, your callset is highly biased. If you get 1.00, your callset has a high error rate. Suppose the exact ts/tv is $\beta$ and you observe ts/tv $\beta'\le\beta$, you can work out the fraction of wrong calls to be (assuming random errors have ts/tv=0.5)&#xA;$$&#xA;\frac{3(\beta-\beta')}{(1+\beta')(2\beta-1)}&#xA;$$&#xA;This is of course an approximate because $\beta$ is not accurate in the first place and because errors are often not random, so their ts/tv is not really 0.5.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How exactly does this ratio imply false positives? Too high ==&gt; high false positive rates? Or too low?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Too low ==&gt; high false positive rate; too high ==&gt; bias. In practice, you rarely see &quot;too high&quot; ts/tv.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-05-26T15:19:43.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4371" PostTypeId="1" AcceptedAnswerId="9008" CreationDate="2018-05-26T21:19:37.693" Score="3" ViewCount="1890" Body="&lt;p&gt;I am learning to use BEAUTI and BEAST software. I followed their tutorial and was able to run example data without any problem. Now, I want to do phylodynamic analysis on my data. I have downloaded 70 picornavirus sequences from the gene bank. My questions are :&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What tool do I have to use to create Nexus file that is used by BEAUTI ?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How do I enter the information about the date (year ) the virus was isolated and location information which is required for rates and dates (evolutionary) analysis in BEAST, spatial analysis by SpreaD3 ?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Notes on &lt;a href=&quot;https://rega.kuleuven.be/cev/ecv/software/SpreaD3_tutorial&quot; rel=&quot;nofollow noreferrer&quot;&gt;SpreaD3&lt;/a&gt; is a update on SPREAD by Bielejec et al. SPREAD: Spatial Phylogenetic Reconstruction of Evolutionary Dynamics. Bioinformatics, 2011, 27:2910-2912. This uses spatial Bayesian reconstruction based on the phylogenetic output of Beast&lt;/p&gt;&#xA;" OwnerUserId="1684" LastEditorUserId="3848" LastEditDate="2022-03-13T16:41:20.537" LastActivityDate="2022-03-13T16:41:20.537" Title="How to create a nexus file for BEAST?" Tags="|sequence-alignment|ngs|covid-19|virus|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4372" PostTypeId="1" AcceptedAnswerId="4444" CreationDate="2018-05-27T02:19:22.767" Score="4" ViewCount="216" Body="&lt;p&gt;I am working on haplotype data and want to make a tree out of haplogroups using &lt;strong&gt;ggtree&lt;/strong&gt;. I have following data in &lt;strong&gt;newick format&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(B(B2b_M112),C(C_RPS4Y711,C1a_V20,C2_M217,C2b1a_F3985,C2C1_Z1338),E(E1b1a_V38,E1b1b_M215,E2_M75),G(G_M201,G1_M342,G2a1_Z6552,G2a2a_PF3147,G2a2b2b_PF3359,G2a2b1_M406,G2a2b2a1a_U1,G2a2b2a1b_L497,G2a2b2a1c_CTS342,G2a_L30,G2b_M3155,G2b_M283),H(H1_M69,H1a_M82),I(I_M170,I1_M253,I2a1_Isles,I2a1_S21825,I2a1a_M26,I2a1b3_L621,I2a2a_M223,I2a2b_L38,I2c2_Y16419),J(J_M304,J1a_PF7257,J1a2a1a2_P58,J1a2a2_PF7264,J1a3_Z1828,J2_M172,J2a_L25,J2a1_L26,J2a1_M319,J2a1_Z1846,J2a1_Z387,J2a1_Z467,J2a1_Z6046,J2a1_Z6063,J2a1_Z6065,J2a1_Z7671,J2a1_Z7700,J2a2_PF5008, J2b_M241,J2b1_M205,J2b2_M241,J2b2_Z2456),K(K_M9),L(L_M20,L1_M22,L1a_M27,L1b_M317,L1c_M357),N(N3,N1a1_M46,N1a2_L666),O(O1a1_B384,O1b1_F2320,O1b2_P49,O2 IMS_JST0213554,O2a1_F51,O_O2a2_F525,O3_M122),Q(Q F1096,Q_L27,Q_L56,Q_M24,Q_M346),R(R_M207,M734,P224,P280xM173,R1a_M417,R1a1_M459,R1a1a_M17,R1b_M343,R1b_M73,R1b1a1b_V1636,R1b1a2_V88,R1b1b_PH155,R2_M479,R2a_M124),T(T_M184,T_L131,T_PF5633));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am using following R code to plot a tree from this data,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; library(&quot;ggtree&quot;)&#xA;&amp;gt; tree &amp;lt;- read.tree(text = &quot;(B(B2b_M112),C(C_RPS4Y711,C1a_V20,C2_M217,C2b1a_F3985,C2C1_Z1338),E(E1b1a_V38,E1b1b_M215,E2_M75),G(G_M201,G1_M342,G2a1_Z6552,G2a2a_PF3147,G2a2b2b_PF3359,G2a2b1_M406,G2a2b2a1a_U1,G2a2b2a1b_L497,G2a2b2a1c_CTS342,G2a_L30,G2b_M3155,G2b_M283),H(H1_M69,H1a_M82),I(I_M170,I1_M253,I2a1_Isles,I2a1_S21825,I2a1a_M26,I2a1b3_L621,I2a2a_M223,I2a2b_L38,I2c2_Y16419),J(J_M304,J1a_PF7257,J1a2a1a2_P58,J1a2a2_PF7264,J1a3_Z1828,J2_M172,J2a_L25,J2a1_L26,J2a1_M319,J2a1_Z1846,J2a1_Z387,J2a1_Z467,J2a1_Z6046,J2a1_Z6063,J2a1_Z6065,J2a1_Z7671,J2a1_Z7700,J2a2_PF5008, J2b_M241,J2b1_M205,J2b2_M241,J2b2_Z2456),K(K_M9),L(L_M20,L1_M22,L1a_M27,L1b_M317,L1c_M357),N(N3,N1a1_M46,N1a2_L666),O(O1a1_B384,O1b1_F2320,O1b2_P49,O2 IMS_JST0213554,O2a1_F51,O_O2a2_F525,O3_M122),Q(Q F1096,Q_L27,Q_L56,Q_M24,Q_M346),R(R_M207,M734,P224,P280xM173,R1a_M417,R1a1_M459,R1a1a_M17,R1b_M343,R1b_M73,R1b1a1b_V1636,R1b1a2_V88,R1b1b_PH155,R2_M479,R2a_M124),T(T_M184,T_L131,T_PF5633));&quot;)&#xA;&amp;gt; p&amp;lt;-ggtree(tree)&#xA;&amp;gt; p+geom_tiplab()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the displayed tree looks like this &#xA;&lt;a href=&quot;https://i.stack.imgur.com/Bwk48.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Bwk48.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have following problems in this tree,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;It gives a low quality image&lt;/li&gt;&#xA;&lt;li&gt;It does not show the haplotype taxa names clearly&lt;/li&gt;&#xA;&lt;li&gt;I want to arrange haplogroups i.e nodes in the desired according to the tree given below,&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7dTqJ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7dTqJ.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e B haplogroups with C haplotypes and with C, E named haplotypes etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to solve these problems but failed to find any solution. Is there a solution?  &lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="3848" LastEditDate="2020-05-07T18:09:40.253" LastActivityDate="2020-05-07T18:09:40.253" Title="Problems in creating desired phylogenetic tree with ggtree" Tags="|r|bioconductor|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4373" PostTypeId="1" AcceptedAnswerId="4764" CreationDate="2018-05-27T14:40:56.613" Score="6" ViewCount="79" Body="&lt;p&gt;I'm new to pairwise and multiple sequence alignment in general, but I thought I understood how clustal works -- k-tuple distance is a cheap metric that's 'probably approximately' monotonic in the real global pairwise-alignment score, so we can just calculate it between all C(n, 2) sequences and do a simple UPGMA. &lt;a href=&quot;https://www-bimas.cit.nih.gov/clustalw/clustalw.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clustalw algorithm description here&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I'm observing odd results. Suppose I have a file with 60 sequences and run clustal on it, and observe that seqA and seqB are paired terminal child nodes. If I then delete a handful (say 10) of the sequences, but keep seqA and seqB, then in the &lt;em&gt;new&lt;/em&gt; clustal alignment, seqA and seqB may not remain paired.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is strange to me, because if Clustal was actually doing the k-tuple distance + UPGMA thing I thought it was doing, then all terminal pairings should always be stable under deletions of other sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So... what am I missing?&lt;/p&gt;&#xA;" OwnerUserId="2860" LastEditorUserId="48" LastEditDate="2018-05-28T06:45:52.740" LastActivityDate="2018-11-21T16:02:55.610" Title="Why aren't clustal alignments stable under deletion of some of the sequences?" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4375" PostTypeId="1" AcceptedAnswerId="4381" CreationDate="2018-05-27T20:19:36.323" Score="1" ViewCount="159" Body="&lt;p&gt;I am trying to find over-expressed motifs in a bipartite transcription factor gene network.My genes are not very well characterized, so finding functional motifs has been a challenge. I used the motif-discovery app on Cytoscape and calculated the total number of 3 node motifs on the network. In my network, where transcription factors regulate genes, the only kind of three node motif that is possible is a simple input module.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/A1jiI.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/A1jiI.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is an example of a simple input module where a transcription factor regulates multiple genes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I further created random networks to my real world network and statistically (Student's t-test) proved that there are more number of single input modules in the real rather than the random network. However, I am just not feeling satisfied with just this one explanation because I would have like to draw some biological inference from this. I was thinking of also analysis the 4 node and 5 node motifs in the network, find the number of times any of my hub nodes occupy a single node of these motifs and maybe use that and prove the importance of my hub nodes (though hub nodes are worthy of more exploration just by virtue of their high degree). Has somebody done something like this previously. It is a given that hub nodes would be part of the maximum number of motifs just by virtue of their high degree and connection to multiple genes. So would this prove to be redundant?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for any help that could be provided. &lt;/p&gt;&#xA;" OwnerUserId="976" LastEditorUserId="976" LastEditDate="2018-05-28T13:50:26.977" LastActivityDate="2018-05-29T07:10:45.227" Title="Motif Discovery in Bipartite networks using Cytoscape" Tags="|networks|motifs|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4376" PostTypeId="1" CreationDate="2018-05-27T20:52:07.567" Score="2" ViewCount="277" Body="&lt;p&gt;I am doing phylodynamic analysis of virus sequences using bayesian method using BEAUti and BEAST. As a part of my analysis I am using Tracer to check how my model is working. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can interpret some of the results like from the Tracer like :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;clock.rate - rate of nucleotide substitution&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;treemodel.rootheight- time when divergence occured&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;ESS- measures sampling quality (ideally has to be greater that 200)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there are other many terms on this analysis which I have no idea how to explain. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please can anybody help me to find out what other aspects of the evolution can be explained using this table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;posterior   -33531.076&lt;br&gt;&#xA;prior   -481.982&lt;br&gt;&#xA;likelihood  -33049.094&lt;br&gt;&#xA;treeModel.rootHeight    32.271&lt;br&gt;&#xA;tmrca(Brazil)   3.853&lt;br&gt;&#xA;tmrca(Canada)   16.644&lt;br&gt;&#xA;tmrca(China )   11.752&lt;br&gt;&#xA;tmrca(Columbia) 1&lt;br&gt;&#xA;tmrca(Thailand) 1.093&lt;br&gt;&#xA;tmrca(USA)  32.271&lt;br&gt;&#xA;tmrca(allseq)   32.271&lt;br&gt;&#xA;constant.popSize    43.595&lt;br&gt;&#xA;CP1.kappa   18.716&lt;br&gt;&#xA;CP2.kappa   17.16&lt;br&gt;&#xA;CP3.kappa   22.796&lt;br&gt;&#xA;frequencies1    0.224&lt;br&gt;&#xA;frequencies3    0.221&lt;br&gt;&#xA;frequencies4    0.26    5001&lt;br&gt;&#xA;CP1.mu  0.76&#xA;CP2.mu  1.221&lt;br&gt;&#xA;CP3.mu  1.019&lt;br&gt;&#xA;clock.rate  1.354E-3&lt;br&gt;&#xA;meanRate    1.354E-3&lt;br&gt;&#xA;CP1.treeLikelihood  -9462.258&lt;br&gt;&#xA;CP2.treeLikelihood  -12520.305&lt;br&gt;&#xA;branchRates 0E0&lt;br&gt;&#xA;coalescent  -467.376    &lt;/p&gt;&#xA;" OwnerUserId="1684" LastActivityDate="2019-04-03T13:01:50.310" Title="Interpreting bayesian phylogenetic model using Tracer?" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4378" PostTypeId="1" AcceptedAnswerId="4380" CreationDate="2018-05-28T05:11:43.127" Score="3" ViewCount="105" Body="&lt;p&gt;&lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-014-0439-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;In this paper&lt;/a&gt;, the authors talk about &lt;em&gt;copy number neutral reads&lt;/em&gt; (as reads that equally represent parental chromosomes) and &lt;em&gt;active reads&lt;/em&gt; (as reads from only one parent chromosome):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We reasoned that the reads mapped into a genomic window can be partitioned in two sets: one set includes reads that equally represent parental chromosomes (copy number neutral reads); and the other set contains reads from only one parent chromosome (active reads).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Nonetheless, I have some difficulties understanding these two concepts. Can somebody try to expand on what they wrote?&lt;/p&gt;&#xA;" OwnerUserId="2862" LastEditorUserId="298" LastEditDate="2018-05-28T09:59:25.430" LastActivityDate="2018-05-28T23:00:05.863" Title="Difference between copy number neutral reads and active reads" Tags="|ngs|genomics|cancer|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4379" PostTypeId="1" CreationDate="2018-05-28T06:23:14.603" Score="3" ViewCount="62" Body="&lt;p&gt;I looked up existing databases but ones like &lt;a href=&quot;http://www.ippidb.cdithem.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;iPPI-DB&lt;/a&gt; or &lt;a href=&quot;https://www.ebi.ac.uk/merops/inhibitors/&quot; rel=&quot;nofollow noreferrer&quot;&gt;enter link description here&lt;/a&gt; either provide non-peptide inhibitors or only sequences. I need structures of proteins bound to other proteins, the latter of which is an inhibitor of the former, and the complex has a structure deposited in the PDB database. Here's an example: &lt;a href=&quot;https://www.rcsb.org/structure/3U1J&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB:3U1J&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case there is no existing database in literature, it'd be great even if I can screen for such PDB entries using some code, but I can't come up with an algorithm for it. Thank you!&lt;/p&gt;&#xA;" OwnerUserId="960" OwnerDisplayName="simple_dozer" LastActivityDate="2018-10-25T19:00:04.210" Title="How do I find a list of PDB structures bound to a peptide inhibitor or peptide-like inhibitor?" Tags="|pdb|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4380" PostTypeId="2" ParentId="4378" CreationDate="2018-05-28T13:17:08.207" Score="1" Body="&lt;p&gt;In genetics normally one gets a copy (allele) of a gene (or actually a chromosome) from each parent. If the alleles differ between parents, for example by a SNP, one would have one copy of each allele. With (NGS) sequencing you would get an even amount of reads for both copies. However with somatic mutations, hence mutations which occur after meiosis and gamete forming, one can lose or gain parts of the chromosome (copy number variation). In this situation, some genomic regions might be deleted or amplified. So the allele of one parent can be deleted, leaving you with only the allele of the other parent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As requested, I have made a simple drawing of a situation where a region differs between parent alleles with SNPs. After a deletion of one of the allele only reads of the other allele are left.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/AMmw3.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AMmw3.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;   &lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="298" LastEditDate="2018-05-28T23:00:05.863" LastActivityDate="2018-05-28T23:00:05.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4381" PostTypeId="2" ParentId="4375" CreationDate="2018-05-28T14:48:13.703" Score="2" Body="&lt;p&gt;I have recently develop an R &lt;a href=&quot;https://github.com/llrs/GSEAdv&quot; rel=&quot;nofollow noreferrer&quot;&gt;package&lt;/a&gt; (sorry not a Cytoscape solution) to deal with this kind of situations of something grouping other elements, like pathways and genes.&lt;br&gt;&#xA;The package is based on GSEABase of Bioconductor to store this relationships. Basically you should store a gene set for each TF and genes it is linked. I would recommend to use the name of the TF as the name of the gene set and then create a GeneSetCollection to be able to use the package.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The package provides with functionalities that allow you to calculate the distribution of genes per gene set (using &lt;code&gt;genesPerPathway&lt;/code&gt;), as well as, &lt;code&gt;pathwaysPerGene&lt;/code&gt; to know in how many other TF regulate each gene. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure of the biological inference you can take from this (I designed the package to test this) but you can also see if there is some specific preference for the TF to link to genes which are also regulated by other TF. This could be done by looking at the conditional probability (&lt;code&gt;condPerPathways&lt;/code&gt; or &lt;code&gt;sizesPerPathway&lt;/code&gt;).&lt;br&gt;&#xA;Sorry the manual pages don't have examples and the vignettes are incomplete yet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also you could use the package &lt;a href=&quot;https://cran.r-project.org/package=tidygraph&quot; rel=&quot;nofollow noreferrer&quot;&gt;tidygraph&lt;/a&gt; to calculate the centrality of the TF or of the genes you want. &lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2018-05-29T07:10:45.227" LastActivityDate="2018-05-29T07:10:45.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4382" PostTypeId="2" ParentId="4379" CreationDate="2018-05-28T16:26:08.403" Score="1" Body="&lt;p&gt;The quickest way to screen the PDB database is &lt;em&gt;Advanced Search&lt;/em&gt; from either &lt;a href=&quot;http://www.rcsb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RCSB&lt;/a&gt; or &lt;a href=&quot;https://www.ebi.ac.uk/pdbe/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDBe&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you need entries with 2+ proteins you can set the number of protein chains to at least 2 in the RCSB search.  Or you can select &quot;protein/protein complex&quot; in &lt;em&gt;Assembly composition&lt;/em&gt; in the PDBe search.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's less clear how to search for inhibitors. If you'd look for the &quot;inhibitor&quot; word in the molecule names you'd get (together with the previous criterium) 1200+ entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/O8mu5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/O8mu5.png&quot; alt=&quot;RCSB advanced search&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2018-05-28T16:26:08.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4384" PostTypeId="1" CreationDate="2018-05-28T20:11:26.690" Score="1" ViewCount="457" Body="&lt;p&gt;I am totally new in Bioinformatics and I would like to apply my knowledge in feature selection on the tag SNP problems.&lt;/p&gt;&#xA;&lt;p&gt;To do that, I've read a lot of papers and books in order to understand the main concepts and how the algorithms to this task work.&lt;/p&gt;&#xA;&lt;p&gt;However, is been very hard for me to understand the organization of the huge amount of genomic data available over the internet.. HapMap, 1000 Genomes, dbSNP and so on... (!) I downloaded GB of data from different sources, but I really do not know what to do with then.&lt;/p&gt;&#xA;&lt;p&gt;--&lt;/p&gt;&#xA;&lt;p&gt;To be more specific in my question:&lt;/p&gt;&#xA;&lt;p&gt;First I would like to run a &lt;a href=&quot;https://link.springer.com/content/pdf/10.1007%2Fs12539-012-0060-x.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genetic Algorithm&lt;/a&gt; to find the tag SNP based on a simple linkage disequilibrium coverage measure. The authors of this GA used the following data&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Three published SNP data sets were downloaded from HapMap (&lt;a href=&quot;http://www.hapmap.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.hapmap.org/&lt;/a&gt;) &lt;em&gt;&lt;strong&gt;(NOT AVAILABLE ANYMORE)&lt;/strong&gt;&lt;/em&gt; for evaluating the prediction accuracies achievable by our method:&lt;/p&gt;&#xA;&lt;p&gt;• ENr112. 343 SNPs genotyped in population ASW (African ancestry in Southwest USA) on chromosome 2 from position 51512208 to 52012208. The genotypes of 83 individuals were used.&lt;/p&gt;&#xA;&lt;p&gt;• ENr113. 380 SNPs genotyped in population ASW on chromosome 4 from position 118466103 to 118966103. The genotypes of 83 individuals were used.&lt;/p&gt;&#xA;&lt;p&gt;• ENm013. 683 SNPs genotyped in population ASW on chromosome 7 from position 89621624 to 90736048. The genotypes of 83 individuals were used.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Besides that, the authors stated that&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We represent a genotype g_i of length L by a sequence over {0, 1, 2}^L (since we assume only bi-allelic SNPs). 0 and 1 stand for the minor and major homozygous types respectively, meaning that the two alleles at that SNP locus are either 0 or 1; the major and minor alleles are determined by the frequency at which each allele appears in the population. A value of 2 denotes the heterozygous type for an SNP having two different alleles.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And also&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;To obtain phasing of genotypes, we used the GEVALT algorithm.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I believe they obtain the aforementioned data in genotype format (something with 0, 1 and 2 values only) and then phasing it to apply in the problem. The image below is a example of this process.&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://image.ibb.co/dptixo/imagem.png&quot; alt=&quot;from paper&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;So&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How and where can I download these data?&lt;/li&gt;&#xA;&lt;li&gt;I know HapMap is not available due to a security issue but there are the FTP, but the FTP folders are really complex to understand. Can someone guide me to download data from there?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I hope I made myself clear.&lt;/p&gt;&#xA;" OwnerUserId="2867" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-06-05T13:34:48.230" Title="How to download SNP data from specific regions, population and positions?" Tags="|snp|public-databases|genomics|data-download|haplotypes|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="4385" PostTypeId="2" ParentId="4384" CreationDate="2018-05-28T23:28:52.343" Score="2" Body="&lt;p&gt;The HapMap website was suspended due to &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/variation/news/NCBI_retiring_HapMap/&quot; rel=&quot;nofollow noreferrer&quot;&gt;a security issue&lt;/a&gt;. Archived HapMap data is available &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/hapmap/&quot; rel=&quot;nofollow noreferrer&quot;&gt;via FTP&lt;/a&gt;, but it's a better idea to download up-to-date International Genome data from the &lt;a href=&quot;http://www.internationalgenome.org/data&quot; rel=&quot;nofollow noreferrer&quot;&gt;IGSR website&lt;/a&gt; (which includes the HapMap individuals as a subset).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The format you use will be dependent on the data analysis program you're using. In most cases involving SNP analysis, the VCF format is probably what would be used.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-05-28T23:28:52.343" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4386" PostTypeId="1" CreationDate="2018-05-29T13:50:53.147" Score="7" ViewCount="169" Body="&lt;p&gt;What methods can we use to compare networks (PPI, gene regulatory etc) created from single-cell gene expression (or proteomics) data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are methods that construct networks by comparing two conditions (e.g. from (gene) co-expression, or differential gene correlation (DGCA, Differential Gene Correlation Analysis)), and infer properties based on these networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm interested here is the comparison of networks constructed for each single cell. The networks can be constructed solely from the data, or  using known interactions (edges) with the help of a database. These comparisons could be used for understanding or classifying cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will probably require a network construction method (which may be hard because we do not have replicates for single cells), some form of network similarity measure / network alignment method. This is a very hard task in general, but here we have 1-to-1 correspondence for each node between the cells (networks), which should help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similar discussions at &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Biostars: &lt;a href=&quot;https://www.biostars.org/p/3853/&quot; rel=&quot;noreferrer&quot;&gt;Question: How Do You Align Or Compare Two Networks&#xA;Content-Wise?&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Researchgate: &lt;a href=&quot;https://www.researchgate.net/post/How_can_I_measure_similarity_between_two_networks&quot; rel=&quot;noreferrer&quot;&gt;How can I measure similarity between two networks?&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;One related paper for single-cells is &lt;a href=&quot;http://dx.doi.org/10.1016/j.cels.2017.08.014&quot; rel=&quot;noreferrer&quot;&gt;Chan et al., 2017&lt;/a&gt;, which uses groups of cells (created by other methods, e.g. clustering) to construct networks, and manually compares them (common edges / Venn diagram).&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2018-05-30T12:33:53.430" Title="Network comparison of single cells (from sequencing data)" Tags="|rna-seq|scrnaseq|networks|single-cell|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4387" PostTypeId="2" ParentId="4367" CreationDate="2018-05-29T15:13:33.497" Score="6" Body="&lt;p&gt;I do not know of such software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I believe this effort is a bit misdirected. The purpose of single-cell sequencing is to get a better understanding of cells; their heterogeneity and functional diversity or developmental / biological processes such as differentiation, using a higher &quot;resolution&quot; method. In other words, if we had the methods you are asking for, there would be no need for the single-cell experiment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Running your data through some pipeline utilizing previous knowledge has the danger of forcing ideas on the data, rather than seeing what the data tells you: it would be better to try and understand and explain the biologically relevant heterogeneity and diversity in your cells, together with a critical comparison of cell type characteristics (expression of genes, pathways) with knowledge described in literature.&#xA;This question is also very similar to the &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/3253/assign-cell-types-to-groups-of-cells-based-on-their-gene-expression-profiles/3296#3296&quot;&gt;one you previously asked&lt;/a&gt; and was answered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also good to recognize that your approach has a lot of implicit assumptions, such as: cells can be clearly categorized as +/- expression for each gene, or it ignores the systems level (networks and pathways), thus again forcing concepts on the data.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This said, one method would be to construct a table listing cell type markers as described in the linked answer above, then write a script that determines a cutoff value for expression of genes in your data (see &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/693/how-can-we-distinguish-between-true-zero-and-dropout-zero-counts-in-single-cell&quot;&gt;this&lt;/a&gt;), then ranks cell types for each cell. For example, you can measure the number or proportion of uniquely genes expressed (although that's very simplistic).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option, if you are familiar with machine learning, is to train a classifier on an annotated dataset, and then use that on new data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also see a convenience function below, which requires a table of marker genes for each cell cluster (i.e. (the &lt;code&gt;Seurat::FindAllMarkers()&lt;/code&gt; output), and a reference df listing genes and corresponding cell types in its HGNC_symbol and Cell_type columns, and returns the table with listing clusters, their marker genes and corresponding cell types.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;getCelltypes &amp;lt;- function(markers, reference) {&#xA;  marker.celltype &amp;lt;- markers&#xA;  marker.celltype$Cell_type &amp;lt;- marker.celltype$gene&#xA;  marker.celltype$Cell_type &amp;lt;- with(reference, celltype[match(marker.celltype$Cell_type, gene)])&#xA;  return(marker.celltype)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Many variants of this reference table and function can be created, feel free to modify it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the future, I think we can expect such classifier software that uses reference data from the Human Protein Atlas, HCA and similar projects.&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2018-05-29T15:13:33.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4388" PostTypeId="1" CreationDate="2018-05-29T16:02:35.670" Score="2" ViewCount="527" Body="&lt;p&gt;I am learning the R package &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/limma.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;limma&lt;/a&gt; using its &lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/limma/inst/doc/usersguide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;user guide&lt;/a&gt;. I need help on understanding the difference between the paired t-test, paired t-test-like (empirical Bayesian) and the mixed effect model in the limma.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paired t test-like example is in the user &lt;em&gt;guide 9.4.1 paired samples&lt;/em&gt;. The mixed effect model is in the &lt;em&gt;9.7 Multi-level Experiments&lt;/em&gt; (I think this is mixed effect model. Correct me if I'm wrong please).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that paired t-test is a special case of linear mixed-effect modeling. So I suppose that the 9.4.1 and 9.7 should return very similar result and the 9.7 should have an identical result with the paired t-test. However, it is not the case. What am I missing here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please see the following small simulation study,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;set.seed(20180528)&#xA;&#xA;y = matrix(rnorm(2*10), nrow=2)&#xA;&#xA;treat = factor(rep(c(&quot;T1&quot;,&quot;T2&quot;),5))&#xA;subject_id = rep(c(&quot;A&quot;,&quot;B&quot;,&quot;C&quot;,&quot;D&quot;,&quot;E&quot;), each = 2)&#xA;design &amp;lt;- model.matrix(~0+treat)&#xA;colnames(design) &amp;lt;- levels(treat)&#xA;# 1 --- 9.7 Multi-level Experiments&#xA;corfit &amp;lt;- duplicateCorrelation(y,design,block=subject_id)&#xA;fit &amp;lt;- lmFit(y,design, block = subject_id, correlation = corfit$consensus)&#xA;cm &amp;lt;- makeContrasts(&#xA;  Diff = T2-T1,&#xA;  levels=design)&#xA;fit2 &amp;lt;- contrasts.fit(fit, cm)&#xA;fit2 &amp;lt;- eBayes(fit2)&#xA;topTable(fit2, coef=&quot;Diff&quot;)&#xA;# logFC     AveExpr          t   P.Value adj.P.Val         B&#xA;# 1 -0.5173411 -0.08265473 -1.3646790 0.1912401 0.3824802 -4.487867&#xA;# 2 -0.1795537 -0.24626262 -0.5743662 0.5737073 0.5737073 -4.785946&#xA;# 2  --- 9.4.1 empirical bayesian paired t test&#xA;design &amp;lt;- model.matrix(~subject_id+treat)&#xA;fit &amp;lt;- lmFit(y, design)&#xA;fit &amp;lt;- eBayes(fit)&#xA;topTable(fit, coef=&quot;treatT2&quot;)&#xA;&#xA;# logFC     AveExpr          t   P.Value adj.P.Val         B&#xA;# 1 -0.5173411 -0.08265473 -1.5408913 0.1619112 0.3238224 -4.340661&#xA;# 2 -0.1795537 -0.24626262 -0.5347977 0.6073295 0.6073295 -4.928314&#xA;&#xA;&#xA;# 3. -----traditional paired t test&#xA;t.test(y[1,]~treat, paired = TRUE)&#xA;# t = 1.3815, df = 4, p-value = 0.2393&#xA;# being equivalent to  &#xA;# ordinary.t &amp;lt;- fit$coef / fit$stdev.unscaled / fit$sigma&#xA;# (1 - pt(abs(ordinary.t[1,ncol(ordinary.t)]), 4))*2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2873" LastEditorUserId="29" LastEditDate="2018-05-30T17:35:49.970" LastActivityDate="2018-05-30T17:35:49.970" Title="Different results between paired t-test, paired t-test-like (empirical Bayesian), and mixed effect model" Tags="|r|limma|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4389" PostTypeId="1" AcceptedAnswerId="4391" CreationDate="2018-05-29T21:30:38.623" Score="6" ViewCount="1321" Body="&lt;p&gt;Let's say I have following example GRanges:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; library(GenomicRanges)&#xA;&amp;gt; gr = GRanges(&#xA;+     seqnames = Rle(c(&quot;chr1&quot;, &quot;chr2&quot;, &quot;chr1&quot;, &quot;chr3&quot;), c(1, 3, 2, 4)),&#xA;+     ranges = IRanges(101:110, end = 111:120, names = head(letters, 10)),&#xA;+     strand = Rle(strand(c(&quot;-&quot;, &quot;+&quot;, &quot;*&quot;, &quot;+&quot;, &quot;-&quot;)), c(1, 2, 2, 3, 2)),&#xA;+     score = 1:10,&#xA;+     GC = seq(1, 0, length=10))&#xA;&amp;gt; gr&#xA;GRanges object with 10 ranges and 2 metadata columns:&#xA;    seqnames     ranges strand |     score                GC&#xA;       &amp;lt;Rle&amp;gt;  &amp;lt;IRanges&amp;gt;  &amp;lt;Rle&amp;gt; | &amp;lt;integer&amp;gt;         &amp;lt;numeric&amp;gt;&#xA;  a     chr1 [101, 111]      - |         1                 1&#xA;  b     chr2 [102, 112]      + |         2 0.888888888888889&#xA;  c     chr2 [103, 113]      + |         3 0.777777777777778&#xA;  d     chr2 [104, 114]      * |         4 0.666666666666667&#xA;  e     chr1 [105, 115]      * |         5 0.555555555555556&#xA;  f     chr1 [106, 116]      + |         6 0.444444444444444&#xA;  g     chr3 [107, 117]      + |         7 0.333333333333333&#xA;  h     chr3 [108, 118]      + |         8 0.222222222222222&#xA;  i     chr3 [109, 119]      - |         9 0.111111111111111&#xA;  j     chr3 [110, 120]      - |        10                 0&#xA;  -------&#xA;  seqinfo: 3 sequences from an unspecified genome; no seqlengths&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is straightforward to subset this GRanges as one would subset an R data.frame, e.g. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; gr$score&#xA; [1]  1  2  3  4  5  6  7  8  9 10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I am writing a function where the user would pass through a string for the metadata column to subset e.g. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;subsetter = function(granges, column){&#xA;    return(granges$column)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I try to use the function as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; subsetter(gr, score)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which outputs &lt;code&gt;NULL&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This obvious doesn't work, as it will subset the &lt;code&gt;gr&lt;/code&gt; by &lt;code&gt;column&lt;/code&gt;, not &lt;code&gt;score&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; subsetter(gr, score)&#xA;NULL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If this was a data.frame, I could subset as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gr[[(score)]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but this gives the error &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in gr[[(score)]] : this S4 class is not subsettable&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Are there any other ways to subset a GRanges besides using &lt;code&gt;$&lt;/code&gt;? &lt;/p&gt;&#xA;" OwnerUserId="1770" LastEditorUserId="146" LastEditDate="2018-05-29T21:37:15.503" LastActivityDate="2018-06-04T14:31:42.183" Title="How to subset a GRanges via an argument passed into a function?" Tags="|r|bioconductor|genomicranges|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4390" PostTypeId="1" AcceptedAnswerId="4393" CreationDate="2018-05-30T01:49:03.213" Score="1" ViewCount="807" Body="&lt;p&gt;I am attempting to get gene regions and their immediate neighborhoods using Bioconductor GenomicRanges packages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is very easy for me to extend the regions around genes if I want to extend it the same distance upstream and downstream as in this example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(GenomicRanges)&#xA;library(Homo.sapiens)&#xA;&#xA;genes = genes(Homo.sapiens)&#xA;genes_5kb_flank = genes + 5000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And if I want to extend both upstream and downstream different amounts centered on the TSS I can use &lt;code&gt;promoters(genes, upstream = 5000, downstream= 1000)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I am having trouble finding a function that is strand-aware and also chromosome length aware that allows me to extend the gene region a different amount upstream vs. downstream... For instance if I wanted a 5kb upstream flanking region and a 1kb downstream flanking region as is used for genomic region annotations on GREAT.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone know of a drop in function for this?  something like &lt;code&gt;promoters()&lt;/code&gt; but for the full gene region instead of only around the TSS?&lt;/p&gt;&#xA;" OwnerUserId="39" LastActivityDate="2018-05-30T08:59:02.270" Title="Expand granges object different amounts upstream vs. downstream" Tags="|bioconductor|sequence-annotation|genomicranges|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4391" PostTypeId="2" ParentId="4389" CreationDate="2018-05-30T08:07:59.397" Score="6" Body="&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;subsetter = function(gr, cname) {&#xA;    return(mcols(gr)[[cname]])&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can then use things like &lt;code&gt;subsetter(gr, &quot;GC&quot;)&lt;/code&gt; and &lt;code&gt;subsetter(gr, &quot;score&quot;)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-05-30T08:07:59.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4392" PostTypeId="2" ParentId="4386" CreationDate="2018-05-30T08:11:18.550" Score="4" Body="&lt;p&gt;You can't build a network of a single cell only with the expression of a single cell. You either need previous known interactions or pathways or you need to use several cells/samples.&#xA;If you use previous known information, you can use pathway information, otherwise you can group some cells and use something along the lines of WGCNA to find a scale-free network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the first case one can add the expression upon it and see what is happening, what edges are kept and which ones not...&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Once you have the network for each cell you can compare the metrics for each cell. Some metrics to look at are the centrality of each gene and the betweneess.&lt;br&gt;&#xA;In a more general way, you can look if it follows a scale-free topology or other type of network structures, or how many nodes are present, how many edges are present... &lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2018-05-30T12:33:53.430" LastActivityDate="2018-05-30T12:33:53.430" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4393" PostTypeId="2" ParentId="4390" CreationDate="2018-05-30T08:59:02.270" Score="2" Body="&lt;p&gt;There doesn't appear to be a built in function for that, so here's a modified version of &lt;code&gt;promoters()&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;expandRange = function(x, upstream=2000, downstream=1000) {&#xA;  strand_is_minus = strand(x) == &quot;-&quot;&#xA;  on_plus = which(!strand_is_minus)&#xA;  on_minus = which(strand_is_minus)&#xA;  start(x)[on_plus] = start(x)[on_plus] - upstream&#xA;  start(x)[on_minus] = start(x)[on_minus] - downstream&#xA;  end(x)[on_plus] = end(x)[on_plus] + downstream&#xA;  end(x)[on_minus] = end(x)[on_minus] + upstream&#xA;  x&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can then call &lt;code&gt;expandedGenes = expandRange(genes, 5000, 1000)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-05-30T08:59:02.270" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4394" PostTypeId="1" AcceptedAnswerId="4397" CreationDate="2018-05-30T13:58:03.240" Score="1" ViewCount="6802" Body="&lt;p&gt;As i have multiple comparison so I m trying this &lt;code&gt;UpSet&lt;/code&gt; plot, but it needs data to be formatted in particular way, one of the ways is as a list, so now the issue is since between one comparison the no of genes coming out may not be same as the other one, so there is a difference in dimension so if i bind them as data frame then i get dimension error, so the best way to get through is make list so what i have tried so far &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A &amp;lt;- read.csv(&quot;GMP_MONO_UP_DOWN_CDS_EPI.txt&quot;,header = TRUE)&#xA;B &amp;lt;- read.csv(&quot;CMP_GMP_UP_DOWN_EPI.txt&quot;,header = TRUE)&#xA;C &amp;lt;- read.csv(&quot;HSC_BLAST_UPDOWN_EPI.txt&quot;,header = TRUE,sep = '\t')&#xA;&#xA;dim(A)&#xA;dim(B)&#xA;ONE &amp;lt;- A[c(1)]#############just taking out the gene column########&#xA;TWO &amp;lt;- B[c(1)]#############just taking out the gene column########&#xA;THREE &amp;lt;- B[c(1)]#############just taking out the gene column########&#xA;dim(ONE)&#xA;dim(TWO)&#xA;b1 &amp;lt;- apply(ONE,2,list) &#xA;b2 &amp;lt;- apply(TWO,2,list)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now this is the code from their help page &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;upset(movies, nsets = 7, nintersects = 30, mb.ratio = c(0.5, 0.5),&#xA;      order.by = c(&quot;freq&quot;, &quot;degree&quot;), decreasing = c(TRUE,FALSE))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;so the data &quot;movies&quot; class is data frame, there is not dimension issue as such as in my case if i want to intersect two comparison which may not have the same length, so how do i overcome the issue any suggestion or help would be highly appreciated. Update my question with one more data to add.&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2018-05-31T06:59:07.013" LastActivityDate="2018-06-04T10:56:54.540" Title="Preparing data for UpSet Plot in R" Tags="|r|upsetr|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4395" PostTypeId="1" CreationDate="2018-05-30T14:27:02.940" Score="2" ViewCount="66" Body="&lt;p&gt;On Encode, it's pretty easy to find out replicate information about the experiments, however I am having some difficulty finding this information on BLUEPRINT DDC, at &lt;a href=&quot;http://dcc.blueprint-epigenome.eu&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dcc.blueprint-epigenome.eu&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where can I find information which describes technical replicates and biological (isogenic) replicates?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="823" LastEditorUserId="298" LastEditDate="2018-05-30T18:50:11.983" LastActivityDate="2018-06-08T11:10:50.837" Title="Blueprint replicates" Tags="|database|data-retrieval|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4397" PostTypeId="2" ParentId="4394" CreationDate="2018-05-30T14:56:48.537" Score="3" Body="&lt;p&gt;Don't you just need a binary matrix with presence/absence data for all of the genes?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(dplyr)&#xA;combined &amp;lt;- full_join(data.frame(gene=ONE, GMP_MONO=1),&#xA;                      data.frame(gene=TWO, CMP=1))&#xA;combined[is.na(combined)] &amp;lt;- 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Edit:&#xA;To loop over a large number of data frames, the easiest solution is to use the &lt;code&gt;reduce&lt;/code&gt; function from the &lt;a href=&quot;https://github.com/tidyverse/purrr&quot; rel=&quot;nofollow noreferrer&quot;&gt;purrr&lt;/a&gt; package:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;combined &amp;lt;- reduce(list(data.frame(gene=c('gene1', 'gene2', 'gene3'), set1=1),&#xA;                        data.frame(gene=c('gene3', 'gene4', 'gene5'), set2=1),&#xA;                        data.frame(gene=c('gene1', 'gene4'), set3=1)&#xA;                       ), full_join)&#xA;combined[is.na(combined)] &amp;lt;- 0&#xA;combined&#xA;#    gene set1 set2 set3&#xA;# 1 gene1    1    0    1&#xA;# 2 gene2    1    0    0&#xA;# 3 gene3    1    1    0&#xA;# 4 gene4    0    1    1&#xA;# 5 gene5    0    1    0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="581" LastEditorUserId="581" LastEditDate="2018-05-31T07:32:42.327" LastActivityDate="2018-05-31T07:32:42.327" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4398" PostTypeId="2" ParentId="4394" CreationDate="2018-05-30T14:57:43.713" Score="1" Body="&lt;p&gt;You need to set a common number of rows, one for each gene (make sure that the gene identifiers are the same for both files). Then in the first column you can set 1 for the file A, and 0 otherwise. In the second column you can do the same.&#xA;Then you add this dataframe as the input of the &lt;code&gt;upset&lt;/code&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;genes &amp;lt;- unique(c(ONE, TWO))&#xA;df &amp;lt;- data.frame(A = ifelse(genes %in% A, 1, 0), B = ifelse(genes %in% b, 1, 0))&#xA;upset(df, ...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-05-30T14:57:43.713" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4399" PostTypeId="2" ParentId="4389" CreationDate="2018-05-30T17:41:06.487" Score="4" Body="&lt;p&gt;In addition to Devon’s answer, you can use &lt;a href=&quot;http://adv-r.had.co.nz/Computing-on-the-language.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;non-standard evaluation&lt;/a&gt; to make this work with R symbol names rather than string arguments:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;subsetter = function (gr, cname) {&#xA;    cname = as.character(substitute(cname))&#xA;    mcols(gr)[[cname]]&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now using &lt;code&gt;subsetter(gr, score)&lt;/code&gt; works analogously to &lt;code&gt;gr$score&lt;/code&gt;, which may be desirable. But, likewise, you’ll no longer be able to use a variable:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;var = 'score'&#xA;subsetter(gr, var) # fails, same as `gr$var`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29" LastActivityDate="2018-05-30T17:41:06.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4401" PostTypeId="1" AcceptedAnswerId="4403" CreationDate="2018-05-30T21:49:33.677" Score="4" ViewCount="129" Body="&lt;p&gt;I am trying to use bedtools and fetch sequences from a whole genome fasta file inside the script I get region coordinates as variables.&#xA;For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr_ = &quot;chr1&quot;&#xA;start = 3000&#xA;end = 3402&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am not sure how to wrap my function, because the example provided by &lt;a href=&quot;https://daler.github.io/pybedtools/autodocs/pybedtools.bedtool.BedTool.sequence.html#pybedtools.bedtool.BedTool.sequence&quot; rel=&quot;nofollow noreferrer&quot;&gt;BedTool.sequence&lt;/a&gt; only uses coordinates in a string form.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you please help me to have something like this?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = pybedtools.BedTool(chr_, start, end, from_variables=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2876" LastEditorUserId="48" LastEditDate="2018-05-31T07:07:43.380" LastActivityDate="2018-05-31T07:07:43.380" Title="How to use variables while calling pybedtools" Tags="|bedtools|pybedtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4402" PostTypeId="2" ParentId="4357" CreationDate="2018-05-31T00:52:35.393" Score="1" Body="&lt;p&gt;This is resolved by installing the more recent version of the code from GitHub.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;git clone https://github.com/lmcinnes/umap&#xA;cd umap&#xA;pip install --user -r requirements.txt&#xA;python setup.py install --user&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As Peter said, this kind of question is best directed to the Issues page on their GitHub repository.&lt;/p&gt;&#xA;" OwnerUserId="163" LastActivityDate="2018-05-31T00:52:35.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4403" PostTypeId="2" ParentId="4401" CreationDate="2018-05-31T01:22:43.957" Score="2" Body="&lt;p&gt;You have to take your variables and make them into a single string variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr_ = &quot;chr1&quot;&#xA;start = 3000&#xA;end = 3402&#xA;as_str = ' '.join([chr_, str(start), str(end)])&#xA;# or &#xA;as_str2 = &quot;{} {} {}&quot;.format(chr_, start, end)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Once you have the string built you can use the example in the &lt;a href=&quot;https://daler.github.io/pybedtools/autodocs/pybedtools.bedtool.BedTool.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedTool docs&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a = BedTool(as_str, from_string=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="964" LastEditorUserId="964" LastEditDate="2018-05-31T01:29:00.713" LastActivityDate="2018-05-31T01:29:00.713" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4404" PostTypeId="2" ParentId="3211" CreationDate="2018-05-31T05:30:23.043" Score="4" Body="&lt;p&gt;&lt;strong&gt;We cannot assume that doublets will produce more UMIs&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would caution the assumption that all doublets will have twice the UMI levels of isolated single-cells. Many &quot;doublets&quot; could contain multiplets of 3 or more cells, depending on how many cells have been loaded into the experiment. Most vendors of single-cell technologies quote multiplet rates for doublets, triplets, etc combined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, we cannot assume that the data from multiplet (doublet) cells is of high quality or produces comparable reads and UMI counts than we would expect from two single cells combined. Some tests of doublets (such as using mixed mouse and human samples) have found that the data generated from doublet droplets or nanowells has relatively low UMI abundance. This may be due to the protocols having been optimised for droplets or nanowells containing only one cell. Especially where tissue preparation is required this is a concern and is consistent with observations that doublet cells appear to be broken or contaminated with debris when they can be identified. This is highly dependent on the tissue you and working with, sample preparation, and the platform used to separate single-cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not been able to reproduce the above result using data from various species (using total UMI counts per droplet or housekeeper genes), although this is work still in progress. Even in the data shown, many multiplet samples have similar total UMI counts to the true single-cells. Therefore, they cannot be separated by a simple threshold (despite being significantly different). There have been several tools developed already for detecting multiplets (and filtering them out) but this is still an area of active development and there are more ways that these could be refined. In many single-cell analyses, arbitrary thresholds are being used to filter the data before further analysis and this may be inadvertently reducing the impact of multiplets if they have produced fewer UMI counts than single-cells for which the protocols have been optimised for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Experimental protocols can be optimised to reduce doublets&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have any insight into how these bioinformatics solutions to issues with multiplets could be used or improved I encourage you to pursue it. However, I suspect that the eventual solution to this issue is in optimising the single-cell NGS experiments for the sample type and platform that you are working with. Technology in single-cell 'omics is continuing to develop and any solution which produces more useful single-cell data than contaminants or doublets (that need to be removed), the more cells can be sampled at the same cost. There is a strong incentive for experimentalists and single-cell technology vendors to optimise this issue.&lt;/p&gt;&#xA;" OwnerUserId="1855" LastEditorUserId="1855" LastEditDate="2018-06-02T14:11:01.300" LastActivityDate="2018-06-02T14:11:01.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4405" PostTypeId="1" AcceptedAnswerId="4407" CreationDate="2018-05-31T14:27:29.163" Score="2" ViewCount="63" Body="&lt;p&gt;I am trying to use HIVdb(&lt;a href=&quot;https://hivdb.stanford.edu/hivdb/by-sequences/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hivdb.stanford.edu/hivdb/by-sequences/&lt;/a&gt;) program. I have a HIV protease which I want to test with most commonly used PIs. But when I entered it in fasta sequence format to the HIVdb, it give me error code saying &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Critical: There were no Protease, Reverse Transcriptase, or Integrase genes found, refuse to process.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;For example, if I download FASTA sequence of 1HPV from PDB and paste it in HIVdb input sequence window as below,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PQITLWQRPLVTIKIGGQLKEALLDTGADDTVLEEMSLPGRWKPKMIGGIGGFIKVRQYDQILIEICGHKAIGTVLVGPT&#xA;PVNIIGRNLLTQIGCTLNF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get the error code. Any suggestion on how to input sequences in HIVdb?&#xA;(Note: I am new to this subject)&lt;/p&gt;&#xA;" OwnerUserId="2880" LastEditorUserId="298" LastEditDate="2018-05-31T14:54:50.900" LastActivityDate="2018-05-31T15:04:07.890" Title="HIVdb program and FASTA sequence input" Tags="|fasta|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4407" PostTypeId="2" ParentId="4405" CreationDate="2018-05-31T15:04:07.890" Score="1" Body="&lt;p&gt;The HIVdb page states the following in the second paragraph (emphasis mine):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Sequences can be entered as plain text if just one sequence is entered. Sequences must be entered using the FASTA format if multiple sequences are entered. Sequences can be pasted in the text box or uploaded using the File Upload option. The upper limit is currently 1000 sequences containing 3000 &lt;strong&gt;nucleotides&lt;/strong&gt; per sequence.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The sequence in your question is a protein, so it consists of amino acids, not nucleotides. So, if you instead give a nucleotide sequence, for example &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/KX692212.1?report=genbank&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;KX692212.1 HIV-1 isolate CH47 from Switzerland pol protein (pol) and gag protein (gag) genes, partial cds&#xA;CCTCAGATCACTCTTTGGCAACGACCCCTCGTCACAATAAAGATAGGGGGGCAACTAAAGGAAGCTCTAT&#xA;TAGATACAGGAGCAGATGATACAGTATTAGAAGAAATGAGTTTGCCAGGAAGATGGAAACCAAAAATGAT&#xA;AGGGGGAATTGGAGGTTTTATCAAAGTAAGACAGTATGATCAGATACTCATAGAAATCTGTGGACATAAA&#xA;GCTATAGGTACAGTATTAGTAGGACCTACACCTGTCAACATAATTGGAAGAAATCTGTTGACTCAGATTG&#xA;GTTGCACTTTAAATTTT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The program will work as expected:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rUzD4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rUzD4.png&quot; alt=&quot;HIVdb output&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For future reference, I found the KX692212 sequence above by taking your protein sequence and running a &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PAGE=Translations&amp;amp;PROGRAM=tblastn&amp;amp;PAGE_TYPE=BlastSearch&amp;amp;BLAST_SPEC=&quot; rel=&quot;nofollow noreferrer&quot;&gt;tBLASTn&lt;/a&gt; analysis against HIV sequences in NCBI's &lt;code&gt;nr&lt;/code&gt; database. That was the first hit.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-05-31T15:04:07.890" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4408" PostTypeId="1" CreationDate="2018-06-01T06:49:45.470" Score="2" ViewCount="585" Body="&lt;p&gt;In my gzipped VCF file I have many variants, and they report allele frequency such as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##INFO=&amp;lt;ID=CAF,Number=.,Type=String,Description=&quot;An ordered, comma delimited list of allele frequencies based on 1000Genomes, starting with the reference allele followed by alternate alleles as ordered in the ALT column. Where a 1000Genomes alternate allele is not in the dbSNPs alternate allele set, the allele is added to the ALT column. The minor allele is the second largest value in the list, and was previuosly reported in VCF as the GMAF. This is the GMAF reported on the RefSNP and EntrezSNP pages and VariationReporter&quot;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CAF=0.5747,0.4253&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to filter out the variants &gt;= 1% allelle frequency. How do I do that? &lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="96" LastEditDate="2019-03-18T17:43:01.337" LastActivityDate="2019-03-18T17:43:01.337" Title="How to filter for &quot;CAF=0.5747,0.4253&quot;" Tags="|vcf|file-formats|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4409" PostTypeId="1" AcceptedAnswerId="4474" CreationDate="2018-06-01T11:17:09.553" Score="5" ViewCount="546" Body="&lt;p&gt;I was reading the KEGG &lt;a href=&quot;https://www.kegg.jp/kegg/docs/plea.html&quot; rel=&quot;noreferrer&quot;&gt;plea&lt;/a&gt; and I found that it doesn't forbid using the KEGG API. Then, what is in the FTP server &lt;a href=&quot;https://www.bioinformatics.jp/docs/subscription_personal.pdf&quot; rel=&quot;noreferrer&quot;&gt;license for personal use/academic use&lt;/a&gt; that it is not covered by the API?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or I could download all the database via the API?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: I couldn't find how to download which genes are in which pathways but it might be I didn't expend enough time with the queries.&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="298" LastEditDate="2018-06-01T11:20:35.073" LastActivityDate="2018-06-21T18:44:44.060" Title="KEGG FTP vs KEGG API" Tags="|database|kegg|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4410" PostTypeId="2" ParentId="4395" CreationDate="2018-06-01T14:52:48.440" Score="0" Body="&lt;p&gt;Unfortunately this information isn't as clearly laid out in Blueprint as it is in ENCODE. You can view metadata for experiments from one of the many data viewers (&lt;a href=&quot;http://epigenomesportal.ca/ihec/grid.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;IHEC Data Portal&lt;/a&gt;, the Blueprint DCC you mentioned, &lt;a href=&quot;http://deepblue.mpi-inf.mpg.de/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepBlue&lt;/a&gt;, or the EGA dataset itself), but these tend to not contain technical/biological replicate information like ENCODE does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You often have to dive into the data from the original publication to get that information&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2018-06-01T14:52:48.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4411" PostTypeId="1" AcceptedAnswerId="4413" CreationDate="2018-06-01T16:52:35.147" Score="1" ViewCount="324" Body="&lt;p&gt;I have to search a specific sequence in a set of cells (&gt; 1k cells) from a single-cell experiment done with &lt;a href=&quot;https://www.10xgenomics.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;10X genomics&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As input file I have a single bam file, and 24 fastqs, therefore each file contains information about several cells. How can I search for a specific sequence and underpin which single cells have it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should be able to search for a specific sequence in a bam using samtools, etc. But the issue here is how to demultiplex the data from different cells to understand which cell provide the sequence.&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-06-01T17:59:01.070" Title="How to search a specific sequence in BAM files for 10X experiment" Tags="|rna-seq|scrnaseq|10x-genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4412" PostTypeId="1" CreationDate="2018-06-01T17:49:13.930" Score="6" ViewCount="218" Body="&lt;p&gt;Problem is to generate a random BED interval given the following constraints:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;minimum &lt;code&gt;start&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;maximum &lt;code&gt;end&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;fixed length&lt;/li&gt;&#xA;&lt;li&gt;maximum number of masked bases (similar to &lt;code&gt;-maxN&lt;/code&gt; option in &lt;code&gt;faSplit&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;set of intervals to avoid overlap with&lt;/li&gt;&#xA;&lt;li&gt;stay within chromosome size constraints&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I can think of computationally intensive ways to code it up from scratch, but am wondering if there is a more efficient approach before re-inventing the wheel.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seq_records = {x.name: x for x in SeqIO.parse('path/to/genome.fa', 'fasta')}&#xA;&#xA;def generate_random_interval(chrom, lower, upper, length, maxrep=1.0, avoid_intervals=None):&#xA;    while True:&#xA;        start = np.random.randint(lower, upper - length)&#xA;        end = start + length&#xA;        regenerate = False&#xA;&#xA;        for interval in avoid_intervals:&#xA;            if (interval.start &amp;lt; end) or (interval.end &amp;gt; start):&#xA;                regenerate = True&#xA;        if 1. * seq_records[chrom].seq[lower, upper].count('N') / length &amp;gt; maxrep:&#xA;            regenerate = True&#xA;&#xA;        if not regenerate:&#xA;            break&#xA;&#xA;    return chrom, start, end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2884" LastEditorUserId="298" LastEditDate="2018-06-01T18:26:01.000" LastActivityDate="2018-07-02T04:56:52.460" Title="Generating random BED intervals given constraints" Tags="|python|bed|bedtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4413" PostTypeId="2" ParentId="4411" CreationDate="2018-06-01T17:59:01.070" Score="2" Body="&lt;p&gt;The bam file has bam tags which say what reads belong to what cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/output/bam&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/output/bam&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2018-06-01T17:59:01.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4414" PostTypeId="1" AcceptedAnswerId="4429" CreationDate="2018-06-01T20:45:57.720" Score="5" ViewCount="156" Body="&lt;p&gt;I am analyzing &lt;strong&gt;viral metagenomics&lt;/strong&gt; data (Illumina Miseq) for the first time. I have used &lt;strong&gt;&lt;a href=&quot;http://denovoassembler.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ray&lt;/a&gt;&lt;/strong&gt; (reference below) for &lt;em&gt;de novo&lt;/em&gt; viral genome assembly before but I haven't done metagenomics analysis before.&lt;/p&gt;&#xA;&lt;p&gt;I know that there are some tools like Metavelvet for metagenomics. But I still need to find time to learn those tools.&lt;/p&gt;&#xA;&lt;p&gt;For now, can I do quick and dirty analysis using following method ?&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;De novo assemby using &lt;a href=&quot;http://denovoassembler.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ray&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Create a personal database of all the viral ref genomes using NCBI database&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Blast all the contigs generated by Ray against personal database.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I assume this should give me some idea about the viral genomes present in my reads. What could be the potential flaws of this method in comparison to the tools like Metavelvet that are dedicated to metagenomics?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Ray Meta: scalable de novo metagenome assembly and profiling.&#xA;Sébastien Boisvert, Frédéric Raymond, Élénie Godzaridis, François Laviolette and Jacques Corbeil.&#xA;Genome Biology (BioMed Central Ltd).&#xA;13:R122, Published: 22 December 2012&lt;/p&gt;&#xA;" OwnerUserId="1684" LastEditorUserId="3848" LastEditDate="2022-03-04T20:38:31.943" LastActivityDate="2022-03-04T20:38:31.943" Title="Viral Metagenomics" Tags="|assembly|covid-19|metagenome|illumina|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4416" PostTypeId="2" ParentId="4412" CreationDate="2018-06-02T01:35:14.480" Score="2" Body="&lt;p&gt;You can perhaps collapse requirements for (1), (2), and (6), so long as bounds from (1) and (2) fall within the bounds of (6). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To help with this, you could use &lt;code&gt;fetchChromSizes&lt;/code&gt; from the Kent UCSC utilities to quickly get chromosomes and chromosomal bounds for your genome of interest. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a chromosome and a &lt;code&gt;maximumEnd&lt;/code&gt; that falls within its bounds, uniformly sample from &lt;code&gt;[0, maximumEnd-minimumStart-maxLength)&lt;/code&gt;, adding &lt;code&gt;minimumStart&lt;/code&gt; to determine start position and &lt;code&gt;maxLength&lt;/code&gt; as length to get the stop position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To obtain the desired number of samples in an efficient way, you could pipe these &quot;candidate elements&quot; into &lt;code&gt;bedmap --echo-map-size&lt;/code&gt; operations on repeatmasked regions (4) and &lt;code&gt;bedops -n 1&lt;/code&gt; on &quot;regions-to-avoid&quot; (5) operations to do &lt;a href=&quot;https://en.wikipedia.org/wiki/Rejection_sampling&quot; rel=&quot;nofollow noreferrer&quot;&gt;rejection sampling&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ REPEATMASK_THRESHOLD=123&#xA;$ SAMPLES=1000&#xA;$ someScriptThatGeneratesCandidateIntervalsWithinBounds \&#xA;    | sort-bed - \&#xA;    | bedmap --echo-map-size --echo --delim '\t' - repeatmaskedRegions.bed \&#xA;    | awk -vthreshold=${REPEATMASK_THRESHOLD} '($1&amp;lt;threshold)' \&#xA;    | cut -f2- \&#xA;    | bedops -n 1 - regionsToAvoid.bed \&#xA;    | head -${SAMPLES} \&#xA;    &amp;gt; qualifyingCandidates.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will be relatively efficient, because once the &lt;code&gt;head&lt;/code&gt; process at the end writes the number of &lt;code&gt;SAMPLES&lt;/code&gt; to &lt;code&gt;qualifyingCandidates.bed&lt;/code&gt;, it will send a &lt;code&gt;SIGINT&lt;/code&gt; signal up the pipeline that will terminate the upstream processes. So you're only doing sampling to the extent needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-07-02T04:56:52.460" LastActivityDate="2018-07-02T04:56:52.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4417" PostTypeId="2" ParentId="4408" CreationDate="2018-06-02T13:35:04.833" Score="0" Body="&lt;p&gt;This could be done with &lt;code&gt;bcftools view&lt;/code&gt;. To include variants where the first element in CAF is &gt; &lt;em&gt;x&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view -i 'INFO/CAF[0] &amp;gt; 0.1' ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606/VCF/common_all_20180418.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, it appears that CAF is encoded as a list of strings in this input so the &lt;code&gt;&amp;gt;&lt;/code&gt; operator cannot be used. In addition, you may want more control on the filtering. In this case, I would suggest using a dedicated library in your favourite language for dealing with VCF files. In python, you could use pysam along these lines:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python -c &quot;import pysam&#xA;invcf= pysam.VariantFile('ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606/VCF/common_all_20180418.vcf.gz')&#xA;&#xA;# Write to stdout&#xA;outvcf= pysam.VariantFile('-', mode= 'w', header= invcf.header)&#xA;&#xA;for line in invcf:&#xA;    if float(line.info['CAF'][0]) &amp;gt; 0.1:&#xA;        outvcf.write(line)&#xA;outvcf.close()&#xA;invcf.close()&#xA;&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would stay away from parsing VCF files with regex, awk, or custom text processing scripts as even if the format is quite well defined they can be quite tricky.&lt;/p&gt;&#xA;" OwnerUserId="339" LastActivityDate="2018-06-02T13:35:04.833" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4418" PostTypeId="2" ParentId="4251" CreationDate="2018-06-02T15:08:15.153" Score="1" Body="&lt;p&gt;It sounds like you’ve already done differential expression analysis (with genes or transcripts) and a pathway enrichment analysis to find these. Differential expression of these functional groups is an interesting finding in its own right but there are several further investigations that could follow, depending on your biological research interests.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clustering the expression patterns of the differentially expressed genes would be able to show if there are relationships between them (such as inter-gene correlations) or relationships to other phenotype data. This is commonly performed with unsupervised hierarchical clustering and displayed by a heatmap. You could also look into whether these differentially expressed genes are expressed in a tissue-specific manner. There is plenty of reference material available from ENCODE, FANTOM4/5, or GTEx that could be used for comparison (as well as cancer datasets) and the Human Cell Atlas Project (currently underway).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach is pathway structure. There are many hierarchical annotation of gene functions such as gene ontology (GO terms) that overlap and are ranked: identifying smaller more specific pathways for future investigations. No pathway annotation is perfect so it’s best to replicate important findings across pathway databases (ideally different expression datasets as well). These functional groups of genes aren’t unordered: the structural relationships between pathways are well characterised in some cases. This includes many gene regulatory networks involving transcription factors. These graph structures are supported on databases such as WikiPathways and Reactome. Network analysis is relatively unexplored compared to differential expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While this is a bioinformatics question, there’s always a need to consider experimental validation as a future direction. If you already have a hypothesis based on your analysis, it may be more appropriate to design an experiment or contact experimental collaborators. If you have access to further genomics data, you could also explore how these differences are regulated: CAGE-Seq for enhancers and lncRNA, ATAC-Seq, and 3C/4C for chromatin structure, or Bisulfite-Seq and ChIP-Seq for epigenetic marks. An integrative analysis of expression and epigenomics data would be especially compelling and would likely account for different activity of transcription factor binding sites. Unfortunately, this can’t be done without more ‘omics data available but it would greatly benefit investigations of histones if it is feasible.&lt;/p&gt;&#xA;" OwnerUserId="1855" LastEditorUserId="1855" LastEditDate="2018-06-02T15:14:39.570" LastActivityDate="2018-06-02T15:14:39.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4420" PostTypeId="2" ParentId="4263" CreationDate="2018-06-02T15:28:38.417" Score="0" Body="&lt;p&gt;You can use the raw counts to normalise your data. It is import to do so before doing a differential expression analysis: otherwise you will detect more reads from samples with higher coverage and from longer genes. A gene with longer mRNA molecules will produce more reads than a shorter one for the same number of transcript molecules. Samples will not produce the same number of reads due to variation in cDNA extraction, amplification, library preparation, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This does not inherently mean that a sample expresses every gene more than another sample. Hence we compare samples by relative rather than absolute expression (for microarray and RNA-Seq data). This is in principle, very similar to using housekeeper genes in qPCR experiments except we adjust for total RNA or reads, gene length, and batch effects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are many ways to achieve this but I recommend using the &lt;em&gt;&lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/limma.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;limma&lt;/a&gt;&lt;/em&gt; package in R. This provides the “voom” normalisation procedure. “voom” is documented in a separate &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/gb-2014-15-2-r29&quot; rel=&quot;nofollow noreferrer&quot;&gt;follow-up&lt;/a&gt; paper to the &lt;a href=&quot;https://academic.oup.com/nar/article-lookup/doi/10.1093/nar/gkv007&quot; rel=&quot;nofollow noreferrer&quot;&gt;limma paper&lt;/a&gt;. It produces normalised data (on a log-scale) which is compatible with downstream analysis provided by the package (including built-in Venn diagram and volcano plot functions). See the package vignette for more details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, everything here only applies to bulk RNASeq experiments and procedures for single-cell analysis are different.&lt;/p&gt;&#xA;" OwnerUserId="1855" LastEditorUserId="1855" LastEditDate="2018-06-02T15:34:35.197" LastActivityDate="2018-06-02T15:34:35.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4421" PostTypeId="2" ParentId="3994" CreationDate="2018-06-02T16:05:23.103" Score="2" Body="&lt;p&gt;Gene Signatures are sets of genes to report a consensus signal or function. They are not defined in a particular way but can be defined by any chosen group depending on the biological function that you are interested in. Typically these are defined by differential gene expression analysis or principal component analysis, although other approaches such as network analysis or time-course data could also be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are plenty of examples of sets of genes published as gene signatures (especially in the field of cancer research). Those that have been cross-validated in other datasets or replicated by other research groups are ideal. If you wish to generate such a “signature” for a particular biological function, you could follow the approaches demonstrated in these published signatures (especially those that have been widely used and cited).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A gene expression signature is used to generate a consensus expression signal for these groups of genes (representing the given biological function). A widely accepted way to do this across samples of bulk RNASeq (or microarray data) is a “meta-gene” from the first eigenvector (v[,1]) of the singular value decomposition (SVD). This dimension reduction identifies a major contributor to variation across samples (in these genes) without negatively correlated genes averaging out. Dimension reduction can also be achieved by PCA and tSNE, depending on the requirements of your analysis.&lt;/p&gt;&#xA;" OwnerUserId="1855" LastActivityDate="2018-06-02T16:05:23.103" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4422" PostTypeId="1" AcceptedAnswerId="4424" CreationDate="2018-06-02T23:44:47.063" Score="4" ViewCount="98" Body="&lt;p&gt;So, I have the genotyping data of about 650,000 SNPs for 96 individuals. I already know the Y DNA haplogroup of these individuals, so to some extent, I have a gross understanding of their ancestry. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the best way to go about doing this? From what I have been reading, maybe a PCA would be a good idea? Suppose individuals 1-10 belong to a particular population (unique ancestary) then their SNP genotypes should be significantly similar to each other than say individuals 11-20 who belong to another population group. So, in theory I should get two clusters in my PCA plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the genotype of each SNP for each individual. How would I go about creating a PCA plot or any other alternative way of analysis for this kind of questions. &lt;/p&gt;&#xA;" OwnerUserId="2887" LastEditorUserId="2887" LastEditDate="2018-06-03T15:57:00.023" LastActivityDate="2018-07-03T22:06:42.913" Title="Statistical approach to compare the SNP genotyping data among set of individuals" Tags="|snp|phylogenetics|snp-chip|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4424" PostTypeId="2" ParentId="4422" CreationDate="2018-06-03T13:52:45.647" Score="3" Body="&lt;p&gt;PCA sounds like a good start to see a rough population structure among your 96 individuals, this step is half a sane thing to do, but also it can give you a hint how many distinct populations there are in your dataset. I have not personally run PCA on SNP data, but it seems that there are quite lot of tutorials online. For instance &lt;a href=&quot;https://privefl.github.io/bigsnpr/articles/how-to-PCA.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt; looks  nice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you know how many categories you can expect in your population, you can use &lt;a href=&quot;https://web.stanford.edu/group/pritchardlab/structure.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Structure&lt;/a&gt; to classify individuals to the categories.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2018-06-03T21:27:05.140" LastActivityDate="2018-06-03T21:27:05.140" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4425" PostTypeId="1" CreationDate="2018-06-03T22:37:49.887" Score="2" ViewCount="302" Body="&lt;p&gt;I have generated proteins structure of capsid protein of two strains of viruses. I have modeled them showing Coulombic electrostatic potential described below where positive charge is blue and negative charge is red. I have opened them in two separate windows for comparison. I have following two issues:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I keep two molecules on the same plane when they are opened in two windows. Is there any function that lets us know which side or which angle the molecule is facing so that I can keep both molecules at same position for comparison.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I was able to label residue on the surface. But whenever I move the molecule some of the residue start moving and some remain static. I want to show that there is difference is electrostatic potential because of difference in the residue at some particular site. I have not been able to do that because those residue keep moving haphazardly?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;It would be great if someone could help me with this.&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://raw.githubusercontent.com/lrjoshi/sample/master/SVA-Indiana-P1-Strucure.png&quot; alt=&quot;Image generated in Chimera&quot; /&gt;&lt;/p&gt;&#xA;" OwnerUserId="1684" LastEditorUserId="3848" LastEditDate="2022-03-15T01:38:44.123" LastActivityDate="2022-03-15T01:38:44.123" Title="Label protein strucuture residues in Chimera" Tags="|proteins|protein-structure|covid-19|3d-structure|virus|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4426" PostTypeId="1" CreationDate="2018-06-04T02:02:13.120" Score="1" ViewCount="293" Body="&lt;p&gt;I am trying to create a SAF file for the virus sequence. I can download .gff3 and .gb format for the virus gene sequence but they are not supported by &lt;strong&gt;featureCounts&lt;/strong&gt;. I need to create my own SAF. I tried making one but I am confused about the GeneID and Chr. The manual says Chr should match Chromosome number in the BAM or SAM file generated by &lt;strong&gt;align&lt;/strong&gt; function. But viruses don't have chromosome. Could anybody help me with this please?&lt;/p&gt;&#xA;&lt;p&gt;Here is the link of the virus sequence which I am trying to create a SAF table for.&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/AY386263.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/AY386263.1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Quoted from &lt;a href=&quot;https://www.rdocumentation.org/packages/Rsubread/versions/1.22.2/topics/featureCounts&quot; rel=&quot;nofollow noreferrer&quot;&gt;R documentation website&lt;/a&gt;&#xA;&lt;strong&gt;featureCounts&lt;/strong&gt; is a general-purpose read summarization function, which assigns to the genomic features (or meta-features) the mapped reads that were generated from genomic DNA and RNA sequencing.&#xA;... Both SAM and BAM format input files are accepted.&lt;/p&gt;&#xA;&lt;p&gt;...paraphrasing&#xA;There is an in-built annotation that uses SAF format however &lt;code&gt;annot.ext&lt;/code&gt; will override this for a custom SAF file.&lt;/p&gt;&#xA;" OwnerUserId="1684" LastEditorUserId="3848" LastEditDate="2022-03-05T14:10:02.867" LastActivityDate="2022-03-05T14:10:02.867" Title="How to create a SAF file for virus sequence?" Tags="|r|bioconductor|covid-19|rna|virus|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4427" PostTypeId="1" CreationDate="2018-06-04T04:06:50.660" Score="3" ViewCount="54" Body="&lt;p&gt;Suppose I have all sequences of genes or proteins of certain species, are there any free tools I can use to generate a &lt;a href=&quot;https://en.wikipedia.org/wiki/Flux_balance_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;flux balance analysis&lt;/a&gt; model out of them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I was thinking is more like to annotate all the enzymes based on some annotation method. Then mapping these enzymes to known meta-metabolic networks that we know for instance the meta-metabolic network in KEGG database.&lt;/p&gt;&#xA;" OwnerUserId="2004" LastEditorUserId="2004" LastEditDate="2018-06-18T23:09:52.193" LastActivityDate="2018-06-18T23:09:52.193" Title="Is there any tools that can generating FBA model from genome sequences or proteome sequences?" Tags="|sequence-analysis|sequence-annotation|modelling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4428" PostTypeId="2" ParentId="4427" CreationDate="2018-06-04T07:31:49.393" Score="2" Body="&lt;p&gt;To generate flux balance, one should now the stoichiometric numbers of all the reactions that are taking place, as well as the conditions (that could affect the reactions). I am not sure we have enough knowledge, time and computationally power to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment I'm only aware of a &lt;a href=&quot;https://www.cell.com/abstract/S0092-8674%2812%2900776-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; simulating the &lt;em&gt;Mycoplasma genitalium&lt;/em&gt; cell. But I'm not sure it was worth the effort as I haven't seen replicated or redone with other species, perhaps the model was not accurate enough to be worth it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tools used are &lt;a href=&quot;http://www.wholecell.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The licenses are not a standard, but at least they seem free.&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2018-06-04T10:06:25.940" LastActivityDate="2018-06-04T10:06:25.940" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4429" PostTypeId="2" ParentId="4414" CreationDate="2018-06-04T09:11:15.370" Score="2" Body="&lt;p&gt;The specifics will depend on the experimental setup and data, but a few general comments...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Metagenomic data is often large volume and high in microbial diversity but low and uneven in coverage; single genome assemblers may suffer particularly since abundant (or closely related) species/strains will tend to be interpreted as repeats within a genome rather than deriving from different genomes, leading to assembly errors. Checkout &lt;a href=&quot;https://www.nature.com/articles/nmeth.4458&quot; rel=&quot;nofollow noreferrer&quot;&gt;CAMI&lt;/a&gt; for some recent benchmarks of mategenomic assmeblers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) You might want to think about filtering your database to certain viruses, e.g. you don't expect to find RNA viruses in DNA sequencing. If the sample is an animal body site (like human gut) then consider restricting to viruses found in that organism (if it's an environmental sample this may not be possible).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) BLAST will be slow, consider &lt;a href=&quot;https://github.com/bbuchfink/diamond&quot; rel=&quot;nofollow noreferrer&quot;&gt;DIAMOND&lt;/a&gt; for a drop in faster replacement for BLASTP. Even faster approaches use only specific marker genes (like Metaphlan2 or Mocat2) or k-mer methods (like kraken or clark). Again CAMI benchmarks many of these and other metagenomic taxonomic classifiers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, I think possibly a missing step is filtering the non-viral sequences. You haven't said if your sequencing was enriched for viruses or depleted for bacteria/host prior to sequencing but if it isn't then the vast majority of your sequences will likely be non-viral (e.g. bacterial or host). Removing or marking these early can speed up downstream analysis and remove false positives (e.g. the human genome contains integrated viruses, bacteria contain prophages etc.). Also it is normally important to look for and remove potential contaminants like lab reagents or cloning vectors.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2018-06-04T09:11:15.370" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4430" PostTypeId="2" ParentId="4340" CreationDate="2018-06-04T10:15:30.050" Score="3" Body="&lt;p&gt;In addition to the other answer (to reduce the number of threads), you might also need to add a &lt;code&gt;--genomeChrBinNbits&lt;/code&gt; argument. Your reference has more than 5000 scaffolds, so you need to adjust the &lt;code&gt;--genomeChrBinNbits&lt;/code&gt; argument to reduce RAM consumption. Found &lt;a href=&quot;https://github.com/alexdobin/STAR/issues/103&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They recommend:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;--genomeChrBinNbits} = min(18, log2(GenomeLength/NumberOfReferences))&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In your case:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Genome size 17G, number of scaffolds 735945&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So &lt;code&gt;--genomeChrBinNbits&lt;/code&gt; should be set to log2(17,000,000,000/735,945) = 14.4955, so let's say 14 or 15.&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2018-06-04T13:59:39.147" LastActivityDate="2018-06-04T13:59:39.147" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4431" PostTypeId="2" ParentId="4394" CreationDate="2018-06-04T10:56:54.540" Score="7" Body="&lt;p&gt;We can plot using lists as well, using &lt;code&gt;fromList&lt;/code&gt; function, see example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(UpSetR)&#xA;&#xA;myGeneSets &amp;lt;- list(&#xA;  set1 = c(&quot;gene1&quot;,&quot;gene2&quot;,&quot;gene3&quot;),&#xA;  set2 = c(&quot;gene1&quot;,&quot;gene4&quot;,&quot;gene5&quot;, &quot;gene7&quot;),&#xA;  set3 = c(&quot;gene1&quot;,&quot;gene7&quot;)&#xA;  )&#xA;&#xA;upset(fromList(myGeneSets))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;fromList&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;A function to convert a list of named vectors to a data frame&#xA;  compatible with UpSetR.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fromList(myGeneSets)&#xA;#   set1 set2 set3&#xA;# 1    1    1    1&#xA;# 2    1    0    0&#xA;# 3    1    0    0&#xA;# 4    0    1    0&#xA;# 5    0    1    0&#xA;# 6    0    1    1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="131" LastActivityDate="2018-06-04T10:56:54.540" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4432" PostTypeId="2" ParentId="4389" CreationDate="2018-06-04T12:41:14.140" Score="5" Body="&lt;p&gt;In such cases it is best to check the &lt;code&gt;str&lt;/code&gt; of an object &lt;em&gt;gr&lt;/em&gt;. Then we could see that meta data is just a &lt;em&gt;dataframe&lt;/em&gt; inside &lt;em&gt;S4&lt;/em&gt; object:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gr@elementMetadata&#xA;# DataFrame with 10 rows and 2 columns&#xA;#        score        GC&#xA;#    &amp;lt;integer&amp;gt; &amp;lt;numeric&amp;gt;&#xA;# 1          1 1.0000000&#xA;# 2          2 0.8888889&#xA;# ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Once we know how to access the &lt;em&gt;dataframe&lt;/em&gt;, we can then subset the same way as we would subset a &lt;em&gt;dataframe&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gr@elementMetadata[, &quot;score&quot; ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't see the point of wrapping this into a function, but if it is needed:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;subsetter &amp;lt;- function(gr, cname) gr@elementMetadata[, cname ]&#xA;&#xA;# test the function&#xA;subsetter(gr, &quot;score&quot;)&#xA;# [1]  1  2  3  4  5  6  7  8  9 10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, we could use &lt;code&gt;GenomicRanges::elementMetadata()&lt;/code&gt; function to extract &lt;em&gt;dataframe&lt;/em&gt; then subset:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;subsetter &amp;lt;- function(gr, cname) elementMetadata(gr)[, cname ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="131" LastEditorUserId="131" LastEditDate="2018-06-04T14:31:42.183" LastActivityDate="2018-06-04T14:31:42.183" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4433" PostTypeId="1" AcceptedAnswerId="4435" CreationDate="2018-06-04T16:03:57.920" Score="3" ViewCount="120" Body="&lt;p&gt;I have a dataset of pdb-format protein structure decoys/models generated by tertiary structure prediction programs. Many of these are missing hydrogen atoms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, I'd like to determine the rough location of side-chain hydrogens in glycine residues, given the CA, C, N, and O coordinates. Is this possible? I don't need too much precision - I just need the general direction relative to the rest of the residue. More specifically, I want a vector originating at the CA that points roughly in the direction of the side chain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've looked at software for generating hydrogen atoms from pdb structures, however, I need to know which ones are glycine side chains, and was wondering if there might be a simpler way to predict hydrogen locations given coordinates of backbone atoms.&lt;/p&gt;&#xA;" OwnerUserId="2897" LastEditorUserId="48" LastEditDate="2018-06-05T14:00:59.760" LastActivityDate="2018-06-05T14:00:59.760" Title="Determining position of side chain hydrogen in glycine residues from coordinates of backbone atoms" Tags="|proteins|protein-structure|pdb|3d-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4435" PostTypeId="2" ParentId="4433" CreationDate="2018-06-04T17:26:05.910" Score="3" Body="&lt;p&gt;Glycine has a single hydrogen atom as its side chain:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BKoQT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BKoQT.png&quot; alt=&quot;Glycine from Wikipedia&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the six bond angles with the CA atom in the middle are about 109°&#xA;(C-CA-N, C-CA-HA3, C-CA-HA2, N-CA-HA3, N-CA-HA2 and HA3-CA-HA2 using the CCD naming convention). This defines the rough direction of the hydrogen atoms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But which of the two hydrogens is the side chain?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is a left-handed protein, I'd assume that it's the hydrogen pointing toward the screen, similarly to the side chain in L-alanine:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YZjyB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YZjyB.png&quot; alt=&quot;L-alanine&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sub&gt;The images are taken from WikiMedia Commons:&#xA;&lt;a href=&quot;https://commons.wikimedia.org/wiki/File:Glycine-3D-balls.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;glycine&lt;/a&gt;&#xA;and &lt;a href=&quot;https://commons.wikimedia.org/wiki/File:L-alanine-3D-balls.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;L-alanine&lt;/a&gt;.&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2018-06-04T17:26:05.910" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4437" PostTypeId="1" AcceptedAnswerId="4438" CreationDate="2018-06-05T10:31:42.533" Score="3" ViewCount="539" Body="&lt;p&gt;I want to perform phylogenetic analysis using &lt;a href=&quot;https://www.dendropy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dendropy&lt;/a&gt;. I have installed dendropy using command &lt;code&gt;sudo pip install -U dendropy&lt;/code&gt; but when I import it in python3.5.1 it gives followinng error: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; &amp;gt; python3&#xA;   Python 3.5.1 |Anaconda 2.5.0 (64-bit)| (default, Dec  7 2015, 11:16:01) &#xA;   [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux&#xA;   Type &quot;help&quot;, &quot;copyright&quot;, &quot;credits&quot; or &quot;license&quot; for more information.&#xA; &amp;gt;&amp;gt;&amp;gt; import dendropy&#xA; Traceback (most recent call last):&#xA; File &quot;&amp;lt;stdin&amp;gt;&quot;, line 1, in &amp;lt;module&amp;gt;&#xA; ImportError: No module named 'dendropy'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I again installed dendropy using the above given command as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sudo pip3 install -U dendropy&#xA;Requirement already up-to-date: dendropy in /usr/local/lib/python3.5/dist-packages (4.4.0)&#xA;Requirement not upgraded as not directly required: setuptools in /usr/local/lib/python3.5/dist-packages (from dendropy) (39.2.0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After doing this again the error remains the same and I am still not able to import dendropy, any solution to this problem?&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="3848" LastEditDate="2020-05-08T20:41:06.743" LastActivityDate="2020-05-08T20:41:06.743" Title="Issues installing Dendropy" Tags="|python|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4438" PostTypeId="2" ParentId="4437" CreationDate="2018-06-05T12:44:17.343" Score="3" Body="&lt;p&gt;You should &lt;strong&gt;never&lt;/strong&gt; install python packages as root or with &lt;code&gt;sudo&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You're already using conda, so &lt;code&gt;conda install -c bioconda -c conda-forge dendropy&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-06-05T12:44:17.343" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4440" PostTypeId="1" AcceptedAnswerId="4442" CreationDate="2018-06-05T18:52:12.590" Score="4" ViewCount="99" Body="&lt;p&gt;When reading about allelic fraction (AF) and SNPs in cancer research, they always mention the fact that they're using heterozygous SNPs (&lt;em&gt;informative&lt;/em&gt; SNPs). Why is this? Why can't we use homozygous SNPs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, what is the reference base (from the reference genome)?&lt;/p&gt;&#xA;" OwnerUserId="2862" LastActivityDate="2018-06-09T15:57:29.113" Title="Use of heterozygous SNPs in cancer research: why?" Tags="|ngs|snp|cancer|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4441" PostTypeId="1" AcceptedAnswerId="4459" CreationDate="2018-06-06T07:04:08.160" Score="3" ViewCount="1634" Body="&lt;p&gt;Hi am trying to align RNA-seq reads using STAR through following commands,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;STAR --runMode alignReads --runThreadN 26  --outSAMtype BAM  SortedByCoordinate Unsorted --sjdbOverhang 100  --genomeDir wheat --readFilesIn G3_cleaned_R1.fastq G3_cleaned_R2.fastq --sjdbGTFtagExonParentTranscript wheat.gff3 &#xA;Jun 05 13:53:59 ..... started STAR run&#xA;Jun 05 13:53:59 ..... loading genome&#xA;Jun 05 14:07:12 ..... started mapping&#xA;Jun 05 18:42:47 ..... started sorting BAM&#xA;&#xA;EXITING because of FATAL ERROR: number of bytes expected from the BAM bin   does not agree with the actual size on disk: 509186583   0   45                                                                                        &#xA;&#xA;EXITING because of FATAL ERROR: number of bytes expected from the BAM bin does not agree with the actual size on disk: 648593301   0   47                                                                                        &#xA;&#xA;Jun 05 18:42:47 ...... FATAL ERROR, exiting                                                                                                                                                                                      &#xA;Jun 05 18:42:47 ...... FATAL ERROR, exiting                                                                                                                                                                                      &#xA;EXITING because of FATAL ERROR: number of bytes expected from the BAM bin does not agree with the actual size on disk: 660412146   0   46                                                                                        &#xA;Jun 05 18:42:47 ...... FATAL ERROR, exiting                                                                                                                                                                                      &#xA;EXITING because of FATAL ERROR: number of bytes expected from the BAM bin does not agree with the actual size on disk: 470931698   0   44                                                                                        &#xA;&#xA;Jun 05 18:42:47 ...... FATAL ERROR, exiting                                                                                                                                                                                      &#xA;EXITING because of FATAL ERROR: number of bytes expected from the BAM bin does not agree with the actual size on disk: 653155717   0   43                                                                                        &#xA;&#xA;Jun 05 18:42:47 ...... FATAL ERROR, exiting                                                                                                                                                                                      &#xA;Segmentation fault (core dumped)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Although there is much free space on my system as shown below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; df -Th&#xA;  Filesystem     Type      Size  Used Avail Use% Mounted on&#xA;  devtmpfs       devtmpfs   63G     0   63G   0% /dev&#xA;  tmpfs          tmpfs      63G  136K   63G   1% /dev/shm&#xA;  tmpfs          tmpfs      63G  1.9M   63G   1% /run&#xA;  tmpfs          tmpfs      63G     0   63G   0% /sys/fs/cgroup&#xA;  /dev/sda3      ext4      3.6T  1.4T  2.2T  39% /&#xA;  /dev/sda2      ext4      9.8G   57M  9.2G   1% /boot&#xA;  tmpfs          tmpfs      13G     0   13G   0% /run/user/480&#xA;  tmpfs          tmpfs      13G   16K   13G   1% /run/user/1000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How this problem can be solved?&lt;/p&gt;&#xA;" OwnerUserId="1194" LastActivityDate="2018-10-29T10:32:23.120" Title="Disk space error while aligning reads using STAR" Tags="|sam|rna-alignment|star|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="4442" PostTypeId="2" ParentId="4440" CreationDate="2018-06-06T09:09:21.067" Score="5" Body="&lt;p&gt;I’m no longer working in tumour sequencing so I’m by no means an expert. But in a nutshell, the reason is that, as indicated, homozygous SNPs &lt;em&gt;aren’t informative&lt;/em&gt;: if your allelic fraction is 100%, meaning you only observe a single nucleotide at a given position, we don’t know whether we’re dealing with a tumour related mutation. In fact, we probably don’t because the sample likely contains healthy as well as tumour cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By contrast, tumour-specific variants always have an AF &amp;lt; 100%, in other words they exhibit some degree of heterozygosity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, in healthy tissues you’d expect to see one of three cases: your variant allele (= sequence base ≠ reference) has a frequency of either 0% or 100% (homozygous), or 50% (heterozygous), ignoring rare somatic mutations. With tumour samples, by contrast, the fraction will vary depending on the penetrance of the variant in the tumour, and on the fraction of healthy to tumour cells in the sample.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2018-06-06T09:09:21.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4443" PostTypeId="1" CreationDate="2018-06-06T15:50:33.837" Score="4" ViewCount="198" Body="&lt;p&gt;In classic phylogenetic inference one is usually given various orthologue sequences of a given gene across various species. Those sequences are then multiple aligned and used to construct a phylogenetic tree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say, I split the multiple alignment of this gene into two parts (part A and B).&#xA;My question now is: What is the most appropriate test to determine if part A and B share the same phylogenetic history?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having looked through literature, it appears this can be done by constructing phylogenetic trees separately for part A and B and test for incongruence using &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Kishino–Hasegawa (&lt;strong&gt;KH test&lt;/strong&gt;); &lt;/li&gt;&#xA;&lt;li&gt;Shimodaira–Hasegawa (&lt;strong&gt;SH-test&lt;/strong&gt;); &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;SOWH-test&lt;/strong&gt; etc. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;as nicely summarized in &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S153204640500081X?via%3Dihub&quot; rel=&quot;nofollow noreferrer&quot;&gt;Plantet et all,&lt;/a&gt; and &lt;a href=&quot;https://doi.org/10.1080/106351500750049752&quot; rel=&quot;nofollow noreferrer&quot;&gt;Goldman et al.&lt;/a&gt; etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However the above test rely on constructed trees beforehand which might be incorrect due to inappropriate algorithms/parameters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was therefore wondering if there are more appropriate test maybe without having to construct trees at all?&lt;/p&gt;&#xA;" OwnerUserId="1461" LastEditorUserId="3848" LastEditDate="2020-05-07T21:53:28.407" LastActivityDate="2020-05-07T21:53:28.407" Title="Test to determine if two genes/exons share the same evolutionary histories?" Tags="|sequence-alignment|phylogenetics|phylogeny|statistics|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="4444" PostTypeId="2" ParentId="4372" CreationDate="2018-06-06T18:41:08.683" Score="1" Body="&lt;p&gt;The above given problems were solved using manual tree construction through &lt;a href=&quot;http://www.megasoftware.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MEGA&lt;/a&gt;. MEGA allows manual construction/manipulation of trees. The newick format obtained after manually constructing the desired tree using MEGA is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;((((((((((((T_M184,T_L131,T_PF5633),T),((R_M207,M734,P224,P280xM173,R1a_M417,R1a1_M459,R1a1a_M17,R1b_M343,R1b_M73,R1b1a1b_V1636,R1b1a2_V88,R1b1b_PH155,R2_M479,R2a_M124),R)),((Q F1096,Q_L27,Q_L56,Q_M24,Q_M346),Q)),(((((((O1a1_B384,O1b1_F2320),O1b2_P49),O2IMS_JST0213554),O2a1_F51),O_O2a2_F525),O3_M122),O)),(((N3,N1a1_M46),N1a2_L666),N)),(((((L_M20,L1_M22),L1a_M27),L1b_M317),L1c_M357),L)),(K,K_M9)),((((((((((((((((((((((J_M304,J1a_PF7257),J1a2a1a2_P58),J1a2a2_PF7264),J1a3_Z1828),J2_M172),J2a_L25),J2a1_L26),J2a1_M319),J2a1_Z1846),J2a1_Z387),J2a1_Z467),J2a1_Z6046),J2a1_Z6063),J2a1_Z6065),J2a1_Z7671),J2a1_Z7700),J2a2_PF5008),J2b_M241),J2b1_M205),J2b2_M241),J2b2_Z2456),J)),(((((((((((I_M170,I1_M253),I2a1_Isles),I2a1_S21825),I2a1a_M26),I2a1b3_L621),I2a2a_M223),I2a2b_L38),I2c2_Y16419),I),((H1_M69,H1a_M82),H)),((((((((((((G_M201,G1_M342),G2a1_Z6552),G2a2a_PF3147),G2a2b2b_PF3359),G2a2b1_M406),G2a2b2a1a_U1),G2a2b2a1b_L497),G2a2b2a1c_CTS342),G2a_L30),G2b_M3155),G2b_M283),G))),((E1b1a_V38,E1b1b_M215),(E2_M75,E))),(((((((C_RPS4Y711,C1a_V20),C2_M217),C2b1a_F3985),C2C1_Z1338),C),B2b_M112),B));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1194" LastActivityDate="2018-06-06T18:41:08.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4446" PostTypeId="2" ParentId="3581" CreationDate="2018-06-06T21:47:11.513" Score="2" Body="&lt;p&gt;It’s right there on the &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/what-is-cell-ranger&quot; rel=&quot;nofollow noreferrer&quot;&gt;cellranger manual&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#aggregate results of counts for separate samples&#xA;cellranger aggr&#xA;#analyse the combined results&#xA;cellranger reanalyze&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note: I’m not sure this applies to the above case where batch effects could be an issue. However, this is how the analysis is performed for 10X data from multiple runs in principle.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/aggregate&quot; rel=&quot;nofollow noreferrer&quot;&gt;cellranger aggr&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/reanalyze&quot; rel=&quot;nofollow noreferrer&quot;&gt;cellranger reanalyze&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1855" LastActivityDate="2018-06-06T21:47:11.513" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4447" PostTypeId="2" ParentId="3581" CreationDate="2018-06-07T12:38:22.383" Score="2" Body="&lt;p&gt;See &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/29608177&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper from the Marioni group&lt;/a&gt;, where they propose a method for correcting batch effects between single cell sequencing experiments when each experiment contains different sub-populations:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Large-scale single-cell RNA sequencing (scRNA-seq) data sets that are&#xA;  produced in different laboratories and at different times contain&#xA;  batch effects that may compromise the integration and interpretation&#xA;  of the data. Existing scRNA-seq analysis methods incorrectly assume&#xA;  that the composition of cell populations is either known or identical&#xA;  across batches. We present a strategy for batch correction based on&#xA;  the detection of mutual nearest neighbors (MNNs) in the&#xA;  high-dimensional expression space. Our approach does not rely on&#xA;  predefined or equal population compositions across batches; instead,&#xA;  it requires only that a subset of the population be shared between&#xA;  batches. We demonstrate the superiority of our approach compared with&#xA;  existing methods by using both simulated and real scRNA-seq data sets.&#xA;  Using multiple droplet-based scRNA-seq data sets, we demonstrate that&#xA;  our MNN batch-effect-correction method can be scaled to large numbers&#xA;  of cells&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-06-07T12:38:22.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4448" PostTypeId="1" AcceptedAnswerId="4450" CreationDate="2018-06-07T16:33:24.770" Score="5" ViewCount="1732" Body="&lt;p&gt;I have paired end ChIP-seq data with 101 bp and 2 biological replicates for each one. I have done peak calling with macs2 but I have some questions about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also faced with an warning:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;WARNING @ Thu, 07 Jun 2018 17:06:05: #2 Since the d (197) calculated from paired-peaks are smaller than 2*tag length, it may be influenced by unknown sequencing problem!&lt;br&gt;&#xA;  WARNING @ Thu, 07 Jun 2018 17:06:05: #2 You may need to consider one of the other alternative d(s): 197&lt;br&gt;&#xA;  WARNING @ Thu, 07 Jun 2018 17:06:05: #2 You can restart the process with --nomodel --extsize XXX with your choice or an arbitrary number. Nontheless, MACS will continute computing. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I have added &lt;code&gt;--nomodel --extsize 197 ; --nomodel --extsize 147&lt;/code&gt; and &lt;code&gt;--nomodel --extsize 202&lt;/code&gt; (separately to macs2 command) and got the results without any warning? Which one is more correct?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are broad peaks extended of narrow peaks? If I apply intersect between them I should expect find 100% overlap between narrow peaks and broad ones?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Which kind of peak (narrow/broad) is proper for H3k27ac, H3k4me1, H3k4me3, H3k27me3 study?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If there is no control group for using as background, can I use default parameters?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2911" LastEditorUserId="48" LastEditDate="2018-06-08T17:25:58.197" LastActivityDate="2018-06-14T14:11:23.530" Title="Macs2 peak calling?" Tags="|chip-seq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4449" PostTypeId="1" AcceptedAnswerId="4451" CreationDate="2018-06-07T17:41:55.033" Score="4" ViewCount="1779" Body="&lt;p&gt;I am trying to perform variant calling on a BAM file generated through STAR version &lt;strong&gt;STAR_2.6.0b&lt;/strong&gt; for wheat genome using GATK haplotypecaller as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gatk HaplotypeCaller -I sorted.bam -R wheat.fa -O st.vc&#xA;Using GATK jar /home/geslab/bin/gatk-package-4.0.4.0-local.jar                                                                                                                                                                   &#xA;Running:                                                                                                                                                                                                                         &#xA;java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /home/geslab/bin/gatk-package-4.0.4.0-local.jar HaplotypeCaller -I sorted.bam -R wheat.fa -O st.vc                                                                                                                                                                                       &#xA;02:34:13.450 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/home/geslab/bin/gatk-package-4.0.4.0-local.jar!/com/intel/gkl/native/libgkl_compression.so                                                 &#xA;02:34:13.600 INFO  HaplotypeCaller - ------------------------------------------------------------                                                                                                                                &#xA;02:34:13.600 INFO  HaplotypeCaller - The Genome Analysis Toolkit (GATK) v4.0.4.0                                                                                                                                                 &#xA;02:34:13.600 INFO  HaplotypeCaller - For support and documentation go to https://software.broadinstitute.org/gatk/                                                                                                               &#xA;02:34:13.601 INFO  HaplotypeCaller - Initializing engine                                                                                                                                                                         &#xA;02:34:40.721 INFO  HaplotypeCaller - Done initializing engine                                                                                                                                                                    &#xA;02:34:48.175 INFO  HaplotypeCallerEngine - Disabling physical phasing, which is supported only for reference-model confidence output                                                                                             &#xA;02:34:48.598 INFO  HaplotypeCaller - Shutting down engine                                                                                                                                                                        &#xA;[June 7, 2018 2:34:48 AM PKT] org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCaller done. Elapsed time: 0.59 minutes.                                                                                      &#xA;Runtime.totalMemory()=10305929216                                                                                                                                                                                                &#xA;java.lang.IllegalArgumentException: samples cannot be empty&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I searched for this error the most proposed solution was to validate BAM file as suggested &lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/10020/illegalargumentexception-samples-cannot-be-empty&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The &lt;a href=&quot;https://broadinstitute.github.io/picard/command-line-overview.html#ValidateSamFile&quot; rel=&quot;nofollow noreferrer&quot;&gt;ValidateSamFile&lt;/a&gt; gave following output for the bam file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gatk ValidateSamFile -I sorted.bam -M SUMMARY&#xA;Using GATK jar /home/geslab/bin/gatk-package-4.0.4.0-local.jar&#xA;Running:&#xA;java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /home/geslab/bin/gatk-package-4.0.4.0-local.jar ValidateSamFile -I sorted.bam -M SUMMARY&#xA;02:53:27.498 INFO  NativeLibraryLoader - Loading libgkl_compression.so   from jar:file:/home/geslab/bin/gatk-package-4.0.4.0-local.jar!/com/intel/gkl/native/libgkl_compression.so&#xA;[Thu Jun 07 02:53:27 PKT 2018] ValidateSamFile  --INPUT sorted.bam --MODE   SUMMARY  --MAX_OUTPUT 100 --IGNORE_WARNINGS false --VALIDATE_INDEX true --INDEX_VALIDATION_STRINGENCY EXHAUSTIVE --IS_BISULFITE_SEQUENCED false --MAX_OPEN_TEMP_FILES 8000 --SKIP_MATE_VALIDATION false --VERBOSITY INFO --QUIET false --VALIDATION_STRINGENCY STRICT --COMPRESSION_LEVEL 2 --MAX_RECORDS_IN_RAM 500000 --CREATE_INDEX false --CREATE_MD5_FILE false --GA4GH_CLIENT_SECRETS client_secrets.json --help false --version false --showHidden false --USE_JDK_DEFLATER false --USE_JDK_INFLATER false&#xA;INFO    2018-06-07 02:54:08     SamFileValidator        Validated Read    10,000,000 records.  Elapsed time: 00:00:36s.  Time for last 10,000,000:   36s.  Last read position: TGACv1_scaffold_080127_1DS:28,522&#xA;INFO    2018-06-07 02:54:44     SamFileValidator        Validated Read    20,000,000 records.  Elapsed time: 00:01:12s.  Time for last 10,000,000:   35s.  Last read position: TGACv1_scaffold_602746_7DL:127,507&#xA;INFO    2018-06-07 02:55:20     SamFileValidator        Validated Read    30,000,000 records.  Elapsed time: 00:01:48s.  Time for last 10,000,000:   35s.  Last read position: TGACv1_scaffold_641946_U:17,161&#xA;INFO    2018-06-07 02:55:55     SamFileValidator        Validated Read    40,000,000 records.  Elapsed time: 00:02:23s.  Time for last 10,000,000:   35s.  Last read position: TGACv1_scaffold_471755_6AL:43,097&#xA;INFO    2018-06-07 02:56:31     SamFileValidator        Validated Read    50,000,000 records.  Elapsed time: 00:02:59s.  Time for last 10,000,000:   35s.  Last read position: TGACv1_scaffold_434075_5DL:29,039&#xA;INFO    2018-06-07 02:57:07     SamFileValidator        Validated Read    60,000,000 records.  Elapsed time: 00:03:34s.  Time for last 10,000,000:   35s.  Last read position: TGACv1_scaffold_062491_1DL:40,466&#xA;INFO    2018-06-07 02:57:42     SamFileValidator        Validated Read    70,000,000 records.  Elapsed time: 00:04:10s.  Time for last 10,000,000:   35s.  Last read position: TGACv1_scaffold_558158_7AL:13,072&#xA;INFO    2018-06-07 02:58:18     SamFileValidator        Validated Read    80,000,000 records.  Elapsed time: 00:04:46s.  Time for last 10,000,000:   35s.  Last read position: TGACv1_scaffold_435811_5DL:29,008&#xA;INFO    2018-06-07 02:58:53     SamFileValidator        Validated Read    90,000,000 records.  Elapsed time: 00:05:21s.  Time for last 10,000,000:   35s.  Last read position: TGACv1_scaffold_572030_7AS:9,114&#xA;&#xA;&#xA;## HISTOGRAM    java.lang.String&#xA;Error Type      Count&#xA;ERROR:MATE_NOT_FOUND    107771&#xA;ERROR:MISSING_READ_GROUP        1&#xA;WARNING:MISSING_TAG_NM  96856336&#xA;WARNING:RECORD_MISSING_READ_GROUP       96856336&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In order to remove the errors shown by ValidateSamFile I used &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/tooldocs/4.0.4.0/picard_sam_FixMateInformation.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;FixMateInformation&lt;/a&gt; as suggested &lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/6709/fixing-the-errors-after-running-validatesamfile&quot; rel=&quot;nofollow noreferrer&quot;&gt;here.&lt;/a&gt; as shown below(sharing part of file):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gatk FixMateInformation -I sorted.bam -O mate.bam -MC&#xA;Using GATK jar /home/geslab/bin/gatk-package-4.0.4.0-local.jar&#xA;Running:&#xA;java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /home/geslab/bin/gatk-package-4.0.4.0-local.jar FixMateInformation -I sorted.bam -O mate.bam -MC&#xA;06:13:13.934 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/home/geslab/bin/gatk-package-4.0.4.0-local.jar!/com/intel/gkl/native/libgkl_compression.so&#xA;[Thu Jun 07 06:13:19 PKT 2018] FixMateInformation  --INPUT sorted.bam --OUTPUT mate.bam --ADD_MATE_CIGAR true  --ASSUME_SORTED false --IGNORE_MISSING_MATES true --VERBOSITY INFO --QUIET false --VALIDATION_STRINGENCY STRICT --COMPRESSION_LEVEL 2 --MAX_RECORDS_IN_RAM 500000 --CREATE_INDEX false --CREATE_MD5_FILE false --GA4GH_CLIENT_SECRETS client_secrets.json --help false --version false --showHidden false --USE_JDK_DEFLATER false --USE_JDK_INFLATER false&#xA;INFO    2018-06-07 06:13:24     FixMateInformation      Sorting input into queryname order.&#xA;INFO    2018-06-07 06:23:26     FixMateInformation      Sorting by queryname complete.&#xA;INFO    2018-06-07 06:23:26     FixMateInformation      Output will be sorted by coordinate&#xA;INFO    2018-06-07 06:23:26     FixMateInformation      Traversing query name sorted records and fixing up mate pair information.&#xA;INFO    2018-06-07 06:23:33     FixMateInformation      Processed      1,000,000 records.  Elapsed time: 00:00:06s.  Time for last 1,000,000:    6s.    Last read position: TGACv1_scaffold_361122_4DS:70,952&#xA;INFO    2018-06-07 06:23:42     FixMateInformation      Processed     2,000,000 records.  Elapsed time: 00:00:15s.  Time for last 1,000,000:    8s.    Last read position: TGACv1_scaffold_044571_1BL:711&#xA;INFO    2018-06-07 06:23:50     FixMateInformation      Processed     3,000,000 records.  Elapsed time: 00:00:23s.  Time for last 1,000,000:    8s.   Last read position: TGACv1_scaffold_433135_5DL:36,647&#xA; .&#xA; .&#xA; .&#xA; .&#xA; [Thu Jun 07 06:40:50 PKT 2018] picard.sam.FixMateInformation done.    Elapsed time: 27.62 minutes.&#xA; Runtime.totalMemory()=8004304896&#xA; Tool returned:&#xA; 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After performing FixMateInformation I again validated the BAM file using ValidateSamFile but the errors are still the same as shown above which shows Mate not found, which means error is not fixed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How these errors can be fixed? So that I can perform variant calling using BAM file?&lt;/p&gt;&#xA;" OwnerUserId="1194" LastActivityDate="2019-05-29T13:39:50.857" Title="Issues performing variant calling with GATK" Tags="|rna-seq|variant-calling|gatk|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4450" PostTypeId="2" ParentId="4448" CreationDate="2018-06-07T18:52:02.940" Score="4" Body="&lt;ol&gt;&#xA;&lt;li&gt;Your original command without &lt;code&gt;--nomodel --extsize ...&lt;/code&gt; is probably the most accurate. This warning stems from a time when reads were much much shorter and likely never made that much sense to begin with.&lt;/li&gt;&#xA;&lt;li&gt;Broad peak calling in MACS2 basically works by finding a bunch of nearish narrow peaks and merging them. If you have really broad signals then use something like &lt;code&gt;histoneHMM&lt;/code&gt; instead.&lt;/li&gt;&#xA;&lt;li&gt;H3K27ac, H3K4me1 and H3K4me3 are narrow. H3K27me3 is broad.&lt;/li&gt;&#xA;&lt;li&gt;Yeah, but you'll need to spend a bit more time with the data to ensure that the peaks are reasonable (you'll have to tweak the parameters if not).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-06-07T18:52:02.940" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4451" PostTypeId="2" ParentId="4449" CreationDate="2018-06-07T19:19:31.770" Score="1" Body="&lt;p&gt;The primary error isn't the missing mate, but likely the missing read groups. Use Picard &lt;code&gt;AddOrReplaceReadGroups&lt;/code&gt; to ensure your BAM files have read groups and then run those through the haplotype caller.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-06-07T19:19:31.770" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4452" PostTypeId="1" AcceptedAnswerId="4481" CreationDate="2018-06-08T00:21:59.170" Score="1" ViewCount="332" Body="&lt;p&gt;I am trying to combine two datasets to be able to compare the cells in them and using &lt;code&gt;Seurat&lt;/code&gt; &lt;code&gt;CCA&lt;/code&gt; for it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://satijalab.org/seurat/Seurat_AlignmentTutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/Seurat_AlignmentTutorial.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I am running &lt;code&gt;AlignSubspace&lt;/code&gt; function I am getting a warning for each single dimension that I selected it to be run on:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seurat_union &amp;lt;- AlignSubspace(object = seurat_union, reduction.type = &#xA;    &quot;cca&quot;, grouping.var = &quot;protocol&quot;, dims.align = 1:13) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, I am getting 13 warnings:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; warnings()&#xA;Warning messages:&#xA;1: In dtw(x = align.1, y = align.2, keep.internals = TRUE,  ... :&#xA;Argument dist.method is only useful with multivariate timeseries&#xA;2: In dtw(x = align.1, y = align.2, keep.internals = TRUE,  ... :&#xA;Argument dist.method is only useful with multivariate timeseries&#xA;3: In dtw(x = align.1, y = align.2, keep.internals = TRUE,  ... :&#xA;Argument dist.method is only useful with multivariate timeseries&#xA;4: In dtw(x = align.1, y = align.2, keep.internals = TRUE,  ... :&#xA;Argument dist.method is only useful with multivariate timeseries&#xA;5: In dtw(x = align.1, y = align.2, keep.internals = TRUE,  ... :&#xA;Argument dist.method is only useful with multivariate timeseries&#xA;6: In dtw(x = align.1, y = align.2, keep.internals = TRUE,  ... :&#xA;Argument dist.method is only useful with multivariate timeseries&#xA;7: In dtw(x = align.1, y = align.2, keep.internals = TRUE,  ... :&#xA;Argument dist.method is only useful with multivariate timeseries&#xA;8: In dtw(x = align.1, y = align.2, keep.internals = TRUE,  ... :&#xA;Argument dist.method is only useful with multivariate timeseries&#xA;9: In dtw(x = align.1, y = align.2, keep.internals = TRUE,  ... :&#xA;Argument dist.method is only useful with multivariate timeseries&#xA;10: In dtw(x = align.1, y = align.2, keep.internals = TRUE,  ... :&#xA;Argument dist.method is only useful with multivariate timeseries&#xA;11: In dtw(x = align.1, y = align.2, keep.internals = TRUE,  ... :&#xA;Argument dist.method is only useful with multivariate timeseries&#xA;12: In dtw(x = align.1, y = align.2, keep.internals = TRUE,  ... :&#xA;Argument dist.method is only useful with multivariate timeseries&#xA;13: In dtw(x = align.1, y = align.2, keep.internals = TRUE,  ... :&#xA;Argument dist.method is only useful with multivariate timeseries&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is this warning telling me? What should I pay attention to with my datasets?&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-06-13T21:50:08.790" Title="Seurat CCA, AlignSubspace, Argument dist.method is only useful with multivariate timeseries" Tags="|seurat|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4453" PostTypeId="2" ParentId="4449" CreationDate="2018-06-08T03:05:40.977" Score="0" Body="&lt;p&gt;You could add read groups: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;['STAR', '--twopassMode', 'Basic', '--genomeDir', '/data2/Fshare/FastaAndIndex/IWGSC_v1.0_STAR/',&#xA;         '--runThreadN', '10', '--limitSjdbInsertNsj', '5000000', '--outSAMtype', 'BAM', 'SortedByCoordinate', '--twopass1readsN', '-1',&#xA;         '--sjdbOverhang', '100', '--readFilesIn', sra1 + '_1.fastq', sra1 + '_2.fastq',&#xA;         '--outSAMattrRGline', 'ID:' + rg1, 'SM:' + rg1, 'PL:ILLUMINA'], shell=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2913" LastEditorUserId="48" LastEditDate="2018-06-08T17:27:36.593" LastActivityDate="2018-06-08T17:27:36.593" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4454" PostTypeId="2" ParentId="4395" CreationDate="2018-06-08T11:00:27.203" Score="1" Body="&lt;p&gt;Blueprint does not have technical replicates, you will find that very few sequencing based experiments ever have technical replicates. As biological replicates: each individual is regarded as a replicate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This makes sense for the sort of experimental design which blueprint was designed to answer questions such as &quot;Do individuals from group A have higher or lower H3K4me3 than individuals from groups B&quot; or &quot;Does cell type A have higher accessibility than cell type B&quot;. In each of these cases the within group variability to be considered is encapsulated in the difference between individuals. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overall technical variation is assumed to be similar in all samples or at least not correlated with the contrast of interest (this may well not be true in the second example design I gave) it is safe to directly measure it, as long we know its distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an aside, one of the reasons that ENCODE got away with only two reps is that their reps are effectively technical (repeated preparations from the same cell lines) and thus there is next to no biological variation (which is what our algorithms are set up to find).&lt;/p&gt;&#xA;" OwnerUserId="235" LastEditorUserId="77" LastEditDate="2018-06-08T11:10:50.837" LastActivityDate="2018-06-08T11:10:50.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4455" PostTypeId="1" AcceptedAnswerId="4456" CreationDate="2018-06-08T11:22:32.420" Score="2" ViewCount="320" Body="&lt;p&gt;I'm using R and I'm trying to import specific records from a PDB file (e.g. &lt;code&gt;HETNAM&lt;/code&gt;, &lt;code&gt;HETSYN&lt;/code&gt;). Unfortunately, it seems the &lt;code&gt;read.pdb&lt;/code&gt; function of either &lt;code&gt;biomaRt&lt;/code&gt; or &lt;code&gt;Rpdb&lt;/code&gt; aren't capable of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="1847" LastActivityDate="2018-06-08T13:56:34.897" Title="Using R to import specific records from a PDB file" Tags="|r|pdb|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4456" PostTypeId="2" ParentId="4455" CreationDate="2018-06-08T13:56:34.897" Score="4" Body="&lt;p&gt;If your file is in PDB format as the one shown &lt;a href=&quot;https://files.rcsb.org/view/6C4A.pdb&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; in .pdb format then you can simply extract your desired fields using &lt;a href=&quot;https://www.gnu.org/software/sed/manual/sed.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sed&lt;/a&gt; which is a programming language used to parse and transform text. As for this&lt;a href=&quot;https://files.rcsb.org/view/6C4A.pdb&quot; rel=&quot;nofollow noreferrer&quot;&gt; structure &lt;/a&gt;i.e 6C4A in .pdb format HETNAM and HETSYN can be extracted using sed as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed  -n -e '/HETNAM/ p' -e '/HETSYN/ p'  6C4A.pdb&#xA;&#xA;HETNAM     EJA S-[(1Z)-2-CARBOXY-N-HYDROXYETHANIMIDOYL]-L-CYSTEINE              &#xA;HETNAM     PYR PYRUVIC ACID                                                     &#xA;HETNAM      MG MAGNESIUM ION                                                    &#xA;HETNAM     3NP 3-NITROPROPANOIC ACID                                            &#xA;HETNAM     PEG DI(HYDROXYETHYL)ETHER                                            &#xA;HETNAM     EPE 4-(2-HYDROXYETHYL)-1-PIPERAZINE ETHANESULFONIC ACID              &#xA;HETSYN     EPE HEPES&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1194" LastActivityDate="2018-06-08T13:56:34.897" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4457" PostTypeId="2" ParentId="4440" CreationDate="2018-06-09T15:57:29.113" Score="2" Body="&lt;p&gt;I'm not sure what I'm talking about, but I guess one could re-phrase Konrad's answer in terms of information theory. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The information content of a message (a packet of bits) is inversely proportional to the frequency of those bits. For example, if you randomly extract a word (the message) from an English book and the word is &quot;the&quot;, you don't get much information about the book because &quot;the&quot; is very frequent and you see it everywhere. If, instead, the word happens to be the much rarer &quot;bioinformatics&quot; you start having a pretty good guess what the book is about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About SNPs, homozygote reference genotypes have very little information because most of the genomic positions in most of the people are homozygote reference. Heterozygote and homozygote non-reference are rarer and thus more informative. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I don't think there is anything wrong with using homozygote SNPs. They are ignored because they are so common that their information content is very small. &lt;/p&gt;&#xA;" OwnerUserId="339" LastActivityDate="2018-06-09T15:57:29.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4458" PostTypeId="1" AcceptedAnswerId="4462" CreationDate="2018-06-10T01:23:29.777" Score="7" ViewCount="253" Body="&lt;p&gt;I used &lt;a href=&quot;http://ccb.jhu.edu/software/stringtie/gffcompare.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;gffcompare&lt;/a&gt; recently to compare a newly assembled transcriptome and a published transcriptome. The class code &quot;p&quot; transcripts are &quot;possible polymerase run on&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://ccb.jhu.edu/software/stringtie/gffcompare_codes_2col.gif&quot; alt=&quot;Image&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does that mean? I tried very hard but still cannot find the answer.&lt;/p&gt;&#xA;" OwnerUserId="2118" LastEditorUserId="292" LastEditDate="2018-06-13T11:42:10.223" LastActivityDate="2018-06-13T11:42:10.223" Title="What is &quot;Possible polymerase run-on fragment&quot;?" Tags="|transcriptome|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4459" PostTypeId="2" ParentId="4441" CreationDate="2018-06-10T13:07:15.620" Score="4" Body="&lt;p&gt;STAR versions 2.6.0b and 2.6.0a are unstable,  as written &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/4449/issues-performing-variant-calling-with-gatk&quot;&gt;Issues performing variant calling with GATK&lt;/a&gt; you are using version 2.6.0b. You should switch to version 2.6.0c which is stable.  Also avoid to perform multiple alignments in the same directory. &lt;/p&gt;&#xA;" OwnerUserId="2914" LastActivityDate="2018-06-10T13:07:15.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4460" PostTypeId="1" AcceptedAnswerId="4466" CreationDate="2018-06-10T22:44:04.473" Score="4" ViewCount="115" Body="&lt;p&gt;I have run psi-blast using the NR database, remotely, with one iteration, for several sequences to calculate an evolutionary profile (PSSM) for each of those sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, many of the PSSM files contain lines of &lt;code&gt;-I -I -I -I -I -I -I -I -I -I -I -I -I -I -I -I -I -I -I -I&lt;/code&gt; rather than numerical values.  What does it mean?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;     Last position-specific scoring matrix computed&#xA;&amp;gt;                 A   R   N   D   C   Q   E   G   H   I   L   K   M   F   P   S   T   W   Y   V&#xA;&amp;gt;         1 G  -I -I -I -I -I -I -I -I -I -I -I -I -I -I -I -I -I -I -I -I  &#xA;&amp;gt;         2 N     4  -1  -2  -2   0  -1  -1   0  -2  -1  -1  -1  -1  -2  -1   1   0  -3  -2   0 &#xA;&amp;gt;         3 K    -2  -1   4   4  -3   0   1  -1   0  -3  -4   0  -3  -3  -2   0  -1  -4  -3  -3 &#xA;&amp;gt;         4 E     0  -3  -3  -3   9  -3  -4  -3  -3  -1  -1  -3  -1  -2  -3  -1  -1  -2  -2  -1 &#xA;&amp;gt;         5 K    -2  -2   1   6  -3   0   2  -1  -1  -3  -4  -1  -3  -3  -1   0  -1  -4  -3  -3 &#xA;&amp;gt;         6 A    -1   0   0   2  -4   2   5  -2   0  -3  -3   1  -2  -3  -1   0  -1  -3  -2  -2 &#xA;&amp;gt;         7 D    -2  -3  -3  -3  -2  -3  -3  -3  -1   0   0  -3   0   6  -4  -2  -2   1   3  -1 &#xA;&amp;gt;         8 R     0  -2   0  -1  -3  -2  -2   6  -2  -4  -4  -2  -3  -3  -2   0  -2  -2  -3  -3 &#xA;&amp;gt;         9 Q    -2   0   1  -1  -3   0   0  -2   8  -3  -3  -1  -2  -1  -2  -1  -2  -2   2  -3 &#xA;&amp;gt;        10 K    -1  -3  -3  -3  -1  -3  -3  -4  -3   4   2  -3   1   0  -3  -2  -1  -3  -1   3 &#xA;&amp;gt;        11 V    -1   2   0  -1  -3   1   1  -2  -1  -3  -2   5  -1  -3  -1   0  -1  -3  -2  -2 &#xA;&amp;gt;        12 V    -1  -2  -3  -4  -1  -2  -3  -4  -3   2   4  -2   2   0  -3  -2  -1  -2  -1   1 &#xA;&amp;gt;        13 S    -1  -1  -2  -3  -1   0  -2  -3  -2   1   2  -1   5   0  -2  -1  -1  -1  -1   1 &#xA;&amp;gt;        14 D    -2   0   6   1  -3   0   0   0   1  -3  -3   0  -2  -3  -2   1   0  -4  -2  -3 &#xA;&amp;gt;        15 L    -1  -2  -2  -1  -3  -1  -1  -2  -2  -3  -3  -1  -2  -4   7  -1  -1  -4  -3  -2 &#xA;&amp;gt;        16 V    -1   1   0   0  -3   5   2  -2   0  -3  -2   1   0  -3  -1   0  -1  -2  -1  -2 &#xA;&amp;gt;        17 A    -1   5   0  -2  -3   1   0  -2   0  -3  -2   2  -1  -3  -2  -1  -1  -3  -2  -3 &#xA;&amp;gt;        18 L     1  -1   1   0  -1   0   0   0  -1  -2  -2   0  -1  -2  -1   4   1  -3  -2  -2 &#xA;&amp;gt;        19 E     0  -1   0  -1  -1  -1  -1  -2  -2  -1  -1  -1  -1  -2  -1   1   5  -2  -2   0 &#xA;&amp;gt;        20 G     0  -3  -3  -3  -1  -2  -2  -3  -3   3   1  -2   1  -1  -2  -2   0  -3  -1   4 &#xA;&amp;gt;        21 A    -3  -3  -4  -4  -2  -2  -3  -2  -2  -3  -2  -3  -1   1  -4  -3  -2  11   2  -3 &#xA;&amp;gt;        22 L    -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1 &#xA;&amp;gt;        23 D    -2  -2  -2  -3  -2  -1  -2  -3   2  -1  -1  -2  -1   3  -3  -2  -2   2   7  -1 &#xA;&amp;gt;        24 M    -1   0   0   1  -3   4   4  -2   0  -3  -3   1  -1  -3  -1   0  -1  -2  -2  -2 &#xA;&amp;gt;        25 Y     0  -3  -3  -3   9  -3  -4  -3  -3  -1  -1  -3  -1  -2  -3  -1  -1  -2  -2  -1 &#xA;&amp;gt;        26 K    -4  -4  -4  -4  -4  -4  -4  -4  -4  -4  -4  -4  -4  -4  -4  -4  -4  -4  -4  -4 &#xA;&amp;gt;        27 L    -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1  -1 &#xA;&amp;gt;        28 D    -1  -2  -3  -3  -1  -2  -3  -4  -3   3   3  -3   2   0  -3  -2  -1  -2  -1   2 &#xA;&amp;gt;        29 N  -I -I -I -I -I -I -I -I -I -I -I -I -I -I -I -I -I -I -I -I  &#xA;&amp;gt;        30 S    -2   1  -1  -1  -2  -1  -2  -2  -1  -1  -1  -1  -2   0  -1   0   0  -1  -1  -3 &#xA;&amp;gt;        31 R     4   0  -2   1   4  -1  -3   0  -3   0  -3  -4   4  -1   0  -1  -3  -1   0  -4 &#xA;&amp;gt;        32 Y    -3  -1  -3  -4  -3  -3  -2  -3  -1  -3  -1  -1  -3  -3  -3  -1  -1  -1  -3  -2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1606" LastEditorUserId="298" LastEditDate="2018-06-11T21:23:37.600" LastActivityDate="2018-06-12T13:14:37.460" Title="What does an output of -I for all amino acids mean in a psi-blast pssm matrix file?" Tags="|blast|sequence-homology|phylogenetics|pssm|psi-blast|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4461" PostTypeId="1" AcceptedAnswerId="4480" CreationDate="2018-06-11T05:30:27.697" Score="1" ViewCount="137" Body="&lt;p&gt;In the following picture, you can see that in normal cells (yellow) we have two copies of each gene, while in tumor cells (violet), we just have one copy due to a clonal mono-allelic deletion. I do not really care about the genotype of the two genes, I just need to know that they are heterozygous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9Wvu3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9Wvu3.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, suppose we are talking about somatic heterozygous SNVs and therefore we do not consider deletions anymore. In this case I would like to know the genotype (AA, AB or BB) of tumor and normal cells. Based on my understanding, I expect the AF of the SNVs in the tumor cells to be roughly 0.5 (since I ask for heterozygous SNVs). But what about normal cells? What is their genotype? If I ask for heterozygous SNVs, does that imply that normal cells are homozygous for that alleles?&lt;/p&gt;&#xA;" OwnerUserId="2862" LastActivityDate="2018-06-13T21:05:30.070" Title="Tumor sample and heterozygous SNVs: what is the genotype of the normal cells?" Tags="|ngs|cancer|snv|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4462" PostTypeId="2" ParentId="4458" CreationDate="2018-06-11T06:59:32.270" Score="5" Body="&lt;p&gt;The general idea behind a run-on fragment is that it's background noise. This derives from an open area of the genome that's next to an area that's actually being transcribed. Thus, all of the machinery for transcription is already present, so if the region adopts a bit more of an open conformation then that machinery can sporadically bind and start transcribing. You can get those either up- or down-stream of genes. A lot of things like this are just background transcriptional noise.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-06-11T06:59:32.270" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4463" PostTypeId="2" ParentId="4458" CreationDate="2018-06-11T10:22:47.427" Score="6" Body="&lt;p&gt;On most coding genes (with the exception of replication dependent histone genes), &lt;em&gt;Transcripts&lt;/em&gt; (as opposed to transcrip&lt;strong&gt;tion&lt;/strong&gt;) are terminated by cleavage polyadenylation sites, where the growing transcript is cleaved by a protein complex and the poly-A tail. However, PolII continues to transcribe until it is terminated by torpedo termination (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Terminator_(genetics)#In_eukaryotes&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;). Normally this extra RNA after the polyA site is degraded by Xrn2, but if you capture any particular snapshot of the RNA in the cell at any one time there will be a small amount that corresponds to this run on transcription. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-06-11T10:22:47.427" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4464" PostTypeId="2" ParentId="4206" CreationDate="2018-06-12T11:04:18.347" Score="5" Body="&lt;p&gt;Yes you can modify the reference PDB file and look for the changes and for this purpose you need &lt;a href=&quot;https://www.rcsb.org/pdb/static.do?p=software/software_links/molecular_graphics.html&quot; rel=&quot;noreferrer&quot;&gt;visualizers.&lt;/a&gt; One among these is &lt;a href=&quot;https://www.cgl.ucsf.edu/chimera/&quot; rel=&quot;noreferrer&quot;&gt;Chimera&lt;/a&gt;. You can easily carry out the energy minimization steps using Chimera, first mutate your PDB file manually and then provide it as an input for Chimera and the steps for doing energy minimization are fairly simple as given in Chimera docs &lt;a href=&quot;https://www.cgl.ucsf.edu/chimera/docs/ContributedSoftware/minimize/minimize.html&quot; rel=&quot;noreferrer&quot;&gt;here.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2927" LastActivityDate="2018-06-12T11:04:18.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4465" PostTypeId="1" CreationDate="2018-06-12T12:21:37.127" Score="1" ViewCount="73" Body="&lt;p&gt;I’m (trying) using the GATK4 germline CNV calling pipeline. I successfully got 57 VCFs from my sample batch, called with segments (obtained by merging the contiguous intervals), like in a classic VCF : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS     ID      REF     ALT     QUAL    FILTER  INFO    FORMAT  2046745451-1006_S4&#xA;M       3288    CNV_M_3288_15907        N       &amp;lt;DEL&amp;gt;,&amp;lt;DUP&amp;gt;     .       .       END=15907       GT:CN:NP:QA:QS:QSE:QSS  2:5:9:17:6:21:21&#xA;1       69071   CNV_1_69071_70028       N       &amp;lt;DEL&amp;gt;,&amp;lt;DUP&amp;gt;     .       .       END=70028       GT:CN:NP:QA:QS:QSE:QSS  1:0:1:204:204:204:204&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I got way too much of those intervals, more than 10k. I would like to know if there is an existing tool which count the different segments (variants / intervals common by +/= 75% of their length) in one VCF and gives me the count of the different segments overlapped by segments in other sample in my batch. By counting the most redundant segment, I could determine which are background noise and maybe decrease the number of variants in my VCF by filtering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you. &lt;/p&gt;&#xA;" OwnerUserId="2929" LastActivityDate="2018-06-12T12:21:37.127" Title="Normalize by counting CNV within a pool of VCF files" Tags="|vcf|cnv|gatk|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4466" PostTypeId="2" ParentId="4460" CreationDate="2018-06-12T13:14:37.460" Score="1" Body="&lt;p&gt;It means -inf. It has to do with how the &lt;a href=&quot;https://en.wikipedia.org/wiki/Position_weight_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;PSSM&lt;/a&gt; score is calculated.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2018-06-12T13:14:37.460" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4467" PostTypeId="2" ParentId="4443" CreationDate="2018-06-12T13:25:45.527" Score="2" Body="&lt;p&gt;Since there hasn't been any other answers, I'd thought to share my approach in case someone is having the same issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Gauging from the comments of tendon and NatWH, it seems using the SH-test valid stategy to tackle this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having done some research, I could only find one functional software package, the R-package &lt;a href=&quot;https://github.com/KlausVigo/phangorn&quot; rel=&quot;nofollow noreferrer&quot;&gt;phangorn&lt;/a&gt; which implements both, the SH- as well as SOWH-test and thought to share my R-workflow:&lt;/p&gt;&#xA;&#xA;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reading in multiple-alignment (MLN). Here: amino-acids (AA)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt; library(phangorn)&#xA; mln &amp;lt;- read.phyDat(&quot;my_mln.fasta&quot;, format = &quot;fasta&quot;, type = &quot;AA&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Partitioning MLN into two parts (part A and B) given a predefined breakpoint (here 100)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;breakpoint &amp;lt;- 100&#xA; # partA: 1:breakpoing&#xA; ( mln_partA &amp;lt;- subset(mln, select = 1:breakpoint, site.pattern = FALSE) )&#xA; # 58 sequences with 100 character and 45 different site patterns.&#xA; # The states are a r n d c q e g h i l k m f p s t w y v&#xA;&#xA; ## partB: breakpoint:end&#xA; ( mln_partB &amp;lt;- subset(mln, select = (breakpoint + 1):length(attr(mln, &quot;index&quot;)), site.pattern = FALSE) )&#xA; # 58 sequences with 1702 character and 720 different site patterns.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Constructing tree(s) separately from part A and B using Neighbourhood Joining (NJ) method&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt; ## Note, other tree building algorithms might be used&#xA; ( treeNJ_partA &amp;lt;- NJ(dist.ml(mln_partA)) )&#xA; # Phylogenetic tree with 58 tips and 56 internal nodes.&#xA; # Unrooted; includes branch lengths.&#xA; ( treeNJ_partB &amp;lt;- NJ(dist.ml(mln_partB)) )&#xA; # Phylogenetic tree with 58 tips and 56 internal nodes.&#xA; # Unrooted; includes branch lengths.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Using topology of previously constructed trees and compute optimal likelihood based on partA data&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt; ( treeNJ_partA_fit1 &amp;lt;- optim.pml(pml(treeNJ_partA, mln_partA)) )&#xA; # optimize edge weights:  -1402.126 --&amp;gt; -1379.043&#xA; #&#xA; #  loglikelihood: -1379.041&#xA; #&#xA; # unconstrained loglikelihood: -302.6292&#xA; ( treeNJ_partB_fit1 &amp;lt;- optim.pml(pml(treeNJ_partB, mln_partA)) )&#xA; # optimize edge weights:  -1633.646 --&amp;gt; -1530.745&#xA; #&#xA; #  loglikelihood: -1530.682&#xA; #&#xA; # unconstrained loglikelihood: -302.6292&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Performing SH-test for for both trees&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt; ( shPart1_t12 &amp;lt;- SH.test(treeNJ_partA_fit1, treeNJ_partB_fit1, data = mln_partA, B = 1e5) )&#xA; #      Trees      ln L Diff ln L p-value&#xA; # [1,]     1 -1379.041     0.000 0.48461&#xA; # [2,]     2 -1530.682   151.641 0.00000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As can be seen in this example, the tree constructed for partB doesn't fit partA data as much as the tree constructed from partA.&lt;/p&gt;&#xA;" OwnerUserId="1461" LastEditorUserId="1461" LastEditDate="2018-06-12T15:15:18.260" LastActivityDate="2018-06-12T15:15:18.260" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="4468" PostTypeId="1" AcceptedAnswerId="4471" CreationDate="2018-06-12T17:54:56.487" Score="1" ViewCount="2620" Body="&lt;p&gt;PSI-Blast on a sequence returns a PSSM matrix file containing two matrices, one to the left, one the the right, for all amino acids:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A   R   N   D   C   Q   E   G   H   I   L   K   M   F   P   S   T   W   Y   V&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is the difference between these two matrices, how can I interpret them?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Last position-specific scoring matrix computed, weighted observed percentages rounded down, information per position, and relative weight of gapless real matches to pseudocounts&#xA;            A   R   N   D   C   Q   E   G   H   I   L   K   M   F   P   S   T   W   Y   V   A   R   N   D   C   Q   E   G   H   I   L   K   M   F   P   S   T   W   Y   V&#xA;    1 R    -2   6  -1  -2  -4   1   0  -3   0  -3  -3   2  -2  -3  -2  -1  -1  -3  -2  -3    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  0.00     0.00&#xA;    2 G     0  -3   0  -2  -3  -2  -2   7  -2  -4  -4  -2  -3  -4  -2   0  -2  -3  -4  -4    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  0.00     0.00&#xA;    3 G     0  -3   0  -2  -3  -2  -2   7  -2  -4  -4  -2  -3  -4  -2   0  -2  -3  -4  -4    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  0.00     0.00&#xA;    4 G     0  -3   0  -2  -3  -2  -2   7  -2  -4  -4  -2  -3  -4  -2   0  -2  -3  -4  -4    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  0.00     0.00&#xA;    5 L    -2  -3  -4  -4  -1  -2  -3  -4  -3   2   4  -3   2   0  -3  -3  -1  -2  -1   1    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  0.00     0.00&#xA;    6 C     0  -4  -3  -4  10  -3  -4  -3  -3  -1  -1  -4  -2  -3  -3  -1  -1  -3  -3  -1    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  0.00     0.00&#xA;    7 Y    -2  -2  -2  -4  -3  -2  -2  -4   2  -2  -1  -2  -1   3  -3  -2  -2   3   8  -1    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  0.00     0.00&#xA;    8 C     0  -4  -3  -4  10  -3  -4  -3  -3  -1  -1  -4  -2  -3  -3  -1  -1  -3  -3  -1    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  0.00     0.00&#xA;    9 R    -2   6  -1  -2  -4   1   0  -3   0  -3  -3   2  -2  -3  -2  -1  -1  -3  -2  -3    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  0.00     0.00&#xA;   10 R    -2   6  -1  -2  -4   1   0  -3   0  -3  -3   2  -2  -3  -2  -1  -1  -3  -2  -3    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  0.00     0.00&#xA;   11 R    -2   6  -1  -2  -4   1   0  -3   0  -3  -3   2  -2  -3  -2  -1  -1  -3  -2  -3    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  0.00     0.00&#xA;   12 F    -3  -3  -3  -4  -3  -4  -4  -4  -1   0   0  -4   0   7  -4  -3  -2   1   3  -1    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  0.00     0.00&#xA;   13 C     0  -4  -3  -4  10  -3  -4  -3  -3  -1  -1  -4  -2  -3  -3  -1  -1  -3  -3  -1    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  0.00     0.00&#xA;   14 V     0  -3  -3  -4  -1  -3  -3  -4  -4   3   1  -3   1  -1  -3  -2   0  -3  -1   4    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  0.00     0.00&#xA;   15 C     0  -4  -3  -4  10  -3  -4  -3  -3  -1  -1  -4  -2  -3  -3  -1  -1  -3  -3  -1    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  0.00     0.00&#xA;   16 V     0  -3  -3  -4  -1  -3  -3  -4  -4   3   1  -3   1  -1  -3  -2   0  -3  -1   4    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  0.00     0.00&#xA;   17 G     0  -3   0  -2  -3  -2  -2   7  -2  -4  -4  -2  -3  -4  -2   0  -2  -3  -4  -4    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  0.00     0.00&#xA;   18 R    -2   6  -1  -2  -4   1   0  -3   0  -3  -3   2  -2  -3  -2  -1  -1  -3  -2  -3    0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0  0.00     0.00&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Additionally, at the end of the file, there is additional information, what does it show exactly?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                      K         Lambda&#xA;Standard Ungapped    0.1628     0.3454&#xA;Standard Gapped      0.0410     0.2670&#xA;PSI Ungapped         0.1628     0.3454&#xA;PSI Gapped           0.0410     0.2670&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1606" LastEditorUserId="57" LastEditDate="2018-12-16T17:23:01.897" LastActivityDate="2018-12-16T17:23:01.897" Title="How to interpret a PSSM matrix from PSI-Blast - what do the numbers represent?" Tags="|blast|phylogenetics|pssm|psi-blast|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4470" PostTypeId="1" AcceptedAnswerId="4478" CreationDate="2018-06-12T20:05:02.963" Score="3" ViewCount="5834" Body="&lt;p&gt;I have 209 cells, I clustered them by Seurat to 4 clusters. By Featureplot I am able to track a gene in clusters:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3uIq9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3uIq9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Higher color shows higher expression. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, for some genes I want to highlight some cells in Featureplot so that apart from yellow or red colours I want to colour a subsets of cells with another color. I mean I want to map a list of cells in Featureplot or tSNE plot. Let's say I want to know the location of cells 1, 4, 80 and highlight them with another color. &#xA;My Seurat object in this &lt;a href=&quot;https://drive.google.com/file/d/1rLsRXrxJPYm8cSzSqa5WUR6NYaUr-e9f/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seurat itself beautifully maps the cells in Featureplot for defined genes with a gradient of colours showing the level of expression. Saying I have genes A and B, in &lt;a href=&quot;https://drive.google.com/file/d/1sks1fwe2yWCQp8n7WFUB480pnECBdhPk/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;excel&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have coloured cells that express a gene &gt; mean + se, &amp;lt; mean - se or between these values. For instance, for this gene, 36 cells express this gene &gt; mean + se, I want to map these cells in Featureplot or tSNE plot in distinct colour so I can locate them in clusters easily. Something like binary (on off) expression to relative expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How I can change the colour of some cells based on my threshold for this gene please?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="48" LastEditDate="2018-06-13T15:52:54.090" LastActivityDate="2020-01-17T00:37:41.667" Title="Mapping a list of cells in seurat featureplot" Tags="|r|single-cell|seurat|ggplot2|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4471" PostTypeId="2" ParentId="4468" CreationDate="2018-06-12T20:23:16.693" Score="1" Body="&lt;p&gt;If you read &lt;a href=&quot;https://en.wikipedia.org/wiki/Position_weight_matrix#Creation&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki&lt;/a&gt; carefully, you can see that PSSM is calculated in 3 steps. First frequency is calculated (how much times was the amino acid or nucleotide on that location in the motif), from that you can calculate probability (in the wiki example there were 10 sequences, so each freq. is divided by 10).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the log likelihood is calculated, which are the PSSM values. These are in your first matrix (rounded down). The second matrix shows how much the values are relative from your pseudocounts (I assume it was set to default = 0).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lambda and kappa are estimated to calculate the normalized score (S') for HSP, if you never heard of this before I suggest you read the original psi-blast paper first which you can find &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/9254694&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.  &lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2018-06-12T20:23:16.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4472" PostTypeId="2" ParentId="4206" CreationDate="2018-06-12T21:23:42.003" Score="3" Body="&lt;p&gt;I would recommend you try the FoldX mutation engine.  It is very easy to use and state of the art (it is the main competitor to expensive commercial software).  It can do both of what you want - the mutation and energy optimisation.  It may not work as well as molecular dynamics software, but for your simple task, it is a reasonable solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple command line can do it:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, repair your structure (since, usually PDBs are not energetically optimised, so you could get better energy anyway without doing this):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FoldX --command=repairpdb --pdb=1AJY.pdb&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, either use AlaScan or PositionScan to mutate.  AlaScan only mutates to Alanine.  PositionScan can mutate to any amino acid. More advanced mutations can be done with the BuildModel command.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FoldX --command=PositionScan --pdb=1AJY_Repair.pdb --output-file=1AJY_Repair_MA30d --out-pdb=false --positions=MA30d&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note: 'd' means try mutate the residue to 24 different amino acids; or, you can specify any of those 24, including the standard 20.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which produces two files:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;PS_1AJY_Repair_MA30d_scanning_output.txt&lt;/li&gt;&#xA;&lt;li&gt;energies_30_1AJY_Repair&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(It can also output the mutated pdb file by setting --out-pdb=true.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Either of those files can be used to determine change in free energy upon mutation (ddg ~ delta delta g).  The second file has additional information (the individual values used to calculate total change in free energy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: The FoldX website has very good documentation for each FoldX command.  You can find any additional information you require there.&lt;/p&gt;&#xA;" OwnerUserId="1606" LastEditorUserId="1606" LastEditDate="2018-06-12T21:31:51.003" LastActivityDate="2018-06-12T21:31:51.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4473" PostTypeId="1" AcceptedAnswerId="4483" CreationDate="2018-06-12T22:25:29.623" Score="0" ViewCount="610" Body="&lt;p&gt;Some time ago I asked the question: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/4357/runumap-in-seurat-not-working-module-umap-has-no-attribute-umap&quot;&gt;RunUMAP in Seurat not working: module &amp;#39;umap&amp;#39; has no attribute &amp;#39;UMAP&amp;#39;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was solved by following the answer in it, however, it is still not working when I run a python script from terminal, giving the same error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python python/umap.py -path &#xA;&quot;/Users/nikitavlasenko/python_scripts/output/&quot; -data &#xA;&quot;/Users/nikitavlasenko/python_scripts/data_files/WT_SCID_CCA.csv&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Traceback (most recent call last):&#xA;    File &quot;python/umap.py&quot;, line 26, in &#xA;      result = umap.UMAP().fit_transform(data.T)&#xA;  AttributeError: module 'umap' has no attribute 'UMAP'&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;From inside &lt;code&gt;spyder&lt;/code&gt; the same script runs perfectly, but somehow when running it from the terminal it is not successful. I am new to python, so maybe I am not pointing correctly to &lt;code&gt;umap&lt;/code&gt; library? Maybe it should be imported in a different way? Here is the whole script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import umap&#xA;import numpy as np&#xA;&#xA;def getopts(argv):&#xA;opts = {}  # Empty dictionary to store key-value pairs.&#xA;while argv:  # While there are arguments left to parse...&#xA;    if argv[0][0] == '-':  # Found a &quot;-name value&quot; pair.&#xA;        opts[argv[0]] = argv[1]  # Add key and value to the dictionary.&#xA;    argv = argv[1:]  &#xA;return opts&#xA;&#xA;if __name__ == '__main__':&#xA;  from sys import argv&#xA;  myargs = getopts(argv)&#xA;  print(myargs)&#xA;  path_to_save = myargs['-path']&#xA;  path_to_data = myargs['-data']&#xA;  data = np.loadtxt(open(path_to_data, &quot;rb&quot;), delimiter=&quot;,&quot;, skiprows=1)&#xA;  result = umap.UMAP().fit_transform(data.T)&#xA;  np.savetxt(path_to_save + &quot;/umap.csv&quot;, result, delimiter=&quot;,&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Python version that is used by &lt;code&gt;spyder&lt;/code&gt; and in the terminal seems to be the same: &lt;code&gt;3.6&lt;/code&gt;. I am using &lt;code&gt;Mac OSX&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="2005" LastEditDate="2018-06-12T22:38:36.073" LastActivityDate="2018-06-14T10:12:01.943" Title="Running python script from terminal: umap issue" Tags="|python|umap|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4474" PostTypeId="2" ParentId="4409" CreationDate="2018-06-13T08:25:47.883" Score="2" Body="&lt;p&gt;I found the way to download the &lt;a href=&quot;http://rest.kegg.jp/link/pathway/hsa&quot; rel=&quot;nofollow noreferrer&quot;&gt;genes and pathways&lt;/a&gt; thanks to this answer using &lt;a href=&quot;https://support.bioconductor.org/p/109871/#109959&quot; rel=&quot;nofollow noreferrer&quot;&gt;KEGGREST&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that the &lt;a href=&quot;https://www.kegg.jp/kegg/rest/keggapi.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;API&lt;/a&gt; allows to fully download the database because it allows to download info, list, find, get, conv, link, ddi.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact in the message of 2011-05-21 it mentions that the API includes more options to download the KGML files that previously where only possible to get via the FTP site.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-06-13T08:25:47.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4475" PostTypeId="1" CreationDate="2018-06-13T10:23:05.627" Score="4" ViewCount="176" Body="&lt;p&gt;I have followed the steps described &lt;a href=&quot;https://github.com/fritzsedlazeck/Sniffles/wiki#how-to-build-sniffles&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; to install Sniffles on my Mac Book Pro running macOS High Sierra.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am struggling with this step:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cmake -D CMAKE_C_COMPILER=/opt/local/bin/gcc-mp-4.7 -D CMAKE_CXX_COMPILER=/opt/local/bin/g++-mp-4.7 .. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I do not have an /opt folder, so I get the following output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-- The C compiler identification is unknown&#xA;-- The CXX compiler identification is unknown&#xA;CMake Error at CMakeLists.txt:2 (project):&#xA;  The CMAKE_C_COMPILER:&#xA;&#xA;    /opt/local/bin/gcc-mp-4.7&#xA;&#xA;  is not a full path to an existing compiler tool.&#xA;&#xA;  Tell CMake where to find the compiler by setting either the environment&#xA;  variable &quot;CC&quot; or the CMake cache entry CMAKE_C_COMPILER to the full path to&#xA;  the compiler, or to the compiler name if it is in the PATH.&#xA;&#xA;&#xA;CMake Error at CMakeLists.txt:2 (project):&#xA;  The CMAKE_CXX_COMPILER:&#xA;&#xA;    /opt/local/bin/g++-mp-4.7&#xA;&#xA;  is not a full path to an existing compiler tool.&#xA;&#xA;  Tell CMake where to find the compiler by setting either the environment&#xA;  variable &quot;CXX&quot; or the CMake cache entry CMAKE_CXX_COMPILER to the full path&#xA;  to the compiler, or to the compiler name if it is in the PATH.&#xA;&#xA;&#xA;-- Configuring incomplete, errors occurred!&#xA;See also &quot;/Users/xxx/Documents/xxx/xxx/Sniffles-master/build/CMakeFiles/CMakeOutput.log&quot;.&#xA;See also &quot;/Users/xxx/Documents/xxx/xxx/Sniffles-master/build/CMakeFiles/CMakeError.log&quot;.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unsurprisingly, the error file says that /opt/local/bin/gcc-mp-4.7 does not exist.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I tried:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cmake -D CMAKE_C_COMPILER=/usr/bin/gcc -D CMAKE_CXX_COMPILER=/usr/bin/gcc ..&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get another long message but the most important bit is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CMake Error at /usr/local/Cellar/cmake/3.8.1/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:137 (message):&#xA;  Could NOT find OpenMP (missing: OpenMP_C_FLAGS OpenMP_CXX_FLAGS)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I do not know what to do to solve this error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do have gcc here: /usr/bin/gcc and cmake version 3.8.1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have created an issue on the &lt;a href=&quot;https://github.com/fritzsedlazeck/Sniffles/issues/80&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sniffles GitHub repository&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Update 13th June 2018&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I installed the latest version of gcc:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;brew install gcc&#xA;Warning: gcc 8.1.0 is already installed and up-to-date&#xA;To reinstall 8.1.0, run `brew reinstall gcc`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, I tried to make cmake use gcc and g++ in the /usr/local/bin folder. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cmake -D CMAKE_C_COMPILER=/usr/local/bin/gcc-8 -D CMAKE_CXX_COMPILER=/usr/local/bin/g++-8 ..&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This recreated the same problem:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CMake Error at /usr/local/Cellar/cmake/3.8.1/share/cmake/Modules/FindPackageHandleStandardArgs.cmake:137 (message):&#xA;  Could NOT find OpenMP (missing: OpenMP_C_FLAGS OpenMP_CXX_FLAGS)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Do I need to set the OpenMP_C*_FLAGS to something?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-06-13T21:27:15.433" LastActivityDate="2018-06-13T21:27:15.433" Title="Problem installing last build of Sniffles" Tags="|software-installation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4476" PostTypeId="2" ParentId="4475" CreationDate="2018-06-13T12:42:05.560" Score="4" Body="&lt;p&gt;You need to install GCC, or a newer version of Clang.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your compiler — though the binary is called &lt;code&gt;gcc&lt;/code&gt; — is actually Clang, and the Clang version that’s shipping with macOS/Xcode still does not support OpenMP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest course of action is probably to install GCC via Homebrew (&lt;code&gt;brew install gcc&lt;/code&gt;) and then configure Sniffles to use Homebrew’s GCC (i.e. setting the &lt;code&gt;CMAKE_C*_COMPILER&lt;/code&gt; paths to &lt;code&gt;/usr/local/bin/g{cc,++}-$version&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2018-06-13T12:42:05.560" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4477" PostTypeId="2" ParentId="4470" CreationDate="2018-06-13T15:33:54.073" Score="2" Body="&lt;p&gt;Since I don't think it is possible to have two color schemes in a single plot (one for expression and one for highlighting some cells) I will suggest other approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A possible option to do this could be to &lt;strong&gt;change the shape&lt;/strong&gt; in the plot of the cells that you want to highlight and maintain the color code for expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I add a code that works for this purpose with the &lt;a href=&quot;https://drive.google.com/file/d/1rLsRXrxJPYm8cSzSqa5WUR6NYaUr-e9f/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;R object provided&lt;/a&gt;. Maybe there are easier ways to do that but this is working: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Run PCA&#xA;seuset_h16 &amp;lt;- RunPCA(seuset_h16, &#xA;                     pc.genes = seuset_h16@var.genes)&#xA;# Run tSNE&#xA;seuset_h16 &amp;lt;- RunTSNE(object = seuset_h16, &#xA;                      dims.use = 1:10, &#xA;                      do.fast = TRUE)&#xA;# Find clusters of cells&#xA;seuset_h16 &amp;lt;- FindClusters(object = seuset_h16, &#xA;                           reduction.type = &quot;pca&quot;, &#xA;                           dims.use = 1:10, &#xA;                           resolution = 0.6, &#xA;                           print.output = 0, &#xA;                           save.SNN = TRUE)&#xA;# Store output from FeaturePlot (do.return = T)&#xA;fp &amp;lt;- FeaturePlot(object = seuset_h16, &#xA;                  features.plot = c(&quot;DDB_G0273311&quot;), &#xA;                  cols.use = c(&quot;grey&quot;, &quot;orange&quot;), &#xA;                  reduction.use = &quot;tsne&quot;, &#xA;                  do.return = T)&#xA;# Add extra column in the data with the cells to be highligthed&#xA;fp$DDB_G0273311$data$cellToDisplay &amp;lt;- as.factor(ifelse(row.names(fp$DDB_G0273311$data) %in% c(&quot;s1.1&quot;, &quot;s1.4&quot;, &quot;s1.80&quot;), &#xA;&quot;group2&quot;, &#xA;&quot;group1&quot;))&#xA;# Modify color and shape in the plot&#xA;# lapply is needed since the object returned in fp is a list &#xA;lapply(fp, `+`, geom_point(aes(shape=cellToDisplay, colour = gene, size = 0.4)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the output, with three cells highlighted as triangles and the rest as dots. You can play a little bit with shapes and sizes if you want a clearer display: &#xA;&lt;a href=&quot;https://i.stack.imgur.com/TiyuG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TiyuG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="678" LastActivityDate="2018-06-13T15:33:54.073" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4478" PostTypeId="2" ParentId="4470" CreationDate="2018-06-13T17:02:58.897" Score="5" Body="&lt;p&gt;To color the TSNEPlot, you can generate a new column in metadata with the expression levels (High, low, etc). Then use pt.shape to set a shape for each identity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To show binary expression based on expression you first have to define the list of cells that are below or over your threshold. Once you have those lists you can use SetIdent() in Seurat to color those groups. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;load(file= &quot;~/seuset_16.RData&quot;)&#xA;&#xA;high &amp;lt;- c(&quot;s1.1&quot;,&quot;s1.4&quot;, &quot;s1.80&quot;)  &#xA;low &amp;lt;- setdiff(colnames(seuset_h16@data), high)  &#xA;seuset_h16 &amp;lt;- SetIdent(seuset_h16, cells.use = high, ident.use = &quot;high&quot;)  &#xA;seuset_h16 &amp;lt;- SetIdent(seuset_h16, cells.use = low, ident.use = &quot;low&quot;) &#xA;seuset_h16 &amp;lt;- StashIdent(seuset_h16, save.name = &quot;expr&quot;)  &#xA;seuset_h16 &amp;lt;- SetAllIdent(seuset_h16, id=&quot;res.1&quot;)  &#xA;t &amp;lt;- TSNEPlot(seuset_h16, pt.size = 2, do.return = T, pt.shape =&quot;expr&quot;)  &#xA;t&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/z9ELG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/z9ELG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;high &amp;lt;- c(&quot;s1.1&quot;,&quot;s1.4&quot;, &quot;s1.80&quot;)  &#xA;low &amp;lt;- setdiff(colnames(seuset_h16@data), high)  &#xA;seuset_h16 &amp;lt;- SetIdent(seuset_h16, cells.use = high, ident.use = &quot;high&quot;)  &#xA;seuset_h16 &amp;lt;- SetIdent(seuset_h16, cells.use = low, ident.use = &quot;low&quot;)   &#xA;t &amp;lt;- TSNEPlot(seuset_h16, pt.size = 2, do.return = T, colors.use = c(&quot;red&quot;, &quot;grey&quot;), do.hover =T, data.hover =c(&quot;ident&quot;, &quot;res.1))  &#xA;t&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/o1dmF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/o1dmF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2675" LastEditorUserId="3501" LastEditDate="2019-04-10T13:29:17.837" LastActivityDate="2019-04-10T13:29:17.837" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4480" PostTypeId="2" ParentId="4461" CreationDate="2018-06-13T21:05:30.070" Score="0" Body="&lt;p&gt;Your initial information/image and actual question seem pretty unrelated to me, but I'll do what I can.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a heterozygous SNV in a malignant population of cells, it really depends on when the mutation occurred for you to know the genotype of wildtype cells. It could have been there since birth (a &lt;strong&gt;germline&lt;/strong&gt; mutation passed down from the parents) or have been acquired throughout the person's life and clonally expanded (&lt;strong&gt;somatic&lt;/strong&gt; mutation). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, you have no way to differentiate between these two types of SNVs just by sequencing the tumor cells. If you want to know true &lt;strong&gt;somatic&lt;/strong&gt; mutations that are found only in the tumor cells, you need to sequence a matched normal control from the same patient (preferably from the same cell type). This is why most large cancer sequencing studies include not only tumor samples, but normal cells from the same patient - see &lt;a href=&quot;https://cancergenome.nih.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TCGA&lt;/a&gt; or &lt;a href=&quot;http://icgc.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ICGC&lt;/a&gt; for examples.&lt;/p&gt;&#xA;" OwnerUserId="1269" LastActivityDate="2018-06-13T21:05:30.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4481" PostTypeId="2" ParentId="4452" CreationDate="2018-06-13T21:50:08.790" Score="1" Body="&lt;p&gt;I don't have enough reputation to comment so I will briefly answer here&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This warning is fixed in Seurat 2.3.2. Without upgrading you can ignore this message. It doesn't mean anything for the actual analysis performed so your results won't be flawed.&lt;/p&gt;&#xA;" OwnerUserId="2675" LastActivityDate="2018-06-13T21:50:08.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4482" PostTypeId="2" ParentId="4473" CreationDate="2018-06-13T21:56:38.777" Score="0" Body="&lt;p&gt;It is pretty weird but the only thing that I needed to do is to rename my &lt;code&gt;python&lt;/code&gt; script from &lt;code&gt;umap.py&lt;/code&gt; to &lt;code&gt;umap_gen.py&lt;/code&gt; and right away it started to work. So, I guess there can be such name collisions which we should avoid. Although I would prefer the file to be named as &lt;code&gt;umap.py&lt;/code&gt; still...&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-06-13T21:56:38.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4483" PostTypeId="2" ParentId="4473" CreationDate="2018-06-14T08:02:10.787" Score="5" Body="&lt;p&gt;You answered your own question, yet it is not a coincidence that you have to change the name of your script.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When Python executes your custom script, it reads:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;import umap&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Python will first search the local sources (the directory of the script) , and afterwards the installed sources.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case Python identifies the local source to be &lt;code&gt;umap.py&lt;/code&gt;, and imports the (currently executed) script, while foregoing the installed sources. &#xA;Since you are calling functions, not defined in your current script, Python will not be able to fully execute your script.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For more information on importing see &lt;a href=&quot;https://docs.python.org/2/tutorial/modules.html#the-module-search-path&quot; rel=&quot;nofollow noreferrer&quot;&gt;Module search path&lt;/a&gt; at docs.pyhton.org&lt;/p&gt;&#xA;" OwnerUserId="2153" LastEditorUserId="2153" LastEditDate="2018-06-14T10:12:01.943" LastActivityDate="2018-06-14T10:12:01.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4484" PostTypeId="2" ParentId="4409" CreationDate="2018-06-14T14:01:37.717" Score="1" Body="&lt;p&gt;If you like to get pathways and corresponding genes as simple table format in a plain text file, use &lt;a href=&quot;https://www.biostars.org/p/108669/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GeneSCF&lt;/a&gt; 'prepare_database' module. This can be simple solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;FTP vs API&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The differences I see is,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;quick access to the updated content via FTP. With that I assume using &lt;a href=&quot;https://www.bioinformatics.jp/en/keggftp.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;FTP version&lt;/a&gt; you will have access to weekly database updates. Also bulk download is possible and more convenient via FTP subscription. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;But &lt;a href=&quot;https://www.kegg.jp/kegg/rest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;KEGG REST API&lt;/a&gt; updates once in month or two months or a year (not a recently updated contents).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1527" LastEditorUserId="1527" LastEditDate="2018-06-14T15:38:16.390" LastActivityDate="2018-06-14T15:38:16.390" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="4487" PostTypeId="1" CreationDate="2018-06-14T15:29:25.690" Score="3" ViewCount="146" Body="&lt;p&gt;If you're looking at the entry for a ligand, for example, &lt;a href=&quot;http://www.rcsb.org/ligand/ADN&quot; rel=&quot;nofollow noreferrer&quot;&gt;adenosine&lt;/a&gt;, there's a link on the webpage to entries for which this entry exists as a free ligand. &#xA;However, I want to access this via the API and can't seem to find docs/info on how to do that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been looking at the fields you can use to get &lt;a href=&quot;https://www.rcsb.org/pdb/results/reportField.do&quot; rel=&quot;nofollow noreferrer&quot;&gt;custom reports&lt;/a&gt;.  &lt;/p&gt;&#xA;" OwnerUserId="2945" LastEditorUserId="48" LastEditDate="2018-06-15T07:29:38.520" LastActivityDate="2018-06-15T07:29:38.520" Title="Can you query the RCSB PDB API with a ligand and return the IDs of all entries in which it appears as a free ligand?" Tags="|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4488" PostTypeId="2" ParentId="4487" CreationDate="2018-06-14T16:30:33.723" Score="2" Body="&lt;p&gt;I'm not sure about the RCSB, but you can get such a list from the PDBe REST API, as documented in the &lt;a href=&quot;http://www.ebi.ac.uk/pdbe/api/doc/compounds.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;compounds section&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ebi.ac.uk/pdbe/api/pdb/compound/in_pdb/ADN&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ebi.ac.uk/pdbe/api/pdb/compound/in_pdb/ADN&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, from an older service:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ebi.ac.uk/pdbe-srv/pdbechem/PDBEntry/download/ADN&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ebi.ac.uk/pdbe-srv/pdbechem/PDBEntry/download/ADN&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2018-06-14T16:30:33.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4489" PostTypeId="1" CreationDate="2018-06-14T17:51:53.507" Score="3" ViewCount="1171" Body="&lt;p&gt;Is there a method to determine if the peaks are broad or narrow? ENCODE provides some &lt;a href=&quot;https://www.encodeproject.org/chip-seq/histone/&quot; rel=&quot;nofollow noreferrer&quot;&gt;guidelines&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VanZO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VanZO.png&quot; alt=&quot;histone peak size&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although those cover common histone marks, there are many others. If you are using one of the ones that are not listed, is there a way to determine the appropriate peak width?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question was inspired by &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/4450/35&quot;&gt;this previous post&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2018-06-20T17:17:20.613" Title="determine if ChIP-seq peaks are broad or narrow" Tags="|chip-seq|peak-calling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4492" PostTypeId="2" ParentId="3958" CreationDate="2018-06-15T09:42:30.913" Score="3" Body="&lt;p&gt;I ended up using &lt;a href=&quot;http://mjsull.github.io/Easyfig/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Easyfig&lt;/a&gt;, which is made for the task and requires little tinkering around, once fed the genbank files.&#xA;(Granted, being so simple, it's hard to make the image more advanced, but it serves my purposes for now).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ARj7n.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ARj7n.gif&quot; alt=&quot;Easyfig example from the homepage.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6498615/bin/12864_2019_5692_MOESM4_ESM.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's an example of my image.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2572" LastEditorUserId="2572" LastEditDate="2019-06-20T12:36:32.657" LastActivityDate="2019-06-20T12:36:32.657" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4493" PostTypeId="1" CreationDate="2018-06-15T13:54:03.650" Score="1" ViewCount="50" Body="&lt;p&gt;I am trying to analyze a number of repetitive sequences and as one step want to calculate a sequence divergence between the elements I found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now in theory I wanted to generate a MSA of the sequences (~100bp times ~15k) and use the MEGA7 option to calculate pairwise and mean divergence. But sadly I run into the problem:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error# 4511 encountered: No common sites were found for the sequence&#xA;  pair (193,4).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and my MSA I generated with MAFFT looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;h1&#xA;-------gat-tt------------tgta--t---c-t------a---ta---t--t-a-&#xA;g-ca--tt-c-tt-t--at----------c----------a--------------ttt--&#xA;--------a---t-aata----------c---------c-------------------t-&#xA;------ag-a--a-a--------------------t-------------g----------&#xA;----t------tt-t-a-a---aa------------------------------------&#xA;------t--g--------c-----a-a-t-----t-----t-------------------&#xA;-------------a-------a--a--c---c-----a---t-t------------tt--&#xA;------------------t---g---g---------t------------------t-t--&#xA;---t-----g-a--tc--------------------at-------a--gt-ag----gc-&#xA;---t------a---t--t------t-t---c--a--a----a---a--aaatatc---&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;basically my sequence normally is 95% conserved just single individuals lead to the accumulation of those gaps, I am not sure how I can reduce these.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I calculate the sequence divergence? Any other programs? Any other methods?&lt;/p&gt;&#xA;" OwnerUserId="2952" LastEditorUserId="48" LastEditDate="2018-06-17T20:02:51.343" LastActivityDate="2018-06-17T20:02:51.343" Title="How can I determine a mean sequence divergence for 10k sequences?" Tags="|sequence-alignment|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4494" PostTypeId="1" AcceptedAnswerId="4512" CreationDate="2018-06-15T13:57:13.960" Score="2" ViewCount="214" Body="&lt;p&gt;I'm attempting to analyze some genomic data with the help of a package called CNSeg. The package &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/CNTools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;is located here&lt;/a&gt; and &lt;a href=&quot;https://confluence.broadinstitute.org/download/attachments/29790363/CNTools_How_To.pdf?version=1&amp;amp;modificationDate=1312551505000&quot; rel=&quot;nofollow noreferrer&quot;&gt;the documentation is here.&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My input data looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; str(segments)&#xA;'data.frame':   11897 obs. of  7 variables:&#xA; $ X                : int  0 1 2 3 4 5 6 7 8 9 ...&#xA; $ SegmentID        : int  72 73 74 75 76 77 78 79 80 81 ...&#xA; $ Chromosome       : int  1 1 2 2 2 3 3 3 3 3 ...&#xA; $ StartPosition    : int  754192 145260908 21494 141215321 141275624 63411 69812903 69884262 126473310 126790130 ...&#xA; $ StopPosition     : int  145258178 249212878 141214996 141275051 243052331 69811900 69884106 126457276 126772699 197852564 ...&#xA; $ Median.Log2.Ratio: num  -0.014 0.311 -0.003 0.059 -0.012 -0.018 -0.106 0.007 -0.171 0.001 ...&#xA; $ FileName         : Factor w/ 95 levels &quot;TSB02183&quot;,&quot;TSB02184&quot;,..: 1 1 1 1 1 1 1 1 1 1 ..&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Note: I did import the segments data with FileName as a vector instead of a factor as well. No difference in the error below).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I followed the vignette to the letter and, at the end of the vignette, the creator tells me that I can use the dist() method. When I try to use it, I get an error: &lt;strong&gt;Error in as.vector(data) : &#xA;  no method for coercing this S4 class to a vector&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my session information:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; segments &amp;lt;- read.csv(&quot;Probe_Segments_CN.csv&quot;)&#xA;&amp;gt; cnseg &amp;lt;- CNSeg(segList = segments, chromosome = &quot;Chromosome&quot;, end = &quot;StopPosition&quot;, start = &quot;StartPosition&quot;, segMean = &quot;Median.Log2.Ratio&quot;, id = &quot;FileName&quot;)&#xA;&amp;gt; rdseg &amp;lt;- getRS(cnseg, by = &quot;region&quot;, imput = FALSE, XY = FALSE, what = &quot;mean&quot;)&#xA;Processing samples ... Done&#xA;&amp;gt; data(&quot;geneInfo&quot;)&#xA;&amp;gt; geneInfo &amp;lt;- geneInfo[sample(1:nrow(geneInfo), 2000), ]&#xA;&amp;gt; &#xA;&amp;gt; rdByGene &amp;lt;- getRS(cnseg, by = &quot;gene&quot;, imput = FALSE, XY = FALSE, geneMap = geneInfo, what = &quot;median&quot;)&#xA;&amp;gt; &#xA;&amp;gt; reducedseg &amp;lt;- rs(rdseg)&#xA;&amp;gt; f1 &amp;lt;- kOverA(5, 1)&#xA;&amp;gt; &#xA;&amp;gt; ffun &amp;lt;- filterfun(f1)&#xA;&amp;gt; &#xA;&amp;gt; filteredrs &amp;lt;- genefilter(rdseg, ffun)&#xA;&amp;gt; filteredrs &amp;lt;- madFilter(rdseg, 0.8)&#xA;&amp;gt; dist(filteredrs)&#xA;Error in as.vector(data) : &#xA;  no method for coercing this S4 class to a vector&#xA;&amp;gt; sessionInfo()&#xA;R version 3.4.0 (2017-04-21)&#xA;Platform: x86_64-w64-mingw32/x64 (64-bit)&#xA;Running under: Windows &amp;gt;= 8 x64 (build 9200)&#xA;&#xA;Matrix products: default&#xA;&#xA;locale:&#xA;[1] LC_COLLATE=English_United States.1252 &#xA;[2] LC_CTYPE=English_United States.1252   &#xA;[3] LC_MONETARY=English_United States.1252&#xA;[4] LC_NUMERIC=C                          &#xA;[5] LC_TIME=English_United States.1252    &#xA;&#xA;attached base packages:&#xA;[1] tools     stats     graphics  grDevices&#xA;[5] utils     datasets  methods   base     &#xA;&#xA;other attached packages:&#xA;[1] CNTools_1.34.0    genefilter_1.60.0&#xA;&#xA;loaded via a namespace (and not attached):&#xA; [1] Rcpp_0.12.17         AnnotationDbi_1.40.0&#xA; [3] BiocGenerics_0.24.0  splines_3.4.0       &#xA; [5] IRanges_2.12.0       bit_1.1-14          &#xA; [7] lattice_0.20-35      xtable_1.8-2        &#xA; [9] blob_1.1.1           parallel_3.4.0      &#xA;[11] grid_3.4.0           Biobase_2.38.0      &#xA;[13] DBI_1.0.0            survival_2.41-3     &#xA;[15] bit64_0.9-7          digest_0.6.15       &#xA;[17] Matrix_1.2-9         S4Vectors_0.16.0    &#xA;[19] bitops_1.0-6         RCurl_1.95-4.10     &#xA;[21] memoise_1.1.0        RSQLite_2.1.1       &#xA;[23] compiler_3.4.0       stats4_3.4.0        &#xA;[25] XML_3.98-1.11        annotate_1.56.2   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1226" LastEditorUserId="1226" LastEditDate="2018-06-15T14:17:55.623" LastActivityDate="2018-06-20T07:09:47.733" Title="CNSeg no method coercing S4 class to vector" Tags="|r|bioconductor|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4495" PostTypeId="1" CreationDate="2018-06-15T14:14:53.660" Score="4" ViewCount="98" Body="&lt;p&gt;I'm a statistician, and am interested in applying the theory of algebraic Markov models to genomics. Here's one paper I'm interested in: &lt;a href=&quot;https://www.nature.com/articles/ng.943&quot; rel=&quot;nofollow noreferrer&quot;&gt;Large-scale genome-wide association analysis of bipolar disorder identifies a new susceptibility locus near ODZ4&lt;/a&gt;. I'm also interested in other diseases (i.e. not just bipolar). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any genomics databases that I can access publicly? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some links in &lt;a href=&quot;http://www.humanconnectomeproject.org/data/hcp-project/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Human Connectome Project&lt;/a&gt;, but I'm not sure if these are genomics databases (or which papers have used their research).&lt;/p&gt;&#xA;" OwnerUserId="2953" LastEditorUserId="48" LastEditDate="2018-06-15T14:59:53.567" LastActivityDate="2018-06-22T11:12:16.337" Title="Public access to genomics databases" Tags="|database|human-genome|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4496" PostTypeId="2" ParentId="4495" CreationDate="2018-06-15T14:22:01.903" Score="3" Body="&lt;p&gt;Most genomics data that is published should be available in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/summary/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GEO&lt;/a&gt; (or SRA). Usually in the paper they mention the accession number. However, not every study ends up in there, and sometime studies that are in GEO are not reproducible (poor description of the samples, or design). But if you look at the original papers, most of the times you'll find an accession number.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2018-06-15T14:22:01.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4497" PostTypeId="2" ParentId="4494" CreationDate="2018-06-15T14:56:43.087" Score="2" Body="&lt;p&gt;If you are using the latest version of Bioconductor you should always look at the latest &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/CNTools/inst/doc/HowTo.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; of the package in Bioconductor. There could be changes from 2011 to 2018 (In this case there aren't any changes in the section you mention).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is the package doesn't define how &lt;code&gt;dist&lt;/code&gt; should behave with that object class (filteredrs is a RS object if I read the source code well). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can extract the metrics to calculate the distance yourself or ask the maintainer to add this method to the class. However it could be that the maintainer answer you if you post the question in the &lt;a href=&quot;http://support.bioconductor.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;support site&lt;/a&gt; of Bioconductor. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-06-15T14:56:43.087" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4498" PostTypeId="1" AcceptedAnswerId="4501" CreationDate="2018-06-15T15:13:50.510" Score="1" ViewCount="2295" Body="&lt;p&gt;I have plotted the log normalized expression of two genes by violonplot for 4 clusters. I have links to my pictures and Seurat object too. My problem is this; in violin plot I can not see the mean or any centennial tendencies so that I don't know if two genes is expressing higher or lower in contrast to each other in each cluster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;violonplot vlnPlot(seuset,c(&quot;&quot;))&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually when I am thinking I see I need a violin or box plot showing  the  ratio of these genes to each other in each cluster and how nice to have mean also in the plot &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Somebody knows how I can do that?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2018-06-17T15:12:29.813" LastActivityDate="2018-06-17T15:12:29.813" Title="Violinplot of gene expression" Tags="|r|single-cell|seurat|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4500" PostTypeId="1" AcceptedAnswerId="4502" CreationDate="2018-06-16T10:53:36.803" Score="5" ViewCount="1538" Body="&lt;p&gt;I have two datasets (DataA and DataB) and I want to find the Spearman correlation between genes and also pull out the gene names (stored in first column of dataset) in &lt;code&gt;R&lt;/code&gt;. I am using &lt;code&gt;fread&lt;/code&gt; from &lt;code&gt;read.table&lt;/code&gt; to read the file and &lt;code&gt;cor.test&lt;/code&gt; to find &lt;code&gt;Rho&lt;/code&gt; and &lt;code&gt;p-value&lt;/code&gt;.&#xA;The code works fine for a smaller data set of 10 obs, 10 variables. But, my actual data set is larger (DataA - 795 obj, 542 variables and DataB - 925 obs, 542 variables). &#xA;When I try to run the code on my actual data, it is consuming a large amount of time (it has been 23 hrs since my code is running). Is there a way to optimize this code? Or is there a bug in the code?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(data.table)&#xA;A.data &amp;lt;- fread(&quot;DataA.csv&quot;, header = TRUE, data.table = FALSE, stringsAsFactors = FALSE)&#xA;B.data &amp;lt;- fread(&quot;DataB.csv&quot;, header = TRUE, data.table = FALSE, stringsAsFactors = FALSE)&#xA;&#xA;correlation.result &amp;lt;- data.frame()&#xA;&#xA;## Computing Correlation&#xA;&#xA;for (i in 1:nrow(A.data)) {&#xA;  for (j in 1:nrow(B.data)) {&#xA;       correln &amp;lt;- cor.test(as.numeric(A.data[i,2:ncol(A.data)]), as.numeric(B.data[j,2:ncol(B.data)]), method = &quot;spearman&quot;, exact = FALSE)&#xA;       rho &amp;lt;- correln$estimate&#xA;       p.val &amp;lt;- correln$p.value&#xA;       Gene &amp;lt;- paste(A.data[i,1], B.data[j,1], sep=&quot;_&quot;, collapse=&quot; &quot;) # To pullout gene name pairwise&#xA;       comb.data &amp;lt;- data.frame(rho, p.val, Gene)&#xA;       correlation.result &amp;lt;- rbind(correlation.result,comb.data)&#xA; } &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2247" LastEditorUserId="2247" LastEditDate="2018-06-18T13:03:12.803" LastActivityDate="2018-06-18T19:49:19.950" Title="Spearman correlation for large dataset" Tags="|r|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4501" PostTypeId="2" ParentId="4498" CreationDate="2018-06-16T15:08:47.667" Score="4" Body="&lt;p&gt;You want to (1) see the mean for each gene, and also to (2) calculate a ratio of expression levels of two genes, then compare it between clusters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) First, notice that &lt;code&gt;vlnPlot()&lt;/code&gt; is deprecated. Use &lt;code&gt;VlnPlot()&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can try using the parameter &lt;code&gt;do.sort=T&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;VlnPlot(object=seuset, features.plot=c(&quot;DDB_G0267412&quot;, &quot;DDB_G0277853&quot;), do.sort=T)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you can return the ggplot2 object and then plot the means etc:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p &amp;lt;- VlnPlot(object=seuset, features.plot=&quot;DDB_G0277853&quot;, do.return=T)&#xA;p &amp;lt;- p + geom_boxplot(width=0.05)&#xA;p&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;(2) There are a few problems with calculating the ratio of gene expression levels. Firstly, what do you mean by gene expression level and how do you measure it? Surely you can have the read counts, but how do you interpret them? Is 30 counts a high level? Is it much more than 60 counts, or is it roughly the same? The same applies to the calculated ratios and the differences between them, even if we ignore amplification, gene length and other biases. And it is very hard to interpret ratios if the reference can also change. The problem is somewhat similar to RT-qPCR, where people use a set of reference genes whose expression has previously been shown to be invariant under the conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In theory, you could use the raw counts (object@raw.data), the log + normalized counts (object@data), or the scaled counts (object@scale.data). The raw counts are biased by sequencing-depth, and the ratio of log or scaled values are not easily interpretable or intuitive. Therefore the library-size normalized (non-log) values seem to be the best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, you could calculate the ratio of two genes either (a) for each cell (paired), or (b) for each group.&#xA;(a) is problematic, because of the zero values: you will have many NaN and Inf values, which cannot be removed without biasing the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some functions for retrieving and plotting data from the object:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ident1 &amp;lt;- WhichCells(seuset, ident=1)&#xA;ident2 &amp;lt;- WhichCells(seuset, ident=2)&#xA;&#xA;FetchData(seuset, vars.all='DDB_G0267412', cells.use=ident1, use.raw=T)&#xA;&#xA;datatoplot1 &amp;lt;- seuset@raw.data['DDB_G0267412', ident1] / seuset@raw.data['DDB_G0277853', ident1]&#xA;datatoplot2 &amp;lt;- seuset@raw.data['DDB_G0267412', ident2] / seuset@raw.data['DDB_G0277853', ident2]&#xA;&#xA;boxplot(datatoplot1, datatoplot2)&#xA;&#xA;stripchart(datatoplot1, &#xA;            vertical = TRUE, method = &quot;jitter&quot;, &#xA;            pch = 21, col = &quot;dark green&quot;, &#xA;            add = TRUE, at=1) &#xA;&#xA;stripchart(datatoplot2, &#xA;            vertical = TRUE, method = &quot;jitter&quot;, &#xA;            pch = 21, col = &quot;dark green&quot;, &#xA;            add = TRUE, at=2) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="208" LastActivityDate="2018-06-16T15:08:47.667" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4502" PostTypeId="2" ParentId="4500" CreationDate="2018-06-16T19:48:58.853" Score="3" Body="&lt;p&gt;Your code looks fine, but &lt;code&gt;rbind&lt;/code&gt; to progressively build and resize a data frame can be gluttonous in terms of memory and time, especially given the A-row x B-row times you are doing this operation, which is fine for very small datasets, but not often realistic for real-world data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, perhaps &lt;code&gt;cat&lt;/code&gt; results to a text file, where results are written in tab-delimited columns. You can then read in that text file later on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to serializing or saving your results to text, which make them easier to work with in non-R contexts, you won't have to deal with R's memory allocation problems when running &lt;code&gt;rbind&lt;/code&gt; repeatedly. You just write out the answer to a file and read it in later on, in one pass.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another approach is to preallocate memory to a presized data frame, and then write to that allocated space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You know the size of your inputs ahead of time, which makes this approach tenable. In this case, it looks like you are storing two floats and a string (&lt;code&gt;rho&lt;/code&gt;, &lt;code&gt;p.val&lt;/code&gt;, and &lt;code&gt;Gene&lt;/code&gt;) and that you are going to store A-row x B-row number of them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's one way you might go about this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# create a presized data frame&#xA;Arows &amp;lt;- nrow(A.data)&#xA;Brows &amp;lt;- nrow(B.data)&#xA;correlation.result &amp;lt;- data.frame(t(matrix(c(0.0, 0.0, &quot;foo&quot;), nrow=3, ncol=Arows * Brows)), stringsAsFactors=FALSE)&#xA;colnames(correlation.result) &amp;lt;- c(&quot;rho&quot;, &quot;p.val&quot;, &quot;Gene&quot;)&#xA;&#xA;# assign values to that presized data frame &#xA;Acols &amp;lt;- ncol(A.data)&#xA;Bcols &amp;lt;- ncol(B.data)&#xA;for (i in 1:Arows) {&#xA;  # get A data&#xA;  Ai &amp;lt;- as.numeric(A.data[i, 2:Acols])&#xA;  Aname &amp;lt;- A.data[i, 1]&#xA;  for (j in 1:Brows) {&#xA;    # calculate row offset in presized data frame&#xA;    ij &amp;lt;- (i - 1) * Brows + j&#xA;    # get B data&#xA;    Bj &amp;lt;- as.numeric(B.data[j, 2:Bcols])&#xA;    Bname &amp;lt;- B.data[j, 1]&#xA;    # run test&#xA;    correln &amp;lt;- cor.test(Ai, Bj, method = &quot;spearman&quot;, exact = FALSE)&#xA;    rho &amp;lt;- correln$estimate&#xA;    p.val &amp;lt;- correln$p.value&#xA;    Gene &amp;lt;- paste(Aname, Bname, sep=&quot;_&quot;, collapse=&quot; &quot;)&#xA;    # assign value to row of result data frame&#xA;    correlation.result[ij,] &amp;lt;- c(rho, p.val, Gene)&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This avoids R needing to resize and recopy the data frame contents as they grow, which I suspect is the major reason it is taking so long to run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chapter or &quot;Circle&quot; 2 of Patrick Burns' book &lt;em&gt;The R Inferno&lt;/em&gt; discusses the problems with growing objects with &lt;code&gt;rbind&lt;/code&gt; and ways to handle it — you might find this a useful read. This book is a PDF download via your search engine of choice.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-06-18T19:49:19.950" LastActivityDate="2018-06-18T19:49:19.950" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4503" PostTypeId="1" CreationDate="2018-06-16T22:50:13.553" Score="2" ViewCount="48" Body="&lt;p&gt;Imagine I have a dataset of highly periodic gene expression, taken at densely spaced timepoints. I know that the expression is periodic, but different genes have different offsets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is trivial to convert my timestamps into a cosine (&lt;code&gt;cos_time = cos(offset + time*pi*2/PERIOD)&lt;/code&gt;), so that this new variable is linear with the genes' expressions -- but only with the ones that have a peak at &lt;code&gt;offset&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What if I could have multiple &lt;code&gt;cos_time&lt;/code&gt;s (let's call them &lt;code&gt;phases&lt;/code&gt;) and then ask DESeq2: &quot;OK, which gene's behavior is most linear with which phase?&quot; Barring comparing p-values of each LRT test, is there a better way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: &lt;em&gt;I know about cosinor, Lomb-Scargle periodograms, ARS, JTK etc, and I know they are likely more applicable in this particular case. I'm interested in whether it is in general possible to ask such a question from DESeq2, even if the variables in question are not periodic. Phase just seemed like the simplest concept to demonstrate this on.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="132" LastActivityDate="2018-06-17T14:07:27.573" Title="Can I ask DESeq2 which variable alone explains which gene's behavior the best?" Tags="|r|rna-seq|differential-expression|deseq2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4504" PostTypeId="2" ParentId="4503" CreationDate="2018-06-17T03:43:13.790" Score="1" Body="&lt;p&gt;If I were to do this, my first guess would be to add in the phases as a separate factor in the statistical model, something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;~ Condition + sex + phase1 + phase2 + phase3 + ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-06-17T03:43:13.790" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4505" PostTypeId="2" ParentId="4500" CreationDate="2018-06-17T12:35:37.597" Score="1" Body="&lt;p&gt;You can use &lt;code&gt;corr.test()&lt;/code&gt; from psych &lt;a href=&quot;https://www.rdocumentation.org/packages/psych/versions/1.8.4/topics/corr.test&quot; rel=&quot;nofollow noreferrer&quot;&gt;package&lt;/a&gt;, e.g. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ct &amp;lt;- corr.test(t(A.data), t(B.data), method='spearman', adjust='none')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then &lt;code&gt;ct$p&lt;/code&gt; and &lt;code&gt;ct$r&lt;/code&gt; would give you (uncorrected) p-value and correlation matrices of size &lt;code&gt;795 x 925&lt;/code&gt;. Note that &lt;code&gt;t()&lt;/code&gt; is required to compare matrices row-wise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That would be orders of magnitude faster than loop-based approach.&lt;/p&gt;&#xA;" OwnerUserId="2920" LastActivityDate="2018-06-17T12:35:37.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4506" PostTypeId="2" ParentId="4503" CreationDate="2018-06-17T14:07:27.573" Score="2" Body="&lt;p&gt;If you have densely spaced time points the using non-linear regression to directly fit the signal to a cosine or sine curve, which would directly produce the offset, would be much simpler. DESeq2 wouldn't be the appropriate tool then, you'd use base R and the &lt;code&gt;nls()&lt;/code&gt; function.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-06-17T14:07:27.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4507" PostTypeId="1" AcceptedAnswerId="4508" CreationDate="2018-06-17T21:27:46.310" Score="14" ViewCount="6789" Body="&lt;p&gt;&lt;code&gt;Bowtie2&lt;/code&gt; is probably the most widely used aligner because of it's speed. Burrow-wheeler (BW) algorithms (including &lt;code&gt;bwa&lt;/code&gt;) tend to be faster. However, they have limitations when it comes to aligning very short reads (e.g. gRNA). Also, setting maximum number of mismatches allowed is complicated by the seed length, overlaps and other parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder if there is any better multi-purpose aligner out there. May be with algorithm other that BW. One which allows special cases e.g. allowing shord reads and high number of mismatches. &lt;/p&gt;&#xA;" OwnerUserId="274" LastActivityDate="2020-09-11T06:01:56.547" Title="Better aligner than bowtie2?" Tags="|sequence-alignment|sequence-analysis|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4508" PostTypeId="2" ParentId="4507" CreationDate="2018-06-17T23:55:32.563" Score="16" Body="&lt;p&gt;Bowtie2 is no longer the fastest aligner. &lt;a href=&quot;http://salmon.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Salmon&lt;/a&gt; and &lt;a href=&quot;https://pachterlab.github.io/kallisto/download&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kallisto&lt;/a&gt; are much faster, but have been designed to optimise RNASeq mapping. Their speed is gained from avoiding a strict base-to-base alignment, but they can output mostly-aligned reads (i.e. position-only, without local alignment) as pseudo-alignments. See &lt;a href=&quot;https://pachterlab.github.io/kallisto/pseudobam.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for more details.&lt;/p&gt;&#xA;&lt;p&gt;Both Kallisto and Salmon can do additional bootstrapping that is interpreted by &lt;em&gt;sleuth&lt;/em&gt; for improved performance in isoform detection. They can also output counts that are equivalent to read-level counts from other programs, which can then be used by other downstream gene-based differential expression analysis software (e.g. &lt;a href=&quot;http://dx.doi.org/10.12688/f1000research.7563.2&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2&lt;/a&gt;). Salmon has additional options that can  correct mappings for sequence-level and GC bias.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://ccb.jhu.edu/software/hisat2/index.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;HISAT2&lt;/a&gt; is from the same group as Bowtie2, and does the same sort of stuff, but with a few optimisations added on top. In particular, it's much better at working out split reads from RNASeq runs. Like Bowtie2, it will do local alignment of reads.&lt;/p&gt;&#xA;&lt;p&gt;For quick alignment of long reads, &lt;a href=&quot;https://github.com/lh3/minimap2#map-long-genomic&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimap2&lt;/a&gt; works well. For high-accuracy alignment (but comparatively slower), &lt;a href=&quot;https://github.com/mcfrith/last-rna/blob/master/last-long-reads.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;LAST&lt;/a&gt; works well.&lt;/p&gt;&#xA;&lt;p&gt;Most bioinformaticians seem to prefer &lt;a href=&quot;https://github.com/alexdobin/STAR/&quot; rel=&quot;nofollow noreferrer&quot;&gt;STAR&lt;/a&gt; for things that Bowtie2 was previously used for. I'm not yet convinced it's a better alternative, and currently prefer HISAT2 for high accuracy short-read alignment.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-09-11T06:01:56.547" LastActivityDate="2020-09-11T06:01:56.547" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="4509" PostTypeId="1" AcceptedAnswerId="4567" CreationDate="2018-06-18T06:44:23.677" Score="3" ViewCount="1204" Body="&lt;p&gt;I am trying to estimate a phylogenetic tree in R with &lt;a href=&quot;https://cran.r-project.org/package=ape&quot; rel=&quot;nofollow noreferrer&quot;&gt;ape&lt;/a&gt; and &lt;a href=&quot;https://cran.r-project.org/package=phangorn&quot; rel=&quot;nofollow noreferrer&quot;&gt;phangorn&lt;/a&gt;. With &lt;code&gt;pratchet()&lt;/code&gt; and &lt;code&gt;optim.parsimony()&lt;/code&gt;, I can estimate a phylogenetic tree, but branch lengths are not estimated. Is there a way to estimate a phylogeny with branch lengths with the following made-up data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x &amp;lt;- data.frame(&quot;L1&quot; = c(0,0,1,0,0,0,1,1,1,1,0,0,0,0,1), &quot;L2&quot; = c(0,1,1,0,1,1,0,1,1,0,1,1,0,1,1),&quot;L3&quot; = c(1,1,1,0,0,0,1,1,1,0,0,0,1,1,1), &quot;L4&quot;=c(0,0,1,1,0,0,0,0,1,1,0,0,0,0,1)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2961" LastEditorUserId="400" LastEditDate="2018-06-22T15:58:55.717" LastActivityDate="2018-06-26T10:14:29.980" Title="How do I estimate branch length of a maximum parsimony phylogeny in R?" Tags="|r|phylogenetics|phylogeny|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4510" PostTypeId="1" CreationDate="2018-06-18T12:46:48.400" Score="4" ViewCount="2951" Body="&lt;p&gt;I'm interested in looking into cells that are positive for two (or in some cases more) genes. I know I have some double positive just by looking at the FeaturePlot of those genes, but now I'm trying to figure it out how many are double positive and further percentage wise. Example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;50 cells are positive for Myh11&lt;br&gt;&#xA;100 cells are positive for Acta2&lt;br&gt;&#xA;30 cells are double positive&lt;br&gt;&#xA;60% of Myh11+ cells are Acta2+&lt;br&gt;&#xA;30% of Acta2+ cells are Myh11+  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is the data used to generated the FeaturePlot the scale.data? If so, I could just make a new dataset out of that, but I'm not sure if that is the case.&lt;/p&gt;&#xA;" OwnerUserId="2964" LastEditorUserId="2719" LastEditDate="2018-06-19T16:37:39.773" LastActivityDate="2018-06-20T10:02:37.080" Title="Is it possible to identify cells that are expressing two or more genes in Seurat?" Tags="|r|scrnaseq|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4511" PostTypeId="1" AcceptedAnswerId="4514" CreationDate="2018-06-18T15:28:05.500" Score="3" ViewCount="195" Body="&lt;p&gt;I'm trying to download around 2000 proteomes from NCBI, but all I have is the genus and species e.g: &lt;code&gt;lophodermium_seditiosum trichoderma_harzianum&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know a good way of doing this in script form?&lt;br&gt;&#xA;My code at the moment is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez&#xA;from Bio import SeqIO&#xA;import os&#xA;&#xA;file_name = &quot;proteome1.faa&quot;&#xA;net_handle = Entrez.efetch(db=&quot;protein&quot;, id=&quot;trichoderma_harzianum&quot;, rettype=&quot;gp&quot;, retmode=&quot;text&quot;)&#xA;out_handle = open(file_name, &quot;w&quot;)&#xA;out_handle.write(net_handle.read())&#xA;out_handle.close()&#xA;net_handle.close()&#xA;print(&quot;Saved&quot;)&#xA;&#xA;&#xA;for seq_record in SeqIO.parse(&quot;proteome1.faa&quot;, &quot;genbank&quot;):&#xA;    print(seq_record.id)&#xA;    print(repr(seq_record.seq))&#xA;    print(len(seq_record))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I think it wants a number as the id, I don't know if it's possible to convert species name to a numerical ID within the script, or if that's the best way of going about this.&lt;/p&gt;&#xA;" OwnerUserId="2965" LastEditorUserId="48" LastEditDate="2018-06-18T16:17:44.800" LastActivityDate="2021-10-27T11:04:17.727" Title="Download proteomes from NCBI based only on binomial names" Tags="|proteins|python|phylogenetics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4512" PostTypeId="2" ParentId="4494" CreationDate="2018-06-18T16:20:31.053" Score="1" Body="&lt;p&gt;I was not able to get the CNSeg clustering method to work, but the work-around was to export the data and cluster it independently of the CNTools package (thanks to Llopis's answer below). I'll post the analysis below. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;require(CNTools)&#xA;segData &amp;lt;- read.csv(&quot;result_cnv.csv&quot;, stringsAsFactors = FALSE)&#xA;head(segData)&#xA;&#xA;# Create inital CN object&#xA;cnseg &amp;lt;- CNSeg(segList = segData, chromosome = &quot;Chromosome&quot;, end = &quot;StopPosition&quot;, start = &quot;StartPosition&quot;, segMean = &quot;Median.Log2.Ratio&quot;, id = &quot;FileName&quot;)&#xA;cnseg&#xA;&#xA;# Create inital RD object&#xA;rdseg &amp;lt;- getRS(cnseg, by = &quot;region&quot;, imput = FALSE, XY = FALSE, what = &quot;mean&quot;)&#xA;rdseg&#xA;&#xA;# Collect gene information&#xA;data(&quot;geneInfo&quot;)&#xA;geneInfo &amp;lt;- geneInfo[sample(1:nrow(geneInfo), 2000), ]&#xA;&#xA;# Create an RD based on gene information&#xA;rdByGene &amp;lt;- getRS(cnseg, by = &quot;gene&quot;, imput = FALSE, XY = FALSE, geneMap = geneInfo, what = &quot;median&quot;)&#xA;&#xA;# Initalize reduced segment&#xA;reducedseg &amp;lt;- rs(rdseg)&#xA;&#xA;# Create a function that evaluates to TRUE if at least 5 of the argument elements are larger than 1&#xA;f1 &amp;lt;- kOverA(5, 1)&#xA;&#xA;# Create a filter based on f1&#xA;ffun &amp;lt;- filterfun(f1)&#xA;&#xA;# Use the CNTools genefilter &#xA;filteredrs &amp;lt;- genefilter(rdseg, ffun)&#xA;&#xA;# Use the CNTools madFilter&#xA;filteredrs &amp;lt;- madFilter(rdseg, 0.8)&#xA;&#xA;filteredrs&#xA;&#xA;# Write the filtered data to file&#xA;write.csv(attributes(filteredrs)[1], &quot;CNseg_CNOut.csv&quot;)&#xA;&#xA;# Reading the recently-written file helps with formatting&#xA;CNseg &amp;lt;- read.csv(&quot;CNseg_CNOut.csv&quot;)&#xA;&#xA;# Keep only the columns with actual sample values&#xA;CNseg &amp;lt;- CNseg[5:ncol(CNseg)]&#xA;&#xA;# Calculate eluclidian distance between samples&#xA;d &amp;lt;- dist(t(CNseg), method = &quot;euclidean&quot;)&#xA;&#xA;# Calculate clustering&#xA;hc1 &amp;lt;- hclust(d, method = &quot;ward.D&quot;)&#xA;plot(hc1, cex = 0.6, hang = -1, main = &quot;Clusters of Copy Number Alterations&quot;, xlab = &quot;Euclidean Distance&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1226" LastEditorUserId="48" LastEditDate="2018-06-20T07:09:47.733" LastActivityDate="2018-06-20T07:09:47.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4513" PostTypeId="1" CreationDate="2018-06-18T20:45:42.160" Score="1" ViewCount="57" Body="&lt;p&gt;I have a gene network that I clustered into 8 parts. I calculated the degree distribution of the clusters using Cytoscape and the slope of the clusters range from $0.7$ to about $1.3$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From literature &lt;a href=&quot;https://en.wikipedia.org/wiki/Scale-free_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;(reference)&lt;/a&gt; usual scale free networks (real world networks) have a slope range of $2&amp;gt;γ&amp;gt;3$. What could be a reason for my network to have the smaller slope value?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: The number of nodes of each cluster ranges from 431 to 187 and the number of edges range from 1009 to 263. The network is bipartite in nature.  The clusters are all very sparse which can be understood by the number of edges in the largest and smallest cluster. Are real world biological networks not always scale free. Can anyone point me towards publications that mention this fact. I am just confused about why my network doesn't conform to the norm but maybe it doesn't need to. &lt;/p&gt;&#xA;" OwnerUserId="976" LastEditorUserId="976" LastEditDate="2018-06-26T13:53:09.987" LastActivityDate="2018-06-26T13:53:09.987" Title="The slopes of gene networks" Tags="|networks|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="4514" PostTypeId="2" ParentId="4511" CreationDate="2018-06-19T08:40:47.010" Score="3" Body="&lt;p&gt;The general procedure is explained &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/guide/howto/retrieve-seq-org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;First step is to search the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;Taxonomy database&lt;/a&gt; with the organism name. Accepted common names usually work at all taxonomic levels. Use the scientific name or formal name if no results are obtained with the common name.&lt;/p&gt;&#xA;&lt;p&gt;In code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez&#xA;&#xA;Entrez.email = 'user@example.org' # Put your email here&#xA;&#xA;def get_tax_id(species):&#xA;    species = species.replace('_', '+').strip()&#xA;    search = Entrez.esearch(term=species, db='taxonomy', retmode='xml')&#xA;    record = Entrez.read(search)&#xA;    return record['IdList'][0]&#xA;&#xA;if __name__ == '__main__':&#xA;    organisms = ['lophodermium_seditiosum', 'trichoderma_harzianum']&#xA;    taxids = [get_tax_id(organism) for organism in organisms]&#xA;    print(taxids)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This gives us the &lt;code&gt;taxid&lt;/code&gt;s for our species, respectively &lt;code&gt;128023&lt;/code&gt; and &lt;code&gt;5544&lt;/code&gt;. Now we can use these to get the proteomes. For &lt;em&gt;Lophodermium seditiosum&lt;/em&gt; we find two results, but &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/protein/?term=txid5544%5BOrganism:exp%5D&quot; rel=&quot;nofollow noreferrer&quot;&gt;for &lt;em&gt;Trichoderma harzianum&lt;/em&gt; there are 65900 results&lt;/a&gt;, so you might want to tweak the query to filter the results. For example, if you only want the sequences appearing in PDB you add &lt;code&gt;AND pdb[filter]&lt;/code&gt; so that the query becomes &lt;code&gt;txid5544[Organism:exp] AND pdb[filter]&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def get_proteomes(taxid):&#xA;    query = 'txid{}[Organism:exp]'.format(taxid)&#xA;    handle = Entrez.esearch(db='protein', term=query)&#xA;    result = Entrez.read(handle)&#xA;    if int(result['Count']) &amp;lt;= 2000:&#xA;        ids = ','.join(result['IdList'])&#xA;        proteomes = Entrez.efetch(db='protein', id=ids, rettype='fasta').read()&#xA;        return proteomes&#xA;    else:&#xA;        raise RuntimeError('Too many results!')&#xA;&#xA;if __name__ == '__main__':&#xA;    print(get_proteomes('128023'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Returns:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;ADI44294.1 actin, partial [Lophodermium seditiosum]&#xA;CKAGFAGDDAPRAVFPSIVGRPRHHGIMIGMGQKDS&#xA;&#xA;&amp;gt;ADI44293.1 actin, partial [Lophodermium seditiosum]&#xA;CKAGFAGDDAPRAVFPSIVGRPRHHGIMIGMGQKDS&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="450" LastEditorUserId="450" LastEditDate="2021-10-27T07:55:20.457" LastActivityDate="2021-10-27T07:55:20.457" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4515" PostTypeId="1" AcceptedAnswerId="4516" CreationDate="2018-06-19T16:17:27.947" Score="0" ViewCount="97" Body="&lt;p&gt;I called &lt;code&gt;pratchet()&lt;/code&gt; on a dataset with binary values for the characters and it returned a parsimony value of 35. I then calculated the parsimony value for the dataset with a tree that I myself constructed and it was 7. How is this possible at all? I thought that &lt;code&gt;pratchet()&lt;/code&gt; was supposed to return the most parsimonious tree on the basis of the data set, but that clearly did not happen in this case. Is there other software that is guaranteed to find the most parsimonious tree?&lt;/p&gt;&#xA;" OwnerUserId="2961" LastActivityDate="2018-06-20T01:23:48.627" Title="Why does `pratchet()` in `phangorn` not find the most parsimonious trees?" Tags="|software-recommendation|phylogenetics|algorithms|phylogeny|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4516" PostTypeId="2" ParentId="4515" CreationDate="2018-06-19T16:47:54.807" Score="1" Body="&lt;p&gt;Perhaps the tree which you have calculated is not being caught in the search heuristic? Have you tried (as a trial) setting the tree which you calculated as the starting topology, just to check that it is indeed being calculated on by pratchet()?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you could try playing around with the search heuristic settings, like varying from ratchet to NNI, or you could also try using an external program, like PAUP*.&lt;/p&gt;&#xA;" OwnerUserId="2841" LastActivityDate="2018-06-19T16:47:54.807" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4517" PostTypeId="1" CreationDate="2018-06-19T17:42:22.713" Score="3" ViewCount="1780" Body="&lt;p&gt;If I subtract input counts from ChIP counts (for every gene, since I have one peak per gene) I get negative values for most genes. This makes sense to me, because (as can be seen in the figure) input and IP have same sequencing depth but IP is very biased towards few genes with lots of counts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZY8Fp.jpg&quot; alt=&quot;https://cdn.sstatic.net/Sites/stackoverflow/img/error-lolcat-problemz.jpg&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Elsevier was informed that the business model which they offered does not meet the demands. How should I implement input normalization?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. The subtraction idea is from &lt;a href=&quot;https://support.bioconductor.org/p/84539/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioconductor&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="2970" LastEditorUserId="48" LastEditDate="2018-06-21T15:54:55.227" LastActivityDate="2018-06-21T15:54:55.227" Title="Input normalization in ChIP-seq" Tags="|bioconductor|normalization|chip-seq|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4518" PostTypeId="2" ParentId="4509" CreationDate="2018-06-20T01:07:37.423" Score="1" Body="&lt;h1&gt;Terminology&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;First, a couple of corrections to your use of terminology. A tree &lt;em&gt;must&lt;/em&gt; have branch lengths associated with it to be called a &lt;strong&gt;phylogenetic tree&lt;/strong&gt; as it represents both the relationships between OTU/taxa/leaves &lt;em&gt;AND&lt;/em&gt; the distances between them (i.e. the branch lengths. A tree where branch lengths have no meaning is called a &lt;strong&gt;cladograms&lt;/strong&gt; or a &lt;strong&gt;dendrogram&lt;/strong&gt;. You can tell if a tree is a cladogram since all the OUTs/taxa/leaves end at the same position and internal branch lengths are usually (not always) of equal length. Whereas a phylogeny the OTUs/taxa/leaves will typically look &quot;staggered&quot; &lt;em&gt;and&lt;/em&gt; internal branches will be of different lengths.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, If the tree you have estimated, truly doesn't have branch lengths estimated and associated with it, it is actually a cladogram.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Example Derived from &lt;code&gt;phangorn&lt;/code&gt; Vignette&lt;/h1&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(phangorn)&#xA;&#xA;# Get example data&#xA;fdir &amp;lt;- system.file(&quot;extdata/trees&quot;, package = &quot;phangorn&quot;)&#xA;primates &amp;lt;- read.phyDat(file.path(fdir, &quot;primates.dna&quot;), format = &quot;phylip&quot;)&#xA;&#xA;# Calculate pairwise distances between OTUs using Maximum Likelihood&#xA;dm &amp;lt;- dist.ml(primates)&#xA;&#xA;# Use the distances to cluster the OTUs using UPGMA algorithm&#xA;# This creates a dendrogram&#xA;treeUPGMA &amp;lt;- upgma(dm)&#xA;plot(treeUPGMA, main=&quot;UPGMA&quot;)&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6SHVE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6SHVE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# Search for different, similar trees to the initial UPGMA tree&#xA;# looking for a tree with the lowest parsimony score&#xA;# There are 2 search algorithms implemented:&#xA;#   optim.parsimony()&#xA;#   pratchet()&#xA;# These are still cladograms&#xA;treePars &amp;lt;- optim.parsimony(treeUPGMA, primates)&#xA;treeRatchet &amp;lt;- pratchet(primates, trace = 0)&#xA;plot(treePars, main=&quot;optim.parsimony()&quot;)&#xA;plot(treeRatchet, main=&quot;pratchet()&quot;)&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2I82V.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2I82V.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/2fcvD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2fcvD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# Calculate branch lengths using the most parsimonious tree identified by&#xA;# the pratchet() searchh algorithm&#xA;phylogeny &amp;lt;- nnls.phylo(treeRatchet, dm)&#xA;plot(phylogeny, main=&quot;Phylogeny&quot;)&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/b8Qdl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/b8Qdl.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Choices, Choices, Choices&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;As you can see from the above example, there are lots of steps to calculating a phylogenetic tree. At each of these steps there may be multiple algorithms to choose from. You choice will depend on the type of data you are analysing, the size of the data set, your available compute resources and the algorithms implemented and available through the tools you use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, you will have to know a fair bit about these algorithms to make an informed choice.&lt;/p&gt;&#xA;" OwnerUserId="400" LastActivityDate="2018-06-20T01:07:37.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4519" PostTypeId="2" ParentId="4515" CreationDate="2018-06-20T01:23:48.627" Score="1" Body="&lt;p&gt;For dendrograms with less than about 10 OTUs/taxa/leaves it is possible to evaluate all possible dendrograms using an exhaustive search algorithm. However, for more than about 10 OTUs, heuristics need to be employed to try and &quot;home-in&quot; on and score a smaller set of &quot;better&quot; trees. However, these heuristics are not guaranteed to find &lt;strong&gt;&lt;em&gt;the&lt;/em&gt;&lt;/strong&gt; best dendrogram.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without some specific code it is hard to do more than speculate as to the problem you are encountering, especially where you get an error using data you manually create. &lt;/p&gt;&#xA;" OwnerUserId="400" LastActivityDate="2018-06-20T01:23:48.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4520" PostTypeId="2" ParentId="3205" CreationDate="2018-06-20T06:58:20.487" Score="2" Body="&lt;p&gt;To extend a bit more to @gringer answer. RIKEN is a large Japanese research institute, that participates in different scientific collaborations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of these is the FANTOM collaboration (Functional ANnotation Of the Mammalian genome), which lately have resulted in the release of &lt;a href=&quot;http://fantom.gsc.riken.jp/cat/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FANTOM CAT&lt;/a&gt;, a browsing tool for exploring lncRNA relative to associated Genes, Traits and Ontologies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many of the transcripts and genes discovered during this work, are not yet named, and are therefore called CAT-genes, following numbering comparable to ensembl (eg. CATG00000118335.1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm unsure how RIK genes have been discovered, as I have not personally encountered these genes yet, but in case you likewise encounter more genes discovered in collaboration with RIKEN, you now also know of CAT genes.&lt;/p&gt;&#xA;" OwnerUserId="2153" LastEditorUserId="2153" LastEditDate="2018-06-21T06:27:39.287" LastActivityDate="2018-06-21T06:27:39.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4521" PostTypeId="1" AcceptedAnswerId="15333" CreationDate="2018-06-20T09:22:06.543" Score="4" ViewCount="255" Body="&lt;p&gt;Such databases have been constructed for example in &lt;a href=&quot;https://ieeexplore.ieee.org/document/5626421/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; or &lt;a href=&quot;https://doi.org/10.1016/j.compbiolchem.2009.10.002&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. But I can't find them anywhere online. Can someone point me to where I can find one available online?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise I'll have to write to these people and I don't know when/whether I'll get a reply.&lt;/p&gt;&#xA;" OwnerUserId="960" LastActivityDate="2021-02-05T10:32:26.533" Title="Is there a database of protein sequences/structure along with their melting temperature?" Tags="|database|public-databases|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4522" PostTypeId="1" AcceptedAnswerId="4525" CreationDate="2018-06-20T09:54:11.133" Score="3" ViewCount="1189" Body="&lt;p&gt;I'm using edgeR for differential analysis. Using glmTreat function I'm detecting differentially expressed genes between Tumor and Normal. I have set the arguments like below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tr &amp;lt;- glmTreat(fit, contrast=contrast.matrix, lfc=2)&#xA;tab2 &amp;lt;- topTags(tr,n=Inf, adjust.method = &quot;BH&quot;)&#xA;keep &amp;lt;- tab2$table$FDR &amp;lt;= 0.05&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This gave me differentially expressed genes with logFC &gt; 2 at FDR &amp;lt; 0.05 and genes with logFC &amp;lt; 2 at FDR &amp;lt; 0.05. [There are few genes with logFC 1.9, 1.89 also]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have also tried with lfc=log2(2).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tr &amp;lt;- glmTreat(fit, contrast=contrast.matrix, lfc=log2(2))&#xA;tab2 &amp;lt;- topTags(tr,n=Inf, adjust.method = &quot;BH&quot;)&#xA;keep &amp;lt;- tab2$table$FDR &amp;lt;= 0.05&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To my surprise I see DE genes with logFC &gt;=1, FDR &amp;lt; 0.05 and also logFC &amp;lt;=1 at FDR &amp;lt; 0.05.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example of DE genes I also found when I used lfc = log2(2)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GeneSymbols logFC   unshrunk.logFC  logCPM  PValue  FDR&#xA;DPYD-IT1    1.14493675  1.560389373 0.67468395  0.002972841 0.018704644&#xA;LINC00945   1.144288473 1.525191847 0.693845996 0.00651175  0.038064312&#xA;ITPKB-AS1   1.14046752  1.484991704 0.703887505 0.006620218 0.038585862&#xA;AC105206.1  1.122495266 1.4871905   0.699167758 0.006545299 0.038194633&#xA;LINC02066   1.120660287 1.558458881 0.66016102  0.004344743 0.026424298&#xA;AC090985.1  1.10235045  1.58968763  0.64343419  0.002745089 0.017426607&#xA;KCNJ6-AS1   1.100399688 1.717028637 0.613324799 0.002224039 0.01438451&#xA;LINC01494   1.090270138 1.490899364 0.676937984 0.006770608 0.039348027&#xA;AC099541.1  1.080957092 1.53817727  0.651356989 0.008449298 0.048356352&#xA;AL133396.2  1.012984207 1.512530231 0.63130643  0.008090259 0.046411928&#xA;AC011363.1  1.01207753  1.499708945 0.635740622 0.007226791 0.041797256&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm very confused with these foldchange cutoffs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which one should I use to select differentially expressed genes based on fold change &gt;=2 and FDR &amp;lt; 0.05?&lt;/p&gt;&#xA;" OwnerUserId="1377" LastEditorUserId="1377" LastEditDate="2018-06-20T11:41:04.960" LastActivityDate="2020-06-21T22:43:57.867" Title="Detecting differentially expressed genes with foldchange &gt;= 2 and FDR &lt; 0.05" Tags="|r|rna-seq|bioconductor|differential-expression|rna|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4523" PostTypeId="2" ParentId="4510" CreationDate="2018-06-20T10:02:37.080" Score="4" Body="&lt;p&gt;The counts stored in the Seurat object are: raw counts (seuratobject@raw.data), the log + normalized counts (seuratobject@data), and the scaled counts (seuratobject@scale.data). &lt;code&gt;FeaturePlot()&lt;/code&gt; plots the log + normalized counts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to identify double-positive cells, you need to identify cells that express a gene (i.e. positive for a gene) and that is not trivial: you need a &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/687/what-methods-are-available-to-find-a-cutoff-value-for-non-expressed-genes-in-rna&quot;&gt;cutoff value&lt;/a&gt; for each gene, and also to &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/693/how-can-we-distinguish-between-true-zero-and-dropout-zero-counts-in-single-cell&quot;&gt;distinguish between true and dropout zero counts&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Provided you solve these, you can retrieve your cells by:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MYH11.cutoff &amp;lt;- 1&#xA;ACTA2.cutoff &amp;lt;- 1&#xA;length(which(FetchData(seuratobject, vars.all='MYH11') &amp;gt; MYH11.cutoff))&#xA;length(which(FetchData(seuratobject, vars.all='ACTA2') &amp;gt; ACTA2.cutoff))&#xA;length(which(FetchData(seuratobject, vars.all='MYH11') &amp;gt; MYH11.cutoff &amp;amp; FetchData(seuratobject, vars.all='ACTA2') &amp;gt; ACTA2.cutoff))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="208" LastActivityDate="2018-06-20T10:02:37.080" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4524" PostTypeId="2" ParentId="4517" CreationDate="2018-06-20T11:34:04.730" Score="4" Body="&lt;p&gt;I strongly suggest that you not try to come up with your own package for this when things like &lt;code&gt;CSAW&lt;/code&gt; already exist in bioconductor and provide a number of useful normalization options.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For visualization purposes, I think it's best to simply take the log2 ratio (e.g., using &lt;code&gt;bamCompare&lt;/code&gt; from deepTools) or alternatively normalize by total coverage (possibly excluding blacklist regions, if applicable for your organism).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-06-20T11:34:04.730" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4525" PostTypeId="2" ParentId="4522" CreationDate="2018-06-20T12:37:08.357" Score="3" Body="&lt;p&gt;Fold-change &gt;= 2 is the same as logFC (&lt;code&gt;log2(fold-change)&lt;/code&gt;) &gt;= 1, so your example is doing exactly what you want. logFC is generally easier to think in and work with than fold-change, since (aside from computational reasons) then increases and decreases in expression differ only in sign and are, thus, easier to compare in magnitude (e.g., it's easier to tell that logFCs of 1.5 and -1.5 are the same magnitude change than it is that 3 and 0.333 are the same magnitude change in the fold-change).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For prioritizing the results, typically you would then rank these filtered results by fold-change and relevance (either given background knowledge or for genes with known function).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-06-20T12:37:08.357" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4526" PostTypeId="1" AcceptedAnswerId="4527" CreationDate="2018-06-20T13:08:11.087" Score="4" ViewCount="250" Body="&lt;p&gt;I'm using TCGA Lung cancer data. I'm interested in doing differential analysis between Lung vs Normal. Before DEA, to check the distance between each pair of samples I plotted an MDS plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/85H6M.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/85H6M.png&quot; alt=&quot;MDS plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this, I see some Tumor samples are clustered with Normals. As this is a two dimensional plot, it is a little hard to say. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I made a clustering heatmap with the top 10% genes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/x2s3E.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/x2s3E.jpg&quot; alt=&quot;clustering heatmap&quot;&gt;&lt;/a&gt; (Where brown color is Normal and black color is Tumor). I also see some Normals clustering with Tumor here, and some Tumor are with the Normals. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: Should I exclude those tumor samples which are clustering with the normals for differential analysis? Is it a good idea to remove samples based on clustering? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I didn't see any technical bias of those samples&lt;/p&gt;&#xA;" OwnerUserId="1369" LastEditorUserId="77" LastEditDate="2018-06-20T13:46:54.923" LastActivityDate="2018-06-20T13:46:54.923" Title="Is removing samples based on clustering for downstream analysis a right choice?" Tags="|rna-seq|differential-expression|clustering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4527" PostTypeId="2" ParentId="4526" CreationDate="2018-06-20T13:16:28.423" Score="5" Body="&lt;p&gt;I would be very hesitant to blindly exclude those samples based on the clustering. Check to see if the clusters actually denote some sort of batch effect, since it's not like all of the TCGA datasets were processed at the same place or time. Check for clinical covariates too. Also, do a PCA and check the genes with high loading on the relevant PC. If it appears that the genes are indicators for cancerous state, then it's likely the samples are just mislabeled and you can reasonably exclude them. Otherwise keep them in so your results better represent the underlying biological variability.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, your heatmap and MDS plots are what I'd expect from a cancer dataset. You occasionally see a tumour with the normal samples and vice versa, since the problematic genes aren't always in the top 10% and there is likely some heterogeneity in the cancer samples.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-06-20T13:46:10.880" LastActivityDate="2018-06-20T13:46:10.880" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4528" PostTypeId="2" ParentId="4489" CreationDate="2018-06-20T17:17:20.613" Score="3" Body="&lt;p&gt;I'd say mostly this is a question of understanding the underlying biology and the relevant literature. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is not known in the literature whether a mark is peaky or broad, evidence might come from FISH studies or Low throughput qPCR. Another way to look would be to examine the signal expressed as fold enrichment over input on a genome browser and look if peaks tended to be small (a few hundred bp, e.g. a transcription factor), or large (several 1kb, e.g. H3K27me3) or contain both (e.g. RNA polymerase II). &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-06-20T17:17:20.613" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4529" PostTypeId="2" ParentId="4495" CreationDate="2018-06-20T17:26:37.043" Score="0" Body="&lt;p&gt;Another places you could look for SNP data would be &lt;a href=&quot;http://www.internationalgenome.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;1000 genomes&lt;/a&gt;, &lt;a href=&quot;http://exac.broadinstitute.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the ExACT database&lt;/a&gt; and the &lt;a href=&quot;http://gnomad.broadinstitute.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;genomAD&lt;/a&gt; database. Cancer specific data is available at the &lt;a href=&quot;https://gdc.cancer.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GDC&lt;/a&gt; and &lt;a href=&quot;https://dcc.icgc.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ICGC&lt;/a&gt; data portal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately a lot of what you'd like to work with is probably covered by restricted access due to privacy concerns. Lots of this data is stored in the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gap&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbGap&lt;/a&gt; database and the &lt;a href=&quot;https://www.ebi.ac.uk/ega/home&quot; rel=&quot;nofollow noreferrer&quot;&gt;EGA&lt;/a&gt;. If you are an academic or employed by a company with a legitimate interest in the data you can usually apply for access to the restricted data, and I've never known anyone to be turned down if you have a legitimate reaons to access the data and can demonstrate your computer systems meet the required security standards. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-06-20T17:26:37.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4530" PostTypeId="1" AcceptedAnswerId="4532" CreationDate="2018-06-20T17:52:00.383" Score="1" ViewCount="42" Body="&lt;p&gt;Where can I find the DNA sequence of the &lt;em&gt;C. elegans&lt;/em&gt; rRNA cluster?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, I would be able to do it using the command-line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried using &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/?term=ribosomal%20RNA%20caenorhabditis%20elegans&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI&lt;/a&gt; but the results are unsatisfactory. I am interested in the DNA sequence that is in the current reference genome of &lt;a href=&quot;http://wormbase.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;WormBase&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried using the Bioconductor biomaRt package and create a table containing the description but when I grep 'rRNA', I only get proteins doing things with rRNA. I used the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ensembl &amp;lt;- biomaRt::useMart('ensembl', dataset = 'celegans_gene_ensembl')&#xA;wormGenes &amp;lt;- biomaRt::getBM(attributes = c('external_gene_name', 'chromosome_name', 'start_position', 'end_position', 'gene_biotype', 'description'), mart = ensembl)&#xA;wormGenes[grep('rRNA', wormGenes$description), 6]&#xA;&#xA;[1] &quot;rRNA 2'-O-methyltransferase fibrillarin  [Source:UniProtKB/Swiss-Prot;Acc:Q22053]&quot;      &#xA;[2] &quot;Pre-rRNA-processing protein pro-1  [Source:UniProtKB/Swiss-Prot;Acc:Q22006]&quot;            &#xA;[3] &quot;Pre-rRNA-processing protein TSR1 homolog  [Source:UniProtKB/Swiss-Prot;Acc:Q19329]&quot;     &#xA;[4] &quot;rRNA methyltransferase 2, mitochondrial  [Source:UniProtKB/Swiss-Prot;Acc:O62251]&quot;      &#xA;[5] &quot;Probable rRNA-processing protein EBP2 homolog  [Source:UniProtKB/Swiss-Prot;Acc:Q09958]&quot;&#xA;[6] &quot;Putative rRNA methyltransferase  [Source:UniProtKB/TrEMBL;Acc:Q5CCJ2]&quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-06-20T21:39:24.590" Title="Where can I find the DNA sequence of the C. elegans rRNA cluster?" Tags="|database|biomart|data-download|sequence-annotation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4531" PostTypeId="1" CreationDate="2018-06-20T19:20:52.383" Score="4" ViewCount="90" Body="&lt;p&gt;Various phylogenetic algorithms estimate ancestral states of a phylogenetic dataset. Is there a way in either maximum parsimony, distance-based methods, or Bayesian inference to indicate what the ancestral states of characters were? &lt;/p&gt;&#xA;" OwnerUserId="2961" LastActivityDate="2019-02-01T10:33:33.850" Title="Do any computational phylogenetic methods enable the specification of ancestral states?" Tags="|software-recommendation|phylogenetics|algorithms|phylogeny|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4532" PostTypeId="2" ParentId="4530" CreationDate="2018-06-20T21:39:24.590" Score="1" Body="&lt;p&gt;The info is actually in the &lt;code&gt;gene_biotype&lt;/code&gt; column of my &lt;code&gt;wormGenes&lt;/code&gt; data frame. I could select the rRNA genes using &lt;code&gt;dplyr::filter(wormGenes, gene_biotype == 'rRNA')&lt;/code&gt;. Unfortunately, no description is available for the genes, so I will have to get it using the gene IDs.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-06-20T21:39:24.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4533" PostTypeId="1" AcceptedAnswerId="4538" CreationDate="2018-06-21T07:52:54.333" Score="2" ViewCount="1422" Body="&lt;p&gt;I have 8 lists of differentially expressed genes for 8 time points (2, 4, 6, 8, 10, 12, 14, 16 hours), the number of genes in each of these lists are not equal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example there are 630 common genes between h2 and h4 time points but I have 1143 genes in h2 list and 768 genes in h4 list , so common genes in h2 vs h4 would be 630/1143 and common genes h4 vs h2 would be 630/768.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That would would be great if I have a code to produce a matrix or plot of all possible pairwise combinations of my 8 time points.&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="2153" LastEditDate="2018-06-21T08:49:40.693" LastActivityDate="2018-06-26T14:41:21.180" Title="Multiway intersection of 8 lists of genes" Tags="|r|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4534" PostTypeId="1" AcceptedAnswerId="4536" CreationDate="2018-06-21T10:23:44.117" Score="7" ViewCount="9643" Body="&lt;p&gt;I am trying to use the &lt;a href=&quot;https://bioinf.shenwei.me/seqkit/usage/#replace&quot; rel=&quot;noreferrer&quot;&gt;seqkit replace&lt;/a&gt; command to replace chromosome names in the format &lt;code&gt;chr_I, chr_II, ...&lt;/code&gt; to &lt;code&gt;I, II, ...&lt;/code&gt;. I am using the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqkit replace -p &amp;quot;(.)&amp;quot; --replacement &amp;quot;{kv}&amp;quot; --kv-file keyValues.txt mySequence.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My &lt;code&gt;keyValues.txt&lt;/code&gt; file contains the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr_I   I&#xA;chr_II  II&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The two columns are separated by a tabulation.&lt;/p&gt;&#xA;&lt;p&gt;I get the following output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[INFO] read key-value file: keyValues.txt&#xA;[INFO] 6 pairs of key-value loaded&#xA;[ERRO] pattern &amp;quot;(.)&amp;quot; matches multiple targets in &amp;quot;chr_I&amp;quot;, this will cause chaos&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I don't know what I am doing wrong. I chose the &lt;code&gt;(.)&lt;/code&gt; pattern to match the whole header but it seems to be wrong. Any help would be appreciated.&lt;/p&gt;&#xA;&lt;h3&gt;Update&lt;/h3&gt;&#xA;&lt;p&gt;All the headers are shown below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep '&amp;gt;' mySequence.fasta&#xA;&amp;gt;chr_I&#xA;&amp;gt;chr_II&#xA;&amp;gt;chr_III&#xA;&amp;gt;chr_IV&#xA;&amp;gt;chr_V&#xA;&amp;gt;chr_X&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2023-04-22T07:55:29.393" Title="How do I rename fasta headers?" Tags="|fasta|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4536" PostTypeId="2" ParentId="4534" CreationDate="2018-06-21T12:09:36.550" Score="10" Body="&lt;p&gt;Why not use &lt;code&gt;sed&lt;/code&gt; instead?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -e 's/chr_I/I/' -e 's/chr_V/V/' -e 's/chr_X/X/' mySequence.fasta &amp;gt; mySeq.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or even simpler:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed 's/chr_//' mySequence.fasta &amp;gt; mySeq.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2018-06-21T12:39:35.483" LastActivityDate="2018-06-21T12:39:35.483" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4537" PostTypeId="2" ParentId="4517" CreationDate="2018-06-21T14:26:49.100" Score="1" Body="&lt;p&gt;If you want to compare conditions (or strains), I don't see why you need Input at all (unless you want to define intervals).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It isn't clear from you question (and subsequent comment) on what you want to achieve, but I can see 2 possible scenarios:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;You already have predefined intervals to test on&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case (Pol2-ChIP), this might be for example gene promoters.&#xA;You could compare the Pol2-ChiP libraries directly using standard test for count data as established for RNA-Seq (i.e. edgeR, baySeq etc) in which the intervals are genes (as opposed to promoters or so). No need for input in this scenario, all you need are ChIP libraries for both conditions (don't forget to do replicates).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;You don't have predefined intervals&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could defined them through &lt;strong&gt;peak calling&lt;/strong&gt;, in this case you'd in fact need input. A nice approach for this is described in &lt;a href=&quot;http://dx.doi.org/10.1186/gb-2012-13-10-r98&quot; rel=&quot;nofollow noreferrer&quot;&gt;Diaz et al&lt;/a&gt; and implemented in &lt;a href=&quot;https://github.com/deeptools/deepTools/wiki/Normalizations&quot; rel=&quot;nofollow noreferrer&quot;&gt;deeptools&lt;/a&gt;. It poses an elegant method to separate the IP signal from background input, which is probably what you want I suppose. Having defined peaks, you could do scenario 1) using them as intervals.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1461" LastActivityDate="2018-06-21T14:26:49.100" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4538" PostTypeId="2" ParentId="4533" CreationDate="2018-06-21T16:09:13.220" Score="3" Body="&lt;p&gt;EDIT: Sorry, I did not see the R tag and wrote the code in bash. I still left the bash example below and added an R version.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depends on what you want to do, but there are two common ways in which you could store this. Either as wide format: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;      h2  h4  h6 ...&#xA;Gene1 1   1   0&#xA;Gene2 0   1   1&#xA;Gene3 1   1   1&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where 1 denotes presence and 0 absence, or as long format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;      timepoint&#xA;Gene1 h2&#xA;Gene1 h4&#xA;Gene2 h4&#xA;Gene2 h6&#xA;Gene3 h2&#xA;Gene3 h4&#xA;Gene3 h6&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The best format probably depends on what you want to do. Since you did not specify a method/language, I took the liberty to write a few lines of bash to go from the gene lists to the matrices. &#xA;Bash examples assume your files are named h2, h4 etc and are tab separated with gene names in the first column. R examples assume &lt;code&gt;tables&lt;/code&gt; is a list of your data frames and &lt;code&gt;timepoints&lt;/code&gt; is a vector &lt;code&gt;c(&quot;h2&quot;, &quot;h4&quot;, ...)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Generate table in wide format:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Concatenate gene names of all tables to a separate file&lt;/li&gt;&#xA;&lt;li&gt;Extract unique values in this file&lt;/li&gt;&#xA;&lt;li&gt;Perform outer join between the separate file and each table&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;R:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;all_names &amp;lt;- c()&#xA;for (tp in timepoints){&#xA;  all_names &amp;lt;- append(all_names, tables[[tp]]$genes)&#xA;}&#xA;all_names &amp;lt;- unique(all_names)&#xA;&#xA;wide_table &amp;lt;- matrix(nrow=length(all_names), ncol=(1+length(timepoints)))&#xA;wide_table &amp;lt;- as.data.frame(wide_table)&#xA;colnames(wide_table) &amp;lt;- append(&quot;genes&quot;, timepoints)&#xA;&#xA;for (tp in timepoints){&#xA; tmp &amp;lt;- tables[[tp]]&#xA; tmp$tp &amp;lt;- 1&#xA; tmp &amp;lt;- merge(wide_table, tmp, by=&quot;genes&quot;, all=T)&#xA; wide_table$tp &amp;lt;- tmp$tp&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Bash:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: you will might need to add headers to this table&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for timefile in $yourfiles;do&#xA;  # make a file with all gene names&#xA;  awk -v tp=$timefile '{print $1}' &amp;gt;&amp;gt; names_file&#xA;done&#xA;&#xA;sort names_file | uniq &amp;gt; tmp &amp;amp;&amp;amp; mv tmp names_file&#xA;&#xA;for timefile in $yourfiles;do&#xA;  join -a1 -a2  -j 1 -o 0,1.2,2.2 -e 0 $names_file &amp;lt;(awk '{print$1,1}' $timefile | sort)&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Generate table in long format:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;For each table, add a column with timepoint&lt;/li&gt;&#xA;&lt;li&gt;Concatenate all tables&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;R:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df &amp;lt;- data.frame()&#xA;for (tp in timepoints){&#xA;  tables[[tp]][[timepoint]] &amp;lt;- tp&#xA;  df &amp;lt;- rbind(df, tables[[tp]] )&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Bash:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   for timefile in $yourfiles;do&#xA;      awk -v tp=$timefile '{print $1,timefile}' &amp;gt;&amp;gt; long_table&#xA;    done &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;EDIT2: To follow up on this, here is a simple way (using the package &lt;a href=&quot;https://www.tidyverse.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dplyr&lt;/a&gt;) to count the number of common genes between all possible combination of time points. Assuming you have your data in long format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df &amp;lt;- data.frame(genes=c(&quot;G1&quot;,&quot;G1&quot;,&quot;G2&quot;,&quot;G3&quot;,&quot;G3&quot;), tp=c(&quot;h2&quot;,&quot;h4&quot;,&quot;h2&quot;,&quot;h4&quot;,&quot;h6&quot;))&#xA;&#xA;  genes tp&#xA;1    G1 h2&#xA;2    G1 h4&#xA;3    G2 h2&#xA;4    G3 h4&#xA;5    G3 h6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can get the co-occurrences using :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(dplyr)&#xA;# Make timepoint ordinal factor&#xA;df &amp;lt;- mutate(df, tp = factor(tp, ordered=T))&#xA;df &amp;lt;- df %&amp;gt;% &#xA;  # Compute all pairwise co-occurrences&#xA;  full_join(df, by=&quot;genes&quot;) %&amp;gt;% &#xA;  group_by(tp.x, tp.y) %&amp;gt;% &#xA;  summarise(N_genes = length(unique(genes))) %&amp;gt;%&#xA;  # Exclude self pairs (gene in h2 and h2)&#xA;  filter(tp.x != tp.y) %&amp;gt;% &#xA;  # Check for duplicate pairs in reverse order (h2-h4 and h4-h2)&#xA;  mutate(comb=ifelse(tp.x &amp;lt; tp.y, &#xA;                     yes = paste(tp.x, tp.y), &#xA;                     no  = paste(tp.y, tp.x)))&#xA;# Remove the duplicate pairs&#xA;df &amp;lt;- df[which(!duplicated(df$comb)),]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And visualise it using for example a heatmap:&#xA;(ofc it looks stupid because I have a tiny sample where not all combinations are even present)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Visualize number of genes present in each combination&#xA;library(ggplot2)&#xA;ggplot(df, aes(x=tp.x, y=tp.y, fill=N_genes)) + geom_tile()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="439" LastEditorUserId="439" LastEditDate="2018-06-25T07:35:24.700" LastActivityDate="2018-06-25T07:35:24.700" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="4539" PostTypeId="1" AcceptedAnswerId="4560" CreationDate="2018-06-21T17:24:00.370" Score="2" ViewCount="129" Body="&lt;p&gt;I have 101 samples from 9 tissues of a nonmodel species, coming either from SRA or sequenced by my lab. I want to generate a reference transcriptome with Trinity for further analysis. In order to do this, I downsampled the reads in order to have 20 to 30M reads for each tissue, for a total of 270M PE reads, which I combined in two files, one for left reads and one for right ones. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now,  I came across the --samples_file flag in Trinity and I'm not sure if I should use it instead of concatenating the transcriptomes. Any help on good practice on this?&lt;/p&gt;&#xA;" OwnerUserId="1637" LastEditorUserId="1637" LastEditDate="2018-06-25T13:53:56.307" LastActivityDate="2018-06-25T13:53:56.307" Title="Correct way to assemble reference transcriptome - what's --samples_file option in Trinity?" Tags="|assembly|transcriptome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4540" PostTypeId="2" ParentId="4409" CreationDate="2018-06-21T18:35:11.780" Score="-1" Body="&lt;p&gt;We implemented &lt;a href=&quot;https://github.com/bio2bel/kegg&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bio2BEL KEGG&lt;/a&gt; package to download the KEGG database via its RESTful API&lt;/p&gt;&#xA;" OwnerUserId="2977" LastEditorUserId="48" LastEditDate="2018-06-21T18:44:44.060" LastActivityDate="2018-06-21T18:44:44.060" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4541" PostTypeId="1" AcceptedAnswerId="6713" CreationDate="2018-06-21T21:32:58.507" Score="5" ViewCount="3703" Body="&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/4061/why-do-neb-adapters-have-non-complementary-sequence&quot;&gt;In a previous post&lt;/a&gt; I asked &lt;strong&gt;Why do NEB adapters have non-complementary sequence?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since then, I realized that there is some other sequence in the p5 indexing primer, as well as in the p7 indexing primer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a diagram of the NEBNext protocol. The parts that I am confused about are in step 5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7Gc73.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7Gc73.png&quot; alt=&quot;NEBNext protocol&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;My first question is about the p5 index. Why is there a sequence &lt;code&gt;5'GATCTACAC 3'&lt;/code&gt; between the p5 flowcell oligo and the index? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1K2yM.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1K2yM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;My second question is about both indices. Why is there no gap between the p7 index and the flowcell annealing oligo?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EhUwu.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EhUwu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference, I fetched these indexing primer sequences from &lt;a href=&quot;https://www.neb.com/-/media/catalog/datacards-or-manuals/manuale7780.pdf&quot; rel=&quot;noreferrer&quot;&gt;NEB on page 18 and 19 of NEBNext Multiplex Oligos for Illumina instruction manual&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-06-21T21:55:30.607" LastActivityDate="2018-12-21T22:20:31.400" Title="Why do NEBNext indexing primers have sequence between the p5 oligo and index?" Tags="|phylogenetics|illumina|adapter|pcr|library|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4542" PostTypeId="2" ParentId="508" CreationDate="2018-06-22T02:09:02.160" Score="4" Body="&lt;p&gt;For much faster and stable&lt;sup&gt;$&lt;/sup&gt; implementation, sambamba can handle this using following one-liner:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sambamba view -t 12 -h -f bam -F &quot;mapping_quality &amp;gt;= 1 and not (unmapped or secondary_alignment) and not ([XA] != null or [SA] != null)&quot; test.bwa-mem.bam -o test-uniq.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;http://lomereiter.github.io/sambamba/docs/sambamba-view.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;manpage&lt;/a&gt; and &lt;a href=&quot;https://github.com/biod/sambamba/wiki/%5Bsambamba-view%5D-Filter-expression-syntax&quot; rel=&quot;nofollow noreferrer&quot;&gt;sambamba synatx for filters&lt;/a&gt; for details on further usage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$: Besides &lt;code&gt;grep&lt;/code&gt; based search being slow, it may return false &lt;del&gt;positive&lt;/del&gt; negative hits (@gringer's reply above). I also do not find &lt;code&gt;awk&lt;/code&gt; based approach reliable because fields like XA, SA, etc. are optional fields in &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAM format&lt;/a&gt;, and not &lt;em&gt;positionally&lt;/em&gt; fixed fields.&lt;/p&gt;&#xA;" OwnerUserId="2978" LastEditorUserId="2978" LastEditDate="2018-06-22T15:48:08.723" LastActivityDate="2018-06-22T15:48:08.723" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4543" PostTypeId="1" CreationDate="2018-06-22T07:26:31.610" Score="0" ViewCount="33" Body="&lt;p&gt;I need good sources of gene expression data in the form of 3rd order tensors. Typically the commonly available datasets are in the form of a matrix, for instance, $sample \times gene$ or $gene \times time$. I need good sources for 3-dimensional data, for example $gene \times sample \times tissue$ or $gene \times sample \times time$ ... etc.&lt;/p&gt;&#xA;" OwnerUserId="459" LastActivityDate="2018-06-22T14:13:46.360" Title="What are some good sources of 3rd order gene expression data" Tags="|rna-seq|public-databases|microarray|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4544" PostTypeId="1" CreationDate="2018-06-22T10:58:54.820" Score="1" ViewCount="2903" Body="&lt;p&gt;Using edgeR for differential analysis between Tumor and Normal gave me differential expressed genes with logFC, logCPM, PValue and FDR. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the details of glmTreat function I see that logCPM is average log2-counts per million, the average taken over all libraries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output table I got from glmTreat:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Geneid     logFC  unshrunk.logFC    logCPM  PValue       FDR&#xA;Gene1   4.985298235 5.573931983 1.236660486 1.55E-54    1.87E-50&#xA;Gene2   4.613438634 5.52126484  0.920343233 2.59E-53    1.56E-49&#xA;Gene3   5.250601296 5.356432653 2.751294034 2.48E-50    9.95E-47&#xA;Gene4   4.943049159 5.379741165 1.361393757 2.06E-45    6.19E-42&#xA;Gene5   6.100121754 6.117580436 5.774385315 2.95E-43    7.11E-40&#xA;Gene6   4.722697891 5.320461275 1.120685402 3.05E-42    6.11E-39&#xA;Gene7   5.246129853 5.497012001 1.902992053 2.78E-40    4.78E-37&#xA;Gene8   3.878773277 4.956636276 0.776208741 8.66E-39    1.30E-35&#xA;Gene9   4.441752496 4.94930499  1.132652682 4.37E-38    5.83E-35&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To calculate logCPM given in the above table manually I did like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;logCPM &amp;lt;- cpm(y, prior.count=2,  log=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I got the logCPM values for all the genes. Then I calculated Average across all the samples for each gene. I got like below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Geneid  Average&#xA;Gene1   0.686560246&#xA;Gene2   0.617115826&#xA;Gene3   1.075975225&#xA;Gene4   0.692050878&#xA;Gene5   1.277556065&#xA;Gene6   0.638358189&#xA;Gene7   0.689323163&#xA;Gene8   0.60700396&#xA;Gene9   0.662115092&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Why there is so difference? Where I'm doing wrong?&lt;/p&gt;&#xA;" OwnerUserId="1377" LastActivityDate="2023-09-05T08:07:19.447" Title="How to calculate logCPM across all samples?" Tags="|r|rna-seq|bioconductor|rna|edger|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4545" PostTypeId="2" ParentId="4495" CreationDate="2018-06-22T11:12:16.337" Score="0" Body="&lt;p&gt;Yet another place I find useful is provided by &lt;a href=&quot;https://repositive.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Repositive&lt;/a&gt;. &#xA;You can think of it as a meta-search engine which includes most of the databases already stated and links hits to relevant papers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With respect to your question about bipolar, &lt;a href=&quot;https://discover.repositive.io/datasets/search?query=bipolar&quot; rel=&quot;nofollow noreferrer&quot;&gt;this search&lt;/a&gt; could get you started.&lt;/p&gt;&#xA;" OwnerUserId="1461" LastActivityDate="2018-06-22T11:12:16.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4546" PostTypeId="2" ParentId="4544" CreationDate="2018-06-22T12:30:23.773" Score="3" Body="&lt;p&gt;If you want the values by group then subset &lt;code&gt;y&lt;/code&gt; to contain the samples of interest and then feed that to &lt;code&gt;aveLogCPM()&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-06-22T12:30:23.773" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4547" PostTypeId="2" ParentId="4543" CreationDate="2018-06-22T14:13:46.360" Score="1" Body="&lt;p&gt;I familiar with the inflammatory bowel disease  which is quite complicated: &#xA;It affects multiple sites and depending on when it appears it is different. So you can look at the GEO for microarrays and RNA-seq data of this disease and I'm sure you'll find datasets from the same patients from several tissues or time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this disease there is a &lt;a href=&quot;https://ibdmdb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;multi'omics project&lt;/a&gt; that fits your needs. You can find (and download) data of high dimensionality: there are samples for the same time, tissue (but different locations), gene, over a period of time and for several omics (not only expression data).&lt;br&gt;&#xA;The only problem is that there are some mistakes on the metadata. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't think they are usually described as 3rd order tensors and I doubt you'll find enough data to train a machine learning method. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-06-22T14:13:46.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4548" PostTypeId="1" AcceptedAnswerId="4554" CreationDate="2018-06-22T19:16:51.357" Score="3" ViewCount="4528" Body="&lt;p&gt;Sorry I just got totally confused conceptually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is my raw count data&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Likely, Seurat divides each value by sum of the column afterward times by 10000. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;which gives so &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                   s1.1        s1.2 s1.3       s1.4 s1.5&#xA;DDB_G0267178 0.00000000 0.009263254    0 0.01286397    0&#xA;DDB_G0267180 0.00000000 0.000000000    0 0.00000000    0&#xA;DDB_G0267182 0.00000000 0.000000000    0 0.03810585    0&#xA;DDB_G0267184 0.00000000 0.000000000    0 0.00000000    0&#xA;DDB_G0267188 0.02640801 0.000000000    0 0.01286397    0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Literally Seurat claims that final step would be taking natural log of the above matrix as I tested with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(seurat)&#xA;mat &amp;lt;- matrix(data = rbinom(n = 25, size = 5, prob = 0.2), nrow = 5)&#xA;mat&#xA;&#xA;    [,1] [,2] [,3] [,4] [,5]&#xA;[1,]    1    0    1    1    1&#xA;[2,]    1    0    2    2    3&#xA;[3,]    1    1    1    1    2&#xA;[4,]    1    1    0    1    0&#xA;[5,]    0    0    0    1    3&#xA;mat_norm &amp;lt;- LogNormalize(data = mat)&#xA;mat_norm&#xA;[1,] 7.824446 .        7.824446 7.419181 7.014015&#xA;[2,] 7.824446 .        8.517393 8.112028 8.112028&#xA;[3,] 7.824446 8.517393 7.824446 7.419181 7.706713&#xA;[4,] 7.824446 8.517393 .        7.419181 .       &#xA;[5,] .        .        .        7.419181 8.112028&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My confusion is: seurat@data dose not give me natural log transformed data rather returns only data divided by column sum and scalled by 10000.&#xA;So when plotting a violin plot if data are log normalised plot should not look as below picture &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I used &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;VlnPlot(object = y, features.plot = &quot;DDB_G0277853&quot;, x.lab.rot = FALSE, y.log = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;plot changed totally. Could someone please help me in getting idea about seurat@data that is not really log transformed and by which data violin plot is being produced??? So if a gene has 10000 read counts, without log transformation violin plot should be so&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my seurat object&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2018-06-24T20:42:33.593" LastActivityDate="2018-06-24T20:42:33.593" Title="Which data is being used for violin plot?" Tags="|r|scrnaseq|seurat|ggplot2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4550" PostTypeId="1" AcceptedAnswerId="4552" CreationDate="2018-06-22T22:18:24.977" Score="3" ViewCount="173" Body="&lt;p&gt;I am doing a project that requires bench-marking the performance of computational tools for alignment and variant calling of human genome sequencing data. In particular I would like to establish benchmarks comparing the performance of &lt;a href=&quot;https://software.broadinstitute.org/gatk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATK&lt;/a&gt; and &lt;a href=&quot;https://github.com/bigdatagenomics/adam&quot; rel=&quot;nofollow noreferrer&quot;&gt;ADAM&lt;/a&gt;/&lt;a href=&quot;https://github.com/bigdatagenomics/avocado&quot; rel=&quot;nofollow noreferrer&quot;&gt;Avocado&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My main question is which genomic data-sets should I use as the input to the data processing pipeline. In Frank Nothaft's (author of ADAM and Avodaco) &lt;a href=&quot;https://www2.eecs.berkeley.edu/Pubs/TechRpts/2017/EECS-2017-204.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;thesis&lt;/a&gt; he reports bench-marking the processing of NA12878 from the 1,000 genome's project in Avocado as compared to GATK's HalotypeCaller. He also reports benchmarking on 270 samples from the Simons Genome Diversity Data-set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in reproducing this portion of his work, but do not know where I can get a hold of these datasets. Where can I get a copy of the NA12878 genome (BAM file) and also were can I get aligned samples from Simons Genome Diversity Data-set? &lt;/p&gt;&#xA;" OwnerUserId="1196" LastEditorUserId="1196" LastEditDate="2018-06-25T16:44:36.547" LastActivityDate="2018-06-25T16:44:36.547" Title="Human Genome Alignment and Variant Calling Benchmarks" Tags="|gatk|adam|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4551" PostTypeId="2" ParentId="4548" CreationDate="2018-06-22T22:22:32.790" Score="2" Body="&lt;p&gt;Those two graphs don't look radically different.   The y axis is definitely not log scaled on the top one, and it looks like it is scaled in the second one (y.log = TRUE)&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2018-06-22T22:22:32.790" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4552" PostTypeId="2" ParentId="4550" CreationDate="2018-06-23T01:43:49.577" Score="2" Body="&lt;h3&gt;NA12878&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;My recommendation: download raw reads from &lt;a href=&quot;https://basespace.illumina.com/home/index&quot; rel=&quot;nofollow noreferrer&quot;&gt;Illumina BaseSpace&lt;/a&gt; and do alignment yourself. Aligning ~40X worth of human reads takes overnight with ~16 CPU cores.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can acquire &lt;a href=&quot;https://www.illumina.com/platinumgenomes.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Platinum Genome&lt;/a&gt; data &lt;a href=&quot;https://www.ebi.ac.uk/ena/data/view/ERX168836&quot; rel=&quot;nofollow noreferrer&quot;&gt;from ENA&lt;/a&gt;. There is a BAM available for download, but that was done by Illumina's own aligner against a wrong reference genome. You'd better do alignment by yourself. Another problem with this dataset is it is too good to be representative. Modern data are worse.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;GIAB provides BAMs here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ftp://ftp-trace.ncbi.nih.gov/giab/ftp/data/NA12878/NIST_NA12878_HG001_HiSeq_300x/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is not &quot;typical&quot;, either.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;SGDP&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Instructions on downloading SGDP BAMs are available &lt;a href=&quot;https://www.simonsfoundation.org/simons-genome-diversity-project/&quot; rel=&quot;nofollow noreferrer&quot;&gt;on this page&lt;/a&gt;, in particular &lt;a href=&quot;http://simonsfoundation.s3.amazonaws.com/share/SCDA/datasets/2014_11_12/StepstodownloadtheSGDPdataset_v4.docx&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Word document&lt;/a&gt;. Warning: you need ~30TB disk space for these BAMs. I also heard it is tricky to download data.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is apparently a mirror &lt;a href=&quot;https://docs.cancergenomicscloud.org/v1.0/docs/simons-genome-diversity-project-sgdp-dataset&quot; rel=&quot;nofollow noreferrer&quot;&gt;from cancer genomics cloud&lt;/a&gt;. I don't know how easy it is to access.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-06-23T01:43:49.577" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4553" PostTypeId="1" CreationDate="2018-06-23T14:21:22.997" Score="1" ViewCount="84" Body="&lt;p&gt;I have 209 cells. I want to show the expression of two genes by ordering my cells. Imagine something like a plot on which y axis shows the read counts range from 0 to 10000 and x axis shows the number of cells (there is a link of an example picture that I wish to produce the same). So that two colours bars each colour for a gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used this code but result is not as I want at all&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Library&#xA; library(tidyverse)&#xA;&#xA; # Create data&#xA; value1=abs(rnorm(26))*2&#xA; data=data.frame(x=LETTERS[1:26], value1=value1, value2=value1+1+rnorm(26, sd=1) )&#xA;&#xA; # Reorder data using average?&#xA; data = data %&amp;gt;% rowwise() %&amp;gt;% mutate( mymean = mean(c(value1,value2) )) %&amp;gt;% arrange(mymean) %&amp;gt;% mutate(x=factor(x, x))&#xA;&#xA; # plot&#xA; ggplot(data) +&#xA;   geom_segment( aes(x=x, xend=x, y=value1, yend=value2), color=&quot;grey&quot;) +&#xA;   geom_point( aes(x=x, y=value1), color=rgb(0.2,0.7,0.1,0.5), size=3 ) +&#xA;   geom_point( aes(x=x, y=value2), color=rgb(0.7,0.2,0.1,0.5), size=3 ) +&#xA;   coord_flip() &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is link of my data and my result picture&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BIcaY.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BIcaY.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And to some &lt;a href=&quot;https://drive.google.com/file/d/1PPFjULGV4CfJvRbubtn6kb5f5qYlpJ5W/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;data&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2018-06-24T20:41:37.493" LastActivityDate="2018-06-24T20:41:37.493" Title="Ordering the cells by the expression of specific genes" Tags="|r|scrnaseq|ggplot2|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4554" PostTypeId="2" ParentId="4548" CreationDate="2018-06-23T16:45:21.043" Score="2" Body="&lt;p&gt;The normalized and log-transformed values are used for the violin plot.&#xA;The argument &lt;code&gt;y.log&lt;/code&gt; changes only the display of the data (scaling of the y axis).&lt;/p&gt;&#xA;&lt;p&gt;Seurat has very good documentation. &lt;a href=&quot;https://satijalab.org/seurat/faq&quot; rel=&quot;nofollow noreferrer&quot;&gt;Section 7 in the FAQ&lt;/a&gt; explains what data is stored in the object:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol start=&quot;7&quot;&gt;&#xA;&lt;li&gt;How is data stored within the Seurat object? What is the difference between raw.data, data, and scale.data?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;raw.data&lt;/strong&gt; The raw data slot&#xA;(object@raw.data) represents the original expression matrix, input&#xA;when creating the Seurat object, and prior to any preprocessing by&#xA;Seurat.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;data&lt;/strong&gt; The data slot&#xA;(object@data) stores normalized and log-transformed single cell&#xA;expression. This maintains the relative abundance levels of all genes,&#xA;and contains only zeros or positive values. See ?NormalizeData for&#xA;more information. This data is used for visualizations, such as violin&#xA;and feature plots, most differential expression tests, finding&#xA;high-variance genes, and as input to ScaleData (see below).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;scale.data&lt;/strong&gt;&#xA;The scale.data slot (object@scale.data) represents a cell’s relative&#xA;expression of each gene, in comparison to all other cells. Therefore&#xA;this matrix contains both positive and negative values. See ?ScaleData&#xA;for more information.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="208" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-06-23T16:45:21.043" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4555" PostTypeId="2" ParentId="4289" CreationDate="2018-06-23T18:56:59.353" Score="0" Body="&lt;p&gt;You can represent, and calculate, the correlations using psych:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(psych)&#xA;pairs.panels(df,df, &#xA;             method = &quot;pearson&quot;, # correlation method&#xA;             hist.col = &quot;#00AFBB&quot;,&#xA;             density = TRUE,  # show density plots&#xA;             ellipses = TRUE,xlim=c(0,500),ylim=c(0,500))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="48" LastEditDate="2018-06-25T07:56:12.137" LastActivityDate="2018-06-25T07:56:12.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4556" PostTypeId="1" AcceptedAnswerId="4561" CreationDate="2018-06-24T16:36:53.100" Score="1" ViewCount="144" Body="&lt;p&gt;I have a data frame containing 4 columns:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Chrom: chromosome name of the feature&lt;/li&gt;&#xA;&lt;li&gt;Start: starting base of the feature &lt;/li&gt;&#xA;&lt;li&gt;End: end base of the feature&lt;/li&gt;&#xA;&lt;li&gt;Name: name of the feature, e.g. 5S ribosomal RNA&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The head of the data frame is shown below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;      Chrom    Start      End                        Name&#xA;    1     1 15390242 15390720 16S ribosomal RNA (partial)&#xA;    2     1 15390900 15391794 16S ribosomal RNA (partial)&#xA;    3     1 15392798 15396046           23S ribosomal RNA&#xA;    4     1 15397439 15397917 16S ribosomal RNA (partial)&#xA;    5     1 15398097 15398991 16S ribosomal RNA (partial)&#xA;    6     1 15399995 15403243           23S ribosomal RNA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to plot these features on the karyotype of &lt;em&gt;C. elegans&lt;/em&gt;. I am using the &lt;a href=&quot;https://bioconductor.org/packages/devel/bioc/vignettes/chromPlot/inst/doc/chromPlot.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;chromPlot&lt;/a&gt; package from Bioconductor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the following code:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Subsetting the original data frame to get a data frame for each feature type: 5S, 16S and 23S rRNAs&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;annot1 &amp;lt;- dplyr::filter(cel, Name == '5S ribosomal RNA')&#xA;annot2 &amp;lt;- dplyr::filter(cel, Name == '23S ribosomal RNA')&#xA;annot3 &amp;lt;- dplyr::filter(cel, Name == '16S ribosomal RNA (partial)')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;chromPlot(gaps = cel, figCols = 2, annot1 = annot1, annot2 = annot2)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4HnwQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4HnwQ.png&quot; alt=&quot;Karyotype with 2 annotations&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is OK because the two features are far away in the genome. However, if I try:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chromPlot(gaps = cel_gap, figCols = 2, annot1 = annot1, annot2 = annot2,&#xA;          annot3 = annot3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hWrx3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hWrx3.png&quot; alt=&quot;Karyotype with 3 annotations&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;one of the features masks the other because they are close to each other in the genome. How can I make the green bar stack on top of the yellow one instead of covering it?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="48" LastEditDate="2018-09-04T10:08:52.457" LastActivityDate="2018-09-04T10:08:52.457" Title="How to superimpose barplots in karyotype graphic using the chromPlot Bioconductor package?" Tags="|r|bioconductor|visualization|chromplot|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4557" PostTypeId="2" ParentId="4534" CreationDate="2018-06-24T17:38:12.380" Score="1" Body="&lt;p&gt;Here is an alternative to sed: &lt;code&gt;cat test | tr -d &quot;chr_&quot;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The pipe is to output the file, while the &lt;code&gt;tr&lt;/code&gt; command is to translate and change the file&lt;/p&gt;&#xA;" OwnerUserId="2970" LastEditorUserId="48" LastEditDate="2018-06-25T07:58:30.013" LastActivityDate="2018-06-25T07:58:30.013" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4558" PostTypeId="2" ParentId="4533" CreationDate="2018-06-24T18:42:58.863" Score="1" Body="&lt;p&gt;I would simply do this...&#xA;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# sepcify your timepoints, each timepoint is a list of genes!!&#xA;timepoints &amp;lt;- list(h2,h4,h6,h8, h10,h12,h14,h16)&#xA;# initialize an empty array that you will fill with your timepoints&#xA;vect = vector()&#xA;# go over all combinations and fill the array with the set intersection values you want (I guess...)&#xA;for (i in timepoints) {&#xA;  for (j in timepoints) {&#xA;    vect &amp;lt;- c(vect, length(intersect(i,j))/length(i))}}&#xA;# reshape your array to a more ordered matrix  &#xA;your_matrix &amp;lt;- matrix(vect, nrow=length(timepoints))&#xA;# assign columns and rows names&#xA;colnames(your_matrix) &amp;lt;- c(&quot;h2&quot;,&quot;h4&quot;,&quot;h6&quot;,&quot;h8&quot;,&quot;h10&quot;,&quot;h12&quot;,&quot;h14&quot;,&quot;h16&quot;)&#xA;rownames(your_matrix) &amp;lt;- c(&quot;h2&quot;,&quot;h4&quot;,&quot;h6&quot;,&quot;h8&quot;,&quot;h10&quot;,&quot;h12&quot;,&quot;h14&quot;,&quot;h16&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2970" LastActivityDate="2018-06-24T18:42:58.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4560" PostTypeId="2" ParentId="4539" CreationDate="2018-06-25T13:42:08.387" Score="2" Body="&lt;p&gt;&lt;code&gt;--samples_file&lt;/code&gt; flag is for &lt;a href=&quot;https://github.com/trinityrnaseq/trinityrnaseq/wiki/Trinity-Differential-Expression&quot; rel=&quot;nofollow noreferrer&quot;&gt;DE analysis&lt;/a&gt;, you can put info about biological replicates in that file. You say that you want to make a ref transcriptome, so you don't need a samples file for DE analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do you down sample? Are you not afraid you will not have enough reads to have a complete transcriptome now?&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2018-06-25T13:42:08.387" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4561" PostTypeId="2" ParentId="4556" CreationDate="2018-06-25T14:09:31.313" Score="2" Body="&lt;pre&gt;&lt;code&gt;chromPlot(gaps = cel_gap, figCols = 2, annot1 = annot1, annot2 = annot3, annot3 = annot2,&#xA;          chrSide=c(1, 1, -1, 1, -1, 1, -1, 1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;chrSide&lt;/code&gt; argument decides on which side the histogram goes, so it can be used to solve my problem. The bars are not stacked but the result is the same: more accurate visualisation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks to Llopis for the guidance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VkMv9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VkMv9.png&quot; alt=&quot;karyotype&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-06-25T14:09:31.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4562" PostTypeId="1" AcceptedAnswerId="4573" CreationDate="2018-06-25T14:09:56.443" Score="2" ViewCount="812" Body="&lt;p&gt;I have the following two small FASTAs, &lt;code&gt;dna1.fa&lt;/code&gt; and &lt;code&gt;dna2.fa&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# dna1.fa&#xA;&amp;gt;chr1&#xA;ACTGACTAGCTAGCTAACTG&#xA;&amp;gt;chr2&#xA;GCATCGTAGCTAGCTACGAT&#xA;&amp;gt;chr3&#xA;CATCGATCGTACGTACGTAG&#xA;&amp;gt;chr4&#xA;ATCGATCGATCGTACGATCG&#xA;&#xA;# dna2.fa&#xA;&amp;gt;chr1&#xA;GCATCGTAGCTAGCTACGAT&#xA;&amp;gt;chr3&#xA;CATCGATCGTACGTACGTAG&#xA;&amp;gt;chr4&#xA;ATCGATCGATCGTACGATCG&#xA;&amp;gt;chr5&#xA;CATCGATCGTACGTACGTAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I concatenate these two files using &lt;code&gt;cat&lt;/code&gt;, the piped result is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat dna1.fa dna2.fa &amp;gt; total1.fa&#xA;$ cat total1.fa&#xA;&amp;gt;chr1&#xA;ACTGACTAGCTAGCTAACTG&#xA;&amp;gt;chr2&#xA;GCATCGTAGCTAGCTACGAT&#xA;&amp;gt;chr3&#xA;CATCGATCGTACGTACGTAG&#xA;&amp;gt;chr4&#xA;ATCGATCGATCGTACGATCG&#xA;&amp;gt;chr1&#xA;GCATCGTAGCTAGCTACGAT&#xA;&amp;gt;chr3&#xA;CATCGATCGTACGTACGTAG&#xA;&amp;gt;chr4&#xA;ATCGATCGATCGTACGATCG&#xA;&amp;gt;chr5&#xA;CATCGATCGTACGTACGTAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here, as expected, these are two files which have concatenated together. It seems this is usually what is done when one concatenates several FASTAs together in bioinformatics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question1: Is it possible to concatenate these &quot;based on the chromosome&quot;? Here is the desired result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;chr1&#xA;ACTGACTAGCTAGCTAACTG&#xA;GCATCGTAGCTAGCTACGAT&#xA;&amp;gt;chr2&#xA;GCATCGTAGCTAGCTACGAT&#xA;&amp;gt;chr3&#xA;CATCGATCGTACGTACGTAG&#xA;CATCGATCGTACGTACGTAG&#xA;&amp;gt;chr4&#xA;ATCGATCGATCGTACGATCG&#xA;ATCGATCGATCGTACGATCG&#xA;&amp;gt;chr5&#xA;CATCGATCGTACGTACGTAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that these are exceptionally small FASTAs---usually one could be dealing with files of several GB in size. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question2: The meta question here is, are there any standard bioinformatic analyses whereby the &quot;concatenated by chromosome&quot; FASTA would be more useful than the straightforward concatenated FASTA via &lt;code&gt;cat&lt;/code&gt; alone? &lt;/p&gt;&#xA;" OwnerUserId="1770" LastActivityDate="2018-07-02T07:42:38.807" Title="How to concatenate FASTAs with respect to chromosome?" Tags="|fasta|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4563" PostTypeId="2" ParentId="343" CreationDate="2018-06-25T16:59:08.893" Score="2" Body="&lt;p&gt;I know this question is a bit old, but this is still an issue for a lot of researchers not being able to access RepBase. It seems now that the most recent version of RepeatMasker is dependent upon RepBase for full functionality if masking anything other than human (currently DFAM only has human models). I recently discovered a &lt;em&gt;de novo&lt;/em&gt; repeat masking approach called REpeat Detector (Red).  This might be a solution for some looking to mask repeats on a genome assembly for annotation.  The paper is &lt;a href=&quot;https://doi.org/10.1186/s12859-015-0654-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I also then wrote a wrapper around Red to make it a bit more easy to soft-mask a genome, which you can find &lt;a href=&quot;https://github.com/nextgenusfs/redmask&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the limitations with Red is that the repeats are not classified, so they are only identified. You would have to use some of the other tools mentioned above to try to classify them. &lt;/p&gt;&#xA;" OwnerUserId="2988" LastEditorUserId="48" LastEditDate="2018-06-26T10:15:32.810" LastActivityDate="2018-06-26T10:15:32.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4564" PostTypeId="1" CreationDate="2018-06-25T17:24:34.033" Score="6" ViewCount="901" Body="&lt;p&gt;I am fairly new to NGS data analysis and I am struggling to understand the exact relationship between a sequencing lane and an NGS dataset. I should add I don't work in the lab, I only do bioinformatics.&lt;/p&gt;&#xA;&lt;p&gt;I understand the basics of how to make NGS libraries (very briefly: shearing, ligation of adapters, optional PCR, ligation to solid surface, following by sequencing-by-synthesis or other method). What I am struggling with is the visualisation of what happens once you have made your libraries. My questions are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;If you have used adapters containing barcode sequences (that identify each patient), you would presumably end up with one single tube with your libraries in it right?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If so, you then &amp;quot;put&amp;quot; the contents of that tube along with DNA polymerase and nucleotides into one single lane of a flow cell? Your provider would then give you raw FASTQ files that have been identified using this barcode? So you would effectively received one (or 2, depending if it's single/paired-end sequencing) FASTQ file per patient right?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you do not use adapters with barcodes, does that mean you would have to use a sequencing lane per patient? If so, that would vastly increase the price you pay your provider, would it not?&lt;/p&gt;&#xA;&lt;p&gt;Apologies if these are basic questions.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2931" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2024-02-07T15:50:53.800" Title="Relationship between sequencing lane and ngs dataset" Tags="|ngs|illumina|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4565" PostTypeId="2" ParentId="4564" CreationDate="2018-06-25T19:01:20.537" Score="0" Body="&lt;p&gt;Indicies matter, lanes do not, unless you have a screw up on the instrument that messes up some lanes but not all.  I'm not sure if anyone skips indices just because they put only one sample on a lane nowadays...it's valuable to be able to use the indices to catch sample mix-ups.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2018-06-25T19:01:20.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4566" PostTypeId="2" ParentId="4564" CreationDate="2018-06-25T20:08:22.463" Score="4" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;If you have used adapters containing barcode sequences (that identify each patient), you would presumably end up with one single tube with your libraries in it right?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Correct, you have to prep each sample into a library.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If so, you then &amp;quot;put&amp;quot; the contents of that tube along with DNA polymerase and nucleotides into one single lane of a flow cell?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You could, or you could pool it with other samples (assuming they have compatible indices). As &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/2388/swbarnes2&quot;&gt;@swbarnes2&lt;/a&gt; mentioned, you don't see a lot of samples run on a single lane these days. Sequencing centers like to split things across lanes in case one lane fails for some reason or there are bubbles in the flow cell or other technical issues.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Your provider would then give you raw FASTQ files that have been identified using this barcode?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Typically, yes, and if your sample was run on multiple lanes they'll often be merged for you automatically. This isn't always the case. I've seen a number of people receive samples either still split by lane (annoying, but you can just &lt;code&gt;cat&lt;/code&gt; things together) or, much worse, not demultiplexed at all. In the latter case, people often receive 3 (or 4) fastq files per lane, one for each read and another for each index read. It's rare that sequencing facilities do this, but it happens.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If you do not use adapters with barcodes, does that mean you would have to use a sequencing lane per patient? If so, that would vastly increase the price you pay your provider, would it not?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You could sequence a sample per lane. Whether that makes economic sense is up to you to decide.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2024-02-07T15:50:53.800" LastActivityDate="2024-02-07T15:50:53.800" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4567" PostTypeId="2" ParentId="4509" CreationDate="2018-06-26T00:18:00.603" Score="2" Body="&lt;p&gt;@Nathan S. Watson-Haigh's answer does give branch lengths, but NOT parsimony branch lengths, which may be what the OP wanted.  &lt;code&gt;nnls.phylo&lt;/code&gt; is giving distances calculated from a pairwise distance matrix. To get the parsimony branch lengths (i.e. number of changes mapped on to each branch), score the tree against the dataset using &lt;code&gt;acctran&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;treeRatchet &amp;lt;- pratchet(primates)&#xA;treeRatchet.BL &amp;lt;- acctran(treeRatchet,primates)&#xA;plot(treeRatchet.BL,main='Phylogeny (Parsimony branch lengths)')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/c1NYZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/c1NYZ.png&quot; alt=&quot;phylogram with parsimony branch lengths&quot;&gt;&lt;/a&gt;&#xA;Note that the branch lengths in the parsimony phylogram are 'chunkier' than those in the nnls.phylo phylogram -- that's because &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;they are all integers, and &lt;/li&gt;&#xA;&lt;li&gt;they aren't corrected with reference to a model of substitution that considers multiple hits or base composition.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2990" LastEditorUserId="48" LastEditDate="2018-06-26T10:14:29.980" LastActivityDate="2018-06-26T10:14:29.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4568" PostTypeId="2" ParentId="4564" CreationDate="2018-06-26T09:23:37.440" Score="3" Body="&lt;p&gt;It is worth noting that the Illumina NovaSeq and NextSeq address all of the lanes of the flow cell from a single loading point (by default). So on those instruments, you end up with all libraries loaded across all lanes. Therefore for each index you will have L001, L002 (+ L003 and L004 if appropriate) FASTQ files (and R1/R2, again if appropriate). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/77/devon-ryan&quot;&gt;@Devon&lt;/a&gt; mentions, these files can be cat-ted together, but for some processes modelling the lane effect does matter (BQSR, for example).  &lt;/p&gt;&#xA;" OwnerUserId="194" LastActivityDate="2018-06-26T09:23:37.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4569" PostTypeId="2" ParentId="4533" CreationDate="2018-06-26T14:41:21.180" Score="3" Body="&lt;p&gt;An &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4720993/&quot; rel=&quot;nofollow noreferrer&quot;&gt;UpSet Plot&lt;/a&gt; is a useful way of visualising these kind of multi-dimensional overlaps. There's an &lt;a href=&quot;https://cran.r-project.org/web/packages/UpSetR/README.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;R package&lt;/a&gt; for producing them, with a reasonable set of examples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input is a binary matrix indicating set membership:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;         h2    h4    h6...&#xA;GeneA     0     1     0&#xA;GeneB     1     1     0&#xA;GeneC     0     1     1&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the output is a pair of bar charts, showing the size of various sets, with a dot matrix showing the set membership. An example from the UpSetR documentation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(UpSetR)&#xA;mutations &amp;lt;- read.csv(system.file(&quot;extdata&quot;, &quot;mutations.csv&quot;, &#xA;                      package = &quot;UpSetR&quot;), header=T, sep = &quot;,&quot;)&#xA;upset(mutations, sets = c(&quot;PTEN&quot;, &quot;TP53&quot;, &quot;EGFR&quot;, &quot;PIK3R1&quot;, &quot;RB1&quot;),&#xA;      sets.bar.color = &quot;#56B4E9&quot;, order.by = &quot;freq&quot;, &#xA;      empty.intersections = &quot;on&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wSfK4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wSfK4.png&quot; alt=&quot;Example UpSet Plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="194" LastActivityDate="2018-06-26T14:41:21.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4571" PostTypeId="1" CreationDate="2018-06-26T22:01:50.913" Score="5" ViewCount="332" Body="&lt;p&gt;Is there a way to annotate the splice junctions output from tophat/STAR output? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I mean by annotate is can I know if it was involved in an alternative splicing event say skipped exon, MXE or retained intron...?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did some research looks like rMATS does it but we do not have access to the data all we have is output from tophat(.bed file) and STAR(SJ.out) files&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Data available:&#xA;Splice junction files from star and tophat2 for some reason they deleted the bams in the same directory (from what i know they purged all files with size greater than something)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Species: human aligned to GRCh37(ensembl). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Goal : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Annotate splice junctions based on event it is involved an event skipped exon, MXE or A3ss or A5ss.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did some more research and saw that ucsc table browser has UCSC Alt events as an option the data looks like this &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#bin    chrom   chromStart  chromEnd    name    score   strand&#xA;0   chr1    67108547    67109226    atacIntron  0   +&#xA;22  chr1    111148925   111149075   altPromoter 0   -&#xA;35  chr1    220046661   220292759   strangeSplice   0   +&#xA;73  chr1    917423  917573  altPromoter 0   -&#xA;75  chr1    2489907 2491261 retainedIntron  0   +&#xA;93  chr1    21494532    21496156    strangeSplice   0   -&#xA;95  chr1    23720526    23724037    atacIntron  0   -&#xA;100 chr1    28835417    28837404    bleedingExon    0   +&#xA;153 chr1    84259617    84379035    strangeSplice   0   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Not sure how they curate this.&lt;/p&gt;&#xA;" OwnerUserId="926" LastEditorUserId="926" LastEditDate="2018-06-27T14:53:10.080" LastActivityDate="2018-06-27T14:53:10.080" Title="Annotating splice junctions from tophat/STAR output" Tags="|rna-seq|" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4573" PostTypeId="2" ParentId="4562" CreationDate="2018-06-27T11:08:11.157" Score="2" Body="&lt;p&gt;Using the TblToFasta and FastaToTbl scripts &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2651/298&quot;&gt;I have posted before&lt;/a&gt;, combined with classic *nix utils, you can do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ join -a1 -a2 &amp;lt;(FastaToTbl dna1.fa | sort) &amp;lt;(FastaToTbl dna2.fa | sort) | &#xA;    sed 's/ /=/2' | TblToFasta | tr = '\n'&#xA;&amp;gt;chr1 &#xA;ACTGACTAGCTAGCTAACTG&#xA;GCATCGTAGCTAGCTACGAT&#xA;&amp;gt;chr2 &#xA;GCATCGTAGCTAGCTACGAT&#xA;&amp;gt;chr3 &#xA;CATCGATCGTACGTACGTAG&#xA;CATCGATCGTACGTACGTAG&#xA;&amp;gt;chr4 &#xA;ATCGATCGATCGTACGATCG&#xA;ATCGATCGATCGTACGATCG&#xA;&amp;gt;chr5 &#xA;CATCGATCGTACGTACGTAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Explanation&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;FastaToTbl&lt;/code&gt; will put everything on the same line, with the sequence's ID at the beginning of the line, then a tab and then the sequence (no matter how many lines of sequence you have). So, we do this to both files, sort them and pass that as input to &lt;code&gt;join&lt;/code&gt;, which will join its input on lines where the 1st field is the same (by default). The &lt;code&gt;-a1 -a2&lt;/code&gt; means &quot;also print non-matching lines from both files&quot; so we don't miss cases where a sequence is in only one of the two files. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since the &lt;code&gt;join&lt;/code&gt; will add a space between the two joined lines, we deal with that by replacing the second space on each line with a &lt;code&gt;=&lt;/code&gt; (&lt;code&gt;sed 's/ /=/2'&lt;/code&gt;) and then pass the whole thing to &lt;code&gt;TblToFasta&lt;/code&gt; to revert to fasta format. Finally, we use &lt;code&gt;tr&lt;/code&gt; to replace the &lt;code&gt;=&lt;/code&gt; with a newline so you can see where each sequence came from as you show in your example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For longer sequences where you want to avoid sorting the input, you can do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ FastaToTbl dna1.fa dna2.fa | &#xA;    awk '{&#xA;            seq[$1] ? seq[$1]=seq[$1]&quot;\n&quot;$2 : seq[$1]=$2 &#xA;         }&#xA;         END{&#xA;              for(chr in seq){&#xA;                printf &quot;&amp;gt;%s\n%s\n&quot;, chr, seq[chr]&#xA;              }&#xA;         }'&#xA;&amp;gt;chr1&#xA;ACTGACTAGCTAGCTAACTG&#xA;GCATCGTAGCTAGCTACGAT&#xA;&amp;gt;chr2&#xA;GCATCGTAGCTAGCTACGAT&#xA;&amp;gt;chr3&#xA;CATCGATCGTACGTACGTAG&#xA;CATCGATCGTACGTACGTAG&#xA;&amp;gt;chr4&#xA;ATCGATCGATCGTACGATCG&#xA;ATCGATCGATCGTACGATCG&#xA;&amp;gt;chr5&#xA;CATCGATCGTACGTACGTAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Explanation&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Here, the tbl format sequences (see above) are passed to an awk script which does the work:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;seq[$1] ? seq[$1]=seq[$1]&quot;\n&quot;$2 : seq[$1]=$2&lt;/code&gt; : if the 1st field (&lt;code&gt;$1&lt;/code&gt;, the sequence identifier) is already in the array &lt;code&gt;seq&lt;/code&gt;, then set the value of &lt;code&gt;seq[$1]&lt;/code&gt; to the current value (so whatever sequence has been already seen for this ID), a newline and the 2nd field (&lt;code&gt;$2&lt;/code&gt;, the sequence). If it hasn't been seen before, then set the value of &lt;code&gt;seq[$1]&lt;/code&gt; to the 2nd field, the sequence. &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;END{ for(chr in seq){ printf &quot;&amp;gt;%s\n%s\n&quot;, chr, seq[chr]}&lt;/code&gt; : after processing everything, go through each element in the array &lt;code&gt;seq&lt;/code&gt; and print out the data in fasta format.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, that will mean keeping both files in memory which may be a problem for huge sequences. You can avoid that by using temp files:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ FastaToTbl dna1.fa dna2.fa | &#xA;    awk '{ &#xA;            if(!seen[$1]++){&#xA;                print &quot;&amp;gt;&quot;$1 &amp;gt; $1&quot;.tmp&quot;&#xA;            } &#xA;            print $2 &amp;gt;&amp;gt; $1&quot;.tmp&quot;&#xA;        }' &amp;amp;&amp;amp; cat *.tmp &amp;gt; out.fa &amp;amp;&amp;amp; rm *.tmp&#xA;$ cat out.fa &#xA;&amp;gt;chr1&#xA;ACTGACTAGCTAGCTAACTG&#xA;GCATCGTAGCTAGCTACGAT&#xA;&amp;gt;chr2&#xA;GCATCGTAGCTAGCTACGAT&#xA;&amp;gt;chr3&#xA;CATCGATCGTACGTACGTAG&#xA;CATCGATCGTACGTACGTAG&#xA;&amp;gt;chr4&#xA;ATCGATCGATCGTACGATCG&#xA;ATCGATCGATCGTACGATCG&#xA;&amp;gt;chr5&#xA;CATCGATCGTACGTACGTAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Explanation&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;This is the same basic idea as the script above, only instead of storing the information in an array, we print it as it comes into temp files. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;if(!seen[$1]++){print &quot;&amp;gt;&quot;$1 &amp;gt; $1&quot;.tmp&quot;}&lt;/code&gt; : if this is the first time we've seen this ID, print it as a fasta header into the file &lt;code&gt;$1&quot;.tmp&quot;&lt;/code&gt; (e.g. &lt;code&gt;chr1.tmp&lt;/code&gt;).&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;print $2 &amp;gt;&amp;gt; $1&quot;.tmp&quot;&lt;/code&gt; append the current sequence to the corresponding temp file.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Once the temp files have been created, we concatenate them (&lt;code&gt;cat *.tmp &amp;gt; out.fa&lt;/code&gt;) and delete them (&lt;code&gt;rm *tmp&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;As for why you would want to do this, an obvious example is reconstructing a reference genome from short sequences. You would want all the sequences of the same schromosome together. Of course, you would also need to assemble them in the right order, which you aren't doing here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you might want to concatenate them this way so as to avoid having the extra, unneeded lines (&lt;code&gt;&amp;gt;chrN&lt;/code&gt;) which would make your file larger. Or you would want to be able to retrieve all sequences with the same ID at once. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I don't really know why you would want to do it. You're the one asking for it, after all! :)&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-07-02T07:42:38.807" LastActivityDate="2018-07-02T07:42:38.807" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4574" PostTypeId="1" AcceptedAnswerId="4588" CreationDate="2018-06-27T14:10:12.520" Score="3" ViewCount="396" Body="&lt;p&gt;I am using &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/chromPlot.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;chromPlot&lt;/a&gt; to visualise the genome of &lt;em&gt;C. elegans&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;library(chromPlot)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have created the following data frame with the lengths of &lt;em&gt;C. elegans&lt;/em&gt; chromosomes.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Chrom Start      End    Name&#xA;1     1     0 15072434 contigs&#xA;2     2     0 15279421 contigs&#xA;3     3     0 13783801 contigs&#xA;4     4     0 17493829 contigs&#xA;5     6     0 20924180 contigs&#xA;6     7     0 17718942 contigs&#xA;7     5     0    13794 contigs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I use &lt;code&gt;chromPlot(dataframe)&lt;/code&gt;, I get a plot with 4 chromosomes and the following output/error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   1 segment clusters.&#xA;   1 segment clusters.&#xA;   1 segment clusters.&#xA;   1 segment clusters.&#xA;   1 segment clusters.&#xA;   1 segment clusters.&#xA;   1 segment clusters.&#xA;Chrom 1 : 15072434 bp&#xA;Chrom 2 : 15279421 bp&#xA;Chrom 3 : 13783801 bp&#xA;Chrom 4 : 17493829 bp&#xA;Error in seq.default(minval, maxval, length = howmany) : &#xA;  'from' must be of length 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does anybody know why I get this error?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't get this error when I use the data frame provided by the package with human segments. The structure of the data frame is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;'data.frame':   457 obs. of  4 variables:&#xA; $ Chrom: chr  &quot;1&quot; &quot;1&quot; &quot;1&quot; &quot;1&quot; ...&#xA; $ Start: int  124535434 121535434 3845268 13219912 17125658 29878082 120697156 120936695 121485434 142731022 ...&#xA; $ End  : int  142535434 124535434 3995268 13319912 17175658 30028082 120747156 121086695 121535434 142781022 ...&#xA; $ Name : chr  &quot;heterochromatin&quot; &quot;centromere&quot; &quot;contig&quot; &quot;contig&quot; ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Compared to my data frame:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;'data.frame':   7 obs. of  4 variables:&#xA; $ Chrom: chr  &quot;1&quot; &quot;2&quot; &quot;3&quot; &quot;4&quot; ...&#xA; $ Start: int  0 0 0 0 0 0 0&#xA; $ End  : int  15072434 15279421 13783801 17493829 20924180 17718942 13794&#xA; $ Name : chr  &quot;contigs&quot; &quot;contigs&quot; &quot;contigs&quot; &quot;contigs&quot; ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am on MacOS X.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Update&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The 20th line of the package-provided hg_gap data frame is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   Chrom Start   End     Name&#xA;20     1     0 10000 telomere&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that the Start is 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also tried changing the start from 0 to 1. The error stays the same.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Update 2&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Sorting the data frame by chromosome name does not improve the situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The traceback of the error is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;6: stop(&quot;'from' must be of length 1&quot;)&#xA;5: seq.default(minval, maxval, length = howmany)&#xA;4: seq(minval, maxval, length = howmany)&#xA;3: pretty_ticks(minlab, maxlab, 2, ...)&#xA;2: draw.scale(y = 0.8 * chr.length[[chrom]] + 0.2 * xylims[3], minval = margin, &#xA;       maxval = maxGeneCount, minlab = annot1_plot_range[1], maxlab = annot1_plot_range[2], &#xA;       lwd = 4, col = colAnnot1, cex = cex, title = scale.title)&#xA;1: chromPlot::chromPlot(gaps)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have emailed the maintainer of the package Karen Orostica with a link to this post.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="48" LastEditDate="2018-09-04T10:09:23.473" LastActivityDate="2018-09-04T10:09:23.473" Title="Error in seq.default in chromPlot" Tags="|r|bioconductor|visualization|chromplot|" AnswerCount="1" CommentCount="14" ContentLicense="CC BY-SA 4.0" />
  <row Id="4575" PostTypeId="1" CreationDate="2018-06-27T16:56:17.273" Score="6" ViewCount="401" Body="&lt;p&gt;After reading the GIAB paper in &lt;a href=&quot;https://www.biorxiv.org/content/early/2018/05/25/281006&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/early/2018/05/25/281006&lt;/a&gt; and its Figure 1, I am still having trouble understanding the data inside the GIAB VCF file for HG001 (HG001_GRCh38_GIAB_highconf_CG-IllFB-IllGATKHC-Ion-10X-SOLID_CHROM1-X_v.3.3.2_all.vcf). In particular, I need to understand the information that led to a FILTER value of PASS for some variants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first case that I would like to discuss is the following variant:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    110176467   .   A   G   50  PASS    platforms=3;platformnames=Illumina,CG,10X;datasets=3;datasetnames=HiSeqPE300x,CGnormal,10XChromium;callsets=4;callsetnames=HiSeqPE300xGATK,CGnormal,HiSeqPE300xfreebayes,10XGATKhaplo;datasetsmissingcall=IonExome,SolidPE50x50bp,SolidSE75bp;callable=CS_HiSeqPE300xGATK_callable,CS_CGnormal_callable,CS_HiSeqPE300xfreebayes_callable    GT:PS:DP:ADALL:AD:GQ    0/1:.:512:114,106:145,148:760&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Translating the information in the line to English, this is what I get: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;We considered data from the following sequencing technologies: Illumina,CG,10X,IonExome,SolidPE50x50bp, and SolidSE75bp. &lt;/li&gt;&#xA;&lt;li&gt;Of those, some of them (IonExome,SolidPE50x50bp,SolidSE75bp) were missing the call, so we discarded them. &lt;/li&gt;&#xA;&lt;li&gt;Therefore, the data for the variant is going to come from 3 platforms (Illumina,CG,10X), 3 datasets (HiSeqPE300x,CGnormal,10XChromium), and 4 callsets (HiSeqPE300xGATK,CGnormal,HiSeqPE300xfreebayes,10XGATKhaplo), because we analyzed the data from HiSeqPE300x with GATK and also with Freebayes.&lt;/li&gt;&#xA;&lt;li&gt;However, the callsets from CS_HiSeqPE300xGATK,CS_CGnormal, and CS_HiSeqPE300xfreebayes had this call in a region with low coverage of high MQ reads.&lt;/li&gt;&#xA;&lt;li&gt;Across all the platforms, DP=512&lt;/li&gt;&#xA;&lt;li&gt;ADALL (Values 114,106) is coming from “all&quot; the datasets. Are those “all” the set (HiSeqPE300x + Cgnormal + 10XChromium), or the set (Illumina + CG + 10X + IonExome + SolidPE50x50bp + SolidSE75bp).&lt;/li&gt;&#xA;&lt;li&gt;AD (Values 145,148) is coming from the same “all” datasets as ADALL, but they are unfiltered, so 145&gt;114 and 148&gt;106.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All this make sense, but I would like to know what case of Figure 1c was applied to arrive at the PASS value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second variant that I would like to mention is this one:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    5705293 .   T   C   50  PASS    platforms=3;platformnames=Illumina,CG,10X;datasets=3;datasetnames=HiSeqPE300x,CGnormal,10XChromium;callsets=4;callsetnames=HiSeqPE300xGATK,CGnormal,HiSeqPE300xfreebayes,10XGATKhaplo;datasetsmissingcall=IonExome,SolidPE50x50bp,SolidSE75bp;callable=CS_HiSeqPE300xGATK_callable,CS_CGnormal_callable,CS_HiSeqPE300xfreebayes_callable;filt=CS_HiSeqPE300xGATK_filt   GT:PS:DP:ADALL:AD:GQ    0/1:.:599:121,139:49,28:627&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Again, my translation would be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;We considered data from the following sequencing technologies: Illumina,CG,10X,IonExome,SolidPE50x50bp, and SolidSE75bp. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Of those, some of them (IonExome,SolidPE50x50bp,SolidSE75bp) were missing the call, so we discarded them. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Therefore, the data for the variant is going to come from 3 platforms (Illumina,CG,10X), 3 datasets (HiSeqPE300x,CGnormal,10XChromium), and 4 callsets (HiSeqPE300xGATK,CGnormal,HiSeqPE300xfreebayes,10XGATKhaplo), because we analyzed the data from HiSeqPE300x with GATK and also with Freebayes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;However, the callsets from CS_HiSeqPE300xGATK,CS_CGnormal, and CS_HiSeqPE300xfreebayes had this call in a region with low coverage of high MQ reads.&lt;/li&gt;&#xA;&lt;li&gt;Also, the data from CS_HiSeqPE300xGATK_filt was filtered/discarded&lt;/li&gt;&#xA;&lt;li&gt;Across all the platforms, DP=599&lt;/li&gt;&#xA;&lt;li&gt;ADALL (Values 121,139) is coming from “all&quot; the datasets. I assume (Cgnormal + 10XChromium), because CS_HiSeqPE300xGATK_filt was filtered. &lt;/li&gt;&#xA;&lt;li&gt;AD (Values 49,28) are unfiltered.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But notice that 49&amp;lt;121 and 28&amp;lt;139. How come that there are less unfiltered reads (AD)? Should not be the case that AD &gt; ADALL for all instances? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don’t know how to reconcile the two variants. Aren’t they providing opposite information in the AD and ADALL fields? Which specific datasets and callsets are involved in getting AD and ADALL for each variant?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also appreciate answers to a couple of other questions: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Why are these two variants a PASS if there are 3 “callable” datasets in regions of low coverage? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Did the variants follow an arbitration process? I guess not, if the INFO flag “arbitrated” is missing.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For reference, here are the descriptions in the header of the VCF file that I have been using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##INFO=&amp;lt;ID=platforms,Number=1,Type=Integer,Description=&quot;Number of different platforms for which at least one callset called this genotype, whether filtered or not&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=platformnames,Number=.,Type=String,Description=&quot;Names of platforms for which at least one callset called this genotype, whether filtered or not&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=datasets,Number=1,Type=Integer,Description=&quot;Number of different datasets for which at least one callset called this genotype, whether filtered or not&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=datasetnames,Number=.,Type=String,Description=&quot;Names of datasets for which at least one callset called this genotype, whether filtered or not&quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=ADALL,Number=R,Type=Integer,Description=&quot;Net allele depths across all datasets&quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=AD,Number=R,Type=Integer,Description=&quot;Net allele depths across all unfiltered datasets with called genotype&quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=DP,Number=1,Type=Integer,Description=&quot;Total read depth summed across all datasets, excluding MQ0 reads&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=datasetsmissingcall,Number=.,Type=Integer,Description=&quot;Names of datasets that are missing a call or have an incorrect call at this location, and the high-confidence call is a variant&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=callable,Number=.,Type=String,Description=&quot;List of callsets that had this call in a region with low coverage of high MQ reads.&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=filt,Number=.,Type=String,Description=&quot;List of callsets that had this call filtered.&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=arbitrated,Number=1,Type=String,Description=&quot;TRUE if callsets had discordant calls so that arbitration was needed.&quot;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2998" LastEditorUserId="96" LastEditDate="2019-02-27T20:11:26.870" LastActivityDate="2020-04-23T17:49:55.777" Title="Meaning of the FORMAT fields of the VCF file coming from GIAB project" Tags="|vcf|ngs|file-formats|genomics|human-genome|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4576" PostTypeId="2" ParentId="4534" CreationDate="2018-06-27T17:41:48.223" Score="2" Body="&lt;p&gt;I like using &lt;code&gt;awk&lt;/code&gt; when there's any kind of lookup involved: something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk 'FNR==NR {&#xA;    hash[&quot;&amp;gt;&quot;$1]=$2;&#xA;    next;&#xA;}&#xA;&#xA;if ($1 ~ /^&amp;gt;/) {&#xA;    print &quot;&amp;gt;&quot;hash[$1];&#xA;} else {&#xA;    print $1;&#xA;}' keyValues.txt mySequence.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically the &lt;code&gt;FNR==NR&lt;/code&gt; check tells awk to work only on the first file (useful primer &lt;a href=&quot;https://stackoverflow.com/questions/32481877/what-is-nr-fnr-in-awk&quot;&gt;here&lt;/a&gt;), &lt;code&gt;keyValues.txt&lt;/code&gt; and create an association for each key (eg. &lt;code&gt;chr_I&lt;/code&gt;) with its value (eg. &lt;code&gt;I&lt;/code&gt;). The rest of the code after the &lt;code&gt;next&lt;/code&gt; works only on &lt;code&gt;mySequence.fasta&lt;/code&gt;, printing out the lookup value only if the line is a fasta header, as checked by the &lt;code&gt;$1 ~ /^&amp;gt;/&lt;/code&gt; condition. &lt;/p&gt;&#xA;" OwnerUserId="541" LastActivityDate="2018-06-27T17:41:48.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4577" PostTypeId="2" ParentId="4575" CreationDate="2018-06-27T21:35:13.770" Score="2" Body="&lt;p&gt;The filtering criteria should be defined in the header of the VCF file. Can you include the header from your VCF file?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.internationalgenome.org/wiki/Analysis/vcf4.0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VCF Format and example header:&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FILTERs that have been applied to the data should be described as follows:&#xA;&#xA;##FILTER=&amp;lt;ID=ID,Description=”description”&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;header:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.0&#xA;##fileDate=20090805&#xA;##source=myImputationProgramV3.1&#xA;##reference=1000GenomesPilot-NCBI36&#xA;##phasing=partial&#xA;##INFO=&amp;lt;ID=NS,Number=1,Type=Integer,Description=&quot;Number of Samples With Data&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=DP,Number=1,Type=Integer,Description=&quot;Total Depth&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=AF,Number=.,Type=Float,Description=&quot;Allele Frequency&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=AA,Number=1,Type=String,Description=&quot;Ancestral Allele&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=DB,Number=0,Type=Flag,Description=&quot;dbSNP membership, build 129&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=H2,Number=0,Type=Flag,Description=&quot;HapMap2 membership&quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=q10,Description=&quot;Quality below 10&quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=s50,Description=&quot;Less than 50% of samples have data&quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&quot;Genotype&quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GQ,Number=1,Type=Integer,Description=&quot;Genotype Quality&quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=DP,Number=1,Type=Integer,Description=&quot;Read Depth&quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=HQ,Number=2,Type=Integer,Description=&quot;Haplotype Quality&quot;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="964" LastActivityDate="2018-06-27T21:35:13.770" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4578" PostTypeId="1" AcceptedAnswerId="4595" CreationDate="2018-06-27T23:03:18.513" Score="2" ViewCount="6631" Body="&lt;p&gt;I am bench-marking the performance of &lt;a href=&quot;https://github.com/bigdatagenomics/adam&quot; rel=&quot;nofollow noreferrer&quot;&gt;ADAM&lt;/a&gt;/&lt;a href=&quot;https://github.com/bigdatagenomics/avocado&quot; rel=&quot;nofollow noreferrer&quot;&gt;Avocado&lt;/a&gt; on NGS whole-genome sequencing datasets. I am trying to repeat the tests reported in Frank Nofthat's (author of &lt;a href=&quot;https://github.com/bigdatagenomics/adam&quot; rel=&quot;nofollow noreferrer&quot;&gt;ADAM&lt;/a&gt; and &lt;a href=&quot;https://github.com/bigdatagenomics/avocado&quot; rel=&quot;nofollow noreferrer&quot;&gt;Avocado&lt;/a&gt;) &lt;a href=&quot;https://www2.eecs.berkeley.edu/Pubs/TechRpts/2017/EECS-2017-204.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;thesis&lt;/a&gt;, however, I am running into an issue acquiring the dbSNP database to use for Base Quality Score Recalibration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Base Quality Score Recalibration command (page 128-129):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;adam-submit -- transformAlignments aln.adam bqsr.adam -recalibrate_base_qualities -known_snps dbsnp.adam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;requires &lt;code&gt;dbsnp.adam&lt;/code&gt;, which is described as &quot;the dnSNP database&quot; (converted to ADAM format) with reference to &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/11125122&quot; rel=&quot;nofollow noreferrer&quot;&gt;the NCBI database of genetic variation&lt;/a&gt;. This &quot;database&quot; can be found at &lt;a href=&quot;ftp://ftp.ncbi.nih.gov/snp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this FTP site&lt;/a&gt; however I do not know which parts of these files to download and convert to ADAM in order to perform this step of the transformation pipeline. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, in order to use this genetic variation databaes do I need to make sure that the reference genome of &quot;known variants&quot; is the same genome that the sequencing data was aligned to?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="1196" LastEditorUserId="1196" LastEditDate="2018-06-27T23:15:53.827" LastActivityDate="2018-06-29T13:54:08.323" Title="How to download dbSNP database?" Tags="|adam|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4579" PostTypeId="1" AcceptedAnswerId="4582" CreationDate="2018-06-27T23:07:07.640" Score="3" ViewCount="133" Body="&lt;p&gt;I would like to convert BAM and VCF files to &lt;a href=&quot;https://github.com/bigdatagenomics/adam&quot; rel=&quot;nofollow noreferrer&quot;&gt;ADAM&lt;/a&gt; format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I do that?&lt;/p&gt;&#xA;" OwnerUserId="1196" LastActivityDate="2018-06-28T01:40:33.247" Title="How to convert files to ADAM format?" Tags="|sam|vcf|adam|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4580" PostTypeId="1" CreationDate="2018-06-27T23:46:14.870" Score="1" ViewCount="5282" Body="&lt;p&gt;This is probably a really quick/simple fix but I am a Noob to R programming/programming in general. I am trying to create a for loop that runs the test multiple times with different FDR values (5,10,20). How do I make it run 3 times with different values and then produce 3 different datasets with names like... &lt;code&gt;atac.glmtop.males.opening.fdr5&lt;/code&gt;, &lt;code&gt;atac.glmtop.males.opening.fdr10&lt;/code&gt;, &lt;code&gt;atac.glmtop.males.opening.fdr20&lt;/code&gt;? Also is there a way to make the &lt;code&gt;write.table&lt;/code&gt; function work so that it exports files with the fdr5, fdr10, fdr20 in the name?&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;fdr.nums = c(&quot;5&quot;,&quot;10&quot;,&quot;20&quot;)&#xA;fdr.nums.vector = c(5,10,20)&#xA;&#xA;for (i in 1:length(fdr.list)){&#xA;  atac.glmtop.males.opening.fdr[i] &amp;lt;- topTags(&#xA;    atac.glmfit.males,&#xA;    n = nrow(atac.dge$counts),&#xA;    sort.by = &quot;none&quot;,&#xA;    adjust.method = &quot;fdr&quot;&#xA;  )$table %&amp;gt;%&#xA;  tibble::rownames_to_column(&quot;peakco&quot;) %&amp;gt;%&#xA;  filter(logFC &amp;lt; 0) %&amp;gt;%&#xA;  filter(FDR &amp;lt; (fdr.vector[i]/100)) %&amp;gt;% &#xA;  merge(.,annotated.atac.pbmc.narrow, by=&quot;peakco&quot; ) %&amp;gt;%&#xA;  tibble::column_to_rownames(&quot;peakco&quot;)&#xA;&#xA;  atac.glmtop.males.opening.fdr[i]&#xA;&#xA;  write.table(&#xA;    atac.glmtop.males.fdr[i].opening[,c(&quot;chr&quot;, &quot;start&quot;,&quot;end&quot;)], &#xA;    file = paste0(&quot;~/Downloads/sexM.age.groupHO.fdr&quot;,i, &quot;.opening.bed&quot;), &#xA;    sep = &quot;\t&quot;,&#xA;    quote = FALSE,&#xA;    row.names = FALSE,&#xA;    col.names = FALSE&#xA;  )&#xA;  write.table(&#xA;    atac.glmtop.males.fdr[i].opening[&quot;GeneName&quot;],&#xA;    file = &quot;~/Downloads/sexM.age.groupHO.fdr&quot;+ i + &quot;.opening.GeneNames.txt&quot;,&#xA;    sep = &quot;\t&quot;,&#xA;    quote = FALSE,&#xA;    row.names = FALSE,&#xA;    col.names = FALSE&#xA;  )&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3001" LastEditorUserId="400" LastEditDate="2018-06-28T06:54:58.070" LastActivityDate="2018-06-28T06:54:58.070" Title="How do I create a for loop to filter through different FDR values?" Tags="|r|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4581" PostTypeId="2" ParentId="4580" CreationDate="2018-06-28T00:25:42.373" Score="2" Body="&lt;p&gt;For loops in most languages can be done in a variety of ways. What you have currently coded, loops over values 1 to 3 by using&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;for (i in 1:length(fdr.list)){&#xA;...&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, you probably want to use a different form and loop over each value in that vector using something more like:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;for (fdr in fdr.nums.vector){&#xA;...&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This way you have convenient access to the FDR value in each loop via the &lt;code&gt;fdr&lt;/code&gt; variable. Which you can then use in constructing your output filenames. Thus your code would become something like:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;fdr.nums.vector = c(5,10,20)&#xA;&#xA;for (fdr in fdr.nums.vector){&#xA;  atac.glmtop.males.opening.fdr[fdr] &amp;lt;- topTags(&#xA;    atac.glmfit.males,&#xA;    n = nrow(atac.dge$counts),&#xA;    sort.by = &quot;none&quot;,&#xA;    adjust.method = &quot;fdr&quot;&#xA;  )$table %&amp;gt;%&#xA;  tibble::rownames_to_column(&quot;peakco&quot;) %&amp;gt;%&#xA;  filter(logFC &amp;lt; 0) %&amp;gt;%&#xA;  filter(FDR &amp;lt; (fdr/100)) %&amp;gt;% &#xA;  merge(.,annotated.atac.pbmc.narrow, by=&quot;peakco&quot; ) %&amp;gt;%&#xA;  tibble::column_to_rownames(&quot;peakco&quot;)&#xA;&#xA;  atac.glmtop.males.opening.fdr[fdr]&#xA;&#xA;  write.table(&#xA;    atac.glmtop.males.fdr[fdr].opening[,c(&quot;chr&quot;, &quot;start&quot;,&quot;end&quot;)], &#xA;    file = paste0(&quot;~/Downloads/sexM.age.groupHO.fdr&quot;,fdr, &quot;.opening.bed&quot;), &#xA;    sep = &quot;\t&quot;,&#xA;    quote = FALSE,&#xA;    row.names = FALSE,&#xA;    col.names = FALSE&#xA;  )&#xA;  write.table(&#xA;    atac.glmtop.males.fdr[fdr].opening[&quot;GeneName&quot;],&#xA;    file = &quot;~/Downloads/sexM.age.groupHO.fdr&quot;+ fdr + &quot;.opening.GeneNames.txt&quot;,&#xA;    sep = &quot;\t&quot;,&#xA;    quote = FALSE,&#xA;    row.names = FALSE,&#xA;    col.names = FALSE&#xA;  )&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;Advancing From &lt;code&gt;for&lt;/code&gt; Loops to &lt;code&gt;apply&lt;/code&gt;&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;As a beginner, &lt;code&gt;for&lt;/code&gt; loops seem quite logical and easy to understand. However, R has a more powerful way to achieve the same thing through what is called &quot;vectorisation&quot; using the &lt;code&gt;apply&lt;/code&gt; function or one of it's relatives &lt;code&gt;lapply&lt;/code&gt;, &lt;code&gt;sapply&lt;/code&gt; etc. Once you have understood and mastered &lt;code&gt;for&lt;/code&gt; loops, you should take a look at &lt;code&gt;apply&lt;/code&gt;, especially if you ever need to do a &lt;code&gt;for&lt;/code&gt; loop many times or you find yourself doing nested &lt;code&gt;for&lt;/code&gt; loops. &lt;code&gt;R&lt;/code&gt; is notoriously slow with &lt;code&gt;for&lt;/code&gt; loops and &lt;code&gt;apply&lt;/code&gt; will likely be many times faster (potentially orders of magnitude). Here are some links for when you are ready to advance from &lt;code&gt;for&lt;/code&gt; loops to &lt;code&gt;vectorisation&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.r-bloggers.com/how-to-write-the-first-for-loop-in-r/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.r-bloggers.com/how-to-write-the-first-for-loop-in-r/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.r-bloggers.com/using-apply-sapply-lapply-in-r/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.r-bloggers.com/using-apply-sapply-lapply-in-r/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="400" LastActivityDate="2018-06-28T00:25:42.373" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4582" PostTypeId="2" ParentId="4579" CreationDate="2018-06-28T01:40:33.247" Score="2" Body="&lt;p&gt;These transformations are quite simple with the &lt;code&gt;adam-submit&lt;/code&gt; script packaged with ADAM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Transformation of BAM files&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;adam-submit -- transformAlignments sample.bam sample.alignments.adam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Transformation of VCF files&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;adam-submit -- transformVariants sample.vcf.gz sample.variants.adam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1196" LastActivityDate="2018-06-28T01:40:33.247" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4583" PostTypeId="1" CreationDate="2018-06-28T04:52:56.413" Score="1" ViewCount="52" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; I've obtained RNAseq datasets from ENCODE and I'm looking to identify external variables contributing to sequence variance among these samples.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;External variables&lt;/strong&gt; For example, clustering effects on the date the sequencing was performed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt; How do I find out the precise date which each sample went through the sequencing machine?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;ENCODE databases are &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/encode/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here on NCBI&lt;/a&gt;&#xA;The &lt;a href=&quot;https://www.encodeproject.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENCODE website is here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="823" LastEditorUserId="3848" LastEditDate="2022-03-26T14:38:50.750" LastActivityDate="2022-03-26T14:38:50.750" Title="Find the date that sequencing was performed" Tags="|rna-seq|ngs|public-databases|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4584" PostTypeId="1" CreationDate="2018-06-28T10:10:39.347" Score="1" ViewCount="87" Body="&lt;p&gt;I read a paper about  (&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0002929712006398&quot; rel=&quot;nofollow noreferrer&quot;&gt;whole genome sequencing analysis of 100 Southeast Asian Malays&lt;/a&gt;. It says &quot;Prior to library preparation, each sample was genotyped on the Illumina HumanOmni1-Quad as an initial DNA QC and for in-house assessment of gender and familial relations&quot;. I can hardly understand how Illumina HumanOmni1-Quad worked here for initial DNA QC and for in-house assessment of gender and familial relations. Could you please kindly explain to me a bit about this sentence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many thank in advance&lt;/p&gt;&#xA;" OwnerUserId="2835" LastEditorUserId="964" LastEditDate="2018-06-28T21:28:18.620" LastActivityDate="2018-07-02T19:48:21.403" Title="Genotyping with Illumina HumanOmni1-Quad before whole genome sequencing" Tags="|illumina|genome-sequencing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4585" PostTypeId="1" CreationDate="2018-06-28T10:12:18.657" Score="1" ViewCount="1234" Body="&lt;p&gt;I am working on ION Torrent S5 XL and I am facing to an issue with VCF output generated by the variant caller of thermofisher. It stating on VCF 4.1 that Phred Quality Score is on a range of 0 to 255. In my VCF I have values over than 255. Do you know if there is a way to transform these values?&lt;/p&gt;&#xA;" OwnerUserId="3004" LastActivityDate="2019-02-07T08:51:06.240" Title="High Phred Quality score VCF file" Tags="|vcf|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4586" PostTypeId="1" AcceptedAnswerId="4605" CreationDate="2018-06-28T10:29:24.533" Score="3" ViewCount="1695" Body="&lt;p&gt;I have about 10,000 genome files all named by either refseq or genbank accession number, do you know if it's possible to convert these numbers to the corresponding NCBI taxon ID or species?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GCA_000005845.2 to 79781&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the case of &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly/GCF_000005845.2&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;E.coli&quot;&gt;&lt;em&gt;E.coli&lt;/em&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The file names look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GCF_000337855.1_ASM33785v1_protein.faa.gz&#xA;GCF_000091125.1_ASM9112v1_protein.faa.gz&#xA;GCF_000184535.1_ASM18453v1_protein.faa.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My operating system is Ubuntu 16.04&lt;/p&gt;&#xA;" OwnerUserId="2965" LastEditorUserId="2965" LastEditDate="2018-07-02T10:44:46.370" LastActivityDate="2023-07-03T23:35:08.380" Title="Retrieving NCBI Taxa IDs from refseq or GenBank assembly accession" Tags="|python|phylogenetics|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4587" PostTypeId="2" ParentId="3335" CreationDate="2018-06-28T13:20:43.110" Score="2" Body="&lt;p&gt;So this is connected to how QMEAN is implemented as a server. It also could be true to many bioinformatics resources so I think this is important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Posting to QMEAN &quot;submit&quot; triggers the job that is processed on our HPC cluster. That takes time and in the meantime JSON response just tells you that the job is QUEUEING/RUNNING or when it is ready it tells you COMPLETED. The answer suggested by you is the correct assuming you wish to wait for the job completion (which might take quite a lot).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So just check in while loop when the status is COMPLETED and fetch the result eg.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import time&#xA;import json&#xA;import requests&#xA;def qmean_score(download_url):&#xA;    # Request the resource&#xA;    zscore = None&#xA;    while True:&#xA;        r = requests.get(download_url, verify=True)&#xA;&#xA;        scores = json.dumps(r.json(), indent=4, sort_keys=True)&#xA;        jdata = json.loads(scores)&#xA;        if jdata[&quot;status&quot;] == &quot;COMPLETED&quot;:&#xA;            zscore = jdata['input_data']['models'][0]['global_scores']['qmean4']['zscore']&#xA;            break&#xA;        else:&#xA;            time.sleep(1000)&#xA;    return zscore&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3005" LastActivityDate="2018-06-28T13:20:43.110" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4588" PostTypeId="2" ParentId="4574" CreationDate="2018-06-28T13:33:42.693" Score="2" Body="&lt;p&gt;Using this data frame:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gaps&#xA;  Chrom Start      End    Name&#xA;1     I     0 15072434 contigs&#xA;2    II     0 15279421 contigs&#xA;3   III     0 13783801 contigs&#xA;4    IV     0 17493829 contigs&#xA;5     V     0 20924180 contigs&#xA;6     X     0 17718942 contigs&#xA;7 MtDNA     0    13794 contigs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and this command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chromPlot(bands = gaps)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I could obtain this plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/M4NiR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/M4NiR.png&quot; alt=&quot;Karyotype of *C. elegans*&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then, if you want to add some segments for a subset of bases, you can do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gaps2 &amp;lt;- gaps[1:2, ]&#xA;gaps2$End[2] &amp;lt;- 7000000&#xA;&#xA;chromPlot(bands = gaps, segment = gaps2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and obtain the following plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MNvzI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MNvzI.png&quot; alt=&quot;*C. elegans* karyotype with segments&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more satisfactory answer could put segments of different colour on the chromosome, so bands but this is the best I've got so far.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-06-28T13:33:42.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4589" PostTypeId="1" AcceptedAnswerId="4592" CreationDate="2018-06-28T16:18:35.890" Score="5" ViewCount="393" Body="&lt;p&gt;I have a &lt;em&gt;de novo&lt;/em&gt; assembly using both multiple SRA and locally sequenced transcriptomes. I started with 270M PE reads from 9 tissues. Here are the assembly stats generated with &lt;code&gt;TrinityStats.pl&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;################################&#xA;## Counts of transcripts, etc.&#xA;################################&#xA;Total trinity 'genes':  543668&#xA;Total trinity transcripts:      1111089&#xA;Percent GC: 33.77&#xA;&#xA;########################################&#xA;Stats based on ALL transcript contigs:&#xA;########################################&#xA;&#xA;        Contig N10: 2117&#xA;        Contig N20: 1324&#xA;        Contig N30: 945&#xA;        Contig N40: 714&#xA;        Contig N50: 561&#xA;&#xA;        Median contig length: 344&#xA;        Average contig: 496.17&#xA;        Total assembled bases: 551293315&#xA;&#xA;&#xA;#####################################################&#xA;## Stats based on ONLY LONGEST ISOFORM per 'GENE':&#xA;#####################################################&#xA;&#xA;        Contig N10: 2172&#xA;        Contig N20: 1333&#xA;        Contig N30: 925&#xA;        Contig N40: 681&#xA;        Contig N50: 523&#xA;&#xA;        Median contig length: 323&#xA;        Average contig: 473.63&#xA;        Total assembled bases: 257499729&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, the total Trinity 'genes' number is very high. I suspect that this is due to a high biological variability of the transcriptomes (coming from organisms sampled in Italy, China, Spain...). &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How can I assess whether the problem is really the biological variability?&lt;/li&gt;&#xA;&lt;li&gt;How can I reduce this number? I'd rather not downsample the reads. It looks like &lt;a href=&quot;https://github.com/weizhongli/cdhit/wiki&quot; rel=&quot;nofollow noreferrer&quot;&gt;CD-Hit&lt;/a&gt; might do what I'm looking for...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1637" LastEditorUserId="1637" LastEditDate="2018-06-28T16:26:02.610" LastActivityDate="2018-06-29T00:03:45.953" Title="Reduce number of transcripts in a highly variable de novo transcriptome assembly" Tags="|assembly|transcriptome|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4590" PostTypeId="2" ParentId="4589" CreationDate="2018-06-28T16:25:51.520" Score="0" Body="&lt;p&gt;Seems to me that your &quot;genes&quot; are very short. What sort organism (prokaryote, lower eukaryote, higher eukaryote) is this? I would look to see if your &quot;genes&quot; contained ORFs, and only keep those with sufficicntly long CDS. You will miss uORFs and ORF fragments this way, but you'd have much more confidence in what you did find. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-06-28T16:25:51.520" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4591" PostTypeId="1" CreationDate="2018-06-28T18:07:18.050" Score="2" ViewCount="229" Body="&lt;p&gt;I am trying to build gene co-expression networks with WGCNA by combining in-house and publicly available RNA-seq datasets. I am interested in identifying gene networks associated with different environmental conditions. I used SVA to adjust for batch effects and &quot;protected&quot; different environmental conditions in the model as I don't want to lose information about them. However, when I used the &quot;batch-adjusted&quot; data to create networks, all the scale-free topology indices were negative (max = -0.05).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried generating the networks with softpower = 16 as it is suggested for sample sizes 20-30. However, only five networks were generated which is far less than I expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What might be the reason for getting negative scale-free toplogy indices? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can combining studies from different environmental conditions cause the networks to be not biologically meaningful? &lt;/p&gt;&#xA;" OwnerUserId="3006" LastActivityDate="2019-04-12T19:02:35.477" Title="Negative scale-free topology" Tags="|rna-seq|networks|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4592" PostTypeId="2" ParentId="4589" CreationDate="2018-06-29T00:03:45.953" Score="3" Body="&lt;p&gt;As you've suggested, CD-HIT works for reducing transcript numbers. We used a mixture of expression-based filtering and CD-HIT for reducing transcript counts for our genome-guided transcriptome assembly. This reduced numbers by a lot, without much change in BUSCO scores:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Map RNA-seq reads to Trinity-generated transcripts using Salmon&lt;/li&gt;&#xA;&lt;li&gt;Use the expression of BUSCO genes to set a credible signal cutoff&lt;/li&gt;&#xA;&lt;li&gt;Subset transcripts based on this threshold [for us it was 50 counts]&lt;/li&gt;&#xA;&lt;li&gt;Identify the longest Met to Stop open reading frame for each transcript (may not be a good idea in all cases)&lt;/li&gt;&#xA;&lt;li&gt;Run cdhit to collapse similar transcripts&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Here's our CD-HIT command line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cdhit -T 10 -c 0.98 -i longest_MetStopORF_HC50_TBNOCFED.fasta -o cdhit_0.98_LMOHC50_TBNOCFED.prot.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;More details here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bmcbiol.biomedcentral.com/articles/10.1186/s12915-017-0473-4#Sec9&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bmcbiol.biomedcentral.com/articles/10.1186/s12915-017-0473-4#Sec9&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-06-29T00:03:45.953" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4593" PostTypeId="2" ParentId="4591" CreationDate="2018-06-29T06:57:44.463" Score="1" Body="&lt;p&gt;Whenever you have batch or other nuisance variables driving variation you're going to get odd networks. You tried using SVA, which is a good method to use, but it seems in this case that it's able to optimally handle this dataset. I also strongly suggest that you use &lt;code&gt;pickSoftThreshold()&lt;/code&gt; to choose a &lt;code&gt;softpower&lt;/code&gt;, since that may produce better results (though since you already have negative values I'm not holding by breath). In general, you're going to need to play with either SVA or filtering to remove the batch-related genes that are causing problems.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-06-29T06:57:44.463" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4594" PostTypeId="1" AcceptedAnswerId="4600" CreationDate="2018-06-29T13:53:56.127" Score="0" ViewCount="810" Body="&lt;p&gt;I have gene expression RNA-seq data of 9 time points (0 hour, 2 hour, ..., 16 hour). I need a plot on which x axis is time point and each data point at a certain time point represents the L2 norm (Euclidean norm) of the difference between the gene expression vector at that time point and the previous time point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the link of my expression data&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://drive.google.com/file/d/1crJSqVdGujy1tFyeoV6E7e-xO9SXteJZ/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://drive.google.com/file/d/1crJSqVdGujy1tFyeoV6E7e-xO9SXteJZ/view?usp=sharing&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2018-06-29T14:07:27.923" LastActivityDate="2018-07-01T17:36:21.970" Title="Calculating L2 norm (Euclidean norm) to set of genes in time series" Tags="|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4595" PostTypeId="2" ParentId="4578" CreationDate="2018-06-29T13:54:08.323" Score="2" Body="&lt;p&gt;In the &lt;a href=&quot;ftp://ftp.ncbi.nih.gov/snp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbSNP FTP site&lt;/a&gt; you linked, you need to go into organisms and select your organism of interest (human obviously). You can then select which release of the SNP database you want (eg. 150 vs 151) [the &lt;em&gt;modified&lt;/em&gt; dates hint at when they were released] and which genome build you want (eg. hg37 vs hg38).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below I am looking at the &lt;em&gt;.vcf&lt;/em&gt; files. I assume this what you want because they can be converted to ADAM format via the &lt;code&gt;adam vcf2adam&lt;/code&gt; command.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example human (newest genome and newest dbSNP release):&#xA;&lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/snp/organisms/human_9606_b151_GRCh38p7/VCF/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ncbi.nlm.nih.gov/snp/organisms/human_9606_b151_GRCh38p7/VCF/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your next decision is whether or not you want to use ALL the annotated SNPs in the database or just the common ones. For the former you choose the &lt;em&gt;All_20180418&lt;/em&gt; file and for the later the &lt;em&gt;common_all_20180418&lt;/em&gt; file. I'm pretty sure the convention here is that the date in the filename is when it was uploaded/updated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case I'm not sure what makes the most sense. Firstly, are you trying to replicate the initial conditions of when that software was published or do you want simply to benchmark the method with the best available data. Besides that, I imagine using all them is computationally much more taxing and if you are just coming up with general trend then using the common ones seems viable.&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2018-06-29T13:54:08.323" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4596" PostTypeId="1" CreationDate="2018-06-29T15:42:40.867" Score="5" ViewCount="1049" Body="&lt;p&gt;I have a set of scRNA-seq samples enriched with FACS for cells expressing a specific gene reporter (TdTomato). In particular the gene I want to report has positive counts in the resulting matrix for 97% of cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I followed &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/advanced/references#header&quot; rel=&quot;nofollow noreferrer&quot;&gt;CellRanger documentation&lt;/a&gt; to create a custom reference. I added TdTomato on an additional chromosome and I added an entry in the gtf:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TdTomato    .   exon    1   1431    .   -   .   gene_id &quot;TdTomato&quot;; transcript_id &quot;TdTomato&quot;; gene_name &quot;TdTomato&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The command I'm using is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cellranger-2.1.1/cellranger mkref --genome=refdata-cellranger-mm10-2.1.0.tdtomato --fasta=genome.tdtomato.fa --genes=genes.tdtomato.gtf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;genome.tdtomato.fa&lt;/code&gt; is the original reference genome (mm10, as provided by cellranger) with an additional chromosome at the end with header:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;TdTomato&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and &lt;code&gt;genes.tdtomato.gtf&lt;/code&gt; the original reference genome gtf with the additional line indicated previously at the end of the file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I actually tried both on the '+' and on the '-' strand, however I guess this should not affect the read mapping, since TdTomato is on an independent chromosome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran &lt;code&gt;cellranger count&lt;/code&gt; using the custom reference:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;s=&quot;SAMPLE&quot;&#xA;cellranger-2.1.1/cellranger count --id=&quot;$s&quot;_tt --transcriptome=refdata-cellranger-mm10-2.1.0.tdtomato --fastqs=original_fastqs_directory/&quot;$s&quot;/fastqs --sample=&quot;$s&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am using the &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/count&quot; rel=&quot;nofollow noreferrer&quot;&gt;default parameters of CellRanger&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I found that less than 1% of cells seems to express TdTomato. If I use the '+' strand I get slightly more counts (2.4% of cells have positive TdTomato).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I checked:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;With IGV I can see that there are reads mapping to TdTomato&lt;/li&gt;&#xA;&lt;li&gt;Using samtools I am able to see the same number of reads mapped on TdTom either with the + or - strand&lt;/li&gt;&#xA;&lt;li&gt;However, following &lt;a href=&quot;https://kb.10xgenomics.com/hc/en-us/articles/360000944391-Why-do-I-have-zero-UMI-counts-for-my-marker-gene-&quot; rel=&quot;nofollow noreferrer&quot;&gt;this documentation&lt;/a&gt; I noticed that by executing the CellRanger filtering steps, more reads are filtered in the case of the - strand&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This reduces the number of reads used for counts to 44 (2.4% of the original 1807 reads, - strand), 240 (13.3%, + strand) for TdTomato, 71620 (21.2%) for the gene reported by TdTomato. Interestingly, there are fewer reads for TdTom than counts, so there may be some kind of normalization afterwards?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The gene reported by TdTomato is longer, 11394 bp than TdTom, 1431 bp. The total length of the exons for the two genes is 5349 vs TdTom: 1431. Does this influence exponentially the amount of reads mapped to the two genes?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2018-07-26T16:54:04.600" LastActivityDate="2018-12-19T11:05:04.197" Title="No counts for added gene in cellranger (scRNA-seq)" Tags="|scrnaseq|cellranger|tdtomato|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="4597" PostTypeId="2" ParentId="769" CreationDate="2018-06-29T17:03:56.600" Score="5" Body="&lt;p&gt;Just to highlight that all the steps can be done within bcftools capabilities, and since I can't just comment on @blmoore 's answer: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view --types indels &amp;lt;vcf&amp;gt; |&#xA;  bcftools norm -m - |&#xA;  bcftools filter --include 'strlen(REF)&amp;lt;strlen(ALT)' |&#xA;  bcftools view -H&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;more bult-in functions for 'bcftools filter' &lt;a href=&quot;https://samtools.github.io/bcftools/bcftools.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="475" LastEditorUserId="48" LastEditDate="2018-07-02T13:49:52.417" LastActivityDate="2018-07-02T13:49:52.417" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4598" PostTypeId="1" CreationDate="2018-06-29T20:00:17.480" Score="0" ViewCount="386" Body="&lt;p&gt;Perhaps my understanding is wildly misguided, but I'm seriously confused about why people are still interested using FPKM values for cross sample gene expression analysis. My understanding is, that for some gene ${ i }$, its $FPKM$ is calculated as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ FPKM_i \propto \frac{ \mathrm{fragments\ mapping \ to\ gene \ }i } { \sum_j \mathrm{ fragments\ mapping\ to \ gene\  } j } \bigg/ &#xA;\mathrm{effective\_length}(i) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For comparing gene expression within a single sample, this doesn't seem too harmful to me. In fact, this seems like a good idea.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the use of $FPKM$ or even the famous $log_2(FPKM+1)$ in a similarity analysis seems to me plain wrong. True differences in RNA abundances of a couple of genes would significantly impact the FPKM values of the other genes leaving  perhaps confused about the true differences between the samples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missing something here? Is it ever meaningful to use FPKM values for analyzing across samples?&lt;/p&gt;&#xA;" OwnerUserId="823" LastActivityDate="2018-06-30T07:24:54.093" Title="Why is FPKM still used for gene expression studies?" Tags="|rna-seq|fpkm|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4599" PostTypeId="2" ParentId="4598" CreationDate="2018-06-30T07:24:54.093" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is it ever meaningful to use FPKM values for analyzing across samples?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One should never use FPKMs for anything important. They can occasionally be useful for plotting, but even in that case one needs to construct the FPKMs from properly normalized data and not use the original definition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's becoming ever rarer to see people using FPKMs these days, thankfully.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-06-30T07:24:54.093" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4600" PostTypeId="2" ParentId="4594" CreationDate="2018-07-01T17:36:21.970" Score="1" Body="&lt;p&gt;If you read your matrix into R and saved it as &lt;code&gt;m&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;diffMatrix = m[,2:9] - m[,1:8]  # The change per-timepoint&#xA;l2n = apply(diffMatrix, 2, function(x) sqrt(sum(x^2)))  # The L2-norm&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;R also has a &lt;code&gt;norm()&lt;/code&gt; function, which can directly produce the L2-norm, but it turns out that &lt;a href=&quot;https://stackoverflow.com/questions/10933945/how-to-calculate-the-euclidean-norm-of-a-vector-in-r&quot;&gt;this is slower&lt;/a&gt; than the simple method shown here. If you click through to the other question you can find somewhat faster implementations of this, but anything will suffice for small datasets like yours.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-07-01T17:36:21.970" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4601" PostTypeId="1" CreationDate="2018-07-01T17:40:44.073" Score="1" ViewCount="38" Body="&lt;p&gt;There are several secondary structure prediction algorithms. I am interested in the algorithms which predict secondary structure &lt;strong&gt;without&lt;/strong&gt; accessing the PDB to cheat by using the DSSP or STRIDE entry for known homologous structures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since DSSP and STRIDE are not performing predictions, rather they analyse the PDB structure to determine whether each residue is a helix, strand or coil.  This in turn means that, the 'so called' secondary structure prediction algorithm, also, is not performing predictions (except, of course, in the cases where no homologous structure/sequence is found in the PDB).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which secondary structure prediction algorithms don't cheat by using DSSP or STRIDE for homologous structures?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a list of secondary structure prediction algorithms, which are accessible via a consensus secondary structure prediction server called @NPS (currently down: &lt;a href=&quot;https://npsa-prabi.ibcp.fr/NPSA/npsa_seccons.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://npsa-prabi.ibcp.fr/NPSA/npsa_seccons.html&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DPM&#xA;DSC&#xA;GOR1&#xA;GOR3&#xA;HNN&#xA;MPSA&#xA;PHD&#xA;PREDA&#xA;SOPM&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1606" LastEditorUserId="77" LastEditDate="2018-07-01T17:56:42.293" LastActivityDate="2018-07-01T17:56:42.293" Title="Which secondary structure prediction algorithms don't query the PDB/DSSP for known homologous sequences?" Tags="|software-recommendation|pdb|software-usage|secondary-structure|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4602" PostTypeId="1" CreationDate="2018-07-01T19:34:15.283" Score="1" ViewCount="23" Body="&lt;p&gt;Any input as to what to study in order to work into genetic engineering from math background?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have troubles understanding what the &quot;interface&quot; would be between mathematicians and biologists/biotechnologists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wikipedia links the first model of an organism having built in 2011:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Computational_genomics#First_Computer_Model_of_an_Organism&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Computational_genomics#First_Computer_Model_of_an_Organism&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And it based on statistically analyzing biology/chemistry papers. This reminds of a typical &quot;stats research&quot;. So there's not necessarily direct interaction between the empirical and the formal. However, for new applications, it's given that there must be some interaction, in order to synthesize knowledge?&lt;/p&gt;&#xA;" OwnerUserId="3017" LastActivityDate="2018-07-01T19:34:15.283" Title="Any input as to what to study in order to work into genetic engineering from math background?" Tags="|statistics|molecular-genetics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4603" PostTypeId="1" CreationDate="2018-07-01T22:04:45.697" Score="3" ViewCount="36" Body="&lt;p&gt;Probably a naive question. I am inexperienced. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in identifying potential CRISP (Cysteine-rich secretory proteins) in a certain tissue transcriptome (ca. 20k sequences in fasta). I have detected signalP and estimated % of cysteine in sequences. However CRISP rely on a certain pattern, and I am not aware of batch-detected algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please would anyone know the best/simplest way of detecting CRISP candidates?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[EDIT]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been asked for the pattern. I summarise what the literature says:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Several of them have a CAP domain;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There should be a region called 'cysteine-rich domain' (CRD) upon the carboxyl terminal half of the protein containing 10 of roughly 16 conserved cysteines. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I am particularly interested in venom-derived CRISPs, but I am not sure whether they cluster in a certain CRISP family.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;References:&#xA;Roberts et al. (2007) Structure and function of epididymal protein cysteine‐rich secretory protein‐1.&#xA;Yamazaki &amp;amp; Morita (2004) Structure and function of snake venom cysteine-rich secretory proteins. &lt;/p&gt;&#xA;" OwnerUserId="2329" LastEditorUserId="2329" LastEditDate="2018-07-03T17:43:07.337" LastActivityDate="2018-07-03T17:43:07.337" Title="Batch detection of CRISP proteins in fasta file" Tags="|sequence-annotation|blast|transcriptome|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4604" PostTypeId="1" CreationDate="2018-07-02T02:12:55.787" Score="5" ViewCount="71" Body="&lt;p&gt;I have a training set of RT-qPCR gene expression data (not run in triplicate) for a batch of samples with two phenotypes &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; on which I've trained a &amp;quot;logistic regression classifier&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I also have another smaller set of samples which have been run twice as technical replicates on the RT-qPCR machine in order to study the effect of qPCR noise on my classifier.&lt;/p&gt;&#xA;&lt;p&gt;I have a particular cut-off probability &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; for my logistic regression model, such that we must be at least &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; confident to classify a test sample as &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; (similarly for &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;), otherwise no classification is returned.&lt;/p&gt;&#xA;&lt;p&gt;I would like to incorporate the effect of RT-qPCR noise into my logistic classifier in order to return a more accurate classification probability for a particular test sample.&lt;/p&gt;&#xA;&lt;p&gt;My model for the RT-qPCR noise for a particular sample &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is given by&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$X_i\sim N(\mu_i, \sigma^2I).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The logit is then given by&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$t_i = \beta_0 + \beta^TX_i,$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;with distribution&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$t_i\sim N(\beta_0+\beta^T\mu_i, \sigma_t^2),$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\sigma_t^2=\|\beta\|_2^2\sigma^2$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Since &lt;span class=&quot;math-container&quot;&gt;$\mu_i$&lt;/span&gt; is a nuisance parameter, we eliminate it by taking the difference of the two replicates for a particular sample&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Z_i=t_i^{(1)} - t_i^{(2)} \sim N(0, 2\sigma_t^2).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;We can then form the MLE of &lt;span class=&quot;math-container&quot;&gt;$\sigma_t^2$&lt;/span&gt; by computing&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\hat{\sigma}_t^2=\frac{1}{2N}\sum_{i=1}^NZ_i^2.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Therefore, for a particular test sample &lt;span class=&quot;math-container&quot;&gt;$x_i$&lt;/span&gt;, we compute the expected value of the logit-normal distribution with location &lt;span class=&quot;math-container&quot;&gt;$t_i$&lt;/span&gt; and squared scale &lt;span class=&quot;math-container&quot;&gt;$\hat{\sigma}_t^2$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$p_i=\int_0^1x\frac{1}{\sqrt{2\pi\hat{\sigma}_t^2}}\frac{1}{x(1-x)}e^{-\frac{(\text{logit}(x)-t_i)^2}{2\hat{\sigma}_t^2}}dx.$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The resulting &lt;span class=&quot;math-container&quot;&gt;$p_i$&lt;/span&gt; is then our corrected probability of phenotype &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Is this a sensible way to achieve my stated goal?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is an example plot of the necessary correction when the squared scale is &lt;span class=&quot;math-container&quot;&gt;$\sigma^2=2$&lt;/span&gt;,&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EWPBKm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EWPBKm.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The black line corresponds to a qPCR assay with no noise.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The violet line indicates the corrected probability.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For example, if the regression model says that a sample has a 75% chance of being phenotype &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, then in the corrected model, this actually corresponds to only having about a 65% chance of being phenotype &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1456" LastEditorUserId="3848" LastEditDate="2020-07-15T07:49:17.903" LastActivityDate="2022-02-07T22:37:46.463" Title="Correcting for noise in RT-qPCR gene expression data" Tags="|gene|statistics|pcr|noise|gene-expression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4605" PostTypeId="2" ParentId="4586" CreationDate="2018-07-02T10:44:13.333" Score="1" Body="&lt;p&gt;Turns out I'd already written some code that did it, human memory is a funny thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This uses biopython to split the field description to where the species is. May not work for all NCBI files, but seems to work on most.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import Bio&#xA;from Bio import SeqIO&#xA;from Bio import AlignIO    &#xA;&#xA;for record in SeqIO.parse (FILE, &quot;fasta&quot;):&#xA;    Speciesname = record.description.split('[', 1)[1].split(']', 1)[0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2965" LastActivityDate="2018-07-02T10:44:13.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4606" PostTypeId="1" AcceptedAnswerId="4607" CreationDate="2018-07-02T14:21:30.287" Score="3" ViewCount="82" Body="&lt;p&gt;I recently got a data set, from which I need to figure out which chromosome a gene is from, but the head of the data reads like: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Gene ID Description     Gene type       Gene End (bp)   Gene Start (bp) Strand  Associated Gene Name    Chromosome/scaffold name&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;while one entry of the data is like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ENSG00000252303 RNA, U6 small nuclear 280, pseudogene [Source:HGNC Symbol;Acc:HGNC:47243]       snRNA   67546754        67546651        1       RNU6-280P       CHR_HG2128_PATCH&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It looks like the last column is the information about which chromosome the gene comes from, but how to interpret the &quot;CHR_HG2128_PATCH&quot;?&lt;/p&gt;&#xA;" OwnerUserId="1082" LastActivityDate="2018-07-02T16:14:36.633" Title="Is there a way to tell which chromosome a gene is on, by looking at the &quot;Chromosome/scaffold name&quot;" Tags="|gene|ensembl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4607" PostTypeId="2" ParentId="4606" CreationDate="2018-07-02T16:14:36.633" Score="2" Body="&lt;p&gt;What you are looking at is a patch, not an actual chromosome location. HG-2128 is an issue ID, something that the Genome Reference Consortium uses to track issue with references. Information regarding that specific patch is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/grc/human/issues/HG-2128&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The issue is described as a &quot;Possible misassembly or indel variation in GRCh38 within AL049860.8.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;During the alignment step before you received your data, someone used a top level assembly version of the reference genome. This includes extra, accessioned scaffolds tacked on to the end of the file that contain the fix. These scaffolds represent what will be found in the next major release of the genome. The fixes aren't applied directly to the reference as soon as they are found due to the inevitable disruption of coordinates. So reads mapped to this FIX patch rather than the primary original scaffold in the primary assembly, on which the RNU6-280P gene resides.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The coordinates are slightly different. In the current GRCh38.p12 &#xA; assembly the RNU6-280P gene is located at chr6:67546651-67546754 (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene/106481886&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;). You can see that the coordinates are very close to those indicated in the table you provided (Gene Start, Gene End) but are indeed different. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/grc/help/patches&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is some more information on patches.&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2018-07-02T16:14:36.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4608" PostTypeId="1" AcceptedAnswerId="4610" CreationDate="2018-07-02T18:10:32.177" Score="3" ViewCount="4500" Body="&lt;p&gt;I am using GATK version 4.0, I tried to use multiple threads for calling variants using HaplotypeCaller using following command &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gatk --java-options -Xmx90G  -nt 28 HaplotypeCaller -I output.bam -R wheat_ref.fa -O final.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the error is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;'-nt' is not a valid command&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have also used &lt;strong&gt;--num_threads&lt;/strong&gt; but again the error was the same, as written &lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/1975/how-can-i-use-parallelism-to-make-gatk-tools-run-faster&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; this parameter is available in previous versions of GATK but I don't know what's its status in current version of GATK i.e GATK 4.0. Whether it is deprecated or replaced by some other parameter? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way of calling multiple threads for variant calling with GATK 4.0?&lt;/p&gt;&#xA;" OwnerUserId="1194" LastActivityDate="2018-07-02T18:59:21.993" Title="Increase number of threads for GATK 4.0 HaplotypeCaller" Tags="|sequence-alignment|variant-calling|gatk|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4609" PostTypeId="2" ParentId="4585" CreationDate="2018-07-02T18:43:41.017" Score="2" Body="&lt;p&gt;See &lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/2403/gatks-quality-scores-on-vcf-files&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt; on the GATK forums.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is normal to see values above 255.  The QUAL score is the posterior probability that all samples in your callset are homozygous reference. I quote Sheila's response from the thread posted above:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The QUAL score is the Phred-scaled posterior of AC = 0. We use the AC priors and the PLs to get the likelihood of the data given each AC, then use those to get the posterior probability for each AC. From there, the calculation is 1 - Pr{AC &gt; 0}.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Please see the GATK post in Methods and Algorithms for the &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/article.php?id=7258&quot; rel=&quot;nofollow noreferrer&quot;&gt;nitty gritty details&lt;/a&gt;. Beware, Bayesian statistics abound.&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="181" LastEditDate="2018-07-02T20:21:16.957" LastActivityDate="2018-07-02T20:21:16.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4610" PostTypeId="2" ParentId="4608" CreationDate="2018-07-02T18:59:21.993" Score="3" Body="&lt;p&gt;I am assuming you are using the non-SPARK version of the method. To specify the number of threads you wish to use with HaplotypeCaller, include &lt;code&gt;--native-pair-hmm-threads&lt;/code&gt; (&lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/tooldocs/current/org_broadinstitute_hellbender_tools_walkers_haplotypecaller_HaplotypeCaller.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;). This will only parallelize the pair hidden Markov models (pair HMM) process. This is an argument for the method HaplotypeCaller itself, so it should be located after the method is called as opposed to one of the &lt;code&gt;--java-options&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do recommend trying out the SPARK implementations of the GATK tools. The whole method will take advantage of the threads you specify. I've been running SPARK locally (i.e. not on Firecloud) on computational servers. I've been impressed with the enhanced performance.&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2018-07-02T18:59:21.993" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4611" PostTypeId="2" ParentId="4584" CreationDate="2018-07-02T19:48:21.403" Score="2" Body="&lt;p&gt;The Illumina HumanOmni1-Quad beadchip is a microarray device consisting of 1,140,419 markers which have derived from the 1,000 genomes project. The markers chosen are is high-value regions of the genome: ABO blood typing SNPs, cSNPs, disease-associated SNPs, eSNPs, SNPs in mRNA splice sites, ADME genes, AIMs, HLA complexes, indels, introns, MHC regions, miRNA binding sites, mitochondrial DNA, PAR, promoter regions, and Y-chromosome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The markers come together to form a sort of individual fingerprint. Germline SNPs are heritable and so can be traced up the line. Some SNPs are more prominent in subpopulations, something that Ancestry.com and other similar services leverage to predict ethnicity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This paper utilized genomic DNA extracted from the Singapore BioBank. BioBanks tend to record age, lifestyle, family history, and health information alongside the genomic DNA. I (conservatively) assume that the Singapore BioBank at least has familial relations and gender information tied to the DNA. My interpretation is that the ivestigators SNP genotyped using the Illumina Beadchip the genomic DNA from the bank and compared their results with the &quot;ground truth&quot; provided by the genetic bank.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.illumina.com/documents/products/datasheets/datasheet_humanomni1_quad.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Illumina brochure&lt;/a&gt; for the HumanOmni1-Quad beadchip.&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2018-07-02T19:48:21.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4612" PostTypeId="2" ParentId="4259" CreationDate="2018-07-02T20:18:58.300" Score="3" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;pct.1&lt;/code&gt;– The percentage of cells where the gene is detected in the first group&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;p_val_adj&lt;/code&gt;– Adjusted p-value, based on bonferroni correction using all genes in the dataset.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This is &lt;strong&gt;not&lt;/strong&gt; also known as a false discovery rate (FDR) adjusted p-value.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;An adjusted p-value of 1.00 means that after correcting for multiple testing, there is a 100% chance that the result (the logFC here) is due to chance.&lt;/p&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;You are correct. The expression of &lt;code&gt;Cttnbp2&lt;/code&gt; donwn-regulated by that factor. If you were to look in the table at the cluster 1 DE genes you would see &lt;code&gt;Cttnbp2&lt;/code&gt; with a logFC value of 1.35264. When you are looking at genes differentailly expressed in only two clusters, the sign is the only thing that changes.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://satijalab.org/seurat/de_vignette.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vignette&lt;/a&gt; posted above by @heathobrien.&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="-1" LastEditDate="2021-11-20T06:31:08.733" LastActivityDate="2021-11-20T06:31:08.733" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4613" PostTypeId="2" ParentId="4408" CreationDate="2018-07-03T09:07:10.373" Score="4" Body="&lt;p&gt;using &lt;strong&gt;&lt;a href=&quot;http://lindenb.github.io/jvarkit/VcfFilterJdk.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;vcffilterjdk&lt;/a&gt;&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;you want to keep the variant if there is no INFO/CAF.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, get All values of CAF &lt;a href=&quot;https://samtools.github.io/htsjdk/javadoc/htsjdk/htsjdk/variant/variantcontext/VariantContext.html#getAttributeAsStringList-java.lang.String-java.lang.String-&quot; rel=&quot;nofollow noreferrer&quot;&gt;as a String list&lt;/a&gt; (because it is declared as a string in the VCF header...), stream those data, skip the first value for REF , convert each string to double, you want to keep the variant if any allele is rare (&amp;lt; 0.01)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar dist/vcffilterjdk.jar -e 'return !variant.hasAttribute(&quot;CAF&quot;) || variant.variant.getAttributeAsStringList(&quot;CAF&quot;,&quot;1.0&quot;).stream().skip(1L).map(A-&amp;gt;Double.parseDouble(A)).anyMatch(A-&amp;gt;A&amp;lt;0.01);' input.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastEditorUserId="48" LastEditDate="2018-07-03T15:09:56.353" LastActivityDate="2018-07-03T15:09:56.353" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4615" PostTypeId="1" AcceptedAnswerId="4616" CreationDate="2018-07-03T09:49:43.190" Score="5" ViewCount="6599" Body="&lt;p&gt;What is difference between samtools mark duplicates and remove duplicates ?  Is it necessary to mark duplicates before removing duplicates with samtools?&lt;/p&gt;&#xA;" OwnerUserId="1194" LastActivityDate="2018-07-03T14:29:53.890" Title="Difference between samtools mark duplicates and samtools remove duplicates?" Tags="|sequence-alignment|ngs|samtools|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4616" PostTypeId="2" ParentId="4615" CreationDate="2018-07-03T14:01:49.103" Score="3" Body="&lt;p&gt;&lt;code&gt;samtools rmdup&lt;/code&gt; and &lt;code&gt;samtools markdup -r&lt;/code&gt; do the same thing. Without the &lt;code&gt;-r&lt;/code&gt; flag &lt;code&gt;samtools markdup&lt;/code&gt; only flags the duplicates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll have to run &lt;code&gt;samtools fixmate -m&lt;/code&gt; and sort the output to add ms and MC tags to prepare the file for &lt;code&gt;markdup&lt;/code&gt;..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the current &lt;a href=&quot;http://www.htslib.org/doc/samtools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;, &lt;code&gt;rmdup&lt;/code&gt; is obsolete. Please do not use &lt;code&gt;rmdup&lt;/code&gt;. &lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2018-07-03T14:01:49.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4617" PostTypeId="2" ParentId="4615" CreationDate="2018-07-03T14:03:51.340" Score="2" Body="&lt;p&gt;&lt;strong&gt;TL;DR:&lt;/strong&gt; just use &lt;code&gt;markdup&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;rmdup&lt;/code&gt; removes duplicates from BAM, while &lt;code&gt;markdup&lt;/code&gt;, like Picard's &lt;code&gt;MarkDuplicates&lt;/code&gt;, marks duplicates by default without hard removal – the latter is usually the desired behavior. In addition, &lt;code&gt;markdup&lt;/code&gt; implements a better algorithm that takes care of more corner cases and gives more consistent results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: to emphasize – I see no point to use &lt;code&gt;samtools rmdup&lt;/code&gt; nowadays. It has been declared obsolete as @kohlkopf said.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2018-07-03T14:29:53.890" LastActivityDate="2018-07-03T14:29:53.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4618" PostTypeId="2" ParentId="4603" CreationDate="2018-07-03T14:26:39.703" Score="1" Body="&lt;p&gt;Is your goal to detect the actual proteins from the sequences? If you'll settle for detection of transcripts from the genes encoding these proteins as opposed to detection of the proteins themselves, here is an approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Aligning your transcriptome to a reference genome (probably with STAR) will be the first step. You have lots of short reads in your FASTA, but in their current state they are no more informative than a box of puzzle pieces. Aligning your reads to the reference will allow you to extrapolate which genes that you short reads correspond with. From here you can quantify the features (genes) in your transcriptome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lucky for you, there are only 3 genes in the &lt;a href=&quot;https://www.genenames.org/cgi-bin/genefamilies/set/1355&quot; rel=&quot;nofollow noreferrer&quot;&gt;cysteine rich secretory protein family&lt;/a&gt;, &lt;code&gt;CRISP1&lt;/code&gt;, &lt;code&gt;CRISP2&lt;/code&gt;, and &lt;code&gt;CRISP3&lt;/code&gt;. Alignment with a reference (and annotations) and quantification of genes that your reads align to will give you all you need. You'll just have to find the three genes in your final counts table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend &lt;a href=&quot;https://github.com/alexdobin/STAR&quot; rel=&quot;nofollow noreferrer&quot;&gt;STAR&lt;/a&gt; as an aligner with the &lt;code&gt;--quantMode GeneCounts&lt;/code&gt; argument. It's very quick if you've got the resources and is geared towards RNA-seq reads.&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2018-07-03T14:26:39.703" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4619" PostTypeId="5" CreationDate="2018-07-03T16:36:01.300" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-07-03T16:36:01.300" LastActivityDate="2018-07-03T16:36:01.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4620" PostTypeId="4" CreationDate="2018-07-03T16:36:01.300" Score="0" Body="For questions about the CRISPER-Cas9 genome editing technology" OwnerUserId="2719" LastEditorUserId="2719" LastEditDate="2018-07-04T07:09:06.343" LastActivityDate="2018-07-04T07:09:06.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4621" PostTypeId="1" AcceptedAnswerId="4624" CreationDate="2018-07-03T17:38:24.847" Score="2" ViewCount="36" Body="&lt;p&gt;I have exomes from 24 individual mice. The exomes are the product of Roche's SeqCap EZ HyperCap target enrichment kit. I see that the mouse exome design comes from mm9/NCBI37, the previous major reference genome released in July 2007. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which reference should I use to align these reads? What artifacts may arise from mapping these reads designed from mm9 to the mm10 reference? I have already aligned the reads to mm10 and noticed an erroneous exon right in the middle of an intron, perhaps a target that was an exon in the mm9 release but revised out of the mm10 release. My goal is to investigate structural variation.&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2018-07-04T06:57:10.660" Title="GRCm37-designed exon target enrichment, which reference to use?" Tags="|sequence-alignment|variant-calling|reference-genome|exome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4623" PostTypeId="1" AcceptedAnswerId="4625" CreationDate="2018-07-03T19:49:04.313" Score="5" ViewCount="1307" Body="&lt;p&gt;I am doing variant calling on RNA-seq datasets from wheat which is hexaploid,the binary alignment (BAM) files were created using STAR version 2.6.0c and variant calling was done using GATK 4.0 HaplotypeCaller.The whole pipeline is as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alignment through &lt;strong&gt;STAR 2.6.0c&lt;/strong&gt; :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;STAR --runMode alignReads --runThreadN 20  --outSAMtype BAM  SortedByCoordinate --sjdbOverhang 100  --genomeDir wheat --readFilesIn G1_cleaned_R1.fastq G1_cleaned_R2.fastq --sjdbGTFtagExonParentTranscript wheat.gff3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then &lt;strong&gt;AddOrReplaceReadGroups&lt;/strong&gt; from &lt;strong&gt;picard&lt;/strong&gt; was used with default settings:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; AddOrReplaceReadGroups \&#xA;   I=input.bam \&#xA;   O=Read_groups_added.bam \&#xA;   RGID=4 \&#xA;   RGLB=lib1 \&#xA;   RGPL=illumina \&#xA;   RGPU=unit1 \&#xA;   RGSM=20&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally &lt;strong&gt;GATK 4.0 haplotypeCaller&lt;/strong&gt; was applied:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gatk HaplotypeCaller --native-pair-hmm-threads 28  \&#xA;    -I Read_groups_added.bam -R wheat_ref.fa -O final.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The command ran successfully as shown below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;20:33:01.018 INFO  ProgressMeter - TGACv1_scaffold_470693_5DS:301            102.2              45108670         441320.0&#xA;20:33:11.039 INFO  ProgressMeter - TGACv1_scaffold_724195_U:301            102.4              45111270         440625.5&#xA;20:33:21.072 INFO  ProgressMeter - TGACv1_scaffold_724579_U:301            102.5              45113880         439932.4&#xA;20:33:31.108 INFO  ProgressMeter - TGACv1_scaffold_248869_3B:301            102.7              45116440         439240.9&#xA;20:33:41.143 INFO  ProgressMeter - TGACv1_scaffold_248918_3B:301            102.9              45118990         438551.6&#xA;20:33:51.150 INFO  ProgressMeter - TGACv1_scaffold_306137_4AL:301            103.0              45121550         437866.7&#xA;20:34:01.176 INFO  ProgressMeter - TGACv1_scaffold_193559_2DS:301            103.2              45124100         437182.5&#xA;20:34:03.134 INFO  HaplotypeCaller - 100112154 read(s) filtered by: ((((((((MappingQualityReadFilter AND MappingQualityAvailableReadFilter) AND MappedReadFilter) AND NotSecondaryAlignmentReadFilter) AND NotDuplicateReadFilter) AND PassesVendorQualityCheckReadFilter) AND NonZeroReferenceLengthAlignmentReadFilter) AND GoodCigarReadFilter) AND WellformedReadFilter)&#xA;100112154 read(s) filtered by: (((((((MappingQualityReadFilter AND MappingQualityAvailableReadFilter) AND MappedReadFilter) AND NotSecondaryAlignmentReadFilter) AND NotDuplicateReadFilter) AND PassesVendorQualityCheckReadFilter) AND NonZeroReferenceLengthAlignmentReadFilter) AND GoodCigarReadFilter)&#xA;100112154 read(s) filtered by: ((((((MappingQualityReadFilter AND MappingQualityAvailableReadFilter) AND MappedReadFilter) AND NotSecondaryAlignmentReadFilter) AND NotDuplicateReadFilter) AND PassesVendorQualityCheckReadFilter) AND NonZeroReferenceLengthAlignmentReadFilter)&#xA;100112154 read(s) filtered by: (((((MappingQualityReadFilter AND MappingQualityAvailableReadFilter) AND MappedReadFilter) AND NotSecondaryAlignmentReadFilter) AND NotDuplicateReadFilter) AND PassesVendorQualityCheckReadFilter)&#xA;100112154 read(s) filtered by: ((((MappingQualityReadFilter AND MappingQualityAvailableReadFilter) AND MappedReadFilter) AND NotSecondaryAlignmentReadFilter) AND NotDuplicateReadFilter)&#xA;100112154 read(s) filtered by: (((MappingQualityReadFilter AND MappingQualityAvailableReadFilter) AND MappedReadFilter) AND NotSecondaryAlignmentReadFilter)&#xA;100112154 read(s) filtered by: ((MappingQualityReadFilter AND MappingQualityAvailableReadFilter) AND MappedReadFilter)&#xA;100112154 read(s) filtered by: (MappingQualityReadFilter AND MappingQualityAvailableReadFilter)&#xA;21066679 read(s) filtered by: MappingQualityReadFilter &#xA;79045475 read(s) filtered by: MappingQualityAvailableReadFilter &#xA;&#xA;20:34:03.134 INFO  ProgressMeter - TGACv1_scaffold_725922_U:301            103.2              45124592         437049.1&#xA;20:34:03.134 INFO  ProgressMeter - Traversal complete. Processed 45124592 total regions in 103.2 minutes.&#xA;20:34:04.749 INFO  VectorLoglessPairHMM - Time spent in setup for JNI call : 0.0&#xA;20:34:04.749 INFO  PairHMM - Total compute time in PairHMM computeLogLikelihoods() : 0.0&#xA;20:34:04.749 INFO  SmithWatermanAligner - Total compute time in java Smith-Waterman : 0.00 sec&#xA;20:34:04.749 INFO  HaplotypeCaller - Shutting down engine&#xA;[July 3, 2018 8:34:04 PM PKT]   org.broadinstitute.hellbender.tools.walkers.haplotypecaller.HaplotypeCaller done. Elapsed time: 103.99 minutes.&#xA;Runtime.totalMemory()=4789370880&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but still the generated vcf file i.e final.vcf does not contain any variants as shown below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[dellemc@localhost bilal]$ tail final.vcf&#xA;##contig=&amp;lt;ID=TGACv1_scaffold_725916_U,length=500&amp;gt;&#xA;##contig=&amp;lt;ID=TGACv1_scaffold_725917_U,length=500&amp;gt;&#xA;##contig=&amp;lt;ID=TGACv1_scaffold_725918_U,length=500&amp;gt;&#xA;##contig=&amp;lt;ID=TGACv1_scaffold_725919_U,length=500&amp;gt;&#xA;##contig=&amp;lt;ID=TGACv1_scaffold_725920_U,length=500&amp;gt;&#xA;##contig=&amp;lt;ID=TGACv1_scaffold_725921_U,length=500&amp;gt;&#xA;##contig=&amp;lt;ID=TGACv1_scaffold_725922_U,length=500&amp;gt;&#xA;##contig=&amp;lt;ID=TGACv1_scaffold_731344_7BL,length=500&amp;gt;&#xA;##source=HaplotypeCaller&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  20&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there any mistake in the haplotypeCaller command or any argument is missing ?&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="298" LastEditDate="2018-07-04T08:05:32.840" LastActivityDate="2018-07-13T15:04:11.323" Title="No variant found using GATK 4.0 HaplotypeCaller" Tags="|vcf|variant-calling|gatk|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4624" PostTypeId="2" ParentId="4621" CreationDate="2018-07-04T06:57:10.660" Score="2" Body="&lt;p&gt;The only possible issue with using GRCm38 as the reference is that since GRCm37 was used in the bait design there are likely a few probes that will show artificial structural variations, due to errors in GRCm37. However, these will be present in all of your mice and if you use liftOver to look at the regions around the probes then such regions should become obvious. In fact, that'd be a convenient thing to do if you want to keep your (likely better) GRCm38 alignments, flank each probe site by a few hundred bases and then use liftOver to get the GRCm37 coordinates. If you go through the result you'll be able to find the false-positive structural variants due to the baits being designed with an older reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, just use GRCm37 and hope that any issues with it don't result in false-negative SV calls.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-07-04T06:57:10.660" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4625" PostTypeId="2" ParentId="4623" CreationDate="2018-07-04T10:55:54.470" Score="3" Body="&lt;p&gt;There are steps of the 'best practices' workflow missing, which I think is leading to the majority of reads being filtered by the haplotype caller, notice these lines in the output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;21066679 read(s) filtered by: MappingQualityReadFilter &#xA;79045475 read(s) filtered by: MappingQualityAvailableReadFilter &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That's &gt;100m reads being filtered by the HC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, you're missing &lt;code&gt;SplitNCigarReads&lt;/code&gt;, which separates reads that bridge a splice junction (padded by Ns in the STAR alignment) and reassigns the mapping quality. At the moment this element of the workflow is not validated or available in GATK4 (RNA-Seq variant calling is largely abandonware in newer versions of GATK, AFAIK), but in the best practices WDL utilises GATK3.5 - see &lt;a href=&quot;https://github.com/gatk-workflows/gatk3-4-rnaseq-germline-snps-indels/blob/master/rna-germline-variant-calling.wdl&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/gatk-workflows/gatk3-4-rnaseq-germline-snps-indels/blob/master/rna-germline-variant-calling.wdl&lt;/a&gt;, line 440-477. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other elements of the recommended workflow missing too, but I think this is the critical bit. I won't harp on here about the unsuitability of RNA-Seq data for variant calling, as I'm all too aware there often isn't much option.&lt;/p&gt;&#xA;" OwnerUserId="194" LastActivityDate="2018-07-04T10:55:54.470" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4626" PostTypeId="1" AcceptedAnswerId="4633" CreationDate="2018-07-04T15:23:37.023" Score="7" ViewCount="6380" Body="&lt;p&gt;I expected the same pattern but here I am not able to compare the patterns as the order of genes does not seem the same. I mean how I can have two heat maps on which the order of genes are the same so I woulds be able to compare the block of yellow and blue colors in two heat maps.&lt;/p&gt;&#xA;&lt;p&gt;I have used this code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;require(&amp;quot;RColorBrewer&amp;quot;)&#xA;myCol &amp;lt;- colorRampPalette(c(&amp;quot;dodgerblue&amp;quot;, &amp;quot;black&amp;quot;, &amp;quot;yellow&amp;quot;))(100)&#xA;myBreaks &amp;lt;- seq(-2, 2, length.out=101)&#xA;heat &amp;lt;- t(scale(t(sc_DEGG)))&#xA;hr &amp;lt;- hclust(as.dist(1-cor(t(y), method=&amp;quot;pearson&amp;quot;)), method=&amp;quot;complete&amp;quot;)&#xA;heatmap.2(&#xA;    heat, &#xA;    Rowv=as.dendrogram(hr), &#xA;    Colv=as.dendrogram(hc), &#xA;    col=myCol, &#xA;    breaks=myBreaks, &#xA;    main=&amp;quot;Title&amp;quot;, &#xA;    key=T, &#xA;    keysize=1.0, &#xA;    scale=&amp;quot;none&amp;quot;, &#xA;    density.info=&amp;quot;none&amp;quot;, &#xA;    reorderfun=function(d,w) reorder(d, w, agglo.FUN=mean), &#xA;    trace=&amp;quot;none&amp;quot;, &#xA;    cexRow=0.2, &#xA;    cexCol=0.8, &#xA;    distfun=function(x) dist(x, method=&amp;quot;euclidean&amp;quot;), &#xA;    hclustfun=function(x) hclust(x, method=&amp;quot;ward.D2&amp;quot;)&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I  have made a pair of heat maps based on your code in complexheatmap package. The right heat map is lighter than the left one. I don't know which one has been clustered based on the another one. How I can make the right one darker so that both  seem more similar??&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/j6GG6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/j6GG6.png&quot; alt=&quot;complexheatmap with different brightness&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="73" LastEditDate="2021-05-23T12:23:32.330" LastActivityDate="2021-05-23T12:25:06.370" Title="plotting two heatmaps with the same order of genes" Tags="|r|heatmap|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4627" PostTypeId="2" ParentId="4626" CreationDate="2018-07-04T18:36:32.280" Score="6" Body="&lt;p&gt;You expect that the order of genes is the same for both heatmaps, but the chance is small that the 4000 genes will exactly be clustered in the same order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you can do, is use the order of genes (rows) from one heatmap and create a new heatmap with the other data set. You'll lose the tree of course because it will not be clustering anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here a small example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(gplots)&#xA;&#xA;# First matrix&#xA;set.seed(2)&#xA;m &amp;lt;- replicate(8, rnorm(26))&#xA;&#xA;rownames(m) &amp;lt;- letters[1:26]&#xA;&#xA;hm &amp;lt;- heatmap.2(&#xA;    m, &#xA;    main=&quot;m&quot;, &#xA;    key=T, &#xA;    keysize=1.0, &#xA;    scale=&quot;none&quot;,&#xA;    trace=&quot;none&quot;, &#xA;    density.info=&quot;none&quot;, &#xA;    Colv = FALSE,&#xA;    distfun=function(x) dist(x, method=&quot;euclidean&quot;), &#xA;    hclustfun=function(x) hclust(x, method=&quot;ward.D2&quot;)&#xA;)&#xA;&#xA;# Second matrix&#xA;set.seed(3)&#xA;m2 &amp;lt;- replicate(8, rnorm(26))&#xA;&#xA;rownames(m2) &amp;lt;- letters[1:26]&#xA;&#xA;hm2 &amp;lt;- heatmap.2(&#xA;    m2, &#xA;    main=&quot;m2&quot;, &#xA;    key=T, &#xA;    keysize=1.0, &#xA;    scale=&quot;none&quot;,&#xA;    trace=&quot;none&quot;, &#xA;    density.info=&quot;none&quot;, &#xA;    Colv = FALSE,&#xA;    distfun=function(x) dist(x, method=&quot;euclidean&quot;), &#xA;    hclustfun=function(x) hclust(x, method=&quot;ward.D2&quot;)&#xA;)&#xA;&#xA;# Second matrix with row order of first&#xA;reorder_m2 &amp;lt;- m2[rev(hm$rowInd),]&#xA;&#xA;hm2_m &amp;lt;- heatmap.2(&#xA;    reorder_m2, &#xA;    main=&quot;m2 in m's order&quot;, &#xA;    key=T, &#xA;    keysize=1.0, &#xA;    scale=&quot;none&quot;,&#xA;    trace=&quot;none&quot;, &#xA;    density.info=&quot;none&quot;, &#xA;    Rowv = FALSE, &#xA;    Colv = FALSE,&#xA;    dendrogram=&quot;none&quot;,&#xA;    distfun=function(x) dist(x, method=&quot;euclidean&quot;), &#xA;    hclustfun=function(x) hclust(x, method=&quot;ward.D2&quot;)&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TZeDB.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TZeDB.jpg&quot; alt=&quot;m&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/IDXMb.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IDXMb.jpg&quot; alt=&quot;m2&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/UrvQD.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UrvQD.jpg&quot; alt=&quot;m2_m&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2018-07-04T20:03:12.113" LastActivityDate="2018-07-04T20:03:12.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4630" PostTypeId="1" CreationDate="2018-07-05T09:22:30.070" Score="2" ViewCount="155" Body="&lt;p&gt;I am trying to understand the working of MutSigCV. But I am not sure about some terms used in MAF file format which is &lt;em&gt;&lt;strong&gt;category&lt;/strong&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;According to &lt;a href=&quot;http://software.broadinstitute.org/cancer/software/genepattern/modules/docs/MutSigCV&quot; rel=&quot;nofollow noreferrer&quot;&gt;MutSigCV&lt;/a&gt;, every mutation can be divided into the following categories:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;CpG transitions&lt;/li&gt;&#xA;&lt;li&gt;CpG transversions&lt;/li&gt;&#xA;&lt;li&gt;C: G transitions&lt;/li&gt;&#xA;&lt;li&gt;C: G transversions&lt;/li&gt;&#xA;&lt;li&gt;A: T transitions&lt;/li&gt;&#xA;&lt;li&gt;A: T transversions&lt;/li&gt;&#xA;&lt;li&gt;null+indel mutations&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I am very much clear about some of this categories like C: G transitions and A: T transitions but for rest, I am unable to make any solid conceptual background. Can anyone suggest me the proper definition of these categories of mutations with proper examples?&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Definition&lt;/p&gt;&#xA;&lt;p&gt;MutSig stands for &amp;quot;Mutation Significance&amp;quot; and is defined &lt;a href=&quot;https://software.broadinstitute.org/cancer/cga/mutsig&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2141" LastEditorUserId="3848" LastEditDate="2022-03-26T14:32:34.313" LastActivityDate="2022-03-26T14:32:34.313" Title="Meaning of category in MutSigCV" Tags="|genomics|cancer|gatk|mutations|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4631" PostTypeId="1" CreationDate="2018-07-05T10:29:30.053" Score="5" ViewCount="170" Body="&lt;p&gt;We generated a (diploid, chordata, highly heterozgous) genome using PacBio and we wanted to see whether it contains lineage-specific duplications (paralogs, basically). The genome is not in Ensembl yet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only data we have at the moment are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;genome&lt;/li&gt;&#xA;&lt;li&gt;transcript annotation&lt;/li&gt;&#xA;&lt;li&gt;RNAseq &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;We found some methods from papers:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;use &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/24/3/319/252715&quot; rel=&quot;noreferrer&quot;&gt;Blast&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;detect segmental duplications in complete genomes with &lt;a href=&quot;https://github.com/nlapalu/SDDetector&quot; rel=&quot;noreferrer&quot;&gt;SDDdetector&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;detecting putative recent segmental duplications or diploid homozygous organisms based on NGS data &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S2214662817300476&quot; rel=&quot;noreferrer&quot;&gt;DuplicationDetector&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;(I just came up with this, but should work) map back the reads to the assembly &amp;amp; analyze the read depth to detect duplicated portions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I'll gladly take advice.&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2018-07-09T04:48:21.887" Title="Find paralogs in a draft genome" Tags="|genome|assembly|duplications|paralogs|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4633" PostTypeId="2" ParentId="4626" CreationDate="2018-07-05T14:40:01.943" Score="11" Body="&lt;p&gt;ComplexHeatmap is built for plotting side-by-side heat maps with the same clustering - you use the &lt;code&gt;+&lt;/code&gt; notation, similar to ggplot2. To use the same example data as &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/939/b-nota&quot;&gt;@b.nota&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(ComplexHeatmap)&#xA;# First matrix&#xA;set.seed(2)&#xA;m &amp;lt;- replicate(8, rnorm(26))&#xA;rownames(m) &amp;lt;- letters[1:26]&#xA;&#xA;# Second matrix&#xA;set.seed(3)&#xA;m2 &amp;lt;- replicate(8, rnorm(26))&#xA;rownames(m2) &amp;lt;- letters[1:26]&#xA;&#xA;Heatmap(m) + Heatmap(m2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the results:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/L3Ju8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/L3Ju8.png&quot; alt=&quot;ComplexHeatmap Results&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There are lots of examples in the &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/ComplexHeatmap.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;extensive documentation&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&lt;p&gt;Since you have a few genes with high values in the 'T' matrix, they are washing out the colour scheme for the rest of the heatmap. You could set the max to be lower, and accept that those genes will saturate - the resulting heatmap will be less 'honest' about the data, but might capture what you want to see. To change colour scheme requires a call to the &lt;code&gt;colorRamp2&lt;/code&gt; function of the &lt;code&gt;circlize&lt;/code&gt; package.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(circlize)&#xA;mycol &amp;lt;- colorRamp2(c(-2,0,2), c(&amp;quot;dodgerblue&amp;quot;, &amp;quot;black&amp;quot;, &amp;quot;yellow&amp;quot;))&#xA;Heatmap(m, col=mycol, cluster_columns = FALSE) + &#xA;    Heatmap(m2, col=mycol, cluster_columns = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This gives:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JwcMy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JwcMy.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="194" LastEditorUserId="73" LastEditDate="2021-05-23T12:25:06.370" LastActivityDate="2021-05-23T12:25:06.370" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4634" PostTypeId="1" CreationDate="2018-07-05T17:09:33.317" Score="1" ViewCount="100" Body="&lt;p&gt;I have a matrix of gene expression for single cell (cells in column and genes in row) in 9 time points with about to 200 cells for each time point. I have also a matrix of gene expression in bulk RNA-seq with the same time point and 2 replications. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have pooled reads from cells in each time point to have a simulated bulk RNA-seq. now I have two matrices; Real bulk RNA-seq in 9 time points and two replication and another matrix a simulated bulk RNA-seq with 9 time points and two replications. I have plotted a heat map of them to show the pattern of gene expression is similar in both matrices so single cell RNA-seq is good enough. Actually I have used the real bulk RNA-seq in a similar study as a gold standard for checking my single cell RNA-seq. My problem is this the pattern of gene expression in 0 time point is very different in two datasets. I should note that 0 time point comes from Fluidigm and another 8 time points come from icell8. I should be able TO adjust this heat map to look similar enough. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I thought to show similarity by plotting cumulative fraction of genes detectable in varying fractions of cells (lets say 20%, 40%, ...), segregated by bulk RNA-seq expression quartile. Something like a plot that x axis is percent of cells and y axis is percent of genes but I think I should solve the fundamental problem in a heat map.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have single cell (matrix with T) and bulk RNA-seq (matrix with R). Whatever I am plotting, time point 0 in single cell RNA-seq seems more yellow and dose not allow a good similarity between single cell and bulk RNA-seq. How I can make zero time point less yellow so another time points become more yellow?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2018-07-10T09:58:56.357" LastActivityDate="2018-07-10T09:58:56.357" Title="Variability in genes across platforms" Tags="|r|scrnaseq|transcriptome|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="4635" PostTypeId="1" AcceptedAnswerId="4636" CreationDate="2018-07-05T19:01:09.040" Score="2" ViewCount="949" Body="&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/324857/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cross-posted&lt;/a&gt; on Biostars (with no answers currently). I hope that's OK. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I thought that &lt;strong&gt;&lt;em&gt;iff&lt;/em&gt;&lt;/strong&gt; a read matches the reference at a position, it would appear in the pileup column for that position. But tracking a single read in pysam shows that that's not the case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This read appears in some pileup columns for positions to which it does not map, and does not appear in some pileup columns to which it does map.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's wrong in my understanding?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Updated code (and updated output) moved to &lt;a href=&quot;https://gist.github.com/jaLasser/15ce9d5f6b91854fe672b5a830ed4f5f&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gist&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: Here's the raw SAM read. I wasn't separating the read from it's mate when I calculated the figures above.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;E00489:44:HNNVYCCXX:1:2120:26524:19135  163     5       345248  0       48M102S =       345446  350     TGTGTGTGTGTGTGTGTGTGTCGGATGATGTCCCTGGCTGTGTGTGGGCGGGAGTGCGTGGGGGAGGGTGAGAGTGTGGATGTCGGTGGTCGCGGCTGCGTGAGAGAGGGGGTGTGTGGGGGGGGGGGGGGGGGGGGGTGTGGGTGGGCG  AAAFFJFJJJJJJJJJJJJJJJJJJJJJJJJJFJJJJJJJJJJJ-J-7-777-7-7-7--7-77-77F---7-7AFJ7----7-7FAA7-7---7----77-777------7-7)7-AAA-A)7FA)&amp;lt;)&amp;lt;--)&amp;lt;)--&amp;lt;-FAJFJFFFF-&amp;lt;  XA:Z:5,+345312,48M102S,0;5,+345554,48M102S,1;5,+345452,4S44M102S,0;5,+345192,48M102S,1; MC:Z:6M2D144M   MD:Z:48 RG:Z:HNNVYCCXX.1       NM:i:0   MQ:i:60 AS:i:48 XS:i:48&#xA;E00489:44:HNNVYCCXX:1:2120:26524:19135  83      5       345446  60      6M2D144M        =       345248  -350    TGGGGATGTGTGTGTGTGTGTCGGATGATGTCCCTGGTTGTGTGTGGGGATGTGTGTGTGTGTGGGTTGATGGTCCTGGCTGTGTGTGTGTGTGGGTGTGCGTGTGTGGGTGTGTGTGTGTGTGTGTCGGATGATGTCCCTGGCTGTGTG  JJAF7-FJJJFJFF-AFJF7--&amp;lt;A--&amp;lt;F&amp;lt;7-7--FA--F-J--A-7--7-FJFJ7JFJJF&amp;lt;JA77-7----7----&amp;lt;F7-&amp;lt;JJF-&amp;lt;&amp;lt;FAJJJJA-7-&amp;lt;JAJJ&amp;lt;7FFJJ&amp;lt;JJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJFFFAA  MC:Z:48M102S    MD:Z:6^TG31C25C2A5T0C76 RG:Z:HNNVYCCXX.1        NM:i:7  MQ:i:0  AS:i:119        XS:i:69&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I guess this explains why the read appears in more than &lt;code&gt;150&lt;/code&gt; pileup columns—in some of the columns, it's really its mate. But given the CIGAR strings for these paired reads, I would expect one of these mates to appear in &lt;code&gt;198&lt;/code&gt; = (&lt;code&gt;48&lt;/code&gt; + &lt;code&gt;6&lt;/code&gt; + &lt;code&gt;144&lt;/code&gt;) pileup columns, at each position where one of them maps?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; I re-ran this script, using @Bioathlete's suggestion of using the flags to distinguish the read and its mate. In case &lt;code&gt;pileup()&lt;/code&gt; was filtering reads with low base quality, I also calculated how many positions the read has &lt;code&gt;baseQ &amp;gt;= 13&lt;/code&gt;. It's &lt;code&gt;150&lt;/code&gt; positions, but this read still appears in pileup columns for only &lt;code&gt;121&lt;/code&gt; positions. &lt;/p&gt;&#xA;" OwnerUserId="3033" LastEditorUserId="3033" LastEditDate="2018-07-06T01:05:42.017" LastActivityDate="2018-07-06T01:05:42.017" Title="pysam pileup: what reads appear in the pileup?" Tags="|python|sam|sequence-alignment|pysam|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4636" PostTypeId="2" ParentId="4635" CreationDate="2018-07-05T22:30:21.297" Score="2" Body="&lt;p&gt;The two entries in the sam file represent mate pairs for the given ID.  You can tell the difference based on the sam flags.  &lt;a href=&quot;http://broadinstitute.github.io/picard/explain-flags.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Picard&lt;/a&gt; has a nice tool to determine the meaning of the flags.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are missing coverage within the first read.  I think that it is related to the low quality bases.  samtools mpileup filters bases with a quality below 13 and pysam just wraps the samtools C functions.&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2018-07-05T22:30:21.297" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4637" PostTypeId="1" AcceptedAnswerId="5211" CreationDate="2018-07-06T09:08:08.217" Score="5" ViewCount="2176" Body="&lt;p&gt;I have downloaded tRNAscan-SE from &lt;a href=&quot;http://lowelab.ucsc.edu/tRNAscan-SE/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. After decompressing and untaring the file, I installed it using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./configure&#xA;make&#xA;make install&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I type &lt;code&gt;tRNAscan-SE --help&lt;/code&gt;, I get the help page:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tRNAscan-SE 2.0 (December 2017)&#xA;&#xA;FATAL: No sequence file(s) specified.&#xA;&#xA;Usage: tRNAscan-SE [-options] &amp;lt;FASTA file(s)&amp;gt;&#xA;&#xA;  Scan a sequence file for tRNAs &#xA;          -- default: use Infernal &amp;amp; tRNA covariance models&#xA;             with eukaryotic sequences &#xA;             (use -B, -A, -M, -O or -G to scan other types of sequences)&#xA;&#xA;Basic Options&#xA;  -E         : search for eukaryotic tRNAs (default)&#xA;  -B         : search for bacterial tRNAs&#xA;  -A         : search for archaeal tRNAs&#xA;  -M &amp;lt;model&amp;gt; : search for mitochondrial tRNAs&#xA;                 options: mammal, vert&#xA;  -O         : search for other organellar tRNAs&#xA;  -G         : use general tRNA model (cytoslic tRNAs from all 3 domains included)&#xA;  -L         : search using the legacy method (tRNAscan, EufindtRNA, and COVE)&#xA;                 use with -E, -B, -A, -O, or -G&#xA;  -I         : search using Infernal (default)&#xA;                 use with -E, -B, -A, -O, or -G&#xA;  -o &amp;lt;file&amp;gt;  : save final results in &amp;lt;file&amp;gt;&#xA;  -f &amp;lt;file&amp;gt;  : save tRNA secondary structures to &amp;lt;file&amp;gt;&#xA;  -m &amp;lt;file&amp;gt;  : save statistics summary for run in &amp;lt;file&amp;gt;&#xA;               (speed, # tRNAs found in each part of search, etc)&#xA;  -H         : show both primary and secondary structure components to&#xA;               covariance model bit scores&#xA;  -q         : quiet mode (credits &amp;amp; run option selections suppressed)&#xA;&#xA;  -h         : print full list (long) of available options&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I then tried to use it with &lt;code&gt;tRNAscan-SE c_elegans.PRJNA13758.WS263.genomic.fa&lt;/code&gt; but it gave the following error:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;FATAL: Unable to find /usr/local/bin/cmsearch executable&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After googling cmsearch, it seems that is is part of the &lt;a href=&quot;http://eddylab.org/infernal/Userguide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Infernal&lt;/a&gt; package. So, I installed infernal with conda using &lt;code&gt;conda install infernal&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I pulled the help from &lt;code&gt;cmsearch&lt;/code&gt; successfully:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cmsearch -h    &#xA;# cmsearch :: search CM(s) against a sequence database&#xA;# INFERNAL 1.1.2 (July 2016)&#xA;# Copyright (C) 2016 Howard Hughes Medical Institute.&#xA;# Freely distributed under a BSD open source license.&#xA;# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -&#xA;Usage: cmsearch [options] &amp;lt;cmfile&amp;gt; &amp;lt;seqdb&amp;gt;&#xA;&#xA;Basic options:&#xA;  -h        : show brief help on version and usage&#xA;  -g        : configure CM for glocal alignment [default: local]&#xA;  -Z &amp;lt;x&amp;gt;    : set search space size in *Mb* to &amp;lt;x&amp;gt; for E-value calculations  (x&amp;gt;0)&#xA;  --devhelp : show list of otherwise hidden developer/expert options&#xA;&#xA;Options directing output:&#xA;  -o &amp;lt;f&amp;gt;       : direct output to file &amp;lt;f&amp;gt;, not stdout&#xA;  -A &amp;lt;f&amp;gt;       : save multiple alignment of all significant hits to file &amp;lt;s&amp;gt;&#xA;  --tblout &amp;lt;f&amp;gt; : save parseable table of hits to file &amp;lt;s&amp;gt;&#xA;  --acc        : prefer accessions over names in output&#xA;  --noali      : don't output alignments, so output is smaller&#xA;  --notextw    : unlimit ASCII text output line width&#xA;  --textw &amp;lt;n&amp;gt;  : set max width of ASCII text output lines  [120]  (n&amp;gt;=120)&#xA;  --verbose    : report extra information; mainly useful for debugging&#xA;&#xA;Options controlling reporting thresholds:&#xA;  -E &amp;lt;x&amp;gt; : report sequences &amp;lt;= this E-value threshold in output  [10.0]  (x&amp;gt;0)&#xA;  -T &amp;lt;x&amp;gt; : report sequences &amp;gt;= this score threshold in output&#xA;&#xA;Options controlling inclusion (significance) thresholds:&#xA;  --incE &amp;lt;x&amp;gt; : consider sequences &amp;lt;= this E-value threshold as significant  [0.01]&#xA;  --incT &amp;lt;x&amp;gt; : consider sequences &amp;gt;= this score threshold as significant&#xA;&#xA;Options controlling model-specific reporting thresholds:&#xA;  --cut_ga : use CM's GA gathering cutoffs as reporting thresholds&#xA;  --cut_nc : use CM's NC noise cutoffs as reporting thresholds&#xA;  --cut_tc : use CM's TC trusted cutoffs as reporting thresholds&#xA;&#xA;Options controlling acceleration heuristics*:&#xA;  --max      : turn all heuristic filters off (slow)&#xA;  --nohmm    : skip all HMM filter stages, use only CM (slow)&#xA;  --mid      : skip first two HMM filter stages (SSV &amp;amp; Vit)&#xA;  --default  : default: run search space size-dependent pipeline  [default]&#xA;  --rfam     : set heuristic filters at Rfam-level (fast)&#xA;  --hmmonly  : use HMM only, don't use a CM at all&#xA;  --FZ &amp;lt;x&amp;gt;   : set filters to defaults used for a search space of size &amp;lt;x&amp;gt; Mb&#xA;  --Fmid &amp;lt;x&amp;gt; : with --mid, set P-value threshold for HMM stages to &amp;lt;x&amp;gt;  [0.02]&#xA;&#xA;Other options*:&#xA;  --notrunc     : do not allow truncated hits at sequence termini&#xA;  --anytrunc    : allow full and truncated hits anywhere within sequences&#xA;  --nonull3     : turn off the NULL3 post hoc additional null model&#xA;  --mxsize &amp;lt;x&amp;gt;  : set max allowed alnment mx size to &amp;lt;x&amp;gt; Mb [df: autodetermined]&#xA;  --smxsize &amp;lt;x&amp;gt; : set max allowed size of search DP matrices to &amp;lt;x&amp;gt; Mb  [128.]&#xA;  --cyk         : use scanning CM CYK algorithm, not Inside in final stage&#xA;  --acyk        : align hits with CYK, not optimal accuracy&#xA;  --wcx &amp;lt;x&amp;gt;     : set W (expected max hit len) as &amp;lt;x&amp;gt; * cm-&amp;gt;clen (model len)&#xA;  --toponly     : only search the top strand&#xA;  --bottomonly  : only search the bottom strand&#xA;  --tformat &amp;lt;s&amp;gt; : assert target &amp;lt;seqdb&amp;gt; is in format &amp;lt;s&amp;gt;: no autodetection&#xA;  --cpu &amp;lt;n&amp;gt;     : number of parallel CPU workers to use for multithreads&#xA;&#xA;*Use --devhelp to show additional expert options.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried the tRNAscan-SE command again but it failed with the same error message.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, tRNAscan-SE is looking for cmsearch in the wrong place. How can I make tRNAscan-SE use cmsearch successfully?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Update&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Forgot to tell you I am working on Mac OS X and that cmsearch is installed here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;/Users/user/anaconda/envs/python3env/bin/cmsearch&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and my &lt;code&gt;python3env&lt;/code&gt; anaconda environment is active.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-07-06T11:18:41.910" LastActivityDate="2018-10-14T14:02:07.053" Title="tRNAscan-SE error: FATAL: Unable to find /usr/local/bin/cmsearch executable" Tags="|sequence-annotation|software-installation|trna|trnascan-se|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4638" PostTypeId="2" ParentId="4637" CreationDate="2018-07-06T11:25:19.713" Score="2" Body="&lt;p&gt;&lt;strong&gt;A much cleaner solution is given in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/4639/298&quot;&gt;@bli's answer&lt;/a&gt;.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The error message is telling you that tRNAScan-SE is looking for the &lt;code&gt;cmsearch&lt;/code&gt; executable at &lt;code&gt;/usr/local/bin/cmsearch&lt;/code&gt;. However, you installed it though conda which apparently placed it at &lt;code&gt;/Users/user/anaconda/envs/python3env/bin/cmsearch&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A simple, if not very elegant solution, would be to make a link to it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sudo ln -s /Users/user/anaconda/envs/python3env/bin/cmsearch /usr/local/bin/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-07-06T15:52:28.600" LastActivityDate="2018-07-06T15:52:28.600" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4639" PostTypeId="2" ParentId="4637" CreationDate="2018-07-06T12:34:18.763" Score="8" Body="&lt;p&gt;You may try to set the correct path through the &lt;code&gt;infernal_dir&lt;/code&gt; variable in &lt;code&gt;tRNAscan-SE.conf.src&lt;/code&gt;. You might need to do this before the installation process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Original file starts with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# tRNAscan-SE 2.0 &#xA;# Configuration File&#xA;&#xA;# default paths&#xA;bin_dir: @bindir@&#xA;lib_dir: @libdir@/tRNAscan-SE&#xA;infernal_dir: {bin_dir}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Try the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# tRNAscan-SE 2.0 &#xA;# Configuration File&#xA;&#xA;# default paths&#xA;bin_dir: @bindir@&#xA;lib_dir: @libdir@/tRNAscan-SE&#xA;infernal_dir: &quot;/Users/user/anaconda/envs/python3env/bin&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="292" LastActivityDate="2018-07-06T12:34:18.763" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4640" PostTypeId="1" CreationDate="2018-07-07T15:04:46.250" Score="3" ViewCount="134" Body="&lt;p&gt;We all know that for physical protein-protein interaction, we need to find the distance between residues from PDB file of that interaction (finding distance between carbon alpha, carbon beta or centroid of two residues in PDB data of two proteins). I need to investigate the physical interaction between a protein and an RNA and I have PDB ID for this interaction but I don't exactly know to find the distance between which parts of residues (calpha, cbeta, centroid and etc.) with which parts of nucleotides (bases, phosphate group, or ribose backbone) from a PDB ID. Is there anyone who knows what can I do for this purpose?&lt;/p&gt;&#xA;" OwnerUserId="818" LastEditorUserId="2719" LastEditDate="2018-07-08T06:40:49.600" LastActivityDate="2020-04-12T15:02:10.407" Title="finding RNA-protein physical interaction" Tags="|pdb|interactions|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4641" PostTypeId="1" AcceptedAnswerId="4642" CreationDate="2018-07-07T21:32:47.347" Score="4" ViewCount="2896" Body="&lt;p&gt;Traditionally, RNA-seq data was quantified on gene level. Newer methods quantify on transcript/isoform level. For example, Kallisto only outputs transcript-level abundances. From the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2 vignette&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A newer and recommended pipeline is to use fast transcript abundance&#xA;  quantifiers upstream of DESeq2, and then to create gene-level count&#xA;  matrices for use with DESeq2 by importing the quantification data&#xA;  using the tximport package.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Is it just for consistency and/or simplicity that the values are converted to gene level? Would it better to proceed with transcript-level table or does that violate some DESeq2 (or similar tools) assumptions?&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2018-07-17T09:35:25.123" Title="gene-level versus transcript-level analysis" Tags="|rna-seq|transcriptome|deseq2|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4642" PostTypeId="2" ParentId="4641" CreationDate="2018-07-08T08:09:41.967" Score="6" Body="&lt;p&gt;There are a variety of reasons people use gene-level quantitations.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Transcript-level differences are difficult to biologically interpret. Let's be honest, few groups are likely to put in the work required to determine what these might mean. Most genes have at least some level of characterization, so it's much more biologically tractable to think in terms of gene-level changes. Granted, &quot;genes&quot; are largely artificial constructs and in an ideal world one would always think in terms of transcripts, since they're what's really changing. But we don't live in an ideal world.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Statistical power increases with counts, so if you're splitting your counts among a handful of isoforms you could be missing something. In practice this isn't a big problem, since there are typically obvious major isoforms, but that won't always be the case.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&quot;Mo transcripts mo problems&quot;, to channel the bioinformatics notorious B.I.G. It's common practice for people to use a subset of transcripts when using tools like Kallisto and Salmon, since the results usually end up being better. But which subset of transcripts should you use? Those in refseq? Gencode basic? A subset of one of those? Do we really understand all the consequences of choosing one over the others? (No, we don't) We all have our own preferences, but this isn't a solved problem.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-07-08T08:09:41.967" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4643" PostTypeId="2" ParentId="4626" CreationDate="2018-07-08T08:56:10.713" Score="2" Body="&lt;p&gt;You have 2 options provided that both datasets have the same genes (rows):&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Keep the original order of rows in the heatmap. If they’re in the same order in the dataset then they will be in the heatmap. You can suppress reordering with any of the following arguments to &lt;code&gt;heatmap.2&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Rowv = NULL&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;dendrogram = “col”&lt;/code&gt; or &lt;code&gt;dendrogram = “none”&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;reorderfun = function(x) return(x)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;You can compute the order of the genes (rows) in advance and pass them to the heatmap. The code shown in the question does this already with &lt;code&gt;Rowv=as.dendrogram(hr)&lt;/code&gt;. To have the same order in a second heatmap, you have to pass the same same dendrogram &lt;code&gt;hr&lt;/code&gt; to both heatmaps. This can be computed from either dataset or another set of samples altogether.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Another option is to compute separate dendrograms (from &lt;code&gt;hclust&lt;/code&gt; of each dataset) for the samples and combine them using &lt;code&gt;merge&lt;/code&gt;. You can combine the datasets with &lt;code&gt;cbind&lt;/code&gt; and plot the datasets together as one heatmap (with 2 sets of samples clustered separately) by passing the merged dendrogram to &lt;code&gt;Colv&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#compute same dendrograms separately&#xA;hc1 &amp;lt;- hclust(as.dist(t(data1), method=&quot;euclidean&quot;)), method=&quot;complete&quot;)&#xA;hr2 &amp;lt;- hclust(as.dist(t(data2), method=&quot;euclidean&quot;)), method=&quot;complete&quot;)&#xA;mergedcols &amp;lt;- merge(as.dendrogram(hr1), as.dendrogram(hr2))&#xA;mergeddata &amp;lt;- cbind(data1, data2)&#xA;#compute gene dendrogram across all samples&#xA;hr &amp;lt;- hclust(as.dist(1-cor(t(mergeddata), method=&quot;pearson&quot;)), method=&quot;complete&quot;)&#xA;heatmap.2(&#xA;mergeddata, &#xA;Rowv=as.dendrogram(hr), &#xA;Colv=as.dendrogram(mergedcols), &#xA;col=myCol, &#xA;breaks=myBreaks, &#xA;main=&quot;Title&quot;, &#xA;key=T, &#xA;keysize=1.0, &#xA;scale=&quot;none&quot;, &#xA;density.info=&quot;none&quot;, &#xA;reorderfun=function(d,w) reorder(d, w, agglo.FUN=mean), &#xA;trace=&quot;none&quot;, &#xA;cexRow=0.2, &#xA;cexCol=0.8, &#xA;distfun=function(x) dist(x, method=&quot;euclidean&quot;), &#xA;hclustfun=function(x) hclust(x, method=&quot;ward.D2&quot;)&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You should be able to show the division between these samples with the &lt;code&gt;colsep&lt;/code&gt; argument.&lt;/p&gt;&#xA;" OwnerUserId="1855" LastActivityDate="2018-07-08T08:56:10.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4644" PostTypeId="1" CreationDate="2018-07-08T11:26:20.370" Score="3" ViewCount="63" Body="&lt;p&gt;Assuming I have SNPs data using hg19, how can I know which SNP belongs into which Gene? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr10_103577643&#xA;chr10_124712463&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and so on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to add a column of Gene, which would tell to which Gene the SNP belongs. &#xA;The file is not a VCF file but a Txt file. &lt;/p&gt;&#xA;" OwnerUserId="3039" LastEditorUserId="3039" LastEditDate="2018-07-09T16:21:07.463" LastActivityDate="2018-07-09T16:21:07.463" Title="Associating SNP and GENE" Tags="|gene|snp|hgp|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4645" PostTypeId="2" ParentId="4631" CreationDate="2018-07-09T04:48:21.887" Score="1" Body="&lt;p&gt;This is really tough to do with highly heterozygous animals. What are your genome assembly stats? Specifically, what is your number of contigs, scaffolds, assembly size, and the N50?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have proximity ligation data it will be easier to determine if potential paralogs are truly from different regions of the genome or are just from homologous locations on sister chromatids that ended up in uncollapsed in your final assembly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I were you and had a good genome, I would start by annotating gene models using RNAseq data, then blasting each gene in the model against all genes in the model. This would give you potential paralogs to start looking at more closely.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-07-09T04:48:21.887" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4646" PostTypeId="2" ParentId="4644" CreationDate="2018-07-09T07:47:20.313" Score="3" Body="&lt;p&gt;Use the &lt;a href=&quot;http://grch37.ensembl.org/info/docs/tools/vep/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Variant Effect Predictor(VEP)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2018-07-09T07:47:20.313" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4647" PostTypeId="2" ParentId="4644" CreationDate="2018-07-09T08:21:43.623" Score="2" Body="&lt;p&gt;You can use &lt;a href=&quot;http://snpeff.sourceforge.net/SnpEff_manual.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;snpEff&lt;/a&gt; or &lt;a href=&quot;http://snpeff.sourceforge.net/SnpSift.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;snpSIFT&lt;/a&gt; java annotator also for hg19.&lt;/p&gt;&#xA;" OwnerUserId="1453" LastActivityDate="2018-07-09T08:21:43.623" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4648" PostTypeId="1" CreationDate="2018-07-09T11:14:17.577" Score="2" ViewCount="78" Body="&lt;p&gt;I have used the command-line version of &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC146525/pdf/250955.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;tRNAscan-SE&lt;/a&gt; by Lowe et al. (1997) on the reference genome sequence of &lt;em&gt;C. elegans&lt;/em&gt; to predict tRNAs. The programme outputs a table with tRNA locations, amino acid, anti codon, intro locations, tRNA type (functional or pseudogene) along with a score. The score in my dataset ranges from 20.0 to 107.4. After reading &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4987944/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the article&lt;/a&gt; Lowe et al (2016) describing this tool, I still do not understand how this score comes about and what it means. I understand that:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the higher the score, the more likely the tRNA is assigned to the correct isotype (amino acid)&lt;/li&gt;&#xA;&lt;li&gt;the lower the score, the more likely the tRNA is a pseudogene&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How is the score calculated?&lt;/li&gt;&#xA;&lt;li&gt;What does the score mean?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The authors of the tool state that is is 99% sensitive and detects 1 false tRNA in 15 GB of DNA. Does this mean that I do not need to filter my results? What about the confidence in the type of tRNA and whether it is functional or pseudogenised?&lt;/p&gt;&#xA;&lt;p&gt;Lowe et al (1997) tRNAscan-SE:  a  program  for  improved  detection  of transfer  RNA  genes  in  genomic  sequence. Nuc. Acids 25&#xA;Lowe et al (2016) tRNAscan-SE On-line: integrating search and context for analysis of transfer RNA genes. Nuc Acids Web Server issue: W54–W57.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="3848" LastEditDate="2020-08-02T22:06:20.830" LastActivityDate="2020-08-02T22:06:20.830" Title="What is tRNAscan-SE bit score?" Tags="|sequence-annotation|trna|trnascan-se|bitscore|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4649" PostTypeId="1" AcceptedAnswerId="4653" CreationDate="2018-07-09T11:29:04.277" Score="3" ViewCount="291" Body="&lt;p&gt;Suppose my genome is 3 million bases and that my reads are 100 nucleotide long. I need to know how many reads I need to cover the entire genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I start from using the equation $C = \frac{N \cdot L}{G}$ where C is the coverage, N the number of reads, L the length of a read and G the length of the haploid genome. I also know that the average coverage follows a Poisson distribution, so I need this for C.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On my slides, in fact, I have the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$&#xA;C = \frac{N \cdot L}{G} \approx ln \left( \frac{G}{L \cdot \epsilon}\right),&#xA;$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if we assume that this is the average coverage needed to cover the entire genome with probability $1-\epsilon$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that I do not understand how to arrive at the expression on the right. I thought that if we want to be sure to have a coverage of at least 1 on average, we would have had to compute $1 - P(X=0) = 1 - e^{-\frac{N \cdot L}{G}} = 0.99$, if we take $\epsilon=0.01$. But I do not get the same result as with the equation above.&lt;/p&gt;&#xA;" OwnerUserId="2862" LastActivityDate="2018-07-09T15:47:24.203" Title="How many reads do I need to cover the entire genome?" Tags="|ngs|coverage|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4650" PostTypeId="1" AcceptedAnswerId="4667" CreationDate="2018-07-09T13:20:59.233" Score="3" ViewCount="46" Body="&lt;p&gt;I am looking for the most appropriate dataset for downloading baseline gene expression level across all human coding genes during development. I am aware that &lt;a href=&quot;https://www.ebi.ac.uk/gxa/home&quot; rel=&quot;nofollow noreferrer&quot;&gt;EMBL Expression Atlas&lt;/a&gt; is one of the resources that provide such information, but I am ideally looking for species averaged (and probably scaled) expression data for all human coding genes during early stages of development!&#xA;Any insight is much appreciated! &lt;/p&gt;&#xA;" OwnerUserId="1250" LastActivityDate="2018-07-11T12:19:15.000" Title="Where to download baseline/average gene expression level of all human coding genes?" Tags="|public-databases|data-retrieval|gene-expression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4652" PostTypeId="1" AcceptedAnswerId="4654" CreationDate="2018-07-09T15:26:25.487" Score="2" ViewCount="70" Body="&lt;p&gt;I've got a fasta file with some RNA seq data and another csv file with the output from plast where I've aligned it to a reference using plastn. I'm struggling with figuring out a command to append my original RNA seq data fasta headers with the annotations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've looked at awk and sed but can't wrap my head around how to use them in this instance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CSV output example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Contig11    ref|XM_021487230.1|_PREDICTED:_Mizuhopecten_yessoensis_UNC93-like_protein_MFSD11_(LOC110443183),_mRNA   78.95   95  20  0   371 465 4627    4721    1e-14   82.4    75  625 1   0   15.2    0   5692    1   0   15.2    0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And example fasta contig:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Contig11&#xA;GACTCTCTTAAGGTAGCCAAATGCCTCGTCATCTAATTAGTGACGCGCATGAATGGATTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How would I be able to parse through the csv to find a matching contig with the fasta file, and append the second csv column to the fasta header. Ideally to produce:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Contig11 ref|XM_021487230.1|_PREDICTED:_Mizuhopecten_yessoensis_UNC93-like_protein_MFSD11_(LOC110443183),_mRNA&#xA;GACTCTCTTAAGGTAGCCAAATGCCTCGTCATCTAATTAGTGACGCGCATGAATGGATTA....    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Cheers for any help!&lt;/p&gt;&#xA;" OwnerUserId="3042" LastEditorUserId="73" LastEditDate="2023-01-15T20:21:09.420" LastActivityDate="2023-01-15T20:21:09.420" Title="RNA seq fasta file annotation from alignment to reference matches" Tags="|rna-seq|sequence-annotation|awk|functional-annotation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4653" PostTypeId="2" ParentId="4649" CreationDate="2018-07-09T15:47:24.203" Score="3" Body="&lt;p&gt;Let's see what your slides are claiming $\epsilon$ to be, using:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ C = \frac{NL}{G} \approx \ln \left( \frac{G}{L\epsilon} \right) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can rearrange to get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \epsilon \approx \frac{G}{L} \times e^{-C} $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, what does it mean for an entire genome to be covered? Having a clear understanding of this definition is crucial, and your misunderstanding seems to partially stem from this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I claim it's natural to consider a genome as consisting of $G/L$ bins. In this setting, we have $N$ balls that each randomly goes into a bin. Then, the entire genome is covered when all bins contain at least $1$ ball/read.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the Poisson distribution, we know that P(a bin has 0 balls) = $e^{-C}$.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, we define $\epsilon$ to be the probability that at least one of the $G/L$ bins has 0 balls. Given the way we've modeled the problem, this is exactly equal to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \epsilon = 1 - (1 - e^{-C})^{G/L}. $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From above, your slides are claiming $ \epsilon \approx \frac{G}{L} \times e^{-C}$. This might be strange, as it appears to be equal to the expected number of bins with 0 balls, instead of the probability that there is at least 1 bin with 0 balls. Furthermore, it's strange because it contains only one exponential, compared to our exact derivation which has a double exponential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mismatch in the number of exponential is a hint that another mathematical approximation is at play. We can use a &lt;a href=&quot;https://en.wikipedia.org/wiki/Binomial_series&quot; rel=&quot;nofollow noreferrer&quot;&gt;first-order approximation of the binomial series&lt;/a&gt; to obtain&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ (1-e^{-C})^{G/L} \approx 1 - (G/L)e^{-C} $$ which holds particularly when $|e^{-C}| \times \frac{G}{L} &amp;lt;&amp;lt; 1$. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case, $\frac{G}{L} = 30,000$, so the approximation only really starts getting accurate when $C &amp;gt; 15$ or so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following through with our approximation, we get&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \epsilon \approx 1 - (1 - (G/L) e^{-C}) $$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$ \epsilon \approx \frac{G}{L} \times e^{-C}. $$&lt;/p&gt;&#xA;" OwnerUserId="3043" LastActivityDate="2018-07-09T15:47:24.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4654" PostTypeId="2" ParentId="4652" CreationDate="2018-07-09T15:56:27.467" Score="1" Body="&lt;p&gt;An &lt;code&gt;awk&lt;/code&gt; approach (assuming your input &quot;csv&quot; file is space-separated):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk '(NR==FNR){a[&quot;&amp;gt;&quot;$1]=$2; next;} (/&amp;gt;/){$1=$1&quot; &quot;a[$1]}1;' foo.csv foo.fa &#xA;&amp;gt;Contig11 ref|XM_021487230.1|_PREDICTED:_Mizuhopecten_yessoensis_UNC93-like_protein_MFSD11_(LOC110443183),_mRNA ref|XM_021487230.1|_PREDICTED:_Mizuhopecten_yessoensis_UNC93-like_protein_MFSD11_(LOC110443183),_mRNA&#xA;GACTCTCTTAAGGTAGCCAAATGCCTCGTCATCTAATTAGTGACGCGCATGAATGGATTA&#xA;GACTCTCTTAAGGTAGCCAAATGCCTCGTCATCTAATTAGTGACGCGCATGAATGGATTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Explanation&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;NR==FNR&lt;/code&gt; : &lt;code&gt;FNR&lt;/code&gt; is the line number of the current input file and &lt;code&gt;NR&lt;/code&gt; is the line number of all input. When processing more than one file, the two will only be equal while reading the first file. &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;(NR==FNR){a[&quot;&amp;gt;&quot;$1]=$2; next;}&lt;/code&gt; : if we're reading the first file, the csv, save the second field as the value in the associative array &lt;code&gt;a&lt;/code&gt;, whose indices are a &lt;code&gt;&amp;gt;&lt;/code&gt; followed by the 1st field. I use the &lt;code&gt;&amp;gt;&lt;/code&gt; only to simplify matching it later in the fasta file. The &lt;code&gt;next&lt;/code&gt; moves on to the next line so the rest of the command isn't run for the 1st file.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;(/&amp;gt;/){$1=$1&quot; &quot;a[$1]}&lt;/code&gt;: Now we're in the 2nd file. If the line starts with a &lt;code&gt;&amp;gt;&lt;/code&gt;, then set the 1st field to be itself, a space and then whatever was stored in the array &lt;code&gt;a&lt;/code&gt; for this 1st field.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;1;&lt;/code&gt; : this is awk shorthand for &quot;print this line&quot;. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You could expand the above into:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{&#xA;        if(NR==FNR){&#xA;            a[&quot;&amp;gt;&quot;$1]=$2; &#xA;        }&#xA;        else{&#xA;            if(/&amp;gt;/){&#xA;                $1=$1&quot; &quot;a[$1]&#xA;            }&#xA;        }&#xA;        print;&#xA;      }' foo.csv foo.fa &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If your files are small enough to be sorted easily, you can use the &lt;code&gt;FastaToTbl&lt;/code&gt; and &lt;code&gt;TblToFasta&lt;/code&gt; scripts &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2651/298&quot;&gt;I have posted before&lt;/a&gt; and then &lt;code&gt;join&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ join -o 1.1,2.2,1.2 &amp;lt;(FastaToTbl foo.fa | sort ) &amp;lt;(sort  foo.csv ) | TblToFasta &#xA;&amp;gt;Contig11 ref|XM_021487230.1|_PREDICTED:_Mizuhopecten_yessoensis_UNC93-like_protein_MFSD11_(LOC110443183),_mRNA &#xA;GACTCTCTTAAGGTAGCCAAATGCCTCGTCATCTAATTAGTGACGCGCATGAATGGATTA&#xA;GACTCTCTTAAGGTAGCCAAATGCCTCGTCATCTAATTAGTGACGCGCATGAATGGATTA&#xA;&amp;gt;Contig12 mamamama|_PREDICTED:_Mizuhopecten_yessoensis_UNC93-like_protein_MFSD11_(LOC110443183),_mRNA &#xA;AKHSALKJHDLKAJBDLKAJDBADADKJALDKJBAD&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-07-09T16:24:50.520" LastActivityDate="2018-07-09T16:24:50.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4655" PostTypeId="1" AcceptedAnswerId="4659" CreationDate="2018-07-09T22:36:55.690" Score="2" ViewCount="385" Body="&lt;p&gt;I'm working with RNA-seq data. I have 40 tumor samples and 5 Normal samples. Differential analysis with &lt;code&gt;Deseq2&lt;/code&gt; based on Fold change &gt; 1.2 and alpha &amp;lt; 0.05 gave very low number of differentially expressed genes. Only 2 upregulated genes. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;res &amp;lt;- results(dds, lfcThreshold = log2(1.2), alpha = 0.05)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To get more number of differential expressed genes I have few questions now &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Instead of FDR &amp;lt; 0.05 can I use FDR &amp;lt; 0.1 (or) FDR &amp;lt; 0.5. Will there be any low confidence with this?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Can I select differential expressed genes only based on FDR &amp;lt; 0.05 without any fold change cutoff?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As I get very low number of DEGs with FDR &amp;lt; 0.05 &amp;amp; Foldchange &gt; 1.2, Can I select DEGs based on Foldchange &gt; 1.2 and p.value &amp;lt; 0.01 or 0.05 ?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;PCA of the samples looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Gfxbq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Gfxbq.png&quot; alt=&quot;PCA plot Tumor vs Normal&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1369" LastEditorUserId="29" LastEditDate="2018-07-10T15:39:15.993" LastActivityDate="2018-07-10T17:15:37.113" Title="Selection of differential expressed genes" Tags="|r|rna-seq|differential-expression|deseq2|rna|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4656" PostTypeId="2" ParentId="4655" CreationDate="2018-07-09T22:45:55.040" Score="0" Body="&lt;p&gt;In general, messing with FDR or log fold change cutoffs beyond the ones you used gets pretty sketchy. I usually only adjust these values in the more conservative direction, although you may be able to get away with FDR &amp;lt; 0.1 if you have a large effect size (which you don't).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might consider investigating why you don't seem to get any differentially expressed genes before adjusting your cutoffs: PCA on the samples based on their top 1,000 most variably expressed genes would be a good start. Do the tumor and normal transcriptomes cluster apart from each other?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With so few normal samples, it is possible that you are not fully characterizing the variability of that population and it is affecting your ability to identify meaningful differences between normal and tumor.&lt;/p&gt;&#xA;" OwnerUserId="643" LastActivityDate="2018-07-09T22:45:55.040" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4657" PostTypeId="1" CreationDate="2018-07-10T01:42:45.003" Score="4" ViewCount="412" Body="&lt;p&gt;I am building a &lt;em&gt;De Novo&lt;/em&gt; transcriptome reference assembly for an eukaryotic organism for which I have a genome.&lt;/p&gt;&#xA;&lt;p&gt;I've created several assemblies with rnaSpades using different kmer sizes (19 to 69 with step 10). Now I would like to merge them into one final transcriptome.&lt;/p&gt;&#xA;&lt;p&gt;How could I do that?&lt;/p&gt;&#xA;&lt;p&gt;Is using a genome assembly reconciliation tool such as metassembler a good idea?&lt;/p&gt;&#xA;" OwnerUserId="3044" LastEditorUserId="3848" LastEditDate="2020-07-18T13:14:28.020" LastActivityDate="2021-08-14T07:03:05.257" Title="Merge / Reconciliate several de novo transcriptome assemblies with different kmers" Tags="|assembly|transcriptome|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4658" PostTypeId="1" CreationDate="2018-07-10T11:37:57.280" Score="3" ViewCount="43" Body="&lt;p&gt;I have an annotated transcriptome and would like to develop PCR primers for particular transcribed genes. My species is a non-model plant. Can I use BLAST or another tool to identify potential PCR primer sequences?  Or more generally, is there a workable bioinformatics approach to identifying potential PCR primers for transcripts arising from a non-model organism? Any literature refs would be great too.&lt;/p&gt;&#xA;" OwnerUserId="2196" LastActivityDate="2018-07-10T14:23:59.703" Title="Bioinformatics approach to dentifing potential PCR primer sequences for transcribed gene" Tags="|blast|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4659" PostTypeId="2" ParentId="4655" CreationDate="2018-07-10T13:07:58.930" Score="3" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You can usually get away with FDR &amp;lt; 0.1, but that's as high as you can go. This all presumes you're doing follow-up experiments of some sort, of course. I guess you could increase the FDR more, but you're then really increasing the odds that your follow-up experiments will fail. Obviously increasing the FDR will decrease the confidence in the results, it's called &quot;false discovery rate&quot; for a reason.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Yes, fold-change cut-offs are mostly useful when you need to prioritize hits further. Small changes are almost impossible to validate with less-sensitive methods (qPCR) and less likely to be biologically relevant.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You're just going to waste your time if you ignore FDR.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Remember that all thresholds are intended to increase the odds of success in follow-up experiments. If you have infinite time and resources then you can set these to whatever you want, but you have neither so you better prioritize reasonably.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's likely that you have a large batch effect in your data, effect sizes in cancer tend to be much larger than this.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-07-10T13:07:58.930" CommentCount="15" ContentLicense="CC BY-SA 4.0" />
  <row Id="4660" PostTypeId="2" ParentId="4658" CreationDate="2018-07-10T14:23:59.703" Score="2" Body="&lt;p&gt;It depends if you want to amplify your target from genomic DNA or cDNA. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on intron size, primers that anneal to different exons may not be able to amplify across the intron(s), and if you're unlucky enough to design a primer that spans a splice junction, you'll definitely be stuffed if you're working with genomic DNA. I would try blastn or translate to an amino acid sequence and use tblastn to see if you can identify the intron/exon structure in a related species. If you don't have a genome sequence available for anything close, I think your only option is to target as small an amplicon as possible and hope for the best.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are working with cDNA, any &lt;a href=&quot;https://www.eurofinsgenomics.eu/en/ecom/tools/pcr-primer-design&quot; rel=&quot;nofollow noreferrer&quot;&gt;standard primer design tool&lt;/a&gt; should do&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-07-10T14:23:59.703" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4661" PostTypeId="1" AcceptedAnswerId="4784" CreationDate="2018-07-10T15:23:31.783" Score="3" ViewCount="74" Body="&lt;p&gt;I want to use &lt;a href=&quot;http://www.vital-it.ch/software/tral&quot; rel=&quot;nofollow noreferrer&quot;&gt;TRAL&lt;/a&gt; to annotate tandem repeats in the reference genome of &lt;em&gt;Caenorhabditis elegans&lt;/em&gt;. For this, I need to install some external software, such as &lt;a href=&quot;http://www.ruhr-uni-bochum.de/ecoevo/cm/cm_phobos.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Phobos&lt;/a&gt;. I've downloaded Phobos and I am using it by typing the path to it's executable like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;input/software/phobos-v3.3.12-mac/bin/phobos-mac-intel-64bit-start-from-command-line genome.fa output.txt&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This works quite well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order for TRAL to find Phobos, I changed TRAL's configuration file as explained &lt;a href=&quot;http://www.vital-it.ch/software/tral&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've replaced &lt;code&gt;PHOBOS = phobos&lt;/code&gt; with &lt;code&gt;PHOBOS = /Users/user/Documents/.../project/input/software/phobos-v3.3.12-mac/bin/phobos-mac-intel-64bit-start-from-command-line&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, I followed the tutorial &lt;a href=&quot;http://www.vital-it.ch/software/tral&quot; rel=&quot;nofollow noreferrer&quot;&gt;Run and parse de novo repeat detection software&lt;/a&gt; and typed the following commands in Python 3 in the Terminal:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;from tral.sequence import sequence&#xA;from tral.paths import PACKAGE_DIRECTORY&#xA;&#xA;proteome_HIV = os.path.join(PACKAGE_DIRECTORY, &quot;examples&quot;, &quot;data&quot;, &quot;HIV-1_388796.faa&quot;)&#xA;sequences_HIV = sequence.Sequence.create(file = proteome_HIV, input_format = 'fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So far so good. But the following command creates an error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tandem_repeats = sequences_HIV[0].detect(denovo = True, detection = {&quot;detectors&quot;: [&quot;Phobos&quot;]})&#xA;&#xA;Did not find Phobos result file in /var/folders/z2/rt6qxntd3tg5bjmqlqnm0mdm0000gn/T/tmpvc20y3i4/001/PHOBOS/phobos.o&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that if I use &lt;code&gt;PHOBOS&lt;/code&gt; instead of &lt;code&gt;Phobos&lt;/code&gt; in the command, I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &quot;&amp;lt;stdin&amp;gt;&quot;, line 1, in &amp;lt;module&amp;gt;&#xA;  File &quot;/Users/user/anaconda/envs/python3env/lib/python3.6/site-packages/tral/sequence/sequence.py&quot;, line 162, in detect&#xA;    **kwargs['detection'])[0]&#xA;  File &quot;/Users/user/anaconda/envs/python3env/lib/python3.6/site-packages/tral/sequence/repeat_detection_run.py&quot;, line 962, in run_detector&#xA;    Detectors(detectors, sequence_type)&#xA;  File &quot;/Users/user/anaconda/envs/python3env/lib/python3.6/site-packages/tral/sequence/repeat_detection_run.py&quot;, line 872, in Detectors&#xA;    FINDER_LIST.keys()))&#xA;Exception: Unknown TR detector supplied (Supplied: ['PHOBOS']. Known TR detectors: dict_keys(['HHrepID', 'Phobos', 'TRED', 'T-REKS', 'TRF', 'TRUST', 'XSTREAM']))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am working on Mac OS X. I installed TRAL with &lt;code&gt;pip install tral&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-07-27T13:07:54.097" Title="TRAL does not find &quot;Phobos result file&quot;" Tags="|sequence-annotation|software-installation|repeat-elements|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4662" PostTypeId="1" CreationDate="2018-07-10T16:51:29.027" Score="0" ViewCount="293" Body="&lt;p&gt;I have installed &lt;a href=&quot;http://globin.bx.psu.edu/dist/gmaj/gmaj_install.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GMAJ&lt;/a&gt; using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wget http://globin.bx.psu.edu/dist/gmaj/gmaj.zip  -O output/software/gmaj.zip&#xA;unzip -u output/software/gmaj.zip -d output/software/gmaj/&#xA;rm output/software/gmaj.zip&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I ran &lt;a href=&quot;http://www.bx.psu.edu/miller_lab/dist/README.lastz-1.02.00/README.lastz-1.02.00a.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;lastz&lt;/a&gt; on two chromosomes from two different assemblies of &lt;em&gt;C. elegans&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;lastz c_elegans.PRJNA13758.WS263.genomicI.fa I.fa --format=maf &amp;gt; MAF.maf&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and got a MAF file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, I tried to visualise the file using:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;java -jar output/software/gmaj/gmaj.jar MAF.maf&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I got the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Exception in thread &quot;main&quot; java.lang.ClassCastException: java.lang.StringBuffer cannot be cast to java.lang.String&#xA;    at edu.psu.bx.gmaj.MajGui.setDefaults(MajGui.java:272)&#xA;    at edu.psu.bx.gmaj.Maj.&amp;lt;init&amp;gt;(Maj.java:53)&#xA;    at edu.psu.bx.gmaj.MajMain.main(MajMain.java:86)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can someone help me resolve this error or suggest an alternative way to visualise my alignment?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I work on Mac OS X.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-07-11T09:50:52.997" Title="GMAJ error: Exception in thread &quot;main&quot; java.lang.ClassCastException: java.lang.StringBuffer cannot be cast to java.lang.String" Tags="|sequence-alignment|visualization|software-installation|maf|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4663" PostTypeId="2" ParentId="4655" CreationDate="2018-07-10T17:15:37.113" Score="1" Body="&lt;p&gt;I think since your PCA shows that your normal samples cluster right alongside the tumor ones, you have to conclude that there is much more variation between tumors than between tumors and controls.  DESeq's differentially expressed gene list was generated properly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your results are what they are, sorry they are so useless.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2018-07-10T17:15:37.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4664" PostTypeId="1" AcceptedAnswerId="4668" CreationDate="2018-07-10T19:54:53.223" Score="4" ViewCount="658" Body="&lt;p&gt;I am trying to install psipred in a unix server.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Psipred is installed, but fails because it can not find the correct database files.  I have downloaded uniprod90.fasta. Psipred also loads the old legacy ncbi BLAST-2.2.26 and BLAST+.&lt;br&gt;&#xA;&lt;code&gt;runpsipred&lt;/code&gt; and &lt;code&gt;runpsipredplus&lt;/code&gt; have been patched to call the correct version of BLAST. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have run a few commands to index the uniprod90 database: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pfilt uniref90.fasta &amp;gt; uniref90filt&#xA;formatdb -t uniref90filt -i uniref90filt&#xA;makeblastdb -in uniref90.fasta -dbtype prot -input_type fasta -out uniref90.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I then ran &lt;code&gt;psipred&lt;/code&gt; and got this error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ./psipred/example/example.fasta&#xA;Running PSI-BLAST with sequence  ./psipred/example/example.fasta ...&#xA;[blastpgp] WARNING: Unable to open uniref90.pin&#xA;[blastpgp] WARNING: Unable to open uniref90.pin&#xA;FATAL: Error whilst running blastpgp - script terminated!&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;makeblastdb&lt;/code&gt; created 26 pin files, but not uniref90.pin.&lt;/p&gt;&#xA;" OwnerUserId="2970" LastEditorUserId="-1" LastEditDate="2020-04-14T16:26:29.840" LastActivityDate="2020-04-14T16:26:29.840" Title="Psipred installation" Tags="|software-installation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4665" PostTypeId="2" ParentId="4662" CreationDate="2018-07-11T09:50:52.997" Score="1" Body="&lt;p&gt;It's a programming error that should look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;StringBuffer sb = (...)&#xA;(...)&#xA;String s = (String)sb;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;while it should be&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;StringBuffer sb = (...)&#xA;(...)&#xA;String s = sb.toString();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;( see also &lt;a href=&quot;https://www.biostars.org/p/177992/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/177992/&lt;/a&gt; ) . You cannot do anything about it but asking the authors of the software.&lt;/p&gt;&#xA;" OwnerUserId="71" LastActivityDate="2018-07-11T09:50:52.997" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4667" PostTypeId="2" ParentId="4650" CreationDate="2018-07-11T12:19:15.000" Score="3" Body="&lt;p&gt;There is not going to be a public database with this data. Apart from anything else, generating data for early human development is difficult and ethically tricky. Also, most people who care about development would probably think that a whole body average would be meaningless. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best you could do would be to select tissues you were interested in from something like Expression Atlas and calculate the averages yourself. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-07-11T12:19:15.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4668" PostTypeId="2" ParentId="4664" CreationDate="2018-07-11T13:23:46.740" Score="4" Body="&lt;p&gt;Try with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;makeblastdb -in uniref90.fasta -dbtype prot -input_type fasta -out uniref90&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;-out&lt;/code&gt; is the name of the DB. So if you use &lt;code&gt;uniref90.fasta&lt;/code&gt;, it creates &lt;code&gt;&lt;/code&gt;uniref90.fasta.pin&lt;code&gt;while the script seems to be expecting&lt;/code&gt;uniref90.pin`.&lt;/p&gt;&#xA;" OwnerUserId="1642" LastEditorUserId="298" LastEditDate="2018-07-12T08:27:51.123" LastActivityDate="2018-07-12T08:27:51.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4671" PostTypeId="1" AcceptedAnswerId="4672" CreationDate="2018-07-11T22:39:50.740" Score="1" ViewCount="284" Body="&lt;p&gt;Posting here because the reddit thread I made didn't gain any traction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm having a little bit of trouble understanding how the odds ratio was calculated in this table, specifically the OR marked in red:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nQYVo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nQYVo.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my understanding, an odds ratio = $$[(\text{# of cases with exposure}) / (\text{# of controls with exposure})] / [(\text{# of cases without exposure}) / (\text{# of controls without exposure})]$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the red-highlighted case, my odds ratio is&lt;/p&gt;&#xA;&#xA;&lt;p&gt;$$(0.661 / 0.6580) / [(1-0.661) / (1-0.6580)] = 1.0134$$&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which is very off from $0.98$. Also, I know I'm using frequencies rather than absolute counts, but it shouldn't matter because if I were to multiply each frequency by the corresponding population values, all the population values would cancel out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what have I done wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1016/j.cell.2018.05.046&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link to the paper&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3054" LastEditorUserId="48" LastEditDate="2018-07-12T07:20:35.617" LastActivityDate="2018-07-12T07:20:35.617" Title="Odds ratio calculations in GWAS paper" Tags="|gwas|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4672" PostTypeId="2" ParentId="4671" CreationDate="2018-07-11T23:38:28.293" Score="2" Body="&lt;p&gt;Those odds ratio aren't calculated from the raw allele frequencies. They're calculated from coefficients estimated from complex linear models with a bunch of covariates that are run on subsets of the data then meta-analysed together. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a bit surprising that the odds ratio is less than one when the frequency is higher in cases, but the p-value indicates that the OR is not significantly different from 1 so it may be down to statistical noise. It's possible that within populations one allele is more common in cases than in controls but that cases are primarily sampled from populations where that allele is uncommon, causing it to be more common in controls overall.  See &lt;a href=&quot;https://en.wikipedia.org/wiki/Simpson%27s_paradox&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Alternatively, 1/0.98 is pretty close to the value you calculated, so it's possible that the reference allele got switched around at some point.&lt;/p&gt;&#xA;" OwnerUserId="581" LastEditorUserId="581" LastEditDate="2018-07-12T00:04:53.140" LastActivityDate="2018-07-12T00:04:53.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4673" PostTypeId="1" CreationDate="2018-07-12T08:24:42.510" Score="4" ViewCount="60" Body="&lt;p&gt;I have recently started working on a substance's effect on a cell line in different dosages. for this, there is a tool called &lt;a href=&quot;https://www.sciome.com/bmdexpress/&quot; rel=&quot;nofollow noreferrer&quot;&gt;bmdexpress2&lt;/a&gt; that I am using. Its input is the normalized counts from RNASeq for each dosage as a big matrix. When it comes to the pathway analysis step, unlike hallmarks of GSEA, this tool uses some databases for defining pathways which involve pathways of even 2-3 genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what I want to discuss here is the strength of small pathways and filtering thresholds. How can we bioinformatically decide on the importance of a pathway made of 3 genes? Should we just filter them out as there are much more comprehensive pathways? Or do their GO Levels matter? Also there are many cases that out of those 3 genes, 1 of them is differentially expressed; is having 1 gene diff. expressed out of X genes in a pathway enough to say that the pathway is enriched?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an &lt;a href=&quot;https://www.ebi.ac.uk/QuickGO/term/GO:0001966&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt; GO term for a small pathway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here is an example of the output I get from the tool. all genes platform column is the number of genes involved in that pathway, input genes column is how many of the genes in that pathway are significantly differentiated. I believe the rest is self explanatory. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QFaPd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QFaPd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at the table, can we really say, ESCRT III Complex, for example is a good pathway to focus while having 1 gene out of 13 in a pathway but that change is significant?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="3056" LastEditorUserId="3056" LastEditDate="2018-07-12T14:57:14.877" LastActivityDate="2018-07-12T15:12:30.907" Title="size of the pathways for analysis and filtration" Tags="|rna-seq|pathway|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4676" PostTypeId="2" ParentId="4673" CreationDate="2018-07-12T15:12:30.907" Score="4" Body="&lt;p&gt;The problem here is that any one GO term with a small number of members is unlikely to include any differentially expressed genes by chance, but if you test a large number of GO terms, some of them definitley will. The solution is to correct the p-values for &lt;a href=&quot;https://en.wikipedia.org/wiki/Multiple_comparisons_problem&quot; rel=&quot;nofollow noreferrer&quot;&gt;multiple comparisons&lt;/a&gt;. If you do this, then none of those GO terms with a single DE member will be significant. The larger GO terms with multiple DE members in the chart you show probably won't survive correction either. Hopefully you have some that are more significant.&lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-07-12T15:12:30.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4677" PostTypeId="1" CreationDate="2018-07-12T18:51:30.450" Score="6" ViewCount="75" Body="&lt;p&gt;We are considering attempting de novo assembly of a species transcriptomes (i.e. without a reference genome) using the combined NGS outputs of Iso-seq and Illumina. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One example I saw (&lt;a href=&quot;https://www.nature.com/articles/celldisc201731&quot; rel=&quot;nofollow noreferrer&quot;&gt;Li et al 2017&lt;/a&gt;), used the standard PacBio tools to assemble a transcriptome, followed by correcting the assembled sequence using a tool called &lt;code&gt;proovread&lt;/code&gt;, followed by &lt;code&gt;cd-hit-est&lt;/code&gt; and then &lt;code&gt;cogent&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would this be considered a reasonable pipeline or is there an alterative recommendation?&lt;/p&gt;&#xA;" OwnerUserId="235" LastEditorUserId="3848" LastEditDate="2019-08-16T08:34:23.730" LastActivityDate="2021-10-09T04:02:45.170" Title="What is the current state-of-the-art in assembling hybrid transcriptomes?" Tags="|assembly|transcriptome|pacbio|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4678" PostTypeId="1" CreationDate="2018-07-12T21:41:28.640" Score="3" ViewCount="524" Body="&lt;p&gt;I am new to bioinformatics, my background is in Electrical Engineering. I am trying to convert Affymetrix Cotton Probe IDs to gene symbols. I have a gene expression dataset and I need the expressions for only certain genes. So in the dataset, I have a probe id such as Ghi.10301.1.S1_s_at, how can I get the gene symbol associated with it? Thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: This is the specific dataset I am looking at &#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE50770&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE50770&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1782" LastEditorUserId="1782" LastEditDate="2018-07-12T22:38:17.880" LastActivityDate="2018-07-13T09:02:41.753" Title="Convert Cotton Probe ID to Gene Symbol" Tags="|gene-expression|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4679" PostTypeId="2" ParentId="4678" CreationDate="2018-07-12T22:25:37.087" Score="1" Body="&lt;p&gt;Your dataset was produced on a microarray. The probe IDs in the FASTA file are similar to read IDs produced from other NGS experiments like RNA-Seq. According to the workflow provided by Ensembl the first step is to align your reads to a reference. You'll only keep the reads that match the reference completely. Once the probes are aligned you'll be able to quantify the number of reads mapping to the gene that the probe was designed to attract.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did find &lt;a href=&quot;https://www.ebi.ac.uk/arrayexpress/files/A-GEOD-8672/A-GEOD-8672.adf.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;a file&lt;/a&gt; providing the database entries associated with each probe ID (one of which is the ID you provided) for the Affymetrix Cotton Genome Array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fDUUK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fDUUK.png&quot; alt=&quot;probe microarray workflow&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="298" LastEditDate="2018-07-13T08:29:41.730" LastActivityDate="2018-07-13T08:29:41.730" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4680" PostTypeId="2" ParentId="4678" CreationDate="2018-07-13T09:02:41.753" Score="1" Body="&lt;p&gt;The first stop for decoding probe IDs for a microarray platform is usually the GEO Platform Description: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GPL8672&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GPL8672&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This description contains a table of annotation, and is provided (in this instance) by the array manufacturer. You can view, or download, the full table: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?view=data&amp;amp;acc=GPL8672&amp;amp;id=1538&amp;amp;db=GeoDb_blob35&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?view=data&amp;amp;acc=GPL8672&amp;amp;id=1538&amp;amp;db=GeoDb_blob35&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GEO Platform annotations do date, and so for up-to-date annotations, download from NetAffx: &lt;a href=&quot;http://www.affymetrix.com/support/technical/byproduct.affx?product=cotton&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.affymetrix.com/support/technical/byproduct.affx?product=cotton&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most recent annotations (dated 2016-04-13) are available here: &lt;a href=&quot;http://www.affymetrix.com/Auth/analysis/downloads/na36/ivt/Cotton.na36.annot.csv.zip&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.affymetrix.com/Auth/analysis/downloads/na36/ivt/Cotton.na36.annot.csv.zip&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the example in your question, looking at this probe in the CSV: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;Ghi.10301.1.S1_s_at&quot;,&quot;Cotton Genome Array&quot;,&quot;Gossypium hirsutum&quot;,&quot;Mar 30, 2016&quot;,&quot;Consensus sequence&quot;,&quot;GenBank&quot;,&quot;Ghi.10301.1&quot;,&quot;gb:DT050231 /DB_XREF=gi:72275634 /DB_XREF=COT_DW_H07 /TID=Ghi.10301.1 /CNT=21 /FEA=EST /TIER=ConsEnd /STK=1 /UG=Ghi.10301 /UG_TITLE=Transcribed locus, moderately similar to NP_568148.1 ubiquitin conjugating enzyme ubiquitin-like activating enzyme (Arabidopsis thaliana) /REP_ORG=G. hirsutum&quot;,&quot;DT050231&quot;,&quot;Ghi.10301&quot;,&quot;Ghi.10301&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;est&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;---&quot;,&quot;This probe set was annotated using the Accession mapped clusters based pipeline to a UniGene identifier using 21 transcripts. // false // Accession mapped clusters // E&quot;,&quot;AI730871,DN803875,DR452973,DR457278,DT050231,DT545642,DT546719,DT546982,DT546983,DT547780,DT547781,DT555433,DT559909,DT564450,DT566924,DT569293,DT569974,DT571307,DT571318,DT571472,DT573548&quot;,&quot;AI730871 // /DB_XREF=BNLGHi8130 // gb // --- // --- /// DN803875 // /DB_XREF=G.hir-8-10 DAA bolls drought stressed751 // gb // --- // --- /// DR452973 // /DB_XREF=CM008C10 // gb // --- // --- /// DR457278 // /DB_XREF=CM053F01 // gb // --- // --- /// DT050231 // /DB_XREF=COT_DW_H07 // gb // --- // --- /// DT545642 // /DB_XREF=EST1056282 // gb // --- // --- /// DT546719 // /DB_XREF=EST1057359 // gb // --- // --- /// DT546982 // /DB_XREF=EST1057622 // gb // --- // --- /// DT546983 // /DB_XREF=EST1057623 // gb // --- // --- /// DT547780 // /DB_XREF=EST1058420 // gb // --- // --- /// DT547781 // /DB_XREF=EST1058421 // gb // --- // --- /// DT555433 // /DB_XREF=EST1066073 // gb // --- // --- /// DT559909 // /DB_XREF=EST1070549 // gb // --- // --- /// DT564450 // /DB_XREF=EST1075090 // gb // --- // --- /// DT566924 // /DB_XREF=EST1077564 // gb // --- // --- /// DT569293 // /DB_XREF=EST1079933 // gb // --- // --- /// DT569974 // /DB_XREF=EST1080614 // gb // --- // --- /// DT571307 // /DB_XREF=EST1081947 // gb // --- // --- /// DT571318 // /DB_XREF=EST1081958 // gb // --- // --- /// DT571472 // /DB_XREF=EST1082112 // gb // --- // --- /// DT573548 // /DB_XREF=EST1084188 // gb // --- // ---&quot;,&quot;---&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="194" LastActivityDate="2018-07-13T09:02:41.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4681" PostTypeId="1" CreationDate="2018-07-13T10:15:41.370" Score="3" ViewCount="224" Body="&lt;p&gt;How much does a single mutation/alteration of a nucleotide affect the presence of a transcription factor binding site (TFBS)? &#xA;I am from computer science background(Obviously). &#xA;I want to make a general assumption about the number of mutated bases in a TFBS to minimize the binding affinity?&lt;/p&gt;&#xA;" OwnerUserId="3064" LastEditorUserId="964" LastEditDate="2018-07-13T17:41:45.803" LastActivityDate="2020-05-10T22:46:43.803" Title="Effect of mutation in DNA sequence on transcription factor binding sites" Tags="|fasta|sequence-analysis|phylogenetics|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4683" PostTypeId="2" ParentId="4623" CreationDate="2018-07-13T15:04:11.323" Score="4" Body="&lt;p&gt;As well as the issues @sjcockell mentions, you have another potential problem you've got a hexaploid organism, by default the HaplotypeCaller (HC) is hardwired to a ploidy of 2, you need to set this to 6 to match your organisms ploidy using &lt;code&gt;-ploidy 6&lt;/code&gt; as an argument.  Since the ploidy is a key term in the underlying Bayesian stats the HC employs (see this &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/article.php?id=4442&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; for an explanation) it's likely that some of the lower allelic frequencies that hexaploidy would generate will mean quite a few variants won't be called as they'd be statistically unlikely for a diploid organism. &lt;/p&gt;&#xA;" OwnerUserId="1769" LastActivityDate="2018-07-13T15:04:11.323" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4684" PostTypeId="2" ParentId="4681" CreationDate="2018-07-13T16:54:46.693" Score="2" Body="&lt;p&gt;I haven't worked on TFBSs but I really, really doubt you will be able to do this. In mutations, like in real estate, it's location, location, location. It isn't a question of how many mutations but what mutations and where. TFBSs are usually described using position weight matrices, precisely because that's how you can show the importance of different changes in different positions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are a couple of random Google image search results for &quot;transcription factor binding sites logo&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/azBCo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/azBCo.png&quot; alt=&quot;CRP binding site logo&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This was taken from &lt;a href=&quot;http://compbio.pbworks.com/w/page/16252929/Transcription%20Regulation%20and%20Transcription%20Factor%20Motif%20Finding&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and you might want to read through it.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1lDSd.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1lDSd.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;(source: Front. Genet., 23 February 2016 | &lt;a href=&quot;https://doi.org/10.3389/fgene.2016.00024&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.3389/fgene.2016.00024&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main take home messages from these images are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Different positions have different importance and potential to disrupt binding. In the first image, I would expect that changing the T at position -3 to anything else would be pretty bad. However, changing an A to a T at -8 would likely be neutral. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Different TFs have very different binding sites and very different binding site lengths.  Look at panel B of the second image. That one appears to be a very conservative site and likely any change would be deleterious. Conversely, changing any of the nucleotides of the 17bp spacer shown in panel A would likely not be important as long as you don't change the length. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Please note that I have not done more than glance at the sources I took these from, so my inferences may well be wrong, I only use them to illustrate the general point: you cannot really put a number to this. You will always need to look at each specific change and each specific position. In some cases, a single change can disrupt binding while in others multiple changes might have no effect at all. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;You might want to look into TFBS databases like JASPAR or TRANSFAC.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-07-13T16:54:46.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4685" PostTypeId="1" CreationDate="2018-07-13T17:44:07.760" Score="2" ViewCount="251" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/EddyRivasLab/ssu-align&quot; rel=&quot;nofollow noreferrer&quot;&gt;ssu-align&lt;/a&gt; is an HMM-based MSA tool for ribosome's small subunit (SSU) sequences. It's bundled with three SSU models: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;archaeal, &lt;/li&gt;&#xA;&lt;li&gt;bacterial  &lt;/li&gt;&#xA;&lt;li&gt;eukaryotic &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Approach&lt;/strong&gt; Given a set of archaeal and bacterial sequences the tool will first identify the best model for each sequence and produce two separate alignments for archaea and bacteria, and the tool provides no merging functionality. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Rationale&lt;/strong&gt; The underlying issue with SSU is the secondary structure, which results in: 1) indels; 2) pairwise point mutations (to maintain the secondary structure)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt; While you can force it to use a single model for all sequences, aligning archaeal sequences to the bacterial model will produce poor results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Solution?&lt;/strong&gt; Nevertheless, the authors of &lt;a href=&quot;https://www.nature.com/articles/ismej2011139&quot; rel=&quot;nofollow noreferrer&quot;&gt;GreenGenes&lt;/a&gt; claim that they've used ssu-align to produce two MSAs (one for archaea and bacteria) and build a joint phylogenetic tree using FastTree, but they say nothing about the way they've managed to merge the alignments. Am I missing some functionality from SSU-align? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; Merging two giant MSAs using conventional tools, such as MAFFT, won't get me far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've already tried contacting the authors of &lt;a href=&quot;https://www.nature.com/articles/ismej2011139&quot; rel=&quot;nofollow noreferrer&quot;&gt;the GreenGenes paper&lt;/a&gt;, but the corresponding author hasn't responded. &lt;/p&gt;&#xA;" OwnerUserId="2696" LastEditorUserId="3848" LastEditDate="2020-04-14T16:22:40.220" LastActivityDate="2023-12-25T22:05:55.697" Title="Merging ssu-align alignments" Tags="|sequence-alignment|fasta|phylogenetics|phylogeny|rna-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4686" PostTypeId="2" ParentId="3859" CreationDate="2018-07-13T19:46:01.440" Score="2" Body="&lt;p&gt;You can calculate allele frequencies for each cluster you have to further verify if they belong to similar population, however if size of your dataset is rather small this may not work for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3813878/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3813878/&lt;/a&gt;, here is an article about differentiating populations based on sequencing data.&lt;/p&gt;&#xA;" OwnerUserId="3067" LastActivityDate="2018-07-13T19:46:01.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4687" PostTypeId="1" AcceptedAnswerId="4688" CreationDate="2018-07-13T20:29:23.807" Score="5" ViewCount="4040" Body="&lt;p&gt;MaxQuant is a software package for mass spectroscopy and proteomics. There is a windows version and a linux version. To run on linux you have to use a program that is called &lt;code&gt;mono&lt;/code&gt;. I think, it is developed by Microsoft, which I find quite nice from them. I installed mono on Linux and started MaxQuant. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mono MaxQuant.exe&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.coxdocs.org/doku.php?id=maxquant:common:download_and_installation&quot; rel=&quot;noreferrer&quot;&gt;documentation&lt;/a&gt; claims it is for running MaxQuant in commandline mode, though a GUI is started anyway. How can I start the program in commandline mode to perform batch analysis? I think, there is not enough information given on the web-page. &lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2022-05-18T00:58:20.340" Title="How to run MaxQuant in command line mode?" Tags="|mass-spectrometry|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4688" PostTypeId="2" ParentId="4687" CreationDate="2018-07-13T22:21:57.477" Score="2" Body="&lt;p&gt;Give &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mono MaxQuantCmd.exe [OPTIONS]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;a try.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.uio.no/english/services/it/research/hpc/abel/help/software/maxquant.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;University of Oslo&lt;/a&gt; description of how to use the tool in their infrastructure.&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2018-07-13T22:21:57.477" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4689" PostTypeId="1" AcceptedAnswerId="4690" CreationDate="2018-07-15T00:58:15.497" Score="1" ViewCount="265" Body="&lt;p&gt;I am using &lt;a href=&quot;https://github.com/najoshi/sickle&quot; rel=&quot;nofollow noreferrer&quot;&gt;sickle&lt;/a&gt; package from bioconda to trim my reads from the Illumina Miseq.&lt;/p&gt;&#xA;&lt;p&gt;When I tried following code, it failed to run&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sickle pe -t illumina -f file1   -r file2  -o output1  -p output2  -s combined_output -q 27 -l 25  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I replaced &lt;strong&gt;illumina&lt;/strong&gt; with &lt;strong&gt;sanger&lt;/strong&gt;, it works. But my reads are from the illumina. Could anyone clarify if using this option is acceptable ? Also, what is the optimum value of -q and -l&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Sickle examines quality and length quality thresholds via a sliding window. This assesses whether to trim the 3'-end of reads, for example removing a sudden drop in quality. It can also assess whether the quality of the 5' end is sufficiently high for trimming.&lt;/p&gt;&#xA;" OwnerUserId="1684" LastEditorUserId="3848" LastEditDate="2022-03-13T16:25:42.520" LastActivityDate="2022-03-13T16:25:42.520" Title="Using sickle for quality trimming" Tags="|ngs|covid-19|trimming|virus|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4690" PostTypeId="2" ParentId="4689" CreationDate="2018-07-15T07:17:40.340" Score="2" Body="&lt;p&gt;It'd be rare to run into Illumina encoded quality scores any more, I doubt there are machines running that still produce those. Everything produced at the moment uses Sanger encoded (phred + 33) scores.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no optimum value for &lt;code&gt;-q&lt;/code&gt; and &lt;code&gt;-l&lt;/code&gt;, though reasonable values would typically be &amp;lt;=5 for &lt;code&gt;-q&lt;/code&gt; and ~5 for &lt;code&gt;-l&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-07-15T07:17:40.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4693" PostTypeId="1" AcceptedAnswerId="4718" CreationDate="2018-07-16T12:57:37.373" Score="5" ViewCount="893" Body="&lt;p&gt;In VCF 4.2, a structural variant (SV) can be described with the &lt;code&gt;BND&lt;/code&gt; keyword in &lt;code&gt;SVTYPE&lt;/code&gt;. For example, the following example is an insertion (from &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://samtools.github.io/hts-specs/VCFv4.2.pdf&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM    POS ID REF ALT                      QUAL FILTER INFO&#xA;2 321682  bnd V  T   ]13 : 123456]AGTNNNNNCAT 6    PASS   SVTYPE=BND;MATEID=bnd U&#xA;13 123456 bnd U  C   CAGTNNNNNCA[2 : 321682[  6    PASS   SVTYPE=BND;MATEID=bnd V&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The keyword can be used to describe any SV event.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to work out how to find out if an event is an insertion, deletion, duplication, inversion, etc. The algorithm/program will tell me the example above is an insertion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not able to find a tool that can do the conversion for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;PS:&lt;/strong&gt; The program I'm using is GRIDSS. This program would only generate &lt;code&gt;BND&lt;/code&gt; in the SVTYPE field. It's not strictly a bug (BND can be used for any SV arrangement), but it's annoying in analysis.&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="96" LastEditDate="2019-02-27T20:09:11.753" LastActivityDate="2019-03-28T05:43:55.263" Title="Simplest way to work out structural variant type?" Tags="|vcf|genome|variation|structural-variation|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="4694" PostTypeId="2" ParentId="4681" CreationDate="2018-07-16T13:06:34.257" Score="1" Body="&lt;p&gt;Every motif has a score. The score is calculated from the number of times each base was observed in that motif in the genome (I'm not sure how exactly these are calculated). Higher numbers indicate that this motif was found more frequently, which we take as a proxy for it providing better binding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://www.ensembl.org/info/docs/tools/vep/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl VEP&lt;/a&gt; can give you the score change for a mutated motif in the output, so this may be the easiest way to analyse this.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2018-07-16T13:06:34.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4697" PostTypeId="1" AcceptedAnswerId="4710" CreationDate="2018-07-16T21:05:22.983" Score="1" ViewCount="1077" Body="&lt;p&gt;I am trying to get gene expression matrix for a list of genes&#xA;I have for my list: Gene ID, Gene Symbol&#xA;How can I get for each gene corresponding expression as array??&#xA;I suppose that I will have &#xA;GENE_ID GENE_NAME SAMPLE_1 SAMPLE_2 .....&#xA;I am new in that, I don't know more details&#xA;I am trying to study relation between diseases and genes(human) using gene expression as features.&#xA;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="2173" LastActivityDate="2018-07-18T11:37:14.067" Title="Get Gene Expression Matrix from GEOquery" Tags="|gene|gene-expression|geoquery|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4698" PostTypeId="1" AcceptedAnswerId="4699" CreationDate="2018-07-16T22:24:03.613" Score="-1" ViewCount="84" Body="&lt;p&gt;I am trying to use the following &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/esATAC.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;ATACseq&lt;/code&gt; pipeline&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately when I am importing it with &lt;code&gt;library(esATAC)&lt;/code&gt;, it is giving me an error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error: package or namespace load failed for ‘rJava’:&#xA; .onLoad failed in loadNamespace() for 'rJava', details:&#xA;  call: dyn.load(file, DLLpath = DLLpath, ...)&#xA;  error: unable to load shared object '/Library/Frameworks/R.framework/Versions/3.5/Resources/library/rJava/libs/rJava.so':&#xA;  dlopen(/Library/Frameworks/R.framework/Versions/3.5/Resources/library/rJava/libs/rJava.so, 6): Library not loaded: /Library/Java/JavaVirtualMachines/jdk-9.jdk/Contents/Home/lib/server/libjvm.dylib&#xA;  Referenced from: /Library/Frameworks/R.framework/Versions/3.5/Resources/library/rJava/libs/rJava.so&#xA;  Reason: image not found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are threads that discuss similar problems &lt;a href=&quot;https://www.r-statistics.com/2012/08/how-to-load-the-rjava-package-after-the-error-java_home-cannot-be-determined-from-the-registry/&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; and &lt;a href=&quot;https://stackoverflow.com/questions/37735108/r-error-onload-failed-in-loadnamespace-for-rjava&quot;&gt;2&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I checked the Java version and it is the right one: &lt;code&gt;64bit&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;d-172-25-60-75:~ nikitavlasenko$ java -version&#xA;java version &quot;1.8.0_172&quot;&#xA;Java(TM) SE Runtime Environment (build 1.8.0_172-b11)&#xA;Java HotSpot(TM) 64-Bit Server VM (build 25.172-b11, mixed mode)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried to do the following in &lt;code&gt;Rstudio&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sys.setenv(JAVA_HOME = &quot;/usr/bin/java&quot;)&#xA;library(rJava)&#xA;library(esATAC)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But &lt;code&gt;library(rJava)&lt;/code&gt; fails. I am using macOS High Sierra 10.13.4. I am not sure what can I try further. Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="1157" LastEditDate="2019-04-21T19:30:03.327" LastActivityDate="2019-04-21T19:30:03.327" Title="rJava issue when trying to use ATACseq pipeline" Tags="|atac-seq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4699" PostTypeId="2" ParentId="4698" CreationDate="2018-07-16T23:21:36.050" Score="0" Body="&lt;p&gt;I uninstalled &lt;code&gt;Java&lt;/code&gt; the way it is written &lt;a href=&quot;https://discussions.apple.com/thread/4772629&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;Click on the Finder icon located in your dock&lt;/li&gt;&#xA;  &lt;li&gt;Click on Applications tab on the sidebar&lt;/li&gt;&#xA;  &lt;li&gt;In the Search box enter JavaAppletPlugin.plugin&lt;/li&gt;&#xA;  &lt;li&gt;This will find the JavaAppletPlugin.plugin file&lt;/li&gt;&#xA;  &lt;li&gt;Right click on JavaAppletPlugin.plugin and select Move to Trash&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Then installed the recommended latest version for mac on the official &lt;code&gt;Java&lt;/code&gt; website and it fixed the issue.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2018-07-17T18:27:15.260" LastActivityDate="2018-07-17T18:27:15.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4700" PostTypeId="1" AcceptedAnswerId="4709" CreationDate="2018-07-17T00:21:28.303" Score="-2" ViewCount="78" Body="&lt;p&gt;I have several &lt;code&gt;atac_seq.gz&lt;/code&gt; files and I need to run the &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/esATAC.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;pipeline&lt;/a&gt; with them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have single end &lt;code&gt;ATACseq&lt;/code&gt;, so the function &lt;code&gt;atacPipe&lt;/code&gt; requires me to have some kind of &lt;code&gt;adapter sequence&lt;/code&gt; for it to run. What is it? Where could I get one?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Update&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I found the adapter sequences, supplied them, but it is writing me that &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error in atacPipe(case = list(&quot;mm10&quot;, fastqInput1 = paste0(base_folder,  : &#xA;    adapter1 should not be NULL for single end sequencing data&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(esATAC)&#xA;base_folder &amp;lt;- &quot;/Users/nikitavlasenko/Downloads/sams_dev/&quot;&#xA;conclusion &amp;lt;- atacPipe(case=list(&quot;mm10&quot;, fastqInput1 = paste0(base_folder, &quot;Micro_atac12.fastq.gz&quot;),&#xA;                   adapter1 = &#xA;&quot;CAAGCAGAAGACGGCATACGAGATGTAGAGAGGTCTCGTGGGCTCGGAGATGT&quot;,&#xA;                   createReport = TRUE))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I supplied it and it is not null, so I am not sure what I am doing wrong.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2018-07-18T09:12:41.577" LastActivityDate="2018-07-18T09:12:41.577" Title="Supplying adapter sequences for ATACseq pipeline" Tags="|atac-seq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4701" PostTypeId="2" ParentId="4425" CreationDate="2018-07-17T01:44:41.027" Score="3" Body="&lt;p&gt;You cant do this in Chimera, but in pymol you can write a script to do this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of the view, you can use get_view on the command line. This returns a section that you can copy and paste in - which will give you the same view every time. I recommend labelling the residue you want, then finding the position you want to view it from. Then using get_view to get that view, which you can use every time (just copy it into a text file) for a reference&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This does presume your different molecules are aligned, but you can do that in pymol as well to make sure that when you use the output of get_view it is consistent. &lt;/p&gt;&#xA;" OwnerUserId="3078" LastEditorUserId="3078" LastEditDate="2018-07-17T10:16:03.727" LastActivityDate="2018-07-17T10:16:03.727" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4702" PostTypeId="1" CreationDate="2018-07-17T04:02:24.360" Score="2" ViewCount="252" Body="&lt;p&gt;I have assembled a virus genome using &lt;a href=&quot;https://github.com/sebhtml/ray&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ray&lt;/a&gt; resulting in approximately 5000 contigs. Now I want to build a scaffold of those contigs to get the full genome (I am aware of the fact that there might be some gaps in the genome).&lt;/p&gt;&#xA;&lt;p&gt;I  found one tool called &lt;a href=&quot;https://github.com/combogenomics/medusa&quot; rel=&quot;nofollow noreferrer&quot;&gt;Medusa&lt;/a&gt; to do this. Unfortunately, this tool cannot take a file above 50 MB and my contig file is about 156 MB.&lt;/p&gt;&#xA;&lt;p&gt;Are there any other tools that can perform reference based scaffolding?&lt;/p&gt;&#xA;&lt;p&gt;It would be great if anyone knows about genome scaffolding packages in Bioconda.&lt;/p&gt;&#xA;" OwnerUserId="1684" LastEditorUserId="3848" LastEditDate="2020-12-05T10:05:28.483" LastActivityDate="2020-12-05T10:05:28.483" Title="Genome scaffolding" Tags="|ngs|covid-19|sars-cov-2|reference-genome|scaffold|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4703" PostTypeId="2" ParentId="4641" CreationDate="2018-07-17T09:35:25.123" Score="4" Body="&lt;p&gt;To add to the list that Devon Ryan outlined (or perhaps to elaborate on point 2?):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although Salmon/Kallisto/RSEM are the more accurate in their transcript quantification than the methods they superseded, transcript level quantification is still not as accurate as gene level quantification (see the &lt;a href=&quot;https://f1000research.com/articles/4-1521&quot; rel=&quot;nofollow noreferrer&quot;&gt;tximport&lt;/a&gt; paper, which would also be the tool i'd recommend for getting gene level estimates from the transcript-level output of these tools).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, the &lt;a href=&quot;https://f1000research.com/articles/4-1521&quot; rel=&quot;nofollow noreferrer&quot;&gt;tximport authors&lt;/a&gt; do claim that considering the transcript-level quantification in your gene-level analysis improves the accuracy of the gene-level analysis.   &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-07-17T09:35:25.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4704" PostTypeId="2" ParentId="4640" CreationDate="2018-07-17T09:58:27.397" Score="1" Body="&lt;p&gt;If you were to treat the nucleotide like a ligand you could follow &lt;a href=&quot;http://vlab.amrita.edu/?sub=3&amp;amp;brch=275&amp;amp;sim=1446&amp;amp;cnt=2&quot; rel=&quot;nofollow noreferrer&quot;&gt;these steps&lt;/a&gt; on pymol, and find the measurements you desire. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Calculating distances to centroids is a little bit more complex, as you have to make a &lt;a href=&quot;https://pymolwiki.org/index.php/Center_of_mass&quot; rel=&quot;nofollow noreferrer&quot;&gt;pseudo atom&lt;/a&gt; to measure against. &lt;/p&gt;&#xA;" OwnerUserId="3078" LastEditorUserId="48" LastEditDate="2018-08-21T10:44:53.283" LastActivityDate="2018-08-21T10:44:53.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4705" PostTypeId="1" CreationDate="2018-07-17T13:09:44.667" Score="4" ViewCount="432" Body="&lt;p&gt;I have RNA-seq raw counts data for 50 samples. 20 Normal and 30 tumor. After differential analysis I got 30 DEGs. I want to make a violin plot showing the expression of each gene. I transformed counts to logCPM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;counts:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Genes   Tumor1  Tumor2  Normal1 Normal2&#xA;RP11-351J23.1   0   5   6   0&#xA;MIR17HG 989 896 8   0&#xA;RP11-563N12.2   0   0   0   0&#xA;LINC04393   0   2   16  0&#xA;RP11-336A10.4   0   0   0   0&#xA;DRH6-AB1    53  13  39  9&#xA;RP11-115J16.1   0   0   50  6&#xA;LINC70518   2   65  0   0&#xA;&#xA;logCPM &amp;lt;- cpm(counts, prior.count=2, log = TRUE)&#xA;&#xA;                Tumor1   Tumor2  Normal1  Normal2&#xA;RP11-351J23.1 11.84477 13.09301 15.71337 11.84477&#xA;MIR17HG      19.84847 19.79600 16.10350 11.84477&#xA;RP11-563N12.2 11.84477 11.84477 11.84477 11.84477&#xA;LINC04393     11.84477 12.47723 17.06532 11.84477&#xA;RP11-336A10.4 11.84477 11.84477 11.84477 11.84477&#xA;DRH6-AB1      15.72257 14.03896 18.32772 19.19283&#xA;RP11-115J16.1 11.84477 11.84477 18.68262 18.61229&#xA;LINC70518     12.44599 16.08368 11.84477 11.84477&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to make violin plot for each gene showing expression for Tumors and Normals with this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I first transformed logCPM. then tried like below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggplot2.violinplot(data=logCPM, xName='RP11-351J23.1',&#xA;    groupName='Tumor', position=position_dodge(0.8), &#xA;    backgroundColor=&quot;white&quot;, groupColors=c('#999999','#E69F00'),&#xA;    legendPosition=&quot;top&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1369" LastEditorUserId="48" LastEditDate="2018-09-04T10:09:42.957" LastActivityDate="2018-09-04T10:09:42.957" Title="Plot to show the expression of genes between tumor and normal" Tags="|r|rna-seq|rna|visualization|ggplot2|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4706" PostTypeId="2" ParentId="4705" CreationDate="2018-07-17T13:30:06.377" Score="2" Body="&lt;p&gt;You'll want to use long-form data for everything:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(dplyr)&#xA;library(ggplot2)&#xA;logCPM$gene = row.names(logCPM)&#xA;d = logCPM %&amp;gt;% gather(Sample, logCPM, -gene)&#xA;d$group = c(rep(&quot;Tumor&quot;, 2), rep(&quot;Normal&quot;, 2))&#xA;geneOfInterest = d %&amp;gt;% filter(gene == 'RP11-351J23.1')&#xA;ggplot(geneOfInterest, aes(x=group, y=logCPM)) + geom_violin()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That's a simple example, you can tweak it to meet your needs.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-07-17T13:30:06.377" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4709" PostTypeId="2" ParentId="4700" CreationDate="2018-07-17T20:07:00.087" Score="0" Body="&lt;p&gt;I was calling the function incorrectly. Here is how it worked:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conclusion &amp;lt;- atacPipe(&quot;mm10&quot;, fastqInput1 = paste0(base_folder, &quot;Micro_atac12.fastq.gz&quot;),&#xA;                   adapter1 = &quot;CAAGCAGAAGACGGCATACGAGATGTAGAGAGGTCTCGTGGGCTCGGAGATGT&quot;,&#xA;                   createReport = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-07-17T20:07:00.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4710" PostTypeId="2" ParentId="4697" CreationDate="2018-07-18T11:23:56.210" Score="3" Body="&lt;p&gt;The answer to this really depends on the type of data you're retrieving from GEO. Microarray data sets should have a normalised matrix of expression values uploaded as part of the entry. &lt;code&gt;getGEO()&lt;/code&gt; from the &lt;code&gt;GEOquery&lt;/code&gt; package will return a list of ExpressionSets - in many cases this list will have a length of 1. An ExpressionSet has an &lt;code&gt;exprs()&lt;/code&gt; accessor for retrieving the expression matrix. An example (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE76510&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSE76510&lt;/a&gt;, an Illumina HT12 v4 BeadArray dataset):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; library(GEOquery)&#xA;&amp;gt; gse76510 &amp;lt;- getGEO('GSE76510')&#xA;&amp;gt; names(gse76510)&#xA;[1] &quot;GSE76510_series_matrix.txt.gz&quot;&#xA;&amp;gt; class(gse76510$GSE76510_series_matrix.txt.gz)&#xA;[1] &quot;ExpressionSet&quot;&#xA;attr(,&quot;package&quot;)&#xA;[1] &quot;Biobase&quot;&#xA;&amp;gt; exprs(gse76510$GSE76510_series_matrix.txt.gz) %&amp;gt;% head&#xA;             GSM2027280 GSM2027281 GSM2027282 GSM2027283 GSM2027284 GSM2027285 GSM2027286&#xA;ILMN_1343291  14.936287  14.983927  15.008199  14.830549  15.008199  14.789385  14.891532&#xA;ILMN_1343295  13.404168  13.329903  13.289727  13.222221  13.386299  13.240755  13.125056&#xA;ILMN_1651199   6.779805   7.189868   7.046952   6.962957   6.697924   6.850950   6.792647&#xA;ILMN_1651209   6.983803   6.769702   7.016746   6.954751   6.890472   6.982947   6.915671&#xA;ILMN_1651210   6.837042   6.999098   6.895225   6.944250   6.955914   6.917959   6.837368&#xA;ILMN_1651221   7.034363   7.025626   6.606457   6.977622   7.051120   6.971884   7.066445&#xA;             GSM2027287 GSM2027288 GSM2027289 GSM2027290 GSM2027291&#xA;ILMN_1343291  14.891532  14.983927  14.848452  14.913474  15.008199&#xA;ILMN_1343295  13.240755  13.010757  13.525999  13.009661  12.998406&#xA;ILMN_1651199   6.786784   6.820451   6.851160   7.013586   6.864316&#xA;ILMN_1651209   6.983701   6.995199   7.270760   6.901377   6.950876&#xA;ILMN_1651210   6.804955   7.033074   6.894117   6.770691   7.000786&#xA;ILMN_1651221   7.003035   6.995199   7.050040   6.923861   6.929933&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You will be able to filter out a list of required genes using the probe annotation (available in the appropriate annotation.db - in this case, for example, &lt;code&gt;illuminaHumanv4.db&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the dataset you're interested in is RNA-Seq, you're probably out of luck. There's currently no requirement on data submitters to provide expression estimates with an RNA-Seq dataset, so the &lt;code&gt;ExpressionSet&lt;/code&gt; that &lt;code&gt;GEOquery&lt;/code&gt; returns will most likely have an empty &lt;code&gt;exprs&lt;/code&gt; slot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, an example (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE117133&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSE117133&lt;/a&gt;): &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; gse117133 &amp;lt;- getGEO('GSE117133')&#xA;&amp;gt; exprs(gse117133$GSE117133_series_matrix.txt.gz)&#xA; GSM3271892 GSM3271893 GSM3271894 GSM3271895 GSM3271896 GSM3271897&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="194" LastEditorUserId="194" LastEditDate="2018-07-18T11:37:14.067" LastActivityDate="2018-07-18T11:37:14.067" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4711" PostTypeId="1" CreationDate="2018-07-18T15:39:06.510" Score="1" ViewCount="101" Body="&lt;p&gt;Are there any tools for finding duplications from a whole genome in FASTA format?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say the genome has gene A duplicated somewhere in the genome and gene B triplicated in tandem elsewhere in the genome. Is there a tool that will output a list of gene A and gene B with location and number of copies?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently using &lt;a href=&quot;http://www.ruhr-uni-bochum.de/ecoevo/cm/cm_phobos.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;phobos&lt;/a&gt; to annotate tandem repeats in the genome of &lt;em&gt;C. elegans&lt;/em&gt; but it becomes very slow beyond repeats of 100 bp. Does anybody know of a tool for gene-size/larger repeats?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried to use &lt;a href=&quot;https://github.com/acg-team/tral&quot; rel=&quot;nofollow noreferrer&quot;&gt;TRAL&lt;/a&gt; but I have got some installation and usage issues, described &lt;a href=&quot;https://github.com/acg-team/tral/issues/12&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot use the annotation because I would like to apply this tool to a non-reference version of a &lt;em&gt;C. elegans&lt;/em&gt; genome, for which no annotation is available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, I would like to annotate the Hawaiian strain, which is a strain very divergent to the reference N2 strain. I am not looking into particular genes, just natural genetic variation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not tried BLAST. It would be a possibility but I wonder if there are others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-07-18T21:10:58.960" LastActivityDate="2018-07-18T21:10:58.960" Title="Detect gene duplication events from a whole genome in FASTA format" Tags="|software-recommendation|sequence-annotation|cnv|repeat-elements|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4712" PostTypeId="5" CreationDate="2018-07-18T15:41:40.360" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-07-18T15:41:40.360" LastActivityDate="2018-07-18T15:41:40.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4713" PostTypeId="4" CreationDate="2018-07-18T15:41:40.360" Score="0" Body="Copy Number Variation. Questions about how to define, find, annotate, study CNVs. Algorithms." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-07-20T11:24:20.870" LastActivityDate="2018-07-20T11:24:20.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4714" PostTypeId="1" AcceptedAnswerId="4717" CreationDate="2018-07-18T17:48:22.013" Score="3" ViewCount="974" Body="&lt;p&gt;Suppose we have a fasta file like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Seq1&#xA;GTTGAGAGGTGTATGGACACGAAAAACGAAACTGTATCCCGTGTTTAGCAAAGAAATCAT&#xA;&amp;gt;Seq1&#xA;AAAAACGAAACTGTATCCCGTGTTT&#xA;&amp;gt;Seq2&#xA;CGTGTTTAGCAAAGAAAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to produce&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Seq1&#xA;GTTGAGAGGTGTATGGACACGAAAAACGAAACTGTATCCCGTGTTTAGCAAAGAAATCATAAAAACGAAACTGTATCCCGTGTTT&#xA;&amp;gt;Seq2&#xA;CGTGTTTAGCAAAGAAAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2687" LastActivityDate="2018-07-19T12:52:12.687" Title="How to merge transcript sequence with same name in a FASTA file." Tags="|fasta|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4715" PostTypeId="5" CreationDate="2018-07-18T21:13:23.363" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-07-18T21:13:23.363" LastActivityDate="2018-07-18T21:13:23.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4716" PostTypeId="4" CreationDate="2018-07-18T21:13:23.363" Score="0" Body="Questions relating to proteins, the chains of amino acids that form three-dimensional structures carrying important functions of life. Protein sequence, structure, similarity, domains, etc." OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-07-20T11:24:31.800" LastActivityDate="2018-07-20T11:24:31.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4717" PostTypeId="2" ParentId="4714" CreationDate="2018-07-19T04:57:22.133" Score="3" Body="&lt;p&gt;Certainly this python script is not the most elegant solution but it will give you the desired result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import sys&#xA;&#xA;seqs = {}&#xA;with open(sys.argv[1], &quot;r&quot;) as fh:&#xA;    curr = &quot;&quot;&#xA;    for line in fh:&#xA;        if line.startswith(&quot;&amp;gt;&quot;):&#xA;            curr = line&#xA;            if curr not in seqs:&#xA;                seqs[curr] = &quot;&quot;&#xA;        else:&#xA;            seqs[curr] += line.strip()&#xA;&#xA;for ident, seq in seqs.items():&#xA;    print(&quot;{}{}&quot;.format(ident, seq))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you saved the script in seq_merge.py just run it with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python seq_merge.py your_file.fasta &amp;gt; result.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3090" LastEditorUserId="3090" LastEditDate="2018-07-19T05:11:33.500" LastActivityDate="2018-07-19T05:11:33.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4718" PostTypeId="2" ParentId="4693" CreationDate="2018-07-19T05:01:35.280" Score="3" Body="&lt;p&gt;GRIDSS's author had posted &lt;a href=&quot;https://github.com/PapenfussLab/gridss/blob/7b1fedfed32af9e03ed5c6863d368a821a4c699f/example/simple-event-annotation.R#L9&quot; rel=&quot;nofollow noreferrer&quot;&gt;his own solution on Github&lt;/a&gt;. The code is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;simpleEventType &amp;lt;- function(gr) {&#xA;    return(ifelse(seqnames(gr) != seqnames(partner(gr)), &quot;ITX&quot;, # inter-chromosomosal&#xA;           ifelse(gr&lt;span class=&quot;math-container&quot;&gt;$insLen &amp;gt;= abs(gr$&lt;/span&gt;svLen) * 0.7, &quot;INS&quot;,&#xA;           ifelse(strand(gr) == strand(partner(gr)), &quot;INV&quot;,&#xA;           ifelse(xor(start(gr) &amp;lt; start(partner(gr)), strand(gr) == &quot;-&quot;), &quot;DEL&quot;, &quot;DUP&quot;)))))&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="174" LastEditorUserId="2878" LastEditDate="2019-03-28T05:43:55.263" LastActivityDate="2019-03-28T05:43:55.263" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4720" PostTypeId="2" ParentId="4714" CreationDate="2018-07-19T07:11:12.090" Score="1" Body="&lt;p&gt;assuming two lines per fasta record (or you can &lt;a href=&quot;https://gist.github.com/lindenb/2c0d4e11fd8a96d4c345&quot; rel=&quot;nofollow noreferrer&quot;&gt;linearize&lt;/a&gt;) and using datamash:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat jeter.fa | paste - - | sort -t $'\t' -k1,1 | datamash -g 1 collapse 2 | tr -d ',' | tr &quot;\t&quot; &quot;\n&quot;&#xA;&#xA;&amp;gt;Seq1&#xA;AAAAACGAAACTGTATCCCGTGTTTGTTGAGAGGTGTATGGACACGAAAAACGAAACTGTATCCCGTGTTTAGCAAAGAAATCAT&#xA;&amp;gt;Seq2&#xA;CGTGTTTAGCAAAGAAAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastActivityDate="2018-07-19T07:11:12.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4721" PostTypeId="1" CreationDate="2018-07-19T10:15:40.813" Score="2" ViewCount="298" Body="&lt;p&gt;I want to plot a Ramachandran Plot for around 5000 PDB files in python that is saved in a folder in my system. I am searching for a way to make it without accessing each file by hard-coding each file name. Please tell if there is a way around this.&lt;/p&gt;&#xA;" OwnerUserId="3093" LastActivityDate="2018-10-17T14:00:34.923" Title="Plotting Ramachandran Plot from more than one PDB file" Tags="|python|file-formats|pdb|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4722" PostTypeId="2" ParentId="4714" CreationDate="2018-07-19T10:28:21.217" Score="1" Body="&lt;p&gt;If you don't care about the order of the sequences and your file is small enough to fit in RAM, you can do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ perl -nle 'if(/^&amp;gt;/){&#xA;                $n=$_&#xA;             }&#xA;             else{&#xA;                $seq{$n}.=$_&#xA;            } &#xA;            END{&#xA;                print &quot;$_\n$seq{$_}&quot; for keys(%seq)&#xA;            }' file.fa &#xA;&amp;gt;Seq1&#xA;GTTGAGAGGTGTATGGACACGAAAAACGAAACTGTATCCCGTGTTTAGCAAAGAAATCATAAAAACGAAACTGTATCCCGTGTTT&#xA;&amp;gt;Seq2&#xA;CGTGTTTAGCAAAGAAAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, if you are into the whole brevity thing&lt;sup&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=vgUHqk-4omc&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; &lt;/sup&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl -nle '/^&amp;gt;/?($n=$_):($s{$n}.=$_);}{print&quot;$_\n$s{$_}&quot;for keys%s' file.fa &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-07-19T10:28:21.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4723" PostTypeId="2" ParentId="4721" CreationDate="2018-07-19T11:31:45.933" Score="2" Body="&lt;p&gt;You could probably use &lt;a href=&quot;https://github.com/gerdos/PyRAMA&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyRAMA&lt;/a&gt; for that. The only problem is that by PyRAMA plots to display, while you probably would like to save to a file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way to get around this is to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Patch PyRAMA to allow &lt;code&gt;ramachandran&lt;/code&gt; function to accept output image filename.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def ramachandran(file_name_list, output_image=None):&#xA;    ...&#xA;    plt.tight_layout()&#xA;    if output_image is not None:&#xA;        plt.savefig(output_file, dpi=300)&#xA;    else:&#xA;        plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If this works as expected I would really &lt;strong&gt;encourage&lt;/strong&gt; you to submit a &lt;strong&gt;pull-request&lt;/strong&gt; (or at least open an issue) on github.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now you can write your main script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from glob import iglob&#xA;&#xA;from rama_lib.ramachandran_calc import ramachandran&#xA;&#xA;&#xA;for pdb_filename in iglob('/my/dir/with/pdb/', '*.pdb'):&#xA;    img_filename = pdb_filename.rsplit('.', 1)[0] + '.png'&#xA;    ramachandran((pdb_filename,), img_filename)&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I did not test the code, but I hope you get an idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also other ways to make a ramachandran plot in python. Please check: &lt;a href=&quot;https://warwick.ac.uk/fac/sci/moac/people/students/peter_cock/python/ramachandran/drawing/&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;http://azevedolab.net/resources/rama_plot_py.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;, &lt;a href=&quot;http://mdtraj.org/latest/examples/ramachandran-plot.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="191" LastActivityDate="2018-07-19T11:31:45.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4724" PostTypeId="2" ParentId="4702" CreationDate="2018-07-19T11:38:55.660" Score="1" Body="&lt;p&gt;What you mean is assembly based scaffolding, as opposed to using reads with long distance information such as mate pair/long jumping distance to scaffold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could reduce your contig file to just the longest contigs, since they have the most information. Probably 156MB of data is overkill for your virus. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other programs out there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/ksahlin/BESST&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ksahlin/BESST&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/institut-de-genomique/MaGuS&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/institut-de-genomique/MaGuS&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your mileage will vary.&lt;/p&gt;&#xA;" OwnerUserId="3094" LastActivityDate="2018-07-19T11:38:55.660" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4725" PostTypeId="2" ParentId="4714" CreationDate="2018-07-19T12:52:12.687" Score="4" Body="&lt;p&gt;This is a scenario where awk excels and yields a much simpler solution than the other examples here (though the logic is essentially equivalent to terdon’s Perl solution):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;awk '&#xA;    /&amp;gt;/  { id = $0 }&#xA;    !/&amp;gt;/ { seq[id] = seq[id] $0 }&#xA;    END  { for (id in seq) print id &quot;\n&quot; seq[id] }&#xA;' input.fasta \&#xA;&amp;gt; output.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That said, for anything beyond a quick’n’dirty application I strongly recommend using a &lt;em&gt;proper&lt;/em&gt; FASTA parser, not an ad-hoc solution.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2018-07-19T12:52:12.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4727" PostTypeId="1" CreationDate="2018-07-19T17:31:57.187" Score="1" ViewCount="86" Body="&lt;p&gt;Some general questions...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it appears with most classification learners one must come up with a series of quantifiable variables to associate with each observation.  This might include mean and std deviation intensity values, texture, and zernike polynomials.  Does anyone have any other suggestions as to how one might quantify 3-dimensional fluorescence pixel data features?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once appropriate quantification's have been acquired for a given class one can then train a classifier.  My next question is - does one then have to segment test images and create a matrix of the same feature variables from that segmented image to use the classifier? Or can one use the raw image itself?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What supervised machine learning techniques will evaluate the raw image data itself - either for training or testing? Can manually annotated data sets or clustered data sets provide a direct model for training data?  That is to say, are there supervised machine learning algorithms which accept pure pixel data as models for training?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there reliable pre-trained classifiers that are already implemented for these type of fluorescence data sets?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any resources or advice is greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance for any input!&lt;/p&gt;&#xA;" OwnerUserId="3096" LastActivityDate="2019-01-20T15:33:03.583" Title="biological fluorescence supervised machine learing" Tags="|3d-structure|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4728" PostTypeId="2" ParentId="4727" CreationDate="2018-07-19T21:27:20.460" Score="2" Body="&lt;p&gt;Whether to quantify things at all with depend on the technique you want to use. An end-to-end CNN (convolutional neural network) for example would just be fed the 3D image for training and predictions. However the major downside to CNNs is that you need a LOT of data to train them, though perhaps you can get lucky and find an existing CNN that you can use and just retrain the last couple layers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other than that, you're going to need to come up with as many (hopefully useful) metrics as you can. There are then a lot of techniques one could try, ranging from simple things like regression to random forests to SVMs, among many others. In general I think simple regression and random forests are the easiest to get started with.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-07-19T21:27:20.460" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4729" PostTypeId="1" AcceptedAnswerId="4730" CreationDate="2018-07-20T05:28:08.383" Score="2" ViewCount="87" Body="&lt;p&gt;I have two file that have overlapping regions. One file has chromosomes, position, CNV data and samples.Other file has chromosomes, positions and ensemble ids of genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is some lines of two files.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;contig  start   end     type  bi-allelic  S1  S2  S3&#xA;chr1    12048   177033    DUP FALSE   -1  22  24&#xA;chr1    12048   89237     DUP FALSE   12  10  -1&#xA;chr1    2985712 3355300   DEL   TRUE  2   2   2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Second file is like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    901877  911245  gene_id &quot;ENSG00000187583.6&quot;                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      &#xA;chr1    22013   41670 gene_id &quot;ENSG00000160075.10&quot;                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    &#xA;chr1    2985732 3355185 gene_id &quot;ENSG00000142611.12&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want output like this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    22013   41670 gene_id &quot;ENSG00000160075.10&quot;  DUP FALSE   12  10  -1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                  &#xA;chr1    2985732 3355185 gene_id &quot;ENSG00000142611.12&quot; DEL   TRUE  2   2   2 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there any way to do this?&lt;/p&gt;&#xA;" OwnerUserId="1247" LastActivityDate="2018-07-20T07:27:15.323" Title="Comparing two files with overlapping regions and get their associated information" Tags="|bedtools|ensembl|cnv|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4730" PostTypeId="2" ParentId="4729" CreationDate="2018-07-20T07:27:15.323" Score="3" Body="&lt;p&gt;You can use &lt;code&gt;bedtools intersect&lt;/code&gt; and cut the results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat file1.bed&#xA;chr1    12048   177033  DUP FALSE   -1  22  24&#xA;chr1    12048   89237   DUP FALSE   12  10  -1&#xA;chr1    2985712 3355300 DEL TRUE    2   2   2&#xA;&#xA;$ cat file2.gtf&#xA;chr1    22013   41670   gene_id &quot;ENSG00000160075.10&quot;&#xA;chr1    22013   41670   gene_id &quot;ENSG00000160075.10&quot;&#xA;chr1    2985732 3355185 gene_id &quot;ENSG00000142611.12&quot;&#xA;&#xA;$ bedtools intersect -a file2.gtf -b file1.bed -wb | cut -f 1-4,8-11&#xA;chr1    22013   41670   gene_id &quot;ENSG00000160075.10&quot;    DUP FALSE   12  10&#xA;chr1    22013   41670   gene_id &quot;ENSG00000160075.10&quot;    DUP FALSE   -1  22&#xA;chr1    2985732 3355185 gene_id &quot;ENSG00000142611.12&quot;    DEL TRUE    2   2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-07-20T07:27:15.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4731" PostTypeId="2" ParentId="4727" CreationDate="2018-07-20T09:42:54.883" Score="2" Body="&lt;p&gt;Adding to Devon's correct answer: Depending on the amount of images you got, there is also the possibility to avoid the feature engineering by building a two level architecture:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The first level identifies the regions of interest in your image and scales these regions to the same size.&lt;/li&gt;&#xA;&lt;li&gt;In the second level, the recognized areas are fed to a (multilayer) neural network/deep neural network to be classified.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;By using this approach, the number of needed datapoints is lower compared to a plain CNN method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The drawback is that identifying the regions of interest itself might be a difficult task as you have either to label these regions manually to apply a machine learning approach at this first level, or you have to &quot;engineer&quot; criteria which make the extraction possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have no idea how these images look like, but maybe you are able to extract the regions using &lt;a href=&quot;https://opencv.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenCV&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3090" LastActivityDate="2018-07-20T09:42:54.883" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4732" PostTypeId="5" CreationDate="2018-07-20T10:57:54.910" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-07-20T10:57:54.910" LastActivityDate="2018-07-20T10:57:54.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4733" PostTypeId="4" CreationDate="2018-07-20T10:57:54.910" Score="0" Body="Use this tag when looking for a reference of a method, data, result or discussion." OwnerUserId="48" LastEditorUserId="48" LastEditDate="2018-07-29T16:09:40.077" LastActivityDate="2018-07-29T16:09:40.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4734" PostTypeId="5" CreationDate="2018-07-20T11:01:44.840" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-07-20T11:01:44.840" LastActivityDate="2018-07-20T11:01:44.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4735" PostTypeId="4" CreationDate="2018-07-20T11:01:44.840" Score="0" Body="Use it when comparing a measure of the expression between two (or more) samples, where the units are relative to the comparison." OwnerUserId="48" LastEditorUserId="48" LastEditDate="2018-07-20T11:24:56.817" LastActivityDate="2018-07-20T11:24:56.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4736" PostTypeId="1" CreationDate="2018-07-20T12:46:32.417" Score="3" ViewCount="273" Body="&lt;p&gt;I'm trying to make a plot showing genome tracks of specific genes in specific cell-lines of RNA-seq and Chip-seq data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should look something like this &lt;a href=&quot;https://i.stack.imgur.com/sR04J.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sR04J.png&quot; alt=&quot;genome tracks&quot;&gt;&lt;/a&gt;I have recently seen this &lt;a href=&quot;https://encode.genemo.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;encode&lt;/a&gt;, but in this I didn't find any MDAMB231 cell-lines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;May I know how to make such plots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I got the Counts data from CCLE portal.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;             BT-549 MDAMB231&#xA;RP11-554I8.2    0   246&#xA;LINC00707      50   107&#xA;   RBM17    17036   10819&#xA;  PFKFB3    18098   7668&#xA;RP11-393K12.2   0   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1377" LastEditorUserId="1377" LastEditDate="2018-07-20T13:32:39.233" LastActivityDate="2022-10-27T14:31:24.587" Title="How to visualize genome track of gene in specific cell-lines?" Tags="|rna-seq|gene|genome|rna|encode|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4737" PostTypeId="1" CreationDate="2018-07-20T18:15:39.377" Score="3" ViewCount="2032" Body="&lt;p&gt;I was experimenting Prokka and RAST annotation tools. So, I took a well-annotated swinepox virus genome from genebank &lt;em&gt;(NCBI Reference Sequence: NC_003389.1)&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I ran those sequences on Prokka and RAST Seed server at the same time. I can see that only a few (may be around 1%) of the genes were annotated. Most of them were predicted as hypothetical protein. And the results were comparable between Prokka and RAST.&lt;/p&gt;&#xA;&lt;p&gt;I would assume that these tools look for similar sequences in NCBI and find the best-match protein. But looks like that is not the case.&lt;/p&gt;&#xA;&lt;p&gt;I am looking for prediction tool(s) that will:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;find a well annotated swinepox virus genome in Genbank to predict most of the proteins.&lt;/li&gt;&#xA;&lt;li&gt;improve on the prediction of Prokka and Rast to assign functions to otherwise &lt;strong&gt;hypothetical protein&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Would prediction tools such as Genemark or other genome annotation tools be beneficial.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively have I misunderstood the concept of annotation?&lt;/p&gt;&#xA;&lt;p&gt;Assistance would be very welcome.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;I have attached the image for comparison. Left one the Swinepox genome in .gb format and right one is the same genome annotated with Prokka.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eefc1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eefc1.png&quot; alt=&quot;Image : sorry for the low quality screenshot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1684" LastEditorUserId="3848" LastEditDate="2022-03-08T16:02:06.407" LastActivityDate="2022-03-08T16:02:06.407" Title="Annotation with Prokka or RAST" Tags="|sequence-annotation|ngs|covid-19|virus|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4739" PostTypeId="1" CreationDate="2018-07-20T21:57:43.827" Score="1" ViewCount="26" Body="&lt;p&gt;I have 3 exomes results (VCF files), I would like to study variants (SNP, INDEL) that can affect the miRNAs, which are related to a complex disease (Schizophrenia). What is the best way to assess the effects of variants on miRNAs and how can I link between the damaged miRNAs and Schizophrenia. Can I find a help or advice please? &lt;/p&gt;&#xA;" OwnerUserId="3103" LastActivityDate="2018-07-23T08:04:50.023" Title="Variants in Exome that affect miRNAs related to Schizophrenia" Tags="|sequence-annotation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4740" PostTypeId="2" ParentId="4737" CreationDate="2018-07-21T09:13:07.217" Score="1" Body="&lt;p&gt;Both of the annotation tools you used are designed for prokaryotic genome annotation. I would not expect them to work very well for viruses. &lt;/p&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-07-21T09:13:07.217" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4741" PostTypeId="1" CreationDate="2018-07-21T22:26:18.847" Score="1" ViewCount="25" Body="&lt;p&gt;I am trying to run &lt;code&gt;ATACseq&lt;/code&gt; pipeline: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/esATAC.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioconductor.org/packages/release/bioc/html/esATAC.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On single end &lt;code&gt;ATACseq&lt;/code&gt; experiments. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conclusion_micro12 &amp;lt;- atacPipe(&quot;mm10&quot;, fastqInput1 = paste0(base_folder, &quot;Micro_atac12.fastq.gz&quot;),&#xA;                   adapter1 = &quot;CAAGCAGAAGACGGCATACGAGATAGCGTAGCGTCTCGTGGGCTCGGAGATGT&quot;,&#xA;                   threads = 4,&#xA;                   createReport = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It generated several folders during its run: &lt;code&gt;esATAC_report&lt;/code&gt;, &lt;code&gt;esATAC_result&lt;/code&gt;, &lt;code&gt;intermediate_results&lt;/code&gt;, &lt;code&gt;refdir&lt;/code&gt;. However, only the last two gets populated, the others being empty. I am not sure what is going on. I looked over various log files in the &lt;code&gt;intermediate_results&lt;/code&gt; folder, but it does not say much. Where should I look to figure out what went wrong? I would expect a report to be generated if everything is fine since I specified &lt;code&gt;createReport = TRUE&lt;/code&gt;. While running the pipeline it does not show any errors in &lt;code&gt;Rstudio&lt;/code&gt; console.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-07-21T22:26:18.847" Title="ATACseq pipeline report generation issue" Tags="|atac-seq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4742" PostTypeId="1" AcceptedAnswerId="4751" CreationDate="2018-07-22T06:43:10.550" Score="5" ViewCount="42220" Body="&lt;p&gt;I have an expression data matrix (120X15; 15 samples and 120 genes), my heatmap looks blurred and raw names (gene names) looks very small and can not read. How can I improve my scripts?&lt;/p&gt;&#xA;&lt;p&gt;Here is the example data&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df&amp;lt;-structure(list(X_T0 = c(1.32606106633333, 0.222768019,  1.27419565966667, 0.293489379),&#xA;X_T1 = c(1.499956424, 0.293117217, 1.11955345266667, 0.324453864666667),&#xA;X_T2 = c(1.118528738, 0.548228048, 0.936319105, 0.846378238666667), &#xA;X_T3 = c(1.025082136, 0.841192647333333, 0.957743318666667, 1.10810667833333),&#xA;Y_T0 = c(0.424537206666667, 3.144197864, 1.01435004766667, 2.195831453),&#xA;Y_T1 = c(0.723243112666667, 0.576764958333333, 0.604810037, 0.896297411666667),&#xA;Y_T2 = c(0.335509156333333, 1.32037215366667, 1.209961935, 0.290895345333333),&#xA;Y_T3 = c(0.328275209, 1.15039119233333, 1.409758678, 0.0922298696666667),&#xA;Z_T0 = c(0.788329993666667, 1.03539976366667, 1.10523299, 0.885098106666667), &#xA;Z_T1 = c(1.125292329, 1.00032109266667, 0.877359233, 1.385935293), &#xA;Z_T2 = c(2.357924224, 0.740699933666667, 0.585095342, 1.98251017733333),&#xA;Z_T3 = c(0.678921448, 0.687992671666667, 1.005347728, 1.05829030233333)), row.names = c(&amp;quot;AA&amp;quot;, &amp;quot;PQ&amp;quot;, &amp;quot;UBQ&amp;quot;, &amp;quot;XY&amp;quot;), class = &amp;quot;data.frame&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Script being used&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;col &amp;lt;- colorRampPalette(c(&amp;quot;red&amp;quot;,&amp;quot;yellow&amp;quot;,&amp;quot;darkgreen&amp;quot;))(30) &#xA;png(&amp;quot;S7.png&amp;quot;)&#xA;heatmap.2(as.matrix(df), Rowv = T, Colv = FALSE, dendrogram = &amp;quot;row&amp;quot;, #scale = &amp;quot;row&amp;quot;, col = col, density.info = &amp;quot;none&amp;quot;, trace = &amp;quot;none&amp;quot;, margins = c(7, 15) )&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For sample data, this script generates perfect heatmap, but when I have more raw data(genes), this problem comes&#xA;I've tried to change these margins = c(7, 15) value, doesn't work well and not sure how to proceed.&#xA;Also, I want to reduce the height of the color key&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ogzDU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ogzDU.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'd appreciate your help with this.&lt;/p&gt;&#xA;" OwnerUserId="3107" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-07-23T10:31:11.520" Title="How to scale the size of heat map and row names font size?" Tags="|r|rna-seq|proteins|clustering|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4743" PostTypeId="1" AcceptedAnswerId="4746" CreationDate="2018-07-22T07:32:07.540" Score="5" ViewCount="486" Body="&lt;p&gt;How can I interpret the PCA results statistically for biological data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used FactoMineR and factoextra libraries for PCA&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Scripts used:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;library(FactoMineR)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;res.PCA = PCA(df, scale.unit=TRUE, ncp=4, graph=F )&#xA;par(mfrow=c(1,2))&#xA;plot.PCA(res.PCA, axes=c(1, 2), choix=&quot;ind&quot;)&#xA;plot.PCA(res.PCA, axes=c(1, 2), choix=&quot;var&quot;)&#xA;dimdesc(res.PCA, axes=c(1,2))&#xA;&#xA;library(&quot;factoextra&quot;)&#xA;fviz_pca_var(res.PCA, arrowsize = 1, labelsize = 3, repel = TRUE, col.var = &quot;contrib&quot;, gradient.cols = c(&quot;#00AFBB&quot;, &quot;#E7B800&quot;, &quot;#FC4E07&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is my PCA output of differnt KO genotypes); how can I explain/interpret the PCA output statistically and if possible biologically: esp, how this &lt;b&gt;colors /contrib explain&lt;/b&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tAo03.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tAo03.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2881" LastEditorUserId="48" LastEditDate="2018-07-23T10:45:34.567" LastActivityDate="2018-07-23T10:45:34.567" Title="How to interpret PCA output statistically and biologically?" Tags="|r|pca|proteomics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4744" PostTypeId="1" AcceptedAnswerId="4754" CreationDate="2018-07-22T15:40:26.600" Score="1" ViewCount="147" Body="&lt;p&gt;I came across an &lt;a href=&quot;https://academic.oup.com/bioinformatics/advance-article-abstract/doi/10.1093/bioinformatics/bty155/4934942?redirectedFrom=fulltext&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; which has defined 3 categories: Buried [A, L, F, C, G, I, V, W], Exposed [ R, K, Q, E, N, D] and Intermediate [M, S, P, T, H, Y] when analyzing the solvent accessibility of a protein. But I could not find any reference validating the claim. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/29/16/2056/204670&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; has come up with 4 divisions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;B=completely buried (0-4% exposed)&lt;br&gt;&#xA;b=partly buried (4-25% exposed)&lt;br&gt;&#xA;e=partly exposed (25-50% exposed)&lt;br&gt;&#xA;E=completely exposed (50+% exposed)   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is authentic when dealing with solvent accessibility, and which should be used? Also for the second article [B,b,e,E], which amino acid corresponds to which class?&lt;/p&gt;&#xA;" OwnerUserId="2817" LastEditorUserId="48" LastEditDate="2018-07-23T10:52:22.623" LastActivityDate="2018-07-23T13:30:32.503" Title="Which is the amino acids classification when analyzing the solvent accessibility of a protein?" Tags="|proteins|protein-structure|3d-structure|computation|secondary-structure|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4745" PostTypeId="2" ParentId="4737" CreationDate="2018-07-22T17:16:33.033" Score="1" Body="&lt;p&gt;I was able to get good annotation using prokka by using following commands in prokka.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;prokka --proteins reference.gb --outdir annotation  --prefix myprotein contigs.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;All the reading frames were annotated. Actually the ones that are hypothetical in reference genomes are also annotated. &lt;/p&gt;&#xA;" OwnerUserId="1684" LastActivityDate="2018-07-22T17:16:33.033" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4746" PostTypeId="2" ParentId="4743" CreationDate="2018-07-22T18:44:26.423" Score="5" Body="&lt;p&gt;All that plots like this are telling you is that there are some genes that contribute more to the variability seen between your various samples than others. In an ideal world these genes will also be differentially expressed between your groups, but since we don't live in an ideal world that might not be the case. In general, the longer the line and the closer it is to the X axis the more it contributes to the overall variation (since it's contributing more the PC1, which is 53.5% of the variation in your data).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you've asked in the comments for a brief &quot;results and description&quot; section style summary of such a PCA I'll include something brief below. Note that one should not over-interpret PCA plots. The axes are (mathematically) orthogonal and don't necessarily correspond to anything biologically relevant.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We performed a PCA on the variance-stabilized counts to check for batch effects and overall clustering of the data. As can be seen, the &quot;3T&quot; and &quot;5T&quot; groups cluster together along the first principal component, while the &quot;0T&quot; and &quot;1T&quot; samples cluster on the opposite side. This suggests that these groups are more similar to each other, as further evidenced by ...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Assuming you had used the &lt;code&gt;vst()&lt;/code&gt; function in DESeq2 (I note you've tagged your post with RNAseq, so perhaps you're using DESeq2) then something like that above would be a reasonable description.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-07-23T06:50:04.617" LastActivityDate="2018-07-23T06:50:04.617" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4747" PostTypeId="2" ParentId="4742" CreationDate="2018-07-22T18:52:15.193" Score="3" Body="&lt;p&gt;Make the image itself bigger (e.g., &lt;code&gt;png(&quot;S7.png&quot;, width=1000, height=1000)&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that, rethink the utility of having the labels there. Are you really going to go through and look at them if you have hundreds of lines? Probably not.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-07-22T18:52:15.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4748" PostTypeId="1" AcceptedAnswerId="4752" CreationDate="2018-07-23T07:19:13.290" Score="8" ViewCount="2013" Body="&lt;p&gt;Can anyone suggest a software/method for kmer analysis using PacBio reads (RSII)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something similar to &lt;a href=&quot;http://www.cbcb.umd.edu/software/jellyfish/&quot; rel=&quot;noreferrer&quot;&gt;Jellyfish&lt;/a&gt;, that I saw in a nice &lt;a href=&quot;https://bioinformatics.uconn.edu/genome-size-estimation-tutorial/#&quot; rel=&quot;noreferrer&quot;&gt;tutorial&lt;/a&gt; - but must be suitable for long, noisy reads. &lt;a href=&quot;https://github.com/BioInfoTools/BBMap/blob/master/sh/kmercountexact.sh&quot; rel=&quot;noreferrer&quot;&gt;kmercounterexact from BBMapTools&lt;/a&gt; might also be an option, but again tricky with long, noisy reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The closest solution &lt;a href=&quot;http://pb-falcon.readthedocs.io/en/latest/tutorial.html#assembly-graph-and-pread-overlaps&quot; rel=&quot;noreferrer&quot;&gt;I could find&lt;/a&gt; was to estimate the genome size from overlap statistics (for the raw reads, the error corrected preassembled reads or by mapping the raw reads to the assembled contigs) by: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;running an initial assembly using a small seed read length&lt;/li&gt;&#xA;&lt;li&gt;then plot the pre-assembled read overlap histogram.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What do you think of this approach?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to use this kmer analysis to estimate genome size, but for now I could not find much. I do not have companion Illumina reads - if so, I could try the approach proposed by &lt;a href=&quot;https://genome.cshlp.org/content/26/12/1710.full.pdf+html&quot; rel=&quot;noreferrer&quot;&gt;Carvalho et al. 2016&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have reads &amp;amp; in-house genome assemblies (Canu) from different isolates of the same species of fungus to compare. &lt;/p&gt;&#xA;" OwnerUserId="294" LastEditorUserId="294" LastEditDate="2018-07-23T07:26:20.550" LastActivityDate="2020-04-19T18:34:18.857" Title="estimate genome size: kmer-based approach from PacBio reads" Tags="|genome|software-recommendation|assembly|k-mer|pacbio|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4749" PostTypeId="2" ParentId="4739" CreationDate="2018-07-23T08:04:50.023" Score="1" Body="&lt;p&gt;I would run them through the &lt;a href=&quot;http://www.ensembl.org/info/docs/tools/vep/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl VEP&lt;/a&gt;, which will tell you which genes they hit and the effect they have on them. Then I would filter by consequence &quot;mature_miRNA_variant&quot;, which will find you all the variants in your file that fall in the mature miRNAs. Or you could filter instead by the gene biotype &quot;miRNA&quot; which will tell you if the variants fall anywhere in the miRNAs (which will include the full stem loop structure). I would then export the list of miRNAs and try to find out about their targets using &lt;a href=&quot;http://www.mirbase.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;miRbase&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2018-07-23T08:04:50.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4750" PostTypeId="1" CreationDate="2018-07-23T08:38:04.690" Score="0" ViewCount="372" Body="&lt;p&gt;I want to code Moreau-Broto autocorrelation, Moran autocorrelation, Geary autocorrelation in Python 3.6 from the scratch for amino acid sequences without using any package. Unfortunately, I have not been able to come across a proper literature which shows how these autocorrelation indices have to be calculated. Can anyone give a clear explanation as to how to calculate it or at least give a link which has got a clear description of how to calculate these?&lt;/p&gt;&#xA;" OwnerUserId="2817" LastEditorUserId="2817" LastEditDate="2018-07-24T08:57:43.267" LastActivityDate="2018-07-24T08:57:43.267" Title="How to calculate Moreau-Broto autocorrelation, Moran autocorrelation, Geary autocorrelation from protein (amino acid) sequences?" Tags="|python|gene|proteins|literature-search|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4751" PostTypeId="2" ParentId="4742" CreationDate="2018-07-23T10:31:11.520" Score="5" Body="&lt;p&gt;&lt;code&gt;heatmap.2&lt;/code&gt; is very configurable, and has options to adjust the things you want to fix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;cexRow&lt;/code&gt;: changes the size of the row label font. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;keysize&lt;/code&gt;: numeric value indicating the size of the key.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The size of the key is also affected by the layout of the plot. &lt;code&gt;heatmap.2&lt;/code&gt; splits your plotting device into 4 panes (see the picture below), and you can control the size of the key partly by controlling the size and layout of these 4 panes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QT5Yc.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QT5Yc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The layout and scale of these panes can be controlled using the &lt;code&gt;lmat&lt;/code&gt;, &lt;code&gt;lwid&lt;/code&gt; and &lt;code&gt;lhei&lt;/code&gt; arguments. For example, in your case, because you're not displaying the column dendrogram in your plot, you could set the height of the first row of the layout to be smaller, which would reduce the size of the colour key: &lt;code&gt;lhei=c(2, 10)&lt;/code&gt; (this is just an example, you'd need to experiment to find values which worked well for your specific heatmap).&lt;/p&gt;&#xA;" OwnerUserId="194" LastActivityDate="2018-07-23T10:31:11.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4752" PostTypeId="2" ParentId="4748" CreationDate="2018-07-23T11:12:42.180" Score="5" Body="&lt;p&gt;I don't think there is a method that would estimate a genome size using raw long reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The genome size estimates based on raw reads are done by fitting a model to kmer spectra (for instance &lt;a href=&quot;https://github.com/schatzlab/genomescope&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genomescope&lt;/a&gt;). The kmer spectra built from long reads are really messy due to the high error rate of long reads. That makes fitting of a model quite difficult. These methods assume small error rates. If the error rate is as high as 0.1 (which is normal for long reads), then the probability of having one kmer of 23 bases correct is ~8%. Meaning that if the coverage is 100x, just ~8x of homozygous kmers would be the real genomic kmers and it's very likely that kmer peaks of the sequencing errors, heterozygous kmers and homozygous kmers will just blend into one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can imagine, that if your coverage would crazy high (like 600x or more) the kmer spectra could separate errors and genomic kmers. You could also try to decrease kmer size to increase the proportion of correct kmers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another option would be to use a trick that they used to generate solid kmers for &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5206522/&quot; rel=&quot;nofollow noreferrer&quot;&gt;genome assembly using generalized de brujin graph using noisy reads&lt;/a&gt;. Maybe it would be better to take a look on the newer version of the assembler, that is called &lt;a href=&quot;https://github.com/fenderglass/Flye&quot; rel=&quot;nofollow noreferrer&quot;&gt;Flye&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But to be honest, I think for a long reads, a genome assembly is much more reliable way how to predict a genome size than any of the kmer tricks. I would simply assemble the genome, remap back reads, check the uniformity of coverage and conclude the genome size out it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- edit --&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to find some comprehensive evaluation of the assembly sizes compared to genome sizes, but I could not, so I just checked two examples, one big, one small and they both match quite well. &lt;em&gt;Ammopiptanthus nanus&lt;/em&gt; was assembled to 823.74 Mb while kmer estimate was 889 Mb (&lt;a href=&quot;https://academic.oup.com/gigascience/article/7/7/giy074/5039704&quot; rel=&quot;nofollow noreferrer&quot;&gt;ref&lt;/a&gt;). &lt;em&gt;Plasmodium falciparum&lt;/em&gt; was assembled to &lt;a href=&quot;https://wellcomeopenresearch.org/articles/3-52/v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;23.5 Mb&lt;/a&gt; while the original genome size estimate is 22.8 Mb. I know it's just two, but there is also a good reason to think that the genome size will be reasonable, feel free to add here more examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would be worried about contamination in the assembly, you can first run it though one of the methods to detect a contamination (like &lt;a href=&quot;https://github.com/blaxterlab/blobology&quot; rel=&quot;nofollow noreferrer&quot;&gt;Blobology&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2018-08-26T20:17:35.090" LastActivityDate="2018-08-26T20:17:35.090" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4753" PostTypeId="1" AcceptedAnswerId="4789" CreationDate="2018-07-23T11:41:12.280" Score="3" ViewCount="184" Body="&lt;p&gt;I want to analyze RNA seq data using R package &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/cummeRbund.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;cummeRbund&lt;/a&gt;. Using this package I have a problem changing sample names. When I run this command in cummeRbund&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samples(cuff) &#xA;sample_index sample_name sample_name parameter value&#xA;1            1          q1        &amp;lt;NA&amp;gt;      &amp;lt;NA&amp;gt;  &amp;lt;NA&amp;gt;&#xA;2            2          q2        &amp;lt;NA&amp;gt;      &amp;lt;NA&amp;gt;  &amp;lt;NA&amp;gt;&#xA;3            3          q3        &amp;lt;NA&amp;gt;      &amp;lt;NA&amp;gt;  &amp;lt;NA&amp;gt;&#xA;4            4          q4        &amp;lt;NA&amp;gt;      &amp;lt;NA&amp;gt;  &amp;lt;NA&amp;gt;&#xA;5            5          q5        &amp;lt;NA&amp;gt;      &amp;lt;NA&amp;gt;  &amp;lt;NA&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It shows q1,q2,q3,q4 and q5 as sample names for the five samples but I want to change the sample names and put following names for the samples Batis, Blue_silver, Chakwal_50, Local_white and Syn_22 for q1,q2,q3,q4 ad q5 respectively. To accomplish this I ran following command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; samples(cuff) &amp;lt;- c(&quot;Batis&quot;, &quot;Blue_silver&quot;,&quot;Chakwal_50&quot;,&quot;Local_white&quot;,&quot;Syn_22&quot;)&#xA;Error in samples(cuff) &amp;lt;- c(&quot;Batis&quot;, &quot;Blue_silver&quot;, &quot;Chakwal_50&quot;, &quot;Local_white&quot;,  : &#xA;could not find function &quot;samples&amp;lt;-&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but this command gave error as shown above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also the BAM files have same names for all the samples as mentioned above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could I change the sample names to desired names in R using cummeRbund? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or I need to re-run cuffdiff and change the names somewhere else?  &lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-07-23T13:49:55.303" LastActivityDate="2018-07-28T11:24:25.427" Title="Change sample names while using cummeRbund" Tags="|rna-seq|differential-expression|cummerbund|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4754" PostTypeId="2" ParentId="4744" CreationDate="2018-07-23T13:30:32.503" Score="2" Body="&lt;p&gt;There is some confusion in the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solvent accessibility of a residue, i.e. the area accessible to the solvent (water), depends on the 3D structure of the protein. It's not a property of  an amino-acid (chemical compound such as alanine, leucine, ...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There may be a correlation -- amino-acids with hydrophobic side chains are more likely to be buried, but in general amino-acids (chemical compounds) cannot be classified as buried or exposed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have access to the first article, but I tried running the prediction server described in the second linked article, and I got a result&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GSGMQIFVKTFRDRLRNLLPQTITLEVEPSDTIENVKAKIQDKEGIPPDQQVLIFSRKRL&#xA;CCCCEEEEEECEEEEEECCCCEEEEEECCCCCHHHHHHHHHHHHCCCHHHEEEEECCEEC&#xA;EEEBeBbBebbEbEbEebEEeEbeBEBeeEeeBeeBbEeBeEeEEbEeEeBeBbbEEeeb&#xA;&#xA;...&#xA;&#xA;Porter (Secondary Structure):&#xA;H = Helix   (DSSP classes H, G and I)&#xA;E = Strand  (DSSP classes E and B)&#xA;C = Coil    (DSSP classes S, T and .)&#xA;&#xA;PaleAle (Relative Solvent Accessibility):&#xA;B = very buried      (&amp;lt;=4% accessible)&#xA;b = somewhat buried  (&amp;gt;4% and &amp;lt;=25% accessible)&#xA;e = somewhat exposed (&amp;gt;25% and &amp;lt;=50% accessible)&#xA;E = very exposed     (&amp;gt;50% accessible)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In one place in the sequence you can see &lt;code&gt;QQ&lt;/code&gt;, predicted as &lt;code&gt;eB&lt;/code&gt;.&#xA;So clearly glutamine is NOT assigned to a particular RSA class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dividing RSA (relative solvent accessibility, 0-100%) into categories is arbitrary, I don't think there is any widely used convention. The easiest way would be to just use numeric values.&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2018-07-23T13:30:32.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4755" PostTypeId="1" CreationDate="2018-07-23T14:35:52.743" Score="1" ViewCount="69" Body="&lt;p&gt;I have next &lt;code&gt;SNPs&lt;/code&gt; data.frame structure&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; str(SNPs)&#xA;'data.frame':   1703 obs. of  4 variables:&#xA; $ group: Factor w/ 2 levels &quot;A&quot;,&quot;B&quot;: 1 1 1 1 1 1 1 1 1 1 ...&#xA; $ rs1  : Factor w/ 3 levels &quot;D/D&quot;,&quot;I/D&quot;,&quot;I/I&quot;: 1 1 2 3 3 2 1 1 1 1 ...&#xA; $ rs2  : Factor w/ 3 levels &quot;a/a&quot;,&quot;a/b&quot;,&quot;b/b&quot;: 3 3 2 3 3 2 2 3 3 2 ...&#xA; $ rs3  : Factor w/ 3 levels &quot;G/G&quot;,&quot;G/T&quot;,&quot;T/T&quot;: 2 1 2 1 1 3 1 1 2 1 ...&#xA; ...other rs&#xA;&#xA;&#xA;&amp;gt; head(SNPs)&#xA;  group rs1 rs2 rs3 ...other rs&#xA;1     A D/D b/b G/T&#xA;2     A D/D b/b G/G&#xA;3     A I/D a/b G/T&#xA;4     A I/I b/b G/G&#xA;5     A I/I b/b G/G&#xA;6     A I/D a/b T/T&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For example, I noticed that &lt;code&gt;rs5 4b/4a&lt;/code&gt; and &lt;code&gt;rs6 G/G&lt;/code&gt; in &lt;code&gt;group A&lt;/code&gt; very often occur together (see below). In &lt;code&gt;group B&lt;/code&gt; they very often occur together too. So I want to know - is it statistical regularity or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can create table with all pairs in both groups&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; SNPs$rs5_rs6 &amp;lt;- paste(SNPs$rs5, SNPs$rs6)&#xA;&amp;gt; tmp &amp;lt;- table(SNPs$rs5_rs6, SNPs$group)&#xA;&amp;gt; tmp&#xA;&#xA;              A   B&#xA;  4a/4a G/G   1  20&#xA;  4b/4a G/G  31  83&#xA;  4b/4a G/T  14  51&#xA;  4b/4a T/T   1   0&#xA;  4b/4b G/G  37 106&#xA;  4b/4b G/T  35 119&#xA;  4b/4b T/T  11  31&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, now I need compare (find p-value) group A and group B: &lt;code&gt;4a/4a G/G&lt;/code&gt; in group A vs &lt;code&gt;4a/4a G/G&lt;/code&gt; in group B, &lt;code&gt;4b/4a G/G&lt;/code&gt; in group A vs &lt;code&gt;4b/4a G/G&lt;/code&gt; in group B, &lt;code&gt;4b/4a G/T&lt;/code&gt; in group A vs &lt;code&gt;4b/4a G/T&lt;/code&gt; in group B and etc. Only this pairs/&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I do that?&#xA;I need p-values for all &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; tmp&#xA;&#xA;              A   B&#xA;  4a/4a G/G   1  20 - p-value?&#xA;  4b/4a G/G  31  83 - p-value?&#xA;  4b/4a G/T  14  51 - p-value?&#xA;  4b/4a T/T   1   0 - p-value?&#xA;  4b/4b G/G  37 106 - p-value?&#xA;  4b/4b G/T  35 119 - p-value?&#xA;  4b/4b T/T  11  31 - p-value?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3108" LastEditorUserId="3108" LastEditDate="2018-07-23T15:29:10.203" LastActivityDate="2018-09-21T17:02:32.547" Title="To find p-value of compounds (pairwise)" Tags="|r|gene|snp|statistics|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4756" PostTypeId="1" AcceptedAnswerId="4766" CreationDate="2018-07-23T15:19:55.697" Score="2" ViewCount="49" Body="&lt;p&gt;I want to get all Pubchem Chemical compounds (CID) that interact with a given UniProt target. I have a list of UniProt targets and I need to study the relationship between these targets and chemical compounds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="2173" LastEditorUserId="1769" LastEditDate="2018-07-27T11:16:30.347" LastActivityDate="2018-07-27T11:16:30.347" Title="How to get all chemical compounds that interact with a given protein target?" Tags="|uniprot|interactions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4757" PostTypeId="2" ParentId="4755" CreationDate="2018-07-23T16:02:05.047" Score="2" Body="&lt;p&gt;You can get standardised residuals from &lt;code&gt;chisq.test&lt;/code&gt; and convert these to p-values, but it's &lt;a href=&quot;https://stats.stackexchange.com/questions/129194/converting-chi-square-adjusted-residuals-to-p-values&quot;&gt;not recommended&lt;/a&gt;. The frequencies of each genotype depend on the counts of all other genotypes, so the p-values aren't independent and they aren't corrected for multiple testing, but if you must:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pnorm(-abs(chisq.test(tmp[,c('A','B')])$stdres)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-07-23T16:02:05.047" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="4758" PostTypeId="2" ParentId="4737" CreationDate="2018-07-23T16:09:50.017" Score="1" Body="&lt;p&gt;If you are looking for another tool, you could try &lt;a href=&quot;https://github.com/jhcepas/eggnog-mapper&quot; rel=&quot;nofollow noreferrer&quot;&gt;eggnog mapper&lt;/a&gt;: it is an annotation tool based on orthology assignments. The EggNOG database happens to have a section for &lt;a href=&quot;http://eggnogdb.embl.de/#/app/viruses&quot; rel=&quot;nofollow noreferrer&quot;&gt;viral Orthologous Groups&lt;/a&gt; so you might obtain better results with it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that emapper annotates coding sequences, not whole genomes. You need an extra step to predict CDS, for instance using &lt;a href=&quot;https://github.com/hyattpd/Prodigal&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prodigal&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basic usage with the optimized database for viral models:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;prodigal -i my.genome.fna -o my.genes -a my.proteins.faa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python emapper.py -i my.proteins.faa --output polb_viruses  -d viruses&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2580" LastEditorUserId="2580" LastEditDate="2018-07-26T06:56:47.623" LastActivityDate="2018-07-26T06:56:47.623" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4759" PostTypeId="1" AcceptedAnswerId="4760" CreationDate="2018-07-23T16:13:31.653" Score="1" ViewCount="177" Body="&lt;p&gt;Reading Cusanovich et al. (2015) I encountered the sentence:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We mixed pairs of cell lines (HEK293T or HL-60 versus GM12878), performed combinatorial cellular indexing, and sequenced the resulting libraries to saturation (65% duplicate rate).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How is PCR duplication rate computed?&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Cusanovich, D. A. et al. &lt;a href=&quot;http://science.sciencemag.org/content/348/6237/910&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multiplex single-cell profiling of chromatin accessibility by combinatorial cellular indexing.&lt;/a&gt; Science 348, 910–914 (2015).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-07-23T17:01:51.810" Title="How is PCR duplication rate computed in scATAC-seq?" Tags="|single-cell|atac-seq|pcr|duplications|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4760" PostTypeId="2" ParentId="4759" CreationDate="2018-07-23T17:01:51.810" Score="3" Body="&lt;p&gt;The answer is in the &lt;a href=&quot;http://science.sciencemag.org/content/sci/suppl/2015/05/06/science.aab1601.DC1/Cusanovich-SM.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;supplementary methods page 3&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Finally, all fragments in the same library with duplicate start and&#xA;  end coordinates were removed using Picard&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="64" LastActivityDate="2018-07-23T17:01:51.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4761" PostTypeId="2" ParentId="4702" CreationDate="2018-07-23T19:08:13.030" Score="2" Body="&lt;p&gt;The number of contigs and total assembly size you have suggest that there was probably more in the sequencing run than a single virus strain. Does the total assembly length correspond to expected genome size of the reference strain? Have you tried just to blast some your contigs? Could you have a contamination or something?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe you can just sort out the contigs by the reference using something like &lt;a href=&quot;https://www.sanger.ac.uk/science/tools/artemis&quot; rel=&quot;nofollow noreferrer&quot;&gt;artemis&lt;/a&gt; or &lt;a href=&quot;http://bioinf.spbau.ru/quast&quot; rel=&quot;nofollow noreferrer&quot;&gt;quast&lt;/a&gt; compares your assembly to reference and showing nice stats (just to be sure that your assembly makes sense so far). Or you could use directly &lt;a href=&quot;https://github.com/mummer4/mummer&quot; rel=&quot;nofollow noreferrer&quot;&gt;MUMmer&lt;/a&gt; (which the alignment tool that is running on the backend of of Medusa and Quast).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not directly answering your question &quot;How to scaffold&quot;, however I think you should first figure out if your contigs make sense before you scaffold them. Also sorting your contigs by artemis using reference is practically the same thing as scaffolding since you won't know the sequence in between in either of the cases.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2018-07-23T19:08:13.030" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4762" PostTypeId="2" ParentId="4750" CreationDate="2018-07-24T07:27:51.650" Score="2" Body="&lt;p&gt;Rajarshi Guha describes the formulas to calculate &lt;a href=&quot;http://www.rguha.net/writing/notes/desc/node2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Moreau Broto autocorrelation, Moran coefficient and Geary coefficient&lt;/a&gt; and has a &lt;a href=&quot;http://www.rguha.net/writing/notes/desc/node8.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bibliography&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But why reinvent the wheel? It seems that the &lt;a href=&quot;http://pybiomed.readthedocs.io/en/latest/_modules/Autocorrelation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Autocorrelation module&lt;/a&gt; of the &lt;a href=&quot;http://pybiomed.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyBioMed&lt;/a&gt; package does already what you want: it is a pure Python implementation without any external dependencies.&lt;/p&gt;&#xA;" OwnerUserId="450" LastActivityDate="2018-07-24T07:27:51.650" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4764" PostTypeId="2" ParentId="4373" CreationDate="2018-07-24T14:06:02.363" Score="3" Body="&lt;p&gt;As the graph of sequences depends on the sequences in the input, if you change the input sequences (by deleting some sequences) you change the graph.&lt;br&gt;&#xA;The alignment between the sequences changes, and thus the relative distance between them changes too. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The deletion of some sequences won't cause any change on other sequences only if the full space of the C(n 2) pairwise comparisons would be done. However for a large number (and long) of sequences, this would require too much computational effort and time, making such methods as the k-tuple distance competitive.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-07-24T14:06:02.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4765" PostTypeId="1" AcceptedAnswerId="4767" CreationDate="2018-07-24T14:29:33.140" Score="6" ViewCount="145" Body="&lt;p&gt;I have 370 samples to sequence, we probably will end up using only 96 samples per run (due to the barcode with the primers we'll use). This means running 4 batches. To minimize the batch effect I need to decide which samples are sequenced in which run. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have several variables from these samples, some of them are categoric (sex, disease status, treatment stage, ...) and some of them are numeric (age, clinical score...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To minimize the batch effect I wouldn't group all samples from the same category in one run, and I would distribute as uniformly as possible the samples according to the numeric variables. However, I could find many papers about adjusting, taking into account, but I couldn't find a paper about software to help designing the runs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any software to help designing each batch?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there are several I would like to point out that I would define best as the one with easier user interface&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2018-07-24T15:06:13.060" LastActivityDate="2018-10-01T07:19:54.767" Title="Deciding which samples go in which batch" Tags="|ngs|software-recommendation|design|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4766" PostTypeId="2" ParentId="4756" CreationDate="2018-07-24T14:40:15.183" Score="0" Body="&lt;p&gt;The easiest solution to the problem of mapping chemical compounds to protein IDs is to use the ChEMBL compound ID to UniProt &lt;a href=&quot;ftp://ftp.ebi.ac.uk/pub/databases/chembl/ChEMBLdb/releases/chembl_24_1/chembl_uniprot_mapping.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;mapping&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing this with PubChem is not so straight forward in part because its downloadable data format is a hideous obfuscated and badly documented mess split over multiple compressed files which in turn contain multiple compressed files...... For a publicly funded project they should really do better.&lt;/p&gt;&#xA;" OwnerUserId="1769" LastActivityDate="2018-07-24T14:40:15.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4767" PostTypeId="2" ParentId="4765" CreationDate="2018-07-24T15:53:29.963" Score="3" Body="&lt;ul&gt;&#xA;&lt;li&gt;I found the &lt;a href=&quot;http://bioconductor.org/packages/OSAT/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OSAT package&lt;/a&gt; which does almost what I want. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However it is specific for Illumina and for a limited number of plates: &quot;plates that hold 2, 4, 8 IlluminaBeadChip chips and have 24, 48, 96 wells, respectively.&quot;&lt;br&gt;&#xA;It doesn't take in consideration the numeric values:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(OSAT)&#xA;inPath &amp;lt;- system.file('extdata', package='OSAT')&#xA;pheno &amp;lt;- read.table(file.path(inPath, 'samples.txt'), &#xA;                    header=TRUE, sep=&quot;\t&quot;, colClasses=&quot;factor&quot;)&#xA;pheno$AgeGrp &amp;lt;- runif(576, 0, 100)&#xA;gs &amp;lt;- setup.sample(pheno, optimal=c(&quot;SampleType&quot;, &quot;Race&quot;, &quot;AgeGrp&quot;))&#xA;gc &amp;lt;- setup.container(IlluminaBeadChip96Plate, 6, batch='plates')&#xA;gSetup &amp;lt;- create.optimized.setup(sample=gs, container=gc, nSim=1000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tNofr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tNofr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;https://github.com/m-Py/minDiff&quot; rel=&quot;nofollow noreferrer&quot;&gt;minDiff&lt;/a&gt; package reported in &lt;a href=&quot;https://stats.stackexchange.com/a/326015/105234&quot;&gt;Stats.SE&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It handles both numeric and categorical data, but it can only optimize for two nominal criteria.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I wrote &lt;a href=&quot;http://github.com/llrs/experDesign&quot; rel=&quot;nofollow noreferrer&quot;&gt;experDesign&lt;/a&gt; which does everything I wanted&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It can handle both numeric and categorical data for an arbitrary number of variables. It is not restricted to an even number of samples per batch. &lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2018-10-01T07:19:54.767" LastActivityDate="2018-10-01T07:19:54.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4768" PostTypeId="1" CreationDate="2018-07-24T16:52:22.057" Score="2" ViewCount="32" Body="&lt;p&gt;I m trying to align different sequences of length not greater than 50 bp. Could I incorporate additional information such as stacking energy, entropy, bonds etc as additional criteria in the alignment?&lt;/p&gt;&#xA;&lt;p&gt;The sequences are very similar.&lt;/p&gt;&#xA;" OwnerUserId="2981" LastEditorUserId="3848" LastEditDate="2020-07-16T21:34:08.433" LastActivityDate="2020-07-16T21:34:08.433" Title="Alignment using secondary and tertiary features of DNA" Tags="|sequence-alignment|phylogenetics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4770" PostTypeId="1" AcceptedAnswerId="4771" CreationDate="2018-07-24T19:16:30.177" Score="1" ViewCount="786" Body="&lt;p&gt;I have a quick doubt on the output of the Genome Indexing, I have used the STAR program along with genome .fasta file and GFF file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Genome size is 3GB, here is the file output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chrLength.txt&#xA;chrNameLength.txt&#xA;chrName.txt&#xA;chrStart.txt&#xA;genomeParameters.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have another small Genome 60MB in size, I did the genome indexing, here is the file output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chrLength.txt&#xA;chrNameLength.txt&#xA;chrName.txt&#xA;chrStart.txt&#xA;exonGeTrInfo.tab&#xA;exonInfo.tab&#xA;geneInfo.tab&#xA;Genome&#xA;genomeParameters.txt&#xA;SA&#xA;SAindex&#xA;sjdbInfo.txt&#xA;sjdbList.fromGTF.out.tab&#xA;sjdbList.out.tab&#xA;transcriptInfo.tab&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My point here is that, why I got the extra information for my small genome size, but I didn't get the same for the big size genome. I do apply the same procedure for the both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here is the below information. Only difference I made for the large Genome size is (&lt;code&gt;--sjdbOverhang 99 --genomeChrBinNbits 15&lt;/code&gt;) to reduce the memory, but the rest of things are same for small genome.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;NUM_THREADS=12&#xA;mkdir DB&#xA;STAR --runMode genomeGenerate --genomeDir DB \&#xA;    --runThreadN $NUM_THREADS \&#xA;    --genomeFastaFiles XL9_2.fa \&#xA;    --sjdbGTFfile XENLA_Frog.gtf+gff3 \&#xA;    --sjdbOverhang 99 \&#xA;    --genomeChrBinNbits 15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Could anyone give an idea, why there is different, I am new to this field, so I am wondering about the difference in this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheer&#xA;San&lt;/p&gt;&#xA;" OwnerUserId="3113" LastEditorUserId="77" LastEditDate="2018-07-24T20:29:02.853" LastActivityDate="2018-07-24T20:34:02.077" Title="STAR Indexing Diference for Small and Large Genome File Output" Tags="|genome|reference-genome|star|genome-index|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4771" PostTypeId="2" ParentId="4770" CreationDate="2018-07-24T20:34:02.077" Score="1" Body="&lt;p&gt;You didn't get extra information with the small genome, the index for the larger one just wasn't made. My guess is that you ran out of RAM. For mammalian sized genomes ensure you have at least 30GB of RAM, likely more like 60GB. Also, make sure to look for error messages.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-07-24T20:34:02.077" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4772" PostTypeId="2" ParentId="4765" CreationDate="2018-07-25T10:47:16.093" Score="2" Body="&lt;p&gt;I've had to go through this pain, and found the best way to do it was to simulate the model fit to start assessing the balance of covariates. I've got two notebooks available on Github, which should help you deal with this issue, although I've not thought about how to accurately assess continuous variables. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While it's not wrapped in a package, you can use these notebooks as a template to fit to your problem. I may wrap this in a package at some point, but I haven't had the chance to do so yet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/AndrewSkelton/Experimental-Design-Simulation/blob/master/Version2_Experiment.Rmd&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; Notebook will allow you to set up all your variables, max size of a batch (may vary due to multiplexing, library set up, plate size, etc). Bin them into appropriate batches, then create a design matrix, model fit, and test. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apologies in advance that it's a bit of a mess, but if anything isn't clear, open an issue on the repo.  &lt;/p&gt;&#xA;" OwnerUserId="222" LastActivityDate="2018-07-25T10:47:16.093" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4773" PostTypeId="1" CreationDate="2018-07-25T10:56:02.727" Score="0" ViewCount="239" Body="&lt;p&gt;I need to use Gene Expression quantification data (using HTSeq-FPKM-UQ) for breast cancer data on the gdc portal (&lt;a href=&quot;https://portal.gdc.cancer.gov/repository&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://portal.gdc.cancer.gov/repository&lt;/a&gt;) to classify breast cancer subtypes (luminal A, luminal B, HER2-like, basal-like).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, since I don't know the portal, I'm having problems in associate the EXP data and the label.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could get the label either by using the old TCGA-barcode and use a dictionary I already have from an article (but I don't know how to get the barcode from the portal / API) or by using clinical data (but I don't know how to find metadata that link the EXP and the clinical data).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there anyone more used to the portal who could help me?&lt;/p&gt;&#xA;" OwnerUserId="3114" LastActivityDate="2019-01-08T21:56:07.987" Title="GDC data portal cancer subtype" Tags="|public-databases|cancer|data-download|data-retrieval|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4774" PostTypeId="1" CreationDate="2018-07-25T16:16:32.963" Score="0" ViewCount="1690" Body="&lt;p&gt;I am using the R (using EdgeR) for the RNA Seq analysis, I had few batch effect samples like Control vs treatment. Could anyone tell me the best way to remove the batch effects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have looked into the cluster dendrogram and MDS post (using EdgeR), the Control and Treatments are not grouping together, and their is differences in the different batch within Control and Treatment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried this syntax to remove the batch effects for the rest of the analysis to get the DE Genelist. I feel it didn't able to remove the batch effects from the rest of the analysis.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# define the experiment&#xA;Batch &amp;lt;- factor(c(1,2,3,1,2,3))&#xA;Treatment &amp;lt;- factor(c(0,0,0,1,1,1))&#xA;design_noI &amp;lt;- model.matrix(~ 0 + Batch + Treatment)&#xA;&#xA;# define our DGEList&#xA;San &amp;lt;- DGEList(counts=x,genes=gene_info)&#xA;&#xA;A &amp;lt;- aveLogCPM(San)&#xA;y2 &amp;lt;- San[A&amp;gt;1,]&#xA;y2 &amp;lt;- calcNormFactors(y2)&#xA;logCPM &amp;lt;- cpm(y2, log=TRUE, prior.count=5)&#xA;plotMDS(logCPM)&#xA;logCPMR &amp;lt;- removeBatchEffect(y2, Clutch = Clutch)&#xA;plotMDS(logCPMR)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can see the difference in MDS plot on removing the batch effects but I couldn't able to carry the same data flow into my rest of analysis.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# filter the lowly expresed genes and normalize the data&#xA;keep &amp;lt;- rowSums(cpm(San/logCPMR)&amp;gt;1) &amp;gt;= 3&#xA;keep2 &amp;lt;- removeBatchEffect(keep, Batch)&#xA;y_group &amp;lt;- San/logCPMR[keep2, , keep2.lib.size=FALSE]&#xA;y_group &amp;lt;- calcNormFactors(San/logCPMR)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have mention the &lt;code&gt;San/logCPMR&lt;/code&gt;, as &lt;code&gt;San&lt;/code&gt; take as data without remove the batch effects, whereas I think &lt;code&gt;logCPMR&lt;/code&gt; as batch effected removed data frame.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion on this more useful to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please let me know, any other information needed to deal with this batch effects.&lt;/p&gt;&#xA;" OwnerUserId="3113" LastEditorUserId="48" LastEditDate="2018-07-26T07:15:00.923" LastActivityDate="2018-07-26T07:15:00.923" Title="Batch Effects in RNA Seq Sample" Tags="|r|rna-seq|edger|batch-effects|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4776" PostTypeId="2" ParentId="4774" CreationDate="2018-07-25T21:24:38.353" Score="3" Body="&lt;p&gt;The help page for &lt;code&gt;removeBatchEffect()&lt;/code&gt; explicitly states that it should not be used in conjunction with differential expression testing, so don't do that. You already have the batch effect included in your model, so you're already correcting for it when testing for differential expression. If your MDS plots suggest that samples are variably affected by the batch effect then look into the &lt;code&gt;combat()&lt;/code&gt; function from the &lt;code&gt;sva&lt;/code&gt; package.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-07-25T21:24:38.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4777" PostTypeId="2" ParentId="4702" CreationDate="2018-07-26T02:23:08.203" Score="1" Body="&lt;p&gt;I found that Viral NGS Pipeline from broad institute can actually do this. There is a python script called &lt;strong&gt;order_and_orient&lt;/strong&gt; which serves this purpose. Here is the link  &lt;a href=&quot;https://viral-ngs.readthedocs.io/en/latest/assembly.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;viral ngs assembly&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1684" LastActivityDate="2018-07-26T02:23:08.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4779" PostTypeId="2" ParentId="3349" CreationDate="2018-07-26T13:24:53.497" Score="2" Body="&lt;p&gt;Mitochondrial genes do not always have to be removed, though you &lt;strong&gt;may&lt;/strong&gt; need to do so for a variety of technical reasons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mitochondrial reads are innately different from rRNA due to rRNA typically being excluded during normal library preparation (so any of it getting carried over is essentially a contaminant). Some scRNAseq programs (e.g., &lt;a href=&quot;https://github.com/dgrun/RaceID&quot; rel=&quot;nofollow noreferrer&quot;&gt;RaceID&lt;/a&gt;) can use sampling at various stages, which can break if you have a lot of your signal going to mitochondrial transcripts. Then it's not that they should be excluded because they're from mitochondria, but rather that the results are heavily affected by the presence of VERY highly expressed genes and your results are suspect if you don't remove these RNAs. When your methods aren't affected by that (e.g., you use more robust scaling methods) or you don't have super high expression of mtRNA then that's not explicitly needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, sometimes you very much &lt;strong&gt;want&lt;/strong&gt; to keep mtRNAs, since you might be interested in metabolic changes. Since I work in an institute partially dedicated to immunology, we often look at things like metabolic changes involved in immune activation and removing mtRNAs would hinder that. But if you're &lt;em&gt;a priori&lt;/em&gt; not interested in metabolism then excluding genes you won't care about isn't unreasonable.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-07-26T13:24:53.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4780" PostTypeId="2" ParentId="3903" CreationDate="2018-07-27T03:54:22.750" Score="3" Body="&lt;p&gt;If I understand correctly, your question boils down to: parsing a bunch of numbers in a row from a .csv file gives a different result from parsing a bunch of numbers as &lt;em&gt;rows&lt;/em&gt; of a text file.  I that right?  I suggest that you look at the source of your text file.  For example, was the text file created on a Windows host (running bash, or in the command terminal?)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try using this modified version of the unix cat command:&#xA;&lt;code&gt;cat -et text_file.txt&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The very end of each line will have a '\$' sign and that will show you if there are any extra non-visible characters in your rows.  For example, a DOS/Windows formatted file will usually have a ^M at the end of each row before the '$'.  If you don't strip off those line endings then they get added to your queries, even if you cannot see them.  You can strip off those linefeed (or are they carriage returns?) using a Perl one-liner like:&#xA;&lt;code&gt;perl -pe 's/\r//' badfile.txt &amp;gt; goodfile.txt&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3124" LastActivityDate="2018-07-27T03:54:22.750" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4781" PostTypeId="1" AcceptedAnswerId="4782" CreationDate="2018-07-27T08:52:49.947" Score="1" ViewCount="184" Body="&lt;p&gt;I'm using &lt;a href=&quot;https://daler.github.io/pybedtools&quot; rel=&quot;nofollow noreferrer&quot;&gt;pybedtools&lt;/a&gt; to process data, and at some point, I loop over a &lt;code&gt;BedTool&lt;/code&gt; object and use &lt;a href=&quot;http://treyhunner.com/2018/03/tuple-unpacking-improves-python-code-readability/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;unpacking&lt;/em&gt;&lt;/a&gt; to extract coordinates from the &lt;code&gt;Interval&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for interval in my_bedtool:&#xA;    (chrom, start, end, _, _, strand) = interval&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Suppose I want to compute the interval length, as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    length = end - start&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This results in a &lt;code&gt;TypeError: unsupported operand type(s) for -: 'str' and 'str'&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It happens that the unpacked &lt;code&gt;end&lt;/code&gt; and &lt;code&gt;start&lt;/code&gt; variables are strings.&#xA;Converting them to ints is probably suboptimal given that &lt;code&gt;Interval&lt;/code&gt;s can be created by providing ints for the coordinates:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from pybedtool import Interval&#xA;interval = Interval(&quot;X&quot;, 0, 100, strand=&quot;+&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;start&lt;/code&gt; and &lt;code&gt;end&lt;/code&gt; attributes of an &lt;code&gt;Interval&lt;/code&gt; are ints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    (chrom, start, end, strand) = (&#xA;        interval.chrom, interval.start, interval.end, interval.strand)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I think this looks a bit ugly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better approach?&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2018-07-27T09:38:11.213" LastActivityDate="2018-07-27T09:38:11.213" Title="How to efficiently extract coordinates from a `pybedtools.Interval`?" Tags="|python|pybedtools|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4782" PostTypeId="2" ParentId="4781" CreationDate="2018-07-27T08:52:49.947" Score="2" Body="&lt;p&gt;The reason why unpacking an &lt;code&gt;Interval&lt;/code&gt; results in strings instead of ints might be related to how &lt;a href=&quot;https://daler.github.io/pybedtools/intervals.html#indexing-into-interval-objects&quot; rel=&quot;nofollow noreferrer&quot;&gt;indexing by position&lt;/a&gt; works.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even without considering the string/int issue, it seems slightly more efficient to access the interval's attributes rather than unpacking them directly:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [311]: %timeit [(chrom, start, end, strand) for (chrom, start, end, _, _, strand) in my_bedtool]&#xA;306 ms ± 26.7 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)&#xA;&#xA;In [312]: %timeit [(interval.chrom, interval.start, interval.end, interval.strand) for interval in my_bedtool]&#xA;238 ms ± 24.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To resolve the ugliness issue, attribute access can be made more compact through the use of &lt;code&gt;operator.attrgetter&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In [313]: from operator import attrgetter&#xA;&#xA;In [314]: get_interval_attrs = attrgetter(&quot;chrom&quot;, &quot;start&quot;, &quot;stop&quot;, &quot;strand&quot;)&#xA;&#xA;In [315]: %timeit [get_interval_attrs(interval) for interval in my_bedtool]&#xA;239 ms ± 26.7 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="292" LastActivityDate="2018-07-27T08:52:49.947" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4784" PostTypeId="2" ParentId="4661" CreationDate="2018-07-27T13:07:54.097" Score="3" Body="&lt;p&gt;TRAL 0.3.5 has bugs running and parsing Phobos. You can follow progress towards a fix on this issue: &lt;a href=&quot;https://github.com/acg-team/tral/issues/16&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/acg-team/tral/issues/16&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Until the issue is fixed, you should remove Phobos from the repeat detector list in your tral config file (~/.tral/config.ini) and use a different repeat detector.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[sequence]&#xA;    [[repeat_detection]]&#xA;        DNA = TRED, T-REKS, TRF, XSTREAM&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3081" LastActivityDate="2018-07-27T13:07:54.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4785" PostTypeId="2" ParentId="4773" CreationDate="2018-07-27T21:19:21.230" Score="1" Body="&lt;p&gt;I personally find the GDC portal not very user-friendly. However, some of the data is available from other sources. For example, you can download TCGA-BRCA FPKMs and phenotypes from UCSC Xena here: &lt;a href=&quot;https://xenabrowser.net/datapages/?cohort=GDC%20TCGA%20Breast%20Cancer%20(BRCA)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://xenabrowser.net/datapages/?cohort=GDC%20TCGA%20Breast%20Cancer%20(BRCA)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the sample info is already combined and the labels are the classic TCGA barcodes.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2018-07-27T21:19:21.230" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4787" PostTypeId="1" AcceptedAnswerId="4791" CreationDate="2018-07-27T22:00:25.053" Score="1" ViewCount="91" Body="&lt;p&gt;I am new to the world world of Bayesian-MCMC inference and have been working my way through various texts. I am having some difficulty understanding how MCMC works. I understand that it is a way of sampling from the posterior distribution. What is puzzling me, however, is how the probabilities of the posterior distribution have been calculated. That is, if the Markov Chain is sampling from the posterior distribution, that posterior distribution must have probability values on some axis. How were those calculated? &lt;/p&gt;&#xA;" OwnerUserId="2961" LastActivityDate="2018-07-30T07:16:30.700" Title="A basic question about phylogenetic inference with Bayes-MCMC" Tags="|statistics|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4789" PostTypeId="2" ParentId="4753" CreationDate="2018-07-28T11:24:25.427" Score="1" Body="&lt;p&gt;Don't know which file was used by cummeRbund for sample names but replacing q1,q2,q3 and q4  with Batis, Blue_silver, Chakwal_50 and Syn_22 for all the files in the directory containing cuffdiff output worked for me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used sed language for this as shown below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -i 's/q4/Syn_22/g' *&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or more generally you can use &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -i 's/sample_name_shown/sample_name_to_change/g' *&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After that I deleted the sqlite object created by cummeRbund and again created this object using &lt;strong&gt;readCufflinks()&lt;/strong&gt; in R.&#xA;This time sample names are changed as desired:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samples(cuff)&#xA;sample_index sample_name sample_name parameter value&#xA;1            1       Batis        &amp;lt;NA&amp;gt;      &amp;lt;NA&amp;gt;  &amp;lt;NA&amp;gt;&#xA;2            2 Blue_silver        &amp;lt;NA&amp;gt;      &amp;lt;NA&amp;gt;  &amp;lt;NA&amp;gt;&#xA;3            3  Chakwal_50        &amp;lt;NA&amp;gt;      &amp;lt;NA&amp;gt;  &amp;lt;NA&amp;gt;&#xA;4            4      Syn_22        &amp;lt;NA&amp;gt;      &amp;lt;NA&amp;gt;  &amp;lt;NA&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1194" LastActivityDate="2018-07-28T11:24:25.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4790" PostTypeId="1" AcceptedAnswerId="4816" CreationDate="2018-07-28T12:05:14.560" Score="4" ViewCount="2132" Body="&lt;p&gt;How to interpret heat map and dendrogram output for biological data (omics) in words (when writing results and discussion)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should I consider (statistics behind?) and what is the best approach?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is one of my HM for proteomics data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Script being used&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;col &amp;lt;- colorRampPalette(c(&quot;red&quot;,&quot;yellow&quot;,&quot;darkgreen&quot;))(30) &#xA;png(&quot;HM1.png&quot;)&#xA;heatmap.2(as.matrix(df), Rowv = T, Colv = FALSE, dendrogram = &quot;row&quot;, #scale = &quot;row&quot;, col = col, density.info = &quot;none&quot;, trace = &quot;none&quot;, margins = c(7, 15) )&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FefR3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FefR3.png&quot; alt=&quot;Heatmap output&quot;&gt;&lt;/a&gt;&#xA;Heat Map output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be greatful if you would give an example.&lt;/p&gt;&#xA;" OwnerUserId="3107" LastActivityDate="2018-07-31T22:13:24.550" Title="Omics data: How to interpret heatmap and dendrogram output?" Tags="|r|clustering|proteomics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4791" PostTypeId="2" ParentId="4787" CreationDate="2018-07-28T12:14:45.893" Score="2" Body="&lt;p&gt;In short, the posterior is a probability value between 0 and 1 which is calculated from &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Bayes%27_theorem&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayes rule&lt;/a&gt;, which In phylogenetic usage is given as the product of the prior probability and the likelihood divided by a normalising constant. The posterior probability allows you to state the probability of some hypothesis given the data at hand (as opposed to the likelihood parlance, which is probability of data given hypothesis).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a phylogenetic setting, the sampler works (roughly) like this: propose some value of a parameter (topology, branch lengths, etc.) and use the specified prior and calculated likelihood for that parameter value given the data to calculate the unnormalised posterior. Next, propose a new value, and repeat the same step. Compare the two values, and if the posterior probability of the second is greater, accept this parameter value and repeat the cycle again. In this way the Markov Chain moves through posterior space until it reaches a reasonably stable optimal approximation of the posterior distribution. Here, we want to get as many samples as possible to maximise the accuracy of our posterior. The maximum a posteriori (MAP) value of the parameter and its credible intervals, etc, will come from the distribution of these samples. &lt;/p&gt;&#xA;" OwnerUserId="2841" LastEditorUserId="2841" LastEditDate="2018-07-30T07:16:30.700" LastActivityDate="2018-07-30T07:16:30.700" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4792" PostTypeId="1" CreationDate="2018-07-28T13:49:41.230" Score="1" ViewCount="1325" Body="&lt;p&gt;I have installed &lt;code&gt;hmmpress&lt;/code&gt; on my Mac OS X. It is here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;which hmmpress&#xA;/Users/user/anaconda/bin/hmmpress&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is available in my root environment:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;hmmpress -h&#xA;# hmmpress :: prepare an HMM database for faster hmmscan searches&#xA;# HMMER 3.1b2 (February 2015); http://hmmer.org/&#xA;# Copyright (C) 2015 Howard Hughes Medical Institute.&#xA;# Freely distributed under the GNU General Public License (GPLv3).&#xA;# - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -&#xA;Usage: hmmpress [-options] &amp;lt;hmmfile&amp;gt;&#xA;&#xA;Options:&#xA;  -h : show brief help on version and usage&#xA;  -f : force: overwrite any previous pressed files&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't remember exactly how I installed it but I thought that I used conda because it is in the &lt;code&gt;anaconda&lt;/code&gt; folder. However, &lt;code&gt;conda search hmmpress&lt;/code&gt; returns no packages. I thought it could be part of the &lt;code&gt;hmmer package&lt;/code&gt;. Indeed, the hmmer package exists in the bioconda channel. However, if I try &lt;code&gt;conda uninstall hmmer&lt;/code&gt;, it says: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conda uninstall hmmer&#xA;Solving environment: failed&#xA;&#xA;PackagesNotFoundError: The following packages are missing from the target environment:&#xA;  - hmmer&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does anyone know how I can remove &lt;code&gt;hmmpress&lt;/code&gt;? I thought of &lt;code&gt;rm -rf&lt;/code&gt; the &lt;code&gt;/Users/user/anaconda/bin/hmmpress&lt;/code&gt; but it does not seem like a clean solution. There might be dependencies elsewhere that I could also get rid of.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="1194" LastEditDate="2018-07-30T12:47:48.857" LastActivityDate="2018-10-28T23:00:31.260" Title="How to uninstall hmmpress from my conda environment?" Tags="|software-installation|hmmer|bioconda|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4793" PostTypeId="5" CreationDate="2018-07-28T17:23:08.070" Score="0" Body="&lt;p&gt;High-throughput sequencing of RNA-fragments is a powerful technique that has many applications, including but not limited to transcript assembly, qantitation, and differential expression analysis. The results of these analyses is often very large data sets with a high degree of relations between various data types and can be somewhat overwhelming. CummeRbund was designed to help simplify the analysis and exploration portion of RNA-Seq data derrived from the output of a differential expression analysis using cuffdiff with the goal of providing fast and intuitive access to your results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CummeRbund takes the various output files from a cuffdiff run and creates a SQLite database of the results describing appropriate relationships betweeen genes, transcripts, transcription start sites, and CDS regions. Once stored and indexed, data for these features, even across multiple samples or conditions, can be retrieved very efficiently and allows the user to explore subfeatures of individual genes, or genesets as the analysis requires. Numerous plotting functions have been implemented for commonly used visualizations.&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="73" LastEditDate="2018-07-29T01:03:27.587" LastActivityDate="2018-07-29T01:03:27.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4794" PostTypeId="4" CreationDate="2018-07-28T17:23:08.070" Score="0" Body="CummeRbund is an R package that is designed to aid and simplify the task of analyzing Cufflinks RNA-Seq output." OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-07-29T16:09:46.960" LastActivityDate="2018-07-29T16:09:46.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4795" PostTypeId="1" AcceptedAnswerId="5502" CreationDate="2018-07-29T01:44:46.730" Score="9" ViewCount="132" Body="&lt;p&gt;I know that I can use &lt;code&gt;bcftools isec&lt;/code&gt; to compare two VCF files containing single nucleotide variants (SNVs); it will generate four possible output files: one with all of the unique called SNVs in file 1, one with all of the unique called SNVs in file 2, one with all of the SNVs from file 1 that were also found in file 2, and finally one with all of the SNVs from file 2 that were also found in file 1.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a similar type of tool that will allow one to compare structural variants (SVs) between two VCF files?  I am about to tweak one of my Perl scripts (that currently uses the VCF.pm Perl module to parse VCF files) to perform this calculation (because thus far I have not found any tools designed for SVs) but I thought I would ask in this forum in case anyone else has already invented this wheel.&lt;/p&gt;&#xA;" OwnerUserId="3124" LastEditorUserId="48" LastEditDate="2018-07-30T08:39:48.887" LastActivityDate="2018-11-25T17:23:19.423" Title="Best practices for deciding if two structural variants are actually the same variant?" Tags="|vcf|ngs|software-recommendation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4796" PostTypeId="5" CreationDate="2018-07-29T08:00:07.787" Score="0" Body="&lt;p&gt;Clustering is a main task of exploratory data mining, and a common technique for statistical data analysis, used in many fields, including machine learning, pattern recognition, image analysis, information retrieval, bioinformatics, data compression, and computer graphics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cluster analysis itself is not one specific algorithm, but the general task to be solved. It can be achieved by various algorithms that differ significantly in their understanding of what constitutes a cluster and how to efficiently find them. Popular notions of clusters include groups with small distances between cluster members, dense areas of the data space, intervals or particular statistical distributions. Clustering can therefore be formulated as a multi-objective optimization problem.&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-07-30T13:38:44.973" LastActivityDate="2018-07-30T13:38:44.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4797" PostTypeId="4" CreationDate="2018-07-29T08:00:07.787" Score="0" Body="Clustering is the task of grouping a set of objects in such a way that objects in the same group (called a cluster) are more similar (in some sense) to each other than to those in other groups (clusters)." OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-07-29T16:09:55.417" LastActivityDate="2018-07-29T16:09:55.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4799" PostTypeId="5" CreationDate="2018-07-29T17:05:00.500" Score="0" Body="&lt;p&gt;A genome is an organism's complete set of DNA, including all of its genes. In contrast to genetics, which refers to the study of individual genes and their roles in inheritance, genomics aims at the collective characterization and quantification of genes, which direct the production of proteins with the assistance of enzymes and messenger molecules.&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-07-30T07:14:40.413" LastActivityDate="2018-07-30T07:14:40.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4800" PostTypeId="4" CreationDate="2018-07-29T17:05:00.500" Score="0" Body="Genomics is an interdisciplinary field of science focusing on the structure, function, evolution, mapping, and editing of genomes." OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-07-31T14:01:00.877" LastActivityDate="2018-07-31T14:01:00.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4801" PostTypeId="5" CreationDate="2018-07-29T17:13:05.850" Score="0" Body="&lt;p&gt;Aligned sequences of nucleotide or amino acid residues are typically represented as rows within a matrix. Gaps are inserted between the residues so that identical or similar characters are aligned in successive columns. Sequence alignments are also used for non-biological sequences, such as calculating the edit distance cost between strings in a natural language or in financial data. Popular alignment programs include blast, clustal omega, dialign etc.&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-07-31T11:09:13.280" LastActivityDate="2018-07-31T11:09:13.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4802" PostTypeId="4" CreationDate="2018-07-29T17:13:05.850" Score="0" Body="Use this tag to refer to the alignment of nucleotide or peptide sequences, producing an arrangement that maximizes similarity between sequences." OwnerUserId="1194" LastEditorUserId="96" LastEditDate="2019-01-10T08:27:30.807" LastActivityDate="2019-01-10T08:27:30.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4803" PostTypeId="5" CreationDate="2018-07-29T17:23:27.783" Score="0" Body="&lt;p&gt;Sequence analysis includes a very wide range of relevant topics:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The comparison of sequences in order to find similarity, often to&#xA;infer if they are related (homologous)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Identification of intrinsic features of the sequence such as active sites, post translational modification sites,gene-structures, reading frames, distributions of introns and exons and regulatory elements.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Identification of sequence differences and variations such as point mutations and single nucleotide polymorphism (SNP) in order to get the genetic marker.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Revealing the evolution and genetic diversity of sequences and organisms&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Identification of molecular structure from sequence alone&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Defines the fields of population genetics, phylogenetics and any field of comparative 'omics'&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="3848" LastEditDate="2020-07-07T20:28:53.327" LastActivityDate="2020-07-07T20:28:53.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4804" PostTypeId="4" CreationDate="2018-07-29T17:23:27.783" Score="0" Body="Umbrella term for understanding any given nucleic acid sequence code,  either as a locus, loci or genome with itself or as a comparison to comparable nucleic acid sequence code either intraspecifically or interspecifically" OwnerUserId="-1" LastEditorUserId="3848" LastEditDate="2020-07-07T20:28:53.327" LastActivityDate="2020-07-07T20:28:53.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4805" PostTypeId="5" CreationDate="2018-07-29T17:30:01.783" Score="0" Body="&lt;p&gt;Use this tag to refer to the questions related to the R package deseq2&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2020-07-05T23:42:13.743" LastActivityDate="2020-07-05T23:42:13.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4806" PostTypeId="4" CreationDate="2018-07-29T17:30:01.783" Score="0" Body="Deseq2 is an R package for analyzing RNA sequencing data" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2020-07-02T15:52:08.253" LastActivityDate="2020-07-02T15:52:08.253" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4807" PostTypeId="5" CreationDate="2018-07-29T17:33:38.183" Score="0" Body="&lt;p&gt;RNA is a polymeric molecule essential in various biological roles in coding, decoding, regulation, and expression of genes. RNA and DNA are nucleic acids, and, along with lipids, proteins and carbohydrates, constitute the four major macromolecules essential for all known forms of life.&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-07-30T13:37:59.717" LastActivityDate="2018-07-30T13:37:59.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4808" PostTypeId="4" CreationDate="2018-07-29T17:33:38.183" Score="0" Body="Use this tag to refer questions that are related to rna sequence." OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-07-30T07:33:12.140" LastActivityDate="2018-07-30T07:33:12.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4809" PostTypeId="5" CreationDate="2018-07-29T17:36:47.063" Score="0" Body="&lt;p&gt;Reference genomes are typically used as a guide on which new genomes are built, enabling them to be assembled much more quickly and cheaply.For example, GRChg37 is used to refer to human genome build 37. &lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-07-31T11:07:50.223" LastActivityDate="2018-07-31T11:07:50.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4810" PostTypeId="4" CreationDate="2018-07-29T17:36:47.063" Score="0" Body="A reference genome (also known as a reference assembly) is a digital nucleic acid sequence database, assembled by experts as a representative example of a species' set of genes." OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-07-30T07:33:22.043" LastActivityDate="2018-07-30T07:33:22.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4811" PostTypeId="5" CreationDate="2018-07-29T17:38:51.390" Score="0" Body="&lt;p&gt;Typically phylogeneticists study one of the following types of question:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What are the evolutionary relationships or histories among my species/individuals/genes of interest?&lt;/li&gt;&#xA;&lt;li&gt;How do sequences evolve?&lt;/li&gt;&#xA;&lt;li&gt;Can I better describe processes of sequence evolution with a mathematical model?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-07-30T13:39:26.817" LastActivityDate="2018-07-30T13:39:26.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4812" PostTypeId="4" CreationDate="2018-07-29T17:38:51.390" Score="0" Body="Phylogenetics is the study of evolutionary relationships among biological entities - often species, individuals or genes." OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-07-30T16:31:29.700" LastActivityDate="2018-07-30T16:31:29.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4813" PostTypeId="1" CreationDate="2018-07-29T21:06:39.113" Score="1" ViewCount="55" Body="&lt;p&gt;I'm struggling these last days on a case/control phenotype prediction problem based on genotype (individual x SNPs with more than 10k individuals and approx 18k SNPs encoded as 0,1,2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks to PCA analysis (with SNPRelate package), I observed that there are sub-populations in my training set. I suspect that there is a bias in my classifier due these sub-population. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current methodology to improve my classifier accuracy is the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;build one model for each population,&lt;/li&gt;&#xA;&lt;li&gt;during prediction phase based on test dataset -&gt; assign each individual to one of the population identified previously. Apply the&#xA;model built for this population.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My difficulties:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;which packages and methods in R would you advise me to assign a&#xA;cluster to each of my individual ? Could I use PC eigenvectors to&#xA;achieve such clustering ?&lt;/li&gt;&#xA;&lt;li&gt;which packages and methods in R could allow me to associate training genotype to cluster prior to apply the specific model ?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thank in advance for your help.&lt;/p&gt;&#xA;" OwnerUserId="3131" LastEditorUserId="3131" LastEditDate="2018-07-30T08:28:11.307" LastActivityDate="2018-07-30T08:28:11.307" Title="case/control phenotype prediction - population stratification with R" Tags="|r|snp|software-recommendation|clustering|data-preprocessing|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4814" PostTypeId="1" AcceptedAnswerId="4817" CreationDate="2018-07-30T07:27:55.693" Score="3" ViewCount="632" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What is the algorithm/concept behind‚&lt;code&gt;makeblastdb&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I know &lt;code&gt;makeblastdb&lt;/code&gt; produces a &lt;code&gt;.phr&lt;/code&gt;, &lt;code&gt;.pin&lt;/code&gt;, and a &lt;code&gt;.psq&lt;/code&gt; file, that represents an index or a data structure that is better optimised for searching a given query. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is exactly stored in these files?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Seems like &lt;code&gt;.phr&lt;/code&gt; and &lt;code&gt;.pin&lt;/code&gt; are storing some metadata (based on the size) and &lt;code&gt;.psq&lt;/code&gt; the actual index.&#xA;It can't be the table of all k-mers since the command doesn't take a value of k? (or maybe it is fixed to 3)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could not find any direct discussion of the algorithm since google is flooded with people asking how to build such a database...&#xA;Maybe someone can give me a link or hint?&lt;/p&gt;&#xA;" OwnerUserId="2766" LastEditorUserId="1769" LastEditDate="2018-07-31T08:21:27.413" LastActivityDate="2018-08-07T07:50:39.673" Title="The algorithm behind makeblastdb?" Tags="|blast|makeblastdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4815" PostTypeId="1" CreationDate="2018-07-30T07:42:43.557" Score="1" ViewCount="894" Body="&lt;p&gt;I have generated phylogenetic trees for virus sequences by using beast software.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I observed in various research articles that the phylogentic trees also have a scale-bar (which mostly represents residue substitution per site). When I visualized the generated trees in FigTree v1.4.3, the scale bar unit (number) seems to be in thousands! While mostly the scale bar represents 0.1 or 0.5 such numbers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I intend to add a scale bar to my year scale bar to phylogenetic tree, please advice me how to generate the optimal scale and value in accordance with the graphs.&lt;/p&gt;&#xA;" OwnerUserId="2957" LastEditorUserId="48" LastEditDate="2018-07-30T08:33:58.610" LastActivityDate="2018-07-30T08:33:58.610" Title="How to make year scale bar unit in figtree software?" Tags="|sequence-alignment|visualization|phylogenetics|" AnswerCount="0" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4816" PostTypeId="2" ParentId="4790" CreationDate="2018-07-30T07:43:35.940" Score="3" Body="&lt;p&gt;The dendrogram summarize the information of a group of values and sort them according to the similarity they have. It can be applied to both, samples and features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dendrogram allows to visualize features that are more similar together, usually revealing patterns that wouldn't have been seen otherwise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In an article usually it is used something like: &quot;we can see that X and Y are clustered together revealing that ... (they have something in common), while Z and A are clustered far&quot;&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-07-30T07:43:35.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4817" PostTypeId="2" ParentId="4814" CreationDate="2018-07-30T15:25:48.310" Score="2" Body="&lt;p&gt;From looking at this publication &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2696921/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Database indexing for production MegaBLAST searches&quot;&lt;/a&gt; which although deals with MegaBLAST and it's indexing method, it would appear that this was rolled into the NCBI C++ toolkit as the command &lt;code&gt;makembindex&lt;/code&gt; so I'm inclined to think the current implementation must have been derived from this strategy. Looking at the relevant part of the current NCBI C++ toolkit I find no reference to &lt;code&gt;makembindex&lt;/code&gt; but only &lt;code&gt;makeblastdb&lt;/code&gt; which I assume has superseded the older command: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/IEB/ToolBox/CPP_DOC/doxyhtml/dir_30f669065d6b13796daf8e29b61b0d9a.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;blastdb Directory Reference&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ulimtately the true details of the index generation lie in the C++ source code for &lt;code&gt;makeblastdb&lt;/code&gt; can be found &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/IEB/ToolBox/CPP_DOC/doxyhtml/makeblastdb_8cpp_source.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.  However following on from that I think the further details of the index generation may be in one of the following files which are includes for the main &lt;code&gt;.cpp&lt;/code&gt; file:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/IEB/ToolBox/CPP_DOC/doxyhtml/blast_2seqdb__reader_2seqdb_8hpp_source.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;seqdb.hpp&lt;/code&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/IEB/ToolBox/CPP_DOC/doxyhtml/blast_2seqdb__writer_2build__db_8hpp_source.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;build_db.hpp&lt;/code&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/IEB/ToolBox/CPP_DOC/doxyhtml/blast_2seqdb__reader_2seqdbcommon_8hpp_source.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;seqdbcommon.hpp&lt;/code&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/IEB/ToolBox/CPP_DOC/doxyhtml/blast_2seqdb__writer_2writedb_8hpp_source.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;writedb.hpp&lt;/code&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would appear from &lt;code&gt;seqdb.hpp&lt;/code&gt; that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The index files have extension &lt;code&gt;.pin&lt;/code&gt; or &lt;code&gt;.nin&lt;/code&gt; and sequence files &lt;code&gt;.psq&lt;/code&gt; or &lt;code&gt;.nsq&lt;/code&gt;, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However it's not at all clear how the current implementation differs to the 2007 reference or where the &lt;code&gt;.phr&lt;/code&gt; file is generated from!  Some degree of reverse engineering the C++ source code will be needed!&lt;/p&gt;&#xA;" OwnerUserId="1769" LastEditorUserId="1769" LastEditDate="2018-08-07T07:50:39.673" LastActivityDate="2018-08-07T07:50:39.673" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4818" PostTypeId="1" AcceptedAnswerId="4824" CreationDate="2018-07-30T18:47:51.673" Score="5" ViewCount="778" Body="&lt;p&gt;This is a theory/good practice question more than a technical one. If samples are being plotted on a PCA projection of gene expression data, I'm wondering whether it is standard (and if so, why) to center and scale the PCs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I ask is that in this case the variables (genes) are being measured in the same scale, so I don't know if it would be necessary to center/scale data. &lt;/p&gt;&#xA;" OwnerUserId="3139" LastEditorUserId="48" LastEditDate="2018-07-31T07:17:38.310" LastActivityDate="2018-09-17T20:22:01.397" Title="Should PCA be standardized for gene expression?" Tags="|statistics|pca|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4819" PostTypeId="2" ParentId="4792" CreationDate="2018-07-30T21:41:53.073" Score="1" Body="&lt;p&gt;This time &lt;code&gt;conda uninstall hmmer&lt;/code&gt; in the root environment did the job. The sequence of commands that led me there is shown below. Thanks Devon Ryan.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conda list | grep hmmer&#xA;hmmer                     3.1b2                         3    bioconda&#xA;&#xA;conda uninstall hmmer&#xA;Solving environment: done&#xA;&#xA;## Package Plan ##&#xA;&#xA;  environment location: /Users/user/anaconda&#xA;&#xA;  removed specs: &#xA;    - hmmer&#xA;&#xA;&#xA;The following packages will be downloaded:&#xA;&#xA;    package                    |            build&#xA;    ---------------------------|-----------------&#xA;    conda-4.5.9                |           py36_0         626 KB  conda-forge&#xA;&#xA;The following packages will be REMOVED:&#xA;&#xA;    barrnap: 0.9-0        bioconda   &#xA;    busco:   3.0.2-py36_6 bioconda   &#xA;    hmmer:   3.1b2-3      bioconda   &#xA;    prokka:  1.12-3       bioconda   &#xA;&#xA;The following packages will be UPDATED:&#xA;&#xA;    conda:   4.5.8-py36_1 conda-forge --&amp;gt; 4.5.9-py36_0      conda-forge&#xA;    openssl: 1.0.2o-0     conda-forge --&amp;gt; 1.0.2o-h470a237_1 conda-forge&#xA;&#xA;Proceed ([y]/n)? y&#xA;&#xA;&#xA;Downloading and Extracting Packages&#xA;conda-4.5.9          |  626 KB | ################################################################################################################# | 100% &#xA;Preparing transaction: done&#xA;Verifying transaction: done&#xA;Executing transaction: done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Checking that &lt;code&gt;hmmpress&lt;/code&gt; is not available anymore:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;hmmpress&#xA;zsh: command not found: hmmpress&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-07-30T21:41:53.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4820" PostTypeId="1" CreationDate="2018-07-31T04:56:35.533" Score="3" ViewCount="263" Body="&lt;p&gt;I have assembled 3 &lt;code&gt;.fasta&lt;/code&gt; files from contigs in&lt;code&gt;fastq&lt;/code&gt; format of 3 different Homo sapiens.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to see if the assembled genomes which are in &lt;code&gt;.fasta&lt;/code&gt; format are in good quality or not. In addition, I would be happy to see heritages of genes from the maternal or the parental genomes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, I have 3 &lt;code&gt;.fasta&lt;/code&gt; files of a mother, father and daughter. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How can I see they are indeed the biological parents?&lt;/li&gt;&#xA;&lt;li&gt;How can I study what genes were inherited from whom of the parents? &lt;/li&gt;&#xA;&lt;li&gt;How can I assess the quality of the assembled genomes? Is there a standard way to do that? &lt;/li&gt;&#xA;&lt;li&gt;Is there a simple frequency analysis of bases n-grams I can use?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="734" LastEditorUserId="3848" LastEditDate="2019-08-20T20:21:51.883" LastActivityDate="2019-08-20T20:21:51.883" Title="How to assess the quality of assembled .fasta genome files?" Tags="|genome|python|assembly|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4822" PostTypeId="2" ParentId="960" CreationDate="2018-07-31T05:30:55.153" Score="1" Body="&lt;p&gt;You can use biopython's code for generation of PSSM.&#xA;Here is the &lt;a href=&quot;http://biopython.org/DIST/docs/api/Bio.motifs.matrix-pysrc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; for the source code which can help you.&lt;/p&gt;&#xA;" OwnerUserId="3142" LastEditorUserId="3142" LastEditDate="2018-07-31T08:57:10.200" LastActivityDate="2018-07-31T08:57:10.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4824" PostTypeId="2" ParentId="4818" CreationDate="2018-07-31T07:42:31.423" Score="4" Body="&lt;p&gt;When the gene expression is scaled and centred you reduce the difference between genes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagine you have one gene A that is highly expressed usually and has a standard deviation of 500 units compared to a gene B that is not much expressed and only have a standard deviation of 5. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the scaled and centred genes both contribute the same because A usually is expressed 10000 and B is usually expressed 100 units. So, for both the standard deviation is 5% of their expression. Meaning that a variation in one is as important as in the other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not scaled (and centred) the first gene A will contribute more to the variation than gene B, because the expression variation in absolute numbers is bigger. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both are used in publications, but I think the scaled and centred is more used, because the first dimension reflects &quot;better&quot; the differences between samples. Of course you can do both and select the one that is better to show what you want.&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2018-09-17T20:22:01.397" LastActivityDate="2018-09-17T20:22:01.397" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4825" PostTypeId="2" ParentId="4790" CreationDate="2018-07-31T12:16:23.250" Score="3" Body="&lt;p&gt;You may be interested in reading up on heatmaps. For a history perspective (pre the biological introduction by &lt;a href=&quot;http://www.pnas.org/content/95/25/14863&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eisen et al.&lt;/a&gt; ) read &lt;em&gt;&lt;a href=&quot;https://www.cs.uic.edu/~wilkinson/Publications/heatmap.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The History of the Cluster Heat Map&lt;/a&gt;&lt;/em&gt; by Wilkinson and Friendly &lt;/p&gt;&#xA;" OwnerUserId="3085" LastEditorUserId="3085" LastEditDate="2018-07-31T22:13:24.550" LastActivityDate="2018-07-31T22:13:24.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4826" PostTypeId="2" ParentId="960" CreationDate="2018-07-31T13:54:21.277" Score="1" Body="&lt;p&gt;I'm not 100% sure what you're trying to do with regards to your search in the first place, but I recommend using Sean Eddy's program &lt;a href=&quot;https://www.ebi.ac.uk/Tools/hmmer/search/jackhmmer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jackhammer&lt;/a&gt; it's similar to PSI-BLAST but uses Hammer HMM underpinnings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As input it can take a single sequence, a multiple sequence alignment, or an existing HMM and then allow you to search in an iterative model building way.  You can also download an HMM too at any stage in the iterative search, as well as the results / alignment. &lt;/p&gt;&#xA;" OwnerUserId="1769" LastEditorUserId="1769" LastEditDate="2018-07-31T15:17:48.247" LastActivityDate="2018-07-31T15:17:48.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4827" PostTypeId="1" AcceptedAnswerId="4848" CreationDate="2018-07-31T14:07:38.527" Score="0" ViewCount="1661" Body="&lt;p&gt;Dear bioinformaticians,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could find two python APIs to mine ENSEMBL genome databases. There is a &lt;a href=&quot;https://github.com/openvax/pyensembl&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;pyensembl&lt;/code&gt;&lt;/a&gt; which is more popular and then there is &lt;a href=&quot;https://github.com/simonvh/genomepy&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;genomepy&lt;/code&gt;&lt;/a&gt; which is less popular. Following is my experience with both.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1. &lt;code&gt;pyensembl&lt;/code&gt;:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;pyensembl&lt;/code&gt; is a fast and powerful API to mine ENSEMBL genome databases. &#xA;I have been using it for a while and in my experience it works very well. The name of the package may make you believe that it is developed/maintained by ENSEMBL. That is not true. Still, being an open source package with a realistic potential to be the default API for mining ENSEMBL genome databases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there are some issues related to it's development that users may find disconcerting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, documentation. I could only find a comprehensive api &lt;a href=&quot;https://media.readthedocs.org/pdf/pyensembl/latest/pyensembl.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation for release 0.8.10&lt;/a&gt;. Current release is 1.1. Not even &lt;a href=&quot;https://github.com/openvax/pyensembl/releases&quot; rel=&quot;nofollow noreferrer&quot;&gt;release notes are available&lt;/a&gt;. Therefore users have to rely on few lines of documentation that is available in the &lt;a href=&quot;https://github.com/openvax/pyensembl/blob/master/README.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;README.md&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, as far as I know, it is not peer reviewed even though it is an excellent analysis resource.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. &lt;code&gt;genomepy&lt;/code&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;genomepy&lt;/code&gt; loads all of the genome files into the RAM, making it immediately unusable, especially if you are working with larger genome such as human genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is: Is there any other better python API for working with ENSEMBL genomes?&lt;/p&gt;&#xA;" OwnerUserId="274" LastEditorUserId="274" LastEditDate="2018-07-31T15:04:13.430" LastActivityDate="2019-07-06T15:57:18.027" Title="Python API for working with ENSEMBL genomes" Tags="|python|genome|sequence-analysis|ensembl|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4828" PostTypeId="1" AcceptedAnswerId="4832" CreationDate="2018-08-02T00:31:58.030" Score="2" ViewCount="328" Body="&lt;p&gt;I have mapped some reads to a reference and want to change SNPs in the reference to SNPs found in the reads, even at low coverage. For example, here is a screenshot of one reference in IGV that I would like to correct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zlUhX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zlUhX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I use pilon to try to correct the reference, however, no bases are ever changed, even if using the option &lt;code&gt;--mindepth 1&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the call that I am using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -Xmx100G -jar {params.pilon}\&#xA;  --genome {input.ref}\&#xA;  --frags {input.bam}\&#xA;  --output {params.outname}\&#xA;  --changes --fix all,amb,breaks,bases\&#xA;  --threads {threads}\&#xA;  --mindepth 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I use even low-coverage information to correct my reference?&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="1769" LastEditDate="2018-08-02T16:48:32.190" LastActivityDate="2018-08-02T17:26:42.623" Title="How to force pilon to correct a reference with low coverage mapped reads?" Tags="|sam|reference-genome|read-correction|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4829" PostTypeId="1" AcceptedAnswerId="4830" CreationDate="2018-08-02T14:05:20.477" Score="2" ViewCount="238" Body="&lt;p&gt;The RNA-Seq fusion detection program &lt;a href=&quot;https://github.com/suhrig/arriba&quot; rel=&quot;nofollow noreferrer&quot;&gt;Arriba&lt;/a&gt; features the ability to white-list known gene fusions, specifically the authors recommend using &lt;a href=&quot;https://cancer.sanger.ac.uk/cosmic/download?genome=37&quot; rel=&quot;nofollow noreferrer&quot;&gt;COSMIC&lt;/a&gt; as a source of these &lt;a href=&quot;http://arriba.readthedocs.io/en/latest/input-files/#known-fusions&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Specifically they state:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A comprehensive list of known fusions can be obtained from CancerGeneCensus in the section titled &quot;Complete Fusion Export&quot;. &lt;em&gt;Depending on the gene annotation that is used to run arriba, some gene names need to be adjusted&lt;/em&gt;. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Emphasis is mine to highlight the issue here!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The state the format of the input file they require to be:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The file has two columns separated by a tab. Each line lists a pair of&#xA;  genes. The order of the genes is irrelevant. arriba searches for both&#xA;  genes as the 5' end and the 3' end of a fusion. Lines starting with a&#xA;  hash (#) are treated as comments. Optionally, the file can be&#xA;  gzip-compressed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;However the format of the download from COSMIC is somewhat different:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sample ID       Sample name     Primary site    Site subtype 1  Site subtype 2  Site subtype 3  Primary histology       Histology subtype 1     Histology subtype 2     Histology subtype 3     Fusion ID       Translocation Name      Fusion Type     PUBMED_PMID     ID_STUDY&#xA;WWWWWWW WWWWWWW thyroid NS      NS      NS      carcinoma       papillary_carcinoma     follicular_variant      NS                              16784981&#xA;WWWWWWW WWWWWWW thyroid NS      NS      NS      carcinoma       papillary_carcinoma     follicular_variant      NS      1271    CCDC6{ENST00000263102}:r.1_535_RET{ENST00000355710}:r.2369_5659 Observed mRNA   16784981&#xA;WWWWWWW WWWWWWW thyroid NS      NS      NS      carcinoma       papillary_carcinoma     follicular_variant      NS      1272    CCDC6{ENST00000263102}:r.1_535_RET{ENST00000355710}:r.2369_5659 Inferred Breakpoint     16784981&#xA;XXXXXXX XXXXXXX thyroid NS      NS      NS      carcinoma       papillary_carcinoma     follicular_variant      NS                              16784981&#xA;YYYYYYY YYYYYYY thyroid NS      NS      NS      carcinoma       papillary_carcinoma     follicular_variant      NS                              16728576&#xA;YYYYYYY YYYYYYY thyroid NS      NS      NS      carcinoma       papillary_carcinoma     follicular_variant      NS      1216    PAX8{NM_003466.2}:r.?_PPARG{NM_015869.2}:r.?    Inferred Breakpoint     16728576&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note: numeric samples IDs have been deliberately replaced with letters here.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this file, as is, will cause arriba to spurt out many warnings that the gene names in the first two columns are not recognised:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;WARNING: unknown gene in known fusions list: XXXXXXX&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the looks of the rest of my arriba output the currency of gene IDs its using would appear to be gene symbols taken from the &lt;code&gt;gencode.gft&lt;/code&gt; I'm supplying it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm thus assuming the format arriba requires for a known gene list to be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SYMB1   SYMB2&#xA;SYMB3   SYMB4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where the record separator is a tab character &lt;code&gt;\t&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how can I reprocess the COSMIC &quot;Complete Fusion Export&quot; &lt;a href=&quot;https://cancer.sanger.ac.uk/cosmic/download?genome=37&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;.tsv&lt;/code&gt; file&lt;/a&gt; to the format required for arriba to act as a white-list?&lt;/p&gt;&#xA;" OwnerUserId="1769" LastEditorUserId="1769" LastEditDate="2018-08-02T14:12:15.327" LastActivityDate="2018-08-02T14:12:15.327" Title="Generating list of known fusions for Arriba from COSMIC" Tags="|rna-seq|fusions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4830" PostTypeId="2" ParentId="4829" CreationDate="2018-08-02T14:05:20.477" Score="1" Body="&lt;p&gt;I managed to solve this with a one-liner:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -v 'Sample ID' CosmicFusionExport.tsv | cut -f 12 | grep -v -e '^$' | perl -lne '/^(\S+?)\{\S+\d+}\S+_(\S+?)\{/; print &quot;$1\t$2&quot;' | sort -u &amp;gt; CosmicFusionList.tsv &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The various parts of the piped command are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Remove header:&lt;code&gt;grep -v 'Sample ID' CosmicFusionExport.tsv |&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Extract 12th field (Translocation Name):&lt;code&gt;cut -f 12 |&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Remove blank entries:&lt;code&gt;grep -v -e '^$' |&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Extract gene symbols from 12th column:&lt;code&gt;perl -lne '/^(\S+?)\{\S+\d+}\S+_(\S+?)\{/; print &quot;$1\t$2&quot;' |&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Remove duplicate lines:&lt;code&gt;sort -u &amp;gt; CosmicFusionList.tsv&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Optionally the final two stages could be:&lt;code&gt;sort -u | gzip -c &amp;gt; CosmicFusionList.tsv.gz&lt;/code&gt; to make use of file compression (for what it's worth on a 3.9 kB file)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Since the authors of arriba have neglected to document this necessary data munging step, I've posted it here in Q and A format in the hope that it will be useful to others, and other solutions can be proposed.&lt;/p&gt;&#xA;" OwnerUserId="1769" LastActivityDate="2018-08-02T14:05:20.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4832" PostTypeId="2" ParentId="4828" CreationDate="2018-08-02T17:26:42.623" Score="1" Body="&lt;p&gt;Pilon will not consider fragments that are not proper pairs if the &lt;code&gt;--frags&lt;/code&gt; option is used to pass the alignment &lt;code&gt;.bam&lt;/code&gt; file. The reads mapped in the above IGV screenshot are unpaired.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, if you want Pilon to consider &lt;em&gt;all&lt;/em&gt; reads that map as legitimate and usable for correction, pass the &lt;code&gt;.bam&lt;/code&gt; file with the &lt;code&gt;--unpaired&lt;/code&gt; option instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To above corrected example would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -Xmx100G -jar {params.pilon}\&#xA;  --genome {input.ref}\&#xA;  --unpaired {input.bam}\&#xA;  --output {params.outname}\&#xA;  --changes --fix all,amb,breaks,bases\&#xA;  --threads {threads}\&#xA;  --mindepth 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-08-02T17:26:42.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4833" PostTypeId="1" AcceptedAnswerId="4835" CreationDate="2018-08-02T17:36:46.897" Score="7" ViewCount="758" Body="&lt;p&gt;I'm trying to learn to use cython to speed up some code based on pysam. My issue is not strictly speaking about bioinformatics, but rather about building tools using a bioinformatics library. I hope this is still relevant in this site (I hesitated posting it as a github issue to pysam, alternatively, but I was not sure that this was a problem with pysam).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my project's folder, the &lt;code&gt;bam25prime/libbam25prime.pyx&lt;/code&gt; file has the following content:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# cython: language_level=3&#xA;from pysam.libchtslib cimport BAM_FREVERSE&#xA;from pysam.libcalignedsegment cimport AlignedSegment&#xA;&#xA;cdef object ccollapse_ali(AlignedSegment ali):&#xA;    &quot;&quot;&quot;&#xA;    Return the bed information corresponding to the 5' collapse&#xA;    of AlignedSegment *ali*.&#xA;    &quot;&quot;&quot;&#xA;    if (ali.flag &amp;amp; BAM_FREVERSE) != 0:&#xA;        return &quot;\t&quot;.join([&#xA;            ali.reference_name,&#xA;            # five prime position&#xA;            str(ali.reference_end - 1),&#xA;            str(ali.reference_end),&#xA;            &quot;.&quot;,&#xA;            &quot;.&quot;,&#xA;            &quot;-&quot;])&#xA;    else:&#xA;        return &quot;\t&quot;.join([&#xA;            ali.reference_name,&#xA;            # five prime position&#xA;            str(ali.reference_start),&#xA;            str(ali.reference_start + 1),&#xA;            &quot;.&quot;,&#xA;            &quot;.&quot;,&#xA;            &quot;+&quot;])&#xA;&#xA;&#xA;def collapse_ali(ali):&#xA;    &quot;&quot;&quot;&#xA;    Return the bed information corresponding to the 5' collapse&#xA;    of AlignedSegment *ali*.&#xA;    &quot;&quot;&quot;&#xA;    return ccollapse_ali(ali)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In my &lt;code&gt;setup.py&lt;/code&gt;, the &lt;code&gt;setup&lt;/code&gt; function is called with this argument:&#xA;&lt;code&gt;ext_modules = cythonize(&quot;bam25prime/libbam25prime.pyx&quot;)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I try to do the build (running &lt;code&gt;python3.6 setup.py build_ext&lt;/code&gt;), cython seems to miss some include paths, because I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[1/1] Cythonizing bam25prime/libbam25prime.pyx&#xA;running build_ext&#xA;building 'bam25prime.libbam25prime' extension&#xA;gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -O3 -march=native -fomit-frame-pointer -fPIC -I/home/bli/.local/lib/python3.6/site-packages/pysam -I${HOME}/include -I/home/bli/include/python3.6m -I/home/bli/.local/include -c bam25prime/libbam25prime.c -o build/temp.linux-x86_64-3.6/bam25prime/libbam25prime.o&#xA;bam25prime/libbam25prime.c:497:28: fatal error: htslib/kstring.h: No such file or directory&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I manually re-run the gcc command, adding &lt;code&gt;-I/home/bli/.local/lib/python3.6/site-packages/pysam/include/htslib&lt;/code&gt;, the header file gets found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How do I get cython to correctly determine the path to the necessary header files?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess another way to formulate the question would be the following: &lt;strong&gt;How is cython supposed to determine what include flags to use?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or maybe there is a problem with my pysam installation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I remember well, I installed it using &lt;code&gt;pip3.6 --user install pysam&lt;/code&gt;. I have a &lt;code&gt;~/.pydistutils.cfg&lt;/code&gt; file containing the following settings&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[install]&#xA;optimize=1&#xA;[build_ext]&#xA;include_dirs=${HOME}/include&#xA;library_dirs=${HOME}/lib&#xA;rpath=${HOME}/lib&#xA;user=1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any advice appreciated.&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2018-08-03T09:26:49.367" LastActivityDate="2018-08-09T12:30:29.140" Title="Using pysam with cython: htslib/kstring.h not found" Tags="|python|pysam|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4834" PostTypeId="1" CreationDate="2018-08-02T19:48:31.683" Score="2" ViewCount="198" Body="&lt;p&gt;I am trying to use a tool called RATT described in this &lt;a href=&quot;https://academic.oup.com/nar/article/39/9/e57/1236534&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; and available &lt;a href=&quot;http://ratt.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I ran the &lt;a href=&quot;http://ratt.sourceforge.net/example.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt; using the following command:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;$RATT_HOME/start.ratt.sh embl F11.fasta F11 Strain&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;embl&lt;/code&gt; is a folder containing the file &lt;code&gt;Tb_H37Rv.embl&lt;/code&gt;, which is an annotation file in EMBL format of the &lt;em&gt;Mycobacterium tuberculosis&lt;/em&gt; genome. F11.fasta contains the genome in FASTA sequence of the strain F11 of &lt;em&gt;Mycobacterium tuberculosis&lt;/em&gt;. This example provided by the RATT developers runs smoothly:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Of the reference chromosome Tb_H37Rv    2.66 per cent   has no synteny with the query&#xA;Of the query chromosome gi_148821191_ref_NC_009565.1_    3.03 per cent  has no synteny with the reference&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Please see the file F11.gi_148821191_ref_NC_009565.1_.Report.txt for reporting the errors and the file F11.gi_148821191_ref_NC_009565.1_.Report.gff for reporting the error in Artemis or other sequence viewer.&#xA;************************ &#xA;If you want to start artemis on this replicon:&#xA;art F11.gi_148821191_ref_NC_009565.1_.final.embl + F11.gi_148821191_ref_NC_009565.1_.Report.gff  + Query/F11.gi_148821191_ref_NC_009565.1_.Mutations.gff&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, I tried it on the genome of &lt;em&gt;E. coli&lt;/em&gt; and it worked:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$RATT_HOME/start.ratt.sh embl bact1.fa bact1 Species&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I used the embl file from &lt;em&gt;Mycobacterium&lt;/em&gt; so not much annotation was transferred due to the large genetic differences between &lt;em&gt;Mycobacterium&lt;/em&gt; and &lt;em&gt;E. coli&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, I tried to transfer the annotation from the reference genome of &lt;em&gt;C. elegans&lt;/em&gt; to a new assembly I have generated. The way I generated the embl annotation file is by running &lt;code&gt;seqret&lt;/code&gt; on the fasta file of &lt;em&gt;C. elegans&lt;/em&gt; and its corresponding annotation file found in GTF format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seqret -sequence c_elegans.PRJNA13758.WS263.genomic.fa -feature -fformat gff -fopenfile c_elegans.PRJNA13758.WS263.annotations.gff3 -osformat embl -auto -outseq c_elegans.PRJNA13758.WS263.embl&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I got the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;RATT_HOME/start.ratt.sh embl scaffolds.fa scaffolds Strain&#xA;&#xA;ERROR: Reference input does not match delta file&#xA;Assertion failed: (Len - Coord + 1 &amp;gt; 0), function revC, file show-coords.cc, line 1118.&#xA;/Users/user/Documents/project/output/software/ratt-code/start.ratt.sh: line 73: 85325 Abort trap: 6           show-coords -clHT $name.delta &amp;gt; $name.coords&#xA;Assertion failed: (Len - Coord + 1 &amp;gt; 0), function revC, file show-coords.cc, line 1118.&#xA;/Users/user/Documents/project/output/software/ratt-code/start.ratt.sh: line 73: 85326 Abort trap: 6           show-coords -clHT $name.filter.delta &amp;gt; $name.filter.coords&#xA;Use of uninitialized value $stats in print at /Users/user/Documents/project/output/software/ratt-code/main.ratt.pl line 82.&#xA;Nucmer is done. Now transfer the annotation.&#xA;perl /Users/user/Documents/project/output/software/ratt-code/main.ratt.pl embl nucmer.scaffolds.snp nucmer.scaffolds.filter.coords scaffolds&#xA;working on c_elegans.PRJNA13758.WS263&#xA;Use of uninitialized value in split at /Users/user/Documents/project/output/software/ratt-code/main.ratt.pl line 1140.&#xA;Can't use an undefined value as an ARRAY reference at /Users/user/Documents/project/output/software/ratt-code/main.ratt.pl line 229.&#xA;6 Sequences where generated&#xA;Done.&#xA;Nucmer is done. Now Correct the annotation for chromosome .&#xA;work on I&#xA;If you want to start artemis on this replicon:&#xA;art scaffolds.I.final.embl + scaffolds.I.Report.gff  + Query/scaffolds.I.Mutations.gff&#xA;work on II&#xA;If you want to start artemis on this replicon:&#xA;art scaffolds.II.final.embl + scaffolds.II.Report.gff  + Query/scaffolds.II.Mutations.gff&#xA;work on III&#xA;If you want to start artemis on this replicon:&#xA;art scaffolds.III.final.embl + scaffolds.III.Report.gff  + Query/scaffolds.III.Mutations.gff&#xA;work on IV&#xA;If you want to start artemis on this replicon:&#xA;art scaffolds.IV.final.embl + scaffolds.IV.Report.gff  + Query/scaffolds.IV.Mutations.gff&#xA;work on V&#xA;If you want to start artemis on this replicon:&#xA;art scaffolds.V.final.embl + scaffolds.V.Report.gff  + Query/scaffolds.V.Mutations.gff&#xA;work on X&#xA;If you want to start artemis on this replicon:&#xA;art scaffolds.X.final.embl + scaffolds.X.Report.gff  + Query/scaffolds.X.Mutations.gff&#xA;rm: scaffolds*embl.tmp.BBA.embl: No such file or directory&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I downloaded the FASTA file &lt;a href=&quot;ftp://ftp.wormbase.org/pub/wormbase/releases/WS263/species/c_elegans/PRJNA13758/c_elegans.PRJNA13758.WS263.genomic.fa.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and the GTF file &lt;a href=&quot;ftp://ftp.wormbase.org/pub/wormbase/releases/WS263/species/c_elegans/PRJNA13758/c_elegans.PRJNA13758.WS263.annotations.gff3.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working on Mac OSX.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Update 6th August 2018&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I have also tried an alternative way of getting a &lt;em&gt;C. elegans&lt;/em&gt; annotation in EMBL format. I downloaded it from the Ensembl FTP website:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;wget -m &lt;a href=&quot;ftp://ftp.ensembl.org/pub/release-91/embl/caenorhabditis_elegans/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ensembl.org/pub/release-91/embl/caenorhabditis_elegans/&lt;/a&gt; --directory-prefix embl/&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, I cat-ed the .dat files (one for each chromosome) into one &lt;code&gt;ensembl.embl&lt;/code&gt; file and used that for the RATT command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$RATT_HOME/start.ratt.sh embl scaffolds.fa scaffolds Strain&#xA;&#xA;ERROR: Could not parse delta file, nucmer.scaffolds.delta&#xA;error no: 402&#xA;ERROR: Could not parse delta file, nucmer.scaffolds.filter.delta&#xA;error no: 402&#xA;ERROR: Could not parse delta file, nucmer.scaffolds.delta&#xA;error no: 402&#xA;ERROR: Could not parse delta file, nucmer.scaffolds.filter.delta&#xA;error no: 402&#xA;Use of uninitialized value $stats in print at /Users/user/Documents/project/output/software/ratt-code/main.ratt.pl line 82.&#xA;Nucmer is done. Now transfer the annotation.&#xA;perl /Users/user/Documents/project/output/software/ratt-code/main.ratt.pl embl nucmer.scaffolds.snp nucmer.scaffolds.filter.coords scaffolds&#xA;working on ensembl&#xA;Use of uninitialized value in split at /Users/user/Documents/project/output/software/ratt-code/main.ratt.pl line 1140.&#xA;Can't use an undefined value as an ARRAY reference at /Users/user/Documents/project/output/software/ratt-code/main.ratt.pl line 229.&#xA;6 Sequences where generated&#xA;Done.&#xA;Nucmer is done. Now Correct the annotation for chromosome .&#xA;work on I&#xA;If you want to start artemis on this replicon:&#xA;art scaffolds.I.final.embl + scaffolds.I.Report.gff  + Query/scaffolds.I.Mutations.gff&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;work on X&#xA;If you want to start artemis on this replicon:&#xA;art scaffolds.X.final.embl + scaffolds.X.Report.gff  + Query/scaffolds.X.Mutations.gff&#xA;rm: scaffolds*embl.tmp.BBA.embl: No such file or directory&#xA;&#xA;Indeed, the nucmer delta file is empty. If that is the only problem, maybe I can run nucmer separately but then how can I run RATT without running nucmer?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The developer of RATT told me to not concatenate the EMBL files because there is no need to do so. So, here is the output of RATT on the .embl files, one for each chromosome. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$RATT_HOME/start.ratt.sh embl scaffolds.fa scaffolds Strain  &#xA;&#xA;Of the reference chromosome Caenorhabditis_elegans.WBcel235.91.chromosome.I 13.76 per cent  has no synteny with the query&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Of the query chromosome X    15.29 per cent has no synteny with the reference&#xA;Nucmer is done. Now transfer the annotation.&#xA;perl /Users/user/Documents/project/output/software/ratt-code/main.ratt.pl embl nucmer.scaffolds.snp nucmer.scaffolds.filter.coords scaffolds&#xA;working on Caenorhabditis_elegans.WBcel235.91.chromosome.I&#xA;Argument &quot;11123466)&quot; isn't numeric in sort at /Users/user/Documents/project/output/software/ratt-code/main.ratt.pl line 985, &amp;lt;F&amp;gt; line 2547.&#xA;Argument &quot;11123466)&quot; isn't numeric in sort at /Users/user/Documents/project/output/software/ratt-code/main.ratt.pl line 985, &amp;lt;F&amp;gt; line 2555.&#xA;Argument &quot;5568506)&quot; isn't numeric in sort at /Users/user/Documents/project/output/software/ratt-code/main.ratt.pl line 985, &amp;lt;F&amp;gt; line 2627.&#xA;1474993 1480045 2526 1477519 &#xA;14358883 14373963 7540 14366423 &#xA;3204189 3216229 6020 3210209 &#xA;3809476 3812106 1315 3810791 &#xA;10232464 10233260 398 10232862 &#xA;10491944 10506864 7460 10499404 &#xA;11020347 11023081 1367 11021714 &#xA;318712 361504 21396 340108 &#xA;working on Caenorhabditis_elegans.WBcel235.91.chromosome.MtDNA&#xA;Can't use an undefined value as an ARRAY reference at /Users/user/Documents/project/output/software/ratt-code/main.ratt.pl line 229.&#xA;6 Sequences where generated&#xA;Done.&#xA;Nucmer is done. Now Correct the annotation for chromosome .&#xA;work on I&#xA;If you want to start artemis on this replicon:&#xA;art scaffolds.I.final.embl + scaffolds.I.Report.gff  + Query/scaffolds.I.Mutations.gff&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;work on X&#xA;If you want to start artemis on this replicon:&#xA;art scaffolds.X.final.embl + scaffolds.X.Report.gff  + Query/scaffolds.X.Mutations.gff&#xA;rm: scaffolds*embl.tmp.BBA.embl: No such file or directory&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is better than before because I get a reasonable delta file from nucmer. But the error &lt;code&gt;Argument &quot;11123466)&quot; isn't numeric&lt;/code&gt; is a problem. It seems that is should be easy enough to solve this. It looks to me like the coordinates from the EMBL file are not parsed properly and the code stumbles on the coordiate followed by a &lt;code&gt;)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="96" LastEditDate="2019-02-22T20:16:08.660" LastActivityDate="2019-02-22T20:16:08.660" Title="RATT works on example bacterial sequence and other bacterial genome but not on C. elegans genome and annotation" Tags="|sequence-annotation|embl|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4835" PostTypeId="2" ParentId="4833" CreationDate="2018-08-03T09:25:52.117" Score="6" Body="&lt;p&gt;Based on a suggestion by Devon Ryan, I searched how to set &lt;code&gt;C_INCLUDE_PATH for&lt;/code&gt; cython, and found &lt;a href=&quot;https://github.com/cython/cython/issues/1480&quot; rel=&quot;nofollow noreferrer&quot;&gt;this issue&lt;/a&gt;. It refers to a (currently not working) feature present in cython's documentation, that should let libraries inform cython of the correct localisation for the header files in the module's &lt;code&gt;setup&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ext_modules = cythonize(&#xA;    &quot;bam25prime/libbam25prime.pyx&quot;,&#xA;    include_path=pysam_get_include())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As noted in the issue discussion, this seems to have no effect, but a workaround exists, consisting in using the &lt;code&gt;Extension&lt;/code&gt; class to store the header localization information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the general answer is: &lt;strong&gt;Some python libraries (including pysam) provide a function that returns the paths to their include directories. This can be used to inform cython, but has to be done &quot;manually&quot;.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My &lt;code&gt;setup.py&lt;/code&gt; now looks as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from setuptools import setup, find_packages&#xA;# If you have .pyx things to cythonize&#xA;from distutils.extension import Extension&#xA;from Cython.Build import cythonize&#xA;from pysam import get_include as pysam_get_include&#xA;&#xA;name = &quot;bam25prime&quot;&#xA;__version__ = &quot;0.1&quot;&#xA;&#xA;# https://github.com/cython/cython/blob/master/docs/src/reference/compilation.rst#configuring-the-c-build&#xA;extensions = [&#xA;    Extension(&#xA;        &quot;bam25prime.libbam25prime&quot;, [&quot;bam25prime/libbam25prime.pyx&quot;],&#xA;        include_dirs=pysam_get_include()),&#xA;    ]&#xA;&#xA;setup(&#xA;    name=name,&#xA;    version=__version__,&#xA;    # [description, author, email, licence, ...]&#xA;    packages=find_packages(),&#xA;    scripts=[&quot;bin/%s&quot; % name],&#xA;    ext_modules = cythonize(extensions),&#xA;    # ext_modules = cythonize(&#xA;    #     &quot;bam25prime/libbam25prime.pyx&quot;,&#xA;    #     include_path=pysam_get_include()),&#xA;    install_requires=[&quot;pysam&quot;, &quot;pybedtools&quot;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, &lt;code&gt;python3.6 setup.py build_ext&lt;/code&gt; succeeds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And for whoever might be interested, here are the other files of the project:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;bin/bam25prime&lt;/code&gt; is a link to &lt;code&gt;bam25prime/bam25prime.py&lt;/code&gt;, which looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;# vim: set fileencoding=&amp;lt;utf-8&amp;gt; :&#xA;&quot;&quot;&quot;&#xA;When executed, this module collapses reads in a bam file to their 5-prime end&#xA;(reducing them to length 1) and writes the result in bed format.&#xA;&quot;&quot;&quot;&#xA;&#xA;import argparse&#xA;from pybedtools import BedTool, Interval&#xA;from pysam import AlignmentFile&#xA;from .libbam25prime import collapse_ali&#xA;&#xA;# pybedtools.Interval | pysam.AlignedSegment&#xA;# chrom               | reference_name&#xA;# start               | reference_start&#xA;# end                 | reference_end&#xA;# bool(int(strand))   | is_reverse&#xA;#&#xA;# Interval(reference_name, reference_start, reference_end, strand=strand)    &#xA;&#xA;&#xA;def collapse_and_sort(alis):&#xA;    &quot;&quot;&quot;&#xA;    Collapse elements from *ali* to their 5 prime end.&#xA;&#xA;    The resulting :class:`pybedtool.BedTool` is sorted.&#xA;&#xA;    :param alis: An iterable of :class:`pysam.AlignedSegment`&#xA;    :rtype: :class:`pybedtool.BedTool`&#xA;&#xA;    :class:`pybedtool.BedTool` is an iterable:&#xA;    It can be iterated over in a for loop, or produce an iterator&#xA;    using :func:`iter` on it&#xA;    (see https://stackoverflow.com/a/28353158/1878788).&#xA;    &quot;&quot;&quot;&#xA;    return BedTool(&#xA;        &quot;\n&quot;.join(map(collapse_ali, alis)),&#xA;        from_string=True).sort(stream=True)&#xA;&#xA;&#xA;def main():&#xA;    &quot;&quot;&quot;Main function of the program.&quot;&quot;&quot;&#xA;    parser = argparse.ArgumentParser(&#xA;        description=__doc__,&#xA;        formatter_class=argparse.ArgumentDefaultsHelpFormatter)&#xA;    parser.add_argument(&#xA;        &quot;-b&quot;, &quot;--bamfile&quot;,&#xA;        required=True,&#xA;        help=&quot;Sorted and indexed bam file containing the aligned reads.&quot;)&#xA;    parser.add_argument(&#xA;        &quot;-o&quot;, &quot;--out_bed&quot;,&#xA;        required=True,&#xA;        help=&quot;Bed file in which the results should be written.&quot;)&#xA;    args = parser.parse_args()&#xA;    with AlignmentFile(args.bamfile, &quot;rb&quot;) as bamfile:&#xA;        collapse_and_sort(bamfile.fetch()).saveas(args.out_bed)&#xA;    return 0&#xA;&#xA;&#xA;if __name__ == &quot;__main__&quot;:&#xA;    exit(main())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;bam25prime/libbam25prime.pyx&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;# vim: set fileencoding=&amp;lt;utf-8&amp;gt; :&#xA;# cython: language_level=3&#xA;&quot;&quot;&quot;&#xA;This library contains a cythonized version of a function to collapse aligned&#xA;segments to their 5' end.&#xA;&quot;&quot;&quot;&#xA;&#xA;from pysam.libchtslib cimport BAM_FREVERSE&#xA;from pysam.libcalignedsegment cimport AlignedSegment&#xA;&#xA;&#xA;cdef object ccollapse_ali(AlignedSegment ali):&#xA;    &quot;&quot;&quot;&#xA;    Return the bed information corresponding to the 5' collapse&#xA;    of AlignedSegment *ali*.&#xA;    &quot;&quot;&quot;&#xA;    if (ali.flag &amp;amp; BAM_FREVERSE) != 0:&#xA;        return &quot;\t&quot;.join([&#xA;            ali.reference_name,&#xA;            # five prime position&#xA;            str(ali.reference_end - 1),&#xA;            str(ali.reference_end),&#xA;            &quot;.&quot;,&#xA;            &quot;.&quot;,&#xA;            &quot;-&quot;])&#xA;    else:&#xA;        return &quot;\t&quot;.join([&#xA;            ali.reference_name,&#xA;            # five prime position&#xA;            str(ali.reference_start),&#xA;            str(ali.reference_start + 1),&#xA;            &quot;.&quot;,&#xA;            &quot;.&quot;,&#xA;            &quot;+&quot;])&#xA;&#xA;&#xA;def collapse_ali(ali):&#xA;    &quot;&quot;&quot;&#xA;    Return the bed information corresponding to the 5' collapse&#xA;    of AlignedSegment *ali*.&#xA;    &quot;&quot;&quot;&#xA;    return ccollapse_ali(ali)&#xA;&#xA;&#xA;def main():&#xA;    &quot;&quot;&quot;Example use case.&quot;&quot;&quot;&#xA;    print(&quot;No examples to show here.&quot;)&#xA;    return 0&#xA;&#xA;&#xA;if __name__ == &quot;__main__&quot;:&#xA;    import sys&#xA;    sys.exit(main())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Any advice or suggestions welcome: I'd like to improve my python package-making skills.)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For those interested, to use cython with &lt;code&gt;pybedtools&lt;/code&gt; I had to define the following &lt;code&gt;Extension&lt;/code&gt; (where &lt;code&gt;glob&lt;/code&gt; is &lt;code&gt;glob.glob&lt;/code&gt; and &lt;code&gt;/home/bli/src/bedtools2-2.27.1/&lt;/code&gt; is the location of the source code of bedtools).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    Extension(&#xA;        &quot;bam25prime.libcollapsebed&quot;, [&quot;bam25prime/libcollapsebed.pyx&quot;],&#xA;        include_dirs=glob(&quot;/home/bli/src/bedtools2-2.27.1/src/utils/*&quot;),&#xA;        language=&quot;c++&quot;),&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2018-08-09T12:30:29.140" LastActivityDate="2018-08-09T12:30:29.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4836" PostTypeId="1" CreationDate="2018-08-03T11:35:05.127" Score="11" ViewCount="331" Body="&lt;p&gt;I’m trying to create a CRAM file that stores its path to the FASTA reference as a relative path, rather than an absolute path, so that I can move the files around. Unfortunately I can’t get this to work; I was expecting the following to work:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;⟩⟩⟩ samtools view -C -T ../reference/ref.fa -o output.cram input.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, the resulting file contains an absolute path in its header:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;⟩⟩⟩ samtools view -H output.cram&#xA;…&#xA;@SQ     SN:1    LN:249250621    M5:hash     UR:/absolute/path/to/data/mapped/../reference/ref.fa&#xA;…&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As a result, I am unable to open the file via a different path mount that results in different absolute paths, and I can’t move the file (+ its reference) around, or to different machines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that I could set the &lt;code&gt;REF_PATH&lt;/code&gt; environment variable or specify &lt;code&gt;-T&lt;/code&gt; when reading the file but I would like to avoid this (the result file needs to be readable by IGV, launched by users who don’t know how to set environment variables).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way of creating a CRAM file that stores a &lt;em&gt;relative path&lt;/em&gt; to its reference?&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2018-10-22T23:52:14.433" Title="Can I create a CRAM file with a relative reference path?" Tags="|sam|samtools|cram|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4837" PostTypeId="1" AcceptedAnswerId="4838" CreationDate="2018-08-03T12:49:14.480" Score="2" ViewCount="1048" Body="&lt;p&gt;I have a file with following information:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    HAVANA  gene    135141  135895  .       -       .       gene_id &quot;ENSG00000268903.1&quot;; gene_type &quot;processed_pseudogene&quot;; gene_name &quot;AL627309.7&quot;; level 2; havana_gene &quot;OTTHUMG00000182518.2&quot;;&#xA;chr1    HAVANA  gene    137682  137965  .       -       .       gene_id &quot;ENSG00000269981.1&quot;; gene_type &quot;processed_pseudogene&quot;; gene_name &quot;AL627309.8&quot;; level 2; havana_gene &quot;OTTHUMG00000182738.2&quot;;&#xA;chr1    HAVANA  gene    139790  140339  .       -       .       gene_id &quot;ENSG00000239906.1&quot;; gene_type &quot;antisense_RNA&quot;; gene_name &quot;AL627309.2&quot;; level 2; havana_gene &quot;OTTHUMG00000002481.1&quot;;&#xA;chr1    HAVANA  gene    141474  173862  .       -       .       gene_id &quot;ENSG00000241860.6&quot;; gene_type &quot;processed_transcript&quot;; gene_name &quot;AL627309.5&quot;; level 2; tag &quot;ncRNA_host&quot;; havana_gene &quot;OTTHUMG00000002480.3&quot;;&#xA;chr1    ENSEMBL gene    157784  157887  .       -       .       gene_id &quot;ENSG00000222623.1&quot;; gene_type &quot;snRNA&quot;; gene_name &quot;RNU6-1100P&quot;; level 3;&#xA;chr1    HAVANA  gene    160446  161525  .       +       .       gene_id &quot;ENSG00000241599.1&quot;; gene_type &quot;lincRNA&quot;; gene_name &quot;AL627309.4&quot;; level 2; havana_gene &quot;OTTHUMG00000002525.1&quot;;&#xA;chr1    HAVANA  gene    182696  184174  .       +       .       gene_id &quot;ENSG00000279928.2&quot;; gene_type &quot;unprocessed_pseudogene&quot;; gene_name &quot;FO538757.2&quot;; level 2; havana_gene &quot;OTTHUMG00000191962.1&quot;;&#xA;chr1    HAVANA  gene    185217  195411  .       -       .       gene_id &quot;ENSG00000279457.4&quot;; gene_type &quot;unprocessed_pseudogene&quot;; gene_name &quot;FO538757.1&quot;; level 2; havana_gene &quot;OTTHUMG00000191963.1&quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the above file there is a &lt;code&gt;gene_type&lt;/code&gt; with different names like processed_pseudogene, lincRNA, snRNA, unprocessed_pseudogene, antisense_RNA, processed_transcript.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to create a new file based on &lt;code&gt;gene_type&lt;/code&gt;. Among all the gene_types I want the new file to be with certain names of &lt;code&gt;gene_type&lt;/code&gt; like only processed_pseudogene, lincRNA, unprocessed_pseudogene, antisense_RNA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The new file should look like below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    HAVANA  gene    135141  135895  .       -       .       gene_id &quot;ENSG00000268903.1&quot;; gene_type &quot;processed_pseudogene&quot;; gene_name &quot;AL627309.7&quot;; level 2; havana_gene &quot;OTTHUMG00000182518.2&quot;;&#xA;chr1    HAVANA  gene    137682  137965  .       -       .       gene_id &quot;ENSG00000269981.1&quot;; gene_type &quot;processed_pseudogene&quot;; gene_name &quot;AL627309.8&quot;; level 2; havana_gene &quot;OTTHUMG00000182738.2&quot;;&#xA;chr1    HAVANA  gene    139790  140339  .       -       .       gene_id &quot;ENSG00000239906.1&quot;; gene_type &quot;antisense_RNA&quot;; gene_name &quot;AL627309.2&quot;; level 2; havana_gene &quot;OTTHUMG00000002481.1&quot;;&#xA;chr1    HAVANA  gene    160446  161525  .       +       .       gene_id &quot;ENSG00000241599.1&quot;; gene_type &quot;lincRNA&quot;; gene_name &quot;AL627309.4&quot;; level 2; havana_gene &quot;OTTHUMG00000002525.1&quot;;&#xA;chr1    HAVANA  gene    182696  184174  .       +       .       gene_id &quot;ENSG00000279928.2&quot;; gene_type &quot;unprocessed_pseudogene&quot;; gene_name &quot;FO538757.2&quot;; level 2; havana_gene &quot;OTTHUMG00000191962.1&quot;;&#xA;chr1    HAVANA  gene    185217  195411  .       -       .       gene_id &quot;ENSG00000279457.4&quot;; gene_type &quot;unprocessed_pseudogene&quot;; gene_name &quot;FO538757.1&quot;; level 2; havana_gene &quot;OTTHUMG00000191963.1&quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1377" OwnerDisplayName="user3351523" LastEditorUserId="29" LastEditDate="2018-08-07T09:41:16.583" LastActivityDate="2018-08-07T09:41:16.583" Title="How do I filter a GFF file by gene type?" Tags="|linux|shell|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="4838" PostTypeId="2" ParentId="4837" CreationDate="2018-08-03T13:02:14.693" Score="4" Body="&lt;pre&gt;&lt;code&gt;grep -E 'gene_type &quot;(processed_pseudogene|lincRNA|unprocessed_pseudogene|antisense_RNA)&quot;;' input_file.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;Explanation&lt;/h1&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;grep -E&lt;/code&gt;: use extended regular expression so you don't need to escape things.&lt;/li&gt;&#xA;&lt;li&gt;This kind of expression &lt;code&gt;(foo|bar|baz)&lt;/code&gt; allows grep to include either of these strings. Hence, the total expression will match &lt;code&gt;gene_type &quot;processed_pseudogene&quot;;'&lt;/code&gt;, &lt;code&gt;gene_type &quot;lincRNA&quot;;'&lt;/code&gt;, etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerDisplayName="Sparhawk" LastActivityDate="2018-08-03T13:02:14.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4839" PostTypeId="1" AcceptedAnswerId="4840" CreationDate="2018-08-03T16:27:51.643" Score="-3" ViewCount="629" Body="&lt;p&gt;I am trying to install STEMNET R package but a permanent error stops me, what can I do please?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; &amp;gt; library(devtools)&#xA;&amp;gt; &amp;gt; devtools::install_git(&quot;https://git.embl.de/velten/STEMNET/&quot;, build_vignettes=TRUE) Downloading git repo&#xA;&amp;gt; https://git.embl.de/velten/STEMNET/ Installation failed: Error in&#xA;&amp;gt; 'git2r_clone': SSL error: syscall failure: Connection reset by peer&#xA;&amp;gt; &#xA;&amp;gt; &amp;gt; &#xA;&amp;gt; &amp;gt; devtools::install_git(&quot;https://git.embl.de/velten/STEMNET/&quot;, build_vignettes=FALSE) Downloading git repo&#xA;&amp;gt; https://git.embl.de/velten/STEMNET/ Installation failed: Error in&#xA;&amp;gt; 'git2r_clone': SSL error: syscall failure: Connection reset by peer&#xA;&amp;gt; &#xA;&amp;gt; &amp;gt; &#xA;&amp;gt; &amp;gt; library(git2r)&#xA;&amp;gt; &#xA;&amp;gt; Attaching package: ‘git2r’&#xA;&amp;gt; &#xA;&amp;gt; The following objects are masked from ‘package:Biobase’:&#xA;&amp;gt; &#xA;&amp;gt;     content, notes&#xA;&amp;gt; &#xA;&amp;gt; The following object is masked from ‘package:igraph’:&#xA;&amp;gt; &#xA;&amp;gt;     tree&#xA;&amp;gt; &#xA;&amp;gt; &amp;gt; &#xA;&amp;gt; &amp;gt; libgit2_features function ()  {&#xA;&amp;gt;     .Call(git2r_libgit2_features) } &amp;lt;bytecode: 0x1d276d918&amp;gt; &amp;lt;environment: namespace:git2r&amp;gt;&#xA;&amp;gt; &amp;gt; devtools::install_git(&quot;https://git.embl.de/velten/STEMNET/&quot;, build_vignettes=FALSE) Downloading git repo&#xA;&amp;gt; https://git.embl.de/velten/STEMNET/ Installation failed: Error in&#xA;&amp;gt; 'git2r_clone': SSL error: syscall failure: Connection reset by peer&#xA;&amp;gt; &#xA;&amp;gt; &amp;gt; &#xA;&amp;gt; &amp;gt; install_git(&quot;c97sr/idd&quot;) Downloading git repo c97sr/idd Installation failed: Error in 'git2r_clone': unsupported URL protocol&#xA;&amp;gt; &#xA;&amp;gt; &amp;gt; &#xA;&amp;gt; &amp;gt; install_git() Error in lapply(url, git_remote, subdir = subdir, branch = branch, credentials = credentials) :    argument &quot;url&quot; is&#xA;&amp;gt; missing, with no default&#xA;&amp;gt; &amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2018-08-03T17:59:05.130" Title="Installing a R package from git" Tags="|r|scrnaseq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4840" PostTypeId="2" ParentId="4839" CreationDate="2018-08-03T17:59:05.130" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;UPDATE 2018/04/22: Some users had problem with this command, a way to&#xA;  install that works for everyone is to download the tar.gz archive,&#xA;  unpack to a directory of your choice (e.g. /path/to) and inside R run&#xA;  devtools::install_local(&quot;/path/to/stemnet_dir&quot;, build_vignettes=T)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://git.embl.de/velten/STEMNET&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://git.embl.de/velten/STEMNET&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2018-08-03T17:59:05.130" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4842" PostTypeId="1" AcceptedAnswerId="4856" CreationDate="2018-08-05T01:24:06.943" Score="7" ViewCount="188" Body="&lt;p&gt;I have a bunch of gene names of &lt;em&gt;Apis mellifera&lt;/em&gt; (specifically 194). I used these gene names as an input on &lt;a href=&quot;https://string-db.org/&quot; rel=&quot;noreferrer&quot;&gt;STRING database&lt;/a&gt; to create a network for &lt;em&gt;Drosophila melanogaster&lt;/em&gt;. 54 of those genes were also present in &lt;em&gt;D. melanogaster&lt;/em&gt; and I received a second network. Is this a credible method of creating homologous gene network. Or should I use the sequences of the 194 genes and BLAST it to the geneset of the &lt;em&gt;D&lt;/em&gt;. &lt;em&gt;melanogaster&lt;/em&gt; and do it that way?&lt;/p&gt;&#xA;" OwnerUserId="976" LastEditorUserId="48" LastEditDate="2018-08-06T14:06:29.067" LastActivityDate="2018-08-08T08:03:49.557" Title="Are gene names same across species?" Tags="|gene|networks|sequence-homology|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4843" PostTypeId="2" ParentId="4842" CreationDate="2018-08-05T04:41:40.240" Score="10" Body="&lt;p&gt;Gene names are not consistent across species. Most of the time genes are automatically annotated and the algorithms try to give names that are similar to their matches in other organisms, but not always. It's also more likely that genes will be automatically named/annotated based on their sequence, and manually named/annotated based on their function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The further away you get from &lt;em&gt;Homo sapiens&lt;/em&gt; &amp;amp; &lt;em&gt;Mus musculus&lt;/em&gt; (and maybe &lt;em&gt;Caenorhabditis elegans&lt;/em&gt;, &lt;em&gt;Arabidopsis thaliana&lt;/em&gt; &amp;amp; &lt;em&gt;Danio rerio&lt;/em&gt;), the less likely it is to find consistently named genes.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-08-05T04:41:40.240" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4845" PostTypeId="2" ParentId="4820" CreationDate="2018-08-05T17:49:29.547" Score="2" Body="&lt;p&gt;To expand on the comment by Devon Ryan, what you really need are (A.) bam alignment files for each individual and (B.) VCF files for each individual showing all of their genomic variants (with respect to the reference human genome).  You mentioned that you had fastq files and that you converted them into fasta files.  Do you know if the bam files and VCF files are also available for download anywhere?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the bam files are not available you will have generate them using sequence aligner software (for example: bwa-mem).  With the bam files you can calculate the average depth of coverage (and many other things).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the VCF files are not available then you will have to generate them from the bam files using variant calling software (for example, the GATK package from the Broad Institute).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of assessing the quality of the sequence runs, a commonly applied tool is name FastQC, but once again, it uses fastq formatted files as input, not fasta files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main use for fasta formatted files these days seems to be for compact storage of various reference genomes from different genome sequencing projects (using gzip or similar type of compression).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the VCF files for each individual in hand you should be able to begin addressing your other questions regarding the frequency of shared variants, etc.&lt;/p&gt;&#xA;" OwnerUserId="3124" LastActivityDate="2018-08-05T17:49:29.547" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4848" PostTypeId="2" ParentId="4827" CreationDate="2018-08-06T15:35:47.527" Score="4" Body="&lt;p&gt;The correct API for Ensembl is the &lt;a href=&quot;http://rest.ensembl.org/documentation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl REST API&lt;/a&gt; which is updated and maintained by Ensembl, and language agnostic.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2018-08-06T15:35:47.527" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4849" PostTypeId="1" AcceptedAnswerId="13642" CreationDate="2018-08-06T22:09:41.733" Score="3" ViewCount="635" Body="&lt;p&gt;In one scRNA-seq sample I encountered the genes: Hbb-bs, Hba-a1 and Hba-a2. These genes appear on top of the list of the highest expressed genes but the 75% percentile of cells have very low counts. Therefore the expression is highly skewed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/D6ptW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D6ptW.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Reading from &lt;a href=&quot;https://www.genecards.org/cgi-bin/carddisp.pl?gene=HBB&amp;amp;keywords=hbb-bs&quot; rel=&quot;nofollow noreferrer&quot;&gt;gene cards&lt;/a&gt;, they appear to be coding hemoglobin subunits and involved in oxygen transport. I think they may need to be removed before normalization, also because the analysis I'm running does not consider oxygen transport.&lt;/p&gt;&#xA;&lt;p&gt;Has anyone encountered this situation and what do I do?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="3848" LastEditDate="2020-06-25T11:15:41.723" LastActivityDate="2021-06-16T19:14:55.767" Title="Hemoglobin subunits genes in scRNA-seq" Tags="|gene|scrnaseq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4851" PostTypeId="1" AcceptedAnswerId="9254" CreationDate="2018-08-07T10:35:42.853" Score="2" ViewCount="2975" Body="&lt;p&gt;I have single cell RNA-seq on 8 time points. I have clustered each time point to 2-3 clusters by seurat. I also have a list of differentially expressed genes between all my 1562 cells in 8 time points. I know how to plot this list of genes in heat map on each cluster of cells or a big heat map of my 1562 cells, but I have also sub-clusters for each cluster (time point). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you please help me to figure out how can I have a heat map with sub-clusters for each time point on which I am showing a list of differentially expressed genes? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something like  the picture below where each day has been clustered to 2-3 sub-population then they plotted common differentially expressed genes between day points&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets say 500 genes commonly are being differentially expressed in 8 time point. time point 1 has 3 sub-clusters of cells and time points 2 has 2 sub-clusters of cells. time point 3 and 4 both have 2 sub-clusters of cells. I want to heat map these 500 genes in cells in time point 1 (3 sub-clusters), time point 2 (2 sub-clusters), time point 23 (2 sub-clusters) and time point 4 (2 sub-clusters) in one unified heat map on which cells in each subclusteres (a,b and c) are in column and genes in rows but I don't have any idea how to do that. column annotation function in ComplexHeatmap likely works in this regard But I don't know how to define cells inside each  sub cluster in different colours &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can heat map of 500 genes only on time points easily by seurat but without underlying sub clusters&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; DoHeatmap(object = SubsetData(object = object, max.cells.per.ident =&#xA;&amp;gt; 100), genes.use = 50 genes, slim.col.label = TRUE, group.label.rot =&#xA;&amp;gt; TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/71AtW.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/71AtW.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Figure 2 in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5668937/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="48" LastEditDate="2019-08-23T15:08:45.687" LastActivityDate="2019-08-23T15:08:45.687" Title="How I can reproduce this heat map" Tags="|r|scrnaseq|seurat|heatmap|figure-reproduction|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4852" PostTypeId="1" AcceptedAnswerId="4853" CreationDate="2018-08-07T16:53:12.963" Score="2" ViewCount="574" Body="&lt;p&gt;I need to run &lt;code&gt;single-end&lt;/code&gt; &lt;code&gt;bulk-RNAseq&lt;/code&gt; &lt;code&gt;salmon&lt;/code&gt; - based alignment and I am not sure what I should supply as &lt;code&gt;--fldMean&lt;/code&gt; and &lt;code&gt;--fldSD&lt;/code&gt; which seem to be crucial for the &lt;code&gt;single-end&lt;/code&gt; experiment alignment. Can they be somehow derived from the &lt;code&gt;FASTQ&lt;/code&gt; files that I have, or should I be provided with them by someone who did the experiment?&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-08-07T18:51:26.807" Title="Where to get '--fldMean' and '--fldSD' for single-end Salmon run" Tags="|rna-seq|salmon|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4853" PostTypeId="2" ParentId="4852" CreationDate="2018-08-07T18:51:26.807" Score="1" Body="&lt;p&gt;From looking at the salmon documentation &lt;a href=&quot;http://salmon.readthedocs.io/en/latest/salmon.html#using-salmon&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, it is correct that both of these are essential for single end experiments:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Since the empirical fragment length distribution cannot be estimated from the mappings of single-end reads&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;From the discussion &lt;a href=&quot;https://github.com/COMBINE-lab/salmon/issues/127&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; it would appear that you can't estimate these parameters at all via the FASTQ files alone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fragment length refers to the physical molecule of D/RNA captured in the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4351865/&quot; rel=&quot;nofollow noreferrer&quot;&gt;library prep&lt;/a&gt; stage which you have sequenced (in part) on your sequencer.  Thus ideally this sort of data should come from a &lt;a href=&quot;https://web.uri.edu/gsc/agilent-2100-bioanalyzer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioanalyzer&lt;/a&gt; trace from whoever performed the library prep or similar QC step, this should ideally show the mean fragment length and standard deviation. &lt;/p&gt;&#xA;" OwnerUserId="1769" LastActivityDate="2018-08-07T18:51:26.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4854" PostTypeId="2" ParentId="3121" CreationDate="2018-08-07T19:30:03.563" Score="3" Body="&lt;p&gt;If you only have one gene and you only need to do this once then the simplest possible workflow is to generate the aliment using STAR (optimally with the two pass method) and open the two resultant &lt;code&gt;.bam&lt;/code&gt; files in IGV, the coverage column at the stop above your alignment track should clearly show the counts of reference and non-reference supporting reads.  The IGV &lt;a href=&quot;http://software.broadinstitute.org/software/igv/alignmentdata#coverage&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; shows a screen shot the pop-up coverage indicator and allele specific read count.&lt;/p&gt;&#xA;" OwnerUserId="1769" LastActivityDate="2018-08-07T19:30:03.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4856" PostTypeId="2" ParentId="4842" CreationDate="2018-08-08T08:03:49.557" Score="4" Body="&lt;p&gt;You could try using &lt;a href=&quot;http://metazoa.ensembl.org/biomart/martview&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioMart in Ensembl Metazoa&lt;/a&gt;. Filter by your list of gene names in honeybee, then get the fly homologues as attributes. The homologues are calculated by &lt;a href=&quot;http://www.ensembl.org/info/genome/compara/homology_method.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sequence comparison and clustering&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2018-08-08T08:03:49.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4858" PostTypeId="1" AcceptedAnswerId="5596" CreationDate="2018-08-08T10:01:06.697" Score="5" ViewCount="126" Body="&lt;p&gt;I am trying to simulate different genome of peoples, I have data (VCF files) of various genes from the 1000K Gene project. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to simulate different whole genomes i.e generate a new population by combining real haplotypes I have. I am wondering what is the best way to approach the problem. That is what is an efficient method of generating realistic (Not just randomly select parts and recombining them) new genotypes based on the real genotypes I already have. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using Bioconductor packages VariantAnnotation to read and manipulate the VCF files and TxDb.Hsapiens.UCSC.hg19.knownGene to determine the Genes positions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data looks as the following: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; gene58@rowRanges&#xA;GRanges object with 91 ranges and 1 metadata column:&#xA;              seqnames    ranges strand | paramRangeID&#xA;                 &amp;lt;Rle&amp;gt; &amp;lt;IRanges&amp;gt;  &amp;lt;Rle&amp;gt; |     &amp;lt;factor&amp;gt;&#xA;  rs551585351        1 229566998      * |         &amp;lt;NA&amp;gt;&#xA;  rs528384854        1 229567027      * |         &amp;lt;NA&amp;gt;&#xA;  rs542093083        1 229567063      * |         &amp;lt;NA&amp;gt;&#xA;  rs561849701        1 229567128      * |         &amp;lt;NA&amp;gt;&#xA;  rs531042647        1 229567160      * |         &amp;lt;NA&amp;gt;&#xA;          ...      ...       ...    ... .          ...&#xA;  rs565479298        1 229569784      * |         &amp;lt;NA&amp;gt;&#xA;  rs572772527        1 229569785      * |         &amp;lt;NA&amp;gt;&#xA;     rs605430        1 229569803      * |         &amp;lt;NA&amp;gt;&#xA;     rs605428        1 229569804      * |         &amp;lt;NA&amp;gt;&#xA;  rs368699658        1 229569810      * |         &amp;lt;NA&amp;gt;&#xA;  -------&#xA;  seqinfo: 86 sequences from &quot;hg19&quot; genome&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The genotype matrix of the gene: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; gene.mat[1:5, 1:5]&#xA;            HG00867 HG02371 HG00759 HG00766 HG00844&#xA;rs551585351 &quot;0|0&quot;   &quot;0|0&quot;   &quot;0|0&quot;   &quot;0|0&quot;   &quot;0|0&quot;  &#xA;rs528384854 &quot;0|0&quot;   &quot;0|0&quot;   &quot;0|0&quot;   &quot;0|0&quot;   &quot;0|0&quot;  &#xA;rs542093083 &quot;0|0&quot;   &quot;0|0&quot;   &quot;0|0&quot;   &quot;0|0&quot;   &quot;0|0&quot;  &#xA;rs561849701 &quot;0|0&quot;   &quot;0|0&quot;   &quot;0|0&quot;   &quot;0|0&quot;   &quot;0|0&quot;  &#xA;rs531042647 &quot;0|0&quot;   &quot;0|0&quot;   &quot;0|0&quot;   &quot;0|0&quot;   &quot;0|0&quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3039" LastEditorUserId="3039" LastEditDate="2018-12-02T14:29:37.127" LastActivityDate="2018-12-06T17:28:37.150" Title="Sampling haplotypes" Tags="|bioconductor|vcf|snp|haplotypes|simulated-data|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4859" PostTypeId="1" AcceptedAnswerId="4860" CreationDate="2018-08-08T13:33:23.693" Score="-1" ViewCount="1345" Body="&lt;p&gt;I have plotted a tSNE plot of my 1643 cells from 9 time points by seurat like below as 9 clusters. But, you know I should not expected each cluster of cells contains only cells from one distinct time point. For instance, cluster 2 includes cells from time point 16, 14 and even few cells from time point 12. My question is, how I can change the colour of cells here so cells from each time point has a distinct colour does not matter some cells are being clustered somewhere else because this is unavoidable. I have named my cells already so I know cells from each time point but I want to colour cells to nine colours based on the time point they have been sequenced not the cluster they are being grouped by seurat. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I first created a file in one column is name of my cells and in second colour the corresponding time point &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like so&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    time&#xA;s0.1    h0&#xA;s0.2    h0&#xA;s0.3    h0&#xA;s0.4    h0&#xA;s0.5    h0&#xA;s0.6    h0&#xA;s0.7    h0&#xA;s0.8    h0&#xA;s0.9    h0&#xA;s0.10   h0&#xA;s0.11   h0&#xA;s0.12   h0&#xA;s0.13   h0&#xA;s0.14   h0&#xA;s0.15   h0&#xA;s0.16   h0&#xA;s0.17   h0&#xA;s0.18   h0&#xA;s0.19   h0&#xA;s0.20   h0&#xA;s0.21   h0&#xA;s0.22   h0&#xA;s0.23   h0&#xA;s0.24   h0&#xA;s0.25   h0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I read my file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time &amp;lt;- read.table(&quot;time.txt&quot;,header = TRUE)&#xA;seurat@meta.data$time &amp;lt;- time&#xA;TSNEPlot(object = seurat, group.by = &quot;time&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Sorry in my code colour of tSNE changed also the position of cells changed that I don't want that. I want to change the color of cells but they are still in the same cluster&lt;/p&gt;&#xA;&#xA;&lt;p&gt;says that    &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; TSNEPlot(object = al)&#xA;Error in seq.default(h[1], h[2], length.out = n) : &#xA;  'to' must be a finite number &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;when I changed &lt;code&gt;names(identity)=sampleInfo$cellName&lt;/code&gt; to &lt;code&gt;names(identity)=src$cellName&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;that worked but again the position of cells changed that I don't want that :(&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="77" LastEditDate="2018-08-23T10:55:54.830" LastActivityDate="2018-08-23T10:55:54.830" Title="Changing the colour of each cell in tSNE plot" Tags="|r|scrnaseq|seurat|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4860" PostTypeId="2" ParentId="4859" CreationDate="2018-08-08T13:54:06.830" Score="2" Body="&lt;p&gt;Seurat puts the label in the tSNE plot according to the &lt;strong&gt;@ident&lt;/strong&gt; slot of the Seurat object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By default, if you do the tSNE without computing the clusters and you have the correct metadata in the object, the labels should be pointing to your timepoints not to the clusters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is not the case, what you have to do its to change this slot to reflect to which timepoint belong each cell you are plotting. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance if your Seurat object is called src:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;identity=as.factor(sampleInfo$timepoints)&#xA;names(identity)=sampleInfo$cellName&#xA;src@ident=identity&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then plot the tSNE. Something like this should work after adapting it to your metadata.&lt;/p&gt;&#xA;" OwnerUserId="678" LastActivityDate="2018-08-08T13:54:06.830" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4862" PostTypeId="1" CreationDate="2018-08-08T16:35:53.547" Score="2" ViewCount="449" Body="&lt;p&gt;I am willing to reproduce part of the analysis from &quot;The human transcriptome across tissues and individuals&quot; (Melé et all, 2015).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I downloaded GTEx v6 FPKM data in txt format from GTEx portal. I want to mimic the MDS plot from Supplementary Figure 3 (or get the equivalent PCA). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my process I did:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Load the FPKC matrix&lt;/li&gt;&#xA;&lt;li&gt;Remove those transcrits not having a level of 0.1 in minim 10 samples&lt;/li&gt;&#xA;&lt;li&gt;log10 transform after adding 1 to the matrix (to avoid 0s transformation) &lt;/li&gt;&#xA;&lt;li&gt;Quantile normalize the matrix&lt;/li&gt;&#xA;&lt;li&gt;Compute the distance matrix between samples&lt;/li&gt;&#xA;&lt;li&gt;MDS and plot&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;But my plot has nothing to to with the one in the paper. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gyXCl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gyXCl.png&quot; alt=&quot;MDS from GTEx v6 transcriptome data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone see any error in my process o see what I am doing that is totally different from GTEx process pipeline?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Extra info&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library( preprocessCore )&#xA;library( smacof )&#xA;&#xA;# Load gene expression matrix from file&#xA;gen &amp;lt;- read.delim( &quot;data/v6/GTEx_Analysis_v6p_RNA-seq_RNA-SeQCv1.1.8_gene_rpkm.gct&quot;, skip = 2, heade = TRUE )&#xA;&#xA;## First two columns corresponds to &quot;Transcript name&quot; and &quot;Gene name&quot;&#xA;gen_ann &amp;lt;- gen[ , c(1,2) ]&#xA;gen_clean &amp;lt;- as.matrix( gen[ , -seq( 2 ) ] )&#xA;&#xA;# Load map file&#xA;sample_map &amp;lt;- read.delim( &quot;data/v6/GTEx_Data_V6_Annotations_SampleAttributesDS.txt&quot; )&#xA;sample_map$SAMPID &amp;lt;- gsub( &quot;-&quot;, &quot;\\.&quot;, sample_map$SAMPID )&#xA;rownames( sample_map ) &amp;lt;- sample_map$SAMPID&#xA;&#xA;# Match data-sets&#xA;common_samples &amp;lt;- intersect( &#xA;    colnames( gen_clean ), rownames( sample_map )&#xA;)&#xA;gen_clean &amp;lt;- gen_clean[ , common_samples ]&#xA;sample_map &amp;lt;- sample_map[ common_samples, ]&#xA;&#xA;# Select tissues&#xA;tissue_to_get &amp;lt;- c( &quot;muscle&quot;, &quot;brain&quot;, &quot;heart&quot;, &quot;blood&quot;, &quot;pituitary&quot;, &quot;adipose tissue&quot;, &quot;nerve&quot;, &quot;lung&quot;, &quot;breast&quot;, &quot;blood vessel&quot;, &quot;thyroid&quot;, &quot;liver&quot;, &quot;skin&quot;, &quot;testis&quot; )&#xA;&#xA;samples_to_use &amp;lt;- sample_map$SAMPID[ &#xA;	    tolower( as.character( sample_map$SMTS ) ) %in% tissue_to_get &#xA;]&#xA;&#xA;# Filter gene expression matrix according to GTEx&#xA;trs_to_use &amp;lt;- apply( X = gen_clean, MARGIN = 1, FUN = function( row, exp_th, min_sam ) {&#xA;  sum( row &amp;gt; exp_th ) &amp;gt;= min_sam&#xA;}, exp_th = 0.1, min_sam = 10 )&#xA;gen_rd &amp;lt;- gen_clean[ trs_to_use, samples_to_use  ]&#xA;&#xA;# log10 transformation and normalization&#xA;gen_norm &amp;lt;- normalize.quantiles( log10( gen_rd[ , list_samples ] + 1 ) )&#xA;&#xA;# MDS&#xA;d &amp;lt;- dist( t( gen_norm ) )&#xA;mds &amp;lt;- mds( d, type = &quot;ratio&quot;, ndim = 3 )&#xA;[ ... creation of color map ... ]&#xA;plot( x = mds$conf[ , 1 ], y = mds$conf[ , 3 ], col = clr2, pch = 20 )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3176" LastEditorUserId="48" LastEditDate="2018-09-04T10:00:54.473" LastActivityDate="2018-09-04T10:00:54.473" Title="Reproducing GTEx transcriptome analysis" Tags="|r|rna-seq|transcriptome|figure-reproduction|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4863" PostTypeId="1" AcceptedAnswerId="4870" CreationDate="2018-08-08T20:27:10.310" Score="2" ViewCount="5571" Body="&lt;p&gt;I'm confused about the meaning of the black dots and the red shape in the violin plots from the &lt;a href=&quot;https://satijalab.org/seurat/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;seurat tutorial&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OVpiU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OVpiU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3171" LastEditorUserId="48" LastEditDate="2018-08-10T11:26:06.200" LastActivityDate="2018-08-10T11:46:17.740" Title="Interpretation of the violin plots from sc-RNA-seq" Tags="|scrnaseq|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4864" PostTypeId="1" AcceptedAnswerId="4884" CreationDate="2018-08-08T21:56:13.793" Score="0" ViewCount="324" Body="&lt;p&gt;I am working with two versions of the &lt;em&gt;C. elegans&lt;/em&gt; genome. I am finding interesting regions (specifically, tRNA genes) in version 1 and then I would like to know if version 2 also has a tRNA gene in that region. For this, I need to know the correspondence between coordinates in genome 1 and coordinates in genome 2. I think a whole-genome alignment programme might help me, e. g. nucmer or mauve. I have run nucmer on the two genomes and got a delta file but I don't know how to interpret it. I have run &lt;code&gt;show-coords&lt;/code&gt; on the delta file and obtained a .coords file. It seems more human-friendly. The first few lines look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/Users/user/Documents/project/output/genome/celegans/ref/wormbase/seq/c_elegans.PRJNA13758.WS245.genomic.fa /Users/user/Documents/project/output/genome/celegans/strain/seq/bristol/scaffolds.fa&#xA;NUCMER&#xA;&#xA;    [S1]     [E1]  |     [S2]     [E2]  |  [LEN 1]  [LEN 2]  |  [% IDY]  |  [LEN R]  [LEN Q]  |  [COV R]  [COV Q]  | [TAGS]&#xA;===============================================================================================================================&#xA;       1      432  |      498      929  |      432      432  |   100.00  | 15072434 15371751  |     0.00     0.00  | I  I&#xA;       1      432  |     1002     1433  |      432      432  |   100.00  | 15072434 15371751  |     0.00     0.00  | I  I&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does this mean that chromosome I starting at 1 and ending at 432 of version 1 corresponds to chromosome I starting at 498 and ending at 929 and that there is 100% identity between these two 432-long sequences? In this case, it's easy to find a correspondence between bases in version 1 and bases in version 2. However, some lines do not align sequences of the same length:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   1   237139  |     4022   241181  |   237139   237160  |    99.99  | 15072434 15371751  |     1.57     1.54  | I  I&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also, in this sequence position 1 of chromosome I is aligned again? So is it aligning to more than one position in version 2 of the genome? Also one sequence is 237139 bases long and the other 237160 bases long. How do I know the internal structure?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody guide me, please?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also aligned the two genomes using Mauve:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;progressiveMauve c_elegans.PRJNA13758.WS245.genomic.fa scaffolds.fa --output scaffolds.mauve &amp;gt; scaffolds.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've tried to import the .mauve file into Python using Biopython but I got an error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;alignment = SeqIO.read('scaffolds.mauve', &quot;mauve&quot;)&#xA;&#xA;Traceback (most recent call last):&#xA;  File &quot;code/mauve.py&quot;, line 4, in &amp;lt;module&amp;gt;&#xA;    alignment = SeqIO.read('scaffolds.mauve', &quot;mauve&quot;)&#xA;  File &quot;/Users/user/anaconda/lib/python3.6/site-packages/Bio/SeqIO/__init__.py&quot;, line 712, in read&#xA;    raise ValueError(&quot;More than one record found in handle&quot;)&#xA;ValueError: More than one record found in handle&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that the alignment opened in the Mauve GUI looks fine. Unfortunately, the picture won't load here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working on Mac OSX.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate your help in understanding the delta file, resolving the Python error or any other support in finding a function between coordinates in version 1 of the genome and coordinates in version 2 of the genome.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2018-08-08T22:15:19.743" LastActivityDate="2018-08-14T08:29:56.280" Title="Find a map/correspondence between two versions of a genome" Tags="|sequence-alignment|python|mauve|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="4865" PostTypeId="1" CreationDate="2018-08-09T03:13:26.910" Score="0" ViewCount="28" Body="&lt;p&gt;I am not sure this kind of question is allowed here. I am asking because some of you might have already experienced this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I submitted some virus genomes in the Genebank. The genomes are on hold and are not public yet.My PI thinks thank we should rename the isolates. I have sent two emails to the admin asking that I want to rename the isolates. I sent an Excel file containing accession number, current isolate name and new name that I want to assign. It has been almost a month and I haven't heard back. So, I don't know if this is the right approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should I do now ? Should I just resubmit the genomes ? Or is there any other way to do that ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks  &lt;/p&gt;&#xA;" OwnerUserId="1684" LastActivityDate="2018-08-09T10:00:43.860" Title="Genebank Submission" Tags="|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4866" PostTypeId="2" ParentId="4865" CreationDate="2018-08-09T10:00:43.860" Score="1" Body="&lt;p&gt;I usually hear back from the GenBank curators within a single day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, I’d still suggest you wait: it’s unlikely that they missed both of your emails (but maybe double-check the address and whether they’re likely to be caught by a spam filter): they’re probably just swamped with other submissions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, it’s holiday season so they might operate at drastically decreased capacity.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2018-08-09T10:00:43.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4867" PostTypeId="1" CreationDate="2018-08-09T11:35:20.330" Score="0" ViewCount="100" Body="&lt;p&gt;I've unmapped cleaned paired end sequence data in fastq format of a bacterial genome. I want to get a sequence data in Genbank format in the end. What are the exact steps that I have to follow in galaxy webserver and if in some point necessary, also using linux based apps? I would appreciate a lot of details.&lt;/p&gt;&#xA;" OwnerUserId="3179" LastEditorUserId="57" LastEditDate="2018-10-13T11:43:51.197" LastActivityDate="2021-01-29T23:29:34.137" Title="How to generate Genbank format out of paired end fastq data?" Tags="|sequence-annotation|genome|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4870" PostTypeId="2" ParentId="4863" CreationDate="2018-08-09T21:44:50.270" Score="1" Body="&lt;p&gt;The black dots represent the values for individual cells. The &quot;nGene&quot; plot (the first one) shows the number of detected genes for every cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The red shape shows the distribution of the data. For the &quot;nGene&quot; plot, you can see that the average number of genes per cell is about 900 and most of the cells have roughly around 700-1100 genes.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2018-08-09T21:44:50.270" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4871" PostTypeId="1" CreationDate="2018-08-10T03:07:36.650" Score="2" ViewCount="78" Body="&lt;p&gt;I am working on the &lt;code&gt;Rivanna&lt;/code&gt; cluster that is of &lt;code&gt;SLURM&lt;/code&gt;-type. Because of that I do not have write permissions and need to install the newest &lt;code&gt;R&lt;/code&gt; version myself locally, and use it. For it to be successfully built from source, I needed to also build from source &lt;code&gt;libcurl4-openssl-dev&lt;/code&gt;, so I downloaded &lt;code&gt;tar.gz&lt;/code&gt; file from here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://packages.debian.org/ru/jessie/libcurl4-openssl-dev&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://packages.debian.org/ru/jessie/libcurl4-openssl-dev&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and did:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mkdir build &#xA;./configure --prefix=/scratch/nv4e/curl/build&#xA;make&#xA;make install&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, in the same way I built from source &lt;code&gt;R&lt;/code&gt; &lt;code&gt;3.5.1&lt;/code&gt;, the latest current version. It was compiled without any errors, but with some warnings. Then, I submitted the &lt;code&gt;R&lt;/code&gt; script that uses &lt;code&gt;SCnorm&lt;/code&gt; package:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/SCnorm.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/html/SCnorm.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inside of the script I am having:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;source(&quot;https://bioconductor.org/biocLite.R&quot;)&#xA;biocLite(&quot;SCnorm&quot;)&#xA;biocLite(&quot;SingleCellExperiment&quot;)&#xA;library(SCnorm)&#xA;library(SingleCellExperiment) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Of course, I am submitting it through &lt;code&gt;.slurm&lt;/code&gt; file where I have the main command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Rscript --vanilla scnorm.R&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It fails, apparently because &lt;code&gt;SCnorm&lt;/code&gt; requires &lt;code&gt;RCurl&lt;/code&gt;, but it can not get one, even though in the log file it is trying to access it correctly:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cjKur.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cjKur.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;libcurl&lt;/code&gt; library was built from source in the &lt;code&gt;/scratch/nv4e/curl/build&lt;/code&gt; folder. The only suspicion that I have here is that maybe its version is wrong and &lt;code&gt;SCnorm&lt;/code&gt; needs another &lt;code&gt;libcurl&lt;/code&gt; version? Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Update&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I tried reinstalling &lt;code&gt;libcurl&lt;/code&gt; library from the official website, using the latest version:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://curl.haxx.se/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://curl.haxx.se/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it did not fix the issue. I have also tried using &lt;code&gt;libcurl&lt;/code&gt; from &lt;code&gt;Anaconda&lt;/code&gt; but it gave the same error. So, neither &lt;code&gt;7.61.0&lt;/code&gt; (latest), nor &lt;code&gt;7.29.0&lt;/code&gt; (&lt;code&gt;Anaconda&lt;/code&gt;), nor &lt;code&gt;7.38.0&lt;/code&gt; (&lt;code&gt;Debian&lt;/code&gt; link) are working.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="2005" LastEditDate="2018-08-10T20:10:15.053" LastActivityDate="2018-12-08T23:01:30.427" Title="RCurl issue when installing SCnorm" Tags="|r|scrnaseq|rcurl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4872" PostTypeId="2" ParentId="4871" CreationDate="2018-08-10T08:44:51.043" Score="1" Body="&lt;p&gt;This is a bug affecting the curl compile, it's been reported &lt;a href=&quot;https://github.com/curl/curl/issues/846&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and also &lt;a href=&quot;https://github.com/curl/curl/pull/1404&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, I suggest trying to compile from the newer release of the libcurl source rather than the &lt;code&gt;.deb&lt;/code&gt; you linked. &lt;/p&gt;&#xA;" OwnerUserId="1769" LastEditorUserId="1769" LastEditDate="2018-08-10T09:09:44.770" LastActivityDate="2018-08-10T09:09:44.770" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4873" PostTypeId="1" CreationDate="2018-08-10T09:03:18.313" Score="1" ViewCount="78" Body="&lt;p&gt;I am looking for the gene expression data in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE15543&quot; rel=&quot;nofollow noreferrer&quot;&gt;GEO GSE15543&lt;/a&gt; with the following keywords,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(Pancreatic Beta Cells) AND &amp;quot;Homo sapiens&amp;quot;[porgn:__txid9606]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to parse the gene expression value of a list of genes from all the samples reported in any particular study.I understand there are two different formats for downloading datasets, SOFT and MINiML.&lt;/p&gt;&#xA;&lt;p&gt;I have downloaded the soft.gz file and I am using BioConductor to read the contents from soft file using the instructions given &lt;a href=&quot;https://mdozmorov.github.io/BIOS567/assets/presentation_Bioconductor/GEO.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; . The file has the content that I 'm looking for , I have checked the contents on the geo2r webpage. I'm still facing challenge in finding out how to fetch out the gene expression values for a list of genes. Any suggestions?&lt;/p&gt;&#xA;&lt;p&gt;From the files that I download, I wish to parse the gene expression value and the Sample Title and Sample Description so that I can classify the values based on disease state and control state. Any suggestions on which format can be downloaded for the above-mentioned task?&lt;/p&gt;&#xA;" OwnerUserId="3186" LastEditorUserId="73" LastEditDate="2021-05-07T09:25:34.890" LastActivityDate="2021-05-07T18:58:03.647" Title="Parsing gene expression value for a list of genes from GEO" Tags="|gene-expression|geoquery|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4874" PostTypeId="2" ParentId="4863" CreationDate="2018-08-10T11:46:17.740" Score="0" Body="&lt;p&gt;The plot includes the data points that were used to generate it, with jitter on the x axis so that you can see them better. In red you see the actual violin plot, a vertical (symmetrical) plot of the distribution/density of the black data points.&lt;/p&gt;&#xA;" OwnerUserId="787" LastActivityDate="2018-08-10T11:46:17.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4875" PostTypeId="2" ParentId="4248" CreationDate="2018-08-10T11:57:39.537" Score="1" Body="&lt;p&gt;A general rule of thumb is that in order to improve the variance $n$ times you need $n^2$ neighbours. This is only applicable if you consider the $n^2$ nearest neighbours of a cell to be biologically identical (i.e. &quot;similar enough&quot;); if your data includes 10 types of cells with 10 cells each, then using the 20 nearest neighbours for smoothing will obscure the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I know, there is no single best answer to this question. I would suggest trying different numbers and sticking to what agrees more with the biology of the dataset.&lt;/p&gt;&#xA;" OwnerUserId="787" LastActivityDate="2018-08-10T11:57:39.537" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4876" PostTypeId="1" CreationDate="2018-08-12T15:42:46.130" Score="3" ViewCount="190" Body="&lt;p&gt;I've been completing various courses on bioinformatics through edX and Coursera, and I've made a strong footing in the material. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure what to pursue next in bioinformatics that would bring me closer to research in the field or employment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't have bioinformatics at my university and thus am unsure of how to progress.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a background in mathematics, chemistry, physics and computer science. So, what would you recommend I do next?&lt;/p&gt;&#xA;" OwnerUserId="3195" LastActivityDate="2019-01-09T15:17:57.627" Title="Path to career in bioinformatics?" Tags="|genomics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4877" PostTypeId="1" AcceptedAnswerId="4885" CreationDate="2018-08-13T06:31:23.780" Score="2" ViewCount="46" Body="&lt;p&gt;I'm filtering the gene expression value of a particular gene using Bioconductor in R. Following the instruction given &lt;a href=&quot;https://mdozmorov.github.io/BIOS567/assets/presentation_Bioconductor/GEO.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, I could filter the gene expression value for a gene from the dataset GDS3782. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output is as follows for the gene of my interest,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GSM524151  &quot;23.82&quot;          &#xA;GSM524152  &quot;27.49&quot;          &#xA;GSM524153  &quot;24.26&quot;          &#xA;GSM524154  &quot;17.98&quot;          &#xA;GSM524155  &quot;25.87&quot;          &#xA;GSM524156  &quot;26.16&quot;          &#xA;GSM524157  &quot;26.02&quot;          &#xA;GSM524158  &quot;25&quot;             &#xA;GSM524159  &quot;29.85&quot;          &#xA;GSM524160  &quot;18.25&quot;          &#xA;GSM524161  &quot;28.86&quot;          &#xA;GSM524162  &quot;27.33&quot;          &#xA;GSM524163  &quot;27.42&quot;          &#xA;GSM524164  &quot;28.23&quot;          &#xA;GSM524165  &quot;22.09&quot;          &#xA;GSM524166  &quot;24.85&quot;          &#xA;GSM524167  &quot;27.11&quot;          &#xA;GSM524168  &quot;32.01&quot;          &#xA;GSM524169  &quot;28.97&quot;          &#xA;GSM524170  &quot;30.98&quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above data corresponds to disease and control which I get using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Meta(gds3782)$description&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I wish to see the description of each value. For example, from the webpage, I have seen that the first ten values correspond to &quot;disease-state&quot; and the last ten correspond to &quot;control&quot;. How do I view this classification while processing the data using Bioconductor in R?&lt;/p&gt;&#xA;" OwnerUserId="3186" LastActivityDate="2018-08-14T08:46:59.727" Title="Filtering the expression values of a gene from GEO" Tags="|bioconductor|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4878" PostTypeId="2" ParentId="4858" CreationDate="2018-08-13T07:29:43.617" Score="0" Body="&lt;p&gt;Ensembl have haplotypes from 1000 Genomes available &lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Transcript/Haplotypes?db=core;g=ENSG00000128573;r=7:114414997-114693768;t=ENST00000403559&quot; rel=&quot;nofollow noreferrer&quot;&gt;by transcript&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2018-08-13T07:29:43.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4879" PostTypeId="1" CreationDate="2018-08-13T10:27:06.547" Score="3" ViewCount="65" Body="&lt;p&gt;There are several popular types of RNA-Seq library prep which are frequently used: total RNA (with and without ribosomal depletion), mRNA-Seq/poly-A, and targeted mRNA-Seq/RNA exome. What would the impact of choice of these various types of RNA-Seq have on downstream fusion detection by programs like: &lt;a href=&quot;https://github.com/ndaniel/fusioncatcher&quot; rel=&quot;nofollow noreferrer&quot;&gt;FusionCatcher&lt;/a&gt;, &lt;a href=&quot;https://github.com/STAR-Fusion/STAR-Fusion/wiki&quot; rel=&quot;nofollow noreferrer&quot;&gt;STAR-Fusion&lt;/a&gt;, &lt;a href=&quot;https://github.com/suhrig/arriba&quot; rel=&quot;nofollow noreferrer&quot;&gt;Arriba&lt;/a&gt;, and &lt;a href=&quot;https://github.com/pmelsted/pizzly&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pizzly&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance whist these methods have various biases and known issues, in particular rRNA depletion &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/24981968&quot; rel=&quot;nofollow noreferrer&quot;&gt;badly effects the accuracy of comparative exon level transcriptomics&lt;/a&gt;.  That and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5072325/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNA degradation is known to affect the efficacy of fusion detection&lt;/a&gt;. To date has anyone studied directly the choice of methodology on the performance of fusion detection directly, or can we infer that one method should be better than another for this usage case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously we can assume that targeted approached will only find partners of known genes, but will the different physical capture techniques introduce biases or shortfalls?&lt;/p&gt;&#xA;" OwnerUserId="1769" LastEditorUserId="1769" LastEditDate="2018-08-13T13:14:33.283" LastActivityDate="2018-08-13T13:14:33.283" Title="RNA-Seq type and and optimal fusion detection" Tags="|rna-seq|transcriptome|fusions|" AnswerCount="0" CommentCount="0" ClosedDate="2021-07-04T04:28:41.333" ContentLicense="CC BY-SA 4.0" />
  <row Id="4880" PostTypeId="1" AcceptedAnswerId="4888" CreationDate="2018-08-13T15:36:41.337" Score="1" ViewCount="718" Body="&lt;p&gt;Is that possible that the percent.mito level is zero? I have done vlnplot and it shows that its percent.mito level is zero. Does that mean my  mitochondrial gene didn't detect?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WvRfE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WvRfE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is violin plot. And the command is:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/x4MoL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/x4MoL.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3171" LastEditorUserId="77" LastEditDate="2018-08-14T07:20:44.207" LastActivityDate="2018-08-14T16:28:07.340" Title="SC-RNA seq percent.mito" Tags="|scrnaseq|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4882" PostTypeId="1" CreationDate="2018-08-14T01:35:35.983" Score="1" ViewCount="722" Body="&lt;p&gt;I am trying to use &lt;code&gt;DiffBind&lt;/code&gt; for my &lt;code&gt;ATACseq&lt;/code&gt; experiment going through the &lt;code&gt;vignette&lt;/code&gt; for &lt;code&gt;DiffBind&lt;/code&gt;. I was able to import the data and make a heatmap with &lt;code&gt;plot(data)&lt;/code&gt;. But somehow calling &lt;code&gt;data &amp;lt;- dba.contrast(data, categories=DBA_TISSUE)&lt;/code&gt; function is not working for me giving a warning message:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Warning message:&#xA;  Model must include count data for contrasts. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I tried running beforehand: &lt;code&gt;data &amp;lt;- dba.count(data)&lt;/code&gt; but it is giving an error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error: Error processing one or more read files. Check warnings().&#xA;  In addition: Warning messages:&#xA;  1:&lt;br&gt;&#xA;  2:&lt;br&gt;&#xA;  3:&lt;br&gt;&#xA;  4:&lt;br&gt;&#xA;  5:&lt;br&gt;&#xA;  6:&lt;br&gt;&#xA;  7:  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My &lt;code&gt;sample_sheet.csv&lt;/code&gt;looks like that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rlftk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rlftk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just need to compare &lt;code&gt;Dura&lt;/code&gt; vs &lt;code&gt;Microglia&lt;/code&gt; which is &lt;code&gt;Tissue&lt;/code&gt; column. How could this error be fixed? Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2023-12-19T01:03:45.460" Title="DiffBind: Model must include count data for contrasts." Tags="|atac-seq|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="4883" PostTypeId="2" ParentId="4867" CreationDate="2018-08-14T08:16:13.880" Score="1" Body="&lt;p&gt;The steps needed to assemble a genome in Galaxy are layed out in the &lt;a href=&quot;http://galaxyproject.github.io/training-material/topics/assembly/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Assembly training material&lt;/a&gt; from the Galaxy Training network. The software for this should be available on the main galaxy servers (e.g., usegalaxy.eu). Since you have a prokaryote, I suggest you follow the &quot;Unicycler Assembly&quot; workflow, which is also available on &lt;a href=&quot;https://usegalaxy.eu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;usegalaxy.eu&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-08-14T08:16:13.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4884" PostTypeId="2" ParentId="4864" CreationDate="2018-08-14T08:29:56.280" Score="1" Body="&lt;p&gt;A relatively convenient method would be to create a .chain file, as used by tools like &lt;code&gt;liftOver&lt;/code&gt; and &lt;code&gt;crossmap&lt;/code&gt; (and probably remap from NCBI). For your use-case, you can simply use the &lt;a href=&quot;http://genomewiki.ucsc.edu/index.php/DoSameSpeciesLiftOver.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;DoSameSpeciesLiftOver.pl&lt;/a&gt; command as described in the link. The resulting chain files are pretty self-explanatory and can be used to map BED files of regions of interest.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-08-14T08:29:56.280" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4885" PostTypeId="2" ParentId="4877" CreationDate="2018-08-14T08:46:59.727" Score="3" Body="&lt;p&gt;You're looking for the &lt;code&gt;Columns()&lt;/code&gt; function:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; Columns(gds3782)&#xA;     sample        disease.state           description&#xA;1  GSM524151 non-diabetic control  Value for GSM524151: beta-cells_non-diabetic condition_donor1; src: human beta-cells_non-diabetic&#xA;2  GSM524152 non-diabetic control  Value for GSM524152: beta-cells_non-diabetic condition_donor2; src: human beta-cells_non-diabetic&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that it's probably easier to work with a more common structure like and expressionSet:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;eset = GDS2eSet(gds3782)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You'd then use the &lt;code&gt;pData()&lt;/code&gt; function to return the same thing.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-08-14T08:46:59.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4887" PostTypeId="1" AcceptedAnswerId="15920" CreationDate="2018-08-14T14:14:48.803" Score="3" ViewCount="309" Body="&lt;p&gt;I was looking for a simple way to do a &lt;a href=&quot;https://en.wikipedia.org/wiki/Sequence_alignment#Global_and_local_alignments&quot; rel=&quot;nofollow noreferrer&quot;&gt;glocal alignment&lt;/a&gt;.&#xA;The case I have is I have a small sequence which should be find in a bigger one, thus typically a glocal alignment.&#xA;Also I can not Install anything on the target machine (Python 3.6 with Biopython is already installed), all I can do is pip install.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is : are there any glocal alignment implementation out there ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was unable to found one and I don't feel proficient enough with Python yet to implement it myself.&lt;/p&gt;&#xA;" OwnerUserId="3209" LastActivityDate="2021-05-14T14:11:13.477" Title="Glocal\semi-local\Hybride Globale-Local alignment with Python" Tags="|sequence-alignment|python|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4888" PostTypeId="2" ParentId="4880" CreationDate="2018-08-14T15:44:21.400" Score="1" Body="&lt;p&gt;I've just come across a similar issue. It seems the mouse and human mitochondrial genes use mixed capitalizations. See the following PR for details:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/Hoohm/dropSeqPipe/pull/43&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Hoohm/dropSeqPipe/pull/43&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bottom line is, if you ignore the case in the grep pattern it might work (assuming you use human/mouse):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mito.genes &amp;lt;- grep(&quot;^mt-&quot;, rownames(shox2crepos_mtmg@data), value=TRUE, ignore.case=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1461" LastActivityDate="2018-08-14T15:44:21.400" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4889" PostTypeId="2" ParentId="4880" CreationDate="2018-08-14T16:28:07.340" Score="1" Body="&lt;p&gt;The simplest answer is that the bit of code you copied from the Seurat vignette is not working on the gene names you have.  Look at the code.  If none of your gene names begin with &quot;MT-&quot;, none will be identified as mitochondrial.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2018-08-14T16:28:07.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4890" PostTypeId="1" AcceptedAnswerId="5024" CreationDate="2018-08-15T10:09:39.660" Score="2" ViewCount="261" Body="&lt;p&gt;I am using Gviz to plot several tracks. I create a &lt;code&gt;GenomeAxisTrack()&lt;/code&gt; and then several &lt;code&gt;AnnotationTrack()&lt;/code&gt;s that I feed to &lt;code&gt;plotTracks()&lt;/code&gt;. This produces a stacked output of tracks with annotated arrows as I want but the displays are linear. In some cases I have the complete genomes and I know they are circular (like some bacterial and viral chromosomes), how can I display the plots as circular?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried feeding to &lt;code&gt;AnnotationTrack()&lt;/code&gt;; a &lt;code&gt;GRanges()&lt;/code&gt; object with a &lt;code&gt;Seqinfo()&lt;/code&gt; object specifying the &lt;code&gt;isCircular=TRUE&lt;/code&gt; flag, but this didn't change the resulting plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is minimal code to create a linear plot, I would like one where the first and last arrow join up in a circle:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Gviz)&#xA;library(stringr)&#xA;&#xA;track &amp;lt;- AnnotationTrack(start=c(1,5,7), end=c(2,6,10), strand=c('+','+','-'), stacking=&quot;dense&quot;, fill=c('red','blue', 'black'))&#xA;png(file=paste(&quot;test&quot;,&quot;png&quot;, sep=&quot;.&quot;))&#xA;plotTracks(track)&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SJ9Nw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SJ9Nw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, I am not interested in alternative software recommendations, I would like to solve this with gviz if it is possible. Also asked this &lt;a href=&quot;https://support.bioconductor.org/p/112040/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2018-09-11T14:22:17.090" Title="How to plot circular chromosomes with Gviz?" Tags="|r|bioconductor|gviz|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4891" PostTypeId="1" CreationDate="2018-08-15T11:18:35.333" Score="0" ViewCount="105" Body="&lt;p&gt;I'm trying to compare the gene expression data reported in the studies documented in GEO for a specific gene expressed in a tissue of my interest in Homo sapiens.&#xA;I compared the values reported in 4 studies, in two studies the expression value of gene A lies between 15-30, in the other two studies the range lies between 1-4.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to ask for advice on whether it is meaningful to compare the data from different studies. What are the factors that one should keep in mind while making such comparisons? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be highly appreciated&lt;/p&gt;&#xA;" OwnerUserId="3186" LastActivityDate="2018-08-15T14:07:59.300" Title="Comparing the gene expression data" Tags="|gene|gene-expression|geoquery|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4892" PostTypeId="2" ParentId="4891" CreationDate="2018-08-15T12:04:00.343" Score="1" Body="&lt;p&gt;There's going to be a batch effect between studies, so unless you have some sort of constant group that you can use for between-study normalization you're going to completely muck up your results from using data from multiple studies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More specifically, this is what packages like &lt;code&gt;sva&lt;/code&gt; and the &lt;code&gt;combat()&lt;/code&gt; command in it are used to correct. Note that you should go back to the raw .CEL or .fastq files if you want to process multiple studies, in order to at least ensure that everything is processed in the same way.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-08-15T12:04:00.343" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4893" PostTypeId="2" ParentId="4891" CreationDate="2018-08-15T14:07:59.300" Score="1" Body="&lt;p&gt;It doesn't make sense to compare gene expression data from different studies unless you have a negative spiked-in control such as &lt;a href=&quot;https://www.sequin.xyz/&quot; rel=&quot;nofollow noreferrer&quot;&gt;sequins&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most obvious thing you will need to do is normlizate the data for various bias such as batch effects, sequencing platforms, sequencing depth, bioinformatic workflows and more ... Your task will be quite &lt;strong&gt;complicated&lt;/strong&gt; as you must justify how your normalization would work for data from different studies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should check how the raw data was generated (methods section), convince yourself, and then apply gene quantification yourself from the raw files.&lt;/p&gt;&#xA;" OwnerUserId="174" LastActivityDate="2018-08-15T14:07:59.300" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4894" PostTypeId="1" AcceptedAnswerId="4921" CreationDate="2018-08-15T15:38:34.193" Score="2" ViewCount="200" Body="&lt;p&gt;I'd like to fuse ~50 pairs of corresponding fastq files &lt;em&gt;on per sequence bases&lt;/em&gt;. I'd be happy for a solution as a Linux script, R script (if feasible with huge files), or any free software that runs preferably under Debian Linux (otherwise Windows). Aim is preprocessing for a Linux pipeline for NGS data. Ideally both the sequence and quality data are fused.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;aim&lt;/strong&gt; of this request is the following: there is an existing pipeline for downstream mapping of immunoreceptor sequences (migec/mixcr/vdjtools etc.). Previously we got Illumina reads (read 1, read 2 = reverse), with the index and UMI sequences attached to one of the reads =&gt; can be alligned and processed to habe 1 biological useful sequence with UMI and index at one end. NOW we get read 1, read 2 and seperately &quot;read 3&quot; which is just the UMI+Index reads, and which was attached by the core facility to read 1 using software we do not have access to. Since the previous pipeline works well, we prefere to use it in the future, but are open for suggestions how to use the index reads with migec/mixcr/vdjtools otherwise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each of the 2 files contains the same number (millions) of short (~300 bp) and corresponding DNA reads:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;File_A.fastq:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;@record1 / aaa... / +... / ...&lt;/li&gt;&#xA;&lt;li&gt;@record2 / ...&lt;/li&gt;&#xA;&lt;li&gt;@record3 / ...&lt;/li&gt;&#xA;&lt;li&gt;... (millions)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;File_B.fastq:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;@record1 / ttt... / +... / ...&lt;/li&gt;&#xA;&lt;li&gt;@record2 / ...&lt;/li&gt;&#xA;&lt;li&gt;@record3 / ...&lt;/li&gt;&#xA;&lt;li&gt;... (millions)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The target file should look like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;@record1 / aaa...ttt... / +... / ...&lt;/li&gt;&#xA;&lt;li&gt;@record2 / ...&lt;/li&gt;&#xA;&lt;li&gt;@record3 / ...&lt;/li&gt;&#xA;&lt;li&gt;... (millions)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;(of course the aaa...ttt... is just an example)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was looking for a solution on Stackexchange and Google, but the solutions, I found do not work for me because&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;manual processing is not an option given the number of sequences&lt;/li&gt;&#xA;&lt;li&gt;I need to fuse the files on a &lt;strong&gt;per sequence&lt;/strong&gt; basis, i.e. a simple Linux command like cat file1 file2 is not what I need (as far as I understand, this would give me (record1_A record2_A.... record1B record2_B).&lt;/li&gt;&#xA;&lt;li&gt;R, package shortread, readFastq + writeFastq would be perfect, but cannot handle such large numbers of input lines&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Maybe I did not use the right search terms, but in any case I'd be happy for a solution&lt;/p&gt;&#xA;" OwnerUserId="3177" LastEditorUserId="3177" LastEditDate="2018-08-20T22:08:54.160" LastActivityDate="2018-08-21T11:49:46.720" Title="fuse fastq files with multiple records" Tags="|fastq|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4895" PostTypeId="2" ParentId="4894" CreationDate="2018-08-15T17:02:19.957" Score="0" Body="&lt;p&gt;What you actually want to do is overlap the reads, for which you can use tools like &lt;a href=&quot;https://ccb.jhu.edu/software/FLASH/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FLASH&lt;/a&gt; or &lt;a href=&quot;https://jgi.doe.gov/data-and-tools/bbtools/bb-tools-user-guide/bbmerge-guide/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BBMerge&lt;/a&gt; (I've had better luck with FLASH).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-08-15T17:02:19.957" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4896" PostTypeId="1" AcceptedAnswerId="4897" CreationDate="2018-08-15T17:43:41.007" Score="4" ViewCount="598" Body="&lt;p&gt;I have some RNA-Seq data from leukaemia patients. I want to do unsupervised clustering on them with some other published leukaemia RNA-Seq data and see how they cluster. There are a few problems I encountered while doing this.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I read mix messages of whether using log2(TPM) or rlog(counts) (e.g. Deseq2 rlog or limma voom transformation) for clustering. Which one should I choose?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If I am filtering out genes with low counts, should I do it prior normalisation to library size or after?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I tried using filterByExpr from edgeR for filtering but it removed many of the genes that are only expressed by subtypes with small sample number. If I am to filter the counts &quot;manually&quot;, is there a recommendation of how the threshold should be set?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If I should use log2(TPM) for unsupervised clustering, can I treat low TPM (e.g. &amp;lt;2) as 0? As this &lt;a href=&quot;https://link.springer.com/article/10.1007%2Fs12064-013-0178-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; suggested TPM &amp;lt; 2 basically means no expression, is it alright to treat them as 0?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3225" OwnerDisplayName="kento1991" LastEditorUserId="48" LastEditDate="2018-08-18T11:05:38.747" LastActivityDate="2018-08-18T11:05:38.747" Title="RNA-Seq: clustering/treatment of genes with low expression" Tags="|rna-seq|clustering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4897" PostTypeId="2" ParentId="4896" CreationDate="2018-08-15T20:24:38.423" Score="3" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;rlog(normalized counts)&lt;/code&gt; is going to be more robust than &lt;code&gt;log2(TPM)&lt;/code&gt;, so use it instead.&lt;/li&gt;&#xA;&lt;li&gt;Do it afterward, keeping the low counts can be helpful for library-size normalization.&lt;/li&gt;&#xA;&lt;li&gt;This is very much more of an art than a science, though have a look at the &lt;code&gt;kOverA()&lt;/code&gt; function from the &lt;code&gt;genefilter&lt;/code&gt; package. That will give you a bit more fine-grained control on filtering.&lt;/li&gt;&#xA;&lt;li&gt;You'd be better off using something like zFPKMs if you want to filter by &quot;expressed or not&quot;, since there's no really great definition of that.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-08-15T20:24:38.423" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4898" PostTypeId="1" AcceptedAnswerId="4902" CreationDate="2018-08-15T21:20:43.983" Score="2" ViewCount="226" Body="&lt;p&gt;I have used minimap2 to map some pacbio reads to a reference genome. I would like to know the &quot;insert size&quot; (true length of sequence) relative to the reference. More specifically, I want to know the first base position in the reference where the read maps and the last base position in the reference where the mapping ends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand it I need the TLEN field from bam files (&lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;sam format field 9&lt;/a&gt;). However, all the alignment files from minimap2 have &lt;code&gt;0&lt;/code&gt; in the TLEN field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know how to get pacbio alignments with the TLEN field? I am also happy using &lt;code&gt;bwa&lt;/code&gt; or another aligner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The command I used was &lt;code&gt;minimap2 -ax map-pb {fasta} {reads} | samtools view -h -b - | samtools sort - &amp;gt; myfile.bam&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-08-16T07:06:30.117" Title="Minimap2 -ax map-pb doesn't output tlen field" Tags="|sequence-alignment|phylogenetics|bwa|minimap2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4899" PostTypeId="1" AcceptedAnswerId="4901" CreationDate="2018-08-16T00:59:49.887" Score="3" ViewCount="442" Body="&lt;p&gt;I was wondering that the threshold that we set after doing QC in Seurat. there is an example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# GenePlot is typically used to visualize gene-gene relationships, but can be used for anything &#xA;# calculated by the object, i.e. columns in object@meta.data, PC scores etc.&#xA;# Since there is a rare subset of cells with an outlier level of high mitochondrial percentage&#xA;# and also low UMI content, we filter these as well&#xA;par(mfrow = c(1, 2))&#xA;GenePlot(object = pbmc, gene1 = &quot;nUMI&quot;, gene2 = &quot;percent.mito&quot;)&#xA;GenePlot(object = pbmc, gene1 = &quot;nUMI&quot;, gene2 = &quot;nGene&quot;)`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BCbyf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BCbyf.png&quot; alt=&quot;QC ANALYSIS&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then its filter cell command is under:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# We filter out cells that have unique gene counts over 2,500 or less than 200&#xA;# Note that low.thresholds and high.thresholds are used to define a 'gate'&#xA;# -Inf and Inf should be used if you don't want a lower or upper threshold.&#xA;pbmc &amp;lt;- FilterCells(object = pbmc, subset.names = c(&quot;nGene&quot;, &quot;percent.mito&quot;), low.thresholds = c(200, -Inf), high.thresholds = c(2500, 0.05))` &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So I want to know whether the threshold setting is depending on the picture I show?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the below is the Geneplot I have got base on my data. How can I set my thresholds?&#xA;&lt;a href=&quot;https://i.stack.imgur.com/IJvQv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IJvQv.png&quot; alt=&quot;my old geneplot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3171" LastEditorUserId="3171" LastEditDate="2018-08-16T14:52:55.230" LastActivityDate="2023-04-21T20:46:10.553" Title="how to filter out cell after doing QC" Tags="|scrnaseq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4900" PostTypeId="1" CreationDate="2018-08-16T06:22:27.677" Score="4" ViewCount="768" Body="&lt;p&gt;I have a problem with using &lt;code&gt;rbind&lt;/code&gt; to combine VCF files using the library &lt;a href=&quot;https://bioconductor.org/packages/VariantAnnotation&quot; rel=&quot;nofollow noreferrer&quot;&gt;VariantAnnotation&lt;/a&gt; from Bioconductor.&lt;/p&gt;&#xA;&lt;p&gt;I am reading two VCF files, when I try to combine them in a certain order with &lt;code&gt;rbind&lt;/code&gt; I'm getting an error. When I combine them in a different order, it works fine.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;vcf.full &amp;lt;- rbind(vcf1, vcf2) ## Doesnt work &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error in as(from, to_class, strict = FALSE) :    no method or default&#xA;for coercing “CompressedCharacterList” to “DNAStringSetList”&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;vcf.full &amp;lt;- rbind(vcf2, vcf1) ## Work &#xA;&#xA;header(vcf1)&#xA;class: VCFHeader &#xA;samples(503): NA06984 NA06989 ... NA20828 NA20832&#xA;meta(2): META contig&#xA;fixed(2): FILTER ALT&#xA;info(27): CIEND CIPOS ... EX_TARGET MULTI_ALLELIC&#xA;geno(1): GT&#xA;&#xA;header(vcf2)&#xA;class: VCFHeader &#xA;samples(503): NA06984 NA06989 ... NA20828 NA20832&#xA;meta(2): META contig&#xA;fixed(2): FILTER ALT&#xA;info(27): CIEND CIPOS ... EX_TARGET MULTI_ALLELIC&#xA;geno(1): GT&#xA;&#xA;dim(vcf1)&#xA;[1]  30 503&#xA;&#xA;dim(vcf2)&#xA;[1] 149 503&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The two VCF files seem identical. Any idea of why it happens? Or any suggestions on a different method to combine multiple VCF files using R?&lt;/p&gt;&#xA;" OwnerUserId="3039" LastEditorUserId="3694" LastEditDate="2022-06-10T14:24:32.040" LastActivityDate="2022-06-10T14:24:32.040" Title="Combine VCF files" Tags="|r|vcf|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4901" PostTypeId="2" ParentId="4899" CreationDate="2018-08-16T06:52:38.453" Score="1" Body="&lt;p&gt;Yes, you end up placing filters where things become inconsistent or similar to background. In the example, which appears to be from the Seurat tutorial, there's a downward plunge in &lt;code&gt;nGenes vs. nUMI&lt;/code&gt; at around 200 genes and few believable cells with more than 2000-2500 genes, so the tutorials sets filters around there. The 0.05 mitochondrial contamination is a more arbitrary round number and is less supported by the plots (one would use more like 0.06 or 0.07 if one based this solely on the plots).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-08-16T06:52:38.453" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4902" PostTypeId="2" ParentId="4898" CreationDate="2018-08-16T07:06:30.117" Score="2" Body="&lt;p&gt;The TLEN for single-end reads is defined as 0 in the SAM specification, which is why this happens. You can still get the end position of the alignment by parsing the CIGAR string. An example using the pysam package in python is below:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pysam&#xA;&#xA;bam = pysam.AlignmentFile(&quot;myfile.bam&quot;)&#xA;for b in bam.fetch():&#xA;    print(&quot;This alignment spans {}:{}-{}&quot;.format(b.reference_name, b.reference_start+1, b.reference_end))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;reference_end&lt;/code&gt; value is automatically calculated from the CIGAR string. &lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-08-16T07:06:30.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4903" PostTypeId="1" AcceptedAnswerId="4904" CreationDate="2018-08-16T23:01:33.613" Score="2" ViewCount="487" Body="&lt;p&gt;Is there a pedigree viewer that works with PLINK pedigree files? I saw an old post by &lt;a href=&quot;https://www.biostars.org/p/60186/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pierre&lt;/a&gt;, but I don't see the script on his blog.&lt;/p&gt;&#xA;" OwnerUserId="136" LastActivityDate="2018-08-17T10:16:56.900" Title="Visualize PLINK pedigree files" Tags="|plink|pedigree|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4904" PostTypeId="2" ParentId="4903" CreationDate="2018-08-17T10:16:56.900" Score="3" Body="&lt;p&gt;We use &lt;code&gt;kinship2&lt;/code&gt; R package to visualize pedigree files (.fam plink files). For some reasons, some characters from plink ped aren't supported (like &quot;-9&quot; for unknown genre or affected status).&lt;br&gt;&#xA;Here is the R script we use:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;args&amp;lt;-commandArgs(TRUE)&#xA;pedPath&amp;lt;-args[1]&#xA;pdfFile&amp;lt;-args[2]&#xA;&#xA;library(kinship2)&#xA;pedfile &amp;lt;- read.table(pedPath,sep=&quot;\t&quot;)&#xA;pedfile$V5 &amp;lt;- apply(pedfile,1,function(data){if(data[5] == 1 || data[5] ==2){data[5]}else{3}}) &#xA;pedfile$affected &amp;lt;- apply(pedfile,1,function(data){if(as.numeric(data[6]) == 1){res &amp;lt;- data[6]}else{if(as.numeric(data[6]) == 2){res &amp;lt;- data[6]}else{res &amp;lt;- NA}}})&#xA;&#xA;familyIDs &amp;lt;- as.character(unique(pedfile$V1))&#xA;&#xA;pdf(file=pdfFile)&#xA;for(famID in familyIDs){&#xA;    ped &amp;lt;- pedigree(id=as.character(pedfile$V2[pedfile$V1==famID]),dadid=as.character(pedfile$V3[pedfile$V1==famID]),momid=as.character(pedfile$V4[pedfile$V1==famID]),sex=as.numeric(pedfile$V5[pedfile$V1==famID]),affected=as.numeric(pedfile$affected[pedfile$V1==famID]))&#xA;    plot.pedigree(ped,cex=0.7,align=TRUE)&#xA;    title(famID)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And we call it this way:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Rscript &amp;lt;kinshipScript.R&amp;gt; &amp;lt;plink ped file&amp;gt; &amp;lt;pdf output&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="373" LastActivityDate="2018-08-17T10:16:56.900" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4907" PostTypeId="1" CreationDate="2018-08-17T14:57:59.793" Score="1" ViewCount="35" Body="&lt;p&gt;I have a table of blast hits in a database that I'd like to chain. What is a good choice for a 'stand alone' chaining tool? Is it simple to implement in Perl, for example?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example 'hit' data is &lt;a href=&quot;https://gist.githubusercontent.com/dbolser/c0e9d570b7023997ca6488bc483d208c/raw/1afc051d356409d8521ff35be086ce6cc8ed27f2/dummy%2520data.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Background is &lt;a href=&quot;http://www.mi.fu-berlin.de/wiki/pub/ABI/GenomeComparisonP4/chaining.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="931" LastEditorUserId="48" LastEditDate="2018-08-18T11:01:07.527" LastActivityDate="2018-08-18T11:01:07.527" Title="Stand alone chaining tool for existing blast hits?" Tags="|sequence-alignment|algorithms|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4908" PostTypeId="2" ParentId="4882" CreationDate="2018-08-17T19:07:31.357" Score="0" Body="&lt;p&gt;The problem was that I was using wrong files. I ran a pipeline from &lt;a href=&quot;http://code.databio.org/PEPATAC/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and it generates several folders with many files. The &lt;code&gt;.bam&lt;/code&gt; files that should be used are located in the &lt;code&gt;aligned_mm10&lt;/code&gt; folder and are with the extension: &lt;code&gt;.pe.q10.sort.rmdup.bam&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2018-12-14T14:56:45.657" LastActivityDate="2018-12-14T14:56:45.657" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4909" PostTypeId="1" AcceptedAnswerId="4914" CreationDate="2018-08-17T21:08:52.493" Score="3" ViewCount="605" Body="&lt;p&gt;I mapped fasta sequences to a reference. The fasta sequences range in length from 17bp up to several hundred bp. I used &lt;code&gt;minimap2&lt;/code&gt; with the following command: &lt;code&gt;minimap2 -ax sr ref.fa seqs.fa&lt;/code&gt;, et cetera. Only the sequences greater than 32bp len mapped though.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -F 4 filename.sorted.bam | \&#xA;  cut -f10 | awk '{print length($1)}' | \&#xA;  sort | uniq -c | sort -k2 -n&#xA;&#xA;   2304 32&#xA;  34931 33&#xA;   5443 34&#xA;   5343 35&#xA;   8811 36&#xA;   5485 37&#xA;   5417 38&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is it possible to make minimap2 map sequences shorter than 32bp? Do I need to use &lt;code&gt;bwa aln&lt;/code&gt; or some other mapper?&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-08-20T11:48:57.550" Title="How to map reads shorter than 32bp with minimap2?" Tags="|fasta|read-mapping|phylogenetics|bwa|minimap2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4910" PostTypeId="1" AcceptedAnswerId="4912" CreationDate="2018-08-18T18:12:48.707" Score="3" ViewCount="80" Body="&lt;p&gt;First question here so sorry if it is about both genetics and evolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if I get a disease which can be clearly identified on the gene level and all my descendants inherit it, after 500 years do all people with the disease share the common ancestor (given that it's only inheritable)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mean, my clear guess first would be &quot;yes&quot; but I really want to check it since I haven't read it anywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Feel free to post materials on this too.&#xA;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="3239" LastActivityDate="2018-08-20T21:53:13.240" Title="Do people today with a disease caused by single gene mutation that is inheritable - all share a common ancestor?" Tags="|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4911" PostTypeId="1" AcceptedAnswerId="4918" CreationDate="2018-08-18T23:43:34.103" Score="4" ViewCount="9746" Body="&lt;p&gt;I found &lt;a href=&quot;http://cagt.bu.edu/w/images/8/81/Avg_read_len_fastq.awk&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; this awk script: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BEGIN {&#xA;  headertype=&quot;&quot;;&#xA;  }&#xA;{&#xA;  if($0 ~ &quot;^@&quot;) {&#xA;    countread++;&#xA;    headertype=&quot;@&quot;;&#xA;  }&#xA;  else if($0 ~ &quot;^+&quot;) {&#xA;    headertype=&quot;+&quot;;&#xA;  }&#xA;  else if(headertype=&quot;@&quot;) { # This is a nuc sequence&#xA;    len=length($0);&#xA;    if (len&amp;gt;4) {&#xA;      readlength[len]++;&#xA;    }&#xA;  }&#xA;}&#xA;END {&#xA;  for (i in readlength){&#xA;    countstored+=readlength[i];&#xA;    lensum+=readlength[i]*i;&#xA;    print i, readlength[i];&#xA;  }&#xA;  print &quot;reads read = &quot;countread &amp;gt; &quot;/dev/stderr&quot;;&#xA;  print &quot;reads stored = &quot;countstored &amp;gt; &quot;/dev/stderr&quot;;&#xA;  print &quot;read average length = &quot;lensum/countstored &amp;gt; &quot;/dev/stderr&quot;;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I just wonder if it is possible to shorten it with bioawk?&lt;/p&gt;&#xA;" OwnerUserId="2477" LastEditorUserId="73" LastEditDate="2023-01-15T20:20:07.620" LastActivityDate="2023-01-15T20:20:07.620" Title="Calculating read average length in a Fastq file with bioawk/awk" Tags="|fastq|awk|bioawk|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4912" PostTypeId="2" ParentId="4910" CreationDate="2018-08-19T01:06:29.223" Score="2" Body="&lt;p&gt;That can happen, but it's more likely that the disease will be present in more than one family. Diseases with a single, small genetic cause will be more likely to flare up elsewhere (assuming the disease isn't &lt;em&gt;too&lt;/em&gt; bad, and it wouldn't be if it's been in the family for 500 years).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, what you will find is that in &lt;em&gt;your&lt;/em&gt; family that disease is almost always present together with some other inherited genetic feature that is nearby on the genome (i.e. an inherited haplotype block). This association is less likely to be shared with other families.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-08-20T21:53:13.240" LastActivityDate="2018-08-20T21:53:13.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4913" PostTypeId="1" AcceptedAnswerId="4917" CreationDate="2018-08-19T08:51:11.920" Score="4" ViewCount="619" Body="&lt;p&gt;The context: I have a DNA-sequence coding for a protein, about 1500 bp in length. Using NGS, a lot of reads of (mutants of) this same sequence were acquired. All of these reads need to be aligned to the reference. We're talking about a lot of reads (100,000 - 1,000,000) of not too short length (120 - 300 bp). All of these reads can belong to different mutants of the template sequence, so the alignment is necessary to determine the exact sequence of every single mutant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I'm just using Smith-Waterman-based local alignment to align every single read to the reference one by one. Yet I can't help but feel there might be a more computationally/time-efficient solution to this specific problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe there exists an algorithm that's not very efficient for most alignment problems, but that becomes very worthwile if it has to map a ton of reads to the same place over and over again. For example, it might do some time-consuming operations on the short template that make it fast to align reads to it, but these operations aren't worth it if only aligning a couple of reads. That's just an idea, I don't know all the different techniques that are out there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So, to recap&lt;/strong&gt;: I want to align a lot (100,000 - 1,000,000) of 120-300 bp reads to the same short 1500 bp reference sequence. If anyone has any suggestions about an algorithm, or just a specific workflow, that is particularly suited to do this, it would be appreciated. I work in R so I can implement some things myself, it doesn't have to be a ready-to-use software package or anything like that. Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="3240" LastActivityDate="2022-08-09T16:21:53.557" Title="Efficiently aligning a lot of reads on the same small reference sequence" Tags="|sequence-alignment|ngs|algorithms|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4914" PostTypeId="2" ParentId="4909" CreationDate="2018-08-20T10:56:03.633" Score="2" Body="&lt;p&gt;Minimap2 is not the most optimal aligner in this case, it was designed for long reads, of which 32bp reads are clearly not. You could try lowering the  mnimizer k-mer length and with &lt;code&gt;-k&lt;/code&gt; the default being 15, also try lowering the minimzer window size with &lt;code&gt;-w&lt;/code&gt;.  However I suggest using &lt;code&gt;bwa aln&lt;/code&gt; and &lt;code&gt;bwa samse&lt;/code&gt; for all reads under 32bp and then merging the alignment later. Although I struggle to see what the utility of these short reads will be against a backdrop of longer reads?&lt;/p&gt;&#xA;" OwnerUserId="1769" LastEditorUserId="1769" LastEditDate="2018-08-20T11:02:40.347" LastActivityDate="2018-08-20T11:02:40.347" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4915" PostTypeId="2" ParentId="4909" CreationDate="2018-08-20T11:48:57.550" Score="3" Body="&lt;p&gt;Minimap2 doesn't work well with 32bp reads by design. Older mappers like bowtie1 and bwa-aln will be better.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-08-20T11:48:57.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4916" PostTypeId="2" ParentId="4911" CreationDate="2018-08-20T12:03:30.090" Score="3" Body="&lt;p&gt;This script is wrong because a quality string may start with &lt;code&gt;@&lt;/code&gt;. With bioawk, it can be simplified to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bioawk -c fastx '{ readlength[length($seq)]++; countread++ } END{...}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;END{}&lt;/code&gt; block is the same as your original version.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: forgot &lt;code&gt;-c fastx&lt;/code&gt; is the original answer. Thank @MattBashton for pointing this out.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2018-08-20T15:49:32.900" LastActivityDate="2018-08-20T15:49:32.900" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4917" PostTypeId="2" ParentId="4913" CreationDate="2018-08-20T14:28:45.690" Score="4" Body="&lt;p&gt;I would suggest using a whole genome aligner like &lt;a href=&quot;http://bio-bwa.sourceforge.net&quot; rel=&quot;nofollow noreferrer&quot;&gt;BWA&lt;/a&gt; or &lt;a href=&quot;http://bowtie-bio.sourceforge.net/bowtie2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BOWTIE2&lt;/a&gt; and swapping out the whole genome reference with your specific reference.&lt;/p&gt;&#xA;&lt;p&gt;These alignment algorithms are designed to align large numbers of NGS reads against a single reference.  They can be used with any reference including short ones like your example or small viruses.&lt;/p&gt;&#xA;&lt;p&gt;To ensure that local alignment is correct, keep in mind that &lt;code&gt;bowtie2&lt;/code&gt; needs the &lt;code&gt;--local&lt;/code&gt; while &lt;code&gt;bwa&lt;/code&gt; should be run with on the &lt;code&gt;BWA-MEM&lt;/code&gt; modality&lt;/p&gt;&#xA;" OwnerUserId="964" LastEditorUserId="15916" LastEditDate="2022-08-09T16:21:53.557" LastActivityDate="2022-08-09T16:21:53.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4918" PostTypeId="2" ParentId="4911" CreationDate="2018-08-20T14:43:26.193" Score="8" Body="&lt;p&gt;This can also be done with regular awk.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{if(NR%4==2) {count++; bases += length} } END{print bases/count}' &amp;lt;fastq_file&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;NR%4==2&lt;/code&gt; count the second line out of every block of 4.  &lt;code&gt;length&lt;/code&gt; is a built-in that defaults to the length of the line, same as &lt;code&gt;length($0)&lt;/code&gt;.  In this case you can inject you custom printing to the &lt;code&gt;END{}&lt;/code&gt; block but &lt;code&gt;countread&lt;/code&gt; and &lt;code&gt;countstored&lt;/code&gt; will always be the same since the averaging is done on the fly.&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2018-08-20T14:43:26.193" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4919" PostTypeId="1" CreationDate="2018-08-20T16:06:08.863" Score="1" ViewCount="513" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0IbOf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0IbOf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;you can see in my feature plot I can only see the cells from which sample but I cannot see the legend. And if I change the command I can see the legend but cannot see the sample identities.&lt;code&gt;FeaturePlot(object = VarMerge, features.plot = c(&quot;Hcn4&quot;), do.hover = T, no.legend = F,)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if I change the command to &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FeaturePlot(object = VarMerge, features.plot = c(&quot;Hcn4&quot;), no.legend = F&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;it will show like the second picture. So I just want to know whether in the first picture can I do something to the command and add the legend like in second picture?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4VIk5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4VIk5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3171" LastEditorUserId="73" LastEditDate="2021-06-07T01:22:17.960" LastActivityDate="2021-06-07T01:22:17.960" Title="How to create a legend in FeaturePlot with do.hover at the same time?" Tags="|scrnaseq|seurat|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4921" PostTypeId="2" ParentId="4894" CreationDate="2018-08-21T10:21:06.363" Score="1" Body="&lt;p&gt;Based on the comments of the question I thought to put together this answer using &lt;a href=&quot;https://www.gnu.org/software/coreutils/coreutils.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GNU coreutils&lt;/a&gt; which should exist on pretty much all systems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is to paste the 2 files together with &lt;code&gt;paste&lt;/code&gt; (assuming they are in the same order and each line correspond to the same line in the other file).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since only the sequence and quality of both files should be kept, the lines starting with &lt;code&gt;@&lt;/code&gt; and &lt;code&gt;+&lt;/code&gt; need to be made empty for one of the files. This can be achived with sed, e.g. &lt;code&gt;sed 's/^@.*//'&lt;/code&gt; empties all lines starting with &lt;code&gt;@&lt;/code&gt; but keeps the empty line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To modify the second file on the fly, one can using &lt;a href=&quot;http://tldp.org/LDP/abs/html/process-sub.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;process substitution&lt;/a&gt; &lt;code&gt;&amp;lt;()&lt;/code&gt;. &#xA;Everything strung together should be something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;file1=File_A.fastq&#xA;file2=File_B.fastq&#xA;paste -d &quot;&quot; $file1 &amp;lt;(cat $file2 | sed 's/^@.*//' | sed 's/^+.*//') &amp;gt; target.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I haven't tested it thoroughly, but could be a good starting point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a lot of files to paste, you might however want to process them first (not on the fly) using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for file in $(ls *.fastq); do &#xA;  cat $file | sed 's/^@.*//' | sed 's/^+.*//' &amp;gt; ${file%.fastq}_cleaned.fastq&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This creates &lt;code&gt;File_A_mod.fastq&lt;/code&gt; from &lt;code&gt;File_A.fastq&lt;/code&gt; etc. &#xA;Best to move them in a separate directory (make sure the first one is the original instead of the _cleaned one since it serves as the base).&#xA;You should have the following in the folder: &lt;code&gt;File_A.fastq File_B_cleaned.fastq File_C_cleaned.fastq ...&lt;/code&gt;. &#xA;Those can be than pasted with &lt;code&gt;paste -d &quot;&quot; *.fastq &amp;gt; target.fastq&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1461" LastEditorUserId="1461" LastEditDate="2018-08-21T11:49:46.720" LastActivityDate="2018-08-21T11:49:46.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4922" PostTypeId="1" AcceptedAnswerId="4926" CreationDate="2018-08-21T14:18:18.517" Score="5" ViewCount="325" Body="&lt;p&gt;I'm trying to assemble a bivalve transcriptome. Since bivalves are filter feeders, their transcriptomes tend to be highly contaminated by bacteria, algae and whatnot. Since I pooled several transcriptomes, I have a high amount of reads (&gt;2B reads). I also have a not-so-high quality reference genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I'm not sure when to perform a BLAST-based filter on these reads:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Do I filter reads with BLAST before assembling them?&lt;/li&gt;&#xA;&lt;li&gt;Do I assemble the transcriptome and then filter out contaminant contigs?&lt;/li&gt;&#xA;&lt;li&gt;Since I have a reference genome, do I simply not care about contaminant reads because they will not align to it? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Having this amount of reads, I'd be inclined to first filter out those reads that don't have a match against Metazoa, and then to perform a second filter after assembly in order to retain only those contigs that match against Lophotrochozoa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you know resources on how to manage read contamination, feel free to share them.&lt;/p&gt;&#xA;" OwnerUserId="1637" LastActivityDate="2018-08-23T15:54:56.220" Title="At what stage of a transcriptome assembly is it better to perform read contaminant filter?" Tags="|transcriptome|reference-genome|read-correction|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4923" PostTypeId="1" AcceptedAnswerId="4928" CreationDate="2018-08-21T16:24:17.400" Score="3" ViewCount="1419" Body="&lt;p&gt;I am heatmaping a list of genes by DoHeatmap function in Seurat R package. I am sure I have 212 genes but heat map shows only a few of my genes&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; DoHeatmap(&#xA;+        object = seurat,&#xA;+        genes.use = genes,&#xA;+        slim.col.label = TRUE, &#xA;+        remove.key = TRUE)&#xA;&amp;gt; dim(as.matrix(seurat@data))&#xA;[1] 12293   209&#xA;&#xA;&amp;gt; length(genes)&#xA;[1] 212&#xA;&amp;gt; class(genes)&#xA;[1] &quot;character&quot;&#xA;&#xA;&amp;gt; class(seurat)&#xA;[1] &quot;seurat&quot;&#xA;attr(,&quot;package&quot;)&#xA;[1] &quot;Seurat&quot;&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Please look at heat map, I was trying to plot 212 genes but you seeing only few genes are among my 212 genes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please somebody save me from this confusion&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2018-08-22T16:08:41.487" LastActivityDate="2018-08-22T16:08:41.487" Title="Why DoHeatmap Does not show all genes in genes.use?" Tags="|r|scrnaseq|seurat|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4924" PostTypeId="1" CreationDate="2018-08-21T19:15:26.427" Score="3" ViewCount="65" Body="&lt;p&gt;Why do some single-cell RNA-seq papers give the number of cells sampled &lt;em&gt;as a fraction of the size of the organ, animal, or embryo&lt;/em&gt;, as opposed to just saying how many cells they sampled? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, Wagner et al.'s paper &quot;Single-cell mapping of gene expression landscapes and lineage in the zebrafish embryo&quot; says &quot;For different developmental stages, we sampled 0.17x to 0.97x of the total cells per embryo, sufficient to detect cell states as rare as 0.1-0.5% of all cells.&quot; A pollster would never write &quot;We sampled 4 out of every 10,000 registered voters in the USA&quot;; they'd just give the number. Statistical results on cluster detection (e.g. &lt;a href=&quot;http://people.csail.mit.edu/moitra/docs/disentangling.pdf&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Disentangling Gaussians by Adam Tauman Kalai, Ankur Moitra, and Gregory Valiant&quot;&gt;Disentangling Gaussians by Adam Tauman Kalai, Ankur Moitra, and Gregory Valiant&lt;/a&gt;) are also phrased in terms of number of samples, not this ratio of sample size to organism size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's worth mentioning that these studies use low-yield techniques. In order to sequence 0.97 times the number of cells in a zebrafish embryo, Wagner et al. had to sequence at least 50-100 embryos. So, it's more like sampling 97 of 100 cells with replacement and less like sampling 97 of 100 cells without replacement.&lt;/p&gt;&#xA;" OwnerUserId="1708" LastEditorUserId="1708" LastEditDate="2018-08-23T17:34:30.367" LastActivityDate="2018-10-22T18:01:12.960" Title="Why is cell count stated relative to animal size?" Tags="|single-cell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4925" PostTypeId="2" ParentId="4924" CreationDate="2018-08-21T22:57:07.343" Score="1" Body="&lt;p&gt;The typical goal of scRNA-seq is to assess the possible cell-types and states that exist in an organism/tissue/etc. Given that, it can be useful to know what portion of the possible cells have been assayed, since that sets bounds on the number and size of cell populations &lt;strong&gt;not&lt;/strong&gt; assayed. If the goal of a study is to assess the variability in a single cell-type, then you're correct that mentioning &quot;0.8% of the population was sequenced&quot; isn't actually informative. Note that &quot;in a single cell-type&quot; is important there, since if one is assaying multiple cell-types then it's important that enough of each are assayed given their prevalence in the relevant population.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Practically speaking, numbers like this are only ever written in papers if they look impressive, so no one is going to write about sequencing 0.000001% of the cells in an organism, since the reader/reviewer will then start to question the relevance of the results.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-08-21T22:57:07.343" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="4926" PostTypeId="2" ParentId="4922" CreationDate="2018-08-22T00:22:19.983" Score="3" Body="&lt;p&gt;First, to answer your question about mapping to a low-quality reference:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;1. Mapping&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;For mapping, low genome contiguity (low N50) doesn't really matter. You will be using a spliced aligner and short reads, so even small portions of your reads will map if they match the reference genome. What does matter though is that you are confident that the draft genome assembly does not contain any scaffolds from contaminating species like bacteria, algae, et cetera.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;2. Reasons to not BLAST your raw reads&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;There are a few reasons that I do not suggest starting with BLAST, but other options may better suit your needs.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;BLASTing 2 billion reads will take several weeks to months even on a machine with many cores (90+).&lt;/li&gt;&#xA;&lt;li&gt;Since you're not working with a model organism, there will be many reads that do not have a BLAST hit but still are probably derived from your bivalve. I do not suggest using BLAST to filter your reads for that reason.&lt;/li&gt;&#xA;&lt;li&gt;Similarly, many contaminating reads will not hit to anything in NCBI databases, so you will miss many of the contaminants as well. Leaving them in for now and removing with the strategy I describe in &lt;strong&gt;#4&lt;/strong&gt;, then reassembling, may work better.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h3&gt;3. Using multiple individuals of highly heterozygous species in assembly.&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Most importantly for non-model organisms and highly heterozygous species - &lt;strong&gt;do not assemble a transcriptome using data from more than one individual&lt;/strong&gt;. The number of allelic differences in highly heterozygous species will make the transcript graphs a jumbled mess and you'll end up with a bunch of partial and erroneous transcripts. You can merge transcriptomes from multiple individuals later with tools like &lt;a href=&quot;https://github.com/davidemms/OrthoFinder&quot; rel=&quot;nofollow noreferrer&quot;&gt;orthoFinder&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;4. A strategy to filter transcripts based on mapping and BLAST&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Given &lt;strong&gt;#1&lt;/strong&gt;, &lt;strong&gt;#2&lt;/strong&gt;, and &lt;strong&gt;#3&lt;/strong&gt; above, I propose the following strategy to filter your reads that may prevent removing true bivalve reads and will guarantee removal of reads from known contaminating species.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would try this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Assemble the transcriptome of each individual separately.&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This avoids misassemblies due to heterozygous sites and over-filtering by Trinity or other assemblers.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Concatenate all of the transcriptomes into one file. From this point on we will use the information from all individuals to help filter. &#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use somthing like &lt;code&gt;cat txome_individual1.fasta txome_individual2.fasta &amp;gt; all_individuals.fasta&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Map all the transcripts to the reference genome s using a spliced aligner like &lt;a href=&quot;https://github.com/alexdobin/STAR&quot; rel=&quot;nofollow noreferrer&quot;&gt;STAR&lt;/a&gt; or &lt;a href=&quot;https://ccb.jhu.edu/software/hisat2/index.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;HISAT2&lt;/a&gt;. &lt;/li&gt;&#xA;&lt;li&gt;Get a bam file of all the transcripts that didn't map to the reference genome. These may be from contaminants. &#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use &lt;code&gt;samtools view -f 4 -b mapped_transcripts.bam | samtools sort - &amp;gt; output.sorted.bam&lt;/code&gt; to only keep the transcripts that &lt;strong&gt;did not&lt;/strong&gt; mapped to the reference. &lt;code&gt;-f 4&lt;/code&gt; is the magic flag here that keeps things that didn't map.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Make a list of all the transcripts that &lt;strong&gt;did not&lt;/strong&gt; map to the reference...&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;samtools view output.sorted.bam | cut -f1 | sort | uniq &amp;gt; unmapped_transcripts.txt&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;...and get only the transcripts from your fasta file that are potential contaminants.&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;seqtk subseq all_transcripts.fa unmapped_transcripts.txt &amp;gt; unmapped.fasta&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;BLAST all of the &lt;code&gt;unmapped.fasta&lt;/code&gt; and identify which ones are from contaminating species. &lt;a href=&quot;https://blobtools.readme.io/docs&quot; rel=&quot;nofollow noreferrer&quot;&gt;blobtools&lt;/a&gt; works well for this if there are too many to keep track of.&lt;/li&gt;&#xA;&lt;li&gt;Use seqtk again to only get the known contaminating seqs from BLAST or blobtools. Let's say you call this &lt;code&gt;known_contaminants.fasta&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Map &lt;strong&gt;all reads&lt;/strong&gt; from &lt;strong&gt;all individuals&lt;/strong&gt; to the &lt;code&gt;known_contaminants.fasta&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Get all the reads that map to known contaminating transcripts, &lt;code&gt;known_contaminants.fasta&lt;/code&gt; using:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;samtools view -F 4 all_reads_to_contams.bam | cut -f1 | sort | uniq &amp;gt; contaminating_reads.txt&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Now remove all of the contaminating reads from your individual fastq files using &lt;a href=&quot;https://www.biostars.org/p/127141/#127143&quot; rel=&quot;nofollow noreferrer&quot;&gt;bbmap filterbyname.sh&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Reassemble the filtered fastq files.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This strategy has the following benefits:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does not require weeks/months of compute time to BLAST 2 billion reads.&lt;/li&gt;&#xA;&lt;li&gt;Ensures that all reads from the reference genome are left untouched.&lt;/li&gt;&#xA;&lt;li&gt;Ensures that all reads from known contaminants are removed.&lt;/li&gt;&#xA;&lt;li&gt;Leaves in reads with no hits to BLAST, which could be from unassembled parts of the reference genome.&lt;/li&gt;&#xA;&lt;li&gt;Avoids potentially filtering out low-abundance contaminants by not inputting all 2 billion reads to the transcriptome assembler, but assembling on a per-individual basis.&lt;/li&gt;&#xA;&lt;li&gt;Leverages contaminants from different individuals to filter reads from all individuals.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-08-23T15:54:56.220" LastActivityDate="2018-08-23T15:54:56.220" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4927" PostTypeId="1" CreationDate="2018-08-22T01:09:17.640" Score="1" ViewCount="73" Body="&lt;p&gt;I have Illumina and PacBio reads and I would like to use &lt;a href=&quot;https://github.com/yechengxi/DBG2OLC&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbg2olc&lt;/a&gt; for hybrid assembly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Part of dbg2olc is &lt;code&gt;SelectLongestReads&lt;/code&gt; which select reads that sum to total_length of bases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How many bases do I need for a 1.8G allotetraploid to remain?&lt;/p&gt;&#xA;" OwnerUserId="2477" LastEditorUserId="48" LastEditDate="2018-08-23T14:06:24.277" LastActivityDate="2019-04-09T22:28:23.627" Title="How many reads do I need for hybrid assembly" Tags="|assembly|illumina|pacbio|coverage|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4928" PostTypeId="2" ParentId="4923" CreationDate="2018-08-22T07:26:29.817" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/satijalab/seurat/issues/287&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/satijalab/seurat/issues/287&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this post there is a answer for my question, however when I set scale to FALSE again the number of genes did not change but the shape of plot dramatically changed. So, maybe the issue is because of scaling that did not solve my problem&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2018-08-22T07:26:29.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4929" PostTypeId="1" AcceptedAnswerId="4969" CreationDate="2018-08-22T16:31:53.497" Score="6" ViewCount="247" Body="&lt;p&gt;I have single cell RNA-seq data on about 2000 cells in 9 time point. I have clustered my cells in each time point by Seurat. I am seeing in some time points I have 3 clusters while in another time points I have 2 clusters of cells (please look at the plot).&lt;/p&gt;&#xA;&lt;p&gt;One hypothesis would be that, a new cell type is being emerged at last time point. But maybe if we had another further time point this third cluster being disappeared because that has been arose just because of undifferentiated cells from the later time point (second hypothesis).&lt;br /&gt;&#xA;Another hypothesis would be this new third cluster is a new cells type that only temporarily expresses a set of genes but if we had cells for another further time point, we would not see the expression of these genes anymore and we had again 2 clusters. I am really struggling with this how I could test my hypothesises.&lt;br /&gt;&#xA;The final hypothesis would be that the third cluster is really a new cell type born from old cell types during the differentiation.&lt;/p&gt;&#xA;&lt;p&gt;Like UAD algorithm addresses my question. I just tried FateID algorithm on my data but got stopped by&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; pr &amp;lt;- prcurve(y,fb,dr,k=2,m=&amp;quot;tsne&amp;quot;,trthr=0.4,start=2)&#xA;Error in svd(xstar) : a dimension is zero&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any suggestion would be highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="73" LastEditDate="2021-05-23T12:29:19.603" LastActivityDate="2021-05-23T12:29:19.603" Title="How I can test my hypothesises computationally" Tags="|r|scrnaseq|seurat|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4931" PostTypeId="2" ParentId="4929" CreationDate="2018-08-23T07:02:26.960" Score="3" Body="&lt;p&gt;This is what packages like &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/29630061&quot; rel=&quot;nofollow noreferrer&quot;&gt;FateID&lt;/a&gt; and &lt;a href=&quot;http://cole-trapnell-lab.github.io/monocle-release/docs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monocle&lt;/a&gt; are for, namely taking single-cell RNAseq data and inferring differentiation trajectories from it. Don't try to reinvent the wheel on this, there are a number of packages out there to do this sort of thing and they're going to have pretty complicated methods that you're really not going to want to reinvent unless you absolutely have to. &lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-08-23T07:02:26.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4933" PostTypeId="1" AcceptedAnswerId="4935" CreationDate="2018-08-23T16:13:47.673" Score="4" ViewCount="481" Body="&lt;p&gt;I'm interested in all proteins that are in any way associated with &lt;em&gt;Danio rerio&lt;/em&gt;. I decided to look them up at Pfam data base and when I just make a keyword search, I get a a nice list which looks like this &lt;a href=&quot;http://pfam.xfam.org/search/keyword?query=danio%20rerio&amp;amp;submit=Submit&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://pfam.xfam.org/search/keyword?query=danio+rerio&amp;amp;submit=Submit&lt;/a&gt;. &#xA;Important thing to note is that most of the associations of these protein families can be found in &lt;strong&gt;seq_info&lt;/strong&gt; column.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: how to extract that list from a python script? Moreover, I want to make a script that when run, will fetch the latest release, filter out the proteins that cannot be found in &lt;em&gt;D. rerio&lt;/em&gt; and do some additional processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been searching for ways to directly download files from Pfam ftp server and mine for the proteins that are associated with &lt;em&gt;D. rerio&lt;/em&gt;, however, I didn't find a single way to do it. I simply cannot see where do they store the information about species in which the protein can be found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to retrieve everything about protein families of interest, because the next step I will need to do is to find the DNA binding proteins and for finding a way to filter out the ones of no interest, I would liketo have as much data as possible. &lt;/p&gt;&#xA;" OwnerUserId="2960" LastActivityDate="2018-08-24T11:53:48.357" Title="Get results of keyword search on Pfam via python script" Tags="|python|public-databases|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4935" PostTypeId="2" ParentId="4933" CreationDate="2018-08-24T11:48:20.223" Score="1" Body="&lt;p&gt;If I understand your question correctly, then you can use the Pfam proteom search to get the data you are looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Go to the &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Pfam website -&gt; Browse -&gt; select the letter D from proteoms category&#xA;  -&gt; D. rerio&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There you can find a download link under the section &lt;code&gt;Domain composition&lt;/code&gt;: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;You can download a list of all regions for this proteome from our FTP&#xA;  site.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The download link seems to be bound to the species ID and if you want to automate your data retrieval you may be able to simply download the file from there in your python script.&lt;/p&gt;&#xA;" OwnerUserId="1786" LastEditorUserId="1786" LastEditDate="2018-08-24T11:53:48.357" LastActivityDate="2018-08-24T11:53:48.357" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4936" PostTypeId="1" AcceptedAnswerId="4948" CreationDate="2018-08-24T16:30:21.467" Score="-1" ViewCount="113" Body="&lt;p&gt;I am using URD R package on my single cell RNA-seq data hoping to be able to trace gene expression changes during the development. After dimension reduction step by PCA and diffusion map whatever I am trying to cluster some cells, but whatever I do I am getting this error &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; object &amp;lt;- graphClustering(object, num.nn = c(5,8,10,15), method=&quot;Louvain&quot;, do.jaccard = TRUE)&#xA;Error in graphClustering(object, num.nn = c(5, 8, 10, 15), method = &quot;Louvain&quot;,  : &#xA;  unused arguments (num.nn = c(5, 8, 10, 15), method = &quot;Louvain&quot;, do.jaccard = TRUE)&#xA;&#xA;&#xA;&amp;gt; print(object.6s) &#xA;URD object: 12451 genes x 1556 cells. &#xA;&#xA;&#xA;object.6s.mnn &amp;lt;- graphClustering(object.6s) &#xA;Show Traceback Rerun with Debug Error in (function (classes, fdef, mtable) : unable to find an inherited method for function ‘graphClustering’ for signature ‘&quot;URD&quot;’&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried to change Louvain, jaccard, ... but I am still getting this error. How can I get rid of it?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="57" LastEditDate="2018-08-26T19:54:16.230" LastActivityDate="2018-08-28T11:53:31.180" Title="A strange error while clustering of single cell RNA-seq using URD package" Tags="|r|scrnaseq|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="4937" PostTypeId="1" AcceptedAnswerId="4939" CreationDate="2018-08-24T18:00:34.187" Score="8" ViewCount="668" Body="&lt;p&gt;&lt;a href=&quot;https://reactome.org/&quot; rel=&quot;noreferrer&quot;&gt;Reactome&lt;/a&gt; uses a three letter code for each species which are listed below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are the letter codes used by Reactome a standard?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so is there a web page or download file that can validate the code and translate it to a common name?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; HSA  Homo sapiens                          &#xA; SSC  Sus scrofa                            &#xA; GGA  Gallus gallus                         &#xA; DME  Drosophila melanogaster               &#xA; RNO  Rattus norvegicus                     &#xA; DDI  Dictyostelium discoideum              &#xA; MMU  Mus musculus                          &#xA; ATH  Arabidopsis thaliana                  &#xA; TGU  Taeniopygia guttata                   &#xA; DRE  Danio rerio                           &#xA; OSA  Oryza sativa                          &#xA; BTA  Bos taurus                            &#xA; XTR  Xenopus tropicalis                    &#xA; CFA  Canis familiaris                      &#xA; SCE  Saccharomyces cerevisiae              &#xA; PFA  Plasmodium falciparum                 &#xA; CEL  Caenorhabditis elegans                &#xA; SPO  Schizosaccharomyces pombe             &#xA; ROT  Rotavirus                             &#xA; FLU  Influenza A virus                     &#xA; HCV  Hepatitis C Virus                     &#xA; HBV  Hepatitis B virus                     &#xA; HER  Human herpesvirus 1                   &#xA; NME  Neisseria meningitidis serogroup B    &#xA; NME  Neisseria meningitidis                &#xA; CTR  Chlamydia trachomatis                 &#xA; STY  Salmonella typhimurium                &#xA; STY  Salmonella enterica                   &#xA; ECO  Escherichia coli                      &#xA; OCU  Oryctolagus cuniculus                 &#xA; XLA  Xenopus laevis                        &#xA; CAE  Cercopithecus aethiops                &#xA; CGR  Cricetulus griseus                    &#xA; HER  Human herpesvirus 8                   &#xA; MCV  Molluscum contagiosum virus subtype 1 &#xA; MCV  Molluscum contagiosum virus           &#xA; HCY  Human cytomegalovirus                 &#xA; HIV  Human immunodeficiency virus 1        &#xA; PCH  Penicillium chrysogenum               &#xA; TAE  Triticum aestivum                     &#xA; VRA  Vigna radiata var. radiata            &#xA; VRA  Vigna radiata                         &#xA; CFS  Crithidia fasciculata                 &#xA; FCA  Felis catus                           &#xA; AMA  Arenicola marina                      &#xA; MGA  Meleagris gallopavo                   &#xA; NGO  Neisseria gonorrhoeae                 &#xA; SAU  Staphylococcus aureus                 &#xA; MVI  Measles virus                         &#xA; HAL  Human alphaherpesvirus 2          &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not a complete answer as this only covers a subset of all organisms but at present better than nothing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using KEGG Organisms&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.genome.jp/kegg-bin/get_htext&quot; rel=&quot;noreferrer&quot;&gt;https://www.genome.jp/kegg-bin/get_htext&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Enter letter code, e.g. GGA&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AOpHS.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AOpHS.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT - After reading &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/4939/1586&quot;&gt;answer&lt;/a&gt; by &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/2085/conchoecia&quot;&gt;conchoecia&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The graph database schema for Reactome only has two labels that use the 3 letter code which is named abbreviation in the schema:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://reactome.org/content/schema/Species&quot; rel=&quot;noreferrer&quot;&gt;Species&lt;/a&gt; and &lt;a href=&quot;https://reactome.org/content/schema/ReferenceTherapeutic&quot; rel=&quot;noreferrer&quot;&gt;ReferenceTherapeutic&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Species has taxid  (Taxon Identifier)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;abbreviation     taxId       displayName                            &#xA;AMA              6344        Arenicola marina                        &#xA;ATH              3702        Arabidopsis thaliana                    &#xA;BAN              1392        Bacillus anthracis                      &#xA;BTA              9913        Bos taurus                              &#xA;CAE              9534        Cercopithecus aethiops                  &#xA;CAL              5476        Candida albicans                        &#xA;CBO              1491        Clostridium botulinum                   &#xA;CEL              6239        Caenorhabditis elegans                  &#xA;CFA              9615        Canis familiaris                        &#xA;CFS              5656        Crithidia fasciculata                   &#xA;CGR              10029       Cricetulus griseus                      &#xA;CPH              10703       Corynephage beta                        &#xA;CPO              10141       Cavia porcellus                         &#xA;CSB              60711       Chlorocebus sabaeus                     &#xA;CTE              1513        Clostridium tetani                      &#xA;CTR              813         Chlamydia trachomatis                   &#xA;CVI              10243       Cowpox virus                            &#xA;DDI              44689       Dictyostelium discoideum                &#xA;DME              7227        Drosophila melanogaster                 &#xA;DRE              7955        Danio rerio                             &#xA;ECO              562         Escherichia coli                        &#xA;FCA              9685        Felis catus                             &#xA;FLU              11320       Influenza A virus                       &#xA;GGA              9031        Gallus gallus                           &#xA;HAL              10310       Human alphaherpesvirus 2                &#xA;HAM              6706        Homarus americanus                      &#xA;HBV              10407       Hepatitis B virus                       &#xA;HCV              11103       Hepatitis C Virus                       &#xA;HCV              31649       Hepatitis C virus genotype 2a           &#xA;HCV              31646       Hepatitis C virus subtype 1a            &#xA;HCY              10359       Human cytomegalovirus                   &#xA;HER              10298       Human herpesvirus 1                     &#xA;HER              37296       Human herpesvirus 8                     &#xA;HGA              10376       Human gammaherpesvirus 4                &#xA;HIV              11676       Human immunodeficiency virus 1          &#xA;HPV              333760      Human papillomavirus type 16            &#xA;HPV              337041      Alphapapillomavirus 9                   &#xA;HSA              9606        Homo sapiens                            &#xA;IBR              11120       Infectious bronchitis virus             &#xA;LMO              1639        Listeria monocytogenes                  &#xA;LMO              1906951     Listeria monocytogenes serovar 1/2a     &#xA;LPN              446         Legionella pneumophila                  &#xA;MCV              10280       Molluscum contagiosum virus subtype 1   &#xA;MCV              10279       Molluscum contagiosum virus             &#xA;MGA              9013        Meleagris gallopavo                     &#xA;MML              9544        Macaca mulatta                          &#xA;MMU              10090       Mus musculus                            &#xA;MTU              1773        Mycobacterium tuberculosis              &#xA;MTU              83332       Mycobacterium tuberculosis H37Rv        &#xA;MVI              11234       Measles virus                           &#xA;NGO              485         Neisseria gonorrhoeae                   &#xA;NME              491         Neisseria meningitidis serogroup B      &#xA;NME              487         Neisseria meningitidis                  &#xA;OAR              9940        Ovis aries                              &#xA;OCU              9986        Oryctolagus cuniculus                   &#xA;OSA              4530        Oryza sativa                            &#xA;PCH              5076        Penicillium chrysogenum                 &#xA;PFA              5833        Plasmodium falciparum                   &#xA;RNO              10116       Rattus norvegicus                       &#xA;ROT              10912       Rotavirus                               &#xA;SAU              1280        Staphylococcus aureus                   &#xA;SCE              4932        Saccharomyces cerevisiae                &#xA;SEV              11191       Sendai virus                            &#xA;SPO              4896        Schizosaccharomyces pombe               &#xA;SSC              9823        Sus scrofa                              &#xA;STY              90371       Salmonella typhimurium                  &#xA;STY              28901       Salmonella enterica                     &#xA;TAE              4565        Triticum aestivum                       &#xA;TGO              5811        Toxoplasma gondii                       &#xA;TGU              59729       Taeniopygia guttata                     &#xA;VRA              3916        Vigna radiata var. radiata              &#xA;VRA              157791      Vigna radiata                           &#xA;VVI              10245       Vaccinia virus                          &#xA;XLA              8355        Xenopus laevis                          &#xA;XTR              8364        Xenopus tropicalis &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;While ReferenceTherapeutic does not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The species can be identified using the NCBI taxon identifier, e.g.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?id=6344&quot; rel=&quot;noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?id=6344&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oVwcY.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oVwcY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT - After reading &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/4940/1586&quot;&gt;answer&lt;/a&gt; by &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/37/user172818&quot;&gt;user172818&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Searching the file &lt;a href=&quot;https://www.uniprot.org/docs/speclist.txt&quot; rel=&quot;noreferrer&quot;&gt;https://www.uniprot.org/docs/speclist.txt&lt;/a&gt; for 6344&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AREMA E    6344: N=Arenicola marina  &#xA;                 C=Lugworm  &#xA;                 S=Lumbricus marinus &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or using the URL: &lt;a href=&quot;https://www.uniprot.org/taxonomy/6344&quot; rel=&quot;noreferrer&quot;&gt;https://www.uniprot.org/taxonomy/6344&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WpFQg.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WpFQg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and with reference to&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I think databases should stop inventing new species abbreviations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The Wikipedia article on &lt;a href=&quot;https://en.wikipedia.org/wiki/Lugworm&quot; rel=&quot;noreferrer&quot;&gt;Lugworm&lt;/a&gt; shows the many &lt;a href=&quot;https://en.wikipedia.org/wiki/Help:Taxon_identifiers&quot; rel=&quot;noreferrer&quot;&gt;Taxon identifiers&lt;/a&gt; in multiple databases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VWqBa.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VWqBa.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1586" LastEditorUserId="1586" LastEditDate="2018-08-25T11:43:18.943" LastActivityDate="2018-08-25T11:43:18.943" Title="Are these standard species abbreviations and how to look up others?" Tags="|database|data-mining|taxonomy|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4939" PostTypeId="2" ParentId="4937" CreationDate="2018-08-25T06:30:00.437" Score="7" Body="&lt;p&gt;No, there are no standardized three-letter abbreviations for species names. Three-letter abbreviations with roman letters is 26^3 combinations, which is only 17,576 possible combinations. As a result there are not enough three-letter abbreviations for all living species.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All three-letter abbreviations are either specific to a database or perhaps are shared between a database or two. There is currently no substitute for a binomial name (although NCBI taxid is a convenient parsable substitute for the species included in NCBI to date).&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-08-25T06:30:00.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4940" PostTypeId="2" ParentId="4937" CreationDate="2018-08-25T07:31:10.550" Score="6" Body="&lt;p&gt;These are not. The closest to &quot;standard&quot; is the 5-character &lt;a href=&quot;https://www.uniprot.org/docs/speclist.txt&quot; rel=&quot;noreferrer&quot;&gt;abbreviation by Swissprot&lt;/a&gt;. It names 25,886 species and has been used for decades. It is also easy to remember for some common species such as HUMAN, RAT, MOUSE and HORSE. I think databases should stop inventing new species abbreviations.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-08-25T07:31:10.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4941" PostTypeId="2" ParentId="4927" CreationDate="2018-08-25T11:06:01.750" Score="1" Body="&lt;p&gt;I have never used dbg2olc. However other OLC assemblers managed to do a solid assembly from 20x of corrected long reads. But obviously, the more the better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In you case of allotetraploid, you need to figure out if the heterozygosity is sufficient to separate haplotypes of not and adjust parameters of the assembler. For instance in &lt;a href=&quot;http://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.2004830&quot; rel=&quot;nofollow noreferrer&quot;&gt;bdelloid rotifers&lt;/a&gt; the tetraploid genome has an AA'BB' structure, all the assemblies were able to separate A and B, but not A and A' and B and B'. I guess an allotetraploid could have a similar structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be good to know what you are dealing with and what is your goal in the assembly. Do you want a single haplotype?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- edit --&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can check the genome structure using short reads by &lt;a href=&quot;https://github.com/KamilSJaron/smudgeplot&quot; rel=&quot;nofollow noreferrer&quot;&gt;smudgeplot&lt;/a&gt;, a tool I made (yes, it's a shameless plug).&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2019-04-09T22:28:23.627" LastActivityDate="2019-04-09T22:28:23.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4942" PostTypeId="1" AcceptedAnswerId="4944" CreationDate="2018-08-26T05:08:23.060" Score="6" ViewCount="5168" Body="&lt;p&gt;I want to find the length of a list of genes, of Homo sapiens, that is reported in the GEO database. I have gathered the ensembl id's of those genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand this information can be parsed from the start and end position given in GTF file available in ensembl.org. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to ask for advice on how to programmatically download the gtf files with ensembl id's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the ensembl webpage a couple of options are listed, API and ftp access. I would like to know if sample scripts are available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: I found a script &lt;a href=&quot;https://github.com/dpryan79/Answers/blob/master/SEQanswers_42420/GTF2LengthGC.R&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, it shows how to parse the data from gtf file. Could someone help in understanding how the code can be modified to obtain gene length from start and end position?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit 1: &#xA;By start and end position, I refer to the start and end values given in the &quot;fields&quot; section &lt;a href=&quot;https://asia.ensembl.org/info/website/upload/gff.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; and the file from which I want to parse the values is the second last file available in this &lt;a href=&quot;ftp://ftp.ensembl.org/pub/current_gtf/homo_sapiens/&quot; rel=&quot;noreferrer&quot;&gt;link&lt;/a&gt;.&lt;br&gt;&#xA;For instance, the end - start value of the gene with ensembl id ENSG00000131482,&#xA;gene :G6PC&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit 2: I get the gene names and the expression values from the SOFT files documented in GEO, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sites/GDSbrowser?acc=GDS4027&quot; rel=&quot;noreferrer&quot;&gt;link&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is a sample output from SOFT file. It gives me the probe_set(ID_REF) , Gene name and expression values from samples (GSMXXXXXX). Here we see fro G6PC, values are reported from two probe_sets. Can we get the transcript id's from the probe_sets?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obtaining the transcript id will help me in finding the length of the specific transcript. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; Table(gds)[2449,0:5]&#xA;           ID_REF IDENTIFIER GSM388749 GSM388750 GSM388753&#xA;2449 1555612_s_at       G6PC   4.39472   4.52235   4.26869&#xA;&amp;gt; Table(gds)[16399,0:5]&#xA;         ID_REF IDENTIFIER GSM388749 GSM388750 GSM388753&#xA;16399 206952_at       G6PC   2.76693   3.45107    2.9815&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3186" LastEditorUserId="3186" LastEditDate="2018-08-26T13:54:20.003" LastActivityDate="2020-03-04T13:12:37.890" Title="Finding gene length using ensembl ID" Tags="|gene|ensembl|" AnswerCount="4" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="4944" PostTypeId="2" ParentId="4942" CreationDate="2018-08-26T13:19:10.677" Score="3" Body="&lt;p&gt;If you really just want the start and end positions of all genes in a list of genes from &lt;a href=&quot;ftp://ftp.ensembl.org/pub/current_gtf/homo_sapiens/Homo_sapiens.GRCh38.93.gtf.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;Homo_sapiens.GRCh38.93.gtf.gz&lt;/a&gt;, you could do this in &lt;code&gt;awk&lt;/code&gt;. For example, if your list of genes is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat genes&#xA;ENSG00000131482&#xA;ENSG00000223972&#xA;ENSG00000228794&#xA;ENSG00000187961&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk '{ &#xA;        if(NR==FNR){&#xA;            wantedGenes[$1]++ &#xA;        } &#xA;        else{&#xA;            if($3==&quot;gene&quot;){&#xA;                gsub(/&quot;;*/,&quot;&quot;,$10); &#xA;                if($10 in wantedGenes){&#xA;                    print $10,$4,$5&#xA;                } &#xA;            }&#xA;        }&#xA;      }' genes Homo_sapiens.GRCh38.93.gtf &#xA;ENSG00000223972 11869 14409&#xA;ENSG00000228794 825138 859446&#xA;ENSG00000187961 960587 965715&#xA;ENSG00000131482 42900797 42913369&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Explanation:&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;if(NR==FNR){ wantedGenes[$1]++ }&lt;/code&gt; : &lt;code&gt;NR&lt;/code&gt; is the input line number and &lt;code&gt;FNR&lt;/code&gt; is the current line number of &lt;em&gt;this&lt;/em&gt; input file. The two are only equal while reading the first file. So, if this is the first file, save the first field of each line (the gene name) in the array &lt;code&gt;wantedGenes&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;if($3==&quot;gene&quot;){&lt;/code&gt; : if this line is a 'gene' entry.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;gsub(/&quot;;*/,&quot;&quot;,$10);&lt;/code&gt; : the gene names in the GTF file are quoted and have an &lt;code&gt;;&lt;/code&gt; after the quote. For example: &lt;code&gt;&quot;ENSG00000187961&quot;;&lt;/code&gt;. Since we don't have this in the list, we need to remove it so we can match the gene name. This &lt;code&gt;gsub&lt;/code&gt; command will remove all &lt;code&gt;&quot;&lt;/code&gt; followed by 0 or more &lt;code&gt;;&lt;/code&gt; (the &lt;code&gt;;*&lt;/code&gt; means 0 or more occurrences of &lt;code&gt;;&lt;/code&gt;) from the 10th field of each line. So, it will replace &lt;code&gt;&quot;ENSG00000187961&quot;;&lt;/code&gt; with &lt;code&gt;ENSG00000187961&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;if($10 in wantedGenes){print $10,$4,$5}&lt;/code&gt; : if the 10th field (the modified gene name from above) is in the &lt;code&gt;wantedGenes&lt;/code&gt; array, then print the 10th field (the gene name), the 4th (start) and the 5th field (end).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can easily modify this to print the length instead of the start and end positions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk '{ &#xA;        if(NR==FNR){&#xA;            wantedGenes[$1]++ &#xA;        } &#xA;        else{&#xA;            if($3==&quot;gene&quot;){&#xA;                gsub(/&quot;;*/,&quot;&quot;,$10); &#xA;                if($10 in wantedGenes){&#xA;                    print $10,$5-$4&#xA;                } &#xA;            }&#xA;        }&#xA;      }' genes Homo_sapiens.GRCh38.93.gtf &#xA;ENSG00000223972 2540&#xA;ENSG00000228794 34308&#xA;ENSG00000187961 5128&#xA;ENSG00000131482 12572&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As requested in chat, here's a python version of the same thing:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;import re&#xA;import sys&#xA;&#xA;wantedGenes = []&#xA;with open(sys.argv[1], mode='r') as f:&#xA;    wantedGenes = f.readlines()&#xA;&#xA;wantedGenes = [ x.rstrip() for x in wantedGenes]&#xA;&#xA;print('# Gene\tstart\tend\tlength')&#xA;with open(sys.argv[2], mode='r') as f:&#xA;    for line in f:&#xA;        if not line.startswith('#'): &#xA;            fields = line.split('\t')&#xA;            annotations = fields[8].split(' ')&#xA;            geneName = re.sub('[&quot;;]', '', annotations[1])&#xA;            if geneName in wantedGenes and fields[2] == 'gene':&#xA;                print('%s\t%s\t%s\t%d' %&#xA;                      (geneName, fields[3],fields[4],&#xA;                       int( int(fields[4]) - int(fields[3]) )&#xA;                      ))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Save the file as &lt;code&gt;foo.py&lt;/code&gt; and run with the file names as arguments:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python foo.py geneList.txt Homo_sapiens.GRCh38.93.gtf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-08-26T15:03:52.113" LastActivityDate="2018-08-26T15:03:52.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4945" PostTypeId="2" ParentId="4942" CreationDate="2018-08-27T14:38:18.427" Score="3" Body="&lt;p&gt;You could also do this in R with the GenomicFeatures library.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(GenomicFeatures)&#xA;&#xA;## make TxDb from GTF file &#xA;txdb &amp;lt;- makeTxDbFromGFF('Homo_sapiens.GRCh38.93.gtf')&#xA;&#xA;## get gene information&#xA;all.genes &amp;lt;- genes(txdb)&#xA;&#xA;## import your list of gene names&#xA;my.genes &amp;lt;- c('ENSG00000141510','ENSG00000184571','ENSG00000011007')&#xA;&#xA;## get the length of each of those genes&#xA;my.genes.lengths &amp;lt;- width(all.genes[my.genes])&#xA;## put the names back on the lengths&#xA;names(my.genes.lengths) &amp;lt;- my.genes&#xA;&#xA;## print lengths&#xA;print(my.genes.lengths)&#xA;&#xA;ENSG00000141510 ENSG00000184571 ENSG00000011007&#xA;      25772           55687           18905&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="383" LastActivityDate="2018-08-27T14:38:18.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4946" PostTypeId="2" ParentId="4942" CreationDate="2018-08-27T18:43:56.030" Score="0" Body="&lt;p&gt;You can do this directly on &lt;a href=&quot;http://useast.ensembl.org/biomart/martview/46d2562c95ab50383e6c3e0e24512ef6&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioMart site from EBI&lt;/a&gt; as well. Using a list of ENSG ids as input, you can get the actual length of each individual transcript, as well as the length of the genomic region coding said transcripts. These are the attributes I used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Gene stable ID&#xA;Transcript stable ID&#xA;Transcript length (including UTRs and CDS)&#xA;Gene start (bp)&#xA;Gene end (bp)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output is a table with the relevant data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Gene ID Transcript ID   Transcript length (including UTRs and CDS)  Gene start (bp) Gene end (bp)&#xA;ENSG00000146648 ENST00000455089 3844    55019021    55211628&#xA;ENSG00000146648 ENST00000342916 2239    55019021    55211628&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can change the build as well if you don't want to use GRCh38.&lt;/p&gt;&#xA;" OwnerUserId="3269" LastActivityDate="2018-08-27T18:43:56.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4948" PostTypeId="2" ParentId="4936" CreationDate="2018-08-28T07:32:26.007" Score="-1" Body="&lt;p&gt;This solved the error&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;object &amp;lt;- URD::graphClustering(object) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;By this function I am telling R to use the graphClustering() function from the URD namespace. I probably had loaded another package with a different graphClustering() function - or defined one myself - which is taking precedence over URD's function.&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2018-08-28T11:53:31.180" LastActivityDate="2018-08-28T11:53:31.180" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4949" PostTypeId="1" CreationDate="2018-08-28T10:53:14.493" Score="0" ViewCount="438" Body="&lt;p&gt;I returned a Seurat object with a diffusion map by &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; seurat_object &amp;lt;- RunDiffusion(seurat_object,genes.use = seurat_object@var.genes)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;as below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NXIKG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NXIKG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, how I could know the order of a cell's projection onto this principal curve representing its predicted progression through the development by fitting a principal curve on my diffusion map?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know Seurat slot for DM1 and DM2 is inside the seurat@dr &lt;/p&gt;&#xA;&#xA;&lt;p&gt;by &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fit=principal_curve(as.matrix(seurat_object@dr$dm@cell.embeddings))&#xA;plot(fit)&#xA;line(fit)&#xA;points(fit)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I draw this graph of my cells&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NB26l.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NB26l.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But how I know the first and last cells on curve? I mean how I know cells from each cluster is being placed where on the curve? By &lt;code&gt;fit$ord&lt;/code&gt; I will obtain orders of cells on the line but how I know the first and end of the line?????&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2018-08-29T13:14:40.640" LastActivityDate="2018-10-21T19:38:24.553" Title="Fitting a principal curve into a diffusion map" Tags="|r|scrnaseq|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4953" PostTypeId="1" AcceptedAnswerId="4956" CreationDate="2018-08-29T06:35:53.690" Score="5" ViewCount="154" Body="&lt;p&gt;I have been using python 3.6 and biopython 1.72 to work with protein data files. I am using a protein sequence file (phylip format), for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    14    678&#xA;Zebrafish  LSSCGVVSGD LISVILPASS LEETQTSSAA AHQTHTDQQA GGSHVSSSSS &#xA;Fugu       LASCGIVSGD LIRVALIRAA LMAADAPDRA DDGGGHSEQA VAMATNQVS- &#xA;Chicken    LASCGIVSGD LIRVALIRAA LMAADAPDRA DDGGGHSEQA VAMATNQVS- &#xA;Zebra      LASYGIVPGD LICLLLEETD AKSSLPPPSS SSPPSLQNGR ETSTLTPKNQ&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to manipulate the file in a way so that;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I may get the number of columns which have all the same letters (amino acids)&lt;/li&gt;&#xA;&lt;li&gt;The number of columns which have at least 2 different amino acids (with no dashes).&lt;/li&gt;&#xA;&lt;li&gt;The number of columns which have any single dash.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I have iterated over the columns of the file with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;alignment = AlignIO.read(open(&quot;example.phy&quot;), &quot;phylip&quot;) &#xA;def column(alignment):&#xA;    n = len(alignment[0].seq)&#xA;&#xA;    for i in range(n):&#xA;        c = []&#xA;&#xA;        for record in alignment:&#xA;            c.append(record.seq[i])&#xA;&#xA;        yield c&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This returns the columns of the file, but I don't understand how I can manipulate them further.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I get some helpful suggestions?&lt;/p&gt;&#xA;" OwnerUserId="3142" LastEditorUserId="3848" LastEditDate="2020-02-22T15:31:06.883" LastActivityDate="2020-02-22T15:31:06.883" Title="Counting letters in phylip alignment columns with Biopython" Tags="|sequence-alignment|python|phylogeny|phylogenetics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4955" PostTypeId="2" ParentId="4953" CreationDate="2018-08-29T21:22:12.313" Score="4" Body="&lt;p&gt;using something like &lt;code&gt;for col in column(alignment):&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I may get the number of columns which have all the same letters (amino acids)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;len(set(col)) == 1&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The number of columns which have at least 2 different amino acids (with no dashes):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;len(set(col)) &amp;gt; 1 and '-' not in col&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The number of columns which have any single dash:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;'-' in col&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;or if you really want columns with a single dash:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    col.count('-') == 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="581" LastActivityDate="2018-08-29T21:22:12.313" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4956" PostTypeId="2" ParentId="4953" CreationDate="2018-08-30T09:18:18.373" Score="3" Body="&lt;p&gt;I suggest you use &lt;code&gt;zip()&lt;/code&gt; to transform your data into columns and then apply the following counts similar to &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/4955/104&quot;&gt;heathobrien suggests&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import AlignIO&#xA;&#xA;alignment = AlignIO.read(open('example.phy'), 'phylip')&#xA;&#xA;var1 = var2 = var3 = 0&#xA;for col in zip(*alignment):&#xA;    num_unique = len(set(col))&#xA;    if num_unique == 1 and col[0] != '-':&#xA;        var1 += 1&#xA;    elif num_unique &amp;gt; 1 and '-' not in col:&#xA;        var2 += 1&#xA;    elif '-' in col:  # assumes 1 or more dashes&#xA;        var3 += 1&#xA;&#xA;print('Number of columns with the same amino acid: {}\n'&#xA;      'Number of columns with at least 2 amino acids (no gaps): {}\n'&#xA;      'Number of columns with at least one gap: {}'&#xA;       .format(var1, var2, var3))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For your example data, this outputs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Number of columns with the same amino acid: 9&#xA;Number of columns with at least 2 amino acids (no gaps): 40&#xA;Number of columns with at least one gap: 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastActivityDate="2018-08-30T09:18:18.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4957" PostTypeId="1" CreationDate="2018-08-30T21:24:07.407" Score="6" ViewCount="151" Body="&lt;p&gt;I have an aligned protein sequence file which I have been using for reconstructing a parsimonious tree. I am currently using NNITreeSearcher._get_neighbors method from Biopython 1.72 but it's way to find one best scored tree only.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def _nni(self, starting_tree, alignment):&#xA;    &quot;&quot;&quot;Search for the best parsimony tree using the NNI algorithm (PRIVATE).&quot;&quot;&quot;&#xA;    best_tree = starting_tree&#xA;    while True:&#xA;        best_score = self.scorer.get_score(best_tree, alignment)&#xA;        temp = best_score&#xA;        for t in self._get_neighbors(best_tree):&#xA;            score = self.scorer.get_score(t, alignment)&#xA;            if score &amp;lt; best_score:&#xA;                best_score = score&#xA;                best_tree = t&#xA;        # stop if no smaller score exist&#xA;        if best_score &amp;gt;= temp:&#xA;            break&#xA;    return best_tree&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I need to generate all NNI-neighbours of the trees (two trees are NNI-neighbours if one can be transformed into another by one nearest neighbour interchange operation).&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I want to know how can I get total number of trees generated before the best tree is reached.&lt;/li&gt;&#xA;&lt;li&gt;Is it possible to return all the trees generated before the best tree is reached, so that the all/required number of trees(other than the best tree) may be printed?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3142" LastActivityDate="2018-10-11T06:00:02.070" Title="Getting all NNI trees of a parsimony tree" Tags="|python|phylogenetics|algorithms|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4959" PostTypeId="1" CreationDate="2018-08-26T04:37:41.757" Score="2" ViewCount="1247" Body="&lt;p&gt;I am working on a project about phosphorylation, and I am creating a database of some proteins. I have a list of protein accession ids for which I need to get the corresponding PDB file for each one to map it to PSM matrix. I want to get the PDB file using Python or BioPython tools. Any suggestions will be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="3284" OwnerDisplayName="Protocol313" LastEditorUserId="57" LastEditDate="2018-08-31T20:06:37.150" LastActivityDate="2018-08-31T20:06:37.150" Title="Get a list of PDB ids from a list of sequence protein number?" Tags="|python|pdb|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4960" PostTypeId="2" ParentId="4959" CreationDate="2018-08-26T11:17:29.610" Score="2" Body="&lt;p&gt;A typical Uniprot URL would look like the example here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.uniprot.org/uniprot/?query=ID:P26378&amp;amp;format=tab&amp;amp;columns=id%2Cdatabase(PDB)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uniprot/?query=ID:P26378&amp;amp;format=tab&amp;amp;columns=id%2Cdatabase(PDB)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The response will be:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Entry Cross-reference (PDB) &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;P26378    1FXL;1G2E;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;We want our query to include the protein ID, the output format should be tab separated and the only contain the columns &lt;code&gt;id&lt;/code&gt; and &lt;code&gt;database (PDB)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get the PDB IDs you would need to split the response, take the last line, split this line after each tab and split it again after each semi-colon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using Python's Requests library the whole code would be.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;import time&#xA;&#xA;uniprot_ids = ['P26378', 'O35433', 'Q02910']&#xA;url = 'https://www.uniprot.org/uniprot/'&#xA;&#xA;protein_to_pdb = {}&#xA;for protein in uniprot_ids:&#xA;    params = {&#xA;        'format': 'tab',&#xA;        'query': 'ID:{}'.format(protein),&#xA;        'columns': 'id,database(PDB)'&#xA;    }&#xA;    contact = &quot;&quot;  # Please set your email address here.&#xA;    headers = {'User-Agent': 'Python {}'.format(contact)}&#xA;    r = requests.get(url, params=params, headers=headers)&#xA;&#xA;    protein_to_pdb[protein] = str(r.text).splitlines()[-1].split('\t')[-1].split(';')&#xA;    protein_to_pdb[protein].pop(-1)&#xA;    time.sleep(1)  # be respectful and don't overwhelm the server with requests&#xA;&#xA;print(protein_to_pdb)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;To accelerate things, you could store your &lt;code&gt;protein_to_pdb&lt;/code&gt; dict in a file and only retrieve the new ones&lt;/li&gt;&#xA;&lt;li&gt;Checking if the returned ID matches your input ID is always a good idea &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="461" OwnerDisplayName="Maximilian Peters" LastActivityDate="2018-08-26T11:17:29.610" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4961" PostTypeId="2" ParentId="4959" CreationDate="2018-08-27T15:21:59.967" Score="0" Body="&lt;p&gt;You could use mapping between UniProt and PDB entries from the &lt;a href=&quot;https://www.ebi.ac.uk/pdbe/docs/sifts/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SIFTS project&lt;/a&gt;.&#xA;The &lt;code&gt;uniprot_pdb.csv&lt;/code&gt; file there has UniProt accession names followed by a semicolon-separated list of PDB four letter codes.&lt;/p&gt;&#xA;" OwnerUserId="266" OwnerDisplayName="marcin" LastActivityDate="2018-08-27T15:21:59.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4962" PostTypeId="1" CreationDate="2018-08-31T18:59:22.733" Score="6" ViewCount="69" Body="&lt;p&gt;let's say I have a list of organism names like the example below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Achromobacter xylosoxidans&#xA;Acidithiobacillus ferrooxidans&#xA;Chloroflexus aurantiacus&#xA;Clostridium perfringens&#xA;Aquaspirillum arcticum&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The goal is to identify the optimal growth conditions for each organism--meaning the optimal growth temperature and pressure. Obviously I can just google this information if the list is composed of a few organisms or even few tens of organisms. However, assuming that the list is too large, let's say several 100s of names, what would be an efficient means to gathering this information.&lt;/p&gt;&#xA;&lt;p&gt;In searching for this info, I have not come across a database that has the information that I'm looking for in one place. Instead I find that this information is scattered across multiple sites, databases, scientific publications, protocols, etc. For each organism, I basically have to search through and extract this manually, and I was wondering if anyone can suggest some way to automate this. I am familiar with Python3, ideally I would accomplish this with python.&lt;/p&gt;&#xA;&lt;p&gt;I though about implementing some sort of a web scraping solution, where I provide the name of the organism, as well as a few key words and see what happens. However, many times this data is not well tabulated and the name and whenever you do find information on the organism, you need to read to be able to interpret the information to get your answer...&lt;/p&gt;&#xA;&lt;p&gt;maybe I'm not looking in the right place, or my approach to this problem is wrong/backwards/etc, any suggestions will help.&lt;/p&gt;&#xA;&lt;p&gt;I haven't been able to find a centralized place to gather this information, that's why I posted here. The closest thing I've found is the &lt;a href=&quot;https://atcc.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ATCC&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3291" LastEditorUserId="1574" LastEditDate="2024-03-28T00:53:21.787" LastActivityDate="2024-03-31T15:57:33.467" Title="Gathering data on bacterial organism growth conditions" Tags="|database|public-databases|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4963" PostTypeId="1" CreationDate="2018-08-31T19:02:18.760" Score="2" ViewCount="1456" Body="&lt;p&gt;I was confused about the VlnPlot that it shows like the screenshot. It only shows the dot but no distribution of the data, is that normal?&lt;a href=&quot;https://i.stack.imgur.com/CYjGQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CYjGQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3171" LastActivityDate="2021-01-07T17:08:39.090" Title="VlnPlot problem" Tags="|scrnaseq|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4964" PostTypeId="1" CreationDate="2018-09-01T22:18:35.140" Score="-1" ViewCount="309" Body="&lt;p&gt;I want to process a plenty of &lt;code&gt;scRNAseq&lt;/code&gt; datasets in &lt;code&gt;python&lt;/code&gt;, and I want to run &lt;code&gt;TMM&lt;/code&gt; normalization on all of them. Unfortunately some of them are already normalized with &lt;code&gt;TPM&lt;/code&gt; method. Is there a way of either converting it back to &lt;code&gt;UMI&lt;/code&gt;matrix, so that &lt;code&gt;TMM&lt;/code&gt; normalization could be applied to it or could we just apply &lt;code&gt;TMM&lt;/code&gt; directly to &lt;code&gt;TPM&lt;/code&gt;-normalized matrices? Would that work? Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-09-02T15:51:57.057" Title="Convert TPM-normalized matrices back to UMI in python" Tags="|python|scrnaseq|normalization|tpm|tmm|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4965" PostTypeId="1" CreationDate="2018-09-01T22:47:48.923" Score="1" ViewCount="522" Body="&lt;p&gt;I have many &lt;code&gt;scRNAseq&lt;/code&gt; datasets saved as &lt;code&gt;hdf5&lt;/code&gt; files (hierarchical data format*), and when I open them keys sometimes are confused, so I need a way of determining from string values themselves whether the array contains &lt;code&gt;gene symbols&lt;/code&gt; or &lt;code&gt;ensembl ids&lt;/code&gt;. What would be a regular expression in python to check for it?&lt;/p&gt;&#xA;&lt;p&gt;*, A database format for quickly accessing large amounts of data, which is accessed using &lt;code&gt;keys&lt;/code&gt;. The format is quicker than pickle/unpickle and can be viewed via a hdf5 viewer, a bit of info is &lt;a href=&quot;https://support.hdfgroup.org/HDF5/whatishdf5.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="3848" LastEditDate="2020-12-12T05:11:28.230" LastActivityDate="2020-12-12T05:11:28.230" Title="Check if a string is ensembl gene id or not in python" Tags="|python|ensembl|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4966" PostTypeId="2" ParentId="4965" CreationDate="2018-09-01T23:19:42.553" Score="2" Body="&lt;p&gt;Fixed it myself actually, not that hard. I found &lt;code&gt;regular expression&lt;/code&gt; for &lt;code&gt;ensembl ids&lt;/code&gt; here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/99142/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/99142/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And used the &lt;code&gt;re&lt;/code&gt; package in &lt;code&gt;python&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;&#xA;ensembl_regexp = 'ENS[A-Z]+[0-9]{11}'&#xA;&#xA;is_ensembl = np.all([bool(re.match(ensembl_regexp, gene)) for gene in gene_symbols])&#xA;    if is_ensembl:&#xA;        gene_symbols = list(f['ensembl_ids']) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where &lt;code&gt;gene_symbols&lt;/code&gt; is a &lt;code&gt;numpy&lt;/code&gt; array that I need to check whether it has &lt;code&gt;ensembl ids&lt;/code&gt;, and if yes, switch to using &lt;code&gt;gene symbols&lt;/code&gt; which are going to be under &lt;code&gt;ensembl_ids&lt;/code&gt; key due to a mistake of someone who made the &lt;code&gt;hdf5&lt;/code&gt; dataset.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="298" LastEditDate="2018-09-03T08:22:47.860" LastActivityDate="2018-09-03T08:22:47.860" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4967" PostTypeId="2" ParentId="4964" CreationDate="2018-09-02T15:51:57.057" Score="1" Body="&lt;p&gt;It depends what your downstream requirement is. If you want to do count based statisitcs, then you definately shouldn't just do TMM normalisation on the TPM data as count statistics only work on real count data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are intersted in doing something like clustering or dimensionality reduction, then I wouldn't use TMM, I'd probably use rLog, which includes normalisation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can have a crack at converting the TPMs back to something that looks like counts, but I not sure how well its going to work. To convert them back to REAL counts, you are going to need to know: the total number UMIs in the original dataset and the precise gene length used for each gene. Its posible the data set provides this ... Otherwise you are going to have problems. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;$\frac{TPM_i}{1,000,000}$ is the fraction of all transcripts in a sample that belong to a given gene. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If $$fL_i = \frac{length_i}{\Sigma_{j\in G} length_j} $$&#xA;is the fraction of the transcribed genome in a gene i. Thus $fR_i = fL_i \times \frac{TPM_i}{1,000,000}$ should be the fraction of all counts that come from a gene $i$. Multiply this by the total number of reads to get the original counts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The total number of reads may or may not be critcal, I think this might come out in the wash of the normalization, but it might mess with the disperision estimates, I'm not sure. But it is absolutely critical that &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) The TPM dataset was counted across exactly the same annotation as the count datasets&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) The gene lengths used are identical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This could be untrue for several reasons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Different annotation or annotation version (ENSEMBL, UCSC, REFSEQ, CCDS etc).&lt;/li&gt;&#xA;&lt;li&gt;Different subset of transcripts used, or different way of combinng transcripts.&lt;/li&gt;&#xA;&lt;li&gt;Different ways of calculating &quot;effective length&quot;, used to correct for things like which parts of the gene reads can or can't map to (reads too close to ends, unmappable regions of the gene etc). &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Much safer to convert the count sets to TPM, but this would preclude doing count based statisitcs (e.g. DESeq2, edgeR, voom). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I'd also worry that if these datasets were processed at this point differently, they are going to have been processed in different ways, and that the data is basically uncomparable. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-09-02T15:51:57.057" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4969" PostTypeId="2" ParentId="4929" CreationDate="2018-09-03T08:49:18.620" Score="1" Body="&lt;p&gt;The answer is behind these papers:&lt;br&gt;&#xA;&lt;a href=&quot;http://science.sciencemag.org/content/early/2018/04/25/science.aar5780&quot; rel=&quot;nofollow noreferrer&quot;&gt;The dynamics of gene expression in vertebrate embryogenesis at single-cell resolution&lt;/a&gt;&lt;br&gt;&#xA; &lt;a href=&quot;http://science.sciencemag.org/content/early/2018/04/25/science.aar4362.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;Single-cell mapping of gene expression landscapes and lineage in the zebrafish embryo&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also &lt;a href=&quot;http://science.sciencemag.org/content/sci/suppl/2018/04/25/science.aar3131.DC1/aar3131-Farrell-SM.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="48" LastEditDate="2018-09-30T14:22:31.487" LastActivityDate="2018-09-30T14:22:31.487" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4970" PostTypeId="2" ParentId="4637" CreationDate="2018-09-03T09:44:41.867" Score="4" Body="&lt;p&gt;tRNAscan-SE 2.0 requires separate installation of Infernal 1.1.2.&#xA;The source code and prebuilt binaries can be obtained at&#xA;&lt;a href=&quot;http://eddylab.org/infernal/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://eddylab.org/infernal/&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The installation directory of Infernal should be the same as the one&#xA;where tRNAscan-SE 2.0 is installed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When install infernal, run the configure command with the option &lt;code&gt;--prefix &amp;lt;your tRNAscan-SE-2.0 install path&amp;gt;&lt;/code&gt;.&#xA;If you didn't add option &lt;code&gt;--prefix&lt;/code&gt; when installing tRNAscan-SE-2.0, then you should install infernal without option &lt;code&gt;--prefix&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3300" LastEditorUserId="29" LastEditDate="2018-09-03T15:11:06.317" LastActivityDate="2018-09-03T15:11:06.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4971" PostTypeId="1" CreationDate="2018-09-04T07:20:13.250" Score="1" ViewCount="21" Body="&lt;p&gt;I have a list binding site motif of SP1 transcription factor which collects from PWMScan tools. Actually this tool scan whole human genome and give the entire binding motif list. From that list of binding motif I want to align with human genome so that I can find the corresponding gene name. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If any body prescribe the suitable tools name or it will require for any programming.&lt;/p&gt;&#xA;" OwnerUserId="3302" LastEditorUserId="48" LastEditDate="2018-09-04T09:10:57.447" LastActivityDate="2018-09-04T09:10:57.447" Title="Finding gene name from human genome using SP1 transcrition factor binding site from Postion Weight Matrix" Tags="|gene|genome|transcriptome|motifs|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4972" PostTypeId="5" CreationDate="2018-09-04T10:04:12.903" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-09-04T10:04:12.903" LastActivityDate="2018-09-04T10:04:12.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4973" PostTypeId="4" CreationDate="2018-09-04T10:04:12.903" Score="0" Body="Use this tag if your question is about reproducing some figure or plot. Your figure should be using the same data or equivalent tools as the original plot." OwnerUserId="48" LastEditorUserId="48" LastEditDate="2020-06-04T07:40:35.503" LastActivityDate="2020-06-04T07:40:35.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4974" PostTypeId="2" ParentId="375" CreationDate="2018-09-04T11:09:00.900" Score="3" Body="&lt;p&gt;I've sorted the visualisation out. Here are three alternative representations of repetitive structures for the same sequence:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OIcNm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OIcNm.png&quot; alt=&quot;Repeat patterns represented as a dotplot&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/QfV3u.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QfV3u.png&quot; alt=&quot;Repeat patterns represented as a linear &amp;quot;ripple&amp;quot; plot&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/L158Z.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/L158Z.png&quot; alt=&quot;Repeat patterns represented as a semicircular &amp;quot;sunset&amp;quot; plot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;These were generated using the same R script, callable from the command line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; &lt;span class=&quot;math-container&quot;&gt;$ repaver.r -style dotplot -prefix dotplot MAOA_chrX_43645715-43756016.fa&#xA; $&lt;/span&gt; repaver.r -style profile -prefix profile MAOA_chrX_43645715-43756016.fa&#xA; $ repaver.r -style semicircular -prefix semi MAOA_chrX_43645715-43756016.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;More details about this are in the &lt;a href=&quot;http://dx.doi.org/10.7490/f1000research.1116032.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;presentation&lt;/a&gt; I gave at Queenstown Research Week, 2018. I also wrote a chapter in a peer-reviewed eBook &lt;a href=&quot;https://www.humboldtaustralia.org.au/wp-content/uploads/2018/09/Our-Changing-World-in-the-South-Pacific.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This is fast enough that I can run it on the nanopore C. elegans genome in about half an hour, producing these plots for each contig. I don't quite have a method to iterate through this plot and pick out the dominant repeat length at each location, but that's a [relatively] simple extension of what I've already done.&lt;/p&gt;&#xA;&lt;p&gt;With a lot of optimisations for speed and memory consumption, I've now been able to run it on the full human genome. It takes a couple of days on my desktop computer (64GB RAM + SSD swap) to categorise the 100bp repeat structure of the assembled T2T/CHM13-v1.0 chromosomes.&lt;/p&gt;&#xA;&lt;p&gt;Code &lt;a href=&quot;https://gitlab.com/gringer/bioinfscripts/-/blob/master/repaver.r&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-10-04T10:15:41.157" LastActivityDate="2020-10-04T10:15:41.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4975" PostTypeId="1" AcceptedAnswerId="5409" CreationDate="2018-09-04T13:59:34.490" Score="3" ViewCount="114" Body="&lt;p&gt;I am trying to build a UCSC track hub (a private one). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have all my hub files on a university server in a simple folder called /hub. I am using Ubuntu 16.04 and the server is also Ubuntu (16.04.3 LTS (GNU/Linux 4.4.0-98-generic x86_64).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried using hubCheck and also simply upload the hub as instructed in the &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgHubConnect?hubId=342899&amp;amp;hgHub_do_disconnect=on&amp;amp;hgHubConnect.remakeTrackHub=on&amp;amp;hgsid=688716027_AHrlcnB4n0FPcPWd8hOPjTqvyPkR&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC instructions&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The URL I have used to indicate the location of my hub is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;http://pcxxx.xx.xx.ac.uk/home/username/projects/project-name/hub/hub.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, in both cases, I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Couldn't open http://webserver-id/home/username/projects/project-name/hub/hub.txt&#xA;TCP non-blocking connect() to webserver timed-out in select() after 10000    milliseconds - Cancelling!&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am not very knowledgeable in servers and protocols (http, ftp etc). But am I right to think that this error means that my hub is somehow not internet accessible? I'm struggling to understand what this error means. When I log onto the server, I automatically arrive in my home folder: &lt;code&gt;/home/username&lt;/code&gt;. My hub is in &lt;code&gt;/projects/project-name/hub&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way I can check if my folder location is indeed internet accessible?&lt;/p&gt;&#xA;" OwnerUserId="2931" LastEditorUserId="48" LastEditDate="2018-09-04T15:13:14.817" LastActivityDate="2018-11-08T17:27:01.387" Title="Finding URL for UCSC data hub" Tags="|ucsc|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="4976" PostTypeId="1" AcceptedAnswerId="5146" CreationDate="2018-09-04T17:48:58.667" Score="5" ViewCount="1455" Body="&lt;p&gt;I need to download all the &lt;strong&gt;completely assembled cyanobacterial genome&lt;/strong&gt;'s GenBank file(.gbff) from NCBI(RefSeq or INSDC ftp data).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this I think, the steps are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Need to find the completely assembled genomes.&lt;/li&gt;&#xA;&lt;li&gt;find the GenBank file URL based on the taxonomic name.&lt;/li&gt;&#xA;&lt;li&gt;download the GenBank file(.gbff file)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is their any way to do this by using any python module or any other idea??&lt;/p&gt;&#xA;" OwnerUserId="211" LastEditorUserId="211" LastEditDate="2018-09-05T07:26:01.637" LastActivityDate="2022-12-06T07:40:04.940" Title="How to automate NCBI genome download" Tags="|python|phylogenetics|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4977" PostTypeId="1" AcceptedAnswerId="4979" CreationDate="2018-09-05T01:16:19.087" Score="8" ViewCount="4181" Body="&lt;p&gt;I do not understand how to specify correctly the parameters on a &lt;code&gt;SLURM&lt;/code&gt; cluster for &lt;code&gt;snakemake&lt;/code&gt; to use them. I tried submitting the following &lt;code&gt;SLURM&lt;/code&gt; file, but it does not work this way and the number of cores used is only &lt;code&gt;1&lt;/code&gt;, not &lt;code&gt;20&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;#SBATCH -p standard&#xA;#SBATCH -A overall &#xA;#SBATCH --time=12:00:00&#xA;#SBATCH --output=snakemake%A.out&#xA;#SBATCH --error=snakemake%A.err&#xA;#SBATCH --nodes=1&#xA;#SBATCH --ntasks=1&#xA;#SBATCH --cpus-per-task=20&#xA;#SBATCH --mem=120000&#xA;&#xA;snakemake&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, I tried following &lt;code&gt;snakemake&lt;/code&gt; &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/snakefiles/configuration.html&quot; rel=&quot;noreferrer&quot;&gt;tutorial&lt;/a&gt;. And created &lt;code&gt;cluster.json&lt;/code&gt; based on the &lt;code&gt;SLURM&lt;/code&gt; parameters that I need:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;    &quot;__default__&quot; :&#xA;    {&#xA;        &quot;A&quot; : &quot;overall&quot;,&#xA;        &quot;time&quot; : &quot;24:00:00&quot;,&#xA;        &quot;nodes&quot;: 1,&#xA;        &quot;ntasks&quot;: 1,&#xA;        &quot;cpus&quot; : 20,&#xA;        &quot;p&quot; : &quot;standard&quot;,&#xA;        &quot;mem&quot;: 120000,&#xA;        &quot;output&quot;: &quot;snakemake%A.out&quot;,&#xA;        &quot;error&quot;: &quot;snakemake%A.err&quot;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And ran &lt;code&gt;snakemake&lt;/code&gt; inside a newly created &lt;code&gt;snakemake.sh&lt;/code&gt; script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;snakemake -j 999 --cluster-config cluster.json --cluster &quot;sbatch -A &#xA;{cluster.A} -p {cluster.p} \ &#xA;-t {cluster.time} --output {cluster.output} --error {cluster.error} -- &#xA;nodes {cluster.nodes} \&#xA;--ntasks {cluster.ntasks} --cpus-per-task {cluster.cpus} --mem &#xA;{cluster.mem}&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And it is giving me an error now:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;sbatch: error: Unable to open file&lt;br&gt;&#xA;  /bin/sh: line 1: -t: command not found&#xA;  Error submitting jobscript (exit code 127):&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am now completely lost at what I shoould actually do. I would prefer plain regular &lt;code&gt;.slurm&lt;/code&gt; file submission, but how to make &lt;code&gt;snakemake&lt;/code&gt; use them? Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I removed &lt;code&gt;\&lt;/code&gt; - line separators in the &lt;code&gt;snakemake.sh&lt;/code&gt; script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;snakemake -j 10 --cluster-config cluster.json --cluster &quot;sbatch -A {cluster.A} -p {cluster.p} -t {cluster.time} --output {cluster.output} --error {cluster.error} --nodes {cluster.nodes} --ntasks {cluster.ntasks} --cpus-per-task {cluster.cpus} --mem {cluster.mem}&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And it started to run. It is not convenient though for me. I would rather prefer submitting just one job using &lt;code&gt;.slurm&lt;/code&gt; file passing all of the parameters from &lt;code&gt;#SBATCH&lt;/code&gt;. Is it possible?&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2018-09-05T07:15:52.950" LastActivityDate="2020-02-11T15:45:13.437" Title="Running SnakeMake on cluster" Tags="|snakemake|slurm|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="4978" PostTypeId="1" AcceptedAnswerId="4980" CreationDate="2018-09-05T03:09:57.093" Score="4" ViewCount="160" Body="&lt;p&gt;I used &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bioawk -cfastx 'length($seq) &amp;gt; 1 {print &quot;@&quot;$name&quot;\n&quot;$seq&quot;\n+\n&quot;$qual}' in.fq.qz | gzip &amp;gt; out.fq.qz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;in order to keep a particular read length, but this command shortened the header  from&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@A00199:161:HF3JLDMXX:1:1101:5882:1063 2:N:0:CTGTAGCC+GAACAGGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@A00199:161:HF3JLDMXX:1:1101:5882:1063&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How is it possible to fix the &lt;code&gt;bioawk&lt;/code&gt; command to remain the full command?&lt;/p&gt;&#xA;" OwnerUserId="2477" LastEditorUserId="77" LastEditDate="2018-09-05T07:20:25.750" LastActivityDate="2018-09-05T07:20:25.750" Title="bioawk removed part of FASTQ header" Tags="|bioawk|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4979" PostTypeId="2" ParentId="4977" CreationDate="2018-09-05T07:05:38.400" Score="10" Body="&lt;p&gt;You can omit &lt;code&gt;--nodes&lt;/code&gt;, you need the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;#SBATCH --ntasks-per-node=1&#xA;#SBATCH -c threads_from_snakemake&#xA;#SBATCH -p some_partition&#xA;&#xA;Commands go here&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For slurm, you may want to modify my &lt;a href=&quot;https://github.com/dpryan79/Misc/blob/master/MPIIE_internal/SlurmEasy&quot; rel=&quot;noreferrer&quot;&gt;SlurmEasy script&lt;/a&gt;. We use that with snakemake all the time, usually of the form:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;snakemake -c &quot;SlurmEasy -t {threads} -n {rule} --mem-per-cpu {cluster.memory}&quot; ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You'll also want units with your memory requests, like &lt;code&gt;12G&lt;/code&gt;. Also, in general it's best not to submit snakemake itself to the cluster, rather run it on an interactive node and have it submit jobs on your behalf (if the admins complain point out that snakemake is barely using any resources on the head node).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-09-05T07:05:38.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4980" PostTypeId="2" ParentId="4978" CreationDate="2018-09-05T07:19:50.107" Score="5" Body="&lt;pre&gt;&lt;code&gt;bioawk -cfastx 'length($seq) &amp;gt; 1 {print &quot;@&quot;$name $comment&quot;\n&quot;$seq&quot;\n+\n&quot;$qual}' in.fq.qz | gzip &amp;gt; out.fq.qz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note the addition of &lt;code&gt;$comment&lt;/code&gt;, which is what follows the name on the first line of each entry.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-09-05T07:19:50.107" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4981" PostTypeId="1" AcceptedAnswerId="4982" CreationDate="2018-09-05T09:29:36.197" Score="2" ViewCount="1117" Body="&lt;p&gt;I am using python 3 and biopython 1.72. I have been using a protein sequence file in clustal format.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CLUSTAL 2.1 multiple sequence alignment&#xA;&#xA;&#xA;Human           ------------------------------------------------------------&#xA;Chimpanzee      ------------------------------------------------------------&#xA;Dog             ------------------------------------------------------------&#xA;Mouse           ------------------------------------------------------------&#xA;Xenopus         ------------------------------------------------------------&#xA;Amphioxus       MQWTGFRVSMTTLMMIMGVVAVLIALLPAKAQQPHDKSLRTTSTLTDTGASADEADMGSA&#xA;&#xA;&#xA;Human           ------------------------------------------------------------&#xA;Chimpanzee      ------------------------------------------------------------&#xA;Dog             ------------------------------------------------------------&#xA;Mouse           ------------------------------------------------------------&#xA;Xenopus         ------------------------------------------------------------&#xA;Amphioxus       HVELLDGDDDVGNGSDQMMVTLHLQSIFQCIRRPCEKVDRAIDPVTQRWRTANTRNDYQK&#xA;&#xA;&#xA;Human           ------------------MRLRVRLLKRTWPLEVPETEPTLGHLRSHLRQSLLCTWGYSS&#xA;Chimpanzee      ------------------MRLRVRLLKRTWPLEVPETEPTLGHLRSRLRQSLLCTWGYSS&#xA;Dog             ------------------MKLRVRLQKRTWPLDLPDAEPTLGQLRAHLSQALLPSWGFGS&#xA;Mouse           ------------------MKLRVRLQKRTQPLEVPESEPTLGQLRAHLSQVLLPTLGFSS&#xA;Xenopus         ------------------MKLRVRVRKQTNRLELEAESPTLGDLRSKLSSVTLPALGYST&#xA;Amphioxus       INVCVVPAYDVSLSTGVRMKLRVKISGQKTRVDVGQDCHTLGTLRTLLAPVLGEQYGLGD&#xA;                                  *:***::  :.  :::     *** **: *        * . &#xA;&#xA;Human           NTRFTITLNYKDPLTGDEETLASYGIVSGDLICLILQDDIPA---------------PNI&#xA;Chimpanzee      NTRFTITLNYKDPLTGDEETLASYGIVSGDLICLILQDDIPA---------------PNI&#xA;Dog             DTRFAITLNNKDALTGDEETLASYGIVSGDLICLILEDAIPA---------------PNL&#xA;Mouse           DTRFAITLNNKDALTGDEETLASYGIVSGDLICLVLEDDMPA---------------PNL&#xA;Xenopus         EANFTITLNGKDALTGDQNTLESAGIISGDLIVVVLPDSQLP---------------PPQ&#xA;Amphioxus       DMPFEISLNGRDALLGDDKPLSDLGIVSGDLIHILLASVDQPSTNHNTQQQGQHPSSPEH&#xA;                :  * *:** :*.* **::.* . **:***** ::* .   .               *  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I need to iterate over the rows of my file so that I could count the occurrence of characters in each row.&#xA;I tried using the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;file_var = input(&quot;File: &quot;)&#xA;count = 0&#xA;with open (file_var , 'r') as fo:&#xA;    for i in fo:&#xA;        count += 1&#xA;        print (i)&#xA;    print (count)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But this results just counting the total number of rows in a file&lt;code&gt;(count variable)&lt;/code&gt;. If I iterate over the file using biopython, it does'nt work also.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for seq_record in SeqIO.parse(&quot;sample.aln&quot;,&quot;clustal&quot;):&#xA;    print (seq_record.id)&#xA;    print (repr(seq_record.seq))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;seq_record.id&lt;/strong&gt; results the names of the sequences used in the file.&#xA;&lt;strong&gt;repr(seq_record.seq)&lt;/strong&gt; results the sequences of each specie alongwith description.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I need to&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;get the rows only with amino acids (exclusing the symbols &lt;code&gt;'*,.,:'&lt;/code&gt; line) and nothing else from the row and for each row I need to count the number of characters (exclusing &lt;code&gt;'-'&lt;/code&gt; dashes)&lt;/li&gt;&#xA;&lt;li&gt;the number of characters of that row be placed at the end of each of the row.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;for example: for above example file, the counter should work like;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;------------------------------------------------------------ 0&#xA;MQWTGFRVSMTTLMMIMGVVAVLIALLPAKAQQPHDKSLRTTSTLTDTGASADEADMGSA 60&#xA;------------------MRLRVRLLKRTWPLEVPETEPTLGHLRSHLRQSLLCTWGYSS 42&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3142" LastActivityDate="2018-09-05T11:08:06.647" Title="Counting the characters in a row using python" Tags="|python|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4982" PostTypeId="2" ParentId="4981" CreationDate="2018-09-05T10:57:55.503" Score="5" Body="&lt;p&gt;Here's one way with pure Python:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;import sys&#xA;import re&#xA;&#xA;with open (sys.argv[1] , 'r') as fo:&#xA;    for line in fo:&#xA;        fields = line.split()&#xA;        if len(fields)==2:&#xA;            print(fields[1],len(fields[1].replace('-','')))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Running this on your file gives:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ foo.py file.aln &#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;MQWTGFRVSMTTLMMIMGVVAVLIALLPAKAQQPHDKSLRTTSTLTDTGASADEADMGSA 60&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;HVELLDGDDDVGNGSDQMMVTLHLQSIFQCIRRPCEKVDRAIDPVTQRWRTANTRNDYQK 60&#xA;------------------MRLRVRLLKRTWPLEVPETEPTLGHLRSHLRQSLLCTWGYSS 42&#xA;------------------MRLRVRLLKRTWPLEVPETEPTLGHLRSRLRQSLLCTWGYSS 42&#xA;------------------MKLRVRLQKRTWPLDLPDAEPTLGQLRAHLSQALLPSWGFGS 42&#xA;------------------MKLRVRLQKRTQPLEVPESEPTLGQLRAHLSQVLLPTLGFSS 42&#xA;------------------MKLRVRVRKQTNRLELEAESPTLGDLRSKLSSVTLPALGYST 42&#xA;INVCVVPAYDVSLSTGVRMKLRVKISGQKTRVDVGQDCHTLGTLRTLLAPVLGEQYGLGD 60&#xA;NTRFTITLNYKDPLTGDEETLASYGIVSGDLICLILQDDIPA---------------PNI 45&#xA;NTRFTITLNYKDPLTGDEETLASYGIVSGDLICLILQDDIPA---------------PNI 45&#xA;DTRFAITLNNKDALTGDEETLASYGIVSGDLICLILEDAIPA---------------PNL 45&#xA;DTRFAITLNNKDALTGDEETLASYGIVSGDLICLVLEDDMPA---------------PNL 45&#xA;EANFTITLNGKDALTGDQNTLESAGIISGDLIVVVLPDSQLP---------------PPQ 45&#xA;DMPFEISLNGRDALLGDDKPLSDLGIVSGDLIHILLASVDQPSTNHNTQQQGQHPSSPEH 60&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Some other options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;awk&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk 'NF==2{l=$2;gsub(/-/,&quot;&quot;,$2); print l,length($2)}' file.aln &#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;MQWTGFRVSMTTLMMIMGVVAVLIALLPAKAQQPHDKSLRTTSTLTDTGASADEADMGSA 60&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;HVELLDGDDDVGNGSDQMMVTLHLQSIFQCIRRPCEKVDRAIDPVTQRWRTANTRNDYQK 60&#xA;------------------MRLRVRLLKRTWPLEVPETEPTLGHLRSHLRQSLLCTWGYSS 42&#xA;------------------MRLRVRLLKRTWPLEVPETEPTLGHLRSRLRQSLLCTWGYSS 42&#xA;------------------MKLRVRLQKRTWPLDLPDAEPTLGQLRAHLSQALLPSWGFGS 42&#xA;------------------MKLRVRLQKRTQPLEVPESEPTLGQLRAHLSQVLLPTLGFSS 42&#xA;------------------MKLRVRVRKQTNRLELEAESPTLGDLRSKLSSVTLPALGYST 42&#xA;INVCVVPAYDVSLSTGVRMKLRVKISGQKTRVDVGQDCHTLGTLRTLLAPVLGEQYGLGD 60&#xA;NTRFTITLNYKDPLTGDEETLASYGIVSGDLICLILQDDIPA---------------PNI 45&#xA;NTRFTITLNYKDPLTGDEETLASYGIVSGDLICLILQDDIPA---------------PNI 45&#xA;DTRFAITLNNKDALTGDEETLASYGIVSGDLICLILEDAIPA---------------PNL 45&#xA;DTRFAITLNNKDALTGDEETLASYGIVSGDLICLVLEDDMPA---------------PNL 45&#xA;EANFTITLNGKDALTGDQNTLESAGIISGDLIVVVLPDSQLP---------------PPQ 45&#xA;DMPFEISLNGRDALLGDDKPLSDLGIVSGDLIHILLASVDQPSTNHNTQQQGQHPSSPEH 60&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Perl&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ perl -lane '$l=$F[1];$l=~s/-//g; $#F==1 &amp;amp;&amp;amp; print &quot;$F[1] &quot;, length($l)' file.aln &#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;MQWTGFRVSMTTLMMIMGVVAVLIALLPAKAQQPHDKSLRTTSTLTDTGASADEADMGSA 60&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;------------------------------------------------------------ 0&#xA;HVELLDGDDDVGNGSDQMMVTLHLQSIFQCIRRPCEKVDRAIDPVTQRWRTANTRNDYQK 60&#xA;------------------MRLRVRLLKRTWPLEVPETEPTLGHLRSHLRQSLLCTWGYSS 42&#xA;------------------MRLRVRLLKRTWPLEVPETEPTLGHLRSRLRQSLLCTWGYSS 42&#xA;------------------MKLRVRLQKRTWPLDLPDAEPTLGQLRAHLSQALLPSWGFGS 42&#xA;------------------MKLRVRLQKRTQPLEVPESEPTLGQLRAHLSQVLLPTLGFSS 42&#xA;------------------MKLRVRVRKQTNRLELEAESPTLGDLRSKLSSVTLPALGYST 42&#xA;INVCVVPAYDVSLSTGVRMKLRVKISGQKTRVDVGQDCHTLGTLRTLLAPVLGEQYGLGD 60&#xA;NTRFTITLNYKDPLTGDEETLASYGIVSGDLICLILQDDIPA---------------PNI 45&#xA;NTRFTITLNYKDPLTGDEETLASYGIVSGDLICLILQDDIPA---------------PNI 45&#xA;DTRFAITLNNKDALTGDEETLASYGIVSGDLICLILEDAIPA---------------PNL 45&#xA;DTRFAITLNNKDALTGDEETLASYGIVSGDLICLVLEDDMPA---------------PNL 45&#xA;EANFTITLNGKDALTGDQNTLESAGIISGDLIVVVLPDSQLP---------------PPQ 45&#xA;DMPFEISLNGRDALLGDDKPLSDLGIVSGDLIHILLASVDQPSTNHNTQQQGQHPSSPEH 60&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-09-05T11:08:06.647" LastActivityDate="2018-09-05T11:08:06.647" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="4983" PostTypeId="1" AcceptedAnswerId="4995" CreationDate="2018-09-05T18:23:50.607" Score="2" ViewCount="1916" Body="&lt;p&gt;I have a folder where the &lt;code&gt;hdf5&lt;/code&gt; files inputs are (matrices of &lt;code&gt;scRNAseq&lt;/code&gt; raw &lt;code&gt;UMI&lt;/code&gt; counts). There is a &lt;code&gt;python&lt;/code&gt; script that loads them, splits them into multiple matrices, normalizes them, and saves as &lt;code&gt;csv&lt;/code&gt; files. When the name pattern of the output files is known, things are easy, I know how to do that, but when one input (&lt;code&gt;hdf5&lt;/code&gt; file) can produce various number of outputs (&lt;code&gt;csv&lt;/code&gt; files) I am not sure how to approach the issue. Currently my &lt;code&gt;Snakemake&lt;/code&gt; file relevant part looks the following way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;hdf5_files,=glob_wildcards('/main/{hdf5_file}.hdf5') &#xA;&#xA;rule all:&#xA;  input:&#xA;---&amp;gt;  expand('data_files/normalized/{hdf5_file}.csv', hdf5_file=hdf5_files)&#xA;    ...&#xA;&#xA;rule normalization:&#xA;  input:&#xA;     script = 'python/normalize.py',&#xA;     path = '/main/{hdf5_file}.hdf5'&#xA;  output:&#xA;---&amp;gt;  path = 'data_files/normalized/{hdf5_file}.csv'&#xA;  shell:&#xA;     &quot;python {input.script} -data {input.path} -path {output.path}&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Above I highlighted the &lt;code&gt;2&lt;/code&gt; lines that I am not sure now how I should write. Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-09-07T09:30:14.340" Title="Output file names are not known beforehand" Tags="|snakemake|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4984" PostTypeId="1" CreationDate="2018-09-05T18:28:40.247" Score="2" ViewCount="63" Body="&lt;p&gt;I am not sure whether this is the best SE community for this question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to publish a transcriptome paper, along with interactive materials enabling readers to peruse the data behind the discussion. The R package 'plotly' enables rendering online-publishable tables that can be filtered &amp;amp; searched for specific entries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to publish such a table as supplementary to a paper (e.g. preprint version) however usually data repositories allow for uploading files and text. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would anyone here know an official data repository with a format such that would enable publishing online an interactive table? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A good example of such a table is given &lt;a href=&quot;http://butterlab.imb-mainz.de/flydev/?data=embryogenesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="2329" LastEditorUserId="77" LastEditDate="2018-09-06T07:05:39.170" LastActivityDate="2018-09-06T14:18:11.047" Title="Best data repository to publish a large 'plotly' table containing all annotations on a transcriptome?" Tags="|r|public-databases|public-dataset|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4985" PostTypeId="1" AcceptedAnswerId="4990" CreationDate="2018-09-06T02:34:07.810" Score="8" ViewCount="2311" Body="&lt;p&gt;edgeR's cpm function has an argument called &lt;code&gt;prior.count&lt;/code&gt;. Based on my understanding of the documentation, it is supposed to be adding a fixed number per sample which is proportional to the library size of said sample. Average of all numbers added to all samples would be equal to &lt;code&gt;prior.count&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, looking at actual data this does not seem to be the case. Given an imaginary data frame of &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df = data.frame(a = c(1,2,3,0),c = c(1,2,3,0)*3, b = c(1,2,3,0)*2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We can try to calculate log cpms by doing&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;logCPM = cpm(df,log = TRUE,prior.count = .5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We can also calculate regular cpms by doing &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CPM = cpm(df)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If we wanted to see, what is the number that is added to the CPMs before getting logged, we could do&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;difference = 2^logCPM - CPM&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To make it look pretty lets use tibble&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tibble::as.tibble(difference)&#xA;&#xA;# A tibble: 4 x 3&#xA;                     a                    c                    b&#xA;                 &amp;lt;dbl&amp;gt;                &amp;lt;dbl&amp;gt;                &amp;lt;dbl&amp;gt;&#xA;1  25641                25641                25641              &#xA;2  12821                12821                12821              &#xA;3 -    0.0000000000582 -    0.0000000000582 -    0.0000000000582&#xA;4  38462                38462                38462   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here we see that the number that is added has no relation to the library size of the sample. What I want to learn is, how is this number that is added to each cpm is calculated based on prior.count. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been digging through the code but that part goes into C++ territory which hinders my understanding of what is going on &lt;/p&gt;&#xA;" OwnerUserId="319" LastEditorUserId="319" LastEditDate="2018-09-06T19:52:36.540" LastActivityDate="2022-09-21T22:36:30.370" Title="How is prior.count used by edgeR's cpm" Tags="|r|edger|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4987" PostTypeId="2" ParentId="4984" CreationDate="2018-09-06T07:12:31.450" Score="1" Body="&lt;p&gt;The example looks a lot like a shiny application. Your best bet then is one of the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ask your IT department to setup a shiny server for you. They can use the free version.&lt;/li&gt;&#xA;&lt;li&gt;Post your shiny app on &lt;a href=&quot;https://shiny.rstudio.com/deploy/&quot; rel=&quot;nofollow noreferrer&quot;&gt;shinyapps.io&lt;/a&gt;. Depending on how popular the page becomes, you might have to pay for an account.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The free version of shiny works for most cases (it's what we use), the only problem with it is that it doesn't scale (but if you care about that then you need to either pay for hosting on a pro server or buy a shiny pro license).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-09-06T07:12:31.450" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4988" PostTypeId="2" ParentId="4984" CreationDate="2018-09-06T07:24:40.017" Score="2" Body="&lt;p&gt;I have seen some (1 or more) articles with interactive data displays in the &lt;a href=&quot;https://f1000research.com/articles/&quot; rel=&quot;nofollow noreferrer&quot;&gt;F1000Research journal&lt;/a&gt; I don't know recall now if they had interactive tables or plots but they used plotly for sure.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-09-06T07:24:40.017" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="4989" PostTypeId="1" AcceptedAnswerId="5005" CreationDate="2018-09-06T07:56:24.757" Score="4" ViewCount="756" Body="&lt;h3&gt;The dilemma:&lt;/h3&gt;&#xA;&lt;p&gt;I have around one million short sequences (21 bp to several 100s of basepairs) for which I need to identify all occurrences of in 20-30x coverage noisy long reads (both pacbio and ONT). All of the short sequences and long reads are derived from the same individual, and there are enough short sequences given the organism's genome size that each long read should have at least one or multiple short sequences.&lt;/p&gt;&#xA;&lt;h3&gt;An example:&lt;/h3&gt;&#xA;&lt;p&gt;Given three short sequences, &lt;code&gt;AA&lt;/code&gt; &amp;amp; &lt;code&gt;BBBB&lt;/code&gt; &amp;amp; &lt;code&gt;CCCCCC&lt;/code&gt; , how to identify all occurrences of these sequences in long, uncorrected reads.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Sequences to identify in long reads:&#xA;  - AA&#xA;  - BBBB&#xA;  - CCCCCC&#xA;&#xA;Long Reads:&#xA;  - 1: ------AA-----------------------CCCCCC------&#xA;  - 2: ------AA--------BBBB-----------------------&#xA;  - 3: ---BBBB-----------CCCC--&#xA;  - 4: ------------------AA-----&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In this example, the output sam file will contain high MAPQ hits for short sequence &lt;code&gt;AA&lt;/code&gt; to reads (1, 2, 3), for short sequence &lt;code&gt;BBBB&lt;/code&gt; to reads (2, 3). For short sequence &lt;code&gt;CCCCCC&lt;/code&gt; there will be a high MAPQ hit to read 1 and a lower MAPQ hit to read 3&lt;/p&gt;&#xA;&lt;h3&gt;Limitations:&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Due to the requirements of this project, I cannot perform a de novo assembly then map the short sequences to the assembly output. It is necessary to directly map the short sequences to the long reads.&lt;/li&gt;&#xA;&lt;li&gt;I cannot correct the long reads before I map/align the short sequences to them.&lt;/li&gt;&#xA;&lt;li&gt;The alignment/mapping should sensitively detect the short sequences in the long reads.&lt;/li&gt;&#xA;&lt;li&gt;The short sequences may be anywhere from 1Mbp to 30Mbp in total bases.&lt;/li&gt;&#xA;&lt;li&gt;The long reads may be anywhere from 3Gbp to 90Gbp in total bases. So the mapper/alignment technique is hopefully fast. Sensitivity is more important though.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Research so far:&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;bwa mem&lt;/code&gt; does not output multiply-mapped in a predictable manner that fits this use case &lt;a href=&quot;https://www.biostars.org/p/304614/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/304614/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;blat&lt;/code&gt; seems to output many hits for a single short sequence, but there are no out-of-the-box parameters for mapping short reads to long reads.&lt;/li&gt;&#xA;&lt;li&gt;Maybe multiple sequence alignment is the most sensitive, but wouldn't that entail running (no. of short seqs) * (no. of long reads) alignments?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="3848" LastEditDate="2020-07-21T17:22:42.617" LastActivityDate="2020-07-21T17:22:42.617" Title="How to map short sequences to long reads, recovering all multiply-mapped high-quality matches" Tags="|sequence-alignment|software-recommendation|phylogenetics|long-reads|homework|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4990" PostTypeId="2" ParentId="4985" CreationDate="2018-09-06T08:34:00.373" Score="7" Body="&lt;p&gt;The prior count ends up getting scaled by the ratio of a library size to the average library size and then multiplied by 2 before getting added to each library size (I'm sure there's a good reason for that, but I don't know what it is). Using the example &lt;code&gt;df&lt;/code&gt; data frame in your post, let's walk step-by-step through what &lt;code&gt;cpm()&lt;/code&gt; is doing:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;df = data.frame(a = c(1,2,3,0),c = c(1,2,3,0)*3, b = c(1,2,3,0)*2)&#xA;prior.count = 0.5&#xA;# First, we need to calculate a library size&#xA;lib.size = colSums(df)&#xA;# Calculate the average library size and the adjusted priors&#xA;ave.lib = mean(lib.size)&#xA;adjusted.prior = prior.count * lib.size / ave.lib&#xA;# Update the library sizes&#xA;adjusted.lib.size = lib.size + 2*adjusted.prior&#xA;# Now we can compute the CPM&#xA;customCPM = t((log(t(df) + adjusted.prior) - log(adjusted.lib.size) + log(1000000))/log(2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The matrix transposition looks really ugly in the last step, but it's needed so R adds things like &lt;code&gt;adjusted.prior&lt;/code&gt; to each row of &lt;code&gt;df&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the prior count is only used if you compute log(CPM), it's unused otherwise.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-09-06T08:34:00.373" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="4992" PostTypeId="2" ParentId="4984" CreationDate="2018-09-06T14:18:11.047" Score="3" Body="&lt;p&gt;You may try &lt;a href=&quot;https://codeocean.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;Code Ocean&lt;/a&gt;. Currently, it is not that easy to use IMHO, but one of their tech leads promises to substantially improve it towards the year end.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-09-06T14:18:11.047" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4993" PostTypeId="1" AcceptedAnswerId="4994" CreationDate="2018-09-06T20:16:15.460" Score="2" ViewCount="185" Body="&lt;p&gt;I thought that &lt;code&gt;UMI&lt;/code&gt; counts are always integers, but when I opened several datasets provided by &lt;code&gt;GEO&lt;/code&gt; I got confused because in some articles they provide &lt;code&gt;counts&lt;/code&gt; that are supposedly non-normalized raw &lt;code&gt;UMI&lt;/code&gt; count values, but they are not integers. For instance: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM2043231&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSM2043231&lt;/a&gt; or &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE77067&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSE77067&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anyone clarify, what these people mean by &lt;code&gt;abundance measurements&lt;/code&gt;? &lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2018-09-07T07:14:40.983" LastActivityDate="2018-09-07T07:14:40.983" Title="Article GEO gives counts that are not integers. Should UMI counts be integers?" Tags="|scrnaseq|umi|geo|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4994" PostTypeId="2" ParentId="4993" CreationDate="2018-09-06T20:27:39.010" Score="3" Body="&lt;p&gt;The data processing section doesn't mention using UMIs at all, so it's likely there were none (whether they'll be present depends on how the libraries are made, with lower throughput methods being less likely to include them). The values are stated as being expected counts from &lt;a href=&quot;https://deweylab.github.io/RSEM/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RSEM&lt;/a&gt;, which is why they're not always integers.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-09-06T20:27:39.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4995" PostTypeId="2" ParentId="4983" CreationDate="2018-09-07T09:30:14.340" Score="4" Body="&lt;p&gt;Could you give an example of names of input files and output files of &lt;code&gt;normalize.py&lt;/code&gt; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me that your case could be similar to the one described about the &lt;code&gt;dynamic&lt;/code&gt; output:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#dynamic-files&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#dynamic-files&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here you would probably use it for the output of &lt;code&gt;normalization&lt;/code&gt; and for the input of &lt;code&gt;all&lt;/code&gt; (with a wildcard different from &lt;code&gt;hdf5_file&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I haven't had the opportunity to use this feature so far.&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2018-09-07T09:30:14.340" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="4996" PostTypeId="1" CreationDate="2018-09-07T11:00:26.393" Score="3" ViewCount="236" Body="&lt;p&gt;I want to get coordinates of human genes from my list (consisting of hgnc genes id) using GenomicFeatures and TxDb.Hsapiens.UCSC.hg19.knownGene R packages from Bioconductor.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(TxDb.Hsapiens.UCSC.hg19.knownGene)&#xA;txdb=(TxDb.Hsapiens.UCSC.hg19.knownGene)&#xA;&#xA;my_genes = c(&quot;INO80&quot;,&quot;NASP&quot;,&quot;INO80D&quot;,&quot;SMARCA1&quot;)&#xA;&#xA;select(txdb, keys = my_genes,&#xA;   columns=c(&quot;TXCHROM&quot;,&quot;TXSTART&quot;,&quot;TXEND&quot;,&quot;TXSTRAND&quot;), &#xA;   keytype=&quot;GENEID&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, it doesn't' work because txdb doesn't take hgnc identifiers; how can it be solved? I couldn't find any appropriate keytype that will support hgnc and not sure how to match hgnc id I have and GENEID from txdb.&lt;/p&gt;&#xA;" OwnerUserId="2876" LastActivityDate="2024-03-20T10:05:40.730" Title="Get genomic coordinates using GenomicFeatures by HGNC gene names" Tags="|r|bioconductor|sequence-annotation|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4997" PostTypeId="1" AcceptedAnswerId="5153" CreationDate="2018-09-07T19:58:22.437" Score="3" ViewCount="545" Body="&lt;p&gt;I was working on certain program using python and I have been using a protein aligned sequence file in two formats, phylip (.phy) and clustal (.aln).&#xA;Example clustal file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CLUSTAL 2.1 multiple sequence alignment&#xA;&#xA;&#xA;    Homo            -------------------------------------------MEMQDLTSPHSRLSGSS&#xA;    Macaca          -------------------------------------------MEMQDLTSPHSRLSGSS&#xA;    Canis           REWSSARPERSKGRRKPVDAAAVSAVQTSQTSSDVAVSSSCRSMEMQDLTSPHSRLSGSS&#xA;    Mus             -------------------------------------------MEMQDLTSPHSRLSGSS&#xA;    Rattus          -------------------------------------------MEMQDLTSPHSRLSGSS&#xA;                                                               *****************&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Example phylip file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     3   120&#xA;Apis       MPWIKCLGGG GGKARRGKPL SVSQPIHLLV KSEFEPQCHV FRTKQLRKPR &#xA;Branchiost ---------- ---------- ---------- ---------- ---------- &#xA;Nematostel ---------- ---------- ---------- ---------- ---------- &#xA;&#xA;           PCHLCHQAVI KQASCCRVCK YICHKTCEDK GARFRDTSQQ IHKQWQADPA &#xA;           ---------- ---------- ---------- ---------- ---------- &#xA;           ---------- ---------- ---------- ---------- ---------- &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to navigate through the file in a way that ;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If the columns of the file have same amino acid, it saves and prints the amino acid that exists in the column.&lt;/li&gt;&#xA;&lt;li&gt;I need to know how can I search the columns by using the amino acids, if particular group of amino acids exist in that column or not. For example, if 'STA'or 'HY' or 'FVLIM' or 'NDEQHK' (with no '-') exists in any of the columns, then put a '@' in the file at the end of that column. &#xA;Following is the code I have been trying to manipulate the file with the following program:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Code below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;alignment = open (&quot;example.aln&quot;)&#xA;a = b = c = 0 &#xA;for col in (alignment): &#xA;    num = len(set(col)) &#xA;        if num == 1 and col[0] != '-':&#xA;            print (num)&#xA;        a += 1&#xA;        elif num &amp;gt; 1 and '-' not in col:&#xA;        print (num)&#xA;        b += 1 elif '-' in col:  # assumes 1 or more dashes&#xA;        c += 1&#xA;        print('Number of columns with the same amino acid: {}\n'&#xA;        'Number of columns with at least 2 amino acids (no gaps): {}\n'&#xA;        'Number of columns with one/more gaps: {}'&#xA;        .format(a, b, c))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These variables return the occurrences of amino acids in the columns of the file but I don't understand how to search using the amino acids.&#xA;What is the best way to iterate over the file and write into a new file with results using python? Should I use python switch statement?&lt;/p&gt;&#xA;" OwnerUserId="3142" LastEditorUserId="3142" LastEditDate="2018-09-10T16:56:33.050" LastActivityDate="2018-10-11T04:50:03.577" Title="How to iterate protein sequences using amino acids?" Tags="|python|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="4998" PostTypeId="1" AcceptedAnswerId="5037" CreationDate="2018-09-08T14:46:10.380" Score="2" ViewCount="71" Body="&lt;p&gt;I would like to know the genomic coordinates of a proteins ligand-binding pocket.  For example, on &lt;a href=&quot;http://www.rcsb.org/structure/1OG5&quot; rel=&quot;nofollow noreferrer&quot;&gt;this PDB page&lt;/a&gt; you can see that there is xray crystallography data for how CYP2C9 interacts with two ligands. I would like to know which amino acids form the space on the protein that binds the ligands, then translate (de-translate?) that back to the nucleotide sequence. Is anyone familiar with either a database that has this information or how to extract the ligand interaction data from the PDB files?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="59" LastActivityDate="2018-09-13T19:23:14.223" Title="Genomic coordinates of predicted binding pocket" Tags="|proteins|genomics|drugs|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="4999" PostTypeId="2" ParentId="4998" CreationDate="2018-09-08T21:47:07.860" Score="1" Body="&lt;p&gt;That PDB page includes a publication reference and abstract, containing the following text:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Mammalian CYP450s recognize and metabolize diverse xenobiotics such as&#xA;drug molecules, environmental compounds and pollutants.&lt;/p&gt;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&lt;p&gt;Here we describe the crystal structure of a human CYP450, CYP2C9, both unliganded and in complex with the anti-coagulant drug warfarin.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genome/gdv/browser/?context=genome&amp;amp;acc=GCF_000001405.38&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's&lt;/a&gt; the genomic context for CYP2C9, with Ensembl page &lt;a href=&quot;https://asia.ensembl.org/Homo_sapiens/Gene/Summary?g=ENSG00000138109&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The binding pocket images from the paper &lt;a href=&quot;https://media.springernature.com/m685/nature-assets/nature/journal/v424/n6947/images/nature01862-f1.2.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://media.springernature.com/m685/nature-assets/nature/journal/v424/n6947/images/nature01862-f2.2.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; indicate that the following amino acids are potentially involved in the interaction:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Arg97&lt;/li&gt;&#xA;&lt;li&gt;Phe100&lt;/li&gt;&#xA;&lt;li&gt;Ala103&lt;/li&gt;&#xA;&lt;li&gt;Val113&lt;/li&gt;&#xA;&lt;li&gt;Phe114&lt;/li&gt;&#xA;&lt;li&gt;Asn217&lt;/li&gt;&#xA;&lt;li&gt;Pro367&lt;/li&gt;&#xA;&lt;li&gt;Phe476&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Using the filtering from the &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTables&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC table browser&lt;/a&gt; (genome: Human; group: Genes &amp;amp; Gene Predictions; Basic table; Region: genome; Identifiers: CYP2C9; Output format: CDS fasta alignment), I get the following amino acid sequences respectively for the exons in CYP2C9. I have added in the protein reference range to the end, with the help of the R &lt;code&gt;cumsum&lt;/code&gt; function applied to the region lengths, &lt;code&gt;cumsum(c(56,54,50,54,59,47,63,47,61))&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;ENST00000260682.7_hg38_1_9 56 0 0 chr10:94938683-94938850+ [1-56]&#xA;MDSLVVLVLCLSCLLLLSLWRQSSGRGKLPPGPTPLPVIGNILQIGIKDISKSLTN&#xA;&amp;gt;ENST00000260682.7_hg38_2_9 54 0 1 chr10:94941858-94942020+ [57-110]&#xA;LSKVYGPVFTLYFGLKPIVVLHGYEAVKEALIDLGEEFSGRGIFPLAERANRGF&#xA;&amp;gt;ENST00000260682.7_hg38_3_9 50 1 1 chr10:94942192-94942341+ [111-160]&#xA;GIVFSNGKKWKEIRRFSLMTLRNFGMGKRSIEDRVQEEARCLVEELRKTK&#xA;&amp;gt;ENST00000260682.7_hg38_4_9 54 1 0 chr10:94947779-94947939+ [161-214]&#xA;ASPCDPTFILGCAPCNVICSIIFHKRFDYKDQQFLNLMEKLNENIKILSSPWIQ&#xA;&amp;gt;ENST00000260682.7_hg38_5_9 59 0 0 chr10:94949108-94949284+ [215-273]&#xA;ICNNFSPIIDYFPGTHNKLLKNVAFMKSYILEKVKEHQESMDMNNPQDFIDCFLMKMEK&#xA;&amp;gt;ENST00000260682.7_hg38_6_9 47 0 1 chr10:94972104-94972245+ [274-320]&#xA;EKHNQPSEFTIESLENTAVDLFGAGTETTSTTLRYALLLLLKHPEVT&#xA;&amp;gt;ENST00000260682.7_hg38_7_9 63 1 0 chr10:94981183-94981370+ [321-383]&#xA;AKVQEEIERVIGRNRSPCMQDRSHMPYTDAVVHEVQRYIDLLPTSLPHAVTCDIKFRNYLIPK&#xA;&amp;gt;ENST00000260682.7_hg38_8_9 47 0 1 chr10:94986033-94986174+ [384-430]&#xA;GTTILISLTSVLHDNKEFPNPEMFDPHHFLDEGGNFKKSKYFMPFSA&#xA;&amp;gt;ENST00000260682.7_hg38_9_9 61 1 0 chr10:94988847-94989028+ [431-491]&#xA;GKRICVGEALAGMELFLFLTSILQNFNLKSLVDPKNLDTTPVVNGFASVPPFYQLCFIPVZ&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And these nucleic acid sequences:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;ENST00000260682.7_hg38_1_9 168 0 0 chr10:94938683-94938850+&#xA;ATGGATTCTCTTGTGGTCCTTGTGCTCTGTCTCTCATGTTTGCTTCTCCTTTCACTCTGGAGACAGAGCTCTGGGAGAGGAAAACTCCCTCCTGGCCCCACTCCTCTCCCAGTGATTGGAAATATCCTACAGATAGGTATTAAGGACATCAGCAAATCCTTAACCAAT&#xA;&amp;gt;ENST00000260682.7_hg38_2_9 163 0 1 chr10:94941858-94942020+&#xA;CTCTCAAAGGTCTATGGCCCTGTGTTCACTCTGTATTTTGGCCTGAAACCCATAGTGGTGCTGCATGGATATGAAGCAGTGAAGGAAGCCCTGATTGATCTTGGAGAGGAGTTTTCTGGAAGAGGCATTTTCCCACTGGCTGAAAGAGCTAACAGAGGATTTG&#xA;&amp;gt;ENST00000260682.7_hg38_3_9 150 1 1 chr10:94942192-94942341+&#xA;GAATTGTTTTCAGCAATGGAAAGAAATGGAAGGAGATCCGGCGTTTCTCCCTCATGACGCTGCGGAATTTTGGGATGGGGAAGAGGAGCATTGAGGACCGTGTTCAAGAGGAAGCCCGCTGCCTTGTGGAGGAGTTGAGAAAAACCAAGG&#xA;&amp;gt;ENST00000260682.7_hg38_4_9 161 1 0 chr10:94947779-94947939+&#xA;CCTCACCCTGTGATCCCACTTTCATCCTGGGCTGTGCTCCCTGCAATGTGATCTGCTCCATTATTTTCCATAAACGTTTTGATTATAAAGATCAGCAATTTCTTAACTTAATGGAAAAGTTGAATGAAAACATCAAGATTTTGAGCAGCCCCTGGATCCAG&#xA;&amp;gt;ENST00000260682.7_hg38_5_9 177 0 0 chr10:94949108-94949284+&#xA;ATCTGCAATAATTTTTCTCCTATCATTGATTACTTCCCGGGAACTCACAACAAATTACTTAAAAACGTTGCTTTTATGAAAAGTTATATTTTGGAAAAAGTAAAAGAACACCAAGAATCAATGGACATGAACAACCCTCAGGACTTTATTGATTGCTTCCTGATGAAAATGGAGAAG&#xA;&amp;gt;ENST00000260682.7_hg38_6_9 142 0 1 chr10:94972104-94972245+&#xA;GAAAAGCACAACCAACCATCTGAATTTACTATTGAAAGCTTGGAAAACACTGCAGTTGACTTGTTTGGAGCTGGGACAGAGACGACAAGCACAACCCTGAGATATGCTCTCCTTCTCCTGCTGAAGCACCCAGAGGTCACAG&#xA;&amp;gt;ENST00000260682.7_hg38_7_9 188 1 0 chr10:94981183-94981370+&#xA;CTAAAGTCCAGGAAGAGATTGAACGTGTGATTGGCAGAAACCGGAGCCCCTGCATGCAAGACAGGAGCCACATGCCCTACACAGATGCTGTGGTGCACGAGGTCCAGAGATACATTGACCTTCTCCCCACCAGCCTGCCCCATGCAGTGACCTGTGACATTAAATTCAGAAACTATCTCATTCCCAAG&#xA;&amp;gt;ENST00000260682.7_hg38_8_9 142 0 1 chr10:94986033-94986174+&#xA;GGCACAACCATATTAATTTCCCTGACTTCTGTGCTACATGACAACAAAGAATTTCCCAACCCAGAGATGTTTGACCCTCATCACTTTCTGGATGAAGGTGGCAATTTTAAGAAAAGTAAATACTTCATGCCTTTCTCAGCAG&#xA;&amp;gt;ENST00000260682.7_hg38_9_9 182 1 0 chr10:94988847-94989028+&#xA;GAAAACGGATTTGTGTGGGAGAAGCCCTGGCCGGCATGGAGCTGTTTTTATTCCTGACCTCCATTTTACAGAACTTTAACCTGAAATCTCTGGTTGACCCAAAGAACCTTGACACCACTCCAGTTGTCAATGGATTTGCCTCTGTGCCGCCCTTCTACCAGCTGTGCTTCATTCCTGTCTGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Associating the pocket protein amino acid numbers in exons 2,3,5,7,9 with genomic sequence using these FASTA sequences is left as an exercise for the reader.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-09-08T21:47:07.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5000" PostTypeId="1" AcceptedAnswerId="5016" CreationDate="2018-09-08T22:43:57.147" Score="2" ViewCount="749" Body="&lt;p&gt;I tried to filter out FASTQ reads which are shorter than 259 bp with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bioawk -cfastx 'length() &amp;gt;= 259 {print &quot;@&quot; &quot; &quot; &quot;\n&quot;&quot;\n+\n&quot;}' good/SZ005_NoIndex_L002_R1_009.good.fq.gz \&#xA;  | gzip &amp;gt;  good-filtered/SZ005_NoIndex_L002_R1_009.good-filtered.fq.gz`. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the output was wrong:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@ &#xA;&#xA;+&#xA;&#xA;@ &#xA;&#xA;+&#xA;&#xA;@ &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What did I miss?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance,&lt;/p&gt;&#xA;" OwnerUserId="3329" LastEditorUserId="964" LastEditDate="2018-09-11T14:48:42.753" LastActivityDate="2018-09-11T14:48:42.753" Title="filter out FASTQ reads which are shorter" Tags="|fastq|bioawk|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5001" PostTypeId="2" ParentId="5000" CreationDate="2018-09-09T00:21:08.497" Score="3" Body="&lt;p&gt;You need to use variable &lt;code&gt;$seq&lt;/code&gt;. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bioawk -cfastx 'length($seq)&amp;gt;=259{print &quot;@&quot;$name&quot;\n&quot;$seq&quot;\n+\n&quot;$qual}' test.fq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are also more convenient tools like &lt;a href=&quot;https://github.com/lh3/seqtk&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqtk&lt;/a&gt; and &lt;a href=&quot;https://github.com/shenwei356/seqkit&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqkit&lt;/a&gt;. With those, you may&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seqtk seq -L260 test.fq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-09-09T00:21:08.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5002" PostTypeId="1" AcceptedAnswerId="7125" CreationDate="2018-09-09T05:39:17.407" Score="3" ViewCount="860" Body="&lt;p&gt;I'm using RepeatMasker to detect, classify the Transposable elements. My Input is a eukaryotic non-reference genome.&#xA;I made a run via RepeatMasker many times to Mask the TEs, but return 0 Annotation tables.&#xA;Further, I used a different &lt;code&gt;-species&lt;/code&gt; option each time, but Annotation results contain only short and simple repeats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I have to create a unique library to detect them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The command I used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./RepeatMasker -no_is -noint - species mammal Genome.fna&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sequences:         32573&#xA;total length: 2004063690 bp  (1981588036 bp excl N/X-runs)&#xA;GC level:         41.28 %&#xA;bases masked:   33282890 bp ( 1.66 %)&#xA;==================================================&#xA;               number of      length   percentage&#xA;               elements*    occupied  of sequence&#xA;--------------------------------------------------&#xA;SINEs:                0            0 bp    0.00 %&#xA;      ALUs            0            0 bp    0.00 %&#xA;      MIRs            0            0 bp    0.00 %&#xA;&#xA;LINEs:                0            0 bp    0.00 %&#xA;      LINE1           0            0 bp    0.00 %&#xA;      LINE2           0            0 bp    0.00 %&#xA;      L3/CR1          0            0 bp    0.00 %&#xA;&#xA;LTR elements:         0            0 bp    0.00 %&#xA;      ERVL            0            0 bp    0.00 %&#xA;      ERVL-MaLRs      0            0 bp    0.00 %&#xA;      ERV_classI      0            0 bp    0.00 %&#xA;      ERV_classII     0            0 bp    0.00 %&#xA;&#xA;DNA elements:         0            0 bp    0.00 %&#xA;     hAT-Charlie      0            0 bp    0.00 %&#xA;     TcMar-Tigger     0            0 bp    0.00 %&#xA;&#xA;Unclassified:         0            0 bp    0.00 %&#xA;&#xA;Total interspersed repeats:        0 bp    0.00 %&#xA;&#xA;&#xA;Small RNA:            0            0 bp    0.00 %&#xA;&#xA;Satellites:           1          267 bp    0.00 %&#xA;Simple repeats:  663679     27953718 bp    1.39 %&#xA;Low complexity:  123705      6154714 bp    0.31 %&#xA;==================================================&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3332" LastEditorUserId="3332" LastEditDate="2020-05-30T22:53:51.963" LastActivityDate="2023-05-30T21:47:29.100" Title="Finding transposable elements using RepeatMasker" Tags="|repeat-elements|transposable-elements|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="5003" PostTypeId="1" CreationDate="2018-09-09T10:12:44.573" Score="2" ViewCount="109" Body="&lt;p&gt;I have multiple large sample datasets in matrix format (each has 15000 rows and 5-50 columns) corresponding to different experiments. Each matrix contains the same number of samples(rows) but the variables(columns) are not the same. My objective is to cluster the samples on the basis of all the experiments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to use Unsupervised multiple kernel learning (UMKL) to integrate the datasets followed by kernelPCA using &quot;mixkernel&quot; package in R (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/29077792&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pubmed/29077792&lt;/a&gt;). The UMKL step calculates kernels for each dataset and combines the kernels using 3 different approaches: 1) calculating a consensus kernel from multiple kernels 2) calculating a sparse kernel preserving the original topology of the data 3) calculating a full kernel preserving the original topology of the data&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The kernel calculation step was fine but the kernel integration step (all 3 approaches) runs very long and my computer hangs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way to handle this problem? More specifically, is there any way to handle multiple kernel integration for large sample datasets?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion alternative to using kernel methods will also work.&lt;/p&gt;&#xA;" OwnerUserId="284" LastEditorUserId="77" LastEditDate="2018-09-09T20:48:47.343" LastActivityDate="2018-10-10T10:00:17.270" Title="How to combine multiple kernels of large sample datasets?" Tags="|r|clustering|multi-omics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5004" PostTypeId="1" CreationDate="2018-09-09T10:57:33.337" Score="5" ViewCount="328" Body="&lt;p&gt;I am working on protein sequence data files for reconstructing phylogenetic tree and I need to generate all &lt;strong&gt;NNI-neighbours&lt;/strong&gt; of the tree (two trees are NNI-neighbours if one can be transformed into another by one nearest neighbour interchange operation). Is there any python &lt;strong&gt;package/sorce code&lt;/strong&gt; I can access to, which can retrieve all NNI neighbors of a tree.&#xA;Earlier I have used biopython 1.72 to generate tree from my dataset. But for now I'm looking for a python package which can do NNI search on my tree and return all NNI-neighbors.&lt;/p&gt;&#xA;" OwnerUserId="3142" LastActivityDate="2018-09-10T10:34:18.140" Title="python package for NNI neighbors" Tags="|python|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5005" PostTypeId="2" ParentId="4989" CreationDate="2018-09-09T11:56:58.633" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://last.cbrc.jp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LAST&lt;/a&gt; has given the best results for me when I've tried to do this, although I agree with @user172818 that it's not a good idea to map really short reads. This is due to a combination of natural sequence duplication in long DNA sequences (e.g. see &lt;a href=&quot;https://twitter.com/gringene_bio/status/1037561956066975744&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), as well as abundant base calling differences present in single-molecule sequencing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Minimising error is not necessarily going to be the best option, and concentrating only on unique hits will miss a lot of signal. There are frequently multiple identically plausible positions, even at zero error, and the long reads will have their own associated errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also find the limitations a little odd. Canu can do read-level correction on long reads based on overlapping reads, and if assembly &lt;em&gt;is&lt;/em&gt; possible, then nanopolish can be used to correct nanopore reads for systematic base-calling error introduced by the methylation of unamplified template.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-09-09T11:56:58.633" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5007" PostTypeId="2" ParentId="5003" CreationDate="2018-09-10T08:28:03.350" Score="0" Body="&lt;p&gt;You can use regularized general canonical correlation analysis to cluster those matrices. I would suggest &lt;a href=&quot;https://cran.r-project.org/package=RGCCA&quot; rel=&quot;nofollow noreferrer&quot;&gt;RGCCA&lt;/a&gt;, which allows to do this for an arbitrary number of matrices but it  doesn't use kernels. &#xA;There has been some efforts to use kernels on the canonical correlation analysis but there isn't any implementation (yet), you can read more on the related literature, specially the &lt;a href=&quot;https://hal-supelec.archives-ouvertes.fr/file/index/docid/553602/filename/JDS2010.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kernel Generalized Canonical Correlation Analysis&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However in this kind of analysis you will need to come up with a design of how those matrices are related between them. It is not like a PCA where you give some data and you get some data back that summarize the matrix. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-09-10T08:28:03.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5008" PostTypeId="2" ParentId="5003" CreationDate="2018-09-10T09:07:12.203" Score="1" Body="&lt;p&gt;Welcome to Bioinformatics Stackexchange @SD1024. A recently published algorithm known as MOFA (Multi-Omics Factor Analysis, &lt;a href=&quot;http://msb.embopress.org/content/14/6/e8124&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, &lt;a href=&quot;https://github.com/bioFAM/MOFA&quot; rel=&quot;nofollow noreferrer&quot;&gt;github&lt;/a&gt;) is generating a lot of interest, and is designed answer exactly the sort of biological question you are describing. It claims to extract axes from multiple matrices with overlapping samples, but not necessarily overlapping features and handle missing data elegantly. I havn't tried it myself you, but I'm quite excited to do so. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-09-10T09:07:12.203" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5009" PostTypeId="2" ParentId="5004" CreationDate="2018-09-10T10:34:18.140" Score="2" Body="&lt;p&gt;Both the &lt;a href=&quot;http://etetoolkit.org/docs/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ETE Toolkit&lt;/a&gt; and &lt;a href=&quot;https://dendropy.org/#&quot; rel=&quot;nofollow noreferrer&quot;&gt;DendroPy&lt;/a&gt; offer methods of node removal and reattachment - I would think it would be easy enough to hack together an NNI movement function with them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also investigate the source code of &lt;a href=&quot;https://biopython.org/wiki/Phylo&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioPython's&lt;/a&gt; &lt;code&gt;NNITreeSearcher()&lt;/code&gt; function.&lt;/p&gt;&#xA;" OwnerUserId="2841" LastActivityDate="2018-09-10T10:34:18.140" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5012" PostTypeId="1" AcceptedAnswerId="5014" CreationDate="2018-09-10T16:35:09.223" Score="0" ViewCount="288" Body="&lt;p&gt;I want to visualise a network in a way that the nodes are along a line (and the edges are not on that line).&#xA;I have included a simple example below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xJ9vo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xJ9vo.png&quot; alt=&quot;Network on a line&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which network visualisation programs (Cytoscape, Gephi, R igraph etc) do have this layout option?&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2022-06-11T13:05:24.270" Title="Visualize network (graph) using &quot;line&quot; layout" Tags="|networks|visualization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5013" PostTypeId="1" AcceptedAnswerId="5066" CreationDate="2018-09-10T16:53:28.273" Score="3" ViewCount="421" Body="&lt;p&gt;I am looking to show how a primer is consistent among some genomic data. I have a primer of about 23bp and looking to compare it to about 5000 genomic sequences of 10kb. I am unsure how to format it the way I need to, since I am just trying to show the nucleotide differences of the primer to the genetic sequences. Here is what I need:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;     1. Cut out the area that my primer is located and about 20bp down each end. &#xA;&amp;gt;     2. Show only the bases that are different from my primer in my analysis.&#xA;&amp;gt;      ex: Primer: -----------ATGTGGAAGCAAATATCAAATGA---------&#xA;&amp;gt;          Gene:   ATGACCATACG----C--------------T---ATCGTAGGG&#xA;&amp;gt;                  ATGAGCATACC-----A----T--------T---TTCGAACGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The data I am using is all dengue sequences (all serotypes) and the primer with the following code: ATGTGGAAGCAAATATCAAATGA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using Biostrings, msa, and seqinr. I use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genomes/VirusVariation/Database/nph-select.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;ncbi&lt;/a&gt; to get the genetic sequences and using FASTA files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Solution:&#xA;I used MAFFT to align the data, working very efficiently and put it into a fasta file. Then I cut the nucleotides I did not need by:&#xA;1. Finding the part i wanted and cutting 10 nucleotides before it and after it.&#xA;2. Putting only the [start:end] into a new list, which I will then manipulate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am still having difficulty finding a way of showing my data nicely because I can't find a suitable package. Any advice?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: &#xA;Hi all,&#xA;Just wanted to show the code I made to solve the problem. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    #So the we will be using a dataframe that has the sequences as a string&#xA;    #Then I created a temporary dataframe to store each sequence, with the primer at the top&#xA;&#xA;    tempDF &amp;lt;- data.frame(matrix(primer1, nrow = 1, ncol = 1), &#xA;    stringsAsFactors = FALSE)&#xA;&#xA;    for(i in sequenceAsString){&#xA;      #Temporary vector for each sequence after compared to primer. &#xA;&#xA;      temporary1 &amp;lt;- &quot;&quot;&#xA;      temporary1 &amp;lt;- paste(temporary1, substring(i,1,11), sep = &quot;&quot;) &#xA;&#xA;      for(j in 12:(nchar(i)-20)){&#xA;        if(substring(i,j,j) == substring(toString(reversePrimer1), j-12,j-12)){&#xA;          temporary1 &amp;lt;- paste(temporary1, &quot;-&quot;, sep = &quot;&quot;)&#xA;        }else{&#xA;          temporary1 &amp;lt;- paste(temporary1, substring(i,j,j), sep = &quot;&quot;)&#xA;        }&#xA;      }&#xA;      temporary1 &amp;lt;- paste(temporary1, substring(i,nchar(i)-19,nchar(i)), sep = &quot;&quot;) &#xA;      tempDF &amp;lt;- rbind(tempDF,temporary1)&#xA;    }  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, I essentially take the sequence I chose apart and compare each character to that of the reverseprimer. I then replace the ones that are the same with an '-'&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks everyone for their help!&lt;/p&gt;&#xA;" OwnerUserId="3340" LastEditorUserId="3340" LastEditDate="2018-10-22T23:48:54.773" LastActivityDate="2018-10-22T23:48:54.773" Title="Aligning sequence and comparing it against primer" Tags="|r|bioconductor|sequence-alignment|primer|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5014" PostTypeId="2" ParentId="5012" CreationDate="2018-09-10T18:48:11.900" Score="2" Body="&lt;p&gt;That plot is called an Arc diagram&#xA;you can google that and use the package or program that best suits you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are 2 examples of the use of Arc diagrams in R:&lt;br&gt;&#xA;&lt;a href=&quot;http://www.gastonsanchez.com/visually-enforced/got-plot/how-to/2013/02/02/Arc-Diagrams-in-R-Les-Miserables/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Example 1&lt;/a&gt;: &lt;a href=&quot;https://i.stack.imgur.com/hkiAS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hkiAS.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sthda.com/english/articles/33-social-network-analysis/135-network-visualization-essentials-in-r/#arc-diagram-layout&quot; rel=&quot;nofollow noreferrer&quot;&gt;Example 2&lt;/a&gt;: &lt;a href=&quot;https://i.stack.imgur.com/HQX6k.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HQX6k.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2675" LastEditorUserId="48" LastEditDate="2018-09-12T07:36:08.897" LastActivityDate="2018-09-12T07:36:08.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5015" PostTypeId="2" ParentId="4957" CreationDate="2018-09-11T05:41:16.137" Score="4" Body="&lt;p&gt;As I've studies the &lt;code&gt;'TreeConstruction'&lt;/code&gt; module of Biopython, I came to know that &lt;code&gt;'_get_neighbors'&lt;/code&gt; method is the one that is used to get all neighbor trees of the given tree. This is all I've concluded from my findings. If anyone wants to add more, that'll be positively appreciated.&lt;/p&gt;&#xA;" OwnerUserId="3142" LastActivityDate="2018-09-11T05:41:16.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5016" PostTypeId="2" ParentId="5000" CreationDate="2018-09-11T08:17:27.123" Score="0" Body="&lt;p&gt;If you have paired-end reads, this solution keeps the two files in-sync (i.e. discard pairs where one of the two reads is shorter than 259). Also, it uses only Unix tools without requiring external programs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;paste &amp;lt;(zcat S1_L001_R1_001.fastq.gz | paste - - - -) &amp;lt;(zcat S1_L001_R2_001.fastq.gz | paste - - - -) \&#xA;| awk -v FS='\t' 'length($2) &amp;gt;= 259 &amp;amp;&amp;amp; length($6) &amp;gt;= 259' \&#xA;| tee &amp;gt;(cut -f 1-4 | tr '\t' '\n' | gzip &amp;gt; r1.fq.gz) \&#xA;| cut -f 5-8 | tr '\t' '\n' | gzip &amp;gt; r2.fq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="339" LastActivityDate="2018-09-11T08:17:27.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5017" PostTypeId="1" AcceptedAnswerId="5018" CreationDate="2018-09-11T08:30:16.157" Score="6" ViewCount="325" Body="&lt;p&gt;I am learning about applying Markov model to sequence alignment. The prof says that the transition probabilities from a gap-residue alignment to a residue-gap alignment and vice versa are both 0. Is there any biological/mathematical reason behind this statement? Why are the (X,Y) and (Y,X) cell 0? This is a lecture slide of lecture 1, week 4 of the &quot;Bioinformatics: Introdcution and Methods&quot; course on coursera.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/NSWgr.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NSWgr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3345" LastActivityDate="2018-09-13T11:52:40.520" Title="Sequence alignment using Markov Model" Tags="|sequence-alignment|hidden-markov-models|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5018" PostTypeId="2" ParentId="5017" CreationDate="2018-09-11T09:09:48.950" Score="6" Body="&lt;p&gt;If I understand your question correctly, then I think for case of pairwise alignment, there is a simple explanation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe the key insight is that: &lt;em&gt;a mismatch should always score better than a gap&lt;/em&gt;.*&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This follows biologically since the insertion/deletion (indel) rate is roughly 1/10th that of the substitution rate (i.e. the occurrence of single nucleotide changes), at least in vertebrates. (This varies across the tree of life but I think the substitution rate virtually always exceeds the indel rate.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To understand why this matters, consider an example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ATG-AG&#xA;ATGT-G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is an 'impossible alignment' under the probabilities you gave since here we have a transition from a gap-residue alignment to a residue-gap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, under our assumption that mismatches are more likely biologically than indels, the correct alignment should be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ATGAG&#xA;ATGTG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Indeed, the latter does look like a better alignment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This also follows for more complex examples, so this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ATG--AAG&#xA;ATGTT-AG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Becomes this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ATG-AAG&#xA;ATGTTAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Or this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ATGA-AG&#xA;ATGTTAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;* Strictly, I mean a substitution should score better than an indel (with the associated gap opening and extension penalties). In fact, for the assumption to always be true, a run of mismatches should still score worse than a single indel. This may not always be a correct assumption, consider this example below, is the true alignment case 1) or 2) or something else? Or is in fact a global alignment bad here and this should be split into 2 local alignments? Is there a likely biological mutational event that could explain this? I ask these questions just to point out it is not black-and-white, I don't have clear answers&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CGTACGTAGAGGAATGCCCCCCCCC--------AGCAACGTAGCAT     &#xA;CGTACGTAGAGGAATG---------TTTTTTTTAGCAACGTAGCAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;2)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CGTACGTAGAGGAATGCCCCCCCCCAGCAACGTAGCAT     &#xA;CGTACGTAGAGGAATGTTTTTTTT-AGCAACGTAGCAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastEditorUserId="104" LastEditDate="2018-09-12T08:27:09.803" LastActivityDate="2018-09-12T08:27:09.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5019" PostTypeId="1" CreationDate="2018-09-11T10:33:41.023" Score="0" ViewCount="181" Body="&lt;p&gt;I'm trying to install Bioconductor packages with Microsoft R Open 3.5.0 seems to fail not sure what is the issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I'm trying to do &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;source(&quot;https://bioconductor.org/biocLite.R&quot;)&#xA;biocLite(&quot;BiocInstaller&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I keep getting an error about &quot;curl&quot; had non-zero exit. &#xA;How can I install the packages of Bioconductor?&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="73" LastEditDate="2018-09-19T22:04:06.797" LastActivityDate="2018-09-19T22:04:06.797" Title="Curl error when installing Bioconductor packages with Microsoft R Open 3.5.0" Tags="|r|bioconductor|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5021" PostTypeId="1" CreationDate="2018-09-11T13:01:49.187" Score="5" ViewCount="98" Body="&lt;p&gt;I would like to detect a change in expression of a molecule present on a cell type by flow cytometry. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming I am able to detect, using an antibody, a signal that represents the amount of the molecule I'm interested into. Also assume that the same experiment will have some noise (ideally near to zero) represented by an isotype antibody. What if the noise is somehow higher than a minimal threshold (i.e. = 1%)? How can I deal with the noise? How can I compare different samples if the noise is higher than the threshold?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking about removing those samples that have a noise above the 1% but is this ok, from a statistical point of view?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility is: can I subtract the noise from the signal and use this information for the final statistical analysis? Is this correct? Is it possible to subtract a signal from another? How is this done?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT just to clarify a bit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a pool of neutrophils which, in resting condition, do not express the molecule on their surface. When stimulated, these cells are supposed to show an increased amount of this molecule (again, on their surface). We would like to quantify the change in the amount of molecule (we used FACS).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We are able to quantify the quantity of surface-molecule but we also get some aspecific signal (the noise due to the isotype antiboby). What we try to do is to keep the isotype signal lower than 1% using a gate-based methodology. What if the isotype signal we get is higher than the threshold (i.e. &gt;1%)? The antibody sometimes work some others don't. What should we do when it returns too much noise? Should we discard the data or should we perform a normalisation over the surface-molecule using the noise?&lt;/p&gt;&#xA;" OwnerUserId="3119" LastEditorUserId="3119" LastEditDate="2018-09-12T09:45:48.033" LastActivityDate="2023-04-25T21:02:00.273" Title="What is the correct way of dealing with the analysis of data from flow cytometry?" Tags="|statistics|single-cell|flow-cytometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5022" PostTypeId="2" ParentId="5019" CreationDate="2018-09-11T13:16:30.810" Score="2" Body="&lt;p&gt;I can't comment yet but from what you said it seems that you need to install additional libraries like the one you mentioned: &quot;curl&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;from the R terminal you should type:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;install.packages(&quot;curl&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then R will install the package. Not sure if there are some other dependencies. If yes R will return you some error message in which there are some package names you need to install.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;of course, after installing all the required packages you should be able to install biocLite too.&lt;/p&gt;&#xA;" OwnerUserId="3119" LastActivityDate="2018-09-11T13:16:30.810" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="5023" PostTypeId="2" ParentId="5002" CreationDate="2018-09-11T14:02:29.930" Score="3" Body="&lt;p&gt;You need to model the repeats in your de novo genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;See:&#xA;&lt;a href=&quot;http://www.repeatmasker.org/RepeatModeler/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.repeatmasker.org/RepeatModeler/&lt;/a&gt; and&#xA;&lt;a href=&quot;https://www.biostars.org/p/154290/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/154290/&lt;/a&gt; (from previous answer)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You first build a database named &quot;name_of_your_database&quot; (insert whatever you want to call it) and run it on the fasta file of your genome (e.g. your_genome.fasta).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Make you find the right paths for the software.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;## build the database&#xA;BuildDatabase -engine ncbi -name name_of_your_database your_genome.fasta&#xA;&#xA;## run the modeler (in this case using ncbi)&#xA;RepeatModeler -database name_of_your_database -engine ncbi&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="383" LastEditorUserId="383" LastEditDate="2018-09-11T14:07:44.673" LastActivityDate="2018-09-11T14:07:44.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5024" PostTypeId="2" ParentId="4890" CreationDate="2018-09-11T14:22:17.090" Score="1" Body="&lt;p&gt;The primary author of Gviz confirmed to me via email that this is not possible:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Gviz will only give you the traditional linear tracks. There are a&#xA;  bunch of solutions available for visualizing circular genomes, most of&#xA;  which are more interactive in nature and not implemented in the rather&#xA;  static R graphics context. GGplot2 has some support for circular axes,&#xA;  too. If you want to stay within the R environment I’d start there.&#xA;  Ggbio is another option providing a somewhat higher-level interface.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="104" LastActivityDate="2018-09-11T14:22:17.090" CommentCount="0" CommunityOwnedDate="2018-09-11T14:22:17.090" ContentLicense="CC BY-SA 4.0" />
  <row Id="5025" PostTypeId="1" CreationDate="2018-09-11T16:49:52.310" Score="1" ViewCount="75" Body="&lt;p&gt;I made a PCA plot and was trying to plot the eigenvenctors in R so that each data point is actually the sample name. All the samples are in column 1. The following are the R commands I am using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    fn &amp;lt;- &quot;Plot.txt&quot;&#xA;    evecdat &amp;lt;- read.table(fn, col.names=c(&quot;Sample&quot;, &quot;PC1&quot;, &quot;PC2&quot;, &quot;PC3&quot;, &quot;PC4&quot;, &quot;PC5&quot;, &quot;PC6&quot;, &quot;PC7&quot;, &quot;PC8&quot;, &quot;PC9&quot;, &quot;PC10&quot;, &quot;pc11&quot;,&quot;pc12&quot;, &quot;pc13&quot;, &quot;pc14&quot;, &quot;pc15&quot;,&quot;pc16&quot;, &quot;pc17&quot;, &quot;pc18&quot;, &quot;pc19&quot;, &quot;pc20&quot;, &quot;pop&quot;, &quot;color&quot;))&#xA;    jpeg(&quot;Plot1.jpeg&quot;, width = 20, height = 20, units = 'in', res = 300)&#xA;    identify(evecdat$PC1, evecdat$PC2  = NULL, labels = seq_along(evecdat$Sample), pos = TRUE, n = length(evecdat$Sample), plot = FALSE, atpen = FALSE, offset = 0.5, tolerance = 0.25, …)&#xA;&#xA;    plot(evecdat$PC1, evecdat$PC2, col=as.vector(evecdat$color), xlab=&quot;PC1&quot;, ylab=&quot;PC2&quot;, pch=&quot;24&quot;, cex.lab=2.5, cex=1.5, cex.axis=1)&#xA;    legend(&quot;topright&quot;, ncol=1, legend=c(&quot;X&quot;, &quot;Y&quot;, &quot;Z&quot;), col=c(&quot;Blue&quot;, &quot;Red&quot;, &quot;Green&quot;), pch=&quot;24&quot;, cex=2, bty = &quot;o&quot;)&#xA;    while (!is.null(dev.list())) dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here, all the row names are under the Sample column name that I defined earlier. &lt;/p&gt;&#xA;" OwnerUserId="2887" LastEditorUserId="48" LastEditDate="2018-09-14T07:11:46.787" LastActivityDate="2018-09-14T07:11:46.787" Title="Error in annotating row names as data points in a PCA plot" Tags="|bioconductor|ggplot2|pca|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5026" PostTypeId="1" CreationDate="2018-09-11T17:26:46.303" Score="8" ViewCount="61" Body="&lt;p&gt;I've converted dog coordinates to human using &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgLiftOver&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC LiftOver&lt;/a&gt;. These are 200bp intergenic regions that are differentially methylated from normal dogs to cancer dogs. I've converted these to human coordinates and found that a lot of them overlap with differentially methylated regions we found in the human model.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is this okay to do?&lt;/li&gt;&#xA;&lt;li&gt;What stringencies should I check or modify, the min ratio of bases needed to overlap?&lt;/li&gt;&#xA;&lt;li&gt;Lastly, is LiftOver taking the 200bp dog sequence and saying that at least 10% of the sequence has to align with a region within the human genome?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I could not find an answer online.&lt;/p&gt;&#xA;" OwnerUserId="3347" LastEditorUserId="2108" LastEditDate="2022-12-19T19:18:38.490" LastActivityDate="2022-12-19T19:58:31.240" Title="dog coordinates (canFam3) to human coordinates (hg19)" Tags="|ngs|methylation|sequence-annotation|liftover|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5028" PostTypeId="1" CreationDate="2018-09-12T00:55:30.780" Score="2" ViewCount="200" Body="&lt;p&gt;I recently tried to reproduce population structure with 1000 Genomes Project data (Phase 3, 2504 people).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to supplementary material,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To reduce the dataset, we used VCFtools to only keep biallelic, non-singleton SNV sites that were a minimum of 2 KB apart from each other. &#xA;  These variants were then merged and converted into PLINK format, giving us 1,286,213 throughout all the autosomes. &#xA;  PLINK was then used to filter the variants further, keeping only variants with a minor allele frequency of at least 0.05, &#xA;  and to convert the variants into binary PED format. After this filter 193,634 sites remained with data from 2,504 individuals in 26 populations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And from &lt;a href=&quot;ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/admixture_files/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp repository&lt;/a&gt;, I found the exact command-line to do it.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;To get biallelic, non-singleton SNVs, all minimum of 2KB apart from each other&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;./vcftools --gzvcf&#xA;ALL.chr1.phase3_shapeit2_mvncall_integrated_v4.20130502.genotypes.vcf.gz&#xA;--thin 2000 --min-alleles 2 --max-alleles 2 --non-ref-ac 2 --plink --chr 1 --out ALL.chr1.phase3_shapeit2_filtered.20141217&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;To get variants with minor allele frequency of at least 0.05&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;./plink --file ALL.wgs.phase3_shapeit2_filtered.20141217.ped --out&#xA;ALL.wgs.phase3_shapeit2_filtered.20141217.maf0.05 --make-bed --maf 0.05&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;To get population structure with ADMIXTURE program (v1.23)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;./admixture ALL.wgs.phase3_shapeit2_filtered.20141217.maf0.05.bed K&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So I applied the same command (although versions of programs were not exactly the same) to the phase 3 data, and got different results (top: from supplementary data, bottom: my reproduction)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/vnwUj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vnwUj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I basically did the same thing as in the supplementary document:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;(for each chromosome \$ch) &lt;code&gt;bcftools view -v snps -O v $gzvcf | vcftools --vcf - --thin 2000 --min-alleles 2 --max-alleles 2 --non-ref-ac 2 --plink --chr ${ch} --out 1KGP_v5a_20130502.chr${ch}&lt;/code&gt; [bcftools-1.9, vcftools 0.1.15]&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;(for each chromosome \$ch) &lt;code&gt;$plink --file 1KGP_v5a_20130502.chr${ch} --maf 0.05 --make-bed --out 1KGP_v5a_20130502.chr${ch}.maf0.05&lt;/code&gt; [plink 1.9b]&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;(merge)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;admixture 1KGP_v5a_20130502.ALL.maf0.05.bed 5&lt;/code&gt; [ADMIXTURE 1.3]&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I would like to know if there's some step I've missed on the process. Otherwise, I cannot understand why the population structure look so different.&lt;/p&gt;&#xA;" OwnerUserId="3350" LastEditorUserId="48" LastEditDate="2018-09-12T07:24:23.637" LastActivityDate="2018-09-12T07:24:23.637" Title="Reproducing population structure from 1000 Genomes Data" Tags="|human-genome|phylogenetics|figure-reproduction|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="5029" PostTypeId="2" ParentId="5017" CreationDate="2018-09-12T18:51:01.613" Score="1" Body="&lt;p&gt;All Chris_Rands said is correct: you set the probability of $X\to Y$ and $Y\to X$ to 0 to forbid adjacent insertions/deletions in the alignment. A lot of textbooks including some classical ones use this rule, but in fact, the rule is questionable. It is easier to see this from Smith-Waterman alignment under the affine gap penalty, which is largely the non-probabilistic view of paired HMM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the affine gap penalty, a gap of length $k&amp;gt;0$ is scored as&#xA;$$&#xA;g(k)=-(d+k\cdot e)&#xA;$$&#xA;where $d\ge0$ is the gap open penalty and $e&amp;gt;0$ is the gap extension penalty. Suppose we are using a simple scoring matrix where a mismatch gets $-b$, $b&amp;gt;0$. We may see an insertion immediately followed by a deletion (and vice versa) if $b&amp;gt;2e$. It is actually not so difficult for this to happen. For example, for the human-mouse alignment (see the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/12529312&quot; rel=&quot;nofollow noreferrer&quot;&gt;blastz paper&lt;/a&gt;), $e=30$ and $b$ is ranged from 31 to 125. It is possible that an $X\to Y$ transition is preferred in the alignment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Theoretically speaking, it makes more sense to consider immediate transitions between insertions and deletions. In practice, though, the difference between allowing/disallowing such transitions is probably minor most of time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: on Chris' example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CGTACGTAGAGGAATGCCCCCCCCC--------AGCAACGTAGCAT     &#xA;CGTACGTAGAGGAATG---------TTTTTTTTAGCAACGTAGCAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If we use a scoring matrix with $b&amp;gt;2e$ but disallow adjacent ins-to-del transitions, we will probably end up with an alignment like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CGTACGTAGAGGAATGCCCCCCCCC-------AGCAACGTAGCAT     &#xA;CGTACGTAGAGGAATG--------TTTTTTTTAGCAACGTAGCAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This alignment score will be lower.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2018-09-13T11:52:40.520" LastActivityDate="2018-09-13T11:52:40.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5030" PostTypeId="1" CreationDate="2018-09-12T19:48:55.027" Score="2" ViewCount="443" Body="&lt;p&gt;I am using &lt;code&gt;rpy2&lt;/code&gt; to run normalization using &lt;code&gt;scran&lt;/code&gt; package. After calling &lt;code&gt;computeSumFactors&lt;/code&gt; I want to do the actual normalization calling &lt;code&gt;normalize()&lt;/code&gt; function from &lt;code&gt;scater&lt;/code&gt; package:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def normalize(counts):&#xA;utils = importr('utils')&#xA;utils.chooseCRANmirror(ind=1) &#xA;&#xA;packnames = ('Matrix', 'limSolve')&#xA;names_to_install = [x for x in packnames if not isinstalled(x)]&#xA;if len(names_to_install) &amp;gt; 0:&#xA;    utils.install_packages(StrVector(names_to_install))&#xA;&#xA;base = importr('base')&#xA;base.source('http://www.bioconductor.org/biocLite.R')&#xA;biocInstaller = importr('BiocInstaller')&#xA;&#xA;biocInstaller.biocLite('SingleCellExperiment', suppressUpdates=True)&#xA;biocInstaller.biocLite('scran', suppressUpdates=True)&#xA;biocInstaller.biocLite('scater', suppressUpdates=True)&#xA;&#xA;sceLib = importr(&quot;SingleCellExperiment&quot;)&#xA;scranLib = importr(&quot;scran&quot;)&#xA;scaterLib = importr(&quot;scater&quot;)&#xA;named_list = robjects.r.list(counts=counts)&#xA;sce = sceLib.SingleCellExperiment(named_list)&#xA;&#xA;try:&#xA;    clusters = scranLib.quickCluster(sce)&#xA;    sce = scaterLib.computeSumFactors(sce, cluster=clusters, positive=True)&#xA;except rpy2.rinterface.RRuntimeError:&#xA;    print(&quot;fewer cells than the minimum cluster size, so no quickCluster is run&quot;)&#xA;    sce = scranLib.computeSumFactors(sce, positive=True)&#xA;&#xA;----&amp;gt; sce = scaterLib.normalize(sce, logExprsOffset=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Everything runs fine except the last line which gives the error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;AttributeError: module 'scater' has no attribute 'normalize'&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is strange since that is what I am supposed to do actually according to &lt;a href=&quot;https://bioconductor.org/packages/3.7/bioc/vignettes/scran/inst/doc/scran.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;docs&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that maybe I am loading a wrong &lt;code&gt;scran&lt;/code&gt; version? I do not know though neither how to check the version number, nor how to install a specific one. Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sce = scaterLib.normalizeSCE(sce, log_exprs_offset=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And it is working. I am afraid though that it is a wrong way to go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to get the normalized matrix but it retrieved the initial raw counts one:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sce = scaterLib.normalizeSCE(sce, log_exprs_offset=1)&#xA;mtx_norm_r = robjects.r.assay(sce)&#xA;mtx_norm = np.array(mtx_norm_r)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, something is definitely wrong here. Either I am retrieving the normalized matrix in the wrong way or the normalization was not run correctly. I tried the following method for retrieval but no luck:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mtx_norm_r = sceLib.normcounts(sce) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;RRuntimeError: Error in assay(object, i = exprs_values) : &#xA;    'assay(, i=&quot;character&quot;, ...)' invalid subscript 'i'&#xA;  'normcounts' not in names(assays())&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I found a way to get the version numbers using &lt;code&gt;getNamespaceVersion()&lt;/code&gt; function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;robjects.r.getNamespaceVersion('scater')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The versions are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;scater - 1.8.4&#xA;scran - 1.8.4&#xA;SingleCellExperiment - 1.2.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;They are the newest as I understand.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2018-09-13T07:29:51.270" LastActivityDate="2018-11-12T08:01:30.280" Title="AttributeError: module 'scater' has no attribute 'normalize'" Tags="|r|python|scran|rpy2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5031" PostTypeId="2" ParentId="5030" CreationDate="2018-09-12T21:52:31.273" Score="1" Body="&lt;p&gt;I was able to get to the normalized values using &lt;code&gt;logcounts&lt;/code&gt; function in the following way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mtx_norm_r = sceLib.logcounts(sce)&#xA;mtx_norm = np.array(mtx_norm_r)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since I used &lt;code&gt;log_exprs_offset=1&lt;/code&gt; the field that I needed to use is &lt;code&gt;logcounts&lt;/code&gt;, otherwise it should be &lt;code&gt;normcounts&lt;/code&gt; which I found &lt;a href=&quot;https://rdrr.io/bioc/scater/man/normalize.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="48" LastEditDate="2018-09-13T07:30:19.950" LastActivityDate="2018-09-13T07:30:19.950" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5032" PostTypeId="1" AcceptedAnswerId="5034" CreationDate="2018-09-13T06:02:45.653" Score="-1" ViewCount="42" Body="&lt;p&gt;So I have expression data set with 4 healthy sample and 4 disease, to determine the number of K I used mclust which i ran on the list of differentially expressed genes so I got around 7 cluster from Mclust which im using as an input for my K-mean .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my data set is like all the differentially expressed genes it contains both UP and down regulated genes so basically it may be UP in my healthy and Down in patient and vice-versa ,4 likely possibilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now I have this doubt that from that I'm getting 7 cluster.Is that result is correct ? &lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2018-09-13T07:30:17.023" Title="K mean clustering issue" Tags="|r|rna-seq|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5033" PostTypeId="2" ParentId="5032" CreationDate="2018-09-13T06:55:40.443" Score="1" Body="&lt;p&gt;7 clusters may be correct in that it's the optimal number according to the Bayesian information criterion (or whatever value you had mclust optimize), but that doesn't make it necessarily the most sensible value. Have a look at your data and visually look at what might make sense. It's likely that the clusters are subdividing up and down regulated genes into various subgroups, which may or may not be useful, depending on your goals.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-09-13T06:55:40.443" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5034" PostTypeId="2" ParentId="5032" CreationDate="2018-09-13T07:30:17.023" Score="1" Body="&lt;p&gt;There is generally no such thing as correct or incorrect when it comes to clustering. When you specify $K$, the program did it's statistical algorithms. Unless the the program had a bug, your results were statistically valid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will need to check the clustered labels visually. Only your biological knowledge can tell you the optimal $K$.&lt;/p&gt;&#xA;" OwnerUserId="174" LastActivityDate="2018-09-13T07:30:17.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5035" PostTypeId="1" CreationDate="2018-09-13T13:55:40.280" Score="1" ViewCount="39" Body="&lt;p&gt;I'm looking at the new version of the Snomed dictionary and the concept &lt;code&gt;707993000&lt;/code&gt; &lt;strong&gt;&lt;em&gt;Clostridium difficile&lt;/em&gt; toxin A + B (substance)&lt;/strong&gt; is deactivated (and outdated) since version &lt;code&gt;20170731&lt;/code&gt;. Can anybody explain me, why this happened? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have used this concept for coding of results of laboratory examinations in our software system. Is there an alternative Snomed concept usable for the purpose of coding of evidence of the Clostridium difficile toxin A + B ?&lt;/p&gt;&#xA;" OwnerUserId="3358" LastEditorUserId="96" LastEditDate="2019-07-11T13:45:33.597" LastActivityDate="2019-07-11T13:45:33.597" Title="Why is Clostridium difficile toxin A + B outdated in snomed?" Tags="|database|identifiers|data-management|ontology|bacteria|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5036" PostTypeId="2" ParentId="5025" CreationDate="2018-09-13T14:37:15.013" Score="2" Body="&lt;p&gt;Maybe try&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plot(evecdat$PC1, evecdat$PC2, xlab=&quot;PC1&quot;, ylab=&quot;PC2&quot;, pch=&quot;24&quot;, cex.lab=2.5, cex.axis=1, type=&quot;n&quot;)  &#xA;text(evecdat$PC1, evecdat$PC2, evecdat$Sample, col=as.vector(evecdat$color))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This should plot the sample names as text (taken from the &lt;code&gt;evecdat$Sample&lt;/code&gt; column), colored as specified in the &lt;code&gt;evecdat$color&lt;/code&gt; column. See &lt;code&gt;?text&lt;/code&gt; for details.&lt;/p&gt;&#xA;" OwnerUserId="3241" LastActivityDate="2018-09-13T14:37:15.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5037" PostTypeId="2" ParentId="4998" CreationDate="2018-09-13T19:23:14.223" Score="3" Body="&lt;p&gt;It turns out there is a tool that does this beautifully.  I used the &lt;code&gt;proteinToGenome&lt;/code&gt; function from the &lt;a href=&quot;https://bioconductor.org/packages/devel/bioc/vignettes/ensembldb/inst/doc/coordinate-mapping.html#5_mapping_protein_coordinates_to_the_genome&quot; rel=&quot;nofollow noreferrer&quot;&gt;ensembldb&lt;/a&gt; package.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Define the Ensemble database object for the &#xA;edbX &amp;lt;- filter(EnsDb.Hsapiens.v75, filter = ~ seq_name == &quot;X&quot;) &#xA;# Set the first and last amino acid you want to convert and the ensembl protein id&#xA;GAGE10_prt &amp;lt;- IRanges(start = 5, end = 9, names = &quot;ENSP00000385415&quot;)&#xA;# Perform the conversion.  GAGE10_prt is an object that has the coordinates&#xA;GAGE10_prt &amp;lt;- proteinToGenome(GAGE10_prt, edbX) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="59" LastActivityDate="2018-09-13T19:23:14.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5038" PostTypeId="1" CreationDate="2018-09-13T19:50:35.627" Score="3" ViewCount="96" Body="&lt;p&gt;Is there a way to identify mixed samples based on SNPs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example input (table of genotypes for multiple samples):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;|      | sample1 | sample2 | sample3 |&#xA;|------|---------|---------|---------|&#xA;| rs1  | AA      | BB      | AB      |&#xA;| rs2  | AA      | BB      | AB      |&#xA;| rs3  | AA      | BB      | AB      |&#xA;| rs4  | AA      | BB      | AB      |&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Desired output: Sample relationships/distances. In this example, sample1 and sample2 are distinct, but sample3 is likely a mix of two (biological or technical).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on this simple example, the calculation seems doable, but maybe it's impossible in more complex realistic scenarios.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a nice tool &lt;a href=&quot;https://github.com/brentp/peddy&quot; rel=&quot;nofollow noreferrer&quot;&gt;peddy&lt;/a&gt; that does something conceptually similar, but it expects pedigree information as well as genotype frequencies for both the input samples and population.&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="35" LastEditDate="2018-09-15T02:16:27.497" LastActivityDate="2018-09-15T02:16:27.497" Title="using SNPs to identify mixed samples" Tags="|snp|genotyping|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5039" PostTypeId="2" ParentId="5038" CreationDate="2018-09-13T23:20:56.530" Score="2" Body="&lt;p&gt;Yes. The general term for identifying subsets from a mixed group is &lt;a href=&quot;https://en.wikipedia.org/wiki/Deconvolution&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Wikipedia Deconvolution&quot;&gt;deconvolution&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is easier to do when there is haplotype-level information available (e.g. sequenced reads that span multiple SNPs), but it is possible to get some indication of sample mixes from unlinked SNPs alone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Deconvolution is helped a lot by having proportional data in the input (e.g. frequencies of genotypes, rather than a single genotype). Most SNPchip platforms have multiple probes per variant, and can provide some indication of variant proportion when digging down into the raw data. Deconvolution is also a lot easier with some knowledge about the component mixes. It's much easier to simulate mixes than to guess at possible mixes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://investigativegenetics.biomedcentral.com/articles/10.1186/s13323-014-0018-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's a paper&lt;/a&gt; about the application of deconvolution in forensics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/nmeth.3337&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's a paper&lt;/a&gt; about a program called CIBERSORT which carries out a deconvolution of immune populations based on gene counts:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's another &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3874291/&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; which discusses the idea in more general terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might find more information on one of these Biostars posts:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.biostars.org/p/160961/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Deconvolution Methods on RNA-Seq Data&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.biostars.org/p/121286/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Differential gene expession analysis in cell populations of mixed tumor and normal cells&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="73" LastEditorUserId="48" LastEditDate="2018-09-14T07:06:31.890" LastActivityDate="2018-09-14T07:06:31.890" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5041" PostTypeId="1" AcceptedAnswerId="5044" CreationDate="2018-09-16T14:36:48.707" Score="3" ViewCount="147" Body="&lt;p&gt;I need the name of the protein in /product=&quot;protein_name&quot; using bash commands. Beware, there is a lot of whitespace between lines.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FEATURES             Location/Qualifiers&#xA;     source          1..1266&#xA;                     /organism=&quot;Sarcophilus harrisii&quot;&#xA;                     /mol_type=&quot;mRNA&quot;&#xA;                     /isolate=&quot;Salem&quot;&#xA;                     /db_xref=&quot;taxon:9305&quot;&#xA;                     /chromosome=&quot;1&quot;&#xA;                     /map=&quot;unlocalized&quot;&#xA;                     /sex=&quot;female&quot;&#xA;                     /country=&quot;Australia: Taronga Zoo, Sydney&quot;&#xA;     gene            1..1266&#xA;                     /gene=&quot;UROS&quot;&#xA;                     /note=&quot;Derived by automated computational analysis using&#xA;                     gene prediction method: Gnomon. Supporting evidence&#xA;                     includes similarity to: 10 Proteins, and 100% coverage of&#xA;                     the annotated genomic feature by RNAseq alignments,&#xA;                     including 5 samples with support for all annotated&#xA;                     introns&quot;&#xA;                     /db_xref=&quot;GeneID:100933602&quot;&#xA;     CDS             60..908&#xA;                     /gene=&quot;UROS&quot;&#xA;                     /codon_start=1&#xA;                     /product=&quot;uroporphyrinogen-III synthase isoform X1&quot;&#xA;                     /protein_id=&quot;XP_003755055.2&quot;&#xA;                     /db_xref=&quot;GeneID:100933602&quot;&#xA;                     /translation=&quot;MRIPTSFSRGAWWKMKVLLLKDAKDDDSGLDPYLQELGSCGFEA&#xA;                     TLIPVLAFEFLSLPSFSEKLSHPEAYGGLVFTSPRAVEALQLSLERAGQLEVWKKSLK&#xA;                     EKWRLKPAYVVGQATASLVKKVGLAAEGEKCGNGEKLAEYICSRETPATLPLLFPCGT&#xA;                     LKGEVLPKTLKQKGILLESLTVYQKVQHPRMQDSLKSYFSTQGVPAGVTFFSPSGLTY&#xA;                     SLPHIQELSGASFDRIKFAAIGPTTARALASAGIPVSCTAEKPCPQDLAAGLQKALQP&#xA;                     SGPSSS&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Right now I have this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while read p;&#xA;do cat &quot;${p}.gb&quot; | tr &quot;\&quot;&quot; &quot; &quot;| egrep -v &quot;/db_xref= taxon&quot;| egrep &quot;/gene=|/db_xref=|/product=|/protein_id=&quot; | sort | uniq | awk '{print($2)}END{print(&quot;&quot;)}'&#xA;done &amp;lt;XM_codes.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which results in this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GeneID:100933602&#xA;UROS&#xA;uroporphyrinogen-111&#xA;XP_003755055.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My output should be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GeneID:100933602&#xA;UROS&#xA;uroporphyrinogen-III synthase isoform X1&#xA;XP_003755055.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3364" LastEditorUserId="48" LastEditDate="2018-09-17T07:43:18.987" LastActivityDate="2018-09-18T20:52:55.287" Title="How to get the product of a CDS" Tags="|proteins|linux|genbank|bash|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5042" PostTypeId="2" ParentId="5041" CreationDate="2018-09-16T16:29:28.810" Score="3" Body="&lt;p&gt;If you change the delimiter to something besides a space in the &lt;code&gt;tr&lt;/code&gt; command (&lt;code&gt;tr &quot;\&quot;&quot; &quot;\t&quot;&lt;/code&gt;) then you can use that same delimiter in the &lt;code&gt;awk&lt;/code&gt; command (&lt;code&gt;-F &quot;\t&quot;&lt;/code&gt;).  In my example below I used a tab, also I filtered the taxon prior to the &lt;code&gt;tr&lt;/code&gt; so that you don't have to worry about that staying in sync if you change the delimiter.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; egrep -v &quot;/db_xref=\&quot;taxon&quot;| tr &quot;\&quot;&quot; &quot;\t&quot; | egrep &quot;/gene=|/db_xref=|/product=|/protein_id=&quot; | sort | uniq | awk -F &quot;\t&quot; '{print($2)}END{print(&quot;&quot;)}';&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GeneID:100933602&#xA;UROS&#xA;uroporphyrinogen-III synthase isoform X1&#xA;XP_003755055.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="964" LastActivityDate="2018-09-16T16:29:28.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5043" PostTypeId="1" CreationDate="2018-09-16T22:31:40.673" Score="1" ViewCount="144" Body="&lt;p&gt;I have a &lt;code&gt;.bam&lt;/code&gt;, as ouput from STAR aligner,  from which I need to extract some info using RSeQC while using all the computational resources available to increase speed. To accomplish that I am using GNU parallel. Currently I am doing it like this:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Split the main .bam per chromosome, generating files with the suffix Aligned.sortedByCoord.out.REF&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bamtools split -in Aligned.sortedByCoord.out.bam  -reference&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate the clipping profile&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ls Aligned.sortedByCoord.out.REF_* | parallel --joblog clipping_profile.log --eta --bar --max-procs 20 --jobs 20 'clipping_profile.py -i {} -s &quot;SE&quot; -o RSeQC.clipping.profile/clipping_profile.{.}'&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This will generate a &lt;code&gt;.r&lt;/code&gt;, &lt;code&gt;.pdf&lt;/code&gt;and a &lt;code&gt;.xlxs&lt;/code&gt; for each chromosome, thus ending up with a lot of files. I am forced to split the &lt;code&gt;.bam&lt;/code&gt; per chromosome as I am not able to find a &lt;code&gt;--recstart&lt;/code&gt; nor a &lt;code&gt;--recend&lt;/code&gt; in the compressed &lt;code&gt;.bam&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Question&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Is there a better way to split a &lt;code&gt;.bam&lt;/code&gt; file using GNU parallel and RSeQC' programs (&lt;em&gt;e.g.&lt;/em&gt; &lt;code&gt;split_bam.py&lt;/code&gt;), such that one gets the outputs from a single input file?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inspired by ole.tange's &lt;a href=&quot;https://www.biostars.org/p/63816/&quot; rel=&quot;nofollow noreferrer&quot;&gt;thread&lt;/a&gt;, I have also tried this, though unsuccessfully:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat star_outputAligned.sortedByCoord.out.bam | parallel --block 1G -k --joblog splitbam3.log --max-procs 20 --jobs 20 --pipe 'split_bam.py -i {} -r ../../My_genomes/GCA.GRCh38/GRCh38_wgEncodeGencodeAttrsV28_all_rRNA.bed12 -o splitbam3'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2936" LastEditorUserId="2936" LastEditDate="2018-09-16T22:37:21.207" LastActivityDate="2018-09-16T22:37:21.207" Title="How to correctly parallelise RSeQC scripts with GNU parallel?" Tags="|rna-seq|rna|rna-alignment|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5044" PostTypeId="2" ParentId="5041" CreationDate="2018-09-17T08:17:16.950" Score="0" Body="&lt;p&gt;If you have GNU &lt;code&gt;grep&lt;/code&gt; (default on Linux), you can do this with a single &lt;code&gt;grep&lt;/code&gt; command using Perl Compatible Regular Expressions (PCREs):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -oP 'GeneID:\d+|/(gene|product|protein_id)=&quot;\K[^&quot;]+' file | sort -u&#xA;GeneID:100933602&#xA;uroporphyrinogen-III synthase isoform X1&#xA;UROS&#xA;XP_003755055.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;-o&lt;/code&gt; flag tells &lt;code&gt;grep&lt;/code&gt; to only print the matching portion of the line and the &lt;code&gt;-P&lt;/code&gt; enables PCREs which let us use &lt;code&gt;d+&lt;/code&gt; for &quot;one or more digits&quot; and &lt;code&gt;\K&lt;/code&gt; which means &quot;discard whatever was matched before the &lt;code&gt;\K&lt;/code&gt;&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I'm not using &lt;code&gt;cat&lt;/code&gt; here. The command from your question is what is known as a &lt;a href=&quot;http://porkmail.org/era/unix/award.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;UUoC&lt;/a&gt; (useless use of cat) and is a surefire way of annoying any *nix geeks around ;).  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If you don't have GNU &lt;code&gt;grep&lt;/code&gt;, you can do something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk -F= '/GeneID:|(gene|product|protein_id)=/{print $2}' file | &#xA;    tr -d '&quot;' | sort -u&#xA;GeneID:100933602&#xA;uroporphyrinogen-III synthase isoform X1&#xA;UROS&#xA;XP_003755055.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-09-17T08:17:16.950" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5046" PostTypeId="1" CreationDate="2018-09-17T14:34:35.607" Score="1" ViewCount="1599" Body="&lt;p&gt;I want to find a cutoff value for each gene, above which we can consider a gene expressed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that not all &lt;em&gt;effectively&lt;/em&gt; non-expressed genes will have 0 counts due to sequencing errors for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is related to &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/687/what-methods-are-available-to-find-a-cutoff-value-for-non-expressed-genes-in-rna&quot;&gt;this one&lt;/a&gt;, but is concerned with single-cell data. The technical differences between bulk and single cell RNA-seq warrant a separate question for this problem. Also similar to &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/693/how-can-we-distinguish-between-true-zero-and-dropout-zero-counts-in-single-cell&quot;&gt;this one&lt;/a&gt;, but focuses on gene expression and not differences between zero counts.&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2018-10-26T06:25:16.290" Title="Find a cutoff value for genes that are expressed in single cell RNA-seq?" Tags="|rna-seq|scrnaseq|gene-expression|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5047" PostTypeId="1" AcceptedAnswerId="5051" CreationDate="2018-09-17T16:37:02.467" Score="1" ViewCount="984" Body="&lt;p&gt;I need to extract certain lines from a GenBank file on Linux.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Input file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;LOCUS       XM_012540068            1713 bp    mRNA    linear   MAM 11-JAN-2018&#xA;DEFINITION  PREDICTED: Sarcophilus harrisii cytochrome c oxidase assembly&#xA;            protein COX15 homolog (LOC100925550), mRNA.&#xA;ACCESSION   XM_012540068&#xA;VERSION     XM_012540068.2&#xA;DBLINK      BioProject: PRJNA167725&#xA;KEYWORDS    RefSeq; corrected model.&#xA;SOURCE      Sarcophilus harrisii (Tasmanian devil)&#xA;  ORGANISM  Sarcophilus harrisii&#xA;            Eukaryota; Metazoa; Chordata; Craniata; Vertebrata; Euteleostomi;&#xA;            Mammalia; Metatheria; Dasyuromorphia; Dasyuridae; Sarcophilus.&#xA;COMMENT     MODEL REFSEQ:  This record is predicted by automated computational&#xA;            analysis. This record is derived from a genomic sequence&#xA;            (NW_003816462.1) and transcript sequence (GEDN01126617.1) annotated&#xA;            using gene prediction method: Gnomon, supported by EST evidence.&#xA;            Also see:&#xA;                Documentation of NCBI's Annotation Process&#xA;&#xA;            On Jan 11, 2018 this sequence version replaced XM_012540068.1.&#xA;&#xA;            ##Genome-Annotation-Data-START##&#xA;            Annotation Provider         :: NCBI&#xA;            Annotation Status           :: Full annotation&#xA;            Annotation Name             :: Sarcophilus harrisii Annotation&#xA;                                           Release 102&#xA;            Annotation Version          :: 102&#xA;            Annotation Pipeline         :: NCBI eukaryotic genome annotation&#xA;                                           pipeline&#xA;            Annotation Software Version :: 8.0&#xA;            Annotation Method           :: Best-placed RefSeq; Gnomon&#xA;            Features Annotated          :: Gene; mRNA; CDS; ncRNA&#xA;            ##Genome-Annotation-Data-END##&#xA;&#xA;            ##RefSeq-Attributes-START##&#xA;            assembly gap :: added 136 transcript bases to patch genome assembly&#xA;                            gap&#xA;            ##RefSeq-Attributes-END##&#xA;PRIMARY     REFSEQ_SPAN         PRIMARY_IDENTIFIER PRIMARY_SPAN        COMP&#xA;            1-136               GEDN01126617.1     1-136&#xA;            137-327             AEFK01003589.1     1071-1261&#xA;            328-450             AEFK01003589.1     2007-2129&#xA;            451-637             AEFK01003589.1     2869-3055&#xA;            638-805             AEFK01003589.1     6071-6238&#xA;            806-887             AEFK01003591.1     516-597&#xA;            888-1042            AEFK01003591.1     1431-1585&#xA;            1043-1156           AEFK01003591.1     4257-4370&#xA;            1157-1713           AEFK01003591.1     6191-6747&#xA;FEATURES             Location/Qualifiers&#xA;     source          1..1713&#xA;                     /organism=&quot;Sarcophilus harrisii&quot;&#xA;                     /mol_type=&quot;mRNA&quot;&#xA;                     /db_xref=&quot;taxon:9305&quot;&#xA;                     /chromosome=&quot;1&quot;&#xA;                     /map=&quot;unlocalized&quot;&#xA;                     /sex=&quot;female&quot;&#xA;     gene            1..1713&#xA;                     /gene=&quot;LOC100925550&quot;&#xA;                     /note=&quot;The sequence of the model RefSeq transcript was&#xA;                     modified relative to its source genomic sequence to&#xA;                     represent the inferred CDS: added 136 bases not found in&#xA;                     genome assembly; Derived by automated computational&#xA;                     analysis using gene prediction method: Gnomon. Supporting&#xA;                     evidence includes similarity to: 10 ESTs, 11 Proteins, and&#xA;                     100% coverage of the annotated genomic feature by RNAseq&#xA;                     alignments, including 5 samples with support for all&#xA;                     annotated introns&quot;&#xA;                     /db_xref=&quot;GeneID:100925550&quot;&#xA;     CDS             53..1288&#xA;                     /gene=&quot;LOC100925550&quot;&#xA;                     /note=&quot;The sequence of the model RefSeq protein was&#xA;                     modified relative to its source genomic sequence to&#xA;                     represent the inferred CDS: added 84 bases not found in&#xA;                     genome assembly&quot;&#xA;                     /codon_start=1&#xA;                     /product=&quot;cytochrome c oxidase assembly protein COX15&#xA;                     homolog&quot;&#xA;                     /protein_id=&quot;XP_012395522.2&quot;&#xA;                     /db_xref=&quot;GeneID:100925550&quot;&#xA;                     /translation=&quot;MMRLLFPPLKAVLGSPGVRLQAAGGAHRCGCHYGQKLSLLQRQH&#xA;                     STLSEVALQSGRGAVSVPTKAAQRIMGRWLLVCSGTVAGAVILGGVTRLTESGLSMVD&#xA;                     WHLIKGMKPPTSQEEWEAEFQKYQQFPEFKILNHNMTLAEFKFIWYMEYSHRMWGRIV&#xA;                     GLAYVLPTVYFWRKGWLTHGMKGRVLALCGLVCFQGLLGWYMVKSGLEEKPDSYDIPR&#xA;                     VSQYRLAAHLGSALVLYCASLWTGLSLLLPRDKLPETRKLLQLRRFAHGTAGLVFLTA&#xA;                     LSGAFVAGLDAGLVYNSFPKMGDHWIPEDLFTFSPVLRNVFENPTMVQFDHRVLGIAT&#xA;                     VTAITGLYFFSRRIPLPRRAKMAVTTLLAVAYAQVALGISTLLTYVPTPLAAVHQSGS&#xA;                     LALLSIALWLMNELRRIPK&quot;&#xA;ORIGIN      &#xA;        1 gtggggcgac cgagggaggc cgctgcgagg ctgtgacttt gagactcgcg cgatgatgcg&#xA;       61 attgctgttt cctcctctga aagcggtgct gggcagtccg ggcgtccggc tgcaggctgc&#xA;      121 tgggggagct caccggtgcg gttgccatta tgggcaaaag ctctctctct tgcaaagaca&#xA;      181 gcacagtact ctgtctgaag tagctttgca gtctggaagg ggagcagtgt ctgtccccac&#xA;      241 aaaggctgcc cagcgcatta tgggtcgttg gctcttggtc tgcagcggaa cagtagcagg&#xA;      301 agcggtgatt cttggtggag tgactaggtt gacagagtct gggctctcca tggtagattg&#xA;      361 gcacttgata aaaggaatga aaccacccac aagccaagaa gaatgggaag cagagttcca&#xA;      421 aaaataccag cagtttccag aatttaaaat cttgaaccat aacatgacat tagcagaatt&#xA;      481 caagtttatc tggtacatgg aatactcaca cagaatgtgg ggtcggattg tgggcctagc&#xA;      541 atatgtcctt cccactgtct acttttggag aaagggctgg ctcacccatg gcatgaaagg&#xA;      601 tcgtgttctt gccctctgtg gtttagtctg tttccaggga ctgttggggt ggtatatggt&#xA;      661 gaagagtgga ttagaagaaa agccagattc ttatgacatc cctcgggtga gccagtatcg&#xA;      721 tctggccgct cacttgggct ctgcccttgt cctctactgt gccagcttgt ggactggact&#xA;      781 ctccctcttg cttcctcgag acaagttgcc agaaacccga aagctcttgc agctgaggcg&#xA;      841 atttgctcat gggacagcag gcctggtttt ccttacagct ctttcaggag cctttgtggc&#xA;      901 aggcctggat gctggacttg tatacaactc ctttcctaag atgggagacc actggatccc&#xA;      961 agaagatctc tttaccttct ctccagttct tagaaatgtc tttgagaatc caaccatggt&#xA;     1021 gcagtttgat catcgagtcc tgggaatcgc tacagtcaca gccattactg gactttactt&#xA;     1081 cttctccagg agaatccctc ttcctcgaag ggccaaaatg gcagtgacaa cattgctggc&#xA;     1141 tgtggcctat gcacaggtgg ccttgggcat tagtactcta ctaacgtatg ttccaactcc&#xA;     1201 tttggctgca gttcaccagt cagggtccct cgctttgctc agcattgccc tttggcttat&#xA;     1261 gaacgaactg aggcgaatcc caaagtaatg cccaggcagg tatttttctg gactcttccc&#xA;     1321 aggcttttga gagctctgta gagggcatga ggcttccctc ctgttgaatt aaagaattga&#xA;     1381 tgtggtagac agatcagaag atcaagaatg aagttactga tctgaggatt ctttttcaga&#xA;     1441 ctaaacaaac tactcttctt ccataagttg tcacttgctt tttaatgtgc tattaagatc&#xA;     1501 ccttccaact ttcactactc tgattctttt gatacagcaa taccactgat aggtctgtgt&#xA;     1561 ctcagagttg ggagggaggg aaggaaaaaa aaaacttttt gtacaaacgt acaaaactat&#xA;     1621 agtagttcct tttgtgggaa ctaaaataag gaaattttct ttctttttta tttattcgtt&#xA;     1681 ttcaacattt acttttataa gatttagatt ttc&#xA;//&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tr &quot;\&quot;&quot; &quot; &quot; &amp;lt; XM_012540068.2.gb | egrep -v &quot;/db_xref= taxon&quot; | awk -F= '/GeneID:|(gene|product|protein_id|translation)=/{print $2}' | sort -u&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What I get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; GeneID:100925550&#xA; LOC100925550&#xA; MMRLLFPPLKAVLGSPGVRLQAAGGAHRCGCHYGQKLSLLQRQH&#xA; XP_012395522.2&#xA; cytochrome c oxidase assembly protein COX15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What I want:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; GeneID:100925550&#xA; LOC100925550&#xA; MMRLLFPPLKAVLGSPGVRLQAAGGAHRCGCHYGQKLSLLQRQH...(sequence goes on...)&#xA; XP_012395522.2&#xA; cytochrome c oxidase assembly protein COX15 homolog&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Preferably, it has to be a mastercode for every GenBank-file. For a project I've got to loop through 78 GenBank-files, and extract the same data. I asked a &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/5041/298&quot;&gt;similar question&lt;/a&gt; earlier, but the solution there doesn't work when the protein's name spans multiple lines.&lt;/p&gt;&#xA;" OwnerUserId="3364" LastEditorUserId="1194" LastEditDate="2018-09-24T12:02:23.203" LastActivityDate="2018-09-24T12:02:23.203" Title="Extract certain fields using from GenBank file using Bash script" Tags="|proteins|genbank|bash|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5048" PostTypeId="1" CreationDate="2018-09-17T17:12:27.960" Score="1" ViewCount="145" Body="&lt;p&gt;I am making a hub on UCSC Genome Browser and I am aiming to display multiple bigWig files as a single track (multiwig container). I decided to use the autoscale option because I wanted a dynamic scale: y axis changing depending on the maximum values in the current observed window.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am confused as to how to the relationship between:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;autoscale on and viewLimits&lt;/li&gt;&#xA;&lt;li&gt;autoscale on and type bigWig &amp;lt;#&gt; &amp;lt;#&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For the &quot;type&quot; option, the official documentation mentions that: &quot;The remote data files of type bigWig must declare the expected signal range for the data&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So my question is, are the following assumptions correct&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If I have autoscale on, I don't need to specify viewLimits&lt;/li&gt;&#xA;&lt;li&gt;I still have to specify &quot;type bigWig&quot; but you no longer have to put a range (i.e. the &amp;lt;#&gt; &amp;lt;#&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If this is correct, my trackDb.txt file will look as such:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;track data&#xA;container multiWig&#xA;configurable on&#xA;shortLabel label data&#xA;longLabel long label data&#xA;visibility full&#xA;priority 1&#xA;type bigWig&#xA;autoScale on&#xA;aggregate transparentOverlay&#xA;windowingFunction mean&#xA;showSubtrackColorOnUi on&#xA;&#xA;    track sample1&#xA;    bigDataUrl path/to/sample1.bw&#xA;    shortLabel s_sample1&#xA;    longLabel s_sample1&#xA;    type bigWig&#xA;    visibility full&#xA;    alwaysZero on&#xA;    priority 120&#xA;    color 220,180,180&#xA;    parent data on&#xA;&#xA;    track sample2&#xA;    bigDataUrl path/to/sample2.bw&#xA;    shortLabel s_sample2&#xA;    longLabel s_sample2&#xA;    type bigWig&#xA;    visibility full&#xA;    alwaysZero on&#xA;    priority 120&#xA;    color 220,180,180&#xA;    parent data on&#xA;&#xA;    etc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2931" LastEditorUserId="2931" LastEditDate="2018-09-18T08:49:32.680" LastActivityDate="2018-09-18T08:49:32.680" Title="Question: UCSC hub: autoscale option for multiwig track" Tags="|ucsc|bigwig|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5049" PostTypeId="1" AcceptedAnswerId="6743" CreationDate="2018-09-17T19:20:02.233" Score="0" ViewCount="204" Body="&lt;p&gt;I am trying to run proteowizzard using wine to be able to convert raw mass spectrometry files to myML format. Following, &lt;a href=&quot;http://tools.proteomecenter.org/wiki/index.php?title=Msconvert_Wine&quot; rel=&quot;nofollow noreferrer&quot;&gt;these instructions&lt;/a&gt;` which seem terribly outdated. I am getting many issues. I am running Ubuntu 16.04.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want to convert output from a ThermoFisher instrument to mzML or mzXML format.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; ~/software/proteowizzard&amp;gt;wine msconvert.exe &#xA;&amp;gt; fixme:heap:HeapSetInformation (nil) 1 (nil) 0&#xA;&amp;gt; fixme:heap:HeapSetInformation (nil) 1 (nil) 0&#xA;&amp;gt; fixme:process:SetProcessShutdownParameters (00000380, 00000000):&#xA;&amp;gt; partial stub. err:module:import_dll Library VCOMP110.DLL (which is&#xA;&amp;gt; needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\baf2sql_c.dll&quot;) not&#xA;&amp;gt; found err:module:import_dll Library baf2sql_c.dll (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msconvert.exe&quot;) not&#xA;&amp;gt; found err:module:import_dll Library MSVCP140.dll (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msconvert.exe&quot;) not&#xA;&amp;gt; found err:module:import_dll Library VCRUNTIME140.dll (which is needed&#xA;&amp;gt; by L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msconvert.exe&quot;) not&#xA;&amp;gt; found err:module:import_dll Library api-ms-win-crt-runtime-l1-1-0.dll&#xA;&amp;gt; (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msconvert.exe&quot;) not&#xA;&amp;gt; found err:module:import_dll Library api-ms-win-crt-heap-l1-1-0.dll&#xA;&amp;gt; (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msconvert.exe&quot;) not&#xA;&amp;gt; found err:module:import_dll Library api-ms-win-crt-convert-l1-1-0.dll&#xA;&amp;gt; (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msconvert.exe&quot;) not&#xA;&amp;gt; found err:module:import_dll Library api-ms-win-crt-stdio-l1-1-0.dll&#xA;&amp;gt; (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msconvert.exe&quot;) not&#xA;&amp;gt; found err:module:import_dll Library api-ms-win-crt-string-l1-1-0.dll&#xA;&amp;gt; (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msconvert.exe&quot;) not&#xA;&amp;gt; found err:module:import_dll Library&#xA;&amp;gt; api-ms-win-crt-environment-l1-1-0.dll (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msconvert.exe&quot;) not&#xA;&amp;gt; found err:module:import_dll Library&#xA;&amp;gt; api-ms-win-crt-filesystem-l1-1-0.dll (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msconvert.exe&quot;) not&#xA;&amp;gt; found err:module:import_dll Library api-ms-win-crt-time-l1-1-0.dll&#xA;&amp;gt; (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msconvert.exe&quot;) not&#xA;&amp;gt; found err:module:import_dll Library api-ms-win-crt-locale-l1-1-0.dll&#xA;&amp;gt; (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msconvert.exe&quot;) not&#xA;&amp;gt; found err:module:import_dll Library api-ms-win-crt-math-l1-1-0.dll&#xA;&amp;gt; (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msconvert.exe&quot;) not&#xA;&amp;gt; found err:module:import_dll Library MSVCP140.dll (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msparser.dll&quot;) not found&#xA;&amp;gt; err:module:import_dll Library VCRUNTIME140.dll (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msparser.dll&quot;) not found&#xA;&amp;gt; err:module:import_dll Library api-ms-win-crt-runtime-l1-1-0.dll (which&#xA;&amp;gt; is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msparser.dll&quot;) not found&#xA;&amp;gt; err:module:import_dll Library api-ms-win-crt-string-l1-1-0.dll (which&#xA;&amp;gt; is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msparser.dll&quot;) not found&#xA;&amp;gt; err:module:import_dll Library api-ms-win-crt-stdio-l1-1-0.dll (which&#xA;&amp;gt; is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msparser.dll&quot;) not found&#xA;&amp;gt; err:module:import_dll Library api-ms-win-crt-filesystem-l1-1-0.dll&#xA;&amp;gt; (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msparser.dll&quot;) not found&#xA;&amp;gt; err:module:import_dll Library api-ms-win-crt-time-l1-1-0.dll (which is&#xA;&amp;gt; needed by L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msparser.dll&quot;)&#xA;&amp;gt; not found err:module:import_dll Library&#xA;&amp;gt; api-ms-win-crt-convert-l1-1-0.dll (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msparser.dll&quot;) not found&#xA;&amp;gt; err:module:import_dll Library api-ms-win-crt-environment-l1-1-0.dll&#xA;&amp;gt; (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msparser.dll&quot;) not found&#xA;&amp;gt; err:module:import_dll Library api-ms-win-crt-locale-l1-1-0.dll (which&#xA;&amp;gt; is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msparser.dll&quot;) not found&#xA;&amp;gt; err:module:import_dll Library api-ms-win-crt-heap-l1-1-0.dll (which is&#xA;&amp;gt; needed by L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msparser.dll&quot;)&#xA;&amp;gt; not found err:module:import_dll Library api-ms-win-crt-math-l1-1-0.dll&#xA;&amp;gt; (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msparser.dll&quot;) not found&#xA;&amp;gt; err:module:import_dll Library api-ms-win-crt-utility-l1-1-0.dll (which&#xA;&amp;gt; is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msparser.dll&quot;) not found&#xA;&amp;gt; err:module:import_dll Library msparser.dll (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msconvert.exe&quot;) not&#xA;&amp;gt; found err:module:import_dll Library api-ms-win-crt-utility-l1-1-0.dll&#xA;&amp;gt; (which is needed by&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msconvert.exe&quot;) not&#xA;&amp;gt; found err:module:LdrInitializeThunk Main exe initialization for&#xA;&amp;gt; L&quot;Z:\\home\\swacker\\software\\proteowizzard\\msconvert.exe&quot; failed,&#xA;&amp;gt; status c0000135&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2018-09-18T17:03:11.280" LastActivityDate="2019-01-02T19:35:36.650" Title="How to run proteowizzard on linux using wine?" Tags="|mass-spectrometry|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5050" PostTypeId="1" AcceptedAnswerId="5052" CreationDate="2018-09-18T00:12:35.127" Score="1" ViewCount="94" Body="&lt;p&gt;In many scientific papers, there is a plus sign:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;... T-circles have also been detected in ALT+ (telomerase-negative) cells9 and ...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What does the &quot;+&quot; mean? Does it mean differentially expressed?&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="1157" LastEditDate="2019-04-01T14:17:35.913" LastActivityDate="2019-04-01T14:17:35.913" Title="What does &quot;ALT+ cancer cells&quot;, the plus sign mean?" Tags="|terminology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5051" PostTypeId="2" ParentId="5047" CreationDate="2018-09-18T03:15:04.413" Score="0" Body="&lt;p&gt;Here is a script for achieving your result not so good but giving the desired output &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;egrep -v &quot;/db_xref= taxon&quot;  $1| awk -F= '/GeneID:|(gene|protein_id)=/{print $2}' | sort -u &amp;gt; file1&#xA;sed -n '/product/,/protein_id/p' $1|grep -v protein_id |sed 's/.*=//' &amp;gt;file2&#xA;awk '/translation/,/ORIGIN/{if(/translation|ORIGIN/) next; print}' $1 &amp;gt;file3&#xA;head -3 file1 &amp;amp;&amp;amp; cat file2 &amp;amp;&amp;amp; printf &quot;\&quot;&quot; &amp;amp;&amp;amp; cat file3 |tr -d &quot; \t\n\r&quot; &amp;amp;&amp;amp;   echo &amp;amp;&amp;amp; rm -rf file1 file2 file3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Save it in a file names as script1 and executing it as &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;`./script5 filename`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Gives following output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;GeneID:100925550&quot;&#xA;&quot;LOC100925550&quot;&#xA;&quot;XP_012395522.2&quot;&#xA;&quot;cytochrome c oxidase assembly protein COX15&#xA;                 homolog&quot;&#xA;&quot;STLSEVALQSGRGAVSVPTKAAQRIMGRWLLVCSGTVAGAVILGGVTRLTESGLSMVDWHLIKGMKPPTSQEEWEAEFQKYQQFPEFKILNHNMTLAEFKFIWYMEYSHRMWGRIVGLAYVLPTVYFWRKGWLTHGMKGRVLALCGLVCFQGLLGWYMVKSGLEEKPDSYDIPRVSQYRLAAHLGSALVLYCASLWTGLSLLLPRDKLPETRKLLQLRRFAHGTAGLVFLTALSGAFVAGLDAGLVYNSFPKMGDHWIPEDLFTFSPVLRNVFENPTMVQFDHRVLGIATVTAITGLYFFSRRIPLPRRAKMAVTTLLAVAYAQVALGISTLLTYVPTPLAAVHQSGSLALLSIALWLMNELRRIPK&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-09-18T08:45:54.777" LastActivityDate="2018-09-18T08:45:54.777" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5052" PostTypeId="2" ParentId="5050" CreationDate="2018-09-18T03:41:50.417" Score="5" Body="&lt;p&gt;Here are a couple of papers (&lt;a href=&quot;http://clincancerres.aacrjournals.org/content/14/13/4134&quot; rel=&quot;noreferrer&quot;&gt;A&lt;/a&gt; and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4993795/&quot; rel=&quot;noreferrer&quot;&gt;B&lt;/a&gt;) that explain ALT.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Telomere maintenance is a hallmark of cancer as it provides cancer cells with cellular immortality. A significant fraction of tumors uses the alternative lengthening of telomeres (ALT) pathway to elongate their telomeres and to gain an unlimited proliferation potential. Since the ALT pathway is unique to cancer cells, it represents a potentially valuable, currently unexploited target for anti-cancer therapies.&#xA;  &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4993795/&quot; rel=&quot;noreferrer&quot;&gt;B&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The ALT+ denotes cell lines that are positive for the ALT pathway.&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2018-09-18T03:41:50.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5053" PostTypeId="1" CreationDate="2018-09-18T11:32:00.763" Score="0" ViewCount="443" Body="&lt;p&gt;I am trying to figure out how to interpret the &quot;Per Base Sequence Quality&quot;? What does Position in reads (bp) mean? Also in order to draw this box-plot graphics, more than one quality scores are needed so we can sort them and calculate the mean, median and quartiles. In this diagram, how these values were calculated for each position? &lt;/p&gt;&#xA;" OwnerUserId="1978" LastEditorUserId="964" LastEditDate="2018-09-18T14:21:33.223" LastActivityDate="2023-06-25T11:45:43.873" Title="Fastqc- Per Base Sequence Quality" Tags="|ngs|sequence-analysis|fastq|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5054" PostTypeId="2" ParentId="5053" CreationDate="2018-09-18T11:57:18.197" Score="2" Body="&lt;p&gt;There is a pretty good description &lt;a href=&quot;https://www.bioinformatics.babraham.ac.uk/projects/fastqc/Help/3%20Analysis%20Modules/2%20Per%20Base%20Sequence%20Quality.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The position in the reads means literally the position in the reads, so position 2 means the quality of all bases on position 2 in the fastq file (all reads). In the description link, you can find how the boxplots and blue and red lines are calculated (and what they mean). They are all based on the total quality values of all bases on a certain position, all bases means all bases of all the reads on a particular position. &lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2018-09-18T12:25:08.353" LastActivityDate="2018-09-18T12:25:08.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5055" PostTypeId="1" CreationDate="2018-09-18T15:41:54.163" Score="1" ViewCount="1188" Body="&lt;p&gt;I'm trying to create a map in &lt;a href=&quot;http://popart.otago.ac.nz/index.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;PopArt&lt;/a&gt; that allows me to visualise the distribution of several species. These are my Traits (=species) and GeoTags blocks:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BEGIN TRAITS;&#xA;    Dimensions NTRAITS=9;&#xA;    Format labels=no missing=? separator=Comma;&#xA;    TraitLabels SNE Lsp A B C D 3n 4n Lep;&#xA;Matrix&#xA;SNE1 1,0,0,0,0,0,0,0,0&#xA;SNE2 1,0,0,0,0,0,0,0,0&#xA;SNE3 1,0,0,0,0,0,0,0,0&#xA;SNE4 1,0,0,0,0,0,0,0,0&#xA;SNE5 1,0,0,0,0,0,0,0,0&#xA;SNE6 1,0,0,0,0,0,0,0,0&#xA;PUR_4n 0,0,0,0,0,0,0,1,0&#xA;GRA 0,1,0,0,0,0,0,0,0&#xA;OJE 0,1,0,0,0,0,0,0,0&#xA;ESC 0,0,0,0,0,0,0,0,1&#xA;LNM 0,0,0,0,0,0,0,0,1&#xA;PIN 0,0,0,0,0,0,0,0,1&#xA;URD 0,0,0,0,0,0,0,0,1&#xA;VIS 0,0,0,0,0,0,0,0,1&#xA;LYE 0,0,0,0,0,0,0,0,1&#xA;BEN_A 0,0,1,0,0,0,0,0,0&#xA;ALJ1_A 0,0,1,0,0,0,0,0,0&#xA;ALJ2_A 0,0,1,0,0,0,0,0,0&#xA;ESP2_B 0,0,0,1,0,0,0,0,0&#xA;ESP3_B 0,0,0,1,0,0,0,0,0&#xA;ALC_C 0,0,0,0,1,0,0,0,0&#xA;AVA1_3n 0,0,0,0,0,0,1,0,0&#xA;AVA2_3n 0,0,0,0,0,0,1,0,0&#xA;PER_3n 0,0,0,0,0,0,1,0,0&#xA;TRI_3n 0,0,0,0,0,0,1,0,0&#xA;VDI_3n 0,0,0,0,0,0,1,0,0&#xA;VIL_3n 0,0,0,0,0,0,1,0,0&#xA;CVE1_4n 0,0,0,0,0,0,0,1,0&#xA;CVE2_4n 0,0,0,0,0,0,0,1,0&#xA;LNA1_4n 0,0,0,0,0,0,0,1,0&#xA;LNA2_4n 0,0,0,0,0,0,0,1,0&#xA;PRA1_4n 0,0,0,0,0,0,0,1,0&#xA;PRA2_4n 0,0,0,0,0,0,0,1,0&#xA;VEN1_4n 0,0,0,0,0,0,0,1,0&#xA;VEN2_4n 0,0,0,0,0,0,0,1,0&#xA;ZAR_4n 0,0,0,0,0,0,0,1,0&#xA;SFI1 0,0,0,0,0,0,0,0,1&#xA;SFI2 0,0,0,0,0,0,0,0,1&#xA;SIB 0,0,0,0,0,0,0,0,1&#xA;TIS1_D 0,0,0,0,0,1,0,0,0&#xA;TIS2_D 0,0,0,0,0,1,0,0,0&#xA;TIS3_D 0,0,0,0,0,1,0,0,0&#xA;TIS4_D 0,0,0,0,0,1,0,0,0&#xA;TVI_D 0,0,0,0,0,1,0,0,0&#xA;PVRFAS0001 1,0,0,0,0,0,0,0,0&#xA;PVRFAS0002 1,0,0,0,0,0,0,0,0&#xA;PVRFAS0024 1,0,0,0,0,0,0,0,0&#xA;PVRFAS0025 1,0,0,0,0,0,0,0,0&#xA;PVRFAS0027 1,0,0,0,0,0,0,0,0&#xA;PVRFAS0028 1,0,0,0,0,0,0,0,0&#xA;;&#xA;END;&#xA;Begin GeoTags;&#xA;    Dimensions NClusts=29;&#xA;    Format labels=no separator=Spaces;&#xA;    [ClustLatitude 37.15 37.77 37.13 39.37 37.11 40.60 40.59 37.04 40.61 38.71 36.70 43.48 40.18 40.24 40.57 40.26 37.10 37.86 38.52 36.56 40.23 38.01 39.58 38.75 40.54 40.55 38.70 40.04 40.69;&#xA;    ClustLongitude -3.70 -3.02 -3.52 -3.72 -3.03 -4.33 -4.33 -3.43 -2.44 0.05 -5.35 3.55 -6.55 -2.06 -4.16 0.26 -3.26 -1.56 -3.56 -4.86 -2.07 -2.87 -3.87 0.18 -4.18 -4.19 -0.49 -7.49 -2.59;]&#xA;Matrix&#xA;SNE1 37.044888 -3.135564&#xA;SNE2 37.044888 -3.135564&#xA;SNE3 37.044888 -3.135564&#xA;SNE4 37.044888 -3.135564&#xA;SNE5 37.044888 -3.135564&#xA;SNE6 37.044888 -3.135564&#xA;PUR_4n 37.133158 -3.524806&#xA;GRA 36.708272 -5.348794&#xA;OJE 36.562847 -4.853522&#xA;ESC 40.5659132 -4.2236186&#xA;LNM 40.609845 -4.344144&#xA;PIN 38.752157 0.180615&#xA;URD 39.370381 -3.720659&#xA;VIS 38.520059 -3.564201&#xA;LYE 39.580131 -3.870267&#xA;BEN_A 38.712603 0.052372&#xA;ALJ1_A 38.704010 -0.490801&#xA;ALJ2_A 38.704010 -0.490801&#xA;ESP2_B 37.865615 -1.569499&#xA;ESP3_B 37.865615 -1.569499&#xA;ALC_C 40.264366 0.267744&#xA;AVA1_3n 43.490194 3.563363&#xA;AVA2_3n 43.490194 3.563363&#xA;PER_3n 40.610461 -2.447806&#xA;TRI_3n 40.697087 -2.595488&#xA;VDI_3n 40.244599 -2.067766&#xA;VIL_3n 40.231843 -2.078962&#xA;CVE1_4n 40.559003 -4.199853&#xA;CVE2_4n 40.559003 -4.199853&#xA;LNA1_4n 40.597289 -4.330246&#xA;LNA2_4n 40.597289 -4.330246&#xA;PRA1_4n 37.113314 -3.035071&#xA;PRA2_4n 37.113314 -3.035071&#xA;VEN1_4n 37.155124 -3.703330&#xA;VEN2_4n 37.155124 -3.703330&#xA;ZAR_4n 40.547662 -4.184395&#xA;SFI1 40.042132 -7.498515&#xA;SFI2 40.042132 -7.498515&#xA;SIB 40.182598 -6.551259&#xA;TIS1_D 37.771593 -3.023286&#xA;TIS2_D 37.771593 -3.023286&#xA;TIS3_D 37.771593 -3.023286&#xA;TIS4_D 37.771593 -3.023286&#xA;TVI_D 38.0140003 -2.8751617&#xA;PVRFAS0001 37.100000 -3.260000&#xA;PVRFAS0002 37.100000 -3.260000&#xA;PVRFAS0024 37.100000 -3.260000&#xA;PVRFAS0025 37.100000 -3.260000&#xA;PVRFAS0027 37.100000 -3.260000&#xA;PVRFAS0028 37.100000 -3.260000&#xA;;&#xA;End;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If I upload my Nexus file like this, the program throws an error saying &quot;Too many rows in GeoTags block.&quot; However, I have 50 samples in both Traits and GeoTags. The same happens if I uncomment (in Nexus format, comments are in brackets) the ClustLatitude and ClustLongitude files. &lt;/li&gt;&#xA;&lt;li&gt;If I set the &quot;labels&quot; flag on &quot;yes&quot;, it takes the traits as the geographical locations and tries to find locations in the world starting with the trait names. &lt;/li&gt;&#xA;&lt;li&gt;If I comment the &quot;GeoTags&quot; block, PopArt correctly outputs a network.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What am I missing? Does somebody have a Nexus example where the traits do not correspond to geographical location of the samples?&lt;/p&gt;&#xA;" OwnerUserId="1637" LastEditorUserId="1637" LastEditDate="2018-09-19T09:40:17.470" LastActivityDate="2018-09-19T09:40:17.470" Title="How to format the Nexus file to create geographical maps with haplotypes in PopArt?" Tags="|networks|haplotypes|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="5056" PostTypeId="1" CreationDate="2018-09-18T16:43:40.800" Score="3" ViewCount="213" Body="&lt;p&gt;I have been working on nucleotide and protein data to reconstruct maximum parsimony trees using MEGA software to get parsimony tree, via &quot;random addition&quot; method is used to generate the initial tree(s).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to know that when randomly chosen sequences are added to reconstruct an initial tree, on what basis the best tree is chosen??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate if anyone can guide me about this method or suggest me some good literature about this in order to make me understand the concept of this method.&lt;/p&gt;&#xA;" OwnerUserId="3142" LastEditorUserId="3848" LastEditDate="2020-05-08T16:38:58.010" LastActivityDate="2020-05-08T16:38:58.010" Title="Random addition method for phylogenetic tree reconstruction" Tags="|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5057" PostTypeId="1" AcceptedAnswerId="5061" CreationDate="2018-09-18T17:50:48.840" Score="-1" ViewCount="638" Body="&lt;pre&gt;&lt;code&gt; gene   H4       H3       H2       H1       L8       L7       L6       L5       L4       L3       L2       L1          clust&#xA;AAGAB  7.540712 7.296325 7.030218 7.408974 8.269541 7.758752 9.253013 8.611002 8.705045 8.940146 8.855133 8.488776     3&#xA;ABCA1  7.112951 6.213973 7.200973 6.784828 8.044299 9.832347 8.558342 7.134570 8.993938 5.794208 8.271960 9.179501     5&#xA;ABCA6  1.822567 2.209167 3.379618 2.813433 3.180290 3.741516 3.060194 2.078485 6.536346 6.881403 5.927874 5.278854     2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is my output after doing kmean clustering so at the end I have my cluster number assigned to each gene. Now I want to show how the expression of my sample varies across each cluster, I am not sure how to transform the data and do it in &lt;code&gt;ggplot2&lt;/code&gt; as of now i doing something like this in base R plot &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;boxplot(da$H4 ~ k$cluster, &#xA;        xlab='Cluster', ylab='Genes', &#xA;        main=' Genes by Cluster')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For each sample I am checking, is there any way to show that with all the samples together, any suggestion or help would be highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="57" LastEditDate="2018-09-18T20:59:22.110" LastActivityDate="2018-09-19T07:21:37.583" Title="Boxplot of kmean cluster" Tags="|r|visualization|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5058" PostTypeId="2" ParentId="5041" CreationDate="2018-09-18T20:52:55.287" Score="0" Body="&lt;p&gt;The solutions here are more robust than mine since I rely on the structure of the file, but if you want a solution that is really easy to understand:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$grep -A 5 &quot;CDS&quot; file | grep -v &quot;CDS&quot; | grep -v &quot;codon_start&quot; | cut -f 2 -d \&quot;&#xA;UROS&#xA;uroporphyrinogen-III synthase isoform X1&#xA;XP_003755055.2&#xA;GeneID:100933602&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I take CDS line and 5 lines bellow, then I kick out CDS and codon_start (lines we are not interested in) and then I just bring what is between quote using cut. I also report the values in the order as they appear in the original file.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2018-09-18T20:52:55.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5060" PostTypeId="1" AcceptedAnswerId="5266" CreationDate="2018-09-19T05:35:08.123" Score="3" ViewCount="73" Body="&lt;p&gt;I am looking for a query to run on &lt;a href=&quot;https://portal.hmpdacc.org/query/s&quot; rel=&quot;nofollow noreferrer&quot;&gt;the HMP database&lt;/a&gt; that will return all subjects who have had BOTH 16s &lt;em&gt;and&lt;/em&gt; whole genomes sequence (WGS) workups.  I am currently using this query...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;file.matrix_type = 16s_community OR file.matrix_type = wgs_community AND (file.format = FASTA OR file.format = FASTQ) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and sifting through the results by adding them to my shopping cart, browsing the file types until I have added all of the samples from one subject, and then I check to see if I have at least one 16s and one WGS.  So far, I have not found one person who has had both 16s and WGS where both files were publicly accessible, and most do not have WGS at all.  There must be a way to get subjects with both workups done to make this faster.&lt;/p&gt;&#xA;" OwnerUserId="2446" LastEditorUserId="48" LastEditDate="2018-09-19T07:25:20.573" LastActivityDate="2018-10-24T02:03:11.790" Title="How to query the Human Microbiome Project (HMP) to find all subjects with both 16s and WGS workups?" Tags="|public-databases|genomics|data-download|metagenome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5061" PostTypeId="2" ParentId="5057" CreationDate="2018-09-19T07:21:37.583" Score="1" Body="&lt;p&gt;Assuming &lt;code&gt;da&lt;/code&gt; is a data-frame with the cluster as factor you can do the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(da) +&#xA;  geom_boxplot(aes(cluster, H4)) +&#xA;  xlab(&quot;Cluster&quot;) +&#xA;  ylab(&quot;Genes&quot;) +&#xA;  ggtitle(&quot;Genes by Cluster&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-09-19T07:21:37.583" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5062" PostTypeId="1" AcceptedAnswerId="5064" CreationDate="2018-09-19T12:57:31.127" Score="6" ViewCount="1556" Body="&lt;p&gt;I'm trying to get the FastA files for some accessions (like NC_001416.1). I did not managed to find an FTP server or direct link to these files (I want to get it from command line with &lt;code&gt;wget&lt;/code&gt;, not from a web browser). But I found an &quot;equivalent&quot; file in &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/sra/refseq/NC_001416.1&quot; rel=&quot;noreferrer&quot;&gt;ftp://ftp.ncbi.nlm.nih.gov/sra/refseq/NC_001416.1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This file is an SRA file that should be processed with the SRA-toolkit. In particular, fastq-dump seems to be the tools of choice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fastq-dump --fasta NC_001416.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I got a &lt;code&gt;NC_001416.1.fasta&lt;/code&gt; file. But the file contains the sequence I'm looking for in chunks of 5000 bases. The IDs in the output file are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;NC_001416.1.1 length=5000&#xA;&amp;gt;NC_001416.1.2 length=5000&#xA;&amp;gt;NC_001416.1.3 length=5000&#xA;&amp;gt;NC_001416.1.4 length=5000&#xA;&amp;gt;NC_001416.1.5 length=5000&#xA;&amp;gt;NC_001416.1.6 length=5000&#xA;&amp;gt;NC_001416.1.7 length=5000&#xA;&amp;gt;NC_001416.1.8 length=5000&#xA;&amp;gt;NC_001416.1.9 length=5000&#xA;&amp;gt;NC_001416.1.10 length=3502&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I need that sequence under a single ID:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;NC_001416.1 length=48502&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I cannot find the proper parameters to supply to &lt;code&gt;fastq-dump&lt;/code&gt; to achieve the desired result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know how could I get the FastA file I'm expecting? How to process properly the SRA file or where I could download the FastA file directly.&lt;/p&gt;&#xA;" OwnerUserId="3037" LastActivityDate="2018-09-19T19:56:34.670" Title="Convert SRA to FastA" Tags="|fasta|format-conversion|data-download|sratoolkit|" AnswerCount="4" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5063" PostTypeId="2" ParentId="5062" CreationDate="2018-09-19T13:07:38.733" Score="0" Body="&lt;p&gt;You can add the &lt;code&gt;--origfmt&lt;/code&gt; option, and see what comes out. Either you will have what you want, or perhaps they uploaded the sequences that way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See: &lt;a href=&quot;https://ncbi.github.io/sra-tools/fastq-dump.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ncbi.github.io/sra-tools/fastq-dump.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3378" LastActivityDate="2018-09-19T13:07:38.733" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5064" PostTypeId="2" ParentId="5062" CreationDate="2018-09-19T13:45:32.473" Score="5" Body="&lt;p&gt;Finally, I found an alternative to the SRA translation: a link that works! For those of you interested in knowing how to download FastA files from NCBI using an accession number, try the following link:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;https://www.ncbi.nlm.nih.gov/search/api/sequence/${accession}/?report=fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using &lt;code&gt;wget&lt;/code&gt; to download the accession used as example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wget -O NC_001416.1.fasta &quot;https://www.ncbi.nlm.nih.gov/search/api/sequence/NC_001416.1/?report=fasta&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3037" LastActivityDate="2018-09-19T13:45:32.473" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5065" PostTypeId="2" ParentId="5062" CreationDate="2018-09-19T14:33:34.333" Score="1" Body="&lt;p&gt;You can either do this directly as you show in your answer or, for a more sophisticated and flexible approach, use NCBI's &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;edirect&lt;/code&gt; tool&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db nucleotide -query 'NC_001416.1' | efetch -format fasta &amp;gt; NC_001416.1.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-09-19T14:33:34.333" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5066" PostTypeId="2" ParentId="5013" CreationDate="2018-09-19T14:42:00.863" Score="3" Body="&lt;p&gt;If I understand you correctly, after your most recent edit, you have 5,000 sequences that are 63 bp long and in each you simply want to replace the nucleotides from 21 through 43 with &lt;code&gt;-&lt;/code&gt; unless that nucleotide does not match the primer. Since &lt;code&gt;-&lt;/code&gt; is normally used to represent gaps, I recommend using &lt;code&gt;.&lt;/code&gt; instead for matching nucleotides.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know you're looking for an R solution, but I don't usually use R for string manipulations and don't have time at the moment to figure it out. So, assuming this might be urgent for you, below is a Python solution. This uses your example data which shows 11 nucleotides before the primer and 9 after it (so you'll need to adjust it to work with your actual case in which 20 nucleotides are before and after the primer). In Python, strings are arrays of characters, by the way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are your example sequences are in the file &lt;code&gt;seqs.txt&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ATGACCATACGATGTCGAAGCAAATATCAATTGAATCGTAGGG&#xA;ATGAGCATACCATGTGAAAGCTAATATCAATTGATTCGAACGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the code that gives you the expected output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/env python&#xA;import sys&#xA;primer = 'ATGTGGAAGCAAATATCAAATGA'&#xA;&#xA;print('-' * 11 + primer + '-' * 9 )&#xA;with open('seqs.txt','r') as f:&#xA;    for gene in f:&#xA;        sys.stdout.write(gene[0:11])&#xA;        for gene_index in range(11,34):&#xA;            primer_index = gene_index - 11&#xA;            if gene[gene_index] == primer[primer_index]:&#xA;                sys.stdout.write('.')&#xA;            else:&#xA;                sys.stdout.write(gene[gene_index])&#xA;        print(gene[34:])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Running this gives you:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-----------ATGTGGAAGCAAATATCAAATGA---------&#xA;ATGACCATACG....C..............T...ATCGTAGGG&#xA;ATGAGCATACC.....A....T........T...TTCGAACGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2878" LastEditorUserId="2878" LastEditDate="2018-09-19T17:22:08.150" LastActivityDate="2018-09-19T17:22:08.150" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5067" PostTypeId="1" CreationDate="2018-09-19T17:01:11.053" Score="1" ViewCount="283" Body="&lt;p&gt;I am downloading a set of Copy Number Variation samples from the &lt;a href=&quot;https://docs.gdc.cancer.gov/Data/Bioinformatics_Pipelines/CNV_Pipeline/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GDC data API&lt;/a&gt;. I noticed that some cases have multiple files (aliquots) for the same case (see for example &lt;a href=&quot;https://portal.gdc.cancer.gov/cases/a8e79b70-76c4-4d4f-8e1d-37daebf2a921&quot; rel=&quot;nofollow noreferrer&quot;&gt;this case&lt;/a&gt; --- &lt;a href=&quot;https://portal.gdc.cancer.gov/query?filters=%7B%22op%22%3A%22and%22%2C%22content%22%3A%5B%7B%22op%22%3A%22and%22%2C%22content%22%3A%5B%7B%22op%22%3A%22in%22%2C%22content%22%3A%7B%22field%22%3A%22cases.case_id%22%2C%22value%22%3A%5B%22a8e79b70-76c4-4d4f-8e1d-37daebf2a921%22%5D%7D%7D%2C%7B%22op%22%3A%22and%22%2C%22content%22%3A%5B%7B%22op%22%3A%22in%22%2C%22content%22%3A%7B%22field%22%3A%22files.data_category%22%2C%22value%22%3A%5B%22Copy%20Number%20Variation%22%5D%7D%7D%2C%7B%22op%22%3A%22and%22%2C%22content%22%3A%5B%7B%22op%22%3A%22in%22%2C%22content%22%3A%7B%22field%22%3A%22files.data_type%22%2C%22value%22%3A%5B%22Masked%20Copy%20Number%20Segment%22%5D%7D%7D%2C%7B%22op%22%3A%22in%22%2C%22content%22%3A%7B%22field%22%3A%22cases.samples.sample_type%22%2C%22value%22%3A%5B%22Primary%20Tumor%22%2C%22Primary%20Blood%20Derived%20Cancer%20-%20Bone%20Marrow%22%2C%22Primary%20Blood%20Derived%20Cancer%20-%20Peripheral%20Blood%22%2C%22Tumor%22%5D%7D%7D%5D%7D%5D%7D%5D%7D%5D%7D&amp;amp;query=cases.case_id%20in%20%5B%22a8e79b70-76c4-4d4f-8e1d-37daebf2a921%22%5D%20and%20files.data_category%20in%20%5B%22Copy%20Number%20Variation%22%5D%20and%20files.data_type%20in%20%5B%22Masked%20Copy%20Number%20Segment%22%5D%20and%20cases.samples.sample_type%20in%20%5B%22Primary%20Tumor%22%2C%22Primary%20Blood%20Derived%20Cancer%20-%20Bone%20Marrow%22%2C%20%22Primary%20Blood%20Derived%20Cancer%20-%20Peripheral%20Blood%22%2C%20%22Tumor%22%5D&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; the set of tumor samples with masked CNV in which it would be necessary to choose one). I need to have one file for each case in the resulting data matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I looked through the &lt;a href=&quot;https://github.com/NCI-GDC&quot; rel=&quot;nofollow noreferrer&quot;&gt;GDC repository&lt;/a&gt; but there's no indication, the same in the &lt;a href=&quot;https://docs.gdc.cancer.gov/Data/Introduction/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GDC API&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think they are samples processed in different institution or at different times. I looked at the metadata but I cannot find a good criteria to choose one of them. Here's the metadata of an example set for a specific TCGA-LUSC case:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{'data_release': '12.0', 'data_type': 'Masked Copy Number Segment', 'updated_datetime': '2018-08-07T15:59:14.863537+00:00', 'file_name': 'ELBOW_p_TCGAb39_SNP_N_GenomeWideSNP_6_A07_598508.nocnv_grch38.seg.v2.txt', 'submitter_id': 'e65d2f5e-96b5-4128-814e-368d830723f5_dr12_nocnv', 'file_id': '9405ea1d-1241-4e87-b1cd-a8e85008964d', 'file_size': 27971, 'id': '9405ea1d-1241-4e87-b1cd-a8e85008964d', 'created_datetime': '2018-05-20T21:25:50.066318-05:00', 'md5sum': '0faad919fb8ccfaa1cbcf04990433b2c', 'data_format': 'TXT', 'acl': ['open'], 'access': 'open', 'platform': 'Affymetrix SNP 6.0', 'state': 'released', 'version': '1', 'data_category': 'Copy Number Variation', 'type': 'copy_number_segment', 'experimental_strategy': 'Genotyping Array'}, &#xA;{'data_release': '12.0', 'data_type': 'Masked Copy Number Segment', 'updated_datetime': '2018-08-07T15:59:14.863537+00:00', 'file_name': 'WHIRR_p_TCGA_168_169_170_redo_N_GenomeWideSNP_6_B10_845144.nocnv_grch38.seg.v2.txt', 'submitter_id': '9fda454b-946a-4760-8f3b-c24bdad9dbfe_dr12_nocnv', 'file_id': '91ac1284-d4e7-4be3-8266-9fec63b41cf3', 'file_size': 19377, 'id': '91ac1284-d4e7-4be3-8266-9fec63b41cf3', 'created_datetime': '2018-05-20T21:50:19.566069-05:00', 'md5sum': 'c2a55b28417de57d00c8e3eda038ca6b', 'data_format': 'TXT', 'acl': ['open'], 'access': 'open', 'platform': 'Affymetrix SNP 6.0', 'state': 'released', 'version': '1', 'data_category': 'Copy Number Variation', 'type': 'copy_number_segment', 'experimental_strategy': 'Genotyping Array'}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In particular I thought about filtering by timestamp, but they have very close timestamps which are probably the timestamp of the final upload/update, not of the sample creation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I noticed that one sample has the word &quot;redo&quot; in the filename. However, I need a methodology that can be automatized through a Python script - I cannot do it by hand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I finally thought about taking the sample with more data, i.e. the sample with highest file size. However, this can be a bad criteria if the sample with smallest file size has a higher quality due to some kind of filtering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have you already encountered this problem and find a solution?&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Edit&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;By looking at the &lt;a href=&quot;https://docs.gdc.cancer.gov/Encyclopedia/pages/TCGA_Barcode/&quot; rel=&quot;nofollow noreferrer&quot;&gt;legacy TCGA barcode&lt;/a&gt;, the difference is due to portions distributed on one/more plates, analyzed by the same center. The two barcodes are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;TCGA-66-2791-01A-01D-&lt;strong&gt;0978&lt;/strong&gt;-01&lt;/li&gt;&#xA;&lt;li&gt;TCGA-66-2791-01A-01D-&lt;strong&gt;1969&lt;/strong&gt;-01&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In boldface the plate id. My hypothesis is that by chance, the biggest one has shorter segments (more resolution, more lines) than the smallest, but both should have the same information (assuming the two aliquot have the same CNV).&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2018-09-19T19:06:08.177" LastActivityDate="2018-09-19T19:06:08.177" Title="Multiple files for case in GDC data portal" Tags="|api|copy-number|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5068" PostTypeId="1" CreationDate="2018-09-19T19:37:58.433" Score="1" ViewCount="1187" Body="&lt;p&gt;I have RNAseq data from a relatively complicated experimental design with variables = genotype, treatment, time, and batch. I have 2 biological replicates for each genotype/condition, however unfortunately in the first iteration of the experiment one sample had poor RNA quality and is unusable. To get around this lack of a replicate, we repeated the experiment for the two biological replicates in the condition of the lost sample, and did a new library prep/sequencing from this repeated experiment.  The repeated experiment obviously lead to a batch effect, and although it isn't the main driver of variance, it is a significant factor.  I would like to model or remove the batch effect from the experiment using DESeq2 if possible, but after reading this similar post: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/232/can-i-model-technical-replicates-in-deseq2&quot; title=&quot;Can I model technical replicates in DESeq2?&quot;&gt;can I model technical replicates in DESeq2?&lt;/a&gt; I am concerned about inflating the power of the test for those conditions that were replicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Here is a diagram of the experimental design to make it more easy to understand:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lzgEI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lzgEI.png&quot; alt=&quot;experiment design diagram&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My &quot;colData&quot; for the DESeq2 object looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lcN17.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lcN17.png&quot; alt=&quot;colData image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My previous thought was to simply collapse the genotype+time+treatment variables into a single combined factor &quot;condition&quot; and then model the batch effect using a design of &quot;~batch + condition&quot;.  However this doesn't really address the concern of inflating the power of the tests due to technical replication.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I Just collapse the counts of these replicated conditions even though the batch affect will affect the combined count values? Or should I use a different tool than DESeq2 ?  Or is there a way for me to remove the batch effect prior to collapsing the counts for replicates?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What would  you guys recommend?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="39" LastEditDate="2018-09-19T20:19:27.053" LastActivityDate="2018-09-20T01:44:46.113" Title="DESeq2 complicated design - effect of replicated samples" Tags="|rna-seq|differential-expression|deseq2|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5069" PostTypeId="2" ParentId="5062" CreationDate="2018-09-19T19:56:34.670" Score="2" Body="&lt;p&gt;Here is a Biopython solution via &lt;code&gt;Entrez&lt;/code&gt; utilies, simply copy the code into a python script and execute it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez, SeqIO&#xA;&#xA;Entrez.email = 'me@email.com'&#xA;&#xA;def get_fasta_from_ids(ids_list):&#xA;    handle = Entrez.efetch(db='nucleotide', id=','.join(ids_list), rettype='fasta')&#xA;    for record in SeqIO.parse(handle, 'fasta'):&#xA;        yield record&#xA;&#xA;SeqIO.write(get_fasta_from_ids(['NC_001416.1']), 'NC_001416.1.fa', 'fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can add more ids to the input list to retrieve multiple records at once.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2018-09-19T19:56:34.670" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5070" PostTypeId="2" ParentId="5068" CreationDate="2018-09-20T01:44:46.113" Score="2" Body="&lt;p&gt;I agree with @devon-ryan that the design matrix looks reasonable, i.e. that it's okay to treat the new samples as biological replicates of the original batch, and that the batch number should be included in that matrix (assuming DESeq2 doesn't complain about it) to filter out most of the batch effects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DESeq2 tends to be conservative in its differential expression calculations. While it's true that more replicates in one sample will increase the number of differentially-expressed genes that are statistically significant (because the biological variation can be better modelled), I wouldn't expect this to change the overall picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, whole-genome differential expression testing should not be used on its own to demonstrate biological relevance. I recommend as a first-pass sanity check that people compare any differential expression results to what is seen on a genome browser to identify any quirky things going on (e.g. expression across only a part of the gene). Ideally, results should be confirmed with targeted analyses on the same (or similar) biological samples, either at the transcript level (e.g. qPCR) or at the protein level (e.g. Western Blot). When aggregating evidence from multiple analyses, the specific way in which one of those analyses was done is less important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Aside: It'd obviously be better [in the future] to make sure that your sample replicates are both robust to drop-out, and allow for outliers to be noticed, reducing the batch-effect problem. In practise this means a &lt;em&gt;minimum&lt;/em&gt; of four replicates per informative condition, but because funders / researchers are well known for only choosing the minimum, I'd just state that six replicates per sample as what is needed to produce good results.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-09-20T01:44:46.113" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5071" PostTypeId="1" AcceptedAnswerId="5072" CreationDate="2018-09-21T02:30:29.430" Score="1" ViewCount="370" Body="&lt;p&gt;There is a &lt;a href=&quot;https://github.com/broadinstitute/gtex-pipeline/tree/master/rnaseq&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github report&lt;/a&gt; for Broad's pipeline on RNA-Seq. It has a docker build file and WDL scripts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I don't see how any of the WDL script is being used in the docker (Dockerfile). It looks like Broad's docker runs on the standard command-line tools, but not via the WDL language.&#xA;How do the Broad bioinformaticians take advantage of the WDL scripts?&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="2776" LastEditDate="2020-04-24T06:36:05.447" LastActivityDate="2020-04-24T06:36:05.447" Title="Using Broad Institute RNAseq pipeline" Tags="|rna-seq|docker|wdl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5072" PostTypeId="2" ParentId="5071" CreationDate="2018-09-21T07:18:08.260" Score="2" Body="&lt;p&gt;You need to follow the &lt;a href=&quot;https://github.com/broadinstitute/gtex-pipeline/blob/master/rnaseq/README.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;readme&lt;/a&gt; of this folder to  use the pipeline, there you will find instructions about how to align, mark duplicates, quantificate, and aggregate the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to execute the WDL scripts you can follow &lt;a href=&quot;https://software.broadinstitute.org/wdl/documentation/execution.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;this guide&lt;/a&gt; using Cromwell. Basically use: &lt;code&gt;java -jar Cromwell.jar run myWorkflow.wdl --inputs myWorkflow_inputs.json&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-09-21T07:18:08.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5073" PostTypeId="1" AcceptedAnswerId="5108" CreationDate="2018-09-21T12:13:48.777" Score="3" ViewCount="110" Body="&lt;p&gt;I'm working on a project about detecting SNP association with a disease. As I understand, SNP is a single variation of the nucleotide that occurs for more than 1% of the population. So, if a gene is AT&lt;strong&gt;T&lt;/strong&gt;G and there's a variation AT&lt;strong&gt;A&lt;/strong&gt;G then the SNP is &lt;strong&gt;A&lt;/strong&gt;. However, I couldn't connect this idea with the dataset in hand. The rows in my dataset represent each patient and the columns contain SNP information. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ID  exm355 exm615&#xA;1     T_T   A_C&#xA;2     T_T   C_C&#xA;3     A_T   C_C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have no idea why the SNP columns contain 2 nucleotides(T_T, A_T, A_C, C_C). As the definition of SNP, I thought it should show only the variant nucleotide or am I misunderstanding anything? How could I interpret T_T or C_C and how could I know which nucleotide is the variation from the common ones in the population?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The source says that the case and control subjects were genotyped by the &lt;a href=&quot;http://www.agrf.org.au&quot; rel=&quot;nofollow noreferrer&quot;&gt;Australian Genome Research Facility&lt;/a&gt; using the Illumina HumanExome BeadChip-12v1_A, which exonic content consists of &lt;span class=&quot;math-container&quot;&gt;$&amp;gt;$&lt;/span&gt; 250 000&#xA;markers representing diverse populations and a range of common&#xA;conditions. An individual was duplicated to test genotyping reliability and&#xA;quality. The identity by descent (IBD) matrix between all pairs of individuals&#xA;was estimated after linkage disequilibrium (LD) pruning and used for&#xA;quality control and for the mixed linear models analyses.&lt;/p&gt;&#xA;" OwnerUserId="3388" LastEditorUserId="48" LastEditDate="2018-09-22T10:51:52.210" LastActivityDate="2018-09-23T06:46:06.273" Title="Understanding SNP coding for association analysis" Tags="|snp|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5074" PostTypeId="1" CreationDate="2018-09-21T15:27:33.293" Score="2" ViewCount="142" Body="&lt;p&gt;I used&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ape::read.GenBank&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and then used the function&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ORFik::findORFs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to find ORFs in the sequence of interest. But the former one results in a &lt;code&gt;DNAbin&lt;/code&gt; class format and the latter one requires &lt;code&gt;FASTA&lt;/code&gt; format.&lt;/p&gt;&#xA;&lt;p&gt;Is there a function available to convert the sequence from &lt;code&gt;DNAbin&lt;/code&gt; class format to &lt;code&gt;FASTA&lt;/code&gt; format?&lt;/p&gt;&#xA;&lt;p&gt;(&lt;em&gt;I could think of directly downloading the FASTA format file and use the function.&lt;/em&gt;)&lt;/p&gt;&#xA;" OwnerUserId="23" LastEditorUserId="3848" LastEditDate="2020-07-18T13:08:53.857" LastActivityDate="2020-07-18T13:08:53.857" Title="How to extract Genbank sequence and identify ORFs from the sequence in R?" Tags="|r|fasta|genbank|orf|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5075" PostTypeId="1" CreationDate="2018-09-21T19:01:43.293" Score="1" ViewCount="211" Body="&lt;p&gt;I have just started using BlastKOALA KEGG which has been useful in annotating (aminoacid) sequences. This is their website: &lt;a href=&quot;https://www.kegg.jp/blastkoala/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kegg.jp/blastkoala/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you get results, there are links for downloading. However these links will not download all detailed query search results, but just general notes already on the screen. To get these results I need to manually click on each query result on the page, which becomes impracticable with &gt;500 entries. Thus I think need is a tool to download all linked contents from a webpage. I have been trying 'wget' however it doesn't work. It says 'Requested Job Not Found' whatever I do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please, did anyone every try to achieve this? Thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. I have provided a working link as an example, below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.kegg.jp/kegg-bin/blastkoala_result_list?id=7345faab3969a3ef10a4a7542ba749386068138f&amp;amp;passwd=NKfm46&amp;amp;type=blastkoala&amp;amp;code=user&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kegg.jp/kegg-bin/blastkoala_result_list?id=7345faab3969a3ef10a4a7542ba749386068138f&amp;amp;passwd=NKfm46&amp;amp;type=blastkoala&amp;amp;code=user&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it should remain active for a week. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have inserted a picture showing examples of query result links below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WrVtw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WrVtw.png&quot; alt=&quot;Picture showing a fraction of results table, where each sequence number is a link to detailed search results; clicking on &amp;quot;Download Detail&amp;quot; merely returns the table as showing on screen already.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this moment I would like to end up with the listed hit results for each of the queries, whose links are showing in blue; one result per file. In short, it is using wget for downloading all links in the page into a dedicated folder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="2329" LastEditorUserId="2329" LastEditDate="2018-09-25T15:31:27.130" LastActivityDate="2018-09-25T15:31:27.130" Title="Downloading full search results from BlastKOALA?" Tags="|blast|sequence-annotation|kegg|functional-annotation|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="5076" PostTypeId="1" AcceptedAnswerId="5107" CreationDate="2018-09-21T22:14:43.973" Score="7" ViewCount="2085" Body="&lt;h2&gt;Library Prep&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I have a &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/22652625&quot; rel=&quot;noreferrer&quot;&gt;Hi-C&lt;/a&gt; library prepped using an enzyme that cuts at &lt;code&gt;GATC&lt;/code&gt;, so it leaves &lt;code&gt;GATCGATC&lt;/code&gt; as the junction sequences. This library was sequenced on a 2x150 PE Illumina run.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;Data Pre-processing&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The reads were adapter and quality trimming with &lt;a href=&quot;http://www.usadellab.org/cms/?page=trimmomatic&quot; rel=&quot;noreferrer&quot;&gt;trimmomatic&lt;/a&gt; paired-end mode, keeping reads that overlap.&lt;/li&gt;&#xA;&lt;li&gt;Finally, the forward and reverse reads were trimmed such that all bases 3' of the junction sequence (&lt;code&gt;GATCGATC&lt;/code&gt;). For example, the read:&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;5' AAAATT**GATCGATC**AATTATGGATAA 3'&lt;/code&gt; becomes: &lt;code&gt;5' AAAATT**GATC 3'&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;In this process, sequences that don't contain the junction sequences were left alone. Trimming junction sequences in this way makes it such that each read pair maps to only two places in the genome. No three-way or four-way chimeric mappings are possible since everything &lt;code&gt;3'&lt;/code&gt; of the junction sequence was removed.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;My Question&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What is the correct way to now map these trimmed Hi-C reads to a genome using &lt;code&gt;bwa mem&lt;/code&gt;?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;Progress so far/things I have tried&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In the past I have just tried mapping using &lt;code&gt;bwa mem&lt;/code&gt; with no special parameters using this command: &lt;code&gt;bwa mem ref.fa hi-c_f.fq.gz hi-c_r.fq.gz&lt;/code&gt;. One concern is that the computations that &lt;code&gt;bwa mem&lt;/code&gt; performs on insert sizes in this mode will cause some mappings to be faulty or that read pairs with chromosome-length information will be removed from the alignment because they fall on the tail of the insert size distribution (non-Gaussian, but log decay).&lt;/li&gt;&#xA;&lt;li&gt;Another option to consider is mapping each file, &lt;code&gt;hi-c_f.fq.gz&lt;/code&gt; and &lt;code&gt;hi-c_r.fq.gz&lt;/code&gt;, independently then adding mate information with &lt;code&gt;samtools fixmate&lt;/code&gt;. However, I do not know how to do this.&lt;/li&gt;&#xA;&lt;li&gt;A last option that I have considered is to use &lt;code&gt;bwa mem&lt;/code&gt; in paired-end mode, but to use a special flag. In the past I believe that &lt;code&gt;-5&lt;/code&gt; was the correct flag, but in &lt;a href=&quot;https://sourceforge.net/p/bio-bwa/mailman/message/36050442/&quot; rel=&quot;noreferrer&quot;&gt;this post&lt;/a&gt; Heng Li discusses that the &lt;code&gt;-5&lt;/code&gt; flag behavior has changed, as well as a new &lt;code&gt;-q&lt;/code&gt; flag. In &lt;a href=&quot;http://lh3.github.io/2018/04/02/minimap2-and-the-future-of-bwa&quot; rel=&quot;noreferrer&quot;&gt;this post&lt;/a&gt; Heng Li discusses that he thinks &lt;code&gt;bwa mem&lt;/code&gt; is still a good option for mapping Hi-C data but does not discuss the proper flags.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;Limitations&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The answer should provide the correct way to map Hi-C using &lt;code&gt;bwa mem&lt;/code&gt;, but other software suggestions will also be considered.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-09-26T01:17:45.883" Title="What is the correct way to map Hi-C data with bwa mem?" Tags="|read-mapping|bwa|illumina|hi-c|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5077" PostTypeId="2" ParentId="756" CreationDate="2018-09-22T00:39:03.073" Score="0" Body="&lt;p&gt;I would recommend using GEOmetadb from Bioconductor. GEOmetadb is a simple wrapper that will download an up-to-date SQLite database with the metadata for all entities in GEO. You can then use your favorite SQL client to query the database quickly, since it's offline, refining your search as you go.&lt;/p&gt;&#xA;" OwnerUserId="3391" LastActivityDate="2018-09-22T00:39:03.073" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5078" PostTypeId="5" CreationDate="2018-09-22T06:49:38.923" Score="0" Body="&lt;p&gt;Rcurl provides a great deal of control over the HTTP/FTP/... connection and the form of the request while providing a higher-level interface than is available just using R socket connections. Additionally, the underlying implementation is robust and extensive, supporting FTP/FTPS/TFTP (uploads and downloads), SSL/HTTPS, telnet, dict, ldap, and also supports cookies, redirects, authentication, etc.&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-09-22T10:47:32.737" LastActivityDate="2018-09-22T10:47:32.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5079" PostTypeId="4" CreationDate="2018-09-22T06:49:38.923" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-09-22T06:49:38.923" LastActivityDate="2018-09-22T06:49:38.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5080" PostTypeId="5" CreationDate="2018-09-22T06:52:53.107" Score="0" Body="&lt;p&gt;An ORF is a continuous stretch of codons that contain a start codon (usually AUG) and a stop codon (usually UAA, UAG or UGA). &lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-09-22T10:47:04.617" LastActivityDate="2018-09-22T10:47:04.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5081" PostTypeId="4" CreationDate="2018-09-22T06:52:53.107" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-09-22T06:52:53.107" LastActivityDate="2018-09-22T06:52:53.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5082" PostTypeId="5" CreationDate="2018-09-22T06:56:09.850" Score="0" Body="&lt;p&gt;Bash is the Bourne Again SHell. Bash is an sh-compatible shell that incorporates useful features from the Korn shell (ksh) and C shell (csh).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;The improvements offered by Bash include:&#xA;&#xA;Command line editing&#xA;Unlimited size command history&#xA;Job Control&#xA;Shell Functions and Aliases&#xA;Indexed arrays of unlimited size&#xA;Integer arithmetic in any base from two to sixty-four&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-09-22T10:47:23.390" LastActivityDate="2018-09-22T10:47:23.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5083" PostTypeId="4" CreationDate="2018-09-22T06:56:09.850" Score="0" Body="Use this tag to refer to the questions related to the Linux and Unix shell abbreviated as bash (Bourne Again SHell). " OwnerUserId="1194" LastEditorUserId="298" LastEditDate="2024-02-23T11:23:00.263" LastActivityDate="2024-02-23T11:23:00.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5084" PostTypeId="5" CreationDate="2018-09-22T06:57:52.627" Score="0" Body="&lt;p&gt;R objects are exposed as instances of Python-implemented classes, with R functions as bound methods to those objects in a number of cases. rpy2 requires an installation of R within Python and it is recommended to use Anaconda 3 for this purpose. Again at the time of writing it is out of date with major upgrades in R, most noably ggplot2 and therefore results in restricted functionality&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="3848" LastEditDate="2020-07-06T13:13:20.550" LastActivityDate="2020-07-06T13:13:20.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5085" PostTypeId="4" CreationDate="2018-09-22T06:57:52.627" Score="0" Body="Python's inferface to R, but at the time of writing is out of date with several upgrades within R, most notably ggplot2" OwnerUserId="-1" LastEditorUserId="3848" LastEditDate="2020-07-06T13:13:20.550" LastActivityDate="2020-07-06T13:13:20.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5086" PostTypeId="5" CreationDate="2018-09-22T07:01:02.157" Score="0" Body="&lt;p&gt;Docker is used to run software packages called &quot;containers&quot;. In a typical example use case, one container runs a web server and web application, while a second container runs a database server that is used by the web application. Containers are isolated from each other and bundle their own tools, libraries and configuration files; they can communicate with each other through well-defined channels.&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-09-22T10:46:33.537" LastActivityDate="2018-09-22T10:46:33.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5087" PostTypeId="4" CreationDate="2018-09-22T07:01:02.157" Score="0" Body="Docker is a computer program that performs operating-system-level virtualization, also known as &quot;containerization&quot;." OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-09-22T10:47:17.033" LastActivityDate="2018-09-22T10:47:17.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5088" PostTypeId="5" CreationDate="2018-09-22T07:03:12.677" Score="0" Body="&lt;p&gt;Historic phylogenetics package which used a specific file format where the header comprised the number of taxa in the alignment together with the length of the alignment. The programs have now been superseded but the alignment format remains central to numerous phylogenetics packages such as RAxML and PhylML. Biopython AlignIO is able to manipulate this alignment format.&lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="3848" LastEditDate="2020-07-06T23:01:11.587" LastActivityDate="2020-07-06T23:01:11.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5089" PostTypeId="4" CreationDate="2018-09-22T07:03:12.677" Score="0" Body="Phylogenetics package and multiple alignment format" OwnerUserId="-1" LastEditorUserId="3848" LastEditDate="2020-07-06T23:01:11.587" LastActivityDate="2020-07-06T23:01:11.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5090" PostTypeId="5" CreationDate="2018-09-22T07:04:28.983" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-09-22T07:04:28.983" LastActivityDate="2018-09-22T07:04:28.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5091" PostTypeId="4" CreationDate="2018-09-22T07:04:28.983" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-09-22T07:04:28.983" LastActivityDate="2018-09-22T07:04:28.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5092" PostTypeId="1" AcceptedAnswerId="5116" CreationDate="2018-09-22T15:54:54.710" Score="3" ViewCount="207" Body="&lt;p&gt;Given a normal protein sequence with the 20 standard amino acids, how can the 'Autocovariances', 'autocorrelations', and 'autocorrelation coefficients' of the sequence be calculated?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is meant by these terms in the context of a protein sequence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the appropriate algorithms and formulas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any existing package/function to calculate these values?&lt;/p&gt;&#xA;" OwnerUserId="1606" LastEditorUserId="1606" LastEditDate="2018-09-25T10:05:39.327" LastActivityDate="2018-09-25T10:05:39.327" Title="How can the Autocovariances, autocorrelations, and autocorrelation coefficients be calculated from a Protein Amino Acid Sequence?" Tags="|proteins|statistics|features|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5093" PostTypeId="5" CreationDate="2018-09-22T16:05:19.090" Score="0" Body="&lt;p&gt;Array- and sequence-based data are accepted. Tools are provided to help users query and download experiments and curated gene expression profiles. &lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-09-24T14:29:14.800" LastActivityDate="2018-09-24T14:29:14.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5094" PostTypeId="4" CreationDate="2018-09-22T16:05:19.090" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-09-22T16:05:19.090" LastActivityDate="2018-09-22T16:05:19.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5095" PostTypeId="5" CreationDate="2018-09-22T16:09:50.450" Score="0" Body="&lt;p&gt;In mathematics and computer science, an algorithm is an unambiguous specification of how to solve a class of problems. Algorithms can perform calculation, data processing and automated reasoning tasks. &lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-09-24T14:28:10.647" LastActivityDate="2018-09-24T14:28:10.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5096" PostTypeId="4" CreationDate="2018-09-22T16:09:50.450" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-09-22T16:09:50.450" LastActivityDate="2018-09-22T16:09:50.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5097" PostTypeId="5" CreationDate="2018-09-22T16:12:28.720" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-09-22T16:12:28.720" LastActivityDate="2018-09-22T16:12:28.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5098" PostTypeId="4" CreationDate="2018-09-22T16:12:28.720" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-09-22T16:12:28.720" LastActivityDate="2018-09-22T16:12:28.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5099" PostTypeId="5" CreationDate="2018-09-22T16:14:32.737" Score="0" Body="&lt;p&gt;The most widely-used tools enable genome arithmetic: that is, set theory on the genome. For example, bedtools allows one to intersect, merge, count, complement, and shuffle genomic intervals from multiple files in widely-used genomic file formats such as BAM, BED, GFF/GTF, VCF. While each individual tool is designed to do a relatively simple task (e.g., intersect two interval files), quite sophisticated analyses can be conducted by combining multiple bedtools operations on the UNIX command line.&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-09-24T14:25:47.040" LastActivityDate="2018-09-24T14:25:47.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5100" PostTypeId="4" CreationDate="2018-09-22T16:14:32.737" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-09-22T16:14:32.737" LastActivityDate="2018-09-22T16:14:32.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5101" PostTypeId="5" CreationDate="2018-09-22T16:17:35.783" Score="0" Body="&lt;p&gt;It supports several common biological data formats, including optionally gzip'ed BED, GFF, SAM, VCF, FASTA/Q and TAB-delimited formats with column names. It also adds a few built-in functions and a command line option to use TAB as the input/output delimiter. When the new functionality is not used, bioawk is intended to behave exactly the same as the original BWK awk.&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1157" LastEditDate="2018-11-07T13:34:37.323" LastActivityDate="2018-11-07T13:34:37.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5102" PostTypeId="4" CreationDate="2018-09-22T16:17:35.783" Score="0" Body="For questions about bioawk, an extension of the unix core utility command awk." OwnerUserId="1157" LastEditorUserId="1157" LastEditDate="2018-11-07T13:34:32.497" LastActivityDate="2018-11-07T13:34:32.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5103" PostTypeId="5" CreationDate="2018-09-22T16:19:16.090" Score="0" Body="&lt;p&gt;Bioconda consists of:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a repository of recipes hosted on GitHub&#xA;a build system that turns these recipes into conda packages&#xA;a repository of more than 3000 bioinformatics packages ready to use with conda install&#xA;Over 250 contributors that add, modify, update and maintain the recipes&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-09-24T14:29:02.417" LastActivityDate="2018-09-24T14:29:02.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5104" PostTypeId="4" CreationDate="2018-09-22T16:19:16.090" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-09-22T16:19:16.090" LastActivityDate="2018-09-22T16:19:16.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5105" PostTypeId="5" CreationDate="2018-09-22T16:22:47.297" Score="0" Body="&lt;p&gt;BigWig files are compressed binary indexed files containing data at several resolutions that allow the high-performance display of next-generation sequencing experiment results in the UCSC Genome Browser.&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-09-24T14:26:00.420" LastActivityDate="2018-09-24T14:26:00.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5106" PostTypeId="4" CreationDate="2018-09-22T16:22:47.297" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2018-09-22T16:22:47.297" LastActivityDate="2018-09-22T16:22:47.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5107" PostTypeId="2" ParentId="5076" CreationDate="2018-09-23T04:37:45.580" Score="4" Body="&lt;p&gt;See &lt;a href=&quot;https://github.com/hms-dbmi/hic-data-analysis-bootcamp&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, in particular &lt;a href=&quot;https://hms-dbmi.github.io/hic-data-analysis-bootcamp/#10&quot; rel=&quot;nofollow noreferrer&quot;&gt;slide #10&lt;/a&gt; from the tutorial:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;bwa mem -5SPM ref.fa read1.fq read2.fq &amp;gt; out.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here, &lt;code&gt;-SP&lt;/code&gt; disables pairing. The &lt;a href=&quot;http://aidenlab.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Aiden lab&lt;/a&gt; and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/30166492&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; also use a similar command line. Beware that there are many Hi-C pipelines, but not many are using bwa-mem.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-09-23T04:37:45.580" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5108" PostTypeId="2" ParentId="5073" CreationDate="2018-09-23T06:46:06.273" Score="3" Body="&lt;p&gt;Humans are diploid. This means they have two copies of each locus (with the exception of sex chromosomes in males). Your two alleles for each individual represent the two copies of this variant locus.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2018-09-23T06:46:06.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5109" PostTypeId="1" CreationDate="2018-09-24T13:47:01.813" Score="2" ViewCount="531" Body="&lt;p&gt;In &lt;a href=&quot;https://www.nature.com/articles/s41586-018-0414-6#Fig7&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;RNA velocity of single cells&quot;&lt;/a&gt;, La Manno et al. look at ratios between spliced and unspliced mRNA as a way of estimating the velocity of a cell through transcriptome space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In checking for a universal steady-state spliced-unspliced ratio, they write &quot;We found that the steady-state behaviour of most genes across a wide range of cell types was consistent with a single fixed [ratio between spliced and unspliced] (Extended Data Fig. 3a–c). However, 11% of genes showed distinct slopes in different subsets of tissues (Extended Data Fig. 3d, e), suggesting tissue-specific alternative splicing (Extended Data Fig. 3f) or degradation rates.&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tissue-specific alternative splicing or degradation rates certainly could account for these patterns, but how do La Manno et al. rule out that some tissues are simply not in a steady state? The figure they cite (screenshotted below) uses data from the Tabula Muris adult mouse cell encyclopedia. In an adult mouse, not all cells would not be expected to be in steady state -- for example, T-cell progenitors from the thymus and epithelial cells in many organs undergo continual replenishment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HwLew.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HwLew.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/JCsxu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JCsxu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1708" LastActivityDate="2018-09-24T14:11:45.460" Title="RNA velocity: competing explanations for variable ratio of spliced to unspliced transcript" Tags="|rna-seq|scrnaseq|rna-splicing|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5110" PostTypeId="2" ParentId="5109" CreationDate="2018-09-24T13:59:19.727" Score="1" Body="&lt;p&gt;If cells aren't in a steady state then one wouldn't expect to see the results in (e), where there are two slopes. One would instead expect to see results more like Plekha3 in panel (c), where there's more variability around the the steady state line (or perhaps no steady state line at all).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-09-24T13:59:19.727" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5111" PostTypeId="2" ParentId="5109" CreationDate="2018-09-24T14:11:45.460" Score="1" Body="&lt;p&gt;In addition to Devon Ryan's data-driven answer, I've found a clue in the text. La Manno et al have not used the full Tabula Muris: the figure legend indicates that they used only terminally differentiated cells. &lt;/p&gt;&#xA;" OwnerUserId="1708" LastActivityDate="2018-09-24T14:11:45.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5112" PostTypeId="2" ParentId="756" CreationDate="2018-09-24T15:51:34.850" Score="3" Body="&lt;p&gt;You could take a look at the &lt;code&gt;rentrez&lt;/code&gt; package for R. You can use the &lt;code&gt;entrez_search&lt;/code&gt; function to get the GEO IDs for the studies related to your query, and then the &lt;code&gt;entrez_summary&lt;/code&gt; function for the results. So, for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x &amp;lt;- rentrez::entrez_search(db=&quot;gds&quot;, term=paste0(&quot;tolvaptan AND gse[ETYP]&quot;),&#xA;                            use_history = T)&#xA;res &amp;lt;- rentrez::entrez_summary(db=&quot;gds&quot;, id=x$ids)&#xA;str(res)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where &lt;code&gt;res&lt;/code&gt; will be a list containing objects of the &lt;code&gt;esummary&lt;/code&gt; class. Then, you can use the following line to get the GSE accession numbers related to tolvaptan:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;lapply(res, `[[`, &quot;gse&quot;)    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So you could put this function in a loop replacing &quot;tolvaptan&quot; with your drugs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;drugs &amp;lt;- c(&quot;drugA&quot;, &quot;drugB&quot;, &quot;drugC&quot;)&#xA;for(drug in drugs){&#xA;    x &amp;lt;- rentrez::entrez_search(db=&quot;gds&quot;, term=paste0(drug, &quot; AND gse[ETYP]&quot;))&#xA;    res &amp;lt;- rentrez::entrez_summary(db=&quot;gds&quot;, id=x$ids)&#xA;    lapply(res, `[[`, &quot;gse&quot;)     &#xA;} &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Take a look at the &lt;a href=&quot;https://cran.r-project.org/web/packages/rentrez/vignettes/rentrez_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;rentrez&lt;/code&gt; documentation&lt;/a&gt; for more details.&lt;/p&gt;&#xA;" OwnerUserId="1815" LastEditorUserId="964" LastEditDate="2018-09-24T17:22:05.357" LastActivityDate="2018-09-24T17:22:05.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5113" PostTypeId="1" AcceptedAnswerId="5119" CreationDate="2018-09-24T16:02:32.947" Score="6" ViewCount="141" Body="&lt;p&gt;I'm looking for official GATK documentation (or a recent manuscript) that defines a general recommendation/requirement for sequencing depth to reliably call a heterozygous point mutation in a diploid organism (WGS). In this case, I'm working with humans. Specifically, suppose we're looking at position X in the genome and we want to classify (True/False) whether there was enough coverage to detect a heterozygous point mutation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What coverage would that be? I'm curious about both: (1) UnifiedGenotyper and (2) HaplotypeCaller&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know general practice is to have 30x coverage genomewide, but I don't see anything in GATK's documentation (i.e., &lt;a href=&quot;https://software.broadinstitute.org/gatk/best-practices/&quot; rel=&quot;noreferrer&quot;&gt;Best Practices&lt;/a&gt;) about required coverage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Really appreciate your help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I also submitted a &lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/13003/recommended-required-sequencing-depth-for-standard-diploid-whole-genome-germline-variant-calling/p1?new=1&quot; rel=&quot;noreferrer&quot;&gt;question to the GATK team&lt;/a&gt; end of last week.&lt;/p&gt;&#xA;" OwnerUserId="136" LastActivityDate="2018-10-22T18:13:42.473" Title="GATK documentation for required depth to reliably call heterozygous mutation in diploid organism?" Tags="|gatk|wgs|coverage|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5115" PostTypeId="1" AcceptedAnswerId="5121" CreationDate="2018-09-24T18:35:53.210" Score="4" ViewCount="9004" Body="&lt;p&gt;I know that in &lt;code&gt;Seurat&lt;/code&gt; we have the function &lt;code&gt;CreateSeuratObject&lt;/code&gt; from which the analysis starts, but it accepts raw count matrix according to the documentation. I have only the already normalized count matrix, so is there a way to work with &lt;code&gt;Seurat&lt;/code&gt; using normalized data? I would try maybe to feed in the normalized data instead of raw to &lt;code&gt;CreateSeuratObject&lt;/code&gt;, but I do not know what &lt;code&gt;Seurat&lt;/code&gt; is doing behind the scenes, whether it is a good idea. Besides, &lt;code&gt;Seurat&lt;/code&gt; provides by default only one &lt;code&gt;log&lt;/code&gt; - normalization method, but I may want to normalize the data by myself with various methods and only then start the analysis with &lt;code&gt;Seurat&lt;/code&gt; - that is the other reason why I want to find a way to start from normalized data.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-09-26T01:11:30.560" Title="Seurat with normalized count matrix?" Tags="|r|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5116" PostTypeId="2" ParentId="5092" CreationDate="2018-09-24T23:31:52.363" Score="2" Body="&lt;p&gt;Protein autocorrelation is similar to usual autocorrelation of time series (that is, function &lt;span class=&quot;math-container&quot;&gt;$r(k)$&lt;/span&gt; measuring correlation between values lagged by &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;), except that instead of time dimension we slide along the backbone of amino acid chain. The &quot;dependent variable&quot; can be any quantifiable property of the amino acids - then the autocorrelation measures similarity in that particular property.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See e.g. Fernandez et al., 2008, for more extensive background:&#xA;&lt;a href=&quot;https://www.tandfonline.com/doi/full/10.1080/08927020802301920?scroll=top&amp;amp;needAccess=true&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tandfonline.com/doi/full/10.1080/08927020802301920?scroll=top&amp;amp;needAccess=true&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="920" LastActivityDate="2018-09-24T23:31:52.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5117" PostTypeId="1" AcceptedAnswerId="5131" CreationDate="2018-09-25T14:18:48.013" Score="5" ViewCount="141" Body="&lt;p&gt;How do population genetics people define a population? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do they define it as a layman will do? say Africans, Americans, or so? Or is there a more scientific way of doing so? For example, I think defining one's population as one's allele frequencies makes more sense, but has this point been discussed previously?&lt;/p&gt;&#xA;" OwnerUserId="1082" LastActivityDate="2018-09-27T08:31:05.557" Title="How do population genetics people define a population?" Tags="|phylogenetics|genetics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5118" PostTypeId="2" ParentId="5117" CreationDate="2018-09-25T14:39:56.177" Score="3" Body="&lt;p&gt;A population is defined as a group of people (or organisms of the same species) living together in a geographical region. However, for human genetic studies, it is wise to use ethnicity as well (so group of people with the same ethnic background). See &lt;a href=&quot;http://www.internationalgenome.org/category/population/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; how they define population in the 1000 genomes project.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2018-09-25T14:39:56.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5119" PostTypeId="2" ParentId="5113" CreationDate="2018-09-25T15:28:54.353" Score="2" Body="&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt; As an update, Sarah Walker (co-author on the poster) responded to &lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/comment/52425#Comment_52425&quot; rel=&quot;nofollow noreferrer&quot;&gt;my question&lt;/a&gt; on the &lt;code&gt;GATK&lt;/code&gt; forum. She clarified with the following statement: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We believe the sites around 30X (and above 150X) are being filtered&#xA;  due to low mapping quality (since it is whole genome so there are many&#xA;  areas that are hard to map), which explains the low sensitivities in&#xA;  these areas. As the error bars show, there are very few variants in&#xA;  these low mapping quality areas.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;She then posted the following plot that falls much more inline with what we would all expect. This is a plot showing sensitivity for &lt;code&gt;NA12878&lt;/code&gt; compared to NIST truth.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/3AWL0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3AWL0.png&quot; alt=&quot;NA12878: sensitivity by depth&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;shlee, from the GATK forums, &lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/comment/52425#Comment_52425&quot; rel=&quot;nofollow noreferrer&quot;&gt;pointed me&lt;/a&gt; to &lt;a href=&quot;https://drive.google.com/drive/folders/1XDXlck-El8uZ7e60ANDSHycEpkgo6NoP&quot; rel=&quot;nofollow noreferrer&quot;&gt;a poster from AGBT 2018&lt;/a&gt; where the authors compared the HiSeqX and NovaSeq instruments. Figure 3 shows sensitivity for SNPs and short INDELs across multiple allelic fractions (AF) from spike-ins. &lt;strong&gt;&lt;em&gt;I was surprised to find that SNPs where AF = 0.4 only had ~50% sensitivity at ~25x coverage, and only ~75% sensitivity at ~50x coverage&lt;/em&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The poster is targeted at somatic variation, but data from figure 3 should behave like a standard diploid germline variant because you often see a wide range of allelic fractions from germline variants. They used spike-ins for figure 3. It's not clear to me whether they used HaplotypeCaller in figure 3, but I don't see how they could have used Mutect2, like they did in figures 1 &amp;amp; 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on this information, &lt;strong&gt;it appears the standard 30x genome-wide coverage is generally insufficient to confidently call many heterozygous mutations, and that ~90-100x is ideal.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Am I missing something? Are we really missing that many heterozygous mutations in standard WGS/WES?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll leave this question unanswered for now to allow others to chime in on interpretation, etc. I'd appreciate additional thoughts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/P8ZMd.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/P8ZMd.jpg&quot; alt=&quot;poster from AGBT 2018&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="136" LastEditorUserId="136" LastEditDate="2018-10-22T18:13:42.473" LastActivityDate="2018-10-22T18:13:42.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5121" PostTypeId="2" ParentId="5115" CreationDate="2018-09-26T01:11:30.560" Score="6" Body="&lt;p&gt;This was addressed by the Seurat developers &lt;a href=&quot;https://github.com/satijalab/seurat/issues/668&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;if you have TPM counts, I suggest you don't use&#xA;  Seurat::NormalizeData(), since TPM counts are already normalized for&#xA;  sequencing depth and transcript/gene length. Note that&#xA;  Seurat::NormalizeData() normalizes the data for sequencing depth, and&#xA;  then transforms it to log space. If you have TPM data, you can simply&#xA;  manually log transform the gene expression matrix in the object@data&#xA;  slot before scaling the data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And &lt;a href=&quot;https://github.com/satijalab/seurat/issues/481&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If you want to run your own normalization function, please place the&#xA;  data in log scale before placing it into object@data&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And &lt;a href=&quot;https://github.com/satijalab/seurat/issues/171&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If you're supplying already normalized values to CreateSeuratObject,&#xA;  you should skip the normalization step. You will likely see a warning&#xA;  when you run ScaleData (which is checking whether you normalized using&#xA;  Seurat). You can ignore this or turn it off by setting check.for.norm&#xA;  to FALSE.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is a lot of quotes, but I found them all to be helpful in different ways.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2018-09-26T01:11:30.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5122" PostTypeId="2" ParentId="5076" CreationDate="2018-09-26T01:17:45.883" Score="1" Body="&lt;p&gt;There are &lt;a href=&quot;https://hicexplorer.readthedocs.io/en/latest/content/example_usage.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;some interesting suggestions from the HiCExplorer developers&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Mates have to be mapped individually to avoid mapper specific&#xA;  heuristics designed for standard paired-end libraries.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# map the reads, each mate individually using&#xA;# for example bwa&#xA;#&#xA;# bwa mem mapping options:&#xA;#       -A INT        score for a sequence match, which scales options -TdBOELU unless overridden [1]&#xA;#       -B INT        penalty for a mismatch [4]&#xA;#       -O INT[,INT]  gap open penalties for deletions and insertions [6,6]&#xA;#       -E INT[,INT]  gap extension penalty; a gap of size k cost '{-O} + {-E}*k' [1,1] # this is set very high to avoid gaps&#xA;#                                  at restriction sites. Setting the gap extension penalty high, produces better results as&#xA;#                                  the sequences left and right of a restriction site are mapped independently.&#xA;#       -L INT[,INT]  penalty for 5'- and 3'-end clipping [5,5] # this is set to no penalty.&#xA;&#xA;$ bwa mem -A1 -B4  -E50 -L0  index_path \&#xA;    -U mate_R1.fastq.gz 2&amp;gt;&amp;gt;mate_R1.log | samtools view -Shb - &amp;gt; mate_R1.bam&#xA;&#xA;$ bwa mem -A1 -B4  -E50 -L0  index_path \&#xA;    -U mate_R2.fastq.gz 2&amp;gt;&amp;gt;mate_R2.log | samtools view -Shb - &amp;gt; mate_R2.bam&#xA;&#xA;# build matrix from independently mated read pairs&#xA;# the restriction sequence GATC is recognized by the DpnII restriction enzyme&#xA;&#xA;$ hicBuildMatrix --samFiles mate_R1.bam mate_R2.bam \&#xA;                 --binSize 10000 \&#xA;                 --restrictionSequence GATC \&#xA;                 --threads 4&#xA;                 --inputBufferSize 100000&#xA;                 --outBam hic.bam \&#xA;                 -o hic_matrix.h5&#xA;                 --QCfolder ./hicQC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="35" LastActivityDate="2018-09-26T01:17:45.883" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5123" PostTypeId="1" AcceptedAnswerId="5130" CreationDate="2018-09-26T04:43:43.513" Score="1" ViewCount="265" Body="&lt;p&gt;I am using &lt;a href=&quot;http://mccarrolllab.org/dropseq/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Drop-seq_tools&lt;/a&gt; to analyze scRNA-seq in a similar way of &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/29227469&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;. I am using the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE97930&quot; rel=&quot;nofollow noreferrer&quot;&gt;same data (GSE97930)&lt;/a&gt; to get used to Drop-seq_tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the GEO repo the authors provide the relevant code (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE97930&amp;amp;format=file&amp;amp;file=GSE97930%5FREADME%2Etxt&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSE97930_README.txt&lt;/a&gt;). However, one tool used in the script is missing: &lt;strong&gt;TagReadWithGene&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The latest version of Drop-seq_tools does not have such method (as it should be provided by this library), and there is no version history for the whole package, so I cannot download previous versions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The best matches for the missing method are &lt;strong&gt;TagReadWithGeneExon&lt;/strong&gt; and &lt;strong&gt;TagReadWithInterval&lt;/strong&gt;.&#xA;In the methods of the cited paper they say:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Reads that mapped to intronic or exonic regions of genes as per the GENCODE gene annotation were recorded. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think that in this case the tool to use should be &lt;strong&gt;TagReadWithInterval&lt;/strong&gt; since the other one should consider only exons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot find any other documentation about the tools. Do you know/already encountered this problem?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-09-27T01:30:12.030" Title="TagReadWithGene missing when using latest version of Drop-seq_tools" Tags="|scrnaseq|gene|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5124" PostTypeId="1" CreationDate="2018-09-26T15:47:39.777" Score="3" ViewCount="194" Body="&lt;p&gt;A preprint describing a new tool and its application to microbiome analysis was recently &lt;a href=&quot;https://www.biorxiv.org/content/early/2018/09/04/408070&quot; rel=&quot;nofollow noreferrer&quot;&gt;published in bioRxiv&lt;/a&gt;&lt;sup&gt;[1]&lt;/sup&gt;. At the core of this new tool, HULK, is a new data structure called a histosketch&lt;sup&gt;[2]&lt;/sup&gt; which is similar in spirit to CountMin sketches and MinHash sketches that have become quite popular in bioinformatics in the last several years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the main advantages the histosketch authors claim over CountMin sketch is that CountMin sketches do not preserve similarity. From section II of their paper&lt;sup&gt;[2]&lt;/sup&gt; (emphasis mine):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It is practically difficult to compute histograms for data streams with typically unknown cardinality and which thus require and unbound amount of memory to maintain the histogram. In this context, the count sketch and the count-min sketch and other online histogram building methods were proposed to approximate the frequency table of elements (i.e., histograms) from a data stream with a fixed-size data structure. However, &lt;strong&gt;the resulting sketches do not preserve similarity between different data streams&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In what sense does the CountMin sketch not preserve similarity between streams? Does this refer to similarity at intermediate states (while data is still streaming) or does it refer to inherent characteristics of the data structure itself?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;[1]&lt;/sup&gt; Rowe WPM, Carrieri AP, Alcon-Giner C, Caim S, Shaw A, Sim K, Kroll JS, Hall L, Pyzer-Knapp EO, Winn MD (2018) Streaming histogram sketching for rapid microbiome analytics. &lt;em&gt;bioRxiv&lt;/em&gt;, 408070, &lt;a href=&quot;https://doi.org/10.1101/408070&quot; rel=&quot;nofollow noreferrer&quot;&gt;doi:10.1101/408070&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;[2]&lt;/sup&gt; Yang D, Li B, Rettig L, Cudré-Mauroux P (2017) HistoSketch: Fast similarity-preserving sketching of streaming histograms with concept drift. &lt;em&gt;Proc. IEEE Int. Conf. Data Mining&lt;/em&gt;, &lt;a href=&quot;https://doi.org/10.1109/ICDM.2017.64&quot; rel=&quot;nofollow noreferrer&quot;&gt;doi:10.1109/ICDM.2017.64&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2021-12-01T08:06:45.640" Title="Histosketch vs Count-Min sketch: preserving similarity" Tags="|k-mer|algorithms|sketch|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5125" PostTypeId="5" CreationDate="2018-09-26T16:02:19.367" Score="0" Body="&lt;p&gt;Sketching data structures provide a compact representation of a data set, and in the context of bioinformatics are typically composed of &lt;em&gt;k&lt;/em&gt;-mer hashes. Just as a &quot;sketch&quot; of a painting, a sketch of a data set bears some sort of &quot;resemblance&quot; to the original data set but requires much less space for storage in memory or disk. Sketches can be used to estimate and compare the size, content, and element frequency of a data set.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2018-09-27T07:41:14.387" LastActivityDate="2018-09-27T07:41:14.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5126" PostTypeId="4" CreationDate="2018-09-26T16:02:19.367" Score="0" Body="questions related to Bloom filters, MinHash sketches, HyperLogLog, Count-Min sketches, and other sketching data structures and related algorithms" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2018-09-27T07:40:55.830" LastActivityDate="2018-09-27T07:40:55.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5128" PostTypeId="1" CreationDate="2018-09-26T18:41:49.467" Score="0" ViewCount="34" Body="&lt;p&gt;I have a network which I clustered into 11 smaller clusters. I have a set of genes and I would like to check if statistically significant number of these genes fall on any specific cluster. One method which I thought of was to find which percentage of a total cluster were made up of my genes of interest. For example, say 60% of the genes in a cluster were the genes in my list. However, I was hoping that there was a better way to do this. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4755009/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This study on honey bees&lt;/a&gt; has done something of the kind.&lt;/p&gt;&#xA;&lt;p&gt;I would appreciate if there was a way to do something similar. I do not know how to replicate what they have done in the study. Any help is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="976" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-09-27T18:14:34.117" Title="Check if a set of genes fall more on a specific cluster than other clusters" Tags="|networks|clustering|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5129" PostTypeId="1" CreationDate="2018-09-26T22:12:46.987" Score="5" ViewCount="94" Body="&lt;p&gt;In the methods of &lt;a href=&quot;http://science.sciencemag.org/content/360/6385/176&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, the authors say:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To simplify analysis, we first removed any dephased reads in our library (last 6 bases of read did not match the expected sequence).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have read &lt;a href=&quot;http://seqanswers.com/forums/showthread.php?t=11553&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt; and &lt;a href=&quot;https://www.france-bioinformatique.fr/sites/default/files/EBA/V3-2014/Claude_Thermes_Intro_NGS_EBA-2014-10_v2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;, and I think that a dephased read is one (according to the first paper) where the last 6 bases are not detected with good quality. In particular, I think this may be an example of a dephased read:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@...&#xA;CACACACACCAGTAAAAAANAGTTGCNNCNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN&#xA;+&#xA;AAAAAEEAEEEEEEEEEE6#EAEEEE##E#####################################&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the quality of the read drops in the middle. Is that correct? If yes, should they be all of bad quality or at least one? And, if not, how can I detect a dephased read?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2018-09-27T19:12:54.363" LastActivityDate="2018-09-27T20:33:27.177" Title="What is and how to detect a dephased read" Tags="|scrnaseq|reads|splitseq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5130" PostTypeId="2" ParentId="5123" CreationDate="2018-09-27T01:30:12.030" Score="1" Body="&lt;p&gt;As per &lt;a href=&quot;https://github.com/broadinstitute/Drop-seq/issues/38#issuecomment-424833407&quot; rel=&quot;nofollow noreferrer&quot;&gt;this response&lt;/a&gt; from Drop-seq_tools team, at the moment intronic regions cannot be annotated and &lt;strong&gt;TagReadWithGeneExon&lt;/strong&gt; should be used instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in the version of the &lt;a href=&quot;https://github.com/broadinstitute/Drop-seq&quot; rel=&quot;nofollow noreferrer&quot;&gt;public repository&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;TagReadWithGeneExonFunction is the old code, for people who want the more simple tagging methodology and rely on it for 3rd party tools, and TagReadWithGeneFunction, which for your purposes replaces TagReadWithGeneExon. [...]  You’ll see additional options to extract expression for CODING, UTR, INTRONIC reads [...]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-09-27T01:30:12.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5131" PostTypeId="2" ParentId="5117" CreationDate="2018-09-27T08:31:05.557" Score="2" Body="&lt;p&gt;It really depends on the study and the ethnicity. Mostly they use self-defined ethnicity and often they will require that three or four grandparents are of that ethnicity. &lt;a href=&quot;https://www.coriell.org/1/NHGRI/Collections/1000-Genomes-Collections/1000-Genomes-Project&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's some detailed info on how the ethnicity was defined for the 1000 Genomes populations.&lt;/a&gt; You'll see that some populations (such as ASW) require all four grandparents to identify as that ethnicity, but some (such as ACB) only need three, and the cultural sensitivities in Japan meant that JPT participants weren't questioned directly but told that the study was only interested in people with Japanese grandparents. The mixed American super-population (AMR) is an exception to this rule due to the extreme admixture of ethnicity in these regions, and is based more in geography than ethnicity – this means that the individuals had to be from the place and have grandparents from the place.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2018-09-27T08:31:05.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5132" PostTypeId="1" AcceptedAnswerId="5270" CreationDate="2018-09-27T14:50:09.687" Score="7" ViewCount="86" Body="&lt;p&gt;The NCBI Genomes database has these dendrograms for (presumably) whole genome comparisons for certain species, e.g. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genome/?term=txid208964[Organism:exp]&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pseudonomas aeruginosa&lt;/a&gt;  or &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genome/167&quot; rel=&quot;nofollow noreferrer&quot;&gt;Escherichia coli&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How were these comparisons done? Someone knows the source / paper?&lt;/p&gt;&#xA;" OwnerUserId="1802" LastEditorUserId="1802" LastEditDate="2018-09-28T17:59:43.160" LastActivityDate="2018-10-24T14:28:40.923" Title="Source for whole genome comparisons at NCBI Genomes" Tags="|phylogenetics|genbank|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5133" PostTypeId="1" AcceptedAnswerId="5136" CreationDate="2018-09-27T15:56:46.940" Score="2" ViewCount="215" Body="&lt;p&gt;In the methods of &lt;a href=&quot;http://science.sciencemag.org/content/360/6385/176&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, the authors say:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Reads were then filtered based on quality score in the UMI region. Any read with &gt;1 low-quality base (phred &amp;lt;=10) were discarded. &lt;strong&gt;Reads with more than one mismatch in any of the three 8 nt cell barcodes were also discarded&lt;/strong&gt;. The cDNA reads (Read 1) were then mapped to either a combined mm10- hg19 genome or the mm10 genome using STAR (44).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I understand how to filter based on mismatch &lt;em&gt;after&lt;/em&gt; mapping but in this case it is done &lt;em&gt;before&lt;/em&gt; mapping. Has this to be interpreted as &quot;Reads with &lt;em&gt;low quality in more than one base&lt;/em&gt; in any of the three 8 nt cell barcodes were also discarded&quot;?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-09-27T20:28:13.003" Title="How to detect mismatch before mapping in RNA-Seq data" Tags="|scrnaseq|read-mapping|reads|barcode|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5134" PostTypeId="2" ParentId="5133" CreationDate="2018-09-27T16:55:31.647" Score="1" Body="&lt;p&gt;It should not be interpreted as &quot;Reads with low quality in more than one base in any of the three 8 nt cell barcodes were also discarded&quot;. They only care if the cell barcodes match what are valid cell barcodes within 1 mismatch, the individual qualities are ignored. This sort of filtering turns out to be pretty common in practice, since you have to write a maximum edit distance into whatever you use to demultiplex by cells (or labeling each read, as was likely done in this case).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-09-27T16:55:31.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5135" PostTypeId="2" ParentId="5128" CreationDate="2018-09-27T18:14:34.117" Score="1" Body="&lt;p&gt;A hypergeometric test is related to the fisher test, in a table with one  dimension being some factors and the other one some other factors it is tested if they are independent. In R you con do this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data &amp;lt;- matrix(c(0, 1, 2, 5), ncol = 2, nro = 2)&#xA;fisher.test(data)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or you can use the specific function &lt;a href=&quot;https://stackoverflow.com/q/8382806/2886003&quot;&gt;&lt;code&gt;phyper&lt;/code&gt;&lt;/a&gt;, to see the relation between them you can read this &lt;a href=&quot;https://stats.stackexchange.com/q/82688/105234&quot;&gt;question and answer&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-09-27T18:14:34.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5136" PostTypeId="2" ParentId="5133" CreationDate="2018-09-27T18:16:19.283" Score="2" Body="&lt;p&gt;At each step in the SLiTing and pooling the barcodes added are not random, but are taken  from a predefined list of possible barcodes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the reads arrive, first you filter by reads that have &gt;1 low quality bases in the UMI region (which is truely random).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you compare the sequences in the 3 cell-barcode regions against the list of allowed barcodes for each of those positions. If a read is not within an edit distance of one from one of those barcodes, it is discarded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally the reads that pass these filters can be aligned.&lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-09-27T18:16:19.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5137" PostTypeId="1" AcceptedAnswerId="5149" CreationDate="2018-09-27T19:14:33.250" Score="2" ViewCount="342" Body="&lt;p&gt;Seurat R package has some functions like &lt;strong&gt;FeaturePlot&lt;/strong&gt;, &lt;strong&gt;DimPlot&lt;/strong&gt; and &lt;strong&gt;DoHeatmap&lt;/strong&gt; by which we can plot the expression of a list of genes on cell clusters. I am working with &lt;strong&gt;URD&lt;/strong&gt; that likely does not have such options or I can not find that. URD has plotDot but would be clear only for a few number of genes (please look at the picture). Could somebody please help me in plotting the expression pattern of a list of genes in URD? As I am not good in R at all to write my own function or adopt Seurat functions to URD&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/R7MRf.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/R7MRf.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example Seurat DoHeatmap gives such a picture by which we can see these set of genes are being expressed on which group of cells&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DoHeatmap(object ,genes.use=a vector of genes)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/j0ACP.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/j0ACP.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or we can plot a set of genes on seurat feature lot to they are being expressed more on which cluster&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FeaturePlot(object = pbmc_small, features.plot = 'PC1')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jX0Ij.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jX0Ij.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In URD only by &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plotDot(object.6s.mnn, genes = a vector of genes, clustering=&quot;Infomap-60&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We can plot a set of genes but as you are being in the first picture that would not be clear much&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how I can plot a vector of genes on URD object&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for example Seurat DimPlot function can visualise the expression pattern of a group of genes on cell clusters but when I am trying to do the same with URD, returns this error&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; plotDim(axial.6somite, c(&quot;DDB_G0291149&quot;,&quot;DDB_G0291029&quot;), plot.title=&quot;SHHA (Notochord marker)&quot;)&#xA;Error in names(data) &amp;lt;- cells.use : &#xA;  invalid to use names()&amp;lt;- on an S4 object of class 'dgCMatrix'&#xA;In addition: Warning messages:&#xA;1: In if (label.type == &quot;meta&quot; | (label.type == &quot;search&quot; &amp;amp; label %in%  :&#xA;  the condition has length &amp;gt; 1 and only the first element will be used&#xA;2: In if (label.type == &quot;group&quot; | (label.type == &quot;search&quot; &amp;amp; label %in%  :&#xA;  the condition has length &amp;gt; 1 and only the first element will be used&#xA;3: In if (label.type == &quot;sig&quot; | (label.type == &quot;search&quot; &amp;amp; label %in%  :&#xA;  the condition has length &amp;gt; 1 and only the first element will be used&#xA;4: In if (label.type == &quot;gene&quot; | (label.type == &quot;search&quot; &amp;amp; label %in%  :&#xA;  the condition has length &amp;gt; 1 and only the first element will be used&#xA;&amp;gt; &#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Actually only one gene can be visualised by this function on URD&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="48" LastEditDate="2018-10-01T07:32:11.417" LastActivityDate="2018-10-01T07:32:11.417" Title="The visualisation of a list of genes on URD object" Tags="|r|scrnaseq|seurat|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5138" PostTypeId="2" ParentId="5133" CreationDate="2018-09-27T20:13:39.853" Score="0" Body="&lt;p&gt;Cell barcodes =/= sequence reads.  The default settings on Illumina's bcl2fastq work exactly the same way; they allow a read to count for a sample if there is a single mismatch in the index, but not more.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastEditorUserId="2388" LastEditDate="2018-09-27T20:28:13.003" LastActivityDate="2018-09-27T20:28:13.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5139" PostTypeId="2" ParentId="5129" CreationDate="2018-09-27T20:33:27.177" Score="4" Body="&lt;p&gt;Phasing in Illumina data means that the read you get is shifted one base before or after what the sequence really should be.  They must be expecting some constant 6 bases at the end of each read, and they are throwing out all reads that don't have those 6 bases.  This would include N's, but a small deletion or insertion would also cause the last 6 bases to not match.  This is in theory detectable and fixable; the dropSeq protocol for instance catches instances where improperly short cell barcodes shift the UMIs, but these authors look like they decided not to try and bother, and just throw out everything that didn't match their exact expectations.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2018-09-27T20:33:27.177" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5141" PostTypeId="1" AcceptedAnswerId="5142" CreationDate="2018-09-28T09:54:16.503" Score="6" ViewCount="1216" Body="&lt;p&gt;Very recently a &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK279690/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BLAST&lt;/a&gt; parameter &lt;code&gt;-max_target_seqs n&lt;/code&gt; got a &lt;a href=&quot;https://academic.oup.com/bioinformatics/advance-article/doi/10.1093/bioinformatics/bty833/5106166&quot; rel=&quot;nofollow noreferrer&quot;&gt;lot of attention&lt;/a&gt;. Instead of the intuitive interpretation (return the best &lt;code&gt;n&lt;/code&gt; sequences) the parameters asks blast to return the first &lt;code&gt;n&lt;/code&gt; sequences that pass the e-value threshold. This is affecting thousands of workflows and analyses that assumed the intuitive interpretation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also found that there is another parameter called &lt;code&gt;-num_alignments&lt;/code&gt;. What is the difference? Related question on &lt;a href=&quot;https://www.biostars.org/p/120287/#340449&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioStars&lt;/a&gt; seems that is misinterpreting the &lt;code&gt;max_target_seqs&lt;/code&gt; parameter. Is &lt;code&gt;num_alignments&lt;/code&gt; also misunderstood?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the main question. How to actually run blast to give me the best &lt;code&gt;n&lt;/code&gt; hits in the database?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Till now I thought that this example will do the job:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blastp -query &lt;span class=&quot;math-container&quot;&gt;$QUERY -db $&lt;/span&gt;PROTEINS -out $BLASTOUT -evalue 1e-10 -outfmt 6 -num_alignments 5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;-- edit --&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I understood that i can not achieve the desired output only by parameters of blast, but I still don't see an answer to how can I get the best n hits. So I wanted to check out how to get all the results and that's not intuitive either. The parameter &lt;code&gt;-max_target_seqs&lt;/code&gt; is set by default to 500. Does it mean that if there are more than 500 significant hits, I don't have a guarantee for the best one? Do I have to specify &lt;code&gt;max_target_seqs&lt;/code&gt; to some crazy high number to be sure that I got them all?&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2018-09-28T12:03:37.247" LastActivityDate="2018-10-03T16:24:07.547" Title="How to achieve blast results according to the intuitive interpretation of `-max_target_seqs`?" Tags="|blast|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5142" PostTypeId="2" ParentId="5141" CreationDate="2018-09-28T10:32:45.987" Score="3" Body="&lt;p&gt;The &lt;code&gt;-max_target_seqs&lt;/code&gt; is also related to &lt;code&gt;-evalue&lt;/code&gt; which &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/3409/blast-hits-disappearing-after-changing-evalue?rq=1&quot;&gt;does not work&lt;/a&gt; as one would think. Looking at the &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMDWeb&amp;amp;PAGE_TYPE=BlastNews&quot; rel=&quot;nofollow noreferrer&quot;&gt;blast news&lt;/a&gt; one finds that since version BLAST 2.2.27+ released, one should use the &lt;code&gt;-max_target_seqs&lt;/code&gt;: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;4.) The output for reports without separate descriptions and alignments sections (all –outfmt greater than 4) should now use –max_target_seqs to control the output rather than –num_descriptions and –num_alignments.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So it seems that &lt;code&gt;num_alignments&lt;/code&gt; is also &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2846/why-does-a-very-strong-blast-hit-get-lost-when-i-change-num-alignments-num-desc&quot;&gt;misunderstood&lt;/a&gt;. I came to the conclusion that one cannot expect to get the best &lt;code&gt;n&lt;/code&gt; alignments, unless you get all and filter yourself the best ones. &lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2018-09-28T10:45:31.567" LastActivityDate="2018-09-28T10:45:31.567" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5143" PostTypeId="1" CreationDate="2018-09-28T12:03:39.790" Score="4" ViewCount="106" Body="&lt;p&gt;We are trying to sort out a pipeline for doing allele specific expression. Our plan is to call SNPs from RNA-seq data and combine with known SNP annotations. A well known problem in ASE is reference bias, where reads are more like to map if they have the reference sequence than the alt sequence.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;We have come accross two possible ways to deal with this. In many papers (e.g. &lt;a href=&quot;https://www.nature.com/articles/s41598-017-12179-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zhuo et al&lt;/a&gt;, &lt;a href=&quot;https://genomemedicine.biomedcentral.com/articles/10.1186/s13073-016-0381-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sigurdsson et al&lt;/a&gt;) a new reference is generated, substitute N bases where there are SNPs using FastaAlternativeReferenceMaker from GATK and reads are remapped to that.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;HISAT2 offers the possibility to provide SNP calls to the index builder, and build an index that accounts for these SNPs.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any advice on which of these might be better? &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-10-17T13:40:16.687" Title="Reference genome for allele specific expression" Tags="|rna-seq|snp|read-mapping|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5144" PostTypeId="1" CreationDate="2018-09-28T14:17:00.273" Score="4" ViewCount="106" Body="&lt;p&gt;I am currently working on a dataset that contains 50 samples (10 samples * 5 blocks). The features of the date set are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The data is perfectly balanced between blocks, with equal treatment representation in each block. Each block contains 2 control samples (CTL) that are pools of all other treatment samples. These are technical replicates that are identical across multiplexes.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There is a lot of missing data (~40%),&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;most of the missing data is missing by block, meaning that the protein is present in some blocks, but not others.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When looking at the expression of these missing proteins in the other blocks, they tend to be around the median value for the experiment, which suggests to me that these values are missing completely at random.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I could proceed using complete cases, but then I'm missing out on a large portion of data. I am considering using an imputation method, but I'm concerned that (due to the nature of the missing data) that I will just be introducing a block effect into the data.&lt;/p&gt;&#xA;&lt;p&gt;Any input on potential methods for imputation would be greatly appreciated.&lt;/p&gt;&#xA;&lt;p&gt;Experimental design:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SampleNumber    Multiplex   Treatment&#xA;1   A   CTL&#xA;2   A   CTL&#xA;3   A   TMT1&#xA;4   A   TMT1&#xA;5   A   TMT2&#xA;6   A   TMT2&#xA;7   A   TMT3&#xA;8   A   TMT3&#xA;9   A   TMT4&#xA;10  A   TMT4&#xA;11  B   CTL&#xA;12  B   CTL&#xA;13  B   TMT1&#xA;14  B   TMT1&#xA;15  B   TMT2&#xA;16  B   TMT2&#xA;17  B   TMT3&#xA;18  B   TMT3&#xA;19  B   TMT4&#xA;20  B   TMT4&#xA;21  C   CTL&#xA;22  C   CTL&#xA;23  C   TMT1&#xA;24  C   TMT1&#xA;25  C   TMT2&#xA;26  C   TMT2&#xA;27  C   TMT3&#xA;28  C   TMT3&#xA;29  C   TMT4&#xA;30  C   TMT4&#xA;31  D   CTL&#xA;32  D   CTL&#xA;33  D   TMT1&#xA;34  D   TMT1&#xA;35  D   TMT2&#xA;36  D   TMT2&#xA;37  D   TMT3&#xA;38  D   TMT3&#xA;39  D   TMT4&#xA;40  D   TMT4&#xA;41  E   CTL&#xA;42  E   CTL&#xA;43  E   TMT1&#xA;44  E   TMT1&#xA;45  E   TMT2&#xA;46  E   TMT2&#xA;47  E   TMT3&#xA;48  E   TMT3&#xA;49  E   TMT4&#xA;50  E   TMT4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The data is in a matrix format, columns = samples and rows = individual proteins, similar to gene expression.&lt;/p&gt;&#xA;&lt;p&gt;The controls are technical replicates, but they are pools of all other 40 samples. This is for TMT proteomics, where the pools are used by the MS to identify peptides across the multiplexes, not necessarily for multiplex correction. I've included multiplex (as well as treatment) in the linear model to identify changes associated with the treatment, while removing the effect of the multiplex. I have confirmed (with PCA and PVCA) that the multiplex effect is negligible after including it in the lm. My problem at the moment is the missing data.&lt;/p&gt;&#xA;&lt;p&gt;I calculated the missing percentage from the data matrix by taking the number of missing values divided by the total number of potential values. he data is in a matrix format, columns = samples and rows = individual proteins, similar to gene expression. To calculate the missing values, I divided the total number of NA's by the total number of values in the matrix (number of samples * number of proteins). Missing data is common in proteomics, but normally this can be attributed to something, most often being close to the limit of detection. But that is not the case here, if a value is missing, it is missing in the whole multiplex, including the pool, where we know it should be found.&lt;/p&gt;&#xA;&lt;p&gt;The missing values in this case are random, and this is definitely due to the MS. It will select the first ~65k peptides that it identifies to trigger an additional MS event, so if it doesn't see that first peptide, it will be missing from the entire multiplex. There are methods for imputation of data that is missing at random, but I'm concerned that it will just be imputing by multiplex.&lt;/p&gt;&#xA;" OwnerUserId="3425" LastEditorUserId="73" LastEditDate="2021-05-14T09:04:01.213" LastActivityDate="2021-05-14T09:04:01.213" Title="Block wise protein imputation" Tags="|statistics|imputation|proteomics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5145" PostTypeId="1" AcceptedAnswerId="5148" CreationDate="2018-09-28T16:56:49.080" Score="4" ViewCount="1503" Body="&lt;p&gt;To calculate bitscore from score you can use this equation: &lt;span class=&quot;math-container&quot;&gt;$S' = (lambda*S - ln(K)) / ln(2)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I am trying to manually calculate the bitscore of an HSP of a pairwise blastn alignment, and I know the alignment score, how do I calculate the parameters K and lambda?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a more simple equation for pairwise nucleotide alignments?&lt;/p&gt;&#xA;" OwnerUserId="3429" LastEditorUserId="298" LastEditDate="2018-09-28T17:30:35.813" LastActivityDate="2018-09-29T20:06:36.537" Title="Calculating bit score: How do you find lambda and K?" Tags="|blast|bitscore|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5146" PostTypeId="2" ParentId="4976" CreationDate="2018-09-28T19:35:37.397" Score="3" Body="&lt;p&gt;Outline of solution:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;get this file: &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/genomes/refseq/bacteria/assembly_summary.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ncbi.nlm.nih.gov/genomes/refseq/bacteria/assembly_summary.txt&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;filter for lines having &quot;Complete Genome&quot; in column 12&lt;/li&gt;&#xA;&lt;li&gt;filter for lines having a taxid (column 6) that is a descendant of taxon id 1117 (phylum Cyanobacteria), you can use nodes.dmp from &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump.tar.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump.tar.gz&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;The file download URL is the concatenation of column 20 and the last field of column 20, after separating it at '/', followed by appending '_genomic.gbff.gz'.  For example: &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/007/365/GCF_000007365.1_ASM736v1/GCF_000007365.1_ASM736v1_genomic.gbff.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/007/365/GCF_000007365.1_ASM736v1/GCF_000007365.1_ASM736v1_genomic.gbff.gz&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;br&gt;&#xA;Since I will need something like that soon, I made a &lt;a href=&quot;https://github.com/pmenzel/download-refseq-genomes&quot; rel=&quot;nofollow noreferrer&quot;&gt;Perl script for downloading genomes&lt;/a&gt;.&lt;br&gt;&#xA;For Cyanobacteria, you would do &lt;code&gt;./download-refseq-genomes.pl 1117&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="1802" LastEditorUserId="1802" LastEditDate="2018-09-29T07:24:04.260" LastActivityDate="2018-09-29T07:24:04.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5148" PostTypeId="2" ParentId="5145" CreationDate="2018-09-29T20:06:36.537" Score="3" Body="&lt;p&gt;NCBI BLAST uses tables of pre-computed values for lambda/K for gapped protein and nucleotide comparison and their values depend on the amino acid substitution matrix or the substitution scores + gap penalties.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find these tables (from line 589 for blastn) in the file &lt;code&gt;c++/src/algo/blast/core/blast_stat.c&lt;/code&gt; from the &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/ncbi-blast-2.7.1+-src.tar.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;blast source package&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1802" LastActivityDate="2018-09-29T20:06:36.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5149" PostTypeId="2" ParentId="5137" CreationDate="2018-09-30T11:18:34.387" Score="1" Body="&lt;p&gt;I just added URD clustering labels for each cel to seurat object and then I did DoHeatmap by using group.by=&quot;URD clustering labels&quot;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2018-09-30T11:18:34.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5150" PostTypeId="1" CreationDate="2018-10-01T10:21:20.490" Score="1" ViewCount="110" Body="&lt;p&gt;I have taken out the data like chromosome number, start, and end coordinate then gene name but the circos need the data in a specific format as given in their tutorial &lt;a href=&quot;http://circos.ca/documentation/tutorials/quick_guide/histograms/&quot; rel=&quot;nofollow noreferrer&quot;&gt;circos&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So far I have my data I have collected is as such&#xA;Symbol  Chr Start   End&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;RBM11   hs21;hs21;hs21;hs21;hs21;hs21;hs21;hs21;hs21;hs21;hs21;hs21;hs21;hs21;hs21;hs21;hs21;hs21;hs21;hs21;hs21;hs21;hs21;hs21 14216130;14216145;14216153;14216154;14216178;14219553;14219553;14219563;14219563;14221097;14221097;14221097;14221097;14221097;14224375;14224438;14224438;14224438;14224438;14226859;14226880;14226880;14226880;14226880 14216282;14216282;14216282;14216282;14216282;14219725;14219725;14219725;14219725;14221453;14221169;14221169;14221169;14221169;14224537;14224537;14224537;14224537;14224537;14227054;14228372;14227384;14228173;14228372&#xA;ELAVL4  hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1;hs1 50048014;50103910;50105690;50106292;50108933;50108966;50109626;50109706;50109816;50141772;50144953;50144957;50144957;50144957;50144957;50144957;50144957;50144957;50144957;50144957;50177089;50177089;50177089;50177089;50177089;50177089;50177089;50177089;50177130;50179544;50193765;50193765;50193765;50193765;50193765;50193765;50193765;50193765;50195561;50195561;50195561;50195561;50195561;50195561;50195561;50195561;50197429;50197429;50197429;50197429;50197429;50197429;50197429;50197429;50200809;50200809;50200851;50200851;50200851;50200851;50200851    50048182;50104000;50105950;50106393;50109198;50109198;50109964;50109964;50109964;50141858;50145197;50145178;50145197;50145197;50145197;50145197;50145197;50145184;50145197;50145197;50177192;50177192;50177192;50177192;50177192;50177192;50177192;50177192;50177192;50181300;50193918;50193918;50193918;50193918;50193918;50193918;50193918;50193918;50195786;50195786;50195786;50195786;50195786;50195786;50195786;50195786;50197585;50197467;50197467;50197467;50197467;50197467;50197467;50197467;50203054;50201402;50202294;50203786;50201399;50203770;50201868&#xA;PTRF    hs17;hs17   42402452;42422627   42405388;42423517&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So i have to make the the data as such&lt;/p&gt;&#xA;&lt;h1&gt;chr start end value [options]&lt;/h1&gt;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;hs3 196000000 197999999 71.0000&#xA;hs3 198000000 199999999 57.0000&#xA;hs4 0 1999999 28.0000&#xA;hs4 2000000 3999999 40.0000&#xA;hs4 4000000 5999999 59.0000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As of now i have multiple&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;RBM11   hs21;hs21;hs21;hs21;hs21;hs21&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;in for a single gene so how do i make it one ?or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;RBM11   hs21&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any help or suggestion would be highly appreciated&lt;/p&gt;&#xA;&lt;p&gt;I tried:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; sed -r ':a; s/\b([[:alnum:]]+)\b(.*)\b\1\b/\1\2/g; ta; s/(, )+/, /g; s/, *$//'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Updated answer so far&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  library(splitstackshape)&#xA;df &amp;lt;- LEUKEMIA_CIRCOS_gene&#xA;Chr &amp;lt;-unique(cSplit(df, &amp;quot;Chr&amp;quot;, sep = ';', direction = &amp;quot;long&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;this gives me this output&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;RBM11&#xA;hs21&#xA;14216130;14216145;14216153;14216154;14216178;14219553;14219553;14219563;14219563;14221097;14221097;14221097;14221097;14221097;14224375;14224438;14224438;14224438;14224438;14226859;14226880;14226880;14226880;14226880&#xA;14216282;14216282;14216282;14216282;14216282;14219725;14219725;14219725;14219725;14221453;14221169;14221169;14221169;14221169;14224537;14224537;14224537;14224537;14224537;14227054;14228372;14227384;14228173;14228372&#xA;2&#xA;ELAVL4&#xA;hs1&#xA;50048014;50103910;50105690;50106292;50108933;50108966;50109626;50109706;50109816;50141772;50144953;50144957;50144957;50144957;50144957;50144957;50144957;50144957;50144957;50144957;50177089;50177089;50177089;50177089;50177089;50177089;50177089;50177089;50177130;50179544;50193765;50193765;50193765;50193765;50193765;50193765;50193765;50193765;50195561;50195561;50195561;50195561;50195561;50195561;50195561;50195561;50197429;50197429;50197429;50197429;50197429;50197429;50197429;50197429;50200809;50200809;50200851;50200851;50200851;50200851;50200851&#xA;50048182;50104000;50105950;50106393;50109198;50109198;50109964;50109964;50109964;50141858;50145197;50145178;50145197;50145197;50145197;50145197;50145197;50145184;50145197;50145197;50177192;50177192;50177192;50177192;50177192;50177192;50177192;50177192;50177192;50181300;50193918;50193918;50193918;50193918;50193918;50193918;50193918;50193918;50195786;50195786;50195786;50195786;50195786;50195786;50195786;50195786;50197585;50197467;50197467;50197467;50197467;50197467;50197467;50197467;50203054;50201402;50202294;50203786;50201399;50203770;50201868&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;my chromosome name are sorted out but the start and end remains because if I make concatenate the column I want to separate I get a really long list&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    Start &amp;lt;- unique(cSplit(df,c(&amp;quot;Chr&amp;quot;,&amp;quot;Start&amp;quot;,&amp;quot;End&amp;quot;),sep = ';', direction = &amp;quot;long&amp;quot;))&#xA;&#xA;RBM11&#xA;hs21&#xA;14216130&#xA;14216282&#xA;2&#xA;RBM11&#xA;hs21&#xA;14216145&#xA;14216282&#xA;3&#xA;RBM11&#xA;hs21&#xA;14216153&#xA;14216282&#xA;4&#xA;RBM11&#xA;hs21&#xA;14216154&#xA;14216282&#xA;5&#xA;RBM11&#xA;hs21&#xA;14216178&#xA;14216282&#xA;6&#xA;RBM11&#xA;hs21&#xA;14219553&#xA;14219725&#xA;7&#xA;RBM11&#xA;hs21&#xA;14219563&#xA;14219725&#xA;8&#xA;RBM11&#xA;hs21&#xA;14221097&#xA;14221453&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="285" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-10-07T21:37:46.580" Title="Circos plot issue" Tags="|r|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="5151" PostTypeId="1" AcceptedAnswerId="5152" CreationDate="2018-10-01T15:51:58.740" Score="1" ViewCount="610" Body="&lt;p&gt;I have FPKM data for three genes like below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     Samples      gene1 gene2 gene3&#xA;TCGA-2Y-A9GS-01A   0.9   7.4  1.0&#xA;TCGA-2Y-A9GT-01A   0.8   1.0  0.3&#xA;TCGA-2Y-A9GU-01A   0.6   2.0  0.2&#xA;TCGA-2Y-A9GV-01A   1.2   0.5  0.1&#xA;TCGA-2Y-A9GW-01A   3.8   2.1  0.4&#xA;TCGA-2Y-A9GX-01A   2.3   2.0  1.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I used &lt;code&gt;cor.test&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cor.test(~ gene1 + gene2, data = df2, method=&quot;spearman&quot;, continuity=FALSE, conf.level=0.95)&#xA;&#xA;    Spearman's rank correlation rho&#xA;&#xA;data:  gene1 and gene2&#xA;S = 5686100, p-value = 6.083e-09&#xA;alternative hypothesis: true rho is not equal to 0&#xA;sample estimates:&#xA;      rho &#xA;0.2984045 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have a warning message which I didn't see before.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Warning message:&#xA;In cor.test.default(x = c(0.9, 0.8, 0.6, 1.2, 3.8, 2.3, 3.8, 0.4,  :&#xA;  Cannot compute exact p-value with ties&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Do I need to care about this warning message? Is it good using FPKM data for correlation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For plotting I used &lt;code&gt;ggscatter&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggscatter(data, x = &quot;gene1&quot;, y = &quot;gene2&quot;, &#xA;          add = &quot;reg.line&quot;, conf.int = TRUE, &#xA;          cor.coef = TRUE, cor.method = &quot;spearman&quot;,&#xA;          xlab = &quot;gene1&quot;, ylab = &quot;gene2&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is this fine or I need to use any &lt;code&gt;log&lt;/code&gt; for the scatter plot?&lt;/p&gt;&#xA;" OwnerUserId="1377" LastActivityDate="2018-10-01T16:44:41.460" Title="Spearman correlation between two genes" Tags="|rna-seq|fpkm|correlation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5152" PostTypeId="2" ParentId="5151" CreationDate="2018-10-01T16:20:10.720" Score="4" Body="&lt;p&gt;Since Spearman is a rank-based test, it relies on you being able to accurately decide on the ranking of your observations by some metric (usually the magnitude of the numbers). If two observations have identical values (are &lt;em&gt;tied&lt;/em&gt;), then they cannot be definitively ranked. Since the ranks are not unique, exact p-values cannot be determined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g. in your data, for gene 2:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                  gene2 rank&#xA;TCGA-2Y-A9GS-01A    7.4    1&#xA;TCGA-2Y-A9GT-01A    1.0    5&#xA;TCGA-2Y-A9GU-01A    2.0   3= &#xA;TCGA-2Y-A9GV-01A    0.5    6&#xA;TCGA-2Y-A9GW-01A    2.1    2&#xA;TCGA-2Y-A9GX-01A    2.0   3=&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;By way of example, it's possible to do the test with either &lt;code&gt;TCGA-2Y-A9GU-01A&lt;/code&gt; or &lt;code&gt;TCGA-2Y-A9GX-01A&lt;/code&gt; ranked number 3 - meaning a definitive p-value cannot be determined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there are only a few ties, you probably don't need to worry too much. If your FPKMs (or better, TPMs) were calculated with greater precision, then ties would be less likely. Is there a reason you only have these numbers to 1 decimal place?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Log or linear scale doesn't make a huge difference (and neither is 'wrong'). If you choose to plot log counts, you need to consider what happens if you have zeroes in your data. &lt;/p&gt;&#xA;" OwnerUserId="194" LastEditorUserId="194" LastEditDate="2018-10-01T16:44:41.460" LastActivityDate="2018-10-01T16:44:41.460" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5153" PostTypeId="2" ParentId="4997" CreationDate="2018-10-02T02:06:36.147" Score="3" Body="&lt;p&gt;In response to question 1, save the following as a python3 script (e.g., &lt;code&gt;aln_parser.py&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3    &#xA;&#xA;from Bio import AlignIO&#xA;import sys&#xA;&#xA;aln = sys.argv[1]&#xA;&#xA;algnmnt = AlignIO.read(open(aln, 'r'), 'clustal')&#xA;&#xA;for col in range(0, algnmnt.get_alignment_length()):&#xA;    proteins = set(algnmnt[:,col])&#xA;    if len(proteins) == 1:&#xA;        print(f&quot;Postion {col}: protein {''.join(proteins)}&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Run the script to query your alignment (&lt;code&gt;clustal.aln&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;python3 aln_parser.py clustal.aln&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To save the output to file:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;python3 aln_parser.py clustal.aln &amp;gt; results.txt&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Requires &lt;code&gt;python3&lt;/code&gt; and &lt;code&gt;Biopython&lt;/code&gt; (&lt;code&gt;pip3 install biopython&lt;/code&gt;).  &lt;/p&gt;&#xA;" OwnerUserId="3381" LastEditorUserId="3381" LastEditDate="2018-10-11T04:50:03.577" LastActivityDate="2018-10-11T04:50:03.577" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5155" PostTypeId="1" AcceptedAnswerId="5166" CreationDate="2018-10-02T15:01:11.547" Score="0" ViewCount="153" Body="&lt;p&gt;I am following &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/5136/1771&quot;&gt;this answer&lt;/a&gt; to select reads with allowed barcodes from a Split-Seq run. In particular, I have a whitelist for each round of barcodes (3 rounds) and I filter out reads which have more than 1 Hamming distance from allowed barcodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, after filtering, the number of reads remaining are only the 0.7% of the original number of reads before filtering (169k from 22M). Is this normal, or what may be the underlying issue here?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="57" LastEditDate="2018-10-02T15:21:27.687" LastActivityDate="2018-10-04T14:40:20.053" Title="Low percentage of reads with consistent barcodes in Split-Seq run" Tags="|scrnaseq|reads|splitseq|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5156" PostTypeId="2" ParentId="692" CreationDate="2018-10-02T18:01:32.747" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://www.dnabaser.com/download/download.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DNA Baser Assembler&lt;/a&gt; will convert ABI to SCF (and other formats). &#xA;Just double click the ABI file you want to convert then go to the &quot;Samples&quot; menu and choose &quot;Save current samples as...&quot;. Plus, it can automatically trim the low quality ends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7GvoY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7GvoY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;There is another tool in that package what will convert all ABI files at batch: &lt;a href=&quot;http://www.dnabaser.com/download/chromatogram-explorer/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chromatogram Explorer&lt;/a&gt;. There is a Lite version that is free, but the conversion to SCF might only work for the first 100 files (if I remember correctly).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related to the comment posted by BioGeek about the Genious license: DNA Baser Assembler does not require a license key. It will work right away.&lt;/p&gt;&#xA;" OwnerUserId="3440" LastEditorUserId="3440" LastEditDate="2018-10-02T18:13:03.940" LastActivityDate="2018-10-02T18:13:03.940" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5157" PostTypeId="1" AcceptedAnswerId="5172" CreationDate="2018-10-03T00:55:17.393" Score="1" ViewCount="39" Body="&lt;p&gt;&lt;a href=&quot;https://youtu.be/-KQ6VtOTfzw?t=127&quot; rel=&quot;nofollow noreferrer&quot;&gt;In this video,&lt;/a&gt; the lecturer says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;You see that phenylalanine is probably still phenylalanine because only one percent of amino acids have changed so roughly 99 percent of the time, phenylalanine is still phenylalanine. Now why is it not exactly 99 percent? Because it’s one percent for all 20 amino acids so some are greater than 99 percent and some are less than 99 percent. It averages out to 99 percent.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What on earth does he mean by this? I either do not have the background knowledge or am not thinking hard enough, &lt;strong&gt;or&lt;/strong&gt; I need this to be explained in a way that is more understandable.&lt;/p&gt;&#xA;" OwnerUserId="3442" LastActivityDate="2018-10-05T14:20:40.983" Title="What does it mean for an amino acid to match with itself 99% of the time in a PAM1 matrix?" Tags="|statistics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5158" PostTypeId="1" CreationDate="2018-10-02T15:34:43.980" Score="5" ViewCount="2333" Body="&lt;p&gt;I am working with single-cell RNA-seq data, using the R package &quot;Seurat&quot; to cluster and visual data-points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had two single cell datasets from which I generated two Seurat objects. I then combined the two using MergeSeurat. I did differential gene expression analysis, performed clustering, and ran a tSNE plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now what I want to do is create that same tSNE plot, but color each cell according to the original object from which it came. How do I do this?&lt;/p&gt;&#xA;" OwnerUserId="3449" OwnerDisplayName="David Tatarakis" LastEditorUserId="57" LastEditDate="2018-10-03T13:01:19.673" LastActivityDate="2018-10-10T20:03:55.893" Title="Seurat Merged objects tSNE - How to paint on original IDs?" Tags="|r|rna-seq|visualization|seurat|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5159" PostTypeId="2" ParentId="5158" CreationDate="2018-10-03T07:09:01.530" Score="0" Body="&lt;p&gt;It is impossible to create the same tSNE plot without knowing which seed you used. tSNE plots are not like PCAs you don't get the same results when using the same data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you want another tSNE plot with a color differentiating from which object each sample comes you'll need a column with these information and color the plot by it. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-10-03T07:09:01.530" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5161" PostTypeId="2" ParentId="5141" CreationDate="2018-10-03T16:24:07.547" Score="1" Body="&lt;p&gt;As a follow up to @Llopis' answer (no native blast solution), I wrote a script &lt;a href=&quot;https://github.com/KamilSJaron/generic_genomics/blob/master/blast_filter.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;blast_filter.py&lt;/a&gt; and I added it into my collection of useful scripts &lt;a href=&quot;https://github.com/KamilSJaron/generic_genomics&quot; rel=&quot;nofollow noreferrer&quot;&gt;generic_genomics&lt;/a&gt;. The script filters out the blast output and it works with streams. To get the &lt;code&gt;n&lt;/code&gt; (5) best blast hits by e-value you can run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blastp -query &lt;span class=&quot;math-container&quot;&gt;$QUERY -db $&lt;/span&gt;PROTEINS -evalue 1e-10 -outfmt 6 | blast_filter.py 5 &amp;gt; $BLASTOUT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I assume that the blast output is sorted primarily by query name and secondarily by e-value. Also, note that default &lt;code&gt;max_target_seqs&lt;/code&gt; in blast is 500, if there will be a query with 500 alignments, script will drop a warning (the query has no guarantee of the best hit found).&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2018-10-03T16:24:07.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5162" PostTypeId="1" AcceptedAnswerId="5164" CreationDate="2018-10-03T19:19:20.257" Score="5" ViewCount="232" Body="&lt;p&gt;I hope someone can lend their thoughts on the below code to generate DNA sequences under the Kimura-2-Parameter model of DNA substitution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue is that each time the code is run and the haplotype distribution is examined, there always is a very skewed distribution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would like is for a different distribution to be generated each time the code is run. that is, there should be multiple specimens sharing the same haplotype (instead of most sharing the most common haplotype). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(pegas)&#xA;set.seed(17)&#xA;&#xA;&#xA;sim.seqs &amp;lt;- TRUE&#xA;length.seqs &amp;lt;- 500&#xA;num.seqs &amp;lt;- 100 # number of DNA sequences&#xA;subst.model &amp;lt;- &quot;K80&quot; # nucleotide substitution model&#xA;transi.rate &amp;lt;- 1e-4 # transition rate&#xA;transv.rate &amp;lt;- transi.rate / 2 # transversion rate&#xA;&#xA;&#xA;if (sim.seqs == TRUE) {&#xA;&#xA;  nucl &amp;lt;- as.DNAbin(c('a','c','g','t'))&#xA;  res &amp;lt;- sample(nucl, size = length.seqs, replace = TRUE, prob = rep(0.25, 4))&#xA;&#xA; if (subst.model == &quot;K80&quot;) {&#xA;    transi.set &amp;lt;- list('a' = as.DNAbin('g'),&#xA;                       'c' = as.DNAbin('t'),&#xA;                       'g' = as.DNAbin('a'),&#xA;                       't' = as.DNAbin('c'))&#xA;&#xA;    transv.set &amp;lt;- list('a' = as.DNAbin(c('c', 't')),&#xA;                       'c' = as.DNAbin(c('a', 'g')),&#xA;                       'g' = as.DNAbin(c('c', 't')),&#xA;                       't' = as.DNAbin(c('a', 'g')))&#xA;&#xA;    transi &amp;lt;- function(res) {&#xA;      unlist(transi.set[as.character(res)])&#xA;    }&#xA;&#xA;    transv &amp;lt;- function(res) {&#xA;      sapply(transv.set[as.character(res)], sample, 1)&#xA;    }&#xA;&#xA;    duplicate.seq &amp;lt;- function(res) {&#xA;      num.transi &amp;lt;- rbinom(n = 1, size = length.seqs, prob = transi.rate) # total number of transitions&#xA;&#xA;      if (num.transi &amp;gt; 0) {&#xA;        idx &amp;lt;- sample(length.seqs, size = num.transi, replace = FALSE)&#xA;        res[idx] &amp;lt;- transi(res[idx])&#xA;      }&#xA;&#xA;      num.transv &amp;lt;- rbinom(n = 1, size = length.seqs, prob = transv.rate) # total number of transversions&#xA;&#xA;      if (num.transv &amp;gt; 0) {&#xA;        idx &amp;lt;- sample(length.seqs, size = num.transv, replace = FALSE)&#xA;        res[idx] &amp;lt;- transv(res[idx])&#xA;      }&#xA;&#xA;      res&#xA;      }&#xA;&#xA;    }&#xA;&#xA;  res &amp;lt;- matrix(replicate(num.seqs, duplicate.seq(res)), byrow = TRUE, nrow = num.seqs)&#xA;&#xA;  class(res) &amp;lt;- &quot;DNAbin&quot;&#xA;&#xA;  # write.dna(res, file = &quot;seqs.fas&quot;, format = &quot;fasta&quot;)&#xA;&#xA;  h &amp;lt;- sort(haplotype(res), decreasing = TRUE, what = &quot;frequencies&quot;)&#xA;  rownames(h) &amp;lt;- 1:nrow(h)&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Output&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;h&#xA;&#xA;Haplotypes extracted from: res&#xA;&#xA;Number of haplotypes: 5&#xA;     Sequence length: 500&#xA;&#xA;Haplotype labels and frequencies:&#xA;&#xA; 1  2  3  4  5&#xA;96  1  1  1  1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If the code is run multiple times (without the seed), the same pattern emerges: h is always skewed toward the most dominant haplotype.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I want to be able to run the code (without seed) and get something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;h&#xA;&#xA;Haplotypes extracted from: res&#xA;&#xA;Number of haplotypes: 5&#xA;     Sequence length: 500&#xA;&#xA;Haplotype labels and frequencies:&#xA;&#xA;  1    2    3    4   5&#xA;  35  25   20   15   5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;i think I would have to specify a distribution for the haplotypes, but the number of haplotypes generated by the mutation process is not known beforehand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any thoughts?&lt;/p&gt;&#xA;" OwnerUserId="3450" LastEditorUserId="3450" LastEditDate="2018-10-04T19:32:47.277" LastActivityDate="2018-10-05T05:10:26.430" Title="Simulating DNA sequence evolution in R" Tags="|r|phylogenetics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5163" PostTypeId="1" AcceptedAnswerId="5170" CreationDate="2018-10-03T20:13:49.267" Score="5" ViewCount="174" Body="&lt;p&gt;I am trying to figure out the best way to extract sequences from a FASTA file which begin with a common 5' region of 43 nucleotides. Preferably, I would like to to allow for &quot;fuzziness&quot; in this region to allow for mutations or read overlaps. The idea is that any sequences that begin with this region, regardless of size, could be extracted into a new file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example of the FASTA file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1-69050-454.08&#xA;GTACGGGGAAGGACGTCAATAGTC&#xA;&#xA;&amp;gt;2-65989-433.95&#xA;AATCTGTAGTACGACTCACTATAGC&#xA;&#xA;&amp;gt;3-62181-408.91&#xA;GATCTGTAATACGACTCACTATAGG&#xA;&#xA;&amp;gt;4-49959-328.53&#xA;GGGGAAGGACGTCAATAGTCACAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And what I would like to get from the code is such:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;2-65989-433.95&#xA;AATCTGTAGTACGACTCACTATAGC&#xA;&#xA;&amp;gt;3-62181-408.91&#xA;GATCTGTAATACGACTCACTATAGG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The FASTA file is quite large and I have tried using a couple grep and awk methods retrieve the sequences. Any help you could provide is much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="3451" LastEditorUserId="73" LastEditDate="2023-01-15T20:20:51.153" LastActivityDate="2023-01-15T20:20:51.153" Title="Extracting sequences from FASTA beginning with common 5' end" Tags="|ngs|fasta|awk|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5164" PostTypeId="2" ParentId="5162" CreationDate="2018-10-03T20:54:34.027" Score="1" Body="&lt;p&gt;What are you expecting these lines to do?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;num.transi &amp;lt;- rbinom(n = 1, size = length.seqs, prob = transi.rate)&#xA;...&#xA;num.transv &amp;lt;- rbinom(n = 1, size = length.seqs, prob = transv.rate)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using the variables that are specified in your code, they are extremely zero-skewed. This is not too surprising given the mutation rate and sequence length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I expect that increasing the sequence length, or increasing the transition rate, should result in more haplotypes being generated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After running this through myself, I see that increasing the mutation rate does increase the number of haplotypes, but the skew is retained. This is because each mutation creates a new haplotype, following something like an infinite allele model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess this is not particularly surprising. It's unlikely that two random mutations from a particular haplotype will result in the same haplotype, and therefore any mutations (from &lt;em&gt;any&lt;/em&gt; existing haplotype) will generally create something new.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-10-05T05:10:26.430" LastActivityDate="2018-10-05T05:10:26.430" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5165" PostTypeId="1" AcceptedAnswerId="5200" CreationDate="2018-10-04T11:23:53.807" Score="2" ViewCount="82" Body="&lt;p&gt;I have a lot of IonTorrent sequencing data in BAM format that I want to call variants on, using the IonTorrent variant caller. However, for some of the BAM files (for about 12 samples out of ~150), the variant caller stops due to what it states is lack of ZM: tags in a few individual reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing is, most of the reads in the BAM file do not have ZM: tags (when viewed through samtools view), but these do not apparently cause problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What settings do I need to change to get these BAM files to process, or do I just filter out the problematic reads and analyse the remainder? &lt;/p&gt;&#xA;" OwnerUserId="2244" LastEditorUserId="57" LastEditDate="2018-10-07T21:38:32.683" LastActivityDate="2018-10-11T11:12:25.207" Title="IonTorrent variant caller failing, lack of ZM: tags in BAM files" Tags="|variant-calling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5166" PostTypeId="2" ParentId="5155" CreationDate="2018-10-04T14:40:20.053" Score="0" Body="&lt;p&gt;In my case the problem was due a wrong set of allowed round1 barcodes, therefore the filtering pipeline was removing most part of the reads. In particular, with the right barcode1 whitelist the total number of reads after filtering is 17M (about 100 times more).&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-10-04T14:40:20.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5167" PostTypeId="1" CreationDate="2018-10-04T16:43:40.290" Score="8" ViewCount="482" Body="&lt;p&gt;I’m trying to process BAM records (of a coordinate-sorted, indexed BAM file) successively in fixed-size, non-overlapping, sliding genome coordinate windows. Unfortunately I am unable to read records via &lt;code&gt;scanBam&lt;/code&gt; in different windows (&lt;code&gt;which&lt;/code&gt; parameter): the first call to &lt;code&gt;scanBam&lt;/code&gt; succeeds but subsequent calls return zero-length results.&lt;/p&gt;&#xA;&lt;p&gt;By contrast, if I don’t specify a coordinate window and instead use &lt;code&gt;yieldSize&lt;/code&gt; when opening the BAM file, I can successfully read records in a loop:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(Rsamtools)&#xA;bamfile = system.file('extdata', 'ex1.bam', package = 'Rsamtools')&#xA;process = function (reads) message(length(reads$seq))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bam = open(BamFile(bamfile, yieldSize = 1000L))&#xA;param = ScanBamParam(what = 'seq')&#xA;&#xA;repeat {&#xA;    reads = scanBam(bam, param = param)[[1L]]&#xA;    if (length(reads$seq) == 0L) break&#xA;    process(reads)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;… where &lt;code&gt;process&lt;/code&gt; performs the read processing (unimportant for the example).&lt;/p&gt;&#xA;&lt;p&gt;Now I’m trying to use the following code to work in genomic coordinate windows instead, and it fails as explained above:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;windowsize = 1000L&#xA;&#xA;bam = open(BamFile(bamfile))&#xA;chr = seqlevels(bam)[1L] # Only do one chromosome for now&#xA;chrlengths = seqlengths(bam)&#xA;&#xA;result = lapply(seq(1L, chrlengths[chr] - windowsize, by = windowsize), function (pos) {&#xA;    which = setNames(RangesList(IRanges(pos, pos + windowsize)), chr)&#xA;    param = ScanBamParam(what = 'seq', which = which)&#xA;    reads = scanBam(bam, param = param)[[1L]]&#xA;    process(reads)&#xA;})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Conversely, it works if I close and re-open the BAM file in each iteration (i.e. before each &lt;code&gt;scanBam&lt;/code&gt; call) — but that’s undesirable: it carries a &lt;strong&gt;substantial&lt;/strong&gt; overhead (running on an &lt;em&gt;empty&lt;/em&gt; BAM file with 16 chromosomes in its header takes several &lt;em&gt;hours&lt;/em&gt;).&lt;/p&gt;&#xA;&lt;p&gt;It's only returning data from the first iteration, not any subsequent iterations, unless I close the BAM file between scanBam calls.&lt;/p&gt;&#xA;&lt;p&gt;Since the &lt;code&gt;process&lt;/code&gt; function is nontrivial and written in R, I’m stuck with R for now. And there’s no reason why this &lt;em&gt;shouldn’t&lt;/em&gt; work in R.&lt;/p&gt;&#xA;&lt;p&gt;Am I overlooking something? Is this a bug in {Rsamtools}? Is it “by design”?&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="73" LastEditDate="2021-05-30T11:46:58.130" LastActivityDate="2021-07-08T03:35:27.210" Title="How can I read multiple different regions from a BAM file in R?" Tags="|r|sam|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5168" PostTypeId="1" CreationDate="2018-10-04T17:06:00.940" Score="4" ViewCount="3908" Body="&lt;p&gt;Is it ever ok to use Seurat for clustering bulk samples?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking at FPKM data from ~750 bulk RNA-seq samples generated using &lt;code&gt;Cufflinks&lt;/code&gt;. As suggested for FPKM data, I manually input log transformed data to the &lt;code&gt;@data&lt;/code&gt; slot [&lt;code&gt;cd138_bm@data &amp;lt;- log(cd138_bm@raw.data + 1)&lt;/code&gt;] and skip the &lt;code&gt;NormalizeData()&lt;/code&gt; function. I then use functions &lt;code&gt;FindVariableGenes&lt;/code&gt;, &lt;code&gt;ScaleData&lt;/code&gt;, &lt;code&gt;RunPCA&lt;/code&gt;, &lt;code&gt;FindClusters&lt;/code&gt;, &lt;code&gt;RunTSNE&lt;/code&gt;, &lt;code&gt;FindAllMarkers&lt;/code&gt; in their usual ways to find clusters &amp;amp; cluster markers. My clustering results are quite reasonable, and reflect published work clustering similar samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the potential pitfalls of using these Seurat functions on bulk data? In &lt;code&gt;FindAllMarkers&lt;/code&gt;, would you recommend I use the &quot;negbinom&quot; test? (currently using wilcox) Any other arguments you would recommend changing from the default?&lt;/p&gt;&#xA;" OwnerUserId="3454" LastEditorUserId="57" LastEditDate="2018-10-05T08:45:43.930" LastActivityDate="2023-06-07T12:34:04.967" Title="Seurat for clustering bulk RNA-seq?" Tags="|rna-seq|seurat|clustering|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5170" PostTypeId="2" ParentId="5163" CreationDate="2018-10-05T03:02:39.043" Score="2" Body="&lt;p&gt;Here is a 'quick' &lt;code&gt;python3&lt;/code&gt; solution, requiring installation of &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK52640/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BLASTN&lt;/a&gt; and a &lt;code&gt;pip3 install biopython&lt;/code&gt;.  The results are output in FASTA format to &lt;code&gt;stdout&lt;/code&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Save the following &lt;code&gt;python3&lt;/code&gt; code in &lt;code&gt;hit_finder.py&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;import sys&#xA;from Bio import SeqIO&#xA;from Bio.Blast.Applications import NcbiblastnCommandline&#xA;&#xA;cline = NcbiblastnCommandline(query=sys.argv[1],&#xA;                              subject=sys.argv[2],&#xA;                              outfmt=&quot;'6 qseqid length qstart'&quot;,&#xA;                              max_hsps=1,&#xA;                              word_size=4)&#xA;&#xA;result = [j[0] for j in [i.split('\t') for i in&#xA;                         cline()[0].rstrip().split('\n')] &#xA;          if int(j[1]) &amp;gt;= int(sys.argv[3]) and int(j[2]) &amp;lt;= int(sys.argv[4])]&#xA;&#xA;for record in SeqIO.parse(sys.argv[1], 'fasta'):&#xA;    if record.id in result:&#xA;        print(record.format('fasta'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Make it executable: &lt;code&gt;chmod +x hit_finder.py&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This script takes four arguments (which in future versions could be enhanced using &lt;code&gt;argparse&lt;/code&gt; instead of &lt;code&gt;sys.argv&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;arg1 = query_fasta_file (file to search in)&#xA;arg2 = subject_fasta_file (database file to contain the sequence to match at 5')&#xA;arg3 = min_alignment_length (this means the length of the hit, allows mismatching in string with blastn and a word_size of 4)&#xA;arg4 = start_of_hit_at_5prime (where should the alignment hit start?)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Let's say you wanted to search for &lt;code&gt;AATCTGTAGTACGACTCACTATAGC&lt;/code&gt; in your query seqeuences.  Save this sequence in &lt;code&gt;subject.fa&lt;/code&gt; as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;subject&#xA;AATCTGTAGTACGACTCACTATAGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Your query sequences are stored in &lt;code&gt;queries.fa&lt;/code&gt; as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1-69050-454.08&#xA;GTACGGGGAAGGACGTCAATAGTC&#xA;&#xA;&amp;gt;2-65989-433.95&#xA;AATCTGTAGTACGACTCACTATAGC&#xA;&#xA;&amp;gt;3-62181-408.91&#xA;GATCTGTAATACGACTCACTATAGG&#xA;&#xA;&amp;gt;4-49959-328.53&#xA;GGGGAAGGACGTCAATAGTCACAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, to search for hits of at least 23 bp starting no further than 1 base from the 5' end of your sequence, you would run this search using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./hit_finder.py queries.fa subject.fa 23 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To search for hits of at least 30 bp no more than 10 bp from the 5' end of your queries, do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./hit_finder.py queries.fa subject.fa 30 10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To save your results, output &lt;code&gt;stdout&lt;/code&gt; to file, e.g.,:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./hit_finder.py queries.fa subject.fa 30 10 &amp;gt; results.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3381" LastEditorUserId="298" LastEditDate="2018-10-08T08:07:51.670" LastActivityDate="2018-10-08T08:07:51.670" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5171" PostTypeId="1" CreationDate="2018-10-05T04:23:03.203" Score="6" ViewCount="1035" Body="&lt;p&gt;I have 2 GFF3 files:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Features using transcript IDs as the landmarks. i.e. &quot;CDS&quot; feature types using coordinates from transcript space.&lt;/li&gt;&#xA;&lt;li&gt;Features using chromosome IDs as the landmarks. i.e. &quot;exon&quot; feature types using coordinates from chromosome space.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I would like to transform the coordinates of the features in file 1 into the coordinate space of file 2. i.e. transcript-based coordinates into genomic-based coodinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example for file1:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat transcript_orfs.gff3&#xA;##gff-version 3&#xA;##sequence-region Tx.1 1 4000&#xA;Tx.1    ORF_finder  gene    1   4000    .   +   .   ID=1&#xA;Tx.1    ORF_finder  mRNA    1   4000    .   +   .   ID=2;Parent=1&#xA;Tx.1    ORF_finder  exon    1501    2500    .   +   0   ID=3;Parent=2&#xA;Tx.1    ORF_finder  CDS 1501    2500    .   +   0   ID=4;Parent=2&#xA;&#xA;gt sketch -addintrons transcript_orfs.png transcript_orfs.gff3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YEeb0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YEeb0.png&quot; alt=&quot;coordinates in transcript space&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example for file1:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat genome.gff3&#xA;##gff-version 3&#xA;##sequence-region chr3A_part1 1 454103970&#xA;chr3A_part1 genome_assembly gene    1001    6000    .   +   .   ID=Tx&#xA;chr3A_part1 genome_assembly mRNA    1001    6000    .   +   .   ID=Tx.1;Parent=Tx&#xA;chr3A_part1 genome_assembly exon    1001    3000    .   +   .   ID=Tx.1.exon1;Parent=Tx.1&#xA;chr3A_part1 genome_assembly five_prime_UTR  1001    2000    .   +   .   ID=Tx.1.utr5;Parent=Tx.1&#xA;chr3A_part1 genome_assembly CDS 2001    3000    .   +   0   ID=Tx.1.cds1;Parent=Tx.1&#xA;chr3A_part1 genome_assembly exon    4001    6000    .   +   .   ID=Tx.1.exon2;Parent=Tx.1&#xA;chr3A_part1 genome_assembly CDS 4001    5000    .   +   2   ID=Tx.1.cds2;Parent=Tx.1&#xA;chr3A_part1 genome_assembly three_prime_UTR 5001    6000    .   +   .   ID=Tx.1.utr3;Parent=Tx.1&#xA;&#xA;gt sketch -addintrons genome.png genome.gff3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XErM1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XErM1.png&quot; alt=&quot;coordinates in genomic space&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to convert the coordinates of the features from file1 into chromosome genomic coordinates. I expect to get something like the following output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat output.gff3&#xA;##gff-version 3&#xA;##sequence-region chr3A_part1 1 454103970&#xA;chr3A_part1 ORF_finder  gene    1001    6000    .   +   .   ID=1&#xA;chr3A_part1 ORF_finder  mRNA    1001    6000    .   +   .   ID=2;Parent=1&#xA;chr3A_part1 ORF_finder  exon    2501    3000    .   +   0   ID=3.1;Parent=2&#xA;chr3A_part1 ORF_finder  CDS 2501    3000    .   +   0   ID=4.1;Parent=2&#xA;chr3A_part1 ORF_finder  exon    4001    4500    .   +   0   ID=3.2;Parent=2&#xA;chr3A_part1 ORF_finder  CDS 4001    4500    .   +   1   ID=4.2;Parent=2&#xA;&#xA;gt sketch -addintrons output.png output.gff3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/h6f6N.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/h6f6N.png&quot; alt=&quot;transformed coordinates&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have looked into using &lt;code&gt;mapFromTranscripts()&lt;/code&gt; from Bioconductor's &lt;code&gt;GenomicRanges&lt;/code&gt; library but I have made little progress trying to decipher the manual.&lt;/p&gt;&#xA;" OwnerUserId="400" LastEditorUserId="96" LastEditDate="2018-12-13T15:12:10.103" LastActivityDate="2018-12-14T11:04:29.183" Title="Transcript Coordinate Ranges to Genomic Coordinates" Tags="|bioconductor|sequence-annotation|gff3|interval|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5172" PostTypeId="2" ParentId="5157" CreationDate="2018-10-05T14:20:40.983" Score="0" Body="&lt;p&gt;It means that the 99% of conservation of amino acids is just a mean value for all the &lt;a href=&quot;https://en.wikipedia.org/wiki/Amino_acid&quot; rel=&quot;nofollow noreferrer&quot;&gt;amino acids&lt;/a&gt;. That there can be amino acids that are less conserved and some that are more conserved. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So from the 21 basic amino acids, arginine is more conserved than lysine in those alignments. But in general they are conserved the 99% of times.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-10-05T14:20:40.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5173" PostTypeId="1" CreationDate="2018-10-05T15:09:14.857" Score="0" ViewCount="175" Body="&lt;p&gt;I am using chromPlot R package to plot some SNPs and their associated score, however, I cannot find a way to remove the scale of such score (circled in the attached picture). Does anybody know how to do it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you so much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kdPhs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kdPhs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3457" LastActivityDate="2018-10-22T21:04:05.363" Title="chromPlot R package: remove scales" Tags="|snp|chromplot|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5174" PostTypeId="2" ParentId="5056" CreationDate="2018-10-05T22:01:29.783" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://www.megasoftware.net/mega4/WebHelp/hc_maximum_parsimony.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;MEGA manual&lt;/a&gt; may help to explain the criteria used in selecting the final maximum parsimony tree. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The random tree additions refers to the generation of random initial trees for a heuristic search to find MP trees. In this case, a tree is generated by randomly selecting a sequence and adding it to the growing tree on a randomly-selected branch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead of using the random tree additions you can use the min-mini algorithm for the initial tree set. &lt;/p&gt;&#xA;" OwnerUserId="3460" LastEditorUserId="57" LastEditDate="2018-10-06T11:45:43.670" LastActivityDate="2018-10-06T11:45:43.670" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5175" PostTypeId="1" CreationDate="2018-10-05T22:33:24.360" Score="1" ViewCount="616" Body="&lt;p&gt;I have two &lt;code&gt;scRNAseq&lt;/code&gt; datasets, and I select a cluster from one of them, and another cluster from the other. I want to find differentially expressed genes between the two clusters. What would be the right way of doing that? Right now I am using &lt;code&gt;DESeq2&lt;/code&gt; in the following way. I select only the selected clusters' cells, merge them together forming a matrix and then feed them into &lt;code&gt;DESeq2&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mtx &amp;lt;- merge(data_1,data_2,by=&quot;row.names&quot;,all=TRUE)&#xA;mtx[is.na(mtx)] &amp;lt;- 0&#xA;mtx_conv &amp;lt;- matrix(as.numeric(unlist(mtx[,2:ncol(mtx)])),nrow=nrow(mtx))&#xA;mtx_int &amp;lt;- round(mtx_conv, 0)&#xA;rownames(mtx_int) &amp;lt;- as.character(mtx$Row.names)&#xA;&#xA;cluster &amp;lt;- as.factor(c(rep(0, dim(data_1)[2]), rep(1, dim(data_2)[2])))&#xA;colData &amp;lt;- data.frame(&#xA;  Cluster = cluster&#xA;)&#xA;&#xA;dds &amp;lt;- DESeqDataSetFromMatrix(countData = mtx_int,&#xA;                          colData = colData,&#xA;                          design = ~ Cluster)&#xA;dds &amp;lt;- DESeq(dds)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am not sure though that it is the correct way to go because I am not using the full original datasets. The reason is that I want the computations to run faster. Another way of doing that is probably combining the whole two datasets, defining 3 clusters: two groups I am interested in, and all other cells, and then running the analysis. Would it differ significantly? Am I on the right track at all?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have another idea: to use &lt;code&gt;Seurat&lt;/code&gt; package. There, I would run &lt;code&gt;CCA&lt;/code&gt; algorithm to align two full datasets, and then run &lt;code&gt;FindMarkers&lt;/code&gt; function between the two clusters. I feel like it may be wrong, because the two datasets may need to be re-normalized together but Seurat does not seem to be doing that:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://satijalab.org/seurat/Seurat_AlignmentTutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/Seurat_AlignmentTutorial.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-10-22T20:59:16.347" Title="Compare clusters from different datasets" Tags="|scrnaseq|differential-expression|deseq2|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5176" PostTypeId="1" CreationDate="2018-10-08T11:13:07.227" Score="5" ViewCount="288" Body="&lt;p&gt;I wish to assign character state changes from a presence-absence matrix to a phylogeny. Thus I want to identify the most parsimonious hypothesis for which node and branch a given &quot;mutation&quot; or &quot;change&quot; has occurred.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried assigning each character to its leaf node, and then if the leaf node's sister has the same character I reassign the character to the parent node (and work back until all nodes are assigned). I am using a dummy dataset to try to achieve this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Matrix&#xA;&amp;gt;Dme_001&#xA;1110000000000111&#xA;&amp;gt;Dme_002&#xA;1110000000000011&#xA;&amp;gt;Cfa_001&#xA;0110000000000011&#xA;&amp;gt;Mms_001&#xA;0110000000000011&#xA;&amp;gt;Hsa_001&#xA;0110000000000010&#xA;&amp;gt;Ptr_002&#xA;0110000000000011&#xA;&amp;gt;Mmu_002&#xA;0110000000000011&#xA;&amp;gt;Hsa_002&#xA;0110000000000011&#xA;&amp;gt;Ptr_001&#xA;0110000000000011&#xA;&amp;gt;Mmu_001&#xA;0110000000000011&#xA;&#xA;Phylogeny&#xA;((Dme_001,Dme_002),(((Cfa_001,Mms_001),((Hsa_001,Ptr_001),Mmu_001)),(Ptr_002,(Hsa_002,Mmu_002))));&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I assign internal nodes using ete3, so my output should be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BranchID    CharacterState    Change&#xA;Node_1:    0    0-&amp;gt;1&#xA;Hsa_001:    15    1-&amp;gt;0 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As my code assigns character states based on their sisters if a loss is encountered it messes up the output so that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BranchID    CharacterState    Change&#xA;Node_1:   0     0-&amp;gt;1&#xA;Node_3    15    0-&amp;gt;1&#xA;Node_5    15    0-&amp;gt;1&#xA;Node_8    15    0-&amp;gt;1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm coding in python 2.7 and welcome assistance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from ete3 import PhyloTree&#xA;from collections import Counter&#xA;import itertools&#xA;&#xA;PAM = open('PAM','r')&#xA;&#xA;gene_tree = '((Dme_001,Dme_002),(((Cfa_001,Mms_001),((Hsa_001,Ptr_001),Mmu_001)),(Ptr_002,(Hsa_002,Mmu_002))));'&#xA;&#xA;NodeIDs = []&#xA;&#xA;tree = PhyloTree(gene_tree)&#xA;edge = 0&#xA;for node in tree.traverse():&#xA;    if not node.is_leaf():&#xA;        node.name = &quot;Node_%d&quot; %edge&#xA;        edge +=1&#xA;        NodeIDs.append(node.name)&#xA;    if node.is_leaf():&#xA;        NodeIDs.append(node.name)&#xA;&#xA;f = open('PAM','r')&#xA;&#xA;taxa = []&#xA;pap = []&#xA;&#xA;for line in f:&#xA;    term = line.strip().split('\t')&#xA;    taxa.append(term[0])&#xA;    p = [p for p in term[1]]&#xA;    pap.append(p)&#xA;&#xA;statesD = dict(zip(taxa, pap))&#xA;&#xA;def PlotCharacterStates():&#xA;&#xA;    Plots = []&#xA;&#xA;    events = []&#xA;&#xA;    for key, value in statesD.iteritems():&#xA;        count = -1&#xA;        for s in value: &#xA;            count+=1&#xA;            if s == CharacterState:&#xA;                a = key, count&#xA;                events.append(a)&#xA;&#xA;    Round3_events = []&#xA;    while len(events) &amp;gt; 0:&#xA;        for rel in Relationships:&#xA;            node_store = []&#xA;            sis_store = []&#xA;            for event in events:&#xA;                if rel[0] == event[0]:&#xA;                    node_store.append(event[1])&#xA;                if rel[1] == event[0]:&#xA;                    sis_store.append(event[1])&#xA;            if (len(node_store) &amp;gt; 0) and (len(sis_store) &amp;gt; 0):&#xA;                place = rel, node_store, sis_store&#xA;                Round3_events.append(place)&#xA;&#xA;        moved = []&#xA;        for placement in Round3_events:&#xA;            intercept = (set(placement[1]) &amp;amp; set(placement[2]))&#xA;            node_plot = (set(placement[1]) - set(placement[2]))&#xA;            sis_plot = (set(placement[2]) - set(placement[1]))&#xA;            if len(node_plot) &amp;gt; 0:&#xA;                for x in node_plot:&#xA;                    y = placement[0][0], x&#xA;                    Plots.append(y)&#xA;                    moved.append(y)&#xA;            if len(sis_plot) &amp;gt; 0:&#xA;                for x in sis_plot:&#xA;                    y = placement[0][1], x&#xA;                    Plots.append(y)&#xA;                    moved.append(y)&#xA;            if len(intercept) &amp;gt; 0:&#xA;                for x in intercept:&#xA;                    y = placement[0][2], x&#xA;                    y1 = placement[0][0], x&#xA;                    y2 = placement[0][1], x&#xA;                    moved.append(y1)&#xA;                    moved.append(y2)&#xA;                    events.append(y)&#xA;&#xA;        for event in events:&#xA;            if event[0] == &quot;Node_0&quot;:&#xA;                Plots.append(event)&#xA;                moved.append(event)&#xA;&#xA;        events2 = (set(events) - set(moved))&#xA;        events = []&#xA;        for event in events2:&#xA;            events.append(event)&#xA;&#xA;&#xA;    pl = set(Plots)&#xA;    Plots = []&#xA;    for p in pl:&#xA;        Plots.append(p)&#xA;&#xA;    print CharacterState, Plots&#xA;&#xA;&#xA;'''&#xA;assign sisters to leaves, internals&#xA;'''&#xA;&#xA;e = []&#xA;round1b_e = []&#xA;round2a_e = []&#xA;placements = []&#xA;Relationships = []&#xA;Rounds = []&#xA;for node in tree.traverse():&#xA;    sisters = node.get_sisters()&#xA;    parent = node.up&#xA;    cycle1 = []&#xA;    if node.is_leaf():&#xA;        for sister in sisters:&#xA;            if sister.is_leaf():&#xA;                round1a = [&quot;Round1a&quot;, node.name, sister.name, parent.name]&#xA;                node_names = node.name, sister.name&#xA;                Rounds.append(round1a)&#xA;                e.append(node_names)&#xA;                x = node.name, sister.name, parent.name, &quot;leaf-leaf&quot;&#xA;                Relationships.append(x)&#xA;            if not sister.is_leaf():&#xA;                round1b =  [&quot;Round1b&quot;, node.name, sister.name, parent.name]&#xA;                node_names = node.name, sister.name&#xA;                Rounds.append(round1b)&#xA;                round1b_e.append(node_names)&#xA;                x = node.name, sister.name, parent.name, &quot;node-leaf&quot;&#xA;                Relationships.append(x)&#xA;    elif not node.is_leaf():&#xA;        if not node.is_root():&#xA;            for sister in sisters:&#xA;                if not sister.is_leaf():&#xA;                    node_names = node.name, sister.name&#xA;                    round2a_e.append(node_names)&#xA;                    x = node.name, sister.name, parent.name, &quot;node-node&quot;&#xA;                    Relationships.append(x)&#xA;&#xA;x = []&#xA;CharacterStates = []                &#xA;for key, value in statesD.iteritems():&#xA;    for value in value:&#xA;        x.append(value)&#xA;&#xA;y = sorted(set(x))&#xA;for x in y:&#xA;    CharacterStates.append(x)&#xA;&#xA;for CharacterState in CharacterStates:&#xA;    PlotCharacterStates()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3394" LastEditorUserId="3848" LastEditDate="2020-05-03T12:26:36.607" LastActivityDate="2020-05-03T12:26:36.607" Title="How to plot character state changes from a presence-absence matrix on to a phylogeny" Tags="|r|python|phylogenetics|phylogeny|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5178" PostTypeId="1" AcceptedAnswerId="5179" CreationDate="2018-10-08T19:20:25.627" Score="11" ViewCount="10607" Body="&lt;p&gt;What is the index fastq file that comes with some Illumina sequencing datasets? (The &lt;code&gt;samplename_I*.fastq.gz&lt;/code&gt; file.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I recently received some 10X Chromium reads for two libraries sequenced on the same lane. This was a 2x150 sequencing run, so there should be two fastq files. The sequencing center demultiplexed the libraries and generated two separate directories - one for each library.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In each directory there are three fastq files:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;Mysample_I1_001.fastq.gz&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;Mysample_R1_001.fastq.gz&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;Mysample_R2_001.fastq.gz&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I know that the &lt;a href=&quot;https://support.illumina.com/content/dam/illumina-support/documents/documentation/system_documentation/miseq/indexed-sequencing-overview-guide-15057455-04.pdf&quot; rel=&quot;noreferrer&quot;&gt;R1 fastq file contains the forward reads and the R2 fastq file contains the reverse reads&lt;/a&gt;, but what is the index file? The first few sequences do not match the 10X indices for this library (&lt;code&gt;ACATTACT&lt;/code&gt;, &lt;code&gt;CAGCCCAC&lt;/code&gt;, &lt;code&gt;GGCAATGG&lt;/code&gt;, &lt;code&gt;TTTGGGTA&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;bioawk -cfastx '{print($seq)}' Mysample_I1_001.fastq.gz | head&#xA;NTTGGGTA&#xA;NGCAATGG&#xA;NAGCCCAC&#xA;NAGCCCAC&#xA;NCATTACT&#xA;NCATTACT&#xA;NGCAATGG&#xA;NAGCCCAC&#xA;NTTGGGTA&#xA;NAGCCCAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-10-09T23:37:19.267" LastActivityDate="2019-02-12T10:13:59.027" Title="What is the index fastq file (sample_I*.fastq.gz) generated when demultiplexing Illumina paired-end runs?" Tags="|ngs|illumina|10x-genomics|demultiplexing|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5179" PostTypeId="2" ParentId="5178" CreationDate="2018-10-08T19:20:25.627" Score="9" Body="&lt;h1&gt;tldr - The I*.fastq.gz file contains the read index sequences.&lt;/h1&gt;&#xA;&lt;h2&gt;long explanation&lt;/h2&gt;&#xA;&lt;p&gt;Illumina uses a program called &lt;a href=&quot;https://support.illumina.com/content/dam/illumina-support/documents/documentation/software_documentation/bcl2fastq/bcl2fastq_letterbooklet_15038058brpmi.pdf&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;bcl2fastq&lt;/code&gt;&lt;/a&gt; to demultiplex sequencing runs.&lt;/p&gt;&#xA;&lt;p&gt;This software takes a list of samples and their associated indices and uses those sequences to make one or more fastq files per sample, binned by one or two index sequences on either end of the sequencing molecule (&lt;a href=&quot;https://support.illumina.com/content/dam/illumina-support/documents/documentation/system_documentation/miseq/indexed-sequencing-overview-guide-15057455-04.pdf&quot; rel=&quot;noreferrer&quot;&gt;i5 and i7 indices, see page 6 for HiSeq&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Illumina sequencing isn't perfect though, and there are sometimes errors in reading the index sequence. For example this index, &lt;code&gt;CAGCCCAC&lt;/code&gt; can easily have read errors in the &lt;code&gt;A&lt;/code&gt; sandwiched between many &lt;code&gt;C&lt;/code&gt;s: &lt;code&gt;CAGCCCAC -&amp;gt; CAGCCCCC&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Instead of throwing out all of the reads with indexing sequencing errors, the &lt;code&gt;bcl2fastq&lt;/code&gt; program includes reads that are discernibly derived from a true index for that sample as long as there is no overlap with another sample.&lt;/p&gt;&#xA;&lt;p&gt;In the example provided above, the four indices for &lt;code&gt;mysample&lt;/code&gt; were: &lt;code&gt;ACATTACT&lt;/code&gt;, &lt;code&gt;CAGCCCAC&lt;/code&gt;, &lt;code&gt;GGCAATGG&lt;/code&gt;, &lt;code&gt;TTTGGGTA&lt;/code&gt;. If we look at all of the index sequences in the &lt;code&gt;Mysample_I1_001.fastq.gz&lt;/code&gt; file, we will see that sequences with the four correct indices are indeed the most prevalent, however there are also reads that have indices with sequencing errors derived from the correct indices.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bioawk -cfastx '{print($seq)}' Mysample_I1_001.fastq.gz | \&#xA;    sort | uniq -c | sort -k1 -nr | head&#xA;&#xA;count      index       source&#xA;41362311   CAGCCCAC    True index&#xA;37209190   GGCAATGG    True index&#xA;36863213   ACATTACT    True index &#xA;33674467   TTTGGGTA    True index&#xA; 1140358   NAGCCCAC    CAGCCCAC&#xA; 1026099   NGCAATGG    GGCAATGG&#xA; 1016754   NCATTACT    ACATTACT&#xA;  933342   NTTGGGTA    TTTGGGTA&#xA;  119626   TTTGGGGA    TTTGGGTA&#xA;   98657   GTTGGGTA    TTTGGGTA&#xA;   96625   GGCAATGA    GGCAATGG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-10-08T19:20:25.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5180" PostTypeId="1" CreationDate="2018-10-09T09:30:06.873" Score="4" ViewCount="102" Body="&lt;p&gt;I recently read the paper &lt;a href=&quot;https://www.nature.com/articles/s41388-018-0293-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;“A novel long non-coding RNA &lt;em&gt;linc-ZNF469-3&lt;/em&gt; promotes lung metastasis through &lt;em&gt;miR-574-5p-ZEB1&lt;/em&gt; axis in triple negative breast cancer”&lt;/a&gt;. In this I see Table1 showing correlation of Linc-ZNF469-3 with different features in TNBC and non-TNBC patients. p-values are calculated based on Pearson's Chi-sq test.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/puQ8D.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/puQ8D.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For non-TNBC patients: I calculated the p-value with the Chi-sq test just to have a look if everything is right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recurrence:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data &amp;lt;- data.frame(x= c(40,62), y=c(56,65))&#xA;data&#xA;chisq.test(data, correct = F)&#xA;&#xA;    Pearson's Chi-squared test&#xA;&#xA;data:  data&#xA;X-squared = 1.1269, df = 1, p-value = 0.2884&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Brain:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data &amp;lt;- data.frame(x= c(6,99), y=c(6,119))&#xA;data&#xA;chisq.test(data, correct = F)&#xA;&#xA;    Pearson's Chi-squared test&#xA;&#xA;data:  data&#xA;X-squared = 0.096461, df = 1, p-value = 0.7561&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can see that the Recurrence p-value doesn't match with what I see in the table. Others, like Brain, Bone, Liver, Lung match with the p-values I calculated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And for TNBC patients, not a single p-value matches with my calculations. I don't even get Lung and Bone as having significant p-values with my calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did it for Lung as shown here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data &amp;lt;- data.frame(x= c(25,143), y=c(9,35))&#xA;data&#xA;chisq.test(data, correct = F)&#xA;&#xA;    Pearson's Chi-squared test&#xA;&#xA;data:  data&#xA;X-squared = 0.8044, df = 1, p-value = 0.3698&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But in the paper they mentioned:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We found that high linc-ZNF469-3 expression was significantly correlated to lung metastasis in TNBC patients (Table 1).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Can anyone tell me what could be the wrong here? Is it my calculation wrong? or in the paper?&lt;/p&gt;&#xA;" OwnerUserId="1377" LastEditorUserId="29" LastEditDate="2018-10-09T11:06:44.937" LastActivityDate="2018-10-09T13:47:51.733" Title="Why are my Chi-squared test results different from those in a published table?" Tags="|r|rna-seq|statistics|gene-expression|correlation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5181" PostTypeId="2" ParentId="5180" CreationDate="2018-10-09T13:47:51.733" Score="2" Body="&lt;p&gt;Your calculations seem right, perhaps there was an error on their side. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also looked at the number of samples reported, but they use the same amount of samples in each case. Because they are testing whether Linc-ZNF469-3 is different in several tissues, they should have used a multiple test correction for the p-value (although reporting the original values is usually also ok). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question in the comments. You can't correlate a categorical variable with a continuous  variable. Or they did another test or they are not explaining clearly what they did.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-10-09T13:47:51.733" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5182" PostTypeId="1" CreationDate="2018-10-09T18:42:14.490" Score="6" ViewCount="1158" Body="&lt;p&gt;I am trying to rewrite a SAM/BAM file with altered read group entries using pysam. In this simplified version, I want to take a BAM, and rewrite the SM tags in all read groups to the same string and copy the alignments with new header into a new BAM. The contents of the read group dictionaries I am changing appear to have no effect on the header that prints or is written to file output. How can I alter the read groups? &lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;import argparse&#xA;import pysam&#xA;&#xA;def rewrite_read_groups(input_filename, output_filename, sample):&#xA;    try:&#xA;        bam_input = pysam.AlignmentFile(input_filename, 'rb')&#xA;        # replace header fields&#xA;        output_header = bam_input.header&#xA;        read_groups = output_header['RG']&#xA;        print(read_groups)&#xA;        for read_group in read_groups:&#xA;            read_group['SM'] = sample&#xA;            print(read_group)&#xA;        output_header.to_dict()&#xA;        print(output_header)&#xA;    finally:&#xA;        bam_input.close()&#xA;&#xA;    print(output_header)&#xA;&#xA;    try:&#xA;        bam_input = pysam.AlignmentFile(input_filename, 'rb')&#xA;        bam_output = pysam.AlignmentFile(output_filename, 'wb', header=output_header)&#xA;&#xA;        # copy alignments&#xA;        for read in bam_input.fetch(until_eof=True):&#xA;            bam_output.write(read)&#xA;&#xA;    finally:&#xA;        bam_input.close()&#xA;        bam_output.close()&#xA;&#xA;if __name__ == &quot;__main__&quot;:&#xA;    parser = argparse.ArgumentParser(description=&quot;Rewrite bam read groups&quot;)&#xA;    parser.add_argument('-s', &quot;--sample&quot;, help=&quot;Set sample for all read groups&quot;)&#xA;&#xA;    parser.add_argument('-i', &quot;--input_file&quot;, help=&quot;Input bam&quot;, required=True)&#xA;    parser.add_argument('-o', &quot;--output_file&quot;, help=&quot;Output bam&quot;, required=True)&#xA;&#xA;    args = parser.parse_args() &#xA;&#xA;    rewrite_read_groups(args.input_file, args.output_file, args.sample)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="272" LastEditorUserId="298" LastEditDate="2018-10-09T19:20:09.353" LastActivityDate="2019-04-10T00:02:20.443" Title="How do I rewrite a read group using pysam?" Tags="|sam|pysam|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5183" PostTypeId="1" AcceptedAnswerId="5198" CreationDate="2018-10-09T22:08:46.233" Score="2" ViewCount="1312" Body="&lt;h1&gt;Question: Why are there more barcodes than GEMs in 10X chromium data?&lt;/h1&gt;&#xA;&#xA;&lt;h2&gt;Introduction&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I have several 10X genomics chromium libraries &lt;a href=&quot;https://assets.ctfassets.net/an68im79xiti/1Jw6vQfW1GOGuO0AsS2gM8/61866afe8c8af5e0eecf6a3d890f58aa/CG00043_GenomeReagentKitv2UserGuide_RevB.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;made with the de novo assembly/genome protocol&lt;/a&gt;. The 10X chromium device is &lt;a href=&quot;https://community.10xgenomics.com/t5/Genome-Exome-Forum/Question-about-linked-reads-in-smaller-genomes/m-p/58/highlight/true#M9&quot; rel=&quot;nofollow noreferrer&quot;&gt;advertised to produce approximately 1,000,000 GEMs&lt;/a&gt; (gel-emulsion droplets). In theory each GEM should have a single barcode that is used to track the provenance of each read pair. If this is true, a single library should have approximately one million barcodes, whether or not it was overloaded or underloaded.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Results&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The libraries actually have more than one million barcodes, confirmed by &lt;code&gt;longranger basic&lt;/code&gt;. Library 1 has 2,369,949 barcodes. Library 2 has 2,311,405 barcodes. Both libraries have approximately 150,000,000 read pairs.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Why this is a problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I'm using the 10X data for some non-standard analyses and want a high number of &lt;a href=&quot;https://community.10xgenomics.com/t5/Genome-Exome-Forum/Question-about-linked-reads-in-smaller-genomes/m-p/58/highlight/true#M9&quot; rel=&quot;nofollow noreferrer&quot;&gt;LinkedReads per molecule (LPM)&lt;/a&gt;. Having multiple barcodes in a GEM will reduce the number of LPMs per barcode proportional to how many barcodes were present in the GEM.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Possible explanations&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If there are more than approximately one million barcodes in a library then this leaves several possibilities:&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Each GEM truly has a single barcode. Therefore the Chromium device actually produced significantly more GEMs than advertised.&lt;/li&gt;&#xA;&lt;li&gt;Each GEM has a variable number of barcodes. Therefore the number of barcodes is independent of how many GEMs are created.&lt;/li&gt;&#xA;&lt;li&gt;Some combination of the above two options. Some variable number of GEMs and a variable number of GEMs per barcode.&lt;/li&gt;&#xA;&lt;li&gt;Additional barcodes are created by sequencing errors and misclassification by &lt;code&gt;longranger basic&lt;/code&gt;. This would probably manifest in many barcodes that only have one or two linked reads total.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Deeper analysis&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I don't know how to test hypotheses 1, 2, &amp;amp; 3 bioinformatically. Hypothesis 4 is testable though by making a histogram of read counts per barcode. To do so, we will use the output of &lt;code&gt;longranger basic&lt;/code&gt; for Lib2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Longranger basic outputs an interleaved &lt;code&gt;fastq.gz&lt;/code&gt; file that contains the whitelisted barcode in the header. For example (pretending the reads are shorter than 150bp):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@J00113:232:HFWCKBBXX:4:2224:17756:14889 BX:Z:AAACACCAGACAATAC-1&#xA;AATTACCCAAAATTAATATTTGTATTAATTTTCATATTAATAAC&#xA;+&#xA;JJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJ&#xA;@J00113:232:HFWCKBBXX:4:2224:17756:14889 BX:Z:AAACACCAGACAATAC-1&#xA;CACAATATCAAAAACAAATCAAAAATTAAGAAAATATAACTAAG&#xA;+&#xA;AAF&amp;lt;7FF-JJFJJFJJJJJJAJFJJJFJJJJJFJJJJJJJJJJJ&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;BX:Z:AAACACCAGACAATAC-1&lt;/code&gt; &lt;a href=&quot;https://support.10xgenomics.com/genome-exome/software/pipelines/latest/output/bam&quot; rel=&quot;nofollow noreferrer&quot;&gt;tells us that the index is whitelisted&lt;/a&gt; (&lt;code&gt;BX:Z&lt;/code&gt;), &lt;a href=&quot;https://support.10xgenomics.com/genome-exome/software/pipelines/latest/output/bam&quot; rel=&quot;nofollow noreferrer&quot;&gt;the index sequence&lt;/a&gt; (&lt;code&gt;AAACACCAGACAATAC&lt;/code&gt;), &lt;a href=&quot;https://support.10xgenomics.com/genome-exome/software/pipelines/latest/output/bam&quot; rel=&quot;nofollow noreferrer&quot;&gt;and the library index for the analysis&lt;/a&gt; (&lt;code&gt;-1&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, we can count the number of occurrences of each barcode with this code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bioawk -cfastx '{print($comment)}' barcoded.fastq.gz | sort | uniq -c &amp;gt; bc_counts.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first column of &lt;code&gt;bc_counts.txt&lt;/code&gt; tells us how many individual reads had that barcode. We divide this number by two to see how many read pairs had that barcode and make a histogram.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{print(&lt;span class=&quot;math-container&quot;&gt;$1)/2}' barcode_counts.txt | sort | uniq -c | sort -k2 -n | awk '{printf(&quot;%d\t%d\n&quot;, $2, $&lt;/span&gt;1)}' &amp;gt; pairs_per_bc.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first column of this file has the number of read pairs in the barcode, and the second column has how many barcodes have that number of read pairs. We see that there are many barcodes that only have one or two read pairs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1       593583&#xA;2       207192&#xA;3       79409&#xA;4       34174&#xA;5       17353&#xA;6       10662&#xA;7       7997&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pVVuG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pVVuG.png&quot; alt=&quot;lib2&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like every barcode with fewer than 10 read pairs may be noise. How many barcodes contain at least 10 reads?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;awk '{if (&lt;span class=&quot;math-container&quot;&gt;$1 &amp;gt;= 10){sum += $&lt;/span&gt;2} } END{print(sum)}' pairs_per_bc.txt&#xA;1347686&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Around one million. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Conclusion&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This analysis doesn't give conclusive evidence for the four hypotheses above, but it seems like there is a noise-generating process in 10X data that causes there to be many barcodes with only a few readpairs. &lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-10-11T10:51:02.103" Title="Why are there more barcodes than GEMs in 10X chromium data?" Tags="|ngs|illumina|10x-genomics|barcode|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5185" PostTypeId="1" CreationDate="2018-10-10T02:32:39.557" Score="5" ViewCount="1469" Body="&lt;p&gt;I tried to use the following command&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;uniq -u reference.fasta &amp;gt;&amp;gt; reference_uniq.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'd like a count of the unique headers.&lt;/p&gt;&#xA;" OwnerUserId="3478" LastEditorUserId="298" LastEditDate="2018-10-10T08:22:51.497" LastActivityDate="2018-10-11T07:58:07.463" Title="finding unique headers in a fasta file using linux command line" Tags="|fasta|linux|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5187" PostTypeId="2" ParentId="5185" CreationDate="2018-10-10T03:14:54.157" Score="5" Body="&lt;p&gt;If you just want the number of unique headers, you can do this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep '&amp;gt;' reference.fasta | sort | uniq | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want a list of the unique headers, you can do this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep '&amp;gt;' reference.fasta | sort | uniq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want a histogram of how many times each header occurs, you can do this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep '&amp;gt;' reference.fasta | sort | uniq -c | awk '{printf(&quot;%s\t%s\n&quot;, &lt;span class=&quot;math-container&quot;&gt;$1, $&lt;/span&gt;2)}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="298" LastEditDate="2018-10-10T08:24:08.857" LastActivityDate="2018-10-10T08:24:08.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5188" PostTypeId="2" ParentId="5185" CreationDate="2018-10-10T04:19:22.000" Score="3" Body="&lt;p&gt;The &lt;code&gt;uniq&lt;/code&gt; command expects sorted input. Interestingly, the &lt;code&gt;sort&lt;/code&gt; command actually has a &quot;unique&quot; option, &lt;code&gt;-u&lt;/code&gt;, which means &lt;code&gt;uniq&lt;/code&gt; is not strictly needed. For the fastest processing, you can look for the '&gt;' character at the start of lines with grep:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep '^&amp;gt;' reference.fasta | sort -u &amp;gt; reference_headers_unique.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For returning the &lt;em&gt;number&lt;/em&gt; of unique lines, pipe through &lt;code&gt;wc -l&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep '^&amp;gt;' reference.fasta | sort -u | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For more information about regular expressions, see &lt;a href=&quot;https://www.gnu.org/software/findutils/manual/html_node/find_html/grep-regular-expression-syntax.html#grep-regular-expression-syntax&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-10-10T04:19:22.000" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5189" PostTypeId="1" AcceptedAnswerId="5199" CreationDate="2018-10-10T07:28:41.963" Score="1" ViewCount="203" Body="&lt;p&gt;I believe many of the experts here have gone through this issue,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have run tRNAscan-SE on my genome to predict all the tRNAs, and have successfully gotten all the tRNA coordinates like start, end and type all  arranged in a table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how can I convert this result to fasta format as long I have all the details about the tRNA ready?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any tool that I can use to get the fasta sequences of the regions described in my table from the original genome?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8JAdp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8JAdp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sequence                tRNA    Bounds  tRNA    Anti    Intron Bounds   Inf       &#xA;Name            tRNA #  Begin   End     Type    Codon   Begin   End     Score   Note&#xA;--------        ------  -----   ------  ----    -----   -----   ----    ------  ------&#xA;NW_011590950.1  1       36704   36632   Ala     AGC     0       0       40.8    pseudo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3332" LastEditorUserId="298" LastEditDate="2018-10-11T08:01:54.037" LastActivityDate="2018-10-11T12:13:28.893" Title="convert tRNA coordinates Result to Fasta format" Tags="|fasta|trnascan-se|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="5191" PostTypeId="2" ParentId="5158" CreationDate="2018-10-10T10:35:24.250" Score="1" Body="&lt;p&gt;If I understand your question correctly, you are trying to present the tSNE plot coloring and labeling the cells representations based on the original sample ID, i.e. which Seurat object it was from. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think they offered a vignette on that. Please check this link. &lt;a href=&quot;https://satijalab.org/seurat/merge_vignette.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/merge_vignette.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, you can add the sample name for the cells in the argument option in &lt;code&gt;MergeSeurat&lt;/code&gt; function. Please check the documentation on &lt;code&gt;MergeSeurat&lt;/code&gt; function as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I did this, I have a lot of samples to be merged, so I simply saved the sample info for every cell in each sample to different objects. You can also use &lt;code&gt;AddMetadata&lt;/code&gt; to do it as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps. &lt;/p&gt;&#xA;" OwnerUserId="3480" LastActivityDate="2018-10-10T10:35:24.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5192" PostTypeId="2" ParentId="5185" CreationDate="2018-10-10T12:57:22.167" Score="1" Body="&lt;p&gt;If, for some reason, you need to keep the original order of the input file, so you only want the &lt;em&gt;first&lt;/em&gt; occurrence of each header, you can do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '/^&amp;gt;/ &amp;amp;&amp;amp; !a[$0]++' reference.fasta &amp;gt; headers&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, to get the number only:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '/^&amp;gt;/ &amp;amp;&amp;amp; !a[$0]++{k++}; END{print k}' reference.fasta &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, these only make sense if you need to keep the order of the original file. If not, use &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/5188/298&quot;&gt;Gringer's &lt;code&gt;grep&lt;/code&gt; approach&lt;/a&gt; which will be much faster (&lt;code&gt;reference.fasta&lt;/code&gt; is hg19):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ time awk '/^&amp;gt;/ &amp;amp;&amp;amp; !a[$&lt;/span&gt;0]++{k++}; END{print k}' reference.fasta &#xA;93&#xA;&#xA;real    0m34.510s&#xA;user    0m31.861s&#xA;sys     0m2.634s&#xA;&#xA;$ time grep '^&amp;gt;' reference.fasta | sort -u | wc -l&#xA;93&#xA;&#xA;real    0m16.597s&#xA;user    0m0.820s&#xA;sys     0m3.725s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-10-11T07:58:07.463" LastActivityDate="2018-10-11T07:58:07.463" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5193" PostTypeId="1" CreationDate="2018-10-10T14:15:17.523" Score="3" ViewCount="354" Body="&lt;h3&gt;BACKGROUND&lt;/h3&gt;&#xA;&lt;p&gt;I work on NGS data (illumina paired ends reads) coming from a full extract of RNA (metagenomic). We are interested in the viral fraction of this extract.&lt;/p&gt;&#xA;&lt;p&gt;I observed a contamination with a PCR amplicon. This contamination has been identified as coming from a PCR result that we used to make near the library preparation. So I know exactly the primers used in the PCR.&lt;/p&gt;&#xA;&lt;p&gt;Here you can see a tablet visualization illustrating the contamination (2 contaminations). I obtained that image with a mapping on a known reference of my virus. &lt;a href=&quot;https://i.stack.imgur.com/92NDo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/92NDo.png&quot; alt=&quot;twin towers and world trade center&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h3&gt;PROBLEM&lt;/h3&gt;&#xA;&lt;p&gt;The problem I have is that these PCR products are exactly the type of sequence I want from my extract (identifying part of the virus). So I have some &amp;quot;false positive&amp;quot; and some misidentification of viruses present in the extract.&lt;/p&gt;&#xA;&lt;p&gt;I have thought about removing all reads containing primer sequences but it leads to a gap in my data and I can't identify the viruses in my extract anymore.&#xA;I am thinking that all of the read pairs that begin before or end after my primers are &amp;quot;true-positives&amp;quot; and I would really like to keep them (even having the primer sequence in them).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eIBzT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eIBzT.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Do you know any tool that can help me in that situation?&lt;/strong&gt; i.e. removing reads containing a sequence but only if it begins with it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;If the tool can also handle the degeneration of my primer it would be perfect.&lt;/em&gt;&lt;br /&gt;&#xA;An example of primer I used is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CNTGGGAGGGCGATCGCAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Complementary information (asked in comments)&lt;/strong&gt;&lt;br /&gt;&#xA;I have around 30 million read pairs (60 millions reads in total). Contamination reads are between 200.000 to a 1 millions reads (depending on sample). &amp;quot;true&amp;quot; reads (not from the contamination) represent around 0.5% of the contamination.&lt;/p&gt;&#xA;" OwnerUserId="3482" LastEditorUserId="3848" LastEditDate="2022-03-11T17:31:37.610" LastActivityDate="2022-06-09T13:50:31.020" Title="Tool to remove a PCR contamination in NGS data" Tags="|rna-seq|ngs|covid-19|pcr|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5194" PostTypeId="2" ParentId="5158" CreationDate="2018-10-10T20:03:55.893" Score="3" Body="&lt;p&gt;Below a few lines of code that accompany BC Wang's answer.&#xA;After using MergeSeurat the sample name will be added to meta data under orig.ident. this can then be used to color the tSNE either using group.by or pt.shape. The former will show colors for each sample, the latter will color each cluster and give sample id another shape.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;path1 &amp;lt;- file.path(&quot;path to 10x sample 1&quot;)&#xA;path2 &amp;lt;- file.path(&quot;path to 10x sample 2&quot;)&#xA;&#xA;S1 &amp;lt;- Read10X(data.dir = path1)&#xA;S2 &amp;lt;- Read10X(data.dir = path2)&#xA;S1 &amp;lt;- CreateSeuratObject(raw.data = S1, min.cells = 3, project = &quot;S1&quot;)&#xA;S2 &amp;lt;- CreateSeuratObject(raw.data = S2, min.cells = 3, project = &quot;S2&quot;)&#xA;&#xA;s.combined &amp;lt;- MergeSeurat(object1 = S1, object2 = S2, add.cell.id1 = &quot;S1&quot;, &#xA;            add.cell.id2 = &quot;S2&quot;, project = &quot;2samples&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;All data analysis goes here:&#xA;RunTSNE etc...&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Plot the data&lt;/h2&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TSNEPlot(s.combined, group.by =&quot;orig.ident&quot;)&#xA;TSNEPlot(s.combined, pt.shape=&quot;orig.ident&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2675" LastActivityDate="2018-10-10T20:03:55.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5195" PostTypeId="2" ParentId="5193" CreationDate="2018-10-10T22:38:16.200" Score="2" Body="&lt;p&gt;This bash script will get you all of the reads that do not start with your primer sequence. It also handles degeneracy. It requires the tool &lt;a href=&quot;https://github.com/lh3/bioawk&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioawk&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;It works by looking at the first bases in the read and only prints out the headers that do not have a match with the degenerate primer. Then, &lt;code&gt;comm&lt;/code&gt; makes sure that the read was retained in both the R1 and R2 filtering. This allows reads to be filtered whether or not the primers start at the beginning of the R1 or R2 read.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;seqtk subseq&lt;/code&gt; is where you recover the reads that don't start with the primer.&lt;/p&gt;&#xA;&lt;p&gt;Be warned though, any off-by-one or sequencing errors will allow some of the sequences with primers at the beginning. Also, this script requires that the headers for the R1 and R2 reads are the same and aren't appended with anything like &amp;quot;\1&amp;quot;, &amp;quot;\2&amp;quot;, et cetera.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;get_primer_free_starts () {&#xA;    bioawk -cfastx 'BEGIN{seqs[0] = &amp;quot;CATGGGAGGGCGATCGCAA&amp;quot;; seqs[1] = &amp;quot;CCTGGGAGGGCGATCGCAA&amp;quot;; \&#xA;        seqs[2] = &amp;quot;CGTGGGAGGGCGATCGCAA&amp;quot;; seqs[3] = &amp;quot;CTTGGGAGGGCGATCGCAA&amp;quot;} \&#xA;        {thisseq = substr(&lt;span class=&quot;math-container&quot;&gt;$seq, 1, length(seqs[0])); printme=1\&#xA;        for (i=0; i&amp;lt;=3; i++) { \&#xA;            if (thisseq != seqs[i]){ \&#xA;                printme=0 \&#xA;            }&#xA;        }; if (printme==1){ \&#xA;            print($seq)&#xA;        }}' &quot;$&lt;/span&gt;{INPUT} | sort &amp;gt; &amp;quot;${OUTPUT}&#xA;}&#xA;&#xA;INPUT=&amp;quot;R1.fastq.gz&amp;quot;&#xA;OUTPUT=&amp;quot;R1_OK.txt&amp;quot;&#xA;get_primer_free_starts&#xA;&#xA;INPUT=&amp;quot;R2.fastq.gz&amp;quot;&#xA;OUTPUT=&amp;quot;R2_OK.txt&amp;quot;&#xA;get_primer_free_starts&#xA;&#xA;comm -12 R1_OK.txt R2_OK.txt | sort | uniq &amp;gt; passing_reads.txt&#xA;&#xA;seqtk subseq R1.fastq.gz passing_reads.txt &amp;gt; R1.filtered.fq&#xA;seqtk subseq R2.fastq.gz passing_reads.txt &amp;gt; R2.filtered.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2022-06-09T13:50:31.020" LastActivityDate="2022-06-09T13:50:31.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5198" PostTypeId="2" ParentId="5183" CreationDate="2018-10-11T10:51:02.103" Score="3" Body="&lt;p&gt;Your initial guess is almost certainly correct. I don't know about the linked read libraries, but in the 10X single cell sequencing protocol, separating real barcodes from noise barcodes is an important and sensitive step in the analysis pipeline. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current analysis pipeline is to look at a plot, like the one you produced and find the &quot;knees&quot;. The plot you produced has two knees, on at around 10 and one at around 200. I'd guess that the ones below 10 are what we call &quot;error barcodes&quot; generated by sequencing is PCR errors from real barcodes, and the ones between 10 and 200 are what we'd call &quot;background&quot; barcodes - in scRNAseq they represent things like GEMs that didn't encapsulate a cell, but did encapsulate some non-cellular DNA or RNA. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As there are more than 1,000,000 possible barcodes, not all of them are used.  I believe 10X knows which sequences are used and which are not (ie the barcodes are not random), so I'm surprised that these sequences make it out of the other end of the analysis. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-10-11T10:51:02.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5199" PostTypeId="2" ParentId="5189" CreationDate="2018-10-11T10:52:58.600" Score="3" Body="&lt;p&gt;You can select “Output BED format” on the tRNAscan-SE server:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5Lcv1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5Lcv1.png&quot; alt=&quot;screenshot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the output BED format and your input reference in FASTA format, you can easily generate a FASTA file of tRNA sequences via &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/getfasta.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;bedtools getfasta&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools getfasta -name -s -fo ‹output.fasta› -fi ‹reference.fasta› -bed ‹tRNAscan-SE-output.bed›&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29" LastActivityDate="2018-10-11T10:52:58.600" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5200" PostTypeId="2" ParentId="5165" CreationDate="2018-10-11T11:12:25.207" Score="1" Body="&lt;p&gt;It turns out, after speaking to tech support, that these are old files from a previous version of the software so won't run through variant caller.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The files themselves need reanalysing from the initial FASTQ -&gt; BAM with the newest version.&lt;/p&gt;&#xA;" OwnerUserId="2244" LastActivityDate="2018-10-11T11:12:25.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5201" PostTypeId="1" CreationDate="2018-10-11T11:40:27.440" Score="4" ViewCount="157" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We want to be able to load (or request) data for a genome including the sequence and gene annotation (bacteria). Then, we want to load our own annotation which should be displayed as a line plot:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;position score&#xA;1        5&#xA;2        6&#xA;etc.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where each position corresponds to the position in the genome (i.e. FASTA).&#xA;Something like this: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QWfgP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QWfgP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a tool that can load a genome sequence, gene annotation, and a position ~ score file, which can be displayed as a line plot? &lt;/p&gt;&#xA;" OwnerUserId="2746" LastEditorUserId="298" LastEditDate="2018-10-11T15:41:36.643" LastActivityDate="2018-10-12T19:24:45.873" Title="Tool to show DNA sequence and allowing upload of own graph data" Tags="|genome|sequence-analysis|visualization|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5202" PostTypeId="2" ParentId="5189" CreationDate="2018-10-11T12:13:28.893" Score="1" Body="&lt;p&gt;You can do this using the tools from the &lt;a href=&quot;https://www.ebi.ac.uk/about/vertebrate-genomics/software/exonerate&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;exonerate&lt;/code&gt; package&lt;/a&gt;. I assume you have all the sequences shown in your table in a single, multifasta file. If so, first create a single file per sequence using &lt;code&gt;fastaexplode&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fastaexplode reference.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will create one file for each sequence in &lt;code&gt;reference.fa&lt;/code&gt;. Now, you can read through the results table, and extract the sequences:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tmpFile=&lt;span class=&quot;math-container&quot;&gt;$(mktemp)&#xA;awk -F&quot;\t&quot; '{ &#xA;                if(NR&amp;gt;3){ printf &quot;%s %s &quot;, $1, $2; &#xA;                    if($4&amp;gt;$3){&#xA;                        print $3, $4-$3&#xA;                    }&#xA;                    else{&#xA;                        print $4,$3-$4,&quot;-&quot;&#xA;                    }&#xA;                }&#xA;            }' file | while read name num start len rev; do &#xA;                fastasubseq &quot;$&lt;/span&gt;name&quot;.fa &lt;span class=&quot;math-container&quot;&gt;$start $&lt;/span&gt;len &amp;gt; &lt;span class=&quot;math-container&quot;&gt;$num.fa&#xA;                if [[ ! -z $&lt;/span&gt;rev ]]; then &#xA;                    fastarevcomp &lt;span class=&quot;math-container&quot;&gt;$num.fa &amp;gt; $&lt;/span&gt;tmpFile &amp;amp;&amp;amp; &#xA;                    mv &lt;span class=&quot;math-container&quot;&gt;$tmpFile $&lt;/span&gt;num.fa&#xA;                fi&#xA;                done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would suggest you go with &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/5199/298&quot;&gt;Konrad's approach&lt;/a&gt;, however, that looks much cleaner and simpler.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-10-11T12:13:28.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5203" PostTypeId="2" ParentId="5182" CreationDate="2018-10-11T22:45:54.400" Score="2" Body="&lt;p&gt;If the goal is to assign all alignments to a single read group (which this code seems to do), then Picard AddOrReplaceReadGroups might help: &lt;a href=&quot;https://broadinstitute.github.io/picard/command-line-overview.html#AddOrReplaceReadGroups&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://broadinstitute.github.io/picard/command-line-overview.html#AddOrReplaceReadGroups&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Modifying headers using PySAM might be tricky – the &lt;a href=&quot;https://github.com/pysam-developers/pysam/issues/618&quot; rel=&quot;nofollow noreferrer&quot;&gt;internal representation of headers&lt;/a&gt; has recently been changed and seems not to be stable across PySAM versions.&lt;/p&gt;&#xA;" OwnerUserId="425" LastEditorUserId="425" LastEditDate="2018-10-11T22:54:02.530" LastActivityDate="2018-10-11T22:54:02.530" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5204" PostTypeId="2" ParentId="5168" CreationDate="2018-10-12T12:44:56.740" Score="2" Body="&lt;p&gt;I'm not sure Seurat is the best tool for this as it was developed for single cell RNA seq data and there are a few intricacies of that type of data that are very different from bulk RNA seq. &#xA;For bulk there are really good packages available and corresponding workflows, e.g. &lt;a href=&quot;https://master.bioconductor.org/packages/release/workflows/html/RNAseq123.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;limma&lt;/a&gt;, &lt;a href=&quot;https://master.bioconductor.org/packages/release/workflows/html/RnaSeqGeneEdgeRQL.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;edgeR&lt;/a&gt; and &lt;a href=&quot;https://master.bioconductor.org/packages/release/workflows/html/rnaseqGene.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main problems with Seurat for bulk RNA-seq:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Seurat expects counts as input - FPKM are not counts nor are they log counts or log norm counts. It's a specific normalisation method that takes into account gene length and library size and breaks the link between gene counts and variance. &lt;/li&gt;&#xA;&lt;li&gt;Because of the distribution of reads from a single cell that you get from the sequencer Seurat scales all unique reads (i.e. UMIs) to 10,000 per cell and usually regresses out the nUMI count. Again, becuase you've got FPKMs this makes no sense.&lt;/li&gt;&#xA;&lt;li&gt;The other functions you can use on bulk data, i.e. tSNE, finding clusters and markers (which is essentially DGE analysis). The one thing to note for the latter is that again because you've got FPKMs they don't follow the negative binomial distribution. Your best bet is limma trend but I'm not sure it's included in Seurat. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3498" LastEditorUserId="57" LastEditDate="2018-10-12T15:41:47.740" LastActivityDate="2018-10-12T15:41:47.740" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5205" PostTypeId="2" ParentId="5201" CreationDate="2018-10-12T19:24:45.873" Score="1" Body="&lt;p&gt;You can use UCSC genome browser, which supports custom genome sequence and annotation. Specifically, one needs to create a &lt;a href=&quot;https://genome.ucsc.edu/goldenpath/help/hgTrackHubHelp.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Track Hub&lt;/a&gt; to hold genomes and annotations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;http://genome.ucsc.edu/cgi-bin/hgTracks?hubUrl=http://zlab-trackhub.umassmed.edu/yfu/insects/hub.txt&amp;amp;db=hub_129615_cl_v0.8.2&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genome.ucsc.edu/cgi-bin/hgTracks?hubUrl=http://zlab-trackhub.umassmed.edu/yfu/insects/hub.txt&amp;amp;db=hub_129615_cl_v0.8.2&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And here is the corresponding &lt;a href=&quot;http://zlab-trackhub.umassmed.edu/yfu/insects/hub.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;hub.txt&lt;/a&gt; file holding the metadata, and URLs to actual fasta, bigWig and bigBed files.&lt;/p&gt;&#xA;" OwnerUserId="3504" LastActivityDate="2018-10-12T19:24:45.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5206" PostTypeId="2" ParentId="4867" CreationDate="2018-10-13T11:57:33.620" Score="0" Body="&lt;p&gt;I think the reason for your confusion is vocabulary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The keywords you search for are genome assembly (see @Devon Ryan's answer) a process that is trying to stick together reads to recreate the sequenced genomic sequence. The assembled sequence needs to be annotated if you are interested where are the genes or what they could possibly do. If you will search for tutorials using these keywords you will find plenty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure why you need it, but &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Sitemap/samplerecord.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genbank format&lt;/a&gt; contains the genome (the product of genome assembly), metadata (who is the submitter, what species it is, etc) and genome annotation (what is in the region of genome).&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2018-10-13T11:57:33.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5207" PostTypeId="1" CreationDate="2018-10-13T12:36:24.900" Score="2" ViewCount="986" Body="&lt;p&gt;I am comparing the gene expression of 2 bacteria under 1 condition. I have now the count tables for 3 tech. replicates for each bacteria.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Bacteria1_1.count&#xA;Bacteria1_2.count&#xA;Bacteria1_3.count&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;...same for the other bacteria.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These files look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gene1 10000&#xA;gene2 500&#xA;gene3 0&#xA;gene4 5000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to use DESeq2 for differential gene expression analysis. But I cannot figure out how to properly execute the DESeqDataSetFromHTSeqCount() command with this type of data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there another intermediate step to add?&lt;/p&gt;&#xA;" OwnerUserId="3507" LastEditorUserId="298" LastEditDate="2018-10-13T13:08:19.123" LastActivityDate="2019-09-12T20:00:50.103" Title="How to input data for DESeq2 from individual HTSeq count?" Tags="|rna-seq|deseq2|hts|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5208" PostTypeId="1" AcceptedAnswerId="5217" CreationDate="2018-10-13T15:13:53.227" Score="4" ViewCount="358" Body="&lt;p&gt;I'm participating in a bioinformatics machine-learning seminar at my university. The main task is predicting binary classification of protein-protein interactions using sequence data as input. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the subtasks is familiarization with the dataset and presenting the dataset. Now I'm wondering which additional information I can get out of the sequence data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to start out with binary classification for the protein interaction with the labels “Interact” and “Non-Interact. The dataset that I got provided consists of two fasta files. One containing the sequence header with the species and the corresponding amino-acid sequences ~300 entries. The other containing the exact same species header and the label “Interact” or &quot;Non-interact”. Species are completely mixed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there something interesting that I can additionally include in my analysis? I'm used to exploratory data analysis though this is different due to the nature of the protein sequences.&lt;/p&gt;&#xA;" OwnerUserId="3510" LastEditorUserId="96" LastEditDate="2019-01-17T14:24:08.457" LastActivityDate="2019-01-17T14:24:08.457" Title="Machine learning using protein-sequences" Tags="|machine-learning|protein-protein-interaction|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5209" PostTypeId="1" CreationDate="2018-10-13T19:21:18.750" Score="2" ViewCount="555" Body="&lt;p&gt;Looking for some guidance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In an introductory bioinformatics class I am tasked with doing the following in R:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Create the sequence ACTGACTG&lt;/li&gt;&#xA;&lt;li&gt;Generate a random string of nucleotides with equal probabilities.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Are there certain functions that I should be looking at? No library or package or set of functions was provided so I am completely lost. I am assuming there must be functions in R that do these things that I need to manipulate, but I cannot seem to find any resources explaining that. I looked at seq() (thinking it might generate DNA sequences), but it only seems to generate sequences of integers. I also looked at randDNA(), but it is random, how could I make it generate the exact string in question 1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any direction will be greatly appreciated.&#xA;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="3511" LastEditorUserId="73" LastEditDate="2018-10-13T23:45:03.437" LastActivityDate="2018-10-13T23:45:03.437" Title="Generating basic dna sequences in R" Tags="|genome-sequencing|homework|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5210" PostTypeId="2" ParentId="5209" CreationDate="2018-10-13T19:54:54.293" Score="1" Body="&lt;p&gt;Without giving away too much for your homework check out the &lt;a href=&quot;http://www.rexamples.com/14/Sample()&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;sample()&lt;/code&gt; function in R&lt;/a&gt;. Think about how you can pass DNA bases to this function to generate random sequences. &lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-10-13T19:54:54.293" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5211" PostTypeId="2" ParentId="4637" CreationDate="2018-10-14T06:46:41.293" Score="5" Body="&lt;p&gt;An easy way to build tRNAscan and their dependencies &quot;all Infernal packages&quot; without confusion is to run&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c bioconda trnascan-se &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3332" LastEditorUserId="964" LastEditDate="2018-10-14T14:02:07.053" LastActivityDate="2018-10-14T14:02:07.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5212" PostTypeId="1" CreationDate="2018-10-15T05:19:29.380" Score="2" ViewCount="92" Body="&lt;p&gt;I have a string, a list of substrings, and a list of positions where substrings are found in string.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;string = 'ATGGTACACGCTACGAGCTAG'&#xA;substrings = ['TACA', 'CTAGAAAAG']&#xA;positions = ['11, 4', '']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Substring &lt;code&gt;'TACA'&lt;/code&gt; appears at positions 11 and 4 in string (i.e. &lt;code&gt;'11, 4'&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Substring &lt;code&gt;'CTAGAAAAG'&lt;/code&gt; does not appear in string (i.e. &lt;code&gt;''&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to remove &lt;code&gt;'CTAGAAAAG'&lt;/code&gt; from &lt;code&gt;substrings&lt;/code&gt; and I need to remove &lt;code&gt;''&lt;/code&gt; from &lt;code&gt;positions&lt;/code&gt;, while creating a new list with &lt;code&gt;'CTAGAAAAG'&lt;/code&gt;. &lt;strong&gt;I need your help here to create code which will give the following results:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;new_substrings = ['TACA']&#xA;new_positions = ['11, 4']&#xA;new_list = ['CTAGAAAAG']&#xA;string = 'ATGGTACACGCTACGAGCTAG' (obtained from above)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Once this is complete, the &lt;strong&gt;given&lt;/strong&gt; code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;new_positions = [map(int, x.split(',')) for x in new_positions]&#xA;seq_pos = dict(zip(new_substrings,new_positions))&#xA;seq_ref = {k: [string[start:start+length] for start in starts] for k, starts, length in zip(new_substrings,new_positions,[len(x) for x in new_substrings])}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Gives:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seq_pos = {'TACA': [11, 4]}&#xA;seq_ref = {'TACA': ['TACG', 'TACA']}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, from seq_ref, I need to create a list of integers which represents the number of mismatches between the key: &lt;code&gt;'TACA'&lt;/code&gt; and both of its values: &lt;code&gt;'TACG'&lt;/code&gt; and &lt;code&gt;'TACA'&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mismatches = [[1, 0]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The code to find the # of mismatches between two strings is provided:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def HammingDistance(p, q):&#xA;    dist = 0&#xA;    for i in range(len(p)):&#xA;        if p[i] != q[i]:&#xA;            dist += 1&#xA;    return dist&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, I need to save my data in a CSV file (i.e. Excel or Numbers).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;new_positions = ['11, 4'] (obtained above)&#xA;new_substrings = ['TACA'] (obtained above)&#xA;actual_substrings = [['TACG', 'TACA']] (values of seq_ref)&#xA;mismatches = [[1, 0]] (obtained above)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Info needs to be sorted based on # of mismatches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LdAa1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LdAa1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand this is a multi-step problem but any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="3515" LastEditorUserId="298" LastEditDate="2018-10-15T08:15:46.283" LastActivityDate="2018-10-15T14:43:14.977" Title="Store contigs and genome substrings based on mismatches" Tags="|python|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5215" PostTypeId="2" ParentId="5212" CreationDate="2018-10-15T14:43:14.977" Score="1" Body="&lt;p&gt;For the first question try something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;new_substrings = []&#xA;new_positions = []&#xA;new_list = []&#xA;for pos, sub in zip(positions, substrings): &#xA;   if not pos:&#xA;       new_list.append(sub)&#xA;   else:&#xA;       new_substrings.append(sub)&#xA;       new_positions.append(pos)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;this gives:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; new_substrings&#xA;['TACA']&#xA;&amp;gt;&amp;gt;&amp;gt; new_positions&#xA;['11, 4']&#xA;&amp;gt;&amp;gt;&amp;gt; new_list&#xA;['CTAGAAAAG']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For the hamming distance you just need to iterate over the values in &lt;code&gt;seq_ref&lt;/code&gt; and append the results of the hamming distance function in &lt;code&gt;mismatches&lt;/code&gt;.  Based on the given code and the example above you should be able to create that block.  Below is an untested quick example to get you on track.  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mismatches = []&#xA;for seq in seq_ref.keys():&#xA;   for value in seq_ref[seq]:&#xA;       mismatches.append(HammingDistance(seq, value))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As far as the CSV and sorting is concerned I suggest that you look into pandas or numpy.  Both of those packages provided functions to help with this.&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2018-10-15T14:43:14.977" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5216" PostTypeId="1" CreationDate="2018-10-15T14:46:10.743" Score="2" ViewCount="94" Body="&lt;p&gt;I'm trying to convert coordinates between the reference mouse genome and CAST/EiJ mice strain. Since Ensebml already has alignment available in the browser, I decided to use Ensembl perl API to lift over the alignment for the regions I'm interested in. The problem is that it doesn't seem to be working. I'm getting following error message: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-------------------- EXCEPTION --------------------&#xA;MSG: The database do not contain any CACTUS_HAL data for Bio::EnsEMBL::Compara::GenomeDB=HASH(0x557db0d661e8)!&#xA;STACK toplevel coord.pl:42&#xA;Date (localtime)    = Mon Oct 15 07:42:17 2018&#xA;Ensembl API version = 94&#xA;---------------------------------------------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which I assume means that I'm either searching for the wrong alignment record or the alignment itself is not available. Following is the perl script which I modified from the Ensembl Compara API &lt;a href=&quot;https://www.ensembl.org/info/docs/api/compara/compara_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;use strict;&#xA;use Bio::EnsEMBL::Registry;&#xA;use Bio::EnsEMBL::Utils::Exception qw(throw);&#xA;use Bio::AlignIO;&#xA;use Getopt::Long;&#xA;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$species = &quot;murinae&quot;;&#xA;my $&lt;/span&gt;targetSpecies = &quot;mus_musculus_casteij&quot;;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$alignment_type = &quot;CACTUS_HAL&quot;;&#xA;my $&lt;/span&gt;output_file = undef;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$coord_system = &quot;chromosome&quot;;&#xA;my $&lt;/span&gt;seq_region = &quot;14&quot;;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$seq_region_start = 5000000;&#xA;my $&lt;/span&gt;seq_region_end = 5010000;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$output_format = &quot;clustalw&quot;;&#xA;my $&lt;/span&gt;genome_dbs;&#xA;&#xA;use Bio::EnsEMBL::Registry;&#xA;Bio::EnsEMBL::Registry-&amp;gt;load_registry_from_db(&#xA;    -host =&amp;gt; 'ensembldb.ensembl.org',&#xA;    -user =&amp;gt; 'anonymous',&#xA;    -port =&amp;gt; 5306);&#xA;&#xA;my $genome_db_adaptor = Bio::EnsEMBL::Registry-&amp;gt;get_adaptor(&#xA;    'Multi', 'compara', 'GenomeDB');&#xA;&#xA;throw(&quot;Cannot connect to Compara&quot;) if (!$genome_db_adaptor);&#xA;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$this_genome_db = $&lt;/span&gt;genome_db_adaptor-&amp;gt;fetch_by_name_assembly(&lt;span class=&quot;math-container&quot;&gt;$species);&#xA;my $&lt;/span&gt;target_genome_db = &lt;span class=&quot;math-container&quot;&gt;$genome_db_adaptor-&amp;gt;fetch_by_name_assembly($&lt;/span&gt;targetSpecies);&#xA;push(@&lt;span class=&quot;math-container&quot;&gt;$genome_dbs, $&lt;/span&gt;this_genome_db);&#xA;push(@&lt;span class=&quot;math-container&quot;&gt;$genome_dbs, $&lt;/span&gt;target_genome_db);&#xA;&#xA;my $method_link_species_set_adaptor = Bio::EnsEMBL::Registry-&amp;gt;get_adaptor(&#xA;    'Multi', 'compara', 'MethodLinkSpeciesSet');&#xA;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$method_link_species_set =&#xA;$&lt;/span&gt;method_link_species_set_adaptor-&amp;gt;fetch_by_method_link_type_GenomeDBs(&#xA;      &lt;span class=&quot;math-container&quot;&gt;$alignment_type, &#xA;  $&lt;/span&gt;genome_dbs);&#xA;&#xA;throw(&quot;The database do not contain any &lt;span class=&quot;math-container&quot;&gt;$alignment_type data for $&lt;/span&gt;target_genome_db!&quot;)&#xA;    if (!$method_link_species_set);&#xA;&#xA;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$slice_adaptor = Bio::EnsEMBL::Registry-&amp;gt;get_adaptor($&lt;/span&gt;species, 'core', 'Slice');&#xA;&#xA;throw(&quot;Registry configuration file has no data for connecting to &amp;lt;&lt;span class=&quot;math-container&quot;&gt;$species&amp;gt;&quot;)&#xA;if (!$&lt;/span&gt;slice_adaptor);&#xA;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$query_slice = $&lt;/span&gt;slice_adaptor-&amp;gt;fetch_by_region('toplevel', &lt;span class=&quot;math-container&quot;&gt;$seq_region, $&lt;/span&gt;seq_region_start, $seq_region_end);&#xA;&#xA;throw(&quot;No Slice can be created with coordinates &lt;span class=&quot;math-container&quot;&gt;$seq_region:$&lt;/span&gt;seq_region_start-&quot;.&#xA;    &quot;&lt;span class=&quot;math-container&quot;&gt;$seq_region_end&quot;) if (!$&lt;/span&gt;query_slice);&#xA;&#xA;# Fetching all the GenomicAlignBlock corresponding to this Slice:&#xA;my $genomic_align_block_adaptor = Bio::EnsEMBL::Registry-&amp;gt;get_adaptor(&#xA;    'Multi', 'compara', 'GenomicAlignBlock');&#xA;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$genomic_align_blocks =&#xA;$&lt;/span&gt;genomic_align_block_adaptor-&amp;gt;fetch_all_by_MethodLinkSpeciesSet_Slice(&#xA;      &lt;span class=&quot;math-container&quot;&gt;$method_link_species_set, &#xA;  $&lt;/span&gt;query_slice);&#xA;&#xA;my $all_aligns;&#xA;&#xA;# Get a Bio::SimpleAlign object from every GenomicAlignBlock&#xA;foreach my &lt;span class=&quot;math-container&quot;&gt;$this_genomic_align_block (@$&lt;/span&gt;genomic_align_blocks) {&#xA;    my &lt;span class=&quot;math-container&quot;&gt;$simple_align = $&lt;/span&gt;this_genomic_align_block-&amp;gt;get_SimpleAlign;&#xA;    push(@&lt;span class=&quot;math-container&quot;&gt;$all_aligns, $&lt;/span&gt;simple_align);&#xA;}&#xA;&#xA;# print all the genomic alignments using a Bio::AlignIO object&#xA;my &lt;span class=&quot;math-container&quot;&gt;$alignIO = Bio::AlignIO-&amp;gt;newFh(&#xA;-interleaved =&amp;gt; 0,&#xA;-fh =&amp;gt; \*STDOUT,&#xA;-format =&amp;gt; $&lt;/span&gt;output_format,&#xA;    -idlength =&amp;gt; 10&#xA;);&#xA;&#xA;foreach my &lt;span class=&quot;math-container&quot;&gt;$this_align (@$&lt;/span&gt;all_aligns) {&#xA;    print &lt;span class=&quot;math-container&quot;&gt;$alignIO $&lt;/span&gt;this_align;&#xA;}&#xA;&#xA;exit; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any pointers on what I'm doing wrong ?&lt;/p&gt;&#xA;" OwnerUserId="1135" LastEditorUserId="1135" LastEditDate="2018-10-16T11:23:08.863" LastActivityDate="2018-12-11T14:24:13.523" Title="Error while performing &quot;lift-over&quot; of genomic coordinates between two species using Ensembl Compara API" Tags="|ensembl|perl|api|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5217" PostTypeId="2" ParentId="5208" CreationDate="2018-10-15T19:30:09.060" Score="8" Body="&lt;p&gt;There are two types of features which can be extracted from protein sequences.  It is not possible to known which, if any, of these features would be useful in classification tasks.  It may not be possible to build a classifier at all, or it may be very straight forward.  To know this, a feature selection technique must be used, such as e.g. forward feature selection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Sequence based features&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sequence based features are derived from sequence-only information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most common used features are the PseAAC (pseudo amino acid composition) style features, including:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Average Position of each amino acid - 20 features&lt;/li&gt;&#xA;&lt;li&gt;Average Distance between each amino acid - 20 features&lt;/li&gt;&#xA;&lt;li&gt;Motifs - 20 features, 20*20 features, and/or 20*20*20 features (depending on motif length)&lt;/li&gt;&#xA;&lt;li&gt;OAAC (Overall Amino Acid Composition) - 20 features&lt;/li&gt;&#xA;&lt;li&gt;SAAC (Split Amino Acid Composition) - 60+ features&lt;/li&gt;&#xA;&lt;li&gt;AAIndex Composition - 1 to 567+ features&lt;/li&gt;&#xA;&lt;li&gt;PCC (Physicochemical Composition) - ~13 features&lt;/li&gt;&#xA;&lt;li&gt;PSSM encoded as 1, 20, 200, 400 or 600 features (i.e. Psi-Blast output using NR or swissprot)&lt;/li&gt;&#xA;&lt;li&gt;Average predicted residue burial/exposure level&lt;/li&gt;&#xA;&lt;li&gt;Average predicted sequence entropy&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Structure based features (predicted)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the sequence, you can either predict the 3d structure using homology, ab initio, or directly use a homologous structure.  You can then extract predicted 3d features, such as:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;3d size (i.e. max of x,y,z)&lt;/li&gt;&#xA;&lt;li&gt;Calculated ASA/SASA&lt;/li&gt;&#xA;&lt;li&gt;3d contacts (type, frequency, etc)&lt;/li&gt;&#xA;&lt;li&gt;any other local/global environmental information&lt;/li&gt;&#xA;&lt;li&gt;cath/scop classification&lt;/li&gt;&#xA;&lt;li&gt;dssp/stride composition&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1606" LastActivityDate="2018-10-15T19:30:09.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5218" PostTypeId="1" AcceptedAnswerId="5219" CreationDate="2018-10-15T20:57:07.927" Score="2" ViewCount="482" Body="&lt;p&gt;When I read about a term related to phylogenetics, for example, &lt;span class=&quot;math-container&quot;&gt;$K_a$&lt;/span&gt;/&lt;span class=&quot;math-container&quot;&gt;$K_s$&lt;/span&gt; ratio, I constantly see the word &lt;em&gt;site&lt;/em&gt;, whose meaning isn't explained and it puzzles me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example substitutions per synonymous site, substitutions per non-synonymous site, substitutions at a site etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you define a phylogenetic &lt;em&gt;site&lt;/em&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hdT1h.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hdT1h.png&quot; alt=&quot;branch&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So there were 0.6 substitutions per the blue-violet site, what does it mean? What is the length of the site? Is the length of sites constant? Is it the length of a sequence read? Are introns the part of the site?&lt;/p&gt;&#xA;" OwnerUserId="3514" LastEditorUserId="3848" LastEditDate="2020-05-08T14:15:07.887" LastActivityDate="2020-05-08T14:15:07.887" Title="What is a site in a phylogenetic tree?" Tags="|phylogenetics|phylogeny|terminology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5219" PostTypeId="2" ParentId="5218" CreationDate="2018-10-15T22:01:39.900" Score="4" Body="&lt;p&gt;A site is simply an individual discrete position, normally a single nucleotide (or amino acid). For example, consider a toy alignment:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sequence1 ATGC&#xA;sequence2 ATCC&#xA;#           ^ arrow points to the third site&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here there is 1 mutation in 4 sites, so 0.25 substitutions per nucleotide site.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure where the term originated, but probably one of the earlier definitions is from renowned population geneticist Motoo Kimura, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1212250/pdf/893.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;who simply stated in 1968&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In this paper, “site” refers to a single nucleotide pair&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A pair in this context, means either a homozygous or heterozygous site.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Synonymous sites are positions where nucleotide changes lead to no amino acid change, these are typically at the 3rd position of the codon where the nucleotide is often degenerate, for example, &lt;code&gt;TAT&lt;/code&gt;, &lt;code&gt;TAC&lt;/code&gt; &lt;code&gt;TAG&lt;/code&gt;, and &lt;code&gt;TAA&lt;/code&gt; all encode the Serine amino acid. By contrast non-synonymous sites are where a single nucleotide change does lead to an amino acid change, and therefore possibly a more important functional change to the resulting protein. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, the ratio of Ka/Ks (or Dn/Ds) is thus often used as a crude proxy of selection, with Ka/Ks &amp;lt; 0, Ka/Ks = 0 and Ka/Ks &gt; 0 implying purifying (negative) selection, neutral evolution and positive selection respectively. Note that this assumes the neutrality of synonymous sites, which is increasingly questioned due codon usage bias and regulatory elements overlapping coding regions (and also ignores mutational biases and biased gene conversion, although these can be accounted for in an evolutionary model, and also assumes no sequencing errors and perfect alignments, which can impact a lot on the results too). &lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="104" LastEditDate="2018-10-16T08:57:42.020" LastActivityDate="2018-10-16T08:57:42.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5220" PostTypeId="1" CreationDate="2018-10-15T23:59:33.377" Score="5" ViewCount="309" Body="&lt;p&gt;I would like to get some summary statistics on a vcf file from one individual, which has over a million variant calls. I've tried to make a histogram of indel sizes with this command, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;vcftools --vcf sample.vcf --out sample --hist-indel-len&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but it returns an empty file. I see thousands of deletions in the vcf, although strangely no insertions. A typical line in the vcf file I have looks like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NC_025968.3 168363 191 N &amp;lt;DEL&amp;gt; . . SVTYPE=DEL;STRANDS=+-:19;SVLEN=-1026;END=169389;CIPOS=-9,330;CIEND=-152,9;CIPOS95=-3,61;CIEND95=-32,3;IMPRECISE;SU=19;PE=19;SR=0 GT:SU:PE:SR ./.:19:19:0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Update: I aligned reads to my reference with BWA MEM, then used samblaster to extract discordant pairs and split-reads. I sorted and indexed the three bam files, then used lumpyexpress to generate the vcf.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samblaster -i sample.sam --excludeDups --addMateTags --maxSplitCount 2 --minNonOverlap 20 | samtools view -S -b - &amp;gt; sample.bam&#xA;&#xA;samtools view -h sample.bam | samblaster -a -e -d sample.disc.sam -s sample.split.sam -o /dev/null&#xA;&#xA;lumpyexpress -B ../sample.sorted.bam -S ../sample.split.sorted.bam -D ../sample.disc.sorted.bam -o sample.vcf &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to see the size distribution of deletions, and if possible, figure out why there are no insertions.&lt;/p&gt;&#xA;" OwnerUserId="3505" LastEditorUserId="3848" LastEditDate="2020-04-15T12:22:03.140" LastActivityDate="2020-04-15T12:22:03.140" Title="vcftools: indel size histogram command returns empty file" Tags="|vcf|ngs|vcftools|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5221" PostTypeId="1" CreationDate="2018-10-16T14:37:08.530" Score="3" ViewCount="182" Body="&lt;p&gt;For an assignment I must do a pairwise optimal local alignment using BLOSUM65 and five protein sequences. The algorithm I want to use is the Smith-Waterman.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Context protein sequencing using Blastp: all the sequences are the &lt;em&gt;hemoglobulin subunit a or b&lt;/em&gt;;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;two times two pairs are from the same organism (but differ in subunit),&lt;/li&gt;&#xA;&lt;li&gt;one is not; &lt;/li&gt;&#xA;&lt;li&gt;all the sequences differ in sequence length (around ~15 amino acids. Gaps!)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The default for the BLOSUM65 penalties are: 10 for a gap, 1 for an extension.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I optimise the gap penalty and gap extension penalty? Should I stick with the default? If so, why?&lt;/p&gt;&#xA;" OwnerUserId="3522" LastEditorUserId="3848" LastEditDate="2019-08-17T16:56:57.130" LastActivityDate="2021-07-08T13:27:24.683" Title="How to assign the best gap penalty and gap extension penalty using BLOSUM65" Tags="|sequence-alignment|proteins|homework|" AnswerCount="0" CommentCount="1" ClosedDate="2021-07-08T13:27:32.627" ContentLicense="CC BY-SA 4.0" />
  <row Id="5222" PostTypeId="1" AcceptedAnswerId="5223" CreationDate="2018-10-16T19:17:44.980" Score="1" ViewCount="113" Body="&lt;p&gt;Is there a way to calculate the area under a curve in a coverage graph?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to calculate the AUC per strand per gene in a coverage graph. I have two wig files (sense and antisense) generated from a single bam (RNA-seq) file (using the &lt;a href=&quot;https://github.com/pmenzel/stranded-coverage&quot; rel=&quot;nofollow noreferrer&quot;&gt;strand-coverage&lt;/a&gt; software).&lt;/p&gt;&#xA;" OwnerUserId="3524" LastEditorUserId="3524" LastEditDate="2018-10-17T15:12:16.610" LastActivityDate="2018-10-17T15:12:16.610" Title="How to calculate AUC in coverage graph" Tags="|rna-seq|coverage|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5223" PostTypeId="2" ParentId="5222" CreationDate="2018-10-16T21:16:25.643" Score="4" Body="&lt;p&gt;Yes, just add up the coverage at each location; that's integration in a nutshell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or, alternatively / equivalently, count the total number of reads within the region.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-10-16T21:16:25.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5224" PostTypeId="2" ParentId="5143" CreationDate="2018-10-17T13:40:16.687" Score="3" Body="&lt;p&gt;As you said, this is a well-known problem for ASE. In &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/25/24/3207/235151&quot; rel=&quot;nofollow noreferrer&quot;&gt;Degner et al. 2009&lt;/a&gt;, they state that, &quot;Perhaps surprisingly, we found that masking known SNPs does little to eliminate inherent biases in read-mapping.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As such, substituting N bases is not likely to completely solve your issue. Depending on the aligner, these N bases may also negatively affect your ability to map reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of your two suggestions, I believe HISAT2 will be the better approach. There are also more intensive methods you can use such as &lt;a href=&quot;https://bmcgenomics.biomedcentral.com/articles/10.1186/1471-2164-15-920&quot; rel=&quot;nofollow noreferrer&quot;&gt;empirical Bayesian methods&lt;/a&gt; or more traditional variant phasing (ex. &lt;a href=&quot;http://alleleseq.gersteinlab.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlleleSeq&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="3499" LastActivityDate="2018-10-17T13:40:16.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5225" PostTypeId="1" CreationDate="2018-10-17T18:46:08.307" Score="5" ViewCount="8568" Body="&lt;p&gt;I have 10x single cell RNA seq data. Which R package is best suited for analysis of the 10x data matrix. What is the script to prepare the data for downstream GSEA analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already processed samples for single cell data (for multiple other purposes). my intention is to focus on subpopulations i.e. cell type x at different time points i.e. time 0, 1, 2 etc and compare the enriched gene pathways in this one cell type at different time points. there are three files that are used (at least in the R Seurat package) with the following file extensions: .mtx, .tsv (barcodes.tsv, genes.tsv and matrix.mtx). the R Seurat package merges these files to create a &quot;Seurat object&quot; which helps mainly with identifying individual diffl gene expression in each barcode cell &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found a package for R: goseq that at least has been used for bulkRNA seq but I was hoping someone has used this package specifically for single cell RNA seq data, and what script has been used to merge the 3 files to create an object for analysis. pardon my terminology, but I am a novice to R and have experience primarily through package use. &lt;/p&gt;&#xA;" OwnerUserId="3529" LastEditorUserId="1855" LastEditDate="2018-11-25T00:02:16.447" LastActivityDate="2020-01-10T15:52:51.487" Title="Script to allow gene set enrichment analysis of 10x genomics data in R" Tags="|r|scrnaseq|10x-genomics|gsea|cellranger|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5226" PostTypeId="1" CreationDate="2018-10-17T19:04:53.200" Score="0" ViewCount="179" Body="&lt;p&gt;I am testing different &lt;code&gt;Protein-Protein Interaction networks&lt;/code&gt; alignment tools. I observed that all alignment tools show some &lt;code&gt;Gene Ontology&lt;/code&gt; terms similarity. How can I calculate &lt;code&gt;Functional similarity&lt;/code&gt; / &lt;code&gt;average functional similarity&lt;/code&gt; for &lt;code&gt;Gene ontology&lt;/code&gt;'s terms &lt;code&gt;Biological processes&lt;/code&gt;, &lt;code&gt;Cellular components&lt;/code&gt;, &lt;code&gt;Molecular functionality&lt;/code&gt; of any two proteins networks&lt;/p&gt;&#xA;" OwnerUserId="3531" LastActivityDate="2018-10-18T08:31:16.950" Title="How to calculate Gene Ontology terms in python" Tags="|sequence-alignment|proteins|networks|protein-protein-interaction|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5227" PostTypeId="1" AcceptedAnswerId="5228" CreationDate="2018-10-17T19:24:32.390" Score="1" ViewCount="42" Body="&lt;p&gt;I am preprocessing a set of scTHSSeq reads in which each read sequence identifier has been replaced with the cell barcode, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@CGAGGCTG_TATCCTCT_AGTCCA&#xA;...&#xA;...&#xA;...&#xA;@CGAGGCTG_TATCCTCT_GGTACG&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The read sequence identifier has been replaced with the cell barcode to be able to demultiplex alignments after bwa into a different bamfile for each cell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran the alignment using &lt;a href=&quot;http://bio-bwa.sourceforge.net/bwa.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;bwa mem&lt;/a&gt; and I received no errors. However I would like to know if there are caveats or reasons why this approach may lead to misleading results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will then proceed to remove PCR duplicates with &lt;code&gt;samtools rmdup&lt;/code&gt; executed in each cell's bamfile and then I'd run peak calling using &lt;a href=&quot;https://github.com/hms-dbmi/spp&quot; rel=&quot;nofollow noreferrer&quot;&gt;spp&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-10-17T20:39:48.517" Title="Does read sequence identifier need to be unique for proper alignment?" Tags="|read-mapping|reads|identifiers|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5228" PostTypeId="2" ParentId="5227" CreationDate="2018-10-17T19:35:25.600" Score="2" Body="&lt;p&gt;What tool did you use to replace the read names? Can you modify that to split up the reads into separate files or encode the cell barcode in the read description instead of the read name? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Typically is it assumed by many tools that the FASTQ read names are unique. you could have issues with BWA and split alignments but I don't know if that pertains to your data.  I think the best you can do is the sanity check the results at each step of the process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the fastq format (&lt;a href=&quot;https://en.wikipedia.org/wiki/FASTQ_format&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/FASTQ_format&lt;/a&gt;) on the first line denoted by the '@' there can be two values separated by a space.  The first is the name and the second is the description.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So an example would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@Original_read_name CGAGGCTG_TATCCTCT_AGTCCA&#xA;ATCGATCATGATGACA&#xA;+&#xA;@&amp;amp;%*@(%@%*($!##!&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="964" LastEditorUserId="964" LastEditDate="2018-10-17T20:39:48.517" LastActivityDate="2018-10-17T20:39:48.517" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5229" PostTypeId="1" CreationDate="2018-10-17T20:25:11.650" Score="7" ViewCount="37647" Body="&lt;p&gt;I'm trying to convert ~20,000 different human gene symbols to ensembl IDs. I've been trying to use biomaRt to do this, but continue getting the following error &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;getBM( attributes=c(&quot;ensembl_gene_id&quot;) , filters= &quot;mgi_symbol&quot;  ,mart=ensembl)&#xA;Error in martCheck(mart) : &#xA;  No dataset selected, please select a dataset first.  You can see the available datasets by using the listDatasets function see ?listDatasets for more information.  Then you should create the Mart object by using the useMart function.  See ?useMart for more information&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a different format I should be using for this? I tried using BioConductor's description of how to use getBM but am a R novice and confused as to how to set this up to convert gene symbols into ensemble IDs. &lt;/p&gt;&#xA;" OwnerUserId="3532" LastEditorUserId="964" LastEditDate="2018-10-17T20:40:32.030" LastActivityDate="2021-09-28T08:26:31.590" Title="Converting Gene Symbol to Ensembl ID in R" Tags="|r|gene|ensembl|biomart|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5230" PostTypeId="2" ParentId="5229" CreationDate="2018-10-18T07:12:57.810" Score="12" Body="&lt;p&gt;You can use the standard annotation package for humans (you can also use biomaRt, but it can be more confusing, see below):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(&amp;quot;org.Hs.eg.db&amp;quot;) # remember to install it if you don't have it already&#xA;symbols &amp;lt;- mapIds(org.Hs.eg.db, keys = ensemblsIDS, keytype = &amp;quot;ENSEMBL&amp;quot;, column=&amp;quot;SYMBOL&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;ensemblsIDS&lt;/code&gt; should be a character vector: &lt;code&gt;c(&amp;quot;ENSG001&amp;quot;, &amp;quot;ENSG002&amp;quot;, ..)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, if your primary key is an ENSEMBL id it is preferred to use a ENSEMBL package centred around the ENSEMBL id :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(&amp;quot;EnsDb.Hsapiens.v86&amp;quot;)&#xA;mapIds &amp;lt;- mapIds(EnsDb.Hsapiens.v86, keys = ensemblsIDS, keytype = &amp;quot;GENEID&amp;quot;, columns = c(&amp;quot;SYMBOL&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The solution with biomaRt:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(&amp;quot;biomaRt&amp;quot;)&#xA;ensembl = useMart(&amp;quot;ensembl&amp;quot;,dataset=&amp;quot;hsapiens_gene_ensembl&amp;quot;)&#xA;getBM(attributes='hgnc_symbol', &#xA;      filters = 'ensembl_gene_id', &#xA;      values = ensemblsIDS, &#xA;      mart = ensembl)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2021-09-28T08:26:31.590" LastActivityDate="2021-09-28T08:26:31.590" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5231" PostTypeId="2" ParentId="5226" CreationDate="2018-10-18T08:31:16.950" Score="2" Body="&lt;p&gt;You can calculate gene ontology similarity with the &lt;a href=&quot;http://bioconductor.org/packages/GOSemSim&quot; rel=&quot;nofollow noreferrer&quot;&gt;GOSemSim&lt;/a&gt; R package (&lt;a href=&quot;http://doi.org/10.1093/bioinformatics/btq064&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;)(disclaimer I'm a contributor to this package). You have several similarity scores implemented, some of them are across the three subontologies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In python you have the &lt;a href=&quot;https://pypi.org/project/goatools/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GOATOOLS&lt;/a&gt; (&lt;a href=&quot;https://www.nature.com/articles/s41598-018-28948-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;). I think there was another Python package but now I don't remember its name. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-10-18T08:31:16.950" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5233" PostTypeId="1" AcceptedAnswerId="5234" CreationDate="2018-10-18T19:15:30.497" Score="8" ViewCount="4592" Body="&lt;p&gt;I'm working through &lt;a href=&quot;http://davidaknowles.github.io/leafcutter/articles/Usage.html&quot; rel=&quot;noreferrer&quot;&gt;the example data set for LeafCutter&lt;/a&gt; and the documentation mentions &quot;split reads&quot;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This will cluster together the introns fond [&lt;em&gt;sic&lt;/em&gt;] in the junc files listed&#xA;  in test_juncfiles.txt, requiring 50 split reads supporting each&#xA;  cluster and allowing introns of up to 500kb.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Searching &quot;split reads&quot; on Google yielded few results, and the &lt;a href=&quot;https://www.biostars.org/p/116201/&quot; rel=&quot;noreferrer&quot;&gt;results I did find&lt;/a&gt; were hard for me understand,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Chimeric reads occur when one sequencing read aligns to two distinct&#xA;  portions of the genome with little or no overlap. Chimeric reads are &#xA;  indicative of structural variation. Chimeric reads are also called&#xA;  split reads.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One sequencing read aligns to two distinct portions of the genome, but then with little or no overlap? Isn't that a self-contradiction?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can more or less deduce what &quot;intron clusters&quot; are but I figured I might as well try to solicit a more formal definition. I know what introns are and I can figure what it means to cluster them together. I guess I just want to know what split reads and intron clusters mean in the context of working out the example data for LeafCutter.&lt;/p&gt;&#xA;" OwnerUserId="3442" LastEditorUserId="964" LastEditDate="2018-10-18T20:23:50.773" LastActivityDate="2019-06-12T19:48:46.570" Title="What are &quot;split reads&quot; and &quot;intron clusters?&quot;" Tags="|sequence-alignment|read-mapping|rna-alignment|rna-splicing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5234" PostTypeId="2" ParentId="5233" CreationDate="2018-10-18T20:23:12.980" Score="5" Body="&lt;p&gt;&lt;strong&gt;split reads&lt;/strong&gt; - These are read that have two or more alignments to the reference from unique region of the read.  In this example a 150bp read sequenced from RNA could have base 1-75 aligning to the 3' end of exon2 and bases 76-150 aligning to the 5' end of exon3.  This would be a split read because it have two alignments (exon2 and exon3) and those alignment are from unique (non overlapping) parts of the read, 1-75 and 76-150.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Chimeric reads are an example of split reads.  Many people use &quot;chimeric reads&quot; to describe a type of noise that occur during PCR or sequencing where the bases from two different molecules are reported as one read and the alignments of that read become split when it aligns back to the location of the two original molecules.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&quot;intron clusters&quot;&lt;/strong&gt; - In this case I think that the LeafCutter software is describing stacks of split reads to cluster to support the detection of an intron.&lt;/p&gt;&#xA;" OwnerUserId="964" LastEditorUserId="964" LastEditDate="2019-06-12T19:48:46.570" LastActivityDate="2019-06-12T19:48:46.570" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5235" PostTypeId="1" CreationDate="2018-10-19T04:19:57.987" Score="3" ViewCount="2357" Body="&lt;p&gt;I need to convert contigs into their respective protein sequences given a reference genome (i.e. I need to take a substring, whose position is already known on the string, and I need to locate the nearest start and stop codons).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is tricky because sometimes the first position of triplet codon within the contig will not be a multiple of three and could be located in the intergenic region (i.e. between genes). The goal is to separate both coding and non-coding DNA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my code so far:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.Seq import Seq&#xA;from Bio.Alphabet import generic_dna, generic_protein&#xA;&#xA;start_codons = ['ATG']&#xA;stop_codons = ['TAG', 'TAA', 'TGA']&#xA;&#xA;string = 'GG*TAG*CCAATT*ATG*AACGAA*TAG*GAC' #remove '*', just for visual&#xA;contigs = ['CCAA', 'TGAAC', 'GAA', 'GGAC']&#xA;positions = [5, 12, 17, 22] #position indices for each contig on string&#xA;&#xA;extended_contigs = []&#xA;extended_position_contigs = []&#xA;intergenic_contigs = []&#xA;intergenic_position_contigs = []&#xA;for i in contigs:&#xA;    extended_contigs.append(#some code)&#xA;    extended_positions_contigs.append(#some code)&#xA;    intergenic_contigs.append(#some code)&#xA;    intergenic_positions_contigs.append(#some code)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I should get &lt;code&gt;extended_contigs = ['ATGAAC', 'ATGAACGAA']&lt;/code&gt; and &lt;code&gt;extended_positions_contigs = [12, 17]&lt;/code&gt;.  These are the contigs that are located within genes. In order to code them into peptides, I need to reach back in the string until I find the start codon and extend the initial contig (e.g. &lt;code&gt;TGAAC&lt;/code&gt; -&gt; &lt;code&gt;ATGAAC&lt;/code&gt; and &lt;code&gt;GAA&lt;/code&gt; -&gt; &lt;code&gt;ATGAACGAA&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should also get &lt;code&gt;intergenic_contigs = ['CCAA', 'GGAC']&lt;/code&gt; and &lt;code&gt;intergenic_positions_contigs = [5, 22]&lt;/code&gt;.  When the missing code is run, the computer searches to the left of the string and finds a stop codon (e.g. &lt;code&gt;TAG&lt;/code&gt;) before the start codon. Thus, the contig is located in between two genes and nothing needs to be added. These intergenic contigs are just stored in a new list.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code continues:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;prot_contigs = []&#xA;for i in extended_contigs:&#xA;    my_dna = Seq(i, generic_dna)&#xA;    my_prot = my_dna.translate()&#xA;    prot_contigs.append(str(my_prot))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here, no new code needs to be added. After the above is run, &lt;code&gt;prot_contigs = ['MN', 'MNE']&lt;/code&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The last step of code (which I need help with) will transform &lt;code&gt;prot_contigs&lt;/code&gt; into &lt;code&gt;new_prot_contigs = ['MN', 'E']&lt;/code&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How? If for any contig (e.g. &lt;code&gt;'TGAAC'&lt;/code&gt;), the beginning or end is part of another codon (not perfect multiple of 3), the extra codon on either end will be maintained (e.g. &lt;code&gt;'MN'&lt;/code&gt; stays &lt;code&gt;'MN'&lt;/code&gt;).  Otherwise, if the contig (e.g. &lt;code&gt;'GAA'&lt;/code&gt;) perfectly matches a codon, anything that was added onto it will be removed (e.g. &lt;code&gt;'MNE'&lt;/code&gt; becomes &lt;code&gt;'E'&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would try to solve the 2 portions of code myself, but I am unsure on how to take a position on a string (i.e. beginning points of contig) and look along the string to find the nearest start/stop codons, so I could determine the DNA's function and accurately sequence the protein coding contigs into peptides.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be appreciated!     &lt;/p&gt;&#xA;" OwnerUserId="3515" LastEditorUserId="3848" LastEditDate="2020-04-21T12:17:31.550" LastActivityDate="2020-07-15T14:57:36.833" Title="Searching for start and stop codons for protein sequencing of contigs" Tags="|proteins|python|phylogenetics|genomics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5237" PostTypeId="2" ParentId="4963" CreationDate="2018-10-19T21:20:41.470" Score="6" Body="&lt;p&gt;I assume you are referring to &lt;code&gt;VlnPlot()&lt;/code&gt; of Seurat. The reason that you are getting such a plot is because your distribution is highly skewed, most of your points are zeros, not surprising for scRNA-seq. And the plot is actually showing your distributions, it is just the points at zero stacking on top of each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can double check this by plotting a histogram for each of the clusters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example violin plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BMK1r.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BMK1r.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cluster 0, which is mostly zeros, kind of looks like yours.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0oMTG.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0oMTG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cluster 6, which I believe is what you would expect to see, contains a lot of cells that express the gene and much less skewed, hence the shape denoting the density.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kBC32.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kBC32.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what you are seeing is normal, your gene is not expressed/not detected is all.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="57" LastEditDate="2018-10-20T08:03:06.593" LastActivityDate="2018-10-20T08:03:06.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5239" PostTypeId="2" ParentId="2163" CreationDate="2018-10-21T18:12:16.310" Score="1" Body="&lt;p&gt;For can do this by accessing the basecall qualities from &lt;a href=&quot;https://pysam.readthedocs.io/en/latest/api.html#pysam.PileupRead.alignment&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;PileupRead.alignment&lt;/code&gt;&lt;/a&gt;. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;_BASE_QUAL_CUTOFF = 30&#xA;with pysam.Samfile(in_file, 'rb') as bam_in:&#xA;    for column in bam_in.pileup():&#xA;        for read in column:&#xA;            qual = read.alignment.query_qualities&#xA;            if all([ord(c)-33 &amp;gt;= _BASE_QUAL_CUTOFF for c in qual]):&#xA;                # do some stuff&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This involves checking every read at every position. You could memoise this task:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def check_quality(read):&#xA;    try:&#xA;        return passed_quality[read.query_name]&#xA;    except KeyError:&#xA;        qual = read.alignment.query_qualities &#xA;        passed = all([ord(c)-33 &amp;gt;= _BASE_QUAL_CUTOFF for c in qual])&#xA;        passed_quality[read.query_name] = passed&#xA;        return passed_quality[read.query_name]&#xA;&#xA;_BASE_QUAL_CUTOFF = 30&#xA;passed_quality = dict()&#xA;with pysam.Samfile(in_file, 'rb') as bam_in:&#xA;    for column in bam_in.pileup():&#xA;        for read in column:&#xA;            if check_quality(read):&#xA;                # do some stuff&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, using &lt;code&gt;functools.lru_cache&lt;/code&gt; (thanks @winni2k):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from functools import lru_cache&#xA;&#xA;@lru_cache(maxsize=1000)&#xA;def check_quality(read):&#xA;    qual = read.alignment.query_qualities &#xA;    return all([ord(c)-33 &amp;gt;= _BASE_QUAL_CUTOFF for c in qual])&#xA;&#xA;_BASE_QUAL_CUTOFF = 30&#xA;passed_quality = dict()&#xA;with pysam.Samfile(in_file, 'rb') as bam_in:&#xA;    for column in bam_in.pileup():&#xA;        for read in column:&#xA;            if check_quality(read):&#xA;                # do some stuff&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="163" LastEditorUserId="163" LastEditDate="2018-11-24T18:25:05.413" LastActivityDate="2018-11-24T18:25:05.413" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5240" PostTypeId="2" ParentId="3208" CreationDate="2018-10-21T18:35:04.123" Score="3" Body="&lt;p&gt;MAGIC assumes input data has been both library-size normalized, and either log or sqrt transformed prior to imputation (see also: &lt;a href=&quot;http://nbviewer.jupyter.org/github/KrishnaswamyLab/magic/blob/master/python/tutorial_notebooks/bonemarrow_tutorial.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;MAGIC tutorial&lt;/a&gt;). Additionally, any graph-based methods (MAGIC, PHATE, t-SNE, UMAP, spectral clustering, Louvain, etc etc) will give flawed results if your data contains a batch effect, since the neighbourhood graph would reflect that structure of your batch effect, and worse, imputation would further reinforce this batch effect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus I would recommend the following pipeline:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Library-size normalization&lt;/li&gt;&#xA;&lt;li&gt;Square root (or log) transform&lt;/li&gt;&#xA;&lt;li&gt;Batch effect removal&lt;/li&gt;&#xA;&lt;li&gt;Imputation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Regarding your &lt;strong&gt;update&lt;/strong&gt;, the reason you don't see the batch effect in the raw counts data is simply that the batch effect is not visible in the most highly expressed genes. Prior to transformation, the principal source of variation in your data is simply the expression of the most highly expressed genes - this is essentially masking the source of the batch effect, not removing it. I recommend never working with raw molecule counts in scRNAseq, as the raw counts data hides much of the heterogenity in your dataset, which is precisely what you are looking in doing single-cell RNA-seq.&lt;/p&gt;&#xA;" OwnerUserId="163" LastActivityDate="2018-10-21T18:35:04.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5241" PostTypeId="2" ParentId="4949" CreationDate="2018-10-21T19:38:24.553" Score="1" Body="&lt;p&gt;If &lt;code&gt;fit$ord&lt;/code&gt; gives you the order of cells on the curve, you should be able to obtain the first and last cells by doing the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;first_cell_index &amp;lt;- which.min(fit&lt;span class=&quot;math-container&quot;&gt;$ord)&#xA;last_cell_index &amp;lt;- which.max(fit$&lt;/span&gt;ord)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I'm not sure this is so informative - if you're looking to get representative gene expression for the start and end of the curve, you should really be looking at the cluster of cells towards the start and end of the curve - for example, by comparing cells in clusters 1 and 3 that you showed in your first plot. A single cell will suffer from dropout and not tell you a lot about the cluster.&lt;/p&gt;&#xA;" OwnerUserId="163" LastActivityDate="2018-10-21T19:38:24.553" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5243" PostTypeId="2" ParentId="5225" CreationDate="2018-10-22T08:46:12.530" Score="-2" Body="&lt;p&gt;I realized that your question is specifically gene set enrichment analysis. However, I personally have never tried any R package doing this. Although topGO may seem to be the one that may work for you? What I have been doing is that for gene enrichment analysis using Gene Ontology, simply paste and run, or DAVID when it was still updating. Sorry, I am not an expert in incorporating biological significance study in the computational result. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that you are also not quite sure what tools you should use for downstream analysis. I think the following publication may be helpful. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/30228881&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pubmed/30228881&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same primary author also published a paper titled, &lt;em&gt;Cluster Headache: Comparing Clustering Tools for 10X Single Cell Sequencing Data&lt;/em&gt; (For some reason I could not find the link to it, will edit the answer if I find it. )&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps. &lt;/p&gt;&#xA;" OwnerUserId="3480" LastEditorUserId="3480" LastEditDate="2018-10-22T08:53:46.163" LastActivityDate="2018-10-22T08:53:46.163" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5244" PostTypeId="1" CreationDate="2018-10-22T09:57:27.220" Score="3" ViewCount="6699" Body="&lt;p&gt;I have seen many posts regarding counts to RPKM and TPM. I haven't seen any post for counts to FPKM. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have RNA-Seq data which is paired-end reads. Extracted the counts using featureCounts for all the samples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a function to convert counts to RPKM: using the gene_length&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rpkm &amp;lt;- function(counts, lengths) {&#xA;  rate &amp;lt;- counts / lengths &#xA;  rate / sum(counts) * 1e6&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I know that RPKM is mainly used for single-end reads data. Do you think I can use the above function for converting counts to FPKM as my data is paired end? [TCGA data: HTSeq counts are converted to HTSeq FPKM]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Somewhere I have seen that the same function applies for single end data which will be RPKM and also for paired end data which will be FPKM. Am Ir right or wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not can anyone show some function or code to convert counts to FPKM please.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanq&lt;/p&gt;&#xA;" OwnerUserId="1369" LastActivityDate="2019-03-30T16:26:39.840" Title="How to convert featureCounts to FPKM?" Tags="|rna-seq|rna|gene-expression|fpkm|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5245" PostTypeId="1" AcceptedAnswerId="5246" CreationDate="2018-10-22T11:05:31.313" Score="3" ViewCount="1530" Body="&lt;p&gt;How can I find ligands of receptors (or &lt;em&gt;vice versa&lt;/em&gt;)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I want to find the receptors of IL-8 (CXCL8).&lt;/p&gt;&#xA;" OwnerUserId="208" LastEditorUserId="48" LastEditDate="2018-10-22T14:20:51.623" LastActivityDate="2020-12-09T02:23:26.690" Title="Receptor-ligand database" Tags="|database|public-databases|protein-protein-interaction|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5246" PostTypeId="2" ParentId="5245" CreationDate="2018-10-22T11:05:31.313" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://www.cellphonedb.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;CellPhoneDB&lt;/a&gt; is a publicly available repository of &lt;em&gt;curated&lt;/em&gt; receptors, ligands and their interactions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The interactions can be searched or downloaded.&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2018-10-22T11:05:31.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5247" PostTypeId="2" ParentId="5245" CreationDate="2018-10-22T12:21:14.633" Score="2" Body="&lt;p&gt;Have you tried &lt;a href=&quot;https://string-db.org/cgi/input.pl?sessionId=QLEnNyQMCbBE&amp;amp;input_page_show_search=on&quot; rel=&quot;nofollow noreferrer&quot;&gt;STRING&lt;/a&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a publicly available protein-protein interaction database, so it won't be a complete picture. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.qiagenbioinformatics.com/products/ingenuity-pathway-analysis/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ingenuity Pathways Analysis (IPA)&lt;/a&gt; is really nice but is a paid service. Ask your institution if they have a license for one. &lt;/p&gt;&#xA;" OwnerUserId="404" LastEditorUserId="48" LastEditDate="2018-10-22T14:20:02.333" LastActivityDate="2018-10-22T14:20:02.333" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5248" PostTypeId="2" ParentId="5244" CreationDate="2018-10-22T12:39:34.817" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I have seen many posts regarding counts to RPKM and TPM.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There’s your answer then: FPKM = RPKM. It’s simply a more accurate name.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Speaking of RPKM for paired-end data is discouraged because the reference to “read” in this context lends itself to ambiguity. But mathematically the quantity is the same: we are counting &lt;em&gt;fragments&lt;/em&gt;, not individual reads (of which each fragment has two, for paired-end data).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But as mentioned in the comments, &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/69/29&quot;&gt;there are good reasons against using FPKMs&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2018-10-22T12:39:34.817" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="5249" PostTypeId="2" ParentId="5175" CreationDate="2018-10-22T20:59:16.347" Score="1" Body="&lt;p&gt;I've been working with the multi-CCA alignment workflow in the Seurat package and am very impressed with functionality. As usual, the CCA tutorial is excellent for an alignment workflow and will get you off the ground quickly. You can even use the DESeq2 algorithm for determining differential expression from within Seurat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you use the function &lt;code&gt;RunCCA()&lt;/code&gt; you can pass &lt;code&gt;rescale.groups=TRUE&lt;/code&gt; if you wish to re-scale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Abstracted workflow&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Data preprocessing and gene selection&lt;/li&gt;&#xA;&lt;li&gt;Define a shared correlation space with canonical correlation analysis&lt;/li&gt;&#xA;&lt;li&gt;Remove dataset specific cells&lt;/li&gt;&#xA;&lt;li&gt;Align correlated subspaces using dynamic time warping&lt;/li&gt;&#xA;&lt;li&gt;Integrated analysis across datasets (clustering, trajectory building, etc ...)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Butler, A., and Satija, R. (2017). Integrated analysis of single cell transcriptomic data across conditions, technologies, and species.&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2018-10-22T20:59:16.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5250" PostTypeId="2" ParentId="5173" CreationDate="2018-10-22T21:04:05.363" Score="1" Body="&lt;p&gt;Setting the &lt;code&gt;legChrom&lt;/code&gt; argument to &lt;code&gt;NA&lt;/code&gt; will omit plotting a legend.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Found in &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/chromPlot/inst/doc/chromPlot.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the manual&lt;/a&gt; for chromPlot on Bioconductor for release 3.7.&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2018-10-22T21:04:05.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5251" PostTypeId="2" ParentId="2375" CreationDate="2018-10-22T22:39:24.693" Score="1" Body="&lt;p&gt;If your strain and batch are confounded, (like you did all of strain one on one day, and all of strain two on another), then there is no clever way to fix that.  Just use one column, and inform whomever designed the experiment that you can't separate batch effect from strain difference.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastEditorUserId="2388" LastEditDate="2018-10-22T23:25:35.040" LastActivityDate="2018-10-22T23:25:35.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5252" PostTypeId="2" ParentId="4836" CreationDate="2018-10-22T23:52:14.433" Score="1" Body="&lt;p&gt;In your CRAM file, make all of the reference sequences present in the &lt;code&gt;./&lt;/code&gt; directory. Then make a bash script to make relative links to your reference for whatever mount you are using at the time - ie &lt;code&gt;ln -s TARGET LINK_NAME&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now your CRAM is portable and you just have to tailor the local working dir to it.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-10-22T23:52:14.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5253" PostTypeId="1" AcceptedAnswerId="5254" CreationDate="2018-10-23T01:37:03.230" Score="1" ViewCount="255" Body="&lt;p&gt;I am encountering a segmentation fault when attempting to run &lt;a href=&quot;http://bioinf.wehi.edu.au/featureCounts/&quot; rel=&quot;nofollow noreferrer&quot;&gt;featureCounts&lt;/a&gt; from &lt;code&gt;subread-1.6.3&lt;/code&gt; on even small test data. I installed featureCounts from &lt;a href=&quot;https://sourceforge.net/projects/subread/files/subread-1.6.3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SourceForge&lt;/a&gt; using the precompiled Linux binary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most existing help threads on this topic (&lt;a href=&quot;http://seqanswers.com/forums/showthread.php?t=50353&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://www.biostars.org/p/169712/&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;, &lt;a href=&quot;https://support.bioconductor.org/p/59473/&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;) show some splash screen output before the error, but I get a segmentation fault before any error is shown.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ featureCounts -T 1 -p -t exon -g gene_id -a /data/sai/annots/musmus_cdna_all_ensembl92.gtf -o test.txt test.bam test2.bam&#xA;Segmentation fault (core dumped)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="163" LastActivityDate="2018-10-23T01:37:03.230" Title="featureCounts segmentation fault on Arch Linux" Tags="|rna-seq|sam|subread|featurecounts|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5254" PostTypeId="2" ParentId="5253" CreationDate="2018-10-23T01:37:03.230" Score="3" Body="&lt;p&gt;I resolved this problem by installing from source; I assume the precompiled binary was not compatible with the Arch Linux kernel.&lt;/p&gt;&#xA;" OwnerUserId="163" LastActivityDate="2018-10-23T01:37:03.230" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5255" PostTypeId="1" AcceptedAnswerId="5256" CreationDate="2018-10-23T02:06:49.263" Score="3" ViewCount="1533" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fXEYw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fXEYw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code I used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DotPlot(object = cluster, genes.plot = c(&quot;Hcn4&quot;,&quot;Shox2&quot;,&quot;Nkx2-5&quot;,&quot;Nppa&quot;,&quot;Tnnt2&quot;,&quot;Gja5&quot;,&quot;Tbx18&quot;,&quot;Tbx3&quot;,&quot;Isl1&quot;,&quot;Scn5a&quot;,&quot;Cacna1d&quot;),  plot.legend=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I was confused about the legend with the Negative number shown for &lt;code&gt;avg.exp.scale&lt;/code&gt;. Does it mean that the cluster expressed the genes at low level or that gene was down regulated? (like the Shox2 in the &quot;0&quot; compared with &quot;2&quot;).&lt;/p&gt;&#xA;" OwnerUserId="3171" LastEditorUserId="298" LastEditDate="2018-10-23T10:07:11.060" LastActivityDate="2018-10-23T10:07:11.060" Title="about dotplot legend meaning" Tags="|r|seurat|single-cell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5256" PostTypeId="2" ParentId="5255" CreationDate="2018-10-23T02:57:24.773" Score="3" Body="&lt;p&gt;The expression values for each gene are scaled / standardized by subtracting the genes mean expression and dividing by its standard deviation. A value of -1 would imply it's one standard deviation below the mean expression for that gene.&lt;/p&gt;&#xA;" OwnerUserId="64" LastActivityDate="2018-10-23T02:57:24.773" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5257" PostTypeId="1" CreationDate="2018-10-23T06:56:49.263" Score="2" ViewCount="52" Body="&lt;p&gt;I would like to analyze my RNAseq profiles from bulk tissue samples (Paired-End, 50M reads/sample, tumour-normal pairs) with &lt;code&gt;varscan2&lt;/code&gt; to detect mutations.&lt;/p&gt;&#xA;&lt;p&gt;Then I would like to use those detected mutations to estimate presence of different tumour clones (likewise pyclone on WGS data). Therefore my questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it possible to estimate number of clones and clonal abundancy from RNA seq profiles from bulk tissue? and if yes: are 50M reads/sample enough coverage.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Could you recommend which package to use for this, for example is pyclone suitable for this?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thank you in advance for the advice!&lt;/p&gt;&#xA;" OwnerUserId="3553" LastEditorUserId="3848" LastEditDate="2020-11-11T22:50:13.853" LastActivityDate="2020-11-11T22:50:13.853" Title="Mutation detection using Varscan2 on RNA sequencing for estimating tumour clones with pyclone or other package" Tags="|rna-seq|bioconductor|variation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5259" PostTypeId="1" AcceptedAnswerId="5260" CreationDate="2018-10-23T07:40:10.910" Score="5" ViewCount="1199" Body="&lt;p&gt;I have looked already here: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/202/tools-for-simulating-oxford-nanopore-reads&quot;&gt;Tools for simulating Oxford Nanopore reads&lt;/a&gt;&#xA;. This doesn't answer my question, because it lists a few Nanopore read simulators, but I have specific problems with each of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to simulate some Nanopore reads. I could use NanoSim, but that tool seems to be very difficult to use. I only have a reference genome, but you need training data as well? anyone have any experience with this? I can only find bacterial and yeast data (and I want to try this on &lt;em&gt;Arabidopsis thaliana&lt;/em&gt; reference)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another promising tool I found, was a paper for SNaReSim, reference here:&#xA;&lt;a href=&quot;https://ieeexplore.ieee.org/document/8031171&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ieeexplore.ieee.org/document/8031171&lt;/a&gt;. However I cannot seem to find the tool itself, which is also mentioned in the other stackexchange post I mentioned above. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have experience simulating PacBio reads with NanoSim, SNaReSim or maybe another tool? Because I am not sure about which one to use. Others like &lt;a href=&quot;https://github.com/ethanagbaker/SiLiCO&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;SiLiCO&quot;&gt;SiLiCO&lt;/a&gt; or &lt;a href=&quot;https://sourceforge.net/projects/readsim/&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;ReadSim&quot;&gt;ReadSim&lt;/a&gt; were updated 2 years ago and I believe that Nanopore technology has improved a lot in that time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To clarify a bit on why I want to simulate data: I just want to test several tools about SV calling and simulating for a school project. All of the points mentioned in the comments I will use for the discussion, but I wanted to focus on 1 read simulator per technology and didn't know what to use, because Nanopore is difficult to simulate in that sense. That was already known to me.&lt;/p&gt;&#xA;" OwnerUserId="3552" LastEditorUserId="3552" LastEditDate="2018-10-24T06:00:46.103" LastActivityDate="2022-09-03T18:38:59.377" Title="How to simulate nanopore reads?" Tags="|nanopore|long-reads|simulated-data|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5260" PostTypeId="2" ParentId="5259" CreationDate="2018-10-23T07:52:47.003" Score="5" Body="&lt;p&gt;It is important to train an error model as NanoSim does, as we do not fully understand the error processes involved in both the nanopore sequencing process and the basecalling process. Any sort of read simulator that does not use a species-specific model is just not going to produce realistic simulated reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as datasets go, &lt;a href=&quot;https://www.nature.com/articles/s41467-018-03016-2#Sec9&quot; rel=&quot;noreferrer&quot;&gt;I would recommend looking at this assembly&lt;/a&gt; and the &lt;a href=&quot;https://www.ebi.ac.uk/ena/data/view/PRJEB21270&quot; rel=&quot;noreferrer&quot;&gt;associated data uploaded on EMBL:&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Michael, Todd P., et al. &quot;High contiguity Arabidopsis thaliana&#xA;  genome assembly with a single nanopore flow cell.&quot; Nature&#xA;  communications 9.1 (2018): 541.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There are several data types there, as well as an assembly to train the models on.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Is it OK to train a model using reads from one individual and a reference assembly from another individual?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Generally, it is not good to map reads from one individual to the reference genome of another individual since this introduces something called &lt;a href=&quot;https://www.sevenbridges.com/reference-bias-challenges-and-solutions/&quot; rel=&quot;noreferrer&quot;&gt;reference bias.&lt;/a&gt; When we align reads to a reference genome generated from the same reads, or just from other reads from the same individual we remove this problem. All true reads will map to correctly assembled parts of the genome.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Why can we train a model for nanopore reads on a reference generated from ... nanopore reads? Won't we get a bad model for indels?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Yes, well, maybe. If the reference genome has been corrected to remove indels using a tool like &lt;a href=&quot;https://github.com/broadinstitute/pilon&quot; rel=&quot;noreferrer&quot;&gt;pilon&lt;/a&gt;, then you can generally presume that the reference genome has very few to no indels present as a result of sequencing error from the Nanopore/PacBio reads. As a result, the model derived from aligning reads should have the correct error profile for indels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, it is important that the long-read-assembly was corrected with Illumina shotgun reads from the same individual from which the long reads were derived. Otherwise the corrected reference would have a chimaera of indels and SNPs from multiple individuals.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-10-23T08:14:01.313" LastActivityDate="2018-10-23T08:14:01.313" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="5261" PostTypeId="1" AcceptedAnswerId="5262" CreationDate="2018-10-23T08:13:48.010" Score="0" ViewCount="76" Body="&lt;p&gt;I am an occasional user of tools in bioinformatics and I am fascinated by the possibility of uncovering deeper patterns in the biological realm using the help of computer algorithms. Yet I don't know much about the current state of this science. As I am considering a post-graduate degree in bioinformatics, I am interested to dig deeper into this subject and check whether the subject indeed aligns with my interest. So I start with the basic question stated in the title: What are some of the major questions pursued by the field bioinformatics? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance. &lt;/p&gt;&#xA;" OwnerUserId="3555" LastActivityDate="2018-10-23T08:35:42.070" Title="What are some of the major questions pursued by the field bioinformatics?" Tags="|algorithms|" AnswerCount="1" CommentCount="1" ClosedDate="2018-10-23T09:44:31.623" ContentLicense="CC BY-SA 4.0" />
  <row Id="5262" PostTypeId="2" ParentId="5261" CreationDate="2018-10-23T08:35:42.070" Score="2" Body="&lt;p&gt;Welcome! Bioinformatics is just a term that describes an approach to doing biology. People who use ( DNA sequencing/genomics/protein informatics + programming ) to answer biological questions are after the same answers as the the people who use ( pipettes/microscopes/molecular biology + a lab notebook ).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My best suggestion is to learn about some biological concepts that are interesting to you, see how people have applied bioinformatics to that branch of biology, and think if there is something you might be interested in doing to work on that question. It is critical to have a strong understanding of the underlying biology to get to the core of new findings using bioinformatics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A lot of the hardcore algorithm development going on today is in genome assembly with long + noisy DNA sequencing reads, aligning long+noisy reads to genomes and trying to figure out the differences between the &lt;a href=&quot;https://en.wikipedia.org/wiki/Haplotype&quot; rel=&quot;nofollow noreferrer&quot;&gt;maternal and paternal haplotypes&lt;/a&gt;, working on evolutionary questions with population-level genetic variation data, and figuring out how things are related to each other with phylogenetics, to name a few.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as a machine-learning type approach which you seem to be hinting at in your question, one very hot topic at the moment is single-cell sequencing. This is a set of technologies that allows us to get DNA, RNA, or protein sequencing from a population of single cells to look at heterogeneity in tissues, timepoints, et cetera.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck! If you have some free time on your hand &lt;a href=&quot;http://rosalind.info/problems/locations/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this website, ROSALIND, is a fun project Euler-type website where you can learn some biology while programming bioinformatics solutions.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-10-23T08:35:42.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5263" PostTypeId="2" ParentId="5220" CreationDate="2018-10-23T08:43:06.453" Score="1" Body="&lt;p&gt;It looks like &lt;code&gt;lumpy&lt;/code&gt; &lt;a href=&quot;https://github.com/arq5x/lumpy-sv&quot; rel=&quot;nofollow noreferrer&quot;&gt;looks for breakpoints in your reference&lt;/a&gt;. Presumably places where the is a haplotype switch detectable with the data on-hand?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My only thought is that since &lt;code&gt;lumpy&lt;/code&gt; is being used to produce the VCF file that it is inherently not looking for insertions. I would suggest running the same BAM file through the GATK best practices pipeline (easier said than done) and vcftools pipeline. If those produce insertions but &lt;code&gt;lumpy&lt;/code&gt; does not, it must be due to how &lt;code&gt;lumpy&lt;/code&gt; produces the calls. Good luck!&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-10-23T08:43:06.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5264" PostTypeId="2" ParentId="2547" CreationDate="2018-10-23T15:07:38.283" Score="2" Body="&lt;p&gt;The figure caption says &quot;the top 40 genes per celltype&quot; - so we can simply use the code you provided for the top 40 genes for a single celltype, repeat this for all celltypes, and then remove any duplicate genes. Here's an example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; expr &amp;lt;- matrix(runif(9*24000), nrow=24000, ncol=9)&#xA;&amp;gt; top_genes_each_celltype &amp;lt;- sapply(1:ncol(expr), function(i) order(expr[,i],decreasing =T)[1:40])&#xA;&amp;gt; dim(top_genes_each_celltype)&#xA;[1] 40 9&#xA;&amp;gt; top_genes_all_celltypes &amp;lt;- unique(as.vector(top_genes_each_celltype))&#xA;&amp;gt; length(top_genes_all_celltypes)&#xA;[1] 358&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you can build a biclustered heatmap with &lt;code&gt;expr[top_genes_all_celltypes,]&lt;/code&gt;, for example, using &lt;code&gt;heatmap2&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="163" LastActivityDate="2018-10-23T15:07:38.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5265" PostTypeId="1" CreationDate="2018-10-24T02:02:38.170" Score="2" ViewCount="47" Body="&lt;p&gt;I am in search of past years' versions of NT and 16s databases from NCBI.  I recently contacted NCBI about this and they replied:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Unfortunately, we do not archive past versions of the blast databases. We update them regularly and simply do not have the resources to store older versions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Any older versions would be OK, but if you happened to have 2012 and 2007 those are preferable.&lt;/p&gt;&#xA;" OwnerUserId="2446" LastEditorUserId="48" LastEditDate="2018-10-24T07:13:06.487" LastActivityDate="2018-10-24T07:13:06.487" Title="Does anyone have older versions of NT and 16s RNA DB from NCBI?" Tags="|database|blast|phylogenetics|data-management|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5266" PostTypeId="2" ParentId="5060" CreationDate="2018-10-24T02:03:11.790" Score="0" Body="&lt;p&gt;After trying a few different queries, I don't believe there is a more specific one than this and I found that by searching and clicking through all of the results given by this query that eventually I found patients that fit the criteria of having both WGS and 16s.&lt;/p&gt;&#xA;" OwnerUserId="2446" LastActivityDate="2018-10-24T02:03:11.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5267" PostTypeId="1" CreationDate="2018-10-24T10:17:41.380" Score="3" ViewCount="163" Body="&lt;p&gt;I am trying to find details for how the ENCODE ChIP seq samples were processed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Linking to a ChIPseq samples on www.encodeproject.org (e.g. &lt;a href=&quot;https://www.encodeproject.org/experiments/ENCSR051DXE/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.encodeproject.org/experiments/ENCSR051DXE/&lt;/a&gt;) offers a PDF &quot;overview&quot; of the pipeline which says basically nothing. The page will also offer you a map of the pipeline, which basically just has headings:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/vVFyS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vVFyS.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clicking on a step gives you a popup with a link to the &quot;&lt;a href=&quot;https://www.encodeproject.org/pipelines/ENCPL220NBH/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chip-seq read mapping&lt;/a&gt;&quot; pipeline. This just contains the same overview PDF and a link to &quot;&lt;a href=&quot;https://www.encodeproject.org/chip-seq/transcription_factor/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ChIP-seq Data Standards and Processing Pipeline&lt;/a&gt;&quot;. This page contains again, a vague outline of the pipeline, a link back to the page we just came from and a list of quality control thresholds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't seem to find anywhere the proper details. For example, if BWA was used, was it MEM or aln? What parameters were used for the mapping? One of the steps is read filtering: what where the filters? How were they applied? There is a link to a &lt;a href=&quot;https://github.com/ENCODE-DCC/chip-seq-pipeline/blob/master/dnanexus/chip_workflow.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;github page&lt;/a&gt; for running the pipeline on DNAnexus, but I can't make head nor tail of it. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2024-01-17T18:07:21.157" Title="Details of ENCODE processing pipelines" Tags="|encode|reproducibility|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5268" PostTypeId="1" AcceptedAnswerId="5271" CreationDate="2018-10-24T10:17:59.727" Score="1" ViewCount="709" Body="&lt;p&gt;I am going to use principal components (PCs) comes from calcPCA function in URD program for clustering my cells in Seurat; So&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seurat@dr$pca=URD_object@pca.load&#xA;&#xA;&amp;gt; seurat &amp;lt;- FindClusters(&#xA;+     object = seurat, &#xA;+     reduction.type = &quot;pca&quot;, &#xA;+     dims.use = 1:8, &#xA;+     resolution = 1.0, &#xA;+     print.output = 0, &#xA;+     save.SNN = TRUE&#xA;+ )&#xA;Error in GetDimReduction(object = object, reduction.type = reduction.type,  : &#xA;  cell.embeddings slot doesn't exist&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There is &lt;code&gt;seuratToURD&lt;/code&gt; function but there is not URD to seurat function I guess so I could use URD object for clustering in seurat&#xA;How I can overcome this error please? I want to see when I am using PCA from URD what would be my clustering by SNN algorithm in seurat&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot for any help&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2018-10-24T15:20:28.790" Title="Using external list of PCs for clustering" Tags="|seurat|clustering|pca|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5269" PostTypeId="1" AcceptedAnswerId="5325" CreationDate="2018-10-24T11:18:18.447" Score="3" ViewCount="84" Body="&lt;h1&gt;edit explaining python tag&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I would still rather have a solution based on the RSVSim package in R, but while waiting for someone who might have an answer I wanted to look at other solutions as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a result like what metadata(sim) produces can be achieved in python as well, I can do it without RSVSim. I need especially the numbers given there, so I do NOT need &quot;Copied BpSeqA BpSeqB_5prime BpSeqB_3prime&quot;. The numbers need to reflect the original position of insert into chr2, because I need to reflect this back to the original (unedited) chr2.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;end edit explaining pyton tag&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I tried using the BioConductor forums for asking this question &lt;a href=&quot;https://support.bioconductor.org/p/114354/&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;question on BioConductor&quot;&gt;Question on BioConductor&lt;/a&gt;. However I do not seem to get an answer there and after some help with other tools on this website, i thought I could also ask the question here, in the hope that someone knows something about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a reference genome, and I need to put in some structural variations. It is really easy to do in RSVSim, and used it quite a lot already.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when processing INS, I found out that it just copies a DNA seq from one part, and puts it somewhere else in the same DNA seq (Bascially a DUP then), or if you specify another an extra DNAseq, to cut from, It randomly cuts from either chr1 or chr2 and puts it in the other. This creates deletions in the DNA seq I want to investigate, which is not what I want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know how to get this kind of result?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;width seq                                                                                                                           names               &#xA;[1]    40 AAAAAAAAAAAAAAAAAAAATTTTTTTTTTTTTTTTTTTT    chr1&#xA;[2]    40 GGGGGGGGGGGGGGGGGGGGCCCCCCCCCCCCCCCCCCCC    chr2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;width seq                                                                                                                           names               &#xA;[1]    32 AAAAAAAAAAAAAAAATTTTTTTTTTTTTTTT                    chr1&#xA;[2]    48 GGGGGGGGGGGGGGAAAATTTTGGGGGGCCCCCCCCCCCCCCCCCCCC    chr2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And do it 30 times, randomly in the genome? (so take a seq from chr1 and put an insertion in chr2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried this and more already for days, and I just cannot figure it out. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    library(RSVSim)    &#xA;&#xA;    seq_random &amp;lt;- readDNAStringSet(&quot;/path/random.fasta&quot;, &quot;fasta&quot;)&#xA;    seq_ref &amp;lt;- readDNAStringSet(&quot;/path/reference1MB.fasta&quot;, &quot;fasta&quot;)&#xA;    genome2 = DNAStringSet(c(seq_random, seq_ref))&#xA;    names(genome2) = c(&quot;chr1&quot;,&quot;chr2&quot;)&#xA;&#xA;    y &amp;lt;- c(30, 30, 30, 30, 30, 50, 50, 50, 50, 50, 100, 100, 100, 100, 100, 500, 500, 500, 500, 500, 1000, 1000, 1000, 1000, 1000, 5000, 5000, 5000, 5000, 5000)&#xA;&#xA;    length_seq = width(genome2[2])&#xA;    knownInsertion = GRanges(IRanges(0,length_seq), seqnames=&quot;chr2&quot;)&#xA;    knownInsertion = GRanges(IRanges(0,length_seq), seqnames=&quot;chr1&quot;, chrB=&quot;chr2&quot;, chrA=&quot;chr1&quot;)&#xA;    knownInsertion = GRanges(seqnames=&quot;chr1&quot;, chrB=&quot;chr2&quot;)&#xA;    sim = simulateSV(output='/folder/of/output/', genome=genome2, &#xA;                     ins = 30, sizeIns=y, regionsIns=knownInsertion)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;EDIT for comment section&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The simple genome example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;genome = DNAStringSet(c(&quot;AAAAAAAAAAAAAAAAAAAATTTTTTTTTTTTTTTTTTTT&quot;,&quot;GGGGGGGGGGGGGGGGGGGGCCCCCCCCCCCCCCCCCCCC&quot;))&#xA;names(genome) = c(&quot;chr1&quot;,&quot;chr2&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I believe I already tried those parameters. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;knownInsertion = GRanges(IRanges(16,25), seqnames=&quot;chr1&quot;, chrB=&quot;chr2&quot;)&#xA;sim = simulateSV(output=NA, genome=genome, ins = 1, regionsIns=knownInsertion, bpSeqSize=6, random=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;results in:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; sim&#xA;  A DNAStringSet instance of length 2&#xA;    width seq                                                                                                                           names               &#xA;[1]    40 AAAAAAAAAAAAAAAAAAAATTTTTTTTTTTTTTTTTTTT                                                                                      chr1&#xA;[2]    40 GGGGGGGGGGGGGGGGGGGGCCCCCCCCCCCCCCCCCCCC                                                                                      chr2&#xA;&amp;gt; metadata(sim)&#xA;$insertions&#xA;         Name ChrA StartA EndA ChrB StartB EndB Size Copied BpSeqA BpSeqB_5prime BpSeqB_3prime&#xA;1 insertion_1 chr1      3   12 chr1      6   15   10  FALSE  AAAAA        AAAAAA        AAAAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;so nothing happened at all.&#xA;If I keep it simple, you get the result I mentioned before, where it is sometimes chr1-&gt;chr2 and sometimes chr2&amp;lt;-chr1&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sim = simulateSV(output=NA, genome=genome, ins = 3, sizeIns = 5, bpSeqSize=6, random=TRUE)&#xA;&#xA;&amp;gt; sim&#xA;  A DNAStringSet instance of length 2&#xA;    width seq                                                                                                                           names               &#xA;[1]    40 AAAAAACCCCCAAAAATTTTTTTTTTTTTTAAAATTTTTT                                                                                      chr1&#xA;[2]    40 GGGGGGGGAAAAAGGGGGGGGGGGGCCCCCCCCCCCCCCC                                                                                      chr2&#xA;&amp;gt; metadata(sim)&#xA;$insertions&#xA;         Name ChrA StartA EndA ChrB StartB EndB Size Copied BpSeqA BpSeqB_5prime BpSeqB_3prime&#xA;1 insertion_1 chr1     17   21 chr1     36   40    5  FALSE AAATTT        TTTAAA        AATTTT&#xA;2 insertion_2 chr1      1    5 chr2      9   13    5  FALSE               GGGAAA        AAAGGG&#xA;3 insertion_3 chr2     33   37 chr1     12   16    5  FALSE CCCCCC        AAACCC        CCCAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this case I dont even know what happened exactly, because an uneven number of Ins, should make one of the chromosomes shorter or longer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, when setting copy on (so no cut/paste)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sim = simulateSV(output=NA, genome=genome, ins = 3, sizeIns = 5, bpSeqSize=6, random=TRUE, percCopiedIns=1.00)&#xA;sim&#xA;metadata(sim)&#xA;&#xA;&amp;gt; sim&#xA;  A DNAStringSet instance of length 2&#xA;    width seq                                                                                                                           names               &#xA;[1]    50 AAAATTTTTAAAAAAAAAAAAAAAATTTTTTATTTTTTTTTTTTTTTTTT                                                                            chr1&#xA;[2]    45 GGGGGGGGGGGGGGGGGAAAAAGGGCCCCCCCCCCCCCCCCCCCC                                                                                 chr2&#xA;&amp;gt; metadata(sim)&#xA;$insertions&#xA;         Name ChrA StartA EndA ChrB StartB EndB Size Copied BpSeqA BpSeqB_5prime BpSeqB_3prime&#xA;1 insertion_1 chr1     36   40 chr1      5    9    5   TRUE               AAATTT        TTTAAA&#xA;2 insertion_2 chr1     20   24 chr1     27   31    5   TRUE               TTTATT        TTTTTT&#xA;3 insertion_3 chr1     13   17 chr2     18   22    5   TRUE               GGGAAA        AAAGGG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Again. all the examples above have random chrA and chrB. Setting these specifically, only seems possible with the GRanges and IRanges, for which I tried tons of versions already.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should be a very simple thing according to the manual, but actually doing it doesn't seem to get the desired results, while my version of Insertions is just the normal one in my opinion? (why would you want to create extra DELS or DUPS while making insertions?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On their own support page, it is only possible to ask a question on the forums, but they never answer it (only look at it)&lt;/p&gt;&#xA;" OwnerUserId="3552" LastEditorUserId="3552" LastEditDate="2018-10-26T11:33:02.520" LastActivityDate="2018-10-30T12:39:37.670" Title="RSVSim insertions from chr1 to chr2" Tags="|r|python|bioconductor|structural-variation|simulated-data|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5270" PostTypeId="2" ParentId="5132" CreationDate="2018-10-24T14:28:40.923" Score="1" Body="&lt;p&gt;I inquired into the details of the dendrograms after becoming frustrated with the lack of information. As with Ensembl, I'm sure that the folks at NCBI have a standardized pipeline that they run the sequences through to generate these dendrograms as no specific source deserving attribution was mentioned:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The tree is based on a pairwise, BLAST comparison of chromosome sequences (an assembly is used for genomes with 2 or more chromosomes). BLAST identity is normalized on sequence length. The distance map is generated by a Neighbor Joining algorithm, where the distance is the blast score.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Dr. Wayne Matten, NCBI&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="181" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-10-24T14:28:40.923" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5271" PostTypeId="2" ParentId="5268" CreationDate="2018-10-24T14:50:10.730" Score="3" Body="&lt;p&gt;There is a way to do this, and even better--there is documentation for how to do it! No surprise coming from the Satija Lab. In the vignette they perform multidimensional scaling, but the idea is the same. &lt;code&gt;cmdscale()&lt;/code&gt; returns the cell embeddings. &lt;code&gt;SetDimReduction()&lt;/code&gt; is the Seurat function you are looking for. No manual editing using &lt;code&gt;@&lt;/code&gt; required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors use add mds embeddings, but if you want to add your own pca embeddings swap out &lt;code&gt;&quot;mds&quot;&lt;/code&gt; for &lt;code&gt;&quot;pca&quot;&lt;/code&gt; and &lt;code&gt;mds&lt;/code&gt; for &lt;code&gt;URD_object@pca.load&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Before running MDS, we first calculate a distance matrix between all pairs&#xA;# of cells.  Here we use a simple euclidean distance metric on all genes,&#xA;# using object@scale.data as input&#xA;d &amp;lt;- dist(x = t(x = pbmc@scale.data))&#xA;# Run the MDS procedure, k determines the number of dimensions&#xA;mds &amp;lt;- cmdscale(d = d, k = 2)&#xA;# cmdscale returns the cell embeddings, we first label the columns to ensure&#xA;# downstream consistency&#xA;colnames(x = mds) &amp;lt;- paste0(&quot;MDS&quot;, 1:2)&#xA;# We will now store this as a new dimensional reduction called 'mds'&#xA;pbmc &amp;lt;- SetDimReduction(object = pbmc, reduction.type = &quot;mds&quot;, slot = &quot;cell.embeddings&quot;, &#xA;    new.data = mds)&#xA;pbmc &amp;lt;- SetDimReduction(object = pbmc, reduction.type = &quot;mds&quot;, slot = &quot;key&quot;, &#xA;    new.data = &quot;MDS&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://satijalab.org/seurat/dim_reduction_vignette.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the dimensional reduction vignette&lt;/a&gt; for more.&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="181" LastEditDate="2018-10-24T15:20:28.790" LastActivityDate="2018-10-24T15:20:28.790" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5272" PostTypeId="2" ParentId="5046" CreationDate="2018-10-24T15:42:19.490" Score="0" Body="&lt;p&gt;Much of the Seurat workflow incorporates normalized, scaled, and mean-centered expression values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Raw gene expression counts for a cell are normalized by total expression, multiplied by 10,000, and log transformed. Expression level for each gene is scaled by dividing the centered gene expression levels by their standard deviations, resulting in z-scores. It's not an ideal solution, but I find a threshold of 1 (in the realm of z-scores) to make sense. A z-score of 1 is one standard deviation from the mean, or what may be interpreted as baseline expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my 2¢. I would love to hear what other people are doing.&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2018-10-24T15:42:19.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5273" PostTypeId="1" CreationDate="2018-10-24T16:58:57.630" Score="3" ViewCount="131" Body="&lt;p&gt;I want to calculate the homoplasy excess ratio for a phylogenetic tree, for example homoplasy resulting from recombination.&lt;/p&gt;&#xA;&lt;p&gt;I am wondering if there are any R packages out there that will do this?&lt;/p&gt;&#xA;&lt;p&gt;I have looked at phyloseq, Rphylopars, phylocurve, PhylogeneticEM, PhyloinformR, but these packages do not have functions that calculate HER. I searched &amp;quot;homoplasy excess ratio&amp;quot; on bioconductor, but it returned ca. 3000 packages.&lt;/p&gt;&#xA;" OwnerUserId="2961" LastEditorUserId="73" LastEditDate="2021-05-05T12:59:40.787" LastActivityDate="2021-05-05T12:59:40.787" Title="Is there an R package that computes homoplasy excess ratios (HER)?" Tags="|r|phylogenetics|phylogeny|covid-19|sars-cov-2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5274" PostTypeId="1" AcceptedAnswerId="5275" CreationDate="2018-10-25T02:11:28.383" Score="1" ViewCount="3691" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sv8xr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sv8xr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FastQC of my sample files, aggregated into a single plot by MultiQC. Blue represents unique reads. Black represents duplicate reads. The x-axis is the number of reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see I have more duplicate reads than unique reads, how do I interpret this plot? This is my attempt:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;FastQC took the first 100,000 sequences in each sample (&lt;a href=&quot;https://www.bioinformatics.babraham.ac.uk/projects/fastqc/Help/3%20Analysis%20Modules/8%20Duplicate%20Sequences.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;details&lt;/a&gt;).&#xA;  Generally, higher duplicated read percentage is welcome as it&#xA;  represents better sequencing depth and lower sequencing error&#xA;  contamination. This dsitribution in this plot looks reasonable and&#xA;  nothing unusual.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="174" LastEditorUserId="174" LastEditDate="2018-10-25T02:19:13.737" LastActivityDate="2018-10-25T02:51:44.017" Title="How to interpret duplication from MultiQC/FastQC?" Tags="|fastqc|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5275" PostTypeId="2" ParentId="5274" CreationDate="2018-10-25T02:51:44.017" Score="0" Body="&lt;p&gt;I would agree with your statement.  Typically the ratio of unique to duplicated reads is a function of input library complexity (number of input molecules) verse sequencing depth (the amount of molecules sequenced).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the example you gave you have about a 2:1 ratio of duplicates to unique reads which suggests that these libraries have been sequenced to their fullest and additional sequencing would yield few new unique reads.&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2018-10-25T02:51:44.017" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5276" PostTypeId="2" ParentId="2580" CreationDate="2018-10-25T02:54:00.713" Score="2" Body="&lt;p&gt;It seems that the author of CRCmapper.py wants to extract &lt;code&gt;sequence_name&lt;/code&gt;, &lt;code&gt;start&lt;/code&gt;, &lt;code&gt;stop&lt;/code&gt; columns from &lt;code&gt;fimo.txt&lt;/code&gt; file. But the code extract &lt;code&gt;motif_alt_id&lt;/code&gt;, &lt;code&gt;sequence_name&lt;/code&gt;, &lt;code&gt;start&lt;/code&gt; columns. So the index of the code should be changed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;fimo.txt&lt;/code&gt; looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;motif_id    motif_alt_id    sequence_name   start   stop    strand  score   p-value q-value matched_sequence&#xA;Transfac.V$AP1_02   AP1 NM_003046931|3|95762|152118 2332    3118    -   11.9802 6.49e-05        TGACTCA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In line 377-382 of the code:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;line[1]&lt;/code&gt; should be changed to &lt;code&gt;line[2]&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for line in fimoTable[1:]:&#xA;    source = motifDatabaseDict[line[0]]&#xA;    region = line[1].split('|')&#xA;    target = refseqToNameDict[region[0]]&#xA;    location = (region[1], int(region[2]), int(region[3]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In line 391-394 of the code:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;line[2]&lt;/code&gt; should be changed to &lt;code&gt;line[3]&lt;/code&gt;, &lt;code&gt;line[3]&lt;/code&gt; should be changed to &lt;code&gt;line[4]&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    # Count unique motifs&#xA;    if (region[1], int(region[2]) + int(line[2]), int(region[2]) + int(line[3])) not in motifDictSE[source]:&#xA;        edgeCountDictSE[(source, target)] += 1&#xA;        motifDictSE[source].append((region[1], int(region[2]) + int(line[2]), int(region[2]) + int(line[3])))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3563" LastEditorUserId="964" LastEditDate="2018-10-25T03:01:17.470" LastActivityDate="2018-10-25T03:01:17.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5277" PostTypeId="1" CreationDate="2018-10-25T03:08:47.363" Score="-1" ViewCount="37" Body="&lt;p&gt;I have a set of eight gene clusters which I obtained from clustering a network. However, I had to update the genes in the network at a later stage which lead to certain deletions (genes not present in the new gene list). I did the clustering again and this time there were a total of 9 clusters. Is there some way in which I could compare the earlier eight clusters with the new nine ones to effectively say that &quot;clustering has happened along the same lines&quot;. It would have been easier to do an eight to eight comparison checking from the largest to the smallest cluster. However, since I have nine clusters now, I am kinda in a fix as how to compare.&#xA;I used the same clustering algorithm to cluster the network in both the cases&lt;/p&gt;&#xA;" OwnerUserId="976" LastActivityDate="2021-09-07T00:02:45.317" Title="Comparing between different set of clusters produced from the same network" Tags="|networks|clustering|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5278" PostTypeId="1" CreationDate="2018-10-25T03:49:14.117" Score="4" ViewCount="1161" Body="&lt;p&gt;To calculate the retention index for a phylogenetic tree, we use the following formula:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\text{maximum number of steps on tree - number of steps on the tree}}&#xA;{\text{maximum number of steps on tree - minimum number of steps in the data}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To calculate the maximum number of steps, we look at the number of observed character states for a character and take the lowest number. For each character, we take the lowest number of observed traits and then sum across all characters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Take the following dataset (columns are characters, taxa are rows):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;          123456&#xA;Outgroup  000000&#xA;Taxa A    100001&#xA;Taxa B    110001&#xA;Taxa C    111001&#xA;Taxa D    111101&#xA;Taxa E    111111&#xA;Taxa F    111110&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The maximum number of steps for the first character is 1, since the character state 0 is attested once and character state 1 occurs six times. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I don't understand is why we do not use 6 for the maximum number of changes for the first character. &lt;/p&gt;&#xA;" OwnerUserId="2961" LastEditorUserId="2961" LastEditDate="2018-10-25T15:55:39.260" LastActivityDate="2020-05-26T10:28:48.943" Title="In calculating the retention index, why do we use the character state with the lowest frequency?" Tags="|phylogenetics|phylogeny|taxonomy|quantification|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5279" PostTypeId="1" CreationDate="2018-10-25T04:54:29.783" Score="1" ViewCount="652" Body="&lt;p&gt;I'm working with a Daphnia data set looking at the 16s gene pulled from the BOLD database. So far I did &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a multiple sequence alignment and &lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;attempted to cluster and plot a dendrogram:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;myClustalWAlignment &amp;lt;- DNAStringSet(msa(MySeqString, cluster=&quot;nj&quot;, gapOpening=30.0,  gapExtension=20.0, maxiters=2, substitutionMatrix=&quot;default&quot;, type=&quot;default&quot;, verbose=TRUE))&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Replace full sequence names with Accession numbers for easy identification&lt;/p&gt;&#xA;&#xA;&lt;p&gt;names(myClustalWAlignment) &amp;lt;- Daphnia_DF$Accession_number&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Converts the alignment object into a DNAbin class. Some of the results are shown and may be useful for further analysis:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ClustalAlignment_dnaBin &amp;lt;- as.DNAbin(myClustalWAlignment)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Creating a distance matrix:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;distmat &amp;lt;- dist.dna(ClustalAlignment_dnaBin, model = &quot;TN93&quot;, as.matrix = TRUE, pairwise.deletion = TRUE)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;clustering the sequences:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Daphnia_Clustered &amp;lt;- IdClusters(distmat,&#xA;                      method = &quot;UPGMA&quot;,&#xA;                      cutoff = 0.01,&#xA;                      showPlot = TRUE,&#xA;                      type = &quot;both&quot;,&#xA;                      myXStringSet = NULL,&#xA;                      model = MODELS,&#xA;                      collapse = 0,&#xA;                      processors = NULL,&#xA;                      verbose = TRUE)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;checking to see the number of clusters:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;length(unique(unlist(Daphnia_Clustered&lt;a href=&quot;https://i.stack.imgur.com/s0dId.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;a href=&quot;https://i.stack.imgur.com/s0dId.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;&lt;/a&gt;)))&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;output: 80&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that I want a dendrogram showing just the 80 clusters if that's possible. Right now the output of the above clustering is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/s0dId.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/s0dId.png&quot; alt=&quot;dendrogram&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to get a smaller number of clusters? Or even being able to plot just the 80 OTU clusters would be great. Is there an easy way of reconstructing the phylogenetic relationship between these clusters?&lt;/p&gt;&#xA;" OwnerUserId="3477" LastEditorUserId="3848" LastEditDate="2020-05-07T08:31:18.607" LastActivityDate="2020-05-07T08:31:18.607" Title="How to create a dendrogram of clusters and reconstruct phylogenetic relationship" Tags="|sequence-alignment|phylogenetics|phylogeny|ggplot2|clustering|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5280" PostTypeId="1" CreationDate="2018-10-25T07:20:41.390" Score="7" ViewCount="333" Body="&lt;p&gt;&lt;strong&gt;My Problem:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Skipping some specific background, what I want to do is &lt;strong&gt;judging whether some soft-clipping sequences are the same&lt;/strong&gt;, which may result by the same SV event. Colored bases in Fig.1 is an example of soft-clipping sequences.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lGaR6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lGaR6.png&quot; alt=&quot;Fig.1 an example of soft-clipping sequences&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I use &lt;code&gt;BWA -MEM&lt;/code&gt; as the aligner. And I am trying to develop an SV caller actually, what I want to do here is refining PE(Paired-Ends) signal. If it is SR(Split-Reads), I don't need to do this, because of SR and its supplemental alignments coming from the same read. But PE signal generated from some cluster procedures need some filters. And I think K-mer is not a good way to do this which adapted by some SV callers. For me, as a bioinformatic fool, imitation and implementation may be a good learning process. It seems still a long way to go.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My solution&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;I suppose this is a sequence alignment problem. Then I use FM-indexing based on BWT to do this. My inexact/approximate matching implementation is referred from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/19451168&quot; rel=&quot;nofollow noreferrer&quot;&gt;BWA&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My workflow&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Extract these soft-clipping sequences&lt;/li&gt;&#xA;&lt;li&gt;Choose the longest one as the reference&lt;/li&gt;&#xA;&lt;li&gt;Iterate others to match with the reference, if the number of unmatched is large than 1, it means these sequences are not the same.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Question 1:&#xA;Is that sounds reasonable to you?  Any suggestions?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Question 2:&#xA;If it is OK, how should I implement &lt;strong&gt;the penalty scoring scheme&lt;/strong&gt;?&lt;br /&gt;&#xA;Here, because these sequences should come from the same SV event, the main differences(Mismatch, InDel) should arise from sequencing errors, not small genomic variants. So I want to set a higher gap open penalty and even higher gap extend penalty, like&lt;/p&gt;&#xA;&lt;p&gt;mismatch_penalty = 4&#xA;gapopen_penalty = 11&#xA;gapextend_penalty = 20&#xA;alignment penalty cutoff = 30&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The later means that one alignment with two more gaps will be considered as spurious alignment.&lt;/p&gt;&#xA;&lt;p&gt;How do I optimize these parameters?&lt;/p&gt;&#xA;" OwnerUserId="1476" LastEditorUserId="73" LastEditDate="2021-06-28T09:18:33.713" LastActivityDate="2021-11-25T10:03:50.280" Title="Sequence alignment using BWT" Tags="|sequence-alignment|bwa|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5281" PostTypeId="1" AcceptedAnswerId="5283" CreationDate="2018-10-25T07:48:37.547" Score="2" ViewCount="4365" Body="&lt;p&gt;I have RNA-Seq data which is &lt;code&gt;FPKM&lt;/code&gt;. In the dataframe &lt;code&gt;df&lt;/code&gt; first column is &lt;code&gt;gene_name&lt;/code&gt; and the other 100 columns are &lt;code&gt;samples&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually if it is &lt;code&gt;counts&lt;/code&gt; data I do like following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df2 &amp;lt;- aggregate(. ~ gene_name, data = df, max)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm not sure what do with the &lt;code&gt;FPKM&lt;/code&gt; data if there are duplicate genes with different &lt;code&gt;FPKM&lt;/code&gt; value for the same sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets say it looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gene_name     sample1        sample2        sample3&#xA;5S_rRNA      0.3206147    0.3327312      0.377578&#xA;5S_rRNA      0.3342000    0.0000000      0.1305166&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any suggestions please.&lt;/p&gt;&#xA;" OwnerUserId="1369" LastActivityDate="2018-10-25T11:13:11.407" Title="How to deal with duplicate genes having different expression values?" Tags="|r|rna-seq|gene-expression|fpkm|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5282" PostTypeId="2" ParentId="5281" CreationDate="2018-10-25T09:44:56.603" Score="3" Body="&lt;p&gt;The best way to deal with this is to use unique gene IDs, for example ensembl accession numbers. So use the ensemble gtf annotation when quantifying the read counts and not the gene symbols. Just to illustrate, when I look for &quot;5S_rRNA&quot; in ensembl's annotation, i see 18 different &quot;genes&quot; with that gene symbol. But which 2 you have is unclear now.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep &quot;5S_rRNA&quot; ensembl_symbol.txt&#xA;&quot;ENSG00000252830&quot;       &quot;5S_rRNA&quot;&#xA;&quot;ENSG00000276442&quot;       &quot;5S_rRNA&quot;&#xA;&quot;ENSG00000274408&quot;       &quot;5S_rRNA&quot;&#xA;&quot;ENSG00000274059&quot;       &quot;5S_rRNA&quot;&#xA;&quot;ENSG00000276861&quot;       &quot;5S_rRNA&quot;&#xA;&quot;ENSG00000274759&quot;       &quot;5S_rRNA&quot;&#xA;&quot;ENSG00000280646&quot;       &quot;5S_rRNA&quot;&#xA;&quot;ENSG00000277411&quot;       &quot;5S_rRNA&quot;&#xA;&quot;ENSG00000201285&quot;       &quot;5S_rRNA&quot;&#xA;&quot;ENSG00000212595&quot;       &quot;5S_rRNA&quot;&#xA;&quot;ENSG00000277418&quot;       &quot;5S_rRNA&quot;&#xA;&quot;ENSG00000277049&quot;       &quot;5S_rRNA&quot;&#xA;&quot;ENSG00000274097&quot;       &quot;5S_rRNA&quot;&#xA;&quot;ENSG00000277488&quot;       &quot;5S_rRNA&quot;&#xA;&quot;ENSG00000274663&quot;       &quot;5S_rRNA&quot;&#xA;&quot;ENSG00000283433&quot;       &quot;5S_rRNA&quot;&#xA;&quot;ENSG00000275305&quot;       &quot;5S_rRNA&quot;&#xA;&quot;ENSG00000278457&quot;       &quot;5S_rRNA&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2018-10-25T11:13:11.407" LastActivityDate="2018-10-25T11:13:11.407" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5283" PostTypeId="2" ParentId="5281" CreationDate="2018-10-25T09:57:56.657" Score="5" Body="&lt;p&gt;I assume you're familiar with the various issues surrounding FPKMs, so I'll not expound upon them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a general rule, you should be using gene IDs rather than gene names, since the former are unique while the latter are not. If you only have access to data quantified on gene names, then the appropriate way to merge RPKMs is with a weighted sum:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$FPKM_{gene} = \frac{FPKM_{copy1} * Length_{copy1} + FPKM_{copy2} * Length_{copy2}}{Length_{copy1} + Length_{copy2}}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an aside, rRNA expression levels will tend to be wrong, since they're normally excluded by either poly-A enrichment or the use of ribo-zero.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-10-25T09:57:56.657" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="5284" PostTypeId="2" ParentId="5278" CreationDate="2018-10-25T14:48:24.350" Score="1" Body="&lt;p&gt;The maximum number of steps (or changes) is the number of taxa with state 1 or 0, whichever is smaller. Conceptually, this is the number of character changes if each taxon evolved its state independently of the other taxa. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If for a given character the character state 0 occurred twice and character state 1 occurred five times, 2 would be the maximum number of changes.&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="181" LastEditDate="2018-10-25T15:15:54.637" LastActivityDate="2018-10-25T15:15:54.637" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5286" PostTypeId="1" CreationDate="2018-10-25T19:31:43.727" Score="1" ViewCount="94" Body="&lt;p&gt;I'm quite new to Bioconductor and GEO.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's my task: I've to develop a simple R script able to read a gene expression dataset on GEO and extract the genes related to it. But... I don't know the command to extract the gene list. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The GEO dataset I want to analyze is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE59867&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSE59867&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's my R code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;options(stringsAsFactors = FALSE)&#xA;&#xA;library(oligo)&#xA;library(GEOquery)&#xA;source(&quot;https://bioconductor.org/biocLite.R&quot;)&#xA;biocLite(&quot;affyio&quot;)&#xA;library(&quot;affyio&quot;)&#xA;&#xA;getGEOSuppFiles(&quot;GSE59867&quot;)&#xA;list.files(&quot;GSE59867&quot;)&#xA;&#xA;untar(&quot;GSE59867/GSE59867_RAW.tar&quot;, exdir = &quot;GSE59867/CEL&quot;)&#xA;list.files(&quot;GSE59867/CEL&quot;)&#xA;&#xA;celfiles &amp;lt;- list.files(&quot;GSE59867&quot;, full = TRUE)&#xA;rawData &amp;lt;- read.celfiles(celfiles)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now I have the &lt;code&gt;rawData&lt;/code&gt; containing the gene expressions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How can I extract the list of genes associated to rawData at this point?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1499" LastActivityDate="2018-10-25T19:31:43.727" Title="Bioconductor, how to retrieve the genes from a GEO gene expression dataset?" Tags="|r|bioconductor|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5287" PostTypeId="1" AcceptedAnswerId="5291" CreationDate="2018-10-25T22:11:35.860" Score="3" ViewCount="6225" Body="&lt;p&gt;I am working on Mac OS X. I am using R version 3.5.1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have important a FASTA file into R using &lt;code&gt;Biostrings::readDNAStringSet&lt;/code&gt;. This creates a DNAStringSet object in R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to transform it into a data frame so that I can then plot the widths with ggplot2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;as.data.frame&lt;/code&gt; on my DNAStringSet object returns a data frame with only one column. Is there a function that returns a data frame with columns &lt;code&gt;width&lt;/code&gt;, &lt;code&gt;seq&lt;/code&gt; and &lt;code&gt;names&lt;/code&gt; present in the DNAStringSet object?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2018-10-26T08:42:10.457" Title="How to transform a DNAStringSet from the Bioconductor package Biostrings to a data frame?" Tags="|r|bioconductor|format-conversion|biostrings|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5288" PostTypeId="1" CreationDate="2018-10-26T01:12:18.023" Score="1" ViewCount="416" Body="&lt;p&gt;I am trying to run &lt;code&gt;TMM&lt;/code&gt; normalization using &lt;code&gt;rpy2&lt;/code&gt; and when I run &lt;code&gt;calcNormFactors()&lt;/code&gt; function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dge_list = edgeRLib.DGEList(counts=rawcounts)&#xA;dge_list = edgeRLib.calcNormFactors(dge_list, method=&quot;TMM&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am getting the error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;RRuntimeError: Error in quantile.default(x, p = p) : &#xA;    missing values and NaN's not allowed if 'na.rm' is FALSE&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, I thought that maybe my &lt;code&gt;rawcounts&lt;/code&gt; &lt;code&gt;2D&lt;/code&gt; &lt;code&gt;numpy&lt;/code&gt; array contains &lt;code&gt;nan&lt;/code&gt; values but it does not. I checked it with both &lt;code&gt;pd.isnull()&lt;/code&gt; and &lt;code&gt;np.isnan()&lt;/code&gt; functions. So, I suspect now that the &lt;code&gt;2D&lt;/code&gt; array itself can not be normalized for some reason and I do not know why. The error is most probably not related to &lt;code&gt;rpy2&lt;/code&gt; set up and &lt;code&gt;R&lt;/code&gt; installation because it was working before for similar &lt;code&gt;numpy&lt;/code&gt; arrays but failed this time. Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-10-26T02:25:47.120" Title="Error in quantile.default(x, p = p) in EgdeR calcNormFactors" Tags="|edger|tmm|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5289" PostTypeId="2" ParentId="5288" CreationDate="2018-10-26T02:25:47.120" Score="4" Body="&lt;p&gt;Ok, it turned out that some rows and/or columns contained all values as zeros and that was the reason for the error. I wrote a function to fix that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def fix_rawcounts(rawcounts, cell_ids, gene_symbols):&#xA;    '''The function checks whether there are &#xA;    any rows or columns with all zeros &#xA;    and removes them'''&#xA;    print('Checking whether rows or columns contain all zeros')&#xA;&#xA;    # Checking columns&#xA;    count_arr = np.apply_along_axis(np.count_nonzero, axis=0, arr=rawcounts )&#xA;    rawcounts = rawcounts[:,count_arr != 0]&#xA;&#xA;    # Checking rows&#xA;    count_arr = np.apply_along_axis(np.count_nonzero, axis=1, arr=rawcounts )&#xA;    rawcounts = rawcounts[count_arr != 0,:]&#xA;&#xA;    print('finished checking zero rows and columns')&#xA;    return rawcounts&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-10-26T02:25:47.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5290" PostTypeId="2" ParentId="5046" CreationDate="2018-10-26T06:25:16.290" Score="1" Body="&lt;p&gt;Not sure if this has anything to do with your question but I think the way how Monocle approach this question sort of makes sense to me. The normalization methods for Seurat and Monocle are somehow similar (correct if I am wrong, this is simply my personal experience), as @kohlkopf mentioned. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For monocle specifically, it has this function where you can display dispersion v.s. expression of the genes to decide the cut off. Here is an example of this plot that I generated when processing the PBMC data downloaded from the Seurat tutorial.&lt;a href=&quot;https://i.stack.imgur.com/tuT7e.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tuT7e.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can assign the cut-off manually but I doubt I answered your question on how to decide the cut-off value. Maybe you need to try a set the expression cut-off from a range to experiment what value may potentially answer a biologically relevant question in your context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested in how monocle approach this, I think the &lt;a href=&quot;http://cole-trapnell-lab.github.io/monocle-release/docs/#classifying-and-counting-cells&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monocle introduction&lt;/a&gt; and &lt;a href=&quot;https://davetang.org/muse/2017/10/01/getting-started-monocle/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dave Tang's Blog&lt;/a&gt; are excellent in explain how it works. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, you are right that 0 could be sequencing error. But I would also suggest that, if you are looking at genes at very small expression level, scRNA-seq, at least droplet-based technique, could be really difficult in such context. I happened to hear someone trying to find lncRNA expression talking about how a &quot;0&quot; sometimes may be a true &quot;0&quot; due the low expression nature of lncRNA in scRNA-seq, etc.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyhow, not sure if I answer the question but I do hope it helps you somehow. I am not able to make comments but this is the best I can do.&lt;/p&gt;&#xA;" OwnerUserId="3480" LastActivityDate="2018-10-26T06:25:16.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5291" PostTypeId="2" ParentId="5287" CreationDate="2018-10-26T08:35:40.497" Score="8" Body="&lt;p&gt;According to the documentation (&lt;code&gt;?Biostrings::DNAStringSet&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;width(x)&lt;/code&gt;: A vector of non-negative integers containing the number of&#xA;            letters for each element in &lt;code&gt;x&lt;/code&gt;.  Note that &lt;code&gt;width(x)&lt;/code&gt; is&#xA;            also defined for a character vector with no NAs and is&#xA;            equivalent to &lt;code&gt;nchar(x, type=&quot;bytes&quot;)&lt;/code&gt;.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;code&gt;names(x)&lt;/code&gt;: &lt;code&gt;NULL&lt;/code&gt; or a character vector of the same length as &lt;code&gt;x&lt;/code&gt;&#xA;            containing a short user-provided description or comment for&#xA;            each element in &lt;code&gt;x&lt;/code&gt;.  These are the only data in an&#xA;            XStringSet object that can safely be changed by the user. All&#xA;            the other data are immutable!  As a general recommendation,&#xA;            the user should never try to modify an object by accessing&#xA;            its slots directly.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;code&gt;as.character(x, use.names=TRUE)&lt;/code&gt;: Converts &lt;code&gt;x&lt;/code&gt; to a character vector&#xA;            of the same length as &lt;code&gt;x&lt;/code&gt;.  The &lt;code&gt;use.names&lt;/code&gt; argument controls&#xA;            whether or not &lt;code&gt;names(x)&lt;/code&gt; should be propagated to the names&#xA;            of the returned vector.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This should be enough for you to create a &lt;code&gt;data.frame&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;dss2df &amp;lt;- function(dss) data.frame(width=width(dss), seq=as.character(dss), names=names(dss))&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, if you have a huge &lt;code&gt;DNAStringSet&lt;/code&gt;, perhaps you do not want to convert all the sequences to characters; storing sequences as characters supposedly is less efficient. For plotting it might be sufficient to have just widths.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;qplot(width(dss), geom='histogram'))&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="191" LastEditorUserId="191" LastEditDate="2018-10-26T08:42:10.457" LastActivityDate="2018-10-26T08:42:10.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5292" PostTypeId="1" CreationDate="2018-10-26T09:55:25.490" Score="7" ViewCount="8650" Body="&lt;p&gt;I am completely new to bioinformatics so I'm looking to learn how to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have multiple directories with fastq files: E.g; 10 Directories with each time series, each with Treatment and control directories, each with rep1 rep2 rep3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example: T9/Infected/Rep1/*.fastq.gz.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking to create a loop to run fastQC on each fastq file instead of having to submit a separate job for each directory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then to either output the fastQC data to a single directory or if possible a directory corresponding to each rep - e.g. rep1 results go into a folder called rep1 and so on.&lt;/p&gt;&#xA;" OwnerDisplayName="Ryan Carter" LastEditorUserId="964" LastEditDate="2018-10-26T15:25:17.320" LastActivityDate="2021-08-20T14:40:31.417" Title="Bash scripting FastQC for multiple fastq files in multiple directories" Tags="|fastq|quality-control|bash|fastqc|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5293" PostTypeId="2" ParentId="5292" CreationDate="2018-10-26T10:56:19.973" Score="1" Body="&lt;p&gt;Example dir structure:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ find FastQC/&#xA;FastQC/&#xA;FastQC/T9&#xA;FastQC/T9/Infected&#xA;FastQC/T9/Infected/Rep1&#xA;FastQC/T9/Infected/Rep1/test11.fastq.gz&#xA;FastQC/T9/Infected/Rep1/test1.fastq.gz&#xA;FastQC/T9/Infected/Rep2&#xA;FastQC/T9/Infected/Rep2/test2.fastq.gz&#xA;FastQC/T9/Infected/Rep3&#xA;FastQC/T9/Infected/Rep3/test3.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If understood well You need to run some job on every *.fastq.gz file. Then You can do something like this (my example job is &lt;strong&gt;gzip test&lt;/strong&gt;, replace with your job):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rookie:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ find FastQC/ -type f -name &quot;*.fastq.gz&quot; | xargs gzip -tv&#xA;FastQC/T9/Infected/Rep1/test11.fastq.gz:         OK&#xA;FastQC/T9/Infected/Rep1/test1.fastq.gz:  OK&#xA;FastQC/T9/Infected/Rep2/test2.fastq.gz:  OK&#xA;FastQC/T9/Infected/Rep3/test3.fastq.gz:  OK&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Solid:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ find FastQC/ -type f -name &quot;*.fastq.gz&quot; -print0 | xargs -0 -I {} gzip -tv {}&#xA;FastQC/T9/Infected/Rep1/test11.fastq.gz:         OK&#xA;FastQC/T9/Infected/Rep1/test1.fastq.gz:  OK&#xA;FastQC/T9/Infected/Rep2/test2.fastq.gz:  OK&#xA;FastQC/T9/Infected/Rep3/test3.fastq.gz:  OK&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;find finds files with name *.fastq.gz and outputs it with zero byte delimited (to support weird characters like space etc. in filenames)&lt;/li&gt;&#xA;&lt;li&gt;xargs represents output as {} and passes it to gzip -tv&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If You want to copy files inside one heap folder:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ find FastQC/ -type f -name &quot;*.fastq.gz&quot; -print0 | xargs -0 -I {} cp -pv {} FastQC_heap/&#xA;`FastQC/T9/Infected/Rep1/test11.fastq.gz' -&amp;gt; `FastQC_heap/test11.fastq.gz'&#xA;`FastQC/T9/Infected/Rep1/test1.fastq.gz' -&amp;gt; `FastQC_heap/test1.fastq.gz'&#xA;`FastQC/T9/Infected/Rep2/test2.fastq.gz' -&amp;gt; `FastQC_heap/test2.fastq.gz'&#xA;`FastQC/T9/Infected/Rep3/test3.fastq.gz' -&amp;gt; `FastQC_heap/test3.fastq.gz'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="Kubator" LastActivityDate="2018-10-26T10:56:19.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5295" PostTypeId="2" ParentId="5292" CreationDate="2018-10-26T17:02:16.727" Score="5" Body="&lt;p&gt;&lt;code&gt;multiqc&lt;/code&gt; kind of glazes over some important information, like the exact adapters and duplicated sequences in a library. If you plan to spend big $$ for sequencing a library it is better to look at both the &lt;code&gt;multiqc&lt;/code&gt; report and the actual fastqc &lt;code&gt;html&lt;/code&gt; report to get a better idea of any error modes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Going off of @Kubator's answer, I noticed that there was no command to run fastqc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a simple one-liner to run fastqc in parallel on all of your fastq files. The &lt;code&gt;-j 25&lt;/code&gt; uses 25 threads. Change &lt;code&gt;25&lt;/code&gt; to however many threads you want/have for max speed.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Run fastqc on everything in parallel.&#xA;&amp;gt; find ../reads/ -name '*.fastq.gz' | awk '{printf(&quot;fastqc \&quot;%s\&quot;\n&quot;, $0)}' | parallel -j 25 --verbose&#xA;# copies all the fastqc files to directory ./    &#xA;&amp;gt; find ../reads/ -name '*fastqc.*' | xargs -I '{}' mv '{}' ./&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These files might be output by &lt;code&gt;multiqc&lt;/code&gt;, anyway!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit - now the command is safe for whitespace in file names. Please, no newlines in filenames though!&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-10-26T19:30:49.320" LastActivityDate="2018-10-26T19:30:49.320" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5296" PostTypeId="1" CreationDate="2018-10-26T19:33:00.740" Score="7" ViewCount="78" Body="&lt;p&gt;Are there any tools that will simulate 3' end tag-based single-cell RNA-seq reads? For example, &lt;code&gt;Drop-seq&lt;/code&gt;, &lt;code&gt;10X Chromium&lt;/code&gt;, &lt;code&gt;CEL-seq&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;There are tools that simulate scRNA-seq gene count data (e.g. &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-017-1305-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Splatter&lt;/a&gt;), but I can't find anything that will simulate reads or even counts at the transcript level.&lt;/p&gt;&#xA;" OwnerUserId="900" LastEditorUserId="3848" LastEditDate="2020-07-25T08:06:39.403" LastActivityDate="2022-06-17T18:04:18.287" Title="Simulating 3' end tag-based scRNA-seq reads" Tags="|scrnaseq|transcriptome|simulated-data|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5297" PostTypeId="2" ParentId="3725" CreationDate="2018-10-26T20:08:33.717" Score="1" Body="&lt;p&gt;Yes, you can stop the run and it should not affect the base calling. We normally do the runs and then do the base calling after the run has completed.&lt;/p&gt;&#xA;" OwnerUserId="3576" LastActivityDate="2018-10-26T20:08:33.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5298" PostTypeId="2" ParentId="296" CreationDate="2018-10-26T20:28:34.207" Score="1" Body="&lt;p&gt;I am working with a fungus and am using CTAB method for DNA extraction and I used rapid kit for my initial library prep but it did not work and since than I have been using ligation kit and it has been giving me very good results. I managed to get data ranging from 4GB to 13GB.&lt;/p&gt;&#xA;" OwnerUserId="3576" LastActivityDate="2018-10-26T20:28:34.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5299" PostTypeId="1" AcceptedAnswerId="5313" CreationDate="2018-10-26T22:01:03.930" Score="1" ViewCount="1851" Body="&lt;p&gt;I have an upcoming run on a HiSeq X and most of the libraries in the pool have both i5 and i7 indices. However, some of the libraries were made with the IS4 p5 oligo and it is unindexed.&lt;/p&gt;&#xA;&lt;p&gt;The IS4 indexing oligo is from the following paper. The sequence is below.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.researchgate.net/publication/44643209_Illumina_Sequencing_Library_Preparation_for_Highly_Multiplexed_Target_Capture_and_Sequencing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Meyer, Matthias, and Martin Kircher. &amp;quot;Illumina sequencing library&#xA;preparation for highly multiplexed target capture and sequencing.&amp;quot;&#xA;Cold Spring Harbor Protocols 2010.6 (2010): pdb-prot5448.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;                                 miseq index&#xA;                              5' TCTTTCC 3'&#xA;                                 &amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&#xA; 5' AATGATACGGCGACCACCGAGATCTACACTCTTTCCCTACACGACGCTCTT 3'&#xA;    ^------------------^         ^--------------------^&#xA;    flowcell binding seq        adapter-binding seq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When we sequence these libraries dual-indexed on the MiSeq, the machine reads into the adapter and returns the index sequence &lt;code&gt;TCTTTCC&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Question&lt;/h2&gt;&#xA;&lt;p&gt;What will the index sequence be when the HiSeq reads it?&lt;/p&gt;&#xA;&lt;p&gt;The more fundamental question is: How exactly do MiSeq and HiSeq machines read index sequences?&lt;/p&gt;&#xA;&lt;h3&gt;side note&lt;/h3&gt;&#xA;&lt;p&gt;Here is what an index p5 and unindexed p5 molecule look like:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vDw4I.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vDw4I.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Update: Sept 2019 - After running into this problem again, an Illumina representative agreed with Devon Ryan and said that setting up a run for single indexing is the best option when there are unindexed p5 ends on one or more libraries.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2019-09-20T20:49:09.063" Title="What i5 index should I use on the Illumina sample sheet for an unindexed p5 primer?" Tags="|ngs|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5300" PostTypeId="1" AcceptedAnswerId="5303" CreationDate="2018-10-26T22:58:04.720" Score="4" ViewCount="473" Body="&lt;p&gt;Grepping out short sequences from a fasta or fastq file is a really useful way to look at sequencing data. Using the option &lt;code&gt;--color=always&lt;/code&gt; makes this even more useful, as you can visualize where the sequences appear in sequencing reads. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zgrep 'GATCGATC' --color=always My_Hiseq_Data.fq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4evAS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4evAS.png&quot; alt=&quot;grep color output&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make this more useful, how can I align the output of &lt;code&gt;grep --color=always&lt;/code&gt; such that the beginning of the highlighted parts are aligned?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wFRmj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wFRmj.png&quot; alt=&quot;aligned output&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="298" LastEditDate="2018-10-27T17:35:25.723" LastActivityDate="2018-10-28T09:54:05.243" Title="How to align output of grep --color=always? (To QC fasta/fastq files)" Tags="|sequence-alignment|fasta|fastq|illumina|bash|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5301" PostTypeId="2" ParentId="4586" CreationDate="2018-10-27T02:07:25.620" Score="2" Body="&lt;p&gt;You can use eutils to get this information as shown below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db assembly -q 'GCF_000337855.1' | esummary | xtract -pattern DocumentSummary -element AssemblyAccession,Taxid&#xA;GCF_000337855.1 469382&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2018-10-27T02:07:25.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5303" PostTypeId="2" ParentId="5300" CreationDate="2018-10-27T18:23:10.967" Score="6" Body="&lt;p&gt;Perhaps, &lt;code&gt;grep&lt;/code&gt; is not the best tool to use in this case, but it should be in principle possible by using &lt;code&gt;grep&lt;/code&gt; &amp;amp; &lt;code&gt;sed&lt;/code&gt;. Here is an example showing three symbols around a match.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;zcat My_Hiseq_Data.fq.gz | \&#xA;      grep -Eo '.{0,3}GATCGATC.*' | \&#xA;      sed -En 's/.*/   \0/; s/.*(.{3}GATCGATC.{0,3}).*/\1/p' | \&#xA;      grep --color=always GATCGATC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here are some explanations:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;zcat&lt;/code&gt; will just decompress a file.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;grep&lt;/code&gt; will output all the matches with 3-symbol context (or less). This is needed because a line can have multiple matches.&lt;/li&gt;&#xA;&lt;li&gt;first &lt;code&gt;sed&lt;/code&gt; command will add 3 symbols before all the matches.&lt;/li&gt;&#xA;&lt;li&gt;second &lt;code&gt;sed&lt;/code&gt; command will output the sequence with 3 surrounding characters on the left and max 3 characters on the right. This is an equivalent of &lt;code&gt;grep -Eo '.{3}GATCGATC.{0,3}'&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Finally, now that all matches are aligned, &lt;code&gt;grep&lt;/code&gt; will color your sequence.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This code is possible to adapt to include more symbols of context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would recommend doing this in another language, though. E.g., in Python:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;import sys&#xA;import re&#xA;&#xA;HL = '\033[93m'&#xA;ENDC = '\033[0m'&#xA;&#xA;&#xA;filename = sys.argv[1]&#xA;seq = sys.argv[2]&#xA;context = 10&#xA;&#xA;seqre = re.compile(r'(.{0,%d})(%s)(.{0,%d})' % (context, sys.argv[2], context))&#xA;&#xA;for line in open(filename):&#xA;    line = line.rstrip()&#xA;    for m in seqre.finditer(line):&#xA;        print(m.group(1).rjust(context),&#xA;              HL,&#xA;              m.group(2),&#xA;              ENDC,&#xA;              m.group(3),&#xA;              sep='')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;UPD&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;A more advanced Python version inspired by &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/5305/191&quot;&gt;this answer&lt;/a&gt; with support for multiple files, gzip, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nHhRS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nHhRS.png&quot; alt=&quot;screenshot of match script&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;import sys&#xA;import re&#xA;import argparse&#xA;import gzip&#xA;from termcolor import  colored&#xA;&#xA;&#xA;def align_matches(f, seqre, context):&#xA;    # for every line&#xA;    for line in f:&#xA;        # get rid of newline character&#xA;        line = line.rstrip()&#xA;        for m in seqre.finditer(line):&#xA;            # for every match print right-aligned left-context,&#xA;            # match colored in red, and right-context &#xA;            print(m.group(1).rjust(context),&#xA;                  colored(m.group(2), 'red'),&#xA;                  m.group(3),&#xA;                  sep='')&#xA;&#xA;&#xA;# this function will check file extension to be able&#xA;# to open text files as well as gzip-compressed files&#xA;def myopen(fn):&#xA;    if fn.endswith('.gz'):&#xA;        return gzip.open(fn, 'rt')&#xA;    else:&#xA;        return open(fn, 'rt')&#xA;&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    # define script arguments&#xA;    parser = argparse.ArgumentParser(description='Aligned grep')&#xA;    parser.add_argument('pattern', type=str,&#xA;                        help='a valid regular expression')&#xA;    parser.add_argument('input', type=myopen, nargs='+',&#xA;                        help='files names, could be a gzip compressed file ending with .gz')&#xA;    parser.add_argument('--context', '-c', default=30, type=int,&#xA;                        help='number of context characters to show')&#xA;    args = parser.parse_args()&#xA;&#xA;    # compile a regular expression with a match surrounded&#xA;    # by at most args.context number of characters&#xA;    seqre = re.compile(r'(.{0,%d})(%s)(.{0,%d})' % (&#xA;        args.context, args.pattern, args.context))&#xA;&#xA;    # for every input file run align_matches&#xA;    for f in args.input:&#xA;        align_matches(f, seqre, args.context)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="191" LastEditorUserId="191" LastEditDate="2018-10-28T09:54:05.243" LastActivityDate="2018-10-28T09:54:05.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5305" PostTypeId="2" ParentId="5300" CreationDate="2018-10-27T19:24:55.807" Score="2" Body="&lt;p&gt;I wrote a Perl script to do this. Just like &lt;code&gt;zgrep&lt;/code&gt;, it can search through as many files as you give it. You can use the &lt;code&gt;-c&lt;/code&gt; switch to tell it how much context (how many characters) you want around the match. So &lt;code&gt;-c10&lt;/code&gt; will search for your target string and 10 characters on either side of it. If you don't use the &lt;code&gt;-c&lt;/code&gt;, it will default to 155, which should include the entire sequence for most cases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To use it, save as &lt;code&gt;alignString.pl&lt;/code&gt; somewhere in your &lt;code&gt;$PATH&lt;/code&gt; (&lt;code&gt;~/bin&lt;/code&gt; should work if you're using Ubuntu, for example), make it executable with &lt;code&gt;chmod a+x ~/bin/alignString&lt;/code&gt; and then run it like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;alignString.pl GAAAATCAG file1.fastq.gz file2.fastq.gz file3.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hARo8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hARo8.png&quot; alt=&quot;example output&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The code:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env perl&#xA;&#xA;use warnings;&#xA;use strict;&#xA;use Term::ANSIColor;&#xA;use Getopt::Std;&#xA;&#xA;my %opts;&#xA;getopts('c:',\%opts);&#xA;my &lt;span class=&quot;math-container&quot;&gt;$context = $&lt;/span&gt;opts{'c'} || 50;&#xA;&#xA;## This subroutine does the alignment&#xA;sub align{&#xA;  ## Read the line and the target pattern&#xA;  my (&lt;span class=&quot;math-container&quot;&gt;$line, $&lt;/span&gt;pat) = @_;&#xA;  ## This will find 0 to &lt;span class=&quot;math-container&quot;&gt;$context characters on either side&#xA;  ## of the search pattern. Note that it will find the first&#xA;  ## occurrence of the pattern.&#xA;  $&lt;/span&gt;line =~ /(.{0,&lt;span class=&quot;math-container&quot;&gt;$context})$&lt;/span&gt;pat(.{0,&lt;span class=&quot;math-container&quot;&gt;$context})/;&#xA;  ## Save the strings to the left and right of the pattern&#xA;  my $&lt;/span&gt;left = &lt;span class=&quot;math-container&quot;&gt;$1;&#xA;  my $&lt;/span&gt;right = &lt;span class=&quot;math-container&quot;&gt;$2;&#xA;  ## If the left hand string is shorter than the value of $&lt;/span&gt;context,&#xA;  ## pad it with spaces so it can be aligned.&#xA;  while (length(&lt;span class=&quot;math-container&quot;&gt;$left) &amp;lt; $&lt;/span&gt;context) {&#xA;    &lt;span class=&quot;math-container&quot;&gt;$left = &quot; &quot; . $&lt;/span&gt;left;&#xA;  }&#xA;  ## Return the padded left hand string, the pattenr colored red&#xA;  ## and the right hand string unchanged.&#xA;  return &quot;&lt;span class=&quot;math-container&quot;&gt;$left&quot; . colored($&lt;/span&gt;pat,&quot;red&quot;) . &quot;$right&quot;;&#xA;}&#xA;&#xA;## The search pattern is the 1st argument&#xA;my $pattern = shift;&#xA;## Any remaining arguments are taken as target files to search&#xA;my @fastqFiles = @ARGV;&#xA;&#xA;## Run the zgrep command&#xA;open(my &lt;span class=&quot;math-container&quot;&gt;$grepStream, '-|', &quot;zgrep $&lt;/span&gt;pattern @fastqFiles&quot;);&#xA;## For each line returned by zgrep&#xA;while (my &lt;span class=&quot;math-container&quot;&gt;$line = &amp;lt;$&lt;/span&gt;grepStream&amp;gt;) {&#xA;  ## Align this line&#xA;  &lt;span class=&quot;math-container&quot;&gt;$line = align($&lt;/span&gt;line, &lt;span class=&quot;math-container&quot;&gt;$pattern);&#xA;  print &quot;$&lt;/span&gt;line\n&quot;;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-10-28T00:21:11.260" LastActivityDate="2018-10-28T00:21:11.260" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5306" PostTypeId="1" CreationDate="2018-10-28T00:02:56.463" Score="6" ViewCount="441" Body="&lt;p&gt;I would like to filter a bed file with intervals, ie. in the format of:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    13800   14301&#xA;chr1    15500   16001&#xA;chr1    19400   19901&#xA;chr1    22800   23301&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In particular, I want to filter out intervals that fall far from known genes, let's say &gt; 1KB.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking of querying each interval (in parallel) and check through a known set of coordinates from a gtf, e.g. from &lt;code&gt;Homo_sapiens.GRCh38.93.gtf&lt;/code&gt; by filtering for gene coordinates:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1       havana  gene    168317497       168376876       .       +       .       gene_id &quot;ENSG00000227722&quot;; gene_version &quot;2&quot;; gene_name &quot;AL022100.1&quot;; gene_source &quot;havana&quot;; gene_biotype &quot;transcribed_processed_pseudogene&quot;;&#xA;1       mirbase gene    168375524       168375621       .       +       .       gene_id &quot;ENSG00000207974&quot;; gene_version &quot;1&quot;; gene_name &quot;MIR557&quot;; gene_source &quot;mirbase&quot;; gene_biotype &quot;miRNA&quot;;&#xA;1       havana  gene    168400829       168495685       .       -       .       gene_id &quot;ENSG00000228697&quot;; gene_version &quot;2&quot;; gene_name &quot;AL023755.1&quot;; gene_source &quot;havana&quot;; gene_biotype &quot;lincRNA&quot;;&#xA;1       havana  gene    168401483       168407274       .       -       .       gene_id &quot;ENSG00000283255&quot;; gene_version &quot;1&quot;; gene_name &quot;AL022100.2&quot;; gene_source &quot;havana&quot;; gene_biotype &quot;lincRNA&quot;;&#xA;1       havana  gene    168449672       168451211       .       -       .       gene_id &quot;ENSG00000215833&quot;; gene_version &quot;3&quot;; gene_name &quot;QRSL1P1&quot;; gene_source &quot;havana&quot;; gene_biotype &quot;processed_pseudogene&quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, since the bedfile is quite big I am looking for the fastest solution since checking each interval may be quite expensive.&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-11-04T12:51:52.710" Title="Fast filtering of intervals not falling within a certain distance from known genes" Tags="|gene|bed|gtf|interval|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5307" PostTypeId="2" ParentId="5306" CreationDate="2018-10-28T00:42:54.767" Score="3" Body="&lt;p&gt;First I prepared a bed file in which the gene intervals are augmented by 1KB before and after the gene start and end coordinates. Then I intersected this bed file with my original one with the option &lt;code&gt;-wa&lt;/code&gt;, therefore retaining only the intervals in my original bed file that intersect with the bedfile produced from the original gtf, filtered by gene regions. The &lt;code&gt;-u&lt;/code&gt; flag is also used to print only one line per overlap between the two files, in case an interval in the first overlaps with multiple intervals in the second.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# download information about chromosomes&#xA;wget http://hgdownload.soe.ucsc.edu/goldenPath/hg38/bigZips/hg38.chrom.sizes ./&#xA;&#xA;# Original GTF has a different notation than UCSC&#xA;cat Homo_sapiens.GRCh38.93.gtf | \&#xA;  sed -e &lt;span class=&quot;math-container&quot;&gt;$'s/^./chr&amp;amp;/' -e $&lt;/span&gt;'s/^chrMT/chrM/' | \&#xA;  awk '&lt;span class=&quot;math-container&quot;&gt;$3==&quot;gene&quot;' | \&#xA;  bedtools slop -b 1000 -g hg38.chrom.sizes -i - | \&#xA;  sed -e $&lt;/span&gt;'s/\tgene_id.*gene_name/\t/' -e &lt;span class=&quot;math-container&quot;&gt;$'s/gene_source.*//' | \&#xA;  tr -d '&quot;; ' | \&#xA;  awk -v OFS=$&lt;/span&gt;'\t' '{print &lt;span class=&quot;math-container&quot;&gt;$1, $&lt;/span&gt;4, &lt;span class=&quot;math-container&quot;&gt;$5, $&lt;/span&gt;9}' &amp;gt; genes_1kb.bed&#xA;&#xA;bedtools intersect -wa -u -a intervals.bed -b genes_1kb.bed &amp;gt; intervals.1kb.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2018-11-03T17:56:09.187" LastActivityDate="2018-11-03T17:56:09.187" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5308" PostTypeId="1" AcceptedAnswerId="5329" CreationDate="2018-10-28T08:09:23.050" Score="2" ViewCount="77" Body="&lt;p&gt;I would like to blast a peptide sequence against all available plant proteome databases, i.e. a &lt;code&gt;blastp&lt;/code&gt; metaproteome analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What tools are available? Which database can I use?&lt;/p&gt;&#xA;" OwnerUserId="3580" LastEditorUserId="3580" LastEditDate="2018-10-29T08:20:57.250" LastActivityDate="2018-10-30T10:45:52.510" Title="API/website for blasting peptide sequence against database of all plant proteomes" Tags="|proteins|blast|metagenome|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="5309" PostTypeId="1" CreationDate="2018-10-28T11:27:32.547" Score="3" ViewCount="432" Body="&lt;p&gt;I am confused with RNA seq alignment. My understanding is that after Mature mRNA is isolated from the cell, it is then fragmented and using reverse transcriptase enzyme a cDNA copy is created which is further amplified. This cDNA is then mapped with the reference genome. If the cDNA does not have introns how does it map with the reference genome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone please help &lt;/p&gt;&#xA;" OwnerUserId="3581" LastEditorUserId="964" LastEditDate="2018-10-29T18:51:13.837" LastActivityDate="2018-10-29T18:51:13.837" Title="cDNA and alignment mapping" Tags="|rna-seq|sequence-alignment|rna-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5310" PostTypeId="2" ParentId="5309" CreationDate="2018-10-28T16:04:18.917" Score="5" Body="&lt;h2&gt;tl;dr&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Spliced aligners just make many small matches along the read pair or single read to determine splice junctions and output an alignment for the whole read, as long as it matches somewhere in the genome.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;mRNA and translation&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In the cell, after transcription, introns are spliced out of RNAs to become mature RNA. The protein-coding sequence does have sequence that is chimeric from cis-located portions of the same chromosome that do not actually exist.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;              |&amp;lt;&amp;lt;exon 1&amp;gt;&amp;gt;|  intron    |&amp;lt;&amp;lt;exon 2&amp;gt;|&#xA;Genome: ------ABCDEFGHIJKLmnopqrstuvwxYZ123456789------------------&#xA;&#xA;mRNA  : ----ABCDEFGHIJKLYZ123456789aaaaaaaaaaa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you said, if we only even consider small stretches of DNA, the sequence &lt;code&gt;JKLYZ&lt;/code&gt; does not exist in the genome, only in the mRNA. So, we need a special way to map mRNA-seq reads to the genome that correctly splits at splice junctions.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;How do we map mRNA-seq reads to a genome?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;As a result, parts of the mRNA-seq reads will align well to one part of the genome, but only for part of the read. This requires a different alignment algorithm that considers multiple local high-score alignments for different parts of the read. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, see how a genomic aligner versus a splice aligner might treat the same mRNA-seq read on our example genome.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                         |&amp;lt;&amp;lt;exon 1&amp;gt;&amp;gt;|  intron    |&amp;lt;&amp;lt;exon 2&amp;gt;|&#xA;Genome:            ------ABCDEFGHIJKLmnopqrstuvwxYZ123456789------------------&#xA;&#xA;spliced aligner  : ------ABCDEFGHIJKL------------YZ123456789&#xA;genomic aligner  : ------ABCDEFGHIJKL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There is a possibility that the genomic aligner will not recognize that the second half of the read is an exon due to the large intron gap, and it may clip the alignment. Spliced aligners just take this into account and output several alignments for the same read regardless of gap size. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3530905/&quot; rel=&quot;noreferrer&quot;&gt;The STAR algorithm, for example, uses many small matching pieces of a single read to determine where the splice junctions are&lt;/a&gt;. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4655817/&quot; rel=&quot;noreferrer&quot;&gt;HiSat2 uses small approximate matches&lt;/a&gt; using a &lt;a href=&quot;https://en.wikipedia.org/wiki/FM-index&quot; rel=&quot;noreferrer&quot;&gt;FM-index&lt;/a&gt; to find seeds for spliced alignment.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-10-28T16:04:18.917" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5312" PostTypeId="1" CreationDate="2018-10-29T03:12:32.973" Score="1" ViewCount="134" Body="&lt;p&gt;I'm running into a segmentation fault running peakranger's bcp on Arch Linux. Binaries were compiled from source just today. Any idea how I can prevent this?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ peakranger bcp -d chip_seq/align_count/Sample_DNA_loA_h3k4me3/alignment/Sample_DNA_loA_h3k4me3.hisat2.bam -c chip_seq/align_count/Sample_DNA_loA_input/alignment/Sample_DNA_loA_input.hisat2.bam --format bam -o chip_seq/align_count/Sample_DNA_loA/call_peaks/bcp --gene_annot_file /data/sai/annots/musmus_GRCm38.94.gtf.gz --verbose&#xA;program version:          1.18&#xA;Data files:&#xA; File format:             bam&#xA; Sample file:             chip_seq/align_count/Sample_DNA_loA_h3k4me3/alignment/Sample_DNA_loA_h3k4me3.hisat2.bam&#xA; Control file:            chip_seq/align_count/Sample_DNA_loA_input/alignment/Sample_DNA_loA_input.hisat2.bam&#xA;Qualities:&#xA; P value cut off:         0.0001&#xA; sliding window size:     500&#xA; Read extension length:   200&#xA;Output:&#xA; Regions:                 chip_seq/align_count/Sample_DNA_loA/call_peaks/macs2_region.bed&#xA; HTML reports:            Enabled&#xA; Plot region length:      6000&#xA; Annotation file:         /data/sai/annots/musmus_GRCm38.94.gtf.gz&#xA;&#xA;Reads statistics:&#xA;&#xA; Treatment reads +:       2125600&#xA; Treatment reads -:       2113802&#xA; Average read length:     20&#xA; Control reads +:         7582218&#xA; Control reads -:         7575995&#xA; Average read length:     20&#xA; Verifying reads...&#xA;Warning: JH584295.1 only contains reads in the positive strand. The chromosome is removed.&#xA;Warning: GL456382.1 only contains reads in the negative strand. The chromosome is removed.&#xA;Warning: No reads were found in GL456382.1 of the treatment dataset. The chromosome is removed.&#xA;Warning: No reads were found in JH584295.1 of the treatment dataset. The chromosome is removed.&#xA;&#xA;Reads statistics after correction:&#xA;&#xA; Treatment reads +:       2125597&#xA; Treatment reads -:       2113800&#xA; Control reads +:         7582196&#xA; Control reads -:         7575947&#xA;&#xA; Calling peaks...&#xA;&#xA;Segmentation fault (core dumped)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Reads were trimmed with trim galore and aligned with hisat2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update 2018-10-29: additional information of input and machine&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ samtools flagstat chip_seq/align_count/Sample_DNA_loA_h3k4me3/alignment/Sample_DNA_loA_h3k4me3.hisat2.bam&#xA;33521569 + 0 in total (QC-passed reads + QC-failed reads)&#xA;946323 + 0 secondary&#xA;0 + 0 supplementary&#xA;0 + 0 duplicates&#xA;5185725 + 0 mapped (15.47% : N/A)&#xA;0 + 0 paired in sequencing&#xA;0 + 0 read1&#xA;0 + 0 read2&#xA;0 + 0 properly paired (N/A : N/A)&#xA;0 + 0 with itself and mate mapped&#xA;0 + 0 singletons (N/A : N/A)&#xA;0 + 0 with mate mapped to a different chr&#xA;0 + 0 with mate mapped to a different chr (mapQ&amp;gt;=5)&#xA;$&lt;/span&gt; samtools flagstat chip_seq/align_count/Sample_DNA_loA_input/alignment/Sample_DNA_loA_input.hisat2.bam&#xA;22508880 + 0 in total (QC-passed reads + QC-failed reads)&#xA;3359461 + 0 secondary&#xA;0 + 0 supplementary&#xA;0 + 0 duplicates&#xA;18517674 + 0 mapped (82.27% : N/A)&#xA;0 + 0 paired in sequencing&#xA;0 + 0 read1&#xA;0 + 0 read2&#xA;0 + 0 properly paired (N/A : N/A)&#xA;0 + 0 with itself and mate mapped&#xA;0 + 0 singletons (N/A : N/A)&#xA;0 + 0 with mate mapped to a different chr&#xA;0 + 0 with mate mapped to a different chr (mapQ&amp;gt;=5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried this with two other input bam files and also hit a segfault.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Machine is a 36-core (72 hyperthread) workstation with 500GB of RAM.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ peakranger --version&#xA;1.18&#xA;$&lt;/span&gt; g++ --version&#xA;g++ (GCC) 8.2.0&#xA;&lt;span class=&quot;math-container&quot;&gt;$ uname -r&#xA;4.18.3-arch1-1-ARCH&#xA;$&lt;/span&gt; pacman -Q boost&#xA;boost 1.67.0-7&#xA;$ pacman -Q zlib&#xA;zlib 1:1.2.11-3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="163" LastEditorUserId="163" LastEditDate="2018-10-29T14:03:49.297" LastActivityDate="2018-10-29T14:03:49.297" Title="peakranger bcp segmentation fault" Tags="|chip-seq|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5313" PostTypeId="2" ParentId="5299" CreationDate="2018-10-29T09:30:57.870" Score="1" Body="&lt;p&gt;Illumina machines read the two indices in separate index sequencing reactions where they have sequencing barcodes specific for indices. If you have no index, then you should either not bother with creating either index read or mask that during demultiplexing. If you're mixing indexed samples with an unindexed sample, you can't really rely on any particular sequence being returned by the machine (i.e., just use the &quot;undetermined indices&quot; fastq file). The unindexed adapters tend to not have the full index sequencing primer binding site on them, so reads without index generally don't return much signal. In practice, they're likely to adopt barcode sequences of neighboring clusters. We see that a lot when we spike in PhiX. It has no barcode but you tend to get a small amount assigned to each sample either due to index hopping or (more likely) bleed through of signal from neighboring clusters (or small amounts of those clusters walking between nanowells and then their minority signal dominating during index sequencing).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-10-29T09:30:57.870" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5315" PostTypeId="1" AcceptedAnswerId="16334" CreationDate="2018-10-29T12:33:26.537" Score="0" ViewCount="44" Body="&lt;p&gt;Does anyone know this python module, bamtags? I'm using a pipeline (Dovetail Genomics, HiRise) that imports it, but it seems it doesn't exist.&lt;/p&gt;&#xA;&lt;p&gt;It's  impossible to install it and there is no information on the internet. Does anyone know it?&lt;/p&gt;&#xA;&lt;p&gt;Additional information:&lt;/p&gt;&#xA;&lt;p&gt;Hirise is a 3 years old pipeline that uses Chromossome Capture (HiC) data to identify misassembled regions in a genome assembly and then rescaffold them. By integrating 2 different sequencing methods, the confidence of the genome assembly is thus increased.&lt;/p&gt;&#xA;&lt;p&gt;The workflow is free (we only paid for the data generation). Unfortunately I have no idea what the module does, as I find no information on internet and the scripts are poorly documented. f python-reading is correct, it looks like that is an internally defined module (class) which is in &lt;a href=&quot;https://github.com/DovetailGenomics/HiRise_July2015_GR/blob/3cdda0a1a44585b52274c01d7e9e02268e2d337f/scripts/bamtags.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. So basically, it's there. I am not a python expert so maybe others more experience with python development will confirm or deny this.&lt;/p&gt;&#xA;&lt;p&gt;The only function that is used is the following: &lt;code&gt;BamTags.mate_mapq(aln)&lt;/code&gt;, which is inside a loop in aln: &lt;code&gt;for aln in sam.fetch(reference=c1)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thanks,&#xA;Ricardo&lt;/p&gt;&#xA;" OwnerUserId="2459" LastEditorUserId="73" LastEditDate="2021-07-08T21:26:39.810" LastActivityDate="2021-07-08T21:30:45.047" Title="Python module bamtags" Tags="|python|sam|genomics|hi-c|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5317" PostTypeId="1" CreationDate="2018-10-29T16:00:38.710" Score="1" ViewCount="145" Body="&lt;p&gt;I was reading the manual of &lt;a href=&quot;http://chibba.pgml.uga.edu/mcscan2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MCScanX&lt;/a&gt; and found that besides a all-vs-all protein blast file the tool takes a gtf file with four columns only: scaffold name, protein name, from, to.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the tool is not taking the strand column, it appears to me that probably the most popular tool for genome collinearity analysis is disregarding orientation of genes. Do I get it right? Is there an analogical tool that would perform the collinearity analysis with respect to orientation of genes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- edit --&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I am almost entirely sure that it MCScanX is not taking orientation into account since I found collinear blocks where some of the gene homologs were on the same strand and some of them two different strands (I do hesitate to call them co-&quot;linear&quot; now). So the updated question would be: Is there a good MCScanX alternative that would take orientation of genes into account?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure if I will improve the question with an example, but i will try anyway. Let's say that &lt;code&gt;A B C D&lt;/code&gt; was detected as collinear with &lt;code&gt;A' B' C' D'&lt;/code&gt;. The orientation of genes is &lt;code&gt;+ + - +&lt;/code&gt; for &lt;code&gt;A B C D&lt;/code&gt; and &lt;code&gt;+ - - -&lt;/code&gt; for &lt;code&gt;A' B' C' D'&lt;/code&gt;. For sure, these are synteny blocks. The order of genes is the same, but are they collinear? How can I search just for blocks that would be same (or completely revers) orientation?&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2018-10-30T14:06:17.237" LastActivityDate="2018-10-30T14:06:17.237" Title="Colinearity analysis with respect to gene orientation" Tags="|software-recommendation|genomics|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5318" PostTypeId="2" ParentId="5306" CreationDate="2018-10-29T17:08:16.627" Score="5" Body="&lt;p&gt;Here's a way to use &lt;a href=&quot;https://github.com/bedops/bedops&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS&lt;/a&gt;, which was designed to work fast by using sorted input. Other tools now use sorting to accomplish similar performance benefits.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Convert GTF annotations to a sorted BED file of genes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk '($&lt;/span&gt;3==&quot;gene&quot;)' annotations.gtf | gtf2bed - &amp;gt; genes.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sort your intervals, if unsorted:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sort-bed intervals.unsorted.bed &amp;gt; intervals.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Filter intervals against 1kb-padded genes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops --merge genes.bed | bedops --range 1000 --everything - | bedops --not-element-of 1 intervals.bed - &amp;gt; intervals.filtered.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;We first &lt;code&gt;--merge&lt;/code&gt; genes to simplify dealing with overlapping genes and further speed downstream filtering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The file &lt;code&gt;intervals.filtered.bed&lt;/code&gt; will contain intervals from &lt;code&gt;intervals.bed&lt;/code&gt; which lie outside of a 1kb-window around merged gene space. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To run all of this as a one-liner (assuming use of the &lt;code&gt;bash&lt;/code&gt; shell):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ bedops --merge &amp;lt;(awk '($&lt;/span&gt;3==&quot;gene&quot;)' annotations.gtf | gtf2bed -) | bedops --range 1000 --everything - | bedops --not-element-of 1 &amp;lt;(sort-bed intervals.unsorted.bed) - &amp;gt; intervals.filtered.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This creates the same output result, but avoids creation of intermediate files and so should run even faster.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: BEDOPS is agnostic about chromosome sizes — this toolkit works with any genome assembly. However, using &lt;code&gt;--range&lt;/code&gt; takes care of ranges that would have a negative start position, setting their start coordinates to &lt;code&gt;0&lt;/code&gt;. Bounds aren't really relevant here, in any case, because the &lt;code&gt;bedops --not-element-of&lt;/code&gt; step queries input intervals, which should already be within chromosome bounds, unless they were generated incorrectly.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-11-02T14:42:35.780" LastActivityDate="2018-11-02T14:42:35.780" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5319" PostTypeId="1" AcceptedAnswerId="5323" CreationDate="2018-10-29T17:28:03.367" Score="3" ViewCount="3051" Body="&lt;p&gt;I would like to make a 3D PCA but not sure how to label group wise which i can do for 2D PCA &lt;a href=&quot;https://i.stack.imgur.com/eMVAZ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eMVAZ.jpg&quot; alt=&quot;I get something like this &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(ggplot2)&#xA;test&amp;lt;- read.csv(&quot;NEW_RBP/TF_DISEASE.txt&quot;,header = T,row.names = 1,sep = '\t')&#xA;&#xA;Sample = c(rep(&quot;HSC&quot;,4),rep(&quot;Blast&quot;,11),rep(&quot;LSC&quot;,8))&#xA;Stage=c(rep(&quot;HSC&quot;,4),rep(&quot;Blast&quot;,11),rep(&quot;LSC&quot;,8))&#xA;&#xA;&#xA;df_final=data.frame(Sample,Stage)&#xA;df_final&#xA;&#xA;pca_data=prcomp(t(test), center=TRUE, scale=TRUE)&#xA;pca_data_perc=round(100*pca_data&lt;span class=&quot;math-container&quot;&gt;$sdev^2/sum(pca_data$&lt;/span&gt;sdev^2),1)&#xA;&#xA;df_pca_data=data.frame(PC1 = pca_data&lt;span class=&quot;math-container&quot;&gt;$x[,1], &#xA;                       PC2 = pca_data$&lt;/span&gt;x[,2], sample = colnames(test))&#xA;myColors &amp;lt;- c(&quot;red&quot;, &quot;black&quot;, &quot;blue2&quot;)&#xA;sample &amp;lt;- as.vector(df_pca_data$sample)&#xA;ggplot(df_pca_data, aes(PC1,PC2, colour = factor(Stage)))+#,label=rownames(t(test)))+&#xA;  geom_point(size=30)+&#xA;  geom_text(aes(label=sample),color=&quot;white&quot;,size=6,angle =0,parse = TRUE,  family=&quot;Bookman&quot;, fontface=&quot;bold&quot;)+&#xA;  labs(x=paste0(&quot;PC1 (&quot;,pca_data_perc[1],&quot;)&quot;), y=paste0(&quot;PC2 (&quot;,pca_data_perc[2],&quot;)&quot;))+&#xA;  theme_minimal(base_size=20) +&#xA;  theme(axis.text.x=element_text(size=rel(3), angle=90))+&#xA;  theme(axis.text.y=element_text(size=rel(3), angle=90))+&#xA;  theme(axis.title.x = element_text(colour=&quot;grey20&quot;,size=55,angle=0,hjust=.5,vjust=0,face=&quot;plain&quot;))+&#xA;  theme(axis.title.y = element_text(colour=&quot;grey20&quot;,size=55,angle=90,hjust=.5,vjust=.5,face=&quot;plain&quot;))+&#xA;  scale_color_manual(values=myColors)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above code give me the figure as i have attached &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For 3D PCA I tried this &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;scores = as.data.frame(pca_data$x) &#xA;head(scores)&#xA;&#xA;plot3d(scores[,1:3], col=c(1:4), size=20, type='p', &#xA;       xlim = c(-50,50), ylim=c(-50,50), zlim=c(-50,50))&#xA;text3d(scores[,1]+2, scores[,2]+1, scores[,3]+1,&#xA;       texts=c(rownames(scores)), cex= 0.7, pos=3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JZ5Ro.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JZ5Ro.jpg&quot; alt=&quot;3dPCA&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get something like this as in case of 2D PCA i can label lets say all of the HSC sample as black and so on..Im not sure how to do that for 3D as I tried using factor i couldn;t do&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help or suggestion would be highly appreciated &lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="964" LastEditDate="2018-10-29T22:31:20.567" LastActivityDate="2018-10-31T10:26:32.760" Title="3D PCA group labelling" Tags="|r|statistics|visualization|pca|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5320" PostTypeId="1" AcceptedAnswerId="5344" CreationDate="2018-10-29T19:38:12.180" Score="3" ViewCount="1176" Body="&lt;p&gt;I returned a FeaturePlot from Seurat to ggplot. My plot has a weird range of colours as below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FhN0Y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FhN0Y.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I produced this plot by this code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(mat[1:4,1:4])&#xA;             s1.1        s1.2 s1.3       s1.4&#xA;DDB_G0267178    0 0.009263254    0 0.01286397&#xA;DDB_G0267180    0 0.000000000    0 0.00000000&#xA;DDB_G0267182    0 0.000000000    0 0.03810585&#xA;DDB_G0267184    0 0.000000000    0 0.00000000&#xA;&amp;gt; &#xA;&#xA;I have converted expression matrix to a binary matrix by 2 as a threshold&#xA;&#xA;mat[mat &amp;lt; 2] &amp;lt;- 0&#xA;mat[mat &amp;gt; 2] &amp;lt;- 1&#xA;&#xA;&amp;gt; head(exp[1:4,1:4])&#xA;             s1.1 s1.2 s1.3 s1.4&#xA;DDB_G0267382    0    0    0    1&#xA;DDB_G0267438    0    0    0    1&#xA;DDB_G0267466    0    0    0    0&#xA;DDB_G0267476    0    0    1    0&#xA;&amp;gt; &#xA;&amp;gt; exp=colSums(exp)&#xA;&amp;gt; exp=as.matrix(exp)&#xA;&amp;gt; colnames(exp)=&quot;value&quot;&#xA;&amp;gt; exp=as.data.frame(exp)&#xA;&amp;gt; cc &amp;lt;- AddMetaData(object = seurat, metadata = exp)&#xA;&amp;gt; cc &amp;lt;- SetAllIdent(object = cc, id = &quot;value&quot;)&#xA;&amp;gt; TSNEPlot(object = cc, do.return= TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How I can convert this range to a gradient of colours for example in 8-18, 18-28, 28-38, 38-48 range in blue to yellow please? Something like below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ph8xz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ph8xz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for any help&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then by ggplot now I scaled my colours but I don't like my clusters as so and I don't know how to retain my clusters as a featureplot by this new color gradient &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(cc@meta.data)&#xA;     nGene    nUMI    orig.ident res.0.7 CELL STAGE GENO dataset stage.nice celltype value&#xA;s1.1  4331  373762 SeuratProject       0 s1.1   H16   WT       1        H16        0    34&#xA;s1.2  5603 1074639 SeuratProject       0 s1.2   H16   WT       1        H16        0    26&#xA;s1.3  2064   49544 SeuratProject       0 s1.3   H16   WT       1        H16        0    27&#xA;s1.4  4680  772399 SeuratProject       1 s1.4   H16   WT       1        H16        1    29&#xA;s1.5  3876  272356 SeuratProject       1 s1.5   H16   WT       1        H16        1    21&#xA;s1.6  2557  122314 SeuratProject       0 s1.6   H16   WT       1        H16        0    31&#xA;&#xA;&amp;gt; ggplot(as.data.frame(cc@meta.data), aes(x = cc@meta.data&lt;span class=&quot;math-container&quot;&gt;$CELL, y = cc@meta.data$&lt;/span&gt;res.0.7, colour =cc@meta.data$value)) + &#xA;+     geom_point(size = 5) +&#xA;+     scale_colour_gradient(low = &quot;yellow&quot;, high = &quot;blue&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GtSTm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GtSTm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By below code I obtained a tsne in link&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; cols &amp;lt;-  scales::seq_gradient_pal(low=&quot;beige&quot;, high=&quot;red&quot;, space=&quot;Lab&quot;)(seq(from=0, to=1,length.out=48))&#xA;&amp;gt; &#xA;&amp;gt; TSNEPlot(cc, colors.use=cols)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://image.ibb.co/mzpZ7f/Rplot21.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;enter link description here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I want to know how I could convert this range to a 8-18, 18-28, 28-38, 38-48 colour range as a gradient of blue to yellow?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="181" LastEditDate="2018-11-01T17:34:08.883" LastActivityDate="2018-11-01T17:34:08.883" Title="Changing a wide range of colours to a limited gradient" Tags="|r|scrnaseq|seurat|single-cell|ggplot2|" AnswerCount="2" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="5321" PostTypeId="2" ParentId="5168" CreationDate="2018-10-29T21:07:22.873" Score="1" Body="&lt;p&gt;FYI - heard from the author of Seurat, see &lt;a href=&quot;https://github.com/satijalab/seurat/issues/826&quot; rel=&quot;nofollow noreferrer&quot;&gt;this github issue&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;His suggestion to use DESeq won't work in my case, because DESeq requires counts data which I don't have. But it is the green light to generally use the package on bulk data. I am using test.use = &quot;wilcox&quot; in FindMarkers() because it's non-parametric...&lt;/p&gt;&#xA;" OwnerUserId="3454" LastActivityDate="2018-10-29T21:07:22.873" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5322" PostTypeId="1" AcceptedAnswerId="5324" CreationDate="2018-10-29T21:33:34.107" Score="3" ViewCount="1207" Body="&lt;p&gt;I ran &lt;code&gt;SVA&lt;/code&gt; to remove batch effects for my &lt;code&gt;bulk RNAseq&lt;/code&gt; experiments, but now I need to somehow correct my data matrix in order to run &lt;code&gt;pca&lt;/code&gt;, &lt;code&gt;mds&lt;/code&gt;. I am using &lt;code&gt;DESeq2&lt;/code&gt; for the analysis. Here is the code that I got now:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- estimateSizeFactors(dds)&#xA;dat = counts(dds, normalized = TRUE)&#xA;idx = rowMeans(dat) &amp;gt; 1&#xA;dat = dat[idx,]&#xA;mod = model.matrix(~Group, colData(dds))&#xA;mod0 = model.matrix(~1, colData(dds))&#xA;&#xA;nsv = num.sv(dat, mod)&#xA;svseq = svaseq(dat, mod, mod0, n.sv = nsv)&#xA;&#xA;dds_sva = dds&#xA;dds_sva&lt;span class=&quot;math-container&quot;&gt;$SV1 = svseq$&lt;/span&gt;sv[,1]&#xA;design(dds_sva) = ~ SV1 + Group&#xA;dds_sva = DESeq(dds_sva)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, I am wondering now, how to use &lt;code&gt;svseq&lt;/code&gt; object to plot, say, the corrected version of &lt;code&gt;MDS plot&lt;/code&gt;.  &lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-10-30T08:09:21.397" Title="How to get the corrected matrix after SVA batch effect correction" Tags="|r|rna-seq|deseq2|batch-effects|sva|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5323" PostTypeId="2" ParentId="5319" CreationDate="2018-10-30T07:22:39.847" Score="5" Body="&lt;p&gt;You can easily color 3D pca plots in R based on the code given below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(&quot;scatterplot3d&quot;)&#xA;colors &amp;lt;- c(&quot;#999999&quot;, &quot;#E69F00&quot;, &quot;#56B4E9&quot;) # Number of color according to the number of groups&#xA;colors &amp;lt;- colors[as.numeric(iris&lt;span class=&quot;math-container&quot;&gt;$Species)] # you can put here the column containing the name of population or sample etc.&#xA;pca1 &amp;lt;- prcomp(iris[, -5])&#xA;s3d &amp;lt;-scatterplot3d(pca1$&lt;/span&gt;x[, 1], pca1&lt;span class=&quot;math-container&quot;&gt;$x[, 2],pca1$&lt;/span&gt;x[, 3],xlab=&quot;Sepal.length&quot;,ylab=&quot;Sepal.width&quot;, zlab=&quot;Petal.length&quot;, pch = 16,color=colors)&#xA;legend(&quot;right&quot;, legend = levels(iris$Species),&#xA;  col =  c(&quot;#999999&quot;, &quot;#E69F00&quot;, &quot;#56B4E9&quot;), pch = 16)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The generated graph is given below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0pOW6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0pOW6.png&quot; alt=&quot;Image1&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For further details about how to manipulate plots in 3D have a look &lt;a href=&quot;http://www.sthda.com/english/wiki/scatterplot3d-3d-graphics-r-software-and-data-visualization&quot; rel=&quot;nofollow noreferrer&quot;&gt;at this site&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edited Response&lt;/strong&gt;&#xA;Using the data given below I plotted a 3D pca that may help solve your problem,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;Col1&quot; &quot;Col2&quot; &quot;Col3&quot; &quot;Col4&quot; &quot;colend&quot;&#xA;&quot;H1&quot; 5.1 3.5 1.4 0.2 &quot;HSC&quot;&#xA;&quot;H2&quot; 4.9 3 1.4 0.2 &quot;HSC&quot;&#xA;&quot;H3&quot; 4.7 3.2 1.3 0.2 &quot;HSC&quot;&#xA;&quot;H4&quot; 4.6 3.1 1.5 0.2 &quot;HSC&quot;&#xA;&quot;H5&quot; 5 3.6 1.4 0.2 &quot;HSC&quot;&#xA;&quot;H6&quot; 5.4 3.9 1.7 0.4 &quot;HSC&quot;&#xA;&quot;H7&quot; 4.6 3.4 1.4 0.3 &quot;HSC&quot;&#xA;&quot;H8&quot; 5 3.4 1.5 0.2 &quot;HSC&quot;&#xA;&quot;H9&quot; 4.4 2.9 1.4 0.2 &quot;HSC&quot;&#xA;&quot;H10&quot; 4.9 3.1 1.5 0.1 &quot;HSC&quot;&#xA;&quot;H11&quot; 5.4 3.7 1.5 0.2 &quot;HSC&quot;&#xA;&quot;H12&quot; 4.8 3.4 1.6 0.2 &quot;HSC&quot;&#xA;&quot;H13&quot; 4.8 3 1.4 0.1 &quot;HSC&quot;&#xA;&quot;H14&quot; 4.3 3 1.1 0.1 &quot;HSC&quot;&#xA;&quot;H15&quot; 5.8 4 1.2 0.2 &quot;HSC&quot;&#xA;&quot;B1&quot; 5.7 4.4 1.5 0.4 &quot;blast&quot;&#xA;&quot;B2&quot; 5.4 3.9 1.3 0.4 &quot;blast&quot;&#xA;&quot;B3&quot; 5.1 3.5 1.4 0.3 &quot;blast&quot;&#xA;&quot;B4&quot; 5.7 3.8 1.7 0.3 &quot;blast&quot;&#xA;&quot;B5&quot; 5.1 3.8 1.5 0.3 &quot;blast&quot;&#xA;&quot;B6&quot; 5.4 3.4 1.7 0.2 &quot;blast&quot;&#xA;&quot;B7&quot; 5.1 3.7 1.5 0.4 &quot;blast&quot;&#xA;&quot;B8&quot; 4.6 3.6 1 0.2 &quot;blast&quot;&#xA;&quot;B9&quot; 5.1 3.3 1.7 0.5 &quot;blast&quot;&#xA;&quot;B10&quot; 4.8 3.4 1.9 0.2 &quot;blast&quot;&#xA;&quot;B11&quot; 5 3 1.6 0.2 &quot;blast&quot;&#xA;&quot;B12&quot; 5 3.4 1.6 0.4 &quot;blast&quot;&#xA;&quot;B13&quot; 5.2 3.5 1.5 0.2 &quot;blast&quot;&#xA;&quot;B14&quot; 5.2 3.4 1.4 0.2 &quot;blast&quot;&#xA;&quot;B15&quot; 4.7 3.2 1.6 0.2 &quot;blast&quot;&#xA;&quot;B16&quot; 4.8 3.1 1.6 0.2 &quot;blast&quot;&#xA;&quot;B17&quot; 5.4 3.4 1.5 0.4 &quot;blast&quot;&#xA;&quot;B18&quot; 5.2 4.1 1.5 0.1 &quot;blast&quot;&#xA;&quot;B19&quot; 5.5 4.2 1.4 0.2 &quot;blast&quot;&#xA;&quot;B20&quot; 4.9 3.1 1.5 0.2 &quot;blast&quot;&#xA;&quot;L1&quot; 5 3.2 1.2 0.2 &quot;LSC&quot;&#xA;&quot;L2&quot; 5.5 3.5 1.3 0.2 &quot;LSC&quot;&#xA;&quot;L3&quot; 4.9 3.6 1.4 0.1 &quot;LSC&quot;&#xA;&quot;L4&quot; 4.4 3 1.3 0.2 &quot;LSC&quot;&#xA;&quot;L5&quot; 5.1 3.4 1.5 0.2 &quot;LSC&quot;&#xA;&quot;L6&quot; 5 3.5 1.3 0.3 &quot;LSC&quot;&#xA;&quot;L7&quot; 4.5 2.3 1.3 0.3 &quot;LSC&quot;&#xA;&quot;L8&quot; 4.4 3.2 1.3 0.2 &quot;LSC&quot;&#xA;&quot;L9&quot; 5 3.5 1.6 0.6 &quot;LSC&quot;&#xA;&quot;L10&quot; 5.1 3.8 1.9 0.4 &quot;LSC&quot;&#xA;&quot;L11&quot; 4.8 3 1.4 0.3 &quot;LSC&quot;&#xA;&quot;L12&quot; 5.1 3.8 1.6 0.2 &quot;LSC&quot;&#xA;&quot;L13&quot; 4.6 3.2 1.4 0.2 &quot;LSC&quot;&#xA;&quot;L14&quot; 5.3 3.7 1.5 0.2 &quot;LSC&quot;&#xA;&quot;L15&quot; 5 3.3 1.4 0.2 &quot;LSC&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The code is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(&quot;scatterplot3d&quot;)&#xA;colors &amp;lt;- c(&quot;#999999&quot;, &quot;#E69F00&quot;, &quot;#56B4E9&quot;) # Number of color according to the number of groups&#xA;colors &amp;lt;- colors[as.numeric(data.c1&lt;span class=&quot;math-container&quot;&gt;$colend)] # you can put here the column containing the name of population or sample etc.&#xA;pca1 &amp;lt;- prcomp(data.c1[, -5]) # PCA on columns except the last column&#xA;s3d&amp;lt;-scatterplot3d(pca1$&lt;/span&gt;x[, 1], pca1&lt;span class=&quot;math-container&quot;&gt;$x[, 2],pca1$&lt;/span&gt;x[, 3],grid=TRUE,xlab=&quot;PC1&quot;,ylab=&quot;PC2&quot;, zlab=&quot;PC3&quot;, pch = 16,color=colors)&#xA;legend(&quot;right&quot;,legend = levels(data.c1&lt;span class=&quot;math-container&quot;&gt;$colend),col =  c(&quot;#999999&quot;, &quot;#E69F00&quot;, &quot;#56B4E9&quot;), pch = 16,inset=-0.04,bty=&quot;n&quot;)&#xA;text(s3d$&lt;/span&gt;xyz.convert(pca1$x[, 1:3]), labels = rownames(data.c1),cex= 0.7, col = &quot;black&quot;,pos=2.5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this way we get following plot:&lt;a href=&quot;https://i.stack.imgur.com/Uvo31.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Uvo31.png&quot; alt=&quot;Image2&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps!!!&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-10-31T10:26:32.760" LastActivityDate="2018-10-31T10:26:32.760" CommentCount="16" ContentLicense="CC BY-SA 4.0" />
  <row Id="5324" PostTypeId="2" ParentId="5322" CreationDate="2018-10-30T08:09:21.397" Score="2" Body="&lt;p&gt;If you want to plot the &quot;corrected&quot; expression, you will need to remove the variation introduced by these surrogate variables. Removing the expression affected can introduce some bias too and it is usually not recommended (despite comBat doing so). You should apply linear algebra, you can look at &lt;a href=&quot;http://llrs.github.io/IEO/IEO_report.html#removing_batch_effect_with_svd&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is an example how to do it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(&quot;corpcor&quot;)&#xA;s &amp;lt;- corpcor::fast.svd(t(scale(t(dat), center = TRUE, scale = TRUE)))&#xA;pcSds &amp;lt;- s&lt;span class=&quot;math-container&quot;&gt;$d&#xA;pcSds[1] &amp;lt;- 0&#xA;svdexp &amp;lt;- s$&lt;/span&gt;u %*% diag(pcSds) %*% t(s$v)&#xA;colnames(svdexp) &amp;lt;- colnames(dat)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But if your surrogate variables do have a big enough influence on your MDS plot to be observable (IMHO) there would be too much batch effect to work with the data. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-10-30T08:09:21.397" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5325" PostTypeId="2" ParentId="5269" CreationDate="2018-10-30T08:31:19.413" Score="2" Body="&lt;p&gt;Answer eventually found on the BioConductor post:&#xA;&lt;a href=&quot;https://support.bioconductor.org/p/114354/#114585&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.bioconductor.org/p/114354/#114585&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It basically comes down to not using RSVSim, but other R packages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Final code snippit:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;genome = DNAStringSet(&#xA;  c(chr1 = &quot;AAAAAAAAAAAAAAAAAAAATTTTTTTTTTTTTTTTTTTT&quot;,&#xA;    chr2 = &quot;GGGGGGGGGGGGGGGGGGGGCCCCCCCCCCCCCCCCCCCC&quot;))&#xA;&#xA;sampleFromGenome &amp;lt;- function(genome, wd) {&#xA;  chrProb &amp;lt;- width(genome) / sum(width(genome))&#xA;  chr &amp;lt;- sample(names(genome), n, TRUE, prob = chrProb)&#xA;  start &amp;lt;- runif(n, 1, nchar(genome[chr]) - wd)&#xA;  GRanges(IRanges(start, width = wd), seqnames = chr)&#xA;}&#xA;&#xA;n &amp;lt;- 2&#xA;wd &amp;lt;- sample(5:10, n, TRUE)&#xA;insertFrom &amp;lt;- sampleFromGenome(genome[1], wd)&#xA;insertAt &amp;lt;- sampleFromGenome(genome[2], 0)&#xA;&#xA;#writeTable(insertFrom, stdout()) . # replace stdout with a file name&#xA;&#xA;sequence &amp;lt;- genome[insertFrom]&#xA;at &amp;lt;- split(insertAt, seqnames(insertAt))[names(genome[2])]&#xA;atSequence &amp;lt;- split(sequence, seqnames(insertAt))[names(genome[2])]&#xA;&#xA;replaceAt(genome[2], unname(ranges(at)), atSequence)&#xA;&#xA;  A DNAStringSet instance of length 1&#xA;    width seq                                                                                                                           names               &#xA;[1]    54 GGGGGGGGGGGGGGGGGGTTTTTTTTTGGCCCCCCCCCCCCTTTTTCCCCCCCC                                                                        chr2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;EDIT with RSVSim solution!&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Finally found a solution based on RSVSim&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;genome = DNAStringSet(&#xA;  c(chr1 = &quot;AAAAAAAAAAAAAAAAAAAATTTTTTTTTTTTTTTTTTTT&quot;,&#xA;    chr2 = &quot;GGGGGGGGGGGGGGGGGGGGCCCCCCCCCCCCCCCCCCCC&quot;))&#xA;&#xA;length_seq_chr1 = width(genome[1]) - 10&#xA;length_seq_chr2 = width(genome[2]) - 10&#xA;sizes &amp;lt;- c(3,3,5,5)&#xA;start_chr1 &amp;lt;- c(&quot;list&quot;, length(sizes))&#xA;start_chr2 &amp;lt;- c(&quot;list&quot;, length(sizes))&#xA;end_chr1 &amp;lt;- c(&quot;list&quot;, length(sizes))&#xA;index = 1&#xA;for(i in sizes){&#xA;  newstartchr1 &amp;lt;- sample(1:length_seq_chr1,1)&#xA;  newstartchr2 &amp;lt;- sample(1:length_seq_chr2,1)&#xA;  start_chr1[[index]] &amp;lt;- newstartchr1&#xA;  start_chr2[[index]] &amp;lt;- newstartchr2&#xA;  end_chr1[[index]] &amp;lt;- newstartchr1 + (i-1)&#xA;  index &amp;lt;- index + 1}&#xA;start_chr1 &amp;lt;- as.integer(start_chr1)&#xA;end_chr1 &amp;lt;- as.integer(end_chr1)&#xA;start_chr2 &amp;lt;- as.integer(start_chr2)&#xA;&#xA;knownInsertion2 = GRanges(IRanges(c(start_chr1),c(end_chr1)),seqnames=&quot;chr1&quot;, chrB=&quot;chr2&quot;, startB=c(start_chr2), copied=TRUE)&#xA;knownInsertion2&#xA;&#xA;sim = simulateSV(output=NA, genome=genome, regionsIns=knownInsertion2, bpSeqSize=5, random=FALSE, verbose=FALSE)&#xA;sim&#xA;metadata(sim)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Gives as an result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; knownInsertion2&#xA;GRanges object with 4 ranges and 3 metadata columns:&#xA;      seqnames    ranges strand |        chrB    startB    copied&#xA;         &amp;lt;Rle&amp;gt; &amp;lt;IRanges&amp;gt;  &amp;lt;Rle&amp;gt; | &amp;lt;character&amp;gt; &amp;lt;integer&amp;gt; &amp;lt;logical&amp;gt;&#xA;  [1]     chr1  [21, 24]      * |        chr2        30      TRUE&#xA;  [2]     chr1  [16, 19]      * |        chr2        21      TRUE&#xA;  [3]     chr1  [ 6, 11]      * |        chr2        10      TRUE&#xA;  [4]     chr1  [ 9, 14]      * |        chr2        27      TRUE&#xA;  -------&#xA;  seqinfo: 1 sequence from an unspecified genome; no seqlengths&#xA;&amp;gt; &#xA;&amp;gt; sim = simulateSV(output=NA, genome=genome, regionsIns=knownInsertion2, bpSeqSize=5, random=FALSE, verbose=FALSE)&#xA;&amp;gt; sim&#xA;  A DNAStringSet instance of length 2&#xA;    width seq                                                                                                                           names               &#xA;[1]    40 AAAAAAAAAAAAAAAAAAAATTTTTTTTTTTTTTTTTTTT                                                                                      chr1&#xA;[2]    60 GGGGGGGGGAAAAAAGGGGGGGGGGGAAAACCCCCCAAAAAACCCTTTTCCCCCCCCCCC                                                                  chr2&#xA;&amp;gt; metadata(sim)&#xA;$insertions&#xA;  Name ChrA StartA EndA ChrB StartB EndB Size Copied BpSeqA BpSeqB_5prime BpSeqB_3prime&#xA;1    1 chr1     21   24 chr2     30   33    4   TRUE                 CCTT          TTCC&#xA;2    2 chr1     16   19 chr2     21   24    4   TRUE                 GGAA          AACC&#xA;3    3 chr1      6   11 chr2     10   15    6   TRUE                 GGAA          AAGG&#xA;4    4 chr1      9   14 chr2     27   32    6   TRUE                 CCAA          AACC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To make this customizable change:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;sizes &amp;lt;- c(whatever, numbers, you, like)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;output=NA (to a file path prefix to get a .csv file with the metadata)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;genome = DNAStringSet() To replace the sample sequence&lt;/p&gt;&#xA;&#xA;&lt;p&gt;bpSeqSize = number (to change how many of the flanking region is shown in the metadata)&lt;/p&gt;&#xA;" OwnerUserId="3552" LastEditorUserId="3552" LastEditDate="2018-10-30T12:39:37.670" LastActivityDate="2018-10-30T12:39:37.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5327" PostTypeId="1" CreationDate="2018-10-30T09:01:57.007" Score="6" ViewCount="2305" Body="&lt;p&gt;I have the following error when running bowtie2:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error: Read HWI-D00466:116:CC62WANXX:3:1102:7363:63646 1:N:0:GCACACG&#xA;  has more read characters than quality values.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I now want to remove all 4 lines of this specific read from the fastq file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I use awk or sed to do this?&lt;/p&gt;&#xA;" OwnerUserId="3593" LastEditorUserId="292" LastEditDate="2018-10-31T12:42:25.153" LastActivityDate="2018-11-21T19:17:19.677" Title="Delete all 4 lines of a fastq read from a fastq file using read ID" Tags="|ngs|fastq|filtering|bioawk|text-processing|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5328" PostTypeId="2" ParentId="5327" CreationDate="2018-10-30T09:40:39.210" Score="6" Body="&lt;p&gt;If you are 100% sure the read only has 4 lines (they can have more), you can use this &lt;code&gt;sed&lt;/code&gt; command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -i.bak '/^@HWI-D00466:116:CC62WANXX:3:1102:7363:63646 1:N:0:GCACACG/,+3d'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;-i.bak&lt;/code&gt; makes &lt;code&gt;sed&lt;/code&gt; modify the original file and create a backup copy with the same name and the extension &lt;code&gt;.bak&lt;/code&gt;. The command just means &quot;delete the line matching the pattern and the next three lines&quot;.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-11-21T19:17:19.677" LastActivityDate="2018-11-21T19:17:19.677" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5329" PostTypeId="2" ParentId="5308" CreationDate="2018-10-30T10:45:52.510" Score="1" Body="&lt;p&gt;A simple BLAST on the &lt;a href=&quot;https://www.uniprot.org/blast/&quot; rel=&quot;nofollow noreferrer&quot;&gt;UniProt website&lt;/a&gt; does the trick (target database &lt;em&gt;plants&lt;/em&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best matches are subsequences &lt;code&gt;ETVEREKE&lt;/code&gt; and &lt;code&gt;EKKRRETV&lt;/code&gt; in &lt;em&gt;Green alga&lt;/em&gt; and &lt;em&gt;Eelgrass&lt;/em&gt; respectively.&lt;/p&gt;&#xA;" OwnerUserId="3580" LastActivityDate="2018-10-30T10:45:52.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5330" PostTypeId="1" AcceptedAnswerId="5335" CreationDate="2018-10-30T14:35:55.930" Score="6" ViewCount="1342" Body="&lt;p&gt;I have two PCR amplicons that have been multiplexed and sequenced using the nanopore minion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have aligned the fastq reads using minimap2 with a reference file containing both amplicon sequences and generated a bam file that I have viewed using IGV.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a way to generate some simple summary statistics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, is there a way to extract the total number of fastq reads aligning to each amplicon reference from the bam file?  &lt;/p&gt;&#xA;" OwnerUserId="3596" LastEditorUserId="48" LastEditDate="2018-10-30T14:44:01.090" LastActivityDate="2018-10-30T18:00:03.153" Title="Total reads aligning to each reference within a bam file" Tags="|sequence-alignment|sam|nanopore|minion|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5331" PostTypeId="2" ParentId="5330" CreationDate="2018-10-30T14:57:16.020" Score="1" Body="&lt;p&gt;This can be done using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   samtools flagstat your_bam_file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1194" LastActivityDate="2018-10-30T14:57:16.020" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5333" PostTypeId="2" ParentId="5330" CreationDate="2018-10-30T15:08:17.377" Score="2" Body="&lt;p&gt;This one-liner below will work better for long reads than &lt;code&gt;samtools flagstat&lt;/code&gt; in that it only counts the primary alignment for each read and &lt;code&gt;samtools flagstat&lt;/code&gt; doesn't seem to calculate some stats for long reads. I have never seen &lt;code&gt;samtools flagstat&lt;/code&gt; output stats on a per-reference basis, but am curious if so!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This answer filters out secondary and supplementary alignments for your reads (&lt;code&gt;-F 2304&lt;/code&gt;) that have some alignment to both amplicon reference and just keep the best one. This might give a more accurate idea of how many reads of each amplicon are in the library.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -F 2304 myfile.bam | awk -F &lt;span class=&quot;math-container&quot;&gt;$'\t' '{a[$1, $3]++} END{for (i in a) {split (i, sep, SUBSEP); print sep[1], sep[2], a[i]}}' | uniq | awk '{print($&lt;/span&gt;2)}' | uniq -c | sort -k1 -nr&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://askubuntu.com/questions/592273/count-the-number-of-unique-values-based-on-two-columns-in-a-spreadsheet&quot;&gt;adapted from this.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-10-30T15:08:17.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5334" PostTypeId="1" AcceptedAnswerId="5361" CreationDate="2018-10-30T17:53:53.247" Score="3" ViewCount="280" Body="&lt;p&gt;I am trying to use beagle 4.1 for phasing and IBD analyses. I was wondering if I need to run the analysis by chromosome or I can put them together in the same vcf file.&lt;/p&gt;&#xA;" OwnerUserId="3536" LastEditorUserId="2085" LastEditDate="2018-10-30T20:29:27.683" LastActivityDate="2018-11-16T10:27:51.230" Title="Using Beagle 4.1 for phasing and IBD - chromosomes" Tags="|snp|phasing|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5335" PostTypeId="2" ParentId="5330" CreationDate="2018-10-30T18:00:03.153" Score="8" Body="&lt;p&gt;The quick way to get the number of alignments on each reference is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools idxstats my_bam.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Number of reads on each reference is column 3. Although, as has been pointed out, this will give you the total number of &lt;em&gt;alignments&lt;/em&gt; per reference, not the total number of reads (each read might give rise to more than one alignment). That said I do tend to us this as generally I'm after a rough approximation, rather than an accurate number. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In theory, only one alignment for each read should be marked as primary, so the following should give you what you need quickly and at low memory usage:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -bF 2304 my_bam.bam &amp;gt; primary_only.bam&#xA;samtools index primary_only.bam&#xA;samtools idxstats primary_only.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-10-30T18:00:03.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5336" PostTypeId="2" ParentId="5327" CreationDate="2018-10-30T19:20:55.403" Score="0" Body="&lt;p&gt;To generalize this to remove all reads that do not have the same length of read and sequence quality, you can use this bioawk one-liner:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bioawk -cfastx '{ if (length(&lt;span class=&quot;math-container&quot;&gt;$seq) == length($&lt;/span&gt;qual) ){printf(&quot;@%s %s\n%s\n+\n%s\n&quot;, &lt;span class=&quot;math-container&quot;&gt;$name, $&lt;/span&gt;comment, &lt;span class=&quot;math-container&quot;&gt;$seq, $&lt;/span&gt;qual)} }' my_fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This strips anything in the &quot;fastq comment field&quot;, &lt;code&gt;+&lt;/code&gt;, but that almost never has anything there anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Append &lt;code&gt;| gzip &amp;gt; filtered.fastq.gz&lt;/code&gt; to the above command to gzip the output.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="298" LastEditDate="2018-10-31T13:50:33.817" LastActivityDate="2018-10-31T13:50:33.817" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5337" PostTypeId="2" ParentId="5320" CreationDate="2018-10-30T19:56:24.700" Score="3" Body="&lt;p&gt;&lt;strong&gt;TSNEPlot()&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;TSNEPlot()&lt;/code&gt; will always treat your variables as discrete. My approach is to manually generate a gradient with unique colors for each factor level and pass it to the &lt;code&gt;cols.use&lt;/code&gt; argument in &lt;code&gt;TSNEPlot()&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#generate values for testing purposes, one value for each cell&#xA;value &amp;lt;- sample(seq(from=8, to=48, by=1), size = length(rownames(unfiltered_cca@meta.data)), replace=TRUE)&#xA;names(value) &amp;lt;- rownames(exp@meta.data)&#xA;exp &amp;lt;- AddMetaData(object=exp, metadata=value, col.name=&quot;value&quot;)&#xA;exp &amp;lt;-  SetIdent(exp, ident.use=exp@meta.data$value)&#xA;#create a gradient between specified colors, multiply by sequence to get appropriate length&#xA;#length.out is the number of different colors, number of factor levels&#xA;cols &amp;lt;-  scales::seq_gradient_pal(low=&quot;beige&quot;, high=&quot;red&quot;, space=&quot;Lab&quot;)(seq(from=0, to=1,length.out=48))&#xA;&#xA;&#xA;TSNEPlot(exp, colors.use=cols)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PKps4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PKps4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would present this with no legend:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TSNEPlot(exp, colors.use=cols, no.legend = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ix0cj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ix0cj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;FeaturePlot()&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also simply use &lt;code&gt;FeaturePlot()&lt;/code&gt; instead of &lt;code&gt;TSNEPlot()&lt;/code&gt; to visualize the gradient. Using the same data as above:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FeaturePlot(object = exp, features.plot = &quot;value&quot;, reduction.use = &quot;tsne&quot;, no.legend = FALSE, cols.use = c(&quot;beige&quot;, &quot;red&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hZCd4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hZCd4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You ask for a continuous scale, but this is not what is shown in your second plot.  You have a different color for each discrete range of values. I assume here that you want a continuous gradient. To achieve something like the second plot you provided requires a different approach.&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="181" LastEditDate="2018-10-30T20:12:08.690" LastActivityDate="2018-10-30T20:12:08.690" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="5338" PostTypeId="1" CreationDate="2018-10-31T09:25:43.883" Score="2" ViewCount="1370" Body="&lt;p&gt;I'm clustering genes in terms of expression after clustering them. I'm taking out clusters and trying to find out what kind of GO terms are coming up. I came across this figure from this &lt;a href=&quot;http://www.jimmunol.org/content/198/6/2238&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;, I want to do as the figure B:&lt;a href=&quot;https://i.stack.imgur.com/igawg.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/igawg.jpg&quot; alt=&quot;the second figure&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://drive.google.com/open?id=1YO7FOlZ7VG1ZSvWXaa1_L1rgoVCn-QLc&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; you can find the file I intend to use to make something like the above plot. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using this the code from &lt;a href=&quot;https://www.biostars.org/p/299161/&quot; rel=&quot;nofollow noreferrer&quot;&gt;a question in Biostar&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DAVIDfile &amp;lt;- &quot;DAVID.txt&quot;&#xA;DAVID &amp;lt;- read.table(DAVIDfile, sep=&quot;\t&quot;, header=TRUE)&#xA;colnames(DAVID)&#xA;class(DAVID)&#xA;&#xA;enrichBcutoff &amp;lt;- 0.05&#xA;DAVID &amp;lt;- subset(DAVID, Benjamini&amp;lt; enrichBcutoff)&#xA;DAVID &amp;lt;- DAVID[,c(1,2,3,5,6,10)]&#xA;&#xA;head(DAVID)&#xA;&#xA;A &amp;lt;- data.frame(sigGeneList)&#xA;&#xA;annGSEA &amp;lt;- data.frame(row.names=A)&#xA;for (j in 1:length(sigGeneList))&#xA;{&#xA;  gene &amp;lt;- sigGeneList[j]&#xA;  pattern &amp;lt;- paste(&quot;^&quot;, gene, &quot;, |, &quot;, gene, &quot;&lt;span class=&quot;math-container&quot;&gt;$| &quot;, gene, &quot;,&quot;, sep=&quot;&quot;)&#xA;    for (k in 1:nrow(DAVID))&#xA;  {&#xA;    if (any(grepl(pattern, DAVID$&lt;/span&gt;Genes[k])))&#xA;    {&#xA;      annGSEA[j,k] &amp;lt;- 1&#xA;    }&#xA;    else&#xA;    {&#xA;      annGSEA[j,k] &amp;lt;- 0&#xA;    }&#xA;  }&#xA;}&#xA;colnames(annGSEA) &amp;lt;- DAVID[,2]&#xA;&#xA;annGSEA &amp;lt;- annGSEA[,apply(annGSEA, 2, mean)!=0]&#xA;&#xA;annGSEA &amp;lt;- annGSEA[apply(annGSEA, 1, mean)!=0,]&#xA;&#xA;&#xA;require(ComplexHeatmap)&#xA;require(circlize)&#xA;&#xA;topTable &amp;lt;- topTable[which(topTable&lt;span class=&quot;math-container&quot;&gt;$Gene %in% rownames(annGSEA)),]&#xA;topTable &amp;lt;- topTable[match(rownames(annGSEA), topTable$&lt;/span&gt;Gene),]&#xA;&#xA;geneLab=10&#xA;termLab=8&#xA;&#xA;#Create heatmap annotations&#xA;dfMinusLog10FDRGenes &amp;lt;- data.frame(-log10(topTable[which(topTable[,1] %in% rownames(annGSEA)),&quot;padj&quot;]))&#xA;dfMinusLog10FDRGenes[dfMinusLog10FDRGenes==&quot;Inf&quot;] &amp;lt;- 0&#xA;dfFoldChangeGenes &amp;lt;- data.frame(topTable[which(topTable[,1] %in% rownames(annGSEA)),&quot;log2FoldChange&quot;])&#xA;dfGeneAnno &amp;lt;- data.frame(dfMinusLog10FDRGenes, dfFoldChangeGenes)&#xA;colnames(dfGeneAnno) &amp;lt;- c(&quot;DEG\nsignificance\nscore&quot;, &quot;Regulation&quot;)&#xA;dfGeneAnno[,2] &amp;lt;- ifelse(dfGeneAnno[,2]&amp;gt;0, &quot;Up-regulated&quot;, &quot;Down-regulated&quot;)&#xA;colours &amp;lt;- list(&quot;Regulation&quot;=c(&quot;Up-regulated&quot;=&quot;royalblue&quot;, &quot;Down-regulated&quot;=&quot;yellow&quot;))&#xA;haGenes &amp;lt;- rowAnnotation(df=dfGeneAnno, col=colours, width=unit(1,&quot;cm&quot;))&#xA;&#xA;dfMinusLog10BenjaminiTerms &amp;lt;- data.frame(-log10(read.table(DAVIDfile, sep=&quot;\t&quot;, header=TRUE)[which(read.table(DAVIDfile, sep=&quot;\t&quot;, header=TRUE)$Term %in% colnames(annGSEA)),&quot;Benjamini&quot;]))&#xA;colnames(dfMinusLog10BenjaminiTerms) &amp;lt;- &quot;GO Term\nsignificance\nscore&quot;&#xA;haTerms &amp;lt;- HeatmapAnnotation(df=dfMinusLog10BenjaminiTerms,&#xA;                             colname=anno_text(colnames(annGSEA), rot=40, just=&quot;right&quot;, offset=unit(1,&quot;npc&quot;)-unit(2,&quot;mm&quot;), gp=gpar(fontsize=termLab)),&#xA;                             annotation_height=unit.c(unit(1, &quot;cm&quot;), unit(8, &quot;cm&quot;)))&#xA;&#xA;pdf(&quot;GO.pdf&quot;, width=7, height=12)&#xA;hmapGSEA &amp;lt;- Heatmap(annGSEA,&#xA;&#xA;                    name=&quot;My enrichment&quot;,&#xA;&#xA;                    split=dfGeneAnno[,2],&#xA;&#xA;                    col=c(&quot;0&quot;=&quot;white&quot;, &quot;1&quot;=&quot;forestgreen&quot;),&#xA;&#xA;                    rect_gp=gpar(col=&quot;grey85&quot;),&#xA;&#xA;                    cluster_rows=T,&#xA;                    show_row_dend=T,&#xA;                    row_title=&quot;Statistically-significant genes&quot;,&#xA;                    row_title_side=&quot;left&quot;,&#xA;                    row_title_gp=gpar(fontsize=12, fontface=&quot;bold&quot;),&#xA;                    row_title_rot=0,&#xA;                    show_row_names=TRUE,&#xA;                    row_names_gp=gpar(fontsize=geneLab, fontface=&quot;bold&quot;),&#xA;                    row_names_side=&quot;left&quot;,&#xA;                    row_names_max_width=unit(15, &quot;cm&quot;),&#xA;                    row_dend_width=unit(10,&quot;mm&quot;),&#xA;&#xA;                    cluster_columns=T,&#xA;                    show_column_dend=T,&#xA;                    column_title=&quot;Enriched terms&quot;,&#xA;                    column_title_side=&quot;top&quot;,&#xA;                    column_title_gp=gpar(fontsize=12, fontface=&quot;bold&quot;),&#xA;                    column_title_rot=0,&#xA;                    show_column_names=FALSE,&#xA;                    #column_names_gp=gpar(fontsize=termLab, fontface=&quot;bold&quot;),&#xA;                    #column_names_max_height=unit(15, &quot;cm&quot;),&#xA;&#xA;                    show_heatmap_legend=FALSE,&#xA;&#xA;                    #width=unit(12.5, &quot;cm&quot;),&#xA;&#xA;                    clustering_distance_columns=&quot;euclidean&quot;,&#xA;                    clustering_method_columns=&quot;ward.D2&quot;,&#xA;                    clustering_distance_rows=&quot;euclidean&quot;,&#xA;                    clustering_method_rows=&quot;ward.D2&quot;,&#xA;&#xA;                    bottom_annotation=haTerms)&#xA;&#xA;draw(hmapGSEA + haGenes, heatmap_legend_side=&quot;right&quot;, annotation_legend_side=&quot;right&quot;)&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However there is an error message after annGSEA: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dput(annGSEA)&#xA;structure(list(), .Names = character(0), row.names = &quot;c(\&quot;LRP1\&quot;, \&quot;ELAVL4\&quot;, \&quot;ENOX1\&quot;, \&quot;TRIP6\&quot;, \&quot;KHDRBS3\&quot;, \&quot;TDRD9\&quot;, \&quot;THOC6\&quot;, \&quot;CPSF1\&quot;, \&quot;HSPB1\&quot;, \&quot;SF3A2\&quot;, \&quot;STXBP1\&quot;, \&quot;DUS3L\&quot;, \&quot;INTS1\&quot;, \&quot;DZIP1L\&quot;, \&quot;HIST1H1B\&quot;, \&quot;TRMT2A\&quot;, \&quot;RPUSD1\&quot;, \&quot;MRPS2\&quot;, \&quot;EXOSC5\&quot;, \&quot;TDRD1\&quot;, \&quot;PURG\&quot;, \&quot;NOL12\&quot;, \&quot;TRMT61A\&quot;, \&quot;XAB2\&quot;, \&quot;TRMT1\&quot;, \&quot;PTRF\&quot;, \&quot;POLR2I\&quot;, \&quot;MAZ\&quot;, \&quot;TBL3\&quot;, \&quot;BOP1\&quot;, \&quot;RBM10\&quot;, \&quot;TSR3\&quot;, \&quot;SART1\&quot;, \&quot;CCDC86\&quot;, \&quot;SAMD4A\&quot;, \&quot;RBM11\&quot;, \&quot;RPS5\&quot;, \&quot;HNRNPLL\&quot;, \&quot;POLRMT\&quot;, \&quot;NSUN7\&quot;, \&quot;PABPC5\&quot;, \&quot;DDX54\&quot;, \&quot;PRRC2B\&quot;, \&quot;RPS15\&quot;, \&quot;MYBBP1A\&quot;, \&quot;DUS1L\&quot;, \&quot;ARHGEF28\&quot;, \&quot;RPL28\&quot;, \&quot;ZNF598\&quot;, \&quot;ZMAT5\&quot;, \&quot;RPL36\&quot;, \&quot;RPL8\&quot;, \&quot;DHX37\&quot;, \&quot;ZC3H7B\&quot;, \n\&quot;RRP9\&quot;, \&quot;PQBP1\&quot;, \&quot;ZNF385A\&quot;, \&quot;PUS1\&quot;, \&quot;DDX49\&quot;, \&quot;TCOF1\&quot;, \&quot;BICC1\&quot;, \&quot;GLTSCR2\&quot;, \&quot;SCAF1\&quot;, \&quot;MRPL4\&quot;, \&quot;FSCN1\&quot;, \&quot;EDF1\&quot;, \&quot;NOVA1\&quot;, \&quot;PCBP3\&quot;, \&quot;FBL\&quot;, \&quot;MRPL55\&quot;, \&quot;PUF60\&quot;, \&quot;NYNRIN\&quot;, \&quot;ZFP36L1\&quot;, \&quot;RPS9\&quot;, \&quot;MKRN3\&quot;, \&quot;GTF2F1\&quot;, \&quot;EEFSEC\&quot;, \&quot;FARSA\&quot;, \&quot;TARBP2\&quot;, \&quot;RPL35\&quot;, \&quot;RPS19\&quot;, \&quot;ZC3H3\&quot;, \&quot;H1F0\&quot;, \&quot;BST2\&quot;, \&quot;FASN\&quot;, \&quot;ZYX\&quot;, \&quot;NOC4L\&quot;, \&quot;LONP1\&quot;, \&quot;RUVBL2\&quot;, \&quot;MRPL38\&quot;, \&quot;TRAP1\&quot;, \&quot;TUFM\&quot;, \&quot;RBM42\&quot;, \&quot;TNS1\&quot;, \&quot;EDC4\&quot;, \&quot;WDR6\&quot;, \&quot;PCBP4\&quot;, \&quot;PUSL1\&quot;, \&quot;EEF2\&quot;, \&quot;RPS28\&quot;, \&quot;RAVER1\&quot;, \&quot;APEH\&quot;, \&quot;TSEN54\&quot;, \&quot;VARS\&quot;, \&quot;C7orf50\&quot;, \&quot;GRN\&quot;, \&quot;ARHGEF1\&quot;, \&quot;SF3B5\&quot;, \&quot;RPL18\&quot;, \&quot;EIF3G\&quot;, \n\&quot;ANG\&quot;, \&quot;EIF5AL1\&quot;, \&quot;HIST1H1C\&quot;, \&quot;RPL13\&quot;, \&quot;U2AF1L4\&quot;, \&quot;C9orf114\&quot;, \&quot;FASTK\&quot;, \&quot;TSEN34\&quot;, \&quot;SRRT\&quot;, \&quot;RPS2\&quot;, \&quot;MRPL54\&quot;, \&quot;CSRP1\&quot;, \&quot;HELZ2\&quot;, \&quot;RPLP2\&quot;, \&quot;RPL18A\&quot;, \&quot;PRRC2A\&quot;, \&quot;GTPBP1\&quot;, \&quot;NOC2L\&quot;, \&quot;RBM19\&quot;, \&quot;RNASEH2A\&quot;, \&quot;ADARB2\&quot;, \&quot;SERPINH1\&quot;, \&quot;CCDC9\&quot;, \&quot;DXO\&quot;, \&quot;GNB2L1\&quot;, \&quot;FLNA\&quot;, \&quot;RNASEH2C\&quot;, \&quot;IPO13\&quot;, \&quot;HABP4\&quot;, \&quot;C11orf68\&quot;, \&quot;QTRT1\&quot;, \&quot;DGCR14\&quot;, \&quot;RPL3\&quot;, \&quot;SETD1A\&quot;, \&quot;GEMIN4\&quot;, \&quot;NPM2\&quot;, \&quot;MRPS26\&quot;, \&quot;RRP7A\&quot;, \&quot;CCDC124\&quot;, \&quot;CACTIN\&quot;, \&quot;ISG20\&quot;, \&quot;INTS5\&quot;, \&quot;SNTB2\&quot;, \&quot;RPUSD3\&quot;, \&quot;RPL29\&quot;, \&quot;PPRC1\&quot;, \&quot;PPAN\&quot;, \&quot;PRPF40B\&quot;, \&quot;RBFOX2\&quot;, \&quot;C15orf52\&quot;, \&quot;ALYREF\&quot;, \&quot;EXOSC4\&quot;, \n\&quot;GNL1\&quot;, \&quot;SIDT2\&quot;, \&quot;REPIN1\&quot;, \&quot;CHERP\&quot;, \&quot;NOB1\&quot;, \&quot;RPLP1\&quot;, \&quot;EIF5A2\&quot;, \&quot;RRP1\&quot;, \&quot;MRPL41\&quot;, \&quot;RPL19\&quot;, \&quot;SUGP1\&quot;, \&quot;NAA38\&quot;, \&quot;CDC42EP4\&quot;, \&quot;EEF1D\&quot;, \&quot;DDX41\&quot;, \&quot;PRPF19\&quot;, \&quot;CLASRP\&quot;, \&quot;PRPF6\&quot;, \&quot;ZC3H12B\&quot;, \&quot;SNRPB\&quot;, \&quot;MEPCE\&quot;, \&quot;COL14A1\&quot;, \&quot;WBSCR16\&quot;, \&quot;LSM7\&quot;, \&quot;PELP1\&quot;, \&quot;VARS2\&quot;, \&quot;TRPT1\&quot;, \&quot;RPS10\&quot;, \&quot;PTBP1\&quot;, \&quot;RPS8\&quot;, \&quot;CPSF3L\&quot;, \&quot;NOSIP\&quot;, \&quot;TCF20\&quot;, \&quot;PTCD1\&quot;, \&quot;APEX1\&quot;, \&quot;C1orf35\&quot;, \&quot;MRPL23\&quot;, \&quot;TWF2\&quot;, \&quot;CTIF\&quot;, \&quot;DHX38\&quot;, \&quot;MRPS34\&quot;, \&quot;SYMPK\&quot;, \&quot;U2AF2\&quot;, \&quot;QARS\&quot;, \&quot;CCAR2\&quot;, \&quot;RPS3\&quot;, \&quot;RPS16\&quot;, \&quot;P4HB\&quot;, \&quot;MVP\&quot;, \&quot;POLR2J\&quot;, \&quot;SKIV2L\&quot;, \&quot;GTPBP3\&quot;, \&quot;ZNF768\&quot;, \&quot;UBA52\&quot;, \&quot;SUPT5H\&quot;, \n\&quot;POLR2L\&quot;, \&quot;POLR2A\&quot;, \&quot;EIF3K\&quot;, \&quot;R3HCC1\&quot;, \&quot;PIH1D1\&quot;, \&quot;ATXN2L\&quot;, \&quot;OAS3\&quot;, \&quot;IPO4\&quot;, \&quot;CCDC137\&quot;, \&quot;SETD1B\&quot;, \&quot;ILF3\&quot;, \&quot;TBRG4\&quot;, \&quot;PKM\&quot;, \&quot;AARS2\&quot;, \&quot;FAU\&quot;, \&quot;ACTN4\&quot;, \&quot;RPS19BP1\&quot;, \&quot;ANGEL1\&quot;, \&quot;PLD6\&quot;, \&quot;MOV10\&quot;, \&quot;EIF6\&quot;, \&quot;DRG2\&quot;, \&quot;AARS\&quot;, \&quot;PRR3\&quot;, \&quot;AC004381.6\&quot;, \&quot;RBPMS\&quot;, \&quot;CPSF4\&quot;, \&quot;NOL6\&quot;, \&quot;RPL13A\&quot;, \&quot;NPM3\&quot;, \&quot;UBA1\&quot;, \&quot;REXO4\&quot;, \&quot;RRP12\&quot;, \&quot;UPF1\&quot;, \&quot;RPL11\&quot;, \&quot;DDX43\&quot;, \&quot;SNRPA\&quot;, \&quot;ISY1\&quot;, \&quot;MRPS18A\&quot;, \&quot;RBMS1\&quot;, \&quot;CSTB\&quot;, \&quot;FKBP3\&quot;, \&quot;MTIF3\&quot;, \&quot;MRPL39\&quot;, \&quot;CLK4\&quot;, \&quot;SRP54\&quot;, \&quot;ERI1\&quot;, \&quot;CLK1\&quot;, \&quot;IMP3\&quot;, \&quot;GRSF1\&quot;, \&quot;GTF3A\&quot;, \&quot;FASTKD1\&quot;, \&quot;DYNC1LI1\&quot;, \&quot;PTGES3\&quot;, \&quot;TRIT1\&quot;, \&quot;EXOSC3\&quot;, \n\&quot;IGF2BP3\&quot;, \&quot;TRIM71\&quot;, \&quot;MCTS1\&quot;, \&quot;GEMIN2\&quot;, \&quot;SUMO1\&quot;, \&quot;NQO1\&quot;, \&quot;HSPA8\&quot;, \&quot;POLR2G\&quot;, \&quot;TAF9\&quot;, \&quot;MRPL33\&quot;, \&quot;FAM46A\&quot;, \&quot;MRPS36\&quot;, \&quot;FNDC3B\&quot;, \&quot;DDX52\&quot;, \&quot;DYNLL1\&quot;, \&quot;RBM43\&quot;, \&quot;MANF\&quot;, \&quot;TDRD7\&quot;, \&quot;AHNAK\&quot;, \&quot;IFIT2\&quot;, \&quot;ZMAT3\&quot;, \&quot;APOBEC3H\&quot;, \&quot;RTN4\&quot;, \&quot;S100A4\&quot;, \&quot;PIH1D2\&quot;, \&quot;ANXA2\&quot;, \&quot;PIWIL3\&quot;, \&quot;TMSB4X\&quot;, \&quot;PIWIL4\&quot;, \&quot;RNASE6\&quot;, \&quot;LGALS1\&quot;, \&quot;RNASE3\&quot;, \&quot;TLR7\&quot;, \&quot;MAEL\&quot;, \&quot;SAMHD1\&quot;, \&quot;RNASE2\&quot;, \&quot;C2orf15\&quot;, \&quot;SIDT1\&quot;, \&quot;HERC5\&quot;, \&quot;PSIP1\&quot;, \&quot;RPL17\&quot;, \&quot;ZC3H12C\&quot;, \&quot;SLC16A3\&quot;, \&quot;TEP1\&quot;, \&quot;HEXIM1\&quot;, \&quot;CPEB4\&quot;, \&quot;ZC3H12D\&quot;, \&quot;CORO1A\&quot;, \&quot;ZFP36\&quot;, \&quot;KIF1C\&quot;, \&quot;WARS\&quot;, \&quot;CCRN4L\&quot;, \&quot;AFF2\&quot;, \n\&quot;TST\&quot;, \&quot;RBM47\&quot;, \&quot;LGALS3\&quot;, \&quot;NXF3\&quot;, \&quot;PPARGC1A\&quot;, \&quot;TLR8\&quot;)&quot;, class = &quot;data.frame&quot;)&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="285" LastEditorUserId="131" LastEditDate="2019-05-11T07:13:34.163" LastActivityDate="2019-05-11T07:13:34.163" Title="GO Term heatmap plot in terms of P value or fold enrichment" Tags="|r|rna-seq|visualization|figure-reproduction|heatmap|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="5339" PostTypeId="2" ParentId="5292" CreationDate="2018-10-31T13:33:36.647" Score="2" Body="&lt;p&gt;This is the typical kind of job for &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;snakemake&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming your have one file per replicate named for instance &lt;code&gt;T9/Infected/Rep1/Rep1.fastq.gz&lt;/code&gt;, you can prepare a file that you call &lt;code&gt;Snakefile&lt;/code&gt; with the following content:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;timepoints = list(range(10))&#xA;conditions = [&quot;control&quot;, &quot;infected&quot;]&#xA;replicates = [1, 2, 3]&#xA;&#xA;rule all:&#xA;    input:&#xA;        expand(&#xA;            &quot;T{time}/{cond}/Rep{rep}/Rep{rep}_fastqc.html&quot;,&#xA;            time=timepoints,&#xA;            cond=conditions,&#xA;            rep=replicates)&#xA;&#xA;rule do_fastqc:&#xA;    input:&#xA;        fastq = &quot;T{time}/{cond}/Rep{rep}/Rep{rep}.fastq.gz&quot;&#xA;    output:&#xA;        html = &quot;T{time}/{cond}/Rep{rep}/Rep{rep}_fastqc.html&quot;&#xA;    shell:&#xA;        &quot;&quot;&quot;&#xA;        fastqc {input.fastq}&#xA;        &quot;&quot;&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Put this file in the directory that contains the &lt;code&gt;T*&lt;/code&gt; directories and run &lt;code&gt;snakemake&lt;/code&gt; from there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The top &lt;code&gt;all&lt;/code&gt; rule explains which files you want. The &lt;code&gt;do_fastqc&lt;/code&gt; rule explains how to make one fastqc report from one fastq.gz file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a bit more work, this can be used to submit jobs to a computing cluster. Snakemake has &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/executable.html#cluster-execution&quot; rel=&quot;nofollow noreferrer&quot;&gt;some tools for this&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't know the exact names of the fastq files but they all follow the same pattern, you will need to use the &lt;code&gt;glob&lt;/code&gt; python module and do a little bit of programming to determine the possible values for &lt;code&gt;rep&lt;/code&gt;, &lt;code&gt;cond&lt;/code&gt; and &lt;code&gt;time&lt;/code&gt;. The &quot;snakefile&quot; can contain any python code you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there are no regular pattern in the file names, fix this issue first ;)&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2018-10-31T13:33:36.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5340" PostTypeId="2" ParentId="5327" CreationDate="2018-10-31T15:41:27.400" Score="1" Body="&lt;p&gt;Here's an &lt;code&gt;awk&lt;/code&gt; one-liner that works as intended, unlike the &lt;code&gt;bioawk&lt;/code&gt; answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This removes all instances of fastq records in which the length of the seq and qual fields do not match. Again, append &lt;code&gt;| gzip &amp;gt; filtered.fastq.gz&lt;/code&gt; to gzip the output.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zcat bad_file.fq.gz | awk '{ pos = NR%4; if ( pos == 1) {h=&lt;span class=&quot;math-container&quot;&gt;$0} else if (pos == 2) \&#xA;  {s=$0} else if (pos==3){c=$0} else if ( pos==0 ) \&#xA;  {q=$&lt;/span&gt;0; if ( length(q) == length(s) ){printf(&quot;%s\n%s\n%s\n%s\n&quot;, h, s, c, q)}}}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Delete the &lt;code&gt;\&lt;/code&gt; characters at the end of the first two lines to make it easy to paste into your terminal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a shorter version of the same thing thanks to @terdon&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zcat file.fastq.gz | awk '{ record[++k]=&lt;span class=&quot;math-container&quot;&gt;$0; if(NR%4==2){slen=length($0)} else if(NR%4==0){if(length($&lt;/span&gt;0) == slen){ for(i in record){print record[i]}} k=0; }}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-11-01T15:51:00.017" LastActivityDate="2018-11-01T15:51:00.017" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5343" PostTypeId="2" ParentId="5327" CreationDate="2018-11-01T15:20:17.277" Score="3" Body="&lt;p&gt;When I can I like to do file processing line-by-line in the spirit of UNIX tools. You can read 4 lines from a Fastq file into 4 tab-separated values on a single line using &lt;code&gt;paste&lt;/code&gt;, and then use &lt;code&gt;grep&lt;/code&gt; to filter out the record in question. (Then you just have to turn the tabs back into line breaks.)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;paste - - - - &amp;lt; reads.fastq \&#xA;    | grep -v 'HWI-D00466:116:CC62WANXX:3:1102:7363:63646' \&#xA;    | tr '\t' '\n' \&#xA;    &amp;gt; reads-fixed.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, it's likely that other reads in your Fastq file are corrupted as well, in which case it's probably better to write a script in Python or some other language that discards all reads whose read length doesn't match the quality string length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But of course the most important question is: how did the data become corrupted in the first place? Where did this file come from, and what pre-processing steps were performed on it? Is getting rid of this read (or all reads with mismatched sequence/quality length) going to solve the problem for you? These aren't questions that we can answer for you, but probably deserve good answers. :-)&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2018-11-01T15:20:17.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5344" PostTypeId="2" ParentId="5320" CreationDate="2018-11-01T15:50:00.467" Score="3" Body="&lt;p&gt;If you would like to color discrete intervals on a gradient as opposed to having a continuous gradient (like your second plot), use this approach. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is similar to the approach in the answer I posted with the continuous scale, but we simply break up the continuous scale in to intervals and color them by these intervals.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#generate values for testing purposes, one value for each cell, add to object&#xA;value &amp;lt;- sample(seq(from=8, to=46, by=1), size = length(rownames(exp@meta.data)), replace=TRUE)&#xA;exp &amp;lt;- AddMetaData(object=exp, metadata=value, col.name=&quot;value&quot;)&#xA;&#xA;#encode the continuous values as factors, determined by the interval they fall in to&#xA;value_breaks &amp;lt;- cut(exp@meta.data$value, breaks = c(8,18,28,38,46), include.lowest=TRUE, right=FALSE)&#xA;&#xA;#name the breaks by cell so they can be added by AddMetaData(), add them&#xA;names(value_breaks) &amp;lt;- rownames(exp@meta.data)&#xA;exp &amp;lt;- AddMetaData(object=exp, metadata=value_breaks, col.name=&quot;value_breaks&quot;)&#xA;&#xA;exp &amp;lt;-  SetIdent(exp, ident.use=exp@meta.data$value_breaks)&#xA;&#xA;#create a gradient between specified intervals&#xA;#length.out is the number of different colors, number of factor levels&#xA;cols &amp;lt;- scales::seq_gradient_pal(low=&quot;blue&quot;, high=&quot;yellow&quot;, space=&quot;Lab&quot;)(seq(from=0, to=1,length.out=4))&#xA;&#xA;TSNEPlot(exp, colors.use=cols)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lyXKc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lyXKc.png&quot; alt=&quot;intervals colored&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The label in the legend are classic mathematical notation for intervals. You can add your own labels if you wish in the &lt;code&gt;cut()&lt;/code&gt; function.&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2018-11-01T15:50:00.467" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5345" PostTypeId="1" CreationDate="2018-11-01T16:09:17.313" Score="7" ViewCount="79" Body="&lt;p&gt;I am studying the ABA network in A. thaliana, consisting of HB7, ABI1 and AREB2. The AGI code I was given are, respectively: AT2G46680, AT4G26080 and AT1G45249.&lt;/p&gt;&#xA;&lt;p&gt;I downloaded the following &lt;a href=&quot;https://www.arabidopsis.org/download_files/Microarrays/Affymetrix/affy_ATH1_array_elements-2010-12-20.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;file&lt;/a&gt; in order to convert the array element name to the AGI one, but I am not able to find AT1G45249 in the above file. How can I check whether &lt;a href=&quot;https://www.arabidopsis.org/servlets/TairObject?type=locus&amp;amp;name=AT1G45249&quot; rel=&quot;nofollow noreferrer&quot;&gt;AT1G45249&lt;/a&gt; has another name? Or even better, how can I check what is the array element name in Affymetrix (GPL198) corresponding to AT1G45249. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="2862" LastEditorUserId="73" LastEditDate="2021-06-25T11:03:46.927" LastActivityDate="2023-09-14T00:00:00.083" Title="Gene not found in Affymetrix expression profiles" Tags="|microarray|gene-expression|identifiers|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5346" PostTypeId="2" ParentId="4341" CreationDate="2018-11-01T17:32:37.097" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/AUCell/inst/doc/AUCell.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;AUCell R package&lt;/a&gt; identifies enriched gene sets within cells. It uses the Area Under the Curve (AUC) to calculate whether a gene set is enriched within the expressed genes for each cell.&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2018-11-01T17:32:37.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5347" PostTypeId="1" CreationDate="2018-11-01T19:29:32.160" Score="6" ViewCount="231" Body="&lt;p&gt;What is the minimum number of individuals required per group (population/species) for an &lt;a href=&quot;https://genome.cshlp.org/content/19/9/1655.full&quot; rel=&quot;noreferrer&quot;&gt;ADMIXTURE&lt;/a&gt; analysis to be appropriate?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to determine this number mathematically based on the amount of genetic data I have or other criteria?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any citations I can use to figure this information out?&lt;/p&gt;&#xA;" OwnerUserId="2338" LastActivityDate="2019-04-16T19:03:25.707" Title="What is the minimum number of individuals per group required for ADMIXTURE analysis?" Tags="|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5348" PostTypeId="2" ParentId="5306" CreationDate="2018-11-01T19:58:53.137" Score="5" Body="&lt;p&gt;In one line, using &lt;code&gt;bedtools&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zcat  Homo_sapiens.GRCh38.93.gtf.gz \&#xA; | awk '$3==&quot;gene&quot;' \&#xA; | bedtools slop -b 10000 -g contigs.tsv -i - \&#xA; | bedtools intersect -u -a intervals.bed -b - &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This first takes the genes and filters them for the third column being gene. This is important because all &lt;code&gt;GTF&lt;/code&gt; lines contain the word &lt;code&gt;gene&lt;/code&gt; as &lt;code&gt;gene_id&lt;/code&gt; is a manditory attribute for &lt;code&gt;GTF&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next it passes it to slop, which adds 1 kb either side (ensuring not to go below 0 or off the end of the chr: this isn't that important because we are never outputting these regions anyway, but &lt;code&gt;bedtools&lt;/code&gt; requires it). We tell &lt;code&gt;bedtools&lt;/code&gt; to work on the standard in by using &lt;code&gt;-i -&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally we intersect these gene regions with the &lt;code&gt;intervals.bed&lt;/code&gt;. &lt;code&gt;-u&lt;/code&gt; says output the line from &lt;code&gt;-a&lt;/code&gt; if it overlaps with at least one entry in &lt;code&gt;-b&lt;/code&gt; and passing &lt;code&gt;-b -&lt;/code&gt; means read file b from the standard in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will only work if your &lt;code&gt;GTF&lt;/code&gt; contains &lt;code&gt;gene&lt;/code&gt; lines, which the GENCODE/Ensembl ones do. However, it is interesting to note that these GTFs are not actaully valid GTF files, as to be valid a GTF file must have both a &lt;code&gt;gene_id&lt;/code&gt; and &lt;code&gt;transcript_id&lt;/code&gt; attribute, and the gene lines in the GENCODE GTFs to not have this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your GTF does not contain gene lines, but can create them by merging all the exons in all the transcripts and filling in the gaps using &lt;code&gt;gtf2gtf&lt;/code&gt; from &lt;code&gt;cgat&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; cgat gtf2gtf --method=merge-transcripts --stdin Homo_sapiens.GRCh38.93.gtf.gz \&#xA; | bedtools slop -b 10000 -g contigs.tsv -i - \&#xA; | bedtools intersect -u -a intervals.bed -b -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="235" LastEditorUserId="235" LastEditDate="2018-11-04T12:51:52.710" LastActivityDate="2018-11-04T12:51:52.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5351" PostTypeId="1" CreationDate="2018-11-01T22:46:43.933" Score="3" ViewCount="71" Body="&lt;p&gt;Consider the following dataset:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fictional.df &amp;lt;- data.frame(L1 = c(0,0,0,0,0,0,0,0), &#xA;                       L2 = c(0,1,0,0,0,1,1,0),&#xA;                       L3 = c(1,1,0,1,1,1,1,1), &#xA;                       L4=c(0,0,1,1,0,0,0,0))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;converted this to a &lt;code&gt;phyDat&lt;/code&gt; object and then&lt;/li&gt;&#xA;&lt;li&gt;created a pairwise distance matrix as follows:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;    fictional.phydat &amp;lt;- as.phyDat(fictional.df,&#xA;                              type=&amp;quot;USER&amp;quot;,levels=c(&amp;quot;1&amp;quot;,&amp;quot;0&amp;quot;),&#xA;                              names=names(fictional.df))&#xA;    fictional.hamming &amp;lt;- dist.hamming(fictional.phydat)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;From this distance matrix, I then estimated a UPGMA tree:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;    fictional.upgma &amp;lt;- upgma(fictional.hamming)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;I then created bootstrap datasets:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;    set.seed(187)&#xA;    fictional.upgma.bs &amp;lt;- bootstrap.phyDat(fictional.phydat, FUN =  &#xA;    function(xx) upgma(dist.hamming(xx)), bs=100)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;5&quot;&gt;&#xA;&lt;li&gt;I then calculated the proportion of partitions in the bootstrap set:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;    upgma.bs.part &amp;lt;- prop.part(fictional.upgma.bs)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;6&quot;&gt;&#xA;&lt;li&gt;So far so good. Here is where I would appreciate some help. When I call the function &lt;code&gt;prop.clades&lt;/code&gt;, I do not understand the result:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;    prop.clades(fictional.upgma,fictional.upgma.bs)&#xA;    [1] 100  NA  71&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;Question&lt;/em&gt; Why does this function return &lt;code&gt;NA&lt;/code&gt; when there is evidence for that clade in the set of bootstrap trees?&lt;/p&gt;&#xA;&lt;p&gt;A second question:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;prop.clades(fictional.upgma,part=upgma.bs.part)&#xA;[1] 100  49 112&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If there are only 100 bootstrap samples, why is the value for the final clade &lt;code&gt;112&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="2961" LastEditorUserId="73" LastEditDate="2021-07-04T04:15:19.130" LastActivityDate="2021-07-04T04:15:19.130" Title="How are the values of prop.part() and prop.clades() calculated?" Tags="|statistics|clustering|phylogenetics|phylogeny|" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5352" PostTypeId="1" CreationDate="2018-09-21T00:39:50.680" Score="4" ViewCount="1581" Body="&lt;p&gt;I am currently working on my own Metagenomic pipeline, utilizing Bowtie 2 to map. Bowtie 2 outputs a SAM file, which I convert to a .BAM and sort it using Samtools. When I try to utilize Samtools to index my .BAM file it gives me this error:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-html prettyprint-override&quot;&gt;&lt;code&gt;[E::hts_idx_push] NO_COOR reads not in a single block at the end 1652 -1.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-html prettyprint-override&quot;&gt;&lt;code&gt;source /opt/asn/etc/asn-bash-profiles-special/modules.sh&#xA;module load samtools/1.3.1&#xA;&#xA;cd ~/gz_files/soapdenovo&#xA;r=20&#xA;###SAM to BAM and sort &#xA;&#xA;samtools faidx soapdenovo.fa &#xA;samtools view -b -T soapdenovo.fa -o sample&lt;span class=&quot;math-container&quot;&gt;${r}bowtiemapping.bam sample$&lt;/span&gt;{r}bowtiemapping.sam&#xA;samtools sort -n -T ~/gz_files/soapdenovo/temp -o sample&lt;span class=&quot;math-container&quot;&gt;${r}bowtiemapping_sorte$&lt;/span&gt;&#xA;samtools index sample${r}bowtiemapping_sorted.bam&#xA;&#xA;################LETS MAKE THE ABUNDANCE TABLES###########################&lt;span class=&quot;math-container&quot;&gt;$&#xA;samtools view -c -f 4  sample$&lt;/span&gt;{r}bowtiemapping_sorted.bam &amp;gt; sample&lt;span class=&quot;math-container&quot;&gt;${r}alignment$&lt;/span&gt;&#xA;&#xA;samtools idxstats sample&lt;span class=&quot;math-container&quot;&gt;${r}bowtiemapping_sorted.bam &amp;gt;  sample$&lt;/span&gt;{r}idxstats.txt&#xA;&#xA;#git clone https://github.com/metajinomics/mapping_tools.git&#xA;#python mapping_tools/get_count_table.py *.idxstats.txt &amp;gt; contig_counts.tsv&#xA;##less contig_counts.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any help would be appreciated!&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;em&gt;Notes on pipeline&lt;/em&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;samtools import&lt;/code&gt; was replaced with &lt;code&gt;samtools view&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;faidx&lt;/code&gt;corrected&lt;/li&gt;&#xA;&lt;li&gt;Instructions used are &lt;a href=&quot;https://github.com/edamame-course/2018-Tutorials/wiki&quot; rel=&quot;nofollow noreferrer&quot;&gt;here on github&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Suggestion for future trouble shooting is given &lt;a href=&quot;https://stackoverflow.com/help/mcve&quot;&gt;here&lt;/a&gt;  by @winni2k&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerDisplayName="Haley" LastEditorUserId="3848" LastEditDate="2020-07-11T09:04:50.283" LastActivityDate="2020-07-14T12:28:23.420" Title="Error given while trying to index a BAM file with Samtools Index - NO COOR?" Tags="|samtools|bash|bugs|" AnswerCount="1" CommentCount="14" ContentLicense="CC BY-SA 4.0" />
  <row Id="5353" PostTypeId="1" CreationDate="2018-11-02T15:37:45.317" Score="3" ViewCount="720" Body="&lt;p&gt;I have a bam file with aligned reads from multiple experiments. For each experiment I have a portion of the sequence identifier, thus for example&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;HWI-D00310:448:CCG0KANXX:7:1101:2570:1976&lt;/code&gt;,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;and is different from other the other of single-run flow cells in the analysis.&lt;/p&gt;&#xA;&lt;p&gt;I want to demultiplex the input bam file into different bam files depending on the experiment. My only concern is a fast methodology to demultiplex a bam using combinations of single-run, flow-cell, and lane. What I came up with (the fastest) is something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Parallelized per chromosome. Assuming only 2 plates here.&#xA;printf '%s\n' $(seq 1 22) X Y MT \&#xA;| parallel 'samtools view input.bam {} \&#xA;  | tee &amp;gt;(grep -f patterns_experiment1.txt &amp;gt; experiment1_chr{}.sam.noheader) \&#xA;        &amp;gt;(grep -f patterns_experiment2.txt &amp;gt; experiment2_chr{}.sam.noheader)' \&#xA;&amp;amp;&amp;amp; seq 1 2 \&#xA;| parallel 'samtools view -H input.bam &amp;amp;&amp;amp; cat experiment{}_*sam.noheader \&#xA;  | samtools view -1 - &amp;gt; input_experiment{}.bam'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there a better/faster way to do it?&lt;/p&gt;&#xA;&lt;p&gt;If I understood correctly &lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/6472/read-groups&quot; rel=&quot;nofollow noreferrer&quot;&gt;read groups&lt;/a&gt; should be specified prior to mapping? I started from the already mapped BAM unfortunately, but looking at the original FASTQs I noticed the pattern in which some parts of the sequence identifier were unique for one experiment vs the others.&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="73" LastEditDate="2021-07-04T04:11:40.890" LastActivityDate="2021-07-04T04:11:40.890" Title="Fastest way to demultiplex bam file based on field" Tags="|sam|ngs|demultiplexing|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5354" PostTypeId="1" AcceptedAnswerId="5377" CreationDate="2018-11-02T18:03:58.177" Score="4" ViewCount="130" Body="&lt;p&gt;I am new to bioinformatics. So if these questions seem you to a bit childish please forgive me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two queries.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I am intending to perform a codon usage analysis followed by correspondence analyses for multiple microbial whole genomes of one bacterial species to find the association with the isolation source and ST type. Is this the right approach?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I have concatenated all the CDS in a single genome and joined all the genomes (average genes per genome is 4100). The problem is all the CDS when concatenated also have the stop codons in it. CodonW, famous for this analyses, cannot begin the analyses in the presence of stop codons. How can I remove stop codons?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Kindly suggest me a solution to the problem and concept.  &lt;/p&gt;&#xA;" OwnerUserId="3626" LastActivityDate="2018-11-05T20:08:56.090" Title="Codon usage analysis for whole genomes" Tags="|sequence-analysis|software-usage|codon|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5355" PostTypeId="1" AcceptedAnswerId="5363" CreationDate="2018-11-02T23:03:55.503" Score="2" ViewCount="1162" Body="&lt;p&gt;When I try to import the data using &lt;code&gt;DESeq2&lt;/code&gt; package:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromTximport(txi, sampleTable, ~Group + Cre)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am getting the error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error in validObject(.Object) : &#xA;    invalid class “DESeqDataSet” object: levels of factors in the design have non-unique level names after make.names() is applied.&#xA;    best to only uobject letters and numbers for levels of factors in the design&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Here is how my &lt;code&gt;sampleTable&lt;/code&gt; looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    ID      Group Cre&#xA;1  Bla_36   IL4   -&#xA;2  Bla_38   IL4   -&#xA;3  Bla_11   IL4   -&#xA;4  Bla_17   IL4   -&#xA;5  Bla_20   IL4   +&#xA;6  Bla_21   IL4   +&#xA;7   Bla_8   IL4   +&#xA;8  Bla_22   IL4   +&#xA;9   Bla_7   IL4   +&#xA;10 Bla_35   PBS   -&#xA;11 Bla_37   PBS   -&#xA;12 Bla_13   PBS   -&#xA;13  Bla_1   PBS   -&#xA;14  Bla_9   PBS   +&#xA;15 Bla_23   PBS   +&#xA;16 Bla_19   PBS   +&#xA;17 Bla_24   PBS   +&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What am I doing wrong here? And how to make it working?&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-11-04T22:17:17.150" Title="levels of factors in the design have non-unique level names after make.names() is applied" Tags="|r|deseq2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5356" PostTypeId="2" ParentId="5355" CreationDate="2018-11-02T23:28:21.367" Score="2" Body="&lt;p&gt;Weirdly enough, &lt;code&gt;+&lt;/code&gt; and &lt;code&gt;-&lt;/code&gt; are apparently not allowed in the column. After mapping them to longer strings it started to work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(plyr)&#xA;cre_id = revalue(samples$cre, c('-' = 'minus', '+' = 'plus'))&#xA;&#xA;sampleTable &amp;lt;- data.frame(&#xA;  ID = samples&lt;span class=&quot;math-container&quot;&gt;$id,&#xA;  Group = samples$&lt;/span&gt;group,&#xA;  Cre = cre_id&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and it looks like that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;       ID   Group   Cre&#xA;1  Bla_36   IL4 minus&#xA;2  Bla_38   IL4 minus&#xA;3  Bla_11   IL4 minus&#xA;4  Bla_17   IL4 minus&#xA;5  Bla_20   IL4  plus&#xA;6  Bla_21   IL4  plus&#xA;7   Bla_8   IL4  plus&#xA;8  Bla_22   IL4  plus&#xA;9   Bla_7   IL4  plus&#xA;10 Bla_35   PBS minus&#xA;11 Bla_37   PBS minus&#xA;12 Bla_13   PBS minus&#xA;13  Bla_1   PBS minus&#xA;14  Bla_9   PBS  plus&#xA;15 Bla_23   PBS  plus&#xA;16 Bla_19   PBS  plus&#xA;17 Bla_24   PBS  plus&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I do not understand though, why...&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2018-11-02T23:28:21.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5357" PostTypeId="1" CreationDate="2018-11-02T23:31:32.600" Score="1" ViewCount="421" Body="&lt;p&gt;I annotated my bacterial genomes using the new &lt;a href=&quot;https://github.com/ncbi/pgap&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Prokaryotic Genome Annotation Pipeline&lt;/a&gt; and now, I want to annotate EC-numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the master annotation file (.gff), I get the RefSeq-ID of every successful annotation, e.g. &lt;code&gt;RefSeq:YP_805528.1&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I visit the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene/?term=YP_805528.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;respective NCBI page&lt;/a&gt;, I see that it sometimes is mapped to an EC number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to automate this process: Convert every RefSeq-ID to an EC number. How can I do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I may already have discovered the first step: I can map the RefSeq-ID to the NCBI-ID (Entrez-ID) using this &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/gene/DATA/gene2refseq.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI conversion database&lt;/a&gt;: &lt;code&gt;YP_805528.1 -&amp;gt; 4419225&lt;/code&gt;. But I don't see how to get from &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/gene/DATA/&quot; rel=&quot;nofollow noreferrer&quot;&gt;these&lt;/a&gt; &lt;a href=&quot;http://barc.wi.mit.edu/entrez_gene/entrez_gene.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;databases&lt;/a&gt; to EC.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there other ways? As far as I know, most people blast their genes to already-mapped databases &lt;a href=&quot;http://breathmintsforpenguins.blogspot.com/2014/07/using-swissprot-and-expasy-enzyme-to.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;like SwissProt and ExPASy ENZYME&lt;/a&gt;. But I don't like this approach because technically, the NCBI annotation pipeline already did this. Why compute the same thing twice?&lt;/p&gt;&#xA;" OwnerUserId="3628" LastEditorUserId="48" LastEditDate="2018-11-05T10:11:06.903" LastActivityDate="2018-12-05T11:01:00.153" Title="Mapping RefSeq-ID or NCBI-ID (Entrez-ID) to EC number" Tags="|sequence-annotation|identifiers|phylogenetics|refseq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5358" PostTypeId="2" ParentId="5357" CreationDate="2018-11-02T23:40:42.797" Score="2" Body="&lt;p&gt;You can use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez Direct&lt;/a&gt; for this as follows: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;efetch -db protein -id YP_805528.1 -format gpc -mode xml | xtract -insd Protein EC_number&#xA;YP_805528.1     6.3.2.13&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3577" LastEditorUserId="3577" LastEditDate="2018-11-03T00:04:44.177" LastActivityDate="2018-11-03T00:04:44.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5359" PostTypeId="1" AcceptedAnswerId="5471" CreationDate="2018-11-03T04:15:40.030" Score="13" ViewCount="846" Body="&lt;p&gt;&lt;strong&gt;&lt;em&gt;edit: Results are current as of Dec 4, 2018 13:00 PST.&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Background&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/K-mer&quot; rel=&quot;noreferrer&quot;&gt;K-mers&lt;/a&gt; have many uses in bioinformatics, and for this reason it would be useful to know the most RAM-efficient and fastest way to work with them programmatically. There have been questions covering &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/3462/how-do-kmer-counters-determine-which-kmer-is-canonical&quot;&gt;what canonical k-mers are&lt;/a&gt;, &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/3619/how-to-calculate-the-memory-usage-of-storing-kmers-in-ram&quot;&gt;how much RAM k-mer storage theoretically takes&lt;/a&gt;, but we have not yet looked at the best data structure to store and access k-mers and associated values with. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Question&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;What data structure in &lt;code&gt;C++&lt;/code&gt; simultaneously allows the most compact k-mer storage, a property, and the fastest lookup time? For this question I choose &lt;code&gt;C++&lt;/code&gt; for speed, ease-of-implementation, and access to lower-level language features if desired. Answers in other languages are acceptable, too.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Setup&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For benchmarking:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I propose to use a standard &lt;code&gt;fasta&lt;/code&gt; file for everyone to use. This program, &lt;code&gt;generate-fasta.cpp&lt;/code&gt;, generates two million sequences ranging in length between 29 and 300, with a peak of sequences around length 60.&lt;/li&gt;&#xA;&lt;li&gt;Let's use &lt;code&gt;k=29&lt;/code&gt; for the analysis, but ignore implementations that require knowledge of the k-mer size before implementation. Doing so will make the resulting data structure more amenable to downstream users who may need other sizes &lt;code&gt;k&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Let's just store the most recent read that the k-mer appeared in as the property to retrieve during k-mer lookup. In most applications it is important to attach some value to each k-mer such as a taxon, its count in a dataset, et cetera.&lt;/li&gt;&#xA;&lt;li&gt;If possible, use the string parser in the code below for consistency between answers.&lt;/li&gt;&#xA;&lt;li&gt;The algorithm should use canonical k-mers. That is, a k-mer and its reverse complement are considered to be the same k-mer.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here is &lt;code&gt;generate-fasta.cpp&lt;/code&gt;. I used the command &lt;code&gt;g++ generate_fasta.cpp -o generate_fasta&lt;/code&gt; to compile and the command &lt;code&gt;./generate_fasta &amp;gt; my.fasta&lt;/code&gt; to run it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;return 0;&#xA;//generate a fasta file to count k-mers&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;random&amp;gt;&#xA;&#xA;char gen_base(int q){&#xA;  if (q &amp;lt;= 30){&#xA;    return 'A';&#xA;  } else if ((q &amp;gt; 30) &amp;amp;&amp;amp; (q &amp;lt;=60) ){&#xA;    return 'T';&#xA;  } else if ((q &amp;gt; 60) &amp;amp;&amp;amp; (q &amp;lt;=80) ){&#xA;    return 'C';&#xA;  } else if (q &amp;gt; 80){&#xA;    return 'G';&#xA;  }&#xA;  return 'N';&#xA;}&#xA;&#xA;int main() {&#xA;  unsigned seed = 1;&#xA;  std::default_random_engine generator (seed);&#xA;  std::poisson_distribution&amp;lt;int&amp;gt; poisson (59);&#xA;  std::geometric_distribution&amp;lt;int&amp;gt; geo (0.05);&#xA;  std::uniform_int_distribution&amp;lt;int&amp;gt; uniform (1,100);&#xA;  int printval;&#xA;  int i=0;&#xA;  while(i&amp;lt;2000000){&#xA;    if (i % 2 == 0){&#xA;      printval = poisson(generator);&#xA;    } else {&#xA;      printval = geo(generator) + 29;&#xA;    }&#xA;    if (printval &amp;gt;= 29){&#xA;      std::cout &amp;lt;&amp;lt; '&amp;gt;' &amp;lt;&amp;lt; i &amp;lt;&amp;lt; '\n';&#xA;      //std::cout &amp;lt;&amp;lt; printval &amp;lt;&amp;lt; '\n';&#xA;      for (int j = 0; j &amp;lt; printval; j++){&#xA;        std::cout &amp;lt;&amp;lt; gen_base(uniform(generator));&#xA;      }&#xA;      std::cout &amp;lt;&amp;lt; '\n';&#xA;      i++;&#xA;    }&#xA;  }&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Example&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;One naive implementation is to add both the observed k-mer and its reverse complement as separate k-mers. This is obviously not space efficient but should have fast lookup. This file is called &lt;code&gt;make_struct_lookup.cpp&lt;/code&gt;. I used the following command to compile on my Apple laptop (OS X): &lt;code&gt;clang++ -std=c++11 -stdlib=libc++ -Wno-c++98-compat make_struct_lookup.cpp -o msl&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;fstream&amp;gt;&#xA;#include &amp;lt;string&amp;gt;&#xA;#include &amp;lt;map&amp;gt;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;chrono&amp;gt;&#xA;//build the structure. measure how much RAM it consumes.&#xA;//then measure how long it takes to lookup in the data structure&#xA;&#xA;#define k 29&#xA;&#xA;std::string rc(std::string seq){&#xA;  std::string rc;&#xA;  for (int i = seq.length()-1; i&amp;gt;=0; i--){&#xA;    if (seq[i] == 'A'){&#xA;      rc.push_back('T');&#xA;    } else if (seq[i] == 'C'){&#xA;      rc.push_back('G');&#xA;    } else if (seq[i] == 'G'){&#xA;      rc.push_back('C');&#xA;    } else if (seq[i] == 'T'){&#xA;      rc.push_back('A');&#xA;    }&#xA;  }&#xA;  return rc;&#xA;}&#xA;&#xA;int main(int argc, char* argv[]){&#xA;  using namespace std::chrono;&#xA;  //initialize the data structure&#xA;  std::string thisline;&#xA;  std::map&amp;lt;std::string, int&amp;gt; kmer_map;&#xA;  std::string header;&#xA;  std::string seq;&#xA;  //open the fasta file&#xA;  std::ifstream inFile;&#xA;  inFile.open(argv[1]);&#xA;&#xA;  //construct the kmer-lookup structure&#xA;  int i = 0;&#xA;  high_resolution_clock::time_point t1 = high_resolution_clock::now();&#xA;  while (getline(inFile,thisline)){&#xA;    if (thisline[0] == '&amp;gt;'){&#xA;      header = thisline.substr(1,thisline.size());&#xA;      //std::cout &amp;lt;&amp;lt; header &amp;lt;&amp;lt; '\n';&#xA;    } else {&#xA;      seq = thisline;&#xA;      //now add the kmers&#xA;      for (int j=0; j&amp;lt; thisline.size() - k + 1; j++){&#xA;        kmer_map[seq.substr(j,j+k)] = stoi(header);&#xA;        kmer_map[rc(seq.substr(j,j+k))] = stoi(header);&#xA;      }&#xA;      i++;&#xA;    }&#xA;  }&#xA;  std::cout &amp;lt;&amp;lt; &quot;  -finished &quot; &amp;lt;&amp;lt; i &amp;lt;&amp;lt; &quot; seqs.\n&quot;;&#xA;  inFile.close();&#xA;  high_resolution_clock::time_point t2 = high_resolution_clock::now();&#xA;  duration&amp;lt;double&amp;gt; time_span = duration_cast&amp;lt;duration&amp;lt;double&amp;gt;&amp;gt;(t2 - t1);&#xA;  std::cout &amp;lt;&amp;lt; time_span.count() &amp;lt;&amp;lt; &quot; seconds to load the array.&quot; &amp;lt;&amp;lt; '\n';&#xA;&#xA;  //now lookup the kmers&#xA;  inFile.open(argv[1]);&#xA;  t1 = high_resolution_clock::now();&#xA;  int lookup;&#xA;  while (getline(inFile,thisline)){&#xA;    if (thisline[0] != '&amp;gt;'){&#xA;      seq = thisline;&#xA;      //now lookup the kmers&#xA;      for (int j=0; j&amp;lt; thisline.size() - k + 1; j++){&#xA;        lookup = kmer_map[seq.substr(j,j+k)];&#xA;      }&#xA;    }&#xA;  }&#xA;  std::cout &amp;lt;&amp;lt; &quot;  - looked at &quot; &amp;lt;&amp;lt; i &amp;lt;&amp;lt; &quot; seqs.\n&quot;;&#xA;  inFile.close();&#xA;  t2 = high_resolution_clock::now();&#xA;  time_span = duration_cast&amp;lt;duration&amp;lt;double&amp;gt;&amp;gt;(t2 - t1);&#xA;  std::cout &amp;lt;&amp;lt; time_span.count() &amp;lt;&amp;lt; &quot; seconds to lookup the kmers.&quot; &amp;lt;&amp;lt; '\n';&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Example output&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I ran the above program with the following command to log peak RAM usage. The amount of time the lookup of all k-mers in two million sequences is reported by the program. &lt;code&gt;/usr/bin/time -l ./msl my.fasta&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output was:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; -finished 2000000 seqs.&#xA;562.864 seconds to load the array.&#xA;  - looked at 2000000 seqs.&#xA;368.734 seconds to lookup the k-mers.&#xA;     1046.94 real       942.38 user        78.96 sys&#xA;11680514048  maximum resident set size&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, the program used &lt;code&gt;11680514048 bytes&lt;/code&gt; = &lt;code&gt;11.68GB&lt;/code&gt; of RAM and it took 368.734 seconds to lookup the k-mers in two million fasta files.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Results&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Below is a plot of the results from each user's answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Z35Ir.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Z35Ir.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-12-04T21:01:35.067" LastActivityDate="2018-12-05T05:45:52.313" Title="What is the most compact data structure for canonical k-mers with the fastest lookup time?" Tags="|k-mer|c++|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="5361" PostTypeId="2" ParentId="5334" CreationDate="2018-11-03T16:29:13.040" Score="4" Body="&lt;p&gt;OK, I ended up contacting one of the authors of the program, who kindly answered to my question:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Either approach should work,&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;[i.e. running the analysis by chromosome vs joining the data together]&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;but I suggest dividing by chromosome because that approach has had&#xA;  more testing.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I hope this might help other users, I saw that several questions regarding Beagle went unanswered on biostars and seqanswers. I guess it is a pretty specialized topic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Max&lt;/p&gt;&#xA;" OwnerUserId="3536" LastActivityDate="2018-11-03T16:29:13.040" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5363" PostTypeId="2" ParentId="5355" CreationDate="2018-11-04T22:17:17.150" Score="3" Body="&lt;p&gt;There's a good reason to not allow &lt;code&gt;+&lt;/code&gt; and &lt;code&gt;-&lt;/code&gt; in the columns. It's because once you do that you get coefficients that can then start with a &lt;code&gt;+&lt;/code&gt; or &lt;code&gt;-&lt;/code&gt;, which is illegal in R and most other languages. In R, legal variable names consist of:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A syntactically valid name consists of letters, numbers and the&#xA;       dot or underline characters and starts with a letter or the dot&#xA;       not followed by a number.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That doesn't really explain the error message you saw, of course. That can be solved by thinking about what R does to invalid character in names. Namely, it replaces them by a dot. So &lt;code&gt;+&lt;/code&gt; and &lt;code&gt;-&lt;/code&gt; get converted to &lt;code&gt;X.&lt;/code&gt; and &lt;code&gt;X.&lt;/code&gt;, which are of course identical and are what's really leading to the error message your saw. If you had used &lt;code&gt;+cre&lt;/code&gt; and &lt;code&gt;-&lt;/code&gt; it would have worked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an aside, you'll want to think in terms of positive and negative Cre if you talk to a biologist, since that's the appropriate terminology for it.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-11-04T22:17:17.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5364" PostTypeId="1" AcceptedAnswerId="5369" CreationDate="2018-11-05T01:37:03.007" Score="0" ViewCount="179" Body="&lt;p&gt;I have been trying to download SPAdes for an important project I wanted to do, however, I have encountered a small problem while doing so. It appears that there is no 'bin' folder in my SPAdes installation file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used 7-Zip as my main method of unpacking the file (As I am using Windows, I had to download the actual source code rather than a nice executable), and it came in the &lt;code&gt;.tar.gz&lt;/code&gt; format, which I believe 7-Zip is rather good with.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few pictures detailing my problems are shown below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope I can get a little help on this matter. It is slightly time-sensitive, so I would really appreciate a little bit of assistance shortly. Thank you in advance!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/v4O2c.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/v4O2c.png&quot; alt=&quot;This is the command I used, the usual test command&quot;&gt;&lt;/a&gt;&lt;a href=&quot;https://i.stack.imgur.com/5Mos7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5Mos7.png&quot; alt=&quot;This is the file structure I got. Notice the absence of the &amp;#39;bin&amp;#39; file.&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SRKur.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SRKur.png&quot; alt=&quot;This is the error I received from running the &amp;lt;code&amp;gt;spades.py --test&amp;lt;/code&amp;gt; command&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As seen above, the first image is the command I used, the usual test command, the second is the file structure I got (Notice the absence of the 'bin' file), and the third is the error I received from running the &lt;code&gt;spades.py --test&lt;/code&gt; command.&lt;/p&gt;&#xA;" OwnerUserId="3630" LastActivityDate="2018-11-05T13:44:34.037" Title="Problem with SPAdes installation - no bin folder" Tags="|assembly|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5365" PostTypeId="5" CreationDate="2018-11-05T10:12:15.860" Score="0" Body="&lt;p&gt;Identifiers are the name given to genes, transcripts, proteins, peptides, protein complex, pathways... there are several naming conventions and usually not 1 to 1 conversion between different databases or identifiers.&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2018-11-06T19:56:39.447" LastActivityDate="2018-11-06T19:56:39.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5366" PostTypeId="4" CreationDate="2018-11-05T10:12:15.860" Score="0" Body="Use this tag if your question is about translating some identifiers to others or about the rules of certain identifiers." OwnerUserId="48" LastEditorUserId="48" LastEditDate="2018-11-06T19:57:24.813" LastActivityDate="2018-11-06T19:57:24.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5367" PostTypeId="1" AcceptedAnswerId="5411" CreationDate="2018-11-05T10:13:29.263" Score="3" ViewCount="931" Body="&lt;p&gt;I would like to use public omics datasets (ChIP-seq, RNA-seq, and ATAC-seq) from different studies to do an integrative analysis as follow:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Normalise samples, within each type of omics, from different public datasets.&lt;/li&gt;&#xA;&lt;li&gt;Convert the normalised values into a uniform scale to make the comparison between ChIP-seq, RNA-seq and ATAC-seq possible.&lt;/li&gt;&#xA;&lt;li&gt;Feed the normalised uniformed values into machine learning to infer one feature (e.g. RNA expression) from other features (e.g. TF or histone marks ChIP-seq).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;How could I integrate these data-sets to predict one group of variables from another one?&lt;/p&gt;&#xA;" OwnerUserId="536" LastEditorUserId="48" LastEditDate="2018-11-07T13:37:00.570" LastActivityDate="2018-11-08T20:02:07.693" Title="Integrative analysis of omics studies using machine learning" Tags="|rna-seq|chip-seq|atac-seq|machine-learning|multi-omics|" AnswerCount="1" CommentCount="7" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5368" PostTypeId="1" CreationDate="2018-11-05T11:11:23.463" Score="4" ViewCount="5830" Body="&lt;p&gt;I have Bed file containing start and end of a sequence, and I need to convert it to fasta format, any recommendations?&lt;/p&gt;&#xA;" OwnerUserId="3632" LastActivityDate="2018-11-08T18:46:44.320" Title="How to convert Bed file to fasta file?" Tags="|file-formats|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5369" PostTypeId="2" ParentId="5364" CreationDate="2018-11-05T13:44:34.037" Score="1" Body="&lt;p&gt;Simply unpacking the source code won't actually install much, you'll need to follow its installation instructions. Note that this might be possible on Windows if you use new bash command line, but in general don't expect any bioinformatics tools to work on windows.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-11-05T13:44:34.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5370" PostTypeId="1" AcceptedAnswerId="5371" CreationDate="2018-11-05T15:00:00.743" Score="1" ViewCount="236" Body="&lt;p&gt;I'm looking for a list of human transcription factor (TF) genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where can I find a TF database or how can I retrieve this list from other databases?&lt;/p&gt;&#xA;" OwnerUserId="208" LastEditorUserId="48" LastEditDate="2018-11-05T16:44:06.423" LastActivityDate="2018-11-05T16:44:06.423" Title="List of human transcription factors" Tags="|database|data-retrieval|human|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5371" PostTypeId="2" ParentId="5370" CreationDate="2018-11-05T15:00:00.743" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0092867418301065?via%3Dihub&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lambert et al.&lt;/a&gt; (2018, Cell. 172(4):650-665) provide a manually curated list of human TFs at &lt;a href=&quot;http://humantfs.ccbr.utoronto.ca/download.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;this website&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2018-11-05T15:00:00.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5372" PostTypeId="2" ParentId="5368" CreationDate="2018-11-05T15:49:16.650" Score="5" Body="&lt;p&gt;&lt;code&gt;bedtools getfasta&lt;/code&gt; can do this. See &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/getfasta.html&quot; rel=&quot;noreferrer&quot;&gt;documentation&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2018-11-05T15:49:16.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5373" PostTypeId="1" AcceptedAnswerId="5376" CreationDate="2018-11-05T16:21:17.787" Score="1" ViewCount="61" Body="&lt;p&gt;I would like to search the literature using the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pubmed search engine&lt;/a&gt; for a set of genes that are associated with a brain region of interest. Unfortunately, there are a lot of them. It's possible there is no solution if there is a character limit on the query. I think it would be easiest to do this with 'nix tools but I'm open to R libraries or Python implementations too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The list is 908 genes long. It's from the &lt;a href=&quot;http://www.gensat.org/AnnotationSearchEngine.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gensat annotation search engine&lt;/a&gt;, with the filters &lt;code&gt;Structure IS olfactory bulb&lt;/code&gt; &amp;amp; &lt;code&gt;Expression level IS moderate to strong signal&lt;/code&gt; &amp;amp; &lt;code&gt;Expression pattern IS region-specific&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The list looks like, scraped from the dropdown list in the HTML source with a regular expression: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Aatf&#xA;Abcb1b&#xA;Abhd3&#xA;Abhd4&#xA;Abhd5&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The Pubmed query should look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(((((((Aatf) OR Abcb1b) OR Abhd3) OR Abhd4) OR Abhd5) OR Abl2) AND olfactory bulb &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but with 908 genes separated by OR and parentheses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bonus points if you find/write a utility which adds gene synonyms to the query!&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="181" LastEditDate="2018-11-05T16:31:10.097" LastActivityDate="2018-11-05T20:42:17.003" Title="Build a Pubmed query given long gene list" Tags="|text-processing|literature-search|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5374" PostTypeId="2" ParentId="5368" CreationDate="2018-11-05T16:21:45.753" Score="1" Body="&lt;p&gt;You could start with a folder of &lt;code&gt;samtools faidx&lt;/code&gt;-indexed FASTA files for your genome of interest:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ mkdir /genomes/hg38 &amp;amp;&amp;amp; cd /genomes/hg38&#xA;$&lt;/span&gt; wget ftp://hgdownload.cse.ucsc.edu/goldenPath/hg38/chromosomes/*.fa.gz&#xA;&lt;span class=&quot;math-container&quot;&gt;$ for fn in `ls *.fa.gz`; do gunzip $&lt;/span&gt;fn; done&#xA;&lt;span class=&quot;math-container&quot;&gt;$ for fn in `ls *.fa`; do samtools faidx $&lt;/span&gt;fn; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then use that folder with the following script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env perl&#xA;&#xA;use strict;&#xA;use warnings;&#xA;use Getopt::Long;&#xA;&#xA;#&#xA;# bed2faidxsta.pl&#xA;# --&#xA;# Reads BED data from standard input, writes FASTA to standard output.&#xA;#&#xA;# Dependent on samtools and indexed FASTA data files located in $fastaDir &#xA;# variable. Set --fastaDir=dir to set custom directory containing a source &#xA;# of per-build, bgzip-compressed FASTA and associated index (fa.gz.fai) &#xA;# files, or leave unset to use data in current working directory. Use the&#xA;# --fastaIsUncompressed option if the FASTA files are not compressed.&#xA;#&#xA;&#xA;# test if samtools is available&#xA;`samtools --version` || die &quot;Error: The samtools application is required to run this script. Try 'module add samtools' or install a local copy of samtools.\n&quot;;&#xA;&#xA;# default FASTA input is current working directory&#xA;my &lt;span class=&quot;math-container&quot;&gt;$fastaDir = `pwd`; chomp $&lt;/span&gt;fastaDir;&#xA;# default is to assume input coordinates use zero-based index scheme&#xA;my &lt;span class=&quot;math-container&quot;&gt;$oneBased;&#xA;# default is to leave IDs alone&#xA;my $&lt;/span&gt;useIDPrefixAsStrand;&#xA;# default is to assume FASTA files are bgzip-compressed&#xA;my $fastaIsUncompressed;&#xA;&#xA;GetOptions ('fastaDir=s' =&amp;gt; \$fastaDir, 'oneBased' =&amp;gt; \$oneBased, 'useIDPrefixAsStrand' =&amp;gt; \$useIDPrefixAsStrand, 'fastaIsUncompressed' =&amp;gt; \$fastaIsUncompressed);&#xA;&#xA;if (! -d $fastaDir) { die &quot;Error: FASTA directory does not exist\n&quot;; }&#xA;&#xA;while (&amp;lt;STDIN&amp;gt;) {&#xA;    chomp;&#xA;    my (&lt;span class=&quot;math-container&quot;&gt;$chr, $&lt;/span&gt;start, &lt;span class=&quot;math-container&quot;&gt;$stop, $&lt;/span&gt;id, &lt;span class=&quot;math-container&quot;&gt;$score, $&lt;/span&gt;strand) = split(&quot;\t&quot;, &lt;span class=&quot;math-container&quot;&gt;$_);&#xA;if (!defined($&lt;/span&gt;chr) || !defined(&lt;span class=&quot;math-container&quot;&gt;$start) || !defined($&lt;/span&gt;stop)) { die &quot;Error: No chromosome name, start or stop position defined\n&quot;; }&#xA;    if (!defined(&lt;span class=&quot;math-container&quot;&gt;$id)) { $id = &quot;.&quot;; }&#xA;if (!defined($&lt;/span&gt;score)) { &lt;span class=&quot;math-container&quot;&gt;$score = &quot;.&quot;; }&#xA;if (!defined($&lt;/span&gt;strand)) { &lt;span class=&quot;math-container&quot;&gt;$strand = &quot;+&quot;; } else { $strand = substr($strand, 0, 1); }&#xA;# adjust coordinates to one-based index, if necessary&#xA;my ($&lt;/span&gt;queryChr, &lt;span class=&quot;math-container&quot;&gt;$queryStart, $&lt;/span&gt;queryStop) = (&lt;span class=&quot;math-container&quot;&gt;$chr, $&lt;/span&gt;start, &lt;span class=&quot;math-container&quot;&gt;$stop);&#xA;if (!$&lt;/span&gt;oneBased) {&#xA;        &lt;span class=&quot;math-container&quot;&gt;$queryStart++;&#xA;}&#xA;# adjust strand if required&#xA;if ($&lt;/span&gt;useIDPrefixAsStrand) {&#xA;        &lt;span class=&quot;math-container&quot;&gt;$strand = substr($&lt;/span&gt;id, 0, 1);&#xA;    }&#xA;    # lookup&#xA;    my &lt;span class=&quot;math-container&quot;&gt;$queryFn = &quot;$&lt;/span&gt;fastaDir/&lt;span class=&quot;math-container&quot;&gt;$chr.fa.gz&quot;;&#xA;if ($&lt;/span&gt;fastaIsUncompressed) {&#xA;        &lt;span class=&quot;math-container&quot;&gt;$queryFn = &quot;$&lt;/span&gt;fastaDir/&lt;span class=&quot;math-container&quot;&gt;$chr.fa&quot;;&#xA;}&#xA;my $&lt;/span&gt;queryKey = &quot;&lt;span class=&quot;math-container&quot;&gt;$queryChr:$&lt;/span&gt;queryStart-&lt;span class=&quot;math-container&quot;&gt;$queryStop&quot;;&#xA;my $&lt;/span&gt;queryResult = `samtools faidx $queryFn $queryKey`; chomp &lt;span class=&quot;math-container&quot;&gt;$queryResult; &#xA;# linearize result&#xA;my @lines = split(&quot;\n&quot;, $&lt;/span&gt;queryResult);&#xA;    my @seqs = @lines[1..(scalar @lines - 1)];&#xA;    my &lt;span class=&quot;math-container&quot;&gt;$seq = join(&quot;&quot;, @seqs);&#xA;# handle reverse-stranded elements&#xA;if ($&lt;/span&gt;strand eq &quot;-&quot;) {&#xA;        &lt;span class=&quot;math-container&quot;&gt;$seq = rc_sequence($&lt;/span&gt;seq);&#xA;    }&#xA;    # print to standard output&#xA;    my &lt;span class=&quot;math-container&quot;&gt;$header = &quot;&amp;gt;&quot;.join(&quot;:&quot;,($&lt;/span&gt;chr, &lt;span class=&quot;math-container&quot;&gt;$start, $&lt;/span&gt;stop, &lt;span class=&quot;math-container&quot;&gt;$id, $&lt;/span&gt;score, &lt;span class=&quot;math-container&quot;&gt;$strand));&#xA;print STDOUT $&lt;/span&gt;header.&quot;\n&quot;.$seq.&quot;\n&quot;;&#xA;}&#xA;&#xA;sub rc_sequence {&#xA;    my &lt;span class=&quot;math-container&quot;&gt;$seq = shift @_;&#xA;my $&lt;/span&gt;reverse_complement = reverse(&lt;span class=&quot;math-container&quot;&gt;$seq);&#xA;$&lt;/span&gt;reverse_complement =~ tr/ACGTacgt/TGCAtgca/;&#xA;    return $reverse_complement;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ perl bed2faidxsta.pl &amp;lt; in.bed &amp;gt; out.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This approach can be useful with stranded input, nonstandard genomes, or just to do things with a bit more knowledge of inputs.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2018-11-05T16:21:45.753" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5375" PostTypeId="1" AcceptedAnswerId="5379" CreationDate="2018-11-05T18:05:25.543" Score="4" ViewCount="896" Body="&lt;p&gt;I have a barcode distribution from single-cell data, e.g:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;11612552 TCCTGAGCACTGCATAACTCAA&#xA;9349711 GCTACGCTACTGCATAAGTCCA&#xA;8343678 CAGAGAGGCTAAGCCTGCACAT&#xA;8161950 CGTACTAGTCTCTCCGCGGCTA&#xA;8102383 TCCTGAGCGTAAGGAGCAGATC&#xA;7110298 AAGAGGCAGTAAGGAGATAATC&#xA;6626630 TAAGGCGAACTGCATAACATGT&#xA;6390489 CGAGGCTGTATCCTCTACTAGA&#xA;6210446 AGGCAGAATCTCTCCGAGTGCT&#xA;5985219 TAAGGCGATCTCTCCGTACGTG&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where the first column is the number of reads associated with the cell barcode in the second column.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to collapse each set of cell barcodes within 1 Hamming distance (i.e. allowing one mismatch between barcodes), keeping only the barcode with the highest read count and adding the read counts of the duplicates to it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking to use &lt;code&gt;awk&lt;/code&gt; and &lt;code&gt;grep&lt;/code&gt; to filter the barcode list in an iterative procedure using regular expressions, but I think this would require too much time. I can share my code if needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to do it using unix tools or already published pipelines (e.g. UMI tools)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my code at the moment (it is not optimized though):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/awk -f&#xA;&#xA;{&#xA;    matching = 0;&#xA;    for (r in regexs) {&#xA;        if (&lt;span class=&quot;math-container&quot;&gt;$2 ~ regexs[r]) {&#xA;        matching = 1;&#xA;        counts[r] += int($1);&#xA;        break;&#xA;    }&#xA;}&#xA;if (!matching) {&#xA;    regex = &quot;&quot;;&#xA;    for (i = 1; i &amp;lt;= length($2); i++) {&#xA;        l_regex = substr($2, 1, i-1) &quot;.&quot; substr($2, i+1);&#xA;        if (length(regex) == 0)&#xA;            regex = l_regex;&#xA;        else&#xA;            regex = regex &quot;|&quot; l_regex;&#xA;    }&#xA;    regexs[$2] = regex;&#xA;	counts[$2] = int($&lt;/span&gt;1);&#xA;    }&#xA;    # so we know that 100 records have been processed&#xA;    if (!(NR % 100)) printf &quot;.&quot; &amp;gt; &quot;/dev/stderr&quot;;&#xA;}&#xA;END {&#xA;    for (c in counts) {&#xA;        print counts[c], c | &quot;sort -k1 -rn&quot;;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2018-11-05T22:47:59.970" LastActivityDate="2018-11-07T12:09:50.073" Title="Collapse cell barcodes distribution within 1 Hamming distance" Tags="|single-cell|barcode|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5376" PostTypeId="2" ParentId="5373" CreationDate="2018-11-05T18:34:22.810" Score="3" Body="&lt;p&gt;After getting it out on paper (so to speak) I was able to accomplish what I wished with bash:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;gene_list=(&lt;span class=&quot;math-container&quot;&gt;$(cat ./markers_clean.txt))&#xA;query=$&lt;/span&gt;{gene_list[@]:0:1}&#xA;parens=&quot;(&quot;  &#xA;for gene in &lt;span class=&quot;math-container&quot;&gt;${gene_list[@]:1} &#xA;    do&#xA;        query=&quot;$&lt;/span&gt;{query} OR &lt;span class=&quot;math-container&quot;&gt;${gene})&quot;&#xA;        parens=&quot;$&lt;/span&gt;{parens}(&quot;&#xA;    done&#xA;echo &quot;&lt;span class=&quot;math-container&quot;&gt;${parens}$&lt;/span&gt;{query} AND olfactory bulb)&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With the output being:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((Aatf OR Abcb1b) OR Abhd3) OR Abhd4) OR Abhd5) OR Abl2) OR Ablim2) OR Acaa2) OR Ackr3) OR Acsbg1) OR Acvr1c) OR Acvr2a) OR Adam11) OR Adam23) OR Adcy1) OR Adcyap1) OR Adgrg1) OR Adora2a) OR Adra2a) OR Adra2c) OR AF529169) OR Agrp) OR Agt) OR Agtr1a) OR Agtr2) OR Ak4) OR Akap12) OR Akap5) OR Akr1c18) OR Aldh1l1) OR Aldh7a1) OR Aldoc) OR Als2) OR Amfr) OR Amigo1) OR Amigo2) OR Angpt2) OR Ank) OR Ank2) OR Ankrd34b) OR Api5) OR Apod) OR Arc) OR Arf6) OR Arhgap33) OR Arhgdig) OR Arhgef6) OR Arhgef7) OR Arl6ip1) OR Arpp21) OR Arvcf) OR Arx) OR Ascl1) OR Aspm) OR Atg4c) OR Atic) OR Atoh1) OR Atp1a1) OR Atp1b1) OR Atp2a2) OR Atp5a1) OR Atp5j2) OR Atp6v1b2) OR Atpif1) OR Avp) OR Avpr1a) OR Avpr1b) OR B2m) OR B3gat1) OR Bahcc1) OR Baiap2) OR Bcar1) OR Bdnf) OR Becn1) OR Bend5) OR Bmp1) OR Bmp6) OR Bmp7) OR Bnip3) OR Bsx) OR Btbd17) OR Btg1) OR Btg2) OR Btg3) OR C1ql2) OR C1ql3) OR C2cd2l) OR C33) OR Cacna1h) OR Cacna1i) OR Cacnb1) OR Cacng3) OR Cacng5) OR Cadps) OR Calb1) OR Calb2) OR Calca) OR Calcb) OR Calcr) OR Camk2a) OR Camkk1) OR Camkk2) OR Camkv) OR Card6) OR Cartpt) OR Casp9) OR Casr) OR Cbln1) OR Cbln2) OR Cbln4) OR Cbx5) OR Ccdc163) OR Ccdc184) OR Ccdc187) OR Ccdc3) OR Cck) OR Cckbr) OR Ccna2) OR Ccnd1) OR Ccne1) OR Ccne2) OR Ccsap) OR Cd63) OR Cdc42bpg) OR Cdc42ep4) OR Cdc6) OR Cdh1) OR Cdh11) OR Cdh3) OR Cdh6) OR Cdh9) OR Cdhr1) OR Cdk17) OR Cdk4) OR Cdk5) OR Cdk5r1) OR Cdk5r2) OR Cdk6) OR Cdkn1a) OR Cdkn1b) OR Cdkn2a) OR Cdkn2c) OR Cdkn2d) OR Cdon) OR Cdr2) OR Celf6) OR Cemip) OR Chd7) OR Chn2) OR Chrm1) OR Chrm4) OR Chrna2) OR Chrna3) OR Chrna4) OR Chrna5) OR Chrna7) OR Chrnb2) OR Chrnb4) OR Chst15) OR Chst2) OR Cic) OR Cisd1) OR Cited1) OR Ckmt1) OR Clcn1) OR Cldn1) OR Clu) OR Cnr1) OR Cntfr) OR Cntn2) OR Cntnap1) OR Coch) OR Col6a1) OR Colgalt2) OR Cpe) OR Cpne6) OR Crabp1) OR Creb1) OR Crh) OR Crhr1) OR Crhr2) OR Crim1) OR Crip2) OR Crk) OR Crym) OR Csnk1g3) OR Ctgf) OR Ctxn1) OR Cx3cl1) OR Cxcl11) OR Cxcl12) OR Cxcr4) OR Cxxc4) OR Cyb561) OR Cyp19a1) OR Cyp4v3) OR Cyr61) OR D8Ertd738e) OR Dbi) OR Dbp) OR Dcdc2a) OR Dct) OR Dctn1) OR Dcx) OR Ddit4l) OR Dgcr14) OR Dgcr6) OR Dhcr24) OR Dhrs3) OR Dhx57) OR Dkk3) OR Dlg4) OR Dlk1) OR Dll1) OR Dlx1) OR Dlx2) OR Dlx5) OR Dlx6) OR Dmbx1) OR Dnajb1) OR Dnajc12) OR Dnmt1) OR Dnmt3a) OR Doc2g) OR Drd1) OR Drd2) OR Drd3) OR Dsp) OR Dtx4) OR Dtymk) OR Dusp1) OR Dusp14) OR Dusp16) OR Dusp3) OR Dusp5) OR Dyrk1a) OR Dyx1c1) OR Dzip1) OR E2f8) OR Ebf2) OR Ebf3) OR Eef1a1) OR Efna1) OR Efnb1) OR Efnb2) OR Efr3a) OR Efs) OR Egr1) OR Eif1ax) OR Eif3h) OR Eif4e3) OR Eif4g2) OR Elavl2) OR Elavl3) OR Elavl4) OR Elfn1) OR Ell3) OR Emp1) OR Emx1) OR En2) OR Enah) OR Enc1) OR Eno1) OR Eno2) OR Entpd4) OR Eomes) OR Epha4) OR Epha7) OR Epha8) OR Eps8l2) OR Erp29) OR Esr2) OR Esyt3) OR Evx1) OR F2r) OR Fam171b) OR Fam81a) OR Fam84a) OR Fbxl5) OR Fbxo32) OR Fezf1) OR Fezf2) OR Fgd5) OR Fgf1) OR Fgf1) OR Fgf13) OR Fgf15) OR Fgfr1) OR Fign) OR Fkbp2) OR Fkbp3) OR Flrt1) OR Flrt2) OR Fn1) OR Fos) OR Foxi1) OR Foxi2) OR Foxm1) OR Foxp4) OR Frat1) OR Fryl) OR Frzb) OR Fxyd6) OR Fxyd7) OR Fzd2) OR Fzd3) OR Fzd5) OR Fzd8) OR Gaa) OR Gabbr1) OR Gabrb2) OR Gabre) OR Gabrg1) OR Gabrg2) OR Gad2) OR Gal) OR Galnt1) OR Galnt5) OR Galnt9) OR Galr3) OR Gap43) OR Gas1) OR Gbx2) OR Gck) OR Gcm2) OR Gdf1) OR Gfap) OR Gja1) OR Gjd2) OR Glce) OR Gldc) OR Glrb) OR Gls) OR Glud1) OR Gng1) OR Gng13) OR Gng3) OR Gng4) OR Gng5) OR Gng7) OR Gpc4) OR Gpm6a) OR Gpr12) OR Gpr139) OR Gpr151) OR Gpr165) OR Gpr21) OR Gpr27) OR Gpr37) OR Gpr68) OR Gpr83) OR Gpr88) OR Gprin3) OR Gpsm1) OR Gria2) OR Grid2ip) OR Grin2c) OR Grm1) OR Grm2) OR Grm4) OR Grp) OR Gsx2) OR Gtf2i) OR Gucy1a3) OR Hap1) OR Hbegf) OR Hcfc1) OR Hcn2) OR Hcn4) OR Hdac11) OR Hdc) OR Herc3) OR Hes6) OR Hey1) OR Hey2) OR Hgs) OR Hmgn2) OR Hmgn3) OR Hnrnpab) OR Hpcal1) OR Hrasls) OR Hrh2) OR Hsd11b1) OR Hspa12a) OR Hspb8) OR Htr1a) OR Htr1b) OR Htr1d) OR Htr3a) OR Htr3b) OR Htr4) OR Htr5b) OR Htr6) OR Icam5) OR Ict1) OR Id1) OR Id4) OR Idh3a) OR Ier5) OR Igf1) OR Igfbp2) OR Igfbp4) OR Igfbp5) OR Igsf9) OR Inpp5j) OR Insig2) OR Iqsec3) OR Irs4) OR Isl2) OR Islr2) OR Itgb3) OR Itm2b) OR Jag1) OR Jag2) OR Jak2) OR Jam3) OR Jph4) OR Jup) OR Kcnc1) OR Kcnc4) OR Kcnip2) OR Kcnj5) OR Kcnk1) OR Kcnq2) OR Kctd13) OR Kctd4) OR Kif11) OR Kif26a) OR Kit) OR Kitl) OR Klf1) OR Klf13) OR Kremen1) OR Krt19) OR Lamb1) OR Lamp5) OR Layn) OR Lbh) OR Ldb1) OR Lef1) OR Lfng) OR Lgals3) OR Lgi1) OR Lhx1) OR Lhx2) OR Lhx4) OR Lhx5) OR Lhx6) OR Lhx8) OR Limk1) OR Lix1) OR Lmo1) OR Lmo4) OR Lpar1) OR Lpl) OR Lrfn5) OR Lrig1) OR Lrig2) OR Lrp1) OR Lrp12) OR Lrp2) OR Lrrk2) OR Lrrn1) OR Lrrn3) OR Lrrtm2) OR Lss) OR Ly6h) OR Lynx1) OR Lypd1) OR Lypd2) OR Lypd6) OR Lzts1) OR Mafb) OR MAGEA8) OR Man1a2) OR Map1b) OR Map1lc3a) OR Map2k6) OR Map4k5) OR Mapk1) OR Mapk8) OR Mapt) OR March4) OR Matk) OR Mb21d2) OR Mbd2) OR Mbd4) OR Mc3r) OR Mc4r) OR Mdk) OR Med12) OR Mef2c) OR Meis1) OR Met) OR Mfap2) OR Mfap4) OR Mfn1) OR Mmp17) OR Mmp2) OR Mpc1) OR Mst1r) OR Msx1) OR Mtap) OR Mturn) OR Mxi1) OR Mycn) OR Nae1) OR Napb) OR Nars) OR Ncor1) OR Ndn) OR Ndnf) OR Ndrg1) OR Ndufv1) OR Ndufv2) OR Nectin3) OR Nefh) OR Nefl) OR Nefm) OR Neurod1) OR Neurod4) OR Neurod6) OR Neurog1) OR Neurog2) OR Nfil3) OR Nhlh1) OR Nhp2) OR NKAIN2) OR Nkain3) OR Nkx2) OR Nlgn3) OR Nmb) OR Nmbr) OR Nmu) OR Nnat) OR Nog) OR Nomo1) OR Nos1) OR Notch1) OR Notch2) OR Nov) OR Nova1) OR Npc1) OR Npc2) OR Nptx1) OR Nptx2) OR Npy) OR Npy1r) OR Npy2r) OR Npy5r) OR Nr2f2) OR Nr4a1) OR Nrep) OR Nrgn) OR Nrip3) OR Nrp1) OR Nrtn) OR Nrxn2) OR Nsf) OR Nsg1) OR Nsmf) OR Ntn1) OR Ntng2) OR Ntrk1) OR Ntrk2) OR Ntrk3) OR Nts) OR Nudt2) OR Numb) OR Numbl) OR Odc1) OR Olfm1) OR Olfr734) OR Olig2) OR Oma1) OR Ophn1) OR Opn3) OR Opn4) OR Oprk1) OR Otof) OR Otx2) OR Oxt) OR Oxtr) OR P2rx7) OR Pacsin3) OR Pak1) OR Pard6a) OR Pcbd1) OR Pcdh2) OR Pcdh9) OR Pcnt) OR Pcp4) OR Pcsk9) OR Pde1b) OR Pde1c) OR Pdp1) OR Pdyn) OR Peg3) OR Penk) OR Per2) OR Pex5l) OR Pgrmc1) OR Phactr1) OR Phactr4) OR Phox2b) OR Pias3) OR Pik3r1) OR Pirt) OR Pkd1) OR Plagl1) OR Plcxd2) OR Pld3) OR Plekhg1) OR Plk2) OR Plk3) OR Plp1) OR Plxdc1) OR Plxna1) OR Plxnc1) OR Pmp22) OR Pnmt) OR Pnoc) OR Podxl2) OR Polr1d) OR Polr3e) OR Pomc) OR Pou2f1) OR Pou3f4) OR Ppfia4) OR Ppm1e) OR Ppp1ca) OR Ppp1r14c) OR Ppp1r16b) OR Ppp1r17) OR Ppp1r1a) OR Ppp1r1b) OR Ppp1r2) OR Ppp1r7) OR Ppp2r2a) OR Ppp4c) OR Ppp6c) OR Prkacb) OR Prkcd) OR Prkcq) OR Prmt2) OR Prnp) OR Prodh) OR Prok2) OR Prokr2) OR Prox1) OR Prss12) OR Prtg) OR Psen1) OR Psen2) OR Ptch2) OR Pten) OR Ptf1a) OR Ptgds) OR Ptn) OR Ptpn2) OR Ptpn5) OR Ptprf) OR Pus1) OR Pygb) OR Rab37) OR Rac1) OR Ramp3) OR Rap1a) OR Rasa3) OR Rasal1) OR Rasgrp1) OR Rasgrp2) OR Rasl11b) OR Rbp1) OR Rcan2) OR Rcn1) OR Rel) OR Rest) OR Rfng) OR Rgs12) OR Rgs16) OR Rgs2) OR Rgs3) OR Rgs4) OR Rgs5) OR Rgs8) OR Rgs9) OR Rhob) OR Rhou) OR Rif1) OR Rlbp1) OR Rnd1) OR Rnd2) OR Rnd3) OR Rorb) OR Rpl18) OR Rpl31) OR Rprm) OR Rraga) OR Rtn4rl1) OR Rubcn) OR Rufy3) OR Runx1) OR Rxfp3) OR S1) OR Satb2) OR Scg2) OR Scg3) OR Scn3b) OR Scrn1) OR Scrt1) OR Scube1) OR Scube2) OR Sdc3) OR Selenom) OR Selenow) OR Sema3a) OR Sema3b) OR Sema3c) OR Sema3e) OR Sema3f) OR Sema4a) OR Sema6d) OR Sema7a) OR Sept3) OR Sept6) OR Serf1) OR Serpina9) OR Serpini1) OR Sertm1) OR Sez6) OR Sez6l2) OR Sf1) OR Sfrp1) OR Sfrp2) OR Sfrp5) OR Sim1) OR Sirpa) OR Sirt2) OR Skil) OR Sla) OR Slc1) OR Slc12a5) OR Slc13a3) OR Slc17a6) OR Slc17a7) OR Slc18a2) OR Slc1a1) OR Slc1a2) OR Slc1a3) OR Slc1a6) OR Slc22a3) OR Slc25a12) OR Slc32a1) OR Slc33a1) OR Slc41a3) OR Slc6a1) OR Slc6a12) OR Slc6a2) OR Slc6a3) OR Slc6a6) OR Slc6a9) OR Slc7a14) OR Slc8a2) OR Slc9a3r1) OR Slit1) OR Slitrk2) OR Slitrk3) OR Slitrk4) OR Slitrk5) OR Slitrk6) OR Smad6) OR Smarce1) OR Smoc2) OR Snap25) OR Snap91) OR Sncg) OR Snhg11) OR Snx17) OR Sort1) OR Sox1) OR Sox11) OR Sox2) OR Sox4) OR Sox8) OR Sox9) OR Sp4) OR Sprn) OR Spry1) OR Sptan1) OR Sqle) OR Srgap3) OR Srsf7) OR Sstr1) OR Sstr2) OR St8sia5) OR Stard5) OR Stau1) OR Strip2) OR Strn) OR Stum) OR Stx1a) OR Sufu) OR Sulf2) OR Sun2) OR Suv39h1) OR Sv2a) OR Sv2b) OR Syndig1l) OR Syngr3) OR Synj2) OR Syt13) OR Syt17) OR Syt2) OR Syt3) OR Syt4) OR Syt6) OR Syt7) OR Tac1) OR Tac2) OR Tacr3) OR Taf13) OR Tagln3) OR Tbc1d8) OR Tcf3) OR Tgif1) OR Thbs1) OR Thbs2) OR Thbs3) OR Thbs4) OR Them6) OR Thra) OR Thrsp) OR Tigar) OR Tle1) OR Tle3) OR Tmeff1) OR Tmem35a) OR Tmem59l) OR Tnc) OR Tor1b) OR Tpbg) OR Tpd52l1) OR Tph2) OR Tppp3) OR Traf4) OR Trappc3) OR Trf) OR Trh) OR Trhr) OR Trib2) OR Tsc22d2) OR Tspan14) OR Tspan7) OR Tuba1b) OR Tubb3) OR Tubb4a) OR Ube3a) OR Ubxn11) OR Uchl1) OR Ucp2) OR Uhmk1) OR Uncx) OR Uso1) OR Vat1) OR Vcan) OR Vdac2) OR Vegfa) OR Vim) OR Vip) OR Vipr2) OR Vldlr) OR Vmn1r2) OR Vsnl1) OR Vsx2) OR Washc2) OR Wdr46) OR Wfs1) OR Wif1) OR Wnt2b) OR Wnt5b) OR Wnt7a) OR Wnt8b) OR Wrb) OR Wscd1) OR Wwc1) OR Ypel4) OR Zbtb18) OR Zbtb2) OR Zcchc12) OR Zfp5) OR Zfp532) OR Zic1) OR Zic2) OR Zic3) OR Zic4) OR Zic5) OR Zwint) AND olfactory bulb)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As far as I can tell, this does not surpass any character limits on Pubmed.&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="181" LastEditDate="2018-11-05T20:42:17.003" LastActivityDate="2018-11-05T20:42:17.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5377" PostTypeId="2" ParentId="5354" CreationDate="2018-11-05T20:08:56.090" Score="2" Body="&lt;p&gt;This python script will remove all stop codons from your fasta file. I called it &lt;code&gt;remove_stops.py&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;import sys&#xA;from Bio import SeqIO&#xA;stop_codons = [&quot;TAG&quot;, &quot;TGA&quot;, &quot;TAA&quot;, &quot;UAG&quot;, &quot;UGA&quot;, &quot;UAA&quot;]&#xA;&#xA;fasta_it = SeqIO.parse(open(sys.argv[1]), 'fasta')&#xA;for fasta in fasta_it:&#xA;    name, sequence = fasta.description, str(fasta.seq)&#xA;    try:&#xA;        assert(len(sequence) % 3 == 0)&#xA;    except:&#xA;        raise IOError(&quot;length of sequence {} was not divisible by three&quot;.format(name))&#xA;    print(&quot;&amp;gt;{}&quot;.format(name))&#xA;    print(&quot;&quot;.join([sequence[i:i+3] for i in range(0, len(sequence) - 3 + 1, 3)&#xA;                   if sequence[i:i+3] not in stop_codons]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can run it like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python remove_stops.py stop_codons.fasta &amp;gt; filtered.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This program won't work if there are any fasta records that do not have a length divisible by three. This indicates a sliding translation stop and these sequences should be trimmed individually. Also, all the genes must be in the 5'-&gt;3' direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It works by looking at each codon &lt;code&gt;[sequence[i:i+3] for i in range(0, len(sequence) - 3 + 1, 3)&lt;/code&gt;, then just makes sure they are not a stop codon: &lt;code&gt;if sequence[i:i+3] not in stop_codons]&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-11-05T20:08:56.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5379" PostTypeId="2" ParentId="5375" CreationDate="2018-11-06T08:32:14.497" Score="3" Body="&lt;p&gt;This doesn't use standard unix tools, but rather the API for UMI tools:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python&#xA;from umi_tools import network&#xA;&#xA;cb = []  # cell barcodes&#xA;cbc = dict()  # cell barcode counts&#xA;for line in open(&quot;UMIs.txt&quot;):&#xA;    cols = line.strip().split(&quot;\t&quot;)&#xA;    cols[1] = bytes(cols[1], &quot;ascii&quot;)&#xA;    cb.append(cols[1])&#xA;    cbc[cols[1]] = int(cols[0])&#xA;&#xA;uc = network.UMIClusterer()&#xA;CBclusters = uc(cb, cbc, 1)&#xA;cbFinal = dict()&#xA;for l in CBclusters:  # This is a list with the first element the dominant barcode&#xA;    cbFinal[l[0]] = 0&#xA;    for x in l:  # Iterate over all barcodes in a cluster&#xA;        cbFinal[l[0]] += cbc[x]&#xA;print(cbFinal)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you give it input in &lt;code&gt;UMIs.txt&lt;/code&gt; of the following form (note the last two lines differ from the first with a distance of 1):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;11612552    TCCTGAGCACTGCATAACTCAA&#xA;9349711 GCTACGCTACTGCATAAGTCCA&#xA;8343678 CAGAGAGGCTAAGCCTGCACAT&#xA;8161950 CGTACTAGTCTCTCCGCGGCTA&#xA;8102383 TCCTGAGCGTAAGGAGCAGATC&#xA;7110298 AAGAGGCAGTAAGGAGATAATC&#xA;6626630 TAAGGCGAACTGCATAACATGT&#xA;6390489 CGAGGCTGTATCCTCTACTAGA&#xA;6210446 AGGCAGAATCTCTCCGAGTGCT&#xA;5985219 TAAGGCGATCTCTCCGTACGTG&#xA;20  TCCAGAGCACTGCATAACTCAA&#xA;2   TCCTGAGCACTGCATAACTCNA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then it will produce:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{b'TAAGGCGATCTCTCCGTACGTG': 5985219,&#xA; b'TAAGGCGAACTGCATAACATGT': 6626630,&#xA; b'GCTACGCTACTGCATAAGTCCA': 9349711,&#xA; b'CGTACTAGTCTCTCCGCGGCTA': 8161950, &#xA; b'TCCTGAGCGTAAGGAGCAGATC': 8102383,&#xA; b'CAGAGAGGCTAAGCCTGCACAT': 8343678, &#xA; b'TCCTGAGCACTGCATAACTCAA': 11612574,&#xA; b'AGGCAGAATCTCTCCGAGTGCT': 6210446, &#xA; b'AAGAGGCAGTAAGGAGATAATC': 7110298,&#xA; b'CGAGGCTGTATCCTCTACTAGA': 6390489}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-11-06T08:32:14.497" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5380" PostTypeId="1" AcceptedAnswerId="5381" CreationDate="2018-11-06T09:35:03.183" Score="2" ViewCount="171" Body="&lt;p&gt;I have an output from vcfutils.pl vcf2fq with specified minimal depth, and it means that nucleotides with not enough depth are lowercase. &#xA;I would like to change them to gaps. I could do it in higher language but I would like to know the solution in bash. &#xA;I have found simple solution to replace all lowercase to uppercase:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN{FS=&quot; &quot;}{if(!/&amp;gt;/){print toupper(&lt;span class=&quot;math-container&quot;&gt;$0)}else{print $&lt;/span&gt;1}}' in.fna &amp;gt; out.fna&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but I would like to replace lowercase with -. The first row need to be omitted. Could you help me? Thanks&lt;/p&gt;&#xA;" OwnerUserId="260" LastActivityDate="2018-11-06T12:09:06.477" Title="Replace lowercase characters with -" Tags="|bash|shell|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5381" PostTypeId="2" ParentId="5380" CreationDate="2018-11-06T10:26:20.770" Score="4" Body="&lt;p&gt;The following &lt;code&gt;sed&lt;/code&gt; command will do the trick.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;sed -e '/^[^&amp;gt;]/ s/[a-z]/\-/g' in.fna &amp;gt; out.fna&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lines starting with &lt;code&gt;&amp;gt;&lt;/code&gt; will be escaped and the lowercase characters will be replaced with &lt;code&gt;-&lt;/code&gt; .&lt;/p&gt;&#xA;" OwnerUserId="206" LastActivityDate="2018-11-06T10:26:20.770" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5382" PostTypeId="2" ParentId="5380" CreationDate="2018-11-06T10:30:03.637" Score="3" Body="&lt;p&gt;You could try it with &lt;code&gt;sed&lt;/code&gt;.&lt;br&gt;&#xA;Replace all lower-case chars with &lt;code&gt;-&lt;/code&gt; in lines not starting with &lt;code&gt;&amp;gt;&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -e '/^&amp;gt;/!s/[a-z]/-/g' in.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1802" LastEditorUserId="48" LastEditDate="2018-11-06T10:57:20.960" LastActivityDate="2018-11-06T10:57:20.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5383" PostTypeId="2" ParentId="5380" CreationDate="2018-11-06T12:09:06.477" Score="1" Body="&lt;p&gt;You can also use this solution where &lt;strong&gt;tr&lt;/strong&gt; command from linux can solve your problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;tail -n +2  in.fna | tr '[:lower:]' '-'&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="1194" LastActivityDate="2018-11-06T12:09:06.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5384" PostTypeId="1" CreationDate="2018-11-06T12:36:02.260" Score="1" ViewCount="2005" Body="&lt;p&gt;Dear knowledgeable bioinformaticians, I'm currently pursuing an undergraduate degree and as a result I do not have so much knowledge in this topic. I'm having some troubles understanding my phylogenetic three construction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the construction of a phylogenetic tree UGENE was used. 11 homologous proteins from different species in the same genus/phyla were aligned. For the tree construction PHYLYP Neighbour Joining (NJ) with the distance matrix model “Jones-Taylor-Thornton” was used. Two trees were constructed, one without bootstrapping and one with bootstrapping (1000 pseudoreplicates). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I interpret my values? Are they unusable since I have low bootstrap-values in a lot of my nodes? All guidance is appreciated, thank you.&lt;a href=&quot;https://i.stack.imgur.com/P0WtG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/P0WtG.png&quot; alt=&quot;Without Bootstrap&quot;&gt;&lt;/a&gt; &lt;a href=&quot;https://i.stack.imgur.com/H97tW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/H97tW.png&quot; alt=&quot;With Bootstrap&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3638" LastEditorUserId="48" LastEditDate="2018-11-06T13:16:07.817" LastActivityDate="2018-11-06T15:36:14.027" Title="How do I interpret my phylogenetic tree construction from Glucosidases in the Actinobacteria class?" Tags="|phylogeny|figure-explanation|phylogenetics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5385" PostTypeId="2" ParentId="5384" CreationDate="2018-11-06T15:36:14.027" Score="2" Body="&lt;p&gt;Bootstrap values usually give you useful information, be they low or high. Usually, a &quot;high&quot; bootstrap value is something &gt;= 70%. That means a clade is strongly supported; lower values are telling you the clade is not clearly separated from others. This could be for several reasons, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) A subpopulation is not really separated from others, e.g. the viral sequences in  town &quot;A&quot; and town &quot;B&quot; are completely admixed, meaning the population is panmictic and there is continuous gene flow between the two of them;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) the populations diverged not long ago, it's too soon to have any phylogenetic signal, at least using the marker you are using (different genes evolve at different rates). Usually, housekeeping genes (which are usually very conserved) are used for populations that supposedly diverged long time ago.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The interpretation of your tree should take into account the biology and epidemiology of your target organism. If you really want to learn more, I suggest a lot of reading, and definitely a class dedicated to molecular phylogenetics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This might be a good start:&#xA;&lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/0521730716&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.amazon.com/Phylogenetic-Handbook-Practical-Approach-Hypothesis/dp/0521730716&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, the NJ algorithm should allow for uneven branch lengths, maybe the tree you are showing is the consensus tree?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Max&lt;/p&gt;&#xA;" OwnerUserId="3536" LastActivityDate="2018-11-06T15:36:14.027" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5386" PostTypeId="1" CreationDate="2018-11-06T16:20:59.703" Score="6" ViewCount="64" Body="&lt;p&gt;I am working on Mac OSX. I have downloaded RepeatMasker from &lt;a href=&quot;http://www.repeatmasker.org/RepeatMasker-open-4-0-7.tar.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;www.repeatmasker.org&lt;/a&gt;. I have downloaded RepBase from &lt;a href=&quot;https://www.girinst.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;www.girinst.org&lt;/a&gt;. I have download the WS266 version of the &lt;em&gt;C. elegans&lt;/em&gt; genome from &lt;a href=&quot;http://www.wormbase.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;WormBase&lt;/a&gt;. I have installed nhmmer using conda.&lt;/p&gt;&#xA;&lt;p&gt;I have then run the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;repeatmasker -s -lib RepBase23.08.fasta/celrep.ref -dir annotation/&#xA;-gff c_elegans.PRJNA13758.WS266.genomic.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then imported the annotation table into R. I notice that some transposons have a length of 1!&lt;/p&gt;&#xA;&lt;p&gt;How is this possible?!&lt;/p&gt;&#xA;&lt;p&gt;Also, how can repeatmasker calculate such percent divergence, deletion, insertion with a query width of 1? For some of these predictions, both the query and repeat lengths are 1 (so an alignment of 1 nucleotide against 1 nucleotide!)! But the Smith-Waterman scores don't match this. There is something important I am missing here. Can someone help me interpret the table below and explain why some transposons have a length of 1 in the genome?&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TnBVZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TnBVZ.png&quot; alt=&quot;Transposons of length 1&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The image only contains transposons with a width of 1 as far as I understand. I've selected the rows where queryBegin == queryEnd. The queryBegin variable = queryEnd for all the lines here. It is a subset of the original data frame.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="73" LastEditDate="2021-05-10T06:11:39.793" LastActivityDate="2021-05-10T06:11:39.793" Title="Why does repeatmasker annotate transposons of length 1?" Tags="|sequence-annotation|repeat-elements|repeat|repeatmasker|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5387" PostTypeId="1" CreationDate="2018-11-06T16:31:57.637" Score="1" ViewCount="81" Body="&lt;p&gt;Here, &lt;a href=&quot;https://www.youtube.com/watch?v=E9-Rm5AoZGw&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=E9-Rm5AoZGw&lt;/a&gt;, ONT claims they can read non-fragmented DNA and that with the hairpin like structure the double strands can be read at once. Then why is that MinION reads are of variable in length and not the length of a whole DNA? BUt here:&lt;a href=&quot;https://www.youtube.com/watch?v=e9wtAIvPPxY&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=e9wtAIvPPxY&lt;/a&gt;, ONT claims that the read lenght can be configured, what does this mean?&lt;/p&gt;&#xA;" OwnerUserId="3627" LastEditorUserId="181" LastEditDate="2018-11-06T19:57:16.370" LastActivityDate="2018-11-06T22:28:45.027" Title="Using ONT MinION, why is that we cannot get a full length DNA read?" Tags="|nanopore|long-reads|minion|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5388" PostTypeId="2" ParentId="5387" CreationDate="2018-11-06T22:28:45.027" Score="2" Body="&lt;p&gt;The MinION should be able to sequence DNA fragments entirely, regardless of the size, if you can get them to the pore intact. That is the problem: during DNA extraction and library preparation you will introduce breakage.&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2018-11-06T22:28:45.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5390" PostTypeId="1" CreationDate="2018-11-07T02:06:15.113" Score="2" ViewCount="23" Body="&lt;p&gt;I'm having some trouble differentiating between CNV, split reads and read pairs and how each one elucidates the presence of a structural variation, say deletion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a reference and donor sequence, I'd like to see a visual depiction (a link to a reference would be welcome) of a deletion event being detected by each of the above read types. In particular, I'd like to see how each of the read types detects the deletion event itself (i.e structural variant type) and its location (i.e the breakpoint junction). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bonus points if it also includes some of the other common structural variants like inversion, duplication and insertion. &lt;/p&gt;&#xA;" OwnerUserId="3642" LastActivityDate="2018-11-07T02:06:15.113" Title="Visual representation of detecting deletion by different read types" Tags="|read-mapping|structural-variation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5391" PostTypeId="1" AcceptedAnswerId="5392" CreationDate="2018-11-07T11:25:50.640" Score="2" ViewCount="47" Body="&lt;p&gt;Sorry, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have analysed my time series single cell data with Scuba algorithm but that says that there is no bifurcations in my data and outputted a single chain of 4 states with no branching like below figure&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RZKSo.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RZKSo.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry, I googled but I am not quite sure what this means&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion please?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-01-06T14:54:53.940" LastActivityDate="2019-01-06T14:54:53.940" Title="What does that mean if there is no bifurcations in my time series single cell data?" Tags="|single-cell|matlab|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5392" PostTypeId="2" ParentId="5391" CreationDate="2018-11-07T11:43:10.040" Score="7" Body="&lt;p&gt;The most obvious interpretation is that the four cell lines are related in a linear pathway rather than a branching one. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-11-07T11:43:10.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5393" PostTypeId="2" ParentId="5375" CreationDate="2018-11-07T12:09:50.073" Score="3" Body="&lt;p&gt;The difficulty with all these things comes with what to do about barcodes that are 1 edit distance away from more than one other barcodes in ways that form complex networks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cell barcodes are a bit different from UMIs. Partly this is because we expect them to be sparser. Partly because the consequences of inncorect error correction at the CB level are much worst than at the UMI level. The way that UMI-tools does its Cell Barcode error correction is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Determine a whitelist of barcodes using the &quot;knee&quot; method&lt;/li&gt;&#xA;&lt;li&gt;Iterate through the list of non-whitelist barcodes and check if they are within one edit distance of one &lt;strong&gt;and only one&lt;/strong&gt; whitelist barcode. (this is done with &lt;code&gt;umi_tools.umi_methods.getErrorCorrectMapping&lt;/code&gt; if you are interested).&lt;/li&gt;&#xA;&lt;li&gt;Things that pass the above are collapsed into the whilelist barcode, else they are discarded. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The function &lt;code&gt;umi_tools.umi_methods.getCellWhitelist&lt;/code&gt; takes a list of barcodes and their frequencies and returns a whitelist and (optionally) a mapping of which barcodes should be corrected to which others. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-11-07T12:09:50.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5395" PostTypeId="1" AcceptedAnswerId="5402" CreationDate="2018-11-07T18:11:58.103" Score="4" ViewCount="74" Body="&lt;p&gt;Using the count of rpkm values from genes in a metagenome sample, I want to group these genes into established categories (for example KEGG or COG). For each sample, my goal is to determine which categories are better represented in each one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Considering the example table whit genes, samples (1-5) and categories.&#xA;Each gene has their expression and the column categories classify them.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import sys&#xA;import numpy&#xA;&#xA;dictionary = {}&#xA;&#xA;table = open(sys.argv[1], &quot;r&quot;)&#xA;&#xA;&#xA;for line in table:&#xA;    next()&#xA;    cols = line.strip().split(&quot;\t&quot;)&#xA;    gene = cols[0]&#xA;    s1 = float(cols[1])&#xA;    s2 = float(cols[2])&#xA;    s3 = float(cols[3])&#xA;    s4 = float(cols[4])&#xA;    s5 = float(cols[5])&#xA;    counts = [s1, s2, s3, s4, s5]&#xA;    kegg = cols[6].strip().split(&quot; &quot;)[0].replace('&quot;','')&#xA;&#xA;&#xA;    if kegg not in dictionary:&#xA;        dictionary[kegg] = [counts]&#xA;    else:&#xA;        dictionary[kegg].append(counts)&#xA;&#xA;&#xA;#print dictionary&#xA;for k, v in sorted(dictionary.iteritems()):&#xA;    m = numpy.array(v)&#xA;    print k,  [sum(m[:,i]) for i in range(5)] # range(number_of_samples)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Original table to count the columns values:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gene    s1  s2  s3  s4  s5  Category&#xA;name01  0   2   2   0   0   A&#xA;name02  3   0   1   0   0   A&#xA;name03  0   2   1   0   0   A&#xA;name04  0   0   1   0   0   B&#xA;name05  5   0   1   0   0   C&#xA;name06  1   0   0   0   0   D&#xA;name07  2   0   0   0   0   D&#xA;name08  0   0   3   0   0   E&#xA;name09  1   0   0   0   0   F&#xA;name10  1   0   0   0   0   F&#xA;name11  3   0   0   0   0   F&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result obtained is this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;&quot;&quot;&#xA;# eliminated from the question&#xA;&#xA;Cat s1  s2  s3  s4  s5&#xA;A   0.0 2.0 1.0 0.0 0.0&#xA;B   0.0 0.0 1.0 0.0 0.0&#xA;C   5.0 0.0 1.0 0.0 0.0&#xA;D   2.0 0.0 0.0 0.0 0.0&#xA;E   0.0 0.0 3.0 0.0 0.0&#xA;F   3.0 0.0 0.0 0.0 0.0&#xA;&quot;&quot;&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But my expected (obtained) table is this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Cat s1  s2  s3  s4  s5&#xA;A   3.0 4.0 4.0 0.0 0.0&#xA;B   0.0 0.0 1.0 0.0 0.0&#xA;C   5.0 0.0 1.0 0.0 0.0&#xA;D   3.0 0.0 0.0 0.0 0.0&#xA;E   0.0 0.0 3.0 0.0 0.0&#xA;F   5.0 0.0 0.0 0.0 0.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3565" LastEditorUserId="3565" LastEditDate="2018-11-08T09:10:49.900" LastActivityDate="2018-11-08T12:27:57.697" Title="Group genes by functional categories suming expression values" Tags="|python|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5396" PostTypeId="2" ParentId="5395" CreationDate="2018-11-07T19:58:07.430" Score="1" Body="&lt;p&gt;Below is code to do that, though if these are really gene counts then I highly suggest you provide more details about why you're trying to do this before you actually do it.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python&#xA;print(&quot;Cat s1  s2  s3  s4  s5&quot;)&#xA;v = []&#xA;category = None&#xA;for line in open(&quot;counts.txt&quot;):&#xA;    if line.startswith(&quot;gene&quot;):&#xA;        continue&#xA;    cols = line.split()&#xA;    if cols[-1] == category:&#xA;        for idx in range(5):&#xA;            v[idx] += int(cols[idx + 1])&#xA;    else:&#xA;        if category:&#xA;            print(&quot;{}   {}&quot;.format(category, &quot; &quot;.join([&quot;{:.1f}&quot;.format(x) for x in v])))&#xA;        v = [int(x) for x in cols[1:-1]]&#xA;        category = cols[-1]&#xA;print(&quot;{}   {}&quot;.format(category, &quot; &quot;.join([&quot;{:.1f}&quot;.format(x) for x in v])))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-11-07T19:58:07.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5397" PostTypeId="1" AcceptedAnswerId="5406" CreationDate="2018-11-08T09:11:03.533" Score="4" ViewCount="246" Body="&lt;p&gt;I am going to get some data from plasmid sequencing to identify SNPs on the plasmids. What is done in the lab is the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The plasmids are purified by size.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We amplify the plasmids using the phi29 polymerase. The polymerase will go through the plasmid multiple times. Hence we get the same sequence concatenated multiple times. &lt;strong&gt;My question is related to this step:&lt;/strong&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Finally, We sequence it using Oxford Nanopore.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;My question is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On step two, I wrote that &quot;we get the same sequence concatenated multiple times&quot;. For me, this a potential source of information to correct the base calls prior to aligning them to the reference. Since you have the same sequence multiple times (concatemers). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would like to know is if there is a tool that uses the information of the concatemers to improve the base calls. I have tried to find some methods but found none. I know that PacBio has a similar flavour using the &quot;circular consensus calling&quot;, but I have not found any methodological explanation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks!&lt;/p&gt;&#xA;" OwnerUserId="706" LastEditorUserId="706" LastEditDate="2018-11-09T21:56:01.647" LastActivityDate="2018-11-09T21:56:01.647" Title="Error correction within the long read" Tags="|nanopore|long-reads|minion|literature-search|minimap2|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5398" PostTypeId="1" AcceptedAnswerId="5399" CreationDate="2018-11-08T09:19:37.557" Score="1" ViewCount="1914" Body="&lt;p&gt;I'm having a bit of confusion about how can I retrieve all the genes that are associated to a specific gene ontology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets say I need to find all the genes that are associated with this GO entry:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;GO:0030098 lymphocyte differentiation&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ebi.ac.uk/QuickGO/GTerm?id=GO:0030098&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; is the graph representing the parents of my GO. In the same page I find the &lt;strong&gt;Child Terms&lt;/strong&gt; that represent the children of my term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I can download the goa file (related to the organism I'm interested into) containing the GO associations and &lt;code&gt;grep&lt;/code&gt; my term to extract a list of associated genes but, as far as I can understand, this list does not represent the whole graph that actually describe my GO term. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I also get the genes in the childred terms of my term? Or in the parents terms? If yes, then again I have to go back in the graph and find the parents/children of the found parents/children and so on, till the whole graph is visited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this the correct approach? Or should I only focus on those genes related to my GO:0030098, and that's it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found this:&#xA;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2533/retrieving-a-list-of-human-genes-having-go-associations&quot;&gt;Retrieving a list of human genes having GO associations&lt;/a&gt;&#xA;and &lt;a href=&quot;https://www.biostars.org/p/52101/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; but I think they don't address my question since they do not describe the parent-children association.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also I found &lt;a href=&quot;http://www.geneontology.org/faq/how-can-i-programmatically-get-list-go-terms-associated-gene-identifier&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;, on the GO FAQ, but what I need to do is quite the opposite.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Summarising: I need to find the genes associated to a specific GO term. Should I also look for its parents or children or not?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any clues?&lt;/p&gt;&#xA;" OwnerUserId="3119" LastEditorUserId="48" LastEditDate="2018-11-08T10:26:33.487" LastActivityDate="2018-11-28T14:45:21.140" Title="Retrieve all genes for a specific gene ontology" Tags="|gene-ontology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5399" PostTypeId="2" ParentId="5398" CreationDate="2018-11-08T09:27:44.450" Score="4" Body="&lt;p&gt;You can use &lt;a href=&quot;http://www.ensembl.org/biomart/martview&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioMart&lt;/a&gt; to filter by your GO term and get the genes as attributes. BioMart is ontology-aware so will pull out all genes associated with your term and with any of its child terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no need to look up the GO child terms as BioMart already deals with this. A term may have more than one parent term, but if it's associated with a gene then everything about that term is associated with the gene. The term is not associated with the gene via parent X or parent Y, it means that both are relevant. So if the child term is associated with the gene, then the parent term is by proxy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're not already familiar with BioMart, there's a &lt;a href=&quot;https://www.youtube.com/watch?v=QvGT2G0-hYA&quot; rel=&quot;nofollow noreferrer&quot;&gt;help video&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ensembl.org/biomart/martview/dc43680a090b326feaf65bec5d81dcef?VIRTUALSCHEMANAME=default&amp;amp;ATTRIBUTES=hsapiens_gene_ensembl.default.feature_page.ensembl_gene_id|hsapiens_gene_ensembl.default.feature_page.external_gene_name|hsapiens_gene_ensembl.default.feature_page.go_id|hsapiens_gene_ensembl.default.feature_page.name_1006&amp;amp;FILTERS=hsapiens_gene_ensembl.default.filters.go_parent_term.%22GO:0030098%22&amp;amp;VISIBLEPANEL=resultspanel&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's a BioMart query that gets you exactly what you need in human using the web interface&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to work with the R interface to biomaRt, &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/biomaRt/inst/doc/biomaRt.html#retrieve-all-entrezgene-identifiers-and-hugo-gene-symbols-of-genes-which-have-a-map-kinase-activity-go-term-associated-with-it.&quot; rel=&quot;nofollow noreferrer&quot;&gt;here's some documentation&lt;/a&gt; and this is the identical query to above:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(biomaRt)&#xA;ensembl=useEnsembl(biomart=&quot;ensembl&quot;, dataset='hsapiens_gene_ensembl')&#xA;getBM(attributes = c('ensembl_gene_id','external_gene_name', 'go_parent_term', 'name_1006'), &#xA;      filters = 'go', &#xA;      values = 'GO:0030098', &#xA;      mart = ensembl)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1332" LastEditorUserId="1332" LastEditDate="2018-11-28T14:45:21.140" LastActivityDate="2018-11-28T14:45:21.140" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="5400" PostTypeId="1" CreationDate="2018-11-08T11:09:14.337" Score="1" ViewCount="5373" Body="&lt;p&gt;I downloaded c5: gene ontology gene sets file from &lt;a href=&quot;http://software.broadinstitute.org/gsea/downloads.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://software.broadinstitute.org/gsea/downloads.jsp&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I opened the &quot;c5.all.v6.2.symbols.gmt&quot; file in csv format and It looks like below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3bkVE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3bkVE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to convert the .gmt file into a dataframe. And it should look like below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4f5qp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4f5qp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;can anyone say how I can do that. Thanq&lt;/p&gt;&#xA;" OwnerUserId="1369" LastActivityDate="2018-12-08T18:00:16.260" Title="How to convert data in gmt format to dataframe?" Tags="|r|file-formats|format-conversion|gsea|conversion|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5401" PostTypeId="2" ParentId="5400" CreationDate="2018-11-08T12:00:40.937" Score="1" Body="&lt;p&gt;You can import the csv file as a table. This does not require further libraries:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#you're reading a csv file, using tab as field separator and considering the first line as the headers of the data table&#xA;data &amp;lt;- read.csv(filename, sep=&quot;\t&quot;, header=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then what you need are just two columns of your &lt;strong&gt;data&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;colnames(data) #will give you the names of the columns that can be accessed&#xA;#to access two columns and merge them into a selectedColumns table&#xA;selectedColumns &amp;lt;- cbind(data&lt;span class=&quot;math-container&quot;&gt;$colName1, data$&lt;/span&gt;colName2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3119" LastEditorUserId="48" LastEditDate="2018-11-08T14:15:16.120" LastActivityDate="2018-11-08T14:15:16.120" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5402" PostTypeId="2" ParentId="5395" CreationDate="2018-11-08T12:08:39.553" Score="4" Body="&lt;p&gt;As suggested in the comments, the &lt;a href=&quot;http://pandas.pydata.org/pandas-docs/stable/groupby.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;pandas&lt;/a&gt; module is quite convenient for this type of work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is how you could do (assuming you have tab-separated values as input, and you want similar file format for the output):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;counts = pd.read_table(&quot;counts.tsv&quot;)&#xA;# At this point, counts looks as follows:&#xA;#       gene  s1  s2  s3  s4  s5 Category&#xA;# 0   name01   0   2   2   0   0        A&#xA;# 1   name02   3   0   1   0   0        A&#xA;# 2   name03   0   2   1   0   0        A&#xA;# 3   name04   0   0   1   0   0        B&#xA;# 4   name05   5   0   1   0   0        C&#xA;# 5   name06   1   0   0   0   0        D&#xA;# 6   name07   2   0   0   0   0        D&#xA;# 7   name08   0   0   3   0   0        E&#xA;# 8   name09   1   0   0   0   0        F&#xA;# 9   name10   1   0   0   0   0        F&#xA;# 10  name11   3   0   0   0   0        F&#xA;&#xA;summed_by_cat = counts.groupby(&quot;Category&quot;).sum()&#xA;# At this point, summed_by_cats looks as follows:&#xA;#           s1  s2  s3  s4  s5&#xA;# Category                    &#xA;# A          3   4   4   0   0&#xA;# B          0   0   1   0   0&#xA;# C          5   0   1   0   0&#xA;# D          3   0   0   0   0&#xA;# E          0   0   3   0   0&#xA;# F          5   0   0   0   0&#xA;&#xA;# Change the index name to get the desired first column header in the output:&#xA;summed_by_cat.index.name = &quot;Cat&quot;&#xA;&#xA;# Write to a file&#xA;summed_by_cat.to_csv(&quot;by_cat.tsv&quot;, sep=&quot;\t&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And resulting the file contains the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Cat s1  s2  s3  s4  s5&#xA;A   3   4   4   0   0&#xA;B   0   0   1   0   0&#xA;C   5   0   1   0   0&#xA;D   3   0   0   0   0&#xA;E   0   0   3   0   0&#xA;F   5   0   0   0   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You could also load your table using &lt;code&gt;counts = pd.read_table(&quot;data.tsv&quot;, index_col=&quot;gene&quot;)&lt;/code&gt; and &lt;code&gt;counts&lt;/code&gt; would have the gene names as index instead of default numbers:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;        s1  s2  s3  s4  s5 Category&#xA;gene                               &#xA;name01   0   2   2   0   0        A&#xA;name02   3   0   1   0   0        A&#xA;name03   0   2   1   0   0        A&#xA;name04   0   0   1   0   0        B&#xA;name05   5   0   1   0   0        C&#xA;name06   1   0   0   0   0        D&#xA;name07   2   0   0   0   0        D&#xA;name08   0   0   3   0   0        E&#xA;name09   1   0   0   0   0        F&#xA;name10   1   0   0   0   0        F&#xA;name11   3   0   0   0   0        F&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This doesn't seem to affect the final outcome.&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2018-11-08T12:27:57.697" LastActivityDate="2018-11-08T12:27:57.697" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5403" PostTypeId="2" ParentId="5400" CreationDate="2018-11-08T12:14:50.920" Score="1" Body="&lt;p&gt;You can use GSA.read.gmt function from &lt;a href=&quot;https://cran.r-project.org/web/packages/GSA/GSA.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSA&lt;/a&gt; package. The following code can be used to convert the file to a dataframe. Just ignore the warnings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Original_response&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(GSA)&#xA;data &amp;lt;-  GSA.read.gmt(&quot;c5.all.v6.2.symbols.gmt&quot;)&#xA;gene_names &amp;lt;- unlist(data&lt;span class=&quot;math-container&quot;&gt;$genesets, use.names=FALSE)&#xA;your_dataframe &amp;lt;- cbind(data$&lt;/span&gt;geneset.names,gene_names)&#xA;colnames(your_dataframe) &amp;lt;- c(&quot;Pathways&quot;,&quot;Genes&quot;)&#xA;mydataframe &amp;lt;- as.data.frame(your_dataframe)&#xA;head(mydataframe)&#xA;                                               Pathway   Gene&#xA;1            GO_POSITIVE_REGULATION_OF_VIRAL_TRANSCRIPTION POLR2C&#xA;2                           GO_CARDIAC_CHAMBER_DEVELOPMENT POLR2J&#xA;3 GO_DNA_DEPENDENT_DNA_REPLICATION_MAINTENANCE_OF_FIDELITY  CTDP1&#xA;4                                      GO_CIRCADIAN_RHYTHM   RDBP&#xA;5              GO_PHOSPHATIDYLSERINE_ACYL_CHAIN_REMODELING COBRA1&#xA;6                               GO_SPINAL_CORD_DEVELOPMENT   RSF1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edited_response&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use the code given below to achieve your desired output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(GSA)&#xA;data &amp;lt;-  GSA.read.gmt(&quot;c5.all.v6.2.symbols.gmt&quot;)&#xA;len_vec=c()           # Now create a vector for containing the length of genes at each position&#xA;len_vec[1] = 3&#xA;for(i in 1:length(data&lt;span class=&quot;math-container&quot;&gt;$genesets)){len_vec[i] &amp;lt;- c(length(data$genesets[[i]]))}&#xA;pathway_vec &amp;lt;- unlist(Vectorize(rep.int)(data$&lt;/span&gt;geneset.names, len_vec),use.names = FALSE) # Now create a vector for all the pathways in the data &#xA;desired_df &amp;lt;- as.data.frame(cbind(pathway_vec,unlist(data$genesets,use.names = FALSE))) # This gives your desired dataframe&#xA;head(desired_df)&#xA;                                    Pathway      Gene&#xA;1 GO_POSITIVE_REGULATION_OF_VIRAL_TRANSCRIPTION POLR2C&#xA;2 GO_POSITIVE_REGULATION_OF_VIRAL_TRANSCRIPTION POLR2J&#xA;3 GO_POSITIVE_REGULATION_OF_VIRAL_TRANSCRIPTION  CTDP1&#xA;4 GO_POSITIVE_REGULATION_OF_VIRAL_TRANSCRIPTION   RDBP&#xA;5 GO_POSITIVE_REGULATION_OF_VIRAL_TRANSCRIPTION COBRA1&#xA;6 GO_POSITIVE_REGULATION_OF_VIRAL_TRANSCRIPTION   RSF1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2018-11-08T17:09:22.203" LastActivityDate="2018-11-08T17:09:22.203" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5404" PostTypeId="1" CreationDate="2018-11-08T12:53:16.883" Score="1" ViewCount="213" Body="&lt;p&gt;I have a data frame presenting different variables in column and for different species name in rows as follow. In the column a have only one independent variables(Proteome size) and all the others are independent variables. I want to use the PIC package to correlate all these dependent variables separately to my independent variables, a way to generate the R coefficient and p-value.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data &amp;lt;- read.table(&quot;t1.txt&quot;, row.names = 1, header = T,sep='')&#xA;&#xA;data &#xA;path1 path2 path3 path4 Proteome_size&#xA;ahli 4.0 -3.2 0.4 -1.0 -0.2&#xA;alayoni 3.8 3.4 -1.8 2.2 -0.3&#xA;alfaroi 3.5 2.6 0.8 -2.4 -0.5&#xA;aliniger 4.0 0.1 -1.7 2.4 0.0&#xA;allisoni 4.4 2.0 -3.7 0.5 -0.1&#xA;allogus 4.0 -2.8 0.6 -1.0 0.6&#xA;altitudinalis 3.8 2.9 -6.1 2.3 1.2&#xA;alumina 3.6 0.7 1.5 -2.7 -1.3&#xA;alutaceus 3.6 1.2 -0.6 -1.7 0.3&#xA;angusticeps 3.8 4.6 -2.0 1.2 1.7&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have this code following code that can correlate any one of the ‘path*’ column to proteome size:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data &amp;lt;- read.csv(&quot;data.csv&quot;, row.names = 1&#xA;tree &amp;lt;- read.tree(&quot;anolis.phy&quot;)&#xA;Proteome &amp;lt;- data[, &quot;Proteome_size&quot;]&#xA;awe &amp;lt;- data[, &quot;path1&quot;]&#xA;names(Proteome_size) &amp;lt;- names(var1) &amp;lt;- rownames(data)&#xA;hPic &amp;lt;- pic(proteome_size, tree)&#xA;aPic &amp;lt;- pic(gene1, tree)&#xA;picModel &amp;lt;- lm(hPic ~ aPic - 1)&#xA;summary(picModel)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question is that, how can I implement this code to do that for all the columns simultaneously and store the statistics info as (R and p_value) for each variable somewhere ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="3646" LastEditorUserId="48" LastEditDate="2018-11-08T14:30:22.653" LastActivityDate="2020-04-07T14:01:40.210" Title="Multiple correlation with R" Tags="|r|correlation|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5405" PostTypeId="2" ParentId="5400" CreationDate="2018-11-08T13:27:42.457" Score="3" Body="&lt;p&gt;Ok. Answering my own question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did this way. And I got the output I want.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;install.packages(&quot;msigdbr&quot;)&#xA;library(msigdbr)&#xA;&#xA;m_df = msigdbr(species = &quot;Homo sapiens&quot;, category = &quot;C5&quot;)&#xA;head(m_df)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gs_name gs_id gs_cat gs_subcat human_gene_symb~ species_name entrez_gene&#xA;  &amp;lt;chr&amp;gt;   &amp;lt;chr&amp;gt; &amp;lt;chr&amp;gt;  &amp;lt;chr&amp;gt;     &amp;lt;chr&amp;gt;            &amp;lt;chr&amp;gt;              &amp;lt;int&amp;gt;&#xA;1 GO_14_~ M184~ C5     MF        AANAT            Homo sapiens          15&#xA;2 GO_14_~ M184~ C5     MF        AKT1             Homo sapiens         207&#xA;3 GO_14_~ M184~ C5     MF        ARRB2            Homo sapiens         409&#xA;4 GO_14_~ M184~ C5     MF        BAD              Homo sapiens         572&#xA;5 GO_14_~ M184~ C5     MF        DAB2IP           Homo sapiens      153090&#xA;6 GO_14_~ M184~ C5     MF        DDIT4            Homo sapiens       54541&#xA;# ... with 2 more variables: gene_symbol &amp;lt;chr&amp;gt;, sources &amp;lt;chr&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;found this solution from here &lt;a href=&quot;https://cran.r-project.org/web/packages/msigdbr/vignettes/msigdbr-intro.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cran.r-project.org/web/packages/msigdbr/vignettes/msigdbr-intro.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1369" LastEditorUserId="1369" LastEditDate="2018-11-08T13:45:13.603" LastActivityDate="2018-11-08T13:45:13.603" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5406" PostTypeId="2" ParentId="5397" CreationDate="2018-11-08T13:51:49.430" Score="2" Body="&lt;p&gt;Yes, there is a tool to do this called R2C2 by the Vollmers lab at UC Santa Cruz.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biorxiv.org/content/early/2018/06/04/338020&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/early/2018/06/04/338020&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-11-08T13:51:49.430" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5407" PostTypeId="1" AcceptedAnswerId="5408" CreationDate="2018-11-08T14:56:37.033" Score="1" ViewCount="153" Body="&lt;p&gt;I am working on macOS High Sierra.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am following the steps described &lt;a href=&quot;http://genomewiki.ucsc.edu/index.php/DoSameSpeciesLiftOver.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; to lift an annotation over from one version of a genome to another. I am now just using the example genomes provided in the tutorial. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this step:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time (doSameSpeciesLiftOver.pl -verbose=2 -buildDir=`pwd` \&#xA;      -ooc=`pwd`/&lt;span class=&quot;math-container&quot;&gt;${target}.ooc -fileServer=localhost -localTmp=&quot;/dev/shm&quot; \&#xA;  -bigClusterHub=localhost -dbHost=localhost -workhorse=localhost \&#xA;  -target2Bit=`pwd`/$&lt;/span&gt;{target}.2bit -targetSizes=`pwd`/&lt;span class=&quot;math-container&quot;&gt;${target}.chrom.sizes \&#xA;  -query2Bit=../$&lt;/span&gt;{query}/&lt;span class=&quot;math-container&quot;&gt;${query}.2bit \&#xA;  -querySizes=../$&lt;/span&gt;{query}/&lt;span class=&quot;math-container&quot;&gt;${query}.chrom.sizes $&lt;/span&gt;{target} ${query}) &amp;gt; do.log 2&amp;gt;&amp;amp;1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get the following error in the &lt;code&gt;do.log&lt;/code&gt; file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Warning: Permanently added 'localhost' (ECDSA) to the list of known hosts.&#xA;username@localhost: Permission denied (publickey,password,keyboard-interactive).&#xA;HgStepManager: executing from step 'align' through step 'cleanup'.&#xA;HgStepManager: executing step 'align' Thu Nov  8 14:27:32 2018.&#xA;Using localhost, /Users/username/Documents/projects/main/output/software/liftover/data/genomes/GCA_000004515.3_Glycine_max_v2.0/GCA_000004515.3_Glycine_max_v2.0.2bit and ../GCF_000004515.3_V1.1/GCF_000004515.3_V1.1.2bit&#xA;align: localhost does not have /Users/username/Documents/projects/main/output/software/liftover/data/genomes/GCA_000004515.3_Glycine_max_v2.0/GCA_000004515.3_Glycine_max_v2.0.ooc -- if that is not the correct location, please run again with -ooc.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think the main problem is here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;username@localhost: Permission denied (publickey,password,keyboard-interactive).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;https://stackoverflow.com/questions/1556056/permission-denied-publickey-keyboard-interactive#4398006&quot;&gt;post&lt;/a&gt; describes this issue. However:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I do not have this file: ~/.ssh/authorized_keys. My files in ~/.ssh are: &lt;code&gt;github_rsa github_rsa.pub known_hosts&lt;/code&gt;. One of the lines of &lt;code&gt;known_hosts&lt;/code&gt; is localhost. Does this presumably allow me to ssh to my own laptop? I still don't understand why I need to ssh in this script if it's run locally.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;PasswordAuthentication yes&lt;/code&gt; is already in &lt;code&gt;/etc/ssh/sshd_config&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;If I run the command without the &lt;code&gt;time&lt;/code&gt; command, I still get the same error. I am not prompted to give a password.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Could anyone help me overcome this problem, please? &lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="181" LastEditDate="2018-11-09T11:41:05.373" LastActivityDate="2018-11-09T11:41:05.373" Title="Accessing UCSC genome via ssh results in a validation error" Tags="|sequence-annotation|ucsc|errors|liftover|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5408" PostTypeId="2" ParentId="5407" CreationDate="2018-11-08T16:12:15.973" Score="2" Body="&lt;p&gt;You need to setup ssh keys.  All the commands in the script are going to use ssh to run the commands.  Note the tips here of how to setup ssh keys:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://genomewiki.ucsc.edu/index.php/Parasol_job_control_system#SSH_keys&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genomewiki.ucsc.edu/index.php/Parasol_job_control_system#SSH_keys&lt;/a&gt;&lt;a href=&quot;http://genomewiki.ucsc.edu/index.php/Parasol_job_control_system#SSH_keys&quot; rel=&quot;nofollow noreferrer&quot;&gt;ssh key setup&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3647" LastActivityDate="2018-11-08T16:12:15.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5409" PostTypeId="2" ParentId="4975" CreationDate="2018-11-08T17:27:01.387" Score="2" Body="&lt;p&gt;Check your webserver functions from the top down.  You can do this from any WEB browser, even one that is on the webserver system itself.  For example, does this link function:&#xA;  &lt;a href=&quot;http://webserver-id/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://webserver-id/&lt;/a&gt;&#xA;and provide you with a home page for that WEB service ?  If not, then there&#xA;is not apache WEB server on that system, or is mis-configured.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, depending upon how that Apache is configured, try this URL&#xA;  &lt;a href=&quot;http://webserver-id/~username/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://webserver-id/~username/&lt;/a&gt;&#xA;which would be a normal way to get to user directories via Apache.&#xA;This would actually be looking for a special directory in your HOME&#xA;directory, perhaps something like $HOME/public_html/&#xA;You would need to place something in public_html/ to access, for example&#xA;a tiny WEB page:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;html&amp;gt;&amp;lt;head&amp;gt;&amp;lt;title&amp;gt;my home directory&amp;lt;/title&amp;gt;&amp;lt;/head&amp;gt;&#xA;&amp;lt;body&amp;gt;&amp;lt;h1&amp;gt;hello world&amp;lt;/h1&amp;gt;&amp;lt;/body&amp;gt;&amp;lt;/html&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If that works, then you need to place your hub files in this&#xA;public_html/ directory.&lt;/p&gt;&#xA;" OwnerUserId="3647" LastActivityDate="2018-11-08T17:27:01.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5410" PostTypeId="2" ParentId="5368" CreationDate="2018-11-08T18:46:44.320" Score="1" Body="&lt;p&gt;UCSC 'kent' command line tool:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;faToTwoBit -bed=yourCoordinates.bed genomeSequence.2bit output.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Binary for this command and others available at: &lt;a href=&quot;http://hgdownload.soe.ucsc.edu/admin/exe/linux.x86_64/&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC kent command line tools&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3647" LastActivityDate="2018-11-08T18:46:44.320" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5411" PostTypeId="2" ParentId="5367" CreationDate="2018-11-08T20:02:07.693" Score="2" Body="&lt;p&gt;The steps you describe are correct. For step 2 it is usually normalized to mean 0 and variance 1. However the &quot;machine learning&quot; part is important. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having several samples being technical replicates will make the integration task easier. However, you have too few samples to make any good prediction. At most I would describe it as an exploratory analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend using regularized canonical correlation analysis for this kind of multi-omics analysis. The method seeks which are the features of one dataset that correlate (or covariate, depending on a parameter) with the variables of other datasets. Then you could use those features to predict the position of the other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several implementations of this methods, the original in the &lt;a href=&quot;https://cran.r-project.org/package=RGCCA&quot; rel=&quot;nofollow noreferrer&quot;&gt;RGCCA&lt;/a&gt; package, and &lt;a href=&quot;https://bioconductor.org/packages/mixOmics&quot; rel=&quot;nofollow noreferrer&quot;&gt;mixOmics&lt;/a&gt; for other procedures with cross-validations also &lt;a href=&quot;https://bioconductor.org/packages/STATegRa&quot; rel=&quot;nofollow noreferrer&quot;&gt;STATegRa&lt;/a&gt; has other related methods. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-11-08T20:02:07.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5412" PostTypeId="1" CreationDate="2018-11-09T03:55:14.463" Score="1" ViewCount="2184" Body="&lt;p&gt;I didn't find way that convert protein sequences in  FASTA to PSSM by python,&#xA;and how to write program in python to find PSSM for our data set ?&#xA;Maybe better to question how to find PSSM from NCBI alignments ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you  &lt;/p&gt;&#xA;" OwnerUserId="3649" LastEditorUserId="3649" LastEditDate="2018-11-09T10:37:49.597" LastActivityDate="2018-11-17T12:24:53.343" Title="How to get PSSM of protein by protein sequence with python?" Tags="|python|psi-blast|pssm|" AnswerCount="1" CommentCount="11" FavoriteCount="0" ClosedDate="2018-12-07T18:17:39.567" ContentLicense="CC BY-SA 4.0" />
  <row Id="5413" PostTypeId="1" AcceptedAnswerId="5415" CreationDate="2018-11-09T08:31:41.483" Score="3" ViewCount="174" Body="&lt;p&gt;I am playing with biomart in order to get the list of genes associated to a specific gene ontology (GO) term.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if I use the webservice as in this &lt;a href=&quot;http://www.ensembl.org/biomart/martview/b222cd479ae7a7ab534cc7d672a80df5&quot; rel=&quot;nofollow noreferrer&quot;&gt;query&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get at least 50 genes. If I download the file I can gen an exact count of them:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat mart_export.txt | wc -l&#xA;347&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, I can do the same operation by using the R library biomaRt. Specifically:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; ensembl &amp;lt;- useEnsembl(biomart=&quot;ENSEMBL_MART_ENSEMBL&quot;, dataset='hsapiens_gene_ensembl')&#xA;&amp;gt; gene &amp;lt;- getBM(attributes = c('external_gene_name'), filters = 'go', values = 'GO:0030098', mart = ensembl)&#xA;&amp;gt; unique(gene)&#xA;  external_gene_name&#xA;1               SPI1&#xA;2              PRKDC&#xA;3                TOX&#xA;4               LY6D&#xA;5               CBFB&#xA;6              IKZF1&#xA;7               RELB&#xA;8             FLT3LG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;By doing so, these are all the genes I retrieve. The two results are totally different and I don't know why it. All these eight genes are found in the biomart file (I grepped it, to be sure).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe this is not &lt;a href=&quot;https://www.biostars.org/p/109246/&quot; rel=&quot;nofollow noreferrer&quot;&gt;an issue related to databases versions&lt;/a&gt; since I checked:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;listMarts()&#xA;               biomart               version&#xA;1 ENSEMBL_MART_ENSEMBL      Ensembl Genes 94&#xA;2   ENSEMBL_MART_MOUSE      Mouse strains 94&#xA;3     ENSEMBL_MART_SNP  Ensembl Variation 94&#xA;4 ENSEMBL_MART_FUNCGEN Ensembl Regulation 94&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;listDatasets(ensembl)&#xA;...&#xA;38 Human genes (GRCh38.p12)&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Both are the same version used by biomart online.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is going on? Is it possible that biomart online is considering &lt;a href=&quot;http://geneontology.org/page/ontology-relations&quot; rel=&quot;nofollow noreferrer&quot;&gt;GO relations&lt;/a&gt; that are not specified in the R version?&lt;/p&gt;&#xA;" OwnerUserId="3119" LastActivityDate="2018-11-09T09:18:38.357" Title="different results coming from biomart online and biomaRt R library" Tags="|r|biomart|gene-ontology|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5415" PostTypeId="2" ParentId="5413" CreationDate="2018-11-09T09:18:38.357" Score="4" Body="&lt;p&gt;Sorry, this was my mistake in the last question. To search down the ontology, rather than just for the specific association with a term, biomaRt needs a different filter: 'go_parent_term'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gene &amp;lt;- getBM(attributes = c('external_gene_name'), filters = 'go_parent_term', values = 'GO:0030098', mart = ensembl)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2018-11-09T09:18:38.357" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5416" PostTypeId="2" ParentId="5327" CreationDate="2018-11-09T09:23:58.110" Score="9" Body="&lt;p&gt;&lt;strong&gt;Don't do it:&lt;/strong&gt; Your FASTQ file is either malformed or your FASTQ record spans more than four lines, which is allowed in FASTQ.  For a detailed description of what can go wrong in FASTQ parsing see for example &lt;a href=&quot;http://biopython.org/DIST/docs/api/Bio.SeqIO.QualityIO-module.html#FastqGeneralIterator&quot; rel=&quot;noreferrer&quot;&gt;http://biopython.org/DIST/docs/api/Bio.SeqIO.QualityIO-module.html#FastqGeneralIterator&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the FASTQ is malformed, then you should really ask yourself how this happened in the first place and fix the source of the problem. If the record is valid FASTQ, then I suggest parsing the read with for example &lt;code&gt;FastqGeneralIterator&lt;/code&gt; and dumping the parsed result back to FASTQ in a 4-line-per-record form.&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2018-11-09T09:23:58.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5418" PostTypeId="1" AcceptedAnswerId="5446" CreationDate="2018-11-11T00:36:25.973" Score="2" ViewCount="861" Body="&lt;p&gt;Using the sample data in the 2,700 PBMC clustering tutorial, the session crashes at the &lt;code&gt;ScaleData()&lt;/code&gt; step. The progress bar remains at 0% and hangs, followed by an R crash. I've tried reducing the size for number of genes to scale at in a single computation with the argument &lt;code&gt;block.size&lt;/code&gt; with no change. The Seurat S4 object is only ~70mb so I can't imagine I'm exceeding the RStudio Cloud 1gb RAM limitations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you'd like to check out the cloud instance I can add you to the workspace temporarily. There is a limit on members allowed in a workspace.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been trying to run this tutorial on another remote cluster in RStudio inside of a container (Singularity here vs. Docker in RStudio Cloud) and am hitting the same snag.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get receive no errors from RStudio Cloud, but in the other evirnment I get this error:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;ERROR session hadabend; LOGGED FROM: rstudio::core::Error {anonymous}::rInit(const rstudio::r::session::RInitInfo&amp;amp;) /home/ubuntu/rstudio/src/cpp/session/SessionMain.cpp:563&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On &lt;a href=&quot;https://github.com/Homebrew/homebrew-science/issues/5504&quot; rel=&quot;nofollow noreferrer&quot;&gt;this issue thread&lt;/a&gt; someone mentions it they receive this error message when hitting their RAM limits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The source for &lt;code&gt;ScaleData()&lt;/code&gt; can be found in the &lt;a href=&quot;https://github.com/satijalab/seurat/blob/65b77a9480281ef9ab1aa8816f7c781752092c18/R/preprocessing.R&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github repo&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="181" LastEditDate="2018-11-12T16:45:22.437" LastActivityDate="2018-11-14T18:56:58.607" Title="ScaleData() from Seurat causes crash on RStudio Cloud" Tags="|r|seurat|rstudio|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5419" PostTypeId="1" CreationDate="2018-11-11T15:47:40.733" Score="2" ViewCount="148" Body="&lt;p&gt;I am working on macOS. I am following the steps described here to lift over an annotation from the reference genome of &lt;em&gt;C. elegans&lt;/em&gt; to another assembly of &lt;em&gt;C. elegans&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have run this code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time (doSameSpeciesLiftOver.pl -verbose=2 -buildDir=`pwd` \&#xA;      -ooc=`pwd`/&lt;span class=&quot;math-container&quot;&gt;${target}.ooc -fileServer=localhost -localTmp=&quot;/dev/shm&quot; \&#xA;  -bigClusterHub=localhost -dbHost=localhost -workhorse=localhost \&#xA;  -target2Bit=`pwd`/$&lt;/span&gt;{target}.2bit -targetSizes=`pwd`/&lt;span class=&quot;math-container&quot;&gt;${target}.chrom.sizes \&#xA;  -query2Bit=../$&lt;/span&gt;{query}/&lt;span class=&quot;math-container&quot;&gt;${query}.2bit \&#xA;  -querySizes=../$&lt;/span&gt;{query}/&lt;span class=&quot;math-container&quot;&gt;${query}.chrom.sizes $&lt;/span&gt;{target} ${query}) &amp;gt; do.log 2&amp;gt;&amp;amp;1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, the &lt;code&gt;do.log&lt;/code&gt; file contains:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zsh:1: command not found: hgsql&#xA;HgStepManager: executing from step 'align' through step 'cleanup'.&#xA;HgStepManager: executing step 'align' Sun Nov 11 15:38:41 2018.&#xA;Using localhost, /Users/username/Documents/projects/main/output/software/liftover/data/genomes/GCA_000004515.3_Glycine_max_v2.0/GCA_000004515.3_Glycine_max_v2.0.2bit and ../GCF_000004515.3_V1.1/GCF_000004515.3_V1.1.2bit&#xA;# chmod a+x /Users/username/Documents/projects/main/output/software/liftover/data/genomes/GCA_000004515.3_Glycine_max_v2.0/run.blat/job.csh&#xA;# chmod a+x /Users/username/Documents/projects/main/output/software/liftover/data/genomes/GCA_000004515.3_Glycine_max_v2.0/run.blat/doAlign.csh&#xA;# ssh -x -o 'StrictHostKeyChecking = no' -o 'BatchMode = yes' localhost nice /Users/username/Documents/projects/main/output/software/liftover/data/genomes/GCA_000004515.3_Glycine_max_v2.0/run.blat/doAlign.csh&#xA;cd /Users/username/Documents/projects/main/output/software/liftover/data/genomes/GCA_000004515.3_Glycine_max_v2.0/run.blat&#xA;rm -rf tParts&#xA;/Users/username/Documents/projects/main/output/software/liftover/data/scripts/partitionSequence.pl 58018742 0 /Users/username/Documents/projects/main/output/software/liftover/data/genomes/GCA_000004515.3_Glycine_max_v2.0/GCA_000004515.3_Glycine_max_v2.0.2bit /Users/username/Documents/projects/main/output/software/liftover/data/genomes/GCA_000004515.3_Glycine_max_v2.0/GCA_000004515.3_Glycine_max_v2.0.chrom.sizes 2000 -lstDir=tParts&#xA;rm -rf qParts&#xA;/Users/username/Documents/projects/main/output/software/liftover/data/scripts/partitionSequence.pl 10000000 0 ../GCF_000004515.3_V1.1/GCF_000004515.3_V1.1.2bit ../GCF_000004515.3_V1.1/GCF_000004515.3_V1.1.chrom.sizes 1000 -lstDir=qParts&#xA;Can't open seq size file ../GCF_000004515.3_V1.1/GCF_000004515.3_V1.1.chrom.sizes: No such file or directory&#xA;Command failed:&#xA;ssh -x -o 'StrictHostKeyChecking = no' -o 'BatchMode = yes' localhost nice /Users/username/Documents/projects/main/output/software/liftover/data/genomes/GCA_000004515.3_Glycine_max_v2.0/run.blat/doAlign.csh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think the biggest problem is that the &lt;code&gt;hgsql&lt;/code&gt; command is not found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I run &lt;code&gt;hgsql&lt;/code&gt; on the command line, I find it even though I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dyld: Library not loaded: /usr/local/opt/mysql/lib/libmysqlclient.20.dylib&#xA;  Referenced from: /usr/local/bin/hgsql&#xA;  Reason: image not found&#xA;[1]    34059 abort      hgsql&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Location of the hgsql installation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;which hgsql&#xA;/Users/username/Documents/projects/main/output/software/liftover/data/bin/hgsql&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me with using hgsql, please?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2019-08-04T22:44:57.280" Title="hgsql not found when running a script despite being accessible on command-line" Tags="|sequence-annotation|software-installation|errors|liftover|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5420" PostTypeId="1" CreationDate="2018-11-11T17:03:26.543" Score="1" ViewCount="136" Body="&lt;p&gt;I'm trying to get some pointers on where to start on the analysis of my dataset. First, some basic info:&#xA;Background:&#xA;We performed Nanopore sequencing of two poolseq samples called UU and DD referring to damage vs undamaged. We were able to obtain approximately 3x coverage (this is currently more of a proof of concept than a hardcore association job, so the coverage is unfortunate but worth pursuing). Between these samples, we know that one is going to have different methylation patterns in response to a phenotypic trait response, but we don't know what regions along the genome are going to be differently methylated. The goal is to compare what's methylated (as in higher or lower levels of methylation) between the two samples, knowing there ought to be a lot that cancels out and come up with ideally a short list of significantly different regions we can associate to the plastic response.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Output of initial methylation calling data:&#xA;Using the raw signal level data against the reference genome and sorted, mapped BAM files the Nanopolish software was used to call methylation. This produces a dataset with the following headers:&#xA;1. Chromosome, 2. +/- Strand 3. Start vs. 4. End of the CG dinucleotide (the difference between these is 1 or two bases) 5. The read ID the dinucleotide came from 6. the log likelihood ratio calculated from the model embedded in the methylation calling process using 7. log likelihood that it's methylated and 8. log likelihood that it's unmethylated. 8. number of calling strands (always appears to be 1) 9. number of motifs (don't know about this) and finally 10. the sequence surrounding the dinucleotide, which is always short but variable in length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can follow the basic interpretation of the log-likelihood... if it's a positive number, there is statistical support that it's methylated. This would probably be about as much as we need, short of a p-value, if we were looking for a specific region, but we are trying to analyze the differences in de/methylation across the entire methylome. How would you suggest using the statistic values provided in our dataset to compare and visualize the significant different modifications of the dinucleotides between strands/samples?&lt;/p&gt;&#xA;" OwnerUserId="3662" LastActivityDate="2018-11-12T15:03:15.330" Title="Continuing analysis based on a log-likelihood ratio? (Comparing methylation between two PoolSeq phenotypes)" Tags="|statistics|methylation|genetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5421" PostTypeId="1" CreationDate="2018-11-11T22:40:48.483" Score="0" ViewCount="42" Body="&lt;p&gt;In Uniprot how do I know which variants are mutations and which are polymorphic?&#xA;I am looking at &lt;a href=&quot;https://www.uniprot.org/uniprot/P38398#function&quot; rel=&quot;nofollow noreferrer&quot;&gt;P38398 (BRCA1_HUMAN)&lt;/a&gt; specifically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many thanks.&lt;/p&gt;&#xA;" OwnerUserId="3663" LastEditorUserId="964" LastEditDate="2018-11-12T04:54:09.130" LastActivityDate="2019-06-11T15:23:31.580" Title="In Uniprot how do I know which variants are mutations and which are polymorphic?" Tags="|uniprot|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5422" PostTypeId="2" ParentId="4006" CreationDate="2018-11-12T11:11:19.027" Score="1" Body="&lt;p&gt;I believe it is mostly a cost issue as spike-ins are &quot;expensive&quot; to use given the large number of cells that are/can be sequenced via droplet based methods (as compared to plate-based ones). The 10x link that you provided actually states that this is possible with 10x and Klein et al. used ERCC spike-ins in 953 droplets (Cell. 2015 May 21;161(5):1187-1201. doi: 10.1016/j.cell.2015.04.044.). &lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2018-11-12T11:11:19.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5423" PostTypeId="2" ParentId="5412" CreationDate="2018-11-12T12:20:46.627" Score="1" Body="&lt;p&gt;We solve our problem and create gitHub repository (&lt;a href=&quot;https://github.com/MsAlEhR/bio-protein-recog&quot; rel=&quot;nofollow noreferrer&quot;&gt;bio-protein&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="3649" LastEditorUserId="48" LastEditDate="2018-11-17T12:24:53.343" LastActivityDate="2018-11-17T12:24:53.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5424" PostTypeId="2" ParentId="5420" CreationDate="2018-11-12T15:03:15.330" Score="2" Body="&lt;p&gt;I assume you don't want to spend 1-2 years developing your own method based on nanopore data, so the path forward is to convert the output into a format used by standard DMR calling programs. There's no standard format for that, annoyingly, but the general things you'll need are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Coordinates of a C/CpG with coverage&lt;/li&gt;&#xA;&lt;li&gt;Number of reads supporting methylation at that site.&lt;/li&gt;&#xA;&lt;li&gt;Number of reads not supporting methylation at that site.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Typically you would do some filtering before/after making such a table so you only have decent quality calls (that would translate into some reasonable log likelihood for your data) and possibly only sites with decent (5x, 10x, 30x, etc.) coverage. You'd then feed that for all of the replicates in all of your groups into programs like &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/bsseq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bsseq&lt;/a&gt;, &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/DSS.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DSS&lt;/a&gt; or &lt;a href=&quot;https://www.bioinf.uni-leipzig.de/Software/metilene/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Metilene&lt;/a&gt; (to not so randomly select 3 examples).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there aren't DMR finding programs already developed specifically for nanopore data I expect someone is developing it. The technology has a higher error rate, but given that bisulfite conversion as required in Illumina data also leads to a higher error rate that's not necessarily a huge issue if the methylation calling itself is reasonably accurate and the depth is high enough. Note that you probably need 10X coverage as a bare minimum unless your effect size is very very large.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-11-12T15:03:15.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5425" PostTypeId="2" ParentId="5404" CreationDate="2018-11-12T18:04:24.063" Score="1" Body="&lt;p&gt;One option is to use one of the &lt;code&gt;apply&lt;/code&gt; family functions. If you enclose your code above (which generates errors by the way) in a function, you can then &quot;apply&quot; this function on the rows or columns of your data with something like &lt;code&gt;apply(X, MARGIN, FUN, …)&lt;/code&gt;, the margin being 1 for rows or 2 for columns (as in your case).&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2018-11-12T18:04:24.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5426" PostTypeId="1" CreationDate="2018-11-12T18:22:07.913" Score="2" ViewCount="222" Body="&lt;p&gt;I have been given the task of assembling a 'new' Ecoli genome and analysing the genes present etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Ecoli is a new strain, and has been taken and run on a Nextseq 500 in high-output mode with 150bp paired end reads. The 'raw' files that I have is the forward and reverse reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have initially QC checked the 'raw' files, and subsequently run them through trim galore and checked the QC after that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the next step, I now need to assemble my genome. I have been told that SPades will run a 'de novo' assembly for me, and then put that assembly into Prokka for Gene annotation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this the best way to assemble the genome and annotate it? Or should I use another method? I am thinking that I should use a 'mapping' technique to assemble the genome using the Ecoli O157:H7 genome as a reference, but I have no idea how to do this. I would say that I am at an intermediate level with unix, but by no means am I a bioinformatician. Some help and guidance would be greatly appreciated!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alex.&lt;/p&gt;&#xA;" OwnerUserId="3668" LastActivityDate="2018-11-14T21:50:58.357" Title="E.coli Sequencing &amp; Analysis" Tags="|ngs|assembly|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5427" PostTypeId="1" AcceptedAnswerId="5485" CreationDate="2018-11-12T23:26:40.987" Score="9" ViewCount="2611" Body="&lt;p&gt;EDIT: I do not want to make any modifications to the mapped reads, I simply want to ignore one read in a read pair if they overlap the same region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used samtools depth to calculate the depth of coverage for samples in the whole Exome region using a GRCh37_ref.bed. These samples are sorted and duplicate marked. I ran this calculation on a few hundred samples to determine how much more sequencing needed to be done and found something interesting. Some samples had 100X coverage at 96,000,000 mapped reads and some samples had 100X coverage at 83,000,000 mapped reads. The samples that had 100X coverage at 83,000,000 reads had read pairs overlapping certain regions of the bedfile (read 1 was covering the same coordinates as read2 to some extent). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then found out that samtools depth double counts these overlapped regions even though they are technically from the same molecule in sequencing and would be a duplicate bases in the read. See the image for an example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7dkOV.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7dkOV.png&quot; alt=&quot;Duplicate Region&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each base in this Duplicate Overlap region is double counted when it shouldn't be. How do you guys deal with this duplicate overlap region when calculating coverage across a bam file?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;samtools (v1.7):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools depth -a -b exome_targets_GRCh37.bed sample1.bam &amp;gt; sample1_depth.txt&#xA;awk '{s+=$3}END{print s/NR}' sample1_depth.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3669" LastEditorUserId="3669" LastEditDate="2018-11-20T19:50:32.767" LastActivityDate="2018-12-11T18:16:36.957" Title="Double-counting coverage of overlapped read pairs" Tags="|samtools|coverage|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5428" PostTypeId="1" CreationDate="2018-11-13T04:45:35.057" Score="4" ViewCount="251" Body="&lt;p&gt;In case of Drop-seq, we have paired end data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Read 1: Cell code + UMI (unique molecule identifier)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Read 2: The transcript information&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I have a problem/doubt with the sample I am working on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sample I am using is the &lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR6261587&quot; rel=&quot;nofollow noreferrer&quot;&gt;following&lt;/a&gt; (Check the &quot;Reads&quot; tab).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you know the Drop-seq is &quot;paired-end&quot;, we are expected to see two reads for a spot. Although this sample say paired-end, it has only one read per spot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example I can share a link of a different scRNA-seq data where you can properly see two reads for a spot&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR8086553&quot; rel=&quot;nofollow noreferrer&quot;&gt;Example sample&lt;/a&gt;, you need to check the &quot;Reads&quot; tab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where I am going wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I asked one of the main authors of the paper. The following is the reply I got :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I recommend that you download the aligned BAM files that are hosted in&#xA;  the same GEO record. Read 1 is already processed into the cell and UMI&#xA;  barcodes and held as custom tags (XC and XM) in the BAM files. The&#xA;  cells are already barcode-corrected, so if you use those files, your&#xA;  cell barcodes will line up with mine; if you start from FASTQs, they&#xA;  will not. For most aligners, you can just use the BAM file as input to&#xA;  realign. (It has all reads, even those that did not align.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But I could not find any &quot;XM&quot; or &quot;XC&quot; keywords in the bam file :(&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To understand his reply you have to be familiar the &lt;a href=&quot;https://github.com/broadinstitute/Drop-seq/files/2425535/Drop-seqAlignmentCookbookv1.2Jan2016.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;processing steps of the Drop-seq&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looks like they have submitted some kind of processed data. I could not figure out how much the data is processed. I am trying to use the data starting at different processing steps but I could not figure out how much the data is processed.&lt;/p&gt;&#xA;" OwnerUserId="16709" OwnerDisplayName="jayesh kumar" LastEditorUserId="48" LastEditDate="2018-11-13T10:10:48.903" LastActivityDate="2019-07-29T23:19:55.283" Title="Problem: &quot;Pair-end&quot; reads scRNA seq data (Drop-seq)" Tags="|scrnaseq|read-mapping|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5429" PostTypeId="1" AcceptedAnswerId="5433" CreationDate="2018-11-13T10:00:16.887" Score="2" ViewCount="332" Body="&lt;p&gt;I’ve run the cellranger analysis pipeline on single cell RNASeq datasets. I can import the matrix and graph-based clusters into R. Doing this I can optimise the dimension reduction and plot cells with coloured by clusters generated by cellranger. I wish to optimise the graph-based clustering as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to either:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;obtain the graph generated by cellranger from the files generated. (I can only find the final clusters in the analysis output)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;reproduce generating a similar graph (used for clustering) from the output matrix (preferably with R or Python.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;With either approach, I am looking for a list of edges for nearest neighbors in the gene-barcode UMI matrix to use as input for a clustering algorithm.&lt;/p&gt;&#xA;" OwnerUserId="1855" LastEditorUserId="1855" LastEditDate="2018-11-14T09:23:34.520" LastActivityDate="2018-12-15T03:04:56.197" Title="Can I get the graph generated by cellranger" Tags="|scrnaseq|single-cell|clustering|10x-genomics|cellranger|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="5430" PostTypeId="1" AcceptedAnswerId="5431" CreationDate="2018-11-13T10:04:37.413" Score="4" ViewCount="227" Body="&lt;p&gt;SAMtools returned an error message for my gzipped genome FASTA:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Indexed block-compressed FASTA file cannot be handled&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The source code for the message is &lt;a href=&quot;https://github.com/samtools/htsjdk/blob/master/src/main/java/htsjdk/samtools/reference/IndexedFastaSequenceFile.java&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;// check if the it is a valid block-compressed file&#xA;if (IOUtil.isBlockCompressed(path, true)) {&#xA;    throw new SAMException(&quot;Indexed block-compressed FASTA file cannot be handled: &quot; + path);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What's a &quot;block-compressed&quot; file?&lt;/p&gt;&#xA;" OwnerUserId="174" LastActivityDate="2018-11-13T10:48:02.000" Title="What is &quot;block-compressed&quot; file in samtools?" Tags="|sam|samtools|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5431" PostTypeId="2" ParentId="5430" CreationDate="2018-11-13T10:48:02.000" Score="7" Body="&lt;p&gt;A block compression usually refers to compressing your file into a series of small blocks (with a tool like bgzip). This allows indexing in that the index can record which record lives in which block so that the whole file does not need to be decompressed and therefore can be accessed randomly. This isn't true of files compressed as a single block using gzip. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I was aware samtools only works with uncompressed FASTA files. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-11-13T10:48:02.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5432" PostTypeId="1" AcceptedAnswerId="5460" CreationDate="2018-11-13T15:12:18.353" Score="3" ViewCount="2153" Body="&lt;p&gt;I am interested in using Seurat to compare wild type vs Mutant. I don't know how to use the package. How can I test whether mutant mice, that have deleted gene, cluster together?&lt;/p&gt;&#xA;" OwnerUserId="3171" LastEditorUserId="57" LastEditDate="2018-11-23T18:47:14.167" LastActivityDate="2018-11-23T18:47:14.167" Title="Using Seurat to compare mutant vs.wt" Tags="|r|scrnaseq|seurat|single-cell|cellranger|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5433" PostTypeId="2" ParentId="5429" CreationDate="2018-11-13T22:54:35.590" Score="2" Body="&lt;p&gt;&lt;strong&gt;Cell Ranger&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can't download the tSNE coordinates for cells directly from the &lt;em&gt;Analysis&lt;/em&gt; tab of the fancy, polished .html document that Cell Ranger produces. If you have access to the machine on which the pipeline was run, you can grab the intermediate files. Everything is &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/output/analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;well documented&lt;/a&gt; by 10X. They explain defaults etc. on that page. The tSNE coordinates are available at:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;analysis/tsne/2_components/projection.csv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the contents look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ head -5 analysis/tsne/2_components/projection.csv&#xA;Barcode,TSNE-1,TSNE-2&#xA;AAACATACAACGAA-1,-13.5494,1.4674&#xA;AAACATACTACGCA-1,-2.7325,-10.6347&#xA;AAACCGTGTCTCGC-1,12.9590,-1.6369&#xA;AAACGCACAACCAC-1,-9.3585,-6.7300&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Seurat&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And the second point: Cell Ranger is replicating the Seurat workflow. Seurat uses the Louvain method for community detection by default. Check out the &lt;a href=&quot;https://satijalab.org/seurat/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;intro tutorial&lt;/a&gt; and see if you can follow along. This tutorial works with 10X data, in fact the first function call in the tutorial reads in the 10X matrices. If you are already plotting in R you'll speed through this tutorial. Seurat is great to work with, you'll have tSNEs and clusters in no time. And there's so much more you can do from there!&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="181" LastEditDate="2018-11-14T21:56:16.010" LastActivityDate="2018-11-14T21:56:16.010" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="5435" PostTypeId="1" AcceptedAnswerId="5440" CreationDate="2018-11-14T00:44:13.430" Score="3" ViewCount="7150" Body="&lt;p&gt;I have some problems in creating a .bed file for hg19, so I will be able to visualize the .bed file in IGV.&lt;/p&gt;&#xA;&lt;p&gt;The .fasta file contains rows of this form:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;chr1:0-1000&#xA;NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN&#xA;&amp;gt;chr1:1000-2000&#xA;NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN&#xA;&amp;gt;chr1:2000-3000&#xA;NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN&#xA;&amp;gt;chr1:3000-4000&#xA;NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried both with biopython:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;with open(f1) as in_f, open(f1+'.seq','w') as out_f:&#xA;    for record in SeqIO.parse(in_f, 'fasta'):&#xA;        out_f.write('{}\t0\t{}\n'.format(record.id, len(record))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and awk:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools faidx file.fasta&#xA;awk 'BEGIN {FS=&amp;quot;\t&amp;quot;}; {print &lt;span class=&quot;math-container&quot;&gt;$1 FS &quot;0&quot; FS $&lt;/span&gt;2}' file.fasta.fai  &amp;gt; file.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Both ways gave me the same output which IGV couldn't work with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1:0-1000 0   1000&#xA;chr1:1000-2000  0   1000&#xA;chr1:2000-3000  0   1000&#xA;chr1:3000-4000  0   1000&#xA;chr1:4000-5000  0   1000&#xA;chr1:5000-6000  0   1000&#xA;chr1:6000-7000  0   1000&#xA;chr1:7000-8000  0   1000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now I tried to see what would be a valid .bed file format from &lt;a href=&quot;http://genome.ucsc.edu/cgi-bin/hgTables&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC&lt;/a&gt; and the format looks different. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#bin    chrom   chromStart  chromEnd&#xA;13  chr1    39845715    39846245&#xA;19  chr1    87030756    87031903&#xA;22  chr1    111148649   111149175&#xA;22  chr1    113246065   113246628&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why my methods don't output the .bed in the same format?&lt;/li&gt;&#xA;&lt;li&gt;How can I make the awk/biopython methods to output a .bed file that IGV can actually load.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/H853Y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/H853Y.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2021-02-07T11:49:23.287" LastActivityDate="2021-09-02T15:58:53.233" Title="How to create a .bed file from .fasta?" Tags="|fasta|samtools|bed|bedtools|igv|" AnswerCount="5" CommentCount="1" ClosedDate="2019-07-16T14:09:08.820" ContentLicense="CC BY-SA 4.0" />
  <row Id="5436" PostTypeId="1" AcceptedAnswerId="5451" CreationDate="2018-11-14T00:45:19.033" Score="4" ViewCount="118" Body="&lt;p&gt;I had a set of genes for honey bees. I found the homologues of those genes in the house fly and created a network of the honey bee genes and a network of the homologous fly genes using STRING. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that I could do a topological comparison between the networks and check if the same set of genes give the same hubs when in two different species. However, I can't find a single study where something similar has been done. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What biological inference can I draw from the topological parameters? For example, if the diameter of the fly network is smaller than the honey bee network, what inference could I draw?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE: It would be great if atleast some publications could be given where network comparisons have been done between different species.&lt;/p&gt;&#xA;" OwnerUserId="976" LastEditorUserId="976" LastEditDate="2018-11-14T16:18:14.743" LastActivityDate="2018-12-17T13:03:45.760" Title="Compare two networks of the same genes between two species" Tags="|networks|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5437" PostTypeId="2" ParentId="5435" CreationDate="2018-11-14T00:57:28.693" Score="3" Body="&lt;p&gt;The UCSC format you linked to isn't a BED file, your method should never produce it. What you posted as your desired output is also not a BED file. Below is a BED file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1    0       1000&#xA;chr1    1000    2000&#xA;chr1    2000    3000&#xA;chr1    3000    4000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;awk and biopython were producing the appropriate thing already. You just need to zoom in on a region in IGV. Since it's not a bigBed file, IGV won't show everything at all zoom levels.&lt;/p&gt;&#xA;&lt;p&gt;As a demonstration, below is the BED file from above in IGV:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8ttlm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8ttlm.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="734" LastEditDate="2021-09-02T15:58:53.233" LastActivityDate="2021-09-02T15:58:53.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5438" PostTypeId="2" ParentId="5435" CreationDate="2018-11-14T01:15:41.003" Score="3" Body="&lt;p&gt;If your headers are &lt;strong&gt;always&lt;/strong&gt; of the form &lt;code&gt;&amp;gt;chrZ:a-b&lt;/code&gt;, where &lt;code&gt;chrZ&lt;/code&gt; is a chromosome name, &lt;code&gt;a&lt;/code&gt; is a zero-indexed, half-open start position, and &lt;code&gt;b&lt;/code&gt; is a zero-indexed, half-open stop position:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk '($&lt;/span&gt;0 ~ /^&amp;gt;/){ print substr($0,2) }' in.fa | sed 's/[:|-]/\t/g' &amp;gt; out.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If your headers contain coordinates using a different index, then you can make adjustments to &lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$1&lt;/code&gt; and &lt;code&gt;$&lt;/span&gt;2&lt;/code&gt; in the &lt;code&gt;awk&lt;/code&gt; statement to create zero-indexed, half-open coordinates to export to BED, depending on the initial index.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2018-11-14T01:18:18.107" LastActivityDate="2018-11-14T01:18:18.107" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5439" PostTypeId="1" CreationDate="2018-11-14T04:27:30.913" Score="1" ViewCount="222" Body="&lt;p&gt;I tried to compile HMMER v3.2.1 in Cygwin using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./configure&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;followed by &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;make&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;make install&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but failed. I got this &lt;a href=&quot;https://drive.google.com/file/d/1aMXNQOiftf09XHW8B0pXVbt-Fdk1CWtD/view&quot; rel=&quot;nofollow noreferrer&quot;&gt;config.log file&lt;/a&gt; to troubleshoot and I suspect the solution is in this line: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;configure:2908: error: no acceptable C compiler found in $PATH&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What does it mean? Is it really all I need to address? I followed the installation steps &lt;a href=&quot;http://eddylab.org/software/hmmer/Userguide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;in this document&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3648" LastEditorUserId="48" LastEditDate="2018-11-14T08:02:51.110" LastActivityDate="2018-11-15T14:53:00.770" Title="Error compiling HMMER v3.2.1" Tags="|software-installation|hmmer|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5440" PostTypeId="2" ParentId="5435" CreationDate="2018-11-14T04:33:58.760" Score="5" Body="&lt;p&gt;Here is a fun little python script I cooked up for the occasion. It will take a standard fasta file as a command-line argument and turn it into the proper bed format. Using your example fasta:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;import sys&#xA;&#xA;inFasta = sys.argv[1] # take fasta as command argument&#xA;&#xA;def fastaParser(fasta):&#xA;    headers = []&#xA;    with open(fasta) as f:&#xA;        header = None&#xA;        for line in f:&#xA;            if line.startswith('&amp;gt;'): # identifies fasta header line&#xA;                headers.append(line[1:-1]) # append all of the line that isnt &amp;gt;&#xA;                header = line[1:] # in reset header&#xA;    newHeader = (header.replace(':',',') for header in headers) # format to be accepted later&#xA;    newnewHeader = (header.replace('-',',') for header in newHeader) # format to accept later&#xA;    bedHead = (header.split(',') for header in newnewHeader) # separate by comma from format above&#xA;    return bedHead&#xA;&#xA;fastaParsed = fastaParser(inFasta) # run the function&#xA;&#xA;headers = list(fastaParsed) # go from generator to list&#xA;bedfile = pd.DataFrame(headers) # create dataframe which will output bedfile&#xA;&#xA;bedfile.to_csv('fastabedfile.bed', sep='\t', index=False, header=None) # output bedfile format&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[dkennetz@nodecn202  fasta_parser]$python fastaParser.py fakeFasta.fa&#xA;&#xA;[dkennetz@nodecn202  fasta_parser]$more fastabedfile.bed&#xA;chr1    0       1000&#xA;chr1    1000    2000&#xA;chr1    2000    3000&#xA;chr1    3000    4000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3669" LastActivityDate="2018-11-14T04:33:58.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5441" PostTypeId="2" ParentId="5435" CreationDate="2018-11-14T05:57:50.240" Score="2" Body="&lt;p&gt;Assuming that you have a reference genome sequence file (e.g., &lt;code&gt;reference.fasta&lt;/code&gt;) available. Getting this kind of file is straightforward.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Index the reference genome and map your reads or FASTA sequences to it (for example with bowtie2)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# index reference genome (should be precomputed)&#xA;bowtie2-build reference.fasta reference&#xA;# map reads&#xA;bowtie2 -x  reference -U file.fasta -S file.sam&#xA;# compress SAM to a BAM (binary) file&#xA;samtools view -bS file.sam &amp;gt; file.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;2) Create a BED files from the BAM file using &lt;code&gt;bedtools&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# extract BED file&#xA;bedtools bamtobed -i file.bam &amp;gt; file.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Alternatively, for sliding windows you can generate these from a reference sequence provided that you know the length of each chromosome (perhaps there is a way to extract these directly from &lt;code&gt;reference.fasta&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# length per chromosome&#xA;samtools view -H file.bam | grep &quot;SQ&quot; | cut -d&quot;:&quot; -f2-3 | sed 's/LN://' &amp;gt; file.chr.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Create sliding windows (using only the lengths per chromosome)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# create sliding windows&#xA;bedtools makewindows -g file.chr.txt -w 1000 &amp;gt; windows.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1855" LastActivityDate="2018-11-14T05:57:50.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5442" PostTypeId="2" ParentId="5435" CreationDate="2018-11-14T10:04:44.823" Score="4" Body="&lt;p&gt;I'm assuming the FASTA header contains all the information you want. You can easily generate a proper bed file with Biopython:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;for record in SeqIO.parse('test.fa', 'fasta'):&#xA;    lst = record.id.split(':')&#xA;    chr_, (start, end) = lst[0], lst[1].split('-')&#xA;    print('\t'.join((chr_, start, end)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that native Python code is only marginally longer:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;with open('test.fa') as f:&#xA;    for line in f:&#xA;        if line.startswith('&amp;gt;'):&#xA;            lst = line.rstrip().split(':')&#xA;            chr_ = lst[0].lstrip('&amp;gt;')&#xA;            start, end = lst[1].split('-')&#xA;            print('\t'.join((chr_, start, end)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Both code snippets output bed format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    0   1000&#xA;chr1    1000    2000&#xA;chr1    2000    3000&#xA;chr1    3000    4000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastEditorUserId="104" LastEditDate="2018-11-14T13:40:11.650" LastActivityDate="2018-11-14T13:40:11.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5443" PostTypeId="2" ParentId="5404" CreationDate="2018-11-14T11:36:28.890" Score="1" Body="&lt;p&gt;Like haci mentioned, you can use the apply family of functions to loop over all your columns.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;apply(data,2, function(path_name){ summary(lm(data&lt;span class=&quot;math-container&quot;&gt;$Proteome_size ~ path_name -1))$&lt;/span&gt;adj.r.squared })    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;this gives you the adjusted r square value for your linear model, and &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;apply(data,2, function(path_name){ summary(lm(data&lt;span class=&quot;math-container&quot;&gt;$Proteome_size ~ path_name -1))$&lt;/span&gt;coefficients[,4] })    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;would give you the p-values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What happens here is apply takes your matrix and chops it up into pieces(a piece is a column here because I gave the option 2) and each pieces is fed into the function as 'path_name' and the output for all the pieces is given back to you. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might get a warning about getting a perfect fit because I am correlating 'Proteome_size' against itself. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awe &amp;lt;- data[, &quot;path1&quot;]&#xA;awe &amp;lt;- data$path1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;are the same when dealing with columns and the second can be more convenient because you can search for autocomplete by hitting tab after typing the $&lt;/p&gt;&#xA;" OwnerUserId="3680" LastActivityDate="2018-11-14T11:36:28.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5444" PostTypeId="1" CreationDate="2018-11-14T13:49:02.130" Score="3" ViewCount="151" Body="&lt;p&gt;I want to paste RSIDs in CADD output as CADD does not give RSID column in its output. For this purpose I am using bedtools intersect to compare two files and have RSID column in my CADD file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the command I am using.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools intersect -a CADD.bed -b Input.vcf| cut -f 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But this command is giving me error&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ERROR: file CADD.bed has non positional records, which are only valid for the groupBy tool.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there any way to solve this error or any other way to get RSID column in CADD output file.&lt;/p&gt;&#xA;" OwnerUserId="1247" LastEditorUserId="96" LastEditDate="2019-01-15T14:23:37.233" LastActivityDate="2019-02-15T10:16:36.767" Title="How to paste RSIDs in CADD output" Tags="|vcf|bedtools|variation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5445" PostTypeId="2" ParentId="5432" CreationDate="2018-11-14T16:37:10.547" Score="2" Body="&lt;p&gt;There is no single ended answer to your question as everything will depend on your data. For example: Is this unsupervised data where you have to browse to 100 cell types for each sample or did you FACS sort on specific markers and are you looking for specific differences between 2 specific cell types etc...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below you can find 4 recent packages that assist in your question. I strongly suggest you read the preprint of each, play around with your data and then follow the suggestion in the comments and ask more specific questions about how to proceed correctly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to use Seurat you can start here:&#xA;&lt;strong&gt;Seurat 3.0 pre-release to integrate your data sets:&lt;/strong&gt; &#xA;&lt;a href=&quot;https://satijalab.org/seurat/pancreas_integration_label_transfer.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/pancreas_integration_label_transfer.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Seurat 2 tutorial on CCA to compare different conditions&lt;/strong&gt;&#xA;you can use these methods on your integrated data sets.&#xA;&lt;a href=&quot;https://satijalab.org/seurat/immune_alignment.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/immune_alignment.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;LIGER&lt;/strong&gt; Package and preprint can be found here&#xA;&lt;a href=&quot;https://macoskolab.github.io/liger/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://macoskolab.github.io/liger/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;HARMONY&lt;/strong&gt;&#xA;&lt;a href=&quot;https://www.biorxiv.org/content/early/2018/11/05/461954&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/early/2018/11/05/461954&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conos&lt;/strong&gt;&#xA;&lt;a href=&quot;https://github.com/hms-dbmi/conos&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/hms-dbmi/conos&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2675" LastActivityDate="2018-11-14T16:37:10.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5446" PostTypeId="2" ParentId="5418" CreationDate="2018-11-14T18:56:58.607" Score="2" Body="&lt;p&gt;It seems RStudio server doesn't know it's limitations. Currently, Projects are limited to 1GB of RAM. Using &lt;a href=&quot;https://github.com/krlmlr/ulimit&quot; rel=&quot;nofollow noreferrer&quot;&gt;the R package&lt;/a&gt; &lt;code&gt;ulimit&lt;/code&gt; which mirrors the function of &lt;code&gt;ulimit&lt;/code&gt; and allows implementation of memory limits in linux I was able to set my own limitations within the R session. After setting the limit to 1GB I see the &lt;code&gt;ulimit&lt;/code&gt; limitation being tripped when invoking &lt;code&gt;ScaleData()&lt;/code&gt;. Looks like a case of exceeding available system resources.&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2018-11-14T18:56:58.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5448" PostTypeId="2" ParentId="5426" CreationDate="2018-11-14T21:50:58.357" Score="1" Body="&lt;p&gt;A de novo assembly with spades should work as well as a mapping with bowtie2.&lt;/p&gt;&#xA;&lt;p&gt;While performing a de novo assembly you should also be aware of possible contamination which you may need to remove. By using a mapping this may not be necessary.&lt;/p&gt;&#xA;&lt;p&gt;When you use a mapping, you must be aware that you may miss relevant and diverged regions because of the mapping to a reference strain. So a mappping would be preferrable if your target genome is closely related to your reference genome.&lt;/p&gt;&#xA;&lt;p&gt;I'd like to give you the advice to use some kind of distribution system like &lt;a href=&quot;https://anaconda.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Anaconda&lt;/a&gt; which helps you to keep your system clean and make it easier to reproduce your steps.&lt;/p&gt;&#xA;&lt;p&gt;Here are some possible steps for both ways. I am using an e.coli k12 sample, sra number SRR6940089 :&lt;/p&gt;&#xA;&lt;h1&gt;De novo assembly&lt;/h1&gt;&#xA;&lt;h2&gt;1. Quality Check ( you already did this )&lt;/h2&gt;&#xA;&lt;h2&gt;2. Run Spades&lt;/h2&gt;&#xA;&lt;pre&gt;&lt;code&gt;spades.py -o k12 -m 8 -t 4 --pe1-1 SRR6940089_1.fastq --pe1-2 SRR6940089_2.fastq &amp;amp;&amp;gt; k12.log&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This command will start an assembly using 4 threads and 8 GB RAM. You can scale it to whatever your system is capable of or what you'd like.&#xA;You now have a folder called &lt;code&gt;k12&lt;/code&gt; which holds the assembly information.&#xA;To get some information about the contigs you can use the &lt;a href=&quot;http://emboss.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;emboss&lt;/a&gt; toolkit.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;infoseq k12/contigs.fasta&#xA;&#xA;fasta::contigs.fasta:NODE_859_length_78_cov_1.000000 -              NODE_859_length_78_cov_1.000000 -              N    78     43.59                      &#xA;fasta::contigs.fasta:NODE_860_length_78_cov_1.000000 -              NODE_860_length_78_cov_1.000000 -              N    78     44.87                      &#xA;fasta::contigs.fasta:NODE_861_length_78_cov_1.000000 -              NODE_861_length_78_cov_1.000000 -              N    78     43.59  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the end of the contigs.fasta. So you see contigs.fasta contains 861 contigs. And you also see thate ther length and coverage is quiet low. So you need to remove this low coverage regions.&lt;/p&gt;&#xA;&lt;h2&gt;3. Remove low quality contigs&lt;/h2&gt;&#xA;&lt;p&gt;There are multiple ways how you could achieve this. I'd like to filter sequence smaller than 500 bp. the tricky part is to identify the coverage threshold. Contigs with a low coverage could be some artifacts or contamination.&lt;/p&gt;&#xA;&lt;h2&gt;4. Check for possible contamination&lt;/h2&gt;&#xA;&lt;p&gt;So to check for possible contamination you can use &lt;a href=&quot;https://ccb.jhu.edu/software/kraken2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;kraken2&lt;/a&gt;. But it needs some disk space. I am using here the minikraken database&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;kraken2 --db minikraken --threads 4 --output ecoli.kraken --report ecoli_kraken.report k12/contigs.fasta &amp;amp;&amp;gt; kraken.log&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;ecoli_kraken.report&lt;/code&gt; shows an overview of how the contigs could be classified to availible species from the database.&lt;/p&gt;&#xA;&lt;p&gt;Based on this results you can also remove some contigs.&lt;/p&gt;&#xA;&lt;h2&gt;5. Run Prokka&lt;/h2&gt;&#xA;&lt;p&gt;Now after you removed unwanted contigs, you can run &lt;a href=&quot;http://www.vicbioinformatics.com/software.prokka.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;prokka&lt;/a&gt; to get the annotation.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;prokka --cpus 4 --outdir prokka k12/contigs.fasta &amp;amp;&amp;gt; prokka.log&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And you will find the information in the prokka folder&lt;/p&gt;&#xA;&lt;h1&gt;Mapping&lt;/h1&gt;&#xA;&lt;p&gt;The mapping process is a little bit easier. You can use &lt;a href=&quot;http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;bowtie2&lt;/a&gt; and &lt;a href=&quot;http://www.htslib.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bowtie2 -x ecoli -1 SRR6940089_1.fastq -2 SRR6940089_2.fastq -p 4 | samtools view -@ 4 -O BAM -o ecoli.bam&#xA;&#xA;158216 reads; of these:&#xA;  158216 (100.00%) were paired; of these:&#xA;    35696 (22.56%) aligned concordantly 0 times&#xA;    120047 (75.88%) aligned concordantly exactly 1 time&#xA;    2473 (1.56%) aligned concordantly &amp;gt;1 times&#xA;    ----&#xA;    35696 pairs aligned concordantly 0 times; of these:&#xA;      4458 (12.49%) aligned discordantly 1 time&#xA;    ----&#xA;    31238 pairs aligned 0 times concordantly or discordantly; of these:&#xA;      62476 mates make up the pairs; of these:&#xA;        56512 (90.45%) aligned 0 times&#xA;        5365 (8.59%) aligned exactly 1 time&#xA;        599 (0.96%) aligned &amp;gt;1 times&#xA;82.14% overall alignment rate&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You now also get some information about the similarity to your reference sequence.&#xA;Now you need to work a little bit more on the BAM file. You need to sort and index this.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools sort ecoli.bam -o ecoli_sort.bam&#xA;samtools index ecoli_sort.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now you can create a consensus sequence using &lt;a href=&quot;https://samtools.github.io/bcftools/bcftools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bcftools&lt;/a&gt; and seqtk&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools mpileup -f sequence.fasta ecoli_sorted.bam | bcftools call -c | vcfutils.pl vcf2fq | seqtk seq -aQ64 -q 20 -n N &amp;gt; ecoli_consensus.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This gives you now a sequence where unknown bases are marked as N.&lt;/p&gt;&#xA;&lt;p&gt;As I wrote before, using anaconda would make the installation a lot easier. So you can write the following lines into the file &lt;code&gt;bio.yaml&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;name: bio&#xA;channels:&#xA;  - bioconda&#xA;  - defaults&#xA;dependencies:&#xA;  - aragorn=1.2.38=h470a237_2&#xA;  - barrnap=0.9=2&#xA;  - bcftools=1.9=h4da6232_0&#xA;  - bedtools=2.27.1=he941832_2&#xA;  - blast=2.6.0=boost1.60_0&#xA;  - bowtie2=2.3.4.3=py36h2d50403_0&#xA;  - emboss=6.6.0=h6debe1e_0&#xA;  - hmmer=3.2.1=hfc679d8_0&#xA;  - infernal=1.1.2=h470a237_1&#xA;  - kraken2=2.0.7_beta=pl526h2d50403_0&#xA;  - libdeflate=1.0=h470a237_0&#xA;  - libidn=7.45.0=2&#xA;  - minced=0.3.2=0&#xA;  - parallel=20160622=1&#xA;  - perl-app-cpanminus=1.7044=pl526_1&#xA;  - perl-bioperl=1.6.924=4&#xA;  - perl-carp=1.38=pl526_1&#xA;  - perl-constant=1.33=pl526_1&#xA;  - perl-data-dumper=2.161=pl526_2&#xA;  - perl-encode=2.88=pl526_1&#xA;  - perl-exporter=5.72=pl526_1&#xA;  - perl-extutils-makemaker=7.34=pl526_2&#xA;  - perl-file-path=2.15=pl526_0&#xA;  - perl-file-temp=0.2304=pl526_2&#xA;  - perl-parent=0.236=pl526_1&#xA;  - perl-threaded=5.22.0=13&#xA;  - perl-xml-namespacesupport=1.12=pl526_0&#xA;  - perl-xml-parser=2.44=pl526h3a4f0e9_6&#xA;  - perl-xml-sax=1.00=pl526_0&#xA;  - perl-xml-sax-base=1.09=pl526_0&#xA;  - perl-xml-sax-expat=0.51=pl526_2&#xA;  - perl-xml-simple=2.25=pl526_0&#xA;  - perl-yaml=1.27=pl526_0&#xA;  - prodigal=2.6.3=1&#xA;  - prokka=1.12=3&#xA;  - samtools=1.9=h8ee4bcc_1&#xA;  - spades=3.12.0=1&#xA;  - tbl2asn=25.3=0&#xA;  - boost=1.60.0=py36_0&#xA;  - bzip2=1.0.6=h14c3975_5&#xA;  - ca-certificates=2018.03.07=0&#xA;  - certifi=2018.10.15=py36_0&#xA;  - curl=7.62.0=hbc83047_0&#xA;  - expat=2.2.6=he6710b0_0&#xA;  - fontconfig=2.13.0=h9420a91_0&#xA;  - freetype=2.9.1=h8a8886c_1&#xA;  - giflib=5.1.4=h14c3975_1&#xA;  - icu=58.2=h9c2bf20_1&#xA;  - jpeg=9b=h024ee3a_2&#xA;  - libcurl=7.62.0=h20c2e04_0&#xA;  - libedit=3.1.20170329=h6b74fdf_2&#xA;  - libffi=3.2.1=hd88cf55_4&#xA;  - libgcc=7.2.0=h69d50b8_2&#xA;  - libgcc-ng=8.2.0=hdf63c60_1&#xA;  - libgd=2.2.5=hceca4fd_3&#xA;  - libpng=1.6.35=hbc83047_0&#xA;  - libssh2=1.8.0=h1ba5d50_4&#xA;  - libstdcxx-ng=8.2.0=hdf63c60_1&#xA;  - libtiff=4.0.9=he85c1e1_2&#xA;  - libuuid=1.0.3=h1bed415_2&#xA;  - libwebp=1.0.0=h222930b_1&#xA;  - libxml2=2.9.8=h26e45fe_1&#xA;  - ncurses=6.1=hf484d3e_0&#xA;  - openjdk=8.0.152=h46b5887_1&#xA;  - openssl=1.1.1=h7b6447c_0&#xA;  - perl=5.26.2=h14c3975_0&#xA;  - pip=18.1=py36_0&#xA;  - python=3.6.7=h0371630_0&#xA;  - readline=7.0=h7b6447c_5&#xA;  - setuptools=40.5.0=py36_0&#xA;  - sqlite=3.25.2=h7b6447c_0&#xA;  - tk=8.6.8=hbc83047_0&#xA;  - wheel=0.32.2=py36_0&#xA;  - xz=5.2.4=h14c3975_4&#xA;  - zlib=1.2.11=ha838bed_2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now you can run following command and most of the installation part is completed. Just the database for kraken is missing.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda env create -f bio.yaml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I hope this gives you some help how to process your data. If you need some information just let me know ;)&lt;/p&gt;&#xA;" OwnerUserId="3360" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-11-14T21:50:58.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5450" PostTypeId="1" CreationDate="2018-11-15T09:12:15.560" Score="2" ViewCount="576" Body="&lt;p&gt;When we get raw sequenced reads in FASTQ format, we usually need to do some pre-processing (QC, demultiplexing, etc.) prior to sequence alignment. The only way to register the results of this preprocessing is to add metadata to each read. This is best done by using the SAM format which has a well-defined system of metadata tags.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it seems that not a single read mapper (except maybe for &lt;em&gt;bwa aln&lt;/em&gt; with option &lt;em&gt;-b&lt;/em&gt;) accepts SAM (or one of the binary equivalents BAM/CRAM) as input to make use of the metadata that has been added during preprocessing. This implies that one has to convert the unaligned reads back to FASTQ format. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, whether one uses FASTQ throughout or converts back to FASTQ, a common annoyance with FASTQ is that metadata is registered as part of the query name - this is not standardized or even well-defined. After alignment, we again have to do some text processing to get the metadata out of the query name and into proper metadata tags defined by SAM. This back and forth hacking is highly unfortunate and error prone. I have seen this, for &lt;strong&gt;example&lt;/strong&gt;, in some single-cell sequencing projects where we have highly multiplexed sequencing runs with reads that need to be QC'd and classified. The new SAM standard from May 2018 does indeed define proper tags for handling cell-multiplexed runs. SAM is the de-facto standard to store both unaligned and aligned reads for years to come so it should be standard for read mappers to accept it as input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I would be grateful for some hints and if we could collect here some read mappers that do accept SAM/BAM/CRAM as input format.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="3684" LastActivityDate="2020-04-23T18:59:03.943" Title="Is there a read mapper that allows input in SAM format instead of FASTA/FASTQ?" Tags="|sam|ngs|read-mapping|data-preprocessing|" AnswerCount="4" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5451" PostTypeId="2" ParentId="5436" CreationDate="2018-11-15T09:15:51.397" Score="2" Body="&lt;p&gt;So, you can give a try to &lt;a href=&quot;http://apps.cytoscape.org/apps/gasoline&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Cytoscape app&lt;/a&gt; that will allow to perform comparison between networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This app is not specifically designed for comparing networks across different species but it will enable you comparing any kind of networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://alpha.dmi.unict.it/~gmicale/gasoline/#references&quot; rel=&quot;nofollow noreferrer&quot;&gt;These are the papers&lt;/a&gt; they published, which can give you some hints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About your specific problem...I can't help you about what to look for when you're comparing networks from different species (not an expert in this specific topic) but as far I know you should look for something related to patterns occurring across different networks (i.e. species), network motifs, distribution of network centralities, conserved structures. In general, to compare networks, I would look for global properties (e.g. diameter, clustering coefficient...) while I would use local properties (e.g. degree, closeness, and network centralities in general) to compare the role of a node that you retrieved in all your networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, since you're looking for some literature, these links may help:  &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;About &lt;a href=&quot;http://www.pnas.org/content/102/6/1974.short&quot; rel=&quot;nofollow noreferrer&quot;&gt;conserved patterns of protein interaction in multiple species&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;About &lt;a href=&quot;https://www.nature.com/articles/ng1291&quot; rel=&quot;nofollow noreferrer&quot;&gt;comparing genomic expression patterns across species&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://onlinelibrary.wiley.com/doi/abs/10.1111/1467-9531.00118&quot; rel=&quot;nofollow noreferrer&quot;&gt;This one&lt;/a&gt; is slightly out of topic but can give you&#xA;some other ideas about comparing networks.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3119" LastEditorUserId="48" LastEditDate="2018-11-17T12:29:02.837" LastActivityDate="2018-11-17T12:29:02.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5453" PostTypeId="2" ParentId="5429" CreationDate="2018-11-15T10:09:46.463" Score="0" Body="&lt;h3&gt;Files in a cellranger run&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Cellranger removes the intermediary files after the clusters have been computed. This can be observed in the following path from the cellranger project directory. This contains empty directories.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SC_RNA_AGGREGATOR_CS/SC_RNA_ANALYZER/RUN_GRAPH_CLUSTERING&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, the clusters generated by cellranger can be found in the file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;outs/analysis/clustering/graphclust/clusters.csv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Clusters can also be &quot;exported&quot; from &lt;code&gt;outs/cloupe.cloupe&lt;/code&gt; file into a CSV using the Cloupe Browser. These CSVs can be imported and used for plotting in R or Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The graph structure used to compute these (with the Louvian algorithm) cannot be found in the files generated by cellranger.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Reproducing graph-based clustering&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The Suerat R package performs graph-based clustering. The shared nearest neighbour (SNN) modularity optimisation uses the same parameters as the Python scripts called by cellranger (&lt;code&gt;k=30&lt;/code&gt;). This generates an SNN matrix which can be used to generate a graph structure from an adjacency matrix. For a Seurat object &lt;code&gt;test&lt;/code&gt; the SNN sparse matrix can be extract with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FindClusters(test, dims.use = 1:20, algorithm = 1,&#xA;             resolution = 0.6, k.param = 30, save.SNN = TRUE,&#xA;             reduction.use = &quot;pca&quot;, metric = &quot;euclidean&quot;)&#xA;SNN_Matrix &amp;lt;- as.matrix(ceiling(as.matrix(test@snn)&#xA;graph_structure &amp;lt;- igraph::graph_from_adjacency(SNN_matrix)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;FindClusters&lt;/code&gt; from the &lt;code&gt;Seurat&lt;/code&gt; package requires a Seurat object that has already has a dimension reduction run on it such as &lt;code&gt;RunPCA&lt;/code&gt;. The &lt;code&gt;algorithm = 1&lt;/code&gt; is the Louvian algorithm as performed by cellranger. The &lt;code&gt;algorithm&lt;/code&gt;, &lt;code&gt;reduction.use&lt;/code&gt; and &lt;code&gt;metric&lt;/code&gt; can be changed as mentioned in the &lt;code&gt;Suerat&lt;/code&gt; package documentation. Clustering algorithms not yet supported by Seurat can be run on the &lt;code&gt;graph_structure&lt;/code&gt; resulting from extracting the SNN matrix. It is not necessary to compute the edges separately, the Seurat functions will compute and return these.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the Leiden algorithm can be computed as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;devtools:install_github(&quot;TomKellyGenetics/leiden&quot;)&#xA;library(leiden)&#xA;clusters &amp;lt;- leiden(SNN_Matrix)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1855" LastActivityDate="2018-11-15T10:09:46.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5454" PostTypeId="1" AcceptedAnswerId="5457" CreationDate="2018-11-15T11:33:41.650" Score="3" ViewCount="840" Body="&lt;p&gt;I am trying to download multiple SRA files located in several SRA accessions. Some of my accession numbers are as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SRR1002678&#xA;SRR1158898&#xA;SRR1158907&#xA;SRR1158950&#xA;SRR1181152&#xA;SRR1181155&#xA;SRR1181196&#xA;SRR1181204&#xA;SRR1181205&#xA;SRR1181208&#xA;SRR1269497&#xA;SRR2100227&#xA;SRR2100419&#xA;SRR2100830&#xA;SRR2101344&#xA;SRR671725&#xA;SRR671726&#xA;SRR671740&#xA;SRR671759&#xA;SRR671760&#xA;SRR671761&#xA;SRR671779&#xA;SRR671782&#xA;SRR671815&#xA;SRR671821&#xA;SRR671842&#xA;SRR671862&#xA;SRR671863&#xA;SRR671864&#xA;SRR671865&#xA;.............&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried the following command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in {SRR1002678..SRR1184123}; do wget ftp://ftp.ncbi.nlm.nih.gov/sra/sra-instant/reads/ByRun/sra/SRR/SRR1*/$i; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but this did not work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried &lt;code&gt;prefetch&lt;/code&gt; as well:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sort -u SRR.txt | parallel -j 4 &quot;prefetch {}&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where SRR.txt is the list of my SRA IDs. However, with this command only a few files are downloaded as *.sra files, while with several other SRA IDs I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;2018-11-15T06:17:20 prefetch.2.8.2: 1) Downloading 'SRR1183962'...&#xA;2018-11-15T06:17:20 prefetch.2.8.2:  Downloading via https...&#xA;2018-11-15T06:20:21 prefetch.2.8.2: 1) 'SRR1183962' was downloaded successfully&#xA;2018-11-15T06:20:27 prefetch.2.8.2: 'SRR1183962' has 0 unresolved dependencies&#xA;2018-11-15T06:20:38 prefetch.2.8.2: 'SRR1183962' has no remote vdbcache&#xA;&#xA;2018-11-15T06:18:59 prefetch.2.8.2: 1) Downloading 'SRR1184034'...&#xA;2018-11-15T06:18:59 prefetch.2.8.2:  Downloading via https...&#xA;2018-11-15T06:20:40 prefetch.2.8.2: 1) failed to download SRR1184034&#xA;2018-11-15T06:20:40 prefetch.2.8.2 sys: libs/kns/unix/syssock.c:606:KSocketTimedRead: timeout exhausted while reading file within network system module - mbedtls_ssl_read returned -76 ( NET - Reading information from the socket failed )&#xA;2018-11-15T06:20:40 prefetch.2.8.2 int: libs/kns/unix/syssock.c:606:KSocketTimedRead: timeout exhausted while reading file within network system module -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I do not have a range of SRA files to download but quite random SRA files to download. What am I doing wrong? Any help is highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="3686" LastEditorUserId="492" LastEditDate="2018-11-15T17:01:44.933" LastActivityDate="2023-06-08T08:35:52.887" Title="Downloading multiple SRA files from several SRA accession IDs does not work" Tags="|linux|sratoolkit|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5455" PostTypeId="2" ParentId="5439" CreationDate="2018-11-15T12:48:25.817" Score="3" Body="&lt;p&gt;I would strongly recommend to use &lt;a href=&quot;https://bioconda.github.io/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioconda&lt;/a&gt; instead of compiling. It is one of the channels (&quot;distributors of packages&quot;) in &lt;a href=&quot;https://anaconda.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Anaconda Cloud&lt;/a&gt; which is basically a package source for &lt;a href=&quot;https://conda.io/docs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;conda&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short - conda allows you to create virtual environments isolated from system and to install various precompiled software packages with one command (so, by the way, you can easily use multiple software versions on one system). Also, Bioconda is just one channel aiming on various bioinformatics software, but many more packages can be found in aforementioned Anaconda Cloud.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tutorial how to start using Bioconda is on its homepage, and it is really easy (far more easier than to struggle with compiling).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioconda.github.io/recipes/hmmer/README.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioconda HMMER package&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. Of course there are cases in which you need a specific build of given software. Than it is really needed to compile manually.&lt;/p&gt;&#xA;" OwnerUserId="3687" LastEditorUserId="3687" LastEditDate="2018-11-15T14:53:00.770" LastActivityDate="2018-11-15T14:53:00.770" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5456" PostTypeId="2" ParentId="3889" CreationDate="2018-11-15T14:23:05.497" Score="1" Body="&lt;p&gt;SNPs, insertions, and deletions are generated by different biological processes (see for example &lt;a href=&quot;https://en.wikipedia.org/wiki/Mutation_rate#Variation_in_mutation_rates&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). I would expect the per-generation rates of these mutational processes to vary across species. I would also expect the degree to which each mutational process varies across species to vary as well. I think the answer to your question is &quot;probably not without further information&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, priors are a feature of Bayesian statistics. In Bayesian inference, it is good practice to redo one's analysis using several priors to establish how sensitive one's results are to the chosen prior. I would suggest that you try to call indels using a range of priors and compare the quality of your call sets to determine how important the choice of prior actually is. &lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2018-11-15T14:23:05.497" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5457" PostTypeId="2" ParentId="5454" CreationDate="2018-11-15T17:11:47.593" Score="6" Body="&lt;p&gt;First of all, the brace expansion you tried to use isn't valid. While bash can expand &lt;code&gt;{1..3}&lt;/code&gt; to &lt;code&gt;1 2 3&lt;/code&gt;, it has no way of knowing how to expand &lt;code&gt;{SRR1002678..SRR1184123}&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ echo {1..3}&#xA;1 2 3&#xA;$&lt;/span&gt; echo {SRR1002678..SRR1184123}&#xA;{SRR1002678..SRR1184123}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Perhaps you meant &lt;code&gt;SRR{1002678..1184123}&lt;/code&gt;? But there's no need for this, you already have the accessions in a file and can read that. I see that the FTP server you are accessing has organized the data like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sra/sra-instant/reads/ByRun/sra/SRR/SRRNNN/SRRNNNXXXX/SRRNNNXXXX.sra&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where &lt;code&gt;NNN&lt;/code&gt; are the 1st three digits of the accession. So, for example, the file for accession &lt;code&gt;SRR1002678&lt;/code&gt; is at:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ftp://ftp.ncbi.nlm.nih.gov/sra/sra-instant/reads/ByRun/sra/SRR/SRR118/SRR1181152/SRR1181152.sra&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With this in mind, you can write a little script that extracts the URL from each accession:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while read i; do &#xA;    n=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;$&lt;/span&gt;i&quot; | sed -E 's/(SRR...).*/\1/'); &#xA;    wget ftp://ftp.ncbi.nlm.nih.gov/sra/sra-instant/reads/ByRun/sra/SRR/&lt;span class=&quot;math-container&quot;&gt;$n/$&lt;/span&gt;i/$i.sra&#xA;done &amp;lt; SRR.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The only trick is the &lt;code&gt;sed&lt;/code&gt; command which simply looks for &lt;code&gt;SRR&lt;/code&gt; and the three next characters and removes everything else on the line (it &quot;captures&quot; the found pattern and then replaces the line with what was captured (&lt;code&gt;\1&lt;/code&gt;)).&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-11-15T17:11:47.593" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5458" PostTypeId="2" ParentId="5450" CreationDate="2018-11-16T10:14:14.160" Score="5" Body="&lt;p&gt;The common solution for scRNA-seq is to put cell barcodes and such in read headers and then post-process things with UMItools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But regarding your actual question, STAR can accept SAM/BAM as input with the &lt;code&gt;--readFilesType SAM PE&lt;/code&gt; option (it's &lt;code&gt;SAM&lt;/code&gt; for both that format and BAM). You can swap &lt;code&gt;SE&lt;/code&gt; for &lt;code&gt;PE&lt;/code&gt; if your data is single-end (or is effectively that way, as is the case in many scRNA-seq experiments). Note that in the case of PE experiments that mates &lt;a href=&quot;https://github.com/alexdobin/STAR/issues/416&quot; rel=&quot;noreferrer&quot;&gt;must be next to each other&lt;/a&gt;. I really don't know what this does with auxiliary tags though, so you would need to see if they're lost before you start processing a lot of data).&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-11-16T10:21:36.733" LastActivityDate="2018-11-16T10:21:36.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5459" PostTypeId="2" ParentId="5334" CreationDate="2018-11-16T10:27:51.230" Score="1" Body="&lt;p&gt;I would also add that splitting your data into chromosomes allows you to run each chromosome in parallel (presuming you have some kind of server you can send each run to), vastly reducing the amount of time your analysis is going to take. I've not run all 22 chromosomes at once, but I can imagine it will take an enormous amount of time if you are using even a small number of markers and references.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2018-11-16T10:27:51.230" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5460" PostTypeId="2" ParentId="5432" CreationDate="2018-11-17T02:08:09.180" Score="7" Body="&lt;p&gt;Single-cell analysis to compare samples is a long a difficult process. There is very good documentation for &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/what-is-cell-ranger&quot; rel=&quot;noreferrer&quot;&gt;10x Genomics cellranger&lt;/a&gt;, the &lt;a href=&quot;http://mccarrolllab.org/dropseq/&quot; rel=&quot;noreferrer&quot;&gt;DropSeq Pipeline&lt;/a&gt; and the &lt;a href=&quot;https://satijalab.org/seurat/&quot; rel=&quot;noreferrer&quot;&gt;Seurat R package&lt;/a&gt;. These tools all have GitHub repositories and the authors are very responsive if you encounter issues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Depending on the technology used to generate the data, you'll need to use either cellranger or DropSeq to process the FASTQ files. These are designed to account for the the different experimental design. Still, it is important to ensure that your samples have been processed in the same way to reduce batch effects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This question is to open-ended to cover all of the details needed to perform the analysis but here is some of the main things to consider and the general process to perform such as analysis. You will need competence using the command-line (shell) and programming in R (or Python) to perform single-cell analysis. I recommend performing this analysis on a remote server as some steps are memory intensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The general overview to compare samples is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Demultiplex the reads for different indicies. You need to do this even if you've only put one sample per lane (to remove the index from the reads). &lt;code&gt;cellranger mkfastq&lt;/code&gt; or Illumina's &lt;code&gt;bcl2fastq&lt;/code&gt; will do this.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Obtain a reference genome (FASTA) and gene annotation (GTF) for the species you are working with. You can prepare a reference transcriptome with &lt;code&gt;cellranger mkgtf&lt;/code&gt; and &lt;code&gt;cellranger mkref&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Run &lt;code&gt;cellranger count&lt;/code&gt; or the DropSeq pipeline on each sample separately. These will both perform STAR (splice-aware) alignment of paired-end RNA-Seq reads and count UMIs for each cell-barcode and gene. Cells will be identified and filtered by an automatic UMI threshold (this can be changed by forcing with expected number of cells). This will return a gene-barcode matrix.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You need to aggregate results of cellranger runs for different samples with &lt;code&gt;cellranger aggr&lt;/code&gt;. This will perform downsampling by default to normalise the number of reads between samples. It will also perform graph-based (Louvian) clustering on the combined samples and return HTML and Cloupe summary data to explore, in addition to a combined gene-barcode matrix for all samples.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Single-cell RNA-Seq experiments are subject to batch effects. The CCA method from &lt;code&gt;Seurat&lt;/code&gt; and the MNN method from &lt;code&gt;scran&lt;/code&gt; are both available in the respective R packages to account for this (with different approaches). Batch effect correction needs to be performed before downstream analysis to ensure comparisons are valid.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This methods require an overlap between the samples to estimate technical errors and account for them. You need to bear in mind that neither of these methods works unless:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;some of the cells detected in each sample are expected to be the same cell type&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;biological replicates of each sample have been performed&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Once you have performed this correction, then standard &lt;code&gt;Seurat&lt;/code&gt; workflows such as those shown in the tutorials can be performed which will identify clusters. You can use these to identify clusters specific to each genotype. Bear in mind that some overlap should be expected and if there isn't one, batch effect correction will over-correct for this.&lt;/p&gt;&#xA;" OwnerUserId="1855" LastActivityDate="2018-11-17T02:08:09.180" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5461" PostTypeId="1" CreationDate="2018-11-17T15:06:48.350" Score="2" ViewCount="44" Body="&lt;p&gt;I  am trying to assemble a plant genome using AWS resources using velvet. Plant genome is huge (&gt; 10 times human genome) and coverage is around 30 x. We are planning for de novo assembly with Velvet (threads enabled).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know if there is a calculator that can provide approximate time taken for assembly, given the necessary details. For eg. if I furnish RAM, CPU (type of instance), genome size, approximate coverage, type of sequencing (PE or SE) and number of client nodes, it should give me the number of hours or days that would take for assembly (reference and/or de novo).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: Previously posted on biostars, but no replies. &lt;/p&gt;&#xA;" OwnerUserId="3697" LastEditorUserId="48" LastEditDate="2018-11-17T16:17:55.527" LastActivityDate="2018-11-17T16:17:55.527" Title="Calculating alignment/mapping time" Tags="|ngs|read-mapping|assembly|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5462" PostTypeId="2" ParentId="5347" CreationDate="2018-11-17T18:29:51.387" Score="1" Body="&lt;p&gt;The following paper provides an in-depth exploration of factors affecting power of admixture analyses under a number of different models. Although the authors state they &quot;provide the open-source R software AdmixPower, to perform power and sample size analysis&quot;, in fact it seems the software is not available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.genetics.org/content/207/3/873.long&quot; rel=&quot;nofollow noreferrer&quot;&gt;AdmixPower: Statistical Power and Sample Size Estimation for Mapping Genetic Loci in Admixed Populations&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tool availability:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://research.cchmc.org/mershalab/Tools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://research.cchmc.org/mershalab/Tools.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Site asks for a password, apparently there is no option to sign up.&lt;/p&gt;&#xA;" OwnerUserId="1642" LastActivityDate="2018-11-17T18:29:51.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5464" PostTypeId="2" ParentId="2412" CreationDate="2018-11-18T08:35:09.460" Score="3" Body="&lt;p&gt;Just use minimap2 in split alignment mode to realign the reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that is not an option, then you could try using &lt;a href=&quot;https://pysam.readthedocs.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pysam&lt;/a&gt; to modify the CIGAR strings. I do not recommend this, as there are many opportunities for subtle bugs because the SAM spec is complex. You would need to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Sort the BAM on read ID so that you can efficiently retrieve reads that you want to merge&lt;/li&gt;&#xA;&lt;li&gt;Iterate through the BAM with pysam while loading all reads of the same read ID&lt;/li&gt;&#xA;&lt;li&gt;For each set of reads with the same read ID:&#xA;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;Order the reads of the same read ID by the number of hard-clipped bases at the start of the CIGAR string.&lt;/li&gt;&#xA;&lt;li&gt;Build a new CIGAR string from the ordered CIGAR strings&lt;/li&gt;&#xA;&lt;li&gt;Merge the meta tags from the ordered reads&lt;/li&gt;&#xA;&lt;li&gt;Write out a single, merged read&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="492" LastActivityDate="2018-11-18T08:35:09.460" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5465" PostTypeId="2" ParentId="1018" CreationDate="2018-11-18T08:39:39.300" Score="2" Body="&lt;p&gt;It looks like you are trying to do a transcript isoform expression analysis. Velvet is a genome assembler, and really not designed for this task. The tools you are looking for are kallisto and salmon.&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2018-11-18T08:39:39.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5466" PostTypeId="1" AcceptedAnswerId="5468" CreationDate="2018-11-19T06:38:04.153" Score="3" ViewCount="595" Body="&lt;p&gt;After alignment using BWA, I have removed the dupliment using the samtools(Version: 1.9).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My procedure is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bwa mem -k 32 -M ref.fa read1 read2 &amp;gt; out.sam &#xA;samtools view -@ 0 -b -T ref.fa -o out.bam in.sam&#xA;samtools sort -n -o out.nameSrt.bam in.bam&#xA;samtools fixmate -r -m in.nameSrt.bam out.fixmate.bam&#xA;samtools sort -o out.fixmate.sort.bam in.fixmate.bam&#xA;samtools markdup -r -S -s in.fixmate.sort.bam out.markdup.bam&#xA;samtools flagstat in.markdup.bam &amp;gt; out.markdup.flagstat&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The flagstat output result is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;21611397 + 0 in total (QC-passed reads + QC-failed reads)&#xA;0 + 0 secondary&#xA;0 + 0 supplementary&#xA;0 + 0 duplicates&#xA;21611397 + 0 mapped (100.00% : N/A)&#xA;21422330 + 0 paired in sequencing&#xA;10711165 + 0 read1&#xA;10711165 + 0 read2&#xA;19797684 + 0 properly paired (92.42% : N/A)&#xA;21422330 + 0 with itself and mate mapped&#xA;0 + 0 singletons (0.00% : N/A)&#xA;1306000 + 0 with mate mapped to a different chr&#xA;727043 + 0 with mate mapped to a different chr (mapQ&amp;gt;=5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Why is the total read number still more than the paired in sequencing after removing the duplicate in samtools flagstat output?&#xA;Is there anything wrong with my procedure?&lt;/p&gt;&#xA;" OwnerUserId="3703" LastEditorUserId="48" LastEditDate="2018-11-19T08:22:36.393" LastActivityDate="2018-11-19T14:14:54.783" Title="Why is the total read number still more than the paired in sequencing after removing the duplicate in samtools flagstat output?" Tags="|ngs|samtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5467" PostTypeId="1" CreationDate="2018-11-19T13:47:28.947" Score="0" ViewCount="450" Body="&lt;p&gt;I have an alignment of some full genome sequences of DENV1 that we have recently sequenced from the serum of patients. I have already cleaned the sequences, built the alignments and done the SNPs calling. After that I got the consensus sequences from each sample and I used MEGA to build the alignments of the consensus sequences and to construct a tree etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From just looking the alignment of the consensus sequences I can see several nt s differences among the sequences. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know if you can export from MEGA like a sort of table where you could see the nt changes for each position and sample?&lt;/p&gt;&#xA;" OwnerUserId="3704" LastEditorUserId="48" LastEditDate="2018-11-19T15:13:26.983" LastActivityDate="2018-11-25T12:37:27.230" Title="Exporting nt's differences from MEGA alignment" Tags="|sequence-alignment|software-recommendation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5468" PostTypeId="2" ParentId="5466" CreationDate="2018-11-19T14:14:54.783" Score="2" Body="&lt;p&gt;Bwa-mem may produce chimeric alignment, where different parts of a read are mapped to distinct loci. flagstat counts them as two reads.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-11-19T14:14:54.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5469" PostTypeId="2" ParentId="5450" CreationDate="2018-11-19T14:45:50.560" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/amplab/snap&quot; rel=&quot;nofollow noreferrer&quot;&gt;SNAP&lt;/a&gt; is a hash-index based genomic aligner that can take SAM and BAM as input. It has been used as an aligner for RNA-seq (see &lt;a href=&quot;https://hood-price.systemsbiology.org/research/snapr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SNAPR&lt;/a&gt;) and BS-seq (see &lt;a href=&quot;https://github.com/khuang28jhu/bs3&quot; rel=&quot;nofollow noreferrer&quot;&gt;BS-Seeker3&lt;/a&gt;). SNAPR seems to accept SAM/BAM but it is not clear whether BS-Seeker3 wraps this functionality as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried SNAP and it seems to do the job well, i.e. it only adds tags during alignment and does not remove metadata that has been deposited prior to alignment, neither from the SAM header nor from individual reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;NOTE&lt;/strong&gt; I post this answer to my own question after I became aware of SNAP.&lt;/p&gt;&#xA;" OwnerUserId="3684" LastEditorUserId="3684" LastEditDate="2018-11-19T16:24:29.050" LastActivityDate="2018-11-19T16:24:29.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5470" PostTypeId="2" ParentId="5450" CreationDate="2018-11-19T14:48:15.237" Score="5" Body="&lt;p&gt;With bwa-mem or minimap2, the recommended way is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools fastq -T BC,RX name-grouped.sam | bwa mem -C -p -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This passes the &lt;code&gt;BC&lt;/code&gt; and &lt;code&gt;RX&lt;/code&gt; tags through bwa-mem and copies them to the SAM output. If you are running paired-end alignment and your BAM is coordinate sorted, you have to run &lt;code&gt;samtools collate&lt;/code&gt; to group reads by name:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools collate -Ol0 in.bam tmp | samtools fastq -T BC,RX - | bwa mem -pC -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here &lt;code&gt;bwa mem -C&lt;/code&gt; or &lt;code&gt;minimap2 -y&lt;/code&gt; copies FASTQ comments to SAM output. This strategy is much easier to deal with than SAM/BAM. A caveat is that it doesn't work well for read groups because each read group requires a separate header line. You have to manually modify the header.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure which short-read RNA-seq mappers have implemented a similar feature. You might consider to send a feature request.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2018-11-19T16:52:53.583" LastActivityDate="2018-11-19T16:52:53.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5471" PostTypeId="2" ParentId="5359" CreationDate="2018-11-19T15:32:44.483" Score="6" Body="&lt;p&gt;The question and the accepted answer are not about k-mer data structure at all, which I will explain in detail below. I will first answer the actual question OP intends to ask.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simplest way to keep k-mers is to use an ordinary hash table. The performance is mostly determined by the hash table library. &lt;code&gt;std::unordered_map&lt;/code&gt; in gcc/clang is one of the worst choices because for integer keys, it is very slow. Google dense, &lt;code&gt;ska::bytell_hash_map&lt;/code&gt; and &lt;code&gt;ska::flat_hash_map&lt;/code&gt;, &lt;code&gt;tsl::robin_map&lt;/code&gt; and &lt;code&gt;absl::flat_hash_map&lt;/code&gt; are much faster. There are a few libraries that focus on smaller footprint, such as google sparse and sparsepp, but those can be a few times slower.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to the choice of hash table, how to construct the key is critical. For k&amp;lt;=32, the right choice is to encode a k-mer with a 64-bit integer, which will be vastly better than &lt;code&gt;std::string&lt;/code&gt;. Memory alignment is also important. In C/C++, as long as there is one 8-byte member in a struct, the struct will be 8-byte aligned on x86_64 by default. Most C++ hash table libraries pack key and value in &lt;code&gt;std::pair&lt;/code&gt;. If you use 64-bit keys and 32-bit values, &lt;code&gt;std::pair&lt;/code&gt; will be 8-byte aligned and use 16 bytes, even though only 12 bytes are actually used – 25% of memory is wasted. In C, we can explicitly define a packed struct with &lt;code&gt;__attribute__ ((__packed__))&lt;/code&gt;. In C++, probably you need to define special key types. A better way to get around memory alignment is to go down to the bit level. For read mapping, for example, we only use 15–23bp seeds. Then we have 18 (=64-23*2) bits left unused. We can use these 18 bits to count k-mers. Such bit-level management is quite common.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above is just basic techniques. There are a few other tricks. For example, 1) instead of using one hash table, we can use 4096 (=2^12) hash tables. Then we can store 12 bits of k-mer information into the 4096 part. This gives us invaluable 12 bits in each bucket to store extra information. This strategy also simplifies parallel k-mer insertions as with a good hash function, it is rare to insert into two tables at the same time. 2) when most k-mers are unique, the faster way to count k-mers is to put k-mers in an array and then sort it. Sorting is more cache friendly and is faster than hash table lookups. The downside is that sort counting can be memory demanding when most k-mers are highly repetitive.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The other answer is spending considerable (probably the majority of) time on k-mer iteration, not on hash table operations. The program loops through each position on the sequence and then each k-mer position. For an &lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt;-long sequence, this is an &lt;span class=&quot;math-container&quot;&gt;$O(kL)$&lt;/span&gt; algorithm. It has worse theoretical time complexity than hash table operations, which is &lt;span class=&quot;math-container&quot;&gt;$O(L)$&lt;/span&gt;. Although hash table operations are slow due to cache misses, a factor of k=29 is quite significant. Another issue is that all programs in the question and in the other answer are compiled without &lt;code&gt;-O3&lt;/code&gt;. Adding this option brings the &lt;code&gt;bytell_hash_map&lt;/code&gt; lookup time from 314s to 34s on my machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The C program at the end of my post shows the proper way to iterate k-mers. It is an &lt;span class=&quot;math-container&quot;&gt;$O(L)$&lt;/span&gt; algorithm with a tiny constant. The program keeps track of both forward and reverse k-mers at the same time and update them with a few bit operations at each sequence position. This echoes my &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/3626/37&quot;&gt;previous comment&lt;/a&gt; &quot;You should not reverse complement the whole k-mer&quot;. On the same machine, the program looks up k-mers in 5.5s using 792MB RAM at the peak. This 6-fold (=34/5.5) speedup mostly comes from k-mer iteration, given that the hash table library in use is known to have comparable performance to &lt;code&gt;bytell_hash_map&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;lt;stdint.h&amp;gt;&#xA;#include &quot;khash.h&quot;&#xA;&#xA;static inline uint64_t hash_64(uint64_t key)&#xA;{ // more sophisticated hash function to reduce collisions&#xA;  key = (~key + (key &amp;lt;&amp;lt; 21)); // key = (key &amp;lt;&amp;lt; 21) - key - 1;&#xA;  key = key ^ key &amp;gt;&amp;gt; 24;&#xA;  key = ((key + (key &amp;lt;&amp;lt; 3)) + (key &amp;lt;&amp;lt; 8)); // key * 265&#xA;  key = key ^ key &amp;gt;&amp;gt; 14;&#xA;  key = ((key + (key &amp;lt;&amp;lt; 2)) + (key &amp;lt;&amp;lt; 4)); // key * 21&#xA;  key = key ^ key &amp;gt;&amp;gt; 28;&#xA;  key = (key + (key &amp;lt;&amp;lt; 31));&#xA;  return key;&#xA;}&#xA;&#xA;KHASH_INIT(64, khint64_t, int, 1, hash_64, kh_int64_hash_equal)&#xA;&#xA;unsigned char seq_nt4_table[128] = { // Table to change &quot;ACGTN&quot; to 01234&#xA;  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,&#xA;  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,&#xA;  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,&#xA;  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,&#xA;  4, 0, 4, 1,  4, 4, 4, 2,  4, 4, 4, 4,  4, 4, 4, 4,&#xA;  4, 4, 4, 4,  3, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4,&#xA;  4, 0, 4, 1,  4, 4, 4, 2,  4, 4, 4, 4,  4, 4, 4, 4,&#xA;  4, 4, 4, 4,  3, 4, 4, 4,  4, 4, 4, 4,  4, 4, 4, 4&#xA;};&#xA;&#xA;static uint64_t process_seq(khash_t(64) *h, int k, int len, char *seq, int is_ins)&#xA;{&#xA;  int i, l;&#xA;  uint64_t x[2], mask = (1ULL&amp;lt;&amp;lt;k*2) - 1, shift = (k - 1) * 2, tot = 0;&#xA;  for (i = l = 0, x[0] = x[1] = 0; i &amp;lt; len; ++i) {&#xA;    int c = (uint8_t)seq[i] &amp;lt; 128? seq_nt4_table[(uint8_t)seq[i]] : 4;&#xA;    if (c &amp;lt; 4) { // not an &quot;N&quot; base&#xA;      x[0] = (x[0] &amp;lt;&amp;lt; 2 | c) &amp;amp; mask;                  // forward strand&#xA;      x[1] = x[1] &amp;gt;&amp;gt; 2 | (uint64_t)(3 - c) &amp;lt;&amp;lt; shift;  // reverse strand&#xA;      if (++l &amp;gt;= k) { // we find a k-mer&#xA;        uint64_t y = x[0] &amp;lt; x[1]? x[0] : x[1];&#xA;        khint_t itr;&#xA;        if (is_ins) { // insert&#xA;          int absent;&#xA;          itr = kh_put(64, h, y, &amp;amp;absent);&#xA;          if (absent) kh_val(h, itr) = 0;&#xA;          tot += ++kh_val(h, itr);&#xA;        } else { // look up&#xA;          itr = kh_get(64, h, y);&#xA;          tot += itr == kh_end(h)? 0 : kh_val(h, k);&#xA;        }&#xA;      }&#xA;    } else l = 0, x[0] = x[1] = 0; // if there is an &quot;N&quot;, restart&#xA;  }&#xA;  return tot;&#xA;}&#xA;&#xA;#include &amp;lt;zlib.h&amp;gt;&#xA;#include &amp;lt;time.h&amp;gt;&#xA;#include &amp;lt;unistd.h&amp;gt;&#xA;#include &quot;kseq.h&quot;&#xA;KSEQ_INIT(gzFile, gzread)&#xA;&#xA;int main(int argc, char *argv[])&#xA;{&#xA;  khash_t(64) *h;&#xA;  int i, k = 29;&#xA;  while ((i = getopt(argc, argv, &quot;k:&quot;)) &amp;gt;= 0)&#xA;    if (i == 'k') k = atoi(optarg);&#xA;  h = kh_init(64);&#xA;  for (i = 1; i &amp;gt;= 0; --i) {&#xA;    uint64_t tot = 0;&#xA;    kseq_t *ks;&#xA;    gzFile fp;&#xA;    clock_t t;&#xA;    fp = gzopen(argv[optind], &quot;r&quot;);&#xA;    ks = kseq_init(fp);&#xA;    t = clock();&#xA;    while (kseq_read(ks) &amp;gt;= 0)&#xA;      tot += process_seq(h, k, ks-&amp;gt;seq.l, ks-&amp;gt;seq.s, i);&#xA;    fprintf(stderr, &quot;[%d] %.3f\n&quot;, i, (double)(clock() - t) / CLOCKS_PER_SEC);&#xA;    kseq_destroy(ks);&#xA;    gzclose(fp);&#xA;  }&#xA;  kh_destroy(64, h);&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2018-12-05T05:45:52.313" LastActivityDate="2018-12-05T05:45:52.313" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5472" PostTypeId="1" CreationDate="2018-11-20T17:30:28.413" Score="1" ViewCount="65" Body="&lt;p&gt;I am using &lt;a href=&quot;https://portal.gdc.cancer.gov/files/a630f0a0-39b3-4aab-8181-89c1dde8d3e2&quot; rel=&quot;nofollow noreferrer&quot;&gt;a mutation MAF file from GDC&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to select for point mutations, therefore I selected &lt;code&gt;Variant_Classification&lt;/code&gt; equal to &lt;code&gt;SNP&lt;/code&gt;, and &lt;code&gt;Variant_Classification&lt;/code&gt; equal to &lt;code&gt;INS&lt;/code&gt; and &lt;code&gt;DEL&lt;/code&gt; - but in this case I need to check against the size of the indel mutation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I check for point mutations in the indel case? I think I have to work it out through checking the length of the fields in columns &lt;code&gt;Reference_Allele&lt;/code&gt;, &lt;code&gt;Tumor_Seq_Allele1&lt;/code&gt;, &lt;code&gt;Tumor_Seq_Allele2&lt;/code&gt; but I am not sure if this is the best solution.&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2018-11-20T17:30:28.413" Title="How to select point mutations from MAF file" Tags="|mutations|indel|maf|point-mutation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5473" PostTypeId="2" ParentId="5427" CreationDate="2018-11-20T19:48:20.707" Score="2" Body="&lt;p&gt;One way to deal with this would be to first merge paired-end reads based upon their overlapping regions, and then map them and calculate the coverage. This way you're only counting once per unique sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Programs like &lt;a href=&quot;https://github.com/jstjohn/SeqPrep&quot; rel=&quot;nofollow noreferrer&quot;&gt;SeqPrep&lt;/a&gt;, &lt;a href=&quot;https://sco.h-its.org/exelixis/web/software/pear/doc.html#cl-usage&quot; rel=&quot;nofollow noreferrer&quot;&gt;PEAR&lt;/a&gt; (Paired-End reAd mergeR), and &lt;a href=&quot;https://github.com/ExpressionAnalysis/ea-utils/blob/wiki/FastqJoin.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;fastq-join&lt;/a&gt; can do this fairly quickly. However, this isn't a perfect solution as the reads will not be joined unless they meet some minimum overlap criteria.&lt;/p&gt;&#xA;" OwnerUserId="3499" LastActivityDate="2018-11-20T19:48:20.707" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5474" PostTypeId="2" ParentId="5225" CreationDate="2018-11-21T10:15:43.510" Score="5" Body="&lt;p&gt;Seruat will give you a list of genes which it thinks are upregulated in a particular cluster. Look at the functions that talk about marker genes - these functions basically do a DE analysis of the genes in one cluster compared to the others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then take that list and feed it to any standard GO analysis tool. Have a look at the topGO topKEGG and geneSetTest functions in the limma package, the GOStats package and the gsea. All should be suitable.  The GOSeq package is designed to compensate for gene length bias in RNA-seq experiments. As 10X only samples 3'-tags, there shouldn't be any gene-length bias in the data, so this shouldn't be an issue. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-11-21T10:15:43.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5475" PostTypeId="1" CreationDate="2018-11-21T15:43:05.703" Score="4" ViewCount="1955" Body="&lt;p&gt;I'm an informatics student who has essentially zero knowledge of biology. I BLASTed my gene and have 1000s of results with very low E values. Where do I go from here if I want to find orthologues?&lt;/p&gt;&#xA;" OwnerUserId="3714" LastEditorUserId="48" LastEditDate="2018-11-22T08:19:16.650" LastActivityDate="2020-06-17T15:24:25.543" Title="Finding orthologues using BLAST on the NCBI database" Tags="|blast|orthologues|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5476" PostTypeId="2" ParentId="4294" CreationDate="2018-11-21T15:48:14.813" Score="1" Body="&lt;p&gt;A SAM file stores alignments to a set of reference sequences.  However, those reference sequences are not usually stored in the file. Instead, they will be in a separate sequence file that is usually in FASTA format. You can get the number of reference sequences stored in that file using the command&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -c '^&amp;gt;' reference.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="492" LastActivityDate="2018-11-21T15:48:14.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5477" PostTypeId="2" ParentId="5427" CreationDate="2018-11-21T16:22:17.793" Score="1" Body="&lt;p&gt;Bases from overlapping reads of the same fragment provide extra information, as their base qualities come from different sequencing cycles on the sequencer. This information can be used by downstream tools. For example, the merging tool of &lt;a href=&quot;https://drive5.com/usearch/manual/merge_pair.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;USEARCH&lt;/a&gt; increases base quality scores of bases that match between overlapping reads. Depending on the context under which coverage information will be used in the future, it may be a good idea to &quot;double-count&quot; overlapping bases.&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="492" LastEditDate="2018-12-11T18:16:36.957" LastActivityDate="2018-12-11T18:16:36.957" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5478" PostTypeId="2" ParentId="5475" CreationDate="2018-11-21T17:02:28.520" Score="7" Body="&lt;p&gt;BLAST is often a sensible way to find gene/protein homologs, but resolving those as orthologs vs paralogs etc. is a non-trivial task, especially when considering large numbers of species.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you are new to bioinformatics I recommend you search an ortholog database that already exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some databases:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.orthodb.org/&quot; rel=&quot;noreferrer&quot;&gt;orthoDB&lt;/a&gt;  &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://omabrowser.org/oma/home/&quot; rel=&quot;noreferrer&quot;&gt;OMA&lt;/a&gt;  &lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://eggnogdb.embl.de/#/app/home&quot; rel=&quot;noreferrer&quot;&gt;EggNog&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All of these should allow you to search either with text or a sequence to identify the orthologous groups that your input belongs to at different taxonomic levels. Sequence searches either follow pairwise alignment (e.g. BLAST, DIAMOND, RAPSEARCH) or profile HMM type (e.g. HMMER3) approaches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more complete list of ortholog databases is available from &quot;&lt;a href=&quot;https://questfororthologs.org/orthology_databases&quot; rel=&quot;noreferrer&quot;&gt;Quest for orthologs&lt;/a&gt;&quot;.&lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="48" LastEditDate="2018-11-22T08:18:01.193" LastActivityDate="2018-11-22T08:18:01.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5481" PostTypeId="1" AcceptedAnswerId="5572" CreationDate="2018-11-16T20:53:36.107" Score="7" ViewCount="428" Body="&lt;p&gt;The DNA sequence sections of the three INSDC databases (i.e., &lt;a href=&quot;https://www.ddbj.nig.ac.jp/index-e.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DDBJ&lt;/a&gt;, &lt;a href=&quot;https://www.ebi.ac.uk/ena&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENA Sequence&lt;/a&gt; and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genbank/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GenBank&lt;/a&gt;) are synchronized periodically and strive to keep their stored data as ubiquitously accessible as possible. Except for idiosyncrasies in their data submission routes, there should be little, if any, reason for preferentially submitting sequence data to one database over another. Yet many researchers display very noticeable preferences!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When asked point blank, many colleagues emphasize that these preferences are not linked to the database-specific data submission or retrieval interfaces available. Yet they cannot provide a technical reason for their choice either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe researchers base their preference on geographic vicinity (&quot;buy local&quot;), but maybe there are genuine technical reasons for their preference (even if they are unaware of these reasons themselves).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, my question is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you think of any technical reason - however minute or seemingly insignificant - in preferentially submitting DNA sequences to NCBI's &lt;em&gt;GenBank&lt;/em&gt; over &lt;em&gt;ENA Sequence&lt;/em&gt; (or vice versa)? For example, there may be differences in data storage or accessibility that are relevant to you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Put differently: Short of flipping a coin, why would you (as a bioinformatics-prone end user) select one database over the other for your data submission?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;--&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 1&lt;/strong&gt;: Many SE posters attempt to reassure me that I can submit my DNA sequences to either database and that &quot;the data will be just fine there&quot;. Having submitted data to both &lt;em&gt;GenBank&lt;/em&gt; and &lt;em&gt;ENA Sequence&lt;/em&gt; for 10+ years myself, this is not what this question is about. Instead, this question is about carving out genuine technical differences from a user perspective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit 2&lt;/strong&gt;: Since no answer has been posted yet despite a bounty, I figure this is a hard question as specified. Thus, I will broaden the question and now also accept non-technical reasons for preferentially submitting DNA sequence data to either &lt;em&gt;GenBank&lt;/em&gt; or &lt;em&gt;ENA Sequence&lt;/em&gt;. (That being said, I still hope for someone listing a technical reason.)&lt;/p&gt;&#xA;" OwnerUserId="3748" OwnerDisplayName="Michael Gruenstaeudl" LastEditorUserId="3748" LastEditDate="2018-12-03T10:11:09.940" LastActivityDate="2018-12-03T19:03:23.063" Title="ANY (technical) reason behind submitting sequences to GenBank versus ENA Sequence" Tags="|database|genomics|phylogenetics|genbank|embl|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="5482" PostTypeId="1" AcceptedAnswerId="7091" CreationDate="2018-11-22T10:58:01.667" Score="9" ViewCount="317" Body="&lt;p&gt;I'd like to call diploid SNV variants from long-reads data (~80SMRTcells PacBio). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have generated a draft reference genome for an indivudual from a heterozygous (~4%) species (&lt;a href=&quot;https://github.com/marbl/canu&quot; rel=&quot;noreferrer&quot;&gt;Canu&lt;/a&gt;+&lt;a href=&quot;https://github.com/mapleforest/HaploMerger2&quot; rel=&quot;noreferrer&quot;&gt;Haplomerger2&lt;/a&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can use this reference for some applications:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;RNAseq mapping;&lt;/li&gt;&#xA;&lt;li&gt;functional annotation;&lt;/li&gt;&#xA;&lt;li&gt;duplicate detection;&lt;/li&gt;&#xA;&lt;li&gt;synteny analysis&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;but not others:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;analysis of compound heterozygotes; &lt;/li&gt;&#xA;&lt;li&gt;allele-specific expression; &lt;/li&gt;&#xA;&lt;li&gt;variant linkage&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This is mainly why I need phasing. FYI, I tried &lt;a href=&quot;https://github.com/PacificBiosciences/FALCON/wiki/FALCON-FALCON-Unzip-%22For-Phased-Diploid-Genome-Assembly-with-Single-Molecule-Real-Time-Sequencing%22&quot; rel=&quot;noreferrer&quot;&gt;Falcon + FalconUnzip&lt;/a&gt; too, but the daligner phase was just taking too much time/resources and I need to find a strategy to make it work in reasonable time (working on it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meanwhile, for phasing, It was suggested to use &lt;a href=&quot;https://github.com/vibansal/HapCUT2/blob/master/README.md&quot; rel=&quot;noreferrer&quot;&gt;HapCUT2&lt;/a&gt; - from Canu's FAQ&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(1) &quot;Avoid collapsing the genome&quot; in Canu to get raw diploid assembly as complete &amp;amp; separate as possible (2) HaploMerger2 to get high-quality reference and alternative haploid assemblies (3) HapCUT2 or other phasing tools to get the high-quality haplotype assembly based on the reference haploid assembly. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;However, HapCUT2 requires the input of a diploid SNV file. Which &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/4019/converting-a-bam-file-into-vcf/4022#4022&quot;&gt;could be done&lt;/a&gt; if I had short-reads data, but yet haven't found a method for doing so with in my hands just long-reads data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Strategies I tried to get the SNVs:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/benedictpaten/marginPhase&quot; rel=&quot;noreferrer&quot;&gt;marginPhase&lt;/a&gt; : &lt;a href=&quot;https://www.biorxiv.org/content/early/2018/04/12/293944&quot; rel=&quot;noreferrer&quot;&gt;method paper&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;but I had a &lt;a href=&quot;https://github.com/benedictpaten/marginPhase/issues/12&quot; rel=&quot;noreferrer&quot;&gt;issue&lt;/a&gt; in running it &lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://github.com/aquaskyline/Clairvoyante&quot; rel=&quot;noreferrer&quot;&gt;Clairvoyante&lt;/a&gt; : &lt;a href=&quot;https://www.biorxiv.org/content/early/2018/04/28/310458&quot; rel=&quot;noreferrer&quot;&gt;method paper&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;but I currently don't have access to suitable GPU machines to build the model - and I can't use the models provided by the authors because my organism is too different from human&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Any input would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="294" LastEditorUserId="294" LastEditDate="2018-11-22T13:12:53.627" LastActivityDate="2019-02-21T10:18:47.657" Title="calling diploid SNVs from long reads" Tags="|vcf|snp|variant-calling|long-reads|pacbio|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5483" PostTypeId="1" AcceptedAnswerId="5488" CreationDate="2018-11-22T15:02:05.717" Score="5" ViewCount="841" Body="&lt;p&gt;I have two draft genomes (aprox. 500Mb each), one with ~10'000 scaffolds (&lt;code&gt;genome1.fasta&lt;/code&gt;) and one with ~1'000 contigs (&lt;code&gt;genome2.fasta&lt;/code&gt;). I would like to visualize their pairwise alignment, which &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/4139/visualisation-of-genome-alignment&quot;&gt;a while ago&lt;/a&gt; I found it could easily be done by &lt;a href=&quot;http://mummer.sourceforge.net/manual/#mummerplot&quot; rel=&quot;nofollow noreferrer&quot;&gt;mummerplot&lt;/a&gt; in &lt;a href=&quot;http://mummer.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MUMmer&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In mummer, &lt;a href=&quot;http://mummer.sourceforge.net/manual/#aligningdraft&quot; rel=&quot;nofollow noreferrer&quot;&gt;pairwise draft genomes alignment&lt;/a&gt; is supported:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Many times it is necessary to align two genomes that have not yet been completed, or two genomes with multiple chromosomes. This can make things a little more complicated, since a separate alignment would have to be generated for each possible pairing of the sequences. However, both NUCmer and PROmer automate this process and accept multi-FastA inputs, thus simplifying the process of aligning two sets of contigs, scaffolds or chromosomes. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is still ok if I do it aligning small portions (e.g. contig vs scaffold), as I could get &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/4155/294&quot;&gt;here&lt;/a&gt;. However, if I do it all-vs-all, I end up with a &quot;confetti&quot;-plot (self-coined term) instead of a dot-plot : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ggmtC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ggmtC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure how to interpret this, and what I would like to see is more of a straight line as the one. Probably it's just a matter of filtering, so I'd like to ask for advice.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used &lt;a href=&quot;http://mummer.sourceforge.net/manual/#nucmer&quot; rel=&quot;nofollow noreferrer&quot;&gt;nucmer&lt;/a&gt; through &lt;a href=&quot;http://mummer.sourceforge.net/manual/#filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;delta-filter&lt;/a&gt; as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# mummer version //4.0.0beta1 &#xA;&#xA;# nucmer&#xA;nucmer --prefix=ref_qry genome1.fasta genome2.fasta&#xA;&#xA;# ref_qry.delta generated from previous step&#xA;delta-filter -r -q ref_qry.delta &amp;gt; ref_qry.filter&#xA;&#xA;# plot&#xA;mummerplot ref_qry.filter -R genome1.fasta -Q genome2.fasta --filter --layout&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Reasons for options &lt;code&gt;-r -q&lt;/code&gt; in &lt;code&gt;delta-filter&lt;/code&gt; step : &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;To determine a one-to-one mapping of each reference and query, combine the options and use delta-filter -r -q.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'll also accept other methods for visualisation (doesn't need to be withh MUMmer). Any input is greatly appreciated!&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;EDIT&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found some solutions which I reported as answers. I will accept anyone giving answer on how to tune MUMmer parameters to achieve the goal I describe in the original question. &lt;/p&gt;&#xA;" OwnerUserId="294" LastEditorUserId="294" LastEditDate="2018-11-26T07:28:58.780" LastActivityDate="2018-11-26T07:28:58.780" Title="full visualisation of draft genomes alignment" Tags="|sequence-alignment|genome|software-recommendation|visualization|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5484" PostTypeId="1" CreationDate="2018-11-22T19:40:21.203" Score="1" ViewCount="49" Body="&lt;p&gt;Well, I'm a MSc student in Information Technology Engineering, trying to become more familiar in the Complex networks specially Bioinformatic field to get more knowledge of cancer detection by Biochemical data analysis. &#xA;beside python / R language what else should I work on to prepare myself for an academic seminar?&lt;/p&gt;&#xA;" OwnerUserId="3719" LastActivityDate="2018-11-22T19:40:21.203" Title="What are fundamentals to work on bioinformatic data analysis ?" Tags="|networks|" AnswerCount="0" CommentCount="2" ClosedDate="2018-11-26T06:28:02.030" ContentLicense="CC BY-SA 4.0" />
  <row Id="5485" PostTypeId="2" ParentId="5427" CreationDate="2018-11-22T23:09:03.470" Score="5" Body="&lt;p&gt;&lt;code&gt;samtools depth&lt;/code&gt; is a simplified version of &lt;code&gt;samtools mpileup&lt;/code&gt;, which handles overlapping regions by default. Since overlapping regions shouldn't be included in coverage calculations (after all, you're not covering a region twice) you can use &lt;code&gt;samtools mpileup&lt;/code&gt; instead:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat foo.sam&#xA;@SQ SN:1    LN:1000&#xA;read1   99  1   1   50  50M =   46  95  ATTTAAAAATTAATTTAATGCTTGGCTAAATCTTAATTACATATATAATT  &amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;  NM:i:0&#xA;read1   147 1   46  50  50M =   1   -95 ATTTAAAAATTAATTTAATGCTTGGCTAAATCTTAATTACATATATAATT  &amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;  NM:i:0&#xA;&#xA;$ samtools mpileup -Q 1 foo.sam | cut -f 1,2,4&#xA;...&#xA;1   42  1&#xA;1   43  1&#xA;1   44  1&#xA;1   45  1&#xA;1   46  1&#xA;1   47  1&#xA;1   48  1&#xA;1   49  1&#xA;1   50  1&#xA;1   51  1&#xA;1   52  1&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I haven't shown the entirety of the output and you'd want to use the &lt;code&gt;-aa&lt;/code&gt; flag to ensure that all bases are returned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;code&gt;samtools mpileup&lt;/code&gt; is doing this internally by setting the base phred scores of overlapping bases in one of the mates to 0, which then get excluded due to &lt;code&gt;-Q 1&lt;/code&gt; (the default is &lt;code&gt;-Q 13&lt;/code&gt;, which you'd want to change).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an aside, you probably don't need exactly correct values, only approximates. Given that, you can use &lt;code&gt;bamPEFragmentSize&lt;/code&gt; from deepTools (or a similar tool from another package) to get the median fragment and read length, determine the median number of covered bases from that and use it to get a quite accurate approximation of the actual coverage (multiply the median fragment base coverage by the number of fragments and divide by the genome size). That may prove faster and close enough.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-11-22T23:09:03.470" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5486" PostTypeId="2" ParentId="5225" CreationDate="2018-11-23T07:57:18.383" Score="2" Body="&lt;p&gt;There is no purpose-built R package to perform gene set enrichment analysis on single-cell data but there does not need to be. You should be able to tools developed for bulk-RNA-Seq or microarray data, although you may not get as significant results from a sparse scRNA-Seq matrix as single-cell technologies have poor sensitivity and miss genes. What you need for a gene set enrichment analysis is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;a database of pathways (or gene sets) to test enrichment for. Packages such as &lt;code&gt;GO.db&lt;/code&gt; or &lt;code&gt;reactome.db&lt;/code&gt; from Bioconductor support these in R.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a gene expression matrix which you can get from the output of &lt;code&gt;cellranger&lt;/code&gt; (as &lt;code&gt;matrix.mtx&lt;/code&gt;). This should be normalised and corrected for batch effects. The &lt;code&gt;Seurat&lt;/code&gt; R package is recommended to do this. Calling &lt;code&gt;object@data&lt;/code&gt; will return the normalised seurat data matrix to pass it to other tools.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;an annotation to identify the genes (depending on the reference data you used for the cellranger &quot;transcriptome&quot;).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;to define groups for differential expression analysis. You can import the clusters generated by &lt;code&gt;cellranger&lt;/code&gt; in &lt;code&gt;outs/analysis&lt;/code&gt; or export clusters of your choosing from the Loupe Browser as CSV files that can be imported into R. These should match the &lt;code&gt;barcodes.tsv&lt;/code&gt; of columns of your data matrix to define the cluster for each cell barcode.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;to run a differential expression analysis to compare clusters. You can use clusters to define cell groups and perform analysis with tools you would use for bulk-RNA-Seq such as &lt;code&gt;limma&lt;/code&gt;, &lt;code&gt;edgeR&lt;/code&gt;, or &lt;code&gt;deseq&lt;/code&gt;. Don't forget to adjust for multiple tests. You also need to correct for batch effects between replicates and log-transform you data to ensure the results reflect biological differences. This will give you a list of differentially expressed genes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;to calculate gene set enrichment of differentially expressed genes in a gene set list such as those supported by pathway databases. You can do this like any other list of significant genes with tools used for bulk-RNA-Seq data, either with dedicated webtools or R packages such as &lt;code&gt;safe&lt;/code&gt; or GSEA. You can also run the hypergeometric test in R yourself.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;cellranger&lt;/code&gt; output gives you all the data you need to use standard tools for which there are already guides on how to do differential analysis. These analyses do not differ from bulk-RNA-Seq analysis once the data has been normalised and the  clusters have been defined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[edit:corrected typo in package name]&lt;/p&gt;&#xA;" OwnerUserId="1855" LastEditorUserId="3677" LastEditDate="2020-01-10T15:52:51.487" LastActivityDate="2020-01-10T15:52:51.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5487" PostTypeId="1" AcceptedAnswerId="5491" CreationDate="2018-11-23T09:33:52.677" Score="2" ViewCount="268" Body="&lt;p&gt;I have Microarray Normalized Expression data for a specific Gene. It looks like below in a dataframe &lt;code&gt;B&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SampleID    Gene    Type&#xA;Sample1     5.02    Tumor&#xA;Sample2     5.06    Tumor&#xA;Sample3     5.1     Tumor&#xA;Sample4     5.11    Tumor&#xA;Sample5    5.127    Normal&#xA;Sample6     5.12    Normal&#xA;Sample7    5.138    Normal&#xA;Sample8    5.149    Normal&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I see that the minimum expression value in the table is 5.0 and maximum expression value is around 5.9.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to show the expression between two conditions with a boxplot and used following code.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;q &amp;lt;- ggboxplot(B, x = &quot;Type&quot;, y = &quot;Gene&quot;,&#xA;               color = &quot;black&quot;, palette = &quot;npg&quot;,&#xA;               add = &quot;jitter&quot;, ylab = 'Gene expression', xlab=FALSE,&#xA;               order=c(&quot;Normal&quot;, &quot;Tumor&quot;))&#xA;q + stat_compare_means(method = &quot;t.test&quot;) + &#xA;  geom_point() + &#xA;  stat_n_text()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This gave a plot like this &lt;a href=&quot;https://i.stack.imgur.com/0wYwx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0wYwx.png&quot; alt=&quot;Boxplot1&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when I remove the &lt;code&gt;jitter&lt;/code&gt; from the code, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;q &amp;lt;- ggboxplot(B, x = &quot;Type&quot;, y = &quot;Gene&quot;,&#xA;               color = &quot;black&quot;, palette = &quot;npg&quot;,&#xA;               ylab = 'Gene expression', xlab=FALSE,&#xA;               order=c(&quot;Normal&quot;, &quot;Tumor&quot;))&#xA;q + stat_compare_means(method = &quot;t.test&quot;) + &#xA;  geom_point() + &#xA;  stat_n_text()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I see many black dots in the middle of the boxplot like this &lt;a href=&quot;https://i.stack.imgur.com/wJPmQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wJPmQ.png&quot; alt=&quot;Boxplot2&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;May I know why I see those dots in the second boxplot after removing the jitter. Is there a way to avoid that?&lt;/p&gt;&#xA;" OwnerUserId="1369" LastEditorUserId="1369" LastEditDate="2018-11-23T12:37:51.070" LastActivityDate="2018-11-23T17:17:40.800" Title="Question about the dots on Quartile groups in boxplot" Tags="|r|rna-seq|microarray|ggplot2|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5488" PostTypeId="2" ParentId="5483" CreationDate="2018-11-23T10:32:53.290" Score="6" Body="&lt;p&gt;Found a solution, using &lt;a href=&quot;http://dgenies.toulouse.inra.fr/&quot; rel=&quot;noreferrer&quot;&gt;D-Genies&lt;/a&gt;, worked great. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some examples from their website:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Hhnul.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Hhnul.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks to &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/37/user172818&quot;&gt;@user172818&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2018-11-23T10:32:53.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5489" PostTypeId="1" AcceptedAnswerId="5492" CreationDate="2018-11-23T12:49:28.110" Score="2" ViewCount="224" Body="&lt;p&gt;I am working with an amplicon kit from Swift Bioscience and they provide the primer locations in what they call a &quot;masterfile&quot;. The format is like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ head -n2 Acel-Amplicon_56g_masterfile.txt&#xA;1       43814951        43815064        MPL_S505N-A519T_F/2     43814932        43814951        MPL_S505N-A519T_F/2_primer      43815064  43815081 MPL_S505N-A519T_R/1_primer      GCCTGGATCTCCTTGGTGA     CTTCGGCTCCACCTGGT&#xA;1       115256480       115256576       NRAS_G60-S65_new_R/1    115256457       115256480       NRAS_G60-S65_new_R/1_primer     115256576 115256602        NRAS_G60-S65_new_F/2_primer     TGATGGCAAATACACAGAGGAAG CAGGATTCTTACAGAAAACAAGTGGT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the format which is understood by &lt;a href=&quot;https://github.com/swiftbiosciences/primerclip&quot; rel=&quot;nofollow noreferrer&quot;&gt;Swift's primer clipping tool&lt;/a&gt;, but in order to use other tools, I would like to convert it to BEDPE so I can then use standard tools to clip the primers from my bam files. However, I can't do that since I am not sure what the fields in this file are. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I think thy are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Chromosome&lt;/li&gt;&#xA;&lt;li&gt;Target region start&lt;/li&gt;&#xA;&lt;li&gt;Target region end&lt;/li&gt;&#xA;&lt;li&gt;5' Primer name&lt;/li&gt;&#xA;&lt;li&gt;5' Primer start&lt;/li&gt;&#xA;&lt;li&gt;5' Primer end&lt;/li&gt;&#xA;&lt;li&gt;3' Primer name&lt;/li&gt;&#xA;&lt;li&gt;3' Primer start&lt;/li&gt;&#xA;&lt;li&gt;3' Primer end&lt;/li&gt;&#xA;&lt;li&gt;5' Primer sequence&lt;/li&gt;&#xA;&lt;li&gt;3' primer sequence&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Is my interpretation correct? Also, is this a standard file format or something specific to the Swift tool? I couldn't find any documentation about a &quot;masterfile&quot; format online.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-11-24T16:49:21.367" LastActivityDate="2018-11-24T17:27:42.603" Title="What is the &quot;primer masterfile&quot; format?" Tags="|file-formats|format-conversion|primer|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5490" PostTypeId="2" ParentId="4341" CreationDate="2018-11-23T15:41:29.863" Score="2" Body="&lt;p&gt;Package &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/hipathia.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hipathia&lt;/a&gt; allows you to compute a value of &quot;activity&quot; for each pathway and each cell, so that the matrix of gene expression is transformed into a matrix of pathway activity. You can use this matrix for further analysis, cell differentiation and clustering. You can also compute functional activity matrices in the same way. &lt;/p&gt;&#xA;" OwnerUserId="3723" LastEditorUserId="492" LastEditDate="2018-11-23T18:47:41.463" LastActivityDate="2018-11-23T18:47:41.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5491" PostTypeId="2" ParentId="5487" CreationDate="2018-11-23T17:17:40.800" Score="3" Body="&lt;p&gt;You add the points with &lt;code&gt;geom_point()&lt;/code&gt;. Just remove it and you will get your &quot;empty&quot; boxplot. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;q &amp;lt;- ggboxplot(B, x = &quot;Type&quot;, y = &quot;Gene&quot;,&#xA;               color = &quot;black&quot;, palette = &quot;npg&quot;,&#xA;               ylab = 'Gene expression', xlab=FALSE,&#xA;               order=c(&quot;Normal&quot;, &quot;Tumor&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unfortunately I couldn't use &lt;code&gt;stat_compare_means(method = &quot;t.test&quot;)&lt;/code&gt; and     &lt;code&gt;stat_n_text()&lt;/code&gt;, but this two just add the labels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uUDCR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uUDCR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2018-11-23T17:17:40.800" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5492" PostTypeId="2" ParentId="5489" CreationDate="2018-11-24T05:27:50.647" Score="2" Body="&lt;p&gt;this is the swift standard primer file. You are a bit off in your interpretation of the columns (seems like you missed one as there are 12 columns, not 11 which threw everything off):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Chromosome&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Target Start&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Target Stop&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) &lt;strong&gt;Target Name&lt;/strong&gt; (change which affected everything downstream in counting) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;5) 5' Primer Start&lt;/p&gt;&#xA;&#xA;&lt;p&gt;6) 5' Primer Stop&lt;/p&gt;&#xA;&#xA;&lt;p&gt;7) 5' Primer Name&lt;/p&gt;&#xA;&#xA;&lt;p&gt;8) 3' Primer Start&lt;/p&gt;&#xA;&#xA;&lt;p&gt;9) 3' Primer Stop&lt;/p&gt;&#xA;&#xA;&lt;p&gt;10) 3' Primer Name&lt;/p&gt;&#xA;&#xA;&lt;p&gt;11) 5' Primer Sequence&lt;/p&gt;&#xA;&#xA;&lt;p&gt;12) 3' Primer Sequence&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To convert to a BEDPE describing the primer locations, you only will need columns 1, 5, 6, 8, and 9 (and optionally 4). The last letter of the Region name for the denotes the strand orientation of the region. For example Region 1 has _F denoting forward or (+) strand for BEDPE usage, while region 2 has _R in name denoting reverse or (-) strand for BEDPE. Putting all this together, you can convert to BEDPE with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk -v OFS=&quot;\t&quot; '{ &#xA;            o=&quot;.&quot;; &#xA;            if($4~/_F\//){&#xA;                o=&quot;+&quot;&#xA;            }&#xA;            else if($4~/_R\//){&#xA;                o=&quot;-&quot;&#xA;            } &#xA;            print $1,$5,$6,$1,$8,$9,$&lt;/span&gt;4,&quot;.&quot;,o}' foo.masterfile&#xA;1   43814932    43814951    1   43815064    43815081    MPL_S505N-A519T_F/2 .   +&#xA;1   115256457   115256480   1   115256576   115256602   NRAS_G60-S65_new_R/1    .   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3669" LastEditorUserId="298" LastEditDate="2018-11-24T17:27:42.603" LastActivityDate="2018-11-24T17:27:42.603" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5493" PostTypeId="2" ParentId="5427" CreationDate="2018-11-24T12:59:09.717" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://broadinstitute.github.io/picard/command-line-overview.html#CollectHsMetrics&quot; rel=&quot;nofollow noreferrer&quot;&gt;picard CollectHsMetrics&lt;/a&gt; has the parameter &lt;code&gt;PER_BASE_COVERAGE&lt;/code&gt; which take this into account.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can run it like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; $ java -jar picard.jar CollectHsMetrics \&#xA;      I=input.bam \&#xA;      O=hs_metrics.txt \&#xA;      R=reference_sequence.fasta \&#xA;      BAIT_INTERVALS=bait.interval_list \&#xA;      TARGET_INTERVALS=target.interval_list \&#xA;      PER_BASE_COVERAGE=base_coverage.csv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;BAIT_INTERVALS&lt;/code&gt; and &lt;code&gt;TARGET_INTERVALS&lt;/code&gt; could be the same file. You will need to convert your &lt;code&gt;bed&lt;/code&gt; file to it using &lt;a href=&quot;http://broadinstitute.github.io/picard/command-line-overview.html#BedToIntervalList&quot; rel=&quot;nofollow noreferrer&quot;&gt;picard BedToIntervalList&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another useful tool is &lt;a href=&quot;https://github.com/brentp/mosdepth&quot; rel=&quot;nofollow noreferrer&quot;&gt;mosdepth&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ mosdepth --by regions.bed SampleName input.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;fin swimmer &lt;/p&gt;&#xA;" OwnerUserId="1310" LastEditorUserId="1310" LastEditDate="2018-11-24T17:40:35.280" LastActivityDate="2018-11-24T17:40:35.280" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5494" PostTypeId="1" CreationDate="2018-11-24T17:40:49.937" Score="4" ViewCount="131" Body="&lt;p&gt;I'd like to evaluate a &lt;a href=&quot;https://github.com/bjtrost/TCAG-WGS-CNV-workflow&quot; rel=&quot;nofollow noreferrer&quot;&gt;CNV project&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My aim is to evaluate if the scripts are sufficient for calling reasonable CNVs. I know they have a paper, but their scripts may be buggy... and all papers are biased...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While there are CNV simulators on Google search, none seem popular in the community. How should I approach this problem?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Simulate CNV variants from a genome&lt;/li&gt;&#xA;&lt;li&gt;Simple evaluation on the called variants against my simulation?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="174" LastEditorUserId="48" LastEditDate="2018-11-26T08:43:59.063" LastActivityDate="2019-04-25T09:02:15.537" Title="Simulate and test CNV workflow?" Tags="|ngs|genome|cnv|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5495" PostTypeId="1" CreationDate="2018-11-24T18:45:22.610" Score="1" ViewCount="223" Body="&lt;p&gt;What is a good method to reconstruct a protein's 3D structure from a pair-wise distance matrix of residues?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the question is self-explanatory. I know of this &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/31/21/3499/194596&quot; rel=&quot;nofollow noreferrer&quot;&gt;method&lt;/a&gt; &#xA;but it seems that it uses contact maps, which actually use a certain cut-off. I don't have a cutoff based contact matrix but a matrix with actual distances.&lt;/p&gt;&#xA;" OwnerUserId="3727" LastEditorUserId="48" LastEditDate="2018-11-26T08:53:19.467" LastActivityDate="2018-11-26T08:53:19.467" Title="Reconstruct a protein's 3D structure from a pair-wise distance matrix" Tags="|pdb|protein-structure|3d-structure|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5496" PostTypeId="1" CreationDate="2018-11-24T19:17:09.107" Score="5" ViewCount="1137" Body="&lt;p&gt;I have a number of sequences and a reference genome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used &lt;a href=&quot;http://www.vicbioinformatics.com/software.snippy.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;snippy&lt;/code&gt;&lt;/a&gt; to align each individual sequence with the reference genome. I then used &lt;code&gt;snippy-core&lt;/code&gt; on the &lt;code&gt;*.aln&lt;/code&gt; files from &lt;code&gt;snippy&lt;/code&gt; which produced a new &lt;code&gt;*.aln&lt;/code&gt; file, and a &lt;code&gt;*.full.aln&lt;/code&gt;. &#xA;It is my understanding that the new &lt;code&gt;*.aln&lt;/code&gt; file contains a core SNP alignment with only bases shared by all of my sequences, and that the &lt;code&gt;*.full.aln&lt;/code&gt; file contains the core genome alignment with all bases of each sequence aligned to the reference genome - regardless of variation between sequences. Is my understanding correct? If not where am I going wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further, the program &lt;code&gt;snp-sites&lt;/code&gt; uses both the new &lt;code&gt;*.aln&lt;/code&gt; and &lt;code&gt;*.full.aln&lt;/code&gt; file to produce a file which can be run through &lt;code&gt;FastTree&lt;/code&gt; to produce a phylogenetic tree. Does this just process the files to give the phylogenetic distance between each of my sequences? I don't really understand what this step achieves at all...&lt;/p&gt;&#xA;" OwnerUserId="3728" LastEditorUserId="3848" LastEditDate="2020-05-06T09:18:38.253" LastActivityDate="2022-02-23T05:59:25.020" Title="Does the &quot;.full.aln&quot; file produced by snippy-core contain all bases of my input sequences aligned to the reference genome?" Tags="|sequence-alignment|snp|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5497" PostTypeId="2" ParentId="5495" CreationDate="2018-11-24T21:47:58.860" Score="-1" Body="&lt;p&gt;Finding coordinates of N points that preserve (as well as possible) pair-wise distances from an NxN matrix is called &lt;a href=&quot;https://en.wikipedia.org/wiki/Multidimensional_scaling&quot; rel=&quot;nofollow noreferrer&quot;&gt;multidimensional scaling&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I don't enough about the context of your problem to tell if/how it is applicable.&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2018-11-24T21:47:58.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5498" PostTypeId="1" CreationDate="2018-11-24T23:45:19.047" Score="1" ViewCount="336" Body="&lt;p&gt;I have a BED6 (BED + name, score, strand info) file that defines some regions of interest.  I also have a set of BAMs corresponding to different samples.  I would like to obtain output similar to &lt;code&gt;bedtools genomecov -d -5&lt;/code&gt;, which generates counts of read start positions, but I would like it restricted to regions of interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can do this for individual BAMs and regions by filtering input with &lt;code&gt;samtools view&lt;/code&gt;, e.g.,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools genomecov -d -5 -strand '+' -ibam &amp;lt;(samtools view -b sample1.bam &quot;chr1:12345-23456&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but this requires extracting position and strand info from the BED file, and then eventually I have to aggregate all the outputs back into a single file. This ends up being very slow (e.g., hours for ~50 regions of interest X 12 samples).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Is there a better way to accomplish this?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, something along the lines of how &lt;code&gt;bedtools multicov&lt;/code&gt; works would be great:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools multicov -bams /path/to/my/bams/*.bam -bed roi.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but it doesn't have the &lt;code&gt;-d&lt;/code&gt; and &lt;code&gt;-3/5&lt;/code&gt; options to extract single-nucleotide read-end coverage.&lt;/p&gt;&#xA;" OwnerUserId="900" LastActivityDate="2018-11-24T23:45:19.047" Title="bedtools single-nucleotide coverage in BED-specified regions for multiple BAMs" Tags="|sam|samtools|bed|bedtools|coverage|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5499" PostTypeId="2" ParentId="5494" CreationDate="2018-11-25T09:20:53.280" Score="1" Body="&lt;p&gt;I cannot tell exactly from the post or the linked project what type of data these scripts are designed for, but I am going to assume that &quot;Scripts involved in our workflow for detecting CNVs from WGS data using read depth-based methods&quot; refers to short-read Illumina sequencing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simulations can be helpful, but they are usually imperfect. Nowadays we have long-read data available. I would suggest obtaining some long-read data for which short-read data is also available, doing CNV detection on both datasets independently, and comparing the results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with using long-read data is that any differences you find will be conflated with the biases of the technology. So, if you find a difference, then this approach will not tell you if the differences are due to the biases of the technologies, or due to bugs in the scripts. If you use simulations, then this conflation of biases remains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Short of re-writing the pipeline, the only other suggestion I have is to borrow from software testing. Try constructing a few minimal examples that might also test some corner cases and see if the scripts behave exactly as expected. &lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="492" LastEditDate="2018-11-25T09:29:39.173" LastActivityDate="2018-11-25T09:29:39.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5501" PostTypeId="2" ParentId="5467" CreationDate="2018-11-25T12:37:27.230" Score="1" Body="&lt;p&gt;You don't say which version of MEGA you are using, but this works for MEGA X on Linux:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Open your alignment with the &quot;Sequence data explorer&quot;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;At the &quot;Highlight&quot; menu, select &quot;Variable sites&quot;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then at the menu &quot;Statistics&quot;, select &quot;Use only highlighted sites&quot; and select what you consider an appropriate output - I choose &quot;Display results as CSV&quot;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now at the menu &quot;Data&quot; select &quot;Export data&quot;; at the &quot;Sequence data export options window&quot; if necessary select the data format again, and select &quot;For each site&quot; at &quot;Writing site numbers&quot;, and select &quot;Only highlighted sites&quot; at &quot;Selected sites to include&quot;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This will result in a comma-separated values file, showing only positions with variable sites.&lt;/p&gt;&#xA;" OwnerUserId="1642" LastActivityDate="2018-11-25T12:37:27.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5502" PostTypeId="2" ParentId="4795" CreationDate="2018-11-25T17:23:19.423" Score="1" Body="&lt;p&gt;The tool that you want to use for this task is &lt;code&gt;bedtools window&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3732" LastActivityDate="2018-11-25T17:23:19.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5503" PostTypeId="2" ParentId="5444" CreationDate="2018-11-25T18:52:24.373" Score="1" Body="&lt;p&gt;It would be useful if you show us some line of your &lt;code&gt;CADD.bed&lt;/code&gt; and &lt;code&gt;Input.vcf&lt;/code&gt; and how your desired output should look like.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;bedtools&lt;/code&gt; is complaining about non positional records in &lt;code&gt;CADD.bed&lt;/code&gt;, which means there are lines in this file where the first three columns aren't &lt;code&gt;CHROM&lt;/code&gt;, &lt;code&gt;START&lt;/code&gt;, &lt;code&gt;END&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess you will have some header lines in the beginning that aren't introduced by an &lt;code&gt;#&lt;/code&gt;. Just delete them (or add in &lt;code&gt;#&lt;/code&gt; to the beginning of the line) and try again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;fin swimmer&lt;/p&gt;&#xA;" OwnerUserId="1310" LastEditorUserId="73" LastEditDate="2019-02-15T10:16:36.767" LastActivityDate="2019-02-15T10:16:36.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5504" PostTypeId="2" ParentId="3670" CreationDate="2018-11-26T01:59:37.483" Score="0" Body="&lt;p&gt;This program in C++ is able to perform the task &lt;a href=&quot;https://github.com/mlafave/sam2pairwise&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mlafave/sam2pairwise&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="60" LastActivityDate="2018-11-26T01:59:37.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5505" PostTypeId="1" CreationDate="2018-11-26T05:13:55.587" Score="1" ViewCount="193" Body="&lt;p&gt;I have performed peak calling on a number of separate ChIP-seq experiments and would like to harmonise the peaks from each of the experiments in order to convert my data into a matrix for further analysis. I would like to do this by taking the union of the called peaks (available in BED / narrowPeak / MACS2 format) and retrieving the enrichment score / q-value for all of the experiments on each peak. Is this possible without writing my own extension to MACS2?&lt;/p&gt;&#xA;" OwnerUserId="163" LastActivityDate="2018-12-29T21:00:45.120" Title="Score predefined ChIP-seq peaks with MACS2 or equivalent" Tags="|chip-seq|peak-calling|macs2|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5506" PostTypeId="2" ParentId="5483" CreationDate="2018-11-26T07:26:32.977" Score="1" Body="&lt;p&gt;Also, &lt;a href=&quot;https://github.com/marbl/MashMap&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mashmap&lt;/a&gt; has an embedded &lt;a href=&quot;https://github.com/marbl/MashMap#visualize&quot; rel=&quot;nofollow noreferrer&quot;&gt;visualisation tool&lt;/a&gt; generating dot-plots to visualize mappings, ideal for:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;highly similar species or to compare different assemblies of the same species. &lt;/li&gt;&#xA;&lt;li&gt;mapping genome assembly or long reads (PacBio/ONT) to reference genome&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;An example from their website: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uIvbw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uIvbw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2018-11-26T07:26:32.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5507" PostTypeId="1" CreationDate="2018-11-26T12:07:49.397" Score="4" ViewCount="3326" Body="&lt;p&gt;After some &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2846/why-does-a-very-strong-blast-hit-get-lost-when-i-change-num-alignments-num-desc&quot;&gt;unexpected results&lt;/a&gt; (and &lt;a href=&quot;https://blastedbio.blogspot.com/2018/11/blast-max-alignment-limits-part-four.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;previously reported&lt;/a&gt;) I heard that there are other tools for finding similar sequences besides blast that are faster and more accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I only found &lt;a href=&quot;http://hmmer.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;hmmer&lt;/a&gt;, but I don't know if it is faster and more accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any review comparing such tools? A quick google scholar search didn't report anything useful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to search with short DNA sequences (~400), mainly interested for taxonomic imputation so I'll be looking for similar sequences and not distant homologs. I won't try to model genes and the sequence will be unspliced as it is from direct sequencing of a a region of the 16S gene.&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="964" LastEditDate="2018-11-26T15:56:38.320" LastActivityDate="2019-09-13T06:26:58.957" Title="Fast and reliable alternatives to blast" Tags="|sequence-alignment|blast|literature-search|" AnswerCount="5" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="5508" PostTypeId="2" ParentId="5482" CreationDate="2018-11-26T12:59:53.623" Score="2" Body="&lt;p&gt;You could try aligning your reads to the draft reference genome with for example &lt;code&gt;minimap2&lt;/code&gt; and calling variants with &lt;code&gt;freebayes&lt;/code&gt;. It appears there is a protocol for long reads: &lt;a href=&quot;https://github.com/ekg/freebayes#re-genotyping-known-variants-and-calling-long-haplotypes&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ekg/freebayes#re-genotyping-known-variants-and-calling-long-haplotypes&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclaimer: I have not run &lt;code&gt;freebayes&lt;/code&gt; myself, and it may be that this is a terrible idea. I would expect that calling SNVs (Single-nucleotide variants) from PacBio reads is very challenging.  However, with enough coverage, I think it could be possible.&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2018-11-26T12:59:53.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5509" PostTypeId="1" AcceptedAnswerId="5516" CreationDate="2018-11-26T13:58:03.200" Score="7" ViewCount="3141" Body="&lt;p&gt;I have two DNA strings:  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;GGAGGAGGGAGAAGGAGGGAGGGAAGAGGAGGGAGAAGGAGGGAGGC &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;AGAAGGAGGGAGGGAAGAGGAGGGAGAAGGAGGGAGGGAAGAGGAGG&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I want a tool that allows me to do something like this on the command line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ aligner  GGAGGAGGGAGAAGGAGGGAGGGAAGAGGAGGGAGAAGGAGGGAGGC AGAAGGAGGGAGGGAAGAGGAGGGAGAAGGAGGGAGGGAAGAGGAGG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and receive an ASCII visualization of a pairwise alignment. Something like this would work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GGAGGAGGGAGAAGGAGGGAGGGAAGAGGAGGGAGAAGGAGGGAGGC---------&#xA;---------AGAAGGAGGGAGGGAAGAGGAGGGAGAAGGAGGGAGGGAAGAGGAGG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I prefer a tool that is packaged on bioconda. A tool that also does protein and RNA sequences is even more preferable.&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="298" LastEditDate="2018-11-26T16:38:25.470" LastActivityDate="2018-11-29T04:32:19.963" Title="What is a simple command line tool for doing Needleman-Wunsch pair-wise alignment on the command line" Tags="|sequence-alignment|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5510" PostTypeId="2" ParentId="5509" CreationDate="2018-11-26T15:40:59.550" Score="9" Body="&lt;p&gt;Install biopython with conda and use the following script:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python&#xA;import sys&#xA;from Bio import Align&#xA;aligner = Align.PairwiseAligner()&#xA;aligner.mode = &quot;local&quot;&#xA;alignments = aligner.align(sys.argv[1], sys.argv[2])&#xA;print(alignments[0])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output is then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;.GGA-GGAGGGAG--AAGGAGGGAGGGAAGA-GGAGGGAG--AAGGAGGGAGGC&#xA;.|-|-||||||||--|||-|||-|||||-||-||||||||--|||-|||-|||.&#xA;AG-AAGGAGGGAGGGAAG-AGG-AGGGA-GAAGGAGGGAGGGAAG-AGG-AGG.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are a LOT of alignments with that same score and you can tweak the details of the mode by changing &lt;code&gt;aligner.mode&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-11-26T15:40:59.550" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5511" PostTypeId="1" CreationDate="2018-11-26T16:07:23.403" Score="1" ViewCount="17" Body="&lt;p&gt;We have to find the conserved region of proteins in a given set of kingdom and we have to give more weightage to distantly related organism. What approach I should take to solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="3737" LastActivityDate="2018-11-26T16:07:23.403" Title="Finding the conserved region of protein in a given set of Kingdom" Tags="|sequence-alignment|proteins|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5512" PostTypeId="2" ParentId="5509" CreationDate="2018-11-26T16:37:33.573" Score="4" Body="&lt;p&gt;I'd just use something like &lt;code&gt;t_coffee&lt;/code&gt; and write a little bash function to run it on the strings you give:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;aligner(){&#xA;  tmp=&lt;span class=&quot;math-container&quot;&gt;$(mktemp);&#xA;  args=(&quot;$&lt;/span&gt;@&quot;)&#xA;  for ((i=0; i&amp;lt;&lt;span class=&quot;math-container&quot;&gt;$#; i++)); do&#xA;    printf '&amp;gt;%s\n%s\n' $&lt;/span&gt;i &lt;span class=&quot;math-container&quot;&gt;${args[i]} &amp;gt;&amp;gt; &quot;$&lt;/span&gt;tmp&quot;&#xA;  done&#xA;  t_coffee &quot;&lt;span class=&quot;math-container&quot;&gt;$tmp&quot;  2&amp;gt;/dev/null | /bin/grep -Pv '^CLUSTAL|^\s*$&lt;/span&gt;'&#xA;  rm &quot;$tmp&quot;&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you add that to your shell's initialization file (e.g. &lt;code&gt;~/.bashrc&lt;/code&gt; if using bash on Linux, &lt;code&gt;~/.profile&lt;/code&gt; if using bash on macOS), you can run &lt;code&gt;align2seqs&lt;/code&gt; with as many sequences as you want to align:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ aligner 'GGAGGAGGGAGAAGGAGGGAGGGAAGAGGAGGGAGAAGGAGGGAGGC' 'AGAAGGAGGGAGGGAAGAGGAGGGAGAAGGAGGGAGGGAAGAGGAGG'&#xA;0               GG--AGGAGGGAGAAGGAGGGAGGGAAGAGGAGGGAGAAGGAGGGAGGC&#xA;1               nAGAAGGAGGGAGGGAAGAGGAGGGAGAAGGAGGGAGG-GAAGAGGAGG&#xA;                    *********      *******  *********  * ** *  * &#xA;$&lt;/span&gt; aligner 'GAGGGAGAAGGAGGGAGGGAAGAGGAGGGAGAAGGAGGGAGGC' 'GGAGGGAGGGAAGAGGAGGGAGAAGGAGGGAGGGAAGAGGAGG' GAGGGAGGGAAGAGGAGGGAGAAGGATCCGAGGAAGAGGAGG AGAAGGAGGGAGGGAAGAGGCCCGAGATATATAAGGATCCGAGGAA&#xA;0               GA-GG----GAGAAGGAGGGAGGGAAGAGGAGGGAGAAGG--AGGGAGGC&#xA;1               GGAGG----GAGGGAAGAGGAGGGAGAAGGAGGGAG-GGA--AGAGGAGG&#xA;2               GA-GG----GAGGGAAGAGGAGGGAGAAGGATCCGA-GGA--AGAGGAGG&#xA;3               AGAAGGAGGGAGGGAAGAGGCCCGAGATATAT--AA-GGATCCGAG-GAA&#xA;                    *    ***      **   **     *       *    * *    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-11-26T16:37:33.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5513" PostTypeId="1" CreationDate="2018-11-26T18:20:36.183" Score="3" ViewCount="358" Body="&lt;p&gt;I installed genometools with anaconda &lt;code&gt;conda install -c bioconda genometools&lt;/code&gt;. Something is installed though apparently no binary called &lt;code&gt;gt&lt;/code&gt;. If I type in gt Ubuntu suggests to install genometools via &lt;code&gt;apt&lt;/code&gt;. What is the difference between both versions? Why is there no &lt;code&gt;gt&lt;/code&gt; binary in the anaconda package?&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2018-11-26T19:21:48.580" Title="Difference between genometools installed with anaconda vs apt-get?" Tags="|bioconda|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5514" PostTypeId="1" AcceptedAnswerId="5521" CreationDate="2018-11-26T18:23:51.213" Score="8" ViewCount="1409" Body="&lt;p&gt;BWA mem has the &lt;code&gt;-S&lt;/code&gt; and &lt;code&gt;-P&lt;/code&gt; tags for &lt;em&gt;skip mate rescue&lt;/em&gt; and &lt;em&gt;skip pairing; mate rescue performed unless -S also in use&lt;/em&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do these do? I presume &lt;code&gt;-P&lt;/code&gt; aligns read pairs independently of each other. Is that correct? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And what does &lt;code&gt;-S&lt;/code&gt; do?&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2018-11-27T06:08:12.213" Title="What is mate rescue in bwa mem?" Tags="|read-mapping|bwa|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5515" PostTypeId="2" ParentId="5513" CreationDate="2018-11-26T19:21:48.580" Score="7" Body="&lt;p&gt;These appear to be two separate collections of tools. The conda version is a collection of python scripts whereas the apt version is based on a C library. So, as you might expect, the python version does not have a &lt;code&gt;gt&lt;/code&gt; binary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They also have separate GitHub accounts &lt;a href=&quot;https://github.com/flo-compbio/genometools&quot; rel=&quot;noreferrer&quot;&gt;conda genometools&lt;/a&gt; vs &lt;a href=&quot;https://github.com/genometools/genometools&quot; rel=&quot;noreferrer&quot;&gt;apt genometools&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3499" LastActivityDate="2018-11-26T19:21:48.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5516" PostTypeId="2" ParentId="5509" CreationDate="2018-11-26T20:13:32.210" Score="11" Body="&lt;p&gt;You are looking for the &lt;code&gt;needle&lt;/code&gt; program from the &lt;code&gt;EMBOSS&lt;/code&gt; suite. Available in bioconda. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://emboss.sourceforge.net/apps/release/6.6/emboss/apps/needle.html&quot; rel=&quot;noreferrer&quot;&gt;http://emboss.sourceforge.net/apps/release/6.6/emboss/apps/needle.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To read sequences from the commandline, you need so specify the format as &lt;code&gt;asis&lt;/code&gt;. To get the output on the screen, you'll need &lt;code&gt;-stdout&lt;/code&gt; and to use the default alignment parameters (gap penalty 10, extend_penalty 0.5) you'll need &lt;code&gt;-auto&lt;/code&gt;. Thus your query above would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ needle -asequence asis:GGAGGAGGGAGAAGGAGGGAGGGAAGAGGAGGGAGAAGGAGGGAGGC -bsequence asis:AGAAGGAGGGAGGGAAGAGGAGGGAGAAGGAGGGAGGGAAGAGGAGG -auto -stdout&#xA;########################################&#xA;# Program: needle&#xA;# Rundate: Tue 27 Nov 2018 10:48:50&#xA;# Commandline: needle&#xA;#    -asequence asis:GGAGGAGGGAGAAGGAGGGAGGGAAGAGGAGGGAGAAGGAGGGAGGC&#xA;#    -bsequence asis:AGAAGGAGGGAGGGAAGAGGAGGGAGAAGGAGGGAGGGAAGAGGAGG&#xA;#    -auto&#xA;#    -stdout&#xA;# Align_format: srspair&#xA;# Report_file: stdout&#xA;########################################&#xA;&#xA;#=======================================&#xA;#&#xA;# Aligned_sequences: 2&#xA;# 1: asis&#xA;# 2: asis&#xA;# Matrix: EDNAFULL&#xA;# Gap_penalty: 10.0&#xA;# Extend_penalty: 0.5&#xA;#&#xA;# Length: 56&#xA;# Identity:      37/56 (66.1%)&#xA;# Similarity:    37/56 (66.1%)&#xA;# Gaps:          18/56 (32.1%)&#xA;# Score: 181.0&#xA;# &#xA;#&#xA;#=======================================&#xA;&#xA;asis               1 GGAGGAGGGAGAAGGAGGGAGGGAAGAGGAGGGAGAAGGAGGGAGGC---     47&#xA;                              |||||||||||||||||||||||||||||||||||||.   &#xA;asis               1 ---------AGAAGGAGGGAGGGAAGAGGAGGGAGAAGGAGGGAGGGAAG     41&#xA;&#xA;asis              47 ------     47&#xA;&#xA;asis              42 AGGAGG     47&#xA;&#xA;&#xA;#---------------------------------------&#xA;#---------------------------------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="235" LastEditorUserId="235" LastEditDate="2018-11-27T10:51:10.863" LastActivityDate="2018-11-27T10:51:10.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5517" PostTypeId="2" ParentId="5514" CreationDate="2018-11-26T20:24:57.097" Score="2" Body="&lt;p&gt;If one read maps and the other doesn't, BWA attempts to rescue the other read by performing Smith-Waterman alignment with the unmapped mate, &lt;code&gt;-S&lt;/code&gt; disables this mate rescue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;-P&lt;/code&gt; does indeed disables pairing, but mate rescues is still enabled - I think &lt;code&gt;-P&lt;/code&gt; disables the setting of the proper pair flag. If you set &lt;code&gt;-P&lt;/code&gt; and &lt;code&gt;-S&lt;/code&gt;, BWA maps the paired reads essentially as single reads.&lt;/p&gt;&#xA;" OwnerUserId="1642" LastActivityDate="2018-11-26T20:24:57.097" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5518" PostTypeId="1" CreationDate="2018-11-26T23:44:50.667" Score="3" ViewCount="623" Body="&lt;p&gt;I have a single VCF file of variants for a population that includes males &amp;amp; females and I would like to pull out all variants that are sex-specific. For example, sites where all males are 0/1 and all females are 0/0. Is there a way to do this using bcftools or vcftools? I could enumerate each individual name, or ask for all individuals named *.M and *.F.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I first tried running smoove on the male and female populations separately then comparing the two VCFs, but I'm working with a draft genome with loads of contigs. As only contigs containing variants are output in each VCF, and these are different between the two files, vcftools couldn't handle the comparison. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Someone here &lt;a href=&quot;https://github.com/samtools/bcftools/issues/118&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/samtools/bcftools/issues/118&lt;/a&gt; managed to get it working ike this: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$bcftools view -f PASS -i 'GT~&quot;0/0.*0/1.*0/1.*0/1.*0/1&quot;' combined.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but the same syntax gives me no variants (yet I can see some that meet my criteria in IGV).&lt;/p&gt;&#xA;" OwnerUserId="3505" LastEditorUserId="3505" LastEditDate="2018-11-27T00:06:12.497" LastActivityDate="2023-02-18T12:46:07.230" Title="how to filter a multi-VCF for sex-specific genotypes?" Tags="|vcf|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5519" PostTypeId="2" ParentId="5518" CreationDate="2018-11-27T01:20:35.917" Score="2" Body="&lt;p&gt;Found a great tool with &lt;a href=&quot;http://snpeff.sourceforge.net/SnpSift.html#filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;SnpSift&lt;/a&gt;!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the examples: I want to keep samples where the genotype for the first sample is homozygous variant and the genotype for the second sample is reference:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat variants.vcf | java -jar SnpSift.jar filter &quot;isHom( GEN[0] ) &amp;amp; isVariant( GEN[0] ) &amp;amp; isRef( GEN[1] )&quot; &amp;gt; filtered.vcf &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3505" LastEditorUserId="48" LastEditDate="2018-11-27T10:23:26.713" LastActivityDate="2018-11-27T10:23:26.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5520" PostTypeId="2" ParentId="5518" CreationDate="2018-11-27T05:30:00.300" Score="1" Body="&lt;p&gt;if you could provide seperate list of male and female samples names you can something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools view -e 'GT[@males.txt]!=&quot;0/1&quot; &amp;amp;&amp;amp; GT[@females.txt]!=&quot;0/0&quot;' combined.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or if you don't want to take care of the exact genotype you can write it more generic:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools view -e 'GT[@males.txt]!=&quot;alt&quot; &amp;amp;&amp;amp; GT[@females.txt]!=&quot;ref&quot;' combined.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;fin swimmer&lt;/p&gt;&#xA;" OwnerUserId="1310" LastEditorUserId="1310" LastEditDate="2018-11-27T05:57:03.123" LastActivityDate="2018-11-27T05:57:03.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5521" PostTypeId="2" ParentId="5514" CreationDate="2018-11-27T06:08:12.213" Score="6" Body="&lt;p&gt;Using &lt;code&gt;-SP&lt;/code&gt; is equivalent to running &lt;code&gt;bwa mem&lt;/code&gt; on each of the two mates as if they are single-end reads, but it formats the output as a proper paired-end output, i.e. with all pair-related flags added properly. Without &lt;code&gt;-SP&lt;/code&gt;, by default &lt;code&gt;bwa mem&lt;/code&gt; forces an alignment of a poorly aligned read if its mate is aligned somewhere. &lt;code&gt;-SP&lt;/code&gt; turns off the forced alignment. We use &lt;code&gt;-SP&lt;/code&gt; for mapping Hi-C reads.&lt;/p&gt;&#xA;" OwnerUserId="3745" LastActivityDate="2018-11-27T06:08:12.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5522" PostTypeId="2" ParentId="5507" CreationDate="2018-11-27T07:21:36.237" Score="4" Body="&lt;p&gt;Besides generic nucleotide aligners, there are also more specialized tools for the alignment of 16S amplicon sequences, e.g. SINA (&lt;a href=&quot;https://academic.oup.com/bioinformatics/article/28/14/1823/218226&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;SINA article&quot;&gt;article&lt;/a&gt;, &lt;a href=&quot;https://github.com/epruesse/SINA&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;SINA at github&quot;&gt;software&lt;/a&gt;) which is part of &lt;a href=&quot;https://www.arb-silva.de/&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;SILVA website&quot;&gt;SILVA&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1802" LastEditorUserId="48" LastEditDate="2019-01-28T08:24:16.763" LastActivityDate="2019-01-28T08:24:16.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5523" PostTypeId="1" CreationDate="2018-11-27T07:26:33.620" Score="2" ViewCount="193" Body="&lt;p&gt;I'm taking data from cbioportal of gene and their mutation frequency with percentage which I'm trying to categorize with various sub class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue I'm facing is I have to plot it in mutation frequency which are already given. I'm not sure how to go about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my code (I'm attaching the data), but what I'm missing is the frequency count of patients. As an example NPM1 altered in 50 patient with 31.00% where the number of patient is 162, I'm going for stacked bar plot, as the frequency is already given, how can I just plot in terms of frequency ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://drive.google.com/open?id=1k8uIQ5hKHyMoMJB8V0Hdc1F17aD5T2s8&quot; rel=&quot;nofollow noreferrer&quot;&gt;data&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df.m1 &amp;lt;- reshape2::melt(ALL_NEW_RBP, id.vars = NULL)&#xA;&#xA;&#xA;library(dplyr)&#xA;d3 &amp;lt;- df.m1 %&amp;gt;% &#xA;  group_by(variable,value) %&amp;gt;% &#xA;  summarise(count=n()) %&amp;gt;% &#xA;  mutate(perc=count/sum(count))&#xA;&#xA;colnames(d3) &amp;lt;- c(&quot;variable&quot;,&quot;value&quot;,&quot;count&quot;,&quot;percentage&quot;)&#xA;&#xA;cbPalette &amp;lt;- c(&quot;black&quot;,&quot;darkblue&quot;,&quot;brown4&quot;,&quot;darkred&quot;,&quot;darkslategrey&quot;,&quot;deeppink&quot;,&quot;orange4&quot;,&quot;mediumorchid1&quot;,&quot;lightpink&quot;,&#xA;               &quot;khaki1&quot;,&quot;gray70&quot;,&quot;cyan&quot;,&quot;coral1&quot;)&#xA;&#xA;&#xA;ggplot(d3, aes(x = factor(variable), y = percentage*100, &#xA;          fill = factor(d3$value,levels = c(&quot;3 &quot;,&quot;4&quot;,&quot;5&quot;,&quot;6&quot;,&quot;7&quot;,&quot;8&quot;,&quot;9&quot;,&quot;10&quot;,&quot;11&quot;,&quot;12&quot;,&quot;13&quot;,&quot;31&quot;,&quot;NA&quot;,&quot;0&quot;)))) +&#xA;  scale_fill_manual(values = cbPalette, na.value = &quot;lightsteelblue&quot;) +&#xA;  geom_bar(stat=&quot;identity&quot;, width = 0.7) +&#xA;  labs(x = &quot;Categories&quot;, y = &quot;percent alteration&quot;, fill = &quot;Percentage&quot;) +&#xA;  theme_bw(base_size=15)+&#xA;&#xA;  theme(axis.text.x=element_text(angle = 45, size=25, face=&quot;bold&quot;, hjust = 1), &#xA;        axis.text.y=element_text(angle=0, size=35, face=&quot;bold&quot;, vjust=0.5),&#xA;        plot.title = element_text(size=40, face=&quot;bold&quot;), &#xA;        legend.title=element_blank(), &#xA;        legend.key.size=unit(1, &quot;cm&quot;),      #Sets overall area/size of the legend&#xA;        legend.text=element_text(size=30))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Fq6l2.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Fq6l2.jpg&quot; alt=&quot;barplot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="48" LastEditDate="2018-11-27T10:36:31.583" LastActivityDate="2018-11-27T10:36:31.583" Title="How to plot gene mutation frequencey in terms of percnatge alteration in samples?" Tags="|r|ggplot2|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5524" PostTypeId="2" ParentId="5507" CreationDate="2018-11-27T11:05:18.290" Score="4" Body="&lt;p&gt;As I understand, the software tool &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/30/17/i349/199474&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lambda&lt;/a&gt; is a viable, yet lesser known alternative to BLAST in the context of taxonomic classification of NGS data.&lt;/p&gt;&#xA;" OwnerUserId="3748" LastActivityDate="2018-11-27T11:05:18.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5525" PostTypeId="1" AcceptedAnswerId="5528" CreationDate="2018-11-27T17:55:29.123" Score="4" ViewCount="2308" Body="&lt;p&gt;Reading the &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-0997-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mash: fast genome and metagenome distance estimation&lt;/a&gt; paper, I see the reference to &quot;2d reads&quot; notion under the &lt;a href=&quot;https://nanoporetech.com/products/minion&quot; rel=&quot;nofollow noreferrer&quot;&gt;minION&lt;/a&gt; context (a sequencing technique)&lt;br&gt;&#xA;e.g&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    In both cases Mash was able to&#xA;correctly differentiate these closely related species&#xA;(ANI ≈ 95 %) using 43,806 and 91,379 sequences collected&#xA;from single MinION R7.3 runs of B. anthracis&#xA;Ames and B. cereus ATCC 10987, respectively (combined&#xA;1D and 2D reads) &amp;lt;----------  HERE. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But couldn't find any reference in Wikipedia or online papers to what it is&lt;br&gt;Can someone shed some light?&lt;/p&gt;&#xA;" OwnerUserId="3753" LastActivityDate="2019-05-31T14:21:26.080" Title="What are 2D reads in the Oxford MinIon?" Tags="|phylogenetics|genome-sequencing|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5526" PostTypeId="2" ParentId="5518" CreationDate="2018-11-27T19:08:30.383" Score="1" Body="&lt;p&gt;Given an input file &lt;code&gt;input.vcf&lt;/code&gt;, I would take this approach:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Create two text files: one with the male sample names (&lt;code&gt;males.txt&lt;/code&gt;) and one with the female sample names (&lt;code&gt;females.txt&lt;/code&gt;) (one sample per line)&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Create a BCF (for speed) for all the samples in each list while dropping the &lt;code&gt;INFO&lt;/code&gt; field so that &lt;code&gt;INFO/AC&lt;/code&gt; is recalculated (I'm not sure dropping the &lt;code&gt;INFO&lt;/code&gt; field is strictly necessary). Then only keep sites with a minor allele count greater than zero:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view --samples-file males.txt input.vcf -Ou | bcftools annotate -x INFO  -Ou | bcftools view --min-ac 1:minor -Ob -o male_variants.bcf&#xA;bcftools view --samples-file females.txt input.vcf -Ou | bcftools annotate -x INFO -Ou | bcftools view --min-ac 1:minor -Ob -o female_variants.bcf&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use &lt;code&gt;isec&lt;/code&gt; to create intersects and complements of the two variant files and store them in &lt;code&gt;dir/*&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools isec male_variants.bcf female_variants.bcf -p dir -Oz&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Double-check &lt;code&gt;dir/README.txt&lt;/code&gt;, but the files you seek should be named &lt;code&gt;dir/0000.vcf.gz&lt;/code&gt; and &lt;code&gt;dir/0001.vcf.gz&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Disclaimer:&lt;/strong&gt; This code is untested.&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="492" LastEditDate="2018-11-27T19:14:52.267" LastActivityDate="2018-11-27T19:14:52.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5527" PostTypeId="2" ParentId="4858" CreationDate="2018-11-27T19:28:01.517" Score="2" Body="&lt;p&gt;The data in your matrix appears to be from a VCF file (For reference, see &lt;a href=&quot;https://github.com/samtools/hts-specs/blob/master/VCFv4.3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/samtools/hts-specs/blob/master/VCFv4.3.pdf&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the VCF specification, a heterozygous genotype can be specified as &lt;code&gt;0|1&lt;/code&gt;. The zero and one indicate the allele numbers (0=ref, 1=first alt) that make up the genotype. The &quot;pipe&quot; (&lt;code&gt;|&lt;/code&gt;) is used to indicate that the alleles are in phase with the previous entry in the file. Therefore, if you observe data like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;0|1&quot;    &quot;0|0&quot;&#xA;&quot;0|1&quot;    &quot;1|1&quot;&#xA;&quot;0|1&quot;    &quot;1|1&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then this means that the first sample has the haplotypes [0,0,0] and [1,1,1]; and that the second sample has the haplotypes [0,1,1] and [0,1,1].   &lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2018-11-27T19:28:01.517" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5528" PostTypeId="2" ParentId="5525" CreationDate="2018-11-27T20:38:43.940" Score="5" Body="&lt;p&gt;Early MinION sequencing runs had forward and reverse DNA templates joined together by a hairpin adapter, so that the sequencer would read both strands from the same template. The consensus sequence that was generated by combining the base calls of opposite strands is referred to as a 2D read.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Due to &lt;a href=&quot;https://nanoporetech.com/about-us/news/oxford-nanopore-announces-favourable-outcome-uk-and-german-patent-litigation-against&quot; rel=&quot;nofollow noreferrer&quot;&gt;a dispute with Pacific Biosciences&lt;/a&gt;, and for other reasons related to chemistry and basecalling, the equivalent mode in current sequencing runs is called 1D². There is no physical ligation of forward and reverse templates, but the kit and flow cell chemistry are modified to encourage the reverse strand to follow the forward strand very soon afterwards through the pore, with the joining and consensus generation carried out in software after the sequencing is done. In the output fast5 files, these 1D² reads can also be referred to as 2D.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="3144" LastEditDate="2019-05-31T14:21:26.080" LastActivityDate="2019-05-31T14:21:26.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5529" PostTypeId="1" AcceptedAnswerId="5533" CreationDate="2018-11-28T04:57:39.347" Score="3" ViewCount="763" Body="&lt;p&gt;I am trying to use &lt;code&gt;DESeqDataSetFromTximport&lt;/code&gt; function from &lt;code&gt;DESeq2&lt;/code&gt; package to construct &lt;code&gt;dds&lt;/code&gt; object:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromTximport(txi, sampleTable, ~Group)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And somehow it is giving me the following error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error in &lt;code&gt;rownames&amp;lt;-&lt;/code&gt;(&lt;code&gt;*tmp*&lt;/code&gt;, value = c(&quot;ENSMUSG00000000001&quot;, &quot;ENSMUSG00000000003&quot;,  : &#xA;    invalid rownames length&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I used the same code for other datasets but for the current one it is somehow not working. My &lt;code&gt;sampleTable&lt;/code&gt; looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;       ID      Group&#xA;1    de_1 denervated&#xA;2    de_2 denervated&#xA;3    de_3 denervated&#xA;4    de_4 denervated&#xA;5   den_5       sham&#xA;6    ho_1        IgG&#xA;7    ho_2        IgG&#xA;8    ho_3        IgG&#xA;9    sh_1       sham&#xA;10   sh_2       sham&#xA;11   sh_3       sham&#xA;12   sh_4       sham&#xA;13   sh_5       sham&#xA;14 VLA_10  anti-VLA4&#xA;15 VLA_12  anti-VLA4&#xA;16  VLA_7  anti-VLA4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;txi&lt;/code&gt; was formed in the following way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;k = keys(txdb, keytype = &quot;GENEID&quot;)&#xA;df = AnnotationDbi::select(txdb, keys = k, keytype = &quot;GENEID&quot;, columns = &quot;TXNAME&quot;)&#xA;tx2gene = df[, 2:1]  # tx ID, then gene ID&#xA;txi &amp;lt;- tximport(names, type = &quot;salmon&quot;, tx2gene = tx2gene)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any suggestions would be greatly appreciated&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2019-04-11T09:37:31.453" Title="DESeqDataSetFromTximport invalid rownames length" Tags="|deseq2|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5530" PostTypeId="2" ParentId="5518" CreationDate="2018-11-28T08:14:06.293" Score="1" Body="&lt;p&gt;using vcffilterjdk  &lt;a href=&quot;http://lindenb.github.io/jvarkit/VcfFilterJdk.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://lindenb.github.io/jvarkit/VcfFilterJdk.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;or ask for all individuals named *.M and *.F.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar dist/vcffilterjdk.jar -e 'return variant.getGenotypes().stream().allMatch(G-&amp;gt;(G.getSampleName().endsWith(&quot;M&quot;) &amp;amp;&amp;amp; G.isHet()) || (G.getSampleName().endsWith(&quot;F&quot;) &amp;amp;&amp;amp; G.isHomRef())); ' input.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;variant.getGenotypes().stream()&lt;/code&gt; get a stream of genotypes 'G' for the current variant.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;allMatch&lt;/code&gt; all genotypes must match the statements below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;G.getSampleName().endsWith(&quot;M&quot;) &amp;amp;&amp;amp; G.isHet()&lt;/code&gt; sample 's name ends with 'M' and is heterozygous&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OR&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;G.getSampleName().endsWith(&quot;F&quot;) &amp;amp;&amp;amp; G.isHet()&lt;/code&gt; sample 's name ends with 'F' and is Hom-ref&lt;/p&gt;&#xA;" OwnerUserId="71" LastActivityDate="2018-11-28T08:14:06.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5531" PostTypeId="1" CreationDate="2018-11-28T09:16:52.870" Score="4" ViewCount="263" Body="&lt;p&gt;Let's say I have an RNA-Seq experiment, where I'm interested in the significantly differentiated genes between pre-treatment and post-treatment conditions. &quot;rep&quot; == biological replicate.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sample PreA (3 reps **all** in 2017) vs Sample PostA (3 reps **all** in 2018) &#xA;Sample PreB (3 reps **all** in 2017) vs Sample PostB (3 reps **all** in 2018) &#xA;Sample PreC (3 reps **all** in 2017) vs Sample PostC (3 reps **all** in 2018) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also have a sample:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sample PreD (3 reps all in 2017) and Sample PreD (3 reps again in 2018)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is safe to assume the samples in sequenced in 2017 were done differently to 2018.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q1:&lt;/strong&gt; Is there problem with batch effects in my study? When I do a test, there should't be any way to split variation from batch effects and biological variation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q2:&lt;/strong&gt; If Q1 is correct. How should I reduce batch effects?&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Can I add a batch factor to my model in DESeq2/edgeR?&lt;/li&gt;&#xA;&lt;li&gt;Should I use my &lt;code&gt;PreD&lt;/code&gt; control samples to derive normalisation factor as in RUV? (assume PreD as a negative control)&lt;/li&gt;&#xA;&lt;li&gt;Should I use &lt;code&gt;sva&lt;/code&gt; package?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Devon suggested for adding a batch variable, but how? That's how I'm doing (ignoring batch effects):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromTximport(txi, data, ~Sample)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;data&lt;/code&gt; holds a data frame with a &lt;code&gt;Sample&lt;/code&gt; column. &lt;code&gt;Sample&lt;/code&gt; would be &quot;PreA&quot; and &quot;PostA&quot; in the first test. Very simple.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I couldn't add &lt;code&gt;Date&lt;/code&gt; to it, like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromTximport(txi, data, ~Date + Sample)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;because the code crashed complaining of the 2017/2018 problem discussed above. I'm not sure how I should add &lt;code&gt;PreD&lt;/code&gt; to the design.&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="174" LastEditDate="2018-11-28T09:33:13.237" LastActivityDate="2018-11-28T09:33:13.237" Title="How should I address batch effects in my experiment?" Tags="|rna-seq|bioconductor|batch-effects|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5532" PostTypeId="2" ParentId="5531" CreationDate="2018-11-28T09:19:59.287" Score="4" Body="&lt;ol&gt;&#xA;&lt;li&gt;Yes, though thankfully Your 2018 PreD samples will help you resolve this.&lt;/li&gt;&#xA;&lt;li&gt;Simply add the batch effect to the design (&lt;code&gt;~Batch + Treatment&lt;/code&gt;) and DESeq2 (or edgeR or Limma) will handle this for you.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You do not need SVA or RUV, thankfully, since you quite cleverly sequenced one group in both batches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To clarify, your coldata will be something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Group    Time    Batch&#xA;A        Pre     A&#xA;A        Pre     A&#xA;A        Pre     A&#xA;A        Post    B&#xA;A        Post    B&#xA;A        Post    B&#xA;...&#xA;D        Pre     A&#xA;D        Pre     A&#xA;D        Pre     A&#xA;D        Pre     B&#xA;D        Pre     B&#xA;D        Pre     B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the design is actually &lt;code&gt;~Batch + Group + Time&lt;/code&gt;. You can probably add an interaction term in if you then modify the model matrix manually to ensure it's full rank.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2018-11-28T09:31:30.483" LastActivityDate="2018-11-28T09:31:30.483" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5533" PostTypeId="2" ParentId="5529" CreationDate="2018-11-28T10:10:13.143" Score="3" Body="&lt;p&gt;I got exactly the same problem not long ago (see my &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/5531/how-should-i-address-batch-effects-in-my-experiment&quot;&gt;other&lt;/a&gt; question what I tried to do). We used the same function!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code ran fine on my HPC but the same code/data failed on my laptop. I later found out I had to upgrade my R to the latest version (3.5.1) and upgrade all my bioconductor packages to the latest. My HPC was on a recent bioconductor docker, so there was no problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please make sure everything in your environment latest (R, R packages, Bioconductor).&lt;/p&gt;&#xA;" OwnerUserId="174" LastActivityDate="2018-11-28T10:10:13.143" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5534" PostTypeId="1" AcceptedAnswerId="5542" CreationDate="2018-11-28T10:26:58.357" Score="1" ViewCount="134" Body="&lt;p&gt;Im trying to classify my list into three type: XDR - A strain must be resistant to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MDR + CIPROFLOXACIN/MOXIFLOXACIN/OFLOXACIN/LEVOFLOXACIN + AMYCACIN/CAPREOMYCIN/KANAMYCIN-&amp;gt; means ISONIAZID + RIFAMPICIN + any of these or two or three or all (CIPROFLOXACIN/MOXIFLOXACIN/OFLOXACIN/LEVOFLOXACIN) + any of these or two or three or all (AMYCACIN/CAPREOMYCIN/KANAMYCIN)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;PRE-XDR - A strain must be resistant to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MDR + CIPROFLOXACIN/MOXIFLOXACIN/OFLOXACIN/LEVOFLOXACIN -&amp;gt; means ISONIAZID + RIFAMPICIN + any of these or two or three or all (CIPROFLOXACIN/MOXIFLOXACIN/OFLOXACIN/LEVOFLOXACIN)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;MDR - A strain must be resistant to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ISONIAZID + RIFAMPICIN &#xA;and if it is resistant to any one/two/three of these or all of these in &#xA;addition to ISONIAZID + RIFAMPICIN than again it is MDR:- &#xA;PYRAZINAMIDE, ETHIONAMIDE, STREPTOMYCIN, ETHAMBUTOL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;SUSCEPTIBLE - A strain must be Susceptible to all drugs&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried the code as follows for MDR:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tail -n +2 drug.txt |\&#xA;awk '{a[&lt;span class=&quot;math-container&quot;&gt;$1] = a[$&lt;/span&gt;1]&quot;,&quot;&lt;span class=&quot;math-container&quot;&gt;$2};END{for(i in a)print i&quot;\t&quot;a[i]}'|\&#xA;grep -P &quot;\t,ISONIAZID,RIFAMPICIN$&lt;/span&gt;&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It gave me MDR starins correctly but than i had to first remove all Susceptible strains and applied it to Resistant one only.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when i applied the code for Pre-XDR:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tail -n +2 drug.txt |\&#xA;awk '{a[&lt;span class=&quot;math-container&quot;&gt;$1] = a[$&lt;/span&gt;1]&quot;,&quot;$2};END{for(i in a)print i&quot;\t&quot;a[i]}'|\&#xA;grep -P &#xA;&quot;\t,ISONIAZID,RIFAMPICIN|CIPROFLOXACIN|MOXIFLOXACIN|OFLOXACIN|LEVOFLOXACIN&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is giving me the result as below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SRR671821       ,CAPREOMYCIN,ETHAMBUTOL,ETHIONAMIDE,ISONIAZID,KANAMYCIN,OFLOXACIN,RIFAMPICIN,STREPTOMYCIN&#xA;SRR671863       ,CAPREOMYCIN,ETHAMBUTOL,ISONIAZID,KANAMYCIN,OFLOXACIN,RIFAMPICIN&#xA;SRR671822       ,ETHAMBUTOL,ISONIAZID,OFLOXACIN,RIFAMPICIN,STREPTOMYCIN&#xA;SRR671864       ,CAPREOMYCIN,ETHAMBUTOL,ISONIAZID,KANAMYCIN,OFLOXACIN,RIFAMPICIN&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which is not correct for Pre-XDR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm looking for a the drugs if match than look for resistant and susceptible if these criteria match with any of the mention type (MDR,XDR...) it put those under that in a new file. Can anyone help? &#xA;my file &lt;a href=&quot;https://drive.google.com/open?id=1TOYNzcwzoBdgpUCA3d45mg2VTCRa4zyB&quot; rel=&quot;nofollow noreferrer&quot;&gt;druglist.sxls&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MDR eg from the file attached is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ERR038736   ISONIAZID   RESISTANT&#xA;ERR038736   RIFAMPICIN  RESISTANT&#xA;ERR038737   ISONIAZID   RESISTANT&#xA;ERR038737   RIFAMPICIN  RESISTANT&#xA;SRR1270146  AMIKACIN    SUSCEPTIBLE &#xA;SRR1270146  CAPREOMYCIN SUSCEPTIBLE &#xA;SRR1270146  CYCLOSERINE SUSCEPTIBLE &#xA;SRR1270146  ETHAMBUTOL  RESISTANT   &#xA;SRR1270146  ETHIONAMIDE SUSCEPTIBLE &#xA;SRR1270146  ISONIAZID   RESISTANT   &#xA;SRR1270146  KANAMYCIN   SUSCEPTIBLE &#xA;SRR1270146  PARA-AMINOSALICYLIC ACID    SUSCEPTIBLE &#xA;SRR1270146  PYRAZINAMIDE    RESISTANT   &#xA;SRR1270146  RIFAMPICIN  RESISTANT   &#xA;SRR1270146  STREPTOMYCIN    RESISTANT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3686" LastEditorUserId="3686" LastEditDate="2018-11-30T10:01:48.240" LastActivityDate="2018-11-30T10:27:16.797" Title="How to separate a microbial strain of MTB resistant and susceptible drugwise?" Tags="|python|perl|text-processing|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="5535" PostTypeId="1" CreationDate="2018-11-28T16:43:29.480" Score="1" ViewCount="54" Body="&lt;p&gt;I am studying about gene regularity network(GRN). I'm interested to work on cancer detection to find start reasons of this particular illness. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to find out how the transcription factor(TF) influence genes expression increase or decrease in a biochemical network. cancer that I try to study and analyze is Liver which is popular in recent years in Iran. the main area of research is developing an influence maximization model for finding pathways in biochemical networks. I am seeking to find out the patterns of TF which can act a key role to initiate cancer illness. &#xA;What would happen that a normal gene held in the mutation process?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;my source is a thesis of MA program major in IT-Bioinformatic &lt;/p&gt;&#xA;" OwnerUserId="3719" LastEditorUserId="3719" LastEditDate="2018-11-29T09:35:14.067" LastActivityDate="2018-11-29T09:35:14.067" Title="How transcription factor influence gene expression?" Tags="|gene|cancer|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5536" PostTypeId="1" AcceptedAnswerId="5537" CreationDate="2018-11-29T00:16:14.600" Score="1" ViewCount="1244" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/g2fvb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/g2fvb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I an confused with the term 'outlier' and 'outgroup' in phylogeny. Could someone please explain with the example phylogenetic tree above which is the correct term? Thank you very much !!!&lt;/p&gt;&#xA;" OwnerUserId="3760" LastEditorUserId="3848" LastEditDate="2020-05-06T16:39:09.517" LastActivityDate="2020-05-07T08:21:18.323" Title="What is outliers and how to determine outliers in the following phylogenetic tree?" Tags="|phylogenetics|phylogeny|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5537" PostTypeId="2" ParentId="5536" CreationDate="2018-11-29T00:43:49.130" Score="3" Body="&lt;p&gt;Distance- and ML-based algorithms using reversible models can't find the root of trees. A classical method to root a tree is to use an outgroup (not outlier), which is a species/sequence &lt;em&gt;known&lt;/em&gt; to directly descend from the root of the tree. In your case, it is relatively easy: add a chicken or fish ortholog to your dataset and put the root on the chicken/fish branch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Outgroup works well for single-copy genes used in species tree construction (e.g. ADH). However, it doesn't always work. The culprit is the &quot;known&quot; part. Say a gene has two copies A and B in vertebrates. Copy A was lost in rodent and copy B lost in primate. If you choose a chicken A gene as the outgroup, the correct tree should be ((primary-A,chicken-A),rodent-B). Without knowing the true history, you may forcefully put the root at the chicken-A branch and build a wrong tree ((primate-A,rodent-B),chicken-A).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few other tree rooting methods. An easy approach is to put the root at the longest branch in the tree, assuming the presence of molecular clock. When the species tree is known, you can root a gene tree by minimizing the number gene duplication/loss events in the history. I generally prefer the latter approach when the relevant information is available.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2018-11-29T13:02:07.610" LastActivityDate="2018-11-29T13:02:07.610" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5538" PostTypeId="1" AcceptedAnswerId="5581" CreationDate="2018-11-29T01:01:36.303" Score="2" ViewCount="3690" Body="&lt;p&gt;I tried running &lt;code&gt;conda install -c bioconda bioconductor-deseq2&lt;/code&gt; in a &lt;code&gt;conda&lt;/code&gt; environment, but when I run &lt;code&gt;R&lt;/code&gt; console and try to import &lt;code&gt;DESeq2&lt;/code&gt; I am getting the error message:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error: package or namespace load failed for ‘GenomeInfoDb’ in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]):&#xA;   there is no package called ‘GenomeInfoDbData’&#xA;  Error: package ‘GenomeInfoDb’ could not be loaded&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;When I try installing &lt;code&gt;GenomeInfoDbData&lt;/code&gt; and &lt;code&gt;GenomeInfoDb&lt;/code&gt; using &lt;code&gt;conda&lt;/code&gt;, it says that the packages were already installed. If I run &lt;code&gt;install.packages(GenomeInfoDb)&lt;/code&gt; from &lt;code&gt;R&lt;/code&gt; console, it is telling me that the package is not available for &lt;code&gt;R&lt;/code&gt; version &lt;code&gt;3.5.1&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to import &lt;code&gt;DESeq2&lt;/code&gt; in &lt;code&gt;Rstudio&lt;/code&gt; though and it is giving me a different error when I try importing &lt;code&gt;DESeq2&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error in library(&quot;DESeq2&quot;) : there is no package called ‘DESeq2’&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I do not get why &lt;code&gt;Rstudio&lt;/code&gt; does not see that I alrerady installed &lt;code&gt;DESeq2&lt;/code&gt; with &lt;code&gt;conda&lt;/code&gt;. If I try running &lt;code&gt;biocLite(&quot;DESeq2&quot;)&lt;/code&gt; from &lt;code&gt;Rstudio&lt;/code&gt; I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error in dyn.load(file, DLLpath = DLLpath, ...) : &#xA;    unable to load shared object '/Users/nikitavlasenko/miniconda3/envs/renv/lib/R/library/cluster/libs/cluster.so':&#xA;    dlopen(/Users/nikitavlasenko/miniconda3/envs/renv/lib/R/library/cluster/libs/cluster.so, 6): Symbol not found: _libintl_dgettext&#xA;    Referenced from: /Users/nikitavlasenko/miniconda3/envs/renv/lib/R/library/cluster/libs/cluster.so&#xA;    Expected in: /Users/nikitavlasenko/miniconda3/envs/renv/lib/R/lib/libR.dylib&#xA;   in /Users/nikitavlasenko/miniconda3/envs/renv/lib/R/library/cluster/libs/cluster.so&#xA;  ERROR: lazy loading failed for package ‘Hmisc’&#xA;  * removing ‘/Users/nikitavlasenko/miniconda3/envs/renv/lib/R/library/Hmisc’&#xA;  ERROR: dependency ‘Hmisc’ is not available for package ‘DESeq2’&#xA;  * removing ‘/Users/nikitavlasenko/miniconda3/envs/renv/lib/R/library/DESeq2’&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, supposedly the issue is with &lt;code&gt;Hmisc&lt;/code&gt;. I installed the package successfully with &lt;code&gt;conda&lt;/code&gt;, but &lt;code&gt;Rstudio&lt;/code&gt; is apparently does not know about it. Running &lt;code&gt;install.packages(&quot;Hmisc&quot;)&lt;/code&gt; gives the first part of the error with missing &lt;code&gt;cluster.so&lt;/code&gt; library.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastActivityDate="2020-04-02T12:17:51.520" Title="Install DESeq2 through anaconda" Tags="|deseq2|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5539" PostTypeId="1" AcceptedAnswerId="5543" CreationDate="2018-11-29T01:31:41.530" Score="6" ViewCount="348" Body="&lt;p&gt;Can anyone help me explain what allelic imbalance is, hopefully shortly? &#xA;Surprisingly, we cannot find any introduction online. &lt;/p&gt;&#xA;" OwnerUserId="1082" LastActivityDate="2018-12-03T16:50:20.507" Title="What is Allelic Imbalance" Tags="|phylogenetics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5540" PostTypeId="1" AcceptedAnswerId="5561" CreationDate="2018-11-29T04:04:51.283" Score="2" ViewCount="577" Body="&lt;p&gt;I am using this script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import csv&#xA;import time&#xA;import sys&#xA;from ete3 import NCBITaxa&#xA;&#xA;ncbi = NCBITaxa()&#xA;&#xA;def get_desired_ranks(taxid, desired_ranks):&#xA;    lineage = ncbi.get_lineage(taxid)   &#xA;    names = ncbi.get_taxid_translator(lineage)&#xA;    lineage2ranks = ncbi.get_rank(names)&#xA;    ranks2lineage = dict((rank,taxid) for (taxid, rank) in lineage2ranks.items())&#xA;    return{'{}_id'.format(rank): ranks2lineage.get(rank, '&amp;lt;not present&amp;gt;') for rank in desired_ranks}&#xA;&#xA;if __name__ == '__main__':&#xA;    file = open(sys.argv[1], &quot;r&quot;)    &#xA;    taxids = []&#xA;    contigs = []&#xA;    for line in file:&#xA;        line = line.split(&quot;\n&quot;)[0]&#xA;        taxids.append(line.split(&quot;,&quot;)[0])&#xA;        contigs.append(line.split(&quot;,&quot;)[1])&#xA;&#xA;    desired_ranks = ['superkingdom', 'phylum']&#xA;    results = list()&#xA;    for taxid in taxids:&#xA;        results.append(list())&#xA;        results[-1].append(str(taxid))&#xA;        ranks = get_desired_ranks(taxid, desired_ranks)&#xA;        for key, rank in ranks.items():&#xA;            if rank != '&amp;lt;not present&amp;gt;':&#xA;                results[-1].append(list(ncbi.get_taxid_translator([rank]).values())[0])&#xA;            else:&#xA;                results[-1].append(rank)&#xA;&#xA;    i = 0&#xA;    for result in results:&#xA;        print(contigs[i] + ','),&#xA;        print(','.join(result))&#xA;        i += 1&#xA;&#xA;    file.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The script takes taxids from a file and fetches their respective lineages from a local copy of NCBI's Taxonomy database.  Strangely, this script works fine when I run it on small sets of taxids (~70, ~100), but most of my datasets are upwards of 280k taxids and these break the script.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get this complete error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &quot;/data1/lstout/blast/scripts/getLineageByETE3.py&quot;, line 31, in &amp;lt;module&amp;gt;&#xA;    ranks = get_desired_ranks(taxid, desired_ranks)&#xA;  File &quot;/data1/lstout/blast/scripts/getLineageByETE3.py&quot;, line 11, in get_desired_ranks&#xA;    lineage = ncbi.get_lineage(taxid)   &#xA;  File &quot;/data1/lstout/.local/lib/python2.7/site-packages/ete3/ncbi_taxonomy/ncbiquery.py&quot;, line 227, in get_lineage&#xA;    result = self.db.execute('SELECT track FROM species WHERE taxid=%s' %taxid)&#xA;sqlite3.Warning: You can only execute one statement at a time.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first two files from the traceback are simply the script I referenced above, the third file is one of ete3's.  And as I stated, the script works fine with small datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I have tried:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Importing the time module and sleeping for a few milliseconds/hundredths of a second before/after my offending lines of code on lines 11 and 31.  No effect.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Went to line 227 in ete3's code...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;result = self.db.execute('SELECT track FROM species WHERE taxid=%s' %merged_conversion[taxid])&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and changed the &quot;execute&quot; function to &quot;executescript&quot; in order to be able to handle multiple queries at once (as that seems to be the problem).  This produced a new error and led to a rabbit hole of me changing minor things in their script trying to fudge this to work.  No result.  This is the complete offending function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    def get_lineage(self, taxid):&#xA;    &quot;&quot;&quot;Given a valid taxid number, return its corresponding lineage track as a&#xA;    hierarchically sorted list of parent taxids.&#xA;    &quot;&quot;&quot;&#xA;    if not taxid:&#xA;        return None&#xA;    result = self.db.execute('SELECT track FROM species WHERE taxid=%s' %taxid)&#xA;    raw_track = result.fetchone()&#xA;    if not raw_track:&#xA;        #perhaps is an obsolete taxid&#xA;        _, merged_conversion = self._translate_merged([taxid])&#xA;        if taxid in merged_conversion:&#xA;            result = self.db.execute('SELECT track FROM species WHERE taxid=%s' %merged_conversion[taxid])&#xA;            raw_track = result.fetchone()&#xA;        # if not raise error&#xA;        if not raw_track:&#xA;            #raw_track = [&quot;1&quot;]&#xA;            raise ValueError(&quot;%s taxid not found&quot; %taxid)&#xA;        else:&#xA;            warnings.warn(&quot;taxid %s was translated into %s&quot; %(taxid, merged_conversion[taxid]))&#xA;&#xA;    track = list(map(int, raw_track[0].split(&quot;,&quot;)))&#xA;    return list(reversed(track))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What bothers me so much is that this works on small amounts of data!  I'm running these scripts from my school's high performance computer and have tried running on their head node and in an interactive moab scheduler.  Nothing has helped.&lt;/p&gt;&#xA;" OwnerUserId="2446" LastEditorUserId="17320" LastEditDate="2023-10-05T20:47:43.317" LastActivityDate="2023-10-05T20:47:43.317" Title="Python scripting with ete3 to query NCBI's Taxonomy: &quot;sqlite3 Warning (can only execute one statement at a time)&quot;" Tags="|python|phylogenetics|database|taxonomy|ete3|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5541" PostTypeId="2" ParentId="5509" CreationDate="2018-11-29T04:32:19.963" Score="3" Body="&lt;p&gt;Here's another option: &lt;a href=&quot;https://github.com/noporpoise/seq-align&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/noporpoise/seq-align&lt;/a&gt;; I've used it in the past as it is small and has no dependencies (just &lt;code&gt;make&lt;/code&gt; and you're set).&lt;/p&gt;&#xA;" OwnerUserId="132" LastActivityDate="2018-11-29T04:32:19.963" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5542" PostTypeId="2" ParentId="5534" CreationDate="2018-11-29T10:37:17.130" Score="2" Body="&lt;p&gt;Since this involves combining multiple conditions, it is easier (and certainly more readable if you ever need to come back to it later) to write a script to do this. For example, in perl:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/perl&#xA;&#xA;use strict;&#xA;use warnings;&#xA;&#xA;## This will hold the information for each strain&#xA;my %strains;&#xA;&#xA;## These are the functions that will test each strain&#xA;sub isMDR{&#xA;  my &lt;span class=&quot;math-container&quot;&gt;$_strain = shift;&#xA;  my %thisStrain = %{$strains{$_strain}};&#xA;  if ($&lt;/span&gt;thisStrain{&quot;ISONIAZID&quot;} &amp;amp;&amp;amp; &lt;span class=&quot;math-container&quot;&gt;$thisStrain{&quot;RIFAMPICIN&quot;}){&#xA;    foreach my $drug (keys(%thisStrain)) {&#xA;      next if $drug eq &quot;ISONIAZID&quot;;&#xA;      next if $drug eq &quot;RIFAMPICIN&quot;;&#xA;      next if $drug eq &quot;PYRAZINAMID&quot;;&#xA;      next if $drug eq &quot;ETHAMBUTOL&quot;;&#xA;      next if $drug eq &quot;STREPTOMYCIN&quot;;&#xA;      ## discard this strain if it is resistant to any other drug&#xA;      if ($thisStrain{$&lt;/span&gt;drug}) {&#xA;        return 0;&#xA;      }&#xA;    }&#xA;&#xA;    return 1;&#xA;  }&#xA;  return 0;&#xA;}&#xA;&#xA;sub isPreXDR{&#xA;  my &lt;span class=&quot;math-container&quot;&gt;$_strain = shift;&#xA;  my %thisStrain = %{$strains{$_strain}};&#xA;  if (&#xA;  ($&lt;/span&gt;thisStrain{&quot;ISONIAZID&quot;} &amp;amp;&amp;amp; &lt;span class=&quot;math-container&quot;&gt;$thisStrain{&quot;RIFAMPICIN&quot;}) &amp;amp;&amp;amp;&#xA;  (&#xA;   $&lt;/span&gt;thisStrain{&quot;CIPROFLOXACIN&quot;}  ||&#xA;       &lt;span class=&quot;math-container&quot;&gt;$thisStrain{&quot;MOXIFLOXACIN&quot;} ||&#xA;   $&lt;/span&gt;thisStrain{&quot;OFLOXACIN&quot;}  ||&#xA;       $thisStrain{&quot;LEVOFLOXACIN&quot;}&#xA;      )&#xA;     ) {&#xA;    return 1;&#xA;  }&#xA;  return 0;&#xA;}&#xA;&#xA;sub isXDR{&#xA;  my &lt;span class=&quot;math-container&quot;&gt;$_strain = shift;&#xA;  my %thisStrain = %{$strains{$_strain}};&#xA;  if (&#xA;  isPreXDR($&lt;/span&gt;_strain) &amp;amp;&amp;amp;&#xA;      (&#xA;       &lt;span class=&quot;math-container&quot;&gt;$thisStrain{&quot;AMYCACIN&quot;} ||&#xA;   $&lt;/span&gt;thisStrain{&quot;CAPREOMYCIN&quot;} ||&#xA;       $thisStrain{&quot;KANAMYCIN&quot;}&#xA;      )&#xA;     ){&#xA;    return 1;&#xA;  }&#xA;  return 0;&#xA;}&#xA;&#xA;sub isSusceptible{&#xA;  my &lt;span class=&quot;math-container&quot;&gt;$_strain = shift;&#xA;  foreach my $&lt;/span&gt;drug (keys(%{&lt;span class=&quot;math-container&quot;&gt;$strains{$_strain}})) {&#xA;if ($strains{$_strain}{$&lt;/span&gt;drug}) {&#xA;      return 0&#xA;    }&#xA;  }&#xA;  return 1&#xA;}&#xA;&#xA;## Iterate over each line in the input file.&#xA;## For each iteration, the special variable &lt;span class=&quot;math-container&quot;&gt;$_&#xA;# will hold the contents of the line.&#xA;while (&amp;lt;&amp;gt;) {&#xA;  ## Skip first line&#xA;  next if $. == 1;&#xA;  ## remove Windows-style line endings&#xA;  s/\r//;&#xA;  ## remove trailing newline&#xA;  chomp;&#xA;  ## split the line on commas&#xA;  my ($strain, $drug, $status) = split(/,/);&#xA;  if ($status eq &quot;SUSCEPTIBLE&quot;){&#xA;$status = 0&#xA;  }&#xA;  else {&#xA;$status = 1;&#xA;  } &#xA;  $strains{$strain}{$drug}=$&lt;/span&gt;status;&#xA;}&#xA;&#xA;foreach my &lt;span class=&quot;math-container&quot;&gt;$strain (keys(%strains)) {&#xA;  print &quot;$strain\t&quot;;&#xA;  if (isMDR($strain)) {&#xA;print &quot;MDR\n&quot;;&#xA;  }&#xA;  elsif (isXDR($strain)) {&#xA;print &quot;XDR\n&quot;&#xA;  }&#xA;  elsif (isPreXDR($strain)) {&#xA;print &quot;PRE-XDR\n&quot;&#xA;  }&#xA;  elsif (isSusceptible($&lt;/span&gt;strain)) {&#xA;    print &quot;SUSCEPTIBLE\n&quot;;&#xA;  }&#xA;  else {&#xA;    print &quot;OTHER\n&quot;;&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you save that script as &lt;code&gt;foo.pl&lt;/code&gt;, you can run it on your file with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl foo.pl drug.csv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Selected output from running this on the &lt;code&gt;drug.csv&lt;/code&gt; file you posted is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ foo.pl drug.csv | grep -E 'ERR038288|ERR038736|ERR038751|SRS553990|ERR133879'&#xA;ERR038751   OTHER&#xA;ERR038288   SUSCEPTIBLE&#xA;SRS553990   XDR&#xA;ERR038736   MDR&#xA;ERR133879   PRE-XDR&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Various details about this question were added in chat: &lt;a href=&quot;https://chat.stackexchange.com/transcript/message/47821646#47821646&quot;&gt;https://chat.stackexchange.com/transcript/message/47821646#47821646&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-11-30T10:27:16.797" LastActivityDate="2018-11-30T10:27:16.797" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="5543" PostTypeId="2" ParentId="5539" CreationDate="2018-11-29T13:13:59.283" Score="5" Body="&lt;p&gt;It might be used differently in different contexts, but generally speaking, in my world - allelic imbalance is when there's a difference in the level of gene expression from different alleles, usually through genetic (e.g. a variant in a promoter) or epigenetic mechanisms (e.g. one copy silenced, as in imprinted regions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a &lt;a href=&quot;http://nathansheffield.com/wordpress/what-is-allelic-imbalance/&quot; rel=&quot;nofollow noreferrer&quot;&gt;few&lt;/a&gt; &lt;a href=&quot;https://www.rna-seqblog.com/direct-testing-for-allele-specific-expression-differences-between-conditions/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog&lt;/a&gt; posts here with some more information, and plenty of papers (google &quot;Allelic imbalance&quot;) if you want to dig deeper.&lt;/p&gt;&#xA;" OwnerUserId="3624" LastEditorUserId="48" LastEditDate="2018-12-03T16:50:20.507" LastActivityDate="2018-12-03T16:50:20.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5544" PostTypeId="1" CreationDate="2018-11-29T14:18:14.337" Score="1" ViewCount="83" Body="&lt;p&gt;I am looking to create a table (on RStudio) that will show the overlap between 8 different gene lists wherever possible. I have saved the 8 lists as .txt files all in one folder and set that particular folder as my working directory. Any idea of code that will work for this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Pn9A1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Pn9A1.png&quot; alt=&quot;Output table with NA values&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anyone knows what I can alter to make the current code work or suggest an alternative way I would be very happy to hear any suggestions!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data.file1 &amp;lt;- &quot;GSE108363_BCGdown_V.txt&quot;&#xA;data.file2 &amp;lt;- &quot;GSE108363_BCGdown_D.txt&quot;&#xA;data.file3 &amp;lt;- &quot;GSE108363_BCGup_V.txt&quot;&#xA;data.file4 &amp;lt;- &quot;GSE108363_BCGup_D.txt&quot;&#xA;data.file5 &amp;lt;- &quot;GSE108363_MTBdown_V.txt&quot;&#xA;data.file6 &amp;lt;- &quot;GSE108363_MTBdown_D.txt&quot;&#xA;data.file7 &amp;lt;- &quot;GSE108363_MTBup_V.txt&quot;&#xA;data.file8 &amp;lt;- &quot;GSE108363_MTBup_D.txt&quot;&#xA;&#xA;genevect1 &amp;lt;- scan(data.file1, what=character(), sep=&quot;\n&quot;)&#xA;genevect2 &amp;lt;- scan(data.file2, what=character(), sep=&quot;\n&quot;)&#xA;genevect3 &amp;lt;- scan(data.file3, what=character(), sep=&quot;\n&quot;)&#xA;genevect4 &amp;lt;- scan(data.file4, what=character(), sep=&quot;\n&quot;)&#xA;genevect5 &amp;lt;- scan(data.file5, what=character(), sep=&quot;\n&quot;)&#xA;genevect6 &amp;lt;- scan(data.file6, what=character(), sep=&quot;\n&quot;)&#xA;genevect7 &amp;lt;- scan(data.file7, what=character(), sep=&quot;\n&quot;)&#xA;genevect8 &amp;lt;- scan(data.file8, what=character(), sep=&quot;\n&quot;)&#xA;&#xA;&#xA;filelist &amp;lt;- list(data.file1, data.file2, data.file3, data.file4, data.file5, data.file6, data.file7, data.file8)&#xA;#-------------------------------------------------------------------------------------------------------&#xA;# &quot;Lapply&quot; is basically making a &quot;for loop&quot;.&#xA;#-------------------------------------------------------------------------------------------------------&#xA;all_gene_vectors &amp;lt;- lapply(filelist, scan, what=character(), sep=&quot;\n&quot;)&#xA;&#xA;#-------------------------------------------------------------------------------------------------------&#xA;# Then make the intersection of *all* genesets&#xA;#-------------------------------------------------------------------------------------------------------&#xA;final_inter &amp;lt;- all_gene_vectors[[1]]&#xA;for (next_genevect in all_gene_vectors[2:length(all_gene_vectors)]) {&#xA;  final_inter &amp;lt;- intersect(final_inter, next_genevect)&#xA;}&#xA;show(final_inter)&#xA;&#xA;#-------------------------------------------------------------------------------------------------------&#xA;# Or compute all pairwise intersections (untested)&#xA;#-------------------------------------------------------------------------------------------------------&#xA;pairwise_inters &amp;lt;- list()&#xA;#-------------------------------------------------------------------------------------------------------&#xA;# Iterate over all possible pairs i,j (i ≠ j)&#xA;#-------------------------------------------------------------------------------------------------------&#xA;N &amp;lt;- length(all_gene_vectors)&#xA;for (i in 1:(N-1)) {&#xA;  ith_inters &amp;lt;- list()&#xA;  for (j in (i+1):N) {&#xA;    ith_inters[[ names(all_gene_vectors)[j ]]] &amp;lt;- intersect(all_gene_vectors[[i]], all_gene_vectors[[j]])&#xA;  }&#xA;  pairwise_inters[[ names(all_gene_vectors)[i] ]] &amp;lt;- ith_inters&#xA;}&#xA;&#xA;#-------------------------------------------------------------------------------------------------------&#xA;#Generate table in wide format:&#xA;#-------------------------------------------------------------------------------------------------------  &#xA;#Concatenate gene names of all tables to a separate file&#xA;#Extract unique values in this file&#xA;#Perform outer join between the separate file and each table&#xA;&#xA;all_names &amp;lt;- c(data.list)&#xA;for (tp in timepoints){&#xA;  all_names &amp;lt;- append(all_names, tables[[tp]]$genes)&#xA;}&#xA;all_names &amp;lt;- unique(all_names)&#xA;&#xA;wide_table &amp;lt;- matrix(nrow=length(all_names), ncol=(1+length(timepoints)))&#xA;wide_table &amp;lt;- as.data.frame(wide_table)&#xA;colnames(wide_table) &amp;lt;- append(&quot;genes&quot;, timepoints)&#xA;&#xA;for (tp in timepoints){&#xA;  tmp &amp;lt;- tables[[tp]]&#xA;  tmp&lt;span class=&quot;math-container&quot;&gt;$tp &amp;lt;- 1&#xA;  tmp &amp;lt;- merge(wide_table, tmp, by=&quot;genes&quot;, all=T)&#xA;  wide_table$&lt;/span&gt;tp &amp;lt;- tmp$tp&#xA;}&#xA;&#xA;&#xA;set.seed(11)&#xA;BCG_validation_Up &amp;lt;- sample(letters[1:429], 20)&#xA;BCG_discovery_Up &amp;lt;- sample(letters[1:250], 20)&#xA;MTB_validation_Up &amp;lt;- sample(letters[1:286], 20)&#xA;MTB_discovery_Up &amp;lt;- sample(letters[1:128], 20)&#xA;BCG_validation_Down &amp;lt;- sample(letters[1:267], 20)&#xA;BCG_discovery_Down  &amp;lt;- sample(letters[1:350], 20)&#xA;MTB_validation_Down &amp;lt;- sample(letters[1:244], 20)&#xA;MTB_discovery_Down &amp;lt;- sample(letters[1:86], 20)&#xA;&#xA;cross_table &amp;lt;- matrix(, nrow = 4, ncol = 4)&#xA;rownames(cross_table) &amp;lt;- c(&quot;BCG_validation_Up&quot;, &quot;BCG_discovery_Up&quot;, &quot;MTB_validation_Up&quot;, &quot;MTB_discovery_Up&quot;)&#xA;colnames(cross_table) &amp;lt;- c(&quot;BCG_validation_Down&quot;, &quot;BCG_discovery_Down&quot;, &quot;MTB_validation_Up&quot;, &quot;MTB_discovery_Up&quot;)&#xA;&#xA;for (i in 1:4){&#xA;  for(j in 1:4){&#xA;    cross_table[i,j] &amp;lt;- length(intersect(get(paste0(&quot;&quot;,i,&quot;_Up&quot;)),(get(paste0(&quot;&quot;,j,&quot;_Down&quot;)))))&#xA;  }&#xA;} &#xA;&#xA;cross_table&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3762" LastEditorUserId="3762" LastEditDate="2018-11-30T00:27:47.793" LastActivityDate="2018-11-30T00:27:47.793" Title="Creating a computational table to show the number of available genes between all possible pairs of combinations in eight gene lists" Tags="|r|rstudio|" AnswerCount="1" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="5545" PostTypeId="1" AcceptedAnswerId="5551" CreationDate="2018-11-29T15:42:32.347" Score="1" ViewCount="802" Body="&lt;p&gt;I have RNA-Seq data for LUNG cancer. 370 tumor and 50 Normal. For differential analysis initially I did some filtering and kept approx. 19k genes for further analysis. I used edgeR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a FC &gt; or &amp;lt; 2 and FDR &amp;lt; 0.05&lt;br&gt;&#xA;&lt;code&gt;tr &amp;lt;- glmTreat(fit, contrast=contrast.matrix, lfc = log2(2))&lt;/code&gt;&lt;br&gt;&#xA;I got approx 3000 differentially expressed genes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Among the 3000 differentially expressed genes I'm interested in a gene &lt;code&gt;OIT3&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GeneSymbol  logFC    unshrunk.logFC   logCPM          PValue      FDR&#xA;  OIT3  2.156487448  2.171599487    -0.437234244    0.000701037 0.005304312&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I see the above gene &lt;code&gt;OIT3&lt;/code&gt; is Upregulated. Why I see logCPM with negative value?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I plotted the logCPM expression of the above gene and did a t-test which gave pvalue 0.1 which means not significant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RKxhR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RKxhR.png&quot; alt=&quot;Boxplot showing OIT3 logCPM expression&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) I see the gene is differentially expressed with Differential analysis using edgeR. Why I see logCPM with negative value?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) With t-test why it is not significant?&lt;/p&gt;&#xA;" OwnerUserId="1369" LastEditorUserId="48" LastEditDate="2018-11-29T16:28:31.630" LastActivityDate="2018-11-30T03:42:18.480" Title="Why the t-test for a specific gene shows different value compared to differential analysis?" Tags="|r|rna-seq|rna|cancer|gene-expression|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5546" PostTypeId="2" ParentId="5544" CreationDate="2018-11-29T16:10:25.900" Score="1" Body="&lt;p&gt;To make a confusion matrix of 4 lists against 4 other lists, you need to get the length of all intersecting genes for each pair. Here an example with letters instead of gene names.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;set.seed(11)&#xA;list1_up &amp;lt;- sample(letters[1:26], 20)&#xA;list1_down &amp;lt;- sample(letters[1:26], 20)&#xA;list2_up &amp;lt;- sample(letters[1:26], 20)&#xA;list2_down &amp;lt;- sample(letters[1:26], 20)&#xA;list3_up &amp;lt;- sample(letters[1:26], 20)&#xA;list3_down &amp;lt;- sample(letters[1:26], 20)&#xA;list4_up &amp;lt;- sample(letters[1:26], 20)&#xA;list4_down &amp;lt;- sample(letters[1:26], 20)&#xA;&#xA;cross_table &amp;lt;- matrix(, nrow = 4, ncol = 4)&#xA;rownames(cross_table) &amp;lt;- c(&quot;list1_up&quot;, &quot;list2_up&quot;, &quot;list3_up&quot;, &quot;list4_up&quot;)&#xA;colnames(cross_table) &amp;lt;- c(&quot;list1_down&quot;, &quot;list2_down&quot;, &quot;list3_down&quot;, &quot;list4_down&quot;)&#xA;&#xA;for (i in 1:4){&#xA;  for(j in 1:4){&#xA;    cross_table[i,j] &amp;lt;- length(intersect(get(paste0(&quot;list&quot;,i,&quot;_up&quot;)),(get(paste0(&quot;list&quot;,j,&quot;_down&quot;)))))&#xA;    }&#xA;} &#xA;&#xA;cross_table&#xA;         list1_down list2_down list3_down list4_down&#xA;list1_up         15         14         15         15&#xA;list2_up         18         15         15         15&#xA;list3_up         17         15         14         16&#xA;list4_up         17         15         15         16&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2018-11-29T20:12:03.087" LastActivityDate="2018-11-29T20:12:03.087" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5547" PostTypeId="1" AcceptedAnswerId="5550" CreationDate="2018-11-29T20:23:21.087" Score="6" ViewCount="3823" Body="&lt;p&gt;I have run into an odd situation, where &lt;a href=&quot;https://anaconda.org/bioconda/blast&quot; rel=&quot;noreferrer&quot;&gt;Anaconda repo page&lt;/a&gt; clearly states that the newest version of Blast in Bioconda channel is 2.7.1, however the version Conda wants to install (command &lt;code&gt;conda install -c bioconda blast&lt;/code&gt;) is 2.6.0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even specifying the version with command &lt;code&gt;conda install -c bioconda blast=2.7.1&lt;/code&gt; does not work and Conda returns a &lt;code&gt;PackagesNotFoundError&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By observing the webpage mentioned above, one can see that there is a &quot;v&quot; before the version number, however adding this letter to the command with specified version did not help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make things even more odd, &lt;code&gt;conda search blast&lt;/code&gt; returns a list of Blast versions available and there is a 2.7.1 version available in Bioconda channel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have added the Bioconda channel into my Conda.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: How can I install the 2.7.1 (by the time of writing this question the newest version available in Anaconda repo) through Conda?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I have Ubuntu Server 18.04 LTS and Conda 4.5.11.&lt;/p&gt;&#xA;" OwnerUserId="1199" LastEditorUserId="1199" LastEditDate="2018-11-29T20:41:53.467" LastActivityDate="2021-03-25T15:01:17.740" Title="Can't install newest Blast from Conda" Tags="|blast|bioconda|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5548" PostTypeId="2" ParentId="5538" CreationDate="2018-11-29T20:48:47.247" Score="2" Body="&lt;p&gt;Sounds like you might have an issue with which &lt;code&gt;R&lt;/code&gt; Rstudio is running.&#xA;If you didn't install Rstudio through &lt;code&gt;conda&lt;/code&gt;, Rstudio may be using its own version of &lt;code&gt;R&lt;/code&gt; that is bundled with Rstudio, instead of the one you've installed via &lt;code&gt;conda&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that your &lt;code&gt;conda&lt;/code&gt; environment name is &lt;code&gt;renv&lt;/code&gt;, try running this in the terminal:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ conda activate renv&#xA;$&lt;/span&gt; conda install r::rstudio -y&#xA;&#xA;$ which rstudio&#xA;/Users/nikitavlasenko/miniconda3/envs/renv/bin/rstudio&#xA;# or something like this ^&#xA;&#xA;# run Rstudio as a background shell process&#xA;rstudio &amp;amp;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This should open up the Rstudio interface, like normal, but using everything defined in your &lt;code&gt;renv&lt;/code&gt; environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you try loading the &lt;code&gt;DEseq2&lt;/code&gt; library now, that might work.&lt;/p&gt;&#xA;" OwnerUserId="240" LastEditorUserId="240" LastEditDate="2018-11-29T21:43:04.903" LastActivityDate="2018-11-29T21:43:04.903" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5549" PostTypeId="2" ParentId="5505" CreationDate="2018-11-29T20:59:33.423" Score="2" Body="&lt;p&gt;You could try setting a high p-value threshold when you're calling peaks to retain the &quot;non-significant&quot; values. Something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;macs2 callpeak -t &amp;lt;ChIP&amp;gt;.bam -c &amp;lt;Control&amp;gt;.bam -f BED -g hs -n test -B -p 0.7&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You probably don't want to set the threshold to 1, since you'll likely want to avoid some noise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But you can then use these &quot;non-significant&quot; values in combination with a &lt;code&gt;bedtools map&lt;/code&gt; command, to map them to your predefined union of &quot;actual&quot; peaks.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2018-11-29T20:59:33.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5550" PostTypeId="2" ParentId="5547" CreationDate="2018-11-29T21:22:40.310" Score="9" Body="&lt;p&gt;I was able to reproduce your problem, even in a blank environment.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ conda create -n test&#xA;$&lt;/span&gt; conda activate test&#xA;$ conda install bioconda::blast=2.7.1&#xA;Solving environment: failed&#xA;&#xA;PackagesNotFoundError: The following packages are not available from current channels:&#xA;&#xA;  - bioconda::blast=2.7.1&#xA;  - boost=1.64&#xA;  - bioconda::blast=2.7.1&#xA;  - gnutls&#xA;  - bioconda::blast=2.7.1&#xA;  - nettle[version='3.3|3.3.*']&#xA;  - bioconda::blast=2.7.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;code&gt;boost&lt;/code&gt;, &lt;code&gt;gnutls&lt;/code&gt;, and &lt;code&gt;nettle&lt;/code&gt; are listed as requirements here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do a search for those packages, you'll see that &lt;code&gt;boost&lt;/code&gt; is available in a default channel, whereas &lt;code&gt;gnutls&lt;/code&gt; and &lt;code&gt;nettle&lt;/code&gt; are only in the &lt;code&gt;conda-forge&lt;/code&gt; channel, which is not a default.&#xA;Adding those packages to the install command resolved this issue, for me.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ conda install bioconda::blast=2.7.1 conda-forge::gnutls conda-forge::nettle --dry-run&#xA;Solving environment: done&#xA;&#xA;## Package Plan ##&#xA;&#xA;environment location: /Users/hawleyj/anaconda3/envs/test&#xA;&#xA;added / updated specs: &#xA;    - bioconda::blast=2.7.1&#xA;    - conda-forge::gnutls&#xA;    - conda-forge::nettle&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As mentioned in the comment below, to avoid this issue entirely, before using Bioconda packages make sure you run the proper channel configuration first.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conda config --add channels defaults&#xA;conda config --add channels bioconda&#xA;conda config --add channels conda-forge&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="240" LastEditorUserId="240" LastEditDate="2018-12-18T20:00:33.633" LastActivityDate="2018-12-18T20:00:33.633" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5551" PostTypeId="2" ParentId="5545" CreationDate="2018-11-29T21:48:11.130" Score="3" Body="&lt;ol&gt;&#xA;&lt;li&gt;The &lt;span class=&quot;math-container&quot;&gt;$log(CPM)$&lt;/span&gt; of any low-moderately expressed gene will be negative. There is nothing unexpected there.&lt;/li&gt;&#xA;&lt;li&gt;Your statistics are inappropriate for a variety of reasons. Firstly, a CPM is not a robust value that's comparable between samples (this is why CPMs aren't used for statistics). &lt;code&gt;edgeR&lt;/code&gt; performs more appropriate normalization and incorporates that into its model. Secondly, though you have a fair number of replicates, edgeR is pooling information from all of the genes to better estimate the variance.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I encourage you to &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/17881408&quot; rel=&quot;nofollow noreferrer&quot;&gt;read the edgeR paper&lt;/a&gt; and likely that of Limma too.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-11-29T21:48:11.130" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5552" PostTypeId="2" ParentId="5545" CreationDate="2018-11-30T03:42:18.480" Score="4" Body="&lt;p&gt;There is no reason your t-test should reproduce edgeR. In fact, edgeR exists because t-test is inappropriate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edgeR does the tests by pooling information from all genes, because with the low number of replicates your t-test doesn't give sufficient statistical power.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you need to do is: &lt;em&gt;check visually your gene and make a decision yourself based on graphical results and edgeR results if your gene shows biologically significant&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="174" LastActivityDate="2018-11-30T03:42:18.480" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5553" PostTypeId="2" ParentId="802" CreationDate="2018-11-30T08:59:00.007" Score="0" Body="&lt;p&gt;I am not sure whether you are looking for how to produce taxonomy file or rep_set file (qiime reference file types). But you can produce fasta file (rep_Set) like that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$blastdbcmd -db 16SMicrobial/16SMicrobial -outfmt '%f' -entry 'all' &amp;gt; 16SMicrobial.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you can create QIIME-compatible id_to_tax maps with this code:&#xA;&lt;a href=&quot;https://github.com/mtruglio/QIIME_utilities/blob/master/id_to_tax_mapmaker.sh&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mtruglio/QIIME_utilities/blob/master/id_to_tax_mapmaker.sh&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="3767" LastActivityDate="2018-11-30T08:59:00.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5554" PostTypeId="1" CreationDate="2018-11-30T10:00:27.883" Score="0" ViewCount="68" Body="&lt;p&gt;well, I'm new to Bio. what I'm going to do is create a network/graph of a dataset related to the genes I think. here's the sample of the nodes dataset with CSV format named lung-nodes : &#xA;&lt;a href=&quot;https://i.stack.imgur.com/Rq8HJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Rq8HJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and this is a sample of edges I think with trrust name :&#xA;&lt;a href=&quot;https://i.stack.imgur.com/MI1N9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MI1N9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;these data are stored in two separate file&lt;/p&gt;&#xA;&#xA;&lt;p&gt;what I must do is create a graph(network) of these nodes and edges by Python and a visualization library like networkX, but I do not how. &#xA;I must make a relation between these to the dataset by creating a graph of them&#xA;can you help me out? &lt;/p&gt;&#xA;" OwnerUserId="3719" LastActivityDate="2018-11-30T13:14:34.560" Title="How to create a graph of multi column gene dataset?" Tags="|gene|networks|visualization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5555" PostTypeId="1" AcceptedAnswerId="5557" CreationDate="2018-11-30T11:25:26.407" Score="3" ViewCount="353" Body="&lt;p&gt;I tried to subset the dataframe in R since there are empty space after each line in the sequence which shows 'NA' when i read the files into R ,as a result of which when i subset it just takes the sequence Protein_IDS and sequence line by line not as whole &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Im looking for this Protein_ID in one column and respective sequence in other &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Proteins_IDs                Sequence                                                    &#xA;  &amp;lt;chr&amp;gt;                       &amp;lt;chr&amp;gt;                                                       &#xA;1 fig|6666666.151743.peg.5284 MERQAVAVVFHDWRRAIQQRNKTAAADRGVDPVGVGHNSKIESLQPRSSVCRASLIQVPG&#xA;2 NA                          RLLNGEIFYSLAEAKVLIEAWRRHYNTVRPHSSLGYRPPAPETATPPYPASGSASLHLRP&#xA;3 NA                          DMAAMGLIH                                                   &#xA;4 NA                          NA                                                          &#xA;5 fig|6666666.151743.peg.5285 MIDRQTAQVRRWSFVLGSSIAAMAAASATPAWAQCSPDPTVADSVTNCTGTDSDGLRVTT&#xA;6 NA                          DGSMVEVAPSASVSGAGAAAITVDIAPVSGSYYARKATINVGGSVDGGTQSGISVLSGSV&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So i have made another file with list of protein ID which i want to subset from this file..I tried removing empty space from the file but couldn;t as it the sequence should come in one line but instead there is line break&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://drive.google.com/open?id=1AO9s3BuLu7nFOGUp6g5Zpm3N7MFrv93K&quot; rel=&quot;nofollow noreferrer&quot;&gt;data files&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried this &lt;code&gt;SEQ &amp;lt;- inner_join(seq,protein_id)&lt;/code&gt; whose output is this its like each ID is associated with multiple sequence &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fig|6666666.151743.peg.5044 MTKNTIHRTILVCASLIAMPAMAQGTTEAGQEGASLADIVVTAQRRSENLQNVPISVTAI&#xA;2 fig|6666666.151743.peg.5063 MTVEQALARMLEGTGLEAQRLSPTAYRLVQAAPPASRRDPVRPAPPLPPEGSVDIIVTGT&#xA;3 fig|6666666.151743.peg.5135 MLRVADGYVEEAADDAIVPSPAVPLSVPSAPVRFEAAPARYGARRGPNVPAILVIVLLHA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2018-11-30T12:11:04.210" LastActivityDate="2018-11-30T14:41:22.553" Title="Subset data frame based on ID" Tags="|r|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5556" PostTypeId="2" ParentId="5554" CreationDate="2018-11-30T12:58:32.627" Score="1" Body="&lt;p&gt;As far as I understand, it seems that the first table you provided contains biological/experimental information related to your genes, i.e. the nodes of the network you will build. The second table is the one you need to build the network: each line is an edge. You can use it as input and build the network using the IGraph library which works for both Python (as you requested) or R (as you suggested as an alternative).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most probably, but I can't say for sure, the first table was just a dataset from which you (or someone for you) recovered a list of nodes that where then submitted to a database, i.e. String, commonly used to recover information related to the interactions between your nodes, i.e. genes. Once you have this info (in the second table) you can build the network!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;if you're not familiar with IGraph, you can have a look at this question describing &lt;a href=&quot;https://stackoverflow.com/questions/36521723/read-edges-from-csv-file-to-igraph-using-python&quot;&gt;how to obtain an IGraph graph from a csv file&lt;/a&gt; or this other one which &lt;a href=&quot;https://stackoverflow.com/questions/32610013/r-igraph-convert-a-csv-file-into-a-graph-file&quot;&gt;uses a different csv file format to build a graph&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3119" LastEditorUserId="3119" LastEditDate="2018-11-30T13:14:34.560" LastActivityDate="2018-11-30T13:14:34.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5557" PostTypeId="2" ParentId="5555" CreationDate="2018-11-30T13:07:10.270" Score="5" Body="&lt;p&gt;I want to advise you to use FASTA format, instead of obscure unknown formats. In R, there are many packages that can handle FASTA format and it is therefore more convenient. I will show you how to make a FASTA file from your data, and then I will show an example of how to import such a FASTA file and sub select your proteins of interest.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;input &amp;lt;- scan(&quot;seq2.txt&quot;, what = character(), sep = &quot;\t&quot;)&#xA;length(input)&#xA;new &amp;lt;- gsub(&quot;f&quot;, &quot;&amp;gt;f&quot;, input[-1:-2])&#xA;write.table(new, &quot;seq2.fasta&quot;, quote = F, row.names = F, col.names = F)&#xA;&#xA;library(seqinr)&#xA;fasta.file &amp;lt;- read.fasta(&quot;seq2.fasta&quot;, seqtype = &quot;AA&quot;, as.string = TRUE)&#xA;prot.id &amp;lt;- scan(&quot;protein_id.txt&quot;, what = character(), sep = &quot;\t&quot;)&#xA;selection &amp;lt;- fasta.file[names(fasta.file) %in% prot.id]&#xA;length(selection)&#xA;[1] 87&#xA;&#xA;head(selection, 2)&#xA;$`fig|6666666.151743.peg.5044`&#xA;[1] &quot;MTKNTIHRTILVCASLIAMPAMAQGTTEAGQEGASLADIVVTAQRRSENLQNVPISVTAITSEQAIKLGITQPDVIAQMVPGLQLNHSAWGLTPFLRGVGTNSVAIGNEPAVALFVDDVYLPTGSAGVFQLNSISSIEVLKGPQGTLFGRNATGGVIHIRTRDPSFDPSASATASYGNYDTISGQLYGSTGLTDTVAINFALQGIDQRKGWGVNGTTGNEANSMSGWGGRAKLLWNMGEETSLLVAGSYDRRSSDQGQPGRSIAGTFTRDGYSADAAGLGFYDVMSNLDTVQKSTTRSLSGKLTHDFGSARLVLISAFSKTVVDPVVSDSDGSPSPFFQTREFAMGGRTFTQEVQLLSPEGSAVKWILGAFYLHDRSFMDATYEGQAFGTSKSHTDAVQRTNSIAGFGQVTTEILPRTNLTLGLRYTSDRRSLDAVARLLPATGAAVTFGPFEDAKTFNSVTGRASIDHHFSDDLMGYIAYNRGFKSGIYNFTGIVAGTTTLPPPVNPETLDAYTIGFKSEFLNRRLRVNAEAFLYEYSDLQVQIVLPTGTTLTNASKATIKGIDFEISAIPVRNLTVSASIGLLDGRYKDFRNAPSFFLLPPNAPTPYDPARCSGPNPYPAVQGSAPATQRSCDVSGNRLIQAPPFSSNLQLLYTLPTDLGDFDLSASWAHGGNYEYDVFSSPLTRQPRTDIVNGSVQWTAPGGTFNVRLWANNLTGEKYYTIATTGSTAGAKYAAAAPRTYGVTLGANF&quot;&#xA;attr(,&quot;name&quot;)&#xA;[1] &quot;fig|6666666.151743.peg.5044&quot;&#xA;attr(,&quot;Annot&quot;)&#xA;[1] &quot;&amp;gt;fig|6666666.151743.peg.5044&quot;&#xA;attr(,&quot;class&quot;)&#xA;[1] &quot;SeqFastaAA&quot;&#xA;&#xA;$`fig|6666666.151743.peg.5063`&#xA;[1] &quot;MTVEQALARMLEGTGLEAQRLSPTAYRLVQAAPPASRRDPVRPAPPLPPEGSVDIIVTGTKREEPLARLPISVAIVQAPGEQGGALLPGTEEVAQRLDGLTLTNLGPGRNRQFIRGVADSPFNGNSQSTVAILINDARATYNAPDPDLKLVDVERVELLKGPQGSLYGTGTLGGVYHIVTRPPDAGSFAAMAGMGASLATHGEMGGSAEAMVNLPVVADRLALRAVAYAVREPGWIDTAGRGDANRGELRGGRVALRALAGDWTVDLNGMAQMRDVADSQYVYARHALERPAQPAEPHDNDFALMALRVAGKLGGADLVYAGNWVDHDVSSTLDASDLSASAFGIAPPALYLEQRRNRLFDQEVRLSQTRDDGLSWLAGLSFVDVRNHLSGVVRGVRDETVGTGIQGVVEFSAFGDMRMPLTGALSLSVGGRLFHSRIEEEQGARHARIEASVARSGFTPSLALSWQAGAHDFLYARYAGALRPGGLSRTGAASAERFESDELQTVEMGWRHSASAWLTVNLDAFLTTWSHMQSDYLLADGLIGTRNVGDARIRGVEASVDWKPAAGWAISAGLVGQSALLTRTNEGLRLEDRHLPVVPDWKARFGVNRHFDLSGWSGSMGVNLAYLGPSRLSFDPGLDRRMGDYALANADLALRRKDWSIRLSGSNLLDCKADSFAFGNPFALARGTQYTPLRPRQWAIGFTKNW&quot;&#xA;attr(,&quot;name&quot;)&#xA;[1] &quot;fig|6666666.151743.peg.5063&quot;&#xA;attr(,&quot;Annot&quot;)&#xA;[1] &quot;&amp;gt;fig|6666666.151743.peg.5063&quot;&#xA;attr(,&quot;class&quot;)&#xA;[1] &quot;SeqFastaAA&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastEditorUserId="298" LastEditDate="2018-11-30T14:41:22.553" LastActivityDate="2018-11-30T14:41:22.553" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5558" PostTypeId="1" CreationDate="2018-11-30T14:51:18.053" Score="3" ViewCount="106" Body="&lt;p&gt;I would like to know why there are different entries for the same mutation in COSMIC, for example &lt;a href=&quot;https://cancer.sanger.ac.uk/cosmic/mutation/overview?id=6954941&quot; rel=&quot;nofollow noreferrer&quot;&gt;COSM6954941&lt;/a&gt; and &lt;a href=&quot;https://cancer.sanger.ac.uk/cosmic/mutation/overview?id=12833&quot; rel=&quot;nofollow noreferrer&quot;&gt;COSM12833&lt;/a&gt; both refer to ERBB4 c.908C&gt;A.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this specific case the field &lt;strong&gt;Gene name&lt;/strong&gt; in the two entries is different, COSM6954941 report ERBB4_ENST00000402597 while &lt;a href=&quot;https://cancer.sanger.ac.uk/cosmic/mutation/overview?id=12833&quot; rel=&quot;nofollow noreferrer&quot;&gt;COSM12833&lt;/a&gt; report ERBB4, but there are many cases in which the &lt;strong&gt;Gene name&lt;/strong&gt; is the same, for example &lt;a href=&quot;https://cancer.sanger.ac.uk/cosmic/mutation/overview?id=1361&quot; rel=&quot;nofollow noreferrer&quot;&gt;COSM1361&lt;/a&gt; and &lt;a href=&quot;https://cancer.sanger.ac.uk/cosmic/mutation/overview?id=1219&quot; rel=&quot;nofollow noreferrer&quot;&gt;COSM1219&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3772" LastEditorUserId="96" LastEditDate="2019-01-15T14:23:18.727" LastActivityDate="2019-01-15T14:23:18.727" Title="multiple COSMIC id for the same mutation" Tags="|variation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5559" PostTypeId="2" ParentId="5538" CreationDate="2018-11-30T16:11:05.687" Score="2" Body="&lt;p&gt;As others have mentioned, this is likely an issue with having multiple versions of R or RStudio installed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I highly recommend that any R/RStudio version not installed inside conda be removed. This includes any installed libraries. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've previously written a &lt;a href=&quot;https://jlboat.github.io/how-to-set-up-a-bioinformatics-environment.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt; about this topic, and I can say from experience that installing R packages through conda can cause lots of headaches. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll have an easier time if you install R/RStudio via conda and then install R packages the traditional way, inside R/RStudio.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, for example. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bad: &lt;code&gt;conda install -c bioconda bioconductor-deseq2&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conda install rstudio&#xA;# Then, inside RStudio&#xA;source(&quot;https://bioconductor.org/biocLite.R&quot;)&#xA;biocLite(&quot;DESeq2&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3499" LastEditorUserId="3499" LastEditDate="2018-11-30T16:24:09.673" LastActivityDate="2018-11-30T16:24:09.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5560" PostTypeId="1" AcceptedAnswerId="5562" CreationDate="2018-11-30T23:03:51.390" Score="5" ViewCount="725" Body="&lt;p&gt;Probably a silly oversight on my part, but I'm trying to filter all the vcfs in a directory with bcftools using a simple loop. My basic command is working fine:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools filter -i 'QUAL &amp;gt; 1000' -o filter/file1out.vcf file1in.vcf &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but when I try to loop it, it echoes through each filename, but generates no output files or error messages.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for f in *.vcf; do echo &quot;filtering &lt;span class=&quot;math-container&quot;&gt;$f&quot;; bcftools filter -i 'QUAL &amp;gt; 1000' -o filter/$&lt;/span&gt;f_out.vcf $f; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Am I not handling the output direction properly? Any help much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="3505" LastEditorUserId="964" LastEditDate="2018-12-01T23:09:58.780" LastActivityDate="2018-12-03T17:18:56.133" Title="bcftools filtering all files in a directory" Tags="|vcf|bcftools|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5561" PostTypeId="2" ParentId="5540" CreationDate="2018-12-01T08:06:11.057" Score="1" Body="&lt;p&gt;I overlooked the fact that (for my data) about 1:50,000 results returned from my blasting yielded a &quot;;&quot; within the hit denoting two taxids and did not account for these in my script.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How a given researcher will respond to their own results having &quot;;&quot;-hits will probably vary (you may want to do extra analysis, etc.).  In my case, I am only interested in the first taxid before the &quot;;&quot;.  So I added a simply python .split() to my script before appending the taxid to the list that I query:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if __name__ == '__main__':&#xA;    file = open(sys.argv[1], &quot;r&quot;)    &#xA;    taxids = []&#xA;    contigs = []&#xA;    for line in file:&#xA;        line = line.split(&quot;\n&quot;)[0]&#xA;        tID = line.split(&quot;,&quot;)[0]&#xA;        if ';' in tID: &#xA;            tID = tID.split(&quot;;&quot;)[0]&#xA;        taxids.append(tID)&#xA;        contigs.append(line.split(&quot;,&quot;)[1])&#xA;&#xA;    desired_ranks = ['superkingdom', 'phylum']&#xA;    results = list()&#xA;    for taxid in taxids:&#xA;        results.append(list())&#xA;        results[-1].append(str(taxid))&#xA;        ranks = get_desired_ranks(taxid, desired_ranks)&#xA;        for key, rank in ranks.items():&#xA;            if rank != '&amp;lt;not present&amp;gt;':&#xA;                results[-1].append(list(ncbi.get_taxid_translator([rank]).values())[0])&#xA;            else:&#xA;                results[-1].append(rank)&#xA;&#xA;    i = 0&#xA;    for result in results:&#xA;        print(contigs[i] + ','),&#xA;        print(','.join(result))&#xA;        i += 1&#xA;&#xA;    file.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2446" LastActivityDate="2018-12-01T08:06:11.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5562" PostTypeId="2" ParentId="5560" CreationDate="2018-12-01T09:57:56.710" Score="5" Body="&lt;p&gt;I assume your shell is looking for the variable &lt;code&gt;$f_out&lt;/code&gt;, but can't find it because you did not define it. It probably generated a hidden file with the name &lt;code&gt;.vcf&lt;/code&gt; in your output directory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Instead, use the following: &lt;code&gt;-o &lt;span class=&quot;math-container&quot;&gt;${f}_out.vcf&lt;/code&gt;. This will &quot;protect/limit&quot; the bash variable to just &lt;code&gt;$&lt;/span&gt;f&lt;/code&gt; rather than &lt;code&gt;$f_out&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When encountering issues like this it's worth putting &lt;code&gt;echo&lt;/code&gt; in your loop right before your command, in this case bcftools (adding quotes whenever necessary). That would show you the command as how it is going to be executed and would show you, in this case, that the output name is incorrect.&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2018-12-01T09:57:56.710" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5563" PostTypeId="2" ParentId="5560" CreationDate="2018-12-02T02:56:35.130" Score="1" Body="&lt;p&gt;For one-liners, &lt;code&gt;xargs&lt;/code&gt; is often more convenient than the &lt;code&gt;for&lt;/code&gt; loop:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-shell prettyprint-override&quot;&gt;&lt;code&gt;ls *.vcf | sed s,.vcf$,, \&#xA;  | xargs -i echo 'bcftools filter -i &quot;QUAL &amp;gt; 1000&quot; -o &quot;filter/{}_out.vcf&quot; &quot;{}.vcf&quot;' \&#xA;  | parallel&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that your command line (with @WouterDeCoster's fix) writes output to &lt;code&gt;foo.vcf_out.vcf&lt;/code&gt;. My command line writes to &lt;code&gt;foo_out.vcf&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2018-12-03T17:18:56.133" LastActivityDate="2018-12-03T17:18:56.133" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5564" PostTypeId="1" CreationDate="2018-12-02T08:11:39.547" Score="2" ViewCount="138" Body="&lt;p&gt;I am a computer science student, who was interested in taking research in the field of bioinformatics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first research idea I had was to make a model that could predict the possibility of type 2 diabetes by using machine learning such as convolutional neural networks or recurrent neural networks by training the model with fasta data of protein that causes diabetes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The research will focus more on the architecture of the machine learning itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this research possible to work on ? If yes, how and where can I get the data?&#xA;All my lecturers have no background in bioinformatic, so I can't ask any of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&#xA;I m sorry, not protein but gene like CAPN10, CDKAL1, CDKN2A, HHEX, HNF4A, IGF2BP2, KCNJ11, PPARG, SLC30A8, and TCF7L2 that shown have risk to be associated in Type 2 Diabetes (&lt;a href=&quot;http://diabetes.diabetesjournals.org/content/56/12/3033&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://diabetes.diabetesjournals.org/content/56/12/3033&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="3744" LastEditorUserId="3744" LastEditDate="2018-12-02T22:26:47.590" LastActivityDate="2018-12-02T22:26:47.590" Title="Question about research in bioinformatics for Computer Science student" Tags="|fasta|machine-learning|" AnswerCount="1" CommentCount="4" ClosedDate="2018-12-07T18:14:20.943" ContentLicense="CC BY-SA 4.0" />
  <row Id="5565" PostTypeId="2" ParentId="5564" CreationDate="2018-12-02T16:48:20.920" Score="1" Body="&lt;p&gt;I think , this kind of research never stop until find way for cure .&#xA;I have some research in heart disease,and write this paper&lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/8472615/authors#authors&quot; rel=&quot;nofollow noreferrer&quot;&gt;ECG&lt;/a&gt;, you can have some research in improve learning method to find fast and more accurate diabetes , there are lot of database your can search in google and more famous is &lt;a href=&quot;https://archive.ics.uci.edu/ml/datasets.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCI&lt;/a&gt;   .&lt;/p&gt;&#xA;" OwnerUserId="3649" LastActivityDate="2018-12-02T16:48:20.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5566" PostTypeId="2" ParentId="5496" CreationDate="2018-12-03T01:23:15.830" Score="4" Body="&lt;p&gt;&lt;code&gt;snippy-core&lt;/code&gt; is a tool that processes the &lt;code&gt;snippy_outdir_{1..n}&lt;/code&gt; folders and produces the following files:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;core.aln&lt;/code&gt; – a fasta format file that contains only the polymorphic sites (of {'A', 'C', 'T' and 'G'}) in your input set, whose length tends to decrease asymptotically as more isolates are added&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;core.full.aln&lt;/code&gt; – a fasta format file that contains all the core sites and base calls at any site where the reference has a site. The reference is not padded but the sequences mapped to it may be padded with gaps and 'N's to match the length of the reference.  The alignment length does not change as more isolates are added.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;snippy{-core}&lt;/code&gt; was written by Torsten Seemann et al.  &lt;code&gt;snp-sites&lt;/code&gt; was written by Andrew Page et al.   &lt;code&gt;snp-sites&lt;/code&gt; will reduce any alignment to the core polymorphic sites (columns) and will still output a fasta.  Therefore the output of a &lt;code&gt;core.aln&lt;/code&gt; processed through &lt;code&gt;snp-sites&lt;/code&gt; should be the identical the input.  All fasta alignments can then be processed downstream using your tree inference package of choice (e.g., &lt;code&gt;FastTree&lt;/code&gt;, &lt;code&gt;IQ-Tree&lt;/code&gt;, &lt;code&gt;MrBayes&lt;/code&gt;, &lt;code&gt;BEAST&lt;/code&gt; ...).  If you want a distance matrix output from a fasta alignment, use something like &lt;code&gt;snp-dists&lt;/code&gt;, also written by Torsten Seemann, or the &lt;code&gt;APE&lt;/code&gt; package in &lt;code&gt;R&lt;/code&gt;.  Be sure to consider how gaps and Ns are treated during this step.  &lt;/p&gt;&#xA;" OwnerUserId="3381" LastActivityDate="2018-12-03T01:23:15.830" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5567" PostTypeId="1" CreationDate="2018-12-01T01:50:20.727" Score="5" ViewCount="13994" Body="&lt;p&gt;I am looking at the supplemental data from the paper &quot;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/26029871&quot; rel=&quot;nofollow noreferrer&quot;&gt;An allelic series of miR-17 ∼ 92-mutant mice uncovers functional specialization and cooperation among members of a microRNA polycistron&lt;/a&gt;&quot; which lists the genes that are differentially expressed between a particular knock-out mouse and the wild-type control. In addition to showing the logFC changes for each of the genes, the table includes their FDR value. In many cases, the FDR value is 1. What does this mean?&lt;/p&gt;&#xA;" OwnerDisplayName="leah" LastEditorUserId="2085" LastEditDate="2018-12-03T21:32:03.320" LastActivityDate="2018-12-03T21:32:03.320" Title="What does an FDR value of 1 in RNA-seq mean?" Tags="|differential-expression|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5568" PostTypeId="2" ParentId="5567" CreationDate="2018-12-01T21:57:04.790" Score="9" Body="&lt;p&gt;FDR stands for &lt;a href=&quot;https://en.wikipedia.org/wiki/False_discovery_rate&quot; rel=&quot;noreferrer&quot;&gt;False Discovery Rate&lt;/a&gt;. It is a statistic tool used in multiple hypothesis testing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you may know, when you use a p-value cutoff (usually 0.05) for your experiments, it means strictly speaking that &quot;is there was actually no signal, there would be a probability of 0.05 to observe this kind of extreme values&quot;. This can be understood as &quot;there's a 5% chance that what you call a hit is actually not signal, just some extreme value that happened by chance, and really there's nothing to see here&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This definition has deep consequences if you run many experiments: imagine that you run 100 experiments (say, you test the effect of 100 different molecules), if none of them has any effect, on average you could still get 5% of false positives. So if you run these 100 experiments, and you get, say, 10 hits with a p-value below your thresold of 0.05; how can you tell which ones are real and which ones are just chance in the absence of biological signal?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is where you can use FDR. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you control for a false discovery rate of, say, 0.2 (20%), it means that after a few computations (the Benjamini-Hochberg procedure described on wikipedia is simple and commonly used) you will lower your p-value cutoff so as to be sure that, &lt;em&gt;among the N hits&lt;/em&gt; (probably less than the 10 you initially selected above) &lt;em&gt;that you got above, a maximum of 20% are actually false positives&lt;/em&gt; (i.e. no biological effect, but surprisingly high/low signal values by pure chance).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the article you mention they have a different FDR value for each sample, which is unusual. The values are not round numbers, so it seems likely that they are numbers obtained after applying the FDR correction (i.e. they are &quot;transformed p-values&quot;) on which they based their decision to include the samples or not. The FDR cutoff that they chose should be written somewhere in the main text.&lt;/p&gt;&#xA;" OwnerUserId="3781" OwnerDisplayName="Mowgli" LastActivityDate="2018-12-03T02:51:46.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5569" PostTypeId="2" ParentId="5567" CreationDate="2018-12-02T23:49:05.523" Score="2" Body="&lt;p&gt;In practice, you can interpret it just like a p-value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An FDR value is a p-value adjusted for multiple tests (by the Benjamini-Hochberg procedure). It stands for the “false discovery rate” it corrects for multiple testing by giving the proportion of tests above threshold alpha that will be false positives (i.e., detected when the null hypothesis is true). Note this is less stringent than the Holm-Bonferroni FWER adjusted p-value (which considers the chance of any false positives among significant results).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for a FDR p-value of 0.05, up to 5% of these tests will be false positives (note this is not the case for uncorrected raw p-values).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an FDR p-value of 1, up to 100% of these tests will be false positives. This makes sense since if you take test with a p-value if 1, then it should include all of the negative results, where the null hypothesis cannot be rejected. Only tests with p-values lower than this, have any chance of being rejected under the null hypothesis as true positives. If you are getting FDR adjusted p-values, you have done so many tests that you do not have the power to detect true positives and cannot rule out these being false positives due to noise of multiple testing. So an FDR p-value of 1 is definitely not significant under any circumstances.&lt;/p&gt;&#xA;" OwnerUserId="1855" OwnerDisplayName="Tom Kelly" LastEditorUserId="1855" LastEditDate="2018-12-03T11:22:50.413" LastActivityDate="2018-12-03T11:22:50.413" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5570" PostTypeId="1" CreationDate="2018-12-03T10:34:30.417" Score="3" ViewCount="114" Body="&lt;p&gt;I am trying to build a features matrix to be used for Random Forest based classification. I'd like to add, as features, short motifs which are common to all the protein sequences belonging to a specific gene family.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to use MEME but I'd like to know if there are also other good tools (online or command line). Keep in mind I have more than 10.000 sequences and that I prefer short motifs (3/4 aa long).&lt;/p&gt;&#xA;" OwnerUserId="2862" LastActivityDate="2018-12-04T08:53:41.757" Title="De novo motif discovery in protein sequences" Tags="|proteins|sequence-analysis|motifs|machine-learning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5571" PostTypeId="2" ParentId="5560" CreationDate="2018-12-03T16:12:51.863" Score="2" Body="&lt;p&gt;Using a &lt;code&gt;for&lt;/code&gt; loop for this is &lt;a href=&quot;https://mywiki.wooledge.org/ParsingLs&quot; rel=&quot;nofollow noreferrer&quot;&gt;by far the safer option&lt;/a&gt;. However, you need to &lt;a href=&quot;https://unix.stackexchange.com/a/131767/22222&quot;&gt;quote your variables&lt;/a&gt; and also make sure you're passing the right variable name. While you're at it, you can also remove the extension so that your filtered file name can be &lt;code&gt;foo_out.vcf&lt;/code&gt; instead of &lt;code&gt;foo.vcf_out.vcf&lt;/code&gt; (which is what your loop would create. Putting all that together:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for f in *.vcf; do &#xA;    echo &quot;filtering &lt;span class=&quot;math-container&quot;&gt;$f&quot;; &#xA;bcftools filter -i 'QUAL &amp;gt; 1000' -o filter/&quot;$&lt;/span&gt;{f//.vcf}&quot;_out.vcf &quot;$f&quot;; done&#xA;fi&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2018-12-03T16:12:51.863" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5572" PostTypeId="2" ParentId="5481" CreationDate="2018-12-03T19:03:23.063" Score="1" Body="&lt;p&gt;&lt;em&gt;Opinion&lt;/em&gt;: I prefer to use EBI/ENA because the turnaround time for &quot;customer service&quot; on editing metadata about a submission is faster than on NCBI. Inquiries to change metadata on NCBI can go unanswered for up to six months in my experience.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Answer to your question&lt;/em&gt;: No, there are no technical reasons that make it preferable to submit data through one website or the other. &lt;strong&gt;All entries end up being hosted by both databases anyway.&lt;/strong&gt; The preference is up to the user to select whichever upload interface they prefer.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-12-03T19:03:23.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5573" PostTypeId="1" CreationDate="2018-12-03T19:26:55.763" Score="4" ViewCount="230" Body="&lt;p&gt;I was trying to find a way to find overlaps between two genomic ranges. I found a post at &lt;a href=&quot;https://www.biostars.org/p/95062/#95098&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt; but it couldn't consider the chromosome information. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(IRanges)&#xA;df1 = data.frame(chr = c(&quot;chr1&quot;, &quot;chr12&quot;), start = c(10000, 10000), end = c(20000, 20000))&#xA;df2 = data.frame(chr = rep(&quot;chr1&quot;, 4), posn = c(100, 12000, 15000, 250000), x = rep(1, 4), y = rep(2, 4), z = rep(3, 4))&#xA;&#xA;df1.ir = IRanges(start = df1&lt;span class=&quot;math-container&quot;&gt;$start, end = df1$&lt;/span&gt;end, names = df1&lt;span class=&quot;math-container&quot;&gt;$chr)&#xA;df2.ir = IRanges(start = df2$&lt;/span&gt;posn, end = df2&lt;span class=&quot;math-container&quot;&gt;$posn, names = df2$&lt;/span&gt;chr) &#xA;&#xA;df1.it = NCList(df1.ir)&#xA;&#xA;overlap_it = findOverlaps(df2.ir, df1.it)&#xA;print(overlap_it) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It returns the overlap from both chromosomes. I am wondering if there is a way to keep the chromosome information intact. &lt;/p&gt;&#xA;" OwnerUserId="561" LastEditorUserId="561" LastEditDate="2018-12-03T21:10:58.147" LastActivityDate="2018-12-04T00:40:10.653" Title="How to find genomic overlaps faster?" Tags="|bioconductor|genomicranges|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5574" PostTypeId="2" ParentId="5573" CreationDate="2018-12-04T00:40:10.653" Score="3" Body="&lt;p&gt;Just wanted to follow up to a question asked at &lt;a href=&quot;https://www.biostars.org/p/95062/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;. My solution and suggestion when dealing with genomic ranges. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(GenomicRanges)&#xA;df1 = data.frame(chr = c(&quot;chr1&quot;,  &quot;chr12&quot;), start = c(10000,  10000), end = c(20000, 20000))&#xA;df2 = data.frame(chr = rep(&quot;chr1&quot;, 4), posn = c(100, 12000, 15000, 250000), x = rep(1, 4), y = rep(2, 4), z = rep(3, 4))&#xA;&#xA;df1.ir = IRanges(start = df1&lt;span class=&quot;math-container&quot;&gt;$start, end = df1$&lt;/span&gt;end, names = df1&lt;span class=&quot;math-container&quot;&gt;$chr)&#xA;df2.ir = IRanges(start = df2$&lt;/span&gt;posn, end = df2&lt;span class=&quot;math-container&quot;&gt;$posn, names = df2$&lt;/span&gt;chr) &#xA;&#xA;df1.it = NCList(df1.ir)&#xA;&#xA;overlap_it = findOverlaps(df2.ir, df1.it)&#xA;print(overlap_it)&#xA;&#xA;# define a genomic interval tree (git) for faster search &#xA;&#xA;df1.gr = GRanges (IRanges(start = df1&lt;span class=&quot;math-container&quot;&gt;$start, end = df1$&lt;/span&gt;end), seqnames=df1&lt;span class=&quot;math-container&quot;&gt;$chr) &#xA;df2.gr = GRanges(IRanges(start=df2$&lt;/span&gt;posn, end = df2&lt;span class=&quot;math-container&quot;&gt;$posn), seqnames = df2$&lt;/span&gt;chr) &#xA;&#xA;df1.git  = GNCList(df1.gr)&#xA;t1 = Sys.time ()&#xA;overlap_git = findOverlaps(df2.gr, df1.git)&#xA;t2 = Sys.time()&#xA;sub1 = difftime(t2, t1, tz, units = c(&quot;auto&quot;))&#xA;print (sub1)&#xA;print (overlap_git)&#xA;&#xA;t1 = Sys.time ()&#xA;overlap_intersect = intersect(df1.gr, df2.gr)&#xA;t2 = Sys.time ()&#xA;print(overlap_intersect)&#xA;sub2 = difftime(t2, t1, tz, units = c(&quot;auto&quot;))&#xA;print(sub2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;One should be informed about differences between IRanges and GRanges (for example chromosome context); for example, &lt;code&gt;findOverlaps&lt;/code&gt; with &lt;code&gt;IRanges&lt;/code&gt; returns intersections from both &lt;code&gt;chr1&lt;/code&gt; and &lt;code&gt;chr12&lt;/code&gt;, but with &lt;code&gt;GRanges&lt;/code&gt; it doesn't &lt;/li&gt;&#xA;&lt;li&gt;Interval Tree based search is way faster (almost 10x) than simple &lt;code&gt;intersect&lt;/code&gt; function&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="561" LastActivityDate="2018-12-04T00:40:10.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5575" PostTypeId="1" CreationDate="2018-12-04T06:37:42.860" Score="2" ViewCount="80" Body="&lt;p&gt;Is there an alternative to mesh2gene for linking MeSH terms to genes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The http://www.ncibi.org/gene2mesh.html site appears to be down for months now (click on &quot;Launch Gene2MeSH via gene2mesh.ncibi.org&quot;) and I've been unable to find an alternative so far...&lt;/p&gt;&#xA;" OwnerUserId="247" LastActivityDate="2018-12-05T09:50:57.587" Title="Linking of MeSH terms and genes" Tags="|gene|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5576" PostTypeId="2" ParentId="5570" CreationDate="2018-12-04T08:53:41.757" Score="3" Body="&lt;p&gt;I think MEME is a good tool for your purpose, but there are others as well. I can think of &lt;a href=&quot;http://www.ebi.ac.uk/interpro/interproscan.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;InterProScan&lt;/a&gt; for example (although I am not sure if it is really &lt;em&gt;de novo&lt;/em&gt;). &lt;a href=&quot;https://molbiol-tools.ca/Motifs.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; a summary of available tools.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2018-12-04T08:53:41.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5577" PostTypeId="1" AcceptedAnswerId="5578" CreationDate="2018-12-04T10:08:25.547" Score="5" ViewCount="253" Body="&lt;p&gt;I wrote script for pasting rsids on CADD output. Here is script.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;cd tmp&#xA;cut -f 1,2 CADD.tsv &amp;gt; fileA&#xA;paste fileA &amp;lt;(cut -f 2,125  CADD.tsv) &amp;gt; myNewFile&#xA;bedtools intersect -a  myNewFile -b New.vcf -wb |cut -f 1-4,7 &amp;gt; CADD.rsids.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have tested commands one by one and they work fine. But when I run script of these commands, it gives me this error.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sh cadd.rsids.sh&#xA;4: cadd.rsids.sh: Syntax error: &quot;(&quot; unexpected&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can anyone please tell me how to resolve this error?&#xA;Here are few lines of CADD.tsv file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;##CADD GRCh38-v1.4 (c) University of Washington, Hudson-Alpha Institute for Biotechnology and Berlin Institute of Health 2013-2018. All rights reserved.&quot;                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                              &#xA;#Chrom  Pos Ref Alt Type    Length  AnnoType    Consequence ConsScore   ConsDetail  GC  CpG motifECount motifEName  motifEHIPos motifEScoreChng oAA nAA GeneID  FeatureID   GeneName    CCDS    Intron  Exon    cDNApos relcDNApos  CDSpos  relCDSpos   protPos relProtPos  Domain  Dst2Splice  Dst2SplType minDistTSS  minDistTSE  SIFTcat SIFTval PolyPhenCat PolyPhenVal priPhCons   mamPhCons   verPhCons   priPhyloP   mamPhyloP   verPhyloP   bStatistic  targetScan  mirSVR-Score    mirSVR-E    mirSVR-Aln  cHmm_E1 cHmm_E2 cHmm_E3 cHmm_E4 cHmm_E5 cHmm_E6 cHmm_E7 cHmm_E8 cHmm_E9 cHmm_E10    cHmm_E11    cHmm_E12    cHmm_E13    cHmm_E14    cHmm_E15    cHmm_E16    cHmm_E17    cHmm_E18    cHmm_E19    cHmm_E20    cHmm_E21    cHmm_E22    cHmm_E23    cHmm_E24    cHmm_E25    GerpRS  GerpRSpval  GerpN   GerpS   tOverlapMotifs  motifDist   EncodeH3K4me1-sum   EncodeH3K4me1-max   EncodeH3K4me2-sum   EncodeH3K4me2-max   EncodeH3K4me3-sum   EncodeH3K4me3-max   EncodeH3K9ac-sum    EncodeH3K9ac-max    EncodeH3K9me3-sum   EncodeH3K9me3-max   EncodeH3K27ac-sum   EncodeH3K27ac-max   EncodeH3K27me3-sum  EncodeH3K27me3-max  EncodeH3K36me3-sum  EncodeH3K36me3-max  EncodeH3K79me2-sum  EncodeH3K79me2-max  EncodeH4K20me1-sum  EncodeH4K20me1-max  EncodeH2AFZ-sum EncodeH2AFZ-max EncodeDNase-sum EncodeDNase-max EncodetotalRNA-sum  EncodetotalRNA-max  Grantham    Dist2Mutation   Freq100bp   Rare100bp   Sngl100bp   Freq1000bp  Rare1000bp  Sngl1000bp  Freq10000bp Rare10000bp Sngl10000bp EnsembleRegulatoryFeature   dbscSNV-ada_score   dbscSNV-rf_score    RemapOverlapTF  RemapOverlapCL  RawScore    PHRED&#xA;1   3362704 T   A   SNV 0   Transcript  INTRONIC    2   intron  0.536423841 0.04    NA  NA  NA  NA  NA  NA  ENSG00000142611 ENST00000270722 PRDM16  CCDS41236.2 12:00:00 AM NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  118572  24215   NA  NA  NA  NA  0   0   0   -0.405  -1.689  -1.593  964 NA  NA  NA  NA  0   1   1   5   0   0   1   0   0   0   0   1   0   1   12  0   0   0   0   0   8   11  5   2   0   NA  NA  2.46    -4.93   1   0.02    4.53362 1.5498  3.49082 1.00224 9.30658 3.84201 5.88121 2.95189 9.31556 5.27366 4.01294 1.08327 25.99   5.10373 4.47027 1.51567 1.55624 0.7046  4.95028 1.49632 8.57929 1.45764 1.19689 0.644106    0.01783 0.01476 NA  16  1   0   15  4   10  184 17  101 1741    NA  NA  NA  5   19  -0.173019   0.424&#xA;1   7785635 T   C   SNV 0   Intergenic  DOWNSTREAM  1   downstream  0.483443709 0.053333333 NA  NA  NA  NA  NA  NA  ENSG00000049245 ENST00000054666 VAMP3   CCDS88.1    NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  112 2704    NA  NA  NA  NA  0.009   0   0   0.418   -0.242  -0.522  726 NA  NA  NA  NA  0   0   0   3   1   0   14  1   0   0   1   0   0   0   22  0   0   0   0   0   1   1   4   0   0   NA  NA  5.52    -11 NA  NA  20.4013 3.22673 13.621  2.38936 11.7271 2.25561 9.94895 2.15059 16.5529 2.79938 11.0724 2.29032 12.2962 2.07181 32.6588 8.63119 15.114  2.76883 7.39549 2.14311 14.1811 3.08082 0.603948    0.246907    0.55734 0.16905 NA  9   1   1   15  3   5   174 35  68  1667    NA  NA  NA  2   4   0.049024    1.923&#xA;1   7785635 T   C   SNV 0   Transcript  INTRONIC    2   intron  0.483443709 0.053333333 NA  NA  NA  NA  NA  NA  ENSG00000049246 ENST00000613533 PER3    CCDS72695.1 12:00:00 AM NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  112 2704    NA  NA  NA  NA  0.009   0   0   0.418   -0.242  -0.522  726 NA  NA  NA  NA  0   0   0   3   1   0   14  1   0   0   1   0   0   0   22  0   0   0   0   0   1   1   4   0   0   NA  NA  5.52    -11 NA  NA  20.4013 3.22673 13.621  2.38936 11.7271 2.25561 9.94895 2.15059 16.5529 2.79938 11.0724 2.29032 12.2962 2.07181 32.6588 8.63119 15.114  2.76883 7.39549 2.14311 14.1811 3.08082 0.603948    0.246907    0.55734 0.16905 NA  9   1   1   15  3   5   174 35  68  1667    NA  NA  NA  2   4   0.049024    1.923&#xA;1   7803233 C   T   SNV 0   Transcript  INTRONIC    2   intron  0.331125828 0.013333333 NA  NA  NA  NA  NA  NA  ENSG00000049246 ENST00000613533 PER3    CCDS72695.1 12:00:00 AM NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  182 726 NA  NA  NA  NA  0.014   0   0   -0.553  -1.498  -0.938  673 NA  NA  NA  NA  0   0   0   6   0   0   23  5   0   0   0   0   0   0   14  0   0   0   0   0   0   0   0   0   0   NA  NA  7.49    -15 NA  NA  4.36329 1.02663 4.55863 1.4118  4.72226 1.51608 6.80823 2.09997 7.44497 2.86816 5.71457 2.35876 3.06623 0.62198 18.3251 3.23263 2.1423  0.84413 8.63178 2.70111 6.29127 1.59229 0.198671    0.048037    0.0465  0.01705 NA  6   1   1   15  2   2   134 26  39  1314    NA  NA  NA  NA  NA  -0.395046   0.067&#xA;1   7808665 T   C   SNV 0   Intergenic  DOWNSTREAM  1   downstream  0.344370861 0   NA  NA  NA  NA  NA  NA  ENSG00000236266 ENST00000451646 Z98884.1    NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  5614    1683    NA  NA  NA  NA  0.082   0   0   0.463   -0.365  -0.373  659 NA  NA  NA  NA  0   0   0   5   3   1   24  0   0   0   0   0   0   0   15  0   0   0   0   0   0   0   0   0   0   NA  NA  5.23    -10.5   NA  NA  7.16527 1.57568 5.63104 1.29085 5.96112 1.41329 8.31159 2.11434 4.41828 1.14843 5.03919 1.48679 3.91787 1.13788 4.97324 1.34708 5.22225 1.10759 7.38244 1.40721 11.4505 2.73652 0.977252    0.19269 0.09436 0.03409 NA  43  1   0   10  3   2   110 30  49  1288    NA  NA  NA  7   26  0.11875 2.796&#xA;1   7808665 T   C   SNV 0   Transcript  INTRONIC    2   intron  0.344370861 0   NA  NA  NA  NA  NA  NA  ENSG00000049246 ENST00000613533 PER3    CCDS72695.1 12:00:00 AM NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  5614    1683    NA  NA  NA  NA  0.082   0   0   0.463   -0.365  -0.373  659 NA  NA  NA  NA  0   0   0   5   3   1   24  0   0   0   0   0   0   0   15  0   0   0   0   0   0   0   0   0   0   NA  NA  5.23    -10.5   NA  NA  7.16527 1.57568 5.63104 1.29085 5.96112 1.41329 8.31159 2.11434 4.41828 1.14843 5.03919 1.48679 3.91787 1.13788 4.97324 1.34708 5.22225 1.10759 7.38244 1.40721 11.4505 2.73652 0.977252    0.19269 0.09436 0.03409 NA  43  1   0   10  3   2   110 30  49  1288    NA  NA  NA  7   26  0.11875 2.796&#xA;1   7827519 C   G   SNV 0   CodingTranscript    NON_SYNONYMOUS  7   missense    0.529801325 0.066666667 NA  NA  NA  NA  P   A   ENSG00000049246 ENST00000613533 PER3    CCDS72695.1 NA  18/22   2854    0.45172523  2590    0.712909441 864 0.714049587 lcompl  NA  NA  177 17277   tolerated   0.05    benign  0.127   0.041   0   0   -0.553  -0.852  0.026   525 NA  NA  NA  NA  1   0   0   3   0   2   8   9   1   1   1   0   0   1   2   6   1   2   9   0   0   0   0   1   0   NA  NA  12.1    -24.2   NA  NA  8.89687 3.01407 58.9843 24.0334 58.3619 18.9077 55.4967 17.2149 10.5523 3.06178 87.6613 48.8498 8.66388 2.84705 31.4762 9.05454 11.8069 3.99886 8.7429  2.26128 72.4647 18.4371 1.08886 0.638979    2.10021 1.18017 27  2   2   1   19  4   8   204 31  59  1420    Promoter    NA  NA  86  94  0.346155    6.007&#xA;1   7827519 C   G   SNV 0   Intergenic  UPSTREAM    1   upstream    0.529801325 0.066666667 NA  NA  NA  NA  NA  NA  ENSG00000236266 ENST00000451646 Z98884.1    NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  177 17277   NA  NA  NA  NA  0.041   0   0   -0.553  -0.852  0.026   525 NA  NA  NA  NA  1   0   0   3   0   2   8   9   1   1   1   0   0   1   2   6   1   2   9   0   0   0   0   1   0   NA  NA  12.1    -24.2   NA  NA  8.89687 3.01407 58.9843 24.0334 58.3619 18.9077 55.4967 17.2149 10.5523 3.06178 87.6613 48.8498 8.66388 2.84705 31.4762 9.05454 11.8069 3.99886 8.7429  2.26128 72.4647 18.4371 1.08886 0.638979    2.10021 1.18017 NA  2   2   1   19  4   8   204 31  59  1420    Promoter    NA  NA  86  94  0.346155    6.007&#xA;1   7827519 C   G   SNV 0   RegulatoryFeature   REGULATORY  4   regulatory  0.529801325 0.066666667 NA  NA  NA  NA  NA  NA  NA  ENSR00000000832 NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  177 17277   NA  NA  NA  NA  0.041   0   0   -0.553  -0.852  0.026   525 NA  NA  NA  NA  1   0   0   3   0   2   8   9   1   1   1   0   0   1   2   6   1   2   9   0   0   0   0   1   0   NA  NA  12.1    -24.2   NA  NA  8.89687 3.01407 58.9843 24.0334 58.3619 18.9077 55.4967 17.2149 10.5523 3.06178 87.6613 48.8498 8.66388 2.84705 31.4762 9.05454 11.8069 3.99886 8.7429  2.26128 72.4647 18.4371 1.08886 0.638979    2.10021 1.18017 NA  2   2   1   19  4   8   204 31  59  1420    Promoter    NA  NA  86  94  0.346155    6.007&#xA;1   7828155 A   G   SNV 0   Intergenic  UPSTREAM    1   upstream    0.364238411 0.066666667 NA  NA  NA  NA  NA  NA  ENSG00000236266 ENST00000451646 Z98884.1    NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  NA  813 17022   NA  NA  NA  NA  0.009   0   0   -0.389  -0.735  -0.731  522 NA  NA  NA  NA  0   0   0   4   0   0   20  4   1   0   0   0   0   0   14  1   0   0   3   0   0   0   0   1   0   NA  NA  4.56    4.56    1   0.12    10.0753 2.35506 38.5512 20.107  21.923  10.3608 21.217  11.8934 5.61317 1.38406 9.74608 6.16783 3.76905 1.10216 24.6436 5.18488 14.8421 7.93141 4.56698 1.13989 18.7058 4.31099 0.500167    0.211141    0.16854 0.06818 NA  20  2   0   14  4   4   159 34  57  1421    NA  NA  NA  11  14  0.225659    4.402&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here are lines of New.vcf&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO&#xA;1   3362704 rs11807862  T   A   923.01  PASS    &quot;BaseCounts=31,0,0,30;BaseQRankSum=-0.108;DB;Dels=0;FS=3.561;GC=61.35;HaplotypeScore=1.7256;MQ=60;MQ0=0;MQRankSum=-0.368;QD=15.13;ReadPosRankSum=0.498;DP=282;AF=0.5;MLEAC=1;MLEAF=0.5;AN=14;AC=7&quot;&#xA;1   7785635 rs228729    T   C   2294.01 PASS    &quot;BaseCounts=0,44,0,0;DB;Dels=0;FS=0;GC=42.39;HaplotypeScore=0;MQ=60;MQ0=0;QD=28.13;BaseQRankSum=2.114;MQRankSum=-1.268;ReadPosRankSum=-0.548;DP=1430;AF=0.5;MLEAC=1;MLEAF=0.5;AN=70;AC=51&quot;&#xA;1   7803233 rs228642    C   T   2082.01 PASS    &quot;BaseCounts=0,19,0,19;BaseQRankSum=-0.963;DB;Dels=0;FS=2.884;GC=44.89;HaplotypeScore=0.9999;MQ=60;MQ0=0;MQRankSum=0.058;QD=14.92;ReadPosRankSum=0.409;DP=1468;AF=0.5;MLEAC=1;MLEAF=0.5;AN=62;AC=44&quot;&#xA;1   7808665 rs228666    T   C   1925.01 PASS    &quot;BaseCounts=0,36,0,26;BaseQRankSum=0.064;DB;Dels=0;FS=0.979;GC=30.42;HaplotypeScore=1.7333;MQ=60;MQ0=0;MQRankSum=-0.82;QD=18.03;ReadPosRankSum=-1.32;DP=939;AF=0.5;MLEAC=1;MLEAF=0.5;AN=42;AC=25&quot;&#xA;1   7827519 rs228697    C   G   786.01  PASS    &quot;BaseCounts=0,14,24,0;BaseQRankSum=0.016;DB;Dels=0;FS=4.925;GC=58.35;HaplotypeScore=0;MQ=60;MQ0=0;MQRankSum=-0.698;QD=18.63;ReadPosRankSum=0.99;DP=355;AF=0.5;MLEAC=1;MLEAF=0.5;AN=18;AC=9&quot;&#xA;1   7828155 rs2859388   A   G   2385.01 PASS    &quot;BaseCounts=0,0,56,0;DB;Dels=0;FS=0;GC=35.41;HaplotypeScore=0;MQ=60;MQ0=0;QD=24.91;BaseQRankSum=-0.231;MQRankSum=0.948;ReadPosRankSum=-0.128;DP=1136;AF=0.5;MLEAC=1;MLEAF=0.5;AN=46;AC=31&quot;&#xA;1   7830057 rs2640909   T   C   1421.01 PASS    &quot;BaseCounts=0,19,0,19;BaseQRankSum=1.051;DB;Dels=0;FS=5.041;GC=56.11;HaplotypeScore=0.734;MQ=60;MQ0=0;MQRankSum=0.496;QD=16.61;ReadPosRankSum=-0.526;DP=638;AF=0.5;MLEAC=1;MLEAF=0.5;AN=32;AC=19&quot;&#xA;1   11107089    rs12139042  G   A   1668.01 PASS    &quot;BaseCounts=43,0,0,0;DB;Dels=0;FS=0;GC=45.64;HaplotypeScore=0.734;MQ=60;MQ0=0;QD=29.7;BaseQRankSum=-1.652;MQRankSum=0.027;ReadPosRankSum=-1.192;DP=354;AF=0.5;MLEAC=1;MLEAF=0.5;AN=18;AC=10&quot;&#xA;1   11787392    rs3737967   G   A   719.01  PASS    &quot;BaseCounts=23,0,23,0;BaseQRankSum=-0.475;DB;Dels=0;FS=4.566;GC=61.85;HaplotypeScore=0.734;MQ=60;MQ0=0;MQRankSum=-0.84;QD=15.63;ReadPosRankSum=0.329;DP=378;AF=0.5;MLEAC=1;MLEAF=0.5;AN=20;AC=10&quot;&#xA;1   11790870    rs2274976   C   T   777.01  PASS    &quot;BaseCounts=0,18,0,24;BaseQRankSum=-0.042;DB;Dels=0;FS=4.677;GC=56.11;HaplotypeScore=0.6651;MQ=60;MQ0=0;MQRankSum=1.204;QD=15.05;ReadPosRankSum=0.836;DP=355;AF=0.5;MLEAC=1;MLEAF=0.5;AN=20;AC=10&quot;&#xA;1   11792243    rs1476413   C   T   1751.01 PASS    &quot;BaseCounts=1,18,0,21;BaseQRankSum=1.028;DB;Dels=0;FS=0;GC=56.11;HaplotypeScore=1.7287;MQ=60;MQ0=0;MQRankSum=-0.408;QD=16.65;ReadPosRankSum=-0.099;DP=954;AF=0.5;MLEAC=1;MLEAF=0.5;AN=50;AC=29&quot;&#xA;1   11794400    rs4846051   G   A   2616.01 PASS    &quot;BaseCounts=46,0,0,0;DB;Dels=0;FS=0;GC=59.35;HaplotypeScore=6.2573;MQ=60;MQ0=0;QD=31.75;DP=1654;AF=1;MLEAC=2;MLEAF=1;AN=78;AC=78&quot;&#xA;1   11794419    rs1801131   T   G   2028.01 PASS    &quot;BaseCounts=0,1,21,21;BaseQRankSum=-0.117;DB;Dels=0;FS=9.995;GC=59.35;HaplotypeScore=2.4328;MQ=60;MQ0=0;MQRankSum=1.03;QD=14.84;ReadPosRankSum=0.665;DP=1176;AF=0.5;MLEAC=1;MLEAF=0.5;AN=54;AC=33&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1247" LastEditorUserId="96" LastEditDate="2019-03-26T19:19:00.027" LastActivityDate="2019-03-26T19:19:00.027" Title="Bash script error at paste command" Tags="|vcf|shell|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="5578" PostTypeId="2" ParentId="5577" CreationDate="2018-12-04T14:34:22.950" Score="5" Body="&lt;p&gt;Your problem is that you have written a script in &lt;code&gt;bash&lt;/code&gt; but are then running it using &lt;code&gt;sh&lt;/code&gt;. &lt;a href=&quot;https://en.wikipedia.org/wiki/Bash_%28Unix_shell%29&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bash&lt;/a&gt;, the Bourne-again shell, is not the same as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Bourne_shell&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bourne shell&lt;/a&gt; (&lt;code&gt;sh&lt;/code&gt;). What is more, on Ubuntu, &lt;code&gt;sh&lt;/code&gt; is actually not even the Bourne shell but another minimal shell called &lt;code&gt;dash&lt;/code&gt;. The &lt;code&gt;/bin/sh&lt;/code&gt; on Ubuntu is a symlink to &lt;code&gt;dash&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ls -l /bin/sh&#xA;lrwxrwxrwx 1 root root 4 Mar  1  2018 /bin/sh -&amp;gt; dash&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Dash is a POSIX compliant shell which means it has the features defined by the &lt;a href=&quot;https://pubs.opengroup.org/onlinepubs/009695399/utilities/sh.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;POSIX specification&lt;/a&gt; for the &lt;code&gt;sh&lt;/code&gt; shell. In your case, the problem is caused by the fact that the &lt;code&gt;&amp;lt;()&lt;/code&gt; input redirection construct is supported by &lt;code&gt;bash&lt;/code&gt; but not by &lt;code&gt;dash&lt;/code&gt;. So, in order to run your script, you can do one of two things:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Make the script executable (&lt;code&gt;chmod a+x cadd.rsids.sh&lt;/code&gt;), and then run it directly:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/path/to/cadd.rsids.sh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the simplest approach since you already have the shebang line (&lt;code&gt;#!/bin/bash&lt;/code&gt;) which will cause the script to be run by &lt;code&gt;bash&lt;/code&gt; and not &lt;code&gt;sh&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Run the script with &lt;code&gt;bash&lt;/code&gt; instead of &lt;code&gt;sh&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bash /path/to/cadd.rsids.sh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Instead of&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sh /path/to/cadd.rsids.sh&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-12-04T18:57:28.410" LastActivityDate="2018-12-04T18:57:28.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5579" PostTypeId="1" CreationDate="2018-12-04T14:40:03.550" Score="5" ViewCount="670" Body="&lt;p&gt;I've been running &lt;code&gt;bwa mem -a&lt;/code&gt; for alignment, using the &lt;code&gt;-a&lt;/code&gt; flag---this will &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;output all alignments for SE or unpaired PE&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I've noticed in the SAM that there are several alignments with &lt;code&gt;*&lt;/code&gt; in the &lt;code&gt;SEQ&lt;/code&gt; and &lt;code&gt;QUAL&lt;/code&gt; fields. Based on the documentation:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;10&quot;&gt;&#xA;  &lt;li&gt;SEQ: segment SEQuence. This field can be a ‘*’ when the sequence is not stored. If not a ‘*’, the length of the sequence must equal the&#xA;  sum of lengths of M/I/S/=/X operations in CIGAR. An ‘=’ denotes the&#xA;  base is identical to the reference base. No assumptions can be made on&#xA;  the letter cases.&lt;/li&gt;&#xA;  &lt;li&gt;QUAL: ASCII of base QUALity plus 33 (same as the quality string in the Sanger FASTQ format). A base quality is the phred-scaled base&#xA;  error probability which equals −10 log10 Pr{base is wrong}. This field&#xA;  can be a ‘*’ when quality is not stored. If not a ‘*’, SEQ must not be&#xA;  a ‘*’ and the length of the quality string ought to equal the length&#xA;  of SEQ.&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;it appears the sequence isn't stored. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would strongly prefer to have the sequence in this case. Is there any to direct bwa to output these sequences? &lt;/p&gt;&#xA;" OwnerUserId="1770" LastEditorUserId="298" LastEditDate="2018-12-04T15:42:29.027" LastActivityDate="2018-12-04T16:48:40.810" Title="How to output all sequences with bwa mem, not `*`?" Tags="|sequence-alignment|bwa|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5580" PostTypeId="2" ParentId="5579" CreationDate="2018-12-04T16:48:40.810" Score="5" Body="&lt;p&gt;I don't believe that is an option.  Internally we created a &lt;a href=&quot;https://github.com/lh3/bwa/pull/36&quot; rel=&quot;noreferrer&quot;&gt;patch&lt;/a&gt; and submitted it to &lt;a href=&quot;https://github.com/lh3/bwa&quot; rel=&quot;noreferrer&quot;&gt;BWA&lt;/a&gt; however that was 4 years ago and it has not been accepted.&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2018-12-04T16:48:40.810" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="5581" PostTypeId="2" ParentId="5538" CreationDate="2018-12-04T17:36:50.927" Score="5" Body="&lt;p&gt;Ultimately my colleague helped me to solve the issue by following the steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Created environment: &lt;code&gt;conda create --name myenv&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Activated it: &lt;code&gt;source activate myenv&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Edited &lt;code&gt;.condarc&lt;/code&gt; to set the priority of the channels to be the following (top to bottom): &lt;code&gt;conda-forge&lt;/code&gt;, &lt;code&gt;bioconda&lt;/code&gt;, &lt;code&gt;r&lt;/code&gt;, &lt;code&gt;default&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Checked that the channels are set in the correct order: &lt;code&gt;conda config --get channels&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;conda install r=3.5.1&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;conda install r-essentials&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;conda install rstudio&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Installation of all of the &lt;code&gt;bioconductor&lt;/code&gt; packages through &lt;code&gt;bioconda&lt;/code&gt; and never install anything from inside &lt;code&gt;R&lt;/code&gt; or &lt;code&gt;Rstudio&lt;/code&gt; (I guess that is how it was broken)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Then launching &lt;code&gt;rstudio&lt;/code&gt; from within the environment. Maybe it is not a kind of quick fix and it could have been solved easier, but I was not able to.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="2005" LastEditDate="2018-12-04T17:59:56.673" LastActivityDate="2018-12-04T17:59:56.673" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5582" PostTypeId="1" CreationDate="2018-12-05T00:10:40.527" Score="2" ViewCount="34" Body="&lt;p&gt;Do any methods exist (or are in the process of development) for investigating transcript data without lysing the cells, i.e, destroying the sample?&lt;/p&gt;&#xA;" OwnerUserId="2875" LastEditorUserId="2085" LastEditDate="2018-12-05T04:24:27.740" LastActivityDate="2018-12-05T04:24:27.740" Title="Real time transcript profiles" Tags="|rna-seq|transcriptome|rna|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5583" PostTypeId="2" ParentId="5582" CreationDate="2018-12-05T03:30:22.010" Score="3" Body="&lt;p&gt;&lt;strong&gt;Answer:&lt;/strong&gt;&#xA;There are no methods that exist for investigating transcript data inside a cell without destroying the cell. Separating RNA from cells requires essentially destroying the cells and using a &lt;a href=&quot;https://en.wikipedia.org/wiki/Trizol&quot; rel=&quot;nofollow noreferrer&quot;&gt;chemical cocktail&lt;/a&gt; to split its constituent components.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Alternative 1:&lt;/strong&gt; One option is to investigate extracellular transcripts of living cells is to centrifuge cells in culture and isolating RNA from the growth media supernatant &lt;a href=&quot;https://currentprotocols.onlinelibrary.wiley.com/doi/abs/10.1002/0471143030.cb0322s30%4010.1002/%28ISSN%291934-2616.SubcellularFractionationandIsolationofOrganelles&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1]&lt;/a&gt;. Cell-free DNA and cell-free RNA are hot topics in contemporary biology and bioinformatics as a potential source for disease biomarkers. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5133845/&quot; rel=&quot;nofollow noreferrer&quot;&gt;[2]&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Alternative 2:&lt;/strong&gt; Another option to monitor a transcript from a living cell is to use a transgenic construct with a reporter molecule like &lt;a href=&quot;https://en.wikipedia.org/wiki/Luciferase&quot; rel=&quot;nofollow noreferrer&quot;&gt;luciferase&lt;/a&gt; or a &lt;a href=&quot;https://en.wikipedia.org/wiki/Green_fluorescent_protein&quot; rel=&quot;nofollow noreferrer&quot;&gt;fluorescent protein&lt;/a&gt; to produce light or become fluorescent when a promotor is activated [&lt;a href=&quot;https://pubs.rsc.org/en/content/articlelanding/2016/cs/c6cs00296j/unauth#!divAbstract&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;, &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0958166907000043&quot; rel=&quot;nofollow noreferrer&quot;&gt;4&lt;/a&gt;]. This does not directly measure transcript quantity but answers a similar question: how does transcription change in a living cell?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sources:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://currentprotocols.onlinelibrary.wiley.com/doi/abs/10.1002/0471143030.cb0322s30%4010.1002/%28ISSN%291934-2616.SubcellularFractionationandIsolationofOrganelles&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1] Théry, C., Amigorena, S., Raposo, G., &amp;amp; Clayton, A. (2006). Isolation and characterization of exosomes from cell culture supernatants and biological fluids. Current protocols in cell biology, 30(1), 3-22.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.mdpi.com/1422-0067/17/11/1845&quot; rel=&quot;nofollow noreferrer&quot;&gt;[2] Yu, X. M., Wu, Y. C., Liu, X., Huang, X. C., Hou, X. X., Wang, J. L., ... &amp;amp; Ling, Z. Q. (2016). Cell-Free RNA content in peripheral blood as potential biomarkers for detecting circulating tumor cells in non-small cell lung carcinoma. International journal of molecular sciences, 17(11), 1845.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://pubs.rsc.org/en/content/articlelanding/2016/cs/c6cs00296j/unauth#!divAbstract&quot; rel=&quot;nofollow noreferrer&quot;&gt;[3] Kaskova, Z. M., Tsarkova, A. S., &amp;amp; Yampolsky, I. V. (2016). 1001 lights: luciferins, luciferases, their mechanisms of action and applications in chemical analysis, biology and medicine. Chemical Society Reviews, 45(21), 6048-6077.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0958166907000043&quot; rel=&quot;nofollow noreferrer&quot;&gt;[4] Rao, J., Dragulescu-Andrasi, A., &amp;amp; Yao, H. (2007). Fluorescence imaging in vivo: recent advances. Current opinion in biotechnology, 18(1), 17-25.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-12-05T04:24:08.397" LastActivityDate="2018-12-05T04:24:08.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5584" PostTypeId="2" ParentId="5575" CreationDate="2018-12-05T09:50:57.587" Score="2" Body="&lt;p&gt;I think you could use the &lt;a href=&quot;https://doi.org/doi:10.18129/B9.bioc.MeSH.db&quot; rel=&quot;nofollow noreferrer&quot;&gt;MESH.db&lt;/a&gt; package in Bioconductor to annotate your genes. I have never used, but you should be able to do something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(&quot;MeSH.db&quot;)&#xA;select(MeSH.db, keys = myGenes, keytype = &quot;ENTREZ&quot;, columns = &quot;MESH&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It depends on the &lt;a href=&quot;https://doi.org/doi:10.18129/B9.bioc.MeSHDbi&quot; rel=&quot;nofollow noreferrer&quot;&gt;MeSHDbi&lt;/a&gt; package from which you could retrieve other data form that database if needed&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2018-12-05T09:50:57.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5585" PostTypeId="1" CreationDate="2018-12-05T09:54:04.577" Score="1" ViewCount="540" Body="&lt;ol&gt;&#xA;&lt;li&gt;I am working with RNA-seq data without a reference genome/transcriptome and am instead using a &lt;strong&gt;Trinity &lt;em&gt;de novo&lt;/em&gt; transcriptome assembly&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;I analyzed both isoform and gene expression abundances using RSEM.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;And got differentially expressed GENES via DESeq2.&#xA;In the next step I want to get fasta sequences of differentially expressed genes to use them in the subsequent BlastX/Blast2Go analyses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Considering the fact that my Trinity created assembly is based on isoform id:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I extract the gene sequence out of it? Is that a right thing to combine all the isoforms of a single gene in order to generate the longest possible isoform assembly? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If so, how can I do it to have a union-exon (combined form of all the exons present in the isoforms of an individual gene) sequence used in my Blast analysis? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, how should I proceed with the data type available?&#xA;Trinity assembled file includes the isoform column as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TRINITY_DN1780_c0_g1_i2 &#xA;TRINITY_DN1780_c0_g1_i1 &#xA;TRINITY_DN1780_c0_g1_i3 &#xA;TRINITY_DN1780_c0_g1_i4 &#xA;TRINITY_DN1780_c0_g1_i5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the DESeq2 produces the output (gene differentials) as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TRINITY_DN1780_c1_g1&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3793" LastEditorUserId="2085" LastEditDate="2018-12-05T14:20:24.143" LastActivityDate="2018-12-09T11:19:35.967" Title="How to get the longest fasta sequence including all possible switching isoforms of a gene out of isoforms" Tags="|rna-seq|blast|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5586" PostTypeId="1" AcceptedAnswerId="5590" CreationDate="2018-12-05T12:05:32.630" Score="1" ViewCount="131" Body="&lt;p&gt;According to the 2017(!) textbook, it worked at that time, for instance, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=snp&amp;amp;term=autism&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this time, this does not work and for each conceivable term it will find 0 records. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the right way to search in &lt;code&gt;snp&lt;/code&gt;, valid in December 2018?&lt;/p&gt;&#xA;" OwnerUserId="956" LastActivityDate="2018-12-05T16:22:36.673" Title="ncbi...eutils/esearch.fcgi: How to search now in the snp database?" Tags="|snp|public-databases|phylogenetics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5587" PostTypeId="2" ParentId="5585" CreationDate="2018-12-05T13:13:28.060" Score="2" Body="&lt;p&gt;From your question it isn't clear if you are trying to identify the isoforms that have a very high or very low differential expression value, or if you are trying to correctly find an orthlog for your top hits using BLAST. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your question is about isoform-level quantification, &lt;a href=&quot;https://support.bioconductor.org/p/43395/#43400&quot; rel=&quot;nofollow noreferrer&quot;&gt;identifying DE at the isoform level is challenging and DESeq2 is only designed for gene-level analyses.&lt;/a&gt; That is why it is only outputting genes rather than a specific isoform.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-12-05T13:13:28.060" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5588" PostTypeId="2" ParentId="5585" CreationDate="2018-12-05T14:18:42.800" Score="2" Body="&lt;p&gt;There's no clear way to go from a trinity &quot;gene&quot; (this is just a cluster of isoforms with shared sequence) to a linear gene sequence that will be blastable and not cause a number of headaches (e.g., due to swapping the order of exons).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would suggest that you do is either Blast2GO the isoforms within the Trinity-defined &quot;genes&quot; or take the highest expressed isoform within a &quot;gene&quot; as the sequence you feed into Blast2GO for that gene. Alternatively just use isoforms for everything and avoid this whole issue.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2018-12-05T14:18:42.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5590" PostTypeId="2" ParentId="5586" CreationDate="2018-12-05T16:22:36.673" Score="3" Body="&lt;p&gt;eUtilities for dbSNP is in the process of being retired. See &lt;a href=&quot;https://ncbiinsights.ncbi.nlm.nih.gov/2018/06/15/dbsnp-updates-json-refsnp-report-api/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt; from NCBI. You may want to check out the &lt;a href=&quot;https://api.ncbi.nlm.nih.gov/variation/v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;new API&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;From the NCBI Insights blog post:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Entrez search is available &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/mailman/pipermail/dbsnp-announce/2018q1/000184.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;only for human&lt;/a&gt;. We will retire all Eutils report formats (RS docsum XML and ASN.1, FASTA, and FLAT text) later this year. Only eSummary XML report will be supported.&lt;/p&gt;&#xA;&lt;p&gt;Batch Query Service for all organisms will be retired October 2018. Users of human dbSNP data should transition to the new API.&lt;/p&gt;&#xA;&lt;p&gt;We will place non-human FTP data in the &lt;a href=&quot;ftp://ftp.ncbi.nih.gov/snp/archive.&quot; rel=&quot;nofollow noreferrer&quot;&gt;archive directory&lt;/a&gt;. Please see the phase-out plan &lt;a href=&quot;https://ncbiinsights.ncbi.nlm.nih.gov/2017/05/09/phasing-out-support-for-non-human-genome-organism-data-in-dbsnp-and-dbvar/&quot; rel=&quot;nofollow noreferrer&quot;&gt;announced last year&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3577" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-12-05T16:22:36.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5591" PostTypeId="1" AcceptedAnswerId="5595" CreationDate="2018-12-05T17:55:44.987" Score="4" ViewCount="1106" Body="&lt;p&gt;I used this previously to get ensembl ID and its gene name from the gencode gtf annotation file &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This was the &lt;a href=&quot;https://www.biostars.org/p/122012/&quot; rel=&quot;nofollow noreferrer&quot;&gt;script&lt;/a&gt; I used &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{                                &#xA;    for (i = 1; i &amp;lt;= NF; i++) {&#xA;        if (&lt;span class=&quot;math-container&quot;&gt;$i ~ /gene_id|gene_name/) {&#xA;        printf &quot;%s &quot;, $&lt;/span&gt;(i+1)&#xA;        }&#xA;    }&#xA;    print &quot;&quot;&#xA;}' Homo_sapiens.GRCh37.70.gtf | sed -e 's/&quot;//g' -e 's/;//g' -e 's/ /\t/' | sort -k1,1 | uniq &amp;gt; Homo_sapiens.GRCh37.70.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the input file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##description: evidence-based annotation of the human genome (GRCh38), version 21 (Ensembl 77)&#xA;##provider: GENCODE&#xA;##contact: gencode@sanger.ac.uk&#xA;##format: gtf&#xA;##date: 2014-09-29&#xA;chr1    HAVANA  gene    11869   14409   .   +   .   gene_id &quot;ENSG00000223972.5&quot;; transcript_id &quot;ENSG00000223972.5&quot;; gene_type &quot;transcribed_unprocessed_pseudogene&quot;; gene_status &quot;KNOWN&quot;; gene_name &quot;DDX11L1&quot;; transcript_type &quot;transcribed_unprocessed_pseudogene&quot;; transcript_status &quot;KNOWN&quot;; transcript_name &quot;DDX11L1&quot;; level 2; havana_gene &quot;OTTHUMG00000000961.2&quot;;&#xA;chr1    HAVANA  transcript  11869   14409   .   +   .   gene_id &quot;ENSG00000223972.5&quot;; transcript_id &quot;ENST00000456328.2&quot;; gene_type &quot;transcribed_unprocessed_pseudogene&quot;; gene_status &quot;KNOWN&quot;; gene_name &quot;DDX11L1&quot;; transcript_type &quot;processed_transcript&quot;; transcript_status &quot;KNOWN&quot;; transcript_name &quot;DDX11L1-002&quot;; level 2; tag &quot;basic&quot;; transcript_support_level &quot;1&quot;; havana_gene &quot;OTTHUMG00000000961.2&quot;; havana_transcript &quot;OTTHUMT00000362751.1&quot;;&#xA;chr1    HAVANA  exon    11869   12227   .   +   .   gene_id &quot;ENSG00000223972.5&quot;; transcript_id &quot;ENST00000456328.2&quot;; gene_type &quot;transcribed_unprocessed_pseudogene&quot;; gene_status &quot;KNOWN&quot;; gene_name &quot;DDX11L1&quot;; transcript_type &quot;processed_transcript&quot;; transcript_status &quot;KNOWN&quot;; transcript_name &quot;DDX11L1-002&quot;; exon_number 1; exon_id &quot;ENSE00002234944.1&quot;; level 2; tag &quot;basic&quot;; transcript_support_level &quot;1&quot;; havana_gene &quot;OTTHUMG00000000961.2&quot;; havana_transcript &quot;OTTHUMT00000362751.1&quot;;&#xA;chr1    HAVANA  exon    12613   12721   .   +   .   gene_id &quot;ENSG00000223972.5&quot;; transcript_id &quot;ENST00000456328.2&quot;; gene_type &quot;transcribed_unprocessed_pseudogene&quot;; gene_status &quot;KNOWN&quot;; gene_name &quot;DDX11L1&quot;; transcript_type &quot;processed_transcript&quot;; transcript_status &quot;KNOWN&quot;; transcript_name &quot;DDX11L1-002&quot;; exon_number 2; exon_id &quot;ENSE00003582793.1&quot;; level 2; tag &quot;basic&quot;; transcript_support_level &quot;1&quot;; havana_gene &quot;OTTHUMG00000000961.2&quot;; havana_transcript &quot;OTTHUMT00000362751.1&quot;;&#xA;chr1    HAVANA  exon    13221   14409   .   +   .   gene_id &quot;ENSG00000223972.5&quot;; transcript_id &quot;ENST00000456328.2&quot;; gene_type &quot;transcribed_unprocessed_pseudogene&quot;; gene_status &quot;KNOWN&quot;; gene_name &quot;DDX11L1&quot;; transcript_type &quot;processed_transcript&quot;; transcript_status &quot;KNOWN&quot;; transcript_name &quot;DDX11L1-002&quot;; exon_number 3; exon_id &quot;ENSE00002312635.1&quot;; level 2; tag &quot;basic&quot;; transcript_support_level &quot;1&quot;; havana_gene &quot;OTTHUMG00000000961.2&quot;; havana_transcript &quot;OTTHUMT00000362751.1&quot;;&#xA;chr1    HAVANA  transcript  12010   13670   .   +   .   gene_id &quot;ENSG00000223972.5&quot;; transcript_id &quot;ENST00000450305.2&quot;; gene_type &quot;transcribed_unprocessed_pseudogene&quot;; gene_status &quot;KNOWN&quot;; gene_name &quot;DDX11L1&quot;; transcript_type &quot;transcribed_unprocessed_pseudogene&quot;; transcript_status &quot;KNOWN&quot;; transcript_name &quot;DDX11L1-001&quot;; level 2; ont &quot;PGO:0000005&quot;; ont &quot;PGO:0000019&quot;; tag &quot;basic&quot;; transcript_support_level &quot;NA&quot;; havana_gene &quot;OTTHUMG00000000961.2&quot;; havana_transcript &quot;OTTHUMT00000002844.2&quot;;&#xA;chr1    HAVANA  exon    12010   12057   .   +   .   gene_id &quot;ENSG00000223972.5&quot;; transcript_id &quot;ENST00000450305.2&quot;; gene_type &quot;transcribed_unprocessed_pseudogene&quot;; gene_status &quot;KNOWN&quot;; gene_name &quot;DDX11L1&quot;; transcript_type &quot;transcribed_unprocessed_pseudogene&quot;; transcript_status &quot;KNOWN&quot;; transcript_name &quot;DDX11L1-001&quot;; exon_number 1; exon_id &quot;ENSE00001948541.1&quot;; level 2; ont &quot;PGO:0000005&quot;; ont &quot;PGO:0000019&quot;; tag &quot;basic&quot;; transcript_support_level &quot;NA&quot;; havana_gene &quot;OTTHUMG00000000961.2&quot;; havana_transcript &quot;OTTHUMT00000002844.2&quot;;&#xA;chr1    HAVANA  exon    12179   12227   .   +   .   gene_id &quot;ENSG00000223972.5&quot;; transcript_id &quot;ENST00000450305.2&quot;; gene_type &quot;transcribed_unprocessed_pseudogene&quot;; gene_status &quot;KNOWN&quot;; gene_name &quot;DDX11L1&quot;; transcript_type &quot;transcribed_unprocessed_pseudogene&quot;; transcript_status &quot;KNOWN&quot;; transcript_name &quot;DDX11L1-001&quot;; exon_number 2; exon_id &quot;ENSE00001671638.2&quot;; level 2; ont &quot;PGO:0000005&quot;; ont &quot;PGO:0000019&quot;; tag &quot;basic&quot;; transcript_support_level &quot;NA&quot;; havana_gene &quot;OTTHUMG00000000961.2&quot;; havana_transcript &quot;OTTHUMT00000002844.2&quot;;&#xA;chr1    HAVANA  exon    12613   12697   .   +   .   gene_id &quot;ENSG00000223972.5&quot;; transcript_id &quot;ENST00000450305.2&quot;; gene_type &quot;transcribed_unprocessed_pseudogene&quot;; gene_status &quot;KNOWN&quot;; gene_name &quot;DDX11L1&quot;; transcript_type &quot;transcribed_unprocessed_pseudogene&quot;; transcript_status &quot;KNOWN&quot;; transcript_name &quot;DDX11L1-001&quot;; exon_number 3; exon_id &quot;ENSE00001758273.2&quot;; level 2; ont &quot;PGO:0000005&quot;; ont &quot;PGO:0000019&quot;; tag &quot;basic&quot;; transcript_support_level &quot;NA&quot;; havana_gene &quot;OTTHUMG00000000961.2&quot;; havana_transcript &quot;OTTHUMT00000002844.2&quot;;&#xA;chr1    HAVANA  exon    12975   13052   .   +   .   gene_id &quot;ENSG00000223972.5&quot;; transcript_id &quot;ENST00000450305.2&quot;; gene_type &quot;transcribed_unprocessed_pseudogene&quot;; gene_status &quot;KNOWN&quot;; gene_name &quot;DDX11L1&quot;; transcript_type &quot;transcribed_unprocessed_pseudogene&quot;; transcript_status &quot;KNOWN&quot;; transcript_name &quot;DDX11L1-001&quot;; exon_number 4; exon_id &quot;ENSE00001799933.2&quot;; level 2; ont &quot;PGO:0000005&quot;; ont &quot;PGO:0000019&quot;; tag &quot;basic&quot;; transcript_support_level &quot;NA&quot;; havana_gene &quot;OTTHUMG00000000961.2&quot;; havana_transcript &quot;OTTHUMT00000002844.2&quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What I'm trying to do now is replace the gene_ID with transcript_ID and the gene_name with transcript_name but I'm doing something wrong as I'm not getting the transcript_ID but the gene Ensembl ID instead. I'm not sure if I'm choosing the right column for parsing the transcript_ID and name.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm expecting to get this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;transcript_id &quot;ENST00000450305.2&quot;;  gene_name &quot;DDX11L1&quot;;transcript_name &quot;DDX11L1-001&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any help or suggestion would be highly appreciated &lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="298" LastEditDate="2018-12-06T12:45:18.363" LastActivityDate="2018-12-06T20:23:02.830" Title="Parsing gtf file for transcript ID and transcript name" Tags="|gtf|text-processing|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5592" PostTypeId="1" AcceptedAnswerId="5594" CreationDate="2018-12-06T02:18:13.990" Score="5" ViewCount="1847" Body="&lt;p&gt;I would like to ask why when we initialize 0 as the starting point, there are 3 gray boxes that are not used as shown (highlighted in the red box) in the pic?&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ZyvEM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZyvEM.png&quot; alt=&quot;pic of matrix&quot;&gt;&lt;/a&gt;&#xA; Are these boxes to show that they are gaps or just to align them properly?&lt;/p&gt;&#xA;" OwnerUserId="3797" LastEditorUserId="964" LastEditDate="2018-12-06T14:45:49.210" LastActivityDate="2018-12-07T14:49:49.640" Title="Understanding the initialization of the Needleman-Wunsch algorithm" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5593" PostTypeId="1" CreationDate="2018-12-06T05:18:23.897" Score="2" ViewCount="633" Body="&lt;p&gt;I did all of the installations from scratch in a virtual &lt;code&gt;conda&lt;/code&gt; environment. Then I tried to run &lt;code&gt;R&lt;/code&gt; script from within &lt;code&gt;javascript&lt;/code&gt; in the following way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;let shCMD = 'Rscript --vanilla R/full_analysis.R ';&#xA;  shCMD += '--diff_1 ' + diff_str_1 + '--diff_2 ' + diff_str_2;&#xA;  shCMD += '--markers ' + filterStr + '--markers-highlighted ' + colorStr;&#xA;  shCMD += '--datasets ' + datasetStr;&#xA;&#xA;  console.log('constructed command for full analysis: ');&#xA;  console.log(shCMD);&#xA;  shell.exec('source activate rmain', function(env_code, env_stdout, &#xA;env_stderr) {&#xA;    console.log('activated rmain environment')&#xA;    if (env_code === 0) {&#xA;      shell.exec(shCMD, function(code, stdout, stderr) {&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As can be seen I am activating the environment by &lt;code&gt;source activate rmain&lt;/code&gt; and I am pretty sure that the next shell command that invokes &lt;code&gt;R&lt;/code&gt; script runs in that environment because &lt;code&gt;R&lt;/code&gt; script needs &lt;code&gt;argparse&lt;/code&gt; package and it is installed only in the &lt;code&gt;rmain&lt;/code&gt; virtual environment: in &lt;code&gt;rmain&lt;/code&gt; &lt;code&gt;argparse&lt;/code&gt; package is successfully imported, but outside of it it gives an error for the &lt;code&gt;argparse&lt;/code&gt; absence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, now the following line in the &lt;code&gt;R&lt;/code&gt; script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sce &amp;lt;- SingleCellExperiment(list(counts=as.matrix(data[,-1])))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gives me the error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in `rownames&amp;lt;-`(`*tmp*`, value = c(&quot;0610007P14Rik&quot;, &quot;0610009B22Rik&quot;,  : &#xA;  invalid rownames length&#xA;Calls: SingleCellExperiment ... elementMetadata -&amp;gt; .local -&amp;gt; rownames&amp;lt;- -&amp;gt; rownames&amp;lt;-&#xA;Execution halted&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, if I launch &lt;code&gt;Rstudio&lt;/code&gt; from within the &lt;code&gt;rmain&lt;/code&gt; environment, the line runs successfully... So, I am puzzled now on what is going on. &lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="298" LastEditDate="2019-03-07T19:00:33.310" LastActivityDate="2019-03-07T19:00:33.310" Title="SingleCellExperiment: invalid rownames length" Tags="|r|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5594" PostTypeId="2" ParentId="5592" CreationDate="2018-12-06T11:48:35.450" Score="7" Body="&lt;p&gt;The first column is added in order to be able to align the sequences. They might be gaps if the alignment ends up beginning in different positions than the first. Without the first empty row and column it wouldn't be possible to have gaps at the beginning of the alignment&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Dynamic_programming&quot; rel=&quot;nofollow noreferrer&quot;&gt;dynamic programming&lt;/a&gt; is useful because you need first to calculate the score for each position, and then once you have the score, you can calculate the optimum path. So you solve your problem by first calculating what would happen if (there was a gap, there was a mismatch...) and then you find the optimum alignment. &lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2018-12-07T14:49:49.640" LastActivityDate="2018-12-07T14:49:49.640" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5595" PostTypeId="2" ParentId="5591" CreationDate="2018-12-06T12:59:50.387" Score="3" Body="&lt;p&gt;I don't see how your &lt;code&gt;awk&lt;/code&gt; command would work since you're using whitespace as a field delimiter. In any case, you can use a much shorter perl command for this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ perl -lne 'print &quot;@m&quot; if @m=(/((?:transcript_id|gene_name|transcript_name)\s+\S+)/g);' Homo_sapiens.GRCh37.70.gtf &#xA;transcript_id &quot;ENSG00000223972.5&quot;; gene_name &quot;DDX11L1&quot;; transcript_name &quot;DDX11L1&quot;;&#xA;transcript_id &quot;ENST00000456328.2&quot;; gene_name &quot;DDX11L1&quot;; transcript_name &quot;DDX11L1-002&quot;;&#xA;transcript_id &quot;ENST00000456328.2&quot;; gene_name &quot;DDX11L1&quot;; transcript_name &quot;DDX11L1-002&quot;;&#xA;transcript_id &quot;ENST00000456328.2&quot;; gene_name &quot;DDX11L1&quot;; transcript_name &quot;DDX11L1-002&quot;;&#xA;transcript_id &quot;ENST00000456328.2&quot;; gene_name &quot;DDX11L1&quot;; transcript_name &quot;DDX11L1-002&quot;;&#xA;transcript_id &quot;ENST00000450305.2&quot;; gene_name &quot;DDX11L1&quot;; transcript_name &quot;DDX11L1-001&quot;;&#xA;transcript_id &quot;ENST00000450305.2&quot;; gene_name &quot;DDX11L1&quot;; transcript_name &quot;DDX11L1-001&quot;;&#xA;transcript_id &quot;ENST00000450305.2&quot;; gene_name &quot;DDX11L1&quot;; transcript_name &quot;DDX11L1-001&quot;;&#xA;transcript_id &quot;ENST00000450305.2&quot;; gene_name &quot;DDX11L1&quot;; transcript_name &quot;DDX11L1-001&quot;;&#xA;transcript_id &quot;ENST00000450305.2&quot;; gene_name &quot;DDX11L1&quot;; transcript_name &quot;DDX11L1-001&quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The scriptlet will print the contents of the &lt;code&gt;@m&lt;/code&gt; array if this line matches what you need. The regex magic is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;(?: )&lt;/code&gt;: this is a non-capturing group. We need this so that &lt;code&gt;@m&lt;/code&gt; only holds what is inside the &lt;em&gt;capturing&lt;/em&gt; parentheses (see below).&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;(  (?:transcript_id|gene_name|transcript_name)\s+\S+)&lt;/code&gt; : the outer parentheses will capture everything matched: one of &lt;code&gt;transcript_id|gene_name|transcript_name&lt;/code&gt; followed by 1 or more whitespace characters (&lt;code&gt;\s+&lt;/code&gt;) and one or more non-whitespace characters.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;print @m if @m=...&lt;/code&gt; : only print if the match was successful.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2018-12-06T20:23:02.830" LastActivityDate="2018-12-06T20:23:02.830" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5596" PostTypeId="2" ParentId="4858" CreationDate="2018-12-06T17:28:37.150" Score="2" Body="&lt;p&gt;For simple haplotype simulation of unrelated individuals, there is Montana's venerable &lt;a href=&quot;https://cran.r-project.org/web/packages/hapsim/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;HapSim&lt;/a&gt;. The model used in that tool is simple and may be good enough for your purposes. You will need to acquire a recombination rate map from for example the HapMap consortium or from &lt;a href=&quot;https://github.com/joepickrell/1000-genomes-genetic-maps&quot; rel=&quot;nofollow noreferrer&quot;&gt;1000G&lt;/a&gt;. A cursory google search also revealed this tool and walkthrough that may be suited to your problem: &lt;a href=&quot;https://adimitromanolakis.github.io/sim1000G/inst/doc/SimulatingFamilyData.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://adimitromanolakis.github.io/sim1000G/inst/doc/SimulatingFamilyData.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are lots of other tools with more advanced models, but it sounds like those would be overkill here.&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2018-12-06T17:28:37.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5597" PostTypeId="2" ParentId="5585" CreationDate="2018-12-06T17:42:07.477" Score="0" Body="&lt;p&gt;There are a lot of questions in this post, and the main ones have been answered.  Here is an answer to the one that has not been answered yet:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How can I do it to have a union-exon (combined form of all the exons present in the isoforms of an individual gene) sequence used in my Blast analysis?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think you should do this, but if you want one of the usually many plausible linearized representations of transcript isoforms, then &lt;a href=&quot;https://github.com/Oshlack/Lace&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lace&lt;/a&gt; would do the trick.&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2018-12-06T17:42:07.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5598" PostTypeId="2" ParentId="5585" CreationDate="2018-12-06T17:46:20.110" Score="0" Body="&lt;p&gt;It appears from the comment in another question that this question is also about transcript-level differential expression analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If instead of using RSEM you use &lt;a href=&quot;https://pachterlab.github.io/kallisto/about&quot; rel=&quot;nofollow noreferrer&quot;&gt;kallisto&lt;/a&gt; for transcript quantification, then you could use &lt;a href=&quot;https://pachterlab.github.io/sleuth/about&quot; rel=&quot;nofollow noreferrer&quot;&gt;sleuth&lt;/a&gt; to perform transcript-level differential expression analysis.&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2018-12-06T17:46:20.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5599" PostTypeId="2" ParentId="4080" CreationDate="2018-12-06T19:25:05.160" Score="0" Body="&lt;p&gt;Easy to visualize volcano plot using Python. Check &lt;a href=&quot;https://reneshbedre.github.io/blog/volcano.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt; for inverted volcano plot&lt;/p&gt;&#xA;" OwnerUserId="3802" LastEditorUserId="57" LastEditDate="2018-12-07T15:13:09.043" LastActivityDate="2018-12-07T15:13:09.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5600" PostTypeId="1" AcceptedAnswerId="5601" CreationDate="2018-12-06T20:01:39.473" Score="4" ViewCount="5998" Body="&lt;p&gt;I have a long list of read IDs of interest to me in a file called read_names.txt. it is simply in the format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;m54197_180831_211346/4981510/ccs&#xA;m54197_180831_211346/6226723/ccs&#xA;m54197_180831_211346/6619607/ccs&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;etc where these are the actual read names from a fastq file. I am then trying to find those in a bam file for which the fastq has been mapped, and I can accomplish this like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for ID in `cat read_names.txt`&#xA;do&#xA;samtools view inbam.bam | grep $ID &amp;gt;&amp;gt; read_locs.sam&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, this method is obscenely slow because it is rerunning samtools view for every ID iteration (several hours now for 600 read IDs), and I was hoping to do this for several &lt;code&gt;read_names.txt&lt;/code&gt; files. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried sort of flipping the script a bit and running samtools view first but it only returned the first read ID present in the file and stopped:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view inbam.bam | for ID in `cat read_names.txt`; do grep $ID &amp;gt;&amp;gt; read_locs.txt; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Am I missing something or is this the only way to accomplish this task?&lt;/p&gt;&#xA;" OwnerUserId="3669" LastActivityDate="2023-03-03T14:22:47.180" Title="Extracting all reads from bam file which match read IDs in another file" Tags="|sam|samtools|reads|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5601" PostTypeId="2" ParentId="5600" CreationDate="2018-12-06T20:04:28.987" Score="9" Body="&lt;p&gt;It is still slow but grep has a &lt;code&gt;-f&lt;/code&gt; option to take in a file&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;samtools view inbam.bam | grep -f read_names.txt &amp;gt; read_locs.txt&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2018-12-06T20:04:28.987" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5604" PostTypeId="1" CreationDate="2018-12-06T22:23:20.617" Score="5" ViewCount="680" Body="&lt;p&gt;I compiled MrBayes v3.2.7 from github, with the configuration options &lt;code&gt;./configure --without-beagle --enable-mpi&lt;/code&gt;. The program is called mb.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried one of the included example files &quot;sceloporus.nex&quot;. I find that if I run &lt;code&gt;mb sceloporus.nex&lt;/code&gt;, then it appears to make progress, and writes files to disk. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used MPI with 1 core, and it behaves as before, making progress, writing files to disk.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if I run with more than 1 core, using MPI, with &lt;code&gt;mpirun -np 2 mb sceloporus.nex&lt;/code&gt;, then it only makes it through some of the analysis setup, doesn't write any files to disk, and just hangs. If I look at &lt;code&gt;top&lt;/code&gt;, I see activity on the number of cores I specified. I finally had to manually terminate the program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there something I should be doing differently to have MrBayes properly run in parallel?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a copy of my terminal from these 3 attempts:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[ptitle@iu mbTest]&lt;span class=&quot;math-container&quot;&gt;$ ls&#xA;sceloporus.nex&#xA;[ptitle@iu mbTest]$&lt;/span&gt; ~/MrBayes/src/mb sceloporus.nex &#xA;&#xA;&#xA;                            MrBayes pre-3.2.7 x86_64&#xA;&#xA;                      (Bayesian Analysis of Phylogeny)&#xA;&#xA;                             (Parallel version)&#xA;                         (1 processors available)&#xA;&#xA;              Distributed under the GNU General Public License&#xA;&#xA;&#xA;               Type &quot;help&quot; or &quot;help &amp;lt;command&amp;gt;&quot; for information&#xA;                     on the commands that are available.&#xA;&#xA;                   Type &quot;about&quot; for authorship and general&#xA;                       information about the program.&#xA;&#xA;&#xA;&#xA;   Executing file &quot;sceloporus.nex&quot;&#xA;   DOS line termination&#xA;   Longest line length = 1632&#xA;   Parsing file&#xA;   Expecting NEXUS formatted file&#xA;   Reading data block&#xA;      Allocated taxon set&#xA;      Allocated matrix&#xA;      Defining new matrix with 123 taxa and 1606 characters&#xA;      Data is Dna&#xA;      Missing data coded as ?&#xA;      Gaps coded as -&#xA;      Taxon   1 -&amp;gt; AZYuJAS289&#xA;      Taxon   2 -&amp;gt; AZYuJAS290&#xA;      Taxon   3 -&amp;gt; AZlaJAS294&#xA;      Taxon   4 -&amp;gt; AZcoTBP271&#xA;      Taxon   5 -&amp;gt; AZcoGM1054&#xA;      Taxon   6 -&amp;gt; AZcoDGM994&#xA;      Taxon   7 -&amp;gt; AZgiP26438&#xA;      Taxon   8 -&amp;gt; AZgrGM1012&#xA;      Taxon   9 -&amp;gt; AZgrGM1013&#xA;      Taxon  10 -&amp;gt; AZgrGM1024&#xA;      Taxon  11 -&amp;gt; AZlaDGM856&#xA;      Taxon  12 -&amp;gt; AZmaDGM984&#xA;      Taxon  13 -&amp;gt; AZmaDGM992&#xA;      Taxon  14 -&amp;gt; AZmaRM4401&#xA;      Taxon  15 -&amp;gt; AZmaRM4402&#xA;      Taxon  16 -&amp;gt; AZmaRM4403&#xA;      Taxon  17 -&amp;gt; AZmoGM1045&#xA;      Taxon  18 -&amp;gt; AZmoDGM492&#xA;      Taxon  19 -&amp;gt; AZmoDGM699&#xA;      Taxon  20 -&amp;gt; AZmoDGM704&#xA;      Taxon  21 -&amp;gt; AZmoRM4394&#xA;      Taxon  22 -&amp;gt; AZpiKWS238&#xA;      Taxon  23 -&amp;gt; AZyuDGM826&#xA;      Taxon  24 -&amp;gt; AZyuDGM838&#xA;      Taxon  25 -&amp;gt; AZyuDGM852&#xA;      Taxon  26 -&amp;gt; AZyuDGM858&#xA;      Taxon  27 -&amp;gt; AZyuDGM865&#xA;      Taxon  28 -&amp;gt; CAfrP25750&#xA;      Taxon  29 -&amp;gt; CAimDGM534&#xA;      Taxon  30 -&amp;gt; CAimDGM558&#xA;      Taxon  31 -&amp;gt; CAimDGM694&#xA;      Taxon  32 -&amp;gt; CAimKWS170&#xA;      Taxon  33 -&amp;gt; CAinCRF530&#xA;      Taxon  34 -&amp;gt; CAinoDGM74&#xA;      Taxon  35 -&amp;gt; CAinCRF529&#xA;      Taxon  36 -&amp;gt; CAkeDS2SP7&#xA;      Taxon  37 -&amp;gt; CAkeP26556&#xA;      Taxon  38 -&amp;gt; CAla78323&#xA;      Taxon  39 -&amp;gt; CAlaDAW220&#xA;      Taxon  40 -&amp;gt; CAlaM14610&#xA;      Taxon  41 -&amp;gt; CAlaM23289&#xA;      Taxon  42 -&amp;gt; CAmoJQR132&#xA;      Taxon  43 -&amp;gt; CArvJOS138&#xA;      Taxon  44 -&amp;gt; CArvESA441&#xA;      Taxon  45 -&amp;gt; CAsaBUR167&#xA;      Taxon  46 -&amp;gt; CAsaMCFMAG&#xA;      Taxon  47 -&amp;gt; CAsarnMCC&#xA;      Taxon  48 -&amp;gt; CAsa150079&#xA;      Taxon  49 -&amp;gt; CAsa150088&#xA;      Taxon  50 -&amp;gt; CAsa179886&#xA;      Taxon  51 -&amp;gt; CAsaM19838&#xA;      Taxon  52 -&amp;gt; CAsdDGM691&#xA;      Taxon  53 -&amp;gt; CAsdDGM748&#xA;      Taxon  54 -&amp;gt; CAsdSWT701&#xA;      Taxon  55 -&amp;gt; MXBCDGM474&#xA;      Taxon  56 -&amp;gt; MXBCDGM475&#xA;      Taxon  57 -&amp;gt; MXBCDGM486&#xA;      Taxon  58 -&amp;gt; MXchRM4860&#xA;      Taxon  59 -&amp;gt; MXduA83134&#xA;      Taxon  60 -&amp;gt; MXsoM14488&#xA;      Taxon  61 -&amp;gt; MXsoP26449&#xA;      Taxon  62 -&amp;gt; MXso210345&#xA;      Taxon  63 -&amp;gt; NMdoTWR230&#xA;      Taxon  64 -&amp;gt; NMedDGM938&#xA;      Taxon  65 -&amp;gt; NMhiU48819&#xA;      Taxon  66 -&amp;gt; NMluGM947&#xA;      Taxon  67 -&amp;gt; NMluDGM948&#xA;      Taxon  68 -&amp;gt; NMsiDGM891&#xA;      Taxon  69 -&amp;gt; NMsoDGM900&#xA;      Taxon  70 -&amp;gt; NMsoDGM904&#xA;      Taxon  71 -&amp;gt; NMsoDGM905&#xA;      Taxon  72 -&amp;gt; NMvaDGM924&#xA;      Taxon  73 -&amp;gt; NVchDGM612&#xA;      Taxon  74 -&amp;gt; NVch162077&#xA;      Taxon  75 -&amp;gt; NVclDGM296&#xA;      Taxon  76 -&amp;gt; NVclDGM678&#xA;      Taxon  77 -&amp;gt; NVesDGM210&#xA;      Taxon  78 -&amp;gt; NVlyDGM610&#xA;      Taxon  79 -&amp;gt; NVwaDGM608&#xA;      Taxon  80 -&amp;gt; UTemGM1058&#xA;      Taxon  81 -&amp;gt; UTgrDGM156&#xA;      Taxon  82 -&amp;gt; UTgrDGM169&#xA;      Taxon  83 -&amp;gt; UTgrDGM175&#xA;      Taxon  84 -&amp;gt; UTkaGM1034&#xA;      Taxon  85 -&amp;gt; UTsnDM142&#xA;      Taxon  86 -&amp;gt; UTsnDG043&#xA;      Taxon  87 -&amp;gt; UTsnnDGM645&#xA;      Taxon  88 -&amp;gt; UTwsGM261&#xA;      Taxon  89 -&amp;gt; UTwsGM263&#xA;      Taxon  90 -&amp;gt; UTwsnDGM751&#xA;      Taxon  91 -&amp;gt; UTwsJRM4437&#xA;      Taxon  92 -&amp;gt; hunsiM376&#xA;      Taxon  93 -&amp;gt; hunsiM377&#xA;      Taxon  94 -&amp;gt; hunsiRWM619&#xA;      Taxon  95 -&amp;gt; lickGM378&#xA;      Taxon  96 -&amp;gt; lineatulus&#xA;      Taxon  97 -&amp;gt; orcuOM14377&#xA;      Taxon  98 -&amp;gt; orcuOM14453&#xA;      Taxon  99 -&amp;gt; orcuOM14673&#xA;      Taxon 100 -&amp;gt; orcuRWM798&#xA;      Taxon 101 -&amp;gt; orcuS201108&#xA;      Taxon 102 -&amp;gt; orcuS201124&#xA;      Taxon 103 -&amp;gt; zostZ16290&#xA;      Taxon 104 -&amp;gt; zosV161292&#xA;      Taxon 105 -&amp;gt; zosV161293&#xA;      Taxon 106 -&amp;gt; zosV161311&#xA;      Taxon 107 -&amp;gt; zosWM1890&#xA;      Taxon 108 -&amp;gt; zosGM364&#xA;      Taxon 109 -&amp;gt; zosGM365&#xA;      Taxon 110 -&amp;gt; zosGM387&#xA;      Taxon 111 -&amp;gt; zosGM393&#xA;      Taxon 112 -&amp;gt; zosRO332&#xA;      Taxon 113 -&amp;gt; zosOM37006&#xA;      Taxon 114 -&amp;gt; zosWM1601&#xA;      Taxon 115 -&amp;gt; clarkii&#xA;      Taxon 116 -&amp;gt; edwardti80&#xA;      Taxon 117 -&amp;gt; smaragdnus&#xA;      Taxon 118 -&amp;gt; graciosus&#xA;      Taxon 119 -&amp;gt; grammicus&#xA;      Taxon 120 -&amp;gt; maculosus&#xA;      Taxon 121 -&amp;gt; bicanthlis&#xA;      Taxon 122 -&amp;gt; torquatus&#xA;      Taxon 123 -&amp;gt; variabilis&#xA;      Successfully read matrix&#xA;      Setting default partition (does not divide up characters)&#xA;      Setting model defaults&#xA;      Seed (for generating default start values) = 1544128156&#xA;      Setting output file names to &quot;sceloporus.nex.run&amp;lt;i&amp;gt;.&amp;lt;p|t&amp;gt;&quot;&#xA;   Exiting data block&#xA;   Reading mrbayes block&#xA;      Excluding character(s)&#xA;      Setting Nst to 6&#xA;      Setting Rates to Gamma&#xA;      Successfully set likelihood model parameters&#xA;      Setting Brlenspr to Unconstrained:GammaDir(1.00,1.0000,0.70,0.50)&#xA;      Successfully set prior model parameters&#xA;      Setting number of generations to 5000000&#xA;      Running Markov chain&#xA;      MCMC stamp = 9284864458&#xA;      Seed = 753696747&#xA;      Swapseed = 1544128156&#xA;      Model settings:&#xA;&#xA;         Data not partitioned --&#xA;            Datatype  = DNA&#xA;            Nucmodel  = 4by4&#xA;            Nst       = 6&#xA;                        Substitution rates, expressed as proportions&#xA;                        of the rate sum, have a Dirichlet prior&#xA;                        (1.00,1.00,1.00,1.00,1.00,1.00)&#xA;            Covarion  = No&#xA;            # States  = 4&#xA;                        State frequencies have a Dirichlet prior&#xA;                        (1.00,1.00,1.00,1.00)&#xA;            Rates     = Gamma&#xA;                        The distribution is approximated using 4 categories.&#xA;                        Shape parameter is exponentially&#xA;                        distributed with parameter (1.00).&#xA;&#xA;      Active parameters: &#xA;&#xA;         Parameters&#xA;         ---------------------&#xA;         Revmat              1&#xA;         Statefreq           2&#xA;         Shape               3&#xA;         Ratemultiplier      4&#xA;         Topology            5&#xA;         Brlens              6&#xA;         ---------------------&#xA;&#xA;         1 --  Parameter  = Revmat&#xA;               Type       = Rates of reversible rate matrix&#xA;               Prior      = Dirichlet(1.00,1.00,1.00,1.00,1.00,1.00)&#xA;&#xA;         2 --  Parameter  = Pi&#xA;               Type       = Stationary state frequencies&#xA;               Prior      = Dirichlet&#xA;&#xA;         3 --  Parameter  = Alpha&#xA;               Type       = Shape of scaled gamma distribution of site rates&#xA;               Prior      = Exponential(1.00)&#xA;&#xA;         4 --  Parameter  = Ratemultiplier&#xA;               Type       = Partition-specific rate multiplier&#xA;               Prior      = Fixed(1.0)&#xA;&#xA;         5 --  Parameter  = Tau&#xA;               Type       = Topology&#xA;               Prior      = All topologies equally probable a priori&#xA;               Subparam.  = V&#xA;&#xA;         6 --  Parameter  = V&#xA;               Type       = Branch lengths&#xA;               Prior      = Unconstrained:GammaDir(1.0,1.0000,0.7,0.5)&#xA;&#xA;&#xA;      Number of chains per MPI processor = 8&#xA;&#xA;      The MCMC sampler will use the following moves:&#xA;         With prob.  Chain will use move&#xA;            0.94 %   Dirichlet(Revmat)&#xA;            0.94 %   Slider(Revmat)&#xA;            0.94 %   Dirichlet(Pi)&#xA;            0.94 %   Slider(Pi)&#xA;            1.89 %   Multiplier(Alpha)&#xA;            9.43 %   ExtSPR(Tau,V)&#xA;            9.43 %   ExtTBR(Tau,V)&#xA;            9.43 %   NNI(Tau,V)&#xA;            9.43 %   ParsSPR(Tau,V)&#xA;           37.74 %   Multiplier(V)&#xA;           13.21 %   Nodeslider(V)&#xA;            5.66 %   TLMultiplier(V)&#xA;&#xA;      Division 1 has 624 unique site patterns&#xA;      Initializing conditional likelihoods&#xA;      Using standard FMA likelihood calculator for division 1 (single-precision)&#xA;&#xA;      Initial log likelihoods and log prior probs for run 1:&#xA;         Chain 1 -- -38239.042144 -- 53.554896&#xA;         Chain 2 -- -37743.352685 -- 53.554896&#xA;         Chain 3 -- -38049.750737 -- 53.554896&#xA;         Chain 4 -- -38759.582835 -- 53.554896&#xA;&#xA;      Initial log likelihoods and log prior probs for run 2:&#xA;         Chain 1 -- -38348.591921 -- 53.554896&#xA;         Chain 2 -- -39602.594468 -- 53.554896&#xA;         Chain 3 -- -38874.065643 -- 53.554896&#xA;         Chain 4 -- -39227.723762 -- 53.554896&#xA;&#xA;&#xA;      Using a relative burnin of 25.0 % for diagnostics&#xA;&#xA;      Chain results (5000000 generations requested):&#xA;&#xA;         0 -- [-38239.042] (-37743.353) (-38049.751) (-38759.583) * [-38348.592] (-39602.594) (-38874.066) (-39227.724) (...0 remote chains...) &#xA;^C&#xA;[ptitle@iu mbTest]&lt;span class=&quot;math-container&quot;&gt;$ ls&#xA;sceloporus.nex  sceloporus.nex.mcmc  sceloporus.nex.run1.p  sceloporus.nex.run1.t  sceloporus.nex.run2.p  sceloporus.nex.run2.t&#xA;[ptitle@iu mbTest]$&lt;/span&gt; mpirun -np 1 ~/MrBayes/src/mb sceloporus.nex &#xA;&#xA;&#xA;                            MrBayes pre-3.2.7 x86_64&#xA;&#xA;                      (Bayesian Analysis of Phylogeny)&#xA;&#xA;                             (Parallel version)&#xA;                         (1 processors available)&#xA;&#xA;              Distributed under the GNU General Public License&#xA;&#xA;&#xA;               Type &quot;help&quot; or &quot;help &amp;lt;command&amp;gt;&quot; for information&#xA;                     on the commands that are available.&#xA;&#xA;                   Type &quot;about&quot; for authorship and general&#xA;                       information about the program.&#xA;&#xA;&#xA;&#xA;   Executing file &quot;sceloporus.nex&quot;&#xA;   DOS line termination&#xA;   Longest line length = 1632&#xA;   Parsing file&#xA;   Expecting NEXUS formatted file&#xA;   Reading data block&#xA;      Allocated taxon set&#xA;      Allocated matrix&#xA;      Defining new matrix with 123 taxa and 1606 characters&#xA;      Data is Dna&#xA;      Missing data coded as ?&#xA;      Gaps coded as -&#xA;      Taxon   1 -&amp;gt; AZYuJAS289&#xA;      Taxon   2 -&amp;gt; AZYuJAS290&#xA;      Taxon   3 -&amp;gt; AZlaJAS294&#xA;      Taxon   4 -&amp;gt; AZcoTBP271&#xA;      Taxon   5 -&amp;gt; AZcoGM1054&#xA;      Taxon   6 -&amp;gt; AZcoDGM994&#xA;      Taxon   7 -&amp;gt; AZgiP26438&#xA;      Taxon   8 -&amp;gt; AZgrGM1012&#xA;      Taxon   9 -&amp;gt; AZgrGM1013&#xA;      Taxon  10 -&amp;gt; AZgrGM1024&#xA;      Taxon  11 -&amp;gt; AZlaDGM856&#xA;      Taxon  12 -&amp;gt; AZmaDGM984&#xA;      Taxon  13 -&amp;gt; AZmaDGM992&#xA;      Taxon  14 -&amp;gt; AZmaRM4401&#xA;      Taxon  15 -&amp;gt; AZmaRM4402&#xA;      Taxon  16 -&amp;gt; AZmaRM4403&#xA;      Taxon  17 -&amp;gt; AZmoGM1045&#xA;      Taxon  18 -&amp;gt; AZmoDGM492&#xA;      Taxon  19 -&amp;gt; AZmoDGM699&#xA;      Taxon  20 -&amp;gt; AZmoDGM704&#xA;      Taxon  21 -&amp;gt; AZmoRM4394&#xA;      Taxon  22 -&amp;gt; AZpiKWS238&#xA;      Taxon  23 -&amp;gt; AZyuDGM826&#xA;      Taxon  24 -&amp;gt; AZyuDGM838&#xA;      Taxon  25 -&amp;gt; AZyuDGM852&#xA;      Taxon  26 -&amp;gt; AZyuDGM858&#xA;      Taxon  27 -&amp;gt; AZyuDGM865&#xA;      Taxon  28 -&amp;gt; CAfrP25750&#xA;      Taxon  29 -&amp;gt; CAimDGM534&#xA;      Taxon  30 -&amp;gt; CAimDGM558&#xA;      Taxon  31 -&amp;gt; CAimDGM694&#xA;      Taxon  32 -&amp;gt; CAimKWS170&#xA;      Taxon  33 -&amp;gt; CAinCRF530&#xA;      Taxon  34 -&amp;gt; CAinoDGM74&#xA;      Taxon  35 -&amp;gt; CAinCRF529&#xA;      Taxon  36 -&amp;gt; CAkeDS2SP7&#xA;      Taxon  37 -&amp;gt; CAkeP26556&#xA;      Taxon  38 -&amp;gt; CAla78323&#xA;      Taxon  39 -&amp;gt; CAlaDAW220&#xA;      Taxon  40 -&amp;gt; CAlaM14610&#xA;      Taxon  41 -&amp;gt; CAlaM23289&#xA;      Taxon  42 -&amp;gt; CAmoJQR132&#xA;      Taxon  43 -&amp;gt; CArvJOS138&#xA;      Taxon  44 -&amp;gt; CArvESA441&#xA;      Taxon  45 -&amp;gt; CAsaBUR167&#xA;      Taxon  46 -&amp;gt; CAsaMCFMAG&#xA;      Taxon  47 -&amp;gt; CAsarnMCC&#xA;      Taxon  48 -&amp;gt; CAsa150079&#xA;      Taxon  49 -&amp;gt; CAsa150088&#xA;      Taxon  50 -&amp;gt; CAsa179886&#xA;      Taxon  51 -&amp;gt; CAsaM19838&#xA;      Taxon  52 -&amp;gt; CAsdDGM691&#xA;      Taxon  53 -&amp;gt; CAsdDGM748&#xA;      Taxon  54 -&amp;gt; CAsdSWT701&#xA;      Taxon  55 -&amp;gt; MXBCDGM474&#xA;      Taxon  56 -&amp;gt; MXBCDGM475&#xA;      Taxon  57 -&amp;gt; MXBCDGM486&#xA;      Taxon  58 -&amp;gt; MXchRM4860&#xA;      Taxon  59 -&amp;gt; MXduA83134&#xA;      Taxon  60 -&amp;gt; MXsoM14488&#xA;      Taxon  61 -&amp;gt; MXsoP26449&#xA;      Taxon  62 -&amp;gt; MXso210345&#xA;      Taxon  63 -&amp;gt; NMdoTWR230&#xA;      Taxon  64 -&amp;gt; NMedDGM938&#xA;      Taxon  65 -&amp;gt; NMhiU48819&#xA;      Taxon  66 -&amp;gt; NMluGM947&#xA;      Taxon  67 -&amp;gt; NMluDGM948&#xA;      Taxon  68 -&amp;gt; NMsiDGM891&#xA;      Taxon  69 -&amp;gt; NMsoDGM900&#xA;      Taxon  70 -&amp;gt; NMsoDGM904&#xA;      Taxon  71 -&amp;gt; NMsoDGM905&#xA;      Taxon  72 -&amp;gt; NMvaDGM924&#xA;      Taxon  73 -&amp;gt; NVchDGM612&#xA;      Taxon  74 -&amp;gt; NVch162077&#xA;      Taxon  75 -&amp;gt; NVclDGM296&#xA;      Taxon  76 -&amp;gt; NVclDGM678&#xA;      Taxon  77 -&amp;gt; NVesDGM210&#xA;      Taxon  78 -&amp;gt; NVlyDGM610&#xA;      Taxon  79 -&amp;gt; NVwaDGM608&#xA;      Taxon  80 -&amp;gt; UTemGM1058&#xA;      Taxon  81 -&amp;gt; UTgrDGM156&#xA;      Taxon  82 -&amp;gt; UTgrDGM169&#xA;      Taxon  83 -&amp;gt; UTgrDGM175&#xA;      Taxon  84 -&amp;gt; UTkaGM1034&#xA;      Taxon  85 -&amp;gt; UTsnDM142&#xA;      Taxon  86 -&amp;gt; UTsnDG043&#xA;      Taxon  87 -&amp;gt; UTsnnDGM645&#xA;      Taxon  88 -&amp;gt; UTwsGM261&#xA;      Taxon  89 -&amp;gt; UTwsGM263&#xA;      Taxon  90 -&amp;gt; UTwsnDGM751&#xA;      Taxon  91 -&amp;gt; UTwsJRM4437&#xA;      Taxon  92 -&amp;gt; hunsiM376&#xA;      Taxon  93 -&amp;gt; hunsiM377&#xA;      Taxon  94 -&amp;gt; hunsiRWM619&#xA;      Taxon  95 -&amp;gt; lickGM378&#xA;      Taxon  96 -&amp;gt; lineatulus&#xA;      Taxon  97 -&amp;gt; orcuOM14377&#xA;      Taxon  98 -&amp;gt; orcuOM14453&#xA;      Taxon  99 -&amp;gt; orcuOM14673&#xA;      Taxon 100 -&amp;gt; orcuRWM798&#xA;      Taxon 101 -&amp;gt; orcuS201108&#xA;      Taxon 102 -&amp;gt; orcuS201124&#xA;      Taxon 103 -&amp;gt; zostZ16290&#xA;      Taxon 104 -&amp;gt; zosV161292&#xA;      Taxon 105 -&amp;gt; zosV161293&#xA;      Taxon 106 -&amp;gt; zosV161311&#xA;      Taxon 107 -&amp;gt; zosWM1890&#xA;      Taxon 108 -&amp;gt; zosGM364&#xA;      Taxon 109 -&amp;gt; zosGM365&#xA;      Taxon 110 -&amp;gt; zosGM387&#xA;      Taxon 111 -&amp;gt; zosGM393&#xA;      Taxon 112 -&amp;gt; zosRO332&#xA;      Taxon 113 -&amp;gt; zosOM37006&#xA;      Taxon 114 -&amp;gt; zosWM1601&#xA;      Taxon 115 -&amp;gt; clarkii&#xA;      Taxon 116 -&amp;gt; edwardti80&#xA;      Taxon 117 -&amp;gt; smaragdnus&#xA;      Taxon 118 -&amp;gt; graciosus&#xA;      Taxon 119 -&amp;gt; grammicus&#xA;      Taxon 120 -&amp;gt; maculosus&#xA;      Taxon 121 -&amp;gt; bicanthlis&#xA;      Taxon 122 -&amp;gt; torquatus&#xA;      Taxon 123 -&amp;gt; variabilis&#xA;      Successfully read matrix&#xA;      Setting default partition (does not divide up characters)&#xA;      Setting model defaults&#xA;      Seed (for generating default start values) = 1544128192&#xA;      Setting output file names to &quot;sceloporus.nex.run&amp;lt;i&amp;gt;.&amp;lt;p|t&amp;gt;&quot;&#xA;   Exiting data block&#xA;   Reading mrbayes block&#xA;      Excluding character(s)&#xA;      Setting Nst to 6&#xA;      Setting Rates to Gamma&#xA;      Successfully set likelihood model parameters&#xA;      Setting Brlenspr to Unconstrained:GammaDir(1.00,1.0000,0.70,0.50)&#xA;      Successfully set prior model parameters&#xA;      Setting number of generations to 5000000&#xA;      Running Markov chain&#xA;      MCMC stamp = 9909051894&#xA;      Seed = 1261289135&#xA;      Swapseed = 1544128192&#xA;      Model settings:&#xA;&#xA;         Data not partitioned --&#xA;            Datatype  = DNA&#xA;            Nucmodel  = 4by4&#xA;            Nst       = 6&#xA;                        Substitution rates, expressed as proportions&#xA;                        of the rate sum, have a Dirichlet prior&#xA;                        (1.00,1.00,1.00,1.00,1.00,1.00)&#xA;            Covarion  = No&#xA;            # States  = 4&#xA;                        State frequencies have a Dirichlet prior&#xA;                        (1.00,1.00,1.00,1.00)&#xA;            Rates     = Gamma&#xA;                        The distribution is approximated using 4 categories.&#xA;                        Shape parameter is exponentially&#xA;                        distributed with parameter (1.00).&#xA;&#xA;      Active parameters: &#xA;&#xA;         Parameters&#xA;         ---------------------&#xA;         Revmat              1&#xA;         Statefreq           2&#xA;         Shape               3&#xA;         Ratemultiplier      4&#xA;         Topology            5&#xA;         Brlens              6&#xA;         ---------------------&#xA;&#xA;         1 --  Parameter  = Revmat&#xA;               Type       = Rates of reversible rate matrix&#xA;               Prior      = Dirichlet(1.00,1.00,1.00,1.00,1.00,1.00)&#xA;&#xA;         2 --  Parameter  = Pi&#xA;               Type       = Stationary state frequencies&#xA;               Prior      = Dirichlet&#xA;&#xA;         3 --  Parameter  = Alpha&#xA;               Type       = Shape of scaled gamma distribution of site rates&#xA;               Prior      = Exponential(1.00)&#xA;&#xA;         4 --  Parameter  = Ratemultiplier&#xA;               Type       = Partition-specific rate multiplier&#xA;               Prior      = Fixed(1.0)&#xA;&#xA;         5 --  Parameter  = Tau&#xA;               Type       = Topology&#xA;               Prior      = All topologies equally probable a priori&#xA;               Subparam.  = V&#xA;&#xA;         6 --  Parameter  = V&#xA;               Type       = Branch lengths&#xA;               Prior      = Unconstrained:GammaDir(1.0,1.0000,0.7,0.5)&#xA;&#xA;&#xA;      Number of chains per MPI processor = 8&#xA;&#xA;      The MCMC sampler will use the following moves:&#xA;         With prob.  Chain will use move&#xA;            0.94 %   Dirichlet(Revmat)&#xA;            0.94 %   Slider(Revmat)&#xA;            0.94 %   Dirichlet(Pi)&#xA;            0.94 %   Slider(Pi)&#xA;            1.89 %   Multiplier(Alpha)&#xA;            9.43 %   ExtSPR(Tau,V)&#xA;            9.43 %   ExtTBR(Tau,V)&#xA;            9.43 %   NNI(Tau,V)&#xA;            9.43 %   ParsSPR(Tau,V)&#xA;           37.74 %   Multiplier(V)&#xA;           13.21 %   Nodeslider(V)&#xA;            5.66 %   TLMultiplier(V)&#xA;&#xA;      Division 1 has 624 unique site patterns&#xA;      Initializing conditional likelihoods&#xA;      Using standard FMA likelihood calculator for division 1 (single-precision)&#xA;&#xA;      Initial log likelihoods and log prior probs for run 1:&#xA;         Chain 1 -- -39098.978535 -- 53.554896&#xA;         Chain 2 -- -38054.044640 -- 53.554896&#xA;         Chain 3 -- -39425.015629 -- 53.554896&#xA;         Chain 4 -- -40157.756230 -- 53.554896&#xA;&#xA;      Initial log likelihoods and log prior probs for run 2:&#xA;         Chain 1 -- -40084.553712 -- 53.554896&#xA;         Chain 2 -- -39002.725314 -- 53.554896&#xA;         Chain 3 -- -38719.752582 -- 53.554896&#xA;         Chain 4 -- -39156.129424 -- 53.554896&#xA;&#xA;&#xA;      Using a relative burnin of 25.0 % for diagnostics&#xA;&#xA;      Chain results (5000000 generations requested):&#xA;&#xA;         0 -- [-39098.979] (-38054.045) (-39425.016) (-40157.756) * [-40084.554] (-39002.725) (-38719.753) (-39156.129) (...0 remote chains...) &#xA;^C[ptitle@iu mbTest]$ mpirun -np 2 ~/MrBayes/src/mb sceloporus.nex &#xA;&#xA;&#xA;                            MrBayes pre-3.2.7 x86_64&#xA;&#xA;                      (Bayesian Analysis of Phylogeny)&#xA;&#xA;                             (Parallel version)&#xA;                         (2 processors available)&#xA;&#xA;              Distributed under the GNU General Public License&#xA;&#xA;&#xA;               Type &quot;help&quot; or &quot;help &amp;lt;command&amp;gt;&quot; for information&#xA;                     on the commands that are available.&#xA;&#xA;                   Type &quot;about&quot; for authorship and general&#xA;                       information about the program.&#xA;&#xA;&#xA;&#xA;   Executing file &quot;sceloporus.nex&quot;&#xA;   DOS line termination&#xA;   Longest line length = 1632&#xA;   Parsing file&#xA;   Expecting NEXUS formatted file&#xA;   Reading data block&#xA;      Allocated taxon set&#xA;      Allocated matrix&#xA;      Defining new matrix with 123 taxa and 1606 characters&#xA;      Data is Dna&#xA;      Missing data coded as ?&#xA;      Gaps coded as -&#xA;      Taxon   1 -&amp;gt; AZYuJAS289&#xA;      Taxon   2 -&amp;gt; AZYuJAS290&#xA;      Taxon   3 -&amp;gt; AZlaJAS294&#xA;      Taxon   4 -&amp;gt; AZcoTBP271&#xA;      Taxon   5 -&amp;gt; AZcoGM1054&#xA;      Taxon   6 -&amp;gt; AZcoDGM994&#xA;      Taxon   7 -&amp;gt; AZgiP26438&#xA;      Taxon   8 -&amp;gt; AZgrGM1012&#xA;      Taxon   9 -&amp;gt; AZgrGM1013&#xA;      Taxon  10 -&amp;gt; AZgrGM1024&#xA;^C[ptitle@iu mbTest]$ &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3804" LastEditorUserId="17320" LastEditDate="2023-10-27T21:34:07.280" LastActivityDate="2023-10-28T01:03:07.463" Title="MrBayes runs as single-core, but hangs when multi-core" Tags="|phylogenetics|phylogeny|molecular-genetics|mrbayes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5605" PostTypeId="1" CreationDate="2018-12-07T01:14:46.587" Score="1" ViewCount="72" Body="&lt;p&gt;The results of &lt;a href=&quot;https://dl.acm.org/citation.cfm?id=3291461&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shekhar et al. (2017)&lt;/a&gt; &quot;indicate that ASTRAL requires &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(f^{-2} \log{n})$&lt;/span&gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gene trees to reconstruct the species tree correctly with high probability where &lt;code&gt;n&lt;/code&gt; is the number of species and &lt;code&gt;f&lt;/code&gt; is the length of the shortest branch in the species tree. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I interpret this function, specifically the big &lt;code&gt;O&lt;/code&gt; notation, when applying it to my data to estimate whether I have enough loci?&lt;/p&gt;&#xA;" OwnerUserId="3806" LastEditorUserId="3848" LastEditDate="2020-05-06T08:54:40.393" LastActivityDate="2020-05-06T08:54:40.393" Title="How do I interpret the function O(f^-2 log n) in Shekhar et al. (2017)?" Tags="|phylogenetics|phylogeny|algorithms|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5606" PostTypeId="1" AcceptedAnswerId="5607" CreationDate="2018-12-08T15:15:23.033" Score="3" ViewCount="2559" Body="&lt;p&gt;Although I know how to get total number of mapped read using samtools flagstat (&lt;code&gt;samtools flagstat file_sorted.bam&lt;/code&gt;) but I want to count total number of mapped read in a &lt;strong&gt;non-overlapping sliding window of fixed size&lt;/strong&gt; (let's say 500 bp) only with respect to reference (hg19) (assigning each read only once by its start position). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of the desired output would be something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Chromosome    Region_Start     Region_Stop    Num_Unique_Reads&#xA;chr2          1                500            200&#xA;chr2          501              1000           202&#xA;chr2          1001             1501           230&#xA;chr2          .                 .              .&#xA;chr2          .                 .              .&#xA;chr2          .                 .              .&#xA;chr2          8001              8500           20&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest me how to perform that window operation?&lt;/p&gt;&#xA;" OwnerUserId="2141" LastEditorUserId="2085" LastEditDate="2018-12-09T23:27:58.203" LastActivityDate="2018-12-10T14:26:20.953" Title="How to count the number of mapped read in 100-bp window from a BAM/SAM file" Tags="|sequence-alignment|sam|ngs|samtools|read-mapping|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5607" PostTypeId="2" ParentId="5606" CreationDate="2018-12-09T05:35:24.907" Score="10" Body="&lt;h2&gt;one-liner&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Here's a gritty one-liner to count the number of reads in a region if you have just one region that you want to investigate. Change the &lt;code&gt;1&lt;/code&gt; in &lt;code&gt;(&lt;span class=&quot;math-container&quot;&gt;$4 &amp;gt;=1)&lt;/code&gt; and the &lt;code&gt;500&lt;/code&gt; in &lt;code&gt;($&lt;/span&gt;4 &amp;lt;=500)&lt;/code&gt; to set your window. Change &lt;code&gt;&quot;hg19&quot;&lt;/code&gt; to your target sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, this one-liner does not double-count reads because of &lt;code&gt;uniq&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view file_sorted.bam | \&#xA;    awk '{ if ((&lt;span class=&quot;math-container&quot;&gt;$3 == &quot;hg19&quot;) &amp;amp;&amp;amp; ($&lt;/span&gt;4 &amp;gt;=1) &amp;amp;&amp;amp; (&lt;span class=&quot;math-container&quot;&gt;$4 &amp;lt;=500)) { \&#xA;  print($&lt;/span&gt;1) }}' | sort | uniq | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;python 3&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If you need something a little more programmatic to get a known region you &lt;a href=&quot;https://pysam.readthedocs.io/en/latest/api.html&quot; rel=&quot;noreferrer&quot;&gt;should consider trying the &lt;code&gt;pysam&lt;/code&gt; package&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;import pysam&#xA;samfile = pysam.AlignmentFile(&quot;file_sorted.bam&quot;, &quot;rb&quot;)&#xA;region_set = set()&#xA;start = 100&#xA;stop =  500&#xA;counter = 0&#xA;for read in samfile.fetch('interesting_contig', 100, 500):&#xA;    region_set.add(read.query_name)&#xA;    counter += 1&#xA;print(&quot;{} unique reads in [{}, {}]&quot;.format(len(region_set), start, stop))&#xA;print(&quot;{} alignments in [{}, {}]&quot;.format(counter, start, stop))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For my &lt;code&gt;bam&lt;/code&gt; file, this program outputs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1618 unique reads in [100, 500]&#xA;2487 alignments in [100, 500]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;python 3, non-overlapping windows&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://pysam.readthedocs.io/en/latest/faq.html#pysam-coordinates-are-wrong&quot; rel=&quot;noreferrer&quot;&gt;It is important to note that the &lt;code&gt;samfile.fetch()&lt;/code&gt; function conforms to 1-based indexing, consistent with SAM files, unlike pysam's otherwise default 0-based indexing that matches the BAM-indexing.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;import pysam&#xA;samfile = pysam.AlignmentFile(&quot;file_sorted.bam&quot;, &quot;rb&quot;)&#xA;print(samfile.references)&#xA;print(samfile.lengths)&#xA;window = 500&#xA;print(&quot;ref\tstart\tstop\tuniq_reads&quot;)&#xA;for i in range(len(samfile.references)):&#xA;    refname = samfile.references[i]&#xA;    seqlen = samfile.lengths[i]&#xA;    for j in range(1, seqlen, window):&#xA;        stop = j+window-1 if j+window-1 &amp;lt; samfile.lengths[i] else samfile.lengths[i]&#xA;        region_set = set()&#xA;        for read in samfile.fetch(refname, j, stop):&#xA;            region_set.add(read.query_name)&#xA;        print(&quot;{}\t{}\t{}\t{}&quot;.format(refname, j, stop, len(region_set)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For the small bam file I looked at the output was:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ref         start  stop  uniq_reads&#xA;OdonB       0      500   1668&#xA;OdonB       501    1000  2647&#xA;OdonB       1001   1500  129400&#xA;OdonB       1501   2000  598&#xA;OdonB       2001   2500  538&#xA;OdonB       2501   3000  872&#xA;OdonB       3001   3500  561&#xA;OdonB       3501   4000  49120&#xA;OdonB       4001   4500  48433&#xA;OdonB       4501   4806  28121&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-12-10T00:02:51.543" LastActivityDate="2018-12-10T00:02:51.543" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5609" PostTypeId="1" AcceptedAnswerId="5610" CreationDate="2018-12-09T12:19:11.107" Score="1" ViewCount="183" Body="&lt;p&gt;I'm trying to automate some of my previous bash scripts with snakemake. Below is one of the recipe I created: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;shell.executable(&quot;/bin/bash&quot;)&#xA;&#xA;workdir: &quot;/proj/uppstore2018136/publication/jupyterNotebooks&quot;&#xA;&#xA;ENSEMBL_RELEASE='94'&#xA;ENSEMBL_GRCm38_BASE='ftp://ftp.ensembl.org/pub/release-'+ENSEMBL_RELEASE+'/fasta/mus_musculus/dna'&#xA;ENSEMBL_GRCm38_GTF_BASE='ftp://ftp.ensembl.org/pub/release-'+ENSEMBL_RELEASE+'/gtf/mus_musculus'&#xA;GTF_FILE='Mus_musculus.GRCm38.'+ENSEMBL_RELEASE+'.gtf'&#xA;&#xA;rule all:&#xA;    input:&#xA;        &quot;genome/genome.fa&quot;,&#xA;        &quot;genome/&quot;+GTF_FILE,&#xA;        &quot;indexes/bowtie2/reference.1.bt2&quot;,&#xA;        &quot;indexes/bowtie2/reference.2.bt2&quot;,&#xA;        &quot;indexes/bowtie2/reference.3.bt2&quot;,&#xA;        &quot;indexes/bowtie2/reference.4.bt2&quot;,&#xA;        &quot;indexes/bowtie2/reference.rev1.bt2&quot;,&#xA;        &quot;indexes/bowtie2/reference.rev2.bt2&quot;,&#xA;        &quot;indexes/hisat2/reference.1.ht2&quot;,&#xA;        &quot;indexes/hisat2/reference.2.ht2&quot;,&#xA;        &quot;indexes/hisat2/reference.3.ht2&quot;,&#xA;        &quot;indexes/hisat2/reference.4.ht2&quot;,&#xA;        &quot;indexes/hisat2/reference.5.ht2&quot;,&#xA;        &quot;indexes/hisat2/reference.6.ht2&quot;,&#xA;        &quot;indexes/hisat2/reference.7.ht2&quot;,&#xA;        &quot;indexes/hisat2/reference.8.ht2&quot;&#xA;&#xA;rule getGenome:&#xA;    output:&#xA;        output1=&quot;genome/genome.fa&quot;,&#xA;        output2=&quot;genome/&quot;+GTF_FILE&#xA;    shell:&#xA;           '''&#xA;           curl -o genome/Mus_musculus.GRCm38.dna.primary_assembly.fa.gz {ENSEMBL_GRCm38_BASE}/Mus_musculus.GRCm38.dna.primary_assembly.fa.gz&#xA;           gunzip genome/Mus_musculus.GRCm38.dna.primary_assembly.fa.gz&#xA;           mv genome/Mus_musculus.GRCm38.dna.primary_assembly.fa genome/genome.fa&#xA;           curl -o genome/{GTF_FILE}.gz {ENSEMBL_GRCm38_GTF_BASE}/{GTF_FILE}.gz&#xA;           gunzip genome/{GTF_FILE}.gz&#xA;           '''&#xA;&#xA;rule bowtie2Build:&#xA;    input: &quot;genome/genome.fa&quot;&#xA;    params:&#xA;        basename=&quot;indexes/bowtie2/reference&quot;&#xA;    output:&#xA;        output1=&quot;indexes/bowtie2/reference.1.bt2&quot;,&#xA;        output2=&quot;indexes/bowtie2/reference.2.bt2&quot;,&#xA;        output3=&quot;indexes/bowtie2/reference.3.bt2&quot;,&#xA;        output4=&quot;indexes/bowtie2/reference.4.bt2&quot;,&#xA;        outputrev1=&quot;indexes/bowtie2/reference.rev1.bt2&quot;,&#xA;        outputrev2=&quot;indexes/bowtie2/reference.rev2.bt2&quot;&#xA;    threads: 20&#xA;    shell: &#xA;        '''&#xA;        module load bowtie2&#xA;        bowtie2-build --threads {threads} {input} {params.basename}&#xA;        '''&#xA;&#xA;rule hisat2Build:&#xA;    input: &quot;genome/genome.fa&quot;,&#xA;        &quot;genome/&quot;+GTF_FILE &#xA;    params:&#xA;        basename=&quot;indexes/hisat2/reference&quot;&#xA;    output:&#xA;        output1=&quot;indexes/hisat2/reference.1.ht2&quot;,&#xA;        output2=&quot;indexes/hisat2/reference.2.ht2&quot;,&#xA;        output3=&quot;indexes/hisat2/reference.3.ht2&quot;,&#xA;        output4=&quot;indexes/hisat2/reference.4.ht2&quot;,&#xA;        output5=&quot;indexes/hisat2/reference.5.ht2&quot;,&#xA;        output6=&quot;indexes/hisat2/reference.6.ht2&quot;,&#xA;        output7=&quot;indexes/hisat2/reference.7.ht2&quot;,&#xA;        output8=&quot;indexes/hisat2/reference.8.ht2&quot;,&#xA;    threads: 20&#xA;    shell: &#xA;        '''&#xA;        module load HISAT2&#xA;        hisat2_extract_splice_sites.py genome/{GTF_FILE} &amp;gt; genome/genome.ss&#xA;        hisat2_extract_exons.py genome/{GTF_FILE} &amp;gt; genome/genome.exon&#xA;        hisat2-build -f -p {threads} {input} --ss genome/genome.ss --exon genome/genome.exon {params.basename}&#xA;        rm genome/genome.ss&#xA;        rm genome/genome.exon&#xA;        '''&#xA;&#xA;rule clean:&#xA;    shell:&#xA;        '''&#xA;        rm genome/*&#xA;        rm indexes/bowtie2/*&#xA;        rm indexes/hisat2/*&#xA;        '''&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this is my driver script which I'm submitting as a job itself:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;module load snakemake&#xA;&#xA;snakemake --jobs 1 --local-cores 20 --cluster-config /crex/proj/uppstore2018136/publication/jupyterNotebooks/recipes/cluster.json --latency-wait 120 --max-jobs-per-second 1 -c &quot;sbatch -A {cluster.account} -p {cluster.partition} -n {-cluster.n} -t {cluster.time} -C {cluster.C}&quot; -s indexMake&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I'm getting this error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Building DAG of jobs...&#xA;Using shell: /bin/bash&#xA;Provided cluster nodes: 1&#xA;Job counts:&#xA;    count   jobs&#xA;    1   all&#xA;    1   bowtie2Build&#xA;    1   getGenome&#xA;    1   hisat2Build&#xA;    4&#xA;&#xA;rule getGenome:&#xA;    output: genome/genome.fa, genome/Mus_musculus.GRCm38.94.gtf&#xA;    jobid: 2&#xA;&#xA;RuleException in line 30 of /crex/proj/uppstore2018136/publication/jupyterNotebooks/recipes/indexMake:&#xA;NameError: The name '-cluster' is unknown in this context. Please make sure that you defined that variable. Also note that braces not used for variable access have to be escaped by repeating them, i.e. {{print $1}&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I really am not sure what I'm doing wrong here ? Is there a conflict with variables or there is something wrong with the snakemake version I'm using (4.5.0).&lt;/p&gt;&#xA;" OwnerUserId="1135" LastActivityDate="2018-12-09T15:51:06.553" Title="Snakemake and SLURM -unable to use it with with --cluster" Tags="|snakemake|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5610" PostTypeId="2" ParentId="5609" CreationDate="2018-12-09T15:51:06.553" Score="5" Body="&lt;p&gt;In your submission script you have &lt;code&gt;-n {-cluster.n}&lt;/code&gt;, hence the error from snakemake I guess. Are you sure it is not a typo and it should be &lt;code&gt;-n {cluster.n}&lt;/code&gt; instead?&lt;/p&gt;&#xA;" OwnerUserId="339" LastActivityDate="2018-12-09T15:51:06.553" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5611" PostTypeId="1" CreationDate="2018-12-09T18:39:35.900" Score="1" ViewCount="367" Body="&lt;p&gt;I can draw a molecule in a jupyter notebook with RDKit. Though, when I use the savefig() method only a white canvas is saved. No molecule is saved.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%pylab inline&#xA;from rdkit import Chem&#xA;from rdkit.Chem import Draw&#xA;&#xA;smiles = 'CCC'&#xA;m = Chem.MolFromSmiles(smiles)&#xA;fig = Draw.MolToMPL(m)&#xA;title('Test')&#xA;fig.savefig('mol.jpeg')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any idea what goes wrong here?&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2018-12-09T18:57:41.933" Title="Rdkit.Chem.Draw.savefig() creates empty figures" Tags="|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5612" PostTypeId="2" ParentId="5611" CreationDate="2018-12-09T18:57:41.933" Score="1" Body="&lt;p&gt;For some reason you have to set &lt;code&gt;bbox_inches&lt;/code&gt; to tight. Without that rdkit does not fit the molecule into the canvas and might plot an empty figure.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fig.savefig('mol.jpeg', bbox_inches='tight')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2018-12-09T18:57:41.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5613" PostTypeId="1" CreationDate="2018-12-09T19:01:19.347" Score="3" ViewCount="390" Body="&lt;p&gt;RDKit can plot molecules, thought the structures are surrounded by an ugly frame with ticks. How can I turn that off and plot the molecules without that frame?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, the &lt;code&gt;%pylab inline&lt;/code&gt; portion of this code is from the Jupyter environment.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;%pylab inline&#xA;from rdkit import Chem&#xA;from rdkit.Chem import Draw&#xA;&#xA;smiles = 'CCC'&#xA;m = Chem.MolFromSmiles(smiles)&#xA;fig = Draw.MolToMPL(m)&#xA;title('Test')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gMDmg.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gMDmg.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="96" LastEditDate="2019-01-29T20:31:47.497" LastActivityDate="2019-01-29T20:31:47.497" Title="How to remove frame on RDKit figures?" Tags="|python|visualization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5614" PostTypeId="1" CreationDate="2018-12-09T22:04:38.270" Score="2" ViewCount="113" Body="&lt;p&gt;I am trying to use the R package &lt;a href=&quot;https://github.com/thierrygosselin/grur&quot; rel=&quot;nofollow noreferrer&quot;&gt;grur&lt;/a&gt; to process GBS SNP data, which is installed from github as instructed.  For starters, I am running the test code in this &lt;a href=&quot;https://www.dropbox.com/s/4zf032g6yjatj0a/vignette_missing_data_analysis.nb.html?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;vignette&lt;/a&gt;.  As suggested I am running it from console in RStudio (v. 1.1.383), and with a recent R installation (below).  I do, and get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; &amp;gt; ibm &amp;lt;- grur::missing_visualization(data = &quot;example_vcf2dadi_ferchaud_2015.vcf&quot;, strata = &quot;strata.stickleback.tsv&quot;)&#xA;#######################################################################&#xA;#################### grur::missing_visualization ######################&#xA;#######################################################################&#xA;Folder created: &#xA;missing_visualization_20181209@1523&#xA;&#xA;Importing data&#xA;&#xA;Reading VCF...&#xA;Generated a filters parameters file: filters_parameters_20181209@1523.tsv&#xA;&#xA;Number of SNPs: 31802&#xA;Number of samples: 177&#xA;&#xA;conversion timing: 1 sec&#xA;&#xA;VCF: biallelic SNPs&#xA;Cleaning VCF sample names&#xA;&#xA;Synchronizing sample IDs in VCF and strata...&#xA;Reads assembly: reference-assisted&#xA;Filters parameters file: updated&#xA;&#xA;&#xA;Number of chromosome/contig/scaffold: 196&#xA;Number of locus: 17095&#xA;Number of markers: 31802&#xA;Number of individuals: 177&#xA;&#xA;Working time: 1 sec&#xA;&#xA;&#xA;&#xA;Deprecated function, update your code to use: filter_monomorphic&#xA;&#xA;&#xA;Scanning for monomorphic markers...&#xA;Number of markers before/blacklisted/after: 31802/0/31802&#xA;&#xA;Tidy genomic data:&#xA;    Number of markers: 31802&#xA;    Number of chromosome/contig/scaffold: 196&#xA;    Number of individuals: 177&#xA;    Number of populations: 8&#xA;&#xA;Informations:&#xA;Number of populations: 8&#xA;Number of individuals: 177&#xA;Number of ind/pop:&#xA;HAD = 21&#xA;HAL = 19&#xA;KIB = 17&#xA;KRO = 20&#xA;MOS = 20&#xA;MAR = 20&#xA;NOR = 20&#xA;ODD = 40&#xA;&#xA;Number of duplicate id: 0&#xA;Number of chrom/scaffolds: 196&#xA;Number of locus: 17095&#xA;Number of SNPs: 31802&#xA;&#xA;Proportion of missing genotypes (overall): 0.022169&#xA;&#xA;Identity-by-missingness (IBM) analysis using&#xA;    Principal Coordinate Analysis (PCoA)...&#xA;Generating Identity by missingness plot&#xA;&#xA;Redundancy analysis...&#xA;&#xA;Error in UseMethod(&quot;ungroup&quot;) : &#xA;  no applicable method for 'ungroup' applied to an object of class &quot;character&quot;&#xA;In addition: Warning message:&#xA;Trying to compute distinct() for variables not found in the data:&#xA;- `strata.select`&#xA;This is an error, but only a warning is raised for compatibility reasons.&#xA;The operation will return the input unchanged. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The final object is not generated, however some plots of the PCoA analysis are. I get this result on two Macs, one with Sierra, the other with High Sierra, but both with the same RStudio and R versions. All R packages are up to date. I get the same error when I run my own .vcf &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone else get these errors on this test dataset from the repo? Can anyone suggest how to deal with this?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; version                          &#xA;platform       x86_64-apple-darwin15.6.0   &#xA;arch           x86_64                      &#xA;os             darwin15.6.0                &#xA;system         x86_64, darwin15.6.0        &#xA;status                                     &#xA;major          3                           &#xA;minor          5.1                         &#xA;year           2018                        &#xA;month          07                          &#xA;day            02                          &#xA;svn rev        74947                       &#xA;language       R                           &#xA;version.string R version 3.5.1 (2018-07-02)&#xA;nickname       Feather Spray          &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2196" LastActivityDate="2018-12-09T22:04:38.270" Title="R package grur: missing_visualization() fails during redundancy analysis of .vcf" Tags="|r|vcf|snp|radseq|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5615" PostTypeId="2" ParentId="5613" CreationDate="2018-12-10T00:27:31.380" Score="2" Body="&lt;p&gt;It looks like &lt;code&gt;rdkit&lt;/code&gt; and &lt;code&gt;rdkit&lt;/code&gt;'s &lt;code&gt;Chem&lt;/code&gt; just use the &lt;code&gt;matplotlib&lt;/code&gt;. So, if you import &lt;code&gt;matplotlib&lt;/code&gt; you can edit your figure like any other &lt;code&gt;matplotlib&lt;/code&gt; figure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I produced this from the command line, so it may work differently in the Jupyter environment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like &lt;code&gt;Chem&lt;/code&gt; doesn't properly scale the drawing based on how large the chemical is, so I added &lt;code&gt;plt.xlim&lt;/code&gt; and &lt;code&gt;plt.ylim&lt;/code&gt; to make the molecule larger in the frame.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;from rdkit import Chem&#xA;from rdkit.Chem import Draw&#xA;import matplotlib.pyplot as plt&#xA;smiles = 'CCC'&#xA;m = Chem.MolFromSmiles(smiles)&#xA;fig = Draw.MolToMPL(m)&#xA;plt.title('Test')&#xA;plt.axis('off')&#xA;coord_min = 0.35&#xA;coord_max = 0.65&#xA;plt.xlim((coord_min, coord_max))&#xA;plt.ylim((coord_min, coord_max))&#xA;fig.savefig('mol.jpeg', bbox_inches='tight')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RBrb2.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RBrb2.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-12-10T00:34:44.660" LastActivityDate="2018-12-10T00:34:44.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5616" PostTypeId="1" AcceptedAnswerId="5617" CreationDate="2018-12-10T02:19:09.900" Score="7" ViewCount="1561" Body="&lt;p&gt;Is there a way to import a GFF file for an organism with Biopython in the same way you can for a Genbank file?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez as ez&#xA;ez.email = '...'&#xA;handle = ez.efetch(db='gene', id='AE015451.2', rettype='genbank', retmode='xml')&#xA;h = handle.read()&#xA;print(h)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will import a Gebank xml file for Pseudomonas putida.  Is there a way to do this for a GFF file?&lt;/p&gt;&#xA;" OwnerUserId="3816" LastEditorUserId="96" LastEditDate="2018-12-13T14:35:53.897" LastActivityDate="2018-12-13T14:35:53.897" Title="Importing GFF file with Biopython" Tags="|python|phylogenetics|genbank|gff|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5617" PostTypeId="2" ParentId="5616" CreationDate="2018-12-10T02:57:42.623" Score="10" Body="&lt;p&gt;No, &lt;a href=&quot;https://biopython.org/wiki/GFF_Parsing&quot; rel=&quot;noreferrer&quot;&gt;there is currently no GFF support in biopython&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/daler/gffutils&quot; rel=&quot;noreferrer&quot;&gt;However, you can read in GFF files into python using this package, gffutils&lt;/a&gt;. There are also a few other packages to read/write GFF files, like &lt;a href=&quot;https://pypi.org/project/gff3/&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;gff3&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-12-10T02:57:42.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5618" PostTypeId="1" CreationDate="2018-12-10T04:40:59.863" Score="3" ViewCount="6757" Body="&lt;p&gt;I would like to ask how BLOSUM matrix is constructed and calculated ? I read the wikipedia and I still do not understand it. I do not understand the mathematical calculations as I have low knowledge about logarithms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe could someone give a simple example and then a real-scenario example of how the BLOSUM matrix can be used and calculated, maybe in relation to the Blast algorithm where it uses the BLOSUM scoring matrix to determine high scoring words for each word in the query sequence ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, why are they different types of BLOSUM matrices like BLOSUM62 and so on ?&lt;/p&gt;&#xA;" OwnerUserId="3797" LastActivityDate="2019-01-08T22:28:15.670" Title="How BLOSUM Matrix is constructed and calculated" Tags="|sequence-alignment|blast|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5620" PostTypeId="2" ParentId="5618" CreationDate="2018-12-10T10:46:12.737" Score="3" Body="&lt;p&gt;Sean Eddy wrote an excellent explanation of the BLOSUM Matrix in his Nature Biotechnology paper &lt;a href=&quot;https://www.nature.com/articles/nbt0804-1035&quot; rel=&quot;nofollow noreferrer&quot;&gt;Where did the BLOSUM62 alignment score matrix come from?&lt;/a&gt; He also gives real examples of how one would calculate the score for different amino acid pairs and where the numbers come from.  Further references given at the end of the paper might also be useful reading, specifically: &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/002228369190193A?via%3Dihub&quot; rel=&quot;nofollow noreferrer&quot;&gt;Amino acid substitution matrices from an information theoretic perspective&lt;/a&gt;, &lt;a href=&quot;https://www.pnas.org/content/87/6/2264.long&quot; rel=&quot;nofollow noreferrer&quot;&gt;Methods for assessing the statistical significance of molecular sequence features by using general scoring schemes&lt;/a&gt;, and &lt;a href=&quot;https://www.pnas.org/content/89/22/10915.long&quot; rel=&quot;nofollow noreferrer&quot;&gt;Amino acid substitution matrices from protein blocks&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1769" LastEditorUserId="1769" LastEditDate="2018-12-11T07:30:19.643" LastActivityDate="2018-12-11T07:30:19.643" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5621" PostTypeId="1" AcceptedAnswerId="5634" CreationDate="2018-12-10T11:48:41.860" Score="5" ViewCount="1311" Body="&lt;h3&gt;Why I need a compatible file&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I’m trying to run &#xA;&lt;a href=&quot;http://velocyto.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;velocyto&lt;/a&gt; with the &lt;a href=&quot;https://github.com/velocyto-team/velocyto.R&quot; rel=&quot;nofollow noreferrer&quot;&gt;R package&lt;/a&gt; to analyse RNA velocity (cell trajectories) with single cell RNASeq data. I have performed single cell analysis from 10x Genomics data using cellranger.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have successfully aligned the reads to get loom files and imported these into R. I can get the velocity from these files by following &lt;a href=&quot;http://pklab.med.harvard.edu/velocyto/notebooks/R/chromaffin2.nb.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the vignettes&lt;/a&gt;. However, I cannot reproduce the RNA velocity analysis based on “gene structure”.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’m working with a different organism to the example (not human or mouse) so the annotation data provided in the example does not work. I have a GTF file for the latest annotation of this organism. However, it only contains “exon” and “CDS” as features. This appears to be the source of the problem. The “find.ip.sites” function requires a GTF with “features” = “gene” and one of the “attributes” to be “protein_coding”. These requirements are hard-coded into the velocyto.R function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following GTF files from the &lt;a href=&quot;https://ics.hutton.ac.uk/atRTD/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AtRTD2&lt;/a&gt; dataset. The chromosome labels match the loom files in R.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Chr1    TAIR10  exon    3631    3913    .   +   .   transcript_id &quot;AT1G01010.1&quot;; gene_id &quot;AT1G01010&quot;; gene_name &quot;AT1G01010&quot;;&#xA;Chr1    TAIR10  exon    3996    4276    .   +   .   transcript_id &quot;AT1G01010.1&quot;; gene_id &quot;AT1G01010&quot;; gene_name &quot;AT1G01010&quot;;&#xA;Chr1    TAIR10  exon    4486    4605    .   +   .   transcript_id &quot;AT1G01010.1&quot;; gene_id &quot;AT1G01010&quot;; gene_name &quot;AT1G01010&quot;;&#xA;Chr1    TAIR10  exon    4706    5095    .   +   .   transcript_id &quot;AT1G01010.1&quot;; gene_id &quot;AT1G01010&quot;; gene_name &quot;AT1G01010&quot;;&#xA;Chr1    TAIR10  exon    5174    5326    .   +   .   transcript_id &quot;AT1G01010.1&quot;; gene_id &quot;AT1G01010&quot;; gene_name &quot;AT1G01010&quot;;&#xA;Chr1    TAIR10  exon    5439    5899    .   +   .   transcript_id &quot;AT1G01010.1&quot;; gene_id &quot;AT1G01010&quot;; gene_name &quot;AT1G01010&quot;;&#xA;Chr1    TAIR10  CDS 3760    3913    .   +   0   transcript_id &quot;AT1G01010.1&quot;; gene_id &quot;AT1G01010&quot;; gene_name &quot;AT1G01010&quot;;&#xA;Chr1    TAIR10  CDS 3996    4276    .   +   2   transcript_id &quot;AT1G01010.1&quot;; gene_id &quot;AT1G01010&quot;; gene_name &quot;AT1G01010&quot;;&#xA;Chr1    TAIR10  CDS 4486    4605    .   +   0   transcript_id &quot;AT1G01010.1&quot;; gene_id &quot;AT1G01010&quot;; gene_name &quot;AT1G01010&quot;;&#xA;Chr1    TAIR10  CDS 4706    5095    .   +   0   transcript_id &quot;AT1G01010.1&quot;; gene_id &quot;AT1G01010&quot;; gene_name &quot;AT1G01010&quot;;&#xA;Chr1    TAIR10  CDS 5174    5326    .   +   0   transcript_id &quot;AT1G01010.1&quot;; gene_id &quot;AT1G01010&quot;; gene_name &quot;AT1G01010&quot;;&#xA;Chr1    TAIR10  CDS 5439    5630    .   +   0   transcript_id &quot;AT1G01010.1&quot;; gene_id &quot;AT1G01010&quot;; gene_name &quot;AT1G01010&quot;;&#xA;Chr1    Araport11   exon    6788    7069    .   -   .   transcript_id &quot;AT1G01020_P2&quot;; gene_id &quot;AT1G01020&quot;; gene_name &quot;AT1G01020&quot;; Note &quot;ARV1&quot;;&#xA;Chr1    Araport11   exon    7157    7450    .   -   .   transcript_id &quot;AT1G01020_P2&quot;; gene_id &quot;AT1G01020&quot;; gene_name &quot;AT1G01020&quot;; Note &quot;ARV1&quot;;&#xA;Chr1    Araport11   exon    7564    7649    .   -   .   transcript_id &quot;AT1G01020_P2&quot;; gene_id &quot;AT1G01020&quot;; gene_name &quot;AT1G01020&quot;; Note &quot;ARV1&quot;;&#xA;Chr1    Araport11   exon    7762    7835    .   -   .   transcript_id &quot;AT1G01020_P2&quot;; gene_id &quot;AT1G01020&quot;; gene_name &quot;AT1G01020&quot;; Note &quot;ARV1&quot;;&#xA;Chr1    Araport11   exon    7942    7987    .   -   .   transcript_id &quot;AT1G01020_P2&quot;; gene_id &quot;AT1G01020&quot;; gene_name &quot;AT1G01020&quot;; Note &quot;ARV1&quot;;&#xA;Chr1    Araport11   exon    8236    8325    .   -   .   transcript_id &quot;AT1G01020_P2&quot;; gene_id &quot;AT1G01020&quot;; gene_name &quot;AT1G01020&quot;; Note &quot;ARV1&quot;;&#xA;Chr1    Araport11   exon    8417    8464    .   -   .   transcript_id &quot;AT1G01020_P2&quot;; gene_id &quot;AT1G01020&quot;; gene_name &quot;AT1G01020&quot;; Note &quot;ARV1&quot;;&#xA;Chr1    Araport11   exon    8571    8737    .   -   .   transcript_id &quot;AT1G01020_P2&quot;; gene_id &quot;AT1G01020&quot;; gene_name &quot;AT1G01020&quot;; Note &quot;ARV1&quot;;&#xA;Chr1    Araport11   CDS 7315    7450    .   -   1   transcript_id &quot;AT1G01020_P2&quot;; gene_id &quot;AT1G01020&quot;; gene_name &quot;AT1G01020&quot;; Note &quot;ARV1&quot;;&#xA;Chr1    Araport11   CDS 7564    7649    .   -   0   transcript_id &quot;AT1G01020_P2&quot;; gene_id &quot;AT1G01020&quot;; gene_name &quot;AT1G01020&quot;; Note &quot;ARV1&quot;;&#xA;Chr1    Araport11   CDS 7762    7835    .   -   2   transcript_id &quot;AT1G01020_P2&quot;; gene_id &quot;AT1G01020&quot;; gene_name &quot;AT1G01020&quot;; Note &quot;ARV1&quot;;&#xA;Chr1    Araport11   CDS 7942    7987    .   -   0   transcript_id &quot;AT1G01020_P2&quot;; gene_id &quot;AT1G01020&quot;; gene_name &quot;AT1G01020&quot;; Note &quot;ARV1&quot;;&#xA;Chr1    Araport11   CDS 8236    8325    .   -   0   transcript_id &quot;AT1G01020_P2&quot;; gene_id &quot;AT1G01020&quot;; gene_name &quot;AT1G01020&quot;; Note &quot;ARV1&quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The specifications for a &lt;a href=&quot;https://asia.ensembl.org/info/website/upload/gff.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GTF or GFF2&lt;/a&gt; are:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;h3&gt;Fields&lt;/h3&gt;&#xA;  &#xA;  &lt;p&gt;Fields must be tab-separated. Also, all but the final field in each feature line must contain a value; &quot;empty&quot; columns should be denoted with a '.'&lt;/p&gt;&#xA;  &#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;&lt;p&gt;seqname - name of the chromosome or scaffold; chromosome names can be given with or without the 'chr' prefix. Important note: the seqname must be one used within Ensembl, i.e. a standard chromosome name or an Ensembl identifier such as a scaffold ID, without any additional content such as species or assembly. See the example GFF output below.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;source - name of the program that generated this feature, or the data source (database or project name)&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;feature - feature type name, e.g. Gene, Variation, Similarity&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;start - Start position of the feature, with sequence numbering starting at 1.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;end - End position of the feature, with sequence numbering starting at 1.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;score - A floating point value.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;strand - defined as + (forward) or - (reverse).&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;frame - One of '0', '1' or '2'. '0' indicates that the first base of the feature is the first base of a codon, '1' that the second base is the first base of a codon, and so on..&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;attribute - A semicolon-separated list of tag-value pairs, providing additional information about each feature.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h3&gt;What I’m looking for&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Is there a way to derive a compatible GTF file containing protein coding genes from the exons and CDS? I want to produce a GTF which contains genes in the features and protein_coding in the attributes. Is it possible to do this with existing tools or scripts? &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;What I’ve tried so far&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I can modify the &lt;a href=&quot;https://github.com/velocyto-team/velocyto.R/blob/666e1db1a6bebf042114eb9eb4c58f54cb002d45/R/momentum_routines.R#L2157-L2242&quot; rel=&quot;nofollow noreferrer&quot;&gt;source code&lt;/a&gt; of the “find.ip.sites” function to run on my GTF file with these features missing. However, this requires running internal functions from the package written in Rcpp and means my workflow override future updates to the package. Running the rest of the vignette returns errors as no introns long enough have been identified (setting the thresholds lower is also incompatible with the general linear models called). Therefore I think it is better to generate a compatible GTF or GFF3 file rather than alerting the source code of the functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While intended for GTF files, the package functions can import GFF3 files, despite being described for GTF in the documentation. I’ve tried generating a GFF3 file using gffread from Cufflinks and replacing “feature” = “mRNA” with “gene” and adding “protein_coding”. This also returns errors when running the velocity algorithm. It does not work for either the GTF file used as an input for cellranger or the file generated by it. Is there a way to annotate protein coding genes and intron/exon boundaries based on a GTF file containing only exon and CDS annotations?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With cufflinks 2.2.1, a GFF3 was generated&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gffread - E AtRTD2_19April2016.gtf -o- &amp;gt; AtRTD2_19April2016.gff&#xA;sed -i '/mRNA/s/gene_name=AT/gene_type=protein_coding;gene_name=AT/g' AtRTD2_19April2016.gff'&#xA;sed -i 's/mRNA/gene/g' AtRTD2_19April2016.gff&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the GFF3 files that I've tried:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# gffread - E AtRTD2_19April2016.gtf -o- &#xA;##gff-version3&#xA;Chr1    TAIR10  gene    3631    5899    .   +   .   ID=AT1G01010.1;geneID=AT1G01010;gene_type=protein_coding;gene_name=AT1G01010&#xA;Chr1    TAIR10  exon    3631    3913    .   +   .   Parent=AT1G01010.1&#xA;Chr1    TAIR10  exon    3996    4276    .   +   .   Parent=AT1G01010.1&#xA;Chr1    TAIR10  exon    4486    4605    .   +   .   Parent=AT1G01010.1&#xA;Chr1    TAIR10  exon    4706    5095    .   +   .   Parent=AT1G01010.1&#xA;Chr1    TAIR10  exon    5174    5326    .   +   .   Parent=AT1G01010.1&#xA;Chr1    TAIR10  exon    5439    5899    .   +   .   Parent=AT1G01010.1&#xA;Chr1    TAIR10  CDS 3760    3913    .   +   0   Parent=AT1G01010.1&#xA;Chr1    TAIR10  CDS 3996    4276    .   +   2   Parent=AT1G01010.1&#xA;Chr1    TAIR10  CDS 4486    4605    .   +   0   Parent=AT1G01010.1&#xA;Chr1    TAIR10  CDS 4706    5095    .   +   0   Parent=AT1G01010.1&#xA;Chr1    TAIR10  CDS 5174    5326    .   +   0   Parent=AT1G01010.1&#xA;Chr1    TAIR10  CDS 5439    5630    .   +   0   Parent=AT1G01010.1&#xA;Chr1    Araport11   gene    6788    8737    .   +   .   ID=AT1G01020_P2;geneID=AT1G01020;gene_type=protein_coding;gene_name=AT1G01020    &#xA;Chr1    Araport11   exon    6788    7069    .   -   .   Parent=AT1G01020_P2&#xA;Chr1    Araport11   exon    7157    7450    .   -   .   Parent=AT1G01020_P2&#xA;Chr1    Araport11   exon    7564    7649    .   -   .   Parent=AT1G01020_P2&#xA;Chr1    Araport11   exon    7762    7835    .   -   .   Parent=AT1G01020_P2&#xA;Chr1    Araport11   exon    7942    7987    .   -   .   Parent=AT1G01020_P2&#xA;Chr1    Araport11   exon    8236    8325    .   -   .   Parent=AT1G01020_P2&#xA;Chr1    Araport11   exon    8417    8464    .   -   .   Parent=AT1G01020_P2&#xA;Chr1    Araport11   exon    8571    8737    .   -   .   Parent=AT1G01020_P2&#xA;Chr1    Araport11   CDS 7315    7450    .   -   1   Parent=AT1G01020_P2&#xA;Chr1    Araport11   CDS 7564    7649    .   -   0   Parent=AT1G01020_P2&#xA;Chr1    Araport11   CDS 7762    7835    .   -   2   Parent=AT1G01020_P2&#xA;Chr1    Araport11   CDS 7942    7987    .   -   0   Parent=AT1G01020_P2&#xA;Chr1    Araport11   CDS 8236    8325    .   -   0   Parent=AT1G01020_P2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The specifications for a &lt;a href=&quot;https://asia.ensembl.org/info/website/upload/gff.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GTF or GFF2&lt;/a&gt; are:     &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The specifications for a &lt;a href=&quot;https://asia.ensembl.org/info/website/upload/gff3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GF3&lt;/a&gt; are:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;h3&gt;Fields&lt;/h3&gt;&#xA;  &#xA;  &lt;p&gt;Fields must be tab-separated. Also, all but the final field in each feature line must contain a value; &quot;empty&quot; columns should be denoted with a '.'&lt;/p&gt;&#xA;  &#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;&lt;p&gt;seqid - name of the chromosome or scaffold; chromosome names can be given with or without the 'chr' prefix. Important note: the seq ID must be one used within Ensembl, i.e. a standard chromosome name or an Ensembl identifier such as a scaffold ID, without any additional content such as species or assembly. See the example GFF output below.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;source - name of the program that generated this feature, or the data source (database or project name)&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;type - type of feature. Must be a term or accession from the SOFA sequence ontology&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;start - Start position of the feature, with sequence numbering starting at 1.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;end - End position of the feature, with sequence numbering starting at 1.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;score - A floating point value.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;strand - defined as + (forward) or - (reverse).&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;phase - One of '0', '1' or '2'. '0' indicates that the first base of the feature is the first base of a codon, '1' that the second base is the first base of a codon, and so on..&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;li&gt;&lt;p&gt;attributes - A semicolon-separated list of tag-value pairs, providing additional information about each feature. Some of these tags are predefined, e.g. ID, Name, Alias, Parent - see the GFF documentation for more details.&lt;/p&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1855" LastEditorUserId="1855" LastEditDate="2018-12-11T03:17:25.497" LastActivityDate="2018-12-13T10:59:15.307" Title="Derive a GTF containing protein coding genes from a GTF file with Exons and CDS" Tags="|scrnaseq|sequence-annotation|single-cell|gtf|10x-genomics|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5622" PostTypeId="2" ParentId="5605" CreationDate="2018-12-10T11:50:28.073" Score="2" Body="&lt;p&gt;Big O notation means &quot;of the order of&quot;. Thus &lt;span class=&quot;math-container&quot;&gt;$\mathcal{O}(f^{-2} \log{n})$&lt;/span&gt; means that the number of trees required goes with &lt;span class=&quot;math-container&quot;&gt;$\log{n}$&lt;/span&gt; for number of species (i.e. if you square the number of genes, you should double the number of trees) and with the inverse square of the shortest branch length (i.e. if you double the shortest branch length, you quarter the number of trees needed). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Big O notation generally allows to to compare two input sizes, but cannot really tell you anything about a single point. I.e. it is hard (impossible?) to tell from this function whether you have sufficient loci. &lt;/p&gt;&#xA;" OwnerUserId="235" LastEditorUserId="29" LastEditDate="2018-12-10T17:48:11.837" LastActivityDate="2018-12-10T17:48:11.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5623" PostTypeId="2" ParentId="5606" CreationDate="2018-12-10T14:26:20.953" Score="2" Body="&lt;p&gt;using a streaming process with &lt;a href=&quot;http://lindenb.github.io/jvarkit/BioAlcidaeJdk.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioalcidaejdk&lt;/a&gt; rather than running many random-access jumps. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The script: &lt;strong&gt;the.code&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;final int WINDOW_SIZE=500;&#xA;&#xA;// map position to count number of reads&#xA;final Map&amp;lt;Integer,Long&amp;gt; pos2count = new TreeMap&amp;lt;&amp;gt;();&#xA;String currChrom=null;&#xA;&#xA;for(;;)&#xA;    {&#xA;    // get next read in stream&#xA;    final SAMRecord rec = iter.hasNext()?iter.next():null;&#xA;    //ignore 'bad' reads&#xA;    if(rec!=null ) {&#xA;        if(rec.getReadUnmappedFlag()) continue;&#xA;        if(rec.getDuplicateReadFlag()) continue;&#xA;        if(rec.isSecondaryOrSupplementary()) continue;&#xA;        }&#xA;    // loop over the windows before the current &#xA;    final Iterator&amp;lt;Integer&amp;gt; posIter = pos2count.keySet().iterator();&#xA;    while(posIter.hasNext())&#xA;        {&#xA;        final int start = posIter.next();&#xA;        final int end = start + WINDOW_SIZE;&#xA;        if(rec!=null &amp;amp;&amp;amp; rec.getContig().equals(currChrom) &amp;amp;&amp;amp; end &amp;gt;= rec.getStart()) break;&#xA;        final long count = pos2count.get(start);&#xA;        out.println(currChrom+&quot;\t&quot;+start+&quot;\t&quot;+end+&quot;\t&quot;+count);&#xA;        //remove this position from pos2count&#xA;        posIter.remove();&#xA;        }&#xA;    //no more read, exit&#xA;    if(rec==null) break;&#xA;&#xA;    //retrieve all the windows for the current read&#xA;    for(Integer pos : IntStream.rangeClosed(rec.getStart(), rec.getEnd()).&#xA;            mapToObj(POS-&amp;gt; WINDOW_SIZE*(int)(POS/(1.0*WINDOW_SIZE))).&#xA;            collect(Collectors.toSet()))&#xA;        {&#xA;        //increment the count of reads&#xA;        final Long count = pos2count.get(pos);&#xA;        pos2count.put(pos,1L + (count==null?0L:count));&#xA;        }&#xA;&#xA;&#xA;    currChrom = rec.getContig();&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;execute:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$java -jar dist/bioalcidaejdk.jar -f the.code  src/test/resources/S1.bam&#xA;&#xA;RF01    0   500 38&#xA;RF01    500 1000    61&#xA;RF01    1000    1500    69&#xA;RF01    1500    2000    80&#xA;RF01    2000    2500    73&#xA;RF01    2500    3000    56&#xA;RF01    3000    3500    20&#xA;RF02    0   500 39&#xA;RF02    500 1000    76&#xA;RF02    1000    1500    73&#xA;RF02    1500    2000    72&#xA;RF02    2000    2500    59&#xA;RF02    2500    3000    13&#xA;RF03    0   500 34&#xA;RF03    500 1000    74&#xA;RF03    1000    1500    85&#xA;RF03    1500    2000    67&#xA;RF03    2000    2500    51&#xA;RF03    2500    3000    7&#xA;RF04    0   500 31&#xA;RF04    500 1000    68&#xA;RF04    1000    1500    90&#xA;RF04    1500    2000    75&#xA;RF04    2000    2500    29&#xA;RF05    0   500 47&#xA;RF05    500 1000    100&#xA;RF05    1000    1500    53&#xA;RF05    1500    2000    5&#xA;RF06    0   500 49&#xA;RF06    500 1000    89&#xA;RF06    1000    1500    29&#xA;RF07    0   500 58&#xA;RF07    500 1000    65&#xA;RF07    1000    1500    7&#xA;RF08    0   500 58&#xA;RF08    500 1000    69&#xA;RF08    1000    1500    3&#xA;RF09    0   500 58&#xA;RF09    500 1000    69&#xA;RF09    1000    1500    2&#xA;RF10    0   500 53&#xA;RF10    500 1000    39&#xA;RF11    0   500 50&#xA;RF11    500 1000    32&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastActivityDate="2018-12-10T14:26:20.953" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5624" PostTypeId="1" CreationDate="2018-12-10T14:59:30.803" Score="1" ViewCount="76" Body="&lt;p&gt;I'm new to using PLINK and I have 2 basic questions about clumping. I'm computing polygenic risk scores for psychiatric diseases and will be using the following command to clump:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plink --noweb --bfile --clump hapmap --clump inputfile --clump-field P --clump-p1 --clump-p2 --clump-r2 --clump-kb&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My first question: What type of file should I be using for the reference? Is it a map file?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My second question: How do I choose the most appropriate reference file?&lt;/p&gt;&#xA;" OwnerUserId="3821" LastEditorUserId="964" LastEditDate="2018-12-10T15:48:03.857" LastActivityDate="2018-12-10T15:48:03.857" Title="What should I use as a reference file for clumping?" Tags="|plink|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5625" PostTypeId="1" AcceptedAnswerId="5641" CreationDate="2018-12-10T15:22:42.360" Score="4" ViewCount="3861" Body="&lt;p&gt;Back again with another snakemake query. This time I decided to port my read cleaning and alignment pipeline to snakemake. Repeating the steps from previous question and trying not to make any typo here, I'm now receiving this &quot;error&quot; while trying a dry run using &lt;code&gt;snakemake snakemake -nr -s alignMake&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Building DAG of jobs...&#xA;MissingRuleException:&#xA;No rule to produce snakemake (if you use input functions make sure that they don't raise unexpected exceptions).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Below is my snakefile: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;shell.executable(&quot;/bin/bash&quot;)&#xA;&#xA;workdir: '/proj/uppstore2018136/publication/jupyterNotebooks'&#xA;&#xA;pathToPEReads = 'raw_data/reads/BL6/totalRNA'&#xA;pathToSMReads = 'raw_data/reads/BL6/smallRNA'&#xA;&#xA;pathToHISAT2index = 'indexes/hisat2/reference'&#xA;pathToBOWTIE2index = 'indexes/bowtie2/reference'&#xA;&#xA;pathToTemp = 'temp/'&#xA;pathToBAM = 'processed_data/BAM'&#xA;&#xA;readsPE, = glob_wildcards('raw_data/reads/BL6/totalRNA/{sample}_1.fastq.gz')&#xA;readsSE, = glob_wildcards('raw_data/reads/BL6/smallRNA/{sample}.fq.gz')&#xA;&#xA;samToolsThreads = 6&#xA;&#xA;rule all:&#xA;    input:&#xA;        expand('processed_data/BAM/{sample}.bam', sample=readsPE),&#xA;        expand('processed_data/BAM/{sample}.bam', sample=readsSE)&#xA;&#xA;rule trimgalorePE:&#xA;    input:&#xA;        fastq1 = pathToPEReads + '{sample}_1.fastq.gz',&#xA;        fastq2 = pathToPEReads + '{sample}_2.fastq.gz'&#xA;    output:&#xA;        output1 = pathToTemp + '{sample}_1_val_1.fq.gz',&#xA;        output2 = pathToTemp + '{sample}_2_val_2.fq.gz'&#xA;    shell:&#xA;        '''&#xA;        module load bioinfo-tools&#xA;        module load cutadapt&#xA;        module load TrimGalore&#xA;        trim_galore --quality 20 --paired --length 80 --illumina --stringency 2 --clip_R1 11 --clip_R2 11 -o {pathToTemp} --paired {input.fastq1} {input.fastq2}&#xA;        '''&#xA;&#xA;rule trimgaloreSEandDust:&#xA;    input:&#xA;        fastq = pathToSMReads + '{sample}.fq.gz',&#xA;    output:&#xA;        output = pathToTemp + '{sample}_trimmed.fq.gz',&#xA;    shell:&#xA;        '''&#xA;        module load bioinfo-tools&#xA;        module load cutadapt&#xA;        module load TrimGalore&#xA;        trim_galore --quality 20 --length 16 --max_length 32 --small_rna --fastqc --stringency 2 -o {pathToTemp} {input.fastq}&#xA;        '''&#xA;&#xA;rule align_hisat2:&#xA;    input:&#xA;        fastq1 = pathToTemp + '{sample}_1_val_1.fq.gz',&#xA;        fastq2 = pathToTemp + '{sample}_2_val_2.fq.gz'&#xA;    output: &#xA;        output = 'processed_data/BAM/{sample}.bam'&#xA;    threads: 10&#xA;    shell:&#xA;        '''&#xA;        module load bioinfo-tools&#xA;        module load HISAT2&#xA;        module load samtools&#xA;        hisat2 -q -p {threads} --no-discordant --no-mixed --rna-strandness RF --dta -x {pathToHISAT2index} -1 {input.fastq1} -2 {input.fastq2} | samtools sort -@ {samToolsThreads} -o {output}&#xA;        '''&#xA;&#xA;rule align_bowtie2:&#xA;    input:&#xA;        fastq = pathToTemp + '{sample}_trimmed.fq.gz'&#xA;    output: &#xA;        output = 'processed_data/BAM/{sample}.bam'&#xA;    threads: 10&#xA;    shell:&#xA;        '''&#xA;        module load bioinfo-tools&#xA;        module load bowtie2&#xA;        module load samtools&#xA;        bowtie2 --no-unal --end-to-end -D 20 -R 3 -N 1 -L 20 -i S,1,0.50 -p {threads} -x {pathToBOWTIE2index} -U {fastq} | samtools sort -@ {samToolsThreads} -o {output}&#xA;        '''&#xA;&#xA;rule clean:&#xA;    shell:&#xA;        '''&#xA;        rm temp/*&#xA;        '''&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I believe it is due snakemake being unable to find the first rule ? Maybe there is something wrong with parsing ? I tested the &lt;code&gt;glob_wildcards&lt;/code&gt; and &lt;code&gt;expand&lt;/code&gt; function independently and they appear to work properly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Here is the directory tree:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;.&#xA;├── citations&#xA;├── doc&#xA;├── genome&#xA;│   ├── genome.fa&#xA;│   └── Mus_musculus.GRCm38.94.gtf&#xA;├── indexes&#xA;│   ├── bowtie2&#xA;│   │   ├── reference.1.bt2&#xA;│   │   ├── reference.2.bt2&#xA;│   │   ├── reference.3.bt2&#xA;│   │   ├── reference.4.bt2&#xA;│   │   ├── reference.rev.1.bt2&#xA;│   │   └── reference.rev.2.bt2&#xA;│   └── hisat2&#xA;│       ├── reference.1.ht2&#xA;│       ├── reference.2.ht2&#xA;│       ├── reference.3.ht2&#xA;│       ├── reference.4.ht2&#xA;│       ├── reference.5.ht2&#xA;│       ├── reference.6.ht2&#xA;│       ├── reference.7.ht2&#xA;│       └── reference.8.ht2&#xA;├── LICENSE&#xA;├── notebooks&#xA;├── plots&#xA;├── processed_data&#xA;│   └── BAM&#xA;├── raw_data&#xA;│   └── reads&#xA;│       └── BL6&#xA;│           ├── smallRNA&#xA;│           │   ├── smBL6_1.fq.gz&#xA;│           │   ├── smBL6_2.fq.gz&#xA;│           │   └── smBL6_3.fq.gz&#xA;│           └── totalRNA&#xA;│               ├── EKIxxxx012_1.fastq.gz&#xA;│               ├── EKIxxxx012_2.fastq.gz&#xA;│               ├── EKIxxxx013_1.fastq.gz&#xA;│               ├── EKIxxxx013_2.fastq.gz&#xA;│               ├── EKIxxxx014_1.fastq.gz&#xA;│               ├── EKIxxxx014_2.fastq.gz&#xA;│               ├── ESSxxxx077_1.fastq.gz&#xA;│               └── ESSxxxx077_2.fastq.gz&#xA;├── README.md&#xA;├── recipes&#xA;│   ├── alignMake&#xA;│   ├── cluster.json&#xA;│   ├── indexMake&#xA;│   ├── indexMakeScript.out&#xA;│   └── indexMakeScript.sh&#xA;├── results&#xA;├── scripts&#xA;├── slurmLogs&#xA;│   ├── slurm-6575541.out&#xA;│   ├── slurm-6575543.out&#xA;│   ├── slurm-6575815.out&#xA;│   └── slurm-6580867.out&#xA;├── source&#xA;└── temp&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1135" LastEditorUserId="3831" LastEditDate="2018-12-12T09:31:20.110" LastActivityDate="2018-12-12T09:31:20.110" Title="Snakemake: Cannot find first rule?" Tags="|python|snakemake|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5626" PostTypeId="1" CreationDate="2018-11-07T18:19:49.010" Score="1" ViewCount="150" Body="&lt;p&gt;I'm taking a genomics course at school, and we have a final project due in a few weeks. &#xA;Now, I can write bash scripts to sort and analyze the data, etc. However, I'm not quite sure how to actually &lt;em&gt;get&lt;/em&gt; the data from the database. I was hoping to get some help there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to get the assemblies of genomes for the species in the &lt;em&gt;Borrelia&lt;/em&gt; genus. Is there any way that I can do this using commands like &quot;wget&quot;, for example. I'm not even sure what URL I'd have to give the computer to go fetch these. Then once I give it the correct URL, how should I tell it what file in that directory to download?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd greatly appreciate some help on this issue.&lt;/p&gt;&#xA;" OwnerDisplayName="The_Questioner" LastActivityDate="2023-11-03T20:10:25.067" Title="Genomics: Downloading genomic data from the NCBI Genbank database" Tags="|genomics|phylogenetics|genbank|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5627" PostTypeId="2" ParentId="5626" CreationDate="2018-12-10T11:23:50.833" Score="3" Body="&lt;p&gt;For downloading data from commandline use the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK25500/&quot; rel=&quot;nofollow noreferrer&quot;&gt;E-utilities&lt;/a&gt; of NCBI.&lt;/p&gt;&#xA;" OwnerUserId="345" OwnerDisplayName="WYSIWYG" LastActivityDate="2018-12-10T11:23:50.833" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5629" PostTypeId="1" AcceptedAnswerId="5644" CreationDate="2018-12-10T17:44:55.417" Score="4" ViewCount="800" Body="&lt;p&gt;Im trying to plot reactome pathway in for of circular bar plot using ggplot2 &lt;/p&gt;&#xA;&#xA;&lt;p&gt;my data&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    dput(df2)&#xA;structure(list(ID = 1:10, Pathway_names = structure(c(5L, 1L, &#xA;7L, 6L, 2L, 3L, 9L, 8L, 10L, 4L), .Label = c(&quot;Antigen Presentation: Folding assembly and peptide loading of class I MHC&quot;, &#xA;&quot;Antigen processing-Cross presentation&quot;, &quot;Class I MHC mediated antigen processing &amp;amp; presentation&quot;, &#xA;&quot;Cytokine Signaling in Immune system&quot;, &quot;Endosomal/Vacuolar pathway&quot;, &#xA;&quot;ER-Phagosome pathway&quot;, &quot;Immunoregulatory interactions between a Lymphoid and a non-Lymphoid cell&quot;, &#xA;&quot;Interferon alpha/beta signaling&quot;, &quot;Interferon gamma signaling&quot;, &#xA;&quot;Interferon Signaling&quot;), class = &quot;factor&quot;), Entities_found = structure(c(1L, &#xA;2L, 3L, 4L, 4L, 5L, 6L, 7L, 8L, 9L), .Label = c(&quot;51&quot;, &quot;54&quot;, &quot;55&quot;, &#xA;&quot;57&quot;, &quot;75&quot;, &quot;77&quot;, &quot;96&quot;, &quot;117&quot;, &quot;176&quot;), class = &quot;factor&quot;), Entities_pValue = c(1.11e-16, &#xA;1.11e-16, 0.003908431, 2.54e-12, 2.77e-10, 0.0053068, 1.64e-13, &#xA;1.11e-16, 1.11e-16, 3.68e-06)), row.names = c(NA, -10L), class = &quot;data.frame&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;my code &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(df2, aes(x = factor(Pathway_names), y = Entities_found)) + ######################################&#xA;  geom_bar(aes(fill = Entities_pValue), stat = &quot;identity&quot;, position = &quot;stack&quot;) + &#xA;  #scale_fill_manual(values = c(blank = &quot;white&quot;, dark = &quot;black&quot;)) + &#xA;  coord_polar() + &#xA;  theme_minimal() + &#xA;  guides(fill = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ohhY1.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ohhY1.jpg&quot; alt=&quot;my output &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far im not sure how to include both p value and entities in the plot so that the pathway that are coming up can be labeled accordingly &lt;/p&gt;&#xA;&#xA;&lt;p&gt;New figure &lt;a href=&quot;https://i.stack.imgur.com/pZjL9.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pZjL9.jpg&quot; alt=&quot;updated figure&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="939" LastEditDate="2018-12-11T10:51:40.047" LastActivityDate="2018-12-11T14:36:02.527" Title="How to plot p-values in a circular barplot?" Tags="|r|visualization|ggplot2|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="5630" PostTypeId="1" AcceptedAnswerId="5633" CreationDate="2018-12-10T19:37:16.047" Score="5" ViewCount="182" Body="&lt;h2&gt;Context&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I am interested in finding all of the promotors specific to a particular sigma factor. I have identified the -35 and -10 sites from the literature, bold denotes -10, -35, binding sites: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  -35 site               -10 site&#xA;5' TTTACAtatttatttcagacaacGTCTTT 3'&#xA;   ^^^^^^                 ^^^^^^&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;CAPITALIZED&lt;/code&gt; and &lt;code&gt;^emphasized^&lt;/code&gt; 5' and 3' sequence components are the conserved sequence that must have an exact match. The lower-case bases are the variable 10-25bp region for which the sequence does not matter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sigma factor is from &lt;em&gt;C. difficile&lt;/em&gt; and I am particularly interested in searching &lt;em&gt;C. acetobutylicum&lt;/em&gt;, however, it would also be interesting to see where else this sequence crops up. Obviously such a short sequence will arise often by random chance, so it is important to define the spacer region, which might be anywhere between 15-25 nucleotides and may be any sequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, I would like to find these binding regions from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NC_003030.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this sequence&lt;/a&gt; and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NC_001988.2&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Desired output&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;A text file that contains the locations and sequences of all of the variable-width sequences as defined above. Something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seq   start  stop  gap  dir  seq&#xA;seq1  1      22    10   F    TTTACAagtcagatctGTCTTT&#xA;seq2  2      23    11   F    TTTACAagtcagactctGTCTTT&#xA;seq2  122    101   10   R    AAAGACatgctagccaTGTAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="3823" LastEditorUserId="2085" LastEditDate="2018-12-11T18:21:09.360" LastActivityDate="2018-12-17T20:45:55.277" Title="How to find all variable-length seqs with an exact 5' and 3' match in a FASTA file" Tags="|fasta|python|genomics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5631" PostTypeId="1" AcceptedAnswerId="5639" CreationDate="2018-12-10T21:49:46.170" Score="3" ViewCount="269" Body="&lt;p&gt;Hello bioinformaticians,&lt;br&gt;&#xA;I've recently downloaded few RNAseq data files from &lt;a href=&quot;https://portal.gdc.cancer.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genomic Data Commons Data&lt;/a&gt; (GDC) portal. These files belong to Broad institutes &lt;a href=&quot;https://portals.broadinstitute.org/ccle&quot; rel=&quot;nofollow noreferrer&quot;&gt;CCLE project&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now the problem is that GDC portal do not provide CCLE cell line names:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zGFXR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zGFXR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is how could I link the GDC ids (prefixes of File Names) to CCLE cell line names? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would imagine because it is an important step in the analysis, there should be some way to do that.&lt;/p&gt;&#xA;" OwnerUserId="274" LastEditorUserId="274" LastEditDate="2018-12-11T11:32:57.133" LastActivityDate="2018-12-19T04:03:11.060" Title="How to link GDC ids to CCLE cell line names?" Tags="|rna-seq|data-download|cell-line|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5633" PostTypeId="2" ParentId="5630" CreationDate="2018-12-11T06:54:43.860" Score="2" Body="&lt;h2&gt;python 3 solution&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This python program will get you what I think you want. It is not elegant given that it (A) stores the whole sequence in memory and (B) spends most of its time slicing strings rather than working with a buffer. This program is not recommended for FASTA files for which the largest sequence will not fit well in RAM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All that you need to do to modify it for other genomes or other binding sites is to change the three lines that begin with &lt;code&gt;forseq&lt;/code&gt;, &lt;code&gt;revseq&lt;/code&gt;, and &lt;code&gt;filename&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio import SeqIO&#xA;from Bio.Seq import Seq&#xA;&#xA;forseq = Seq(&quot;tttaca&quot;.upper())&#xA;revseq = Seq(&quot;gtcttt&quot;.upper())&#xA;filename = &quot;NC_003030.fasta&quot;&#xA;&#xA;forcom = forseq.reverse_complement()&#xA;revcom = revseq.reverse_complement()&#xA;print(&quot;seq\tstart\tstop\tgap\tdir\tseq&quot;)&#xA;for record in SeqIO.parse(filename, &quot;fasta&quot;):&#xA;    for i in range(len(record.seq)):&#xA;        for v1, v2, direc in [ (forseq, revseq, &quot;F&quot;), (revcom, forcom, &quot;R&quot;) ]:&#xA;            if record.seq[i:i+len(v1)] == v1:&#xA;                for j in range(10,25):&#xA;                    if record.seq[i+len(v1)+j:i+len(v2)+j+len(v2)] == v2:&#xA;                        print(&quot;{}\t{}\t{}\t{}\t{}\t{}&quot;.format(&#xA;                            record.id, i,&#xA;                            i + len(v1)+j+len(v2)-1,&#xA;                            j, direc,&#xA;                            record.seq[i: i+len(v1) + j + len(v2)] ))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here are the first few lines of example output from the first fasta file that you linked.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seq          start    stop     gap  dir  seq&#xA;NC_003030.1  97512    97543    20   F    TTTACAAAAGAAAAGATATGCATAATGTCTTT&#xA;NC_003030.1  283173   283205   21   F    TTTACACTTATGTTTATATAGTTCAACGTCTTT&#xA;NC_003030.1  303104   303125   10   R    AAAGACGCAAGCATAATGTAAA&#xA;NC_003030.1  1127691  1127725  23   F    TTTACAAGGTGTTACTGATAGAACAAAGGGTCTTT&#xA;NC_003030.1  1301958  1301988  19   F    TTTACATAAAAAGATGATAGTTTTTGTCTTT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;python 3 with redundancy&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If you need sequence redundancy, this version works. &lt;a href=&quot;https://reverse-complement.com/ambiguity.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;I used the redundancy codes available from www.reverse-complement.com&lt;/a&gt; You may use the redundant characters in [ R, Y, S, W, K, M, B, V, D, H, N ] or lowercase variants. I'm not sure how to handle the '-' character at the moment but am open to suggestions. Code and results below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, the algorithm isn't great given all the for loops so other answers are appreciated!&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio.Seq import Seq&#xA;&#xA;forseq = &quot;tttaca&quot; # Just use python strings here. No Seq() as above&#xA;revseq = &quot;ntnntn&quot;&#xA;filename = &quot;NC_003030.fasta&quot;&#xA;&#xA;revcomp = {'A':'T', 'C':'G', 'G':'C', 'T':'A',&#xA;           'R':'Y', 'Y':'R', 'S':'S', 'W':'W',&#xA;           'K':'M', 'M':'K', 'B':'V', 'V':'B',&#xA;           'D':'H', 'H':'D', 'N':'N'}&#xA;redund = {'A':['A'], 'C':['C'], 'G':['G'], 'T':['T'],&#xA;          'R':['A','G'], 'Y':['C','T'],&#xA;          'S':['C','G'], 'W':['A','T'],&#xA;          'K':['G','T'], 'M':['A','C'],&#xA;          'B':['C','G','T'], 'V':['A','C','G'],&#xA;          'D':['A','G','T'], 'H':['A','C','T'],&#xA;          'N':['A','C','G','T']}&#xA;&#xA;def revcom(seq):&#xA;    return &quot;&quot;.join(revcomp.get(base, base) for base in reversed(seq))&#xA;&#xA;def seqmatch(query, reference):&#xA;    &quot;&quot;&quot;Checks if the query sequence (perhaps a promoter with redundant bases)&#xA;    matches the reference. Both query and reference must be the same length.&#xA;&#xA;    Returns true if there is a match (redundancy allowed), returns false&#xA;    otherwise.&#xA;    &quot;&quot;&quot;&#xA;    assert len(query) == len(reference)&#xA;    for i in range(len(query)):&#xA;        if reference[i] in redund[query[i]]:&#xA;            pass&#xA;        else:&#xA;            return False&#xA;    return True&#xA;&#xA;forseq = forseq.upper()&#xA;revseq = revseq.upper()&#xA;forcom = revcom(forseq)&#xA;revcom = revcom(revseq)&#xA;print(&quot;seq\tstart\tstop\tgap\tdir\tseq&quot;)&#xA;for record in SeqIO.parse(filename, &quot;fasta&quot;):&#xA;    for i in range(len(record.seq)):&#xA;        for v1, v2, direc in [ (forseq, revseq, &quot;F&quot;), (revcom, forcom, &quot;R&quot;) ]:&#xA;            if seqmatch( v1, record.seq[i:i+len(v1)] ):&#xA;                for j in range(10,25):&#xA;                    if seqmatch( v2, record.seq[i+len(v1)+j:i+len(v2)+j+len(v2)] ):&#xA;                        print(&quot;{}\t{}\t{}\t{}\t{}\t{}&quot;.format(&#xA;                            record.id, i,&#xA;                            i + len(v1)+j+len(v2)-1,&#xA;                            j, direc,&#xA;                            record.seq[i: i+len(v1) + j + len(v2)] ))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is some sample output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seq          start  stop  gap  dir  seq&#xA;NC_003030.1  733    767   23   R    GATGAAGATCAAGAAACCGATACAAACAATGTAAA&#xA;NC_003030.1  736    767   20   R    GAAGATCAAGAAACCGATACAAACAATGTAAA&#xA;NC_003030.1  739    767   17   R    GATCAAGAAACCGATACAAACAATGTAAA&#xA;NC_003030.1  742    767   14   R    CAAGAAACCGATACAAACAATGTAAA&#xA;NC_003030.1  743    767   13   R    AAGAAACCGATACAAACAATGTAAA&#xA;NC_003030.1  2665   2697  21   R    TATTAAAATTTTAGAGGATGATGGTGATGTAAA&#xA;NC_003030.1  2668   2697  18   R    TAAAATTTTAGAGGATGATGGTGATGTAAA&#xA;NC_003030.1  3798   3822  13   R    CACCAGAAGACTTAAAAATTGTAAA&#xA;NC_003030.1  3801   3822  10   R    CAGAAGACTTAAAAATTGTAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-12-17T20:45:55.277" LastActivityDate="2018-12-17T20:45:55.277" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="5634" PostTypeId="2" ParentId="5621" CreationDate="2018-12-11T10:23:37.900" Score="4" Body="&lt;p&gt;In your case, I would definitely suggest following @Emily_Ensembl's advice and using the Arabidopsis GTF from Ensembl. But for future reference, if an Ensembl GTF wasn't available, you could build something like this using the &lt;code&gt;gtf&lt;/code&gt; class from &lt;code&gt;cgat&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cgat module and dependancies can be installed by following the instructions &lt;a href=&quot;https://github.com/cgat-developers/cgat-apps&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Specifically, you can download and run their installation script. This will set up a dedicated conda environment to run the versions of dependancies needed. Further documentation can be found &lt;a href=&quot;https://www.cgat.org/downloads/public/cgat/documentation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# download installation script:&#xA;curl -O https://raw.githubusercontent.com/cgat-developers/cgat-apps/master/install.sh&#xA;&#xA;# install the development version (recommended, no production version yet):&#xA;bash install.sh --devel&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This requires python3 and anaconda to install. You may need to run this script in the conda environment:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# show available environments&#xA;conda info --envs&#xA;# activate by path&#xA;source activate /home/user/local/bin/conda-install/envs/cgat-a&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then run the follow as a python3 script. Copy the contents in a file such as &lt;code&gt;convert_gtf.py&lt;/code&gt;. Then run it in the terminal on your gtf file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python convert_gtf.py genes.gtf &amp;gt; genes_new.gtf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;#python 3.6.4&#xA;&#xA;import sys&#xA;&#xA;from cgat import GTF&#xA;from cgatcore import iotools&#xA;&#xA;for gene in GTF.flat_gene_iterator(&#xA;     GTF.iterator(iotools.open_file(sys.argv[1])), strict = False):&#xA;    gene_start = min (e.start for e in gene)&#xA;    gene_end = max(e.end for e in gene)&#xA;&#xA;    if any(e.feature == &quot;CDS&quot; for e in gene):&#xA;        gene_type = &quot;protein_coding&quot;&#xA;    else:&#xA;        gene_type = &quot;non_coding&quot;&#xA;&#xA;    for exon in gene:&#xA;        exon_line.setAttribute(&quot;gene_biotype&quot;, gene_type)&#xA;&#xA;    gene_line = GTF.Entry().fromGTF(gene[0])&#xA;    gene_line.feature = &quot;gene&quot;&#xA;    gene_line.start = gene_start&#xA;    gene_line.end = gene_end&#xA;&#xA;    # Its not clear what the transcript_id for a gene line is.&#xA;    # Technically speaking, Ensembl GTFs do not follow the GTF&#xA;    # standard as GTF entries must include transcript_ids, but&#xA;    # the ensembl ones don't.&#xA;&#xA;    if hasattr(gene_line, &quot;gene_name&quot;):&#xA;        gene_line.attributes[&quot;gene_name&quot;] = gene[0].gene_name&#xA;    else:&#xA;        gene_line.attributes[&quot;gene_name&quot;] = gene[0].gene_id&#xA;&#xA;    gene_line.attributes[&quot;gene_biotype&quot;] = gene_type&#xA;&#xA;    print (str(gene_line)+&quot;\n&quot;)&#xA;    for exon in gene:&#xA;        print(str(exon_line)+&quot;\n&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="235" LastEditorUserId="235" LastEditDate="2018-12-13T10:59:15.307" LastActivityDate="2018-12-13T10:59:15.307" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5635" PostTypeId="1" AcceptedAnswerId="5640" CreationDate="2018-12-11T10:42:33.140" Score="3" ViewCount="1915" Body="&lt;p&gt;Hi im doing Variant calling on fastq files for which i have 4000 fq files and the variant calling are done in different 9 steps. Each step generate different files that are output to next steps.&lt;/p&gt;&#xA;&lt;p&gt;So can anybody help me with this to do it in a loop it pick up names from previous steps for all files. Step 1 command is here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bwa mem -M -R '@RG\tID:ERR040140\tLB:2517707\tPL:ILLUMINA\tPM:HiSeq2000\tSM:ERR040140' ../ref/M._tuberculosis_H37Rv_2015-11-13.fasta ERR040140_* &amp;gt; ERR040140.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So i made an excel file to break the command in columns so that i can change the file names keeping remaining command same just like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uj6vl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uj6vl.png&quot; alt=&quot;command_column_wise&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So changes are occurring 5th, 7th and 9th column. which are:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tID:ERR040140&#xA;tLB:2517707&#xA;tSM:ERR040140&#xA;ERR040140_*&#xA;ERR040140.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can anyone help me with loop or something?&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="3686" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2019-01-05T01:57:30.633" Title="How to run same command on multiple files?" Tags="|variant-calling|linux|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="5636" PostTypeId="2" ParentId="5635" CreationDate="2018-12-11T10:51:24.073" Score="5" Body="&lt;p&gt;if you have a repeating workflow, I strongly recommend to have a look at workflow management systems like &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/tutorial/basics.html&quot; rel=&quot;noreferrer&quot;&gt;snakemake&lt;/a&gt;. I've also wrote a little tutorial on &lt;a href=&quot;https://www.biostars.org/p/335903/&quot; rel=&quot;noreferrer&quot;&gt;biostars&lt;/a&gt; about this topic, which might be useful for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;fin swimmer&lt;/p&gt;&#xA;" OwnerUserId="1310" LastActivityDate="2018-12-11T10:51:24.073" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5637" PostTypeId="2" ParentId="5635" CreationDate="2018-12-11T12:29:47.003" Score="8" Body="&lt;p&gt;Something like this might do it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;# make an array to hold your list of samples/base filenames&#xA;samples=( S1 S2 S3 S4 S5 )&#xA;&#xA;# make a loop to go through the sample IDs&#xA;for sample in &quot;&lt;span class=&quot;math-container&quot;&gt;${samples[@]}&quot;&#xA;do&#xA;  :&#xA;  # do something with each sample/file, e.g. make a directory for it&#xA;  mkdir &quot;$&lt;/span&gt;{sample}&quot;_home&#xA;&#xA;  # do something else with the sample, e.g. unzip the fastq.gz file&#xA;  gunzip &quot;&lt;span class=&quot;math-container&quot;&gt;${sample}&quot;.fq.gz &amp;gt; &quot;$&lt;/span&gt;{sample}&quot;_home/&quot;${sample}&quot;.fq&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One disadvantage with this is that it will process the files one after the other.  If you have a cluster, you might prefer to run them all at the same time to make the whole thing quicker.  If you have a SLURM cluster, you can use job array to do something similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have your list of samples in a text file, you could try replacing the samples array command in the script above with something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samples=( `cat samples_list.txt` )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will only work if the samples are listed on a single line, space-separated. You may need to process your samples file further to get them in this format, or use the bash readarray command.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastEditorUserId="3828" LastEditDate="2019-01-04T16:39:30.057" LastActivityDate="2019-01-04T16:39:30.057" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="5638" PostTypeId="1" CreationDate="2018-12-11T12:50:30.673" Score="1" ViewCount="148" Body="&lt;p&gt;Looking at the sample &lt;a href=&quot;https://www.bioinformatics.babraham.ac.uk/projects/bismark/PE_report.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;report&lt;/a&gt; from &lt;a href=&quot;https://www.bioinformatics.babraham.ac.uk/projects/bismark/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bismark webpage&lt;/a&gt;, there are two sections: &lt;em&gt;Cytosine Methylation&lt;/em&gt; and &lt;em&gt;Cytosine Methylation after Extraction&lt;/em&gt;. However even after reading &lt;a href=&quot;https://www.bioinformatics.babraham.ac.uk/projects/bismark/Bismark_User_Guide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;User Guide&lt;/a&gt;, I still cannot understand the difference between these two sections.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I understand this, &lt;em&gt;Cytosine Methylation&lt;/em&gt; section shows summary statistics about Cytosine Methylation in sequenced data and &lt;em&gt;Cytosine Methylation after Exctraction&lt;/em&gt; shows summary statistics after a Methylation calling, but is not the Methylation calling already part of the first section?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question therefore is: What is the difference between these two sections of Bismark report?&lt;/p&gt;&#xA;" OwnerUserId="1199" LastActivityDate="2020-04-21T20:37:46.333" Title="Bismark report before and after Cytosine Methylation Extraction" Tags="|methylation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5639" PostTypeId="2" ParentId="5631" CreationDate="2018-12-11T13:05:42.493" Score="2" Body="&lt;p&gt;The filenames of those bams contain the cell line names. They are constructed:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SAMPLEID.CELL_LINE_NAME.REPLICATE.bam&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus the first 5 cell lines in your screen capture are SK-N-F1, TE-9, CJM, HCI-H1915 and CAL-33. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the information that is need is also present in the experimental metadata &lt;code&gt;xml&lt;/code&gt; file attached to each sample record. This is accessible by downloading it from the record on the website, or by querying the GDC metadata API (&lt;a href=&quot;https://gdc.cancer.gov/developers/gdc-application-programming-interface-api&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gdc.cancer.gov/developers/gdc-application-programming-interface-api&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="235" LastEditorUserId="235" LastEditDate="2018-12-17T10:22:14.323" LastActivityDate="2018-12-17T10:22:14.323" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="5640" PostTypeId="2" ParentId="5635" CreationDate="2018-12-11T13:08:13.020" Score="10" Body="&lt;p&gt;Another solution for repeating one command with different parameters is to use &lt;code&gt;parallel&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Create a tab-delimited file &lt;code&gt;samples.txt&lt;/code&gt;, which contains the sample names and the information for your read groups, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sample1 Lib1&#xA;Sample2 Lib2&#xA;Sample3 Lib3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you can use &lt;code&gt;parallel&lt;/code&gt; like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;parallel --colsep &quot;\t&quot; 'bwa mem -M -R &quot;@RG\tID:{1}\tLB:{2}\tPL:ILLUMINA\tPM:HiSeq2000\tSM:{1}&quot; ../ref/M._tuberculosis_H37Rv_2015-11-13.fasta {1}_* &amp;gt; {1}.sam' :::: samples.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For each line in &lt;code&gt;samples.txt&lt;/code&gt; a new process is started, replacing &lt;code&gt;{1}&lt;/code&gt; and &lt;code&gt;{2}&lt;/code&gt; with the values in column 1 and 2 of this line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;fin swimmer&lt;/p&gt;&#xA;" OwnerUserId="1310" LastActivityDate="2018-12-11T13:08:13.020" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5641" PostTypeId="2" ParentId="5625" CreationDate="2018-12-11T13:36:57.480" Score="9" Body="&lt;p&gt;you are starting &lt;code&gt;snakemake&lt;/code&gt; the wrong way. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;snakemake snakemake&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;means &quot;Running the pipeline in &lt;code&gt;Snakefile&lt;/code&gt; until you reach a rule named &lt;code&gt;snakemake&lt;/code&gt; or a file name &lt;code&gt;snakemake&lt;/code&gt; is created.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your snakefile is called &lt;code&gt;snakemake&lt;/code&gt; and this is the pipeline you want to start, the correct syntax would be: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;snakemake -s snakemake&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1310" LastEditorUserId="57" LastEditDate="2018-12-11T14:15:52.097" LastActivityDate="2018-12-11T14:15:52.097" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5642" PostTypeId="1" CreationDate="2018-12-11T13:45:40.567" Score="1" ViewCount="2044" Body="&lt;p&gt;I need some help with forgi library for visualisation of RNA secondary structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have an RNA sequence and dot-bracket notation of the secondary structure and I would like to plot 2D graph like that you can find &lt;a href=&quot;https://viennarna.github.io/forgi/graph_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input file is a &lt;code&gt;.fx&lt;/code&gt; file, or a fasta file where the sequence is single-line and the third line is the &lt;a href=&quot;http://eternawiki.org/wiki/index.php5/Dot-Bracket_Notation&quot; rel=&quot;nofollow noreferrer&quot;&gt;dot-bracket notation&lt;/a&gt; of the RNA structure.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1y26&#xA;CGCUUCAUAUAAUCCUAAUGAUAUGGUUUGGGAGUUUCUACCAAGAGCCUUAAACUCUUGAUUAUGAAGUG&#xA;((((((((((..(((((((.......)))))))......).((((((.......))))))..)))))))))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OlrLz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OlrLz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there any other possibilities to plot it in real-time (for instance, in jupyter notebook avoiding any crossing external scripts), I will be very thankful to learn them!&lt;/p&gt;&#xA;" OwnerUserId="3829" LastEditorUserId="2085" LastEditDate="2018-12-12T15:28:04.577" LastActivityDate="2022-06-28T23:41:48.237" Title="How to plot a 2D RNA structure in python using dot-bracket input?" Tags="|python|visualization|secondary-structure|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="5643" PostTypeId="2" ParentId="5216" CreationDate="2018-12-11T14:24:13.523" Score="2" Body="&lt;p&gt;While I wouldn't consider this as the best answer or solution - I decided to use Ensembl REST api to extract the coordinates of alignment blocks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The quick fix - in python looks like this: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#This is the main function which uses Ensembl REST API to access the Enesembl Compara database &#xA;#And retrive the alignment blocks which correspond to the MM10 standard. &#xA;#The function accepts chromosome, start and array and returns the alignment blocks&#xA;#reported by the API. Since the blocks can be on different chromosomes/and/or/contigs, it &#xA;#returns the whole output in form of dataframe - containing chromosome, start and stop of the&#xA;#alignment blocks corresponding to input and all the other alignment blocks corresponding the &#xA;#16 mice species, if available. &#xA;#!!!!!!!----------Caution start----------!!!!!!!&#xA;#The API is a bit buggy, so it is advised not to request for specific species.&#xA;#!!!!!!!-----------Caution end-----------!!!!!!!&#xA;def liftover(chrom, start, end):&#xA;    mmStart, mmEnd, mmchr = [], [], []&#xA;    caStart, caEnd, cachr = [], [], []&#xA;    #ryStart, ryEnd, rychr = [], [], []&#xA;    server = &quot;https://rest.ensembl.org&quot;&#xA;    ext = &quot;/alignment/region/mus_musculus/&quot; + str(chrom) + &quot;:&quot; + str(start) + &quot;-&quot; + str(end) + &quot;:1?species_set_group=murinae;method=CACTUS_HAL;content-type=application/json&quot;&#xA;    headers={ &quot;Content-Type&quot; : &quot;application/json&quot;}&#xA;    r = requests.get(server+ext, headers={ &quot;Content-Type&quot; : &quot;application/json&quot;})&#xA;    if(r.ok):&#xA;        json_string = json.dumps(r.json())&#xA;        jData = json.loads(r.content)&#xA;        results = r.json()&#xA;        print(&quot;The response contains {0} properties&quot;.format(len(jData)))&#xA;        print(&quot;\n&quot;)&#xA;    else:&#xA;        r.raise_for_status()&#xA;&#xA;    for element in jData:&#xA;        element.pop('tree', None)&#xA;&#xA;    for element in jData:&#xA;        for key, value in element.items():&#xA;            for z in value:&#xA;                if (z[&quot;species&quot;]==&quot;mus_musculus&quot;):&#xA;                    mmStart.append(z[&quot;start&quot;])&#xA;                    mmEnd.append(z[&quot;end&quot;])&#xA;                    mmchr.append(z[&quot;seq_region&quot;])&#xA;                if (z[&quot;species&quot;]==&quot;mus_musculus_casteij&quot;):&#xA;                    caStart.append(z[&quot;start&quot;])&#xA;                    caEnd.append(z[&quot;end&quot;])&#xA;                    cachr.append(z[&quot;seq_region&quot;])&#xA;    if (len(mmStart) != len(caStart)):&#xA;        print(&quot;Length mismatch between species blocks - proceed with caution!&quot;)&#xA;    return [min(mmStart), max(mmEnd), min(caStart), max(caEnd), len(mmStart)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Again, it depends on how alignment blocks are reported - and requires further digging to resolve which mm10 alignment block resembles CAST alignment block.&lt;/p&gt;&#xA;" OwnerUserId="1135" LastActivityDate="2018-12-11T14:24:13.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5644" PostTypeId="2" ParentId="5629" CreationDate="2018-12-11T14:36:02.527" Score="3" Body="&lt;p&gt;The &lt;code&gt;adjustcolor&lt;/code&gt; function from &lt;code&gt;grDevices&lt;/code&gt; can be used to generate colours with transparency. What you want is colours based on the pathway and transparency to form a colour gradient for the p-value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;adjustcolor( &quot;red&quot;, alpha.f = 0.2)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the ggplot example, the &lt;code&gt;aes&lt;/code&gt; function has an &lt;code&gt;alpha&lt;/code&gt; parameter.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(df2, aes(x = factor(Pathway_names), y = Entities_found), alpha = Entities_pValue) + ######################################&#xA;geom_bar(aes(fill = Entities_found, alpha = Entities_pValue), stat = &quot;identity&quot;, position = &quot;stack&quot;) +  &#xA;coord_polar() + &#xA;theme_minimal() + &#xA; guides(fill = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1855" LastActivityDate="2018-12-11T14:36:02.527" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5645" PostTypeId="2" ParentId="5616" CreationDate="2018-12-11T19:53:27.250" Score="2" Body="&lt;p&gt;I would use &lt;code&gt;BCBio&lt;/code&gt; for gff handling as it is written to directly interface with BioPython’s object model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only downside is that I believe it is no longer actively supported. It is however the package that the &lt;a href=&quot;https://biopython.org/wiki/GFF_Parsing&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioPython docs&lt;/a&gt; use generally. There are plans to properly incorporate GFF/GTF parsers in to BP in the not too distant future according to that link though.&lt;/p&gt;&#xA;" OwnerUserId="929" LastActivityDate="2018-12-11T19:53:27.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5646" PostTypeId="1" AcceptedAnswerId="5647" CreationDate="2018-12-11T21:09:22.263" Score="4" ViewCount="438" Body="&lt;p&gt;I have some sequencing data from a captured region that is a known paralog &lt;strong&gt;edited&lt;/strong&gt;. For now, I have been mapping the data using standard minimap2 flags for PacBio DNA sequencing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 -ax map-pb GRCh37.fa fastqfile.fastq &amp;gt; samfile.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I captured a 10kb region and what I am seeing in IGV is puzzling. The region directly before and directly after the paralogues are covered at about 1000X, but the genes themselves are not covered at all:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ecXKR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ecXKR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two genes in the middle are the paralogues. I left them off to protect data. This data is also only the sorted data, the duplicate marked data has lower coverage (600X). &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Question 1&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I was wondering if there is a different approach to mapping I could take that would output &lt;strong&gt;all&lt;/strong&gt; alignments for a read, and then specifically only multi-mapped reads?&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Question 2&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Is this process normally accomplished by minimap2, or are other aligners more effective for paralogous mapping?&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Question 3&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The interesting thing is, while my average read length was 6kb, I am seeing the reads here are about 4kb. I have another region with very high coverage where the reads spanned both paralogues and mapping was successful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question: &lt;strong&gt;Why am I only seeing reads that are about 4kb long when the average read length for this dataset is 6kb?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help on this would be great! &#xA;Thanks and I hope I have been clear on my question,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dennis&lt;/p&gt;&#xA;" OwnerUserId="3669" LastEditorUserId="2085" LastEditDate="2018-12-11T22:27:01.170" LastActivityDate="2018-12-14T23:34:41.653" Title="Mapping Reads to Known Gene Paralogs with Long Read Technology" Tags="|long-reads|pacbio|mappability|minimap2|paralogs|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5647" PostTypeId="2" ParentId="5646" CreationDate="2018-12-11T22:08:14.657" Score="3" Body="&lt;h3&gt;Question 1&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;This command will get you all of supplementary alignments for the reads. This isn't exactly what you want though. You want all of the reads that have more than one mapping.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -f 2048 -h myfile_sorted.bam &amp;gt; supp_only.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This bash script returns a bam file that contains all alignments for reads that have multiple mappings. It works by only selecting reads that have supplementary alignments (2048 flag), or are not the primary alignment (256 flag), then using picard tools to filter the &lt;code&gt;bam&lt;/code&gt; file.  &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/3390/2085&quot;&gt;Using picard tools is the fastest way I have seen to filter a &lt;code&gt;bam&lt;/code&gt; file given a list of reads.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bsh prettyprint-override&quot;&gt;&lt;code&gt;#/bin/bash&#xA;BAMFILE=/path/to/my_file.bam&#xA;OUTFILE=/path/to/multiply_mapped.bam&#xA;PICARD_JAR=/path/to/picard.jar&#xA;TEMP=multiply_mapped.txt&#xA;&#xA;for filt_int in 256 2048; \&#xA;do samtools view -f &quot;&lt;span class=&quot;math-container&quot;&gt;$filtint&quot; $&lt;/span&gt;{BAMFILE} | \&#xA;  grep -v '#' | cut -f1; done | sort | uniq &amp;gt; &lt;span class=&quot;math-container&quot;&gt;${TEMP}; \&#xA;java -jar $&lt;/span&gt;{PICARD_JAR} FilterSamReads \&#xA;  I=&lt;span class=&quot;math-container&quot;&gt;${BAMFILE} \&#xA;  O=$&lt;/span&gt;{OUTFILE} \&#xA;  READ_LIST_FILE=&lt;span class=&quot;math-container&quot;&gt;${TEMP} \&#xA;  FILTER=includeReadList&#xA;rm $&lt;/span&gt;{TEMP}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This command took 21 seconds to execute for my bam file with ~300,000 long reads mapped and several repetitive regions.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Question 2&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Filtering the &lt;code&gt;bam&lt;/code&gt; file should be pretty standard, as addressed above in my response to &lt;strong&gt;Q1&lt;/strong&gt;. However, different long-read aligners are more sensitive to distantly-related paralogs. You might consider trying the &lt;a href=&quot;https://github.com/PacificBiosciences/blasr&quot; rel=&quot;nofollow noreferrer&quot;&gt;blasr aligner&lt;/a&gt; since it is tuned for PacBio long reads, or try bwa-mem to compare your results with minimap2. Realistically though, most of these aligners will probably behave similarly to one another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The fact that there are paralogs does not matter to the aligners.&lt;/strong&gt; The aligner just sees these regions as two different places to which a single read can map just about equally well. When there are multiple similar regions to which a read can map, as in the case of paralogs, the aligner will report both alignments. &lt;strong&gt;I do not know of any long-read paralog-aware aligners.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to why none of the reads mapped to the paralogs: Perhaps they have low mappability, as in they are repetitive? Also, maybe the capture didn't work properly? It looks like there is serious sequencing bias to the dataset given the uneven coverage.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Question 3&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Are the reads that are mapping actually 4kb, or are the alignments only 4kb long? You may have many 6kb-long sequences that have been clipped at either the 5' or 3' end. If this is not a heavily-curated genome then the assembly may be incorrect around the paralogs, preventing proper mapping. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One suggestion is to take whole-genome long reads from the same organism and to map them. Then check if the mapping to this region is equally strange compared to your capture data. This will give you a better idea of what caused the uneven coverage and seeming lack of coverage around the paralogs.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-12-11T22:47:12.230" LastActivityDate="2018-12-11T22:47:12.230" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5648" PostTypeId="2" ParentId="402" CreationDate="2018-12-11T23:12:02.330" Score="4" Body="&lt;p&gt;With this function, you can subsample a given BAM file to a user-specified number of reads using SamBamba. The function automatically calculates the scaling factor. Ideas in part taken from &lt;a href=&quot;http://crazyhottommy.blogspot.com/2016/05/downsampling-for-bam-files-to-certain.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;function SubSample {&#xA;&#xA;## Calculate the sampling factor based on the intended number of reads:&#xA;FACTOR=&lt;span class=&quot;math-container&quot;&gt;$(samtools idxstats $&lt;/span&gt;1 | cut -f3 | awk -v COUNT=&lt;span class=&quot;math-container&quot;&gt;$2 'BEGIN {total=0} {total += $&lt;/span&gt;1} END {print COUNT/total}')&#xA;&#xA;if [[ &lt;span class=&quot;math-container&quot;&gt;$FACTOR &amp;gt; 1 ]]&#xA;  then &#xA;  echo '[ERROR]: Requested number of reads exceeds total read count in' $&lt;/span&gt;1 '-- exiting' &amp;amp;&amp;amp; exit 1&#xA;fi&#xA;&#xA;sambamba view -s &lt;span class=&quot;math-container&quot;&gt;$FACTOR -f bam -l 5 $&lt;/span&gt;1&#xA;&#xA;}&#xA;&#xA;## Usage example, selecting 100.000 reads:&#xA;SubSample in.bam 100000 &amp;gt; subsampled.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;code&gt;$2&lt;/code&gt; is the number of reads. Say you choose 10000 reads but have paired-end data, you'll end up with 5000 fragments.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2018-12-11T23:12:02.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5649" PostTypeId="1" AcceptedAnswerId="5650" CreationDate="2018-12-12T11:36:15.670" Score="4" ViewCount="410" Body="&lt;p&gt;For example, I have two allelic genes. How can I identify a recessive and dominant gene? Are there any databases with this information? Or the answer to my question is to study the concept of &quot;haplosufficiency&quot;?&lt;/p&gt;&#xA;" OwnerUserId="3637" LastActivityDate="2018-12-12T14:06:01.717" Title="How can I identify a recessive and dominant gene?" Tags="|genome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5650" PostTypeId="2" ParentId="5649" CreationDate="2018-12-12T11:45:25.113" Score="5" Body="&lt;p&gt;I'd probably look up the genes in &lt;a href=&quot;https://www.omim.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OMIM&lt;/a&gt;, which provides detailed description of the functions and phenotypes associated with genes, including inheritance mode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, this is what you would get for TP53:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fDdry.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fDdry.png&quot; alt=&quot;OMIM Gene-Phenotype Relationships table for P53&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1332" LastEditorUserId="298" LastEditDate="2018-12-12T14:06:01.717" LastActivityDate="2018-12-12T14:06:01.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5651" PostTypeId="2" ParentId="402" CreationDate="2018-12-12T18:31:35.037" Score="1" Body="&lt;p&gt;A totally different tactic...it helps if you aren't super picky about how many reads you have at the end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Illumina read names contain the lane, tile, and xy coordinates of the read in them.  If you filtered by these, you would be guaranteed to get either both members of a pair, or neither.  I would recommend trying to avoid the edges of the flowcell.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2018-12-12T18:31:35.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5652" PostTypeId="1" CreationDate="2018-12-12T18:54:02.600" Score="4" ViewCount="567" Body="&lt;h2&gt;question 1&lt;/h2&gt;&#xA;&lt;p&gt;I wanted to use sumt command after completion of mcmc run. Whenever I use, &lt;code&gt;sumt Filename=infile.nex&lt;/code&gt;, it gives me the error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A matrix must be specified before sumt can be used&#xA;Error when setting parameter &amp;quot;Filename&amp;quot; (1) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But for the same analysis, I can use sump without any problem. What should I do now?&lt;/p&gt;&#xA;&lt;h2&gt;question 2&lt;/h2&gt;&#xA;&lt;p&gt;Another issue, I used burn in as 25% where as in Tracer, it is always showing 10%. No matter how much burn in I use, Tracer always shows 10% as burn in.&lt;/p&gt;&#xA;&lt;h3&gt;system stats&lt;/h3&gt;&#xA;&lt;pre&gt;&lt;code&gt;Operating system : Centos7&#xA;Version of MrBayes: 3.2.6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thanks in advance,&#xA;Best Regards,&#xA;Zillur&lt;/p&gt;&#xA;" OwnerUserId="3839" LastEditorUserId="17320" LastEditDate="2023-10-27T21:34:00.583" LastActivityDate="2023-12-04T22:44:19.577" Title="mrbayes, Sumt ; Error when setting parameter &quot;Filename&quot; (1)" Tags="|phylogenetics|phylogeny|mrbayes|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5653" PostTypeId="1" AcceptedAnswerId="5654" CreationDate="2018-12-12T23:33:20.827" Score="5" ViewCount="199" Body="&lt;p&gt;I recently had some Iso-Seq sequencing done on my organism catfish on the new Sequel platform and got weird alignments for a size selected 4 Kilobase and up fraction after running the isoseq3 pipeline.  When I investigated the exon intron junctions using data from (Pac Bio Iso-Seq, Illumina RNA-seq) against our private reference genome, I get PacBio Iso-Seq reads where the sequence length is greater than the seq length on the reference for my bam alignments:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Seq length:&lt;/strong&gt; 6751&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Seq length on ref:&lt;/strong&gt; 5267&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When blasting the sequence in NCBI it shows 5% similarity to the publicly-available genome.  I'll attach a JBrowse image with the same RNA sequenced with Illumina paired RNA-seq.  I also included a image and circled the pacbio contig and Illumina is above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vtf1j.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vtf1j.jpg&quot; alt=&quot;Pac Bio Iso-seq and Illumina RNA-seq&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Question 1:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Is this a problem with JBrowse?&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Question 2:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Does this mean that some DNA was sequenced with the catfish cDNA?&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Question 3:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Why is this aligned sequence's length longer than the length of the reference genome region to which it is aligned?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="989" LastEditorUserId="2085" LastEditDate="2018-12-13T00:13:41.883" LastActivityDate="2018-12-13T00:17:09.517" Title="Why is a PacBio read length larger than the aligned reference region?" Tags="|rna-seq|sequence-alignment|pacbio|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5654" PostTypeId="2" ParentId="5653" CreationDate="2018-12-13T00:06:42.297" Score="5" Body="&lt;h3&gt;Q1&lt;/h3&gt;&#xA;&lt;p&gt;Probably not - this is probably just displaying the contents of the input (I presume a &lt;code&gt;bam&lt;/code&gt; file) exactly as they are encoded.&lt;/p&gt;&#xA;&lt;h3&gt;Q2&lt;/h3&gt;&#xA;&lt;p&gt;The splicing looks a little weird, but &lt;strong&gt;the exon-mapping pattern in the 5' part of the gene makes it seem like RNA rather than DNA&lt;/strong&gt;. I think that the large intron is what makes you think that it is DNA, correct?&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Explanations for weird splicing&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The weird read is a rare splice variant but a true mRNA.&lt;/li&gt;&#xA;&lt;li&gt;The weird read is not completely matured and has an intron that has not yet been spliced. The IsoSeq library prep protocol picked this up somehow and it ended up being sequenced.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;Q3&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Explanations for why &lt;strong&gt;Seq length: 6751&lt;/strong&gt; and &lt;strong&gt;Seq length on ref: 5267&lt;/strong&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There is probably a large amount of clipping on the 5' or 3' end of the read. This could be polyA or some other transcribed-but-not-yet-completely-matured seq. &lt;strong&gt;Or&lt;/strong&gt;, the extra sequence could be sequencing error, PacBio adapters, or a chimeric molecule.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;things to try&lt;/h3&gt;&#xA;&lt;p&gt;How to check what is going on:&lt;/p&gt;&#xA;&lt;p&gt;I'm guessing that you had to upload a &lt;code&gt;bam&lt;/code&gt; file to JBrowse to make the IsoSeq track. Use the sequence name that you got from Jbrowse and run the following command to extract the alignment info from the bam file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view /path/to/your/isoseq/file.bam | grep 'weird_read_id'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Look at the &lt;a href=&quot;http://bioinformatics.cvr.ac.uk/blog/tag/cigar-string/&quot; rel=&quot;noreferrer&quot;&gt;cigar string&lt;/a&gt; to see what it says about 5' or 3' clipping and corroborate with the sequence data.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-12-13T00:17:09.517" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5655" PostTypeId="2" ParentId="3444" CreationDate="2018-12-13T06:25:38.967" Score="2" Body="&lt;p&gt;Several years ago we have written a tool for this: &lt;a href=&quot;http://kodomo.fbb.msu.ru/~avkitex/projects/pdbparser.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://kodomo.fbb.msu.ru/~avkitex/projects/pdbparser.html&lt;/a&gt;&#xA;&lt;a href=&quot;https://github.com/avkitex/pdbparserTools&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/avkitex/pdbparserTools&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The description is only in Russian but you can contact the author.&lt;/p&gt;&#xA;" OwnerUserId="3841" LastEditorUserId="3841" LastEditDate="2018-12-15T13:54:16.693" LastActivityDate="2018-12-15T13:54:16.693" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="5656" PostTypeId="2" ParentId="5616" CreationDate="2018-12-13T14:35:19.103" Score="1" Body="&lt;p&gt;Most GFF parsers handle the work of reading annotation data into objects for convenient data access, but most do not handle the important task of resolving relationships between features. The &lt;a href=&quot;https://tag.readthedocs.io/en/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;tag&lt;/strong&gt; Python library&lt;/a&gt; does both, grouping related features together for inspection, traversal, and feature-by-feature processing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;CAVEAT&lt;/strong&gt;: The tag library only supports GFF3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;DISCLAIMER&lt;/strong&gt;: I'm the developer of the tag library, so this is a shameless plug. :-)&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2018-12-13T14:35:19.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5657" PostTypeId="2" ParentId="5171" CreationDate="2018-12-13T15:11:31.120" Score="2" Body="&lt;p&gt;This task has the same &quot;flavor&quot; as many I've done before, but each case is so subtly different that it's impossible to write a generalized tool that will work correctly for all circumstances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;R isn't my wheelhouse, but I was able to throw something together pretty quickly with standard Python. This should get you at least 95% of the way to where you're going. :-)&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;from argparse import ArgumentParser, FileType&#xA;from collections import defaultdict&#xA;from re import search, sub&#xA;&#xA;cli = ArgumentParser()&#xA;cli.add_argument('txspace', type=FileType('r'))&#xA;cli.add_argument('chrspace', type=FileType('r'))&#xA;args = cli.parse_args()&#xA;&#xA;exons = defaultdict(list)&#xA;for line in args.txspace:&#xA;    if line.startswith('#') or line.strip() == '':&#xA;        continue&#xA;    values = line.split('\t')&#xA;    featuretype = values[2]&#xA;    if featuretype not in ('CDS', 'exon'):&#xA;        continue&#xA;    txid = values[0]&#xA;    exons[txid].append(values)&#xA;&#xA;for line in args.chrspace:&#xA;    values = line.split('\t')&#xA;    if len(values) != 9:&#xA;        print(line, end='')&#xA;        continue&#xA;    featuretype = values[2]&#xA;    if featuretype not in ('gene', 'mRNA'):&#xA;        continue&#xA;    print(line, end='')&#xA;    if featuretype == 'mRNA':&#xA;        txid = search('ID=([^;\n]+)', values[8]).group(1)&#xA;        start = int(values[3])&#xA;        txexons = exons[txid]&#xA;        for exon in txexons:&#xA;            exon[0] = values[0]&#xA;            exon[3] = str(int(exon[3]) + start)&#xA;            exon[4] = str(int(exon[4]) + start)&#xA;            exon[8] = sub(r'Parent=[^;\n]+', 'Parent=' + txid, exon[8])&#xA;            print('\t'.join(exon), end='')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="96" LastEditorUserId="37" LastEditDate="2018-12-13T15:23:22.673" LastActivityDate="2018-12-13T15:23:22.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5658" PostTypeId="5" CreationDate="2018-12-13T15:14:52.667" Score="0" Body="&lt;p&gt;Biological annotations are typically represented as intervals on a 1-dimensional space, and many of the most common data processing problems in bioinformatics can be reduced to interval arithmetic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two most common interval notations used in bioinformatics are 1-based closed interval notation and 0-based half-open interval notation.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;1-based closed interval notation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In 1-based closed interval notation, each element in the sequence is numbered and the interval is denoted by the first and last element.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  1   2   3   4   5   6   7   8   9  10  11  12  13  14  15&#xA;  |   |   |   |   |   |   |   |   |   |   |   |   |   |   |&#xA;  A   C   G   T   G   A   T   T   A   C   A   A   C   G   T&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The interval containing the sequence &lt;code&gt;GATTACA&lt;/code&gt; is denoted &lt;code&gt;5-11&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Length calculations: &lt;code&gt;11 - 5 + 1 = 7&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Consecutive intervals: &lt;code&gt;100001-20000&lt;/code&gt;, &lt;code&gt;20001-30000&lt;/code&gt;, &lt;code&gt;30001-40000&lt;/code&gt;, ...&lt;/li&gt;&#xA;&lt;li&gt;No unambiguous representation of zero-length features (such as insertion sites)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;0-based open interval notation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In 0-based half-open notation, each element's coordinate is its offset from the beginning of the sequence. It can be helpful to think of coordinates as the spaces between elements in the sequence, as in the example below. Each interval is then denoted by the coordinates spanning the sequence of interest.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15 &#xA;|   |   |   |   |   |   |   |   |   |   |   |   |   |   |   |&#xA;  A   C   G   T   G   A   T   T   A   C   A   A   C   G   T&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The interval containing the sequence &lt;code&gt;GATTACA&lt;/code&gt; is denoted &lt;code&gt;4-11&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Length calculations: &lt;code&gt;11 - 4 = 7&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Consecutive intervals: &lt;code&gt;10000-20000&lt;/code&gt;, &lt;code&gt;20000-30000&lt;/code&gt;, &lt;code&gt;30000-40000&lt;/code&gt;, ...&lt;/li&gt;&#xA;&lt;li&gt;Zero-length features (such as insertion sites) are represented using the same coordinate twice: &lt;code&gt;4-4&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2019-02-01T08:42:36.903" LastActivityDate="2019-02-01T08:42:36.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5659" PostTypeId="4" CreationDate="2018-12-13T15:14:52.667" Score="0" Body="Questions related to processing, analysis, arithmetic, and set operations involving sequence intervals." OwnerUserId="96" LastEditorUserId="96" LastEditDate="2019-02-01T08:42:40.310" LastActivityDate="2019-02-01T08:42:40.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5660" PostTypeId="1" CreationDate="2018-12-14T04:34:41.810" Score="7" ViewCount="599" Body="&lt;p&gt;I noticed that many scRNA-seq papers normalize TPMs to 10k or 100k as opposed to 1M (as the abbreviation defines them). It doesn't really matter since you are just moving the decimal point, so why mess with an established convention?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, the values might be log-transformed. In that case, log(x+1) would affect the data less if the total is 1M.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2021-03-10T17:43:00.800" Title="Why are TPMs per 10k or 100k in many scRNA-seq studies?" Tags="|rna-seq|scrnaseq|normalization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5661" PostTypeId="1" CreationDate="2018-12-14T09:41:55.860" Score="2" ViewCount="59" Body="&lt;p&gt;Im trying to plot log fold change for a condition as well as no of patient for a particular gene mutated information but whenever im trying to do that i see only only the fold changes labeling not the patient but my idea is to see the fold change as well as mutation information for that gene im posting the .conf file along with the data files im using any help would be highly appreciated .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or there is a flaw in my idea to show both of them which im not sure but i guess whatever i seen so far examples it can work!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My conf file code and my data files &lt;a href=&quot;https://drive.google.com/open?id=1ZoOR-QusndH-Pwqs-WnkCGRTcOsJTULW&quot; rel=&quot;nofollow noreferrer&quot;&gt;data&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;use_rules = yes&#xA;karyotype = data/karyotype/karyotype.human.txt&#xA;# spacing = 10000000&#xA;#&#xA;# you can write&#xA;#&#xA;# spacing = 10u&#xA;#&#xA;# See ticks.conf for examples.&#xA;&#xA;chromosomes_units = 1000000&#xA;#chromosomes_display_default = no&#xA;#chromosomes                 = /hs[1234]$/&#xA;#chromosomes=hs5;hs7; hs17; hs11; hs9&#xA;&#xA;&amp;lt;&amp;lt;include ideogram.conf&amp;gt;&amp;gt;&#xA;&#xA;&amp;lt;&amp;lt;include four_ticks_4chr.conf&amp;gt;&amp;gt;&#xA;&#xA;&amp;lt;image&amp;gt;&#xA;&amp;lt;&amp;lt;include etc/image.conf&amp;gt;&amp;gt;                &#xA;&amp;lt;/image&amp;gt;&#xA;&amp;lt;&amp;lt;include etc/colors.conf&amp;gt;&amp;gt; &#xA;&#xA;&amp;lt;&amp;lt;include etc/colors_fonts_patterns.conf&amp;gt;&amp;gt; &#xA;&#xA;&#xA;&#xA;&#xA;&amp;lt;&amp;lt;include etc/housekeeping.conf&amp;gt;&amp;gt; &#xA;&#xA;&amp;lt;plots&amp;gt;&#xA;&amp;lt;plot&amp;gt;&#xA;#this is for two direction expression data visualisation&#xA;type = histogram &#xA;#file=data2/HSC_VS_CMP.txt&#xA;#file = /home/punit/circos-0.69-5/data_files/LEUKEMIA_value_tab.txt&#xA;file = /home/punit/circos-0.69-5/data_files/CIRCOS_VAL1.txt&#xA;&#xA;r0   = 0.60r&#xA;r1   = 0.35r&#xA;thickness = 10&#xA;min=-1.5&#xA;max=1.5&#xA;extend_bin = no&#xA;&#xA;&amp;lt;rules&amp;gt;&#xA;&amp;lt;rule&amp;gt;&#xA;condition    = var(value) &amp;gt; 1.5&#xA;color        = dred&#xA;fill_color   = dred&#xA;&amp;lt;/rule&amp;gt;&#xA;&#xA;&amp;lt;rule&amp;gt;&#xA;condition    = var(value) &amp;lt; -1.5&#xA;color        = dgreen&#xA;fill_color   = dgreen&#xA;&amp;lt;/rule&amp;gt;&#xA;&amp;lt;/rules&amp;gt;&#xA;&amp;lt;/plot&amp;gt;&#xA;#########################################################&#xA;&#xA;###########################################################&#xA;&#xA;&#xA;&amp;lt;plot&amp;gt;&#xA;type    = text&#xA;color = black&#xA;#file = /home/punit/circos-0.69-5/LEUKEMIA_gene_tab.txt&#xA;file = /home/punit/circos-0.69-5/CIRCOS_LOC1.txt&#xA;&#xA;&#xA;r0   = 0.75r&#xA;r1   = 0.95r&#xA;show_links     = yes&#xA;link_dims      = 0p,0p,50p,0p,10p&#xA;# link_dims      = 4p,4p,8p,4p,4p&#xA;link_thickness = 2p&#xA;link_color     = grey&#xA;label_snuggle             = yes&#xA;max_snuggle_distance      = 2r&#xA;label_size   = 25p&#xA;label_font   = condensed&#xA;padding  = 0p&#xA;rpadding = 0p&#xA;&amp;lt;/plot&amp;gt;&#xA;&#xA;###############################################################################&#xA;&#xA;&#xA;#############################################################################################&#xA;&#xA;    &amp;lt;plot&amp;gt;&#xA;    #this is for two direction expression data visualisation&#xA;    type = histogram&#xA;    color = blue&#xA;    #file=data2/HSC_VS_CMP.txt&#xA;    #file = /home/punit/circos-0.69-5/data_files/LEUKEMIA_value_tab.txt&#xA;    file = /home/punit/circos-0.69-5/M_DATA.txt&#xA;&#xA;    r0 = 2.0r&#xA;    r1 = 1.8r&#xA;    orientation = out&#xA;&#xA;    #max       = 20&#xA;    #min       = -20&#xA;    stroke_type = outline&#xA;    thickness   = 18&#xA;    extend_bin  = yes&#xA;    fill_color = blue&#xA;&#xA;    &amp;lt;rules&amp;gt;&#xA;    &amp;lt;rule&amp;gt;&#xA;    condition    = var(value) &amp;gt; 10&#xA;    color        = dblue&#xA;    fill_color   = dblue&#xA;    &amp;lt;/rule&amp;gt;&#xA;&#xA;    &amp;lt;rule&amp;gt;&#xA;    condition    = var(value) &amp;lt; 8&#xA;    color        = dyellow&#xA;    fill_color   = dyellow&#xA;    &amp;lt;/rule&amp;gt;&#xA;    &amp;lt;/rules&amp;gt;&#xA;&#xA;    &amp;lt;/plot&amp;gt;&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;    &amp;lt;/plots&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="285" LastEditorUserId="77" LastEditDate="2018-12-16T15:32:38.100" LastActivityDate="2018-12-16T15:32:38.100" Title="Cicros plot issue using two data track" Tags="|circos|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5662" PostTypeId="2" ParentId="5171" CreationDate="2018-12-14T11:04:29.183" Score="1" Body="&lt;p&gt;The linked gist contains a class I wrote for doing this on GTF files:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gist.github.com/IanSudery/d8349c22823a475ceb489c3e8aeb448e&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gist.github.com/IanSudery/d8349c22823a475ceb489c3e8aeb448e&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It uses the GTF class from cgat, which can be found here:&#xA;&lt;a href=&quot;https://github.com/cgat-developers/cgat-apps&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/cgat-developers/cgat-apps&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would use it to do this task like so:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;from cgat import gtf&#xA;from cgat import iotools&#xA;import TranscriptCoordsInterconverter&#xA;&#xA;&#xA;trans_coord_file = gtf.iterator(iotools.open_file(args[1])&#xA;transcripts = dict()&#xA;&#xA;for transcript in gtf.transcript_iterator(trans_coords_file):&#xA;    transcript_id = transcript[0].transcript_id&#xA;    transcripts[transcript_id] = transcript&#xA;&#xA;trans_coord_file.close()&#xA;&#xA;genome_coord_file = gtf.iterator(iotools.open_file(args[2]))&#xA;for transcript in gtf.transcript_iterator(genome_coord_file):&#xA;&#xA;    transcript_id = transcript[0].transcript_id&#xA;    contig = transcript[0].contig&#xA;    strand = transcript[0].strand&#xA;&#xA;    if transcript_id not in transcripts:&#xA;        continue&#xA;&#xA;    converter = TranscriptCoordsInterconverter(transcript)&#xA;    for exon in transcripts[transcript_id]:&#xA;        genome_intervals = converter.transcript_interval2genome_intervals((exon.start,exon.end))&#xA;        for interval in genome_intervals:&#xA;            new_entry = GTF.Entry().fromGTF(exon)&#xA;            new_entry.contig = contig&#xA;            new_entry.strand = strand&#xA;            new_entry.start = interval[0]&#xA;            new_entry.end = interval[1]&#xA;            print (str(new_entry)+&quot;\n&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="235" LastActivityDate="2018-12-14T11:04:29.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5663" PostTypeId="2" ParentId="5604" CreationDate="2018-12-14T13:34:45.963" Score="1" Body="&lt;p&gt;It is not trivial getting MB to run in parallel because from my experience it required granting privileges to regular users. I used this via MPI (MPICH) on a Slurm/SGE cluster.&lt;/p&gt;&#xA;&lt;p&gt;Essentially, this is coarse grain parallel processing.&lt;/p&gt;&#xA;&lt;p&gt;It is certainly worth parallelizing MB because each chain can be run on a different nodes and avoids RAM bottlenecks on complex models and big data sets. For example, 8 to 10+ chains can be run, ensuring convergence quickly by running chains across nodes.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt; I later found it was much easier to run via EasyBuild with OpenMPI using the command &lt;code&gt;runmpi&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-10-28T01:03:07.463" LastActivityDate="2023-10-28T01:03:07.463" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="5664" PostTypeId="2" ParentId="5652" CreationDate="2018-12-14T21:50:07.910" Score="2" Body="&lt;p&gt;Having thought this through, I think you've made a mistake. The file it wants is the outfile, you have given it the infile only. My original answer is below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;====================&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From memory (my personal memory, not RAM, SSD or Stata), in the run command, you need to specify,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nruns=2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;sumt will then combine both runs. If the file name is mystuff.run and nruns=2, there will be an output of mystuff.run1.t and mystuff.run2.t.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If nruns isn't set then sumt will not work, or if the MCMCMC output ends up with different names again sumt will not work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the burnin, you simply specify this in MrBayes. It is easy to write a script that will perform a 25% burnin. Tracer is nicely written, presumably C, its fast, but it has been around for a long time.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2018-12-17T22:25:04.760" LastActivityDate="2018-12-17T22:25:04.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="5665" PostTypeId="2" ParentId="5646" CreationDate="2018-12-14T23:34:41.653" Score="0" Body="&lt;p&gt;I have found an alternative solution that has been excellent for this paralog mapping issue, so I thought I'd post what I had worked on:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 1+2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if there is a different approach to mapping I could take that would output all alignments for a read, and then specifically only multi-mapped reads?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using blasr, pacbio specified some options in their github docs for long, nearly identical reads:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blasr input.bam /path/to/reference.fa --hitpolicy all --bam --out alignments.bam --minMatch 8 --maxMatch 15 --nproc 4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The link to the docs as to why this is useful is &lt;a href=&quot;https://github.com/PacificBiosciences/blasr/wiki/To-align-long-and-almost-identical-sequences&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, although I did tweak the parameters to smaller seed sizes to create more anchors to the reference, which produced more secondary alignments. This is effective because this will output up to 10 secondary alignments per read, which I saw quite frequently for my region of interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After this, sorting was accomplished using Picard SortSam, with the following flags:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar picard.jar SortSam INPUT=&lt;span class=&quot;math-container&quot;&gt;${SAMPLE} OUTPUT=./sorted/$&lt;/span&gt;{SAMPLE}\&#xA;SORT_ORDER=coordinate VALIDATION_STRINGENCY=LENIENT\&#xA;CREATE_INDEX=true MAX_RECORDS_IN_RAM=100000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The blasr files have much more information than normal sams produced by aligners like minimap2 and bwa mem, so the MAX_RECORDS_IN_RAM=100000 as opposed to 500000 helps to ensure less frequent job failures (this was done on a cluster).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, I parsed out the region of interest:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h input.bam &quot;chrX:1000-2000&quot; &amp;gt; extracted.bam # this is fake&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Parse out mapq=0 reads:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '$5==0' extracted.bam &amp;gt; mapq0.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Get read names to grep for later:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{print $1}' mapq0.txt &amp;gt; read_names_mapq0.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Search for all reads with readname from the original bam to get mapping positions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view infile.bam | grep -f read_names_mapq0.txt &amp;gt; all_reads_mapq0_mapq_high.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Keep only columns: 1) read_name, 2) Sam Flag, 3) Chr, 4) Pos, 5) MapQ:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{print &lt;span class=&quot;math-container&quot;&gt;$1,$&lt;/span&gt;2,&lt;span class=&quot;math-container&quot;&gt;$3,$&lt;/span&gt;4,$5}' OFS='\t' all_reads_mapq0_mapq_high.txt &amp;gt; mapq_read_info.txt &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I wrote a simple python script to sort by read name, count the number of times a read shows up, and see all the positions it maps to. An example of the output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;        Read_Name       Flag    Chr     Pos     MAPQ&#xA;read1/10158955/ccs       272     19      3395674 0&#xA;read1/10158955/ccs       0       16      217682  254&#xA;read1/10158955/ccs       272     12      64990527        0&#xA;read1/10158955/ccs       256     16      222913  0&#xA;read1/10158955/ccs       272     19      21328060        0&#xA;read1/10158955/ccs       256     19      3224052 0&#xA;read1/10158955/ccs       272     17      34878621        0&#xA;read1/10158955/ccs       272     16      84204312        0&#xA;read1/10158955/ccs       256     1       244788680       0&#xA;read1/10158955/ccs       272     2       222951342       0&#xA;read2/10223868/ccs       16      16      217057  254&#xA;read2/10223868/ccs       256     20      3078724 0&#xA;read2/10223868/ccs       272     6       133521134       0&#xA;read2/10223868/ccs       256     7       152515722       0&#xA;read2/10223868/ccs       256     21      45258117        0&#xA;read2/10223868/ccs       256     1       249160685       0&#xA;read2/10223868/ccs       272     16      223631  0&#xA;read3/10551872/ccs       272     GL000220.1      144982  0&#xA;read3/10551872/ccs       256     18      72831954        0&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;read_counts output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;        Read_Name       Flag    Chr     Pos     MAPQ&#xA;read1/10158955/ccs       10      10      10      10&#xA;read2/10223868/ccs       7       7       7       7&#xA;read3/10551872/ccs       9       9       9       9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The numbers in each column correspond to the number of supplementary reads seen. &#xA;A brief explanation of sam flags which I feel is relevant for unpaired reads:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Flag=0: read was mapped to + strand&#xA;Flag=16: read was mapped to - strand&#xA;Flag=256: read was mapped to + as secondary alignment&#xA;Flag=272: read was mapped to - as secondary alignment&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3669" LastActivityDate="2018-12-14T23:34:41.653" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="5666" PostTypeId="1" CreationDate="2018-12-15T02:31:43.200" Score="3" ViewCount="721" Body="&lt;p&gt;I'm starting to do sc-rnaseq using 10x cellranger pipelines, and i add TdTomato sequence to mouse reference genome and add an entry in the gtf. My code is But when I makref, it remindered that: &#xA;Writing genes GTF file into reference folder...&#xA;Invalid number of columns in GTF line &#xA;Can anyone tell me the reason?&lt;/p&gt;&#xA;" OwnerUserId="3850" LastEditorUserId="3850" LastEditDate="2018-12-24T15:26:58.423" LastActivityDate="2018-12-24T15:26:58.423" Title="scRNA-seq, 10x cellranger pipelines" Tags="|scrnaseq|fasta|gtf|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="5670" PostTypeId="2" ParentId="5666" CreationDate="2018-12-15T13:05:29.603" Score="2" Body="&lt;p&gt;As was suggested by @benn, you will need to add the TdTomato sequence to the end of your fasta formatted genome file. If your TdTomato sequence is in fasta format already, &lt;code&gt;cat&lt;/code&gt; will do the trick. If not, just go the end of the file, start a new line and add a header line starting with &quot;&gt;&quot; then immediately the name of your gene, no whitespcaes in between, something like &quot;&gt;TdTomato&quot;. Then add your sequence in a new line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The format of the GTF file is explained in the image you posted, the trick is to use the exact same name (in column 1) that is used in the genome file you will create at the previous step (no &quot;&gt;&quot; though). Again, this line, in which the different fields (columns) will be separated by tabs, will be added to the end of your GTF file.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2018-12-15T15:09:52.840" LastActivityDate="2018-12-15T15:09:52.840" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6670" PostTypeId="2" ParentId="5475" CreationDate="2018-12-16T03:38:05.830" Score="0" Body="&lt;p&gt;Two solutions based around the same premise:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Repeat the blast using &quot;blastp&quot;. I suspect you have used blastn (nucleotides). To translate the gene Expasy is okay and find the common &quot;translate&quot;, &lt;a href=&quot;https://web.expasy.org/translate/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Blasted via Blastx without any modification to the gene.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Blast is capable of doing just about anything if you are aware of all its parameters. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reasoning is that the gene you have will be under-represented in the database. By switching to amino acids there is a much greater chance of finding orthologous hits. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2018-12-16T03:38:05.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6672" PostTypeId="1" AcceptedAnswerId="6673" CreationDate="2018-12-16T08:00:14.557" Score="5" ViewCount="1952" Body="&lt;p&gt;I have around 100 chromatograms (&lt;code&gt;.ab1&lt;/code&gt; files) from Sanger sequencing a genome at loci believed to have an indel. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm new to interpreting this kind of data in general, but I've read a bit on the general idea—mostly in guides like &lt;a href=&quot;https://seqcore.brcf.med.umich.edu/sites/default/files/html/interpret.html&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt;. What I'm not able to figure out from the resources I've checked is &lt;strong&gt;how to identify insertions and deletions in a chromatogram&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm most concerned with  heterozygous indels, and these seem to be simpler. (And since my samples are (supposed to be) larger indels that are more rare and more likely to be het, I think.) A heterozygous indel, according to my understanding, would like something like consecutive heterozygous SNPs—overlapping peaks, maybe not with the same height, but with the same horizontal placement—until the end of the sample. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, something like the top track in the open window in this image. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BFkKb.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BFkKb.png&quot; alt=&quot;Indel Chromatogram&quot;&gt;&lt;/a&gt;&#xA;(Source: &lt;a href=&quot;https://www.codoncode.com/aligner/mutation-detection.htm&quot; rel=&quot;noreferrer&quot;&gt;CodonCode&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I'm trying to do this manually, and not with packages like this for several reasons, including: I want to see how this is actually done, I want to do it as thoroughly as possible—I will probably check my results with an automated package—and I was asked to do it like this.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How can evidence of (het) indels like this be distinguished from noise in the chromatogram?&lt;/em&gt; My guess is it includes things like&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the overlapping peaks last until the end of the sample (and don't start at the beginning, but somewhere in the middle); and &lt;/li&gt;&#xA;&lt;li&gt;the peaks are still at the same horizontal positions; i.e., one is not shifted right or left relative to the other?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is this correct?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;With my own chromatograms, I have a lot of doubt about distinguishing noise from indel. Here's an example, visualized with 4Peaks. (Full chromatogram &lt;a href=&quot;http://s000.tinyupload.com/?file_id=41522383324429952799&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Zii0P.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Zii0P.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the left, the chromatogram appears high quality. There are sharp, regularly spaced, single peaks. On the right, there appear to be two sets of peaks lined up with each other. Does this constitute evidence for an indel? Does the homopolymer region—I've noticed in many of these samples, the putative indel seems to start in a homopolymer region—cast doubt on this? &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I've also noticed that in some situations like this, there appear to be not two but &lt;em&gt;three&lt;/em&gt; peaks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5efEm.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5efEm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is one of these just noise and can be disregarded? Or does this cast doubt on the validity of the data—which is supposed to be coming from the diploid human genome?&lt;/p&gt;&#xA;" OwnerUserId="3033" LastActivityDate="2019-09-24T11:47:37.817" Title="Identifying Indels from Chromatograms" Tags="|sequence-alignment|ngs|genome|variant-calling|" AnswerCount="6" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6673" PostTypeId="2" ParentId="6672" CreationDate="2018-12-16T14:56:18.683" Score="4" Body="&lt;p&gt;If you want to call heterozygous SNPs and INDELs, you may try &lt;a href=&quot;https://droog.gs.washington.edu/polyphred/&quot; rel=&quot;nofollow noreferrer&quot;&gt;polyphred&lt;/a&gt;. I believe there are a couple of other similar tools, but I have forgotten their names.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How can evidence of (het) indels like this be distinguished from noise in the chromatogram?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Your two points are correct. Basically, at some position, you start to see two streaks of overlapping peaks. Once identified, the two streaks can be aligned to each other if you shift one of them. This is a strong constraint, but it is hard to tell by eye because it is not obvious how to assign peaks to streaks without running an automatic tool. Doing this is even more difficult given noisy traces like your last figure. I think you'd better run polyphred or similar.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;the putative indel seems to start in a homopolymer region—cast doubt on this?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The INDEL mutation rate in long homopolymer runs is orders of magnitude higher than elsewhere in the genome. This has two implications: 1) you expect to see more homopolymer indels; 2) PCR polymerase is more likely to introduce errors in homopolymers––a higher fraction of homopolymer indels tend to be wrong.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2018-12-16T14:56:18.683" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6674" PostTypeId="2" ParentId="6672" CreationDate="2018-12-16T18:00:30.307" Score="1" Body="&lt;p&gt;Sanger sequencing produces a bit of messy signal at the start and a lot at the end of the sequence, but should have a region of clean signal in the middle. If the signal is getting mixed up in the middle, it suggests that there are sequences that are out of phase with each other. That &lt;em&gt;could&lt;/em&gt; be a heterozygous INDEL, but it could also indicate that multiple template sequences have been amplified by the same primer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sanger sequencing is frequently carried out in both directions (forward and reverse complement), and if similar things are happening in both directions at the same place, it makes it more likely that there is a heterozygous INDEL and not something else funny going on. It's a very good idea to have homozygous templates for all expected variants, if possible. If the secondary peaks in the signal match what is observed in the homozygous variants, that can also be used as evidence. In the absence of control sequences, it is also possible to eyeball the peaks to find self-similarity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where triple peaks are seen, that could point to a region where there are two INDEL variants in close proximity, or multiple regions getting amplified at the same time, or even a single INDEL variant with more than two variant states (e.g. a homopolymer with 8 As that is sometimes 7 As, and sometimes 9 As).&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2018-12-17T10:37:45.160" LastActivityDate="2018-12-17T10:37:45.160" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6675" PostTypeId="2" ParentId="6672" CreationDate="2018-12-17T05:22:17.720" Score="3" Body="&lt;p&gt;to understand how to interpret sanger sequencing chromatograms I'd like to give a brief introduction to how this data are generated (sorry, if you already know this)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The target region is amplified by PCR. The following sequencing reaction is quite similar to a PCR, despite two important differences:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Only one strand is used as a template, by adding just 1 primer instead of two.&lt;/li&gt;&#xA;&lt;li&gt;In addition two normal dNTPs, we have modified ddNTPS. Those are fluorescence labeled and the 3'-OH is missing, so there is no chance to add more dNTPS.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The result of this reaction are single stranded DNA fragments of different length. The last base is fluorescence labeled. So we have fragments that looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A&#xA;AT&#xA;ATC&#xA;ATCG&#xA;ATCGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Of course there a many copies of these fragments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now these fragments are separated by capillary electrophoresis. Small fragments are detected first. We can differ the several bases by its fluorescence color.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's imagine a DNA strand like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ATCGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What happens if we had on the second strand a SNP?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ATCGA&#xA;ATAGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now the sequence would detect two signal of equal intensity on base 3 - one for C and one for A. This have two effects in the chromatogram.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Two peaks of equal heights with different colors&lt;/li&gt;&#xA;&lt;li&gt;The relative peak height is just half the height, compared to those sequences that have just one base at this position.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Let's see what happen in case of a heterozygous deletion. An alignment would look like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ATCGA&#xA;AT-GA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But our capillary electrophoresis doesn't know about alignment. It only knows about fragment length. So this is what it will detect&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ATCGA&#xA;ATGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With the beginning of the deletion it will detect to different signals. Just like in case of the SNP, the relativ signal intensity is now just about the half.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this knowledge let's go back to your initial questions:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How can evidence of (het) indels like this be distinguished from noise&#xA;  in the chromatogram? My guess is it includes things like&lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;the overlapping peaks last until the end of the sample (and don't start at the beginning, but somewhere in the middle); and&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;They don't start at the beginning, if your indel is not right after the primer, correct. In case of an insertion or deletion you have strands of different size. So if your target region is quite small you may have luck, that you can see the end of the overlap at the end of the sequence, because the shorter strand reaches its end.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;the peaks are still at the same horizontal positions; i.e., one is not shifted right or left relative to the other?&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes. And one of the best things to distinguish between noise and true heterozygous variant is a relativ drop off of the signal intensities by approximately 50% compared to homozygous samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concerning your second chromatogram: This is not an indel. As the other already said, this is a mistake made by the polymerase. Invitro polymerases doesn't like homopolymer stretches. They begin to add extra bases or &quot;forget&quot; one. As we do PCR this error accumulates. In my experience up to 9 time the same base is not a big problem, 10 only if you have luck, from 11 upwards your sequence are terrible. If you are interested in the sequence part behind this stretch, you have to sequence from the other direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;fin swimmer&lt;/p&gt;&#xA;" OwnerUserId="1310" LastActivityDate="2018-12-17T05:22:17.720" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6676" PostTypeId="1" CreationDate="2018-12-17T07:16:35.287" Score="7" ViewCount="715" Body="&lt;p&gt;I have been using biopython 1.72 to display my phylogenetic tree files.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Using the function &lt;code&gt;'Phylo.draw(pars_tree, branch_labels=lambda c: c.branch_length)'&lt;/code&gt; to display branch lengths as well on tree, the tree displays the branch lengths as including all numbers after decimal. While I want the numbers to be truncated to two decimal places.&#xA;Following is a python method that truncates upto &lt;code&gt;&quot;the number required&quot;&lt;/code&gt; decimal places.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;def truncate(n, decimals=0):&#xA;    multiplier = 10 ** decimals&#xA;    return int(n * multiplier) / multiplier&#xA;truncate (12.567, 2)  #Using function&#xA;12.56                 #Output&#xA;Can I use such method with biopython for branch lengths?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;'Phylo.draw'&lt;/code&gt; function opens the window which is very narrow and if the tree file is larger, all the branches are overlapped and it gets difficult to read what is written. Does anyone know any better display method?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Following is the example code, I have been using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Phylo&#xA;from Bio.Phylo.TreeConstruction import *&#xA;from Bio import AlignIO&#xA;&#xA;aln = AlignIO.read(open('example.phy'), 'phylip')&#xA;&#xA;calculator = DistanceCalculator()&#xA;dm = calculator.get_distance(aln)&#xA;&#xA;constructor = DistanceTreeConstructor()&#xA;njtree = constructor.nj(dm)&#xA;starting_tree = njtree&#xA;scorer = ParsimonyScorer()&#xA;searcher = NNITreeSearcher(scorer)&#xA;constructor = ParsimonyTreeConstructor(searcher, starting_tree)&#xA;pars_tree = constructor.build_tree(aln)&#xA;Phylo.draw(pars_tree, branch_labels=lambda c: c.branch_length)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have an msa file as input and it gives a tree, which looks like:                                                 &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                                                         _______ Human&#xA;                                                 _______|&#xA;                                                |       |_______ Chimpanzee&#xA;                                         _______|&#xA;                                        |       |        _______ Dog&#xA;                                 _______|       |_______|&#xA;                                |       |               |_______ Cow&#xA;                         _______|       |&#xA;                        |       |       |_______________________ Elephant&#xA;                 _______|       |&#xA;                |       |       |_______________________________ Mouse&#xA;                |       |&#xA;         _______|       |_______________________________________ Platypus&#xA;        |       |&#xA;        |       |        _______________________________________ Anole_lizard&#xA;        |       |_______|&#xA;  ______|               |        _______________________________ Chicken&#xA; |      |               |_______|&#xA; |      |                       |_______________________________ Zebra_finch&#xA; |      |&#xA; |      |_______________________________________________________ Xenopus&#xA;_|&#xA; |       _______________________________________________________ Zebrafish&#xA; |______|&#xA; |      |_______________________________________________________ Fugu&#xA; |&#xA; |______________________________________________________________ Amphioxus&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3142" LastEditorUserId="3848" LastEditDate="2020-05-05T12:14:54.747" LastActivityDate="2020-05-05T12:14:54.747" Title="Truncating branch length values of Phylogenetic tree with biopython" Tags="|phylogenetics|python|phylogeny|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6677" PostTypeId="1" CreationDate="2018-12-17T17:53:06.257" Score="2" ViewCount="747" Body="&lt;p&gt;Very conceptual doubt not sure if im doing it right or something terribly wrong so im using this corrplot library to calucate the correlation of my desired data frame after that i rounding it off but when i use my original calculated matrix such as &lt;code&gt;M&lt;/code&gt; i get a different plot again when i used the round of matrix i get a different plot in terms of correlation ,in case of &lt;code&gt;M&lt;/code&gt; all seems to be positively correlated but when i used the &lt;code&gt;corr&lt;/code&gt; i get plot which shows my cell types are both positively and negatively correlated .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Im posting my code below as well as the plot along with the data&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(corrplot)&#xA;gendat = read.csv(&quot;NEW_RBP/TF_MYELOID_DISEASE.txt&quot;,row.names = 1,sep = &quot;\t&quot;)&#xA;&#xA;dim(gendat)&#xA;head(gendat)&#xA;gen &amp;lt;- gendat&#xA;head(gen)&#xA;M&amp;lt;- cor(gen, use=&quot;complete.obs&quot;, method=&quot;spearman&quot;)&#xA;corr &amp;lt;- round(cor(M),1)&#xA;&#xA;&#xA;&#xA;&#xA;names(gen)&#xA;dim(gen)&#xA;&#xA;mycolors &amp;lt;- rep(NA,23)&#xA;names(mycolors) &amp;lt;- names(gen)&#xA;mycolors[1:4]   &amp;lt;- 'black' # mpg, cyl, disp, hp&#xA;mycolors[5:15]   &amp;lt;- 'red'   # drat, wt, qsec, vs, am&#xA;mycolors[16:23] &amp;lt;- 'blue'&#xA;mycolors[13:16] &amp;lt;- 'darkgreen'&#xA;&#xA;&#xA;&#xA;ord &amp;lt;- corrMatOrder(corr, order=&quot;hclust&quot;)&#xA;newcolours &amp;lt;- mycolors[ord]&#xA;newcolours&#xA;corrplot(M, tl.col = newcolours, method = &quot;color&quot;,is.corr = FALSE, &#xA;         tl.srt = 45,diag = TRUE&#xA;         ,order = &quot;hclust&quot;,hclust.method = c(&quot;complete&quot;),addrect = 5,tl.cex = 3,number.cex=1.5)&#xA;&#xA;corrplot(corr, tl.col = newcolours, method = &quot;color&quot;,is.corr = TRUE, &#xA;         tl.srt = 45,diag = TRUE&#xA;         ,order = &quot;hclust&quot;,hclust.method = c(&quot;complete&quot;),addrect = 3,tl.cex = 3,number.cex=1.5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AKvoq.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AKvoq.jpg&quot; alt=&quot;plot when the corr matrix is used &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/obO8f.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/obO8f.jpg&quot; alt=&quot;plot when i the M matrix is used&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know what is happening in both the cases&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://drive.google.com/open?id=1DJLEg-jYiQxWH8EIWCIRcoZyG9wmxq3d&quot; rel=&quot;nofollow noreferrer&quot;&gt;data file&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2018-12-17T17:53:06.257" Title="corrplot issue with correlation plot output" Tags="|r|visualization|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6678" PostTypeId="1" AcceptedAnswerId="6680" CreationDate="2018-12-17T18:41:14.230" Score="1" ViewCount="124" Body="&lt;p&gt;I want to implement the analysis found in the paper &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/26525104&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNA-Seq of Tumor-Educated Platelets Enables Blood-Based Pan-Cancer, Multiclass, and Molecular Pathway Cancer Diagnostics&lt;/a&gt; The project id is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/bioproject/?term=PRJNA281708&quot; rel=&quot;nofollow noreferrer&quot;&gt;281708&lt;/a&gt;. It has 285 samples comprising about 300 Gb of read data. The reads are available on &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/bioproject/?term=PRJNA281708&quot; rel=&quot;nofollow noreferrer&quot;&gt;sra&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to apply a machine-learning pipeline to find important features in these data but I do not have access to computing resources to finish the project.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Question&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Are there free cloud computing platforms that I can use to perform this research?&lt;/p&gt;&#xA;" OwnerUserId="3771" LastEditorUserId="2085" LastEditDate="2018-12-17T19:32:45.397" LastActivityDate="2018-12-17T19:36:04.330" Title="Are there free cloud computing platforms for biology projects?" Tags="|fastq|genomics|phylogenetics|sratoolkit|" AnswerCount="1" CommentCount="1" ClosedDate="2018-12-18T10:33:36.733" ContentLicense="CC BY-SA 4.0" />
  <row Id="6679" PostTypeId="2" ParentId="3463" CreationDate="2018-12-17T18:51:37.453" Score="5" Body="&lt;p&gt;It's worth noting two things as of Dec 2018:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Albacore is being deprecated (but is still available from the Nanopore developer portal).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Guppy is under active development, so Ryan Wick's comparisons may not reflect the current state of things. (The claim is that the current, just-released version of Guppy uses a new &quot;flip-flop&quot; algorithm that improves accuracy over Albacore.)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;MinKnow contains the current version of the production basecaller. As of the release today, this has switched from Albacore to Guppy. If you want, you can run MinKnow for your basecalling (which will run Guppy), but you will likely need a beefy computer with access to a good amount of storage to do that.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3860" LastActivityDate="2018-12-17T18:51:37.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6680" PostTypeId="2" ParentId="6678" CreationDate="2018-12-17T19:28:23.803" Score="1" Body="&lt;p&gt;If you're getting started with a project and need some free computational resources you should look at &lt;a href=&quot;https://galaxyproject.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://galaxyproject.org/&lt;/a&gt;. There are some free servers that you may be able to use. However, Galaxy is designed to have pre-fabricated pipelines and a GUI to avoid having to deal with command-line tools. Therefore any sort of custom analysis will have to go through the &lt;a href=&quot;https://galaxyproject.org/develop/api/&quot; rel=&quot;nofollow noreferrer&quot;&gt;galaxy API&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, most of the free computational resources that were available several years ago like iPlantCollaborative are no longer available. Running servers costs money, after all, and the funding for such projects often runs out after a few years.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You may consider asking researchers at your local universities to collaborate and provide you with access to their servers.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-12-17T19:36:04.330" LastActivityDate="2018-12-17T19:36:04.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6681" PostTypeId="2" ParentId="6672" CreationDate="2018-12-17T19:49:40.823" Score="0" Body="&lt;p&gt;One important point that @gringer briefly touched on is that there is often off-target amplification when performing PCR on whole-genome DNA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the &lt;strong&gt;noise in the chromatograms very well could be from another part of the genome and not related to indels at all&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only way to ensure that this is not a problem is to clone your PCR products into a vector, transform into something like E. coli, grow up a single colony, cut the product with restriction enzymes, size-select, then sequence the correct-sized insert. Or you could just re-sequence your PCR product with Illumina or PacBio to accurately sequence the complete population of molecules -- indels and all. Check out:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bmcgenomics.biomedcentral.com/articles/10.1186/s12864-018-4611-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hebert, Paul DN, et al. &quot;A Sequel to Sanger: amplicon sequencing that scales.&quot; BMC genomics 19.1 (2018): 219.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The safe thing to do is to not read too much into the noise of Sanger chromatograms unless it is a simple and very clean SNP.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-12-17T20:48:19.580" LastActivityDate="2018-12-17T20:48:19.580" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6682" PostTypeId="2" ParentId="6672" CreationDate="2018-12-17T23:17:26.713" Score="1" Body="&lt;p&gt;Honestly I would not try to call heterozygous indels with Sanger sequencing.  It's just not the right technology.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you were trying to confirm a known indel that didn't flank a homopolymer stretch, and you had sequence from the other side that got messy at exactly the same point, maybe you could safely conclude there was something going on there, and possibly you could eyeball the exact right sequence of mixed peaks, but I don't see that what's what you have in your examples at all.  It just looks like noise.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2018-12-17T23:17:26.713" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6683" PostTypeId="1" CreationDate="2018-12-18T03:26:36.583" Score="5" ViewCount="81" Body="&lt;p&gt;&lt;strong&gt;Are there any databases that have aggregated information on cell volumes?&lt;/strong&gt;  Most useful would be a database keyed by cell ontology (e.g., &lt;code&gt;CL:0000236&lt;/code&gt; for B cell).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;http://book.bionumbers.org/how-big-is-a-human-cell/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioNumbers book has a small table&lt;/a&gt;, but none of the references appear to link back to systematic catalogues.  In fact, just to put that small table together, they had to reference a dozen or so papers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://shogoin.stemcellinformatics.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The SHOGoiN database&lt;/a&gt; is a nice overview of the taxonomy, but the only morphological data available is attached to the images (&lt;a href=&quot;http://shogoin.stemcellinformatics.org/image/cell/40&quot; rel=&quot;nofollow noreferrer&quot;&gt;example B cell&lt;/a&gt;), which only includes areas and other histological measurements. Also, they don't seem to provide an API for programmatic access.&lt;/p&gt;&#xA;" OwnerUserId="900" LastEditorUserId="900" LastEditDate="2020-03-09T22:05:30.947" LastActivityDate="2023-03-02T19:56:14.170" Title="Database of Cell Volumes by Cell Ontology?" Tags="|database|single-cell|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6685" PostTypeId="1" AcceptedAnswerId="6689" CreationDate="2018-12-18T16:53:45.377" Score="2" ViewCount="1511" Body="&lt;p&gt;I am doing a gene-gene pairwise correlation, I get a plot and I do see positive and negative correlation &lt;a href=&quot;https://i.stack.imgur.com/56IzN.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/56IzN.jpg&quot; alt=&quot;correlation plot &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(corrplot)&#xA;library(Hmisc)&#xA;gen = read.csv('NEW_RBP/CORPLOT/TF_NORMAL_NEW_40.txt',row.names = 1)&#xA;head(gen)&#xA;dim(gen)&#xA;gendat &amp;lt;- t(gen)&#xA;#gendat &amp;lt;- gen&#xA;head(gendat)&#xA;dim(gendat)&#xA;&#xA;&#xA;macolor = colorRampPalette(c(&quot;navyblue&quot;, &quot;white&quot;, &quot;red&quot;))(100)&#xA;&#xA;cor_5 &amp;lt;- rcorr(as.matrix(gendat))&#xA;M &amp;lt;- cor_5&lt;span class=&quot;math-container&quot;&gt;$r&#xA;p_mat &amp;lt;- cor_5$&lt;/span&gt;P&#xA;row_names &amp;lt;- rownames(M)&#xA;&#xA;pdf(&quot;TF_NORMAL_NEW_40_gene.pdf&quot;,20,15,paper = &quot;a4r&quot;)&#xA;&#xA;&#xA;corrplot(M, method = &quot;color&quot;,is.corr = FALSE,&#xA;         tl.srt = 45,diag = TRUE,tl.pos='n'&#xA;         ,order = &quot;hclust&quot;,hclust.method = c(&quot;complete&quot;),tl.cex = 3,number.cex=1.5)&#xA;&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How do I extract those gene pairs? One way what I have read and done is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(dplyr)&#xA;flat_cor_mat &amp;lt;- function(cor_r, cor_p){&#xA;    #This function provides a simple formatting of a correlation matrix&#xA;    #into a table with 4 columns containing :&#xA;    # Column 1 : row names (variable 1 for the correlation test)&#xA;    # Column 2 : column names (variable 2 for the correlation test)&#xA;    # Column 3 : the correlation coefficients&#xA;    # Column 4 : the p-values of the correlations&#xA;    library(tidyr)&#xA;    library(tibble)&#xA;    cor_r &amp;lt;- rownames_to_column(as.data.frame(cor_r), var = &quot;row&quot;)&#xA;    cor_r &amp;lt;- gather(cor_r, column, cor, -1)&#xA;    cor_p &amp;lt;- rownames_to_column(as.data.frame(cor_p), var = &quot;row&quot;)&#xA;    cor_p &amp;lt;- gather(cor_p, column, p, -1)&#xA;    cor_p_matrix &amp;lt;- left_join(cor_r, cor_p, by = c(&quot;row&quot;, &quot;column&quot;))&#xA;    cor_p_matrix&#xA;}&#xA;&#xA;&#xA;#cor_3 &amp;lt;- rcorr(as.matrix(gendat))&#xA;&#xA;my_cor_matrix &amp;lt;- flat_cor_mat(cor_5&lt;span class=&quot;math-container&quot;&gt;$r, cor_5$&lt;/span&gt;P)&#xA;head(my_cor_matrix)&#xA;&#xA;dim(my_cor_matrix)&#xA;&#xA;Delete.na &amp;lt;- function(DF, n=0) {&#xA;      DF[rowSums(is.na(DF)) &amp;lt;= n,]&#xA;    }&#xA;&#xA;TF_NORMAL_CORR = Delete.na(my_cor_matrix)&#xA;dim(RBP_DISEASE_40_CORR)&#xA;&#xA;TF_NORMAL_HIGH = TF_NORMAL_CORR[ TF_NORMAL_CORR&lt;span class=&quot;math-container&quot;&gt;$cor &amp;gt; 0.9 &amp;amp; RBP_DISEASE_40_CORR$&lt;/span&gt;p &amp;lt; 0.00000001, ]&#xA;&#xA;TF_NORMAL_LOW = TF_NORMAL_CORR[TF_NORMAL_CORR&lt;span class=&quot;math-container&quot;&gt;$cor &amp;lt; -0.9 &amp;amp; TF_NORMAL_CORR$&lt;/span&gt;p &amp;lt; 0.00000001, ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now even if I do that sort of high cut-off filtering I still get considerably high number of correlation pairs like few thousands if positive correlation is considered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So few of my issues which I would like to get input or help &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How do I extract the cluster of regions with positive correlation or negative correlation  ?&lt;/li&gt;&#xA;&lt;li&gt;If I found certain gene pair/pairs how do I label them ? should I make a vector or map or any other way like if say more than 0.8 correlation I would like to put a label on those regions only &lt;/li&gt;&#xA;&lt;li&gt;Considering label the gene order is important to be considered, what I have read and found is this how to find the order of the label&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;tempCor &amp;lt;- cor(gendat)&#xA;tempCor &amp;lt;- data.frame(tempCor)&#xA;dim(tempCor)&#xA;names(tempCor) &amp;lt;- c(1:741)&#xA;&#xA;out &amp;lt;- corrplot(t(tempCor),method = &quot;color&quot;,is.corr = FALSE,&#xA;                 tl.srt = 45,diag = TRUE,tl.pos='n'&#xA;                 ,order = &quot;hclust&quot;,hclust.method = c(&quot;complete&quot;),tl.cex = 3,number.cex=1.5)&#xA;&#xA;dimnames(out)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Now with all these how can I extract those patches of high and low correlation pairs and add selected label or highlight them and add label &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I m not sure if all these can be done with corrplot! like adding selected label if yes i would be really glad to use that..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My data file &lt;a href=&quot;https://drive.google.com/open?id=1try36CdLgsBSCdwd52jWHLiLcDS8kkNe&quot; rel=&quot;nofollow noreferrer&quot;&gt;data used &lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;In which groups are you interested in? In the upper left region and the bottom right square? In which format do you want it ? A list for each group? BTW, how many samples do you have? If it is so high, or they are mostly similar or there are few samples.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its symmetric so would it matter if i use upper left or bottom right? I guess anyone would do good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Format - a list for each group if i understand if there is one gene with many positive correlation then that would be considered as group?is that you are saying ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 16 samples in this case but here I using gene-gene. I'm bit curious with a naive question: Is it possible to do gene to sample correlation or if i know what genes are positively or negatively correlated can I trace back without have to plot another heatmap of found genes which are positively or negatively correlated to see their expression in sample ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have given the data file it might help you better with my context..&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="48" LastEditDate="2018-12-19T08:29:36.377" LastActivityDate="2018-12-19T12:19:15.700" Title="Extracting genes from corrplot and adding labels based on high and low corelation" Tags="|r|visualization|correlation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6688" PostTypeId="1" AcceptedAnswerId="6690" CreationDate="2018-12-18T20:25:52.790" Score="3" ViewCount="749" Body="&lt;p&gt;I got a &lt;code&gt;PCA&lt;/code&gt; plot of &lt;code&gt;bulk RNA-seq&lt;/code&gt; experiment that looks the following way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gbQ1P.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gbQ1P.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was generated by the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pcaData &amp;lt;- plotPCA(rld_sva, intgroup=c(&quot;Group&quot;), returnData=TRUE)&#xA;percentVar &amp;lt;- round(100 * attr(pcaData, &quot;percentVar&quot;))&#xA;ggplot(pcaData, aes(PC1, PC2, color=Group)) +&#xA;  geom_point(size=3) +&#xA;  xlab(paste0(&quot;PC1: &quot;,percentVar[1],&quot;% variance&quot;)) +&#xA;  ylab(paste0(&quot;PC2: &quot;,percentVar[2],&quot;% variance&quot;)) + &#xA;  coord_fixed()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;First &lt;code&gt;sva&lt;/code&gt; correction was run to correct for batch effects and then &lt;code&gt;rlog&lt;/code&gt; transformed values were plugged in to &lt;code&gt;plotPCA&lt;/code&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first issue that catches the eye is that &lt;code&gt;100%&lt;/code&gt; of the variance is explained by just 2 dimensions. I am not sure what can one say about the data in this case. The second issue is that I get only around &lt;code&gt;20&lt;/code&gt; differentially expressed genes by using &lt;code&gt;DESeq2&lt;/code&gt; analysis (&lt;code&gt;log2FoldChange &amp;gt; 1&lt;/code&gt;, &lt;code&gt;p_adj &amp;lt; 0.05&lt;/code&gt;). I know that we can not directly state that if there is a large difference on &lt;code&gt;PCA&lt;/code&gt; there will be present a plenty of differentially expressed genes, but why is it not the case? Simple logic tells me that &lt;code&gt;pca&lt;/code&gt; shows the difference between the samples in their gene expression, so I would expect seeing a plenty of differentially expressed genes.&lt;/p&gt;&#xA;" OwnerUserId="2005" LastEditorUserId="2005" LastEditDate="2018-12-19T00:11:05.143" LastActivityDate="2020-04-05T10:31:25.670" Title="PCA plot shows big difference but not many differentially expressed genes are found" Tags="|scrnaseq|deseq2|pca|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6689" PostTypeId="2" ParentId="6685" CreationDate="2018-12-19T03:57:30.973" Score="4" Body="&lt;p&gt;Here is a solution using the &lt;a href=&quot;https://cran.r-project.org/web/packages/pheatmap/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;pheatmap&lt;/a&gt; library to cluster and visualise the correlation matrix, then extract the groups from the cluster dendrograms:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gen &amp;lt;- read.csv(file = &quot;TF_NORMAL_NEW_40.txt&quot;, header = TRUE, row.names = 1, sep = &quot;,&quot;)&#xA;gendat &amp;lt;- t(gen)&#xA;library(corrplot)&#xA;library(Hmisc)&#xA;macolor = colorRampPalette(c(&quot;navyblue&quot;, &quot;white&quot;, &quot;red&quot;))(100)&#xA;&#xA;cor_5 &amp;lt;- rcorr(as.matrix(gendat))&#xA;M &amp;lt;- cor_5&lt;span class=&quot;math-container&quot;&gt;$r&#xA;str(M)&#xA;p_mat &amp;lt;- cor_5$&lt;/span&gt;P&#xA;row_names &amp;lt;- rownames(M)&#xA;&#xA;library(pheatmap)&#xA;pdf(&quot;TF_NORMAL_NEW_40_gene.pdf&quot;,20,15,paper = &quot;a4r&quot;)&#xA;plot_data_pheatmap &amp;lt;- pheatmap(M, color = rev(macolor), clustering_method = &quot;complete&quot;, fontsize_row = 0.8, fontsize_col = 0.8)&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GSdVX.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GSdVX.jpg&quot; alt=&quot;jpeg version of pdf&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tree_cut &amp;lt;- cutree(plot_data_pheatmap&lt;span class=&quot;math-container&quot;&gt;$tree_row, h = 20)&#xA;tc &amp;lt;- data.frame(tip = names(tree_cut), clust_membership = as.character(unname(tree_cut)))&#xA;row.names(tc) &amp;lt;- tc$&lt;/span&gt;tip&#xA;tc &amp;lt;- tc['clust_membership']&#xA;head(tc)&#xA;str(tc)&#xA;class(tc$clust_membership)&#xA;pdf(&quot;TF_NORMAL_NEW_40_gene_clustcut.pdf&quot;,20,15,paper = &quot;a4r&quot;)&#xA;pheatmap(M, color = rev(macolor), clustering_method = &quot;complete&quot;, annotation_row = tc, annotation_col = tc, fontsize_row = 0.8, fontsize_col = 0.8)&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PAT1v.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PAT1v.jpg&quot; alt=&quot;clusters at height 20 jpeg version&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3381" LastEditorUserId="3381" LastEditDate="2018-12-19T12:19:15.700" LastActivityDate="2018-12-19T12:19:15.700" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6690" PostTypeId="2" ParentId="6688" CreationDate="2018-12-19T03:58:20.637" Score="7" Body="&lt;p&gt;You only have 4 samples total. I think it would be difficult to not have the PCA show big differences between the groups with so few points.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, for differential expression, it is hard to get something to be statistically significant with only 2 replicates.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2018-12-19T03:58:20.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6691" PostTypeId="2" ParentId="5631" CreationDate="2018-12-19T04:03:11.060" Score="1" Body="&lt;p&gt;I would recommend using &lt;a href=&quot;https://xenabrowser.net/datapages/?cohort=Cancer%20Cell%20Line%20Encyclopedia%20(CCLE)&amp;amp;removeHub=https%3A%2F%2Fxena.treehouse.gi.ucsc.edu%3A443&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC Xena&lt;/a&gt; instead of GDC for CCLE data. They don't have everything. For example, BAM files are not available. However, the data that is available is organized in standard text files with consistent identifiers.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2018-12-19T04:03:11.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6693" PostTypeId="2" ParentId="4596" CreationDate="2018-12-19T11:05:04.197" Score="1" Body="&lt;p&gt;The first column of your GTF must correspond exactly to the name of the sequence in your FASTA file. Typically this is the chromosome so a GTF line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    hg38_knownCanonical exon    11106535    11262507    0.000000    .   .   gene_id &quot;gene1&quot;; transcript_id &quot;tx1&quot;; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This must have a corresponding sequence in the FASTA file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;chr1&#xA;ATCG ATCG ATCG ATCG ATCG ATCG...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Therefore you need to add an additional sequence to your FASTA file with &lt;code&gt;&amp;gt;TdTomato&lt;/code&gt;. This must have exactly the same sequence as the construct that you are expecting reads for. The run &lt;code&gt;cellranger mkref&lt;/code&gt; and &lt;code&gt;cellranger count&lt;/code&gt; on these FASTA and GTF files. Cellranger also can take a GTF with “CDS” as a feature in addition to “exons”. The STAR alignment algorithm used by cellranger is splice aware so it can handle split reads but you will need to ensure the GTF corresponds to exons of the mRNA/cDNA rather than the full sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;10x Genomics is a platform with a high dropout rate, especially for lowly expressed genes. You should bear this in mind for what you expect to observe. It may be worth comparing the mapping rate of your construct to housekeeper genes to see if this is the reason for the low counts. Since whole mRNA molecules are captured on the beads before preparing the library, there should not be a difference in UMI counts between long and short genes.&lt;/p&gt;&#xA;" OwnerUserId="1855" LastActivityDate="2018-12-19T11:05:04.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6694" PostTypeId="1" CreationDate="2018-12-19T11:57:15.130" Score="2" ViewCount="2563" Body="&lt;p&gt;I have a set of scRNA-seq samples expressing TdTomato, which has high content in microscope. I followed the 10x cellranger pipelines to finsh the work, my procedures are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;added TdTomato on an additional chromosome:&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/dzDLv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dzDLv.png&quot; alt=&quot;addition tdtomato chromosome sequence&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;added an entry in the gtf &lt;a href=&quot;https://i.stack.imgur.com/NDflA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NDflA.png&quot; alt=&quot;additional gtf&quot;&gt;&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;cellranger mkref&lt;/li&gt;&#xA;&lt;li&gt;cellranger count&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I got the result, which includes:&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/BWSbG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BWSbG.png&quot; alt=&quot;result folder&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But strangely, the tdtomato expression content in my three sample are all very low! And cells with tdtomato are also very low and disperse!! Take sample 1 as example, you can see the picture:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QQyCF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QQyCF.png&quot; alt=&quot;loupe cell browser picture&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not normal, but I can't find the reason. Following the &lt;a href=&quot;https://kb.10xgenomics.com/hc/en-us/articles/360000944391-Why-do-I-have-zero-UMI-counts-for-my-marker-gene-&quot; rel=&quot;nofollow noreferrer&quot;&gt;10x genomics help documentation&lt;/a&gt; noticed that by executing the CellRanger filtering steps, maybe my tdtomato reads are filtered too much, but because I'm not very skillful at programming and I'm first time to do RNA-seq (and scRNA-seq), I just don't know how to analysis the bam file as suggested by the official website and find the real reason of the problem:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/HIRc3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HIRc3.png&quot; alt=&quot;Suggestions on the official website&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can show me the code need to do, I will be very grateful! &lt;/p&gt;&#xA;" OwnerUserId="3851" LastEditorUserId="3694" LastEditDate="2021-10-27T16:11:43.650" LastActivityDate="2024-01-18T00:22:31.497" Title="Low custom tdtomato gene content" Tags="|scrnaseq|single-cell|10x-genomics|cellranger|tdtomato|" AnswerCount="5" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6695" PostTypeId="1" CreationDate="2018-12-19T14:34:20.000" Score="5" ViewCount="308" Body="&lt;p&gt;I need to use these PMC tables from the medical journals for the purpose of integrating the data with a decision-making software. Though, I can see that one can access full-text for an article from PMC &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/oai/oai.cgi?verb=GetRecord&amp;amp;identifier=oai:pubmedcentral.nih.gov:6148705&amp;amp;metadataPrefix=pmc&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; (which includes the article table), but is there a way to extract the table from this XML document and export it to a local MySQL database or any other relational database?&lt;/p&gt;&#xA;" OwnerUserId="3015" LastActivityDate="2019-02-03T07:55:12.913" Title="Is there a way to import tables in PubMed Central to MySQL or other Relational databases?" Tags="|public-databases|text-processing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6696" PostTypeId="1" CreationDate="2018-12-19T20:18:59.847" Score="2" ViewCount="49" Body="&lt;h2&gt;Background&lt;/h2&gt;&#xA;&lt;p&gt;I have two networks with homologous genes from two different species.&lt;/p&gt;&#xA;&lt;p&gt;To identify homologs I performed a functional comparison. One of the species is not well studied while the other one is the house fly (D. melanogaster). I used ClueGO to predict the functions of the genes in the D. melanogaster network.&lt;/p&gt;&#xA;&lt;p&gt;I found a function of interest influenced by a set of genes and predicted that the same genes in the other network (thus far uncharacterized) should also perform a similar function.&lt;/p&gt;&#xA;&lt;h2&gt;Questions&lt;/h2&gt;&#xA;&lt;p&gt;&lt;strong&gt;Q1:&lt;/strong&gt; How do I check if there are overlapping patterns between both the networks on Cytoscape?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Q2:&lt;/strong&gt; There are more than 100 homologs in this list and I would like to minimize the list by only considering those homologous genes that are arranged the same as well.&lt;/p&gt;&#xA;" OwnerUserId="976" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-12-20T11:23:55.540" Title="Visualize similar interactions in two homologous networks" Tags="|visualization|networks|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="6697" PostTypeId="2" ParentId="6696" CreationDate="2018-12-19T21:07:14.327" Score="4" Body="&lt;p&gt;An easy way to visualize this is to make a tab-delimited table that contains edge information for both species and both networks, color the edges depending on the species, and make the thickness and transparency dependent on the interaction strength.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, say you have a list of orthologs for the two species and just assign them a number or a name. Each edge has a &quot;source&quot; and a &quot;target&quot;. The edge weight is the interaction strength.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Note do not copy-paste the below table as the mixed tabs/spaces may make the graph topology incorrect). In my toy dataset the data look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;source  target  variable    value&#xA;gene1   gene2   speciesA    3&#xA;gene1   gene3   speciesA    42&#xA;gene2   gene3   speciesA    3&#xA;gene1   gene2   speciesB    9&#xA;gene2   gene3   speciesB    5&#xA;gene2   gene4   speciesB    41&#xA;gene3   gene4   speciesB    32&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Import your network to cytoscape as a network from a file:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8BhSw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8BhSw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then go to the style tab:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/poiDr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/poiDr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;... and make sure edge is selected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/S7eT2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/S7eT2.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you can adjust the transparency, edge color, and edge stroke settings, like so using edge stroke color:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vRD48.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vRD48.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;... and transparency panels as examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ci0Ap.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ci0Ap.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Don't forget to save your style when it is complete!&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you have adjusted these settings you will have something that looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RucPs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RucPs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-12-19T21:07:14.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6698" PostTypeId="2" ParentId="6694" CreationDate="2018-12-19T21:39:28.280" Score="3" Body="&lt;p&gt;A few suggestions: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Manually inspecting the BAM file with a genome browser like IGV, as @GWW suggested and checking the qualities of the alignments in your BAM/SAM file (SAM is the human readable version of BAM, there are tools to convert one to another);&lt;/li&gt;&#xA;&lt;li&gt;Using another single cell analysis tool like Seurat so that you can manually set thresholds for QC and not lose to many (tdtomato) expressing cells at the QC step (I don't know about your experiment but if transfected, cells could be at a worse condition than non-transfected ones and hence might not pass QC);&lt;/li&gt;&#xA;&lt;li&gt;Double checking your tdtomato reference sequence, I don't know about tdtomato but for example GFP has flavors like EGFP, similar but different sequences, might explain low alignment score if you observe in step 1;&lt;/li&gt;&#xA;&lt;li&gt;Taking the limitations of scRNA-seq (especially droplet-based) technologies into account. The figure below is from the CITE-seq paper (from supp info), &lt;em&gt;Simultaneous epitope and transcriptome measurement in single cells&lt;/em&gt; (﻿Stoeckius et al., 2017), showing that the protein - RNA measurements of a given gene might show poor correlation.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/TYmiz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TYmiz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2018-12-19T21:39:28.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6699" PostTypeId="1" AcceptedAnswerId="6700" CreationDate="2018-12-19T23:48:54.063" Score="3" ViewCount="824" Body="&lt;h3&gt;Background&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I have a fasta file with protein sequences. The first sequence in the fasta file is the reference sequence and every subsequent sequence is something I want to align to the reference. The end goal is to identify every unique sequence and be able to compare the regions of variation.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;What I have tried&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I'm looking into different tools such as Clustal Omega and MAFFT, but there is no option to select the reference sequence against which all the other protein sequences will be aligned.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Questions&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q1:&lt;/strong&gt; I want perform one pairwise global alignment of each sequence in the file against the first (reference) sequence. &lt;strong&gt;What software can I use to do this?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q2:&lt;/strong&gt; &lt;strong&gt;What specific commands can I use to achieve Q1?&lt;/strong&gt; I'm looking for specific instructions on how to complete this, not just a suggestion of a tool, since I'm not able to find anything from a relatively comprehensive search of available tools.&lt;/p&gt;&#xA;" OwnerUserId="3585" LastEditorUserId="2085" LastEditDate="2018-12-20T01:33:37.573" LastActivityDate="2018-12-20T01:47:52.637" Title="Multiple sequence pairwise global alignment against reference sequence" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6700" PostTypeId="2" ParentId="6699" CreationDate="2018-12-20T01:24:53.943" Score="2" Body="&lt;h3&gt;Q1&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;From this phrase, &quot;The end goal is to identify every unique sequence&quot;, I am not 100% sure of what you mean by &quot;every unique sequence&quot;. It sounds like maybe you want to group alignments of homologs and visualize them together?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the proteins are similar enough, you might also benefit from trying BLAST to look at similar regions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise MAFFT, Clustal, or MUSCLE are all appropriate tools to generate the alignments that you need. However I also do not know of any software aside from Geneious that has the option to align many proteins to a reference. If you were working with DNA I'm sure the stock suggestion would be to use a read aligner like &lt;code&gt;bwa mem&lt;/code&gt; or &lt;code&gt;minimap2&lt;/code&gt; and to use a bam file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I don't know of any software to do exactly what you would like, I wrote a small python script to run MAFFT many times for each sequence in a fasta after the first sequence, the reference.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Q2 - MAFFT/Python&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In any case, here is a bit of python code that performs a MAFFT alignment of all of the sequences in a fasta file against the first sequence in the fasta file, which you have designated as the reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is &lt;a href=&quot;https://en.wikipedia.org/wiki/FASTA_format&quot; rel=&quot;nofollow noreferrer&quot;&gt;the fasta file from Wikipedia&lt;/a&gt;. I called it &lt;code&gt;seqs.fasta&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;reference&#xA;MDSKGSSQKGSRLLLLLVVSNLLLCQGVVSTPVCPNGPGNCQVSLRDLFDRAVMVSHYIHDLSS&#xA;EMFNEFDKRYAQGKGFITMALNSCHTSSLPTPEDKEQAQQTHHEVLMSLILGLLRSWNDPLYHL&#xA;VTEVRGMKGAPDAILSRAIEIEEENKRLLEGMEMIFGQVIPGAKETEPYPVWSGLPSLQTKDED&#xA;ARYSAFYNLLHCLRRDSSKIDTYLKLLNCRIIYNNNC*&#xA;&#xA;&amp;gt;MCHU - Calmodulin - Human, rabbit, bovine, rat, and chicken&#xA;ADQLTEEQIAEFKEAFSLFDKDGDGTITTKELGTVMRSLGQNPTEAELQDMINEVDADGNGTID&#xA;FPEFLTMMARKMKDTDSEEEIREAFRVFDKDGNGYISAAELRHVMTNLGEKLTDEEVDEMIREA&#xA;DIDGDGQVNYEEFVQMMTAK*&#xA;&#xA;&amp;gt;gi|5524211|gb|AAD44166.1| cytochrome b [Elephas maximus maximus]&#xA;LCLYTHIGRNIYYGSYLYSETWNTGIMLLLITMATAFMGYVLPWGQMSFWGATVITNLFSAIPYIGTNLV&#xA;EWIWGGFSVDKATLNRFFAFHFILPFTMVALAGVHLTFLHETGSNNPLGLTSDSDKIPFHPYYTIKDFLG&#xA;LLILILLLLLLALLSPDMLGDPDNHMPADPLNTPLHIKPEWYFLFAYAILRSVPNKLGGVLALFLSIVIL&#xA;GLMPFLHTSKHRSMMLRPLSQALFWTLTMDLLTLTWIGSQPVEYPYTIIGQMASILYFSIILAFLPIAGX&#xA;IENY&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is a script, &lt;code&gt;mafft_ref.py&lt;/code&gt;. Run it with &lt;code&gt;python mafft_ref.py seqs.fasta&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio import SeqIO&#xA;from Bio.Align.Applications import MafftCommandline&#xA;import tempfile&#xA;&#xA;fasta = sys.argv[1]&#xA;&#xA;reference = 0&#xA;count = 0&#xA;with tempfile.NamedTemporaryFile() as temp:&#xA;    for record in SeqIO.parse(fasta, &quot;fasta&quot;):&#xA;        if count == 0:&#xA;            reference = record&#xA;        else:&#xA;            SeqIO.write([reference, record], temp.name, &quot;fasta&quot;)&#xA;            mafft_cline = MafftCommandline(input=temp.name)&#xA;            stdout,stderr=mafft_cline()&#xA;            print(stdout)&#xA;        count += 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the output of &lt;code&gt;python mafft_ref.py seqs.fasta&lt;/code&gt; printed to the terminal. Use &lt;code&gt;python mafft.py prots.pep &amp;gt; output.txt&lt;/code&gt; to save the output to &lt;code&gt;output.txt&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;reference&#xA;------------------MDSKGSSQKGSRLLLLLVVSNLLLCQGVVSTPVCPNGPGNCQ&#xA;VSLRDLFDRA--------------VMVSHYIHDLSS-----EMFNEFDKRYAQGKGFITM&#xA;ALNSCHTSSLPTPEDKEQAQQTHHEVLMSLILGLLRSWNDPLYHLVTEVRGMKGAPDAIL&#xA;SRAIEIEEENKRLLEGMEMIFGQVIPGAKETEPYPVWSGLPSLQTKDEDARYSAFYNLLH&#xA;CLRRDSSKIDTYLKLLNCRIIYNNNC&#xA;&amp;gt;MCHU - Calmodulin - Human, rabbit, bovine, rat, and chicken&#xA;ADQLTEEQIAEFKEAFSLFDKDGDGTITTKEL------------GTVMRSL---GQNPTE&#xA;AELQDMINEVDADGNGTIDFPEFLTMMARKMKDTDSEEEIREAFRVFDK---DGNGYISA&#xA;A---------------------------------------ELRHVMTN-----------L&#xA;GEKLTDEEVD-------EMIREADIDG-------------------DGQVNYEEFVQMM-&#xA;-----TAK------------------&#xA;&#xA;&amp;gt;reference&#xA;------------MDSKGSSQKGSRLLLLLVVSNLLLCQGVVSTPVCPNGPGNCQVS----&#xA;--LRDLF--------------------DRAVM----VSHYIHDLSSEMFNEFDKRYAQGK&#xA;GFITMALNSCHTSSL-------PTPEDKEQAQQTHHE-------------VLMSLILGLL&#xA;RS--WNDPLYHLVTEVRGMKGAPDAILSRAIEIEEE--------------NKRLLEGMEM&#xA;IFGQVI-------------------P-------------------GAKETE-PYPVWSGL&#xA;PSLQTKDEDARYSAFYNLLHCLRRDSSKIDTYLKLLNCRIIYNNNC&#xA;&amp;gt;gi|5524211|gb|AAD44166.1| cytochrome b [Elephas maximus maximus]&#xA;LCLYTHIGRNIYYGSYLYSETWNTGIMLLLITMATAFMGY----VLPWG----QMSFWGA&#xA;TVITNLFSAIPYIGTNLVEWIWGGFSVDKATLNRFFAFHFILP-----------------&#xA;-FTMVALAGVHLTFLHETGSNNPLGLTSDSDKIPFHPYYTIKDFLGLLILILLLLLLALL&#xA;SPDMLGDPDNHM----------PADPLNTPLHIKPEWYFLFAYAILRSVPNK--LGGVLA&#xA;LFLSIVILGLMPFLHTSKHRSMMLRPLSQALFWTLTMDLLTLTWIGSQPVEYPYTIIGQM&#xA;ASI----------LYFSIILAFLPIAGXIENY--------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;References:&#xA;&lt;a href=&quot;https://www.biostars.org/p/343605/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Some credit of &lt;code&gt;mafft_ref.py&lt;/code&gt; to a biostars answer found here.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-12-20T01:47:52.637" LastActivityDate="2018-12-20T01:47:52.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6703" PostTypeId="2" ParentId="6696" CreationDate="2018-12-20T11:23:55.540" Score="0" Body="&lt;p&gt;I think what you're looking for, as I suggested in this &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/5436/compare-two-networks-of-the-same-genes-between-two-species/5451#5451&quot;&gt;old question&lt;/a&gt; (quite similar to this one, I believe), is the &lt;a href=&quot;http://apps.cytoscape.org/apps/gasoline&quot; rel=&quot;nofollow noreferrer&quot;&gt;GASOLINE app&lt;/a&gt; for Cytoscape that allow you to perform an analysis that will tell you which parts of the two networks are conserved and which genes belong to the overlapping subnetworks.&lt;/p&gt;&#xA;" OwnerUserId="3119" LastActivityDate="2018-12-20T11:23:55.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6704" PostTypeId="1" CreationDate="2018-12-20T14:09:04.850" Score="3" ViewCount="613" Body="&lt;p&gt;I have &lt;a href=&quot;https://github.com/tbenavi1/smudgeplot/blob/master/hetkmers.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;a script&lt;/a&gt; that extracts the set of unique kmer pairs from a kmer dump file (&amp;amp; then using them downstream to guess ploidy).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The script seems to work correctly (it's reporting a meaningful set of kmer pairs), but since we have added a parallelization it does not quit when the job is done for really big data sets. The problem is that I never managed to reproduce the behavior with a small reproducible example and it takes around 400G of memory and a day of computation to get to the problem. My imagination is quite limited, I have no idea how should I debug such a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't really get it; the script actually prints the last line &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;*_families_2.tsv and *_coverages_2.tsv files saved.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;, but then it runs further (&amp;amp; forever). I thought it could be related to longer time to free the memory as discussed &lt;a href=&quot;https://stackoverflow.com/questions/2412715/python-program-doesnt-quit-when-finished&quot;&gt;here&lt;/a&gt;, but it does not seem like the case because it does not quit even after a really long time (I tried to just leave it for a day or so and when i kill it the memory is freed in like 20s).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anybody an idea where the problem could be? Or how to figure out?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reported problem on &lt;a href=&quot;https://github.com/tbenavi1/smudgeplot/issues/18&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="37" LastEditDate="2018-12-20T15:29:58.480" LastActivityDate="2019-01-22T23:01:50.373" Title="python does not quit when the input file is too big" Tags="|python|k-mer|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6705" PostTypeId="1" AcceptedAnswerId="6712" CreationDate="2018-12-20T15:15:32.653" Score="2" ViewCount="1352" Body="&lt;p&gt;The question of how one can extract the longest isoform per gene from a multi-fasta file has been answered expertly in this thread: &#xA;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/595/how-can-longest-isoforms-per-gene-be-extracted-from-a-fasta-file&quot;&gt;How can longest isoforms (per gene) be extracted from a FASTA file?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wonder: how can the python script &quot;filter.py&quot; by Devon Ryan, that is marked as best answer, be modified so that it extracts the N longest isoform sequences per gene instead of only the single longest isoform? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help on this to a python novice is very much appreciated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example of how the data looks (two genes with 5 isoformes each, and I need the 3 longest isoforms for each gene):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Animal1|gene000001_1  &#xA;TTLQLLNELVLLLHYTTLLLHTKKTKESTINNYFKQKEQK  &#xA;&amp;gt;Animal1|gene000001_2  &#xA;CYYYTTQHYYYIQKKQKNLQSTITSNKKNKNKHAK  &#xA;&amp;gt;Animal1|gene000001_3  &#xA;ARDMLDYTTTTKRVSATTTLHNTTTTYKKNKRIYNQQLLQTKRTKISMQNKNLNQINYKK  &#xA;LQETTEKHAK  &#xA;&amp;gt;Animal1|gene000001_4  &#xA;HHAVNFQPMSEATGIYLNSTEIRNSLLQYSSWSWFTFHTCKLWNVTF  &#xA;&amp;gt;Animal1|gene000001_5  &#xA;EILCYSIPPGVGLLFTPVNFGMLRSKLIKVRILDIINFNCSLLICLPPYLFQIDVGEDKD  &#xA;DPVVEIVLYLACSI  &#xA;&amp;gt;Animal1|gene000002_1  &#xA;TTLQLLNELVLLLHYTTLLLHTKKTKESTINNYFKQKEQK  &#xA;&amp;gt;Animal1|gene000002_2  &#xA;CYYYTTQHYYYIQKKQKNLQSTITSNKKNKNKHAK  &#xA;&amp;gt;Animal1|gene000002_3  &#xA;ARDMLDYTTTTKRVSATTTLHNTTTTYKKNKRIYNQQLLQTKRTKISMQNKNLNQINYKK  &#xA;LQETTEKHAK  &#xA;&amp;gt;Animal1|gene000002_4  &#xA;LCSAVARQKTKSRKYDATVDTCFSNIKININVLSFSLIVNPKYSKVSYQYGTNDAFRKFF  &#xA;ITSRSLR  &#xA;&amp;gt;Animal1|gene000002_5  &#xA;ETLEYFGLTINEKESTLILILMLLKQVSTVASYFLDLVFCLATALHS  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As output, I would like to have a fasta file like the following: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Animal1|gene000001_5  &#xA;EILCYSIPPGVGLLFTPVNFGMLRSKLIKVRILDIINFNCSLLICLPPYLFQIDVGEDKD  &#xA;DPVVEIVLYLACSI  &#xA;&amp;gt;Animal1|gene000001_3  &#xA;ARDMLDYTTTTKRVSATTTLHNTTTTYKKNKRIYNQQLLQTKRTKISMQNKNLNQINYKK  &#xA;LQETTEKHAK  &#xA;&amp;gt;Animal1|gene000001_4  &#xA;HHAVNFQPMSEATGIYLNSTEIRNSLLQYSSWSWFTFHTCKLWNVTF  &#xA;&amp;gt;Animal1|gene000002_3  &#xA;ARDMLDYTTTTKRVSATTTLHNTTTTYKKNKRIYNQQLLQTKRTKISMQNKNLNQINYKK  &#xA;LQETTEKHAK  &#xA;&amp;gt;Animal1|gene000002_4  &#xA;LCSAVARQKTKSRKYDATVDTCFSNIKININVLSFSLIVNPKYSKVSYQYGTNDAFRKFF  &#xA;ITSRSLR  &#xA;&amp;gt;Animal1|gene000002_5  &#xA;ETLEYFGLTINEKESTLILILMLLKQVSTVASYFLDLVFCLATALHS  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3876" LastEditorUserId="298" LastEditDate="2018-12-21T11:01:29.167" LastActivityDate="2018-12-21T14:04:11.763" Title="How can I extract the longest N isoforms per gene from a fasta file?" Tags="|rna-seq|filtering|isoform|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6706" PostTypeId="1" AcceptedAnswerId="8516" CreationDate="2018-12-20T15:25:59.390" Score="5" ViewCount="196" Body="&lt;p&gt;I have a large number of loci I would like to examine manually with &lt;code&gt;samtools tview&lt;/code&gt;. Rather than typing or copy-n-pasting dozens of coordinates, I was hoping to automate the generation of pileup &quot;images&quot; (text files).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With some experimenting just now, I was happy to learn that if I specify the display option &lt;code&gt;-d H&lt;/code&gt; or &lt;code&gt;-d T&lt;/code&gt; the output can easily be redirected to a file for later inspection. However, it appears that I cannot control the window captured. Changing the width of the terminal window doesn't change the output, nor does adjusting the &lt;code&gt;-p&lt;/code&gt; flag (i.e. &lt;code&gt;-p chr1:501-700&lt;/code&gt;). No matter what, &lt;code&gt;tview&lt;/code&gt; always reports an 80bp window in text or HTML mode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way I can adjust this?&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-04-24T14:53:40.727" Title="Control width of samtools tview &quot;snapshot&quot; when redirecting" Tags="|samtools|read-mapping|visualization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6707" PostTypeId="2" ParentId="2054" CreationDate="2018-12-20T16:18:01.330" Score="0" Body="&lt;p&gt;&lt;a href=&quot;http://deweylab.biostat.wisc.edu/detonate/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DETONATE&lt;/a&gt; offers a pipeline for filtering in this way as well. But the aligner used by default is Bowtie2...&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2018-12-20T16:18:01.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6708" PostTypeId="1" AcceptedAnswerId="6709" CreationDate="2018-12-20T19:14:12.023" Score="7" ViewCount="855" Body="&lt;h2&gt;Background&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I am using the wonderful &lt;a href=&quot;http://lh3lh3.users.sourceforge.net/parsefastq.shtml&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;kseq.h&lt;/code&gt;&lt;/a&gt; as my fastq/fasta parser for a C++ project. &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/5471/2085&quot;&gt;My actual implementation uses something like this&lt;/a&gt;, where I am counting kmers of each sequence and performing a bunch of other processing on every read pair.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks to @user1782818's code my program is very fast, but I want it to be faster. The only way I know how to do this is with multithreading. However, I have never worked with multithreading in C++.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each read pair I am accessing a vector or map of structs. Within each struct I am incrementing counters or pushing values onto vectors. Essentially I am doing something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;//compiled with g++ 7.3 using command g++ this_prog.cpp -o this_prog -lz&#xA;// execute with ./this_prog my_file.fastq.gz&#xA;#include &amp;lt;iostream&amp;gt;//cout&#xA;#include &amp;lt;map&amp;gt;&#xA;#include &amp;lt;zlib.h&amp;gt;&#xA;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &quot;kseq.h&quot;&#xA;KSEQ_INIT(gzFile, gzread)&#xA;&#xA;int main(int argc, char **argv)&#xA;{&#xA;  //initialize prefix&#xA;  std::map&amp;lt;std::string, int&amp;gt; prefix_counter;&#xA;  std::string prefix;&#xA;  prefix.resize(2);&#xA;&#xA;  gzFile fp;&#xA;  kseq_t *seq;&#xA;  int n = 0, slen = 0, qlen = 0;&#xA;  fp = gzopen(argv[1], &quot;r&quot;);&#xA;  seq = kseq_init(fp);&#xA;  while (kseq_read(seq) &amp;gt;= 0){&#xA;    ++n;&#xA;    //COUNT PREFIXES&#xA;    std::copy(seq-&amp;gt;seq.s, seq-&amp;gt;seq.s + 2, begin(prefix));&#xA;    if (prefix_counter.count(prefix) == 0){&#xA;      prefix_counter[prefix] = 1;&#xA;    } else prefix_counter[prefix]++;&#xA;  }&#xA;  std::cout &amp;lt;&amp;lt; &quot;Total n. of sequences: &quot; &amp;lt;&amp;lt; n &amp;lt;&amp;lt; '\n';&#xA;  kseq_destroy(seq);&#xA;  gzclose(fp);&#xA;&#xA;  //print counts&#xA;  for( const auto&amp;amp; sm_pair : prefix_counter ){&#xA;    std::cout &amp;lt;&amp;lt; sm_pair.first &amp;lt;&amp;lt; &quot;: &quot; &amp;lt;&amp;lt; sm_pair.second &amp;lt;&amp;lt; '\n';&#xA;  }&#xA;  return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Example output from the above program is something like this for a 100,000-read test fastq file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Total n. of sequences: 100000&#xA;AA: 8561&#xA;AC: 5178&#xA;AG: 7634&#xA;AT: 8523&#xA;CA: 7526&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Question&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q1:&lt;/strong&gt; Is it possible to perform a task like the one shown above using multithreading and &lt;code&gt;kseq.h&lt;/code&gt; to read in a fastq file and process the reads in chunks? I only want to use standard libraries. C++11 is OK - I know that multithreading support was added in C++11.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q2:&lt;/strong&gt; Can you please provide a C++ example of how to convert the above or similar program to use multithreading using &lt;code&gt;kseq.h&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2018-12-20T21:55:45.890" LastActivityDate="2018-12-21T02:59:45.817" Title="Multithread fastq processing with kseq.h in C++11?" Tags="|fastq|c++|multithreading|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6709" PostTypeId="2" ParentId="6708" CreationDate="2018-12-21T02:50:45.937" Score="7" Body="&lt;p&gt;You might take a look at my FQFeeder library (&lt;a href=&quot;https://github.com/rob-p/FQFeeder&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/rob-p/FQFeeder&lt;/a&gt;), which pairs the kseq parser with a multi-producer / multi-consumer queue to allow efficient parallel processing of the parsed reads.  The library contains an example program to demonstrate how to use it with multiple threads.  It's worth noting that kseq is already &lt;em&gt;very&lt;/em&gt; fast, so multi-threading only really makes sense if worker threads are doing non-trivial work on each read so that you can then make the parsing and processing concurrent (have the processing keep pace with kseq).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll note that my library relies on a header-only MPMC implementation and so technically isn't &quot;only using the standard library&quot;, but you're already relying on &lt;code&gt;kseq.h&lt;/code&gt;, so omitting third-party libraries must not be a hard constraint.&lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="491" LastEditDate="2018-12-21T02:59:45.817" LastActivityDate="2018-12-21T02:59:45.817" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6710" PostTypeId="2" ParentId="6705" CreationDate="2018-12-21T08:24:54.040" Score="1" Body="&lt;p&gt;I know you are asking for a python solution. But maybe you are interested in other ways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use &lt;code&gt;samtools faidx&lt;/code&gt; to index your fasta file. The resulting file contains the information about the length of every sequence. We can sort by length and extract the first n transcripts per gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1. Index fasta&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools faidx input.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. Extract IDs of the n longest transcripts per gene&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed 's/_.\+//g' input.fa.fai | sort -u | &#xA;    parallel -k 'grep {} -m 3 &amp;lt;(sort -k2,2 -r --sort g input.fa.fai) |&#xA;        cut -f1' &amp;gt; longestID.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;sed&lt;/code&gt; command replaces everything from &lt;code&gt;_&lt;/code&gt; onwards in each line, leaving behind only the gene name. &lt;code&gt;sort -u&lt;/code&gt; is used to get a list of distinct gene names. For each of theses gene names we &lt;code&gt;grep&lt;/code&gt; for the first 3 matches in the descending sorted by length index file. As we just want the sequence ids, we select only the first column with &lt;code&gt;cut&lt;/code&gt;. Adopt the &lt;code&gt;-m&lt;/code&gt; parameter if you want more or fewer transcripts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3. Get sequences&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools faidx input.fa -r longestID.txt &amp;gt; output.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1310" LastEditorUserId="298" LastEditDate="2018-12-21T11:02:40.580" LastActivityDate="2018-12-21T11:02:40.580" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6711" PostTypeId="2" ParentId="6706" CreationDate="2018-12-21T08:36:49.013" Score="2" Body="&lt;p&gt;Partially a shameless plug. See if &lt;a href=&quot;https://asciigenome.readthedocs.io/en/latest/description.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ASCIIGenome&lt;/a&gt; can help you with that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To review a number of regions that you saved in, say, a bed file, you can either process them in &lt;a href=&quot;https://asciigenome.readthedocs.io/en/latest/examples.html#batch-and-non-interactive-mode&quot; rel=&quot;nofollow noreferrer&quot;&gt;batch&lt;/a&gt; or load BAM(s) and bed file and jump from one bed interval to the next using the &lt;a href=&quot;https://asciigenome.readthedocs.io/en/latest/commandHelp.html#next&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;next -start&lt;/code&gt;&lt;/a&gt; command.&lt;/p&gt;&#xA;" OwnerUserId="339" LastActivityDate="2018-12-21T08:36:49.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6712" PostTypeId="2" ParentId="6705" CreationDate="2018-12-21T12:25:01.593" Score="0" Body="&lt;p&gt;Here a python solution:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;from __future__ import print_function&#xA;from Bio import SeqIO&#xA;from argparse import ArgumentParser&#xA;&#xA;&#xA;def main():&#xA;    args = get_args()&#xA;    last_gene = None&#xA;    transcripts = []&#xA;&#xA;    for record in SeqIO.parse(args.input, &quot;fasta&quot;):&#xA;        gene = record.id.split(&quot;_&quot;)[0]&#xA;&#xA;        if last_gene is None:&#xA;            last_gene = gene&#xA;&#xA;        if last_gene == gene:&#xA;            transcripts.append(record)&#xA;        else:&#xA;            print_longest_transcripts(transcripts, args.number)&#xA;            last_gene = gene&#xA;            transcripts = [record]&#xA;&#xA;    print_longest_transcripts(transcripts, args.number)&#xA;&#xA;&#xA;def print_longest_transcripts(transcripts, number):&#xA;    longest = sorted(transcripts, key=lambda x: len(x.seq), reverse=True)&#xA;&#xA;    for record in longest[:number]:&#xA;        print(&quot;&amp;gt;&quot;+record.id, record.seq, sep=&quot;\n&quot;)&#xA;&#xA;&#xA;def get_args():&#xA;    parser = ArgumentParser(description=&quot;&quot;)&#xA;    parser.add_argument(&quot;-i&quot;, &quot;--input&quot;, help=&quot;input fasta&quot;, required=True)&#xA;    parser.add_argument(&quot;-n&quot;, &quot;--number&quot;, help=&quot;number of longest transcript per gene&quot;, type=int, required=True)&#xA;    return parser.parse_args()&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Run it like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python longest.py -i input.fasta -n 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1310" LastEditorUserId="1310" LastEditDate="2018-12-21T14:04:11.763" LastActivityDate="2018-12-21T14:04:11.763" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6713" PostTypeId="2" ParentId="4541" CreationDate="2018-12-21T22:20:31.400" Score="5" Body="&lt;p&gt;Question 1: The additional sequences is needed because that is complimentary to the P5 sequence anchored to the flow cell. It is also the site for priming the Index 2 read on a MiSeq. The additional sequence is also needed for the read 1 primer in the cartridge to anneal to the correct place for the molecules.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question 2: The &quot;GT&quot; sequence you're referring to is necessary for the Index 1 and Read 2 primer to anneal to the correct place.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LHZV0.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LHZV0.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3878" LastActivityDate="2018-12-21T22:20:31.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6714" PostTypeId="2" ParentId="6676" CreationDate="2018-12-23T12:07:59.537" Score="0" Body="&lt;p&gt;The universal coding way to do this is the &lt;code&gt;printf&lt;/code&gt; command regardless of language (sprintf is used language depending), for Python this is,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blength = 0.22342&#xA;printf(&quot;Taxa A, blength=%.2f&quot;, blength)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taxa A, 0.22&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The following may be useful to your context:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy&#xA;allclades = list(pars_tree.find_clades(order='level'))&#xA;for blength in enumerate(allclades):&#xA;        printf(blength=%.2f, blength)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Goodluck&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2018-12-23T12:07:59.537" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6715" PostTypeId="2" ParentId="6704" CreationDate="2018-12-23T22:56:35.087" Score="2" Body="&lt;p&gt;You've hit the RAM bottleneck. Its very common in de novo assemblies and resolved using template assemblies. I don't know this program but suspect it is performing a pairwise analysis for every piece of data in a genomics data set.&lt;/p&gt;&#xA;&lt;p&gt;The solution is to either:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;find a bigger machine, e.g. a big multi-core machine and make sure the program distributes across every core.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;parallelise the calculation (an excellent way to overcome RAM problems), or,&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;find an approximation to the calculation that is less computationally demanding, templates are a good way to circumvent this.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2018-12-23T22:56:35.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6716" PostTypeId="1" CreationDate="2018-12-24T01:52:08.683" Score="4" ViewCount="76" Body="&lt;h3&gt;Context&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I'm an MSc student working on writing up my thesis (back home now) from my laptop and, therefore, unfortunately don't have access to a workstation/server capable of doing the tBLASTn search that I wanted to do. As a result I have been trying to use as many online resources as possible. &lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Question&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Is there a way to export the tBLASTn results from the NCBI website to a simple text file? I want the output to be something similar to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FASTA_Name Max_score Total_score Query_cover E_value Identity Accession Gene_ID&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Question Details &amp;amp; requirements&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Essentially, I have a yeast  &lt;a href=&quot;https://yeastmine.yeastgenome.org/yeastmine/begin.do&quot; rel=&quot;nofollow noreferrer&quot;&gt;database&lt;/a&gt; with gene expression information (where the gene names are named in the &lt;em&gt;S. cerevisiae&lt;/em&gt; convention &lt;sup&gt;eg&lt;/sup&gt; YMR105C or YDL204W) and I'm trying to relate each of those expression profiles to their respective homologous gene in my species of yeast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've exported the systemic file names (and their respective protein residue sequences) for &lt;em&gt;S. cerevisiae&lt;/em&gt;, turned that into a big fasta file (~6,000 sequences) and then run a tBLASTn against my yeast genome to give a single Report page where you select each submitted FASTA sequence from the &quot;Results for:&quot; dropdown and it brings up the alignments for that specified sequence as normal. To manually curate this would take forever and I was wondering if there's a way to generate a table containing all the results where the E-value is below a certain threshold?&#xA;Ideally it'd be something along the lines of:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;FASTA Name&quot;  &quot;Max score&quot;  &quot;Total score&quot;  &quot;Query cover&quot;  &quot;E value&quot; &quot;Identity&quot; &#xA; &quot;Accession&quot; &quot;Gene*&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*where &quot;Gene&quot; is what you get when you get the title of the page if you were to click that specific &quot;Accession&quot; hyperlink... appreciate that might be beyond the scope of this question though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this possible (or indeed is there a completely different strategy) or should do I need to download the relevant databases and run something along the lines of &lt;a href=&quot;https://www.biostars.org/p/5684/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for all your help in advance!&lt;/p&gt;&#xA;" OwnerUserId="3883" LastEditorUserId="2085" LastEditDate="2018-12-24T06:41:58.707" LastActivityDate="2018-12-24T06:41:58.707" Title="How to export web NCBI tBLASTn results in table format with many queries?" Tags="|public-databases|phylogenetics|blastp|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6717" PostTypeId="1" AcceptedAnswerId="6719" CreationDate="2018-12-24T05:15:52.993" Score="10" ViewCount="715" Body="&lt;p&gt;As we know that in human DNA sequence, one copy of chromosome comes from mother's DNA and another copy comes from father's DNA so as to form two copies of each chromosome in human DNA. So, if we extract exome sequence from its DNA then each exome must also have two copies of each chromosome (one from mother and another from father). But in either fastq file or bam file in WES data, I always find only one copy of each chromosome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q1:&lt;/strong&gt; Where is the other gene copy in the sequence or have I have missed something? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q2:&lt;/strong&gt; How can I check the ploidy and find if there are two copies of each chromosome in WES data? How can I do this if my WES data are mapped to the reference and stored in a bam file?&lt;/p&gt;&#xA;" OwnerUserId="2141" LastEditorUserId="2085" LastEditDate="2018-12-24T06:45:43.153" LastActivityDate="2018-12-24T14:48:12.190" Title="Why does this human bam file only have one copy of each chromosome?" Tags="|sam|ngs|fastq|exome|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6718" PostTypeId="2" ParentId="6716" CreationDate="2018-12-24T06:00:11.337" Score="5" Body="&lt;p&gt;It sounds like your easiest option is to just install blast on your local machine and set up the databases there. It really will only take a few minutes to blast 6,000 sentences against a yeast genome on a modern laptop. Any solution interfacing with web results or parsing the output will be painfully convoluted. There are executables for blast that will work on Mac, Linux, or Windows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some beginning instructions for installing it on your laptop can be found here: &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&amp;amp;PAGE_TYPE=BlastDocs&amp;amp;DOC_TYPE=Download&quot; rel=&quot;noreferrer&quot;&gt;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&amp;amp;PAGE_TYPE=BlastDocs&amp;amp;DOC_TYPE=Download&lt;/a&gt; . Make sure to pass the --num_threads option to speed up the search. &lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2018-12-24T06:00:11.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6719" PostTypeId="2" ParentId="6717" CreationDate="2018-12-24T06:21:41.077" Score="16" Body="&lt;p&gt;The maternal and paternal copies of a chromosome are called &lt;strong&gt;haplotypes&lt;/strong&gt;. Many metazoans (animals) are diploid and have maternal and paternal chromosome contribution during sexual reproduction, not just humans as your question states. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Response to Q1&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Your question, in other words, is: Why do &lt;code&gt;.bam&lt;/code&gt; files not differentiate between haplotypes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your question gets to a more fundamental kernel of how most people do &quot;genomics&quot; today. When most people assemble a genome or create a reference genome, they actually just produce a fasta file that contains a single sequence for each chromosome. This, of course, is not biologically accurate as there are two very distinct sequences per chromosome. Most genomes only report a chimeric consensus between the two and call this a reference. These reference genomes are haploid, or haplotype-collapsed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;This is where your question comes in:&lt;/strong&gt; You have mapped transcripts that biologically are derived from two different haplotypes to a reference genome that only contains one consensus sequence that (inadequately) represents both haplotypes. &lt;strong&gt;As a result, at a single locus in the bam file there will be mapped reads from both haplotypes.&lt;/strong&gt; If your reference fasta file contained the exact sequence of both haplotypes, if you mapped reads to the reference, and if you only look at primary alignments, the reads will mostly map to their haplotype-of-origin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This gets into another topic called &lt;strong&gt;phasing&lt;/strong&gt;, wherein the order and orientation of most of the polymorphisms unique to each haplotype are determined using sequencing data. There are some problems with this as it relies on properly detecting variant sites. Software like &lt;a href=&quot;https://software.broadinstitute.org/gatk/&quot; rel=&quot;noreferrer&quot;&gt;GATK&lt;/a&gt; and others can find single nucleotide polymorphisms (SNPs) if there is good sequencing coverage, however detecting insertions and deletions is much more difficult. This gives a very SNP-biased view of the haplotype differences in any given genome. After finding variant sites, phasing software like  &lt;a href=&quot;https://github.com/vibansal/HapCUT2&quot; rel=&quot;noreferrer&quot;&gt;hapcut2&lt;/a&gt; determines which variants fall on which haplotype and outputs blocks of variants that are thought to belong to the same haplotype.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Phasing alone is not enough to accurately reconstruct the exact sequence of both haplotypes due to the inability to detect all variants with read mapping. The gold standard of the future is &lt;strong&gt;diploid de novo genome assembly&lt;/strong&gt;, in which each haplotype is assembled independently. This is an active area of research for people who develop genome assemblers and is tightly linked to the advances in PacBio and Oxford Nanopore long reads. &lt;a href=&quot;https://www.biorxiv.org/content/biorxiv/early/2018/02/26/271486.full.pdf&quot; rel=&quot;noreferrer&quot;&gt;This paper on trio canu is a good start for learning about one successful dioploid assembly technique.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Response to Q2&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If you want to check ploidy of all of the chromosomes then you need at least 10X coverage of whole-genome shotgun data and a tool like &lt;a href=&quot;https://github.com/tbenavi1/smudgeplot&quot; rel=&quot;noreferrer&quot;&gt;smudgeplot&lt;/a&gt; and &lt;a href=&quot;http://qb.cshl.edu/genomescope/&quot; rel=&quot;noreferrer&quot;&gt;genomescope&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are trying to check local duplications or whole-chromosome duplications for something like a cancer sample you would also need whole-genome shotgun data. WES data does not give reliable information on ploidy since the read count at a given locus is dependent on the transcription level, not the actual quantity of homologous chromosomal DNA for the region in question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are trying to (use a bam file to) look for evidence of chromosomal duplication or duplication of a locus in cancer samples you would look for an increase in whole genome shotgun coverage at that locus compared to a known normal locus. For example if all chromosomes but one have a coverage of 22, and one chromosome has an average coverage of 33, this is evidence of a trisomy. This logic can be applied to smaller regions as well (barring repetitive regions, paralogs, et cetera).&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="96" LastEditDate="2018-12-24T14:48:12.190" LastActivityDate="2018-12-24T14:48:12.190" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6720" PostTypeId="1" CreationDate="2018-12-24T07:58:37.730" Score="2" ViewCount="305" Body="&lt;p&gt;I have used multiple sequences aligned file of protein data to generate a maximum parsimony tree, then I used the &lt;code&gt;&quot;bootstrap_trees(msa, times, tree_constructor)&quot;&lt;/code&gt; function from biopython to generate bootstrap replicates of my MSA data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have my original tree and bootstrap replicates for my tree, I want to ask how do I compare the topology (using python/biopython) for my tree with the bootstrap replicates so that for each edge in original tree the bootstrap support is shown. (Bootstrap support is the number of bootstrap trees that edge appears in.)&lt;/p&gt;&#xA;" OwnerUserId="3142" LastEditorUserId="3848" LastEditDate="2020-04-20T17:27:28.893" LastActivityDate="2020-04-20T17:27:28.893" Title="How to get bootstrap support of phylogenetic tree?" Tags="|phylogenetics|python|phylogeny|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6722" PostTypeId="2" ParentId="6676" CreationDate="2018-12-24T15:26:32.447" Score="1" Body="&lt;p&gt;Apologies, try the following,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;allclades = list(pars_tree.find_clades(order='level'))&#xA;for i, blength in enumerate(allclades):&#xA;        printf(blength=%.2f, blength)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&quot;allclades&quot; is a dictionary rather than a simple array.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this fails I would examine the allclades data structure either using &lt;code&gt;pprint&lt;/code&gt; (directly) or using &lt;code&gt;printStruct&lt;/code&gt; via the following code, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; d = [{'a1':1, 'a2':2, 'a3':3}, [1,2,3], [{'b1':1, 'b2':2}, {'c1':1}], 'd1', 'd2', 'd3']&#xA;&amp;gt;&amp;gt;&amp;gt; printStruct(d)&#xA;&#xA;def printStruct(struc, indent=0):&#xA;  if isinstance(struc, dict):&#xA;    print '  '*indent+'{'&#xA;    for key,val in struc.iteritems():&#xA;      if isinstance(val, (dict, list, tuple)):&#xA;        print '  '*(indent+1) + str(key) + '=&amp;gt; '&#xA;        printStruct(val, indent+2)&#xA;      else:&#xA;        print '  '*(indent+1) + str(key) + '=&amp;gt; ' + str(val)&#xA;    print '  '*indent+'}'&#xA;  elif isinstance(struc, list):&#xA;    print '  '*indent + '['&#xA;    for item in struc:&#xA;      printStruct(item, indent+1)&#xA;    print '  '*indent + ']'&#xA;  elif isinstance(struc, tuple):&#xA;    print '  '*indent + '('&#xA;    for item in struc:&#xA;      printStruct(item, indent+1)&#xA;    print '  '*indent + ')'&#xA;  else: print '  '*indent + str(struc)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Once you know the structure you unravel it accordingly to get to the branch lengths and then issue a printf command to round them to 2 dp.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Goodluck&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2018-12-24T15:31:35.387" LastActivityDate="2018-12-24T15:31:35.387" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6723" PostTypeId="1" CreationDate="2018-12-25T15:15:16.357" Score="1" ViewCount="189" Body="&lt;p&gt;I'm trying to find the amino acid context around a particular position in a protein (5 aa amino acid window on each side).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used to have some code that works and then biomaRt stopped working. I think some of these issues would be fixed in later versions of biomaRt, but I would have to install a new version of R, which would require me to install a new version of my OS.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;require('biomaRt')&#xA;require('Biostrings')&#xA;hs_ensembl_mart = &#xA;useMart(biomart=&quot;ENSEMBL_MART_ENSEMBL&quot;,dataset=&quot;hsapiens_gene_ensembl&quot;)&#xA;&#xA;gene_names=c('KRAS')&#xA;aa_affected_loc=12&#xA;&#xA;biomart_query_results=getBM(attributes=c('hgnc_symbol', 'entrezgene','coding'), &#xA;  filters = 'hgnc_symbol', &#xA;  values = gene_names, &#xA;  mart = hs_ensembl_mart)&#xA;&#xA;preceding_peptide= substr(as.character(translate(DNAString(biomart_query_results[1,'coding']))),aa_affected_loc-5,aa_affected_loc-1)&#xA;&#xA;trailing_peptide= substr(as.character(translate(DNAString(biomart_query_results[1,'coding']))),aa_affected_loc+1,aa_affected_loc+5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does anyone have a workaround for this?&lt;/p&gt;&#xA;" OwnerUserId="673" LastEditorUserId="673" LastEditDate="2019-01-05T14:00:58.987" LastActivityDate="2019-01-08T23:12:26.160" Title="How to obtain amino acid context at a given position" Tags="|r|biomart|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6724" PostTypeId="1" AcceptedAnswerId="6727" CreationDate="2018-12-26T12:50:16.153" Score="4" ViewCount="231" Body="&lt;p&gt;I recently started doing the analysis on nanopore sequencing data. As I was searching for some help on pre-processing of the data, I found your nice setup pipeline created here: &lt;a href=&quot;https://www.protocols.io/view/stranded-mapping-from-long-reads-n8cdhsw&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.protocols.io/view/stranded-mapping-from-long-reads-n8cdhsw&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I was trying to follow the instruction, I ran into a question on part where it uses lastal to divide the input file as strand specific output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Part that uses the following script below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;lastal -P 10 ONT_barcodes_adapters.fa &amp;lt;(zcat 4T1_BC06.correctedReads.uniqueOnly.fasta.gz) | \  ~/scripts/maf_bcsplit.pl | grep -e 'ONT_SSP,+' -e 'ONT_VNP,-' | \   awk -F',' '{print &lt;span class=&quot;math-container&quot;&gt;$1}' | sort | uniq &amp;gt; fwdQry_seqs_BC06.txt lastal -P 10 barcodes_primerSeqs.fa &amp;lt;(zcat 4T1_BC06.correctedReads.uniqueOnly.fasta.gz) | \   ~/scripts/maf_bcsplit.pl | grep -e 'ONT_SSP,-' -e 'ONT_VNP,+' | \   awk -F',' '{print $&lt;/span&gt;1}' | sort | uniq &amp;gt; revQry_seqs_BC06.txt &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In here, I was wondering what the ONT_barcodes_adapters.fa file actually looks like. I wanted to find the information in other places and could not find it elsewhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you could help me with this, I would really appreciate it!&lt;/p&gt;&#xA;" OwnerUserId="3885" LastEditorUserId="964" LastEditDate="2018-12-26T14:52:22.053" LastActivityDate="2018-12-27T09:09:56.780" Title="Question on nanopore sequencing data process pipeline (cDNA-PCR)" Tags="|nanopore|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6726" PostTypeId="1" CreationDate="2018-12-27T08:38:51.270" Score="2" ViewCount="651" Body="&lt;p&gt;I have written some codes in R for my personal use. The package successfully installed on my personal laptop (where I have written those commands) and working correctly. But when I distributed it to my other colleagues, this package is not installing and returned these errors. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In R CMD INSTALL &#xA;&#xA;Warning in install.packages : running command&#xA;&#xA;&quot;C:/PROGRA~1/R/R-34~1.3/bin/x64/R&quot; CMD INSTALL -l &quot;C:\Program&#xA;&#xA;Files\R\R-3.4.3\library&quot; &quot;E:/test/Gene_1.0.tar.gz&quot;' had status 1&#xA;&#xA;Warning in install.packages :   installation of package&#xA;&#xA;E:/test/Gene_1.0.tar.gz had non-zero exit status&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Secondly, I ran &quot;CHECK PACKAGE&quot; command in R-studio. Those results expressed two errors and two notes.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; checking DESCRIPTION meta-information ... WARNING&#xA;  Dependence on R version '3.5.1' not with patchlevel 0&#xA;&#xA;&amp;gt; checking for unstated dependencies in examples ... OK&#xA;   WARNING&#xA;  'qpdf' is needed for checks on size reduction of PDFs&#xA;0 errors v | 2 warnings x | 2 notes x&#xA;&#xA;Error: R CMD check found WARNINGs&#xA;&#xA;Execution halted&#xA;&#xA;Exited with status 1.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Kindly help me out with these two issues.&lt;/p&gt;&#xA;" OwnerUserId="3626" LastActivityDate="2018-12-27T08:38:51.270" Title="R package CMD check errors" Tags="|r|bioconductor|rstudio|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6727" PostTypeId="2" ParentId="6724" CreationDate="2018-12-27T09:09:56.780" Score="1" Body="&lt;p&gt;I recommend that you look at the most recent version of the protocol, which includes example adapter and barcode files:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.protocols.io/view/stranded-mapping-from-long-reads-syheft6?step=2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.protocols.io/view/stranded-mapping-from-long-reads-syheft6?step=2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[check in the protocol to see if there's a more recent version]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These files contains the public sequences that ONT have released for their sequencing kits (as briefly mentioned in the &quot;Before Starting&quot; section of the protocol). The sequences are available on the ONT community, but I have converted them into FASTA files for use in my protocol.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The adapter and barcode files were combined into a single FASTA file in the previous version of my protocol. For the PCR cDNA barcoding, it looked something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;ONT_VNP&#xA;ACTTGCCTGTCGCTCTATCTTCTTTTT&#xA;&amp;gt;ONT_SSP&#xA;TTTCTGTTGGTGCTGATATTGCTGCCATTACGGCCGGG&#xA;&amp;gt;BC01&#xA;AAGAAAGTTGTCGGTGTCTTTGTG&#xA;&amp;gt;BC02&#xA;TCGATTCCGTTTGTAGTCGTCTGT&#xA;&amp;gt;BC03&#xA;GAGTCTTGTGTCCCAGTTACCAGG&#xA;&amp;gt;BC04&#xA;TTCGGATTCTATCGTGTTTCCCTA&#xA;&amp;gt;BC05&#xA;CTTGTCCAGGGTTTGTGTAACCTT&#xA;&amp;gt;BC06&#xA;TTCTCGCAAAGGCAGAAAGTAGTC&#xA;&amp;gt;BC07&#xA;GTGTTACCGTGGGAATGAATCCTT&#xA;&amp;gt;BC08&#xA;TTCAGGGAACAAACCAAGTTACGT&#xA;&amp;gt;BC09&#xA;AACTAGGCACAGCGAGTCTTGGTT&#xA;&amp;gt;BC10&#xA;AAGCGTTGAAACCTTTGTCCTCTC&#xA;&amp;gt;BC11&#xA;GTTTCATCTATCGGAGGGAATGGA&#xA;&amp;gt;BC12&#xA;CAGGTAGAAAGAAGCAGAATCGGA&#xA;&amp;gt;RB12A&#xA;GTTGAGTTACAAAGCACCGATCAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I separate out adapters and barcodes in the newer versions of the protocol because I was getting too many incorrect hits.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2018-12-27T09:09:56.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6730" PostTypeId="1" AcceptedAnswerId="6731" CreationDate="2018-12-27T19:01:37.417" Score="5" ViewCount="89" Body="&lt;p&gt;Does Prokka do six-frame genome annotations? The corresponding publication does not mention it.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Seemann, Torsten. 2014. “Prokka: Rapid Prokaryotic Genome Annotation.” Bioinformatics  30 (14): 2068–69.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="48" LastEditDate="2019-07-02T15:54:55.577" LastActivityDate="2019-07-02T15:54:55.577" Title="Does Prokka do six-frame translations?" Tags="|sequence-annotation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6731" PostTypeId="2" ParentId="6730" CreationDate="2018-12-27T21:42:52.467" Score="7" Body="&lt;p&gt;Yes, Prokka will consider all six-frames, the gene prediction is actually done with Prodigal, paper:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2848648/&quot; rel=&quot;noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2848648/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;github:&#xA;&lt;a href=&quot;https://github.com/hyattpd/Prodigal&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/hyattpd/Prodigal&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2018-12-27T21:42:52.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6732" PostTypeId="1" CreationDate="2018-12-28T05:03:49.470" Score="5" ViewCount="1057" Body="&lt;p&gt;I wish to perform differential expression analysis for cluster-specific gene expression in single-cell data (with a tool such as MAST or SCDE).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have data for 3 biological replicates. I performed analysis on 10x data with cellranger and have filtered out cells with low UMIs. I have corrected for batch effects and identified clusters by performing graph-based clustering on the UMAP projections.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I understand it, the components from batch effect correction algorithms are &lt;em&gt;only&lt;/em&gt; intended to be used for the purpose of plotting and clustering. They are &lt;em&gt;not&lt;/em&gt; to be used for analysis of individual gene expression or differential expression. This has been stated unequivocally by the original authors of MNN and CCA batch correction algorithms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have derived a UMI count matrix for each sample which corrects for amplification bias. Differences between the cells and samples still persist in the raw data. These are obvious from the mean UMI counts to the UMAP of uncorrected data. The number of UMIs differs considerably between samples (batches) and individual cells within a sample (the total UMI per cell). The amount of data from each of the cells varies considerably. The read-depth between samples is also different. Therefore, I need a simple approach to correct for differences in read-depth, total UMI or sensitivity between cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to normalise UMI counts between cells? Should UMI count data be treated differently to reads per transcript in bulk RNA-Seq? Would it be more appropriate to do so at the level of individual cells or for batches across samples?&lt;/p&gt;&#xA;" OwnerUserId="1855" LastActivityDate="2018-12-28T15:21:21.613" Title="How to normalise scRNASeq data for differential expression analysis" Tags="|scrnaseq|differential-expression|single-cell|normalization|10x-genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6733" PostTypeId="2" ParentId="6732" CreationDate="2018-12-28T15:21:21.613" Score="3" Body="&lt;p&gt;I would suggest using a likelihood ratio test for differential expression using logistic regression with batch as a latent variable. In Seurat you can do:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;markers &amp;lt;- FindAllMarkers(object, test.use = 'LR', latent.vars = 'batch')&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(change &quot;object&quot; and &quot;batch&quot; accordingly)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://www.biorxiv.org/content/early/2018/02/14/258566&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/early/2018/02/14/258566&lt;/a&gt; from Lior Pachter's group for some more info on the LR test.&lt;/p&gt;&#xA;" OwnerUserId="3415" LastActivityDate="2018-12-28T15:21:21.613" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6734" PostTypeId="2" ParentId="5618" CreationDate="2018-12-28T20:39:15.773" Score="1" Body="&lt;p&gt;Amino acid matrices are contructed by maximum likelihood and easily performed using PAML. You present a known tree and an amino acid alignment and ask what amino acid matrix would maximise the likelihood. Hence all amino acid matrices using this old school approach are dependent on the alignment composition. JTT was generally regarded as obtaining the highest likelihood. However, I reitereate that things have moved on a lot since this 2-D reversible amino acid substitution matrix was devised.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2018-12-28T20:39:15.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6735" PostTypeId="2" ParentId="4531" CreationDate="2018-12-28T21:35:02.293" Score="1" Body="&lt;p&gt;Yes this approach is done in parsimony, i.e. you can indicate the outgroup and the ancestral state is calculated from there. I assume this is also performed using 'Beast' because it is a non-reversible matrix. Regular ML and Bayes approaches use reversible mutation matrices therefore the tree is calculated first, you can then specify an outgroup to calculate the ancestral state. However, in a reversible matrix the ancestral state is not used in the calcuation. I hope that makes sense.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2018-12-28T21:35:02.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6736" PostTypeId="1" AcceptedAnswerId="6738" CreationDate="2018-12-28T22:05:56.863" Score="3" ViewCount="4963" Body="&lt;p&gt;I am using the following function from seurat package to generate multiple violon plots and I am interested in adding box plots to them but it doesn't work when I have plotted different data at once.&#xA;Is there a way to solve it ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, this works:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Seurat)&#xA;VlnPlot(object = pbmc_small, features.plot = 'PC1') +&#xA;  geom_boxplot()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zsDsn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zsDsn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this will simply lead into an empty box on top of my plots:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;VlnPlot(object = pbmc_small, features.plot = c('PC1', 'PC2')) +&#xA;  geom_boxplot()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oD4Fp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oD4Fp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3901" LastActivityDate="2018-12-31T16:11:12.663" Title="Separate boxplots for multiple violin plot" Tags="|r|scrnaseq|seurat|ggplot2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6737" PostTypeId="2" ParentId="6736" CreationDate="2018-12-29T03:54:48.347" Score="2" Body="&lt;p&gt;The &lt;code&gt;vioplot&lt;/code&gt; package comes built in with boxplots. You can download it from CRAN or there are more features (including formula input and separate colours) in the development version on GitHub: &lt;a href=&quot;https://github.com/TomKellyGenetics/vioplot&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/TomKellyGenetics/vioplot&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;devtools::install_github(“TomKellyGenetics/vioplot”)&#xA;library(“vioplot”)&#xA;vioplot(pbmc_small@dr@pca[,1]~pbmc_small@meta.data$ident)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To plot several PCs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(“RColorBrewer”)&#xA;colours &amp;lt;- brewer.pal(length(unique(pbmc_small@meta.data&lt;span class=&quot;math-container&quot;&gt;$ident)), “Set3”)&#xA;par(mfrow = c(1, 2))&#xA;vioplot(pbmc_small@dr@pca[,1]~pbmc_small@meta.data$&lt;/span&gt;ident, main = “PC1”, xlab = “Identity”, col = colours)&#xA;vioplot(pbmc_small@dr@pca[,2]~pbmc_small@meta.data$ident , main = “PC2”, xlab = “Identity”, col = colours)&#xA;par(mfrow = c(1, 1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1855" LastActivityDate="2018-12-29T03:54:48.347" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6738" PostTypeId="2" ParentId="6736" CreationDate="2018-12-31T16:11:12.663" Score="2" Body="&lt;p&gt;This happens because the violin plots are combined using &lt;code&gt;cowplot::plot_grid&lt;/code&gt; before being returned by VlnPlot. You can prevent the plots from being combined by setting &lt;code&gt;combine=FALSE&lt;/code&gt;, then modify each one by adding a boxplot, then combine the modified plots using &lt;code&gt;Seurat::CombinePlots&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the combine argument is currently broken in VlnPlot. I have pushed a fix but it's not on the public branch yet. VlnPlot is just a wrapper around ExIPlot (expression by identity plot) in Seurat v3 so right now this will work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Seurat)&#xA;library(ggplot2)&#xA;&#xA;plots &amp;lt;- ExIPlot(object = pbmc_small,&#xA;                 features = c('PC1', 'PC2'),&#xA;                 pt.size = 1, &#xA;                 combine = FALSE)&#xA;&#xA;for(i in 1:length(plots)) {&#xA;  plots[[i]] &amp;lt;- plots[[i]] + geom_boxplot() + theme(legend.position = 'none')&#xA;}&#xA;CombinePlots(plots)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NwrRy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NwrRy.png&quot; alt=&quot;VlnPlot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note this answer is for Seurat v3&lt;/p&gt;&#xA;" OwnerUserId="3415" LastActivityDate="2018-12-31T16:11:12.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6739" PostTypeId="1" AcceptedAnswerId="6740" CreationDate="2019-01-01T10:16:26.307" Score="3" ViewCount="994" Body="&lt;p&gt;I am running a multiple comparison using the non-parametric Kruskal Wallis test (K-W), using the ggpubr library and I am a bit confused about the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When i just run the KW-test using &quot;base R&quot; the result is different and I am not sure if there is an issue with the data or I am doing something wrong. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case of ggpubr im getting significant result, i mean the p value is much lower than the critical threshold, but in case of base R this does not occer. I am posting the code and results figure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion would be highly appreciated why results are varying ..&#xA; &lt;a href=&quot;https://drive.google.com/open?id=10vanPZEW77qfImINZvoYiAc8CH2Y0XOi&quot; rel=&quot;nofollow noreferrer&quot;&gt;Data&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ApzVd.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ApzVd.jpg&quot; alt=&quot;K-W test output &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(ggpubr)&#xA;df1 = read.csv(&quot;NEW_RBP/CORPLOT/RBP_DISEASE_CLUSTER/RBP_DISEASE_C1_C4_AVG.txt&quot;,header = TRUE,sep = &quot;\t&quot;)&#xA;head(df1)&#xA;my_comparisons &amp;lt;- list( c(&quot;HSC&quot;, &quot;LSC&quot;), c(&quot;LSC&quot;, &quot;Blast&quot;), c(&quot;HSC&quot;, &quot;Blast&quot;) )&#xA;&#xA;ex &amp;lt;- melt(df1, id.vars=c(&quot;gene&quot;))&#xA;&#xA;head(ex)&#xA;&#xA;ggboxplot(ex, x = &quot;variable&quot;, y = &quot;value&quot;,ggtheme = theme_bw(base_size = 30),&#xA;          color = &quot;variable&quot;, palette = &quot;jco&quot;, add = &quot;jitter&quot;)+ &#xA;  stat_compare_means(comparisons = my_comparisons)+ # Add pairwise comparisons p-value&#xA;  stat_compare_means(label.y = 50)     # Add global p-value&#xA;&#xA;&#xA;&#xA;&#xA;kruskal.test(HSC ~ LSC, data = df1)&#xA;&#xA;    Kruskal-Wallis rank sum test&#xA;&#xA;data:  HSC by LSC&#xA;Kruskal-Wallis chi-squared = 416, df = 416, p-value = 0.4908&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2019-01-02T09:02:21.073" LastActivityDate="2019-01-03T22:24:17.953" Title="Kruskal-Wallis rank sum test output confusion" Tags="|r|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6740" PostTypeId="2" ParentId="6739" CreationDate="2019-01-02T01:28:35.697" Score="0" Body="&lt;p&gt;The OP says KW is giving different results between &quot;base R&quot; and ggpubr. Background information: KW test is a non-parametric (assumption free) version of 1-way ANOVA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The analysis you have presented provides a clear result: HSC and Blast are of an approximately equivalent distribution, however LSC is an outlier to both HSC and Blast. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The global value is very highly significant (because of LSC). In other words the global value (HSC, Blast and LSC used in a single analysis) is a much lower probability than the pairwise tests because it will amplify the discrepancy between LSC-HSC and LSC-Blast. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I personally don't see any discrepancy in behaviour within this analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suspect that HSC and Blast might be significantly different in different analyses because the result is quite low p=0.14, although it isn't borderline (e.g. if p=0.5 I would be happy). The basic issue in my opinion is that non-parametric tests are not particularly powerful and p=0.14 isn't hugely above p=0.05.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this concerns you, I would examine the HSC and Blast groups using 1-way ANOVA and see if this sheds light on the problem, i.e. the global test will also be rejected by 1-way ANOVA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1-way ANOVA requires the assumption of normality, but would be supportive of the KW-result. If 1-way ANOVA resulted in HSC and Blast being not significantly different from each other, that is your answer because 1-way ANOVA is a more powerful test (i.e, statistical power), thus it is more likely to reject a relationship between HSC and Blast.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-01-02T02:35:22.420" LastActivityDate="2019-01-02T02:35:22.420" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6741" PostTypeId="2" ParentId="6739" CreationDate="2019-01-02T10:21:29.673" Score="2" Body="&lt;p&gt;Your problem comes from using an incorrect test, the formula parameter says:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;a formula of the form response ~ group where response gives the data values and group a vector or factor of the corresponding groups.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;When you apply the kruskal  test, you don't have groups in the LSC variables! You aren't comparing the mean of several groups based on LSC!&#xA;When you use &lt;code&gt;stat_compare_means&lt;/code&gt; it is doing a wilcox.test (it hints to it in the help page &quot;a list of additional arguments used for the test method. For example one might use method.args = list(alternative = &quot;greater&quot;) for wilcoxon test.&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you use the wilcox.test we get the same p-value:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wilcox.test(df1&lt;span class=&quot;math-container&quot;&gt;$HSC, df1$&lt;/span&gt;LSC)&#xA;&#xA;    Wilcoxon rank sum test with continuity correction&#xA;&#xA;data:  df1&lt;span class=&quot;math-container&quot;&gt;$HSC and df1$&lt;/span&gt;LSC&#xA;W = 110980, p-value = 4.849e-12&#xA;alternative hypothesis: true location shift is not equal to 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In addition, probably you should read about the paired argument, if this two values where measured on the same samples you should use it (but I can't tell it for sure because I don't know what does it mean each variable name). &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2019-01-02T10:21:29.673" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6743" PostTypeId="2" ParentId="5049" CreationDate="2019-01-02T15:09:05.303" Score="3" Body="&lt;p&gt;Try this Docker &lt;a href=&quot;https://hub.docker.com/r/chambm/pwiz-skyline-i-agree-to-the-vendor-licenses&quot; rel=&quot;nofollow noreferrer&quot;&gt;container&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the vendor formats work except T2D, YEP, and FID.&lt;/p&gt;&#xA;" OwnerUserId="3913" LastEditorUserId="964" LastEditDate="2019-01-02T15:31:20.417" LastActivityDate="2019-01-02T15:31:20.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6744" PostTypeId="2" ParentId="5049" CreationDate="2019-01-02T19:35:36.650" Score="1" Body="&lt;p&gt;I sympathize ...&#xA;If it is a Mac just run 'PlayonMac' its a brilliant frontend for Wine without all the bugs and hassles it causes. Its free and available &lt;a href=&quot;https://www.playonmac.com/en/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-01-02T19:35:36.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6745" PostTypeId="2" ParentId="6739" CreationDate="2019-01-03T15:03:10.717" Score="2" Body="&lt;p&gt;Just to mention the KW test is appropriate here and not an error as suggested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each variable is a compartment and you have 3 compartments. There is no satisfactory equivalent to a non-parametric 2-way ANOVA (possible Friedman test), which is what is being implied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wilcoxon will only compare 2 samples, there is no restriction in the number under KW.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are comparing alot of samples, KW is much preferred because of type 2 error which will result from Wilcoxon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would present both results,KW and Wilcoxon. The only error was that we didn't know what test was run in base R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore if you can split each sample into say two compartments, then you would run KW on each compartment (ie run it twice), at this point Wilcoxon becomes erroneous because you would need to run it 6 times at 0.05 critical probability and the chance of a false positive (two data sets are different when they are not) becomes statistically significant. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-01-03T22:24:17.953" LastActivityDate="2019-01-03T22:24:17.953" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6746" PostTypeId="1" CreationDate="2019-01-03T19:41:18.470" Score="3" ViewCount="198" Body="&lt;p&gt;I have been working on RNA-Seq data from two different cohorts, and they show very strong batch effect (~35% variance explained by 1st component in PCA). Since I am trying to do a class discovery from a data set with the subtype of only some samples are known, the only methods I have been using are ComBat and pSVA from SVA package. I would like to compare them with more methods to make the clustering result more trust-worthy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found that some papers used sva package, but not citing the paper on pSVA, meaning perhaps there are some other ways to use sva for this type of data that I am not aware of. Is there a method to construct a null model and a full model for sva only by knowing the batch the samples are from?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any other suggestion of batch effect correction tools for a data set with no known phenotype/subtype are also welcome.&lt;/p&gt;&#xA;" OwnerUserId="3225" LastActivityDate="2021-06-27T08:03:10.303" Title="sva for RNA-Seq data without known phenotype" Tags="|rna-seq|clustering|batch-effects|combat|sva|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6747" PostTypeId="2" ParentId="5635" CreationDate="2019-01-05T01:57:30.633" Score="0" Body="&lt;p&gt;You can create a script to pass arguments to as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#! /bin/bash&#xA;&#xA;bwa mem -M -R &quot;@RG\tID:&lt;span class=&quot;math-container&quot;&gt;${1}\tLB:$&lt;/span&gt;{2}\tPL:ILLUMINA\tPM:HiSeq2000\tSM:&lt;span class=&quot;math-container&quot;&gt;${1}&quot; ../ref/M._tuberculosis_H37Rv_2015-11-13.fasta $&lt;/span&gt;{1}_* &amp;gt; ${1}.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now if you run &lt;code&gt;bash script.sh sample lib&lt;/code&gt; then &lt;code&gt;sample&lt;/code&gt; and &lt;code&gt;lib&lt;/code&gt; will be  the script as arguments &lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;${1}&lt;/code&gt; and &lt;code&gt;$&lt;/span&gt;{2}&lt;/code&gt; respectively. Then all you need is a loop to pass arguments to the script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for sample in sample1 sample2 sample3&#xA;   do&#xA;   for lib in lib1 lib2 lib3&#xA;     do&#xA;     bash script.sh &lt;span class=&quot;math-container&quot;&gt;$sample $&lt;/span&gt;lib&#xA;     done&#xA;   done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can use &lt;code&gt;nohup&lt;/code&gt; to run each sample in the background and proceed with next one. This will in essence, submit each command as a separate job to run on a different core. These will print to separate log files sample.out instead of to the terminal. It will also continue to run, even if you disconnect from SSH or close the terminal.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for sample in sample1 sample2 sample3&#xA;   do&#xA;   for lib in lib1 lib2 lib3&#xA;     do&#xA;     nohup bash script.sh &lt;span class=&quot;math-container&quot;&gt;$sample $&lt;/span&gt;lib &amp;gt; ${sample}.out &amp;amp;&#xA;     done&#xA;   done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You should only do this for single-threaded processes. For multi-threaded processes or memory-intensive jobs, this will demand a lot from the machine. Even if this isn’t a shared server, it will perform poorly if you run out of memory and have to “swap” memory by writing to disk. So it’s best to submit them one-by-one (above) or use a cluster with a job scheduler. &lt;/p&gt;&#xA;" OwnerUserId="1855" LastActivityDate="2019-01-05T01:57:30.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6748" PostTypeId="2" ParentId="6723" CreationDate="2019-01-05T14:02:53.893" Score="2" Body="&lt;p&gt;It turned out that I needed to reinstall OSX to upgrade my R version and biomaRt install.&lt;/p&gt;&#xA;" OwnerUserId="673" LastActivityDate="2019-01-05T14:02:53.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6749" PostTypeId="1" CreationDate="2019-01-06T10:52:07.423" Score="0" ViewCount="76" Body="&lt;p&gt;I have a Normal esophageal Fibroblasts (NOFs) cultured in DMEM media; The same NOF also have been cultured with a tumor sample from a patient named 005 on DMEM media; I have also Cancer Associated Fibroblasts (CAFs) cultured on DMEM, on DMEM + tumor from a patient named as 005 and DMEM + tumor from a patient named 036 something like this picture;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TwjbK.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TwjbK.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know if NOF and CAF are different in terms of gene expression but I am not sure about design for DESeq2 or Edger and ANOVA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By @Devon's help I have done so&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; mycols&#xA;   cell_type co_cultured&#xA;G2       NOF          NO&#xA;G3       NOF         YES&#xA;G4       CAF          NO&#xA;G5       CAF         YES&#xA;G6       CAF         YES&#xA;&#xA;&#xA;dds=DESeqDataSetFromMatrix(countData = NOFCAF,colData = mycols, design        =~ cell_type*co_cultured) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, I am wondering what would be a suitable contrast for results function in DESeq2&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; resultsNames(dds)&#xA;[1] &quot;Intercept&quot;                   &quot;cell_type_NOF_vs_CAF&quot;        &quot;co_cultured_YES_vs_NO&quot;      &#xA;[4] &quot;cell_typeNOF.co_culturedYES&quot;&#xA;&#xA;&#xA;&#xA;&amp;gt; res=results(dds)&#xA;&amp;gt; res&#xA;log2 fold change (MLE): cell typeNOF.co culturedYES &#xA;Wald test p-value: cell typeNOF.co culturedYES &#xA;DataFrame with 2545 rows and 6 columns&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot for correcting me&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-01-08T12:45:12.717" LastActivityDate="2019-01-08T12:45:12.717" Title="Experimental Design for Differential expreression analysis" Tags="|r|rna-seq|deseq2|edger|design|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="6750" PostTypeId="1" CreationDate="2019-01-06T14:47:26.107" Score="6" ViewCount="767" Body="&lt;p&gt;I'm kind of new to bioinformatics and trying to self-study. I'm reading a bioinformatics book: &lt;em&gt;An Introduction to Bioinformatics Algorithms&lt;/em&gt; and ran into some problems about understanding the proof of the Breakpoint Reversal Sort algorithm in chapter 5.4. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, the book states a theorem: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If a permutation &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt; contains a decreasing strip, then there is a reversal &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; that decreases the number of breakpoints in &lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;, that is, breakpoint (&lt;span class=&quot;math-container&quot;&gt;$\rho(\pi)$&lt;/span&gt;) &amp;lt; breakpoint(&lt;span class=&quot;math-container&quot;&gt;$\pi$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This theorem guarantees that the algorithm will terminate and lead to no breakpoint. The book later explains the proof for the theorem:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; be the smallest element of decreasing strip, so element &lt;span class=&quot;math-container&quot;&gt;$k-1$&lt;/span&gt; would be the ending of an increasing strip. Therefore element &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$k-1$&lt;/span&gt; corresponds to a breakpoint. Then reversing the segment between &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$k-1$&lt;/span&gt; will decrease the number of breakpoints.  &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm quite confused about this proof and don't really understand what the authors mean by reversing the segment between &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$k-1$&lt;/span&gt;. Can someone please explain the proof for me? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the lecture slide about reversal sorting that based on the book &lt;a href=&quot;http://csbio.unc.edu/mcmillan/Media/Lecture07Spring2015.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://csbio.unc.edu/mcmillan/Media/Lecture07Spring2015.pdf&lt;/a&gt;.  &lt;/p&gt;&#xA;" OwnerUserId="3914" LastEditorUserId="4719" LastEditDate="2019-11-11T21:33:47.330" LastActivityDate="2019-12-19T10:00:55.240" Title="Proof of Breakpoint Reversal Sorting Approximation Algorithm" Tags="|sequence-alignment|genomics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6751" PostTypeId="1" AcceptedAnswerId="6752" CreationDate="2019-01-06T22:02:08.580" Score="4" ViewCount="336" Body="&lt;p&gt;I am using BLAT to align two versions of the genome of &lt;em&gt;C. elegans&lt;/em&gt;. I can see in the Activity Monitor of my Mac Book Pro High Sierra that &lt;code&gt;blat&lt;/code&gt; is using 100% of a CPU. However, is this programme able to use more than one core / CPU to speed up the alignment?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot find such option in the help page of the tool &lt;a href=&quot;https://genome.ucsc.edu/goldenPath/help/blatSpec.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I installed blat using &lt;code&gt;conda install -c bioconda blat&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the latest version of &lt;code&gt;blat&lt;/code&gt;, i.e. 36.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2019-01-07T18:28:36.673" Title="Can blat use more than one core/CPU to speed up the alignment?" Tags="|sequence-alignment|software-usage|blat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6752" PostTypeId="2" ParentId="6751" CreationDate="2019-01-06T22:32:02.033" Score="6" Body="&lt;p&gt;BLAT can only use one CPU. It is actually not the right tool for full-genome alignment. For &quot;two versions&quot; of the same species, MUMmer and minimap2 are orders of magnitude faster and probably give better alignment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT (moving comment to answer): OP comments that the purpose of this alignment is for lifting over annotations using the UCSC PSL-based pipeline. So far there are no converters for minimap2's SAM/PAF or MUMmer's delta formats. With the UCSC pipeline, you have to write a converter by yourself. An alternative approach is to use minimap2's liftover. The &lt;a href=&quot;https://github.com/lh3/minimap2/blob/master/cookbook.md#liftover&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimap2 cookbook&lt;/a&gt; gives an example:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-shell prettyprint-override&quot;&gt;&lt;code&gt;minimap2 -cx asm5 --cs ecoli_ref.fa ecoli_canu.fa &amp;gt; ecoli_canu.paf&#xA;echo -e 'tig00000001\t200000\t300000' | paftools.js liftover ecoli_canu.paf -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that this pipeline is not as mature as UCSC liftOver and is not as scalable.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2019-01-07T18:28:36.673" LastActivityDate="2019-01-07T18:28:36.673" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6753" PostTypeId="1" AcceptedAnswerId="6757" CreationDate="2019-01-07T13:19:42.397" Score="1" ViewCount="809" Body="&lt;p&gt;I tried to install gviz in a conda environment, but that library seems to be incompatible to python and r. I tried to setup a clean environment using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conda create -n r -c conda-forge r-essentials jupyter&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then add the library with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;source activate r&#xA;conda install -c bioconda bioconductor-gviz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;getting &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;UnsatisfiableError: The following specifications were found to be in conflict:&#xA;    - atk&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I remove atk, but now I get:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;UnsatisfiableError: The following specifications were found to be in conflict:&#xA;    - bioconductor-gviz&#xA;    - r-bindr&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Does anyone manage to use gviz from within an jupyter notebook?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried to install gviz from a running R-notebook with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;install.package('gviz')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Warning in install.packages :&#xA;    package ‘gviz’ is not available (for R version 3.4.3)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Same when I try 'bioconductor-gviz'.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2019-01-09T17:21:51.807" Title="How can I install bioconductor-gviz and use it in jupyter notebook?" Tags="|bioconductor|gviz|bioconda|jupyter|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6754" PostTypeId="1" CreationDate="2019-01-07T14:38:53.430" Score="2" ViewCount="103" Body="&lt;p&gt;This was my earlier &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/5661/cicros-plot-issue-using-two-data-track&quot;&gt;question&lt;/a&gt; but now I would like to see patient sample mutation which i have tried and im getting an output but it looks really dumb &lt;a href=&quot;https://i.stack.imgur.com/3MNDt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3MNDt.png&quot; alt=&quot;Circos output&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So here are my data files and conf file &lt;a href=&quot;https://drive.google.com/open?id=1ZoOR-QusndH-Pwqs-WnkCGRTcOsJTULW&quot; rel=&quot;nofollow noreferrer&quot;&gt;files&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my file which im trying to plot is gene having mutation in terms of patient which ranges from 1  to 53 patient at max .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NPM1 is having the highest mutation with 53 patient in the plot what i tried is to set a threshold of between &gt; 10 and &amp;lt; 3 to set the color label ,which labels the number it contains. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How can i add the gene names to the plot ?&lt;/li&gt;&#xA;&lt;li&gt;Is there a way where i can show the genes with only 1 mutation are linked together ,similarly for 2 linked together so on and so forth .&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I tried with the links but couldn;t make it work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion or help would be highly appreciated . &lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2019-01-07T14:38:53.430" Title="Circos plot with mutation data" Tags="|circos|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6755" PostTypeId="2" ParentId="6723" CreationDate="2019-01-07T15:12:10.413" Score="2" Body="&lt;p&gt;You could also save your &quot;biomart_query_result&quot; object after the first time it is generated. Then you can re-use it without having to depend on the biomaRt API functionality.&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2019-01-07T15:12:10.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6756" PostTypeId="1" CreationDate="2019-01-07T19:41:13.913" Score="2" ViewCount="184" Body="&lt;p&gt;I have a multiple sequence alignment (MSA) of protein sequences, with which I have performed multidimensional scaling to visualize their clustering.  Using R, I used the &lt;code&gt;seqinr&lt;/code&gt; package to create an identity matrix (using &lt;code&gt;dist.alignment&lt;/code&gt;).  Is there a similar method that is more &quot;generalizable&quot;?  Specifically, I'm interested in classifying the amino acids by some functional characteristics, such as polarity, instead of having the alignment list the amino acids by their identity (A, G, I, etc.).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a very short set of fake input data, with columns 2-4 representing amino acid residue positions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a &amp;lt;- c(&quot;Strain 1&quot;, &quot;Strain 2&quot;, &quot;Strain 3&quot;, &quot;Strain 4&quot;)&#xA;b &amp;lt;- c(&quot;Polar_hydrox&quot;, &quot;Nonpolar&quot;, &quot;Nonpolar&quot;, &quot;Polar_hydrox&quot;)&#xA;c &amp;lt;- c(&quot;Nonpolar&quot;, &quot;Nonpolar&quot;, &quot;Nonpolar&quot;, &quot;Nonpolar&quot;)&#xA;d &amp;lt;- c(&quot;Nonpolar_cyclic&quot;, &quot;Nonpolar_cyclic&quot;, &quot;Polar_amide&quot;, &quot;Polar_amide&quot;)&#xA;e &amp;lt;- c(&quot;Polar_amide&quot;, &quot;Acidic&quot;, &quot;Polar_amide&quot;, &quot;Acidic&quot;)&#xA;x &amp;lt;- data.frame(a, b, c, d, e)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is what I'm asking a feasible/correct use of MDS?  Or should I be looking into a different method entirely?  I attempted to use Gower's formula to create a dissimilarity matrix on the original dataset (i.e. using the actual amino acid identities instead of the functional groupings), but compared to what I got using &lt;code&gt;dist.alignment&lt;/code&gt;, the MDS plots look different.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Gower's formula and the &lt;code&gt;dist.alignment&lt;/code&gt; function are obviously calculating the distance matrices differently, but I don't know the details on the different methods.  Would using Gower's formula be a good approach for calculating distance matrices for the example dataset?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: clarified functions used for calculating distance matrix.&lt;/p&gt;&#xA;" OwnerUserId="3932" LastEditorUserId="3932" LastEditDate="2019-01-08T15:35:30.267" LastActivityDate="2019-01-09T11:45:09.720" Title="Using multidimensional scaling to visualize protein sequences by functionality" Tags="|r|clustering|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6757" PostTypeId="2" ParentId="6753" CreationDate="2019-01-07T20:30:18.450" Score="3" Body="&lt;p&gt;Did you try following the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/Gviz.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;installation instructions&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try this in R:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;source(&quot;https://bioconductor.org/biocLite.R&quot;)&#xA;BiocInstaller::biocLite(c(&quot;Gviz&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1269" LastActivityDate="2019-01-07T20:30:18.450" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6758" PostTypeId="1" AcceptedAnswerId="6799" CreationDate="2019-01-07T22:13:50.400" Score="2" ViewCount="129" Body="&lt;p&gt;I have used a python script to identify target sequences in a DNA sequence file. There are two classes of sequence: coding and non-coding. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have identified 728 sequences of interest. 597 of these fall into the coding regions and 131 of these fall into the non-coding regions. This is the equivalent of 18% non-coding. The total non-coding region in the sequence file is 13%. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a statistical tool to demonstrate the python script identified target sequences in a non-random fashion?&lt;br&gt;&#xA;If the script identified sequences that were randomly distributed then 13% of them would have been found in the non-coding region, from a total of 728 this seems like it should be reliable.&lt;/p&gt;&#xA;" OwnerUserId="3823" LastEditorUserId="3848" LastEditDate="2023-05-19T18:52:23.187" LastActivityDate="2023-05-19T18:52:23.187" Title="Genomics Statistics Problem" Tags="|phylogenetics|phylogeny|statistics|genomics|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6759" PostTypeId="2" ParentId="6749" CreationDate="2019-01-08T09:50:33.527" Score="3" Body="&lt;p&gt;You're really running out of degrees of freedom and the only actual replicate is a single sample from another patient, so please take the results with a huge grain of salt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use a design of &lt;code&gt;~cellType * coculture&lt;/code&gt; where &lt;code&gt;cellType&lt;/code&gt; has levels &lt;code&gt;NOF&lt;/code&gt; and &lt;code&gt;CAF&lt;/code&gt; and &lt;code&gt;coculture&lt;/code&gt; has levels &lt;code&gt;yes&lt;/code&gt; and &lt;code&gt;no&lt;/code&gt; (ideally you'd block by patient, but that's not an option for you).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-01-08T09:50:33.527" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6761" PostTypeId="2" ParentId="6758" CreationDate="2019-01-08T17:18:57.327" Score="2" Body="&lt;p&gt;A Poisson or binomial would suffice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not at all sure the difference will significantly deviate from randomness. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perl: 95% confidence interval of a non-parametric bootstrap for e.g. 1000 replicates is an alternative approach. I'd be more comfortable coding that solution in Perl than I would in Python, so is unlikely to be of much use here. Anyway, I would use a 2-dimensional array for your total data-set, the second dimension being coding/non-coding, I would then use the inbuilt random-number generator and multiply by the size of the array to randomly pluck out each element. 95% CI of the mean works by ranking the data and removing the top and bottom 2.5% and taking the interval for a single replicate (which is repeated 999 times more).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There will be a deep learning approach to this data, which would be easier to implement in Python, but I'd need to think about this because it is not simple classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again my warning is that given the sample size, the result at best might be borderline. There are likely to be lots of interesting in your data, but coding/non-coding may not be one of them.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-01-08T17:18:57.327" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6762" PostTypeId="2" ParentId="2798" CreationDate="2019-01-08T17:56:50.360" Score="1" Body="&lt;p&gt;SEM specialises in identifying 'latent variables'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SEM are extensively used for questionnaire data. In questionnaire data, latent variables are the 'psychological' factors behind given (human) behaviours. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Applying SEM to explicit molecular mechanisms doesn't make much sense unless there is a hidden component in a mechanism that has not been identified. However, even here the SEM will not tell you what the 'latent variable' represents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus I don't believe there is a good application to bioinformatics, outside human phenotypes derived from questionnaire data, which might be mapped on biological variables, but I could be wrong. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-01-08T17:56:50.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6763" PostTypeId="2" ParentId="6756" CreationDate="2019-01-08T19:29:17.647" Score="1" Body="&lt;p&gt;I personally don't think this will work well for some variables, primarily because you are not mapping conformational epitopes, where amino acids will cluster in physical space by their charge. In particular, the charge is critical to retaining this conformational structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, hydrophobic/hydrophilic characters are far more amenable to linear style epitopes that this analysis is primed towards.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't recommend a multi-dimensional array for this analysis, because you have loads of information, but rather a 'dictionary' (Python), hash (Perl) (or 'object' [any]) or XML, which retains all the information for a given sequence and then strips this in preparation for your multivariate statistic. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-01-08T19:29:17.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6764" PostTypeId="1" AcceptedAnswerId="6765" CreationDate="2019-01-08T20:58:38.270" Score="-2" ViewCount="58" Body="&lt;p&gt;I have written a package for my personal use and my other colleagues. I have uploaded the files and source package to different websites. I am able to install the source package in my computer. But my colleagues cannot install it either by github.com (&lt;a href=&quot;https://github.com/furqan915/Epi-gene&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/furqan915/Epi-gene&lt;/a&gt;) or by sourceforge.net (&lt;a href=&quot;https://sourceforge.net/projects/epi-gene/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sourceforge.net/projects/epi-gene/&lt;/a&gt;). &#xA;Common error is package cannot be installed and it does not add to user library.&#xA;I am unable to find out the problem.&#xA;Kindly help me out.  &lt;/p&gt;&#xA;" OwnerUserId="3626" LastActivityDate="2019-01-09T13:39:30.377" Title="R package installation in r studio" Tags="|r|" AnswerCount="2" CommentCount="2" ClosedDate="2019-01-10T12:16:44.973" ContentLicense="CC BY-SA 4.0" />
  <row Id="6765" PostTypeId="2" ParentId="6764" CreationDate="2019-01-08T21:11:20.147" Score="3" Body="&lt;p&gt;Can you use &lt;code&gt;devtools::install_github()&lt;/code&gt; giving your github username/reposname&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, you need to check that your R package is passing &lt;code&gt;R CMD check&lt;/code&gt; or &lt;code&gt;devtools::check()&lt;/code&gt;. See &lt;a href=&quot;http://r-pkgs.had.co.nz/check.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://r-pkgs.had.co.nz/check.html&lt;/a&gt; and &lt;a href=&quot;http://kbroman.org/pkg_primer/pages/check.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://kbroman.org/pkg_primer/pages/check.html&lt;/a&gt; for info&lt;/p&gt;&#xA;" OwnerUserId="3939" LastEditorUserId="57" LastEditDate="2019-01-09T08:07:17.307" LastActivityDate="2019-01-09T08:07:17.307" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6766" PostTypeId="2" ParentId="6753" CreationDate="2019-01-08T21:14:16.540" Score="4" Body="&lt;pre&gt;&lt;code&gt;# Install the BiocManager package first&#xA;install.packages(&quot;BiocManager&quot;) &#xA;library(&quot;BiocManager&quot;)&#xA;&#xA;# Then install the Gviz package&#xA;BiocManager::install(&quot;Gviz&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3939" LastEditorUserId="3938" LastEditDate="2019-01-09T17:21:51.807" LastActivityDate="2019-01-09T17:21:51.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6767" PostTypeId="2" ParentId="6746" CreationDate="2019-01-08T21:26:00.047" Score="1" Body="&lt;p&gt;You can just remove the first component from the dataset by setting the first eigenvalue to 0 in the diagonal matrix and then multiplying the SVD matrices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure which R code you are using to estimate the components (I have a comparison of PCA code in R on &lt;a href=&quot;https://github.com/aedin/ODSC_2018/tree/master/PCA_vignette&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/aedin/ODSC_2018/tree/master/PCA_vignette&lt;/a&gt;)  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In ade4 R package, there is a function &lt;code&gt;reconst()&lt;/code&gt; which reconstructs data from a selection of components: &lt;a href=&quot;https://rdrr.io/rforge/ade4/man/reconst.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rdrr.io/rforge/ade4/man/reconst.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Happy to provide code to do this in other PCA versions if needed&#xA;Aedin Culhane&lt;/p&gt;&#xA;" OwnerUserId="3939" LastEditorUserId="3541" LastEditDate="2020-02-03T05:11:26.483" LastActivityDate="2020-02-03T05:11:26.483" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6768" PostTypeId="2" ParentId="4773" CreationDate="2019-01-08T21:56:07.987" Score="1" Body="&lt;p&gt;These data are in Experiment Hub in Bioconductor&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(curatedTCGAData)&#xA;brcaRNA &amp;lt;- curatedTCGAData(&quot;BRCA&quot;, &quot;RNASeq2GeneNorm&quot;, dry.run = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3939" LastActivityDate="2019-01-08T21:56:07.987" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6769" PostTypeId="2" ParentId="6756" CreationDate="2019-01-08T22:06:57.717" Score="2" Body="&lt;p&gt;Some of the first MSA analysis, eg of codon usage of bacteria were performed using matrix factorization (see old papers from Des Higgins). His group also published discriminative methods for finding functional residues using supervised correspondence analysis &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Wallace IM1, Higgins DG BMC Bioinformatics. 2007 Apr 23;8:135.&#xA;  Supervised multivariate analysis of sequence groups to identify&#xA;  specificity determining residues.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The method is available in the R package &lt;a href=&quot;https://bioconductor.org/packages/ade4&quot; rel=&quot;nofollow noreferrer&quot;&gt;ade4&lt;/a&gt; and Bioconductor package &lt;a href=&quot;https://bioconductor.org/packages/made4&quot; rel=&quot;nofollow noreferrer&quot;&gt;made4&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also have a look at the phylogenetic R package &lt;a href=&quot;https://cran.r-project.org/package=adephylo&quot; rel=&quot;nofollow noreferrer&quot;&gt;adephylo&lt;/a&gt; and several related packages&lt;/p&gt;&#xA;" OwnerUserId="3939" LastEditorUserId="48" LastEditDate="2019-01-09T11:45:09.720" LastActivityDate="2019-01-09T11:45:09.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6770" PostTypeId="2" ParentId="5618" CreationDate="2019-01-08T22:28:15.670" Score="2" Body="&lt;p&gt;BLOSUM stands for BLOcks SUbstitution Matrices (Henikoff &amp;amp; Henikoff, 1992), and were created by observing substitution frequencies in local ungapped multiple sequence alignments. The score reflects the chance (log-odds) one amino acid is substituted for another in a set of protein multiple sequence alignments. BLOSUM scoring matrices are normally followed by a number eg BLOSUM62. The number is % sequence identity between the sequences in the multiple sequence alignment (MSA) used to create the score matrix. eg BLOSUM62 matrices were created from multiple sequence alignments with blocks that shared 62% identity. Note because these are ungapped alignments, the multiple sequence alignments were generated from conserved protein families.  With the older PAM matrices, the number is opposite, a higher number is greater distance. PAM matrices reflect evolutionarily distant in global (aka gapped) alignments of the full-length sequences that include both conserved and diverged regions.  See Wikipedia &lt;a href=&quot;https://en.wikipedia.org/wiki/BLOSUM&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/BLOSUM&lt;/a&gt;. There are several newer substitution matrices.  The R package SubVis allows you to compare/visualize different substitution matrices. &lt;/p&gt;&#xA;" OwnerUserId="3939" LastActivityDate="2019-01-08T22:28:15.670" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6771" PostTypeId="2" ParentId="2392" CreationDate="2019-01-08T22:38:26.773" Score="0" Body="&lt;pre&gt;&lt;code&gt;library(EnsDb.Hsapiens.v86)&#xA;library(ensembldb)&#xA;genes(EnsDb.Hsapiens.v86)&#xA;genes(EnsDb.Hsapiens.v86, filter=GeneIdFilter('ENSG00000235857'))&#xA;transcripts(EnsDb.Hsapiens.v86, filter = GeneIdFilter(&quot;ENSG00000184895&quot;),&#xA;              order.by = &quot;tx_seq_start&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(The ids you listed are older than v86). See the vignette for the ensembldb Bioconductor package.&lt;/p&gt;&#xA;" OwnerUserId="3939" LastActivityDate="2019-01-08T22:38:26.773" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6772" PostTypeId="2" ParentId="6723" CreationDate="2019-01-08T23:12:26.160" Score="1" Body="&lt;p&gt;Would this be useful to you?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;   library(EnsDb.Hsapiens.v86)  &#xA;   library(ensembldb)&#xA;   mygene &amp;lt;- genes(EnsDb.Hsapiens.v86, filter= GenenameFilter(&quot;KRAS&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt;to get the protein info and protein sequence for each transcript&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;proteins(EnsDb.Hsapiens.v86, filter= GenenameFilter(&quot;KRAS&quot;))&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also have a look at functions&#xA;    proteinToTranscript&#xA;    proteinToGenome&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; x&amp;lt;- proteins(EnsDb.Hsapiens.v86, filter= GenenameFilter(&quot;KRAS&quot;), return.type = &quot;AAStringSet&quot;)&#xA;&amp;gt; x&#xA;  A AAStringSet instance of length 4&#xA;    width seq                                    names               &#xA;[1]   188 MTEYKLVVVGAGGVGKSA...KDGKKKKKKSKTKCVIM ENSP00000308495&#xA;[2]    75 MTEYKLVVVGAGGVGKSA...KDGKKKKKKSKTKCVIM ENSP00000452512&#xA;[3]   189 MTEYKLVVVGAGGVGKSA...KEEKTPGCVKIKKCIIM ENSP00000256078&#xA;[4]    43 MTEYKLVVVGAGGVGKSA...HFVDEYDPTIEVLLSSM ENSP00000451856&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are also functions  exons, exonsBy, transcriptsBy, cdsBy, fiveUTRsByTranscript and threeUTRsByTranscript.  These functions return a GenomicRanges object and inherit all of the methods in that package too. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x&amp;lt;-  genes(EnsDb.Hsapiens.v86, filter= GenenameFilter(&quot;KRAS&quot;))&#xA;start(x)&#xA;end(x)&#xA;strand(x)&#xA;length(x)&#xA;flank(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can manipulate the sequence selected as an IRanges object&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ranges(x)   # gives an IRanges object&#xA;narrow(ranges(x), start=10, end=20)  # select the 10th position &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; ranges(x)&#xA;IRanges object with 1 range and 0 metadata columns:&#xA;                      start       end     width&#xA;                  &amp;lt;integer&amp;gt; &amp;lt;integer&amp;gt; &amp;lt;integer&amp;gt;&#xA;  ENSG00000133703  25204789  25250936     46148&#xA;&amp;gt; narrow(ranges(x), start=10, end=19)&#xA;IRanges object with 1 range and 0 metadata columns:&#xA;                      start       end     width&#xA;                  &amp;lt;integer&amp;gt; &amp;lt;integer&amp;gt; &amp;lt;integer&amp;gt;&#xA;  ENSG00000133703  25204798  25204807        10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is worth having a look at these vignettes. Let me know if you've questions or post to the Bioconductor support forum ;-)&lt;/p&gt;&#xA;" OwnerUserId="3939" LastActivityDate="2019-01-08T23:12:26.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6773" PostTypeId="2" ParentId="5558" CreationDate="2019-01-09T11:57:09.963" Score="3" Body="&lt;p&gt;Harry Boutselakis addressed the question via the customer support of the COSMIC web site.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Dear Ivan, Thank you for using COSMIC.&lt;/p&gt;&#xA;  &#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;I would like to know why there are different entries for the same mutation   in COSMIC, for example COSM6954941 and COSM12833 both refer&#xA;    to ERBB4   c.908C&gt;A.&lt;/p&gt;&#xA;    &#xA;    &lt;p&gt;In this specific case the field Gene name in the two entries is different,   COSM6954941 report ERBB4_ENST00000402597 while COSM12833&#xA;    report ERBB4,&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;  &#xA;  &lt;p&gt;The mutations are annotated onto two different isoforms of ERBB4. So&#xA;  this annotation refers to two different transcripts. In these cases&#xA;  the consequence of the mutation on the mRNA and polypeptide is the&#xA;  same reflected by the syntax.&lt;/p&gt;&#xA;  &#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;but there are many cases in which the Gene name is the same, for example   COSM1361 and COSM1219.&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;  &#xA;  &lt;p&gt;This case the consequence one mutation on the mRNA is unknown , but&#xA;  the amino acid change is the same on both cases.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The information stored in COSMIC is curated from experimental data,&#xA;  depending on the evidence and experimental methodology used, the&#xA;  mutations are differentiated accordingly. I hope this answers your&#xA;  question.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Kind regards,&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Harry&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3772" LastEditorUserId="298" LastEditDate="2019-01-09T12:17:30.580" LastActivityDate="2019-01-09T12:17:30.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6774" PostTypeId="1" CreationDate="2019-01-09T12:14:08.907" Score="5" ViewCount="241" Body="&lt;p&gt;I have a problem with a tool, that possibly ignores supplementary reads. I want to find out a bit how this tool works by converting all supplementary reads to primary reads, and then change the names of those reads (because I guess that primary reads with the same name are ignored?). If the tool produces something, instead of nothing (like now), then I know for sure its the supplementary read flag that could be the problem.&lt;/p&gt;&#xA;&lt;p&gt;I use SV detection tools. The longer the reads are that I produce (like simulating improvement in the long-read sequencing technology), far less SVs gets detected, while my hypothesis is that longer reads should be able to detect more. Then I found that the alignment files look clean (sam/bam), but with far more supplementary tags. I thought I could test this by changing the flags.&lt;/p&gt;&#xA;&lt;p&gt;I am searching the internet, but changing read flags is not something they seem to do, only things like omitting reads with certain flags from the file. Maybe I am looking with the wrong search terms, in that case I am sorry.&lt;/p&gt;&#xA;&lt;p&gt;I guess I need to change column 1 and 2 of the sam file (QNAME and FLAG). But I was also wondering if this is going to work at all. Maybe someone with more knowledge of alignment files could help me with this problem? suggestions how to change the sam/bam file to check if supplementary reads are ignored?&lt;/p&gt;&#xA;&lt;p&gt;I think this, because I made on purpose a very unrealistic alignment. A 15Mb region against a 15Mb region with differences (x10 times to &amp;quot; simulate&amp;quot; 10 reads / 10x coverage). The alignment file looks quite good actually in IGV, but clicking on any read reveals that there are of course MANY supplementary reads. I also have a &amp;quot;normal&amp;quot; alignment file, where I also want to check if changing the supplementary tag could possibly help me with this tool.&lt;/p&gt;&#xA;" OwnerUserId="3552" LastEditorUserId="73" LastEditDate="2021-05-10T06:02:34.840" LastActivityDate="2021-05-10T06:02:34.840" Title="convert supplementary reads to primary in sam or bam" Tags="|sequence-alignment|sam|samtools|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6775" PostTypeId="2" ParentId="6753" CreationDate="2019-01-09T13:06:13.280" Score="1" Body="&lt;p&gt;Use BiocManager::install()&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Code&#xA;if (!requireNamespace(&quot;BiocManager&quot;, quietly = TRUE))&#xA;    install.packages(&quot;BiocManager&quot;)&#xA;BiocManager::install(&quot;Gviz&quot;, version = &quot;3.8&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, If R &amp;lt; 3.5.0 , please use the following for installing Bioconductor packages.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BiocInstaller::biocLite(&quot;Gviz&quot;) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3938" LastActivityDate="2019-01-09T13:06:13.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6776" PostTypeId="2" ParentId="6764" CreationDate="2019-01-09T13:39:30.377" Score="0" Body="&lt;p&gt;A quick workaround is to install the package directly from source:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;install.packages(&quot;~/Download/myPackage.tar.gz&quot;, repos = NULL, type=&quot;source&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You should be able to pass also &lt;code&gt;.zip&lt;/code&gt; and URLs (other than CRAN).&lt;/p&gt;&#xA;" OwnerUserId="3631" LastActivityDate="2019-01-09T13:39:30.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6777" PostTypeId="2" ParentId="3181" CreationDate="2019-01-09T14:36:19.940" Score="4" Body="&lt;p&gt;Among other things, the &lt;code&gt;canon-gff3&lt;/code&gt; program in the &lt;a href=&quot;https://aegean.readthedocs.io&quot; rel=&quot;nofollow noreferrer&quot;&gt;AEGeAn Toolkit&lt;/a&gt; infers and prints missing features like UTRs.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ canon-gff3 &amp;lt; genes.gff3 &amp;gt; genes-with-utrs.gff3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-01-09T14:36:19.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6778" PostTypeId="1" CreationDate="2019-01-09T14:42:49.520" Score="1" ViewCount="365" Body="&lt;p&gt;I have 2 cell types (NOF and CAF) cultures individually and also have been co-cultured with tumour like this picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ibb.co/x8Ty1Wz&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ibb.co/x8Ty1Wz&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is the log cpm of these 4 samples &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(onco[,1:4])&#xA;             A10       A11        A12        A2&#xA;A2M    11.755100 12.145143 12.9787977 12.407221&#xA;ABCB11  2.955457  2.278363  0.1229266  1.493507&#xA;ABCC2   6.711643  6.974025  0.1229266  7.628936&#xA;ABCC6   6.443439  6.094670  6.1389496  7.483394&#xA;ABCF1   9.995098 10.045613  9.6627424 10.314842&#xA;ABCG2   7.125483  7.448127  6.8669702  8.492895&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have used log of count per million (cpm) for Kruskal-Wallis rank sum test that says these samples are significantly difference like below;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; NOF1=c(NOF1&lt;span class=&quot;math-container&quot;&gt;$`onco$&lt;/span&gt;A10`)&#xA;&amp;gt; NOF2=c(NOF2$`onco&lt;span class=&quot;math-container&quot;&gt;$A11`)&#xA;&amp;gt; CAF1=c(CAF1$&lt;/span&gt;`onco$A12`)&#xA;&amp;gt; CAF2=c(CAF2&lt;span class=&quot;math-container&quot;&gt;$`onco$&lt;/span&gt;A2`)&#xA;&amp;gt; my_data &amp;lt;- data.frame( &#xA;+     group = rep(c(&quot;NOF1&quot;, &quot;NOF2&quot;,&quot;CAF1&quot;,&quot;CAF2&quot;), each = 719),&#xA;+     cell_type = c(NOF1,NOF2,  CAF1,CAF2)&#xA;+ )&#xA;&#xA;&amp;gt; head((my_data))&#xA;  group cell_type&#xA;1  NOF1 11.755100&#xA;2  NOF1  2.955457&#xA;3  NOF1  6.711643&#xA;4  NOF1  6.443439&#xA;5  NOF1  9.995098&#xA;6  NOF1  7.125483&#xA;&amp;gt; &#xA;&#xA;&amp;gt; kruskal.test(cell_type ~ group, data = my_data)&#xA;&#xA;    Kruskal-Wallis rank sum test&#xA;&#xA;data:  cell_type by group&#xA;Kruskal-Wallis chi-squared = 22.006, df = 3, p-value = 6.505e-05&#xA;&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I then did Pairwise comparisons using Wilcoxon rank sum test to see which samples are different with each other.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; pairwise.wilcox.test(my_data&lt;span class=&quot;math-container&quot;&gt;$cell_type, my_data$&lt;/span&gt;group,&#xA;+                      p.adjust.method = &quot;BH&quot;)&#xA;&#xA;    Pairwise comparisons using Wilcoxon rank sum test &#xA;&#xA;data:  my_data&lt;span class=&quot;math-container&quot;&gt;$cell_type and my_data$&lt;/span&gt;group &#xA;&#xA;     CAF1   CAF2   NOF1  &#xA;CAF2 0.0058 -      -     &#xA;NOF1 0.2809 4e-05  -     &#xA;NOF2 0.6818 0.0016 0.3811&#xA;&#xA;P value adjustment method: BH &#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My Question is; how I can extract the genes behind the difference between each pair of samples? I mean something like differential expression by non parametric test. also I am not sure if I am right in using log cpm values for these test&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If possible give me a hand to correct myself and get intuition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks &lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2019-01-09T14:48:54.957" Title="Extracting genes differentially expressed by Wilcox test" Tags="|r|rna-seq|statistics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6779" PostTypeId="2" ParentId="6778" CreationDate="2019-01-09T14:48:54.957" Score="2" Body="&lt;p&gt;All of the genes are behind the difference, you're doing a rank test comparing all of them against each other at once. Your results have more to do with samples having slightly different outlier genes than any interesting biology. You don't have enough replicates to do a non-parametric test.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-01-09T14:48:54.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6780" PostTypeId="2" ParentId="4246" CreationDate="2019-01-09T15:06:27.920" Score="3" Body="&lt;p&gt;As @DevonRyan mentioned, it's very likely that those samples were degraded, which is good justification for excluding them from subsequent analysis.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-01-09T15:06:27.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6781" PostTypeId="2" ParentId="4876" CreationDate="2019-01-09T15:17:57.627" Score="3" Body="&lt;p&gt;The &lt;a href=&quot;http://rosalind.info&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioinformatics Stronghold at Rosalind&lt;/a&gt; is an excellent resource for building your understanding of many concepts and algorithms you'll see frequently in bioinformatics and genomics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd also say that, if you can make it work, an internship where you can &quot;get your hands dirty&quot; with real problems is a very valuable experience. This option may not be available to everyone due to a variety of factors, but when it is an option it provides a tremendous opportunity to build experience with little risk to the intern or the employer.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-01-09T15:17:57.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6784" PostTypeId="2" ParentId="4246" CreationDate="2019-01-10T12:26:16.717" Score="2" Body="&lt;p&gt;I would expect at least 30% of reads from a total-cell, ribo-depleted RNA-seq to be exonic. Less suggests something when wrong. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As well as degradation, another explanation would be contamination with genomic DNA. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;RNA obtained from the nuclear or cytoplasmic fraction of the cell might have a exonic content lower than 30%, while poly-A selected RNA would be expected to have higher, more like 66%. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2019-01-10T12:26:16.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6786" PostTypeId="1" AcceptedAnswerId="6788" CreationDate="2019-01-10T14:35:50.690" Score="4" ViewCount="2965" Body="&lt;p&gt;I want to ask about why the sequence duplication levels are high after I trimmed by using Trimmomatic? I am using the following Trimmomatic operations: &lt;code&gt;HEADCROP = 19 TRAILING = 20 MINLEN = 66&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can i solve this problem? Thank You.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9rPCe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9rPCe.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/e04D8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/e04D8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3946" LastEditorUserId="96" LastEditDate="2019-01-10T16:14:44.957" LastActivityDate="2019-01-10T16:14:44.957" Title="&quot;Sequence Duplication Levels&quot; module still fails after pre-processing Illumina data" Tags="|illumina|data-preprocessing|trimming|fastqc|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6787" PostTypeId="2" ParentId="6786" CreationDate="2019-01-10T14:45:59.020" Score="6" Body="&lt;p&gt;FastQC assumes that all samples are for whole genome sequencing and will flag them as failed if they differ too much from that assumption. This will, for example, cause essentially all RNA-seq, ChIP-seq, and ATAC-seq samples to fail in one module or another. This is not any cause for concern and is completely expected. Primarily concern yourself with whether all of your samples are similar in their metrics.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-01-10T14:45:59.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6788" PostTypeId="2" ParentId="6786" CreationDate="2019-01-10T16:12:54.763" Score="6" Body="&lt;p&gt;To answer your direct question, there are a few reasons why there might be high levels of sequence duplication. From the FastQC help:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;The underlying assumption of this module is of a diverse unenriched library. Any deviation from this assumption will naturally generate duplicates and can lead to warnings or errors from this module.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;As @DevonRyan mentioned, with certain sequencing protocols such as RNA-Seq, two sequence reads at exactly the same location aren't that uncommon. This isn't a problem with RNA-Seq data, or with Trimmomatic, or with FastQC. It's just that this kind of data violates the assumption, and therefore should be ignored in those circumstances.&lt;/li&gt;&#xA;&lt;li&gt;PCR duplicates are another possible cause. PCR duplicates can give the false impression of high coverage at a particular locus when in fact it's just a single observed read that has been duplicated many times (&lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2867/96&quot;&gt;see here&lt;/a&gt; for more details). PCR duplicates can usually be detected and removed if your analysis involves mapping to a reference genome. But whether this is actually a problem you need to fix depends on what type of data you have and what types of analysis you want to do.&lt;/li&gt;&#xA;&lt;li&gt;Large numbers of adapter dimers or rRNA may be present in your sample.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;But I think it's also important to address how quality control (QC) is run. It can be tempting to run and re-run QC tools like Trimmomatic until all errors go away, but to be blunt these tools cannot think for you. For example, it's &lt;em&gt;possible&lt;/em&gt; to get rid of most adapters by aggressively cropping/trimming both ends of each read, but you'll likely throw away a lot of good data that way. You may want to look into Trimmomatic's ILLUMINACLIP operation. It's also may be tempting to crop/trim reads aggressively if there are compositional biases near the beginning or end of the read. In fact, random hexamer priming can cause the &lt;strong&gt;Per-base Sequence Content&lt;/strong&gt; module to fail on almost any RNA-Seq sample. Again, from the FastQC help (emphasis mine):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;Libraries produced by priming using random hexamers (including nearly all RNA-Seq libraries) and those which were fragmented using transposases inherit an intrinsic bias in the positions at which reads start. This bias does not concern an absolute sequence, but instead provides enrichement of a number of different K-mers at the 5' end of the reads. Whilst this is a true technical bias, &lt;strong&gt;it isn't something which can be corrected by trimming&lt;/strong&gt; and in most cases &lt;strong&gt;doesn't seem to adversely affect the downstream analysis&lt;/strong&gt;. It will however produce a warning or error in this module.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So in other words, it's best to determine what can cause each FastQC module to fail, investigate whether this is actually a problem for your data set (referring to &lt;a href=&quot;https://www.bioinformatics.babraham.ac.uk/projects/fastqc/Help/3%20Analysis%20Modules/&quot; rel=&quot;noreferrer&quot;&gt;documentation&lt;/a&gt; as needed), and make a deliberate QC plan that addresses the issues that need attention.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-01-10T16:12:54.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6789" PostTypeId="1" AcceptedAnswerId="6790" CreationDate="2019-01-10T17:32:12.067" Score="0" ViewCount="752" Body="&lt;p&gt;I have got the problem on how to write the italic script in RStudio. I used the script in the screenshot and got the error &quot;Error in grobs[[i]]: subscript out of bounds.&quot; &lt;a href=&quot;https://i.stack.imgur.com/emCgW.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/emCgW.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3171" LastActivityDate="2019-01-10T18:42:59.803" Title="how to write italic script in Rstudio" Tags="|rna-seq|single-cell|" AnswerCount="1" CommentCount="4" ClosedDate="2019-01-24T12:52:05.597" ContentLicense="CC BY-SA 4.0" />
  <row Id="6790" PostTypeId="2" ParentId="6789" CreationDate="2019-01-10T18:42:59.803" Score="4" Body="&lt;p&gt;The title is misleading as this error doesn't have anything to do with making the font italic. In Seurat v2, FeaturePlot does not return a ggplot2 object by default, so &lt;code&gt;p&lt;/code&gt; in your case is &lt;code&gt;NULL&lt;/code&gt;. You need to set &lt;code&gt;do.return&lt;/code&gt; to &lt;code&gt;TRUE&lt;/code&gt; in the FeaturePlot call.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should instead do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Seurat)&#xA;&#xA;p &amp;lt;- FeaturePlot(pbmc_small, head(pbmc_small@var.genes), do.return = TRUE)&#xA;&#xA;for(i in 1:length(p)) {&#xA;  p[[i]] &amp;lt;- p[[i]] + theme(plot.title = element_text(face = 'italic'))&#xA;}&#xA;&#xA;cowplot::plot_grid(plotlist = p)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/h9me4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/h9me4.png&quot; alt=&quot;featureplot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3415" LastActivityDate="2019-01-10T18:42:59.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6791" PostTypeId="1" AcceptedAnswerId="7070" CreationDate="2019-01-10T21:18:05.047" Score="8" ViewCount="143" Body="&lt;p&gt;I have short (67bp) Hi-C reads from a highly heterozygous organism (~15% between-haplotype SNP divergence) and I have both reference haplotypes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to try and compare different haplotyping softwares for Hi-C reads using these reads as benchmarking dataset. When mapping the reads separately to each haplotype, I get good mapping statistics. When I map the reads to a single haplotype with all the heterozygous SNPs masked (into N's), I get  very poor mapping rates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to be able to map the reads when the real haplotypes are unknown (reference is a mix of haplotypes).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use minimap2 to map the reads with the sr preset. I tried decreasing the mismatch penalty (-B) to 1 and increasing the --score-N value, but this had no effect.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As shown in the attached IGV screenshot, coverage drops to 0 when the SNP density increases.&lt;a href=&quot;https://i.stack.imgur.com/ycZ4G.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ycZ4G.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Is it feasible to map reads with such a high heterozygosity on a single (masked) reference ? Should I use another tool ?&lt;/p&gt;&#xA;" OwnerUserId="439" LastEditorUserId="439" LastEditDate="2019-02-19T09:16:07.640" LastActivityDate="2019-02-19T14:15:12.180" Title="Highly heterozygous reads mapping" Tags="|read-mapping|haplotypes|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6792" PostTypeId="1" CreationDate="2019-01-10T22:40:40.990" Score="6" ViewCount="105" Body="&lt;p&gt;Hi I'm a first time user of IADMIX. I tested on one known Finnish sample from the  1000Genome project using the softwares provided frequency table hapmap3.8populations.hg19.freqs and the prediction is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm also want to do the same using population frequencies from the gnomAD database but my prediction is wrong - comes out to be African.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My steps are: Create my own population frequency table using the gnomad database from &lt;a href=&quot;http://gnomad.broadinstitute.org/downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://gnomad.broadinstitute.org/downloads&lt;/a&gt; For each VCF file i do the following&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view -Ou -I -q 0.05  -v snps -k -m2 -M2 -f PASS gnomad.genomes.r2.0.1.sites.22.vcf.gz | bcftools query -f '%CHROM\t%POS\t%ID\t%REF\t%ALT\t%INFO/AF_nfe\t%INFO/AF_afr\t%INFO/AF_amr\t%INFO/AF_eas\t%INFO/AF_asj\t%INFO/AF_fin\t%INFO/AF_oth\n'  -o iadmix-22.freq.bcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which gives me tabulated data like this containing only &quot;known&quot; variants with RSID != &quot;.&quot; , min threshold for total AF &gt;= 0.05 , only bi-allelic SNPs that also have FILTER==PASS &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;7 16692 rs370739206 G C 1 . . . 1 . 1  7 16719 rs151261124 G A 1 1 1 .&#xA;  1 . 1  7 20460 rs1466291194 T C 0.0381589 0.0630435 0.191919 0.0357143&#xA;  0.0333333 0.144909 0.121212  7 20471 rs370790239 C T 0.631696 0.716216 0.766667 0.666667 . 0.661765 0.625&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Then I have a small R script to - exclude sites that have any population groups AF values == &quot;.&quot; - include sites that has mean of all AFs for the snps must be &gt;= 0.05 (to include only common variants)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I tried running IADMIX using this frequency table on a know finnish 1KG sample, but the prediction is incorrect&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python runancestry.py --freq MY.FREQUENCY.FILE --plink SAMPLE_file --out testoutput --path full/path/&#xA;&#xA;snps in file 455423 pops 7 AF_NFE:0.167 AF_AFR:0.189 AF_AMR:0.133 AF_EAS:0.188 AF_ASJ:0.238 AF_FIN:0.037 AF_OTH:0.047 initial sol 1082621.766160 RUNNING THE L-BFGS-B CODE&#xA;&#xA;    * * *&#xA;&#xA;Machine precision = 2.220446e-16 ... ...&#xA;&#xA;final maxval -1044248.568804 ADMIX_PROP AF_NFE:0.0000 AF_AFR:0.5822 AF_AMR:0.1137 AF_EAS:0.1797 AF_ASJ:0.1244 AF_FIN:0.0000 AF_OTH:0.0000 AF_AFR:0.5822 AF_AMR:0.1137 AF_EAS:0.1797 AF_ASJ:0.1244 FINAL_NZ_PROPS&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any one know whats happening here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference the output using the frequency table provided by IADMIX is as follows&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;snps in file 106250 pops 8 YRI:0.015 CHB:0.003 CHD:0.275 TSI:0.207&#xA;  MKK:0.101 LWK:0.169 CEU:0.194 JPT:0.037  initial sol 76610.617803&#xA;  RUNNING THE L-BFGS-B CODE&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    * * *&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;Machine precision = 2.220446e-16 final maxval -72537.379103 ADMIX_PROP&#xA;  YRI:0.0000 CHB:0.0000 CHD:0.0000 TSI:0.1031 MKK:0.0000 LWK:0.0000&#xA;  CEU:0.8190 JPT:0.0779 TSI:0.1031 CEU:0.8190 JPT:0.0779 FINAL_NZ_PROPS&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;so the CEU proportion is 82% , acceptable for this finnish sample, right?&lt;/p&gt;&#xA;" OwnerUserId="3948" LastEditorUserId="3848" LastEditDate="2019-08-12T11:15:41.547" LastActivityDate="2021-07-08T13:57:15.807" Title="making my own population allele frequency table for input to IADMIX from Gnomad data (population admixture)?" Tags="|r|gwas|phylogenetics|" AnswerCount="0" CommentCount="2" ClosedDate="2021-07-08T13:57:27.027" ContentLicense="CC BY-SA 4.0" />
  <row Id="6793" PostTypeId="1" CreationDate="2019-01-11T02:05:49.547" Score="1" ViewCount="112" Body="&lt;p&gt;I have .fastq files from a targeted amplicon sequencing by Illumina MiSeq run. I want to get the target sequences to perform Blast search on them. What is the pipeline to use for processing amplicon sequencing data? I also have 20% PhiX. &lt;/p&gt;&#xA;" OwnerUserId="3949" LastActivityDate="2019-01-11T02:05:49.547" Title="amplicon sequencing set up" Tags="|ngs|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6794" PostTypeId="1" CreationDate="2019-01-11T08:45:39.713" Score="2" ViewCount="79" Body="&lt;p&gt;I have 4 pandas data frames containing the following data:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;affinity_score1: affinity between each couple (Drug-Target) measured using  tool1&lt;/li&gt;&#xA;&lt;li&gt;affinity_score2: affinity between each couple (Drug-Target) measured using tool2 &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The two tools (tool1, tool2) give measure in different scales and are not 100% accurate.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Drug_similarity: a similarity matrix between drugs&lt;/li&gt;&#xA;&lt;li&gt;Target_similarity: a similarity matrix between targets.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Here also the tool used to measure the similarity is not 100% accurate.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;affinity_score1 = [{'Drug0': -24.5, 'Drug1': -17.1, 'Drug2': -13.1},&#xA;           {'Drug0': -20.0, 'Drug1': -21.6, 'Drug2': -18.1},&#xA;           {'Drug0': -19.0, 'Drug1': -15.4, 'Drug2': -20.0}]&#xA;&#xA;affinity_score2 = [{'Drug0': 1.5, 'Drug1': 0.8, 'Drug2': 0.4},&#xA;           {'Drug0': 1.2, 'Drug1': 1.3, 'Drug2': 0.9},&#xA;           {'Drug0': 0.8, 'Drug1': 0.7, 'Drug2': 0.9}]&#xA;affinity_score1_df = pd.DataFrame(affinity_score1).rename(index={0:'TargetX',1:'TargetY', 2:'TargetZ'})&#xA;affinity_score2_df = pd.DataFrame(affinity_score2).rename(index={0:'TargetX',1:'TargetY', 2:'TargetZ'})&#xA;Drug_similarity = [{'Drug0': 1, 'Drug1': 0.8, 'Drug2': 0.2},&#xA;           {'Drug0': 0.8, 'Drug1': 1, 'Drug2': 0.9},&#xA;           {'Drug0': 0.2, 'Drug1': 0.9, 'Drug2': 1}]&#xA;Target_similarity = [{'TargetX': 1, 'TargetY': 0.8, 'TargetZ': 0.2},&#xA;           {'TargetX': 0.8, 'TargetY': 1, 'TargetZ': 0.9},&#xA;           {'TargetX': 0.2, 'TargetY': 0.9, 'TargetZ': 1}]&#xA;Drug_similarity = pd.DataFrame(Drug_similarity).rename(index={0:'Drug0',1:'Drug1', 2:'Drug2'})&#xA;Target_similarity = pd.DataFrame(Target_similarity).rename(index={0:'TargetX',1:'TargetY', 2:'TargetZ'})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I am trying to find the Drug who can have as many Targets as possible.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, a simple approach could be to select the Drug having the highest average score in both affinity scoring tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This approach does not use the information in the similarity matrices. &#xA;Given that if a Drug1 is successful with TargetX, and TargetX is similar to  TargetY then Drug1 can also easily be successful with TargetY. Considering the inaccuracy of the scoring tools, I think including this information could improve the accuracy of the answer, eliminating some false positives. &#xA;Also, if a Drug from a cluster of similar drugs (information that can be derived from the drugs similarity matrix) is the only one predicted to have good affinities with some targets, then this is more likely to be a false positive. &#xA;But I don't know how to efficiently integrate all the information. Scipy may help with clustering the two similarity matrices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another way of analysis is to set affinity and similarity thresholds, in which case each data frame could be transformed into a network:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Each node is a drug or a target.&lt;/li&gt;&#xA;&lt;li&gt;Two types of edges: similarity (between drugs or targets) and affinity (drug-target)&lt;/li&gt;&#xA;&lt;li&gt;The connected nodes have passed the thresholds. &lt;/li&gt;&#xA;&lt;li&gt;The edges have weights: affinity or similarity score. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The problem then becomes finding the most connected node (a drug node) with the highest weight values (scores on tool1 and tool2). Connected nodes through similarity may then be used to validate the number of targets for each drug. &#xA;It seems a more complicated approach but more rigorous with most likely higher positive predictive value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any python package to implement such an approach?&#xA;And also a visualization tool.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The original data has 38 Targets with 806 drugs.&lt;/p&gt;&#xA;" OwnerUserId="3861" OwnerDisplayName="BND" LastEditorUserId="48" LastEditDate="2019-01-11T14:04:36.463" LastActivityDate="2019-01-11T14:44:23.913" Title="Drug with as many targets as possible through a system biology graph?" Tags="|python|drugs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6795" PostTypeId="1" CreationDate="2019-01-11T11:19:27.910" Score="3" ViewCount="508" Body="&lt;p&gt;I downloaded the genome of Staph aureus as DNA sequence from NCBI. I would like to visualize it using Gviz. Is Gviz the right tool to visualize genomes? What are the necessary steps from the sequence file to an object that can be used in Gviz? I searched the user guide and the reference manual. It is stated that it is possible to use fasta file, but not how to use them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We will generate lots of genomes that we will have to annotate and compare. S aureus is not S aureus. There is a spectrum of phenotypes, they all have a different genomes. And we will also not be limited to SA. The question here is not how to annotate the genome, but rather the steps necessary to visualize something and get going. It is my first steps with genomic data, I don't even know what makes sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the head of the file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;NC_007795.1 Staphylococcus aureus subsp. aureus NCTC 8325 chromosome, complete genome&#xA;CGATTAAAGATAGAAATACACGATGCGAGCAATCAAATTTCATAACATCACCATGAGTTTGGTCCGAAGCATGAGTGTTT&#xA;ACAATGTTCGAACACCTTATACAGTTCTTATACATACTTTATAAATTATTTCCCAAACTGTTTTGATACACTCACTAACA&#xA;GATACTCTATAGAAGGAAAAGTTATCCACTTATGCACATTTATAGTTTTCAGAATTGTGGATAATTAGAAATTACACACA&#xA;AAGTTATACTATTTTTAGCAACATATTCACAGGTATTTGACATATAGAGAACTGAAAAAGTATAATTGTGTGGATAAGTC&#xA;GTCCAACTCATGATTTTATAAGGATTTATTTATTGATTTTTACATAAAAATACTGTGCATAACTAATAAGCAAGATAAAG&#xA;TTATCCACCGATTGTTATTAACTTGTGGATAATTATTAACATGGTGTGTTTAGAAGTTATCCACGGCTGTTATTTTTGTG&#xA;TATAACTTAAAAATTTAAGAAAGATGGAGTAAATTTATGTCGGAAAAAGAAATTTGGGAAAAAGTGCTTGAAATTGCTCA&#xA;AGAAAAATTATCAGCTGTAAGTTACTCAACTTTCCTAAAAGATACTGAGCTTTACACGATTAAAGATGGTGAAGCTATCG&#xA;TATTATCGAGTATTCCTTTTAATGCAAATTGGTTAAATCAACAATATGCTGAAATTATCCAAGCAATCTTATTTGATGTT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2019-01-11T18:02:40.717" LastActivityDate="2019-01-11T18:02:40.717" Title="How to plot genomic.fna fasta file in R using Gviz?" Tags="|r|genome|visualization|gviz|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6796" PostTypeId="1" CreationDate="2019-01-11T14:06:02.097" Score="2" ViewCount="433" Body="&lt;p&gt;I have this data from one of the cluster which show pathway in context of those genes but when i do a bit of filtering i see plenty of pathways with same or similar FDR which are sort of redundancy i suppose in terms of over-representation ,so i came up something like this &lt;a href=&quot;https://i.stack.imgur.com/3uypv.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3uypv.jpg&quot; alt=&quot;pathway&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;may be i need a bit more streamlining into it which as of now i can;t think up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So few question just as in terms of gene clustering can i cluster those FDR or say same or similar FDR are getting clustered together which might be helpful in a way to concise and precise the result a bit better .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My data &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(Function = c(&quot;covalent chromatin modification&quot;, &#xA;&quot;histone modification&quot;, &quot;histone methyltransferase complex&quot;, &#xA;&quot;SWI/SNF superfamily-type complex&quot;, &quot;histone methyltransferase activity&quot;, &#xA;&quot;N-methyltransferase activity&quot;, &quot;methyltransferase complex&quot;, &#xA;&quot;protein methyltransferase activity&quot;, &quot;histone methylation&quot;, &#xA;&quot;S-adenosylmethionine-dependent methyltransferase activity&quot;, &#xA;&quot;protein alkylation&quot;, &quot;protein methylation&quot;, &quot;macromolecule methylation&quot;, &#xA;&quot;methylation&quot;, &quot;methyltransferase activity&quot;, &quot;transferase activity, transferring one-carbon groups&quot;, &#xA;&quot;DNA helicase complex&quot;, &quot;Ino80 complex&quot;, &quot;peptidyl-lysine modification&quot;, &#xA;&quot;SWI/SNF complex&quot;, &quot;protein-DNA complex disassembly&quot;, &quot;nucleosome disassembly&quot;, &#xA;&quot;chromatin remodeling&quot;, &quot;chromatin disassembly&quot;, &quot;protein-DNA complex subunit organization&quot;, &#xA;&quot;BAF-type complex&quot;, &quot;INO80-type complex&quot;, &quot;histone-lysine N-methyltransferase activity&quot;, &#xA;&quot;histone acetyltransferase complex&quot;, &quot;nucleosome organization&quot;, &#xA;&quot;lysine N-methyltransferase activity&quot;, &quot;protein-lysine N-methyltransferase activity&quot;&#xA;), variable = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, &#xA;1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, &#xA;1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = &quot;FDR&quot;, class = &quot;factor&quot;), &#xA;    value = c(1.47992665134053e-13, 1.47992665134053e-13, 7.9468967129082e-09, &#xA;    7.9468967129082e-09, 1.1841058070482e-08, 1.75996948641137e-08, &#xA;    1.75996948641137e-08, 4.71690557750932e-08, 9.63620605173175e-08, &#xA;    3.94626427029223e-07, 4.55323040178062e-07, 4.55323040178062e-07, &#xA;    4.55323040178062e-07, 8.9471201991238e-07, 1.38187416380405e-06, &#xA;    2.07900888100729e-06, 2.60629648189489e-05, 2.60629648189489e-05, &#xA;    2.91531936861022e-05, 3.19327841103331e-05, 5.04466386223372e-05, &#xA;    5.04466386223372e-05, 5.45938617344686e-05, 5.4956792922593e-05, &#xA;    5.4956792922593e-05, 5.4956792922593e-05, 8.32575963760242e-05, &#xA;    0.000287420280100368, 0.000372921199469602, 0.000387140224358431, &#xA;    0.000447883223319561, 0.000447883223319561)), row.names = c(NA, &#xA;-32L), class = &quot;data.frame&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p &amp;lt;- ggplot(aes(x=Function, y=variable, fill=value), data=plot.data)&#xA;fig &amp;lt;- p + geom_tile() + scale_fill_gradient2(low=&quot;blue&quot;, mid=&quot;white&quot;, high=&quot;red&quot;) + &#xA;  #   geom_text(aes(label=stars, color=value), size=8) + scale_colour_gradient(low=&quot;grey30&quot;, high=&quot;white&quot;, guide=&quot;none&quot;) +&#xA;  #geom_text(aes(label=stars), color=&quot;black&quot;, size=5) + &#xA;  #labs(y=NULL, x=NULL, fill=&quot;t-value&quot;) + geom_vline(xintercept=1.5, size=1.5, color=&quot;grey50&quot;) + &#xA;  theme_bw() + theme(axis.text.x=element_text(angle = -45, hjust = 0))&#xA;fig&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/I42lS.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/I42lS.jpg&quot; alt=&quot;I would like to replicate something like the middle one &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help or suggestion or help would be highly appreciated &lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2019-01-11T14:06:02.097" Title="Ploting FDR along with the pathway as heatmap any simple way" Tags="|r|visualization|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="6797" PostTypeId="2" ParentId="6794" CreationDate="2019-01-11T14:44:23.913" Score="2" Body="&lt;p&gt;I don't know any python (or R package) to do this), but this is how I would do it based on these matrices:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Do a multidimensional scaling (MDS) plot of the target_similarity matrix.  Explore the visualization: &lt;/li&gt;&#xA;&lt;li&gt;Find which drug targets the drug closest to the 0,0 point in the 2D space&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;However this doesn't say anything about the affinity between the drug and the target.&#xA;If you wanted to add this then you could weight how much close is the distance between the drug and the target vs how close are they to the center of the MDS (a simple division could work if the distance is scaled between 0 and 1 and averaged).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that depending on the original information used to calculate the similarities and the affinities it could be done in other ways&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2019-01-11T14:44:23.913" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6798" PostTypeId="2" ParentId="6795" CreationDate="2019-01-11T15:11:09.627" Score="2" Body="&lt;p&gt;I am not familiar with Gviz, but I am familiar with a variety of tools for visualizing genome features. At the scale of entire microbial genomes, the primary sequence itself is an overwhelming amount of information and can't really be visualized in a meaningful way by itself. I mean, you could color the nucleotides, but few patterns in DNA can be discerned by an unassisted human eye.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What we're typically interested in are features encoded in the genome sequence: protein-coding genes, non-coding RNAs, transposable elements, regulatory elements, binding sites, epigenetic modifications, and so on. A particular scientist is usually only interested in a subset of these features, based on what particular questions they are investigating. In your comment you say you &quot;&lt;em&gt;would like to annotate protein coding regions and compare them to other genomes&lt;/em&gt;&quot;, so it seems like we're on the same page here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Visualization tools often display the genome linearly (even, for example, if it's a circular bacterial chromosome), zoomed out so that the nucleotide sequence isn't visible, and overlayed with boxes, arrows, or other glyphs representing the genes or other features of interest. Here are some examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/q/5171/96&quot;&gt;This question thread&lt;/a&gt; contains images generated with the &lt;a href=&quot;http://genometools.org/annotationsketch.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GenomeTools AnnotationSketch program&lt;/a&gt; (&lt;code&gt;gt sketch&lt;/code&gt; command).&lt;/li&gt;&#xA;&lt;li&gt;The UCSC genome browser which provides web-based access to &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTracks?db=hg38&amp;amp;lastVirtModeType=default&amp;amp;lastVirtModeExtraState=&amp;amp;virtModeType=default&amp;amp;virtMode=0&amp;amp;nonVirtPosition=&amp;amp;position=chr1%3A11102837%2D11267747&amp;amp;hgsid=706397415_aaU7aADCAPm3vbNa7iTImwAxTqcu&quot; rel=&quot;nofollow noreferrer&quot;&gt;dynamic displays of the human genome&lt;/a&gt; and many other mammalian genomes.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;http://software.broadinstitute.org/software/igv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IGV browser&lt;/a&gt; can be downloaded and easily run on your personal laptop or desktop computer.&lt;/li&gt;&#xA;&lt;li&gt;I'm not familiar with &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/Gviz.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gviz&lt;/a&gt;, but it definitely looks like it belongs in this class of tools.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In each of these cases, the visualization software is not doing the work to determine the positions of the genes and other genome features. This was done independently using tools designed for this task, storing the information in an &lt;strong&gt;annotation&lt;/strong&gt; file in &lt;a href=&quot;https://github.com/The-Sequence-Ontology/Specifications/blob/master/gff3.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;GFF3&lt;/a&gt;, &lt;a href=&quot;http://mblab.wustl.edu/GTF22.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GTF&lt;/a&gt;, or &lt;a href=&quot;https://genome.ucsc.edu/FAQ/FAQformat.html#format1&quot; rel=&quot;nofollow noreferrer&quot;&gt;BED&lt;/a&gt; format. The visualization tools need both the genome sequence (in Fasta format) and one or more annotation files (in GFF3, GTF, or BED format) to make meaningful visualizations.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-01-11T15:11:09.627" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6799" PostTypeId="2" ParentId="6758" CreationDate="2019-01-12T01:10:09.217" Score="3" Body="&lt;p&gt;If your null hypothesis is that the event of being selected by your program is statistically independent from the event of being non-coding, you can use &lt;a href=&quot;https://en.wikipedia.org/wiki/Fisher%27s_exact_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fisher's exact test&lt;/a&gt; to reject the null hypothesis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suppose you have 10,000 (I'm making that up but presumably you have the real number) total sequences, 8,700 coding and 1,300 noncoding. Given that your program selects 728 sequences, you would expect around 13%*728 ~= 95 noncoding sequences. But your program picked 131 noncoding sequences and you want to know if 131 is far enough away from 95 to conclude that the difference is not just due to chance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The probability of selecting 131 or more of these noncoding sequences (given that we are choosing 728 total sequences) is given by the so-called &lt;a href=&quot;https://en.wikipedia.org/wiki/Hypergeometric_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;hypergeometric distribution&lt;/a&gt;. The two-sided test will also incorporate the probably of getting an equally unlikely low number (in this case, somewhere around 57) or less.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In R for example, you can calculate this using the 2-by-2 contingency table as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fisher.test(rbind(c(597, 131), c(8103, 1169)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which gives:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p-value = 7.399e-05&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So in this situation (and assuming the probability model described), there's around a 0.0074% chance of observing a ratio as &quot;extreme&quot; as the one you observed, which indicates it's unlikely that the two events are independent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: As others have mentioned, a binomial test could be appropriate too. If your number of total sequences is very large, then the two results will converge. But if it isn't, then Fisher's exact test will be more appropriate (essentially since you're sampling without replacement).&lt;/p&gt;&#xA;" OwnerUserId="462" LastEditorUserId="462" LastEditDate="2019-01-12T23:23:43.450" LastActivityDate="2019-01-12T23:23:43.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6800" PostTypeId="1" CreationDate="2019-01-12T17:54:23.473" Score="-2" ViewCount="93" Body="&lt;p&gt;I have this matrix of expression data&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(data[,1:6])&#xA;  Symbol      cl1    cl1.1     cl1.2     cl1.3    cl1.4&#xA;1   ACTB 14.07922 14.25211 13.547206 14.046814 15.33551&#xA;2 ATP5F1 10.43386 10.34219  9.671036  9.862641 10.39786&#xA;3   DDX5 13.99250 13.29734 13.706894 13.084000 13.73379&#xA;4  EEF1G 14.35303 14.26112 13.523434 13.327333 13.97747&#xA;5  GAPDH 16.07695 15.61435 15.454911 15.233670 15.90100&#xA;6    NCL 14.35149 13.32919 13.051861 12.704973 13.55680&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have 24 samples as cl1 and 30 samples as cl2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to do a t.tset, I have this function but I am not sure Why I am getting error by &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;volcan &amp;lt;- function(data, cl1, cl2) {&#xA;   tt &amp;lt;- t(apply(data[, -1], 1, function(d){z &amp;lt;- t.test(d[cl1], d[cl2]); c(z&lt;span class=&quot;math-container&quot;&gt;$estimate, z$&lt;/span&gt;p.value)}))&#xA;     tt[, 1] &amp;lt;- tt[, 1] - tt[, 2]&#xA;     tt &amp;lt;- tt[, -2]&#xA;     tt &amp;lt;- cbind(symbol=data[, 1], as.data.frame(tt), rank=NA)&#xA;     tt[, 3] &amp;lt;- -log(tt[, 3], 10)&#xA;     colnames(tt)[2:3] &amp;lt;- c(&quot;log.ratio&quot;, &quot;neg.log(t.test)&quot;)&#xA;     tt[, 4] &amp;lt;- sqrt(tt[, 2]^4  tt[, 3]^2)&#xA;     tt &amp;lt;- tt[order(tt[, 4], decreasing=T), ]&#xA;     tt&#xA;}&#xA;    Error: unexpected symbol in &quot;tt[, 4] &amp;lt;- sqrt(tt[, 2]^4 tt&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I am running function as whole says that &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error: unexpected '}' in &quot;}&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I am putting &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;tt= volcan(data,cl1,cl2) &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and doing line by line going down after &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tt[, 4] &amp;lt;- sqrt(tt[, 2]^4  tt[, 3]^2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Says&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error: unexpected symbol in &quot;tt[, 4] &amp;lt;- sqrt(tt[, 2]^4  tt&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am not sure why, I put comma in&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; tt[, 4] &amp;lt;- sqrt(tt[, 2]^4 , tt[, 3]^2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But another error&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-01-12T18:52:29.903" LastActivityDate="2019-01-12T18:56:12.030" Title="Changing this function to work" Tags="|r|rna-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6801" PostTypeId="2" ParentId="6800" CreationDate="2019-01-12T18:00:23.187" Score="5" Body="&lt;p&gt;EDIT: I'm editing my answer as I don't have comment privileges and don't wish to add multiple answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please don't change the entire premise of the question after you get an answer. If you do, retain the original content so the post makes &lt;em&gt;some&lt;/em&gt; sense to people.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Run &lt;code&gt;?sqrt&lt;/code&gt; and check how the function is to be invoked, and how you're invoking it. Does it even accept 2 input parameters?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h2&gt;Original Content&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;You are subsetting the 2D object &lt;code&gt;data&lt;/code&gt; improperly. How can data be subset as both &lt;code&gt;data[,1:6]&lt;/code&gt; and &lt;code&gt;d[data[,1:24]]&lt;/code&gt; where &lt;code&gt;d&lt;/code&gt; = &lt;code&gt;data&lt;/code&gt;? Fix that and your r code will work OK (syntactically at least).&lt;/p&gt;&#xA;" OwnerUserId="650" LastEditorUserId="650" LastEditDate="2019-01-12T18:56:12.030" LastActivityDate="2019-01-12T18:56:12.030" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6802" PostTypeId="1" AcceptedAnswerId="8621" CreationDate="2019-01-13T16:20:53.530" Score="2" ViewCount="1603" Body="&lt;p&gt;After i updated my complexheatmap library i can;t use my old annotation to label sample this is the image how i used to get something like this &#xA;&lt;a href=&quot;https://i.stack.imgur.com/pHgIP.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pHgIP.jpg&quot; alt=&quot;with family and sample annotation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code for the below image&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;require(ComplexHeatmap)&#xA;require(circlize)&#xA;require(cluster)&#xA;df &amp;lt;- read.csv(&quot;heatmap_folder/NORMAL_EPI_TF1.txt&quot;,header = TRUE,sep = &quot;\t&quot;)&#xA;&#xA;head(df)&#xA;tail(df)&#xA;dim(df)&#xA;heat &amp;lt;- t(scale(t(df[,3:ncol(df)])))&#xA;head(heat)&#xA;&#xA;rownames(heat) &amp;lt;- df$Gene&#xA;Sample = c(rep(&quot;HSC&quot;,4),rep(&quot;CMP&quot;,4),rep(&quot;GMP&quot;,4),rep(&quot;Mono&quot;,4))&#xA;&#xA;anno &amp;lt;- as.data.frame(Sample)&#xA;categories &amp;lt;- data.frame(anno)&#xA;colnames(anno) &amp;lt;- c(&quot;Condition&quot;)&#xA;&#xA;ColAnn &amp;lt;- HeatmapAnnotation(df=anno, which=&quot;col&quot;, col=list(Condition=c(&quot;HSC&quot;=&quot;black&quot;,&quot;CMP&quot;=&quot;brown&quot;,&#xA;                                                                       &quot;GMP&quot;=&quot;maroon1&quot;,&quot;Mono&quot; = &quot;darkgreen&quot;)))&#xA;RowAnn &amp;lt;- data.frame(df$Family)&#xA;colnames(RowAnn) &amp;lt;- c(&quot;Gene family&quot;)&#xA;RowAnn &amp;lt;- HeatmapAnnotation(df=RowAnn, col=colours, which=&quot;row&quot;)&#xA;&#xA;&#xA;boxAnnCol &amp;lt;- HeatmapAnnotation(boxplot=anno_boxplot(heat, border=TRUE, gp=gpar(fill=&quot;#CCCCCC&quot;), &#xA;                                                    pch=&quot;.&quot;, size=unit(2, &quot;mm&quot;), axis=TRUE, axis_side=NULL, axis_gp=gpar(fontsize=10)), &#xA;                               annotation_width=unit(c(1, 6.5), &quot;cm&quot;))&#xA;&#xA;boxAnnRow &amp;lt;- rowAnnotation(boxplot=row_anno_boxplot(heat, border=TRUE, gp=gpar(fill=&quot;#CCCCCC&quot;),&#xA;                                                    pch=&quot;.&quot;, size=unit(3, &quot;cm&quot;), axis=TRUE, axis_side=&quot;top&quot;, axis_gp=gpar(fontsize=12)),&#xA;                           annotation_width=unit(c(3), &quot;cm&quot;))&#xA;&#xA;&#xA;&#xA;myCol &amp;lt;- colorRampPalette(c(&quot;navyblue&quot;, &quot;white&quot;, &quot;red&quot;))(100)&#xA;myBreaks &amp;lt;- seq(-2, 2, length.out=100)&#xA;hmap &amp;lt;- Heatmap(heat,&#xA;                name=&quot;Z-score&quot;,&#xA;                col=colorRamp2(myBreaks, myCol),&#xA;                heatmap_legend_param=list(color_bar=&quot;continuous&quot;, &#xA;                                          legend_direction=&quot;vertical&quot;, legend_width=unit(5,&quot;cm&quot;),&#xA;                                          title_position=&quot;topcenter&quot;, title_gp=gpar(fontsize=10, fontface=&quot;bold&quot;)),&#xA;&#xA;                #Split heatmap rows by gene family&#xA;                split=df$Family,&#xA;                #split = NULL,&#xA;&#xA;                #Row annotation configurations&#xA;                cluster_rows=TRUE,&#xA;                show_row_dend=FALSE,&#xA;                row_dend_side = &quot;left&quot;,&#xA;                #row_title=&quot;Transcript&quot;, #overridden by 'split' it seems&#xA;                row_title_side=&quot;left&quot;,&#xA;                row_title_gp=gpar(fontsize=8,fontface=&quot;bold&quot;),&#xA;                row_names_gp = gpar(fontsize = 8),&#xA;                show_row_names=TRUE,&#xA;                row_names_side=&quot;right&quot;,&#xA;                row_title_rot=0,&#xA;&#xA;                #Column annotation configuratiions&#xA;                cluster_columns=FALSE,&#xA;                show_column_dend=TRUE,&#xA;                column_title=&quot;Samples&quot;,&#xA;                column_title_side=&quot;top&quot;,&#xA;                column_title_gp=gpar(fontsize=10, fontface=&quot;bold&quot;),&#xA;                column_title_rot=0,&#xA;                column_names_gp = gpar(fontsize = 20, fontface=&quot;bold&quot;),&#xA;                show_column_names=TRUE,&#xA;&#xA;                #Dendrogram configurations: columns&#xA;                clustering_distance_columns=&quot;euclidean&quot;,&#xA;                clustering_method_columns=&quot;complete&quot;,&#xA;                column_dend_height=unit(10,&quot;mm&quot;),&#xA;&#xA;                #Dendrogram configurations: rows&#xA;                clustering_distance_rows=&quot;euclidean&quot;,&#xA;                clustering_method_rows=&quot;complete&quot;,&#xA;                row_dend_width=unit(15,&quot;mm&quot;),&#xA;&#xA;                top_annotation_height=unit(0.5,&quot;cm&quot;),&#xA;                top_annotation=ColAnn,&#xA;&#xA;                bottom_annotation_height=unit(2, &quot;cm&quot;),&#xA;                bottom_annotation=boxAnnCol)&#xA;hmap + RowAnn&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The code remains the except the data &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now not sure may be im messing up the annotation so that i can;t see sample label and the family annotation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since this was the error &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;Error: &lt;code&gt;axis_gp&lt;/code&gt; is removed from the arguments. Use &lt;code&gt;axis_param =&#xA;  list(gp = ...)&lt;/code&gt; instead.&quot; which i modified a bit &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Then i modified my code to this &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;RowAnn &amp;lt;- data.frame(df$Family)&#xA;colnames(RowAnn) &amp;lt;- c(&quot;Gene family&quot;)&#xA;RowAnn &amp;lt;- HeatmapAnnotation(df=RowAnn, col=colours, which=&quot;row&quot;)&#xA;&#xA;ColAnn &amp;lt;- HeatmapAnnotation(df=anno, which=&quot;col&quot;, col=list(Condition=c(&quot;HSC&quot;=&quot;black&quot;,&quot;CMP&quot;=&quot;brown&quot;,&#xA;                                                                       &quot;GMP&quot;=&quot;maroon1&quot;,&quot;Mono&quot; = &quot;darkgreen&quot;)))&#xA;&#xA;boxplotCol &amp;lt;- HeatmapAnnotation(boxplot=anno_boxplot(heat,&#xA;  border=TRUE, gp=gpar(fill=&quot;#CCCCCC&quot;), pch=&quot;.&quot;, &#xA;  size=unit(2, &quot;mm&quot;), axis=TRUE, axis_param = list(side = &quot;left&quot;) ,&#xA;  annotation_width=unit(c(1, 5.0), &quot;cm&quot;), which=&quot;col&quot;)&#xA;)&#xA;&#xA;hmap &amp;lt;- Heatmap(heat, name = &quot;expression&quot;,col=colorRamp2(myBreaks, myCol),&#xA;                split=df$Family,&#xA;                #split = NULL,&#xA;&#xA;                show_row_names = TRUE, show_column_names = TRUE, cluster_rows = TRUE,&#xA;                cluster_columns = TRUE, show_column_dend = TRUE, show_row_dend = TRUE,&#xA;                row_dend_reorder = TRUE, column_dend_reorder = TRUE, clustering_method_rows = &quot;ward.D2&quot;,&#xA;                clustering_method_columns = &quot;ward.D2&quot;, width = unit(100, &quot;mm&quot;),&#xA;&#xA;                #top_annotation_height=unit(1.0,&quot;cm&quot;), top_annotation=colAnn,&#xA;&#xA;                bottom_annotation=boxplotCol)&#xA;&#xA;draw(hmap+RowAnn, heatmap_legend_side=&quot;left&quot;, annotation_legend_side=&quot;right&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;my fig&lt;a href=&quot;https://i.stack.imgur.com/KAipK.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KAipK.jpg&quot; alt=&quot;new &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be glad to know  what im doing wrong as i can;t she condition getting annotated &lt;/p&gt;&#xA;&#xA;&lt;p&gt;my data &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dput(df)&#xA;structure(list(Family = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, &#xA;1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, &#xA;2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, &#xA;2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L), .Label = c(&quot;EPI&quot;, &#xA;&quot;TF&quot;), class = &quot;factor&quot;), Gene = structure(c(31L, 23L, 32L, 40L, &#xA;26L, 34L, 14L, 17L, 39L, 21L, 15L, 30L, 9L, 1L, 33L, 2L, 5L, &#xA;11L, 18L, 16L, 7L, 27L, 8L, 19L, 22L, 12L, 24L, 36L, 6L, 20L, &#xA;28L, 41L, 43L, 44L, 45L, 47L, 48L, 49L, 50L, 13L, 17L, 38L, 42L, &#xA;21L, 25L, 35L, 37L, 10L, 11L, 4L, 46L, 3L, 29L), .Label = c(&quot;BABAM1&quot;, &#xA;&quot;BAP1&quot;, &quot;BRF2&quot;, &quot;CASZ1&quot;, &quot;CBX8&quot;, &quot;CCDC17&quot;, &quot;CEBPA&quot;, &quot;CUX2&quot;, &quot;DAPK3&quot;, &#xA;&quot;DEAF1&quot;, &quot;DOT1L&quot;, &quot;EBF1&quot;, &quot;ERF&quot;, &quot;HCFC1&quot;, &quot;HDAC7&quot;, &quot;HES6&quot;, &quot;HMG20B&quot;, &#xA;&quot;INO80E&quot;, &quot;IRF8&quot;, &quot;KLF16&quot;, &quot;MBD3&quot;, &quot;MEF2D&quot;, &quot;MTA3&quot;, &quot;MXD4&quot;, &quot;PIN1&quot;, &#xA;&quot;PIWIL4&quot;, &quot;PREB&quot;, &quot;REPIN1&quot;, &quot;RFX2&quot;, &quot;SETD1A&quot;, &quot;SFMBT1&quot;, &quot;SFMBT2&quot;, &#xA;&quot;SMARCA4&quot;, &quot;SMYD2&quot;, &quot;SNAPC4&quot;, &quot;SREBF2&quot;, &quot;TCF20&quot;, &quot;TCF7&quot;, &quot;TFPT&quot;, &#xA;&quot;UBE2A&quot;, &quot;ZBTB17&quot;, &quot;ZFHX2&quot;, &quot;ZNF142&quot;, &quot;ZNF319&quot;, &quot;ZNF414&quot;, &quot;ZNF446&quot;, &#xA;&quot;ZNF469&quot;, &quot;ZNF581&quot;, &quot;ZNF646&quot;, &quot;ZNF76&quot;), class = &quot;factor&quot;), HSC1 = c(1.38256100990102, &#xA;6.03508433561645, 8.53718694596963, 9.7579, 4.9089643439748, &#xA;6.08652168925513, 10.85362, 5.791612, 4.717658, 5.034092, 7.6573073927514, &#xA;8.214206, 6.087519, 7.795752, 9.147491, 9.16065391580272, 5.475244, &#xA;6.839048, 4.307532, 0.4008883181761, 3.681525, 5.39357937244362, &#xA;0.048702874829245, 4.884674, 9.42704096616912, 0.331449426359954, &#xA;8.70131868583511, 9.860997, 5.96926894038996, 5.122815, 9.84364182958174, &#xA;4.99646644487682, 8.64613509396529, 6.221252, 4.788327, 4.221885, &#xA;6.904238, 7.027222, 6.90290115677703, 5.910114, 5.791612, 7.85232611449929, &#xA;4.30633834185685, 5.034092, 6.587824, 3.77111448331312, 4.816565, &#xA;5.83621984608683, 6.839048, 5.38002542796833, 2.499083, 4.07296733289583, &#xA;7.384605), HSC2 = c(3.3250440950498, 5.69710569315867, 7.53535864842004, &#xA;9.46294, 4.63015366085456, 6.27268560573257, 10.61808, 5.936903, &#xA;5.754081, 5.17955, 7.34546746135814, 8.156995, 4.959736, 7.1724, &#xA;9.519366, 8.399964626033, 3.79417, 6.580323, 3.006501, 0.465543296427136, &#xA;3.854056, 6.19801925234426, 1.35661238539194, 2.520579, 10.1179535946398, &#xA;0.438637606305892, 8.02299949380521, 9.599955, 5.14592215971399, &#xA;5.362818, 9.62067506274524, 5.81177950264385, 8.6388082408386, &#xA;6.580456, 5.507812, 4.647564, 5.933962, 7.036506, 6.61438422233323, &#xA;7.081847, 5.936903, 7.25061985280206, 6.14119015507351, 5.17955, &#xA;5.51992, 4.7980903060972, 4.688224, 5.75480261400296, 6.580323, &#xA;4.61480421865767, 2.473632, 2.58901029457056, 7.141953), HSC3 = c(4.5609308176212, &#xA;6.31335736623603, 8.08532370258223, 9.954938, 5.33971874563167, &#xA;6.31519305036765, 8.686445, 3.652774, 1.705097, 3.590782, 5.82598582988908, &#xA;7.160368, 2.730684, 6.544555, 8.308113, 7.28529068473834, 2.181592, &#xA;6.153712, 3.769699, 0.686412239668329, 3.181293, 5.81660138585872, &#xA;0.513314099277583, 3.807037, 9.40253894125251, 0.789263521770297, &#xA;6.25643885795079, 8.4667, 3.59885151800005, 1.959458, 8.16946518123644, &#xA;3.78892554507724, 8.17931637873862, 6.871673, 2.207662, 2.586419, &#xA;4.611411, 6.342816, 6.24601538582422, 4.504617, 3.652774, 6.1448035804027, &#xA;4.32474896196878, 3.590782, 4.283432, 3.97377694799887, 5.081613, &#xA;4.67353303760197, 6.153712, 2.95856351440761, 1.797669, 2.61144324906345, &#xA;6.927889), HSC4 = c(4.33336593087016, 5.51218211320577, 7.98927439884575, &#xA;10.02818, 4.72376095157983, 7.06408657043682, 10.6532, 6.332262, &#xA;4.356266, 5.063746, 7.40605558817077, 7.943625, 4.577773, 7.114057, &#xA;10.10959, 8.39439253099313, 0.843862, 7.275198, 3.125764, 0.461081947661647, &#xA;4.914624, 6.47822557945205, 0.151047922603591, 4.233378, 9.76778876120242, &#xA;1.52153581933298, 7.74186897540428, 9.848883, 4.27034862337376, &#xA;5.461201, 9.20573611283668, 5.57160015049504, 8.90499660789483, &#xA;5.972766, 4.37926, 4.914465, 6.288035, 6.667292, 6.48347655570048, &#xA;6.525983, 6.332262, 6.67910931337338, 5.93921207229708, 5.063746, &#xA;5.391649, 5.14382963917955, 3.682687, 6.17864627473077, 7.275198, &#xA;4.55820389425563, 2.464959, 3.36552973589941, 7.155267), CMP1 = c(6.87345497292835, &#xA;5.51412758368083, 7.31352446967914, 10.55862, 6.25444074607741, &#xA;7.17818981714724, 11.09789, 6.353011, 4.176257, 5.988031, 8.45885235860671, &#xA;8.239023, 4.831419, 8.439534, 9.741489, 9.12973766558991, 1.63498, &#xA;7.616455, 4.40003, 3.35836219130892, 6.782445, 8.09320436383016, &#xA;0.034087182831375, 7.328077, 10.1348328626404, 3.77197918672699, &#xA;8.22105235434344, 9.586611, 3.96736165755976, 6.120908, 9.66405823977144, &#xA;5.5410623284247, 7.98439042490943, 5.617969, 4.389489, 3.320773, &#xA;7.259381, 6.342179, 7.80157963446663, 6.00359, 6.353011, 4.88520066048698, &#xA;4.57336386910792, 5.988031, 7.534274, 4.22034258843835, 4.407149, &#xA;6.04554630191099, 7.616455, 5.5286242078284, 2.938658, 6.05835585199437, &#xA;7.254343), CMP2 = c(4.57632794651802, 3.78653056181795, 7.38323687896953, &#xA;10.18887, 5.42618113653324, 7.14753074148422, 11.10573, 6.444083, &#xA;5.364945, 5.56755, 7.87006213457315, 8.183806, 5.547153, 7.408484, &#xA;10.41511, 8.5734105703595, 3.332623, 7.927959, 3.044397, 2.23975823913519, &#xA;5.948004, 6.8523382099618, -0.208208601046198, 2.502613, 9.61958525630466, &#xA;0.083082776643226, 7.96816766770755, 9.658819, 4.28029266262047, &#xA;6.064432, 9.51407062066367, 6.0002887227803, 8.84860518364821, &#xA;6.63929, 5.418921, 4.246354, 6.265858, 6.726515, 6.59033513636581, &#xA;6.599091, 6.444083, 6.42468503826279, 5.39157032731654, 5.56755, &#xA;5.975197, 5.49143188875569, 4.882971, 6.3269787223519, 7.927959, &#xA;4.52014312274917, 3.02202, 3.32107044636493, 7.234948), CMP3 = c(5.85154481845215, &#xA;4.70657054545281, 7.81269743162002, 10.51041, 6.02525567584962, &#xA;7.6061510389796, 8.938033, 4.185122, 3.453336, 4.250061, 6.95084523398581, &#xA;7.016165, 3.711631, 6.691113, 9.291314, 6.79870749458327, 2.556527, &#xA;6.640901, 2.472737, 0.713500223112626, 4.610029, 7.09501723097758, &#xA;0.554991742557483, 3.663828, 8.64957961123419, 0.830920666997163, &#xA;5.03120897448563, 8.193863, 3.66385830275847, 2.514419, 8.57920717695102, &#xA;3.68106350062846, 7.99067978896729, 4.917504, 1.705394, 2.940584, &#xA;4.935565, 4.679952, 5.45403468978382, 4.097788, 4.185122, 4.7017134210513, &#xA;3.36996680968093, 4.250061, 4.841303, 4.16528140537129, 4.311709, &#xA;4.34280333979994, 6.640901, 3.46696931369128, 1.836416, 1.2954688478789, &#xA;6.759544), CMP4 = c(5.70607301130076, 4.90995364796149, 8.07668148492955, &#xA;10.49127, 6.12272998130959, 8.1070578791571, 11.14272, 6.069435, &#xA;3.77716, 5.905144, 7.80284615350651, 8.193236, 5.266133, 7.994962, &#xA;10.62971, 8.76622729912917, 1.728943, 8.018743, 2.64787, 3.17362412630739, &#xA;6.675791, 8.17425862958478, 1.3062705649262, 8.305058, 9.60100662458438, &#xA;3.64083547966437, 7.03151917955059, 9.697494, 4.22387311586196, &#xA;5.872986, 9.48040032459999, 5.9054376735385, 9.0993222961645, &#xA;6.641411, 4.042785, 4.175351, 6.040673, 7.025595, 6.8701243794046, &#xA;6.474802, 6.069435, 5.98875791029547, 5.44536133412665, 5.905144, &#xA;6.253757, 5.55713735563886, 3.183721, 6.23569788279688, 8.018743, &#xA;4.14168543985438, 3.140228, 2.85944735485205, 7.19532), GMP1 = c(6.00003380096822, &#xA;6.45782274084263, 10.1386873750581, 11.18212, 4.99574155426759, &#xA;8.22548270576452, 11.80921, 6.139842, 4.716938, 6.208399, 7.25702470622416, &#xA;9.216353, 4.459075, 7.604192, 10.214, 9.71979154806924, 4.045693, &#xA;6.892321, 4.36733, 4.54209088244969, 4.363115, 7.42142574723769, &#xA;8.16658165526266, 13.13801, 11.7030896033115, 1.38480651276548, &#xA;7.70549837966537, 10.51655, 5.6799276696584, 7.117451, 10.4613436109924, &#xA;7.51060888343217, 9.45360701936786, 11.02082, 4.741664, 4.37359, &#xA;6.757818, 7.101331, 6.51838172412493, 6.234017, 6.139842, 7.16147421243415, &#xA;5.80674368005962, 6.208399, 6.860234, 4.69772329684801, 5.404108, &#xA;6.45675195710298, 6.892321, 5.30230900977469, 4.155601, 4.36874064027233, &#xA;7.42003), GMP2 = c(5.74747276487971, 5.97553824097351, 8.84397150139003, &#xA;10.54739, 7.01942050935297, 8.32418895763336, 11.28484, 6.100887, &#xA;5.371866, 5.782092, 6.81459497341517, 8.300889, 4.573727, 7.155416, &#xA;10.55908, 8.53584780642442, 3.288921, 7.981419, 3.343707, 1.78160205563733, &#xA;7.106303, 6.97534663269409, 4.30909405726115, 10.94426, 10.9546928815164, &#xA;6.74862372900741, 7.1870575783895, 9.650893, 4.16099729988147, &#xA;6.067064, 9.97780340347263, 5.92239739476255, 9.00840651100957, &#xA;5.952921, 5.034178, 4.255218, 5.533772, 6.671744, 6.58968157482356, &#xA;6.572507, 6.100887, 6.06061888187195, 4.67987580614607, 5.782092, &#xA;5.976481, 5.57605980178547, 4.822138, 5.9343704738438, 7.981419, &#xA;4.0036610231274, 2.709241, 2.35084397835661, 7.122418), GMP3 = c(6.86258223460792, &#xA;6.20078322375291, 9.61446552902509, 10.80572, 7.36372917472567, &#xA;8.3826076358515, 11.15008, 5.201591, 4.123945, 5.045783, 6.24154676477513, &#xA;8.276453, 4.263376, 7.519041, 10.51785, 8.30200623104098, 2.495567, &#xA;7.857207, 2.962492, 1.93915883789869, 7.249551, 7.84810925448229, &#xA;5.84602007858824, 11.77414, 11.1027524778454, 5.63020389504886, &#xA;6.50727637405781, 9.456897, 3.75134646474614, 5.932853, 10.1039559568546, &#xA;4.9400044296399, 9.31472753734898, 6.039605, 3.336343, 3.975937, &#xA;5.736387, 6.905666, 6.30344093479224, 5.426484, 5.201591, 5.47026133533826, &#xA;4.69976751703472, 5.045783, 5.749752, 5.23937658540781, 5.154187, &#xA;6.34478978856221, 7.857207, 3.50805939441741, 2.657933, 2.78283797334871, &#xA;7.091083), GMP4 = c(6.05880869638876, 6.28654396576068, 9.96479543345143, &#xA;10.73202, 7.213420739218, 8.81669595423024, 11.31576, 5.679182, &#xA;4.155314, 5.673872, 6.43309255133785, 8.167664, 4.513147, 7.768381, &#xA;10.68025, 8.7651559260891, 2.485242, 8.41621, 3.457858, 2.07304058721841, &#xA;6.412872, 8.09668562132081, 5.97986866984983, 12.08623, 11.28032878957, &#xA;6.57392060927003, 6.41585754599334, 9.749641, 3.66579392385564, &#xA;5.878271, 9.75282317403084, 5.77880844881496, 9.31419691100924, &#xA;5.048616, 3.784422, 4.082659, 6.015668, 6.789027, 6.44947035574528, &#xA;6.177861, 5.679182, 5.40028170642952, 4.40100191946731, 5.673872, &#xA;6.189491, 5.32217020480945, 3.106884, 5.98621706786421, 8.41621, &#xA;3.507010215991, 2.810708, 2.26597487448895, 6.822586), Mono1 = c(5.2563684147039, &#xA;6.45633325704225, 8.35683375157621, 10.4683, 6.89578871222261, &#xA;6.89453589831897, 10.76922, 5.79888, 3.938016, 4.625604, 8.14811443428627, &#xA;8.89819, 4.565085, 7.582637, 9.690787, 9.26841770880937, 2.680727, &#xA;7.256856, 5.506176, 1.73004263683502, 7.168846, 7.34133258574044, &#xA;2.04394933289888, 10.88391, 10.8831286687177, 2.76098817217713, &#xA;7.62084289021189, 9.376334, 5.89374731019185, 5.059797, 9.58406921227736, &#xA;6.10083260496288, 8.90601784817498, 8.926934, 4.514644, 4.671412, &#xA;6.824876, 7.160325, 7.12646156474527, 6.029051, 5.79888, 7.45127447114964, &#xA;6.58727075697179, 4.625604, 6.726376, 4.54492833429885, 5.482923, &#xA;5.75789405204644, 7.256856, 5.01661617666884, 3.912327, 4.21481739920369, &#xA;7.311136), Mono2 = c(5.85774218629914, 5.24626015596457, 7.87393184409552, &#xA;10.74763, 6.1115477644807, 8.59285304183197, 11.30387, 6.487999, &#xA;5.641296, 5.605635, 6.83338037544917, 8.158746, 5.031193, 7.791207, &#xA;10.12365, 8.83146107303257, 3.839879, 7.995034, 3.864015, 1.95600811707846, &#xA;6.547528, 7.15767104450585, 1.13087340949285, 10.60541, 10.6662569861529, &#xA;2.45950067788357, 7.80075639569934, 9.772267, 4.10768104209722, &#xA;6.039518, 9.45126490091168, 5.43166366925388, 8.53913541490463, &#xA;6.852944, 5.210281, 3.909477, 6.10825, 6.759114, 6.10127101835935, &#xA;6.394382, 6.487999, 6.77443082143061, 4.65095032762504, 5.605635, &#xA;6.292733, 5.06179443435785, 4.687693, 5.97873952456647, 7.995034, &#xA;4.53685791636727, 2.036068, 3.79175025763666, 7.80933), Mono3 = c(5.79480048210947, &#xA;5.93841021449181, 8.4456428811783, 10.96997, 5.90368127873194, &#xA;7.57633522462406, 10.77521, 5.357294, 3.631277, 5.573695, 6.02937219504517, &#xA;7.823033, 4.89389, 7.134801, 9.942904, 8.29297682562671, 2.376142, &#xA;7.942619, 2.938838, 2.58663314475541, 5.473615, 6.54736080917345, &#xA;1.50818476189011, 10.39009, 9.81896143227971, 2.61179572528994, &#xA;7.11294273571537, 9.321912, 4.59467076052555, 4.846162, 9.0889984655259, &#xA;5.5580711224014, 8.60684080318993, 6.807437, 3.781213, 3.349656, &#xA;5.352098, 6.732808, 6.18873397274794, 5.580769, 5.357294, 6.02896733254353, &#xA;4.09719798968178, 5.573695, 5.576249, 4.76264681239467, 4.599397, &#xA;5.85107919870538, 7.942619, 4.46991267319193, 2.359072, 2.49361466378781, &#xA;7.168415), Mono4 = c(5.79136484133828, 5.22885638262284, 8.66188302349651, &#xA;10.80097, 6.02816710441609, 8.28180453176157, 10.95579, 5.606963, &#xA;4.781397, 6.146859, 7.35941757150855, 8.279649, 5.035288, 8.131193, &#xA;10.46082, 8.7442177262891, 2.904981, 8.009644, 2.855958, 2.31248706248342, &#xA;7.171729, 7.85215015030227, 1.5674747074644, 10.34766, 10.4041048711624, &#xA;1.7198811173707, 7.7219124608883, 9.652986, 4.80881131147327, &#xA;5.631832, 9.37540653744023, 5.93520028734304, 8.83460813839155, &#xA;7.145424, 4.609511, 4.550807, 6.666205, 6.956604, 6.66113029874611, &#xA;6.54366, 5.606963, 5.47022342513485, 4.59138757680696, 6.146859, &#xA;5.969648, 5.20105550006015, 3.857023, 6.22204033269913, 8.009644, &#xA;4.44000233834592, 2.854773, 2.88863557103663, 6.788677)), class = &quot;data.frame&quot;, row.names = c(NA, &#xA;-53L))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="285" LastEditorUserId="131" LastEditDate="2019-05-11T07:13:44.073" LastActivityDate="2019-05-11T15:56:25.707" Title="Complexheatmap update issue" Tags="|r|visualization|heatmap|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6803" PostTypeId="1" CreationDate="2019-01-13T17:08:59.513" Score="2" ViewCount="96" Body="&lt;p&gt;I have a whole-gene CRISPR screening of an assay at different time points. I created a table of the gene number and the count of its reads:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gene_id, count_t0, count_t1, count_t3&#xA;ABCA1       3        6          7&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What would be a standard way to process such type of data to inference information on up-regulated/down-regulated genes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I saw this tool: &lt;a href=&quot;http://crispr-analyzer.dkfz.de/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://crispr-analyzer.dkfz.de/&lt;/a&gt;, but it requires my NGS raw data and these file are very large in my case. I'd expect that the information for analysis purposes can be derived solely from the counting table above. &lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2019-01-15T00:28:39.120" LastActivityDate="2019-01-15T00:28:39.120" Title="How to analyze and visualize CRISPR-CAS9 screening results?" Tags="|crispr|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6804" PostTypeId="1" AcceptedAnswerId="6809" CreationDate="2019-01-13T21:41:25.683" Score="3" ViewCount="4914" Body="&lt;p&gt;I'm using Snakemake and I'm trying to get all my QC reports into multiqc but I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;WorkflowError in line 120 of /rst1/2017-0205_illuminaseq/scratch/swo-406/test_snakemake_full/Snakefile:&#xA;'Wildcards' object has no attribute 'sample'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Rule 120 is the start of my multiqc rule which looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule multiqc:&#xA;    input:&#xA;        expand([os.path.join(analyzed_dir, '{sample}.stat', '{sample}.cnt'),&#xA;                os.path.join(rseqc_dir, '{sample}.bam_stat.txt'),&#xA;                os.path.join(rseqc_dir, '{sample}.clipping_profile.xls'),&#xA;                os.path.join(rseqc_dir, '{sample}.deletion_profile.txt'),&#xA;                os.path.join(rseqc_dir, '{sample}.infer_experiment.txt'),&#xA;                os.path.join(rseqc_dir, '{sample}.geneBodyCoverage.txt'),&#xA;                os.path.join(rseqc_dir, '{sample}.inner_distance.txt'),&#xA;                os.path.join(rseqc_dir, '{sample}.insertion_profile.xls'),&#xA;                os.path.join(rseqc_dir, '{sample}.junction.xls'),&#xA;                os.path.join(rseqc_dir, '{sample}.junctionSaturation_plot.r'),&#xA;                os.path.join(rseqc_dir, '{sample}.mismatch_profile.xls'),&#xA;                os.path.join(rseqc_dir, '{sample}.read_distribution.txt'),&#xA;                os.path.join(rseqc_dir, '{sample}.pos.DupRate.xls'),&#xA;                os.path.join(rseqc_dir, '{sample}.seq.DupRate.xls'),&#xA;                os.path.join(rseqc_dir, '{sample}.GC.xls'),&#xA;                os.path.join(rseqc_dir, '{sample}.NVC.xls'),&#xA;                os.path.join(rseqc_dir, '{sample}.qual.r'),&#xA;                os.path.join(rseqc_dir, '{sample}.RNA_fragment_size.txt'),&#xA;                os.path.join(rseqc_dir, '{sample}.STAR.genome.sorted.summary.txt'),&#xA;                os.path.join(fastqc_dir, '{sample}_R{read}_001_fastqc.html')],&#xA;                sample=samples['samples'], read=reads)&#xA;    output:&#xA;        os.path.join(qc_dir, 'multiqc_report.html') &#xA;    conda:&#xA;        &quot;envs/multiqc.yaml&quot;&#xA;    shell:&#xA;        '''&#xA;        #!/bin/bash&#xA;        multiqc . --outdir qc --ignore .snakemake --force&#xA;        '''&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My &quot;rule all looks like this:&quot;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule all:&#xA;    input:&#xA;        expand([os.path.join(analyzed_dir, '{sample}.genes.results'),&#xA;                os.path.join(qc_dir, 'multiqc_report.html')],&#xA;                sample=samples['samples'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't see anything wrong with it, in fact, it is very similar to my &quot;rule all&quot; I used before and which worked. Can anyone see an error?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I limited the rule to only one input, I still have the same error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule multiqc:&#xA;    input:&#xA;        expand(os.path.join(rseqc_dir, '{sample}.geneBodyCoverage.txt'),&#xA;                sample=samples['samples'])&#xA;    output:&#xA;        os.path.join(qc_dir, 'multiqc_report.html') &#xA;    conda:&#xA;        &quot;envs/multiqc.yaml&quot;&#xA;    shell:&#xA;        '''&#xA;        #!/bin/bash&#xA;        multiqc . --outdir qc --ignore .snakemake --force&#xA;        '''&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the ouput, you can see it read the samples correctly:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;run_snakemake.sh &#xA;/rst1/2017-0205_illuminaseq/scratch/swo-406/snakemake/lib/python3.6/site-packages/snakemake/workflow.py:12: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'.&#xA;  from collections import OrderedDict&#xA;Building DAG of jobs...&#xA;/rst1/2017-0205_illuminaseq/scratch/swo-406/snakemake/lib/python3.6/site-packages/snakemake/workflow.py:12: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'.&#xA;  from collections import OrderedDict&#xA;Building DAG of jobs...&#xA;Using shell: /bin/bash&#xA;Provided cluster nodes: 256&#xA;Job counts:&#xA;    count   jobs&#xA;    1   all&#xA;    1   multiqc&#xA;    2&#xA;&#xA;[Mon Jan 14 17:46:40 2019]&#xA;rule multiqc:&#xA;    input: qc/rseqc/0067_P2018SEQE27R01_S1.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R02_S2.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R03_S3.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R04_S4.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R05_S5.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R06_S6.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R07_S7.geneBodyCoverage.txt, qc/rseqc/0067_P2018SEQE27R08_S8.geneBodyCoverage.txt&#xA;    output: qc/multiqc_report.html&#xA;    jobid: 2&#xA;&#xA;WorkflowError in line 119 of /rst1/2017-0205_illuminaseq/scratch/swo-406/test_snakemake_full/Snakefile:&#xA;'Wildcards' object has no attribute 'sample'&#xA;17:46 nlv24077@kiato /rst1/2017-0205_illuminaseq/scratch/swo-406/test_snakemake_full &amp;gt; ll qc/rseqc/0067_P2018SEQE27R07_S7.geneBodyCoverage.txt&#xA;-rw-rw-r-- 1 nlv24077 2017-0205_illuminaseq 303 Jan  7 16:00 qc/rseqc/0067_P2018SEQE27R07_S7.geneBodyCoverage.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;EDIT 2:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how I start Snakemake:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; cat run_snakemake.sh&#xA;source activate /rst1/2017-0205_illuminaseq/scratch/swo-406/snakemake&#xA;snakemake --dag | dot -T svg &amp;gt; dag.svg&#xA;snakemake --cluster-config cluster.json --cluster &quot;qsub -l nodes={cluster.nodes}:ppn={cluster.ppn} -N {wildcards.sample}_{rule}&quot; --jobs 256 --use-conda --rerun-incomplete&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;EDIT 3:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ok, that last remark (by @finswimmer) triggered me, I removed the &quot;{wildcards.sample}_&quot; from the above command and then it works. Or, works, I get a very different error, it is multiqc specific and solved as specified here: &lt;a href=&quot;https://multiqc.info/docs/#locale-error-messages&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://multiqc.info/docs/#locale-error-messages&lt;/a&gt;. When adding the two lines as suggested this error disappears and everything works as expected, even with the long inputs list.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;RuntimeError: Click will abort further execution because Python 3 was configured to use ASCII as encoding for the environment. Consult https://click.palletsprojects.com/en/7.x/python3/ for mitigation steps.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Anyway, I'm guessing Snakemake does not understand how to name this task when it uses an entire list of {sample}'s... Is this a bug? What should be used? I guess it is impossible to use {sample} for naming for this rule. It's a shame I have to kill the naming for all rules because using sample_rule is very nice for keeping track.&lt;/p&gt;&#xA;" OwnerUserId="3964" LastEditorUserId="3964" LastEditDate="2019-01-14T19:40:22.367" LastActivityDate="2020-01-16T11:23:18.193" Title="'Wildcards' object has no attribute 'sample'" Tags="|snakemake|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="6805" PostTypeId="1" AcceptedAnswerId="6812" CreationDate="2019-01-14T13:27:11.400" Score="0" ViewCount="409" Body="&lt;p&gt;How do i interpret the overall survival when the p value reported is not significant going by the standard convention .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not sure what kind of test it runs while comparing the expression may be one non-parametric test , but the survival is kaplan meier on cox ph model which they have mentioned .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So is it a significant one if i have to tell in context of survival ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qIJX8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qIJX8.png&quot; alt=&quot;log rank&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="48" LastEditDate="2019-01-15T08:09:03.530" LastActivityDate="2019-01-15T09:56:02.787" Title="log rank p value interpretation from gepia 2" Tags="|rna-seq|figure-explanation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6806" PostTypeId="1" CreationDate="2019-01-14T20:13:33.553" Score="2" ViewCount="2364" Body="&lt;p&gt;I am new to single cell sequencing data analysis but I have basic programming background in R and python. I want to be able to plot differential expression of two genes, Gene1 and Gene2, across three cell types and across three conditions on one plot. I have followed this &lt;a href=&quot;https://satijalab.org/seurat/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; to perform QC and various differential analyses using Seurat package on my data up till now. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My data set is one object, containing information about the three cell types under three various conditions. The metadata slot of my data set contains information about my cell types as well as the conditions under which they are tested. Using the following DotPlot commands I am able to generate separate plots of gene expression with respect to cell type and with respect to condition:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Seurat::DotPlot(object = object, genes.plot = c(&quot;Gene1&quot;, &quot;Gene2&quot;), plot.legend = TRUE, group.by = &quot;cell_types&quot;)&#xA;&#xA;Seurat::DotPlot(object = object, &#xA;        genes.plot = c(&quot;Gene1&quot;, &quot;Gene2&quot;), plot.legend = TRUE, group.by = &quot;tissue_types&quot; )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For elegant representation of my data, I want to be able to have the information of the following plots into one plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will be very grateful for any hint on how I can proceed to achieve an elegant presentation as well as information about other possible ways to graphically represent gene expression data for publication or presentation purposes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for your help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lbsMg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lbsMg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3968" LastEditorUserId="3968" LastEditDate="2019-01-14T22:15:40.173" LastActivityDate="2019-01-14T22:15:40.173" Title="Visualising gene expression across cell type and conditions in one plot, in Single Cell Sequencing data" Tags="|scrnaseq|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6807" PostTypeId="1" CreationDate="2019-01-14T21:39:48.543" Score="3" ViewCount="202" Body="&lt;p&gt;I have two matrices like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(DEG_log[,1:4])&#xA;            cl1       cl1       cl1       cl1&#xA;ALB    4.653796  7.046317  7.732135  7.758708&#xA;AQP9   7.798208  8.231112  8.734799  8.054087&#xA;CALML5 5.791524  9.058341  6.977369  9.503477&#xA;CCL4   9.739961  7.663303  8.934939  7.324958&#xA;CDK6   9.832115  9.710043 11.439345 10.330253&#xA;CHGA   5.139340 12.024606  5.988773  8.696836&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have the same patients and genes in two separate matrices coming from two sequencing panels with the same chemistry. Each sample has a match in another panel. I want to remove genes that have low correlation between two matched samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have normalised and log-transformed the data by DESeq2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just plotted the correlation between each pairs of samples: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(&quot;ggpubr&quot;)&#xA;ggscatter(my_data, x = &quot;sample1&quot;, y = &quot;sample1_1&quot;, &#xA;          add = &quot;reg.line&quot;, conf.int = TRUE, &#xA;          cor.coef = TRUE, cor.method = &quot;pearson&quot;,&#xA;          xlab = &quot;onco&quot;, ylab = &quot;biomarker)&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/64yCv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/64yCv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;by:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;corr_test(a, b, 0.95) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can calculate correlation, but I don't know how to remove genes with low or non-significant correlation between matched samples.&lt;/p&gt;&#xA;" OwnerDisplayName="user3962" LastEditorUserId="3848" LastEditDate="2023-05-13T02:12:36.140" LastActivityDate="2023-05-13T02:12:36.140" Title="Removing genes with less than a correlation cut-off between two matrices" Tags="|r|rna-seq|phylogenetics|phylogeny|deseq2|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6808" PostTypeId="2" ParentId="6806" CreationDate="2019-01-14T22:10:05.453" Score="2" Body="&lt;p&gt;Try using &lt;code&gt;SplitDotPlotGG&lt;/code&gt;, this will allow you to color the dots by some additional variable (eg condition).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case you would want to do something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Seurat)&#xA;&#xA;SplitDotPlotGG(object = object,&#xA;               genes.plot = c(&quot;Gene1&quot;, &quot;Gene2&quot;),&#xA;               plot.legend = TRUE,&#xA;               group.by = &quot;cell_types&quot;,&#xA;               grouping.var = &quot;tissue_types&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This would give a result similar to below (an example from &lt;a href=&quot;https://satijalab.org/seurat/immune_alignment.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/immune_alignment.html&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ynq5g.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ynq5g.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3415" LastActivityDate="2019-01-14T22:10:05.453" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6809" PostTypeId="2" ParentId="6804" CreationDate="2019-01-14T23:37:39.667" Score="5" Body="&lt;p&gt;Snakemake gets &lt;code&gt;wildcards&lt;/code&gt; from the parsing the input/output file names.&#xA;For example, if you had&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule sort:&#xA;    input: &quot;{file}.bed&quot;&#xA;    output: &quot;{file}.sorted.bed&quot;&#xA;    shell: &quot;sort -k1,1 -k2,2n {input} &amp;gt; {output}&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then the &lt;code&gt;wildcards&lt;/code&gt; variable would be set as &lt;code&gt;wildcards = {&quot;file&quot;: ...}&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your &lt;code&gt;multiqc&lt;/code&gt; rule, you have the &lt;code&gt;{sample}&lt;/code&gt; variable within your &lt;code&gt;expand()&lt;/code&gt;, but this variable doesn't exist outside of &lt;code&gt;expand&lt;/code&gt;'s local context. You don't actually have any wildcards there, nor in your output file (&lt;code&gt;qc_dir&lt;/code&gt; and &lt;code&gt;rseqc_dir&lt;/code&gt; are variables you define elsewhere, but not &quot;wildcards&quot; as Snakemake interprets them).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you're getting the error &lt;code&gt;'Wildcards' object has no attribute 'sample'&lt;/code&gt; because there are no wildcards for that rule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your rule is currently set up to process every sample that you have listed in  &lt;code&gt;samples['sample']&lt;/code&gt;. If you want your &lt;code&gt;multiqc&lt;/code&gt; rule to work for a single sample, simply remove the &lt;code&gt;sample=samples['sample']&lt;/code&gt; and double brace the &lt;code&gt;{sample}&lt;/code&gt; wildcard.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule multiqc:&#xA;    input:&#xA;        expand([os.path.join(analyzed_dir, '{{sample}}.stat', '{{sample}}.cnt'),&#xA;                os.path.join(rseqc_dir, '{{sample}}.bam_stat.txt'),&#xA;                os.path.join(rseqc_dir, '{{sample}}.clipping_profile.xls'),&#xA;                os.path.join(rseqc_dir, '{{sample}}.deletion_profile.txt'),&#xA;                os.path.join(rseqc_dir, '{{sample}}.infer_experiment.txt'),&#xA;                os.path.join(rseqc_dir, '{{sample}}.geneBodyCoverage.txt'),&#xA;                os.path.join(rseqc_dir, '{{sample}}.inner_distance.txt'),&#xA;                os.path.join(rseqc_dir, '{{sample}}.insertion_profile.xls'),&#xA;                os.path.join(rseqc_dir, '{{sample}}.junction.xls'),&#xA;                os.path.join(rseqc_dir, '{{sample}}.junctionSaturation_plot.r'),&#xA;                os.path.join(rseqc_dir, '{{sample}}.mismatch_profile.xls'),&#xA;                os.path.join(rseqc_dir, '{{sample}}.read_distribution.txt'),&#xA;                os.path.join(rseqc_dir, '{{sample}}.pos.DupRate.xls'),&#xA;                os.path.join(rseqc_dir, '{{sample}}.seq.DupRate.xls'),&#xA;                os.path.join(rseqc_dir, '{{sample}}.GC.xls'),&#xA;                os.path.join(rseqc_dir, '{{sample}}.NVC.xls'),&#xA;                os.path.join(rseqc_dir, '{{sample}}.qual.r'),&#xA;                os.path.join(rseqc_dir, '{{sample}}.RNA_fragment_size.txt'),&#xA;                os.path.join(rseqc_dir, '{{sample}}.STAR.genome.sorted.summary.txt'),&#xA;                os.path.join(fastqc_dir, '{{sample}}_R{read}_001_fastqc.html')],&#xA;                read=reads)&#xA;    output:&#xA;        # this {sample} is in single {} and is recognized as a wildcard&#xA;        os.path.join(qc_dir, '{sample}', 'multiqc_report.html') &#xA;    conda:&#xA;        &quot;envs/multiqc.yaml&quot;&#xA;    shell:&#xA;        '''&#xA;        #!/bin/bash&#xA;        # using {wildcards.sample} here since MultiQC likes directories&#xA;        multiqc . --outdir qc/{wildcards.sample} --ignore .snakemake --force&#xA;        '''&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When wildcards are recognized, you'll see them in the job list.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Running &lt;code&gt;snakemake 0067_P2018SEQE27R01_S1/multiqc_report.html&lt;/code&gt;, for example, should give you something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule multiqc:&#xA;    input: qc/rseqc/0067_P2018SEQE27R01_S1.geneBodyCoverage.txt, ...&#xA;    output: qc/0067_P2018SEQE27R01_S1/multiqc_report.html&#xA;    jobid: 2&#xA;    wildcards: sample=0067_P2018SEQE27R01_S1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="240" LastEditorUserId="240" LastEditDate="2019-01-15T14:46:22.620" LastActivityDate="2019-01-15T14:46:22.620" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6810" PostTypeId="1" AcceptedAnswerId="6811" CreationDate="2019-01-15T04:35:14.150" Score="6" ViewCount="7483" Body="&lt;p&gt;What is the difference between a Single Nucleotide Polymorphism (SNP) and a point mutation? I am quite confused in understanding these term as both of them refer to one base difference from the reference sequence?&lt;/p&gt;&#xA;" OwnerUserId="3760" LastEditorUserId="96" LastEditDate="2019-01-15T14:14:36.547" LastActivityDate="2019-01-15T14:21:15.530" Title="SNP vs Point Mutation" Tags="|snp|terminology|variation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6811" PostTypeId="2" ParentId="6810" CreationDate="2019-01-15T04:48:06.523" Score="6" Body="&lt;p&gt;The difference between the two depends on to whom you talk ;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are right: both refer to one base difference from the sequence. Usually the term &quot;mutation&quot; is used if the change has an impact on the phenotype. The &quot;P&quot; in &quot;SNP&quot; means &quot;polymorphism&quot;. A lot of people use this term if more than 1% of the individual in a given population have this variant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My recommendation: only use &quot;mutation&quot; if you are sure that your variant has an important impact on the phenotype. Only use &quot;SNP&quot; if you are sure that it is a commonly occurring variant. In any other case just use &quot;variant&quot;. It is slowly becoming more common to talk about &quot;SNV&quot; (Single Nucleotide Variant) than about &quot;SNP&quot;.&lt;/p&gt;&#xA;" OwnerUserId="1310" LastEditorUserId="96" LastEditDate="2019-01-15T14:21:15.530" LastActivityDate="2019-01-15T14:21:15.530" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6812" PostTypeId="2" ParentId="6805" CreationDate="2019-01-15T09:56:02.787" Score="1" Body="&lt;p&gt;The plot on the left shows that the gene has very different expression between two groups. However, according to the graph on the right, these high expression is not directly related to survival because both groups have similar survival time.&lt;br&gt;&#xA;The gene expression so high might be to compensate for other deficiencies or problems. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2019-01-15T09:56:02.787" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="6813" PostTypeId="1" CreationDate="2019-01-15T11:22:50.500" Score="3" ViewCount="131" Body="&lt;p&gt;I want to find some good predictors (genes). I have checked the Spearman correlation of the expression of each of 23 genes with dependent variables (responders Vs non-responders and I saw only the 5 genes identified initially showed significant correlation with the dependent variable.&lt;/p&gt;&#xA;&lt;p&gt;This is my data, log transformed RNA-seq:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;              TRG    CDK6 EGFR  KIF2C CDC20&#xA;    Sample 1  TRG12  11.39 10.62  9.75 10.34&#xA;    Sample 2  TRG12  10.16  8.63  8.68  9.08&#xA;    Sample 3  TRG12   9.29 10.24  9.89 10.11&#xA;    Sample 4  TRG45  11.53  9.22  9.35  9.13&#xA;    Sample 5  TRG45   8.35 10.62 10.25 10.01&#xA;    Sample 6  TRG45  11.71 10.43  8.87  9.44&#xA;    ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;23 differentially expressed genes between responder patients (TRG12) and non-responder patients (TRG45).&lt;/p&gt;&#xA;&lt;p&gt;1- I tested each of 23 genes individually in this code and each of them gives p-value &amp;lt; 0.05 remained as a good predictor; For example for CDK6 I have done&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; glm=glm(TRG ~ CDK6, data = df, family = binomial(link = 'logit'))&#xA;&amp;gt; &#xA;&amp;gt; summary(glm)&#xA;&#xA;Call:&#xA;glm(formula = TRG ~ CDK6, family = binomial(link = &amp;quot;logit&amp;quot;), &#xA;    data = df)&#xA;&#xA;Deviance Residuals: &#xA;    Min       1Q   Median       3Q      Max  &#xA;-1.8286  -0.9317   0.6382   0.9500   1.5966  &#xA;&#xA;Coefficients:&#xA;            Estimate Std. Error z value Pr(&amp;gt;|z|)   &#xA;(Intercept)  26.8950     9.6627   2.783  0.00538 **&#xA;CDK6         -2.6932     0.9765  -2.758  0.00582 **&#xA;---&#xA;Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1&#xA;&#xA;(Dispersion parameter for binomial family taken to be 1)&#xA;&#xA;    Null deviance: 77.347  on 55  degrees of freedom&#xA;Residual deviance: 64.062  on 54  degrees of freedom&#xA;AIC: 68.062&#xA;&#xA;Number of Fisher Scoring iterations: 5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Finally I obtained five genes and I put them in this model:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;final &amp;lt;- glm(TRG ~ CDK6 + CXCL8 + IL6 + ISG15 + PTGS2 , data = df, family = binomial(link = 'logit'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; summary(final)&#xA;&#xA;Call:&#xA;glm(formula = TRG ~ CDK6 + CXCL8 + IL6 + ISG15 + PTGS2, family = binomial(link = &amp;quot;logit&amp;quot;), &#xA;    data = df)&#xA;&#xA;Deviance Residuals: &#xA;    Min       1Q   Median       3Q      Max  &#xA;-1.5696  -0.9090   0.2233   0.7125   2.0131  &#xA;&#xA;Coefficients:&#xA;            Estimate Std. Error z value Pr(&amp;gt;|z|)  &#xA;(Intercept)   3.2258    13.3877   0.241   0.8096  &#xA;CDK6         -2.6101     1.0860  -2.403   0.0162 *&#xA;CXCL8         0.8702     0.9783   0.889   0.3737  &#xA;IL6           0.9591     0.8883   1.080   0.2803  &#xA;ISG15         0.9574     0.5101   1.877   0.0605 .&#xA;PTGS2        -0.4623     1.1005  -0.420   0.6744  &#xA;---&#xA;Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1&#xA;&#xA;(Dispersion parameter for binomial family taken to be 1)&#xA;&#xA;    Null deviance: 77.347  on 55  degrees of freedom&#xA;Residual deviance: 53.917  on 50  degrees of freedom&#xA;AIC: 65.917&#xA;&#xA;Number of Fisher Scoring iterations: 6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am just seeing CDK6 as significant. I done cross validation&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; cv.glm(df, final, K=nrow(df))$delta&#xA;[1] 0.2130050 0.2125748&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Likely the delta values are not greatly differ (Model is right ???)&lt;/p&gt;&#xA;&lt;p&gt;Then I tried Sensitivity and Specificity&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; threshold=0.5&#xA;&amp;gt; predicted_values&amp;lt;-ifelse(predict(final,type=&amp;quot;response&amp;quot;)&amp;gt;threshold,1,0)&#xA;&amp;gt; actual_values&amp;lt;-final$y&#xA;&amp;gt; conf_matrix&amp;lt;-table(predicted_values,actual_values)&#xA;&amp;gt; conf_matrix&#xA;                actual_values&#xA;predicted_values  0  1&#xA;               0 16  7&#xA;               1 10 23&#xA;&amp;gt; sensitivity(conf_matrix)&#xA;[1] 0.6153846&#xA;&amp;gt; specificity(conf_matrix)&#xA;[1] 0.7666667&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;2- I went this way for step-wise method&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;full.model &amp;lt;- glm(TRG ~., data = df, family = binomial)&#xA;&#xA;library(MASS)&#xA;step.model &amp;lt;- full.model %&amp;gt;% stepAIC(trace = FALSE)&#xA;coef(step.model)&#xA;&#xA;&amp;gt; coef(step.model)&#xA;(Intercept)        CDK6        CHGA        CSF3       CXCL8       ERBB2         IL6       ISG15       KRT14       KRT17 &#xA;-23770.6261   -469.3660    376.3528   -238.5697   1404.5905    590.2586    789.5295    842.2058    423.0402   -651.3403 &#xA;     MAGEA1         OSM       PTGS2 &#xA;   496.6933   -637.0583   -500.1040 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In addition to 5 genes another genes remains as predictors but likely none of them are significant by Pr(&amp;gt;|z|) &amp;gt; 0.05&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; summary(step.model)&#xA;&#xA;Call:&#xA;glm(formula = TRG ~ CDK6 + CHGA + CSF3 + CXCL8 + ERBB2 + IL6 + &#xA;    ISG15 + KRT14 + KRT17 + MAGEA1 + OSM + PTGS2, family = binomial, &#xA;    data = df)&#xA;&#xA;Deviance Residuals: &#xA;       Min          1Q      Median          3Q         Max  &#xA;-1.317e-04  -2.100e-08   2.100e-08   2.100e-08   1.396e-04  &#xA;&#xA;Coefficients:&#xA;             Estimate Std. Error z value Pr(&amp;gt;|z|)&#xA;(Intercept)  -23770.6  2922188.9  -0.008    0.994&#xA;CDK6           -469.4    64920.6  -0.007    0.994&#xA;CHGA            376.4    46191.2   0.008    0.993&#xA;CSF3           -238.6    30857.1  -0.008    0.994&#xA;CXCL8          1404.6   172558.7   0.008    0.994&#xA;ERBB2           590.3    71989.5   0.008    0.993&#xA;IL6             789.5   100130.7   0.008    0.994&#xA;ISG15           842.2   102934.0   0.008    0.993&#xA;KRT14           423.0    52565.5   0.008    0.994&#xA;KRT17          -651.3    79489.6  -0.008    0.993&#xA;MAGEA1          496.7    60319.6   0.008    0.993&#xA;OSM            -637.1    82121.8  -0.008    0.994&#xA;PTGS2          -500.1    73541.7  -0.007    0.995&#xA;&#xA;(Dispersion parameter for binomial family taken to be 1)&#xA;&#xA;    Null deviance: 7.7347e+01  on 55  degrees of freedom&#xA;Residual deviance: 1.4741e-07  on 43  degrees of freedom&#xA;AIC: 26&#xA;&#xA;Number of Fisher Scoring iterations: 25&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;3- I went to lasso regression&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; library(glmnet)&#xA;&amp;gt; lassoModel &amp;lt;- glmnet(&#xA;+   x=data.matrix(df[,-1]),&#xA;+   y=df&lt;span class=&quot;math-container&quot;&gt;$TRG,&#xA;+   standardize=TRUE,&#xA;+   alpha=1.0,&#xA;+   family=&quot;multinomial&quot;)&#xA;&amp;gt; #Derive coefficients for each gene to each subtype&#xA;&amp;gt; co &amp;lt;- coef(lassoModel, s=idealLambda, exact=TRUE)&#xA;&amp;gt; co&#xA;$&lt;/span&gt;`TRG12`&#xA;24 x 1 sparse Matrix of class &amp;quot;dgCMatrix&amp;quot;&#xA;                       1&#xA;(Intercept)  4.441322326&#xA;ALB         -0.010867331&#xA;AQP9        -0.051908908&#xA;CALML5       0.126359364&#xA;CCL4        -0.059869284&#xA;CDK6         0.283781437&#xA;CHGA        -0.088283426&#xA;CREB3L3      0.109988758&#xA;CSF3        -0.012311176&#xA;CXCL5       -0.030532485&#xA;CXCL6       -0.063610112&#xA;CXCL8       -0.084385732&#xA;ERBB2       -0.102303095&#xA;FGFRL1       0.148759702&#xA;IL1B        -0.072225606&#xA;IL6         -0.122671026&#xA;ISG15       -0.297386587&#xA;KLK5         0.039891341&#xA;KRT14       -0.004536929&#xA;KRT17        0.041693665&#xA;MAGEA1      -0.154075458&#xA;OSM          0.028987648&#xA;PTGS2       -0.034960405&#xA;S100A7A     -0.008190105&#xA;&#xA;$TRG45&#xA;24 x 1 sparse Matrix of class &amp;quot;dgCMatrix&amp;quot;&#xA;                       1&#xA;(Intercept) -4.441322326&#xA;ALB          0.010867331&#xA;AQP9         0.051908908&#xA;CALML5      -0.126359364&#xA;CCL4         0.059869284&#xA;CDK6        -0.283781437&#xA;CHGA         0.088283426&#xA;CREB3L3     -0.109988758&#xA;CSF3         0.012311176&#xA;CXCL5        0.030532485&#xA;CXCL6        0.063610112&#xA;CXCL8        0.084385732&#xA;ERBB2        0.102303095&#xA;FGFRL1      -0.148759702&#xA;IL1B         0.072225606&#xA;IL6          0.122671026&#xA;ISG15        0.297386587&#xA;KLK5        -0.039891341&#xA;KRT14        0.004536929&#xA;KRT17       -0.041693665&#xA;MAGEA1       0.154075458&#xA;OSM         -0.028987648&#xA;PTGS2        0.034960405&#xA;S100A7A      0.008190105&#xA;&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Likely none of genes is significant so I just tried 5 genes from first glm&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; finalLasso &amp;lt;- glm(df$TRG ~ CDK6 + IL6 + ISG15 + CXCL8 + CALML5 , data=df, family=binomial(link=&amp;quot;logit&amp;quot;))&#xA;&amp;gt; summary(finalLasso)&#xA;&#xA;Call:&#xA;glm(formula = df$TRG ~ CDK6 + IL6 + ISG15 + CXCL8 + CALML5, family = binomial(link = &amp;quot;logit&amp;quot;), &#xA;    data = df)&#xA;&#xA;Deviance Residuals: &#xA;    Min       1Q   Median       3Q      Max  &#xA;-1.6780  -0.6733   0.1643   0.6925   1.9182  &#xA;&#xA;Coefficients:&#xA;            Estimate Std. Error z value Pr(&amp;gt;|z|)  &#xA;(Intercept)  -0.4178    14.7456  -0.028   0.9774  &#xA;CDK6         -2.2672     1.1537  -1.965   0.0494 *&#xA;IL6           1.2524     0.9197   1.362   0.1733  &#xA;ISG15         1.1763     0.5479   2.147   0.0318 *&#xA;CXCL8         0.5413     0.6443   0.840   0.4008  &#xA;CALML5       -0.7890     0.4219  -1.870   0.0615 .&#xA;---&#xA;Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1&#xA;&#xA;(Dispersion parameter for binomial family taken to be 1)&#xA;&#xA;    Null deviance: 77.347  on 55  degrees of freedom&#xA;Residual deviance: 49.839  on 50  degrees of freedom&#xA;AIC: 61.839&#xA;&#xA;Number of Fisher Scoring iterations: 6&#xA;&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That 2 genes are significant; really confusing :(&lt;/p&gt;&#xA;&lt;p&gt;Please somebody save me from this horrible confusion, finally which genes are significant remaining in model and which not&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;First confusion: Why when I am putting each of 23 genes individually&#xA;in glm model, 5 of them returns p-value &amp;lt; 0.05 but when summary of&#xA;final model containing these 5 genes says only p-value of one of them&#xA;is &amp;lt; 0.05&lt;/p&gt;&#xA;&lt;p&gt;Second confusion: When I am using stepwise regression, summary of&#xA;model says none of my genes is significant while some of them had been&#xA;remained as predictors by &lt;code&gt;coef(step.model)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Third confusion: CDK6 as my significant genes in two of model shows&#xA;negative coefficient while I know that show be correlated with&#xA;dependent variable positively&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="73" LastEditDate="2021-06-25T11:26:47.997" LastActivityDate="2024-03-19T16:49:07.870" Title="How do I resolve disagreements in the determination of significant genes?" Tags="|r|rna-seq|linear-regression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6814" PostTypeId="5" CreationDate="2019-01-15T14:40:57.640" Score="0" Body="&lt;p&gt;Structural variants are genetic variants that involve large rearrangements of genomic sequence. There is no universal consensus as to what qualifies as &quot;large&quot;, though 1kb and 10kb are commonly used as lower thresholds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term &quot;structural variant&quot; is used to distinguish large-scale variation from smaller variants such as single-nucleotide substitutions and short indels which are less disruptive to the genome structure.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2019-01-15T21:51:08.643" LastActivityDate="2019-01-15T21:51:08.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6815" PostTypeId="4" CreationDate="2019-01-15T14:40:57.640" Score="0" Body="Refers to large-scale genomic variation in a population or between individuals, including inversions, translocations, and copy number variants" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2019-01-16T00:17:53.470" LastActivityDate="2019-01-16T00:17:53.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6816" PostTypeId="2" ParentId="422" CreationDate="2019-01-15T15:15:20.917" Score="0" Body="&lt;p&gt;The terms &quot;second-generation sequencing&quot;, &quot;third-generation sequencing&quot;, &quot;short-read sequencing&quot;, and &quot;long-read sequencing&quot; are used commonly but (as far as I know) lack commonly used acronyms/initialisms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the forensic genetics community, the acronym &lt;strong&gt;MPS&lt;/strong&gt; is used commonly to refer to &quot;massively parallel sequencing.&quot; This definitely includes 2G/short-read sequencing (such as Illumina). It's less clear whether MPS includes 3G/long-read sequencing such as PacBio or Oxford Nanopore, but since these are rarely used in a forensics context it's a moot point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, it's important to acknowledge (as @burger has done) that there is a lot of imprecision and opinion regarding the appropriate scope of each term. Some folks seem to want a term to refer to everything post-Sanger, which includes an incredible diversity of platforms with respect to sequencing technology, read length, throughput, and error profiles. Some very reasonable questions should make us think seriously about our chosen terminology: How long do (for example) Illumina reads have to get before they're no longer classified as short reads? How much throughput is required of a sequencer to be classified as &quot;high&quot; throughput? When do these distinctions &lt;em&gt;really&lt;/em&gt; matter?&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-01-15T15:15:20.917" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6817" PostTypeId="1" AcceptedAnswerId="6821" CreationDate="2019-01-15T17:56:29.110" Score="2" ViewCount="177" Body="&lt;h2&gt;Background&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In C++ I want to implement a simple class that performs &lt;a href=&quot;http://www.cs.jhu.edu/~langmea/resources/lecture_notes/assembly_olc.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;overlap layout consensus assembly&lt;/a&gt; but I can not figure out the most logical data structure to use for the consensus alignment step. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ocw.mit.edu/courses/biology/7-91j-foundations-of-computational-and-systems-biology-spring-2014/lecture-slides/MIT7_91JS14_Lecture6.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This is the step where all of the reads from the overlap graph are aligned and the consensus is called (slide 48)&lt;/a&gt;. An example alignment is below. Asterisks above the column indicate some disagreement among all reads. Alignment position is shown above and below for convenience.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                         *    * *         *&#xA;     pos           111111111122222222223333333333444&#xA;         0123456789012345678901234567890123456789012&#xA;read 1   TAGATTACACAGATTACTGAT TGA&#xA;read 2      ATTACACAGATTACTGACTTGATG CGTAAACTA&#xA;read 3                   tTGACTTcATGGCGTAA CTA&#xA;read 4                     GACTTGATGGCGTAA CTAAG&#xA;read 5                           ATGGCGTAA CTAAGATCG&#xA;     pos           111111111122222222223333333333444&#xA;         0123456789012345678901234567890123456789012&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Question:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q1:&lt;/strong&gt; What is the most logical way to implement the consensus alignment step in a OLC assembler? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Q2:&lt;/strong&gt; What are the pros and cons of this and other possibilities?&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Ideas&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;Idea - Vector of vectors. One column per position.&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;My initial idea for a data structure to store the alignment is to use one vector per read to store the read's character at that position. We would end up with j arrays (number of reads) all of length i (the total length of the overlapped alignments). This, however, would be space-inefficient if the locus were large and many positions in the vectors would contain a null value. In the example alignment above the first 23 positions of read 5 would be null values.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Idea - Graph. One node per position.&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In another implementation idea, each position in the alignment would be represented by a node in a directed graph. Each node would have a property keeping track of the nucleotide counts at each position and might keep the read id's stored as well if there is a good reason to do so. &lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2019-01-16T02:05:21.090" Title="How is consensus alignment for OLC assembly usually implemented?" Tags="|assembly|algorithms|c++|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6818" PostTypeId="2" ParentId="285" CreationDate="2019-01-15T19:17:46.453" Score="2" Body="&lt;p&gt;If the goal is to extract the consensus sequence for given regions, this is how it works in 2019.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;bgzip&lt;/code&gt; and index your &lt;code&gt;vcf&lt;/code&gt; file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ bgzip -c input.vcf &amp;gt; input.vcf.gz&#xA;$&lt;/span&gt; tabix input.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Create a &lt;code&gt;regions.txt&lt;/code&gt; which contains one region per line in the format &lt;code&gt;chr:from-to&lt;/code&gt;. If you have already a &lt;code&gt;bed&lt;/code&gt; file, you can use this little &lt;code&gt;awk&lt;/code&gt;script to create it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk '{print $1&quot;:&quot;$2+1&quot;-&quot;$&lt;/span&gt;3}' input.bed &amp;gt; regions.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now run this combination of &lt;code&gt;samtools&lt;/code&gt; and &lt;code&gt;bcftools&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ samtools faidx -r regions.txt genome.fa|bcftools consensus input.vcf.gz -o consensus.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1310" LastActivityDate="2019-01-15T19:17:46.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6819" PostTypeId="2" ParentId="2223" CreationDate="2019-01-15T20:10:30.333" Score="2" Body="&lt;p&gt;Here's an update how this work in 2019. (The answer is quite similar to my update in this &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/6818/1310&quot;&gt;thread&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;bgzip&lt;/code&gt; and index your &lt;code&gt;vcf&lt;/code&gt; file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ bgzip -c input.vcf &amp;gt; input.vcf.gz&#xA;$&lt;/span&gt; tabix input.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now use &lt;a href=&quot;http://www.htslib.org/doc/bcftools.html#consensus&quot; rel=&quot;nofollow noreferrer&quot;&gt;bcftools consensus&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools consensus -f genome.fa input.vcf.gz &amp;gt; consensus.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1310" LastActivityDate="2019-01-15T20:10:30.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6820" PostTypeId="2" ParentId="6807" CreationDate="2019-01-16T01:58:55.303" Score="3" Body="&lt;p&gt;Nice data and good question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is called &quot;outlier&quot; analysis or the analysis of the residual. Its an easy analysis, but . You calculated the standard deviation using the regression slope as the 'mean' and retain data within 1.96 standard deviations. You can plot the data like this and if you are lucky will see the residual forming a nice 'normal distribution'. Outliers outside 1.96 sd can be formally excluded.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Calling (and plotting) the residual from the regression slope will be possible in R, but I use external packages because its easy. BTW you should also try Spearman's coefficient to check you are getting the same answer.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-01-16T01:58:55.303" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6821" PostTypeId="2" ParentId="6817" CreationDate="2019-01-16T02:05:21.090" Score="2" Body="&lt;p&gt;For short reads, mainstream assemblers don't have a consensus step. The contig path directly spells the consensus sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For long reads, the most popular way to derive a consensus is via partial-order alignment (POA). This is used by error correctors (similar to consensus) like pbdagcon and falcon_sense, and consensus tools like canu, racon, wtdbg2 and nanopolish. Basically, you start with an empty directed acyclic graph (DAG), align each read in turn to the graph and augment the DAG by inserting unaligned segments. When you align all reads onto the DAG, the most weighted path is taken as the consensus. You should read &lt;a href=&quot;http://simpsonlab.github.io/2015/05/01/understanding-poa/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jared Simpson's blog&lt;/a&gt; on this topic. It is much more detailed than &quot;basically&quot;. You should also read the papers on all the mentioned tools above if you want to implement by yourself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternative approach is Gene Myers' realigner (&lt;a href=&quot;http://myerslab.mpi-cbg.de/wp-content/uploads/2014/06/realigner.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDF here&lt;/a&gt;). It is not often used, though.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2019-01-16T02:05:21.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6822" PostTypeId="2" ParentId="481" CreationDate="2019-01-16T02:14:15.717" Score="2" Body="&lt;p&gt;Adding another answer that, while much more involved, offers flexibility, walks through minimal steps, and provides visualizations.&lt;br&gt;&#xA;I created a &lt;a href=&quot;http://tgviz.herokuapp.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dash app&lt;/a&gt; and &lt;a href=&quot;https://github.com/arvkevi/tgviz&quot; rel=&quot;nofollow noreferrer&quot;&gt;repository&lt;/a&gt; that explores this type of analysis using published ancestry informative SNPs and 1000 Genomes Project Data. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Identify published ancestry informative SNPs (AISNPs).&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed?db=pubmed&amp;amp;cmd=Retrieve&amp;amp;dopt=citation&amp;amp;list_uids=24508742&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kidd et al.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed?cmd=Retrieve&amp;amp;dopt=citation&amp;amp;list_uids=18683858&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kosoy et al.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Download 1000 Genomes Project genotype &lt;a href=&quot;http://tgviz.herokuapp.com/ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/bcf_files/&quot; rel=&quot;nofollow noreferrer&quot;&gt;bcf data&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Limit genotype data from (2) to AISNPs loci from (1).&lt;/li&gt;&#xA;&lt;li&gt;One-hot encode the genotypes.&lt;/li&gt;&#xA;&lt;li&gt;Perform dimensionality reduction (PCA, t-SNE, or UMAP)&lt;/li&gt;&#xA;&lt;li&gt;Plot components. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The original post seemed to want to project their genome against a reference sample. The &lt;a href=&quot;https://github.com/arvkevi/tgviz/blob/master/notebooks/plot_walkthrough.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;plot_walkthrough.ipynb&lt;/a&gt; in the &lt;code&gt;tgviz&lt;/code&gt; repo shows how this can be achieved.&lt;br&gt;&#xA;Output from the notebook on my own data:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wo2JH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wo2JH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="84" LastActivityDate="2019-01-16T02:14:15.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6824" PostTypeId="1" CreationDate="2019-01-16T13:56:54.143" Score="5" ViewCount="86" Body="&lt;p&gt;I am generating &lt;a href=&quot;https://sites.google.com/a/poissonboltzmann.org/software/file-formats/biomolecular-structurw/pqr&quot; rel=&quot;nofollow noreferrer&quot;&gt;PQR&lt;/a&gt; files for a membrane protein that is almost entirely buried in the membrane. The goal is to calculate the electrostatic charge across the surface of the protein. There are no lipids in the structure.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://nbcr-222.ucsd.edu/pdb2pqr_2.1.1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB2PQR&lt;/a&gt; has predefined options of &lt;code&gt;amber&lt;/code&gt;, &lt;code&gt;charmm&lt;/code&gt;, &lt;code&gt;parse&lt;/code&gt;, &lt;code&gt;tyl06&lt;/code&gt;, &lt;code&gt;peoepb&lt;/code&gt; and &lt;code&gt;swanson&lt;/code&gt;. It also allows user supplied forcefields.&lt;/p&gt;&#xA;&lt;p&gt;The protein is an ion channel in the plasma membrane of a human. I'll be processing in memAPBS. However, I would greatly appreciate a &amp;quot;rule of thumb&amp;quot; answer as I plan to use the same method on TMPs from other organisms in various subcellular membranes.&lt;/p&gt;&#xA;&lt;p&gt;What is a sensible forcefield choice for membrane proteins when generating PQR files for electrostatic analysis?&lt;/p&gt;&#xA;" OwnerUserId="747" LastEditorUserId="3848" LastEditDate="2020-07-13T12:09:56.830" LastActivityDate="2020-07-13T12:09:56.830" Title="What is a sensible forcefield choice for membrane proteins when using PDB2PQR?" Tags="|protein-structure|pdb|structural-biology|membrane-protein|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6825" PostTypeId="1" AcceptedAnswerId="6827" CreationDate="2019-01-16T14:26:37.880" Score="2" ViewCount="1731" Body="&lt;p&gt;I have the raw counts for RNA-Seq data. I converted counts data to logCPM using edgeR package.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets say I have a dataframe &lt;code&gt;A&lt;/code&gt; with 15000 genes as rows and 100 samples as columns with counts data. I have used following code from edgeR tutorial.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;logCPM &amp;lt;- cpm(A, prior.count=2, log=TRUE)&#xA;head(logCPM)[1:5,1:5]&#xA;               Sample1          Sample2          Sample3         Sample4&#xA;A1BG          -1.019429        0.8333739        -1.192960       -0.6717287&#xA;A2M            7.622280        7.8057767         7.932570        8.6409077&#xA;A2ML1          1.004541        1.0909119         5.462613        6.4149348&#xA;A4GALT         2.126502        1.8403954         5.322601        2.9469345&#xA;AAAS           5.785888        5.1639501         5.259789        5.3248026&#xA;               Sample5&#xA;A1BG          0.3121975&#xA;A2M           9.1949911&#xA;A2ML1         9.4740594&#xA;A4GALT        2.9214979&#xA;AAAS          4.3708063&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After converting counts to &lt;code&gt;logCPM&lt;/code&gt; I see there is another command for scaling which is used for heatmaps.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;logCPM &amp;lt;- t(scale(t(logCPM)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is given there that &lt;code&gt;scaling each row (each gene) to have mean zero and standard deviation one&lt;/code&gt; in page 15 &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4934518/pdf/f1000research-5-9996.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;edgeR&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this nothing but z-score? or Is there any otherway to calculate z-score from logCPM data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanq&lt;/p&gt;&#xA;" OwnerUserId="1377" LastActivityDate="2019-01-16T17:48:33.573" Title="Calculating Z-score from logCPM values using edgeR" Tags="|r|rna-seq|normalization|edger|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6826" PostTypeId="1" AcceptedAnswerId="6828" CreationDate="2019-01-16T17:24:23.863" Score="6" ViewCount="6398" Body="&lt;p&gt;Due to tabix constraints, I need to sort a vcf so that contigs and then positions within contigs occur in numerical order in the vcf.  I don't know if the following will sort positions within contigs, but I tried this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat in.vcf | vcf-sort -c &amp;gt; out.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but was informed:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Old version of sort command installed, please run without the -c option.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've read that can cause problems &lt;a href=&quot;http://bo-peng.blogspot.com/2011/05/upgrading-sorting-and-merging-next-gen.html&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. Further, my contigs are't numeric, but are something like this: 'Erumb1_s00000030'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see two possible solutions.  (1) I am on Mac OS Mojave, and if there were a way to install the a new unix-style sort, I could use it. I tried homebrew and didn't find anything. (2) there are likely some bash commands that would do it.  If anyone has an idea in either of these directions, I would appreciate it. &lt;/p&gt;&#xA;" OwnerUserId="2196" LastActivityDate="2019-05-08T08:40:48.460" Title="Sort vcf by contig and position within contig" Tags="|vcf|bash|vcftools|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6827" PostTypeId="2" ParentId="6825" CreationDate="2019-01-16T17:31:02.737" Score="1" Body="&lt;p&gt;Yes this is nothing but a z-score. The scale function calculates column z-scores, but since you use t (transpose), it is actually performed on rows. There are many other ways to calculate z-scores, but if done correctly they will all give the same results.&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2019-01-16T17:48:33.573" LastActivityDate="2019-01-16T17:48:33.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6828" PostTypeId="2" ParentId="6826" CreationDate="2019-01-16T17:45:39.340" Score="5" Body="&lt;p&gt;You could simply use &lt;a href=&quot;http://www.htslib.org/doc/bcftools.html#sort&quot; rel=&quot;noreferrer&quot;&gt;bcftools sort&lt;/a&gt; for it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools sort input.vcf &amp;gt; output.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you really want to use bash only, you can do this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ grep &quot;^#&quot; input.vcf &amp;gt; output.vcf&#xA;$&lt;/span&gt; grep -v &quot;^#&quot; input.vcf| sort -k1,1V -k2,2g &amp;gt;&amp;gt; output.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first command will write the header information to the new vcf file.&#xA;The second will sort by contig name and position and append the result to the new vcf file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;fin swimmer&lt;/p&gt;&#xA;" OwnerUserId="1310" LastActivityDate="2019-01-16T17:45:39.340" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6829" PostTypeId="1" AcceptedAnswerId="6832" CreationDate="2019-01-16T21:42:49.520" Score="1" ViewCount="108" Body="&lt;p&gt;There was a nice question on &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/807/&quot;&gt;random access on a fastq file&lt;/a&gt;, however I have the opposite problem. I want to programatically access non-random fastq entries quickly (using C++).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say I have several pairs of &lt;code&gt;fastq.gz&lt;/code&gt; files that could be hundreds of millions or &gt; 1 billion read pairs total. I know the number of read pairs in each file and I have a list of 10-20 reads that I want to access from each file. I only know the 0-based index of the fastq entry that I want to access, not the read ID. So if I want the  234th and 863rd reads the naive solution is to open the fastq file and scan through 864 records. I need to perform some tasks with these reads, then I move on and access/process another batch of reads.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I need to perform the above task tens of thousands of times, so the options are:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Low RAM usage. Many file IO ops. No disk usage:&lt;/strong&gt; Opening/scanning/closing the &lt;code&gt;fastq.gz&lt;/code&gt; files then processing the desired reads &lt;strong&gt;tens of thousands of times.&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;strong&gt;Extreme RAM usage. One round of file reading. No disk usage:&lt;/strong&gt; Read in each &lt;code&gt;fastq.gz&lt;/code&gt; file once, store all of the desired reads in RAM and access and process those reads as needed&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;strong&gt;Low RAM usage. One round of file reading. Some disk usage:&lt;/strong&gt; Coming up with an intermediate file format similar to a &lt;code&gt;bam&lt;/code&gt; that will allow me to parse and sort the reads with their groups and other metadata later on.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If it makes a difference the fastq.gz files are on spinning disks, not SSDs.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2019-01-17T05:01:03.353" Title="Non-random access on a fastq file" Tags="|fastq|algorithms|c++|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6831" PostTypeId="1" CreationDate="2019-01-17T03:36:13.387" Score="2" ViewCount="131" Body="&lt;p&gt;I am learning about vcf file formatting and have a multi-sample (&amp;gt;300) vcf file and aim to &lt;strong&gt;group&lt;/strong&gt; (take) &lt;strong&gt;samples with shared mutations&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Can someone suggest a tool/command/script solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="1513" LastEditorUserId="3848" LastEditDate="2020-07-16T16:48:26.330" LastActivityDate="2020-07-16T16:48:26.330" Title="group samples based on shared mutations in a single multi samples vcf file" Tags="|r|shell|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6832" PostTypeId="2" ParentId="6829" CreationDate="2019-01-17T05:01:03.353" Score="2" Body="&lt;p&gt;You don't have a RAM problem because you only have about:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;10-20 reads that I want to access from each file.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Please go for (2). If you are concerned about RAM usage, you may selectively write some of your data back to disk.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Tens of thousands of 10-20 reads is not a problem for modern computing. &lt;/p&gt;&#xA;" OwnerUserId="174" LastActivityDate="2019-01-17T05:01:03.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6833" PostTypeId="1" CreationDate="2019-01-17T09:41:02.977" Score="4" ViewCount="181" Body="&lt;p&gt;I want to predict origins of replication (ORI) on hundreds of prokaryotic genomes. The most straight-forward solution would be to use most commonly used tool, &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-9-79&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ori-Finder&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It uses &lt;a href=&quot;https://academic.oup.com/bib/advance-article/doi/10.1093/bib/bbx174/4796864&quot; rel=&quot;nofollow noreferrer&quot;&gt;integrated gene prediction, analysis of base composition asymmetry, distribution of DnaA boxes, occurrence of genes frequently close to oriC regions and phylogenetic relationships&lt;/a&gt; to predict ORIs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I came across a paper by &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0303264715000027&quot; rel=&quot;nofollow noreferrer&quot;&gt;Parikh et al&lt;/a&gt; which describes a single measure (correlated entropy measure / CEM) with which they achieved 100% ORI-identification on 500 genomes.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/cSn9n.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cSn9n.png&quot; alt=&quot;comparison of ORI detection algorithms&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I like the CEM approach more than a close-source Windows-only black-box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parikh tested their algorithm on NCBI genomes. How are ORI annotated on these reference genomes? I couldn't find such an annotation. Does the sequence simply begin at the ORI in leading strand direction by convention? Were the ORI of the reference genomes identified by experiment?&lt;/p&gt;&#xA;" OwnerUserId="3628" LastEditorUserId="3848" LastEditDate="2020-03-13T06:52:39.743" LastActivityDate="2020-03-13T07:02:54.177" Title="Prediction of prokaryotic origins of replication (ORI)" Tags="|genome|algorithms|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6836" PostTypeId="2" ParentId="4727" CreationDate="2019-01-17T11:48:16.793" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What supervised machine learning techniques will evaluate the raw&#xA;  image data itself - either for training or testing?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You can use convolutional neural networks (CNN). They will directly use the raw image data itself.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;After applying CNNs, you can also use recurrent neural layers (RNN), to achieve better results (e.g. gated recurrent layers).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Hope this helps. As a reference you have &lt;a href=&quot;http://ncbi.nlm.nih.gov/pubmed/?term=3D%20CNN%20fluorescence&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Gudla PR1,2, Nakayama K2,3, Pegoraro G1,2, Misteli T2. Article:&#xA;  SpotLearn: Convolutional Neural Network for Detection of Fluorescence&#xA;  In Situ Hybridization (FISH) Signals in High-Throughput Imaging&#xA;  Approaches. Send to Cold Spring Harb Symp Quant Biol. 2017;82:57-70.&#xA;  doi: 10.1101/sqb.2017.82.033761. Epub 2017 Nov 28.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3980" LastEditorUserId="48" LastEditDate="2019-01-20T15:33:03.583" LastActivityDate="2019-01-20T15:33:03.583" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6837" PostTypeId="1" CreationDate="2019-01-17T18:09:49.247" Score="1" ViewCount="315" Body="&lt;p&gt;This is the &lt;a href=&quot;https://stackoverflow.com/questions/42917349/r-circlize-circos-plot-how-to-plot-unconnected-areas-between-sectors-with-min/42996677&quot;&gt;post&lt;/a&gt; where they made circos using R I'm just following that the part answered by Zuguang Gu, not not sure what I'm doing wrong my &lt;a href=&quot;https://drive.google.com/open?id=1UNkUO6numTeoLiBZlcl4kc7ah-roELWN&quot; rel=&quot;nofollow noreferrer&quot;&gt;file&lt;/a&gt;, and the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(ComplexHeatmap)&#xA;library(circlize)&#xA; d11 &amp;lt;- read.csv(&quot;DC3_7_MUT.txt&quot;,stringsAsFactors = FALSE)&#xA;a &amp;lt;- as.data.frame(d11)&#xA;head(a)&#xA;&#xA;&#xA;ex &amp;lt;- melt(a, id.vars=c(&quot;Gene&quot;))&#xA;head(ex)&#xA;dim(ex)&#xA;&#xA;bb1 &amp;lt;- ex %&amp;gt;% na.omit() %&amp;gt;%&#xA;  group_by(Gene,value) %&amp;gt;% &#xA;  mutate(new = n()) %&amp;gt;% &#xA;  data.frame()&#xA;&#xA;#head(bbb)&#xA;#dim(bbb)&#xA;&#xA;bb1[bb1==&quot;&quot;]&amp;lt;-NA&#xA;&#xA;df &amp;lt;- na.omit(bb1)&#xA;&#xA;bbb &amp;lt;- df&#xA;&#xA;features1 = unique(bbb[[3]])&#xA;features_col1 = structure(rand_color(length(features1)), names = features1)&#xA;&#xA;patients_col1 = structure(1:187, names = unique(bbb[[2]])) # I don't understand this&#xA;&#xA;&#xA;&#xA;col1 = ifelse(bbb[[4]], features_col1[bbb[[3]]], &quot;#FFFFFF00&quot;)&#xA;col1 = gsub(&quot;FF$&quot;, &quot;80&quot;, col1) # half transparent&#xA;features_count1 = tapply(bbb[[4]], bbb[[3]], sum)&#xA;head(bbb)&#xA;# set color to white if it only exists in one patient/cell&#xA;col1[features_count1[bbb[[3]]] == 1] = &quot;#FFFFFF00&quot;&#xA;pdf(&quot;CIRCOS.pdf&quot;,20,25)&#xA;chordDiagram(bbb, col = col1, grid.col = c(features_col1, patients_col1))&#xA;&#xA;&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The error &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;chordDiagram(bbb, col = col1, grid.col = c(features_col1,&#xA;  patients_col1)) Error in abs(df$value1[i]) :    non-numeric argument&#xA;  to mathematical function&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm doing something wrong but I'm not able to find out, secondly I didn't get the idea about the usage of structure the one I commented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I using the right dimension for the structure argument?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A bit of progress with a bit of tweak in the code &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;d11 &amp;lt;- read.csv(&quot;DC3_7_MUT.txt&quot;,stringsAsFactors = FALSE)&#xA;&#xA;&#xA;blank2na &amp;lt;- function(x){ &#xA;  z &amp;lt;- gsub(&quot;\\s+&quot;, &quot;&quot;, x)  #make sure it's &quot;&quot; and not &quot; &quot; etc&#xA;  x[z==&quot;&quot;] &amp;lt;- NA &#xA;  return(x)&#xA;}&#xA;&#xA;a1 &amp;lt;- data.frame(sapply(d11,  blank2na))&#xA;&#xA;df &amp;lt;- a1&#xA;head(df)&#xA;&#xA;df2 &amp;lt;- df[, colSums(is.na(df)) != nrow(df)]&#xA;&#xA;head(df2)&#xA;dim(df2)&#xA;&#xA;&#xA;&#xA;a &amp;lt;- as.data.frame(df2)&#xA;head(a)&#xA;ex &amp;lt;- melt(a, id.vars=c(&quot;Gene&quot;))&#xA;head(ex)&#xA;dim(ex)&#xA;ex &amp;lt;- with(ex, table(paste(Gene,variable,sep='|'), value))&#xA;ex = as.data.frame(ex, stringsAsFactors = FALSE)&#xA;head(ex)&#xA;ex &amp;lt;- ex[which(ex&lt;span class=&quot;math-container&quot;&gt;$value != &quot;&quot;),]&#xA;head(ex)&#xA;dim(ex)&#xA;ex &amp;lt;- (ex[which(ex$&lt;/span&gt;Freq == 1),])&#xA;head(ex)&#xA;dim(ex)&#xA;&#xA;&#xA;features = unique(ex[[2]])&#xA;features_col = structure(rand_color(length(features)), names = features)&#xA;patients_col = structure(1:280, names = unique(ex[[2]]))&#xA;col = ifelse(ex[[3]], features_col[ex[[2]]], &quot;#FFFFFF00&quot;)&#xA;col = gsub(&quot;FF$&quot;, &quot;80&quot;, col) # half transparent&#xA;features_count = tapply(ex[[3]], ex[[2]], sum)&#xA;# set color to white if it only exists in one patient/cell&#xA;col[features_count[ex[[2]]] == 1] = &quot;red&quot; &#xA;&#xA;#And the final chord diagram:&#xA;&#xA;pdf(&quot;CIRCOS.pdf&quot;,20,25)&#xA;&#xA;#png(&quot;circos.png&quot;,width =20,height=20,unit=&quot;cm&quot;,res=1000)&#xA;#chordDiagram(ex, col = col, grid.col = c(features_col, patients_col))&#xA;&#xA;chordDiagram(ex, col = col, grid.col = patients_col, directional = 1,&#xA;             transparency = 0.25 , xmax = NULL,&#xA;             symmetric = FALSE, keep.diagonal = FALSE,&#xA;             direction.type = &quot;diffHeight&quot;, diffHeight = convert_height(2, &quot;mm&quot;),&#xA;             reduce = 1e-5, self.link = 2,big.gap = 10,&#xA;             preAllocateTracks = NULL,&#xA;             annotationTrack = c(&quot;name&quot;, &quot;grid&quot;, &quot;axis&quot;),&#xA;             annotationTrackHeight = convert_height(c(3, 2), &quot;mm&quot;),&#xA;             link.border = NA, link.lwd = par(&quot;lwd&quot;), link.lty = par(&quot;lty&quot;),&#xA;             link.sort = FALSE, link.decreasing = TRUE,scale = TRUE,&#xA;             link.arr.lwd = par(&quot;lwd&quot;), link.arr.col = par(&quot;col&quot;),&#xA;             link.largest.ontop = FALSE, link.visible = TRUE)&#xA;&#xA;&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fUU8m.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fUU8m.png&quot; alt=&quot;this is somewhat im trying to but still not understandable &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2019-01-18T10:41:59.347" LastActivityDate="2019-01-18T10:41:59.347" Title="Circos plot with circlize" Tags="|r|circos|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6838" PostTypeId="1" CreationDate="2019-01-18T12:51:16.373" Score="3" ViewCount="2403" Body="&lt;p&gt;I am analysing singe cell sequence data and I have followed this tutorial, &lt;a href=&quot;https://satijalab.org/seurat/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/pbmc3k_tutorial.html&lt;/a&gt; to perform QC and various differential analyses using the Seurat package on my data up till now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I rearrange on the resulting plot, the order in which the groups of my samples appear when I use the &lt;code&gt;FeatureHeatmap()&lt;/code&gt; function to visualise the differential gene expression of Gene1 and Gene2 across the cell clusters?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FeatureHeatmap(object = object, features.plot = c(&quot;Gene1&quot;, &quot;Gene2&quot;), &#xA;               group.by = &quot;Conditions&quot;,&#xA;               pt.size = 0.25, key.position = &quot;top&quot;,&#xA;               max.exp = 3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;produces:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/HsHVd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HsHVd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I want:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/QFkwQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QFkwQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3968" LastEditorUserId="3968" LastEditDate="2019-01-18T13:48:23.547" LastActivityDate="2019-01-18T14:04:22.837" Title="How to set the position of groups in a Seurat object on a FeatureHeatmap plot" Tags="|scrnaseq|seurat|ggplot2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6839" PostTypeId="1" AcceptedAnswerId="6851" CreationDate="2019-01-18T13:17:15.287" Score="0" ViewCount="333" Body="&lt;p&gt;I have two normalized gene expression values (log2 of cpm)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(biomarker[,1:5])&#xA;               A2        A3        A4        A6        A7&#xA;A2M    12.4071618 12.631601 12.889748 11.842511 11.574134&#xA;ABCB11  0.5151957  6.176788  5.414905  7.134915  1.247590&#xA;ABCC2   6.7244303  6.487794 10.178615  6.384132  6.089462&#xA;ABCC6   6.3977122  5.823627  7.370761  7.397091  6.071587&#xA;ABCF1   9.1609847  9.893258 10.116638 10.225520 10.838486&#xA;ABCG2   7.6841874  7.738293  6.833152  6.888041  6.030032&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(immune[,1:5])&#xA;              A2        A3        A4        A6        A7&#xA;A2M    10.904748 11.388404  9.910614 11.513439 12.963609&#xA;ABCB11  5.011380  6.359443  7.145992  8.451947  7.722605&#xA;ABCC2   5.040461  6.477014  3.873996  6.409777  9.133971&#xA;ABCC6   7.798441  7.601848  9.948072 11.628533 12.701460&#xA;ABCF1   8.553597  8.615425 11.145903 10.289098 11.444140&#xA;ABCG2   6.224294  5.629375  8.293416  7.979859  8.603793&#xA;&amp;gt; &#xA;&#xA;&amp;gt; dim(biomarker)&#xA;[1] 719  56&#xA;&amp;gt; dim(immune)&#xA;[1] 719  56&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have matched samples in these experiments and same gene too. I could roughly say in both experiments everything has been constant. However, I want to do Student's t-Test to find genes that they had inconsistent gene expression patterns between the two data sets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By this function I am getting a list but I don't know how to extract the names of genes with p-value &gt; 0.05 or whatever cut-off&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; f &amp;lt;- function(x,y){&#xA;+     test &amp;lt;- t.test(x,y, paired=TRUE)&#xA;+     out &amp;lt;- data.frame(stat = test&lt;span class=&quot;math-container&quot;&gt;$statistic,&#xA;+                       df   = test$&lt;/span&gt;parameter,&#xA;+                       pval = test&lt;span class=&quot;math-container&quot;&gt;$p.value,&#xA;+                       conl = test$&lt;/span&gt;conf.int[1],&#xA;+                       conh = test$conf.int[2]&#xA;+     )&#xA;+     return(out)&#xA;+ }&#xA;&amp;gt; t=sapply(seq(ncol(t(biomarker))), function(x) f(t(biomarker)[,x], t(immune)[,x]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any help please?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  str(t)&#xA;&#xA;    [list output truncated]&#xA;     - attr(*, &quot;dim&quot;)= int [1:2] 5 719&#xA;     - attr(*, &quot;dimnames&quot;)=List of 2&#xA;      ..&lt;span class=&quot;math-container&quot;&gt;$ : chr [1:5] &quot;stat&quot; &quot;df&quot; &quot;pval&quot; &quot;conl&quot; ...&#xA;  ..$&lt;/span&gt; : NULL&#xA;    &amp;gt;&#xA;&#xA;&#xA;&amp;gt; sapply(t, &quot;[[&quot;, &quot;pval&quot;)&#xA;Error in FUN(X[[i]], ...) : subscript out of bounds&#xA;&amp;gt;  &#xA;&#xA;&#xA;&amp;gt; pvalueGenes &amp;lt;- sapply(colnames(biomarker),function(i) t.test(biomarker[i, ],immune[i, ], paired = TRUE)&lt;span class=&quot;math-container&quot;&gt;$p.value)&#xA;Error in if (stderr &amp;lt; 10 * .Machine$&lt;/span&gt;double.eps * abs(mx)) stop(&quot;data are essentially constant&quot;) : &#xA;  missing value where TRUE/FALSE needed&#xA;In addition: Warning messages:&#xA;1: In mean.default(x) : argument is not numeric or logical: returning NA&#xA;2: In if (stderr &amp;lt; 10 * .Machine$double.eps * abs(mx)) stop(&quot;data are essentially constant&quot;) :&#xA;&#xA; Show Traceback&#xA;&#xA; Rerun with Debug&#xA; Error in if (stderr &amp;lt; 10 * .Machine$double.eps * abs(mx)) stop(&quot;data are essentially constant&quot;) : &#xA;  missing value where TRUE/FALSE needed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-01-21T17:06:25.747" LastActivityDate="2019-01-22T23:19:34.137" Title="Extracting some part of a list" Tags="|r|rna-seq|statistics|" AnswerCount="2" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="6840" PostTypeId="2" ParentId="6838" CreationDate="2019-01-18T14:04:22.837" Score="5" Body="&lt;p&gt;I don't think this is possible in Seurat v2, but in v3 you can change the factor levels of the grouping variable to change the plot order:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Seurat)&#xA;&#xA;FeaturePlot(object = pbmc_small,&#xA;            features = head(VariableFeatures(pbmc_small), 2),&#xA;            split.by = 'groups')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/c51So.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/c51So.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Change the order:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pbmc_small&lt;span class=&quot;math-container&quot;&gt;$groups &amp;lt;- factor(pbmc_small$&lt;/span&gt;groups, levels = c('g2', 'g1'))&#xA;&#xA;FeaturePlot(object = pbmc_small,&#xA;            features = head(VariableFeatures(pbmc_small), 2),&#xA;            split.by = 'groups')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Rg2BG.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Rg2BG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3415" LastActivityDate="2019-01-18T14:04:22.837" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="6841" PostTypeId="1" CreationDate="2019-01-18T14:49:07.073" Score="5" ViewCount="277" Body="&lt;p&gt;I am looking for a modular way to query data from &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTracks&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENCODE&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I would like to get CHiP-seq or similar tracks for a specific cell line. What's the proper way to do it? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, is there an API to do it?  &lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2019-01-25T04:14:54.467" LastActivityDate="2020-09-04T11:03:59.413" Title="How do you query and explore ENCODE data?" Tags="|public-databases|data-download|api|encode|genetics|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6842" PostTypeId="2" ParentId="6841" CreationDate="2019-01-18T15:13:20.970" Score="7" Body="&lt;p&gt;I don't know whether there is an API, but &lt;a href=&quot;https://www.encodeproject.org/&quot; rel=&quot;noreferrer&quot;&gt;ENCODE's website&lt;/a&gt; does provide an interactive data matrix where you can filter data based on assay and sample type, place data sets in a &quot;shopping cart&quot;, and then proceed to &quot;checkout&quot; to download the files of interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mws6W.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mws6W.png&quot; alt=&quot;Splash page&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1yJCj.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1yJCj.png&quot; alt=&quot;Data matrix&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yMUxb.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yMUxb.png&quot; alt=&quot;Checkout&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-01-18T15:13:20.970" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6843" PostTypeId="2" ParentId="481" CreationDate="2019-01-18T17:39:58.820" Score="1" Body="&lt;p&gt;Related: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/3259/ethnicity-check-either-from-bam-or-vcf-files/3300#3300&quot;&gt;ethnicity check either from bam or vcf files&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The missing piece of your puzzle is an ancestry-labeled reference dataset.  1000 Genomes phase 3 usually works well enough here, and it now has variant callsets for both GRCh37 and GRCh38.&lt;/p&gt;&#xA;" OwnerUserId="472" LastActivityDate="2019-01-18T17:39:58.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6844" PostTypeId="1" AcceptedAnswerId="6845" CreationDate="2019-01-19T01:29:06.777" Score="6" ViewCount="1105" Body="&lt;p&gt;I appologize for asking this, but I really am really bad with regex...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone help me transform the headers of my fasta files from this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Agaricus_chiangmaiensis|JF514531|SH174817.07FU|reps|k__Fungi;p__Basidiomycota;c__Agaricomycetes;o__Agaricales;f__Agaricaceae;g__Agaricus;s__Agaricus_chiangmaiensis&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;JF514531|SH174817.07FU;tax=d:Fungi,p:Basidiomycota,c:Agaricomycetes,o:Agaricales,f:Agaricaceae,g:Agaricus,s:Agaricus_chiangmaiensis_SH174817.07FU;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;seems like I could use the command &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed 's/__/:/g;s/;/,/g;s/.*|/|/g' &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to substitute &lt;code&gt;:&lt;/code&gt; for &lt;code&gt;__&lt;/code&gt;, and &lt;code&gt;,&lt;/code&gt; for &lt;code&gt;;&lt;/code&gt;.  I'm not entirely sure how to proceed from there though...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="3990" LastEditorUserId="298" LastEditDate="2019-01-19T13:06:27.850" LastActivityDate="2019-01-22T19:20:40.970" Title="How do I re-name the headers of my Fasta file?" Tags="|fasta|text-processing|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6845" PostTypeId="2" ParentId="6844" CreationDate="2019-01-19T08:09:15.683" Score="5" Body="&lt;p&gt;If &lt;code&gt;awk&lt;/code&gt; is good as well:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk -v FS=&quot;|&quot; -v OFS=&quot;|&quot; '/&amp;gt;/{&#xA;    gsub(&quot;;&quot;,&quot;,&quot;, $5); &#xA;    gsub(&quot;__&quot;,&quot;:&quot;,$5); &#xA;    print &quot;&amp;gt;&quot;$2,$3&quot;;tax=&quot;$5&quot;_&quot;$&lt;/span&gt;3&quot;;&quot;;&#xA;    next;&#xA;}1' file.fa&#xA;&amp;gt;JF514531|SH174817.07FU;tax=k:Fungi,p:Basidiomycota,c:Agaricomycetes,o:Agaricales,f:Agaricaceae,g:Agaricus,s:Agaricus_chiangmaiensis_SH174817.07FU;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This relies on the order of the parts separated by &lt;code&gt;|&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand why &lt;code&gt;k__Fungi&lt;/code&gt; becomes &lt;code&gt;d:Fungi&lt;/code&gt;, so I ignored it for now. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some explanation for what &lt;code&gt;awk&lt;/code&gt;is doing.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;|&lt;/code&gt; is defined as a field (column) separator for the input and output&lt;/li&gt;&#xA;&lt;li&gt;If the line contains a &lt;code&gt;&amp;gt;&lt;/code&gt; the code between &lt;code&gt;{&lt;/code&gt; and &lt;code&gt;}&lt;/code&gt; is executed.&lt;/li&gt;&#xA;&lt;li&gt;the first &lt;code&gt;gsub&lt;/code&gt; replaces &lt;code&gt;;&lt;/code&gt; with &lt;code&gt;,&lt;/code&gt; in the 5th field&lt;/li&gt;&#xA;&lt;li&gt;the second &lt;code&gt;gsub&lt;/code&gt; replaces &lt;code&gt;__&lt;/code&gt; with &lt;code&gt;:&lt;/code&gt; in the 5th field&lt;/li&gt;&#xA;&lt;li&gt;then we print the new header line. The &lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$2&lt;/code&gt;, &lt;code&gt;$&lt;/span&gt;3&lt;/code&gt; and &lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$5&lt;/code&gt; get replaced by the values of the second, third and 5th field of the input. &lt;code&gt;,&lt;/code&gt; get replaced by the defined output field separator&lt;/li&gt;&#xA;&lt;li&gt;with &lt;code&gt;next&lt;/code&gt; we tell &lt;code&gt;awk&lt;/code&gt; to go to the next input line and repeat the code&lt;/li&gt;&#xA;&lt;li&gt;The &lt;code&gt;1&lt;/code&gt; is in equivalent to &lt;code&gt;print $&lt;/span&gt;0&lt;/code&gt; for printing the whole input line. In our code this is only executed if the line is no header line. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1310" LastEditorUserId="1310" LastEditDate="2019-01-22T19:20:40.970" LastActivityDate="2019-01-22T19:20:40.970" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6846" PostTypeId="2" ParentId="6844" CreationDate="2019-01-19T13:20:05.290" Score="3" Body="&lt;p&gt;It seems like you want to do four things:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Remove all text up to and including the first &lt;code&gt;|&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Now that that has been removed, take the string between what is now (after step 1) the first &lt;code&gt;|&lt;/code&gt; and what is now the second &lt;code&gt;|&lt;/code&gt;, and append it to the end of the header with a &lt;code&gt;_&lt;/code&gt;. &lt;/li&gt;&#xA;&lt;li&gt;Replace all &lt;code&gt;__&lt;/code&gt; with &lt;code&gt;:&lt;/code&gt; and all &lt;code&gt;;&lt;/code&gt; with &lt;code&gt;,&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Change the (fixed?) string &lt;code&gt;reps|k&lt;/code&gt; with &lt;code&gt;tax=d&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If so, and assuming that you only have one occurrence of &lt;code&gt;reps|&lt;/code&gt; in each header, you can do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sed 's/&amp;gt;[^|]*|/&amp;gt;/; &#xA;       s/reps|k/tax=d/; &#xA;       s/__/:/g; &#xA;       s/;/,/g; &#xA;       s/|\([^|]*\)\(|.*\)/|\1\2_\1;/; ' file.fa &#xA;&amp;gt;JF514531|SH174817.07FU;tax=d:Fungi,p:Basidiomycota,c:Agaricomycetes,o:Agaricales,f:Agaricaceae,g:Agaricus,s:Agaricus_chiangmaiensis_SH174817.07FU;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;s/&amp;gt;[^|]*|/&amp;gt;/&lt;/code&gt; : replace the longest stretch of non&lt;code&gt;-|&lt;/code&gt; characters after a &lt;code&gt;&amp;gt;&lt;/code&gt; and until the first &lt;code&gt;|&lt;/code&gt; with &lt;code&gt;&amp;gt;&lt;/code&gt; alone.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;s/reps|k/tax=d/;&lt;/code&gt; : replace the first occurrence of the string &lt;code&gt;reps|k&lt;/code&gt; with &lt;code&gt;tax=&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;s/__/:/g; s/;/,/g&lt;/code&gt; : replace &lt;em&gt;all&lt;/em&gt; (that's what the &lt;code&gt;/g&lt;/code&gt; means) occurrences of &lt;code&gt;__&lt;/code&gt; with &lt;code&gt;:&lt;/code&gt; and of &lt;code&gt;;&lt;/code&gt; with &lt;code&gt;,&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;s/|\([^|]*\)\(|.*\)/|\1\2_\1;/&lt;/code&gt; : the &lt;code&gt;\(foo\)&lt;/code&gt; construct is called a &lt;a href=&quot;https://www.regular-expressions.info/brackets.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;capturing group&lt;/a&gt; and will &lt;em&gt;capture&lt;/em&gt; &lt;code&gt;foo&lt;/code&gt; and save it as &lt;code&gt;\1&lt;/code&gt;, which can be used on the right hand side of the replacement operator. The next captured group will be &lt;code&gt;\2&lt;/code&gt; then &lt;code&gt;\3&lt;/code&gt; etc. Here, with &lt;code&gt;|\([^|]*\)&lt;/code&gt; we are capturing the longest string of non&lt;code&gt;|&lt;/code&gt; found after the 1st &lt;code&gt;|&lt;/code&gt; (this happens after the first operation, so the 1st &lt;code&gt;|&lt;/code&gt; is the one that was 3rd in the original) and then everything after the next &lt;code&gt;|&lt;/code&gt; (&lt;code&gt;|.*&lt;/code&gt;) until the end of the line. We then replace everything matched with a &lt;code&gt;|&lt;/code&gt;, then whatever was matched by the first capture group (&lt;code&gt;\1&lt;/code&gt;), then the second (&lt;code&gt;\2&lt;/code&gt;), then we add a &lt;code&gt;_&lt;/code&gt; and repeat the &lt;code&gt;\1&lt;/code&gt;. &#xA;Finally, if you want to make the changes to the original input file, use &lt;code&gt;-i&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -i.bak 's/&amp;gt;[^|]*|/&amp;gt;/; s/reps|k/tax=d/; s/__/:/g; s/;/,/g; s/|\([^|]*\)\(|.*\)/|\1\2_\1;/; ' file.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That will save the original file as &lt;code&gt;file.fa.bak&lt;/code&gt;. use &lt;code&gt;-i&lt;/code&gt; alone without any argument if you don't need the backup. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-01-22T17:37:23.440" LastActivityDate="2019-01-22T17:37:23.440" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6847" PostTypeId="1" AcceptedAnswerId="6848" CreationDate="2019-01-21T04:17:22.033" Score="2" ViewCount="240" Body="&lt;p&gt;I am doing linear association calculations for SNP effects, I see it take a lot of time to do, does anyone have any experience in parallelizing this job. I have used &lt;code&gt;--threads &amp;lt;num of cores&amp;gt;&lt;/code&gt; but it says it only uses 1 thread&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here is my code for the calculation in plink:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plink --bfile ukbb_dis --pheno phenotype_dis.pheno --mpheno 4 --allow-no-sex --covar phenotype_dis.pheno --covar-number 5-21 --missing-phenotype -10000000 --linear --adjust --out assoc_SNP_height_linear --threads 4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the current output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;257682 MB RAM detected; reserving 128841 MB for main workspace.&#xA;1133273 variants loaded from .bim file.&#xA;284516 people (0 males, 0 females, 284516 ambiguous) loaded from .fam.&#xA;Ambiguous sex IDs written to assoc_SNP_height_linear.nosex .&#xA;284516 phenotype values present after --pheno.&#xA;Using 1 thread.&#xA;Warning: This run includes BLAS/LAPACK linear algebra operations which&#xA;currently disregard the --threads limit.  If this is problematic, you may want&#xA;to recompile against single-threaded BLAS/LAPACK.&#xA;--covar: 17 out of 21 covariates loaded.&#xA;Before main variant filters, 284516 founders and 0 nonfounders present.&#xA;Calculating allele frequencies... done.&#xA;Total genotyping rate is 0.992737.&#xA;1133273 variants and 284516 people pass filters and QC.&#xA;Phenotype data is quantitative.&#xA;Writing linear model association results to&#xA;assoc_SNP_height_linear.assoc.linear ... 0%&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Your help is really appreciated!&lt;/p&gt;&#xA;" OwnerUserId="3973" LastActivityDate="2019-01-21T06:34:30.040" Title="Parallel in PLINK for linear association for SNP effects" Tags="|gwas|plink|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6848" PostTypeId="2" ParentId="6847" CreationDate="2019-01-21T06:34:30.040" Score="2" Body="&lt;p&gt;Try the latest plink 2.0 alpha build for this.  It’s an incomplete program, but its implementation of —linear is far better than v1.9’s.&lt;/p&gt;&#xA;" OwnerUserId="472" LastActivityDate="2019-01-21T06:34:30.040" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6849" PostTypeId="1" CreationDate="2019-01-21T10:20:10.510" Score="3" ViewCount="303" Body="&lt;p&gt;How can I prevent a function from sorting my data groups (factors) in alphabetical order without affecting the integrity of the data? I am analysing single cell RNA sequencing data using Seurat 2.3.4. I want to be able to visualise the expression of two genes, Gene1 and Gene2 across cell clusters using the &lt;code&gt;FeatureHeatmap()&lt;/code&gt; function from Seurat. I have assigned to the metadata slot of my Seurat object the factors or conditions being tested in the experiment. However on the resulting FeatureHeatmap plot, the data are sorted and grouped in alphabetical order to my disfavour. I have attempted a solution given on &lt;a href=&quot;https://stackoverflow.com/questions/16897043/prevent-sorting-of-factor-levels&quot;&gt;stackoverflow&lt;/a&gt; such as the one below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conditions &amp;lt;- c(rep(&quot;WTNaClctrl&quot;, 1229), &#xA;               rep(&quot;WTBlu&quot;, 1328),&#xA;               rep(&quot;ILKOBlu&quot;, 349))&#xA;&#xA;&#xA;conditions &amp;lt;- as.data.frame(conditions)&#xA;&#xA;seurat_object@meta.data&lt;span class=&quot;math-container&quot;&gt;$conditions &amp;lt;- factor(conditions$&lt;/span&gt;conditions, levels=unique(conditions$conditions))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I use the &lt;code&gt;FeatureHeatmap()&lt;/code&gt; function, the plot &lt;strong&gt;still&lt;/strong&gt; sorts the data groups in alphabetical order:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FeatureHeatmap(seurat_object, features.plot = c(&quot;Gene1&quot;, &quot;Gene2&quot;), &#xA;               pt.size = 0.25,  &#xA;               group.by = &quot;conditions&quot;,&#xA;               max.exp = 3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This plot gets produced: &lt;a href=&quot;https://i.stack.imgur.com/4WCei.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4WCei.png&quot; alt=&quot;in alphabetical order&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like groupings not to appear in alphabetical order as such:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/6bmOv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6bmOv.png&quot; alt=&quot;preferred order&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As previously suggested &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/6838/how-to-set-the-position-of-groups-in-a-seurat-object-on-a-featureheatmap-plot/6840?noredirect=1#comment9831_6840&quot;&gt;StackExchange&lt;/a&gt;, I have tried to upgrade my Seurat object to Seurat V3:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;UpdateSeuratObject(Seurat_object): &#xA;Object representation is consistent with the most current Seurat version.&#xA;&#xA;devtools::install_github(repo = 'satijalab/seurat', ref = 'release/3.0') : package ‘curl’ successfully unpacked and MD5 sums checked: &#xA;&#xA;Error: (converted from warning) cannot remove prior installation of package ‘curl’.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for any suggestions.&lt;/p&gt;&#xA;" OwnerUserId="3968" LastActivityDate="2019-04-22T02:01:25.247" Title="How do I prevent the FeatureHeatmap function from the Seurat package, from sorting my data groups in alphabetical order when plotting data?" Tags="|r|scrnaseq|seurat|ggplot2|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="6850" PostTypeId="2" ParentId="6839" CreationDate="2019-01-21T12:55:13.007" Score="2" Body="&lt;p&gt;From your example we are comparing samples not genes, I think we need to run the test row wise, something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pvalueGenes &amp;lt;- sapply(colnames(biomarker),&#xA;                      function(i) t.test(biomarker[i, ],&#xA;                                         immune[i, ], paired = TRUE)$p.value)&#xA;&#xA;names(pvalueGenes)[ pvalueGenes &amp;lt; 0.05 ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="131" LastActivityDate="2019-01-21T12:55:13.007" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="6851" PostTypeId="2" ParentId="6839" CreationDate="2019-01-21T19:43:39.003" Score="2" Body="&lt;p&gt;Starting from the definition of &lt;code&gt;t&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;&amp;gt; t &amp;lt;- sapply(seq(ncol(t(biomarker))), function(x) f(t(biomarker)[,x], t(immune)[,x]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Set the column names to the gene names from one or the other of your conditions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; colnames(t) &amp;lt;- rownames(immune)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Set the cutoff:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; cutoff &amp;lt;- 0.05&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Filter for rows which meet the p-value cutoff criterium, and get their column names (gene names):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; colnames(t)[which(unlist(lapply(t[&quot;pval&quot;,], function(x) x &amp;lt; cutoff)))]&#xA;[1] &quot;ABCC6&quot; &#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Only one gene name is reported here, using your demo datasets. Adjust your cutoff and criterium logic, as needed. For example, I filter here for gene names where the p-value is &lt;em&gt;less&lt;/em&gt; than the cutoff; however, from your question, you may be asking for the inverse, for whatever reason, so you could change &lt;code&gt;&amp;lt;&lt;/code&gt; to &lt;code&gt;&amp;gt;=&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2019-01-22T23:19:34.137" LastActivityDate="2019-01-22T23:19:34.137" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6852" PostTypeId="1" AcceptedAnswerId="11410" CreationDate="2019-01-21T22:31:03.027" Score="1" ViewCount="3116" Body="&lt;p&gt;Can I calculate the molecular mass from a formula like this &lt;code&gt;C9H15N4O8P&lt;/code&gt; with Python e.g. using &lt;code&gt;rdkit&lt;/code&gt;? I think &lt;code&gt;rdkit.Chem.Descriptors.MolWt&lt;/code&gt; needs a molecule object. Though, to generate that I would need e.g. a SMILES string. Is there any other python module that can do the trick?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found this: &lt;a href=&quot;http://verahill.blogspot.com/2012/10/molecular-weight-calculator-in-python.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://verahill.blogspot.com/2012/10/molecular-weight-calculator-in-python.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2020-02-14T18:08:59.533" LastActivityDate="2020-02-14T18:08:59.533" Title="Calculate molecular mass from chemical formula C9H15N4O8P with Python?" Tags="|python|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6853" PostTypeId="1" CreationDate="2019-01-21T22:40:00.477" Score="2" ViewCount="47" Body="&lt;p&gt;I have run smoove then duphold on 46 individuals, then pasted the duphold output back into a squared vcf with all 46 samples. &#xA;I have been using SU/AC as one of my filters, looking for variants that have a minimum threshold of support per alternate allele called. Now I'm confused by some of the variants that are getting filtered out, and wonder why SU (number of pieces of evidence supporting the variant) is not calculated as I had expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For one variant, if I sum the AS values (split-read observation counts) and the AP values (paired-end observation counts) over all 46 individuals, I get AS=24 and AP=15, so 39 total reads supporting the variant in the population. However, SU is only 14 (PE=5 + SR=9).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone tell me why the values for SU, PE, SR are so much lower than the total AS &amp;amp; AP counts? Thank you!&lt;/p&gt;&#xA;" OwnerUserId="3505" LastEditorUserId="96" LastEditDate="2019-01-22T14:35:20.977" LastActivityDate="2019-01-22T14:35:20.977" Title="structural variants: why doesn't AS + AP = SU?" Tags="|vcf|structural-variation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6854" PostTypeId="2" ParentId="6841" CreationDate="2019-01-22T00:08:18.590" Score="5" Body="&lt;p&gt;You can just add &lt;code&gt;/?format=json&lt;/code&gt; to any page to get the JSON output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ENCODE REST API documentation: &lt;a href=&quot;https://www.encodeproject.org/help/rest-api/&quot; rel=&quot;noreferrer&quot;&gt;https://www.encodeproject.org/help/rest-api/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example scripts: &lt;a href=&quot;https://github.com/ENCODE-DCC/submission_sample_scripts&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/ENCODE-DCC/submission_sample_scripts&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2019-01-22T00:08:18.590" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6855" PostTypeId="1" AcceptedAnswerId="6858" CreationDate="2019-01-22T20:26:15.110" Score="6" ViewCount="538" Body="&lt;p&gt;I am trying to basecall data produced by the MinION using the SQK-LSK109 kit and FLO-MIN106 flowcell via the command-line. My version of albacore is the latest (v2.3.4). I tried running using the following command-line (the name of the sequencing run and fast5s are different and are default from machine output):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python3.6 /home/dkennetz/.local/bin/read_fast5_basecaller.py -i ./seqrun/fast5/ -t 1 -s ./good_reads/ -f FLO-MIN106 -k SQK-LSK109 -r -o fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;-t 1 is just for this test. It is running and finishing almost instantly with 0 errors but outputting no reads. I have previously run this same command-line using the SQK-LSK108 kit and had no problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then went on to validate the fast5s using the h5_validate tool:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python3.6 /home/dkennetz/.local/bin/h5_validate /home/dkennetz/.local/lib/python3.6/site-packages/h5_validator/schemas/single_read_fast5.yaml seqrun/fast5/0/seqrun.fast5 -v&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output from this is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    Matcher &amp;lt;'UniqueGlobalKey', type=group match_type=exact count=&#xA;{'minimum_count': 1, 'maximum_count': 1}&amp;gt; was not satisfied after matching &amp;lt;HDF5 file &quot;seqrun.fast5&quot; (mode r)&amp;gt;&#xA;&#xA;Error at /:&#xA;&#xA;Matcher &amp;lt;'PreviousReadInfo', type=group match_type=exact count=&#xA;{'minimum_count': 1, 'maximum_count': 1}&amp;gt; was not satisfied after matching &amp;lt;HDF5 file &quot;seqrun.fast5&quot; (mode r)&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I read on the community page that this may be because the header is missing some relevant meta-data information? Any help on how to fix this would be greatly appreciated!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is cross-posted on the community page for ONT here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://community.nanoporetech.com/posts/albacore-basecalling-runni&quot; rel=&quot;noreferrer&quot;&gt;https://community.nanoporetech.com/posts/albacore-basecalling-runni&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3669" LastActivityDate="2019-03-09T22:49:53.410" Title="Albacore basecalling running but outputs 0 reads" Tags="|nanopore|albacore|base-calling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6856" PostTypeId="1" CreationDate="2019-01-23T04:56:09.247" Score="7" ViewCount="719" Body="&lt;p&gt;I downloaded a VCF file from dbSNP, and I'm curious if the indels in the file are left-aligned for GRCH37 genome. The documentation doesn't say anything.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can we tell if a VCF file has left or right aligned indels?&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="96" LastEditDate="2019-03-26T19:16:47.653" LastActivityDate="2019-03-26T19:16:47.653" Title="How to check if indels in VCF files are left or right aligned?" Tags="|sequence-alignment|vcf|indel|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6857" PostTypeId="2" ParentId="6856" CreationDate="2019-01-23T05:26:59.587" Score="5" Body="&lt;p&gt;By definition in the &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.3.pdf&quot; rel=&quot;noreferrer&quot;&gt;VCF format specification&lt;/a&gt; the POS field is left aligned &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;POS int32 t 0-based leftmost coordinate&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;To confirm that you VCF is following the standard you would have to look at the reference and the match up the entries in the VCF.  However if it is from a reliable source like dbSNP you can probably assume they follow the spec &lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2019-01-23T05:26:59.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6858" PostTypeId="2" ParentId="6855" CreationDate="2019-01-23T07:07:53.700" Score="5" Body="&lt;p&gt;I believe you have multiread fast5 files, as generated by recent versions of MinKNOW. A &lt;a href=&quot;https://github.com/nanoporetech/ont_fast5_api/blob/master/README.rst&quot; rel=&quot;nofollow noreferrer&quot;&gt;conversion script&lt;/a&gt; is available to convert these to the older, one-read one-file format. AFAIK Albacore is being deprecated and no version will be made supporting the multiread fast5 files, as Albacore is being replaced by Guppy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The command line to convert from multi read fast5s to single read fast5s is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$multi_to_single_fast5 -i /path/to/fast5/ -s /path/to/save/output/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="681" LastEditorUserId="3669" LastEditDate="2019-01-23T20:09:44.013" LastActivityDate="2019-01-23T20:09:44.013" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6859" PostTypeId="2" ParentId="6856" CreationDate="2019-01-23T08:24:07.100" Score="6" Body="&lt;p&gt;One possibility is to run &lt;a href=&quot;http://www.htslib.org/doc/bcftools.html#norm&quot; rel=&quot;noreferrer&quot;&gt;bcftools norm&lt;/a&gt; on the file. At the end it will print out a statistic about how many variants were realigned. I did this for dbSNPv151 (GRCh38) only for chromosome 22 like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools view -i 'TYPE=&quot;indel&quot;' dbSNP_GRCh38.p7_b151.vcf.gz 22|bcftools norm -Ou -f GRCh38.fa &amp;gt; /dev/null&#xA;Lines   total/split/realigned/skipped:  901503/0/490/0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1310" LastActivityDate="2019-01-23T08:24:07.100" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6860" PostTypeId="1" CreationDate="2019-01-23T10:53:30.400" Score="1" ViewCount="1364" Body="&lt;p&gt;Supposing upcoming RNA-seq for 3 conditions and 3 replications for each, how much is data storage requirements?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2019-01-23T11:09:28.357" Title="data storage requirements for RNA-seq and WGS" Tags="|r|rna-seq|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6861" PostTypeId="2" ParentId="6860" CreationDate="2019-01-23T11:09:28.357" Score="3" Body="&lt;p&gt;The breakdown would be:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1.5-2 GB per sample for fastq files.&lt;/li&gt;&#xA;&lt;li&gt;~3GB per BAM file, depending on the aligner&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So ~5GB per BAM file, which is about 50GB total. That can vary by maybe 20%, depending on how compressible the results are (I rearrange our fastq files to make them more compressible, so I'm probably underestimating what others would see).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously using Salmon/Kallisto would halve the storage requirements. Using CRAM would also decrease things, though obviously fewer tools support it.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-01-23T11:09:28.357" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6862" PostTypeId="1" CreationDate="2019-01-23T17:19:04.393" Score="2" ViewCount="156" Body="&lt;p&gt;Many of the modern gene-quantification tools (Salmon/Kallisto) output transcript-level (as opposed to gene-level) data. All of the scRNA-seq analysis I have seen just uses the gene-level values. I suppose I can just use the transcript-level matrix, but then I lose any links between the different isoforms of the same gene. Maybe it does not really matter?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few options for bulk RNA-seq and there was a &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/774/analysis-of-differential-transcript-usage-dtu&quot;&gt;similar question here&lt;/a&gt;, but are there any single-cell options?&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2019-01-28T20:16:15.203" Title="scRNA-seq differential transcript usage" Tags="|scrnaseq|single-cell|transcriptome|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6863" PostTypeId="1" AcceptedAnswerId="6864" CreationDate="2019-01-23T17:35:01.780" Score="1" ViewCount="3984" Body="&lt;p&gt;I have a read count data (RNAseq) and want to perform quantile normalization. Could you please help me how to do it. I tried some scripts in R but it didn't work. I want the result output in matrix form.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  gene_id   SRR896664   SRR896663   SRR896665&#xA;  ENSG00000000003   46106   36353   40614&#xA;  ENSG00000000005   198 399 1200&#xA;  ENSG00000000419   40364   37769   40849&#xA;  ENSG00000000457   18924   16211   16057&#xA;  ENSG00000000460   31040   28888   29901&#xA;  ENSG00000000938   200 0   394&#xA;  ENSG00000000971   14935   14353   12522&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The script which i tried is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  data &amp;lt;- read.csv(&quot;testquantile.csv&quot;,header=T)&#xA;  head(data)&#xA;  rownames(data) &amp;lt;- data[,1]&#xA;  data_mat &amp;lt;- data.matrix(data[,-1]) &#xA;  head(data_mat)&#xA;  data_norm &amp;lt;- normalize.quantiles(m, copy = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3766" LastEditorUserId="3766" LastEditDate="2019-01-23T17:59:31.697" LastActivityDate="2020-12-09T10:13:52.107" Title="How to quantile normalization on RNA seq counts" Tags="|rna-seq|gene|normalization|gene-expression|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6864" PostTypeId="2" ParentId="6863" CreationDate="2019-01-23T19:11:33.150" Score="4" Body="&lt;p&gt;On google there are many tutorials about quantile normalzation, for example &lt;a href=&quot;https://davetang.org/muse/2014/07/07/quantile-normalisation-in-r/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. In that tutorial they made a function to calculate quantile normalization. Here an example with that function on your small data set.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data&#xA;          gene_id SRR896664 SRR896663 SRR896665&#xA;1 ENSG00000000003     46106     36353     40614&#xA;2 ENSG00000000005       198       399      1200&#xA;3 ENSG00000000419     40364     37769     40849&#xA;4 ENSG00000000457     18924     16211     16057&#xA;5 ENSG00000000460     31040     28888     29901&#xA;6 ENSG00000000938       200         0       394&#xA;7 ENSG00000000971     14935     14353     12522&#xA;&#xA;rownames(data) &amp;lt;- data$gene_id&#xA;&#xA;quantile_normalisation &amp;lt;- function(df){&#xA;  df_rank &amp;lt;- apply(df,2,rank,ties.method=&quot;min&quot;)&#xA;  df_sorted &amp;lt;- data.frame(apply(df, 2, sort))&#xA;  df_mean &amp;lt;- apply(df_sorted, 1, mean)&#xA;&#xA;  index_to_mean &amp;lt;- function(my_index, my_mean){&#xA;    return(my_mean[my_index])&#xA;  }&#xA;&#xA;  df_final &amp;lt;- apply(df_rank, 2, index_to_mean, my_mean=df_mean)&#xA;  rownames(df_final) &amp;lt;- rownames(df)&#xA;  return(df_final)&#xA;}&#xA;&#xA;new_data &amp;lt;- quantile_normalisation(data[,2:4])&#xA;new_data&#xA;                 SRR896664  SRR896663  SRR896665&#xA;ENSG00000000003 41574.6667 39110.3333 39110.3333&#xA;ENSG00000000005   197.3333   599.6667   599.6667&#xA;ENSG00000000419 39110.3333 41574.6667 41574.6667&#xA;ENSG00000000457 17064.0000 17064.0000 17064.0000&#xA;ENSG00000000460 29943.0000 29943.0000 29943.0000&#xA;ENSG00000000938   599.6667   197.3333   197.3333&#xA;ENSG00000000971 13936.6667 13936.6667 13936.6667&#xA;&#xA;boxplot(data[,2:4])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CtN0l.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CtN0l.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;boxplot(new_data)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dTUfu.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dTUfu.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="939" LastEditorUserId="939" LastEditDate="2019-01-23T19:21:37.007" LastActivityDate="2019-01-23T19:21:37.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6865" PostTypeId="1" CreationDate="2019-01-23T21:02:31.540" Score="3" ViewCount="7543" Body="&lt;p&gt;I downloaded an annotated genome file in gff format &lt;a href=&quot;https://www.phe-culturecollections.org.uk/products/bacteria/detail.jsp?refId=NCTC+13373&amp;amp;collection=nctc&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I would like to use it for proteomics. Though I need it in fasta format. Is there any tool that converts gff to annotated fasta? I see the file contains the genome sequence in fasta format, but not annotated. Are those file formats convertable?&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="2085" LastEditDate="2019-01-28T18:51:02.340" LastActivityDate="2019-01-28T19:21:08.103" Title="Can a gff file be converted to a fasta file?" Tags="|fasta|file-formats|format-conversion|gff|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6866" PostTypeId="2" ParentId="6865" CreationDate="2019-01-23T22:08:47.943" Score="5" Body="&lt;p&gt;A gff is a file of annotation.  It generally doesn't include sequence information, so you can not inter-convert.  But the file on your page has a fasta entry at the bottom..  I'd just grab the bottom half of it, and if the sequence length looks right, that's likely what you want.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-01-23T22:08:47.943" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6867" PostTypeId="2" ParentId="6865" CreationDate="2019-01-23T22:35:27.943" Score="2" Body="&lt;p&gt;I extracted the sequence contained in the gff file into a separate file and added a header according to the fasta file format specifications. I named that file &lt;code&gt;ERS654933.fasta&lt;/code&gt; and then used &lt;code&gt;prokka&lt;/code&gt; to generate my own annotation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;File header ERS654933.fasta:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;ATCC_NCTN13373&#xA;GTGTATAACTTAAAAATTTAAGAAAGATGGAGTAAATTTATGTCGGAAAAAGAAATTTGG&#xA;GAAAAAGTGCTTGAAATTGCTCAAGAAAAATTATCAGCTGTAAGTTACTCAACTTTCCTA&#xA;AAAGATACTGAGCTTTACACGATCAAAGATGGTGAAGCTATCGTATTATCGAGTATTCCT&#xA;TTTAATGCAAATTGGTTAAATCAACAATATGCTGAAATTATCCAAGCAATCTTATTTGAT&#xA;GTTGTAGGCTATGAAGTAAAACCTCACTTTATTACTACTGAAGAATTAGCAAATTATAGT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;prokka ERS654933.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which generated a folder with several files. Here is the header of the generated file &lt;code&gt;PROKKA_01232019.faa&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NEOBMGJD_00001 Chromosomal replication initiator protein DnaA&#xA;MSEKEIWEKVLEIAQEKLSAVSYSTFLKDTELYTIKDGEAIVLSSIPFNANWLNQQYAEI&#xA;IQAILFDVVGYEVKPHFITTEELANYSNNETATPKEATKPSTETTEDNHVLGREQFNAHN&#xA;TFDTFVIGPGNRFPHAASLAVAEAPAKAYNPLFIYGGVGLGKTHLMHAIGHHVLDNNPDA&#xA;KVIYTSSEKFTNEFIKSIRDNEGEAFRERYRNIDVLLIDDIQFIQNKVQTQEEFFYTFNE&#xA;LHQNNKQIVISSDRPPKEIAQLEDRLRSRFEWGLIVDITPPDYETRMAILQKKIEEEKLD&#xA;IPPEALNYIANQIQSNIRELEGALTRLLAYSQLLGKPITTELTAEALKDIIQAPKSKKIT&#xA;IQDIQKIVGQYYNVRIEDFSAKKRTKSIAYPRQIAMYLSRELTDFSLPKIGEEFGGRDHT&#xA;TVIHAHEKISKDLKEDPIFKQEVENLEKEIRNV&#xA;&amp;gt;NEOBMGJD_00002 Beta sliding clamp&#xA;MMEFTIKRDYFITQLNDTLKAISPRTTLPILTGIKIDAKEHEVILTGSDSEISIEITIPK&#xA;TVDGEDIVNISETGSVVLPGRFFVDIIKKLPGKDVKLSTNEQFQTLITSGHSEFNLSGLD&#xA;PDQYPLLPQVSRDDAIQLSVKVLKNVIAQTNFAVSTSETRPVLTGVNWLIQENELICTAT&#xA;DSHRLAVRKLQLEDVSENKNVIIPGKALAELNKIMSDNEEDIDIFFASNQVLFKVGNVNF&#xA;ISRLLEGHYPDTTRLFPENYEIKLSIDNGEFYHAIDRASLLAREGGNNVIKLSTGDDVVE&#xA;LSSTSPEIGTVKEEVDANDVEGGSLKISFNSKYMMDALKAIDNDEVEVEFFGTMKPFILK&#xA;PKGDDSVTQLILPIRTY&#xA;&amp;gt;NEOBMGJD_00003 hypothetical protein&#xA;MVQEVVVEGDINLGQFLKTEGIIESGGQAKWFLQDVEVLINGVRETRRGKKLEHQDRIDI&#xA;PELPEDAGSFLIIHQGEQ&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, that way I could create my own annotation, but I am loosing the annotation information in the &lt;code&gt;gff&lt;/code&gt; file.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2019-01-23T22:35:27.943" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6868" PostTypeId="1" AcceptedAnswerId="6869" CreationDate="2019-01-24T01:58:18.900" Score="3" ViewCount="2422" Body="&lt;p&gt;I am reading about GWAS in heritability. They usually say to calculate heritability on a liability scale but after searching for this word &quot;liability&quot;, I still don't understand clearly what does liability mean? Does anyone have any easy-to-understand explanation regarding this term? Thank you very much!&lt;/p&gt;&#xA;" OwnerUserId="3973" LastActivityDate="2019-01-24T09:17:15.307" Title="What does liability mean in GWAS heritability?" Tags="|gwas|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6869" PostTypeId="2" ParentId="6868" CreationDate="2019-01-24T09:17:15.307" Score="5" Body="&lt;p&gt;When you use linear mixed models to estimate heritability you assume that the underlying trait is normally distributed which is called a disease liability scale. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For continuous traits this is not a problem but for binary traits, this becomes an issue because you have a 0/1 value for a phenotype and usually there is a higher proportion of cases in the study sample than the general population prevalence of disease, which leads to an ascertainment bias. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The population prevalence also varies between populations, for example the prevalence of malaria in one continent is vastly different from another. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So to make the heritability estimates comparable and also to correct for the ascertainment bias, the observed scale heritabilities for dichotomous traits are transformed to the liability scale taking population prevalence and the case proportion into account.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some good papers for reading:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Falconer D.S. The inheritance of liability to certain diseases, estimated from the incidence among relatives. Ann. Hum. Genet. 1965;29:51–71&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3059431/&quot; rel=&quot;noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3059431/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="601" LastActivityDate="2019-01-24T09:17:15.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6870" PostTypeId="2" ParentId="6862" CreationDate="2019-01-24T15:20:56.440" Score="2" Body="&lt;p&gt;There is a paper that takes advantage of transcript compatibility counts (TCCs). A TCC describes all of the potential transcripts that a cDNA alignment can be derived from.  The senior author, Lior Pachter also has a nice &lt;a href=&quot;https://liorpachter.wordpress.com/2018/02/15/gde%C2%B2-dge%C2%B2-dtu%C2%B2-dte%E2%82%81%C2%B2-dte%E2%82%82%C2%B2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt; about differential transcript usage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/s41592-018-0303-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;A discriminative learning approach to differential expression analysis for single-cell RNA-seq&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Single-cell RNA-seq makes it possible to characterize the&#xA;  transcriptomes of cell types across different conditions and to&#xA;  identify their transcriptional signatures via differential analysis.&#xA;  Our method detects changes in transcript dynamics and in overall gene&#xA;  abundance in large numbers of cells to determine differential&#xA;  expression. When applied to transcript compatibility counts obtained&#xA;  via pseudoalignment, our approach provides a quantification-free&#xA;  analysis of 3′ single-cell RNA-seq that can identify previously&#xA;  undetectable marker genes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="64" LastActivityDate="2019-01-24T15:20:56.440" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6871" PostTypeId="1" AcceptedAnswerId="8682" CreationDate="2019-01-24T18:44:47.103" Score="4" ViewCount="186" Body="&lt;p&gt;Accuracies of protein structure predictors have improved quite a lot in recent years. Algorithms such as &lt;a href=&quot;https://www.rosettacommons.org/software&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rosetta&lt;/a&gt; have gotten robust enough to &lt;a href=&quot;http://science.sciencemag.org/content/355/6322/294&quot; rel=&quot;nofollow noreferrer&quot;&gt;predict structures of large number of proteins&lt;/a&gt;. However, I could't find any initiative to make a database for proteome-wide predictions of protein structures or may be integrate the predicted structures in existing databases (PDB or Uniprot).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If any such exists, please let me know.&lt;/p&gt;&#xA;" OwnerUserId="274" LastActivityDate="2022-01-10T11:56:42.923" Title="Database for proteome-wide predictions of protein structures" Tags="|database|proteins|protein-structure|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6873" PostTypeId="1" AcceptedAnswerId="6875" CreationDate="2019-01-24T23:13:46.927" Score="0" ViewCount="820" Body="&lt;p&gt;I tied to install genometools with anaconda:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c bioconda genometools&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Though, after running that successfully the command &lt;code&gt;gt&lt;/code&gt; is not available in my conda environment.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The program 'gt' is currently not installed. You can install it by&#xA;  typing: sudo apt install genometools&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;While conda tells me the program is installed. Are there multiple packages called genometools? What is the difference and can I install the package providing &lt;code&gt;gt&lt;/code&gt; with anaconda at all? There is also a package called &lt;code&gt;genometools-genometools&lt;/code&gt;, but it only has two downloads. So, I am not sure that is what I need. &lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2020-07-25T23:02:30.267" Title="How to install genometools with anaconda?" Tags="|genometools|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6874" PostTypeId="2" ParentId="6863" CreationDate="2019-01-25T03:52:39.400" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://www.rdocumentation.org/packages/limma/versions/3.28.14/topics/normalizeQuantiles&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rdocumentation.org/packages/limma/versions/3.28.14/topics/normalizeQuantiles&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;is a possibility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another popular method is &lt;a href=&quot;https://www.rdocumentation.org/packages/preprocessCore/versions/1.34.0/topics/normalize.quantiles&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rdocumentation.org/packages/preprocessCore/versions/1.34.0/topics/normalize.quantiles&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(preprocessCore)&#xA;normalize.quantiles&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="174" LastActivityDate="2019-01-25T03:52:39.400" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6875" PostTypeId="2" ParentId="6873" CreationDate="2019-01-25T05:07:58.047" Score="2" Body="&lt;p&gt;&lt;code&gt;genometools-genometools&lt;/code&gt; seem to be the current version. By installing the &lt;code&gt;genometools&lt;/code&gt; package I received the same error like you. I guess it's because the binary isn't copy to &lt;code&gt;bin&lt;/code&gt; during installation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;genometools-genometools&lt;/code&gt; seems to work fine. And it has &lt;code&gt;2k&lt;/code&gt; (2000) and not &lt;code&gt;2&lt;/code&gt; :)&lt;/p&gt;&#xA;" OwnerUserId="1310" LastActivityDate="2019-01-25T05:07:58.047" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6877" PostTypeId="1" CreationDate="2019-01-25T19:16:19.063" Score="1" ViewCount="170" Body="&lt;p&gt;I would like to export the data from Protein-Shaker proteomics analysis software and use it in downstream analysis. For example the overview data, numbers of validated peptides per protein, confidence, etc. Ideally I would like to generate this output from commandline without the gui and, if possible to plain txt or csv.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/K0eRE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/K0eRE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2019-01-25T19:23:30.570" LastActivityDate="2019-01-25T20:15:18.907" Title="How to export peptide-shaker data from commandline?" Tags="|peptide-shaker|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6878" PostTypeId="2" ParentId="6877" CreationDate="2019-01-25T19:59:12.820" Score="1" Body="&lt;p&gt;You have to start PeptideShaker calling &lt;code&gt;java&lt;/code&gt; with the &lt;code&gt;-cp&lt;/code&gt; argument an input project file and provide a comma/space-separated list of integers for the reports you want to export: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -cp PeptideShaker-1.16.37.jar eu.isas.peptideshaker.cmd.ReportCLI -in input.cpsx -reports &quot;0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10&quot;`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the list of reports you can export: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-reports                   Comma separated list of types of report to export. &#xA;                           0: Certificate of Analysis, &#xA;                           1: Default Hierarchical Report, &#xA;                           2: Default PSM Phosphorylation Report, &#xA;                           3: Default PSM Report, &#xA;                           4: Default PSM Report with non-validated matches, &#xA;                           5: Default Peptide Phosphorylation Report, &#xA;                           6: Default Peptide Report, &#xA;                           7: Default Peptide Report with non-validated matches, &#xA;                           8: Default Protein Phosphorylation Report, &#xA;                           9: Default Protein Report, &#xA;                           10: Default Protein Report with non-validated matches, &#xA;                           11: Extended PSM Report, &#xA;                           12-n: Your own custom reports.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't know whether you can also use the SearchGUI output zip to generate this files. The program accepts zip files as input, but I think that means PeptideShaker project that are exported to zip files, but not SearchGUI output zips. So, in the end you might need to run the GUI at least one time, unfortunately. &lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2019-01-25T20:15:18.907" LastActivityDate="2019-01-25T20:15:18.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6879" PostTypeId="2" ParentId="2677" CreationDate="2019-01-26T01:39:58.580" Score="9" Body="&lt;p&gt;It may be necessary to distinguish between methods that use unique molecular identifiers (UMIs), such as 10X's Chromium, Drop-seq, etc, and non-UMI methods, such as SMRT-seq.  At least for UMI-based methods, the alternative perspective, that &lt;strong&gt;there is no significant zero-inflation in scRNA-seq&lt;/strong&gt;, is also advocated in the single-cell research community.  The argument is straight-forward: the empirical mean expression vs. dropout rate curve matches the theoretically predicted one, &lt;em&gt;given the current levels of capture efficiency&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Examples&lt;/h1&gt;&#xA;&#xA;&lt;h3&gt;Svensson Blog&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;A couple of blog posts from &lt;a href=&quot;https://twitter.com/vallens&quot; rel=&quot;noreferrer&quot;&gt;Valentine Svensson&lt;/a&gt; argue this point rather pedagogically, and include citations from across the literature:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.nxn.se/valent/2017/11/16/droplet-scrna-seq-is-not-zero-inflated&quot; rel=&quot;noreferrer&quot;&gt;Droplet scRNA-seq is not zero-inflated&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.nxn.se/valent/2018/1/30/count-depth-variation-makes-poisson-scrna-seq-data-negative-binomial&quot; rel=&quot;noreferrer&quot;&gt;Count-depth variation makes Poisson scRNA-seq data negative binomial&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;bayNorm&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;There is a more extensive preprint by Tang, Shahrezaei, et al. &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/384586v2&quot; rel=&quot;noreferrer&quot;&gt;(BioRxiv, 2018)&lt;/a&gt; that claims to show a binomial model is sufficient to account for the observed dropout noise.  Here is a snippet of a relevant conclusion:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Importantly, as bayNorm recovered dropout rates successfully in both UMI-based and non-UMI protocols without the need for specific assumptions, we conclude that invoking zero-inflation models is not required to describe scRNA-seq data.  Consistent with this, the differences in mean expression levels of lowly expressed genes observed between bulk and scRNA-seq data, which were suggested to be indicative of zero-inflation, were recovered by our simulated data using the binomial model only.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h3&gt;Multinomial Modeling&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;There is also a very clearly written preprint by Townes, Irizarry, et al. &lt;a href=&quot;https://doi.org/10.1101/574574&quot; rel=&quot;noreferrer&quot;&gt;(BioRxiv, 2019)&lt;/a&gt; where the authors consider scRNA-seq as a proper compositional sampling (i.e., multinomial process) and they come to a similar conclusion, though &lt;em&gt;specifically for UMI-based methods&lt;/em&gt;. From the paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The multinomial model makes two predictions which we verified using negative control data. First, the fraction of zeros in a sample (cell or droplet) is inversely related to the total number of UMIs in that sample. Second, the probability of an endogenous gene or ERCC spike-in having zero counts is a decreasing function of its mean expression (equations provided in Methods). Both of these predictions were validated by the negative control data (Figure 1). In particular, the empirical probability of a gene being zero across droplets was well calibrated to the theoretical prediction based on the multinomial model.  This also demonstrates that UMI counts are not zero inflated.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Furthermore, by comparing raw read counts (prior to UMI-based deduplication) and UMI counts, they conclude that PCR is indeed the cause of zero-inflation:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The results suggest that while read counts appear zero-inflated and multimodal, UMI counts follow a discrete distribution with no zero inflation (Figure S1). The apparent zero inflation in read counts is a result of PCR duplicates.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I highly recommend giving this a read, especially because it nicely situates other common generative models (e.g., binomial, Poisson) as valid simplifying assumptions of the multinomial model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should be noted that this same group previously published a work (&lt;a href=&quot;https://academic.oup.com/biostatistics/article/19/4/562/4599254&quot; rel=&quot;noreferrer&quot;&gt;Hicks, Irizarry, et al. 2018&lt;/a&gt;), mostly focused on non-UMI-based datasets (SMRT-seq), where they showed evidence that, relative to bulk RNA-seq, there was significant zero-inflation.&lt;/p&gt;&#xA;" OwnerUserId="900" LastEditorUserId="900" LastEditDate="2019-03-17T20:45:22.530" LastActivityDate="2019-03-17T20:45:22.530" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6880" PostTypeId="1" CreationDate="2019-01-26T13:47:00.443" Score="2" ViewCount="77" Body="&lt;p&gt;I have fitted a cox model on a pooled dataset of multiple studies, say studies A, B, and C. As these studies have different baseline hazard functions, I stratified for 'study' in the model.&lt;/p&gt;&#xA;&lt;p&gt;Like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df$SurvObj &amp;lt;- with(df, Surv(event_rd, event == 1))&#xA;&#xA;fit &amp;lt;- coxph(SurvObj ~ cov1 + cov2 + cov3 + cov4 + strata(study),data=df)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, I want to assess whether I can use the beta coefficients from the above model to predict the event probability in a new study, study D.&#xA;The baseline hazard in study D is different from those in studies A, B, and C.&lt;/p&gt;&#xA;&lt;p&gt;When I have&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;predict(fit,type=&amp;quot;lp&amp;quot;,newdata=studyD,reference=&amp;quot;strata&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get this error:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error in model.frame.default(data = studyD, formula = ~hba1c + sbp +&#xA;: factor strata(study) has new levels study=4&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Why does R require that I match the strata of study D with those in study A, B, and/or C when I only want to output the linear predictor (&lt;code&gt;type=&amp;quot;lp&amp;quot;&lt;/code&gt;)? As the linear predictor is independent of the baseline hazard function in this case.&lt;/p&gt;&#xA;&lt;p&gt;I suppose it should be possible to extract the linear predictor for individuals in study D and then manually calculate the event probability using the baseline hazard of study D.&lt;/p&gt;&#xA;&lt;p&gt;Does anybody know how to do this?&lt;br /&gt;&#xA;I could not find any question like this so maybe I am thinking the wrong way.&lt;/p&gt;&#xA;" OwnerUserId="4015" LastEditorUserId="9651" LastEditDate="2021-04-14T01:07:51.453" LastActivityDate="2021-04-14T01:07:51.453" Title="Output the linear predictor from a stratified cox model?" Tags="|r|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6881" PostTypeId="2" ParentId="6863" CreationDate="2019-01-27T05:20:47.037" Score="3" Body="&lt;p&gt;Ma be &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/cqn.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;CQN&lt;/a&gt; from Bioconductor will be useful, though it doesn't perform just quantile normalisation. &lt;/p&gt;&#xA;" OwnerUserId="4017" LastEditorUserId="4017" LastEditDate="2019-01-29T04:43:27.153" LastActivityDate="2019-01-29T04:43:27.153" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6882" PostTypeId="2" ParentId="2551" CreationDate="2019-01-27T21:04:31.210" Score="0" Body="&lt;p&gt;&lt;strong&gt;Two answers:&lt;/strong&gt;&#xA;What you have asked for is a straight Mantel's test. You could read up on the test, it is traditionally used to assess genetic vs geographic distance but could be applied.&#xA;This is easy to do in R:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mantel.test(m1, m2, nperm = 999, graph = FALSE,&#xA;            alternative = &quot;two.sided&quot;,  ...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You will need to import your data (recommend csv) and parse to a data.frame both for your observed (m1) and expected (m2). Its easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The documentation is &lt;a href=&quot;https://www.rdocumentation.org/packages/ape/versions/5.2/topics/mantel.test&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&#xA;My advice is to think carefully about the contruction of the matrix: its an everything versus everything comparison.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Alternative approach&lt;/strong&gt;&#xA;My personal choice would be to use a covariance/correlation matrix. That can be done in R too, but you need to understand what covariance is and how it relates to your problem. If you don't really get covariance, I'd leave it. Hence I'll avoid putting code down - but is part of ggplot2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note, I don't supply detailed codes (I'm not your supervisor), unless its Perl, which this isn't&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-01-27T21:04:31.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6883" PostTypeId="2" ParentId="5507" CreationDate="2019-01-28T10:46:43.527" Score="0" Body="&lt;p&gt;Why not shift the parameter inputs of Blast to exclude the species/subspecies which are being queried? Happy to show you how to do it, but that will pick up homologues. Alternative parse the blast output for e.g. 20000 outputs using E or identity within the threshold required (that's the way I'd do it).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-01-28T10:46:43.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6884" PostTypeId="2" ParentId="6865" CreationDate="2019-01-28T11:58:25.000" Score="5" Body="&lt;p&gt;You can use &lt;a href=&quot;http://ccb.jhu.edu/software/stringtie/gff.shtml#gffread_ex&quot; rel=&quot;noreferrer&quot;&gt;GFF utilities&lt;/a&gt; to get a fasta sequence from a GFF file using a reference genome file.&lt;/p&gt;&#xA;" OwnerUserId="4017" LastActivityDate="2019-01-28T11:58:25.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6885" PostTypeId="1" AcceptedAnswerId="6899" CreationDate="2019-01-28T12:20:20.097" Score="4" ViewCount="737" Body="&lt;p&gt;I'm trying to plot &quot;meta&quot; coverage of annotation: i.e. features (eg. gene class) over certain regions. It is similar to read coverage plots over gene body, except my input is two bed files (both in BED6 format) - (A) one containing the regions for which the meta plot is needed, and another being the list of genes or any feature whose coverage over the regions in (A) is needed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any package or tool which can create such plots (my domain is limited to python but I can try to work with R) ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something akin to this but for whole gene body&lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/gb-2012-13-11-r107&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1]&lt;/a&gt;:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/X2buW.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/X2buW.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[&lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/gb-2012-13-11-r107&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;]- Kelley, D, Rinn, J (2012). Transposable elements reveal a stem cell-specific class of long noncoding RNAs. Genome Biol., 13, 11:R107.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: The BED file containing list of regions looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr3    39218734    40053659    region1 0.92426187419769    +&#xA;chr4    140163762   140453127   region2 0.896103896103896   -&#xA;chr7    40549151    41205036    region3 0.986072423398329   +&#xA;chr8    81291743    81963246    region4 0.94184168012924    -&#xA;chr9    12284032    12539789    region5 0.95539033457249    -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the bed file containing features to be plotted looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr3    39218100    40053200    LINE    1   +&#xA;chr4    140163962   140453027   LINE 1  -&#xA;chr7    40549002    41204999    SINE 1  +&#xA;chr8    81291143    81963846    LTR 1   -&#xA;chr9    12284332    12539720    LTR 1   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1135" LastEditorUserId="1135" LastEditDate="2019-01-30T08:12:15.877" LastActivityDate="2019-01-30T08:12:15.877" Title="Plotting coverage of annotation over collection of region" Tags="|python|software-recommendation|visualization|coverage|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6886" PostTypeId="1" CreationDate="2019-01-28T14:14:23.230" Score="1" ViewCount="152" Body="&lt;p&gt;I have been using BPGA to perform pan-genome analysis on my set of isolates. Now, I need the functional analysis as well. However I am having problem with the function, as the program says I am missing two files: usearch.exe (which I am sure is already present) and cog.udb, which I do not know what it is and where to find it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions on how to solve the error?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4024" LastActivityDate="2019-01-28T14:14:23.230" Title="BPGA - functional analysis" Tags="|software-usage|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6887" PostTypeId="1" CreationDate="2019-01-28T17:23:13.470" Score="2" ViewCount="100" Body="&lt;p&gt;I am currently working in a project that needs MD simulation of a small RNA hairpin. I am currently trying it on GROMACS but it seems to be protein based, hence am facing several issues.&#xA;Is there any software that is designed for the same?&lt;/p&gt;&#xA;" OwnerDisplayName="AKV" LastActivityDate="2019-02-01T10:20:58.207" Title="Which software is used to run Molecular Dynamics simulation of a small RNA hairpin?" Tags="|rna|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6888" PostTypeId="2" ParentId="6865" CreationDate="2019-01-28T19:21:08.103" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there any tool that converts gff to annotated fasta?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is not an uncommon question / request, but perhaps a crash course refresher on file formats would provide some valuable context.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;FASTA files&lt;/strong&gt; are designed to hold nucleotide or amino acid sequences. The &quot;defline&quot; (header) of each record may include some unique identifiers or descriptive metadata. There are no strict requirements, although there are many common conventions. The information that &lt;em&gt;should&lt;/em&gt; be the defline of a FASTA record depends primarily on the tool(s) used to produce and analyze the data.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;GFF files&lt;/strong&gt; are designed to hold annotations corresponding to sequences, most often genomic sequences such as chromosomes or (if it's a draft genome assembly) scaffolds or contigs. Each record in the GFF file specifies a sequence (such as &lt;code&gt;chr6&lt;/code&gt; or &lt;code&gt;scaffold1985&lt;/code&gt;) and an interval (such as &lt;code&gt;19920557-19997344&lt;/code&gt; or &lt;code&gt;2488-2489&lt;/code&gt;). The GFF file labels that interval as a gene or transposable element or transcription factor binding site or whatever genomic feature you're interested in.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So in this context, a &quot;&lt;em&gt;tool that converts gff to annotated fasta&lt;/em&gt;&quot; doesn't make sense since it's an underdefined task. There are many ways to extract sequences from a Fasta file that has been annotated with a GFF file. Many, many scientists have written throwaway programs to do this type of task. It can be done in a hundred subtly different ways, and most of these programs are designed for a single narrow use case. Do you want to:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;pull out the sequence corresponding to every GFF record? Usually not helpful, since many records are children of other records.&lt;/li&gt;&#xA;&lt;li&gt;pull out the sequence corresponding to every gene? More common, but since you want to do proteomics you're probably not interested in promoters or introns (if either of these are annotated in your GFF file). &lt;/li&gt;&#xA;&lt;li&gt;pull out the coding sequence of each gene and translate it into its corresponding peptide sequence?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;It looks like that last point is most likely, but unless I missed something it looks like Prokka is already doing that for you. So the question remains: &lt;strong&gt;precisely what information exists in your GFF file that you would like in your FASTA file&lt;/strong&gt;? You've provided an example of what you have (the FASTA files at least), but can you provide an example of what you &lt;em&gt;want&lt;/em&gt;? Only then can we help you to identify the right tool for the task. There's a good chance it will involve writing some code in a programming language like Python or Perl.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-01-28T19:21:08.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6889" PostTypeId="2" ParentId="6862" CreationDate="2019-01-28T20:16:15.203" Score="4" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I was hoping to find some tool that already exists.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The short answer is, &lt;strong&gt;none exist yet&lt;/strong&gt;. I see a number of hurdles to overcome first.&lt;sup&gt;1&lt;/sup&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, &lt;strong&gt;no one has yet characterized saturation for transcript discovery&lt;/strong&gt;, i.e., the reads per cell needed to maximize transcript information recovery.  This is a typical stepping-stone as a sequencing approach matures, and until it's done, it makes it unlikely that most scRNA-seq datasets are going to be appropriate vis-à-vis statistical power, even if there was a tool.  That is, if people are only aiming for cell-type discrimination via gene-level markers, datasets will likely be underpowered to measure transcript level differences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another issue is the &lt;strong&gt;lack of any transcript-level scRNA-seq simulators&lt;/strong&gt;. All the simulators out there that I know of produce gene-level matrices.  Having a transcript-level simulator is crucial for benchmarking and characterizing the performance of differential expression calling methodologies.  Without anything to test against, that leaves a huge blindspot in terms of estimating the uncertainty (false positive rates, sensitivity) that any tool might have.  However, since some scRNA-seq models (e.g., &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/26040288&quot; rel=&quot;nofollow noreferrer&quot;&gt;Velten, et al. 2015&lt;/a&gt;, or &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/30504886&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lopez, et al., 2018&lt;/a&gt;) are generative, perhaps this is shouldn't be too difficult to adapt to generating FASTQs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simulators are fine and all, but ground truth is probably the key hurdle of the enterprise right now. While there are definitely groups using smFISH to distinguish isoforms quantitatively on a single-cell level, &lt;strong&gt;there doesn't appear to be enough smFISH data out there to measure a transcriptome-wide scRNA-seq method against&lt;/strong&gt;. Hopefully this will change in the coming year or two, as results from more highly-multiplexed single-molecule technologies become available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;&lt;strong&gt;1&lt;/strong&gt; This is just my opinion formed as I try to keep up with the literature, so I could be missing things.  Comments/corrections certainly welcome!&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="900" LastActivityDate="2019-01-28T20:16:15.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6890" PostTypeId="1" AcceptedAnswerId="6891" CreationDate="2019-01-28T20:17:05.003" Score="3" ViewCount="499" Body="&lt;p&gt;I'm new to data filtering on vcf data and vcftools.&lt;br&gt;&#xA;I performed variant calling on my dataset, CHR22, homo sapiens. I'd like to remove sites that are missing in more than 5% of individuals.  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;vcftools --missing-site --vcf updated_ids68.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt; &lt;/h1&gt;&#xA;&#xA;&lt;p&gt;This gives me a file with out.lmiss. That is missingness per locus.&lt;br&gt;&#xA;I had 68 individuals in my input vcf file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CHR     POS     N_DATA  N_GENOTYPE_FILTERED     N_MISS  F_MISS  &#xA;22      16848278        106     0       30      0.283019  &#xA;22      16848492        68      0       68      1  &#xA;22      16849180        69      0       67      0.971014  &#xA;22      16849229        68      0       68      1  &#xA;22      16849376        133     0       3       0.0225564  &#xA;22      16849476        132     0       6       0.0454545  &#xA;22      16851734        126     0       36      0.285714  &#xA;22      16852588        123     0       13      0.105691&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h1&gt; &lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I'm unable to understand the output here. What does third column - &lt;em&gt;N_DATA&lt;/em&gt; tell?&lt;br&gt;&#xA;133, 126 doesn't tell that this is number of individuals if my input had 68 individuals only.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;VCFtools (0.1.15)&lt;br&gt;&#xA;fileformat=VCFv4.1&lt;/p&gt;&#xA;" OwnerUserId="862" LastEditorUserId="96" LastEditDate="2019-01-28T20:41:55.033" LastActivityDate="2019-01-29T02:22:37.567" Title="Output from vcftools missingness" Tags="|variant-calling|variation|filtering|vcftools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6891" PostTypeId="2" ParentId="6890" CreationDate="2019-01-28T20:45:33.503" Score="4" Body="&lt;p&gt;Humans are diploid, so you can expect to see up to 2*N (2*68=136) alleles, so N_DATA is the number of observances of that allele.&lt;/p&gt;&#xA;" OwnerUserId="934" LastEditorUserId="934" LastEditDate="2019-01-29T02:22:37.567" LastActivityDate="2019-01-29T02:22:37.567" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6892" PostTypeId="1" AcceptedAnswerId="6893" CreationDate="2019-01-29T07:21:56.050" Score="2" ViewCount="127" Body="&lt;p&gt;So I'm doing a pathway analysis using &lt;a href=&quot;https://genetrail2.bioinf.uni-sb.de/start.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GeneTrail&lt;/a&gt; database. One thing I am not able to get is this similarity measure being calculated and plotted: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cG2f7.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cG2f7.jpg&quot; alt=&quot;figure&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know how this is being made, what is the output being plotted because each gene is may be involved in multiple pathways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to be able to make it in R again. Any suggestion or help would be appreciated. How to plot the enrichment result in this form ?&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="48" LastEditDate="2019-01-29T08:32:07.210" LastActivityDate="2019-01-29T08:44:27.873" Title="Pathway plot in GeneTrail data base" Tags="|r|rna-seq|pathway|figure-explanation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6893" PostTypeId="2" ParentId="6892" CreationDate="2019-01-29T08:44:27.873" Score="3" Body="&lt;p&gt;This is a a symmetric matrix with the pathways in both axes. So for each pair a similarity index is calculated (usually it takes into account the number of genes in each pathway and the total number of genes in both pathways). There are several similarity indexes available. A typical one is &lt;a href=&quot;https://en.wikipedia.org/wiki/Jaccard_index&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jaccard&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/S%C3%B8rensen%E2%80%93Dice_coefficient&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sørensen-Dice&lt;/a&gt; index, usually 0 means no similarity and 1 equal pathways.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can do this kind of analysis in R using &lt;a href=&quot;https://bioconductor.org/packages/BioCor/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioCor&lt;/a&gt; (Disclosure: I am the author of this package) which uses the Dice similarity. You will need the name of the genes of each pathway, after calculating the similarity of pathways using &lt;code&gt;mpathSim&lt;/code&gt;, you can then plot it with a heatmap (using the &lt;code&gt;heatmap&lt;/code&gt; function).&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2019-01-29T08:44:27.873" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6894" PostTypeId="2" ParentId="6887" CreationDate="2019-01-29T10:55:21.800" Score="2" Body="&lt;p&gt;Google found this review that might be of interest: &lt;a href=&quot;https://pubs.acs.org/doi/10.1021/acs.chemrev.7b00427&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pubs.acs.org/doi/10.1021/acs.chemrev.7b00427&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a link to a free full-text version:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/pmid/29297679/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/pmid/29297679/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors &quot;here provide a comprehensive overview of the fast-developing field of MD simulations of RNA molecules. We begin with an in-depth, evaluatory coverage of the most fundamental methodological challenges that set the basis for the future development of the field, in particular, the current developments and inherent physical limitations of the atomistic force fields and the recent advances in a broad spectrum of enhanced sampling methods. We also survey the closely related field of coarse-grained modeling of RNA systems. After dealing with the methodological aspects, we provide an exhaustive overview of the available RNA simulation literature, ranging from studies of the smallest RNA oligonucleotides to investigations of the entire ribosome. Our review encompasses tetranucleotides, tetraloops, a number of small RNA motifs, A-helix RNA, kissing-loop complexes, the TAR RNA element, the decoding center and other important regions of the ribosome, as well as assorted others systems. Extended sections are devoted to RNA–ion interactions, ribozymes, riboswitches, and protein/RNA complexes. Our overview is written for as broad of an audience as possible, aiming to provide a much-needed interdisciplinary bridge between computation and experiment, together with a perspective on the future of the field.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastEditorUserId="3828" LastEditDate="2019-02-01T10:20:58.207" LastActivityDate="2019-02-01T10:20:58.207" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6895" PostTypeId="2" ParentId="6885" CreationDate="2019-01-29T11:00:55.103" Score="0" Body="&lt;p&gt;There is a reasonably nice looking tutorial on metaplots in R here: &lt;a href=&quot;https://rpubs.com/achitsaz/94710&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rpubs.com/achitsaz/94710&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-01-29T11:00:55.103" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6896" PostTypeId="2" ParentId="4531" CreationDate="2019-01-29T11:14:06.697" Score="2" Body="&lt;p&gt;For maximum parsimony, Fitch's method seems simple and intuitive, and can be done in the R phangorn package: Fitch, Walter M. (1971). &quot;Toward Defining the Course of Evolution: Minimum Change for a Specific Tree Topology&quot;. Systematic Zoology. 20 (4): 406. doi:10.2307/2412116. ISSN 0039-7989. JSTOR 2412116&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are various methodological criticisms of parsimony that you can find in the literature, but its appeal is you can easily understand why the algorithm came to the conclusions it did. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;RAxML can do maximum likelihood reconstructions based on a tree and tip values, and I think Mr. Bayes can do a Bayesian version.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastEditorUserId="3828" LastEditDate="2019-02-01T10:33:33.850" LastActivityDate="2019-02-01T10:33:33.850" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6897" PostTypeId="1" CreationDate="2019-01-29T17:47:06.250" Score="5" ViewCount="31" Body="&lt;p&gt;I need to predict disorder of proteins and &lt;a href=&quot;http://d2p2.pro/about/predictors&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; there is a description of &lt;a href=&quot;https://iupred2a.elte.hu&quot; rel=&quot;nofollow noreferrer&quot;&gt;Iupred&lt;/a&gt; in its two variants, &lt;strong&gt;long&lt;/strong&gt; and &lt;strong&gt;short&lt;/strong&gt;-disorder. &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0022283605001294?via%3Dihub&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; is a manuscript defining the method. I couldn't find a precise definition of short or long-disorder.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to predict disorder in &lt;strong&gt;30-mers&lt;/strong&gt;, but I expect that as with secondary structure prediction, the longer the input sequence the more accurate the result. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will appreciate if anyone has experience with this software and can share ideas or suggestion on how to proceed.&lt;/p&gt;&#xA;" OwnerUserId="2970" LastEditorUserId="298" LastEditDate="2019-01-30T10:30:35.320" LastActivityDate="2019-01-30T10:30:35.320" Title="Iupred definition of long/short form disorder" Tags="|software-recommendation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6898" PostTypeId="1" CreationDate="2019-01-30T00:17:23.547" Score="3" ViewCount="128" Body="&lt;p&gt;So I aligned roughly 5k sequences and I got my output using mafft. However, I want to restrict the output to only present conserved segments of a specific length (let's say 25bp). Does mafft have parameters to set this that I seem to be missing?&lt;/p&gt;&#xA;" OwnerUserId="4035" LastActivityDate="2019-01-30T20:56:06.883" Title="Restricting match output for multiple sequence alignment using mafft?" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6899" PostTypeId="2" ParentId="6885" CreationDate="2019-01-30T07:42:45.817" Score="3" Body="&lt;p&gt;If one assumes that repeat elements of a given type (e.g., LINEs) don't overlap each other, then the following will work:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Split your BED file by repeat element, such that you have a LINE.bed, SINE.bed, etc.&lt;/li&gt;&#xA;&lt;li&gt;Convert those to bedGraph (e.g., &lt;code&gt;awk 'BEGIN{OFS=&quot;\t&quot;}{print &lt;span class=&quot;math-container&quot;&gt;$1,$&lt;/span&gt;2,$3,&quot;1.0&quot;}' LINE.bed &amp;gt; LINE.bedGraph&lt;/code&gt;).&lt;/li&gt;&#xA;&lt;li&gt;Use UCSC tools to convert those bedGraph files to BigWig.&lt;/li&gt;&#xA;&lt;li&gt;Install deepTools and run &lt;code&gt;computeMatrix reference-point -b 2500 -a 20 -S LINE.bigWig SINE.bigWig LTR.bigWig -R Regions_Of_Interest.BED -o foo.mat.gz&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Make a profile plot with plotProfile (&lt;code&gt;plotProfile -m foo.mat.gz -o foo.png --perGroup&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If you do have overlapping regions you'll need to first make a disjoint set of intervals (you can use bedops for this), find the coverage of them (e.g., &lt;code&gt;bedtools coverage&lt;/code&gt;) and then continue on with that.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-01-30T07:42:45.817" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6900" PostTypeId="1" CreationDate="2019-01-30T12:59:49.163" Score="2" ViewCount="101" Body="&lt;p&gt;I have 25 highly differentially expressed genes among and patients to chemotherapy.&lt;/p&gt;&#xA;&lt;p&gt;I have made a neural network of these genes. The accuracy of the model is &lt;code&gt;0.73&lt;/code&gt; but I don't know from 25 genes how I could select the best predictors or remove the weaker ones.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; # Convert column from /to 1/0&#xA;&amp;gt; = as.numeric(data$)-1&#xA;&amp;gt; data1 = cbind(,df)&#xA;&#xA;&amp;gt; head(data1[,1:4])&#xA;   ALB CALML5 CALML6&#xA;A2       0 4.64   5.77   3.16&#xA;A3       0 7.12   9.13   5.13&#xA;A4       0 7.84   7.09   4.85&#xA;A6       0 7.90   9.64   6.52&#xA;A7       0 5.84   6.26   0.00&#xA;A8       0 4.83   5.77   2.15&#xA;&amp;gt;&#xA;&amp;gt; library(caTools)&#xA;&amp;gt; set.seed(101)&#xA;&amp;gt; # Create Split (any column is fine)&#xA;&amp;gt; split = sample.split(data1&lt;span class=&quot;math-container&quot;&gt;$, SplitRatio = 0.70)&#xA;&amp;gt; # Split based off of split Boolean Vector&#xA;&amp;gt; train = subset(data1, split == TRUE)&#xA;&amp;gt; test = subset(data1, split == FALSE)&#xA;&amp;gt; feats &amp;lt;- names(data[,-1])&#xA;&amp;gt; # Concatenate strings&#xA;&amp;gt; f &amp;lt;- paste(feats,collapse=' + ')&#xA;&amp;gt; f &amp;lt;- paste('TG ~',f)&#xA;&amp;gt; # Convert to formula&#xA;&amp;gt; f &amp;lt;- as.formula(f)&#xA;&amp;gt; f&#xA;~ ALB + CALML5 + CALML6 + CDK6 + CHGA + CREB3L3 + CSF3 + &#xA;    CXCL5 + CXCL6 + CXCL8 + DKK1 + EGLN2 + FGF20 + FGF4 + IL11 + &#xA;    IL1B + IL6 + ISG15 + KLK5 + KRT17 + MMP3 + MMP7 + PKP1 + &#xA;    PTGS2 + S100A7A&#xA;&amp;gt; #install.packages('neuralnet')&#xA;&amp;gt; library(neuralnet)&#xA;Warning message:&#xA;package ‘neuralnet’ was built under R version 3.5.2 &#xA;&amp;gt; nn &amp;lt;- neuralnet(f,train,hidden=c(25,2,1),linear.output=FALSE)&#xA;&amp;gt; View(test)&#xA;&amp;gt; # Compute Predictions off Test Set&#xA;&amp;gt; predicted.nn.values &amp;lt;- compute(nn,test[2:26])&#xA;&amp;gt; # Check out net.result&#xA;&amp;gt; print(head(predicted.nn.values$&lt;/span&gt;net.result))&#xA;             [,1]&#xA;A4  0.26890819169&#xA;B5  0.94836576493&#xA;B7  0.19753554824&#xA;B8  0.99790838807&#xA;B9  0.99764782073&#xA;B12 0.05230351043&#xA;&amp;gt; predicted.nn.values&lt;span class=&quot;math-container&quot;&gt;$net.result &amp;lt;- sapply(predicted.nn.values$&lt;/span&gt;net.result,round,digits=0)&#xA;&amp;gt; table(test&lt;span class=&quot;math-container&quot;&gt;$Private,predicted.nn.values$&lt;/span&gt;net.result)&#xA;   &#xA;    0 1&#xA;  0 4 3&#xA;  1 1 7&#xA;&#xA;&amp;gt; accuracy&amp;lt;-(table(test&lt;span class=&quot;math-container&quot;&gt;$Private,predicted.nn.values$&lt;/span&gt;net.result)[1,1]+table(test&lt;span class=&quot;math-container&quot;&gt;$Private,predicted.nn.values$&lt;/span&gt;net.result)[2,2])/(sum(table(test&lt;span class=&quot;math-container&quot;&gt;$Private,predicted.nn.values$&lt;/span&gt;net.result)))&#xA;&#xA;&#xA;&amp;gt; accuracy&#xA;[1] 0.7333333333&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="9651" LastEditDate="2021-04-14T01:07:30.507" LastActivityDate="2021-04-14T01:07:30.507" Title="How I know which gene is a good predictor in this neural network or not?" Tags="|r|rna-seq|transcriptome|clustering|machine-learning|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6901" PostTypeId="2" ParentId="6887" CreationDate="2019-01-30T17:29:01.230" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://ambermd.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AMBER&lt;/a&gt; has force fields specifically for viewing RNA free energy and structural dynamics. Specifically, AMBERff99 is useful for RNA analysis. It is fairly easy to use and a comprehensive software for your analysis. It has plenty of &lt;a href=&quot;http://ambermd.org/tutorials/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorials&lt;/a&gt; from basic to advanced usage, as well as many publications. A specific publication related to your question can be found at &lt;/p&gt;&#xA;&#xA;&lt;p&gt;doi: 10.1016/j.bpj.2009.10.040&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in general, the &lt;a href=&quot;https://pubs.acs.org/journal/jpcbfk&quot; rel=&quot;nofollow noreferrer&quot;&gt;J Phys Chem B&lt;/a&gt; is relevant to MD studies. &lt;/p&gt;&#xA;" OwnerUserId="3669" LastActivityDate="2019-01-30T17:29:01.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6902" PostTypeId="2" ParentId="6900" CreationDate="2019-01-30T18:01:53.543" Score="1" Body="&lt;p&gt;Yes, there is a formal solution. You want a heat map of the layers weighting and this is the information sought. It is more difficult to interpret than other supervised learning algorithms, but that is the nature of neural networks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know Sci-test Learn (Python), but I don't know R for neural networks, so I can't advise you on the code, only that it will be a few lines of code and a graphical (relative) output. I imagine Tensorflow will have a really brilliant heat map layer weighting.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-01-30T23:30:16.840" LastActivityDate="2019-01-30T23:30:16.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6904" PostTypeId="1" CreationDate="2019-01-30T18:34:49.157" Score="6" ViewCount="491" Body="&lt;p&gt;EDIT: I am updating this question to make it more specific to my issue.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;For context - original question prior to edit:&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;How do I obtain a deamination metric when doing the variant calling using the IonTorrent variant caller, and secondly, how do I correct my called variants for deamination to ensure that these don't provide false positives for the downstream analysis? &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Background&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I'm calling variants using the IonTorrent TorrentSuite on DNA which has been sequenced from formalin-fixed paraffin-embedded (FFPE) tissue. This has a major issue in that without addition of uracil-N-glycosylase, some of the Ts in the original DNA are deaminated to Cs, which upon sequencing and calling variants can show up as mutations, either as T&gt;C transitions or G&gt;A (from the opposite strand, due to PCR in the library prep). I do not have any idea how long these samples were stored without UNG before sequencing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;TVC (Torrent Variant Caller) gives a deamination metric (essentially, sum of T&gt;C and G&gt;A variants over all variants called), and for our samples, the highest value seen is ~0.92. Naively postprocessing the variants show that for these samples, C&gt;T/T&gt;C transitions overwhelm the remaining variants among my samples.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Application&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;My use case is this: these are medical samples, which have been inspected by a pathologist (hence the FFPE treatment), and I want to determine which variants are &lt;em&gt;predictive&lt;/em&gt; of outcome, hence I have two potentially contradictory goals: reduce false positives and capture the rarer variants which may hold predictive power.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vpKxA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vpKxA.png&quot; alt=&quot;T&amp;gt;C and C&amp;gt;T transitions&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Question&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Given the IonTorrent variant calling pipeline (sequencing &gt; BAM file &gt; TVC &gt; VCF file with deamination statistic):&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;(Q1)&lt;/strong&gt; Is there a way of correcting the output VCF to remove transition and deamination errors?&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;(Q2)&lt;/strong&gt; Alternatively, is there a set of filters to use in bcftools to reduce the effect of the errors on how variants are called?&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2244" LastEditorUserId="2085" LastEditDate="2019-02-06T21:21:33.733" LastActivityDate="2019-02-07T11:08:49.283" Title="How to correctly call a VCF file using damaged DNA? (IonTorrent &amp; FFPE)" Tags="|variant-calling|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6905" PostTypeId="2" ParentId="6898" CreationDate="2019-01-30T20:19:03.780" Score="2" Body="&lt;h2&gt;Q - Is there some parameter for this?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;No, there is no parameter for this. Getting the conserved regions is something that you will have to perform post-alignment. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;How to get the conserved regions?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;You will need to define what exactly you mean by conservation and calculate a conservation score for each position of the alignment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance you can look at each column of the alignment and calculate the percent of queries that had the same base/AA at that position. If using AAs you can define conservation more loosely depending on what properties are important to you, or using a scoring matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have a co-occurrence matrix you can define the regions of interest based on a conservation threshold that you define.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Is there software to find conserved columns?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;There is a program &lt;a href=&quot;https://github.com/gcollet/MstatX&quot; rel=&quot;nofollow noreferrer&quot;&gt;MstatX&lt;/a&gt; which outputs conservation scores of individual columns.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;How MstatX works&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The example of MstatX provides this example alignment. Each column is progressively less conserved than the last. The last column is conserved, but has some gaps -- this lowers the conservation score.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   col          1&#xA; seq1  1234567890&#xA; seq2  DDDDDDIPDL&#xA; seq3  DDDDDDIPVL&#xA; seq4  DDDDDDIPYL&#xA; seq5  DDDDDDIPA-&#xA; seq6  DDDDDDLWT-&#xA; seq7  DDEDEELWK-&#xA; seq8  DDEDEELWP-&#xA; seq9  DDEDEFVSR-&#xA; seq10 DEEFFFVSH-&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The conservation scores for each column were:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1       2.71272e-08&#xA;2       0.243433&#xA;3       0.312789&#xA;4       0.243433&#xA;5       0.483364&#xA;6       0.480391&#xA;7       0.498757&#xA;8       0.498757&#xA;9       0.973514&#xA;10      0.267319&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;How to get conserved regions&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The python program below will output all windows of conserved regions when using a fasta alignment and the default output from &lt;code&gt;MstatX&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Positional args &#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;1.&lt;/strong&gt; &lt;code&gt;MstatX_alignment_example.mali&lt;/code&gt;, is the fasta alignment file. &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;2.&lt;/strong&gt; &lt;code&gt;output.txt&lt;/code&gt; is the default output of &lt;code&gt;mstatx&lt;/code&gt; when called with &lt;code&gt;mstatx -i MstatX_alignment_example.mali&lt;/code&gt;. &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;3.&lt;/strong&gt; 0.4 is the minimum conservation score&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;4.&lt;/strong&gt; 3 is the minimum number of bases that must have that conservation score.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Calling this program with the following command on the above example: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;python print_conserved_regions.py MstatX_alignment_example.mali output.txt 0.4 3&lt;/code&gt; prints:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{1,4}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Meaning that there was only one window at least 3 bases long in which all positions had a conservation score below 0.4. This window was positions 1 through 4 in the alignment.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#call this file cons_windows.py&#xA;import sys&#xA;from Bio import AlignIO&#xA;&#xA;align_filename = sys.argv[1]&#xA;cons_filename = sys.argv[2]&#xA;cutoff = float(sys.argv[3])&#xA;minchar = int(sys.argv[4])&#xA;&#xA;def parse_conservation(filename):&#xA;    cons = {}&#xA;    with open(filename, &quot;r&quot;) as f:&#xA;        for line in f:&#xA;            if line.strip():&#xA;                temp = line.split()&#xA;                cons[int(temp[0])] = float(temp[1])&#xA;    return cons&#xA;&#xA;align = AlignIO.read(align_filename, &quot;fasta&quot;)&#xA;cons = parse_conservation(sys.argv[2])&#xA;&#xA;i = 1&#xA;while i &amp;lt; max(cons):&#xA;    if cons[i] &amp;lt;= cutoff:&#xA;        j = i&#xA;        done = False&#xA;        while j &amp;lt; max(cons) and not done:&#xA;            if (cons[j] &amp;gt; cutoff) and (j-i &amp;gt;= minchar):&#xA;                print(&quot;{{{},{}}}&quot;.format(i,j-1))&#xA;                done = True&#xA;                i = j&#xA;            j+=1&#xA;    i+=1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2019-01-30T20:56:06.883" LastActivityDate="2019-01-30T20:56:06.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6906" PostTypeId="1" CreationDate="2019-01-30T20:48:14.017" Score="1" ViewCount="60" Body="&lt;p&gt;Suppose I have a short chain string of oligos, and I want to rate them from 0 to 1 based on how clustered the &quot;A&quot;s are in respect to the chain. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;AAAAACCCCC&quot; #should score as 1 for being as clustered as it possibly could&#xA;&quot;ACACACACAC&quot; #should be scored as 0 for being as thinly distributed as it could&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Current idea is to strip both ends of A's, then remove first and last characters, then &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gaps = ''.join(str(x) for x in deck).split(&quot;A&quot;)&#xA;listofCs = [x for x in gaps if x] &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to join it all together as a list of how big the gaps are (i.e. the &quot;C&quot;s), and simply count the average size &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;average = sum(map(len,listofCs))/len(listofCs)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I feel like there should be a much more elegant solution to this. It feels messy. How can I improve the scoring?&lt;/p&gt;&#xA;" OwnerUserId="4039" LastEditorUserId="2085" LastEditDate="2019-02-03T06:04:53.487" LastActivityDate="2019-02-03T15:51:15.817" Title="How to score how densely bases are distributed along a given oligo?" Tags="|ngs|algorithms|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6907" PostTypeId="2" ParentId="6904" CreationDate="2019-01-30T21:05:22.300" Score="2" Body="&lt;p&gt;Sites that were deaminated are often called abasic sites.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Q1: How to obtain DNA damage metrics?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;You might want to check out &lt;a href=&quot;https://github.com/ginolhac/mapDamage&quot; rel=&quot;nofollow noreferrer&quot;&gt;mapDamage&lt;/a&gt; - this software is used to perform these sorts of analyses and produce plots of types of DNA damage along sequencing reads.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Q2: How to correct for abasic sites and other DNA damage errors?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;You can use your choice of kmer-based or alignment based error correction software. The errors will be random and if the sequencing depth is sufficient the erroneous kmers at the ends of reads will be corrected. However, depending on what you are trying to do downstream with your reads you may be better served by not correcting the reads, then dealing with transitions/damage later. &lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2019-01-30T21:11:49.673" LastActivityDate="2019-01-30T21:11:49.673" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6908" PostTypeId="1" CreationDate="2019-01-30T21:40:29.110" Score="4" ViewCount="568" Body="&lt;p&gt;I'm looking at Broad Institute's orignal GSEA-P algorithm R script which I downloaded here: &lt;a href=&quot;http://software.broadinstitute.org/gsea/downloads.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://software.broadinstitute.org/gsea/downloads.jsp&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to adapt their GSEA.1.0.R script to process datasets that have 1 gene expression profile as for class 1, and N gene expression profiles for class 2. The bottleneck seems to be the GSEA.MetaboliteRanking() method. I looked and it appears they are using &quot;signal to noise ratio&quot; (SNR) metric to rank genes in patient profiles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I was wondering how the signal to noise is calculated from gene expression profiles?&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I am using/modifying their script because it's the only implementation of the GSEA algorithm that I could find where they expose their code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i.e., what is the mathematical reasoning behind the following lines of code? Note, there are little to no comments in the code, but the following variables, from what I can tell, are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A: Matrix of gene expression values (rows are genes, columns are samples) &#xA;class.labels: Phenotype of class distinction of interest. A vector of &#xA;              binary labels having first the 1s and then the 0s &#xA;&#xA;&#xA;&#xA;P = class.labels1 * subset.mask&#xA;n1 = sum(P[,1])         &#xA;M1 = A %*% P&#xA;M1 = M1/n1      &#xA;A2 = A*A        &#xA;S1 = A2 %*% P   &#xA;S1 = S1/n1 - M1*M1    &#xA;S1 = sqrt(abs((n1/(n1-1)) * S1))&#xA;&#xA;P = class.labels2 * subset.mask&#xA;n2 = sum(P[,1])           &#xA;M2 = A %*% P           &#xA;M2 = M2/n2          &#xA;A2 = A*A           &#xA;S2 = A2 %*% P      &#xA;S2 = S2/n2 - M2*M2 &#xA;S2 = sqrt(abs((n2/(n2-1)) * S2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4040" LastEditorUserId="4040" LastEditDate="2019-01-31T23:41:50.677" LastActivityDate="2019-03-04T09:44:33.767" Title="Gene Ranking - signal to noise ratio used in GSEA-P algorithm?" Tags="|r|gene-expression|gsea|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6909" PostTypeId="2" ParentId="2554" CreationDate="2019-01-31T05:04:07.483" Score="1" Body="&lt;p&gt;Here is another way to do it in 2019:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools view -T regions.bed -i 'ID=@&amp;lt;myrsid&amp;gt;.txt' input.vcf &amp;gt; output.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Things can be sped up if we compress and index the &lt;code&gt;input.vcf&lt;/code&gt;. Then &lt;code&gt;bcftools&lt;/code&gt; will have near-immediate random access to the position.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ bgzip -c input.vcf &amp;gt; input.vcf.gz&#xA;$&lt;/span&gt; tabix input.vcf.gz&#xA;$ bcftools view -T regions.bed -i 'ID=@&amp;lt;myrsid&amp;gt;.txt' input.vcf.gz &amp;gt; output.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Internally &lt;code&gt;bcftools&lt;/code&gt; works with &lt;code&gt;bcf&lt;/code&gt;. Whenever the input and output is &lt;code&gt;vcf&lt;/code&gt; it converts &lt;code&gt;vcf&lt;/code&gt;&gt;&lt;code&gt;bcf&lt;/code&gt;&gt;&lt;code&gt;vcf&lt;/code&gt;. When dealing with large files, this produces a large overhead. Converting &lt;code&gt;vcf&lt;/code&gt; to &lt;code&gt;bcf&lt;/code&gt; first can reduce runtime for further downstream analyses dramatically. In my experience 2-3x faster.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ bcftools view -Ob input.vcf &amp;gt; input.bcf&#xA;$&lt;/span&gt; bcftools index input.bcf&#xA;$ bcftools view -Ob -T regions.bed -i 'ID=@&amp;lt;myrsid&amp;gt;.txt' input.bcf &amp;gt; output.bcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1310" LastEditorUserId="96" LastEditDate="2019-03-01T15:34:47.657" LastActivityDate="2019-03-01T15:34:47.657" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6910" PostTypeId="1" AcceptedAnswerId="6915" CreationDate="2019-01-31T05:39:45.503" Score="8" ViewCount="10828" Body="&lt;p&gt;PacBio provides longer read length than Illumina's short-length reads. Longer reads offer better opportunity for genome assembly, structural variant calling. It is not worse than short reads for calling SNP/indels, quantifying transcripts. Sounds like PacBio can do whatever Illumina platform can offer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would that be any reason for going with Illumina if PacBio's long reads can do everything and more?&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="3694" LastEditDate="2023-07-28T13:25:37.333" LastActivityDate="2023-07-28T13:25:37.333" Title="What reasons are there to choose Illumina if PacBio provides longer and better reads?" Tags="|phylogenetics|ngs|phylogeny|long-reads|pacbio|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6911" PostTypeId="1" AcceptedAnswerId="6931" CreationDate="2019-01-31T10:20:26.127" Score="3" ViewCount="787" Body="&lt;p&gt;I'm running a custom perl script using Snakemake. I use this rule:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule complexity_20mer_counter:&#xA;    input:&#xA;       os.path.join(fastq_trimmed_dir, '{sample}_{read}' + fastq_extension.split('.')[0] + '_val_' + '{read}'.strip('R') + '.fq.gz')&#xA;    output:&#xA;        os.path.join(custom_qc_dir, '{sample}_{read}_20mer_counter.txt')&#xA;    shell:&#xA;        '''&#xA;        #!/bin/bash&#xA;        perl scripts/20mer_counter.pl {input} &amp;gt; {output}&#xA;        '''&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It fails (I'll get to why later) with this error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Building DAG of jobs...&#xA;MissingInputException in line 1 of /rst1/2017-0205_illuminaseq/scratch/swo-406/2019-01-24/rules/20mer_counter.smk:&#xA;Missing input files for rule complexity_20mer_counter:&#xA;fastq_trimmed/xs0067_P2018SEQE27R01_S1_R1_001_val_R1.fq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;... this is because I use the output of trim_galore, this rule looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule trim_galore:&#xA;    input:&#xA;       [os.path.join(fastq_raw_dir, '{sample}_'+ read + fastq_extension)&#xA;       for read in reads]&#xA;    output:&#xA;        timmed_fastq_files = [os.path.join(fastq_trimmed_dir, '{sample}_' + read + fastq_extension.split('.')[0] + '_val_' + read.strip('R') + '.fq.gz')&#xA;        for read in reads],&#xA;        trimming_reports = [os.path.join(trim_galore_qc_dir, '{sample}_'+ read + fastq_extension + '_trimming_report.txt')&#xA;        for read in reads]&#xA;    conda:&#xA;        &quot;../envs/fastqc.yaml&quot;&#xA;    shell:&#xA;        '''&#xA;        #!/bin/bash&#xA;        trim_galore --fastqc --gzip -o {fastq_trimmed_dir} --paired --retain_unpaired {input}&#xA;        # Move all qc reports to the fastq_trimmed_dir&#xA;        mv {fastq_trimmed_dir}/*.zip \&#xA;           {fastq_trimmed_dir}/*.html \&#xA;           {fastq_trimmed_dir}/*.txt \&#xA;           {trim_galore_qc_dir}&#xA;        '''&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, trim_galore not only uses my {read} parameter in the output filename but again adds it at the end, only now without the &quot;R&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried solving it in my first rule by applying a python .strip('R') on my {read} wildcard, but this does not seem to work. How can I correctly get '1' and '2' from 'R1' and 'R2' (which are my {read} wildcards)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Extra bonus points for a nice output.trim_galore.something as input for my complexity_20mer_counter rule, which does not work because the output is a list (it needs 2 fastq files at the same time), but my complexity_20mer_counter rule &lt;em&gt;can&lt;/em&gt; be run in parallel again.)&lt;/p&gt;&#xA;" OwnerUserId="3964" LastActivityDate="2019-02-02T12:25:50.770" Title="Python string editing in Snakemake" Tags="|python|snakemake|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6912" PostTypeId="2" ParentId="6910" CreationDate="2019-01-31T11:33:23.457" Score="3" Body="&lt;p&gt;Three reasons for Illumina:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Much better for a large number of samples (easily handle 96 samples).&lt;/li&gt;&#xA;&lt;li&gt;SNP calling is much better - much greater depth&lt;/li&gt;&#xA;&lt;li&gt;Hardware costs, an Illumina MiSeq machine is cheap&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;PacBio SNP calling has been improved towards the standard of Illumina by DNA modification to the inputs to be sequenced but it depends on the application.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3694" LastEditDate="2022-12-17T16:24:55.147" LastActivityDate="2022-12-17T16:24:55.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6913" PostTypeId="2" ParentId="6910" CreationDate="2019-01-31T12:37:02.290" Score="5" Body="&lt;p&gt;Many analyses performed on Illumina machines these days require large numbers of reads. For example, most analyses in ChIP-seq, RNA-seq, ATAC-seq etc, need 10s or even 100s of millions of reads for the statistics to work out properly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But this isn't limited to just sequencing as an assay experiments. High depths are important for things like somatic variant calling as well. Or simply sequencing of medical gene pannels, where you might only have 20 amplicons of a 500bp each, but need to do it on 100s of patients as cheaply as possible. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2019-01-31T12:37:02.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6914" PostTypeId="1" AcceptedAnswerId="6923" CreationDate="2019-01-31T15:17:35.283" Score="9" ViewCount="3991" Body="&lt;p&gt;I have been experimenting a lot lately with Snakemake, I love it. Recently I also switched to using conda (--use-conda) in the way that is advertised. However, I have some issues with it, mostly related to the way we work. I work in a lab that processes data from our sequencer but also external datasets, moreover, multiple people use my pipeline.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, with respect to conda this means: currently I have 5 envs, averaging 1.5 GB per env, this takes about 20 min to set up and it's quite some data to pull down. Especially since it is pulled down every time someone analyses data (sometime multiple times a day). This feels like quite a waste of resources. Also, I fix some package versions in the yaml files, but certainly not all (this may lead to issues when repos get reshuffled I read), so I also can't guarantee the exact environment long term (say over years, dependencies may change/update).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm considering switching to a central conda env which I version and of which I export lists of all packages installed for reproducibility reasons. This will then contain all tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, is there a nice way to do this? I had issues with putting a &quot;source activate&quot; line in every shell command (and how to do this for Python code under &quot;run&quot; anyway) before, see this: &lt;a href=&quot;https://stackoverflow.com/questions/54024862/snakemake-conda-env-parameter-is-not-taken-from-config-yaml-file&quot;&gt;https://stackoverflow.com/questions/54024862/snakemake-conda-env-parameter-is-not-taken-from-config-yaml-file&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, is there a way, also when using a qsub cluster, to always use a specific conda env, for everything you do?&lt;/p&gt;&#xA;" OwnerUserId="3964" LastActivityDate="2020-04-08T06:45:03.903" Title="Running Snakemake in one single conda env" Tags="|python|snakemake|bioconda|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6915" PostTypeId="2" ParentId="6910" CreationDate="2019-01-31T16:12:46.933" Score="16" Body="&lt;p&gt;There are &lt;strong&gt;so&lt;/strong&gt; many reasons why one might want to prefer Illumina over PacBio (also note that it's a false dichotomy, at least Oxford Nanopore is a competitive sequencing platform):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The first (IMHO and the most common reason) is still the cost of both sequencing and the instruments. &lt;a href=&quot;http://allseq.com/knowledge-bank/sequencing-platforms/illumina/&quot; rel=&quot;noreferrer&quot;&gt;Illumina can sequence a Gbp of data&lt;/a&gt; for \$7 - \$93. PacBio sequencing is according &lt;a href=&quot;http://allseq.com/knowledge-bank/sequencing-platforms/pacific-biosciences/&quot; rel=&quot;noreferrer&quot;&gt;the same webpage&lt;/a&gt; \$115 per Gbp, however at our sequencing center it's ~$200. Though ONT might have already a cheaper solution. Edit, I just found &lt;a href=&quot;https://docs.google.com/spreadsheets/d/1GMMfhyLK0-q8XkIo3YxlWaZA5vVMuhU1kg41g4xLkXc/edit?usp=sharing&quot; rel=&quot;noreferrer&quot;&gt;a google sheat with prices&lt;/a&gt; that seems to be frequently updated, seems that the ratio still holds Illumina short reads ~10x cheaper than PacBio.&lt;/li&gt;&#xA;&lt;li&gt;RNA-seq (i.e. analysis of a gene expression) is not possible with PacBio due to &lt;a href=&quot;https://www.pacb.com/proceedings/getting-the-most-out-of-your-pacbio-libraries-with-size-selection/&quot; rel=&quot;noreferrer&quot;&gt;preferential sequencing of smaller fragments&lt;/a&gt;; shorter genes would always be shown to be more expressed. To be clear, it's possible to sequence RNA with PacBio (the keyword is iso-seq), but the analysis of gene expression is problematic.&lt;/li&gt;&#xA;&lt;li&gt;It's way easier to extract fragmented DNA (concerns small non-model organisms; although recently &lt;a href=&quot;https://www.mdpi.com/2073-4425/10/1/62&quot; rel=&quot;noreferrer&quot;&gt;a single mosquito was sequenced&lt;/a&gt;, so we can expect a further improvement)&lt;/li&gt;&#xA;&lt;li&gt;other sequencing techniques as RAD-seq that allow genotyping with very little effort and cost, I have never seen anybody even considering using long reads for such genotyping&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/schatzlab/genomescope&quot; rel=&quot;noreferrer&quot;&gt;Genome profiling&lt;/a&gt; (assembly-less genome evaluation) based on kmer spectra analysis is not possible with PacBio data due to higher error rates. Conflict of interest: I am a developer of one of the tools for genome profiling (&lt;a href=&quot;https://github.com/KamilSJaron/smudgeplot&quot; rel=&quot;noreferrer&quot;&gt;smudgeplot&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;DNA in Formalin-Fixed Paraffin-Embedded samples is fragmented anyway (~300bp), therefore, there is no point in sequencing them with more expensive long-read technology (contributed by &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/3564/mrotten&quot;&gt;@mRoten&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;I bet there will be a plenty other applications I am not aware of&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2019-04-09T22:29:03.913" LastActivityDate="2019-04-09T22:29:03.913" CommentCount="2" CommunityOwnedDate="2019-03-19T09:06:00.370" ContentLicense="CC BY-SA 4.0" />
  <row Id="6917" PostTypeId="1" CreationDate="2019-01-31T21:16:25.637" Score="3" ViewCount="391" Body="&lt;p&gt;I downloaded the H. sapiens, NCBI GRCh38 files from Bowtie's website. After unzipping, there are 6 files, 4 that end in set 1.ebwt, set2.ebwt, set3.ebwt, and set4.ebwt and two that end in set.rev.1.ebwt and set.rev.2.ebwt. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am unable to find any explanation for what these files are either online.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which one do I use for bowtie2 mapping?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4050" LastEditorUserId="964" LastEditDate="2019-01-31T22:03:25.633" LastActivityDate="2019-01-31T22:03:25.633" Title="Seeking explanation of the hg38 files downloaded from bowtie 2 website" Tags="|sequence-alignment|genome|human|bowtie2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6918" PostTypeId="2" ParentId="6917" CreationDate="2019-01-31T22:01:43.393" Score="3" Body="&lt;p&gt;Bowtie is providing you with the index files for hg38.  This are the result of the &lt;a href=&quot;http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml#the-bowtie2-build-indexer&quot; rel=&quot;nofollow noreferrer&quot;&gt;bowtie2-build indexer&lt;/a&gt;.  However you have downloaded the indexes for Bowtie1 not bowtie2&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Bowtie 2’s .bt2 index format is different from Bowtie 1’s .ebwt&#xA;  format, and they are not compatible with each other.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Then according to the &lt;a href=&quot;http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml#command-line&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt; you pass the basename of the index for the reference genome. So this should be the name prior to the extension, .1.bt2 for Bowtie2.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bowtie2 [options]* -x &amp;lt;bt2-idx&amp;gt; {-1 &amp;lt;m1&amp;gt; -2 &amp;lt;m2&amp;gt; | -U &amp;lt;r&amp;gt; | --interleaved &amp;lt;i&amp;gt;} -S [&amp;lt;sam&amp;gt;]&#xA;Main arguments&#xA;-x &amp;lt;bt2-idx&amp;gt;&#xA;The basename of the index for the reference genome. The basename is the name of any of the index files up to but not including the final .1.bt2 / .rev.1.bt2 / etc. bowtie2 looks for the specified index first in the current directory, then in the directory specified in the BOWTIE2_INDEXES environment variable.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="964" LastActivityDate="2019-01-31T22:01:43.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6919" PostTypeId="2" ParentId="4376" CreationDate="2019-02-01T00:17:57.877" Score="1" Body="&lt;p&gt;When I have a moment I will fill in the blanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You must have an RNA virus (nothing else will mutate that fast). The basal group and root is the USA. It estimates the time to that basal event is 32.3 (likely years).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;frequencies&quot; frequency of ACGT where the missing one will be 0.25. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its a simple model of constant population size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ESS is the most important parameter to ensure convergence of the Bayesian simulation . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Likelihood is the second most important parameter. I'll edit again shortly.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I'll fill in one more blank and summarise ... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Kappa&lt;/strong&gt; ...&#xA;This will be the transition transversion ratio in HKY. I suspect it also will be transition transversion ratio in Kimuras 3-parameter model, this is because ... the transversions are further split into two mutation groups, ie. purine to pyrimidine has 2 further combinations and you end up with 2 ratios.  that makes sense, transition/transversion (kappa 1), transition/transversion 1 (kappa 2), transition/transversion2 (kappa 3) .... &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;CP1.kappa 18.716&lt;/li&gt;&#xA;&lt;li&gt;CP2.kappa 17.16&lt;/li&gt;&#xA;&lt;li&gt;CP3.kappa 22.796&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;CP2 and CP3 are either side of CP1 .. which is what you would expect on my interpretation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are very high values by the way, essentially there are very few transversions in this data, so ... I dunno. You need to check the data, if you see lots (and lots) of A-G and T-C mutations, then thats what I believe it is saying.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Briefly posterior and prior are the fundamental method of a Bayesian calculations, the output is probability expressed as a log, so the more negative the higher the probability (the prior looks like a random tree, or a reasonable tree with really rubbish parameters, likely the former).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&#xA;Essentially I am going to call it a day here. The real issue is that the output summarise the entire modern theory in phylogenetics, so its a very long response required to address all of the parameters and I honestly don't think it helps your analysis. What I am clear about is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;* You've omitted a gamma distribution (I think Tracer would summarise this) *&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my advice is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ESS MUST be homogeneous between runs - big trouble if not&lt;/li&gt;&#xA;&lt;li&gt;gamma distribution MUST be run&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Beast can be tricky getting gamma to converge via ESS, just a word of warning. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't see the full robustness, because its absent... However, from the output, I think its pretty clear, that you (well me) can more or less see the model you ran. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-03-04T12:17:04.233" LastActivityDate="2019-03-04T12:17:04.233" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6920" PostTypeId="2" ParentId="6914" CreationDate="2019-02-01T08:21:12.453" Score="5" Body="&lt;p&gt;A good practice is to provide &lt;code&gt;yml&lt;/code&gt; file along with your &lt;code&gt;Snakefile&lt;/code&gt;. This file should contain all the enviroment definitions that are needed to run your pipeline. The minimum content of this file would be to install &lt;code&gt;snakemake&lt;/code&gt; itself. Any other program used in your pipeline should be included as well. For reproducibility version numbers must be used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, if someone would like to use your pipeline, (s)he first creates a new enviroment based on your definition:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ conda env create -f my_pipeline.yml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Afterwards activate this environment and run the pipeline:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ conda activate my_pipeline&#xA;$&lt;/span&gt; snakemake&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;conda&lt;/code&gt; directive in your &lt;code&gt;Snakefile&lt;/code&gt; is only neccessary if you need to use program which dependencies cannot be resolved within the main environment. If you start snakemake with the &lt;code&gt;--use-conda&lt;/code&gt; parameter, snakemake will create a new environment based on the provided definition file and activate it for the rule.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Especially since it is pulled down every time someone analyses data (sometime multiple times a day).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The environment creation is only done once. If you run the pipeline a second time &lt;code&gt;snakemake&lt;/code&gt; can activate the environment diretcly without downloading and installing something again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://www.biostars.org/p/335903/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; I wrote some time ago on biostars about &lt;code&gt;snakemake&lt;/code&gt; and &lt;code&gt;conda&lt;/code&gt; might be useful for you.&lt;/p&gt;&#xA;" OwnerUserId="1310" LastEditorUserId="450" LastEditDate="2019-02-01T08:50:45.590" LastActivityDate="2019-02-01T08:50:45.590" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6921" PostTypeId="2" ParentId="6914" CreationDate="2019-02-01T08:42:23.687" Score="1" Body="&lt;p&gt;Inside your Snakefile you can provide the same &lt;code&gt;conda:&lt;/code&gt; directive to each rule. If you install your pipeline in a central location and have everyone use the same conda when running it (hint: use a wrapper script that sets the path to conda) then everyone will be using the same environment. This has the added benefit that you can make this readable to everyone, but disallow them to make any modifications (so users can't mess things up).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tack we've taken is to have a basic conda env with snakemake and little else and then separate envs for various pipelines (and versions thereof if software versions in your YAML files change) or parts thereof, in cases where you have rules using tools with conflicting dependencies. This ends up being fairly convenient for more complicated pipelines, though if you can get away with just a single environment then finswimmer's answer is simpler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that all of this should work with any scheduler, since snakemake will insert the environment activation stuff in the script it submits to your scheduler.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an aside, this is exactly how &lt;a href=&quot;https://snakepipes.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;snakePipes&lt;/a&gt; works.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-02-01T08:42:23.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6922" PostTypeId="2" ParentId="6906" CreationDate="2019-02-01T10:39:11.890" Score="2" Body="&lt;p&gt;You could count the number of transitions between an A and a 'not A', and divide this number by the total number of As.  You might also need to scale by the sequence length, if you have different length sequences that you want to compare.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: also, there might be a correction to make if one or both ends of the sequence are an A&lt;/p&gt;&#xA;" OwnerUserId="3828" LastEditorUserId="3828" LastEditDate="2019-02-01T10:49:40.930" LastActivityDate="2019-02-01T10:49:40.930" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6923" PostTypeId="2" ParentId="6914" CreationDate="2019-02-01T13:07:09.493" Score="5" Body="&lt;p&gt;Using only the &lt;code&gt;--use-conda&lt;/code&gt; directive, snakemake copies environments somewhere in the local &lt;code&gt;.snakemake&lt;/code&gt; directory. This is repeated for each directory you run it in and replicates all environments effectively for each dataset.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can however tell snakemake to use a shared environment location using &lt;code&gt;--conda-prefix&lt;/code&gt; as described &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/executing/cli.html#CONDA&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Specify a directory in which the ‘conda’ and ‘conda-archive’&#xA;  directories are created. These are used to store conda environments&#xA;  and their archives, respectively. If not supplied, the value is set to&#xA;  the ‘.snakemake’ directory relative to the invocation directory. If&#xA;  supplied, the –use-conda flag must also be set. The value may be given&#xA;  as a relative path, which will be extrapolated to the invocation&#xA;  directory, or as an absolute path.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This way you could only download/copy the environment once and share it, e.g. this way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;snakemake --use-conda --conda-prefix /path/to/shared/environment&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1461" LastEditorUserId="1461" LastEditDate="2020-04-08T06:45:03.903" LastActivityDate="2020-04-08T06:45:03.903" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6924" PostTypeId="1" AcceptedAnswerId="6927" CreationDate="2019-02-01T16:52:06.140" Score="0" ViewCount="87" Body="&lt;p&gt;Having trouble building RNAz 2.1 on MacOS. Following instructions at &lt;a href=&quot;https://www.tbi.univie.ac.at/software/RNAz/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tbi.univie.ac.at/software/RNAz/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tar -xzf RNAz-2.1-tar.gz&#xA;cd RNAz-2.1&#xA;./configure&#xA;make&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;make&lt;/code&gt; fails with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/usr/bin/clang++ -o RNAz RNAz.o rnaz_utils.o svm_helper.o zscore.o cmdline.o strand.o svm.o -L../librna -lRNA -lm &#xA;Undefined symbols for architecture x86_64:&#xA;  &quot;_HairpinE&quot;, referenced from:&#xA;      _alifold in libRNA.a(alifold.o)&#xA;      _fill_arrays in libRNA.a(fold.o)&#xA;      _backtrack in libRNA.a(fold.o)&#xA;      _stack_energy in libRNA.a(fold.o)&#xA;  &quot;_LoopEnergy&quot;, referenced from:&#xA;      _alifold in libRNA.a(alifold.o)&#xA;      _fill_arrays in libRNA.a(fold.o)&#xA;      _backtrack in libRNA.a(fold.o)&#xA;      _stack_energy in libRNA.a(fold.o)&#xA;ld: symbol(s) not found for architecture x86_64&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thinking that compiling for 32-bit might work I tried setting &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CXXFLAGS = -m32&#xA;LDFLAGS = -m32&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;without success (same result). Same result with &lt;code&gt;gmake&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Configure&lt;/strong&gt;&#xA;Configure checks for &lt;code&gt;gcc&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;checking for gcc... gcc&#xA;checking whether the C compiler works... yes&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Makefile&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Makefile specifies the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CC = gcc&#xA;CCDEPMODE = depmode=gcc3&#xA;CFLAGS = -g -O2&#xA;CPP = gcc -E&#xA;CPPFLAGS = &#xA;CXX = g++&#xA;CXXDEPMODE = depmode=gcc3&#xA;CXXFLAGS = -g -O2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Compiler&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Apple LLVM version 10.0.0 (clang-1000.11.45.5)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Target: x86_64-apple-darwin18.2.0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thread model: posix&lt;/p&gt;&#xA;&#xA;&lt;p&gt;InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone built this successfully on MacOS? Or perhaps suggestions as to how I might build this successfully? I'm at a loss.&lt;/p&gt;&#xA;" OwnerUserId="2813" LastEditorUserId="2813" LastEditDate="2019-02-01T17:20:26.563" LastActivityDate="2019-02-01T20:42:33.770" Title="Building RNAz on MacOS" Tags="|rna|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6925" PostTypeId="2" ParentId="6924" CreationDate="2019-02-01T17:17:58.450" Score="1" Body="&lt;p&gt;The first thing to try is to perform the install as sudo.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sudo ./configure # definitely&#xA;sudo make ... sudo make install # possibly (?)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The clang++ compiler may be using a path that is not setup in your profile and sudo will overcome this. If sudo works you know what the problem is and its a case of changing permissions and configurations accordingly. There is a good chance this will resolve the matter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other possibilities, it could be the compiler. It looks like the compiler is fine however looking at the config file and the check. The final issue is it could be the program ;-)&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Tried this. Same result. Will continue scratching head&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Try an installation on Linux&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise its usual stuff, namely upgrading Xtools&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-02-01T17:25:47.763" LastActivityDate="2019-02-01T17:25:47.763" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6926" PostTypeId="2" ParentId="6924" CreationDate="2019-02-01T17:31:06.743" Score="2" Body="&lt;p&gt;The other option would be to use RNAz via a docker container instead on installing it directly.  The docker container can be downloaded from &lt;a href=&quot;https://quay.io/repository/biocontainers/rnaz&quot; rel=&quot;nofollow noreferrer&quot;&gt;biocontainers&lt;/a&gt;.  If you are new to docker here is a &lt;a href=&quot;https://docs.docker.com/docker-for-mac/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial for mac&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2019-02-01T17:31:06.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6927" PostTypeId="2" ParentId="6924" CreationDate="2019-02-01T20:42:33.770" Score="3" Body="&lt;p&gt;Compile with gcc or remove &lt;code&gt;inline&lt;/code&gt; from &lt;code&gt;librna/fold.c:776&lt;/code&gt; and &lt;code&gt;librna/fold.c:846&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2019-02-01T20:42:33.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6928" PostTypeId="2" ParentId="3377" CreationDate="2019-02-02T03:06:29.507" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cDkJT.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cDkJT.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There you go, its Microcystis aeruginosa aspartate/glutamate racemase. Easy does it. If you want to know how you can do this in 15 seconds let me know.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Its one of the new features of NCBI's Blast&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Go to Blast here, &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blast.ncbi.nlm.nih.gov/Blast.cgi&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Enter your sequence into the box (it doesn't accept PDB codes alone)&lt;/li&gt;&#xA;&lt;li&gt;Enter the protein database - when I first did this calculation I used SwissProt, thinking there would be alot of sequences - I then used &quot;nr&quot;&lt;/li&gt;&#xA;&lt;li&gt;Under the algorithm parameters enter &quot;50&quot; (default is too many)&lt;/li&gt;&#xA;&lt;li&gt;Hit &quot;Blast&quot;&lt;/li&gt;&#xA;&lt;li&gt;Once the search is complete at the top of the page are the hyperlinks:&#xA;&quot;Other reports: Search Summary [Taxonomy reports] [Distance tree of results]&quot;&lt;/li&gt;&#xA;&lt;li&gt;Click on &quot;Distance tree of results&quot;&lt;/li&gt;&#xA;&lt;li&gt;The following page will load automatically, automatically aligning your sequences and producing, in this case a parsimony based tree, but there is also the option of a nj tree (recommended) .... here: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/blast/treeview/treeView.cgi?request=page&amp;amp;blastRID=5A1ZBRU1014&amp;amp;queryID=lcl%7CQuery_210080&amp;amp;entrezLim=&amp;amp;ex=&amp;amp;exl=&amp;amp;exh=&amp;amp;ns=50&amp;amp;screenWidth=1280&amp;amp;screenHeight=800&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/blast/treeview/treeView.cgi?request=page&amp;amp;blastRID=5A1ZBRU1014&amp;amp;queryID=lcl|Query_210080&amp;amp;entrezLim=&amp;amp;ex=&amp;amp;exl=&amp;amp;exh=&amp;amp;ns=50&amp;amp;screenWidth=1280&amp;amp;screenHeight=800&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Click &quot;Tool&quot;, &quot;Download&quot;, &quot;PDF&quot; ...&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="77" LastEditDate="2019-02-03T18:54:45.577" LastActivityDate="2019-02-03T18:54:45.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6929" PostTypeId="2" ParentId="6908" CreationDate="2019-02-02T08:31:45.490" Score="1" Body="&lt;p&gt;The noise is comming from the subset.mask, which is created above in a loop with the number of permutations.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for (r in 1:nperm) { #L90&#xA;....&#xA;        subset.mask[, r] &amp;lt;- as.numeric(c(subset.class1, subset.class2)) # L107&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So by multiplying the random selection of subsets by the expression we get the &quot;noise&quot;. Later on line 251 we get the ratio of noise/signal by dividing the signal from the contrast and the signal by the noise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Those transformation on the middle seem to come from normalizing the signal in order to be able to compare the resampling, but I don't fully understand why they do this. Also note that the R scripts are outdated and no longer maintained, so they can contain errors.&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2019-03-04T09:44:33.767" LastActivityDate="2019-03-04T09:44:33.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6930" PostTypeId="1" AcceptedAnswerId="6996" CreationDate="2019-02-01T20:00:11.757" Score="2" ViewCount="61" Body="&lt;p&gt;I would like to model a biological compartment with &lt;a href=&quot;https://github.com/Vessy/sysBio&quot; rel=&quot;nofollow noreferrer&quot;&gt;sysBio R package&lt;/a&gt;. It is fine but I'm having a difficult time with modeling the change in pH due to acidosis. I tried several things like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;addMAreaction(glu, &quot;pH = null&quot;, &quot;acidosis&quot;, &quot;base excess&quot;)&#xA;addMAreactRate(glu, &quot;acidosis&quot;, &quot;assigned&quot;, &quot;ka*pH&quot;)&#xA;addMAreactRate(glu, &quot;base excess&quot;, &quot;assigned&quot;, &quot;ka*pH*-1&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which caused continuously decreasing pH levels. I'll leave out the intermediate steps but the simplified version of the last code looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(&quot;sysBio&quot;)&#xA;&#xA;glu &amp;lt;- newModel(&quot;glu&quot;)&#xA;addSpecies(glu, &quot;Lactate&quot;, 0)&#xA;addSpecies(glu, &quot;Glucose&quot;, 1000)&#xA;addSpecies(glu, &quot;pH&quot;, 7.4)&#xA;addSpecies(glu, &quot;Acid&quot;, 0)&#xA;&#xA;addMAreaction(glu, &quot;Glucose = Lactate + 0.01*Acid&quot;, &quot;forwardL&quot;, &#xA;              &quot;backwardL&quot;, name = &quot;glycolysis, sort of&quot;)&#xA;&#xA;addMAreactRate(glu, &quot;forwardL&quot;, &quot;assigned&quot;, &quot;kgl*Glucose&quot;)  &#xA;addMAreactRate(glu, &quot;backwardL&quot;, &quot;assigned&quot;, &quot;ka*pH+klg*Lactate&quot;)&#xA;&#xA;addParameters(glu, &quot;kgl&quot;, 0.01)&#xA;addParameters(glu, &quot;klg&quot;, 0.01)&#xA;addParameters(glu, &quot;ka&quot;, 0.05)&#xA;addParameters(glu, &quot;a&quot;, 0.001)&#xA;&#xA;addRule(glu, &quot;pH rule&quot;, &quot;ODEs&quot;, &quot;pH=7.4-a*Acid&quot;)&#xA;&#xA;makeModel(glu)&#xA;simResults &amp;lt;- simulateModel(glu, times = seq(0,10,0.1))&#xA;plotResults(simResults, title=&quot;glu&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the resulting plot is this where pH is increasing forever despite &lt;code&gt;Acid&lt;/code&gt; is at a steady state concentration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qUych.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qUych.jpg&quot; alt=&quot;Plot of the glu.df&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Please note that the abovementioned code is just a simplification. It doesn't even look like aerobic glycolysis where 1 glucose produces  lactate and 2 hydrogen atoms etc.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I intend to include several acid- and base-creating reactions and evaluate the resulting pH. So could someone point me in the right direction about how to model the relationship between pH and acids or bases?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer does not need to be explicit code. I'll appreciate any solid comment on my design errors.&lt;/p&gt;&#xA;" OwnerUserId="4059" OwnerDisplayName="barerd" LastEditorDisplayName="user100" LastEditDate="2019-02-04T11:35:08.667" LastActivityDate="2019-02-08T22:26:03.303" Title="How to model the effect of acid/base on pH" Tags="|r|modelling|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6931" PostTypeId="2" ParentId="6911" CreationDate="2019-02-02T12:25:50.770" Score="2" Body="&lt;p&gt;It seems that it is not possible to apply &lt;code&gt;strip()&lt;/code&gt; directly on the wildcard placeholders. Luckily it is possible to executed functions in the &lt;code&gt;input&lt;/code&gt; directive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So try changing your input of the &lt;code&gt;complexity_20mer_counter&lt;/code&gt; rule to this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;lambda wc: os.path.join(fastq_trimmed_dir, wc.sample + '_' + wc.read + fastq_extension.split('.')[0] + '_val_' + wc.read.strip('R') + '.fq.gz')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The wildcard object is passed to our anonymous function. So have access to the wildcards value via &lt;code&gt;wc.sample&lt;/code&gt; and &lt;code&gt;wc.read&lt;/code&gt;, which are now handled as normal strings and we can apply &lt;code&gt;strip&lt;/code&gt; on them.&lt;/p&gt;&#xA;" OwnerUserId="1310" LastActivityDate="2019-02-02T12:25:50.770" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6932" PostTypeId="2" ParentId="956" CreationDate="2019-02-03T02:49:28.850" Score="4" Body="&lt;p&gt;I maintain several Python libraries for interacting with flow cytometry data (FCS files). The most recent of which, &lt;a href=&quot;https://github.com/whitews/flowkit&quot; rel=&quot;nofollow noreferrer&quot;&gt;FlowKit&lt;/a&gt;, provides functionality to export the event data as a NumPy array, Pandas DataFrame, or CSV text. Using the &lt;code&gt;h5py&lt;/code&gt; library, &lt;a href=&quot;https://stackoverflow.com/questions/20928136/input-and-output-numpy-arrays-to-h5py&quot;&gt;it's trivial to export as an HDF5 file&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4057" LastActivityDate="2019-02-03T02:49:28.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6933" PostTypeId="2" ParentId="6906" CreationDate="2019-02-03T05:24:14.000" Score="0" Body="&lt;p&gt;To define the base density score, or clustering score, of a sequence &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; we can think of each position in the sequence as being part of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Markov_chain&quot; rel=&quot;nofollow noreferrer&quot;&gt;Markov process&lt;/a&gt;. For the sake of symmetry around a single base we consider odd-order Markov models from 3-onward where we ask what is the probability that we see the character at &lt;span class=&quot;math-container&quot;&gt;$S_i$&lt;/span&gt; in the &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; bases from &lt;span class=&quot;math-container&quot;&gt;$[S_{i-j}, S_{i+j}]$&lt;/span&gt;. For each column we have &lt;code&gt;length(S) -1&lt;/code&gt;, &lt;span class=&quot;math-container&quot;&gt;$|S|-1$&lt;/span&gt;, possible probabilities to calculate per window. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can't look left or right of the edges of the sequences so the probabilities are skewed, but we can get a rough idea of what is going on.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For each base position we calculate a clustering score of &lt;span class=&quot;math-container&quot;&gt;$ \frac{1}{w}\Sigma_{j=1}^{j=w} -log_2 \frac{m(S_{i-j}, S_{i+j})-1 + \frac{1}{|S|}}{|S_{\{i-j, i+j\}}| -1 }$&lt;/span&gt; where:&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; is the window size we can look at. The max is &lt;span class=&quot;math-container&quot;&gt;$|S|-1$&lt;/span&gt; and the min is &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;. This just means that we can calculate a score in the context of the whole sequence or we maybe only care within some smaller window, like 5 bp flanking either side of the base in question.&lt;/li&gt;&#xA;&lt;li&gt;The function &lt;span class=&quot;math-container&quot;&gt;$m(S_{i-j}, S_{i+j})$&lt;/span&gt; is just the number of counts of matching bases in the window &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; on either side of the base &lt;span class=&quot;math-container&quot;&gt;$S_i$&lt;/span&gt;. For example in the sequence &lt;code&gt;gatAgga&lt;/code&gt;, if &lt;span class=&quot;math-container&quot;&gt;$S_i$&lt;/span&gt; is the uppercase &lt;code&gt;A&lt;/code&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; is 3, the function &lt;span class=&quot;math-container&quot;&gt;$m(S_{i-3}, S_{i+3})$&lt;/span&gt; returns 3, since there is the &lt;code&gt;A&lt;/code&gt; base at &lt;span class=&quot;math-container&quot;&gt;$S_i$&lt;/span&gt; and the two additional &lt;code&gt;a&lt;/code&gt; bases at &lt;span class=&quot;math-container&quot;&gt;$i-2$&lt;/span&gt; and at &lt;span class=&quot;math-container&quot;&gt;$i+3$&lt;/span&gt;. We have to subtract one from this number because we don't want to factor in the matching base at &lt;span class=&quot;math-container&quot;&gt;$S_i$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;We then add &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{|S|}$&lt;/span&gt; to that number. This is a pseudocount that we need to add to prevent from taking the log of zero. This number is pretty arbitrary but I chose &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{|S|}$&lt;/span&gt; so that non-clusting regions' scores aren't too high.&lt;/li&gt;&#xA;&lt;li&gt;We then divide that numerator by the length of &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; from the little window that we just looked at minus one, &lt;span class=&quot;math-container&quot;&gt;$|S_{\{i-j, i+j\}}| -1 $&lt;/span&gt;. For example in the sequence &lt;code&gt;gatAgga&lt;/code&gt; from above we only want to consider the bases &lt;code&gt;gat gga&lt;/code&gt;. This is 6 bases.&lt;/li&gt;&#xA;&lt;li&gt;We then take the negative log to convert the proportional scale to a log scale.&lt;/li&gt;&#xA;&lt;li&gt;The summation just says that we do this for every symmetrical window size around the base in question then sum them up. For example we would calculate the ratio for &lt;code&gt;tAg&lt;/code&gt;, &lt;code&gt;atAgg&lt;/code&gt;, and &lt;code&gt;gatAgga&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{w}$&lt;/span&gt; just gives the average log clustering score of all the windows that we looked at. One nice thing about this is that the impact of each increasing window size has less effect on the ratio, so there is a sort of convergence that happens.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Implemented in python:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;from math import log as mlog&#xA;s = &quot;GGATAGGACATTTTTTTTTTGAGCTTTTATTTTCCGA&quot;&#xA;assert len(s) &amp;gt; 0&#xA;pseudocount = 1/len(s)&#xA;#maxdepth = 5&#xA;maxdepth = len(s)&#xA;score = []&#xA;for i in range(len(s)): # iterate positions&#xA;    temp = []&#xA;    for j in range(1, maxdepth): # iterate depth&#xA;        rmin = max(i-j, 0)&#xA;        rmax = i+j+1&#xA;        num_matches_min_one = sum([1 for k in s[rmin:rmax] if k == s[i]])-1&#xA;        len_s_min_one = len(s[rmin:rmax]) -1&#xA;        sim = abs(-1 * mlog((num_matches_min_one/len_s_min_one) + pseudocount, 2))&#xA;        #print(&quot;i={}, j={}, rmin={}, rmax={}, sim={}&quot;.format(i, j, rmin, rmax, sim))&#xA;        temp.append(sim)&#xA;    print(&quot;{0:d}\t{1}\t&quot;.format(i, s[i]), end='\t')&#xA;    score.append(sum(temp)/len(temp))&#xA;    print(&quot;{0:.4f}&quot;.format(score[-1]))&#xA;&#xA;import matplotlib.pyplot as plt&#xA;fig, ax = plt.subplots()&#xA;x = [i+1 for i in range(len(s))]&#xA;plt.bar(x, score)&#xA;plt.xticks(x, [k for k in s])&#xA;ax.set_yscale(&quot;log&quot;, nonposy='clip')&#xA;plt.xlabel('sequence')&#xA;plt.ylabel('-log2(cluster-ness)')&#xA;plt.title(&quot;cluster-ness score&quot;)&#xA;plt.savefig('output.png', dpi=150)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This calculates the matrix, prints out a tab-delimited file of the position, base, and log clusting score, then saves a plot of the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the test sequence &lt;code&gt;GGATAGGACATTTTTTTTTTGAGCTTTTATTTTCCGA&lt;/code&gt; and using the whole sequence as context, I got the following plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/C3SdP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/C3SdP.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, the log clustering score is low around the things that are cluster-y and high around things that aren't cluster-y.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; OP requested that the score be scaled from 0 to 1. I instead used the log scale to make visualization easier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Warning 1:&lt;/strong&gt; If you plan on doing this for large sequences, it is really important to limit the &lt;code&gt;maxdepth&lt;/code&gt;, or &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt;, parameter. The time complexity for this algorithm is &lt;span class=&quot;math-container&quot;&gt;$O(|S|*w)$&lt;/span&gt;, so the number of calculations can blow up to &lt;span class=&quot;math-container&quot;&gt;$O(|S|^2)$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Warning 2:&lt;/strong&gt; It is also important to limit the &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; parameter because the edge effects propagate through the entire sequence when &lt;span class=&quot;math-container&quot;&gt;$w=|S|$&lt;/span&gt;. Look at these examples built off of OP's initial suggestions of &lt;code&gt;AAAAACCCCC&lt;/code&gt; and &lt;code&gt;ACACACACAC&lt;/code&gt;. The clustering score never stabilizes when &lt;span class=&quot;math-container&quot;&gt;$w=|S|$&lt;/span&gt;, however it does stabilize away from the edges when &lt;span class=&quot;math-container&quot;&gt;$w=5$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For 50 characters of &lt;code&gt;ACACAC...&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/t8E0M.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/t8E0M.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For 50 characters of &lt;code&gt;[25xA][25xC]&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bPPvA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bPPvA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2019-02-03T15:51:15.817" LastActivityDate="2019-02-03T15:51:15.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6934" PostTypeId="2" ParentId="6695" CreationDate="2019-02-03T07:55:12.913" Score="1" Body="&lt;p&gt;If you can use R, consider using web scraping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See here : &lt;a href=&quot;http://bradleyboehmke.github.io/2015/12/scraping-html-tables.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://bradleyboehmke.github.io/2015/12/scraping-html-tables.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4058" LastActivityDate="2019-02-03T07:55:12.913" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6935" PostTypeId="2" ParentId="6906" CreationDate="2019-02-03T08:19:00.933" Score="0" Body="&lt;p&gt;The Markov chain approach is very interesting. You could deploy a maximum likelihood approach where instead of scoring the genetic distance within a homologue, the algorithm looks at its physically linked neighbour. The genetic distance of the transition point i.e. AAAA then CCCC is deducted from the final genetic distance so that AAAAACCCCCC would be zero. It would be fairly easy to script and use PAML or equivalent where the default phylogeny would be a star tree.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The challenge is to establish a Deep Learning approach. Its not impossible, you just assign pre-determined classes of oligos which fall into the definition you want and use these as your training set.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-02-03T08:19:00.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6936" PostTypeId="1" CreationDate="2019-02-03T20:11:50.260" Score="3" ViewCount="7487" Body="&lt;p&gt;I know this is a somewhat general or vague question, but I’m interested in your opinions. I must build a desktop for general bioinformatics activities with human genomes. I will work with Python and R libraries, I will also need to install denovo sequence assembly software, alignment tools, variant viewers/browsers, sql, among other software. I don’t have much experience customizing hardware that will be dedicated to bioinformatics and genome data analyses. I’m pretty sure I work with linux and/or iOS. What is the recommended RAM, processor speed, HD capacity, graphics cards, etc. I appreciate your opinions on the matter. Thank you. &lt;/p&gt;&#xA;" OwnerUserId="2621" LastEditorUserId="492" LastEditDate="2019-02-04T07:48:50.067" LastActivityDate="2019-02-04T14:04:08.920" Title="Hardware Requirements (specs) for Bioinformatics-dedicated desktop" Tags="|sequence-alignment|ngs|sequence-analysis|" AnswerCount="2" CommentCount="1" ClosedDate="2019-02-04T14:13:11.250" ContentLicense="CC BY-SA 4.0" />
  <row Id="6939" PostTypeId="2" ParentId="6936" CreationDate="2019-02-04T07:59:18.687" Score="4" Body="&lt;p&gt;The RAM requirements for some bioinformatics analyses like assembly can be quite high (in the hundreds of Gigabytes). My recommendation is to get a fast laptop. Something with an i7 quad-core processor, 16 GB of RAM, and 1TB of storage should do. Do your analyses on this laptop. If you run out of RAM or storage, then use a cloud service to get access to bigger machines or buy an external hard drive. Once you have a feeling for what your actual needs are, start looking at a dedicated rack server solution in your local institution's server farm or again, in the cloud. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't mention any analyses that clearly benefit from a GPU, but as you get more familiar with your analyses, perhaps that will become useful as well.&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2019-02-04T07:59:18.687" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6940" PostTypeId="1" CreationDate="2019-02-04T09:51:07.267" Score="2" ViewCount="66" Body="&lt;p&gt;I am currently looking at microarray data from a bioconductor microarray dataset. Specifically, I have data (a snippet) which looks like the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   GSM794885   GSM794886   GSM794887   GSM794888   GSM794889 &#xA;   -1.67170290 -0.07803822  0.99990560 -1.94692850 -1.78671360 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am wondering how to interpret the values in this microarray dataset. Is it generally the case that larger values mean a more significant effect?&lt;/p&gt;&#xA;" OwnerUserId="4062" LastActivityDate="2019-03-08T21:02:05.410" Title="How can I interpret gene expression data from Bioconductor packages?" Tags="|rna-seq|gene|rna|gene-expression|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6941" PostTypeId="1" AcceptedAnswerId="6943" CreationDate="2019-02-04T11:50:30.977" Score="5" ViewCount="175" Body="&lt;p&gt;I am searching various sources about phylogenetics. I saw some materials about &lt;a href=&quot;https://en.wikipedia.org/wiki/Perfect_phylogeny&quot; rel=&quot;nofollow noreferrer&quot;&gt;perfect phylogeny&lt;/a&gt; and also phylogenies acquired from &lt;a href=&quot;https://en.wikipedia.org/wiki/Maximum_parsimony_(phylogenetics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;maximum parsimony&lt;/a&gt; constraint. They seem very similar to me. Are they the same?&lt;/p&gt;&#xA;" OwnerUserId="3165" LastEditorUserId="3848" LastEditDate="2020-11-20T14:27:35.870" LastActivityDate="2023-08-15T11:27:34.123" Title="Perfect Phylogeny vs Maximum parsimony" Tags="|phylogenetics|ngs|phylogeny|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6943" PostTypeId="2" ParentId="6941" CreationDate="2019-02-04T12:34:35.850" Score="3" Body="&lt;p&gt;No. A perfect phylogeny is one such that all characters evolve on the tree with no homoplasy, i.e. for a binary character, changes occur once from 0 -&amp;gt; 1 but never from 1 -&amp;gt; 0. Maximum parsimony inference &lt;em&gt;may&lt;/em&gt; produce a perfect phylogeny, but typically for real datasets some degree of homoplasy is required to explain character patterns.&lt;/p&gt;&#xA;" OwnerUserId="2841" LastEditorUserId="2841" LastEditDate="2023-08-15T11:27:34.123" LastActivityDate="2023-08-15T11:27:34.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6944" PostTypeId="2" ParentId="6941" CreationDate="2019-02-04T13:29:21.850" Score="2" Body="&lt;p&gt;Long discussion that took many years to resolve. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;MP doesn't account for back mutation, which is a HUGE problem for nucleotide data because in theory 1:4 mutations is a back mutation. Maximum likelihood has resumed is crown here, however Beast deployment of a Bayesian calculation is also very widely, particularly for molecular dating. ML uses  a reversible matrix A &amp;lt;-&gt; T, whilst Beast uses a directional matrix A-&gt;T and will score T-&gt;A separately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Happy to discuss the approaches and modern interpretations at length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The model which accounts for back-mutation is the Jukes-Cantor correction (JC correction), this is a basic model present in every phylogenetic algorithm except p-distances). Essentially JC extrapolates back-calculation for the true number of mutations against the observed number of mutations against divergence time. However, when the P-distance (uncorrected observed distance) exceeds 0.75 for nucleotides the model is not viable. Basically, nucleotide divergence &gt;0.75 is saturated and the phylogenetic information is essentially random.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-02-04T16:31:03.863" LastActivityDate="2019-02-04T16:31:03.863" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6945" PostTypeId="2" ParentId="6936" CreationDate="2019-02-04T13:43:36.367" Score="2" Body="&lt;p&gt;If you deal with genomics NGS data then heavy RAM will be essential. I don't think you will get this from a reasonable priced laptop.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using a Deep Learning based algorhithms or Bayesian or maximum likelihood heavy duty processors are needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would go for an allround machine. &#xA;Two hard drives:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;0.5 to 1 Tb SSD (I use 1TB),&lt;/li&gt;&#xA;&lt;li&gt;2 to 4 Tb Stata data storage,&lt;/li&gt;&#xA;&lt;li&gt;4-core (light) or 8-core, i7 (10-core is common now),&lt;/li&gt;&#xA;&lt;li&gt;16GB-32GB RAM.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Important&lt;/strong&gt;: For your purpose if you are doing de novo NGS assemblies use:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ALOT of RAM.&lt;/li&gt;&#xA;&lt;li&gt;Sufficient harddrive - make sure this is backed up&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you want evidence why RAM is important simply look at the threads here and on Stackexchange which read ... de novo NGS assembly PC has stopped working?(!). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following threads give an insight into the sort of problems RAM creates&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/6704/python-does-not-quit-when-the-input-file-is-too-big/6715#6715&quot;&gt;here&lt;/a&gt;:&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/questions/53524163/random-forest-survival-analysis-randomforestsrc-in-r-crashes/53524842#53524842&quot;&gt;a similar RAM problem is here, but not related to NGS is here&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The following looking like a particularly bad case:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/questions/54508739/how-to-fix-bowtie2-died-with-signal-9-kill-error&quot;&gt;grasshopper NGS&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-02-04T14:04:08.920" LastActivityDate="2019-02-04T14:04:08.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6946" PostTypeId="1" CreationDate="2019-02-04T14:33:04.007" Score="2" ViewCount="1544" Body="&lt;p&gt;I am trying to installing Vep on Linux but an error is permanent&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[fi1d18@cyan01 ensembl-vep-release-95]$ perl INSTALL.pl&#xA;&#xA;Hello! This installer is configured to install v95 of the Ensembl API for use by the VEP.&#xA;It will not affect any existing installations of the Ensembl API that you may have.&#xA;&#xA;It will also download and install cache files from Ensembl's FTP server.&#xA;&#xA;Checking for installed versions of the Ensembl API...done&#xA;&#xA;Setting up directories&#xA;Destination directory ./Bio already exists.&#xA;Do you want to overwrite it (if updating VEP this is probably OK) (y/n)? y&#xA; - fetching BioPerl&#xA; - unpacking ./Bio/tmp/release-1-6-924.zip&#xA; - moving files&#xA;Attempting to install Bio::DB::HTS and htslib.&#xA;&#xA;&amp;gt;&amp;gt;&amp;gt; If this fails, try re-running with --NO_HTSLIB&#xA;&#xA; - checking out HTSLib&#xA;fatal: destination path 'htslib' already exists and is not an empty directory.&#xA; - building HTSLIB in ./htslib&#xA;In /home/fi1d18/ensembl-vep/ensembl-vep-release-95/htslib&#xA;gcc -g -Wall -O2 -fPIC -Wno-unused -Wno-unused-result -I.  -c -o cram/cram_io.o cram/cram_io.c&#xA;cram/cram_io.c:61:18: error: lzma.h: No such file or directory&#xA;cram/cram_io.c: In function 'lzma_mem_deflate':&#xA;cram/cram_io.c:679: warning: implicit declaration of function 'lzma_stream_buffer_bound'&#xA;cram/cram_io.c:685: error: 'LZMA_OK' undeclared (first use in this function)&#xA;cram/cram_io.c:685: error: (Each undeclared identifier is reported only once&#xA;cram/cram_io.c:685: error: for each function it appears in.)&#xA;cram/cram_io.c:685: warning: implicit declaration of function 'lzma_easy_buffer_encode'&#xA;cram/cram_io.c:685: error: 'LZMA_CHECK_CRC32' undeclared (first use in this function)&#xA;cram/cram_io.c: In function 'lzma_mem_inflate':&#xA;cram/cram_io.c:695: error: 'lzma_stream' undeclared (first use in this function)&#xA;cram/cram_io.c:695: error: expected ';' before 'strm'&#xA;cram/cram_io.c:701: error: 'LZMA_OK' undeclared (first use in this function)&#xA;cram/cram_io.c:701: warning: implicit declaration of function 'lzma_stream_decoder'&#xA;cram/cram_io.c:701: error: 'strm' undeclared (first use in this function)&#xA;cram/cram_io.c:701: warning: implicit declaration of function 'lzma_easy_decoder_memusage'&#xA;cram/cram_io.c:719: warning: implicit declaration of function 'lzma_code'&#xA;cram/cram_io.c:719: error: 'LZMA_RUN' undeclared (first use in this function)&#xA;cram/cram_io.c:720: error: 'LZMA_STREAM_END' undeclared (first use in this function)&#xA;cram/cram_io.c:732: error: 'LZMA_FINISH' undeclared (first use in this function)&#xA;cram/cram_io.c:743: warning: implicit declaration of function 'lzma_end'&#xA;At top level:&#xA;cc1: warning: unrecognized command line option &quot;-Wno-unused-result&quot;&#xA;make: *** [cram/cram_io.o] Error 1&#xA;Compile didn't complete. No libhts.a library file found at INSTALL.pl line 820.&#xA;[fi1d18@cyan01 ensembl-vep-release-95]$&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[fi1d18@cyan01 ensembl-vep]$ perl INSTALL.pl --NO_HTSLIB&#xA;&#xA;Hello! This installer is configured to install v95 of the Ensembl API for use by the VEP.&#xA;It will not affect any existing installations of the Ensembl API that you may have.&#xA;&#xA;It will also download and install cache files from Ensembl's FTP server.&#xA;&#xA;Checking for installed versions of the Ensembl API...done&#xA;&#xA;Setting up directories&#xA;Destination directory ./Bio already exists.&#xA;Do you want to overwrite it (if updating VEP this is probably OK) (y/n)? y&#xA; - fetching BioPerl&#xA; - unpacking ./Bio/tmp/release-1-6-924.zip&#xA; - moving files&#xA;&#xA;Downloading required Ensembl API files&#xA; - fetching ensembl&#xA; - unpacking ./Bio/tmp/ensembl.zip&#xA; - moving files&#xA; - getting version information&#xA; - fetching ensembl-variation&#xA; - unpacking ./Bio/tmp/ensembl-variation.zip&#xA; - moving files&#xA; - getting version information&#xA; - fetching ensembl-funcgen&#xA; - unpacking ./Bio/tmp/ensembl-funcgen.zip&#xA; - moving files&#xA; - getting version information&#xA; - fetching ensembl-io&#xA; - unpacking ./Bio/tmp/ensembl-io.zip&#xA; - moving files&#xA; - getting version information&#xA;&#xA;Testing VEP installation&#xA;ERROR: Testing VEP script failed with the following error&#xA;Can't locate Time/HiRes.pm in @INC (@INC contains: /home/fi1d18/ensembl-vep/modules /home/fi1d18/ensembl-vep ./Bio /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /home/fi1d18/ensembl-vep/Bio/EnsEMBL/Utils/SqlHelper.pm line 93.&#xA;BEGIN failed--compilation aborted at /home/fi1d18/ensembl-vep/Bio/EnsEMBL/Utils/SqlHelper.pm line 93.&#xA;Compilation failed in require at /home/fi1d18/ensembl-vep/Bio/EnsEMBL/DBSQL/DBConnection.pm line 81.&#xA;BEGIN failed--compilation aborted at /home/fi1d18/ensembl-vep/Bio/EnsEMBL/DBSQL/DBConnection.pm line 81.&#xA;Compilation failed in require at /home/fi1d18/ensembl-vep/Bio/EnsEMBL/DBSQL/DBAdaptor.pm line 69.&#xA;BEGIN failed--compilation aborted at /home/fi1d18/ensembl-vep/Bio/EnsEMBL/DBSQL/DBAdaptor.pm line 69.&#xA;Compilation failed in require at /home/fi1d18/ensembl-vep/Bio/EnsEMBL/Registry.pm line 137.&#xA;BEGIN failed--compilation aborted at /home/fi1d18/ensembl-vep/Bio/EnsEMBL/Registry.pm line 137.&#xA;Compilation failed in require at /home/fi1d18/ensembl-vep/modules/Bio/EnsEMBL/VEP/BaseVEP.pm line 58.&#xA;BEGIN failed--compilation aborted at /home/fi1d18/ensembl-vep/modules/Bio/EnsEMBL/VEP/BaseVEP.pm line 58.&#xA;Compilation failed in require at (eval 7) line 3.&#xA;        ...propagated at /usr/share/perl5/base.pm line 94.&#xA;BEGIN failed--compilation aborted at /home/fi1d18/ensembl-vep/modules/Bio/EnsEMBL/VEP/BaseRunner.pm line 56.&#xA;Compilation failed in require at (eval 6) line 3.&#xA;        ...propagated at /usr/share/perl5/base.pm line 94.&#xA;BEGIN failed--compilation aborted at /home/fi1d18/ensembl-vep/modules/Bio/EnsEMBL/VEP/Runner.pm line 71.&#xA;Compilation failed in require at ./vep line 20.&#xA;BEGIN failed--compilation aborted at ./vep line 20.&#xA;&#xA;[fi1d18@cyan01 ensembl-vep]$&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="96" LastEditDate="2019-02-04T15:50:24.530" LastActivityDate="2019-02-04T15:50:24.530" Title="Error in installing Vep" Tags="|ensembl|linux|perl|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6947" PostTypeId="2" ParentId="6946" CreationDate="2019-02-04T14:35:34.210" Score="1" Body="&lt;p&gt;What you can try is stated out in the message:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If this fails, try re-running with --NO_HTSLIB&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1310" LastActivityDate="2019-02-04T14:35:34.210" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6949" PostTypeId="1" AcceptedAnswerId="6950" CreationDate="2019-02-04T15:08:01.203" Score="3" ViewCount="187" Body="&lt;p&gt;I am trying to download a file from the following repository: &lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR7276474&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR7276474&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, there are several layers to the webpage. For example, clicking on the download tab doesn't change the URL and, the link is not a 'download link' per ce - where simply clicking the link automates a download. I have tried some of the answers on the forum, where they have advised using quotes and the operation: &#xA;&lt;code&gt;wget &quot;url/?target=link&quot;&lt;/code&gt;. This however does not work in the following instance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, I want to download the file labeled 'P1TLH.bam', in the download tab of the link provided:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Fv2nN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Fv2nN.png&quot; alt=&quot;screenshot showing the link&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4001" OwnerDisplayName="h3ab74" LastEditorUserId="298" LastEditDate="2019-02-04T15:40:09.683" LastActivityDate="2019-02-06T16:05:29.083" Title="wget for links inside html pages" Tags="|phylogenetics|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6950" PostTypeId="2" ParentId="6949" CreationDate="2019-02-04T15:48:04.753" Score="1" Body="&lt;p&gt;If you want to download a file, you need to use the link &lt;em&gt;to that file&lt;/em&gt;. Your original attempt, &lt;code&gt;wget  https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR7276474&lt;/code&gt; wouldn't work since that's a link to the trace page of the relevant run. If you want to download something else, just right click on the link (the one in the screenshot in your question), copy the URL and use that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wget https://sra-download.ncbi.nlm.nih.gov/traces/sra64/SRZ/007276/SRR7276474/P1TLH.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-02-04T15:48:04.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6952" PostTypeId="1" AcceptedAnswerId="6955" CreationDate="2019-02-04T16:31:01.857" Score="1" ViewCount="104" Body="&lt;p&gt;I am trying to run cn.MOPS to detect CNVs in multiple bam files. Following &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/vignettes/cn.mops/inst/doc/cn.mops.R&quot; rel=&quot;nofollow noreferrer&quot;&gt;the example code&lt;/a&gt;, I have run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;source(&quot;https://bioconductor.org/biocLite.R&quot;)&#xA;biocLite(&quot;cn.mops&quot;)&#xA;options(width=75)&#xA;set.seed(0)&#xA;library(cn.mops)&#xA;library(Biobase)&#xA;library(GenomicRanges)&#xA;library(GenomeInfoDb)&#xA;cn.mopsVersion &amp;lt;- packageDescription(&quot;cn.mops&quot;)$Version&#xA;library(cn.mops)&#xA;BAMFiles &amp;lt;- c('bam1.bam', 'bam2.bam', ..., 'bamN.bam')&#xA;bamDataRanges &amp;lt;- getReadCountsFromBAM(BAMFiles)&#xA;&#xA;PLEASE BE PATIENT... this might take a while. Consider using the parallel version of this function&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since this is indeed taking a while, I'd like to follow the tool's suggestion and use &quot;the parallel version of this function&quot;. But how do I do that? I don't see any mention of &quot;parallel&quot; in &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/vignettes/cn.mops/inst/doc/cn.mops.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the manual&lt;/a&gt;. Is there some standard Bioconductor/R trick to make a function parallel? Am I missing something in the manual? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I run &lt;code&gt;bamDataRanges&lt;/code&gt; parallelized? &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-02-05T11:55:14.933" Title="Parallel version of cn.mops' bamDataRanges" Tags="|cnv|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6953" PostTypeId="2" ParentId="6949" CreationDate="2019-02-04T16:33:32.517" Score="-1" Body="&lt;p&gt;NCBI has its own digger for this purpose. No, my bad the traditional Perl approach of LWP and HTTP::Request::Common to interrogate NCBI, isn't going to work no how much data munging or spider crawling is done, because its just an NGS data dump. ... well its diggable, but it would be easier to dump the data and  interrogate it locally.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-02-06T16:05:29.083" LastActivityDate="2019-02-06T16:05:29.083" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6954" PostTypeId="1" AcceptedAnswerId="6971" CreationDate="2019-02-05T10:33:37.737" Score="5" ViewCount="293" Body="&lt;p&gt;I have a nexus formatted &lt;a href=&quot;http://beast.community/&quot; rel=&quot;noreferrer&quot;&gt;BEAST&lt;/a&gt; output containing 20,000 phylogenetic trees of seven taxa. Is there any way to get the percentage of each unique phylogenetic tree contained in this output?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already made an unsuccessful attempt with &lt;a href=&quot;https://stackoverflow.com/questions/54523725/compare-list-of-complex-structures&quot;&gt;R&lt;/a&gt;.  &lt;/p&gt;&#xA;" OwnerUserId="4067" OwnerDisplayName="Justine Vandendorpe" LastEditorUserId="3848" LastEditDate="2020-07-31T16:09:31.103" LastActivityDate="2020-07-31T16:09:31.103" Title="Calculate the percentage of each unique phylogenetic tree in a BEAST output" Tags="|r|sequence-alignment|phylogenetics|phylogeny|beast|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6955" PostTypeId="2" ParentId="6952" CreationDate="2019-02-05T11:55:14.933" Score="0" Body="&lt;p&gt;This is an option for the function documented in its help page. Just run &lt;code&gt;?bamDataRanges&lt;/code&gt; and you will see:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Usage:&#xA;&#xA;     getReadCountsFromBAM(BAMFiles, sampleNames, refSeqNames, WL = 25000,&#xA;       parallel = 0, ...)&#xA;&#xA;[...]&#xA;&#xA;parallel: The number of parallel processes to be used for this&#xA;          function. Default=0.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So all you need to run with 10 cores is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bamDataRanges &amp;lt;- getReadCountsFromBAM(BAMFiles, parallel=10)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-02-05T11:55:14.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6956" PostTypeId="2" ParentId="6954" CreationDate="2019-02-05T17:08:41.107" Score="2" Body="&lt;p&gt;You require a consensus phylogeny. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is available in RAxML, PAUP, the old Phylip package and possibly the stats companion to Beast, Tracer. I'd need to dig up the code for RAxML and PAUP. The easiest is Consense &lt;a href=&quot;http://evolution.genetics.washington.edu/phylip/doc/consense.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; because you just download it and input the sequence. NOTE their is possibly a format difference (RAxML and Consense use Phylip format), I suspect you've got nexus, or Newick format. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beast is likely to have its own consensus formating OR the Beast companion  &quot;Tracer&quot; may have this (I can't remember). Tracer is &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwis-s2llqXgAhWkSRUIHYimDR0QFjAAegQICRAB&amp;amp;url=http%3A%2F%2Ftree.bio.ed.ac.uk%2Fsoftware%2Ftracer%2F&amp;amp;usg=AOvVaw372z_FAnVlQErSu7tFxUF2&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and would be cool if it did consensus phylogeny because there will be no conversion problems (it doesn't use Phylip format). The alternative would be to use PAUP (no conversion problems), which is currently in a state of flux from various stages of freeware and updates. PAUP is contained within the Genious package, which is available as a 1 month free-ware.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have a look and let me know: because if you want to use RAxML or Consense you'll likely need to remind me of the Beast output, if it is what I think it is a format conversion app is needed.  The calculation is easy, its easy to script a conversion program, but I think tree formating will be an issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could use Ape, within R, but as you have gathered there are alot of other solutions. There are more recent, impressive, solutions as stand-alones which I've not used yet.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-02-06T00:37:14.450" LastActivityDate="2019-02-06T00:37:14.450" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6957" PostTypeId="1" CreationDate="2019-02-05T20:50:11.123" Score="2" ViewCount="41" Body="&lt;p&gt;I know that there are different types of biological networks. Protein-protein interaction, gene interaction, metabolic networks, gene expression networks, etc. Each of these networks can be represented as a graph. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Is there a principled way to integrate these graphs into a single large graph? For example, probably most of these networks have nodes that are genes, or gene related things, and this could allow us to match nodes in one network to nodes in another network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Is there a principled way to ask joint questions about these networks?&lt;/p&gt;&#xA;" OwnerUserId="4069" LastActivityDate="2019-02-06T13:06:54.693" Title="General framework for fusing biological networks" Tags="|machine-learning|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6958" PostTypeId="2" ParentId="6957" CreationDate="2019-02-05T21:07:06.460" Score="1" Body="&lt;p&gt;You are talking network statistics and this is a complex area of stats. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For generalised simplicity, Google fusion tables have a simple to use but impressively interactive network tool. This is a great starting point for exploration. Have a go and see what you think. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are looking at a haplotype map, there is a formal solution to this (parsimony network), which is standalone program written some time ago but still useful today. In my area split-decomposition is an important important technique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest question is the number of connecting nodes and Google fusion tables is as good as any routine interactive networking tool. As I say beyond this network stats is a specialised field.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-02-06T00:35:18.370" LastActivityDate="2019-02-06T00:35:18.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6959" PostTypeId="1" CreationDate="2019-02-05T22:23:53.443" Score="2" ViewCount="159" Body="&lt;p&gt;I would like to run the CNVkit workflow on &lt;a href=&quot;https://www.cancergenomicscloud.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cancer Genomics Cloud&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the two workflows that I'd like to use:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://cgc.sbgenomics.com/raw/admin/sbg-public-data/cnvkit-reference/7&quot; rel=&quot;nofollow noreferrer&quot;&gt;CNVkit Reference&lt;/a&gt;, for building a reference from normal samples&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://cgc.sbgenomics.com/raw/admin/sbg-public-data/cnvkit-pipeline-from-baseline-093/10&quot; rel=&quot;nofollow noreferrer&quot;&gt;CNVkit CNV Calling&lt;/a&gt;, for calling CNVs by comparing tumors to the reference&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;don't have parameters that can be set to define in which folder to save the results or which prefix to add to the results. This means that if you're generating multiple references or calling CNV for multiple tumor samples you will end up with results having the same name for each run (tumor).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to modify the CWL workflow, online and through Rabix composer, but I was unable to make it work. I would like to use the published workflow to not code everything from the ground up.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know how to update/use the current CWL workflow in order to avoid having the same filename/folder for all the results?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1157" LastEditDate="2019-09-03T22:21:55.527" LastActivityDate="2021-09-23T02:03:05.260" Title="Save output to a specific folder and/or with a specific prefix in Cancer Genomics Cloud" Tags="|cnv|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6960" PostTypeId="1" AcceptedAnswerId="6962" CreationDate="2019-02-06T09:51:22.560" Score="3" ViewCount="2661" Body="&lt;p&gt;I have 2151 cells, I clustered them by Seurat to 5 clusters. With the code below, I am able to have the number of cells per cluster and per condition:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;number_perCluster&amp;lt;- table(object@meta.data&lt;span class=&quot;math-container&quot;&gt;$conditions, &#xA;                          object@meta.data$&lt;/span&gt;clusterID)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Could I please get a hint on how I could proceed to produce a figure where I say: X% of cells treated with condition &quot;Y&quot; are located in cluster &quot;C&quot; and so forth?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="3968" LastActivityDate="2021-02-01T17:27:33.083" Title="Percentage distribution of cells in all clusters based on their treatment condition?" Tags="|r|scrnaseq|seurat|ggplot2|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6961" PostTypeId="2" ParentId="6957" CreationDate="2019-02-06T13:06:54.693" Score="2" Body="&lt;p&gt;I believe you're referring to something related to multi-layer networks. Here you can find an interesting reading about this kind of networks:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://academic.oup.com/comnet/article/2/3/203/2841130&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://academic.oup.com/comnet/article/2/3/203/2841130&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also you can try with this one:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0370157314002105&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sciencedirect.com/science/article/pii/S0370157314002105&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a matter of fact, I'm not an expert of multi-layer networks but, as far as I can tell, the main problem (as you mentioned) is to find &lt;em&gt;correlations&lt;/em&gt;, or &lt;em&gt;connections&lt;/em&gt;, between the different layers in order to be able to draw some meaningful conclusion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure you can call this a multi-layer network but this can be an example of what you're looking for: suppose you have a protein-protein interaction network. You can enrich your network using biological information. You can associate biological functions, or cellular locations or whatever you can find in GeneOntology (just to make an example of a biological database), as a further layer of information regarding your protein-protein interaction network.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will help you in elucidating the role a node (or more nodes) is playing with respect to its neighbours: let's say five nodes form a clique and you also know they share the same cellular location...then you're able to say something about these nodes (maybe they form a complex?) that has a dual nature: biological and &lt;em&gt;graph-based&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps!&lt;/p&gt;&#xA;" OwnerUserId="3119" LastActivityDate="2019-02-06T13:06:54.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6962" PostTypeId="2" ParentId="6960" CreationDate="2019-02-06T14:35:44.000" Score="4" Body="&lt;p&gt;Here is a solution using &lt;code&gt;dplyr&lt;/code&gt; and &lt;code&gt;ggplot2&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(Seurat) &#xA;library(dplyr)&#xA;library(ggplot2)&#xA;&#xA;meta.data &amp;lt;- pbmc_small[[]]&#xA;&#xA;# create random classifications for the sake of this example&#xA;meta.data$condition &amp;lt;- sample(c('A', 'B', 'C'), nrow(meta.data), replace = TRUE)&#xA;&#xA;counts &amp;lt;- group_by(meta.data, condition, res.1) %&amp;gt;% summarise(count = n())&#xA;&#xA;ggplot(counts, aes(res.1, count, fill = condition)) +&#xA;  geom_bar(stat = 'identity')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LL3B5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LL3B5.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3415" LastEditorUserId="3415" LastEditDate="2021-02-01T17:27:33.083" LastActivityDate="2021-02-01T17:27:33.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6964" PostTypeId="2" ParentId="6940" CreationDate="2019-02-06T20:41:12.657" Score="2" Body="&lt;p&gt;Processed microarray data is commonly represented as log_2(expression). This data transformation is used because the data more closely fit a normal distribution in log space. With such a transformation, negative values represent very small amounts, rather than the absence of something.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Larger values mean that more stuff is being produced (presumably transcripts), but its significance can only be assessed when combined with other data (e.g. through differential expression analysis).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2019-02-06T20:41:12.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6966" PostTypeId="1" CreationDate="2019-02-06T23:18:35.857" Score="-3" ViewCount="144" Body="&lt;p&gt;I have a couple of modified ML methods that I want to test with biological binary data - specifically binary gene expression data - e.g. &lt;a href=&quot;https://eprints.soton.ac.uk/69359/1/Thesis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://eprints.soton.ac.uk/69359/1/Thesis.pdf&lt;/a&gt; that might include a variable that could be used to be the target variable in ML classification. What I mean by binary data is the features are all either 0 or 1, so I expect a large dataset e.g this thesis. I am looking for any suggestions.&lt;/p&gt;&#xA;" OwnerUserId="4078" LastEditorUserId="4079" LastEditDate="2019-02-13T20:45:31.893" LastActivityDate="2019-02-13T20:45:31.893" Title="Recommendation for a binary dataset for Computational Biology open access (as in a dataset with 1 and 0 that can be used to apply ML techniques?" Tags="|machine-learning|public-dataset|" AnswerCount="1" CommentCount="1" ClosedDate="2019-02-15T12:07:08.243" ContentLicense="CC BY-SA 4.0" />
  <row Id="6968" PostTypeId="2" ParentId="6694" CreationDate="2019-02-07T08:38:24.773" Score="0" Body="&lt;p&gt;you might want to try adding the 3'UTR sequence of you tdtomato construct used in your cells. that should do the trick. &lt;/p&gt;&#xA;" OwnerUserId="4081" LastActivityDate="2019-02-07T08:38:24.773" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6969" PostTypeId="2" ParentId="4585" CreationDate="2019-02-07T08:51:06.240" Score="0" Body="&lt;p&gt;GATK's &lt;code&gt;HaplotypeCaller&lt;/code&gt; and &lt;code&gt;freebayes&lt;/code&gt; are two more variant caller that output QUAL values greater than 255. The reason for that is, that each read have its part on calculation that value. This results in the higher the read depth at this position the higher the QUAL value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get something useful out of this value, the usual thing that is done, is to calculate the quality per depth (QD) by just divide the QUAL value by the read depth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note whenever you try a different variant caller, you have to find out whether the QUAL value is read depth dependend or not.&lt;/p&gt;&#xA;" OwnerUserId="1310" LastActivityDate="2019-02-07T08:51:06.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6970" PostTypeId="2" ParentId="6904" CreationDate="2019-02-07T11:08:49.283" Score="1" Body="&lt;p&gt;Having inspected one of my samples (with estimated deamination metric, the proportion of called SNPs consisting of C&gt;T or G&gt;A transitions, of 0.837), this looks like a straightforward filtering procedure. Looking at the minor allele fraction (AF) and the quality score (-10 log10(p-value)) per type of SNP:&lt;a href=&quot;https://i.stack.imgur.com/6WUc0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6WUc0.png&quot; alt=&quot;AF v QUAL per SNP type&quot;&gt;&lt;/a&gt;&#xA;it can be seen that the C&gt;T and G&gt;A transitions likely caused by deamination (and PCRing up the deaminated DNA to get the libraries) all cluster in the bottom right of the plot - low AF and low QUAL, so setting thresholds to the left (higher QUAL) and above (higher AF) the groups will remove these from the called variants.&lt;/p&gt;&#xA;" OwnerUserId="2244" LastActivityDate="2019-02-07T11:08:49.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6971" PostTypeId="2" ParentId="6954" CreationDate="2019-02-07T12:03:07.163" Score="4" Body="&lt;p&gt;I finally managed to do it in &lt;a href=&quot;https://stackoverflow.com/questions/54523725/compare-list-of-complex-structures?noredirect=1#comment95877920_54523725&quot;&gt;R&lt;/a&gt;. Here is my code: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;install.packages('devtools')&#xA;library(devtools)&#xA;install_github('santiagosnchez/rBt')&#xA;library(rBt)&#xA;&#xA;beast_output &amp;lt;- read.annot.beast('beast_output.trees')&#xA;beast_output_rooted &amp;lt;- root.multiPhylo(beast_output, c('taxon_A', 'taxon_B'))&#xA;unique_topologies &amp;lt;- unique.multiPhylo(beast_output_rooted)&#xA;&#xA;count &amp;lt;- function(item, list) {&#xA;  total = 0&#xA;  for (i in 1:length(list)) {&#xA;    if (all.equal.phylo(item, list[[i]], use.edge.length = FALSE)) {&#xA;      total = total + 1&#xA;    }&#xA;  }&#xA;  return(total)&#xA;}&#xA;&#xA;result &amp;lt;- data.frame(unique_topology = rep(0, length(unique_topologies)),&#xA;                     count = rep(0, length(unique_topologies)))&#xA;for (i in 1:length(unique_topologies)) {&#xA;  result[i, ] &amp;lt;- c(i, count(unique_topologies[[i]], beast_output_rooted))&#xA;}&#xA;&#xA;result&lt;span class=&quot;math-container&quot;&gt;$percentage &amp;lt;- ((result$&lt;/span&gt;count/length(beast_output_rooted))*100)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4067" LastActivityDate="2019-02-07T12:03:07.163" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6976" PostTypeId="1" AcceptedAnswerId="6983" CreationDate="2019-02-07T15:23:39.787" Score="1" ViewCount="87" Body="&lt;p&gt;Is there any RNAseq gene/transcript read count database for the mouse? I already know about &lt;a href=&quot;https://amp.pharm.mssm.edu/archs4/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ARCHS4&lt;/a&gt;, looking for some other source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="3766" LastEditorUserId="298" LastEditDate="2019-02-07T15:55:12.580" LastActivityDate="2019-02-08T11:39:47.863" Title="RNA-seq gene/transcript read counts database for Mouse" Tags="|rna-seq|reads|data-mining|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6977" PostTypeId="1" CreationDate="2019-02-07T17:18:21.060" Score="3" ViewCount="5138" Body="&lt;p&gt;When I run my BruteForce function with only one input it works and the result is correct.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;def BruteForce(s, t):&#xA;    occurrences = []&#xA;    for i in range(len(s)-len(t)+1): # loop over alignment&#xA;        match = True&#xA;        for j in range(len(t)): # loop over characters&#xA;            if s[i+j] != t[j]:  # compare characters&#xA;                match = False   # mismatch&#xA;                break&#xA;        if match:   # allchars matched&#xA;            occurrences.append(i)&#xA;&#xA;    print(occurrences) &#xA;&#xA;t = 'CAACTTCCA'&#xA;&#xA;s = 'GACAACTTCCAACTTCCAACTTCCCGTCCCAACTTCACAACTTCGGCCCAACTTCCATGCAACTTCACCATCAACTTCGCTCGAAGCTGCCTTCCACTCCAACTTCACAACTTCCTCAACTTCCTCACCAACTTCAGCAACTTCTCTAGGGCCAACTTCCAACTTCTCAACTTCTCAACTTCCAACTTCCGACAACTTCTCCTGGCAACTTCCAACTTCCAACTTCAATACAACTTCGCAGACAACTTCCGCAACTTCGAACAACTTCCAACTTCCCCAACTTCCAACTTCCAACTTCGCCAACTTCCAACTTCCAACTTCCCAACTTCAGATAGCAACTTCGATCTTACACAACTTCACGCAACTTCTCCAACTTCCAACTTCTGTGCAACTTCTCTGAACAACTTCCTCAACTTCCAACTTCGCAACTTCCCCAACTTCCTCAACTTCATGCAACTTCGAGGCAACTTCCCAACTTCGCAACTTCCTATTCCCAACTTCTGTGGCAACTTCTCAACTTCTGGACAACTTCTATGCCCAACTTCACAACTTCCCCAACTTCTTTACAACTTCGACAACTTCATCAACTTCTAGTCAACTTCTGGTCCAACTTCCAACTTCCCCAACTTCCAAAGTGCCGCAACTTCGTAACAACTTCACGCGCTCAACTTCAACCAACTTCTTTTCCCGCAACTTCGCAACTTCACAACTTCTAATCAACTTCCAACTTCGGATCAACTTCCAACTTCGCCAACTTCCAACTTCCAACTTCTCCAGGGACAACTTCAAGTACAACTTCCAACTTCGCAACTTCACAACTTCCCAACTTCGCAACTTCTACACGCAACTTCCAACTTCTGGTCCCAACTTCATCAACTTCAGTCAACTTC'&#xA;&#xA;BruteForce(s, t)&#xA;&#xA;[2, 9, 48, 152, 175, 205, 212, 261, 277, 284, 300, 307, 370, 410, 607, 623, 717, 735, 751, 758, 792, 844]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But when I use it in the following loop it returns nothing! I have used the print() to see if it reads the strings and motifs from the input_2.txt file and it does but it returns an empty list of occurrences [].&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;with open('input_2.txt', 'r') as input_2:&#xA;    nt = input_2.readline() #nt is the no of strings+motifs in 1st line&#xA;    nt = int(nt) #in this case it is 10&#xA;&#xA;    for i in range(nt):&#xA;        s = input_2.readline()&#xA;        print(s)&#xA;        t = input_2.readline()&#xA;        print(t)&#xA;        BruteForce(s, t)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Could you please let me know what changes are needed here? &#xA;Many thanks,&#xA;Beeta &lt;/p&gt;&#xA;&#xA;&lt;p&gt;input_2.txt: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;10&#xA;&#xA;GACAACTTCCAACTTCCAACTTCCCGTCCCAACTTCACAACTTCGGCCCAACTTCCATGCAACTTCACCATCAACTTCGCTCGAAGCTGCCTTCCACTCCAACTTCACAACTTCCTCAACTTCCTCACCAACTTCAGCAACTTCTCTAGGGCCAACTTCCAACTTCTCAACTTCTCAACTTCCAACTTCCGACAACTTCTCCTGGCAACTTCCAACTTCCAACTTCAATACAACTTCGCAGACAACTTCCGCAACTTCGAACAACTTCCAACTTCCCCAACTTCCAACTTCCAACTTCGCCAACTTCCAACTTCCAACTTCCCAACTTCAGATAGCAACTTCGATCTTACACAACTTCACGCAACTTCTCCAACTTCCAACTTCTGTGCAACTTCTCTGAACAACTTCCTCAACTTCCAACTTCGCAACTTCCCCAACTTCCTCAACTTCATGCAACTTCGAGGCAACTTCCCAACTTCGCAACTTCCTATTCCCAACTTCTGTGGCAACTTCTCAACTTCTGGACAACTTCTATGCCCAACTTCACAACTTCCCCAACTTCTTTACAACTTCGACAACTTCATCAACTTCTAGTCAACTTCTGGTCCAACTTCCAACTTCCCCAACTTCCAAAGTGCCGCAACTTCGTAACAACTTCACGCGCTCAACTTCAACCAACTTCTTTTCCCGCAACTTCGCAACTTCACAACTTCTAATCAACTTCCAACTTCGGATCAACTTCCAACTTCGCCAACTTCCAACTTCCAACTTCTCCAGGGACAACTTCAAGTACAACTTCCAACTTCGCAACTTCACAACTTCCCAACTTCGCAACTTCTACACGCAACTTCCAACTTCTGGTCCCAACTTCATCAACTTCAGTCAACTTC&#xA;&#xA;CAACTTCCA&#xA;&#xA;ATTGTGTATCGCTATGTATCGTGTATCGGATTTTGTATCGTGTATCGGTGTATCGTGTATCGTGTATCGTATCACTGTATCGTTGTATCGTAGCGTTGTATCGTAATGTATCGCTCTGTATCGTGTATCGGGTTTGTATCGATGTGTATCGCTGTATCGGTGTATCGGTGTATCGCTTGTATCGACAGCGCTTGTATCGTGTATCGACCTGTATCGGTGTATCGTGTATCGAATGTATCGTTGTATCGAATTGTGTATCGTGTATCGTGTATCGTGTATCGTATGTATCGACTGTATCGCTGTATCGTAGCCTGTATCGTGTATCGGTGTATCGGTGTATCGGCGTGTATCGAATGTATCGTTGTATCGCTGTATCGTGTGCTGTGTATCGGTGTATCGACCCGTTGTATCGTGTATCGTGGGTAAGTGTATCGTTGTATCGTAGTGTATCGTGTATCGTGTATCGTCATGGTATTGTATCGTTATAGCTGTATCGCTCGGCTGTATCGTATGTATCGTGTATCGTGTATCGCATGTATCGGTGTATCGCTGTATCGACCATGTATCGTGTATCGGTGTATCGATGTATCGCTTCCATTAGAAATGTATCGTATGTATCGTGTATCGCTGTATCGTTTGTATCGCATGTATCGATTGTGTATCGTGTATCGTTGTATCGTGTATCGTGTATCGTTGTATCGTAATAACGATGTATCGAATGATGTATCGTTGTGATGTATCGTGTATCGACTGTATCGATGTATCGGGTTGTATCGTGTATCGTGTATCGTAGATGTATCGAGAGCCATTGTATCGCCTTGTATCGCGTGTATCGGTGTGTATCGTTTGTATCGTGTTTGAATGTATCGCTGTATCG&#xA;&#xA;TGTATCGTG&#xA;&#xA;TTTCAGCGTGTTTGGCTTTCAGCACTACTTTCAGCGGTTTCAGCTTTTTCAGCTTTTCAGCTATTTTCAGCTTTCAGCCATCTTTCAGCCCAGGTTTCAGCTGTTTCAGCTTTCAGCGAGTTTCAGCTTTCAGCGCGGGGATTTCAGCGTATTTCAGCGACATTTCAGCAGTTTCAGCTGGAGTGAAGCCGTTTCAGCGCTTTTCAGCACCATTTCAGCTAGTAGGTTTCAGCTTTTCAGCCTTTTCAGCTTTCAGCTATTTCAGCTCTAGAAGTCGTTTCAGCTTTCAGCTTCCTTTTCAGCTTTCAGCTTTCAGCCTTTCAGCTTTCAGCAAACTTTCAGCGATTTCAGCTTTCAGCTTTCAGCTTTCAGCTTTCAGCATTTCAGCTTTCAGCCGTTTTCAGCGACTCTTTCAGCGTTTCAGCTTTCAGCTGATCGTTTTCAGCTTTTCAGCCGGTTTCAGCGAAAGTTGGTCTTTTCAGCCAAATTTTCAGCTTTCAGCTTTCAGCGTTTCAGCTTTTCAGCCATTTCAGCCTATTTTCAGCAATCTTTCAGCAATTTTCAGCGCAATTTCAGCCAAAATTTCAGCATTTCAGCTTTCAGCGTAGTTTCAGCTTTCAGCTGAGCCTTTCAGCTTTTCAGCTCTCTTTTCAGCTTTCAGCGATTTCAGCCTTTTCAGCGCTTTCAGCACCACGCCTTTCAGCTTTCAGCTTTTCAGCCTTTCAGCCTATTTTCAGCGGTTTCAGCGTTTCAGCTTTCAGCACTTTCAGCTTTCAGCTTTTCAGCGGAAGGTTTTCAGCA&#xA;&#xA;TTTCAGCTT&#xA;&#xA;AGCATCCGACTGCATCCGTGCATCCGACGGCATCCGGCATCCGCAGCATCCGCACAGCCGCATCCGAGCCCGCATCCGAAGCATCCGGCATCCGGGCATCCGGCCGCATCCGGCATCCGGGCATCCGTGCATCCGGCATCCGTGTGCATCCGGCATCCGGGGTGCTTGCATCCGCGCATCCGTGCATCCGCGCATCCGGCATCCGCAGTCCGCATCCGCGCATCCGGCATCCGAGCATCCGTATTCGCATCCGTCGAGCATCCGTACGTCGCATCCGTGCATCCGGGAGAGGCATCCGGGCATCCGCGCATCCGGTGGACTATAACGCGCATCCGGCGAGCATCCGCCGCGGCATCCGTGCATCCGGTACGGCATCCGGGCTGAGGCATCCGCTCCCGGGGCATCCGCGGCGCATCCGCACGCGCATCCGGCATCCGAGCATCCGTTGCATCCGGAGCATCCGTAGCATCCGATAGCATCCGCAAGCTTGCATCCGGCATCCGCTACAGCATCCGGTGGCATCCGAAGCATCCGCGCATCCGCGGATTGCATCCGCACACGGCATCCGTAGCATCCGTGCATCCGGAGTGCATCCGCCGGGCGCATCCGCTGCATCCGCAGCGCATCCGAGCATCCGATCTTGCATCCGGCATCCGGGAATGCATCCGGGACTGCATCCGGTCTTAAGGGTGCATCCGAATGCATCCGCTGTAGCATCCGGCATCCGAGTAAGCATCCGAGTTCTGCATCCGGAAGCAGCATCCGGCATCCGGACACCAGCATCCGCGCATCCGGGGCGAGCATCCGAGCATCCGGCATCCGGGGCAAGTGGCATCCGGCATCCGTCGTGCATCCGGGCATCCGAA&#xA;&#xA;GCATCCGGC&#xA;&#xA;GACGAGCTGACGAGCGTGACGAGCAGGCGACGAGCGACGAGCATGCGGTGACGAGCGGACGAGCGACGACGAGCGCGGACGAGCGACGAGCGGACGAGCTCCGACGAGCGACGAGCACGACGAGCGCCCGACGAGCGACGAGCATGATGACGAGCGACGAGCTTAGACGAGCATGCTGACGAGCTGACGAGCTGACGAGCTCGTACATCGACGAGCGGACGAGCAGCCCGATAAGCCTTCGACGAGCTGACGAGCCACGACGAGCGACGAGCGACGAGCGGACGAGCGGGACGAGCGTCGGACGAGCGGACGAGCAGGACGAGCACCTCAATCGACGAGCTGACGAGCGGACGAGCCGACGAGCTGAACTGAAGGACGAGCCTACGTTCTAACGTGCCGTCACTGACGAGCGACGAGCGGACGAGCGATGGACGAGCTGACGAGCGCAGGACGAGCGAGAAGGCTGACGAGCAGACGAGCTGACGAGCTCGACGAGCAGACGAGCGGACGAGCTTTGATAAGACGAGCACGTCGACGAGCCTCGTGACGAGCTGACGAGCGACGAGCTAGACGAGCGACGAGCTGACGAGCGACGAGCACTAAACGCGACGAGCTCGACGACGAGCATGGACGAGCGACGAGCTAGGACGAGCGGCGACGAGCAGACGAGCGACGAGCTGACGAGCTTATAGACGAGCCTGACGACGAGCCAAGACGAGCGACGAGCGGACGAGCGACGAGCACGACGAGCCGACGAGCCGCGGACGAGCCGTAGACGAGCCAATCATTAAGACGAGCCGACGAGCCATTTGGGGACGAGCCTCCTCGACGAGCCATAAGACGAGCTGACGAGCCATGACGAGCATGCCCGACGAGC&#xA;&#xA;GACGAGCGA&#xA;&#xA;GAGGACCCCGGACCCCTGGACCCCAGGACCCCGACGGTGGGGCGGACCCCGAGAAGGACCCCTGGACCCCCGCAGGACCCCTTTATCGGACCCCGGACCCCGGACCCCCGGACCCCGGCTGGACCCCGTCGTAAGGACCCCGAATCGGACCCCTAGGACCCCAGGGACCCCTCCCCGGTTGGACCCCCTGGACCCCCTTGAGAGGGACCCCGGACCCCACGCCGTGCTTAAGGACCCCACTATGGACCCCATACGGACCCCGGACCCCGATCAGAGCGACCAGGACCCCCGGCCTGGACCCCTCGGACCCCAGAAGGACCCCTCGGACCCCTGGACCCCGGACCCCACAGGGACCCCGGACCCCTAGCCGCGGTGGACCCCCGGACCCCCGCAGATGGGGACCCCTCATGCGGACCCCCTGACGGACCCCTGGACCCCCGGGACCCCCAGGGACCCCATTTCGAGGACCCCATGGGGACCCCAAGCTGGACCCCGGACCCCTGGGACCCCCGGACCCCGGACCCCGGACCCCATAGGACCCCCGTTTGTTGCCATGGACCCCTTGGGACCCCGAGTCGGACCCCGGACCCCCAGGACCCCACAGGACCCCGGGACCCCGGACCCCATCGATCGGACCCCGGACCCCGGACCCCTACGGACCCCGCTAAGGGACCCCGGACCCCGTGGACCCCCACGGACCCCTAGGACCCCGGGACCCCGGACCCCGGACCCCAGTTGGACCCCCCTTAGGACCCCAGGACCCCACATGAGGACCCCGGGACCCCGGGACCCCTGGACCCCTGGACCCC&#xA;&#xA;GGACCCCGG&#xA;&#xA;AAACCTTCGGACCTTCGTACCTTCGACCTTCGTGAAACCTTCGGACCTTCGTGACCTTCGGGTGACCTTCGGGTACTCACCTTCGGCACCTTCGTATACCTTCGAGACCTTCGCTGGACTGTAAACCTTCGATCGTTATTTTCTACCTTCGCACACCTTCGACCTTCGAGCCGAACCTTCGGACTGGCCTCACCTTCGATCACCTTCGACCTTCGAACCTTCGACCTTCGACCATACCTTCGACCTTCGGACCTTCGGACCTTCGCACCTTCGAACCTTCGCCACCTTCGACCTTCGAACCTTCGGTGGGCCACCTTCGTTACCTTCGAACCTTCGGGCACCTTCGACCTTCGGCTTACACCTTCGGACCTTCGATGACTGACCTTCGAAACCTTCGACCTTCGACCTTCGATTCCACACCTTCGACCTTCGACACCTTCGACCTTCGCGTTCACCTTCGACCTTCGACCTTCGGGCCAATACCTTCGACCTTCGGCCTAATACCAACCTTCGAACCTTCGAACCTTCGGGTGACCTTCGGACCTTCGCGCACACCTTCGCACTTACCTTCGACCTTCGGTGACCTTCGACCTTCGTGCACCTTCGCGTCTCACCTTCGCACCTTCGTTACCTTCGAACCTTCGACCTTCGACAACCTTCGTACCTTCGCTACCTTCGGGAACCTTCGATGTCCTACCTTCGACCTTCGCGACCTTCGACCTTCGTAACCTTCGAACCTTCGACCACCTTCGAACCTTCGAAGGAAGGACCTTCGAATCATTACCTTCGTTACCTTCGTAGTACCTTCGCCGACCTTCGGACCTTAACCTTCGTGA&#xA;&#xA;ACCTTCGAC&#xA;&#xA;AAGTGGTATTCCCAAAAAGATATTCCCGTTATTCCCGCTATAAATATTCCCAACTGTTTATTCCCACTTATTCCCGTATTCCCGTATTCCCCGGGCCTTTTATTCCCCATTCTATTCCCGTATTCCCATATTCCCACTTATTCCCCTATTCCCATATTCCCTATTCCCTATTCCCGTAAGTATTCCCCCCCTTATTCCCCCATTTGTCCATATTCCCTCAATATTCCCTATTCCCCTATTCCCGCTTCCCCAGTCTATCCCAAATTGTATTCCCAATATTCCCCTTTATTCCCTTTATTCCCGTATTCCCTATTCCCGTGGCTATTCCCCCGACGTTATTCCCTCTATATTCCCGTCTCAAAGTATTCCCTTATTCCCAGTATTCCCATTCGCCTGGTATTCCCGTATTCCCGGTATTCCCGTTTATTCCCATCTATTCCCATTATTCCCTGGTATTCCCGCTATTCCCGCAACGGTATTCCCTATTCCCTATTCCCGTATTCCCAATATTCCCTTCCTATTCCCGTATTCCCTAGCTATTCCCATATTCCCGATATTCCCTATTCCCGTATTCCCTATTCCCTTATTCCCTATTCCCTAGTTATTCCCAAACAGGCTATTCCCTATTCCCTATTCCCATATTCCCTTATTCCCTACGTATTCCCTCTTAAAAATATTATTCCCCTATTCCCTATTCCCGTATTCCCCGAAAGTATTCCCCCATATTCCCAATTATATTCCCGGTATTCCCCCGTATTCCCTATTCCCTTATTCCCATATTCCCATATTCCCAAAGTTTATTCCCGTATTCCCCACTATTCCCATAGATTATTCCCTATTCCCTATTCCCTTATTCCCCTATTCCCTATTCCCTTATTCCCGTATTCCCAGTTTATTCCC&#xA;&#xA;TATTCCCTA&#xA;&#xA;GAAAGGATAAAGGATGTCAAAGGATGCAGATATAAAGGATAAAGGATACAAAGGATTAAGTATGTCCGAAAAAGGATAAAGGATTAAAAGGATCGCTGAACCTTACACAAAGGATAGTGAACAAAGGATTCAATAAAAGGATAAAGGATCAAAGGATCAAAAGGATACAAAGGATGCCGATGAAAGGATAAAGGATCTAAAGGATAAAGGATGAGAAAGGATTGGAAAGGATTATGAGATCAAAGGATAAAGGATGGTGTAAAAGCTAAAGGATTCGGCAAAGGATAAAGGATTAAAGGATAAAAAGGATAAAGGATGAAAGGATCCGCAGGGACCAGCAAAAGGATAAAGGATCGAATGGGTAAGAAAGGATCAAAGGATGAAAAAGGATTACTAAAGGATAAAGGATCCTGAAAGGATTACAAAGGATCTTAAAAGGATTCGGAAAGGATCCATAGGAAAAGGATAAAGGATCGCGAAAGGATAAAGGATTAAAGGATAAAGGATAAAAGGATTCAAAGGATAAAGGATAGACAAGGGAGAAAGGATGCAAAGGATGAAAGGATAAAGGATAAAGGATAAAGGATTAGGTTAAAGGATCGAAAGGATCCAAAGAGAGCGAAAAGGATGAGGACGAAAGGATCAAAGGATCCAAAGGATCAAAGGATAAAGGATTGCGATGGAAAGGATGTCAAAGGATCACCAAAGGATAAAGGATAATAAAGGATAAAGGATAAAGGATCAAAGGATTTGAAAAGGATAAAGGATCACGAAAGGATAAAAGGATTGGCAAAGGATGAAAAGGATGAAAGGATAAGCCCTCCAAAGGAT&#xA;&#xA;AAAGGATAA&#xA;&#xA;ATTCAAATACTTCAAATAGTCAAATATCAAATATGCTTCAAATATCAAATATCGGTCAAATAAGTCAAATAAGTCAAATATCAAATATCAAATACTCAAATATCAAATAGTGTCAAATACGAATTGGGTCAAATATCAAATATGTGTTCAAATATTCTTCAAATACTGGACACTCAAATAGGAGTCAAATATCAAATATCAAATACGTGAAGTGCTTGTCAAATATTTCAAATACTTTCAAATATTCAAATACTCAAATATGTTCAAATATCAAATATCAAATATTTCAAATATCAAATATCAAATAGTCCTCAAATAGCAAACCAGTCAAATATCAAATATGTCAAATATGCTCACGGCAACCTCAAATATCAAATATCAAATAGATGTCAAATAAGTTTCAAATATCAAATATCAAATATCAAATATCAAATATTCAAATATCAAATAGCGGGCTCAAATAAGCTCAAATACGTCAAATAGGGGGTCAAATAGATCAAATAGTCAAATATTCAAATACATCAAATAGTCAAATACAAGAACCACCGAGATCTCAAATAATCAAATATCAAATATGGTCAAATAAATATTCAAATAGGTCAAATAAGATCAAATATCAAATAAGTCGTCCATCAAATAGTCAAATAGCTCAAATAACCTCAAATATCAAATATTCAAATACCGGTCATCAAATAGGACAAATCAAATAGTCAAATAAGATCCTCTCAAATAATTCAAATAGCTGTTCAAATACTCAAATATCAAATAGTCAAATATCAAATATTCAAATATCAAATACTTCAAATATGTTCAAATAATCAAATACTCAAATATCAAATAATCAAATAATACTTCAAATATACCAAACGCTCAAATATTAGTTGGATCAAATATCTTCAAATATCAAATAA&#xA;&#xA;TCAAATATC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2726" LastEditorUserId="3848" LastEditDate="2023-06-02T23:26:19.613" LastActivityDate="2023-06-02T23:26:19.613" Title="Python - Finding a motif - input: a txt file with 10 sequences and 10 motifs" Tags="|python|phylogenetics|sequence-alignment|phylogeny|motifs|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6978" PostTypeId="2" ParentId="6977" CreationDate="2019-02-07T17:49:57.243" Score="3" Body="&lt;p&gt;This looks hard work. In Perl the function &quot;index&quot; and &quot;rindex&quot; as well as &quot;substr&quot; will perform this without the incremental loop. There must be an equivalent in Python?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've not tried this, but at a guess it would be something like,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;my @array;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$target = 'AGATAGAGAGATAGAGAGATAGAGAGATAGAGAGATAGAG';&#xA;my $&lt;/span&gt;query = 'AGAGA';&#xA;while(length(&lt;span class=&quot;math-container&quot;&gt;$target) &amp;gt; 4){&#xA;    my $location = index ($query, $target);&#xA;    push @array, $location;&#xA;    $target = substr ($location, length($target), $&lt;/span&gt;target);&#xA;}&#xA;print join(&quot;\s&quot;, @array), &quot;\n&quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-02-07T17:49:57.243" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6979" PostTypeId="2" ParentId="6977" CreationDate="2019-02-07T20:26:24.397" Score="5" Body="&lt;p&gt;Let's start by doing some explaining about why your code doesn't work.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;with open('input_2.txt', 'r') as input_2:&#xA;    nt = input_2.readline() #This is reading the first line of the file (which is 10)&#xA;    nt = int(nt) # this is storing &quot;10&quot; as an int&#xA;&#xA;    for i in range(nt): # This is the same as &quot;for i in range(10):&quot; which just returns nums 0-9, does not access lines of your file&#xA;        s = input_2.readline() # this is reading the second line which is blank &#xA;        print(s) # this will print a blank line 10 blank lines&#xA;        t = input_2.readline() # this is reading the rest of the file&#xA;        print(t)&#xA;        BruteForce(s, t) # this is calling your function on 10 blank lines and the restof your file, not each seq/motif&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I made some modifications that will help you achieve your goal I think:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def BruteForce(s, t):&#xA;    occurrences = []&#xA;    for i in range(len(s)-len(t)+1): # loop over alignment&#xA;        match = True&#xA;        for j in range(len(t)): # loop over characters&#xA;            if s[i+j] != t[j]:  # compare characters&#xA;                match = False   # mismatch&#xA;                break&#xA;        if match:   # allchars matched&#xA;            occurrences.append(i)&#xA;&#xA;    print(occurrences)&#xA;&#xA;def group(lst, n):&#xA;    &quot;&quot;&quot; This function will be used to group your sequences and motifs later &quot;&quot;&quot;&#xA;    for i in range(0, len(lst), n):&#xA;        val = lst[i:i+n]&#xA;        if len(val) == n:&#xA;            yield tuple(val)&#xA;&#xA;with open('input_2.txt') as input_2: #standard open in read mode, &#xA;    lines = input_2.readlines()[2:] # skip 10 and the blank space bc we don't care about them&#xA;    split_line = [line.strip('\n') for line in lines if line!='\n'] #this will store lines that aren't blank into a list and strip new_line chars '\n'&#xA;    seq_tuple = list(group(split_line, 2)) # this will use group func above to group the list into tuples of 2's which are sequence and motif.&#xA;&#xA;for x in seq_tuple: # this accesses the list of tuples (seq_tuple)&#xA;    BruteForce(x[0], x[1]) # x[0] in each tuple is seq, or BruteForce(s), x[1] in each tuple is motif, or BruteForce(t)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Running this will output Each motifs start position in each sequence:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[2, 9, 48, 152, 175, 205, 212, 261, 277, 284, 300, 307, 370, 410, 607, 623, 717, 735, &#xA;751, 758, 792, 844]&#xA;[14, 33, 48, 55, 116, 192, 217, 253, 260, 267, 312, 370, 405, 412, 445, 452, 511, &#xA;518, 561, 613, 657, 672, 679, 735, 769, 776, 847]&#xA;[37, 46, 64, 103, 120, 226, 243, 277, 284, 296, 303, 318, 345, 352, 359, 366, 381, 418, 439, 488, 495, 510, 590, 608, 628, 648, 697, 704, 748, 764, 771]&#xA;[29, 80, 95, 105, 128, 145, 191, 219, 328, 423, 489, 642, 713, 758, 808, 831]&#xA;[28, 57, 77, 102, 129, 148, 258, 265, 404, 419, 449, 553, 569, 584, 629, 664, 713, 728]&#xA;[2, 87, 94, 109, 204, 254, 332, 350, 487, 511, 518, 577, 603, 611]&#xA;[18, 154, 201, 216, 223, 235, 283, 339, 390, 397, 418, 425, 434, 453, 460, 481, 565, 582, 637, 644, 695, 711, 735]&#xA;[154, 161, 221, 303, 476, 483, 526, 554, 569, 584, 591, 617, 624, 647, 685, 754, 829, 836, 859]&#xA;[1, 33, 70, 135, 182, 198, 241, 279, 294, 303, 340, 395, 460, 478, 493, 500, 517, 559, 566, 573, 661, 704, 711, 721, 728, 754, 772, 808]&#xA;[19, 37, 44, 73, 80, 95, 128, 184, 191, 263, 270, 286, 293, 327, 364, 371, 400, 407, 414, 421, 436, 561, 609, 659, 754, 769, 784, 826, 886, 896]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3669" LastActivityDate="2019-02-07T20:26:24.397" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6980" PostTypeId="1" AcceptedAnswerId="6981" CreationDate="2019-02-08T06:11:26.820" Score="5" ViewCount="1059" Body="&lt;p&gt;I'm a starting my analysis on nanopore whole genome sequencing. I start my analysis from this popular &lt;a href=&quot;https://github.com/nanopore-wgs-consortium/NA12878&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sample I downloaded was WGS for NA12878, so I would assume it's alignment to GRCh38 shouldn't be that bad.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But ... I'm getting lot's of insertions...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/63m8T.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/63m8T.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure it was caused by the sequencing platform, the MiniMap aligner or my bugs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I supply all my commands, starting from sample download.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wget https://s3.amazonaws.com/nanopore-human-wgs/rel5-guppy-0.3.0-chunk10k.fastq.gz&#xA;gunzip rel5-guppy-0.3.0-chunk10k.fastq.gz&#xA;wget http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/technical/reference/GRCh38_reference_genome/GRCh38_full_analysis_set_plus_decoy_hla.fa&#xA;minimap2 -d GRCh38_full_analysis_set_plus_decoy_hla.fa.index GRCh38_full_analysis_set_plus_decoy_hla.fa&#xA;minimap2 -ax map-ont -t 20 GRCh38_full_analysis_set_plus_decoy_hla.fa.index rel5-guppy-0.3.0-chunk10k.fastq | samtools view -bS - | samtools sort -@ 5 &amp;gt; rel5-guppy-0.3.0-chunk10k.bam &#xA;samtools index rel5-guppy-0.3.0-chunk10k.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="174" LastEditorUserId="96" LastEditDate="2019-02-08T14:33:13.943" LastActivityDate="2019-02-08T14:33:13.943" Title="Why do I get so many insertions from Minimap2 on my Nanopore WGS?" Tags="|sequence-alignment|ngs|nanopore|errors|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6981" PostTypeId="2" ParentId="6980" CreationDate="2019-02-08T07:10:47.560" Score="8" Body="&lt;p&gt;Insertions and deletions are the dominant error mode of long read sequencing, including nanopore sequencing. What you see is not unexpected. Things may have improved by now if you would download the raw fast5 data and repeat the basecalling. There is no need to &lt;code&gt;gunzip&lt;/code&gt; the fastq.gz prior to alignment. Your commands for alignment look alright, except that (if you are using a recent version of samtools) you don't really need samtools view and can pipe directly to samtools sort, using -o to specify the output format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 -ax map-ont -t 20 &amp;lt;index&amp;gt; &amp;lt;fastq&amp;gt; | samtools sort -@5 -o alignment.bam &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In a recent version of igv you can right-click the alignment and chose a &quot;Quick consensus mode&quot; (or something like that), which will make your alignments at least a bit prettier. Note how the consensus sequence (top of the graph) is quite clean - while individual reads may be noisy this is mostly randomly distributed and as such ironed out in consensus.&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2019-02-08T07:10:47.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6982" PostTypeId="1" CreationDate="2019-02-08T09:51:54.553" Score="1" ViewCount="101" Body="&lt;p&gt;I am interested in creating an averaged value for genotype across two parents:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g. Input (three unrelated couples, 1, 2 and 3):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Individual    SNP1    SNP2    SNP3    SNP4 ...&#xA;Father_1      1       0       1       2    ...&#xA;Mother_1      0       0       2       1    ...&#xA;Father_2      1       0       1       1    ...&#xA;Mother_2      2       1       0       1    ...&#xA;Father_3      1       2       0       1    ...&#xA;Mother_3      1       .       0       2    ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;E.g. Output (the midparental genotypes for couple 1, 2 and 3):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Individual    SNP1    SNP2    SNP3    SNP4 ...&#xA;Midparent_1   0.5     0       1.5     1.5  ...&#xA;Midparent_2   1.5     0.5     0.5     1    ...&#xA;Midparent_3   1       .       0       1.5  ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can think of a way to do this using e.g. a PLINK .raw file and manipulating that in R, but am concerned that would become memory- and storage-heavy when assessing 100000s of SNPs and 1000s of pairs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know of an existing tool that can do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4092" LastEditorUserId="96" LastEditDate="2019-02-08T15:22:34.543" LastActivityDate="2019-02-08T15:31:53.057" Title="Calculate average parental genotype" Tags="|snp|plink|data-management|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6983" PostTypeId="2" ParentId="6976" CreationDate="2019-02-08T11:39:47.863" Score="2" Body="&lt;p&gt;EBI has an expression atlas, including mouse: &lt;a href=&quot;https://www.ebi.ac.uk/gxa/experiments?species=mus+musculus&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/gxa/experiments?species=mus+musculus&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you click on one of the individual experiments you can download tables of TPM or RPKM per gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are currently about 1000 individual experiments in different tissues etc.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-02-08T11:39:47.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6984" PostTypeId="2" ParentId="6966" CreationDate="2019-02-08T11:44:12.593" Score="2" Body="&lt;p&gt;Kaggle has a cats v dogs dataset if you want to try your hand at biological image classification: &lt;a href=&quot;https://www.kaggle.com/c/dogs-vs-cats&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kaggle.com/c/dogs-vs-cats&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The DREAM challenge competitions are usually specifically oriented to solving biological classification or learning problems with ML: &lt;a href=&quot;http://dreamchallenges.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dreamchallenges.org/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-02-08T11:44:12.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6985" PostTypeId="2" ParentId="6982" CreationDate="2019-02-08T12:05:13.173" Score="2" Body="&lt;p&gt;Beside the correct question by @Kamil, this sounds like a job for &lt;a href=&quot;https://www.gnu.org/software/datamash/&quot; rel=&quot;nofollow noreferrer&quot;&gt;datamash&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First let's get the number of columns in the file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ head -n1 genotypes.txt| awk -v FS=&quot;\t&quot; '{print NF}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now lets calculate the average per column with &lt;code&gt;datamash&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ datamash -H mean 2-&amp;lt;n&amp;gt; &amp;lt; genotypes.txt &amp;gt; mean.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Replace &lt;code&gt;&amp;lt;n&amp;gt;&lt;/code&gt; with the number of columns we received with command above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your example the output will look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mean(SNP1)  mean(SNP2)  mean(SNP3)  mean(SNP4)&#xA;0,5 0   1,5 1,5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One can remove the &lt;code&gt;mean&lt;/code&gt; with &lt;code&gt;sed&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sed -i -e 's/mean(//g' -e 's/)//g' mean.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1310" LastActivityDate="2019-02-08T12:05:13.173" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6986" PostTypeId="1" AcceptedAnswerId="6994" CreationDate="2019-02-08T12:59:59.647" Score="2" ViewCount="248" Body="&lt;p&gt;I have a dataset with 88 tumor and 113 normal samples. Among the 50k genes after filtering there are a total of 28k genes (both mRNAs and lncRNAs)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to do co-expression analysis between lncRNAs and mRNAs to find the targets for each lncRNA. I'm thinking to use WGCNA for co-expression network analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Few doubts:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) For Co-expression network analysis (WGCNA) should I use only tumor samples (or) both tumor and normal samples?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) If not WGCNA, if I use cor function in R with method Pearson, on what cutoff I should select the target genes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Heard that with bedtools we can know which genes are 10kb upstream/ downstream of lncRNAs to find neighboring pc genes. But no idea how to do that. Could you please give me an example for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="1377" LastActivityDate="2019-02-11T09:38:07.903" Title="Question about Co-expression analysis and finding targets for lncRNAs" Tags="|r|rna-seq|bioconductor|rna|wgcna|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="6987" PostTypeId="1" CreationDate="2019-02-08T14:09:17.993" Score="2" ViewCount="378" Body="&lt;p&gt;I frequently make use of &lt;a href=&quot;http://rseqc.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RSeQC&lt;/a&gt;, I used to use it with my own bed file, generated using gtf2bed, but now I thought I'd use the one provided by RSeQC themselves from &lt;a href=&quot;https://sourceforge.net/projects/rseqc/files/BED/Human_Homo_sapiens/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, I got the file hg38_Gencode_V28.bed.gz, when looking at it with IGV it looks similar to my self generated bedfile however, read_distribution.py (a script in RSeQC), does not report any reads in any of the categories (CDS_exons, Introns, etc).&lt;/p&gt;&#xA;&lt;p&gt;Any idea why?&lt;/p&gt;&#xA;&lt;p&gt;Edit: I see that my own bed file (from the GTF) had ENST identifiers like this ENST00000466557 whereas the file from RSeQC supplies them like ENST00000611746.1 (with decimals) that seems to be the only difference as far as I can see.&lt;/p&gt;&#xA;" OwnerUserId="3964" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2019-02-08T15:48:29.890" Title="RSeQC provided bedfile does not work with read_distribution.py" Tags="|rna-seq|qc|rseqc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6988" PostTypeId="1" CreationDate="2019-02-08T14:10:32.267" Score="3" ViewCount="275" Body="&lt;p&gt;I am evaluating the tool somalier (&lt;a href=&quot;https://github.com/brentp/somalier&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/brentp/somalier&lt;/a&gt;) and I need to create a list of about 65,000 SNPs where the allele frequency (AF) is as close to 0.5 as possible across the most representative set of populations possible with today's data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My alignments are in GRCh38, so I need a vcf input file for somalier also in GRCh38, not the one that is available in the somalier website, which is in GRCh37.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am starting as input with the vcfs from the 1000genomes project in GRCh38 positions, and I would like to come up with a filtering strategy that:&#xA;(1) Sorts all SNPs by the AF being close to 0.5 in as many populations as possible.&#xA;(2) Takes the first 65,000 SNPs from that list and produces a vcf file from them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My simplistic strategy so far has been to take the global AF value if it's between 0.3-0.7 and print it if it's a SNP with this perl one-liner:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gunzip -c my.vcf.gz | grep -v '#' | perl -lne 'if (&lt;span class=&quot;math-container&quot;&gt;$_ =~ /\;AF=0\.[34567]/) { if ($_ =~ /\w+\t\d+\t\S+\t\w\t\w\t/) {print $&lt;/span&gt;_} }'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any recommendations?&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="180" LastEditDate="2019-02-08T14:48:14.433" LastActivityDate="2019-02-08T14:49:49.247" Title="Selecting 65000 SNPs where AF is close to 0.5 in all or most populations" Tags="|vcf|snp|genomics|variation|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6989" PostTypeId="2" ParentId="6988" CreationDate="2019-02-08T14:31:12.120" Score="5" Body="&lt;p&gt;Just use &lt;a href=&quot;http://www.htslib.org/doc/bcftools.html#view&quot; rel=&quot;noreferrer&quot;&gt;bcftools view&lt;/a&gt; for filtering:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools view -i 'AF&amp;gt;0.3 &amp;amp;&amp;amp; AF&amp;lt;0.7' input.vcf.gz &amp;gt; output.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To truncate this list to 65,000 SNPs count the header lines, sum them to the number of SNPs you like to have and use &lt;code&gt;head -n&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools view -h input.vcf.gz|wc -l&#xA;255&#xA;&#xA;$ bcftools view -i 'AF&amp;gt;0.3 &amp;amp;&amp;amp; AF&amp;lt;0.7' input.vcf.gz | head -n 65255 &amp;gt; output.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1310" LastActivityDate="2019-02-08T14:31:12.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6990" PostTypeId="2" ParentId="6988" CreationDate="2019-02-08T14:49:49.247" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/a/6989/298&quot;&gt;Using &lt;code&gt;bcftools&lt;/code&gt;&lt;/a&gt; is the right way, but you could also greatly simplify the command you're using. You can do the same thing with a single &lt;code&gt;zgrep&lt;/code&gt; call:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zgrep -P '^[^#]+\t\d+\t\S+\t\w\t\w\t.*;AF=0\.[3-7]' my.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And to limit to the first 65000 results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zgrep -m 65000 -P '^[^#]+\t\d+\t\S+\t\w\t\w\t.*;AF=0\.[3-7]' my.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, to keep the header of the original vcf unchanged, you could do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zgrep -m 65000 -P '^(#|[^#]+\t\d+\t\S+\t\w\t\w\t.*;AF=0\.[3-7])' my.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also suggest you use hg19 instead of hg38 data since the best available source for frequencies, the gnomAD database, is based on hg19 and the tool you want to test also provides a site file for hg19. But &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/6988/selecting-65000-snps-where-af-is-close-to-0-5-in-all-or-most-populations#comment10101_6988&quot;&gt;that may not be possible&lt;/a&gt; for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your VCF file has multiple AF entries for different sub-populations and you want to use those, you'll probably need to script something. We can't really help with that unless you show us some example input lines and start writing the basic script yourself though.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-02-08T14:49:49.247" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="6991" PostTypeId="2" ParentId="6982" CreationDate="2019-02-08T15:23:41.230" Score="1" Body="&lt;p&gt;These are unusual parents because they all appear haploid. Anyway, if I was doing this I would calculate the allele frequency per progeny such as makefreq in adegenet in R-cran (just my personal thoughts). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt; The issue is that you are using a continuous distribution 1,2,3 etc... for catagorical data and will result in confusing homozygotes with heterozygotes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;For example&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you have allele 3 (father) and allele 1 (mother) the &quot;progeny&quot; wil&#xA;be &quot;allele&quot; 2&lt;/li&gt;&#xA;&lt;li&gt;If you have allele 2 (father) and allele 2 (mother) the &quot;progeny&quot;&#xA;will be &quot;allele&quot; 2&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;However, one is homozygous and the other is heterozygous and the allelic background for each is quite differnt. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The alternative is to use letters rather than numbers. You can always use &quot;OneHotKeyEncoder&quot; to convert back from AA, aa etc... to numerical catagories for analysis later (its a standard feature in Python).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you used letters rather and then converted to numbers every &quot;progeny&quot; would have its own numerical catagory (but you would lose homozygous vs. heterozygous classification). It is standard practice to switch from letters to numbers and the &quot;encoder&quot; will ensure no mistakes are made. The other way around this is to designate the variable as a string, rather than as an integer... this might be more to your liking, but you are stuck in base 10.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With regards the throughput, 100000s 1000 individuals, a 100 million SNPs is doable via R, the operations are simple.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to clarify: it would be easy to script to a solution (even from base code), but I'm not sure about how to interpret the output. All I'm suggesting is you have another think about the goal of this analysis. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-02-08T15:31:53.057" LastActivityDate="2019-02-08T15:31:53.057" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="6992" PostTypeId="2" ParentId="6986" CreationDate="2019-02-08T15:43:57.853" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;2) If not WGCNA, if I use cor function in R with method Pearson, on&#xA;  what cutoff I should select the target genes?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Good call, this is a nice approach. It is a good question because the central issue is that a purists approach of 0.7 results in insufficient information to understand the behaviour of the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The guys here were using 0.4, in my experience co-variance of 0.5 and above is good evidence, whilst 0.7+ is strong evidence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've done this (not on RNAseq) and I used 0.25 to 0.45 and 0.45 and above then used a scatterplot with one half of the graph showing low co-variance and the other half strong co-variance. So in one sense I ignored my own advice (0.5+). In my defence its easy to reassign the boundaries. Each scatterplot (1000s) was then subject to tSNE analysis to assess the group behaviour. Its a different context but gives an idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Networks are cool for visualisation but they are not particularly  quantitive for most (not all) network theory.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Just to explain to everyone else, what happens if you get boundary effects with the data and you are assigning a discrete boundary on a continous distribution (its a negative binomial distribution in my case). I observed a vast amount of results  occurring between 0.2 to 0.25 - but 0.2 isn't good evidence, so that data has to be lost. These sort of boundaries were in days before genomic throughput. Thus in my context I didn't consider a hard boundary particularly informative.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-02-08T16:01:58.460" LastActivityDate="2019-02-08T16:01:58.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6993" PostTypeId="2" ParentId="6987" CreationDate="2019-02-08T15:48:29.890" Score="2" Body="&lt;p&gt;Ok, I figured it out already, it's and old issue I come accross every now and then. The original bed file lists the chromosomes as chr1,chr2,...,chrX whereas my bam files uses 1,2,...,X&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I fixed the bed file like this (python3):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;bedFile = '/hg38_Gencode_V28.bed'&#xA;bedFile = pd.read_csv(bedFile, sep='\t', header=None)&#xA;bedFile[0] = [value.strip('chr') for value in bedFile[0]]&#xA;newBedFile = 'hg38_Gencode_V28_reformatted.bed'&#xA;bedFile.to_csv(newBedFile, sep='\t', header=None, index=None)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And now read_distribution.py works.&lt;/p&gt;&#xA;" OwnerUserId="3964" LastActivityDate="2019-02-08T15:48:29.890" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="6994" PostTypeId="2" ParentId="6986" CreationDate="2019-02-08T17:42:18.030" Score="4" Body="&lt;p&gt;You have several options to approach this with &lt;a href=&quot;https://horvath.genetics.ucla.edu/html/CoexpressionNetwork/Rpackages/WGCNA/&quot; rel=&quot;nofollow noreferrer&quot;&gt;WGCNA&lt;/a&gt; (weighted correlation network analysis). You can run a WGCNA on the combined set, identify modules and select those lncRNAs for further follow-up that have relatively high intra-modular connectivity (i.e., are intramodular hub genes). I would suggest this approach first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second option is to run WGCNA on mRNA for coding genes only and screen module eigengenes and lncRNAs for association with one another. This approach is usually helpful when the two data types (mRNA and lncRNAs) are different which I suppose is not really your case (mRNA and lncRNAs were probably measured together and are overall similar).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would start using all samples together (tumor and normal). Hopefully you'll find some interesting and plausible information; in the second pass I'd perhaps create separate (or consensus) networks for tumor and normal samples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not familiar with bedtools, but you should have some sort of gene/transcript annotation with your data (it was surely aligned to a reference genome and summarized to some sort of annotated set of genes). The annotation should contain chromosome and base pair position, which would make it rather easy to find genes within a specific interval.&lt;/p&gt;&#xA;" OwnerUserId="851" LastEditorUserId="3848" LastEditDate="2019-02-10T00:56:02.353" LastActivityDate="2019-02-10T00:56:02.353" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="6995" PostTypeId="1" CreationDate="2019-02-08T18:59:48.210" Score="1" ViewCount="47" Body="&lt;p&gt;I should get and analyse isomiR (isoform of miRNA) profile from RNA-seq with FASTA format. The best tool for this is &lt;a href=&quot;https://mcg.ustc.edu.cn/bsc/deanniso/upload.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;DeAnnIso&quot;&lt;/a&gt;. But here is a problem with this tool, it only allows you to upload FASTA format files  with a maximum size of 50MB. However, the size of my FASTA file is more than 1000MB. I tried to compress the file in different ways and uploading, but it doesn't work.&#xA;Does anybody know any way to solving this problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or Does anybody know &lt;strong&gt;any other easy way to get isomiR profile from RNA-seq with FASTA format?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A sample of RNA-seq like this: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;RR42357.95441.1 I-D00105:274:H7MDXX:1:112:8124:865 length=51&#xA;TGGAATTCTCGGGTGCCAAGGAACTCCAGTCACCTTGTAATCTCGTATGCC&#xA;&amp;gt;RR42357.95442.1 I-D00105:274:H7MDXX:1:112:8133:866 length=51&#xA;AAAGTGCTACTACTTTTGAGTCTTGGAATTCTCGGGTGCCAAGGAACTCCA&#xA;&amp;gt;RR42357.95443.1 I-D00105:274:H7MDXX:1:112:8144:8637 length=51&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thank you very very much!&lt;/p&gt;&#xA;" OwnerUserId="4091" LastEditorUserId="298" LastEditDate="2019-02-09T12:31:49.513" LastActivityDate="2019-02-09T12:31:49.513" Title="Get &amp; Annotation of isomiR (isoform of miRNA) profile from RNA-seq" Tags="|rna-seq|fasta|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="6996" PostTypeId="2" ParentId="6930" CreationDate="2019-02-08T22:26:03.303" Score="2" Body="&lt;p&gt;I figured that one has to include only the changes in the rules formula. Keeping the initial value of the &lt;code&gt;Species&lt;/code&gt; or &lt;code&gt;Parameter&lt;/code&gt; when defining circular or self-referencing formulas caused the error. So the correct rule becomes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;pH=-a*Acid&lt;/code&gt; instead of &lt;code&gt;pH=pH-a*Acid&lt;/code&gt; or &lt;code&gt;pH=7.4-a*Acid&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For convenience, I include the correct code crowded with several reactions producing both acids and bases, where pH gradually decreases as expected. The names are fictional, and just are better placeholders instead of x1 and x2, etc.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(&quot;sysBio&quot;)&#xA;&#xA;glu &amp;lt;- newModel(&quot;glu&quot;)&#xA;addSpecies(glu, &quot;Glucose&quot;, 1000)&#xA;addSpecies(glu, &quot;Lactate&quot;, 0)&#xA;addSpecies(glu, &quot;Pyruvate&quot;, 0)&#xA;addSpecies(glu, &quot;Oxalate&quot;, 0)&#xA;addSpecies(glu, &quot;Acid&quot;, 0)&#xA;addSpecies(glu, &quot;Base&quot;, 0)&#xA;addSpecies(glu, &quot;pH&quot;, 7.4)&#xA;&#xA;addMAreaction(glu, &quot;Glucose = Lactate + Acid&quot;, &quot;forwardL&quot;, &#xA;              &quot;backwardL&quot;, name = &quot;glycolysis&quot;)&#xA;addMAreaction(glu, &quot;Lactate = Pyruvate + Acid&quot;, &quot;forwardP&quot;, &#xA;              &quot;backwardP&quot;, name = &quot;lactolysis&quot;)&#xA;addMAreaction(glu, &quot;Pyruvate = Oxalate + Base&quot;, &quot;forwardO&quot;, &#xA;              &quot;backwardO&quot;, name = &quot;oxalate synthesis&quot;)&#xA;&#xA;addMAreactRate(glu, &quot;forwardL&quot;, &quot;assigned&quot;, &#xA;               &quot;k_glucose_to_lactate*Glucose&quot;)&#xA;addMAreactRate(glu, &quot;backwardL&quot;, &quot;assigned&quot;, &#xA;               &quot;k_acid*pH+k_lactate_to_glucose*Lactate&quot;)&#xA;addMAreactRate(glu, &quot;forwardP&quot;, &quot;assigned&quot;, &#xA;               &quot;k_lactate_to_pyruvate*Lactate&quot;)&#xA;addMAreactRate(glu, &quot;backwardP&quot;, &quot;assigned&quot;, &#xA;               &quot;k_acid*pH+k_pyruvate_to_lactate*Pyruvate&quot;)&#xA;addMAreactRate(glu, &quot;forwardO&quot;, &quot;assigned&quot;, &#xA;               &quot;k_pyruvate_to_oxalate*Pyruvate&quot;)&#xA;addMAreactRate(glu, &quot;backwardO&quot;, &quot;assigned&quot;, &#xA;               &quot;k_base*pH+k_oxalate_to_pyruvate*Oxalate&quot;)&#xA;&#xA;addParameters(glu, &quot;k_glucose_to_lactate&quot;, 0.02)&#xA;addParameters(glu, &quot;k_lactate_to_glucose&quot;, 0.001)&#xA;addParameters(glu, &quot;k_lactate_to_pyruvate&quot;, 0.01)&#xA;addParameters(glu, &quot;k_pyruvate_to_lactate&quot;, 0.0001)&#xA;addParameters(glu, &quot;k_pyruvate_to_oxalate&quot;, 0.01)&#xA;addParameters(glu, &quot;k_oxalate_to_pyruvate&quot;, 0.001)&#xA;addParameters(glu, &quot;k_acid&quot;, 0.0001)&#xA;addParameters(glu, &quot;k_base&quot;, 0.0001)&#xA;&#xA;addRule(glu, &quot;pH rule&quot;, &quot;ODEs&quot;, &quot;pH=-k_acid*Acid+k_base*Base&quot;)&#xA;&#xA;makeModel(glu)&#xA;simResults &amp;lt;- simulateModel(glu, seq(0,3,0.01))&#xA;plotResults(simResults, title=&quot;glu&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The head of the resulting data.frame is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  time   Glucose  Lactate   Pyruvate      Oxalate     Acid         Base       pH&#xA;1 0.00 1000.0000   0.0000  0.0000000 0.000000e+00   0.0000 0.000000e+00 7.400000&#xA;2 0.01  844.0001 155.0615  0.9384259 1.402282e-05 156.9384 1.402282e-05 7.399914&#xA;3 0.02  785.0869 210.3646  4.5476263 7.892761e-04 219.4615 7.892761e-04 7.399719&#xA;4 0.03  772.1557 218.6462  9.1924839 5.655403e-03 237.0425 5.655403e-03 7.399489&#xA;5 0.04  768.8212 217.2720 13.8876293 1.917807e-02 245.0856 1.917807e-02 7.399248&#xA;6 0.05  767.0383 214.4883 18.4278053 4.550630e-02 251.4350 4.550630e-02 7.399000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and resulting graph looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kg6bG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kg6bG.png&quot; alt=&quot;correct plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4059" LastActivityDate="2019-02-08T22:26:03.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6997" PostTypeId="2" ParentId="6995" CreationDate="2019-02-09T07:42:19.267" Score="1" Body="&lt;p&gt;The obvious answer is to break up the file size and upload each separately,&#xA;First I'd check the file size,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wc infile.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;XXXXX      YYYY     ZZZZZ infile.fa&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The number you want is XXXX - the number of lines in the file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'd then decide how many files you wanted to break it up into ... in your case divide XXXXX by 3. Then .. for example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -n '1,5000000p' infile.fa &amp;gt; outfile.fa&#xA;sed -n '5000001,2000000p' infile.fa &amp;gt; outfile2.fa&#xA;sed -n '2000001,4000000p' infile.fa &amp;gt; outfile3.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I'm not certain this would be cool however. An miRNA expert could answer this question very quickly. I think is you should check your data for sequence duplication. Simply piping the fasta file through a simple distance matrix and screening for &quot;0&quot; values would confirm this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Creating a frequency table of duplicate RNA must be a feature of Bioconductor, but in any case screening for duplication (technically duplicate records) is a formal SQL database command. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can be done a Perl/Python but its a bit clunky for a data set this size. I'd do it by setting the sequence as the key within a hash, the inefficiency has been solved (but its a weird way to do it). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would have a database for &quot;headers&quot; for a given sequence, which would take up space, but I would assess the potential level of duplication first.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-02-09T07:42:19.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6998" PostTypeId="2" ParentId="6995" CreationDate="2019-02-09T12:30:23.060" Score="1" Body="&lt;p&gt;The tool you want to use accepts compressed files, so the first thing you should do is compress your file. I created a 111M file of random, 51bp sequences, each with a random, 52 character header and a random 51 nt sequence:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ls -lh file.fa&#xA;-rw-r--r-- 1 terdon terdon 111M Feb  9 12:21 file.fa&#xA;&#xA;$ head file.fa&#xA;&amp;gt;YZF8SGLVBPCKMY18GJ94XTRCS61BB4OZKUZ61D 1J6R7H3 EJQA5 length=51&#xA;ACGAAGGGCATTTGGCAACGAACCTATACTATATCACGAGCAATTAGGCAG&#xA;&amp;gt;SPR0INGWKMILAXZZPF73BKRV73ELL1ROZSHYO1E9JX6P5X6MUZ20 length=51&#xA;TTAAGATTCCGAGAGAGTCTGTGGTTGGGTCCCTCTCCATAATGCTTTCAT&#xA;&amp;gt;U CB82L2M3BWJ9ZZZG3V6DL0Y0FTW8BXIFTU0TK7SL VN5QE42J6 length=51&#xA;TGGCTGGGAGGGATAGTAGCGAGATTATCCAGTCCGGTGTTCACAAGCCGG&#xA;&amp;gt;TGH62IM2TJK2QJTIVWGZR IVKOW94X7D52KGM7DJP61KKF81KPYA length=51&#xA;GCCGCAATACCAAAAAGATGGTGTGATTAGGAAATAAATAGGATCCAACAA&#xA;&amp;gt;L5VS9AVEF3UTEO3341STET092YEZ43ESVH32W7GI6Q27OIR NTXF length=51&#xA;CTATAGACGACTACGAACGACTGCAGGTGTGCCAGTTCTCGGTGAAACGGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If we now compress that file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ tar cvzf file.fa.tar.gz file.fa&#xA;file.fa&#xA;$&lt;/span&gt; ls -lh file.fa.tar.gz &#xA;-rw-r--r-- 1 terdon terdon 56M Feb  9 12:22 file.tar.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You will likely get a significantly better compression ratio since your headers won't be random and will likely be very repetitive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that isn't enough, you should try simplifying your headers. For instance, just use an increasing number:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ perl -pe 's/^&amp;gt;.*/&quot;&amp;gt;&quot; . ++$&lt;/span&gt;i/e' file.fa &amp;gt; file.short.fa&#xA;$ ls -lh file.short.fa &#xA;-rw-r--r-- 1 terdon terdon 58M Feb  9 12:23 file.short.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;See what a huge difference that one change made? And if we now compress the file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ tar cvzf file.short.tar.gz file.short.fa&#xA;file.short.fa&#xA;$&lt;/span&gt; ls -lh file.short.tar.gz&#xA;-rw-r--r-- 1 terdon terdon 19M Feb  9 12:24 file.short.tar.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Well below the threshold! &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If compression isn't enough, you should make sure you have no duplicate sequences in the file. Use the &lt;code&gt;FastaToTbl&lt;/code&gt; and &lt;code&gt;TblToFasta&lt;/code&gt; scripts I have posted &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2651/298&quot;&gt;here&lt;/a&gt; and:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FastaToTbl file.fa | sort | uniq | TblToFasta &amp;gt; file.uniq.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, if none of these helps enough, split your file into multiple smaller ones:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;split -l 100000 file.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That will create multiple files of 100000 lines (so 50000 sequences, assuming all your sequences are one line) each. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-02-09T12:30:23.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="6999" PostTypeId="1" AcceptedAnswerId="7000" CreationDate="2019-02-10T02:34:04.337" Score="1" ViewCount="873" Body="&lt;p&gt;I heard the term Feature Barcoding a lot so I find resources to understand it. I have read docs of 10xgenomics and I see that &lt;code&gt;Feature Barcoding technology is a method for adding extra channels of information to cells by running single-cell gene expression in parallel with other assays&lt;/code&gt;, this technology also do something with t-SNE and something with matrices. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't imagine the whole picture because I don't know what are those &lt;code&gt;adding extra channels of information to cells&lt;/code&gt;, &lt;code&gt;single-cell gene expression&lt;/code&gt; and &lt;code&gt;parallel with other assays&lt;/code&gt; mean. And what is the input and output of this technology? Is there any paper go along with this technology?&lt;/p&gt;&#xA;" OwnerUserId="3984" LastActivityDate="2019-02-10T19:50:52.333" Title="What is Feature Barcoding technology?" Tags="|barcode|features|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7000" PostTypeId="2" ParentId="6999" CreationDate="2019-02-10T12:12:19.620" Score="3" Body="&lt;p&gt;&quot;Feature Barcoding&quot; refers to using antibodies linked to specific oligos. The antibodies can then bind to cell surface markers of interest and the oligos they're conjugated with turned into normal scRNA-seq reads. This effectively means you can get a FACs profile for each cell without needing to run it through a FACs sorter and individually label it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;feature barcoding&quot; nomenclature comes from 10X I think, but there are a few similar protocols (maybe they're exactly the same in the end, I'm not sure about the details) such as &lt;a href=&quot;https://www.nature.com/articles/nmeth.4380&quot; rel=&quot;nofollow noreferrer&quot;&gt;CITE-seq&lt;/a&gt; and &lt;a href=&quot;https://www.nature.com/articles/nbt.3973&quot; rel=&quot;nofollow noreferrer&quot;&gt;REAP-seq&lt;/a&gt;. You can see a talk from 10X on this &lt;a href=&quot;https://www.youtube.com/watch?v=jzVMVAO7Af0&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2019-02-10T19:50:52.333" LastActivityDate="2019-02-10T19:50:52.333" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7001" PostTypeId="1" CreationDate="2019-02-10T19:01:26.587" Score="4" ViewCount="2959" Body="&lt;p&gt;I am trying to install DESeq2 in my Ubuntu with R version 3.5.1. According to the package repository in Bioconductor the version should be 3.5.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; R.version&#xA;&#xA;platform       x86_64-pc-linux-gnu         &#xA;arch           x86_64                      &#xA;os             linux-gnu                   &#xA;system         x86_64, linux-gnu           &#xA;status                                     &#xA;major          3                           &#xA;minor          5.1                         &#xA;year           2018                        &#xA;month          07                          &#xA;day            02                          &#xA;svn rev        74947                       &#xA;language       R                           &#xA;version.string R version 3.5.1 (2018-07-02)&#xA;nickname       Feather Spray&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then I attempt to install DESeq2, following the instructions in Bioconductor:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; if (!requireNamespace(&quot;BiocManager&quot;, quietly = TRUE))&#xA;       install.packages(&quot;BiocManager&quot;)&#xA;   BiocManager::install(&quot;DESeq2&quot;, version = &quot;3.8&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and at the end I get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Warning messages:&#xA;1: In install.packages(pkgs = doing, lib = lib, repos = repos, ...) :&#xA;  installation of package ‘RCurl’ had non-zero exit status&#xA;2: In install.packages(pkgs = doing, lib = lib, repos = repos, ...) :&#xA;  installation of package ‘GenomeInfoDb’ had non-zero exit status&#xA;3: In install.packages(pkgs = doing, lib = lib, repos = repos, ...) :&#xA;  installation of package ‘annotate’ had non-zero exit status&#xA;4: In install.packages(pkgs = doing, lib = lib, repos = repos, ...) :&#xA;  installation of package ‘GenomicRanges’ had non-zero exit status&#xA;5: In install.packages(pkgs = doing, lib = lib, repos = repos, ...) :&#xA;  installation of package ‘genefilter’ had non-zero exit status&#xA;6: In install.packages(pkgs = doing, lib = lib, repos = repos, ...) :&#xA;  installation of package ‘geneplotter’ had non-zero exit status&#xA;7: In install.packages(pkgs = doing, lib = lib, repos = repos, ...) :&#xA;  installation of package ‘SummarizedExperiment’ had non-zero exit status&#xA;8: In install.packages(pkgs = doing, lib = lib, repos = repos, ...) :&#xA;  installation of package ‘DESeq2’ had non-zero exit status&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have visited &lt;span class=&quot;math-container&quot;&gt;$2*10^{9999}$&lt;/span&gt; discussions on this issue and none of them helped me solving the problem. In addition, I didn't have any problem installing other packages like limma and edgeR, so I assume that this might be specific to DESeq2 or one of it's dependencies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for the help!&lt;/p&gt;&#xA;" OwnerUserId="2970" LastActivityDate="2019-02-11T01:47:18.953" Title="Installing DESeq2 in Ubuntu" Tags="|deseq2|software-installation|linux|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7002" PostTypeId="2" ParentId="7001" CreationDate="2019-02-10T23:11:40.113" Score="1" Body="&lt;p&gt;Have you tried just typing,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;install.packages(&quot;BiocManager&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Without all the proceeding if statement? If this doesn't install correctly nothing else will.  Personally I would be tempted install each of the named libraries independently. If that works okay, e.g. its not a problem with the mirrors then I'd contact the developer michaelisaiahlove at gmail.com&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-02-10T23:11:40.113" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7003" PostTypeId="2" ParentId="7001" CreationDate="2019-02-11T01:47:18.953" Score="4" Body="&lt;p&gt;First I installed this libraries:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sudo apt-get install libcurl4-openssl-dev libxml2-dev&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I then installed RCurl: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BiocManager::install(&quot;RCurl&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and lastly:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BiocManager::install(&quot;DESeq2&quot;, version = &quot;3.8&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2970" LastActivityDate="2019-02-11T01:47:18.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7004" PostTypeId="2" ParentId="6986" CreationDate="2019-02-11T09:38:07.903" Score="2" Body="&lt;p&gt;A cursory search to &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/closest.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools documentation&lt;/a&gt; will reveal the &lt;code&gt;bedtools closest&lt;/code&gt; feature - which might be exactly what you are looking in your third question. You can download the list of protein-coding genes from Ensembl biomart or UCSC table browser in bed format, and also convert the list of your lncRNAs into a bed compatible file. Rest is easy:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools closest -a lncRNAs.bed -b proteinCoding.bed [-iu/-id] -D ref&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1135" LastActivityDate="2019-02-11T09:38:07.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7005" PostTypeId="1" AcceptedAnswerId="8788" CreationDate="2019-02-11T17:53:56.843" Score="2" ViewCount="210" Body="&lt;p&gt;In creating a parser for the UniProt flat file &lt;code&gt;uniprot_sprot.dat&lt;/code&gt; (&lt;a href=&quot;ftp://ftp.uniprot.org/pub/databases/uniprot/current_release/knowledgebase/complete/&quot; rel=&quot;nofollow noreferrer&quot;&gt;directory&lt;/a&gt;) in the &lt;a href=&quot;https://www.uniprot.org/docs/userman.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt; for the section &lt;code&gt;SUBCELLULAR LOCATION&lt;/code&gt; is listed &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The format of SUBCELLULAR LOCATION is:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;CC   -!- SUBCELLULAR LOCATION:(( Molecule:)?( Location.)+)?(&#xA;  Note=Free_text( &lt;code&gt;Flag&lt;/code&gt;)?.)?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Where:  &lt;/p&gt;&#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Molecule: Isoform, chain or peptide name  &lt;/li&gt;&#xA;  &lt;li&gt;Location = Subcellular_location( &lt;code&gt;Flag&lt;/code&gt;)?(; Topology( &lt;code&gt;Flag&lt;/code&gt;)?)?(; Orientation( &lt;code&gt;Flag&lt;/code&gt;)?)? &#xA;  &#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;Subcellular_location: SL-line of subcell.txt ID-record &lt;/li&gt;&#xA;  &lt;li&gt;Topology: SL-line of subcell.txt IT-record &lt;/li&gt;&#xA;  &lt;li&gt;Orientation: SL-line of subcell.txt IO-record&lt;/li&gt;&#xA;  &lt;/ul&gt;&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What does &lt;code&gt;Flag&lt;/code&gt; mean?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My best guess at present is that these are the text surrounded by parenthesis, e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; Note=Not associated with microtubules (PubMed:21558460).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but that does not help in understanding why it is named flag.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I can use my parser to verify this is correct when this section is working, it would be nicer to have a better understanding of what the word flag means in this context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also checked the XML &lt;a href=&quot;ftp://ftp.uniprot.org/pub/databases/uniprot/current_release/knowledgebase/complete/uniprot.xsd&quot; rel=&quot;nofollow noreferrer&quot;&gt;schema&lt;/a&gt; but no help there.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Per request.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More examples from file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These where gathered using grep. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -A 3 -n 'CC   -!- SUBCELLULAR LOCATION'  /mnt/.../uniprot_sprot.dat     &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;71:CC   -!- SUBCELLULAR LOCATION: Host membrane {ECO:0000305}; Single-pass&#xA;72-CC       membrane protein {ECO:0000305}.&#xA;&#xA;2163:CC   -!- SUBCELLULAR LOCATION: Membrane {ECO:0000305}; Single-pass membrane&#xA;2164-CC       protein {ECO:0000305}.&#xA;&#xA;6003:CC   -!- SUBCELLULAR LOCATION: Secreted {ECO:0000305}.&#xA;&#xA;7039:CC   -!- SUBCELLULAR LOCATION: Cell membrane {ECO:0000305}; Lipid-anchor,&#xA;7040-CC       GPI-anchor {ECO:0000305}. Note=In microneme/rhoptry complexes.&#xA;7041-CC       {ECO:0000250}.&#xA;&#xA;7409:CC   -!- SUBCELLULAR LOCATION: Secreted {ECO:0000305}.&#xA;&#xA;8677:CC   -!- SUBCELLULAR LOCATION: Virion {ECO:0000269|PubMed:9499098}. Host&#xA;8678-CC       endoplasmic reticulum-Golgi intermediate compartment&#xA;8679-CC       {ECO:0000269|PubMed:15016891, ECO:0000269|PubMed:9499098}.&#xA;&#xA;12871:CC   -!- SUBCELLULAR LOCATION: Nucleus {ECO:0000250|UniProtKB:P48349}.&#xA;12872-CC       Cytoplasm {ECO:0000250|UniProtKB:P48349}. Note=Translocates from&#xA;12873-CC       the cytosol to the nucleus when phosphorylated.&#xA;12874-CC       {ECO:0000250|UniProtKB:P48349}.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -A 6 -n -m 200 'CC   -!- SUBCELLULAR LOCATION.*(.*)'  /mnt/.../uniprot_sprot.dat&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1560049:CC   -!- SUBCELLULAR LOCATION: Secreted. Note=Dense core vesicles (DCV).&#xA;&#xA;2452012:CC   -!- SUBCELLULAR LOCATION: DNA-(apurinic or apyrimidinic site) lyase,&#xA;2452013-CC       mitochondrial: Mitochondrion. Note=Translocation from the&#xA;2452014-CC       cytoplasm to the mitochondria is mediated by ROS signaling and&#xA;2452015-CC       cleavage mediated by granzyme A. Tom20-dependent translocated&#xA;2452016-CC       mitochondrial APEX1 level is significantly increased after&#xA;2452017-CC       genotoxic stress (By similarity). The cleaved APEX2 is only&#xA;2452018-CC       detected in mitochondria. {ECO:0000250}.&#xA;&#xA;9243083:CC   -!- SUBCELLULAR LOCATION: Isoform Soluble copper chaperone CopA(Z):&#xA;9243084-CC       Cytoplasm {ECO:0000250|UniProtKB:Q59385}.&#xA;&#xA;38197671:CC   -!- SUBCELLULAR LOCATION: E(rns) glycoprotein: Host cell membrane&#xA;38197672-CC       {ECO:0000269|PubMed:28290554}; Peripheral membrane protein. Virion&#xA;38197673-CC       membrane {ECO:0000269|PubMed:28290554}; Peripheral membrane&#xA;38197674-CC       protein {ECO:0000305}. Note=The C-terminus membrane anchor of Erns&#xA;38197675-CC       represents an amphipathic helix embedded in plane into the&#xA;38197676-CC       membrane.&#xA;&#xA;4924705:CC   -!- SUBCELLULAR LOCATION: Note=(Microbial infection) In the presence&#xA;4924706-CC       of M.tuberculosis EsxA-EsxB complex decreased amounts of B2M are&#xA;4924707-CC       found on the cell surface (PubMed:25356553).&#xA;4924708-CC       {ECO:0000269|PubMed:25356553}.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;02/18/2019&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the DE line it is noted&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A block of DE lines may further contain multiple Includes: and/or Contains: sections and a separate field &lt;code&gt;Flags&lt;/code&gt;: to indicate whether the protein sequence is a precursor or a fragment:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;At present this seems to be related to the answer. When I have the file correctly parsed, then I can check for a correspondence as verification.&lt;/p&gt;&#xA;" OwnerUserId="1586" LastEditorUserId="1586" LastEditDate="2019-02-19T01:45:43.510" LastActivityDate="2019-06-11T15:16:25.480" Title="UniProt flat file. Interpreting format of SUBCELLULAR LOCATION. What does flag mean?" Tags="|file-formats|uniprot|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7006" PostTypeId="2" ParentId="5178" CreationDate="2019-02-12T00:06:29.563" Score="2" Body="&lt;p&gt;It took me a while to figure out that the &quot;&lt;strong&gt;index&lt;/strong&gt;&quot; is the same thing as the &quot;&lt;strong&gt;barcode&lt;/strong&gt;&quot; that says which sample each sequence is from on a multiplexed run.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your data is &lt;em&gt;not demultiplexed&lt;/em&gt; (single &lt;code&gt;R1.fastq&lt;/code&gt; and &lt;code&gt;R2.fastq&lt;/code&gt; files contain the information for multiple samples), then this &lt;code&gt;I1.fastq&lt;/code&gt; file is what you use to assign each sequence to a sample (ie to &quot;demultiplex&quot;). The spreadsheet listing which barcode/index is which sample is often called a &quot;mapping file.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your data is &lt;em&gt;already demultiplexed&lt;/em&gt; (you have separate &lt;code&gt;R1.fastq&lt;/code&gt; and &lt;code&gt;R2.fastq&lt;/code&gt; files for each sample), then you don't really need it... however some people use the index sequences in their quality control workflows. For example, this 2016 paper in BMC Genomics by Wright and Vetsigien &quot;Quality filtering of Illumina index reads mitigates sample cross-talk&quot;: &lt;a href=&quot;https://doi.org/10.1186/s12864-016-3217-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1186/s12864-016-3217-x&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4110" LastActivityDate="2019-02-12T00:06:29.563" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7007" PostTypeId="1" CreationDate="2019-02-12T07:31:10.000" Score="8" ViewCount="1740" Body="&lt;p&gt;I would like to get a list of the licenses of all packages I installed using Conda. pip has what I need in the form of &lt;a href=&quot;https://pypi.org/project/pip-licenses/&quot; rel=&quot;noreferrer&quot;&gt;pip-licenses&lt;/a&gt;, is there a similar thing for conda?&lt;/p&gt;&#xA;" OwnerUserId="3964" LastEditorUserId="96" LastEditDate="2019-02-12T14:14:54.143" LastActivityDate="2022-10-01T04:40:59.277" Title="Getting the licenses of all conda installed packages" Tags="|bioconda|conda|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7008" PostTypeId="2" ParentId="7007" CreationDate="2019-02-12T08:00:33.833" Score="7" Body="&lt;p&gt;To my knowledge there's no single listing of the licenses, but you can:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conda list | awk '{if(NR&amp;gt;3) printf(&quot;%s=%s&quot;, &lt;span class=&quot;math-container&quot;&gt;$1, $&lt;/span&gt;2)}' | xargs conda info | grep license&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Allegedly &lt;code&gt;conda info --license&lt;/code&gt; should work, but at least in conda 4.6.3 it doesn't show any license information. It unfortunate that there's no convenient way to get a tabular list of package:version:channel:license information from the command line.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-02-12T08:00:33.833" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7009" PostTypeId="2" ParentId="5178" CreationDate="2019-02-12T10:13:59.027" Score="0" Body="&lt;p&gt;I looked at 10x data a couple of years ago, and at that time, their protocol would use four separate indexes, with balanced index sequences, as a way of introducing diversity into the library.  These four indexes were separate from the individual oligos used to mark single cells.  For an individual sample, you would need to combine the reads from all four separate indexes, which is how it looks like in your data.  There is a brief explanation here: &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/index/doc/specifications-sample-index-sets-for-single-cell-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.10xgenomics.com/single-cell-gene-expression/index/doc/specifications-sample-index-sets-for-single-cell-3&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-02-12T10:13:59.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7010" PostTypeId="1" AcceptedAnswerId="7012" CreationDate="2019-02-12T15:41:23.217" Score="4" ViewCount="18815" Body="&lt;p&gt;I tried to run the following code in R studio. Everything worked fine, except at the last step [write.table(mdat, &quot;recount_mdat.csv&quot;)] when I tried to export the 'mdat', I got the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in as.vector(x) : no method for coercing this S4 class to a vector&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; library('dplyr')&#xA; library('recount')&#xA; library('magrittr')&#xA; library('ffpe')&#xA; library('RSkittleBrewer')&#xA; library('SummarizedExperiment')&#xA; library('devtools')&#xA; trop &amp;lt;- RSkittleBrewer::RSkittleBrewer('tropical')&#xA; colon_proj &amp;lt;- c('SRP025982', 'SRP42161')&#xA; if(any(!file.exists(file.path(colon_proj, 'rse_gene.Rdata')))) {&#xA; sapply(colon_proj, download_study)&#xA; }&#xA;&#xA; blood_proj &amp;lt;- c('SRP066834', 'SRP041736', 'SRP060416')&#xA; if(any(!file.exists(file.path(blood_proj, 'rse_gene.Rdata')))) {&#xA; sapply(blood_proj, download_study)&#xA; }&#xA; proj &amp;lt;- c(colon_proj,blood_proj)&#xA;&#xA; dat &amp;lt;- lapply(proj, function(x) {&#xA; load(file.path(x, 'rse_gene.Rdata'))&#xA; return(rse_gene)&#xA; })&#xA; proj&#xA;&#xA; sapply(dat, dim)&#xA;&#xA; metadata &amp;lt;- all_metadata('sra')&#xA;&#xA; write.table(metadata, &quot;recount_metadata.csv&quot;)&#xA;&#xA; mdat &amp;lt;- do.call(cbind, dat)&#xA;&#xA; write.table(mdat, &quot;recount_mdat.csv&quot;)&#xA;&#xA;is(mdat)&#xA;## &quot;RangedSummarizedExperiment&quot; &quot;SummarizedExperiment&quot; &quot;Vector&quot; &quot;Annotated&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3766" LastEditorUserId="48" LastEditDate="2019-02-13T09:32:32.917" LastActivityDate="2019-02-13T09:32:32.917" Title="Error in as.vector(x) : no method for coercing this S4 class to a vector" Tags="|rna-seq|bioconductor|rstudio|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7011" PostTypeId="1" CreationDate="2019-02-12T16:48:19.153" Score="2" ViewCount="262" Body="&lt;p&gt;How can I get plant Genome size, total number of genes, EST and chromosome number from phytozom and plantDB?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wtr33.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wtr33.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4114" LastEditorUserId="2719" LastEditDate="2019-02-13T22:40:52.093" LastActivityDate="2019-02-15T10:57:24.337" Title="How can I get plant Genome size, total number of genes, EST, and chromosomes number" Tags="|public-databases|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7012" PostTypeId="2" ParentId="7010" CreationDate="2019-02-12T17:10:02.887" Score="6" Body="&lt;p&gt;You can't cbind a bunch of obscure object types.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want merged count tables you should do this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mdat &amp;lt;- do.call(cbind,lapply(dat,assay))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where &lt;strong&gt;row.names&lt;/strong&gt; are &lt;em&gt;Ensembl gene IDs&lt;/em&gt; and &lt;strong&gt;col.names&lt;/strong&gt; are the &lt;em&gt;SRR accessions&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then run your table writing command.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want the coordinates of your genes then do this to make a bed with the genomic locations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(rtracklayer)&#xA;export.bed(rowRanges(dat[[1]]),'gene_coords.bed')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="383" LastActivityDate="2019-02-12T17:10:02.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7013" PostTypeId="1" AcceptedAnswerId="7014" CreationDate="2019-02-12T20:30:16.290" Score="1" ViewCount="97" Body="&lt;p&gt;I am new to bash and the processes behind cluster computing in general and need some help with understanding some basics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After looking all over the internet and this forum (+ askUbuntu) I found nothing that addresses this issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a series of raw RNA sequencing files: .bam and wish to utilize samtools (or others) to begin the data preprocessing steps defined in many workflows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have installed samtools and many other programs necessary for the workflow, however, once I am on the remote server in bash, and try and run a function over the files of interest, it gives me the error: &lt;code&gt;-bash-samtools: command not found&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I try to install the package, say samtools, by running &lt;code&gt;sudo apt install samtools&lt;/code&gt;, I get an error stating I do not have permission to do such.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do I do? Must I bring up this issue to the server provider or is there a way around it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your time,&lt;/p&gt;&#xA;" OwnerUserId="4001" LastActivityDate="2019-02-13T15:33:53.453" Title="Using preprocessing/alignment functions on the server" Tags="|rna-seq|scrnaseq|samtools|bash|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7014" PostTypeId="2" ParentId="7013" CreationDate="2019-02-12T20:40:14.337" Score="3" Body="&lt;p&gt;The short answer is to &lt;a href=&quot;https://docs.conda.io/en/latest/miniconda.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;use conda&lt;/a&gt;. In the &lt;a href=&quot;https://bioconda.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioconda channel&lt;/a&gt; we have most of the tools used in bioinformatics, such as samtools. You do not need administrator permissions to install conda or packages with it, so your lack of &lt;code&gt;sudo&lt;/code&gt; ability is not an issue.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I have installed samtools and many other programs necessary for the workflow, however, once I am on the remote server in bash, and try and run a function over the files of interest, it gives me the error&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Installing packages on your local machine won't affect the remote server unless they share a file system, so this is expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an aside, BAM files aren't really &quot;raw&quot;, they're considered processed data (fastq/fast5 files are considered truly &quot;raw&quot; data in the sequencing world).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-02-12T20:40:14.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7015" PostTypeId="2" ParentId="7013" CreationDate="2019-02-12T22:05:09.693" Score="2" Body="&lt;p&gt;At a very wild guess&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-bash-samtools&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This looks shaky. For a start you need a space between the command and the application in all Linux/Unix/Mac OS X applications (its complicated to explain right now [see below] just accept it).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Wait you are using &lt;code&gt;sudo&lt;/code&gt; and thats how you are installing on a local machine. &lt;code&gt;sudo&lt;/code&gt; on a remote machine will never work, because you are not the systems admin. &lt;code&gt;sudo&lt;/code&gt; on a local machine is not cool (see below)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Anyway I need to do some next-gen and have 6 MiSeq kits waiting for action so I installed it from here:&#xA;&lt;a href=&quot;http://www.htslib.org/download/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.htslib.org/download/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Okay so here's my tcsh history&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 4  21:44   cd sam* # like I was in the Desktop&#xA; 7  21:44   more README # I needed to know how they wanted the installation&#xA; 8  21:45   ./configure # okay there are setting the paths for me - saves me a job&#xA;11  21:46   make # like fairytale stuff for a load of C scripts&#xA;16  21:49   ls | grep samtools # I wanna know if its compiled and I see samtools* ... yep&#xA;17  21:49   ln -s ./samtools* ~/bin # I'm gonna pop a link in my bin so after I type&#xA;18  21:49   rehash # 'cause I still insist on tcsh rather than bash (don't do it in bash)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, the following is a bit creepy because I get loads of perl scripts in my /usr/local/bin ...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;make install&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Anyway,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It took 4 mins on an OS X machine, using tsch, after I just quaffed 2 pints at my  pub (10pm London time). So probability is set against me .... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;OUTPUT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Program: samtools (Tools for alignments in the SAM format)&#xA;Version: 0.1.18 (r982:295)&#xA;&#xA;Usage:   samtools &amp;lt;command&amp;gt; [options]&#xA;&#xA;Command: view        SAM&amp;lt;-&amp;gt;BAM conversion&#xA;         sort        sort alignment file&#xA;         mpileup     multi-way pileup&#xA;         depth       compute the depth&#xA;         faidx       index/extract FASTA&#xA;         tview       text alignment viewer&#xA;         index       index alignment&#xA;         idxstats    BAM index stats (r595 or later)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;etc ...&#xA;BTW seriously, fun over, avoid &quot;&lt;code&gt;sudo&lt;/code&gt;&quot; if you can, for lots of reasons (which I'm not gonna explain). Its a last resort and usually means your paths are incorrectly set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;sudo will be refused on a remote machine, without question. You COULD ask systems admin to do this for you. HOWEVER, I'd try and install yourself without &lt;code&gt;sudo&lt;/code&gt; and your bioinformatics will benefit.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Program: samtools (Tools for alignments in the SAM format)&#xA;Version: 1.9 (using htslib 1.9)&#xA;&#xA;Usage:   samtools &amp;lt;command&amp;gt; [options]&#xA;&#xA;Commands:&#xA;  -- Indexing&#xA;     dict           create a sequence dictionary file&#xA;     faidx          index/extract FASTA&#xA;     fqidx          index/extract FASTQ&#xA;     index          index alignment&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt; following &lt;code&gt;make install&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-02-13T15:33:53.453" LastActivityDate="2019-02-13T15:33:53.453" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7016" PostTypeId="1" CreationDate="2019-02-13T03:09:22.230" Score="4" ViewCount="128" Body="&lt;p&gt;I have around 500 annotated proteomes of different bacterial strains and would like to quantify their similarity (or difference). I found &lt;code&gt;gt genomediff&lt;/code&gt; from &lt;a href=&quot;http://genometools.org/tools/gt_genomediff.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;genometools&lt;/a&gt; gives me some scores that I can use to generate nice clusters, but I am not sure whether that tools really works. The fasta-files that I use contain multiple sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran some tests and it looks ok.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1_reference.fna:&#xA;&amp;gt;1&#xA;TAAGTTACT&#xA;&amp;gt;2&#xA;TAAGTTACA&#xA;&#xA;2_eq_to_ref.fna:&#xA;&amp;gt;1&#xA;TAAGTTACT&#xA;&amp;gt;2&#xA;TAAGTTACA&#xA;&#xA;3_tags_diff.fna:&#xA;&amp;gt;1asdfadf&#xA;TAAGTTACT&#xA;&amp;gt;2asdfasdfffa&#xA;TAAGTTACA&#xA;&#xA;4_orde_diff.fna:&#xA;&amp;gt;2&#xA;TAAGTTACA&#xA;&amp;gt;1&#xA;TAAGTTACT&#xA;&#xA;5_add_subse.fna:&#xA;&amp;gt;1&#xA;TAAGTTACT&#xA;&amp;gt;2&#xA;TAAGTTACA&#xA;&amp;gt;2&#xA;TTACA&#xA;&#xA;6_point_mut.fna:&#xA;&amp;gt;1&#xA;AAAGTTACT&#xA;&amp;gt;2&#xA;TAAGTTACA&#xA;&#xA;7_different.fna:&#xA;&amp;gt;1&#xA;TAAGTTACT&#xA;ATTACCTAA&#xA;&amp;gt;2&#xA;AAAAAAAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gt genomediff --indexname test *fna&#xA;7&#xA;1_reference.fna 0.000000        0.000000        0.000000        0.000000        0.206969        0.199527        0.794496&#xA;2_eq_to_ref.fna 0.000000        0.000000        0.000000        0.000000        0.206969        0.199527        0.794496&#xA;3_tags_diff.fna 0.000000        0.000000        0.000000        0.000000        0.206969        0.199527        0.794496&#xA;4_orde_diff.fna 0.000000        0.000000        0.000000        0.000000        0.206969        0.199527        0.794496&#xA;5_add_subse.fna 0.206969        0.206969        0.206969        0.206969        0.000000        0.212596        1.349206&#xA;6_point_mut.fna 0.199527        0.199527        0.199527        0.199527        0.212596        0.000000        0.569180&#xA;7_different.fna 0.794496        0.794496        0.794496        0.794496        1.349206        0.569180        0.000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am running the clustering on the difference matrix calculated with genomediff: &quot;These distances are Jukes-Cantor corrected divergence between the pairs of genomes, that is, the number of mutations per base between them.&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, we are studying S aureus. We genomes are assembled genomes (three different methods). My guess is that the sequences from the plasmids are present. Furthermore, we do have drug-resistance measured in culture. So, we will be able to compare the genomes and the resistances. &lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3848" LastEditDate="2023-07-08T20:46:29.740" LastActivityDate="2023-07-08T20:46:29.740" Title="How to quantify similarity of genomes and find differences in set of S aureus genomes?" Tags="|phylogenetics|sequence-alignment|phylogeny|bacteria|genometools|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7017" PostTypeId="1" CreationDate="2019-02-13T10:21:03.330" Score="2" ViewCount="488" Body="&lt;p&gt;I am very new to bioinformatics and Linux and am trying to run psipred locally on Ubuntu 18.04 but cannot get PSI-BLAST to work. I have downloaded BLAST legacy version 2.2.26 and unpacked it, uniref90 database and psipred version 4.01&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I installed psipred according to instructions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cd psipred/src&#xA;make&#xA;make install&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this seems to work fine&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used the following code to create a database, which proceeded without errors:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;makeblastdb -in uniref90.fasta -dbtype prot -out uniref90&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In runpsipred I changed the directories:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# The name of the BLAST data bank&#xA;set dbname = /home/hilda/uniref90&#xA;&#xA;# Where the NCBI programs have been installed&#xA;set ncbidir = /home/hilda/blast/bin&#xA;&#xA;# Where the PSIPRED V4 programs have been installed&#xA;set execdir = ./bin&#xA;&#xA;# Where the PSIPRED V4 data files have been installed&#xA;set datadir = ./data&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I then try to run psipred:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tsch ./runpsipred example/example.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And get the error&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Running PSI-BLAST with sequence example/example.fasta ...&#xA;FATAL: Error whilst running blastpgp - script terminated!&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am not sure how to solve this as I do not understand what is going wrong. As I said I am new to Linux. I use the command tsch to run psipred (otherwise I get the error &quot;command not found&quot;), but not to install the database, could this be a problem? Or have I missed something in the setup process? Running psipred does create a copy of the example.fasta in the psipred folder named psitmp4286007f010.fasta, but that is all that seems to happen. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would greatly appreciate any suggestions on how to make it work!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&#xA;I tried to run with BLAST+ (which seemed to work without tsch) and get the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BLAST Database error: No alias or index file found for protein database [/home/hilda/uniref90] in search path [/home/hilda/psipred::]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have also tried setting dbname to home/hilda/uniref90/uniref90 and got the same results. there is a uniref90.pal file in the uniref90 directory, which I understand is the alias file? Other forum answers seem to use pfilt to index (?) the database, but there is no pfilt in my psipred directory. Any ideas how to fix the problem?&lt;/p&gt;&#xA;" OwnerUserId="4118" LastEditorUserId="4118" LastEditDate="2019-02-14T10:19:18.670" LastActivityDate="2021-02-01T02:20:09.740" Title="Error in PSI-BLAST when running psipred locally" Tags="|software-installation|psi-blast|makeblastdb|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7018" PostTypeId="2" ParentId="7017" CreationDate="2019-02-13T14:13:17.787" Score="4" Body="&lt;p&gt;This error could be due to the fact that you are using legacy blast, have you tried using BLAST+ instead?&#xA;If you wish to use PSIPRED with BLAST+, then use the runpsipredplus script&#xA;here, rather than the normal runpsipred script.&lt;/p&gt;&#xA;" OwnerUserId="4121" LastActivityDate="2019-02-13T14:13:17.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7019" PostTypeId="1" CreationDate="2019-02-13T17:07:17.383" Score="6" ViewCount="6169" Body="&lt;p&gt;I have around a hundred Fasta files (and will collect several thousand) with DNA sequences and +50x coverage. What is a recommended method to construct a phylogenetic tree? Solutions in Python or R are sought.&lt;/p&gt;&#xA;&lt;p&gt;I found &lt;code&gt;Phylo&lt;/code&gt; from &lt;code&gt;Biopython&lt;/code&gt; only handles already calculated trees.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2020-09-11T22:31:20.267" LastActivityDate="2023-10-06T15:59:59.727" Title="How to create Phylogenetic Trees from fasta files in Python or R?" Tags="|sequence-alignment|phylogenetics|python|phylogeny|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7020" PostTypeId="2" ParentId="7019" CreationDate="2019-02-13T18:41:17.643" Score="3" Body="&lt;p&gt;The obvious single answer is R &quot;ape&quot;. This will give you access to PhylML for tree building and Clustal/Muscle for alignment building. The paths to the binarys are important. There are several distance methods in there such as NJ and BIONJ. Its distance approaches however don't look mainstream, but I could be wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are functions within ape which are cool, the tree sorting is very cool and I need to read through this with much greater care. Personally I wouldn't perform a core phylogenetic analysis within R, because the standalones are sufficient and the analysis is intensive. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://cran.r-project.org/web/packages/ape/ape.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cran.r-project.org/web/packages/ape/ape.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-02-13T18:41:17.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7021" PostTypeId="2" ParentId="7019" CreationDate="2019-02-13T21:14:56.727" Score="7" Body="&lt;p&gt;I would not look for a package for this, but instead build a small pipeline calling external tools with something like the following workflow:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Cluster the ~100 sequences with &lt;a href=&quot;https://sites.google.com/view/cd-hit&quot; rel=&quot;nofollow noreferrer&quot;&gt;CD-HIT-EST/PSI-CD-HIT&lt;/a&gt; or many other options&lt;/li&gt;&#xA;&lt;li&gt;Take all the sequences that form one individual cluster and build a multiple sequence alignment (MSA) with &lt;a href=&quot;https://mafft.cbrc.jp/alignment/software/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MAFFT&lt;/a&gt;/&lt;a href=&quot;http://www.clustal.org/omega/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ClustalOmega&lt;/a&gt; or similar&lt;/li&gt;&#xA;&lt;li&gt;Take the MSA and build a phylogenetic tree with a Maximum-Likelihood approach like &lt;a href=&quot;http://www.iqtree.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IQ-TREE&lt;/a&gt; or similar&lt;/li&gt;&#xA;&lt;li&gt;Visualize the tree file with &lt;a href=&quot;https://www.jalview.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jalview&lt;/a&gt; or similar&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Of course this is rather general and depending on exactly what you're doing you may want a different workflow and/or different tools. You should also explore the parameter space, &lt;em&gt;do not&lt;/em&gt; assume the defaults are necessarily good choices&lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="17320" LastEditDate="2023-10-05T20:47:57.320" LastActivityDate="2023-10-05T20:47:57.320" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7022" PostTypeId="2" ParentId="7016" CreationDate="2019-02-13T23:22:23.957" Score="4" Body="&lt;p&gt;... stuff of original post deleted.&#xA;On second thoughts what you might be doing is a template assembly of your genomes. It is a possible interpretation of the 1.2. fasta sequences above (i.e. 1 is template).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A microbial genomics professional would advise also performing a de novo assembly, particularly if you are interested in presence and absence. The reason is that if a gene is present in your query and absent in your template, you will miss it. Again it all depends on what bacteria are being assessed, some are more prone to &quot;genetic islands&quot; than others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need a collaborator beyond that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the onlookers here bacterial genetic behaviour is very different to eukaryotes and what they get up, switching DNA etc.., to would appear bizzare from a higher eukaryotic world.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;You mentioned a comparative analysis of 500 strains: I have worked on this in MRSA. Note, the bacteria is important considering the approach you will adopt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, you want a single aligned file to produce a phylogeny, Bayes or likelihood. This is a model of point mutations. Bootstrapping doesn't really help because of SNP differences between isolates. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a complex problem. Generic phylogeny can hit the buffers in my opinion because often:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;low numbers of SNP differences between isolates.&lt;/li&gt;&#xA;&lt;li&gt;The error across the genome is poorly defined&lt;/li&gt;&#xA;&lt;li&gt;Multi-clonal infection ignored and your isolates not cloned. &lt;/li&gt;&#xA;&lt;li&gt;The other issue is the QC of the genome, I've been fairly astonished by the variation (it affects the tree)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Put it all together and you can get a nice tree but the topology of a given MLST is in my opinion unlikely to be correct. A referee may not bother with this, but they might.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You then map the phenotype against the phylogeny (the tips of the tree) and look for clusters. HOWEVER, there are problems:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Drug-resistant e.g. mec cassette (methicillin-resistant ) are often on plasmids and these are lost during isolation. Albeit they can integrate onto the genome. So an isolate can be drug resistance but you fail to find the gene.&lt;/li&gt;&#xA;&lt;li&gt;Drug-resistance transmission is unlikely to cluster, its too quick, so you don't see nice tight clusters against a theoretically perfect tree&lt;/li&gt;&#xA;&lt;li&gt;The best approach is drug-resistance in culture.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Hierarchical clustering, has many meanings. It is used in presence/absence data, I demonstrated the method failed (on bacteria) based on cluster analysis. It is currently revised and used in unsupervised deep (or machine) learning prior a training method.  The question then is what is what are you modelling? Presence/absence (of genes), point mutations, epidemiological data, drug-resistance/non-drug resistance? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My assessment is that given the potential complexity of these biological scenarios and the quality of the question you should seek formal collaboration and this is before considering what bacteria are being assessed. Some of epidemic drug-resistent bacteria rip up this rule book.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-02-14T16:36:27.230" LastActivityDate="2019-02-14T16:36:27.230" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7023" PostTypeId="1" AcceptedAnswerId="7038" CreationDate="2019-02-14T05:55:15.080" Score="4" ViewCount="427" Body="&lt;p&gt;Currently I'm performing whole analysis (pipeline from *.fastq to *.vcf) of 41  samples (targeted NGS). I rely on GATK best practices, however with some modifications. I decided to use the following tools:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#mapping&#xA;bwa mem (with mem - alternate haplotypes)&#xA;&#xA;#preprocessing&#xA;RevertSam (Picard)&#xA;MergeBamAlignment (Picard)&#xA;MarkDuplicates (Picard)&#xA;BaseRecalibrator (GATK)&#xA;ApplyBQSR (GATK)&#xA;AnalyzeCovariates (GATK)&#xA;SortSam (Picard)&#xA;&#xA;#call, filter and annotate variants&#xA;HaplotypeCaller (GATK)&#xA;GenotypeGVCFs (GATK)&#xA;snpEff, SnpSift&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Nevertheless, after marking the duplicates with &lt;code&gt;MarkDuplicates&lt;/code&gt; about 99% of reads have been marked as duplicated, hence the coverage decreased from several hundreds to literally up to five. So I decided to omit this step.&lt;/p&gt;&#xA;&lt;p&gt;Have you ever encountered a similar problem? What is your opinion about this?&lt;/p&gt;&#xA;&lt;p&gt;Best Regards,&#xA;Adam&lt;/p&gt;&#xA;&lt;h1&gt;EDIT&lt;/h1&gt;&#xA;&lt;p&gt;The Agilent Haloplex custom target enrichment kit (HaloPlex Target Enrichment for Illumina Kit) has been used to prepare the libraries, sequencing done using Illumina HiSeq 2500. Targeted NGS of all exons (only) of 90 genes.&lt;/p&gt;&#xA;&lt;p&gt;Here I provide how the final bam look like. This is the result without duplicate marking, otherwise there're only several reads per exon...&#xA;&lt;a href=&quot;https://i.stack.imgur.com/k8yze.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/k8yze.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2869" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2019-02-15T10:23:45.743" Title="Targeted NGS, up to 99% of reads have been marked as duplicates" Tags="|gatk|markduplicate|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7024" PostTypeId="2" ParentId="7023" CreationDate="2019-02-14T06:31:30.423" Score="3" Body="&lt;p&gt;If you're doing target capture, especially of a small region, it is entirely possible that there are only five unique reads. Target capture is molecularly finicky and sometimes the entire region is not captured.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2019-02-14T06:31:30.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7025" PostTypeId="1" CreationDate="2019-02-14T13:23:25.143" Score="1" ViewCount="66" Body="&lt;p&gt;I want to download the next generation RNA sequencing data in different cell lines. I want the data (alongwith normal) for p53 wild type, p53 knock-out and p53 null cell lines (human colon cancer). From where do I download this data? In fastq format if possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I am going through SRA, to look for the datasets, using their search engine. But the process is extremely tedious. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also can someone please hint about how to download the data from SRA, in case that's the only way to get data. I'm getting very confused. &lt;/p&gt;&#xA;" OwnerUserId="4127" LastEditorUserId="4127" LastEditDate="2019-02-14T13:29:45.067" LastActivityDate="2019-02-15T15:23:54.873" Title="Retrieve RNA sequencing data for human p53 colon cancer cell lines" Tags="|sequence-analysis|rna|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7026" PostTypeId="1" AcceptedAnswerId="7040" CreationDate="2019-02-14T16:43:48.907" Score="2" ViewCount="5130" Body="&lt;p&gt;I am analysing my single cell RNA seq data with the Seurat package. I want to know if there is a possibilty to obtain the percentage expression of a list of genes per identity class, as actual numbers (e.g. a matrix) which I can write out to say an excel file. For example I want to be able to say gene X is expressed in 30% of cell A, 10% of cell B etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can generate something comparable with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DotPlot(object = my_object, genes.plot = &quot;my_gene&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However the results are only graphic and I wish to have further processible numbers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AverageExpression(object, genes.use = &quot;my_gene&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Produces expression values which I cannot transform to percentages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will be very grateful on any hints.&lt;/p&gt;&#xA;" OwnerUserId="3968" LastEditorUserId="48" LastEditDate="2019-02-14T16:51:21.627" LastActivityDate="2020-04-02T06:49:41.693" Title="How can I obtain the percentage gene expression per identity class in Seurat as further processible numbers (e.g. matrix)?" Tags="|r|scrnaseq|seurat|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7027" PostTypeId="1" CreationDate="2019-02-14T19:19:04.730" Score="5" ViewCount="3088" Body="&lt;p&gt;I wish to extract metadata from a list of runs on NCBI's short read archive. For instance, I'd like to extract the library name (&quot;HS0798&quot;) from the following run info:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/sra.cgi?view=run_browser&amp;amp;run=SRR390728&quot; rel=&quot;noreferrer&quot;&gt;https://trace.ncbi.nlm.nih.gov/Traces/sra/sra.cgi?view=run_browser&amp;amp;run=SRR390728&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to extract metadata for a few hundred run numbers. Is there a existing command-line tool (sratoolkit, python, perl, bash, etc) that will extract the &lt;em&gt;metadata&lt;/em&gt; about the run?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could grep it out of a webpage dump, but I notice there are XML schemas provided on the site, which to me indicates that there's some sort of XML API available somewhere. &lt;/p&gt;&#xA;" OwnerUserId="3235" LastActivityDate="2022-05-09T11:59:55.393" Title="How to extract metadata from NCBI's short read archive (SRA) for a few runs?" Tags="|phylogenetics|sratoolkit|" AnswerCount="6" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7028" PostTypeId="2" ParentId="7027" CreationDate="2019-02-14T19:39:45.057" Score="1" Body="&lt;p&gt;Its got to be the &lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/sra.cgi?view=software&quot; rel=&quot;nofollow noreferrer&quot;&gt;SRA Toolkit&lt;/a&gt;&#xA;The way I would do it (a traditional Perl digger) doesn't work on the Tracer NCBI site, at least not readily and dumping the XML is likewise unlikely to succeed.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-02-14T19:39:45.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7029" PostTypeId="1" CreationDate="2019-02-14T21:07:59.580" Score="7" ViewCount="378" Body="&lt;p&gt;I am using &lt;a href=&quot;https://github.com/ruanjue/wtdbg2&quot; rel=&quot;noreferrer&quot;&gt;wtdbg2&lt;/a&gt; 2.3 to assemble a human genome (sequenced on PromethION from a cell line). I filtered out reads with low average quality, and now I am trying to determine the parameters that will optimize overall assembly quality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The wtdbg2 assembler has two k-mer related settings, documented as follows.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; -k &amp;lt;int&amp;gt;    Kmer fsize, 0 &amp;lt;= k &amp;lt;= 25, [0]&#xA; -p &amp;lt;int&amp;gt;    Kmer psize, 0 &amp;lt;= p &amp;lt;= 25, [21]&#xA;             k + p &amp;lt;= 25, seed is &amp;lt;k-mer&amp;gt;+&amp;lt;p-homopolymer-compressed&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The requirement that k + p is &amp;lt;= 25&lt;sup&gt;**&lt;/sup&gt; suggests that these parameters can be set simultaneously. However, in all of the presets for the various long read sequencing technologies, one of the two settings is always set to 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are as follows.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Are &lt;code&gt;k&lt;/code&gt; and &lt;code&gt;p&lt;/code&gt; ever intended to be used simultaneously?&lt;/li&gt;&#xA;&lt;li&gt;If so, how does the behavior change with &lt;code&gt;-k 2 -p 19&lt;/code&gt; versus &lt;code&gt;-k 19 -p 2&lt;/code&gt;?&lt;/li&gt;&#xA;&lt;li&gt;I'm familiar with some common implications of choosing k size for k-mer based analyses of Illumina reads. (Smaller k-sizes are more sensitive but also more subject to spurious matches. Larger k-sizes are more specific but also more likely to include a sequencing error.) Do similar implications apply for &lt;code&gt;-k&lt;/code&gt; and &lt;code&gt;-p&lt;/code&gt; with wtdbg2?&lt;/li&gt;&#xA;&lt;li&gt;BONUS: Is there an explanation anywhere of how &lt;code&gt;-k&lt;/code&gt; and &lt;code&gt;-p&lt;/code&gt; are used? It looks to me like &lt;code&gt;-k&lt;/code&gt; is related to k-mers in the traditional sense, and &lt;code&gt;-p&lt;/code&gt; is a homopolymer compressed k-mer. Are seeds made by concatenating the regular k-mer and the compressed k-mer? It's possible I missed some important details in &lt;a href=&quot;https://doi.org/10.1101/530972&quot; rel=&quot;noreferrer&quot;&gt;the preprint&lt;/a&gt; but I haven't been able to make the connection yet.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;**&lt;/sup&gt;Incidentally, the documentation here is wrong. During runtime wtdbg2 enforces a constraint of &amp;lt;= 23, not 25. This matches the description in the bioRxiv preprint stating that 46 bits (2 bits per base) are used to store each k-mer.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2021-01-29T20:04:40.543" Title="wtdbg2: practical implications of k-mer fsize and psize choice" Tags="|genome|nanopore|assembly|long-reads|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7031" PostTypeId="1" CreationDate="2019-02-14T21:11:21.547" Score="1" ViewCount="147" Body="&lt;p&gt;I have TMT tagged MS2 mass spec run processed with Peptideshaker. I saved the project in a peptideshaker format &lt;code&gt;cpsx&lt;/code&gt;. Now, I would like to do the quantification with Reporter. I can load the file with Reporter GUI. Though Reporter is still in development and cannot be run from commandline yet. Are there other tools available that can run from Peptideshaker exported reports for downstream analysis? I tried to run Reporter from commandline version 0.7.16. I will try 0.7.17 which was just released.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2019-02-15T16:55:01.247" LastActivityDate="2019-02-15T16:55:01.247" Title="How to do protein quantification with Peptideshaker output (TMT labeled, MS2)?" Tags="|software-recommendation|mass-spectrometry|proteomics|peptide-shaker|tmt|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7032" PostTypeId="1" CreationDate="2019-02-14T21:47:36.803" Score="6" ViewCount="182" Body="&lt;p&gt;So if you see the BLAST parameters it says &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The Expected value E  is a parameter that describes the number of hits one can &quot;expect&quot; to get by chance when searching a database of particular size. It decreases exponentially as the score (S) increases. Essential, the E value describes the randomness background noise. For example an E value of 1 assigned to a hit can be interpreted as meaning that a database of the current size one might expect to see 1 match with a similar score simply by chance. Lower the E value the better.&quot; &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How could there be something &quot;BY CHANCE&quot; it's a software right ?&lt;/p&gt;&#xA;" OwnerUserId="4131" LastActivityDate="2019-02-18T09:18:08.300" Title="Randomness in BLAST" Tags="|blast|" AnswerCount="3" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7033" PostTypeId="2" ParentId="7032" CreationDate="2019-02-14T22:21:20.533" Score="0" Body="&lt;p&gt;Calculation of E for Blast is a good question. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two methods:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use a Poison or binomial distribution.&lt;/li&gt;&#xA;&lt;li&gt;Use randomization&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I think its no. 1 because no. 2 would use too much computational power.  I did know but someone borrowed by O'Reilly book on the subject.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the contributors here will know, because I suspect their genomics developers.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-02-15T12:03:35.940" LastActivityDate="2019-02-15T12:03:35.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7034" PostTypeId="1" CreationDate="2019-02-14T22:24:29.063" Score="9" ViewCount="996" Body="&lt;p&gt;I am new to Nanopore sequencing analysis. I have a couple of questions regarding it which are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How do I know if my fast5 file is multiread or single read file? &lt;/li&gt;&#xA;&lt;li&gt;Is there a way to combine all the fast5 files into a single fast5 file?&lt;/li&gt;&#xA;&lt;li&gt;How can I extract the flowcell type and sequencing kit information from the fast5 file? Is there a command that I can use to extract it? I know I can use the following command to retrieve it: &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;h5dump -a /UniqueGlobalKey/context_tags/sequencing_kit -a /UniqueGlobalKey/context_tags/flowcell_type &amp;lt;fast5 file&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, it works well for single read file and not for multiread files. Please correct me if I am wrong. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="4132" LastActivityDate="2024-01-21T23:05:50.453" Title="Questions regarding Nanopore sequencing analysis" Tags="|nanopore|long-reads|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7035" PostTypeId="2" ParentId="7027" CreationDate="2019-02-15T00:32:55.693" Score="2" Body="&lt;p&gt;Here's a very crude script version, which should work without downloading fancy software tools. It will only work as long as the website stays in its current layout.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It extracts the particular &lt;code&gt;&amp;lt;td&amp;gt;&lt;/code&gt; from the webpage, based on a nearby table link ID. and prints it out. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: This is brittle. It will only work for library names that fit in a single-line of the HTML, and will ignore some webserver errors. The content is also not html-decoded (so its output might show things like &lt;code&gt;&amp;amp;amp;&lt;/code&gt; if the field contained html-special characters).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;set -e -o pipefail&#xA;&#xA;# pass the run as a single argument to the script&#xA;run=&quot;$1&quot;&#xA;&#xA;# extract library name field. second cell in visual order.&#xA;curl -o - &quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/sra.cgi?view=run_browser&amp;amp;run=&lt;span class=&quot;math-container&quot;&gt;$run&quot; 2&amp;gt;/dev/null | \&#xA;   grep 'a id=&quot;exp-accession&quot;' -A 7 | \&#xA;   tail -n +3 | head -n 1 | \&#xA;   sed -e 's@^&amp;lt;td&amp;gt;@@' -e 's@&amp;lt;/td&amp;gt;$&lt;/span&gt;@@'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3235" LastEditorUserId="3235" LastEditDate="2019-03-02T04:08:23.427" LastActivityDate="2019-03-02T04:08:23.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7036" PostTypeId="2" ParentId="7027" CreationDate="2019-02-15T01:27:14.823" Score="10" Body="&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez Direct&lt;/a&gt; is perfect for this sort of thing. You can do something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ esearch -db sra -query 'SRR390728' \&#xA;  | efetch -format runinfo \&#xA;  | cut -f1,12 -d,&#xA;Run,LibraryName&#xA;SRR292241,HS0798&#xA;SRR390728,HS0798&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;runinfo&lt;/code&gt; table format has 47 fields including experiment accession, project accessions, bioproject and biosample identifiers and such. You may want to explore that a little bit more if you are interested in downloading metadata in addition to the LibraryName.&lt;/p&gt;&#xA;&lt;p&gt;Also, if you prefer to parse XML output, you can append &lt;code&gt;-mode xml&lt;/code&gt; to the &lt;code&gt;efetch&lt;/code&gt; command to retrieve the runinfo table in XML format. &lt;code&gt;xtract&lt;/code&gt; tool that comes with the Entrez Direct package can then be used to convert that into a table.&lt;/p&gt;&#xA;" OwnerUserId="3577" LastEditorUserId="3577" LastEditDate="2021-10-05T10:23:58.073" LastActivityDate="2021-10-05T10:23:58.073" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7037" PostTypeId="2" ParentId="7034" CreationDate="2019-02-15T10:07:07.957" Score="6" Body="&lt;p&gt;&lt;em&gt;[Note: As of 2024, ONT now uses pod5 as the standard output format from their MinKNOW sequencing program, and &lt;em&gt;all&lt;/em&gt; pod5 files are &amp;quot;multi&amp;quot; pod5 files.]&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;1.&lt;/strong&gt; The quickest way to know if it's a single or multi fast5 is the file name. If it mentions channels and read numbers (e.g. &lt;code&gt;..._read_9_ch_471_...&lt;/code&gt;, then it's a single read file. Alternatively, the multi-read files have only read IDs at the top level (i.e. read_&amp;lt;32-character-string-with-additional-dashes&amp;gt;). Example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ h5ls &amp;lt;read_file_name&amp;gt;.fast5 | head&#xA;read_0115573a-042a-42a3-b4e2-f0338b4a6c66 Group&#xA;read_015b870d-8a03-4f22-a73e-fb9f833b94c7 Group&#xA;read_0167393a-dede-4a27-91af-494622b63261 Group&#xA;read_01e658de-853b-4f58-93ad-87fbd735254c Group&#xA;read_0202acc2-33f4-4d29-9dac-c7f80c758c86 Group&#xA;read_022d68f0-6f8e-4124-bfb0-11262efa4a67 Group&#xA;read_0258a3c9-4ffe-42ab-a42d-89466fcefbf5 Group&#xA;read_032da6db-e84a-43ec-8c25-bcaead992066 Group&#xA;read_03383754-be43-420b-87fc-60c2eb833963 Group&#xA;read_036b4824-bb0a-42dd-ade0-29d89112ff07 Group&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;2.&lt;/strong&gt; ONT provide a &lt;code&gt;single_to_multi&lt;/code&gt; program for converting to the multi-read format. Details can be found on &lt;a href=&quot;https://github.com/nanoporetech/ont_fast5_api/blob/master/README.rst#single_to_multi_fast5&quot; rel=&quot;nofollow noreferrer&quot;&gt;this github page&lt;/a&gt;. Here's a quick example of how it could be run:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ single_to_multi_fast5 --input_path fast5 --save_path fast5_multi \&#xA;    --filename_base multi_file_myRunID --batch_size 4000 --recursive&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;3a.&lt;/strong&gt; For multi-fast5 files, the kit/cell attributes are in the &lt;code&gt;context_tags&lt;/code&gt; group of the file (because a multi-fast5 file could contain fast5 elements from multiple different sequencing runs with different run parameters):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ h5dump -g /read_&amp;lt;readID&amp;gt;/context_tags multi_file_myRunID_0.fast5 | \&#xA;  grep -A 11 -e 'sequencing_kit' -e 'flowcell_type'&#xA;   ATTRIBUTE &amp;quot;flowcell_type&amp;quot; {&#xA;      DATATYPE  H5T_STRING {&#xA;         STRSIZE 11;&#xA;         STRPAD H5T_STR_NULLTERM;&#xA;         CSET H5T_CSET_ASCII;&#xA;         CTYPE H5T_C_S1;&#xA;      }&#xA;      DATASPACE  SCALAR&#xA;      DATA {&#xA;      (0): &amp;quot;flo-min106&amp;quot;&#xA;      }&#xA;   }&#xA;--&#xA;   ATTRIBUTE &amp;quot;sequencing_kit&amp;quot; {&#xA;      DATATYPE  H5T_STRING {&#xA;         STRSIZE 11;&#xA;         STRPAD H5T_STR_NULLTERM;&#xA;         CSET H5T_CSET_ASCII;&#xA;         CTYPE H5T_C_S1;&#xA;      }&#xA;      DATASPACE  SCALAR&#xA;      DATA {&#xA;      (0): &amp;quot;sqk-pbk004&amp;quot;&#xA;      }&#xA;   }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;3b.&lt;/strong&gt; For single fast5 files (at least, in the most recent version of them), they are found in the &lt;code&gt;UniqueGlobalKey&lt;/code&gt; group:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ h5dump -g /UniqueGlobalKey single_file_read_X_ch_Y_strand.fast5 | \&#xA;  grep -A 11 -e 'sequencing_kit' -e 'flowcell_type'&#xA;      ATTRIBUTE &amp;quot;flowcell_type&amp;quot; {&#xA;         DATATYPE  H5T_STRING {&#xA;            STRSIZE 11;&#xA;            STRPAD H5T_STR_NULLTERM;&#xA;            CSET H5T_CSET_ASCII;&#xA;            CTYPE H5T_C_S1;&#xA;         }&#xA;         DATASPACE  SCALAR&#xA;         DATA {&#xA;         (0): &amp;quot;flo-min106&amp;quot;&#xA;         }&#xA;      }&#xA;--&#xA;      ATTRIBUTE &amp;quot;sequencing_kit&amp;quot; {&#xA;         DATATYPE  H5T_STRING {&#xA;            STRSIZE 11;&#xA;            STRPAD H5T_STR_NULLTERM;&#xA;            CSET H5T_CSET_ASCII;&#xA;            CTYPE H5T_C_S1;&#xA;         }&#xA;         DATASPACE  SCALAR&#xA;         DATA {&#xA;         (0): &amp;quot;sqk-pcs108&amp;quot;&#xA;         }&#xA;      }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;[note that this file structure may change in the future]&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2024-01-21T23:05:50.453" LastActivityDate="2024-01-21T23:05:50.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7038" PostTypeId="2" ParentId="7023" CreationDate="2019-02-15T10:23:45.743" Score="1" Body="&lt;p&gt;Most duplicate marking programs do it by looking at the start and end location of hits. In this case, if two reads have identical start and end locations for both reads, then they will be considered [PCR] duplicates of each other. When running through a duplicate filter, you will only get one read for each unique pair.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is much more likely for targeted sequencing, because the length covered by the template is much smaller. It's also more likely that you actually &lt;em&gt;do&lt;/em&gt; have lots of PCR duplicates for the same reason.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2019-02-15T10:23:45.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7039" PostTypeId="2" ParentId="7025" CreationDate="2019-02-15T10:37:25.220" Score="1" Body="&lt;p&gt;I agree with Kamil on the downloading method.&#xA;Moreover, SRA will give you your file in the raw format which is usually a &lt;strong&gt;Binary Alignment Map (BAM)&lt;/strong&gt; or &lt;strong&gt;Sequence Alignment Map (SAM)&lt;/strong&gt; &#xA;Download &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra/docs/toolkitsoft/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SRA toolkit&lt;/a&gt; available online, to convert your raw reads to fastq format. Follow the steps using command line. It is a very simple command to convert your SRA file to fastq which uses the &lt;em&gt;fastq-dump&lt;/em&gt; command.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Further reading: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK158900/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/books/NBK158900/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps! &lt;/p&gt;&#xA;" OwnerUserId="4121" LastActivityDate="2019-02-15T10:37:25.220" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7040" PostTypeId="2" ParentId="7026" CreationDate="2019-02-15T10:40:25.197" Score="5" Body="&lt;p&gt;This can be solved like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Seurat)&#xA;&#xA;my_genes &amp;lt;- c(&quot;gene1&quot;, &quot;gene2&quot;, &quot;gene3&quot;)&#xA;&#xA;exp &amp;lt;- FetchData(object, my_genes)&#xA;&#xA;matrix &amp;lt;- as.matrix(colMeans(exp  &amp;gt; 0))*100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3968" LastEditorUserId="3968" LastEditDate="2019-02-15T11:08:30.890" LastActivityDate="2019-02-15T11:08:30.890" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7041" PostTypeId="2" ParentId="7011" CreationDate="2019-02-15T10:57:24.337" Score="2" Body="&lt;p&gt;Ensembl plants has a REST API, you might be able to get all these details (except possibly EST numbers) there: &lt;a href=&quot;http://ensemblgenomes.org/info/access/rest&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ensemblgenomes.org/info/access/rest&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example this query gives you the Arabidopsis genome size: &lt;a href=&quot;http://rest.ensemblgenomes.org/info/genomes/arabidopsis_thaliana?content-type=application/json&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://rest.ensemblgenomes.org/info/genomes/arabidopsis_thaliana?content-type=application/json&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-02-15T10:57:24.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7042" PostTypeId="2" ParentId="7032" CreationDate="2019-02-15T12:22:40.293" Score="7" Body="&lt;p&gt;The Blast E is the expected frequency of obtaining false positive. This will depend on your query size (number of nucleotide or amino acid residues) and the size of the database. With a short query and a large database you are more likely to have a sequence in your database that matches your query by simple chance. In summary, the smaller the query and the larger the database the greater the chance of a spurious result and that is what E is measuring.&lt;/p&gt;&#xA;" OwnerUserId="3818" LastEditorUserId="3818" LastEditDate="2019-02-18T09:18:08.300" LastActivityDate="2019-02-18T09:18:08.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7043" PostTypeId="2" ParentId="7025" CreationDate="2019-02-15T15:23:54.873" Score="2" Body="&lt;p&gt;I suggest you to try difference resources, including ArrayExpress. &#xA;A simple search at &lt;a href=&quot;https://www.ebi.ac.uk/arrayexpress/browse.html?keywords=p53&amp;amp;organism=Homo%20sapiens&amp;amp;exptype%5B%5D=&amp;amp;exptype%5B%5D=%22sequencing%20assay%22&amp;amp;array=&amp;amp;page=1&amp;amp;pagesize=500&quot; rel=&quot;nofollow noreferrer&quot;&gt;ArrayExpress&lt;/a&gt; returns multiple experiments which might be of your interest. Just a pointer, ArrayExpress is also synced with GEO.&lt;/p&gt;&#xA;" OwnerUserId="1135" LastActivityDate="2019-02-15T15:23:54.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7044" PostTypeId="2" ParentId="7032" CreationDate="2019-02-15T15:41:27.047" Score="5" Body="&lt;p&gt;Sure, so you start out with what is called a bitscore, which is a normalized to the score calculated from the alignment between the 2 seqs which depends on the following equation. It is &lt;strong&gt;independent&lt;/strong&gt; of database size:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(lambda * S - ln(k))/(ln)2&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Then, the p-value of a local blast is just:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;1/2^bitscore&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So if your bitscore is 15, you need 1/32768 alignments before you will get a score as good or better (highly similar sequences) &lt;strong&gt;BY CHANCE ALONE&lt;/strong&gt;. This addresses your &quot;BY CHANCE&quot; question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Earlier I said the bitscore was &lt;strong&gt;independent&lt;/strong&gt; of database size. The E value is just the p-value above normalized to the database size (so it is &lt;strong&gt;dependent&lt;/strong&gt; on the db size) by the following equation:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(query length * database length * p-value)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;which simplifies a bit to:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;E = (query length* (db length/2^bitscore)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I hope this helps! &lt;/p&gt;&#xA;" OwnerUserId="3669" LastEditorUserId="3669" LastEditDate="2019-02-15T19:48:51.473" LastActivityDate="2019-02-15T19:48:51.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7046" PostTypeId="1" CreationDate="2019-02-17T02:23:01.417" Score="0" ViewCount="2053" Body="&lt;p&gt;I install hdf5r following &lt;a href=&quot;https://github.com/hhoeflin/hdf5r%EF%BC%8Cit&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/hhoeflin/hdf5r&lt;/a&gt;， it shows two methods:&#xA;&lt;img src=&quot;https://user-images.githubusercontent.com/30619010/52907396-e8e2ad00-329b-11e9-998b-96431b055ba0.png&quot; alt=&quot;image&quot;&gt;&#xA;When I use the second method to install from github,it showed many errors：&#xA;&lt;img src=&quot;https://user-images.githubusercontent.com/30619010/52907457-e0d73d00-329c-11e9-9921-14408bd51967.png&quot; alt=&quot;image&quot;&gt;&#xA;&lt;img src=&quot;https://user-images.githubusercontent.com/30619010/52907426-5989c980-329c-11e9-95a6-e122f7196071.png&quot; alt=&quot;image&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;The downloaded source packages are in&#xA;‘C:\Users\dell\AppData\Local\Temp\RtmpI7TP9f\downloaded_packages’&#xA;√ checking for file 'C:\Users\dell\AppData\Local\Temp\RtmpI7TP9f\remotes8c87faaf3d\hhoeflin-hdf5r-cc278c1/DESCRIPTION'&#xA;&#xA;* preparing 'hdf5r': (4.9s)&#xA;  √  checking DESCRIPTION meta-information ...&#xA;* cleaning src&#xA;* checking for LF line-endings in source and make files and shell scripts (676ms)&#xA;* checking for empty or unneeded directories (1.5s)&#xA;* building 'hdf5r_1.0.1.tar.gz'&#xA;  Warning: file 'hdf5r/cleanup' did not have execute permissions: corrected&#xA;  Warning: file 'hdf5r/configure' did not have execute permissions: corrected&#xA;&#xA;* installing _source_ package 'hdf5r' ...&#xA;  checking hdf5 headers and libraries&#xA;  Environment variable LIB_HDF5 is not set. Attempting to download HDF5 files from repository.&#xA;  试开URL’https://github.com/mannau/h5-libwin/archive/master.zip'&#xA;  Content type 'application/zip' length unknown&#xA;  ** libs&#xA;  错误: (由警告转换成)this package has a non-empty 'configure.win' file,&#xA;  so building only the main architecture&#xA;* removing 'D:/soft ware/R-3.5.2/library/hdf5r'&#xA;  In R CMD INSTALL&#xA;  Error in i.p(...) :&#xA;  (converted from warning) installation of package ‘C:/Users/dell/AppData/Local/Temp/RtmpI7TP9f/file8c863b6795/hdf5r_1.0.1.tar.gz’ had non-zero exit status&#xA;  In addition: Warning messages:&#xA;  1: In untar2(tarfile, files, list, exdir) :&#xA;  skipping pax global extended headers&#xA;  2: In untar2(tarfile, files, list, exdir) :&#xA;  skipping pax global extended headers&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But when I use the first command:install.packages(&quot;hdf5r&quot;), it worked:&#xA;  &lt;img src=&quot;https://user-images.githubusercontent.com/30619010/52907413-23e4e080-329c-11e9-9e32-916eccdb0b2d.png&quot; alt=&quot;image&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me? I search many times, but I could't find the answer.&lt;/p&gt;&#xA;" OwnerUserId="3851" LastEditorUserId="48" LastEditDate="2019-02-18T08:32:21.023" LastActivityDate="2019-02-28T19:21:18.407" Title="R studio failed to install the latest development version of hdf5r from Github (win10)" Tags="|r|seurat|software-installation|rstudio|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7047" PostTypeId="1" CreationDate="2019-02-17T08:12:26.993" Score="1" ViewCount="57" Body="&lt;p&gt;I failed to install hdf5r using this command provided by &lt;a href=&quot;https://satijalab.org/seurat/mca_loom.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/mca_loom.html&lt;/a&gt;: devtools::install_github(repo = 'hhoeflin/hdf5r'),below is the code,can you help me ?&#xA;&lt;img src=&quot;https://user-images.githubusercontent.com/30619010/52910146-2c560f00-32ce-11e9-9195-f4748f9f06cb.png&quot; alt=&quot;image&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/30619010/52910141-22341080-32ce-11e9-910f-c9b5c8b1a1be.png&quot; alt=&quot;image&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/30619010/52910131-07fa3280-32ce-11e9-8a6b-59823801c23f.png&quot; alt=&quot;image&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="3851" LastActivityDate="2019-02-17T08:12:26.993" Title="failed to install hdf5r" Tags="|r|seurat|software-installation|rstudio|" AnswerCount="0" CommentCount="0" ClosedDate="2019-02-18T18:42:33.307" ContentLicense="CC BY-SA 4.0" />
  <row Id="7048" PostTypeId="1" CreationDate="2019-02-17T10:38:50.020" Score="5" ViewCount="4881" Body="&lt;p&gt;I am not talking about consensus sequence, I know how to get consensus sequence using mpileup in samtools/bcftools. As I understand , SAM/BAM files are basically sequence alignment format so it's  natural to expect a straightforward way of conversion between them. Sadly I have yet to find one. Vcf-kit allows for conversion between vcf file and fasta alignment, but the fasta alignment contains only the SNPs. I need the whole alignment.  Does samtools/bcftools provide option for such conversion? If not, is there any other tool?&lt;/p&gt;&#xA;" OwnerUserId="3555" LastEditorUserId="3555" LastEditDate="2019-02-17T11:48:57.187" LastActivityDate="2021-12-15T13:41:10.440" Title="How to get fasta alignment file from SAM/BAM file?" Tags="|sequence-alignment|samtools|assembly|bcftools|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7050" PostTypeId="2" ParentId="7048" CreationDate="2019-02-17T15:04:41.580" Score="8" Body="&lt;p&gt;I am not sure what you mean by &quot;fasta alignment file&quot;. If you mean a multi-sequence alignment (MSA) in the fasta format, you can't get that because SAM keeps pairwise alignments only and doesn't align inserted sequences. Even if you don't care about inserted sequences, a MSA in fasta is far to big to be practical. Alternatively, by &quot;fasta alignment file&quot;, you could mean pairwise alignment, but it is still impractical to output every pairwise alignment in a separate fasta file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are tools to convert SAM to BLAST-like format if that is what you want. The following shows one of them. I believe there are other tools easier to use, but I forget what are they.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, to install:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;wget https://raw.githubusercontent.com/lh3/minimap2/master/misc/paftools.js&#xA;curl -L https://github.com/attractivechaos/k8/releases/download/v0.2.4/k8-0.2.4.tar.bz2 | tar -jxf -&#xA;cp k8-0.2.4/k8-`uname -s` k8&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you can convert your &lt;code&gt;ALN.sam&lt;/code&gt; with:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;./k8 paftools.js sam2paf -L ALN.sam | ./k8 paftools.js view -l0 - | less -S&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or generate a SAM stream from &lt;code&gt;ALN.bam&lt;/code&gt; and pipe it with:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;samtools view -h ALN.bam | ./k8 paftools.js sam2paf -L - | ./k8 paftools.js view -l0 -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is an example output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;MT_orang   16499   0   16025   +   MT_human    16569   576 16569   13773   16095   60  tp:A:P  mm:i:2150   gn:i:172    go:i:101&#xA;Ref+:        577 GTTTATGTAGCTTACCTCCT---CAAAGCAATACACTGAAAATGTTTAGACGGGCTCACATCACCCCATAAACAAATAGGTTTGGTCCTAGCCTTTCTATTAGCTCTTAGTAAGATTACACATGCAAGCATCCCCGTTCCAGTGAGTTCACCCTCTAAATCACCACGATCAAAAGGAACAAGCATCA&#xA;                 ||||||||||||||  | ||   ||||||||| |||||||||||| | ||||||| |||| | |||||||||||||||||||||||||||||||||||||||||||||||| || |||||||||||||||||||||  |||||||| || ||||| || ||||   ||  || |||| |||||||||&#xA;Qry+:          1 GTTTATGTAGCTTA--TTCTATCCAAAGCAATGCACTGAAAATGTCTCGACGGGCCCACA-CGCCCCATAAACAAATAGGTTTGGTCCTAGCCTTTCTATTAGCTCTTAGTGAGGTTACACATGCAAGCATCCCCGCCCCAGTGAG-TCGCCCTCCAAGTCACTCTGACTAAGAGGAGCAAGCATCA&#xA;//&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can change &lt;code&gt;-l0&lt;/code&gt; to &lt;code&gt;-l80&lt;/code&gt; to print alignments in multiple lines.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2019-02-17T15:04:41.580" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7051" PostTypeId="1" CreationDate="2019-02-17T20:44:05.370" Score="0" ViewCount="552" Body="&lt;p&gt;I would like to make a &lt;a href=&quot;https://www.data-to-viz.com/graph/chord.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;chord plot&lt;/a&gt; from my data. I have a list of genes that according to my experiments are divided into 64 clusters of enrichment pattern. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would combine my 64 clusters with a second clustering. A clustering of all the human genes into subgroups of &quot;interacting&quot; genes (e.g. both are pathway, co-expressed) will allow me to combine the two clustering data on a chord plot. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the analogy of the following graph the continents will be my 64 clusters of the genes and the edges/chords will be assigned based on the biological function  clustering. I guess that such a functional clustering can be done ore has been done using platforms such as: KEGG, genescf, or reactome.org.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ykKrl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ykKrl.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there a biological pathway clustering of genes for lymphoblastoid or breast cell lines?&lt;/li&gt;&#xA;&lt;li&gt;Are there any suggestions how to build the clustering between genes whose functions are strongly correlated in the cell?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: the question is about getting the clustering from a system-biological perspective not how to plot chord diagrams. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there such a metric to quantify how likely two genes are correlated in their enrichment, function etc. For example, using &lt;a href=&quot;http://version10.5.string-db.org/cgi/network.pl?taskId=KbO2wnULZxKQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;STRING&lt;/a&gt; we can see that PIK3CA and PTEN are more co-functioning than PIK3CA and SF3B1. How can I do something like String on genome wide scale. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8Lpz1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8Lpz1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2019-02-20T20:00:01.477" LastActivityDate="2019-02-20T20:00:01.477" Title="How to cluster the human genes by pathways/system-biology/metabolic properties?" Tags="|ngs|gene-expression|pathway|human|genetics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7052" PostTypeId="1" CreationDate="2019-02-17T20:55:13.253" Score="1" ViewCount="1107" Body="&lt;p&gt;I have a bam file with 1 billion alignment reads of which there are 700 million unique reads. I want to split the alignments into chunks for parallel-processing. Multi-alignments of the same read should be in the same chunk. I want to use pysam to solve the problem and would be very grateful for suggestions.&lt;/p&gt;&#xA;" OwnerUserId="4141" LastEditorUserId="48" LastEditDate="2019-02-18T08:36:29.553" LastActivityDate="2019-02-19T17:59:03.320" Title="Chunk alignment in a name sorted bam for parallel processing" Tags="|python|sam|pysam|multithreading|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="7053" PostTypeId="2" ParentId="7051" CreationDate="2019-02-17T22:15:22.343" Score="1" Body="&lt;p&gt;Production of a chord diagram is described in detail &lt;a href=&quot;https://stackoverflow.com/questions/14599150/chord-diagram-in-r&quot;&gt;here&lt;/a&gt;. You use the R package with either, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(chorddiag)&#xA;library (circulize)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first level clusters are predefined and the second layer of clustering is what you need to work out. The obvious stat is correlation, secondly (better approach IMO) you might look at covariance, although I don't exactly know what this &quot;interaction&quot; constitutes, which makes it difficult.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My personal view is the diagramatic representation doesn't really matter, its the robustness of the stats behind the diagram that counts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was a bit surprised to learn ggplot2 doesn't do circular diagrams :-(&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Lymphoblastoid Cell lines&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I think you need to clarify the experiment on these cells lines, I assume its a time course, i.e. time series. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;At a guess this is RNA expression, if you are using a chip then its between predetermined genes (i.e. spotted on the chip). If it is RNA seq then it can be anything. I suspect you are treating the cells and monitoring changes in RNA expression as a function of their biological (immunological?) activity. If the basic measure is time, i.e. a time series - thus two genes being expressed in the same unit time the best opening statistic is covariance. If all you want to do is make a chord diagram out of it, I think thats fine, this will form a 2x2 variance matrix for which any standard clustering will work (Jaccard distances [there will be better ones]) or direct plotting from the matrix. If you looking at unique expression behaviour to attempt to unlock some sort of pathway then you are into more serious regression time series (autoregression that sort of thing - this is complicated). If its oscillations then its a separate branch of stats based on sinosoids (sine waves) called periodicity analysis. HOWEVER, if the interaction is genetic similarity (gene families) e.g. mRNA then you want a phylogenetics measure, which if its clustering neighbor-joining is fine.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;My personal view if it is the experiment I think, I would,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Construct a matrix of the covariance of the (I assume) time series  &lt;/li&gt;&#xA;&lt;li&gt;Make sure the covariance is standarised between -1 to 1 (Pearson&#xA;approach), R-cran.&lt;/li&gt;&#xA;&lt;li&gt;Assign a tight cut-off (tricky because there's no &quot;hard cut-off&quot;) but say 0.7 (you need to experiment a bit)&lt;/li&gt;&#xA;&lt;li&gt;Plot the raw covariance on the chord diagram.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Easy, simple and highly visual.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-02-17T23:28:26.140" LastActivityDate="2019-02-17T23:28:26.140" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7054" PostTypeId="1" CreationDate="2019-02-17T22:28:43.373" Score="0" ViewCount="74" Body="&lt;p&gt;I have inferred a confusion matrix of training and test set by neural network. I want to know which members are in the confusion matrix.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; table(test&lt;span class=&quot;math-container&quot;&gt;$TRG,predicted.nn.values$&lt;/span&gt;net.result)&#xA;&#xA;    0 1&#xA;  0 5 4&#xA;  1 5 3&#xA;&amp;gt; &#xA;&#xA;&amp;gt; head(train[,1:4])&#xA;  TRG      ALB     AQP9   CALML5&#xA;1   1 5.865827 8.190945 6.705303&#xA;2   1 6.998435 8.424261 8.505591&#xA;3   1 7.424512 8.716471 7.249556&#xA;4   1 7.442049 8.325263 8.809286&#xA;6   1 5.893411 8.199990 6.677618&#xA;7   1 7.288030 9.143510 7.088598&#xA;&amp;gt; &#xA;&#xA;&amp;gt; head(test[,1:4])&#xA;   TRG      ALB     AQP9    CALML5&#xA;5    1 6.369307 7.954310  6.920290&#xA;8    1 6.181902 8.651442  7.225389&#xA;10   1 6.119359 9.345270  6.829623&#xA;13   1 7.775533 9.016272  7.976813&#xA;14   1 5.913656 9.484457 10.609013&#xA;18   1 6.603138 7.908560  8.827173&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I know which patients are false positive or true negative? For example I have 5 patients predicted correctly, how can I know the name of these patients?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="298" LastEditDate="2019-02-18T08:51:46.153" LastActivityDate="2019-02-18T23:33:42.127" Title="Finding the members in a confusion matrix" Tags="|r|machine-learning|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7055" PostTypeId="2" ParentId="7054" CreationDate="2019-02-17T22:47:21.620" Score="1" Body="&lt;p&gt;This does not look a great confusion matrix, but it is a very, very cool approach. The cool thing about machine learning is the ability to examine lots of models within confusion matrices.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To answer your question, you want a heat map of the regression weights to establish who is who, i.e. which patient is false positive etc... This is the only way I'm aware of to analyse a neural network. This looks like R, I only know Scikit learn.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-02-18T23:33:42.127" LastActivityDate="2019-02-18T23:33:42.127" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7056" PostTypeId="2" ParentId="7051" CreationDate="2019-02-18T09:45:54.187" Score="2" Body="&lt;p&gt;To measure if two genes are functional similar I developed the &lt;a href=&quot;https://bioconductor.org/packages/BioCor&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioCor&lt;/a&gt; package. It calculates a similarity score between genes by the measuring the amount of pathways  shared. However, it doesn't take into account the strength of the connection between two proteins. You could multiply the similarity by a value between 0 and 1 according to the number of evidences of the relation between the two proteins.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is optimized for large amounts of data, you could even perform them in parallel, but I doubt you find information about pathways for most of the genes (In my informal test, I think that I could calculate similarities for about 1600 genes not the ~20000 of the human genome).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I am interested in co-expression data I calculate the co-expression by correlation and add the similarity of the genes calculated with BioCor. The resulting distance matrix can be used for clustering. Maybe the section about using &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/BioCor/inst/doc/BioCor_1_basics.html#8_wgcna_and_biocor&quot; rel=&quot;nofollow noreferrer&quot;&gt;weighted co-expression network analysis and BioCor&lt;/a&gt; can be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To have a &quot;biological pathway clustering of genes for lymphoblastoid or breast cell lines&quot;, you would need to have a specific signature/pathway of the cell lines. Currently I am not aware of accepted standard signature database. The &lt;a href=&quot;https://www.ebi.ac.uk/gxa/home&quot; rel=&quot;nofollow noreferrer&quot;&gt;single cell atlas&lt;/a&gt; could be useful to find expression of your cells, but I don't know if they provide a signature for cell line&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2019-02-18T10:22:23.960" LastActivityDate="2019-02-18T10:22:23.960" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7057" PostTypeId="1" CreationDate="2019-02-18T10:02:41.590" Score="3" ViewCount="114" Body="&lt;p&gt;I have a qqplot of my whole genome sequencing data; A plot is for showing possibly significant driver genes. I tried to read about qqplot though but people only say about the skewedness while I want to know from these two genes which one are more likely to be driver&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7Otcg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7Otcg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2019-02-19T08:27:39.450" Title="Which gene I should select from this qqplot" Tags="|variant-calling|mutations|wgs|" AnswerCount="3" CommentCount="0" ClosedDate="2021-05-23T12:39:16.493" ContentLicense="CC BY-SA 4.0" />
  <row Id="7058" PostTypeId="2" ParentId="7057" CreationDate="2019-02-18T10:18:39.267" Score="3" Body="&lt;p&gt;Given that TP53 is the most significant and is already known to have driver mutations in cancer it would seem to be the logical choice.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-02-18T10:18:39.267" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7059" PostTypeId="1" AcceptedAnswerId="7093" CreationDate="2019-02-18T10:21:03.677" Score="0" ViewCount="238" Body="&lt;p&gt;I have a list of gene names from the analysis of my mouse single seq data. I want to perform a simple grouping of the genes based on the pathways to which they belong or pathways through which they relate. Results from my online searches strongly recommend the ingenuity pathway analysis (IPA) tool. My laboratory has a license for the tool, but I need help with the workflow/steps. I have no prior experience with pathway analysis, and I have a background in R and Python programming. Can anyone also suggest any link which can help me proceed with the required IPA analysis? I will also be very grateful for any other tools (databases, R or Python packages) that can be helpful here.&lt;/p&gt;&#xA;" OwnerUserId="3968" LastEditorUserId="7340" LastEditDate="2020-06-18T13:17:01.820" LastActivityDate="2020-06-18T13:17:01.820" Title="How do I perform a pathway related grouping of genes?" Tags="|scrnaseq|bioconductor|proteins|pathway|gene-ontology|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7060" PostTypeId="1" CreationDate="2019-02-18T11:27:25.643" Score="3" ViewCount="215" Body="&lt;p&gt;Which Read Groups are correct:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -Xmx4G -jar picard.jar AddOrReplaceReadGroups \&#xA;I=$SNIC_TMP/WCRO84_S23_L005.bam \&#xA;O=WCRO84_S23_L005.bam \&#xA;RGID=WCRO84_S23_L005 \&#xA;RGLB=WCRO84 \&#xA;RGPL=illumina \&#xA;RGPU=WCRO84_S23_L005 \&#xA;RGSM=WCRO84&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -Xmx4G -jar picard.jar AddOrReplaceReadGroups \&#xA;I=$SNIC_TMP/WCRO84_S23_L005.bam \&#xA;O=WCRO84_S23_L005.bam \&#xA;RGID=WCRO84_S23_L005 \&#xA;RGLB=WCRO84_S23_L005 \&#xA;RGPL=illumina \&#xA;RGPU=WCRO84_S23_L005 \&#xA;RGSM=WCRO84_S23_L005&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or is a better way to determine read groups?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance,&lt;/p&gt;&#xA;" OwnerUserId="2477" LastActivityDate="2019-02-18T11:33:19.420" Title="Determining Read Groups" Tags="|sequence-alignment|sam|bwa|picard|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7061" PostTypeId="2" ParentId="7060" CreationDate="2019-02-18T11:33:19.420" Score="4" Body="&lt;p&gt;Usually it doesn't actually matter, but if you want it to be very correct:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;RGID=SomeLibraryID \&#xA;RGLB=SomeLibraryID \&#xA;RGPL=illumina \&#xA;RGPU=Some_Sequencer_ID \&#xA;RGSM=SomeSampleID&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That assumes you have one library per sample and aren't splitting the output BAM files by lane.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-02-18T11:33:19.420" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7062" PostTypeId="1" AcceptedAnswerId="7064" CreationDate="2019-02-18T15:47:46.980" Score="5" ViewCount="629" Body="&lt;p&gt;I have 23bp long reads and want to find all possible alignments of them to the human genome (hg19, hg38) for an arbitrary number of mismatches (&amp;lt;7), possibly also small indels. I've read in literature that people use bowtie2 for this, so I've tried the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;echo &amp;quot;GGAGAACAAGTGGCAGATAGAGG&amp;quot; | bowtie2 --all -f -U- -x hg19 -r --no-hd --quiet --score-min C,-36 -p 3 -L 4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm not very experienced with bowtie2, and I hoped that this would give me all alignments of the piped-in sequence with up to 6 mismatches (6*-6 = -36) to the hg19 genome which I previously indexed from fasta files. It however only gives me the correct full alignment:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;0   0   chr2    173327389   255 23M *   0   0   GGAGAACAAGTGGCAGATAGAGG IIIIIIIIIIIIIIIIIIIIIII AS:i:0  XN:i:0  XM:i:0  XO:i:0  XG:i:0  NM:i:0  MD:Z:23 YT:Z:UU&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've read that bowtie2 by design only allows a maximum of one mismatch per seed. Even if I implement the smallest possible seed length and interval (-L 4 -i C,1 -N 1), I only get this one result which I think is highly unlikely, given that I'm aligning to the whole genome.&lt;/p&gt;&#xA;&lt;p&gt;The last three bp will have to match exactly - I was going to enforce this manually later on, maybe there's a better option for doing this. Taking the indels into account is also a secondary aim.&lt;/p&gt;&#xA;&lt;p&gt;I'm fine with ~10k alignments per query for &amp;lt;=6 mismatches plus the last 3bp matching exactly. Haven't done the combinatorics but that sounds realistic to me - question is how to make bowtie do it, or a different tool.&lt;/p&gt;&#xA;&lt;p&gt;What would be the right way of achieving this using bowtie, or a different tool?&lt;/p&gt;&#xA;" OwnerUserId="4145" LastEditorUserId="73" LastEditDate="2022-10-21T18:31:21.143" LastActivityDate="2022-10-21T18:33:21.030" Title="Alignment with arbitrary number of mismatches or gaps" Tags="|sequence-alignment|bowtie2|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7063" PostTypeId="2" ParentId="7062" CreationDate="2019-02-18T17:16:09.523" Score="0" Body="&lt;p&gt;Did you see&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;-k  Report up to  valid alignments per read or pair (default: 1).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-02-18T17:16:09.523" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7064" PostTypeId="2" ParentId="7062" CreationDate="2019-02-18T18:24:33.357" Score="6" Body="&lt;p&gt;This is not something mappers will be good at, because you're going to hit most of the genome. Bowtie won't do it, and I don't think LAST will either. The majority of fast mapping programs rely on seed-based mapping where the seed is essentially free of errors.&lt;/p&gt;&#xA;&lt;p&gt;Doing only mismatches makes it much easier to determine, but it's still too relaxed. Consider that the example sequence you have provided is 10 mismatches away from &lt;code&gt;GGGGGGGGGGGGGGGGAGAGAGG&lt;/code&gt;, and also 10 mismatches away from &lt;code&gt;AAAAAAAAAAAAAAAGAGAGAGG&lt;/code&gt; (with the last 3 bp matching exactly)&lt;/p&gt;&#xA;&lt;p&gt;Depending on how many reads you have it may be better to work in reverse, iterating over all the kmers in the genome and working out whether or not they are fewer than 7 mismatches (plus indels) away from any reads:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for gp in genomePositions&#xA;    genomeKmer = genome[gp:(gp+23-1)]&#xA;    for read in reads&#xA;        errors = 0&#xA;        for rp in 1..23&#xA;            if (genomeKmer[rp] != read[rp])&#xA;                errors += 1&#xA;        if (errors &amp;lt; 7)&#xA;            print gp &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-10-21T18:33:21.030" LastActivityDate="2022-10-21T18:33:21.030" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7065" PostTypeId="1" AcceptedAnswerId="7071" CreationDate="2019-02-18T20:08:42.290" Score="0" ViewCount="2719" Body="&lt;p&gt;I was using Seurat to analysis single-cell RNA Seq. And I was interested in only one cluster by using the Seurat. Does anyone know how to achieve the cluster's data(.csv file) by using Seurat or any other method that can do this function? Thank you!&lt;/p&gt;&#xA;" OwnerUserId="3171" LastEditorUserId="48" LastEditDate="2019-02-19T10:10:10.163" LastActivityDate="2019-02-19T14:50:51.793" Title="how to obtain only one cluster's data (.csv file) in Seurat" Tags="|seurat|single-cell|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7066" PostTypeId="2" ParentId="7057" CreationDate="2019-02-19T01:28:59.560" Score="1" Body="&lt;p&gt;This looks to be a pretty good Q-Q plot. My recollection of doing Q-Q plots is that a good Q-Q requires a &quot;more or less&quot; linear relationship for the model to be considered okay. Again this model looks good, its 1:1, albeit it does deviate a little (for the gene of interest). TP15 falls outside the 1:1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;@Devon Ryan states TP15 has good biological credentials, which is clearly important. It maybe worth checking  MAP15 because this is &quot;more or less&quot; within your model (for my Q-Q plots definitely within the model :-) ). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, its just my opinion - that on the basis of the Q-Q plot the driver is the not the expected candidate.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-02-19T01:36:34.230" LastActivityDate="2019-02-19T01:36:34.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7067" PostTypeId="2" ParentId="7052" CreationDate="2019-02-19T07:02:12.363" Score="2" Body="&lt;p&gt;python will be to slow for this job. Here's a &lt;code&gt;awk&lt;/code&gt; solution. One need to sort by read name and take track over the number of reads per chunk. If the number is reached the next chunk can only created of the read name is different.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ samtools sort -n -O SAM input.bam|awk -v n=1000000 -v FS=&quot;\t&quot; '&#xA;  BEGIN { part=0; line=n }       &#xA;  /^@/ {header = header$0&quot;\n&quot;; next;} &#xA;  { if( line&amp;gt;=n &amp;amp;&amp;amp; $1!=last_read ) {part++; line=1; printf header &amp;gt; part&quot;.sam&quot;; print $0 &amp;gt;&amp;gt; part&quot;.sam&quot; } &#xA;    else { print $0 &amp;gt;&amp;gt; part&quot;.sam&quot;; line++; } &#xA;    last_read = $&lt;/span&gt;1;&#xA;  }'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output will be in sam format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can controll the number of reads per chunk by setting the &lt;code&gt;n&lt;/code&gt; value for the &lt;code&gt;awk&lt;/code&gt; command.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A second version that will output in &lt;code&gt;bam&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ samtools sort -n -O SAM input.bam|awk -v n=1000000 -v FS=&quot;\t&quot; ' &#xA;BEGIN { part=0; line=n }  &#xA;/^@/ {header = header$0&quot;\n&quot;; next;}  &#xA;{ if( line&amp;gt;=n &amp;amp;&amp;amp; $1!=last_read ) {print part,line; part++; line=1;}  &#xA;  print line==1 ? header&quot;&quot;$0 : $0 | &quot;samtools view -b -o &quot;part&quot;.bam&quot; &#xA;  last_read = $&lt;/span&gt;1;&#xA;  line++;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1310" LastEditorUserId="1310" LastEditDate="2019-02-19T17:59:03.320" LastActivityDate="2019-02-19T17:59:03.320" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7068" PostTypeId="2" ParentId="7057" CreationDate="2019-02-19T08:27:39.450" Score="2" Body="&lt;p&gt;You can't know which is the driver and which is the carrier. At most you can say that a specific gene deviate more of the expected underlying hypothesis. See also &lt;a href=&quot;https://stats.stackexchange.com/a/101290/105234&quot;&gt;other&lt;/a&gt; &lt;a href=&quot;http://onlinestatbook.com/2/advanced_graphs/q-q_plots.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;resources&lt;/a&gt; &lt;a href=&quot;https://support.sas.com/documentation/cdl/en/procstat/63104/HTML/default/viewer.htm#procstat_univariate_sect040.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;online&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also seem to ignore other genes that deviate more of your null hypothesis. I recommend to plot the histogram of the p-values to see if the distribution is uniform&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2019-02-19T08:27:39.450" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7069" PostTypeId="2" ParentId="7052" CreationDate="2019-02-19T11:55:18.650" Score="2" Body="&lt;p&gt;I don't know what the timings would be like, but the python code below will output in BAM rather than SAM, so you won't earn your PIs ire for using all that disk space, and I guess your processing code if going the be the slow bit.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pysam&#xA;infile = pysam.AlignmentFile(&quot;input.bam&quot;)&#xA;&#xA;chunk_size = 10000000&#xA;outfile_pattern = &quot;output_segement%d.bam&quot;&#xA;&#xA;chunk = 0&#xA;reads_in_this_chunk = 0&#xA;old_name = None&#xA;outfile = pysam.AlignmentFile(outfile_pattern % chunk, &quot;w&quot;, template = infile)&#xA;&#xA;for read in infile.fetch(until_eof=True):&#xA;&#xA;    if old_name != read.query_name and reads_in_this_chunk &amp;gt; chunk_size:&#xA;        reads_in_this_chunk = 0&#xA;        chunk += 1&#xA;        outfile.close()&#xA;        outfile = pysam.AlignmentFile(outfile_pattern % chunk, &quot;w&quot;, template = infile)&#xA;&#xA;    outfile.write(read)&#xA;    old_name = read.query_name&#xA;    reads_in_this_chunk += 1&#xA;&#xA;outfile.close()&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="235" LastActivityDate="2019-02-19T11:55:18.650" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7070" PostTypeId="2" ParentId="6791" CreationDate="2019-02-19T14:15:12.180" Score="4" Body="&lt;p&gt;I believe you may be able to map your reads, but I don't know how to do that with minimap2. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend running gsnap, which is more SNP tolerant and provides a number of parameters which are likely to help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, I believe that most aligners will treat 'N' characters as mismatches when you align. GSNAP has parameters to account for this. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;--query-unk-mismatch=INT       Whether to count unknown (N) characters in the query as a mismatch&#xA;                               (0=no (default), 1=yes)&#xA;--genome-unk-mismatch=INT      Whether to count unknown (N) characters in the genome as a mismatch&#xA;                               (0=no, 1=yes (default))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It also has a mismatch parameter similar to the one you described for minimap2.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-m, --max-mismatches=FLOAT     Maximum number of mismatches allowed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Try running with the 'genome-unk-mismatch' parameter above (with the masked reference). That may be your best bet. There are also other parameters that may help, but this should be a good start.&lt;/p&gt;&#xA;" OwnerUserId="3499" LastActivityDate="2019-02-19T14:15:12.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7071" PostTypeId="2" ParentId="7065" CreationDate="2019-02-19T14:50:51.793" Score="1" Body="&lt;p&gt;You can access data within the Seurat object using &lt;code&gt;GetAssayData&lt;/code&gt;, and extract a list of cell names for the cluster you're interested in using &lt;code&gt;WhichCells&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Seurat)&#xA;&#xA;cells.use &amp;lt;- WhichCells(object = pbmc_small, ident = 1)&#xA;expr &amp;lt;- GetAssayData(object = pbmc_small, assay.type = &quot;RNA&quot;, slot = &quot;data&quot;)[, cells.use]&#xA;expr &amp;lt;- as(Class = 'matrix', object = expr)&#xA;write.csv(x = expr, file = &quot;expression_cluster1.csv&quot;, quote = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3415" LastActivityDate="2019-02-19T14:50:51.793" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7072" PostTypeId="1" CreationDate="2019-02-19T17:06:42.933" Score="1" ViewCount="43" Body="&lt;p&gt;I am analyzing the effect of one putative tumor suppressor (and its loss) on a specific cancer phenotype. This putative tumor suppressor is known in literature to have an effect on the growth of tumors in mouse models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in the same region of this gene there's a second well-known tumor suppressor - actually a very powerful one. By looking at the data from &lt;a href=&quot;http://www.cbioportal.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;cBio portal&lt;/a&gt;, there's a very strong co-occurrence of loss on both tumor suppressors in a pan-cancer analysis, that is they are almost always lost together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, I would like to disentangle the effect of the two genes, and to analyze only the effect of my putative candidate on the phenotype. The simplest approach is to use only the samples that have a single loss (only the loss of my putative candidate, but not the loss of the confounding tumor suppressor). However, only ~10% of all samples shows such behavior, and this 10% is distributed across all cancer types, decreasing the statistical power of the analysis - since I would like to test each cancer type separately to avoid mixing incompatible data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which kind of strategy/analysis can be used in this case?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2019-02-20T11:13:17.347" Title="Strategies to disentangle the effect of two genes on phenotype" Tags="|gene|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7073" PostTypeId="1" CreationDate="2019-02-19T19:22:19.477" Score="4" ViewCount="567" Body="&lt;p&gt;I constructed a phylogenetic tree using mole blast, and it created a neighbor joining tree, so I used the alignment of those sequences with mine (1-43) in mega with TN93+G+I model but I don't understand how to interpret why my species in question (29) is so much different (or maybe novel?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Attached is my original tree (Picture 1) and my bootstrap analysis (Picture 2):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dG4SC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dG4SC.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/1syFE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1syFE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4148" LastEditorUserId="3848" LastEditDate="2019-08-20T22:04:29.783" LastActivityDate="2019-08-20T22:04:29.783" Title="Help interpreting my phylogenetic tree construction of bacterial species in same genus" Tags="|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7074" PostTypeId="2" ParentId="7072" CreationDate="2019-02-20T10:28:22.510" Score="2" Body="&lt;p&gt;Maybe you could fit a mixed model to predict phenotype, using the two genes as fixed effects and cancer type as a random effect, and ask the model for the separate effects of each gene, and the interaction between them.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-02-20T10:28:22.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7075" PostTypeId="2" ParentId="7072" CreationDate="2019-02-20T11:13:17.347" Score="1" Body="&lt;p&gt;Or even more simply using generalized linear modeling (GLM), where the regression weights are the genes and the accuracy of the model is measured in the residual. You also Q-Q plots to assess the fit of the model to the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the residual is high and Q-Q &quot;wobbly&quot; transformations works wonders.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Machine learning such as a neural network is a very cool way to do this calculation, but IMO you need Python - although people on this board do this in R. The concept is very easy you are catagorising (keep in mind I'm not an oncologist) gene vs. cancer resistance and you want neural network, linear-based regression etc.. to make the call (you will see it visually). You then use the weights to assess the relative contribution of each gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 10% issue is formally assessed, its not hard to obtain the power because its a common problem. The formal way to assess this, from memory, is you assign a &quot;null model&quot; and the accuracy of a ML model needs to a lot higher than the null model. I forget the details, I've never been in these situations. There's a specific technical name (which I should look up).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The explicit GLM / GLMM (non-ML) is the easiest approach however. If you use a Mac there is a great (relatively cheap) package for GLM, but any good software package will do. I assume GLMM is available in R (certainly Stata).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-02-20T11:13:17.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7076" PostTypeId="1" AcceptedAnswerId="7077" CreationDate="2019-02-20T11:32:19.300" Score="2" ViewCount="164" Body="&lt;p&gt;I have been looking ways to use data from various micro-array platform such as agilient, rosetta/merk. Some of the established method I came across is combat, but I'm not sure if it can be done for cross platform. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A similar question &lt;a href=&quot;https://support.bioconductor.org/p/77363/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; where the answer is kind of can't be or shouldn't be done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion how to take account of batch effects across multiple platforms.&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="48" LastEditDate="2019-02-20T13:34:58.810" LastActivityDate="2019-02-20T13:34:58.810" Title="Combat for multiplatform batch correction" Tags="|r|microarray|batch-effects|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7077" PostTypeId="2" ParentId="7076" CreationDate="2019-02-20T12:10:31.057" Score="3" Body="&lt;p&gt;Usually with microarrays you want to make a case/control comparison, so I am going to assume that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Data from different array platforms is generally difficult to compare:  each platform is measuring potentially a different part of the expression of the gene (different exons or different regions), each platform is likely to need different normalisation approaches, and have different dynamic range.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have case/control data from several different platforms, then the most robust thing to do is to analyse each platform separately, and produce a list of up and down-regulated genes according to each platform.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, it would be reasonably safe to combine the lists of genes in a 'meta-analysis' approach.  I would expect that in a lot of cases, 'up' and 'down' regulation would be correlated across platforms, but I would not necessarily expect very strong correlation in the magnitude of changes between platforms for individual genes.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-02-20T12:10:31.057" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7078" PostTypeId="2" ParentId="7073" CreationDate="2019-02-20T15:22:08.817" Score="3" Body="&lt;p&gt;The top phylogeny is likely mid-point rooted, but I suspect the rooting is fine. The lower diagram has been rooted on the query sequence. It is taxonomic analysis using 16S which in bacteria is considered to represent species delineation at a given percentage of divergence. I thought it was 5%, which is impossible based on the phylogram in the top picture.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bootstrap values lower than 80% mean that the clade is not supported (monophyletic group is a better term). The delineation between 10001 and 10013 against the massive ingroup of 13 16S sequences is robustly supported by bootstraps of 95% and 99%. The divergence between them is around 2%. There is no taxonomic distinction between the 13 16S sequences based on the phylogram.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One question is whether this is full length 16S? If it is there appears to be a species designation confusion in the lower clade supporting 10001 and 10013 because it is identical to to Dermococcus species and there is no way to assess which one is valid, nor whether they are really members of the same species. The very small divergence suggesting in D. abyssis might also call this into question. 10013 is in fact a robust outgroup to 10001 and the three Dermococcus spp. but the genetic distance involved is small.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this is not the major result. You have recovered what appears to be a large population (if that is how you sampled it) with close genetic homology to Dermococcus strain DSM20448. Given the complexity of species lineation within this genus it is impossible to say whether this represents the same species as DSM20448 but you could call it a &quot;species complex&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Overall the population structure (if that is how it was sampled) is heavily skewed towards a bacterial species which is hitherto received minimal attention. This population skew (if that is how it was sampled), could be reversed in different ecotopes/niches/ecoregions/patients? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, I wouldn't get caught up in systematics because in this genus its a bit messy. I would focus on the population structure and these results are very clear.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-02-20T15:22:08.817" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7079" PostTypeId="1" CreationDate="2019-02-20T15:53:21.387" Score="1" ViewCount="745" Body="&lt;p&gt;I want to extract the expression matrix from the GSE object in GEOquery.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(GEOquery)&#xA;&#xA;&#xA;gse1 &amp;lt;- getGEO(&quot;GSE97168&quot;,GSEMatrix=TRUE)&#xA;datExpr = exprs (gse1[[1]])&#xA;datExpr&#xA;&#xA;GSM2553017 GSM2553018 GSM2553019 GSM2553020 GSM2553021 GSM2553022 GSM2553023 GSM2553024 GSM2553025&#xA;     GSM2553026&#xA;&#xA;&#xA;Table(GSMList(gse1)[[1]])[1:5,]&#xA;&#xA;Table(GSMList(gse1)[[1]])[1:5,]&#xA;Error in (function (classes, fdef, mtable)  : &#xA;  unable to find an inherited method for function ‘GSMList’ for signature ‘&quot;list&quot;’&#xA;&#xA;Table(GPLList(gse1)[[1]])[1:5,]&#xA;Error in (function (classes, fdef, mtable)  : &#xA;  unable to find an inherited method for function ‘GPLList’ for signature ‘&quot;list&quot;’&#xA;&#xA;Meta(gse1)$description&#xA;Error in (function (classes, fdef, mtable)  : &#xA;  unable to find an inherited method for function ‘Meta’ for signature ‘&quot;list&quot;’&#xA;&#xA;Columns(gse1)&#xA;Error in (function (classes, fdef, mtable)  : &#xA;  unable to find an inherited method for function ‘Columns’ for signature ‘&quot;list&quot;’&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Why it's not working with R version 3.5.1?&lt;/p&gt;&#xA;" OwnerUserId="4155" LastEditorUserId="77" LastEditDate="2019-02-21T08:14:00.380" LastActivityDate="2020-03-17T10:02:25.313" Title="GEOquery errors due to list input" Tags="|single-cell|geoquery|gse|geo|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7080" PostTypeId="1" CreationDate="2019-02-20T17:02:44.117" Score="0" ViewCount="348" Body="&lt;p&gt;Does anyone know where I can download cancer raw reads (fastq files), tumor and Germline for non-humans? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to make a study with human data but I don't have the control access to download raw reads.&lt;/p&gt;&#xA;" OwnerUserId="4156" LastActivityDate="2019-02-21T07:51:31.200" Title="Where to Download Cancer Raw Reads (fastq)?" Tags="|rna-seq|ngs|fastq|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7081" PostTypeId="1" AcceptedAnswerId="7092" CreationDate="2019-02-20T17:35:18.533" Score="0" ViewCount="75" Body="&lt;p&gt;This is my &lt;a href=&quot;https://drive.google.com/open?id=1BNfxuEuagZ0BSjMAN24cTIbIDxtBpBSg&quot; rel=&quot;nofollow noreferrer&quot;&gt;file&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df1 = read.csv(&quot;HSC_LSC_BLAST_karyotyope.txt&quot;,header = TRUE,sep = &quot;,&quot;,row.names = 1)&#xA;&#xA;metadata &amp;lt;- data.frame(row.names = colnames(df1))&#xA;metadata&lt;span class=&quot;math-container&quot;&gt;$Group &amp;lt;- rep(NA, ncol(df1))&#xA;metadata$&lt;/span&gt;Group[seq(1,4,1)] &amp;lt;- 'HSC'&#xA;metadata&lt;span class=&quot;math-container&quot;&gt;$Group[seq(5,15,1)] &amp;lt;- 'Blast'&#xA;metadata$&lt;/span&gt;Group[seq(16,23,1)] &amp;lt;- 'LSC'&#xA;#metadata&lt;span class=&quot;math-container&quot;&gt;$Group[seq(13,16,1)] &amp;lt;- 'Mono'&#xA;metadata$&lt;/span&gt;Cytogenetics[seq(1,4,1)] &amp;lt;- 'Healthy'&#xA;metadata&lt;span class=&quot;math-container&quot;&gt;$Cytogenetics[seq(5,21,1)] &amp;lt;- 'NK'&#xA;metadata$&lt;/span&gt;Cytogenetics[seq(22,23,1)] &amp;lt;- 'Abnormal'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;s metadata&lt;span class=&quot;math-container&quot;&gt;$Cytogenetics[seq(1,4,1)] &amp;lt;- 'Healthy' Error in `$&lt;/span&gt;&amp;lt;-.data.frame`(`*tmp*`, Cytogenetics, value = c(&quot;Healthy&quot;,  :   &#xA;&amp;gt; replacement has 4 rows, data has 23&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Im trying to add other metadata information based on clinical information apart from giving sample labeling , im certainly doing something wrong but im not sure what it is. As my idea is the another column named Cytogenetics which would have respective information status but im getting that error.&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="57" LastEditDate="2019-02-23T00:13:57.020" LastActivityDate="2019-02-23T00:13:57.020" Title="Error in adding data in columns of R data frame" Tags="|r|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7082" PostTypeId="2" ParentId="7081" CreationDate="2019-02-20T20:37:27.400" Score="2" Body="&lt;p&gt;Add the &lt;code&gt;Cytogenetics&lt;/code&gt; columns and then assign things to a subset of it.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;metadata &amp;lt;- data.frame(row.names = colnames(df1))&#xA;metadata&lt;span class=&quot;math-container&quot;&gt;$Group &amp;lt;- NA     # you don't need to write a vector; just NA does the job&#xA;metadata$&lt;/span&gt;Group[seq(1,4,1)] &amp;lt;- 'HSC'&#xA;metadata&lt;span class=&quot;math-container&quot;&gt;$Group[seq(5,15,1)] &amp;lt;- 'Blast'&#xA;metadata$&lt;/span&gt;Group[seq(16,23,1)] &amp;lt;- 'LSC'&#xA;metadata&lt;span class=&quot;math-container&quot;&gt;$Cytogenetics &amp;lt;- NA    # initiate the column with NA values&#xA;metadata$&lt;/span&gt;Cytogenetics[seq(1,4,1)] &amp;lt;- 'Healthy'&#xA;metadata&lt;span class=&quot;math-container&quot;&gt;$Cytogenetics[seq(5,21,1)] &amp;lt;- 'NK'&#xA;metadata$&lt;/span&gt;Cytogenetics[seq(22,23,1)] &amp;lt;- 'Abnormal'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastEditorUserId="57" LastEditDate="2019-02-21T12:51:09.340" LastActivityDate="2019-02-21T12:51:09.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7084" PostTypeId="1" AcceptedAnswerId="7095" CreationDate="2019-02-20T21:30:43.197" Score="3" ViewCount="274" Body="&lt;p&gt;I downloaded the annotation of the &lt;em&gt;C. elegans&lt;/em&gt; genome in GFF3 format from &lt;a href=&quot;ftp://ftp.ensembl.org/pub/release-95/gff3/caenorhabditis_elegans/Caenorhabditis_elegans.WBcel235.95.gff3.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl&lt;/a&gt;.&#xA;I typed the following command, hoping to get the header of the file (lines starting with &lt;code&gt;#&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep '^#' Caenorhabditis_elegans.WBcel235.95.gff3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This returned 46798 lines, many many more than the dozen or so header lines. After searching the file for the &lt;code&gt;#&lt;/code&gt; character, I noticed many lines that contain only three hashtag (&lt;code&gt;#&lt;/code&gt;) symbols. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qWbuD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qWbuD.png&quot; alt=&quot;screenshot showing hashtags in GFF3 file&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why are these lines there?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="96" LastEditDate="2019-02-21T17:57:17.567" LastActivityDate="2020-04-25T21:18:43.537" Title="Purpose of ### (3 consecutive pound signs / hashtags / octothorps) in GFF3" Tags="|ensembl|sequence-annotation|gff3|gff|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7085" PostTypeId="2" ParentId="7084" CreationDate="2019-02-20T22:22:52.310" Score="-1" Body="&lt;p&gt;Hmmm, I dunno. If I was doing this I'd want an easy way to parse the file and ### would be a good marker. Anyway just pop a space character after the #. Using sed ...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -n '/^# /p' &amp;lt; ./Caenorhabditis_elegans.WBcel235.95.gff3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That should remove the ### lines&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also &lt;code&gt;head -20&lt;/code&gt; and &lt;code&gt;sed -n '/1,20/p' ./Caenorhabditis_elegans.WBcel235.95.gff3&lt;/code&gt; would get around the problem with a bit of tweaking.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The grep verion is (removing unnecessary jargon)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep '^# ' Caenorhabditis_elegans.WBcel235.95.gff3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the same command,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep '^#\s' Caenorhabditis_elegans.WBcel235.95.gff3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The point behind the code is there is no negation operator, in sed (I assume grep is the same). In Perl you would simple [^#]{3} (need checking) in your regex (remove all ###). In sed (or grep) you can exclude them using the ^ (beginning of a line) and suffixed with a &quot;space&quot; with only a single character in the middle all 3 letter patterns are removed.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Another approach without the space character is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -n '/^###/d; /^#/p' ./Caenorhabditis_elegans.WBcel235.95.gff3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Albeit this may require,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -i '/^###/d; /^#/p' ./file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-02-21T15:44:41.507" LastActivityDate="2019-02-21T15:44:41.507" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="7086" PostTypeId="2" ParentId="7080" CreationDate="2019-02-21T07:51:31.200" Score="2" Body="&lt;p&gt;You can try to browse SRA (sequence read archive) at &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/sra&lt;/a&gt;. &#xA;To download fastq files from SRA, you'll need their sratoolkit: &lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/sra.cgi?view=software&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://trace.ncbi.nlm.nih.gov/Traces/sra/sra.cgi?view=software&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easy command to download fastq files by their accession number is :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fastq-dump --split-files accession_number&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2619" LastActivityDate="2019-02-21T07:51:31.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7087" PostTypeId="2" ParentId="7084" CreationDate="2019-02-21T07:53:52.353" Score="4" Body="&lt;p&gt;The three &lt;code&gt;#&lt;/code&gt; are used for splitting group of features that belong together, e.g. a transcript and it's exons. Sometimes you see with a blank line instead of the &lt;code&gt;#&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To &lt;code&gt;grep&lt;/code&gt; just the header, which has 1 or 2 &lt;code&gt;#&lt;/code&gt; from the line start on, you can use extended regular expression:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -E &quot;^#{1,2}[^#]&quot; Caenorhabditis_elegans.WBcel235.95.gff3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which means: &quot;grep all lines that started with 1 or 2 &lt;code&gt;#&lt;/code&gt; and are not followed by &lt;code&gt;#&lt;/code&gt;&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you like to create a new &lt;code&gt;gff3&lt;/code&gt; to get rid of this separator lines you can do this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -v &quot;^###&quot; Caenorhabditis_elegans.WBcel235.95.gff3 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;-v&lt;/code&gt; prints all line that doesn't match the pattern. (I guess this is what @Michael mean by &quot;negator&quot;)&lt;/p&gt;&#xA;" OwnerUserId="1310" LastActivityDate="2019-02-21T07:53:52.353" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7089" PostTypeId="2" ParentId="7079" CreationDate="2019-02-21T08:32:45.117" Score="1" Body="&lt;p&gt;Use &lt;code&gt;GSEMatrix=FALSE&lt;/code&gt;, the information you want appears to be in the soft file, rather than the matrix. Yes, this is moderately annoying since it means you need two objects to describe this particular dataset.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-02-21T08:32:45.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7090" PostTypeId="2" ParentId="7084" CreationDate="2019-02-21T09:27:35.827" Score="3" Body="&lt;p&gt;As others have stated, those are just there to separate the entries for easier parsing. They enable you to do nifty tricks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk -v RF='###' '/Y74C9A.3/' Caenorhabditis_elegans.WBcel235.95.gff3 &#xA;I   WormBase    mRNA    4116    10230   .   -   .   ID=transcript:Y74C9A.3;Parent=gene:WBGene00022277;Name=Y74C9A.3;biotype=protein_coding;transcript_id=Y74C9A.3&#xA;I   WormBase    three_prime_UTR 4116    4220    .   -   .   Parent=transcript:Y74C9A.3&#xA;I   WormBase    exon    4116    4358    .   -   .   Parent=transcript:Y74C9A.3;Name=Y74C9A.3.e5;constitutive=1;ensembl_end_phase=-1;ensembl_phase=0;exon_id=Y74C9A.3.e5;rank=5;version=1&#xA;I   WormBase    CDS 4221    4358    .   -   0   ID=CDS:Y74C9A.3;Parent=transcript:Y74C9A.3;protein_id=Y74C9A.3&#xA;I   WormBase    exon    5195    5296    .   -   .   Parent=transcript:Y74C9A.3;Name=Y74C9A.3.e4;constitutive=1;ensembl_end_phase=0;ensembl_phase=0;exon_id=Y74C9A.3.e4;rank=4;version=1&#xA;I   WormBase    CDS 5195    5296    .   -   0   ID=CDS:Y74C9A.3;Parent=transcript:Y74C9A.3;protein_id=Y74C9A.3&#xA;I   WormBase    exon    6037    6327    .   -   .   Parent=transcript:Y74C9A.3;Name=Y74C9A.3.e3;constitutive=1;ensembl_end_phase=0;ensembl_phase=0;exon_id=Y74C9A.3.e3;rank=3;version=1&#xA;I   WormBase    CDS 6037    6327    .   -   0   ID=CDS:Y74C9A.3;Parent=transcript:Y74C9A.3;protein_id=Y74C9A.3&#xA;I   WormBase    exon    9727    9846    .   -   .   Parent=transcript:Y74C9A.3;Name=Y74C9A.3.e2;constitutive=1;ensembl_end_phase=0;ensembl_phase=0;exon_id=Y74C9A.3.e2;rank=2;version=1&#xA;I   WormBase    CDS 9727    9846    .   -   0   ID=CDS:Y74C9A.3;Parent=transcript:Y74C9A.3;protein_id=Y74C9A.3&#xA;I   WormBase    CDS 10095   10148   .   -   0   ID=CDS:Y74C9A.3;Parent=transcript:Y74C9A.3;protein_id=Y74C9A.3&#xA;I   WormBase    exon    10095   10230   .   -   .   Parent=transcript:Y74C9A.3;Name=Y74C9A.3.e1;constitutive=1;ensembl_end_phase=0;ensembl_phase=-1;exon_id=Y74C9A.3.e1;rank=1;version=1&#xA;I   WormBase    five_prime_UTR  10149   10230   .   -   .   Parent=transcript:Y74C9A.3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To extract the headers, you just need the lines that start with exactly 2 &lt;code&gt;#&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep '^##[^#]' Caenorhabditis_elegans.WBcel235.95.gff3 &#xA;##gff-version 3&#xA;##sequence-region   I 1 15072434&#xA;##sequence-region   II 1 15279421&#xA;##sequence-region   III 1 13783801&#xA;##sequence-region   IV 1 17493829&#xA;##sequence-region   MtDNA 1 13794&#xA;##sequence-region   V 1 20924180&#xA;##sequence-region   X 1 17718942&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, to also include the comments, lines that start with a &lt;code&gt;#&lt;/code&gt;, then either a &lt;code&gt;#&lt;/code&gt; or a &lt;code&gt;!&lt;/code&gt; and then a non-&lt;code&gt;#&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep '^#[#!][^#]' Caenorhabditis_elegans.WBcel235.95.gff3 &#xA;##gff-version 3&#xA;##sequence-region   I 1 15072434&#xA;##sequence-region   II 1 15279421&#xA;##sequence-region   III 1 13783801&#xA;##sequence-region   IV 1 17493829&#xA;##sequence-region   MtDNA 1 13794&#xA;##sequence-region   V 1 20924180&#xA;##sequence-region   X 1 17718942&#xA;#!genome-build WormBase WBcel235&#xA;#!genome-version WBcel235&#xA;#!genome-date 2012-12&#xA;#!genome-build-accession NCBI:GCA_000002985.3&#xA;#!genebuild-last-updated 2017-06&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or just a &lt;code&gt;#&lt;/code&gt;, any character and then a non&lt;code&gt;-#&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep '^#.[^#]' Caenorhabditis_elegans.WBcel235.95.gff3 &#xA;##gff-version 3&#xA;##sequence-region   I 1 15072434&#xA;##sequence-region   II 1 15279421&#xA;##sequence-region   III 1 13783801&#xA;##sequence-region   IV 1 17493829&#xA;##sequence-region   MtDNA 1 13794&#xA;##sequence-region   V 1 20924180&#xA;##sequence-region   X 1 17718942&#xA;#!genome-build WormBase WBcel235&#xA;#!genome-version WBcel235&#xA;#!genome-date 2012-12&#xA;#!genome-build-accession NCBI:GCA_000002985.3&#xA;#!genebuild-last-updated 2017-06&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="1075" LastEditDate="2019-02-21T12:22:35.550" LastActivityDate="2019-02-21T12:22:35.550" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7091" PostTypeId="2" ParentId="5482" CreationDate="2019-02-21T10:18:47.657" Score="2" Body="&lt;p&gt;You may want to try the longshot tool (&lt;a href=&quot;https://github.com/pjedge/longshot&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/pjedge/longshot&lt;/a&gt;) developed for calling variants in diploid genomes from long read data. &lt;/p&gt;&#xA;" OwnerUserId="4162" LastActivityDate="2019-02-21T10:18:47.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7092" PostTypeId="2" ParentId="7081" CreationDate="2019-02-21T13:04:32.333" Score="1" Body="&lt;p&gt;I know this is not really an answer, but I strongly discourage you from doing what you do, it's dangerous to impute data manually because it's prone to human mistakes - that you might mess up which columns should carry with metadata.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually, I think I have spotted a mistake already. This is your table using the corrected version of your code (with &lt;code&gt;metadata$Cytogenetics &amp;lt;- NA&lt;/code&gt;, see Devon's answer):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;metadata&#xA;        Group Cytogenetics&#xA;HSC1      HSC      Healthy&#xA;HSC2      HSC      Healthy&#xA;HSC3      HSC      Healthy&#xA;HSC4      HSC      Healthy&#xA;Blast11 Blast           NK&#xA;Blast12 Blast           NK&#xA;Blast1  Blast           NK&#xA;Blast2  Blast           NK&#xA;Blast3  Blast           NK&#xA;Blast4  Blast           NK&#xA;Blast6  Blast           NK&#xA;Blast7  Blast           NK&#xA;Blast8  Blast           NK&#xA;Blast9  Blast           NK&#xA;LSC1    Blast           NK&#xA;LSC2      LSC           NK&#xA;LSC3      LSC           NK&#xA;LSC4      LSC           NK&#xA;LSC6      LSC           NK&#xA;LSC7      LSC           NK&#xA;LSC8      LSC           NK&#xA;Blast5    LSC     Abnormal&#xA;LSC5      LSC     Abnormal&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I sort of think that the column &lt;code&gt;LSC1&lt;/code&gt; should be in the &lt;code&gt;LSC&lt;/code&gt; group and visa versa the &lt;code&gt;Blast5&lt;/code&gt; should be in &lt;code&gt;Blast&lt;/code&gt; group. In this case it seems that it's derived from name, you can actually write a code that will extract the group from the name.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df1 = read.csv(&quot;HSC_LSC_BLAST_karyotyope.txt&quot;, header = TRUE, sep = &quot;,&quot;, row.names = 1)&#xA;&#xA;metadata &amp;lt;- data.frame(row.names = colnames(df1))&#xA;metadata$Group &amp;lt;- sub(&quot;[^[:alpha:]]+&quot;, &quot;&quot;, (colnames(df1)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I don't know how do you know which sample is Healthy, NK or Abnormal. I suppose you have a table where this is written, if it's the case, use that table, don't impute the data manually.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2019-02-22T09:58:02.773" LastActivityDate="2019-02-22T09:58:02.773" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7093" PostTypeId="2" ParentId="7059" CreationDate="2019-02-21T13:05:53.500" Score="2" Body="&lt;p&gt;The DAVID tool is always interesting, and easy to try: &lt;a href=&quot;https://david.ncifcrf.gov/gene2gene.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://david.ncifcrf.gov/gene2gene.jsp&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You submit a list of genes, it uses several different annotation databases to annotate the gene list, and then it clusters the results and tests for statistical enrichment. You can potentially learn about functional enrichment in your gene list from several different independent annotations.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-02-21T13:05:53.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7094" PostTypeId="1" AcceptedAnswerId="18954" CreationDate="2019-02-21T14:54:17.997" Score="4" ViewCount="894" Body="&lt;p&gt;I making a bed file for RSeQC, so it can do things like compute the number of reads from exons, introns, 5&quot;UTRs, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to use a bed file that corresponds to my GTF file, so I use gtf2bed to make a bed file, like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{ if (&lt;span class=&quot;math-container&quot;&gt;$0 ~ &quot;transcript_id&quot;) print $&lt;/span&gt;0; else print &lt;span class=&quot;math-container&quot;&gt;$0&quot; transcript_id \&quot;\&quot;;&quot;; }' Homo_sapiens.GRCh38.gtf | gtf2bed - &amp;gt; $&lt;/span&gt;Homo_sapiens.GRCh38.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This file throws some errors, among other when using it with junction_annotation.py:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Reading reference bed file:  /rst1/2017-0205_illuminaseq/data/references/Reference_Genomes/GRCh38.95/Homo_sapiens.GRCh38.87.bed  ... Traceback (most recent call last):&#xA;  File &quot;/rst1/2017-0205_illuminaseq/tools/snakemake_conda_env/f4c16777/bin/junction_annotation.py&quot;, line 125, in &amp;lt;module&amp;gt;&#xA;    main()&#xA;  File &quot;/rst1/2017-0205_illuminaseq/tools/snakemake_conda_env/f4c16777/bin/junction_annotation.py&quot;, line 109, in main&#xA;    obj.annotate_junction(outfile=options.output_prefix,refgene=options.ref_gene_model,min_intron=options.min_intron, q_cut = options.map_qual)&#xA;  File &quot;/rst1/2017-0205_illuminaseq/tools/snakemake_conda_env/f4c16777/lib/python2.7/site-packages/qcmodule/SAM.py&quot;, line 3762, in annotate_junction&#xA;    exon_starts = map( int, fields[11].rstrip( ',\n' ).split( ',' ) )&#xA;ValueError: invalid literal for int() with base 10: 'gene_version'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and when I compare this bed file to the RSeQC provide (dated) one these are the differences:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My bed file I created:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1       11868   12227   ENSG00000223972 .       +       havana  exon    .       gene_id &quot;ENSG00000223972&quot;; gene_version &quot;5&quot;; transcript_id &quot;ENST00000456328&quot;; transcript_version &quot;2&quot;; exon_number &quot;1&quot;; gene_name &quot;DDX11L1&quot;; gene_source &quot;havana&quot;; gene_biotype &quot;transcribed_unprocessed_pseudogene&quot;; transcript_name &quot;DDX11L1-202&quot;; transcript_source &quot;havana&quot;; transcript_biotype &quot;processed_transcript&quot;; exon_id &quot;ENSE00002234944&quot;; exon_version &quot;1&quot;; tag &quot;basic&quot;; transcript_support_level &quot;1&quot;;&#xA;1       11868   14409   ENSG00000223972 .       +       havana  gene    .       gene_id &quot;ENSG00000223972&quot;; gene_version &quot;5&quot;; gene_name &quot;DDX11L1&quot;; gene_source &quot;havana&quot;; gene_biotype &quot;transcribed_unprocessed_pseudogene&quot;; transcript_id &quot;&quot;;&#xA;1       11868   14409   ENSG00000223972 .       +       havana  transcript      .       gene_id &quot;ENSG00000223972&quot;; gene_version &quot;5&quot;; transcript_id &quot;ENST00000456328&quot;; transcript_version &quot;2&quot;; gene_name &quot;DDX11L1&quot;; gene_source &quot;havana&quot;; gene_biotype &quot;transcribed_unprocessed_pseudogene&quot;; transcript_name &quot;DDX11L1-202&quot;; transcript_source &quot;havana&quot;; transcript_biotype &quot;processed_transcript&quot;; tag &quot;basic&quot;; transcript_support_level &quot;1&quot;;&#xA;1       12009   12057   ENSG00000223972 .       +       havana  exon    .       gene_id &quot;ENSG00000223972&quot;; gene_version &quot;5&quot;; transcript_id &quot;ENST00000450305&quot;; transcript_version &quot;2&quot;; exon_number &quot;1&quot;; gene_name &quot;DDX11L1&quot;; gene_source &quot;havana&quot;; gene_biotype &quot;transcribed_unprocessed_pseudogene&quot;; transcript_name &quot;DDX11L1-201&quot;; transcript_source &quot;havana&quot;; transcript_biotype &quot;transcribed_unprocessed_pseudogene&quot;; exon_id &quot;ENSE00001948541&quot;; exon_version &quot;1&quot;; tag &quot;basic&quot;; transcript_support_level &quot;NA&quot;;&#xA;1       12009   13670   ENSG00000223972 .       +       havana  transcript      .       gene_id &quot;ENSG00000223972&quot;; gene_version &quot;5&quot;; transcript_id &quot;ENST00000450305&quot;; transcript_version &quot;2&quot;; gene_name &quot;DDX11L1&quot;; gene_source &quot;havana&quot;; gene_biotype &quot;transcribed_unprocessed_pseudogene&quot;; transcript_name &quot;DDX11L1-201&quot;; transcript_source &quot;havana&quot;; transcript_biotype &quot;transcribed_unprocessed_pseudogene&quot;; tag &quot;basic&quot;; transcript_support_level &quot;NA&quot;;&#xA;1       12178   12227   ENSG00000223972 .       +       havana  exon    .       gene_id &quot;ENSG00000223972&quot;; gene_version &quot;5&quot;; transcript_id &quot;ENST00000450305&quot;; transcript_version &quot;2&quot;; exon_number &quot;2&quot;; gene_name &quot;DDX11L1&quot;; gene_source &quot;havana&quot;; gene_biotype &quot;transcribed_unprocessed_pseudogene&quot;; transcript_name &quot;DDX11L1-201&quot;; transcript_source &quot;havana&quot;; transcript_biotype &quot;transcribed_unprocessed_pseudogene&quot;; exon_id &quot;ENSE00001671638&quot;; exon_version &quot;2&quot;; tag &quot;basic&quot;; transcript_support_level &quot;NA&quot;;&#xA;1       12612   12697   ENSG00000223972 .       +       havana  exon    .       gene_id &quot;ENSG00000223972&quot;; gene_version &quot;5&quot;; transcript_id &quot;ENST00000450305&quot;; transcript_version &quot;2&quot;; exon_number &quot;3&quot;; gene_name &quot;DDX11L1&quot;; gene_source &quot;havana&quot;; gene_biotype &quot;transcribed_unprocessed_pseudogene&quot;; transcript_name &quot;DDX11L1-201&quot;; transcript_source &quot;havana&quot;; transcript_biotype &quot;transcribed_unprocessed_pseudogene&quot;; exon_id &quot;ENSE00001758273&quot;; exon_version &quot;2&quot;; tag &quot;basic&quot;; transcript_support_level &quot;NA&quot;;&#xA;1       12612   12721   ENSG00000223972 .       +       havana  exon    .       gene_id &quot;ENSG00000223972&quot;; gene_version &quot;5&quot;; transcript_id &quot;ENST00000456328&quot;; transcript_version &quot;2&quot;; exon_number &quot;2&quot;; gene_name &quot;DDX11L1&quot;; gene_source &quot;havana&quot;; gene_biotype &quot;transcribed_unprocessed_pseudogene&quot;; transcript_name &quot;DDX11L1-202&quot;; transcript_source &quot;havana&quot;; transcript_biotype &quot;processed_transcript&quot;; exon_id &quot;ENSE00003582793&quot;; exon_version &quot;1&quot;; tag &quot;basic&quot;; transcript_support_level &quot;1&quot;;&#xA;1       12974   13052   ENSG00000223972 .       +       havana  exon    .       gene_id &quot;ENSG00000223972&quot;; gene_version &quot;5&quot;; transcript_id &quot;ENST00000450305&quot;; transcript_version &quot;2&quot;; exon_number &quot;4&quot;; gene_name &quot;DDX11L1&quot;; gene_source &quot;havana&quot;; gene_biotype &quot;transcribed_unprocessed_pseudogene&quot;; transcript_name &quot;DDX11L1-201&quot;; transcript_source &quot;havana&quot;; transcript_biotype &quot;transcribed_unprocessed_pseudogene&quot;; exon_id &quot;ENSE00001799933&quot;; exon_version &quot;2&quot;; tag &quot;basic&quot;; transcript_support_level &quot;NA&quot;;&#xA;1       13220   13374   ENSG00000223972 .       +       havana  exon    .       gene_id &quot;ENSG00000223972&quot;; gene_version &quot;5&quot;; transcript_id &quot;ENST00000450305&quot;; transcript_version &quot;2&quot;; exon_number &quot;5&quot;; gene_name &quot;DDX11L1&quot;; gene_source &quot;havana&quot;; gene_biotype &quot;transcribed_unprocessed_pseudogene&quot;; transcript_name &quot;DDX11L1-201&quot;; transcript_source &quot;havana&quot;; transcript_biotype &quot;transcribed_unprocessed_pseudogene&quot;; exon_id &quot;ENSE00001746346&quot;; exon_version &quot;2&quot;; tag &quot;basic&quot;; transcript_support_level &quot;NA&quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The RSeQC provided bed file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1       11868   14409   ENST00000456328 0       +       11868   14409   0       3       359,109,1189,   0,744,1352,     DDX11L1&#xA;1       12009   13670   ENST00000450305 0       +       12009   13670   0       6       48,49,85,78,154,218,    0,169,603,965,1211,1443,        DDX11L1&#xA;1       17368   17436   ENST00000619216 0       -       17368   17436   0       1       68,     0,      MIR6859-1&#xA;1       14403   29570   ENST00000488147 0       -       14403   29570   0       11      98,34,152,159,198,136,137,147,99,154,37,        0,601,1392,2203,2454,2829,3202,3511,3864,10334,15130,   WASH7P&#xA;1       29553   31097   ENST00000473358 0       +       29553   31097   0       3       486,104,122,    0,1010,1422,    MIR1302-2&#xA;1       30266   31109   ENST00000469289 0       +       30266   31109   0       2       401,134,        0,709,  MIR1302-2&#xA;1       30365   30503   ENST00000607096 0       +       30365   30503   0       1       138,    0,      MIR1302-2&#xA;1       34553   36081   ENST00000417324 0       -       34553   36081   0       3       621,205,361,    0,723,1167,     FAM138A&#xA;1       35244   36073   ENST00000461467 0       -       35244   36073   0       2       237,353,        0,476,  FAM138A&#xA;1       52472   53312   ENST00000606857 0       +       52472   53312   0       1       840,    0,      OR4G4P&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Why are the files so different? How would I make a simpler bed file form my GTF file?&lt;/p&gt;&#xA;" OwnerUserId="3964" LastActivityDate="2022-04-19T19:21:53.153" Title="Making a bed file for RSeQC" Tags="|rna-seq|bed|gtf|rseqc|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7095" PostTypeId="2" ParentId="7084" CreationDate="2019-02-21T16:28:00.800" Score="4" Body="&lt;p&gt;There are some good answers so far, but I don't think any of them fully communicate the significance of the &lt;code&gt;###&lt;/code&gt; directive. The &lt;a href=&quot;https://github.com/The-Sequence-Ontology/Specifications/blob/master/gff3.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;GFF3 specification&lt;/a&gt; states:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;This directive (three &lt;code&gt;#&lt;/code&gt; signs in a row) indicates that all forward references to feature IDs that have been seen to this point have been resolved. After seeing this directive, a program that is processing the file serially can close off any open objects that it has created and return them, thereby allowing iterative access to the file. Otherwise, software cannot know that a feature has been fully populated by its subfeatures until the end of the file has been reached. It is recommended that complex features, such as the canonical gene, be terminated with the &lt;code&gt;###&lt;/code&gt; notation.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In other words, without the &lt;code&gt;###&lt;/code&gt; directive you have to load the entire GFF3 file into memory to make sure you've inferred all feature relationships completely (as encoded by &lt;code&gt;ID&lt;/code&gt; and &lt;code&gt;Parent&lt;/code&gt; attributes). With the &lt;code&gt;###&lt;/code&gt; directive, it's possible to process the annotations in smaller chunks, with a MUCH smaller peak memory usage.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2020-04-25T21:18:43.537" LastActivityDate="2020-04-25T21:18:43.537" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7096" PostTypeId="1" CreationDate="2019-02-21T18:57:22.463" Score="5" ViewCount="3163" Body="&lt;p&gt;I am trying to reproduce some results of a scRNASeq experiment. However I am new to the server-side aspect of such analyses and am very confused at the moment.&lt;/p&gt;&#xA;&lt;p&gt;The data provided by the authors of the paper is in .BAM format and from there I wish to derive a gene expression matrix (UMI counts per gene per cell). The authors stated that they did this using the 10X genomics pipeline, however it seems that .BAM outputs are sort of the &amp;quot;final version&amp;quot; of the pipeline and, the pipeline only really deals with truly raw sequencing files (bcl/fastq). I am trying to learn how to do all this processing myself as I will need to eventually. Unfortunately, most of the documents I have read end up making me more confused. I know BAM files are technically already processed and aligned but, how would I get the counts from the file?&lt;/p&gt;&#xA;&lt;p&gt;I have considered converting it back to &lt;code&gt;fastq&lt;/code&gt; format and following the pipeline from the beginning, but it just seems like I would be moving backwards by doing such.&lt;/p&gt;&#xA;&lt;p&gt;I have also found some R packages that can integrate BAM files in the environment like &lt;code&gt;Rsamtools&lt;/code&gt;, but I would much rather do this on the server and then upload a csv into R for the downstream analysis as the files are +60GB each.&lt;/p&gt;&#xA;&lt;p&gt;Essentially I am just asking for some advice, or a vignette/article I can read that would clear up how to proceed with the BAM outputs to derive the gene expression data. I have already read the &lt;code&gt;cellranger&lt;/code&gt; pages on the 10X genomics website but they were of no help in answering this quesion.&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your time,&lt;/p&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="73" LastEditDate="2023-06-12T21:16:03.557" LastActivityDate="2023-06-12T21:16:03.557" Title="BAM to gene expression matrix (UMI counts per gene per cell),10X" Tags="|sam|10x-genomics|cellranger|bash|umi|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7097" PostTypeId="2" ParentId="7096" CreationDate="2019-02-21T21:45:06.520" Score="4" Body="&lt;p&gt;You should be able to parse out what you need using the tags in the .bam.  10xGenomics' website says what tags they add.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/output/bam&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/output/bam&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Going backwards would also involve parsing the tags, because you have to make two fastq files, and a simple bam -&gt; fastq pipeline won't do that correctly.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastEditorUserId="2388" LastEditDate="2019-02-28T07:10:50.190" LastActivityDate="2019-02-28T07:10:50.190" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="7098" PostTypeId="1" CreationDate="2019-02-22T12:20:35.460" Score="6" ViewCount="798" Body="&lt;p&gt;I have &lt;a href=&quot;ftp://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_19/gencode.v19.annotation.gtf.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; GTF file and I use the command below on a Linux machine to extract the coding regions of the genome:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '{if(&lt;span class=&quot;math-container&quot;&gt;$3==&quot;transcript&quot; &amp;amp;&amp;amp; $&lt;/span&gt;20==&amp;quot;\&amp;quot;protein_coding\&amp;quot;;&amp;quot;){print $0}}' gencode.gtf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How I could do the inverse and keep only non coding regions?&lt;/p&gt;&#xA;&lt;p&gt;I need non coding regions of human genome, but for asking my question here I referred to coding parts too.&lt;/p&gt;&#xA;&lt;p&gt;After googling I see I actually need enhancers, promoters, or other regulatory elements of human genome to find cancer driver genes placed in these regions.&lt;/p&gt;&#xA;&lt;p&gt;Likely non-coding regions of genome are here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://elifesciences.org/download/aHR0cHM6Ly9jZG4uZWxpZmVzY2llbmNlcy5vcmcvYXJ0aWNsZXMvMjE3NzgvZWxpZmUtMjE3Nzgtc3VwcDQtdjMueGxzeA==/elife-21778-supp4-v3.xlsx?_hash=KQi5jfO3kT2c4Qw44j4Rg6YAyCBQilYuWHVYXcRDuuo%3D&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://elifesciences.org/download/aHR0cHM6Ly9jZG4uZWxpZmVzY2llbmNlcy5vcmcvYXJ0aWNsZXMvMjE3NzgvZWxpZmUtMjE3Nzgtc3VwcDQtdjMueGxzeA==/elife-21778-supp4-v3.xlsx?_hash=KQi5jfO3kT2c4Qw44j4Rg6YAyCBQilYuWHVYXcRDuuo%3D&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="73" LastEditDate="2021-05-23T12:32:11.430" LastActivityDate="2021-05-23T12:32:11.430" Title="Identify non-coding regions from a genome annotation" Tags="|sequence-annotation|genome|gtf|text-processing|interval|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7099" PostTypeId="2" ParentId="7098" CreationDate="2019-02-22T12:50:18.610" Score="4" Body="&lt;p&gt;Getting the non coding regions of a protein coding transcript, sounds like you are looking for &lt;a href=&quot;https://en.wikipedia.org/wiki/Untranslated_region&quot; rel=&quot;nofollow noreferrer&quot;&gt;UTR&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;UTR&lt;/code&gt; has its own feature in the gtf file. So you can do this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk -v FS=&quot;\t&quot; '$&lt;/span&gt;3==&amp;quot;UTR&amp;quot;' gencode.gtf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If the gtf file is compressed use this instead:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ zcat gencode.gtf.gz | awk -v FS=&quot;\t&quot; '$&lt;/span&gt;3==&amp;quot;UTR&amp;quot;'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;BTW: Why are you using such an old release of gencode? The current version is v29.&lt;/p&gt;&#xA;&lt;p&gt;As @Wouter &lt;a href=&quot;https://www.biostars.org/p/365086/#365131&quot; rel=&quot;nofollow noreferrer&quot;&gt;tells&lt;/a&gt; you, the non coding region of a genome is the complement of the coding regions. Coding regions have its own feature in the gtf file. You can get them with &lt;code&gt;$ awk -v FS=&amp;quot;\t&amp;quot; '$3==&amp;quot;CDS&amp;quot;' gencode.gtf&lt;/code&gt;. Reading the manual for &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/complement.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools complement&lt;/a&gt; is your task.&lt;/p&gt;&#xA;" OwnerUserId="1310" LastEditorUserId="73" LastEditDate="2021-05-23T12:31:30.700" LastActivityDate="2021-05-23T12:31:30.700" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7100" PostTypeId="1" AcceptedAnswerId="7101" CreationDate="2019-02-22T13:31:27.143" Score="3" ViewCount="319" Body="&lt;p&gt;Imagine a DNA sample containing a mixture of different intact plasmids. These samples are sequenced using either MiSeq or HiSeq sequencing. Would it possible to assemble these plasmids post-sequencing as they would have been when sequenced individually? &lt;/p&gt;&#xA;" OwnerUserId="2968" LastEditorUserId="57" LastEditDate="2019-02-22T13:57:34.847" LastActivityDate="2019-02-22T15:27:29.233" Title="Separation of mixed plasmid DNA sequences post whole-plasmid sequencing" Tags="|ngs|assembly|genome-sequencing|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7101" PostTypeId="2" ParentId="7100" CreationDate="2019-02-22T14:36:25.813" Score="4" Body="&lt;p&gt;You would expect to have high coverage, given the plasmids are short, so de novo assembly would be likely very easy.  Given that each plasmid is present in different multiples, you would expect different coverage on each plasmid, so it might be best to approach it as a metagenome-type or transcriptome assembly, rather than a classic genome assembly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, Google search just told me there is a tool called plasmidSPADES designed for just your purpose.  SPADES itself is a reliable genome assembler, so I would imagine plasmidSPADES to be a good choice.  It does assume that there is also some whole bacterial genomes in the sequencing mix though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As @terdon commented, it does depend on whether you have many identical copies of each plasmid, or many similar but not identical plasmids in the mix.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastEditorUserId="3828" LastEditDate="2019-02-22T14:57:44.093" LastActivityDate="2019-02-22T14:57:44.093" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7102" PostTypeId="2" ParentId="7100" CreationDate="2019-02-22T15:27:29.233" Score="2" Body="&lt;p&gt;Let's take a step back and consider the &quot;perfect&quot; output for a &lt;em&gt;de novo&lt;/em&gt; assembly algorithm. Ideally, you would like to see one complete sequence for molecule (chromosome, plasmid, etc.). In reality, this is difficult to achieve due to a couple factors.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;By random chance some regions may have low coverage, meaning that there are an insufficient number of reads spanning the region to reconstruct it &lt;em&gt;de novo&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;li&gt;A bigger problem is that some portions of the genome are repetitive and occur at many different locations, sometimes even on different chromosomes.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So in practice, it's very common that &lt;em&gt;de novo&lt;/em&gt; assemblies can only recover partial fragments of each molecule, and sometimes fragments are erroneously fused due to shared repetitive content.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Back to your question&lt;/strong&gt;: it's certainly possible to assemble distinct plasmids that were sequenced together without any special sample prep. How well this will work in practice will depend on the amount of coverage/redundancy in the sequenced reads and the amount of DNA shared between the plasmids. If they don't share much in common, and you sample to a sufficient depth of coverage, you shouldn't have any problem assembling them.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-02-22T15:27:29.233" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7103" PostTypeId="2" ParentId="7098" CreationDate="2019-02-22T18:34:06.030" Score="2" Body="&lt;p&gt;If you want all transcripts from that gtf file whose type isn't &quot;protein_coding&quot;, you can use almost the same command, just change the &lt;code&gt;==&lt;/code&gt; (&quot;is&quot;) to &lt;code&gt;!=&lt;/code&gt; (&quot;isn't&quot;): &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{if(&lt;span class=&quot;math-container&quot;&gt;$3==&quot;transcript&quot; &amp;amp;&amp;amp; $&lt;/span&gt;20!=&quot;\&quot;protein_coding\&quot;;&quot;){print $0}}' gencode.gtf &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, a simpler version:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '&lt;span class=&quot;math-container&quot;&gt;$3==&quot;transcript&quot; &amp;amp;&amp;amp; $&lt;/span&gt;20!=&quot;\&quot;protein_coding\&quot;;&quot;' gencode.gtf &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that this will not include any of the havana transcripts in the file, but I am assuming that's what you want since that's what your original command did. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Specifically, the command will return the following types of transcript (the numbers on the left are the number of such transcripts in the file):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '&lt;span class=&quot;math-container&quot;&gt;$3==&quot;transcript&quot; &amp;amp;&amp;amp; $&lt;/span&gt;20!=&quot;\&quot;protein_coding\&quot;;&quot;{print $20}' gencode.gtf  | sort | uniq -c | sort -nk1&#xA;      1 &quot;translated_processed_pseudogene&quot;;&#xA;      2 &quot;Mt_rRNA&quot;;&#xA;      3 &quot;IG_J_pseudogene&quot;;&#xA;      3 &quot;TR_D_gene&quot;;&#xA;      4 &quot;TR_J_pseudogene&quot;;&#xA;      5 &quot;TR_C_gene&quot;;&#xA;     10 &quot;IG_C_pseudogene&quot;;&#xA;     18 &quot;IG_C_gene&quot;;&#xA;     18 &quot;IG_J_gene&quot;;&#xA;     22 &quot;Mt_tRNA&quot;;&#xA;     25 &quot;3prime_overlapping_ncrna&quot;;&#xA;     27 &quot;TR_V_pseudogene&quot;;&#xA;     37 &quot;IG_D_gene&quot;;&#xA;     58 &quot;non_stop_decay&quot;;&#xA;     59 &quot;polymorphic_pseudogene&quot;;&#xA;     74 &quot;TR_J_gene&quot;;&#xA;     97 &quot;TR_V_gene&quot;;&#xA;    144 &quot;IG_V_gene&quot;;&#xA;    182 &quot;unitary_pseudogene&quot;;&#xA;    196 &quot;IG_V_pseudogene&quot;;&#xA;    330 &quot;sense_overlapping&quot;;&#xA;    387 &quot;pseudogene&quot;;&#xA;    442 &quot;transcribed_processed_pseudogene&quot;;&#xA;    531 &quot;rRNA&quot;;&#xA;    802 &quot;sense_intronic&quot;;&#xA;    860 &quot;transcribed_unprocessed_pseudogene&quot;;&#xA;   1529 &quot;snoRNA&quot;;&#xA;   1923 &quot;snRNA&quot;;&#xA;   2050 &quot;misc_RNA&quot;;&#xA;   2549 &quot;unprocessed_pseudogene&quot;;&#xA;   3116 &quot;miRNA&quot;;&#xA;   9710 &quot;antisense&quot;;&#xA;  10623 &quot;processed_pseudogene&quot;;&#xA;  11780 &quot;lincRNA&quot;;&#xA;  13052 &quot;nonsense_mediated_decay&quot;;&#xA;  25955 &quot;retained_intron&quot;;&#xA;  28082 &quot;processed_transcript&quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You might also want to remove that &quot;translated_processed_pseudogene&quot; since that is actually translated into protein and is therefore technically coding:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '&lt;span class=&quot;math-container&quot;&gt;$3==&quot;transcript&quot; &amp;amp;&amp;amp; &#xA;     $&lt;/span&gt;20!=&quot;\&quot;protein_coding\&quot;;&quot; &amp;amp;&amp;amp;&#xA;     $20!=&quot;\&quot;translated_processed_pseudogene\&quot;;&quot;' gencode.gtf  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-02-22T20:22:06.987" LastActivityDate="2019-02-22T20:22:06.987" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7104" PostTypeId="2" ParentId="7098" CreationDate="2019-02-22T19:03:51.273" Score="3" Body="&lt;p&gt;This isn't a problem that's easily solved with awk. It's not like you're extracting a feature that's annotated in the GTF file. Instead, you want the &lt;em&gt;empty space&lt;/em&gt; between annotated features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few years ago I wrote a program called &lt;a href=&quot;https://aegean.readthedocs.io/en/stable/locuspocus.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;LocusPocus&lt;/a&gt; for a similar task. It uses a gene annotation to break down a genome into gene loci and intergenic regions. It handles overlapping annotations and other weirdness pretty robustly. The output will include both coding regions and non-coding regions, but you can identify the intergenic spaces as those with &lt;code&gt;iLocus_type&lt;/code&gt; equal to &lt;code&gt;iiLocus&lt;/code&gt; or &lt;code&gt;fiLocus&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: the &lt;code&gt;--delta&lt;/code&gt; parameter will extend each gene/transcript by 500bp by default.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Caveat&lt;/strong&gt;: the program only accepts GFF3 input by default. Hopefully it won't be too hard to convert your GTF to GFF3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Another caveat&lt;/strong&gt;: eventual interpretation of these data will depend on what features are annotated in the genome and which annotations you include vs ignore. Do you want your non-coding regions to include non-coding genes, or should these be treated separately? Some non-coding regions will be full of transposable elements and other repetitive DNA, while others will have enhancers, promoters, or other regulatory elements. It's important to tread carefully before you jump to any conclusions.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2019-02-22T19:14:00.553" LastActivityDate="2019-02-22T19:14:00.553" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7105" PostTypeId="1" AcceptedAnswerId="8757" CreationDate="2019-02-23T16:43:58.607" Score="4" ViewCount="639" Body="&lt;p&gt;I have been trying to understand an error that the &lt;a href=&quot;https://github.com/EBIvariation/vcf-validator&quot; rel=&quot;nofollow noreferrer&quot;&gt;EBI's vcf_validator&lt;/a&gt; gives when run on my vcf file. Consider this minimal file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.1&#xA;##reference=foo&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&quot;Genotype&quot;&amp;gt;&#xA;##contig=&amp;lt;ID=chr12&amp;gt;&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  Sample1&#xA;chr12   529514  .   AACAC   AATAC   0.00    .   .   GT  0/1&#xA;chr12   529516  .   C   T   .   PASS    .   GT  0/1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I then run vcf_validator on it, it complains about:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ cat $&lt;/span&gt;(LC_ALL=&quot;C&quot; ~/vcf_validator_linux  -i test.vcf |&amp;amp; awk '/written to/{print $NF}' )&#xA;According to the VCF specification, the input file is not valid&#xA;Error: Duplicated variant chr12:529516:C&amp;gt;T found. This occurs 2 time(s), first time in line 6.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is because the two variants are indeed the same. Changing &#xA;&lt;code&gt;AACAC&lt;/code&gt; to &lt;code&gt;AATAC&lt;/code&gt; at position 529514 just means change &lt;code&gt;C&lt;/code&gt; to &lt;code&gt;T&lt;/code&gt; at position 529516.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OK, but how is that against the VCF standard? What makes that an invalid VCF file? I am trying to automatically validate input VCF files and stop the analysis if they're invalid, but this seems like a perfectly valid file. Why is  &lt;code&gt;vcf_validator&lt;/code&gt; not letting it through? The &lt;code&gt;vcf-validator&lt;/code&gt; perl script from vcftools doesn't complain about this. So is it just a bug in  EBI's &lt;code&gt;vcf_validator&lt;/code&gt; or is it really somehow illegal to have duplicate variants in a VCF file according to &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the specs&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-02-23T21:19:32.130" LastActivityDate="2019-06-05T04:19:33.577" Title="Are duplicate variants against the VCF standard?" Tags="|vcf|variation|vcftools|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7106" PostTypeId="1" AcceptedAnswerId="9337" CreationDate="2019-02-24T18:25:50.583" Score="0" ViewCount="659" Body="&lt;p&gt;I have mutation data of patients.How do i add that mutation info or annotate those as column with my annotation information &lt;a href=&quot;https://drive.google.com/open?id=1T-zSJ4pOYf7rB5Hoe6-fHzj6c54t6d5h&quot; rel=&quot;nofollow noreferrer&quot;&gt;expression&lt;/a&gt; and &lt;a href=&quot;https://drive.google.com/open?id=1TE55ORXGg_QEdlLp5GSyY-6_jirtt4cj&quot; rel=&quot;nofollow noreferrer&quot;&gt;mutation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To help myself i did try this &lt;a href=&quot;http://jokergoo.github.io/supplementary/ComplexHeatmap-supplementary1-4/supplS4_GBM_450K/supplS4_gbm_450K.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zuguang Gu&lt;/a&gt; but couldn;t able to do it, i would like similar to the one to show patient mutations .&#xA;any help or suggestion would be highly appreciated .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted something like this , as here they annotated with batch ,type and cell clone. I would like to have something similar where i would like to see how mutation in patients affects expression profile .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bDJDn.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bDJDn.jpg&quot; alt=&quot;figure&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df &amp;lt;- read.csv(&quot;HSC_pHSC_LSC_data/heatma_cluster.txt&quot;,header = TRUE,sep = &quot;,&quot;)&#xA;        head(df)&#xA;        tail(df)&#xA;        dim(df)&#xA;        heat &amp;lt;- t(scale(t(df[,3:ncol(df)])))&#xA;&#xA;        head(heat)&#xA;&#xA;        rownames(heat) &amp;lt;- df&lt;span class=&quot;math-container&quot;&gt;$Gene&#xA;        ColAnn &amp;lt;- data.frame(colnames(heat))&#xA;        Sample = c(rep(&quot;HSC&quot;,4),rep(&quot;pHSC&quot;,12),rep(&quot;LSC&quot;,8))&#xA;        anno &amp;lt;- as.data.frame(Sample)&#xA;        categories &amp;lt;- data.frame(anno)&#xA;        colnames(anno) &amp;lt;- c(&quot;Condition&quot;)&#xA;        ColAnn &amp;lt;- HeatmapAnnotation(df=anno,which=&quot;col&quot;, col=list(Condition=c(&quot;HSC&quot;=&quot;black&quot;,&quot;pHSC&quot;=&quot;red&quot;,&quot;LSC&quot;=&quot;dodgerblue4&quot;)))&#xA;        RowAnn &amp;lt;- data.frame(df$&lt;/span&gt;Family)&#xA;        colnames(RowAnn) &amp;lt;- c(&quot;Gene family&quot;)&#xA;        colours &amp;lt;- list(&quot;Gene family&quot;=c(&quot;Cluster_one&quot;=&quot;red&quot;,&quot;Cluster_two&quot;=&quot;blue&quot;,&quot;Cluster_three&quot;=&quot;brown&quot;,&quot;Cluster_four&quot;=&quot;yellow&quot;,&#xA;                                       &quot;Cluster_five&quot;=&quot;green&quot;,&quot;Cluster_six&quot;=&quot;maroon&quot;,&quot;Cluster_seven&quot;=&quot;pink&quot;,&quot;Cluster_eight&quot;=&quot;navyblue&quot;,&#xA;                                       &quot;Cluster_nine&quot;=&quot;darkgrey&quot;,&quot;Cluster_ten&quot;=&quot;violet&quot;))&#xA;&#xA;&#xA;&#xA;        RowAnn &amp;lt;- HeatmapAnnotation(df=RowAnn, col=colours, which=&quot;row&quot;)&#xA;        boxAnnCol &amp;lt;- HeatmapAnnotation(boxplot=anno_boxplot(heat, border=TRUE, gp=gpar(fill=&quot;#CCCCCC&quot;), &#xA;                                                            pch=&quot;.&quot;, size=unit(2, &quot;mm&quot;), &#xA;                                                            axis=TRUE,  axis_param = list(side = &quot;left&quot;)), &#xA;                                       annotation_width=unit(c(1, 6.5), &quot;cm&quot;))&#xA;&#xA;        boxAnnRow &amp;lt;- rowAnnotation(boxplot=row_anno_boxplot(heat, border=TRUE, gp=gpar(fill=&quot;#CCCCCC&quot;),pch=&quot;.&quot;,&#xA;                                                            size=unit(3, &quot;cm&quot;), axis=TRUE, axis_param = list(side = &quot;bottom&quot;)),&#xA;                                   annotation_width=unit(c(3), &quot;cm&quot;))&#xA;        pdf(&quot;ALL1_CLUSTER_DISEASE.pdf&quot;,width = 10, height = 20)&#xA;        #png(&quot;HSC_CMP_DIFF_HEATMAP.png&quot;, units=&quot;in&quot;, width=20, height=10, res=100)&#xA;&#xA;        #dev.off()&#xA;&#xA;        myCol &amp;lt;- colorRampPalette(c(&quot;navyblue&quot;, &quot;white&quot;, &quot;red&quot;))(100)&#xA;        myBreaks &amp;lt;- seq(-2, 2, length.out=100)&#xA;        hmap &amp;lt;- Heatmap(heat,&#xA;                        name=&quot;Z-score&quot;,&#xA;                        col=colorRamp2(myBreaks, myCol),&#xA;                        heatmap_legend_param=list(color_bar=&quot;continuous&quot;, &#xA;                                                  legend_direction=&quot;vertical&quot;, legend_width=unit(5,&quot;cm&quot;),&#xA;                                                  title_position=&quot;topcenter&quot;, title_gp=gpar(fontsize=10, fontface=&quot;bold&quot;)),&#xA;&#xA;                        #Split heatmap rows by gene family&#xA;                        split=df$Family,&#xA;                        #split = NULL,&#xA;&#xA;                        #Row annotation configurations&#xA;                        cluster_rows=TRUE,&#xA;                        show_row_dend=FALSE,&#xA;                        row_dend_side = &quot;left&quot;,&#xA;                        #row_title=&quot;Transcript&quot;, #overridden by 'split' it seems&#xA;                        row_title_side=&quot;left&quot;,&#xA;                        row_title_gp=gpar(fontsize=8,fontface=&quot;bold&quot;),&#xA;                        row_names_gp = gpar(fontsize = 8),&#xA;                        show_row_names=TRUE,&#xA;                        row_names_side=&quot;right&quot;,&#xA;                        row_title_rot=0,&#xA;&#xA;                        #Column annotation configuratiions&#xA;                        cluster_columns=TRUE,&#xA;                        show_column_dend=TRUE,&#xA;                        column_title=&quot;Samples&quot;,&#xA;                        column_title_side=&quot;top&quot;,&#xA;                        column_title_gp=gpar(fontsize=10, fontface=&quot;bold&quot;),&#xA;                        column_title_rot=0,&#xA;                        column_names_gp = gpar(fontsize = 20, fontface=&quot;bold&quot;),&#xA;                        show_column_names=TRUE,&#xA;&#xA;                        #Dendrogram configurations: columns&#xA;                        clustering_distance_columns=&quot;euclidean&quot;,&#xA;                        clustering_method_columns=&quot;complete&quot;,&#xA;                        column_dend_height=unit(10,&quot;mm&quot;),&#xA;&#xA;                        #Dendrogram configurations: rows&#xA;                        clustering_distance_rows=&quot;euclidean&quot;,&#xA;                        clustering_method_rows=&quot;complete&quot;,&#xA;                        row_dend_width=unit(15,&quot;mm&quot;),&#xA;&#xA;                        #top_annotation_height=unit(0.5,&quot;cm&quot;),&#xA;                        top_annotation=ColAnn,&#xA;&#xA;                        #bottom_annotation_height=unit(2, &quot;cm&quot;),&#xA;                        bottom_annotation=boxAnnCol)&#xA;        hmap + RowAnn&#xA;&#xA;        dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The image i get after running the code is something like this sample numbers vary, as i have only HSC ,pHSC and LSC samples &#xA;&lt;a href=&quot;https://i.stack.imgur.com/L55LN.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/L55LN.jpg&quot; alt=&quot;heatmap&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2019-03-05T10:06:14.407" LastActivityDate="2019-09-09T14:08:24.170" Title="Mutation annotation to the heatmap" Tags="|r|visualization|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="7108" PostTypeId="1" AcceptedAnswerId="7109" CreationDate="2019-02-25T00:38:34.327" Score="1" ViewCount="94" Body="&lt;p&gt;I'm looking to splice out specific amino acid alignment positions as a continuous motif from an alignment (~15 000 sequences). The code is written in Perl (I agree not many developers now) but splicing method is awk which a lot of you know. Please note the hash is &quot;tied&quot; (not shown), this just means the order of the sequences is preserved.  I can get the awk code below to return single amino acid positions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am struggling to get a loop working in awk.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I agree the Perl wrapper makes it more complicated, but its part of a huge data pipeline (all written in Perl).&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/local/ActivePerl-5.20/bin/env perl&#xA;#use strict;&#xA;use warnings;&#xA;&#xA;# I loaded the alignment into a hash, where key = sequence and value = protein&#xA;my %dna = (&#xA;    virus1 =&amp;gt; &quot;LGVLVILLMVQEGLKKRMTTKIIISTSMAVLVAMILGGFSMSDLAKLAILMGATFAEMNTGGDVAHLALIAAFKVRPALLVSFIF&quot;, &#xA;    virus2 =&amp;gt; &quot;LGVLVILLMVQEGLKKRMTTKIIISTSMAVLVAMILGGFSMSDLAKLAILMGATFAEMNTGGDVXXXXXXAAFKVRPALLVSFIF&quot;, &#xA;    virus3 =&amp;gt; &quot;LGVLVILLMVQEGLKKRMTTKIIISTSMAVLVAMILGGFSMSDLAKLAILMGATFAEMNTGGZZZZZZLIAAFKVRPALLVSFIF&quot;&#xA;);&#xA;&#xA;# I define the start and end of the alignment positions I am splicing&#xA;my &lt;span class=&quot;math-container&quot;&gt;$n = 1;&#xA;my $&lt;/span&gt;z = 5;&#xA;&#xA;# I turn the alignment into a string with separated by carriage returns &#xA;my @array = map {&quot;&lt;span class=&quot;math-container&quot;&gt;$dna{$_}&quot;} keys %dna;&#xA;my $&lt;/span&gt;string = join &quot;\n&quot;, @array;&#xA;&#xA;# This is the awk loop, note in Perl the &quot;``&quot; means issue a shell command &#xA;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$splice = `echo &quot;$&lt;/span&gt;string&quot; | awk 'BEGIN {FS=&quot;&quot;} {x = &lt;span class=&quot;math-container&quot;&gt;$n; while (x &amp;lt; $&lt;/span&gt;z); print \$ x; ++x}' 2&amp;gt;1`;&#xA;print $splice;&#xA;__DATA__&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Expected output&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;LGVLV&#xA;LGVLV&#xA;LGVLV&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Observed output&lt;/strong&gt;&#xA;I get the original sequences returned&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am familiar with Python/Perl solutions of line by line processing but awk is more efficient IMO. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-02-25T00:44:55.403" LastActivityDate="2019-02-25T06:14:56.173" Title="awk for motif splicing from an amino acid alignment" Tags="|perl|awk|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7109" PostTypeId="2" ParentId="7108" CreationDate="2019-02-25T01:14:44.037" Score="1" Body="&lt;p&gt;I &lt;em&gt;think&lt;/em&gt; you are trying to use &lt;code&gt;awk&lt;/code&gt; to get characters 3 through 7 (inclusive) of each input sequence. If so, a working version of your approach would be this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;my &lt;span class=&quot;math-container&quot;&gt;$splice = `echo &quot;$&lt;/span&gt;string&quot; | awk 'BEGIN {FS=\&quot;\&quot;} {x = &lt;span class=&quot;math-container&quot;&gt;$n; while (x &amp;lt; $&lt;/span&gt;z){ printf \&quot;%s\&quot;,\$x; ++x} print &quot;&quot;}' 2&amp;gt;1`;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That, when added to your perl script, produces:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ foo.pl&#xA;VLVIL&#xA;VLVIL&#xA;VLVIL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That isn't the output you show, but the output you show is just the first 5 letters of each sequence, which doesn't seem to correspond to what your script is doing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, the reason your version didn't work, is that the &lt;code&gt;while&lt;/code&gt; was like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while (x &amp;lt; $z); print \$ x; ++x}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Assuming the &lt;code&gt;\$ x&lt;/code&gt; was a typo, the problem is that the &lt;code&gt;++x&lt;/code&gt; isn't part of the while loop. So that should never even have exited. Seeing as we're in the same time zone, and given the time, I am guessing you're tired and that you miscopied something when trying to give us a reproducible example. In any case, to get the loop right, you need &lt;code&gt;{}&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while (x &amp;lt; $z){print \$x; ++x}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, if all you want is a substring, a loop is a very inefficient way of doing that. You can instead use &lt;code&gt;awk&lt;/code&gt;'s &lt;code&gt;substr&lt;/code&gt; command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  substr(s, m[, n ])&#xA;             Return  the at most n-character substring of s that begins at position&#xA;             m, numbering from 1. If n is omitted, or if n specifies  more  charac‐&#xA;             ters than are left in the string, the length of the substring shall be&#xA;             limited by the length of the string s.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, in your script, that would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;my &lt;span class=&quot;math-container&quot;&gt;$splice = `echo &quot;$&lt;/span&gt;string&quot; | awk '{print substr(\$0,$n,5)}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Since you're already using a language that excels in string manipulation, I really don't understand why you would try to use &lt;code&gt;awk&lt;/code&gt; instead. It will not be faster or more efficient than the same thing in perl:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print substr(&lt;span class=&quot;math-container&quot;&gt;$_,$&lt;/span&gt;n,5) . &quot;\n&quot; for @array;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;By calling &lt;code&gt;awk&lt;/code&gt; you are also calling the shell which will be slower by definition, especially if you're &lt;code&gt;echo&lt;/code&gt;ing thousands of lines. In fact, I tried this with 15000 sequences (just repetitions of your first one) and using either the &lt;code&gt;awk&lt;/code&gt; loop I gave above, or the &lt;code&gt;awk&lt;/code&gt;+&lt;code&gt;substr&lt;/code&gt;, or the native perl &lt;code&gt;substr&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;awk loop, average time taken to run the same thing 100 times: 0.0525&lt;/li&gt;&#xA;&lt;li&gt;awk substr, average time taken to run the same thing 100 times: 0.0522s&lt;/li&gt;&#xA;&lt;li&gt;perl substr, average time taken to run the same thing 100 times: 0.0464&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;And you can make it even faster by skipping the array and iterating over the keys directly:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print substr(&lt;span class=&quot;math-container&quot;&gt;$dna{$_},$&lt;/span&gt;n,5) . &quot;\n&quot; for keys(%dna);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That averaged at 0.0376s over 100 runs.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-02-25T01:35:36.460" LastActivityDate="2019-02-25T01:35:36.460" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7111" PostTypeId="2" ParentId="7108" CreationDate="2019-02-25T06:14:56.173" Score="1" Body="&lt;p&gt;If you have the sequences in a seperate file like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat input&#xA;LGVLVILLMVQEGLKKRMTTKIIISTSMAVLVAMILGGFSMSDLAKLAILMGATFAEMNTGGDVAHLALIAAFKVRPALLVSFIF&#xA;LGVLVILLMVQEGLKKRMTTKIIISTSMAVLVAMILGGFSMSDLAKLAILMGATFAEMNTGGDVXXXXXXAAFKVRPALLVSFIF&#xA;LGVLVILLMVQEGLKKRMTTKIIISTSMAVLVAMILGGFSMSDLAKLAILMGATFAEMNTGGZZZZZZLIAAFKVRPALLVSFIF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;you can use a simple &lt;code&gt;cut&lt;/code&gt; to fetch a substring:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cut -c1-5 input&#xA;LGVLV&#xA;LGVLV&#xA;LGVLV&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1310" LastActivityDate="2019-02-25T06:14:56.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7112" PostTypeId="1" CreationDate="2019-02-25T06:34:01.620" Score="1" ViewCount="189" Body="&lt;p&gt;I am trying to get base coverage information, and am using the &lt;code&gt;--PER_BASE_COVERAGE&lt;/code&gt; output from Picard tools to get a text file of base level coverage. However, some of my targets are being grouped together. In the targets file, these are separated by one base as such:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;12      12258595        12258695        +       TARGET1&#xA;12      12258696        12258795        +       TARGET2&#xA;12      12258796        12259791        +       TARGET3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, when I look at my covg file, I see the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr    pos    target   covg &#xA;12  12258595        TARGET1|TARGET2|TARGET3 510&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Why would HSmetrics group these targets together, and how can I separate them?&lt;/p&gt;&#xA;" OwnerUserId="4174" LastEditorUserId="57" LastEditDate="2019-02-25T09:31:37.147" LastActivityDate="2019-02-25T09:31:37.147" Title="CollectHsMetrics base coverage output has overlapping targets" Tags="|coverage|picard|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7114" PostTypeId="1" CreationDate="2019-02-25T13:49:06.393" Score="3" ViewCount="151" Body="&lt;p&gt;I searched for the package 'snakemake' on my SLURM cluster using:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;conda search --channel bioconda snakemake&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and I get many versions, up to 5.4.2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then try to install it using:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;conda install --channel bioconda --yes snakemake=5.4.2&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but it fails with the following message:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PackagesNotFoundError: The following packages are not available from current channels:&#xA;&#xA;  - snakemake&#xA;&#xA;Current channels:&#xA;&#xA;  - https://repo.anaconda.com/pkgs/main/linux-64&#xA;  - https://repo.anaconda.com/pkgs/main/noarch&#xA;  - https://repo.anaconda.com/pkgs/free/linux-64&#xA;  - https://repo.anaconda.com/pkgs/free/noarch&#xA;  - https://repo.anaconda.com/pkgs/r/linux-64&#xA;  - https://repo.anaconda.com/pkgs/r/noarch&#xA;  - https://repo.anaconda.com/pkgs/pro/linux-64&#xA;  - https://repo.anaconda.com/pkgs/pro/noarch&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is the bioconda channel not included in the current channels? I thought that was what the &lt;code&gt;--channel bioconda&lt;/code&gt; did in my install command.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;code&gt;conda install --channel bioconda --yes samtools=1.9&lt;/code&gt; worked fine and installed samtools successfully.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2019-02-25T15:13:30.647" Title="Why can I not install snakemake on my SLURM computer even though I can find it in the bioconda channel?" Tags="|software-installation|snakemake|bioconda|anaconda|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7115" PostTypeId="2" ParentId="7114" CreationDate="2019-02-25T15:13:30.647" Score="2" Body="&lt;p&gt;I created a file .condarc in my home directory with the following content:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;channels:&#xA;  - conda-forge&#xA;  - bioconda&#xA;  - defaults&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and it works now. I don't understand why this is necessary though. I thought the channels in the .condarc file would be used if I did not specify them in my command.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2019-02-25T15:13:30.647" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7116" PostTypeId="1" AcceptedAnswerId="7118" CreationDate="2019-02-25T15:53:30.263" Score="4" ViewCount="71" Body="&lt;p&gt;&lt;a href=&quot;https://jimb.stanford.edu/giab&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genome in a Bottle&lt;/a&gt; is an excellent resource that provides many types of DNA and RNA sequencing reads for a single individual/cell line to test genome assembly and analysis tools. For example there are Illumina WGS reads, Oxford Nanopore, PacBio, and 10X reads available for download at high coverage depth. In addition, there are pre-aligned bam files and verified variant calls.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any aggregation of similar robust single-individual sequencing resources for non-human, non-mammalian metazoan species? The only way I can consider finding such resources at the moment is going through SRA projects one at a time and hoping for the best.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="3848" LastEditDate="2023-04-22T20:28:02.020" LastActivityDate="2023-04-22T20:28:02.020" Title="Are there any Genome in a Bottle-like resources for non-humans (especially for invertebrates)?" Tags="|phylogenetics|ngs|phylogeny|genomics|public-dataset|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7117" PostTypeId="2" ParentId="7019" CreationDate="2019-02-25T15:55:00.997" Score="3" Body="&lt;p&gt;I agree with Chris Rands that a reasonable approach would be to call external tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you really want to do the phylogeny from within Python, you could use the P4 package, which is a bit complicated to handle but gives you lots of options in the way to build MCMC-based bayesian phylogenies:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/pgfoster/p4-phylogenetics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/pgfoster/p4-phylogenetics&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would still need something else to align the sequences before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To visualize the tree using python, you could use the ete toolkit, which is likely more powerful than what you can find in Biopython: &lt;a href=&quot;http://etetoolkit.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://etetoolkit.org/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2019-02-25T15:55:00.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7118" PostTypeId="2" ParentId="7116" CreationDate="2019-02-25T17:32:55.940" Score="2" Body="&lt;p&gt;In addition I would like to add &quot;&lt;a href=&quot;https://www.vectorbase.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vectorbase&lt;/a&gt;&quot;, the curator &quot;Dan&quot; is a good guy (I forget his surname right now) and it's a good team. However, please read my rather somber assessment of arthropod genomics first.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;We appear to be talking about lower metazoans, e.g. not protists, or yeasts (the latter I suspect are really good for your question). I don't know anything about worms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its important to understand that arthropods are several (soccer) divisions lower than mammals in terms of the question you are asking, except for two model organisms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.flybase.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Flybase&lt;/a&gt;, Drosophila melanogaster is the most important species for your purposes. The second most important species within the Class Insecta will be Anopheles gambiae (malaria mosquito). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally I would stick to the main players because overall whilst there are a lot of insect genomes (because there are a lot of insects), their genomes remain rough because of the difficulties in assembly and requirement to physical mapping their genomes in comparison to their genome size. Also except for D. melenogaster the (incredible) tardiness in producing a draft genome. For example the idea of PacBio is in its infancy for many entomologists: and this would help them a lot in assembling their genomes correctly. I agree things have picked a lot recently, but I remain a skeptic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Within the arthropods I would be very surprised if the crustacea, myriapods or Chelicerata (e.g. spiders) were in better condition than hexapods. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are interested in protists certainly Plasmodium falciparum (African malaria) has the sort of information you seek (&lt;a href=&quot;http://plasmodb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Plasmodb&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&#xA;I don't want to discourage you, insects are great, but you need to be selective if you are from a human/mammalian genetics background.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="298" LastEditDate="2019-02-25T18:19:55.530" LastActivityDate="2019-02-25T18:19:55.530" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7119" PostTypeId="1" CreationDate="2019-02-26T13:22:53.797" Score="1" ViewCount="552" Body="&lt;p&gt;I am very new to &lt;a href=&quot;http://jorgensen.biology.utah.edu/wayned/ape/&quot; rel=&quot;nofollow noreferrer&quot;&gt;APE&lt;/a&gt; software. I am trying to align complementary oligo-DNA strands using APE on macOS. I have both the forward and reverse sequence of an oligo-DNA insert in two separate files. But I am unable to align them as complementary sequences and show the sticky ends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible with APE? I tried &lt;strong&gt;Align two sequences&lt;/strong&gt; option. But couldn't get the expected result.&lt;/p&gt;&#xA;" OwnerUserId="4117" LastEditorUserId="298" LastEditDate="2019-02-26T15:52:12.637" LastActivityDate="2019-03-28T19:23:11.893" Title="Aligning nucleotide sequences in APE software" Tags="|sequence-alignment|phylogenetics|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7121" PostTypeId="1" CreationDate="2019-02-26T17:01:55.717" Score="1" ViewCount="102" Body="&lt;p&gt;I am trying to a recently published software called diffTF and contacted the authors on how to obtain the input &lt;a href=&quot;http://hocomoco11.autosome.ru/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HOCOMOCO&lt;/a&gt; files for GRCh38. While I wait for the answer, I thought I would post the question here as well:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tool requires bed files for each of the transcription factors (TF) in the genome, such as this bed file (TAB separated), which seems to give the locations of different TF binding sites in the genome:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    3457132     3457141     GTTGCGTGA   1266   +&#xA;chr1    4595374     4595383     GTTGCGTGA   1266   -&#xA;chr1    4996631     4996640     TTTGCGTGC   1276   +&#xA;chr1    5005357     5005366     TTTGCGTGC   1276   +&#xA;chr1    5170456     5170465     TTTGCGTGC   1276   -&#xA;chr1    5567302     5567311     GTTGCGTGA   1266   -&#xA;chr1    8676279     8676288     TTTGCGTGC   1276   -&#xA;chr1    8859719     8859728     GTTGCGTGA   1266   +&#xA;chr1    9600920     9600929     GTTGCGTGA   1266   +&#xA;chr1    9650971     9650980     GTTGCGTGA   1266   +&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then there is a symbols file to map the TF to their gene(?) that looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SYMBOL  ENSEMBL HOCOID&#xA;Ahr     ENSMUSG00000019256      AHR&#xA;Aire    ENSMUSG00000000731      AIRE&#xA;Alx1    ENSMUSG00000036602      ALX1&#xA;Ar      ENSMUSG00000046532      ANDR&#xA;Tfap2a  ENSMUSG00000021359      AP2A&#xA;Tfap2b  ENSMUSG00000025927      AP2B&#xA;Tfap2c  ENSMUSG00000028640      AP2C&#xA;Tfap2d  ENSMUSG00000042596      AP2D&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I googled for HOCOMOCO and found a bunch of pwn or words files, but none of them seems to be similar to the bed file above. Any ideas where to get these for GRCh38?&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="298" LastEditDate="2019-02-26T17:37:20.133" LastActivityDate="2019-02-27T22:31:32.843" Title="how to obtain GRCh38 bed and symbol files for HOCOMOCOv11" Tags="|transcription-factors|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7122" PostTypeId="1" AcceptedAnswerId="7124" CreationDate="2019-02-26T20:56:43.530" Score="3" ViewCount="138" Body="&lt;p&gt;I'm writing a tool to automate Sanger sequencing primer design for a production lab that uses a universal-tail chemistry Sanger sequencing to verify NGS results. Essentially, the template DNA is amplification using primers with universal 5' tails (M13F/R), and then the &quot;BigDye&quot; reaction is performed with an M13F and M13R primers in separate wells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm able to design logical primers for each region in a request using Primer3 (v2.5.0), but then I'd like to calculate folding thermodynamics for the M13-appended primers to determine the best primer partner for the 5' M13 appendages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, in the last step, I would like to know if I should use:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;M13F-forward_primer + M13R-reverse_primer&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;M13R-forward_primer + M13F-reverse_primer&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The problem is that Primer3 won't fold an oligo &gt; 36 nt, since the algorithm for Tm is diminishingly accurate for longer oligos. However, I believe thermodynamic predictions are valuable, even if only relatively, i.e. comparing one 5' appendage to another 5' appendage on the same allele-specific sequence, so I'd like the most painless answer, even if it isn't accurate in absolute terms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm aware that UNAFold will do this, but it seems like extreme overkill for what I need to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For lack of a better solution, I've implemented a workaround wherein I pair each allele-specific primer with either M13F or M13R and submit the pair as a &lt;code&gt;check_primer&lt;/code&gt; task. In other words, the left primer is one of the allele-specific primers that primer3 designed, and the right primer is either M13F, or M13R. I then choose the M13 appendage orientation based on the &lt;code&gt;PRIMER_PAIR_0_COMPL_ANY_TH&lt;/code&gt; output values for the two possible orientations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know: Is there a better solution? Primer3 docs say that the Tm calculation (and not necessarily folding thermodynamic calculations) is inaccurate for 37+ mers. Is there a way to force Primer3 to fold a sequence &gt; 36 nt? If not, is there a simple tool for folding &gt;36-mers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Follow-Up:&lt;/strong&gt; The workaround described in my OP was inadequate and UNAFold would cost $9k (which would take months for my employer to approve - don't get me started), so I tried @MichaelG's solution below. It turned out to work quite well. I'm computing the maximum Tm value for all fragments from every oligo, and that number agrees quite well with the browser-based UNAFold output.&lt;/p&gt;&#xA;" OwnerUserId="3564" LastEditorUserId="3848" LastEditDate="2023-07-15T16:50:54.230" LastActivityDate="2023-07-15T16:50:54.230" Title="programmatic secondary structure prediction for &gt;36-mer DNA oligonucleotides" Tags="|phylogenetics|phylogeny|software-recommendation|primer|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7124" PostTypeId="2" ParentId="7122" CreationDate="2019-02-27T11:08:56.360" Score="2" Body="&lt;p&gt;If you really insisted on using Primer3 you could perform a 36bp sliding window analysis. You would submit your primers in 36bp 'chunks' with a step size of say 3 bp. You would need to &quot;circularise&quot; your primer within the sliding window to assess the &quot;beginning&quot; and the &quot;end&quot; and remove &quot;artifactual&quot; hairpins. Its doable via a script and you would automate the web upload and data extraction via a &quot;digger/spider&quot; or else just use a download equivalent (I've used a standalone). There would be a &quot;blindspot&quot; using this approach, which you could overcome if you wanted. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So summary, its complicated and I think your suggestion of using UNAFold is a better idea.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-02-27T11:08:56.360" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7125" PostTypeId="2" ParentId="5002" CreationDate="2019-02-27T15:53:23.903" Score="2" Body="&lt;p&gt;That's one answer ^, not sure if you NEED to do that, which wouldn't actually classify your elements as LTRs, it would just find repetitive sequences in your new genome (as you discovered).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get them classified one standard approach is to use evidence of homology to sequences of LTR elements with classifications already, and you can get those in Repbase. If you want to download Repbase, you have to sign up for an account with GIRI but I think they give to anyone who asks. Then you can download the section of Repbase you're interested in (e.g. the whole thing, LTR, DNA-TE, ...) and provide the path to the downloaded library (in FASTA format I think) using &lt;code&gt;-lib&lt;/code&gt;, like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;RepeatMasker -lib Repbase_LTR.fa genome.fa&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then your RepeatMasker output table should have info about other kinds of TEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to check out my approach to classification by homology you might find some ideas you'd like to try. I used Dfam+nhmmer and Repbase+tblastx to identify evidence for LTR retrotransposon classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My software pipeline is available as open-source software. If you want detailed high-quality annotations of LTR retrotransponsons, PhyLTR will automatically annotate putative protein-coding sequence regions, classify them, and remove false-positives like a tandem array of DNA TEs which look like LTR-Rs (good diagram of LTR-R false positives here: &lt;a href=&quot;https://github.com/oushujun/LTR_retriever/blob/master/Manual.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/oushujun/LTR_retriever/blob/master/Manual.pdf&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Phylogenetic Analysis of LTR retrotransposons&#xA;&lt;a href=&quot;https://github.com/mcsimenc/PhyLTR&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mcsimenc/PhyLTR&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4187" LastEditorUserId="4187" LastEditDate="2019-02-27T16:00:26.193" LastActivityDate="2019-02-27T16:00:26.193" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="7126" PostTypeId="1" AcceptedAnswerId="7127" CreationDate="2019-02-27T15:56:46.077" Score="5" ViewCount="2934" Body="&lt;p&gt;This is related to the question I asked &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/7105/298&quot;&gt;here&lt;/a&gt;. Consider a vcf file that contains duplicate variants, but where the duplicates aren't simply the same thing in the same notation but instead one is a subset of the other. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.1&#xA;##reference=foo&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&quot;Genotype&quot;&amp;gt;&#xA;##contig=&amp;lt;ID=chr12&amp;gt;&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  Sample1&#xA;chr12   529514  .   AACAC   AATAC   .   PASS    .   GT  0/1&#xA;chr12   529516  .   C   T   .   PASS    .   GT  0/1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These the two variants are actually the same. They result in exactly the same genotype. Changing  AACAC to AATAC at position 529514 just means change C to T at position 529516.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any tool that can detect such duplicates and remove them? I tried &lt;a href=&quot;https://github.com/vcflib/vcflib#vcfuniq&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;vcfuniq&lt;/code&gt;&lt;/a&gt; from &lt;code&gt;vcflib&lt;/code&gt;, but that doesn't seem to recognize this as a duplicate. I think it only looks at the 1st 4 fields and only considers duplicates those variants with exactly the same values in the 1st 4 fields:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ./bin/vcfuniq test.vcf&#xA;##fileformat=VCFv4.1&#xA;##reference=foo&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&quot;Genotype&quot;&amp;gt;&#xA;##contig=&amp;lt;ID=chr12&amp;gt;&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  Sample1&#xA;chr12   529514  .   AACAC   AATAC   .   PASS    .   GT  0/1&#xA;chr12   529516  .   C   T   .   PASS    .   GT  0/1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, as explained in the &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/7105/298&quot;&gt;linked question&lt;/a&gt;, EBI's &lt;code&gt;vcf_validator&lt;/code&gt; considers this invalid. And it doesn't really make sense to have these duplicates in any case, so is there any way I can detect and remove them? Preferably an existing tool, but I am open to scripting solutions as well.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This is further complicated by cases like this one:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.1&#xA;##reference=foo&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&quot;Genotype&quot;&amp;gt;&#xA;##contig=&amp;lt;ID=chr12&amp;gt;&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  Sample1&#xA;chr12   529514  529514  AACAC   AAT,AATAC       0.00    .       .     GT    0/1&#xA;chr12   529516  529516  C       T       .       PASS    .     GT    0/1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, this one won't be caught by the approach in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/7128/298&quot;&gt;Daniel's clever script&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat test2.vcf | foo.py&#xA;##fileformat=VCFv4.1&#xA;##reference=foo&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&quot;Genotype&quot;&amp;gt;&#xA;##contig=&amp;lt;ID=chr12&amp;gt;&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  Sample1&#xA;chr12   529514  529514  AACAC   AAT,AATAC   0.00    .   .   GT  0/1&#xA;chr12   529516  529516  C   T   .   PASS    .   GT  0/1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="96" LastEditDate="2019-02-27T19:55:54.913" LastActivityDate="2019-03-01T15:21:03.897" Title="How can I remove (non-trivial) duplicates from a VCF file?" Tags="|vcf|variation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7127" PostTypeId="2" ParentId="7126" CreationDate="2019-02-27T16:10:34.370" Score="3" Body="&lt;p&gt;It turns out that &lt;code&gt;bcftools&lt;/code&gt; can do this (tested on bcftools-1.8), if you give it the reference genome to test against:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools norm -d none -f hg19.fa test.vcf&#xA;##fileformat=VCFv4.1&#xA;##FILTER=&amp;lt;ID=PASS,Description=&quot;All filters passed&quot;&amp;gt;&#xA;##reference=foo&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&quot;Genotype&quot;&amp;gt;&#xA;##contig=&amp;lt;ID=chr12&amp;gt;&#xA;##bcftools_normVersion=1.8+htslib-1.8&#xA;##bcftools_normCommand=norm -d none -f hg19.fa test.vcf; Date=Wed Feb 27 16:08:44 2019&#xA;#CHROM  POS     ID      REF     ALT     QUAL    FILTER  INFO    FORMAT  Sample1&#xA;chr12   529516  .       C       T       .       PASS    .       GT      0/1&#xA;Lines   total/split/realigned/skipped:  2/0/1/0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For the more complex case of the multi-allelic variant in the second VCF example from the question, you can run it through bcftools twice. Once using &lt;code&gt;norm&lt;/code&gt; to left-align and split multi-allelic variants, and then again to remove the duplicates:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools norm -m -any -NO z -O v -o - ~/test2.vcf |&#xA;    bcftools norm -d none -f hg19.fa&#xA;Lines   total/split/realigned/skipped:  2/1/0/0&#xA;##fileformat=VCFv4.1&#xA;##FILTER=&amp;lt;ID=PASS,Description=&quot;All filters passed&quot;&amp;gt;&#xA;##reference=foo&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&quot;Genotype&quot;&amp;gt;&#xA;##contig=&amp;lt;ID=chr12&amp;gt;&#xA;##bcftools_normVersion=1.8+htslib-1.8&#xA;##bcftools_normCommand=norm -m -any -NO z -O v -o - test2.vcf; Date=Wed Feb 27 18:18:32 2019&#xA;##bcftools_normCommand=norm -d none -f hg19.fa -; Date=Wed Feb 27 18:18:32 2019&#xA;#CHROM  POS     ID      REF     ALT     QUAL    FILTER  INFO    FORMAT  Sample1&#xA;chr12   529516  529514  CAC     T       0       .       .       GT      0/1&#xA;chr12   529516  529514  C       T       0       .       .       GT      0/0&#xA;Lines   total/split/realigned/skipped:  3/0/2/0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-03-01T15:21:03.897" LastActivityDate="2019-03-01T15:21:03.897" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7128" PostTypeId="2" ParentId="7126" CreationDate="2019-02-27T16:15:59.773" Score="2" Body="&lt;p&gt;I'm no expert with VCF (few can say they are!) but I have worked a lot with VCF data in the last few years, both tools to consume and produce VCF. I've never seen variants encoded in this fashion, and it seems to be non-canonical. Typically:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Single nucleotide variants (SNVs) are encoded with a single base as the REF allele and a single base as the ALT allele.&lt;/li&gt;&#xA;&lt;li&gt;In the case of insertions or deletions, the shorter of the REF and ALT alleles will be a single base, the base preceding the inserted/deleted sequence. Thus the first base of the REF and ALT alleles is always the same.&lt;/li&gt;&#xA;&lt;li&gt;In the rarer case of two or more consecutive substitutions forming a multinucleotide variant (MNV) the REF and ALT alleles will have the same length.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Using multi-bp strings of the same length to encode SNVs is unnecessary and, as you've pointed out, problematic. This makes me think its a bug or a &quot;feature&quot; of the variant predictor that produced the VCF.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case, I'd write a small script that would check for variants where the REF and ALT alleles have the same length. If the base is the same for REF and ALT in any position, drop it, and adjust the position accordingly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The script below will convert these funky SNVs to the canonical representation, and will also work on MNVs. Standard tools should then work to remove the duplicates.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;&#xA;def canonicalize(instream):&#xA;    for line in instream:&#xA;        if not line.startswith('#'):&#xA;            values = line.split('\t')&#xA;            pos = int(values[1])&#xA;            ref, alt = values[3:5]&#xA;            if len(ref) &amp;gt; 1 and len(ref) == len(alt):&#xA;                # How many bp to trim off the end&#xA;                for n, (r, a) in enumerate(zip(ref[::-1], alt[::-1])):&#xA;                    if r != a:&#xA;                        revoffset = -1 * n&#xA;                        break&#xA;&#xA;                # How many bp to trim off the front&#xA;                for n, (r, a) in enumerate(zip(ref, alt)):&#xA;                    if r != a:&#xA;                        offset = n&#xA;                        values[1] = str(pos + offset)&#xA;                        values[3] = ref[offset:revoffset]&#xA;                        values[4] = alt[offset:revoffset]&#xA;                        break&#xA;                line = '\t'.join(values)&#xA;        yield line&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    import sys&#xA;    for line in canonicalize(sys.stdin):&#xA;        print(line, end='')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;: Upon further reflection, the more complicated example you listed actually makes sense. In the reference we have the sequence &lt;code&gt;AACAC&lt;/code&gt;, and the alternate alleles represent two variations on this: deletion of the final two bp (in the first case), and a point mutation of the middle &lt;code&gt;C&lt;/code&gt; to &lt;code&gt;T&lt;/code&gt; (in both cases). Usually, only a single bp precedes the definition of an indel, so I would've encoded this complex variant as &lt;code&gt;ref=ACAC alt=AT,ATAC&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the SNV is &quot;implied by&quot;/&quot;encoded in&quot;/&quot;redundant with&quot; the complex variant, but it's not strictly a duplicate. I'm curious whether the VCF validator complains about these cases as well?&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2019-02-27T20:05:10.100" LastActivityDate="2019-02-27T20:05:10.100" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7129" PostTypeId="2" ParentId="343" CreationDate="2019-02-27T16:17:13.620" Score="4" Body="&lt;p&gt;AFAIK Dfam and Repbase are currently the two best sources of (a variety of) TE sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my genome annotations I have used RepeatModeler+RepeatMasker and then later used Repbase+tblastx and Dfam+nhmmer to classify them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The classification process in my pipeline PhyLTR (&lt;a href=&quot;https://github.com/mcsimenc/PhyLTR&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mcsimenc/PhyLTR&lt;/a&gt;) is based on Dfam and Repbase. The process I used for LTR identification is&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Putative ID with LTRHarvest (based on structural sequence characteristics)&lt;/li&gt;&#xA;&lt;li&gt;Classification-by-homology to Repbase and Dfam&lt;/li&gt;&#xA;&lt;li&gt;Removal of elements without homology to sequences in Repbase or Dfam.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This results in a set of LTR-Rs which are full-length and have evidence that they are LTR-Rs.&lt;/p&gt;&#xA;" OwnerUserId="4187" LastEditorUserId="4187" LastEditDate="2019-02-27T18:42:54.970" LastActivityDate="2019-02-27T18:42:54.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7130" PostTypeId="1" AcceptedAnswerId="7131" CreationDate="2019-02-27T18:16:43.147" Score="2" ViewCount="428" Body="&lt;p&gt;I am trying to replicate some results of a scRNASeq experiment and, when I looked at the data provided by the author, I noticed that some of the counts in the expression matrix are represented as decimals. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would this be because the authors uploaded a normalised count matrix or for other reasons? Also, please note that the authors used the 10X genomics &lt;code&gt;cellranger&lt;/code&gt; pipeline for the processing of raw data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have provided a brief snippet of the data frame here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(P1TLH_AAACCTGAGCAGCCTC_1 = c(0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0), P1TLH_AAACCTGTCCTCATTA_1 = c(0, &#xA;0, 0.314759545319035, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0.314759545319035, 0, 0), P1TLH_AAACCTGTCTAAGCCA_1 = c(0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 1.18024827050334, 0, 0, 0, 0, 0, 1.18024827050334, &#xA;0, 0, 0), P1TLH_AAACGGGAGTAGGCCA_1 = c(0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0.644905372424165, 0, 0, 0), P1TLH_AAACGGGGTTCGGGCT_1 = c(0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.780522072779651, &#xA;0, 0, 0), P1TLH_AAAGCAACAGTAAGAT_1 = c(0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0)), row.names = c(&quot;RP11-34P13.7&quot;, &#xA;&quot;FO538757.2&quot;, &quot;AP006222.2&quot;, &quot;RP4-669L17.10&quot;, &quot;RP5-857K21.4&quot;, &#xA;&quot;RP11-206L10.9&quot;, &quot;LINC00115&quot;, &quot;FAM41C&quot;, &quot;RP11-54O7.1&quot;, &quot;RP11-54O7.3&quot;, &#xA;&quot;SAMD11&quot;, &quot;NOC2L&quot;, &quot;KLHL17&quot;, &quot;PLEKHN1&quot;, &quot;RP11-54O7.17&quot;, &quot;HES4&quot;, &#xA;&quot;ISG15&quot;, &quot;AGRN&quot;, &quot;C1orf159&quot;, &quot;RP11-465B22.8&quot;), class = &quot;data.frame&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also, I am quite confused of the structure they used - as I don't have much experience with single cell datasets. Just to give some further context:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;colnames&lt;/code&gt; of the &quot;full&quot; dataframe are coded from P1TLH_sequence_1 to P5TLH_sequence_1. There are 5 patients in the study. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm assuming that each of the &lt;code&gt;colnames&lt;/code&gt; represents a cell for a particular patient while the rows are the genes.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this assumption is correct, how would I do the conversion from &lt;code&gt;colnames&lt;/code&gt; in the dataframe to the &quot;cell names&quot;? And lastly, why are there decimals in this expression matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you all for your time,&lt;/p&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="4001" LastEditDate="2019-02-27T20:15:28.943" LastActivityDate="2019-02-27T20:15:28.943" Title="scRNASeq expression matrix with decimal values" Tags="|r|scrnaseq|sequence-annotation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7131" PostTypeId="2" ParentId="7130" CreationDate="2019-02-27T18:49:13.157" Score="3" Body="&lt;p&gt;It looks like you have normalized counts. My guess is that they are log-normalized with some scale factor, probably 10000.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the whole matrix you should be able to work out the scale factor for each cell. See &lt;a href=&quot;https://www.nxn.se/valent/2018/10/25/unscaling-scaled-counts-in-scrna-seq-data&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nxn.se/valent/2018/10/25/unscaling-scaled-counts-in-scrna-seq-data&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;They typical format for scRNA-seq data is to have cells in the columns and genes in the rows. So here your rownames are the gene names and colnames are the cell names.&lt;/p&gt;&#xA;" OwnerUserId="3415" LastActivityDate="2019-02-27T18:49:13.157" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7132" PostTypeId="2" ParentId="7121" CreationDate="2019-02-27T21:15:04.653" Score="1" Body="&lt;p&gt;I guess you're right that you need predictions of transcription factor binding site (TFBS) through genome.&#xA;There're two options to get them: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You can generate predictions using any motif scanning tool. E.g. you can use our tool &lt;a href=&quot;https://github.com/VorontsovIE/sarus&quot; rel=&quot;nofollow noreferrer&quot;&gt;SPRY-SARUS&lt;/a&gt; which is able to generate bed-file in necessary format.&#xA;In this case, you can choose more or less strict thresholds for each prediction (i.e. more false positives vs more false negatives). Standard thresholds for HOCOMOCO motifs can be found at its downloads &lt;a href=&quot;http://hocomoco11.autosome.ru/downloads_v11&quot; rel=&quot;nofollow noreferrer&quot;&gt;page&lt;/a&gt; (we, as HOCOMOCO authors, recommend thresholds corresponding to P-value of 0.0005).&lt;/li&gt;&#xA;&lt;li&gt;I had no experience with diffTF but it seems that authors prepared TFBS predictions and put them on their site. Please, take a look at this &lt;a href=&quot;https://difftf.readthedocs.io/en/latest/chapter2.html#parameter-hocomoco-mapping&quot; rel=&quot;nofollow noreferrer&quot;&gt;page of manual&lt;/a&gt;. If you don't need extra-flexibility in the generation of TFBS predictions, it can be the simplest way to proceed.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4190" LastEditorUserId="57" LastEditDate="2019-02-27T22:31:32.843" LastActivityDate="2019-02-27T22:31:32.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7133" PostTypeId="1" AcceptedAnswerId="7135" CreationDate="2019-02-28T06:22:40.843" Score="1" ViewCount="1552" Body="&lt;p&gt;I am doing some whole-genome sequence analysis and to speed things up split the data into chromosomes and also into regions within those chromosomes. (So I have my dataset split into ~300ish files) I am doing some IBD analysis with plink files and to do this IBD analysis I am combining the files into one big plink file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran into issues today where when using plink 1.9 and the --bfile and --merge-list commands, the output fam file is different from the input fam file.  My dataset has related individuals so the fam file is very important.  Also its is important to note that when using binary files the order in the fam file needs to be exactly the same as the order the individuals appear in the other files (I learned this the hard way haha).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know why plink 1.9 changes the order in the fam file when merging plink files, and how to fix this?  It works with plink 1.07 but with 1.9 I get a different fam file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Performing &quot;diff&quot; on input and output fam files shows they are different:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plink=$mydir/bin/plink_1_9/plink&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;$plink --noweb --bfile $&lt;/span&gt;indir/chr22/285/&lt;span class=&quot;math-container&quot;&gt;$prefix.chr.22.region.285.qc --merge-list $&lt;/span&gt;indir/chr22/merge_list.txt --make-bed --out &lt;span class=&quot;math-container&quot;&gt;$indir/chr22/$&lt;/span&gt;prefix.chr.22.qc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using the same code but using plink-1.07 creates matching fam files as expected&quot;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plink1=bin/plink-1.07-x86_64/plink&#xA;$plink1 --noweb --bfile $indir/chr22/285/$prefix.chr.22.region.285.qc --merge-list $indir/chr22/merge_list.txt --make-bed --out $indir/chr22/$prefix.chr.22.qc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The merge_list.txt file is in this format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$indir/chr22/286/$prefix.chr.22.region.286.qc.bed $indir/chr22/286/$prefix.chr.22.region.286.qc.bim $indir/chr22/286/$prefix.chr.22.region.286.qc.fam&#xA;$indir/chr22/287/$prefix.chr.22.region.287.qc.bed $indir/chr22/287/$prefix.chr.22.region.287.qc.bim $indir/chr22/287/$prefix.chr.22.region.287.qc.fam&#xA;$indir/chr22/288/$prefix.chr.22.region.288.qc.bed $indir/chr22/288/$prefix.chr.22.region.288.qc.bim $indir/chr22/288/$prefix.chr.22.region.288.qc.fam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can someone explain to me why this could be happening and if there is a way to fix it? Thanks so much!!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Note, I know I can manually replace the fam file with the correct one with cp or mv, but I would like to know how to handle this with plink 1.9, if possible, as I am afraid it could be indicative of an underlying error in the merge process :) )&lt;/p&gt;&#xA;" OwnerUserId="1574" LastEditorUserId="1574" LastEditDate="2019-02-28T06:33:56.023" LastActivityDate="2019-03-04T18:37:04.523" Title="Plink 1.9 --merge-list changing order of fam file" Tags="|plink|wgs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7134" PostTypeId="2" ParentId="3667" CreationDate="2019-02-28T06:48:50.053" Score="3" Body="&lt;p&gt;Additional tidbit I learned in the last year that I wanted to share with anyone else that's working on this.&lt;/p&gt;&#xA;&lt;p&gt;Plink creates an extremely generic &lt;a href=&quot;https://www.cog-genomics.org/plink2/formats#fam&quot; rel=&quot;nofollow noreferrer&quot;&gt;fam&lt;/a&gt; file for you, but if you are updating this fam file to reflect your population's family structure, sex, etc., note that when using plink binary files (bed/bim/fam) the order of the individuals in the fam file needs to match the order of the individuals in the other files (which is the order of the individuals in the generic fam file)&lt;/p&gt;&#xA;&lt;p&gt;Do be careful if you care about which alleles are major/minor, as sometimes the order gets switched around in plink. For more info see &amp;quot;Variant Call Format&amp;quot; section on this &lt;a href=&quot;https://www.cog-genomics.org/plink2/input&quot; rel=&quot;nofollow noreferrer&quot;&gt;page&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1574" LastEditorUserId="73" LastEditDate="2022-02-22T20:58:20.653" LastActivityDate="2022-02-22T20:58:20.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7135" PostTypeId="2" ParentId="7133" CreationDate="2019-02-28T07:14:10.973" Score="2" Body="&lt;p&gt;Thanks Christopher Chang for the great answer via the plink2 google group! (See &lt;a href=&quot;https://groups.google.com/forum/#!topic/plink2-users/QGBJ_Xn62Uk&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;His answer was as follows:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Yes, plink 1.9 did change the default merged sample order.  However, you can request plink 1.07’s behavior by adding “--indiv-sort 0” during the merge.  (You can also use --indiv-sort + --make-bed at any time to change the sample order to whatever you want.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for why the default was changed, take a look at the new default order: it will probably make a lot of sense.  It’s “natural-sorted” which tends to make it easy for humans to manually search through regardless of what mix of letters and numbers are used in the IDs.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also the Plink 1.9 documentation for &lt;a href=&quot;https://www.cog-genomics.org/plink/1.9/data#indiv_sort&quot; rel=&quot;nofollow noreferrer&quot;&gt;--indiv-sort&lt;/a&gt; which makes the point that this behavior is default only for merges--in other commands Plink 1.9 uses the same ordering as in Plink 1.07, which is the order the samples were in.&lt;/p&gt;&#xA;" OwnerUserId="1574" LastEditorUserId="1574" LastEditDate="2019-03-04T18:37:04.523" LastActivityDate="2019-03-04T18:37:04.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7137" PostTypeId="1" AcceptedAnswerId="7138" CreationDate="2019-02-28T12:31:21.357" Score="2" ViewCount="288" Body="&lt;p&gt;I have analysed  and clustered my single cell rna seq data with methods in the Seurat package. After clustering, I obtained two main clusters: 0 and 1. I used the Seurat &lt;strong&gt;SplitDotPlotGG&lt;/strong&gt; function to visualise the differential expression of a gene of interest X. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SplitDotPlotGG(object,&#xA;               grouping.var = &quot;clusterID&quot;, &#xA;               genes.plot = c(&quot;geneX&quot;),&#xA;               cols.use = c(&quot;blue&quot;, &quot;red&quot;), plot.legend = T) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FYhqE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FYhqE.png&quot; alt=&quot;Expression of geneX&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plot shows clearly that there could be a significant differential expression of geneX between the conditions represented in clusters 0 and 1. However, I cannot see geneX in the list of differentially expressed genes returned when I run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FindMarkers(object, ident.1 = 0, ident.2 = 1, genes.use = &quot;geneX&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there be any explanation for this?&#xA;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="3968" LastEditorUserId="3968" LastEditDate="2019-02-28T15:20:27.213" LastActivityDate="2019-02-28T15:20:27.213" Title="Why does my SplitDotPlotGG plot show differential expression of a gene but Seurat::FindMarkers() function does not return it?" Tags="|seurat|single-cell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7138" PostTypeId="2" ParentId="7137" CreationDate="2019-02-28T14:41:55.663" Score="2" Body="&lt;p&gt;You're showing groups 1 and 2 in the plot but comparing groups 0 and 1 in the differential expression test, so you need to instead set &lt;code&gt;ident.1 = 1, ident.2 = 2&lt;/code&gt; in &lt;code&gt;FindMarkers&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;FindMarkers&lt;/code&gt; function by default only returns genes with &gt;0.25 log fold-change expression, with &gt;3 cells in each group. It's possible that these criteria are not being met for gene X. You can try changing these defaults, setting &lt;code&gt;logfc.threshold = 0&lt;/code&gt; to return all genes.&lt;/p&gt;&#xA;" OwnerUserId="3415" LastActivityDate="2019-02-28T14:41:55.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7139" PostTypeId="1" AcceptedAnswerId="7145" CreationDate="2019-02-28T16:18:38.140" Score="6" ViewCount="3200" Body="&lt;p&gt;We have a problem trying to demultiplex MinION sequences with custom barcodes. Do you have any software recommendations we can try for demultiplexing or how to demultiplex these custom barcodes with Albacore? We have tried using albacore but it only recognizes barcodes provided by Nanopore Technologies.&lt;/p&gt;&#xA;" OwnerUserId="4198" LastEditorUserId="4198" LastEditDate="2019-02-28T18:47:48.453" LastActivityDate="2023-10-23T22:11:23.780" Title="Demultiplex nanopore reads with custom barcodes" Tags="|nanopore|minion|albacore|demultiplexing|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7140" PostTypeId="2" ParentId="7046" CreationDate="2019-02-28T19:21:18.407" Score="2" Body="&lt;p&gt;Try running this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;source(&quot;https://bioconductor.org/biocLite.R&quot;)&#xA;biocLite(&quot;hdf5r&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Just a different method of installing packages besides github. It is more centred around bioinformatics packages as well so it is pretty useful.&lt;/p&gt;&#xA;" OwnerUserId="4001" LastActivityDate="2019-02-28T19:21:18.407" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7141" PostTypeId="2" ParentId="3212" CreationDate="2019-02-28T19:25:28.097" Score="2" Body="&lt;p&gt;It is certainly possible to encode polycistrons in GFF3. In fact, this is discussed explicitly in the &lt;a href=&quot;https://github.com/The-Sequence-Ontology/Specifications/blob/master/gff3.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;GFF3 specification&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;h3&gt;&lt;em&gt;Polycistronic transcripts&lt;/em&gt;&lt;/h3&gt;&#xA;  &#xA;  &lt;p&gt;This is the case in which a single (possibly spliced) transcript encodes multiple open reading frames that generate independent protein products.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;code&gt;-----&amp;gt;XXXXXXX*--&amp;gt;BBBBBB*---&amp;gt;ZZZZ*--&amp;gt;AAAAAA*-----&lt;/code&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Since the single transcript corresponds to multiple genes that can be identified by genetic analysis, the recommended solution here is to create four &quot;gene&quot; objects and make them the parent for a single transcript. The transcript will contain a single exon (in the unspliced case) and four separate CDSs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chrX  . gene XXXX YYYY  .  +  . ID=gene01;name=resA&#xA;chrX  . gene XXXX YYYY  .  +  . ID=gene02;name=resB&#xA;chrX  . gene XXXX YYYY  .  +  . ID=gene03;name=resX&#xA;chrX  . gene XXXX YYYY  .  +  . ID=gene04;name=resZ&#xA;chrX  . mRNA XXXX YYYY  .  +  . ID=tran01;Parent=gene01,gene02,gene03,gene04&#xA;chrX  . exon XXXX YYYY  .  +  . ID=exon00001;Parent=tran01&#xA;chrX  . CDS  XXXX YYYY  .  +  . Parent=tran01;Derives_from=gene01&#xA;chrX  . CDS  XXXX YYYY  .  +  . Parent=tran01;Derives_from=gene02&#xA;chrX  . CDS  XXXX YYYY  .  +  . Parent=tran01;Derives_from=gene03&#xA;chrX  . CDS  XXXX YYYY  .  +  . Parent=tran01;Derives_from=gene04&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;To disambiguate the relationship between which genes encode which CDSs, you may use the &lt;code&gt;Derives_from&lt;/code&gt; relationship.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This example doesn't indicate what the coordinates of each feature should be, but I think it makes most sense to have the &lt;code&gt;gene&lt;/code&gt; and &lt;code&gt;CDS&lt;/code&gt; feature coordinates correspond to the region coding each protein product, and the &lt;code&gt;mRNA&lt;/code&gt; and &lt;code&gt;exon&lt;/code&gt; feature coordinates correspond to the entire span.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;BIG CAVEAT&lt;/strong&gt;: Polycistrons are not typically considered a normal feature of eukaryotic genomes—they're much more common in bacteria and archaea. However, there are some confirmed cases of eukaryotic polycistronic mRNAs. That said, if overlapping gene predictions are observed in a GFF3 file, one cannot immediately assume that these are polycistronic. One or both of the gene predictions might be technical artifacts, or real genes that are incorrectly annotated, or (more rarely) actual overlapping genes. A claim that multiple protein products are expressed from a single transcript would require some convincing experimental validation.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-02-28T19:25:28.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7142" PostTypeId="1" CreationDate="2019-02-28T20:07:26.480" Score="3" ViewCount="384" Body="&lt;p&gt;I have two bulk RNA-Seq samples, already tpm-normalized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know what is a reasonable normalization procedure to enable downstream log fold-change estimation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distribution of the two samples using the common set of genes looks similar:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mcXNW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mcXNW.png&quot; alt=&quot;TPM distribution&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the two samples have only been tpm-normalized, is it enough to guarantee reliable fold-change estimation? Should I use another normalization procedure, e.g. &lt;a href=&quot;https://en.wikipedia.org/wiki/Quantile_normalization&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quantile Normalization&lt;/a&gt;, before comparison?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My objective is to define a signature using the genes that are up-regulated in Sample1 with respect to Sample0, and vice versa. I'm using log fold-changes, but I'm concerned that their value may be affected by each sample distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you also have suggestions for the definition of up-regulated genes with these data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aVoPa.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aVoPa.png&quot; alt=&quot;scatter&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2019-03-01T21:56:13.760" Title="Normalization for two bulk RNA-Seq samples to enable reliable fold-change estimation between genes" Tags="|rna-seq|normalization|fold-change|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7143" PostTypeId="2" ParentId="7142" CreationDate="2019-02-28T21:35:00.897" Score="3" Body="&lt;p&gt;It's not a good idea to do tpm normalisation prior to differential expression analysis, because the actual read counts are useful to determine shot noise and statistical significance. &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2&lt;/a&gt; includes read normalisation as part of its methods for differential expression analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think shot noise is best explained in terms of shooting photons at a target, as found on &lt;a href=&quot;https://en.wikipedia.org/wiki/Shot_noise#Origin&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Shot noise exists because phenomena such as light and electric current&#xA;  consist of the movement of discrete (also called &quot;quantized&quot;)&#xA;  'packets'. Consider light—a stream of discrete photons—coming out of a&#xA;  laser pointer and hitting a wall to create a visible spot. The&#xA;  fundamental physical processes that govern light emission are such&#xA;  that these photons are emitted from the laser at random times; but the&#xA;  many billions of photons needed to create a spot are so many that the&#xA;  brightness, the number of photons per unit of time, varies only&#xA;  infinitesimally with time. However, if the laser brightness is reduced&#xA;  until only a handful of photons hit the wall every second, the&#xA;  relative fluctuations in number of photons, i.e., brightness, will be&#xA;  significant, just as when tossing a coin a few times. These&#xA;  fluctuations are shot noise.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Sequencing also has shot noise effects because the sequencing process is a random sampling method. For low-abundance targets, the likelihood of sampling a target is low, so the effect of a single hit is amplified. This reduces the statistical significance when comparing different low-abundance expression values, because the variation in expression is highly dependent on which targets won the sampling lottery. The shot noise variation / expression curve can be seen in &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/gb-2010-11-10-r106#Fig7&quot; rel=&quot;nofollow noreferrer&quot;&gt;Figure 7 of this paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2019-03-01T21:56:13.760" LastActivityDate="2019-03-01T21:56:13.760" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7144" PostTypeId="2" ParentId="7142" CreationDate="2019-02-28T21:36:59.937" Score="3" Body="&lt;p&gt;What I have generally done in the past is to process the data using &lt;code&gt;voom&lt;/code&gt; in the &lt;code&gt;limma&lt;/code&gt; package for bulk RNASeq. Inside &lt;code&gt;voom&lt;/code&gt; you can call for different normalization methods to be used - &quot;TMM&quot; works fine for me and, is advocated by many in the field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;voom&lt;/code&gt; will output an object containing the normalized expression values in a log2 scale, which, also in my experience, has worked out just fine for calculating log fold changes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check out this link for more info on the package as well as normalization methods: &lt;a href=&quot;https://www.bioconductor.org/packages/devel/workflows/vignettes/RNAseq123/inst/doc/limmaWorkflow.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioconductor.org/packages/devel/workflows/vignettes/RNAseq123/inst/doc/limmaWorkflow.html&lt;/a&gt;&#xA;It is a very thorough introduction to the package and all of its capabilities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;" OwnerUserId="4001" LastActivityDate="2019-02-28T21:36:59.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7145" PostTypeId="2" ParentId="7139" CreationDate="2019-02-28T21:45:10.430" Score="5" Body="&lt;p&gt;I've written some &lt;a href=&quot;https://gitlab.com/gringer/bioinfscripts/-/blob/master/fastq-dental.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;Perl code&lt;/a&gt; for demultiplexing nanopore reads using &lt;a href=&quot;https://gitlab.com/mcfrith/last&quot; rel=&quot;nofollow noreferrer&quot;&gt;LAST&lt;/a&gt;. A workflow demonstrating the use of this script (and how it works under the hood) can be found here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://dx.doi.org/10.17504/protocols.io.14egnxw4zl5d&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://dx.doi.org/10.17504/protocols.io.14egnxw4zl5d&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You feed the script a barcode file and input fastq file, and it creates a trained mismatch matrix, aligns to the barcodes, then demultiplexes the reads into separate files for each barcode:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./fastq-dental.pl -barcode barcode_full_PBK004.fa -mat bc.mat reads_all.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Example output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;         9 BC01&#xA;        39 BC02&#xA;         1 BC02_BC07&#xA;        16 BC03&#xA;        38 BC04&#xA;         2 BC04_BC06&#xA;         1 BC04_BC07&#xA;         1 BC04_BC09&#xA;       643 BC05&#xA;         2 BC05_BC07&#xA;    493175 BC06&#xA;       115 BC06_BC06&#xA;         2 BC06_BC07&#xA;    663161 BC07&#xA;         1 BC07_BC05&#xA;       117 BC07_BC07&#xA;         1 BC07_BC09&#xA;       106 BC08&#xA;         1 BC08_BC06&#xA;        75 BC09&#xA;        72 BC10&#xA;         1 BC10_BC07&#xA;        41 BC11&#xA;        44 BC12&#xA;         1 BC12_BC06&#xA;     13535 BCchim&#xA;    237796 BCnoadapt&#xA;       152 BCnone&#xA;        90 RB12A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;While the approach designed around using the ONT barcodes, the barcode fasta file can be replaced by another file [using barcode ID in the header] without change the protocol.&lt;/p&gt;&#xA;&lt;p&gt;FWIW, there is also now some support for detecting strand direction (e.g. using the strand switch primer provided by ONT), but I haven't yet got it working properly doing both demultiplexing and strand orientation at the same time.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-10-23T22:11:23.780" LastActivityDate="2023-10-23T22:11:23.780" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7146" PostTypeId="2" ParentId="7142" CreationDate="2019-02-28T23:03:02.447" Score="5" Body="&lt;p&gt;You have only two samples?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You aren't going to be able to draw strong conclusions from that no matter what you do.  Clever statistics don't work without replicates.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-02-28T23:03:02.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7147" PostTypeId="1" AcceptedAnswerId="7150" CreationDate="2019-03-01T05:13:55.683" Score="2" ViewCount="370" Body="&lt;p&gt;I have no idea why I used the subsetdata in Seurat after using CCA the TSNEPlot shows the cluster all in the left side and merge， did I miss something? Here are the code and figure I got.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cluster &amp;lt;-SubsetData(object = immune.combined, ident.use= &quot;3&quot;)&#xA;&#xA;cluster &amp;lt;- NormalizeData(object = cluster, normalization.method = &quot;logNormalize&quot;, scale.factor = 10000)&#xA;&#xA;&#xA;cluster &amp;lt;- FindVariableGenes(object = cluster, mean.function = ExpMean, dispersion.function = LogVMR, x.low.cutoff = 0, x.high.cutoff = 6, y.cutoff = 0, do.plot = F)&#xA;&#xA;&#xA;&#xA;cluster &amp;lt;- ScaleData(object = cluster, display.progress = FALSE)&#xA;cluster &amp;lt;- RunPCA(object = cluster, pc.genes = cluster@var.genes, pcs.print = 1:10, genes.print =5)&#xA;&#xA;&#xA;&#xA;PCAPlot(object = cluster, group.by= &quot;orig.ident&quot;, dim.1 = 1, dim.2 = 2)&#xA;&#xA;cluster &amp;lt;- FindClusters (object = cluster, reduction.type = &quot;pca&quot;, dims.use = 1:10, resolution =0.3, print.output = 0, save.SNN = TRUE, force.recalc = T)&#xA;&#xA;TSNEPlot(object = cluster, do.label = TRUE,do.return = TRUE,pt.size = 2,label.size = 5, no.legend=FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fn9yz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fn9yz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3171" LastEditorUserId="48" LastEditDate="2019-03-01T08:06:10.993" LastActivityDate="2019-03-01T16:35:32.397" Title="Why are the clusters in the left side of a tSNE plot and not in the centre?" Tags="|seurat|single-cell|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7148" PostTypeId="2" ParentId="7147" CreationDate="2019-03-01T15:52:54.773" Score="1" Body="&lt;p&gt;I'm going to stick my neck out here. Overall PCA followed by tSNE is a very powerful method of unsupervised learning however, I agree with @GWW,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cluster &amp;lt;- ScaleData(object = cluster, display.progress = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If this is a max-min rescale transformation (common in supervised learning) then the outliers will badly affect the clustering. You can remove outliers that are over 2 sd from a formal analysis and re-run the calculation. That is just the risk of a max-min rescaling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The one issue I'm concerned about is the initial quasi-standardisation followed by log-transformation,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;log1p(value/colSums[cell-idx] *scale_factor)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this case you are multiplying by 10000, this is to remove the ratio (used as a quasi-standardisation) and then taking the log ??? That is not the way to transform a ratio, it would be better to take the reciprocal of the ratio. There are other (better) methods, a formal standardisation would be useful, before a transformation is considered. Possibly if that was done correctly you wouldn't have the problem of outliers potentially crunching your data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you fiddle with the parameters you might get something - but its a symptom not the cause of the problem. However, if removing the outliers works (I have not checked the ScaleData function), then you are sorted and all else is forgotten (at least for this data set). The information I read was &lt;a href=&quot;https://learn.gencore.bio.nyu.edu/single-cell-rnaseq/seurat-part-3-data-normalization/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-03-01T16:00:05.427" LastActivityDate="2019-03-01T16:00:05.427" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7149" PostTypeId="1" CreationDate="2019-03-01T16:29:24.573" Score="3" ViewCount="71" Body="&lt;p&gt;We have a test plate that was done with 10 of the 11plex TMT labels. So, the intensities of the 11th label should be almost zero, plus the leakage from the 10th label due to 13C content. Although we observe a substantial drop in the intensities, we see ratios up to 50 percent compared to the highest reference intensity, which is the maximum of the intensities of the other labels, if we sum up the intensities for all proteins. For individual proteins the intensity sometimes is higher than for all &amp;quot;real&amp;quot; samples. Only talking about proteins that are identified &amp;quot;Confidential&amp;quot;. Is that something you would expect? Is this due to the normalization?&lt;/p&gt;&#xA;&lt;p&gt;I was using Reporter 0.7.17 from command line without specifying a reference sample. Pretty much default configuration and based on the default protein report.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bd9d1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bd9d1.png&quot; alt=&quot;Ratios (intensities) reported by the program.&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Fig: Ratios (intensities) reported by the program.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2019-04-04T00:01:25.217" Title="Why does a missing label 11plex TMT shows up at almost 50% intensity compared to other labels?" Tags="|proteomics|tmt|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7150" PostTypeId="2" ParentId="7147" CreationDate="2019-03-01T16:35:32.397" Score="1" Body="&lt;p&gt;The issue here is simply that you did not re-compute the tSNE after subsetting the data, so you're looking at a subset of cells from a larger embedding. You would never get an embedding that looks like that by running tSNE on only those cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You also don't need to re-normalize the data after subsetting the cells. Since this is a per-cell calculation, the results will be identical to those in the larger matrix (you can check this for yourself).&lt;/p&gt;&#xA;" OwnerUserId="3415" LastActivityDate="2019-03-01T16:35:32.397" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7151" PostTypeId="1" CreationDate="2019-03-01T16:43:48.350" Score="1" ViewCount="211" Body="&lt;p&gt;I have a SNP dataset with 56 populations and 430 samples that was produced by GBS sequencing. I need to identify groups of samples that have similar patterns of genotype missingness, since there is substantial structure in missing genotypes, shown here in a PCoA ordination of genotype matrix with 1-0 values, where 0 represents missing gt and 1 is presence of a gt value: &#xA;&lt;a href=&quot;https://i.stack.imgur.com/0znDe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0znDe.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Here the missingess threshold for genotype inclusion is &amp;lt;20%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to cluster the samples using the scores on 20-30 PCoA axes, which is 17%-21% of variation, using the R package mclust. I ran &lt;code&gt;ape::PCoA()&lt;/code&gt; with an Euclidian distance matrix from &lt;code&gt;paralleldist::pardist()&lt;/code&gt;. When I run &lt;code&gt;mclust::mclustBIC()&lt;/code&gt;, I get a weird result in that the there is no modal value for any of the better-supported models, here shown in the legend: &#xA;&lt;a href=&quot;https://i.stack.imgur.com/gsJl4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gsJl4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had expected to find a model that would support the existence of a small number of clusters (2-6 maybe).  I also don't understand why the VVE model only has values to 3 clusters, where it presents the maximum value. Any ideas?  I have tried relaxing the minimum missing level, and using data with differing minimum locus depth levels.  I get very similar results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone please help me understand what is going on here and how I might address it?  Once I have clusters, I can look at the assignment of samples to clusters by population, flowcell, and other stratifying variables. I would be happy to edit to include more information, if that would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="2196" LastEditorUserId="48" LastEditDate="2019-03-08T10:42:59.010" LastActivityDate="2019-04-07T17:01:10.187" Title="GBS: clustering of PCoA axes with R package mclust to describe identity by missingness" Tags="|r|snp|clustering|radseq|genotyping|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7153" PostTypeId="1" AcceptedAnswerId="7155" CreationDate="2019-03-01T20:52:01.287" Score="4" ViewCount="1837" Body="&lt;p&gt;I am currently trying to understand the purpose of these PCHeatmaps - part of the &lt;code&gt;seurat&lt;/code&gt; package in R:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/A6NRi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/A6NRi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the online documentation I have searched for has only highlighted how it is used to observe heterogeneity in the data.. This much I understand. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Are the genes in the rows of each PC plot just the most variable x&#xA;number of genes defined in &lt;code&gt;FindVariableGenes&lt;/code&gt;?&lt;/li&gt;&#xA;&lt;li&gt;If not, then how are each of these genes 'chosen' for use in the &lt;code&gt;PCHeatmap&lt;/code&gt;&#xA;visualization?&lt;/li&gt;&#xA;&lt;li&gt;What would be the different conclusions/deductions drawn from the PC1 heatmap and PC2 heatmap with regards to heterogeneity for example?&lt;/li&gt;&#xA;&lt;li&gt;And finally, how can these conclusions help in the analysis of scRNASeq data?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="48" LastEditDate="2019-03-02T08:26:44.157" LastActivityDate="2019-03-02T08:26:44.157" Title="Understanding PCHeatmap outputs" Tags="|r|scrnaseq|statistics|seurat|pca|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7155" PostTypeId="2" ParentId="7153" CreationDate="2019-03-01T22:15:43.620" Score="3" Body="&lt;p&gt;The &lt;code&gt;PCHeatmap&lt;/code&gt; function (renamed &lt;code&gt;DimHeatmap&lt;/code&gt; in Seurat v3) can be used to help determine the number of principal components to use in downstream analysis, as well as to visualize the top genes contributing to each PC. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both cells and genes are ordered by their PC scores, and by default the 15 genes with the highest and 15 genes with the lowest PC loadings are displayed for each PC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case with PC1 and PC2, my conclusion would be that PC1 and PC2 both strongly separate different populations of cells. For PC1 there is a smaller population with very strong separation, and PC2 splits the cells roughly in half.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also set the &lt;code&gt;cells.use&lt;/code&gt; parameter (renamed &lt;code&gt;cells&lt;/code&gt; in Seurat v3) to only show the top &lt;em&gt;n&lt;/em&gt; cells for each PC which can help with visualization.&lt;/p&gt;&#xA;" OwnerUserId="3415" LastActivityDate="2019-03-01T22:15:43.620" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7156" PostTypeId="1" AcceptedAnswerId="7158" CreationDate="2019-03-02T17:28:59.233" Score="2" ViewCount="429" Body="&lt;p&gt;For Oxford Nanopore contigs produced by any long-read assembler has anyone performed any benchmarks to compare the polishing tools &lt;a href=&quot;https://github.com/isovic/racon&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;racon&lt;/code&gt;&lt;/a&gt;, &lt;a href=&quot;https://github.com/jts/nanopolish&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;nanopolish&lt;/code&gt;&lt;/a&gt;, or something like &lt;a href=&quot;https://github.com/ruanjue/wtdbg2&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;wtpoa-cns&lt;/code&gt;&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The compute time of &lt;code&gt;nanopolish&lt;/code&gt; is quite long and I would rather use another tool if it will suffice. After long-read polishing I will polish more with two rounds of &lt;a href=&quot;https://github.com/broadinstitute/pilon&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;pilon&lt;/code&gt;&lt;/a&gt; using 200x coverage Illumina shotgun data from the same individual.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2019-03-02T17:51:05.560" LastActivityDate="2019-03-03T14:44:19.827" Title="Is nanopolish worth it since faster polishing software is available?" Tags="|nanopore|assembly|long-reads|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7157" PostTypeId="1" CreationDate="2019-03-02T18:49:19.107" Score="2" ViewCount="62" Body="&lt;p&gt;I'm starting a thesis in Computer Science with topic the RFLCS (Repetition-free Longest Common Subsequence). I know very little about Bioinformatics problems and Genomics. I would like to understand the motivations behind this problem.&#xA;More precisely, I'd like to understand why one need a similarity measure between genomes, and how the RFLCS is an useful measure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[Adi et al.] introduce the problem in &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0166218X09001802&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4205" LastEditorUserId="4205" LastEditDate="2019-03-03T20:40:37.050" LastActivityDate="2019-03-03T20:40:37.050" Title="Repetition-free Longest Common Subsequence as similarity measure" Tags="|genomics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7158" PostTypeId="2" ParentId="7156" CreationDate="2019-03-03T14:44:19.827" Score="3" Body="&lt;p&gt;Nanopolish is necessary if you want to get high-quality consensus. racon etc don't use signal data. They can't achieve high quality. pilon at times doesn't work well. Even if it worked perfectly, it wouldn't help repetitive regions inaccessible by short reads.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2019-03-03T14:44:19.827" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7159" PostTypeId="2" ParentId="7157" CreationDate="2019-03-03T15:00:24.467" Score="1" Body="&lt;p&gt;A brief literature review can give you the answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1305.4883&quot; rel=&quot;nofollow noreferrer&quot;&gt;This preprint&lt;/a&gt; gives the definition of RFLCS more clearly:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A repetition free Longest Common Subsequence (LCS) of two sequences x and y is an LCS of x and y where each symbol may appear at most once.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0166218X09001802&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adi et al&lt;/a&gt; proposed to solve the &lt;a href=&quot;https://en.wikipedia.org/wiki/Chromosomal_rearrangement&quot; rel=&quot;nofollow noreferrer&quot;&gt;genome rearrangement&lt;/a&gt; problem raised in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/10743557&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;. If you want to know more, follow the references and citations of these papers and read them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, Adi et al think the length of RFLCS measures the distance between two genomes. The longer the more similar. I am not sure if this is a good measurement, though.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2019-03-03T15:00:24.467" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7160" PostTypeId="1" CreationDate="2019-03-03T19:55:55.293" Score="2" ViewCount="1095" Body="&lt;p&gt;I would like to convert 23andme text file to NextGen to BAM file for Yfull.com to read.  It is difficult to get answers on how to convert to SAM/BAM file for the 23andme text file to be converted to SAM/BAM tools&lt;/p&gt;&#xA;" OwnerUserId="4208" LastEditorUserId="96" LastEditDate="2019-03-26T18:41:00.173" LastActivityDate="2019-03-26T23:14:48.967" Title="I have 23andme text files and would like to convert to SAM/BAM format" Tags="|sam|file-formats|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="7161" PostTypeId="2" ParentId="7160" CreationDate="2019-03-04T08:19:20.857" Score="11" Body="&lt;p&gt;Having done 23andme myself I can tell you that your variant file, which contains SNP genotypes, cannot be converted to a bam file. It does not contain the same information as a bam file. It may be helpful to familiarize yourself with those filetypes and the technologies used to obtain them. A SAM/BAM file contains alignments of reads obtained by sequencing. Your standard* 23andme analysis doesn't have reads, as it uses a SNP array for determining the alleles at predefined positions. Markers which are not included in your 23andme file cannot be added by other means because those were not genotyped by the array. Therefore also Y-DNA cannot use your 23andme file or add additional markers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*For the sake of completeness: I am aware that under rare circumstances 23andme can perform WGS for additional variant typing, but I do not assume this is the case for OP.&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2019-03-04T08:19:20.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7162" PostTypeId="1" CreationDate="2019-03-04T10:55:59.653" Score="3" ViewCount="174" Body="&lt;p&gt;I have 6 RNA-seq samples like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;4 patients (005, 036, 121, 013)&#xA;&#xA;I have 3 tumour samples and 3 cancer models (organoid)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is PCA of log transformed data by DESeq2 (tumour Vs. organoid)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/n7fID.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/n7fID.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this paper &lt;a href=&quot;https://www.nature.com/articles/s41467-018-05190-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41467-018-05190-9&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They are saying &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Finally, for each patient, the genes that are specifically up- or&#xA;  down regulated in each patient’s organoid (1 vs. all comparison, p ≤&#xA;  0.05 and abs (log2-FoldChange) ≤ 1) and that are among the top-50 genes with highest base mean expression were selected.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;They have tumour and organoid like my case; My confusion is what they mean by &quot;&lt;em&gt;each patient’s organoid (1 vs. all comparison&lt;/em&gt;&quot;? Because like them I need up or down regulated genes in each patient's organoid but I can not figure out I should compare each organoid versus what?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ultimate goal would finding a signature of genes specific to each organoid as each organoid originates from tumour of a patient by these signatures one would be able to classify cancer to sub types &lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-03-04T11:21:52.683" LastActivityDate="2019-03-06T11:59:13.520" Title="Getting genes specially up or down regulated" Tags="|r|rna-seq|" AnswerCount="0" CommentCount="4" ClosedDate="2021-05-23T12:36:15.890" ContentLicense="CC BY-SA 4.0" />
  <row Id="7163" PostTypeId="1" CreationDate="2019-03-04T14:27:31.293" Score="0" ViewCount="329" Body="&lt;p&gt;I am analysing microarray ChIP data generated several years ago. I want to use the &lt;em&gt;affy&lt;/em&gt; package for R. However, in order to load the CEL files and proceed with the analysis, I need the pombe annotation file which comes under the name of &lt;strong&gt;Sp20b_M_v04.cdf&lt;/strong&gt;. I have looked everywhere for this file but I am unable to locate it. Is anyone in possession of this file or know where I can download it from?&lt;/p&gt;&#xA;" OwnerUserId="4209" LastEditorUserId="57" LastEditDate="2019-03-04T15:25:25.867" LastActivityDate="2019-03-05T14:07:45.980" Title="Where can I find .cdf files for Affymetrix ChIP-chip data analysis?" Tags="|r|microarray|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7165" PostTypeId="2" ParentId="3034" CreationDate="2019-03-04T16:33:02.677" Score="1" Body="&lt;p&gt;You simply need to transform your data.frame into a matrix. Just use these  two lines: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(SingleCellExperiment)&#xA;sce &amp;lt;- SingleCellExperiment(assays = list(counts = as.matrix(all.counts)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4210" LastActivityDate="2019-03-04T16:33:02.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7166" PostTypeId="1" CreationDate="2019-03-04T17:33:23.960" Score="3" ViewCount="1256" Body="&lt;p&gt;I am trying to install bedtools on windows 10, but I get an error I don't understand. How can I fix it?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Building BEDTools:&#xA;=========================================================&#xA;DETECTED_VERSION = v2.27.1&#xA;CURRENT_VERSION  =&#xA;Updating version file.&#xA; * Creating BamTools API&#xA;- Building in src/utils/FileRecordTools&#xA;  * compiling FileRecordMgr.cpp&#xA;make[1]: g++: Command not found&#xA;Makefile:36: recipe for target '../../../obj//FileRecordMgr.o' failed&#xA;make[1]: *** [../../../obj//FileRecordMgr.o] Error 127&#xA;Makefile:186: recipe for target 'src/utils/FileRecordTools' failed&#xA;make: *** [src/utils/FileRecordTools] Error 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4211" LastEditorUserId="57" LastEditDate="2019-03-05T08:37:55.270" LastActivityDate="2019-03-06T20:37:24.430" Title="Unable to install bedtools on windows 10 ubuntu" Tags="|genome|software-installation|bedtools|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7167" PostTypeId="2" ParentId="7166" CreationDate="2019-03-04T17:35:37.833" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;make&lt;a href=&quot;https://www.cs.odu.edu/~zeil/cs250PreTest/latest/Public/installingACompiler/&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;: g++: Command not found&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You do not have g++ installed on your machine.  This is used to compile the bedtools into an executable file.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.cs.odu.edu/~zeil/cs250PreTest/latest/Public/installingACompiler/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is a tutorial on installing g++&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2019-03-04T17:35:37.833" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7169" PostTypeId="1" CreationDate="2019-03-04T20:39:34.647" Score="3" ViewCount="3532" Body="&lt;p&gt;I'm using &lt;a href=&quot;https://github.com/roblanf/minion_qc&quot; rel=&quot;nofollow noreferrer&quot;&gt;MinIONQC&lt;/a&gt; to do quality control on some ONT data. The software plots several read characteristics over time (hours passed during the sequencing process). These plots contain several vertical red lines. From the documentation:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Muxes, which occur every 8 hours, are shown as red dashed lines&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hh0IF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hh0IF.png&quot; alt=&quot;Read quality by hour&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reading &lt;a href=&quot;https://dx.doi.org/10.12688/f1000research.7201.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper (doi:10.12688/f1000research.7201.1)&lt;/a&gt; I was able to get a rough idea of what a &quot;mux&quot; or a &quot;mux scan&quot; is. If I understand correctly, it's a calibration step that scans each well containing 4 nanopores to determine which nanopores will be used for data collection in which order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I understand the basic idea of the mux scan correctly? Does it occur at the same frequency on all ONT instruments? Does it always occur in 8hr intervals, even if the sequencing reaction is run for longer than the standard run time?&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-03-04T21:27:25.603" Title="Significance and timing of &quot;mux scans&quot;" Tags="|ngs|nanopore|quality-control|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7170" PostTypeId="2" ParentId="7169" CreationDate="2019-03-04T21:27:25.603" Score="6" Body="&lt;p&gt;Yes, your understanding is largely correct. This originates from the situation that for each detector on a nanopore array there are 4 pores. I'll explain mux scans and groups, but this is outdated information as now another system is used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So a MinION FC has 2048 pores with 512 sensors. Not all of these pores will be equally suitable for sequencing. At the beginning of the run originally a mux scan was performed, to test for each sensor which the best pore would be (if any) out of the 4 available. The best pores will be used for sequencing first, the rest will be turned off. That is group 1. After 8 hours the expectation is that many of these pores have degraded/accumulated damage or are not functional for other reasons. Then it is time to swith to group 2. So you change to another set of pores, which in these cases often lead to a temporary increase in yield. The default script used 8 hours, although some used adaptive software to select new pores when the first group started going bad. Also note that during the run no new mux scan was performed: the software did not check if the selection it made in the beginning still made sense. Pores in group 1 got turned off after 8 hours and switched to group 2, without taking their current 'health' into account. Cumulative yield plots get a characteristic bumpy pattern from this. Each bump is less than the previous one, eventually flatting off.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now that's history, as the software now uses dynamic muxing and will continuously check for the best pore per channel - and as such maximize yield by having as many healthy pores as possible available for sequencing. Cumulative yield plots are now smoother.&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2019-03-04T21:27:25.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7171" PostTypeId="2" ParentId="7149" CreationDate="2019-03-04T23:01:45.850" Score="1" Body="&lt;p&gt;The developers answered along these lines:&#xA;This might be a normalization artefact. Reporter is normalizing the ratios so that the median is 1. So, if the label is almost not present the noise gets increased. To check that you can turn of the normalization. This function is currently not documented. The developers are working on it though. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/compomics/reporter/issues/10&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub issue&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2019-03-04T23:01:45.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7172" PostTypeId="1" AcceptedAnswerId="7176" CreationDate="2019-03-05T02:16:55.307" Score="0" ViewCount="14" Body="&lt;p&gt;I am using MirDeep star tool for RNA sequence alignment. I have tried both GUI version and command line version of this tool. But every time, I am getting empty . result file and .known_mir file. This tool is neither giving any error nor any warning with my data. I have also tried the demo file given with this and this tool is working fine with this (i.e. giving some results in .result and known_mir file).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest me what could be the possible reason for this. Why I am getting empty file.&lt;/p&gt;&#xA;" OwnerUserId="2141" LastActivityDate="2019-03-05T10:51:14.257" Title="Empty .result in MirDeep Star" Tags="|rna-seq|sequence-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7173" PostTypeId="1" CreationDate="2019-03-05T04:49:30.827" Score="0" ViewCount="56" Body="&lt;p&gt;I have followed the Seurat T&lt;a href=&quot;https://satijalab.org/seurat/immune_alignment.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;utorial:Integrating stimulated vs. control PBMC datasets&lt;/a&gt; to learn cell-type specific responses by using my data. And I am confused about how to write the script to calculate the cell number of each sample on each cluster. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have run TSENPlot to know the samples and I'm using another TSENPlot to get the clusters. If I want to know find the cluster1 of each sample's number how can I write the script? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MbRGr.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MbRGr.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3171" LastEditorUserId="298" LastEditDate="2019-03-05T10:22:27.033" LastActivityDate="2019-03-05T10:25:58.553" Title="how to write the script to know each sample's cluster number?" Tags="|single-cell|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7174" PostTypeId="1" AcceptedAnswerId="7302" CreationDate="2019-03-05T09:09:05.050" Score="0" ViewCount="52" Body="&lt;p&gt;could you explain me how to interpret such situation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Same tag aligned in several distint places, near different genes...&#xA;Is the annotation able to distuinguish such 'same' tags, during DGE analysis?&#xA;Why tags arent unique?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ICk1R.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ICk1R.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/7ErRX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7ErRX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2869" LastActivityDate="2019-03-21T08:58:39.727" Title="Affymetrix tags (same ID's) present in different places of the genome" Tags="|microarray|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7175" PostTypeId="2" ParentId="7173" CreationDate="2019-03-05T10:08:46.403" Score="2" Body="&lt;p&gt;Assuming you are following the basic seurat guide:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pbmc@ident&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can access the &lt;code&gt;'ident'&lt;/code&gt; slot of your Seurat object and you will get a vector of the cluster IDs and the names will be the respective cells.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;head(seuratobj@ident)&#xA;Cell1A7 Cell1C4 Cell1D1 Cell1D2 Cell1D3 Cell1D5&#xA;0 3 3 1 3 1 &#xA;Levels: 0 1 2 3 4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="383" LastEditorUserId="48" LastEditDate="2019-03-05T10:25:58.553" LastActivityDate="2019-03-05T10:25:58.553" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7176" PostTypeId="2" ParentId="7172" CreationDate="2019-03-05T10:51:14.257" Score="0" Body="&lt;p&gt;I have found reason for not getting any result. I was using default adapter given by mirdeep* which gives me zero alignment with my data. This error pointed out by author of mirdeep* ( thanks to Jiyuan and colleen). After giving right adapter, I got the right result.&lt;/p&gt;&#xA;" OwnerUserId="2141" LastActivityDate="2019-03-05T10:51:14.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7177" PostTypeId="2" ParentId="7166" CreationDate="2019-03-05T10:52:34.233" Score="3" Body="&lt;p&gt;My personal advice would be to avoid compiling tools like this if you can avoid it, especially where you are using a &quot;barebones&quot; setup like the one you get with WSL Ubunutu as you'll almost always run into dependency issues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The linux version of &lt;code&gt;conda&lt;/code&gt; installs just fine in WSL and &lt;code&gt;bioconda&lt;/code&gt; hae pretty much any bioinformatics package you might need, already compiled.&lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2019-03-05T10:52:34.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7179" PostTypeId="2" ParentId="7163" CreationDate="2019-03-05T13:38:31.653" Score="1" Body="&lt;p&gt;It looks like it's in GEO here: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GPL10187&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GPL10187&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is apparently a zipped copy linked at the bottom in the supplementary files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When people upload a study based on an array, there is a corresponding 'platform' record where the array details should be deposited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: you will probably need to gunzip the file first, to get it into CDF format.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastEditorUserId="3828" LastEditDate="2019-03-05T14:07:45.980" LastActivityDate="2019-03-05T14:07:45.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7180" PostTypeId="1" AcceptedAnswerId="7185" CreationDate="2019-03-05T14:48:58.893" Score="3" ViewCount="916" Body="&lt;p&gt;I am new to using bioinformatic tools and I was hoping this community could help clear something up for me. I need to generate a gtf file. My data are a set of complete HA genes for influenza B viruses in fasta format. From reading through forums and other internet searches, my understanding is that fasta cannot be transformed to gtf or gff because these last two provide location information which fasta does not (unless available in the header). I saw someone suggest mapping sequences to a reference sequence and generating the gtf from that mapping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, would it make sense to use the consensus sequence selected for alignment as this reference sequence? The alignment algorithm is a simple majority rule. Because all my data are complete genes, I would think the mapping of locations (start codon and stop codon) would be the same no matter which sequence in my data set I choose to use. Is this not so?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking at viral evolution in the HA segment and my end goal is to calculate the dS/dN ratio for a set influenza B viruses collected during a particular season. The program I want to use requires a gtf, but likely other options would only require the reference genome. Would it make more sense to use the strain used as the vaccine candidate as the reference genome?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I appreciate any help you can provide me!&lt;/p&gt;&#xA;" OwnerUserId="4214" LastEditorUserId="4214" LastEditDate="2019-03-05T18:01:20.620" LastActivityDate="2019-03-05T21:02:13.850" Title="GTF from consensus sequence" Tags="|fasta|gtf|gff|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="7181" PostTypeId="1" CreationDate="2019-03-05T16:24:29.800" Score="1" ViewCount="25" Body="&lt;p&gt;MSstatsTMT is an R library for statistical analysis of TMT labeled mass-spectrometry (MS) proteomics data. The &lt;a href=&quot;http://msstats.org/wp-content/uploads/2018/12/MSstatsTMT.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; does not explicitly mention the kind of MS that can be processed. The examples contain comments that contain both MS2 and MS3, but I would like to make sure that is indeed the case. Can anyone confirm that and maybe provide an example script?&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2019-03-05T16:24:29.800" Title="Can MSstatsTMT be used for both MS2 and MS3 runs?" Tags="|proteomics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7182" PostTypeId="1" CreationDate="2019-03-05T17:01:50.187" Score="1" ViewCount="221" Body="&lt;p&gt;The objective of the following analysis is to identify an activation signature of a specific phenotype on bulk RNASeq and to apply it to single-cell RNA-Seq, in order to identify the population of activated cells in scRNA-Seq samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the first step I used DESeq2 to identify genes that increase during different timepoints (I have 4 timepoints):&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Bulk samples are normalized using VST (Variance Stabilizing Transformation - DESeq2)&lt;/li&gt;&#xA;&lt;li&gt;For each timepoint, differentially expressed genes are computed with respect to the previous timepoint (i.e. 2 vs 1, 3 vs 2, 4 vs 3)&lt;/li&gt;&#xA;&lt;li&gt;Differentially expressed genes that increase with timepoint (log-FC &gt; 1, padj &amp;lt; .05) are selected (n=24)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;As a control, I plotted the activation score (&lt;a href=&quot;https://icb-scanpy.readthedocs-hosted.com/en/latest/api/scanpy.api.tl.score_genes.html?highlight=score&quot; rel=&quot;nofollow noreferrer&quot;&gt;computed with scanpy&lt;/a&gt;) based on the 24-gene signature applied to the bulk samples from which it has been extracted. As you can see, the score always increases from the first timepoint (blue) to the last (red).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RZjMJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RZjMJ.png&quot; alt=&quot;activation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I use the same scoring procedure on a single-cell RNASeq sample containing the same populations, I think something may be wrong.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the following the violin plot of the extracted activation score on an unsupervised clustering on single-cell data. Excluding cluster 4, which we identified as contamination, while cluster 1 and cluster 0 are supposed to have the correct degree of activation, clusters 2 and 3 have a lower degree of activation with respect to cluster 0 (which we identified as the first timepoint - lowest activation on bulk).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4scS2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4scS2.png&quot; alt=&quot;activation-sc&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What may be the causes of this behavior? Right now the possible explanation I came out with is that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Clusters 2 and 3 have &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/3231/1771&quot;&gt;global shifts on gene expression&lt;/a&gt; that make TPM normalization over-normalize activation genes&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Are there other possible explanations for this behavior, and how to check for them?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2019-03-05T19:13:20.363" LastActivityDate="2019-03-05T20:25:46.347" Title="Discordance in gene signature behavior between bulk and single-cell RNASeq" Tags="|rna-seq|scrnaseq|normalization|tpm|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7183" PostTypeId="2" ParentId="7180" CreationDate="2019-03-05T17:31:53.860" Score="3" Body="&lt;p&gt;&quot;Transforming&quot; Fasta files to GTF or GFF3 files is a common request, as are similar tasks such as &quot;converting&quot; BAM files to VCF files. But these are underdefined questions, and &quot;transform&quot; or &quot;convert&quot; isn't really the right way to think about the task.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as your particular request is concerned, Fasta files and GTF files contain fundamentally different types data.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fasta files contain sequences, which could be chromosomes, assembled contigs, polypeptides, transcripts, or other biological sequences.&lt;/li&gt;&#xA;&lt;li&gt;GTF (and GFF3 and BED) files contain &lt;em&gt;intervals&lt;/em&gt;, indicating that some feature of interest occurs at such-and-such position on such-and-such sequence. A hypothetical example: a transcription factor binding site occurs on chromosome 2 from position 40011 to 40017.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So you are correct: it's not really helpful to think of your task in terms of &quot;transforming&quot; Fasta to GTF. The process of creating a GTF/GFF3/BED file is often called &lt;em&gt;annotation&lt;/em&gt;. There are many different methods to annotate many different types of genomic features: transposable elements, segmental duplications, protein-coding genes, non-coding RNAs, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a result, it's meaningless to say &quot;I need a GTF file&quot; without clearly specifying what that GTF file will annotate. (It's kind of like saying &quot;I need an XML file&quot; or &quot;I need an Excel spreadsheet&quot; without indicating what the contents of those files would be.) The things you need to clarify are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What will the intervals in my GTF file represent?&lt;/li&gt;&#xA;&lt;li&gt;What precise sequences will serve as the reference and provide interval coordinates for the GTF file?&lt;/li&gt;&#xA;&lt;li&gt;How are those intervals to be determined?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It sounds like 3. is a big part of your question. If I'm reading things correctly, you want to annotate a particular class of genes on an influenza genome. Is that correct? Did you perform &lt;em&gt;de novo&lt;/em&gt; sequencing of influenza viruses, and now hope to characterize variation in their HA genes? Stating what you hope to get out of your analysis would be helpful in finding a good way forward.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2019-03-05T17:37:55.610" LastActivityDate="2019-03-05T17:37:55.610" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7184" PostTypeId="1" AcceptedAnswerId="7190" CreationDate="2019-03-05T17:55:10.793" Score="3" ViewCount="3088" Body="&lt;p&gt;Consider the following scenario.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;├── sample-alice&#xA;│   ├── sequence_1.fastq&#xA;│   ├── sequence_2.fastq&#xA;│   ├── ...&#xA;│   └── sequence_n.fastq&#xA;└── sample-bob&#xA;    ├── sequence_1.fastq&#xA;    ├── sequence_2.fastq&#xA;    ├── ...&#xA;    └── sequence_m.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm trying to write a single rule in my Snakefile that will preprocess each sample generically. The rule will execute a single command that takes all of Fastq files associated with the sample simultaneously. The samples do not have the same number of Fastq files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it was a single sample, I could do something like this.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;rule preprocess:&#xA;    input: glob('sample-alice/*.fastq')&#xA;    output: 'alice-clean.fastq'&#xA;    shell: 'mypreproccmd {input} &amp;gt; {output}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;On the other hand, if there was a single Fastq file for each sample, I could use a wildcard to write a generic rule like this.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;rule preprocess:&#xA;    input: 'sample-{samp}/sequence_1.fastq'&#xA;    output: '{samp}-clean.fastq'&#xA;    shell: 'mypreproccmd {input} &amp;gt; {output}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is it possible to combine these two approaches to write a rule for handling multi-file samples generically?&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2023-12-07T20:21:35.277" Title="Mix globbing and wildcards when specifying rule input" Tags="|snakemake|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7185" PostTypeId="2" ParentId="7180" CreationDate="2019-03-05T18:38:25.643" Score="1" Body="&lt;p&gt;I was wanting more time, but okay here's what I'm thinking,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Would it make more sense to use the strain used as the vaccine&#xA;  candidate as the reference genome?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Neither, because the GTF format is based on an HA protein structure (likely a crystal structure but could be cryoEM). This is not the same as an alignment position within a surface antigen, because most alignments of pathogen surface antigen genes will contain indels, e.g. between the GTF and your influenza epidemic. If HA doesn't have any indels fine, but I would be surprised. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The indels will cause artifactual shifts in the GTF domain boundaries unless they are corrected. Put simply an alignment position (particularly for pathogen surface antigen proteins) is rarely the same as the amino acid position of GFT because a protein structure of a surface antigen doesn't comprise indels in its native form. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you do this work in detail, i.e. shifting between protein structure and phylogeny, you always write scripts which convert between aligned positions and &quot;reference&quot; (crystal/ cryoEM structure) positions. Its a common way of thinking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in summary, your reference sequence is the GTF sequence because that is the basis of the structure&amp;lt;-&gt; alignment conversion  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;For positive selection analysis it does not matter what the reference is, &lt;strong&gt;all that matters is the correct rooting&lt;/strong&gt;. For demarcation of domains within HA the GFT sequence needs to be at the top of the alignment 1) because it helps easily identify whether it contains indels 2) structure programs will expect the reference sequence to be there. If you are using the vaccine in a positive selection analysis ... you need to think hard about the question you are asking and the coverage of the vaccine. Epidemiologically speaking thats complicated because the passage of the vaccine will be an important source of selection and you will mistake that as the wild-type &quot;escaping&quot; the vaccine.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-03-05T21:02:13.850" LastActivityDate="2019-03-05T21:02:13.850" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7186" PostTypeId="2" ParentId="7180" CreationDate="2019-03-05T18:38:41.873" Score="1" Body="&lt;p&gt;Since you know that your sequences in your fastas are your genes of interest, why not just dummy up a gtf where every sequence is a gene?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point of a gtf is to tell software which genomic regions matter for a certain procedure, and which regions do not.  But if you already know that your sequences are pared down to genes, then you know that the whole thing is what's relevant.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-03-05T18:38:41.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7187" PostTypeId="1" CreationDate="2019-03-05T20:08:58.030" Score="0" ViewCount="37" Body="&lt;p&gt;I am running an analysis and I want to edit the output that I received from that analysis. However I cannot define the output name in my pipeline, because after running the analysis it is asking me to name the output file. How can I include that output file in my pipeline ?&lt;/p&gt;&#xA;" OwnerUserId="4216" LastEditorUserId="4216" LastEditDate="2019-03-05T20:17:30.327" LastActivityDate="2019-03-05T20:19:08.713" Title="How to use analysis output in my pipeline" Tags="|database|bash|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7188" PostTypeId="2" ParentId="7187" CreationDate="2019-03-05T20:19:08.713" Score="1" Body="&lt;p&gt;I assume you are referring to the &lt;a href=&quot;http://emboss.sourceforge.net/apps/cvs/emboss/apps/pepstats.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;pepstats program from EMBOSS&lt;/a&gt;. The program has a &lt;code&gt;-outfile&lt;/code&gt; flag that allows you to specify your desired name for the output file when you're not running the program interactively.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-03-05T20:19:08.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7189" PostTypeId="2" ParentId="7182" CreationDate="2019-03-05T20:25:46.347" Score="3" Body="&lt;p&gt;One explanation could be that your mapping of clusters to timepoints is not accurate. There are other methods you could look at for doing this, for example scMap, scPred, or Seurat v3 (disclosure: I am one of the Seurat developers).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility is that there is some problem in the calculation of the activation signature. You could instead look at the expression of the 24 genes themselves in each cluster (for example by drawing a heatmap).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference between bulk and single-cell here could also be biological. This could happen if the increase in expression of those genes in bulk was due to a change in cell type frequency over time rather than increased expression.&lt;/p&gt;&#xA;" OwnerUserId="3415" LastActivityDate="2019-03-05T20:25:46.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7190" PostTypeId="2" ParentId="7184" CreationDate="2019-03-05T20:38:54.180" Score="5" Body="&lt;p&gt;A nice solution comes from a colleague of mine, as seen in &lt;a href=&quot;https://github.com/AlexanderLabWHOI/tara-euk-metaG/blob/ef0f140b7d79540da22da48d4b8e939b95e21f0b/Snakefile#L150-L172&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Snakemake workflow&lt;/a&gt;. The trick is to access the wildcards programmatically using an anonymous (lambda) function. In my example above, it would be implemented as follows.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from glob import glob&#xA;&#xA;rule preprocess:&#xA;    input: lambda wildcards: glob('sample-{samp}/*.fastq'.format(samp=wildcards.samp))&#xA;    output: '{samp}-clean.fastq'&#xA;    shell: 'mypreproccmd {input} &amp;gt; {output}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2023-12-07T20:21:35.277" LastActivityDate="2023-12-07T20:21:35.277" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7191" PostTypeId="1" AcceptedAnswerId="7193" CreationDate="2019-03-05T20:53:45.013" Score="2" ViewCount="223" Body="&lt;p&gt;I'm using the &lt;code&gt;coverage&lt;/code&gt; function of &lt;code&gt;bedtools&lt;/code&gt; to check whether a certain set of regions (file B) has any overlap with known regions of epigenetic markers (file A). When that is the case, I want to access the name of that marker, which is given as the third column (name) in file A. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I run &lt;code&gt;bedtools coverage -a fileA.bed -b fileB.bed&lt;/code&gt;, I only get the columns of file B as an output, together with the overlap fraction. Is there a built-in way of accessing also the columns of the file A to which I aligned, or an easy way of cross-referencing to it?&lt;/p&gt;&#xA;" OwnerUserId="4145" LastActivityDate="2019-03-07T18:54:32.140" Title="bedtools: get name column of alignment file A" Tags="|read-mapping|bedtools|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7192" PostTypeId="1" CreationDate="2019-03-06T01:46:11.593" Score="0" ViewCount="91" Body="&lt;p&gt;I am a PhD student who inherited some log2cpm data of expression data from bulk kidney tissue from a UUO(unilateral urethral obstruction) experiment that tests a new drug. The sample material consists of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;6 x Ligated Kidney (Untreated&lt;/li&gt;&#xA;&lt;li&gt;6 x Ligated Kideny (Treated)&lt;/li&gt;&#xA;&lt;li&gt;3 x Unligated Kidney (Untreated)&lt;/li&gt;&#xA;&lt;li&gt;3 x Unligated Kidney (Treated)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The previous study showed that this drug was effective against fribrosis, so the aim of my study is to investigate how this drug affects inflammation and mitochondrial function in the diseased kidney.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I have filtered my original dataset based on MitoCarta v2 DB, so investigate how the drug affected mitochondrial function. I afterwards run a PCA analysis on log2cpm data which revealed the following:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/mUE34.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mUE34.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following figure shows PC1(84.51%) and PC2(4.69%).&#xA;Red (Ligated, Untreated), purple (Ligated, Treated), green(healthy kidney, not treated), orange (Healthy kidney, treated).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I interpret this as there is a very small difference between treated and not treated ligated kidneys, however there is a small difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have just started doing bioinformatics and I am very unsure which approach I should choose further, however I have though maybe do a highly variable genes analysis (HVG) &lt;a href=&quot;https://academic.oup.com/bib/advance-article/doi/10.1093/bib/bby011/4898116&quot; rel=&quot;nofollow noreferrer&quot;&gt;see link&lt;/a&gt;, isolate the 500 most variable genes and run a heatmap with hieraichal clustering. I was also thinking of looking at the loadings in PC2, and take this further. The truth is that I am very unsure what is the right approach, and my research group does not have bioinformatician I can ask anymore.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What kind of analysis you guys would suggest I use to investigate&#xA;further (Hieraichal clustering, HVG etc?)&lt;/li&gt;&#xA;&lt;li&gt;Is there any alternative preprocessing I could use before I run the PCA, to make the analysis more accurate?&lt;/li&gt;&#xA;&lt;li&gt;Are there any packages that you can suggest?&lt;/li&gt;&#xA;&lt;li&gt;Any other tips?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Any other tips?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I asked this same question on Biostars as well&lt;/strong&gt;&#xA;&lt;a href=&quot;https://www.biostars.org/p/367722/#367730&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/367722/#367730&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4217" LastEditorUserId="4217" LastEditDate="2019-03-06T10:11:26.897" LastActivityDate="2019-04-05T14:03:05.620" Title="I need some tips and suggestions for further analysis of NGS expression data (log2cpm)" Tags="|rna-seq|differential-expression|transcriptome|rna|gene-expression|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7193" PostTypeId="2" ParentId="7191" CreationDate="2019-03-06T03:32:19.697" Score="2" Body="&lt;p&gt;I would recommend using &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/intersect.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools intersect&lt;/a&gt; for that type of comparison.  This will allow you to see the specific segments that overlap.  bedtools coverage is used more for determining how many things cover a region but not which ones.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The documentation linked above will provide a much better desciption of intersect than I would be able to in this post but I would recommend using the &lt;code&gt;-loj&lt;/code&gt; or &lt;code&gt;-u&lt;/code&gt; depending on what you want.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-loj    Perform a “left outer join”. That is, for each feature in A report each overlap with B. If no overlaps are found, report a NULL feature for B.&#xA;&#xA;-u  Write original A entry once if any overlaps found in B. In other words, just report the fact at least one overlap was found in B. Restricted by -f and -r.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="964" LastActivityDate="2019-03-06T03:32:19.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7194" PostTypeId="2" ParentId="7191" CreationDate="2019-03-06T07:26:23.707" Score="2" Body="&lt;p&gt;If you're working with &lt;code&gt;fileB.bed&lt;/code&gt; (a &lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/file-management/sorting/sort-bed.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sorted&lt;/a&gt; BED file) and &lt;code&gt;fileA.txt&lt;/code&gt; (a three-column text file, where the name is in the third column), then you could pipe single-base pseudo-BED elements to &lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/statistics/bedmap.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;bedmap&lt;/code&gt;&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ bedmap --echo --echo-map-id --delim '\t' fileB.bed &amp;lt;(awk -vFS=&quot;\t&quot; -vOFS=&quot;\t&quot; '{ print $1, $2, ($2+1), $&lt;/span&gt;3 }' fileA.txt) &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;--echo-map-id&lt;/code&gt; option reports the list of marker IDs from &lt;code&gt;fileA.txt&lt;/code&gt;, where the single-base marker interval from &lt;code&gt;fileA.txt&lt;/code&gt; overlaps a region in &lt;code&gt;fileB.bed&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If all the marker IDs are not unique (if there are duplicate marker IDs) it may make sense to instead use &lt;code&gt;--echo-map-id-uniq&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This assumes that marker positions in the second column and that the intervals in &lt;code&gt;fileB.bed&lt;/code&gt; are using a 0-based index scheme. If marker positions are 1-based and &lt;code&gt;fileB.bed&lt;/code&gt; is 0-based, then a small adjustment is needed to make both inputs use the same indexing, to calculate correct overlaps:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ bedmap --echo --echo-map-id --delim '\t' fileB.bed &amp;lt;(awk -vFS=&quot;\t&quot; -vOFS=&quot;\t&quot; '{ print $1, ($2-1), $2, $&lt;/span&gt;3 }' fileA.txt) &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I mention this, as SNVs (such as those from VCF) can often be reported with a 1-based index scheme, which adds some nuance to doing set operations against BED files.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2019-03-07T18:54:32.140" LastActivityDate="2019-03-07T18:54:32.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7195" PostTypeId="1" CreationDate="2019-03-06T07:30:22.703" Score="4" ViewCount="881" Body="&lt;p&gt;I installed Cytoscape 3.7.1 with java 1.8.0_191 in windows server 2012. I have a 7.4 GB &lt;code&gt;csv&lt;/code&gt; file (about 1,500,000 reords) and when I tried to load it into the Cytoscape it throws an error &lt;code&gt;java.lang.OutOfMemoryErroe:Java heap space.&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My system has 500GB ram and then I changed &lt;code&gt;Cytoscape.vmoptions&lt;/code&gt; file according to &lt;code&gt;-Xms102400M&lt;/code&gt; but when I load my file, in task manager, the memory increased until 50GB and then Cytoscape crashed and again it threw the same error: &lt;code&gt;java.lang.OutOfMemoryErroe:Java heap space.&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My data is like this sample:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;45677888,test1,3453453453,test2,3534523453,235412352,3453452345,test3,235423452,test4,1980/09/02,14:13,23523525,test5&#xA;45234288,test11,34234553453,test12,353434534553,23453452352,342422345,test13,23456543452,test14,1980/04/01,14:12,2323423425,test15&#xA;4243424888,test12,3235253453,test22,3533456343453,233534532352,2343452345,test33,23345342,test44,1980/03/4,14:11,23674575,test55&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My file is in CSV format. col2 is source node, col1 and col3 are attributes for source node,col 4 is destination node, col5,col6,col7 are attributes for destination node, col8 until end are attributes for edge. I have about 200,000 nodes and 230000 edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I read this file in Cytoscape?&lt;/p&gt;&#xA;" OwnerUserId="4219" LastEditorUserId="298" LastEditDate="2019-03-06T11:28:47.293" LastActivityDate="2020-10-20T20:14:45.613" Title="Error while loading a big file with Cytoscape" Tags="|networks|" AnswerCount="3" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="7196" PostTypeId="1" AcceptedAnswerId="7207" CreationDate="2019-03-06T10:20:36.757" Score="1" ViewCount="470" Body="&lt;p&gt;I am trying to use &lt;a href=&quot;https://www.bioinf.jku.at/software/cnmops/cnmops.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;cn.MOPS&lt;/a&gt; to call CNVs on a set of whole exome sequencing data bam files. My script (up to the point where it fails) is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;source(&quot;https://bioconductor.org/biocLite.R&quot;)&#xA;library(cn.mops)&#xA;BAMFiles &amp;lt;- c('sample1.bam', 'sample2.bam', 'sample3.bam', 'sample4.bam', 'sample5.bam', 'sample6.bam', 'sample7.bam')&#xA;segments &amp;lt;- read.table(&quot;region.bed&quot;,sep=&quot;\t&quot;,as.is=TRUE)&#xA;gr &amp;lt;- GRanges(segments[,1],IRanges(segments[,2],segments[,3]))&#xA;X &amp;lt;- getSegmentReadCountsFromBAM(BAMFiles,GR=gr,parallel=11)&#xA;resCNMOPS &amp;lt;- exomecn.mops(X)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The last command, &lt;code&gt;resCNMOPS &amp;lt;- exomecn.mops(X)&lt;/code&gt; fails with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in if (all(segMedianT == 0)) { : &#xA;  missing value where TRUE/FALSE needed&#xA;In addition: Warning message:&#xA;In normalizeChromosomes(X, chr = chr, normType = normType, qu = normQu,  :&#xA;  Normalization for reference sequence  chrUn_gl000228 not applicable, because of low number of segments&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I run &lt;code&gt;traceback&lt;/code&gt; immediately afterwards, I get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; traceback()&#xA;4: FUN(newX[, i], ...)&#xA;3: apply(sINI[chrIdx, , drop = FALSE], 2, segment, minSeg = minWidth, &#xA;       ...)&#xA;2: cn.mops(input = input, I = I, classes = classes, priorImpact = priorImpact, &#xA;       cyc = cyc, parallel = parallel, normType = normType, normQu = normQu, &#xA;       norm = norm, sizeFactor = sizeFactor, quSizeFactor = quSizeFactor, &#xA;       lowerThreshold = lowerThreshold, upperThreshold = upperThreshold, &#xA;       minWidth = minWidth, segAlgorithm = segAlgorithm, minReadCount = minReadCount, &#xA;       useMedian = useMedian, returnPosterior = returnPosterior, &#xA;       ...)&#xA;1: exomecn.mops(X)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am using R version &lt;code&gt;3.4.4&lt;/code&gt; on an Ubuntu 16.04.5 LTS machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My bed file only has one region which is on &lt;code&gt;chrUn_gl000228&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep chrUn_gl000228 region.bed &#xA;chrUn_gl000228  112604  113879&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I remove that region and rerun the same code, it works as expected. So, my question is, how can I detect this and fix it automatically? Given a collection of bam files and the bed file describing the capture regions used to sequence the samples and generate the bam files, how can I check if cn.MOPS will complain about any of the segments?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presumably, once I have built these two objects:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;segments &amp;lt;- read.table(&quot;region.bed&quot;,sep=&quot;\t&quot;,as.is=TRUE)&#xA;gr &amp;lt;- GRanges(segments[,1],IRanges(segments[,2],segments[,3]))&#xA;X &amp;lt;- getSegmentReadCountsFromBAM(BAMFiles,GR=gr,parallel=11)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I should be able to then check them and see if they have the right data and, if not, remove the offending segment from the &lt;code&gt;GRanges&lt;/code&gt; object. But how can I do this? &lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-03-06T11:46:14.180" LastActivityDate="2019-05-31T08:35:59.723" Title="cn.MOPS fails with 'missing value where TRUE/FALSE needed'" Tags="|r|bioconductor|cnv|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="7197" PostTypeId="1" CreationDate="2019-03-06T10:29:16.433" Score="8" ViewCount="589" Body="&lt;p&gt;Hi there :) Thanks for the tool!&lt;/p&gt;&#xA;&lt;p&gt;I recently updated to the new salmon (from 0.8... its been a couple years) and I noticed that my mapping percentages change dramatically between the two versions.&lt;/p&gt;&#xA;&lt;p&gt;For example, using the default settings in v0.8, I see a mapping rate of 96.25% however upon upgrading to v0.13, these mapping rates go down to 82-84% depending on which of the parameters I play with (I've permutated  the validateMappings, incompatPrior, consensusSlack, minScoreFraction, ma, etc). Below I have a snippet of one of these tests.  I'm currently assembling the unmapped reads to see if I can determine why these reads were not mapped.&lt;/p&gt;&#xA;&lt;p&gt;What do you think is the most likely cause for this dramatic change in performance?&lt;/p&gt;&#xA;&lt;p&gt;Perhaps it could be the transcript index? (I used a kmer of 31 and my reads are 100 bp).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have run the indexing and quasi-mapping  using salmon v0.8, v0.10 (when validateMappings was released), v0.12, v0.13 and had mapping rates of 96, 90, 90 and 83 respectively.  So it looks like a gradual decay of mapping rates.  Or alternatively, a gradual reduction in spurious mapping?&lt;/p&gt;&#xA;&lt;p&gt;Seems like what ever improvements from 0.12 to 0.13 changed the mapping rates significantly.  I will go ahead with both mappings (v0.12 and v0.13) to see if there are differences in the DESEQ2 outputs.&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot!&#xA;(while I have presented values here for one set of fastq files, the same pattern is seen throughout all 72 of my samples from different experiments)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for F in &lt;span class=&quot;math-container&quot;&gt;$FILES ; do&#xA;  R1=$&lt;/span&gt;{F}_L001_R1_001.fastq.gz&#xA;  R2=&lt;span class=&quot;math-container&quot;&gt;${F}_L001_R2_001.fastq.gz&#xA;  salmon quant \&#xA;      -p 35 \&#xA;      -i $&lt;/span&gt;ref_dir/&lt;span class=&quot;math-container&quot;&gt;$transcript_index \&#xA;      -l A \&#xA;      -1 $&lt;/span&gt;R1 \&#xA;      -2 &lt;span class=&quot;math-container&quot;&gt;$R2 \&#xA;      -o $&lt;/span&gt;(basename $F).VM.rFB4.IP9.quant \&#xA;      --incompatPrior 9.9999999999999995e-21 \  # default of v0.8&#xA;      --writeUnmappedNames \&#xA;      --validateMappings \&#xA;      --rangeFactorizationBins 4 &#xA;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;em&gt;Update&lt;/em&gt; &lt;code&gt;salmon&lt;/code&gt; is now released with a &lt;code&gt;--allowDovetail&lt;/code&gt; option in  0.13.1 and inspected via a &lt;code&gt;-z&lt;/code&gt; dump.&lt;/p&gt;&#xA;" OwnerUserId="4221" LastEditorUserId="3848" LastEditDate="2020-07-11T17:27:20.710" LastActivityDate="2023-03-24T14:04:10.527" Title="lower mapping rates in salmon v0.13 compared to previous versions" Tags="|rna-seq|salmon|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7199" PostTypeId="1" CreationDate="2019-03-06T11:02:33.150" Score="0" ViewCount="241" Body="&lt;p&gt;I am working on a script to create a correlation matrix and mutual_rank matrix from RNAseq gene counts for 57992 genes in 7027 samples. I have already tested this script for a smaller file (for 10,000 genes) and it is working, however, when I am running this script on my final input file, it is giving the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; corrupt matrix -- dims not not match length (this error comes after gene to &#xA; gene correlation step)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Script which i am running is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;E &amp;lt;- read.table(&quot;mdat_human_test.txt&quot;, header = TRUE, row.names=1)&#xA;&#xA;# gene-to-gene correlation&#xA;C &amp;lt;- cor (t(E))&#xA;write.table(C, &quot;human_correlation_genes_mdat.txt&quot;, quote=FALSE, sep=&quot;\t&quot;, &#xA;row.names=TRUE)&#xA;&#xA;# correlation rank matrix&#xA;R &amp;lt;- t(apply(1-C, 1, rank))&#xA;write.table(R, &quot;human_rank_genes_mdat.txt&quot;, quote=FALSE, sep=&quot;\t&quot;, &#xA;row.names=TRUE)&#xA;&#xA;# mutual rank matrix&#xA;M &amp;lt;- sqrt(R * t(R))&#xA;write.table(M, &quot;human_mutual_rank_genes_mdat.txt&quot;, quote=FALSE, sep=&quot;\t&quot;, &#xA;row.names=TRUE)&#xA;&#xA;q()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3766" LastActivityDate="2019-03-06T13:33:02.020" Title="Error in formation of correlation matrix-(corrupt matrix -- dims not not match length)" Tags="|r|rna-seq|correlation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7200" PostTypeId="2" ParentId="7195" CreationDate="2019-03-06T11:40:32.550" Score="1" Body="&lt;p&gt;The first thing you can try is to separate the network from the meta information. Assuming you have access to a *nix machine, you can run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -F&quot;,&quot; '{print &lt;span class=&quot;math-container&quot;&gt;$2,$&lt;/span&gt;4}' file.csv &amp;gt; network.csv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That will produce a file like this (based on your example data):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;test1 test2&#xA;test11 test12&#xA;test12 test22&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That should radically decrease the file size and will give you the best chance of loading it in Cytoscape. You can then similarly extract the attributes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -F&quot;,&quot; '{print &lt;span class=&quot;math-container&quot;&gt;$2,$&lt;/span&gt;1}' file.csv &amp;gt; source.column1.attributes&#xA;awk -F&quot;,&quot; '{print &lt;span class=&quot;math-container&quot;&gt;$2,$&lt;/span&gt;3}' file.csv &amp;gt; source.column3.attributes&#xA;awk -F&quot;,&quot; '{print &lt;span class=&quot;math-container&quot;&gt;$4,$&lt;/span&gt;5}' file.csv &amp;gt; target.column5.attributes&#xA;awk -F&quot;,&quot; '{print &lt;span class=&quot;math-container&quot;&gt;$4,$&lt;/span&gt;6}' file.csv &amp;gt; target.column6.attributes&#xA;awk -F&quot;,&quot; '{print &lt;span class=&quot;math-container&quot;&gt;$4,$&lt;/span&gt;7}' file.csv &amp;gt; target.column7.attributes&#xA;awk -F',' '{print &lt;span class=&quot;math-container&quot;&gt;$2,$&lt;/span&gt;4,&lt;span class=&quot;math-container&quot;&gt;$8}' file.csv &amp;gt; edge.column8.attributes&#xA;awk -F',' '{print $2,$4,$9}' file.csv &amp;gt; edge.column9.attributes&#xA;awk -F',' '{print $2,$4,$10}' file.csv &amp;gt; edge.column10.attributes&#xA;awk -F',' '{print $2,$4,$11}' file.csv &amp;gt; edge.column11.attributes&#xA;awk -F',' '{print $2,$4,$12}' file.csv &amp;gt; edge.column12.attributes&#xA;awk -F',' '{print $2,$4,$13}' file.csv &amp;gt; edge.column13.attributes&#xA;awk -F',' '{print $2,$4,$&lt;/span&gt;14}' file.csv &amp;gt; edge.column14.attributes&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, try loading &lt;code&gt;network.csv&lt;/code&gt; into Cytoscape and then load each of the node and edge attributes separately. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;However, a network with 200,000 nodes and 230,000 edges is just too large to be able to manipulate in any useful way in a GUI. It will just look like a horrible hairball. I really urge you to do some preprocessing and pare the network down to something more manageable. I used to work with a human PPI network of ~13000 nodes and ~15000 edges. While I &lt;em&gt;could&lt;/em&gt; load that into Cytoscape, that was already far too big to be able to do anything useful with. I could use the data analysis aspects of Cytoscape, but not really the visualization. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-03-06T11:40:32.550" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7201" PostTypeId="2" ParentId="7062" CreationDate="2019-03-06T13:20:16.503" Score="2" Body="&lt;p&gt;I've ended up using a package called &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/22689389&quot; rel=&quot;nofollow noreferrer&quot;&gt;BatMis&lt;/a&gt; for this, which can detect alignments with an arbitrary number of mismatches (not sure about indels). It is quite performant and yields about the order of magnitude of alignments that I was expecting (several thousands for alignment of a 20bp read to the whole human genome with up to 6 mismatches).&lt;/p&gt;&#xA;" OwnerUserId="4145" LastActivityDate="2019-03-06T13:20:16.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7202" PostTypeId="2" ParentId="7192" CreationDate="2019-03-06T13:24:39.413" Score="1" Body="&lt;p&gt;Since you indicate already having performed differential expression and using gene set enrichment the next step is to look at what changes due to drug treatment in the pathways of interest while having a decent understanding of those pathways. Things like IPA or pathview (from Bioconductor) can aid in this, but at the end of the day you need to have a decent understanding of the pathways yourself to judge what seems to be like the best targets for future work.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-03-06T13:24:39.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7203" PostTypeId="2" ParentId="7199" CreationDate="2019-03-06T13:33:02.020" Score="3" Body="&lt;p&gt;You can't compute a correlation matrix when you have more than ~46000 rows, since a standard R matrix can have a maximum of 2^31-1 values. Have a look at packages like &lt;a href=&quot;https://www.rdocumentation.org/packages/propagate/versions/1.0-6/topics/bigcor&quot; rel=&quot;nofollow noreferrer&quot;&gt;bigcor&lt;/a&gt;. Alternatively, consider if having 3+ billion correlation values is really useful.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-03-06T13:33:02.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7204" PostTypeId="1" CreationDate="2019-03-06T14:20:19.723" Score="3" ViewCount="327" Body="&lt;p&gt;I have a task in a training that I have to read and filter the 'good' reads of big fastq files. &#xA;I downsampled, got the code working, saving in a python dictionary. But turns out the original files are huge and I rewrite the code to give a generator. It did work for the down-sampled sample. But I was wondering if it's a good idea to get out all the data and filtering in a dictionary. Does anybody here a better idea? I am asking because I am learning python on my own and I am not sure if it makes sense.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I got some ideas from a code in Biostar:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import sys&#xA;import gzip&#xA;&#xA;filename = sys.argv[1]&#xA;&#xA;def parsing_fastq_files(filename):&#xA;&#xA;    with gzip.open(filename, &quot;rb&quot;) as infile:&#xA;        count_lines = 0&#xA;            for line in infile:&#xA;            line = line.decode()&#xA;            if count_lines % 4 == 0:&#xA;                ids = line[1:].strip()&#xA;                yield ids&#xA;            if count_lines == 1:&#xA;                reads = line.rstrip()&#xA;                yield reads&#xA;        count_lines += 1&#xA;&#xA;total_reads = parsing_fastq_files(filename)&#xA;print(next(total_reads))&#xA;print(next(total_reads))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I now need to figure out to get the data filtered by using 'if value.endswith('expression'):' but if I use a dict for example, but that's my doubt because of the number of keys and values.&lt;/p&gt;&#xA;" OwnerUserId="4222" LastEditorUserId="3669" LastEditDate="2019-03-08T11:23:22.950" LastActivityDate="2019-03-08T11:23:22.950" Title="Getting data from fastq by generator" Tags="|python|fastq|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7205" PostTypeId="2" ParentId="7204" CreationDate="2019-03-06T14:25:26.090" Score="2" Body="&lt;p&gt;Have a function that returns a single-end read (or pair) and another that does any relevant filtering on that. Wrap both of them in another function that checks if you've hit the end of the file. That's both simple, will handle any size of file, and allows single and paired-end reads.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-03-06T14:25:26.090" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7206" PostTypeId="2" ParentId="7204" CreationDate="2019-03-06T16:56:39.773" Score="2" Body="&lt;p&gt;I do not have an example of fastqs with 'expression' as the last string in the header, so instead I will filter based off of a specific barcode used for demultiplexing. I have a gzipped fastq of 250 reads in size. Based on Devon's answer, I have first generated a fastq parser which stores only (header, seq) as a list of tuples, and then looks inside the list of tuples for a specific index. Some of the indexes vary by 1 nucleotide so those will be left out. I can prove this by printing the length of the 2 lists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import sys&#xA;import gzip&#xA;&#xA;fastq = sys.argv[1]&#xA;&#xA;def parser(filename):&#xA;    &quot;&quot;&quot;&#xA;    Generate a list of tuples (header, read)&#xA;    &quot;&quot;&quot;&#xA;    fastq_parsed = []&#xA;    try:&#xA;&#xA;        with gzip.open(filename) as fq:&#xA;            header = next(fq)&#xA;            read = next(fq)&#xA;            fastq_parsed.append((header[1:-1], read[:-1])) # we don't want new-line chars or @ sign in reads&#xA;            while True:&#xA;                next(fq) # skip + line&#xA;                next(fq) # skip qscore&#xA;                header = next(fq) # store next read header&#xA;                read = next(fq) # store next read seq&#xA;                fastq_parsed.append((header[1:-1], read[:-1]))&#xA;    except:&#xA;        StopIteration # when we run out of reads, stop&#xA;    return fastq_parsed&#xA;&#xA;def expression_finder(read_list):&#xA;    &quot;&quot;&quot;&#xA;    Find headers that end with expression assuming the list format generated above.&#xA;    &quot;&quot;&quot;&#xA;    filtered_reads = []&#xA;    for header, read in read_list:&#xA;        if (header.endswith(b'ATCACGAT')): #gzip files in bytes format, so we pass bytes object.&#xA;            filtered_reads.append((header.decode(&quot;utf-8&quot;), read.decode(&quot;utf-8&quot;))) # list items decoded to utf-8&#xA;    return filtered_reads&#xA;&#xA;&#xA;read_list = parser(fastq)&#xA;print(len(read_list))&#xA;subset_reads = expression_finder(read_list)&#xA;print(len(subset_reads))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which returns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(base) [dkennetz@nodecn203  test]$ python3.6 fastq_parser.py subset.fastq.gz&#xA;250&#xA;173&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We can see that of the 250 reads, 173 have the correct index. I have hardcoded the filter in the second function, but you could also pass it as an argument to the function, like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def expression_finder(read_list, filter):&#xA;    ...&#xA;        if header.endswith(filter):&#xA;            ...&#xA;&#xA;subset_reads = expression_finder(read_list, b'expression')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which would also work. But include the &lt;code&gt;b''&lt;/code&gt; in your string syntax because the first function still passes a bytes object.&lt;/p&gt;&#xA;" OwnerUserId="3669" LastActivityDate="2019-03-06T16:56:39.773" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7207" PostTypeId="2" ParentId="7196" CreationDate="2019-03-06T17:01:53.317" Score="2" Body="&lt;p&gt;I managed to track down what was producing this error. The problem was that my bed file only had one segment for chromosome &lt;code&gt;chrUn_gl000228&lt;/code&gt;. However, the cn.MOPS method relies on normalizing read counts across multiple segments for each chromosome. Since this chromosome only had a single segment, the &lt;code&gt;exomecn.mops(X)&lt;/code&gt; was failing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, either remove the single segment from the bed file, or use a bed file that has at least 2 segments on each chromosome. I created a fake segment on &lt;code&gt;chrUn_gl000228&lt;/code&gt; and confirmed that the program worked as expected and didn't throw any errors.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-05-31T08:35:59.723" LastActivityDate="2019-05-31T08:35:59.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7208" PostTypeId="1" CreationDate="2019-03-06T18:04:41.270" Score="0" ViewCount="897" Body="&lt;p&gt;I run this cmd &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./stringtie G1_sorted.bam -B -o G1.gtf -G Triticum_aestivum.IWGSC.42.gtf -p 4  -C G1.refs.gtf -A G1.abund.tab&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Error is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-WARNING: no reference transcripts were found for the genomic sequences where reads were mapped!&#xA;Please make sure the -G annotation file uses the same naming convention for the genome sequences.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I resolve it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have downloaded both (GTF and Refernce Genome)files from Ensemble. Can you confirm me that whether theu have same chromosomes naming? So that i can perform my mapping  task of GTF File with Reference Genome which usually i remain fail to do due to difference in chromosomes naming.                                                           Here The Reference genome file is shown as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; &amp;gt;1A dna:chromosome chromosome:IWGSC:1A:1:594102056:1 REF&#xA;CTAAACCCTAAACCCTAAACCCTAAACCCTAAACCCCTAACCCTAAACCCTAACCAAAAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where as The annotation file (gtf file ) is as                                                                                                 &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    #!genome-build IWGSC&#xA;#!genome-version IWGSC&#xA;#!genome-date 2018-07&#xA;#!genome-build-accession GCA_900519105.1&#xA;3B      IWGSC   gene    212892  214491  .       -       .       gene_id &quot;TraesCS3B02G000100&quot;; gene_source &quot;IWGSC&quot;; gene_biotype &quot;protein_coding&quot;;&#xA;3B      IWGSC   transcript      212892  214491  .       -       .       gene_id &quot;TraesCS3B02G000100&quot;; transcript_id &quot;TraesCS3B02G000100.1&quot;; gene_source &quot;IWGSC&quot;; gene_biotype &quot;protein_coding&quot;; transcript_source &quot;IWGSC&quot;; transcript_biotype &quot;protein_coding&quot;; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can it be confirmend that these both files are same ? If not same with chromosome naming convention , then please suggest me how can I change it ?&lt;/p&gt;&#xA;" OwnerUserId="4194" LastEditorUserId="77" LastEditDate="2019-03-12T11:13:08.127" LastActivityDate="2019-03-12T11:13:08.127" Title="Assembly by stringtie" Tags="|rna-seq|assembly|transcriptome|gtf|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7211" PostTypeId="1" CreationDate="2019-03-06T21:02:37.020" Score="1" ViewCount="1358" Body="&lt;p&gt;I got the following error when attempting to intersect two files. The file below is supposed to be the -a file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error: Unable to open file compare_me_to.txt. Exiting.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I attempted to make it tab-delimited through Ubuntu (see code below)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl -p -i -e 's/ /\t/g' compare_me_to.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I got the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Can't open compare_me_to: Permission denied.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Additionally, the ownership of the folder the files are in is of my username.&#xA;I can't tell if this is a bedtools problem or a Ubuntu problem or something else.&lt;/p&gt;&#xA;" OwnerUserId="4211" LastActivityDate="2019-03-11T17:39:16.347" Title="bedtools unable to open file despite being tab-delimited" Tags="|bedtools|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7212" PostTypeId="1" CreationDate="2019-03-06T21:52:41.193" Score="1" ViewCount="893" Body="&lt;p&gt;MaxQuant is a tool for proteomics quantification. In the settings under &lt;code&gt;Global parameters / Folder locations&lt;/code&gt; are three folder locations that are possible to be changed. Thought the manual does not even mention these options. Is there any benefit in changing these locations? Or can it be bad to do that? For example, if I run multiple MaxQuant instances at the same time doing different searches?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the meaning of the &lt;code&gt;Fixed andromeda index folder&lt;/code&gt;, &lt;code&gt;Temporary folder&lt;/code&gt; and &lt;code&gt;Combined folder location&lt;/code&gt; in MaxQuant? What is it used for and when would you (not) change it?&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2019-06-17T15:59:39.553" Title="What is the purpose of 'Folder locations' in MaxQuant?" Tags="|proteomics|maxquant|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7213" PostTypeId="2" ParentId="7211" CreationDate="2019-03-07T03:00:30.303" Score="1" Body="&lt;p&gt;I suspect the reason is the &quot;-i&quot;. The following will work,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl -pe 's/  */\t/g' compare_me_to.txt &amp;gt; compare_me_to.txt2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;-i will want to write onto the file and create a backup. I suspect you'll have spotted &lt;code&gt;chmod +w compare_me_to.txt&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just make sure that a double space is inserted before the * character, otherwise it will go wrong. Copying and pasting will avoid this.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This is a clearer solution because it avoids missing a space character,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl -e 's/[ ][ ]*/\t/g' compare_me_to.txt &amp;gt; compare_me_to.txt2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the solution you used .. (I did on OS X), what I'd recommend is,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl -p -e 's/[ ][ ]*/\t/g' compare_me_to.txt # to see if it works by printing on screen without altering the file&#xA;&#xA;perl -i -e 's/[ ][ ]*/\t/g' compare_me_to.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The  -i flag will alter the original file and store a backup without requiring renaming the file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What's it doing?&lt;/strong&gt;&#xA;The regex [ ][ ]* structure is saying look a space and if one or more spaces occur adjacent to each other replace everything with a single tab. [ ]* means a &quot;space&quot; that is present or absent and if present it can be present an infinite number of times. If the [ ]* is by itself the command risks saying replace '' (zip) or 'space' with a tab, so tabs can appear at every '' (zilch), so tabs can appear everywhere. The &quot;*&quot; symbol in regex work differently to the &quot;*&quot; wildcard character in shell.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-03-11T17:39:16.347" LastActivityDate="2019-03-11T17:39:16.347" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7214" PostTypeId="1" CreationDate="2019-03-07T09:07:45.327" Score="1" ViewCount="247" Body="&lt;p&gt;I have a vcf containing SNV and MNV (short indells or substitutions), I have annotated that file with annovar and in the resulting txt file I have this row:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr7    140453136   140453137     AC   TT  exonic   BRAF    .     nonframeshift substitution              BRAF:NM_004333:exon15:c.1798_1799AA:p.V600K&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If I convert the txt file in maf using annovarToMaf function of &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/maftools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;maftools&lt;/a&gt; bioconductor package I obtain &lt;code&gt;NA&lt;/code&gt; as value for &lt;code&gt;Variant_Classification&lt;/code&gt; field. In other words, the field is included in the original vcf file but it is converted into NA when saved as maf with maftools.&lt;/p&gt;&#xA;&lt;p&gt;What is the correct value of the Variant_Classification field in a .maf file for non-frameshift substitutions like BRAF V600K?&lt;/p&gt;&#xA;&lt;p&gt;I tried vcf2maf and that tool put &lt;code&gt;Missense_Mutation&lt;/code&gt; as a value for &lt;code&gt;Variant_Classification&lt;/code&gt; of BRAF V600K mutations.&lt;/p&gt;&#xA;&lt;p&gt;I tried to download the .maf file of TCGA melanoma (that should contain BRAF V600K mutations) but the access to that file is restricted.&lt;/p&gt;&#xA;" OwnerUserId="3772" LastEditorUserId="73" LastEditDate="2022-06-21T09:53:23.313" LastActivityDate="2022-06-21T09:53:23.313" Title="maf file format: Variant_Classification value for non frameshift substitution" Tags="|r|mutations|maf|maftools|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7215" PostTypeId="2" ParentId="3421" CreationDate="2019-03-07T09:09:47.827" Score="6" Body="&lt;p&gt;You can use annovar to annotate the vcf, then convert it to maf using the function &lt;code&gt;annovarToMaf&lt;/code&gt; of &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/maftools.html&quot; rel=&quot;noreferrer&quot;&gt;maftools&lt;/a&gt; bioconductor package.&lt;/p&gt;&#xA;" OwnerUserId="3772" LastEditorUserId="3772" LastEditDate="2019-03-19T10:11:56.000" LastActivityDate="2019-03-19T10:11:56.000" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7216" PostTypeId="2" ParentId="7208" CreationDate="2019-03-07T09:11:41.320" Score="3" Body="&lt;p&gt;This seems to be a common problem. As @llrs pointed out, Ensembl, UCSC(genome browser and associated data) and NCBI records are not directly interchangeable. The ambiguity refers to annotation as well as the reference genome. The reference genome is identical, except for the chromosome naming convention, meaning a reference from Ensembl might not contain CHR prefix, whereas it will be present in one downloaded from UCSC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To prevent such conflicts in downstream tools, it is good to stick to the reference and annotation from the same repository throughout the analysis and map the genes to gene names or whatever towards the end of the investigation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can download all the associated data for &lt;em&gt;T. aestivum&lt;/em&gt; which I beleive is the organism you want to study from Ensembl plants FTP &lt;a href=&quot;http://plants.ensembl.org/info/website/ftp/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;download&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="1135" LastActivityDate="2019-03-07T09:11:41.320" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7217" PostTypeId="1" AcceptedAnswerId="7223" CreationDate="2019-03-07T12:19:23.797" Score="1" ViewCount="1726" Body="&lt;p&gt;While going through the &lt;a href=&quot;https://ccb.jhu.edu/software/hisat2/manual.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;HISAT2 manual&lt;/a&gt;, one can see this: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Two alignments for the same individual read are &quot;distinct&quot; if they map&#xA;  the same read to different places. Specifically, we say that two&#xA;  alignments are distinct if there are no alignment positions where a&#xA;  particular read offset is aligned opposite a particular reference&#xA;  offset in both alignments with the same orientation. E.g. if the first&#xA;  alignment is in the forward orientation and aligns the read character&#xA;  at read offset 10 to the reference character at chromosome 3, offset&#xA;  3,445,245, and the second alignment is also in the forward orientation&#xA;  and also aligns the read character at read offset 10 to the reference&#xA;  character at chromosome 3, offset 3,445,245, they are not distinct&#xA;  alignments.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Two alignments for the same pair are distinct if either the mate 1s in&#xA;  the two paired-end alignments are distinct or the mate 2s in the two&#xA;  alignments are distinct or both.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Default mode: search for one or more alignments, report each&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and then there is -k parameter (analogous to bowtie2 -K parameter)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;which defines: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;HISAT2 searches for up to N distinct, primary alignments for each&#xA;  read, where N equals the integer specified with the -k parameter.&#xA;  Primary alignments mean alignments whose alignment score is equal or&#xA;  higher than any other alignments. It is possible that multiple&#xA;  distinct alignments have the same score. That is, if -k 2 is&#xA;  specified, HISAT2 will search for at most 2 distinct alignments. The&#xA;  alignment score for a paired-end alignment equals the sum of the&#xA;  alignment scores of the individual mates. Each reported read or pair&#xA;  alignment beyond the first has the SAM 'secondary' bit (which equals&#xA;  256) set in its FLAGS field. See the SAM specification for details.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is a bit confusing, as the first quote means that HISAT2 finds all distinct valid alignment and reports them by default, and the second quote says that when -k option is used, it will only report a subset of alignments which it would have reported by default. Am I correct in this inference or does one needs to use -k option to enable &quot;multi-mapping&quot;?&lt;/p&gt;&#xA;" OwnerUserId="1135" LastEditorUserId="96" LastEditDate="2019-03-07T18:01:56.017" LastActivityDate="2019-03-07T18:01:56.017" Title="HISAT2 - are all multi mapped reads reported by default?" Tags="|read-mapping|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7218" PostTypeId="1" AcceptedAnswerId="7247" CreationDate="2019-03-07T14:11:05.170" Score="1" ViewCount="33" Body="&lt;p&gt;I will preface this by saying I'm not a microbiologist, so I apologise for any haziness and I'll be happy to expand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to download the assemblies of different E. coli genomes and group them by their pathotypes (e.g. EPEC, ETEC, EIEC). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used enterobase previously to download salmonella genomes broadly by their subspecies, and it does have a set of E. coli genomes. However, they are mainly annotated by their serotype, and not with any broad nomination of pathogenesis. I think I would be able to use the API to grab these serotypes and map them to a type of pathogenicity but that would probably be quite messy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does such a database exist? Most I can find are broadly annotated or a are very detailed, but don't allow access to the raw annotation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="252" LastActivityDate="2019-03-14T11:57:01.997" Title="Is there a resource where I can donwload E. coli assemblies grouped by their pathotypes?" Tags="|genome|public-databases|assembly|microbiology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7219" PostTypeId="1" CreationDate="2019-03-07T15:04:50.473" Score="2" ViewCount="38" Body="&lt;p&gt;Inference of structural variants from short read data is always sneaky (for a question about pair-end reads, see &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/120/how-can-i-call-structural-variants-svs-from-pair-end-short-read-resequencing-d&quot;&gt;this&lt;/a&gt;). Recently I bumped into &lt;a href=&quot;https://onlinelibrary.wiley.com/doi/abs/10.1111/mec.15016&quot; rel=&quot;nofollow noreferrer&quot;&gt;a paper&lt;/a&gt; where they used mate-pairs for inference of structural variants and it made me wonder if the approach might be problematic or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mate-pairs have a known problem of &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/27153683&quot; rel=&quot;nofollow noreferrer&quot;&gt;contamination by pair-end reads&lt;/a&gt; and there is even a tool for calling structural variations considering this problem called &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5629590/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SVachara&lt;/a&gt;. However, authors have used methods &lt;a href=&quot;https://doi.org/10.1093/bioinformatics/btq665&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lumpy&lt;/a&gt; and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3436805/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Delly2&lt;/a&gt;. Have they chosen an appropriate approach?&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="3848" LastEditDate="2019-08-25T22:52:56.440" LastActivityDate="2019-08-25T22:52:56.440" Title="Is there a risk in inference of structural variations using mate-pairs?" Tags="|structural-variation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7220" PostTypeId="1" AcceptedAnswerId="13048" CreationDate="2019-03-07T15:06:40.397" Score="0" ViewCount="553" Body="&lt;p&gt;I am sure I set the path right but whatever I am trying the command not working&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help please?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[fi1d18@cyan02 fi1d18]&lt;span class=&quot;math-container&quot;&gt;$ picard MarkDuplicates \&#xA;    I=DNA.dupmarked.bam \&#xA;    O=DNA.dupmarked1.bam \&#xA;    M= marked-dup-metrics.txt &#xA;[Thu Mar 07 17:33:42 GMT 2019] picard.sam.markduplicates.MarkDuplicates INPUT=[/temp/hgig/fi1d18/1631_WTSI-OESO_005_a_DNA/mapped_sample/HUMAN_1000Genomes_hs37d5_genomic_WTSI-OESO_005_a_DNA.dupmarked.bam] OUTPUT=/temp/hgig/fi1d18/1631_WTSI-OESO_005_a_DNA/mapped_sample/HUMAN_1000Genomes_hs37d5_genomic_WTSI-OESO_005_a_DNA.dupmarked1.bam METRICS_FILE=marked-dup-metrics.txt MAX_SEQUENCES_FOR_DISK_READ_ENDS_MAP=50000 MAX_FILE_HANDLES_FOR_READ_ENDS_MAP=8000 SORTING_COLLECTION_SIZE_RATIO=0.25 REMOVE_SEQUENCING_DUPLICATES=false TAGGING_POLICY=DontTag REMOVE_DUPLICATES=false ASSUME_SORTED=false DUPLICATE_SCORING_STRATEGY=SUM_OF_BASE_QUALITIES PROGRAM_RECORD_ID=MarkDuplicates PROGRAM_GROUP_NAME=MarkDuplicates READ_NAME_REGEX=&amp;lt;optimized capture of last three ':' separated fields as numeric values&amp;gt; OPTICAL_DUPLICATE_PIXEL_DISTANCE=100 VERBOSITY=INFO QUIET=false VALIDATION_STRINGENCY=STRICT COMPRESSION_LEVEL=5 MAX_RECORDS_IN_RAM=500000 CREATE_INDEX=false CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json&#xA;[Thu Mar 07 17:33:42 GMT 2019] Executing as fi1d18@cyan02 on Linux 2.6.32-754.el6.x86_64 amd64; Java HotSpot(TM) 64-Bit Server VM 1.8.0_51-b16; Picard version: 2.8.3-SNAPSHOT&#xA;INFO 2019-03-07 17:33:42 MarkDuplicates Start of doWork freeMemory: 2012347496; totalMemory: 2027945984; maxMemory: 3817865216&#xA;INFO 2019-03-07 17:33:42 MarkDuplicates Reading input file and constructing read end information.&#xA;INFO 2019-03-07 17:33:42 MarkDuplicates Will retain up to 14684096 data points before spilling to disk.&#xA;WARNING: BAM index file /temp/hgig/fi1d18/1631_WTSI-OESO_005_a_DNA/mapped_sample/HUMAN_1000Genomes_hs37d5_genomic_WTSI-OESO_005_a_DNA.dupmarked.bam.bai is older than BAM /temp/hgig/fi1d18/1631_WTSI-OESO_005_a_DNA/mapped_sample/HUMAN_1000Genomes_hs37d5_genomic_WTSI-OESO_005_a_DNA.dupmarked.bam&#xA;[Thu Mar 07 17:33:42 GMT 2019] picard.sam.markduplicates.MarkDuplicates done. Elapsed time: 0.01 minutes.&#xA;Runtime.totalMemory()=2027945984&#xA;To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp&#xA;Exception in thread &quot;main&quot; htsjdk.samtools.SAMFormatException: SAM validation error: ERROR: Record 3752, Read name HX3_22030:3:2114:23155:23319, bin field of BAM record does not equal value computed based on alignment start and end, and length of sequence to which read is aligned&#xA;at htsjdk.samtools.SAMUtils.processValidationErrors(SAMUtils.java:448)&#xA;at htsjdk.samtools.BAMFileReader$&lt;/span&gt;BAMFileIterator.advance(BAMFileReader.java:665)&#xA;at htsjdk.samtools.BAMFileReader&lt;span class=&quot;math-container&quot;&gt;$BAMFileIterator.next(BAMFileReader.java:650)&#xA;at htsjdk.samtools.BAMFileReader$&lt;/span&gt;BAMFileIterator.next(BAMFileReader.java:620)&#xA;at htsjdk.samtools.SamReader&lt;span class=&quot;math-container&quot;&gt;$AssertingIterator.next(SamReader.java:569)&#xA;at htsjdk.samtools.SamReader$&lt;/span&gt;AssertingIterator.next(SamReader.java:543)&#xA;at picard.sam.markduplicates.MarkDuplicates.buildSortedReadEndLists(MarkDuplicates.java:438)&#xA;at picard.sam.markduplicates.MarkDuplicates.doWork(MarkDuplicates.java:222)&#xA;at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:205)&#xA;at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:94)&#xA;at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:104)&#xA;[fi1d18@cyan02 fi1d18]$&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The error message is:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Exception in thread &quot;main&quot; htsjdk.samtools.SAMFormatException: validation error: ERROR: Record 3752, Read name HX3_22030:3:2114:23155:23319, bin field of BAM record does not equal value computed based on alignment start and end, and length of sequence to which read is aligned&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;EDITED&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[fi1d18@cyan02 sambamba]$ samtools view -H /temp/hgig/fi1d18/1631_WTSI-OESO_005_a_DNA/mapped_sample/HUMAN_1000Genomes_hs37d5_genomic_WTSI-OESO_005_a_DNA.dupmarked.bam&#xA;[W::bam_hdr_read] EOF marker is absent. The input is probably truncated.&#xA;@HD     VN:1.5  SO:coordinate&#xA;@SQ     SN:1    LN:249250621    AS:NCBI37       M5:1b22b98cdeb4a9304cb5d48026a85128     SP:Human  UR:/lustre/scratch117/core/sciops_repository/references/Homo_sapiens/1000Genomes_hs37d5/all/fasta/hs37d5.fa&#xA;@SQ     SN:2    LN:243199373    AS:NCBI37       M5:a0d9851da00400dec1098a9255ac712e     SP:Human  UR:/lustre/scratch117/core/sciops_repository/references/Homo_sapiens/1000Genomes_h&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="2776" LastEditDate="2020-04-24T18:30:25.930" LastActivityDate="2020-04-24T18:30:25.930" Title="Picard validation error regarding bin field of BAM file" Tags="|sam|wgs|picard|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="7221" PostTypeId="1" AcceptedAnswerId="7222" CreationDate="2019-03-07T16:28:11.780" Score="3" ViewCount="83" Body="&lt;p&gt;I have asked this question at BioStars but have not gotten any suggestions so far. So, I am asking the community here, perhaps someone here has an idea how I can solve my problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am dealing with a bunch of inversions and running some analysis on each of them. One of them is located on chromosome one, the starting position is 6199548 and ending position is 9699260. So the length of this variant is 3499712 bp. I want to generate non-overlapping windows based on the length of these focal positions (3499712) and assign the focal position to one of these windows that overlap with the starting and ending position of my focal position. I am achieving this by using the &lt;code&gt;GenomicRanges::tileGenome&lt;/code&gt; function in &lt;code&gt;R&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seqlength &amp;lt;-c(chr1=159217232,chr2=184765313,chr3=182121094,chr4=221926752,chr5=187413619)&#xA;bin_length&amp;lt;-c(3499712) &#xA; bins&amp;lt;-tileGenome(seqlength,tilewidth=bin_length,cut.last.tile.in.chrom&#xA;        = T)&#xA;            #bins_u&amp;lt;-unlist(bins)&#xA;            write.table(bins,file=paste(&quot;bins_good_&quot;,bin_length,sep=&quot;&quot;),col.names&#xA;        = FALSE, row.names = TRUE) }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This generates non-overlapping 3499712 bps windows across the genome.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;window_id chromosome start end length&#xA;&quot;1&quot; &quot;chr1&quot; 1 3499712 3499712&#xA;&quot;2&quot; &quot;chr1&quot; 3499713 6999424 3499712 &#xA;&quot;3&quot; &quot;chr1&quot; 6999425 10499136 3499712 &#xA;&quot;4&quot; &quot;chr1&quot; 10499137 13998848 3499712 &#xA;&quot;5&quot; &quot;chr1&quot; 13998849 17498560 3499712&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I have a major problem here that I have not been able to sort out. I want one of these windows to overlaps exactly with the starting and ending position of my focal position (6199548:9699260). Only the focal position not any other position outside this region. This means 2 windows (one before and one after the focal position) will have a different length which my analysis is fine with. I just do not know how to do it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my desired output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;window_id chromosome start end length&#xA;&quot;1&quot; &quot;chr1&quot; 1 3499712 3499712 &quot;*&quot;                  &#xA;&quot;2&quot; &quot;chr1&quot; 3499713 6199547 2699834 &quot;*&quot;&#xA;&quot;3&quot; &quot;chr1&quot; 6199548  9699260 3499712 &quot;*&quot;&#xA;&quot;4&quot; &quot;chr1&quot; 9699261 13998848 4299587 &quot;*&quot;&#xA;&quot;5&quot; &quot;chr1&quot; 13998849 17498560 3499712 &quot;*&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you see, the the second window is now smaller, the third window accommodates the focal position and the forth window is a little larger than the bin size. Does anyone have any idea how can I do this task in R? &lt;/p&gt;&#xA;" OwnerUserId="1749" LastEditorUserId="298" LastEditDate="2019-03-07T18:56:33.997" LastActivityDate="2019-03-07T18:56:33.997" Title="How can I assign a genomic region into a window using R?" Tags="|r|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7222" PostTypeId="2" ParentId="7221" CreationDate="2019-03-07T17:17:23.490" Score="5" Body="&lt;p&gt;I'm not sure how you did your calculation but for me it looks like your region is &lt;strong&gt;3499713&lt;/strong&gt; bp in length.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;site &amp;lt;- GRanges(&quot;chr1:6199548-9699260&quot;)&#xA;&#xA;seqlength &amp;lt;- c(chr1=159217232,chr2=184765313,chr3=182121094,chr4=221926752,chr5=187413619)&#xA;&#xA;bin_length &amp;lt;- width(site)&#xA;&#xA;## check length of region&#xA;print(bin_length)&#xA;##3499713&#xA;&#xA;bins &amp;lt;- tileGenome(seqlength, tilewidth=bin_length, cut.last.tile.in.chrom=TRUE)&#xA;&#xA;## extract overlapping bins&#xA;flank.bins &amp;lt;- subsetByOverlaps(bins,site)&#xA;&#xA;## find new coordinates excluding region of interest&#xA;new.flank.bins &amp;lt;- setdiff(reduce(flank.bins),site)&#xA;&#xA;## combine new bins with your region of interest&#xA;new.bins &amp;lt;- c(new.flank.bins,site)&#xA;&#xA;## remove old bins&#xA;bins &amp;lt;- subsetByOverlaps(bins,new.bins,invert=T)&#xA;&#xA;## add new bins&#xA;bins &amp;lt;- sort(c(bins,new.bins))&#xA;&#xA;## check that it makes sense&#xA;##table(width(bins[seqnames(bins) == 'chr1']))&#xA;&#xA;write.table(bins,file=paste(&quot;bins_good_&quot;,bin_length,sep=&quot;&quot;),col.names= FALSE, row.names = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="383" LastActivityDate="2019-03-07T17:17:23.490" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7223" PostTypeId="2" ParentId="7217" CreationDate="2019-03-07T18:00:30.253" Score="3" Body="&lt;p&gt;You said:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;...the first quote means that HISAT2 finds all distinct valid alignment and reports them by default.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Not exactly. The first section you quoted doesn't say it will find and report &lt;em&gt;all&lt;/em&gt; distinct alignments, but rather that it would find and report one or more distinct alignments. It's a subtle difference, but an important one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That section describes how distinct mappings are defined, not how many are reported. The second section you quoted, as well as the documentation for the &lt;code&gt;-k&lt;/code&gt; parameter, are all consistent with this interpretation.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;-k &amp;lt;int&amp;gt;        It searches for at most &amp;lt;int&amp;gt; distinct, primary alignments for&#xA;                each read. Primary alignments mean alignments whose alignment&#xA;                score is equal or higher than any other alignments. The search&#xA;                terminates when it can't find more distinct valid alignments, or&#xA;                when it finds &amp;lt;int&amp;gt;, whichever happens first. The alignment&#xA;                score for a paired-end alignment equals the sum of the alignment&#xA;                scores of the individual mates. Each reported read or pair&#xA;                alignment beyond the first has the SAM 'secondary' bit (which&#xA;                equals 256) set in its FLAGS field. For reads that have more&#xA;                than &amp;lt;int&amp;gt; distinct, valid alignments, hisat2 does not guarantee&#xA;                that the &amp;lt;int&amp;gt; alignments reported are the best possible in&#xA;                terms of alignment score. Default: 5 (HFM) or 10 (HGFM)&#xA;&#xA;                Note: HISAT2 is not designed with large values for -k in mind,&#xA;                and when aligning reads to long, repetitive genomes large -k can&#xA;                be very, very slow.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;By default, 5 or 10 distinct mappings are reported (depending on the mode), but you can control this with the &lt;code&gt;-k&lt;/code&gt; parameter.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-03-07T18:00:30.253" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7224" PostTypeId="1" CreationDate="2019-03-07T18:44:44.220" Score="1" ViewCount="178" Body="&lt;p&gt;I am helping an undergrad with their project and their thesis advisor has requested they use GENELAND for their analysis. I haven't worked on this sort of analysis, so all I have done is install it in their R, hand over the manual, and help troubleshoot any errors they can't resolve. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We've gotten the analysis through the MCMC stage and are getting an error with the PostProcessingChain command.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Their MCMC command is: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;MCMC(geno.hap=geno1, spatial=FALSE, nb.nuclei.max=3, &#xA;    varnpop=TRUE, freq.model=&quot;Uncorrelated&quot;, &#xA;    filter.null.alleles=FALSE, nit=100000, &#xA;    thinning=100, npopmin=1, npopinit=3, &#xA;    npopmax=10, rate.max=3, delta.coord=0, &#xA;    path.mcmc=&quot;/Users/myname/localdoc/GENELAND&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Their PostProcessChain command is: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PostProcessChain(coordinates=gene1coord.utm, &#xA;    path.mcmc =&quot;/Users/myname/localdoc/GENELAND/&quot;, &#xA;    nxdom=100, nydom=100, burnin=200)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is exactly the same command as in the manual, and &lt;code&gt;gene1coord.utm&lt;/code&gt; is an R object with the coordinates transformed into UTM coordinates. The error we receive is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/avlCv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/avlCv.png&quot; alt=&quot;Fortran runtime error: File already opened in another unit&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;([1] &quot;Calling Fortran function postprocesschain2&quot;&#xA;&#xA;At line 12334 of file main.f (unit = 10)&#xA;&#xA;Fortran runtime error: File already opened in another unit)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have tried &lt;code&gt;fix(PostProcessChain)&lt;/code&gt; to see if there's something I can adjust and this is what I see for the section that shows up:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tqUoE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tqUoE.png&quot; alt=&quot;fix(PostProcessChain)&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far, here is the list of things I have tried:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Reinstall X11&lt;/li&gt;&#xA;&lt;li&gt;Reinstall gfortran&lt;/li&gt;&#xA;&lt;li&gt;make sure that R is 3.4.x as the GENELAND package suggests&lt;/li&gt;&#xA;&lt;li&gt;Tried different parameters on MCMC just in case it was user error&lt;/li&gt;&#xA;&lt;li&gt;Tried different parameters on PostProcessChain, but they're the same as the manual really&lt;/li&gt;&#xA;&lt;li&gt;Tried different kinds of coordinates&lt;/li&gt;&#xA;&lt;li&gt;Tried a spatial analysis instead of a non-spatial&lt;/li&gt;&#xA;&lt;li&gt;Googled the error&lt;/li&gt;&#xA;&lt;li&gt;Restarted the computer&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Alternatively, is there another program that does similar analysis to geneland? I'm not sure if the advisor recommended it because it's the only thing that does what it does or because that's just what they used back in the day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: There were 3 blank files in the folder titled %FO, P, and x. When I deleted them the &quot;file open&quot; error went away but it still hangs up on [ 1 ] &lt;code&gt;Calling Fortran function postprocesschain2&lt;/code&gt; and has a fatal error that crashes R.&lt;/p&gt;&#xA;" OwnerUserId="4230" LastEditorUserId="77" LastEditDate="2019-03-07T23:35:37.093" LastActivityDate="2019-03-07T23:35:37.093" Title="Geneland R + Fortran error?" Tags="|r|phylogenetics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7226" PostTypeId="2" ParentId="7139" CreationDate="2019-03-09T22:41:20.033" Score="3" Body="&lt;p&gt;There are two other ways in which you could demultiplex reads if guppy doesn't have the config file for the kit you are using.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/calacademy-research/minibar&quot; rel=&quot;nofollow noreferrer&quot;&gt;Minibar&lt;/a&gt;: This works for dual index barcodes specifically&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/rrwick/Porechop&quot; rel=&quot;nofollow noreferrer&quot;&gt;Porechop&lt;/a&gt;: This is intended for the ONT sets, but you can also replace them with your own. However, it is designed to work with single index barcodes. That means, you will have to run it once with your forward indices, then take that output and run again with the reverse, and then do some gymnastics to combine it all together&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4235" LastEditorUserId="57" LastEditDate="2019-03-10T12:37:57.583" LastActivityDate="2019-03-10T12:37:57.583" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7228" PostTypeId="1" CreationDate="2019-03-11T06:09:23.587" Score="4" ViewCount="93" Body="&lt;p&gt;UCSC provides a database of telomere regions for each chromosome:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;ftp://hgdownload.cse.ucsc.edu/goldenPath/hg19/database/gap.txt.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://hgdownload.cse.ucsc.edu/goldenPath/hg19/database/gap.txt.gz&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It reported the telomere section in chr21 as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep chr21 gap.txt | grep telomere&#xA;585 chr21   0   10000   1   N   10000   telomere    no&#xA;952 chr21   48119895    48129895    515 N   10000   telomere    no&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Telomere is the first 10000 bases in chr21?? But ... the region are all &quot;N&quot; in the reference genome!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How does UCSC confidently give out telomere regions if they don't even know anything about the sequences?&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="77" LastEditDate="2019-03-11T08:24:21.143" LastActivityDate="2019-03-12T11:03:08.830" Title="Does UCSC definition for telomere making sense?" Tags="|genomics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7229" PostTypeId="1" AcceptedAnswerId="20633" CreationDate="2019-03-11T10:23:11.457" Score="2" ViewCount="1677" Body="&lt;p&gt;I am working on subclonal evolution of multiple myeloma. After literature survey, I found people use Fish plot to represent tumor evolution. The example of fish plot can be found &lt;a href=&quot;https://www.rdocumentation.org/packages/fishplot/versions/0.4&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. But I am not able to interpret this plot. How this plot represent tumor evolution and subclones in tumor. Can anybody explain with example. Thanks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hNl5C.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hNl5C.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2141" LastEditorUserId="48" LastEditDate="2019-03-11T10:51:17.567" LastActivityDate="2023-02-27T22:27:32.863" Title="How to interpret fish plot in R" Tags="|cancer|figure-explanation|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7230" PostTypeId="2" ParentId="7229" CreationDate="2019-03-11T14:21:54.040" Score="5" Body="&lt;p&gt;Each color is a clone (set of mutations) or an individual mutation (depends on how the input data was generated). This is a cancer with the bottleneck likely being induced by a treatment (e.g. UV, chemo, drugs, etc...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the vignette this is likely an AML (leukemia).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like grey are healthy cells. At &lt;strong&gt;day 0&lt;/strong&gt; a specific clone (red) existed at a low frequency (~20-40%). The patient (or cells) was then exposed to a treatment and on &lt;strong&gt;day 150&lt;/strong&gt; another measurement was taken. In this case two novel clones (subclones of red) have taken over the population. They seem to co-exist or are co-dominate clones with orange being 50-60% of the population and yellow being 30-40%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The &lt;a href=&quot;https://www.rdocumentation.org/packages/fishplot/versions/0.4&quot; rel=&quot;noreferrer&quot;&gt;example&lt;/a&gt; says &quot;&lt;em&gt;Simple plot with multiple subclones&lt;/em&gt;&quot; so: the orange and yellow populations have additional (likely mutually exclusive) mutations that were not present in the red clone - but biologically also have all the mutations in the red clone. So my first assumption was right - each color is a proxy for a set of mutations.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could have generated the mutation data (i.e. allele frequencies) from a wide variety of sources (scRNA-Seq, exome-sequencing, PCR genotyping, etc...)&lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="383" LastEditDate="2019-03-11T14:26:57.803" LastActivityDate="2019-03-11T14:26:57.803" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7232" PostTypeId="1" CreationDate="2019-03-11T18:02:54.730" Score="1" ViewCount="59" Body="&lt;p&gt;I am running the Picard tool CollectSequencingArtifactMetrics in galaxy (locally in my pc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output files are saved into disk and not passed to &lt;em&gt;stdout&lt;/em&gt; and (I guess) they can't be passed as output from picad to galaxy .  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the xml file I am including:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;outputs&amp;gt;&#xA;    &amp;lt;data name='preSummary' format=&quot;tabular&quot;  from_work_dir=&quot;pref.pre_adapter_summary_metrics&quot;/&amp;gt;&#xA;&amp;lt;/outputs&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;br&gt;&lt;br&gt;&#xA;When I click on &lt;em&gt;View details&lt;/em&gt;, in &lt;em&gt;&quot;stderr&quot;&lt;/em&gt;, I see: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[Mon Mar 11 13:43:07 EDT 2019]&#xA;  picard.analysis.artifacts.CollectSequencingArtifactMetrics done.&#xA;  Elapsed time: 0.19 minutes. Runtime.totalMemory()=3323461632&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;1 - It looks like it worked but shouldn't I see the highlighted information in &lt;em&gt;&quot;stdout&quot;&lt;/em&gt; rather than in the &lt;em&gt;&quot;stderr&quot;&lt;/em&gt;?&lt;br&gt;&#xA;2- If it worked, How can I get my output files? I tracked the working directory printing &lt;code&gt;pwd&lt;/code&gt; from the &lt;code&gt;&amp;lt;command&amp;gt;&lt;/code&gt; but I guess all files are deleted immediately once the picard process has finished.&#xA;&lt;br&gt;&lt;br&gt;&lt;br&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea of what am I doing wrong? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What basic knowledge am I missing here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="2970" LastActivityDate="2019-03-12T08:10:23.107" Title="Including Picard tool in galaxy" Tags="|picard|galaxy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7233" PostTypeId="1" AcceptedAnswerId="7257" CreationDate="2019-03-11T23:00:47.950" Score="1" ViewCount="472" Body="&lt;p&gt;I am working on a SLURM-based cluster. I am currently using the following slurm json file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;    &quot;__default__&quot; :&#xA;    {&#xA;        &quot;A&quot; : &quot;overall&quot;,&#xA;        &quot;time&quot; : &quot;96:00:00&quot;,&#xA;        &quot;nodes&quot;: 1,&#xA;        &quot;ntasks&quot;: 1,&#xA;        &quot;cpus&quot; : 24,&#xA;        &quot;p&quot; : &quot;LONG&quot;,&#xA;        &quot;mem&quot;: 16G,&#xA;        &quot;output&quot;: &quot;snakemake%A.out&quot;,&#xA;        &quot;error&quot;: &quot;snakemake%A.err&quot;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;along with the following command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;snakemake --jobs 200 --use-conda --printshellcmds  --cluster-config slurm.json --cluster &quot;sbatch -A {cluster.A} -p {cluster.p} -t {cluster.time} --output {cluster.output} --error {cluster.error} --ntasks {cluster.ntasks} --cpus-per-task {cluster.cpus} --mem {cluster.mem}&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to be able to specify my email and receive an email when a job finishes. I have tried adding a line in the json file with &lt;code&gt;&quot;email&quot;: my@email.com&lt;/code&gt; and an option to the sbatch command &lt;code&gt;--email {cluster.email}&lt;/code&gt; but that did not work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone knows how to do it?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2019-03-15T14:09:51.257" Title="How can I specify an email address in the SLURM snakemake json configuration file?" Tags="|snakemake|slurm|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7235" PostTypeId="2" ParentId="7232" CreationDate="2019-03-12T08:10:23.107" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;Picard, like most tools, prints logs to stderr.&lt;/li&gt;&#xA;&lt;li&gt;It should be in the Galaxy history. Are you using a version of picard from the toolshed (if so, which one)? You might set &lt;code&gt;cleanup_job&lt;/code&gt; to &lt;code&gt;never&lt;/code&gt; in the config, which will prevent galaxy from cleaning the working directory. This will enable seeing what the job actually produced.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Most likely there's more in the log file that you're not showing is us and picard itself never ran to completion. In lieu of that information, what I wrote above is usually sufficient to debug things.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-03-12T08:10:23.107" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7236" PostTypeId="2" ParentId="7228" CreationDate="2019-03-12T11:03:08.830" Score="3" Body="&lt;p&gt;It seems so, yes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;UCSC uses a 10Kb definition and representation for telomeres in human. In the table you shared this is consistent across all chromosomes, not only chr21. This 10kb representation is an approximation to the mean value observed in newborns and is supported by evidence like the one linked below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Taken from &lt;a href=&quot;https://en.wikipedia.org/wiki/Telomere&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A telomere is a region of repetitive nucleotide sequences at each end of a chromosome, which protects the end of the chromosome from deterioration or from fusion with neighboring chromosomes. &lt;strong&gt;For vertebrates&lt;/strong&gt;, the sequence of nucleotides in telomeres is AGGGTT, with the complementary DNA strand being TCCCAA, with a single-stranded TTAGGG overhang. This sequence of &lt;strong&gt;TTAGGG is repeated approximately 2,500 times in humans. In humans, average telomere length declines from about 11 kilobases at birth[ to less than 4 kilobases in old age&lt;/strong&gt;, with the average rate of decline being greater in men than in women.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2,500 x 6 (TCCCAA) = 15,000&lt;br&gt;&#xA;(approx. 10K)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two interesting additional facts: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://science.sciencemag.org/content/350/6265/1193.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;The minimum telomere length needed to ensure human telomere protective stability in white blood cells is 3.81 kb&lt;/a&gt;&lt;br&gt;&#xA;&lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTables?hgsid=715252179_oJrvVDkmdEEyM8d3cov3mFWLDTRZ&amp;amp;clade=mammal&amp;amp;org=Mouse&amp;amp;db=0&amp;amp;hgta_group=map&amp;amp;hgta_track=gap&amp;amp;hgta_table=gap&amp;amp;hgta_regionType=genome&amp;amp;position=&amp;amp;hgta_outputType=primaryTable&amp;amp;hgta_outFileName=&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mouse has a 100Kb telomere representation in USCS&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="475" LastActivityDate="2019-03-12T11:03:08.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7237" PostTypeId="1" CreationDate="2019-03-12T14:44:26.387" Score="3" ViewCount="341" Body="&lt;p&gt;I'm looking for a tool that is capable to do quality trimming at 5' end and it is configurable. For example, I can choose the quality trheshold, the read length, etc. Any recommendations? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently using sickle for quality trimming. However, one of my samples have very low quality at the beginning as well and sickle doesn't seem to be able to deal well with it. &lt;/p&gt;&#xA;" OwnerUserId="2245" LastEditorUserId="2245" LastEditDate="2019-03-12T16:34:42.313" LastActivityDate="2019-03-22T20:29:29.557" Title="Tools for quality trimming at 5 prime?" Tags="|ngs|illumina|quality-control|trimming|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7238" PostTypeId="2" ParentId="7237" CreationDate="2019-03-12T15:12:58.773" Score="3" Body="&lt;p&gt;I never spent too much time on choosing my trimming software, therefore I might have missed some jewels. I use &lt;a href=&quot;http://www.usadellab.org/cms/?page=trimmomatic&quot; rel=&quot;nofollow noreferrer&quot;&gt;trimmomatic&lt;/a&gt; when I need versatility and I am entirely sure that it trims reads on both ends. A more convenient option is &lt;a href=&quot;https://github.com/relipmoc/skewer&quot; rel=&quot;nofollow noreferrer&quot;&gt;skewer&lt;/a&gt;, they do perform quality filtering, but I could not find an explicit confirmation that it works on 5' too.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2019-03-12T15:12:58.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7239" PostTypeId="2" ParentId="7237" CreationDate="2019-03-12T15:33:32.753" Score="4" Body="&lt;p&gt;You can have a look at &lt;a href=&quot;https://cutadapt.readthedocs.io/en/stable/installation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;cutadapt&lt;/a&gt;. It is capable of quality trimming for both ends as you can read &lt;a href=&quot;https://cutadapt.readthedocs.io/en/stable/guide.html#quality-trimming&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2019-03-12T15:33:32.753" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7240" PostTypeId="1" AcceptedAnswerId="7244" CreationDate="2019-03-12T15:43:05.740" Score="3" ViewCount="1998" Body="&lt;p&gt;I am trying to derive an expression matrix from BAM files using &lt;code&gt;htseq-count&lt;/code&gt;. These are bulk RNASeq BAM's by the way.&lt;/p&gt;&#xA;&lt;p&gt;I have read the &lt;code&gt;htseq-count&lt;/code&gt; documentation as well as &lt;code&gt;samtools&lt;/code&gt; and figured that the following command &lt;em&gt;should&lt;/em&gt; work:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools sort -on file.bam | samtools view -h - | htseq-count --type=exon --idattr=gene_id --additional-attr=gene_name - Homo_sapiens.GRCh38.77.gtf &amp;gt; counts.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;.gtf&lt;/code&gt; file here is from ensembl.&lt;/p&gt;&#xA;&lt;p&gt;I have tried many variations of the above command but have failed to generate a count matrix. The &lt;em&gt;error&lt;/em&gt; message states:&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;0 SAM alignments processed&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Here are the first few lines of my .gtf file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!genome-build GRCh37.p13&#xA;#!genome-version GRCh37&#xA;#!genome-date 2009-02&#xA;#!genome-build-accession NCBI:GCA_000001405.14&#xA;#!genebuild-last-updated 2013-09&#xA;1       pseudogene      gene    11869   14412   .       +       .       gene_id &amp;quot;ENSG00000223972&amp;quot;; gene_name &amp;quot;DDX11L1&amp;quot;;&#xA;gene_source &amp;quot;ensembl_havana&amp;quot;; gene_biotype &amp;quot;pseudogene&amp;quot;;&#xA;1       processed_transcript    transcript      11869   14409   .       +       .       gene_id &amp;quot;ENSG00000223972&amp;quot;; trans&#xA;cript_id &amp;quot;ENST00000456328&amp;quot;; gene_name &amp;quot;DDX11L1&amp;quot;; gene_source &amp;quot;ensembl_havana&amp;quot;; gene_biotype &amp;quot;pseudogene&amp;quot;; transcript_nam&#xA;e &amp;quot;DDX11L1-002&amp;quot;; transcript_source &amp;quot;havana&amp;quot;;&#xA;1       processed_transcript    exon    11869   12227   .       +       .       gene_id &amp;quot;ENSG00000223972&amp;quot;; transcript_id&#xA; &amp;quot;ENST00000456328&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;DDX11L1&amp;quot;; gene_source &amp;quot;ensembl_havana&amp;quot;; gene_biotype &amp;quot;pseudogene&amp;quot;; trans&#xA;cript_name &amp;quot;DDX11L1-002&amp;quot;; transcript_source &amp;quot;havana&amp;quot;; exon_id &amp;quot;ENSE00002234944&amp;quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And all the genes in the text file have an expression value of 0. What exactly is going wrong in this step? Is the conversion to a SAM file the problem? I'm completely lost so any help would me much appreciated.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&lt;p&gt;EDIT:: For anybody who sees this post cause they have a similar problem. Just note that the newer versions of &lt;code&gt;htseq-count&lt;/code&gt; don't require sorted .bam files and, so following the editing of the &lt;code&gt;.gtf&lt;/code&gt; file, all I needed to do was convert it to &lt;code&gt;.sam&lt;/code&gt; using &lt;code&gt;samtools view -h&lt;/code&gt; and then pipe this to &lt;code&gt;htseq-count&lt;/code&gt;. Sorting the files prior to this conversion would cause errors.&lt;/p&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="4001" LastEditDate="2021-03-26T18:39:21.193" LastActivityDate="2021-03-26T18:39:21.193" Title="Running htseq-count over BAM files" Tags="|rna-seq|sam|samtools|bash|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7242" PostTypeId="1" CreationDate="2019-03-12T21:53:50.813" Score="5" ViewCount="399" Body="&lt;p&gt;I'm looking for a tool which, given a truth vcf file and a test vcf file, calculates the phase/switch error rate. I performed phasing of a vcf using &lt;a href=&quot;https://whatshap.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;WhatsHap&lt;/a&gt; and want to compare the outcome to some ground truth phased vcf I have. I can't find a tool, and I don't want to write it myself. It seems like a rather common thing... so I'd expect it to exist, but my google-fu is weak today.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example of test lines:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS     ID      REF     ALT     QUAL    FILTER  INFO    FORMAT  sample&#xA;&#xA;chr10   100588  .       G       A       .       PASS    AF=0.18;AC=1;NS=150;AN=2;EAS_AF=0.14;EUR_AF=0.39;AFR_AF=0.08;AMR_AF=0.31;SAS_AF=0.2;VT=SNP;DP=11907     GT:PS   1|0:48005&#xA;chr10   102385  .       A       G       .       PASS    AF=0.07;AC=1;NS=150;AN=2;EAS_AF=0.02;EUR_AF=0.0;AFR_AF=0.1;AMR_AF=0.19;SAS_AF=0.01;VT=SNP;DP=22665      GT:PS   0|1:48005&#xA;chr10   105170  .       G       A       .       PASS    AF=0.03;AC=1;NS=150;AN=2;EAS_AF=0.02;EUR_AF=0.0;AFR_AF=0.0;AMR_AF=0.19;SAS_AF=0.0;VT=SNP;DP=17140       GT:PS   0|1:48005&#xA;chr10   105365  .       G       T       .       PASS    AF=0.19;AC=1;NS=150;AN=2;EAS_AF=0.14;EUR_AF=0.39;AFR_AF=0.09;AMR_AF=0.31;SAS_AF=0.2;VT=SNP;DP=21845     GT:PS   1|0:48005&#xA;chr10   106057  .       T       C       .       PASS    AF=0.13;AC=1;NS=150;AN=2;EAS_AF=0.02;EUR_AF=0.0;AFR_AF=0.32;AMR_AF=0.19;SAS_AF=0.01;VT=SNP;DP=22559     GT:PS   0|1:48005&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Example of truth lines&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS     ID      REF     ALT     QUAL    FILTER  INFO    FORMAT  sample&#xA;chr10   100554  .       C       CT,CTT  30      .       .       GT:AD   1|2:0,1,1&#xA;chr10   100588  .       G       A       30      .       .       GT:AD   0|1:1,1&#xA;chr10   102385  .       A       G       30      .       .       GT:AD   1|0:1,1&#xA;chr10   102636  .       T       C       30      .       .       GT:AD   1|1:0,2&#xA;chr10   102757  .       T       C       30      .       .       GT:AD   0|1:1,1&#xA;chr10   104815  .       GCC     G,GC    30      .       .       GT:AD   2|1:0,1,1&#xA;chr10   105170  .       G       A       30      .       .       GT:AD   1|0:1,1&#xA;chr10   105365  .       G       T       30      .       .       GT:AD   0|1:1,1&#xA;chr10   106057  .       T       C       30      .       .       GT:AD   1|0:1,1&#xA;chr10   106110  .       C       G       30      .       .       GT:AD   1|1:0,2&#xA;chr10   106261  .       A       G       30      .       .       GT:AD   1|0:1,1&#xA;chr10   108612  .       T       G       30      .       .       GT:AD   1|0:1,1&#xA;chr10   108646  .       A       AT      30      .       .       GT:AD   0|1:1,1&#xA;chr10   108834  .       G       A       30      .       .       GT:AD   1|0:1,1&#xA;chr10   110840  .       C       T       30      .       .       GT:AD   1|1:0,2&#xA;chr10   111743  .       A       AT      30      .       .       GT:AD   1|0:1,1&#xA;chr10   112016  .       T       A       30      .       .       GT:AD   1|0:1,1&#xA;chr10   112262  .       C       T       30      .       .       GT:AD   1|1:0,2&#xA;chr10   113006  .       C       T       30      .       .       GT:AD   0|1:1,1&#xA;chr10   113031  .       G       A       30      .       .       GT:AD   0|1:1,1&#xA;chr10   113136  .       G       A       30      .       .       GT:AD   1|0:1,1&#xA;chr10   113359  .       A       T       30      .       .       GT:AD   1|1:0,2&#xA;chr10   113583  .       C       G       30      .       .       GT:AD   0|1:1,1&#xA;chr10   113997  .       G       C       30      .       .       GT:AD   1|1:0,2&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="681" LastEditorUserId="681" LastEditDate="2019-03-13T13:06:54.753" LastActivityDate="2019-03-13T18:19:08.557" Title="Searching tool to calculate phase/switch error rate" Tags="|vcf|haplotypes|phasing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7244" PostTypeId="2" ParentId="7240" CreationDate="2019-03-13T09:04:52.473" Score="2" Body="&lt;p&gt;If your BAM file is using UCSC chromosome names (chr1, chr2, etc.) and your GTF file is using Ensembl chromosome names (1, 2, etc.) then htseq-count won't know how to match your alignments to any of the genes. The simplest solution is to not use htseq-count, but instead to use featureCounts, which will handle the chromosome name difference transparently (it's one of the few programs that does) and is much faster anyway. In the future, download your GTF and fasta files from the same source, don't mix and match them.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-03-13T09:04:52.473" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7245" PostTypeId="2" ParentId="7242" CreationDate="2019-03-13T13:32:40.913" Score="3" Body="&lt;p&gt;Assuming your files are as you show, and the genotype field is always the 1st entry of the sample information field (the 10th), then you can just do this directly with a simple awk script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk -F&quot;\t&quot; -vOFS=&quot;\t&quot; 'BEGIN{print &quot;#CHROM&quot;,&quot;POS&quot;,&quot;ID&quot;,&quot;REF&quot;,&quot;ALT&quot;,&quot;Truth&quot;,&quot;Test&quot;}{ if(/^[^#]/){var=$1$2$4$5; gt=substr($10,1,3); if(NR==FNR){a[var]=gt; next;} if(var in a &amp;amp;&amp;amp; a[var]!=gt){print $1,$2,$3,$4,$&lt;/span&gt;5,gt,a[var]}}}' truth.vcf test.vcf &#xA;#CHROM  POS     ID  REF ALT Truth   Test&#xA;chr10   100588  .   G   A   1|0 0|1&#xA;chr10   102385  .   A   G   0|1 1|0&#xA;chr10   105170  .   G   A   0|1 1|0&#xA;chr10   105365  .   G   T   1|0 0|1&#xA;chr10   106057  .   T   C   0|1 1|0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's the script split into separate lines for clarity (you can still copy paste directly into a terminal):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -F&quot;\t&quot; -vOFS=&quot;\t&quot; '&#xA;    BEGIN{&#xA;        print &quot;#CHROM&quot;,&quot;POS&quot;,&quot;ID&quot;,&quot;REF&quot;,&quot;ALT&quot;,&quot;Truth&quot;,&quot;Test&quot;&#xA;    }&#xA;    { &#xA;        if(/^[^#]/){&#xA;            var=&lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;2&lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt;5; &#xA;            gt=substr(&lt;span class=&quot;math-container&quot;&gt;$10,1,3); &#xA;            if(NR==FNR){&#xA;                a[var]=gt; &#xA;                next;   &#xA;            } &#xA;            if(var in a &amp;amp;&amp;amp; a[var]!=gt){&#xA;                print $1,$2,$3,$4,$&lt;/span&gt;5,gt,a[var]&#xA;            }&#xA;        }&#xA;    }' truth.vcf test.vcf &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The basic idea is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;awk -F&quot;\t&quot; -vOFS=&quot;\t&quot;&lt;/code&gt;: set the input and output field separators to a tab.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;BEGIN{print &quot;#CHROM ... }&lt;/code&gt;: print a header.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;if(/^[^#]/){&lt;/code&gt;: skip the header lines of the input vcf files&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;var=$1$2$4$5&lt;/code&gt;: get a &quot;signature&quot; for this variant, by concatenating the chromosome, position, ref and alt fields.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;gt=substr($10,1,3);&lt;/code&gt; : get the genotype value.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;if(NR==FNR){&lt;/code&gt;: if this is the 1st file/&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;a[var]=gt; next&lt;/code&gt;: save the genotype for this variant and skip to the next line.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;if(var in a &amp;amp;&amp;amp; a[var]!=gt){&lt;/code&gt;: this part will only be run for the 2nd file because of the 'next' in the previous block. If this variant was in the truth set and if its genotype there isn't the same as in the test set, print the variant and the two genotypes.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If you need the error rate, you can extend the script thusly:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -F&quot;\t&quot; -vOFS=&quot;\t&quot; '&#xA;BEGIN{&#xA;    print &quot;#CHROM&quot;,&quot;POS&quot;,&quot;ID&quot;,&quot;REF&quot;,&quot;ALT&quot;,&quot;Truth&quot;,&quot;Test&quot;&#xA;}&#xA;{ &#xA;    if(/^[^#]/){&#xA;        var=&lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;2&lt;span class=&quot;math-container&quot;&gt;$4$&lt;/span&gt;5; &#xA;        gt=substr(&lt;span class=&quot;math-container&quot;&gt;$10,1,3); &#xA;        if(NR==FNR){&#xA;            a[var]=gt; &#xA;            next;   &#xA;        }&#xA;        tot++; &#xA;        if(var in a &amp;amp;&amp;amp; a[var]!=gt){&#xA;            bad++;&#xA;            print $1,$2,$3,$4,$&lt;/span&gt;5,gt,a[var]&#xA;        }&#xA;    }&#xA;}&#xA;END{&#xA;    printf &quot;Correct: %s of %s (%s%)\n&quot;,tot-bad,tot,(tot-bad)*100/tot;&#xA;    printf &quot;Wrong: %s of %s (%s%)\n&quot;,bad,tot,bad*100/tot;&#xA;}' truth.vcf test.vcf &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;On your example dataset, that prints:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT Truth   Test&#xA;chr10   100588  .   G   A   1|0 0|1&#xA;chr10   102385  .   A   G   0|1 1|0&#xA;chr10   105170  .   G   A   0|1 1|0&#xA;chr10   105365  .   G   T   1|0 0|1&#xA;chr10   106057  .   T   C   0|1 1|0&#xA;Correct: 0 of 5 (0%)&#xA;Wrong: 5 of 5 (100%)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-03-13T18:19:08.557" LastActivityDate="2019-03-13T18:19:08.557" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7246" PostTypeId="1" CreationDate="2019-03-13T20:13:53.963" Score="3" ViewCount="266" Body="&lt;p&gt;I'm trying to figure out how to do a DMP analysis (using minfi dmpFinder) on a related sample (if it's even possible). Right now the code (not written by me) is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dmps = dmpFinder(mVals, &#xA;                 pheno = targets1&lt;span class=&quot;math-container&quot;&gt;$X8yrfactor, &#xA;                 type = &quot;continuous&quot;)&#xA;pheno.exists &amp;lt;- !is.na(targets1$&lt;/span&gt;X8yrfactor) &#xA;pheno &amp;lt;- targets1&lt;span class=&quot;math-container&quot;&gt;$X8yrfactor[pheno.exists] &#xA;mVal.phenoNotNA &amp;lt;- mVals[, pheno.exists]&#xA;dmps = dmpFinder(mVal.phenoNotNA, pheno = pheno, type = &quot;continuous&quot;)&#xA;dmps$&lt;/span&gt;probe = paste(rownames(dmps))&#xA;rownames(dmps) = NULL &#xA;dmps = dmps[,c(6,1:5)] &#xA;write.csv(dmps, file = &amp;quot;dmps_8yr_minfi.csv&amp;quot;, row.names = F)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The sample size is 96, i.e. 48 twin pairs, which is why we want to be able to control for relatedness in the sample. By &amp;quot;twin pairs&amp;quot;, I mean the methylation data we're using is from a sample of 48 pairs of identical human twins. At the moment I believe we're only going to be testing one at a time if we do find a way to test them on this sample.&lt;/p&gt;&#xA;&lt;p&gt;Is there anyway to code it similarly to a mixed linear model where a randomization variable for each related individual is taken into account? I've been referred/found &lt;a href=&quot;https://jdblischak.github.io/singleCellSeq/analysis/batch-limma.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;, &lt;a href=&quot;https://support.bioconductor.org/p/29768/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/4203/hierarchical-models-with-limma&quot;&gt;this&lt;/a&gt; resource but I'm not sure they capture what I need. Correct me if I'm wrong.&lt;/p&gt;&#xA;&lt;p&gt;A simple linear model will not do this, which is why we're trying to figure out how to do this similarly to a mixed model. 'dmpFinder' identifies CpGs where methylation is associated with a continuous or categorical phenotype ('pheno', which we have multiple we want to test, we're trying to write the code first). 'targets1' is our cases (3 of them) that were removed following QC.&lt;/p&gt;&#xA;" OwnerUserId="4252" LastEditorUserId="73" LastEditDate="2021-06-25T11:20:44.310" LastActivityDate="2021-06-25T11:20:44.310" Title="Differentially methylated position analysis in a related sample?" Tags="|r|bioconductor|microarray|methylation|limma|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7247" PostTypeId="2" ParentId="7218" CreationDate="2019-03-14T11:57:01.997" Score="1" Body="&lt;p&gt;If ENA archive contains such information, then you can access it programatically through the &lt;a href=&quot;https://www.ebi.ac.uk/ena/browse/programmatic-access&quot; rel=&quot;nofollow noreferrer&quot;&gt;EBI api&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also you can access the NCBI database programatically through their &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK25500/&quot; rel=&quot;nofollow noreferrer&quot;&gt;eutils&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In both cases you can access annotations, though not sure if they contain what you are looking for in all the cases.&lt;/p&gt;&#xA;" OwnerUserId="2245" LastActivityDate="2019-03-14T11:57:01.997" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7248" PostTypeId="1" CreationDate="2019-03-14T12:11:36.657" Score="1" ViewCount="169" Body="&lt;p&gt;I have been extracted number of abstracts from pubmed. now I am having a bunch of abstracts of pubmed articles now from this abstracts I need to extract gene names which are included in abstracts.&lt;/p&gt;&#xA;" OwnerUserId="4258" LastEditorUserId="96" LastEditDate="2019-03-14T13:59:51.020" LastActivityDate="2019-03-14T13:59:51.020" Title="extracting gene names from pubmed abstracts" Tags="|python|text-processing|text-mining|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7250" PostTypeId="1" CreationDate="2019-03-14T14:25:25.203" Score="2" ViewCount="174" Body="&lt;p&gt;I have WGS .bam files for 3 patients (tumour and its matched derived model namely organoid) but I don't matched normal sample. If I call variants of each patients (tumour and its matched organoid), how I can use read counts at germline heterozygous positions estimated by GATK 3.2-2 to compensate for the absence of matched normal sample? I heard people use dbsnp VCF instead of the matched normal small variant VCF. But, I don't know start from where? which GATK function does that?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I should mention, Calling SNV and indel in many tools returns vcf but&#xA;  I called copy number by varscan that did not return .vcf output so I&#xA;  am not sure what to do for CNV&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Any helps please?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-03-14T17:20:43.327" LastActivityDate="2019-03-27T14:53:39.453" Title="Variant calling without matched normal sample" Tags="|sam|snp|gatk|wgs|vcftools|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7251" PostTypeId="2" ParentId="7250" CreationDate="2019-03-14T14:42:15.717" Score="6" Body="&lt;p&gt;If you just want to filter out calls present in dbSNP then use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar \&#xA;   -T SelectVariants \&#xA;   -R reference.fasta \&#xA;   -V patient.vcf \&#xA;   --discordance dbSNP.vcf \&#xA;   -o patient.filtered.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;--discordance&lt;/code&gt; will produce calls not present in dbSNP.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2019-03-14T14:56:20.050" LastActivityDate="2019-03-14T14:56:20.050" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7252" PostTypeId="1" CreationDate="2019-03-14T17:28:37.063" Score="0" ViewCount="42" Body="&lt;p&gt;I have RNA-seq .bam files for 3 patients, tumour and its &lt;strong&gt;matched derived model&lt;/strong&gt;, namely &lt;strong&gt;organoid&lt;/strong&gt;, but I don't have any matched normal sample. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Differentially expressed genes between a tumour and its model mainly come from the presence and lack of &lt;strong&gt;microenvironment&lt;/strong&gt; in cancers and organoids respectively and response to stimuli (artificial environment in organoid culture vs cancer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How I can get the differentially expressed genes specific to microenvironment?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any intuition please?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="48" LastEditDate="2019-03-15T15:45:55.803" LastActivityDate="2019-03-15T15:50:11.347" Title="Finding genes especific to microenvironment" Tags="|rna-seq|deseq2|cancer|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7255" PostTypeId="1" CreationDate="2019-03-15T04:08:43.333" Score="3" ViewCount="567" Body="&lt;p&gt;I get the following error when I try to load a .chip file to &lt;a href=&quot;https://www.gsea-msigdb.org/gsea/index.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSEA&lt;/a&gt; software. What is the problem?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;---- Full Error Message ----&#xA;There were errors: ERROR(S) #:1&#xA;Parsing trouble&#xA;java.lang.NumberFormatException: ...&#xA;&#xA;---- Stack Trace ----&#xA;# of exceptions: 1&#xA;------For input string: &amp;quot;YBL028C&amp;quot;------&#xA;java.lang.NumberFormatException: For input string: &amp;quot;YBL028C&amp;quot;&#xA;    at sun.misc.FloatingDecimal.readJavaFormatString(Unknown Source)&#xA;    at sun.misc.FloatingDecimal.parseFloat(Unknown Source)&#xA;    at java.lang.Float.parseFloat(Unknown Source)&#xA;    at edu.mit.broad.genome.parsers.TxtDatasetParser._parseNoDesc(TxtDatasetParser.java:169)&#xA;    at edu.mit.broad.genome.parsers.TxtDatasetParser.parse(TxtDatasetParser.java:131)&#xA;    at edu.mit.broad.genome.parsers.TxtDatasetParser.parse(TxtDatasetParser.java:87)&#xA;    at edu.mit.broad.genome.parsers.ParserFactory.readDatasetTXT(ParserFactory.java:202)&#xA;    at edu.mit.broad.genome.parsers.ParserFactory.read(ParserFactory.java:749)&#xA;    at edu.mit.broad.genome.parsers.ParserFactory.read(ParserFactory.java:726)&#xA;    at edu.mit.broad.genome.parsers.ParserWorker.doInBackground(ParserWorker.java:52)&#xA;    at javax.swing.SwingWorker&lt;span class=&quot;math-container&quot;&gt;$1.call(Unknown Source)&#xA;    at java.util.concurrent.FutureTask.run(Unknown Source)&#xA;    at javax.swing.SwingWorker.run(Unknown Source)&#xA;    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)&#xA;    at java.util.concurrent.ThreadPoolExecutor$&lt;/span&gt;Worker.run(Unknown Source)&#xA;    at java.lang.Thread.run(Unknown Source)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yzSaF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yzSaF.png&quot; alt=&quot;Here is the first few lines of the table I am using &quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am also using GSEA 3.0. What does not make sense is the other files have the right extensions and are not getting this error, but only this file is failing to upload.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.gsea-msigdb.org/gsea/index.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.gsea-msigdb.org/gsea/index.jsp&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Gene Set Enrichment Analysis (GSEA) is a computational method that determines whether an a priori defined set of genes shows statistically&#xA;significant, concordant differences between two biological states&#xA;(e.g. phenotypes). &amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="4266" LastEditorUserId="3848" LastEditDate="2022-04-11T11:19:29.193" LastActivityDate="2023-06-05T00:01:40.280" Title="I am getting a parsing error when uploading .chip file to GSEA software" Tags="|errors|gsea|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7256" PostTypeId="1" AcceptedAnswerId="7260" CreationDate="2019-03-15T11:37:26.007" Score="1" ViewCount="59" Body="&lt;p&gt;I am trying to calculate the ΔG of TM insertion for many protein sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone come across the source code or a programmatic version for the von Heijne ΔG for TM insertion? The web server is at &lt;a href=&quot;http://dgpred.cbr.su.se/index.php?p=instructions&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dgpred.cbr.su.se/index.php?p=instructions&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't find a link in the papers or on the site.&lt;/p&gt;&#xA;" OwnerUserId="747" LastEditorUserId="747" LastEditDate="2019-03-15T12:23:37.510" LastActivityDate="2019-03-15T16:33:03.960" Title="Download program for ΔG for TM insertion" Tags="|proteins|sequence-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7257" PostTypeId="2" ParentId="7233" CreationDate="2019-03-15T14:09:51.257" Score="4" Body="&lt;p&gt;As noted by the comments and the SLURM manual (not so much bioinformatics related?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need the following terms in your batch/queuing command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;--mail-type=ALL&#xA;--mail-user=you@email.com&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I used &quot;ALL&quot; which will send you &lt;em&gt;all&lt;/em&gt; reports. If you just want the finish message then you would use &quot;END&quot; and &quot;FAIL&quot; like such:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;--mail-type=END,FAIL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here are the specifics from the &lt;a href=&quot;https://slurm.schedmd.com/sbatch.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt; if you wish to make a different selection:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;--mail-type=&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Notify user by email when certain event types occur. Valid type values&#xA;  are NONE, BEGIN, END, FAIL, REQUEUE, ALL (equivalent to BEGIN, END,&#xA;  FAIL, REQUEUE, and STAGE_OUT), STAGE_OUT (burst buffer stage out and&#xA;  teardown completed), TIME_LIMIT, TIME_LIMIT_90 (reached 90 percent of&#xA;  time limit), TIME_LIMIT_80 (reached 80 percent of time limit),&#xA;  TIME_LIMIT_50 (reached 50 percent of time limit) and ARRAY_TASKS (send&#xA;  emails for each array task). Multiple type values may be specified in&#xA;  a comma separated list. The user to be notified is indicated with&#xA;  --mail-user. Unless the ARRAY_TASKS option is specified, mail notifications on job BEGIN, END and FAIL apply to a job array as a&#xA;  whole rather than generating individual email messages for each task&#xA;  in the job array.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;--mail-user=&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;User to receive email notification of state changes as defined by&#xA;  --mail-type. The default value is the submitting user.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="383" LastActivityDate="2019-03-15T14:09:51.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7258" PostTypeId="1" AcceptedAnswerId="7277" CreationDate="2019-03-15T15:31:04.723" Score="1" ViewCount="132" Body="&lt;p&gt;Related to &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/4408/how-to-filter-for-caf-0-5747-0-4253&quot;&gt;this other question&lt;/a&gt;&#xA;but in this case the CAF is defined as &quot;An ordered, comma delimited list of allele frequencies...&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried using &lt;a href=&quot;http://lindenb.github.io/jvarkit/VcfFilterJdk.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;vcffilterjdk&lt;/a&gt; and the command suggested by Pierre &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar dist/vcffilterjdk.jar -e 'variant.variant.getAttributeAsStringList(&quot;CAF&quot;,&quot;1.0&quot;).stream().skip(1L).map(A-&amp;gt;Double.parseDouble(A)).anyMatch(A-&amp;gt;A&amp;lt;0.01);' input.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but predictably returned an error &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java.lang.NumberFormatException: For input string: &quot;0.1]&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't know javascript so how should I change the command to make it work?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;HEADER&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##INFO=&amp;lt;ID=CAF,Number=.,Type=String,Description=&quot;An ordered, comma delimited list of allele frequencies based on 1000Genomes, starting with the reference allele followed by alternate alleles as ordered in the ALT column. Where a 1000Genomes alternate allele is not in the dbSNPs alternate allele set, the allele is added to the ALT column.  The minor allele is the second largest value in the list, and was previuosly reported in VCF as the GMAF.  This is the GMAF reported on the RefSNP and EntrezSNP pages and VariationReporter&quot;&amp;gt;  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;LINE&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    51479   rs116400033     T       A       .       PASS    ASP;CAF=[0.8926,0.1074];COMMON=1;G5;G5A;KGPROD;KGPhase1;OTHERKG;RS=116400033;RSPOS=51479;SAO=0;SSR=0;VC=SNV;VLD;VP=0x050000000005170016000100;WGT=1;dbSNPBuildID=132&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4269" LastEditorUserId="96" LastEditDate="2019-03-18T17:42:18.177" LastActivityDate="2019-03-18T17:42:18.177" Title="Filter VCF for &quot;CAF=[0.9,0.1]&quot;" Tags="|vcf|file-formats|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7259" PostTypeId="2" ParentId="7252" CreationDate="2019-03-15T15:50:11.347" Score="1" Body="&lt;p&gt;The genes related to the environment can be identified using a linear model where each organoid is compared to the correspondent original tumor. This is assuming that the environment for all the organoids is the same and that the differences between tumors is just from different patient but are from the same organ/region.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2019-03-15T15:50:11.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7260" PostTypeId="2" ParentId="7256" CreationDate="2019-03-15T16:01:20.183" Score="2" Body="&lt;p&gt;The application is stored in &lt;a href=&quot;https://github.com/ElofssonLab/dgpred&quot; rel=&quot;nofollow noreferrer&quot;&gt;github&lt;/a&gt; (ElofssonLab/dgpred) with a link to the website. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To find it I searched for: dgpred download github  &lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2019-03-15T16:33:03.960" LastActivityDate="2019-03-15T16:33:03.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7261" PostTypeId="1" AcceptedAnswerId="7264" CreationDate="2019-03-15T18:15:24.740" Score="2" ViewCount="86" Body="&lt;p&gt;In a bioinformatics programming course I am currently teaching (using Python, and approached from a mostly CS rather than a biology background) I assigned a project which involves finding sites where restriction enzymes bind to a DNA sequence. The project is heavily influenced by Chapter 9 of the now classic text &quot;Beginning Perl for Bioinformatics&quot; by James Tisdall. I am using the REBASE database, in particular &lt;a href=&quot;http://rebase.neb.com/rebase/rebase.f9.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;REBASE Format #9&lt;/a&gt;. In this format, each restriction enzyme has a recognition sequence which looks like e.g. &lt;code&gt;C^YCGRG&lt;/code&gt; where &lt;code&gt;^&lt;/code&gt; gives information about the exact cut-point and upper-case letters such as &lt;code&gt;Y&lt;/code&gt; and &lt;code&gt;R&lt;/code&gt; other than &lt;code&gt;ACGT&lt;/code&gt; are ambiguity codes. The point of the project is to turn these into regular expressions where cleavage information is discarded and the ambiguity codes are expanded into character classes, and then use these regular expressions to search for matches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All fairly straightforward, but one very observant student noticed that some enzymes had multiple sequences, for example &lt;code&gt;M.BssHIII&lt;/code&gt; has a total of 5 sequences: &lt;code&gt;ACGCGT,CCGCGG,RGCGCY,RCCGGY,GCGCGC&lt;/code&gt;. It is easy to see that these correspond to a total of 11 sequences (after the Rs and Ys are expanded), far fewer than the 144 sequences which would match &lt;code&gt;VSSSSB&lt;/code&gt; (which is the least ambiguous single sequence capable of matching the above).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, what is going on here? Am I safe in simply replacing &lt;code&gt;,&lt;/code&gt; by &lt;code&gt;|&lt;/code&gt; (for &quot;or&quot;) in the regular expression, or is there something more subtle going on? Only 8 of the nearly 5000 enzymes have such multiple sequences, and neither the Tisdall book nor even the description of the format (both of which seem to assume single sequences) shed any light on the situation.&lt;/p&gt;&#xA;" OwnerUserId="4115" LastEditorUserId="3848" LastEditDate="2023-07-22T19:34:44.053" LastActivityDate="2023-07-22T19:34:44.053" Title="Restriction enzymes with multiple recognition sequences" Tags="|phylogenetics|phylogeny|database|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7262" PostTypeId="1" AcceptedAnswerId="7348" CreationDate="2019-03-15T19:08:37.170" Score="2" ViewCount="397" Body="&lt;p&gt;&lt;strong&gt;Summary:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm writing an automated tool for designing Sanger sequencing primers (other tools don't do &lt;em&gt;exactly&lt;/em&gt; what I want) that will be incorporated in a production diagnostic laboratory for on-demand NGS variant verification. I'm stuck on a problem wherein off-target BLAST hits found by NCBI's Primer-BLAST aren't showing up in my programmatic BLAST query of the same genome assembly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Background:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After each iteration of design with primer3, I'm using &lt;code&gt;blastn -task blastn&lt;/code&gt; to search a local hg38 (patch 12) database for off-target hits. I have logic that does round-robin pairwise comparisons (i.e. compares every hit to every other hit) to determine if each hit pair could result in an amplicon (on same chromosome, opposite-stranded, perfect match at 3'-ends, plus-strand hit is upstream of minus-strand hit, are separated by &amp;lt;= 1000 bp). If my logic finds a viable off-target amplicon, it triggers a total re-design with different design parameters. If no off-targets are found, those primers are finalized, so it's important that I find any and all off-targets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make sure that code for viable off-target amplicons is working correctly (that it's triggering re-design or reporting a design failure when appropriate), I'm using NCBI's browser-based Primer-BLAST to verify the absence of viable off-targets, which also uses genome build hg38 patch 12.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The problem:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Primer-BLAST is finding hits that I can't find with my code. Even when I look at every hit returned from the &lt;code&gt;blastn&lt;/code&gt; query, I can't find hits that Primer-BLAST is finding. I would like to know if there is some parameter that I am missing, or parameters I could use in addition to those I'm already using, to produce the hits I am getting from Primer-BLAST.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe my blast database an my query fasta file are built correctly, because I am getting logical output from the query below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Programmatic BLAST Query Details:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Query file (query.fasta):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;for_primer&#xA;GGATCTGCCTCCTTGCC&#xA;&amp;gt;rev_primer&#xA;CCCAACAGGGCCCTC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;BLASTn Query Statement:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blastn -db blastn -db /path/to/hg38.fasta -evalue 100 -num_threads 8 -outfmt 6 std gaps nident -dust no -word_size 7 -query /path/to/query.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The BLAST database used hg38.fa.gz (which I believe to be patch 12) from &lt;a href=&quot;http://hgdownload.soe.ucsc.edu/goldenPath/hg38/bigZips/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. That was unzipped and processed with &lt;code&gt;makeblastdb&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Primer-BLAST Results:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Primer-BLAST results are &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/tools/primer-blast/primertool.cgi?ctg_time=1552618788&amp;amp;job_key=QEqf-GL6b1JIbHVpeAlRWwISQGkvAVt0Lg&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. To re-create, use the template DNA below, search database = &quot;Genomes for selected organisms...&quot;, organism = &lt;em&gt;Homo sapiens&lt;/em&gt;, max target size = 1000. The template DNA is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; GAGGAGTCAGGCCTGATCTGAAGGCAGTGGGGTACCCTCTAGGGTGTCAGCAGGGCTGTGATGTCACCAGTCTTGGCTGCAGTGTGGATTAGGTAGGCGGAGGCTGGGAGGCAGTGGGAAGTGGCCGCAGGGACCGTGTGGTCAGTCAGGGTGGTGCCATTTCCTGTGTGCACACTCGGGGGCTCCCGCCACACCAGGGTGCAGACCACCCCGCCGCTCTCCCGCCAGGCAGCCTGCAGGGTGAAGGACTGACATGGCGGGAGGAGTGGCCTTCTCTGTTGGGGCGCCTGTGGCCCCCCAGCTCTTTGATAGGCAGAGCTGCAGGATCTGCCTCCTTGCCCCTGTGAGCCCAGCAGTGATTCTTGTGTCCTCCCCAGATCCATGGCTGCGGTGAGAGGGCTCGCGCCCATGCCCGGCAGAGAATCAGCCGGGAAGGGGTCCTCTATGCCGGCAGCGGGACCAAGAACGGATCCCTGGACCCAGCCAAGAGGGCCCAGCTGCAGAGGAGGCTGGATAAGAAGCTGAGTGAGCTCAGCAACCAGAGGACCAGCCGGAGGAAGGAGAGGGGGACGTGACCGGCCGCATCCTTGCACGCCCCGCGGAGCTCTCTCCATGGTAGCCCAGGGCGCTGGCAGACCATGCTCCGCCTCCACCAGGGCCACAGAGGAGCGGAGGGGCCTATGGGGGAGGAGCGGAGGGCCCTGTTGGGGAAGGTTGGGTTTTTGGACCCCAGGGATAAGCTTTTCCGATGTCACAATGTGGAGGAAAGCACCTGGGGGACAACAGTGCTCGTGCAGGTGGGGCTTGGGAAATGCACGTCCCTTCCCCTTACTCCCCGCCAAAACCCACATCCCAGCCTCTGGATCCTGGGGAAGGTTCCAGTCCCTGGAGAATACCCAGGGCCTCAAACTTGAAGTCACTCCTCCAATGTCTGGGACTTGCCAGCTCAGCCCGTTAGGATGAGGGTGCTGAGAGGAAACAGGAAACAAGACTGCGAATGGCG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;blastn&lt;/code&gt; output from the query above does not contain either of these two Primer-BLAST hits for 'rev_primer' that comprise a potential amplicon from chromosome 11:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;product length = 110&#xA;Features flanking this product:&#xA;   12759 bp at 5' side: neurexin-2-beta isoform X3&#xA;   735 bp at 3' side: RAS guanyl-releasing protein 2 isoform X5&#xA;&#xA;Reverse primer  1         CCCAACAGGGCCCTC  15&#xA;Template        64726566  AGT............  64726552&#xA;&#xA;Reverse primer  1         CCCAACAGGGCCCTC  15&#xA;Template        64726457  ....G..........  64726471&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Why aren't either of these two hits returned in my &lt;code&gt;blastn&lt;/code&gt; query? Thanks in advance for helping me with this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I noticed in IGV that off-target sequence above is lower-case in the hg38 reference genome. I've tried passing &lt;code&gt;-soft_masking false&lt;/code&gt; to blastn, but got the same result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought maybe the problem was due to my using hg38 from UCSC instead of that from NCBI, so I downloaded hg19 from &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/001/405/GCA_000001405.15_GRCh38&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt; and made the blast database from that file. Same result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then unmasked the hg38.fna file by converting all lowercase nt to uppercase, as recommended in the NCBI genome assemblies &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/001/405/GCA_000001405.15_GRCh38/README.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;README.txt&lt;/a&gt;, created a BLAST database from that and ran my code again using the unmasked database. Same result, still am not detecting the off-target hit above.&lt;/p&gt;&#xA;" OwnerUserId="3564" LastEditorUserId="3564" LastEditDate="2019-03-19T18:36:58.137" LastActivityDate="2019-03-28T22:49:50.650" Title="Hits in Primer-BLAST not found with programmatic blastn query" Tags="|blast|primer|makeblastdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7263" PostTypeId="2" ParentId="7026" CreationDate="2019-03-15T21:42:56.900" Score="3" Body="&lt;p&gt;You can get the table that is used to make the dot plot if you modify the DotPlot function to return it instead of the ggplot, and use the argument do.return=T.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To edit the function, the command is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;trace(&quot;DotPlot&quot;,edit=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then replace the last line &quot;return(p)&quot; by &quot;return(data.to.plot)&quot; and save the edit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can call the function and store the return in a variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The editing doesn't affect the source code, so the function will be untouched at your next R session.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best,&lt;/p&gt;&#xA;" OwnerUserId="4272" LastActivityDate="2019-03-15T21:42:56.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7264" PostTypeId="2" ParentId="7261" CreationDate="2019-03-15T23:04:32.940" Score="3" Body="&lt;p&gt;I would honestly just remove the non-homologous cutting site enzymes from the database. Firstly, these are biology students and require a simple picture to grasp the concept of coding via regex pattern matching. Secondly, in a wet-lab nobody uses these enzymes, I was not even aware the &quot;8&quot; existed until your post. In a teaching lab the best motto is &quot;keep it simple&quot;.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-03-15T23:04:32.940" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7265" PostTypeId="1" AcceptedAnswerId="7267" CreationDate="2019-03-16T03:22:52.247" Score="7" ViewCount="3211" Body="&lt;p&gt;I would like to know what is meant by &quot;seed&quot; for various sequence aligners. How is it important?&lt;/p&gt;&#xA;" OwnerUserId="2040" LastEditorUserId="96" LastEditDate="2019-03-26T18:44:34.070" LastActivityDate="2019-03-26T18:44:34.070" Title="Definition of &quot;seed&quot; in sequence alignment" Tags="|sequence-alignment|read-mapping|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7267" PostTypeId="2" ParentId="7265" CreationDate="2019-03-16T12:52:40.310" Score="12" Body="&lt;p&gt;The seed is the subset of a read used in the first step of an alignment. Many aligners work by a seed-and-extend model, wherein they first find all regions matching the &quot;seed&quot; and then extend the alignment around that allowing mistmatches and indels until it either gives up (and therefore uses a different seed) or finds a sufficiently good alignment.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-03-16T12:52:40.310" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7268" PostTypeId="1" CreationDate="2019-03-17T07:02:06.307" Score="2" ViewCount="642" Body="&lt;p&gt;I am trying to visualize a GSEA analysis I conducted using the Broad Institute's GSEA Enrichment Map Visualization, however I get the following error and am not sure what the issue is: any help would be greatly appreciated. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;INFO - successfully connected to cytoscape rest&#xA;INFO - successfully connected to cytoscape rest&#xA;INFO - Found enrichment map command&#xA;INFO - Get URL:http://localhost:1234/v1/commands/enrichmentmap/gseabuild?edbdir=C%3A%5CUsers%5CHatim%5Cgsea_home%5Coutput%5Cmar15%5Cmy_analysis.Gsea.1552686798457%5Cedb&amp;amp;pvalue=0.005&amp;amp;qvalue=0.1&amp;amp;overlap=0.5&amp;amp;similaritymetric=OVERLAP&amp;amp;combinedconstant=0.5&amp;amp;expressionfile=C%3A%5CUsers%5CHatim%5CDesktop%5CJunior%5Csem+2%5CKarras+Lab%5CGSEA%5CKarrasEXP.gct.txt&#xA;DEBUG - CookieSpec selected: default&#xA;DEBUG - Auth cache not set in the context&#xA;DEBUG - Connection request: [route: {}-&amp;gt;http://localhost:1234][total kept alive: 0; route allocated: 0 of 2; total allocated: 0 of 20]&#xA;DEBUG - Connection leased: [id: 4][route: {}-&amp;gt;http://localhost:1234][total kept alive: 0; route allocated: 1 of 2; total allocated: 1 of 20]&#xA;DEBUG - Opening connection {}-&amp;gt;http://localhost:1234&#xA;DEBUG - Connecting to localhost/127.0.0.1:1234&#xA;DEBUG - Connection established 127.0.0.1:51968&amp;lt;-&amp;gt;127.0.0.1:1234&#xA;DEBUG - Executing request GET /v1/commands/enrichmentmap/gseabuild?edbdir=C%3A%5CUsers%5CHatim%5Cgsea_home%5Coutput%5Cmar15%5Cmy_analysis.Gsea.1552686798457%5Cedb&amp;amp;pvalue=0.005&amp;amp;qvalue=0.1&amp;amp;overlap=0.5&amp;amp;similaritymetric=OVERLAP&amp;amp;combinedconstant=0.5&amp;amp;expressionfile=C%3A%5CUsers%5CHatim%5CDesktop%5CJunior%5Csem+2%5CKarras+Lab%5CGSEA%5CKarrasEXP.gct.txt HTTP/1.1&#xA;DEBUG - Target auth state: UNCHALLENGED&#xA;DEBUG - Proxy auth state: UNCHALLENGED&#xA;DEBUG - http-outgoing-4 &amp;gt;&amp;gt; GET /v1/commands/enrichmentmap/gseabuild?edbdir=C%3A%5CUsers%5CHatim%5Cgsea_home%5Coutput%5Cmar15%5Cmy_analysis.Gsea.1552686798457%5Cedb&amp;amp;pvalue=0.005&amp;amp;qvalue=0.1&amp;amp;overlap=0.5&amp;amp;similaritymetric=OVERLAP&amp;amp;combinedconstant=0.5&amp;amp;expressionfile=C%3A%5CUsers%5CHatim%5CDesktop%5CJunior%5Csem+2%5CKarras+Lab%5CGSEA%5CKarrasEXP.gct.txt HTTP/1.1&#xA;DEBUG - http-outgoing-4 &amp;gt;&amp;gt; Host: localhost:1234&#xA;DEBUG - http-outgoing-4 &amp;gt;&amp;gt; Connection: Keep-Alive&#xA;DEBUG - http-outgoing-4 &amp;gt;&amp;gt; User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_201)&#xA;DEBUG - http-outgoing-4 &amp;gt;&amp;gt; Accept-Encoding: gzip,deflate&#xA;DEBUG - http-outgoing-4 &amp;gt;&amp;gt; &quot;GET /v1/commands/enrichmentmap/gseabuild?edbdir=C%3A%5CUsers%5CHatim%5Cgsea_home%5Coutput%5Cmar15%5Cmy_analysis.Gsea.1552686798457%5Cedb&amp;amp;pvalue=0.005&amp;amp;qvalue=0.1&amp;amp;overlap=0.5&amp;amp;similaritymetric=OVERLAP&amp;amp;combinedconstant=0.5&amp;amp;expressionfile=C%3A%5CUsers%5CHatim%5CDesktop%5CJunior%5Csem+2%5CKarras+Lab%5CGSEA%5CKarrasEXP.gct.txt HTTP/1.1[\r][\n]&quot;&#xA;DEBUG - http-outgoing-4 &amp;gt;&amp;gt; &quot;Host: localhost:1234[\r][\n]&quot;&#xA;DEBUG - http-outgoing-4 &amp;gt;&amp;gt; &quot;Connection: Keep-Alive[\r][\n]&quot;&#xA;DEBUG - http-outgoing-4 &amp;gt;&amp;gt; &quot;User-Agent: Apache-HttpClient/4.5.2 (Java/1.8.0_201)[\r][\n]&quot;&#xA;DEBUG - http-outgoing-4 &amp;gt;&amp;gt; &quot;Accept-Encoding: gzip,deflate[\r][\n]&quot;&#xA;DEBUG - http-outgoing-4 &amp;gt;&amp;gt; &quot;[\r][\n]&quot;&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; &quot;HTTP/1.1 500 Internal Server Error[\r][\n]&quot;&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; &quot;Date: Sun, 17 Mar 2019 06:55:04 GMT[\r][\n]&quot;&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; &quot;Content-Type: text/plain[\r][\n]&quot;&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; &quot;Access-Control-Allow-Origin: *[\r][\n]&quot;&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; &quot;Access-Control-Allow-Headers: origin, content-type, accept, authorization[\r][\n]&quot;&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; &quot;Access-Control-Allow-Credentials: true[\r][\n]&quot;&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; &quot;Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS, HEAD[\r][\n]&quot;&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; &quot;Content-Length: 20[\r][\n]&quot;&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; &quot;Server: Jetty(9.4.10.v20180503)[\r][\n]&quot;&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; &quot;[\r][\n]&quot;&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; &quot;Failed: empty String&quot;&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; HTTP/1.1 500 Internal Server Error&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; Date: Sun, 17 Mar 2019 06:55:04 GMT&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; Content-Type: text/plain&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; Access-Control-Allow-Origin: *&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; Access-Control-Allow-Headers: origin, content-type, accept, authorization&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; Access-Control-Allow-Credentials: true&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS, HEAD&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; Content-Length: 20&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; Server: Jetty(9.4.10.v20180503)&#xA;DEBUG - Connection can be kept alive indefinitely&#xA;INFO - status:Internal Server Error&#xA;INFO - Unable to create Enrichment Map:Internal Server ErrorResponseEntityProxy{[Content-Type: text/plain,Content-Length: 20,Chunked: false]}&#xA;DEBUG - Connection manager is shutting down&#xA;DEBUG - http-outgoing-4: Close connection&#xA;DEBUG - Connection manager shut down&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4266" LastActivityDate="2023-09-08T05:01:35.227" Title="I get the following error when using the GSEA software 'Enrichment Map Visualization' tool using the Cytoscape 3.7.1 application" Tags="|errors|gsea|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7269" PostTypeId="1" AcceptedAnswerId="7272" CreationDate="2019-03-18T00:19:08.267" Score="1" ViewCount="48" Body="&lt;p&gt;How can I get the dataset for all the 43 metabolic networks for this paper:&#xA;&lt;a href=&quot;https://www.nature.com/articles/35036627&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/35036627&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like the WIT database does not exist anymore.&lt;/p&gt;&#xA;" OwnerUserId="4079" LastEditorUserId="57" LastEditDate="2019-03-18T10:37:59.200" LastActivityDate="2019-03-18T10:37:59.200" Title="Retrieving Metabolic networks from WIT Database" Tags="|networks|data-retrieval|public-dataset|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7270" PostTypeId="1" AcceptedAnswerId="7271" CreationDate="2019-03-18T00:51:59.367" Score="0" ViewCount="488" Body="&lt;p&gt;I am using &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/30/7/923/232889&quot; rel=&quot;nofollow noreferrer&quot;&gt;featureCounts&lt;/a&gt; from the subread package to count the number of fragments per gene in an RNA-seq experiment. I feed it a BAM file with paired-end reads aligned to a reference genome. In the help page of the executable, which I have installed using &lt;code&gt;conda install subread&lt;/code&gt;, there is a section about overlaps, including for example the &lt;code&gt;-O&lt;/code&gt; option which will assign the reads/fragments to all overlapping features. What is the default behaviour? Assigning to reads/fragments to the feature with largest overlap? not assigning multi-overlapping reads/fragments?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate your help on this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;OS: macOS X&lt;/p&gt;&#xA;&#xA;&lt;p&gt;featureCounts version: 1.6.3&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2019-03-18T08:12:53.997" Title="How do I tell featureCounts to ignore fragments that overlap with more than one feature?" Tags="|software-usage|subread|featurecounts|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7271" PostTypeId="2" ParentId="7270" CreationDate="2019-03-18T08:12:53.997" Score="1" Body="&lt;p&gt;If a fragment overlaps multiple genomic features then it's unassignable, which is the appropriate way to handle ambiguous mappings when the resulting counts are then given to tools like DESeq2.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-03-18T08:12:53.997" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7272" PostTypeId="2" ParentId="7269" CreationDate="2019-03-18T09:46:42.027" Score="2" Body="&lt;p&gt;It seems like there are some files related to the databases at the ftp site: &lt;a href=&quot;http://ftp.mcs.anl.gov/pub/WIT2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ftp.mcs.anl.gov/pub/WIT2/&lt;/a&gt;. The related files were last updated on 2002. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found it via searching at google: &lt;code&gt;site:*.mcs.anl.gov/ WIT&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which reports also a &lt;a href=&quot;http://ftp.mcs.anl.gov/pub/tech_reports/reports/P694.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;page&lt;/a&gt; about the database. There seems to be a project to develop WIT3, because there is a mailing list for &lt;a href=&quot;https://lists.mcs.anl.gov/mailman/listinfo/wit3_development_team&quot; rel=&quot;nofollow noreferrer&quot;&gt;developers&lt;/a&gt;, maybe you can ask them more details if you have problems with the database. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2019-03-18T09:46:42.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7273" PostTypeId="1" AcceptedAnswerId="7274" CreationDate="2019-03-18T13:27:53.327" Score="5" ViewCount="2599" Body="&lt;p&gt;I have to split this fasta files into smaller files and write them into individual files my files  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;lcl|CP000522.1_prot_ABO13860.1_1 [locus_tag=A1S_3471] [protein=hypothetical protein] [protein_id=ABO13860.1] [location=1..957] [gbkey=CDS]&#xA;&amp;gt;lcl|CP000522.1_prot_ABO13850.1_2 [locus_tag=A1S_3461] [protein=DNA replication protein] [protein_id=ABO13850.1] [location=950..1504] [gbkey=CDS]&#xA;&amp;gt;lcl|CP000522.1_prot_ABO13851.1_3 [locus_tag=A1S_3462] [protein=hypothetical protein] [protein_id=ABO13851.1] [location=complement(2523..3437)] [gbkey=CDS]&#xA;&amp;gt;lcl|CP000522.1_prot_ABO13852.1_4 [locus_tag=A1S_3463] [protein=YPPCP.09C-like protein] [protein_id=ABO13852.1] [location=3538..4788] [gbkey=CDS]&#xA;&amp;gt;lcl|CP000522.1_prot_ABO13853.1_5 [locus_tag=A1S_3464] [protein=Cro-like protein] [protein_id=ABO13853.1] [location=5039..5629] [gbkey=CDS]&#xA;&amp;gt;lcl|CP000522.1_prot_ABO13854.1_6 [locus_tag=A1S_3465] [protein=hypothetical protein] [protein_id=ABO13854.1] [location=complement(6340..6906)] [gbkey=CDS]&#xA;&amp;gt;lcl|CP000522.1_prot_ABO13855.1_7 [locus_tag=A1S_3466] [protein=Resolvase] [protein_id=ABO13855.1] [location=complement(7074..7685)] [gbkey=CDS]&#xA;&amp;gt;lcl|CP000522.1_prot_ABO13856.1_8 [locus_tag=A1S_3467] [protein=hypothetical protein] [protein_id=ABO13856.1] [location=complement(8602..9732)] [gbkey=CDS]&#xA;&amp;gt;lcl|CP000522.1_prot_ABO13857.1_9 [locus_tag=A1S_3468] [protein=putative lipoprotein] [protein_id=ABO13857.1] [location=complement(10072..10374)] [gbkey=CDS]&#xA;&amp;gt;lcl|CP000522.1_prot_ABO13858.1_10 [locus_tag=A1S_3469] [protein=Diaminopimelate decarboxylase] [protein_id=ABO13858.1] [location=complement(10367..10723)] [gbkey=CDS]&#xA;&amp;gt;lcl|CP000522.1_prot_ABO13859.1_11 [locus_tag=A1S_3470] [protein=regulatory protein LysR] [protein_id=ABO13859.1] [location=complement(12076..12444)] [gbkey=CDS]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The other pattern is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;lcl|CP000523.1_prot_ABO13861.1_1 [locus_tag=A1S_3472] [protein=DNA replication protein] [protein_id=ABO13861.1] [location=1..951] [gbkey=CDS]&#xA;&amp;gt;lcl|CP000523.1_prot_ABO13862.1_2 [locus_tag=A1S_3473] [protein=hypothetical protein] [protein_id=ABO13862.1] [location=3048..4262] [gbkey=CDS]&#xA;&amp;gt;lcl|CP000523.1_prot_ABO13863.1_3 [locus_tag=A1S_3474] [protein=hypothetical protein] [protein_id=ABO13863.1] [location=4357..5133] [gbkey=CDS]&#xA;&amp;gt;lcl|CP000523.1_prot_ABO13864.1_4 [locus_tag=A1S_3475] [protein=hypothetical protein] [protein_id=ABO13864.1] [location=6197..8608] [gbkey=CDS]&#xA;&amp;gt;lcl|CP000523.1_prot_ABO13865.1_5 [locus_tag=A1S_3476] [protein=secretory lipase] [protein_id=ABO13865.1] [location=8705..9403] [gbkey=CDS]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So now my idea is how do i parse and write them into individual files such that CP000522 output written to one file and CP000523 written to another file so forth and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far what i understand is i have to match the pattern after &gt;lcl &#xA;so there are other patterns like &quot;&lt;code&gt;LN997847&lt;/code&gt;&quot; in the file &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not sure how to proceed tried it in R but failed &lt;/p&gt;&#xA;&#xA;&lt;p&gt;it can be done with awk and sed which i tried but i can;t define something that parse all header like takes into account &lt;code&gt;CP&lt;/code&gt; as well as &lt;code&gt;LN&lt;/code&gt; .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion or help would be highly appreciated .&#xA;here is my &lt;a href=&quot;https://drive.google.com/open?id=1TpTOYTilUVTvWCXx9BBUsG2g67t86sb_&quot; rel=&quot;noreferrer&quot;&gt;file&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="73" LastEditDate="2023-01-15T20:25:39.730" LastActivityDate="2023-01-15T20:25:39.730" Title="Splitting fasta file into smaller files based on header pattern" Tags="|fasta|awk|shell|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7274" PostTypeId="2" ParentId="7273" CreationDate="2019-03-18T14:39:53.417" Score="9" Body="&lt;p&gt;Here's a simple awk approach:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{if(/^&amp;gt;/){split($1,a,&quot;[|.]&quot;)}print &amp;gt;&amp;gt; a[2]&quot;.fa&quot;}' Protein_FASTA.txt &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, more concisely, just:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '/^&amp;gt;/{split($1,a,&quot;[|.]&quot;)}{print &amp;gt;&amp;gt; a[2]&quot;.fa&quot;}' Protein_FASTA.txt &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When run on the file linked to in your question, that results in the following files:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ls&#xA;AP014650.fa  CP003848.fa  CP007713.fa  CP012005.fa  CP015122.fa  CP017645.fa  CP018422.fa  CP020594.fa  CP023021.fa  CP024577.fa  CP026712.fa  CP027245.fa  CP030108.fa&#xA;CP000522.fa  CP003850.fa  CP007714.fa  CP012007.fa  CP015365.fa  CP017647.fa  CP018678.fa  CP020596.fa  CP023023.fa  CP024578.fa  CP026748.fa  CP027529.fa  CP030109.fa&#xA;CP000523.fa  CP003887.fa  CP008707.fa  CP012008.fa  CP015366.fa  CP017649.fa  CP018679.fa  CP021322.fa  CP023024.fa  CP025267.fa  CP026749.fa  CP027531.fa  CU459137.fa&#xA;CP000864.fa  CP003888.fa  CP008708.fa  CP012953.fa  CP015484.fa  CP017651.fa  CP019218.fa  CP021327.fa  CP023025.fa  CP026126.fa  CP027121.fa  CP027532.fa  CU459138.fa&#xA;CP000865.fa  CP003907.fa  CP008709.fa  CP012954.fa  CP015485.fa  CP017653.fa  CP020573.fa  CP021348.fa  CP023027.fa  CP026127.fa  CP027122.fa  CP027608.fa  CU459139.fa&#xA;CP001183.fa  CP003908.fa  CP008850.fa  CP012955.fa  CP015486.fa  CP017655.fa  CP020575.fa  CP021783.fa  CP023028.fa  CP026128.fa  CP027124.fa  CP027609.fa  CU459140.fa&#xA;CP001922.fa  CP003968.fa  CP008851.fa  CP012956.fa  CP016296.fa  CP017657.fa  CP020576.fa  CP021784.fa  CP023030.fa  CP026129.fa  CP027179.fa  CP027610.fa  JN377410.fa&#xA;CP001923.fa  CP004359.fa  CP010398.fa  CP013925.fa  CP016297.fa  CP018144.fa  CP020577.fa  CP021785.fa  CP023032.fa  CP026339.fa  CP027180.fa  CP029570.fa  LN865144.fa&#xA;CP001938.fa  CP006769.fa  CP010399.fa  CP014216.fa  CP016299.fa  CP018255.fa  CP020580.fa  CP021786.fa  CP023033.fa  CP026340.fa  CP027181.fa  CP029571.fa  LN997847.fa&#xA;CP002523.fa  CP007578.fa  CP010400.fa  CP014217.fa  CP016301.fa  CP018257.fa  CP020585.fa  CP021787.fa  CP023035.fa  CP026705.fa  CP027182.fa  CP029572.fa  LT594096.fa&#xA;CP002524.fa  CP007579.fa  CP010780.fa  CP014292.fa  CP016302.fa  CP018333.fa  CP020589.fa  CP022284.fa  CP024125.fa  CP026706.fa  CP027243.fa  CP029573.fa  Protein_FASTA.txt&#xA;CP003501.fa  CP007580.fa  CP010782.fa  CP014293.fa  CP017643.fa  CP018334.fa  CP020593.fa  CP022285.fa  CP024419.fa  CP026708.fa  CP027244.fa  CP030107.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Explanation&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;if(/^&amp;gt;/){split($1,a,&quot;[|.]&quot;)&lt;/code&gt; : if this line starts with a &lt;code&gt;&amp;gt;&lt;/code&gt;, split the first field on any occurrence of either &lt;code&gt;|&lt;/code&gt; or &lt;code&gt;.&lt;/code&gt; and save the results in the array &lt;code&gt;a&lt;/code&gt;. Since your header lines all start with &lt;code&gt;&amp;gt;lcl|&lt;/code&gt;, then the string you are looking for and a &lt;code&gt;.&lt;/code&gt;, this means that the second value in the &lt;code&gt;a&lt;/code&gt; array will be your target string. &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;print &amp;gt;&amp;gt; a[2]&quot;.fa&quot;&lt;/code&gt; : print (append, &lt;code&gt;&amp;gt;&amp;gt;&lt;/code&gt;) the current line to a file called &quot;whatever the name of this sequence is&quot; (&lt;code&gt;a[2]&lt;/code&gt;) and &lt;code&gt;.fa&lt;/code&gt;. This is run for every line in your input file. &lt;strong&gt;&lt;em&gt;Note that if you run the same command again, you will need to first delete the files created the first time&lt;/em&gt;&lt;/strong&gt;. If you don't, because I am using the &lt;code&gt;&amp;gt;&amp;gt;&lt;/code&gt;, you will just append to the existing files. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-03-18T16:55:45.187" LastActivityDate="2019-03-18T16:55:45.187" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="7277" PostTypeId="2" ParentId="7258" CreationDate="2019-03-18T17:40:32.233" Score="0" Body="&lt;p&gt;Printing a bracketed list of numerical values is not typical for VCF—I'd guess it's not strictly allowed. A list should be printed only with the commas to separate values. Perhaps the solution is as simple as removing the square brackets.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat input.vcf | perl -ne 's/CAF=\[([^\]]+)\]/CAF=$1/; print' &amp;gt; fixed.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Replace &lt;code&gt;cat&lt;/code&gt; with &lt;code&gt;gunzip -c&lt;/code&gt; if your VCF file is gzip-compressed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Perl regular expression is&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;looking for the string &lt;code&gt;CAF=[...]&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;grabbing the characters in &lt;code&gt;...&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;replacing the entire match with &lt;code&gt;CAF=...&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here's a detailed breakdown of the Perl regular expression.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;s/.../.../;&lt;/code&gt; — substitution syntax; replace the contents of the left block with the contents of the right block&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;[^\]]+&lt;/code&gt; — one or more non-&lt;code&gt;]&lt;/code&gt; characters; this is the trickiest bit; see &lt;a href=&quot;https://www.regular-expressions.info/charclass.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt; for more info on the &lt;code&gt;[]&lt;/code&gt; syntax for character classes and the special &lt;code&gt;^&lt;/code&gt; symbol for character class negation&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;(...)&lt;/code&gt; — the parentheses capture the matched pattern into the &lt;code&gt;$1&lt;/code&gt; variable&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;CAF=$1&lt;/code&gt; — replaces everything matched in the first block with &lt;code&gt;CAF=&lt;/code&gt; followed by the captured characters&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-03-18T17:40:32.233" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7278" PostTypeId="2" ParentId="7265" CreationDate="2019-03-18T17:57:34.177" Score="6" Body="&lt;p&gt;Devon's answer gives a good, concise definition. But it's also helpful to consider why seed-and-extend is used and what benefits it provides.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finding approximate string matches requires operations that are expensive computationally. On the other hand, finding exact string matches can be performed using much less expensive operations—that is, it can be done more quickly with less memory. So quickly finding the location of all seed matches and then applying the approximate matching ONLY AT THOSE LOCATIONS (the seed-and-extend strategy) makes sure that less time is wasted on chasing spurious similarities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how long should a seed be? Riffing off &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/7265/seed-for-various-sequence-aligners#comment10746_7267&quot;&gt;your comment&lt;/a&gt;, a seed 3bp in length would be entirely ineffective since that same 3bp sequence is going to occur ALL OVER THE GENOME IN RANDOM PLACES: genes, pseudogenes, transposable elements, regulatory sequences, you name it. On the other hand, a seed 51bp in length would also be ineffective since almost every seed would contain at least one sequencing error or variant/polymorphism, thus preventing a perfect match.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, mappers designed for aligning 100-300bp Illumina reads typically use seeds in the high teens or low 20s.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-03-18T17:57:34.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7279" PostTypeId="1" CreationDate="2019-03-18T19:48:40.637" Score="0" ViewCount="57" Body="&lt;p&gt;I'm working on a somewhat unusual transcriptome focused on a killer X chromosome. This chromosome has some new genes, with expression levels quite different from their autosomal paralogs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've found that Trinity does an ok job of assembling the distinct genes, but when I try to do a gene-level DE analysis, it lumps the distinct genes together. So, I lose important information by doing a gene-level DE analysis and need to use a transcript-level approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've done blastx with all of my transcripts using diamond, and want to extract the isoform from each gene with the best score (bitscore). I'm trying to write a script to parse this, but my scripting skills aren't up to the task. Does anyone know of (or have) a script that can do this?&lt;/p&gt;&#xA;" OwnerUserId="4281" LastEditorUserId="96" LastEditDate="2019-03-19T14:50:28.557" LastActivityDate="2019-03-19T14:50:28.557" Title="How to retrieve the best-scoring Trinity isoform from blastx results" Tags="|python|blast|differential-expression|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7280" PostTypeId="1" AcceptedAnswerId="7281" CreationDate="2019-03-19T03:55:37.193" Score="2" ViewCount="41" Body="&lt;p&gt;What is the difference between C383S and C383A? For C383S, does it mean Cysteine(C) position 383 in the sequence and its next amino acid connect to it is Serine(S)?&lt;/p&gt;&#xA;" OwnerUserId="3760" LastEditorUserId="96" LastEditDate="2019-03-19T13:41:46.567" LastActivityDate="2019-03-19T13:41:46.567" Title="How to interpret amino acid representation" Tags="|proteins|variation|nomenclature|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7281" PostTypeId="2" ParentId="7280" CreationDate="2019-03-19T04:02:59.790" Score="4" Body="&lt;p&gt;This means that at position 383 the Cysteine(C) is mutated to a Serine(S).  Depending on the organism and gene usually the mutation notation is linked to a specific transcript since the position could vary depending on transcript splicing. &lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2019-03-19T04:02:59.790" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7282" PostTypeId="1" CreationDate="2019-03-19T05:26:36.407" Score="3" ViewCount="1324" Body="&lt;p&gt;I would like to trim/mark adapters using trimmomatic or picard MarkIlluminaAdapters from a series of Illumina Paired-End read fastqs. The fastq files may have been done using different kits or different generations/versions of illumina chips.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without knowing ahead of time how the libraries were prepared (or which kit was used), Is there a safe &quot;catch all&quot; adapter sequence (or set of such sequences) that can be passed to a trimming tool and that would cover &lt;em&gt;all&lt;/em&gt; Illumina kits? In other words, what are we expected to know about the fastq to ensure correct trimming?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;edit:&lt;/em&gt; Picard defines a few built-in adapters which cover many standard kits, but it's not immediately obvious if you need to pass a common one (&lt;a href=&quot;https://github.com/broadinstitute/picard/blob/1531278515aaf39c9049b5792ae569b92326dd90/src/main/java/picard/util/IlluminaUtil.java#L141&quot; rel=&quot;nofollow noreferrer&quot;&gt;built-in sequences defined here&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="3235" LastEditorUserId="3235" LastEditDate="2019-03-21T17:01:58.710" LastActivityDate="2019-03-22T17:33:13.610" Title="Is there a safe catch-all adapter sequence for trimming?" Tags="|illumina|picard|trimming|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7283" PostTypeId="2" ParentId="7282" CreationDate="2019-03-19T08:46:31.690" Score="2" Body="&lt;p&gt;You're best off just using &lt;a href=&quot;https://github.com/OpenGene/fastp&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;fastp&lt;/code&gt;&lt;/a&gt; or &lt;a href=&quot;https://www.bioinformatics.babraham.ac.uk/projects/trim_galore/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;Trim Galore!&lt;/code&gt;&lt;/a&gt;, both of which will determine the adapter sequence for you. Trim Galore! uses a built-in list of known sequences for this, whereas fastp uses read overlap.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="96" LastEditDate="2019-03-19T14:56:20.830" LastActivityDate="2019-03-19T14:56:20.830" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="7284" PostTypeId="1" CreationDate="2019-03-19T10:40:01.887" Score="3" ViewCount="95" Body="&lt;p&gt;In creating a parser for the UniProt flat file &lt;code&gt;uniprot_sprot.dat&lt;/code&gt; (&lt;a href=&quot;ftp://ftp.uniprot.org/pub/databases/uniprot/current_release/knowledgebase/complete/&quot; rel=&quot;nofollow noreferrer&quot;&gt;directory&lt;/a&gt;) in the &lt;a href=&quot;https://www.uniprot.org/docs/userman.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt; for the comments &lt;code&gt;POLYMORPHISM&lt;/code&gt; there are listed patterns for alleles, e.g. for &lt;code&gt;1A11_HUMAN&lt;/code&gt; is &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The following alleles of A-11 are known:&lt;br&gt;&#xA;  A*11:01 (A-11E), A*11:02 (A-11K), A*11:03, A*11:04, A*11:05 and A*11:07.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I tried searching for a description of the patterns using the words &lt;code&gt;allele&lt;/code&gt; and/or &lt;code&gt;polymorphism&lt;/code&gt; and found nothing.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How are the allele patterns to be interpreted?  &lt;/li&gt;&#xA;&lt;li&gt;Is there a search term to be used when looking for the patterns?  &lt;/li&gt;&#xA;&lt;li&gt;Also, is there an official repository of the alleles?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The answer does not have to explain the specifics of the example given.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Example of explanation of other allele pattern from &lt;a href=&quot;http://hla.alleles.org/nomenclature/naming.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jZhwr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jZhwr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This explanation of the patterns doesn't explain the part of the pattern in UniProt like &lt;code&gt;(A-11E)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1586" LastEditorUserId="3848" LastEditDate="2020-07-29T08:14:10.693" LastActivityDate="2020-07-29T08:14:10.693" Title="How to interpret UniProt allele patterns?" Tags="|sequence-alignment|phylogenetics|phylogeny|file-formats|uniprot|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7285" PostTypeId="2" ParentId="7284" CreationDate="2019-03-19T13:53:28.097" Score="4" Body="&lt;p&gt;These are cytotoxic T-cell HLA alleles. HLA genotyping is very common and easy to do, so Genbank is the repository.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A*11:01 has very high frequency in Aborigine populations &lt;a href=&quot;http://www.allelefrequencies.net/hla6006a.asp?hla_locus_type=Classical&amp;amp;hla_locus=&amp;amp;hla_allele1=A*11%3A01&amp;amp;hla_allele2=A*11%3A01&amp;amp;hla_selection=&amp;amp;hla_pop_selection=&amp;amp;hla_population=&amp;amp;hla_country=&amp;amp;hla_dataset=&amp;amp;hla_region=&amp;amp;hla_ethnic=&amp;amp;hla_study=&amp;amp;hla_order=order_1&amp;amp;hla_sample_size_pattern=equal&amp;amp;hla_sample_size=&amp;amp;hla_sample_year_pattern=equal&amp;amp;hla_sample_year=&amp;amp;hla_level_pattern=equal&amp;amp;hla_level=&amp;amp;standard=a&amp;amp;hla_show=&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; . You can explore the population genetics and the past, present and current population genetics distributions per population of HLA at &lt;a href=&quot;http://www.allelefrequencies.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.allelefrequencies.net/&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-03-19T13:53:28.097" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7286" PostTypeId="2" ParentId="7174" CreationDate="2019-03-19T14:10:43.293" Score="1" Body="&lt;p&gt;I always assumed that these tags, such as 239963_at you show, mean that the sequence is not specific of a genome region. This means it cannot be assigned to any gene for DGE. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It could be due to changes on the reference genome or due to a faulty design of the tag sequence. However once in the microarray template they cannot be changed without introducing more problems, that's why doing RNA-seq prevent this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry I don't have any reference for this (but if someone wants to add this edit the answer). &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2019-03-19T14:10:43.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7288" PostTypeId="1" CreationDate="2019-03-19T14:16:15.317" Score="0" ViewCount="189" Body="&lt;p&gt;I recently starting studying bioinformatics and have a question. I need to build a model of protein using &lt;a href=&quot;https://swissmodel.expasy.org/interactive#structure&quot; rel=&quot;nofollow noreferrer&quot;&gt;Swiss-Model&lt;/a&gt;. First, I need to download a protein record from PDB (for example, 2HZI), then I must open PDB file using simple text editor. I know this usually requires biopython, but i don't use it yet. Now I'm trying to study how to build a model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Swiss-model needs only one chain to build model, and I remove from file all extra information like AUTHOR,REMARK,HETATM etc. But Swiss-model still gives an error: &quot;Sequence String contains illegal characters. Only letters and gaps allowed.&quot;&lt;/p&gt;&#xA;" OwnerUserId="4289" LastEditorUserId="96" LastEditDate="2019-03-19T19:18:13.900" LastActivityDate="2019-03-20T16:41:17.770" Title="Preparing PDB file for modeling with Swiss-Model" Tags="|pdb|proteomics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7289" PostTypeId="1" CreationDate="2019-03-19T17:07:11.063" Score="1" ViewCount="52" Body="&lt;p&gt;What is the purpose of changing the &lt;code&gt;fixedSearchFolder&lt;/code&gt; and &lt;code&gt;fixedCombinedFolder&lt;/code&gt; in MaxQuant proteomics software? Is there any data that can be reused in other runs, so would it be good to use the same search folder for multiple runs? Or is it simply to direct temporary data and results data into different locations and each run should use a different folder?&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="96" LastEditDate="2019-03-19T18:30:33.737" LastActivityDate="2019-03-19T18:30:33.737" Title="What is the purpose of the &quot;fixedSearchFolder&quot; and &quot;fixedCombinedFolder&quot; in MaxQuant?" Tags="|proteomics|maxquant|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7290" PostTypeId="1" AcceptedAnswerId="7291" CreationDate="2019-03-19T20:25:18.713" Score="0" ViewCount="97" Body="&lt;p&gt;I have two matrices; I want to convert the row names of first matrix to gene symbol from matched ensemble=gene symbol from second matrix&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(mat1[,1:2])&#xA;                TCGA-L5-A4OG-11A-12R-A260-31&#xA;ENSG00000000003                         1818&#xA;ENSG00000000005                            0&#xA;ENSG00000000419                         1436&#xA;ENSG00000000457                         1175&#xA;ENSG00000000460                          242&#xA;ENSG00000000938                          536&#xA;                TCGA-IC-A6RE-11A-12R-A336-31&#xA;ENSG00000000003                         4596&#xA;ENSG00000000005                            3&#xA;ENSG00000000419                          751&#xA;ENSG00000000457                          840&#xA;ENSG00000000460                          205&#xA;ENSG00000000938                          253&#xA;&amp;gt; &#xA;&#xA;&amp;gt; dim(mat1)&#xA;[1] 56925    11&#xA;&amp;gt; &#xA;&#xA;&amp;gt; head(mat2)&#xA;                ensembl              symbol&#xA;ENSG00000274572 ENSG00000274572       ZYXP1&#xA;ENSG00000159840 ENSG00000159840         ZYX&#xA;ENSG00000162378 ENSG00000162378      ZYG11B&#xA;ENSG00000232242 ENSG00000232242    ZYG11AP1&#xA;ENSG00000203995 ENSG00000203995      ZYG11A&#xA;ENSG00000070476 ENSG00000070476        ZXDC&#xA;&amp;gt; &#xA;&#xA;&amp;gt; dim(mat2)&#xA;[1] 36848     2&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How I can do that in R?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2019-03-19T22:46:19.973" Title="Matching these matrices in R" Tags="|r|rna-seq|ensembl|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7291" PostTypeId="2" ParentId="7290" CreationDate="2019-03-19T22:46:19.973" Score="4" Body="&lt;p&gt;Since they share the ensembl_ID column, you can merge them, then assign the symbol column to the rownames, then delete the symbol column.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;merged &amp;lt;- merge(mat1, mat2, by = 0) &#xA;rownames(merge) &amp;lt;- merged[,'symbol'] &#xA;merged[,symbol] &amp;lt;- NULL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stat.ethz.ch/R-manual/R-devel/library/base/html/merge.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://stat.ethz.ch/R-manual/R-devel/library/base/html/merge.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-03-19T22:46:19.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7292" PostTypeId="1" CreationDate="2019-03-20T02:25:20.750" Score="3" ViewCount="390" Body="&lt;p&gt;I have been trying to obtain some preliminary data from &lt;a href=&quot;http://hyphy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HyPhy&lt;/a&gt; selection analyses to inform a larger project. I have obtained a number of assembled mammalian genomes from NCBI with the initial goal of extracting from each of them their corresponding sequence for a specific gene. I wanted to use HyPhy to determine if there are signatures of positive selection in any of the mammalian lineages for that specific gene. However, I have been struggling to make headway with this for some time and am realizing that I may be out of my depth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was hoping that people could suggest, or at least point me in the direction of, a pipeline/guide/best practices for how one would go about achieving what I am trying to do. I would like to start from scratch and make sure I am performing my analyses in a proper way, using advice from other users.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I mentioned, I currently have a collection of assembled genomes from NCBI which I would like to pull specific genes from. My understanding is that I will then need to extract the exons (I imagine there is a way to do this without having to extract the entire gene first), align the sequences from each species while preserving reading frames, and then use that alignment as input in HyPhy's codon models. It is very possible that I am missing out some fundamental steps in the process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If anybody could offer direction/advice, I would be very appreciative.&lt;/p&gt;&#xA;" OwnerUserId="4293" LastActivityDate="2019-03-20T10:42:13.263" Title="Pipeline for extracting gene from multiple genomes for use in HyPhy selection analyses?" Tags="|sequence-alignment|genome|codon|exon|positive-selection|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7293" PostTypeId="1" CreationDate="2019-03-20T07:01:04.927" Score="1" ViewCount="96" Body="&lt;p&gt;Im not able to figure it out why the lower quartile of the two data points missing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://drive.google.com/open?id=1DsUAkSCyMWRYWtShuT0gZ_6Cn0bi1toL&quot; rel=&quot;nofollow noreferrer&quot;&gt;Data&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;require(reshape2)&#xA;require(ggplot2)&#xA;require(RColorBrewer)&#xA;#require(dplyr)&#xA;require(tidyr)&#xA;require(tibble)&#xA;df &amp;lt;- read.csv(&quot;NC_II_box_plot_input.txt&quot;,sep = &quot;,&quot;)&#xA;&#xA;df_melt = melt(df,id.vars = NULL)&#xA;&#xA;df_melt&lt;span class=&quot;math-container&quot;&gt;$Group &amp;lt;- gsub(&quot;HSC&quot;,'',df_melt$&lt;/span&gt;variable)&#xA;head(df_melt)&#xA;df_melt&lt;span class=&quot;math-container&quot;&gt;$Group &amp;lt;- gsub(&quot;CMP&quot;,'',df_melt$&lt;/span&gt;Group)&#xA;df_melt&lt;span class=&quot;math-container&quot;&gt;$Group &amp;lt;- gsub(&quot;GMP&quot;,'', df_melt$&lt;/span&gt;Group)&#xA;df_melt&lt;span class=&quot;math-container&quot;&gt;$Group &amp;lt;- gsub(&quot;Mono&quot;,'', df_melt$&lt;/span&gt;Group)&#xA;df_melt&lt;span class=&quot;math-container&quot;&gt;$Group &amp;lt;- factor(df_melt$&lt;/span&gt;Group,levels = c(&quot;m&quot;,&quot;nc&quot;,&quot;r&quot;,&quot;rr&quot;,&quot;sn&quot;,&quot;t&quot;))&#xA;&#xA;ggplot(df_melt, aes(variable,value)) +&#xA;  stat_boxplot(geom=&quot;errorbar&quot;, width=.5)+&#xA;  geom_boxplot(aes(fill=Group), position = position_dodge2(preserve = &quot;total&quot;))+&#xA;  scale_fill_manual(values=c(&quot;grey&quot;,&quot;dodgerblue4&quot;,&quot;red&quot;,&quot;darkgreen&quot;,&quot;brown&quot;,&quot;maroon1&quot;)) + #for boxplot&#xA;  # scale_fill_manual(values=c(&quot;grey&quot;,&quot;dodgerblue4&quot;,&quot;red&quot;,&quot;darkgreen&quot;,&quot;brown&quot;)) + #for boxplot&#xA;  geom_jitter(position=position_jitter(width=0.3), size=0.01, colour=&quot;black&quot;) +&#xA;&#xA;  #scale_fill_manual(values = c(&quot;black&quot;,&quot;brown&quot;,&quot;maroon1&quot;,&quot;darkgreen&quot;))+&#xA;  #stat_summary(fun.y=mean, colour=&quot;red&quot;, geom=&quot;line&quot;, aes(group = 1))+&#xA;  #geom_jitter(position = position_jitter(0.0001),size=.00001)+&#xA;  #theme_bw(base_size=15)+&#xA;  theme_bw()+&#xA;&#xA;  theme(axis.text.x=element_text(angle = 45, size=25, face=&quot;bold&quot;, hjust = 1), &#xA;        axis.text.y=element_text(angle=0, size=35, face=&quot;bold&quot;, vjust=0.5),&#xA;        plot.title = element_text(size=40, face=&quot;bold&quot;), &#xA;        legend.title=element_blank(), &#xA;        legend.key.size=unit(1, &quot;cm&quot;),      #Sets overall area/size of the legend&#xA;        legend.text=element_text(size=30)) +&#xA;&#xA;  labs(x=&quot;Cell_Type&quot;, y=expression(paste(&quot;Expression&quot;), title=&quot;Expression&quot;)) + &#xA;  guides(colour = guide_legend(override.aes = list(size=5)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Y2NNy.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Y2NNy.jpg&quot; alt=&quot;rrCMP and rrMono lower quartile missing &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;#&#xA;&#xA;&lt;p&gt;to find the quartile i used this as well &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A &amp;lt;- boxplot(df)&#xA;mytable &amp;lt;- A&lt;span class=&quot;math-container&quot;&gt;$stats&#xA;colnames(mytable)&amp;lt;-A$&lt;/span&gt;names&#xA;rownames(mytable)&amp;lt;-c('min','lower quartile','median','upper quartile','max')&#xA;mytable &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any help or suggestion would be highly appreciated &lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2019-03-20T10:19:49.217" LastActivityDate="2019-03-20T10:19:49.217" Title="Missing lower IQR in boxplot" Tags="|r|ggplot2|" AnswerCount="0" CommentCount="5" ClosedDate="2019-03-23T06:13:21.683" ContentLicense="CC BY-SA 4.0" />
  <row Id="7294" PostTypeId="2" ParentId="7288" CreationDate="2019-03-20T08:57:59.707" Score="3" Body="&lt;p&gt;As explained in the website the data introduced must be:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Format must be FASTA, Clustal, plain string, or a valid UniProtKB AC&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So you need to paste just the protein sequence. Without seeing the data you paste I would guess that you don't use a valid plain string.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As Jonathan Moore explains below a FASTA format is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;SEQUENCE_1 &#xA;MTEITAAMVKELRESTGAGMMDCKNALSETNGDFDKAVQLLREKGLGKAAKKADRLAAEG&#xA;LVSVKVSDDFTIAAMRPSYLSYEDLDMTFVENEYKALVAELEKENEERRRLKDPNKPEHK&#xA;IPQFASRKQLSDAILKEAEEKIKEELKAQGKPEKIWDNIIPGKMNSFIADNSQLDSKLTL&#xA;MGQFYVMDDKKTVEQVIAEKEKEFGGKIKIVEFICFEVGEGLEKKTEDFAAEVAAQL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(breaking the sequence in several lines is optional but usually done fore easy reading)&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2019-03-20T16:41:17.770" LastActivityDate="2019-03-20T16:41:17.770" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7295" PostTypeId="1" CreationDate="2019-03-20T10:18:34.163" Score="4" ViewCount="111" Body="&lt;p&gt;I have come across a modeling toolbox, &lt;a href=&quot;https://www.nature.com/articles/s41598-018-22031-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;ATLANTIS&lt;/a&gt;, which is able to determine cell fates in silico based on the input models provided. This MATLAB-based toolbox is built on a method called &quot;Attractor Landscape Analysis.&quot; I tried to find a paper dedicated to explaining this method but I did not have any luck so far. Can any of you folks please explain it to me, especially the term &quot;attractor&quot;? &lt;/p&gt;&#xA;" OwnerUserId="4996" LastEditorUserId="57" LastEditDate="2019-03-20T11:10:03.610" LastActivityDate="2019-03-21T08:47:45.633" Title="Attractor Landscape Analysis" Tags="|networks|algorithms|pathway|literature-search|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7296" PostTypeId="2" ParentId="7292" CreationDate="2019-03-20T10:42:13.263" Score="2" Body="&lt;p&gt;Okay well, because you're using HYPHY, you're going to need a gene tree of every gene you intend to analyze. This requires you to understand the homology relationships between the genomes at hand. Depending on your project, you might already have a list of homologs you're trying to analyze, or you can start from scratch. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If starting from scratch, you will need the proteomes of the species you're interested in (that is, the translation of the coding sequences of each gene, or to put it another way, the concatenated exons). Genome releases should typically have a proteome available for download, along with its corresponding coding sequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next, you need some way to get the homolog groups for the genes in your genomes. You can use a pipeline like &lt;a href=&quot;https://github.com/davidemms/OrthoFinder&quot; rel=&quot;nofollow noreferrer&quot;&gt;OrthoFinder&lt;/a&gt;, or &lt;a href=&quot;https://academic.oup.com/mbe/article/31/11/3081/2925722&quot; rel=&quot;nofollow noreferrer&quot;&gt;iterative tree building and trimming&lt;/a&gt;, to infer homolog clusters (OrthoFinder calls them orthogroups but they're the same thing - homologous genes for the species of interest including both single copy orthologs and paralogs). &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For this step, if you already have a set of genes and you know the copy numbers, or know that they're single copy orthologs, then it is more easy - you can just pull the sequence from each proteome and go from there. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Once you have your homologs, you need to align the amino acids using a tool like &lt;a href=&quot;https://mafft.cbrc.jp/alignment/software/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MAFFT&lt;/a&gt;, and then align the coding sequences to the amino acid alignment to attain a coding sequence alignment. For this you can use a tool like &lt;a href=&quot;http://www.bork.embl.de/pal2nal/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pal2nal&lt;/a&gt;, or check out &lt;code&gt;pxaa2cdn&lt;/code&gt; from the &lt;a href=&quot;https://github.com/FePhyFoFum/phyx&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;phyx&lt;/code&gt;&lt;/a&gt; package written by some of my collaborators. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If the sequences aren't too divergent, and intron-exon structure is conserved, you could possibly also include alignments of the introns to provide more phylogenetic information. If you do use this approach however, you'd need to align the exons and introns separately. Also, the introns would not be included in the selection analysis. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Next, I would typically reinfer the gene tree of each gene using nucleotide models, which depending on the time, level of selective constraint, etc. can give more signal for the phylogenetic relationships. For this you can use a tree inference program like &lt;a href=&quot;https://cme.h-its.org/exelixis/software.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;RAxML&lt;/a&gt;. With this tree and your coding sequence alignment, you'll be ready to run analyses in HYPHY (aBS-REL, MEME, FUBAR, etc., depending on your requirements).&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Note that if you want, you can also do this in a Bayesian framework to integrate over uncertainty in the model parameters, tree topology, etc. Several Bayesian phylogenetic inference tools (e.g. &lt;a href=&quot;http://nbisweden.github.io/MrBayes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MrBayes&lt;/a&gt;, &lt;a href=&quot;http://www.atgc-montpellier.fr/phylobayes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PhyloBayes&lt;/a&gt;), allow you to infer putatively positively selected sites at the same time as tree inference.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2841" LastActivityDate="2019-03-20T10:42:13.263" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7298" PostTypeId="1" AcceptedAnswerId="7299" CreationDate="2019-03-20T16:12:06.437" Score="3" ViewCount="485" Body="&lt;p&gt;I have a set (a couple of millions) of kmer pairs of known length (usually 21) that I know that are heterozygous in the middle nucleotide. For instance:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AAAAAAAAAANAAAAAAAAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where N has always two states. I would like to map these nucleotides on a fragmented genome assembly that is presumably haploid. I bet that this problem is resolved, but I have hard times to make a practical solution, preferably within python environment. I considered following&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Using an existing mapper&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I tried to use python binding of &lt;a href=&quot;https://github.com/KamilSJaron/minimap2/tree/master/python#class-mappyaligner&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimap2&lt;/a&gt;, thinking that I just need to tell it that I search for exact matches only, but I did not manage to set it up. I set up smaller minimisers but it always returned no hits for kmers of length 21 (I tried to tweak parameters &lt;code&gt;k&lt;/code&gt;, &lt;code&gt;w&lt;/code&gt; and &lt;code&gt;min_cnt&lt;/code&gt;, but to be honest I don't have a good understanding of how minimizers work). Is there a mapper that would be easily set up to do these searches?&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;strong&gt;Hash table&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The second thing I tried was to build a hash table of all k - 1 / 2 mers of the genome to map the left and right sides of the kmers. Where sequences were mapped to lists of corresponding genomic positions, but this have blown my memory very soon (~50m of kmers were indexed using ~8GB of memory; i.e. this approach does not seems to be viable on a genomic scale).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also suspecting that there will be an elegant solution using a suffix array, but I have not cracked that one yet.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2019-03-21T08:28:33.967" LastActivityDate="2021-09-13T19:08:00.553" Title="mapping heteryzygous kmers on a genome" Tags="|genome|algorithms|k-mer|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7299" PostTypeId="2" ParentId="7298" CreationDate="2019-03-20T16:34:34.347" Score="5" Body="&lt;p&gt;I don't know if this is the best answer, but one quick-n-dirty approach I commonly use involves tweaking &lt;code&gt;bwa mem&lt;/code&gt; to report only perfect matches. So in your case, the idea is to make two copies of each 21-mer, replace the N with the appropriate allele, and then map the resulting sequences to your bwa-indexed assembly like so.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bwa mem -k 21 -T 21 -a -c 5000 assembly.fasta your-21mers.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;-k 21&lt;/code&gt; says to use a minimum seed length of 21, which forces exact matches. The &lt;code&gt;-T 21&lt;/code&gt; requires a minimum score of 21, which also enforces an exact match. The &lt;code&gt;-a&lt;/code&gt; parameter reports all matches, since a &amp;quot;best&amp;quot; match doesn't make sense in this situation. The &lt;code&gt;-c&lt;/code&gt; parameter limits how many matches are reported, which may need to be adjusted depending on how repetitive the 21-mer is.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt; If you don't want to introduce a dependency like &lt;a href=&quot;https://github.com/nanoporetech/bwapy&quot; rel=&quot;nofollow noreferrer&quot;&gt;bwapy&lt;/a&gt;, it's fairly straightforward to implement this in pure Python.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;with tempfile.TemporaryFile() as kmerfile:&#xA;    for kmer in kmers:&#xA;        print('&amp;gt;{k}\n{k}'.format(k=kmer), file=kmerfile)&#xA;    kmerfile.flush()             # Often works without these two lines&#xA;    os.fsync(kmerfile.fileno())  # But best to be safe&#xA;    bwacmd = ['bwa', 'mem', '-k', ksize, '-t', ksize, '-a', '-c', '5000', refrfile, kmerfile.name]&#xA;    bwaproc = subprocess.Popen(bwacmd, stdout=subprocess.PIPE, universal_newlines=True)&#xA;    for line in bwaproc.stdout:&#xA;        # process SAM records line by line&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Also, one can &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/511/96&quot;&gt;call bwa mem via a C API&lt;/a&gt;, but the mechanics of this are left as an exercise for the reader. &lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="57" LastEditDate="2021-09-13T19:08:00.553" LastActivityDate="2021-09-13T19:08:00.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7300" PostTypeId="1" CreationDate="2019-03-21T06:09:57.933" Score="2" ViewCount="213" Body="&lt;p&gt;From the &lt;a href=&quot;https://drive.google.com/file/d/1MfVRpxPfT0cfmkFxnEm2Nu0LEF-a3acj/view&quot; rel=&quot;nofollow noreferrer&quot;&gt;log&lt;/a&gt; file it seems &lt;a href=&quot;https://github.com/tseemann/snippy&quot; rel=&quot;nofollow noreferrer&quot;&gt;snippy&lt;/a&gt; doesn't do assembly. It only does mapping: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fastq --&amp;gt; SAM --&amp;gt; BAM --&amp;gt; VCF --&amp;gt; consensus_seq/snps&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But if multiple samples are given (with snippy-multi option) it outputs an &lt;a href=&quot;https://drive.google.com/file/d/13QNtnAKSbqDqJ9UMcCFRp4rR10DQUX9k/view&quot; rel=&quot;nofollow noreferrer&quot;&gt;aligned fasta format&lt;/a&gt;. My understanding so far is it doesn't go through any MSA alignment algorithm, but use the pairwise alignment information from SAM files. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How is the aligned fasta file generated? The log file doesn't contain information about how snippy calculates it.  Is there any other tool that I can use to get similar aligned fasta file from SAM/VCF file without going through any actual MSA algorithm? &lt;/p&gt;&#xA;" OwnerUserId="3555" LastEditorUserId="3848" LastEditDate="2020-05-05T00:35:01.647" LastActivityDate="2020-05-05T00:35:01.647" Title="How snippy makes MSA-like aligned fasta output from multiple samples?" Tags="|phylogenetics|phylogeny|ngs|variant-calling|read-mapping|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7301" PostTypeId="2" ParentId="7295" CreationDate="2019-03-21T08:47:45.633" Score="1" Body="&lt;p&gt;From &lt;a href=&quot;https://en.wikipedia.org/wiki/Attractor&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;:&#xA;In the mathematical field of dynamical systems, an attractor is a set of numerical values &lt;strong&gt;toward which a system tends to evolve&lt;/strong&gt;, for a wide variety of starting conditions of the system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In biology you can think of an attractor as a state towards the system is moving to. A good example is the process of differentiation where stem cells compromise to more specialized cells, that could be thought as the attractors of the process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aJwlG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aJwlG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="678" LastActivityDate="2019-03-21T08:47:45.633" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7302" PostTypeId="2" ParentId="7174" CreationDate="2019-03-21T08:58:39.727" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Is the annotation able to distinguish such 'same' tags, during DGE analysis?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, you'll end up discarding such probes (assuming you have a reason to actually use microarrays still).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why aren't tags unique?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Platforms such as HG-U133 date back to 2002 (meaning they were designed a few years before that), when the human reference genome (and the reference transcriptome) were not nearly as &quot;finished&quot; as they are today. Probes that may have made sense given what we knew almost 20 years ago may no longer makes sense now given what we know. Further, microarrays are rather conservative, in that they tend to primarily gain probes over time and not have them removed (lest companies lose customers who are relying on given probes).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-03-21T08:58:39.727" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7304" PostTypeId="2" ParentId="7298" CreationDate="2019-03-21T13:23:32.123" Score="4" Body="&lt;p&gt;Indeed, with suffix arrays, you can map a kmer in ~0.0007s. The way to go is to build a suffix array (for instance using &lt;a href=&quot;https://bitbucket.org/alex-warwickvesztrocy/pysais/src&quot; rel=&quot;nofollow noreferrer&quot;&gt;PySAIS package)&lt;/a&gt; and then do a convoluted binary search of the four possible kmers. A full implementation of such mapper can be found &lt;a href=&quot;https://github.com/KamilSJaron/smudgeplot/blob/dev/smudgeplot/map.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To build suffix array&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;from Bio.Seq import Seq&#xA;from PySAIS import sais&#xA;from bisect import bisect_right&#xA;&#xA;# load the genome&#xA;kmer_genome_file = 'my_genome.fasta'&#xA;ffile = SeqIO.parse(kmer_genome_file, &quot;fasta&quot;)&#xA;sequences = []&#xA;scf_names = []&#xA;for seq_record in ffile:&#xA;    scf_names.append(seq_record.name)&#xA;    sequences.append(str(seq_record.seq).upper())&#xA;&#xA;ffile.close()&#xA;&#xA;# paste all scaffolds into a single string and build a suffix array&#xA;genome = '$'.join(sequences)&#xA;sa = sais(genome)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To transform genome coordinate to scaffold/position coordinates we define a function &lt;code&gt;evaluated2assembly_position&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def evaluated2assembly_position(eval_pos):&#xA;    scf_index = bisect_right(scf_sizes, eval_pos)&#xA;    eval_pos - scf_sizes[scf_index - 1]&#xA;    if scf_index &amp;gt; 0:&#xA;        return((scf_names[scf_index], eval_pos - scf_sizes[scf_index - 1]))&#xA;    else:&#xA;        return((scf_names[scf_index], eval_pos))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and finally the function for a convoluted binary search (assuming all kmers being 21 nt long and having the unknown nucleotide in the middle, on 11th nt). Note that you need to map to both strands of DNA, which is equivalent of mapping kmer and it's reverse complementary sequence&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def searchKmer(self, kmer):&#xA;    reverse_complementary_kmer = str(Seq(kmer).reverse_complement())&#xA;    alignments = []&#xA;    for strand in ['+', '-']:&#xA;        if strand == '-':&#xA;            kmer = reverse_complementary_kmer&#xA;        L_kmer = kmer[0:10]&#xA;        l = 0&#xA;        r = len(sa) - 1&#xA;        while l &amp;lt;= r:&#xA;            m = (l + r) // 2&#xA;            eval_pos = sa[m]&#xA;            L_genome_kmer = genome[eval_pos:(eval_pos + 10)]&#xA;            if L_kmer == L_genome_kmer:&#xA;                r_L_converged = r&#xA;                for N in ['A', 'C', 'G', 'T']:&#xA;                    kmer = kmer[0:10] + N + kmer[11:21]&#xA;                    while l &amp;lt;= r:&#xA;                        m = (l + r) // 2&#xA;                        eval_pos = sa[m]&#xA;                        genome_kmer = genome[eval_pos:(eval_pos + 21)]&#xA;                        if kmer == genome_kmer:&#xA;                            m_hit = m&#xA;                            # following ms&#xA;                            while kmer == genome_kmer:&#xA;                                scf, pos = evaluated2assembly_position(eval_pos)&#xA;                                alignments.append([scf, pos, strand, N])&#xA;                                m += 1&#xA;                                eval_pos = sa[m]&#xA;                                genome_kmer = genome[eval_pos:(eval_pos + 21)]&#xA;                            # previous ms&#xA;                            m = m_hit - 1&#xA;                            eval_pos = sa[m]&#xA;                            genome_kmer = genome[eval_pos:(eval_pos + 21)]&#xA;                            while kmer == genome_kmer:&#xA;                                scf, pos = evaluated2assembly_position(eval_pos)&#xA;                                alignments.append([scf, pos, strand, N])&#xA;                                m -= 1&#xA;                                eval_pos = sa[m]&#xA;                                genome_kmer = genome[eval_pos:(eval_pos + 21)]&#xA;                            break&#xA;                        elif genome_kmer &amp;lt; kmer:&#xA;                            l = m + 1&#xA;                        else:&#xA;                            r = m - 1&#xA;                    r = r_L_converged # recover the r after converging on the left sub-kmer&#xA;                break&#xA;            elif L_genome_kmer &amp;lt; L_kmer:&#xA;                l = m + 1&#xA;            else:&#xA;                r = m - 1&#xA;    return(alignments)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Taking one of the fist kmers in the scaffold one to test it&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;searchKmer('ACGGCCACCCNGTGTCGTTGT')                                                                                                                                                                                                                                                    &#xA;Out[23]: [['scaf000001', 28, 'G']]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And a softmasked kmer that we can expect more than once&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;kmer = 'ccgaaccacgNtggaggcctg'.upper()                                                                                                                                                                                                                              &#xA;&#xA;searchKmer(kmer)                                                                                                                                                                                                                                                    &#xA;Out[31]: &#xA;[['CCGAACCACGATGGAGGCCTG', '1_Tps_b3v07_scaf000306', 5196, 'A'],&#xA; ['CCGAACCACGCTGGAGGCCTG', '1_Tps_b3v07_scaf008224', 15781, 'C'],&#xA; ['CCGAACCACGGTGGAGGCCTG', '1_Tps_b3v07_scaf001152', 71261, 'G'],&#xA; ['CCGAACCACGTTGGAGGCCTG', '1_Tps_b3v07_scaf002780', 1721, 'T']]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So to map all your kmers (assuming a file with a kmer per line)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;kmer_file_name = 'my_kmer_file.txt'&#xA;with open(kmer_file_name, 'r') as kmer_file:&#xA;    kmers = [kmer.rstrip() for kmer in kmer_file]&#xA;&#xA;mapping_list = [searchKmer(kmer) for kmer in kmers]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Stuff can be probably done in a more elegant way, but this is a working solution that scales to any genome size.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2019-04-26T09:04:11.413" LastActivityDate="2019-04-26T09:04:11.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7305" PostTypeId="1" CreationDate="2019-03-21T15:10:20.277" Score="0" ViewCount="492" Body="&lt;p&gt;I have 3 tumour samples from 3 patients from an experiment, I also downloaded 10 normal samples from TCGA. My design is like this &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; mycols&#xA;               condition batch&#xA;normal1           normal     1&#xA;normal2           normal     1&#xA;normal3           normal     1&#xA;normal4           normal     1&#xA;normal5           normal     1&#xA;normal6           normal     1&#xA;normal7           normal     1&#xA;normal8           normal     1&#xA;normal9           normal     1&#xA;normal10          normal     1&#xA;OESO_036_a_RNA    cancer     2&#xA;OESO_013_a_RNA    cancer     2&#xA;OESO_005_a_RNA    cancer     2&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But DESeq2 returns error when trying for differential expression analysis&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dds &amp;lt;- DESeqDataSetFromMatrix(countData=counts, colData=mycols, design=~batch+condition)&#xA;Error in checkFullRank(modelMatrix) : &#xA;  the model matrix is not full rank, so the model cannot be fit as specified.&#xA;  One or more variables or interaction terms in the design formula are linear&#xA;  combinations of the others and must be removed.&#xA;&#xA;  Please read the vignette section 'Model matrix not full rank':&#xA;&#xA;  vignette('DESeq2')&#xA;In addition: Warning message:&#xA;In DESeqDataSet(se, design = design, ignoreRank) :&#xA;  some variables in design formula are characters, converting to factors&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then how I could extract deferentially expressed genes between tumour and normal samples while they are coming from two different batches?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help please?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2019-03-21T16:40:16.270" Title="Differential expression analysis when nested effects" Tags="|r|rna-seq|deseq2|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7306" PostTypeId="2" ParentId="7305" CreationDate="2019-03-21T16:40:16.270" Score="3" Body="&lt;p&gt;llrs is right.  &quot;Matrix Model not full rank&quot; means that two of your conditions are confounded.  You can't get your normals from one place and your experimentals from another&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And no, there generally is no clever way out of this.  For all you know, &lt;strong&gt;any&lt;/strong&gt; differences you see between your normals and your cancers are 100% due to being sequenced somewhere else, and have nothing to do with real biology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to run this through DESeq, drop batch effect.  But tell whoever you are reporting the results to that batch is hopelessly confounded with cancer state, and you can't tell which differences are due to what.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-03-21T16:40:16.270" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7307" PostTypeId="1" CreationDate="2019-03-21T20:05:01.370" Score="2" ViewCount="109" Body="&lt;p&gt;I would like to assign a URL/URI to accession runs (i.e. SRR IDs) from the NCBI short read archive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In program listings, I used to list them as &lt;code&gt;ftp://&lt;/code&gt; URLs, because they were available on their servers as such, e.g. &lt;code&gt;SRR3418242&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ftp://ftp-trace.ncbi.nlm.nih.gov/sra/sra-instant/reads/ByRun/sra/SRR/SRR341/SRR3418242/SRR3418242.sra&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But this approach is deprecated, plus, not all runs are available this way anymore. The now recommended approach is to use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra/docs/sradownload/&quot; rel=&quot;nofollow noreferrer&quot;&gt;sra-tools to download the data&lt;/a&gt;. This tool takes the SRR Ids directly, i.e. &lt;code&gt;prefetch &quot;SRR3418242&quot;&lt;/code&gt;, and resolves them to one of many possible locations (home directory, http, etc).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an official URI scheme that can be used to reference accession runs from the short-read-archive ? I would like to disambiguate them from other types of files and URLs (S3 keys, http, file). For file manifests, spreadsheets, and publications, is there something similar to &lt;code&gt;srr://SRR3418242&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="3235" LastEditorUserId="3235" LastEditDate="2019-03-21T21:10:18.177" LastActivityDate="2020-06-21T00:05:35.623" Title="What is the recognized way to reference accession data (.sra files) from NCBI as a URI?" Tags="|phylogenetics|sratoolkit|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7308" PostTypeId="1" CreationDate="2019-03-22T00:14:49.087" Score="1" ViewCount="88" Body="&lt;p&gt;I have this script to trim adaptors from my fastq sequencing files&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for PAIR in $(fastq | sed 's/_R.//g' | uniq)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am gitting this error: bad substitution&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could you help &lt;/p&gt;&#xA;" OwnerUserId="4299" LastEditorUserId="4299" LastEditDate="2019-03-24T05:10:28.750" LastActivityDate="2019-03-24T05:10:28.750" Title="&quot;Bad substitution&quot; error while trimming adapters" Tags="|fastq|quality-control|shell|" AnswerCount="0" CommentCount="2" ClosedDate="2019-04-08T22:10:03.150" ContentLicense="CC BY-SA 4.0" />
  <row Id="7309" PostTypeId="2" ParentId="7282" CreationDate="2019-03-22T16:52:34.170" Score="2" Body="&lt;p&gt;Alternatively, I really like using bbduk which is part of the &lt;a href=&quot;https://github.com/BioInfoTools/BBMap&quot; rel=&quot;nofollow noreferrer&quot;&gt;BBMap suite&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've processed every nascent sequencing dataset that has been published, and found a lot of quirky errors with older datasets using TrimGalore. bbduk is a little more fine-tuneable relative to cutadapt/trimmomatic/trimGalore (built on top of cutadapt)/fastp and the run time and memory usage is significantly less.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you clone/install the repository, there is a directory of adapter fa files that you can choose from for trimming in the resources/ subdirectory which you can specify using the &quot;ref&quot; argument (i.e. ref=adapters.fa). The &quot;adapters.fa&quot; file is an excellent catchall for adapters, but you will likely over-trim a little. Alternatively, if you know what adapters were used in your dataset, you can choose one of the other pre-configured fa files. There are also a number of stats/marking/reporting/trimmed read save options which you can see here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/BioInfoTools/BBMap/blob/master/sh/bbduk.sh&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/BioInfoTools/BBMap/blob/master/sh/bbduk.sh&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4298" LastEditorUserId="964" LastEditDate="2019-03-22T17:33:13.610" LastActivityDate="2019-03-22T17:33:13.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7310" PostTypeId="2" ParentId="7237" CreationDate="2019-03-22T20:29:29.557" Score="2" Body="&lt;p&gt;Out of all of the major trimming tools available and widely used (trimmomatic, cutadapt/trimGalore (trimGalore is built on top of cutadapt), fastp), I actually instead prefer bbduk which is part of the bbmap suite:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/BioInfoTools/BBMap&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/BioInfoTools/BBMap&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/BioInfoTools/BBMap/blob/master/sh/bbduk.sh&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/BioInfoTools/BBMap/blob/master/sh/bbduk.sh&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's much more &quot;fine tuneable&quot; than the other tools and runs in much less time using much less memory (as long as you set the memory limit as it's written mostly in java!). You can trim both ends or use a sliding window approach similar to what you see in Trimmomatic. Even better you can limit the number of bp in that it will trim from either left/right (5'/3'). It's by far the best all-around trimming tool for any type of sequencing dataset ({RNA, ChIP, PRO, GRO, ATAC, DNA}-seq etc...) in my opinion.&lt;/p&gt;&#xA;" OwnerUserId="4298" LastActivityDate="2019-03-22T20:29:29.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7311" PostTypeId="1" CreationDate="2019-03-22T21:55:28.690" Score="4" ViewCount="379" Body="&lt;p&gt;I've downloaded this &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE43013&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSE43013&lt;/a&gt; dataset using GEOquery in R. My understanding is that it contains expression data from liver, kidney, and brain for multiple species. I would like to produce gene expression tables for each species and tissue type. The goal would then be to perform PCA and regression analyses to see if there is a relationship between expression of certain genes and e.g. taxonomic position, diet, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I have never worked with gene expression data before so I have very little understanding of the GSE file. I have failed to extract any kind of expression data from the file. Solutions that I have found online do not seem to be working for this dataset (for example producing empty tables where I assume there would normally be data on expression levels). Because of my inexperience I cannot determine why they are not working in this situation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate some help in understanding the format of this file, but most importantly, in figuring out how to pull expression data from it, if that is possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I still haven't managed to pull data from the GSE file, but looking through the samples associated with the data set (such as &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM1055020&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;), I can see that there are files formatted like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;mus.lv.3&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ENSMUSG00000000001  26.3376&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ENSMUSG00000000003  0&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ENSMUSG00000000028  0.501533&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ENSMUSG00000000031  0.116078&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume that the values in the left column refer to specific genes. Perhaps the values in the right column represent expression intensity? If so, would these values be normalized and comparable between species as they are?&lt;/p&gt;&#xA;" OwnerUserId="4293" LastEditorUserId="4293" LastEditDate="2019-03-23T02:25:17.903" LastActivityDate="2019-03-24T00:13:19.907" Title="How to extract gene expression tables from this GEO dataset?" Tags="|r|bioconductor|gene-expression|gse|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7313" PostTypeId="2" ParentId="6849" CreationDate="2019-03-23T01:32:39.607" Score="1" Body="&lt;p&gt;I had this same issue. I reordered the levels and that worked for most functions, but would switch back with the FeatureHeatmap. Turns out there is a line in the function that changes it to a character. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ended up making a &quot;new&quot; function and just changed that one line from:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data.plot$ident &amp;lt;- as.character(x = object@ident)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data.plot$ident &amp;lt;- object@ident&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Once that was fixed, the order that I assigned worked out just fine.&lt;/p&gt;&#xA;" OwnerUserId="4304" LastEditorUserId="4304" LastEditDate="2019-03-23T01:41:40.603" LastActivityDate="2019-03-23T01:41:40.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7314" PostTypeId="2" ParentId="5244" CreationDate="2019-03-23T17:10:56.673" Score="2" Body="&lt;p&gt;You can use &lt;a href=&quot;https://github.com/AAlhendi1707/countToFPKM&quot; rel=&quot;nofollow noreferrer&quot;&gt;countToFPKM&lt;/a&gt; package.&#xA;This package provides an easy to use function to convert the read count matrix into FPKM matrix; following the equation in &#xA;&lt;img src=&quot;https://s0.wp.com/latex.php?latex=%5Ctext%7BFPKM%7D_i%20%3D%20%5Cdfrac%7BX_i%7D%7B%20%5Cleft%28%5Cdfrac%7B%5Cwidetilde%7Bl%7D_i%7D%7B10%5E3%7D%5Cright%29%20%5Cleft%28%20%5Cdfrac%7BN%7D%7B10%5E6%7D%20%5Cright%29%7D%20%3D%20%5Cdfrac%7BX_i%7D%7B%5Cwidetilde%7Bl%7D_i%20N%7D%20%5Ccdot%2010%5E9%20%20&amp;amp;bg=ffffff&amp;amp;fg=000000&amp;amp;s=0&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;fpkm()&lt;/code&gt; function requires three inputs to return FPKM as numeric matrix normalized by library size and feature length:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;counts&lt;/code&gt; A numeric matrix of raw feature counts. &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;featureLength&lt;/code&gt; A    numeric vector with feature lengths that can be&#xA;obtained using&lt;br&gt;&#xA;&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/biomaRt/inst/doc/biomaRt.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;biomaRt&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;meanFragmentLength&lt;/code&gt; A numeric vector with mean fragment lengths,&lt;br&gt;&#xA;which can be calculate with &lt;a href=&quot;https://broadinstitute.github.io/picard/command-line-overview.html#CollectInsertSizeMetrics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Picard&lt;/a&gt;&#xA;using CollectInsertSizeMetrics.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://github.com/AAlhendi1707/countToFPKM&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/AAlhendi1707/countToFPKM&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4305" LastEditorUserId="4305" LastEditDate="2019-03-30T16:26:39.840" LastActivityDate="2019-03-30T16:26:39.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7315" PostTypeId="2" ParentId="7160" CreationDate="2019-03-23T19:13:06.817" Score="3" Body="&lt;p&gt;If you just want to get the markers which were not reported/genotyped by 23andMe (as you mentioned in the comment), you don´t want to convert your 23andMe data to SAM/BAM (which is most probably really difficult or impossible, as explained by @Wouter De Coster), but rather to perform &lt;em&gt;imputation&lt;/em&gt; on your data file. This will give you a genotype file with genotype or allele probabilities for all the markers. While it will not give you a 100% certainty for the lacking variants, it can certainly give you more information on many of them. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB: you can imputate the Y-chromosome variants &lt;em&gt;only&lt;/em&gt; if your 23andMe file contain data from Y-chromosome. If you want to know about that option, let me know.&lt;/p&gt;&#xA;" OwnerUserId="4306" LastEditorUserId="4306" LastEditDate="2019-03-23T21:20:18.453" LastActivityDate="2019-03-23T21:20:18.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7316" PostTypeId="2" ParentId="7311" CreationDate="2019-03-23T23:02:20.973" Score="2" Body="&lt;p&gt;Assuming that you have downloaded processed file, according to your link, it is either plain text (compressed) or GTF. If you would have plain text you probably would have posted a snapshot or sample of it, so I assume you have the GTF. According to format description for GTF-files: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Format consists of one line per feature, each containing 9 columns of data, plus optional track definition lines. &lt;/li&gt;&#xA;&lt;li&gt;The 9 columns (or fields) are tab-separated and all fields contain a value (missing/NA-values are usually denoted with &quot;.&quot;). &lt;/li&gt;&#xA;&lt;li&gt;Fields are: &#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;seqname&lt;/strong&gt; - name of the chromosome or scaffold; chromosome names can be given with or without the 'chr' prefix. Important note: the seqname must be one used within Ensembl, i.e. a standard chromosome name or an Ensembl identifier such as a scaffold ID, without any additional content such as species or assembly. See the example GFF output below.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;source&lt;/strong&gt; - name of the program that generated this feature, or the data source (database or project name)&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;feature&lt;/strong&gt; - feature type name, e.g. Gene, Variation, Similarity&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;start&lt;/strong&gt; - Start position of the feature, with sequence numbering starting at 1.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;end&lt;/strong&gt; - End position of the feature, with sequence numbering starting at 1.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;score&lt;/strong&gt; - A floating point value.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;strand&lt;/strong&gt; - defined as + (forward) or - (reverse).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;frame&lt;/strong&gt; - One of '0', '1' or '2'. '0' indicates that the first base of the feature is the first base of a codon, '1' that the second base is the first base of a codon, and so on..&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;attribute&lt;/strong&gt; - A semicolon-separated list of tag-value pairs, providing additional information about each feature.&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can find more information on it &lt;a href=&quot;http://gmod.org/wiki/GFF2&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://www.ensembl.org/info/website/upload/gff.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;IF you don´t have the file and just have the query output, you probably will find the answer &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2408/extracting-expression-data-from-gse-dataset-downloaded-from-geo&quot;&gt;here&lt;/a&gt;, where extraction of expression data has been discussed in more detail.&lt;/p&gt;&#xA;" OwnerUserId="4306" LastEditorUserId="4306" LastEditDate="2019-03-24T00:13:19.907" LastActivityDate="2019-03-24T00:13:19.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7317" PostTypeId="1" CreationDate="2019-03-24T10:48:11.577" Score="1" ViewCount="40" Body="&lt;p&gt;In stochastic (probabilistic) models, the evolution of a system over time is calculated using a &amp;quot;Master Equation&amp;quot; which also factors the external and internal noise / perturbations in order to more accurately replicate the behavior of a biological system (e.g. cell growth). I am unable to understand what is it, specifically, about the &amp;quot;Master Equation&amp;quot; that allows computational biologists to replicate a biological network with noise?&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Master_equation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Master Equation&lt;/a&gt; is the phenominological description of a non-stationary (i.e., evolving with time) process.&lt;/p&gt;&#xA;" OwnerUserId="4996" LastEditorUserId="73" LastEditDate="2022-06-21T09:59:10.267" LastActivityDate="2022-06-21T09:59:10.267" Title="Kinetics &quot;Master Equation&quot;" Tags="|networks|algorithms|literature-search|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7318" PostTypeId="1" CreationDate="2019-03-24T18:41:23.613" Score="3" ViewCount="174" Body="&lt;p&gt;I did RNA-seq of mammalian cells infected with pox virus. Now, I have read files which contain both host and virus reads. I want to align the reads both to host and viral genome. I was thinking I could concatenate the host and virus genome into one file and run &lt;a href=&quot;https://combine-lab.github.io/salmon/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Salmon&lt;/a&gt; against this concatenated genome.&lt;/p&gt;&#xA;&lt;p&gt;However, salmon recommends transcriptome file for assembly which are not available for viruses. Virus genome are available as GenBank or GFF3 format from NCBI. Is there any way I can concatenate these formats into the format that can be used by salmon? Or is there any way around to use virus genome as reference in salmon?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Citing the website:&lt;/p&gt;&#xA;&lt;p&gt;Salmon is a tool for quantifying the expression of transcripts using RNA-seq data. Salmon uses new algorithms (specifically, coupling the concept of quasi-mapping with a two-phase inference procedure) to provide accurate expression estimates very quickly (i.e. wicked-fast) and while using little memory.&lt;/p&gt;&#xA;&lt;p&gt;Patro, R., Duggal, G., Love, M. I., Irizarry, R. A., &amp;amp; Kingsford, C. (2017). Salmon provides fast and bias-aware quantification of transcript expression. Nature Methods.&lt;/p&gt;&#xA;" OwnerUserId="1684" LastEditorUserId="3848" LastEditDate="2022-03-05T03:20:40.303" LastActivityDate="2022-03-05T03:20:40.303" Title="RNA-seq analysis of mixed viral/host reads with salmon" Tags="|rna-seq|covid-19|salmon|virus|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7319" PostTypeId="2" ParentId="7318" CreationDate="2019-03-24T18:50:41.480" Score="2" Body="&lt;p&gt;You provide Salmon with a transcriptome fasta... so merging the human transcriptome with the pox virus genome fasta file should work. You don't mention that in your post but the viral genome should exist as a fasta. You can extract the sequences from genbank or maybe you can find our virus here: &lt;a href=&quot;https://www.ebi.ac.uk/genomes/virus.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/genomes/virus.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find the coding transcriptome at the link below in the &lt;strong&gt;cdna&lt;/strong&gt; folder.&#xA;The non-coding transcriptome is in the &lt;strong&gt;ncrna&lt;/strong&gt;. You should probably merge them:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;ftp://ftp.ensembl.org/pub/release-95/fasta/homo_sapiens/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ensembl.org/pub/release-95/fasta/homo_sapiens/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2019-03-24T18:50:41.480" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7320" PostTypeId="2" ParentId="4182" CreationDate="2019-03-24T21:22:58.737" Score="1" Body="&lt;p&gt;If you have information on which part of the protein are interacting, and its sequence, you can search for homologous or similar sequences in other proteins. This would give you a list of potential targets and probably a similarity score which you can also use. Then you would have something to run simulations on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I think that with 3D structure you should be able to run simulations &lt;em&gt;in silico&lt;/em&gt; in this case (eventhough it is probably more common to look for the ligands to the protein, not proteins to the ligand). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;In silico&lt;/em&gt; simulations are computationally intensive, but I´ve also seen some being run on a laptop, so you don´t necessarily need access to HPC-clusters, or supercomputer. &lt;/p&gt;&#xA;" OwnerUserId="4306" LastEditorUserId="4306" LastEditDate="2019-03-25T11:29:04.333" LastActivityDate="2019-03-25T11:29:04.333" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7321" PostTypeId="2" ParentId="527" CreationDate="2019-03-25T00:56:11.580" Score="6" Body="&lt;p&gt;If you are still interested, last year miRBase generated new updates. Currently, according to &lt;a href=&quot;ftp://mirbase.org/pub/mirbase/CURRENT/&quot; rel=&quot;noreferrer&quot;&gt;ftp site&lt;/a&gt; the last release is 22.1. So, it is not a dead project and for more specific information you should reference the &lt;a href=&quot;http://www.mirbase.org/blog/&quot; rel=&quot;noreferrer&quot;&gt;miRBase blog&lt;/a&gt;.&#xA;In order to get the best set of miRNA annotations, you have to define your target species or even if exists some related annotated species and then, retrieve the current annotation along different miRNA databases, but most importantly you have to be really clear about the definition of miRNA families, because it is clearly different in &lt;a href=&quot;http://www.mirbase.org/blog/2011/04/whats-in-a-name/&quot; rel=&quot;noreferrer&quot;&gt;miRBase&lt;/a&gt; and also in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4383904/&quot; rel=&quot;noreferrer&quot;&gt;RFAM&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4314" LastActivityDate="2019-03-25T00:56:11.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7322" PostTypeId="1" AcceptedAnswerId="7323" CreationDate="2019-03-25T13:40:23.227" Score="1" ViewCount="485" Body="&lt;p&gt;I have numerous gzip-compressed paired-end Fastq files with ChIP-seq data that I would like to align with bowtie2. I confirmed that bowtie2 takes &lt;code&gt;.gz&lt;/code&gt; files, although I do see this warning when I run for two samples:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Warning: gzbuffer added in zlib v1.2.3.5. Unable to change buffer size&#xA;  from default of 8192.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My files are named like so.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;10_S10_L002_R1_001.fastq.gz  12_S12_L002_R1_001.fastq.gz  2_S2_L001_R1_001.fastq.gz  4_S4_L001_R1_001.fastq-003.gz  6_S6_L001_R1_001.fastq-002.gz  8_S8_L002_R1_001.fastq.gz  &#xA;10_S10_L002_R2_001.fastq.gz  12_S12_L002_R2_001.fastq.gz  2_S2_L001_R2_001.fastq.gz  4_S4_L001_R2_001.fastq-002.gz  6_S6_L001_R2_001.fastq-003.gz  8_S8_L002_R2_001.fastq.gz  &#xA;11_S11_L002_R1_001.fastq.gz  1_S1_L001_R1_001.fastq.gz    3_S3_L001_R1_001.fastq.gz  5_S5_L001_R1_001.fastq-002.gz  7_S7_L002_R1_001.fastq.gz      9_S9_L002_R1_001.fastq.gz&#xA;11_S11_L002_R2_001.fastq.gz  1_S1_L001_R2_001.fastq.gz    3_S3_L001_R2_001.fastq.gz  5_S5_L001_R2_001.fastq-003.gz  7_S7_L002_R2_001.fastq.gz      9_S9_L002_R2_001.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The bash commands I tried are below. When I run this test command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in $(ls *.fastq*.gz | grep -v 6 | rev | cut -c 16- | rev | uniq); do &#xA;    echo $i; &#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;...I get this output.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;12_S12_L002_R1_001.fastq.gz  12_S12_L002_R2_001.fastq.gz &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I run this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in $(ls *.fastq*.gz | rev | cut -c 16- | rev | uniq); do &#xA;    echo $i &#xA;    bowtie2 -q -p20 -x /run/media/punit/data1/BowtieIndex/hg38 -1 ${i}R1_001.fastq.gz -2 ${i}R2_001.fastq.gz | samtools view -bS - &amp;gt;${i%}.bam&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I do see one BAM for my paired end files. But as I see the second set of files which are named:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;6_S6_L001_R1_001.fastq-002.gz  6_S6_L001_R2_001.fastq-003.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;if I run the same script over them, I would get an error as the loop can't read the paired end file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the tedious way is I can make a separate folder for these sets of data with &lt;code&gt;fastq-002.gz&lt;/code&gt; and &lt;code&gt;fastq-003.gz&lt;/code&gt; and can run the script by tweaking the &lt;code&gt;cut&lt;/code&gt; parameter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I would like to know what is the programmatic way of resolving the issue.&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="96" LastEditDate="2019-03-26T18:39:40.460" LastActivityDate="2019-03-26T18:39:40.460" Title="Batch alignment of inconsistently named Fastq files" Tags="|read-mapping|fastq|shell|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7323" PostTypeId="2" ParentId="7322" CreationDate="2019-03-25T15:53:44.790" Score="3" Body="&lt;p&gt;A very simple approach, but one which will work with the names you show, is to remove any occurrence of &lt;code&gt;00N&lt;/code&gt; (where &lt;code&gt;N&lt;/code&gt; is any number) from right before the &lt;code&gt;.gz&lt;/code&gt;. To do so, edit your script to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in &lt;span class=&quot;math-container&quot;&gt;$(ls *.fastq*.gz | sed 's/00[0-9]\.gz/.gz/' | rev | cut -c 16- | rev | uniq); do &#xA;  bowtie2 -q -p20 -x /run/media/punit/data1/BowtieIndex/hg38 \&#xA;    -1 $&lt;/span&gt;{i}R1_001.fastq.gz \&#xA;    -2 &lt;span class=&quot;math-container&quot;&gt;${i}R2_001.fastq.gz | &#xA;        samtools view -bS - &amp;gt;$&lt;/span&gt;{i%}.bam&quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-03-25T15:53:44.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7324" PostTypeId="1" AcceptedAnswerId="7355" CreationDate="2019-03-25T16:19:30.267" Score="3" ViewCount="126" Body="&lt;p&gt;Medical Subject Headings (MeSH) is a very comprehensive vocabulary to index the biomedical literature on Medline or PubMed (which is the major Biomedical literature database). MeSH is arranged into Descriptor &gt; Concept &gt; term hierarchy, but I still do not understand the difference between a MeSH descriptor and a MeSH concept.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, for Knee&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;MeSH descriptor is &lt;a href=&quot;https://meshb.nlm.nih.gov/record/ui?ui=D007717&quot; rel=&quot;nofollow noreferrer&quot;&gt;D007717&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;MeSH concept is &lt;a href=&quot;https://meshb.nlm.nih.gov/record/ui?ui=D007717&quot; rel=&quot;nofollow noreferrer&quot;&gt;M0012066&lt;/a&gt; (Under the concepts section in the link)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Even after reading the description &lt;a href=&quot;https://www.nlm.nih.gov/mesh/concept_structure.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://www.nlm.nih.gov/mesh/xmlmesh.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, I am still not sure about the purpose of MeSH concept and MeSH descriptor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anyone please shed some light on it?&lt;/p&gt;&#xA;" OwnerUserId="3015" LastEditorUserId="96" LastEditDate="2019-03-25T20:04:10.730" LastActivityDate="2019-03-29T10:03:15.887" Title="What is the difference between a MeSH concept and a MeSH descriptor" Tags="|ontology|literature|nomenclature|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7325" PostTypeId="1" AcceptedAnswerId="7327" CreationDate="2019-03-25T17:22:56.150" Score="3" ViewCount="347" Body="&lt;p&gt;I am currently working on multiple datasets where each is managed by a separate &lt;code&gt;Seurat&lt;/code&gt; object. For each I'd like to also compute the marker genes using the &lt;code&gt;FindAllMarkers&lt;/code&gt; function. Where would it be a good practice to save the markers returned from the &lt;code&gt;FindAllMarkers&lt;/code&gt; function? I tried adding an extra slot to the &lt;code&gt;Seurat&lt;/code&gt; object with no success.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I saw the &lt;code&gt;calc.params&lt;/code&gt; slot and though I could add an item to that list. However, it doesn't seem like the right place as that slot is a list of the parameters used for the different functions executed on the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I have some wrapping object that has a slot for the &lt;code&gt;Seurat&lt;/code&gt; object and a slot for the results from the &lt;code&gt;FindAllMarkers&lt;/code&gt;. Is there any better place to put it? Somewhere within the &lt;code&gt;Seurat&lt;/code&gt; object?&lt;/p&gt;&#xA;" OwnerUserId="3501" LastEditorUserId="3501" LastEditDate="2019-03-26T10:18:33.717" LastActivityDate="2019-03-26T13:40:41.073" Title="Storing FindAllMarkers results in Seurat object" Tags="|seurat|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7326" PostTypeId="1" AcceptedAnswerId="7329" CreationDate="2019-03-26T13:20:38.173" Score="0" ViewCount="567" Body="&lt;p&gt;Whatever I am googling I am not able to find fasta file of each chromosome for hs37d5.fa. hs37d5.fa whole fasta file however is here &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/phase2_reference_assembly_sequence/hs37d5.fa.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/phase2_reference_assembly_sequence/hs37d5.fa.gz&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know where I can get fasta file for each chromosome for this genome?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="96" LastEditDate="2019-03-26T14:07:32.767" LastActivityDate="2019-03-26T14:08:43.253" Title="Downloading Fasta for each chromosome of a complete genome" Tags="|fasta|reference-genome|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7327" PostTypeId="2" ParentId="7325" CreationDate="2019-03-26T13:40:41.073" Score="3" Body="&lt;p&gt;You can stash anything you like in the &lt;code&gt;misc&lt;/code&gt; slot (present in both v2 and v3 Seurat objects). I often use it for storing marker information to help with organization. &lt;/p&gt;&#xA;" OwnerUserId="3415" LastActivityDate="2019-03-26T13:40:41.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7328" PostTypeId="2" ParentId="7326" CreationDate="2019-03-26T13:58:10.673" Score="2" Body="&lt;p&gt;It is common that reference genomes will be published only as a complete genome, without separate files for each chromosome. Fortunately, a google search for &quot;split fasta by chromosome&quot; will give you many options for downloading a complete genome Fasta file and splitting it by chromosome. The top answer on &lt;a href=&quot;https://www.biostars.org/p/173723/#173819&quot; rel=&quot;nofollow noreferrer&quot;&gt;this biostar thread&lt;/a&gt; looks promising.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pip install pyfaidx&#xA;pyfaidx -x sequences.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Alternatively, I typically use a homebrew Python script. If there are unlocalized/unplaced scaffolds in the file, this will create a bunch of files with ugly unexpected names. These are easy to ignore or delete.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;# usage: ./split.py &amp;lt; genome.fasta&#xA;from Bio import SeqIO&#xA;import sys&#xA;reader = SeqIO.parse(sys.stdin, 'fasta')&#xA;for record in reader:&#xA;    outfile = 'chr{chrid}.fasta'.format(chrid=record.id)&#xA;    with open(outfile, 'w') as outstream:&#xA;        count = SeqIO.write(record, outstream, 'fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/help/how-to-ask&quot;&gt;With a little bit of searching&lt;/a&gt;, you will find a wealth of helpful information on blogs, forums, and library documentation.&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2019-03-26T14:04:29.027" LastActivityDate="2019-03-26T14:04:29.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7329" PostTypeId="2" ParentId="7326" CreationDate="2019-03-26T14:08:43.253" Score="2" Body="&lt;p&gt;Personally, I would download the entire file and split it using something like &lt;code&gt;fastaexplode&lt;/code&gt; from the &lt;code&gt;exonerate&lt;/code&gt; package. On an Ubuntu machine, you can do that with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wget ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/phase2_reference_assembly_sequence/hs37d5.fa.gz&#xA;gunzip hs37d5.fa.gz&#xA;sudo apt install exonerate&#xA;fastaexplode hs37d5.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you could do this on the fly while downloading:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wget -O - ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/phase2_reference_assembly_sequence/hs37d5.fa.gz | &#xA;  gunzip | awk '/&amp;gt;/{name=$1; sub(/&amp;gt;/,&quot;&quot;,name)} {print &amp;gt;&amp;gt; name&quot;.fa&quot;}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-03-26T14:08:43.253" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7330" PostTypeId="1" CreationDate="2019-03-26T14:47:58.260" Score="1" ViewCount="166" Body="&lt;p&gt;Is it possible that on the same &lt;a href=&quot;https://pysam.readthedocs.io/en/latest/api.html?highlight=pileupcolumn#pysam.PileupColumn&quot; rel=&quot;nofollow noreferrer&quot;&gt;pileupcolumn&lt;/a&gt; object (i.e a specific aligned base) I find the same base but aligned to different strands? For example, can I find a coverage of 50 'A's that align to strand '+', and on the same time 50 'A's that align to strand '-' within the same index? I expect that if 'A' is aligned to strand '+', I will find 'T' aligned to strand '-'.&lt;/p&gt;&#xA;" OwnerUserId="4321" LastEditorUserId="96" LastEditDate="2019-03-26T18:00:36.563" LastActivityDate="2019-03-26T18:04:26.853" Title="Aligned base strand in pysam `pileupcolumn`" Tags="|python|pysam|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7331" PostTypeId="1" AcceptedAnswerId="20203" CreationDate="2019-03-26T15:39:31.977" Score="2" ViewCount="28" Body="&lt;p&gt;Chromosome capture experiments, like Hi-C, typically bin the genome into distinct bins. A contact matrix, &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt;, is given by &lt;span class=&quot;math-container&quot;&gt;$M_{i,j} = |\{pairs | mate_1 \in bin_i, mate_2 \in bin_j\}|$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If one of the read mates spans a bin boundary, where should that pair be assigned?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, if &lt;span class=&quot;math-container&quot;&gt;$mate_1$&lt;/span&gt; overlaps &lt;span class=&quot;math-container&quot;&gt;$bin_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$mate_2$&lt;/span&gt; overlaps both &lt;span class=&quot;math-container&quot;&gt;$bin_j$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$bin_{j+1}$&lt;/span&gt;, should this read pair be assigned to &lt;span class=&quot;math-container&quot;&gt;$M_{i,j}$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$M_{i,j+1}$&lt;/span&gt;? Should it be randomly assigned to either bin with equal probability? Should it be assigned to both?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know this may be a rare event, but if Hi-C datasets are generated using billions of reads (such as &lt;a href=&quot;https://doi.org/10/xqj&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rao &lt;em&gt;et al.&lt;/em&gt;, Cell, 2014&lt;/a&gt;), this is bound to happen.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2022-12-18T05:57:40.153" Title="Where to assign a read pair when a mate spans a HiC bin boundary?" Tags="|hi-c|chromosome-capture|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7332" PostTypeId="1" CreationDate="2019-03-26T16:17:17.837" Score="2" ViewCount="651" Body="&lt;p&gt;I have a vcf file with individual level genetic data, that I read using the R package &lt;code&gt;vcfR&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gen &amp;lt;- read.vcfR('test.vcf')&#xA;&#xA;Scanning file to determine attributes.&#xA;File attributes:&#xA;  meta lines: 9&#xA;  header_line: 10&#xA;  variant count: 152&#xA;  column count: 3798&#xA;Meta line 9 read in.&#xA;All meta lines processed.&#xA;gt matrix initialized.&#xA;Character matrix gt created.&#xA;  Character matrix gt rows: 152&#xA;  Character matrix gt cols: 3798&#xA;  skip: 0&#xA;  nrows: 153&#xA;  row_num: 0&#xA;Processed variant: 153&#xA;All variants processed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How do I convert this file to a data frame that I can use for a GWAS or Mendelian Randomization study?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was told that this should be a data frame where each column is a genetic variant and it take values 0, 1, or 2, depending on the number of &quot;mutations&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I never worked with genetic data before and do not know much about genetics.&lt;/p&gt;&#xA;" OwnerUserId="4322" LastActivityDate="2019-03-27T15:15:33.530" Title="how do I convert a vcf file for a GWAS study (using R package vcfR)" Tags="|r|vcf|gwas|data-management|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7333" PostTypeId="2" ParentId="7330" CreationDate="2019-03-26T18:04:26.853" Score="1" Body="&lt;p&gt;As @terdon mentions, it's certainly possible to have an &lt;code&gt;A&lt;/code&gt; aligned on both strands. If a read has a mismatch at the location of interest and that mismatch is the complement of the reference base, then you are likely to see &lt;code&gt;A&lt;/code&gt; in both strands. However, it is very unlikely you will see the same base &lt;em&gt;at high coverage&lt;/em&gt; in both strands. For example, if your total coverage is 55, and you see 50 &lt;code&gt;A&lt;/code&gt;s on the + strand and 50 &lt;code&gt;A&lt;/code&gt;s on the - strand, it's likely that the pileup object is reporting the reverse complement sequence of the reads that align to the - strand for your convenience.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-03-26T18:04:26.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7334" PostTypeId="1" CreationDate="2019-03-26T17:30:59.607" Score="0" ViewCount="293" Body="&lt;p&gt;I am exploring application of Graph convolution network on gene similarity network. I am trying to explore&#xA;genemania.org and &lt;a href=&quot;https://string-db.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://string-db.org/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please let me know where can I find the data for my research in gene interaction network, which allows me to download the data in the form of an adjacency matrix.&lt;/p&gt;&#xA;" OwnerDisplayName="Susha Suresh" LastEditorUserId="96" LastEditDate="2019-03-26T18:54:25.813" LastActivityDate="2019-08-08T06:34:04.827" Title="How do I get the adjacency matrix from gene similarity network?" Tags="|gene|networks|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7335" PostTypeId="2" ParentId="7160" CreationDate="2019-03-26T23:14:48.967" Score="3" Body="&lt;p&gt;I think the closest approximation for a SNP chip would be the intensity values.  When I asked 23andMe about obtaining these, the answer was that they don't plan on adding this feature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some programs use the .vcf file format instead of the 23andMe format, but I think the 23andMe format is compatible with relatively more 3rd party software than some other vendors.  If I remember correctly, I think &lt;a href=&quot;https://dna.land/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DNA.land&lt;/a&gt; even has an intermediate .vcf file, which you can download.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I actually think it is important that 23andMe &lt;em&gt;doesn't&lt;/em&gt; impute the variants in your genotype table.  However, if you are trying to dig deeper into some SNP chip data, I believe you can still get genotyping for free from &lt;a href=&quot;https://genesforgood.sph.umich.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genes for Good&lt;/a&gt;.  They provide a few different &quot;raw data&quot; formats (such as .vcf files, with and without imputation, as well as a &quot;23andMe format&quot; file) and I got the impression that they would consider providing an even more raw form of data (to test the effect of generating genotypes on your own, even thought I don't think most sites will change).  However, to be clear, they also do not currently provide intensity values (or some other even more raw form of data).&lt;/p&gt;&#xA;" OwnerUserId="4264" LastActivityDate="2019-03-26T23:14:48.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7336" PostTypeId="1" CreationDate="2019-03-27T00:45:57.037" Score="6" ViewCount="2570" Body="&lt;p&gt;What is the reason for adding hydrogen and removing unnecessary water molecules from the protein structure before protein-ligand docking? FYI, the tools I used for docking is GOLD.&lt;/p&gt;&#xA;" OwnerUserId="3760" LastEditorUserId="96" LastEditDate="2019-03-27T13:03:18.517" LastActivityDate="2020-09-09T06:33:41.630" Title="Why is it necessary to add hydrogen and delete water before protein-ligand docking?" Tags="|protein-structure|docking|structural-biology|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7337" PostTypeId="1" CreationDate="2019-03-27T00:55:17.443" Score="13" ViewCount="284" Body="&lt;p&gt;There have been a few methods proposed for integration (or batch correction) of scRNA-seq datasets, such as Seurat CCA, MNN Correct, Scanorama, and Harmony. The concern is generally about the maximum number of cells that they handle, but I haven't seen any discussion about the minimum number of cells. I am confident they can all handle 10k cells reasonably well and will fail with 10 cells, but where do you draw the line? Is there a method that works best for small datasets?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, with plate-based platforms like Fluidigm, many experiments only have 96 cells and potentially much less after quality filtering. How can those be used?&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="35" LastEditDate="2019-03-27T23:30:23.277" LastActivityDate="2021-04-12T15:31:45.030" Title="scRNA-seq multi-dataset integration for small datasets" Tags="|scrnaseq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7339" PostTypeId="1" CreationDate="2019-03-27T07:05:49.567" Score="6" ViewCount="42" Body="&lt;p&gt;I've done my 1st trial of 1D^2 whole genome sequencing using LSK-SQK308 kit with R9.5 flowcell. Without doing library fragmentation, I encountered the unexpected library shearing during sequencing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MdRPp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MdRPp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I didn't see significant portion of low-molecular-weight fragments in the QC gel photo before and after library preparation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MQuY6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MQuY6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone using nanopore has encountered this before? Any explanation? And how to avoid it? I could not find evidence of the problem in my QC steps...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="4325" LastEditorUserId="96" LastEditDate="2019-03-27T13:06:37.037" LastActivityDate="2021-05-06T10:09:19.373" Title="Help with 1D^2 library shearing" Tags="|nanopore|quality-control|1d2-reads|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7340" PostTypeId="1" AcceptedAnswerId="10432" CreationDate="2019-03-27T12:31:19.730" Score="4" ViewCount="164" Body="&lt;p&gt;I have Illumina MiSeq paired-end reads from 150bp amplicons mapped to my reference genome (&gt; 1000X coverage). These reads have indels that may or may not induce frameshifts. If the indel induces a frameshift (i.e. indel is not a multiple of 3bp), I want to know if that leads to a stop codon later in the read. Is there any package/method that would allow me to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Few extra details&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I might be wrong but I think this is different than gene prediction as I would want the tool to scan each read in the correct frame; not try different frames to detect ORFs etc.&lt;/li&gt;&#xA;&lt;li&gt;I expect mosaicism, so I do not want to compute a single assembly/consensus and predict stop codons in it.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Here is how I imagine such a script/tool would work:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Take the position of the first translated codon of the exon I am interested in (from Ensembl for instance?). &lt;em&gt;Eg. gene abc/exon1 starts at chr1:1000.&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Then for each read:&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Look at which position the read is mapped to my reference. &lt;em&gt;Eg. read 1 is mapped to chr1:1050.&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;Compute the position of the first full codon in the correct frame in my read. &lt;em&gt;Eg. start codon at chr1:1000 but my read starts at chr1:1050, the first full translated codon in my read should start at chr1:1052 (I think?).&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;Split all my read into 3-mers starting at that position and build something like a 3-mer/position dictionnary. &lt;/li&gt;&#xA;&lt;li&gt;Look for stop codons and report the positions.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Does it exist?&lt;/p&gt;&#xA;" OwnerUserId="868" LastEditorUserId="868" LastEditDate="2019-03-27T15:04:43.903" LastActivityDate="2019-09-18T15:35:52.823" Title="How to predict stop codons in Illumina reads?" Tags="|sequence-alignment|sequence-annotation|illumina|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7341" PostTypeId="2" ParentId="7250" CreationDate="2019-03-27T14:53:39.453" Score="0" Body="&lt;p&gt;Here is a clue by scatngs&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/cancerit/ascatNgs/wiki/Human-reference-files-from-1000-genomes-VCFs&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/cancerit/ascatNgs/wiki/Human-reference-files-from-1000-genomes-VCFs&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and already calculated file is here &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;ftp://ftp.sanger.ac.uk/pub/cancer/support-files/CPIB/ascatNgs/Human/GRCh37/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.sanger.ac.uk/pub/cancer/support-files/CPIB/ascatNgs/Human/GRCh37/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although installing scatngs itself is a big burden &lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2019-03-27T14:53:39.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7342" PostTypeId="2" ParentId="7332" CreationDate="2019-03-27T15:15:33.530" Score="3" Body="&lt;p&gt;If you want to do GWAS or mendelian randomisation, you can do it with &lt;a href=&quot;https://www.cog-genomics.org/plink/2.0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Plink&lt;/a&gt; (v2) which should be faster than R.  &lt;/p&gt;&#xA;" OwnerUserId="4306" LastActivityDate="2019-03-27T15:15:33.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7343" PostTypeId="2" ParentId="7340" CreationDate="2019-03-27T18:29:38.503" Score="1" Body="&lt;p&gt;With 150bp reads, you might be able to blastx all the unique sequences you've got against the target proteins.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-03-27T18:29:38.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7344" PostTypeId="1" CreationDate="2019-03-28T05:13:39.857" Score="3" ViewCount="539" Body="&lt;p&gt;I am trying to reproduce the work of &lt;a href=&quot;https://www.nature.com/articles/nprot.2016.095&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;&#xA;&lt;a href=&quot;https://www.nature.com/articles/nprot.2016.095&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1]&lt;/a&gt;, and I have run StringTie successfully, but after that I have to run Ballgown but could not understand this command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;pheno_data = read.csv(&quot;geuvadis_phenodata.csv&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Actually, I dont know how to make the .csv file of my samples. What phenotype information should this csv contain?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have checked the supplementary data of this research article but didn't find any .csv file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How I can make the .csv file for my samples to load the phenotype data for ballgown?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ol start=&quot;8&quot;&gt;&#xA;  &lt;li&gt;&lt;p&gt;Load the phenotype data for the samples. An example file called&#xA;  geuvadis_phenodata.csv is included with the data files for this protocol&#xA;  (ChrX_data). In general, you will have to create this file yourself. It contains&#xA;  information about your RNA sequencing samples, formatted as illustrated in&#xA;  this csv (comma-separated values) file. Each sample should be described on&#xA;  one row of the file and each column should contain one variable. To read this&#xA;  file into R we use the command read.csv. In this file the values are&#xA;  separated by commas, but if the file were tabdelimited you could use the&#xA;  function read.table.&lt;/p&gt;&#xA;  &#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;pheno_data = read.csv(&quot;geuvadis_phenodata.csv&quot;)&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Pertea &lt;em&gt;et al&lt;/em&gt;, Transcript-level expression analysis of RNA-seq experiments with HISAT, StringTie and Ballgown, &lt;em&gt;Nature Protocols&lt;/em&gt; volume 11, pages 1650–1667 (2016) &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4194" LastEditorUserId="298" LastEditDate="2019-03-28T09:31:13.917" LastActivityDate="2020-09-29T10:06:55.710" Title="load the phenotype data for ballgown" Tags="|rna-seq|bioconductor|differential-expression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7345" PostTypeId="1" CreationDate="2019-03-28T10:26:48.183" Score="1" ViewCount="933" Body="&lt;p&gt;I want to open the expression data GSE9838 dataset from GEO using GEOquery R package.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gset &amp;lt;- getGEO(filename=&quot;GSE9838_family.soft.gz&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result of this code said it's S4 object of class GSE.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I get the expression data of GSE9838?&lt;/p&gt;&#xA;" OwnerUserId="4332" LastEditorUserId="57" LastEditDate="2019-03-28T10:32:50.310" LastActivityDate="2019-03-29T19:00:58.187" Title="Extracting expression data in r from GEO and got a S4 object" Tags="|r|bioconductor|microarray|geoquery|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="7346" PostTypeId="2" ParentId="3181" CreationDate="2019-03-28T15:56:39.460" Score="1" Body="&lt;p&gt;You can use the &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/genomes/TOOLS/add_utrs_to_gff/add_utrs_to_gff.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;add_utrs_to_gff.py&lt;/code&gt;&lt;/a&gt; script provided by NCBI RefSeq for this purpose. Running the script on your example produces the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./add_utrs_to_gff.py example.gff3&#xA;ctg123  .  gene             1050  9000  .  +  .  ID=gene00001;Name=EDEN&#xA;ctg123  .  mRNA             1050  9000  .  +  .  ID=mRNA00001;Parent=gene00001;Name=EDEN.1&#xA;ctg123  .  exon             1050  1500  .  +  .  ID=exon00002;Parent=mRNA00001&#xA;ctg123  .  exon             3000  3902  .  +  .  ID=exon00003;Parent=mRNA00001&#xA;ctg123  .  exon             5000  5500  .  +  .  ID=exon00004;Parent=mRNA00001&#xA;ctg123  .  exon             7000  9000  .  +  .  ID=exon00005;Parent=mRNA00001&#xA;ctg123  .  CDS              1201  1500  .  +  0  ID=cds00001;Parent=mRNA00001;Name=edenprotein.1&#xA;ctg123  .  CDS              3000  3902  .  +  0  ID=cds00001;Parent=mRNA00001;Name=edenprotein.1&#xA;ctg123  .  CDS              5000  5300  .  +  0  ID=cds00001;Parent=mRNA00001;Name=edenprotein.1&#xA;ctg123  .  five_prime_UTR   1050  1200  .  +  .  ID=utr00002;Parent=mRNA00001&#xA;ctg123  .  three_prime_UTR  5301  5500  .  +  .  ID=utr00004;Parent=mRNA00001&#xA;ctg123  .  three_prime_UTR  7000  9000  .  +  .  ID=utr00005;Parent=mRNA00001&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2019-03-28T15:56:39.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7347" PostTypeId="1" AcceptedAnswerId="7352" CreationDate="2019-03-28T16:50:49.220" Score="15" ViewCount="3252" Body="&lt;p&gt;I want to understand the difference between pipeline systems and workflow engines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After reading &lt;a href=&quot;https://link.springer.com/article/10.1007/s41019-017-0047-z&quot; rel=&quot;noreferrer&quot;&gt;A Review of Scalable Bioinformatics Pipelines&lt;/a&gt; I had a good overview of current bioinformatics pipelines. After some further research I found that there is collection of highly capable &lt;a href=&quot;https://github.com/meirwah/awesome-workflow-engines&quot; rel=&quot;noreferrer&quot;&gt;workflow engines&lt;/a&gt;. My question is then based on what I saw for &lt;a href=&quot;https://github.com/argoproj/argo&quot; rel=&quot;noreferrer&quot;&gt;argo&lt;/a&gt;. I would say I can be used as a bioinformatics pipeline as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how do bioinformatics pipelines differ from workflow engines?&lt;/p&gt;&#xA;" OwnerUserId="2026" LastEditorUserId="96" LastEditDate="2019-04-24T15:13:23.717" LastActivityDate="2019-04-24T15:13:23.717" Title="What is the difference between a Bioinformatics pipeline and workflow?" Tags="|software-usage|terminology|workflow-management|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7348" PostTypeId="2" ParentId="7262" CreationDate="2019-03-28T17:54:20.187" Score="1" Body="&lt;p&gt;I've found the problem, and it's way less interesting than I initially thought. Essentially, my &quot;eValue&quot; was way too low. Instead of &lt;code&gt;-evalue 100&lt;/code&gt;, I'm now using &lt;code&gt;-evalue 5000&lt;/code&gt; (Primer-BLAST actually uses 30,000).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some primers, &lt;code&gt;-evalue 5000&lt;/code&gt; gives ~20,000 results. It's beyond the scope of this question, but I'm now filtering on the query/subject Tm for each hit, perfect match at the query 3'-end, and number of query/subject mismatches. My pairwise &#xA;hit comparisons are now detecting plausible off-target amplicons that Primer-BLAST is detecting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, I was relying on NCBI's hit ranking system to determine what hits were relevant, but I needed cast a wider net with blastn, and then implement a rational ranking scheme of my own.&lt;/p&gt;&#xA;" OwnerUserId="3564" LastEditorUserId="3564" LastEditDate="2019-03-28T22:49:50.650" LastActivityDate="2019-03-28T22:49:50.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7349" PostTypeId="1" AcceptedAnswerId="7354" CreationDate="2019-03-28T18:37:57.483" Score="2" ViewCount="148" Body="&lt;p&gt;While I have used my title, it is more of a general statistics question regarding how these significance tests are carried out, these include &lt;code&gt;decideTests&lt;/code&gt; and &lt;code&gt;mroast&lt;/code&gt; / &lt;code&gt;roast&lt;/code&gt; among others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To give a brief overview of my experimental design, I am analysing bulk RNASeq datasets and there are two main factors in the design matrix: &lt;code&gt;patientID&lt;/code&gt; and &lt;code&gt;casetimeViremia&lt;/code&gt;; where the latter identifies whether a patient is viremia positive or negative at a particular time point (1-4). This leaves us with 5 different possibilities for &lt;code&gt;casetimeViremia&lt;/code&gt;: 1-min, 2-plus, 3-min, 3-plus, 4-min.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The design matrix was created in R by running: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;model.matrix(~patientID + casetimeViremia, data = dgeObject$samples)&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;code&gt;dgeObject&lt;/code&gt; is an object of the function &lt;code&gt;DGEList&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the purpose of answering this biological question, we wish to conduct the differential expression testing by seeing whether all the previously mentioned &lt;code&gt;casetimeViremia&lt;/code&gt; timepoints are separately and significantly expressed relative to each person's &lt;em&gt;first&lt;/em&gt; timepoint, which is labelled as: 1-min.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I understood that should I wish to conduct this test, I do not need to specify a contrast matrix in the &lt;code&gt;mroast&lt;/code&gt; function and can just do this by calling the column number of interest.. But in that case, how does &lt;code&gt;mroast&lt;/code&gt; &lt;em&gt;know&lt;/em&gt; that I wish to do the contrast relative to the 1-min timepoint? And, just wondering, should I wish to define a contrast matrix, how would I do this given that there are two &lt;code&gt;colnames&lt;/code&gt; missing from the design, in this case one patientID and the 1-min timepoint?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you all for your time.&lt;/p&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="4001" LastEditDate="2019-03-29T20:25:46.087" LastActivityDate="2019-03-29T20:25:46.087" Title="Significant gene set testing - limma" Tags="|r|rna-seq|statistics|limma|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7350" PostTypeId="1" CreationDate="2019-03-28T18:40:52.673" Score="5" ViewCount="407" Body="&lt;p&gt;There are nicer genomics visualization tools available, but the &lt;code&gt;samtools tview&lt;/code&gt; command is almost always my go-to for a quick first look at read alignments. I just brought up the following locus in &lt;code&gt;tview&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;81055951  81055961  81055971  81055981                                                81055991  81056001  81056011  81056021&#xA;GTGGAGGTCGGCAGCAGAGCCTGTGGCAGCT**********************************************CTGAGGGTCGGCGTGGCCTCCTGGTGGAAGTTGCACCTGGTTTGCTT&#xA;...............................                                              ...............................................&#xA;...............................**********************************************..       ......................................&#xA;   ,,,,,,,,,,,,,,,,,,,,,,,,,,,,**********************************************,,       .....T................................&#xA;.....                  ,,,,,,,,**********************************************,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,&#xA;.......                      ,,**********************************************,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,&#xA;,,,,,,,,,                                                                    ...............................................&#xA;.............                                                                .........................................A.....&#xA;........................                                                     ...............................................&#xA;...............................**********************************************..         ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,&#xA;,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,**********************************************,,           ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,&#xA;...............................**********************************************.A......                 ,,,,,,,,,,,,,,,,,,,,,,&#xA;...............................**********************************************..                         ....................&#xA;,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,**********************************************,,                         ....................&#xA;,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,**********************************************,,,,,,,,,,,,,,,,,,,,,,,,,,,,    ,,,,,,,,,,,,,,,&#xA;,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,**********************************************,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,        ,,,,,&#xA;...............................**********************************************..                               ,,,,,,,,,,,,,,&#xA;...............................**********************************************..                                             &#xA;...............................**********************************************..                                             &#xA;,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,**********************************************,,                                             &#xA;,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,**********************************************,,                                             &#xA;..................T............**********************************************...............................................&#xA;...............................**********************************************...............................................&#xA;...............................**********************************************...............................................&#xA;,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,**********************************************,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,&#xA;...............................**********************************************...............................................&#xA;...............................**********************************************...............................................&#xA;,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,**********************************************,,                                             &#xA;..............G................**********************************************...............................................&#xA;,,c,,,,,,,,,,,,,,,,,,,,,,,,,,,,**********************************************,,                                             &#xA;...............................CTAAGCAAGTAACTGTGGTGTTAGAACCTACCTCCCTCGCCGGGCA...............................................&#xA;...............................**********************************************...............................................&#xA;...............................**********************************************........................................A......&#xA;...............................**********************************************...........................................T...&#xA;,,,,,,,,,,,,,,,,,,,,,g,,,,,,,,,**********************************************,,                                             &#xA;,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,**********************************************,,                                             &#xA;                                                                             ...............................................&#xA;                                                                             ...............................................&#xA;                                                                             ...............................................&#xA;                                                                             ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,&#xA;                                                                             ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,a, &#xA;                                                                             ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,                &#xA;                                                                             ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,&#xA;                                                                             ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,&#xA;                                                                             ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;At first glance there appears to be only a single read spanning this insertion, although the abrupt end of many reads at the insertion breakpoints made me suspicious. Indeed, upon closer inspection, 19 additional reads have primary alignments with soft clipping at the insertion breakpoints.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By default, soft-clipped portions of the reads are not displayed. &lt;strong&gt;Is there any way to toggle this behavior?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2021-12-17T01:07:20.717" Title="Displaying soft-clipped nucleotides in samtools tview" Tags="|samtools|read-mapping|visualization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7351" PostTypeId="2" ParentId="7345" CreationDate="2019-03-28T19:06:35.613" Score="1" Body="&lt;p&gt;Well, if you use a SOFT formatted files:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#You can read your file &#xA;gds &amp;lt;- getGEO(filename=&quot;GSE9838_family.soft.gz&quot;)&#xA;&#xA;# and check it&#xA;class(gds)&#xA;show(gds)&#xA;&#xA;#then you usually would try to use  an exprs() accessor to retrieve the expression matrix:&#xA;exprs (gds[[1]])&#xA;&#xA;# but in this case you will get an error revealing that it is a GSE datastructure&#xA;# eg. is composite structure having GSM and GPL object lists: &#xA;names(GSMList(gds))&#xA;names(GPLList(gds))&#xA;&#xA;# now GPLList is probably not interesting in this case&#xA;# but you can look at the GSM-objects from GSMList:&#xA;first &amp;lt;- GSMList(gds)[[1]]&#xA;first&#xA;class(first)&#xA;Meta(first)&#xA;&#xA;# at this point if you read the results you´ll notice that &#xA;#    title &#xA;#    [1] &quot;X_intercept&quot;&#xA;&#xA;# and &#xA;#   An object of class &quot;GEODataTable&quot;&#xA;#   ****** Column Descriptions ******&#xA;#   [1] Column      Description&#xA;#   &amp;lt;0 rows&amp;gt; (or 0-length row.names)&#xA;#   ****** Data Table ******&#xA;#   data frame with 0 columns and 0 rows&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So here you would probably wonder if the data you want is these objects. If you go back go GEO page for GSE9838, you can find that &lt;em&gt;&quot;Raw data provided as supplementary file. Processed data is available on Series record&quot;&lt;/em&gt;. So one may want to check series record instead:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# you can query the GSE9838 like this: &#xA;data = getGEO(&quot;GSE9838&quot;)    &#xA;&#xA;# then you can check what you´ve got&#xA;names(data)&#xA;#[1] &quot;GSE9838_series_matrix.txt.gz&quot;&#xA;class(data$GSE9838_series_matrix.txt.gz)&#xA;&#xA;# and check what´s inside:&#xA;exprs(data$GSE9838_series_matrix.txt.gz)&#xA;# but apparently it just shows one header row. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So it wasn´t there either. And if you check the size of the file, you´ll find that it is just 4.8 kb. My educated guess would be that the expression values are not in the dataset itself but probably in some of the supplementary files?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to download them, you can:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;filePaths = getGEOSuppFiles(&quot;GSE9838&quot;)&#xA;filePaths&#xA;untar(&quot;GSE9838/GSE9838_RAW.tar&quot;, exdir = &quot;GSE9838/CEL&quot;)&#xA;list.files(&quot;GSE9838/CEL&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;...what will allow you to see all the CEL files with the data. &lt;/p&gt;&#xA;" OwnerUserId="4306" LastEditorUserId="4306" LastEditDate="2019-03-29T19:00:58.187" LastActivityDate="2019-03-29T19:00:58.187" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7352" PostTypeId="2" ParentId="7347" CreationDate="2019-03-29T02:51:12.670" Score="18" Body="&lt;p&gt;Great question! Note that from a prescriptive standpoint, the terms &lt;strong&gt;&lt;em&gt;pipeline&lt;/em&gt;&lt;/strong&gt; and &lt;strong&gt;&lt;em&gt;workflow&lt;/em&gt;&lt;/strong&gt; don't have any strict or precise definitions. But it's still useful to take a descriptive standpoint and discuss how the terms are commonly used in the bioinformatics community.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But before talking about pipelines and workflows, it's helpful to talk about programs and scripts. A &lt;strong&gt;&lt;em&gt;program&lt;/em&gt;&lt;/strong&gt; or &lt;strong&gt;&lt;em&gt;script&lt;/em&gt;&lt;/strong&gt; typically implements a single data analysis task (or set of related tasks). Some examples include the following.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.bioinformatics.babraham.ac.uk/projects/fastqc/&quot; rel=&quot;noreferrer&quot;&gt;FastQC&lt;/a&gt;, a program that checks NGS reads for common quality issues&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.usadellab.org/cms/?page=trimmomatic&quot; rel=&quot;noreferrer&quot;&gt;Trimmomatic&lt;/a&gt;, a program for cleaning NGS reads&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://combine-lab.github.io/salmon/&quot; rel=&quot;noreferrer&quot;&gt;salmon&lt;/a&gt;, a program for estimating transcript abundance from NGS reads&lt;/li&gt;&#xA;&lt;li&gt;a custom R script that uses &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/DESeq2.html&quot; rel=&quot;noreferrer&quot;&gt;DESeq2&lt;/a&gt; to perform differential expression analysis&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A &lt;strong&gt;&lt;em&gt;pipeline&lt;/em&gt;&lt;/strong&gt; or a &lt;strong&gt;&lt;em&gt;workflow&lt;/em&gt;&lt;/strong&gt; refers to a particular kind of program or script that is intended primarily to &lt;em&gt;combine other independent programs or scripts&lt;/em&gt;. For example, I might want to write an RNA-seq workflow that executes Trimmomatic, FastQC, salmon, and the R script using a single command. This is particularly useful if I have to run the same command many times, or if the commands take a long time to run. It's very inconvenient when you have to babysit your computer and wait for step 3 to finish so that you can launch step 4!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So when does a program become a pipeline? Honestly, there are no strict rules. In some cases it's clear: the 10-line Python script I wrote to split Fasta files is definitely &lt;em&gt;NOT&lt;/em&gt; a pipeline, but the 200-line Python script I wrote that does nothing but invoke 6 other bioinformatics programs definitely &lt;em&gt;IS&lt;/em&gt; a pipeline. There are a lot of tools that fall in the middle: they may require running multiple steps in a certain order, or implement their own processing but also delegate processing to other tools. Usually nobody worries too much about whether it's &quot;correct&quot; to call a particular tool a pipeline.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, a &lt;strong&gt;&lt;em&gt;workflow engine&lt;/em&gt;&lt;/strong&gt; is the software used to actually execute your pipeline/workflow. As mentioned above, general-purpose scripting languages like Bash, Python, or Perl can be used to implement workflows. But there are other languages that are designed specifically for managing workflows. Perhaps the earliest and most popular of these is GNU Make, which was originally intended to help engineers coordinate software compilation but can be used for just about any workflow. More recently there has been a proliferation of tools intended to replace GNU Make for numerous languages in a variety of contexts. The most popular in bioinformatics seems to be &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/&quot; rel=&quot;noreferrer&quot;&gt;Snakemake&lt;/a&gt;, which provides a nice balance of simplicity (through shell commands), flexibility (through configuration), and power-user support (through Python scripting). Build scripts written for these tools (i.e., a &lt;code&gt;Makefile&lt;/code&gt; or &lt;code&gt;Snakefile&lt;/code&gt;) are often called pipelines or workflows, and the &lt;strong&gt;&lt;em&gt;workflow engine&lt;/em&gt;&lt;/strong&gt; is the software that executes the workflow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The workflow engines you listed above (such as argo) can certainly be used to coordinate bioinformatics workflows. Honestly though, these are aimed more at the broader tech industry: they involve not just workflow execution but also hardware and infrastructure coordination, and would require a level of engineering expertise/support not commonly available in a bioinformatics setting. This could change, however, as bioinformatics becomes more of a &quot;big data&quot; endeavor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a final note, I'll mention a few more relevant technologies that I wasn't able to fit above.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.docker.com/&quot; rel=&quot;noreferrer&quot;&gt;Docker&lt;/a&gt;: managing a consistent software environment across multiple (potentially dozens or hundreds) of computers; Singularity is Docker's less popular step-sister&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.commonwl.org/&quot; rel=&quot;noreferrer&quot;&gt;Common Workflow Language (CWL)&lt;/a&gt;: a generic language for declaring how each step of a workflow is executed, what inputs it needs, what outputs it creates, and approximately what resources (RAM, storage, CPU threads, etc.) are required to run it; designed to write workflows that can be run on a variety of workflow engines&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://dockstore.org/&quot; rel=&quot;noreferrer&quot;&gt;Dockstore&lt;/a&gt;: a registry of bioinformatics workflows (heavy emphasis on genomics) that includes a Docker container and a CWL specification for each workflow&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://toil.readthedocs.io/en/latest/&quot; rel=&quot;noreferrer&quot;&gt;toil&lt;/a&gt;: a production-grade workflow engine used primarily for bioinformatics workflows&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2019-03-29T13:42:48.003" LastActivityDate="2019-03-29T13:42:48.003" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7354" PostTypeId="2" ParentId="7349" CreationDate="2019-03-29T09:26:52.257" Score="2" Body="&lt;p&gt;The &lt;code&gt;mroast&lt;/code&gt; function has an argument to specify which contrast do you want to test, quoting from the help page:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;contrast&lt;br&gt;&#xA;  contrast for which the test is required. Can be an integer specifying a column of design, or the name of a column of design, or a numeric contrast vector of length equal to the number of columns of design.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So if you have the contrast of 1-min timepoint positive vs 1-min timepoint negative you can use this.&lt;br&gt;&#xA;Read the help pages in detail, specially the help pages in limma are quite extensive and detailed to answer most questions if not all about how to use the software.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2019-03-29T09:26:52.257" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="7355" PostTypeId="2" ParentId="7324" CreationDate="2019-03-29T10:03:15.887" Score="1" Body="&lt;p&gt;MeSH is organized into &lt;code&gt;Descriptor &amp;gt; Concept &amp;gt; Term hierarchy&lt;/code&gt;, where all the synonymous &lt;code&gt;&amp;lt;term&amp;gt;&lt;/code&gt;'s are grouped together under a &lt;code&gt;&amp;lt;concept&amp;gt;&lt;/code&gt;. For example, consider the MeSH terms,&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Blood-Brain Barrier&lt;/li&gt;&#xA;&lt;li&gt;Hemato-Encephalic Barrier&lt;/li&gt;&#xA;&lt;li&gt;Brain-Blood Barrier&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;All these terms are group under the umbrella concept of &quot;&lt;a href=&quot;https://meshb.nlm.nih.gov/record/ui?ui=D001812&quot; rel=&quot;nofollow noreferrer&quot;&gt;Blood-brain barrier&lt;/a&gt;&quot;.&lt;/p&gt;&#xA;" OwnerUserId="3015" LastActivityDate="2019-03-29T10:03:15.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7356" PostTypeId="1" CreationDate="2019-03-30T23:22:59.550" Score="2" ViewCount="338" Body="&lt;p&gt;I know if there is one control and one treatment group it is pretty straight forward to interpret log 2 fold change. But, I have time course experiment. I have infected cells with viruses and I harvested cells at different time points: o hr, 6 hour, 12 hr and 24 hour. Here o hr is my non-infected control. What is the best way to show log 2 fold change for each gene in this type of time course experiment. I ultimate aim to find highly expressed genes at different time points. &lt;/p&gt;&#xA;" OwnerUserId="1684" LastActivityDate="2019-04-30T12:02:05.377" Title="How to get log2 fold change of RNA-Seq data for time series experiment?" Tags="|rna-seq|deseq2|limma|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7357" PostTypeId="2" ParentId="7356" CreationDate="2019-03-31T11:22:16.790" Score="3" Body="&lt;p&gt;If you just want the change with time then use &lt;code&gt;time&lt;/code&gt; as a continuous variable. The log2FC will then be the change per hour (you can use the default Wald test to extract it).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For time-point specific comparisons you'll instead need to treat the time points as groups and use contrasts as appropriate.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-03-31T11:22:16.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7358" PostTypeId="1" AcceptedAnswerId="7359" CreationDate="2019-03-31T17:33:17.780" Score="-2" ViewCount="85" Body="&lt;p&gt;I have this dataframe &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(mut)&#xA;  sample chr     pos ref alt&#xA;1  t_005   1  928006   G   A&#xA;2  t_005   1 1649842   G   T&#xA;3  t_005   1 2020408   G   A&#xA;4  t_005   1 2031677   T   A&#xA;5  t_005   1 2901878   G   A&#xA;6  t_005   1 3039338   G   A&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How I can add &lt;code&gt;&quot;chr&quot;&lt;/code&gt; to the &lt;code&gt;chr&lt;/code&gt; column? I mean, instead of &lt;code&gt;&quot;1&quot;&lt;/code&gt; I want &lt;code&gt;&quot;chr1&quot;&lt;/code&gt;, etc.&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="29" LastEditDate="2019-04-03T08:52:11.570" LastActivityDate="2019-04-03T08:52:11.570" Title="Adding a string to start of one column" Tags="|r|" AnswerCount="1" CommentCount="1" ClosedDate="2019-04-08T22:10:49.760" ContentLicense="CC BY-SA 4.0" />
  <row Id="7359" PostTypeId="2" ParentId="7358" CreationDate="2019-03-31T17:57:27.563" Score="4" Body="&lt;p&gt;This should work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mut&lt;span class=&quot;math-container&quot;&gt;$chr &amp;lt;- paste(&quot;chr&quot;, mut$&lt;/span&gt;chr, sep = &quot;&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="4001" LastEditDate="2019-03-31T18:02:45.357" LastActivityDate="2019-03-31T18:02:45.357" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7361" PostTypeId="1" CreationDate="2019-04-01T13:01:35.513" Score="0" ViewCount="317" Body="&lt;p&gt;I have called SNP and INDEL in two matched samples by &lt;em&gt;strelka&lt;/em&gt; and extract this information from .vcf file and I have these&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CHROMOSOME  POS REF ALT SAMPLE&#xA;1   928006  G   A   t_005&#xA;1   1649842 G   T   t_005&#xA;1   2020408 G   A   t_005&#xA;1   2031677 T   A   t_005&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CHROMODOME  POSITION    REF ALT SAMPLE&#xA;1   14115   A   T   o_005&#xA;1   541052  T   C   o_005&#xA;1   1088123 T   G   o_005&#xA;1   1232501 A   G   o_005&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I need to show if and how extend the mutations between these two samples are consistent by a circos plot but really I don't know how people do that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something like this &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OuDvH.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OuDvH.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3DfJb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3DfJb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="131" LastEditDate="2019-04-04T12:54:35.787" LastActivityDate="2019-04-04T12:54:35.787" Title="Plot a circos plot to show the consistency between 2 samples" Tags="|r|mutations|wgs|software-usage|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="7362" PostTypeId="2" ParentId="7361" CreationDate="2019-04-01T13:43:42.953" Score="1" Body="&lt;p&gt;A circos plot is most likely not the most appropriate solution here. What I would suggest is a confusion matrix, of which you can find an example here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/T4yqN.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/T4yqN.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For every variant in your vcf you'll add a number in this matrix. One sample is the columns, the other is the lines. If your variant is homozygous in both, then you add in that square +1 (the cell with 5845 in the example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A perfect concordant sample pair will have only variants on the diagonal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is some python code to get such a matrix. It uses cyvcf2 and pandas, and expects as input a vcf file with both samples.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from argparse import ArgumentParser&#xA;from cyvcf2 import VCF&#xA;import pandas as pd&#xA;&#xA;&#xA;def main():&#xA;    args = get_args()&#xA;    confusion_matrix(args.vcf)&#xA;&#xA;&#xA;def confusion_matrix(vcff):&#xA;    &quot;&quot;&quot;&#xA;    First level of the dict is the &quot;first&quot; call, second level is the &quot;second&quot; sample&#xA;    0: hom_ref&#xA;    1: heterozygous&#xA;    2: unknown/nocall&#xA;    3: hom_alt&#xA;    &quot;&quot;&quot;&#xA;    zygosities = {0: {0: 0, 1: 0, 2: 0, 3: 0},&#xA;                  1: {0: 0, 1: 0, 2: 0, 3: 0},&#xA;                  2: {0: 0, 1: 0, 2: 0, 3: 0},&#xA;                  3: {0: 0, 1: 0, 2: 0, 3: 0},&#xA;                  }&#xA;    for v in VCF(vcff):&#xA;        zygosities[v.gt_types[0]][v.gt_types[1]] += 1&#xA;    zygs = [2, 0, 1, 3]&#xA;    df = pd.DataFrame(index=zygs, columns=zygs)&#xA;    for tr in zygs:&#xA;        for te in zygs:&#xA;            df.loc[tr, te] = zygosities[tr][te]&#xA;    df.columns = ['nocall', 'hom_ref', 'het', 'hom_alt']&#xA;    df.index = ['nocall', 'hom_ref', 'het', 'hom_alt']&#xA;    print(df)&#xA;&#xA;&#xA;def get_args():&#xA;    parser = ArgumentParser(description=&quot;Create confusion matrix of SNV calls&quot;)&#xA;    parser.add_argument(&quot;vcf&quot;, help=&quot;vcf containing two samples&quot;)&#xA;    return parser.parse_args()&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="681" LastActivityDate="2019-04-01T13:43:42.953" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7363" PostTypeId="2" ParentId="7350" CreationDate="2019-04-01T18:03:18.533" Score="1" Body="&lt;p&gt;A program called gridss does this. It can be found &lt;a href=&quot;https://github.com/PapenfussLab/gridss&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. It basically extracts the clipped bases and repeatedly realigns them to reference with the aligner of your choice with bwa as default. It will do this until there are no further realignments. The simple command-line for this tool after install is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -Xmx512M -cp gridss-VERSION-with-dependencies.jar gridss.SoftClipsToSplitReads I=your_input.bam O=your_output.bam REFERENCE_SEQUENCE=your_reference.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I hope this helps! &lt;/p&gt;&#xA;" OwnerUserId="3669" LastActivityDate="2019-04-01T18:03:18.533" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7365" PostTypeId="1" CreationDate="2019-04-02T02:13:54.360" Score="1" ViewCount="39" Body="&lt;p&gt;After an ExpressionSet is established,&#xA;I want it to contain only the genes of my choice (&quot;selected.txt&quot;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the best approach??&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="2306" LastEditorUserId="29" LastEditDate="2019-04-03T08:53:03.980" LastActivityDate="2019-04-03T08:53:03.980" Title="ExpressionSet with a custom gene list" Tags="|r|filtering|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7366" PostTypeId="1" CreationDate="2019-04-02T11:29:40.480" Score="3" ViewCount="477" Body="&lt;p&gt;I am new to Bioinformatics and I am exploring the refGene.txt files from the UCSC genome annotation database for several species.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question concerns the Dec. 2011 (GRCm38/mm10) assembly of the mouse genome. I have seen that the Human one (hg39) contains both coding and non-coding transcripts. But the Mouse (mm10) RefGene.txt only contains coding transcripts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source of the file: &lt;a href=&quot;http://hgdownload.soe.ucsc.edu/goldenPath/mm10/database/refGene.txt.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://hgdownload.soe.ucsc.edu/goldenPath/mm10/database/refGene.txt.gz&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4351" LastEditorUserId="4351" LastEditDate="2019-04-04T21:11:09.797" LastActivityDate="2021-05-29T13:03:10.283" Title="Why doesn't the mouse GRCm38/mm10 refGene genome annotation file contain non-coding transcripts?" Tags="|sequence-annotation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7367" PostTypeId="1" CreationDate="2019-04-03T05:31:22.683" Score="1" ViewCount="72" Body="&lt;p&gt;Currently, I am working on Galactose oxidase enzyme for protein docking and I am totally new to it. I have to study how mutation on specific residue will make mutated galactose oxidase better than the wild type. My questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;There are so many structure available on PDB, how to know which structure is the so called wild type structure? Is the structure with PDB Code: 1GOG belongs to wild type?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I was requested to extract the ligand(substrate) out of the structure and docked it back. But seem all the structures available on PDB do not complexed with ligand. For what I know, the substrate for galactose oxidase could be D-galactose. Could I use the galactose oxidase structure from pubmed and dock it to the enzyme? Btw, what is purpose of extract the ligand out and dock it back?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3760" LastActivityDate="2019-04-03T05:31:22.683" Title="How to find Wild Type (WT) protein structure on Protein Data Bank (PDB)?" Tags="|pdb|protein-structure|docking|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7368" PostTypeId="1" AcceptedAnswerId="7381" CreationDate="2019-04-03T10:00:36.363" Score="2" ViewCount="77" Body="&lt;pre&gt;&lt;code&gt;colnames(countdata) = gsub(&quot;Aligned.sortedByCoord.out.bam_rem.bam&quot;, &quot;&quot;, colnames(countdata))&#xA;colnames(countdata)&#xA;dput(colnames(countdata))&#xA;    c(&quot;BAF60A_KD_1_S13&quot;, &quot;BAF60A_KD_2_S14&quot;, &quot;BAF60A_RA_1_S17&quot;, &quot;BAF60A_RA_2_S18&quot;, &#xA;    &quot;BAF60A_VD3_1_S15&quot;, &quot;BAF60A_VD3_2_S16&quot;, &quot;BAF60C_KD_1_UI_S1&quot;, &#xA;    &quot;BAF60C_KD_2_UI_S2&quot;, &quot;BAF60C_KD_RA_1_S11&quot;, &quot;BAF60C_KD_RA_2_S12&quot;, &#xA;    &quot;BAF60C_KD_VD3_1_S9&quot;, &quot;BAF60C_KD_VD3_2_S10&quot;, &quot;VC_1_UI_S3&quot;, &quot;VC_2_UI_S4&quot;, &#xA;    &quot;VC_RA1_S5&quot;, &quot;VC_RA_2_S6&quot;, &quot;VC_VD3_1_S7&quot;, &quot;VC_VD3_2_S8&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After removing the above pattern I want to remove this part from each of the colnames &lt;code&gt;_S[anynumber]&lt;/code&gt; so that I can end up like this as an example  &lt;code&gt;BAF60A_KD_1&lt;/code&gt; but not sure how to make that pattern and match the same and remove or replace it with empty space. I tried as below which ends up like all the digits are removed.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;colnames(countdata) = gsub(&quot;[[:digit:]_S]&quot;, &quot;&quot;, colnames(countdata))&#xA;&#xA;colnames(countdata)&#xA;&#xA;c(&quot;BAFAKD&quot;, &quot;BAFAKD&quot;, &quot;BAFARA&quot;, &quot;BAFARA&quot;, &quot;BAFAVD&quot;, &quot;BAFAVD&quot;, &#xA;&quot;BAFCKDUI&quot;, &quot;BAFCKDUI&quot;, &quot;BAFCKDRA&quot;, &quot;BAFCKDRA&quot;, &quot;BAFCKDVD&quot;, &quot;BAFCKDVD&quot;, &#xA;&quot;VCUI&quot;, &quot;VCUI&quot;, &quot;VCRA&quot;, &quot;VCRA&quot;, &quot;VCVD&quot;, &quot;VCVD&quot;)    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any suggestion would be helpful&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="48" LastEditDate="2019-04-03T12:47:17.177" LastActivityDate="2019-04-04T08:23:19.077" Title="Pattern matching and removing from the name" Tags="|r|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="7369" PostTypeId="1" AcceptedAnswerId="7373" CreationDate="2019-04-03T19:19:11.717" Score="1" ViewCount="129" Body="&lt;p&gt;Is there a out of the box to convert a .bam file to a .wig file at specific resolution/bin size?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking to use igvtools, but I am not sure how it works:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;igvtools count -z 5 -w 25 -e 250 alignments.bam  alignments.cov.tdf  hg19&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the diffrence between &lt;code&gt;-w,-z,-e&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-z num  Specifies the maximum zoom level to precompute. The default&#xA;               value is 7 and is sufficient for most files. To reduce file&#xA;               size at the expense of IGV performance this value can be&#xA;               reduced.&#xA;&#xA; -e, --extFactor num&#xA;&#xA;The read or feature is extended by the specified distance in bp prior to counting. This option is useful for chip-seq and rna-seq applications. The value is generally set to the average fragment length of the library minus the average read length.&#xA;&#xA;-w, --windowSize num&#xA;&#xA;The window size over which coverage is averaged. Defaults to 25 bp.&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2019-04-03T23:51:03.447" LastActivityDate="2019-04-03T23:51:03.447" Title="How to convert ChiP-seq track in .bam file format into an array at fixed bin resolution?" Tags="|chip-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7370" PostTypeId="1" CreationDate="2019-04-03T19:52:38.030" Score="1" ViewCount="687" Body="&lt;p&gt;I have constructed a gene co-expression network from RNA-seq data. The network has more than 10000 nodes and more than 1 million edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The network file as an edge list format of memory around 1gb which was created by calculating Pearson correlation of each gene pairs and gene pairs having correlation greater than 95% we're selected to create the edge list. The file is too large to run in cytoscape due to limitation of computer configuration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In R, I am able to calculate the topological properties of the network. And to cluster the gene coexperssion network, I tried using different community detection methods and but I am unable to identify which community detection is good for obtaining clusters?.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After clustering how to rank the subclusters and save each subcluster in separate files.&lt;/p&gt;&#xA;" OwnerUserId="2811" LastEditorUserId="2811" LastEditDate="2019-04-06T08:07:50.913" LastActivityDate="2019-04-06T08:07:50.913" Title="Clustering of gene co-expression network by igraph R package" Tags="|r|rna-seq|gene|networks|gene-expression|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7371" PostTypeId="1" AcceptedAnswerId="7372" CreationDate="2019-04-03T20:14:28.317" Score="3" ViewCount="2342" Body="&lt;p&gt;I have installed &lt;a href=&quot;https://anaconda.org/bioconda/busco&quot; rel=&quot;nofollow noreferrer&quot;&gt;busco&lt;/a&gt; on my Mac computer using &lt;code&gt;conda install --channel bioconda busco=3.0.2&lt;/code&gt; into an environment called &lt;code&gt;busco3_0_2&lt;/code&gt;. However, when I type &lt;code&gt;busco&lt;/code&gt;, the command is not found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone knows what the name of the binary is? Usually when I install something with conda, the name of the command is the same as the name of the package. In this case, it does not seem to be so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the package installation finished successfully.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="96" LastEditDate="2019-04-04T13:27:08.513" LastActivityDate="2020-01-17T14:45:05.060" Title="How to use busco installed through conda?" Tags="|software-installation|conda|busco|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7372" PostTypeId="2" ParentId="7371" CreationDate="2019-04-03T20:16:58.337" Score="4" Body="&lt;p&gt;From the &lt;a href=&quot;https://gitlab.com/ezlab/busco&quot; rel=&quot;nofollow noreferrer&quot;&gt;README&lt;/a&gt;, it looks like the command-line entry point is not &lt;code&gt;busco&lt;/code&gt; but &lt;code&gt;run_BUSCO.py&lt;/code&gt;. This seems to agree with the &lt;a href=&quot;https://github.com/bioconda/bioconda-recipes/blob/master/recipes/busco/meta.yaml&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioconda recipe&lt;/a&gt;, where &lt;code&gt;run_BUSCO.py&lt;/code&gt; and &lt;code&gt;run_busco&lt;/code&gt; are listed among the commands.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-04-03T20:16:58.337" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7373" PostTypeId="2" ParentId="7369" CreationDate="2019-04-03T22:06:46.640" Score="1" Body="&lt;p&gt;Using deepTools to get a bin size of 10 and 20 threads:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bamCoverage -of bedgraph -bs 10 -b input.bam -p 20&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You'll probably prefer the default bigWig format if you need random access or to use something like IGV.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the options to igvtools, &lt;code&gt;-w&lt;/code&gt; is equivalent to &lt;code&gt;-bs&lt;/code&gt;/&lt;code&gt;--binSize&lt;/code&gt; in deepTools and denotes the resolution of the output data. &lt;code&gt;-e&lt;/code&gt; (it's the same option in bamCoverage) denotes whether and how much to extend a read in its 3' direction. This done such that the output represents the sequences fragments. &lt;code&gt;-z&lt;/code&gt; is specific to things like bigWig files (and apparently tdf files), where there are precomputed values that IGV will load as you zoom out (the various resolutions are &quot;zoom levels&quot;, which is parlance taken from the bigWig format).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-04-03T22:06:46.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7374" PostTypeId="1" CreationDate="2019-04-03T23:39:54.573" Score="1" ViewCount="26" Body="&lt;p&gt;I obtained a genome from NCBI. For instance, Haloquadratum walsbyi strain AM180088.1. But the FASTA file contain two parts. The first part marked as &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&gt;AM180088.1 Haloquadratum walsbyi DSM 16790 complete genome&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second part marked as&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&gt;AM180089.1 Haloquadratum walsbyi DSM 16790 plasmid PL47 complete genome&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does that mean? In particular, the second one &quot;plasmid&quot;? If it is called complete genome should it all in one? Why divide into parts?&lt;/p&gt;&#xA;" OwnerUserId="4367" LastActivityDate="2019-04-03T23:39:54.573" Title="What does this genome file from NCBI means?" Tags="|genome|" AnswerCount="0" CommentCount="6" ClosedDate="2019-04-04T08:03:14.483" ContentLicense="CC BY-SA 4.0" />
  <row Id="7375" PostTypeId="2" ParentId="6841" CreationDate="2019-04-04T00:27:08.333" Score="2" Body="&lt;p&gt;&lt;code&gt;So let's say I want to get the bed file of this experiment using curl encodeproject.org/experiments/ENCSR000CKC&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also search for files that belong to a given experiment, e.g. &lt;a href=&quot;https://www.encodeproject.org/search/?type=File&amp;amp;dataset=/experiments/ENCSR000CKC/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.encodeproject.org/search/?type=File&amp;amp;dataset=/experiments/ENCSR000CKC/&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/N9Nuj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/N9Nuj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Then further select file properties (status, format, etc.) using the facets on the left.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This basically builds up a search URL for you, which is also possible to do manually. For example all BED files in ENCSR000CKC with GRCh38 assembly, returned as JSON: &lt;a href=&quot;https://www.encodeproject.org/search/?type=File&amp;amp;dataset=/experiments/ENCSR000CKC/&amp;amp;file_format=bed&amp;amp;assembly=GRCh38&amp;amp;format=json&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.encodeproject.org/search/?type=File&amp;amp;dataset=/experiments/ENCSR000CKC/&amp;amp;file_format=bed&amp;amp;assembly=GRCh38&amp;amp;format=json&lt;/a&gt; (See properties tab here &lt;a href=&quot;https://www.encodeproject.org/profiles/file.json&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.encodeproject.org/profiles/file.json&lt;/a&gt; for all of the file fields you can use.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also specify a &lt;code&gt;field&lt;/code&gt; parameter to return only fields you are interested in. For example &lt;a href=&quot;https://www.encodeproject.org/search/?type=File&amp;amp;dataset=/experiments/ENCSR000CKC/&amp;amp;file_format=bed&amp;amp;assembly=GRCh38&amp;amp;field=s3_uri&amp;amp;field=cloud_metadata.url&amp;amp;field=dataset&amp;amp;field=file_format&amp;amp;format=json&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.encodeproject.org/search/?type=File&amp;amp;dataset=/experiments/ENCSR000CKC/&amp;amp;file_format=bed&amp;amp;assembly=GRCh38&amp;amp;field=s3_uri&amp;amp;field=cloud_metadata.url&amp;amp;field=dataset&amp;amp;field=file_format&amp;amp;format=json&lt;/a&gt; only returns the &lt;em&gt;s3_uri&lt;/em&gt;, &lt;em&gt;cloud_metadata.url&lt;/em&gt;, &lt;em&gt;dataset&lt;/em&gt;, and &lt;em&gt;file_format&lt;/em&gt; fields. These objects are returned in the &lt;code&gt;@graph&lt;/code&gt; list:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Znnoi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Znnoi.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Putting this all together to parse in Python:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; import requests&#xA;&amp;gt;&amp;gt;&amp;gt; import pandas as pd&#xA;&amp;gt;&amp;gt;&amp;gt; url = (&#xA;...     'https://www.encodeproject.org'&#xA;...     '/search/?type=File'&#xA;...     '&amp;amp;dataset=/experiments/ENCSR000CKC/'&#xA;...     '&amp;amp;file_format=bed'&#xA;...     '&amp;amp;assembly=GRCh38'&#xA;...     '&amp;amp;field=file_format'&#xA;...     '&amp;amp;field=assembly'&#xA;...     '&amp;amp;field=s3_uri'&#xA;...     '&amp;amp;field=cloud_metadata.url'&#xA;...     '&amp;amp;format=json'&#xA;... )&#xA;&amp;gt;&amp;gt;&amp;gt; r = requests.get(url)&#xA;&amp;gt;&amp;gt;&amp;gt; files = r.json()['@graph']&#xA;&amp;gt;&amp;gt;&amp;gt; df = pd.DataFrame(files)&#xA;&amp;gt;&amp;gt;&amp;gt; df['cloud_metadata'] = df['cloud_metadata'].apply(lambda x: x.get('url'))&#xA;&amp;gt;&amp;gt;&amp;gt; df[['@id', 'assembly', 'file_format', 's3_uri', 'cloud_metadata']]&#xA;                   @id assembly file_format                                                                                 s3_uri                                                                                             cloud_metadata&#xA;0  /files/ENCFF368QQU/   GRCh38         bed  s3://encode-public/2016/08/04/10493464-917f-4290-bb13-c5fec8319887/ENCFF368QQU.bed.gz  https://encode-public.s3.amazonaws.com/2016/08/04/10493464-917f-4290-bb13-c5fec8319887/ENCFF368QQU.bed.gz&#xA;1  /files/ENCFF554NVI/   GRCh38         bed  s3://encode-public/2016/08/04/865f220e-7a47-427a-a7c1-9bdc049d4658/ENCFF554NVI.bed.gz  https://encode-public.s3.amazonaws.com/2016/08/04/865f220e-7a47-427a-a7c1-9bdc049d4658/ENCFF554NVI.bed.gz&#xA;2  /files/ENCFF217CRP/   GRCh38         bed  s3://encode-public/2016/08/04/43aa693e-1581-4886-8ace-7e1609f45790/ENCFF217CRP.bed.gz  https://encode-public.s3.amazonaws.com/2016/08/04/43aa693e-1581-4886-8ace-7e1609f45790/ENCFF217CRP.bed.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The s3_uri and cloud_metadata.url both point to the same thing. You can use the s3_uri to download using awscli or boto3. You can pass the https link to cURL.&lt;/p&gt;&#xA;" OwnerUserId="4368" LastActivityDate="2019-04-04T00:27:08.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7376" PostTypeId="1" CreationDate="2019-04-04T00:32:37.703" Score="4" ViewCount="91" Body="&lt;p&gt;I am trying to assemble RNA virus genome using Broad Viral NGS pipeline &lt;a href=&quot;https://viral-ngs.readthedocs.io/en/latest/assembly.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BROAD VIRAL NGS PIPELINE&lt;/a&gt;. I am two questions : &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) As this pipeline requires unaligned bam format as input, how do I convert fastq to bam ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Some of the tools like   assemble_trinity require Trimmomatic clipDb. How can I get this clipDb ?&lt;/p&gt;&#xA;" OwnerUserId="1684" LastEditorUserId="3848" LastEditDate="2020-12-11T00:40:31.803" LastActivityDate="2020-12-11T00:40:31.803" Title="Viral genome assembly using broad viral ngs pipeline?" Tags="|genome|assembly|covid-19|sars-cov-2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7377" PostTypeId="2" ParentId="7370" CreationDate="2019-04-04T01:02:39.490" Score="2" Body="&lt;p&gt;For the first part, do you mean that the file is too large to be run on your computer?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the second, if I understood correctly, you can use &lt;code&gt;igraph&lt;/code&gt; or &lt;code&gt;ggnet2&lt;/code&gt; to color code the nodes based on the conditions you have and see if communities are formed based on condition. This would be a simple way to do this kind of thing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I personally prefer using &lt;code&gt;ggnet2&lt;/code&gt; as I found it more user friendly. What you can do if you have an adjacency matrix, say &lt;code&gt;adj_mat&lt;/code&gt; is to create a color vector based on the conditions for each node in the network, where &lt;code&gt;length(color_vec)&lt;/code&gt; = &lt;code&gt;ncol(adj_mat)&lt;/code&gt; = &lt;code&gt;nrow(adj_mat)&lt;/code&gt; - since it is a square matrix. Following that, you could do something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(ggplot2)&#xA;library(network)&#xA;&#xA;neto &amp;lt;- network(adj_mat, directed = FALSE, loops = FALSE, hyper = FALSE)&#xA;#assuming that it is undirected here and a simple network diagram. &#xA;#run ?network for more details in case your analysis is to be more complex&#xA;&#xA;color_vec &amp;lt;- as.factor(color_vec)&#xA;&#xA;col_pal &amp;lt;- c(&quot;blue&quot;, &quot;red&quot;, &quot;green&quot;) #assuming you only have 3 colors..&#xA;&#xA;names(col_pal) &amp;lt;- levels(color_vec)&#xA;neto %v% &quot;color&quot; &amp;lt;- as.character(col_vec) &#xA;&#xA;ggnet2(neto, node.size = .9, color = &quot;color&quot;, palette = col_pal,&#xA;       edge.size = .05, edge.color = &quot;white&quot;, mode = &quot;adj&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4001" LastActivityDate="2019-04-04T01:02:39.490" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7378" PostTypeId="2" ParentId="7376" CreationDate="2019-04-04T01:52:32.413" Score="4" Body="&lt;p&gt;Since you are already using the Broad Tools sets you can use &lt;a href=&quot;https://broadinstitute.github.io/picard/command-line-overview.html#FastqToSam&quot; rel=&quot;nofollow noreferrer&quot;&gt;Picard FastqToSam&lt;/a&gt; to make the conversion&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far a clipDb I am unfamiliar with that and a quick google search and look at the trimmomatic manual were unhelpful&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2019-04-04T01:52:32.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7380" PostTypeId="2" ParentId="159" CreationDate="2019-04-04T08:13:28.417" Score="1" Body="&lt;p&gt;This may not be the whole answer to the question, but one key difference between Phyre and ITasser (I think I'm correct in saying), is that Phyre only returns regions of the model it considers to be sufficiently well modelled, so it may not be a full length model. ITasser on the other hand, always generates full size models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As one of the other answers alludes to, it is very dependent on your protein of interest. If it isn't well represented in the databases, the initial threading stages will not yield good models, and the subsequent steps all suffer as a result.&lt;/p&gt;&#xA;" OwnerUserId="929" LastActivityDate="2019-04-04T08:13:28.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7381" PostTypeId="2" ParentId="7368" CreationDate="2019-04-04T08:23:19.077" Score="3" Body="&lt;p&gt;Without &lt;em&gt;regex&lt;/em&gt;, split on &lt;code&gt;&quot;_&quot;&lt;/code&gt;, drop last item, then &lt;em&gt;paste&lt;/em&gt; it back again:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sapply(strsplit(x, &quot;_&quot;), function(i) paste(head(i, -1), collapse = &quot;_&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or with &lt;em&gt;regex&lt;/em&gt; (stealing the code from &lt;code&gt;tools::file_path_sans_ext&lt;/code&gt; function which is used to drop file extensions):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sub(&quot;([^_]+)\\_[[:alnum:]]+$&quot;, &quot;\\1&quot;, x)&#xA;&#xA;# [1] &quot;BAF60A_KD_1&quot;     &quot;BAF60A_KD_2&quot;     &quot;BAF60A_RA_1&quot;     &quot;BAF60A_RA_2&quot;     &quot;BAF60A_VD3_1&quot;    &quot;BAF60A_VD3_2&quot;   &#xA;# [7] &quot;BAF60C_KD_1_UI&quot;  &quot;BAF60C_KD_2_UI&quot;  &quot;BAF60C_KD_RA_1&quot;  &quot;BAF60C_KD_RA_2&quot;  &quot;BAF60C_KD_VD3_1&quot; &quot;BAF60C_KD_VD3_2&quot;&#xA;# [13] &quot;VC_1_UI&quot;         &quot;VC_2_UI&quot;         &quot;VC_RA1&quot;          &quot;VC_RA_2&quot;         &quot;VC_VD3_1&quot;        &quot;VC_VD3_2&quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="131" LastActivityDate="2019-04-04T08:23:19.077" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="7384" PostTypeId="1" AcceptedAnswerId="7388" CreationDate="2019-04-04T14:54:56.077" Score="4" ViewCount="917" Body="&lt;p&gt;I am looking for a solution to exlude certain entries when searching a local blast nt database (with blastn), specifically the sequences from uncultured / environmental samples, ideally using their taxon ids.&#xA;This should ideally replicate the results from the BLAST web interface, when checking the option &quot;&lt;em&gt;Exclude Uncultured/environmental sample sequences&lt;/em&gt;&quot;, see screenshot below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Three approaches that I am aware of so far are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(1) grep taxon IDs for any taxa with &quot;uncultured&quot; or &quot;environmental&quot; in their scientific name from names.dmp and fetch the GIs belonging to these taxa using the E-utils. Then use the -negative_gilist parameter for blastn. However, GIs are discontinued for future records, so this method should be replaced by Acc.Ver identifiers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(2) locally format the blast database after prepocessing the nt file by myself and exclude records belonging to the taxa list as obtained in (1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(3) Postprocess the blast output and remove the relevant lines, but this approach might remove a lot of alignments, whereas (1) and (2) would always generate the desired number of output alignments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I am looking for other suggestions, preferably without preprocessing the nt file.&#xA;Ideally, one would like to have a -negative_taxalist option for blastn, which would resemble what is possible on the NCBI BLAST web interface, where one can exclude taxa, see screenshot below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BLwXE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BLwXE.png&quot; alt=&quot;NCBI BLAST interface&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="1802" LastActivityDate="2019-08-08T14:25:35.870" Title="taxon exclude list for searching local blast database using blastn" Tags="|blast|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7385" PostTypeId="1" CreationDate="2019-04-04T15:17:57.427" Score="2" ViewCount="319" Body="&lt;p&gt;I was wondering if any one has had success merging pre-generated reference index files for BWA, bowtie2 from multiple references? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a concrete example I have hg19 and a bunch of viral and bacterial genomes that all have been index separately but I want to do a single alignment to all of them.  Is there a way I could say combine hg19, virus_A, virus_D and bacteria_X into a single reference without having to regenerate the indexes?  Since the problem could get combinatorial quickly and indexing anything with hg19 is time consuming I was wondering if there was a way around it.&lt;/p&gt;&#xA;" OwnerUserId="964" LastEditorUserId="964" LastEditDate="2019-04-04T20:01:14.150" LastActivityDate="2019-04-04T20:01:14.150" Title="Merging/concatenating index files" Tags="|read-mapping|genome-index|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="7386" PostTypeId="1" AcceptedAnswerId="7390" CreationDate="2019-04-04T20:04:21.217" Score="2" ViewCount="891" Body="&lt;p&gt;I was wondering if any one has had success merging BLAST databases from multiple references? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a concrete example I have hg19 and a bunch of viral and bacterial genomes that all have been index separately but I want to do a single alignment to all of them.  Is there a way I could say combine hg19, virus_A, virus_D and bacteria_X into a single database without having to regenerate?  Since the problem could get combinatorial quickly and making a db with hg19 is time consuming I was wondering if there was a way around it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is similar to &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/7385/merging-concatenating-index-files&quot;&gt;another ticket&lt;/a&gt; but asking for BLAST instead of alignment indexes.&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2019-04-05T09:16:37.937" Title="Merging / Concatentating BLAST Databases" Tags="|blast|makeblastdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7388" PostTypeId="2" ParentId="7384" CreationDate="2019-04-05T08:32:06.467" Score="1" Body="&lt;p&gt;Hurray, the solution seems already to be implemented in the new BLAST database format (v5), which is usable with blast+ versions &gt;= 2.8.0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible to specify an exclusion as well as inclusion list of taxon ids, which, however, have to be on species level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Species level taxon ids can be obtained from higher level taxon ids using the script get_species_taxids.sh, e.g.: &lt;code&gt;get_species_taxids.sh -n Enterobacterales&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See docs here: &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/blast/db/v5/blastdbv5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nlm.nih.gov/blast/db/v5/blastdbv5.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;br&gt;&#xA;For creating taxon ID lists locally from the NCBI taxonomy files, the program &lt;a href=&quot;https://github.com/pmenzel/taxonomy-tools#subtree&quot; rel=&quot;nofollow noreferrer&quot;&gt;subtree&lt;/a&gt; can be used. See example therein for compiling a list of taxon IDs from environmental or unclassified taxa.&lt;/p&gt;&#xA;" OwnerUserId="1802" LastEditorUserId="1802" LastEditDate="2019-08-08T14:25:35.870" LastActivityDate="2019-08-08T14:25:35.870" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7389" PostTypeId="1" CreationDate="2019-04-05T09:03:31.463" Score="1" ViewCount="119" Body="&lt;p&gt;Wiki has the algorithm for finding position-specific weight matrix (PSWM) for a list of nucleotide sequences having an equal number of nucleotide. How do we find the PSSM for a list of nucleotide sequences having different sizes?&lt;/p&gt;&#xA;" OwnerUserId="2817" LastActivityDate="2019-04-05T14:01:53.707" Title="How to find the position-specific weight matrix (PSWM) for a list of genes having different sizes?" Tags="|sequence-alignment|proteins|sequence-analysis|computation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7390" PostTypeId="2" ParentId="7386" CreationDate="2019-04-05T09:16:37.937" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK279693/&quot; rel=&quot;nofollow noreferrer&quot;&gt;There's an app for that&lt;/a&gt;, apparently. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Often we need to search multiple databases together or wish to search a specific subset of sequences within an existing database. At the BLAST search level, we can provide multiple database names to the “-db” parameter, or to provide a GI file specifying the desired subset to the “-gilist” parameter. However for these types of searches, a more convenient way to conduct them is by creating virtual BLAST databases for these. Note: When combining BLAST databases, all the databases must be of the same molecule type. The following examples assume that the two databases as well as the GI file are in the current working directory.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;Aggregate existing BLAST databases&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;To combine the two nematode nucleotide databases, named “nematode_mrna” and “nematode_genomic&quot;, we use the following command line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ blastdb_aliastool -dblist &quot;nematode_mrna nematode_genomic&quot; -dbtype nucl \&#xA;  -out nematode_all -title &quot;Nematode RefSeq mRNA + Genomic&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, you can use the &lt;code&gt;blastdb_aliastool&lt;/code&gt; to create merged databases just like you asked for.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-04-05T09:16:37.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7391" PostTypeId="1" CreationDate="2019-04-05T09:22:12.310" Score="0" ViewCount="355" Body="&lt;p&gt;I am working with single cell RNA-seq data. I obtained the squared coefficient of variation (CV²) as a measure of gene expression variability:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://latex.codecogs.com/gif.latex?%5Cfrac%7Bvariance(gene)%7D%7Bmean&amp;space;(gene)%5E%7B2%7D%7D&quot; title=&quot;\frac{variance(gene)}{mean (gene)^{2}}&quot; /&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I want a metric to express gene expression noise that eliminates/takes into account the observed relationship between CV² of a gene and its mean expression&lt;/strong&gt; so I can compare this metric across genes/conditions:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/haktu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/haktu.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I have found two metrics that try to tackle this problem:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Distance to median (DM)&lt;/strong&gt;:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A median-based trend is fitted to the log-transformed CV² against the log-transformed mean. The DM is defined as the residual from the trend for each gene. This statistic is a measure of the relative variability of each gene, after accounting for the empirical mean-variance relationship &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;residual CV² (rCV²)&lt;/strong&gt;: &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Fit between the mean log2 expression and CV² using a gamma generalised linear model as parametrisation then computing the absolute deviation of the CV² values from the fitted CV² values.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Do you have any clue about which metric is better or if there exists any other similar metrics?&lt;/p&gt;&#xA;" OwnerUserId="678" LastActivityDate="2019-04-05T20:24:16.617" Title="residual Squared Coefficient of Variation (rCV²) vs Distance to Median (DM)" Tags="|scrnaseq|statistics|gene-expression|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7392" PostTypeId="2" ParentId="7366" CreationDate="2019-04-05T13:06:17.637" Score="1" Body="&lt;p&gt;It wasn't the correct file.&#xA;I downloaded the file again from UCSC, compared it with the file that I was using, and they were different. This new file contained both information from coding and non-coding transcripts.&lt;/p&gt;&#xA;&lt;p&gt;To all the kind people that took interest in my problem, I am sorry.&lt;/p&gt;&#xA;&lt;p&gt;Edited 30/12/2020:&#xA;To add a bit of clarity; the link in the question is the correct link from which to download the correct file. The file with which I was working from the beginning was the wrong one, but I don't really know where I got it from the first time.&lt;/p&gt;&#xA;" OwnerUserId="4351" LastEditorUserId="4351" LastEditDate="2020-12-30T12:29:39.760" LastActivityDate="2020-12-30T12:29:39.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7393" PostTypeId="1" CreationDate="2019-04-05T13:59:16.553" Score="2" ViewCount="269" Body="&lt;p&gt;I am working on a GNU/Linux machine. I have installed &lt;a href=&quot;https://busco.ezlab.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BUSCO&lt;/a&gt; using conda. I have downloaded the nematode dataset &lt;code&gt;nematoda_odb9.tar.gz&lt;/code&gt;&lt;a href=&quot;https://busco.ezlab.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I downloaded the reference sequence of &lt;em&gt;C. elegans&lt;/em&gt; from &lt;a href=&quot;https://www.ensembl.org/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl&lt;/a&gt;. I then ran busco using the following command:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;run_busco --in Caenorhabditis_elegans.WBcel235.dna.toplevel.fa --cpu 1 --out busco --lineage_path ./nematoda_odb9/ --mode genome --force &amp;gt; busco.o 2&amp;gt; busco.e&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;tblastn seems to have run properly. &lt;code&gt;run_busco/blast_output/tblastn_busco.tsv&lt;/code&gt; contains lots of hits for many of the buscos.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I have 982 &quot;Missing BUSCOs&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The file that contains logs and errors related to augustus (run_busco/augustus_output/augustus.log) contains:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Will create parameters for a EUKARYOTIC species!&#xA;creating directory ...anaconda/envs/busco3_0_2/config/species/BUSCO_busco_1395218643/ ...&#xA;creating ...anaconda/envs/busco3_0_2/config/species/BUSCO_busco_1395218643/BUSCO_busco_1395218643_parameters.cfg ...&#xA;creating ...anaconda/envs/busco3_0_2/config/species/BUSCO_busco_1395218643/BUSCO_busco_1395218643_weightmatrix.txt ...&#xA;creating ...anaconda/envs/busco3_0_2/config/species/BUSCO_busco_1395218643/BUSCO_busco_1395218643_metapars.cfg ...&#xA;The necessary files for training BUSCO_busco_1395218643 have been created.&#xA;Now, either run etraining or optimize_parameters.pl with --species=BUSCO_busco_1395218643.&#xA;etraining quickly estimates the parameters from a file with training genes.&#xA;optimize_augustus.pl alternates running etraining and augustus to find optimal metaparameters.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It does not seem to contain any errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following error:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Failed to open sequence file run_busco/augustus_output/extracted_proteins/EOG091H07ZC.faa.1&lt;/code&gt; In fact, the folder &lt;code&gt;run_busco/augustus_output/extracted_proteins/&lt;/code&gt; is empty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone would know why I have no extracted proteins?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2019-05-10T07:16:53.697" Title="BUSCO does not find buscos in reference sequence of C. elegans (technical problem)" Tags="|software-usage|hmmer|busco|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7394" PostTypeId="2" ParentId="7389" CreationDate="2019-04-05T14:01:53.707" Score="3" Body="&lt;p&gt;If your sequences are different lengths, this implies a gapped alignment with insertions/deletions. This is not possible with a regular PSSM, so you have to create a separate PSSM for each group of homologous sequences of the same sequence length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The alternative is to use a different approach that allows gaps like profile HMMs (as implemented in HMMER3 or HH-Suite3).&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2019-04-05T14:01:53.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7395" PostTypeId="1" CreationDate="2019-04-05T14:08:29.627" Score="1" ViewCount="48" Body="&lt;p&gt;Is the visual cortex of newborn babies right off the bat capable of making sense of raw visual data, for instance, converting the constant stream of raw RGB images perceived by the eyes into a meaningful higher-level representation of objects in motion in a 3D world? Yes? No? If not, then does it mean this skill has to be developed over time by means of some &lt;strong&gt;learning mechanism&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the visual cortex needs time to learn advanced visual skills, how does the actual learning mechanism work? Does the visual cortex have to optimize the connections between neurons, in a way analogous to how artificial neural networks optimize their parameters through backpropagation algorithm (machine learning)? If this is the case, then where does the visual cortex get its error signals from? To make the last question clear, in machine learning the typical approach is to compute the gradient of a loss function which compares the model's prediction with the ground truth, and the model's parameters are updated by moving the parameters in the direction of the gradient. If the visual cortex is learning advanced visual skills by virtue of a similar learning mechanism, then what kind of loss function is the visual cortex optimizing?&lt;/p&gt;&#xA;" OwnerUserId="4384" LastEditorUserId="4384" LastEditDate="2019-04-07T15:42:18.677" LastActivityDate="2019-04-07T15:42:18.677" Title="Is the visual cortex of a newborn baby immediately capable of object detection or is this skill learned over time, and if so, how?" Tags="|machine-learning|" AnswerCount="0" CommentCount="7" ClosedDate="2019-04-05T15:36:14.957" ContentLicense="CC BY-SA 4.0" />
  <row Id="7396" PostTypeId="1" CreationDate="2019-04-05T14:49:28.587" Score="1" ViewCount="59" Body="&lt;p&gt;I am using Biopython to examine protein sequences including the Kyte and Doolittle hydrophobicity. However, for short sequences, this returns an empty value; the value is not allowed if the window does not have 20 values.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.SeqUtils.ProtParam import ProteinAnalysis&#xA;&#xA;window_length=20&#xA;edge=1&#xA;kyte = {'A': 1.8, 'R': -4.5, 'N': -3.5, 'D': -3.5, 'C': 2.5, 'Q': -3.5, 'E': -3.5, 'G': -0.4, 'H': -3.2, 'I': 4.5,&#xA;        'L': 3.8, 'K': -3.9, 'M': 1.9, 'F': 2.8, 'P': -1.6, 'S': -0.8, 'T': -0.7, 'W': -0.9, 'Y': -1.3, 'V': 4.2}&#xA;&#xA;full_sequence_analysis = ProteinAnalysis(&quot;CYGTGAVYLLGLGIGGFS&quot;)&#xA;kyte_window = full_sequence_analysis.protein_scale(kyte, window_length, edge)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there are a built in Biopython option in this for &quot;allow half windows/short windows&quot; for incomplete windows? There is nothing I can see to suggest so in &lt;a href=&quot;http://biopython.org/DIST/docs/api/Bio.SeqUtils.ProtParam-pysrc.html#ProteinAnalysis.protein_scale&quot; rel=&quot;nofollow noreferrer&quot;&gt;the docs&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="747" LastEditorUserId="3848" LastEditDate="2020-04-29T10:53:23.610" LastActivityDate="2020-04-29T10:53:23.610" Title="How to allow &quot;half windows&quot; in biopython?" Tags="|phylogenetics|proteins|python|sequence-analysis|phylogeny|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7397" PostTypeId="1" CreationDate="2019-04-05T16:11:57.943" Score="2" ViewCount="496" Body="&lt;p&gt;I'm trying to execute pymol from a shell script (and it is not working). I'm not executing the script on  PyMOL but even if I do this, it doesn't work neither. However, if instead of a script the commands are manually written in PyMOL, everything goes well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;pwd&#xA;echo 'fetch 1OYT &#xA;remove resn HOH&#xA;h_add elem O or elem N&#xA;select ligand, resn FSN &#xA;' &amp;gt; scriptt.pml&#xA;pymol scriptt.pml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One of the problems, you will see, is that PyMOL does not remove the water molecules. Another issue is that the program does not exit. Do you have any idea what's going on there?&lt;/p&gt;&#xA;" OwnerDisplayName="user4378" LastEditorDisplayName="user4378" LastEditDate="2019-04-05T17:05:29.660" LastActivityDate="2019-04-05T17:05:29.660" Title="Executing PyMOL from a Shell script" Tags="|shell|pymol|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7398" PostTypeId="2" ParentId="7397" CreationDate="2019-04-05T16:18:56.547" Score="1" Body="&lt;p&gt;Your shell script is fine, and works as expected. The reason it hangs is that you aren't telling PyMol to exit after running your script, so it's still running and waiting for the next command. Just add a &lt;code&gt;quit&lt;/code&gt; at the end and it should work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;pwd&#xA;echo 'fetch 1OYT &#xA;remove resn HOH&#xA;h_add elem O or elem N&#xA;select ligand, resn FSN &#xA;select prot, 1OYT and not ligand, &#xA;save lig.pdb, ligand &#xA;save prot.pdb, prot&#xA;quit&#xA;' &amp;gt; scriptt.pml&#xA;pymol scriptt.pml&#xA;obabel prot.pdb -xr -O prot.pdbqt &#xA;obabel lig.pdb -O lig.pdbqt&#xA;echo 'babel conversion is finished'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-04-05T16:18:56.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7399" PostTypeId="2" ParentId="7391" CreationDate="2019-04-05T17:27:43.870" Score="2" Body="&lt;p&gt;There has been a lot of work done on this problem already, particularly these two papers:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biorxiv.org/content/10.1101/576827v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/10.1101/576827v2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biorxiv.org/content/10.1101/574574v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/10.1101/574574v1&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3415" LastActivityDate="2019-04-05T17:27:43.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7400" PostTypeId="1" CreationDate="2019-04-05T17:51:25.250" Score="1" ViewCount="20" Body="&lt;p&gt;I have a set of 414 GenBank genomes and I need to count a number of insertion sequences (ISs, transposases) in each of them. Is there any traditional way to perform such an analysis?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I came up with three possible ways:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Count a number of &quot;transposase&quot; as a product of CDS for all CDS in annotation file of each genome. -- it depends on annotation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Use a tool named OASIS. -- it depends on annotation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Use a web-server named ISsaga. -- it can analyse only 10 replicons at once &lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) Perform blast(n or p) against ISfinder database and count a number of hits with low e-value. -- it seems not reliable&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is appreciated!&lt;/p&gt;&#xA;" OwnerUserId="4387" LastActivityDate="2019-04-05T17:51:25.250" Title="The best tool to count the number of insertion sequences in a set of bacterial genome" Tags="|sequence-annotation|genbank|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7401" PostTypeId="1" CreationDate="2019-04-05T19:15:40.540" Score="5" ViewCount="1582" Body="&lt;p&gt;&lt;strong&gt;The short version&lt;/strong&gt;: If I have a SAM record, is there any simple way to retrieve the base aligned to a particular reference position without computing a pileup?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The long version&lt;/strong&gt;: I'm using pysam to write some genotyping code. I have a BAM file with reads aligned to some amplicons. I'd now like to genotype a couple of positions of interest in each read while maintaining the phase information. For example, at amplicon 1, if read 1 has &lt;code&gt;A&lt;/code&gt; for SNP1 and &lt;code&gt;T&lt;/code&gt; at SNP2, then I want to report an &lt;code&gt;A,T&lt;/code&gt; genotype for read 1 at amplicon 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The typical way to genotype SNPs of interest is to compute a pileup and look at all of the aligned bases position-by-position. The pysam API includes the following example of how to work with pileups.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pysam&#xA;samfile = pysam.AlignmentFile(&quot;ex1.bam&quot;, &quot;rb&quot; )&#xA;for pileupcolumn in samfile.pileup(&quot;chr1&quot;, 100, 120):&#xA;    print (&quot;\ncoverage at base %s = %s&quot; %&#xA;           (pileupcolumn.pos, pileupcolumn.n))&#xA;    for pileupread in pileupcolumn.pileups:&#xA;        if not pileupread.is_del and not pileupread.is_refskip:&#xA;            # query position is None if is_del or is_refskip is set.&#xA;            print ('\tbase in read %s = %s' %&#xA;                  (pileupread.alignment.query_name,&#xA;                   pileupread.alignment.query_sequence[pileupread.query_position]))&#xA;&#xA;samfile.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I lose the phasing information if I analyze the data position-by-position. Is there any way to flip this example so that I'm iterating over reads in the outer loop, and then over positions in the inner loop? In other words, &lt;strong&gt;if I'm iterating over reads, is there any simple way to retrieve the base aligned to a particular reference position &lt;em&gt;without&lt;/em&gt; computing a pileup&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a potential solution, I've though about storing genotype information in a nested dictionary indexed first by read ID and then by position, but depending on coverage that could be incredibly memory intensive.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2021-05-27T09:13:49.007" Title="Access base aligned to particular reference position" Tags="|sam|pysam|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7402" PostTypeId="2" ParentId="7391" CreationDate="2019-04-05T20:24:16.617" Score="1" Body="&lt;p&gt;You should do both. Distance to median (DM) is the most powerful transformation IMO.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The important feature of the coefficient of variation which is more famously known as the index of dispersion (sd2/mean) is if:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;IoD is approximately 1, the distribution is random (Poison), &lt;/li&gt;&#xA;&lt;li&gt;IoD &amp;lt;1 the distribution is a uniform distribution (very rare in biological systems)&lt;/li&gt;&#xA;&lt;li&gt;IoD &gt;&gt; 1 the distribution is skewed (clustered), such as a negative binomial distribution. This is common in biological systems.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;An example of a uniform distribution are the distance between apple trees in an orchard. In general biological statistics IoD is rarely useful because many distributions are moderately over dispersed. This could be different however for transcriptomics. Thus if IoD results in a moderately skewed distribution you can comment on it, but its not worth taking further. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Distance to median appears to be a &quot;non-parametric&quot; standardisation and that is exactly what you are looking for here. Standardisation is a formal process of accommodating natural variation, e.g. time of day of the experiment, to extract the underlying trend. It is very common (in fact almost essential) in time series.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-04-05T20:24:16.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7403" PostTypeId="1" CreationDate="2019-04-06T09:44:27.570" Score="3" ViewCount="417" Body="&lt;p&gt;I  downloaded virulence factor database from &lt;a href=&quot;http://www.mgc.ac.cn/VFs/main.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; , to predict virulence genes from few genomes that my lab  sequenced. I  performed local blast on the database with default setting and found around 4000 genes. I want to filter this result. I am not sure how to proceed. There are evalue, pident (percent identity), qcovs (query coverage) etc that I can tweak and I did tweak. For example, if I set pident above 40% the qcovs automatically goes close to 100%. That gave me indication above 40% is pretty significant and biologically relevant. But I am not sure what percent I should consider for treating the query as having the same function as the subject -- in this case that function is virulence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Need some guidance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit1: I ran blastp&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit2: I am blasting protein (cds predicted from genome by annotation tool prokka). My question here is what is the most reasonable way to filter putative virulence factor using blast. &lt;/p&gt;&#xA;" OwnerUserId="3555" LastEditorUserId="3555" LastEditDate="2019-04-06T14:35:59.963" LastActivityDate="2019-04-07T09:42:17.767" Title="How to filter blast results from blast run against virulence factor database" Tags="|blast|blastp|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7404" PostTypeId="1" CreationDate="2019-04-06T15:18:47.300" Score="3" ViewCount="105" Body="&lt;p&gt;I have assembled poxvirus genome using Ray. The assembly is good. Out of several thousand contigs I got, I was able to get one scaffold using Contiguator tool, which is about 90% of my genome. I have well annotated reference sequence. If there any good tool for filling gaps and checking assembly quality?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Things I've tried so far:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;IMAGE from PAGIT. It asks for ace file which I guess is obtained from Newbler. So, I am not able to use this.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;gapfill_py script from broad: There are some gaps still missing.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I came to know that Consed is good. But, I have windows which is not compatible for Consed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I checked my BAM file with Tablet. There are reads that cover almost entire gene. Are there any good tools or manual ways to finish my assembly so that I can get the full genome?&lt;/p&gt;&#xA;" OwnerUserId="1684" LastEditorUserId="3848" LastEditDate="2020-11-28T14:12:23.363" LastActivityDate="2020-11-28T14:12:23.363" Title="Viral genome finishing" Tags="|genome|assembly|covid-19|sars-cov-2|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7405" PostTypeId="1" AcceptedAnswerId="7408" CreationDate="2019-04-06T22:12:43.677" Score="-1" ViewCount="349" Body="&lt;p&gt;I am trying to run &lt;code&gt;CMSclassifier::classifyCMS&lt;/code&gt;function on my data but I am getting this error&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(CMSclassifier)&#xA;&#xA;&amp;gt; Rfcms &amp;lt;- CMSclassifier::classifyCMS(my_data,method=&quot;RF&quot;)[[3]]&#xA;Error in match.names(clabs, names(xi)) : &#xA;  names do not match previous names&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This code classifies gene expression data&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/Sage-Bionetworks/CMSclassifier/blob/master/R/cmsClassifier.R&quot; rel=&quot;nofollow noreferrer&quot;&gt;Code&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But on example data code works&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have attached my data and example data here&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could somebody please help me in solving this error?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/g684pg5izjrqs8w/my_data.txt?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;My data&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/m4mxpise8fsiito/example_data.txt?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;Example data&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2019-04-07T20:23:46.790" Title="How I debug this code?" Tags="|r|rna-seq|networks|modelling|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7406" PostTypeId="2" ParentId="7401" CreationDate="2019-04-07T07:26:36.660" Score="4" Body="&lt;p&gt;You can use the &lt;a href=&quot;https://pysam.readthedocs.io/en/latest/api.html#pysam.AlignedSegment.get_reference_positions&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;get_reference_positions()&lt;/code&gt;&lt;/a&gt; function in pysam to get a vector of the positions. Searching that for your positions will allow you to output phased calls.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for read in bam.fetch(some_position):&#xA;    positions = read.get_reference_positions():&#xA;    if some_other_position in positions:&#xA;        idx = positions.index(some_position)&#xA;        idx2 = positions.index(some_other_positions)&#xA;        ... and so on...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That's not exactly correct syntax, but you get the idea.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="4396" LastEditDate="2019-04-08T14:21:13.610" LastActivityDate="2019-04-08T14:21:13.610" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7407" PostTypeId="2" ParentId="7403" CreationDate="2019-04-07T09:42:17.767" Score="4" Body="&lt;p&gt;The answer is it depends. 40% pident with ~100% qcovs (assuming length of protein is not very small) tells you that the virulence factor (VF) and your prokka predicted protein are likely homologs, they belong to the same (or closely related) protein family. For some research studies, you might annotate your prokka proteins as 'putative' VFs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, in a clinical context, even 90% pident might not be enough evidence to say these predictions are real VFs. A single amino acid change alters the protein and therefore potentially the function. You could investigate bioinformtiaclly how the changes impact the protein structure and domains etc. Ideally validate it experimentally, e.g. for a new putative E. coli toxin, clone the protein and express it in a commensal E. coli strain and test the toxicity of the new strain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practise, think about what question you're asking and who you want to convince. Try numerous thresholds and see how robust your results are to threshold changes (4,000 VFs seems like a lot for a 'few' genomes). You could also try the VirulenceFinder database. When annotating antibiotic resistance genes, if you use 80% pident and qcovs thresholds with a minimum protein length then most researchers will accept your predicted annotations as 'conservative', maybe the VF community has similar arbitrary but pragmatic standards.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2019-04-07T09:42:17.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7408" PostTypeId="2" ParentId="7405" CreationDate="2019-04-07T12:54:34.700" Score="2" Body="&lt;p&gt;The problem was I should use ENTREZ rather than gene symbols&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2019-04-07T12:54:34.700" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7409" PostTypeId="2" ParentId="7405" CreationDate="2019-04-07T16:40:12.933" Score="0" Body="&lt;p&gt;I see what you mean but I'm not sure that is the source of the error. I kinda suspect that the example data set you provided works fine. What I think is happening is the labels between the classifier, test and training data are different at some point and so classification ceases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your code (albeit I rarely use R) and approach and data look fine. In fact very professional, the data appears scaled (not shown). It looks good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps what you refer to is the row name consistency in labelling against the example data set. For example row name, ARHGEF19, this is a gene ID rather than a NCBI code, viz.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;efetch -db nucleotide -format gpc -id &quot;LS482351&quot; | xtract -insd gene gene&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;LS482351.1  ARHGEF19&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Elsewhere you use an AL590644.1 as the label, I assume this is the genomic region for PADI4&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;efetch -db nucleotide -format gpc -id &quot;AL590644&quot; | xtract -insd source organism&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AL590644.14 Homo sapiens&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(There appears to be version 14 for this subgenomic locus).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the labelling is a bit mixed and you can feasible convert between them using efetch and scripting a hash/dictionary. However, I don't think its the problem here because its a random forest classification and a label is a label.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is possible the package you are using is buggy and some of the labels are not getting parsed properly e.g. with a XXX-1 tag or even if a XXX.1 tag. The example data set uses &quot;nice whole&quot; integers. I don't know about R but if regex approaches are used for label parsing things can easily go wrong (its not a good approach). Just a guess however and its easy to check.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-04-07T20:23:46.790" LastActivityDate="2019-04-07T20:23:46.790" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7410" PostTypeId="1" AcceptedAnswerId="7412" CreationDate="2019-04-07T22:35:03.513" Score="5" ViewCount="1182" Body="&lt;p&gt;I am using snakemake to create workflows. It is very convenient to visualise my DAG using &lt;code&gt;snakemake --dag target{sampleA,sampleB,sampleC}File | dot -Tpdf &amp;gt; dag.pdf&lt;/code&gt;. The resulting pdf shows all the rules' dependencies to get to the target files. However, there is one node per rule/sample combination in the pdf image. When I have many samples, this creates an unreadable image with little added value. Is there an option to merge all the nodes from the same rule together so that I have the workflow for just one sample?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2019-04-08T04:27:28.917" Title="How can I ask snakemake to produce a dag where each node represents a rule?" Tags="|snakemake|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7411" PostTypeId="1" AcceptedAnswerId="7434" CreationDate="2019-04-07T22:39:37.480" Score="1" ViewCount="106" Body="&lt;p&gt;I have a .wig file in this format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;variableStep chrom=chr1 span=20&#xA;59965   7&#xA;59985   10&#xA;60005   10&#xA;60025   10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I would like to format it to something like this format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#bedGraph section chr16:0-103584&#xA;chr16   0       59901   0&#xA;chr16   59901   59927   1&#xA;chr16   59927   59931   2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Please assume, I am working with a reference hg19 genome, in case the info isn't deducible from the &lt;code&gt;variableStep&lt;/code&gt; .wig file. I couldn't find anything in &lt;a href=&quot;https://github.com/Ensembl/WiggleTools&quot; rel=&quot;nofollow noreferrer&quot;&gt;WiggleTools&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2019-04-08T15:29:02.090" LastActivityDate="2019-04-10T12:17:24.910" Title="Is there a tool to convert a variableStep .wig to fixedStep one?" Tags="|python|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7412" PostTypeId="2" ParentId="7410" CreationDate="2019-04-08T04:27:28.917" Score="7" Body="&lt;p&gt;I think you are looking for &lt;code&gt;--rulegraph&lt;/code&gt; instead of &lt;code&gt;--dag&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  --rulegraph           Do not execute anything and print the dependency graph&#xA;                        of rules in the dot language. This will be less&#xA;                        crowded than above DAG of jobs, but also show less&#xA;                        information. Note that each rule is displayed once,&#xA;                        hence the displayed graph will be cyclic if a rule&#xA;                        appears in several steps of the workflow. Use this if&#xA;                        above option leads to a DAG that is too large.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1310" LastActivityDate="2019-04-08T04:27:28.917" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7413" PostTypeId="1" AcceptedAnswerId="7414" CreationDate="2019-04-08T05:56:33.213" Score="1" ViewCount="601" Body="&lt;p&gt;From a &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/29628141&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;(D) Expression analysis of multiple lineage-specific differentiation markers in WT and PUS7-KO EBs (14 days). Heatmap shows log2 fold change (FC) PUS7-KO to WT for each individual gene (rows) in three independent experiments (columns).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;They have analysed the data in EdgeR but I was wondering how did they plot fold change when they have two condition but they have three independent experiment, I haven't used EdgeR so far, use DESeq2 where I group my biological replicate. For example if I have two condition with three biological replicate then I would group them and analyse, in case of DESeq2 I would get final result with one fold change report.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How did they do in this case reporting three fold change ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE101004&quot; rel=&quot;nofollow noreferrer&quot;&gt;Data sets&lt;/a&gt; where I see three wild type and three knockout.&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="48" LastEditDate="2019-04-08T09:08:22.253" LastActivityDate="2019-04-08T09:08:22.253" Title="log2fold change as heatmap" Tags="|rna-seq|figure-reproduction|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7414" PostTypeId="2" ParentId="7413" CreationDate="2019-04-08T07:46:25.020" Score="1" Body="&lt;p&gt;If I read quickly thru the paper, I see you refer to Fig 5D. In the methods they describe that they use Fluidigm (i.e. RT-qPCR) for gene expression, not RNA-seq. The data link you give is for small-RNA-seq.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2019-04-08T07:46:25.020" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7415" PostTypeId="1" CreationDate="2019-04-08T13:10:24.010" Score="4" ViewCount="483" Body="&lt;p&gt;For benchmark purposes I’m trying to find an old version of the human reference genome (pre-GRCh37) and a matching version of dbSNP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately it appears as though older versions of dbSNP aren’t archived, or at least these archives are not publicly accessible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In particular, &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/snp/organisms/archive&quot; rel=&quot;nofollow noreferrer&quot;&gt;the NCBI FTP archive&lt;/a&gt; is fairly sparse. The only pre-GRCh37 version it contains is ostensibly &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/projects/SNP/snp_summary.cgi?build_id=124&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbSNP build 124 from January 2005&lt;/a&gt;, for hg17 (NCBI build 35). Unfortunately &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/snp/organisms/archive/human_9606_b124_NCBI35/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the corresponding FTP folder is actually &lt;em&gt;empty&lt;/em&gt;&lt;/a&gt;. dbSNP build 138 has been “removed to recover storage”, whatever that means, and I’m unclear which version of the human reference genome it refers to (its &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/projects/SNP/snp_summary.cgi?build_id=138&quot; rel=&quot;nofollow noreferrer&quot;&gt;official page&lt;/a&gt; does not contain this rather crucial bit of information).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there another authoritative source of this data?&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2020-02-01T00:23:46.150" Title="Old versions of the reference genome and dbSNP" Tags="|database|public-databases|variant-calling|reference-genome|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7416" PostTypeId="1" CreationDate="2019-04-08T13:10:50.470" Score="1" ViewCount="184" Body="&lt;p&gt;I've previously asked a different question about the ChIP-chip microarray data that I'm working on. I'm new to this so I'm sorry for asking here again. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have .CEL files and I'm using the &lt;em&gt;Affy&lt;/em&gt; package for R to read them. After I load all the files in, I then normalise the data and  generate a .txt file containing the expression values of each file (ranging from 11 to 12). Column 1 contains the gene names and the rest of the  columns contain the expression values for each of the files. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if there is a way to convert these data to bedgraph or BED format? What I'd like to do is a metagene analysis but I don't know how that would be possible with expression value data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I looked online and there are some kinda relevant suggestions/solutions, but they are for Windows machines and I'm working on MacOS.&lt;/p&gt;&#xA;" OwnerUserId="4209" LastEditorUserId="48" LastEditDate="2019-04-08T14:57:16.067" LastActivityDate="2019-04-08T14:57:16.067" Title="How can I convert Affymetrix microarray data to bedgraph format?" Tags="|r|microarray|format-conversion|chip-seq|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7417" PostTypeId="1" CreationDate="2019-04-08T13:32:52.450" Score="0" ViewCount="1947" Body="&lt;p&gt;I have the following alignment script which uses BWA:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;#usage: sh align_pbs.sh /scratch/waterhouse_team/benth/dbg2olc-40x/good/ output.racon2.fasta bam-soap&#xA;&#xA;mkdir $3&#xA;&#xA;for r1 in &lt;span class=&quot;math-container&quot;&gt;$(find $&lt;/span&gt;1 -name &quot;*R1*.gz&quot;);&#xA;do&#xA;  base=&lt;span class=&quot;math-container&quot;&gt;${r1%_R1*}&#xA;  output=$&lt;/span&gt;(basename &lt;span class=&quot;math-container&quot;&gt;$(echo $&lt;/span&gt;r1 | sed 's/R1//g'))&#xA;  r2=&lt;span class=&quot;math-container&quot;&gt;$(echo $&lt;/span&gt;r1 | sed 's/R1/R2/g') &#xA;&#xA;  #Get read group infomration:&#xA;  #Source: https://www.biostars.org/p/280837/#310132&#xA;  header=&lt;span class=&quot;math-container&quot;&gt;$(zcat $&lt;/span&gt;r1 | head -n 1)&#xA;  id=&lt;span class=&quot;math-container&quot;&gt;$(echo $&lt;/span&gt;header | head -n 1 | cut -f 1-4 -d&quot;:&quot; | sed 's/@//' | sed 's/:/_/g')&#xA;  sm=&lt;span class=&quot;math-container&quot;&gt;$(echo $&lt;/span&gt;header | head -n 1 | grep -Eo &quot;[ATGCN]+&lt;span class=&quot;math-container&quot;&gt;$&quot;)&#xA;  #echo &quot;Read Group @RG\tID:$&lt;/span&gt;id\tSM:&lt;span class=&quot;math-container&quot;&gt;$id&quot;_&quot;$&lt;/span&gt;sm\tLB:&lt;span class=&quot;math-container&quot;&gt;$id&quot;_&quot;$&lt;/span&gt;sm\tPL:ILLUMINA&quot;&#xA;  echo &quot;@RG\tID:&lt;span class=&quot;math-container&quot;&gt;${output}\tSM:$&lt;/span&gt;{output}\tLB:${output}\tPL:ILLUMINA&quot;&#xA;&#xA;  #cat &amp;lt;&amp;lt;EOF&#xA;  qsub &amp;lt;&amp;lt;EOF&#xA;#!/bin/bash -l&#xA;&#xA;#PBS -N $output&#xA;#PBS -l walltime=48:00:00&#xA;#PBS -j oe&#xA;#PBS -l mem=70G&#xA;#PBS -l ncpus=12&#xA;#PBS -M m.lorenc@qut.edu.au&#xA;##PBS -m bea&#xA;&#xA;cd \$PBS_O_WORKDIR&#xA;&#xA;conda activate bwa-hic&#xA;bwa mem \&#xA;&lt;span class=&quot;math-container&quot;&gt;$2 $&lt;/span&gt;r1 &lt;span class=&quot;math-container&quot;&gt;$r2 \&#xA;-t 12 \&#xA;-R &quot;$&lt;/span&gt;(echo &quot;@RG\tID:&lt;span class=&quot;math-container&quot;&gt;${output}\tSM:$&lt;/span&gt;{output}\tLB:&lt;span class=&quot;math-container&quot;&gt;${output}\tPL:ILLUMINA&quot;)&quot; | samblaster -r | samtools view -@ 12 -h -f 0x2 -F 2316 - | samtools fixmate - - | sed 's/^/LP1-/' | samtools sort -@ 12 - -o $&lt;/span&gt;{3}/${output}.sorted.dedup.bam&#xA;&#xA;EOF&#xA;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unfortunatelye, this pipeline cuased the following erros:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samblaster: Version 0.1.24&#xA;samblaster: Inputting from stdin&#xA;samblaster: Outputting to stdout&#xA;[M::bwa_idx_load_from_disk] read 0 ALT contigs&#xA;[M::process] read 1200000 sequences (120000000 bp)...&#xA;[M::process] read 1200000 sequences (120000000 bp)...&#xA;[M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (2, 316829, 6, 4)&#xA;[M::mem_pestat] skip orientation FF as there are not enough pairs&#xA;[M::mem_pestat] analyzing insert size distribution for orientation FR...&#xA;[M::mem_pestat] (25, 50, 75) percentile: (432, 474, 509)&#xA;[M::mem_pestat] low and high boundaries for computing mean and std.dev: (278, 663)&#xA;[M::mem_pestat] mean and std.dev: (471.59, 53.66)&#xA;[M::mem_pestat] low and high boundaries for proper pairs: (201, 740)&#xA;[M::mem_pestat] skip orientation RF as there are not enough pairs&#xA;[M::mem_pestat] skip orientation RR as there are not enough pairs&#xA;[M::mem_process_seqs] Processed 1200000 reads in 615.287 CPU sec, 58.490 real sec&#xA;samblaster: Loaded 19 header sequence entries.&#xA;[W::sam_read1] Parse error at line 1&#xA;samtools sort: truncated file. Aborting&#xA;sed: couldn't write 567 items to stdout: Broken pipe&#xA;[E::bgzf_flush] File write failed (wrong size)&#xA;samtools fixmate: Couldn't write to output file: Broken pipe&#xA;[E::bgzf_close] File write failed&#xA;[bam_mating] error while closing output file&#xA;samtools view: writing to standard output failed: Broken pipe&#xA;samtools view: error closing standard output: -1&#xA;samblaster: Unable to write to output file.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance,&lt;/p&gt;&#xA;" OwnerUserId="2477" LastActivityDate="2019-04-08T13:56:39.323" Title="samtools view: writing to standard output failed: Broken pipe" Tags="|samtools|bwa|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7418" PostTypeId="2" ParentId="7417" CreationDate="2019-04-08T13:56:39.323" Score="3" Body="&lt;pre&gt;&lt;code&gt;[W::sam_read1] Parse error at line 1&#xA;samtools sort: truncated file. Aborting&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;SAMtools sort has been unable to parse its input, which it thought was SAM (mostly because it couldn't be recognised as another format e.g. BAM).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is because &lt;code&gt;sed 's/^/LP1-/'&lt;/code&gt; is putting &lt;code&gt;LP1-&lt;/code&gt; at the front of every line. If the output of samtools fixmate is SAM, then this LP1 is garbling the SAM header lines. If the output of samtools fixmate is BAM (as it probably is), this LP1 is completely garbling the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What were you trying to achieve with this &lt;code&gt;sed&lt;/code&gt; command?&lt;/p&gt;&#xA;" OwnerUserId="134" LastActivityDate="2019-04-08T13:56:39.323" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7419" PostTypeId="2" ParentId="7411" CreationDate="2019-04-08T15:27:27.040" Score="2" Body="&lt;p&gt;This is a temporary solution I came up with that worked for my specific case:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def convert_line(line, chrom, span, start=None, fixedStep=None, count=None):&#xA;        new_line=''&#xA;        if fixedStep is None or count is None or start is None:&#xA;                line_arr  = line.split()&#xA;                start = line_arr[0]&#xA;                end   = int(line_arr[0]) + int(span)&#xA;                val   = line_arr[1]&#xA;                new_line = '{0}\t{1}\t{2}\t{3}\n'.format(chrom, start, end, val)&#xA;        else:&#xA;                line_arr  = line.split()&#xA;                val   = np.float(line_arr[0])&#xA;                start = int(start) + int(fixedStep)*int(count)&#xA;                end   = start + int(fixedStep)&#xA;                new_line = '{0}\t{1}\t{2}\t{3}\n'.format(chrom, start, end, val)&#xA;&#xA;        return new_line&#xA;&#xA;def convert_file(input_file, output_file):&#xA;        chrom=None&#xA;        span=None&#xA;        fixedStep=None&#xA;        count=None&#xA;        start=None&#xA;        with open(input_file, 'r') as fin, open(output_file, 'w') as fout:&#xA;                for line in fin:&#xA;                        if len(line)&amp;lt;1 or line[0] == '#':&#xA;                                continue&#xA;                        elif 'variableStep' in line or 'fixedStep' in line:&#xA;                                for key in line.split():&#xA;                                        if 'chrom=' in key:&#xA;                                                chrom = key.split('chrom=')[-1]&#xA;                                        if 'span=' in key:&#xA;                                                span = key.split('span=')[-1]&#xA;                                        if 'step=' in key:&#xA;                                                fixedStep = key.split('step=')[-1]&#xA;                                        if 'start=' in key:&#xA;                                                start = key.split('start=')[-1]&#xA;                                                count = 0&#xA;                        else:&#xA;                                line_out = convert_line(line, chrom, span, start, fixedStep, count)&#xA;                                fout.write(line_out)&#xA;                                if count is not None:&#xA;                                        count += 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;variableStep chrom=chr16 span=20&#xA;59965   7&#xA;59985   10&#xA;60005   10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr16   59965   59985   7&#xA;chr16   59985   60005   10&#xA;chr16   60005   60025   10&#xA;chr16   60025   60045   10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2019-04-08T17:40:53.393" LastActivityDate="2019-04-08T17:40:53.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7420" PostTypeId="1" AcceptedAnswerId="7422" CreationDate="2019-04-08T21:01:00.073" Score="1" ViewCount="223" Body="&lt;p&gt;I have RNA seq data of raw gene counts that I want to transform for linear modelling. I am trying to voom transform, to do a weighted analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Data frame: prac_count_10&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gene         S1     S2    S3   S4    S5 &#xA;ENG0000456   0      1     10   145   24&#xA;ENG0000458   7      2     9    0     0&#xA;ENG0000657   76     12    56   10    2 &#xA;ENG0000689   0      0     0    3     5&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;prac_prac_10 &amp;lt;- voom(prac_count_10[,2:5], design=NULL, lib.size=NULL, &#xA;  normalize.method = &quot;none&quot;, span = 0.5&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However this doesn't give an output with the gene sample names and logCPM values. Would like an output with gene id's and logCPM values for linear modelling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also tried:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cpm &amp;lt;- cpm(prac_count_10)&#xA;lcpm &amp;lt;- cpm(prac_count_10, log=TRUE)&#xA;&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;````````````&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2019-04-09T07:31:36.970" Title="Voom transformation of RNA seq raw counts data" Tags="|r|limma|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7421" PostTypeId="1" AcceptedAnswerId="7440" CreationDate="2019-04-09T05:15:37.410" Score="7" ViewCount="1181" Body="&lt;p&gt;Are phylogenetic tree construction algorithms any different from clustering algorhithms? I suspect the answer is no.&lt;/p&gt;&#xA;&lt;p&gt;Of course phylogenetic tree construction uses biological knowledge, e.g special distance metrics, but does it brings anything new to clustering algorithm (e.g hierarchical, neighbor-joining etc).&lt;/p&gt;&#xA;" OwnerUserId="3555" LastEditorUserId="3848" LastEditDate="2020-07-26T08:10:08.540" LastActivityDate="2020-07-26T08:10:08.540" Title="Are phylogenetic tree construction algorithms any different than general clustering algorithms?" Tags="|sequence-alignment|phylogenetics|phylogeny|clustering|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7422" PostTypeId="2" ParentId="7420" CreationDate="2019-04-09T07:31:36.970" Score="1" Body="&lt;p&gt;Try to use &lt;code&gt;DElist()&lt;/code&gt; function before you transform, and also make rownames first.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;counts &amp;lt;- prac_count_10[,-1]&#xA;&#xA;rownames(counts) &amp;lt;- prac_count_10[,1]&#xA;&#xA;DGE1 &amp;lt;- DGEList(counts)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then continue with your voom...&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2019-04-09T07:31:36.970" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7424" PostTypeId="2" ParentId="125" CreationDate="2019-04-09T09:49:39.053" Score="1" Body="&lt;p&gt;The answer is the logDet algorithm was constructed to overcome GC% clustering. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the time it was devised only a distance method was available/implemented, so it wasn't very powerful. The posts here imply that a Bayesian or ML approach are available and these do tightly stick to the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Original publication &lt;a href=&quot;https://academic.oup.com/mbe/article/13/10/1375/1020361&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-04-09T11:20:50.910" LastActivityDate="2019-04-09T11:20:50.910" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7425" PostTypeId="2" ParentId="7401" CreationDate="2019-04-09T09:58:48.090" Score="1" Body="&lt;p&gt;This is not [yet] a complete answer, but hopefully it helps get you on the right path.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wrote &lt;a href=&quot;https://gitlab.com/gringer/bioinfscripts/blob/master/sam2mapped.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;some code&lt;/a&gt; to retrieve the subsequence from reads in a BAM file that are actually mapped to the reference, which required calculating mapped base locations. Here's a subsection of the code that does this (slightly paraphrased):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  my &lt;span class=&quot;math-container&quot;&gt;$refPos = $&lt;/span&gt;F[3];&#xA;  my &lt;span class=&quot;math-container&quot;&gt;$cigar = $&lt;/span&gt;F[5];&#xA;  &lt;span class=&quot;math-container&quot;&gt;$cigar =~ s/[0-9]S$&lt;/span&gt;//;&#xA;  my &lt;span class=&quot;math-container&quot;&gt;$seq = $&lt;/span&gt;F[9];&#xA;  my &lt;span class=&quot;math-container&quot;&gt;$qual = $&lt;/span&gt;F[10];&#xA;  my &lt;span class=&quot;math-container&quot;&gt;$startTrim = 0;&#xA;  my $&lt;/span&gt;matchLen = 0;&#xA;  while(&lt;span class=&quot;math-container&quot;&gt;$cigar =~ s/^([0-9]+)([MIDNSHP=X])//){&#xA;    my $subLen = $1;&#xA;    my $op = $2;&#xA;    if($op eq &quot;S&quot;){&#xA;        $seq = substr($seq, $subLen);&#xA;        $qual = substr($qual, $subLen);&#xA;      }&#xA;    }&#xA;    if($&lt;/span&gt;op =~ /[M=XI]/){&#xA;      &lt;span class=&quot;math-container&quot;&gt;$matchLen += $&lt;/span&gt;subLen;&#xA;    }&#xA;  }&#xA;  &lt;span class=&quot;math-container&quot;&gt;$seq = substr($&lt;/span&gt;seq, 0, &lt;span class=&quot;math-container&quot;&gt;$matchLen + $&lt;/span&gt;addSeq);&#xA;  &lt;span class=&quot;math-container&quot;&gt;$qual = substr($&lt;/span&gt;qual, 0, &lt;span class=&quot;math-container&quot;&gt;$matchLen + $&lt;/span&gt;addSeq);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, some things to take note of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The entire CIGAR string (excluding terminal soft clips) should be parsed up to the required base location&lt;/li&gt;&#xA;&lt;li&gt;Any soft clips at the start of the sequence need to be accounted for&lt;/li&gt;&#xA;&lt;li&gt;Only update the read location counter if there's a match, mismatch, or insertion&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="73" LastActivityDate="2019-04-09T09:58:48.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7426" PostTypeId="1" AcceptedAnswerId="7428" CreationDate="2019-04-09T11:50:23.367" Score="6" ViewCount="132" Body="&lt;p&gt;My work involves searching for marker genes/fragments in metagenomic databases (like the Sequence Read Archive). Once I find these sequences, I would like to know more about the neighboring genomic region. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way I could assemble only sequences that create a contig which contains my region of interest? Contigs which don't contain this region are not useful to me. My organism of interest might represent a minority of the metagenome, and assembling everything in the dataset would use a lot of computing power.&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2019-04-09T16:18:30.580" Title="Is there a way to assemble contigs starting from a specific sequence?" Tags="|sequence-alignment|fasta|assembly|metagenome|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7427" PostTypeId="1" CreationDate="2019-04-09T12:39:52.917" Score="1" ViewCount="804" Body="&lt;p&gt;From this link &lt;a href=&quot;https://support.bioconductor.org/p/67237/&quot; rel=&quot;nofollow noreferrer&quot;&gt;remove X and Y chromosome genes in RNA-seq data using DESeq2 pipeline&lt;/a&gt;, I have learned that depending on context, it is perfectly valid to remove X and Y chromosomal genes in an RNA-seq dataset before doing differential expression analysis. However I only have access to the count matrix and not the &lt;strong&gt;bam&lt;/strong&gt; files of the RNA seq data I am analysing. Is it advisable to do the X and Y chromosome gene removal at the level of the count matrix? I am talking about removing all rows that contain X and Y chromosomal genes from the &lt;strong&gt;.csv&lt;/strong&gt; count matrix file read in to &lt;strong&gt;RStudio&lt;/strong&gt; before proceeding to create the &lt;strong&gt;DESeq2&lt;/strong&gt; object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for your kind response.&lt;/p&gt;&#xA;" OwnerUserId="3968" LastEditorUserId="3968" LastEditDate="2019-04-09T12:45:00.107" LastActivityDate="2019-04-09T12:45:00.107" Title="Is it advisable to remove X and Y chromosome genes in a bulk RNA-seq dataset at the level of the count matrix?" Tags="|rna-seq|deseq2|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7428" PostTypeId="2" ParentId="7426" CreationDate="2019-04-09T16:18:30.580" Score="2" Body="&lt;p&gt;This is one of the primary use cases for which spacegraphcats (&lt;a href=&quot;https://www.biorxiv.org/content/10.1101/462788v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;preprint&lt;/a&gt; and &lt;a href=&quot;https://github.com/spacegraphcats/spacegraphcats&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;) was designed. The &quot;neighborhood queries&quot; discussed in the paper sound particularly relevant. I don't have any personal experience with spacegraphcats, but the &lt;a href=&quot;https://github.com/spacegraphcats/spacegraphcats/blob/master/doc/running-spacegraphcats.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;run guide&lt;/a&gt; provides some examples of how to index the complete data set and how to query sequences of interest.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-04-09T16:18:30.580" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7429" PostTypeId="2" ParentId="7421" CreationDate="2019-04-09T17:14:31.633" Score="5" Body="&lt;p&gt;A great question, though a little ambiguous. I don't know what &quot;general clustering algorithms&quot; refer to. For biological sequences, building a tree can be thought as a way of clustering. Anyway...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are different tree building algorithms. Max parsimony (MP), max likelihood (ML) and bayesian algorithms directly take sequences as input. They are distinct from distance based clustering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then there is a class of distance based algorithms in phylogenetics. They start from an all-pair distance matrix and aim to find a tree that is most compatible with the matrix. Among them, &lt;a href=&quot;https://en.wikipedia.org/wiki/UPGMA&quot; rel=&quot;noreferrer&quot;&gt;UPGMA&lt;/a&gt; is largely hierarchical clustering. Neighbor-joining is somewhat similar to hierarchical clustering, but it builds unrooted trees. &lt;a href=&quot;http://www.atgc-montpellier.fr/fastme/&quot; rel=&quot;noreferrer&quot;&gt;FastME&lt;/a&gt; uses a very different approach. Briefly, for each topology it tries to find the best branch lengths by weighted least square (for details, see &lt;a href=&quot;https://academic.oup.com/mbe/article/21/3/587/1079580&quot; rel=&quot;noreferrer&quot;&gt;its paper&lt;/a&gt;); the best topology minimizes total branch lengths. FastME finds the best topology via nearest neighbor interchange (NNI), closer to some ML algorithms (e.g. PhyML).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;does it brings anything new at the level of clustering algorithm?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Personally, I think the distance based tree building methods are more sophisticated than naive hierarchical clustering, &lt;em&gt;provided that&lt;/em&gt; the real relationship follows a tree topology.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2019-04-09T17:14:31.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7430" PostTypeId="1" CreationDate="2019-04-10T08:17:21.357" Score="1" ViewCount="251" Body="&lt;p&gt;We have been experimenting with VEP (Variant Effect Predictor). One of the meta data attributes that we are interested in is the SIFT score, indeed when we apply the dbNSFP plug we get a column containing the scores (named SIFT_score). However, I don't understand why there are sometimes dots or multiple values in the fields. For example, the gene ENSG00000196924 below has 5 transcripts:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8FdyW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8FdyW.png&quot; alt=&quot;DataFrame of VEP output&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The SIFT_score column contains several values, not 1 per transcript/rs-number...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is another example that confuses me (I added the SIFT_pred column this time):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Jmjyt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Jmjyt.png&quot; alt=&quot;DataFrame of another gene with mutations&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two mutations, the lower one can be expressed in 4 transcripts and I thus understand that there can be 4 SIFT scores, but why are all for in given in every row? Is the first one the SIFT_score for the first transcript?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One last example, again 4 transcripts, but now 2 of the scores are dots, what does that mean?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Lo0Qb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Lo0Qb.png&quot; alt=&quot;And another pd.DataFrame&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been looking for quite some time now how to interpret this data, any help is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="3964" LastActivityDate="2019-04-10T12:13:28.950" Title="VEP output SIFT_score unclear" Tags="|ngs|variant-calling|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7431" PostTypeId="2" ParentId="7404" CreationDate="2019-04-10T09:52:27.687" Score="2" Body="&lt;p&gt;Assuming that gap filling is a domain-independent and works the same for viruses as for bacteria or eukarytes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two real options I am aware of: &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-015-0663-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sealer&lt;/a&gt; a successor of GapFiller from &lt;a href=&quot;http://www.bcgsc.ca/platform/bioinfo/software/abyss&quot; rel=&quot;nofollow noreferrer&quot;&gt;ABySS&lt;/a&gt; package and GapCloser from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3626529/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SOAPdenovo2&lt;/a&gt; package. According to Sealer paper it's better, but I am not sure if any indepented benchmarking have confirmed their conclusion. On my insect data GapCloser worked better than GapFiller (back then I have not tried to compare it to Sealer).&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2019-04-10T09:52:27.687" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7432" PostTypeId="2" ParentId="7430" CreationDate="2019-04-10T10:05:19.823" Score="3" Body="&lt;p&gt;The &lt;a href=&quot;https://github.com/Ensembl/VEP_plugins/blob/release/96/dbNSFP.pm&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbNSFP plugin&lt;/a&gt; from VEP accesses tables of data for each variant from dbNSFP and pulls out the values. dbNSFP provide their SIFT scores in that format: a score for every transcript affected by the variant, all on one line. The lookup is just for the variant, not for the variant/transcript combo, so they provide scores for all variant/transcript combos. You can also get a column that gives you a list of the transcripts or proteins (Ensembl_transcriptid or Ensembl_proteinid) in order so you know which score goes with which transcript.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A better way to get SIFT scores with VEP is to get them directly from VEP, rather than using dbNSFP. This will get the SIFT score that goes with the transcript on the line with the relevant transcript.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastEditorUserId="1332" LastEditDate="2019-04-10T10:45:58.443" LastActivityDate="2019-04-10T10:45:58.443" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7433" PostTypeId="2" ParentId="7430" CreationDate="2019-04-10T12:06:38.850" Score="3" Body="&lt;p&gt;The first gene you mention, ENSG00000196924, actually &lt;a href=&quot;https://varsome.com/variant/hg38/rs371839875&quot; rel=&quot;nofollow noreferrer&quot;&gt;has 6 transcripts&lt;/a&gt; (link to the VarSome.com page of variant rs371839875), not 5. It's just that one of them is non-coding:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rUyYq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rUyYq.png&quot; alt=&quot;VarSome genome browser showing transcripts&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the Sift scores you see are indeed one per transcript, it's just that there are 6 because dbNSFP also includes a score for the non-coding transcript of the gene. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The dots are just there as placeholders, they mean there was no value associated with that transcript. Many tools will show some sort of symbol instead of an empty field both for clarity and for practical technical reasons. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Visiting the &lt;a href=&quot;https://varsome.com/variant/hg38/rs371839875&quot; rel=&quot;nofollow noreferrer&quot;&gt;variant's page&lt;/a&gt; on VarSome gives you a clearer picture since we collapse the identical scores and also include the converted rankscore provided by dbNSFP so you can have a single number for your variant:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cyfSO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cyfSO.png&quot; alt=&quot;VarSome screenshot showing SIFT score&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Disclaimer&lt;/strong&gt;: I work for the company behind &lt;a href=&quot;https://varsome.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;VarSome&lt;/a&gt;, but it's a free tool. You need to pay to annotate VCF files (unlike the 100% free VEP), but it's free to use as a lookup tool for single variants.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-04-10T12:13:28.950" LastActivityDate="2019-04-10T12:13:28.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7434" PostTypeId="2" ParentId="7411" CreationDate="2019-04-10T12:17:24.910" Score="1" Body="&lt;p&gt;When I had to do something similar previously, I had used the following method using &lt;code&gt;sed&lt;/code&gt; and &lt;code&gt;awk&lt;/code&gt;. It is not as robust in terms of error detection and correction as the python solution but it is a simple one-liner. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat temp.wig&#xA;variableStep chrom=chr1 span=20&#xA;59965   7&#xA;59985   10&#xA;60005   10&#xA;60025   10&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;$ sed -r 's/variableStep chrom=(.*) span=([0-9]*)/\1\t\2/g' temp.wig \&#xA;        | awk 'BEGIN{FS=&quot;\t&quot;;OFS=&quot;\t&quot;}{if ($1~/^chr/) {c=$1 ; s=$2} else {print c,$1,$1+s,$&lt;/span&gt;2}}' \&#xA;    &amp;gt; temp.bedgraph&#xA;&#xA;$ cat temp.bedgraph&#xA;chr1    59965   59985   7&#xA;chr1    59985   60005   10&#xA;chr1    60005   60025   10&#xA;chr1    60025   60045   10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2019-04-10T12:17:24.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7435" PostTypeId="1" CreationDate="2019-04-10T12:55:49.570" Score="1" ViewCount="59" Body="&lt;p&gt;I have constructed a gene co-expression network from RNA-seq data. The network file is in edge list format of memory around 1gb which was created by calculating Pearson correlation of each gene pairs and the gene pairs which having correlation &gt;95% were selected to create the edge list. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have clustered this gene network (edge list) using igraph R package &quot;cluster_louvian&quot; community detection algorithm and obtained 534 subclusters. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;c &amp;lt;- cluster_louvian(g)&#xA;c$membership&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to get largest subcluster&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sub1 &amp;lt;- induced.subgraph(g, membership(cl) ==order(-sizes(cl))[1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am finding difficulty to write the subcluster into a text file as an edge list&#xA;How can I save each subcluster in separate files (text files) as edge list?&lt;/p&gt;&#xA;" OwnerUserId="2811" LastEditorUserId="48" LastEditDate="2019-04-11T12:51:02.350" LastActivityDate="2019-04-11T12:51:02.350" Title="How to write the subclusters in file?" Tags="|r|rna-seq|networks|clustering|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7436" PostTypeId="1" AcceptedAnswerId="7444" CreationDate="2019-04-10T15:51:29.480" Score="5" ViewCount="847" Body="&lt;p&gt;I am used to &lt;code&gt;gzip&lt;/code&gt;/&lt;code&gt;biopython&lt;/code&gt; solutions when dealing with sequencing data, but now I wish to switch to more elegant &lt;code&gt;pysam&lt;/code&gt;. So I looked at &lt;a href=&quot;https://pysam.readthedocs.io/en/latest/api.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the manual&lt;/a&gt;, but ran into quite bizarre troubles with the first couple of lines using my bam file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pysam&#xA;samfile = pysam.AlignmentFile(&quot;3_Tms_1_mapped.bam&quot;, &quot;rb&quot;)&#xA;for read in samfile.fetch('3_Tms_b3v08_scaf000159'):&#xA;    print(read)&#xA;samfile.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;returns &lt;code&gt;ValueError: fetching by region is not available for SAM files&lt;/code&gt;. Well, the file is &lt;code&gt;bam&lt;/code&gt;. I tried to google the error but the only hits I found are &lt;a href=&quot;https://github.com/pysam-developers/pysam/blob/master/pysam/libcalignmentfile.pyx#L1085-L1109&quot; rel=&quot;nofollow noreferrer&quot;&gt;the lines in the source code of pysam&lt;/a&gt; that check if the file is bam/cram or sam, so somehow &lt;code&gt;pysam&lt;/code&gt; thinks that my bam is a sam. How can I convince it otherwise? I have also noticed that the manual is for python 2.7, that's maybe where the problem comes from...&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2019-07-12T16:10:59.320" LastActivityDate="2019-07-12T16:10:59.320" Title="What does &quot;fetching by region is not available for SAM files&quot; mean?" Tags="|python|sam|pysam|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7437" PostTypeId="2" ParentId="7436" CreationDate="2019-04-10T16:10:46.163" Score="5" Body="&lt;p&gt;That isn't actually a bam file &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/7444/298&quot;&gt;as John Marshall figured out&lt;/a&gt;. I am keeping the rest of my answer since it could be useful to someone else, but the issue here was that you had a compressed (bgzipped) sam file and not an actual bam file and that's why you were getting that error. When I sorted your file in preparation for indexing it, I converted to a bam which is why the rest of my answer worked.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;You don't have the index file for the bam file you're using. I used this script on the file you linked to (changing the name so that it corresponds to the right file and a contig in that file):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;import pysam&#xA;samfile = pysam.AlignmentFile(&quot;3_Tce_1_mapped.bam&quot;, &quot;rb&quot;)&#xA;for read in samfile.fetch('3_Tce_b3v08_scaf005149'):&#xA;    print(read)&#xA;samfile.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The directory I ran it in had:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ls 3*&#xA;3_Tce_1_mapped.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I got the error you described:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ foo.py&#xA;Traceback (most recent call last):&#xA;  File &quot;/home/terdon/scripts/foo.py&quot;, line 5, in &amp;lt;module&amp;gt;&#xA;    for read in samfile.fetch('3_Tce_b3v08_scaf005149'):&#xA;  File &quot;pysam/libcalignmentfile.pyx&quot;, line 1107, in pysam.libcalignmentfile.AlignmentFile.fetch&#xA;ValueError: fetching by region is not available for SAM files&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, indexing the bam file fixed it:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ samtools sort 3_Tce_1_mapped.bam &amp;gt; 3_Tce_1_mapped.sorted.bam &#xA;$&lt;/span&gt; mv 3_Tce_1_mapped.sorted.bam 3_Tce_1_mapped.bam&#xA;&lt;span class=&quot;math-container&quot;&gt;$ samtools index 3_Tce_1_mapped.bam &#xA;$&lt;/span&gt; ls 3*&#xA;3_Tce_1_mapped.bam  3_Tce_1_mapped.bam.bai&#xA;&#xA;$ foo.py | wc&#xA;227    2724   16725&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So just sort and index your files before attempting to seek in them. Which makes sense since the index's job is primarily to allow seeking.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-04-11T09:54:03.857" LastActivityDate="2019-04-11T09:54:03.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7438" PostTypeId="1" AcceptedAnswerId="7439" CreationDate="2019-04-10T16:42:48.980" Score="1" ViewCount="305" Body="&lt;p&gt;I have been given a bunch of .bam files, that come from the alignment on GRCh37d5 of human genome by Star tool. I want to extract raw read counts from these bam files by FEATURECOUNTS tool so I need a GTF. I googled a lot but I did not find GTF for GRCh37d5. I used another GTFs and I obtained weird features. Please one of you help me in finding this specific GTF. I converted a bam to fastq and I used Salmon to quantify gene expression by a transcripome.fasta but rate of mapping was 0.06&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-04-10T18:42:12.907" LastActivityDate="2019-04-10T18:42:12.907" Title="A GTF annotation for GRCH375d version" Tags="|rna-seq|sequence-alignment|sequence-annotation|gtf|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7439" PostTypeId="2" ParentId="7438" CreationDate="2019-04-10T17:07:57.997" Score="2" Body="&lt;p&gt;The most recent annotation of GRCh37 assembly by NCBI RefSeq is here: &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/genomes/Homo_sapiens/GRCh37.p13_interim_annotation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nlm.nih.gov/genomes/Homo_sapiens/GRCh37.p13_interim_annotation/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can download the annotation in GFF3 format from that path and use it with featurecounts tool by first converting it to the SAF format using the commands shown below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ zgrep -v '^#' annotation.gff.gz \&#xA;    | awk 'BEGIN{FS=&quot;\t&quot;;OFS=&quot;\t&quot;}($&lt;/span&gt;3==&quot;exon&quot;){print &lt;span class=&quot;math-container&quot;&gt;$1,$&lt;/span&gt;4,&lt;span class=&quot;math-container&quot;&gt;$5,$&lt;/span&gt;7,&lt;span class=&quot;math-container&quot;&gt;$9}' \&#xA;    | sed -r 's/ID.*GeneID:([0-9]*).*/\1/g' \&#xA;    | awk 'BEGIN{FS=&quot;\t&quot;;OFS=&quot;\t&quot;}{print $5,$1,$2,$3,$&lt;/span&gt;4}'&#xA;    &amp;gt; annotation.saf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You should confirm that the chromosome sequence identifiers in the BAM file and the SAF file match. NCBI annotation has identifiers in the accession.version format whereas UCSC uses &lt;code&gt;chr#&lt;/code&gt; format and Ensembl uses &lt;code&gt;#&lt;/code&gt;. For this, you can use the &lt;code&gt;-A&lt;/code&gt; parameter from &lt;code&gt;featureCounts&lt;/code&gt;. Assuming that your BAM file uses UCSC-style identifiers, you can generate a 'chromosome name alias file' that can be used with featureCounts as follows: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;## get NCBI assembly report for GRCh37&#xA;wget -nv ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/001/405/GCF_000001405.24_GRCh37.p12/GCF_000001405.24_GRCh37.p12_assembly_report.txt&#xA;&#xA;## generate chr_alias.csv file&#xA;grep -v '^#' GCF_000001405.24_GRCh37.p12_assembly_report.txt \&#xA;    | dos2unix \&#xA;    | awk 'BEGIN{FS=&quot;\t&quot;;OFS=&quot;,&quot;}(&lt;span class=&quot;math-container&quot;&gt;$10!=&quot;na&quot;){print $10,$&lt;/span&gt;7}' \&#xA;    &amp;gt; chr_alias.csv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2019-04-10T17:07:57.997" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7440" PostTypeId="2" ParentId="7421" CreationDate="2019-04-10T17:18:52.907" Score="6" Body="&lt;p&gt;The goal of a phylogeny is to estimate the &quot;expected&quot; number of mutations between all taxa in the analysis and their hypothetical common ancestors. A cluster-analysis will only identify the &quot;observed&quot; mutations and &quot;expected&quot; and &quot;observed&quot; mutations can be majorly different due to the major artefact of reversion mutation. This is particularly true of nucleotide phylogenies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key difference between clustering algorithms based on a &quot;uncorrected&quot; distance matrix and phylogeny is the latter is based on an explicit model to accommodate reversion mutations. The real problem is that there are only 4 bases, so randomly there is 1/4 chance that a mutation at a given position will mutate back to the original, e.g. A-&gt;C-&gt;A. Observed mutational differences = 0, expected (real) mutational differences = 2. What phylogeny is concerned about is reconstructing that &quot;2&quot;. The problem is significant because almost every gene has regions of rapid mutations and regions of low mutations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The primary way of doing this is via the Jukes-Cantor correction and is foundational in all nucleotide trees expect &quot;p-distances&quot;. If nucleotide divergence is lower than 75% then it is possible to estimate the expected number of mutations using the observed via the JC correction. Moreover, when combined with a method of estimating rate variation within a gene, (usually the discrete gamma distribution), JC correction is very effective at recovering the &quot;true tree&quot;. This is because fast evolving homologous sites are grouped together - big correction via JC, slow evolving sites are grouped together - small correction via JC. Other approaches to improving the JC correction are via identifying the bias between purine to purine mutations and pyrimidine to pyrimidine and purine to pyrimidine mutations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The importance of JC was demonstrated by simulation studies of 4 taxa ((a,b), (c,d)), if two strains evolved very quickly when there sister lineages evolve slowly a clustering algorithm will report the fast strains are a sister group, i.e. ((b, c),(a, d)). If the JC method is implemented via maximum likelihood (or Bayesian) then it correctly recovers the true tree ((a,b), (c,d)). The artefact is known as long branch attraction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clustering algorithms based on a distance matrix that implements the JC correction, tend to perform poorly for long branch-attraction artefacts. This doesn't mean the correction is useless, just not particularly powerful. The issue is that distance matrix methods don't &quot;stick&quot; to the model and the clustering will introduce a layer of inaccuracy. Normally presentation of a &quot;corrected&quot; distance matrix combined with a clustering algorithm will require bootstrapping (resampling with replacement) to assess whether a given cluster is supported. Parameterised distance matrices, using neighbour-joining clustering in in-conjunction with a bootstrap are considered okay.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The @user172818 mentioned parsimony and this method is considered less reliable because it can't implement a JC correction. IMO it is possible weighted parsimony could make a &quot;come-back&quot;, but it would be really complicated to implement a biological weighting method and would require extensive, independent calculations.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-04-11T04:59:04.753" LastActivityDate="2019-04-11T04:59:04.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7441" PostTypeId="1" CreationDate="2019-04-10T19:41:03.983" Score="2" ViewCount="481" Body="&lt;p&gt;I am trying to perform reference-based assembly. Most of the tutorials teach how to create a bam file and view alignemnts in IGV or Tablet. But, I want a assembled genome sequence in fasta format. How can I do that ? Is there any tool or linux syntax available for this purpose ?&lt;/p&gt;&#xA;" OwnerUserId="1684" LastActivityDate="2019-04-10T19:52:57.977" Title="Consensus sequence from SAM or BAM file?" Tags="|sam|genome|assembly|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7442" PostTypeId="2" ParentId="7441" CreationDate="2019-04-10T19:52:57.977" Score="1" Body="&lt;p&gt;A few years ago I used AlignGraph (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4058956/&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; and &lt;a href=&quot;https://github.com/baoe/AlignGraph&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;) for a task like this. I gave AlignGraph my raw reads, so I'm not sure if it accepts SAM/BAM input or if you have to go back to the original Fastq reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure if any related/improved tools have been published since AlignGraph. Internet searches for &quot;reference-guided assembly&quot; return mostly transcriptome assembly tools. It might be worth looking for &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed?linkname=pubmed_pubmed_citedin&amp;amp;from_uid=24932000&quot; rel=&quot;nofollow noreferrer&quot;&gt;articles that cite the AlignGraph paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-04-10T19:52:57.977" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7443" PostTypeId="2" ParentId="5529" CreationDate="2019-04-10T20:00:07.630" Score="2" Body="&lt;p&gt;The same error occurred to me with &lt;code&gt;DESeq2_1.22.2&lt;/code&gt; using &lt;code&gt;R 3.5.0&lt;/code&gt;.&#xA;By trial and error, I managed to solve the issue when I updated &lt;code&gt;SummarizedExperiment&lt;/code&gt; from &lt;code&gt;1.10.1&lt;/code&gt; to &lt;code&gt;1.12.0&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4410" LastEditorUserId="57" LastEditDate="2019-04-11T09:37:31.453" LastActivityDate="2019-04-11T09:37:31.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7444" PostTypeId="2" ParentId="7436" CreationDate="2019-04-10T20:43:55.863" Score="5" Body="&lt;p&gt;Your &lt;em&gt;3_Tms_1_mapped.bam&lt;/em&gt; file, despite its filename extension, is in fact a bgzipped &lt;strong&gt;SAM&lt;/strong&gt; file. You can verify this using &lt;a href=&quot;http://www.htslib.org/doc/htsfile.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;htsfile&lt;/code&gt;&lt;/a&gt;, which is a small utility packaged with HTSlib:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;$ htsfile 3_Tms_1_mapped.bam &#xA;3_Tms_1_mapped.bam: SAM version 1.3 BGZF-compressed sequence data&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(For files that really are in BAM format, it reports &lt;em&gt;BAM version 1 compressed sequence data&lt;/em&gt;.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the error message is accurate in this case.&lt;/p&gt;&#xA;" OwnerUserId="134" LastEditorUserId="134" LastEditDate="2019-04-11T06:36:38.400" LastActivityDate="2019-04-11T06:36:38.400" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7447" PostTypeId="1" CreationDate="2019-04-11T16:38:34.413" Score="1" ViewCount="92" Body="&lt;p&gt;What to do after protein-ligand docking? Is there any necessary or enhancement step recommended after docking to validate the confirmation of highest score is the best? For your information, I am using CCDC GOLD for docking and GOLD score as my scoring function.&lt;/p&gt;&#xA;" OwnerUserId="3760" LastActivityDate="2023-11-02T06:20:39.693" Title="Post docking evaluation" Tags="|proteins|protein-structure|docking|protein-protein-interaction|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7448" PostTypeId="1" CreationDate="2019-04-12T14:09:14.430" Score="0" ViewCount="509" Body="&lt;p&gt;I have a .vcf file&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/8v73nppwg3a1tnd/LP2000109-DNA_A01_vs_LP2000103-DNA_A01.SVannotated.txt?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/8v73nppwg3a1tnd/LP2000109-DNA_A01_vs_LP2000103-DNA_A01.SVannotated.txt?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;with this header&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##startTime=Fri Mar 29 16:46:32 2019&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  NORMAL  TUMOR&#xA;1   54586   .   T   C   .   PASS    DP=39;MQ=50.55;MQ0=0;NT=ref;QSS=48;QSS_NT=48;ReadPosRankSum=1.92;SGT=TT-&amp;gt;CT;SNVSB=0.00;SOMATIC;SomaticEVS=10.83;TQSS=1;TQSS_NT=1    AU:CU:DP:FDP:GU:SDP:SUBDP:TU    0,0:0,0:20:0:0,0:0:0:20,20  0,0:6,6:18:0:0,0:0:0:12,13&#xA;1   103241  .   C   T   .   PASS    DP=120;MQ=24.94;MQ0=35;NT=ref;QSS=47;QSS_NT=47;ReadPosRankSum=2.09;SGT=CC-&amp;gt;CT;SNVSB=0.00;SOMATIC;SomaticEVS=9.44;TQSS=2;TQSS_NT=2   AU:CU:DP:FDP:GU:SDP:SUBDP:TU    0,1:32,47:33:1:0,0:0:0:0,5  0,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &amp;quot;DP&amp;quot; field in the vcf shows the depth of the individual samples; So in this file, the first locus has the following format fields:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;AU:CU:DP:FDP:GU:SDP:SUBDP:TU    0,0:0,0:20:0:0,0:0:0:20,20  0,0:6,6:18:0:0,0:0:0:12,13&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So according to this (DP field of normal and tumor samples), normal sample has a depth of 20 and tumor sample has a depth of 18.&lt;/p&gt;&#xA;&lt;p&gt;So how I could extract the read depth for all loci as described for the first position? The desired output would be like this [note that the VCF is taken from my own data but the table is my desired format that I don't know how to get that from my own data. chr have been added manually because my reference genome is hg19]:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  Sample   Type   CHROM       POS    REF  ALT       Tumor_Depth Normal_Depth&#xA;  CHC2432T SNV    chr1  102961055    G    A                  64           62      &#xA;  CHC2432T SNV    chr1  105492588    A    T                  66           73     &#xA;  CHC2432T SNV    chr1  108628724    C    T                  45           54    &#xA;  CHC2432T SNV    chr1  109692113    G    T                  53           29     &#xA;  CHC2432T SNV    chr1  109692114    G    T                  53           31     &#xA;  CHC2432T SNV    chr1  120676701    T    C                  48           87   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="3694" LastEditDate="2021-06-08T10:54:00.147" LastActivityDate="2021-06-08T10:54:00.147" Title="Parsing .vcf file for this information" Tags="|vcf|wgs|snv|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7449" PostTypeId="2" ParentId="7448" CreationDate="2019-04-12T19:27:27.120" Score="1" Body="&lt;p&gt;To extract the DP fields from a VCF file, you could use a tool like &lt;a href=&quot;https://www.htslib.org/doc/bcftools.html#query&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;bcftools query&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Extracts fields from VCF or BCF files and outputs them in user-defined format.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You could start from something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools query -Hf 'CHC2432T\t%TYPE\t%CHROM\t%POS\t%REF\t%ALT[\t%DP]\n' file.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3351" LastActivityDate="2019-04-12T19:27:27.120" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7450" PostTypeId="1" CreationDate="2019-04-13T00:30:38.403" Score="0" ViewCount="1006" Body="&lt;p&gt;I am new to &lt;code&gt;python&lt;/code&gt; and trying to learn. The below is an attempt to filter out secondary reads in a &lt;code&gt;bam&lt;/code&gt; using &lt;code&gt;pysam&lt;/code&gt;. I ran &lt;code&gt;mark duplicates&lt;/code&gt; and got an error on several &lt;code&gt;bam&lt;/code&gt; files due to secondary alignments. I added comments to each line to illustrate my thought process.  Thank you :).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The specific &lt;code&gt;mark duplicates&lt;/code&gt; error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Value was put into PairInfoMap more than once&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#! /usr/bin/python  ## call python script&#xA;import sys ## import python system functions&#xA;import pysam  ## import module&#xA;&#xA;bam = pysam.AlignmentFile(*.bam, &quot;rb&quot;)  ## open bam and read&#xA;click.echo(&quot;Reading BAM file&quot;)  ## output message&#xA;for read in bam:  ## start loop and iterate over each bam&#xA;    if read.is_secondary=true  ## not the primary alignment&#xA;    if read.has_tag('XS')  ## look for Secondary Alignment Score tag&#xA;        XS = read.get_tag('XS')  ## read and store AS value&#xA;    if read.has_tag('AS')  ## look for Alignment Score tag&#xA;        AS = read.get_tag('AS')  ## read and store value os XS tag  &#xA;    if AS &amp;lt; XS  ## Alignment score less then &#xA;        read.write(read)  ## only print primary alignments&#xA;bam.close()  ## end processing&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4424" LastEditorUserId="57" LastEditDate="2019-04-15T15:55:13.707" LastActivityDate="2019-04-15T15:55:13.707" Title="filter secondary alignments using pysam" Tags="|python|pysam|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7451" PostTypeId="2" ParentId="7450" CreationDate="2019-04-13T02:41:48.677" Score="3" Body="&lt;p&gt;Quick look at your script suggests that you are comparing the XS and AS from different reads. I would recommend removing the continue from the &lt;code&gt;read.has_tag('XS')&lt;/code&gt; continue block.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that if you filter out &lt;code&gt;is_secondary&lt;/code&gt; and &lt;code&gt;is_supplementary&lt;/code&gt; you don't need to do the XS to AS comparison assuming whatever generated for BAM did so correctly.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for read in bam.fetch()  &#xA;    if read.is_secondary=true  &#xA;        continue&#xA;    if read.has_tag('XS') &amp;lt;- This block stores XS and then moves on to the next read&#xA;        XS = read.get_tag('XS')  &#xA;        continue &#xA;    if read.has_tag('AS') &#xA;        AS = read.get_tag('AS')  &#xA;    if AS &amp;lt; XS  &amp;lt;- This block compares the AS of the current read to the XS of the previous read that had one.&#xA;        read.write(read)  &#xA;bam.close()  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="964" LastActivityDate="2019-04-13T02:41:48.677" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7453" PostTypeId="1" AcceptedAnswerId="7462" CreationDate="2019-04-14T21:56:03.733" Score="2" ViewCount="674" Body="&lt;p&gt;I have data that includes 'cases' and 'controls' and have carried out hierarchical clustering. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used the &lt;code&gt;pvclust&lt;/code&gt; package in R to bootstrap the results and significant branches are highlighted with red rectangles (based on au&gt;0.95):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/C3kwC.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/C3kwC.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is clear is that no clustering occurs that separates 'cases' and 'controls', and this is in fact what we expected and want to show. We want to show that the measured variable does not distinguish between cases and controls.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;List &lt;code&gt;itemApart&lt;/code&gt; from saying visually no clear clusters emerge that distinguishes between cases and controls are there any objective measures that can be used to say no significant clustering occurs between two groups?&lt;/li&gt;&#xA;&lt;li&gt;One observation I have here is that the AU values and the BP values are very different, even though both p-values should be interpreted in a similar fashion, am I missing something?&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Perhaps &lt;code&gt;pvclust&lt;/code&gt; (bootstrapping) is not the right option here, is there a better way of showing quantifiably that no significant clustering occurs between two groups? Perhaps some kind of supervised clustering (I am not sure what this even means in this context)?&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2159" LastEditorUserId="57" LastEditDate="2019-04-15T09:17:09.213" LastActivityDate="2019-04-17T17:27:01.810" Title="Interpretation of hierarchical clustering with bootstrapping" Tags="|r|clustering|" AnswerCount="2" CommentCount="7" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7456" PostTypeId="1" CreationDate="2019-04-15T04:03:33.463" Score="1" ViewCount="105" Body="&lt;p&gt;I finished the sequencing of the genome of a biocontrol activity strain &lt;em&gt;Bacillus amyloliquefaciens&lt;/em&gt; by Illumina hiseq. I have already gone through some papers but there were no details of processing in it.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is anywhere I can find some resources to get my research going on to genome annotation, compare it with other microbial genome data, and analyze non-ribosomal peptide synthetase, find out more genomic information about biocontrol activity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, how to locate 16S DNA and some lipopeptide antibiotic genes (fungycin gene, surfactin gene) in my bacterial genome, and compare it with the reference strain and draw a phylogenetic tree. And how to use my bacterial genome and 2 reference genomes to draw a genomic circle diagram.&lt;/p&gt;&#xA;" OwnerUserId="4432" LastEditorUserId="4432" LastEditDate="2020-04-05T07:00:14.790" LastActivityDate="2021-01-29T23:39:49.007" Title="Genome annotation of a Bacillus strain" Tags="|sequence-annotation|genome|sequence-analysis|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7457" PostTypeId="1" AcceptedAnswerId="7467" CreationDate="2019-04-15T05:49:52.290" Score="0" ViewCount="161" Body="&lt;p&gt;My professor in class said the following statements. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Short reads can map break points&lt;/li&gt;&#xA;&lt;li&gt;Long reads can disambiguate repeat related issues&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I'm very new to genomics, so I don't quite understand the first statement. What is meant by short reads can map break points. What are break points to begin with?&lt;/p&gt;&#xA;" OwnerUserId="2519" LastActivityDate="2019-04-16T07:43:30.890" Title="Short reads vs long reads with regards to break points" Tags="|ngs|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7458" PostTypeId="1" AcceptedAnswerId="7459" CreationDate="2019-04-15T09:46:12.963" Score="14" ViewCount="7473" Body="&lt;p&gt;I have a DNA sequence for which I would like to quickly find the reverse complement. Is there a quick way of doing this on the bash command line using only GNU tools?&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="492" LastEditDate="2021-02-07T19:22:54.950" LastActivityDate="2021-02-07T19:22:54.950" Title="What is a quick way to find the reverse complement in bash" Tags="|sequence-analysis|phylogenetics|bash|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7459" PostTypeId="2" ParentId="7458" CreationDate="2019-04-15T09:46:12.963" Score="32" Body="&lt;p&gt;Thanks to &lt;a href=&quot;http://manutamminen.info/posts/unix_tricks/&quot; rel=&quot;noreferrer&quot;&gt;Manu Tamminen&lt;/a&gt; for this solution:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;echo ACCTTGAAA | tr ACGTacgt TGCAtgca | rev&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="492" LastEditorUserId="104" LastEditDate="2019-04-15T14:03:06.827" LastActivityDate="2019-04-15T14:03:06.827" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7460" PostTypeId="1" CreationDate="2019-04-15T14:51:57.273" Score="2" ViewCount="135" Body="&lt;p&gt;In the &lt;a href=&quot;https://www.tbi.univie.ac.at/RNA/tutorial/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; of RNAfold it states,  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;RNAfold&lt;/code&gt; reads RNA sequences from stdin, calculates their minimum free energy (MFE) structure, prints the MFE structure in dot-bracket notation and its free energy to stdout.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the example using the &lt;code&gt;-p --MEA&lt;/code&gt; options, the &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;MFE = -5.00&lt;/strong&gt; and the &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;ensemble Free Energy = -5.72&lt;/strong&gt;. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;How can the free energy of the ensemble be lower than the minimum free energy?&lt;/p&gt;&#xA;" OwnerUserId="2970" LastEditorUserId="3848" LastEditDate="2020-04-18T08:52:16.457" LastActivityDate="2020-04-18T08:52:16.457" Title="How is that ensemble Free energy is lower than the MFE in RNAfold from ViennaRNA" Tags="|rna|rna-structure|thermodynamics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7461" PostTypeId="2" ParentId="2078" CreationDate="2019-04-16T00:13:57.217" Score="1" Body="&lt;p&gt;A concise description for using Arlecore on terminal for MAC, LINUX or Windows is provided here:&#xA;&lt;a href=&quot;http://cmpg.unibe.ch/software/arlequin35/man/arlecore_readme.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cmpg.unibe.ch/software/arlequin35/man/arlecore_readme.txt&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4438" LastActivityDate="2019-04-16T00:13:57.217" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7462" PostTypeId="2" ParentId="7453" CreationDate="2019-04-16T00:16:45.440" Score="1" Body="&lt;p&gt;Good question. You have performed the first step via unsupervised learning and AU bootstraps give great results, orthodox bootstraps give no significant (or very few) clusters. The contrast between these approaches is unusual because they should be somewhere close to each other. If you can resolve that, your analysis is fine. So at present IMO the unsupervised learning needs further investigation. &#xA;... deleted &lt;/p&gt;&#xA;&#xA;&lt;p&gt;... &lt;strong&gt;antibodies&lt;/strong&gt; ... oh I see. Trees, like you've done, don't work here because of the cross-reactivity creates network phylogenies. This is the antonym of bifurcating trees. If someone has a robust bifurcating phylogeny with antibody data and gets good bootstraps .. utterly amazing. However, if you subject your data to a network analysis you'll find connections all over the place - in practical terms this really messes up bootstraps. Thats the likely reason for you results. There is no point doing a complex bifurcating tree analysis because the antibody cross-reactivity will mess it up. k-means has been implemented as unsupervised learning in these instances and would provide the starting point in using clustering to define a supervised learning problem.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-04-16T19:22:25.900" LastActivityDate="2019-04-16T19:22:25.900" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7463" PostTypeId="1" AcceptedAnswerId="7464" CreationDate="2019-04-16T04:12:21.217" Score="0" ViewCount="209" Body="&lt;p&gt;Does one chunk correspond to the raw signal of a single read read? Why does messing with this parameter affect basecalling results?&lt;/p&gt;&#xA;" OwnerUserId="4439" LastActivityDate="2019-04-16T06:24:17.443" Title="What is a signal 'chunk' in the context of Nanopore sequencing?" Tags="|nanopore|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7464" PostTypeId="2" ParentId="7463" CreationDate="2019-04-16T06:24:17.443" Score="1" Body="&lt;p&gt;If you have access to the nanopore community website you can read more about this here: &lt;a href=&quot;https://community.nanoporetech.com/posts/not-chunky-enough&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://community.nanoporetech.com/posts/not-chunky-enough&lt;/a&gt; (I'm not sure to which extent I can copy paste from there so I won't).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A TLDR version: basecalling doesn't happen simultaneously for an entire read, but it is broken in segments which are processed separately and combined afterward. Those segments are chunks. Basecalling results are affected mostly by low-complexity regions, in which a heavily biased region (in terms of nucleotide content) leads to artifacts in the per-chunk basecalling. Those issues are solved by taking a chunking-size larger than most low-complexity regions.&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2019-04-16T06:24:17.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7465" PostTypeId="2" ParentId="7457" CreationDate="2019-04-16T07:15:41.427" Score="3" Body="&lt;p&gt;A &quot;break point&quot; typically denotes one end of a structural variation, such as an inversion or deletion. You can simply think of it as the position where the normal chromosome was broken to allow the variation. One can find these with short (Illumina) reads relatively easily since with paired end sequencing you'll have pairs spanning these breakpoints with odd relative orientations or insert sizes or such. Note that this is not a particular strength of short reads, since long reads are probably even better at this (they are not as good for finding SNPs at the moment though).&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2019-04-16T07:42:12.397" LastActivityDate="2019-04-16T07:42:12.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7466" PostTypeId="1" AcceptedAnswerId="7468" CreationDate="2019-04-16T07:18:51.067" Score="1" ViewCount="1177" Body="&lt;p&gt;This is the one i used for a single bam file to filter its mitochondrial as well as unassembled read&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h SRR2920466.bam | awk '{if(&lt;span class=&quot;math-container&quot;&gt;$3!= &quot;chrM&quot; &amp;amp;&amp;amp; $&lt;/span&gt;3 != &amp;quot;chrUn&amp;quot;){print $0}}' | samtools view -Shb - &amp;gt; this.filter.bam &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there any way i can do it on multiple files with parallel as i do to index and sort bam files using samtools .&lt;/p&gt;&#xA;&lt;p&gt;So i tried this ,which is of course not working as its not able to pipe properly because it shows me error regarding additional fonts&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ls *.bam | parallel &amp;quot;samtools view -h {} | awk '{if(&lt;span class=&quot;math-container&quot;&gt;$3 != &quot;chrM&quot; &amp;amp;&amp;amp; $&lt;/span&gt;3 != &amp;quot;chrUn&amp;quot;){print $0}}'| samtools view -Shb {.}.bam&amp;quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any help with my above codes? As i get it there is wrong with my pipe .&lt;/p&gt;&#xA;&lt;p&gt;After updating the samtools to version 1.9 i ran the small script&lt;/p&gt;&#xA;&lt;p&gt;i get this error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk: cmd. line:1: {if( != chrM &amp;amp;&amp;amp;  != chrUn){print ./index_sort.sh}}&#xA;awk: cmd. line:1:      ^ syntax error&#xA;awk: cmd. line:1: {if( != chrM &amp;amp;&amp;amp;  != chrUn){print ./index_sort.sh}}&#xA;awk: cmd. line:1:                                    ^ unterminated regexp&#xA;awk: cmd. line:1: {if( != chrM &amp;amp;&amp;amp;  != chrUn){print ./index_sort.sh}}&#xA;awk: cmd. line:1:                                                   ^ unexpected newline or end of string&#xA;awk: cmd. line:1: {if( != chrM &amp;amp;&amp;amp;  != chrUn){print ./index_sort.sh}}&#xA;awk: cmd. line:1:      ^ syntax error&#xA;awk: cmd. line:1: {if( != chrM &amp;amp;&amp;amp;  != chrUn){print ./index_sort.sh}}&#xA;awk: cmd. line:1:                                    ^ unterminated regexp&#xA;awk: cmd. line:1: {if( != chrM &amp;amp;&amp;amp;  != chrUn){print ./index_sort.sh}}&#xA;awk: cmd. line:1:                                                   ^ unexpected newline or end of string&#xA;awk: cmd. line:1: {if( != chrM &amp;amp;&amp;amp;  != chrUn){print ./index_sort.sh}}&#xA;awk: cmd. line:1:      ^ syntax error&#xA;awk: cmd. line:1: {if( != chrM &amp;amp;&amp;amp;  != chrUn){print ./index_sort.sh}}&#xA;awk: cmd. line:1:                                    ^ unterminated regexp&#xA;awk: cmd. line:1: {if( != chrM &amp;amp;&amp;amp;  != chrUn){print ./index_sort.sh}}&#xA;awk: cmd. line:1:                                                   ^ unexpected newline or end of string&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have 30 bam files i tried with 2 to see if its working or not , after the suggestion of @WouterDeCoster i tried the above one after updating samtools . the basic samtool with egrep works but for multiple file i couldn;t do it.&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="73" LastEditDate="2023-01-15T20:21:24.307" LastActivityDate="2023-01-15T20:21:24.307" Title="removing mitochondrial read and unassembled &quot;random&quot; from multiple bam files" Tags="|awk|shell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7467" PostTypeId="2" ParentId="7457" CreationDate="2019-04-16T07:43:30.890" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Short reads can map break points&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;With short reads you do an &quot;indirect observation/inference&quot; of a structural variant: based on your paired-end reads you figure out something is wrong because their distance or orientation is not what you expect. With long reads, you often get a direct observation of a breakpoint, with sufficient long reads on both sides (either split alignment or spanning the SV) allowing almost nucleotide accuracy. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Long reads can disambiguate repeat related issues&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Short reads cannot align unambiguously in many repetitive regions, because the repeat is longer than the read. Long reads, however, span most common repeats entirely, allowing you to unambiguously assign a location for each read, as such resolving the issues.&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2019-04-16T07:43:30.890" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7468" PostTypeId="2" ParentId="7466" CreationDate="2019-04-16T10:54:04.467" Score="1" Body="&lt;p&gt;Your awk command is inside double quotes (&lt;code&gt;&quot;&quot;&lt;/code&gt;) which means that all of the awk field variables (e.g. &lt;code&gt;$3&lt;/code&gt;) are being interpreted by the shell. Since &lt;code&gt;$3&lt;/code&gt; is not set in the shell, your awk is trying to compare an empty string and that's why you get the error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk: cmd. line:1: {if( != chrM &amp;amp;&amp;amp;  != chrUn){print ./index_sort.sh}}&#xA;awk: cmd. line:1:      ^ syntax error&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note how the error message shows &lt;code&gt;if( != chrM&lt;/code&gt; and not &lt;code&gt;if($3 != chrM&lt;/code&gt;. Since you &lt;em&gt;need&lt;/em&gt; to use double quotes here because you need both an outer (for &lt;code&gt;parallel&lt;/code&gt;) and an inner (for awk) quote, and single quotes can't be nested, you will have to escape the awk variables:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ls *.bam | parallel &quot;samtools view -h {} | &#xA;    awk '{if(\$3 != &quot;chrM&quot; &amp;amp;&amp;amp; \$3 != &quot;chrUn&quot;){print \$0}}' | samtools view -Shb {.}.bam&quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-04-16T10:54:04.467" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="7469" PostTypeId="2" ParentId="2825" CreationDate="2019-04-16T13:32:08.047" Score="3" Body="&lt;p&gt;I tried the &lt;code&gt;pyfaidx&lt;/code&gt; approach reported by Jared, but I got unhandled exceptions that I could not handle easily within the framework. I had &lt;code&gt;bcftools&lt;/code&gt; lying around, so I tried &lt;a href=&quot;https://samtools.github.io/bcftools/howtos/consensus-sequence.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;bcftools consensus&lt;/code&gt;&lt;/a&gt; and it worked like a charm. However, you do need an indexed VCF.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;bcftools consensus all-site.vcf.gz &amp;lt; input.fasta &amp;gt; output.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;NB: &lt;code&gt;bcftools consensus&lt;/code&gt; has &lt;a href=&quot;https://samtools.github.io/bcftools/bcftools.html#consensus&quot; rel=&quot;nofollow noreferrer&quot;&gt;a few options&lt;/a&gt; specified with the &lt;code&gt;--haplotype&lt;/code&gt; argument for choosing which alleles should be incorporated into the FASTA file.&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="492" LastEditDate="2021-07-03T15:12:45.313" LastActivityDate="2021-07-03T15:12:45.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7470" PostTypeId="1" CreationDate="2019-04-16T20:16:44.703" Score="2" ViewCount="60" Body="&lt;p&gt;Using &lt;code&gt;limma&lt;/code&gt; within Bioconductor &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/limma.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, which is a linear model package for microarray data and &lt;code&gt;lmFit&lt;/code&gt; on gene expression data, I can perform a series of regressions &lt;code&gt;gene ~ covariates&lt;/code&gt; by specifying &lt;code&gt;design = model.matrix(~ covariates)&lt;/code&gt; and calling &lt;code&gt;lmFit(eset, design)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;How can I instead run the regressions &lt;code&gt;phenotype ~ gene + covariates&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="4443" LastEditorUserId="3848" LastEditDate="2020-07-15T14:10:53.383" LastActivityDate="2020-07-15T14:10:53.383" Title="How can `limma` be used to determine genes predictive of a given phenotype?" Tags="|r|gene|gene-expression|limma|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7471" PostTypeId="1" AcceptedAnswerId="7472" CreationDate="2019-04-16T22:57:49.503" Score="1" ViewCount="51" Body="&lt;p&gt;Why exactly does genotyping not maintain phase? I guess I'm more confused on what's the difference between sequencing and genotyping, and as a result how phase is maintained in one, but not the other. This is an image trying to do phasing using genotyping methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand how phasing works, I just don't quite get what is the difference between genotyping and sequencing, and why one of them doesn't work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eIHCa.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eIHCa.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2519" LastEditorUserId="2519" LastEditDate="2019-04-17T06:08:28.183" LastActivityDate="2019-04-18T17:07:27.587" Title="Genotyping technologies do not maintain phase?" Tags="|ngs|haplotypes|genotyping|phasing|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7472" PostTypeId="2" ParentId="7471" CreationDate="2019-04-17T04:13:22.113" Score="2" Body="&lt;p&gt;Phasing means that you are able to distinguish the both copies of your target sequence. (Usually you have one from your mother and one from your father). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can do this, depends mainly on the sequencing technology you are using. For example in case of sanger sequencing, you cannot do this, because you (pre)amplify all your copies at once and detect the sequence at the same time at the same location.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case of ngs you have read information at the end. One read represents the sequence of one original dna molecule. If you detect multiple variants in one read, you can be sure that they are from the same chromosome and thus they are in phase. Now you can go on and try to find other reads that have unambiguous overlaps with these variants on your read. If you find some, you can also say that these variants have their origin on the same chromosome.  &lt;/p&gt;&#xA;" OwnerUserId="1310" LastActivityDate="2019-04-17T04:13:22.113" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7473" PostTypeId="1" CreationDate="2019-04-17T07:01:03.543" Score="2" ViewCount="443" Body="&lt;p&gt;What's the best way to test if a pair of Illumina FASTQ files from blood DNA extraction contain any reads are from viral or bacterial infection from human samples? Thanks in advance.&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&lt;p&gt;Looking at the available options, I started testing &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/gb-2014-15-3-r46&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kraken&lt;/a&gt;, the metagenomics classification (Wood and Salzberg Gen Biol 2014) for samples where I have seen an indication of viral reads. I will update with results / further questions.&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="3848" LastEditDate="2022-03-02T17:47:13.913" LastActivityDate="2022-03-02T17:47:13.913" Title="Testing for viral/bacterial infection FASTQ files" Tags="|fastq|ngs|covid-19|" AnswerCount="5" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7474" PostTypeId="1" AcceptedAnswerId="7479" CreationDate="2019-04-17T10:58:22.297" Score="2" ViewCount="307" Body="&lt;p&gt;I'm currently working with the PLINK file system to store large amounts of genotype data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plain format consists of three files, two files for storing phenotype and marker information as well as one file (.PED) containing the genotype matrix. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There exists a binary version of the .PED file (.BED), which stores the data in a number of consecutive byte blocks, with each block encoding the genotype of all samples for a specific marker. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While there exists a transposed version of the .PED file (.TPED) which is also supported by PLINK, I couldn't find out whether there also exists a transposed version of the binary .BED file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it exist at all? Are there other file formats to compactly store a genotype matrix for fast sample / marker extraction?&lt;/p&gt;&#xA;" OwnerUserId="4000" LastEditorUserId="4000" LastEditDate="2019-04-17T12:45:39.463" LastActivityDate="2019-04-17T16:33:05.700" Title="PLINK - Transposed BED file?" Tags="|gwas|plink|genotyping|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7475" PostTypeId="2" ParentId="7473" CreationDate="2019-04-17T11:20:25.523" Score="1" Body="&lt;p&gt;To know if there are bacterial genomes in the sequenced sample I have used an approach based on kmers analysis. An useful tool to do that is &lt;a href=&quot;https://bitbucket.org/genomicepidemiology/kmerfinder&quot; rel=&quot;nofollow noreferrer&quot;&gt;kmerfinder&lt;/a&gt; which uses a kmers database built from bacterial genomes. Quoting from the kmerfinder web site: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The method is used to find the best match (species identification) to&#xA;  the reads in one or more fastq files or one fasta file in a (kmer)&#xA;  database produced using the KMA program.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I haven't use it for viruses genome analysis, but conceptually it should be work as well.&#xA;Hope this would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="4444" LastEditorUserId="57" LastEditDate="2019-04-17T12:32:09.050" LastActivityDate="2019-04-17T12:32:09.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7476" PostTypeId="2" ParentId="7473" CreationDate="2019-04-17T12:13:22.630" Score="2" Body="&lt;p&gt;If you're talking RNA, I have a blog post from a while back about Diamond/MEGAN:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://achri.blogspot.com/2017/04/data-for-free-viral-metagenomics-in.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://achri.blogspot.com/2017/04/data-for-free-viral-metagenomics-in.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In principle this will work for DNA if you have a non-trivial number of pathogen reads (since they tend to be gene dense in their genomes). Otherwise I use Kraken on a database for all the NCBI nt database, but this is slower and doesn't let you sanity check the false positive taxonomy assignments as well as I describe in the post. Of course, ideally, you run multiple methods if you have the resources to do so.&lt;/p&gt;&#xA;" OwnerUserId="4445" LastEditorUserId="4445" LastEditDate="2019-04-17T22:12:57.830" LastActivityDate="2019-04-17T22:12:57.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7478" PostTypeId="1" AcceptedAnswerId="7481" CreationDate="2019-04-17T16:10:06.373" Score="2" ViewCount="105" Body="&lt;p&gt;Let's say I'm sequencing a donor, and I have 20X coverage. And let's say all the reads covering a particular base has the same allele at the position. Then, is that position considered a &quot;variant&quot; with only 1 allele in that position or is that not even considered a variant?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I'm asking is, I was given some read data, and it's in the form location:variant - location:variant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there was often the case where for all the variants at a particular location, there was only one allele. But this data is labelled to only have variants. Hence, how is a particular location deemed a &quot;variant.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, lets say I'm told that the following is &lt;strong&gt;&quot;variant reads given in the form read# location:variant - location:variant&quot;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;read 1 6523432: A - 6710432 T&#xA;read 2 6523432: C - 6710432 G&#xA;read 3 6523432: A - 6710432 T&#xA;read 4 6523432: C - 6710432 G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, why are these two locations considered variants? It seems like it's just a heterozygous locations. Why were these locations considered to carry variants? Similarly, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;read 1 6523432: A - 6710432 T&#xA;read 2 6523432: A - 6710432 T&#xA;read 3 6523432: A - 6710432 T&#xA;read 4 6523432: A - 6710432 T&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In those reads, it just seems like it's homozygous. Why is it being called a variant? How was it classified as such?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Put another way, if I'm given 100000 reads, and I said I only want to look at &lt;strong&gt;informative reads&lt;/strong&gt;, and so I'm given a subset with only 100 reads, and those 100 reads look something like the above, then how were these reads selected?&lt;/p&gt;&#xA;" OwnerUserId="2519" LastEditorUserId="2519" LastEditDate="2019-04-17T17:45:13.437" LastActivityDate="2019-04-17T18:42:27.563" Title="Knowing which positions are variants" Tags="|variant-calling|terminology|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7479" PostTypeId="2" ParentId="7474" CreationDate="2019-04-17T16:25:53.640" Score="2" Body="&lt;p&gt;Yes, this exists, and can be efficiently generated by plink 2.0's &lt;code&gt;--export ind-major-bed&lt;/code&gt; command.  (The third byte is 0 instead of 1, and the specification is otherwise identical to that of regular plink .bed files with samples and variants swapped.)&lt;/p&gt;&#xA;" OwnerUserId="472" LastEditorUserId="298" LastEditDate="2019-04-17T16:33:05.700" LastActivityDate="2019-04-17T16:33:05.700" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7480" PostTypeId="2" ParentId="7453" CreationDate="2019-04-17T17:27:01.810" Score="1" Body="&lt;p&gt;I've got it. &#xA;The question you are asking is homogeneity between you target group and your general population of patients. What you want to show is &quot;complete&quot; homogeneity between the two groups regardless of any sub-sampling, ie. if there is groups within groups that might break your hypothesis.&#xA;K-means is perfect here. The issue with K-means is designating the number of groups, but here thats irrelevant because its not the question you are asking. You are asking homogeneity and if your hypothesis is true no matter how many groups you designate for K-means, e.g. 2,3,4,5,6,7,8,9,10 (or maybe a more reduced sampling) homogeneity between target (treatment?) and general population is observed. This would therefore be a definitive analysing ASSUMING that you don't encounter a group where there is exclusively one population and the other is absent. Obviously the number of K-mean groups can't exceed the number of treated/target patients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be a really good application of k-means because it avoids the question, &#xA;So why was that a priori selected? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally I would first present your preferred a priori group size and then say same result for all group sizes. Its simple, recognised and neat and easy for a reviewer to follow. There are ML solutions to K-means, but I think &quot;keep it simple&quot;.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-04-17T17:27:01.810" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7481" PostTypeId="2" ParentId="7478" CreationDate="2019-04-17T18:42:27.563" Score="3" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Set 1&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;read 1 6523432: A - 6710432 T&#xA;read 2 6523432: C - 6710432 G&#xA;read 3 6523432: A - 6710432 T&#xA;read 4 6523432: C - 6710432 G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I understand this as showing the location of 2 variants, supported by 2 reads each. Now, if the reference has an A at position 6523432, your sample could be heterozygous for an A-&gt;C variant at position 6523432. If the reference has a C at that position, then your sample could be heterozygous for C-&gt;A. If the reference has something else again, for example G, then your sample could be heterozygous for G-&gt;A/C. Similarly for location 6710432.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Set 2&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;read 1 6523432: A - 6710432 T&#xA;read 2 6523432: A - 6710432 T&#xA;read 3 6523432: A - 6710432 T&#xA;read 4 6523432: A - 6710432 T&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this case, if the reference has an A at position 6523432, the reads do not support a variant. However, if the reference has anything else, then the reads would support a homozygous variant e.g. G-&gt;A. And the same for position 6710432.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Now, in all of these cases, you cannot know whether a variant is real without taking into account the number of reads that support it, the quality of the alignment, and the quality of the reads. That's why we do this using dedicated variant callers, software that is designed to take all these factors into account and decide whether to call a variant at that position. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in all cases, the variant must be defined with respect to a reference genome. You cannot call a variant just by looking at reads without any knowledge of a reference. A &quot;variant&quot; just means &quot;not what is found in the reference&quot;. It is largely a matter of random chance whether something is a &quot;variant&quot; or &quot;normal&quot;. If the individual(s) whose genome was taken as a reference has the same sequence, then you call it normal. If not, then you call it a variant. There are many cases where the standard hg19 or hg38 genomes have a &quot;normal&quot; residue which is actually far less common than the known &quot;variant&quot;. There's nothing special about the reference genome, after all, it's just the one we chose to call a reference.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-04-17T18:42:27.563" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7482" PostTypeId="1" CreationDate="2019-04-17T18:45:00.707" Score="3" ViewCount="974" Body="&lt;p&gt;I recently finished mapping an RNAseq run using STAR2.3.0 and noticed the read1 and read2 counts are different, according to samtools flagstat. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The map% is ~80% but the R1 R2 counts are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;R1=139283692&lt;/p&gt;&#xA;&#xA;&lt;p&gt;R2=137472495&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This usually does not happen for PE sequencing, which makes me a bit nervous to proceed with analysis. What are some potential reasons something like this could occur? I do not have experience with this happening and can't seem to find reasons by searching the web.&lt;/p&gt;&#xA;" OwnerUserId="3669" LastActivityDate="2019-04-17T18:45:00.707" Title="What could cause differing counts of R1 and R2 in Paired End Sequencing (RNASEQ)" Tags="|rna-seq|illumina|star|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7483" PostTypeId="1" AcceptedAnswerId="7484" CreationDate="2019-04-18T09:13:05.500" Score="3" ViewCount="52" Body="&lt;p&gt;I would like to find proteins with exactly one SS bond. Is there a database where I can search this? I've tried advanced search on &lt;a href=&quot;https://www.rcsb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/&lt;/a&gt;, but no such option, at least I could not find it.&lt;/p&gt;&#xA;" OwnerUserId="4449" LastEditorUserId="298" LastEditDate="2019-04-18T09:52:43.813" LastActivityDate="2019-04-18T09:52:43.813" Title="Proteins with one SS bond?" Tags="|public-databases|protein-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7484" PostTypeId="2" ParentId="7483" CreationDate="2019-04-18T09:36:38.087" Score="3" Body="&lt;p&gt;In cases when you cannot find answer through the web interface, you can &lt;a href=&quot;http://www.wwpdb.org/download/downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;download all the data&lt;/a&gt; and search it locally.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Relying on the PDB annotations of SS bonds, you could search for PDB files that have exactly one SSBOND record:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ cd $&lt;/span&gt;PDB_DIR/structures/divided/pdb/&#xA;&lt;span class=&quot;math-container&quot;&gt;$ zgrep -c ^SSBOND aa/pdb*.ent.gz | grep :1$&lt;/span&gt;&#xA;aa/pdb2aaq.ent.gz:1&#xA;aa/pdb3aad.ent.gz:1&#xA;aa/pdb5aap.ent.gz:1&#xA;aa/pdb6aa4.ent.gz:1&#xA;aa/pdb6aa5.ent.gz:1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;All PDB files (gzipped) take about 30GB of disk space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The caveat is that some entries are available only in the mmCIF format. Using mmCIF would be slighty more complex: you would need to count &lt;code&gt;_struct_conn.conn_type_id&lt;/code&gt; being &lt;code&gt;disulf&lt;/code&gt;. And you would need to download 10GB more.&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2019-04-18T09:36:38.087" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7485" PostTypeId="2" ParentId="2817" CreationDate="2019-04-18T13:06:15.737" Score="1" Body="&lt;p&gt;Here is my free program on Github  &lt;strong&gt;Sequence database curator&lt;/strong&gt;&#xA;(&lt;a href=&quot;https://github.com/Eslam-Samir-Ragab/Sequence-database-curator&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Eslam-Samir-Ragab/Sequence-database-curator&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a very fast program and it can deal with:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Nucleotide sequences&lt;/li&gt;&#xA;&lt;li&gt;Protein sequences&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It can work under Operating systems:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Windows&lt;/li&gt;&#xA;&lt;li&gt;Mac&lt;/li&gt;&#xA;&lt;li&gt;Linux&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;It also works for:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Fasta format&lt;/li&gt;&#xA;&lt;li&gt;Fastq format&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Best Regards&lt;/p&gt;&#xA;" OwnerUserId="4450" LastActivityDate="2019-04-18T13:06:15.737" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="7486" PostTypeId="2" ParentId="7473" CreationDate="2019-04-18T15:16:58.647" Score="0" Body="&lt;p&gt;What type of NGS data is it? If smallRNA or RNAseq data, you can simply assemble the reads into longer contigs and do the blast search using your reference viral or bacterial sequences. &lt;/p&gt;&#xA;" OwnerUserId="2425" LastActivityDate="2019-04-18T15:16:58.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7487" PostTypeId="2" ParentId="7471" CreationDate="2019-04-18T16:59:33.883" Score="0" Body="&lt;p&gt;Genotyping (there are of course many different methods) is likely to give you information in the format of &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Locus 1: 100% A&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Locus 2: 50% C, 50% T&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Locus 3: 50% C, 50% A&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's not phased.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Short or long read technology will give you data in the form of reads, each of which originates from one molecule, and if a read covers two loci, you'll know how they are phased.  You will know if one DNA molecule has ACC, or ACA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sanger sequencing will give you mixed peaks at the locus of a mixed SNP, so there will be no phasing data.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-04-18T16:59:33.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7488" PostTypeId="2" ParentId="7471" CreationDate="2019-04-18T17:07:27.587" Score="1" Body="&lt;p&gt;Many (small) variant calling algorithms use pileups to genotype variants position-by-position. This approach alone is insufficient to phase adjacent variants. However, if the variants of interest are close enough to be covered by a single read, it's certainly possible to phase them even with &amp;lt;100bp Illumina reads. In fact, I was implementing just such an approach recently when I asked &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/7401/96&quot;&gt;this question about accessing bases by &lt;em&gt;aligned&lt;/em&gt; position rather than by read position&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-04-18T17:07:27.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7489" PostTypeId="2" ParentId="7473" CreationDate="2019-04-18T19:25:21.477" Score="2" Body="&lt;p&gt;I think you could try &lt;a href=&quot;https://microbiomejournal.biomedcentral.com/articles/10.1186/2049-2618-2-33&quot; rel=&quot;nofollow noreferrer&quot;&gt;pathoscope&lt;/a&gt; for this.&lt;/p&gt;&#xA;" OwnerUserId="829" LastActivityDate="2019-04-18T19:25:21.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7490" PostTypeId="2" ParentId="156" CreationDate="2019-04-19T00:08:29.877" Score="14" Body="&lt;p&gt;To expand on the answer above, in case it isn't clear, we show:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why palindromic sequences must be even in length&lt;/li&gt;&#xA;&lt;li&gt;Why palindromic sequences induce self-loops in a de Bruijn graph&lt;/li&gt;&#xA;&lt;li&gt;Why self loops in a de Bruijn graph are problematic&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h2&gt;1. Palindromic sequence ⇒ sequence is of even length&lt;/h2&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Idea:&lt;/strong&gt; in an odd-length k-mer, its middle nucleotide is 'flipped' in its reverse complement, so the two can never be equal.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Suppose you have a palindromic sequence &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt;. Then &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is identical to its reverse complement, which we will label &lt;span class=&quot;math-container&quot;&gt;$\bar{X}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Suppose &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is of odd length. Then it is of the form &lt;span class=&quot;math-container&quot;&gt;$AbC$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$len(A) = len(C) = \frac{len(X)-1}{2}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$len(b) = 1$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Then&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$X = \bar{X} \implies AbC = \overline{AbC} = \bar{C}\bar{b}\bar{A}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;And hence:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$b = \bar{b}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;(since &lt;span class=&quot;math-container&quot;&gt;$len(A) = len(C) = len(\bar{C}))$&lt;/span&gt;. But this is a contradiction, since &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; is a single nucleotide, and cannot be equal to its complement. Therefore k-mers of odd length cannot form palindromes.&lt;/p&gt;&#xA;&lt;p&gt;Hence the length of a k-mer that forms a palindrome must be even.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h2&gt;2. Why palindromic k-mers induce self loops&lt;/h2&gt;&#xA;&lt;p&gt;Each node in a traditional de Bruijn graph is a unique string, but in most bio-informatics implementations each pair of reverse-complementary k-1-mers are identified as a single node, e.g for &lt;span class=&quot;math-container&quot;&gt;$k=6$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://pmelsted.wordpress.com/2014/01/17/edge-cases-in-de-bruijn-graphs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Hbakm.png&quot; alt=&quot;&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;A palindromic k-mer (of &lt;span class=&quot;math-container&quot;&gt;$k \geq 2$&lt;/span&gt;) is of the form:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$xAy$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$len(A) = k-2$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$x=\bar{y}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$A=\bar{A}$&lt;/span&gt; (possibly the empty string).&lt;/p&gt;&#xA;&lt;p&gt;Hence it will contribute two nodes in the de Bruijn graph:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;its left k-1-mer &lt;span class=&quot;math-container&quot;&gt;$xA$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;its right k-1-mer &lt;span class=&quot;math-container&quot;&gt;$Ay$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;And an edge going from 1 to 2.&lt;/p&gt;&#xA;&lt;p&gt;But since this k-mer is palindromic, &lt;span class=&quot;math-container&quot;&gt;$xA = \overline{Ay}$&lt;/span&gt; and hence these two nodes are reverse-complementary, and thus the 'same' node, and so this edge is a self loop at this node.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h2&gt;3. Why are self loops problematic?&lt;/h2&gt;&#xA;&lt;p&gt;Self-loops (if they occur in a node with &lt;span class=&quot;math-container&quot;&gt;$in\_degree \geq 2$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$out\_degree \geq 1$&lt;/span&gt;) increase the number of possible Eulerian paths in a de Bruijn graph (or more specifically, in the connected component containing this node, which represents a &lt;em&gt;contig&lt;/em&gt;, of which their may be multiple), as you have an additional possible Eulerian path for each time you traverse this node.&lt;/p&gt;&#xA;&lt;p&gt;This increases ambiguity in reading the graph, as each possible Eulerian path is an extra possible reconstruction of the full sequence.&lt;/p&gt;&#xA;&lt;p&gt;Consider the example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jZSpf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jZSpf.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There is only one possible Eulerian path:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$A B C D B E$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, if we include a self loop at &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, which is visited twice above, this doubles to two possible Eulerian paths:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0tkDw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0tkDw.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$A B B C D B E$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$A B C D B B E$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Depending on whether we traverse the self loop during the first time we reach &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;, or the second.&lt;/p&gt;&#xA;" OwnerUserId="4327" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2020-04-24T12:03:22.037" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7493" PostTypeId="1" CreationDate="2019-04-19T13:52:47.923" Score="4" ViewCount="3161" Body="&lt;p&gt;How can I detect a split read in a BAM file?&#xA;Is there any sign in the CIGAR string that describes split read? &lt;/p&gt;&#xA;" OwnerUserId="4456" LastEditorUserId="29" LastEditDate="2019-05-01T18:37:05.513" LastActivityDate="2020-05-25T21:01:10.147" Title="How do I find split reads?" Tags="|sequence-alignment|sam|structural-variation|cigar|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="7494" PostTypeId="1" CreationDate="2019-04-19T14:29:14.200" Score="1" ViewCount="39" Body="&lt;p&gt;I am trying to build a database of different cancer genes i.e lung cancer, throat cancer etc. Searched through NCBI website by searching for nucleotide sequences, but I am unable to find appropriate results. Any help in this regard is much appreciated&lt;/p&gt;&#xA;" OwnerUserId="4457" LastActivityDate="2019-04-19T14:29:14.200" Title="Building cancer gene database" Tags="|gene|sequence-analysis|cancer|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="7495" PostTypeId="1" CreationDate="2019-04-21T18:53:46.087" Score="2" ViewCount="145" Body="&lt;p&gt;I'm a mathematician studying knotted curves and am trying to find some applications of my work in proteins. Using &lt;a href=&quot;https://www.rcsb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/&lt;/a&gt;, how do I find proteins that have similar  (but perhaps not the same) tertiary structure? Preferably they would have similar primary structure and function... &#xA;Perhaps what I'm looking for is &quot;similar proteins&quot;, if there is such a thing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, is there a way in the database to select more &quot;common proteins&quot; (important or well studied, not just some rare exotic one (if there is such a thing)?&lt;/p&gt;&#xA;" OwnerUserId="4449" LastActivityDate="2019-04-22T00:19:54.763" Title="Find similar proteins with similar tertiary structure?" Tags="|database|protein-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7496" PostTypeId="2" ParentId="7495" CreationDate="2019-04-21T22:54:20.127" Score="1" Body="&lt;p&gt;As a general question on the board, more detail is often good, because of the answers can be inaccurate due to misunderstanding of the biological context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several ways to do this and at least 3 that I'm aware of. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Blast - sequence similarity searching the PDB database&lt;/li&gt;&#xA;&lt;li&gt;Contact map identity, I use this approach, but its complicated - and its precision or utility is context specific, i.e. it could be loads of work and miss the vital detail you need.&lt;/li&gt;&#xA;&lt;li&gt;Direct mapping one protein structure against another, this in particular is true for specific domains, or sites within the protein and the result is angstroms. Its not difficult and I assume this can be scaled to the whole protein.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Point 1 &quot;Blackboxes&quot;&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By far the easiest is option one and I suspect I-Tasser (&lt;a href=&quot;https://zhanglab.ccmb.med.umich.edu/I-TASSER/download/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) use this principle before instigating &quot;threading&quot;. This package is considered &quot;good&quot; (others appear to consider it the best of its type), I believe it has done well in CASP tests (Critical Assessment of Techniques for Protein Structure Prediction). So the simple answer is dump your sequence in there an watch the output. Pyre is something similar. These are interesting tools but certainly not definitive, really there a starting point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Point 1 &quot;Formal Blasting PDB&quot;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the easiest and most mature approach for your question, without knowing the details. I've run a test to ask the following question.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Which protein structure available on PBD is most similar to Japanese encephalitis virus (JEV) based on protein homology?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step A&lt;/strong&gt;&#xA;I've no specific interest in the virus BTW. The most robust Blast engine IMO is NCBI &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; . I dump a JEV sequence into the box, and click three important options (otherwise it doesn't work). &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Database = protein databank (PDB), i.e. structural data&lt;/li&gt;&#xA;&lt;li&gt;Organism &quot;&lt;em&gt;Japanese encephalitis virus (taxid:11072)&lt;/em&gt;&quot;: the taxid will come up automatically (if not just proceed as normal)&lt;/li&gt;&#xA;&lt;li&gt;Next to the Organism box I click &quot;&lt;em&gt;Exclude&lt;/em&gt;&quot;, otherwise this whole approach doesn't work&lt;/li&gt;&#xA;&lt;li&gt;Obviously I select &quot;&lt;em&gt;blastp&lt;/em&gt;&quot;, otherwise it definitely will not work&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The output is/was &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi#alnHdr_1545825802&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (but is now inactive). Summarising what it presented, E is so high its &quot;0&quot; (&gt;2e-139) protein ID &quot;77.59%&quot; and just states &quot;envelope protein&quot; Chain A. I hit the link and the protein that comes up has a name which ends in &quot;&lt;strong&gt;6A0P_A&lt;/strong&gt;&quot;. This is the PDB reference and Percentage ID and PDB reference are the only two bits of info you need. The PDB reference needs the &quot;_A&quot; removing, thus its 6A0P. The _A just refers to chain A of a dimer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Step B&lt;/strong&gt;&#xA;You then go back to the Web site you mentioned and input the PDB ID you stated into the search bar. This results in this quite interesting page &lt;a href=&quot;https://www.rcsb.org/structure/6A0P&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The structure is Usutu virus, which surprised me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Point 2&lt;/strong&gt; . This is much harder and not only involves specific Python code, but a post unsupervised learning analysis and results in an average similarity of structure over the whole protein. There are a number of published algorithms for generating a contact map, or its easy to write your own. Its more advanced, its complicated and unless you've seen loads of them they not immediately understandable. Contact maps angle at a very specialist approach of predicting and understanding protein structure. It is not for the faint hearted and some biological details will be omitted, which may or may not be important for your purpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Point 3&lt;/strong&gt; . I don't have research grade expertise here, so I forget the exact button to click, but its quite easy via PyMol and other graphical viewers to perform spatial similarity for specific epitopes and this is something you need to learn within your sphere. Basically you will load two proteins into the image viewer and then they will be a &quot;similarity aligning&quot; function and you zoom in on your output to generate an answer in angstroms. You will need to learn this approach for the later parts of your project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;, The orthodox way would be to perform point 1, select several targets and then examine them in detail via point 3. I definitely recommend direct Blast if you go down this route, because it is the most tractable and avoids potential unknown subsampling.  I personally would not recommend &lt;em&gt;Point 2&lt;/em&gt; in a general context because you have to consider an unsupervised learning approach towards processing a heap of contact maps. Ultimately, which ever approach you take you can never avoid Point 3 (which someone else does, in my case).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; This may or may not be obvious to your but just because two sequence have the closest homology doesn't mean they will have the closest structural similarity.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-04-22T00:19:54.763" LastActivityDate="2019-04-22T00:19:54.763" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7497" PostTypeId="1" CreationDate="2019-04-22T05:31:08.900" Score="1" ViewCount="30" Body="&lt;p&gt;I am using &lt;a href=&quot;https://deeptools.readthedocs.io&quot; rel=&quot;nofollow noreferrer&quot;&gt;deeptools&lt;/a&gt; to generate a &lt;a href=&quot;https://deeptools.readthedocs.io/en/develop/content/feature/metagene.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;metagene&lt;/a&gt; plot. I am wondering how the deeptools scale genes if the size of genes is shorter than bin size. For example, if my bin size is 1000 bp and the gene is about 450bp, how does the deeptools computeMatrix scale it to make a metagene plot of 1000 bp?&lt;/p&gt;&#xA;" OwnerUserId="4460" LastEditorUserId="57" LastEditDate="2019-04-22T09:45:24.373" LastActivityDate="2019-04-22T09:45:24.373" Title="scaling algorithm of deeptools" Tags="|deeptools|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7498" PostTypeId="1" AcceptedAnswerId="8570" CreationDate="2019-04-22T09:44:32.910" Score="2" ViewCount="107" Body="&lt;p&gt;I am using &lt;a href=&quot;https://github.com/dellytools/delly/&quot; rel=&quot;nofollow noreferrer&quot;&gt;delly&lt;/a&gt; to call structural variants on a human WGS sample. I noticed that it finds certain variants that are essentially duplicates. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr4    177594033       DEL00005339     G       &amp;lt;DEL&amp;gt;   .       PASS    PRECISE;SVTYPE=DEL;SVMETHOD=EMBL.DELLYv0.8.1;CHR2=chr4;END=177594095;PE=0;MAPQ=0;CT=3to5;CIPOS=-3,3;CIEND=-3,3;SRMAPQ=60;INSLEN=0;HOMLEN=3;SR=5;SRQ=0.993243;CONSENSUS=GACAAAAACATATAAACAAATACAAATTATTATTATATGAAGGGTGAAAAATTATGTTCAGCTTTAATATTATGTGGCCTGAAACAAATTTGTATAAGTCATATGATCCGTTTTGTATATTCATTAAAATAGTCATAAAAATATAAAC;CE=1.75466     GT:GL:GQ:FT:RCL:RC:RCR:CN:DR:DV:RR:RV   0/1:-51.3797,0,-42.791:10000:PASS:1115:1821:1820:1:0:0:23:27&#xA;chr4    177594033       DEL00005340     G       &amp;lt;DEL&amp;gt;   .       PASS    PRECISE;SVTYPE=DEL;SVMETHOD=EMBL.DELLYv0.8.1;CHR2=chr4;END=177594095;PE=0;MAPQ=0;CT=3to5;CIPOS=-3,3;CIEND=-3,3;SRMAPQ=60;INSLEN=0;HOMLEN=3;SR=12;SRQ=0.994186;CONSENSUS=ATATAAACAAATACAAATTATTATTATATGAAGGGTGAAAAATTATGTTCAGCTTTAATATTATGTGGCCTGAAACAAATTTGTATAAGTCATATGATCCGTTTTGTATATTCATTAAAATAGTCATAAAAATATAAACAATGTAATAAGTATGTTTGAATAGTATTATGAT;CE=1.72637    GT:GL:GQ:FT:RCL:RC:RCR:CN:DR:DV:RR:RV   0/1:-51.3797,0,-42.791:10000:PASS:1115:1821:1820:1:0:0:23:27&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The two variants above have the same start position and the same end position (&lt;code&gt;END=177594095&lt;/code&gt;). However, they differ in the &lt;code&gt;SR&lt;/code&gt;, &lt;code&gt;SRQ&lt;/code&gt; and &lt;code&gt;CONSENSUS&lt;/code&gt; fields. These are defined in the VCF header as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##INFO=&amp;lt;ID=SR,Number=1,Type=Integer,Description=&quot;Split-read support&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=SRQ,Number=1,Type=Float,Description=&quot;Split-read consensus alignment quality&quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=CONSENSUS,Number=1,Type=String,Description=&quot;Split-read consensus sequence&quot;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Of course, the consensus sequences are quite similar, but they aren't identical (I have simply added some spaces to the beginning of the 2nd sequence here to align them):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GACAAAAACATATAAACAAATACAAATTATTATTATATGAAGGGTGAAAAATTATGTTCAGCTTTAATATTATGTGGCCTGAAACAAATTTGTATAAGTCATATGATCCGTTTTGTATATTCATTAAAATAGTCATAAAAATATAAAC&#xA;         ATATAAACAAATACAAATTATTATTATATGAAGGGTGAAAAATTATGTTCAGCTTTAATATTATGTGGCCTGAAACAAATTTGTATAAGTCATATGATCCGTTTTGTATATTCATTAAAATAGTCATAAAAATATAAACAATGTAATAAGTATGTTTGAATAGTATTATGAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So what am I looking at here, exactly? If this is a deletion on chromosome 4 from position 177594033 to position 177594095, then how can the length of the consensus sequence be different? This is the &quot;split read consensus&quot;, so presumably the sequence that appears to have been deleted based on how the reads covering that region are split. This means that these reads align to the genome with a gap and this gap indicates the deletion (&lt;a href=&quot;https://www.researchgate.net/figure/Five-approaches-to-detect-CNVs-from-NGS-short-reads-A-Paired-end-mapping-PEM-strategy_fig1_260376412&quot; rel=&quot;nofollow noreferrer&quot;&gt;image source&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kodIq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kodIq.png&quot; alt=&quot;split read example&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, if the start and end position are the same, the consensus should also be identical. And, in any case, if the start and end positions are the same, then this should be the same structural variant, so why is it being reported twice? How can I interpret this result?&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-05-02T13:00:02.367" Title="Delly finds identical deletions with different consensus sequences" Tags="|structural-variation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7499" PostTypeId="1" AcceptedAnswerId="8580" CreationDate="2019-04-22T12:20:27.657" Score="1" ViewCount="509" Body="&lt;p&gt;Long time I am struggling how to provide &lt;strong&gt;&lt;em&gt;GISTIC2&lt;/em&gt;&lt;/strong&gt; required input like &lt;code&gt;Num_Probes&lt;/code&gt; and &lt;code&gt;Segment_Mean&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have called somatic copy number by scatngs and I have copy number output file (attached); &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/r6nef2v1j2di6q6/my.copynumber.caveman.csv?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/r6nef2v1j2di6q6/my.copynumber.caveman.csv?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is few lines of scatngs copy number output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1,1,13116,665401,2,1,3,1&#xA;2,1,668374,96324627,2,1,2,1&#xA;3,1,96326095,96419709,2,1,1,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In scatngs reference paper says in this file we have&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Segment number&#xA;&#xA;Chromosome&#xA;&#xA;Start position (origin-1)&#xA;&#xA;End position (origin-1)&#xA;&#xA;Major copy number—normal&#xA;&#xA;Minor copy number—normal&#xA;&#xA;Major copy number—tumor&#xA;&#xA;Minor copy number—tumor&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;scatngs also returns another output like below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/7i28vooixdp06c2/WTSI-OESO_121_1pre.copynumber.txt?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/7i28vooixdp06c2/WTSI-OESO_121_1pre.copynumber.txt?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Chromosome  Position    Log R   segmented LogR  BAF segmented BAF   Copy number Minor allele    Raw copy number&#xA;rs62635286  1   13116   -0.0290741822638953 0.245246748556562   0.25    NA  3   1   2.35806652809599&#xA;rs75454623  1   14930   0.402869048014209   0.245246748556562   0.3095  NA  3   1   4.14031506947036&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am wondering how I can get &lt;code&gt;Num_Probes&lt;/code&gt; and &lt;code&gt;Segment_Mean&lt;/code&gt; From this files?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;People say&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Segment_Mean = log2(tumour copy number / 2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, does that mean I should add up &lt;code&gt;Major copy number—tumor&lt;/code&gt; and &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Minor copy number—tumor&lt;/code&gt; to get tumour copy number? for getting Num_Probes what should I do?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;EDITED&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;people says&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;scatngs uses read counts at SNPs to generate calls and so the&#xA;  locations of the SNPs is what I need to&#xA;  generate the number of markers in each segment. (eg each SNP = 1&#xA;  marker).&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Do they mean, for instance if the position of SNP  rs62635286 is&#xA;  13116, so the num_prob is 13116?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Any help please?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot in advance &lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-04-23T09:46:01.317" LastActivityDate="2019-05-03T13:03:04.327" Title="Getting this information for GISTIC2" Tags="|wgs|copy-number|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="7500" PostTypeId="1" CreationDate="2019-04-22T23:25:07.243" Score="0" ViewCount="1386" Body="&lt;p&gt;I have a set of SNPs (single nucleotide polymorphisms) { S&lt;sub&gt;1&lt;/sub&gt;, S&lt;sub&gt;2&lt;/sub&gt;, ..., S&lt;sub&gt;N&lt;/sub&gt; } from approximately 200 humans. I wish to determine the linkage disequilibrium (LD) for a defined subset of these SNPs. By this I mean e.g. I would like to compute e.g. LD between S1 and S2, S3, S4, but not the LD between S2 and S3. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could this calculation be performed in PLINK?&lt;/p&gt;&#xA;" OwnerUserId="255" LastEditorUserId="96" LastEditDate="2019-08-27T16:31:32.850" LastActivityDate="2019-09-26T17:02:05.243" Title="How to compute LD for pairs of variants with Plink" Tags="|vcf|snp|variation|plink|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7501" PostTypeId="2" ParentId="7500" CreationDate="2019-04-23T00:58:08.223" Score="0" Body="&lt;p&gt;Anyway, it would be fairly easy via the filtering options &lt;a href=&quot;https://www.cog-genomics.org/plink/2.0/filter#snps_only&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;--exclude-snp&lt;/code&gt;  is what you are wanting.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I have worked in LD and subsampling a genome is open to forcing a result. The alternative approach is the subsample is the population, or examine disequilibrium with respect to the physical chromosome. A local shift in disequilibrium  or heterozygosity  (sliding window along a chromosome) is an indication of selective pressure. This approach is frequently used in eukaryotic parasites to identify drug resistance. I don't have experience of human genetics but there are certainly enough here who do. However, I think you need to define the biological rationale for the subsample. Just having loads of data is easily resolved because any algorithm can be written to handle automation.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2019-04-23T19:42:28.323" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7502" PostTypeId="1" CreationDate="2019-04-23T07:28:53.593" Score="1" ViewCount="317" Body="&lt;p&gt;I have two results from two separate GSEA runs (2 different inputs / samples with same gene sets), and i want to know which sample has the most enrichment about specific pathway (gene set).&lt;/p&gt;&#xA;&lt;p&gt;I want to compare the ES from the two different runs, looking at the same gene set. Can I use ES (or NES) from two different GSEA analysis about same gene set?&lt;/p&gt;&#xA;&lt;p&gt;or should I combine the input dataset into one, and run the analysis again?&lt;/p&gt;&#xA;" OwnerUserId="4466" LastEditorUserId="73" LastEditDate="2022-06-21T10:01:17.607" LastActivityDate="2022-06-21T10:01:17.607" Title="Question about enrichment score (ES) and NES from GSEA" Tags="|gene-expression|pathway|gsea|go-enrichment|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="7504" PostTypeId="1" AcceptedAnswerId="10593" CreationDate="2019-04-23T14:14:48.140" Score="1" ViewCount="92" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the purpose of correlating docking score with catalytic efficiency (kcat/km)? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I saw some papers are doing this, is it to validate the docking protocol? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Lets say the docking protocol is correct, the docking score will positively correlate with kcat/km?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3760" LastActivityDate="2019-10-15T13:30:32.017" Title="Purpose of correlating docking score with catalytic efficiency(Kcat/km)" Tags="|docking|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="7505" PostTypeId="1" CreationDate="2019-04-23T16:20:35.857" Score="7" ViewCount="728" Body="&lt;p&gt;I am running some analysis on an RNA-seq dataset. I have a list of transcripts that are potential lncRNA for which I ran both Kallisto and Salmon aligners. The input data for index building and quantification includes mRNA as well as these potential lncRNA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Commands for one replicate: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Salmon&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;salmon index --keepDuplicates -t transcripts.fasta -i index_directory/ -k 31&#xA;salmon quant --seqBias --gcBias -i index_directory/ --libType ISR --auxDir index_directory/ -1 sample_R1.fastq.gz -2 sample_R2.fastq.gz -o output_name&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Kallisto&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;kallisto index -i kallisto_index.idx transcripts.fasta&#xA;kallisto quant -i kallisto_index.idx -o output_name --rf-stranded sample_R1.fastq.gz sample_R2.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I used the tpm values, averaged replicates (3) and plotted the number of transcripts per condition that had a tpm &gt; 2. Same result for tpm &gt; 5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following graph shows the for mRNA:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dGJhe.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dGJhe.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And for lncRNA:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZC3CA.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZC3CA.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, the analysis were done with both lncRNA and mRNA transcripts together in the same fasta file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The overlap is not shown here but is always smaller than the count for Kallisto, for the lncRNA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In literature both tools seem to have similar results. I cannot find a reason for this. If there would be an error somewhere I would expect to see also differences in the conding RNAs which does not happen.&lt;/p&gt;&#xA;" OwnerUserId="4469" LastEditorUserId="96" LastEditDate="2019-04-25T13:45:16.480" LastActivityDate="2019-05-14T22:14:15.647" Title="Why are my kallisto and salmon results differing so much just for lncRNA transcripts?" Tags="|rna-seq|salmon|kallisto|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="7507" PostTypeId="2" ParentId="7473" CreationDate="2019-04-23T21:28:12.607" Score="1" Body="&lt;p&gt;You may find this blog post helpful: &lt;a href=&quot;https://www.sevenbridges.com/centrifuge/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Identifying viral sequences in TCGA data using Kraken and Centrifuge&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Our goal was to evaluate the performance of Centrifuge and Kraken for&#xA;  detecting viral transcripts across a cancer cohort. Centrifuge and&#xA;  Kraken produce very similar results and consistently detect HPV&#xA;  strains in cervical cancer.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is also a review &lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fmicb.2018.00749/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;Overview of Virus Metagenomic Classification Methods and Their Biological Applications&lt;/a&gt;, but they focus specifically on viruses:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We evaluated 49 published computational classification workflows for&#xA;  virus metagenomics in a literature review. To this end, we described&#xA;  the methods of existing workflows by breaking them up into five&#xA;  general steps and assessed their ease-of-use and validation&#xA;  experiments. Performance scores of previous benchmarks were summarized&#xA;  and correlations between methods and performance were investigated. We&#xA;  indicate the potential suitability of the different workflows for (1)&#xA;  time-constrained diagnostics, (2) surveillance and outbreak source&#xA;  tracing, (3) detection of remote homologies (discovery), and (4)&#xA;  biodiversity studies.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="35" LastEditorUserId="35" LastEditDate="2019-04-23T21:48:38.787" LastActivityDate="2019-04-23T21:48:38.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="7508" PostTypeId="1" CreationDate="2019-04-24T00:19:40.397" Score="3" ViewCount="874" Body="&lt;p&gt;I have 2 .csv files that contains a list of accession codes. For example for this &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/biosample?LinkName=bioproject_biosample_all&amp;amp;from_uid=520914&quot; rel=&quot;nofollow noreferrer&quot;&gt;experiment&lt;/a&gt; a .csv file will contain:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SAMN10870244&#xA;SAMN10870243&#xA;SAMN10870242&#xA;SAMN10870241&#xA;SAMN10870240&#xA;SAMN10870239&#xA;SAMN10870238&#xA;SAMN10870237&#xA;SAMN10870236&#xA;SAMN10870235&#xA;SAMN10870234&#xA;SAMN10870233&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I run the following &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in &lt;span class=&quot;math-container&quot;&gt;$(cat *.csv); do echo $&lt;/span&gt;{i}; /usr/bin/sratoolkit/bin/fastq-dump --gzip ${i} &amp;amp; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Some of my downloads takes a lot of time days. Is there a faster way to get stuff from NCBI?&lt;/li&gt;&#xA;&lt;li&gt;Are there mirrors of NCBI to accelerate the process?&lt;/li&gt;&#xA;&lt;li&gt;Why is it so slow? &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2019-04-24T00:26:44.260" LastActivityDate="2019-05-01T14:34:27.090" Title="How do you download stuff from NCBI fast?" Tags="|phylogenetics|" AnswerCount="2" CommentCount="0" ClosedDate="2019-04-29T12:24:16.643" ContentLicense="CC BY-SA 4.0" />
  <row Id="8509" PostTypeId="2" ParentId="7508" CreationDate="2019-04-24T08:36:07.600" Score="6" Body="&lt;p&gt;Answers to this question can be found on &lt;a href=&quot;https://www.biostars.org/p/91885/&quot; rel=&quot;noreferrer&quot;&gt;biostars&lt;/a&gt;. To give you a short summary:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Faster way might be to use the &lt;code&gt;parallel-fastq-dump&lt;/code&gt;, as suggested in this &lt;a href=&quot;https://www.biostars.org/p/91885/#300457&quot; rel=&quot;noreferrer&quot;&gt;answer&lt;/a&gt;. I never tested that tool though, my own experience is that &lt;code&gt;prefetch&lt;/code&gt; is more stable than &lt;code&gt;fastq-dump&lt;/code&gt; command.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In that same post it seems that the Japanese mirror seems faster, don't know if that is still valid.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The reason this is never optimized might be that downloading from &lt;code&gt;SRA&lt;/code&gt; is not a task you'll have to do repeatedly, as described here in this &lt;a href=&quot;https://www.biostars.org/p/91885/#91895&quot; rel=&quot;noreferrer&quot;&gt;post&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="939" LastActivityDate="2019-04-24T08:36:07.600" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8510" PostTypeId="1" CreationDate="2019-04-24T11:30:14.543" Score="1" ViewCount="43" Body="&lt;p&gt;I'm currently using &lt;a href=&quot;https://ccb.jhu.edu/software/stringtie/&quot; rel=&quot;nofollow noreferrer&quot;&gt;StringTie&lt;/a&gt; with option &lt;code&gt;-A&lt;/code&gt; for calculating gene abundance (I'm interested in TPM). While checking the manual, I couldn't find any reference to the record type StringTie uses to calculate this value. Does anyone know if it is total gene length or total effective (i.e. exon) gene length?&lt;/p&gt;&#xA;" OwnerUserId="1135" LastEditorUserId="298" LastEditDate="2019-04-24T11:56:44.243" LastActivityDate="2019-04-24T11:56:44.243" Title="Does StringTie use exon length for gene abundance or complete gene length?" Tags="|quantification|stringtie|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8512" PostTypeId="1" AcceptedAnswerId="8515" CreationDate="2019-04-24T12:52:31.567" Score="2" ViewCount="336" Body="&lt;p&gt;Beside it (maximum parsimony) being compuationally cheap what other good arguments are there for it? Is there any model behind this principle ? Why would one expect this principle to provide right phylogeny in any situation at all ?&lt;/p&gt;&#xA;" OwnerUserId="3555" LastActivityDate="2019-04-24T14:54:44.720" Title="What is the best argument for maximum parsimony method in phylogenetic tree construction?" Tags="|phylogenetics|phylogeny|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8513" PostTypeId="2" ParentId="8512" CreationDate="2019-04-24T13:06:55.330" Score="2" Body="&lt;p&gt;I'd say maximum parsimony approaches assume genetic similarity is unlikely to occur by convergence, but likely a result of divergence. If this assumptions holds, finding the tree configuration that minimizes the number of unlikely events makes sense.&lt;/p&gt;&#xA;" OwnerUserId="3818" LastActivityDate="2019-04-24T13:06:55.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8515" PostTypeId="2" ParentId="8512" CreationDate="2019-04-24T14:47:22.523" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;The principle of parsimony states that the simplest explanation consistent with a data set should be chosen over more complex explanations...&lt;/em&gt; (Stewart 1993)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Sometimes independent events in different lineages cause similarities in molecular sequences (or morphology or behavior or whatever characteristics one happens to be studying). These events are not true evidence of recently shared ancestry, but it is easy to mistake them as such evidence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The principle of parsimony asserts that we will be better off assuming the simplest case--that is, that identical characters &lt;strong&gt;&lt;em&gt;are&lt;/em&gt;&lt;/strong&gt; evidence of recent shared ancestry, since the simple explanation should be true more frequently than alternative explanations. There are cases for which the maximum parsimony model works very well, and there are cases for which maximum likelihood, distance matrix, Bayesian, or other models perform better.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Stewart CB (1993) The powers and pitfalls of parsimony. &lt;em&gt;Nature&lt;/em&gt; &lt;strong&gt;361&lt;/strong&gt;, 603-607, &lt;a href=&quot;https://doi.org/10.1038/361603a0&quot; rel=&quot;nofollow noreferrer&quot;&gt;doi:10.1038/361603a0&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-04-24T14:47:22.523" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8516" PostTypeId="2" ParentId="6706" CreationDate="2019-04-24T14:53:40.727" Score="1" Body="&lt;p&gt;I cross-posted this question on the &lt;a href=&quot;https://github.com/samtools/samtools/issues/994&quot; rel=&quot;nofollow noreferrer&quot;&gt;official samtools issue tracker&lt;/a&gt;, and finally got a response.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Looking at the code for the HTML view, I see &lt;a href=&quot;https://github.com/samtools/samtools/blob/develop/bam_tview_html.c#L320&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://github.com/samtools/samtools/blob/develop/bam_tview_html.c#L348-L352&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;.&#xA;  So maybe try&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;code&gt;COLUMNS=200 samtools tview -d H -p CHR:X-Y aln.bam ref.fasta&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-04-24T14:53:40.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8517" PostTypeId="2" ParentId="8512" CreationDate="2019-04-24T14:54:44.720" Score="1" Body="&lt;p&gt;Parsimony was a nice idea and developed the algorithms and tree methods towards its successors, maximum likelihood and Bayesian. The fundamental assumption of mutational independence at each alignment position was founded via parsimony. Just the recursion algorithm involved in  reading a tree isn't trivial coding. Also Walter Fitch (Fitch Marigold method) was a nice guy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Weighted parsimony has performed well in simulation studies to prevent 'long branch attraction' artefacts. The key issue that was never resolved was how to objectively weight an MP analysis. It could be resurrected however, because machine learning ML could independently provide the weights in specific contexts. That a lot of &quot;coulds&quot; though.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-04-24T14:54:44.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8519" PostTypeId="1" AcceptedAnswerId="8526" CreationDate="2019-04-24T17:33:10.293" Score="2" ViewCount="698" Body="&lt;p&gt;I'm working with an unassembled genome of cephalopod and I would to find transposable elements. My idea is to use repeatscout to generate a library and repeatmasker to find the elements. After creating a library with repeatscout,  I run Repeatmasker but it generates a table like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; file name: OB_100DEC.fa             &#xA;sequences:           100&#xA;total length:  209466439 bp  (184235452 bp excl N/X-runs)&#xA;GC level:         35.24 %&#xA;bases masked:   73100024 bp ( 34.90 %)&#xA;==================================================&#xA;               number of      length   percentage&#xA;               elements*    occupied  of sequence&#xA;--------------------------------------------------&#xA;SINEs:                0            0 bp    0.00 %&#xA;      ALUs            0            0 bp    0.00 %&#xA;      MIRs            0            0 bp    0.00 %&#xA;&#xA;LINEs:                0            0 bp    0.00 %&#xA;      LINE1           0            0 bp    0.00 %&#xA;      LINE2           0            0 bp    0.00 %&#xA;      L3/CR1          0            0 bp    0.00 %&#xA;&#xA;LTR elements:         0            0 bp    0.00 %&#xA;      ERVL            0            0 bp    0.00 %&#xA;      ERVL-MaLRs      0            0 bp    0.00 %&#xA;      ERV_classI      0            0 bp    0.00 %&#xA;      ERV_classII     0            0 bp    0.00 %&#xA;&#xA;DNA elements:         0            0 bp    0.00 %&#xA;     hAT-Charlie      0            0 bp    0.00 %&#xA;     TcMar-Tigger     0            0 bp    0.00 %&#xA;&#xA;Unclassified:    560941     66374119 bp   31.69 %&#xA;&#xA;Total interspersed repeats: 66374119 bp   31.69 %&#xA;&#xA;&#xA;Small RNA:            0            0 bp    0.00 %&#xA;&#xA;Satellites:           0            0 bp    0.00 %&#xA;Simple repeats:  290711     15358198 bp    7.33 %&#xA;Low complexity:       0            0 bp    0.00 %&#xA;==================================================&#xA;&#xA;* most repeats fragmented by insertions or deletions&#xA;  have been counted as one element&#xA;&#xA;&#xA;The query species was assumed to be homo          &#xA;RepeatMasker Combined Database: Dfam_Consensus-20181026&#xA;&#xA;run with rmblastn version 2.6.0+&#xA;The query was compared to unclassified sequences in &quot;.../OB_100DEC_repeats_filtered1.fa&quot;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This was my command to run repeatmasker:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./RepeatMasker -s -lib /home/RepeatScout-1.0.5/OB_100DEC_repeats_filtered1.fa /home/Workdirectory/OB_100DEC.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The file &lt;code&gt;OB_100DEC_repeats_filtered1.fa&lt;/code&gt; is the output of repeatscout and looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;R=3 (RR=4.  TRF=0.000 NSEG=0.000)&#xA;TAAGGCGGCGAGCTGGCAGAATCGTTAGCACGCCGGGCGAAATGCTTAGCGGTATTTCGTCTGTCTTTACGTTCTGAGTT&#xA;CAAATTCCGCCGAGGTCGACTTTGCCTTTCATCCTTTCGGGGTCGATAAAATAAGTACCAGTTGAGCACTGGGGTCGATG&#xA;TAATCGACTTACCCCCTCCCCCAAAATTTCTGGCCTTGTGCCTATATTAGAAACGATTATT&#xA;&amp;gt;R=4 (RR=5.  TRF=0.122 NSEG=0.226)&#xA;ACACACACACACACACACACACACACATATATATATATATACATATATACGACGGGCTTCTTTCAGTTTCCGTCTACCAA&#xA;ATCCACTCACAAGGCTTTGGTCGGCCCGAGGCTATAGTAGAAGACACTTGCCCAAGGTGCCACGCAGTGGGACTGAACCC&#xA;GGAACCATGTGGTTGGTAAGCAAGCTACTTACCACACAGCCACTCCTGCGCCTATATATAT&#xA;&amp;gt;R=6 (RR=7.  TRF=0.134 NSEG=0.247)&#xA;TTGTTTCAGTCATTTGACTGCGGCCATGCTGGAGCACCGCCTTTAGTCGAGCAAATCGACCCCAGGACTTATTCTTTGTA&#xA;AGCCTAGTACTTATTCTATCGGTCTCTTTTGCCGAACCGCTAAGTTACGGGGACGTAAACACACCAGCATCGGTTGTCAA&#xA;GCGATGTTGGGGGGACAAACACAGACACACAAACACACACACACACATACATATATATATATATATATATA&#xA;&amp;gt;R=8 (RR=9.  TRF=0.391 NSEG=0.000)`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and it is composed by all the TE (not classified) find in all the scaffolds of the genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone explain why there are no TE?&#xA;The fasta file that I used is only a part of the whole genome, it is about 200 mb and it is composed by the 100 greatest scaffolds of my genome.&#xA;Maybe I made a mistake with the use of libraries. Anyway I installed RepeatMasker with rmblastn version 2.6.0+ and RepeatMasker Combined Database: Dfam_Consensus-20181026.&#xA;thank you&lt;/p&gt;&#xA;" OwnerUserId="4475" LastEditorUserId="4475" LastEditDate="2019-04-25T09:33:45.190" LastActivityDate="2019-04-25T12:33:01.170" Title="Repeatmasker output table" Tags="|transposable-elements|repeatmasker|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8520" PostTypeId="1" AcceptedAnswerId="8525" CreationDate="2019-04-24T17:34:02.400" Score="4" ViewCount="71" Body="&lt;p&gt;I am looking for a database that would keep track of  human cytogenetic regions and genomic coordinates per genome assembly. I had expected the Genome Reference Consortium to have it, or Ensembl, or NCBI, but I could not find it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a place with a translation table between human cytogenetic regions and genomic coordinates?&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;EDIT&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;After benn's answer I found that you can also get this information from Ensembl's REST API: &lt;a href=&quot;https://rest.ensembl.org/info/assembly/homo_sapiens?content-type=application/json&amp;amp;bands=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rest.ensembl.org/info/assembly/homo_sapiens?content-type=application/json&amp;amp;bands=1&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have transformed the data provided by Ensembl and generated a CSV file with the cytogenetic regions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/ramiromagno/gwasrapidd/blob/master/data-raw/cytogenetic_bands.csv&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ramiromagno/gwasrapidd/blob/master/data-raw/cytogenetic_bands.csv&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1503" LastEditorUserId="1503" LastEditDate="2019-05-13T23:01:05.097" LastActivityDate="2019-05-13T23:01:05.097" Title="Authoritative source on human cytogenetic regions?" Tags="|assembly|human-genome|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8522" PostTypeId="1" CreationDate="2019-04-24T23:28:39.883" Score="0" ViewCount="43" Body="&lt;p&gt;Anyone knows of an example of a protein that, without coming from a recent duplication event, underwent a mutation(s) that caused it to have a novel interaction with a new ligand, substrate, other protein or molecule?&#xA;By novel I imply that it doesn't &quot;go back&quot; to an ancestral interaction or doesn’t develops an interaction that it's already performed by a close relative. &#xA;I assume that these mutations are extremely rare, and that a duplicate is almost always necessary to have them fixed in a lineage.&lt;/p&gt;&#xA;" OwnerUserId="4476" LastEditorUserId="4476" LastEditDate="2019-04-25T05:52:15.833" LastActivityDate="2019-04-25T06:39:47.623" Title="Evidence of emergence of &quot;genuine&quot; novel protein interactions by aa mutation" Tags="|sequence-annotation|phylogenetics|protein-structure|variation|" AnswerCount="2" CommentCount="0" ClosedDate="2019-04-29T16:19:08.987" ContentLicense="CC BY-SA 4.0" />
  <row Id="8523" PostTypeId="2" ParentId="8522" CreationDate="2019-04-25T04:27:52.220" Score="0" Body="&lt;p&gt;That's easy, red blood cells and the single mutation causing the switch in charge resulting in sickle cell anaemia. There are five different convergent mutations although the concept of convergence here has been challenged. The mutation results in the physical collapse of the protein structure and massively reduces its binding with oxygen. The receptor interactions with the malarial parasite render infection of the red blood cell impossible. Even in the heterozygote state, oxygen binding with the heme group is not massively disrupted, but the ability of the malaria parasite in particular plasmodium falciparum, to invade the red blood cell is again greatly impaired. It is an incredible example of human evolution and demonstrates the selective pressure of holoedemica malaria. There is no gene duplication involved here and again due stress there are potentially 5 convergent events resulting in the same phenotype and the scale of protein collapse from circular red blood cell to sickle cell shape is the result of what would otherwise appear to be a small change in amino acid charge for a single amino acid substitution.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-04-25T04:27:52.220" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8524" PostTypeId="2" ParentId="8522" CreationDate="2019-04-25T06:39:47.623" Score="1" Body="&lt;p&gt;HA Sialic acid receptor binding in influenza virus determining host specificity, viz. Bird flu vs human flu. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-04-25T06:39:47.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8525" PostTypeId="2" ParentId="8520" CreationDate="2019-04-25T09:24:06.323" Score="5" Body="&lt;p&gt;Do you mean cytoband coordinates? For the human genome you can find them for example at UCSC.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;hgdownload.cse.ucsc.edu/goldenPath/hg38/database/cytoBand.txt.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastActivityDate="2019-04-25T09:24:06.323" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8526" PostTypeId="2" ParentId="8519" CreationDate="2019-04-25T12:33:01.170" Score="1" Body="&lt;p&gt;As far as I can tell, you use the &lt;code&gt;-lib&lt;/code&gt; option for species where there is no existing RepeatMaster database. Therefore, all it does is attempt to locate versions of the repetitive elements from the &lt;code&gt;-lib&lt;/code&gt; file in the target genome file. However, since your lib file doesn't have any information on what type of element each of your hits is, it looks like RepeatMasker is treating all of them as &quot;Unclassified&quot;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note the following from the &lt;code&gt;repeatmasker.help&lt;/code&gt; file you should have downloaded along with RepeatMasker:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The recommended format for IDs in a custom library is:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&gt;repeatname#class/subclass &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;or simply&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&gt;repeatname#class&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;In this format, the data will be processed (overlapping repeats are&#xA;  merged etc), alternative output (.ace or .gff) can be created and an&#xA;  overview .tbl file will be created. Classes that will be displayed in&#xA;  the .tbl file are 'SINE', 'LINE', 'LTR', 'DNA', 'Satellite', anything&#xA;  with 'RNA' in it, 'Simple_repeat', and 'Other' or 'Unknown' (the&#xA;  latter defaults when class is missing).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This makes sense. RepeatMasker cannot know if an arbitrary sequence you give it is a SINE or a LINE or anything else. It will therefore rely on the categories present in the fasta headers. Since your file has no categories, it identifies everything in your lib file as &quot;Unknown&quot;, which is presumably what the &quot;Unclassified&quot; section in your output refers to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Unclassified:    560941     66374119 bp   31.69 %&#xA;&#xA;Total interspersed repeats: 66374119 bp   31.69 %&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-04-25T12:33:01.170" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8527" PostTypeId="1" AcceptedAnswerId="8530" CreationDate="2019-04-26T00:37:48.353" Score="1" ViewCount="50" Body="&lt;p&gt;I am trying to fetch genome assemblies that include splicing information.&lt;br&gt;&#xA;E.g. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/CP003820.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt;, is a genome assembly of a yeast, where genes show variants like this: &lt;code&gt;join(5928..7626,7685..7785,7857..7982)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am retrieving the information from NCBI with the following commands:&lt;br&gt;&#xA;&lt;code&gt;efetch -db nuccore -id CP003820.1 -format tabular&lt;/code&gt;. I tried different formats and it changes but I don't get the &lt;code&gt;join(...)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Am I correct and the &lt;code&gt;join(...)&lt;/code&gt; indicates &quot;splicing-like&quot; modifications in yeast?&lt;br&gt;&#xA;2) How can I get the &lt;code&gt;join(...)&lt;/code&gt; (any other format is fine too) with efetch?&lt;/p&gt;&#xA;" OwnerUserId="2970" LastEditorUserId="48" LastEditDate="2019-04-26T07:48:56.817" LastActivityDate="2019-04-26T12:14:57.327" Title="NCBI retrieve genome assembly with splicing or &quot;modifications&quot;" Tags="|phylogenetics|data-retrieval|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8528" PostTypeId="1" CreationDate="2019-04-26T10:03:08.523" Score="0" ViewCount="33" Body="&lt;p&gt;For a project, I want to predict the subcellular localization of some proteins. I am a student in software Informatics at the University of applied Science HEIG-VD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Subcellular localization prediction is a common problem in the bioinformatics, but I have a problem&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the data I found. In this &lt;a href=&quot;http://locate.imb.uq.edu.au/downloads.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;database&lt;/a&gt;: of human and mouse proteins, we can &lt;a href=&quot;http://locate.imb.uq.edu.au/faq.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;see&lt;/a&gt; that there are more than 30 different localizations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At my level, and for the use of my exercise, 30 classes to predict is too much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where can I find a mapping to reduce those 30 classes to 4 or 5?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried QuickGO, to see the ancestor tree of the different classes, but nothing conclusive. Somebody might have done that, but I lack knowledge on bioinformatic web research.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are the classes I wanted to map to reduce the number of classes :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Cytoplasm (GO:0005737) &lt;/li&gt;&#xA;&lt;li&gt;Cytoplasmic Vesicles (GO:0016023) &lt;/li&gt;&#xA;&lt;li&gt;Endoplasmic Reticulum (GO:0005783) &lt;/li&gt;&#xA;&lt;li&gt;Endosomes (GO:0005768) &lt;/li&gt;&#xA;&lt;li&gt;Extracellular (GO:0005576) &lt;/li&gt;&#xA;&lt;li&gt;Golgi Apparatus (GO:0005794) &lt;/li&gt;&#xA;&lt;li&gt;Lipid Particles (GO:0005811) &lt;/li&gt;&#xA;&lt;li&gt;Lysosomes (GO:0005764) &lt;/li&gt;&#xA;&lt;li&gt;Melanosome (GO:0042470)&lt;/li&gt;&#xA;&lt;li&gt;Mitochondria (GO:0005739) &lt;/li&gt;&#xA;&lt;li&gt;Nucleus (GO:0005634) &lt;/li&gt;&#xA;&lt;li&gt;Peroxisome (GO:0005777) &lt;/li&gt;&#xA;&lt;li&gt;Plasma Membrane (GO:0005886) &lt;/li&gt;&#xA;&lt;li&gt;Synaptic Vesicles (GO:0008021) &lt;/li&gt;&#xA;&lt;li&gt;Cellular Component Unknown (GO:0008372) &lt;/li&gt;&#xA;&lt;li&gt;Apical Plasma Membrane (GO:0016324) &lt;/li&gt;&#xA;&lt;li&gt;Basolateral Plasma Membrane (GO:0016323)&lt;/li&gt;&#xA;&lt;li&gt;Centrosome (GO:0005813) &lt;/li&gt;&#xA;&lt;li&gt;Golgi Cis Cisterna (GO:0000137) &lt;/li&gt;&#xA;&lt;li&gt;Cytoskeleton (GO:0005856) &lt;/li&gt;&#xA;&lt;li&gt;Early Endosomes (GO:0005769) &lt;/li&gt;&#xA;&lt;li&gt;ERGIC (GO:0005793) &lt;/li&gt;&#xA;&lt;li&gt;Inner Mitochondrial Membrane (GO:0005743) &lt;/li&gt;&#xA;&lt;li&gt;Late Endosomes (GO:0005770)&lt;/li&gt;&#xA;&lt;li&gt;Medial-Golgi (GO:0005797) &lt;/li&gt;&#xA;&lt;li&gt;Outer Mitochondrial Membrane (GO:0005741)&lt;/li&gt;&#xA;&lt;li&gt;Secretory Granule (GO:0030141) &lt;/li&gt;&#xA;&lt;li&gt;Golgi Trans Cisterna (GO:0000137)&lt;/li&gt;&#xA;&lt;li&gt;Golgi Trans Face (GO:0005802) &lt;/li&gt;&#xA;&lt;li&gt;Tight Junction (GO:0005923) &lt;/li&gt;&#xA;&lt;li&gt;Transport Vesicle (GO:0030133)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4486" LastEditorUserId="4486" LastEditDate="2019-04-26T11:14:09.780" LastActivityDate="2019-05-29T11:34:49.473" Title="How do i map subcellular localizations to lower the number of classes to predict" Tags="|proteins|gene-ontology|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8529" PostTypeId="2" ParentId="8528" CreationDate="2019-04-26T11:47:08.337" Score="1" Body="&lt;p&gt;One way to do this would be to plot the first 10 GO categories of the the CC sub ontology (CC from cellular component). Or you can just visit &lt;a href=&quot;http://amigo.geneontology.org/amigo/term/GO:0005575&quot; rel=&quot;nofollow noreferrer&quot;&gt;the official&lt;/a&gt; webpage to see the list of children terms of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However there aren't only 4 or 5 categories, so you could arbitrary pick some of those. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2019-04-26T11:47:08.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8530" PostTypeId="2" ParentId="8527" CreationDate="2019-04-26T12:14:57.327" Score="3" Body="&lt;p&gt;The page you link to is a GenBank Flat file, the default format used by GenBank. So, to get that format, which includes the &lt;code&gt;join&lt;/code&gt; lines, use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./efetch -db nuccore -id CP003820.1 -format gb&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;join()&lt;/code&gt; lines give the different ways in which a sequence can be built from this gene. For example, one mRNA sequence produced by the locus you mention (CP003820) is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  mRNA            complement(join(100..1497,1556..1767,1823..2274,&#xA;                 2322..2782,2846..3126,3206..3890,3958..5263,5322..5422,&#xA;                 5494..&amp;gt;5645))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The corresponding coding sequence is for the mRNA above is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; CDS             complement(join(168..1497,1556..1767,1823..2274,&#xA;                 2322..2782,2846..3126,3206..3890,3958..5263,5322..5422,&#xA;                 5494..5645))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These aren't necessarily splicing &quot;modifications&quot;, and aren't &quot;variants&quot;, they are simply the way that the exons are joined to make an mRNA. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that since CP003820.1 is &quot;Cryptococcus neoformans var. grubii H99 chromosome 1, complete sequence&quot;, there are multiple genes described in there and each will have its own (or multiple, if there is alternative splicing) &lt;code&gt;join&lt;/code&gt; statements. Specifically, I see 1014 genes described in the file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -Pc '^\s+gene\s+' CP003820.gb &#xA;1014&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;950 mRNAs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -c 'mRNA.*join(' CP003820.gb &#xA;950&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And 955 proteins:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -Pc 'protein_id=' CP003820.gb &#xA;955&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-04-26T12:14:57.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8531" PostTypeId="1" AcceptedAnswerId="8532" CreationDate="2019-04-26T12:17:09.593" Score="-9" ViewCount="559" Body="&lt;p&gt;I have called copy number and I have 2 files (I have shared the link of my files ); One contains some ranges &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(cndata[,c(2,3,4)])&#xA;     Chromosome   Start      End&#xA;4470          1   51479   817980&#xA;4471          1  818499  1136753&#xA;4472          1 1138735  2558308&#xA;4473          1 2558740  5724264&#xA;4474          1 5724940  5749083&#xA;4475          1 5749226 12529544&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/wei3i787c9nrz8q/WTSI-OESO_121_1pre.copynumber.caveman.txt.csv?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/wei3i787c9nrz8q/WTSI-OESO_121_1pre.copynumber.caveman.txt.csv?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have another file like below &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;              Chromosome         Position&#xA;rs62635286       1                 51479   &#xA;rs75454623       1                 114930&#xA;rs806731         1                 30923&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/tiqjhkrpfr9d9gu/WTSI-OESO_121_1pre.copynumber.txt?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/tiqjhkrpfr9d9gu/WTSI-OESO_121_1pre.copynumber.txt?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In first  column I have some SNPs, how I can count the number of SNP in each range? For example based on the second table how many SNP are in range of &lt;code&gt;51479&lt;/code&gt; to  &lt;code&gt;817980&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-04-26T12:33:04.233" LastActivityDate="2019-04-26T20:16:53.257" Title="Extracting the number SNP in each range" Tags="|snp|bedtools|wgs|copy-number|text-mining|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8532" PostTypeId="2" ParentId="8531" CreationDate="2019-04-26T20:16:53.257" Score="3" Body="&lt;p&gt;There is like a thousand different ways how to achieve this. You could use a specialised software for this (like bedtools) or calculate it simply in R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;R solution: You can make a function that calculates the number of SNPs in a range and than you can apply it on all the ranges in the table with genomic ranges.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;snp_table &amp;lt;- read.table('WTSI-OESO_121_1pre.copynumber.txt')&#xA;genomic_ranges &amp;lt;- read.table('WTSI-OESO_121_1pre.copynumber.caveman.txt.csv', sep = ',', col.names = c('Chromosome', 'Start', 'End'), stringsAsFactors = F)&#xA;&#xA;how_many_in_range &amp;lt;- function(coords){&#xA;    coords = as.vector(coords)&#xA;    sum(snp_table&lt;span class=&quot;math-container&quot;&gt;$Chromosome == coords[1] &amp;amp; snp_table$&lt;/span&gt;Position &amp;gt; as.numeric(coords[2]) &amp;amp; snp_table$Position &amp;lt; as.numeric(coords[3]))&#xA;}&#xA;&#xA;genomic_ranges$number_of_snps &amp;lt;- apply(genomic_ranges, 1, how_many_in_range)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now the table &lt;code&gt;genomic_ranges&lt;/code&gt; has a variable that sais how many SNPs are in the range.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2019-04-26T20:16:53.257" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8533" PostTypeId="1" CreationDate="2019-04-26T23:37:16.717" Score="2" ViewCount="1105" Body="&lt;p&gt;I'm trying to use igv.js for a very simple visualization of some genomic alignments, but I can't get past the first step.  I have a FASTA file that I want to use as the reference, but igv.js consistently gives an error when I try to load that file.  The file is a valid FASTA (have tested others from other sources to be sure - same error), and I can specify a remote https:// file, but not a local file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the HTML page I'm working on:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;!DOCTYPE html&amp;gt;&#xA;&amp;lt;html lang=&quot;en&quot;&amp;gt;&#xA;&#xA;&amp;lt;head&amp;gt;&#xA;    &amp;lt;meta charset=&quot;UTF-8&quot;&amp;gt;&#xA;    &amp;lt;meta name=&quot;viewport&quot; content=&quot;width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no&quot;&amp;gt;&#xA;&#xA;    &amp;lt;title&amp;gt;Local FASTA Test&amp;lt;/title&amp;gt;&#xA;&#xA;    &amp;lt;!-- IGV JS--&amp;gt;&#xA;    &amp;lt;script src=&quot;https://igv.org/web/release/2.1.0/dist/igv.min.js&quot;&amp;gt;&amp;lt;/script&amp;gt;&#xA;&#xA;&amp;lt;/head&amp;gt;&#xA;&#xA;&amp;lt;body&amp;gt;&#xA;&#xA;    &amp;lt;h1&amp;gt;Local File Test&amp;lt;/h1&amp;gt;&#xA;&#xA;    &amp;lt;div id=&quot;igv-div&quot; style=&quot;padding-top: 10px;padding-bottom: 10px; border:1px solid lightgray&quot;&amp;gt;&amp;lt;/div&amp;gt;&#xA;&#xA;    &amp;lt;script type=&quot;text/javascript&quot;&amp;gt;&#xA;        var igvDiv = document.getElementById(&quot;igv-div&quot;);&#xA;        var options = {&#xA;            reference: {&#xA;                fastaURL: &quot;test.fasta&quot;&#xA;            }&#xA;        };&#xA;&#xA;        igv.createBrowser(igvDiv, options)&#xA;            .then(function(browser) {&#xA;                console.log(&quot;Created IGV browser&quot;);&#xA;            })&#xA;    &amp;lt;/script&amp;gt;&#xA;&#xA;&amp;lt;/body&amp;gt;&#xA;&#xA;&amp;lt;/html&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and here's the FASTA file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;HSBGPG Human gene for bone gla protein (BGP)&#xA;GGCAGATTCCCCCTAGACCCGCCCGCACCATGGTCAGGCATGCCCCTCCTCATCGCTGGGCACAGCCCAGAGGGT&#xA;ATAAACAGTGCTGGAGGCTGGCGGGGCAGGCCAGCTGAGTCCTGAGCAGCAGCCCAGCGCAGCCACCGAGACACC&#xA;ATGAGAGCCCTCACACTCCTCGCCCTATTGGCCCTGGCCGCACTTTGCATCGCTGGCCAGGCAGGTGAGTGCCCC&#xA;CACCTCCCCTCAGGCCGCATTGCAGTGGGGGCTGAGAGGAGGAAGCACCATGGCCCACCTCTTCTCACCCCTTTG&#xA;GCTGGCAGTCCCTTTGCAGTCTAACCACCTTGTTGCAGGCTCAATCCATTTGCCCCAGCTCTGCCCTTGCAGAGG&#xA;GAGAGGAGGGAAGAGCAAGCTGCCCGAGACGCAGGGGAAGGAGGATGAGGGCCCTGGGGATGAGCTGGGGTGAAC&#xA;CAGGCTCCCTTTCCTTTGCAGGTGCGAAGCCCAGCGGTGCAGAGTCCAGCAAAGGTGCAGGTATGAGGATGGACC&#xA;TGATGGGTTCCTGGACCCTCCCCTCTCACCCTGGTCCCTCAGTCTCATTCCCCCACTCCTGCCACCTCCTGTCTG&#xA;GCCATCAGGAAGGCCAGCCTGCTCCCCACCTGATCCTCCCAAACCCAGAGCCACCTGATGCCTGCCCCTCTGCTC&#xA;CACAGCCTTTGTGTCCAAGCAGGAGGGCAGCGAGGTAGTGAAGAGACCCAGGCGCTACCTGTATCAATGGCTGGG&#xA;GTGAGAGAAAAGGCAGAGCTGGGCCAAGGCCCTGCCTCTCCGGGATGGTCTGTGGGGGAGCTGCAGCAGGGAGTG&#xA;GCCTCTCTGGGTTGTGGTGGGGGTACAGGCAGCCTGCCCTGGTGGGCACCCTGGAGCCCCATGTGTAGGGAGAGG&#xA;AGGGATGGGCATTTTGCACGGGGGCTGATGCCACCACGTCGGGTGTCTCAGAGCCCCAGTCCCCTACCCGGATCC&#xA;CCTGGAGCCCAGGAGGGAGGTGTGTGAGCTCAATCCGGACTGTGACGAGTTGGCTGACCACATCGGCTTTCAGGA&#xA;GGCCTATCGGCGCTTCTACGGCCCGGTCTAGGGTGTCGCTCTGCTGGCCTGGCCGGCAACCCCAGTTCTGCTCCT&#xA;CTCCAGGCACCCTTCTTTCCTCTTCCCCTTGCCCTTGCCCTGACCTCCCAGCCCTATGGATGTGGGGTCCCCATC&#xA;ATCCCAGCTGCTCCCAAATAAACTCCAGAAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4491" LastEditorUserId="96" LastEditDate="2019-04-29T14:30:27.340" LastActivityDate="2019-10-30T05:41:52.877" Title="Loading local FASTA file in igv.js" Tags="|visualization|igv|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8534" PostTypeId="1" CreationDate="2019-04-27T22:36:41.090" Score="2" ViewCount="52" Body="&lt;p&gt;Obviously there is a gap in my understanding of how sex chromosomes are annotated. I've been working on some CNV calls and noticed that most of the 2nd half of ChrY is missing annotations and no reads are being mapped there. Yq12 seems to be all NNNNNNN&lt;a href=&quot;https://i.stack.imgur.com/0STLv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0STLv.png&quot; alt=&quot;UCSC table browser&quot;&gt;&lt;/a&gt; in the hg38 reference.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone know why? &lt;/p&gt;&#xA;" OwnerUserId="460" LastActivityDate="2019-04-29T10:08:13.387" Title="No refseq transcripts in 2nd half of Y chromosome for HG38" Tags="|sequence-annotation|read-mapping|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8535" PostTypeId="2" ParentId="8534" CreationDate="2019-04-27T23:04:16.860" Score="2" Body="&lt;p&gt;Nevermind, Answered my own question. It's a repetitive region that isn't very well understood&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The human Y chromosome, because it is enriched in repetitive DNA, has been very intractable to genetic and molecular analyses. There is no previous evidence for developmental stage- and testis-specific transcription from the male-specific region of the Y (MSY). Here, we present evidence for the first time for a developmental stage- and testis-specific transcription from MSY distal heterochromatic block. We isolated two novel RNAs, which localize to Yq12 in multiple copies, show testis-specific expression, and lack active X-homologs. Experimental evidence shows that one of the above Yq12 noncoding RNAs (ncRNAs) trans-splices with CDC2L2 mRNA from chromosome 1p36.3 locus to generate a testis-specific chimeric β sv13 isoform. This 67-nt 5′UTR provided by the Yq12 transcript contains within it a Y box protein-binding CCAAT motif, indicating translational regulation of the β sv13 isoform in testis. This is also the first report of trans-splicing between a Y chromosomal and an autosomal transcript.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1832090/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1832090/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="460" LastEditorUserId="298" LastEditDate="2019-04-29T10:08:13.387" LastActivityDate="2019-04-29T10:08:13.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8536" PostTypeId="2" ParentId="500" CreationDate="2019-04-29T00:39:30.740" Score="1" Body="&lt;p&gt;If you're looking for a python API, nanopore have created one (although I am not sure how &quot;maintained&quot; it is) &lt;a href=&quot;https://github.com/nanoporetech/bwapy&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nanoporetech/bwapy&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="149" LastActivityDate="2019-04-29T00:39:30.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8537" PostTypeId="1" AcceptedAnswerId="8538" CreationDate="2019-04-29T03:49:21.187" Score="0" ViewCount="79" Body="&lt;p&gt;I ran into the following error when using &lt;code&gt;cooler&lt;/code&gt; to load a contact pairs file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ubuntu@ip-172-31-8-42:/data/hg-tmp$ FILE=GSE109344_monkey_fibro_allValidPairs.txt.gz; sudo docker run -v $(pwd):/data $TAG cooler cload pairs -c1 2 -p1 3 -c2 5 -p2 6 /data/chromInfo.rheMac2.&#xA;simple.txt:${BINSIZE} /data/$FILE /data/${FILE}.cool;&#xA;Traceback (most recent call last):&#xA;  File &quot;/usr/local/bin/cooler&quot;, line 11, in &amp;lt;module&amp;gt;&#xA;    sys.exit(cli())&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/click/core.py&quot;, line 764, in __call__&#xA;    return self.main(*args, **kwargs)&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/click/core.py&quot;, line 717, in main&#xA;    rv = self.invoke(ctx)&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/click/core.py&quot;, line 1137, in invoke&#xA;    return _process_result(sub_ctx.command.invoke(sub_ctx))&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/click/core.py&quot;, line 1137, in invoke&#xA;    return _process_result(sub_ctx.command.invoke(sub_ctx))&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/click/core.py&quot;, line 956, in invoke&#xA;    return ctx.invoke(self.callback, **ctx.params)&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/click/core.py&quot;, line 555, in invoke&#xA;    return callback(*args, **kwargs)&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/cooler/cli/cload.py&quot;, line 477, in pairs&#xA;    map(pipeline, reader),&#xA;  File &quot;cytoolz/functoolz.pyx&quot;, line 493, in cytoolz.functoolz.Compose.__call__&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/cooler/create/_ingest.py&quot;, line 425, in _aggregate_records&#xA;    .aggregate(agg)&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/pandas/core/groupby/generic.py&quot;, line 1315, in aggregate&#xA;    return super(DataFrameGroupBy, self).aggregate(arg, *args, **kwargs)&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/pandas/core/groupby/generic.py&quot;, line 186, in aggregate&#xA;    result, how = self._aggregate(arg, _level=_level, *args, **kwargs)&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/pandas/core/base.py&quot;, line 498, in _aggregate&#xA;    result = _agg(arg, _agg_1dim)&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/pandas/core/base.py&quot;, line 449, in _agg&#xA;    result[fname] = func(fname, agg_how)&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/pandas/core/base.py&quot;, line 432, in _agg_1dim&#xA;    return colg.aggregate(how, _level=(_level or 0) + 1)&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/pandas/core/groupby/generic.py&quot;, line 760, in aggregate&#xA;    return getattr(self, func_or_funcs)(*args, **kwargs)&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/pandas/core/groupby/groupby.py&quot;, line 1227, in size&#xA;    result = self.grouper.size()&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/pandas/core/groupby/ops.py&quot;, line 225, in size&#xA;    ids, _, ngroup = self.group_info&#xA;  File &quot;pandas/_libs/properties.pyx&quot;, line 36, in pandas._libs.properties.CachedProperty.__get__&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/pandas/core/groupby/ops.py&quot;, line 252, in group_info&#xA;    comp_ids, obs_group_ids = self._get_compressed_labels()&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/pandas/core/groupby/ops.py&quot;, line 272, in _get_compressed_labels&#xA;    return compress_group_index(group_index, sort=self.sort)&#xA;  File &quot;/usr/local/lib/python2.7/site-packages/pandas/core/sorting.py&quot;, line 379, in compress_group_index&#xA;    comp_ids, obs_group_ids = table.get_labels_groupby(group_index)&#xA;  File &quot;pandas/_libs/hashtable_class_helper.pxi&quot;, line 1255, in pandas._libs.hashtable.Int64HashTable.get_labels_groupby&#xA;  File &quot;pandas/_libs/hashtable_class_helper.pxi&quot;, line 199, in pandas._libs.hashtable.Int64Vector.resize&#xA;MemoryError: cannot allocate memory for array&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The machine has 8Gb of RAM. Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="9" LastEditorUserId="3541" LastEditDate="2019-12-01T12:43:04.117" LastActivityDate="2019-12-01T12:43:04.117" Title="cooler out of memory" Tags="|python|errors|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8538" PostTypeId="2" ParentId="8537" CreationDate="2019-04-29T04:00:40.107" Score="5" Body="&lt;p&gt;It seems to be some kind of a memory leak in pandas's &lt;code&gt;read_csv&lt;/code&gt;, maybe due to it doing gzip-decompression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing the decompression in a separate process and piping the result into &lt;code&gt;cooler cload pairs&lt;/code&gt; should do the trick.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gzip -dc GSE109344_monkey_fibro_allValidPairs.txt.gz | cooler cload pairs -c1 2 -p1 3 -c2 5 -p2 6 /data/chromInfo.rheMac2.&#xA;simple.txt:${BINSIZE} - /data/${FILE}.cool&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4497" LastActivityDate="2019-04-29T04:00:40.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8540" PostTypeId="2" ParentId="3022" CreationDate="2019-04-29T10:56:20.643" Score="2" Body="&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; I suspect this question is about the phylogenetics of methylation and the approach the investigator is proposing would be the last approach to use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&#xA;The approaches to assess the phylogenetics of methylation in order of preference are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;dN/dS between CpG sites and non-CpG sites, &lt;/li&gt;&#xA;&lt;li&gt;An explicit molecular clock between CpG sites and non-CpG sites&lt;/li&gt;&#xA;&lt;li&gt;A likelihood ration test (LRT) based on a null distribution of mutations generated by a Monte Carlo algorithm &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Background/Rationale&lt;/strong&gt;&#xA;The approaches you are using have been around for a long while and generate the random distribution of mutations for a given phylogeny. Its phylogenetics in reverse and the technique is known as Monte Carlo, you start with a randomised probability and send it through a parameterised model to predict the amino acid/nucleotide. Thus it is ML(max likelihood), Bayesian, &quot;phylogenetic-HMM&quot; in reverse. It is used within likelihood ratio test calculations. The model is determined by a standard ML, Bayesian phylogeny algorithm, i.e. its circular mathematics because there is no independent means of calculating the mutation behaviour, so the precise context you are using this for needs to be carefully considered.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Packages&lt;/strong&gt; There are lots of packages that do this type of Monte Carlo simulation, SOWHat being a good one (LRT) and you generate a large number of replicate data sets (100 or 1000). The one of the base algorithms is &quot;seq-gen&quot;, although PAML may have implemented this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Considerations&lt;/strong&gt; To use this style of approach you need to carefully consider your question. The &quot;mutation simulations&quot; when analysed via a phylogenetic program will produce the same parameters that you initially set and trees of basically the same length. If you are using this to generate a null distribution for a phylogenetics test these approaches are useful, you then compare the observed likelihood against the null distribution. If you are using it to work out whether the mutation rate at CpG sites is higher than at other sites, its one approach amongst a number of alternatives. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Drawbacks&lt;/strong&gt; Calculating a de novo null distribution of the mutation rate is computationally very expensive and therefore tends to the last calculation performed. It will address a single hypothesis, usually regarding topology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Crux of the problem&lt;/strong&gt; To really do what (I think) you want to do you need an independent measure of mutation and mutational behaviour and its not trivial to achieve. You would have to consider it like a machine learning calculation, with a formal training set, I don't know whether such an approach has been implemented. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Strengths/Summary&lt;/strong&gt; In summary, Monte Carlo phylogenetics simulation has a limited application because of circularity of the calculation, BUT, BUT, BUT what I have omitted is when it is appropriate it is a very powerful test indeed.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Points of interest&lt;/strong&gt; I do like the population dynamic simulations within Beast, this certainly &quot;a priori&quot; and its potential IMO needs further exploration. However, I don't think you looking at how molecular epidemiology might impact the mutational behaviour methylation. I have not looked at Indelible, but looks interesting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was going to talk about &quot;processor war simulations&quot; but its probably off-topic.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-04-29T12:54:56.010" LastActivityDate="2019-04-29T12:54:56.010" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8541" PostTypeId="2" ParentId="7508" CreationDate="2019-04-29T11:59:58.650" Score="1" Body="&lt;p&gt;You can try &lt;code&gt;wget&lt;/code&gt; to download SRA files from NCBI server. This seems to be 2x faster in terms of CPU time than &lt;code&gt;prefetch&lt;/code&gt; and has the advantage of not clogging your &lt;code&gt;/home/ncbi/public/sra/&lt;/code&gt; directory in case you have space limitation. The tutorial for &lt;code&gt;wget&lt;/code&gt; downloading method is available at &lt;a href=&quot;https://github.com/johanzi/ncbi_tutorial&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/johanzi/ncbi_tutorial&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here my timing of the &lt;code&gt;prefetch&lt;/code&gt; method:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time prefetch SRR3300113&#xA;real    3m5.609s&#xA;user    0m6.332s&#xA;sys     0m2.076s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Versus the &lt;code&gt;wget&lt;/code&gt; method:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time wget ftp://ftp-trace.ncbi.nih.gov/sra/sra-instant/reads/ByRun/sra/SRR/SRR330/SRR3300113/SRR3300113.sra&#xA;real    3m4.696s&#xA;user    0m0.456s&#xA;sys     0m3.332s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That makes a user+sys time of 8.402s for &lt;code&gt;prefetch&lt;/code&gt; and 3.788s for &lt;code&gt;wget&lt;/code&gt;. Not so much difference for real time though (about a second).&lt;/p&gt;&#xA;" OwnerUserId="4500" LastEditorUserId="4500" LastEditDate="2019-05-01T14:34:27.090" LastActivityDate="2019-05-01T14:34:27.090" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8544" PostTypeId="2" ParentId="8533" CreationDate="2019-04-29T13:27:14.143" Score="2" Body="&lt;p&gt;It looks as though the script is looking for a URL (fastaURL) so you could try specifying the file as file://directory/test.fasta&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By default though, javascript cannot read local files from disk for security reasons. Are you sure it is even possible to specify a local file in your script in this way?  If yes, then the problem is likely that you need to specify the path correctly.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-04-29T13:27:14.143" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8545" PostTypeId="1" CreationDate="2019-04-29T14:05:14.510" Score="0" ViewCount="213" Body="&lt;p&gt;I have installed some ucsc tools on my Mac computer and it worked fine:&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Installation&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conda create --name ucsc-netchainsubset377 --channel bioconda --yes ucsc-netchainsubset=377&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Conda environment activation&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;source activate ucsc-netchainsubset377&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Checking the tool's help&lt;/h3&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    netchainsubset&#xA;netChainSubset - Create chain file with subset of chains that appear in the net&#xA;usage:&#xA;   netChainSubset in.net in.chain out.chain&#xA;options:&#xA;   -gapOut=gap.tab - Output gap sizes to file&#xA;   -type=XXX - Restrict output to particular type in net file&#xA;   -splitOnInsert - Split chain when get an insertion of another chain&#xA;   -wholeChains - Write entire chain references by net, don't split&#xA;    when a high-level net is encoundered.  This is useful when nets&#xA;    have been filtered.&#xA;   -skipMissing - skip chains that are not found instead of generating&#xA;    an error.  Useful if chains have been filtered.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The same commands on GNU/Linux produced the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;netchainsubset&#xA;netchainsubset: command not found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;even though the tool seems to be installed successfully:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Downloading and Extracting Packages&#xA;ucsc-netchainsubset- | 310 KB    | ############################################################################ | 100%&#xA;mysql-connector-c-6. | 4.4 MB    | ############################################################################ | 100%&#xA;Preparing transaction: done&#xA;Verifying transaction: done&#xA;Executing transaction: done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The same happened with &lt;code&gt;ucsc-fatotwobit&lt;/code&gt; and &lt;code&gt;ucsc-liftup&lt;/code&gt; and I would not be surprised if this behaviour affected the whole suite of ucsc programmes installed through conda.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone can suggest a solution?&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Update 5 minutes after posting&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Thanks to terdon, I printed the PATH when the conda environment was active and listed what was in the folder where conda installed the UCSC tool. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ls /mnt/home2/miska/cr517/anaconda3/envs/ucsc-netchainsubset377/bin&#xA;c_rehash         libpng-config      mysql_config    openssl  pngfix&#xA;libpng16-config  my_print_defaults  netChainSubset  perror   png-fix-itxt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I typed &lt;code&gt;netChainSubset&lt;/code&gt; and got the help page, suggesting the tool is installed and working. Interestingly, on Mac, the tool is also called &lt;code&gt;netChainSubset&lt;/code&gt; but it seems it can be called with &lt;code&gt;netchainsubset&lt;/code&gt;or &lt;code&gt;NETCHAINSUBset&lt;/code&gt; for that matter. The case doesn't seem to matter on mac. To make my code more portable, I am going to follow the case in the &lt;code&gt;bin&lt;/code&gt; of where the tool is installed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers terdon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2019-04-29T14:20:20.890" LastActivityDate="2019-04-29T14:26:15.103" Title="UCSC packages/tools ucsc-netchainsubset and ucsc-fatotwobit not working on GNU/Linux" Tags="|software-installation|ucsc|bioconda|conda|anaconda|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8546" PostTypeId="1" CreationDate="2019-04-29T14:13:30.840" Score="1" ViewCount="144" Body="&lt;p&gt;I'm running Repeatscout. I built the l-mer table called myfile.freq of myfile.fa&#xA;Can anyone tell me what do they mean the second and third columns produced as output?&#xA;here I report an example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AAAAAAAAGCGGGA  3       107776875 &#xA;AAAAAAACTGTATG  10  83440519&#xA;AAAAAAAAGGCGTA  3   41037187&#xA;AAAAAAACTTGAAT  7   94493612&#xA;CATACATGCATGCA  1065    125671338&#xA;CATACATGCTTGAA  7   121799834&#xA;AAAAAAATCATGCA  10  95493021&#xA;AAAAAAAGTCCAGT  3   125127980&#xA;AATTCACATGTATG  7   102505668&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I read the tutorial that explains how to run the program: &lt;a href=&quot;https://bix.ucsd.edu/repeatscout/readme.1.0.3.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bix.ucsd.edu/repeatscout/readme.1.0.3.txt&lt;/a&gt;. &#xA;It says: &quot;First, build_lmer_table creates a file that tabulates the frequency of all l-mers in the sequence to be analyzed&quot;&lt;/p&gt;&#xA;" OwnerUserId="4475" LastEditorUserId="4475" LastEditDate="2019-04-29T21:35:55.823" LastActivityDate="2019-04-29T21:35:55.823" Title="repeatscout freq table" Tags="|transposable-elements|repeatmasker|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8547" PostTypeId="2" ParentId="8545" CreationDate="2019-04-29T14:26:15.103" Score="1" Body="&lt;h2&gt;Short answer: respect the case of the tool name&lt;/h2&gt;&#xA;&#xA;&lt;h2&gt;Longer story below&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Thanks to terdon, I printed the PATH when the conda environment was active and listed what was in the folder where conda installed the UCSC tool.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ls /mnt/home2/miska/cr517/anaconda3/envs/ucsc-netchainsubset377/bin&#xA;c_rehash         libpng-config      mysql_config    openssl  pngfix&#xA;libpng16-config  my_print_defaults  netChainSubset  perror   png-fix-itxt&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I typed netChainSubset and got the help page, suggesting the tool is installed and working. Interestingly, on Mac, the tool is also called netChainSubset but it seems it can be called with netchainsubsetor NETCHAINSUBset for that matter. The case doesn't seem to matter on mac. To make my code more portable, I am going to follow the case in the bin of where the tool is installed.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2019-04-29T14:26:15.103" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8548" PostTypeId="1" CreationDate="2019-04-29T20:53:55.540" Score="2" ViewCount="112" Body="&lt;p&gt;I am trying to calculate p-values and ultimately FDR values for RI introns for 2 conditions of experiment I have(control and knockdown)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For every condition I have 3 biological replicates each: 3 control replicates and 3 knockdown replicates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the header line of my input file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chrom   Istart  Iend    strand  avg-L_R_counts-1    Sk_counts-1 avg-L_R_counts-2    Sk_counts-2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;-1 and -2 are 2 sample conditions control and knockout respectively&lt;/p&gt;&#xA;&#xA;&lt;p&gt;avg-L_R_counts = for a particular intron, this is the average of reads 5bp up stream and downstream of the 5' end and at the 3'end of the intron&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sk_counts = This is the number of reads spanning exon-exon junction for that intron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have approximately 350 introns(this is yeast). I can get the Percent Spliced In values but I am interested in getting a p-value and then I can calculate the FDR values.&#xA;Can anyone point me as to what approach I should take to calculate a p-value, or what test should I use?&lt;/p&gt;&#xA;" OwnerUserId="2040" LastEditorUserId="96" LastEditDate="2019-04-30T15:31:50.857" LastActivityDate="2019-10-01T10:15:25.140" Title="Calculating p-value for introns which are retained (or not) between 2 conditions" Tags="|rna-seq|rna-splicing|molecular-genetics|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8549" PostTypeId="1" AcceptedAnswerId="8800" CreationDate="2019-04-29T22:50:43.053" Score="0" ViewCount="172" Body="&lt;p&gt;I want to lift over the annotation from &lt;em&gt;E. coli&lt;/em&gt; release 29 to &lt;em&gt;E. coli&lt;/em&gt; release 42. I have followed the steps for an annotation liftover mentioned &lt;a href=&quot;http://genomewiki.ucsc.edu/index.php/Minimal_Steps_For_LiftOver&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I got the chain file for the final step and ran:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;liftOver -gff Escherichia_coli.GCA_000967155.1.gtf Chromosome.chain conversions.gtf unmapped.gtf&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I get the following warning:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;WARNING: -gff is not recommended.&#xA;Use 'ldHgGene -out=&amp;lt;file.gp&amp;gt;' and then 'liftOver -genePred &amp;lt;file.gp&amp;gt;'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to heed the warning. However, I don't know how to use the &lt;code&gt;ldHgGene&lt;/code&gt; tool. Here is its help page:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ldHgGene - load database with gene predictions from a gff file.&#xA;usage:&#xA;     ldHgGene database table file(s).gff&#xA;options:&#xA;     -bin         Add bin column (now the default)&#xA;     -nobin       don't add binning (you probably don't want this)&#xA;     -exon=type   Sets type field for exons to specific value&#xA;     -oldTable    Don't overwrite what's already in table&#xA;     -noncoding   Forces whole prediction to be UTR&#xA;     -gtf         input is GTF, stop codon is not in CDS&#xA;     -predTab     input is already in genePredTab format&#xA;     -requireCDS  discard genes that don't have CDS annotation&#xA;     -out=gpfile  write output, in genePred format, instead of loading&#xA;                  table. Database is ignored.&#xA;     -genePredExt create a extended genePred, including frame&#xA;                  information and gene name&#xA;     -impliedStopAfterCds - implied stop codon in GFF/GTF after CDS&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;From what I understand, I am trying to give the liftOver tool the annotation in &lt;a href=&quot;http://genome.ucsc.edu/FAQ/FAQformat.html#format9&quot; rel=&quot;nofollow noreferrer&quot;&gt;GenePred format&lt;/a&gt; instead of GTF format. So I need to convert the GTF format into a GenePred format and that's what the ldHgGene tool is for. So the command should be something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ldHgGene Escherichia_coli.GCA_000967155.1.gtf -out=Escherichia_coli.GCA_000967155.1.gp&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but this returns nothing. The &lt;code&gt;-out&lt;/code&gt; option does mean that the &lt;code&gt;database&lt;/code&gt; parameter is not needed but the tool still seems to need the parameter &lt;code&gt;table&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the &lt;code&gt;table&lt;/code&gt; parameter?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="96" LastEditDate="2019-04-30T12:55:55.530" LastActivityDate="2019-06-13T18:26:31.503" Title="What is the `table` parameter in the UCSU ldHgGene tool?" Tags="|format-conversion|sequence-annotation|ucsc|liftover|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8550" PostTypeId="2" ParentId="8548" CreationDate="2019-04-30T09:51:03.703" Score="1" Body="&lt;p&gt;Did you look into literature? I am not an expert in this field, but a small search on google leads me to &lt;a href=&quot;https://github.com/williamritchie/IRFinder/wiki&quot; rel=&quot;nofollow noreferrer&quot;&gt;IRFinder&lt;/a&gt; (&lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-017-1184-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;ref.&lt;/a&gt;), a tool that can assess intron retention. My advice is to see if this tool can help you, before reinventing the wheel...&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2019-04-30T09:51:03.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8552" PostTypeId="1" AcceptedAnswerId="8554" CreationDate="2019-04-30T14:52:22.037" Score="3" ViewCount="1814" Body="&lt;p&gt;Does anyone know where and how could I download a list of all human genes and the length of the coding sequence for each gene? Is it possible to do this on the NCBI site, ensembl?&lt;/p&gt;&#xA;" OwnerUserId="4504" LastEditorUserId="57" LastEditDate="2019-05-01T07:51:02.403" LastActivityDate="2019-11-02T00:48:39.777" Title="CDS length for each human gene" Tags="|gene|sequence-analysis|phylogenetics|ensembl|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8553" PostTypeId="2" ParentId="8548" CreationDate="2019-04-30T15:40:25.003" Score="0" Body="&lt;p&gt;You might take a look at the MISO algorithm for inspiration, or as a method for calculating confidence intervals of alternatively spliced isoforms: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3037023/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3037023/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-04-30T15:40:25.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8554" PostTypeId="2" ParentId="8552" CreationDate="2019-04-30T15:48:34.807" Score="4" Body="&lt;p&gt;While I haven't found a way to limit the results to the canonical transcript only, you can get a list of genes, transcripts and their CDS lengths using Ensemble's BioMart. I have already set it up for you, you can see the results, and modify them, &lt;a href=&quot;http://www.ensembl.org/biomart/martview/7bd5b4f80f2c922856260b66fd9587fa?VIRTUALSCHEMANAME=default&amp;amp;ATTRIBUTES=hsapiens_gene_ensembl.default.structure.ensembl_gene_id|hsapiens_gene_ensembl.default.structure.ensembl_transcript_id|hsapiens_gene_ensembl.default.structure.cds_length&amp;amp;FILTERS=hsapiens_gene_ensembl.default.filters.transcript_biotype.&amp;quot;protein_coding&amp;quot;|hsapiens_gene_ensembl.default.filters.biotype.&amp;quot;protein_coding&amp;quot;&amp;amp;VISIBLEPANEL=resultspanel&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (click on the &quot;Results&quot; link if you don't see them). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially, you just need to go to &lt;a href=&quot;http://www.ensembl.org/biomart/martview&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioMart&lt;/a&gt;, and &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;select &quot;Ensembl Genes 96&quot; (the number will change if the version changes) as the database and &quot;uman Genes&quot; as the dataset.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Click on &quot;Filters&quot;, and set &lt;code&gt;Gene type&lt;/code&gt; to &lt;code&gt;coding&lt;/code&gt; and &lt;code&gt;Transcript type&lt;/code&gt; to &lt;code&gt;protein_coding&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;From &quot;Attributes&quot;, select whatever you want to see. The &quot;CDS Length&quot; is under &quot;Structures&quot;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-04-30T22:23:43.663" LastActivityDate="2019-04-30T22:23:43.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8555" PostTypeId="1" CreationDate="2019-04-30T16:06:12.787" Score="2" ViewCount="266" Body="&lt;p&gt;I am struggling to use biopython to gather cross references such as GO annotation from Uniprot/swissprot text files (&quot;&lt;code&gt;DR&lt;/code&gt;&quot; in the .txt files) using biopython.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the &lt;a href=&quot;https://biopython.org/DIST/docs/api/Bio.SwissProt-pysrc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;source code&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def _read_dr(record, value): &#xA;    cols = value.rstrip(&quot;.&quot;).split('; ') &#xA;    record.cross_references.append(tuple(cols)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This leads me to think that the following code should work, as this is how other features like annotation and features work.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for record in SeqIO.parse(filename, &quot;swiss&quot;):&#xA;    print(record.cross_references) #does not work, should return a list of DR rows&#xA;    print(record.features) #works, returns a list of parsed FT rows&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="747" LastActivityDate="2019-05-06T15:44:21.613" Title="How to get Uniprot and swissprot cross references using Biopython" Tags="|python|uniprot|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8556" PostTypeId="2" ParentId="7493" CreationDate="2019-04-30T23:10:24.257" Score="3" Body="&lt;p&gt;The answer to this question depends on the aligner used to generate SAM/BAM file. For BWA method that is part of the pipeline mentioned in comments, split reads will be flagged as supplementary alignment (2048). You can modify this behavior by using option -M as described in the BWA-MEM &lt;a href=&quot;http://bio-bwa.sourceforge.net/bwa.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual page&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in case of split alignment, you will end up with multiple aligned segments that have identical query name and they are flagged as 2048. These segments may only have part of the original read sequence, but the complete read sequence is present in at least one of the segments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, there is no unique sign in CIGAR of an individual segment to identify split reads. Each aligned segment of the original read may have long deletion in its CIGAR or aligned pairs. However, without looking at all segments with the read ID it is not clear whether the segment has a novel sequence in it that doesn't align with reference or that part matches different chromosome/coordinates.&lt;/p&gt;&#xA;" OwnerUserId="1392" LastActivityDate="2019-04-30T23:10:24.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8557" PostTypeId="2" ParentId="8552" CreationDate="2019-05-01T11:33:31.283" Score="0" Body="&lt;p&gt;Ensembl has an FTP site that allows you to select and download only the coding sequences from many different genomes.&#xA;&lt;a href=&quot;https://useast.ensembl.org/info/data/ftp/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://useast.ensembl.org/info/data/ftp/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To determine the length of those sequences, download the associated gtf or gff3 annotation file. The annotation file is tab delim. The fourth and fifth column represent genomic loci of the annotated region. Subtract the amount in the fourth column from the amount in the fifth column to yield the length of all the annotated features.&#xA;You can easily do this in R after loading the file in the environment using the Magrittr library. The following code will create a new column called gene_length with the associated gene lengths.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;install.packages(&quot;magrittr&quot;)&#xA;# this only needs to be done once&#xA;library(magrittr)&#xA;# must be run each time the library is neaded&#xA;annotation.gtf &amp;lt;- read.table(&quot;path/to/annotation.gtf&quot;)&#xA;annotation.gtf$start &amp;lt;- annotation.gtf[,4]&#xA;annotation.gtf$end &amp;lt;- annotation.gtf[,5]&#xA;annotation.new-column.gtf &amp;lt;- annotation.gtf %&amp;gt;%&#xA;          mutate(gene_length=end-start)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4507" LastEditorUserId="57" LastEditDate="2019-05-01T13:03:03.197" LastActivityDate="2019-05-01T13:03:03.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8558" PostTypeId="1" AcceptedAnswerId="8559" CreationDate="2019-05-01T13:54:49.753" Score="0" ViewCount="91" Body="&lt;p&gt;I've installed Repeatscout from &lt;a href=&quot;https://bix.ucsd.edu/repeatscout/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bix.ucsd.edu/repeatscout/&lt;/a&gt;.&#xA;I've also installed TRF and nseg required by the program. &#xA;I followed the steps:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;build_lmer_table -l 16 -sequence ~/file_fasta/od_1000.fa -freq od_1000.freq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; RepeatScout -sequence od_1000.fa -output od_1000_repeats.fa -freq od_1000.freq -l 16&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the I put this command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat od_1000_repeats.fa| filter-stage-1.prl &amp;gt;od_1000_repeats_filt1.fa &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but this message appears:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bash: filter-stage-1.prl: command not found...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Why? It is all correctly installed, I can't understand the problem&lt;/p&gt;&#xA;" OwnerUserId="4475" LastActivityDate="2019-05-01T15:50:29.220" Title="Repeatscout installation problem" Tags="|software-installation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8559" PostTypeId="2" ParentId="8558" CreationDate="2019-05-01T15:50:29.220" Score="2" Body="&lt;p&gt;If you're in the RepeatScout directory you need to execute the Perl script with a &lt;code&gt;./&lt;/code&gt; prefix, like so.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat od_1000_repeats.fa | ./filter-stage-1.prl &amp;gt; od_1000_repeats_filt1.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you're not in the RepeatScout directory, you need to provide the full relative or absolute path when executing the script. Alternatively, you can put the RepeatScout directory in your &lt;code&gt;$PATH&lt;/code&gt; environmental variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are all standard UNIX/Linux concepts that many bioinformaticians take for granted, for better and for worse. If you're lost in all the terminology, a crash course in UNIX basics is probably where you would want to start.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-05-01T15:50:29.220" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8560" PostTypeId="1" AcceptedAnswerId="8563" CreationDate="2019-05-01T17:41:39.157" Score="5" ViewCount="2416" Body="&lt;p&gt;I understand what GWAS is and I'm able to perform certain tests with the p-values, etc. But what I am having a hard time wrapping my head around is what PCA on GWAS means.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So let's say I have 100,000 individuals and genotype data for 10 million SNPs. And I'm analyzing this for 10 different diseases. I also have p-value information for each of the 10 diseases for the 10 million SNPs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At my work, they were throwing around the following phrases. I'm not quite sure what exactly it means.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&quot;Perform PCA on GWAS.&quot; Does this mean, I'm performing PCA on the genotype data or p-value data? I've done PCA on classic machine learning problems like &quot;iris.&quot; But I'm not able to quite wrap my head around what PCA information tells me in terms of this genetic data. &lt;/li&gt;&#xA;&lt;li&gt;&quot;PCA on individual diseases vs. PCA on entire GWAS&quot; What exactly is the difference. I realize this is a broken statement, but they said we have two sets of PCs. One on the individual diseases which are unique, and then PCs on the entire GWAS for all 10 diseases. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Sorry if my questions is not properly asked, I realize it's probably heavily abusing notation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Added clarifications:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;So, my matrix has 100k rows and 10 million columns. Then, PC1, for example, is a vector with 100k values, each value corresponding to each individual. What exactly does PC1 value that's associated with the first individual tell me with regards to the PC1 value that's associated with 2nd individual, and so on. My current understanding if you think of a number line that represents PC1, then each of the PC values (corresponding to each individual) represents where on the number line that individual falls. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How exactly does this eliminate/account for variation in population stratification?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2519" LastEditorUserId="3848" LastEditDate="2023-07-01T01:37:37.863" LastActivityDate="2023-07-01T01:37:37.863" Title="What does PCA mean on GWAS" Tags="|phylogenetics|phylogeny|snp|gwas|pca|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8561" PostTypeId="2" ParentId="7268" CreationDate="2019-05-01T19:42:38.677" Score="1" Body="&lt;p&gt;This is a tough one, because the only relevant lines from your GSEA debug log were these two:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; &quot;Failed: empty String&quot;&#xA;DEBUG - http-outgoing-4 &amp;lt;&amp;lt; HTTP/1.1 500 Internal Server Error&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My recommendations to you at the time would've been&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;try closing all the programs, restarting them, possibly rebooting and restarting them, and even analyzing a completely different set of data to see if you get the same results, &lt;strong&gt;—or—&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;update to the latest Java 8, reinstall Cytoscape 3.7.whatever and the EnrichmentMap plugin from scratch, and re-download GSEA, &lt;strong&gt;—or—&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;lastly, try reinstalling everything on a completely different machine/OS and see if you get different results (&lt;em&gt;e.g.,&lt;/em&gt; try on a Mac or Linux if you had problems with Windows originally, or vice-versa)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I know that's lame, but sometimes that's how you solve weird computer errors, when the error message the application itself gives you is non-specific and unhelpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you had seen the error message&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;None of the gene sets have passed the filter. Try relaxing the gene set filter parameters.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;in the GSEA application logs, then you would've had the problem described in EnrichmentMap's GitHub &lt;a href=&quot;https://github.com/BaderLab/EnrichmentMapApp/issues/358&quot; rel=&quot;nofollow noreferrer&quot;&gt;issue #358&lt;/a&gt;, and the solution would be to use less stringent p- and FDR Q-value cutoffs, as prescribed in the &quot;&lt;a href=&quot;https://enrichmentmap.readthedocs.io/en/latest/Parameters.html?highlight=filter#id1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tips on Parameter Choice&lt;/a&gt;&quot; section of the &lt;a href=&quot;https://enrichmentmap.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;EnrichmentMap documentation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;My&lt;/em&gt; problem was that GSEA's default cutoffs (p-value &amp;lt; 0.005 and FDR Q-value &amp;lt; 0.1) were entirely too stringent for my results set, and using the &quot;Very permissive&quot; settings of p-value &amp;lt; 0.05 and FDR &amp;lt; 0.25 from the documentation solved it for me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully this benefits someone who ends up here after googling for &quot;Unable to create Enrichment Map:Internal Server ErrorResponseEntityProxy,&quot; which is just a stupendously unhelpful error message when &quot;None of the gene sets have passed the filter&quot; is &lt;em&gt;actually&lt;/em&gt; what went wrong. Maybe go thumbs-up &lt;a href=&quot;https://github.com/BaderLab/EnrichmentMapApp/issues/358#issuecomment-488370349&quot; rel=&quot;nofollow noreferrer&quot;&gt;this comment&lt;/a&gt; in the EnrichmentMap issue tracker, if you think the maintainers need to work on that.&lt;/p&gt;&#xA;" OwnerUserId="4191" LastEditorUserId="4191" LastEditDate="2019-05-01T19:48:06.460" LastActivityDate="2019-05-01T19:48:06.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8562" PostTypeId="1" CreationDate="2019-05-01T20:34:27.030" Score="0" ViewCount="508" Body="&lt;p&gt;I have been running PAML, via codeml, to try to identify positive selection in certain branches of my tree for one gene (but will eventually be trying to automate the analysis over multiple genes). This was done by setting &lt;code&gt;model = 2&lt;/code&gt; and &lt;code&gt;NSsites = 2&lt;/code&gt; in the codeml.ctl file. I attempted to run both the null and alternative models by setting &lt;code&gt;fix_omega = 1&lt;/code&gt; in the null.ctl and &lt;code&gt;fix_omega = 0&lt;/code&gt; in the alternative.ctl.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, after running both models, they have returned exactly identical log-likelihood (lnL) values. I wasn't expecting to find anything significant, but the fact that they were identical struck me as odd. Out of curiosity, I altered the branch of interest in the newick tree file and reattempted the analysis, but again, the lnL values remain the same between the null and alternative models (although slightly different to the value returned with the different branch selected).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have since repeated this many times, choosing a different branch(es) each time to see if I can get a difference between the null and alternative, but no luck. Am I missing something obvious? Should I really be getting the same exact result between null and alternative models, regardless of where in the tree I place &lt;code&gt;#1&lt;/code&gt;? To reiterate, I wasn't expecting to find significant results, but was at least expecting (non-significantly) different numbers for lnL...&lt;/p&gt;&#xA;" OwnerUserId="4293" LastEditorUserId="3848" LastEditDate="2019-05-02T14:26:09.427" LastActivityDate="2020-09-23T20:02:45.277" Title="PAML branch-site model: Identical lnL value between null and alternative models?" Tags="|positive-selection|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8563" PostTypeId="2" ParentId="8560" CreationDate="2019-05-01T22:15:50.357" Score="5" Body="&lt;p&gt;From my memory of what a statistician told me, a PCA aims to determine independent linear combinations of variables (i.e. genotypes) that account for the most variation in the dataset. With 10 million SNPs, the vectors describing the genotypes of all individuals reside somewhere inside a 10-million-dimension hypersphere, and the first principle component determines the best vantage point / angle from outside that sphere that will create the greatest spread of data when data is projected onto a perpendicular / orthogonal axis. The second principle component is the perpendicular / orthogonal axis that will create the second-greatest spread of data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Aside: There's some wonderful linear algebra (which I've forgotten) that makes it very quick (computationally) to do this angle calculation, exploiting the nature of linear transformations and orthogonality.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The point of the PCA for GWAS is to identify (and preferably exclude) differences that might be due to the structure of populations, rather than the disease of interest. It's fairly common for researchers to do a PCA on the entire set of genotype data, then use the first 5, 10, or 20 principal components as covariates in the association model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's an underlying assumption that the majority of genetic variation is due to population structure - if this is not the case, then the disease would have to be extremely omnigenetic, with almost the entirety of the genome contributing to the disease. The approach also won't deal with &lt;em&gt;all&lt;/em&gt; the population structure (because there may be non-linear contributions of genotypes to population structure), but it will get rid of the largest problems.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additional to this, I'd recommend the following things:&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Structure Visualisation&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Carry out a STRUCTURE analysis or PCA on the combined cases and controls, and display the results of the first two principle components, labelling, colouring, or otherwise indicating both groups. As an additional check, it may be useful to do addititional PCAs for cases and controls separately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If they appear different, it suggests that the cases and controls were drawn from separate population clusters, which should be commented on in the discussion of the paper. There's no need to panic and find more data, just make the observation.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Subsampling&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Carry out association tests on an equal-sized subsample of both datasets. This will mean that area-under-the-curve statistics can be properly interpreted. If you've got unbalanced datasets, use a little bit less than the smaller of the two groups (e.g. for 5k diseased and 95k healthy, use a random subsample of 4.5k from both groups).&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Multiple testing / Internal validation&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Run the association tests multiple times (with different subsamples), recording the individuals that were subsampled for future reference / reproducibility.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If an association is only present in one of the many different subsamples, it should be ignored as a spurious result. It means that there is some association that is not generalisable to all (or most) of the cases and controls. These variants might be interesting in themselves on an individual level, but not in a general population.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I explain some of this in more detail in the following preprint (which I'm not planning to submit to any journal, due to it being a bit stale and needing a lot of work to fix up):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1101/104497&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bootstrap Distillation: Non-parametric Internal Validation of GWAS Results by Subgroup Resampling&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2019-05-02T11:20:15.067" LastActivityDate="2019-05-02T11:20:15.067" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8564" PostTypeId="1" CreationDate="2019-05-02T01:00:44.490" Score="1" ViewCount="789" Body="&lt;p&gt;I have been trying to conduct a classic GSEA using the broad institute software and can not figure out how to compile it in R for the life of me. I am aware that there exists other packages in R that are &lt;em&gt;almost&lt;/em&gt; the same, such as &lt;code&gt;fgsea&lt;/code&gt; however, I would like to use this one first and then compare it with these other methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have already generated a ranked list for my contrast of interest and save it as a &lt;code&gt;.rnk&lt;/code&gt; file. Here is what I tried to do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gseaCallH &amp;lt;- &#xA;  paste(&quot;java -Xmx2G -cp C:/Users/omk/Desktop/BIOINFORMATIC/MsigDB/gsea2-2.2.4.jar xtools.gsea.GseaPreranked -param_file C:/Users/omk/Desktop/BIOINFORMATIC/MsigDB/GSEAPrerankedParameters.txt -rnk C:/Users/omk/Desktop/BIOINFORMATIC/MsigDB/gsea_treatFU_min.rnk -gmx C:/Users/omk/Desktop/BIOINFORMATIC/MsigDB/h.all.v6.2.symbols.gmt -rpt_label gsea_treatFU_min.rnk -out C:/Users/omk/Desktop/BIOINFORMATIC/MsigDB/C7&quot;,sep=&quot;&quot;)&#xA;&#xA;testGSEA &amp;lt;- system(command = gseaCallH, intern = TRUE, ignore.stderr = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I run the &lt;code&gt;system&lt;/code&gt; command I get the following error:&#xA;&lt;code&gt;had status 1&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At first I thought maybe it had something to do with JAVA not being properly installed however, I have installed it and still nothing. Maybe the installation was not done correctly? I have tried running it directly from cmd (I'm running windows 10) and got the following error: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error occurred during initialization of VM&#xA;Could not reserve enough space for 2097152KB object heap&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can anyone advise on how to proceed? &lt;/p&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="48" LastEditDate="2019-05-02T07:07:59.870" LastActivityDate="2019-05-03T20:11:09.613" Title="GSEA - Gene Set Enrichment Analysis - in R studio" Tags="|r|rna-seq|gsea|java|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8565" PostTypeId="2" ParentId="8562" CreationDate="2019-05-02T02:17:41.390" Score="1" Body="&lt;p&gt;The issue with PAML is that it doesn't throw many exceptions (programming jargon for trapping and reporting errors) if something is wrong, generally the &quot;exceptions&quot; are identified in the failure to report an expected value in the output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Possible explanation&lt;/strong&gt; fix_omega is not turning the omega parameter on or off it is simply defining what the value of omega will be. If the default value, which is set at .4, but can be changed, is very close to the estimated value the likelihoods will be similar - or even identical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Goto debugging&lt;/strong&gt; As computer/data scientists, the very first thing we should do is just check there is run time variation between the options you are assessing via,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;time codeml ./file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The thing to remember is fix_omega=0 is where Omega is being estimated from the data, whilst =1 means it will be supplied with a value. There should be longer run time on the &quot;0&quot; option. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Nitty gritty debugging&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1 Please check you have &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seqtype = 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If not, then that is your answer, omega was never being set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2 For debugging only make sure you have,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is global dN/dS for all the data not for a specific clade within the tree, the reason for this is given in the final step below &lt;/p&gt;&#xA;&#xA;&lt;p&gt;3, Then try a different NSsites model,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NSsites = 8&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This should produce real differences. The NSsites options are,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;ul&gt;&#xA;  &lt;li&gt;0:one w;1:neutral;2:selection; 3:discrete;4:freqs; * 5:gamma;6:2gamma;7:beta;8:beta&amp;w;9:beta&amp;gamma;&lt;/li&gt;&#xA;  &lt;li&gt;10:beta&amp;amp;gamma+1; 11:beta&amp;amp;normal&gt;1; 12:0&amp;amp;2normal&gt;1; * 13:3normal&gt;0&lt;/li&gt;&#xA;  &lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;w here is for omega&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4 The proof of your system is that when you &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fix_omega = 1&#xA;omega = .4 * try lots of different values, e.g. 0.3, 0.4, 0,7, 1, 1.5, 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This should produce likelihood variation&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Finally &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;model = 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then go to your tree file and make sure that the nomenclature used follows the manual, I think from memory the # is deployed. The output file should change to specify that you have two values for dN/dS from your treefile, thus the output from model=2 should vary from the output of model=1 if not the delineation has been ignored. If you do see two rates, great and now you are into parameter exploration. Parameter exploration is basically the options 1 to 4 described above, plus the stuff you mentioned.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Oh you mentioned using Newick format. I haven't used PAML for a while (dNdS stuff is ancient) but it it only used phylip format when I used it.  I suspect if it got the wrong tree format PAML will throw an exception, but it's worth checking.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-05-02T16:45:56.973" LastActivityDate="2019-05-02T16:45:56.973" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8566" PostTypeId="2" ParentId="8560" CreationDate="2019-05-02T02:54:51.317" Score="2" Body="&lt;p&gt;PCA = principle component analysis and a multivariate statistic, today it is trendily retermed &quot;unsupervised learning&quot; and here is likely being deployed for individuals within your data set. It works by identifying the maximum variance within multidimensional space, shearing it and describing this as the first principle component. The second principle component is sections the variance at 90o to the first in all axes (orthogonal is the posh word), and so on ... The maximum information is within the first two components and they are sometimes plotted on x and y axes to look for potential clusters within your data. In this case the first few components are all that is needed because you not identifying the genes of interest just checking the data looks sound.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Point 1&lt;/strong&gt;&#xA;PCA on all GWAS is attempting to assess that the genetic variation between diseased and healthy people is evening spread across the genome. For example if your diseased population came from e.g. the indigenous people of Western Samoa (purely as an example), which has a very defined genetic heritage (at least for HLA) then most of the SNPs will be unrelated to the disease your are attempting to identify. This will mean you have to redefine the healthy population, the null distribution, to screen out the artefacts. In this case a better representation would be to select healthy individuals from Western Samoa as your comparison. So within the first two components you want to see lots of mixing between your patients vs healthy people. If the patients are clumping (clustering) thats bad here and you have to find out why. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PCA has weaknesses BTW for complicated reasons which are boring, but it is often not assessed directly but via a tSNE analysis and then plotted on two axes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Point 2&lt;/strong&gt; &#xA;If PCA from point 1 looks cool, then you might split the data set into diseased and another one of healthy and perform a PCA on each but now look at the SNPs ... not the people. I hope that makes sense. If you compare the PCA plot of SNPs between these two groups, the differences MIGHT and I stress might highlight the SNPs of interest, i.e. the SNPs causing your disease. However and its a big however the number of SNPs you are dealing with is huge and if you are manually going through all that data to assess anomalous clusters, or outliers to make a list of candidate SNPs for further analysis (@gringer describes some of the further options above) -  I wish you good health and long life, and you will need it because you will be very old at the end of that optical analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you really do is an additional analysis (I would deploy tSNE) to assess the differences between the two PCA plots and end up with a third plot indicating the differences between all the other PCA plots. In this context it is complicated to deploy and would be nested within a tSNE analyse of all components which is what I alluded to in point 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Point 2 - issues&lt;/strong&gt; Generally if someone thought I was going to eye-ball 50000 SNPs between two graphs, or throw them back a reanalysis which conceptually made their brain hurt ... their gonna have conceptualise real hard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; Having said all that PCA is merely an exploratory tool and is not the defining significance test required, to find the &quot;one SNP&quot; which will explain your disease. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The imbalance of the data set makes advanced SNP identification statistically difficult, but the 30:70 split is okay. However a subsample of the healthy population, which is what Point 1 is all about, would likely correct for this imbalance.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;On the question of mixing Point 1, what you don't want to see is your diseased patients forming a &quot;blob&quot;, or even a circle, or any sort of defined cluster against the spread of healthy patients on a scatterplot of PC1 Vs PC2. Perhaps I should have said if your patients don't cluster on the output to Point 1 that's good. There are statistics that identify clusters on a PC1 PC2 scatter plot, and you want it to fail that test.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-05-02T06:10:41.577" LastActivityDate="2019-05-02T06:10:41.577" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8568" PostTypeId="1" AcceptedAnswerId="8569" CreationDate="2019-05-02T09:51:01.940" Score="0" ViewCount="32" Body="&lt;p&gt;I have called and annotated structural variants and I have a table by name of samples and structural events like below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sample  Type&#xA;t_005   DEL&#xA;t_005   INS&#xA;t_005   DEL&#xA;t_005   DEL&#xA;t_005   DEL&#xA;t_005   DEL&#xA;t_005   DUP&#xA;t_005   DUP&#xA;t_005   DEL&#xA;t_005   DEL&#xA;t_005   DEL&#xA;t_005   DEL&#xA;t_005   DUP&#xA;t_005   BND&#xA;t_005   DEL&#xA;t_005   DEL&#xA;t_005   BND&#xA;t_005   DEL&#xA;t_005   DEL&#xA;t_005   DUP&#xA;t_005   DEL&#xA;t_005   DEL&#xA;t_005   DEL&#xA;t_005   DEL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I count the number of events to get the below table?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(SV_Counts[,1:4])&#xA;               t_005      o_005       t_013         o_013&#xA;    DEL          27          40          25          30&#xA;    DUP          11          10          13          22&#xA;    BND          23          20          10          12&#xA;    TRA           3           2           4           4&#xA;    &amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried this from google but I failed to adapt my data to that&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    &amp;gt; melt(table(a))&#xA;   Sample Type value&#xA;1   o_036  BND   280&#xA;2   o_005  BND     8&#xA;3   o_013  BND   160&#xA;4   o_021  BND   144&#xA;5   o_036  BND     0&#xA;6   t_005  BND    15&#xA;7   t_013  BND   108&#xA;8   o_036  DEL    63&#xA;9   o_005  DEL    15&#xA;10  o_013  DEL   121&#xA;11  o_021  DEL   243&#xA;12  o_036  DEL     0&#xA;13  t_005  DEL    40&#xA;14  t_013  DEL    76&#xA;15  o_036  DUP    34&#xA;16  o_005  DUP     4&#xA;17  o_013  DUP    57&#xA;18  o_021  DUP    88&#xA;19  o_036  DUP     0&#xA;20  t_005  DUP    14&#xA;21  t_013  DUP    48&#xA;22  o_036  INS    58&#xA;23  o_005  INS     0&#xA;24  o_013  INS     4&#xA;25  o_021  INS    10&#xA;26  o_036  INS     1&#xA;27  t_005  INS     1&#xA;28  t_013  INS     2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But hereafter I don't know what to do&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="298" LastEditDate="2019-05-02T12:01:47.793" LastActivityDate="2019-05-02T12:01:47.793" Title="Generate table for total number of SV events per sample" Tags="|r|structural-variation|wgs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8569" PostTypeId="2" ParentId="8568" CreationDate="2019-05-02T10:22:40.117" Score="2" Body="&lt;p&gt;So you have data in &quot;long format&quot; and want to convert it to &quot;wide format&quot;. You can use &lt;code&gt;spread()&lt;/code&gt; from &lt;code&gt;tidyr&lt;/code&gt; for that:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;require(tidyr)&#xA;# Assuming your input is called `foo`&#xA;spread(foo, Sample, value)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-05-02T10:22:40.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8570" PostTypeId="2" ParentId="7498" CreationDate="2019-05-02T13:00:02.367" Score="1" Body="&lt;p&gt;It turns out that this was just a bug which has been fixed by &lt;a href=&quot;https://github.com/dellytools/delly/commit/0fed1a17d0df1a085bdc9af73ae768bd74aab10a&quot; rel=&quot;nofollow noreferrer&quot;&gt;this commit&lt;/a&gt;. The current &lt;code&gt;master&lt;/code&gt; version of delly no longer exhibits this behavior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are facing this issue, either build the current &lt;code&gt;master&lt;/code&gt; version or wait a bit for the next release which should also include this fix.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-05-02T13:00:02.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8571" PostTypeId="2" ParentId="8564" CreationDate="2019-05-02T13:51:35.800" Score="2" Body="&lt;p&gt;Are you calling java with sufficient memory? Try increasing &lt;code&gt;-Xmx2G&lt;/code&gt; to &lt;code&gt;-Xmx8G&lt;/code&gt;, provided your PC has 8 GB RAM.&lt;/p&gt;&#xA;" OwnerUserId="3818" LastActivityDate="2019-05-02T13:51:35.800" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8572" PostTypeId="1" CreationDate="2019-05-02T14:58:20.150" Score="3" ViewCount="315" Body="&lt;p&gt;I was wondering if anyone had any experience 'counting' the frequency of a particular viral sequence in an individual's fastq sequence. So basically, counting the number of occurrences of a particular sequence given someone's fastq file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing is, we must speak with the sequencing company regarding what format we'd like the files and must choose between &lt;code&gt;.bam&lt;/code&gt; and &lt;code&gt;.fastq&lt;/code&gt; - both would be extra $. Usually I'd choose &lt;code&gt;.bam&lt;/code&gt; out of convenience but I was wondering, given that it has already been aligned the human genome I might not be able to proceed with this 'counting' process. So, my question is: if I were to use &lt;code&gt;samtools&lt;/code&gt; - or any other program - to convert the &lt;code&gt;.bam&lt;/code&gt; files to &lt;code&gt;.fastq&lt;/code&gt;, is there any possibility that I could lose such information - e.g. viral seqs - that would have otherwise been available should I have just gotten the &lt;code&gt;.fastq&lt;/code&gt; files directly? &lt;/p&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="3848" LastEditDate="2020-11-27T19:24:03.840" LastActivityDate="2020-11-27T19:24:03.840" Title="Frequency of specific viral sequence in .BAM or .fastq" Tags="|sequence-alignment|samtools|fastq|covid-19|sars-cov-2|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8573" PostTypeId="1" AcceptedAnswerId="8586" CreationDate="2019-05-02T15:12:10.157" Score="1" ViewCount="253" Body="&lt;p&gt;I am using WGCNA and code has worked with no problems with some of my datasets however with most recent attempt when completing gene network &amp;amp; identification of modules. I get following error code... &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;net__30= blockwiseModules(t5, power=6,&#xA;                              TOMType = &quot;signed&quot;, minModuleSize=30,&#xA;                              reassignThreshold =0, mergeCutHeight = 0.25,&#xA;                              numericLabels = TRUE, pamRespectsDendro = FALSE,&#xA;                              saveTOMs = TRUE,&#xA;                              saveTOMFileBase = &quot;PBCreponseTOMB&quot;,&#xA;                              verbose= 3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I come up with error code saying: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;            Number of items to replace is not a multiple of replacement length.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does anyone please know what this indicts to? I have looked on all the WGCNA tutorial pages but cannot find this error message anywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you kindly &lt;/p&gt;&#xA;" OwnerUserId="4398" LastActivityDate="2019-05-28T00:09:21.440" Title="WGCNA error codes in Network Construction &amp; moduel detection" Tags="|r|wgcna|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="8574" PostTypeId="2" ParentId="8572" CreationDate="2019-05-02T15:24:24.787" Score="3" Body="&lt;p&gt;If you are interested in a particular viral sequence, it seems wasteful to request bams that are aligned to the human genome, but you could request bams aligned to the human genome with unaligned reads retained. As an alternative, you can generate unaligned bams, where all of the same reads are present that would be present in a fastq but in a bam format. You can revert this to a fastq or align the fastq directly to a viral reference yourself. Aligning to small references is fast. &lt;/p&gt;&#xA;" OwnerUserId="272" LastEditorUserId="272" LastEditDate="2019-05-02T15:49:54.820" LastActivityDate="2019-05-02T15:49:54.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8575" PostTypeId="2" ParentId="8572" CreationDate="2019-05-02T15:24:34.007" Score="6" Body="&lt;p&gt;Most read aligners will report unaligned reads as well, which presumably will include your viral sequences. I would ask them to formally confirm that the BAM files will contain unaligned reads before choosing that option.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-05-02T15:24:34.007" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8576" PostTypeId="1" CreationDate="2019-05-02T15:43:07.097" Score="-4" ViewCount="109" Body="&lt;p&gt;I have called structural variants and now I have&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sample  Type    Chrom1   Start1        End1 Strand1&#xA;t_005   DEL 1   28677074    28677079    +&#xA;t_005   INS 1   186383731   186383738   +&#xA;t_005   DEL 1   245902589   245902590   +&#xA;t_005   DEL 2   56345643    56345645    +&#xA;t_005   DEL 3   59766214    59766217    +&#xA;t_005   DEL 3   60270545    60270548    +&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have an annotation file like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(sv[,1:5])&#xA;  ENSG00000233440 13 23708313 23708703   HMGA1P6&#xA;1 ENSG00000207157 13 23726725 23726825    RNY3P4&#xA;2 ENSG00000229483 13 23743974 23744736 LINC00362&#xA;3 ENSG00000229483 13 23743974 23744736 LINC00362&#xA;4 ENSG00000252952 13 23791571 23791673  RNU6-58P&#xA;5 ENSG00000235205 13 23817659 23821323  TATDN2P3&#xA;6 ENSG00000232849 13 93708910 93710179 LINC00363&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How I could relate each gene name from annotation file to my genomic range file?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;EDITED&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sample  Type    Chrom1  Start1  End1    Strand1 Chrom2  Start2  End2    Strand2&#xA;t_005   DEL 1   28677074    28677079    +   1   28706324    28706329    -&#xA;t_005   INS 1   186383731   186383738   +   1   186383731   186383732   -&#xA;t_005   DEL 1   245902589   245902590   +   1   246007384   246007385   -&#xA;t_005   DEL 2   56345643    56345645    +   2   56345829    56345830    -&#xA;t_005   DEL 3   59766214    59766217    +   3   59785325    59785328    -&#xA;t_005   DEL 3   60270545    60270548    +   3   60705591    60705594    -&#xA;t_005   DUP 3   60744344    60744345    -   3   60806306    60806307    +&#xA;t_005   DUP 3   60882109    60882112    -   3   60934155    60934158    +&#xA;t_005   DEL 3   77973091    77973093    +   3   77991398    77991400    -&#xA;t_005   DEL 3   95972796    95972800    +   3   95974511    95974515    -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-05-02T16:49:59.737" LastActivityDate="2019-05-02T16:49:59.737" Title="Finding matched genes for each genomic range" Tags="|r|structural-variation|wgs|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8578" PostTypeId="2" ParentId="8576" CreationDate="2019-05-02T16:17:02.273" Score="3" Body="&lt;p&gt;You received an answer from Kamil for a similar question here: &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/8532/650&quot;&gt;https://bioinformatics.stackexchange.com/a/8532/650&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can adapt the code they gave you to compare &lt;code&gt;Chrom1 == sv[2] &amp;amp; Start1 &amp;gt;= sv[3] &amp;amp; End1&amp;lt;=sv[4]&lt;/code&gt; to find events that happen within a gene. For events spanning genes, you're going to need to write a collapsing function for the gene names and a different way to compare overlaps instead of &lt;code&gt;Start &amp;gt;= sv[3] &amp;amp; End &amp;lt;= sv[4]&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2019-05-02T16:17:02.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8580" PostTypeId="2" ParentId="7499" CreationDate="2019-05-03T13:03:04.327" Score="0" Body="&lt;p&gt;Depends on the software to generate the segmentation file;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example I have used scatngs for segmentation that for sample returns &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Segment number&#xA;Chromosome&#xA;Start position (origin-1)&#xA;End position (origin-1)&#xA;Major copy number—normal&#xA;Minor copy number—normal&#xA;Major copy number—tumor&#xA;Minor copy number—tumor&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;By  log2(Major copy number—tumor +  Minor copy number—tumor /ploidy of each sample) you would get segment mean&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For number of probes &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;The number of probes depends on where the data came from.&#xA;1) From SNP arrays:  Number of array probes within each segment.&#xA;2) From exon capture data:   Probes are the ends of exons and so the number of probes are equal to the number of exons  X 2.&#xA;3) From whole genome sequencing:  Probes are the boundaries of bins used to generate the copy number data and so the number of probes are equal to the number of bins + &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For example in my case for scatngs&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Each SNP is a marker, and so you have to count how many of these SNPs are on each segment&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have asked this question in &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/8531/extracting-the-number-snp-in-each-range?noredirect=1#comment11314_8531&quot;&gt;Extracting the number SNP in each range&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2019-05-03T13:03:04.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8581" PostTypeId="1" CreationDate="2019-05-03T13:46:43.143" Score="3" ViewCount="211" Body="&lt;p&gt;I'm planning to build a transcriptome by pooling all existing transcriptomes in SRA for a non-model species (which has no reference genome) to study differentially expressed genes and the like. It might worth mentioning that the transcriptomes I want to pool mostly come from different tissues and sampling sites. These transcriptomes would come from five SRA BioProjects plus one project from my laboratoryfor a total of 102 single transcriptomes coming from about 20 individuals (plus three pooled larvae transcriptomes). I'm especially interested in differentially expressed genes in different tissues, on genes specific to different tissues and on the UTRs of differentially expressed genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My idea is to build a common reference transcriptome against which I'd re-assemble the individual transcriptomes, and only then I'd get differential expression data etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm looking for literature (or personal experience) dealing with assemblies of transcriptomes / comparing transcriptomes coming from different datasets in order to understand whether this is a &lt;strong&gt;good idea or not&lt;/strong&gt; and get a grasp of what &lt;strong&gt;technical and biological problems&lt;/strong&gt; I could face. I'm (vaguely) aware of the batch effect and of &lt;a href=&quot;https://rdrr.io/bioc/sva/man/ComBat.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;one way to remove it&lt;/a&gt;, but I'd like to read papers dealing with my same problem (different SRA databases, without reference genomes). Up until now I found just projects dealing with microarray data.&lt;/p&gt;&#xA;" OwnerUserId="1637" LastEditorUserId="1637" LastEditDate="2019-05-13T14:23:58.447" LastActivityDate="2019-05-14T12:46:05.133" Title="Merging transcriptomes coming from different experiments" Tags="|database|differential-expression|transcriptome|literature-search|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="8582" PostTypeId="2" ParentId="8564" CreationDate="2019-05-03T18:52:18.090" Score="3" Body="&lt;p&gt;I have answered my own question however, it is specific to windows users.. If you get the same error from the &lt;code&gt;cmd&lt;/code&gt; as I did, you must manually put in environment variables related to Java. Do as the user Mohamed did in this SO post: &lt;a href=&quot;https://stackoverflow.com/questions/9303889/error-occurred-during-initialization-of-vm-could-not-reserve-enough-space-for&quot;&gt;https://stackoverflow.com/questions/9303889/error-occurred-during-initialization-of-vm-could-not-reserve-enough-space-for&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following this, you &lt;strong&gt;must avoid&lt;/strong&gt; using paths such as &lt;code&gt;C:/absolute_path&lt;/code&gt; and instead use &lt;code&gt;/absolute_path&lt;/code&gt;, if you do the former, you will get the exception: &lt;code&gt;java.net.MalformedURLException: unknown protocol: c&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the end, after manually entering the environment variables and running the following command: &lt;code&gt;java -Xmx2G -cp /Users/omk/Desktop/BIOINFORMATIC/MsigDB/gsea2-2.2.4.jar xtools.gsea.GseaPreranked -param_file /Users/omk/Desktop/BIOINFORMATIC/MsigDB/GSEAPrerankedParameters.txt -rnk /Users/omk/Desktop/BIOINFORMATIC/MsigDB/gsea_treatFU_min.rnk -gmx /Users/omk/Desktop/BIOINFORMATIC/MsigDB/h.all.v6.2.symbols.gmt -rpt_label gsea_treatFU_min.rnk -out /Users/omk/Desktop/BIOINFORMATIC/MsigDB/C7&lt;/code&gt; it has worked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where the command can be executed directly from the command line or in R using the &lt;code&gt;system&lt;/code&gt; function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: A '#' separated by a &lt;em&gt;space&lt;/em&gt; must be present at the beginning of your &lt;code&gt;.rnk&lt;/code&gt; file for the program to work. Therefore, &lt;code&gt;.rnk&lt;/code&gt; files must have the following format:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;# Genes  Coeffecients&#xA;TSPAN6    0.2345676&lt;/code&gt; &#xA;etc. for every gene.&lt;/p&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="4001" LastEditDate="2019-05-03T20:11:09.613" LastActivityDate="2019-05-03T20:11:09.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8583" PostTypeId="1" AcceptedAnswerId="8587" CreationDate="2019-05-04T10:41:49.673" Score="2" ViewCount="705" Body="&lt;p&gt;I have thousands of PDB files and I need to extract the following information from each of them: an ID or the name of the small molecule (ligand) present in the complex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to do that with BioPython or similar tools? Taking &lt;a href=&quot;http://www.rcsb.org/structure/5TO8&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; PDB entry as an example, the tools should return 7FM.&lt;/p&gt;&#xA;" OwnerUserId="2862" LastEditorUserId="2862" LastEditDate="2019-05-04T17:21:16.483" LastActivityDate="2023-10-27T22:22:43.167" Title="Retrieve ID ligand from PDB file" Tags="|python|pdb|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8584" PostTypeId="1" CreationDate="2019-05-04T10:41:57.833" Score="2" ViewCount="190" Body="&lt;p&gt;I am currently working on a Computer Science project where we are trying to build a large set of orthogonal single-stranded DNA sequences. The goal would be to ensure that when put in solution, the ssDNA do not interact with each other to form double-stranded DNA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope that this makes sense, but my knowledge  biological understanding is developing because I have a CS background. Please excuse any inadvertent errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As one of the first steps, we used NUPACK to predict the percent of the resulting AB complex, given two single-stranded sequences A, B in concentration 1μM each. We would use the concentration of AB to classify if the strands are likely to form double-stranded DNA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, we cannot use NUPACK for large sets, as we would need to do the above check against each of the other sequences, which is a quadratic problem. For a set of size 30,000 we would have to do 450 million NUPACK computations which is simply unfeasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I turned to the idea of approximating NUPACK. &lt;a href=&quot;https://www.pnas.org/content/115/10/E2183&quot; rel=&quot;nofollow noreferrer&quot;&gt;In this paper&lt;/a&gt;, it is claimed that &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Ideally, the thermodynamics of hybridization between a candidate&#xA;  probe and potential off-target sites would be modeled in&#xA;  silico and employed as a means of identifying probe oligos likely&#xA;  to only bind their intended targets in a given set of reaction&#xA;  conditions. Although powerful utilities such as NUPACK (52–&#xA;  54) are capable of performing such simulations, the limited&#xA;  throughput of these programs renders a direct thermodynamic&#xA;  approach impractical for genome-scale probe design. However,&#xA;  we hypothesized that features in rapidly calculated data such as&#xA;  alignment scores may be predictive of thermodynamic behavior&#xA;  and could therefore serve as a proxy for the information that&#xA;  would be produced by thermodynamic simulations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xb15Q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xb15Q.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The alignment scores are then used to predict the interaction. &lt;a href=&quot;http://bowtie-bio.sourceforge.net/bowtie2/index.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;bowtie2&lt;/a&gt; is used in the paper for alignment purposes. However, the context is different as their computation is done on genome information. Very sensitive settings of bowtie2 seem to be slow (hours) on my laptop and result in enormous files (tens of GB). I used less sensitive settings as a compromise but it seems to negatively affect the LDA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking at alternative applications for alignment, but there are many alternatives (some of the more promising ones so far, in my opinion: &lt;a href=&quot;http://snap.cs.berkeley.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SNAP&lt;/a&gt;, &lt;a href=&quot;https://github.com/soedinglab/MMseqs2&quot; rel=&quot;nofollow noreferrer&quot;&gt;MMseqs2&lt;/a&gt;, &lt;a href=&quot;https://github.com/felipealbrecht/Genoogle&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genoogle&lt;/a&gt;). Would any of these, or others not mentioned, be better suited for my purpose?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also any other advice or suggestions are welcome!&lt;/p&gt;&#xA;" OwnerUserId="4527" LastEditorUserId="4527" LastEditDate="2019-05-04T12:16:56.223" LastActivityDate="2019-06-03T13:04:10.530" Title="Alignment for predicting DNA hybridization?" Tags="|sequence-alignment|fasta|blast|phylogenetics|thermodynamics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8585" PostTypeId="2" ParentId="8584" CreationDate="2019-05-04T12:27:46.937" Score="1" Body="&lt;p&gt;As a rough and ready and very easy to perform test I would put the entire universe of dsDNA through Primer3 (details below). This could be done by &quot;splicing&quot; your probe of interest onto each &quot;chunk&quot; of dsDNA you pass through primer3 and restrict the primer search to the 20bp primer you are interested in. In other words you are forcing the program to consider the internal and trans-hybridisation of every possible dsDNA against your probe. If you probe is not recovered from a given search you would flag this on a subsequent parsing script and then that &quot;chunk&quot; of DNA would be assessed further. All the components required are in the link below. You would need to provide your own wrapper, but its fairly easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Primer3 will assess the internal thermodynamics of your probe and it has loads of options to assess the full range of behaviour, for example 3' hybridisation (which could be extended because you want 5' to 3'). It would be easy to construct a parameter grid around all the parameters of interest. Ultimately it would provide an additional layer of certainty, and everyone knows and trusts this algorithm (its also written in C, so its fast). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The description of all the components is &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/8543/directed-mutagenesis-primer-design-by-back-to-back-pcr-of-constructs&quot;&gt;here&lt;/a&gt; , my specific purpose for customising the pipeline is different.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Bowtie2 will take a lot of RAM, I am not sure I would use it for this purpose, particularly if RAM is limiting (its used for genome assemblies against a reference). I'd ask the users whether Blat would be useful, because that is friendlier to the calculation you are performing. Anyway there some heavy duty Bowtie2 users around and their input would be useful.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-05-04T12:47:44.943" LastActivityDate="2019-05-04T12:47:44.943" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8586" PostTypeId="2" ParentId="8573" CreationDate="2019-05-04T16:40:15.360" Score="2" Body="&lt;p&gt;Looks like a bug in the code which I will try to hunt down and fix. In the meantime, I would suggest playing with maxBlockSize argument to blockwiseModules. Try increasing it as much as your available RAM allows (see the paragraph &quot;A second word of caution concerning block size&quot; on page 6 of &lt;a href=&quot;https://horvath.genetics.ucla.edu/html/CoexpressionNetwork/Rpackages/WGCNA/Tutorials/FemaleLiver-02-networkConstr-blockwise.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;WGCNA tutorial I, section 2c&lt;/a&gt;, for some guidance on how to set maxBlockSize). Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2019-05-04T16:40:15.360" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8587" PostTypeId="2" ParentId="8583" CreationDate="2019-05-05T17:25:51.487" Score="2" Body="&lt;p&gt;I found out that the package &lt;a href=&quot;https://github.com/williamgilpin/pypdb&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyPDB&lt;/a&gt; has the function &lt;code&gt;get_ligands&lt;/code&gt; that does exactly what I'm looking for.&lt;/p&gt;&#xA;" OwnerUserId="2862" LastActivityDate="2019-05-05T17:25:51.487" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8588" PostTypeId="1" AcceptedAnswerId="8594" CreationDate="2019-05-05T22:01:06.833" Score="2" ViewCount="66" Body="&lt;p&gt;I am fairly new to genomics and population genetics. And I am reading this article 'Demographic history and rare allele sharing among human populations':&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3142009/pdf/pnas.201019276.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3142009/pdf/pnas.201019276.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this article they propose a demographic model in Figure 4 on page 11986. However, I am confused how does African Americans fit in this demographic model that are known to have both African and European ancestry. And if I am to consider African American population than what is the split time between the two populations, time they merge into admixed population and the admixed proportion. &#xA;My apologies if it's a really basic question but, any insights would be appreciated. &lt;/p&gt;&#xA;" OwnerUserId="4514" LastActivityDate="2019-05-07T13:16:04.417" Title="Demographic model for admixed African Americans" Tags="|phylogenetics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8589" PostTypeId="1" AcceptedAnswerId="8595" CreationDate="2019-05-06T02:39:46.047" Score="4" ViewCount="1471" Body="&lt;p&gt;Is there a paper or web page describing the procedure for creating the nr database used by NCBI's BLAST implementation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I presume it's some type of clustering, but I'm curious about how exactly sequences are condensed into non-redundant representatives.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2020-12-30T19:55:29.377" Title="How is BLAST's nr database created?" Tags="|blast|makeblastdb|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8590" PostTypeId="2" ParentId="8589" CreationDate="2019-05-06T08:39:33.353" Score="4" Body="&lt;p&gt;The Refseq team and also the NCBI resource coordinators team publish a new paper every few years, so check out the many papers (e.g. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5753331/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; or &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4383943/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), but to answer your 2nd question, non-redundancy here is (I think) defined very strictly as proteins that are identical in terms of sequence and length, so the clustering is trivial, without the need for a sophisticated clustering algorithm as required to detect more remote homologs.&lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="104" LastEditDate="2019-05-06T16:35:55.953" LastActivityDate="2019-05-06T16:35:55.953" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8592" PostTypeId="2" ParentId="8555" CreationDate="2019-05-06T15:44:21.613" Score="3" Body="&lt;p&gt;I believe the attribute you should be looking for is &lt;code&gt;dbxrefs&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[x for x in record.dbxrefs if x.startswith('GO:')]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Apparently &lt;code&gt;SeqIO&lt;/code&gt; handles uniprot text files differently than the &lt;code&gt;SwissProt&lt;/code&gt; module:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SwissProt&#xA;record = SwissProt.read(open('single_record_from_uniprot.txt'))&#xA;record.cross_references&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So you should change you code to use the &lt;code&gt;SwissProt&lt;/code&gt; module to get the behavior you expect&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2019-05-06T15:44:21.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8593" PostTypeId="1" AcceptedAnswerId="8617" CreationDate="2019-05-06T20:27:44.650" Score="1" ViewCount="1175" Body="&lt;p&gt;With tabix one can index a VCF file for rapid variant retrieval based on genomic position. I'm wondering if there are any tools that will index a VCF file to allow rapid retrieval using rsIDs and/or other metadata? I'm aware of awk/grep/vcftools one-liners for this purpose, but I'd like to avoid scanning a huge VCF each time I need to retrieve the coordinates of a new batch of rsIDs.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2023-03-02T15:01:47.590" Title="Index VCF file for rapid access by rsID?" Tags="|vcf|file-formats|variation|indexing|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8594" PostTypeId="2" ParentId="8588" CreationDate="2019-05-07T10:06:17.730" Score="1" Body="&lt;p&gt;Just to further my answer, the problem with molecular dating within a species (I don't do humans [excepting immunology], but it's the same theory), is it requires an assumption of clonality (bifurcating trees). Mendelian species recombine towards panmixia, which will make any attempt to date via de novo point mutation, appear far younger than they really are, or for panmixia, incalculable (it breaks the central assumption). Dating through admixture requires a lot of extra data and/or assumptions for example the generation time,  putative admixture per generation, and shifts in population dynamics. Maybe you can calculate some of those requirements but  alot will remain assumptions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not read the paper you suggested, because I'm not involved in human genetics [outside immunology], but it is possible the authors may (and I stress may) have simplified an assumption that could be difficult to estimate. It is not clear what the objective of the paper was.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Quantitative analysis is extremely important in modern population genetics/ evolutionary theory because historically it was overlooked and resulted in lots of hypothese which were difficult to test. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-05-07T13:16:04.417" LastActivityDate="2019-05-07T13:16:04.417" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8595" PostTypeId="2" ParentId="8589" CreationDate="2019-05-07T16:35:23.487" Score="2" Body="&lt;p&gt;Did a little more searching and found the answer in the README on BLAST's ftp site: &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/blast/db/README&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ncbi.nlm.nih.gov/blast/db/README&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;6. Non-redundant defline syntax&#xA;&#xA;The non-redundant databases are nr, nt and pataa. Identical sequences are &#xA;merged into one entry in these databases. To be merged two sequences must&#xA;have identical lengths and every residue at every position must be the &#xA;same.  The FASTA deflines for the different entries that belong to one &#xA;record are separated by control-A characters invisible to most &#xA;programs. In the example below both entries Q57293.1 and AAB05030.1&#xA;have the same sequence, in every respect:&#xA;&#xA;&amp;gt;Q57293.1 RecName: Full=Fe(3+) ions import ATP-binding protein FbpC ^AAAB05030.1 afuC &#xA;[Actinobacillus pleuropneumoniae] ^AAAB17216.1 afuC [Actinobacillus pleuropneumoniae]&#xA;MNNDFLVLKNITKSFGKATVIDNLDLVIKRGTMVTLLGPSGCGKTTVLRLVAGLENPTSGQIFIDGEDVTKSSIQNRDIC&#xA;IVFQSYALFPHMSIGDNVGYGLRMQGVSNEERKQRVKEALELVDLAGFADRFVDQISGGQQQRVALARALVLKPKVLILD&#xA;EPLSNLDANLRRSMREKIRELQQRLGITSLYVTHDQTEAFAVSDEVIVMNKGTIMQKARQKIFIYDRILYSLRNFMGEST&#xA;ICDGNLNQGTVSIGDYRFPLHNAADFSVADGACLVGVRPEAIRLTATGETSQRCQIKSAVYMGNHWEIVANWNGKDVLIN&#xA;ANPDQFDPDATKAFIHFTEQGIFLLNKE&#xA;&#xA;Individual sequences are now identifed simply by their accession.version.  &#xA;&#xA;For databases whose entries are not from official NCBI sequence databases, &#xA;such as Trace database, the gnl| convention is used. For custom databases, &#xA;this convention should be followed and the id for each sequence must be &#xA;unique, if one would like to take the advantage of indexed database, &#xA;which enables specific sequence retrieval using blastdbcmd program included &#xA;in the blast executable package.  One should refer to documents &#xA;distributed in the standalone BLAST package for more details.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Landed on that README from this question on biostars.org: &lt;a href=&quot;https://www.biostars.org/p/217456/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/217456/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In that same README file is some information on the origin of the sequences in the non-redundant sets:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;+-----------------------+-----------------------------------------------------+&#xA;|File Name              | Content Description                                 |&#xA;+-----------------------+-----------------------------------------------------+&#xA;nr.gz*                  | non-redundant protein sequence database with entries&#xA;                           from GenPept, Swissprot, PIR, PDF, PDB, and RefSeq&#xA;nt.gz*                  | nucleotide sequence database, with entries from all&#xA;                          traditional divisions of GenBank, EMBL, and DDBJ;&#xA;                          excluding bulk divisions (gss, sts, pat, est, htg)&#xA;                          and wgs entries. Partially non-redundant.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9" LastEditorUserId="9" LastEditDate="2020-12-30T19:55:29.377" LastActivityDate="2020-12-30T19:55:29.377" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8596" PostTypeId="1" CreationDate="2019-05-07T16:47:03.563" Score="3" ViewCount="63" Body="&lt;p&gt;I am interested in looking at germline copy number variations in individuals that are at high risk of developing cancer. Are there any databases where I can look, if a CNV seen in our test case has been seen/reported earlier in a patient with cancer?&#xA;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="4537" LastEditorUserId="96" LastEditDate="2019-05-07T18:43:29.453" LastActivityDate="2019-05-07T18:43:29.453" Title="Database for germline copy number variations in cancer" Tags="|variant-calling|variation|cancer|cnv|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8597" PostTypeId="2" ParentId="8596" CreationDate="2019-05-07T17:22:21.730" Score="2" Body="&lt;p&gt;You can find CNV (and other structural variant) information in:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://decipher.sanger.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DECIPHER&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;DECIPHER is used by the clinical community to share and compare phenotypic and genotypic data. The DECIPHER database contains data from 29603 patients who have given consent for broad data-sharing; DECIPHER also supports more limited sharing via consortia. &lt;a href=&quot;https://decipher.sanger.ac.uk/about#stats&quot; rel=&quot;nofollow noreferrer&quot;&gt;Have a look at the numbers&lt;/a&gt;.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Anyone can browse publicly-available patient data on DECIPHER and request to be put in contact with the responsible clinician. Why? &lt;a href=&quot;https://decipher.sanger.ac.uk/about#overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;Because sharing benefits everyone&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://exac.broadinstitute.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ExAC&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;http://exac.broadinstitute.org/about&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Exome Aggregation Consortium (ExAC)&lt;/a&gt; is a coalition of investigators seeking to aggregate and harmonize exome sequencing data from a wide variety of large-scale sequencing projects, and to make summary data available for the wider scientific community.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The data set provided on this website spans 60,706 unrelated individuals sequenced as part of various disease-specific and population genetic studies. The ExAC Principal Investigators and groups that have contributed data to the current release are listed &lt;a href=&quot;http://exac.broadinstitute.org/about&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;ExAC has included CNV calls since August 2016.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://www.clinvar.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ClinVar&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;ClinVar is a freely accessible, public archive of reports of the relationships among human variations and phenotypes, with supporting evidence. ClinVar thus facilitates access to and communication about the relationships asserted between human variation and observed health status, and the history of that interpretation. ClinVar processes submissions reporting variants found in patient samples, assertions made regarding their clinical significance, information about the submitter, and other supporting data. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://dgv.tcag.ca/dgv/app/faq&quot; rel=&quot;nofollow noreferrer&quot;&gt;Database of Genomic Variants&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The objective of the Database of Genomic Variants is to provide a comprehensive summary of structural variation in the human genome. We define structural variation as genomic alterations that involve segments of DNA that are larger than 50bp. The content of the database is only representing structural variation identified in healthy control samples.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The Database of Genomic Variants provides a useful catalog of control data for studies aiming to correlate genomic variation with phenotypic data. The database is continuously updated with new data from peer reviewed research studies. We always welcome suggestions and comments regarding the database from the research community.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Information from all of these is also displayed in the Structural Variant browser of &lt;a href=&quot;https://varsome.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;VarSome&lt;/a&gt;&lt;sup&gt;*&lt;/sup&gt;. While you can't search by patient status, so you can't use VarSome to know the clinical history of any sample with the CNV, you can at least look up a specific gene and see the structural variants listed for it. For example, &lt;a href=&quot;https://varsome.com/gene/TP53&quot; rel=&quot;nofollow noreferrer&quot;&gt;for TP53, you would see&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eSWN3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eSWN3.png&quot; alt=&quot;varsome structural variant genome browser&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;*&lt;/sup&gt; &lt;sub&gt;Disclaimer: I work for the company behind VarSome.&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-05-07T17:22:21.730" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8598" PostTypeId="2" ParentId="6826" CreationDate="2019-05-08T08:03:33.990" Score="2" Body="&lt;p&gt;Use a subshell. First print the header and then the variant lines, sorting by chromosome and position:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(grep ^&quot;#&quot; input.vcf; grep -v ^&quot;#&quot; input.vcf | sort -k1,1 -k2,2n) &amp;gt; sorted.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3730" LastEditorUserId="298" LastEditDate="2019-05-08T08:40:48.460" LastActivityDate="2019-05-08T08:40:48.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8599" PostTypeId="2" ParentId="8593" CreationDate="2019-05-08T09:41:26.713" Score="2" Body="&lt;p&gt;I am not aware of any widespread index formats for query-by-QNAME for BAM or query-by-ID for VCF. I thought there were historical &lt;em&gt;samtools{-devel,-help}&lt;/em&gt; threads to the effect of “in principle one could index a named-sorted BAM file, but there's insufficient demand so no-one's ever implemented it”, but I can't find any just now.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover note that VCF files are by definition sorted by genomic position, so such an index would just be a hash table of rsID to file offset or genomic position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using dbSNP, probably you have a dbSNP database or API that you can use to query by rsID and get genomic position back. So you can retrieve your VCF contents by rsID in two stages: do that dbSNP query, and use its results to find the variants in your VCF file by position.&lt;/p&gt;&#xA;" OwnerUserId="134" LastActivityDate="2019-05-08T09:41:26.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8600" PostTypeId="1" AcceptedAnswerId="8601" CreationDate="2019-05-08T11:46:17.617" Score="0" ViewCount="118" Body="&lt;p&gt;I am trying to read a txt file in R but always I am getting &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in scan(file = file, what = what, sep = sep, quote = quote, dec = dec,  : &#xA;  line 113 did not have 10 elements&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I noticed that in the header of my file I have something like &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; 0000775 €   g¿´�€   4_®™00000016125 13404727020 017402  0                                                                                                    ustar   devons01                        stlab-icgc &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Even by removing these characters I am not able to read this file&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is link of my file&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/rpc5jgti5aa433b/txt.txt?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/rpc5jgti5aa433b/txt.txt?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could somebody please heap me in reading this file in R?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2019-05-08T20:20:45.290" Title="Reading a txt in R with weird header" Tags="|r|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8601" PostTypeId="2" ParentId="8600" CreationDate="2019-05-08T11:55:09.410" Score="2" Body="&lt;p&gt;Delete the corrupt portion preceeding &lt;code&gt;CFS&lt;/code&gt; in the header and it works with with &lt;code&gt;read.delim()&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-05-08T11:55:09.410" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8602" PostTypeId="2" ParentId="8593" CreationDate="2019-05-08T13:04:33.220" Score="1" Body="&lt;p&gt;Take a look at &lt;a href=&quot;https://github.com/arq5x/grabix&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;grabix&lt;/code&gt;&lt;/a&gt;. Using &lt;code&gt;grabix&lt;/code&gt; you could implement a binary search on a VCF sorted by rsID and compressed using &lt;code&gt;bgzip&lt;/code&gt;. &lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2019-05-08T13:04:33.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8603" PostTypeId="2" ParentId="8581" CreationDate="2019-05-08T14:35:49.133" Score="2" Body="&lt;p&gt;As mentioned in the comments, your biological question is going to be vitally important in determining the best way to tackle this issue. However, there are some things that you should generally keep in mind. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Most &lt;em&gt;de novo&lt;/em&gt; assemblers will perform kmer normalization.&lt;/strong&gt; This means that the merged RNA-Seq data will be normalized across all libraries. This is almost certainly not what you want because you'll be randomly selecting reads from all libraries without regard to their treatments. So, if you're going to allow for kmer normalization, you should normalize prior to merging the libraries so that you have equal representation of the reads by library. If you don't normalize, you're going to have a very fragmented assembly, and it will take a long time to generate.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;&lt;em&gt;De novo&lt;/em&gt; assemblers using short reads assume all identified exons with junction support may be used to generate splice variants with equal probability.&lt;/strong&gt; I'm not entirely sure how to best phrase that, but here's an example. Suppose in a cold-treated library that Gene A generates a transcript using exons 1-2-3, but a heat-treated library generates transcripts using 3-4-5 and 2-3-5. It may be that merging the libraries results in transcripts generated containing various permutations of those exons (e.g. 1-2-3-5, 2-3-4, etc.) that aren't generated in any of the biological settings but can no longer be distinguished. This will lead to false positive transcripts which will detrimentally affect expression quantification.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Merging assemblies is difficult.&lt;/strong&gt; So, to be true to each treatment, it makes more sense to generate individual assemblies (which will also result in library-specific kmer normalization). StringTie essentially does this using reference genomes. You assemble using each library (all bio-reps for a treatment), and then merge after the fact and re-quantify. This is the best approach. However, it can be difficult and computationally taxing. Some programs like CD-HIT (specifically cd-hit-est) are typically used in these cases. Keep in mind that CD-HIT will drop sequences with high similarity, not merge them. This may lead to loss of some uniquely assembled sequence. Other approaches involve using programs like CAP3 (a DNA assembler) to treat the transcripts like longer reads for assembly. Depending on the parameter choice, this might work, but you will risk generating chimeric transcripts.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;How you move forward with the assembly process will again depend on your biological questions. I would tend to promote that you tackle the questions one at a time rather than using a single transcriptome to address them all, but there may be circumstances in which merging assemblies makes more sense. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your specific case, I would perform multiple assemblies using a variety of the approaches described above and then annotate the assemblies (ex. using Trinotate) to see how their quality metrics compare. I would discourage generating a single transcriptome and then re-assembling individual transcriptomes from that assembly -- the single transcriptome may have misassemblies that would then propagate to the individual assemblies, and this process would generally be messy/difficult. &lt;/p&gt;&#xA;" OwnerUserId="3499" LastEditorUserId="3499" LastEditDate="2019-05-14T12:46:05.133" LastActivityDate="2019-05-14T12:46:05.133" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8604" PostTypeId="1" CreationDate="2019-05-08T14:52:56.513" Score="1" ViewCount="320" Body="&lt;p&gt;A collaborator of mine is using ComBat for some RNA-seq data. I would like to understand what it's doing, and I have a specific question about the structure of the model. Equation 2.1 of &lt;a href=&quot;https://academic.oup.com/biostatistics/article/8/1/118/252073&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper&lt;/a&gt; reads &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Y_{ijg} = \alpha_g + X\beta_g + \gamma_{ig} + \delta_{ig}\epsilon_{ijg}$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is the sample, &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; is the batch, &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is a gene, &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; is observed expression, &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; is an intercept term, &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; is a vector of coefficients, &lt;span class=&quot;math-container&quot;&gt;$\gamma$&lt;/span&gt; is an additive batch effect, &lt;span class=&quot;math-container&quot;&gt;$\delta$&lt;/span&gt; is a multiplicative batch effect, and &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is a design matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; not indexed by &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;? If there are &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; samples, &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; genes, and &lt;span class=&quot;math-container&quot;&gt;$d$&lt;/span&gt; variables in the analysis, what are the dimensions of &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; (or &lt;span class=&quot;math-container&quot;&gt;$X_i$&lt;/span&gt;) and &lt;span class=&quot;math-container&quot;&gt;$\beta_g$&lt;/span&gt;? &lt;/p&gt;&#xA;" OwnerUserId="1708" LastActivityDate="2019-05-08T18:28:18.140" Title="ComBat batch correction: understanding the model" Tags="|batch-effects|combat|" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8605" PostTypeId="1" AcceptedAnswerId="8611" CreationDate="2019-05-08T16:53:19.213" Score="1" ViewCount="1682" Body="&lt;p&gt;I need to calculate the occurrences of a motif (including overlaps) in sequences (motif is passed in the first line of standard input and FASTA sequences in subsequent lines). The sequence name starts with &gt;, and after whitespace is just a comment about the sequence that needs to be neglected. The input of program is like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; AT&#xA; &amp;gt;seq1 Comment......&#xA; AGGTATA&#xA; TGGCGCC&#xA; &amp;gt;seq2 Comment.....&#xA; GGCCGGCGC&#xA; GCCGGCGGG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output should be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seq1: 2&#xA;seq2: 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I decided to save the first line as a motif, strip the comment from sequence name, join lines of sequence in one line and save sequence names (keys) and sequences (values) in a dictionary. I also wrote a function for motif_count and want to call it on dictionary values and then save it in a separate dictionary for final output. However, it does not work the way I want it as the method motif_count does not produce any result when called on my d values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; #!/usr/bin/env python3&#xA;&#xA; import sys&#xA;&#xA; sequence = sys.stdin.readlines()&#xA; d = {}&#xA; temp_genename = None&#xA; temp_sequence = None&#xA; motif = sequence[0]&#xA;&#xA;def motif_count(motif, temp_sequence):&#xA;    count = 0&#xA;    start = -1&#xA;    while True:&#xA;        start = temp_sequence.find(motif, start +1)&#xA;    if start &amp;lt;0:&#xA;        return count&#xA;    count += 1&#xA;&#xA;if sequence[1][0] != '&amp;gt;':&#xA;&#xA;    print(&quot;ERROR&quot;)&#xA;&#xA;exit(1)&#xA;&#xA;for line in sequence[1:]:&#xA;    if line[0] == '&amp;gt;':&#xA;       temp_genename = line.split(' ')[0].strip()&#xA;       tmp_genename = temp_genename.replace('&amp;gt;', '')&#xA;       temp_sequence = &quot;&quot;&#xA;&#xA;else:&#xA;&#xA;      temp_sequence += line.strip()&#xA;&#xA;d[tmp_genename] = temp_sequence&#xA;for key in d:&#xA;    motif_count(motif, key)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4542" LastEditorUserId="298" LastEditDate="2019-05-09T13:19:37.727" LastActivityDate="2019-05-09T14:52:03.103" Title="Calculate the occurrence of motif in sequences (including overlaps) using Python" Tags="|python|phylogenetics|homework|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8607" PostTypeId="2" ParentId="8605" CreationDate="2019-05-08T19:46:08.877" Score="3" Body="&lt;p&gt;This is a good task for &lt;a href=&quot;https://biopython.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biopython&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;with open('input_file') as f:&#xA;    motif = next(f).rstrip().lower()&#xA;    for r in SeqIO.parse(f, 'fasta'):&#xA;        print(r.id, ':', r.seq.lower().count_overlap(motif))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note: in future I would recommend don't add the motif to the start of your FASTA file, keep it separate so as not to alter the standard file format.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2019-05-08T19:46:08.877" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8608" PostTypeId="2" ParentId="8600" CreationDate="2019-05-08T20:20:45.290" Score="3" Body="&lt;p&gt;We can skip the first row:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;read.delim(&quot;txt.txt&quot;, skip = 1, header = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Or&lt;/strong&gt; we can get the &lt;em&gt;&quot;uncorrupted&quot;&lt;/em&gt; version of the table, it looks like it is from &lt;a href=&quot;https://doi.org/10.1016/j.celrep.2013.07.003&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S2211124713003501?via%3Dihub#mmc4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Table S4&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="131" LastActivityDate="2019-05-08T20:20:45.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8609" PostTypeId="2" ParentId="8605" CreationDate="2019-05-09T04:09:23.063" Score="0" Body="&lt;p&gt;I like @Chris_Rands tidy use of OO, objects are really important per se.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There appears a paradox with your assignment that you might want to refer back to your supervisor/tutors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &quot;AT&quot; is the motif, this cannot overlap with another &quot;AT&quot; motif, therefore the request for &quot;overlapping motifs&quot; makes this part of the code superfluous.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be better expressed if the motif was &quot;ATA&quot; for example. Thus if the sequence was ATATATA the motif is present 3 times, but only twice if the motif was contiguous. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally the code essentially searches for AT microsatellite motifs and formally performed using e.g. tandem repeat finder.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-05-09T04:15:24.647" LastActivityDate="2019-05-09T04:15:24.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8610" PostTypeId="1" CreationDate="2019-05-09T13:01:59.807" Score="2" ViewCount="75" Body="&lt;p&gt;I'm analyzing mygenome.fa with RepeatScout and RepeatMasker to find transposable elements. I produced a library of Repetitive elements with repeatscout and the I masked TE using Repeatmasker. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./build_lmer_table -l 14 -sequence mygenome.fa -freq mygenome.freq&#xA;./RepeatScout -sequence mygenome.fa -output mygenome_repeats.fa -freq mygenome.freq -l 14&#xA;cat mygenome_repeats.fa| ./filter-stage-1.prl &amp;gt;mygenome_repeats_filtered1.fasta&#xA;./RepeatMasker -s -lib mygenome_repeats_filtered1.fa mygenome.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Generate a masked genome using (non-low-complexity, non-tandem) repeats&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat mygenome_repeats_filtered1.fa | ./filter-stage-2.prl --cat mygenome.fa.out &amp;gt;mygenome_repeats_filtered2.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Filter out all (non-low-complexity, non-tandem) repeats that have less than 10 repeats&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./RepeatMasker -pa 4 -s -lib mygenome_repeats_filtered2.fa -nolow -norna -no_is -gff mygenome.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After this I ran Repeatmasker: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./RepeatMasker no_is mygenome_repeats_filtered2.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I produced:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mygenome_repeats_filt2.fa.masked  mygenome_repeats_filt2.fa.tbl&#xA;mygenome_repeats_filt2.fa.cat  mygenome_repeats_filt2.fa.out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would to know a way to find the location of transposable elements masked by RepeatMasker on my genome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Supposing that this is a part of my Repetitive elements library produced with RepeatScout as I showed above:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;less mygenome_repeats_filt2.fa&#xA;&#xA;&amp;gt;R=3 (RR=4.  TRF=0.000 NSEG=0.000)&#xA;TAAGGCGGCGAGCTGGCAGAATCGTTAGCACGCCGGGCGAAATGCTTAGCGGTATTTCGTCTGTCTTTACGTTCTGAGTT&#xA;CAAATTCCGCCGAGGTCGACTTTGCCTTTCATCCTTTCGGGGTCGATAAAATAAGTACCAGTTGAGCACTGGGGTCGATG&#xA;TAATCGACTTACCCCCTCCCCCAAAATTTCTGGCCTTGTGCCTATATTAGAAACGATTATT&#xA;&amp;gt;R=4 (RR=5.  TRF=0.122 NSEG=0.226)&#xA;ACACACACACACACACACACACACACATATATATATATATACATATATACGACGGGCTTCTTTCAGTTTCCGTCTACCAA&#xA;ATCCACTCACAAGGCTTTGGTCGGCCCGAGGCTATAGTAGAAGACACTTGCCCAAGGTGCCACGCAGTGGGACTGAACCC&#xA;GGAACCATGTGGTTGGTAAGCAAGCTACTTACCACACAGCCACTCCTGCGCCTATATATAT&#xA;&amp;gt;R=6 (RR=7.  TRF=0.134 NSEG=0.247)&#xA;TTGTTTCAGTCATTTGACTGCGGCCATGCTGGAGCACCGCCTTTAGTCGAGCAAATCGACCCCAGGACTTATTCTTTGTA&#xA;AGCCTAGTACTTATTCTATCGGTCTCTTTTGCCGAACCGCTAAGTTACGGGGACGTAAACACACCAGCATCGGTTGTCAA&#xA;GCGATGTTGGGGGGACAAACACAGACACACAAACACACACACACACATACATATATATATATATATATATA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;from the file mygenome_repeats_filt2.fa.out I can see that in R=3 there is a trasposable element:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   SW   perc perc perc  query     position in query    matching        repeat              position in repeat&#xA;score   div. del. ins.  sequence  begin end   (left)   repeat          class/family      begin  end    (left)  ID&#xA;&#xA;348   20.6  0.0  0.0  R=3          78   140   (81) + AmnSINE2        SINE/tRNA-Deu         67    129  (229) 234  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see I have the coordinates to find this element in the library, but I would to find its exactly in the file that contain my assembled genome. &lt;/p&gt;&#xA;" OwnerUserId="4475" LastEditorUserId="4475" LastEditDate="2019-05-09T13:57:14.467" LastActivityDate="2019-05-09T13:57:14.467" Title="find repetitive elements on genome" Tags="|repeatmasker|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8611" PostTypeId="2" ParentId="8605" CreationDate="2019-05-09T14:35:39.677" Score="1" Body="&lt;p&gt;I tried to fix the indentation of the code you posted, a typo in some variable names, plus a few other issues that caused it to not work as expected (see the comment in the code):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;import sys&#xA;&#xA;sequence = sys.stdin.readlines()&#xA;d = {}&#xA;temp_genename = None&#xA;temp_sequence = None&#xA;# The lines returned by readlines() contain end-of-line characters,&#xA;# this is why strip() is needed&#xA;motif = sequence[0].strip()&#xA;&#xA;def motif_count(motif, temp_sequence):&#xA;    count = 0&#xA;    start = -1&#xA;    while True:&#xA;        start = temp_sequence.find(motif, start +1)&#xA;        if start &amp;lt;0:&#xA;            return count&#xA;        count += 1&#xA;&#xA;if sequence[1][0] != '&amp;gt;':&#xA;&#xA;    print(&quot;ERROR&quot;)&#xA;&#xA;    exit(1)&#xA;&#xA;for line in sequence[1:]:&#xA;    if line[0] == '&amp;gt;':&#xA;        # Record the previous one in the dictionary&#xA;        if temp_genename is not None:&#xA;            d[temp_genename] = temp_sequence&#xA;        temp_genename = line.split(' ')[0].strip()&#xA;        temp_genename = temp_genename.replace('&amp;gt;', '')&#xA;        temp_sequence = &quot;&quot;&#xA;&#xA;    else:&#xA;&#xA;        temp_sequence += line.strip()&#xA;&#xA;# This should be done each time a new sequence is found,&#xA;# not only at the end&#xA;d[temp_genename] = temp_sequence&#xA;for key in d:&#xA;    # We need to use the result&#xA;    # motif_count(motif, key)&#xA;    # We want to count the motif in the sequence, not in its name&#xA;    count = motif_count(motif, d[key])&#xA;    print(f&quot;{key}: {count}&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Testing it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ cat tmp.fa &#xA;AT&#xA;&amp;gt;seq1 Comment......&#xA;AGGTATA&#xA;TGGCGCC&#xA;&amp;gt;seq2 Comment.....&#xA;GGCCGGCGC&#xA;GCCGGCGGG&#xA;$&lt;/span&gt; cat tmp.fa | ./count_motif.py&#xA;seq1: 2&#xA;seq2: 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is another way to do, using &lt;code&gt;valmap&lt;/code&gt; to make a dict of the motif counts from the dict of sequences, and with a few other programming style changes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;import sys&#xA;from cytoolz import valmap&#xA;&#xA;motif = sys.stdin.readline().strip()&#xA;&#xA;&#xA;# We can make this a function of only one parameter&#xA;# because motif is fixed&#xA;def motif_count(temp_sequence):&#xA;    count = 0&#xA;    start = -1&#xA;    while True:&#xA;        start = temp_sequence.find(motif, start +1)&#xA;        if start &amp;lt;0:&#xA;            return count&#xA;        count += 1&#xA;&#xA;&#xA;seq_dict = {}&#xA;temp_genename = None&#xA;temp_sequence = None&#xA;&#xA;for line in sys.stdin:&#xA;    if line[0] == '&amp;gt;':&#xA;        # Record the previous one in the dictionary&#xA;        if temp_genename is not None:&#xA;            seq_dict[temp_genename] = temp_sequence&#xA;        temp_genename, *_ = line[1:].strip().split()&#xA;        temp_sequence = &quot;&quot;&#xA;    else:&#xA;        temp_sequence += line.strip()&#xA;&#xA;msg = f&quot;&quot;&quot;File format might be wrong&quot;&#xA;motif: {motif}&#xA;temp_genename: {temp_genename}&#xA;temp_genename: {temp_sequence}&quot;&quot;&quot;&#xA;&#xA;assert temp_sequence is not None and len(temp_sequence), msg&#xA;&#xA;# Record the last sequence&#xA;seq_dict[temp_genename] = temp_sequence&#xA;&#xA;# Make a new dict where the values are the motif counts&#xA;# valmap needs the function motif_count to take only one parameter&#xA;counts_dict = valmap(motif_count, seq_dict)&#xA;# Print the results, iterating over dict items&#xA;for (name, count) in counts_dict.items():&#xA;    print(f&quot;{name}: {count}&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2019-05-09T14:52:03.103" LastActivityDate="2019-05-09T14:52:03.103" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8612" PostTypeId="1" CreationDate="2019-05-09T15:33:42.780" Score="2" ViewCount="4030" Body="&lt;p&gt;I am using &lt;strong&gt;Seurat v2&lt;/strong&gt; for professional reasons (I am aware of the availablity of &lt;strong&gt;Seurat v3&lt;/strong&gt;). I am clustering and analysing single cell RNA seq data. How do I add a coloured annotation bar to the heatmap generated by the &lt;strong&gt;DoHeatmap&lt;/strong&gt; function from Seurat v2? I want to be able to demarcate my cluster numbers on the heatmap over a coloured annotation bar. I want to have the an image like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GwS2x.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GwS2x.png&quot; alt=&quot;this&quot;&gt;&lt;/a&gt; (taken from &lt;a href=&quot;https://satijalab.org/seurat/v3.0/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Satija V3 tutorial&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My original code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Seurat)&#xA;DoHeatmap(object = object, genes.use = top20_pos$gene)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;produces the following image which is not what I want:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/74ice.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/74ice.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware that &lt;strong&gt;DoHeatmap&lt;/strong&gt; function returns a &lt;strong&gt;ggplot2 object&lt;/strong&gt; but after searching the internet for a long time, I have not been able to find  which &lt;strong&gt;geom&lt;/strong&gt; layer I can add to achive my objective.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On the other hand, I have tried to generate a separate coloured annotation bar from another package called &lt;strong&gt;ComplexHeatmap&lt;/strong&gt; with the intention of cropping it to fit the cluster demarcations on my heatmap generated with &lt;strong&gt;DoHeatmap&lt;/strong&gt; but this has proven to be very time-consuming and inaccurrate:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(ComplexHeatmap)&#xA;&#xA;df = data.frame(type = c(rep(&quot;a&quot;, 18), rep(&quot;b&quot;, 3), rep(&quot;c&quot;, 2), rep(&quot;d&quot;, 1), rep(&quot;e&quot;, 1)))&#xA;ha = HeatmapAnnotation(df = df, col = list(type = c(&quot;a&quot; =  &quot;blue&quot;, &quot;b&quot; = &quot;red&quot;, &quot;c&quot; = &quot;green&quot;, &quot;d&quot; = &quot;cyan&quot;, &quot;e&quot; = &quot;magenta&quot;)))&#xA;draw(ha, 1:25)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which produces:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/CkKIm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CkKIm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will be very grateful for any hint on how to achieve my goal automatically and accurately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for your help.&lt;/p&gt;&#xA;" OwnerUserId="3968" LastEditorUserId="131" LastEditDate="2019-05-11T03:02:52.980" LastActivityDate="2021-04-23T12:40:46.987" Title="How do I add a colour annotation bar to the heatmap generated by DoHeatmap function of Seurat v2?" Tags="|r|scrnaseq|seurat|ggplot2|heatmap|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8613" PostTypeId="1" CreationDate="2019-05-09T19:03:59.463" Score="-2" ViewCount="100" Body="&lt;p&gt;This is a silly post, but hope it makes a point. We have BioPerl (which never impressed me, in fact there isn't even a BioPerl tag), BioPython (which admittedly is impressive), and even now BioRust. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why not BioMoose? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moose is Perl5s answer to objects and is really impressive in its simplicity and ease of use. Before Moose Perl5 objects, were/are, how can I say, make evolutionary chaos theory look hip and relevant. Moose changed that completely, but stacking Moose code into BioPerl is never gonna work.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-05-09T20:01:13.580" Title="How about BioMoose?" Tags="|perl|" AnswerCount="1" CommentCount="6" ClosedDate="2019-05-10T10:59:47.943" ContentLicense="CC BY-SA 4.0" />
  <row Id="8614" PostTypeId="2" ParentId="8613" CreationDate="2019-05-09T20:01:13.580" Score="2" Body="&lt;p&gt;I fear this entire thread is going to veer dangerously close to opinionation. But regardless of how one feels about the Perl language and its various libraries and communities, I think we can objectively say that the era of Perl's dominance in the bioinformatics community has passed. A lot of tools and libraries implemented in Perl are still widely used, and there is a tremendous amount of legacy Perl code around. The ability to read and write Perl remains a valuable skill.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But in 2019, few people (with a choice) are learning Perl or implementing new tools and frameworks in Perl. Python fits snuggly into the niche once occupied by Perl, and its native OO support is (I dare to say) objectively superior to any &lt;em&gt;post hoc&lt;/em&gt; OO support Perl 3rd-party libraries provide.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The broader web/tech community moved on from Perl to Python (and Ruby and others) even before the bioinformatics community did, and this broader community's offerings (IPython, Jupyter, scipy, numpy, pytest, etc.) make Python an even more attractive alternative to Perl for bioinformatics work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the meantime, the &quot;hadleyverse&quot; ecosystem of R packages for data manipulation make R much more capable of the types of data pre-processing tasks for which an R programmer might have previously used a language like Perl or Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, Perl is waning in popularity (if not relevance), and I would not expect much enthusiasm from the bioinformatics community for a new framework built around a relatively obscure Perl extension.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-05-09T20:01:13.580" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8615" PostTypeId="2" ParentId="7393" CreationDate="2019-05-10T07:16:53.697" Score="2" Body="&lt;p&gt;Most likely your Augustus installation is not functioning properly. Did you install Augustus from source or using conda? I believe the conda version of Augustus is known to have some issues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something worth trying: &lt;a href=&quot;https://busco.ezlab.org/files/BUSCO_3_Ubuntu_Gnome_16.04.ova&quot; rel=&quot;nofollow noreferrer&quot;&gt;Running a complete busco install in a VM&lt;/a&gt;. This image has all dependencies installed and properly configured.&lt;/p&gt;&#xA;" OwnerUserId="4552" LastActivityDate="2019-05-10T07:16:53.697" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8616" PostTypeId="1" AcceptedAnswerId="8641" CreationDate="2019-05-10T18:22:58.790" Score="1" ViewCount="737" Body="&lt;p&gt;I have created a ComplexHeatmap containing 2 &lt;code&gt;Heatmap&lt;/code&gt;s, each with their own legend. The legend on the first Heatmap is continuous, whereas that of the second is discrete.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I concatenate these &lt;code&gt;Heatmap&lt;/code&gt;s vertically, and &lt;code&gt;draw&lt;/code&gt; the &lt;code&gt;HeatmapList&lt;/code&gt;, their legends are auto-aligned to the center of the plot. Is there any way I can have these legends positioned centered to the respective heatmaps and not to the center of the plot?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EmdRY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EmdRY.png&quot; alt=&quot;Current_Image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's what I'd like to have:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/00LSG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/00LSG.png&quot; alt=&quot;Desired_Image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I go about solving this problem? I'd really like to avoid creating customized &lt;code&gt;Legend&lt;/code&gt;s if I can, but if that's what it takes, that works too.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for any help/pointers!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I tried adding the &lt;code&gt;ComplexHeatmap&lt;/code&gt; tag, but it doesn't exist yet and I don't have the reputation to create new tags yet. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Since this is a Bioconductor package, I also created an identical post on the &lt;a href=&quot;https://support.bioconductor.org/p/120958/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioconductor support forum&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dummy data and code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;devtools::install_github('jokergoo/ComplexHeatmap');&#xA;library(ComplexHeatmap);&#xA;library(viridis);&#xA;&#xA;dummy_top_mat &amp;lt;- structure(c(0.4563, 0.2211, 1.2235, 0.067, 1.6859, 1.0936, 0.4533, &#xA;1.6844, 1.0039, 0.5402, 0.6841, 1.498, 1.042, 0.1711, 0.3565, &#xA;2.2814, 3.0516, 1.5012, 0.5367, 3.0846, 1.1909, 0.235, 1.4266, &#xA;0.4858, 2.9054, 0.3733, 0.6902, 0.8555, 0.4234, 2.6778, 0.1568, &#xA;0.5556, 2.0172, 0.8034, 2.2897, 0.1166, 3.8033, 0.1431, 2.0606, &#xA;1.2725, 1.5365, 0.4123, 1.2087, 1.1264, 0.8334, 1.1943, 1.58, &#xA;1.5849, 0.3004, 0.3722, 0.0362, 0.0532, 1.4867, 0.4053, 0.3615, &#xA;0.0897, 1.3217, 1.1447, 1.3058, 0.1903, 0.1067, 0.9482, 1.3382, &#xA;3.2955, 0.391, 1.0418, 0.2041, 1.208, 1.5857, 3.5313, 0.472, &#xA;1.389, 0.2143, 0.0226, 0.029, 0.444, 2.0521, 0.3955, 0.3495, &#xA;0.5062, 1.3236, 1.3234, 0.7111, 0.1176, 2.2223, 1.2073, 0.3964, &#xA;2.1175, 0.3382, 0.2816, 0.71, 3.1417, 0.2402, 0.5793, 0.7662, &#xA;1.6782, 0.0986, 0.087, 0.5447, 2.6672, 1.2498, 1.0676, 1.8608, &#xA;1.8146, 0.1422, 0.4221, 0.0303, 0.9541, 0.7358, 1.7664, 1.5144, &#xA;0.2034, 0.9366, 0.7837, 0.3284, 0.1477, 1.8306, 1.3564, 0.1126, &#xA;0.0171, 2.9858, 0.0233, 0.2796, 0.6995, 1.6081, 0.215, 1.7093, &#xA;0.5178, 1.7061, 2.473, 1.8912, 0.7661, 4.4102, 1.2963, 0.6542, &#xA;0.4281, 0.4491, 0.6, 0.4076, 1.6869, 0.4747, 3.9823, 1.1226, &#xA;2.7355, 2.7036, 0.2241, 0.983, 1.0992, 1.4736, 0.1584, 0.2995, &#xA;0.2272, 0.5744, 0.9314, 0.6924, 0.0812, 1.361, 0.727, 0.1525, &#xA;1.3367, 0.566, 2.7801, 0.2349, 3.2655, 1.0675, 0.449, 0.5411, &#xA;0.8291, 0.52, 0.0507, 0.6538, 0.4636, 1.2063, 0.9784, 0.1925, &#xA;0.0756, 0.136, 1.2529, 0.317, 0.0281, 0.8668, 3.4138, 5.3898, &#xA;0.521, 0.087, 3.4819, 0.1114, 0.0061, 0.9804, 1.139, 1.4159, &#xA;1.0297, 0.6503, 1.0828, 2.3527, 0.057, 0.5602, 0.4017, 0.9985, &#xA;0.8673), .Dim = c(10L, 20L), .Dimnames = list(c(&quot;gene01&quot;, &quot;gene02&quot;, &#xA;&quot;gene03&quot;, &quot;gene04&quot;, &quot;gene05&quot;, &quot;gene06&quot;, &quot;gene07&quot;, &quot;gene08&quot;, &quot;gene09&quot;, &#xA;&quot;gene10&quot;), c(&quot;sample01&quot;, &quot;sample02&quot;, &quot;sample03&quot;, &quot;sample04&quot;, &#xA;&quot;sample05&quot;, &quot;sample06&quot;, &quot;sample07&quot;, &quot;sample08&quot;, &quot;sample09&quot;, &quot;sample10&quot;, &#xA;&quot;sample11&quot;, &quot;sample12&quot;, &quot;sample13&quot;, &quot;sample14&quot;, &quot;sample15&quot;, &quot;sample16&quot;, &#xA;&quot;sample17&quot;, &quot;sample18&quot;, &quot;sample19&quot;, &quot;sample20&quot;)));&#xA;&#xA;dummy_bottom_mat &amp;lt;- structure(c(&quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Negative&quot;, &#xA;&quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Negative&quot;, &quot;Positive&quot;, &#xA;&quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Negative&quot;, &quot;Positive&quot;, &#xA;&quot;Negative&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &#xA;&quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &#xA;&quot;Negative&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &#xA;&quot;Negative&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Negative&quot;, &quot;Positive&quot;, &#xA;&quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &#xA;&quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &#xA;&quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &#xA;&quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &#xA;&quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &#xA;&quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Negative&quot;, &#xA;&quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &#xA;&quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &#xA;&quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &#xA;&quot;Positive&quot;, &quot;Negative&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &#xA;&quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &#xA;&quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &#xA;&quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &quot;Positive&quot;, &#xA;&quot;Positive&quot;), .Dim = c(6L, 20L), .Dimnames = list(c(&quot;marker1&quot;, &#xA;&quot;marker2&quot;, &quot;marker3&quot;, &quot;marker4&quot;, &quot;marker5&quot;, &quot;marker6&quot;), c(&quot;sample01&quot;, &#xA;&quot;sample02&quot;, &quot;sample03&quot;, &quot;sample04&quot;, &quot;sample05&quot;, &quot;sample06&quot;, &quot;sample07&quot;, &#xA;&quot;sample08&quot;, &quot;sample09&quot;, &quot;sample10&quot;, &quot;sample11&quot;, &quot;sample12&quot;, &quot;sample13&quot;, &#xA;&quot;sample14&quot;, &quot;sample15&quot;, &quot;sample16&quot;, &quot;sample17&quot;, &quot;sample18&quot;, &quot;sample19&quot;, &#xA;&quot;sample20&quot;)));&#xA;&#xA;hmap1 &amp;lt;- Heatmap(dummy_top_mat, col = colorRampPalette(viridis(15))(100), name = 'log2 (TPM)', na_col = 'grey60', cluster_rows = FALSE, cluster_columns = FALSE, row_names_side = 'left', column_title_side = 'top', rect_gp = gpar(col='white', lwd=0.5), height=unit(10*0.75,'cm'), width = unit(20*0.75, 'cm'), column_title = 'Gene Expression');&#xA;&#xA;hmap2 &amp;lt;- Heatmap(dummy_bottom_mat, col = c('Negative'='red', 'Positive'='blue'), na_col = 'grey60', row_split = factor( c('gp1','gp1','gp1','gp1','gp2','gp2'), c('gp1','gp2'), ordered = TRUE), name = 'Marker\nStatus', rect_gp = gpar(col='white',lwd=1), row_title = NULL, height = unit(6*0.75,'cm'), width = unit(20*0.75, 'cm'), row_names_side = 'left', column_title = 'Biomarkers');&#xA;&#xA;draw(hmap1 %v% hmap2, ht_gap = unit(1, 'cm'), auto_adjust = FALSE);&#xA;&#xA;sessionInfo()&#xA;&#xA;R version 3.5.1 (2018-07-02)&#xA;Platform: x86_64-apple-darwin13.4.0 (64-bit)&#xA;Running under: macOS  10.14.4&#xA;&#xA;Matrix products: default&#xA;BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib&#xA;LAPACK: /Users/ram/miniconda3/lib/R/lib/libRblas.dylib&#xA;&#xA;locale:&#xA;[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8&#xA;&#xA;attached base packages:&#xA;[1] grid      stats     graphics  grDevices utils     datasets  methods   base     &#xA;&#xA;other attached packages:&#xA; [1] viridis_0.5.1        viridisLite_0.3.0    ComplexHeatmap_2.1.0 forcats_0.4.0       &#xA; [5] stringr_1.4.0        dplyr_0.8.0.1        purrr_0.2.5          readr_1.3.1         &#xA; [9] tidyr_0.8.3          tibble_2.1.1         ggplot2_3.1.1        tidyverse_1.2.1     &#xA;&#xA;loaded via a namespace (and not attached):&#xA; [1] Rcpp_1.0.0          lubridate_1.7.4     lattice_0.20-38     circlize_0.4.6      prettyunits_1.0.2  &#xA; [6] png_0.1-7           ps_1.3.0            rprojroot_1.3-2     assertthat_0.2.1    digest_0.6.18      &#xA;[11] R6_2.3.0            cellranger_1.1.0    plyr_1.8.4          backports_1.1.4     reprex_0.2.1       &#xA;[16] httr_1.4.0          pillar_1.3.1        GlobalOptions_0.1.0 rlang_0.3.1         lazyeval_0.2.2     &#xA;[21] readxl_1.3.1        rstudioapi_0.10     callr_3.2.0         GetoptLong_0.1.7    desc_1.2.0         &#xA;[26] devtools_2.0.2      munsell_0.5.0       broom_0.5.2         compiler_3.5.1      modelr_0.1.4       &#xA;[31] pkgconfig_2.0.2     pkgbuild_1.0.3      shape_1.4.4         tidyselect_0.2.5    gridExtra_2.3      &#xA;[36] crayon_1.3.4        withr_2.1.2         nlme_3.1-137        jsonlite_1.6        gtable_0.3.0       &#xA;[41] magrittr_1.5        scales_1.0.0        cli_1.1.0           stringi_1.4.3       remotes_2.0.4      &#xA;[46] fs_1.3.1            xml2_1.2.0          generics_0.0.2      rjson_0.2.20        RColorBrewer_1.1-2 &#xA;[51] tools_3.5.1         glue_1.3.0          hms_0.4.2           processx_3.3.1      pkgload_1.0.2      &#xA;[56] parallel_3.5.1      clue_0.3-57         colorspace_1.4-1    cluster_2.0.7-1     sessioninfo_1.1.1  &#xA;[61] rvest_0.3.3         memoise_1.1.0       haven_2.1.0         usethis_1.5.0 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastEditorUserId="650" LastEditDate="2019-05-10T21:58:32.980" LastActivityDate="2019-05-15T19:27:49.933" Title="Align legends to each heatmap in a ComplexHeatmap with 2 heatmaps" Tags="|r|rna-seq|heatmap|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8617" PostTypeId="2" ParentId="8593" CreationDate="2019-05-10T19:26:37.633" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/a/8599/96&quot;&gt;John Marshall said&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;Moreover note that VCF files are by definition sorted by genomic position, so such an index would just be a hash table of rsID to file offset or genomic position.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I just published &lt;a href=&quot;https://github.com/bioforensics/rsidx&quot; rel=&quot;nofollow noreferrer&quot;&gt;rsidx&lt;/a&gt;, a new package for indexing VCF files by rsID. As John suggests, I simply created a mapping from rsID to genomic coordinates. These are stored in an sqlite3 database.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It currently takes...several...hours to index build 151 of dbSNP on GRCh38. I'm looking into optimizing sqlite3 parameters to accelerate the indexing, as well as an alternative approach based on a two step process (populate an in-memory sqlite3 db, then dump DB to disk). But in the meantime, the search capability works great.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-05-10T19:26:37.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8618" PostTypeId="1" AcceptedAnswerId="8625" CreationDate="2019-05-11T01:55:02.043" Score="4" ViewCount="450" Body="&lt;p&gt;I am using the biopython Entrez.efetch command to retrieve all features (CDS, mRNA, ...) of some genomes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case (NC_014649, Acanthamoeba polyphaga mimivirus), it works as expected:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez, SeqIO&#xA;handle = Entrez.efetch(id='NC_014649', db='nuccore', rettype='gb')&#xA;records = SeqIO.read(handle, 'gb')&#xA;records.features&#xA;&#xA;[SeqFeature(FeatureLocation(ExactPosition(0), ExactPosition(1181549), strand=1), type='source'),&#xA; SeqFeature(FeatureLocation(ExactPosition(191), ExactPosition(2609), strand=1), type='gene'),&#xA; SeqFeature(FeatureLocation(ExactPosition(191), ExactPosition(2609), strand=1), type='mRNA'),&#xA; SeqFeature(FeatureLocation(ExactPosition(191), ExactPosition(2579), strand=1), type='CDS'),&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However in this case (NC_010830, Candidatus Amoebophilus asiaticus), it only returns one record for the whole genome.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;handle = Entrez.efetch(id='NC_010830', db='nuccore', rettype='gb')&#xA;records = SeqIO.read(handle, 'gb')&#xA;records.features&#xA;[SeqFeature(FeatureLocation(ExactPosition(0), ExactPosition(1884364), strand=1), type='source')]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, if I check the corresponding file in the browser, it does have features: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NC_010830.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/NC_010830.1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What could be the reason for this inconsistency ? Did I  miss something ?&lt;/p&gt;&#xA;" OwnerUserId="439" LastEditorUserId="439" LastEditDate="2019-05-15T13:34:23.223" LastActivityDate="2019-05-15T13:34:23.223" Title="Entrez.efetch returns incomplete genbank records" Tags="|genbank|entrez|efetch|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8619" PostTypeId="2" ParentId="8618" CreationDate="2019-05-11T03:42:03.830" Score="0" Body="&lt;p&gt;I use the &lt;code&gt;efetch&lt;/code&gt; bash command from &lt;strong&gt;edirect&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With &lt;code&gt;efetch -db nuccore -id CP001102.1 -format gb&lt;/code&gt;, I get the &lt;em&gt;Candidatus Amoebophilus asiaticus 5a2&lt;/em&gt;, complete genome in &lt;strong&gt;GenBank format&lt;/strong&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might try your command with &lt;code&gt;CP001102.1&lt;/code&gt; instead of &lt;code&gt;NC_010830&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="2970" LastActivityDate="2019-05-11T03:42:03.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8621" PostTypeId="2" ParentId="6802" CreationDate="2019-05-11T15:56:25.707" Score="1" Body="&lt;p&gt;I see that you created the &lt;code&gt;ColAnn&lt;/code&gt; variable with the Condition annotation but did not include it in the &lt;code&gt;draw&lt;/code&gt; call (or even in the &lt;code&gt;Heatmap()&lt;/code&gt; function). Could that be the problem here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Oops, just noticed that this post is &lt;em&gt;really old&lt;/em&gt;. Doesn't have an answer, so I guess I'll let mine stay.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2019-05-11T15:56:25.707" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8622" PostTypeId="1" CreationDate="2019-05-11T21:34:56.597" Score="1" ViewCount="3366" Body="&lt;p&gt;I am trying to use the GPU enabled version of the guppy_basecaller on an HPC cluster.  I am requesting a node that has 2 GPUs and am requesting 1 of the two GPUs.  So, I am trying to set the &lt;code&gt;-x cuda:--&lt;/code&gt; flag.  What I was thinking this needed was the device number, ie if the allocation gave me a node and had the &lt;code&gt;CUDA_VISIBLE_DEVICES=&lt;/code&gt; flag set, I would use &lt;code&gt;-x cuda:0&lt;/code&gt; or &lt;code&gt;-x cuda:1&lt;/code&gt; depending on which GPU I was assigned.  When I set it to 0 or auto, it seems to work when I have a whole.  When I set it to 1 when I am given a job with that device id flag set, the program errors out.  Am I thinking about the flag wrong?  The documents I have found on this are a bit sparse.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CUDA_VISIBLE_DEVICES=1&#xA;working on run id  8138&#xA;working on subset  0&#xA;&#xA;git er done ...&#xA;ONT Guppy basecalling software version 3.0.3+7e7b7d0&#xA;config file:        /groups/ju_lab/softwares/ont-guppy/data/dna_r9.4.1_450bps_hac.cfg&#xA;model file:         /groups/ju_lab/softwares/ont-guppy/data/template_r9.4.1_450bps_hac.jsn&#xA;input path:         ./&#xA;save path:          ./&#xA;chunk size:         1000&#xA;chunks per runner:  1000&#xA;records per file:   4000&#xA;num basecallers:    4&#xA;gpu device:         cuda:1&#xA;kernel path:&#xA;runners per device: 4&#xA;&#xA;[guppy/error] main: Could not create CUDA device 1: CUDA error at /builds/ofan/ont_core_cpp/ont_core/basecall_nn/nn_caller_cuda.&#xA;cpp:651: CUDA_ERROR_INVALID_DEVICE&#xA;[guppy/warning] main: An error occurred in the basecaller. Aborting.&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4558" LastActivityDate="2019-06-21T14:45:30.157" Title="using guppy_basecaller on node with 2 GPUs" Tags="|minion|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8623" PostTypeId="1" CreationDate="2019-05-12T00:19:25.600" Score="3" ViewCount="340" Body="&lt;p&gt;I know this is a broad question and I'm sorry if this is not the right place to post (this is my first post ever), but is there some sort of roadmap for bioinformatics?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm an undergraduate student and I have a strong background in statistics, computer science, and molecular biology so I feel like I should be able to get around, but every time I talk or read about bioinformatics I feel lost. It's such a vast area that I don't even know where to begin or what even exists.&#xA;Is there some kind of all-encompassing textbook or roadmap for me to follow?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most textbooks I could find expend way to much time explaining really basic stuff like what is a cell and how to program and I feel they fail to convey the bigger picture of bioinformatics. Things like what exists in the area, what are the current technologies, conventions, and challenges that we face. What should I know in order to call myself a bioinformatician?&lt;/p&gt;&#xA;" OwnerUserId="4559" LastActivityDate="2019-05-12T09:28:28.910" Title="How to learn/get started bioinformatics?" Tags="|python|" AnswerCount="1" CommentCount="4" ClosedDate="2019-05-13T09:26:56.053" ContentLicense="CC BY-SA 4.0" />
  <row Id="8624" PostTypeId="2" ParentId="8623" CreationDate="2019-05-12T05:48:11.403" Score="2" Body="&lt;p&gt;The formal answer is O'Reilly &quot;Bioinformatics programming, using Python&quot;. Check out the classic O'Reilly texts on &quot;Mastering Perl for bioinformatics&quot;, and possibly&quot;Beginning Perl for bioinformatics&quot; (which is likely far too basic), which establish some core principles and have a solid reputation in teaching. However, the Perl/Python thing might get confusing (I dunno).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are statistics orientated at a guess, O'Reilly's, Bioinformatics data skills... I don't know, but something that would focus on R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The real goal is core-technical competance because the biological understanding will be taught by your supervisor. Thereafter you might turn your attention to teaming up with a lab for project training.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-05-12T09:28:28.910" LastActivityDate="2019-05-12T09:28:28.910" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8625" PostTypeId="2" ParentId="8618" CreationDate="2019-05-12T11:29:44.107" Score="5" Body="&lt;p&gt;This has been answered before. See: &lt;a href=&quot;https://stackoverflow.com/a/55402322/6262370&quot;&gt;https://stackoverflow.com/a/55402322/6262370&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, you need to either use &lt;code&gt;rettype='gbwithparts'&lt;/code&gt; or &lt;code&gt;rettype='gb', style='withparts'&lt;/code&gt; to download the entire genbank flat file.&lt;/p&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2019-05-12T11:29:44.107" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8626" PostTypeId="1" AcceptedAnswerId="8627" CreationDate="2019-05-13T14:08:47.673" Score="1" ViewCount="353" Body="&lt;p&gt;I'm noobie in bioinformatics. I use &lt;code&gt;cal_dn_ds&lt;/code&gt; from &lt;code&gt;Bio.codonalign.codonseq&lt;/code&gt; with method 'NG86'. Sometimes I get dS=-1. What does it mean? How can number of substitutions be negative? Does it mean that something is wrong with my data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Upd.&#xA;1) Chris_Rands pointed to sources, I read them already, but the question remain. What is that &lt;code&gt;ps&lt;/code&gt; variable, and why if it's greater than 3/4, dS = -1? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Some more details. &#xA;&lt;code&gt;cal_dn_ds&lt;/code&gt; requires sequences from &lt;code&gt;CodonAlignment&lt;/code&gt; object, which is created by function &lt;code&gt;Bio.codonalign.build&lt;/code&gt; that requires (quote from docs) &lt;code&gt;protein alignment and corresponding nucleotides&lt;/code&gt;. I have nucleotide sequences from NCBI at the beginning. So I trim non-coding regions from nucleotide sequences, translate all trimmed sequences, write them to a file, perform multiple alignment of the protein sequences, and then pass the list of aligned protein sequences and the list of nucleotide sequences to the &lt;code&gt;build&lt;/code&gt; function.&#xA;I analize protein DKK1, and example sequences that get negatives are &lt;em&gt;Empidonax traillii XM_027898752.1&lt;/em&gt; and &lt;em&gt;Physeter catodon XM_007110600.1&lt;/em&gt;. But I just want to understand general idea behind dS = -1, what can it mean it different cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Michael G. gave an interesting suggestion that it means zero, but it's strange because the function returns dN and dS separately, so Biopython might as well return zero, so the user would handle those cases himself. And for some sequences Biopython actually returns normal positive dS and dN = 0 at the same time. If it's an error, what are the possible causes? And why in those error cases Biopython can calculate dN, but dS is -1?&lt;/p&gt;&#xA;" OwnerUserId="4566" LastEditorUserId="3848" LastEditDate="2019-08-20T22:02:26.210" LastActivityDate="2019-08-20T22:02:26.210" Title="What do negative dN or dS mean in cal_dn_ds in Biopython?" Tags="|python|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8627" PostTypeId="2" ParentId="8626" CreationDate="2019-05-13T18:34:26.257" Score="0" Body="&lt;p&gt;dS = -1 &#xA;.. there are two suggestions&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I suggest its an error, possibly based on nucleotide calls within the S, check for anomylous nucleotides, particularly over a non-synonymous codon&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It might mean their are no synonymous mutations. It is not the best idea to assign this a value of 0 in a ratio because it will cause the ratio to fall over when in fact their could be strong selection pressure, ie non-synon mutations. Having an incrementally small value is problematic because dN if positive could anything up to infinity, so -1 might flag this problem.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Okay I now know why... You are correct to be concerned with the result because its serious. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My sincere advice to you is do not use any dN/dS ratio where -1 is occurring under any circumstances. On this occasion I'm not going to explain why for two reasons:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;First and foremost, as a phylogeneticist you should know why, it relates to the fundamentals of the discipline.&lt;/li&gt;&#xA;&lt;li&gt;(Hint) The answer is in my past posts on this forum.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;To conclude, I'm pretty nuanced in statistical dogma, you can check my record, IMO most dogma is tribalism because every calculation has weaknesses. HERE HOWEVER, there is a real problem and again I urge you to ditch any data set where -1 is occurring and everything associated with it. That is a fairly dire warning should give you a clue about whats wrong. If you are not prepared to do this, then you must report the result and the situations where it is arising (don't hide it).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-05-16T09:26:34.513" LastActivityDate="2019-05-16T09:26:34.513" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8628" PostTypeId="1" AcceptedAnswerId="8645" CreationDate="2019-05-13T22:42:48.393" Score="1" ViewCount="91" Body="&lt;p&gt;I'm using efetch to dig out a middle &amp;quot;envelope&amp;quot; protein within a viral genome specifically within fasta format. The code below works but may not be robust for an entire database. Thus I want to replace the two &amp;quot;perl&amp;quot; lines with formal code within the Entrez package. I tried using &lt;code&gt;efilter -query&lt;/code&gt; but couldn't get it to work. Genome positions are one approach but are not absolute from one virus to another. Rewriting this may be the best option.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;ncbi2=NC_002021&#xA;efetch -db protein -format gpc -id &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$ncbi2&quot; |\&#xA;xtract -pattern INSDSeq -element INSDQualifier_value |\&#xA;perl -p -e 's/^(.+?)\s+taxon.*.*\s+\b([A-Z]{300}[A-Z]+).*/&amp;gt;$&lt;/span&gt;1\n$2/'|\&#xA;perl -p -e &amp;quot;s/[\t \;\/]+/_/g&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt;InfluenzaAvirus_A_PuertoRico_8_1934&#xA;MDVNPTLLFLKVPAQNAISTTFPYTGD.....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I will provide a more detailed explanation shortly.&#xA;Any ideas welcome.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-03-11T17:29:28.910" LastActivityDate="2022-03-11T17:29:28.910" Title="Multi-line efetch digging" Tags="|covid-19|bash|sars-cov-2|perl|efetch|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8629" PostTypeId="1" CreationDate="2019-05-14T12:31:01.300" Score="4" ViewCount="208" Body="&lt;p&gt;&lt;em&gt;[This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/379586&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;]&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I received an hg38 VCF file that's had variants imputed with 1000 genomes. I've encountered some issues with the VCF; REF alleles that do not align to a reference genome, ALT alleles that do not appear to be reported anywhere in the literature, and, most recently, variants that flat-out do not align to the human genome (variants on chr19 with bp-pos 100 million+ when the whole chromosome is in the 50 million bp range).&lt;/p&gt;&#xA;&lt;p&gt;I've worked out hack-y solutions to most of the issues that I've encountered, but this latest one has been an issue for me. I only detected these variants when I ran VEP and it flagged them as not mapping to the genome. As such, I'm more or less removing these variants one at a time using &lt;code&gt;grep -v&lt;/code&gt;. I'd like a solution where I can just remove any variants from the vcf that appear to map to regions that do not exist in the human genome. Bonus points if the solution also encompasses some of the other issues I mentioned, although I think I've already found solutions to those. Is there anything out there that does this?&lt;/p&gt;&#xA;" OwnerUserId="4548" LastEditorUserId="73" LastEditDate="2020-10-05T20:32:06.493" LastActivityDate="2020-10-05T20:32:06.493" Title="Remove variants that do not map to human genome" Tags="|vcf|variant-calling|variation|quality-control|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8630" PostTypeId="2" ParentId="8629" CreationDate="2019-05-14T13:09:57.213" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/379586/#379589&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Biostars&amp;#39; answer&quot;&gt;Answer&lt;/a&gt; I received on Biostars that appears to work as expected:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;bcftools norm -Ob --check-ref wx -f &amp;lt;reference_fasta&amp;gt; &amp;lt;vcf.gz&amp;gt; -o &amp;lt;output_name&amp;gt;&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="4548" LastEditorUserId="48" LastEditDate="2019-05-14T13:22:46.807" LastActivityDate="2019-05-14T13:22:46.807" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8631" PostTypeId="1" CreationDate="2019-05-14T15:57:52.630" Score="0" ViewCount="1914" Body="&lt;p&gt;I'm trying to map RNA-Seq reads generated using the &lt;code&gt;NEB Ultra Directional kit is first strand reversed&lt;/code&gt; protocal. I'm using &lt;a href=&quot;http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml#default-mode-search-for-multiple-alignments-report-the-best-one&quot; rel=&quot;nofollow noreferrer&quot;&gt;bowtie-2.3.5_1&lt;/a&gt; to align against the mus musculuous (mm10) genome.  The reads are single ended.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems that &lt;code&gt;--fr&lt;/code&gt; and &lt;code&gt;--rf&lt;/code&gt; are relevant only for paired end options. I tried these options as well as not specifying either (default).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bowtie2 --fr \&#xA;        --threads 20 \&#xA;        --very-sensitive \&#xA;        -x /reference/mus_musculus/mm10/ucsc_assmebly/illumina_download/Sequence/Bowtie2Index/genome \&#xA;        -U trimmed_out/my_trimmed.fq.gz \&#xA;        -S bowtie_fr_1.sam&#xA;&#xA;$ samtools sort -n -o bowtie_fr_sort_1.sam bowtie_def_1.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I then compared the output of &lt;code&gt;bowtie_rf_sort.sam&lt;/code&gt; &lt;code&gt;bowtie_default_sort.sam&lt;/code&gt; and &lt;code&gt;bowtie_fr_sort.sam&lt;/code&gt;.  They are basically identical excluding the header.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does strandedness matter in this alignment? Perhaps I should care about the &lt;code&gt;--nofw/--norc&lt;/code&gt; options? Am I missing something here?&lt;/p&gt;&#xA;" OwnerUserId="3886" LastEditorUserId="3886" LastEditDate="2019-05-14T16:08:10.287" LastActivityDate="2019-05-14T19:06:48.340" Title="bowtie2 options when mapping stranded single end reads" Tags="|rna-seq|read-mapping|bowtie2|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8633" PostTypeId="2" ParentId="8631" CreationDate="2019-05-14T19:06:48.340" Score="1" Body="&lt;p&gt;I repeat what user172818 says...don't waste your time on bowtie or tophat.  Use STAR or HISAT2, or a pseudoaligner like kallisto with a transcriptome.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-05-14T19:06:48.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8634" PostTypeId="1" CreationDate="2019-05-14T19:10:56.457" Score="6" ViewCount="76" Body="&lt;p&gt;&lt;em&gt;Note: this question was also asked on &lt;a href=&quot;https://github.com/ncbi/sra-tools/issues/202&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have 241 SRA files that appear to be reference compressed. I didn't even know this was a thing until I tried to convert them to Fastq files without an internet connection. I got the &amp;quot;name not found while resolving...&amp;quot; error described at the bottom of &lt;a href=&quot;https://ncbi.github.io/sra-tools/fastq-dump.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt; when I attempted to do the conversion.&lt;/p&gt;&#xA;&lt;p&gt;The instructions say:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;If you have elected to prevent the toolkit from contacting NCBI, you will need to manually acquire the reference(s) here&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;We have many NCBI databases mirrored locally, including RefSeq, so it's likely that we already have all the references. I have two questions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How do I determine the reference used to compress a reference-compressed SRA file? (Note this needs to be scripted for 241 SRA files, so manual inspection is out of the question.)&lt;/li&gt;&#xA;&lt;li&gt;Once I've determined the correct reference and ensured it is available locally, how to I tell &lt;code&gt;fastq-dump&lt;/code&gt; where to find it?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="96" LastEditorUserId="73" LastEditDate="2021-06-25T11:18:58.623" LastActivityDate="2022-03-07T14:49:17.370" Title="Determine reference for reference-compressed SRA file" Tags="|file-formats|fastq|format-conversion|sratoolkit|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8635" PostTypeId="2" ParentId="7505" CreationDate="2019-05-14T22:14:15.647" Score="2" Body="&lt;p&gt;There has been a detailed comparison of kallisto and salmon for lncRNA transcripts by Zheng et al &lt;a href=&quot;https://www.biorxiv.org/content/biorxiv/early/2018/01/09/241869.full.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/biorxiv/early/2018/01/09/241869.full.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It shows them to be near identical. This suggests there is some problem with the way you made the comparison/plot.&lt;/p&gt;&#xA;" OwnerUserId="4575" LastActivityDate="2019-05-14T22:14:15.647" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8636" PostTypeId="1" AcceptedAnswerId="8637" CreationDate="2019-05-15T08:49:38.920" Score="0" ViewCount="280" Body="&lt;p&gt;I would to use blastx from command line to analize a library produced with RepeatScout in format .fasta to find new transposable elements.&#xA;I use this command &lt;code&gt;mylaptop$ tblastx OB150DEC_2_repeats.fa&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but it reports an error: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;USAGE&#xA;  tblastx [-h] [-help] [-import_search_strategy filename]&#xA;    [-export_search_strategy filename] [-db database_name]&#xA;    [-dbsize num_letters] [-gilist filename] [-seqidlist filename]&#xA;    [-negative_gilist filename] [-entrez_query entrez_query]&#xA;    [-db_soft_mask filtering_algorithm] [-db_hard_mask filtering_algorithm]&#xA;    [-subject subject_input_file] [-subject_loc range] [-query input_file]&#xA;    [-out output_file] [-evalue evalue] [-word_size int_value]&#xA;    [-qcov_hsp_perc float_value] [-max_hsps int_value]&#xA;    [-xdrop_ungap float_value] [-searchsp int_value] [-sum_stats bool_value]&#xA;    [-max_intron_length length] [-seg SEG_options]&#xA;    [-soft_masking soft_masking] [-matrix matrix_name]&#xA;    [-threshold float_value] [-culling_limit int_value]&#xA;    [-best_hit_overhang float_value] [-best_hit_score_edge float_value]&#xA;    [-window_size int_value] [-lcase_masking] [-query_loc range]&#xA;    [-strand strand] [-parse_deflines] [-query_gencode int_value]&#xA;    [-db_gencode int_value] [-outfmt format] [-show_gis]&#xA;    [-num_descriptions int_value] [-num_alignments int_value]&#xA;    [-line_length line_length] [-html] [-max_target_seqs num_sequences]&#xA;    [-num_threads int_value] [-remote] [-version]&#xA;&#xA;DESCRIPTION&#xA;   Translated Query-Translated Subject BLAST 2.5.0+&#xA;&#xA;Use '-help' to print detailed descriptions of command line arguments&#xA;========================================================================&#xA;&#xA;Error: Too many positional arguments (1), the offending value: OB150DEC_2_repeats.fa&#xA;Error:  (CArgException::eSynopsis) Too many positional arguments (1), the offending value: OB150DEC_2_repeats.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't understand what does it mean &lt;code&gt;too many positional arguments&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="4475" LastActivityDate="2019-05-15T08:53:46.587" Title="blast command line" Tags="|blast|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8637" PostTypeId="2" ParentId="8636" CreationDate="2019-05-15T08:53:46.587" Score="1" Body="&lt;p&gt;A positional argument is one that doesn't follow something like &lt;code&gt;-foo&lt;/code&gt; or &lt;code&gt;--bar&lt;/code&gt;. That is, the variable that it should internally map to depends on it's position in the command. You're looking for something like &lt;code&gt;tblastx -db something -query OB150DEC_2_repeats.fa&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-05-15T08:53:46.587" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="8638" PostTypeId="1" CreationDate="2019-05-15T08:56:37.007" Score="2" ViewCount="1082" Body="&lt;p&gt;I want to perform a correlation test between genes in on my single cell RNA seq data set. I perfomed the differential expression analysis using the &lt;strong&gt;Seurat&lt;/strong&gt; version 2 package, after performing stages of normalisation, scaling, PCA, TSNE analyses and clustering. My correlation plot is far from what I expect and I want to know if the expression values obtained with &lt;strong&gt;Seurats&lt;/strong&gt; V2's &lt;strong&gt;FetchData&lt;/strong&gt; function (imputed, scaled, or raw) may have an effect on correlation analysis' outcome. This is what I did:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# load Seurat v2&#xA;library(Seurat)&#xA;&#xA;# load corrplot package&#xA;library(corrplot)&#xA;&#xA;# obtain expression values of the genes I want to test rom the Seurat object&#xA;expression_matrix &amp;lt;- FetchData(object, vars.all = c(&quot;gene1&quot;, &quot;gene2&quot;, &quot;gene3&quot;))&#xA;&#xA;expression_matrix &amp;lt;- round(expression_matrix , 1)&#xA;&#xA;cor_expression_matrix &amp;lt;- cor(expression_matrix)&#xA;&#xA;corrplot(cor_expression_matrix , is.corr = T, method = &quot;square&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which gene expression values should be best used for the analysis of correlation between genes: imputed, scaled, or raw?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for your kind response.&lt;/p&gt;&#xA;" OwnerUserId="3968" LastActivityDate="2024-01-27T06:06:30.857" Title="Which values from Seurat::FetchData function are to be used for correlation analysis between genes?" Tags="|seurat|single-cell|correlation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8639" PostTypeId="1" CreationDate="2019-05-15T11:05:24.860" Score="0" ViewCount="734" Body="&lt;p&gt;I am using Recount2 TCGA data and was wondering is it right to use GTEx data as control data for this. I would really appreciate your views on this?&lt;/p&gt;&#xA;" OwnerUserId="3766" LastEditorUserId="3766" LastEditDate="2019-05-15T15:13:05.740" LastActivityDate="2019-05-21T21:46:45.163" Title="Can we use GTEx data as control data for TCGA data?" Tags="|rna-seq|ngs|cancer|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8641" PostTypeId="2" ParentId="8616" CreationDate="2019-05-15T19:27:49.933" Score="1" Body="&lt;p&gt;Turns out, this is not possible. Concatenating two heatmaps causes their legends to be grouped, and while they can be positioned together at the top, bottom, at the left or the right sides, they cannot be separated spatially.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The discussion can be viewed at the bioconductor support link from the question. Relevant snippets:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;No, since you put the two heatmaps as one plot, the legends are grouped. You can either edit in Inkscape or make two heatmaps in two plots. You can get the column (order) of the first heatmap ... and assign to the second heatmap&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&amp;nbsp;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The thing is if you add two heatmaps, the legends are always plotted together. One way I can think to improve this is to move the legends to the bottom of the heatmap.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="650" LastActivityDate="2019-05-15T19:27:49.933" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8644" PostTypeId="1" AcceptedAnswerId="8647" CreationDate="2019-05-16T10:14:22.560" Score="3" ViewCount="21069" Body="&lt;p&gt;I have a large number of .fastq.gz files of different lane and reads. I have to merge them each reads group files into single .fastq.gz files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;**eg:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1st type&lt;/strong&gt;&lt;br&gt;&#xA;NA24694_GCCAAT_L001_R1_001.fastq.gz&lt;br&gt;&#xA;NA24694_GCCAAT_L001_R1_002.fastq.gz&lt;br&gt;&#xA;…....&lt;br&gt;&#xA;…....&lt;br&gt;&#xA;NA24694_GCCAAT_L001_R1_033.fastq.gz&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2nd type&lt;/strong&gt;&lt;br&gt;&#xA;NA24694_GCCAAT_L001_R2_001.fastq.gz&lt;br&gt;&#xA;NA24694_GCCAAT_L001_R2_002.fastq.gz&lt;br&gt;&#xA;…....&lt;br&gt;&#xA;…....&lt;br&gt;&#xA;NA24694_GCCAAT_L001_R2_027.fastq.gz&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3rd type&lt;/strong&gt;&lt;br&gt;&#xA;NA24694_GCCAAT_L002_R1_001.fastq.g&lt;br&gt;&#xA;NA24694_GCCAAT_L002_R1_002.fastq.g&lt;br&gt;&#xA;…....&lt;br&gt;&#xA;…....&lt;br&gt;&#xA;NA24694_GCCAAT_L002_R1_040.fastq.g    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;4th type&lt;/strong&gt;&lt;br&gt;&#xA;NA24694_GCCAAT_L002_R2_001.fastq.gz&lt;br&gt;&#xA;NA24694_GCCAAT_L002_R2_002.fastq.gz&lt;br&gt;&#xA;…....&lt;br&gt;&#xA;…....&lt;br&gt;&#xA;NA24694_GCCAAT_L002_R2_040.fastq.gz&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so now i need to merge all the corresponding files into single file.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output:&lt;/strong&gt;&lt;br&gt;&#xA;EA00694_GCCAAT_L001_R1.fastq.gz&lt;br&gt;&#xA;EA00694_GCCAAT_L001_R2.fastq.gz&lt;br&gt;&#xA;EA00694_GCCAAT_L002_R1.fastq.gz&lt;br&gt;&#xA;EA00694_GCCAAT_L002_R2.fastq.gz  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried &quot;&quot;cat'&quot; but its not possible to do for large files. Could any one help me to run in script parallel in perl or python or shell. or any linux command which can be used for large file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks all.&lt;/p&gt;&#xA;" OwnerUserId="1162" LastEditorUserId="96" LastEditDate="2019-05-20T13:17:41.723" LastActivityDate="2020-02-27T10:05:59.570" Title="How to merge .fastq.qz files into a single .fastq.gz with their same id without losing any content in parallel" Tags="|bash|shell|data-management|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8645" PostTypeId="2" ParentId="8628" CreationDate="2019-05-16T11:00:23.290" Score="2" Body="&lt;p&gt;Not exactly what you were looking for but may be you can use something like this?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ncbi2=NC_002021&#xA;efetch -db protein -format gpc -id &amp;quot;$ncbi2&amp;quot; \&#xA;    | xtract -pattern INSDSeq_feature-table \&#xA;        -group INSDFeature \&#xA;        -if INSDFeature_key -equals 'mat_peptide' \&#xA;        -tab '\n' \&#xA;        -element INSDFeature_location,INSDQualifier_value \&#xA;    | grep 'Segment 2'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For explorations of the INSDSet XMLs, &lt;code&gt;xtract&lt;/code&gt; has &lt;code&gt;-insd&lt;/code&gt; option that you can use. In this particular case, you can do something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;efetch -db nuccore -id &amp;quot;$ncbi2&amp;quot; -format gpc \&#xA;    | xtract -insd mat_peptide product peptide&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For more information, checkout &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/#_chapter6_Sequence_Records_&quot; rel=&quot;nofollow noreferrer&quot;&gt;EDirect documentation&lt;/a&gt; and some  &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/#_chapter6_Examples_&quot; rel=&quot;nofollow noreferrer&quot;&gt;example queries&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3577" LastEditorUserId="3848" LastEditDate="2020-12-17T16:28:32.750" LastActivityDate="2020-12-17T16:28:32.750" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8647" PostTypeId="2" ParentId="8644" CreationDate="2019-05-16T16:49:36.707" Score="10" Body="&lt;p&gt;All you need is &lt;code&gt;cat&lt;/code&gt;. You won't find any better tool for a simple job like this. Just run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat NA24694_GCCAAT_L001_R1*fastq.gz &amp;gt; EA00694_GCCAAT_L001_R1.fastq.gz&#xA;cat NA24694_GCCAAT_L001_R2*fastq.gz &amp;gt; EA00694_GCCAAT_L001_R2.fastq.gz&#xA;cat NA24694_GCCAAT_L001_R3*fastq.gz &amp;gt; EA00694_GCCAAT_L002_R1.fastq.gz&#xA;cat NA24694_GCCAAT_L001_R4*fastq.gz &amp;gt; EA00694_GCCAAT_L002_R2.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That should work just fine. You say you tried &lt;code&gt;cat&lt;/code&gt; and it &quot;didn't work&quot; but since you don't tell us how it failed, I can't really help. The only issues I can think of is that either there are too many files (as in several hundred thousand, whatever the value returned by &lt;code&gt;getconf ARG_MAX&lt;/code&gt; on your system which is 2097152 on mine) or, more likely, you are running out of disk space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it's a disk space issue, you might be able to get around it by adding each file and then deleting it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in 1 2 3 4; do &#xA;    for file in NA24694_GCCAAT_L001_R&lt;span class=&quot;math-container&quot;&gt;${i}_*fastq.gz; do&#xA;        cat &quot;$&lt;/span&gt;file&quot; &amp;gt;&amp;gt; EA00694_GCCAAT_L001_R&lt;span class=&quot;math-container&quot;&gt;${i}.fastq.gz &amp;amp;&amp;amp; rm &quot;$&lt;/span&gt;file&quot;&#xA;    done&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;IMPORTANT&lt;/em&gt;&lt;/strong&gt;: the command above is destructive. It will delete each file after it has been added to the new one. If everything works fine that's not a problem, but if I've made a mistake or your file names are slightly different you might lose your data. So I strongly urge you to simply run the &lt;code&gt;cat&lt;/code&gt; commands in the first section on a machine with enough disk space to deal with them.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2020-02-27T10:05:59.570" LastActivityDate="2020-02-27T10:05:59.570" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8649" PostTypeId="1" CreationDate="2019-05-17T19:35:23.303" Score="5" ViewCount="2301" Body="&lt;p&gt;I'm looking for a way to input a vcf or bed file (with specific base positions) and a bam file, and get the coverage at each base position (ie single base bins) using the bam file. I also want the strand information so ideal output would be something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr base_position total_coverage fwd_coverage rev_coverage&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've tried bamCoverage and bedtools Coverage but with bamCoverage, I can't figure out how to get single-base pair resolution and with bedtools Coverage, I can't figure out how to get the strand coverage information. &lt;/p&gt;&#xA;" OwnerUserId="4592" LastEditorUserId="57" LastEditDate="2019-05-18T11:54:41.553" LastActivityDate="2019-05-21T15:52:52.357" Title="How can I calculate coverage at single bases using a bam file?" Tags="|sam|bedtools|coverage|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8650" PostTypeId="1" AcceptedAnswerId="8653" CreationDate="2019-05-17T22:04:57.543" Score="1" ViewCount="258" Body="&lt;p&gt;I can do something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ tabix $&lt;/span&gt;{url} ${range}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where &lt;code&gt;${range}&lt;/code&gt; spans a region on a single chromosome, &lt;em&gt;e.g.&lt;/em&gt;, &lt;code&gt;chrA:X-Y&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can a &lt;code&gt;tabix&lt;/code&gt; query span multiple chromosomes, &lt;em&gt;e.g.&lt;/em&gt;, where &lt;code&gt;${range}&lt;/code&gt; looks like &lt;code&gt;chrA:X-chrN:Y&lt;/code&gt; (or similar), which returns data between &lt;code&gt;chrA:X&lt;/code&gt; and &lt;code&gt;chrN:Y&lt;/code&gt; (using either a lexicographical or natural sort order between &lt;code&gt;chrA&lt;/code&gt; and &lt;code&gt;chrN&lt;/code&gt;)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note: I do not want to pass in a BED file of intervals. If it can't be done without passing in a separate file, that's fine. I only want to provide a one-liner to give to someone who can run &lt;code&gt;tabix&lt;/code&gt; on their end, to do this type of query directly and remotely.&lt;/em&gt; &lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2019-05-17T22:11:43.897" LastActivityDate="2019-05-18T07:44:37.170" Title="Tabix queries spanning chromosomes" Tags="|tabix|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8651" PostTypeId="2" ParentId="8649" CreationDate="2019-05-17T23:49:00.650" Score="4" Body="&lt;p&gt;&lt;em&gt;Note: not yet tested, so there may be some additional fiddling with command line options needed&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The per-base depth can be obtained from &lt;em&gt;samtools depth&lt;/em&gt; (-a includes zero-coverage positions):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools depth -a in1.bam &amp;gt; depth_in1_both.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To split this by forward and reverse, you can use an initial pipe through &lt;em&gt;samtools view&lt;/em&gt; to exclude or include reverse-complement mappings:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -b -F 0x10 in1.bam | samtools depth -a &amp;gt; depth_in1_fwd.tsv&#xA;samtools view -b -f 0x10 in1.bam | samtools depth -a &amp;gt; depth_in1_rev.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To filter for particular regions, this can be pre-piped through &lt;em&gt;bedtools intersect&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools intersect -a in1.bam -b regions.bed | \&#xA;  samtools view -b -F 0x10 in1.bam | \&#xA;  samtools depth -a &amp;gt; depth_in1_regions_fwd.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Combining both, forward, and reverse is a bit trickier. I'd recommend doing it in a more manual fashion in R. But if you want to keep to the command line...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are absolutely sure that the 'depth' output will appear in the same order in all files (it might be worth using &lt;code&gt;samtools depth -aa&lt;/code&gt;), then the lines can be &lt;em&gt;paste&lt;/em&gt;d, with anonymous pipes, with unnecessary fields filtered out using &lt;em&gt;cut&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;paste &amp;lt;(samtools depth -aa in1.bam) \&#xA;      &amp;lt;(samtools view -b -F 0x10 in1.bam | samtools depth -aa) \&#xA;      &amp;lt;(samtools view -b -f 0x10 in1.bam | samtools depth -aa) | \&#xA;  cut -f 1,2,3,6,9 &amp;gt; depth_in1_bfr.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or equivalently, if using intermediate files that have been generated already:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;paste depth_in1_both.tsv depth_in1_fwd.tsv depth_in1_rev.tsv |&#xA;  cut -f 1,2,3,6,9 &amp;gt; depth_in1_bfr.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It would also be possible to use &lt;em&gt;join&lt;/em&gt; to combine the files (in cases where file output is &lt;em&gt;not&lt;/em&gt; consistent), but the first two fields of each file would then need to be combined, because 'join' only joins on the first field... but if you need to get that complex, it'd be better to use something other than pipes on the command line.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2019-05-17T23:49:00.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8652" PostTypeId="2" ParentId="8649" CreationDate="2019-05-18T02:38:35.590" Score="4" Body="&lt;pre&gt;&lt;code&gt;git clone https://github.com/lh3/htsbox&#xA;cd htsbox &amp;amp;&amp;amp; make&#xA;./htsbox pileup -cCf ref.fa aln.bam | less -S&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This output a VCF containing positions covered by at least one read. Tag &lt;code&gt;ADF&lt;/code&gt; and &lt;code&gt;ADR&lt;/code&gt; give the forward/reverse depth for each allele. You can combine them to get the total counts on both strands.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2019-05-18T02:38:35.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8653" PostTypeId="2" ParentId="8650" CreationDate="2019-05-18T07:44:37.170" Score="3" Body="&lt;p&gt;Looking at the &lt;a href=&quot;https://github.com/samtools/htslib/blob/5c7ff9c6812f1e6ef1fd83ba73b8e42cfbabba93/hts.c#L2627-L2645&quot; rel=&quot;nofollow noreferrer&quot;&gt;htslib source code&lt;/a&gt;, it seems like the answer is no, a single region specifier cannot span multliple chromosomes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seems like the closest you can get is to enumerate all the chromosomes as separate regions on the &lt;a href=&quot;http://www.htslib.org/doc/tabix.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;command line&lt;/a&gt;. So to get everything between &lt;code&gt;chrA:X&lt;/code&gt; and &lt;code&gt;chrN:Y&lt;/code&gt;, you would have to write:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tabix inputfile.gz chrA:X chrB chrC ... chrN:1-Y&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(If you omit the end coordinate like &lt;code&gt;chrA:X&lt;/code&gt;, the region encompasses &lt;code&gt;chrA&lt;/code&gt; from the specified coordinate to the end. If only specify the chromosome name without the coordinates, like &lt;code&gt;chrB&lt;/code&gt;, the region encompasses the whole named chromosome, and for the last chromosome, you can specify up to the desired point by starting from the beginning: &lt;code&gt;chrN:1-Y&lt;/code&gt;.) &lt;/p&gt;&#xA;" OwnerUserId="3351" LastActivityDate="2019-05-18T07:44:37.170" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8654" PostTypeId="1" CreationDate="2019-05-18T17:42:30.427" Score="1" ViewCount="91" Body="&lt;p&gt;I am trying to understand metagenomeSeq, I will be very much thankful if you could help me to understand this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do have three treatment condition (T1, T2, and T3) and want to see how these treatments affect the microbial diversity in rice roots. I am interested in the pairwise comparison between them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used below code but I am not sure whether it is right or wrong, I do also have few queries regarding below code. I initially imported whole data into phyloseq and then filter the root data and convert into metagenomeSeq object.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;root_data &amp;lt;- phyloseq_to_metagenomeSeq(root_M_filter)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;DO I need to used normalize count for further processing? what is setting?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;normalizedMatrix &amp;lt;- MRcounts(root_data, norm = TRUE)&#xA;settings = zigControl(maxit = 1, verbose = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is it correct way to make model matrix?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mod = model.matrix(~root_data&lt;span class=&quot;math-container&quot;&gt;$Treatment)&#xA;colnames(mod) = levels(root_data$&lt;/span&gt;Treatment)&#xA;res = fitZig(obj = root_data, mod = mod, control = settings)&#xA;res&#xA;An object of class &quot;fitZigResults&quot;&#xA;Slot &quot;fit&quot;:&#xA;An object of class &quot;MArrayLM&quot;&#xA;$coefficients&#xA;                                      T1           T2          T3 &#xA;71f84e7910006f22684121564206e8ca -2.8129112 -0.321366092 0.06113294&#xA;54e204fb99c80764e964456dadd6a0e5 0.5643061 0.019018800 1.01351768&#xA;55cd1fc570879d645bbf7a3642e9b0a8 -3.1159264 0.072348076 0.02205983&#xA;65f5c31e12c277aec319e2096463f9d2 -0.1355134 -0.004567206 0.53119153&#xA;7bed62f0fef250fd831dcf13bf43f4fc 1.9469670 -0.596315540 -0.35124338&#xA;                             scalingFactor &#xA;71f84e7910006f22684121564206e8ca 1.62101725&#xA;54e204fb99c80764e964456dadd6a0e5 -0.24292482&#xA;55cd1fc570879d645bbf7a3642e9b0a8 1.50462644&#xA;65f5c31e12c277aec319e2096463f9d2 0.05833637&#xA;7bed62f0fef250fd831dcf13bf43f4fc -0.56247683&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is design here?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;finalMod = slot(res, &quot;fit&quot;)$design&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is correct option for contract.make, or should I also involved some&#xA;more options?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;contrast.matrix = makeContrasts(T1 - T2, levels = finalMod)&#xA;fit2 = contrasts.fit(zigFit, contrast.matrix)&#xA;fit2 = eBayes(fit2)&#xA;topTable(fit2)&#xA;                                 logFC   AveExpr         t      P.Value &#xA;3147790f0d5a78316fb9dd64f53b9473 11.262064 14.091887 16.331949&#xA;3.482815e-15&#xA;d2ec9f3b77975c0f457e4b7413b217ff 9.466087 12.212476 10.606122&#xA;6.159311e-11&#xA;7b908379d1fd2e75d84fd9aaecfd3f77 6.907182 9.784971 9.163780&#xA;1.267984e-09&#xA;d5fdc1df6dfd31acce9a34b9d8a76590 5.404459 8.373729 7.035926&#xA;1.802316e-07&#xA;19ddf8e59673c884734cf6093c672da0 5.381863 8.449413 7.011910&#xA;1.912363e-07&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="1008" LastEditDate="2019-05-24T08:06:33.067" LastActivityDate="2019-05-24T08:06:33.067" Title="Queries regarding MetagenomeSeq code" Tags="|bioconductor|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8655" PostTypeId="1" AcceptedAnswerId="8657" CreationDate="2019-05-19T09:10:18.413" Score="2" ViewCount="58" Body="&lt;p&gt;I have mass-spectrometry data (Human cellsanalyzed and annotated by &lt;a href=&quot;http://coxdocs.org/doku.php?id=maxquant:start&quot; rel=&quot;nofollow noreferrer&quot;&gt;MaxQuant&lt;/a&gt;).&#xA;I want to find which peptides are covering positions of former splice junctions (on the mRNA sequence), based on protein id and peptide positions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where could I find an appropriate data? I want to know, based on peptide-sequence, annotaion and position on protein, whether this peptide is a single exon, or if it contained areas of former splice junctions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Itai&lt;/p&gt;&#xA;" OwnerUserId="4565" LastEditorUserId="3848" LastEditDate="2019-05-19T16:30:58.200" LastActivityDate="2019-05-19T17:05:30.827" Title="Alternative splicing positions on protein" Tags="|proteins|isoform|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8656" PostTypeId="2" ParentId="8638" CreationDate="2019-05-19T15:31:16.590" Score="0" Body="&lt;p&gt;You should use normalized data. The function has a parameter slot=&amp;quot;data&amp;quot; that let's you select to access the normalized data.&lt;/p&gt;&#xA;&lt;p&gt;Any reason for rounding to 1 decimal place here? I imagine that would have a substantial impact on the correlation you compute.&lt;/p&gt;&#xA;" OwnerUserId="3415" LastEditorUserId="73" LastEditDate="2021-05-12T01:18:59.403" LastActivityDate="2021-05-12T01:18:59.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8657" PostTypeId="2" ParentId="8655" CreationDate="2019-05-19T16:14:58.827" Score="0" Body="&lt;p&gt;If you are considering the human genome as &quot;data&quot; (not mRNA), I would run your single peptide via &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PROGRAM=tblastn&amp;amp;PAGE_TYPE=BlastSearch&amp;amp;LINK_LOC=blasthome&quot; rel=&quot;nofollow noreferrer&quot;&gt;tblastn&lt;/a&gt; which will take your protein(s) translate them to nucleotides and run a blast against Genbank. The splice sites will be immediately apparent because they force indels in your query. If there are no indels there are no introns (splice sites). If you want to process several proteins, one solution is to use eblast, a command line version of tblastn  (Anaconda package). You can confirm this against a full genome annotation (note, I don't work on humans).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alot of work has been done on splice sites, a friend worked on this, but I think you just want to focus on a restricted number of target peptides in your mass spec. output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;mRNA should be avoided IMO if you are looking for splice sites, i.e. intron boundaries.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-05-19T17:05:30.827" LastActivityDate="2019-05-19T17:05:30.827" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8658" PostTypeId="1" CreationDate="2019-05-20T08:58:55.493" Score="0" ViewCount="844" Body="&lt;p&gt;I have a group of cells that have yellow (EYFP) expression and cells that don't. I want to define the yellow and non-yellow genes in two cell identities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I clustered the cells and then created Violin Plot for EYFP expression. Then I did FindMarkers between clusters that express high EYFP with low EYFP expression. However, this is not a very accurate way to isolate cells of high and low EYFP expression. I need to use ident to separate EYFP +ve and EYFP -ve cells (but don't know the exact syntax). Then FindMarkers between these two new identities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to subset a certain set of barcodes that express a marker gene (in this case Olfm4).&lt;/p&gt;&#xA;" OwnerUserId="4600" LastEditorUserId="48" LastEditDate="2019-05-22T09:56:42.547" LastActivityDate="2021-02-10T16:07:30.767" Title="How to set new identities for new clusters based on a gene expression in Seurat?" Tags="|seurat|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8659" PostTypeId="1" CreationDate="2019-05-20T14:50:32.390" Score="0" ViewCount="435" Body="&lt;p&gt;I want to define the identity of some cells based on a gene marker (for example the colour yellow&amp;lt; EYFP gene). One group of cells has the EYFP gene the other doesn't and I want to find the differential gene expression between those groups that are found on the same cell sample. Do you have any tips for a code in Seurat? Thanks&lt;/p&gt;&#xA;" OwnerUserId="4600" LastActivityDate="2019-06-19T15:03:25.400" Title="Differential expression between two types of cells of specific marker gene expression" Tags="|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8660" PostTypeId="2" ParentId="8659" CreationDate="2019-05-20T14:59:47.953" Score="1" Body="&lt;p&gt;Take a look at the section &lt;strong&gt;&quot;Finding Differentialy Expressed Feautres (cluster biomarkers)&quot;&lt;/strong&gt; of &lt;a href=&quot;https://satijalab.org/seurat/v3.0/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seurat tutorial&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have to define your clusters (if not computed automatically) by modifying the &lt;strong&gt;&lt;em&gt;ident&lt;/em&gt;&lt;/strong&gt; slot of the Seurat object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you call the function &lt;em&gt;FindMarkers&lt;/em&gt; of your Seurat object (to define markers of a given cluster against the rest) or &lt;em&gt;FindAllMarkers&lt;/em&gt; (to look for all the combinations) &lt;/p&gt;&#xA;" OwnerUserId="678" LastActivityDate="2019-05-20T14:59:47.953" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="8661" PostTypeId="2" ParentId="8644" CreationDate="2019-05-20T16:32:47.027" Score="3" Body="&lt;p&gt;We sequence DNA on daily basis. Like &lt;code&gt;terdon&lt;/code&gt; said, you can use &lt;code&gt;cat&lt;/code&gt; to group all the FastQ of all your sample whether they are bgzipped or not like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;root@slurm1: ~  # ls&#xA;am224_S6_L001_R1_001.fastq.gz&#xA;am224_S6_L001_R2_002.fastq.gz&#xA;am224_S6_L002_R1_003.fastq.gz&#xA;am224_S6_L002_R2_004.fastq.gz&#xA;am232_S10_L001_R1_001.fastq.gz&#xA;am232_S10_L001_R2_002.fastq.gz&#xA;am232_S10_L002_R1_003.fastq.gz&#xA;am232_S10_L002_R2_004.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You could do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# initiate am empty array&#xA;listSamples=()&#xA;&#xA;# loop to add the unique samples to an array&#xA;for i in *.fastq.gz;&#xA;do&#xA;    if [[ &quot;&lt;span class=&quot;math-container&quot;&gt;$sampleName&quot; =~ &quot;$&lt;/span&gt;{listSamples[@]}&quot; ]]; then&#xA;        sampleName=&lt;span class=&quot;math-container&quot;&gt;$(echo $&lt;/span&gt;i | cut -d'_' -f1)&#xA;        listSamples+=(&quot;$sampleName&quot;)&#xA;    fi&#xA;done&#xA;&#xA;# loop in the newly created array of samples and group them by reads number&#xA;for i in &quot;&lt;span class=&quot;math-container&quot;&gt;${listSamples[@]}&quot;&#xA;do&#xA;    cat &quot;$&lt;/span&gt;i&quot;_*_R1_* &amp;gt; &quot;&lt;span class=&quot;math-container&quot;&gt;$i&quot;_1.fastq.gz&#xA;    cat &quot;$&lt;/span&gt;i&quot;_*_R2_* &amp;gt; &quot;$i&quot;_2.fastq.gz&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I highly suggest you to compress your files as &lt;code&gt;.bz2&lt;/code&gt; to save storage as much as possible. You can achieve this using the multithreaded tool &lt;a href=&quot;http://lbzip2.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;lbzip2&lt;/a&gt;. Assuming that you have 10 CPUs available, you can do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in &quot;${listSamples[@]}&quot;&#xA;do&#xA;    bzcat &quot;$i&quot;_*_R1_* | lbzip2 -n 10 --best &amp;gt; &quot;$i&quot;_1.fastq.bz2&#xA;    bzcat &quot;$i&quot;_*_R2_* | lbzip2 -n 10 --best &amp;gt; &quot;$i&quot;_2.fastq.bz2&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Hope this helps !&lt;/p&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2019-05-20T16:32:47.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8662" PostTypeId="1" CreationDate="2019-05-21T04:55:09.997" Score="0" ViewCount="547" Body="&lt;p&gt;I have a multiplexed lane of GBS sequencing reads as a fastq file. I understand the first step is to demultiplex and trim the adapter sequences from the reads.&lt;/p&gt;&#xA;&lt;p&gt;This yields many individual fastq files that correspond to individual libraries.&lt;/p&gt;&#xA;&lt;p&gt;I would like to genotype my samples, but I need to map the reads first. My question is how should I perform the BWA MEM alignment?&lt;/p&gt;&#xA;&lt;p&gt;Do I perform BWA MEM on each of the resulting fastq files that correspond to individual libraries...&lt;/p&gt;&#xA;&lt;p&gt;OR&lt;/p&gt;&#xA;&lt;p&gt;I should I perform BWA MEM on the large original fastq file that contains all my sequence reads before demux and read trimming?&lt;/p&gt;&#xA;&lt;p&gt;Forgive my ignorance, I've only ever used TASSEL (which works great) and I'm trying to expand my understanding and flexibility.&lt;/p&gt;&#xA;" OwnerUserId="4607" LastEditorUserId="57" LastEditDate="2021-02-15T16:40:41.453" LastActivityDate="2021-02-15T16:40:41.453" Title="Proper use of BWA MEM on multiplexed GBS sample" Tags="|bwa|genotyping|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8663" PostTypeId="2" ParentId="8662" CreationDate="2019-05-21T08:15:26.807" Score="0" Body="&lt;p&gt;Trimming is something you always want to do before mapping to make sure that the bases at each position are of sufficient quality, and to make sure that you are not aligning your adapters to the reference genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not entirely sure what the biological difference is represented in your individual libraries (that depends on your research question). But most likely you want to demultiplex the reads before mapping with BWA as well. As a result you would indeed get a sam/bam file per library. Let me know if anything is still unclear!&lt;/p&gt;&#xA;" OwnerUserId="4552" LastActivityDate="2019-05-21T08:15:26.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8664" PostTypeId="2" ParentId="8662" CreationDate="2019-05-21T08:19:30.897" Score="1" Body="&lt;p&gt;The first thing to do is to group the FastQ files together following this &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/8644/how-to-merge-fastq-qz-files-into-a-single-fastq-gz-with-their-same-id-without&quot;&gt;post&lt;/a&gt;. Once you have your reads grouped, you could trim your files before mapping. Then you could perform your BWA alignment. Since you mentioned &lt;code&gt;MEM&lt;/code&gt;, I suppose your read length should be between 70-100bp as suggested by the author:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;It consists of three algorithms: BWA-backtrack, BWA-SW and BWA-MEM.&#xA;The first algorithm is designed for Illumina sequence reads up to 100bp, while the rest two for&#xA;longer sequences ranged from 70bp to 1Mbp. BWA-MEM and BWA-SW share similar features such &#xA;as long-read support and split alignment, but BWA-MEM, which is the latest, &#xA;is generally recommended for high-quality queries as it is faster and more accurate.&#xA;BWA-MEM also has better performance than BWA-backtrack for 70-100bp Illumina reads.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then you could run your BWA MEM, supposing you have 10 CPUs available, like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bwa mem \                                  # bwa mem algorithm&#xA;-M \                                       # mark shorter split hits as secondary (for Picard compatibility).&#xA;-t 10 \                                    # CPUs thread&#xA;reference.fasta \                          # reference genome to be aligned to&#xA;&amp;lt; sample_r1.fastq \                        # fastQ read 1 (can be .gz)&#xA;&amp;lt; sample_r2.fastq | \                      # optional FastQ read 2 (can be .gz)&#xA;samtools view -bS -@ 10 &amp;gt; sample.bam       # pipe to samtools to convert SAM to BAM on the fly.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Hope this helps !&lt;/p&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="2108" LastEditDate="2021-02-15T14:15:32.853" LastActivityDate="2021-02-15T14:15:32.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8665" PostTypeId="1" CreationDate="2019-05-21T09:04:58.797" Score="1" ViewCount="21" Body="&lt;p&gt;I mutated a wild type protein structure using Scwrl4. How can I evaluate or validate the reliability of the predicted structure generated by Scwrl4?&lt;/p&gt;&#xA;" OwnerUserId="3760" LastEditorUserId="298" LastEditDate="2019-05-21T14:04:07.753" LastActivityDate="2019-05-21T14:04:07.753" Title="Possible evaluation or validation of predicted 3D protein structure generated by SCWRL4" Tags="|protein-structure|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8667" PostTypeId="2" ParentId="8649" CreationDate="2019-05-21T15:52:52.357" Score="0" Body="&lt;p&gt;Brent Pedersen (&lt;a href=&quot;https://twitter/com/brentp&quot; rel=&quot;nofollow noreferrer&quot;&gt;@brentp&lt;/a&gt;) claims &lt;a href=&quot;https://github.com/brentp/mosdepth&quot; rel=&quot;nofollow noreferrer&quot;&gt;mosdepth&lt;/a&gt; is 2x as fast as &lt;code&gt;samtools depth&lt;/code&gt;. I'm not sure whether this provides strand information, but it might be the fastest tool available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another tool of possible interest (also written by Brent) is &lt;a href=&quot;https://github.com/brentp/goleft/tree/master/indexcov#indexcov&quot; rel=&quot;nofollow noreferrer&quot;&gt;indexcov&lt;/a&gt;. This doesn't give single bp precision, but is able to provide even quicker estimates of coverage at 16Kb resolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Installing these tools from source is non-trivial, and I struggled with installation in the past. However, they are both now available in bioconda, which should make installation simple.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-05-21T15:52:52.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8668" PostTypeId="2" ParentId="8639" CreationDate="2019-05-21T16:16:17.403" Score="1" Body="&lt;p&gt;Definately not directly. Probably not even if used indirectly. Different projects, done at different times, with different protocols, different machines, different library preps etc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;We have tested combining GTEx and TCGA data for several different purposes where one might expect the analyses' internal controls would have allowed compatibilty, but even then we have always found a persistent batch effect which we were unable to remove. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2019-05-21T16:16:17.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8669" PostTypeId="2" ParentId="8639" CreationDate="2019-05-21T21:46:45.163" Score="1" Body="&lt;p&gt;I am not sure how appropriate it is, but GTEx data was used as the normal controls for TCGA data in &lt;a href=&quot;https://doi.org/10.1038/s41467-017-01027-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;Comprehensive analysis of normal adjacent to tumor transcriptomes&lt;/a&gt;. It was not just a single panel, but a major part of the paper.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, there is &lt;a href=&quot;https://github.com/mskcc/RNAseqDB&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNAseqDB&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Here, we developped a pipeline that processes and unifies RNA-seq data&#xA;  from different studies. Using the pipeline, we have processed data&#xA;  from the GTEx and TCGA and have successfully corrected for&#xA;  study-specific biases, allowing comparative analysis across studies.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="35" LastActivityDate="2019-05-21T21:46:45.163" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8670" PostTypeId="1" CreationDate="2019-05-22T04:44:25.583" Score="3" ViewCount="242" Body="&lt;p&gt;Could you please suggest suitable R packages for statistical coupling and direct coupling analysis of protein sequences?&lt;br&gt;&#xA;I know there are Python packages for those: &lt;a href=&quot;https://reynoldsk.github.io/pySCA&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;pySCA&quot;&gt;SCA&lt;/a&gt; and &lt;a href=&quot;https://github.com/giopina/pydca&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;pyDCA&quot;&gt;DCA&lt;/a&gt;.&#xA;I was looking for similar packages in R so that I can continue with my present R workflow.&lt;/p&gt;&#xA;" OwnerUserId="4612" LastEditorUserId="48" LastEditDate="2019-05-24T08:45:20.877" LastActivityDate="2019-08-10T01:03:10.397" Title="Protein SCA and DCA" Tags="|r|sequence-analysis|sequence-homology|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8672" PostTypeId="2" ParentId="8658" CreationDate="2019-05-22T13:38:55.150" Score="1" Body="&lt;p&gt;You have to define your &lt;code&gt;ident&lt;/code&gt; it has to have the structure of a table. You put the cell names as names and clusters as value (for instance cell 1 = YEPF+ cell 2 = YEFP-):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pident=as.factor(clusters)&#xA;names(pident)=cellNames    &#xA;object1@ident=pident&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you can run the &lt;code&gt;FindAllMarkers&lt;/code&gt; function on this object:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FindAllMarkers(object1, min.pct = 0.25, min.diff.pct = 0.25)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="678" LastActivityDate="2019-05-22T13:38:55.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8675" PostTypeId="1" AcceptedAnswerId="8679" CreationDate="2019-05-23T04:08:14.763" Score="1" ViewCount="53" Body="&lt;p&gt;Which type of genomic elements in the UCSC browser can be used as an example of the highly labile = highly variable = low conservation regions?&lt;/p&gt;&#xA;" OwnerUserId="4617" LastActivityDate="2019-05-23T08:09:04.937" Title="Highly variable elements?" Tags="|genomics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8678" PostTypeId="1" AcceptedAnswerId="8680" CreationDate="2019-05-23T08:03:01.917" Score="2" ViewCount="1461" Body="&lt;p&gt;In my scRNA seq experiment, single-cell libraries were generated using the GemCode Single-Cell Instrument and Single Cell 3′ Library &amp;amp; Gel Bead Kit v2 and Chip Kit (10x Genomics) according to the manufacturer’s protocol. About 8700 cells were added to each channel with a targeted cell recovery estimate of 5000 cells. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, after processing my raw data with &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/count&quot; rel=&quot;nofollow noreferrer&quot;&gt;CellRanger&lt;/a&gt; software, I get 737,280 cells (barcodes) and about 31,000 genes. I cannot understand why I get so many cells and why this barcode whitelist is not up to 10,000 cells. After filtering this data retaining genes expressed in &gt;= 5 cells as well as retaining cells with at least 200 detected genes, I get around 2200 cells. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4620" LastEditorUserId="48" LastEditDate="2019-05-23T13:31:28.567" LastActivityDate="2019-05-25T11:38:47.560" Title="Why do I have &gt;10,000 cells in the 10X matrix produced by cellranger?" Tags="|scrnaseq|10x-genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8679" PostTypeId="2" ParentId="8675" CreationDate="2019-05-23T08:09:04.937" Score="1" Body="&lt;p&gt;Transposable elements (TEs) are the classic example of rapidly evolving regions in eukaryotic genomes. These are often used as a proxy for neutrally evolving sequences, which are by definition not conserved/constrained. The UCSC genome browser includes various repeat tracks (e.g. RepeatMasker) that includes TEs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Within protein coding regions, 4-fold degenerate sites are often used as a proxy for neutrally evolving sequences. These sites are where any nucleotide at this position leads to the same amino acid and therefore protein (synonymous mutations).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Potentially even more variable sequences would be those under positive selection, but there is not a single class of element that falls into this category i think.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2019-05-23T08:09:04.937" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8680" PostTypeId="2" ParentId="8678" CreationDate="2019-05-23T13:45:34.880" Score="3" Body="&lt;p&gt;The raw data from cell ranger contains all of the barcodes detected in the experiment.  These raw barcodes are filtered by cell ranger to identify the barcodes that likely represent cells rather than empty droplets / dead cells. You can find the filtered data in the filtered &lt;code&gt;feature-barcode matrices&lt;/code&gt;.  You can also examine the &lt;code&gt;Run summary HTML&lt;/code&gt; file to see a summary of the cell ranger barcode filtering. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason why there are so many raw barcodes is an important optimization by 10X to increase the cell capture rate compared to techniques such as drop-seq.  In a 10X experiment the number of gel beads is much higher than the number of cells which leads to a higher probability of a cell and a bead ending up in the same droplet.  This also leads to a large number of empty droplets with just a bead and some ambient RNA or a dead cell.  These all end up being sequenced and the empty drops tend to have a small number of unique molecules (the majority less than 10 UMIs / barcode).&lt;/p&gt;&#xA;" OwnerUserId="64" LastEditorUserId="64" LastEditDate="2019-05-23T13:58:28.600" LastActivityDate="2019-05-23T13:58:28.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8681" PostTypeId="2" ParentId="2649" CreationDate="2019-05-23T15:39:34.233" Score="2" Body="&lt;p&gt;In cases where there is no sequence wrapping and each sequence occupies only a single line, the following shell command is probably going to be fastest, easiest, and most convenient.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;paste - - &amp;lt; your.fasta &amp;gt; your.new.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-05-23T15:39:34.233" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8682" PostTypeId="2" ParentId="6871" CreationDate="2019-05-23T17:08:18.930" Score="4" Body="&lt;p&gt;There are three initiatives I know of to have a go at this:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://pconsfam.bioinfo.se/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PConsFam&lt;/a&gt;, which collects data from &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/33/14/i23/3953955&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The Baker group's &lt;a href=&quot;https://science.sciencemag.org/content/355/6322/294&quot; rel=&quot;nofollow noreferrer&quot;&gt;metagenomic study&lt;/a&gt; which you mention in the question.&lt;/li&gt;&#xA;&lt;li&gt;The recent &lt;a href=&quot;https://arxiv.org/abs/1811.12355&quot; rel=&quot;nofollow noreferrer&quot;&gt;DMPfold&lt;/a&gt; work from our lab. This compares its results to the above two studies and discusses the effect on various model organisms.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The second and third of these don't provide a database per se, but the models are available for download and are linked to the Pfam database.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: things have moved on significantly since I first wrote this answer. As other answers mention, the &lt;a href=&quot;https://alphafold.ebi.ac.uk&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaFold Protein Structure Database&lt;/a&gt; is now the best resource in this area.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastEditorUserId="4622" LastEditDate="2022-01-10T11:56:42.923" LastActivityDate="2022-01-10T11:56:42.923" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8683" PostTypeId="1" CreationDate="2019-05-24T02:53:56.180" Score="1" ViewCount="93" Body="&lt;p&gt;I ran &lt;a href=&quot;https://github.com/etabari/PorthoMCL/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PorthoMCL&lt;/a&gt; but I could not find an explanation about the output format and I would like to visualise the results, maybe as a &lt;a href=&quot;http://circos.ca/&quot; rel=&quot;nofollow noreferrer&quot;&gt;circos plot&lt;/a&gt;. PorthoMCL produces two output files:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/etabari/PorthoMCL/blob/master/sample/8.all.ort.group&quot; rel=&quot;nofollow noreferrer&quot;&gt;Orthologs&lt;/a&gt; sample output file and&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/etabari/PorthoMCL/blob/master/sample/8.all.par.group&quot; rel=&quot;nofollow noreferrer&quot;&gt;Paralogs&lt;/a&gt;  sample output.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To create the above outputs they used &lt;a href=&quot;https://github.com/etabari/PorthoMCL/blob/master/sample/taxon_list&quot; rel=&quot;nofollow noreferrer&quot;&gt;12 genes&lt;/a&gt; as input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are the first 5 lines of each file to illustrate the format:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Orthologs&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NC_000913|NP_416270.1   NC_000916|NP_276057.1   NC_000913|NP_415023.1   NC_000916|NP_275620.1   NC_000913|NP_416372.1   NC_000916|NP_275748.1   NC_000913|NP_418459.1   NC_000913|NP_415400.1   NC_000916|NP_275839.1   NC_000913|YP_026282.1   NC_000913|NP_415281.1   NC_000913|NP_415120.1   NC_000916|NP_276816.1   NC_000913|NP_415635.4   NC_000913|NP_415958.1   NC_000913|NP_417907.2   NC_000916|NP_276221.1   NC_000913|NP_414796.2   NC_000913|NP_415028.1   NC_000913|NP_416705.2   NC_000913|NP_417937.1   NC_000913|NP_416000.1   NC_000916|NP_275276.1   NC_000913|YP_025300.1   NC_000913|NP_416809.1   NC_000913|NP_415315.4   NC_000916|NP_276601.1   NC_000913|NP_415807.1   NC_000916|NP_276150.1   NC_000913|NP_414669.1   NC_000913|NP_417997.1   NC_000913|NP_415286.1   NC_000913|NP_417668.1   NC_000913|NP_414900.1   NC_000913|NP_414693.1   NC_000913|NP_415644.1   NC_000913|NP_415453.1   NC_000913|NP_415385.1   NC_000913|NP_417737.1   NC_000913|NP_416224.1   NC_000913|NP_416685.1   NC_000913|NP_415763.1   NC_000913|NP_417163.1   NC_000913|NP_415185.1   NC_000913|NP_416917.1   NC_000913|NP_416001.1   NC_000916|NP_276486.1   NC_000913|NP_414741.1   NC_000913|NP_415330.1   NC_000913|NP_416427.1   NC_000913|NP_415376.4   NC_000913|YP_026225.1   NC_000913|NP_418707.1   NC_000913|NP_417998.1   NC_000913|NP_414608.1   NC_000913|NP_415350.2   NC_000913|NP_418520.1   NC_000913|NP_415806.1   NC_000913|NP_416633.1   NC_000913|NP_417920.1   NC_000913|NP_417936.1   NC_000913|NP_418530.1   NC_002570|NP_243996.1   NC_002570|NP_244512.1   NC_002570|NP_242964.1   NC_002570|NP_242329.1   NC_002570|NP_241947.1   NC_002570|NP_244161.1   NC_002570|NP_244348.1   NC_002570|NP_244469.1   NC_002570|NP_241038.1   NC_002570|NP_241216.1   NC_002570|NP_241378.1   NC_002570|NP_242006.1   NC_002570|NP_244540.1   NC_002570|NP_244761.1   NC_002570|NP_243136.1   NC_002570|NP_241905.1   NC_002570|NP_240894.1   NC_002570|NP_241920.1   NC_002570|NP_241942.1   NC_002570|NP_241217.1   NC_002570|NP_242260.1   NC_002570|NP_240893.1   NC_002570|NP_242077.1   NC_002570|NP_241436.1   NC_002570|NP_244513.1   NC_002570|NP_241306.1   NC_002570|NP_243988.1   NC_003080|NP_203144.1   NC_006512|YP_155450.1   NC_006512|YP_155127.1   NC_006512|YP_154507.1   NC_006512|YP_154789.1   NC_006512|YP_154619.1   NC_006512|YP_156439.1   NC_006512|YP_155126.1   NC_006512|YP_155546.1   NC_006512|YP_155904.1   NC_006512|YP_154704.1   NC_006512|YP_154666.1   NC_006512|YP_154873.1   NC_006512|YP_156149.1   NC_006512|YP_155495.1   NC_006512|YP_156640.1   NC_009901|YP_001501520.1    NC_009901|YP_001503463.1    NC_009901|YP_001503247.1    NC_009901|YP_001502530.1    NC_009901|YP_001503231.1    NC_009901|YP_001503183.1NC_009901|YP_001500118.1    NC_009901|YP_001500809.1    NC_009901|YP_001503824.1    NC_009901|YP_001502529.1    NC_009901|YP_001500708.1    NC_009901|YP_001501695.1    NC_009901|YP_001504047.1NC_009901|YP_001503376.1    NC_009901|YP_001501406.1    NC_009901|YP_001500078.1    NC_009901|YP_001500551.1    NC_016147|YP_004931203.1    NC_016147|YP_004931685.1    NC_016147|YP_004930231.1NC_016147|YP_004929818.1    NC_016147|YP_004928762.1    NC_016147|YP_004931616.1    NC_016147|YP_004929081.1    NC_016147|YP_004930265.1    NC_016147|YP_004931494.1    NC_016147|YP_004928942.1NC_016147|YP_004930600.1    NC_016147|YP_004930540.1    NC_016147|YP_004928882.1    NC_016147|YP_004931319.1    NC_017248|YP_005602919.1    NC_017248|YP_005602966.1    NC_017248|YP_005603551.1NC_017248|YP_005603622.1    NC_017248|YP_005604109.1    NC_017248|YP_005604328.1    NC_017248|YP_005604706.1    NC_017248|YP_005604668.1    NC_017248|YP_005604330.1    NC_017248|YP_005604329.1NC_017248|YP_005603027.1    NC_017248|YP_005603041.1    NC_017248|YP_005604355.1    NC_017248|YP_005604414.1    NC_017248|YP_005604665.1    NC_017248|YP_005604493.1    NC_017248|YP_005602900.1NC_020238|YP_007420349.1    NC_020238|YP_007421075.1    NC_020238|YP_007420090.1    NC_020238|YP_007419317.1    NC_020238|YP_007420030.1    NC_020238|YP_007424892.1    NC_020238|YP_007423937.1NC_020238|YP_007422295.1    NC_020238|YP_007424674.1    NC_020238|YP_007419587.1    NC_020238|YP_007423552.1    NC_020238|YP_007420512.1    NC_020238|YP_007420129.1    NC_020238|YP_007421619.1NC_020238|YP_007421019.1    NC_020238|YP_007420614.1    NC_020238|YP_007423400.1    NC_020238|YP_007423686.1    NC_020238|YP_007419795.1    NC_020238|YP_007420130.1    NC_020238|YP_007419815.1NC_020238|YP_007419455.1    NC_020238|YP_007424758.1    NC_020238|YP_007420513.1    NC_020238|YP_007419247.1    NC_020238|YP_007422156.1    NC_020238|YP_007423074.1    NC_020238|YP_007419316.1NC_020238|YP_007424193.1    NC_021033|YP_007821261.1    NC_021033|YP_007821721.1    NC_021033|YP_007822326.1    NC_021033|YP_007821823.1    NC_021033|YP_007821411.1    NC_021033|YP_007821867.1NC_021033|YP_007822359.1    NC_021033|YP_007821473.1    NC_021033|YP_007821472.1    NC_021033|YP_007821938.1    NC_021033|YP_007821903.1    NC_021033|YP_007821718.1    NC_021033|YP_007822061.1NC_021033|YP_007821870.1    NC_022132|YP_008472828.1    NC_022132|YP_008473609.1    NC_022132|YP_008472967.1    NC_022132|YP_008474121.1    NC_022132|YP_008474068.1    NC_022132|YP_008473983.1NC_022132|YP_008473447.1    NC_022132|YP_008474008.1    NC_022132|YP_008473247.1    NC_022132|YP_008473161.1    NC_022132|YP_008473246.1    NC_022132|YP_008473838.1    NC_022132|YP_008474228.1NC_022132|YP_008473820.1    NC_022132|YP_008474146.1    NC_000916|NP_276806.1   NC_002570|NP_244526.1   NC_000916|NP_276784.1   NC_000916|NP_275294.1   NC_002570|NP_241030.1   NC_002570|NP_241031.1   NC_006512|YP_156785.1   NC_009901|YP_001504036.1    NC_009901|YP_001503932.1    NC_000916|NP_275838.1   NC_009901|YP_001501699.1    NC_009901|YP_001502339.1    NC_009901|YP_001504020.1    NC_000916|NP_276196.1   NC_016147|YP_004930361.1    NC_016147|YP_004928885.1    NC_017248|YP_005602814.1    NC_017248|YP_005604115.1    NC_017248|YP_005604412.1    NC_017248|YP_005604130.1    NC_020238|YP_007422466.1    NC_020238|YP_007424757.1    NC_020238|YP_007423053.1    NC_020238|YP_007422501.1    NC_020238|YP_007424454.1    NC_020238|YP_007419248.1    NC_021033|YP_007821993.1NC_002570|NP_242453.1   NC_002570|NP_241518.1   NC_002570|NP_242666.1   NC_002570|NP_241893.1   NC_002570|NP_242057.1   NC_002570|NP_241437.1   NC_002570|NP_241335.1   NC_002570|NP_244762.1   NC_002570|NP_241311.1   NC_002570|NP_241314.1   NC_002570|NP_244781.1   NC_002570|NP_241147.1   NC_002570|NP_242944.1   NC_002570|NP_241045.1   NC_002570|NP_241619.1   NC_002570|NP_242031.1   NC_002570|NP_243565.1   NC_002570|NP_243840.1   NC_002570|NP_242255.1   NC_002570|NP_241869.1   NC_002570|NP_244080.1   NC_002570|NP_244847.1   NC_002570|NP_241152.1   NC_002570|NP_241139.1   NC_002570|NP_242032.1   NC_002570|NP_244158.1   NC_002570|NP_241156.1   NC_002570|NP_243546.1   NC_002570|NP_242622.1   NC_002570|NP_243514.1   NC_002570|NP_244372.1   NC_002570|NP_241465.1   NC_002570|NP_244360.1   NC_002570|NP_244507.1   NC_002570|NP_241248.1   NC_002570|NP_241684.1   NC_002570|NP_244506.1   NC_002570|NP_242665.1   NC_002570|NP_242943.1   NC_009901|YP_001500350.1    NC_002570|NP_244541.1   NC_009901|YP_001504033.1    NC_002570|NP_243987.1   NC_016147|YP_004929013.1    NC_016147|YP_004928965.1    NC_016147|YP_004931492.1    NC_016147|YP_004930297.1    NC_017248|YP_005603335.1    NC_020238|YP_007424682.1    NC_020238|YP_007423929.1    NC_020238|YP_007424789.1    NC_020238|YP_007424238.1    NC_020238|YP_007419737.1    NC_020238|YP_007422965.1    NC_020238|YP_007420686.1    NC_020238|YP_007423624.1    NC_020238|YP_007424548.1    NC_020238|YP_007423477.1    NC_020238|YP_007424911.1    NC_020238|YP_007420302.1    NC_020238|YP_007424153.1    NC_020238|YP_007419419.1    NC_020238|YP_007422171.1NC_020238|YP_007419960.1    NC_020238|YP_007420599.1    NC_020238|YP_007422781.1    NC_020238|YP_007419982.1    NC_020238|YP_007424788.1    NC_021033|YP_007821124.1    NC_021033|YP_007821704.1NC_021033|YP_007821695.1    NC_022132|YP_008473209.1    NC_022132|YP_008472940.1    NC_022132|YP_008473367.1    NC_006512|YP_156645.1   NC_009901|YP_001503213.1    NC_020238|YP_007424863.1    NC_020238|YP_007420652.1    NC_020238|YP_007422376.1    NC_020238|YP_007422580.1    NC_021033|YP_007822031.1    NC_009901|YP_001501606.1    NC_009901|YP_001500374.1    NC_009901|YP_001503941.1NC_016147|YP_004931619.1    NC_009901|YP_001503958.1    NC_009901|YP_001503487.1    NC_009901|YP_001503590.1    NC_009901|YP_001500490.1    NC_020238|YP_007419614.1    NC_020238|YP_007423774.1NC_020238|YP_007423118.1    NC_020238|YP_007420688.1    NC_020238|YP_007422691.1    NC_016147|YP_004929180.1    NC_020238|YP_007419271.1    NC_017248|YP_005604413.1    NC_020238|YP_007421708.1NC_020238|YP_007423902.1    NC_020238|YP_007421569.1    NC_020238|YP_007419703.1    NC_020238|YP_007421870.1    NC_020238|YP_007424247.1    NC_020238|YP_007424404.1    NC_020238|YP_007422671.1NC_020238|YP_007423844.1    NC_020238|YP_007422533.1    NC_021033|YP_007821309.1&#xA;NC_000913|NP_415787.1   NC_002570|NP_243076.1   NC_000913|NP_417319.1   NC_002570|NP_243033.1   NC_000913|NP_417378.1   NC_002570|NP_243357.1   NC_000913|NP_415026.1   NC_002570|NP_242372.1   NC_000913|NP_416921.4   NC_000913|NP_418670.1   NC_000913|NP_417476.1   NC_002570|NP_242377.1   NC_000913|NP_417036.1   NC_000913|NP_418687.1   NC_000913|NP_416136.1   NC_000913|NP_416057.1   NC_000913|NP_415128.1   NC_000913|NP_415611.1   NC_000913|NP_416641.1   NC_000913|NP_417185.1   NC_002570|NP_242416.1   NC_000913|NP_417254.4   NC_006512|YP_155429.1   NC_006512|YP_155255.1   NC_006512|YP_155729.1   NC_006512|YP_155327.1   NC_006512|YP_155181.1   NC_006512|YP_155081.1   NC_000913|NP_416123.1   NC_006512|YP_155918.1   NC_009901|YP_001501509.1    NC_009901|YP_001502003.1    NC_009901|YP_001502349.1    NC_009901|YP_001500292.1    NC_009901|YP_001501986.1    NC_009901|YP_001501933.1    NC_009901|YP_001503747.1    NC_009901|YP_001502120.1    NC_016147|YP_004931151.1    NC_016147|YP_004931013.1    NC_016147|YP_004931458.1    NC_016147|YP_004929444.1    NC_016147|YP_004930364.1    NC_016147|YP_004930559.1    NC_016147|YP_004931475.1    NC_016147|YP_004928712.1    NC_016147|YP_004928757.1    NC_016147|YP_004930325.1    NC_017248|YP_005603267.1    NC_017248|YP_005604744.1    NC_017248|YP_005604687.1    NC_017248|YP_005603278.1    NC_017248|YP_005604375.1    NC_017248|YP_005604362.1NC_017248|YP_005603504.1    NC_017248|YP_005603046.1    NC_020238|YP_007421222.1    NC_020238|YP_007424124.1    NC_020238|YP_007423535.1    NC_020238|YP_007423235.1    NC_020238|YP_007421123.1NC_020238|YP_007419985.1    NC_020238|YP_007422926.1    NC_020238|YP_007421688.1    NC_021033|YP_007821536.1    NC_021033|YP_007821016.1    NC_021033|YP_007821777.1    NC_022132|YP_008474258.1NC_022132|YP_008473927.1    NC_022132|YP_008473949.1    NC_002570|NP_242708.1   NC_002570|NP_241372.1   NC_002570|NP_242709.1   NC_002570|NP_244048.1   NC_006512|YP_154934.1   NC_002570|NP_242376.1   NC_002570|NP_243257.1   NC_002570|NP_241182.1   NC_002570|NP_241573.1   NC_002570|NP_241804.1   NC_002570|NP_244764.1   NC_002570|NP_241798.1   NC_002570|NP_241666.1   NC_002570|NP_241933.1   NC_002570|NP_243010.1   NC_016147|YP_004931673.1    NC_016147|YP_004929856.1    NC_016147|YP_004931082.1    NC_016147|YP_004930495.1    NC_016147|YP_004929610.1    NC_002570|NP_241276.1   NC_017248|YP_005603899.1    NC_017248|YP_005604104.1    NC_020238|YP_007419269.1    NC_020238|YP_007424017.1    NC_020238|YP_007422803.1    NC_020238|YP_007421962.1    NC_020238|YP_007423170.1    NC_020238|YP_007421052.1NC_020238|YP_007423374.1    NC_022132|YP_008473466.1    NC_009901|YP_001500418.1    NC_006512|YP_155874.1   NC_009901|YP_001500747.1    NC_006512|YP_155775.1   NC_009901|YP_001501955.1    NC_009901|YP_001502632.1    NC_006512|YP_156827.1   NC_006512|YP_154534.2   NC_006512|YP_154485.1   NC_006512|YP_154502.1   NC_006512|YP_154475.1   NC_006512|YP_155993.1   NC_009901|YP_001502647.1    NC_016147|YP_004931529.1    NC_016147|YP_004930815.1    NC_016147|YP_004930642.1    NC_017248|YP_005602925.1    NC_020238|YP_007423415.1    NC_020238|YP_007422305.1    NC_009901|YP_001502641.1    NC_016147|YP_004928774.1    NC_009901|YP_001502028.1    NC_016147|YP_004930367.1    NC_009901|YP_001502017.1    NC_009901|YP_001501992.1    NC_009901|YP_001502023.1    NC_009901|YP_001501969.1    NC_009901|YP_001500395.1    NC_016147|YP_004931412.1    NC_016147|YP_004930099.1    NC_016147|YP_004931667.1    NC_017248|YP_005604363.1    NC_017248|YP_005604510.1    NC_016147|YP_004930411.1NC_016147|YP_004930388.1    NC_016147|YP_004931520.1    NC_017248|YP_005603520.1    NC_020238|YP_007420644.1    NC_020238|YP_007421901.1    NC_017248|YP_005604749.1    NC_017248|YP_005604374.1NC_020238|YP_007420315.1    NC_020238|YP_007423401.1    NC_020238|YP_007422918.1    NC_020238|YP_007420064.1    NC_020238|YP_007422689.1    NC_020238|YP_007421824.1    NC_020238|YP_007421665.1&#xA;NC_000913|NP_416477.1   NC_000916|NP_275587.1   NC_000913|NP_418536.1   NC_000913|NP_415102.1   NC_000913|NP_415513.2   NC_000913|NP_418347.1   NC_000913|NP_416723.1   NC_000913|NP_416126.1   NC_000913|NP_418431.1   NC_000913|NP_414934.1   NC_000913|NP_416582.1   NC_000913|NP_417266.1   NC_000913|YP_026207.1   NC_000913|NP_418816.1   NC_002570|NP_241239.1   NC_002570|NP_242811.1   NC_002570|NP_242786.1   NC_002570|NP_244022.1   NC_000913|NP_418305.1   NC_002570|NP_242668.1   NC_002570|NP_243129.1   NC_000913|NP_416722.2   NC_000913|NP_417498.1   NC_000913|NP_417863.1   NC_002570|NP_242413.1   NC_006512|YP_156029.1   NC_006512|YP_156814.1   NC_006512|YP_155592.1   NC_006512|YP_156491.1   NC_006512|YP_156114.1   NC_006512|YP_154713.1   NC_006512|YP_156817.1   NC_000913|NP_416871.1   NC_000913|NP_417051.2   NC_009901|YP_001503954.1    NC_009901|YP_001503779.1    NC_009901|YP_001502820.1    NC_009901|YP_001501220.1    NC_009901|YP_001500739.1    NC_009901|YP_001503863.1    NC_009901|YP_001503774.1    NC_009901|YP_001500491.1    NC_009901|YP_001500771.1    NC_015963|YP_004821697.1    NC_016147|YP_004929463.1    NC_016147|YP_004931113.1    NC_016147|YP_004930812.1    NC_016147|YP_004928847.1    NC_016147|YP_004929107.1    NC_016147|YP_004929562.1    NC_016147|YP_004930161.1    NC_016147|YP_004930186.1    NC_017248|YP_005603425.1    NC_017248|YP_005604274.1    NC_017248|YP_005603133.1NC_017248|YP_005603895.1    NC_020238|YP_007419743.1    NC_020238|YP_007419826.1    NC_020238|YP_007420786.1    NC_020238|YP_007423982.1    NC_020238|YP_007421221.1    NC_020238|YP_007420629.1NC_021033|YP_007822333.1    NC_021033|YP_007821567.1    NC_021033|YP_007822214.1    NC_022132|YP_008473871.1    NC_022132|YP_008473566.1    NC_002570|NP_244894.1   NC_009901|YP_001501042.1    NC_016147|YP_004929074.1    NC_017248|YP_005603428.1    NC_020238|YP_007421237.1    NC_000916|NP_275317.1   NC_020238|YP_007422472.1    NC_002570|NP_244292.1   NC_002570|NP_242039.1   NC_002570|NP_242254.1   NC_002570|NP_242675.1   NC_002570|NP_241685.1   NC_002570|NP_242447.1   NC_002570|NP_242879.1   NC_006512|YP_155836.1   NC_002570|NP_244344.1   NC_002570|NP_243815.1   NC_002570|NP_242020.1   NC_016147|YP_004930588.1    NC_017248|YP_005604351.1    NC_020238|YP_007422447.1    NC_020238|YP_007419413.1    NC_020238|YP_007423024.1    NC_020238|YP_007420376.1    NC_020238|YP_007424261.1    NC_020238|YP_007423930.1    NC_020238|YP_007422463.1    NC_021033|YP_007822035.1    NC_022132|YP_008473570.1    NC_022132|YP_008473326.1    NC_006512|YP_154994.1   NC_006512|YP_156768.1   NC_006512|YP_156232.1   NC_009901|YP_001503548.1    NC_006512|YP_154488.1   NC_006512|YP_156386.1   NC_016147|YP_004929745.1    NC_006512|YP_156501.1   NC_006512|YP_156041.1   NC_020238|YP_007420252.1    NC_020238|YP_007420805.1    NC_020238|YP_007424071.1    NC_006512|YP_156533.1   NC_020238|YP_007421868.1    NC_009901|YP_001501039.1    NC_009901|YP_001502374.1    NC_009901|YP_001500149.1    NC_009901|YP_001502230.1    NC_009901|YP_001501294.1    NC_009901|YP_001501944.1    NC_016147|YP_004931542.1    NC_009901|YP_001503424.1    NC_016147|YP_004929588.1    NC_009901|YP_001501613.1NC_017248|YP_005603109.1    NC_017248|YP_005604799.1    NC_009901|YP_001502892.1    NC_020238|YP_007419575.1    NC_009901|YP_001502412.1    NC_016147|YP_004930781.1    NC_016147|YP_004930045.1NC_016147|YP_004928931.1    NC_020238|YP_007424074.1    NC_016147|YP_004929598.1    NC_016147|YP_004931348.1    NC_016147|YP_004930778.1    NC_022132|YP_008474131.1    NC_020238|YP_007424980.1NC_017248|YP_005603821.1    NC_017248|YP_005603394.1    NC_017248|YP_005603151.1    NC_017248|YP_005604804.1    NC_020238|YP_007419756.1    NC_020238|YP_007419420.1    NC_020238|YP_007422557.1NC_020238|YP_007424496.1    NC_020238|YP_007422215.1    NC_020238|YP_007420664.1    NC_020238|YP_007424185.1&#xA;NC_000913|NP_417864.1   NC_000916|NP_275590.1   NC_000913|NP_414933.1   NC_000916|NP_275588.1   NC_000913|NP_416396.1   NC_000913|NP_416478.2   NC_000913|NP_415751.1   NC_000913|NP_415648.1   NC_000913|NP_418537.1   NC_002570|NP_241238.1   NC_002570|NP_244023.1   NC_000913|NP_415222.1   NC_002570|NP_242019.1   NC_000913|NP_416583.1   NC_002570|NP_242810.1   NC_002570|NP_242446.1   NC_000913|NP_418348.1   NC_002570|NP_244895.1   NC_000913|NP_418815.1   NC_000913|NP_415103.1   NC_000913|NP_416125.2   NC_000913|NP_415515.1   NC_000913|NP_418818.1   NC_000913|NP_417497.1   NC_006512|YP_156492.1   NC_006512|YP_156030.1   NC_006512|YP_154712.1   NC_006512|YP_156042.1   NC_006512|YP_155505.1   NC_006512|YP_156815.1   NC_006512|YP_154487.1   NC_006512|YP_154883.1   NC_009901|YP_001501604.1    NC_009901|YP_001503862.1    NC_009901|YP_001502821.1    NC_009901|YP_001501241.1    NC_009901|YP_001503955.1    NC_009901|YP_001503778.1    NC_009901|YP_001503359.1    NC_009901|YP_001500740.1    NC_009901|YP_001503197.1NC_009901|YP_001500595.1    NC_015963|YP_004821696.1    NC_016147|YP_004929561.1    NC_016147|YP_004931840.1    NC_016147|YP_004928846.1    NC_016147|YP_004929764.1    NC_016147|YP_004930185.1NC_016147|YP_004929462.1    NC_016147|YP_004930128.1    NC_016147|YP_004931112.1    NC_016147|YP_004930813.1    NC_017248|YP_005603416.1    NC_017248|YP_005604273.1    NC_017248|YP_005604848.1NC_017248|YP_005604798.1    NC_017248|YP_005603424.1    NC_020238|YP_007422448.1    NC_020238|YP_007419421.1    NC_020238|YP_007420628.1    NC_020238|YP_007424981.1    NC_020238|YP_007422974.1NC_020238|YP_007423931.1    NC_020238|YP_007419742.1    NC_021033|YP_007821503.1    NC_021033|YP_007822334.1    NC_021033|YP_007821568.1    NC_022132|YP_008473565.1    NC_022132|YP_008473850.1NC_022132|YP_008474130.1    NC_000916|NP_275583.1   NC_000916|NP_276719.1   NC_000916|NP_276038.1   NC_000916|NP_275691.1   NC_000916|NP_276039.1   NC_009901|YP_001500416.1    NC_009901|YP_001502232.1    NC_000916|NP_276870.1   NC_000916|NP_275600.1   NC_016147|YP_004930279.1    NC_016147|YP_004928930.1    NC_016147|YP_004929075.1    NC_020238|YP_007419827.1    NC_020238|YP_007424243.1    NC_002570|NP_241686.1   NC_002570|NP_242776.1   NC_002570|NP_243099.1   NC_002570|NP_244779.1   NC_002570|NP_243130.1   NC_002570|NP_241154.1   NC_002570|NP_242038.1   NC_002570|NP_241621.1   NC_002570|NP_244293.1   NC_002570|NP_241137.1   NC_002570|NP_243567.1   NC_002570|NP_242674.1   NC_009901|YP_001503476.1    NC_002570|NP_241392.1   NC_009901|YP_001501292.1    NC_002570|NP_244661.1   NC_009901|YP_001503928.1    NC_002570|NP_240874.1   NC_016147|YP_004930589.1    NC_017248|YP_005604344.1    NC_017248|YP_005602945.1    NC_020238|YP_007423846.1    NC_020238|YP_007422507.1    NC_020238|YP_007420250.1    NC_020238|YP_007420804.1    NC_020238|YP_007420378.1    NC_020238|YP_007419412.1    NC_020238|YP_007423983.1    NC_020238|YP_007424262.1    NC_006512|YP_154558.1   NC_006512|YP_155485.1   NC_009901|YP_001501253.1    NC_006512|YP_155597.1   NC_009901|YP_001500492.1    NC_006512|YP_156095.1   NC_020238|YP_007424223.1    NC_020238|YP_007422379.1    NC_020238|YP_007422214.1NC_020238|YP_007421220.1    NC_009901|YP_001502787.1    NC_009901|YP_001502373.1    NC_009901|YP_001500198.1    NC_009901|YP_001502663.1    NC_009901|YP_001502091.1    NC_009901|YP_001503425.1NC_009901|YP_001503449.1    NC_009901|YP_001500165.1    NC_020238|YP_007419757.1    NC_022132|YP_008472776.1    NC_016147|YP_004931541.1    NC_016147|YP_004930044.1    NC_016147|YP_004929587.1NC_016147|YP_004928825.1    NC_017248|YP_005604712.1    NC_020238|YP_007421867.1    NC_020238|YP_007424495.1    NC_020238|YP_007421236.1    NC_020238|YP_007424764.1    NC_020238|YP_007424552.1NC_020238|YP_007424070.1    NC_020238|YP_007422555.1    NC_022132|YP_008473569.1&#xA;NC_000913|NP_416724.1   NC_002570|NP_243814.1   NC_000913|NP_417880.1   NC_002570|NP_241403.1   NC_000913|NP_417049.1   NC_000913|NP_417211.1   NC_000913|NP_415839.1   NC_002570|NP_242745.1   NC_000913|NP_414864.1   NC_000913|NP_417189.2   NC_000913|NP_417345.1   NC_002570|NP_241858.1   NC_000913|NP_416986.4   NC_002570|NP_244767.1   NC_000913|NP_415819.4   NC_000913|NP_418432.1   NC_000913|NP_418304.1   NC_000913|NP_415153.1   NC_006512|YP_156963.1   NC_006512|YP_156816.1   NC_000913|NP_417727.1   NC_006512|YP_156670.1   NC_006512|YP_155593.1   NC_006512|YP_155113.1   NC_006512|YP_154588.1   NC_006512|YP_155797.1   NC_006512|YP_155122.1   NC_000913|NP_415719.2   NC_009901|YP_001501295.1    NC_009901|YP_001501614.1    NC_009901|YP_001500252.1    NC_009901|YP_001503170.1    NC_009901|YP_001501310.1    NC_009901|YP_001502525.1    NC_016147|YP_004929744.1    NC_016147|YP_004931524.1    NC_016147|YP_004929108.1    NC_016147|YP_004929327.1    NC_016147|YP_004931622.1    NC_017248|YP_005603894.1NC_020238|YP_007419730.1    NC_020238|YP_007422240.1    NC_020238|YP_007419468.1    NC_020238|YP_007422327.1    NC_020238|YP_007420168.1    NC_021033|YP_007821613.1    NC_021033|YP_007821101.1NC_022132|YP_008473759.1    NC_000916|NP_275692.1   NC_002570|NP_243834.1   NC_002570|NP_243632.1   NC_002570|NP_244654.1   NC_006512|YP_155591.1   NC_002570|NP_244812.1   NC_002570|NP_244705.1   NC_009901|YP_001501219.1    NC_009901|YP_001499949.1    NC_002570|NP_242692.1   NC_002570|NP_243605.1   NC_009901|YP_001503938.1    NC_009901|YP_001501221.1    NC_016147|YP_004929023.1    NC_017248|YP_005603892.1NC_020238|YP_007424841.1    NC_006512|YP_156531.1   NC_006512|YP_156442.1   NC_009901|YP_001503210.1    NC_006512|YP_154927.1   NC_009901|YP_001503292.1    NC_016147|YP_004929583.1    NC_020238|YP_007421638.1    NC_009901|YP_001503216.1    NC_009901|YP_001501634.1    NC_009901|YP_001503775.1    NC_009901|YP_001500582.1    NC_009901|YP_001501943.1    NC_020238|YP_007419660.1    NC_020238|YP_007423568.1    NC_021033|YP_007822213.1    NC_016147|YP_004931436.1&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Paralogs&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NC_020238|YP_007420194.1    NC_020238|YP_007420199.1    NC_020238|YP_007421388.1    NC_020238|YP_007422556.1    NC_020238|YP_007421295.1    NC_020238|YP_007422519.1    NC_020238|YP_007422868.1    NC_020238|YP_007419313.1    NC_020238|YP_007422870.1    NC_020238|YP_007423429.1    NC_020238|YP_007424554.1    NC_020238|YP_007419582.1    NC_020238|YP_007424027.1    NC_020238|YP_007421297.1    NC_020238|YP_007424852.1    NC_020238|YP_007419896.1    NC_020238|YP_007421517.1    NC_020238|YP_007424277.1    NC_020238|YP_007419311.1    NC_020238|YP_007422308.1    NC_020238|YP_007424965.1    NC_020238|YP_007424556.1    NC_020238|YP_007420196.1    NC_020238|YP_007423077.1    NC_020238|YP_007419782.1    NC_020238|YP_007424967.1    NC_020238|YP_007422134.1    NC_020238|YP_007422606.1    NC_020238|YP_007421515.1    NC_020238|YP_007424854.1    NC_020238|YP_007422892.1    NC_020238|YP_007424026.1    NC_020238|YP_007420846.1    NC_020238|YP_007421386.1    NC_020238|YP_007422608.1    NC_020238|YP_007419894.1    NC_020238|YP_007419780.1&#xA;NC_002570|NP_244374.1   NC_002570|NP_243243.1   NC_002570|NP_242247.1   NC_002570|NP_243020.1   NC_002570|NP_242567.1   NC_002570|NP_242552.1   NC_002570|NP_241820.1   NC_002570|NP_244452.1   NC_002570|NP_242339.1   NC_002570|NP_243788.1   NC_002570|NP_243343.1   NC_002570|NP_242204.1   NC_002570|NP_243507.1   NC_002570|NP_243844.1   NC_002570|NP_244275.1   NC_002570|NP_242049.1   NC_002570|NP_242829.1   NC_002570|NP_243517.1   NC_002570|NP_243070.1   NC_002570|NP_244113.1   NC_002570|NP_243338.1   NC_002570|NP_244432.1&#xA;NC_009901|YP_001501354.1    NC_009901|YP_001501833.1    NC_009901|YP_001500880.1    NC_009901|YP_001499887.1    NC_009901|YP_001503552.1    NC_009901|YP_001500436.1    NC_009901|YP_001500544.1    NC_009901|YP_001502124.1    NC_009901|YP_001503914.1    NC_009901|YP_001500967.1    NC_009901|YP_001501483.1    NC_009901|YP_001500660.1    NC_009901|YP_001502935.1    NC_009901|YP_001499880.1    NC_009901|YP_001503109.1    NC_009901|YP_001502279.1    NC_009901|YP_001501475.1    NC_009901|YP_001500265.1    NC_009901|YP_001502320.1    NC_009901|YP_001502427.1    NC_009901|YP_001501772.1    NC_009901|YP_001502186.1&#xA;NC_002570|NP_242278.1   NC_002570|NP_242213.1   NC_002570|NP_240989.1   NC_002570|NP_244003.1   NC_002570|NP_240955.1   NC_002570|NP_242812.1   NC_002570|NP_244366.1   NC_002570|NP_243640.1   NC_002570|NP_244619.1   NC_002570|NP_241534.1   NC_002570|NP_242036.1   NC_002570|NP_243833.1   NC_002570|NP_241136.1   NC_002570|NP_242795.1   NC_002570|NP_241633.1   NC_002570|NP_241282.1   NC_002570|NP_242173.1   NC_002570|NP_241111.1   NC_002570|NP_241245.1   NC_002570|NP_241915.1   NC_002570|NP_242412.1&#xA;NC_000916|NP_275317.1   NC_000916|NP_276388.1   NC_000916|NP_275266.1   NC_000916|NP_276120.1   NC_000916|NP_275589.1   NC_000916|NP_276039.1   NC_000916|NP_275762.1   NC_000916|NP_275602.1   NC_000916|NP_276038.1   NC_000916|NP_276252.1   NC_000916|NP_275499.1   NC_000916|NP_275927.1   NC_000916|NP_275962.1   NC_000916|NP_275611.1   NC_000916|NP_275503.1   NC_000916|NP_275435.1&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;How can I visualise the Ortholog and Paralog genes?&lt;/p&gt;&#xA;" OwnerUserId="2477" LastEditorUserId="298" LastEditDate="2019-05-25T12:12:00.920" LastActivityDate="2019-05-25T12:12:00.920" Title="Visualizing Ortholog and Paralog genes" Tags="|gene|orthologues|paralogs|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="8689" PostTypeId="2" ParentId="165" CreationDate="2019-05-25T21:34:11.070" Score="1" Body="&lt;p&gt;Some information seems to be available on the module level utilising the API. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reaction &lt;a href=&quot;http://www.genome.jp/dbget-bin/www_bget?rn:R00352&quot; rel=&quot;nofollow noreferrer&quot;&gt;R00352&lt;/a&gt; points to the module &lt;a href=&quot;https://www.genome.jp/kegg-bin/show_module?M00173&quot; rel=&quot;nofollow noreferrer&quot;&gt;M00173&lt;/a&gt;. While some information is in the &lt;code&gt;Definition&lt;/code&gt;, it's not trivial to link it to the particular reaction. This linkage is however easier accessible via &lt;a href=&quot;http://rest.kegg.jp/get/md:M00173&quot; rel=&quot;nofollow noreferrer&quot;&gt;API&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/orSvB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/orSvB.png&quot; alt=&quot;Reaction orthology defined on module level&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4628" LastEditorUserId="4628" LastEditDate="2019-05-26T07:07:10.983" LastActivityDate="2019-05-26T07:07:10.983" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8690" PostTypeId="2" ParentId="2881" CreationDate="2019-05-25T21:55:07.397" Score="3" Body="&lt;p&gt;Since this question's been asked there's been a (what looks like a first useful?) release of Drug Ontology &lt;a href=&quot;http://bioportal.bioontology.org/ontologies/DRON&quot; rel=&quot;nofollow noreferrer&quot;&gt;DRON&lt;/a&gt;. Perhaps worth checking again?&lt;/p&gt;&#xA;" OwnerUserId="4628" LastActivityDate="2019-05-25T21:55:07.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8691" PostTypeId="1" CreationDate="2019-05-26T23:35:44.717" Score="2" ViewCount="342" Body="&lt;p&gt;I have a list of human genes that have been proven to be up-regulated in a disease condition of interest, using microarray analysis. I also have a protein which according to literature is said to be a DNA binding protein. I wish to be able to computationally predict if the protein mentioned binds to any of the promoter sequences of my differentially expressed genes and if yes , then at which position?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any Bioinformatic tools available for this task?&lt;/p&gt;&#xA;" OwnerUserId="3968" LastEditorUserId="3968" LastEditDate="2019-05-27T20:44:27.693" LastActivityDate="2019-05-28T17:51:20.840" Title="How do I computationally predict the binding position of a DNA binding protein on the promoters of differentially expressed genes?" Tags="|computation|homer|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8692" PostTypeId="2" ParentId="8691" CreationDate="2019-05-27T04:53:42.147" Score="4" Body="&lt;p&gt;You could run &lt;a href=&quot;http://meme-suite.org/doc/fimo.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;FIMO&lt;/a&gt; on your entire genome for TFs (transcription factors; DNA-binding proteins) of interest, which gives you binding sites: genomic intervals where those TFs bind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2491/776&quot;&gt;https://bioinformatics.stackexchange.com/a/2491/776&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then map or intersect promoter regions with those TF binding sites with BEDOPS &lt;code&gt;bedmap&lt;/code&gt; or &lt;code&gt;bedops&lt;/code&gt;, respectively:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/statistics/bedmap.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedops.readthedocs.io/en/latest/content/reference/statistics/bedmap.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/set-operations/bedops.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedops.readthedocs.io/en/latest/content/reference/set-operations/bedops.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The use of &lt;code&gt;bedmap&lt;/code&gt; is recommended to assign TFs to a promoter of interest. This tool will return both the promoter region and any TFs that overlap it by the specified overlap criteria.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2019-05-28T17:51:20.840" LastActivityDate="2019-05-28T17:51:20.840" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="8696" PostTypeId="2" ParentId="8691" CreationDate="2019-05-27T13:41:29.443" Score="0" Body="&lt;p&gt;If you work with human, I would start by intersecting your promoter coordinates with the &lt;a href=&quot;http://tagc.univ-mrs.fr/remap/index.php?page=annotation&quot; rel=&quot;nofollow noreferrer&quot;&gt;ReMap2018&lt;/a&gt; database which is a comprehensive collection of published ChIP-seq datasets. Extract those TFs that fit your research question. Alternatively you can browse NCBI for published datasets of your TF and then intersect those with your annotations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also cross-posted: &lt;a href=&quot;https://www.biostars.org/p/381751/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/381751/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2019-05-28T07:37:44.310" LastActivityDate="2019-05-28T07:37:44.310" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8697" PostTypeId="1" AcceptedAnswerId="8700" CreationDate="2019-05-27T16:57:25.830" Score="1" ViewCount="47" Body="&lt;p&gt;I am making a demographic model for African American population in which I say 80% of ancestry comes from African population and 20% comes from European population. I am using the time/population time from &lt;a href=&quot;https://www.pnas.org/content/111/2/757&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gazave et al. (2014)&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The authors doe not mention the effective population size (Ne) of African American population after admixture. Is there a publication someone can point me to that mentions the effective population size of African American (AA) or even European American (EA). I know there is not much difference between the two. It's just I need a figure to have my simulation up and running.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; &lt;em&gt;N&lt;/em&gt;e is an estimate of the people size based genetic diversity and migration and is associated with the Fst statistics&lt;/p&gt;&#xA;" OwnerUserId="4636" LastEditorUserId="3848" LastEditDate="2020-05-03T10:41:24.013" LastActivityDate="2020-05-03T10:41:24.013" Title="Confusion related to effective population size" Tags="|phylogeny|phylogenetics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8698" PostTypeId="1" AcceptedAnswerId="8699" CreationDate="2019-05-27T17:01:10.293" Score="-1" ViewCount="80" Body="&lt;p&gt;I have average log fold change for a cluster of cells versus another cluster of cells by Seurat like below &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;         avg_logFC&#xA;TAGLN   -2.507833541&#xA;TXNRD1  1.093672244&#xA;MYL9    -1.579165463&#xA;ACTB    -1.482386539&#xA;ACTA2   -2.379757433&#xA;SQSTM1  1.227562767&#xA;ACTG2   -2.144757086&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have also mean log2 ratio of a set of proteins in treatment versus control like below (I converted protein IDs to gene symbol)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mean log2ratio  Gene Symbol&#xA;-0.3    CYFIP1&#xA;0.4     GNPTG&#xA;-0.3    WWC2&#xA;-0.5    TRPV4&#xA;-0.4    CHRM2&#xA;0.3     MFSD11&#xA;-0.3    ANXA11&#xA;0.3     CENPE&#xA;-0.4    RPL22L1&#xA;0.3     SAMD1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How I can normalize these two data or make them compatible for plotting a heatmap, pathway analysis, etc?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="77" LastEditDate="2019-05-27T19:20:30.277" LastActivityDate="2019-05-27T19:23:14.000" Title="How I normalize these two sets of data" Tags="|r|scrnaseq|normalization|proteomics|heatmap|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8699" PostTypeId="2" ParentId="8698" CreationDate="2019-05-27T19:23:14.000" Score="2" Body="&lt;p&gt;There's no need to normalize them, you're not comparing them. Just use them as they are.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-05-27T19:23:14.000" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8700" PostTypeId="2" ParentId="8697" CreationDate="2019-05-27T19:26:54.433" Score="0" Body="&lt;p&gt;That is a fantastic question and in my opinion a really important question in context. The Ne of Africa will be much bigger than the European population, however I'm less sure about the West African population which gave rise to the Afro-American population. I suspect it is still large because of the &quot;out of Africa&quot; theory. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My advice is to start by taking a complete look at the entire subject, from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1832099/pdf/520.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tenesa et al (2007)&lt;/a&gt;.  Ne is a really important parameter in any population study and extremely useful. I have feeling this could be complex, but I cannot hypotheses any further because I don't do human genetics (except for immunology). &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-05-27T19:34:51.767" LastActivityDate="2019-05-27T19:34:51.767" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8702" PostTypeId="1" CreationDate="2019-05-28T04:58:33.257" Score="1" ViewCount="27" Body="&lt;p&gt;We usually denote the origin of a mutation as either somatic or germline. This information is usually available in certain databases such as &lt;a href=&quot;https://civicdb.org/home&quot; rel=&quot;nofollow noreferrer&quot;&gt;CIVIC&lt;/a&gt;, ClinVar, COSMIC etc. But when we come to variants that have been denoted as &quot;Overexpression&quot;, &quot;Upregulation&quot; , &quot;Methylation&quot; etc., what would be the origin? Any leads would be very helpful. Thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/30098223&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cristina Aguirre‐Portolés et al., 2018&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;ABCA1 is significantly overexpressed in patients at advanced stages of&#xA;  colorectal cancer, and its overexpression confers proliferative&#xA;  advantages together with caveolin‐1 dependent‐increased migratory and&#xA;  invasive capacities. Overexpression of ABCA1 was associated with&#xA;  overall survival independently of the molecular subtype of the cancer.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Does this overexpression phenomenon arise in the germline or does it happen somatically? How do we conclude?&lt;/p&gt;&#xA;" OwnerUserId="4640" LastEditorUserId="57" LastEditDate="2019-05-28T18:00:24.360" LastActivityDate="2019-05-28T18:00:24.360" Title="Need help in identifying origin of variants in CIVIC" Tags="|database|mutations|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8703" PostTypeId="2" ParentId="2153" CreationDate="2019-05-28T09:30:22.383" Score="0" Body="&lt;p&gt;In addition to UniProt and EnsEMBL as suggested above you might like:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ebi.ac.uk/interpro/&quot; rel=&quot;nofollow noreferrer&quot;&gt;InterPro&lt;/a&gt;, functional analysis of proteins by classifying them into families and predicting domains and important sites.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://prodata.swmed.edu/ecod/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ECOD&lt;/a&gt;, a classification of protein domains by evolutionary relationship.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://pfam.xfam.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pfam&lt;/a&gt;, a collection of protein families.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.cathdb.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CATH&lt;/a&gt;, protein structural domains.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.genome3d.eu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genome3D&lt;/a&gt;, which combines various resources to structurally annotate proteins.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2019-05-28T09:30:22.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8704" PostTypeId="1" AcceptedAnswerId="8708" CreationDate="2019-05-28T09:39:51.960" Score="5" ViewCount="957" Body="&lt;p&gt;&lt;strong&gt;What makes tSNE being the preferred dimensional reduction&lt;/strong&gt; for visualization in single cell RNA-seq over PCA?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware that tSNE works better at showing local structures and fails to capture global structures of the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I think I don't fully get the reason of &lt;strong&gt;why is this an advantage&lt;/strong&gt;? It offers better resolution? Or better separation of the cells as compared to PCA?  &lt;/p&gt;&#xA;" OwnerUserId="678" LastActivityDate="2019-06-04T11:00:10.367" Title="PCA vs tSNE in single cell RNA-seq" Tags="|scrnaseq|ngs|single-cell|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8706" PostTypeId="1" CreationDate="2019-05-28T13:18:02.017" Score="1" ViewCount="643" Body="&lt;p&gt;I am trying to perform a cellranger count on fastq files generated from a 10x genomics single cell RNA Seq run.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to provide some background, I ssh’d into the AWS using our AWS IP with MobaXTerm linux command line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My files are in the Isilon. The file folders are &lt;code&gt;path/to/runs/&lt;/code&gt; - this folder contains the &lt;code&gt;.fastq&lt;/code&gt; files for each sample (ex: &lt;code&gt;SI_GA_A1_S1&lt;/code&gt;, &lt;code&gt;SI_GA_B1_S2&lt;/code&gt;, &lt;code&gt;SI_GA_C1_S3&lt;/code&gt;, and &lt;code&gt;SI_GA_D1_S4&lt;/code&gt;). I can see these folders listed in the linux command line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cellranger is found when I write the following: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;path/to/cellranger-3.0.1/cellranger-3.0.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then &lt;code&gt;ls&lt;/code&gt; brings up 13 items located in the folder, one of them being &lt;code&gt;cellranger&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I write the following command, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/path/to/cellranger-3.0.1/cellranger-3.0.1/cellranger count \&#xA;    --id=run2\&#xA;    --transcriptome=/path/to/refseq/refdata-cellranger-hg19-3.0.0\&#xA;    --sample=sample1\&#xA;    --expect-cells=5000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and it gives me: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-bash: /path/to/cellranger-3.0.1/cellranger-3.0.1/cellranger: No such file or directory&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This looks correct from what the 10x website says for single cell RNA Seq. Please advise.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ls -l /path/to/cellranger-3.0.1/cellranger-3.0.1/cellranger &#xA;-rwxrwx--- 1 1003080 1000000 460 Dec 5 19:41 /path/to/cellranger-3.0.1/cellranger-3.0.1/cellranger&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4643" LastEditorUserId="298" LastEditDate="2019-05-28T18:39:18.043" LastActivityDate="2019-06-28T16:00:50.110" Title="processing fastq files using cellranger on linux" Tags="|linux|bash|cellranger|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="8707" PostTypeId="2" ParentId="8706" CreationDate="2019-05-28T14:17:46.490" Score="0" Body="&lt;p&gt;It could be that &lt;code&gt;cellranger&lt;/code&gt; does not have execute permissions. Try&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chmod u+x [insert internal isilon path here]/[insert windows username here]/cellranger-3.0.1/cellranger-3.0.1/cellranger&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(or the Windows equivalent, which seems to be to give Execute permission using the GUI)&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2019-05-28T14:17:46.490" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="8708" PostTypeId="2" ParentId="8704" CreationDate="2019-05-28T14:22:52.430" Score="11" Body="&lt;p&gt;tSNE often offers better visual representation (separation) on such complicated data than PCA. As Micheal pointed out, computing a tSNE embedding over 20.000 gene dimensions is computationally unfeasible, so a number of PCs are normally calculated and these are used as input for calculating the tSNE. They are used in tandem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for global vs. local, we are much more interested to see similarity of cells to a limited number of neighbours indicating a celltype and grouping these close together. This is more important than the distance between such celltypes. (assuming the separation in your tSNE is driven by a biologically meaningful factor such as celltype    and not some confounder)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since I just made these images for myself anyway I might as well post them. The first 2 PCs show some separation by celltype, tSNE computed over 100 PCs gets very nice separation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/anIfM.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/anIfM.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/IzQ2o.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IzQ2o.png&quot; alt=&quot; &quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3818" LastEditorUserId="3818" LastEditDate="2019-06-04T11:00:10.367" LastActivityDate="2019-06-04T11:00:10.367" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8709" PostTypeId="1" CreationDate="2019-05-29T02:14:33.930" Score="2" ViewCount="110" Body="&lt;p&gt;IGV is turning QUAL &quot;.&quot; into -10 from this VCF of structural variations.  I'm wondering if this means anything or if I should report a bug to the IGV devs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a row from a VCF:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT SAMPLE&#xA;&#xA;chrY    11292098    DUP00008569 C   &amp;lt;DUP&amp;gt;   .   PASS    IMPRECISE;SVTYPE=DUP;SVMETHOD=EMBL.DELLYv0.8.1;CHR2=chrY;END=56834684;PE=3;MAPQ=24;CT=5to3;CIPOS=-265,265;CIEND=-265,265    GT:GL:GQ:FT:RCL:RC:RCR:CN:DR:DV:RR:RV0/1:-5.71458,0,-20.0913:57:PASS:0:90774:0:-1:6:5:0:0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You see the QUAL value is &quot;.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IGV reports it as -10&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oAXiB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oAXiB.png&quot; alt=&quot;igv screenshot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(This call is probably a false positive but it's just an example. There are others)&lt;/p&gt;&#xA;" OwnerUserId="460" LastActivityDate="2019-05-29T02:40:01.703" Title="Why does IGV turn a VCF QUAL of &quot;.&quot; into -10?" Tags="|vcf|variant-calling|igv|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8710" PostTypeId="2" ParentId="8709" CreationDate="2019-05-29T02:40:01.703" Score="3" Body="&lt;p&gt;Looks like they are treating the &quot;.&quot; as a value of 1.  According to the &lt;a href=&quot;http://samtools.github.io/hts-specs/VCFv4.2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VCF spec&lt;/a&gt; the quality is &quot;−10log10 prob&quot;.  Where are you getting your QUALs from?  I have never seen the quals as a &quot;.&quot; and IGV probably needs to do some translation into an int but a QUAL of -10 makes no sense. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would report it and let them explain.&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2019-05-29T02:40:01.703" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8711" PostTypeId="2" ParentId="2282" CreationDate="2019-05-29T03:56:35.337" Score="0" Body="&lt;p&gt;Some of the researchers in my institute have researched on this and use UMIs (Unique Molecular Identifiers) to eliminate PCR duplicates. They have been able to increase the reproducibility of the &lt;a href=&quot;https://www.basepairtech.com/analysis/rna-seq/&quot; rel=&quot;nofollow noreferrer&quot;&gt;rna seq analysis&lt;/a&gt; they had been working on. The UMIs capture some lengths of diverse nucleotides from the rna-sequence. I cannot validate how much UMIs can help in deduplication, as even Nature journal says that UMIs specifically work on the 3’ end. But it has been observed that eliminating duplicates doesn’t improve the accuracy of quantification and it could introduce a bias in the data. So do not eliminate the duplicates, it's just a waste of time.&lt;/p&gt;&#xA;" OwnerUserId="4618" LastEditorUserId="4618" LastEditDate="2019-05-29T14:28:18.280" LastActivityDate="2019-05-29T14:28:18.280" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8712" PostTypeId="1" AcceptedAnswerId="8716" CreationDate="2019-05-29T10:55:42.127" Score="3" ViewCount="1114" Body="&lt;p&gt;I am creating a pipeline in Nextflow. One step is creating a pangenome with Roary. Roary takes threads as an argument and if no number of threads is supplied as an argument it defaults to one. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way in Nextflow to pass the maximum number of available threads as an argument? At the moment the best I can come up with is defining an arbitrarily large number of threads as an argument which will cap out and run with the max number of threads. I would like to know, is there a 'cleaner' or more canonical way of doing this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment my process looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;process roary {&#xA;&#xA;    publishDir &quot;${params.outdir}/roary&quot;, mode: 'copy'&#xA;&#xA;    cpus 200&#xA;&#xA;    input:&#xA;    file gff from gff.collect()&#xA;&#xA;    output:&#xA;    file(&quot;*&quot;) into roary&#xA;    file(&quot;pan_genome_reference.fa&quot;) into pan_genome&#xA;    file(&quot;pan_genome_sequences/*&quot;) into alignment_files&#xA;    file(&quot;gene_presence_absence.Rtab&quot;) into gene_presence&#xA;    set file(&quot;*accessory*&quot;),&#xA;        file(&quot;*.Rtab&quot;),&#xA;        file(&quot;_*&quot;),&#xA;        file(&quot;*.txt&quot;),&#xA;        file(&quot;*csv&quot;) into roary_out&#xA;&#xA;&#xA;    script:&#xA;    &quot;&quot;&quot;&#xA;    roary -p &lt;span class=&quot;math-container&quot;&gt;${task.cpus} -e -n -v -z $&lt;/span&gt;gff&#xA;    &quot;&quot;&quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance. Also, apologies as I can see why this might not strictly be bioinformatics, but it's where Nextflow has the most traction.&lt;/p&gt;&#xA;" OwnerUserId="252" LastEditorUserId="96" LastEditDate="2019-05-29T13:06:02.143" LastActivityDate="2020-04-08T00:54:31.753" Title="Autodetect max number of cores and pass as an argument in Nextflow" Tags="|workflow-management|nextflow|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8713" PostTypeId="1" AcceptedAnswerId="8715" CreationDate="2019-05-29T11:31:07.237" Score="1" ViewCount="476" Body="&lt;p&gt;I'm trying to count the GC in regions leading up to polyA clusters. I found the bedtools command nuc, one of the outputs of which is the GC content. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I took the mouse polyA cluster database from &lt;a href=&quot;http://polyasite.unibas.ch/#22&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, and the mm10 primary assembly soft masked from &lt;a href=&quot;ftp://ftp.ensembl.org/pub/release-96/fasta/mus_musculus/dna/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I imported the polyA clusters bed file to R. I then got a list of all the polyA clusters in the genes I'm interested in. For positive strands on polyA I took the start position, took 100 away from it and made that the start of the region of interest, then made the start of the polyA signal the end. For things on the negative strand I added 100 to the end of the polyA signal. I then exported the results to a bed file format. The code I used to do this is below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I try to run:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;bedtools nuc -fi Mus_musculus.GRCm38.dna_sm.primary_assembly.fa -bed PolyA.bed&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Every signal returns some variant of: Feature (chr10:21,080,315-21,080,415) beyond the length of chr10 size (0 bp).  Skipping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can however look at several of these regions in the UCSC genome browser without a problem, at least I don't think I had a problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone point me to what's going wrong or to another way I can solve this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GetRanges &amp;lt;- function(FilteredClusters){&#xA;  for (x in 1:nrow(FilteredClusters)){&#xA;    if(FilteredClusters&lt;span class=&quot;math-container&quot;&gt;$Strand[x] == '+'){&#xA;  FilteredClusters$RegionEnd[x] &amp;lt;- FilteredClusters$ClusterStart[x]&#xA;  FilteredClusters$RegionStart[x] &amp;lt;- FilteredClusters$RegionEnd[x] - 100&#xA;}&#xA;else if(FilteredClusters$&lt;/span&gt;Strand[x] == '-')&#xA;    {&#xA;      FilteredClusters&lt;span class=&quot;math-container&quot;&gt;$RegionStart[x] &amp;lt;- FilteredClusters$&lt;/span&gt;ClusterEnd[x]&#xA;      FilteredClusters&lt;span class=&quot;math-container&quot;&gt;$RegionEnd[x] &amp;lt;- FilteredClusters$&lt;/span&gt;RegionStart[x] + 100&#xA;    }&#xA;  }&#xA;    ClusterCut &amp;lt;- select(FilteredClusters, 'Chr', 'RegionStart', 'RegionEnd')&#xA;&#xA;    return(ClusterCut)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1843" LastEditorUserId="77" LastEditDate="2019-05-29T11:49:28.947" LastActivityDate="2019-05-29T11:51:31.523" Title="bedtools feature out of bounds" Tags="|rna-seq|bedtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8714" PostTypeId="2" ParentId="8528" CreationDate="2019-05-29T11:34:49.473" Score="0" Body="&lt;p&gt;This is probably one of those things where as long as you can justify your choice somewhat.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It might be worth seeing what others have done (see &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4086075/&quot; rel=&quot;nofollow noreferrer&quot;&gt;loctree3&lt;/a&gt; and &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/26/13/1608/201357&quot; rel=&quot;nofollow noreferrer&quot;&gt;PSORT&lt;/a&gt;) and seeing how well they match your case, then add in any subdomains you have a particular interest in.&lt;/p&gt;&#xA;" OwnerUserId="252" LastActivityDate="2019-05-29T11:34:49.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8715" PostTypeId="2" ParentId="8713" CreationDate="2019-05-29T11:51:31.523" Score="3" Body="&lt;p&gt;There is no chromosome named &lt;code&gt;chr10&lt;/code&gt;, but there is one named &lt;code&gt;10&lt;/code&gt;. The problem is that you're mixing chromosome name systems with a tool that, like most, isn't able to handle that. Remove all instances of &lt;code&gt;chr&lt;/code&gt; in your BED file and fix things like &lt;code&gt;chrM&lt;/code&gt; -&gt; &lt;code&gt;MT&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-05-29T11:51:31.523" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8716" PostTypeId="2" ParentId="8712" CreationDate="2019-05-29T12:20:39.397" Score="5" Body="&lt;p&gt;I have not read the full Nextflow manual in depth, but I think there is no Nextflow command to detect the maximum number of threads you can use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you can try to automatize this task by using the &lt;code&gt;nproc&lt;/code&gt; command.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;nproc&lt;/code&gt; gives you the number of processing units available. An idea would be to include something such &lt;code&gt;nproc - 1&lt;/code&gt; as the max number of threads for Roary.  &lt;/p&gt;&#xA;" OwnerUserId="678" LastEditorUserId="298" LastEditDate="2019-05-29T13:57:42.187" LastActivityDate="2019-05-29T13:57:42.187" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8717" PostTypeId="2" ParentId="4449" CreationDate="2019-05-29T13:03:24.207" Score="0" Body="&lt;p&gt;The problem is that you are missing read groups. You can add them directly to bam files you already have using picard tools. Something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar ./software/picard.jar AddOrReplaceReadGroups \&#xA;  I=input.bam \&#xA;  O=output.bam \&#xA;  RGLB=lib1 \&#xA;  RGPL=illumina \&#xA;  RGPU=unit1 \&#xA;  RGSM=hello&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4651" LastEditorUserId="57" LastEditDate="2019-05-29T13:39:50.857" LastActivityDate="2019-05-29T13:39:50.857" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8718" PostTypeId="2" ParentId="8706" CreationDate="2019-05-29T15:04:00.900" Score="1" Body="&lt;p&gt;this has been resolved. I downloaded the windows file and this caused trouble but when downloading the file within the Linux, it seems to be working. &lt;/p&gt;&#xA;" OwnerUserId="4643" LastActivityDate="2019-05-29T15:04:00.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8719" PostTypeId="1" AcceptedAnswerId="8720" CreationDate="2019-05-29T16:45:57.920" Score="1" ViewCount="121" Body="&lt;p&gt;I'd like to download domain-domain interactions that were in ipfam (which doesn't seem to be up-and-running anymore).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for an example of what I would like to be able to download via ftp or another source, see this list of interactions with Death domain:&#xA;&lt;a href=&quot;https://pfam.xfam.org/family/PF00531#tabview=tab8&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pfam.xfam.org/family/PF00531#tabview=tab8&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this available in one of these files in the current release??&#xA;&lt;a href=&quot;ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4260" LastActivityDate="2019-05-29T18:53:10.393" Title="ipfam or pfam domain-domain interactions" Tags="|interactions|domains|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8720" PostTypeId="2" ParentId="8719" CreationDate="2019-05-29T18:46:56.477" Score="0" Body="&lt;p&gt;I think you're looking for &lt;a href=&quot;ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/database_files/pfamA_interactions.txt.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ebi.ac.uk/pub/databases/Pfam/current_release/database_files/pfamA_interactions.txt.gz&lt;/a&gt;. Based on the file's name, that's a list of domain interactions. The file has two columns, so presumably each line represents an interaction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's what I get for the Death domain (PF00531):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ zgrep PF00531 pfamA_interactions.txt.gz &#xA;PF00036 PF00531&#xA;PF00531 PF14786&#xA;PF00531 PF00531&#xA;PF00791 PF00531&#xA;PF01335 PF00531&#xA;PF13499 PF00531&#xA;PF13833 PF00531&#xA;PF14786 PF00531&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Those are 8 interactions, which matches what the PFam link you gave in your question shows. If you want to convert the accessions to something more informative, you could try Pfam's REST API and do something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ zgrep PF00531 pfamA_interactions.txt.gz | &#xA;    while read a b; do &#xA;    name1=$&lt;/span&gt;(wget -O - &quot;https://pfam.xfam.org/family/&lt;span class=&quot;math-container&quot;&gt;${a}?output=xml&quot; 2&amp;gt;/dev/null | &#xA;     perl -00lne 'm|&amp;lt;description&amp;gt;.*?CDATA\[\s*(.*?)\s\].*?&amp;lt;/description&amp;gt;|s &amp;amp;&amp;amp; print $&lt;/span&gt;1'); &#xA;    name2=&lt;span class=&quot;math-container&quot;&gt;$(wget -O - &quot;https://pfam.xfam.org/family/$&lt;/span&gt;{b}?output=xml&quot; 2&amp;gt;/dev/null | &#xA;     perl -00lne 'm|&amp;lt;description&amp;gt;.*?CDATA\[\s*(.*?)\s\].*?&amp;lt;/description&amp;gt;|s &amp;amp;&amp;amp; print &lt;span class=&quot;math-container&quot;&gt;$1'); &#xA;    printf &quot;%s\t%s\t%-20s\t%-25s\n&quot; &quot;$&lt;/span&gt;a&quot; &quot;&lt;span class=&quot;math-container&quot;&gt;$b&quot; &quot;$&lt;/span&gt;name1&quot; &quot;$name2&quot;; &#xA;done&#xA;PF00036 PF00531 EF hand                 Death domain             &#xA;PF00531 PF14786 Death domain            Tube Death domain        &#xA;PF00531 PF00531 Death domain            Death domain             &#xA;PF00791 PF00531 ZU5 domain              Death domain             &#xA;PF01335 PF00531 Death effector domain   Death domain             &#xA;PF13499 PF00531 EF-hand domain pair     Death domain             &#xA;PF13833 PF00531 EF-hand domain pair     Death domain             &#xA;PF14786 PF00531 Tube Death domain       Death domain         &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Obviously, if you need anything more fancy than that, it's time to write a proper parser using Perl or Python or any other language that can understand XML.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-05-29T18:53:10.393" LastActivityDate="2019-05-29T18:53:10.393" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8722" PostTypeId="1" CreationDate="2019-05-30T05:27:50.610" Score="2" ViewCount="1465" Body="&lt;p&gt;This is a question from the Oxford Nanopore community, from user &lt;em&gt;Michael Radzieta&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I have sequenced some plasmids using the rapid barcoding kit, I have attempted to assemble the data using several programs (unicycler, spades, ra) but nothing has been able to assemble the plasmids I am expecting. I have read that if the coverage is too high, then it can negatively affect the assembly. Based on the approximate size of my plasmids and the number of filtered reads I have, the coverage is estimated at &gt;5000. &lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Can anyone recommend a tool to subsample or normalise nanopore reads to a certain depth, such as 100x coverage?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-12-14T09:58:01.337" Title="How do you normalise read coverage in a BAM file?" Tags="|sam|nanopore|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8723" PostTypeId="2" ParentId="8722" CreationDate="2019-05-30T05:27:50.610" Score="2" Body="&lt;p&gt;I just updated one of my SAM-normalisation tools to more accurately normalise mapped reads to a target coverage. It uses reservoir sampling on a SAM-formatted file to choose reads to bring the coverage up to 100X (if possible).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://gitlab.com/gringer/bioinfscripts/blob/master/samNormalise.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gitlab.com/gringer/bioinfscripts/blob/master/samNormalise.pl&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Example Usage:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h input.bam | ~/scripts/samNormalise.pl | samtools view -b &amp;gt; normalised.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output BAM file can be converted into a fastq file using 'samtools fastq':&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools fastq normalised.bam &amp;gt; normalised.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here are some example before and after shots ['samtools mpileup input.bam'; 'samtools mpileup normalised.bam']:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ODrgG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ODrgG.png&quot; alt=&quot;Before normalisation&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XAl2s.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XAl2s.png&quot; alt=&quot;After normalisation&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Notice that where the coverage is below 100, the coverage is identical. Coverage is below 100 in some areas because some reads end, and no additional reads start until later on in the reference sequence.&lt;/p&gt;&#xA;&lt;p&gt;The options are coded as variables at the beginning of the script at the moment, but I can easily convert them to command-line options in the future if there's demand for it.&lt;/p&gt;&#xA;&lt;p&gt;This will work for long reads (i.e. PacBio, Nanopore), bearing in mind that there may be more low-coverage gaps if the reads are sparsely placed.&lt;/p&gt;&#xA;&lt;p&gt;As implemented, this won't work properly with paired reads and will leave behind singletons.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-12-14T09:58:01.337" LastActivityDate="2023-12-14T09:58:01.337" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8724" PostTypeId="1" AcceptedAnswerId="19305" CreationDate="2019-05-30T07:51:29.547" Score="5" ViewCount="287" Body="&lt;p&gt;I'm looking at better database/storage solutions for NCBI virus data, with all attributes particularly year and country of isolation, together with structural data, possible antibody data, T-cell data and bioinformatics data such as %GC etc... The amount of data is comparatively small, with a maximum size of &lt;code&gt;3 x ([6000 &amp;lt; no. viruses &amp;lt; 20000] for 10 genes)&lt;/code&gt; and each entry will contain nucleotide, amino acid and one or more pieces of alignment data (i.e. with indels). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data will be used for numerous different calculations from structural biology, phylogenetics, and deep learning. Some of the calculations will require parallel processing. A typical calculation would be to identify a clade/similarity/difference then screen the database to look at the isolates information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Previously I've used lots of flat files which are loaded at initiation and are held within a data pipeline either as an object (Python) or especially a multi-dimensional hash (Perl, preferred).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two solutions that I think look suitable are: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://biosql.org&quot; rel=&quot;noreferrer&quot;&gt;BioSQL&lt;/a&gt; - The official Genbank solution, however my SQL commands are very basic and the documentation appears minimal (for BioSQL)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.hdfgroup.org/solutions/hdf5/&quot; rel=&quot;noreferrer&quot;&gt;HDF5&lt;/a&gt; (hierarchical data format) - this looks good because it uses complex hashes (dictionaries), and allows layering of multiple information onto a single Genbank sequence code (key). It has strong documentation. However, I understand this isn't a &quot;database&quot; (long term storage), but thats how I intend to use it.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;JSON/Pickling&lt;/strong&gt; I had a look at JSON, which to me looks okay, because I'm fond of complex hashes / dictionaries, but it appears to be more of a Web solution. Finally, &quot;pickling&quot; (Python) is cool, very cool, but I understand this not a database solution, because it risk compatibility issues, e.g. with future Python releases. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in knowing what archival systems board members are using, and a critique whether my assessment is ok for moving forward.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-05-30T08:44:33.347" LastActivityDate="2022-06-26T13:32:16.943" Title="HDF5 and BioSQL solutions" Tags="|python|perl|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8725" PostTypeId="1" CreationDate="2019-05-30T13:05:57.503" Score="1" ViewCount="112" Body="&lt;p&gt;I have recently downloaded &lt;a href=&quot;http://annovar.openbioinformatics.org/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Annovar&lt;/a&gt;, and succesfully filtered my files with a few databases, downloaded from Annovar. But when I try to do the exact same thing with the 1000 genomes project database, I get this error many times in the terminal, one for each line of **.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Argument &quot;&amp;amp;&quot; isn't numeric in numeric eq (==) at annotate_variation.pl line 1962, &amp;lt;DB&amp;gt; line &amp;amp;&amp;amp;.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&amp;amp;&lt;/code&gt; can be &lt;code&gt;A&lt;/code&gt;, &lt;code&gt;C&lt;/code&gt;, &lt;code&gt;G&lt;/code&gt;, or &lt;code&gt;T&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&amp;amp;&amp;amp;&lt;/code&gt; every line of the database&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried filtering with &lt;code&gt;1000g&lt;/code&gt;, &lt;code&gt;1000g2010&lt;/code&gt; and &lt;code&gt;1000g2010jul&lt;/code&gt; databases, but all give these errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of the commands I use for filtering:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl annotate_variation.pl --filter --dbtype CEU.sites.2010_07 testfile1.txt humandb/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can anyone tell me if I'm doing something wrong, if this is known, and if I can do anything about it?&lt;/p&gt;&#xA;" OwnerUserId="4658" LastEditorUserId="57" LastEditDate="2019-05-30T15:54:40.600" LastActivityDate="2019-06-29T16:01:55.503" Title="ANNOVAR, 1000g database calling" Tags="|sequence-annotation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8726" PostTypeId="1" CreationDate="2019-05-30T13:40:48.207" Score="2" ViewCount="90" Body="&lt;p&gt;I have orf virus sequence in GFF3 format. I am trying to create a txDb using &lt;code&gt;GenomicFeatures::makeTxDbFromGFF()&lt;/code&gt; function. But, it is not able to capture any CDS: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;transcript_nrow: 0&#xA;exon_nrow: 0&#xA;cds_nrow: 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Command I used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;orf &amp;lt;- GenomicFeatures::makeTxDbFromGFF(&quot;orf.gff3&quot;,format=&quot;gff3&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/AY386263.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link&lt;/a&gt; to the genome in NCBI.&lt;/p&gt;&#xA;" OwnerUserId="1684" LastEditorUserId="48" LastEditDate="2019-05-30T15:05:17.930" LastActivityDate="2019-05-30T15:05:17.930" Title="How to create custom txDb using GenomeFeatures?" Tags="|bioconductor|genomefeatures|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8727" PostTypeId="1" CreationDate="2019-05-30T15:42:59.600" Score="0" ViewCount="46" Body="&lt;p&gt;I need to store a large number of variants with their annotations. I need to import dbSNP, as well as Clinvar, and ExAC etc. Also I have sample variant data from patients. I need to be able to search on genomic regions as well as annotations. And it needs to be scalable.&#xA;What would be a preferred data storage technology for such a case? What is the industry using? It needs to be accessible by APIs/REST.&#xA;We are using ElasticSearch, but that is proving to be expensive in terms of servers needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something similar to this has been discussed in &#xA;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/195/the-state-limitations-and-comparisons-of-large-variant-stores&quot;&gt;The state, limitations and comparisons of large variant stores&lt;/a&gt;, but that post is a little old, I am sure there has been progress since then.&lt;/p&gt;&#xA;" OwnerUserId="4660" LastEditorUserId="4660" LastEditDate="2019-05-30T17:53:20.920" LastActivityDate="2019-05-30T17:53:20.920" Title="Store and search on variant and annotation data" Tags="|database|snp|indel|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8728" PostTypeId="2" ParentId="8725" CreationDate="2019-05-30T15:53:37.467" Score="1" Body="&lt;p&gt;That's a warning. Presumably, a match failed (&lt;code&gt;$&amp;amp;&lt;/code&gt; is the default Perl variable which holds the result of a match operation). So something in your input is not exactly what the script was expecting. You can probably safely ignore these messages. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, update your question and add a few example lines of input which we can use to reproduce the error and maybe we can figure something out. If that isn't possible, at least show us line 1962 of the &lt;code&gt;annotate_variation.pl&lt;/code&gt; script since that's the one that's failing. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-05-30T15:53:37.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8729" PostTypeId="2" ParentId="8727" CreationDate="2019-05-30T17:12:39.923" Score="1" Body="&lt;p&gt;The question you linked is about storing genotypes. Your question is about keeping site information, which is different and much easier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to solutions, you can parse the information you want and store them in a standard SQL database. You can use UCSC binning for region queries. Modern SQL databases all provide R-tree indices, which can be adapted for region query and is probably better these days.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2019-05-30T17:12:39.923" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8732" PostTypeId="1" AcceptedAnswerId="8733" CreationDate="2019-05-31T08:00:31.107" Score="3" ViewCount="5420" Body="&lt;p&gt;I have two VCF files and I want to compare the missing rate in each of them, but I want to only look at sites that are present in both files.  How could I go about getting a list of positions that are included in both VCF files and then filtering those VCF files to only include those positions?&lt;/p&gt;&#xA;" OwnerUserId="1574" LastEditorUserId="96" LastEditDate="2019-06-03T15:02:01.837" LastActivityDate="2019-06-03T15:02:01.837" Title="Find overlap between VCF files" Tags="|vcf|variant-calling|variation|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8733" PostTypeId="2" ParentId="8732" CreationDate="2019-05-31T08:06:24.483" Score="2" Body="&lt;p&gt;Something like &lt;a href=&quot;http://vcftools.sourceforge.net/perl_module.html#vcf-isec&quot; rel=&quot;nofollow noreferrer&quot;&gt;vcf-isec&lt;/a&gt;? Quoting:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Creates intersections and complements of two or more VCF files. Given multiple VCF files, it can output the list of positions which are shared by at least N files, at most N files, exactly N files, etc. The first example below outputs positions shared by at least two files and the second outputs positions present in the files A but absent from files B and C. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3818" LastEditorUserId="96" LastEditDate="2019-06-03T14:59:14.647" LastActivityDate="2019-06-03T14:59:14.647" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8734" PostTypeId="1" AcceptedAnswerId="8738" CreationDate="2019-05-31T10:11:29.847" Score="1" ViewCount="761" Body="&lt;p&gt;I have bit of confusion regarding how ggpubr has implemented the KW and wilcox.test. By default usage of ggpubr reports p value for wilcox.test() but when it comes to Kruskal–Walis it is different .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But as i have to compare more that two groups im opting for Kruskal–Wallis so now the confusion is i get a p value which is significant but if i do the test independently then the p value is not significant not sure why is that &#xA;I have asked similar question &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/6739/kruskal-wallis-rank-sum-test-output-confusion&quot;&gt;old question&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My data &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dput(df1)&#xA;structure(list(mHSC = c(7.7559725, 3.307901197229, 7.91049997141824, &#xA;10.9659075, 5.66225942792448, 9.40594075, 8.03212169895329, 10.929870369027, &#xA;9.80897606357472, 4.29298594445745, 8.58579242742772, 12.0936123426199, &#xA;8.61371460307564, 11.0174749823542, 10.5147718423478, 5.53565725, &#xA;8.12612925, 10.6881344317581, 8.48826962785442, 5.52294757621421, &#xA;8.38046925, 9.44552809599148, 9.19142493299942, 7.03072998196466, &#xA;8.88679975, 0.3126697975, 10.4907798679713, 7.864884, 12.7182883447252, &#xA;6.3180225, 5.87640020441158, 7.87333318550091, 8.15221192095875, &#xA;4.51086624773858, 5.957332, 6.90057575, 5.64708909092112, 10.7124580115802, &#xA;9.82466018308675, 8.49078, 8.63046025, 6.96721675, 7.57871225, &#xA;8.815752, 6.72449933154642, 10.7951750189282, 8.01780331997795, &#xA;6.07469990935439, 8.31037653431384, 9.60791091202699, 7.11738304581177, &#xA;6.27364283988744, 10.8712275, 4.76963356477331, 8.22113922937348, &#xA;4.68111125, 7.81847825, 8.4123615, 7.1236546846872, 5.5773485, &#xA;10.72958, 7.716703, 7.606539, 9.3070835272045, 7.75540042667672, &#xA;9.8687305, 8.98201221147182, 8.93182675, 10.868215, 10.8990875, &#xA;10.9377104128057, 8.60048104950787, 7.81762687302046, 9.24377416902964, &#xA;6.57057975, 7.52549392459474, 5.9971085, 10.7335671888057), mCMP = c(8.08511525, &#xA;3.15081874988432, 8.21803244045432, 12.7012625, 6.95234868015311, &#xA;10.24927775, 8.02201688082928, 11.1415993774267, 8.6576364168622, &#xA;4.33979841956685, 9.20120340882164, 12.6360469440726, 8.82394480426142, &#xA;11.7614369660881, 10.518813583537, 5.9224525, 8.3939755, 10.9519826793289, &#xA;9.07562352307607, 4.68003855905254, 8.7914695, 9.35721245249627, &#xA;9.52072790562384, 7.52153767036027, 9.37291475, 5.323077, 11.316187036788, &#xA;8.08079325, 12.9250246147509, 6.3281185, 6.36628784788811, 8.26354221670866, &#xA;8.80757801591239, 5.08139074215397, 6.6258195, 7.56676875, 6.30618317691088, &#xA;12.8268812686057, 10.4361513118013, 9.12211125, 8.94618675, 7.25775925, &#xA;8.7313635, 9.8282665, 7.10318403743241, 10.9016853993601, 8.75786307748107, &#xA;6.2613305781586, 7.24503928286349, 10.1900260653741, 6.04352467875968, &#xA;5.41919427988363, 11.689205, 5.2447156735349, 7.81023802636728, &#xA;5.484589, 7.9321635, 8.8598865, 7.26208235921707, 11.7873, 11.256575, &#xA;8.574748, 8.3324525, 9.69726624009281, 8.51491553090231, 10.792235, &#xA;9.97113940283311, 10.08911925, 11.867595, 11.8185525, 11.9422571106972, &#xA;9.19311951548278, 8.61818925695215, 9.34701182556221, 7.063838, &#xA;7.41275116679947, 6.84360325, 10.478012600725), mGMP = c(8.07654075, &#xA;3.12607874540186, 7.91384610943697, 12.9717825, 6.533351883655, &#xA;10.394515, 8.26093661667729, 11.2346407524235, 11.1019372952288, &#xA;4.34824706553879, 9.19896153987714, 12.6475029547774, 8.53466968298702, &#xA;11.6750087661439, 10.7705094222618, 5.60385575, 8.32733375, 10.6929416341222, &#xA;8.85393492258581, 4.40609754458633, 9.2379455, 9.43854664655775, &#xA;9.66597253836355, 7.99640783151295, 9.60382225, 0.40676415, 11.3480173068737, &#xA;8.2298565, 13.3988341481968, 5.2305385, 6.12320931339542, 8.16212407932226, &#xA;8.98321413854527, 5.21894298832466, 6.88443175, 7.9935035, 6.66952804861909, &#xA;11.3240966560241, 10.795029283416, 8.830129, 8.58251875, 7.6113605, &#xA;8.508246, 9.63661975, 6.87344684043907, 11.2237814311451, 9.43948359781618, &#xA;6.10350570815157, 8.37769527299503, 9.98272594007505, 7.15195399920282, &#xA;5.48458261850125, 11.8302525, 5.66643624946888, 8.36658108147322, &#xA;5.9928725, 8.56603575, 9.670526, 7.70714893990735, 9.552538, &#xA;11.5112725, 8.99551975, 9.504391, 9.76913665769114, 8.58112436662299, &#xA;11.2680775, 10.5535730280208, 10.226635, 11.9211375, 12.0610625, &#xA;12.3249064903557, 9.52625419135532, 8.66295412469679, 10.0469451814325, &#xA;7.700204, 7.44384030300093, 7.40615025, 10.222430040369), mMono = c(6.86074675, &#xA;0.899186661852538, 5.72102113412742, 11.0054, 5.22275722758541, &#xA;8.56569375, 7.25543228485158, 9.37557140457706, 7.92766781596895, &#xA;1.22333714157354, 7.54589243251616, 11.0881258432345, 7.91303390155615, &#xA;10.3482735440588, 7.2731491579751, 3.10091425, 6.91085225, 9.03107467062931, &#xA;7.91879733419338, 2.71978845710368, 8.31396425, 6.89137159431571, &#xA;8.68813944060236, 6.65073272892769, 8.41960525, 0.283334725, &#xA;10.0529313221945, 6.7606855, 11.6800576702458, 3.6173355, 3.81146596960356, &#xA;6.93219836413735, 7.92727013246883, 4.09314670396697, 5.78634575, &#xA;6.55346475, 5.05937426539404, 10.0514401266737, 9.21573912287031, &#xA;7.80863325, 7.80450125, 5.96109, 6.07350675, 7.14264825, 5.20991063841094, &#xA;8.9541923959891, 5.48494732937368, 3.46755807112085, 5.91011468781522, &#xA;6.19799161521768, 4.09986325310833, 1.58331376348435, 11.021745, &#xA;4.62557956951436, 7.48457597125806, 4.9819695, 7.567706, 8.547291, &#xA;7.13353425765299, 9.4336955, 10.6991475, 7.3555725, 8.32300975, &#xA;9.14731426222114, 8.2750379299376, 10.85821, 9.63317457338075, &#xA;9.6021585, 11.4682025, 11.4990425, 11.4153866521691, 9.02074441245752, &#xA;7.96069342402473, 9.45598020603536, 6.462563, 6.78696488936524, &#xA;5.50898575, 8.79251593030914)), class = &quot;data.frame&quot;, row.names = c(NA, &#xA;-78L))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Code for the first figure with wilcox test and KW as argument &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  head(df1)&#xA;&#xA;ex &amp;lt;- melt(df1, id.vars=c(NULL))&#xA;&#xA;head(ex)&#xA;my_comparisons &amp;lt;- list( c(&quot;mHSC&quot;,&quot;mCMP&quot;),c(&quot;mCMP&quot;,&quot;mGMP&quot;),c(&quot;mGMP&quot;,&quot;mMono&quot;),c(&quot;mHSC&quot;,&quot;mMono&quot;),c(&quot;mHSC&quot;,&quot;mGMP&quot;))&#xA;&#xA;head(ex)&#xA;&#xA;&#xA;&#xA;ggboxplot(ex, x = &quot;variable&quot;, y = &quot;value&quot;,ggtheme = theme_bw(base_size = 30),&#xA;          color = &quot;variable&quot;, palette = &quot;jco&quot;, add = &quot;jitter&quot;)+ &#xA;  stat_compare_means(comparisons = my_comparisons,method = &quot;kruskal.test&quot;)+ # Add pairwise comparisons p-value&#xA;  stat_compare_means(label.y = 50)     # Add global p-value&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NzN1U.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NzN1U.jpg&quot; alt=&quot;Global KW p value&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;New figure &lt;a href=&quot;https://i.stack.imgur.com/kV7AV.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kV7AV.jpg&quot; alt=&quot;KW test with same global KW value&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The one im not able to get is in the first figure even with Wilcox and KW as argument the global p value for KW doesn't change where as the KW p value for comparison gives p value which is not significant .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am i doing it correctly ?&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2019-06-01T16:47:34.977" Title="Kruskal–Wallis and wilcox.test choice of test for expression data" Tags="|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8735" PostTypeId="1" AcceptedAnswerId="8742" CreationDate="2019-05-31T11:19:19.173" Score="6" ViewCount="3392" Body="&lt;p&gt;I have found a couple of sources&lt;sup&gt;&lt;a href=&quot;http://www.bioconductor.org/packages//release/data/experiment/vignettes/minionSummaryData/inst/doc/creating-Styphi-Dataset.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;,&lt;a href=&quot;https://porecamp.github.io/2016/tutorials/PoreCamp2016-02-MinIONData.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;&lt;/sup&gt; that indicate that a read in a 1D² run is classified by MinKNOW as &quot;pass&quot; and put into the &lt;code&gt;fastq_pass&lt;/code&gt; folder if both of the following conditions are met:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Both strands were read&lt;/li&gt;&#xA;&lt;li&gt;The average Phred score of all bases in the read was &gt;9&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;(Thus a read in a 1D² run goes into the &lt;code&gt;fastq_fail&lt;/code&gt; folder if either its Phred is too low &lt;em&gt;or&lt;/em&gt; if the second strand didn't follow the first through the pore.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I have been unable to track down any source indicating how the pass/fail classification works for 1D runs (where only one strand is supposed to be read per read). One might reasonably guess that it's based solely upon whether the mean Phred score is above 9, but inspection of the data from a 1D run reveals that this is not the case. While all &quot;pass&quot; reads seem to have a mean Phred score of at least 9 (the lowest I see is 9.45), some fail reads have scores above 9 too (with the highest one being 12.9):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CRgzT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CRgzT.png&quot; alt=&quot;Histogram illustrating the distribution described above&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus it looks to me like this is not the condition used to classify 1D reads. But then what is?&lt;/p&gt;&#xA;" OwnerUserId="3144" LastEditorUserId="3144" LastEditDate="2019-06-03T11:22:37.353" LastActivityDate="2019-06-03T11:22:37.353" Title="How does MinKNOW classify 1D reads as &quot;pass&quot; or &quot;fail&quot;?" Tags="|nanopore|minion|minknow|base-calling|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8736" PostTypeId="2" ParentId="7027" CreationDate="2019-05-31T19:29:54.080" Score="1" Body="&lt;p&gt;I've been told by the support team that there's a way to pull it out in csv form by GETting the following URL, replacing &lt;code&gt;&amp;lt;SRRNUMBER&amp;gt;&lt;/code&gt;, with, e.g. &lt;code&gt;SRR000001&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;https://trace.ncbi.nlm.nih.gov/Traces/sra/sra.cgi?save=efetch&amp;amp;db=sra&amp;amp;rettype=runinfo&amp;amp;term=&amp;lt;SRRNUMBER&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So this would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# download the file attachment&#xA;wget -nv -O foo.csv  &quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/sra.cgi?save=efetch&amp;amp;db=sra&amp;amp;rettype=runinfo&amp;amp;term=SRR000001&quot;&#xA;&#xA;# or just get the output on stdout&#xA;curl -v &quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/sra.cgi?save=efetch&amp;amp;db=sra&amp;amp;rettype=runinfo&amp;amp;term=SRR000001&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As an example, the output looks like this for SRR000001. It namely contains additional info on other runs from the same SRX:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Run,ReleaseDate,LoadDate,spots,bases,spots_with_mates,avgLength,size_MB,AssemblyName,download_path,Experiment,LibraryName,LibraryStrategy,LibrarySelection,LibrarySource,LibraryLayout,InsertSize,InsertDev,Platform,Model,SRAStudy,BioProject,Study_Pubmed_id,ProjectID,Sample,BioSample,SampleType,TaxID,ScientificName,SampleName,g1k_pop_code,source,g1k_analysis_group,Subject_ID,Sex,Disease,Tumor,Affection_Status,Analyte_Type,Histological_Type,Body_Site,CenterName,Submission,dbgap_study_accession,Consent,RunHash,ReadHash&#xA;SRR000001,2008-04-04 14:36:13,2015-04-07 21:54:15,470985,129515135,236041,274,298,,https://sra-download.ncbi.nlm.nih.gov/traces/sra27/SRR/000000/SRR000001,SRX000007,SID2748,WGS,RANDOM,GENOMIC,PAIRED,3000,0,LS454,454 GS FLX,SRP000001,PRJNA33627,1,33627,SRS000100,SAMN00001583,simple,9606,Homo sapiens,NA18505,,,,,female,,no,,,,,454MSC,SRA000197,,public,25D46C95842BCD2F235DDD48070341C4,5FC1B56C9A5332BD21A6FCC6713C3159&#xA;SRR000013,2008-04-04 15:15:44,2012-01-19 15:14:06,478304,131410350,316908,274,307,,https://sra-download.ncbi.nlm.nih.gov/traces/sra4/SRR/000000/SRR000013,SRX000007,SID2748,WGS,RANDOM,GENOMIC,PAIRED,3000,0,LS454,454 GS FLX,SRP000001,PRJNA33627,1,33627,SRS000100,SAMN00001583,simple,9606,Homo sapiens,NA18505,,,,,female,,no,,,,,454MSC,SRA000197,,public,A576F73351E329617F91C92845ADE3BE,0EFE27254FB8790C80E4A8A3A3CCBF1D&#xA;SRR000019,2008-04-04 15:20:15,2012-01-19 15:14:06,537808,147928455,358612,275,347,,https://sra-download.ncbi.nlm.nih.gov/traces/sra4/SRR/000000/SRR000019,SRX000007,SID2748,WGS,RANDOM,GENOMIC,PAIRED,3000,0,LS454,454 GS FLX,SRP000001,PRJNA33627,1,33627,SRS000100,SAMN00001583,simple,9606,Homo sapiens,NA18505,,,,,female,,no,,,,,454MSC,SRA000197,,public,5E42E66B2972A28C3C1B714499B3CE1E,A44F729DF562E11BDA4F950F171085B3&#xA;SRR000033,2008-04-04 15:26:19,2012-01-19 15:14:16,476737,131230950,316780,275,307,,https://sra-download.ncbi.nlm.nih.gov/traces/sra4/SRR/000000/SRR000033,SRX000007,SID2748,WGS,RANDOM,GENOMIC,PAIRED,3000,0,LS454,454 GS FLX,SRP000001,PRJNA33627,1,33627,SRS000100,SAMN00001583,simple,9606,Homo sapiens,NA18505,,,,,female,,no,,,,,454MSC,SRA000197,,public,8D775F0FB1E78F2818166B4E6C7BC320,6C794108B0172AE98323C9A809BE892E&#xA;SRR000047,2008-04-04 15:34:57,2012-01-19 15:14:25,322279,88301883,214744,273,208,,https://sra-download.ncbi.nlm.nih.gov/traces/sra4/SRR/000000/SRR000047,SRX000007,SID2748,WGS,RANDOM,GENOMIC,PAIRED,3000,0,LS454,454 GS FLX,SRP000001,PRJNA33627,1,33627,SRS000100,SAMN00001583,simple,9606,Homo sapiens,NA18505,,,,,female,,no,,,,,454MSC,SRA000197,,public,B4D7BC3DAD22AE58EADC18B2F05B2732,154C9532706F955A63CF685C4D71D9E6&#xA;SRR000049,2008-04-04 15:35:58,2012-01-19 15:14:34,506408,139029601,328595,274,336,,https://sra-download.ncbi.nlm.nih.gov/traces/sra4/SRR/000000/SRR000049,SRX000007,SID2748,WGS,RANDOM,GENOMIC,PAIRED,3000,0,LS454,454 GS FLX,SRP000001,PRJNA33627,1,33627,SRS000100,SAMN00001583,simple,9606,Homo sapiens,NA18505,,,,,female,,no,,,,,454MSC,SRA000197,,public,60D02918CD7686DA084607BFD9D8E6A5,E9A732F2A8F36B8EB1CD38CA41BA8FB5&#xA;SRR000061,2008-04-04 15:41:48,2012-01-19 15:14:37,532884,146330433,351541,274,348,,https://sra-download.ncbi.nlm.nih.gov/traces/sra4/SRR/000000/SRR000061,SRX000007,SID2748,WGS,RANDOM,GENOMIC,PAIRED,3000,0,LS454,454 GS FLX,SRP000001,PRJNA33627,1,33627,SRS000100,SAMN00001583,simple,9606,Homo sapiens,NA18505,,,,,female,,no,,,,,454MSC,SRA000197,,public,8F6E1CA597BCAA19A852036775FA1FAB,2C035CCBD62808A01CEF6EB13BAEB40F&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Providing an invalid SRR number will generate an empty HTTP 200 response.&lt;/p&gt;&#xA;" OwnerUserId="3235" LastEditorUserId="3235" LastEditDate="2019-05-31T20:54:27.530" LastActivityDate="2019-05-31T20:54:27.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8737" PostTypeId="2" ParentId="8732" CreationDate="2019-05-31T23:01:49.547" Score="2" Body="&lt;p&gt;You can try something like this in &lt;code&gt;bcftools&lt;/code&gt;, which is a bit quicker than VCFtools, as it uses block gzipping and indexing.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# make sure both files are indexed and in bgzip format &#xA;bcftools index vcfA.vcf.gz&#xA;bcftools index vcfB.vcf.gz&#xA;&#xA;# find overlapping positions between the two files&#xA;bcftools isec -n~11 -c all vcfA.vcf.gz vcfB.vcf.gz &amp;gt; sharedPositions.txt&#xA;&#xA;# filter vcf files for the overlapping positions&#xA;bcftools view -T sharedPositions.txt vcfA.vcf.gz -Oz &amp;gt; vcfA.sharedPositions.vcf.gz&#xA;&#xA;bcftools view -T sharedPositions.txt vcfB.vcf.gz -Oz &amp;gt; vcfB.sharedPositions.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The perhaps use something like plink2 to look at the missing rate per variant and individual for each file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plink2 --vcf vcfA.vcf.gz --missing --out vcfA.missingStats&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2019-05-31T23:01:49.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8738" PostTypeId="2" ParentId="8734" CreationDate="2019-06-01T04:50:17.967" Score="1" Body="&lt;p&gt;The lower graph doesn't look good. The &quot;Wilcoxan values&quot; are all identical and I can't see how if all pairwise comparisons were 0.46 that would result in a KW of &amp;lt;&amp;lt;0.001. The way the values hold together in the upper graph looks fine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just to summarise &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;KW = 1 way ANOVA ... non-parametric version&lt;/li&gt;&#xA;&lt;li&gt;Wilcoxan = T test ... non-parametric version&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Looking at your code you specify the pairwise comparisons but then in the ggplot2 code you state &lt;code&gt;method = &quot;kruskal.test&quot;&lt;/code&gt; ... are you sure that isn't &lt;code&gt;method = &quot;wilcoxan.test&quot;&lt;/code&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;KW will want to perform a global comparison across multiple single columns of data and this might have resulted in some sort of weird mean value being generated, because it will just want to output a single value (my bet is that is why all values are identical). Wilcoxan will be expecting to a request to perform pairwise tests.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Okay my advice is to perform an independent Wilcoxan test outside of ggpubr&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wtest &amp;lt;- wilcox.test(mCMP ~ mHSC, data = df1, paired = TRUE)&#xA;wtest&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&#xA;    Wilcoxon signed rank test&#xA;data:  weight by group&#xA;.....&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wtest$p.value&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&#xA;p value&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would then simply remove the calculation from with the boxplot and present the p values independently. &lt;em&gt;The only other possibility is whether the data is paired or not and you may have changed that setting between calculations&lt;/em&gt;. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-06-01T16:47:34.977" LastActivityDate="2019-06-01T16:47:34.977" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8739" PostTypeId="1" CreationDate="2019-06-02T12:49:31.040" Score="1" ViewCount="101" Body="&lt;p&gt;I have a list of cell markers differentially expressed between 2 groups of cells, that I got with the FindMarkers command in Seurat. I want to enrich for those genes that are expressed in the cell membrane. Are there any specific tools/commands I can use to do that? Thanks! &lt;/p&gt;&#xA;" OwnerUserId="4600" LastActivityDate="2019-07-03T08:01:45.187" Title="Enrich for genes expressed in the plasma membrane" Tags="|seurat|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8740" PostTypeId="2" ParentId="8739" CreationDate="2019-06-03T07:06:50.777" Score="1" Body="&lt;p&gt;You can do functional enrichment analysis, for instance with &lt;code&gt;gProfiler&lt;/code&gt; in R and see if your marker genes are enriched in any GO term related with the plasma membrane. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you can generate a list with genes whose product are located at the plasma membrane and perform Gene Set Enrichment Analysis, for instance with &lt;code&gt;fgsea&lt;/code&gt; in R.&lt;/p&gt;&#xA;" OwnerUserId="678" LastEditorUserId="678" LastEditDate="2019-06-03T07:23:30.197" LastActivityDate="2019-06-03T07:23:30.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8741" PostTypeId="1" CreationDate="2019-06-03T09:07:31.740" Score="0" ViewCount="174" Body="&lt;p&gt;is there a way to subset a group of cells based on the foldchange of a gene expression? I have the KRAS object KRAS An object of class Seurat 53805 features across 6826 samples within 1 assay Active assay: RNA (53805 features). I want to subset on the expression of the Olfm4 gene, but with a statistical threshold, for example a logfold2 change of 1. In the meantime I use subset(KRAS,subset='Olfm4'&gt;1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks &lt;/p&gt;&#xA;" OwnerUserId="4600" LastEditorUserId="4600" LastEditDate="2019-06-03T12:44:16.197" LastActivityDate="2019-06-03T12:44:16.197" Title="Subset cells based on a changefold level in gene expression" Tags="|seurat|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8742" PostTypeId="2" ParentId="8735" CreationDate="2019-06-03T11:21:10.927" Score="6" Body="&lt;p&gt;The classification is indeed based purely on comparing a mean to a threshold... but it's &lt;em&gt;not&lt;/em&gt; the mean of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Phred_quality_score&quot; rel=&quot;noreferrer&quot;&gt;Phred scores&lt;/a&gt; that represent base quality on a logarithmic scale, but rather the mean of the underlying quality values prior to being transformed into Phreds.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I compute the &quot;mean&quot; quality for a read in this way - by converting each Phred in the read back to a probability, taking the mean of those probabilities, and then converting that mean probability to a Phred - I find that whether a read is a pass or fail is determined by comparing that score to a threshold of 7:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9YuQj.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9YuQj.png&quot; alt=&quot;Historgram illustrating the claim above&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A mean quality score computed in this way is also included in the &lt;code&gt;mean_qscore_template&lt;/code&gt; column of the sequencing summary file. If you open that file in a spreadsheet program (it's just a TSV) and sort by &lt;code&gt;mean_qscore_template&lt;/code&gt;, you will find that the &lt;code&gt;passes_filter&lt;/code&gt; column is always &lt;code&gt;FALSE&lt;/code&gt; for &lt;code&gt;mean_qscore_template&lt;/code&gt; values below 7 and always &lt;code&gt;TRUE&lt;/code&gt; for values above 7.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Hat tip to &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/681/wouter-de-coster&quot;&gt;Wouter De Coster&lt;/a&gt; for successfully guessing that this was the answer in the comments and pointing me on the right path.)&lt;/p&gt;&#xA;" OwnerUserId="3144" LastActivityDate="2019-06-03T11:21:10.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8743" PostTypeId="1" CreationDate="2019-06-03T11:44:06.570" Score="1" ViewCount="23" Body="&lt;p&gt;I am doing simulations using a program 'ms':&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://snoweye.github.io/phyclust/document/msdoc.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://snoweye.github.io/phyclust/document/msdoc.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at page 3 of this pdf, it explains the output produced by ms. The line which starts with positions tells us the position of segregating sites. But it is between 0 and 1. My simulation produces a chromosome of length 1.1Mb and I want to divide it into 100kb subwindows each. However, I don't know quite understand how I should determine which segregating site (which is a float number) falls into which sub window. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone explain how this can be done? &lt;/p&gt;&#xA;" OwnerUserId="4636" LastActivityDate="2019-06-03T11:44:06.570" Title="the positions computed by ms" Tags="|python|phylogenetics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8744" PostTypeId="1" CreationDate="2019-06-03T13:11:35.057" Score="1" ViewCount="128" Body="&lt;p&gt;As the title says I would like to know which alignment method(global or local) should I use for sequences for which is assumed that they have common ancestry?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sequences are 21 aa long peptide chains (proteins).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the background of the sequences, it's only stated that it's ASSUMED that they are from the same ancestry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing that we assume that they are from the same ancestry makes me confused. I know that on similar sequences with the same length usually, we use global.  &lt;/p&gt;&#xA;" OwnerUserId="4676" LastEditorUserId="57" LastEditDate="2019-06-03T13:48:29.000" LastActivityDate="2019-06-03T14:32:50.977" Title="Global or local alignment on sequences for which it is assumed that they have common ancestry" Tags="|sequence-alignment|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8745" PostTypeId="2" ParentId="8724" CreationDate="2019-06-03T14:11:16.600" Score="1" Body="&lt;p&gt;Just to conclude, @user172818 and @AlexReynolds are collectively saying either learn SQL or stick with CSV. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Update,&#xA;I can provide an answer. It's HDF5 with pandas dataframes (Python).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Simply&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df.to_hdf('path/file.h5', 'key', format='table')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This isn't SQL but it works:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What it gives,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;it's fast, that's it's purpose (not my main reason though)&lt;/li&gt;&#xA;&lt;li&gt;it perfectly preserves all pandas dataframe information, this is  important&lt;/li&gt;&#xA;&lt;li&gt;The 'key' ( object) allows you to store multiple dataframes into the same file, you just provide different key names&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The last point is very useful, because you can pull out a dataframe of choice and the SQL commands are mimicked within pandas&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran into trouble using CSV to store/recover complex pandas dataframe formats. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it's good enough and much faster than SQL. I could have problems with format changes in time (unlike SQL), but for the here and now it works well. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Just to add a caveat, for dictionaries the json dump works well, but it is just shunting dictionaries around, so isn't relevant.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-09-28T22:20:52.947" LastActivityDate="2019-09-28T22:20:52.947" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8746" PostTypeId="2" ParentId="8744" CreationDate="2019-06-03T14:32:22.103" Score="1" Body="&lt;p&gt;My initial reaction is that 21 amino acids is too little information to demonstrate these proteins are homologues (which is the question behind your question). A homologous site means it shares a common ancestry and is distinguishable from random chance or convergent evolution. A common ancestor can be demonstrated using phylogeny (trees).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a regular alignment program cannot align these sequences, such as clustalo or muscle, the assumption they are homologues cannot be easily tested using phylogeny. That would then become complicated at which point you would need to ask whether it is worth pursing the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My recommendation is to perform a simple protein-protein blast and assess the e-scores to get some feel about you data. Your question is achievable, but you would solve this in context to additional sequence data. Whether that sequence data is a denser sample or else contiguous sequence data needs further exploration.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-06-03T14:32:22.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8747" PostTypeId="2" ParentId="8744" CreationDate="2019-06-03T14:32:50.977" Score="1" Body="&lt;p&gt;If you are trying to figure out the proteins' evolutionary history (e.g. if they are homologous) then you want a global alignment. In fact, there are very few cases where a local alignment is better, local is usually done only because it is far more efficient and less computationally expensive than global. But with tiny sequences of just as few residues such as the ones you have, you would want global almost every time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only case I can think of where you might prefer local alignment is when you have sequences that share little overall sequence similarity but you suspect they carry some sort of shared motif. For example, in proteins, that could be a protein domain. In that case, you would want to do a local alignment (possibly using something like blast) to identify those sub-regions of higher similarity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that simple sequence similarity isn't really enough for you to decide whether they are indeed homologous. A more sophisticated approach such as &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&amp;amp;PAGE=Proteins&amp;amp;PROGRAM=blastp&amp;amp;RUN_PSIBLAST=on&quot; rel=&quot;nofollow noreferrer&quot;&gt;psi-blast&lt;/a&gt; or &lt;a href=&quot;http://hmmer.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HMMER&lt;/a&gt; will give better results. Nevertheless, a simple alignment will help you at least discard some obviously non-homologous sequences.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-06-03T14:32:50.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8748" PostTypeId="2" ParentId="8732" CreationDate="2019-06-03T15:01:08.687" Score="3" Body="&lt;p&gt;The &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/intersect.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools intersect&lt;/a&gt; command is my go-to for operations like this. From the manual:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;By far, the most common question asked of two sets of genomic features is whether or not any of the features in the two sets “overlap” with one another. This is known as feature intersection. &lt;code&gt;bedtools intersect&lt;/code&gt; allows one to screen for overlaps between two sets of genomic features. Moreover, it allows one to have fine control as to how the intersections are reported. &lt;code&gt;bedtools intersect&lt;/code&gt; works with both BED/GFF/VCF and BAM files as input.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-06-03T15:01:08.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8749" PostTypeId="1" CreationDate="2019-06-03T20:46:48.753" Score="0" ViewCount="142" Body="&lt;p&gt;Is there any computational tool or any other method to predict if the binding site of DNA binding protein is actually located on a promoter sequence?&lt;/p&gt;&#xA;" OwnerUserId="3968" LastActivityDate="2019-06-04T02:31:35.953" Title="How can I predict if the binding site of a DNA-binding protein is located on the promoter sequence of a gene?" Tags="|bioconductor|genome|proteins|computation|transposable-elements|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8750" PostTypeId="2" ParentId="8749" CreationDate="2019-06-04T02:31:35.953" Score="1" Body="&lt;p&gt;You could run &lt;a href=&quot;http://meme-suite.org/doc/fimo.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;FIMO&lt;/a&gt; on your entire genome for TFs (transcription factors; DNA-binding proteins) of interest, which gives you predicted binding sites: genomic intervals where those TFs bind, with some statistical significance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2491/776&quot;&gt;https://bioinformatics.stackexchange.com/a/2491/776&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then map or intersect promoter regions with those TF binding sites with BEDOPS &lt;code&gt;bedmap&lt;/code&gt; or &lt;code&gt;bedops&lt;/code&gt;, respectively:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/statistics/bedmap.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedops.readthedocs.io/en/latest/content/reference/statistics/bedmap.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/set-operations/bedops.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedops.readthedocs.io/en/latest/content/reference/set-operations/bedops.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can define your promoter regions however you like, e.g., from the TSS to 2.5kb upstream of the TSS. It's up to you and your experiment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The use of &lt;code&gt;bedmap&lt;/code&gt; is recommended to assign TFs to a promoter of interest. This tool will return both the promoter region and any TFs that overlap it by the specified overlap criteria.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2019-06-04T02:31:35.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8753" PostTypeId="1" AcceptedAnswerId="8754" CreationDate="2019-06-04T14:27:35.700" Score="1" ViewCount="469" Body="&lt;p&gt;I have a &lt;code&gt;vcf v4.2&lt;/code&gt; file with structural variants, the 8 columns describing the variant are clear, but in the genotype information I got confused by values of &lt;code&gt;GL&lt;/code&gt; entries. In the &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;vcf specification&lt;/a&gt; they mention that &lt;code&gt;GL&lt;/code&gt; are likelihoods of the individual genotypes, but as far as I am concerned, likelihood should be a number between 0 and 1, therefore I am suprised to see negative numbers. Then I thought that it might be log-likelihood, but then it's strange that the higher the number is the better. Here is a sample of 5 variants with various called genotypes&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;...       GT:GL:GQ:FT:RCL:RC:RCR:CN:DR:DV:RR:RV   0/1:-22.4949,0,-4.49485:45:PASS:20:1:11:0:1:4:0:0&#xA;...       GT:GL:GQ:FT:RCL:RC:RCR:CN:DR:DV:RR:RV   0/1:-3.1888,0,-43.9892:32:PASS:16:26:13:2:9:2:0:0&#xA;...       GT:GL:GQ:FT:RCL:RC:RCR:CN:DR:DV:RR:RV   0/1:-0.190635,0,-49.2888:4:LowQual:26:51:31:2:10:1:0:0&#xA;...       GT:GL:GQ:FT:RCL:RC:RCR:CN:DR:DV:RR:RV   0/0:0,-1.01435,-75.899:10:LowQual:18:54:26:2:14:1:0:0&#xA;...       GT:GL:GQ:FT:RCL:RC:RCR:CN:DR:DV:RR:RV   1/1:-19.198,-1.5031,0:15:PASS:70:0:58:0:0:5:0:0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;vcf&lt;/code&gt; file was generated by &lt;a href=&quot;https://github.com/dellytools/delly&quot; rel=&quot;nofollow noreferrer&quot;&gt;delly&lt;/a&gt; genotyper. I followed their recommendations for germline SV calling. However, negative likelihoods are even in the &lt;code&gt;vcf&lt;/code&gt; specification, therefore it seems that it's not something &lt;code&gt;delly&lt;/code&gt; specific.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, what exactly is &lt;code&gt;GL&lt;/code&gt; in &lt;code&gt;vcf&lt;/code&gt; files? And how do they relate to other entries &lt;code&gt;GP&lt;/code&gt; (genotype posterior probabilities, not in the file by delly) and &lt;code&gt;GQ&lt;/code&gt; (conditional genotype quality)?&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2019-06-04T15:00:35.483" Title="Quality GL entries in vcf file" Tags="|vcf|structural-variation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8754" PostTypeId="2" ParentId="8753" CreationDate="2019-06-04T15:00:35.483" Score="4" Body="&lt;p&gt;The &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VCF spec&lt;/a&gt; clearly states that these are indeed log likelihood values:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;• GL : genotype likelihoods comprised of comma separated floating point log10-scaled likelihoods for all possible&#xA;  genotypes given the set of alleles defined in the REF and ALT fields. In presence of the GT field the same&#xA;  ploidy is expected and the canonical order is used; without GT field, diploidy is assumed. If A is the allele in&#xA;  REF and B,C,... are the alleles as ordered in ALT, the ordering of genotypes for the likelihoods is given by:&#xA;  F(j/k) = (k*(k+1)/2)+j. In other words, for biallelic sites the ordering is: AA,AB,BB; for triallelic sites the&#xA;  ordering is: AA,AB,BB,AC,BC,CC, etc. For example: GT:GL 0/1:-323.03,-99.29,-802.53 (Floats)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;With log-likelihood values, the higher the number, the better the value. So -2 is better than -3. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-06-04T15:00:35.483" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8755" PostTypeId="1" CreationDate="2019-06-04T17:34:37.057" Score="0" ViewCount="61" Body="&lt;p&gt;I am looking for an induction into RNA-seq analysis using R. I have yet to decide a data set, or a  subject area. Any help in this regard would be welcome. &lt;/p&gt;&#xA;" OwnerUserId="4683" LastEditorUserId="3848" LastEditDate="2019-06-04T21:23:34.477" LastActivityDate="2019-06-04T21:23:34.477" Title="how can i download rna-seq data sample for starting rna-seq analysis by R?" Tags="|r|rna-seq|" AnswerCount="1" CommentCount="3" ClosedDate="2019-06-05T10:11:33.627" ContentLicense="CC BY-SA 4.0" />
  <row Id="8756" PostTypeId="2" ParentId="8755" CreationDate="2019-06-04T18:32:31.400" Score="2" Body="&lt;p&gt;You must have found the airway dataset by googling &quot;RNASeq R tutorial&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.bioconductor.org/packages/devel/workflows/vignettes/rnaseqGene/inst/doc/rnaseqGene.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioconductor.org/packages/devel/workflows/vignettes/rnaseqGene/inst/doc/rnaseqGene.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why won't this work for you?&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-06-04T18:32:31.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8757" PostTypeId="2" ParentId="7105" CreationDate="2019-06-05T04:19:33.577" Score="7" Body="&lt;p&gt;No. The only point the specification makes is that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;It is permitted to have multiple records with the same POS&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I suspect this is &lt;em&gt;intended&lt;/em&gt; to let the file list different variants occurring at the same position, but this incidentally allows duplicate variants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This said, you probably don't &lt;em&gt;want&lt;/em&gt; duplicate variants, because it will mess with any statistics you apply over the variants, and will unnecessarily clutter the file and slow down processing, if this happens a lot. You might also get variant annotations on one variant but not another, which can become a problem depending on how you query your variants.&lt;/p&gt;&#xA;" OwnerUserId="4645" LastActivityDate="2019-06-05T04:19:33.577" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8758" PostTypeId="1" AcceptedAnswerId="8778" CreationDate="2019-06-05T15:22:35.480" Score="1" ViewCount="162" Body="&lt;p&gt;I want to get transcription factor targets for my list of genes (For Human and Mouse)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found ENCODE Transcription Factor Targets database (The link is given below). But I am looking for some other databases also if available. I will really appreciate if anyone can suggest any other database.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://amp.pharm.mssm.edu/Harmonizome/dataset/ENCODE+Transcription+Factor+Targets&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://amp.pharm.mssm.edu/Harmonizome/dataset/ENCODE+Transcription+Factor+Targets&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="3766" LastActivityDate="2020-09-04T11:01:39.533" Title="Transcription factor targets for Human and Mouse genes" Tags="|genomics|transcriptome|encode|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8759" PostTypeId="1" CreationDate="2019-06-05T20:12:38.210" Score="1" ViewCount="153" Body="&lt;p&gt;The &lt;a href=&quot;http://www.repeatmasker.org/cgi-bin/WEBRepeatMasker&quot; rel=&quot;nofollow noreferrer&quot;&gt;repeat masker site&lt;/a&gt; has a size limit of 90kb. &#xA;Where can I mask repeats in a 500kb sequence?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, I need a thorough repeat masking of known and de-novo found repeats of various complexity. I am working on Windows but can access Unix too.&lt;/p&gt;&#xA;" OwnerUserId="4617" LastActivityDate="2022-05-26T02:24:34.043" Title="Looking for a good tool for repeat masking in long sequences" Tags="|repeat-elements|repeatmasker|repeat|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8760" PostTypeId="2" ParentId="8759" CreationDate="2019-06-06T02:03:21.207" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://www.cbcb.umd.edu/software/RepeatFinder/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RepeatFinder&lt;/a&gt; may suffice by Volfovsky et al. (2001) and is available for download.&lt;/p&gt;&#xA;" OwnerUserId="4686" LastEditorUserId="3848" LastEditDate="2019-06-06T06:59:11.733" LastActivityDate="2019-06-06T06:59:11.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8761" PostTypeId="2" ParentId="4297" CreationDate="2019-06-06T08:08:33.160" Score="3" Body="&lt;p&gt;Have a look into &lt;a href=&quot;https://cran.r-project.org/web/packages/clustree/vignettes/clustree.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;clustree&lt;/a&gt; to assess the different clusters by clustering them and see different levels.&lt;/p&gt;&#xA;&lt;h3&gt;Example:&lt;/h3&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(clustree)&#xA;data(&amp;quot;iris_clusts&amp;quot;)&#xA;&#xA;# plot the tree of clusters&#xA;# K1 to K5 correspond to different clustering resolutions&#xA;clustree(iris_clusts, prefix = &amp;quot;K&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3&gt;Using the stability index to assess clusters&lt;/h3&gt;&#xA;&lt;p&gt;The stability index from the SC3 package (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5410170/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kiselev et al. 2017&lt;/a&gt;) measures the stability of clusters across resolutions. The stability index is automatically calculated when a clustering tree is built. Note that each level of clustering corresponds to a different resolution.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;clustree(iris_clusts, prefix = &amp;quot;K&amp;quot;, node_colour = &amp;quot;sc3_stability&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DKT34.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DKT34.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4690" LastEditorUserId="3355" LastEditDate="2022-01-06T07:10:32.537" LastActivityDate="2022-01-06T07:10:32.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8763" PostTypeId="1" CreationDate="2019-06-06T11:36:57.640" Score="1" ViewCount="305" Body="&lt;p&gt;I used &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if (!requireNamespace(&quot;BiocManager&quot;, quietly = TRUE))&#xA;    install.packages(&quot;BiocManager&quot;)&#xA;BiocManager::install()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;for installing &quot;bioconductor&quot; but I encountered this error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error: invalid version specification ‘Bioconductor version cannot be validated; no internet connection?’&#xA;In addition: Warning messages:&#xA;1: In file(con, &quot;r&quot;) : InternetOpenUrl failed: 'à¿q'&#xA;2: In file(con, &quot;r&quot;) : InternetOpenUrl failed: 'pen the ð¥Xn'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can anybody help me?&lt;/p&gt;&#xA;" OwnerUserId="4683" LastEditorUserId="298" LastEditDate="2019-06-06T12:12:01.560" LastActivityDate="2019-06-06T12:12:01.560" Title="install bioconductor" Tags="|bioconductor|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8764" PostTypeId="1" CreationDate="2019-06-06T11:42:15.663" Score="4" ViewCount="1473" Body="&lt;p&gt;I’m trying to run a Nextflow workflow in a custom Docker container. Without Docker, the workflow succeeds. But running it inside the container leads to an error because a dependent file cannot be found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here’s a minimal script to illustrate the issue (similar to the &lt;a href=&quot;https://www.nextflow.io/docs/latest/basic.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;basic Nextflow script example&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;params.reference = &quot;/data/reference/GRCh38.fa&quot;&#xA;&#xA;reference = file(params.reference)&#xA;fastq_channel = Channel.fromFilePairs('/data/fastq/*_R{1,2}.fastq.gz')&#xA;&#xA;process map_reads {&#xA;    input:&#xA;    set sample, '?.fastq.gz' from fastq_channel&#xA;&#xA;    output:&#xA;    file(&quot;${sample}.sam&quot;) into sam_channel&#xA;&#xA;    script:&#xA;    &quot;&quot;&quot;&#xA;    echo 'bwa mem -Y &quot;&lt;span class=&quot;math-container&quot;&gt;$reference&quot; 1.fastq.gz 2.fastq.gz &amp;gt;$&lt;/span&gt;{sample}.sam'&#xA;    stat -t &quot;${reference}.bwt&quot;&#xA;    &quot;&quot;&quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Running this inside a container will cause the path of the query, &lt;code&gt;/some/fastq/&lt;/code&gt;, to be mounted inside the container. However, the path to the database, &lt;code&gt;/data/reference&lt;/code&gt;, will &lt;em&gt;not&lt;/em&gt; be mounted. Consequently, I get the following execution error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;stat: cannot stat 'GRCh38-chr21.fa.bwt': No such file or directory&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Without the container, the command succeeds.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I understand &lt;em&gt;why&lt;/em&gt; the error occurs: Nextflow doesn’t track the dependency on &lt;code&gt;$reference&lt;/code&gt; inside the command, and thus doesn’t mount its path in the container.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is, &lt;strong&gt;how do I fix this?&lt;/strong&gt; The documentation is silent on this regard, and none of the workflows I can find online address the problem. For example, the &lt;a href=&quot;https://github.com/IARCbioinfo&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATK4 IARC workflows&lt;/a&gt; would presumably have the same issue, and aren’t configured for use with Docker.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just adding the &lt;code&gt;reference&lt;/code&gt; as a process input is insufficient, since BWA requires various other index files. Am I supposed to add these all manually as inputs?&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2019-06-07T10:08:32.027" Title="Run Nextflow with file dependencies inside a Docker container?" Tags="|docker|nextflow|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8765" PostTypeId="2" ParentId="8764" CreationDate="2019-06-06T12:38:55.310" Score="3" Body="&lt;p&gt;The following workaround seems to work but ideally I don’t want to have to change the &lt;code&gt;script&lt;/code&gt; portion of my process so that it works with Docker.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;params.ref = 'data/reference/GRCh38.fa'&#xA;&#xA;ref = file(params.ref)&#xA;ref_index_dir = ref.parent&#xA;fastq_channel = Channel.fromFilePairs('data/fastq/*_R{1,2}.fastq.gz')&#xA;&#xA;process map_reads {&#xA;    input:&#xA;    set sample, '?.fastq.gz' from fastq_channel&#xA;    file ref&#xA;    file ref_index_dir&#xA;&#xA;    output:&#xA;    file(&quot;${sample}.sam&quot;) into sam_channel&#xA;&#xA;    script:&#xA;    &quot;&quot;&quot;&#xA;    echo 'bwa mem -Y &quot;reference/&lt;span class=&quot;math-container&quot;&gt;$ref&quot; 1.fastq.gz 2.fastq.gz &amp;gt;$&lt;/span&gt;{sample}.sam'&#xA;    stat -t reference/&lt;span class=&quot;math-container&quot;&gt;${ref}.bwt &amp;amp;&amp;amp; touch $&lt;/span&gt;{sample}.sam&#xA;    &quot;&quot;&quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here, &lt;code&gt;reference&lt;/code&gt; inside the script section refers to the parent directory name of the reference, and needs to be adjusted accordingly. Having to refer to &lt;code&gt;reference/$ref&lt;/code&gt; is annoying but at least it works — both inside a Docker container and outside of it.&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2019-06-06T12:38:55.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8766" PostTypeId="2" ParentId="8759" CreationDate="2019-06-06T13:48:11.893" Score="4" Body="&lt;p&gt;&lt;a href=&quot;http://www.repeatmasker.org/RMDownload.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;RepeatMasker&lt;/a&gt; can downloaded an run locally. There's then no size limit, though do note that it's quite slow for mammalian-sized sequences (you'll want to split it into chunks and run in parallel in such cases). For a small 500kb sequence that won't pose an issue.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-06-06T13:48:11.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8767" PostTypeId="1" CreationDate="2019-06-06T16:20:02.193" Score="1" ViewCount="3158" Body="&lt;p&gt;bcl2fastq is a program that converts Illumina run output from .bcl to fastq. I'm having difficulties installing the software I downloaded from Illumina on my Ubuntu 18.04. I've followed their installation manual, but the software is not building when I enter the command &quot;make&quot;.  I receive the error message... &quot;make: *** No targets specified and no makefile found. Stop. Has anyone had similar experience? Is there another software that will install easier on Ubuntu?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Software:&#xA;&lt;a href=&quot;https://support.illumina.com/downloads/bcl2fastq-conversion-software-v2-20.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.illumina.com/downloads/bcl2fastq-conversion-software-v2-20.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Manual:&#xA;&lt;a href=&quot;https://support.illumina.com/content/dam/illumina-support/documents/documentation/software_documentation/bcl2fastq/bcl2fastq2-v2-20-software-guide-15051736-03.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.illumina.com/content/dam/illumina-support/documents/documentation/software_documentation/bcl2fastq/bcl2fastq2-v2-20-software-guide-15051736-03.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Steps followed:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;export TMP=/tmp&#xA;export SOURCE=&lt;span class=&quot;math-container&quot;&gt;${TMP}/bcl2fastq&#xA;export BUILD=$&lt;/span&gt;{TMP}/bcl2fastq2-v2.20.0-build&#xA;export INSTALL_DIR=/usr/local/bcl2fastq2-v2.20.0&#xA;&#xA;cd ${TMP}&#xA;tar -xvzf bcl2fastq2-v2.20.0.499-Source.tar.gz&#xA;&#xA;mkdir &lt;span class=&quot;math-container&quot;&gt;${BUILD}&#xA;cd $&lt;/span&gt;{BUILD}&#xA;chmod ugo+x &lt;span class=&quot;math-container&quot;&gt;${SOURCE}/src/configure&#xA;chmod ugo+x $&lt;/span&gt;{SOURCE}/src/cmake/bootstrap/installCmake.sh&#xA;&lt;span class=&quot;math-container&quot;&gt;${SOURCE}/src/configure --prefix=$&lt;/span&gt;{INSTALL_DIR}&#xA;&#xA;cd ${BUILD}&#xA;make&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The final command is supposed to be &lt;code&gt;make install&lt;/code&gt;, but the &lt;code&gt;configure&lt;/code&gt; step fails with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[ . . . ]&#xA;-- Configuring incomplete, errors occurred! &#xA;Couldn't configure the project: &#xA;/tmp/bcl2fastq2-v2.20.0-build/bootstrap_cmake/bin/cmake -H'/tmp/bcl2fastq/src' -B'/tmp/bcl2fastq2-v2.20.0-build' -G'Unix Makefiles' -DBCL2FASTQ_NAME_SHORT:STRING=&quot;bcl2fastq&quot; -DBCL2FASTQ_NAME_LONG:STRING=&quot;BCL to FASTQ file converter&quot; -DBCL2FASTQ_COPYRIGHT:STRING=&quot;Copyright (c) 2007-2017 Illumina, Inc.&quot; -DBCL2FASTQ_VERSION_MAJOR:STRING=&quot;2&quot; -DBCL2FASTQ_VERSION_MINOR:STRING=&quot;20&quot; -DBCL2FASTQ_VERSION_PATCH:STRING=&quot;0&quot; -DBCL2FASTQ_VERSION_BUILD:STRING=&quot;422&quot; -DBCL2FASTQ_VERSION:STRING=&quot;2.20.0.422&quot; -DBCL2FASTQ_SOURCE_DIR:STRING=&quot;/tmp/bcl2fastq/src&quot; -DBCL2FASTQ_PREFIX:PATH=&quot;/usr/local/bcl2fastq2-v2.20.0&quot; -DBCL2FASTQ_EXEC_PREFIX:PATH=&quot;&quot; -DCMAKE_INSTALL_PREFIX:PATH=&quot;/usr/local/bcl2fastq2-v2.20.0&quot; -DBCL2FASTQ_BINDIR:PATH=&quot;&quot; -DBCL2FASTQ_LIBDIR:PATH=&quot;&quot; -DBCL2FASTQ_LIBEXECDIR:PATH=&quot;&quot; -DBCL2FASTQ_INCLUDEDIR:PATH=&quot;&quot; -DBCL2FASTQ_DATADIR:PATH=&quot;&quot; -DBCL2FASTQ_DOCDIR:PATH=&quot;&quot; -DBCL2FASTQ_MANDIR:PATH=&quot;&quot; -DCMAKE_BUILD_TYPE:STRING=&quot;RelWithDebInfo&quot; -DCMAKE_PARALLEL:STRING=&quot;1&quot; &#xA;Moving CMakeCache.txt to CMakeCache.txt.removed &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2621" LastEditorUserId="298" LastEditDate="2019-06-06T19:14:09.267" LastActivityDate="2020-06-25T13:32:19.867" Title="bcl2fastq2 v2.20.0 converstion software on Ubuntu 18.04" Tags="|software-installation|" AnswerCount="2" CommentCount="13" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8768" PostTypeId="2" ParentId="8764" CreationDate="2019-06-07T10:08:32.027" Score="1" Body="&lt;p&gt;Process input files need to be declared properly to allow nextflow to create the required docker mounts accordingly. Therefore the &lt;code&gt;reference&lt;/code&gt; file should be declared in the process input block, eh: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;input:&#xA; set sample, file(fastq) from fastq_channel&#xA; file reference &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="662" LastActivityDate="2019-06-07T10:08:32.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8770" PostTypeId="1" AcceptedAnswerId="16225" CreationDate="2019-06-07T14:29:42.273" Score="3" ViewCount="50" Body="&lt;ol&gt;&#xA;&lt;li&gt;I used &lt;code&gt;samtools depth&lt;/code&gt; to find the per base-pair read coverage over a number of isoform contigs from my &lt;a href=&quot;https://github.com/trinityrnaseq/trinityrnaseq/wiki&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trinity assembly&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;I have also conducted a multiple sequence alignment of those isoforms using Clustal Omega DNA aligner.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In my final step, I want to assess the per-residue read coverage of the consensus amino acid sequence to demonstrate an idea of confidence in that sequence.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have any tips on how to go about doing this and whether there are existing tools for such things?&lt;/p&gt;&#xA;" OwnerUserId="2727" LastEditorUserId="3848" LastEditDate="2020-07-24T04:14:35.490" LastActivityDate="2021-06-25T11:17:26.477" Title="RNASeq read coverage in protein space?" Tags="|sequence-alignment|read-mapping|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8771" PostTypeId="1" CreationDate="2019-06-07T16:51:45.033" Score="3" ViewCount="101" Body="&lt;p&gt;What kind of tool would be appropriate do detect &quot;broad peaks&quot; in small RNA-seq sequencing data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;MACS2 appears to be  developed for ChIP-seq data, but I see that there is a &lt;code&gt;--nomodel&lt;/code&gt; option. Would that make this program usable in my case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I suppose I should also set the &lt;code&gt;--shift&lt;/code&gt; and &lt;code&gt;--extsize&lt;/code&gt; options, but I'm not sure how:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  --nomodel             Whether or not to build the shifting model. If True,&#xA;                        MACS will not build model. by default it means&#xA;                        shifting size = 100, try to set extsize to change it.&#xA;                        DEFAULT: False&#xA;  --shift SHIFT         (NOT the legacy --shiftsize option!) The arbitrary&#xA;                        shift in bp. Use discretion while setting it other&#xA;                        than default value. When NOMODEL is set, MACS will use&#xA;                        this value to move cutting ends (5') towards 5'-&amp;gt;3'&#xA;                        direction then apply EXTSIZE to extend them to&#xA;                        fragments. When this value is negative, ends will be&#xA;                        moved toward 3'-&amp;gt;5' direction. Recommended to keep it&#xA;                        as default 0 for ChIP-Seq datasets, or -1 * half of&#xA;                        EXTSIZE together with EXTSIZE option for detecting&#xA;                        enriched cutting loci such as certain DNAseI-Seq&#xA;                        datasets. Note, you can't set values other than 0 if&#xA;                        format is BAMPE for paired-end data. DEFAULT: 0.&#xA;  --extsize EXTSIZE     The arbitrary extension size in bp. When nomodel is&#xA;                        true, MACS will use this value as fragment size to&#xA;                        extend each read towards 3' end, then pile them up.&#xA;                        It's exactly twice the number of obsolete SHIFTSIZE.&#xA;                        In previous language, each read is moved 5'-&amp;gt;3'&#xA;                        direction to middle of fragment by 1/2 d, then&#xA;                        extended to both direction with 1/2 d. This is&#xA;                        equivalent to say each read is extended towards 5'-&amp;gt;3'&#xA;                        into a d size fragment. DEFAULT: 200. EXTSIZE and&#xA;                        SHIFT can be combined when necessary. Check SHIFT&#xA;                        option&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In my case the tags are the same things as the fragments, so this &quot;extension&quot; thing is probably not appropriate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, I would like to call peaks based on the plain real coverage of my small reads. Are there other tools that would be more appropriate?&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2019-06-09T14:01:30.563" Title="Detecting broad peaks in sRNA-seq data" Tags="|software-recommendation|hts|peak-calling|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8772" PostTypeId="1" CreationDate="2019-06-07T18:50:30.660" Score="2" ViewCount="211" Body="&lt;p&gt;I am a novice to bioinformatics trying to run a cuffdiff job in cufflinks and keep getting this error message: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error: sort order of reads in BAMs must be the same&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;GENERATING SAM FILES:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cd /fs/project/PMIU0157/Claire_Stoll/RNASeq/B73_hisat2/B73_hisat2_11-5&#xA;module load hisat2  &#xA;hisat2 -k 1 -p 8 -S 11-5_B73_accepted_hits.sam -x /fs/project/PMIU0157/Claire_Stoll/RNASeq/B73_hisat2/B73_Zea_mays.AGPv4.dna.toplevel -1 /fs/project/PMIU0157/Claire_Stoll/RNASeq/11-5_FCHHL7JBBXX_L7_DLPRMzP300H3AAAPEI-27_1.fq  -2 /fs/project/PMIU0157/Claire_Stoll/RNASeq/11-5_FCHHL7JBBXX_L7_DLPRMzP300H3AAAPEI-27_2.fq&#xA;&#xA;&#xA;cd /fs/project/PMIU0157/Claire_Stoll/RNASeq/Mo17_hisat2/Mo17_hisat2_11R-5&#xA;module load hisat2  &#xA;hisat2 -k 1 -p 8 -S 11-5_Mo17_accepted_hits.sam -x /fs/project/PMIU0157/Claire_Stoll/RNASeq/Mo17_hisat2//Mol17_Ref_Lai -1 /fs/project/PMIU0157/Claire_Stoll/RNASeq/11R-5_DLPRMzP300B4AAAPEI-2_1.fq  -2 /fs/project/PMIU0157/Claire_Stoll/RNASeq/11R-5_DLPRMzP300B4AAAPEI-2_2.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;CONVERTING SAM TO BAM:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cd /fs/project/PMIU0157/Claire_Stoll/RNASeq/B73_hisat2/B73_hisat2_11-5&#xA;module load samtools &#xA;samtools view -bS a_B73_11-5_accepted_hits.sam &amp;gt; a_B73_11-5_accepted_hits.bam&#xA;samtools sort a_B73_11-5_accepted_hits.bam -o a_B73_11-5_accepted_hits.sorted.bam&#xA;&#xA;cd /fs/project/PMIU0157/Claire_Stoll/RNASeq/Mo17_hisat2/Mo17_hisat2_11-5&#xA;module load samtools &#xA;samtools view -bS a_Mo17_11-5_accepted_hits.sam &amp;gt; a_Mo17_11-5_accepted_hits.bam&#xA;samtools sort a_Mo17_11-5_accepted_hits.bam -o a_Mo17_11-5_accepted_hits.sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;CUFFDIFF JOB&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cd /fs/project/PMIU0157/Claire_Stoll/RNASeq/B73_hisat2/B73_hisat2_11-5&#xA;module load cufflinks/2.2.1&#xA;cuffdiff -b /fs/project/PMIU0157/Claire_Stoll/RNASeq/B73_hisat2/B73_hisat2_11-5 -u -N -p 8 /fs/project/PMIU0157/Claire_Stoll/Zea_mays.AGPv4.36.chr.gff3 /fs/project/PMIU0157/Claire_Stoll/RNASeq/B73_hisat2/B73_hisat2_11-5/a_B73_11-5_accepted_hits.sorted.bam /fs/project/PMIU0157/Claire_Stoll/RNASeq/Mo17_hisat2/Mo17_hisat2_11-5/a_Mo17_11-5_accepted_hits.sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4699" LastEditorUserId="298" LastEditDate="2019-06-07T19:04:14.683" LastActivityDate="2019-06-08T08:19:07.777" Title="Error: sort order of reads in BAMs must be the same" Tags="|sequence-alignment|sam|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8773" PostTypeId="2" ParentId="8772" CreationDate="2019-06-07T20:39:19.560" Score="3" Body="&lt;p&gt;I think cuffdiff is not quite giving you the right error...it looks like the crux of your problem is you aligned each sample to a different reference.  &lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-06-07T20:39:19.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8774" PostTypeId="1" AcceptedAnswerId="8775" CreationDate="2019-06-08T07:24:59.867" Score="-1" ViewCount="118" Body="&lt;p&gt;These are my files &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SO_8543_IP_3_ablt_R1.fq.gz&#xA;SO_8543_IP_4_ablt_R1.fq.gz&#xA;SO_8543_INPUT_1_ablt_R1.fq.gz  SO_8543_IP_5_ablt_R1.fq.gz&#xA;SO_8543_INPUT_2_ablt_R1.fq.gz  SO_8543_IP_6_ablt_R1.fq.gz&#xA;SO_8543_INPUT_3_ablt_R1.fq.gz  SO_8543_IP_7_ablt_R1.fq.gz&#xA;SO_8543_INPUT_4_ablt_R1.fq.gz  SO_8543_IP_8_ablt_R1.fq.gz&#xA;SO_8543_IP_1_ablt_R1.fq.gz     &#xA;SO_8543_IP_2_ablt_R1.fq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Im running the below line &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in &lt;span class=&quot;math-container&quot;&gt;$(ls *.fq*.gz | cut -f1-5 -d&quot;_&quot; | uniq);do echo $&lt;/span&gt;i; done&#xA;&#xA;bowtie2 -q -p20 -x /run/media/punit/data1/BowtieIndex/hg38  -1 &lt;span class=&quot;math-container&quot;&gt;${i}_R1.fq.gz | samtools view -bS - &amp;gt;$&lt;/span&gt;{i%}.bam;&#xA;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So if i use run this &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in &lt;span class=&quot;math-container&quot;&gt;$(ls *.fq*.gz | cut -f1-5 -d&quot;_&quot; | uniq);do echo $&lt;/span&gt;i; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get files as such &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SO_8543_INPUT_1_ablt&#xA;SO_8543_INPUT_2_ablt&#xA;SO_8543_INPUT_3_ablt&#xA;SO_8543_INPUT_4_ablt&#xA;SO_8543_IP_1_ablt&#xA;SO_8543_IP_2_ablt&#xA;SO_8543_IP_3_ablt&#xA;SO_8543_IP_4_ablt&#xA;SO_8543_IP_5_ablt&#xA;SO_8543_IP_6_ablt&#xA;SO_8543_IP_7_ablt&#xA;SO_8543_IP_8_ablt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The part of the file name that is consistent is &lt;code&gt;_R1.fq.gz&lt;/code&gt; which goes after the input argument. As these are single end files so only 1 im giving one end as input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Error message &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;***&#xA;Error: Must specify at least one read input with -U/-1/-2&#xA;(ERR): bowtie2-align exited with value 1&#xA;./Chip_SEQ.sh: line 25: syntax error near unexpected token `done'&#xA;./Chip_SEQ.sh: line 25: `done'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;it looks like its not able to recognize as input .Any help or suggestion would be highly appreciated &lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2019-06-08T08:13:07.967" Title="Regular expression struggle" Tags="|shell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8775" PostTypeId="2" ParentId="8774" CreationDate="2019-06-08T08:13:07.967" Score="6" Body="&lt;p&gt;You have an extra &lt;code&gt;done&lt;/code&gt;, instead you want:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in &lt;span class=&quot;math-container&quot;&gt;$(ls *.fq*.gz | cut -f1-5 -d&quot;_&quot; | uniq); do&#xA;    echo $&lt;/span&gt;i&#xA;    bowtie2 -q -p20 -x /run/media/punit/data1/BowtieIndex/hg38 -1 &lt;span class=&quot;math-container&quot;&gt;${i}_R1.fq.gz | samtools view -bS - &amp;gt;$&lt;/span&gt;{i}.bam&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As an aside, I'm not a big fan of piping &lt;code&gt;ls&lt;/code&gt; to cut and then uniq. Why not instead:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for f in *.fq.gz; do&#xA;    sampleName=&lt;span class=&quot;math-container&quot;&gt;${f%%_R1.fq.gz}&#xA;    echo $&lt;/span&gt;sampleName&#xA;    bowtie2 -q ...&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That will do the same thing, but (A) it's clearer what's going on and (B) it can be reused if your sample names change. Also, stop writing bash scripts and use snakemake or something like that instead, it scales much better and is less error-prone.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-06-08T08:13:07.967" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="8776" PostTypeId="2" ParentId="8772" CreationDate="2019-06-08T08:19:07.777" Score="2" Body="&lt;p&gt;As @swbarnes2 noted, the error message is because the chromosome names, sizes, and orders differ between the two reference genomes/species you aligned against. Even if they were the same, you're comparing one sample against another between organisms, which is innately not something likely to work well. Further, I wouldn't expect the Zea Mays annotation to be very useful with whatever &lt;code&gt;Ref_Lai&lt;/code&gt; is.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-06-08T08:19:07.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8778" PostTypeId="2" ParentId="8758" CreationDate="2019-06-08T17:28:54.487" Score="2" Body="&lt;p&gt;JASPAR is another useful transcription factor profiling database.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The JASPAR CORE contains a curated, non-redundant set of profiles, derived from published and experimentally defined transcription factor binding sites for eukaryotes. It should be used, when seeking models for specific factors or structural classes, or if experimental evidence is paramount.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://jaspar.genereg.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://jaspar.genereg.net/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3384" LastActivityDate="2019-06-08T17:28:54.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8779" PostTypeId="2" ParentId="8759" CreationDate="2019-06-08T17:45:52.017" Score="2" Body="&lt;p&gt;From: &lt;a href=&quot;http://meme-suite.org/doc/glam2_tut.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://meme-suite.org/doc/glam2_tut.html&lt;/a&gt;, here's a few tools that do various repeat masking tasks.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;These tools mask various kinds of repeats: &lt;a href=&quot;http://www.repeatmasker.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Repeatmasker&lt;/a&gt;, &lt;a href=&quot;http://www.girinst.org/censor/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Censor&lt;/a&gt;, &lt;a href=&quot;ftp://ftp.ncbi.nih.gov/pub/seg/&quot; rel=&quot;nofollow noreferrer&quot;&gt;seg/pseg/nseg&lt;/a&gt;, &lt;a href=&quot;http://meme-suite.org/doc/dust.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DUST&lt;/a&gt;,&#xA;      &lt;a href=&quot;http://meme-suite.org/doc/purge.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;XNU&lt;/a&gt;, &lt;a href=&quot;https://www.liebertpub.com/doi/abs/10.1089/106652703321825955&quot; rel=&quot;nofollow noreferrer&quot;&gt;DSR&lt;/a&gt;, &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/22/24/2980/208627&quot; rel=&quot;nofollow noreferrer&quot;&gt;GBA&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Censor, seg/pseg/nseg, dust, DSR all have locally available applications. DSR, GBA, and CAST are algorithms for repeat masking.&lt;/p&gt;&#xA;" OwnerUserId="3384" LastActivityDate="2019-06-08T17:45:52.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8781" PostTypeId="1" CreationDate="2019-06-09T09:54:10.847" Score="1" ViewCount="110" Body="&lt;p&gt;I'm having difficulty normalizing my data.&lt;/p&gt;&#xA;&lt;p&gt;I was searching for transposable elements in my genome, and after this step, I made counts of reads in some transcripts. I produced something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head(table_tissues_filtered_TE)&#xA;                  Lengths ova testes lobe retina suckers brain1 brain2 skin stage&#xA;Simple_repeat_80      134  58     77   48     69     115     137  131  195     75&#xA;tRNA_1                 59   0     14   12      1      19      12   14   21    104&#xA;Simple_repeat_87       26   1     33   12      3      15      24   21   19    180&#xA;Simple_repeat_114      22   0      0    0      1       0       0    0    2      7&#xA;Simple_repeat_115      30   0      0    0      0       0       0    0    0      1&#xA;Simple_repeat_123      22   2      3   317     45      13    652  651   15     21&#xA;                              axial                    gland viscera&#xA;Simple_repeat_80                 99                       35     557&#xA;tRNA_1                            9                        0       3&#xA;Simple_repeat_87                  9                        0       4&#xA;Simple_repeat_114                 0                      204       0&#xA;Simple_repeat_115                 0                       42       0&#xA;Simple_repeat_123               333                        5       4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where Lengths are the Length of each elements (simple repeats, etc), and the other columns indicate the reads counted with FeatureCounts.&#xA;I have another table with the number of reads for each tissue:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head(reads_table)&#xA;       ova   testes lobe   retina  suckers   brain1  brain2  skin   stage   axial&#xA;      522444 310243 226146  102307  126055   489389  668243  372728 262536  233754&#xA;  gland  viscera&#xA;  24817   25689&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to do an RPkM analysis to normalize the data using R, but I don't know exactly how to do it. Can anyone can help me? thank you!!!&lt;/p&gt;&#xA;" OwnerUserId="4475" LastEditorUserId="73" LastEditDate="2022-06-21T10:08:51.023" LastActivityDate="2022-06-21T10:08:51.023" Title="Normalization of data with RPkM" Tags="|r|rna-seq|normalization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8782" PostTypeId="2" ParentId="8771" CreationDate="2019-06-09T14:01:30.563" Score="3" Body="&lt;p&gt;The sure why any software should be confined to only broad peaks, but there is two packages that should do what I thing you have in mind. Just to clarify, for smallRNAs peak detection is sometimes also referred to as loci or cluster detection:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://sites.psu.edu/axtell/software/shortstack/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ShortStack&lt;/a&gt; which is developed for alignment, annotation, and quantification of small RNAs. The &lt;code&gt;de-novo cluster finding&lt;/code&gt; option as copied from the manual might do the trick:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DE-NOVO CLUSTER FINDING&#xA;    Unless options --locus or --locifile are used (see below), ShortStack&#xA;    will de-novo identify clusters of small RNA accumulation genome-wide.&#xA;    Cluster definition is simple: First, all regions containing at least one&#xA;    primary alignment are found where the maximum distance between the ends&#xA;    of the alignments is &amp;lt;= option --pad (default: 75). Second, if the&#xA;    number of alignments in the cluster is &amp;gt;= option --mincov (default:&#xA;    0.5rpm), the cluster is kept. The mincov threshold can also be specified&#xA;    in terms of reads per million by using a value such as 3.2rpm (which&#xA;    specifies the threshold to be 3.2 reads per million). Using a rpm&#xA;    threshold allows the sensitivity of cluster discovery to be normalized&#xA;    between libraries of different sizes. Alternatively, reads per million&#xA;    mapped (rpmm) can be specified: A --mincov of 1.2rpmm indicates 1.2&#xA;    reads per million mapped is the threshold. rpm is a fraction of total&#xA;    library size, while rpmm is a fraction of only the aligned &amp;amp; placed&#xA;    fraction of the library.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/segmentSeq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;segmentSeq&lt;/a&gt;&#xA;is an R-package useing a Bayesian approach for small RNA locus detection. It's however really slow and not developed any further by the main author.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1461" LastActivityDate="2019-06-09T14:01:30.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8783" PostTypeId="1" CreationDate="2019-06-09T19:01:44.213" Score="1" ViewCount="143" Body="&lt;p&gt;I have a data frame with gene-expression ratio. Is it possible to input this into limma/voom to find signinficannt gene-ratios between groups of samples? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;my data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                                   S1         S2         S3         S4         S5&#xA;hsa-let-7a-5p_hsa-let-7b-3p 80.066116 197.808511 275.322581 220.421053 242.100000&#xA;hsa-let-7a-5p_hsa-let-7b-5p  2.083889   2.393050   2.337716   1.062944   1.704930&#xA;hsa-let-7a-5p_hsa-let-7c-5p 83.517241  57.388889  85.350000  40.660194  71.205882&#xA;hsa-let-7a-5p_hsa-let-7d-3p  4.199393   5.792523   4.569058   3.193290   6.022388&#xA;hsa-let-7a-5p_hsa-let-7d-5p 32.729730  17.027473  14.490662   9.507378  22.416667&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1595" LastActivityDate="2019-06-09T19:01:44.213" Title="Gene ratio as imput in limma" Tags="|differential-expression|edger|limma|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8784" PostTypeId="1" AcceptedAnswerId="8785" CreationDate="2019-06-09T22:40:50.593" Score="1" ViewCount="21" Body="&lt;p&gt;I'm using snpEff to annotate some VCFs and getting some spurious annotations that don't have transcript ids. Some of these annotations have a gene name followed by ampersand such as PCDHB@ and I'm not finding any explanation in the docs.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any idea what this refers to? Entire gene family? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.  &lt;/p&gt;&#xA;" OwnerUserId="460" LastActivityDate="2019-06-09T22:55:31.527" Title="SnpEff gene annotation with @ such as PCDHB@" Tags="|vcf|sequence-annotation|variant-calling|functional-annotation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8785" PostTypeId="2" ParentId="8784" CreationDate="2019-06-09T22:55:31.527" Score="1" Body="&lt;p&gt;The &lt;code&gt;@&lt;/code&gt; at the end of a name denotes a &quot;gene cluster in chromosome region&quot;. This is mentioned in the official &lt;a href=&quot;https://www.genenames.org/about/guidelines/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HUGO nomenclature guidelines&lt;/a&gt; in the &quot;Characters reserved for specific usage&quot; section:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ywv8s.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ywv8s.png&quot; alt=&quot;[HUGO reserved characters table&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's also mentioned in the &lt;a href=&quot;https://www.genecards.org/Guide/GeneCard&quot; rel=&quot;nofollow noreferrer&quot;&gt;GeneCards documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8ltki.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8ltki.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-06-09T22:55:31.527" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8786" PostTypeId="1" AcceptedAnswerId="8787" CreationDate="2019-06-10T04:39:30.933" Score="1" ViewCount="119" Body="&lt;p&gt;I have a dataset composed on numerous RefSeq transcripts with names of the form NM_###### - essentially standard annotated mRNA accession numbers.  I notice that numerous transcripts which have different sequences (different start and stop sites in a gene) all have the same NM_#### accession number.  The interpretation is that they are all from the same gene, but this makes no sense to me.  Why should different transcripts with different sequences have the same name?  I understand this identifies their gene of origin, but they are still different transcripts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example from the .bed file I'm using.  Note they have different start and stop locations and even have different strand notations, but all have the same name.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr     start        stop       name            score   strand&#xA;chrX    3441730     3443690     NM_001085545    0       -   &#xA;chrX    3700232     3702192     NM_001085545    0       +   &#xA;chrX    3750925     3752885     NM_001085545    0       -   &#xA;chrX    31336368    31338328    NM_001085545    0       +   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4491" LastEditorUserId="4491" LastEditDate="2019-06-10T13:31:55.560" LastActivityDate="2019-06-10T19:33:27.543" Title="RefSeq transcript naming - transcript vs gene" Tags="|refseq|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="8787" PostTypeId="2" ParentId="8786" CreationDate="2019-06-10T19:33:27.543" Score="4" Body="&lt;p&gt;No, this is the same sequence. A single RefSeq accession will correspond to one sequence. However, nobody said that sequence needs to be unique in the genome. The specific sequence you are talking about, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NM_001085545.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;NM_001085545.1&lt;/a&gt;, is a mouse transcript which can be aligned to many different regions of the mouse's chromosome X. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is your sequence:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;NM_001085545.1&#xA;agagaagtctaggcaggactggcagactacagcaatgcctcaaccaagaagcctctgcga&#xA;gagcttgttctaggaattttcatcaatggggcttttagtcagcagggtcttgagatgcag&#xA;ggattccagtctgttagagccacagccagaagccatagccggagccagctacattcctgg&#xA;cagtcgcaagcgaaaaagaaacagtttggaggagttggcaacaagttctaatgttcatgg&#xA;acctcaaaaccagggaatgtatccacatcaagttctcaactacatctactggaaaagggt&#xA;taagatctcatctaatgatgcttatcaaaacttatttttggacgggcatgatagcgacat&#xA;taaaatccgtgctctgggaaaaacatggtgtttacacaaagtatttttatgtcagtcagg&#xA;ctactttgctaacattctcaaaggtacttggagagaatcacaccatggtgttataaatct&#xA;gatcattaagaatgaggatattgatacccgatctctgcattttgtgtttggtgctttgta&#xA;cacggatgcggatttgtcaataacacctctggaagttcctcaagttttggcagcagcatg&#xA;cctgcttcgggtggatcaagtaattcagcagtgtgaaggaatcatgaaagaaactatcaa&#xA;caggaacactgtgtgctcctattatttggcagcagaaacctatagattaaaagctgtaaa&#xA;gacgagatgctttgaatggcttctttgcaatttgatggtacatccaagtgtggcacttta&#xA;caaggaagtagacttgaagttgatgtatcttctagcactgtcttctgacttactagtcat&#xA;gcaaaaggagattgatgtatataccacactaaaaatatggatgttcctttatcttaatcc&#xA;atgctggaacggaaccatgaaacagcttttacaacacgcaaacaactggctttccaccca&#xA;catggcatatgttgataacatcagttttcttgaaagtgaagaaggactaatatttcaacc&#xA;agtgtttaaaaagctgagatttcagcacattatctgtgacttgacttccacaactattct&#xA;tgaacaagatcgactaatacctatggcatggttgtcacccatttacaaacaacagtggtt&#xA;gactttgctgcgaacacaagaatatggggtaattggaccacaagttatcaatgaacaaga&#xA;acttgaagaatgcaccatgaggtgtggtacaatgatccccaaggatggaagatatacttg&#xA;gaagtggtcagttggacgacttggctttcctttacgtgtgacctttaccaggcagtgtgt&#xA;aattttaaggcaacggtgtcagaggtgtgatggttctgcttgccacaaccatatccgaaa&#xA;tgtcatattcagaataactttggtgtgttttgattccaacaaaagagtaactttcagaaa&#xA;gacaacaggttataaaatcctcacctttgaatataaggaggagcaaattgtaatgaaatt&#xA;ggatagtgatgttctaaccttccctatgtgtatattctgcaatttcctttttgtaaacct&#xA;aggaaatgcagaaaacaagtaatcttatcaattattagactgtaagtattttacaagtat&#xA;tgaatggttcatttaatatgaaggctgtgtaatgactacagtgaagaagaccaaaaacaa&#xA;attcaatattctttggagtctagtaccaccagaagtgactgcttccctccatcaccttat&#xA;ttccatgtatttctgaagaaaacctttacttccaacaaagtatttgtcaaatcaaataaa&#xA;cctaatcttctgatttaaatttaccatcgagatccaagcttggaatggaaataagagact&#xA;atatttgcaaggtaaaacatctcaatcaatttgttaagtagatttggccctagctaaaat&#xA;tgagacaaaataaaatcatgccattgttgaaactgttgtg&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you BLAT it against the mouse genome using the &lt;a href=&quot;http://genome.ucsc.edu/cgi-bin/hgBlat&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC BLAT tool&lt;/a&gt;, you will see many hits, most of them almost perfect matches and some of them overlapping the regions given in your bed file:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ywBxj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ywBxj.png&quot; alt=&quot;blat results&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your bed file matches the following hits in the image above:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NM_001085545.1  1954     1  1960  1960    99.9%  chrX   -     3441731   3443690   1960&#xA;NM_001085545.1  1960     1  1960  1960   100.0%  chrX   +     3700233   3702192   1960&#xA;NM_001085545.1  1954     1  1960  1960    99.9%  chrX   -     3750926   3752885   1960&#xA;NM_001085545.1  1954     1  1960  1960    99.9%  chrX   +    31336369  31338328   1960&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't know why your bed file has all of these various matches for NM_001085545.1, since you haven't told us what the bed file is supposed to be doing. A bed file is just a collection of regions. For all I know, that bed file is just &quot;a few hits for sequence NM_001085545.1 in the genome&quot;. There is no reason to assume that each region in a bed file corresponds to a gene. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So there's nothing wrong with your file, you're just dealing with a non-unique sequence from the mouse genome. And you don't have multiple sequences with the same accession, you have one sequence, one accession, but multiple hits for that sequence in the genome.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-06-10T19:33:27.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8788" PostTypeId="2" ParentId="7005" CreationDate="2019-06-11T15:16:25.480" Score="1" Body="&lt;p&gt;Thank you for asking this question. &quot;Flag&quot; is actually an obsolete notation which predates the introduction of proper &lt;a href=&quot;https://www.uniprot.org/help/evidences&quot; rel=&quot;nofollow noreferrer&quot;&gt;evidence attribution&lt;/a&gt; in UniProtKB, and which escaped us in our efforts to update documentation to use the correct terminology. The correct  term would now be &quot;Evidence&quot;, and the next version of the user manual will have this corrected (release 2019_06 due on July 3rd, 2019).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before the current way of evidencing annotations was introduced in UniProtKB, the following &quot;non-experimental qualifiers&quot; or &quot;flags&quot; were used: By similarity, Potential, Probable. Absence of such a flag would generally imply that the information is based on an experiment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When in doubt, please don't hesitate to contact the UniProt helpdesk. &lt;/p&gt;&#xA;" OwnerUserId="1459" LastActivityDate="2019-06-11T15:16:25.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8789" PostTypeId="2" ParentId="5421" CreationDate="2019-06-11T15:23:31.580" Score="2" Body="&lt;p&gt;This answer is very late but may still help others:&#xA;Have a look at the UniProt index of human polymorphisms and disease mutations at &lt;a href=&quot;https://www.uniprot.org/docs/humsavar&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/docs/humsavar&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please don't hesitate to contact the UniProt helpdesk if you have any additional questions.&lt;/p&gt;&#xA;" OwnerUserId="1459" LastActivityDate="2019-06-11T15:23:31.580" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8791" PostTypeId="1" CreationDate="2019-06-12T07:36:01.380" Score="3" ViewCount="295" Body="&lt;p&gt;I am trying to write python script for customized filtering for fastq file (size &gt;3 GB). My proposed script is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def filtering(read):&#xA;    time.sleep(0.1)&#xA;&#xA;    if len(read) &amp;gt;= 15 and \&#xA;       len(read) &amp;lt;= 30 and \&#xA;       np.mean(read.qual) &amp;gt;= 30 and \&#xA;       sum(i &amp;lt; 20 for i in read.qual) &amp;lt;=2:&#xA;           flag1 = True&#xA;           return  flag1&#xA;&#xA;pool = multiprocessing.Pool(5)&#xA;pool.ncpus = 4&#xA;for file in os.listdir('/moRNA_data/trimmed_fastq'):&#xA;    if file.endswith('.fq'):&#xA;        fastq_file = HTSeq.FastqReader( os.path.join('/moRNA_data/trimmed_fastq',file))&#xA;        name = str(file.split('.')[0]) + '.fastq'&#xA;        file_out = open(os.path.join('/moRNA_data/filtered_fastq',name), &quot;w&quot; )&#xA;        for read in tqdm(fastq_file):    &#xA;            flag1 = pool.apply_async(filtering,read)&#xA;            if flag1:&#xA;                read.write_to_fastq_file( file_out )&#xA;pool.close()&#xA;pool.join()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The problem with this script is that it works well with small file like upto 100-200 MB but when it goes to directory having multiple sample each having size more than 3 &#xA;GB, it works well for first file but after that it hangs so severely that I had to restart the server [ I can't even kill the process]. I need suggestions to improve this script to make fastq file processing faster. Platform details is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;OS : Ubuntu-18.02&#xA;Architecture:        x86_64&#xA;CPU op-mode(s):      32-bit, 64-bit&#xA;Byte Order:          Little Endian&#xA;CPU(s):              12&#xA;On-line CPU(s) list: 0-11&#xA;Thread(s) per core:  2&#xA;Core(s) per socket:  6&#xA;Socket(s):           1&#xA;NUMA node(s):        1&#xA;Vendor ID:           GenuineIntel&#xA;CPU family:          6&#xA;Model:               158&#xA;Model name:          Intel(R) Core(TM) i7-8700 &#xA;CPU @ 3.20GHz&#xA;Stepping:            10&#xA;CPU MHz:             800.560&#xA;CPU max MHz:         4600.0000&#xA;CPU min MHz:         800.0000&#xA;BogoMIPS:            6384.00&#xA;Virtualization:      VT-x&#xA;L1d cache:           32K&#xA;L1i cache:           32K&#xA;L2 cache:            256K&#xA;L3 cache:            12288K&#xA;NUMA node0 CPU(s):   0-11&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="2141" LastEditorUserId="104" LastEditDate="2019-06-12T11:25:34.197" LastActivityDate="2019-06-12T14:27:56.713" Title="Fast processing of fastq data" Tags="|fastq|filtering|text-processing|data-preprocessing|" AnswerCount="1" CommentCount="18" ContentLicense="CC BY-SA 4.0" />
  <row Id="8793" PostTypeId="2" ParentId="8791" CreationDate="2019-06-12T14:27:56.713" Score="6" Body="&lt;p&gt;Knowing a high-performance language will make a huge difference here. See the example below in C. I haven't tested, but it should be easy to modify for your purpose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;C++, Rust, Go, Nim and Julia can be as fast.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;// Download https://raw.githubusercontent.com/lh3/minimap2/master/kseq.h&#xA;// Compile with: gcc -O2 -o myprog this-file.c -lz&#xA;#include &amp;lt;zlib.h&amp;gt;  &#xA;#include &amp;lt;stdio.h&amp;gt;  &#xA;#include &quot;kseq.h&quot;&#xA;// STEP 1: declare the type of file handler and the read() function  &#xA;KSEQ_INIT(gzFile, gzread)  &#xA;&#xA;int main(int argc, char *argv[])  &#xA;{  &#xA;    gzFile fp;  &#xA;    kseq_t *seq;  &#xA;    if (argc == 1) {  &#xA;        fprintf(stderr, &quot;Usage: %s &amp;lt;in.seq&amp;gt;\n&quot;, argv[0]);  &#xA;        return 1;  &#xA;    }  &#xA;    fp = gzopen(argv[1], &quot;r&quot;); // STEP 2: open the file handler  &#xA;    seq = kseq_init(fp); // STEP 3: initialize seq  &#xA;    while (kseq_read(seq) &amp;gt;= 0) { // STEP 4: read sequence  &#xA;        int i, sum, low;&#xA;        if (seq-&amp;gt;qual.l == 0) continue; // no quality&#xA;        if (seq-&amp;gt;seq.l &amp;lt; 15 || seq-&amp;gt;seq.l &amp;gt; 30) continue;&#xA;        for (i = sum = low = 0; i &amp;lt; seq-&amp;gt;seq.l; ++i) {&#xA;            int qual = seq-&amp;gt;qual.s[i] - 33; // assuming Sanger quality&#xA;            sum += qual;&#xA;            if (qual &amp;lt; 20) ++low;&#xA;        }&#xA;        if (sum &amp;lt; seq-&amp;gt;seq.l * 30.0 || low &amp;gt; 2) continue;&#xA;        printf(&quot;@%s\n&quot;, seq-&amp;gt;name.s); // output can be made faster&#xA;        puts(seq-&amp;gt;seq.s);&#xA;        printf(&quot;+\n&quot;);&#xA;        puts(seq-&amp;gt;qual.s);&#xA;    }  &#xA;    kseq_destroy(seq); // STEP 5: destroy seq  &#xA;    gzclose(fp); // STEP 6: close the file handler  &#xA;    return 0;  &#xA;}  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To apply the program to all &lt;code&gt;*.fq&lt;/code&gt; in a directory (again, &lt;em&gt;not&lt;/em&gt; tested):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;mkdir -p filtered&#xA;ls *.fq | xargs -i echo ./myprog {} \&amp;gt; filtered/{} | sh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can also replace the final &lt;code&gt;sh&lt;/code&gt; with &lt;code&gt;parallel&lt;/code&gt;, but the above should be fast enough.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2019-06-12T14:27:56.713" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8794" PostTypeId="1" AcceptedAnswerId="8796" CreationDate="2019-06-13T07:20:36.677" Score="1" ViewCount="346" Body="&lt;p&gt;There are difference annotation file for UCSC and gencode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if I use the reference assembly from UCSC and the GTF from Genocode or vice versa would my downstream results would be wrong? &lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="77" LastEditDate="2019-06-13T07:49:16.487" LastActivityDate="2019-06-13T09:00:04.287" Title="How drastic would the result be if I use fasta or reference assembly from ucsc and gtf from gencode?" Tags="|rna-seq|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8795" PostTypeId="2" ParentId="8794" CreationDate="2019-06-13T07:46:45.880" Score="1" Body="&lt;p&gt;The main difference is in the way the chromosomes are named - UCSC uses the &quot;chr&quot; prefix (so chromosome 1 is &quot;chr1&quot;) while in gencode the &quot;chr&quot; isn't used (so chromosome 1 is just &quot;1&quot;). Depending on your use case, this can obviously cause problems - if you're trying to match a locus (e.g. from gencode 1:1000002) between them, whatever tool you use is going to be looking in your aligned data for &quot;1:1000002&quot;, but in your aligned data, it'll be named &quot;chr1:1000002&quot;, so it won't match the two up. &lt;/p&gt;&#xA;" OwnerUserId="3624" LastActivityDate="2019-06-13T07:46:45.880" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8796" PostTypeId="2" ParentId="8794" CreationDate="2019-06-13T08:08:54.850" Score="5" Body="&lt;p&gt;&lt;strong&gt;Never use genomes or annotations from UCSC&lt;/strong&gt;, they're poorly versioned and only recently with mouse and human have they even included all of the contigs. For fasta/GTF files from early in the GRCh38 release, you can tell whether you're using UCSC or Gencode by the presence/absence of &lt;code&gt;_random&lt;/code&gt; contigs, which will only exist for UCSC. These were mostly later split into the actual contigs, so recent download from UCSC should more closely match what you find at Gencode/Ensembl. Further, that time predated UCSC beginning to adopt Gencode's vastly superior annotations, so if your GTF file has instances where the same gene ID is on either multiple strands or multiple chromosomes (this is obviously biologically impossible) then you have a UCSC GTF file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, with early GRCh38 releases your only real issues will be with _random contigs, which are a minority of the genome and don't have all that many genes. But really, you should be keeping track of the sources of your files and ensuring that they're compatible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;: I should expand a bit on my &quot;Never use genomes or annotations from UCSC&quot; comment. In point of fact the genomes themselves aren't so terrible. Early on UCSC had the bad habit of concatenating contigs together into &lt;code&gt;_random&lt;/code&gt; &quot;chromosomes&quot;, but they seem to have mostly kicked that habit as of late. Note, however, that there are no versions for their genomes. Since reference genomes continue to get updates over time (mostly through the addition of patches) the lack of actual versions means you have to manually check if a recently downloaded file matches what may have been downloaded either previously or by someone else. This has obvious consequences when it comes to reproducibility. The same issue occurs for annotations from UCSC, but they have the additional problem of historically having biologically incoherent concepts of genes. That is, they will contain the same gene in multiple places with multiple orientations, which will break many tools in both obvious and completely unclear ways. For example, DEXSeq will simply break with an error message if given a UCSC annotation, since they break biological plausibility. If you were to use these annotations files with deepTools, you wouldn't get an error message, but the resulting output would be only partial, due to the biologically impossible annotation effectively corrupting most obvious ways of storing annotation data in a data structure (i.e., you can no longer treat IDs as unique). This could have downstream ramifications on biological interpretation of results.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2019-06-13T09:00:04.287" LastActivityDate="2019-06-13T09:00:04.287" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="8797" PostTypeId="1" AcceptedAnswerId="8798" CreationDate="2019-06-13T12:50:57.907" Score="2" ViewCount="1252" Body="&lt;p&gt;I have the series of coordinates in &lt;code&gt;id.txt&lt;/code&gt; file, whose coordinates sequences are in &lt;code&gt;genome.fasta&lt;/code&gt; file.&#xA;The coordinates of id.txt file is shown below,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Contig3:15-7&#xA;Contig2:5-10&#xA;Contig1:12-3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The genome.fasta file is shown below,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Contig1&#xA;AAGGCCATCAAGGACGTGGATGAGGTCGTCAAG&#xA;&amp;gt;Contig2&#xA;ACGAAGCGCTCGCCAAGGCCGAAGAAGAAGGCC &#xA;&amp;gt;Contig3&#xA;GCTGCGGCGCTGATCCTGGCGGCCCGCGCCGAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have used the following codes to extract the sequences from &lt;code&gt;genome.fasta&lt;/code&gt; based on the coordinates in &lt;code&gt;id.txt&lt;/code&gt; by using the following command &lt;code&gt;xargs samtools faidx genome.fasta &amp;lt;id.txt &amp;gt; result.fasta&lt;/code&gt;&#xA;It has extracted only &lt;code&gt;Contig2:5-10&lt;/code&gt;sequence. Because, the coordinates of &lt;code&gt;Contig2:5-10&lt;/code&gt;is in proper order. But &lt;code&gt;Contig3:15-7&lt;/code&gt; and &lt;code&gt;Contig1:12-3&lt;/code&gt; coordinates are in inverse/reverse order, So samtools could not fetch those sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to extract those sequences from &lt;code&gt;Contig1:12-3&lt;/code&gt; and &lt;code&gt;Contig3:15-7&lt;/code&gt; coordinates and also I want to &lt;code&gt;reverse complement&lt;/code&gt; them. I have  larger fasta file and plenty of coordinates sequences to be extracted, So I need to automate this process. So, please help me to automate this process.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The expected output is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Contig3    &#xA;GATCAGCGC&#xA;&amp;gt;Contig2&#xA;AGCGCT&#xA;&amp;gt;Contig1&#xA;CTTGATGGCC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="4712" LastEditorUserId="4712" LastEditDate="2019-06-14T09:28:38.153" LastActivityDate="2019-06-14T18:41:01.200" Title="How to reverse complement the DNA sequences for given inverse/reverse coordinates?" Tags="|python|fasta|samtools|shell|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8798" PostTypeId="2" ParentId="8797" CreationDate="2019-06-13T15:00:15.163" Score="1" Body="&lt;p&gt;The &lt;code&gt;exonerate&lt;/code&gt; tool comes with some very handy utilities for sequence manipulation. Here, we are interested in &lt;code&gt;fastasubseq&lt;/code&gt; and &lt;code&gt;fasterevcomp&lt;/code&gt;. First, install &lt;code&gt;exonerate&lt;/code&gt; as described &lt;a href=&quot;https://www.ebi.ac.uk/about/vertebrate-genomics/software/exonerate&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. If you're using Debian or Ubuntu or one of their derivatives, it might be enough to run&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sudo apt install exonerate&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, follow these steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Create one fasta file for each sequence in &lt;code&gt;genome.fasta&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fastaexplode genome.fasta &#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A little preprocessing&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk -F[:-] '{ &#xA;                if($2&amp;gt;$3){&#xA;                    start=$3-1; &#xA;                    len=$2-start&quot; -&quot;&#xA;                }&#xA;                else{&#xA;                    start=$2-1; &#xA;                    len=$3-start&#xA;                } &#xA;                print $&lt;/span&gt;1,start,len}' id.txt &#xA;Contig3 7 8 -&#xA;Contig2 5 5&#xA;Contig1 3 9 -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, that will print the target contig's name, the start coordinate of the target sequence, the length of the target sequence and a &lt;code&gt;-&lt;/code&gt; if it is on the reverse strand. The &lt;code&gt;fastasubseq&lt;/code&gt; program I will use below starts counting from 0 and not from 1, so we also need to subtract 1 from the coordinates to get them right.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Now, feed that into &lt;code&gt;fastafetch&lt;/code&gt; and &lt;code&gt;fastarevcomp&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tmpFile=&lt;span class=&quot;math-container&quot;&gt;$(mktemp); &#xA;    &amp;gt; subseqs.fa&#xA;    awk -F[:-] '{ &#xA;                if($2&amp;gt;$3){&#xA;                    start=$3-1; &#xA;                    len=$2-start&quot; -&quot;&#xA;                }&#xA;                else{&#xA;                    start=$2-1; &#xA;                    len=$3-start&#xA;                } &#xA;                print $1,start,len}' id.txt |&#xA;    while read cont start len rev; do &#xA;        fastasubseq &quot;$&lt;/span&gt;cont&quot;.fa &lt;span class=&quot;math-container&quot;&gt;$start $&lt;/span&gt;len &amp;gt; &lt;span class=&quot;math-container&quot;&gt;$tmpFile; &#xA;        if [[ -n $&lt;/span&gt;rev ]]; then &#xA;            fastarevcomp &quot;&lt;span class=&quot;math-container&quot;&gt;$tmpFile&quot; &amp;gt;&amp;gt; subseqs.fa; &#xA;        else &#xA;            cat &quot;$&lt;/span&gt;tmpFile&quot; &amp;gt;&amp;gt; subseqs.fa; &#xA;    fi&#xA;    done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will produce the following file when run on your data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat subseqs.fa &#xA;&amp;gt;Contig3:subseq(6,9) [revcomp]&#xA;GATCAGCGC&#xA;&amp;gt;Contig2:subseq(4,6)&#xA;AGCGCT&#xA;&amp;gt;Contig1:subseq(2,10) [revcomp]&#xA;CTTGATGGCC&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Finally, remove the per-contig fasta files&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rm Contig*fa&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-06-14T09:56:17.523" LastActivityDate="2019-06-14T09:56:17.523" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="8799" PostTypeId="1" CreationDate="2019-06-13T15:23:30.877" Score="2" ViewCount="104" Body="&lt;p&gt;In my first approach to the use of Gene Ontology, I have found a discordance if we consult either Mouse Genome Informatics (MGI) or Ensembl for these data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Concretely, for the mouse genes Zic1 (NCBI 2271) and Zic2 (NCBI 2272), MGI assigns them  up to &lt;a href=&quot;http://www.informatics.jax.org/go/marker/MGI:106683&quot; rel=&quot;nofollow noreferrer&quot;&gt;23&lt;/a&gt; and &lt;a href=&quot;http://www.informatics.jax.org/go/marker/MGI:106679&quot; rel=&quot;nofollow noreferrer&quot;&gt;24&lt;/a&gt; different GO terms, respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However these numbers are different according to Ensembl data retrieved through R/Bioconductor. In particular, for Zic2 (Id 22772) the difference is very significant (24 terms in MGI vs. only 3 terms in Ensembl):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; my_ensembl = useMart (&quot;ensembl&quot;, dataset = &quot;mmusculus_gene_ensembl&quot;)&#xA;&amp;gt; getBM (attributes = &quot;go_id&quot;, filters =&quot;entrezgene&quot; , values = '22771', mart = my_ensembl)&#xA;        go_id&#xA;1  GO:0003676&#xA;2  GO:0046872&#xA;3  GO:0003677&#xA;4  GO:0005634&#xA;../..&#xA;22 GO:0008589&#xA;23 GO:0007389&#xA;24 GO:0021510&#xA;25 GO:0042472&#xA;&amp;gt; getBM (attributes = &quot;go_id&quot;, filters =&quot;entrezgene&quot; , values = '22772', mart = my_ensembl)&#xA;       go_id&#xA;1 GO:0003676&#xA;2 GO:0005634&#xA;3 GO:0016604&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, my questions are: Which is the reason for this discordance? Has it something to do with the different types of evidence to assign GO terms? Also, in order to proceed to analysis like GO enrichment, which of these two databases would be more appropriate? &lt;/p&gt;&#xA;" OwnerUserId="4543" LastEditorUserId="48" LastEditDate="2019-06-14T09:28:01.273" LastActivityDate="2019-06-14T12:16:10.113" Title="Discordance of Gene Ontology data between Mouse Genome Informatics and Ensembl sources" Tags="|bioconductor|ensembl|biomart|gene-ontology|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8800" PostTypeId="2" ParentId="8549" CreationDate="2019-06-13T18:26:31.503" Score="3" Body="&lt;p&gt;I stumbled across this issue after getting the same warning message from &lt;code&gt;liftOver&lt;/code&gt;. By using the &lt;code&gt;-out&lt;/code&gt; option of &lt;code&gt;lgHgGene&lt;/code&gt;, the &lt;code&gt;database&lt;/code&gt; parameter is ignored but must still be provided. The &lt;code&gt;table&lt;/code&gt; parameter should be the second column from your GTF file. Note that you must also specify the &lt;code&gt;-gtf&lt;/code&gt; option (otherwise it expects GFF format). In my case, the GTF file looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    mm10_knownGene  exon    3073253 3074322 0.000000        +       .       gene_id &quot;ENSMUST00000193812.1&quot;; transcript_id &quot;ENSMUST00000193812.1&quot;;&#xA;chr1    mm10_knownGene  exon    3102016 3102125 0.000000        +       .       gene_id &quot;ENSMUST00000082908.1&quot;; transcript_id &quot;ENSMUST00000082908.1&quot;;&#xA;chr1    mm10_knownGene  exon    3206523 3207317 0.000000        -       .       gene_id &quot;ENSMUST00000159265.1&quot;; transcript_id &quot;ENSMUST00000159265.1&quot;;&#xA;chr1    mm10_knownGene  exon    3213439 3215632 0.000000        -       .       gene_id &quot;ENSMUST00000159265.1&quot;; transcript_id &quot;ENSMUST00000159265.1&quot;;&#xA;chr1    mm10_knownGene  exon    3205901 3207317 0.000000        -       .       gene_id &quot;ENSMUST00000162897.1&quot;; transcript_id &quot;ENSMUST00000162897.1&quot;;&#xA;chr1    mm10_knownGene  exon    3213609 3216344 0.000000        -       .       gene_id &quot;ENSMUST00000162897.1&quot;; transcript_id &quot;ENSMUST00000162897.1&quot;;&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;By default, &lt;code&gt;ldHgGene&lt;/code&gt; outputs a leading 'bin' column which won't work with &lt;code&gt;liftOver&lt;/code&gt;; this can be avoided with the &lt;code&gt;-nobin&lt;/code&gt; option. So my command was:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ldHgGene -gtf -nobin -out=mm10.GENCODE_VM20.gp ignored mm10_knownGene mm10.GENCODE_VM20.gtf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The resulting genePred output can be fed directly into &lt;code&gt;liftOver -genePred&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ENSMUST00000193812.1    chr1    +       3073252 3074322 3074322 3074322 1       3073252,        3074322,&#xA;ENSMUST00000082908.1    chr1    +       3102015 3102125 3102125 3102125 1       3102015,        3102125,&#xA;ENSMUST00000162897.1    chr1    -       3205900 3216344 3216344 3216344 2       3205900,3213608,        3207317,3216344,&#xA;ENSMUST00000159265.1    chr1    -       3206522 3215632 3215632 3215632 2       3206522,3213438,        3207317,3215632,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4713" LastActivityDate="2019-06-13T18:26:31.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8802" PostTypeId="2" ParentId="8799" CreationDate="2019-06-14T12:16:10.113" Score="1" Body="&lt;p&gt;Short answer, we get our annotations from GOA, which has the three listed. We're in touch with GOA to see why they differ from MGI.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2019-06-14T12:16:10.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8803" PostTypeId="1" AcceptedAnswerId="8807" CreationDate="2019-06-14T12:45:27.520" Score="2" ViewCount="228" Body="&lt;p&gt;I have a &quot;test&quot; &lt;code&gt;data.frame&lt;/code&gt; object in R, which is basically a small subset of a 66000 row dataframe, which looks as follows:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr   start    end      PC1     comp&#xA;chr1  3360001  3400000  -0.009  B&#xA;chr1  3400001  3440000  -0.004  B&#xA;chr1  3440001  3480000  -0.001  B&#xA;chr1  3480001  3520000  0.003  A&#xA;chr1  3520001  3560000  0.002  A&#xA;chr1  3560001  3600000  -0.001  B&#xA;chr1  3600001  3640000  0.002  A&#xA;chr1  3640001  3680000  0.004  A&#xA;chr1  3720001  3760000  0.003  A&#xA;chr1  3760001  3800000  0.003  A&#xA;chr1  3800001  3840000  0.007  A&#xA;chr1  3840001  3880000  0.006  A&#xA;chr1  3880001  3920000  -0.004  B&#xA;chr1  3920001  3960000  -0.017  B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, the interval between each start and end position is &lt;code&gt;40kb&lt;/code&gt;. The column &lt;code&gt;comp&lt;/code&gt; can have only two values, &lt;code&gt;A&lt;/code&gt; or &lt;code&gt;B&lt;/code&gt;. Going from &lt;code&gt;row 1&lt;/code&gt; to the last row of my dataframe and based on the values of the column &lt;code&gt;comp&lt;/code&gt;, I want to get the whole stretch/block of the genomic region, once the compartment(character in column &lt;code&gt;comp&lt;/code&gt;) is changed, and perform a &lt;code&gt;mean&lt;/code&gt; of the &lt;code&gt;PC1&lt;/code&gt; values in that stretch.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in this example, I basically want to shrink my test dataframe as follows: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1  3360001  3480000  mean(c(-0.009,-0.004,-0.001))  B&#xA;chr1  3480001  3560000  mean(c(0.003,0.002))  A&#xA;chr1  3560001  3600000  mean(c(-0.001))  B  &amp;lt;---- #notice this is a 'singleton'&#xA;chr1  3600001  3880000  mean(c(0.002,0.004,0.003,0.003,0.007,0.006))  A&#xA;chr1  3880001  3960000  mean(c(-0.004,-0.017))  B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried with a simple code, but it only runs until the first compartment value change(so row 4 in this test), and I am unable to figure out how to make it run for the whole dataset of 66000 rows. Also the use of multiple &lt;code&gt;for&lt;/code&gt; loops, which is what I tried, may not be the prettiest way to do this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess there exists a &lt;code&gt;Bioconductor&lt;/code&gt; package which performs this task very easily. If anyone can point me to that direction, it'd be great.  &lt;/p&gt;&#xA;" OwnerUserId="2094" LastEditorUserId="678" LastEditDate="2019-06-14T13:12:22.197" LastActivityDate="2019-08-29T15:15:04.747" Title="Getting a stretch of genomic ranges from a dataframe/granges object based on metadata column" Tags="|r|bioconductor|data-management|genomicranges|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8804" PostTypeId="1" AcceptedAnswerId="8809" CreationDate="2019-06-14T12:57:53.543" Score="2" ViewCount="2924" Body="&lt;p&gt;I can make a selection in Pymol using the resi parameter. Is there a way to get this resi data back out of a selection object? The only thing I found so far is how to get the position, as in the x/y/z coordinates from a selection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For context, I have a table with a set of data points for each residue in the amino acid sequence of the displayed structure and I want to write a function where I can print the data based on the current selection. So my thought would be something like the following, where the content of the &lt;em&gt;'get_residue_positions_for_selection'&lt;/em&gt; function would be what I'm asking about.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def show_selection_position_data():&#xA;    residue_data = {&#xA;        &quot;header&quot;: ['variation', 'rmsf']&#xA;        , &quot;1&quot;: ['high', 0.1]&#xA;        , ...&#xA;    }&#xA;    selections = cmd.get_names('selection')&#xA;&#xA;    if sele not in selections:&#xA;        print(&quot;No selection&quot;)&#xA;        return&#xA;&#xA;    residues = get_residue_positions_for_selection(&quot;sele&quot;)&#xA;&#xA;    for residue in residue_selections:&#xA;        print(residue_data[residue])&#xA;&#xA;    return&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4441" LastActivityDate="2019-06-14T15:47:21.523" Title="Get residue position from Pymol Selection" Tags="|pymol|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8805" PostTypeId="2" ParentId="8803" CreationDate="2019-06-14T13:44:30.573" Score="0" Body="&lt;p&gt;Take a look at the &lt;code&gt;rle&lt;/code&gt; function:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Compute the lengths and values of runs of equal values in a vector – or the reverse operation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In this way you can mark the moment in which A changes to B and the length of the streches of A and B. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With that I think you can reduce the number of for loops you have to use (I can not think a way of doing this without at least one loop, but for sure it may exist)&lt;/p&gt;&#xA;" OwnerUserId="678" LastActivityDate="2019-06-14T13:44:30.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8806" PostTypeId="2" ParentId="8797" CreationDate="2019-06-14T14:05:20.187" Score="2" Body="&lt;p&gt;Since you tagged python, here is a solution that solves this in a single script using Biopython:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import sys&#xA;from Bio import SeqIO&#xA;&#xA;d = {}&#xA;with open('id.txt') as f:&#xA;    for line in f:&#xA;        contig, coords = line.split(':')&#xA;        start, end = map(int, coords.split('-'))&#xA;        d[contig] = start, end&#xA;&#xA;for record in SeqIO.parse('genome.fasta', 'fasta'):&#xA;    start, end = d[record.id]&#xA;    if start &amp;gt; end:&#xA;        record.seq = record.seq[end - 1: start].reverse_complement()&#xA;    else:&#xA;        record.seq = record.seq[start - 1: end]&#xA;    SeqIO.write(record, sys.stdout, 'fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output on your example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Contig1&#xA;CTTGATGGCC&#xA;&amp;gt;Contig2&#xA;AGCGCT&#xA;&amp;gt;Contig3&#xA;GATCAGCGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastActivityDate="2019-06-14T14:05:20.187" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8807" PostTypeId="2" ParentId="8803" CreationDate="2019-06-14T14:13:02.030" Score="2" Body="&lt;p&gt;The GenomicRanges package in bioconductor will do this.  It lets you do 'set' operations on genomic ranges, such as concatenating them, finding overlaps, etc.  Hopefully something in the following code will be inspiring, even if I haven't solved the exact problem for you here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(GenomicRanges)&#xA;&#xA;# make a GenomicRanges set corresponding to the original table&#xA;test.gr = makeGRangesFromDataFrame(df = test, start.field = &quot;start&quot;, end.field = &quot;end&quot;, seqnames.field = &quot;chr&quot;)&#xA;&#xA;# make separate GenomicRanges sets for each of the two partitions of the genome&#xA;test_A.gr = makeGRangesFromDataFrame(df = test[test&lt;span class=&quot;math-container&quot;&gt;$comp==&quot;A&quot;,], start.field = &quot;start&quot;, end.field = &quot;end&quot;, seqnames.field = &quot;chr&quot;)&#xA;test_B.gr = makeGRangesFromDataFrame(df = test[test$&lt;/span&gt;comp==&quot;B&quot;,], start.field = &quot;start&quot;, end.field = &quot;end&quot;, seqnames.field = &quot;chr&quot;)&#xA;&#xA;# apply the reduce() function to concatenate segments which are adjoining each other into single segments&#xA;reduced_test_A.gr = reduce(test_A.gr)&#xA;reduced_test_B.gr = reduce(test_B.gr)&#xA;&#xA;# group PC1 values by segment classification&#xA;test[queryHits(findOverlaps(test.gr, reduced_test_A.gr)), &quot;PC1&quot;]&#xA;test[queryHits(findOverlaps(test.gr, reduced_test_B.gr)), &quot;PC1&quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-06-14T14:13:02.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8808" PostTypeId="1" AcceptedAnswerId="8810" CreationDate="2019-06-14T15:41:41.570" Score="0" ViewCount="1199" Body="&lt;p&gt;I'm attempting to perform differential expression analysis using DESeq2 on a dataset of ~90 samples and ~20000 transcripts. I have a number of variables I'd like to test against, some of which are qualitative (factors) and some of which are quantitative (numeric). A sample of my data are below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(data.phenotype.test)&#xA;# A tibble: 6 x 7&#xA;  Novogene_ID BORIS_ID     treatment queenness ovaries   elo sampletime&#xA;  &amp;lt;fct&amp;gt;       &amp;lt;fct&amp;gt;        &amp;lt;fct&amp;gt;         &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;fct&amp;gt;     &#xA;1 W03BLGR     003W-blu-grn Qr12        0.0202    124.  1000  12        &#xA;2 W03GR       003W-grn     Qr12        0.00210    66.8  923. 12        &#xA;3 W03SL       003W-sil     Qr12        0.969     316.  1077. 12        &#xA;4 W04BLOR     004W-blu-ora Qr12        0.00168    66.4  876. 12        &#xA;5 W04GRSL     004W-grn-sil Qr12        0.175     210.   931. 12        &#xA;6 W04YLWH     004W-yel-whi Qr12        0.984     302.  1192. 12        &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, let's say I want to test which transcripts are differentially expressed together with a numeric variable such as 'ovaries': &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dds.transcript.ovaries = DESeqDataSetFromMatrix(countData = data.transcript.count.ovaries,&#xA;                                        colData = as.matrix(data.phenotype.ovaries),&#xA;                                        design = ~ ovaries)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This &lt;strong&gt;should&lt;/strong&gt; work, but DESeq2 does something strange- it converts 'ovaries' to a factor, with the message &lt;em&gt;&quot;some variables in design formula are characters, converting to factors&quot;&lt;/em&gt;. But there are no character values in the supplied data frame, and certainly not in the column 'ovaries'! This conversion to factors is a big problem for me because, having converted a numeric column to factors, DESeq2 unsurprisingly reports that differential expression analysis cannot be performed because there are an equal number of factor levels to samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found a temporary workaround: if I reduce the data frame to just the 'ovaries' column, DESeq2 no longer converts the numeric data to factor levels and I'm able to perform differential expression analysis as normal. The problem is that I'd eventually like to perform multivariate analyses, e.g. with a design like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;~ ovaries + elo + treatment&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, doing so isn't going to be possible if DESeq2 keeps converting the numeric columns to factors, which it persistently does unless the numeric columns are supplied alone (which of course they can't be for a multivariate design).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What's going on here? Why does DESeq2 convert the numeric columns to factors, seemingly having misinterpreted those columns as characters?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB: I have replicated this issue with different data on a different machine and the result is the same, so I don't believe this an issue with my R setup.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Here's the structure of the df:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; str(data.phenotype.test)&#xA;Classes ‘tbl_df’, ‘tbl’ and 'data.frame':   64 obs. of  7 variables:&#xA; &lt;span class=&quot;math-container&quot;&gt;$ Novogene_ID: Factor w/ 64 levels &quot;W03BLGR&quot;,&quot;W03GR&quot;,..: 1 2 3 4 5 6 7 8 9 10 ...&#xA; $&lt;/span&gt; BORIS_ID   : Factor w/ 64 levels &quot;003W-blu-grn&quot;,..: 1 2 3 4 5 6 7 8 9 10 ...&#xA; &lt;span class=&quot;math-container&quot;&gt;$ treatment  : Factor w/ 4 levels &quot;Qc&quot;,&quot;Qr12&quot;,&quot;Qr3&quot;,..: 2 2 2 2 2 2 2 2 2 2 ...&#xA; $&lt;/span&gt; queenness  : num  0.02017 0.0021 0.96853 0.00168 0.17529 ...&#xA; &lt;span class=&quot;math-container&quot;&gt;$ ovaries    : num  124.1 66.8 316.1 66.4 209.9 ...&#xA; $&lt;/span&gt; elo        : num  1000 923 1077 876 931 ...&#xA; $ sampletime : Factor w/ 3 levels &quot;0&quot;,&quot;3&quot;,&quot;12&quot;: 3 3 3 3 3 3 3 3 3 3 ...&#xA; - attr(*, &quot;na.action&quot;)= 'omit' Named int  1 2 3 4 5 6 7 8 9 10 ...&#xA;  ..- attr(*, &quot;names&quot;)= chr  &quot;1&quot; &quot;2&quot; &quot;3&quot; &quot;4&quot; ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4717" LastEditorUserId="57" LastEditDate="2019-06-14T22:30:34.787" LastActivityDate="2019-06-14T22:30:34.787" Title="Why does DESeq2 convert numeric columns to factor during differential expression analysis?" Tags="|r|rna-seq|differential-expression|deseq2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8809" PostTypeId="2" ParentId="8804" CreationDate="2019-06-14T15:44:59.617" Score="2" Body="&lt;p&gt;Ok, after some more searching I found the answer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You have to use the &lt;strong&gt;cmd.iterate&lt;/strong&gt; function to iterate over the default selection (&quot;sele&quot;, first parameter). With the second parameter you can push the variables you need into an array for later access. In this case I wanted the residue number so I used the variable 'resv'. A full list of the variables exposed by the iterate function can be found on the &lt;a href=&quot;https://www.pymolwiki.org/index.php/Iterate&quot; rel=&quot;nofollow noreferrer&quot;&gt;pymolwiki&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that you can use a function call as the second parameter and if you need a more complex selection than 'sele' you can use selector.process in the iterate function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my final working function. Note that I used a set for storing the residue ids, since the iterate function technically iterates over the atoms and so the same residue will likely appear multiple times in your selection.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def show_selection_position_data():&#xA;    residue_data = {&#xA;            &quot;header&quot;: ['variation', 'rmsf']&#xA;            , 1: ['high', 0.1]&#xA;            , 2: ['high', 0.2]&#xA;            , 3: ['low', 0.1]&#xA;            , 4: ['high', 0.05]&#xA;            , 5: ['middle', 0.2]&#xA;            , 6: ['high', 0.1]&#xA;            , 7: ['high', 0.1]&#xA;        }&#xA;    selections = cmd.get_names('selections')&#xA;&#xA;    if &quot;sele&quot; not in selections:&#xA;        print(&quot;No selection&quot;)&#xA;        return&#xA;&#xA;    userSelection = &quot;sele&quot;&#xA;&#xA;    stored.residues = set()&#xA;    cmd.iterate(selector.process(userSelection), 'stored.residues.add(resv)')&#xA;&#xA;    for residue in stored.residues:&#xA;        if residue in residue_data:&#xA;            print(residue_data[residue])&#xA;        else:&#xA;            print(&quot;{}: no data&quot;.format(residue))&#xA;&#xA;    return&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4441" LastEditorUserId="4441" LastEditDate="2019-06-14T15:47:21.523" LastActivityDate="2019-06-14T15:47:21.523" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8810" PostTypeId="2" ParentId="8808" CreationDate="2019-06-14T16:03:11.897" Score="1" Body="&lt;p&gt;Edited:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's the &quot;as.matrix&quot; that is causing the problem.  When I use that on my sample annotation data, and make dds with that, it turns my numbers into factors&lt;/p&gt;&#xA;" OwnerUserId="2388" LastEditorUserId="2388" LastEditDate="2019-06-14T17:01:56.113" LastActivityDate="2019-06-14T17:01:56.113" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8811" PostTypeId="2" ParentId="8797" CreationDate="2019-06-14T18:41:01.200" Score="1" Body="&lt;p&gt;If you're okay with using a Perl script, you could convert your headers into stranded BED6 and use the following script &lt;code&gt;bed2faidxsta.pl&lt;/code&gt; with &lt;code&gt;samtools&lt;/code&gt;-indexed FASTA to get back stranded sequence:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gist.github.com/alexpreynolds/fa9b0f90e181e3b4f640&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gist.github.com/alexpreynolds/fa9b0f90e181e3b4f640&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This script looks at the sixth column of your BED file of chromosome, start, and stop positions, to decide whether to do a reverse complement of the queried sequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could use &lt;code&gt;awk&lt;/code&gt; to quickly build such a BED6 file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk -vFS='[:-]' '($&lt;/span&gt;0~/^&amp;gt;/){ if (&lt;span class=&quot;math-container&quot;&gt;$2 &amp;lt; $&lt;/span&gt;3) { strand = &quot;+&quot;; } else if (&lt;span class=&quot;math-container&quot;&gt;$2 &amp;gt; $&lt;/span&gt;3) { strand = &quot;-&quot;; } else { printf(&quot;Coordinates must differ\n&quot;); exit 1; } printf(&quot;%s\t%d\t%d\t.\t0\t%c\n&quot;, &lt;span class=&quot;math-container&quot;&gt;$1, $&lt;/span&gt;2, $3, strand; }' in.fa &amp;gt; out.bed6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To set up indexed FASTA, here's a generic shell-based approach, which uses [Kent utilities][1] &lt;code&gt;fetchChromSizes&lt;/code&gt; to get a list of chromosomes for the build &lt;code&gt;mm9&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ mkdir mm9&#xA;$&lt;/span&gt; cd mm9&#xA;&lt;span class=&quot;math-container&quot;&gt;$ while read chr; do echo &quot;$&lt;/span&gt;chr&quot;; wget -qO- &quot;http://hgdownload.cse.ucsc.edu/goldenpath/mm9/chromosomes/&lt;span class=&quot;math-container&quot;&gt;$chr.fa.gz&quot; | gunzip -c - &amp;gt; $&lt;/span&gt;chr.fa; samtools faidx $chr.fa; done &amp;lt; &amp;lt;(fetchChromSizes mm9 | cut -f1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This can be adjusted for &lt;code&gt;mm10&lt;/code&gt;, &lt;code&gt;hg19&lt;/code&gt;, &lt;code&gt;hg38&lt;/code&gt;, and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have indexed FASTA and the above Perl script, you could do something as easy as the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ./bed2faidxsta.pl --fastaDir=&quot;/path/to/indexed/mm9&quot; &amp;lt; in.bed &amp;gt; out.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you're working with a reference genome, the nice thing about this approach is that &lt;code&gt;awk&lt;/code&gt;, Perl, and indexed lookups are fast. You can reuse indexed FASTA for other projects that also require sequence lookups by genomic interval.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2019-06-14T18:41:01.200" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8812" PostTypeId="1" AcceptedAnswerId="8818" CreationDate="2019-06-15T00:47:38.283" Score="4" ViewCount="118" Body="&lt;p&gt;I would like some advice on potential strategies to address the following problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to write a program that will generate DNA sequences that are optimized on two constraints based on an input genome:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The GC content should be as close to the genome as possible.&lt;/li&gt;&#xA;&lt;li&gt;When taking all k-mers (of a predefined length) in the sequence, the sum of their frequencies in the genome should be as small as possible (i.e. it should be made of rare k-mers).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I currently score sequences using  :&#xA;&lt;span class=&quot;math-container&quot;&gt;$S = w_{GC} * (1 - |GC_{genome} - GC_{seq}|) + w_{Kmer} * (1 - \frac{mean(f(Kmer_{seq})}{max(f(Kmer_{genome}))}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;$w_{GC}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$w_{Kmer}$&lt;/span&gt; are user-defined weights defining the relative importance of GC content and k-mer profile) and &lt;span class=&quot;math-container&quot;&gt;$f(Kmer_{seq})$&lt;/span&gt; is the frequency of k-mers in the sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course I could just use bruteforce and generate many random sequences of appropriate GC content, score them all and pick the very best. But 1) that's no fun and 2) it's slow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first thing I attempted was to extend the sequence one basepair at a time. Each iteration has 4 candidates (with a new A, C, T or G added). I pick the candidate using their score (&lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;) as probability weight. This &quot;blind&quot; method is too simple and does not yield satisfactory results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I thought a dynamic approach could work well (filling a table and backtracking),  but I am not sure this is applicable since the cost of extending the sequence would be dependent on all new k-mers generated and not be a fixed value (such as in sequence alignment).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also thought something like reinforcement learning could be appropriate here, but I don't know much about it, so I am not sure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know all of this is probably overkill, but this is mostly for the challenge / learning :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which strategy would be most appropriate for this ? Did I miss something simple ? All advice / suggestions are welcome !&lt;/p&gt;&#xA;" OwnerUserId="439" LastEditorUserId="439" LastEditDate="2019-06-15T01:08:40.137" LastActivityDate="2019-06-16T16:43:54.023" Title="Generating DNA sequences with constraints" Tags="|sequence-analysis|genomics|k-mer|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8813" PostTypeId="1" CreationDate="2019-06-15T08:58:10.007" Score="1" ViewCount="353" Body="&lt;p&gt;Currently I'm working on a project, which combines deep learning with RNA sequences. I'll try to predict pseudotorsion angles [1] from raw rna sequence. The ideas is to train a neural network with raw rna sequences and for each nucleotide their corresponding pseudotorsion angles, and than to predict the angles of the remaining sequences in the test set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How my data is structured:&#xA;example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seq1:   A  C  G    G   U   A   C&#xA;Eta:   169 87 110 87  45  187 78&#xA;Theta: 123 10 45  168 132 34  100 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;[1] These are angles describing the backbone conformation of a rna molecule.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm pretty new to the field of deep learning, and so far I build a simple feedforward neural network, but it's prediction accuracy is pretty low with only one percent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone some tips for me how to improve this? &#xA;How do I preprocess this kind of data correctly for deep learning?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I appreciate any help.&lt;/p&gt;&#xA;" OwnerUserId="4721" LastActivityDate="2019-06-20T11:16:45.377" Title="Deep learning RNA sequences" Tags="|python|sequence-analysis|rna|machine-learning|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8814" PostTypeId="2" ParentId="8813" CreationDate="2019-06-15T13:38:43.620" Score="2" Body="&lt;p&gt;The question of folding of RNA sequences looks slightly similar to protein folding - perhaps searching in this domain might bring more suggestions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of (current) state of the art of deep learning approach to protein folding is &lt;a href=&quot;https://deepmind.com/blog/alphafold/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaFold&lt;/a&gt; developed by DeepMind for &lt;a href=&quot;http://predictioncenter.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;CASP&lt;/a&gt; competition . They basically decompose the angles to an image representation and look for structures in the proximity of each atom. While it requires a &lt;strong&gt;significant&lt;/strong&gt; processing power, RNA structures might be easier due to the size of the sequence.&lt;/p&gt;&#xA;" OwnerUserId="4628" LastActivityDate="2019-06-15T13:38:43.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8815" PostTypeId="2" ParentId="8813" CreationDate="2019-06-15T16:27:42.537" Score="0" Body="&lt;p&gt;I think the solution is more in the DL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would look at simpler ML, using a random forest as a starting point and then maybe SVC. If you get some reasonable predictive power on this then move towards neural network/Tensorflow type calculation.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-06-15T16:27:42.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8816" PostTypeId="1" AcceptedAnswerId="8826" CreationDate="2019-06-15T21:26:56.950" Score="1" ViewCount="1302" Body="&lt;p&gt;I am comparing single block module detection with 2 block module detection in WGCNA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know what this relates to? I have rechecked and rechecked my codes for both the single and 2 module detection and cannot see an error. Thank you&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get the error code: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in plotOrderedColors(dendro$order, colors = colors, rowLabels = rowLabels,  : &#xA;  ERROR: length of colors vector not compatible with number of objects in 'order'.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;net= blockwiseModules(Th1_h5, power=6,&#xA;                      TOMType = &quot;signed&quot;, minModuleSize=30,&#xA;                      reassignThreshold =0, mergeCutHeight = 0.25,&#xA;                      numericLabels = TRUE, pamRespectsDendro = FALSE,&#xA;                      saveTOMs = TRUE,&#xA;                      saveTOMFileBase = &quot;PBCreponseTOMB&quot;,&#xA;                      verbose= 3 &#xA;mergedColors = labels2colors(net$colors)&#xA;&#xA;plotDendroAndColors(net&lt;span class=&quot;math-container&quot;&gt;$dendrograms[[1]], mergedColors[net$&lt;/span&gt;blockGenes[[1]]],&#xA;                    &quot;Module colors&quot;,&#xA;                    dendroLabels = FALSE, hang = 0.03,&#xA;                    addGuide = TRUE, guideHang = 0.05)&#xA;moduleLabels = net&lt;span class=&quot;math-container&quot;&gt;$colors&#xA;moduleColors = labels2colors(net$&lt;/span&gt;colors)&#xA;MEs = net&lt;span class=&quot;math-container&quot;&gt;$MEs&#xA;geneTree = net$&lt;/span&gt;dendrograms[[1]]&#xA;&#xA;bwet = blockwiseModules(Th1_h5, maxBlockSize = 8000,&#xA;                        power=6, TOMType = &quot;signed&quot;, minModuleSize = 30,&#xA;                        reassignThreshold = 0, mergeCutHeight = 0.25,&#xA;                        numericLabels = TRUE,&#xA;                        saveTOMs = TRUE, &#xA;                        saveTOMFileBase = &quot;Th1_NR_R_TOM-blockwise&quot;,&#xA;                        verbose=3)&#xA;bwLabels = matchLabels(bwet$colors,moduleLabels)&#xA;&#xA;bwModuleColors = labels2colors(bwLabels)&#xA;&#xA;&#xA;plotDendroAndColors(geneTree,&#xA;                    cbind(moduleColors, bwModuleColors),&#xA;                    c(&quot;Single block&quot;, &quot;2 blocks&quot;),&#xA;                    main = &quot;Single block gene dendrogram and module colours&quot;,&#xA;                    dendroLabels = FALSE, hang = 0.03,&#xA;                    addGuide = TRUE, guideHang =  0.05) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="48" LastEditDate="2019-06-16T09:28:22.983" LastActivityDate="2019-06-17T22:40:34.813" Title="Error Code in WGCNA: Dealing with large data sets: block-wise network construction andmodule detection" Tags="|r|wgcna|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8817" PostTypeId="1" AcceptedAnswerId="8823" CreationDate="2019-06-16T11:20:24.280" Score="2" ViewCount="425" Body="&lt;p&gt;I would like to visualize tree with multiple sequence alignment. &#xA;My &lt;code&gt;din.newick&lt;/code&gt; file is shown below,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(org1:0.49999997,org2:0.47298786,org3:28.37858179);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My &lt;code&gt;din.fasta&lt;/code&gt; alignment file shown below,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;org1&#xA;ATGCTA&#xA;&amp;gt;org2&#xA;TAGCAT&#xA;&amp;gt;org3&#xA;AGTAGT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have tried the following code,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse)&#xA;library(ggtree) &#xA;library(seqinr)&#xA;tree = read.tree(&quot;din.newick&quot;) &#xA;msaplot(p=ggtree(tree), fasta = &quot;din.fasta&quot;)&#xA;msaplot(p=ggtree(tree) + geom_tiplab(align=TRUE), fasta = &quot;din.fasta&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above mentioned code is generating tree with the alignment matrix, but the tree root tip labels are not visible. It is either overlapped with matrix or it is not at all visible. &lt;a href=&quot;https://i.stack.imgur.com/y9LbQ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/y9LbQ.jpg&quot; alt=&quot;msaplot(p=ggtree(tree) + geom_tiplab(align=TRUE), fasta = &amp;quot;din.fasta&amp;quot;)&quot;&gt;&lt;/a&gt; Kindly help me to fix this issue. &lt;/p&gt;&#xA;" OwnerUserId="4712" LastEditorUserId="4712" LastEditDate="2019-06-18T12:20:38.820" LastActivityDate="2022-08-29T20:32:04.273" Title="Issue with visualising cladogram/phylogenetic tree with multiple sequence alignment data in R?" Tags="|r|bioconductor|fasta|phylogenetics|phylogeny|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8818" PostTypeId="2" ParentId="8812" CreationDate="2019-06-16T16:37:25.837" Score="4" Body="&lt;p&gt;You can use a &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;-order Markov chain. Here is the procedure:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Count &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;-mers in your genome. For small genomes, you can do that in Python. For large genomes, you may need jellyfish or KMC3.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Draw a &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;-mer randomly based on the distribution of &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;-mers. This generates the first &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; bases.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Let &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; be the last &lt;span class=&quot;math-container&quot;&gt;$(l-1)$&lt;/span&gt; subsequence from the generated sequence. There will be at most four &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;-mers prefixed with &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. It gives you a distribution of the next A/C/G/T base conditional on &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;. Randomly draw a base from this distribution. You get the next base.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Repeat step 3 until you generate a sequence long enough.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The generated sequence will have the same &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;-mer distribution to the original sequence. The GC content will be the same, too. In your case, if &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is small enough, you can let &lt;span class=&quot;math-container&quot;&gt;$l=k$&lt;/span&gt;. If &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is large, you have to choose a smaller &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The procedure above doesn't consider the strand symmetry of DNA, but probably this doesn't matter much. It is also possible to adapt the algorithm to address this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As to implementation, you can use python, but a high performance language like C will be tens of times faster at least.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2019-06-16T16:43:54.023" LastActivityDate="2019-06-16T16:43:54.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8819" PostTypeId="2" ParentId="8817" CreationDate="2019-06-16T17:21:32.333" Score="0" Body="&lt;p&gt;I suspect the solution is to realign the two plots within &quot;ggarrange&quot;, available &lt;a href=&quot;https://www.rdocumentation.org/packages/ggpubr/versions/0.2/topics/ggarrange&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is stacking ggtree with geom_tiplab. The ggarrange library is described on STHDA &lt;a href=&quot;http://www.sthda.com/english/articles/24-ggpubr-publication-ready-plots/81-ggplot2-easy-way-to-mix-multiple-graphs-on-the-same-page/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have not used this library however, but it should be easy because you are working singly within R. I tend to use ggplot2 within Python via rpy2, which is much more difficult and restricted. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-06-18T15:00:47.333" LastActivityDate="2019-06-18T15:00:47.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8821" PostTypeId="1" AcceptedAnswerId="8834" CreationDate="2019-06-17T15:49:26.410" Score="3" ViewCount="108" Body="&lt;p&gt;The occupancy of SMARCD3 in the target genes listed below. I want to see average, normalized ChIP-seq signal at the promoter proximal region (1000bp upstream and downstream of the TSS). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 4 different experimental conditions (overlayed in one plot, per target gene using different color codes) where we want to visualize the changes in occupancy of GATA1 under all the conditions at individual gene promoters. The following are our target genes &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;ABCB1&lt;/li&gt;&#xA;&lt;li&gt;ABCC1&lt;/li&gt;&#xA;&lt;li&gt;ABCC2&lt;/li&gt;&#xA;&lt;li&gt;SNAI2&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So how GATA1 occupancy is changed across all these target genes across 4 different treatment condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TJ0I1.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TJ0I1.jpg&quot; alt=&quot;representative figure&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far what i have done&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Aligned the samples &lt;/li&gt;&#xA;&lt;li&gt;Done peak calling&lt;/li&gt;&#xA;&lt;li&gt;Annotated the peak &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I did steps 2 and 3 using homer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now after all this how do I find the occupancy of a TF with its target genes? Do I get the information from my peaks annotated output or from the peak files?&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2019-06-19T13:18:36.793" LastActivityDate="2019-06-19T13:28:18.383" Title="Occupancy of TFs with the target genes" Tags="|visualization|chip-seq|peak-calling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8822" PostTypeId="2" ParentId="7212" CreationDate="2019-06-17T15:59:39.553" Score="2" Body="&lt;p&gt;These folders are usually generated in a default directory (where the raw files are). It can however be beneficial to set the paths to these folders manually.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Temporary folder&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;that is obviously where temporary files go, so having an SSD or other fast drive will benefit the I/O operations. Set that path to a fast storage partition.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Combined folder&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;all the output, including &lt;code&gt;txt&lt;/code&gt; directory will go here. Setting it to fast storage might speed up writing the txt files. If you set it to the same path while running multiple instances of MQ it will obviously override output. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Andromeda index&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;this is where index files from Andromeda search are. By default these are inside a &lt;code&gt;combined/search&lt;/code&gt; folder. Since these index files can get big, once generated by MQ they can be reused. Set this path if you want to reuse previously generated index files. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4732" LastActivityDate="2019-06-17T15:59:39.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8823" PostTypeId="2" ParentId="8817" CreationDate="2019-06-17T16:14:55.147" Score="4" Body="&lt;p&gt;You were very close, you just need to supply the &lt;code&gt;offset&lt;/code&gt; argument to &lt;code&gt;msaplot()&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So using your code block above with one extra argument in &lt;code&gt;msaplot()&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse)&#xA;library(ggtree) &#xA;library(seqinr)&#xA;tree = read.tree(&quot;din.newick&quot;) &#xA;msaplot(p=ggtree(tree), fasta = &quot;din.fasta&quot;)&#xA;msaplot(p=ggtree(tree) + geom_tiplab(align=TRUE), fasta = &quot;din.fasta&quot;, offset = 5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;yields&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3thEk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3thEk.png&quot; alt=&quot;msaplot with offset&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for supplying a minimal and reproducible example!&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2019-06-17T16:14:55.147" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8824" PostTypeId="1" AcceptedAnswerId="8825" CreationDate="2019-06-17T18:51:46.597" Score="4" ViewCount="210" Body="&lt;p&gt;I am trying to run some frequency based stats to identify selective sweep in my study system using Rpackage &quot;PopGenome&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To proceed with it I have split my genome data (whole genome sequencing) into 5MB chunks&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cregions &amp;lt;- character(31)&#xA;cregions[1] &amp;lt;- &quot;1-5000000&quot;&#xA;cregions[2] &amp;lt;- &quot;5000001-10000000&quot;&#xA;cregions[3] &amp;lt;- &quot;10000001-15000000&quot;&#xA;cregions[4] &amp;lt;- &quot;15000001-20000000&quot;&#xA;cregions[5] &amp;lt;- &quot;20000001-25000000&quot;&#xA;cregions[6] &amp;lt;- &quot;25000001-30000000&quot;&#xA;cregions[7] &amp;lt;- &quot;30000001-35000000&quot;&#xA;....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have a bunch of genes and I want to know each of my genes falls within the range of which intervals in the above list.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;these are my genes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chrom start stop gene_id&#xA;Chr01 107310787 107323633 gene2748&#xA;Chr01 113789595 113794851 gene2894&#xA;Chr01 159041389 159068535 gene4806&#xA;Chr02 2742629 2758060 gene4846 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'd like to get an out put like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gene2748  window_2&#xA;gene2894  window_3&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any idea how I can do it in R?&lt;/p&gt;&#xA;" OwnerUserId="1749" LastEditorUserId="96" LastEditDate="2019-07-02T18:03:02.137" LastActivityDate="2019-07-03T09:51:50.267" Title="Find intervals that genes fall within their range" Tags="|r|genomicranges|interval|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8825" PostTypeId="2" ParentId="8824" CreationDate="2019-06-17T19:04:33.640" Score="8" Body="&lt;p&gt;The GenomicRanges package is a great place to start, something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(GenomicRanges)&#xA;&#xA;# make a GenomicRanges set corresponding to the genes&#xA;genes.gr = makeGRangesFromDataFrame(df = genes, start.field = &quot;start&quot;, end.field = &quot;stop&quot;, seqnames.field = &quot;chrom&quot;)&#xA;&#xA;# turn your windows into a data frame and make a GenomicRanges set from that&#xA;windows = ...&#xA;windows.gr = makeGRangesFromDataFrame(df = windows ...&#xA;&#xA;# find overlaps between the two sets&#xA;olaps = findOverlaps(genes.gr, windows.gr)&#xA;&#xA;# output relevant fields&#xA;cat(paste(genes[queryHits(olaps),&quot;gene_id&quot;], windows[subjectHits(olaps),&quot;window_id&quot;]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3828" LastEditorUserId="3828" LastEditDate="2019-07-03T09:51:50.267" LastActivityDate="2019-07-03T09:51:50.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8826" PostTypeId="2" ParentId="8816" CreationDate="2019-06-17T22:40:34.813" Score="2" Body="&lt;p&gt;The problem is that blockwiseModules split your data into (probably 3) blocks, because the default maxBlockSize is 5000 and smaller than the number of genes in your data. If you have enough RAM (16GB is recommended for 12k genes, but 8GB just might be enough), rerun blockswiseModules again with maxBlockSize raised to above the number of your genes. Since you were able to run the function with maxBlockSize=8000, hoepfully you'll be able to do the single block calculation.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2019-06-17T22:40:34.813" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8827" PostTypeId="1" CreationDate="2019-06-18T13:41:41.583" Score="0" ViewCount="162" Body="&lt;p&gt;My undergraduate thesis is on the topic &quot;Ligand design from protein-protein interfaces&quot; and my PI has asked me to learn the basics of Structural bioinformatics over the summer so that I'll be able to start with my project as I begin my next semester(last week of July). Can someone please suggest me top resources to learn the basics of structural bioinformatics. Though I have done projects in Genomics, and Systems biology, this will be my first time with structural bioinformatics, so, I'll really appreciate some good leads. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="2883" LastActivityDate="2019-06-19T10:43:17.383" Title="What is the best way to start with Structural bioinformatics with zero prior experience?" Tags="|protein-structure|protein-protein-interaction|structural-biology|" AnswerCount="2" CommentCount="1" ClosedDate="2019-07-08T18:53:18.687" ContentLicense="CC BY-SA 4.0" />
  <row Id="8828" PostTypeId="2" ParentId="8827" CreationDate="2019-06-18T13:50:55.673" Score="1" Body="&lt;p&gt;A good starting point could be this book by Gu and Bourne:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.wiley.com/en-us/Structural+Bioinformatics%2C+2nd+Edition-p-9780470181058&quot; rel=&quot;nofollow noreferrer&quot;&gt;Structural Bioinformatics, 2nd Edition&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="678" LastActivityDate="2019-06-18T13:50:55.673" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8829" PostTypeId="1" CreationDate="2019-06-18T19:01:45.880" Score="0" ViewCount="50" Body="&lt;p&gt;I'm currently doing a python project where I need to write code that extracts the ATP binding site from a FASTA file using a regular expression.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried the following code but it does not work. It opens a new file but the file is empty. I don't know what's wrong.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio import SeqIO&#xA;import re&#xA;import os&#xA;path=&quot;C:/Users/nessr/Desktop/info projet&quot;&#xA;os.chdir(path)&#xA;records=list(SeqIO.parse(&quot;plasmo.fasta&quot;,&quot;fasta&quot;))&#xA;ATPM=&quot;[LIV]G[^P]G[^P][FYWMGSTNH][SGA][^PW][LIVCAT][^PD].[GSTACLIVMFY].{5,18}[LIVMFYWCSTAR][AIVP][LIVMFAGCKR]K&quot;&#xA;patern=re.compile(r&quot;(ATPM)&quot;)&#xA;&#xA;def atp_binding(records,patern):&#xA;&#xA;    for item in records:&#xA;        for m in patern.findall(str(item)):&#xA;            istart=m.start()&#xA;            iend=m.end()&#xA;&#xA;            iend=iend+50&#xA;&#xA;            print(istart,m.group())&#xA;&#xA;&#xA;file = open(&quot;output.txt&quot;,&quot;w&quot;)&#xA;&#xA;for seq_record in SeqIO.parse(records, &quot;fasta&quot;): &#xA;    file.write(atp_binding(seq_record.records,patern)+&quot;\n&quot;)&#xA;    file.write('-------------------\n')&#xA;file.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4738" LastEditorUserId="96" LastEditDate="2019-06-18T19:26:25.637" LastActivityDate="2019-07-20T21:01:40.173" Title="Function to find ATP binding sites" Tags="|fasta|phylogenetics|homework|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8830" PostTypeId="1" CreationDate="2019-06-18T19:46:21.573" Score="2" ViewCount="2832" Body="&lt;p&gt;Is there a way to import gene list into Seurat to define cell type? The default cell types in Seurat is not enough for our research. For example, we want to mark a subtype of B cells in Seurat, but seems like Seurat only have general B cell, I'm planning to download gene list from Genomic Cytometry, and then import the gene list into Seurat and then Seurat define cell types in the cluster based on the gene list I import. Does Seurat support this function?&lt;/p&gt;&#xA;" OwnerUserId="4739" LastEditorUserId="57" LastEditDate="2019-06-19T11:31:27.940" LastActivityDate="2019-06-30T19:01:27.807" Title="Import gene list to Seurat to define cell types" Tags="|r|seurat|single-cell|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8833" PostTypeId="2" ParentId="8827" CreationDate="2019-06-19T10:43:17.383" Score="2" Body="&lt;p&gt;As others have said, there is no short cut and you will need to spend some time with a good book.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would say there are 3 broad areas of understanding you have to have as a structural bioinformatician:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Solid understanding of the basic science. This includes molecular biology but also chemistry, which is often overlooked.&lt;/li&gt;&#xA;&lt;li&gt;Knowledge of the databases and data formats available in the field. You need to know where to go to get protein structures, protein sequences etc.&lt;/li&gt;&#xA;&lt;li&gt;Programming and algorithmic skills. You don't have to be a computer whiz but being effective in a high-level language (e.g. Python, R, Julia) and being able to parse data and carry out basic data analyses are essential.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you want to take it further and have a bit more time then I would consider implementing something yourself such as a protein secondary structure predictor. Getting this working is challenging and would show you have competency in the above three areas.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2019-06-19T10:43:17.383" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8834" PostTypeId="2" ParentId="8821" CreationDate="2019-06-19T13:28:18.383" Score="4" Body="&lt;p&gt;I would ignore peak calling for this and instead compute enrichment of ChIP/input for the genome (e.g., with deepTools or presumably homer) and then plot it for the genes of interest individually (e.g., using IGV or pyGenomeTracks) or as a group (e.g., with computeMatrix). If the peaks are obvious and you trust your peak calling then sure you can just use that instead, but given that that's prone to error it's usually better to simply use normalized signal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding actually looking at occupancy, I don't think that's useful. Occupancy isn't binary. If you're instead wanting to associate peaks with genes then using the nearest gene is common (it's not great, but it's not terrible either).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-06-19T13:28:18.383" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="8835" PostTypeId="1" AcceptedAnswerId="8836" CreationDate="2019-06-19T22:39:34.200" Score="1" ViewCount="64" Body="&lt;p&gt;am experimenting preprocessed Affymetrix microarrays expression data matrix (Affymetrix probe-sets in rows (32830 probesets), and RNA samples in columns (735 samples)) for my downstream analysis. Here is how the data looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dim(eset_HTA20)&#xA;[1] 32830   735&#xA;## load assayData&#xA;HTA20_rma &amp;lt;- load(&quot;HTA20_RMA.RData&quot;)&#xA;&#xA;&amp;gt; eset_HTA20[1:10,1:3]&#xA;             Tarca_001_P1A01 Tarca_003_P1A03 Tarca_004_P1A04&#xA;1_at                6.062215        6.125023        5.875502&#xA;10_at               3.796484        3.805305        3.450245&#xA;100_at              5.849338        6.191562        6.550525&#xA;1000_at             3.567779        3.452524        3.316134&#xA;10000_at            6.166815        5.678373        6.185059&#xA;100009613_at        4.443027        4.773199        4.393488&#xA;100009676_at        5.836522        6.143398        5.898364&#xA;10001_at            6.330018        5.601745        6.137984&#xA;10002_at            4.922339        4.711765        4.628124&#xA;10003_at            2.689344        2.771010        2.556756&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;objective&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am interested in to reduce the dimension of the dataset by tossing off low expressed in genes in the experiment. To do so, I need to quantify the variation of the gene that expressed. I checked the possible approach and using the coefficient of variation (cv) could be one of the approaches I could try. It is not intuitive for me how to quantify the variation of genes that expressed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone point me out how to lay out this procedure on the above expression data? Is there any Bioconductor package to do this kind of task? any idea to make this happen? thanks&lt;/p&gt;&#xA;" OwnerUserId="4729" LastActivityDate="2021-01-13T10:21:38.430" Title="Any way to quantify the variation of genes that expressed in Affymetrix expression data?" Tags="|r|microarray|gene-expression|limma|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8836" PostTypeId="2" ParentId="8835" CreationDate="2019-06-20T07:12:44.867" Score="2" Body="&lt;p&gt;You can use the following code to calculate the coefficient of variation:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# expr is your expression matrix.&#xA;SD &amp;lt;- apply(expr, 1, sd)&#xA;CV &amp;lt;- sqrt(exp(SD^2) - 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It might be implemented in some package but it is so brief that you can write it again yourself. Then you can filter out those that are below certain percentage of the distribution of CV (like 0.1)&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;expr[CV &amp;gt; quantile(CV, probs = 0.1), ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="48" LastActivityDate="2019-06-20T07:12:44.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8837" PostTypeId="2" ParentId="8813" CreationDate="2019-06-20T09:49:24.467" Score="2" Body="&lt;p&gt;I come from a protein background and this problem is analogous to protein torsion angle prediction, which in turn is a variant of protein secondary structure prediction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conventional ways to go here would be to use a sliding window of &lt;em&gt;x&lt;/em&gt; bases either side to make a prediction for each point, or use a CNN/LSTM-type model to run over the whole sequence and output angles.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are other technical concerns such as how to split the training and test sets, how to output the angles from the model (usually transformed by sin or cos), and whether to predict a single value or the probability of the value being in certain ranges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reference to protein tertiary prediction and CASP in another answer isn't directly relevant here unless you intend to build 3D models from your torsion angles, which you might very well do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some references on torsion angle prediction for proteins:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-017-1834-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;A recent review&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-018-2065-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;A recent state of the art method&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0022283699930917?via%3Dihub&quot; rel=&quot;nofollow noreferrer&quot;&gt;A classic paper describing some of the approaches&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4622" LastEditorUserId="4622" LastEditDate="2019-06-20T11:16:45.377" LastActivityDate="2019-06-20T11:16:45.377" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8838" PostTypeId="1" CreationDate="2019-06-20T10:11:55.597" Score="0" ViewCount="193" Body="&lt;p&gt;What is the most complete vcf file of population allele frequencies that can be built/downloaded from public datasets nowadays?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About 5 years or so ago, it used to be the latest release of the CSHL HapMap 12 populations that were part of the 1000 genomes project. These were:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CHB CHD MEX GIH TSI LWK ASW MKK HCB JPT CEU YRI&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the EnsEMBL project currently has these 12 populations available as a vcf here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;http://ftp.ensembl.org/pub/current_variation/vcf/homo_sapiens/&#xA;&#xA;CSHL-HAPMAP-HAPMAP-CHB.vcf.gz&#xA;CSHL-HAPMAP-HAPMAP-CHD.vcf.gz&#xA;CSHL-HAPMAP-HAPMAP-MEX.vcf.gz&#xA;CSHL-HAPMAP-HAPMAP-GIH.vcf.gz&#xA;CSHL-HAPMAP-HAPMAP-TSI.vcf.gz&#xA;CSHL-HAPMAP-HAPMAP-LWK.vcf.gz&#xA;CSHL-HAPMAP-HAPMAP-ASW.vcf.gz&#xA;CSHL-HAPMAP-HAPMAP-MKK.vcf.gz&#xA;CSHL-HAPMAP-HapMap-HCB.vcf.gz&#xA;CSHL-HAPMAP-HapMap-JPT.vcf.gz&#xA;CSHL-HAPMAP-HapMap-CEU.vcf.gz&#xA;CSHL-HAPMAP-HapMap-YRI.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Each SNP has an AF entry from which a multi-populations vcf with rsids, alleles and frequencies can be built, where the AFs are such as AF_CHB, AF_CHD, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The 1000 genomes project populations documentation describes more than these 12 populations, but I haven't seen equivalent population vcfs with AFs built from the individuals within the population for the remainder of these, apart from the original HapMap 12 marked with a &lt;code&gt;*&lt;/code&gt; below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;###Populations and codes&#xA;&#xA;*         CHB   Han Chinese             Han Chinese in Beijing, China&#xA;*         JPT   Japanese                Japanese in Tokyo, Japan&#xA;         CHS    Southern Han Chinese    Han Chinese South&#xA;         CDX    Dai Chinese             Chinese Dai in Xishuangbanna, China&#xA;         KHV    Kinh Vietnamese         Kinh in Ho Chi Minh City, Vietnam&#xA;*         CHD   Denver Chinese          Chinese in Denver, Colorado (pilot 3 only)&#xA;&#xA;         CEU    CEPH                    Utah residents (CEPH) with Northern and Western European ancestry &#xA;*         TSI   Tuscan                  Toscani in Italia &#xA;         GBR    British                 British in England and Scotland &#xA;         FIN    Finnish                 Finnish in Finland &#xA;         IBS    Spanish                 Iberian populations in Spain &#xA;&#xA;*         YRI   Yoruba                  Yoruba in Ibadan, Nigeria&#xA;*        LWK    Luhya                   Luhya in Webuye, Kenya&#xA;         GWD    Gambian                 Gambian in Western Division, The Gambia &#xA;         MSL    Mende                   Mende in Sierra Leone&#xA;         ESN    Esan                    Esan in Nigeria&#xA;&#xA;*         ASW   African-American SW     African Ancestry in Southwest US  &#xA;         ACB    African-Caribbean       African Caribbean in Barbados&#xA;         MXL    Mexican-American        Mexican Ancestry in Los Angeles, California&#xA;         PUR    Puerto Rican            Puerto Rican in Puerto Rico&#xA;         CLM    Colombian               Colombian in Medellin, Colombia&#xA;         PEL    Peruvian                Peruvian in Lima, Peru&#xA;&#xA;*         GIH   Gujarati                Gujarati Indian in Houston, TX&#xA;         PJL    Punjabi                 Punjabi in Lahore, Pakistan&#xA;         BEB    Bengali                 Bengali in Bangladesh&#xA;         STU    Sri Lankan              Sri Lankan Tamil in the UK&#xA;         ITU    Indian                  Indian Telugu in the UK&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I presume there is more public data nowadays that can be accessed to build a more complete vcf files of allele frequencies per population, with as many populations as possible to use as a reference dataset with new data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="180" LastEditDate="2019-06-24T07:33:45.717" LastActivityDate="2019-06-24T07:33:45.717" Title="what is the most complete vcf file of population allele frequencies that can be built/downloaded from public datasets?" Tags="|vcf|genomics|1000genomes|allele-frequency|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8839" PostTypeId="2" ParentId="8830" CreationDate="2019-06-20T11:45:30.217" Score="1" Body="&lt;p&gt;Seurat does not define cell types by name. It clusters and assigns each cell to a cluster, from 0 to X.&#xA;If your data has the cell type (e.g. B,T, Mast cells) it means that someone annotate the clusters so that they have a biological meaning.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can assign different names to the clusters by using the &lt;code&gt;AddMetaData&lt;/code&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sampleID  cellType&#xA;cell1     B&#xA;cell2     B&#xA;cell3     T&#xA;cell4     T&#xA;&#xA;newMeta &amp;lt;- oldMEta&#xA;newMeta&lt;span class=&quot;math-container&quot;&gt;$origType &amp;lt;- newMeta$&lt;/span&gt;cellType&#xA;newMeta$cellType &amp;lt;- c(&quot;B1&quot;,&quot;B&quot;,&quot;Treg&quot;,&quot;Tkill&quot;)&#xA;&#xA;object &amp;lt;- AddMetaData(object, newMeta)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The new metadata should look like &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sampleID  origType  cellType&#xA;cell1     B         B1&#xA;cell2     B         B&#xA;cell3     T         Treg&#xA;cell4     T         Tkill&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can then swap the labels using &lt;code&gt;SetAllIdent(object = test, id = &quot;cellType&quot;)&lt;/code&gt;. Your tSNE or uMAp, will now show the new cell types.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have a look at &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/4249/manually-define-clusters-in-seurat-and-determine-marker-genes?rq=1&quot;&gt;Manually define clusters in Seurat and determine marker genes&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3631" LastEditorUserId="3631" LastEditDate="2019-06-20T12:06:50.690" LastActivityDate="2019-06-20T12:06:50.690" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8841" PostTypeId="2" ParentId="8838" CreationDate="2019-06-20T18:52:35.620" Score="4" Body="&lt;p&gt;The best set I'm aware of can be downloaded from &lt;a href=&quot;https://gnomad.broadinstitute.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gnomad.broadinstitute.org&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="472" LastActivityDate="2019-06-20T18:52:35.620" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8843" PostTypeId="2" ParentId="8829" CreationDate="2019-06-20T20:28:28.813" Score="1" Body="&lt;p&gt;The problem in your code is that the function atp_binding is not returning any value, but printing it to the stdout.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should add a return statement with the result you want to save in your output file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, you should change how the resulting value is being saved. As the return value of your function is None, the file handler will write nothing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would use something similar to this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def atp_binding(records,patern):&#xA;    s = &quot;&quot;&#xA;    for item in records:&#xA;        for m in patern.findall(str(item)):&#xA;            istart=m.start()&#xA;            iend=m.end()&#xA;            iend=iend+50&#xA;            s+=&quot;{0}\t{1}\n&quot;.format(istart,m.group())&#xA;    return s&#xA;&#xA;&#xA;&#xA;file = open(&quot;output.txt&quot;,&quot;w&quot;)&#xA;&#xA;for seq_record in SeqIO.parse(records, &quot;fasta&quot;): &#xA;   file.write(atp_binding(seq_record.records,patern)+&quot;\n&quot;)&#xA;   file.write('-------------------\n')&#xA;file.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4747" LastActivityDate="2019-06-20T20:28:28.813" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8845" PostTypeId="1" CreationDate="2019-06-22T17:06:20.450" Score="0" ViewCount="153" Body="&lt;p&gt;I am working on a script on bash linux to get the CDS (coding sequence) of a gene using extractfeat by EMBOSS. It gives me the error: Warning: No sequences written to output file&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am unsure if my syntax is right, but this is what I have: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;extractfeat inputFile outputFile -type CDS&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4756" LastActivityDate="2024-02-27T22:03:45.447" Title="How to use extractfeat?" Tags="|sequence-analysis|bash|emboss|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8847" PostTypeId="2" ParentId="8845" CreationDate="2019-06-23T12:59:26.487" Score="0" Body="&lt;p&gt;Perhaps,removing introns (if it is a higher eukaryote)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;extractfeat tembl:x65921 -type CDS -join &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is difficult to know without knowing the gene. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW NCBI's Entrez &quot;efetch&quot; will do essentially the same thing, but its more complicated&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-06-23T16:07:47.540" LastActivityDate="2019-06-23T16:07:47.540" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8848" PostTypeId="1" CreationDate="2019-06-24T02:33:29.157" Score="1" ViewCount="55" Body="&lt;p&gt;I wonder if there's some mathematical details that outline what the algorithms does. &lt;/p&gt;&#xA;" OwnerUserId="4761" LastActivityDate="2019-06-24T02:33:29.157" Title="What are the mathematical assumptions and algorithmic procedure behind MACS2?" Tags="|statistics|peak-calling|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8849" PostTypeId="1" CreationDate="2019-06-24T11:38:30.950" Score="1" ViewCount="862" Body="&lt;p&gt;I have 2 biological replicates for chip-seq transcription factor data and I have applied macs2 peak calling for each replicate separately. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can i make use of the biological replicates to extract the most out the data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="4763" LastActivityDate="2019-07-25T20:07:59.223" Title="Biological replicates on Chip-seq Transcription factor data" Tags="|chip-seq|macs2|transcription-factors|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8851" PostTypeId="1" AcceptedAnswerId="8856" CreationDate="2019-06-24T18:26:24.350" Score="1" ViewCount="141" Body="&lt;p&gt;I am analyzing a family-based whole-genome sequence (WGS) dataset.  My identity-by-descent(IBD) analysis shows they share about 50% of their DNA. I think my pedigree structure may have gotten mixed up though and I want to see if one individual is the mother or the sister of a male individual.  Is there anyway to do this either with Plink or &lt;a href=&quot;http://people.virginia.edu/~wc9c/KING/manual.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;KING&lt;/a&gt;? &lt;/p&gt;&#xA;" OwnerUserId="1574" LastActivityDate="2019-06-25T23:44:48.063" Title="Sibling vs Parent/Child relationship detection using whole-genome sequence data" Tags="|wgs|plink|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8852" PostTypeId="2" ParentId="8849" CreationDate="2019-06-24T22:15:53.200" Score="2" Body="&lt;p&gt;You could first look at the degree of correlation between the two replicates - what proportion of peaks are shared between the two, versus peaks found in one sample only?  This will give an idea of how repeatable the analysis is, and how many peaks are variable between samples or due to artefacts of the method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When it comes to interpreting the biology behind the binding patterns, you are probably going to want to only interpret peaks which you find in both replicates.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-06-24T22:15:53.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8853" PostTypeId="1" AcceptedAnswerId="19391" CreationDate="2019-06-25T02:24:04.303" Score="2" ViewCount="130" Body="&lt;p&gt;I have Affymetrix gene level expression matrix (genes in the rows and sample ID on the columns), and I have annotation data of microarray experiment observation where sample ID in the rows and description identification on the columns.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;example data&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;here is Affymetrix gene level expression matix:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dim(exprs_mat)&#xA;[1] 112830   735&#xA;&amp;gt; exprs_mat[1:4, 1:3]&#xA;        Tarca_001_P1A01 Tarca_003_P1A03 Tarca_004_P1A04&#xA;1_at           6.062215        6.125023        5.875502&#xA;10_at          3.796484        3.805305        3.450245&#xA;100_at         5.849338        6.191562        6.550525&#xA;1000_at        3.567779        3.452524        3.316134&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;here is annotation data which contain experiment observation:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dim(ano)&#xA;[1] 735   6&#xA;&amp;gt; head(ano)&#xA;                       SampleID   GA Batch     Set Train Platform&#xA;Tarca_001_P1A01 Tarca_001_P1A01 11.0     1 PRB_HTA     1    HTA20&#xA;Tarca_013_P1B01 Tarca_013_P1B01 15.3     1 PRB_HTA     1    HTA20&#xA;Tarca_025_P1C01 Tarca_025_P1C01 21.7     1 PRB_HTA     1    HTA20&#xA;Tarca_037_P1D01 Tarca_037_P1D01 26.7     1 PRB_HTA     1    HTA20&#xA;Tarca_049_P1E01 Tarca_049_P1E01 31.3     1 PRB_HTA     1    HTA20&#xA;Tarca_061_P1F01 Tarca_061_P1F01 32.1     1 PRB_HTA     1    HTA20&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I intend to see how the genes in each sample are correlated with GA value of corresponding samples in the annotation data. How can I get my expected correlation matrix and filter out the genes by correlation value? any idea to make this happen correctly?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;my attempt&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(limma)&#xA;fit &amp;lt;- limma::lmFit(exprs_mat, design = model.matrix( ~ 0 + t(ano$GA))&#xA;fit &amp;lt;- eBayes(fit)&#xA;topTable(fit, coef=2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but this attempt gave me dimension error and I still didn't get highly correlated genes from &lt;code&gt;exprs_mat&lt;/code&gt; with each &lt;code&gt;ano$GA&lt;/code&gt;in &lt;code&gt;ano&lt;/code&gt;data. I just removed &lt;code&gt;0+&lt;/code&gt; and ran the code, then I got this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;row dimension of design doesn't match column dimension of data object&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am pretty sure I did &lt;code&gt;df[!is.na(df$GA), ]&lt;/code&gt;, removed &lt;code&gt;NA&lt;/code&gt; in both &lt;code&gt;exprs_mat&lt;/code&gt; and &lt;code&gt;ano&lt;/code&gt; metadata already, but the same error still persists. I also tried &lt;code&gt;df[complete.cases(df),]&lt;/code&gt; to make sure I removed all &lt;code&gt;NA&lt;/code&gt; values in my data.&lt;/p&gt;&#xA;&lt;p&gt;How can I get those highly correlated genes correctly? any possible way to make this happen in R? any thought?&lt;/p&gt;&#xA;&lt;p&gt;How can I cut down &lt;code&gt;exprs_mat&lt;/code&gt; with the shortlist of genes? any idea?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;desired output&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I intend to filter out genes in the gene expression matrix &lt;code&gt;expr_mat&lt;/code&gt; by using correlation value (only keep highly correlated genes), expected sub expression matrix should have same data structure as &lt;code&gt;expr_mat&lt;/code&gt;. any way to get this done?&lt;/p&gt;&#xA;" OwnerUserId="4729" LastEditorUserId="73" LastEditDate="2022-06-21T10:12:20.080" LastActivityDate="2022-07-14T03:16:12.347" Title="Any way to filter out highly correlated genes with limma linear model?" Tags="|r|microarray|gene-expression|limma|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8854" PostTypeId="1" AcceptedAnswerId="8892" CreationDate="2019-06-25T07:43:11.377" Score="0" ViewCount="518" Body="&lt;p&gt;What's the recommended way to create population-level Allele Frequency variant entries from a set of individuals (vcfs?) in the 1000 genomes (or similar) project?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, given the IBS individuals in the 1000 Genomes project:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.internationalgenome.org/data-portal/population/IBS&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.internationalgenome.org/data-portal/population/IBS&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to create an IBS &lt;code&gt;.vcf.gz&lt;/code&gt; that contains an IBS population-level AF entry for each of the variants present.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas on how to achieve this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: this is related to my earlier question on how to create the most complete file for population allele frequencies, here:&#xA;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/8838/what-is-the-most-complete-vcf-file-of-population-allele-frequencies-that-can-be&quot;&gt;what is the most complete vcf file of population allele frequencies that can be built/downloaded from public datasets?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen that there is an Allele Frequency Calculator tool in EnsEMBL, which is at least available with GRCh37:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.internationalgenome.org/allele-frequency-calculator-documentation&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.internationalgenome.org/allele-frequency-calculator-documentation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://grch37.ensembl.org/Homo_sapiens/Tools/AlleleFrequency?db=core&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://grch37.ensembl.org/Homo_sapiens/Tools/AlleleFrequency?db=core&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also instructions on how to do this genome-wide using vcftools:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;vcf-subset -c ~/1000g/IBS.samples.list ~/1000g/ALL.chr1.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz | fill-an-ac | bgzip -c &amp;gt; ~/1000g/IBS.chr1.20130502.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This seems to be generating correct files, and the AF can be calculated with a tool that combines &lt;code&gt;AC&lt;/code&gt; and &lt;code&gt;AN&lt;/code&gt; values, such as it is described here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/123468/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/123468/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any other ideas / warnings / pitfalls of the approach?&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="180" LastEditDate="2019-06-25T10:00:15.860" LastActivityDate="2019-07-02T00:57:28.837" Title="creating population AF from set of individuals in 1000 genomes population?" Tags="|vcf|genomics|allele-frequency|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8856" PostTypeId="2" ParentId="8851" CreationDate="2019-06-25T23:44:48.063" Score="1" Body="&lt;p&gt;With at least three samples, you can determine this by looking at the IBS0 output column (from either plink2 --make-king-table or KING): if the relationship in question has a very low IBS0 value compared to the other pairs, that's parent-child.  You can try to do the same thing with only two samples, but with no other IBS0 values to compare against it's easier to misjudge.&lt;/p&gt;&#xA;" OwnerUserId="472" LastActivityDate="2019-06-25T23:44:48.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8857" PostTypeId="1" CreationDate="2019-06-26T03:45:05.013" Score="2" ViewCount="2178" Body="&lt;p&gt;I used Seurat 2.4 on our scRNA dataset to obtain the following tSNE plot.&lt;a href=&quot;https://i.stack.imgur.com/ZhB3T.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZhB3T.png&quot; alt=&quot;very good cluster definition&quot;&gt;&lt;/a&gt;I was able to successfully extract cell IDs from the different clusters, and generate gene expression profiles. The analysis, and the biology makes sense. I have been however stuck in trying to highlight specific cells we are interested in using the Cell IDs (barcodes). For e.g, in this plot I am hoping to highlight specific cells (~ 120 of them) within Cluster 5 (blue cluster on the top). It can be either in featureplot mode or in this plot itself by an overlay, it doesn't matter. All I have to show are the 120 cells within the cluster. &#xA;For eg. if cluster 5 has cells atag-1 atgc-2 atat-3, cacc-4 cat-5... i want to recreate this plot using atag-1 atgc-2, cacc-4&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried using do.identify = TRUE but that is a visual selection and can select regions, but is not useful to select specific cells. I was thinking if there is a way to subset and replace cluster 5 to include just the cells (~120 odd out of the total 200) but was unable to crack it. I tried by using cells.use but then realized that doesn't work either.Any ideas will be greatly appreciated, to get me out of this rut.&#xA;Thank you&lt;/p&gt;&#xA;" OwnerUserId="4774" LastEditorUserId="4774" LastEditDate="2019-06-26T15:21:04.730" LastActivityDate="2019-06-28T22:10:20.403" Title="How to highlight specific cells in Seurat 2.4" Tags="|scrnaseq|seurat|visualization|single-cell|transcriptome|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8858" PostTypeId="1" CreationDate="2019-06-26T16:22:47.557" Score="3" ViewCount="361" Body="&lt;p&gt;I'm trying to run Repeatmasker in Linux on the command line with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./RepeatMasker -species human -alu -gff -dir /mnt/lustre/users/Analysis/RepeatMaskerOutput /mnt/lustre/users/Software/STAR/bin/Linux_x86_64/HRG_Newest_and_Annotation/GCA_000001405.28_GRCh38.p13_genomic.fna&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but I am getting this error:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;perl: warning: Setting locale failed.&#xA;perl: warning: Please check that your locale settings:&#xA;        LANGUAGE = &amp;quot;en_GB.utf8&amp;quot;,&#xA;        LC_ALL = &amp;quot;&amp;quot;,&#xA;        LC_CTYPE = &amp;quot;en_GB.utf8&amp;quot;,&#xA;        LANG = &amp;quot;en_GB.utf8&amp;quot;&#xA;    are supported and installed on your system.&#xA;perl: warning: Falling back to the standard locale (&amp;quot;C&amp;quot;).&#xA;RepeatMasker version open-4.0.9&#xA;TRF program not configured! This version of RepeatMasker requires a&#xA;local installatoin of TRF.  Please visit:&#xA;               http://tandem.bu.edu/trf/trf.html&#xA;to obtain the current version.  Once installed please re-run the&#xA;RepeatMasker configure script to setup RepeatMasker to use the new&#xA;installation. at ./RepeatMasker line 442.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have downloaded rmBlast and the latest TRF file. I have put the TRF file in my &lt;code&gt;Users/username/bin&lt;/code&gt; PATH folder and used &lt;code&gt;chmod a+x&lt;/code&gt; (and &lt;code&gt;chmod 755&lt;/code&gt;) on it. I then changed my &lt;code&gt;LD_LIBRARY_PATH&lt;/code&gt; using &lt;code&gt;export&lt;/code&gt; to my glibc directory.&lt;/p&gt;&#xA;&lt;p&gt;After this I used &lt;code&gt;perl ./configure&lt;/code&gt; and directed the program to the PATH folder where the TRF file was. It gave a series of the same error messages as above but was 'successful' as shown here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;[user@nodec23() RepeatMasker]$ perl ./configure&#xA;perl: warning: Setting locale failed.&#xA;perl: warning: Please check that your locale settings:&#xA;        LANGUAGE = (unset),&#xA;        LC_ALL = (unset),&#xA;        LANG = &amp;quot;en_GB&amp;quot;&#xA;    are supported and installed on your system.&#xA;perl: warning: Falling back to the standard locale (&amp;quot;C&amp;quot;).&#xA;RepeatMasker Configuration Program&#xA;&#xA;**TRF PROGRAM**&#xA;&#xA;  This is the full path to the TRF program.&#xA; This is now used by RepeatMasker to mask simple repeats.&#xA;&#xA;Enter path  [ /mnt/lustre/users/Software/RepeatMasker/RepeatMasker/TRF.pm ]&#xA;&#xA;Add a Search Engine:&#xA;   1. Crossmatch - Phrap             [ Configured ]&#xA;   2. RMBlast - Improved NCBI Blast  [ Configured, Default ]&#xA;   3. HMMER3.1 &amp;amp; DFAM                [ Un-configured ]&#xA;   4. ABBlast/WUBlast                [ Un-configured ]&#xA;   5. Done&#xA;&#xA;Enter Selection: 5&#xA; -- Setting perl interpreter...&#xA;perl: warning: Setting locale failed.&#xA;perl: warning: Please check that your locale settings:&#xA;        LANGUAGE = (unset),&#xA;        LC_ALL = (unset),&#xA;        LANG = &amp;quot;en_GB&amp;quot;&#xA;    are supported and installed on your system.&#xA;perl: warning: Falling back to the standard locale (&amp;quot;C&amp;quot;).&#xA;perl: warning: Setting locale failed.&#xA;perl: warning: Please check that your locale settings:&#xA;        LANGUAGE = (unset),&#xA;        LC_ALL = (unset),&#xA;        LANG = &amp;quot;en_GB&amp;quot;&#xA;    are supported and installed on your system.&#xA;perl: warning: Falling back to the standard locale (&amp;quot;C&amp;quot;).&#xA;perl: warning: Setting locale failed.&#xA;perl: warning: Please check that your locale settings:&#xA;        LANGUAGE = (unset),&#xA;        LC_ALL = (unset),&#xA;        LANG = &amp;quot;en_GB&amp;quot;&#xA;    are supported and installed on your system.&#xA;perl: warning: Falling back to the standard locale (&amp;quot;C&amp;quot;).&#xA;perl: warning: Setting locale failed.&#xA;perl: warning: Please check that your locale settings:&#xA;        LANGUAGE = (unset),&#xA;        LC_ALL = (unset),&#xA;        LANG = &amp;quot;en_GB&amp;quot;&#xA;    are supported and installed on your system.&#xA;perl: warning: Falling back to the standard locale (&amp;quot;C&amp;quot;).&#xA;perl: warning: Setting locale failed.&#xA;perl: warning: Please check that your locale settings:&#xA;        LANGUAGE = (unset),&#xA;        LC_ALL = (unset),&#xA;        LANG = &amp;quot;en_GB&amp;quot;&#xA;    are supported and installed on your system.&#xA;perl: warning: Falling back to the standard locale (&amp;quot;C&amp;quot;).&#xA;perl: warning: Setting locale failed.&#xA;perl: warning: Please check that your locale settings:&#xA;        LANGUAGE = (unset),&#xA;        LC_ALL = (unset),&#xA;        LANG = &amp;quot;en_GB&amp;quot;&#xA;    are supported and installed on your system.&#xA;perl: warning: Falling back to the standard locale (&amp;quot;C&amp;quot;).&#xA;perl: warning: Setting locale failed.&#xA;perl: warning: Please check that your locale settings:&#xA;        LANGUAGE = (unset),&#xA;        LC_ALL = (unset),&#xA;        LANG = &amp;quot;en_GB&amp;quot;&#xA;    are supported and installed on your system.&#xA;perl: warning: Falling back to the standard locale (&amp;quot;C&amp;quot;).&#xA;perl: warning: Setting locale failed.&#xA;perl: warning: Please check that your locale settings:&#xA;        LANGUAGE = (unset),&#xA;        LC_ALL = (unset),&#xA;        LANG = &amp;quot;en_GB&amp;quot;&#xA;    are supported and installed on your system.&#xA;perl: warning: Falling back to the standard locale (&amp;quot;C&amp;quot;).&#xA;perl: warning: Setting locale failed.&#xA;perl: warning: Please check that your locale settings:&#xA;        LANGUAGE = (unset),&#xA;        LC_ALL = (unset),&#xA;        LANG = &amp;quot;en_GB&amp;quot;&#xA;    are supported and installed on your system.&#xA;perl: warning: Falling back to the standard locale (&amp;quot;C&amp;quot;).&#xA;&#xA;Congratulations!  RepeatMasker is now ready to use.&#xA;The program is installed with a the following repeat libraries:&#xA;  Dfam database version Dfam_3.0&#xA;  RepeatMasker Combined Database: Dfam_3.0&#xA;Further documentation on the program may be found here:&#xA;  /mnt/lustre/users/Software/RepeatMasker/RepeatMasker/repeatmasker.help&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm at a loss of what to do next. I have googled and tried changing each of the locale settings with export to the following:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;en_GB&lt;/li&gt;&#xA;&lt;li&gt;en_US.utf8&lt;/li&gt;&#xA;&lt;li&gt;en_GB:2000&lt;/li&gt;&#xA;&lt;li&gt;en_GB.utf8&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;but none have changed the error.&lt;/p&gt;&#xA;&lt;p&gt;I am running a Windows 10 machine and using PuTTY to connect to an HPC which uses a Linux environment.&lt;/p&gt;&#xA;&lt;p&gt;Locale settings are as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;locale: Cannot set LC_CTYPE to default locale: No such file or directory&#xA;locale: Cannot set LC_MESSAGES to default locale: No such file or directory&#xA;locale: Cannot set LC_ALL to default locale: No such file or directory&#xA;LANG=en_GB&#xA;LC_CTYPE=&amp;quot;en_GB&amp;quot;&#xA;LC_NUMERIC=&amp;quot;en_GB&amp;quot;&#xA;LC_TIME=&amp;quot;en_GB&amp;quot;&#xA;LC_COLLATE=&amp;quot;en_GB&amp;quot;&#xA;LC_MONETARY=&amp;quot;en_GB&amp;quot;&#xA;LC_MESSAGES=&amp;quot;en_GB&amp;quot;&#xA;LC_PAPER=&amp;quot;en_GB&amp;quot;&#xA;LC_NAME=&amp;quot;en_GB&amp;quot;&#xA;LC_ADDRESS=&amp;quot;en_GB&amp;quot;&#xA;LC_TELEPHONE=&amp;quot;en_GB&amp;quot;&#xA;LC_MEASUREMENT=&amp;quot;en_GB&amp;quot;&#xA;LC_IDENTIFICATION=&amp;quot;en_GB&amp;quot;&#xA;LC_ALL=&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Also please note that I do not have administrator access.&lt;/p&gt;&#xA;&lt;p&gt;Typing 'perl -v' gives the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl: warning: Setting locale failed.&#xA;perl: warning: Please check that your locale settings:&#xA;        LANGUAGE = (unset),&#xA;        LC_ALL = (unset),&#xA;        LANG = &amp;quot;en_GB&amp;quot;&#xA;    are supported and installed on your system.&#xA;perl: warning: Falling back to the standard locale (&amp;quot;C&amp;quot;).&#xA;&#xA;This is perl, v5.10.1 (*) built for x86_64-linux-thread-multi&#xA;&#xA;Copyright 1987-2009, Larry Wall&#xA;&#xA;Perl may be copied only under the terms of either the Artistic License or the&#xA;GNU General Public License, which may be found in the Perl 5 source kit.&#xA;&#xA;Complete documentation for Perl, including FAQ lists, should be found on&#xA;this system using &amp;quot;man perl&amp;quot; or &amp;quot;perldoc perl&amp;quot;.  If you have access to the&#xA;Internet, point your browser at http://www.perl.org/, the Perl Home Page.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Typing &amp;quot;LC_ALL=C perl ./configure&amp;quot; and going throught the configurations leads to the same outcome but without the perl errors:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Enter Selection: 5&#xA; -- Setting perl interpreter...&#xA;&#xA;Congratulations!  RepeatMasker is now ready to use.&#xA;The program is installed with a the following repeat libraries:&#xA;  Dfam database version Dfam_3.0&#xA;  RepeatMasker Combined Database: Dfam_3.0&#xA;Further documentation on the program may be found here:&#xA;  /mnt/lustre/users/k1894998/Software/RepeatMasker/RepeatMasker/repeatmasker.help&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, the error remains when trying to run Repeatmasker.&lt;/p&gt;&#xA;" OwnerUserId="4780" LastEditorUserId="3848" LastEditDate="2020-11-13T09:26:05.470" LastActivityDate="2024-03-27T16:03:30.420" Title="&quot;perl: warning: Setting locale failed.&quot; in RepeatMasker" Tags="|rna-seq|perl|linux|repeatmasker|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8859" PostTypeId="1" CreationDate="2019-06-26T19:40:46.283" Score="1" ViewCount="433" Body="&lt;p&gt;I am reading through the paper &quot;A scaling normalization method for differential expression analysis of RNA-seq data&quot; by Mark D Robinson, Alicia Oshlack, available &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/gb-2010-11-3-r25&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this paper they introduce a normalization method called &quot;trimmed mean of M-values&quot; (TMM), which is apparently a normalization used in the edgeR package. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is confusing to me is that on page 3 they define the value &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$M_g = \log_2\left(\frac{Y_{gk}/N_k}{Y_{gk'}/N_k'}\right)$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is an experiment to be normalized and &lt;span class=&quot;math-container&quot;&gt;$k'$&lt;/span&gt; is a reference experiment. A few pages down on page 7, however, they define&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$M_{gk}^r = \frac{\log_2\left(Y_{gk}/N_k\right)}{\log_2\left(Y_{gr}/N_r'&#xA;\right)}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is an experiment to be normalized and &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is a reference experiment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are these two formulae intended to represent the same quantity? If they are supposed to be the same, which one has the correct order of logarithm/division? If they are not supposed to be the same, can anyone explain the difference? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="4782" LastActivityDate="2019-06-26T19:40:46.283" Title="What is the formula for Mg values in TMM normalization for RNA Seq data?" Tags="|rna-seq|normalization|edger|tmm|" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8860" PostTypeId="1" AcceptedAnswerId="8861" CreationDate="2019-06-26T21:43:24.460" Score="1" ViewCount="96" Body="&lt;p&gt;File1:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ENST00000000233 ARF5&#xA;ENST00000000412 M6PR&#xA;ENST00000001008 FKBP4&#xA;ENST00000001146 CYP26B1&#xA;ENST00000002501 DBNDD1&#xA;ENST00000002596 HS3ST1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;File 2:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ENST00000000412 0.779142&#xA;ENST00000001008 0.738143&#xA;ENST00000002501 0.715315&#xA;ENST00000002596 0.713664&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Desired Output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ENST00000000412 0.779142    M6PR&#xA;ENST00000001008 0.738143    FKBP4&#xA;ENST00000002501 0.715315    DBNDD1&#xA;ENST00000002596 0.713664    HS3ST1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4783" LastActivityDate="2019-06-27T18:21:10.383" Title="How to extract values from second file on the basis common first column?" Tags="|bash|awk|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8861" PostTypeId="2" ParentId="8860" CreationDate="2019-06-26T22:54:06.377" Score="6" Body="&lt;p&gt;I like to use the &lt;code&gt;join&lt;/code&gt; command for this. For your example above, you can simply use the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ join file2 file1&#xA;ENST00000000412 0.779142 M6PR&#xA;ENST00000001008 0.738143 FKBP4&#xA;ENST00000002501 0.715315 DBNDD1&#xA;ENST00000002596 0.713664 HS3ST1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This assumes your files are sorted. If they aren't sorted, you can use input redirection if your shell supports it (bash and other modern shells do):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;join &amp;lt;(sort -k 1b,1 file2) &amp;lt;(sort -k 1b,1 file1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If your shell doesn't support &lt;code&gt;&amp;lt;()&lt;/code&gt;, sort the files first:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sort file1 &amp;gt; file1.sorted; sort file2 &amp;gt; file2.sorted&#xA;join &amp;lt;(sort -k 1b,1 file2.sorted) &amp;lt;(sort -k 1b,1 file1.sorted)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="92" LastEditorUserId="298" LastEditDate="2019-06-27T15:48:46.800" LastActivityDate="2019-06-27T15:48:46.800" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8862" PostTypeId="1" AcceptedAnswerId="8876" CreationDate="2019-06-27T01:28:55.950" Score="4" ViewCount="350" Body="&lt;p&gt;I have a folder with 150 pairs of fastq files from an illumina sequence run. I need to precess them with mixcr, how can I do this in Bash with a single command?&lt;/p&gt;&#xA;" OwnerUserId="2621" LastEditorUserId="3694" LastEditDate="2022-05-31T15:17:25.923" LastActivityDate="2023-01-26T16:27:20.327" Title="Mixcr batch processing" Tags="|ngs|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8863" PostTypeId="2" ParentId="3581" CreationDate="2019-06-27T03:11:00.333" Score="1" Body="&lt;p&gt;Have you tried seurat3? Here is the reference:&#xA;&lt;a href=&quot;https://satijalab.org/seurat/v3.0/pancreas_integration_label_transfer.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/v3.0/pancreas_integration_label_transfer.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4784" LastActivityDate="2019-06-27T03:11:00.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8865" PostTypeId="2" ParentId="3146" CreationDate="2019-06-27T03:30:13.147" Score="0" Body="&lt;p&gt;seurat3 may help. You can compare the marker genes from the 2 cell types or treat one cell type as a control. Please see the example analysis from: &lt;a href=&quot;https://satijalab.org/seurat/v3.0/immune_alignment.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/v3.0/immune_alignment.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4784" LastActivityDate="2019-06-27T03:30:13.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8866" PostTypeId="2" ParentId="2885" CreationDate="2019-06-27T04:39:45.553" Score="1" Body="&lt;p&gt;You can express the annotations of the genome with a dot plot, x and y represent the two annotations along the genome. The you can summary the result with Spearman correlation coefficient(s) based on whole set of annotations or particular gene families.&lt;/p&gt;&#xA;" OwnerUserId="4784" LastActivityDate="2019-06-27T04:39:45.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8868" PostTypeId="1" CreationDate="2019-06-27T16:33:36.190" Score="2" ViewCount="734" Body="&lt;p&gt;I am running &lt;code&gt;bcftools merge&lt;/code&gt; to create a single &lt;code&gt;.vcf.gz&lt;/code&gt; file out of a collection of chromosomal &lt;code&gt;.vcf.gz&lt;/code&gt; files from the 1000g project.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The merge process has been running for a couple of days, and I want to estimate how much longer it will take. The file is a growing &lt;code&gt;.vcf.gz&lt;/code&gt; that is being &lt;code&gt;bgzip -c&lt;/code&gt; compressed from a Linux pipe:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools merge [...] | some processing and filtering | bgzip -c &amp;gt; final.file.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I know at which point is the file currently, to predict how much longer it will take to complete the merge?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to do something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tail --with-some-magic final.file.vcf.gz | bgzip -d -c&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So that I can read at which chromosome is currently on?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to guesstimate the final size of the merge file and compare from the sum of sizes of the original files? E.g. the sum of compressed sizes originally (before filtering and munging) is 20G:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;find &lt;span class=&quot;math-container&quot;&gt;$PWD -name &quot;ALL.chr*.vcf.gz&quot; | xargs ls -l | awk '{print $&lt;/span&gt;5}' | csvtk summary -H -f 1:sum | numfmt --to-si&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This gives 20G, but the merge file is at 27G and still running.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2019-07-02T00:44:18.747" Title="extracting last few lines of a growing bgzip compressed .vcf.gz file" Tags="|vcf|vcftools|bcftools|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8869" PostTypeId="2" ParentId="8860" CreationDate="2019-06-27T18:04:57.560" Score="1" Body="&lt;p&gt;Using awk, this can be done in the following way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk 'NR==FNR{a[&lt;span class=&quot;math-container&quot;&gt;$1]=$&lt;/span&gt;2; next}&lt;span class=&quot;math-container&quot;&gt;$1 in a{print $1, $2, a[$&lt;/span&gt;1]}' file1.txt file2.txt &amp;gt; same.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Explanation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NR==FNR (along with the next command) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Will go through the first file only and create an associative array of of the geneIDs corresponding with the geneNames. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then if the 1st column in the 2nd file is present in the array, a, it will print both columns from the file plus the geneName.&lt;/p&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="4001" LastEditDate="2019-06-27T18:21:10.383" LastActivityDate="2019-06-27T18:21:10.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8873" PostTypeId="2" ParentId="2885" CreationDate="2019-06-28T11:27:48.117" Score="1" Body="&lt;p&gt;I would start with metaplots for a visual/exploratory analysis to guide your statistical testing.  Cut the genome into short segments e.g. centred on the centre of each nucleosome peak, overlay all the segments on top of each other, and plot the abundance of secondary structure features at each base count as you step away from the centres.  This should give an easy view on whether there is any clear structure in the relative positions of the different objects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do the plots both ways round, and try different subsets of the objects (e.g. ones overlapping genes or not, different stringency thresholds for peak calling).  This should hopefully give you some insights on what is worth testing for.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-06-28T11:27:48.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8874" PostTypeId="2" ParentId="8868" CreationDate="2019-06-28T13:26:31.247" Score="5" Body="&lt;p&gt;27GB is not huge. You can &lt;code&gt;zcat&lt;/code&gt; the file and see the last few lines. It will take ~10 minutes. It is theoretically possible to peek a bgzf file, but no such tools are available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I am running bcftools merge to create a single .vcf.gz file out of a collection of chromosomal .vcf.gz files from the 1000g project.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Probably you should use &lt;code&gt;bcftools concat&lt;/code&gt;, not &lt;code&gt;bcftools merge&lt;/code&gt;. &lt;code&gt;merge&lt;/code&gt; combines VCFs from different samples. &lt;code&gt;concat&lt;/code&gt; concatenates VCFs for individual chromosomes; each input VCF contains the same set of samples.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2019-06-28T19:05:02.320" LastActivityDate="2019-06-28T19:05:02.320" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8876" PostTypeId="2" ParentId="8862" CreationDate="2019-06-28T18:12:27.167" Score="4" Body="&lt;p&gt;Here's how I'd approach this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for READ1 in /Documents/Data/Input/*R1*&#xA;do&#xA;    READ2=${READ1/R1/R2}&#xA;    mixcr analyze amplicon --s hs \&#xA;        --starting-material dna \&#xA;        --5-end v-primers \&#xA;        --3-end j-primers \&#xA;        --adapters no-adapters \&#xA;        --receptor-type trb \&#xA;        --only-productive &quot;$READ1&quot; &quot;$READ2&quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically, loop over all the R1 FASTQ files in a directory, and calculate the R2 names as needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is using BASH string manipulation, and assumes that substituting &lt;code&gt;R2&lt;/code&gt; for &lt;code&gt;R1&lt;/code&gt; will correctly calculate the R2 filename from R1. It also assumes that all the FASTQ files are in a single directory.&lt;/p&gt;&#xA;" OwnerUserId="92" LastEditorUserId="92" LastEditDate="2019-06-29T01:07:55.447" LastActivityDate="2019-06-29T01:07:55.447" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8877" PostTypeId="2" ParentId="8857" CreationDate="2019-06-28T22:10:20.403" Score="1" Body="&lt;p&gt;You can &quot;highlight&quot; any cells you'd like by creating a new column (or modifying an existing column) in the meta data slot within the Seurat object. From there you can modify the contents of that vector to your liking and plot the results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not clear if you've been successful in obtaining the barcodes for the subpopulation to begin with, because that's a separate issue all together, and you need to clarify what attribute you are trying to subset the cells on because there are a number of ways to accomplish that. If you've already obtained the barcodes then I'll describe how you would plot the results below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume you'd like to keep the original cluster IDs for all other cells. We'll start off by duplicating the cluster ID column (this isn't necessary, but it's good practice to preserve the original IDs).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n &amp;lt;- seurat_obj@meta.data # for convenience &#xA;n&lt;span class=&quot;math-container&quot;&gt;$new_column &amp;lt;- n$&lt;/span&gt;res.1.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now you need to modify the ids for cluster 5 to break them up into 2 populations. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;n[rownames(n) %in% my_barcodes, &quot;new_column&quot;] &amp;lt;- &quot;5b&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can plot the new column using &lt;a href=&quot;https://www.rdocumentation.org/packages/Seurat/versions/2.3.4/topics/DimPlot&quot; rel=&quot;nofollow noreferrer&quot;&gt;DimPlot&lt;/a&gt; with the argument &lt;code&gt;group.by = &quot;new_column&quot;&lt;/code&gt;, which will make the subpopulation stand out from the rest of the cells in cluster 5.&lt;/p&gt;&#xA;" OwnerUserId="1645" LastActivityDate="2019-06-28T22:10:20.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8878" PostTypeId="1" CreationDate="2019-06-28T22:13:19.557" Score="1" ViewCount="18" Body="&lt;p&gt;I've searched around and found about 80 specific secondary metabolites for lichen, but I can't seem to find anymore. They were mostly from research papers. I read that there are more than 800 secondary metabolites of lichen. Is there some website that has all of them? I tried using HMDB but it yielded nothing. &lt;/p&gt;&#xA;" OwnerUserId="4796" LastActivityDate="2019-06-28T22:13:19.557" Title="Where to find secondary metabolites in lichen (and herbal medicine)?" Tags="|database|" AnswerCount="0" CommentCount="3" ClosedDate="2019-07-22T16:26:15.407" ContentLicense="CC BY-SA 4.0" />
  <row Id="8880" PostTypeId="1" CreationDate="2019-06-29T14:21:25.407" Score="4" ViewCount="2732" Body="&lt;p&gt;I would like to run an analysis on Nanopore data. I'm trying to download sample data from &lt;a href=&quot;https://github.com/nanopore-wgs-consortium/chm13&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nanopore-wgs-consortium/chm13&lt;/a&gt;. I would like to run minimap2 on the FASTQ files, resulting in a single BAM file.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9q15C.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9q15C.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Q: What do the file names mean? What is &amp;quot;S13&amp;quot;? What is &amp;quot;L002_l1&amp;quot;? What is &amp;quot;L002_R1&amp;quot;? Am I supposed to download them and combine them into a single FASTQ file, and then align them?&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="73" LastEditDate="2021-07-05T04:16:51.717" LastActivityDate="2021-07-05T04:19:37.633" Title="What do the FASTQ file names mean here?" Tags="|fastq|genomics|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8882" PostTypeId="2" ParentId="8880" CreationDate="2019-06-29T22:48:32.817" Score="5" Body="&lt;p&gt;&lt;em&gt;What is &quot;S13&quot;?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;S# is most likely the sample number. If so, you do not want to combine these.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;What is &quot;L002_l1&quot;?&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;L00# is the lane number (of the 8 Illumina flow cell lanes). 'I1_' (not 'L') are index files and are not needed (see also &lt;a href=&quot;https://www.biostars.org/p/374581/&quot; rel=&quot;noreferrer&quot;&gt;https://www.biostars.org/p/374581/&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;What is &quot;L002_R1&quot;?&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;R1 is one end of paired-end sequencing, R2 is the other end. These filepairs must have the same number of lines. Use an aligner that utilizes both.&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2019-06-29T22:48:32.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8883" PostTypeId="2" ParentId="8830" CreationDate="2019-06-29T23:08:10.763" Score="0" Body="&lt;p&gt;Maybe you can try &lt;code&gt;Seurat::AddModuleScore()&lt;/code&gt;, then &lt;code&gt;FeaturePlot()&lt;/code&gt; and see if some of your B cells are different. After plotting this on &lt;code&gt;GenePlot()&lt;/code&gt;, perhaps you can set a cutoff, then assign identities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, use your B cell gene list in &lt;code&gt;RunPCA(object, pc.genes = yourgenelist)&lt;/code&gt; instead of the usual variable genes. You can also subset B cells first with &lt;code&gt;SubsetData()&lt;/code&gt;, and run the above.&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2019-06-29T23:08:10.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8884" PostTypeId="2" ParentId="8830" CreationDate="2019-06-30T19:01:27.807" Score="2" Body="&lt;p&gt;You can use &lt;a href=&quot;https://github.com/BatadaLab/scID&quot; rel=&quot;nofollow noreferrer&quot;&gt;scID&lt;/a&gt; with your gene list to find the cells that match your gene list. You can then find which cluster in Seurat have these cells. &lt;/p&gt;&#xA;" OwnerUserId="4799" LastActivityDate="2019-06-30T19:01:27.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8885" PostTypeId="1" AcceptedAnswerId="8886" CreationDate="2019-07-01T10:16:13.297" Score="0" ViewCount="1743" Body="&lt;p&gt;I have all file names for each paired-end fastq pair as shown below. Could you please suggest how I can extract all fastq.gz files in a single folder?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;sample name:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;188833984]$ ls&#xA;&#xA;Leaf_T1_FD_R10_L001-ds.3b884c360b1e4ae185408a613b90a3bc&#xA;Leaf_T1_FD_R2_L001-ds.7db8eb7e3426486db549426601b3a0bd&#xA;Leaf_T1_FD_R3_L001-ds.147177ecc03a46ccbbce3162d1185a0a&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;pair end file within sample name Leaf_T1_FD_R10_L001-ds.3b884c360b1e4ae185408a613b90a3bc&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;188833984]$ cd Leaf_T1_FD_R10_L001-ds.3b884c360b1e4ae185408a613b90a3bc&#xA;&#xA;Leaf_T1_FD_R10_L001-ds.3b884c360b1e4ae185408a613b90a3bc]$ ls&#xA;Leaf-T1-FD-R10_S73_L001_R1_001.fastq.gz&#xA;Leaf-T1-FD-R10_S73_L001_R2_001.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I need all in fastq.gz format but the problem is that each sample, pair end R1 and R2 fastq reads in a single folder like this: and I need to extract all samples files in a single folder&lt;/p&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="1008" LastEditDate="2019-07-01T15:38:38.917" LastActivityDate="2019-07-01T16:54:24.860" Title="How to extract all pair end fastq.gz files from multiple subdirectory to a single folder" Tags="|bash|awk|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8886" PostTypeId="2" ParentId="8885" CreationDate="2019-07-01T14:22:29.003" Score="3" Body="&lt;p&gt;Go to the common parent folder under which all FASTQ.GZ files reside. They could be in sub-folders under the folder, but all of them should have the folder you're in as a common parent folder. From such a folder, you should be able to run a &lt;code&gt;find&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;find . -name &quot;*.fastq.gz&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to get your FASTQ.GZ files. Now, there are surely better ways of doing this but here's one straightforward way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say you need to extract the FASTQs to &lt;code&gt;~/target/dir/&lt;/code&gt;, you'd do something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for f in &lt;span class=&quot;math-container&quot;&gt;$(find . -name &quot;*.fastq.gz&quot;)&#xA;do&#xA;    tgt_file=&quot;~/target/dir/$&lt;/span&gt;{f%.*}&quot; #removed .gz from the name&#xA;    gzip -dc &lt;span class=&quot;math-container&quot;&gt;$f &amp;gt;$&lt;/span&gt;tgt_file #pigz -dc would be better if you have pigz&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;EDIT: OP has clarified that they wish to copy the files as such, not decompress them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two ways to achieve this goal - you can either make physical copies of the files (thereby doubling the storage space you're currently using for them), or you can create symbolic links to the files (which are like shortcuts pointing to the actual files). I'll show you both ways here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume the files are all under &lt;code&gt;/fastq/files&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;find /fastq/files/ -name &quot;*.fastq.gz&quot; -exec cp {} ~/target/dir/ \; #This creates physical copies&#xA;&#xA;cd ~/target/dir&#xA;find /fastq/files/ -name &quot;*.fastq.gz&quot; | xargs -I v_f ln -s v_f #These lines create symbolic links that point to the original files. Do not delete/move/rename the original files or these will become invalid.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastEditorUserId="650" LastEditDate="2019-07-01T16:54:24.860" LastActivityDate="2019-07-01T16:54:24.860" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8887" PostTypeId="1" AcceptedAnswerId="8896" CreationDate="2019-07-01T14:39:41.727" Score="0" ViewCount="2361" Body="&lt;p&gt;I have used the following script for calculating the median for one file (gene). However, I have 20,000 files for which I want to run this R script and get 200 separate output files&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; #!/usr/bin/R&#xA; setwd(&quot;/location folder&quot;)&#xA;&#xA; d &amp;lt;- read.table(&quot;tgf.txt&quot;, header=FALSE)&#xA;&#xA; x &amp;lt;- aggregate(d[, 2], list(d$V3), median)&#xA;&#xA; write.table(x, &quot;tgf.output.txt&quot;, quote=FALSE, row.names=TRUE)&#xA;&#xA; q()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4783" LastEditorUserId="131" LastEditDate="2019-07-03T11:55:41.630" LastActivityDate="2019-07-03T11:55:41.630" Title="Run Rscript for multiple samples (files)" Tags="|r|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8888" PostTypeId="2" ParentId="8887" CreationDate="2019-07-01T16:14:38.640" Score="3" Body="&lt;p&gt;Please do not use &lt;code&gt;setwd&lt;/code&gt; in a script, it changes working folder without explicit specifications from the user, so the results could be unpredictable for people that are not familiar with the content of the script.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Create a tab-separated file with input filename and output filename, or at least a file with all input files, unless you can pick them using a &lt;code&gt;list.files(pattern = &quot;&amp;lt;pattern_here&amp;gt;&quot;)&lt;/code&gt;. Read this file as a &lt;code&gt;data.frame&lt;/code&gt; and use it with an &lt;code&gt;apply&lt;/code&gt;, so a function processes the df row-by-row. Let's say the file (named &lt;code&gt;metadata.txt&lt;/code&gt;) is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;in_file out_file&#xA;tgf.txt tgf.output.txt&#xA;abc.txt abc.filtered.txt&#xA;xyz.txt xyz.randomname.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Define a function called &lt;code&gt;process_row()&lt;/code&gt; in the Rscript:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;process_row &amp;lt;- function(r) {&#xA;    d &amp;lt;- read.table(r[[1]], header = FALSE);&#xA;    x &amp;lt;- aggregate(d[, 2], list(d$V3), median);&#xA;    write.table(x, r[[2]], quote=FALSE, row.names=TRUE);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And do the actual work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;meta_data &amp;lt;- read.table('./metadata.txt', sep = ' ', header = TRUE, stringsAsFactors = FALSE);&#xA;apply(meta_data, 1, process_row(r))&#xA;# You don't need a q() in an Rscript because it exits after the last line of code anyway.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastEditorUserId="650" LastEditDate="2019-07-02T15:11:33.410" LastActivityDate="2019-07-02T15:11:33.410" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8889" PostTypeId="1" CreationDate="2019-07-01T18:19:09.160" Score="0" ViewCount="78" Body="&lt;p&gt;I want to predict from protein sequence if the protein binds to metal, nuclear or small ligand. How can I do this ?  Which features are relevant if I want to use them in a machine learning algorithm ?&lt;/p&gt;&#xA;" OwnerUserId="4805" LastEditorUserId="48" LastEditDate="2019-07-02T15:47:47.723" LastActivityDate="2019-11-29T17:00:55.300" Title="How to predict protein binding to ligands?" Tags="|proteins|machine-learning|interactions|features|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8890" PostTypeId="1" AcceptedAnswerId="8907" CreationDate="2019-07-01T20:41:07.967" Score="2" ViewCount="738" Body="&lt;p&gt;Question moved here from the &lt;a href=&quot;https://biology.stackexchange.com/questions/85289/is-the-genome-a-programming-language-e-g-like-lisp-can-we-analyze-it-as-soft/&quot;&gt;biology stack exchange&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like the genome is a kind of LISP language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Operon structure of genome:&#xA;- &lt;a href=&quot;https://github.com/philschatz/microbiology-book/blob/master/resources/OSC_Microbio_11_07_Operon.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/philschatz/microbiology-book/blob/master/resources/OSC_Microbio_11_07_Operon.jpg&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LISP function structure:&#xA;- &lt;a href=&quot;http://support.ircam.fr/docs/om/om6-manual/res/listprefix.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://support.ircam.fr/docs/om/om6-manual/res/listprefix.png&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nested genes:&#xA;- &lt;a href=&quot;https://player.slideplayer.com/31/9782148/data/images/img11.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://player.slideplayer.com/31/9782148/data/images/img11.jpg&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nested LISP cons cells:&#xA;- &lt;a href=&quot;https://www.tutorialspoint.com/lisp/images/treestructure.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tutorialspoint.com/lisp/images/treestructure.jpg&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic similarity I am seeing is of general structure: (func var0 var1 ... varn) and nested cons cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The correspondence with genome structure is the operon portion, which has:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;operator = function&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;structural genes = variables&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;structural genes can be nested like cons cells&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is this a global structure for the genome, then a specific prediction is we can do something simple like counting open and close markers in the nested genes and they will be equal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, why would such a correspondence matter, if true?  The basic idea is that if the LISP structure is consistent, then we could maybe look for other programming constructs built on the structure, such as recursion/looping, maybe templating or even object oriented programming. Or just identify simple properties like balanced opening closing markers. The mere fact it has a LISP structure would be fascinating if true.  And since LISP is essentially an abstract syntax tree (AST), which all programming languages have, perhaps this structure I see in the genome is actually an AST, which implies an even more fundamental correspondence between the genome and programming languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally, we have a wealth of knowledge reverse engineering human programming languages.  If the genome is also a programming language similar in some respects to human programming languages, such as being a LISP variant, then we could apply this same reverse engineering knowledge base to better understand the genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, first of all, is my high level observation correct?  Does the genome, at least for bacteria, have a global LISP like structure?  If yes, then if the genome is a kind of programming language, i.e. LISP, can we analyze it like software to understand its functionality?  Has anyone done this before?  Simple searching on Google Scholar turns up lots of software for analyzing genomes, but nothing about analyzing the genome as software.  It would be interesting to know whether anyone has seriously examined the question before, and either made some useful discoveries, or found out there is only a very loose analogy with programming languages, and thus knowledge regarding programming languages is not of use regarding the genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully this question is clear enough now, if not please ask me for further clarification.  I would like to get some sort of decent answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE: &lt;a href=&quot;https://ds9a.nl/amazing-dna/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Someone&lt;/a&gt; who has put thought into &quot;DNA is code&quot; idea.  At a more general level than whether specific genome structures match specific programming structures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE #2: The late Scott Federhen of NCBI wrote an article &lt;a href=&quot;https://www.biorxiv.org/content/biorxiv/early/2015/05/01/018804.full.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;comparing genomic replication with the lambda operator&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE #3: It might be the case that &lt;a href=&quot;https://groups.csail.mit.edu/mac/users/gjs/6.945/readings/robust-systems.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;LISP was inspired by the genome&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4806" LastEditorUserId="4806" LastEditDate="2019-07-07T23:57:12.697" LastActivityDate="2019-07-07T23:57:12.697" Title="Is the genome a programming language (i.e. LISP)? Can we analyze it with computer science theory?" Tags="|genome|" AnswerCount="5" CommentCount="17" ContentLicense="CC BY-SA 4.0" />
  <row Id="8891" PostTypeId="2" ParentId="8868" CreationDate="2019-07-02T00:44:18.747" Score="0" Body="&lt;p&gt;You could find which Chromosome the merged file is currently writing on by:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools query -f'%CHROM\n' final.file.vcf.gz | tail&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that also you might want to try multithreading the command that you are using to write the file, as bcftools is capable of using several threads for the compression of output files. E.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools merge -Ob --threads 32 file1.vcf.gz file2.vcf.gz &amp;gt; mergedFile.vcf.gz &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2019-07-02T00:44:18.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8892" PostTypeId="2" ParentId="8854" CreationDate="2019-07-02T00:57:28.837" Score="1" Body="&lt;p&gt;bcftools has a plugin called &lt;a href=&quot;https://samtools.github.io/bcftools/howtos/plugin.fill-tags.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;fill-tags&lt;/a&gt; which should do the job for you easily, without having to mess around with lots of complicated bash scripting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could do something along the lines of this, presuming you have a .vcf file with all the IBS individuals in it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools +fill-tags in.vcf.gz -Oz -o out.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will give each position in the .vcf file new INFO tags, one of which will be allele-frequency. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then pull off the frequency for a particular variant (CHR:POS) by:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools query -t CHR:POS -f'%CHROM\t%POS\t%AF\n' out.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2019-07-02T00:57:28.837" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8893" PostTypeId="2" ParentId="8889" CreationDate="2019-07-02T08:57:24.307" Score="1" Body="&lt;p&gt;If you are training a machine learning algorithm you probably want to train against things you know to be true, rather than making predictions from the sequence and training against the predictions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case you will want to use protein structures with the relevant ligands bound as your data. If you go to the &lt;a href=&quot;https://www.rcsb.org/pdb/search/advSearch.do?search=new&quot; rel=&quot;nofollow noreferrer&quot;&gt;RCSB PDB advanced search&lt;/a&gt; you can search for entries with any ligand present, specific ligands present, protein and DNA present, etc. Then you can extract the FASTA sequences for these entries.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See also databases like &lt;a href=&quot;http://metalweb.cerm.unifi.it/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MetalPDB&lt;/a&gt; for specific ligand types, and one of &lt;a href=&quot;https://www.nature.com/articles/s41598-018-34533-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;our papers&lt;/a&gt; for an example of creating a dataset of metal binding proteins.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Features to use would be the raw sequence as a one-hot encoding on amino acid types. You could add other sequence level features such as secondary structure predictions, disorder predictions or physico-chemical properties. However a decent network would be able to predict much of this if it needs to. You could create alignments and put in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Position_weight_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;sequence profile&lt;/a&gt; or covariation data, which would be giving the network more information.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastEditorUserId="4622" LastEditDate="2019-07-02T16:06:57.187" LastActivityDate="2019-07-02T16:06:57.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8896" PostTypeId="2" ParentId="8887" CreationDate="2019-07-02T14:21:56.647" Score="1" Body="&lt;p&gt;Another option is to use &lt;code&gt;list.files&lt;/code&gt; and &lt;code&gt;grep&lt;/code&gt; to select only the &lt;code&gt;.txt&lt;/code&gt; in your directory. A &lt;code&gt;for&lt;/code&gt; loop can read-aggregate and save the files one at the time.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;all.files &amp;lt;- list.files(&quot;/location folder&quot;)&#xA;txt.files &amp;lt;- grep(&quot;.txt&quot;,all.files,value=T)&#xA;&#xA;for(i in txt.files){&#xA;  d &amp;lt;- read.table(paste(&quot;/location folder&quot;,i,sep=&quot;&quot;),header=F)&#xA;  x &amp;lt;- aggregate(d[, 2], list(d$V3), median)&#xA;  write.table(x, paste0(i,&quot;.output.txt&quot;), quote=FALSE, row.names=TRUE)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3631" LastActivityDate="2019-07-02T14:21:56.647" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8897" PostTypeId="2" ParentId="4172" CreationDate="2019-07-02T18:13:53.690" Score="1" Body="&lt;p&gt;Developments in this area over the last year are worth pointing out. All use residue-residue covariation information from a protein family so require a multiple sequence alignment to work, though it sounds like you have that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most accurate way to do this would be to use pre-trained machine learning methods, which would not require you to train your own version. Examples include:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://arxiv.org/abs/1811.12355&quot; rel=&quot;nofollow noreferrer&quot;&gt;DMPfold&lt;/a&gt; from our lab, which is freely-available open source.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/465955v4&quot; rel=&quot;nofollow noreferrer&quot;&gt;RaptorX&lt;/a&gt; method from the Xu lab.&lt;/li&gt;&#xA;&lt;li&gt;The state of the art is AlphaFold from DeepMind, though this has not been published yet.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;These predict the probabilities of a residue-residue distance being in various distance 'bins', e.g. the probability of being 5 - 6 Angstrom apart. They don't predict beyond ~20 Angstrom.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you specifically wanted a statistical, non-trained method, then consider:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://academic.oup.com/bioinformatics/article/28/2/184/198108&quot; rel=&quot;nofollow noreferrer&quot;&gt;PSICOV&lt;/a&gt; from our lab.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://academic.oup.com/bioinformatics/article/30/21/3128/2422261&quot; rel=&quot;nofollow noreferrer&quot;&gt;CCMPred&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://evfold.org/evfold-web/evfold.do&quot; rel=&quot;nofollow noreferrer&quot;&gt;EVfold&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;All of which are available in some form. The problem here is that they all predict binary contacts at 8 Angstrom, whereas it sounds like you need more fine-grained distances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a general sense, you can use any 3D structure prediction method and then read the distances back from the predicted structure.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2019-07-02T18:13:53.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8898" PostTypeId="2" ParentId="8880" CreationDate="2019-07-02T21:53:47.750" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/a/8882/240&quot;&gt;Peter's answer&lt;/a&gt; is a good one and to the point. I'll be a bit more verbose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The breakdown of FASTQ file names that come directly from the sequencer typically have the following format:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;{sample_name}_S{sample number}_L{lane number}_{R/I}{read or index number}_001.fastq.gz&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;sample_name&lt;/code&gt; is the sample name provided by you (or whoever sequenced the data) to the sequencer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To save time and money, samples are often &quot;multiplexed&quot; on sequencing runs. This multiplexing means you aggregate samples (&quot;S&quot;) on the same run and split samples on different lanes (&quot;L&quot;) to limit some technical noise. Lanes are essentially technical replicates for the sequencing data on the same sequencing run, and these numbers tell you which sample and which lane that FASTQ comes from.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sequencing can be done in single-end or paired-end. Single-end sequencing generates a single FASTQ file (&lt;code&gt;_R1&lt;/code&gt;), whereas paired-end sequencing produces 2 FASTQ files (&lt;code&gt;_R1&lt;/code&gt; and &lt;code&gt;_R2&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data you highlight isn't actually Nanopore:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Approximately 50x of data was generated on a NovaSeq instrument.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;10x includes &quot;indexes&quot; (&quot;I&quot;) in addition to the sample and lane information to keep track of things like single cells or other unique molecular identifiers, and that information is listed after the &lt;code&gt;I&lt;/code&gt;. &lt;code&gt;_R1&lt;/code&gt;, &lt;code&gt;_R2&lt;/code&gt;, and &lt;code&gt;_I1&lt;/code&gt; all go together for a unique sample and lane combination&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2019-07-02T21:53:47.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8899" PostTypeId="2" ParentId="8890" CreationDate="2019-07-03T13:18:50.773" Score="5" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;So, first of all, is my high level observation correct? Does the genome, at least for bacteria, have a global LISP like structure? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No. I'm afraid this is what it boils down to. I don't see any real similarity to LISP or other programming languages at all. Perhaps it would be clearer if you could explain &lt;em&gt;in detail&lt;/em&gt; why you think they're similar, but as it stands, no, there is no such structure apart from a very simplistic similarity which is sort of the same as saying that the genome is like a car because it carries genes and could therefore be studied by a car mechanic.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-07-03T13:18:50.773" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="8900" PostTypeId="1" AcceptedAnswerId="8901" CreationDate="2019-07-03T13:52:47.737" Score="1" ViewCount="111" Body="&lt;p&gt;Is there a tool already established to &quot;swap&quot; the ref/alt information in the &lt;code&gt;GT&lt;/code&gt;, &lt;code&gt;AD&lt;/code&gt; and &lt;code&gt;PL&lt;/code&gt; fields of a &lt;code&gt;vcf&lt;/code&gt; file?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the &lt;code&gt;GT&lt;/code&gt; field, I need to transform: &lt;code&gt;0/0 -&amp;gt; 1/1&lt;/code&gt; ;  &lt;code&gt;1/1 -&amp;gt; 0/0&lt;/code&gt; ;  &lt;code&gt;0/1 -&amp;gt; 1/0&lt;/code&gt; ; and finally &lt;code&gt;1/0 -&amp;gt; 0/1&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the &lt;code&gt;AD&lt;/code&gt; field, I need to swap the columns: &lt;code&gt;x,y -&amp;gt; y,x&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the &lt;code&gt;PL&lt;/code&gt; field, I need to swap the columns: &lt;code&gt;x,y,z -&amp;gt; z,y,x&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sample input: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Contig111   11  Contig111_11_T_G    T   G   4278.22 PASS    AC=8;AF=0.073;AN=110;BaseQRankSum=1.35;DP=1176;ExcessHet=0.1448;FS=1.841;InbreedingCoeff=0.1904;QD=13.41;ReadPosRankSum=0.351;set=Intersection  GT:AD:DP:GQ:PL  0/0:17,0:17:48:0,48,720 0/1:14,8:22:99:167,0,347 .....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Sample output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    Contig111   11  Contig111_11_T_G    T   G   4278.22 PASS    AC=8;AF=0.073;AN=110;BaseQRankSum=1.35;DP=1176;ExcessHet=0.1448;FS=1.841;InbreedingCoeff=0.1904;QD=13.41;ReadPosRankSum=0.351;set=Intersection  GT:AD:DP:GQ:PL  1/1:0,17:17:48:720,48,0 1/0:8,14:22:99:347,0,167 .....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;note: the same operations will need to be carried out for all samples if the VCF file has multiple samples (the &lt;code&gt;...&lt;/code&gt; stand for other potential samples).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before I dive into scripting ( basically playing with regex), I would like to know if something to process this is already available. I looked into &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=1&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwjpz7e77pjjAhU14KYKHdpiD5UQFjAAegQIARAB&amp;amp;url=https%3A%2F%2Fsamtools.github.io%2Fbcftools%2Fbcftools.html&amp;amp;usg=AOvVaw0zRNAyxe78GwcQ_MweVbi1&quot; rel=&quot;nofollow noreferrer&quot;&gt;bcftools&lt;/a&gt; and &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=5&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwingcLE7pjjAhVI0aYKHXHbD6kQFjAEegQIABAB&amp;amp;url=https%3A%2F%2Fgithub.com%2Fsamtools%2Fbcftools%2Fissues%2F806&amp;amp;usg=AOvVaw1Jw-IzAWoMXgaN-bxeGSlV&quot; rel=&quot;nofollow noreferrer&quot;&gt;fixref&lt;/a&gt; but they don't seem to help for this. &lt;/p&gt;&#xA;" OwnerUserId="294" LastEditorUserId="294" LastEditDate="2019-07-04T11:51:54.443" LastActivityDate="2019-07-04T11:51:54.443" Title="swap genotypes for specific fields of a vcf file" Tags="|vcf|file-formats|variant-calling|text-processing|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8901" PostTypeId="2" ParentId="8900" CreationDate="2019-07-03T14:36:00.260" Score="1" Body="&lt;p&gt;If we can always assume your fields will be &lt;code&gt;GT:AD:DP:GQ:PL&lt;/code&gt;, you can use this relatively simple, brute approach awk script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -vOFS=&quot;\t&quot; -F&quot;\t&quot; '{&#xA;  ## just print the header lines&#xA;  if(/^#/){&#xA;    print;&#xA;  }&#xA;  else{    &#xA;    ## The number of samples will be the extra number of fields &#xA;    ## in addition to the default VCF 9&#xA;    sampleNum = NF - 9&#xA;    ## Fix each sample&#xA;    for(i=1; i&amp;lt;=sampleNum; i++){&#xA;      ## get the fields&#xA;      num=split(&lt;span class=&quot;math-container&quot;&gt;$(9+i), fields,&quot;:&quot;);&#xA;      ## genotype&#xA;      if(fields[1] ~ &quot;0/0&quot;){ fields[1] = &quot;1/1&quot;}&#xA;      else if(fields[1] == &quot;1/1&quot;){ fields[1] = &quot;0/0&quot; }&#xA;      else if(fields[1] == &quot;0/1&quot;){ fields[1] = &quot;1/0&quot; }&#xA;      else if(fields[1] == &quot;1/0&quot;){ fields[1] = &quot;0/1&quot; }&#xA;      ## AD&#xA;      split(fields[2],values,&quot;,&quot;)&#xA;      fields[2] = values[2]&quot;,&quot;values[1]&#xA;      ## PL&#xA;      split(fields[2],values,&quot;,&quot;)&#xA;      ## Now set these as the sample data&#xA;      $&lt;/span&gt;(9+i) = fields[1];&#xA;      for(k=2;k&amp;lt;=num;k++){&#xA;        &lt;span class=&quot;math-container&quot;&gt;$(9+i) = $&lt;/span&gt;(9+i)&quot;:&quot;fields[k]&#xA;      }&#xA;    }&#xA;    print;&#xA;  }&#xA;}' file.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;On your example input, this returns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Contig111   11  Contig111_11_T_G    T   G   4278.22 PASS    AC=8;AF=0.073;AN=110;BaseQRankSum=1.35;DP=1176;ExcessHet=0.1448;FS=1.841;InbreedingCoeff=0.1904;QD=13.41;ReadPosRankSum=0.351;set=Intersection  GT:AD:DP:GQ:PL  1/1:0,17:17:48:0,48,720 1/0:8,14:22:99:167,0,347    .....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-07-03T14:36:00.260" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8902" PostTypeId="2" ParentId="8890" CreationDate="2019-07-03T15:59:47.297" Score="0" Body="&lt;p&gt;The only possible parallel was the &quot;processor 'core' wars&quot; in &lt;a href=&quot;https://en.wikipedia.org/wiki/Avida&quot; rel=&quot;nofollow noreferrer&quot;&gt;Avida&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Tierra_(computer_simulation)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tierra&lt;/a&gt;. Tierra was initiated by Tom Ray, around 1990. Here they had a 'population' of compiled binaries were randomly mutated (yep the binary code was mutated) and reproduction was dependent on how much of the processor could be consumed. The simulation worked and an in silico ecosystem evolved, but was heavily dependent on the preconditions of the simulation and proved no more reliabe at understanding evolution than differential equestions. So it was a nice idea, and very cool it worked, but it came to zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That aside the best example was the concept of the &quot;selfish gene&quot;, but has very little to do with programming languages IMO and everything to do with understanding kinselection.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-07-03T20:21:39.227" LastActivityDate="2019-07-03T20:21:39.227" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8903" PostTypeId="2" ParentId="8890" CreationDate="2019-07-03T16:01:41.007" Score="-5" Body="&lt;p&gt;Believe it or not, there are people who know more than you do about programming, and who also know a hell of a lot more than you do about the genome.  This is not an avenue those people think is worth pursuing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DNA is a chemical, and interacts with other chemicals based on its shape.  There is no equivalent to that in a human designed programming language.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-07-03T16:01:41.007" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8904" PostTypeId="2" ParentId="8890" CreationDate="2019-07-03T20:54:17.240" Score="2" Body="&lt;p&gt;You mention this as the base similarity you find:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;operator = function&lt;br&gt;&#xA;       structural genes = variables&lt;br&gt;&#xA;       structural genes can be nested like cons cells&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As other has pointed, this just considers the genome as a sequence of where only the linear order is important. It doesn't take into account other things that happen on the genome that are know to affect the cells like the chromatin structure, the duplication of certain regions, the transversion, the effect of moving elements, methylation, or simply that a single gene can have multiple functions (pleiotropism, moonlighting) while a programming function does not. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even if we considered it as code, certainly by using tools like HMM we could also learn more about the structure of the genes we have. While certainly, we need more tools than just the sequence to discover what does each gene do. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a lot more differences than similarities on this comparison, it can be helpful for certain aspects but it fails for other aspects of cells (without even going into the virus discussion or the mitochondrial). &lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="298" LastEditDate="2019-07-04T08:54:26.067" LastActivityDate="2019-07-04T08:54:26.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8905" PostTypeId="1" AcceptedAnswerId="8906" CreationDate="2019-07-03T21:07:40.657" Score="2" ViewCount="863" Body="&lt;p&gt;Could you please suggest me how I can remove range from fasta header:like these number from below sequences which has some colons indicating the range of the genome&lt;/p&gt;&#xA;&#xA;&lt;p&gt;:147010-147657 &lt;/p&gt;&#xA;&#xA;&lt;p&gt;:149201-149845&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; FVJT01000004.1:147010-147657 Mycobacterium abscessus subsp. massiliense strain 974 &#xA;GTGACCAGCAGCGCAGACCGATCCCCCGTCGAGGCGATGGTGAGCCACGCCGAGGGCGCCATCTCCGAGG&#xA;ATGACATCGTCGCCGCCGCCCGGGAACGTGCCGTTGACCTGGGCGCGGGCCCGGTGACACCCGCGGTAGG&#xA;&#xA;&amp;gt;FSAX01000002.1:149201-149845 Mycobacterium abscessus subsp. bolletii strain 286 genome &#xA;ATGACGACGCCCGTCAGCTTCGATAACAGATTCTTCTCCGCAACGTGGTCTGCTATCGCGCGGCGCGAAC&#xA;CTCAGGAGATCAAGGAGCTGCGTGCGGAGAATCTACGCGGCCTGACCGGCCACGTTCTGGAGGTGGGAGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="650" LastEditDate="2019-07-04T08:55:45.503" LastActivityDate="2019-07-10T13:03:00.690" Title="how to remove range from fasta header" Tags="|bash|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8906" PostTypeId="2" ParentId="8905" CreationDate="2019-07-03T22:13:37.497" Score="6" Body="&lt;p&gt;Invest some time in learning unix tools such as &lt;code&gt;grep&lt;/code&gt;, &lt;code&gt;sed&lt;/code&gt; and &lt;code&gt;awk&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -r '/^&amp;gt;/s/:[0-9]+-[0-9]+//' input.fasta &amp;gt; output.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This line of code matches the pattern &lt;code&gt;:&amp;lt;numbers&amp;gt;-&amp;lt;numbers&amp;gt;&lt;/code&gt; on lines that start with &lt;code&gt;&amp;gt;&lt;/code&gt; (FASTA sequence headers) and replaces that pattern with &lt;code&gt;''&lt;/code&gt;, essentially removing text that matches the pattern.&lt;/p&gt;&#xA;" OwnerUserId="650" LastEditorUserId="298" LastEditDate="2019-07-04T08:57:55.713" LastActivityDate="2019-07-04T08:57:55.713" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8907" PostTypeId="2" ParentId="8890" CreationDate="2019-07-04T12:19:06.633" Score="6" Body="&lt;p&gt;I see what the metaphor has been inspired by, where I think it got turned around, and might have a better one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to back up a bit and establish some common ground.  It isn't wrong to expect that some aspects of genetic code and software code are similar, and while biology is usually very specific, some aspects can be seen productively from the light of computer science ideas.  For example, mass-action molecular dynamics, which are often a good model for protein interactions, are Turing-complete.  Here's a fun example where people have written a programming language out of molecular dynamics:  &lt;a href=&quot;https://arxiv.org/abs/1809.07430&quot; rel=&quot;noreferrer&quot;&gt;https://arxiv.org/abs/1809.07430&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's also true that proteins have particular substructures leading to certain functions, such as using alpha-helices to react less with the outside world.  Learning about larger substructures, called protein domains, help identify the functions of families of proteins.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that the parts of the genome which appear nested like LISP cells actually aren't included in the protein.  The regulatory regions, promoters, inhibitors, introns, and so forth never become part of the key &quot;computational&quot; machinery.  This nesting isn't actively part of the computational activity proteins do perform.  The main quality of LISP is that the structure and function are perfectly self-similar throughout an expression, but the structure of the regulatory regions are neither replicated down into the protein nor are necessarily recapitulated on a larger scale.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, we don't expect the computational structure of proteins to be the same from one protein to another.  This is because &quot;binds only with X&quot; and &quot;binds only with Y&quot; must be implemented differently to successfully be distinct operations, and it is these binding constraints by which molecular dynamics implement &quot;computational&quot; operations, such as &quot;not X&quot; and &quot;not Y&quot;, and so those are distinct as well.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think a better way to think about the regular structures of the regulatory apparatus is more like a network protocol.  Along an otherwise undifferentiated binary channel, one has to identify when and where the data is delivered to an application.  Similarly, along a strand of bases, the machinery of translation has to have an indication of where the &quot;application&quot; of the protein starts and ends, and when that protein should be delivered to the cell.  We might compare some of the images of genetic structure that provided in the original question with layouts of network headers: &lt;a href=&quot;http://slideplayer.com/slide/4798836/15/images/16/TCP/IP+Packet+Structures.jpg&quot; rel=&quot;noreferrer&quot;&gt;http://slideplayer.com/slide/4798836/15/images/16/TCP/IP+Packet+Structures.jpg&lt;/a&gt;  We might compare bacterial versus multi-cellular regulation as having different delivery mechanisms and data structures for delivering the application information, for different functional consequences, in the same way TCP and UDP have different structures for different trade-offs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is all complicated somewhat by the fact that proteins do then interact with the regulation of genes, leading to interesting &quot;computational&quot; dynamics such as feed-forward regulation.  However, I'm not aware of any particular structure of the regulatory apparatus that necessarily correlates with these larger functional patterns.  The opposite is frequently biologically useful: the same structure of a protein helping to activate one gene might simultaneously inhibit another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One book I've found very accessible and useful in understanding where computation is useful biologically is Dennis Bray's &quot;Wetware: a computer in every living cell&quot;.  It gave me a clearer picture of how computation occurs in cells.  That should open up some introductions to systems biology which explains the biological-role of larger molecular &quot;computational&quot; patterns.&lt;/p&gt;&#xA;" OwnerUserId="4818" LastEditorUserId="4818" LastEditDate="2019-07-05T12:48:47.750" LastActivityDate="2019-07-05T12:48:47.750" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="8908" PostTypeId="1" AcceptedAnswerId="11216" CreationDate="2019-07-04T23:26:35.613" Score="3" ViewCount="949" Body="&lt;p&gt;I'm having problems properly looking for a solution because I'm a layman in Bioinformatics not familiar with the terminology. I'm hoping you can nudge me in the right direction, please! Thank you very much.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;What I want in the end&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;A FASTA file containing a single sequential sequence (chromosome 1-22..) of the human genome.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;What I have&lt;/h1&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;23gb of &quot;clean_data&quot; which are .fq.gz files.&lt;/li&gt;&#xA;&lt;li&gt;21gb of &quot;result_alignment&quot; which is .bam+.bai&lt;/li&gt;&#xA;&lt;li&gt;&quot;result_variation&quot; indel.vcf.gz and snp.vcf.gz&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h1&gt;What I tried&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I used &lt;code&gt;samtools fasta reads.bam &amp;gt; reads.fasta&lt;/code&gt; which results in many lines like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;V100005418L4C001R001322578/1&#xA;AAATAGCCCACAC...&#xA;&amp;gt;V100005418L4C001R006084507/1&#xA;GCCATAAAGCCTA...&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But these include many many duplicate lines and the reference genome from the human genome project GRCh38 is only 3GB in size, so I thought that this FASTA sequence needs to be properly &quot;aligned&quot; somehow?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I thinking correctly here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm really confused with all of the terminology and don't know exactly what I need to look for. Could someone kindly put the process into simpler words for me?&lt;/p&gt;&#xA;" OwnerUserId="4362" LastEditorUserId="4362" LastEditDate="2019-07-05T14:00:23.730" LastActivityDate="2020-01-22T18:31:42.317" Title="Produce a single sequential FASTA sequence out of BAM" Tags="|sam|sequence-alignment|ngs|phylogenetics|genome-sequencing|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8909" PostTypeId="1" CreationDate="2019-07-04T23:45:39.273" Score="1" ViewCount="1923" Body="&lt;p&gt;I got an input matrix which RNA expression and ADT capture are combined into one file. I loaded the file into Seurat successfully, however, when I tried to create Seurat object, it threw out an error saying &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in CreateAssayObject(counts = counts, min.cells = min.cells, min.features = min.features) : &#xA;  No cell names (colnames) names present in the input matrix&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I create Seurat object when RNA expression and ADT data are combine together?&lt;/p&gt;&#xA;" OwnerUserId="4739" LastEditorUserId="650" LastEditDate="2019-07-05T12:48:16.587" LastActivityDate="2020-05-09T07:25:51.837" Title="How to create Seurat object while RNA expression and ADT combined into one matrix" Tags="|seurat|single-cell|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8910" PostTypeId="2" ParentId="8908" CreationDate="2019-07-05T07:41:16.977" Score="3" Body="&lt;p&gt;If you just want the human chromosomes in a FASTA format, why not &lt;a href=&quot;http://hgdownload.cse.ucsc.edu/goldenpath/hg38/chromosomes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;downloading&lt;/a&gt; directly the individual chromosome files (&lt;code&gt;chr*.fa.gz&lt;/code&gt;)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't rely on the &quot;official&quot; releases (for whatever reason), then what you need is first to assemble your reads into genomes, possibly (but not surely) going down to a chromosome-level resolution. Different approaches for assembly are possible/available, mainly depending on (1) what do you biologically need to answer and (2) the type of sequencing data you have - we'll need more details to be able to help you in this direction. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;samtools fasta&lt;/code&gt; &lt;a href=&quot;http://www.htslib.org/doc/samtools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;utility&lt;/a&gt; will &quot;just&quot; convert your reads (likely given from a sequencing facility) from an alignment (BAM) format into a reads (FASTA) format, there is no concept of chromosomes here. You won't get assembled chromosomes this way, just extract the sequence in FASTA format from your alignment file. &lt;/p&gt;&#xA;" OwnerUserId="294" LastEditorUserId="294" LastEditDate="2019-07-05T07:50:05.420" LastActivityDate="2019-07-05T07:50:05.420" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8911" PostTypeId="1" AcceptedAnswerId="8912" CreationDate="2019-07-05T11:02:30.173" Score="2" ViewCount="406" Body="&lt;p&gt;I am wondering if anyone can help to merge two fasta file based on the full header as initially gene ID may be similar in some of the sequences and to remove the duplicate entry based on full fasta header &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do have two multifasta file:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;file1:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;AOIT01000069.1:1403-2242 Haloterrigena limicola JCM 13563 contig_69, whole genome shotgun sequence&#xA;ATGAGTAACGATACGGTAGCTGGTGACCGCGACCCCGAGGAGGCCCGTGAGATGGTGCGCGAACGCTACG&#xA;GGACGATCGCTTCGGACGGTCAGGACTGCTGTGGCGATGTCGGCATCGATGTCACCGACGACGGTGGGTG&#xA;&#xA;&amp;gt;LOPV01000533.1:1144-1992 Haloferax sp. SB29 contig_613, whole genome shotgun sequence&#xA;ATGAGTGACGACACCCAGACTGGCACGACAGCCGACGACCGCGATCCAGAGGCGACTCGCCAGTTGGTAC&#xA;GGGAACGCTACGCCGGAATCGCATCGAGCGACCAGGGCTGTTGTAGCGACGTCGGCCTCGATGTCACGGG&#xA;&#xA;&#xA;&amp;gt;AOLJ01000027.1:50569-51417 Haloferax gibbonsii ATCC 33959 contig_27, whole genome shotgun sequence&#xA;TCAGTCGTCGAAGGTCGGTTTCCGGGCTTCGATGGTGGCCGACACGAGGTACTCGCCGAGGTCGCGGTCG&#xA;GCCTCCCAGTCGCTGATGAACTCAGTACTCTCGTCCTTGGGTGCAATCTCGACCGCCTCGAAGCCAGCGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;file2:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;AOLJ01000027.1:50569-51417 Haloferax gibbonsii ATCC 33959 contig_27, whole genome shotgun sequence&#xA;TCAGTCGTCGAAGGTCGGTTTCCGGGCTTCGATGGTGGCCGACACGAGGTACTCGCCGAGGTCGCGGTCG&#xA;GCCTCCCAGTCGCTGATGAACTCAGTACTCTCGTCCTTGGGTGCAATCTCGACCGCCTCGAAGCCAGCGC&#xA;&#xA;&#xA;&amp;gt;AM774418.1:280304-281149 Halobacterium salinarum R1 plasmid PHS3 complete genome&#xA;ATGAGTAATGACAACGAGACGATGGTCGCCGATCGCGATCCCGAGGAGACTCGCGAGATGGTGCGGGAAC&#xA;GCTACGCGGGAATCGCGACGAGCGGCCAGGACTGCTGTGGTGACGTCGGTTTGGATGTCTCTGGCGACGG&#xA;&#xA;&amp;gt;AOJK01000067.1:53467-54312 Halorubrum californiensis DSM 19288 contig_67, whole genome shotgun sequence&#xA;TCAGTCGTCGCGGGCTGGTTTTCGGGCTTCGATGGTAGCGGAAACGAGGTACTCGCCGAGGTCGCGGTCG&#xA;GCGTCCCAGTCGCTGATGAACTCGGTGCTCTCGTCCTTCGGCGCGATCTCGACCGCCTCGAAGCCCGCCT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Expected output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;AOIT01000069.1:1403-2242 Haloterrigena limicola JCM 13563 contig_69, whole genome shotgun sequence&#xA;ATGAGTAACGATACGGTAGCTGGTGACCGCGACCCCGAGGAGGCCCGTGAGATGGTGCGCGAACGCTACG&#xA;GGACGATCGCTTCGGACGGTCAGGACTGCTGTGGCGATGTCGGCATCGATGTCACCGACGACGGTGGGTG&#xA;&#xA;&amp;gt;LOPV01000533.1:1144-1992 Haloferax sp. SB29 contig_613, whole genome shotgun sequence&#xA;ATGAGTGACGACACCCAGACTGGCACGACAGCCGACGACCGCGATCCAGAGGCGACTCGCCAGTTGGTAC&#xA;GGGAACGCTACGCCGGAATCGCATCGAGCGACCAGGGCTGTTGTAGCGACGTCGGCCTCGATGTCACGGG&#xA;&#xA;&amp;gt;AOLJ01000027.1:50569-51417 Haloferax gibbonsii ATCC 33959 contig_27, whole genome shotgun sequence&#xA;TCAGTCGTCGAAGGTCGGTTTCCGGGCTTCGATGGTGGCCGACACGAGGTACTCGCCGAGGTCGCGGTCG&#xA;GCCTCCCAGTCGCTGATGAACTCAGTACTCTCGTCCTTGGGTGCAATCTCGACCGCCTCGAAGCCAGCGC&#xA;&#xA;&amp;gt;AM774418.1:280304-281149 Halobacterium salinarum R1 plasmid PHS3 complete genome&#xA;ATGAGTAATGACAACGAGACGATGGTCGCCGATCGCGATCCCGAGGAGACTCGCGAGATGGTGCGGGAAC&#xA;GCTACGCGGGAATCGCGACGAGCGGCCAGGACTGCTGTGGTGACGTCGGTTTGGATGTCTCTGGCGACGG&#xA;&#xA;&amp;gt;AOJK01000067.1:53467-54312 Halorubrum californiensis DSM 19288 contig_67, whole genome shotgun sequence&#xA;TCAGTCGTCGCGGGCTGGTTTTCGGGCTTCGATGGTAGCGGAAACGAGGTACTCGCCGAGGTCGCGGTCG&#xA;GCGTCCCAGTCGCTGATGAACTCGGTGCTCTCGTCCTTCGGCGCGATCTCGACCGCCTCGAAGCCCGCCT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="298" LastEditDate="2019-07-05T12:54:01.577" LastActivityDate="2019-07-06T14:52:24.493" Title="how to merge two fasta based on full fasta header and remove the duplicate" Tags="|fasta|text-processing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8912" PostTypeId="2" ParentId="8911" CreationDate="2019-07-05T13:04:24.760" Score="2" Body="&lt;p&gt;You can use the &lt;code&gt;FastaToTbl&lt;/code&gt; and &lt;code&gt;TblToFasta&lt;/code&gt; scripts I have posted &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2651/298&quot;&gt;here&lt;/a&gt; for this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FastaToTbl file1.fa file2.fa | sort | uniq | TblToFasta &amp;gt; merged.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you run this on your example, you will get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ FastaToTbl file1.fa file2.fa | sort | uniq | TblToFasta &amp;gt; merged.fa&#xA;$&lt;/span&gt; cat merged.fa &#xA;&amp;gt;AM774418.1:280304-281149 Halobacterium salinarum R1 plasmid PHS3 complete genome &#xA;ATGAGTAATGACAACGAGACGATGGTCGCCGATCGCGATCCCGAGGAGACTCGCGAGATG&#xA;GTGCGGGAACGCTACGCGGGAATCGCGACGAGCGGCCAGGACTGCTGTGGTGACGTCGGT&#xA;TTGGATGTCTCTGGCGACGG&#xA;&amp;gt;AOIT01000069.1:1403-2242 Haloterrigena limicola JCM 13563 contig_69, whole genome shotgun sequence &#xA;ATGAGTAACGATACGGTAGCTGGTGACCGCGACCCCGAGGAGGCCCGTGAGATGGTGCGC&#xA;GAACGCTACGGGACGATCGCTTCGGACGGTCAGGACTGCTGTGGCGATGTCGGCATCGAT&#xA;GTCACCGACGACGGTGGGTG&#xA;&amp;gt;AOJK01000067.1:53467-54312 Halorubrum californiensis DSM 19288 contig_67, whole genome shotgun sequence &#xA;TCAGTCGTCGCGGGCTGGTTTTCGGGCTTCGATGGTAGCGGAAACGAGGTACTCGCCGAG&#xA;GTCGCGGTCGGCGTCCCAGTCGCTGATGAACTCGGTGCTCTCGTCCTTCGGCGCGATCTC&#xA;GACCGCCTCGAAGCCCGCCT&#xA;&amp;gt;AOLJ01000027.1:50569-51417 Haloferax gibbonsii ATCC 33959 contig_27, whole genome shotgun sequence &#xA;TCAGTCGTCGAAGGTCGGTTTCCGGGCTTCGATGGTGGCCGACACGAGGTACTCGCCGAG&#xA;GTCGCGGTCGGCCTCCCAGTCGCTGATGAACTCAGTACTCTCGTCCTTGGGTGCAATCTC&#xA;GACCGCCTCGAAGCCAGCGC&#xA;&amp;gt;LOPV01000533.1:1144-1992 Haloferax sp. SB29 contig_613, whole genome shotgun sequence &#xA;ATGAGTGACGACACCCAGACTGGCACGACAGCCGACGACCGCGATCCAGAGGCGACTCGC&#xA;CAGTTGGTACGGGAACGCTACGCCGGAATCGCATCGAGCGACCAGGGCTGTTGTAGCGAC&#xA;GTCGGCCTCGATGTCACGGG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will remove any cases where both the header and the sequence are identical. If you instead want to do this based on the header only, not taking the sequences into account, you can just process both files and only print sequences from the 2nd if their ID line hasn't been seen before. For example, with &lt;code&gt;awk&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk '{ &#xA;        if(NR==FNR){ &#xA;            if(/^&amp;gt;/){seen[$0]++;} &#xA;            print&#xA;        }&#xA;        else{ &#xA;            if(/^&amp;gt;/){ skip=seen[$&lt;/span&gt;0] } &#xA;            if(!skip){ print }&#xA;        }&#xA;       }' file1.fa file2.fa &#xA;&amp;gt;AOIT01000069.1:1403-2242 Haloterrigena limicola JCM 13563 contig_69, whole genome shotgun sequence&#xA;ATGAGTAACGATACGGTAGCTGGTGACCGCGACCCCGAGGAGGCCCGTGAGATGGTGCGCGAACGCTACG&#xA;GGACGATCGCTTCGGACGGTCAGGACTGCTGTGGCGATGTCGGCATCGATGTCACCGACGACGGTGGGTG&#xA;&#xA;&amp;gt;LOPV01000533.1:1144-1992 Haloferax sp. SB29 contig_613, whole genome shotgun sequence&#xA;ATGAGTGACGACACCCAGACTGGCACGACAGCCGACGACCGCGATCCAGAGGCGACTCGCCAGTTGGTAC&#xA;GGGAACGCTACGCCGGAATCGCATCGAGCGACCAGGGCTGTTGTAGCGACGTCGGCCTCGATGTCACGGG&#xA;&#xA;&#xA;&amp;gt;AOLJ01000027.1:50569-51417 Haloferax gibbonsii ATCC 33959 contig_27, whole genome shotgun sequence&#xA;TCAGTCGTCGAAGGTCGGTTTCCGGGCTTCGATGGTGGCCGACACGAGGTACTCGCCGAGGTCGCGGTCG&#xA;GCCTCCCAGTCGCTGATGAACTCAGTACTCTCGTCCTTGGGTGCAATCTCGACCGCCTCGAAGCCAGCGC&#xA;&amp;gt;AM774418.1:280304-281149 Halobacterium salinarum R1 plasmid PHS3 complete genome&#xA;ATGAGTAATGACAACGAGACGATGGTCGCCGATCGCGATCCCGAGGAGACTCGCGAGATGGTGCGGGAAC&#xA;GCTACGCGGGAATCGCGACGAGCGGCCAGGACTGCTGTGGTGACGTCGGTTTGGATGTCTCTGGCGACGG&#xA;&#xA;&amp;gt;AOJK01000067.1:53467-54312 Halorubrum californiensis DSM 19288 contig_67, whole genome shotgun sequence&#xA;TCAGTCGTCGCGGGCTGGTTTTCGGGCTTCGATGGTAGCGGAAACGAGGTACTCGCCGAGGTCGCGGTCG&#xA;GCGTCCCAGTCGCTGATGAACTCGGTGCTCTCGTCCTTCGGCGCGATCTCGACCGCCTCGAAGCCCGCCT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-07-05T13:04:24.760" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8913" PostTypeId="1" AcceptedAnswerId="8955" CreationDate="2019-07-05T17:12:33.580" Score="2" ViewCount="758" Body="&lt;p&gt;I'm trying to perform pretty standard differential expression analysis using RNA-seq data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've used &lt;a href=&quot;https://pachterlab.github.io/kallisto/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kallisto&lt;/a&gt; to perform RNA quantification and am using &lt;a href=&quot;https://pachterlab.github.io/sleuth/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sleuth&lt;/a&gt; to perform the differential expression analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 2 conditions that I'd like to compare, which is stored in the &lt;code&gt;metadata&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; metadata&#xA;    sample cell replicate batch                   path&#xA;1:  Base-1 Base         1   1   /path/to/Batch1/Base-1&#xA;2:  Base-2 Base         2   2   /path/to/Batch2/Base-2&#xA;3:  Base-3 Base         3   2   /path/to/Batch2/Base-3&#xA;4:  Test-1 Test         1   2   /path/to/Batch2/Test-1&#xA;5:  Test-2 Test         2   2   /path/to/Batch2/Test-2&#xA;6:  Test-3 Test         3   3   /path/to/Batch3/Test-3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm attempting to test for differentially expressed genes between the &lt;code&gt;Base&lt;/code&gt; and &lt;code&gt;Test&lt;/code&gt; cells while controlling for &lt;code&gt;batch&lt;/code&gt; (the sequencing batch). The R code to do this is adapted from &lt;a href=&quot;https://pachterlab.github.io/sleuth_walkthroughs/pval_agg/analysis.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sleuth's manual&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sleuth_obj = sleuth_prep(&#xA;    metadata,&#xA;    extra_bootstrap_summary = TRUE,&#xA;    num_cores = 1,&#xA;    target_mapping = transcripts_genes, #ENSEMBL information mapping transcripts to genes&#xA;    aggregation_column = &quot;ens_gene&quot;&#xA;)&#xA;&#xA;# smooth raw Kallisto abundances&#xA;sleuth_obj = sleuth_fit(sleuth_obj, ~cell + batch, &quot;full&quot;)&#xA;&#xA;# fitting error measurements on reduced model&#xA;sleuth_obj = sleuth_fit(sleuth_obj, ~batch, &quot;reduced&quot;)&#xA;&#xA;# compare models to calculate differentially expressed transcripts&#xA;sleuth_obj = sleuth_lrt(sleuth_obj, &quot;reduced&quot;, &quot;full&quot;)&#xA;&#xA;# extract results&#xA;sleuth_table = as.data.table(sleuth_results(&#xA;    sleuth_obj,&#xA;    'reduced:full',&#xA;    'lrt',&#xA;    show_all = FALSE&#xA;))&#xA;sleuth_table_tx = as.data.table(sleuth_results(&#xA;    sleuth_obj,&#xA;    'reduced:full',&#xA;    'lrt',&#xA;    show_all = FALSE,&#xA;    pval_aggregate = FALSE&#xA;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To check the quality of the null hypothesis, I've checked the p-value histograms of the gene-level and transcript-level analyses, and I get the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FOKOx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FOKOx.png&quot; alt=&quot;Gene-level p-value histogram&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/TAWCX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TAWCX.png&quot; alt=&quot;Transcript-level p-value histogram&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neither of these plots show well-behaved p-values, which makes me question the validity of the reduced model. How can I go about adjusting my analysis to get more reliable results?&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Edit following from @swbarnes2's answer:&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I've used &lt;code&gt;fdrtool&lt;/code&gt; to create an empirical null model of Sleuth's likelihood ratio test.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(fdrtool)&#xA;local_fdr_txs = fdrtool(so&lt;span class=&quot;math-container&quot;&gt;$tests$&lt;/span&gt;lrt&lt;span class=&quot;math-container&quot;&gt;$`reduced:full`$&lt;/span&gt;test_stat, statistic=&quot;normal&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which gives the following plots:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/37rsY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/37rsY.png&quot; alt=&quot;fdrtool&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plotting the distribution of these p-values produces:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/O3hjw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/O3hjw.png&quot; alt=&quot;pvalues after fdrtool&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This still has a hump, but looks more anti-conservative than before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My remaining question is about whether I can use &lt;code&gt;fdrtool&lt;/code&gt; to model the null hypothesis as normal, since I'm not certain about the distribution for Sleuth's likelihood ratio test&lt;/p&gt;&#xA;" OwnerUserId="240" LastEditorUserId="240" LastEditDate="2019-07-09T17:31:19.150" LastActivityDate="2019-07-10T14:40:46.627" Title="Strange p-value histogram for differential gene expression analysis" Tags="|rna-seq|differential-expression|kallisto|p-values|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8915" PostTypeId="1" CreationDate="2019-07-05T22:16:29.403" Score="5" ViewCount="1501" Body="&lt;p&gt;Many popular SV callers output a VCF. Unfortunately, there isn't a unified system at the present to label events with the same notation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, is there a standard method for converting these VCFs to BEDPEs? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;svtools comes to mind: &lt;a href=&quot;https://github.com/hall-lab/svtools&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/hall-lab/svtools&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also some Bioconductor packages. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, is there currently a standard widely used? Normally this is because a certain consortium did it in a certain way. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: There was a question in the comments about VCF vs BEDPE for structural variants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/jaquol/blog-like-old/blob/master/01-How-interpret-structural-variants-BEDPE-format.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/jaquol/blog-like-old/blob/master/01-How-interpret-structural-variants-BEDPE-format.md&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a structural variant in VCF format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.2                                    &#xA;##source=LUMPY                                  &#xA;##INFO=&amp;lt;ID=SVTYPE,Number=1,Type=String,Description=&quot;Type of structural variant&quot;&amp;gt;                                    &#xA;##INFO=&amp;lt;ID=SVLEN,Number=.,Type=Integer,Description=&quot;Difference in length between REF and ALT alleles&quot;&amp;gt;                                  &#xA;##INFO=&amp;lt;ID=END,Number=1,Type=Integer,Description=&quot;End position of the variant described in this record&quot;&amp;gt;                                    &#xA;##INFO=&amp;lt;ID=STRANDS,Number=.,Type=String,Description=&quot;Strand orientation of the adjacency in BEDPE format (DEL:+-, DUP:-+, INV:++/--)&quot;&amp;gt;                                  &#xA;##INFO=&amp;lt;ID=IMPRECISE,Number=0,Type=Flag,Description=&quot;Imprecise structural variation&quot;&amp;gt;                                   &#xA;##INFO=&amp;lt;ID=CIPOS,Number=2,Type=Integer,Description=&quot;Confidence interval around POS for imprecise variants&quot;&amp;gt;                                 &#xA;##INFO=&amp;lt;ID=CIEND,Number=2,Type=Integer,Description=&quot;Confidence interval around END for imprecise variants&quot;&amp;gt;                                 &#xA;##INFO=&amp;lt;ID=CIPOS95,Number=2,Type=Integer,Description=&quot;Confidence interval (95%) around POS for imprecise variants&quot;&amp;gt;                                 &#xA;##INFO=&amp;lt;ID=CIEND95,Number=2,Type=Integer,Description=&quot;Confidence interval (95%) around END for imprecise variants&quot;&amp;gt;                                 &#xA;##INFO=&amp;lt;ID=MATEID,Number=.,Type=String,Description=&quot;ID of mate breakends&quot;&amp;gt;                                  &#xA;##INFO=&amp;lt;ID=EVENT,Number=1,Type=String,Description=&quot;ID of event associated to breakend&quot;&amp;gt;                                 &#xA;##INFO=&amp;lt;ID=SECONDARY,Number=0,Type=Flag,Description=&quot;Secondary breakend in a multi-line variants&quot;&amp;gt;                                  &#xA;##INFO=&amp;lt;ID=SU,Number=.,Type=Integer,Description=&quot;Number of pieces of evidence supporting the variant across all samples&quot;&amp;gt;                                   &#xA;##INFO=&amp;lt;ID=PE,Number=.,Type=Integer,Description=&quot;Number of paired-end reads supporting the variant across all samples&quot;&amp;gt;                                 &#xA;##INFO=&amp;lt;ID=SR,Number=.,Type=Integer,Description=&quot;Number of split reads supporting the variant across all samples&quot;&amp;gt;                                  &#xA;##INFO=&amp;lt;ID=EV,Number=.,Type=String,Description=&quot;Type of LUMPY evidence contributing to the variant call&quot;&amp;gt;                                   &#xA;##INFO=&amp;lt;ID=PRPOS,Number=.,Type=String,Description=&quot;LUMPY probability curve of the POS breakend&quot;&amp;gt;                                    &#xA;##INFO=&amp;lt;ID=PREND,Number=.,Type=String,Description=&quot;LUMPY probability curve of the END breakend&quot;&amp;gt;                                    &#xA;##ALT=&amp;lt;ID=DEL,Description=&quot;Deletion&quot;&amp;gt;                                   &#xA;##ALT=&amp;lt;ID=DUP,Description=&quot;Duplication&quot;&amp;gt;                                    &#xA;##ALT=&amp;lt;ID=INV,Description=&quot;Inversion&quot;&amp;gt;                                  &#xA;##ALT=&amp;lt;ID=DUP:TANDEM,Description=&quot;Tandem duplication&quot;&amp;gt;                                  &#xA;##ALT=&amp;lt;ID=INS,Description=&quot;Insertion of novel sequence&quot;&amp;gt;                                    &#xA;##ALT=&amp;lt;ID=CNV,Description=&quot;Copy number variable region&quot;&amp;gt;                                    &#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&quot;Genotype&quot;&amp;gt;                                    &#xA;##FORMAT=&amp;lt;ID=SU,Number=1,Type=Integer,Description=&quot;Number of pieces of evidence supporting the variant&quot;&amp;gt;                                    &#xA;##FORMAT=&amp;lt;ID=PE,Number=1,Type=Integer,Description=&quot;Number of paired-end reads supporting the variant&quot;&amp;gt;                                  &#xA;##FORMAT=&amp;lt;ID=SR,Number=1,Type=Integer,Description=&quot;Number of split reads supporting the variant&quot;&amp;gt;                                   &#xA;##FORMAT=&amp;lt;ID=BD,Number=1,Type=Integer,Description=&quot;Amount of BED evidence supporting the variant&quot;&amp;gt;                                  &#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  3128_T47D&#xA;chr1    1584528 11  N   &amp;lt;DUP&amp;gt;   .   .   SVTYPE=DUP;STRANDS=-+:6;SVLEN=66541;END=1651069;CIPOS=-449,10;CIEND=-10,503;CIPOS95=-374,0;CIEND95=0,366;IMPRECISE;SU=6;PE=6;SR=0;PRPOS=2.44491e-07,3.26458e-07,4.33697e-07,5.7608e-07,7.57136e-07,9.87322e-07,1.28796e-06,1.66553e-06,2.14862e-06,2.75607e-06,3.52645e-06,4.48654e-06,5.68684e-06,7.14253e-06,8.95608e-06,1.11311e-05,1.38146e-05,1.70081e-05,2.09074e-05,2.54614e-05,3.0947e-05,3.7325e-05,4.47552e-05,5.33515e-05,6.32491e-05,7.45332e-05,8.7438e-05,0.00010173,0.000117821,0.000135629,0.000155258,0.000177125,0.000200949,0.000227194,0.000255827,0.000286183,0.000319177,0.000353501,0.000390203,0.000428746,0.000468895,0.000511378,0.000555476,0.000600756,0.000648001,0.000695831,0.000745204,0.000795631,0.000845517,0.000896997,0.00094752,0.000998877,0.00104988,0.00110053,0.00115056,0.00120038,0.00124937,0.00129849,0.00134617,0.00139206,0.00143762,0.0014814,0.00152386,0.00156521,0.00160519,0.00164404,0.00168131,0.00171717,0.00175139,0.00178475,0.00181657,0.00184663,0.00187563,0.00190327,0.00192975,0.00195479,0.00197842,0.00200183,0.00202369,0.00204549,0.00206583,0.00208582,0.00210445,0.00212224,0.00213857,0.00215452,0.00216896,0.00218263,0.00219584,0.00220845,0.00221975,0.00223061,0.00224049,0.00224956,0.00225841,0.00226617,0.00227317,0.00227931,0.00228505,0.00228996,0.00229495,0.00229948,0.00230347,0.00230755,0.00231109,0.0023146,0.00231767,0.00232087,0.0023235,0.00232593,0.00232843,0.00233037,0.00233266,0.00233471,0.0023365,0.00233822,0.00233974,0.00234121,0.00234261,0.00234391,0.00234524,0.00234656,0.00234787,0.00234901,0.00235006,0.0023513,0.0023522,0.00235324,0.00235429,0.00235519,0.00235614,0.00235695,0.00235786,0.00235864,0.00235957,0.00236033,0.00236117,0.00236198,0.00236288,0.00236365,0.00236441,0.00236514,0.00236581,0.00236671,0.0023675,0.00236851,0.0023693,0.00237023,0.00237106,0.00237196,0.00237283,0.00237371,0.00237471,0.00237556,0.00237647,0.00237733,0.00237819,0.00237897,0.00237973,0.00238056,0.00238145,0.00238222,0.00238294,0.00238394,0.0023848,0.00238545,0.00238618,0.0023871,0.00238796,0.0023889,0.00238976,0.00239053,0.00239142,0.00239222,0.0023932,0.00239412,0.00239504,0.00239599,0.00239704,0.00239815,0.00239902,0.00239984,0.00240073,0.00240184,0.00240288,0.00240385,0.00240489,0.00240597,0.00240695,0.00240787,0.00240872,0.00240953,0.00241046,0.00241129,0.00241228,0.00241313,0.00241411,0.00241502,0.00241603,0.00241712,0.00241811,0.00241934,0.00242034,0.00242141,0.00242256,0.00242369,0.00242488,0.00242598,0.00242723,0.00242845,0.00242958,0.00243082,0.00243182,0.00243282,0.00243389,0.00243485,0.00243593,0.00243699,0.00243818,0.00243921,0.00244038,0.00244162,0.00244278,0.00244386,0.002445,0.00244621,0.00244734,0.00244858,0.00244991,0.0024512,0.00245246,0.00245379,0.00245524,0.00245665,0.00245774,0.00245913,0.00246048,0.00246179,0.00246295,0.00246419,0.00246564,0.00246697,0.00246825,0.00246954,0.0024709,0.00247228,0.00247353,0.002475,0.00247639,0.00247812,0.0024795,0.00248075,0.00248215,0.00248359,0.00248517,0.00248655,0.00248793,0.00248927,0.00249072,0.00249223,0.00249351,0.00249512,0.00249648,0.00249801,0.00249976,0.00250133,0.00250293,0.00250464,0.00250618,0.00250758,0.00250932,0.00251113,0.00251256,0.00251406,0.00251567,0.00251749,0.00251919,0.00252051,0.00252197,0.00252351,0.00252518,0.00252675,0.00252834,0.00252988,0.00253136,0.00253307,0.00253469,0.00253652,0.0025383,0.0025403,0.00254212,0.00254387,0.00254575,0.00254754,0.00254942,0.0025514,0.00255316,0.00255534,0.0025573,0.00255935,0.00256123,0.00256313,0.00256486,0.00256668,0.00256838,0.00257033,0.00257235,0.00257408,0.00257612,0.00257812,0.00258004,0.00258208,0.00258394,0.00258613,0.00258805,0.00259035,0.00259238,0.00259449,0.00259664,0.00259868,0.00260091,0.0026033,0.00260538,0.00260742,0.00260951,0.00261187,0.00261382,0.00261579,0.00261808,0.00262029,0.00262253,0.00262481,0.00262695,0.00262929,0.00263173,0.00263396,0.0026362,0.00263851,0.00264069,0.00264344,0.0026458,0.00264821,0.00265024,0.00265247,0.00265488,0.00265735,0.00265965,0.00266185,0.00266402,0.00266663,0.00266919,0.00267144,0.00267383,0.00267621,0.002679,0.00268127,0.00268393,0.00268632,0.00268891,0.00269145,0.00269389,0.00269661,0.00269912,0.00270165,0.00270404,0.0027067,0.00270939,0.00271195,0.00271455,0.00271722,0.00271969,0.00272252,0.0027254,0.002728,0.00273082,0.00273369,0.00273648,0.00273952,0.00274227,0.00274511,0.00274816,0.00275103,0.00275394,0.00275687,0.00275977,0.00276241,0.00276495,0.00276767,0.00277064,0.00277351,0.00277641,0.00277909,0.00278202,0.00278452,0.00278729,0.00278998,0.00279275,0.0027958,0.00279878,0.0028016,0.00280472,0.00280771,0.0028109,0.00281415,0.00281717,0.00282038,0.00282346,0.00282626,0.00282919,0.00283207,0.00283518,0.00283843,0.00284122,0.0028439,0.00284708,0.00284994,0.00285294,0.00285587,0.00285848,0.00286126,0.0028643,0.00286701,0.00287003,0.00287284,0.00287589,0.00287886,0.00288176,0.00288469,0.00288762,0.0028909,0.00289392,0.00289664,0.00289945,0.00290244,0.00290543,0.00290824,0.00291089,0.00291354,0.00291634,0.00291902,0.00292172,0.00292413,0.0029271,0.00292984,0.002933,0.0029355,0.00293802,0.0029405,0.00294333,0.00294613,0.00117372,0.000467584,0.000186264,7.42093e-05,2.95642e-05,1.17781e-05,4.69226e-06,1.86919e-06,7.44696e-07,2.96676e-07;PREND=3.02026e-15,4.78471e-14,7.57973e-13,1.20067e-11,1.90178e-10,3.01282e-09,4.77324e-08,7.56151e-07,1.19779e-05,0.000189743,0.00300582,0.0030039,0.00300127,0.00299837,0.00299541,0.00299255,0.00299047,0.00298793,0.00298555,0.00298267,0.00298033,0.0029782,0.00297579,0.00297295,0.00297017,0.00296769,0.00296562,0.0029632,0.00296072,0.00295816,0.00295557,0.00295308,0.00295062,0.0029479,0.0029447,0.00294211,0.00293963,0.00293734,0.00293449,0.00293173,0.00292904,0.0029266,0.00292425,0.00292122,0.00291848,0.00291567,0.00291333,0.00291077,0.00290832,0.00290537,0.00290246,0.00289957,0.00289704,0.00289385,0.00289114,0.0028884,0.00288549,0.00288284,0.00287995,0.00287726,0.00287463,0.00287144,0.00286885,0.00286569,0.0028629,0.00286073,0.00285801,0.00285472,0.00285196,0.00284879,0.00284594,0.00284346,0.00284056,0.00283777,0.00283502,0.0028323,0.00282939,0.00282645,0.00282353,0.00282053,0.00281778,0.00281457,0.00281189,0.00280891,0.00280615,0.00280355,0.00280057,0.00279736,0.00279468,0.00279185,0.00278916,0.00278699,0.0027844,0.00278164,0.00277904,0.00277575,0.00277311,0.00277044,0.00276776,0.00276461,0.00276186,0.00275906,0.00275677,0.00275435,0.00275168,0.00274942,0.00274617,0.00274363,0.00274107,0.00273819,0.0027355,0.00273297,0.00273059,0.00272816,0.00272563,0.00272344,0.00272085,0.00271833,0.00271609,0.00271369,0.00271133,0.00270894,0.0027064,0.00270412,0.00270171,0.0026994,0.00269703,0.00269418,0.00269178,0.00268936,0.00268728,0.00268506,0.00268312,0.00268036,0.00267787,0.00267564,0.00267347,0.00267118,0.00266877,0.00266653,0.00266441,0.00266271,0.00266054,0.00265825,0.0026559,0.00265383,0.00265161,0.00264968,0.00264732,0.00264539,0.00264295,0.00264098,0.00263869,0.00263636,0.00263434,0.00263215,0.00263023,0.00262822,0.00262628,0.00262422,0.00262236,0.00262036,0.00261831,0.00261628,0.00261451,0.00261257,0.00261059,0.00260871,0.00260687,0.00260501,0.00260267,0.00260094,0.00259868,0.00259649,0.00259478,0.00259308,0.00259143,0.00258955,0.00258721,0.00258551,0.00258347,0.00258174,0.00258005,0.00257832,0.00257656,0.00257497,0.00257335,0.00257165,0.00256984,0.00256803,0.00256624,0.00256436,0.00256254,0.0025607,0.00255925,0.00255792,0.00255616,0.00255439,0.0025528,0.00255149,0.00254971,0.00254832,0.00254703,0.00254555,0.00254428,0.00254283,0.00254166,0.00254023,0.00253862,0.00253728,0.002536,0.00253436,0.00253252,0.00253096,0.00252943,0.00252819,0.00252656,0.00252508,0.00252379,0.00252243,0.00252084,0.00251975,0.00251831,0.00251666,0.00251521,0.00251391,0.00251267,0.00251144,0.00251024,0.00250897,0.00250758,0.00250646,0.00250525,0.00250396,0.00250248,0.00250084,0.00249962,0.00249836,0.00249712,0.00249567,0.00249435,0.00249341,0.00249217,0.00249093,0.00248988,0.00248849,0.00248716,0.00248589,0.00248488,0.00248398,0.00248264,0.00248155,0.00248047,0.00247944,0.00247824,0.00247724,0.00247609,0.00247492,0.00247375,0.00247262,0.00247147,0.0024706,0.00246888,0.0024678,0.00246675,0.00246572,0.00246487,0.00246414,0.00246316,0.0024619,0.00246083,0.00245995,0.00245894,0.002458,0.0024569,0.00245599,0.00245492,0.0024538,0.00245275,0.0024515,0.00245035,0.0024494,0.00244835,0.00244731,0.00244625,0.00244542,0.00244473,0.0024439,0.00244295,0.00244172,0.00244076,0.00243982,0.00243907,0.00243812,0.00243725,0.00243639,0.00243555,0.00243464,0.0024338,0.00243277,0.00243145,0.00243048,0.00242965,0.0024285,0.00242736,0.00242654,0.00242575,0.00242483,0.00242391,0.00242298,0.00242193,0.00242078,0.00241981,0.00241862,0.00241771,0.00241663,0.0024155,0.0024146,0.00241367,0.0024125,0.00241116,0.00241008,0.00240902,0.00240729,0.00240548,0.00240419,0.00240253,0.00240081,0.00239908,0.00239741,0.00239517,0.0023929,0.00239052,0.00238798,0.00238505,0.00238189,0.00237904,0.00237565,0.00237201,0.00236829,0.00236374,0.00235923,0.00235403,0.00234858,0.00234288,0.00233677,0.00233038,0.00232438,0.0023171,0.00230981,0.00230198,0.00229329,0.00228476,0.00227551,0.00226594,0.00225598,0.00224564,0.00223392,0.00222138,0.00220881,0.00219607,0.00218362,0.00217038,0.00215639,0.00214182,0.00212751,0.00211102,0.00209334,0.00207535,0.00205704,0.00203711,0.00201595,0.00199504,0.00197257,0.00194934,0.0019241,0.00189856,0.0018717,0.00184359,0.00181463,0.00178446,0.00175283,0.0017206,0.0016859,0.00165096,0.00161563,0.00157919,0.00154094,0.00150137,0.00146059,0.00141945,0.00137825,0.0013367,0.00129363,0.00124908,0.00120422,0.00115974,0.00111459,0.00106967,0.00102485,0.000979928,0.000935386,0.000891343,0.000847616,0.000805325,0.000763099,0.000721605,0.000680876,0.000641258,0.000602415,0.000564516,0.000529212,0.000494813,0.000461726,0.000429703,0.000398541,0.000369522,0.000342268,0.000316742,0.000292816,0.000270237,0.000249087,0.000228998,0.000210083,0.000192775,0.000176194,0.000160682,0.000146541,0.000133475,0.000121171,0.000110027,9.98983e-05,9.04025e-05,8.1898e-05,7.37485e-05,6.64247e-05,5.991e-05,5.34662e-05,4.79254e-05,4.28124e-05,3.80863e-05,3.37871e-05,2.98488e-05,2.64158e-05,2.31778e-05,2.02341e-05,1.77006e-05,1.53314e-05,1.32501e-05,1.13287e-05,9.72139e-06,8.17381e-06,6.84904e-06,5.73097e-06,4.74769e-06,3.91083e-06,3.20076e-06,2.6024e-06,2.08961e-06,1.69223e-06,1.36759e-06,1.08247e-06,8.47588e-07,6.69622e-07,5.17384e-07,4.00954e-07,3.11072e-07,2.34849e-07,1.76165e-07,1.31673e-07,9.73364e-08,7.19841e-08,5.32892e-08,3.90585e-08,2.83402e-08,2.02418e-08,1.4579e-08,1.06446e-08,7.59531e-09,5.37474e-09,3.79538e-09,2.64724e-09,1.84573e-09,1.29566e-09,9.10203e-10,6.14793e-10,4.26199e-10,2.87287e-10,1.936e-10,1.30732e-10,9.00388e-11,6.2542e-11,4.32799e-11,2.90205e-11,1.95416e-11,1.3365e-11,8.69465e-12,5.75237e-12,3.71099e-12,2.54394e-12,1.73591e-12,1.14175e-12,7.46542e-13,4.68409e-13,3.08728e-13,1.94589e-13,1.17053e-13,7.58227e-14,4.59278e-14,2.89681e-14,1.97988e-14,1.19182e-14,7.29947e-15,4.37438e-15  GT:SU:PE:SR ./.:6:6:0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And here is the same event as a BEDPE:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    1584078 1584538 chr1    1651058 1651572 11  .   -   +   DUP .   SVTYPE=DUP;SVLEN=66541;END=1651069;STRANDS=-+:6;IMPRECISE;CIPOS=-449,10;CIEND=-10...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/cancerit/BRASS/wiki/BEDPE&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/cancerit/BRASS/wiki/BEDPE&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is a table describing how SVs are represented in BEDPE format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Heading Type    Description&#xA;chr1    String  Chromosome of lower coordinate&#xA;start1  0-based int Start coordinate of lower coordinate&#xA;end1    1-based int End coordinate of lower coordinate&#xA;chr2    String  Chromosome of high coordinate&#xA;start2  0-based int Start coordinate of high coordinate&#xA;end2    1-based int End coordinate of high coordinate&#xA;id/name String  ID of event, correlates with VCF&#xA;brass_score int Number of aberrant pairs contributing to the rearrangement group.&#xA;strand1 [+-]    Strand of end in 'genomic' context - see table&#xA;strand2 [+-]    Strand of end in 'genomic' context&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1770" LastEditorUserId="298" LastEditDate="2019-07-07T15:50:15.873" LastActivityDate="2019-07-07T15:53:37.363" Title="Is there a standard tool used to convert a VCF to a BEDPE?" Tags="|vcf|structural-variation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8916" PostTypeId="1" CreationDate="2019-07-05T19:45:18.970" Score="0" ViewCount="74" Body="&lt;p&gt;So I have a list of several thousand RNA transcripts, and I am trying to find out how they are spliced. Right now I am running &lt;code&gt;matchLRpatterns()&lt;/code&gt; from the Biostrings package with a max gap length of 0, after running a &lt;code&gt;matchpattern&lt;/code&gt; function to categorize the transcripts by donor sites (where the first cut in an RNA transcript is made to cut out introns). But applying that to several thousand transcripts is quite time consuming, when you have 5 donor sequences to match to 9 acceptor sequences in 100 nucleotide long transcripts. Instead, I am thinking to sort by donor, remove all that comes before the splice junction by converting it to a string then using &lt;code&gt;gsub&lt;/code&gt;, then using &lt;code&gt;matchpattern&lt;/code&gt; again, but I only want to check the first 8 nucleotides after the splice junction for the consensus sequence that follows my list of acceptor sites. If I just run &lt;code&gt;matchpattern&lt;/code&gt; without shortening the length of the transcript, I will just find acceptor sites in the RNA transcript, not the acceptor site at the splice junction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking &lt;code&gt;lapply&lt;/code&gt; the &lt;code&gt;gsub&lt;/code&gt; function to the transcripts to remove up to the splice junction, then &lt;code&gt;lapply&lt;/code&gt; the &lt;code&gt;strsplit&lt;/code&gt; function to make a list of lists of groups of 1 character strings, &lt;code&gt;lapply&lt;/code&gt; the &lt;code&gt;head&lt;/code&gt; function to get the first 8 single character strings in each list, &lt;code&gt;lapply&lt;/code&gt; the &lt;code&gt;paste&lt;/code&gt; function to make them one string again, &lt;code&gt;lapply&lt;/code&gt; the &lt;code&gt;DNAString&lt;/code&gt; function to make the sequences readable by matchpattern, then lapply the &lt;code&gt;matchpattern&lt;/code&gt; function to check the acceptor sequence at the splice junction. Is there a better way? This feels a bit clunky and roundabout way of doing it. I know gsub can remove characters after a certain character or sequence of characters, but I am not sure what the first sequence in each transcript is at the splice junction. Can I still use gsub, but tell it to remove characters after a certain number of characters? Or better yet, is there a way to get matchpattern to only check the first x nucleotides in a sequence?&lt;/p&gt;&#xA;" OwnerUserId="4823" OwnerDisplayName="Joe Kowzun" LastActivityDate="2019-07-06T14:43:32.370" Title="Reducing a string to a certain amount of the first characters in R" Tags="|r|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8917" PostTypeId="1" CreationDate="2019-07-06T02:59:31.850" Score="0" ViewCount="497" Body="&lt;p&gt;Using python 3 I need to process qPCR sequencing raw data outputs by searching for the first occurrence of a user defined string and then making a new data frame using all lines after that string. I am trying to find solutions in the pandas doc but so far unsuccessful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a raw output .csv file that I need to process. (couldn't paste complete csv as exceeds character limit, this is lines 40-50 and am hoping this text is useful?). I need to tell pandas to create a new data frame that 1. starts at the line containg the first occurance of str(&quot;Sample Name&quot;) with that line as header and containing all lines following. And then 2., only including columns (&quot;Sample Name&quot;), (&quot;Target Name&quot;), (&quot;CT&quot;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone please help me so that I can use python to help me analyze biological data?&#xA;Many thanks,&#xA;Luke &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;40,Quantification Cycle Method,Ct,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,&#xA;&#xA;41,Signal Smoothing On,true,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,&#xA;&#xA;42,Stage where Melt Analysis is performed,Stage3,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,&#xA;&#xA;43,Stage/ Cycle where Ct Analysis is performed,&quot;Stage2, Step2&quot;,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,&#xA;&#xA;44,User Name,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,&#xA;&#xA;45,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,&#xA;&#xA;46,Well,Well Position,Omit,Sample Name,Target Name,Task,Reporter,Quencher,Quantity,Quantity Mean,SE,RQ,RQ Min,RQ Max,CT,Ct Mean,Ct SD,Delta Ct,Delta Ct Mean,Delta Ct SD,Delta Ct SE,Delta Delta Ct,Automatic Ct Threshold,Ct Threshold,Automatic Baseline,Baseline Start,Baseline End,Amp Status,Comments,Cq Conf,CQCONF,HIGHSD,OUTLIERRG,Tm1,Tm2,Tm3,Tm4&#xA;&#xA;47,1,A1,False,WT1,AtTubulin,UNKNOWN,SYBR,None,,,,,,,23.357698440551758,23.4766845703125,0.5336655378341675,,,,,,True,20959.612776965325,True,3,17,Amp,,0.9588544573203085,N,Y,N,81.40960693359375,,,&#xA;&#xA;48,2,A2,False,WT1,AtTubulin,UNKNOWN,SYBR,None,,,,,,,24.05980110168457,23.4766845703125,0.5336655378341675,,,,,,True,20959.612776965325,True,3,15,Amp,,0.9592687354496955,N,Y,N,81.40960693359375,,,&#xA;&#xA;49,3,A3,False,WT1,AtTubulin,UNKNOWN,SYBR,None,,,,,,,23.012556076049805,23.4766845703125,0.5336655378341675,,,,,,True,20959.612776965325,True,3,16,Amp,,0.9592714462250367,N,Y,N,81.40960693359375,,,&#xA;&#xA;50,4,A4,False,fla11fla12-1,AtTubulin,UNKNOWN,SYBR,None,,,,,,,23.803699493408203,24.419523239135742,0.5669151544570923,,,,,,True,20959.612776965325,True,3,17,Amp,,0.9671570584141241,N,Y,N,81.40960693359375,,,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the code that I have so far:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;import numpy as np&#xA;&#xA;import matplotlib.pyplot as plt&#xA;&#xA;data = pd.read_excel (&quot;2019-02-27_161601 AtWAKL8 different version expressions.xls&quot;, sheet_name='Results').fillna(0)&#xA;&#xA;data.to_csv('df1' + '.csv', index=True)&#xA;&#xA;df1 = pd.read_csv (&quot;df1.csv&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4822" LastEditorUserId="4822" LastEditDate="2019-07-09T13:02:30.423" LastActivityDate="2020-08-02T19:01:55.170" Title="Raw Data frame manipulation in python" Tags="|python|data-retrieval|jupyter|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8918" PostTypeId="2" ParentId="8916" CreationDate="2019-07-06T14:43:32.370" Score="1" Body="&lt;p&gt;Is there any particular reason why you want to do this via pattern matching in R? This sounds like something that SuperTranscripts and Lace might be useful for:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-017-1284-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-017-1284-1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Finally, Lace will annotate each superTranscript with blocks either by using the graph structure itself or alternatively using a script called Mobius (packaged with Lace) which infers the annotation from spliced junctions discovered when mapping reads to the superTranscript.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastActivityDate="2019-07-06T14:43:32.370" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8920" PostTypeId="2" ParentId="4827" CreationDate="2019-07-06T15:57:18.027" Score="6" Body="&lt;p&gt;Now there's another option, the &lt;code&gt;ensembl_rest&lt;/code&gt; module, a thin wrapper around the &lt;a href=&quot;http://rest.ensembl.org&quot; rel=&quot;noreferrer&quot;&gt;Ensembl REST API&lt;/a&gt; to simplify its usage and make it more pythonic. You can find the documentation &lt;a href=&quot;https://ensemblrest.readthedocs.io&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To clarify things, I'm the creator and maintainer, but still think it's a legitimate alternative.&lt;/p&gt;&#xA;" OwnerUserId="4824" LastActivityDate="2019-07-06T15:57:18.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8922" PostTypeId="2" ParentId="8915" CreationDate="2019-07-07T09:30:24.180" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/fritzsedlazeck/SURVIVOR&quot; rel=&quot;nofollow noreferrer&quot;&gt;SURVIVOR&lt;/a&gt; has a subcommand to convert an SV vcf to bed, although I'm not sure if it's actual bedpe format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To install:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;git clone https://github.com/fritzsedlazeck/SURVIVOR.git&#xA;cd SURVIVOR/Debug&#xA;make&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./SURVIVOR vcftobed input.vcf &lt;span class=&quot;math-container&quot;&gt;$minSize $&lt;/span&gt;maxSize output.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you run this on your example file, you get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ./SURVIVOR vcftobed foo.vcf 1 10000000000000000 bar.bed; cat bar.bed &#xA;chr1    1584528 1584528 chr1    1651069 1651069 11  ,   -   +   DUP&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="681" LastEditorUserId="298" LastEditDate="2019-07-07T15:53:37.363" LastActivityDate="2019-07-07T15:53:37.363" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8924" PostTypeId="1" CreationDate="2019-07-08T02:35:18.333" Score="1" ViewCount="261" Body="&lt;p&gt;I intend to see data points distribution (each reows) within labeled groups (different batchs such as 1,2,3, and so on) in 3D scatter plot, because I want to see the distribution of the data points and want to see how similar each group of data points in 3D space. To do so, I used &lt;code&gt;scatterplot3d&lt;/code&gt; package from CRAN to get 3D scatter plot, didn't get correct plot for my data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;reproducible data:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the reproducible data that simulated from the original dataset.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;persons_df &amp;lt;- data.frame(ID= LETTERS[1:20], ages=sample(1:50,20, replace = FALSE),&#xA;                    batch=c(rep(1,5), rep(2,6),rep(3,4),rep(4,5)),&#xA;                    platform=c(rep(&quot;CATH&quot;,12), rep(&quot;ANN&quot;,8)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;basically, I want to render 3D plot for the data points that belongs to the same batch with the same color (1 is red, 2 is blue, 3 is green, 4 is yellow).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;this was my attempt to render PCA plot&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;require(scatterplot3d)&#xA;par(mar=c(4,4,4,4), cex=1.0, cex.main=0.8, cex.axis=0.8)&#xA;scatterplot3d(persons_df$batch)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but above code doesn't work because first I should group data points that belongs to different batches then render them in 3D space. This is not very intuitive to me how to do it in R? Any idea to render possible 3D scatter plot for my data? any thought?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;desired 3D plot&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want 3D plot something like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fkTH6.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fkTH6.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;how can I make this happen? any idea to get this done?&lt;/p&gt;&#xA;" OwnerUserId="4729" LastEditorUserId="4729" LastEditDate="2019-07-11T00:06:47.143" LastActivityDate="2019-07-11T00:06:47.143" Title="How can I correctly get 3D scatter plot for micro-array pheno data in R [updated]?" Tags="|r|microarray|ggplot2|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8926" PostTypeId="1" CreationDate="2019-07-08T10:41:11.680" Score="0" ViewCount="955" Body="&lt;p&gt;How do I solve the problem of loading the  bioconductor &lt;strong&gt;Chipseeker&lt;/strong&gt; package? I have installed in the following way and I having an error while loading it and I do not know why.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BiocManager::install(&quot;ChIPseeker&quot;)&#xA;BiocManager::install(&quot;TxDb.Hsapiens.UCSC.hg38.knownGene&quot;)&#xA;BiocManager::install(&quot;TxDb.Hsapiens.UCSC.hg19.knownGene&quot;)&#xA;&#xA;library(ChIPseeker) # produces and error&#xA;&#xA;Error: package or namespace load failed for 'ChIPseeker': &#xA;  'TxDb.Hsapiens.UCSC.hg19.knownGene' is not exported from 'namespace: TxDb.Hsapiens.UCSC.hg19.knownGene'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for any help on how to go about this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;session info:&#xA;&#xA;R version 3.5.1 (2018-07-02)&#xA;Platform: x86_64-w64-mingw32/x64 (64-bit)&#xA;Running under: Windows 7 x64 (build 7601) Service Pack 1&#xA;&#xA;Matrix products: default&#xA;&#xA;locale:&#xA;[1] LC_COLLATE=German_Germany.1252  LC_CTYPE=German_Germany.1252    LC_MONETARY=German_Germany.1252&#xA;[4] LC_NUMERIC=C                    LC_TIME=German_Germany.1252    &#xA;&#xA;attached base packages:&#xA;[1] stats4    parallel  stats     graphics  grDevices utils     datasets  methods   base     &#xA;&#xA;other attached packages:&#xA; [1] BiocInstaller_1.32.1                    TxDb.Hsapiens.UCSC.hg19.knownGene_3.2.2&#xA; [3] TxDb.Hsapiens.UCSC.hg38.knownGene_3.4.0 GenomicFeatures_1.34.8                 &#xA; [5] AnnotationDbi_1.44.0                    GenomicRanges_1.34.0                   &#xA; [7] GenomeInfoDb_1.18.2                     IRanges_2.16.0                         &#xA; [9] S4Vectors_0.20.1                        cellrangerRkit_1.1.0                   &#xA;[11] Rmisc_1.5                               plyr_1.8.4                             &#xA;[13] lattice_0.20-35                         bit64_0.9-7                            &#xA;[15] bit_1.1-14                              ggplot2_3.1.1                          &#xA;[17] RColorBrewer_1.1-2                      Biobase_2.42.0                         &#xA;[19] BiocGenerics_0.28.0                     Matrix_1.2-14                          &#xA;&#xA;loaded via a namespace (and not attached):&#xA;Error in x[[&quot;Version&quot;]] : subscript out of bounds&#xA;In addition: Warning message:&#xA;In FUN(X[[i]], ...) : DESCRIPTION von Paket 'digest' fehlt oder zerstört&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3968" LastActivityDate="2020-11-29T17:04:50.447" Title="Error: package or namespace load failed for 'ChIPseeker'" Tags="|r|bioconductor|chip-seq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8927" PostTypeId="1" CreationDate="2019-07-08T14:59:04.783" Score="3" ViewCount="4965" Body="&lt;p&gt;I have sequenced a bacterial genome with a GridIon from ONT. Basically what I want to check is whether or not trimming 50 bps at the beginning of the reads will improve alignment against the reference genome and ultimately the call of a consensus sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For that what I have done so far is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;concatenated all the fastqs generated from the Gridion into a single (very large) fastq file: &lt;code&gt;ref494_cat_all.fastq&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;aligned this concatenated fastq against the reference genome &lt;code&gt;ref_genbank.fasta&lt;/code&gt; with &lt;code&gt;minimap2&lt;/code&gt;: &lt;code&gt;~/Downloads/minimap2-2.17_x64-linux/minimap2 -ax map-ont ref_genbank/ref_genbank.fasta ref494_cat_all.fastq &amp;gt; ref494_aligned_trim50.sam&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Performed the general &lt;code&gt;samtools&lt;/code&gt; steps to covert sam to bam, index and sort&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Then, in order to get the consensus sequence I first try to generate a &lt;code&gt;.vcf.gz&lt;/code&gt; file, according to &lt;a href=&quot;https://samtools.github.io/bcftools/howtos/variant-calling.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools mpileup -f ref_genbank/ref_genbank.fasta ref494_aligned.sorted.bam | bcftools call -mv -Oz -o calls.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Immediatelly after calling the command above I got:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Note: none of --samples-file, --ploidy or --ploidy-file given, assuming all sites are diploid&#xA;[mpileup] 1 samples in 1 input files&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, after a minute or two, I got:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Failed to open -: unknown file type&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm pretty sure that there's no issue with &lt;code&gt;.bam&lt;/code&gt; file, as I could visualise the alignment with IGV or tablet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know what the issue can be?&lt;/p&gt;&#xA;" OwnerUserId="3452" LastActivityDate="2021-03-30T19:04:15.537" Title="Error while calling bcftools mpileup - Failed to open -: unknown file type" Tags="|sequence-alignment|sam|vcf|samtools|nanopore|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="8928" PostTypeId="2" ParentId="8926" CreationDate="2019-07-08T15:34:24.403" Score="1" Body="&lt;p&gt;Simple, use the latest version of R and Bioconductor, now on version 3.6.1 and Bioconductor 3.9. Those are the versions that are supported. If you can't upgrade R, check that &lt;code&gt;BiocManager::valid()&lt;/code&gt; is true, you might have ended in some inconsistency between Bioconductor versions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at your &lt;code&gt;sessionInfo&lt;/code&gt;, you seem to have some problems with the installed libraries, check and correct those too if possible&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2019-07-08T15:34:24.403" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8929" PostTypeId="1" CreationDate="2019-07-08T16:02:54.707" Score="1" ViewCount="39" Body="&lt;p&gt;I have single cell data which I have analysed for differential expression. In my experiment,  I subjected two groups of mice (a control and a treatment group) to treatment that will lead to the proliferation of my cells of interest in the treated mice. Other scientists have said that the increase of cell numbers caused by this treatment results from increased recruitment from precursor cells from the bone marrow while other scientists argue that the increase in cell number is due to local proliferation.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish to address this question computationally, based on my single cell rna seq data using R or any other method. I have googled but I can find no suitable tool. Does it suffice to test the expression of known cell migration markers, produce a migration score  and judge based on the migration scores of my proliferating cells?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, I will be very grateful for suggestion of a computational tool, method or package that can help me address this question even indirectly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="3968" LastActivityDate="2020-07-27T20:25:05.410" Title="Is there a computational method/method/way to predict if a cell population in a tissue immigrated or is enriched locally?" Tags="|scrnaseq|bioconductor|software-recommendation|computation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8930" PostTypeId="2" ParentId="8927" CreationDate="2019-07-08T16:15:40.243" Score="1" Body="&lt;p&gt;The &lt;code&gt;-&lt;/code&gt; is usually used to mean standard input when reading data. So the message would suggest that the second &lt;code&gt;bcftools&lt;/code&gt; command fails to read from stdin, so it fails to read the output of the first &lt;code&gt;bcftools&lt;/code&gt; command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ echo &quot;foo bar&quot; | /ref/software/bcftools-1.8/bcftools call -mv -Oz -o calls.vcf.gz&#xA;Note: none of --samples-file, --ploidy or --ploidy-file given, assuming all sites are diploid&#xA;Failed to open -: unknown file type&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So there is something wrong with either your input file or with the output produced when running the first &lt;code&gt;bcftools&lt;/code&gt; command on that input file. We can't really help more unless you share the file. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-07-08T16:15:40.243" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8931" PostTypeId="1" AcceptedAnswerId="8933" CreationDate="2019-07-08T18:32:50.893" Score="4" ViewCount="5490" Body="&lt;p&gt;I was looking at a bam file in the IGV viewer and saw:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7JGv1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7JGv1.png&quot; alt=&quot;IGV screenshot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do the different read colors mean? Why is one read a light blue color, another green, another aquamarine (?), another purple and another two orange? What do these different read colors represent? The &lt;a href=&quot;http://software.broadinstitute.org/software/igv/UserGuide&quot; rel=&quot;nofollow noreferrer&quot;&gt;IGV docs&lt;/a&gt; have sections on interpreting color &lt;a href=&quot;http://software.broadinstitute.org/software/igv/interpreting_insert_size&quot; rel=&quot;nofollow noreferrer&quot;&gt;by insert size&lt;/a&gt;, p&lt;a href=&quot;http://software.broadinstitute.org/software/igv/interpreting_pair_orientations&quot; rel=&quot;nofollow noreferrer&quot;&gt;air orientation&lt;/a&gt;, &lt;a href=&quot;http://software.broadinstitute.org/software/igv/interpreting_bisulfite_mode&quot; rel=&quot;nofollow noreferrer&quot;&gt;bisulphide mode&lt;/a&gt;, &lt;a href=&quot;http://software.broadinstitute.org/software/igv/splice_junctions&quot; rel=&quot;nofollow noreferrer&quot;&gt;splice junctions&lt;/a&gt; and &lt;a href=&quot;http://software.broadinstitute.org/software/igv/Sashimi&quot; rel=&quot;nofollow noreferrer&quot;&gt;sashimi plot&lt;/a&gt;, but none of them seems to be describing what I'm looking at there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally, where are all these default behaviors documented? Are they somewhere in &lt;a href=&quot;http://software.broadinstitute.org/software/igv/UserGuide&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://software.broadinstitute.org/software/igv/UserGuide&lt;/a&gt; and I'm missing them?&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="96" LastEditDate="2019-07-09T13:40:35.803" LastActivityDate="2019-07-09T19:07:58.667" Title="What do the read colors in IGV mean?" Tags="|sequence-alignment|visualization|igv|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8932" PostTypeId="1" AcceptedAnswerId="8940" CreationDate="2019-07-08T19:04:20.693" Score="0" ViewCount="570" Body="&lt;p&gt;The &lt;code&gt;RunPCA()&lt;/code&gt; in Seurat 3.0 function has &lt;code&gt;seed.use=42&lt;/code&gt; by default. How does one choose a seed number? Is this chosen simply because it is the Answer to the Ultimate Question of Life, the Universe, and Everything? Or maybe Lewis Carroll's curious obsession with the number?&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2019-07-09T12:42:06.477" Title="Why is 42 the seed used in RunPCA() in Seurat?" Tags="|r|seurat|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8933" PostTypeId="2" ParentId="8931" CreationDate="2019-07-08T19:18:18.620" Score="5" Body="&lt;p&gt;Could it be that these are reads with their mates found on other chromosomes? That's &lt;a href=&quot;http://software.broadinstitute.org/software/igv/interpreting_insert_size&quot; rel=&quot;nofollow noreferrer&quot;&gt;the part of the IGV documentation&lt;/a&gt; which seems to make sense in this context:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/l8dSy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/l8dSy.png&quot; alt=&quot;color codes from IGV docs&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;for paired end reads that are coded by the chromosome on which their mates can be found&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="650" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2019-07-09T19:07:58.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8935" PostTypeId="2" ParentId="8913" CreationDate="2019-07-08T22:37:11.443" Score="1" Body="&lt;p&gt;Try the protocol here for generating a different (smaller) dispersion estimate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.huber.embl.de/users/klaus/Teaching/DESeq2Predoc2014.html#inspection-and-correction-of-pvalues&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.huber.embl.de/users/klaus/Teaching/DESeq2Predoc2014.html#inspection-and-correction-of-pvalues&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-07-08T22:37:11.443" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8936" PostTypeId="2" ParentId="8927" CreationDate="2019-07-09T05:43:51.213" Score="1" Body="&lt;p&gt;It's possible you're looking at instructions for a recent version of samtools / bcftools, whereas your computer has an older version. Can you try &lt;code&gt;samtools mpileup -g -f ...&lt;/code&gt; to output the pileup as BCF format instead of &lt;code&gt;bcftools mpileup -f ...&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: it's good to know that the samtools / bcftools pipelines are at least producing consistent results, which suggests that the error is not associated with the output format on the mpileup side of the pipeline, and also good to see that you've identified that this was a memory issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most frequent cause of memory issues I've had with sorted BAM files has been trying to figure out variants in extremely high coverage data (e.g. over 1000X), which would make sense for a bacterial isolate sequenced on a GridION. I use &lt;a href=&quot;https://gitlab.com/gringer/bioinfscripts/blob/master/samNormalise.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;my own script&lt;/a&gt; to carry out a reservoir sampling on mapped BAM format files, which reduces coverage down to a more manageable level. Here's an example command to normalise to 100X coverage where possible:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view reads.bam | ./samNormalise.pl -coverage 100 | \&#xA;   samtools view -b &amp;gt; reads_normalised.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2019-07-09T14:35:02.110" LastActivityDate="2019-07-09T14:35:02.110" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8937" PostTypeId="1" AcceptedAnswerId="8945" CreationDate="2019-07-09T08:39:54.447" Score="1" ViewCount="280" Body="&lt;p&gt;The module &lt;code&gt;DigitalExpression&lt;/code&gt; which is part of the popular &lt;a href=&quot;https://github.com/broadinstitute/Drop-seq&quot; rel=&quot;nofollow noreferrer&quot;&gt;Drop-seq tools&lt;/a&gt; digitally count gene transcripts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The manual is not very clear on how exactly it resolves the UMI information. &#xA;Maybe I'm missing something but I was wondering if UMI are used in isolation or in conjunction with the barcode and or transcript?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The manual i.e. &lt;a href=&quot;https://github.com/broadinstitute/Drop-seq/blob/v2.3.0/doc/Drop-seq_Alignment_Cookbook.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Drop-seq Alignment Cookbook&lt;/a&gt; only states:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;To digitally count gene transcripts, a list of UMIs in each gene, within each cell, is assembled, and UMIs&#xA;within edit distance = 1 are merged together. The total number of unique UMI sequences is counted,&#xA;and this number is reported as the number of transcripts of that gene for a given cell.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To clarify my point, I'll use an example. Say my reads are only 4 bp and read 1 has the cell barcode at position 1+2 and UMI at position 3+4.  read 2 constitutes the transcript read. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the following example reads:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seqence1:   read1 = AATT  read2 = ATGC&#xA;seqence2:   read1 = AATT  read2 = TTTT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;UMI and barcode are the same but read2 is different, would they be collapsed to 1 UMI count or 2?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Similarly as case with same UMI and transcript but different cell barcode:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seqence1:   read1 = AATT  read2 = ATGC&#xA;seqence2:   read1 = GGTT  read2 = ATGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also of note, UMI deduplication is the default setting as of the parameter below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;OUTPUT_READS_INSTEAD=Boolean  Output number of reads instead of number of unique molecular barcodes.  Default value:&#xA;                              false. This option can be set to 'null' to clear the default value. Possible values:&#xA;                              {true, false} &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1461" LastActivityDate="2019-07-13T13:05:02.157" Title="How are UMIs used to dedupulicate in Drop-seq tools?" Tags="|scrnaseq|umi|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8938" PostTypeId="1" AcceptedAnswerId="8941" CreationDate="2019-07-09T11:22:10.347" Score="6" ViewCount="2526" Body="&lt;p&gt;I thought I had figured this one out. But apparently not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to convert a BAM file of paired-end alignments to two FASTQ files of paired reads to realign them, with a twist: I only want reads that fall within a defined region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I am using the following command (the region is an example; it also fails on different regions; at any rate, this is on hs37d5 but presumably the issue would exist for other references too):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;samtools view -b -u -@ 48 alignments.bam 21:31,831,502-31,896,094 \&#xA;| samtools collate -n 128 -u -O -@ 48 - /tmp/bam-to-fastq- \&#xA;| samtools fastq -F 0x900 -@ 48 \&#xA;    -0 /dev/null -1 reads_R1.fastq.gz -2 reads_R2.fastq.gz -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Apart from the region filtering, this command is essentially the same that &lt;a href=&quot;https://twitter.com/lh3lh3/status/1132756684789768202&quot; rel=&quot;noreferrer&quot;&gt;word of god&lt;/a&gt; tells us to use.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resulting files are not properly paired:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;⟩⟩⟩ diff &amp;lt;(gunzip -c reads_R1.fastq.gz | sed -n -e 1~4p -e 10q) \&#xA;1 ⟩   &amp;lt;(gunzip -c reads_R2.fastq.gz | sed -n -e 1~4p -e 10q)&#xA;1,2d0&#xA;&amp;lt; @A00346:17:H52H7DSXX:3:2142:32316:24518&#xA;&amp;lt; @A00346:17:H52H7DSXX:3:1153:6705:26412&#xA;3a2,3&#xA;&amp;gt; @A00346:17:H52H7DSXX:3:1173:9118:3129&#xA;&amp;gt; @A00346:17:H52H7DSXX:3:1109:29414:11757&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As far as I can see from performing a spot check, these are all reads which are marked as paired in the BAM file, but for which only one mate falls within the filtered region.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And running &lt;code&gt;bwa mem&lt;/code&gt; on the files gives me the error:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;[mem_sam_pe] paired reads have different names: […]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How can I ensure that only properly paired reads are included in the FASTQ files?&lt;/p&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2019-07-09T12:54:44.170" LastActivityDate="2019-07-09T14:07:58.503" Title="Convert BAM to properly paired FASTQ files" Tags="|sam|samtools|fastq|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="8939" PostTypeId="2" ParentId="4367" CreationDate="2019-07-09T11:49:30.173" Score="1" Body="&lt;p&gt;SingleR, a novel computational method for unbiased cell type recognition of scRNA-seq:&#xA;&lt;a href=&quot;https://github.com/dviraran/SingleR&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/dviraran/SingleR&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4831" LastActivityDate="2019-07-09T11:49:30.173" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8940" PostTypeId="2" ParentId="8932" CreationDate="2019-07-09T12:42:06.477" Score="3" Body="&lt;p&gt;Given that it is virtually impossible for a human to predict what random numbers will be generated given a certain seed, and what effects they will have for a given application, the choice of the seed can be considered arbitrary.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What matters is that fixing the seed should fix the behaviour of the application. What seed has been chosen does not really matter, as long as it is documented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The choice of &lt;code&gt;42&lt;/code&gt; could therefore well come from one of the reasons you propose. Only the authors of the programs could confirm.&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2019-07-09T12:42:06.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8941" PostTypeId="2" ParentId="8938" CreationDate="2019-07-09T14:07:58.503" Score="6" Body="&lt;p&gt;As noted in the comments, the problem is “some reads fall in the target region but their pairs fall outside it”, leading to non-trivial numbers of singleton reads coming out of &lt;code&gt;samtools collate&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;… | samtools fastq -F 0x900 -@ 48 \&#xA;    -0 /dev/null -1 reads_R1.fastq.gz -2 reads_R2.fastq.gz -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Your &lt;code&gt;samtools fastq&lt;/code&gt; command is not doing anything to siphon off these singleton reads. You should add &lt;code&gt;-s /dev/null&lt;/code&gt; to get rid of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the discussion in &lt;a href=&quot;https://github.com/samtools/samtools/issues/874&quot; rel=&quot;noreferrer&quot;&gt;samtools/samtools#874&lt;/a&gt; (especially the part starting &lt;a href=&quot;https://github.com/samtools/samtools/issues/874#issuecomment-399403984&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;). I have not looked at how the &lt;a href=&quot;https://github.com/samtools/samtools/commit/f69e678bc16ebb93b6f5b16cd23a4ef7af0f7319&quot; rel=&quot;noreferrer&quot;&gt;recent changes from jkbonfield&lt;/a&gt;, which will land in an upcoming samtools version (after 1.9), might affect this.&lt;/p&gt;&#xA;" OwnerUserId="134" LastActivityDate="2019-07-09T14:07:58.503" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8942" PostTypeId="2" ParentId="8917" CreationDate="2019-07-09T14:16:50.747" Score="2" Body="&lt;pre&gt;&lt;code&gt;data = pd.read_excel (&quot;2019-02-27_161601 AtWAKL8 different version expressions.xls&quot;, sheet_name='Results').fillna(0)&#xA;&#xA;data.to_csv('df1' + '.csv', index=True)&#xA;&#xA;df1 = pd.read_csv (&quot;df1.csv&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This doesn't make much sense ... you read in Excel to a dataframe, you then read out to csv - the dataframe doesn't have an index, but you include an index and then read it in again. This could do more harm than good. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway your supervisor is right it is very easy to parse this data using pandas, although there will be more expertise doing this via R. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't want to peer through the gloom of a CSV output, but I would like to highlight how easy it is to parse this stuff into a dataframe. The dataframe '&lt;code&gt;data&lt;/code&gt;' is fine. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Parse&lt;/strong&gt; You can simply remove the top rows of bumf &lt;code&gt;data = data.iloc[6:]&lt;/code&gt; (I might have miss counted, plus see below),&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Headers&lt;/strong&gt; You will probably have to remove the &quot;headers&quot; row to a list and&#xA;then introduce back onto the dataframe using &lt;code&gt;data.columns =&#xA;['header1', 'header2' ...]&lt;/code&gt; ... rather this is the list (array in Perl) which you can automatically generate using a magic &lt;code&gt;to_list&lt;/code&gt; type pandas command which I forget, its not &lt;code&gt;list&lt;/code&gt; - its cuter&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Columns&lt;/strong&gt; You next might use &lt;code&gt;drop&lt;/code&gt; to remove the columns you don't want (that might be alot of columns) or else make a subset of&#xA;the dataframe into a new dataframe, you can also use &lt;code&gt;iloc&lt;/code&gt; for&#xA;columns too.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Index&lt;/strong&gt; You then assign A1, A2, ... etc  as an index &lt;code&gt;data.index = ['A1',&#xA;'A2', ]&lt;/code&gt;...etc..  you can reshape column 3 to automate this&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;to_numeric&lt;/strong&gt; You might have to assign the CT values as floats&#xA;    or better &lt;code&gt;to_numeric&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Extrapolation&lt;/strong&gt; You would then define a subroutine (or whatever they call them in&#xA;Python) so perform the extrapolation and use the &lt;code&gt;apply&lt;/code&gt; command on&#xA;your dataframe and assign it to a new column, where the input is your CT values.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Output&lt;/strong&gt; Finally, issue your &lt;code&gt;to_csv&lt;/code&gt; command &lt;code&gt;to_pickle&lt;/code&gt; is also a good idea because you have a df that is worth importing at a future date.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Plotting&lt;/strong&gt; this is easy using &lt;code&gt;data.plot&lt;/code&gt; which appears to be some sort of inbuilt matplotlib, keep in mind seaborn DONT try using R's &lt;code&gt;ggplot2&lt;/code&gt; within Python using rpy2 .... its a NIGHTMARE to get working and doesn't support the latest &lt;code&gt;ggplot2&lt;/code&gt; (!!!). R&#xA;s &lt;code&gt;ggplot2&lt;/code&gt; is stronger  than seaborn or matplotlib (providing you are working within R), but these two are good for quick graphs.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;If I had time I'd write the code for you, but I'm afraid I'll have to leave it there. I'm still in debt with a Perl efetch script I promised a previous OP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, Python pandas is fabulous stuff for this purpose, I love it. Doing this via Perl and &quot;subprocessing&quot; inside R... its doable, well in this particular case I suspect its just as easy, but generally its alot more code and often fiddly.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-07-09T15:35:34.727" LastActivityDate="2019-07-09T15:35:34.727" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8943" PostTypeId="1" CreationDate="2019-07-09T14:49:15.670" Score="-1" ViewCount="134" Body="&lt;pre&gt;&lt;code&gt;## ------------------------------------------------------------------------&#xA;library(limma)&#xA;library(statmod)&#xA;library(GEOquery)&#xA;#library(GEOmetadb)&#xA;require(Biobase)&#xA;library(ggplot2)&#xA;library(rio)&#xA;library(affy)&#xA;library(affyPLM)&#xA;library(reshape)&#xA;library(stringr)&#xA;library(cowplot)&#xA;#library(makecdfenv)&#xA;#library(aroma.affymetrix)&#xA;#library(RTN)&#xA;library(gProfileR)&#xA;library(corpcor)&#xA;library(viper)&#xA;library(rio)&#xA;library(dplyr)&#xA;library(igraph)&#xA;library(RColorBrewer)&#xA;library(reshape2)&#xA;&#xA;&#xA;## ------------------------------------------------------------------------&#xA;gse89988 &amp;lt;- getGEO(filename = &quot;GSE89988_family.soft.gz&quot;, GSEMatrix = TRUE)&#xA;&#xA;&#xA;## ------------------------------------------------------------------------&#xA;# define samples&#xA;samples_nt &amp;lt;- c('GSM2394292', 'GSM2394303')&#xA;samples_pam &amp;lt;- c('GSM2394295', 'GSM2394306')&#xA;samples_lps &amp;lt;- c('GSM2394296', 'GSM2394307')&#xA;&#xA;# make table with samples metadata&#xA;samples &amp;lt;- c(samples_nt, samples_pam, samples_lps)&#xA;metadata &amp;lt;- as.data.frame(samples)&#xA;colnames(metadata) &amp;lt;- 'sample_id'&#xA;&#xA;# add group information both with the group acronym as a dummy variable -- which&#xA;# will be useful for the DEA; also add color for differentiate groups in plots&#xA;metadata&lt;span class=&quot;math-container&quot;&gt;$group &amp;lt;- c(rep(0, 2), rep(1, 2), rep(2, 2))&#xA;metadata$&lt;/span&gt;group_name &amp;lt;- c(rep('NT', 2), rep('PAM', 2), rep('LPS', 2))&#xA;metadata$group_color &amp;lt;- c(rep('gray', 2), rep('tomato', 2), rep('dodgerblue', 2))&#xA;&#xA;# get positions in GSE object&#xA;match(samples, names(GSMList(gse89988)))&#xA;&#xA;# with the following I confirm that the samples correspond to the expected samples&#xA;# and get that expression values are already normalized and log2-transformed&#xA;GSMList(gse89988)[[49]]&#xA;GSMList(gse89988)[[60]]&#xA;GSMList(gse89988)[[52]]&#xA;GSMList(gse89988)[[63]]&#xA;GSMList(gse89988)[[53]]&#xA;GSMList(gse89988)[[64]]&#xA;&#xA;&#xA;## ------------------------------------------------------------------------&#xA;#First, we need to make sure that all of the GSMs are from the same platform&#xA;gsmplatforms &amp;lt;- lapply(GSMList(gse89988),function(x) {Meta(x)$platform_id})&#xA;head(gsmplatforms)&#xA;# from the output of the command above we see that indeed all GSMs derive from&#xA;# the same platform &quot;GPL17543&quot;&#xA;&#xA;# We can filter the original GSMList to include only those GSMs with the GPL96 platform and use this list for further processing&#xA;gsmlist = Filter(function(gsm) {Meta(gsm)$platform_id=='GPL17543'},GSMList(gse89988))&#xA;length(gsmlist)&#xA;&#xA;# get the probeset ordering&#xA;probesets &amp;lt;- Table(GPLList(gse89988)[[1]])$ID&#xA;&#xA;# make the data matrix from the VALUE columns from each GSM&#xA;# being careful to match the order of the probesets in the platform&#xA;# with those in the GSMs&#xA;data.matrix &amp;lt;- do.call('cbind', lapply(gsmlist,function(x) &#xA;                                      {tab &amp;lt;- Table(x)&#xA;                                      mymatch &amp;lt;- match(probesets,tab&lt;span class=&quot;math-container&quot;&gt;$ID_REF)&#xA;                                      return(tab$&lt;/span&gt;VALUE[mymatch])&#xA;                                     }))&#xA;data.matrix &amp;lt;- apply(data.matrix, 2, function(x) {as.numeric(as.character(x))})&#xA;&#xA;# go through the necessary steps to make a compliant ExpressionSet&#xA;# and filter for the samples to be analyzed&#xA;rownames(data.matrix) &amp;lt;- probesets&#xA;colnames(data.matrix) &amp;lt;- names(gsmlist)&#xA;data.matrix &amp;lt;- data.matrix[, samples]&#xA;pdata &amp;lt;- data.frame(samples = samples)&#xA;rownames(pdata) &amp;lt;- samples&#xA;pheno &amp;lt;- as(pdata,&quot;AnnotatedDataFrame&quot;)&#xA;eset &amp;lt;- new('ExpressionSet', exprs = data.matrix, phenoData = pheno)&#xA;&#xA;&#xA;## ------------------------------------------------------------------------&#xA;load_gene_annotation &amp;lt;- function() {&#xA;&#xA;  ifile &amp;lt;- 'GPL17543-38936.txt'&#xA;  df &amp;lt;- read.delim(ifile, comment.char = '#')&#xA;&#xA;  return(df)&#xA;&#xA;}&#xA;&#xA;gene_annotation &amp;lt;- load_gene_annotation()&#xA;&#xA;&#xA;## ------------------------------------------------------------------------&#xA;apply_dea &amp;lt;- function(eset, fc_cutoff = 2, adj_p_cutoff = 0.05) {&#xA;&#xA;  # output directory&#xA;  ODIR &amp;lt;- 'results/step1/'&#xA;  dir.create(ODIR, showWarnings = FALSE, recursive = TRUE)&#xA;&#xA;  # make design matrix&#xA;  design &amp;lt;- model.matrix(~ 0 + factor(c(0, 0, 1, 1, 2, 2)))&#xA;  colnames(design) &amp;lt;- c(&quot;group0&quot;, &quot;group1&quot;, &quot;group2&quot;)&#xA;&#xA;  # plot with the normalized array weights&#xA;  aWeights &amp;lt;- arrayWeights(eset, design = design)&#xA;  ofile &amp;lt;- paste0(ODIR, 'array_weights_gse89988_samples_used_in_dea.pdf')&#xA;  pdf(ofile, width = 10, height = 5)&#xA;  dotchart(aWeights, labels = samples,&#xA;           groups = as.factor(metadata&lt;span class=&quot;math-container&quot;&gt;$group_name),&#xA;           pch = 19,&#xA;           color = metadata$&lt;/span&gt;group_color,&#xA;           main = &quot;Array Weights&quot;)  &#xA;  dev.off()&#xA;&#xA;  # linear model fit&#xA;  fit &amp;lt;- lmFit(eset, design)&#xA;  contrast.matrix &amp;lt;- makeContrasts(group1-group0, group2-group0,&#xA;                                   levels = design)&#xA;  fit2 &amp;lt;- contrasts.fit(fit, contrast.matrix)&#xA;  fit2 &amp;lt;- eBayes(fit2)&#xA;&#xA;&#xA;  # save &#xA;  ODIR='deliverables/'&#xA;  dir.create(ODIR, showWarnings = FALSE)&#xA;  ofile=paste0(ODIR, 'fit_for_figures_7_and_8.rds')&#xA;  saveRDS(fit2, ofile)&#xA;&#xA;&#xA;  # extract results, annotate and visualize DEA results&#xA;&#xA;  extract_annotate_and_viz_dea_results &amp;lt;- function(coef_index, ref, treat) {&#xA;&#xA;    # Extract DEA results for a given comparison&#xA;    df &amp;lt;- topTable(fit2, coef = coef_index, adjust = &quot;BH&quot;, number = Inf)&#xA;    df&lt;span class=&quot;math-container&quot;&gt;$reference &amp;lt;- ref&#xA;    df$&lt;/span&gt;treatment &amp;lt;- treat&#xA;&#xA;    # remove genes with missing values&#xA;    df &amp;lt;- df[complete.cases(df), ]&#xA;&#xA;    # annotate with gene information -- gene annotation has many fields which&#xA;    # are not relevant here so I will make a reduced gene information version&#xA;    fields_included &amp;lt;- c('ID', 'Species', 'Source', 'Entrez_Gene_ID', 'Symbol')&#xA;    df &amp;lt;- merge(df, gene_annotation[fields_included], by.x = 0, by.y = 'ID',&#xA;                sort = FALSE)&#xA;    colnames(df)[1] &amp;lt;- 'probe_id'&#xA;&#xA;    # prepare data for visualization through volcano plot&#xA;    cond1 &amp;lt;- df&lt;span class=&quot;math-container&quot;&gt;$adj.P.Val &amp;lt; adj_p_cutoff&#xA;    cond2d &amp;lt;- df$&lt;/span&gt;logFC &amp;lt; -log2(fc_cutoff)&#xA;    cond2u &amp;lt;- df&lt;span class=&quot;math-container&quot;&gt;$logFC &amp;gt; log2(fc_cutoff)&#xA;    log10p &amp;lt;- -log10(df$&lt;/span&gt;adj.P.Val)&#xA;    down &amp;lt;- nrow(df[cond1 &amp;amp; cond2d, ])&#xA;    up &amp;lt;- nrow(df[cond1 &amp;amp; cond2u, ])&#xA;    df&lt;span class=&quot;math-container&quot;&gt;$category &amp;lt;- ifelse(cond1 &amp;amp; cond2d, 'down-regulated',&#xA;                          ifelse(cond1 &amp;amp; cond2u, 'up-regulated', 'non-regulated'))&#xA;    abs_max_logfc &amp;lt;- max(abs(df$&lt;/span&gt;logFC))&#xA;&#xA;    # volcano plot&#xA;    g &amp;lt;- ggplot(df, aes(x = logFC, y = -log10(adj.P.Val), colour = category)) +&#xA;          geom_point(alpha = 0.4, size = 1.75) +&#xA;          scale_color_manual(values = c('dodgerblue', 'gray', 'tomato')) +&#xA;          xlim(-abs_max_logfc, abs_max_logfc) +&#xA;          ylim(c(0, max(-log10(df$adj.P.Val)))) +&#xA;          xlab(&quot;log2 fold change&quot;) +&#xA;          ylab(&quot;-log10(adjusted p-value)&quot;) +&#xA;          annotate(&quot;text&quot;, x = -4, y = 3, size =3, &#xA;                   label = paste0(down, &quot;\ndown-regulated DEGs&quot;),&#xA;                   color = 'dodgerblue') +&#xA;          annotate(&quot;text&quot;, x = 4, y = 3, size = 3,&#xA;                   label = paste0(up, &quot;\nup-regulated DEGs&quot;),&#xA;                   color = 'tomato') +&#xA;          annotate(&quot;text&quot;, x = -abs_max_logfc, y = 0, size = 2,&#xA;                   label = paste0(&quot;DEGs defined with:\nAdj. p-value &amp;lt; &quot;,&#xA;                                  adj_p_cutoff,&#xA;                                  '\nFold change &amp;gt; ',&#xA;                                  fc_cutoff),&#xA;                   color = 'black', hjust = 0) +&#xA;          ggtitle(paste0(ref, ' vs. ', treat)) +&#xA;          theme(plot.title = element_text(hjust = 0.5))&#xA;&#xA;    # save plot to file&#xA;    ofile &amp;lt;- paste0(ODIR, 'volcano_plot_', tolower(ref), '_vs_', tolower(treat),&#xA;                    '.pdf')&#xA;    ggsave(ofile, width = 7, height = 5)&#xA;&#xA;    return(df)&#xA;&#xA;  }&#xA;&#xA;  res &amp;lt;- rbind(extract_annotate_and_viz_dea_results(1, 'NT', 'PAM'),&#xA;               extract_annotate_and_viz_dea_results(2, 'NT', 'LPS'))&#xA;&#xA;&#xA;  # export table with results&#xA;  ofile &amp;lt;- paste0(ODIR, 'differential_expression_analysis_results_combined.xlsx')&#xA;  export(res, ofile)&#xA;&#xA;}&#xA;&#xA;&#xA;## ------------------------------------------------------------------------&#xA;apply_dea(eset)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After all these i have to run this &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;gsca &amp;lt;- pathwayAnalysis(fit2, pvaluecutoff=0.05)&lt;/code&gt;,where fit2 is made after running &lt;code&gt;apply_dea(eset)&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I run the above code i get this &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error in pathwayAnalysis(fit2, pvaluecutoff = 0.05) :    could not&#xA;  find function &quot;pathwayAnalysis&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I would share the .rdata file for reproduciblty&lt;br&gt;&#xA;&lt;a href=&quot;https://drive.google.com/file/d/1TGHa0TKV9J6glq7Q5JhoFMUM94IXa0eN/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;rdata&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The gse &amp;amp; the annotation &lt;a href=&quot;https://drive.google.com/open?id=1QkxCaIlzNwa4BoBZ_KEhaVYocUrJNbuD&quot; rel=&quot;nofollow noreferrer&quot;&gt;files&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be really glad if i get to know what am i doing wrong.&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="57" LastEditDate="2019-07-09T15:54:54.960" LastActivityDate="2019-07-09T17:10:12.887" Title="R error: could not find function &quot;pathwayAnalysis&quot;" Tags="|r|microarray|" AnswerCount="1" CommentCount="6" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8944" PostTypeId="2" ParentId="8943" CreationDate="2019-07-09T15:56:23.813" Score="1" Body="&lt;p&gt;It seems like you are missing function &quot;&lt;a href=&quot;https://rdrr.io/bioc/BLMA/src/R/PathwayAnalysis.R&quot; rel=&quot;nofollow noreferrer&quot;&gt;pathwayAnalysis&lt;/a&gt;&quot; presumably from package BLMA, i.e. &lt;code&gt;library('BLMA')&lt;/code&gt; should resolve your error once you install that package. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- edit 1 --&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ha, stupid me. Package &lt;code&gt;BLMA&lt;/code&gt; contains file &quot;pathwayAnalysis.R&quot;, not the function. A package that contains a FUNCTION called &lt;code&gt;pathwayAnalysis&lt;/code&gt; is &lt;a href=&quot;https://www.rdocumentation.org/packages/IntClust/versions/0.1.0/topics/PathwayAnalysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;IntClust&lt;/a&gt;. You should try that one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- edit 2 --&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alright, stupid me again, now I have not checked that IntClust has capitalized &lt;code&gt;PathwayAnalysis&lt;/code&gt;, you need &lt;code&gt;pathwayAnalysis&lt;/code&gt;. I found &lt;a href=&quot;https://www.rdocumentation.org/packages/metaX/versions/1.4.0/topics/pathwayAnalysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;one more function&lt;/a&gt; matching the name form package &lt;code&gt;metaX&lt;/code&gt;. However, it could be that it's function defined somewhere in their code. You can try to &lt;a href=&quot;https://readwrite.com/2010/11/10/how-to-search-your-source-with/&quot; rel=&quot;nofollow noreferrer&quot;&gt;search the source code for example using grep&lt;/a&gt; or you try to use google to find packages that contain the function of this name (this is exactly what I was doing, many you should be a bit more careful than I was).&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2019-07-09T17:10:12.887" LastActivityDate="2019-07-09T17:10:12.887" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="8945" PostTypeId="2" ParentId="8937" CreationDate="2019-07-09T16:01:42.143" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;UMI and barcode are the same but read2 is different, would they be&#xA;  collapsed to 1 UMI count or 2?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If the two reads share cell barcodes and UMI sequences and align to the same &lt;strong&gt;gene&lt;/strong&gt;, (even if they are different sequences) then they originate from the same RNA molecule, and should only count as 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/library-prep/doc/user-guide-chromium-single-cell-3-reagent-kits-user-guide-v3-chemistry&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.10xgenomics.com/single-cell-gene-expression/library-prep/doc/user-guide-chromium-single-cell-3-reagent-kits-user-guide-v3-chemistry&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2388" LastEditorUserId="2388" LastEditDate="2019-07-10T16:19:04.237" LastActivityDate="2019-07-10T16:19:04.237" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="8946" PostTypeId="1" AcceptedAnswerId="8948" CreationDate="2019-07-09T16:38:53.957" Score="0" ViewCount="126" Body="&lt;p&gt;I have a strange protein fasta file. There are several entries for the same gene and I need to extract the entry with the longest width.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;lcl|NW_017095468.1_prot_XP_017786561.1_8 [gene=LOC108569215] [db_xref=GeneID:108569215] [protein=NADH dehydrogenase [ubiquinone] 1 a&#xA;lpha subcomplex subunit 9, mitochondrial] [protein_id=XP_017786561.1] [location=complement(join(88030..88986,89044..89243,89412..8944&#xA;8))] [gbkey=CDS]&#xA;MAAVIFTGAQLLKQQSGLVGIAYIRVNNYSSDAKVYNLASLKRGTGGRSSFNGIVATVFGSTGFIGRYVANKLGKIGTQL&#xA;ILPYRGNNYETMRLKLCGDLGQVLFQPYYLKDLESIDKALKYSNVVINLVGRDWETKNFSFQDVHVKGARDLARAAKKAG&#xA;VEKFIHLSALNCDGPNEAIFSRQGSKFLSSKWEGEQAVLEEFPEATIFRPSDVYGQEDRFLRYYAHNWRRQGQYMPMWKN&#xA;GEATIKQPVHVSDLAAGIVAAIKDPEAAGKVYQAVGPRRYQLNELVDWFYRVMRKDSEWGYKRYDMKYDMFFKLKVSLTQ&#xA;KFSPAFPVGNLHWERLEREFVTDRVNQAIPTLEDLGVNLRRMEDQVPWELKPFTYGLYFGTDSEEPVVEPKPPKYVS&#xA;&amp;gt;lcl|NW_017095468.1_prot_XP_017770525.1_9 [gene=LOC108557919] [db_xref=GeneID:108557919] [protein=spondin-1 isoform X1] [protein_id=XP_017770525.1] [location=join(95471..95648,106560..106774,106834..107025,107093..107299,107562..107816,107872..108474,108526..108799,108885..109058,109127..109237,109573..109812,109882..110114)] [gbkey=CDS]&#xA;MRLKVAFLWLVSSISWIGEALRCDRTPEGTFSPRTRADGRFVIEVSGNPDTYVPGEQYNIFLRSNGEYQAKNKFKDFLLL&#xA;VEHEPSEKILGEVHNPSVGTLQLLGDMLMKFSEKCRNAVMQTNSLPKSEVQVLWVAPPSGSGCVAIRATVVESKEFWYTD&#xA;DGPLSKILCEEVQENEDTQPNMLRQCCACDEAKYEVTFEGLWSRNTHPKDFPSNGWLTRFSDIIGASHTFDYTFWNYGEI&#xA;ASNGLRQLAENGNTRMLESELKAKSEHIRTIIKARGISYPNITGKTFAVFRVDKRHHLMSLVSMIDPSPDWIVGVSGLEL&#xA;CLRNCSWVESRVLNLYPWDAGTDDGPTYISANQPSMPPHPIRRIKSNSPNDPRSPFYDPTGTEMKPLARLYLSRQRLYEK&#xA;NCVAQVDVSEEDGGVEGDKCEMEEWSEWSKCTVTCGRGFKYKQRAYKNPASNFVCNKPLTKRASCVAILEHCSNQQRPQE&#xA;ADPSCSLTGWGNWSSCTAPCGPGWKTRSRRYKNRKAAKRCAAGNENPEPLEQNLECMERECGPSDRRPLQESKECEGRAW&#xA;SNWSPCSSTCGKGIKVRRRMAYRSLWGRSPARYNRGLFDTEDTSRDDDDDGSDEDPCMNLDEKVECINDDVPVCEDTVDN&#xA;SAVVCGFPRDEGGCMSNVDRWYFDVIKGNCDIFSYSGCQGNKNNFKTLERCENVCDSYKKELLANRTAYKRQLGVTVSGV&#xA;LSYNLHHMQNDDADNCVPGSQTRQDQDKKIIQEPIGEVVDCQMSEWTNWSGCNATCGRGFSTKHRFIRVHPSNGGKRCPQ&#xA;KILKKRKCKIPCPGDYTKRDPMLPTWGTANSLEHVQIDCVMTGWSAWSPCSRSCGPNAVQQRTRGILLPPSGRGEPCLHR&#xA;TEERPCSLLACPE&#xA;&amp;gt;lcl|NW_017095468.1_prot_XP_017771299.1_10 [gene=LOC108557919] [db_xref=GeneID:108557919] [protein=spondin-1 isoform X1] [protein_id=XP_017771299.1] [location=join(95471..95648,106560..106774,106834..107025,107093..107299,107562..107816,107872..108474,108526..108799,108885..109058,109127..109237,109573..109812,109882..110114)] [gbkey=CDS]&#xA;MRLKVAFLWLVSSISWIGEALRCDRTPEGTFSPRTRADGRFVIEVSGNPDTYVPGEQYNIFLRSNGEYQAKNKFKDFLLL&#xA;VEHEPSEKILGEVHNPSVGTLQLLGDMLMKFSEKCRNAVMQTNSLPKSEVQVLWVAPPSGSGCVAIRATVVESKEFWYTD&#xA;DGPLSKILCEEVQENEDTQPNMLRQCCACDEAKYEVTFEGLWSRNTHPKDFPSNGWLTRFSDIIGASHTFDYTFWNYGEI&#xA;ASNGLRQLAENGNTRMLESELKAKSEHIRTIIKARGISYPNITGKTFAVFRVDKRHHLMSLVSMIDPSPDWIVGVSGLEL&#xA;CLRNCSWVESRVLNLYPWDAGTDDGPTYISANQPSMPPHPIRRIKSNSPNDPRSPFYDPTGTEMKPLARLYLSRQRLYEK&#xA;NCVAQVDVSEEDGGVEGDKCEMEEWSEWSKCTVTCGRGFKYKQRAYKNPASNFVCNKPLTKRASCVAILEHCSNQQRPQE&#xA;ADPSCSLTGWGNWSSCTAPCGPGWKTRSRRYKNRKAAKRCAAGNENPEPLEQNLECMERECGPSDRRPLQESKECEGRAW&#xA;SNWSPCSSTCGKGIKVRRRMAYRSLWGRSPARYNRGLFDTEDTSRDDDDDGSDEDPCMNLDEKVECINDDVPVCEDTVDN&#xA;SAVVCGFPRDEGGCMSNVDRWYFDVIKGNCDIFSYSGCQGNKNNFKTLERCENVCDSYKKELLANRTAYKRQLGVTVSGV&#xA;LSYNLHHMQNDDADNCVPGSQTRQDQDKKIIQEPIGEVVDCQMSEWTNWSGCNATCGRGFSTKHRFIRVHPSNGGKRCPQ&#xA;KILKKRKCKIPCPGDYTKRDPMLPTWGTANSLEHVQIDCVMTGWSAWSPCSRSCGPNAVQQRTRGILLPPSGRGEPCLHR&#xA;TEERPCSLLACPE&#xA;&amp;gt;lcl|NW_017095468.1_prot_XP_017772069.1_11 [gene=LOC108557919] [db_xref=GeneID:108557919] [protein=spondin-1 isoform X2] [protein_id=XP_017772069.1] [location=join(95471..95648,106560..106774,106834..107025,107093..107299,107562..107816,107872..108474,108526..108799,109546..109812,109882..110114)] [gbkey=CDS]&#xA;MRLKVAFLWLVSSISWIGEALRCDRTPEGTFSPRTRADGRFVIEVSGNPDTYVPGEQYNIFLRSNGEYQAKNKFKDFLLL&#xA;VEHEPSEKILGEVHNPSVGTLQLLGDMLMKFSEKCRNAVMQTNSLPKSEVQVLWVAPPSGSGCVAIRATVVESKEFWYTD&#xA;DGPLSKILCEEVQENEDTQPNMLRQCCACDEAKYEVTFEGLWSRNTHPKDFPSNGWLTRFSDIIGASHTFDYTFWNYGEI&#xA;ASNGLRQLAENGNTRMLESELKAKSEHIRTIIKARGISYPNITGKTFAVFRVDKRHHLMSLVSMIDPSPDWIVGVSGLEL&#xA;CLRNCSWVESRVLNLYPWDAGTDDGPTYISANQPSMPPHPIRRIKSNSPNDPRSPFYDPTGTEMKPLARLYLSRQRLYEK&#xA;NCVAQVDVSEEDGGVEGDKCEMEEWSEWSKCTVTCGRGFKYKQRAYKNPASNFVCNKPLTKRASCVAILEHCSNQQRPQE&#xA;ADPSCSLTGWGNWSSCTAPCGPGWKTRSRRYKNRKAAKRCAAGNENPEPLEQNLECMERECGPSDRRPLQESKECEGRAW&#xA;SNWSPCSSTCGKGIKVRRRMAYRSLWGRSPARYNRGLFDTEDTSRDDDDDGSDEDPCMNLDEKVECINDDVPVCEDTVDN&#xA;SVTENYFGRIVPGSQTRQDQDKKIIQEPIGEVVDCQMSEWTNWSGCNATCGRGFSTKHRFIRVHPSNGGKRCPQKILKKR&#xA;KCKIPCPGDYTKRDPMLPTWGTANSLEHVQIDCVMTGWSAWSPCSRSCGPNAVQQRTRGILLPPSGRGEPCLHRTEERPC&#xA;SLLACPE&#xA;&amp;gt;lcl|NW_017095468.1_prot_XP_017772815.1_12 [gene=LOC108557919] [db_xref=GeneID:108557919] [protein=spondin-1 isoform X3] [protein_id=XP_017772815.1] [location=join(95471..95648,106560..106774,106834..107025,107093..107299,107562..107816,107872..108474,108526..108799,109573..109812,109882..110114)] [gbkey=CDS]&#xA;MRLKVAFLWLVSSISWIGEALRCDRTPEGTFSPRTRADGRFVIEVSGNPDTYVPGEQYNIFLRSNGEYQAKNKFKDFLLL&#xA;VEHEPSEKILGEVHNPSVGTLQLLGDMLMKFSEKCRNAVMQTNSLPKSEVQVLWVAPPSGSGCVAIRATVVESKEFWYTD&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've tried using BioStrings but I'm unfamiliar with how to index this type of file. Any help appreciated.&lt;/p&gt;&#xA;" OwnerUserId="4834" LastEditorUserId="4834" LastEditDate="2019-07-09T19:22:18.470" LastActivityDate="2019-08-08T20:01:55.213" Title="Subset a protein fasta based on sequence length?" Tags="|fasta|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="8948" PostTypeId="2" ParentId="8946" CreationDate="2019-07-09T19:46:33.960" Score="1" Body="&lt;p&gt;Using the &lt;code&gt;FastaToTbl&lt;/code&gt; and &lt;code&gt;TblToFasta&lt;/code&gt; scripts I have &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2651/298&quot;&gt;posted before&lt;/a&gt;, you can do:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;FastaToTbl file.fa | &#xA;    awk '{&#xA;            len=length(&lt;span class=&quot;math-container&quot;&gt;$NF); &#xA;            if(len &amp;gt; l[$&lt;/span&gt;2]){&#xA;                l[&lt;span class=&quot;math-container&quot;&gt;$2]=len; &#xA;                seq[$&lt;/span&gt;2]=$0; &#xA;            }&#xA;          } &#xA;          END{&#xA;            for(s in seq){&#xA;                print seq[s]&#xA;            }&#xA;        }' | TblToFasta &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Running this on your example gives:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ FastaToTbl file.fa | awk '{len=length($NF); if(len &amp;gt; l[$2]){l[$2]=len; seq[$2]=$&lt;/span&gt;0; }} END{for(s in seq){print seq[s]}}' | TblToFasta &#xA;&amp;gt;lcl|NW_017095468.1_prot_XP_017770525.1_9 [gene=LOC108557919] [db_xref=GeneID:108557919] [protein=spondin-1 isoform X1] [protein_id=XP_017770525.1] [location=join(95471..95648,106560..106774,106834..107025,107093..107299,107562..107816,107872..108474,108526..108799,108885..109058,109127..109237,109573..109812,109882..110114)] [gbkey=CDS] &#xA;MRLKVAFLWLVSSISWIGEALRCDRTPEGTFSPRTRADGRFVIEVSGNPDTYVPGEQYNI&#xA;FLRSNGEYQAKNKFKDFLLLVEHEPSEKILGEVHNPSVGTLQLLGDMLMKFSEKCRNAVM&#xA;QTNSLPKSEVQVLWVAPPSGSGCVAIRATVVESKEFWYTDDGPLSKILCEEVQENEDTQP&#xA;NMLRQCCACDEAKYEVTFEGLWSRNTHPKDFPSNGWLTRFSDIIGASHTFDYTFWNYGEI&#xA;ASNGLRQLAENGNTRMLESELKAKSEHIRTIIKARGISYPNITGKTFAVFRVDKRHHLMS&#xA;LVSMIDPSPDWIVGVSGLELCLRNCSWVESRVLNLYPWDAGTDDGPTYISANQPSMPPHP&#xA;IRRIKSNSPNDPRSPFYDPTGTEMKPLARLYLSRQRLYEKNCVAQVDVSEEDGGVEGDKC&#xA;EMEEWSEWSKCTVTCGRGFKYKQRAYKNPASNFVCNKPLTKRASCVAILEHCSNQQRPQE&#xA;ADPSCSLTGWGNWSSCTAPCGPGWKTRSRRYKNRKAAKRCAAGNENPEPLEQNLECMERE&#xA;CGPSDRRPLQESKECEGRAWSNWSPCSSTCGKGIKVRRRMAYRSLWGRSPARYNRGLFDT&#xA;EDTSRDDDDDGSDEDPCMNLDEKVECINDDVPVCEDTVDNSAVVCGFPRDEGGCMSNVDR&#xA;WYFDVIKGNCDIFSYSGCQGNKNNFKTLERCENVCDSYKKELLANRTAYKRQLGVTVSGV&#xA;LSYNLHHMQNDDADNCVPGSQTRQDQDKKIIQEPIGEVVDCQMSEWTNWSGCNATCGRGF&#xA;STKHRFIRVHPSNGGKRCPQKILKKRKCKIPCPGDYTKRDPMLPTWGTANSLEHVQIDCV&#xA;MTGWSAWSPCSRSCGPNAVQQRTRGILLPPSGRGEPCLHRTEERPCSLLACPE&#xA;&amp;gt;lcl|NW_017095468.1_prot_XP_017786561.1_8 [gene=LOC108569215] [db_xref=GeneID:108569215] [protein=NADH dehydrogenase [ubiquinone] 1 alpha subcomplex subunit 9, mitochondrial] [protein_id=XP_017786561.1] [location=complement(join(88030..88986,89044..89243,89412..89448))] [gbkey=CDS] &#xA;MAAVIFTGAQLLKQQSGLVGIAYIRVNNYSSDAKVYNLASLKRGTGGRSSFNGIVATVFG&#xA;STGFIGRYVANKLGKIGTQLILPYRGNNYETMRLKLCGDLGQVLFQPYYLKDLESIDKAL&#xA;KYSNVVINLVGRDWETKNFSFQDVHVKGARDLARAAKKAGVEKFIHLSALNCDGPNEAIF&#xA;SRQGSKFLSSKWEGEQAVLEEFPEATIFRPSDVYGQEDRFLRYYAHNWRRQGQYMPMWKN&#xA;GEATIKQPVHVSDLAAGIVAAIKDPEAAGKVYQAVGPRRYQLNELVDWFYRVMRKDSEWG&#xA;YKRYDMKYDMFFKLKVSLTQKFSPAFPVGNLHWERLEREFVTDRVNQAIPTLEDLGVNLR&#xA;RMEDQVPWELKPFTYGLYFGTDSEEPVVEPKPPKYVS&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-07-09T19:46:33.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8949" PostTypeId="1" CreationDate="2019-07-09T19:55:13.857" Score="1" ViewCount="174" Body="&lt;p&gt;I have two files of genomic coordinates. I am trying to output only the overlapping region of fileA if it intersects with fileB. Not the original fileA coordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;fileA&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1 25 50&#xA;chr1 75 200&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;fileB&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1 10 60&#xA;chr1 80 90&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;desired output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1 25 50&#xA;chr1 80 90&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4836" LastEditorUserId="96" LastEditDate="2019-07-09T20:40:56.440" LastActivityDate="2019-07-09T20:40:56.440" Title="How to output only the overlapping region of two files?" Tags="|genome|interval|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8950" PostTypeId="2" ParentId="8949" CreationDate="2019-07-09T20:38:59.933" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools&lt;/a&gt; is my go-to program for operations on genomic intervals. In particular, the &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools intersect&lt;/a&gt; operation is what you're looking for here.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat fileA&#xA;chr1    25      50&#xA;chr1    75      200&#xA;$ cat fileB&#xA;chr1    10      60&#xA;chr1    80      90&#xA;$ bedtools intersect -a fileA -b fileB                                                           &#xA;chr1    25      50&#xA;chr1    80      90&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Make sure that the values are separated by tabs rather than spaces, or bedtools will have trouble reading the files.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-07-09T20:38:59.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8951" PostTypeId="2" ParentId="8905" CreationDate="2019-07-09T20:47:21.120" Score="1" Body="&lt;p&gt;Alternatively, Perl provides a slightly simpler regular expression syntax (&lt;code&gt;\d&lt;/code&gt; instead of &lt;code&gt;[0-9]&lt;/code&gt;) that I prefer over &lt;code&gt;sed&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl -pe 's/:\d+-\d+//' input.fasta &amp;gt; output.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2019-07-10T13:03:00.690" LastActivityDate="2019-07-10T13:03:00.690" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8952" PostTypeId="1" AcceptedAnswerId="8953" CreationDate="2019-07-10T10:23:27.910" Score="0" ViewCount="328" Body="&lt;p&gt;I have multiple files (n=86000) with one column each and I want to combine them all into one file with 86000 columns.&lt;/p&gt;&#xA;&lt;p&gt;I tried the following command&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Paste *.txt &amp;gt; mean.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And as a for loop&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in *.txt, do paste $i &amp;gt; mean.txt; done &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This did not appear to work because there are 86000 to combine, I then divided my files into multiple folders with each folder not containing more than 15000 files but this did not working&lt;/p&gt;&#xA;&lt;p&gt;Format of my single file is as follows (It has 16000 lines)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;0.047202943506056&#xA;0.0383549703366668&#xA;0.0159162908608477&#xA;0.0238048058550539&#xA;0.0694728928685859&#xA;0.0389052658410832&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What is the bug in the code?&lt;/p&gt;&#xA;" OwnerUserId="3766" LastEditorUserId="3848" LastEditDate="2022-11-18T23:32:38.410" LastActivityDate="2022-11-18T23:32:38.410" Title="How to combine multiple files into one file?" Tags="|sequence-alignment|phylogenetics|phylogeny|bash|" AnswerCount="1" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="8953" PostTypeId="2" ParentId="8952" CreationDate="2019-07-10T13:50:32.017" Score="4" Body="&lt;p&gt;The bug in your loop appears be &quot;&lt;code&gt;&amp;gt; filename&lt;/code&gt;&quot; per column, this over-rides the last iteration resulting in a file with a single column, which has been over-written 85999 times. Thus the file output you observe should be the single column present in the last file of your array. If you replaced &quot;&lt;code&gt;&amp;gt; filename&lt;/code&gt;&quot; with &quot;&lt;code&gt;&amp;gt;&amp;gt; filename&lt;/code&gt;&quot; within the loop you would then append to the file a new column with each iteration. If the loop fell over at some point you could then simply use one of the outputs of &lt;code&gt;wc filename&lt;/code&gt; to assess how many columns you had (you might have to transponse the data for wc to work (lots of scripts for this in Perl), although there will almost certainly be an easy solution for counting columns in awk. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally, I would use a dataframe for this operation (possibly R, definately Python pandas), because 86000 columns is a lot of columns in any other walk of life, even if it is just post-processing. Generally, if you just want bash, awk is a better solution in this case (I don't code with it, but some here certainly do). However, I do get the idea that &lt;code&gt;paste&lt;/code&gt; will work if each column is an identical length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW &lt;code&gt;paste&lt;/code&gt; must be lower case, you've got &lt;code&gt;Paste&lt;/code&gt; in the first line of your code, but its okay in the second bit of code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW BTW, I did not know shell would take *.txt directly as an array, with Perl/Python you need to &quot;glob&quot; it and assign it as an array/list.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-07-10T14:06:53.557" LastActivityDate="2019-07-10T14:06:53.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8955" PostTypeId="2" ParentId="8913" CreationDate="2019-07-10T14:40:46.627" Score="1" Body="&lt;p&gt;I found that using the Wald test instead of the Likelihood Ratio Test produced very different results with respect to the p-values.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sleuth_obj = sleuth_prep(&#xA;    metadata,&#xA;    extra_bootstrap_summary = TRUE,&#xA;    read_bootstrap_tpm = TRUE,&#xA;    num_cores = 1,&#xA;    target_mapping = transcripts_genes,&#xA;    aggregation_column = &quot;ens_gene&quot;&#xA;)&#xA;&#xA;# same model of covariates&#xA;sleuth_obj = sleuth_fit(sleuth_obj, ~cell + batch, &quot;full&quot;)&#xA;&#xA;# using Wald test instead of LRT&#xA;sleuth_obj = sleuth_wt(sleuth_obj, &quot;cellLT1&quot;, &quot;full&quot;)&#xA;&#xA;# extract results&#xA;sleuth_table_genes &amp;lt;- as.data.table(sleuth_results(sleuth_obj, 'cellLT1', 'wt', show_all = FALSE, pval_aggregate = TRUE))&#xA;sleuth_table_txs &amp;lt;- as.data.table(sleuth_results(sleuth_obj, 'cellLT1', 'wt', show_all = FALSE, pval_aggregate = FALSE))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8UExE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8UExE.png&quot; alt=&quot;Wald test gene-level&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/8D4je.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8D4je.png&quot; alt=&quot;Wald test transcript-level&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The p-values are different, but surprisingly the rankings of the genes aren't too different from each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bht5o.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bht5o.jpg&quot; alt=&quot;LRT vs Wald&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2019-07-10T14:40:46.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8956" PostTypeId="2" ParentId="3049" CreationDate="2019-07-10T17:34:04.733" Score="2" Body="&lt;p&gt;CRT (CRISPR Recognition Tool) &lt;a href=&quot;http://www.room220.com/crt/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.room220.com/crt/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Crisprs Finder Online Tool &lt;a href=&quot;https://crispr.i2bc.paris-saclay.fr/Server/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://crispr.i2bc.paris-saclay.fr/Server/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Piler-CR &lt;a href=&quot;http://www.drive5.com/pilercr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.drive5.com/pilercr/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Omic Tools List) &lt;a href=&quot;https://omictools.com/crispr-detection-category&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://omictools.com/crispr-detection-category&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1833" LastActivityDate="2019-07-10T17:34:04.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8957" PostTypeId="1" AcceptedAnswerId="8960" CreationDate="2019-07-10T18:24:50.783" Score="0" ViewCount="43" Body="&lt;p&gt;I have two types of files for a gene in a folder&lt;/p&gt;&#xA;&#xA;&lt;p&gt;File 1: FOS.tf.txt &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ABL2&#xA;ACTN4&#xA;ADGRE5&#xA;ADIPOR2&#xA;ADRB2&#xA;ERCC4&#xA;EZR&#xA;FAS&#xA;FMO4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;File 2: FOS.tt.txt&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ABL2&#xA;CASP9&#xA;CCK&#xA;DDIT3&#xA;ERCC4&#xA;EZR&#xA;FAS&#xA;FMO4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After running command &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;join FOS.tf.txt FOS.tt.txt &amp;gt; FOS_final.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I got output file: FOS_final.txt&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ABL2&#xA;ERCC4&#xA;EZR&#xA;FAS&#xA;FMO4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question is that I have these two type of files (tf and tt) for 605 genes in a folder and I want to run join command together for all my genes and get 605 output files.&lt;/p&gt;&#xA;" OwnerUserId="4845" LastEditorUserId="96" LastEditDate="2019-07-10T18:47:19.527" LastActivityDate="2019-07-10T18:47:19.527" Title="How to run 'join' command for multiple files (of 2 types) in a folder" Tags="|bash|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8958" PostTypeId="1" AcceptedAnswerId="8959" CreationDate="2019-07-10T18:28:30.217" Score="2" ViewCount="445" Body="&lt;p&gt;In order to use &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/tooldocs/current/picard_sam_MergeBamAlignment.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;MergeBamAlignment (Picard)&lt;/a&gt;, I need an unmapped BAM file and a mapped BAM file. I have two mapped BAM files: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;one with reads mapped to the reference I want but without metadata such as read group&lt;/li&gt;&#xA;&lt;li&gt;a second one with reads mapped to the incorrect reference but with metadata&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;When I try to use MergeBamAlignment with these two files, I get errors, such as &quot;Second read from pair not found in unmapped bam&quot;. So I am trying to unmap the reads from my second BAM file. How can I do that? Is there a tool for it? Otherwise, I'll have to manually change the bitwise FLAGs, reference names, CIGARS, etc.&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2019-07-11T05:47:56.127" Title="How can I transform a mapped BAM file into an unmapped BAM file?" Tags="|sam|read-mapping|software-recommendation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8959" PostTypeId="2" ParentId="8958" CreationDate="2019-07-10T18:38:00.173" Score="2" Body="&lt;p&gt;Looks like you need Picard's &lt;code&gt;RevertSam&lt;/code&gt; operation (described in context &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/article?id=6484#optionB&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, manual &lt;a href=&quot;https://broadinstitute.github.io/picard/command-line-overview.html#RevertSam&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar picard.jar RevertSam \&#xA;     I=input.bam \&#xA;     O=reverted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-07-10T18:38:00.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8960" PostTypeId="2" ParentId="8957" CreationDate="2019-07-10T18:42:46.477" Score="1" Body="&lt;p&gt;If all of your &lt;code&gt;.tf.txt&lt;/code&gt; and &lt;code&gt;.tt.txt&lt;/code&gt; files are in the same directory, the following command should work.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for tffile in $(ls *.tf.txt)&#xA;do&#xA;    prefix=$(basename $tffile .tf.txt)&#xA;    join ${prefix}.tf.txt ${prefix}.tt.txt &amp;gt; ${prefix}_final.txt&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-07-10T18:42:46.477" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8961" PostTypeId="2" ParentId="8670" CreationDate="2019-07-11T00:21:10.543" Score="3" Body="&lt;p&gt;While my suggestion doesn't technically fit your qualifications, I do think it may be useful. I bring this up because you mention you want to continue with your R workflow. I am assuming you don't have anything against Python, but just want to stick with R to keep a tidy project. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The R package &lt;code&gt;reticulate&lt;/code&gt; (&lt;a href=&quot;https://rstudio.github.io/reticulate/&quot; rel=&quot;nofollow noreferrer&quot;&gt;homepage&lt;/a&gt;) allows for a very easy interface between Python and R. I use it to run the Python package &lt;code&gt;scrublet&lt;/code&gt; to remove doublets from scRNA-Seq data before analysis with a popular R package. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just for reference, here is what using a Python package in R might look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;## setup&#xA;Sys.setenv(&quot;RETICULATE_PYTHON&quot;=&quot;/n/apps/CentOS7/install/pyenv-1.0.0/pyenv/shims/python3&quot;)&#xA;use_python(&quot;/n/apps/CentOS7/install/pyenv-1.0.0/pyenv/shims/python3&quot;, required=TRUE)&#xA;reticulate::py_config()&#xA;scr &amp;lt;- import(module=&quot;scrublet&quot;, convert=FALSE)&#xA;&#xA;## usage&#xA;E150_scrub &amp;lt;- scr&lt;span class=&quot;math-container&quot;&gt;$Scrublet(r_to_py(seurat_object@assays$&lt;/span&gt;RNA@counts)&lt;span class=&quot;math-container&quot;&gt;$T$&lt;/span&gt;tocsc(), expected_doublet_rate=0.06)&#xA;doublet_results &amp;lt;- E150_scrub$scrub_doublets(min_counts=2, min_cells=3, min_gene_variability_pctl=85)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The Python package function calls look like &lt;code&gt;scr$Scrublet()&lt;/code&gt;. There are some very convenient features like easy conversion of numpy tables to dataframes and the like. If you use RMarkdown, you can even display inline plots from &lt;code&gt;matplotlib&lt;/code&gt;. I highly recommend giving it a try, it smoothly bridges the gap between the spheres of development of bioinformatic tools.&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2019-07-11T00:21:10.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8962" PostTypeId="2" ParentId="8958" CreationDate="2019-07-11T05:47:56.127" Score="0" Body="&lt;p&gt;What happens if you use the bam with the metadata as the input to your aligner?  Maybe it will keep all those fields.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-07-11T05:47:56.127" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8965" PostTypeId="1" AcceptedAnswerId="9253" CreationDate="2019-07-11T10:29:21.730" Score="0" ViewCount="882" Body="&lt;p&gt;I am running a single cell dataset (count data - exon) through Seurat. After running tsne I see a cluster (13) split in 3 different locations on the plot. Here are the commands I am running:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library('Seurat')&#xA;## rows (genes) that have 99% zeros were filtered out, &#xA;## and then columns (cells) that have 99% zeros were filtered out&#xA;isp.data &amp;lt;- read.table(file = &quot;genes.cells99.percent.filtered&quot;, sep = &#xA;            &quot;,&quot;, row.names = 1, header=TRUE)&#xA;isp &amp;lt;- CreateSeuratObject(raw.data = isp.data, project = &#xA;       &quot;brain_regions&quot;)&#xA;isp &amp;lt;- NormalizeData(object = isp, normalization.method = &#xA;       &quot;LogNormalize&quot;,scale.factor = 10000)&#xA;isp &amp;lt;- FindVariableGenes(object = isp, mean.function = ExpMean, &#xA;       dispersion.function = LogVMR,do.plot = FALSE)&#xA;hv.genes &amp;lt;- head(rownames(isp@hvg.info), 1000)&#xA;isp &amp;lt;- ScaleData(object = isp, genes.use = hv.genes, num.cores = 3)&#xA;isp &amp;lt;- RunPCA(object = isp, pc.genes = hv.genes, pcs.compute = 100, &#xA;       do.print = TRUE, pcs.print = 1:5, genes.print = 5)&#xA;PCElbowPlot(object = isp, num.pc = 100)&#xA;isp &amp;lt;- JackStraw(object = isp, num.replicate = 100, display.progress = &#xA;                 FALSE, num.pc = 30)&#xA;JackStrawPlot(object = isp, PCs = 1:30)&#xA;isp &amp;lt;- FindClusters(object = isp, reduction.type = &quot;pca&quot;, dims.use = &#xA;        1:26, resolution= 0.75, print.output = 0, save.SNN = TRUE, &#xA;        force.recalc=TRUE)&#xA;isp &amp;lt;- RunTSNE(isp, dims.use = 1:26, do.fast = T)&#xA;TSNEPlot(isp, do.label = T, pt.size = 0.5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the tsne plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oOIg7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oOIg7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried varying parameters for resolution in FindClusters (from 0.5-3.5), but that just increases the clusters found. Further I tried to play around with parameters min.dist (0.75,1), n_neighbors (10 to 30) and perplexity(5 to 50) but I still get at least 1-2 clusters split in the tsne plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing I have tried is to reduce the dimension (dims.use=10) on which tsne is ran but the clusters seem to be too close together, like so:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JhQWH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JhQWH.png&quot; alt=&quot;dims_reduced&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: Dimensionality of dataset (after filtering) - 23824 genes across 8967 samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question being why does such behavior happen? Is my filtering strategy not correct? Have I not normalized/scaled data correctly? &lt;/p&gt;&#xA;" OwnerUserId="4850" LastEditorUserId="4850" LastEditDate="2019-07-11T10:35:43.287" LastActivityDate="2019-08-22T13:03:43.063" Title="Cluster is split in 2-3 locations on tsne plot - Suerat" Tags="|rna-seq|seurat|single-cell|clustering|normalization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8967" PostTypeId="1" CreationDate="2019-07-11T13:43:39.120" Score="1" ViewCount="67" Body="&lt;p&gt;I understand the basic principles under which RPS-BLAST operates: search a protein (&lt;code&gt;rpsblast&lt;/code&gt;) or nucleotide (&lt;code&gt;rpstblastn&lt;/code&gt;) query against a database of protein profiles, a &quot;reverse PSI-BLAST&quot; search. My question has to do with the practical application of this tool to annotate conserved protein domains in bacterial assemblies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume I have downloaded &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Structure/cdd/cdd.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI's Conserved Domain Database (CDD)&lt;/a&gt; and would like to annotate conserved protein domains in a bacterial genome I have just assembled from Illumina reads. I see two potential approaches.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Run a gene prediction tool on the assembly, translate the predicted gene models, use &lt;code&gt;rpsblast&lt;/code&gt; to search the translated sequences against the CDD profiles, and then transform the matches from local protein coordinates to global genome coordinates.&lt;/li&gt;&#xA;&lt;li&gt;Use &lt;code&gt;rpstblastn&lt;/code&gt; to search the genome sequences against the profiles directly.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The latter option would be more compute intensive, since it requires computing a 6-frame translation of each genome sequence. But perhaps it is more sensitive in the presence of small misassemblies?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In practice, is either of these approaches preferred over the other? Are there any considerations I'm missing?&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-10-08T16:25:18.120" Title="Annotating conserved protein domains in bacterial assemblies" Tags="|proteins|bacteria|sequence-annotation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8968" PostTypeId="1" CreationDate="2019-07-11T14:28:08.153" Score="4" ViewCount="2864" Body="&lt;p&gt;I have R installed and managed by conda (miniconda) on my MacBook Pro. The version of R I use most frequently (3.5.1) is installed on the base environment and I have other version-specific environments as well. I did this as it was easier than managing R on homebrew.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The downside I now face is that I cannot use &lt;code&gt;install.packages()&lt;/code&gt;, &lt;code&gt;BiocManager::install()&lt;/code&gt; or &lt;code&gt;biocLite()&lt;/code&gt; to install most packages. &lt;code&gt;devtools::install_github()&lt;/code&gt; seems to work fine for the few packages I have installed from GitHub.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to use &lt;code&gt;conda install -c [conda-forge|bioconda] [r|bioconductor]-package_name&lt;/code&gt; to install my packages and it breaks my workflow and I need to step outside of R to install packages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I doing anything wrong, or is this the only way packages can be managed when R in installed through conda?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A few sample packages that I needed to install using &lt;code&gt;conda&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;tidyverse&lt;/li&gt;&#xA;&lt;li&gt;tximport&lt;/li&gt;&#xA;&lt;li&gt;deseq2&lt;/li&gt;&#xA;&lt;li&gt;apeglm&lt;/li&gt;&#xA;&lt;li&gt;maftools&lt;/li&gt;&#xA;&lt;li&gt;sva&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="650" LastActivityDate="2019-07-12T17:07:57.990" Title="Using `install.packages` with conda-managed R" Tags="|r|bioconductor|conda|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="8970" PostTypeId="1" AcceptedAnswerId="8971" CreationDate="2019-07-11T17:40:13.400" Score="1" ViewCount="142" Body="&lt;p&gt;I am working with a specific GWAS.  If I were to run this on the command line&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep &amp;lt;rs_id_of_interest&amp;gt; GWAS.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would see the GWAS p-value to be on the order of 10^-200. Very tiny p-value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if I were to do the same but in pandas, with the following code, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gwas = pd.read_csv(GWAS_file,sep='\t')&#xA;gwas_rs_id = gwas.loc[gwas['marker'] == 'rs_id'].copy()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then print gwas_rs_id, it shows the gwas p-value to be 0.0. Is there a way to read the file and not have it automatically round the tiny tiny p-values?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="4855" LastEditorUserId="3848" LastEditDate="2023-05-13T02:13:33.200" LastActivityDate="2023-05-13T02:13:33.200" Title="Pandas automatically rounds GWAS P-value" Tags="|python|phylogenetics|phylogeny|gwas|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8971" PostTypeId="2" ParentId="8970" CreationDate="2019-07-11T19:59:05.177" Score="2" Body="&lt;p&gt;The solution you likely want is here,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pd.set_eng_float_format(accuracy=x, use_eng_prefix=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;x = whatever is required&#xA;The function set_eng_float_format has been moved around a bit and is now a top level function&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;You might be dealing with maximum likelihood, so convert to log likelihood -200 that sort of thing&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best idea&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gwas_rs_id['logged'] = np.log(gwas_rs_id.marker)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;First idea,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import decimal as D&#xA;.... &#xA;gwas_rs_id['marker'] = gwas_rs_id['marker'].astype(D.decimal)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Last attempt,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pd.set_option('display.precision', x) # x whatever dp you need, this might work in combination with a log likelihood&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Try a &lt;code&gt;gwas_rs_id.dtype()&lt;/code&gt; check whether you're using float32, float64 or still in object. It should convert to scientific notion (E values) automatically. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-07-12T11:51:49.350" LastActivityDate="2019-07-12T11:51:49.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8972" PostTypeId="1" CreationDate="2019-07-11T21:56:15.050" Score="1" ViewCount="67" Body="&lt;p&gt;What is ChrQ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume it then means chromosme Q or something like that yet when I google for it I get very weird answers, but as I saw that a couple times when looking on freebayes pipelines manuals I wanted to be sure.&lt;/p&gt;&#xA;" OwnerUserId="4857" LastEditorUserId="48" LastEditDate="2019-07-12T07:31:28.580" LastActivityDate="2022-06-21T10:18:05.493" Title="ChrQ definition" Tags="|chromosome-capture|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8973" PostTypeId="1" CreationDate="2019-07-12T00:30:45.577" Score="1" ViewCount="7516" Body="&lt;p&gt;I have a list of genes that I'd like to visualize using the DoHeatmap function in Seurat. However, the output of the heatmap does not result in hierarchical clustering and therefore makes it very difficult to interpret. My code is,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; DoHeatmap(object = obj, genes.use = genes), &#xA;          slim.col.label = TRUE, remove.key = T,group.label.rot = F, use.scaled = T)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/x85hy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/x85hy.png&quot; alt=&quot;Seurat&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to adjust the &lt;code&gt;DoHeatmap&lt;/code&gt; command to rank the cells by the intensity of gene expression? Here's an example output:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Jy8Xs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Jy8Xs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4858" LastEditorUserId="3848" LastEditDate="2019-07-14T21:12:02.183" LastActivityDate="2019-08-16T14:00:55.077" Title="DoHeatmap Hierarchical Clustering Seurat" Tags="|r|seurat|heatmap|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8974" PostTypeId="1" CreationDate="2019-07-12T08:52:28.770" Score="2" ViewCount="36" Body="&lt;p&gt;I've been interested in plasmid detection in bacterial genomes. I tried &lt;a href=&quot;https://cge.cbs.dtu.dk/services/PlasmidFinder/&quot; rel=&quot;nofollow noreferrer&quot;&gt;plasmidfinder&lt;/a&gt; and this software found only one plasmid in each of my two bacterial genomes. I was dubious about these results, because I read many genome announcements which found something like from 6 to 10 plasmids per genome around 5 -6 MB. I then used &lt;a href=&quot;https://github.com/bioinfo-ut/PlasmidSeeker&quot; rel=&quot;nofollow noreferrer&quot;&gt;PlasmidSeeker&lt;/a&gt; and what I received in the result file was a list of 9 or 11 so called &quot;plasmid clusters&quot;, each containing a few plasmids with complete ID. I know that my genome cannot contain tens of plasmids, so my question is which one(s) of them is (are) the real one(s)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, can you suggest other bioinformatics tools specially designed to detect plasmids in bacterial genomes?&lt;/p&gt;&#xA;" OwnerUserId="4821" LastEditorUserId="48" LastEditDate="2019-07-12T09:10:31.157" LastActivityDate="2019-07-12T09:10:31.157" Title="How to interprete PlasmidSeeker results concerning plasmids?" Tags="|perl|software-usage|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8975" PostTypeId="1" CreationDate="2019-07-12T10:28:38.843" Score="5" ViewCount="858" Body="&lt;p&gt;I am facing an issue when trying to align short reads against a region in human chr5. The two Sensory Motor Neuron genes, (SMN1 and SMN2) are almost 100% identical and this causes the aligner to fail to align reads correctly since each read matches perfectly to two separate regions of the reference, leading to very low alignment scores.&lt;/p&gt;&#xA;&lt;p&gt;If I BLAT the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NG_008691.1?report=fasta&quot; rel=&quot;nofollow noreferrer&quot;&gt;SMN1 gene's full genomic sequence&lt;/a&gt; against the human genome (hg19) on UCSC, I get (only showing the 1st two lines):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;QUERY       SCORE START   END QSIZE IDENTITY  CHROM  STRAND  START       END   SPAN&#xA;--------------------------------------------------------------------------------------------------------------&#xA;NG_008691.1 35072     1 35072 35072   100.0%  chr5   +    70215768  70250839  35072&#xA;NG_008691.1 34279     1 35072 35072    99.8%  chr5   +    69340350  69375383  35034&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And &lt;a href=&quot;http://genome.ucsc.edu/cgi-bin/hgTracks?db=hg19&amp;amp;lastVirtModeType=default&amp;amp;lastVirtModeExtraState=&amp;amp;virtModeType=default&amp;amp;virtMode=0&amp;amp;nonVirtPosition=&amp;amp;position=chr5%3A69311399%2D70350568&amp;amp;hgsid=739135289_3EijLmAz1FaCi8vZojAHjpMDEwU0&quot; rel=&quot;nofollow noreferrer&quot;&gt;on the browser&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4osMP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4osMP.png&quot; alt=&quot;UCSC genome browser BLAT results&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;To get around this, I want to mask the SMN2 gene. However, the region of similarity extends beyond the annotated boundaries of the gene, so I would like to identify the full extent of this duplication.&lt;/p&gt;&#xA;&lt;p&gt;So, given a large nucleotide sequence in fasta format, how can I compare it to itself and identify identical or near identical regions?&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2021-07-05T08:13:53.727" LastActivityDate="2021-07-05T08:13:53.727" Title="How can I locate duplicated regions in a sequence?" Tags="|fasta|genome|sequence-analysis|sequence-homology|duplications|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8976" PostTypeId="2" ParentId="8975" CreationDate="2019-07-12T15:21:26.860" Score="2" Body="&lt;p&gt;If you know approximately where are the duplicated regions, you can cut them out from the genome and align the two block using tools like &lt;a href=&quot;https://github.com/garviz/MUMmer&quot; rel=&quot;nofollow noreferrer&quot;&gt;MUMmer&lt;/a&gt; or &lt;a href=&quot;http://darlinglab.org/mauve/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mauve&lt;/a&gt;. Although Mauve is not scalable solution (You can not scan genomes for duplicates with it) it's very handy for aligning small(ish) blocks. It can also align multiple sequences if you would like to check some tri/multiplications.&lt;/p&gt;&#xA;&lt;p&gt;I downloaded &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NC_000005.9&quot; rel=&quot;nofollow noreferrer&quot;&gt;the sequence of ch5&lt;/a&gt; you linked and extracted big regions around the genes (300bp blocks). The first block is from 70100000 - 70400000 (with SMN1 sequence at 115768 - 150839) and the second from 69200000 - 69500000 (with SMN2 sequence at 140350 - 175383).&lt;/p&gt;&#xA;&lt;p&gt;I used progressive mauve to align the two (click on File -&amp;gt; align with ProgressiveMauve -&amp;gt; add Sequence ... -&amp;gt; Align).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RAT8G.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RAT8G.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I found that the duplicated region is almost 200k long and maps between 44690 - 225520 of the first block and 19760 - 200940 of the second block, which we still need translate to the original genome coordinates, but I believe I can leave this to you :-).&lt;/p&gt;&#xA;&lt;p&gt;P.S. you can also somehow upload annotations to Mauve, but I was lazy, so I &amp;quot;hand drawn&amp;quot; the arrow where I knew the genes are.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="73" LastEditDate="2021-07-05T04:00:47.227" LastActivityDate="2021-07-05T04:00:47.227" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8977" PostTypeId="1" AcceptedAnswerId="8997" CreationDate="2019-07-12T16:25:51.223" Score="3" ViewCount="356" Body="&lt;p&gt;I'm trying to separate RNA from protein in a complex protein/RNA PDB file and I want all RNA info with the hetero atoms in between the bases BUT without H20 etc. In short I want RNA part of pdb file without discontinuous lines.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I managed to separate RNA from protein with Bio PDB Select but it consider hetero atoms as amino acid when I use is_aa(residue). So hetero atoms wont appear in my &quot;only RNA&quot; file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.PDB import *&#xA;from Bio.PDB import PDBParser, PDBIO, Select&#xA;import os&#xA;&#xA;class ProtSelect(Select):&#xA;    def accept_residue(self, residue):&#xA;        return 1 if is_aa(residue) == True else 0&#xA;&#xA;class RNASelect(Select):&#xA;    def accept_residue(self, residue):&#xA;        return 1 if is_aa(residue) == False and residue.id[0] != &quot;W&quot; else 0&#xA;&#xA;pdb = PDBParser().get_structure(&quot;2bh2&quot;, &quot;pdb2bh2.ent&quot;)&#xA;io = PDBIO()&#xA;io.set_structure(pdb)&#xA;io.save(&quot;seqprotest.pdb&quot;, ProtSelect())&#xA;io.save(&quot;seqRNAtest.pdb&quot;, RNASelect())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;EDIT: I managed to do it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class RNASelect(Select):&#xA;    def accept_residue(self, residue):&#xA;        return 1 if ((residue.id[0] != 'H_ MG') and ((residue.get_resname() == '  A' or residue.get_resname() == '  U' or   residue.get_resname() == '  G' or  residue.get_resname() == '  C' or residue.get_resname() == '  T') or  (residue.id[0][0:2] == 'H_' and (residue.get_resname()[2] == 'U'  or residue.get_resname()[2] == 'A' or residue.get_resname()[2] == 'C' or residue.get_resname()[2] == 'G' or residue.get_resname()[2] == 'T'))))  else 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It's not pretty but it do the job.&#xA;The idea is to get everything that finish by a nucleotide. I figured out that hetatom's names of RNA chains always finish by a nucleotide letter so i just have to check everything that begins by 'H_' and ends by 'A,T,C,G,U'.&#xA;Just remove 'H_ MG' and you're good.&lt;/p&gt;&#xA;" OwnerUserId="4864" LastEditorUserId="4864" LastEditDate="2019-07-16T15:12:20.967" LastActivityDate="2019-07-16T15:16:42.553" Title="How to select only RNA with Hetero atoms from pdb file with python?" Tags="|python|pdb|rna|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8978" PostTypeId="1" AcceptedAnswerId="8981" CreationDate="2019-07-12T16:30:47.150" Score="1" ViewCount="2222" Body="&lt;p&gt;I have a bigfolder where i have a lot of fastq.gz files and I want to remove the adapters from all of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying then the following loop:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;for file in LotsofRaws\/cannam/*_R1.fastq.gz; do cutadapt &#xA;     -a runs/new/Adapters/IlluNexteraPE.fa &#xA;        -o LotsofRaws/RemovedAdapter/Adapted$file   &#xA;           $file done;  done &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However I am getting the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cutadapt: error: &#xA;&#xA;*It appears you want to trim paired-end data because you provided two input files, but then you also need to provide two output files (with -o and -p) or use the --interleaved option.*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I just want to take the adapters from the files I am not looking to do it in the same way I would do with a single file is there any way to do this because if I try to do &lt;code&gt;--interleaved&lt;/code&gt; I get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cutadapt: error: Character '/' in adapter sequence 'RTNS/NEW/ADAPTERS/ILLTNEXTERAPE.FA' is not a valid IUPAC code. Use only characters XACGTURYSWKMBDHVN.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is cutadapt 2.3 with Python 3.6.7&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and if instead of using the file I directly use the sequences:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for file in LotsofRaws/cannam/*; &#xA;         do cutadapt -a CTGTCTCTTATACACATCT &#xA;               -a AGATCGGAAGAGC &#xA;               --interleaved &#xA;               -o LotsofRaws/RemovedAdapter/&lt;span class=&quot;math-container&quot;&gt;$file   $&lt;/span&gt;file ; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FileNotFoundError: [Errno 2] No such file or directory: 'LotsofRaws/RemovedAdapter/LotsofRaws/cannam/NAM7_Glenlea_R1.fastq.gz'&#xA;During handling of the above exception, another exception occurred:&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;many times&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What should I then do for general variant calling it is appropriate to do the --interleaved option?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I just the do the trimming manually or can I loop it in any way that still resembles doing it one by one but without me having to go and write line and line again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the files on the folder look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NAM11_BW415_R1.fastq.gz         NAM13_BW968_R2.fastq.gz     NAM17_AC_Barrie_R1.fastq.gz  NAM3_PI648600_R2.fastq.gz  NAM7_Glenlea_R1.fastq.gz   PI648598_R2.fastq.gz&#xA;NAM11_BW415_R2.fastq.gz         NAM13_Frontana_R1.fastq.gz  NAM17_AC_Barrie_R2.fastq.gz  NAM3_PI648842_R1.fastq.gz  NAM7_Glenlea_R2.fastq.gz   PI648611_R1.fastq.gz&#xA;NAM11_Reeder_R1.fastq.gz        NAM13_Frontana_R2.fastq.gz  NAM17_BW388_R1.fastq.gz      NAM3_PI648842_R2.fastq.gz  NAM7_Laura_R1.fastq.gz     PI648611_R2.fastq.gz&#xA;NAM11_Reeder_R2.fastq.gz        NAM14_BW874_R1.fastq.gz     NAM17_BW388_R2.fastq.gz      NAM4_PI613279_R1.fastq.gz  NAM7_Laura_R2.fastq.gz     PI648672_R1.fastq.gz&#xA;NAM11_Webill_R1.fastq.gz        NAM14_BW874_R2.fastq.gz     NAM17_RedFife_R1.fastq.gz    NAM4_PI613279_R2.fastq.gz  NAM7_Marquis_R1.fastq.gz   PI648672_R2.fastq.gz&#xA;NAM11_Webill_R2.fastq.gz        NAM14_Glenn_R1.fastq.gz     NAM17_RedFife_R2.fastq.gz    NAM4_PI613280_R1.fastq.gz  NAM7_Marquis_R2.fastq.gz   PI648724_R1.fastq.gz&#xA;NAM12_BW431_L002_R1.fastq.gz    NAM14_Glenn_R2.fastq.gz     NAM1_PI648517_R1.fastq.gz    NAM4_PI613280_R2.fastq.gz  NAM8_FL62R1_R1.fastq.gz    PI648724_R2.fastq.gz&#xA;NAM12_BW431_L002_R2.fastq.gz    NAM14_Unity_R1.fastq.gz     NAM1_PI648517_R2.fastq.gz    NAM4_PI648524_R1.fastq.gz  NAM8_FL62R1_R2.fastq.gz    PI648768_R1.fastq.gz&#xA;NAM12_BW431_L008_R1.fastq.gz    NAM14_Unity_R2.fastq.gz     NAM1_PI648519_R1.fastq.gz    NAM4_PI648524_R2.fastq.gz  NAM8_Muchmore_R1.fastq.gz  PI648768_R2.fastq.gz&#xA;NAM12_BW431_L008_R2.fastq.gz    NAM15_BW881_R1.fastq.gz     NAM1_PI648519_R2.fastq.gz    NAM5_PI648523_R1.fastq.gz  NAM8_Muchmore_R2.fastq.gz  PI648860_R1.fastq.gz&#xA;NAM12_Penhold_L002_R1.fastq.gz  NAM15_BW881_R2.fastq.gz     NAM1_PI648541_R1.fastq.gz    NAM5_PI648523_R2.fastq.gz  NAM8_Stettler_R1.fastq.gz  PI648860_R2.fastq.gz&#xA;NAM12_Penhold_L002_R2.fastq.gz  NAM15_Lillian_R1.fastq.gz   NAM1_PI648541_R2.fastq.gz    NAM5_PI648585_R1.fastq.gz  NAM8_Stettler_R2.fastq.gz  PI648862_R1.fastq.gz&#xA;NAM12_Penhold_L008_R1.fastq.gz  NAM15_Lillian_R2.fastq.gz   NAM2_PI648493_R1.fastq.gz    NAM5_PI648585_R2.fastq.gz  PI277012_R1.fastq.gz       PI648862_R2.fastq.gz&#xA;NAM12_Penhold_L008_R2.fastq.gz  NAM15_PT245_R1.fastq.gz     NAM2_PI648493_R2.fastq.gz    NAM5_PI648622_R1.fastq.gz  PI277012_R2.fastq.gz       PI648864_R1.fastq.gz&#xA;NAM12_Sumai3_L002_R1.fastq.gz   NAM15_PT245_R2.fastq.gz     NAM2_PI648602_R1.fastq.gz    NAM5_PI648622_R2.fastq.gz  PI648478_R1.fastq.gz       PI648864_R2.fastq.gz&#xA;NAM12_Sumai3_L002_R2.fastq.gz   NAM16_BA51B92_R1.fastq.gz   NAM2_PI648602_R2.fastq.gz    NAM6_Neepawa_R1.fastq.gz   PI648478_R2.fastq.gz       run_701_150904_7001410_0194_BC79VNANXX_NAM.list&#xA;NAM12_Sumai3_L008_R1.fastq.gz   NAM16_BA51B92_R2.fastq.gz   NAM2_PI648609_R1.fastq.gz    NAM6_Neepawa_R2.fastq.gz   PI648487_R1.fastq.gz       run_760_160912_7001343F_0065_AC9EV0ANXX_NAM.list&#xA;NAM12_Sumai3_L008_R2.fastq.gz   NAM16_BW928_R1.fastq.gz     NAM2_PI648609_R2.fastq.gz    NAM6_PI648532_R1.fastq.gz  PI648487_R2.fastq.gz&#xA;NAM13_BW462_R1.fastq.gz         NAM16_BW928_R2.fastq.gz     NAM3_PI648511_R1.fastq.gz    NAM6_PI648532_R2.fastq.gz  PI648566_R1.fastq.gz&#xA;NAM13_BW462_R2.fastq.gz         NAM16_BW963_R1.fastq.gz     NAM3_PI648511_R2.fastq.gz    NAM6_Thatcher_R1.fastq.gz  PI648566_R2.fastq.gz&#xA;NAM13_BW968_R1.fastq.gz         NAM16_BW963_R2.fastq.gz     NAM3_PI648600_R1.fastq.gz    NAM6_Thatcher_R2.fastq.gz  PI648598_R1.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4857" LastEditorUserId="650" LastEditDate="2019-07-13T09:55:11.013" LastActivityDate="2019-07-13T10:42:45.163" Title="for loop cutadapt for files on a single directory" Tags="|shell|cutadapt|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8979" PostTypeId="2" ParentId="8968" CreationDate="2019-07-12T17:07:57.990" Score="3" Body="&lt;p&gt;Installing packages outside of an R session is arguably the point of using &lt;code&gt;conda&lt;/code&gt;.  Package management isn't something that should be done within scripts, only the loading of packages. Doing so ensures that your environment setup is somewhat robust, which should be set before you perform actual calculations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using &lt;code&gt;conda install ...&lt;/code&gt; is the best way to ensure environment stability here, and, thankfully, lots of people at Bioconductor have worked very hard to make sure &lt;code&gt;bioconda&lt;/code&gt; packages fit with the conda framework.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need to find packages, you can search online at &lt;a href=&quot;https://anaconda.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://anaconda.org&lt;/a&gt;, or from the command line via &lt;code&gt;conda search &amp;lt;pkg&amp;gt;&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2019-07-12T17:07:57.990" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8981" PostTypeId="2" ParentId="8978" CreationDate="2019-07-13T10:42:45.163" Score="0" Body="&lt;p&gt;You have various different issues here. First, the first version of the command you show would give a syntax error, not the error you show, so I am guessing what you actually ran was this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for file in LotsofRaws\/cannam/*_R1.fastq.gz; do &#xA;    cutadapt -a runs/new/Adapters/IlluNexteraPE.fa \ &#xA;             -o LotsofRaws/RemovedAdapter/Adapted&lt;span class=&quot;math-container&quot;&gt;$file $&lt;/span&gt;file done;  &#xA;done &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The error there is that you have the word &lt;code&gt;done&lt;/code&gt; after &lt;code&gt;$file&lt;/code&gt;. So what &lt;code&gt;cutadapt&lt;/code&gt; sees is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cutadapt [OPTIONS] $file done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So it reads that as being given two input files instead of one. What you wanted to do was this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for file in LotsofRaws\/cannam/*_R1.fastq.gz; do &#xA;    cutadapt -a runs/new/Adapters/IlluNexteraPE.fa \ &#xA;             -o LotsofRaws/RemovedAdapter/Adapted&lt;span class=&quot;math-container&quot;&gt;$file $&lt;/span&gt;file  &#xA;done &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, on one line, this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for file in LotsofRaws\/cannam/*_R1.fastq.gz; do cutadapt -a runs/new/Adapters/IlluNexteraPE.fa -o LotsofRaws/RemovedAdapter/Adapted&lt;span class=&quot;math-container&quot;&gt;$file $&lt;/span&gt;file; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One &lt;code&gt;done&lt;/code&gt;, not two and separated from the command by a newline or a &lt;code&gt;;&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your next attempt, with &lt;code&gt;--interleaved&lt;/code&gt; failed because the &lt;code&gt;-a&lt;/code&gt; option expects an actual sequence, not a file unless you use it with &lt;code&gt;file:&lt;/code&gt;. See &lt;code&gt;cutadapt --help&lt;/code&gt; (emphasis mine):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Finding adapters:&lt;br&gt;&#xA;  Parameters -a, -g, -b specify adapters to be&#xA;  removed from each read (or   from the first read in a pair if data is&#xA;  paired). If specified multiple   times, only the best matching adapter&#xA;  is trimmed (but see the --times   option). &lt;strong&gt;When the special notation&#xA;  'file:FILE' is used, adapter sequences   are read from the given FASTA&#xA;  file&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, the &lt;code&gt;-a&lt;/code&gt; expects to be given a sequence unless you use it as &lt;code&gt;-a file:/path/to/file&lt;/code&gt;. Because you used it as &lt;code&gt;-a runs/new/Adapters/IlluNexteraPE.fa&lt;/code&gt; instead of &lt;code&gt;-a file:withruns/new/Adapters/IlluNexteraPE.fa&lt;/code&gt;, it tried to interpret &lt;code&gt;runs/new/Adapters/IlluNexteraPE.fa&lt;/code&gt; as a sequence and complained because the character &lt;code&gt;/&lt;/code&gt; isn't valid in a DNA sequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, your last lop failed because you're giving the wrong name. The first rule of debugging is &quot;print all the things&quot;. When you see a command that doesn't work as you expect, your first step should always be to print out the command and see what is actually being run. When you do a &lt;code&gt;for file in /long/path/to/file&lt;/code&gt; loop, the entire path is included in your &lt;code&gt;$file&lt;/code&gt; variable. Try running this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for file in LotsofRaws/cannam/*; do &#xA;    echo &quot;cutadapt -a CTGTCTCTTATACACATCT -a AGATCGGAAGAGC --interleaved -o LotsofRaws/RemovedAdapter/&lt;span class=&quot;math-container&quot;&gt;$file $&lt;/span&gt;file&quot;; &#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You will see that that will print commands like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cutadapt -a CTGTCTCTTATACACATCT -a AGATCGGAAGAGC --interleaved \&#xA;         -o LotsofRaws/RemovedAdapter/LotsofRaws/cannam/NAM11_BW415_R1.fastq.gz \&#xA;          LotsofRaws/cannam/NAM11_BW415_R1.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What you need is to take the name of the file, without the path. And you don't want to escape the slash after &lt;code&gt;LotsofRaws&lt;/code&gt;. So putting all this together and going back to your first attempt, what you want is something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for f in LotsofRaws\/cannam/*_R1.fastq.gz; do &#xA;    fname=&lt;span class=&quot;math-container&quot;&gt;$(basename file)&#xA;    cutadapt -a &quot;file:runs/new/Adapters/IlluNexteraPE.fa&quot; &#xA;             -o &quot;LotsofRaws/RemovedAdapter/Adapted$&lt;/span&gt;{fname}&quot; $file&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-07-13T10:42:45.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8982" PostTypeId="2" ParentId="8937" CreationDate="2019-07-13T13:05:02.157" Score="1" Body="&lt;p&gt;The code does EXACTLY what it says in the first quote.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say you have 5 reads on a cell barcode that align to gene A.  Those reads have the following molecular barcodes:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AAAA,BBBB,AAAA,CCCC,DDDD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are 4 UNIQUE molecular barcodes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The only complication to this is that we collapse barcodes with an edit distance of 1, which takes care of over-inflation of UMI counts due to PCR/Sequencing errors.  If we added a barcode with the sequence AAAB to the original list, it would be collapsed into AAAA as it has an edit distance of 1, and you'd still have 4 unique molecular barcodes.&lt;/p&gt;&#xA;" OwnerUserId="4866" LastActivityDate="2019-07-13T13:05:02.157" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8983" PostTypeId="1" AcceptedAnswerId="8984" CreationDate="2019-07-13T15:01:29.657" Score="0" ViewCount="1718" Body="&lt;p&gt;I'm having some problems with finding the right parameters to trim my small RNA Illumina reads (51 nt long) with Trimmomatic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before trimming, one of the samples (21M reads) looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/powRQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/powRQ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for my understanding, it's quite good and the last two issues can be easily solved with adapter trimming.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then tried to trim it with the command &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar /opt/Trimmomatic-0.38/trimmomatic-0.38.jar SE -threads 24 -phred33 FemaleMito3.fastq FemaleMito3_trimmed.fastq ILLUMINACLIP:adapters.ultimate.fa:2:30:10 AVGQUAL:25 LEADING:3 TRAILING:3 SLIDINGWINDOW:4:15 MINLEN:18&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but I got the following result:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eF2y0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eF2y0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now I have a problem with the sequence quality per tile (why?), an overall drop of the quality at the end of the reads and a warning with the sequence length distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I tried trimming just the adapters with this command: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar /opt/Trimmomatic-0.38/trimmomatic-0.38.jar SE -threads 24 -phred33 FemaleMito1.fastq FemaleMito1_noadapters.fastq ILLUMINACLIP:adapters.ultimate.fa:2:30:10 AVGQUAL:25&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but again I get some new issues:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zlNDJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zlNDJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again the per tile sequence quality issue, and now a warning about the GC content.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I didn't find anywhere some precise instruction on how to trim a small RNA &#xA;library with Trimmomatic. &lt;strong&gt;How should I properly trim my small RNA library with Trimmomatic?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="1637" LastActivityDate="2019-07-14T19:35:12.333" Title="What are the right parameters to trim a small RNA transcriptome with trimmomatic?" Tags="|rna-seq|transcriptome|trimming|data-preprocessing|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8984" PostTypeId="2" ParentId="8983" CreationDate="2019-07-14T19:35:12.333" Score="3" Body="&lt;p&gt;The command where you trim with adapters and by quality is perfectly fine. That FastQC isn't perfectly happy is expected. It's a tool made for whole genome sequencing QC, you SHOULD see a number of &quot;fail&quot;s with any RNA-seq protocol. Steps that should always fail in RNA-seq:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;per-base sequence content (there's &quot;random priming&quot; performed that isn't completely random, so the first ~8 bases show a typical bias pattern).&lt;/li&gt;&#xA;&lt;li&gt;per-sequence GC content (particularly in sRNA-seq you're enriching for a subset of genes, they won't have some nice GC distribution and that's perfectly OK).&lt;/li&gt;&#xA;&lt;li&gt;Sequence duplication level (if this doesn't fail then everything is uniformly expressed...which would be quite unlikely)&lt;/li&gt;&#xA;&lt;li&gt;Overrepresented sequences (as above, though this will be worse for sRNA-seq since reads will often cover the entire transcript, so there's even less sequence diversity)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-07-14T19:35:12.333" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8985" PostTypeId="2" ParentId="5259" CreationDate="2019-07-14T23:34:48.100" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;I could use NanoSim, but that tool seems to be very difficult to use. I only have a reference genome, but you need training data as well?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Karel Břinda's fork, &lt;a href=&quot;https://github.com/karel-brinda/NanoSim-H&quot; rel=&quot;nofollow noreferrer&quot;&gt;NanoSim-H&lt;/a&gt;, ships with pretrained models (or &amp;quot;error profiles&amp;quot;, as they're called there), eliminating the need for training data.&lt;/p&gt;&#xA;&lt;p&gt;Minimal usage instructions to run with the default error model (trained on 1D² reads of E. Coli on an R9 flowcell):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pip install nanosim-h&#xA;nanosim-h your-reference.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3144" LastEditorUserId="3144" LastEditDate="2022-09-03T18:38:59.377" LastActivityDate="2022-09-03T18:38:59.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8986" PostTypeId="1" AcceptedAnswerId="8990" CreationDate="2019-07-15T05:54:53.507" Score="0" ViewCount="249" Body="&lt;p&gt;I want to use either of Proteomics or Transcriptomics data for integrating it into my &lt;a href=&quot;https://biology.stackexchange.com/questions/85360/how-to-integrate-transcriptomics-data-with-kinetic-metabolic-models&quot;&gt;kinetic model&lt;/a&gt;. Before proceeding, I want to know what are the advantages of using either of them so that I could make an informed decision on it!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many studies have shown that the best we can do is integrating both transcriptomics and proteomics data with our kinetic model, but I've some time constraints and have to proceed with only one of those.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;My effort and findings:&lt;/em&gt; I've found from discussions with researchers that gathering transcriptomics data has an amplification step which increases the chance of finding a particular one whereas gathering proteomics data has no such step but has fragmentation and then rejoining which creates many problems(such as splice variants etc) and thus leads to a loss of data. But a PostDoc told me that even after the loss of that data, I'll get more information from Proteomics data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know such type of points and want to know if these are valid or not!&lt;/p&gt;&#xA;" OwnerUserId="2883" LastEditorUserId="2883" LastEditDate="2019-07-15T08:27:06.437" LastActivityDate="2019-07-15T16:59:58.273" Title="Proteomics data Vs Transcriptomics data?" Tags="|transcriptome|networks|modelling|proteomics|multi-omics|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="8987" PostTypeId="2" ParentId="8986" CreationDate="2019-07-15T10:10:39.047" Score="0" Body="&lt;p&gt;This is what I found from doing some research. Comments are welcome at any point!&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Capture percentage for data gathering:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Transcriptomics Data: There's an amplification step in Transcriptomics data gathering methods. Hence, it's possible to capture almost the totality of the Transcriptome using those methods(scRNA seq methods, Nanopore tech, Spatial transcriptomics, etc)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Proteomics Data: There's no amplification step in Proteomics data gathering methods. Those methods have fragmentation and a rejoining step which leads to a loss of the data.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Scalability of methods:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Transcriptomics Data: Scalable methods available&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Proteomics Data: Less scalable methods for protein studies&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reference availability:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Transcriptomics Data: Fully annotated references available on consortiums such as ensembl biomart, etc&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Proteomics Data: Universal and Comprehensive Human Proteome reference is still in question and because of the incomplete and inaccurate references, much of the newly generated data is being rendered useless&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Uniformity&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Transcriptomics Data: Lots of publications on pipelines, and their benchmarking available&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Proteomics Data: Lack of uniformity across labs/research groups and lack of related literature lead to differences in protein fragmentation and solubilization and differences in algorithms to run analyses&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Technical Bias&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Transcriptomics Data: Many methods available to tackle cell-bias, noise in data, and batch effects, so as to capture maximum biological variability&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Proteomics Data: Cell-wide Mass spectrometry has a bias towards the identification of peptides with higher concentration or contamination from other experiments&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2883" LastActivityDate="2019-07-15T10:10:39.047" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="8988" PostTypeId="1" CreationDate="2019-07-15T11:33:01.173" Score="0" ViewCount="182" Body="&lt;p&gt;This is a follow-up of &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/8975/298&quot;&gt;my other question&lt;/a&gt;. I have been having trouble calling variants in the human SMN1 and SMN2 genes, because the human genome has a large segmental duplication there and these two genes (and their surrounding sequences) are essentially identical :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0UiCX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0UiCX.png&quot; alt=&quot;UCSC browser screenshot&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;http://genome.ucsc.edu/cgi-bin/hgTracks?db=hg19&amp;amp;lastVirtModeType=default&amp;amp;lastVirtModeExtraState=&amp;amp;virtModeType=default&amp;amp;virtMode=0&amp;amp;nonVirtPosition=&amp;amp;position=chr5%3A69311399%2D70350568&amp;amp;hgsid=739135289_3EijLmAz1FaCi8vZojAHjpMDEwU0&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;sub&gt;Click here to see the original image on the UCSC browser.&lt;/sub&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since variants in the SMN1 gene have been linked to disease, it is important for me to be able to call them in my NGS data. However, because of this segmental duplication, aligning reads to this region correctly is very hard since the presence of the duplication makes each read align almost perfectly to separate locations in the genome, leading to low alignment scores. The low scores will, in turn, block variant callers from calling in those regions.&lt;/p&gt;&#xA;&lt;p&gt;I thought that the only real solution for this would be to mask the SMN2 gene and its associated duplication in my reference genome which would allow the aligner to align all reads to the region of SMN1. I would then miss any real variants in SMN2, but since the primary gene of interest for  spinal muscular atrophy is SMN1 and not SMN2, I would be OK with that.&lt;/p&gt;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is this a reasonable approach? I was told that &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/8975/how-can-i-locate-duplicated-regions-in-a-sequence#comment12265_8976&quot;&gt;&amp;quot;generally, masking segdups is not recommended&amp;quot;&lt;/a&gt; by someone who most certainly knows what he's talking about. So is this not a good approach for my stated goal?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If not, then what else can I do that will let me call variants in the SMN1 gene?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I would like something similar to the approach described &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1707-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;: mask one of the regions to rescue mutations in the other.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="73" LastEditDate="2021-07-05T02:57:59.877" LastActivityDate="2021-12-02T06:02:15.263" Title="How should I deal with segmental duplications when aligning NGS reads to a reference genome?" Tags="|sequence-alignment|variant-calling|reference-genome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8989" PostTypeId="1" CreationDate="2019-07-15T13:18:07.407" Score="1" ViewCount="88" Body="&lt;p&gt;I have data for 396 transcriptional factors (TF), where I have calculated the number of Transcriptional targets for each transcriptional factor present between the two groups:&lt;/p&gt;&#xA;&lt;p&gt;Group-I(G-I): where co-exp genes have a correlation coefficient is more than 0.6 and&lt;/p&gt;&#xA;&lt;p&gt;Group-II(G-II): where the co-exp genes have r values less than -0.6.&lt;/p&gt;&#xA;&lt;p&gt;I have co-expression map for all the human transcription factors. My hypothesis is that transcriptional targets of a transcriptional factor are among the highly co-expressed gene for that transcription factor. For that, I have divided my group into high and low co-exp genes.&lt;/p&gt;&#xA;&lt;p&gt;My question which statistical test I should use for evaluating the difference between the two groups?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  TF    G-I G-II&#xA;  AATF  1   0&#xA;  ABL1  5   1&#xA;  AES   0   1&#xA;  AHR   1   2&#xA;  AIP   2   0&#xA; APBB1  0   1&#xA;APEX1   2   1&#xA;  AR    0   2&#xA;ARNT    3   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4845" LastEditorUserId="73" LastEditDate="2022-06-21T10:19:43.103" LastActivityDate="2022-06-21T10:19:43.103" Title="Which statistical test to use for comparing two groups?" Tags="|statistics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="8990" PostTypeId="2" ParentId="8986" CreationDate="2019-07-15T16:59:58.273" Score="0" Body="&lt;p&gt;Ultimately, proteomics data can actually measure the actual concentrations of the enzymes that you are interested, while transcriptomics only measures their rates of change due to ongoing transcription from mRNA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rates of change are often treated as proxy for concentrations. This makes sense when you have a constant degradation/dilution environment and are near steady state, since then the rate of change is proportional to the converged concentration. If you're interested in more dynamic environments or believe that modulation of degradation or sequestration might play a significant role in a system, however, that assumption breaks done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, if you have to choose between proteomics and transcriptomics data, I would suggest proteomics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a final note, however, since the link suggests you're interested in metabolites, however, have you consider going directly to metabolomics data?  That would let you really get direct information about the particular species of interests.&lt;/p&gt;&#xA;" OwnerUserId="4869" LastActivityDate="2019-07-15T16:59:58.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8992" PostTypeId="1" AcceptedAnswerId="8999" CreationDate="2019-07-15T20:20:13.590" Score="3" ViewCount="2990" Body="&lt;p&gt;I am trying to find novel transcripts from an RNA-seq database. Based on the advice I got, it seemed that using Stringtie for transcript assembly is a good way to go, and it supports novel transcript discovery even with the reference GTF file provided (which apparently significantly improves the assembly process). So I tried to follow the protocol provided in the &lt;a href=&quot;https://www.nature.com/articles/nprot.2016.095&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nature Protocols&lt;/a&gt; paper that described the use of Stringtie; the paper also suggested using GFFcompare for comparing the assembled transcripts with the reference, to quantify and find the novel transcripts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I followed the entire protocol, and this was the output of GFFcompare when comparing the reference GTF with that of one of the samples after mapping - &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#= Summary for dataset: ./hisat2/ERR188044_chrX.gtf &#xA;#-----------------| Sensitivity | Precision  |&#xA;        Base level:    51.7     |    79.5    |&#xA;        Exon level:    46.7     |    85.2    |&#xA;      Intron level:    47.2     |    93.9    |&#xA;Intron chain level:    31.2     |    64.4    |&#xA;  Transcript level:    31.6     |    52.3    |&#xA;       Locus level:    36.6     |    50.1    |&#xA;&#xA;     Matching intron chains:     580&#xA;       Matching transcripts:     664&#xA;              Matching loci:     397&#xA;&#xA;          Missed exons:    4395/8804    ( 49.9%)&#xA;           Novel exons:     426/4874    (  8.7%)&#xA;        Missed introns:    3832/7946    ( 48.2%)&#xA;         Novel introns:      83/3992    (  2.1%)&#xA;           Missed loci:     610/1086    ( 56.2%)&#xA;            Novel loci:     273/795 ( 34.3%)&#xA;&#xA; Total union super-loci across all input datasets: 749 &#xA;1270 out of 1270 consensus transcripts written in 188044_only.annotated.gtf (0 discarded as redundant)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the paper however, the number of novel &lt;strong&gt;genes&lt;/strong&gt; and &lt;strong&gt;transcripts&lt;/strong&gt; is clearly mentioned - &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ML19h.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ML19h.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I was wondering how they were able to calculate the number of novel &lt;strong&gt;transcripts and genes&lt;/strong&gt;, from the data that GFFcompare provides (they even mention in the paper that they got it from GFFcompare). I see the numbers related to &lt;strong&gt;novel exons and novel introns&lt;/strong&gt;, but how do I translate it to the number and identities of the novel transcripts/genes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/389330/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cross-posted in Biostars&lt;/a&gt;. Let me know if I shouldn't do this.&lt;/p&gt;&#xA;" OwnerUserId="907" LastEditorUserId="907" LastEditDate="2019-07-16T18:44:36.630" LastActivityDate="2019-07-17T12:13:55.627" Title="How to find novel transcripts using GFFcompare?" Tags="|rna-seq|transcriptome|reference-genome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8993" PostTypeId="1" CreationDate="2019-07-16T01:25:13.570" Score="1" ViewCount="1050" Body="&lt;p&gt;I have been using &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/edgeR.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;EdgeR&lt;/a&gt; from the &lt;strong&gt;Bioconductor&lt;/strong&gt; package to perform differential expression analysis on bulk RNASeq data sets.&lt;/p&gt;&#xA;&lt;p&gt;This analysis gives me an output dataframe that has a list of all the genes, ranked by the their adjP values. This dataframe also contains the t-statistic and lfc values for each gene as well.&lt;/p&gt;&#xA;&lt;p&gt;I have recently been using &lt;a href=&quot;https://maayanlab.cloud/Enrichr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;enrichR&lt;/a&gt; to perform gene set enrichment analysis on all the differentially expressed genes to determine what biological, molecular, transcriptional, etc pathways these genes are involved in. I really like the enrichR package for a few key reasons:&lt;/p&gt;&#xA;&lt;p&gt;1.) It has access to many different databases that I can call on with a simple vector of their names. Most of my data is murine RNASEq, so for those analyses, I can simply select the murine databases:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;enrichR_dbs &amp;lt;- c(&amp;quot;WikiPathways_2019_Mouse&amp;quot;, &amp;quot;KEGG_2019_Mouse&amp;quot;, &amp;quot;GO_Biological_Process_2018&amp;quot;, &amp;quot;GO_Cellular_Component_2018&amp;quot;, &amp;quot;GO_Molecular_Function_2018&amp;quot;, &amp;quot;GWAS_Catalog_2019&amp;quot;, &amp;quot;TRRUST_Transcription_Factors_2019&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;2.) It uses a single function, enrichR, to perform the enrichment. The arguments for that function are the list of gene symbols and the list of databases.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;enrichr(as.character(Gene_Dataframe$SYMBOL[1:500]), enrichR_dbs)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Anyway, I am looking for another enrichment analysis because I want to be able to see if I get similar pathways for my genes when using another method/algorithm.&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, I can't seem to find another enrichment analysis( &lt;a href=&quot;https://biodatamining.biomedcentral.com/articles/10.1186/s13040-018-0166-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;competitive or self-contained&lt;/a&gt;) that is both as user-friendly and that gives me easy access to murine databases (without requiring to convert my genes through changes in capitalization, like you need to with GSEA) as enrichR.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Does anyone have recommendations about enrichment analysis R packages that don't require too many more inputs than the dataframe I generated through my EdgeR analysis?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;P.S. I'm ok with downloading databases if I need to. Though inclusion of a PDF with a tutorial would be fantastic, if possible.&lt;/p&gt;&#xA;" OwnerUserId="4876" LastEditorUserId="3848" LastEditDate="2020-12-17T16:33:06.873" LastActivityDate="2020-12-17T16:33:06.873" Title="Alternative to enrichR for enrichment analysis?" Tags="|go-enrichment|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8995" PostTypeId="2" ParentId="6694" CreationDate="2019-07-16T12:07:10.747" Score="2" Body="&lt;p&gt;The solution to your problem probably is adding the full mRNA sequence of your transgene to the reference (as also suggested by acrux).&lt;br&gt;&#xA;I had a very similar problem recently when tdTomato expression was only detectable in a couple of cells. The transgene was introduced into the cells using a lentiviral vector and therefore the mRNA had the tdTomato sequence followed by a WPRE element. In the picture below you can see that a large number of reads is mapping to the WPRE element in the 3' UTR (blue region) and and almost none to the tdTomato CDS (red region).&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ZULWo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZULWo.png&quot; alt=&quot;reads mapping to the exogenous tdTomato sequence&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, the untranslated regions are considered exons because they are part of the final mRNA. So if you have the fasta sequence beginning at the transcription start and ending at the polyadenylation signal, this is the full mRNA. Then you just need to update the GTF with the longer full mRNA length and you are all set.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2019-07-16T12:07:10.747" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="8996" PostTypeId="2" ParentId="8993" CreationDate="2019-07-16T14:48:07.060" Score="2" Body="&lt;p&gt;A couple of years ago I wrote a blog post about &lt;a href=&quot;https://b101nfo.blogspot.com/2017/03/gsea-in-bioconductor.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSEA methods in Bioconductor&lt;/a&gt;, in addition to these I would recommend &lt;a href=&quot;https://bioconductor.org/packages/clusterProfiler/&quot; rel=&quot;nofollow noreferrer&quot;&gt;clusterProfiler&lt;/a&gt; or &lt;a href=&quot;https://bioconductor.org/packages/ReactomePA/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ReactomePA&lt;/a&gt;.  In most of them you need to provide the gene set/pathway database.&lt;br&gt;&#xA;If you want to do gene ontology enrichment then I recomment the &lt;a href=&quot;http://bioconductor.org/packages/topGO/&quot; rel=&quot;nofollow noreferrer&quot;&gt;topGO&lt;/a&gt; package.&#xA;In most, if not all, of these methods you just need an ordered list and a gene set source. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2019-07-16T14:48:07.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8997" PostTypeId="2" ParentId="8977" CreationDate="2019-07-16T15:16:42.553" Score="3" Body="&lt;p&gt;Solution I found: (c/p of body)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;class RNASelect(Select):&#xA;    def accept_residue(self, residue):&#xA;        return 1 if ((residue.id[0] != 'H_ MG') and ((residue.get_resname() == '  A' or residue.get_resname() == '  U' or   residue.get_resname() == '  G' or  residue.get_resname() == '  C' or residue.get_resname() == '  T') or  (residue.id[0][0:2] == 'H_' and (residue.get_resname()[2] == 'U'  or residue.get_resname()[2] == 'A' or residue.get_resname()[2] == 'C' or residue.get_resname()[2] == 'G' or residue.get_resname()[2] == 'T'))))  else 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It's not pretty but it do the job. The idea is to get everything that finish by a nucleotide. I figured out that hetatom's names of RNA chains always finish by a nucleotide letter so i just have to check everything that begins by 'H_' and ends by 'A,T,C,G,U'. Just remove 'H_ MG' and you're good.&lt;/p&gt;&#xA;" OwnerUserId="4864" LastActivityDate="2019-07-16T15:16:42.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="8999" PostTypeId="2" ParentId="8992" CreationDate="2019-07-17T12:13:55.627" Score="4" Body="&lt;p&gt;The output of &lt;a href=&quot;https://ccb.jhu.edu/software/stringtie/gffcompare.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;gffcompare&lt;/a&gt; includes several files per run (just like &lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/cuffcompare/&quot; rel=&quot;nofollow noreferrer&quot;&gt;cuffcompare&lt;/a&gt;). Example for a run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ls cuffcmp.* | sed 's/\t/\n/'&#xA;cuffcmp.combined.gtf&#xA;cuffcmp.loci&#xA;cuffcmp.output.gtf.refmap&#xA;cuffcmp.output.gtf.tmap&#xA;cuffcmp.stats&#xA;cuffcmp.tracking&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;From my experience, the easiest file to manipulate is the &lt;code&gt;tmap&lt;/code&gt; file:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Tab delimited file lists the most closely matching reference transcript for each query transcript. There is one row per query transcript, and the columns are as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1     Reference gene name    The gene_name attribute of the reference GTF record for this transcript, if present. Otherwise gene_id is used.&#xA;2     Reference transcript id     The transcript_id attribute of the reference GTF record for this transcript&#xA;3     Class code  The type of relationship between the query transcripts in column 4 and the reference transcript (as described in the Class Codes section below)&#xA;4     Query gene id   The query (e.g., Stringtie) internal gene id&#xA;5     Query transcript id     The query internal transcript id&#xA;6     Number of exons     The number of exons in the query transcript&#xA;7     FPKM    The expression of this transcript expressed in FPKM&#xA;8     TPM     the estimated TPM for the transcript, if found in the query input file&#xA;9     Coverage    The estimated average depth of read coverage across the transcript.&#xA;10    Length  The length of the transcript&#xA;11    Major isoform ID    The query ID of the gene's major isoform&#xA;12    Reference match length  The length of the longest overlap with a reference, ‘-’ if there is no such exonic overlap&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The important column to look at for transcript classification is actually the 3rd, &quot;class code&quot;. The content is meaningful if you used option &lt;code&gt;-r&lt;/code&gt; when running gffcompare. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the tmap file you can access info on both transcript and gene ID directly, both from the assembler and the reference annotation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now what you need is: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;n. assembled genes&lt;/li&gt;&#xA;&lt;li&gt;n. novel genes&lt;/li&gt;&#xA;&lt;li&gt;n. novel transcripts&lt;/li&gt;&#xA;&lt;li&gt;n. transcripts matching annotation&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;At this point, it is essential to understand what the authors mean by &quot;novel&quot; (gene/transcript), does it mean &quot;non previously annotated&quot; (= class code &lt;code&gt;u&lt;/code&gt;, for &quot;unknown&quot;)? Also, what about &quot;matching&quot;, does it mean &quot;perfectly matching&quot; (= class code &lt;code&gt;=&lt;/code&gt;, for &quot;complete, exact match of intron chain&quot;)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Under these assumptions, you can do some easy stats with some bash help. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;n. assembled genes: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;$cat cuffcmp.output.gtf.tmap | sed &quot;1d&quot; | cut -f4 | sort | uniq | wc -l&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;n. novel genes: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;$ cat cuffcmp.output.gtf.tmap | awk '$3==&quot;u&quot;{print $0}' | cut -f4 | sort | uniq | wc -l&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;n. novel transcripts: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;$ cat cuffcmp.output.gtf.tmap | awk '$3==&quot;u&quot;{print $0}' | cut -f5 | sort | uniq | wc -l&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;n. transcripts matching annotation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;$ cat cuffcmp.output.gtf.tmap | awk '$3==&quot;=&quot;{print $0}' | cut -f5 | sort | uniq | wc -l&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I hope this helps. I think it's an easier approach than trying to &quot;translate the numbers related to novel exons and novel introns to the number and identities of the novel transcripts/genes&quot; from the stats file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Although you have to think a bit what &quot;class codes&quot; to include in each count. Maybe my assumptions are too strict for your use case. &lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2019-07-17T12:13:55.627" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9000" PostTypeId="2" ParentId="8973" CreationDate="2019-07-17T12:23:40.313" Score="1" Body="&lt;p&gt;You need to order the marker matrix (e.g. by &lt;code&gt;avg_logFC&lt;/code&gt;) before calling &lt;code&gt;DoHeatMap&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(dplyr)&#xA;all.markers &amp;lt;- FindAllMarkers(object = obj)&#xA;top20 &amp;lt;- all.markers %&amp;gt;% group_by(cluster) %&amp;gt;% top_n(20, avg_logFC)&#xA;DoHeatmap(object = obj, genes.use = top20$gene, slim.col.label = TRUE, remove.key = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3631" LastEditorUserId="3631" LastEditDate="2019-07-17T12:28:48.373" LastActivityDate="2019-07-17T12:28:48.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9001" PostTypeId="2" ParentId="8612" CreationDate="2019-07-17T12:59:27.407" Score="1" Body="&lt;p&gt;I am not sure if it's a v3 or also v2 thing, but have you tried by setting &lt;code&gt;group.by&lt;/code&gt; and &lt;code&gt;label&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;code&gt;?Seurat::DoHeatmap&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;group.by   A vector of variables to group cells by; pass 'ident' to group by cell identity classes&#xA;label      Label the cell identies above the color bar&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you prefer external tools, you can use &lt;code&gt;pheatmap&lt;/code&gt; and setting &lt;code&gt;annotation_col&lt;/code&gt; e &lt;code&gt;annotation_colors&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First define a data.frame (df) containing the data you want in the bar (clusters in your case)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Description Condition&#xA;liver       Tumour&#xA;lung        Normal&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you have to assign the colours&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;anno &amp;lt;- as.data.frame(df)&#xA;anno_colori &amp;lt;- list(&#xA;   Condition = c(Normal = '#FFC000', Tumour = '#0070C0'),&#xA;   Description = c(liver = '#D55E00', lung = '#66CC99')&#xA;)&#xA;&#xA;pheatmap(mat,&#xA;     annotation_col = anno,&#xA;     annotation_colors = anno_colori,&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The code above will add 2 bars on the top of your heatmap, one for &lt;code&gt;Condition&lt;/code&gt; and another for &lt;code&gt;Description&lt;/code&gt;. &lt;/p&gt;&#xA;" OwnerUserId="3631" LastEditorUserId="3631" LastEditDate="2019-07-17T19:29:55.013" LastActivityDate="2019-07-17T19:29:55.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9002" PostTypeId="2" ParentId="8993" CreationDate="2019-07-17T13:53:24.960" Score="2" Body="&lt;p&gt;Try &lt;a href=&quot;https://www.rdocumentation.org/packages/gProfileR/versions/0.6.7/topics/gprofiler&quot; rel=&quot;nofollow noreferrer&quot;&gt;gProfileR&lt;/a&gt;. It is very intuitive and easy to use in R.&lt;/p&gt;&#xA;" OwnerUserId="678" LastActivityDate="2019-07-17T13:53:24.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9003" PostTypeId="1" CreationDate="2019-07-17T19:08:35.320" Score="1" ViewCount="69" Body="&lt;p&gt;I'm trying to use rentrez package to retrieve NIH files, and ran the following code (I followed what was on the rentrez tutorial):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pubmed_search &amp;lt;- entrez_search(db = &quot;pubmed&quot;, &#xA;                    term = &quot;Case Reports[Filter] AND cardiovascular disease AND English[lang] AND 2009:2019[PDat])&quot;, &#xA;                    retmax = 792711, &#xA;                    use_history = TRUE)&#xA;&#xA;for(i in seq(1,10000,100)){   &#xA;    recs &amp;lt;- entrez_summary(db=&quot;pubmed&quot;, &#xA;                           web_history=pubmed_search$web_history,&#xA;                           retmax=100, &#xA;                           retstart=i)   cat(    &#xA;seq_start+99, &quot;sequences downloaded\r&quot;) }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, instead of the 10,000 texts I expected, I only downloaded 100 files. Could so&lt;/p&gt;&#xA;" OwnerUserId="4888" LastEditorUserId="48" LastEditDate="2019-07-19T09:02:07.033" LastActivityDate="2019-08-18T10:03:00.330" Title="rentrez to retrieve NIH files" Tags="|r|entrez|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9004" PostTypeId="1" AcceptedAnswerId="9006" CreationDate="2019-07-17T19:11:10.580" Score="0" ViewCount="213" Body="&lt;p&gt;I have condition as such mouse liver and spleen data. &#xA;So in liver I have WT (Wild Type) and KO (knock out) as well as in spleen WT (Wild Type) and KO (knock out).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to compare the parasite load in WT and KO condition in both liver and spleen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of data points what I got is this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;10 mice for WT, 6 mice for KO in case of liver&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;10 mice for WT, 5 mice for KO in case of spleen.  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;WT 20 8 21 9 22 10 23 11 24 &#xA;KO 12 25 26 27 28&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So far what I know is I have to perform a parametric test.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I'm not sure if I have to go for a paired t-test or an unpaired t-test.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because here I'm comparing with between two groups, only so I don't have to go for anova if I'm not wrong. Secondly there is a difference in number of observation that is available for WT as compared to KO. So would that affect my choice of test as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suggestion would be highly appreciated &lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="48" LastEditDate="2019-07-18T07:27:51.183" LastActivityDate="2019-07-18T07:27:51.183" Title="Statistical test design question" Tags="|statistics|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9005" PostTypeId="2" ParentId="9004" CreationDate="2019-07-17T20:10:18.970" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;So far what i know is i have to perform a parametric test .&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How did you determine this?&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;But im not sure if I have to go for a&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Paired t-test or Unpaired t-test&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you can't tell the difference between when to do a paired t-test and an unpaired one, you need to stop and talk to someone who can teach you basic statistics.  Throwing statistical tests at numbers when you don't understand them is a recipe for disaster.  &lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-07-17T20:10:18.970" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9006" PostTypeId="2" ParentId="9004" CreationDate="2019-07-18T06:31:47.527" Score="2" Body="&lt;p&gt;Here's a combination of general advice and my thoughts about your specific data:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the samples are not matched, e.g. by time point, then it will be unpaired. You need a way to say sample 1 KO and sample 1 WT are uniquely matched to be paired, sample 2 KO and sample WT will be paired etc ...  pairing is great for a time series. If you have replicates within the paired groups then ANOVA is the way to go, particularly if the sample sizes are uneven.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: in your case the sample sizes are uneven, you cannnot perform a paired T-test between two uneven sample sizes. HOWEVER you appear to a natural pairing, but I don't know whether this is biologically justified, i.e. liver and spleen. So you appear to have a biological pairing with uneven sample sizes. If this is correct it suggests a &lt;strong&gt;1-way ANOVA&lt;/strong&gt; is a better idea than a T-test.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parametric test: check for normality in each data set, e.g. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A Q-Q plot against the normal distribution (a cool way to do this) &lt;/li&gt;&#xA;&lt;li&gt;Homogeneity of variances is Hartley's Fmax test (good for T-test and ESSENTIAL for 1-way ANOVA)&lt;/li&gt;&#xA;&lt;li&gt;68% of the data should be less than ±1 standard deviation around the mean, 95% less than ±2 standard deviation and 99% less than ±3&lt;/li&gt;&#xA;&lt;li&gt;Kolmogorov-Smirnov and/or Shapiro-Wilk test&lt;/li&gt;&#xA;&lt;li&gt;Skewness &amp;amp; kurtosis magnitudes &amp;lt; 1.96 respectively&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; if you combine liver and spleen data sets and find a nice normal distribution, and one or both is not normal by itself, then you would need to consider the biological justification for combining the data sets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the data passes all those tests, you are good to go. If not .... gets complicated,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Normalisation, this is where you transform the data, in your case I'm not sure, but typically log transformation and then perform the checks for normality above. There are lots of ways to normalise data.&lt;/li&gt;&#xA;&lt;li&gt;Repeat the T-test/1-way ANOVA with normalised data&lt;/li&gt;&#xA;&lt;li&gt;Ignore it - yup its top level statistical advice, it is context specific and never a universally shared idea ... I suspect that is where the advice &quot;just use a parametric test&quot; came from. Please be aware if you do this, someone could turn around and sau &quot;you've done it wrong&quot;. The statistical arguments about why this assumption can be valid get complicated.&lt;/li&gt;&#xA;&lt;li&gt;Use a non-parametric test, orthodox statistical advice, in this case it is the Mann-Whitney U test and &lt;strong&gt;Kruskal–Wallis test&lt;/strong&gt; if you have paired replicates (i.e. non-parametric version of 1-way ANOVA).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Non-parametric tests often lack power and I suspect for this sample size that will be the case. Personally I would try and ignore all this advice :-) - okay I would definately check for normality and investigate avenues towards normality - then I would perform a general linear model (GLM). The reason is that it is a much more powerful method for identifying whether the two gropus are different. Performing a single GLM with two paired groups is not trivial, so I would have to give serious consideration to 1-way ANOVA, or KW-test, which would be a cleaner solution - but could miss a critical part of the data. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-07-18T06:52:28.590" LastActivityDate="2019-07-18T06:52:28.590" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9007" PostTypeId="2" ParentId="9003" CreationDate="2019-07-18T06:46:57.037" Score="1" Body="&lt;p&gt;I don't program in R, but replace the last line of you code with the following and I suspect you will find it works,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for(i in seq(1,10000,1)){ recs &amp;lt;- entrez_summary(db=&quot;pubmed&quot;, web_history=pubmed_search$web_history, retmax=100, retstart=i) cat(seq_start+99, &quot;sequences downloaded\r&quot;) }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Rationale&lt;/strong&gt; I would guess &lt;code&gt;seq&lt;/code&gt; is similar to the &lt;code&gt;linspace&lt;/code&gt; function in Python, and put simply &lt;code&gt;10000 (total sample size) / 100 (the step size) = 100&lt;/code&gt;, 100 being the number of downloads you observed. In the code above its &lt;code&gt;10000/1&lt;/code&gt;. I would also suspect &quot;step size = 1&quot; is default, if this was correct then &lt;code&gt;seq(1,10000)&lt;/code&gt; would give you your required output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a random aside I do like the presence of curly brackets defining the loop, it reminds me of Perl.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-07-18T06:55:32.317" LastActivityDate="2019-07-18T06:55:32.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9008" PostTypeId="2" ParentId="4371" CreationDate="2019-07-18T07:09:28.480" Score="1" Body="&lt;p&gt;Its easy, &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Download the data from NCBI in fasta format (its an option in one of the pull down headers).&lt;/li&gt;&#xA;&lt;li&gt;Load it into the latest version of MEGA, version 7(?), perform the alignment therein.&lt;/li&gt;&#xA;&lt;li&gt;Export from MEGA as Nexus format and upload it into Beast&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You could of course perform the alignment in Clustalx, Clustalo or Muscle or T-Coffee, (hmmm I could write an alignment program just to name it T-4two (sorry British joke)). If the output option of &quot;nexus&quot; was not present in the alignment program of your choice, then again upload the alignment into MEGA and export it as nexus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, I could go into specific detail about the various points and clicks therein, but I think its easy to figure out. Most phylogeny programs will do an automatic fasta-&gt;nexus conversion, but anyway. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its easy, Beast is not trivial to use properly however and ensuring convergence is not at all easy, in addition the multiple simulation a priori models, whilst powerful, give different answers. To try and explain this a different way, it is easy to get an answer in Beast, but difficult to get a robust answer from it and thats the bit I would give serious consideration about. It does amaze how widespread the program is (I guess its the nice GUI), given how complicated it is to obtain robust result.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Just to add you could try the online tool &lt;a href=&quot;http://sequenceconversion.bugaco.com/converter/biology/sequences/fasta_to_nexus.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for the fasta to nexus conversion:  . I've not tried it, its just an after thought. If you do Python, Biopython will do it, but its just as easy to code.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-07-18T07:48:11.553" LastActivityDate="2019-07-18T07:48:11.553" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9010" PostTypeId="1" CreationDate="2019-07-18T08:58:34.967" Score="6" ViewCount="107" Body="&lt;p&gt;Assume I have an infix expression such as k1*s1, how do I use libsbml to translate this expression to mathml using the python interface?&lt;/p&gt;&#xA;" OwnerUserId="4894" LastEditorUserId="96" LastEditDate="2019-07-19T12:56:57.213" LastActivityDate="2019-07-29T08:42:41.447" Title="How can I translate infix to mathml using the python interface to libsbml?" Tags="|modelling|systems-biology|sbml|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9011" PostTypeId="2" ParentId="9010" CreationDate="2019-07-18T08:58:34.967" Score="4" Body="&#xA;&#xA;&lt;p&gt;It’s quite straightforward to convert infix to mathml using libsbml. Make sure you have libsbml installed in python by typing&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;pip install python-libsbml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;at the windows/Mac/Linux command line or &lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;!pip install python-libsbml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;if you are running a Jupyter notebook. Some tools such as &lt;a href=&quot;http://tellurium.analogmachine.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tellurium&lt;/a&gt; already have libsbml preinstalled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the python command line type&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import libsbml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to import the package. To enter the infix expression type the following:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;p = libsbml.parseFormula ('k1*s1')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To generate the mathml type&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;print (libsbml.writeMathMLToString (p))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4894" LastEditorUserId="4959" LastEditDate="2019-07-29T08:42:41.447" LastActivityDate="2019-07-29T08:42:41.447" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9013" PostTypeId="1" AcceptedAnswerId="9117" CreationDate="2019-07-18T15:42:30.647" Score="2" ViewCount="103" Body="&lt;p&gt;I was trying to use &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgLiftOver&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC liftover tool&lt;/a&gt; to convert human coordinates to mouse. While most of the files worked, one file didn't:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr16   86430087    86430726    Human_chr16:86430087-86430726___element_1___positive____neural_tube[12/12]___hindbrain_(rhombencephalon)[12/12]___limb[3/12]___cranial_nerve[8/12]&#xA;chr16   85620095    85621736    Human_chr16:85620095-85621736___element_2___negative_&#xA;chr16   80423343    80424652    Human_chr16:80423343-80424652___element_3___negative_&#xA;chr16   80372593    80373755    Human_chr16:80372593-80373755___element_4___positive____neural_tube[6/10]___hindbrain_(rhombencephalon)[10/10]___midbrain_(mesencephalon)[10/10]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does anyone know why this bed file can only be converted to chr:start-end format but not bed4 format?&lt;/p&gt;&#xA;" OwnerUserId="1852" LastEditorUserId="298" LastEditDate="2019-07-18T15:47:22.230" LastActivityDate="2019-07-31T10:10:17.087" Title="UCSC liftover cannot produce bed format output" Tags="|bed|ucsc|liftover|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9014" PostTypeId="1" CreationDate="2019-07-18T15:43:21.697" Score="3" ViewCount="1118" Body="&lt;p&gt;I have a Seurat object existing of an aggregate of two samples namely; RD1 and RD2. I am trying to make a subset of each sample. But it's not really working. I manage to make a subset of the cells but I don't manage to make a subset of the whole object. Isn't there a way to do this, based on the subset from cell names a subset to make from the whole object. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seuratObj&lt;span class=&quot;math-container&quot;&gt;$orignal_1 &amp;lt;- seuratObj@meta.data$&lt;/span&gt;orig.ident&#xA;seuratObj&lt;span class=&quot;math-container&quot;&gt;$orignal_2 &amp;lt;- seuratObj@meta.data$&lt;/span&gt;orig.ident&#xA;seuratObj&lt;span class=&quot;math-container&quot;&gt;$orignal_1 &amp;lt;- gsub(&quot;RD1&quot;, &quot;TRUE&quot;,seuratObj$&lt;/span&gt;orignal_1 )&#xA;seuratObj&lt;span class=&quot;math-container&quot;&gt;$orignal_1  &amp;lt;- gsub(&quot;RD2&quot;, &quot;False&quot;,seuratObj$&lt;/span&gt;orignal_1  )&#xA;seuratObj&lt;span class=&quot;math-container&quot;&gt;$orignal_2  &amp;lt;- gsub(&quot;RD1k&quot;, &quot;TRUE&quot;,seuratObj$&lt;/span&gt;orignal_2  )&#xA;seuratObj&lt;span class=&quot;math-container&quot;&gt;$orignal_2  &amp;lt;- gsub(&quot;RD2&quot;, &quot;False&quot;,seuratObj$&lt;/span&gt;orignal_2  )&#xA;&#xA;subset_DR1 &amp;lt;- data.frame(orignal_1 = seuratObj$orignal_1)&#xA;subset_DR1 &amp;lt;- subset(subset_DR1, orignal_1 == TRUE,select = c(&quot;orignal_1&quot;))&#xA;&#xA;subset_DR2 &amp;lt;- data.frame(orignal_2 = seuratObj$orignal_2)&#xA;subset_DR2 &amp;lt;- subset(subset_DR2 , orignal_2 == TRUE,select = c(&quot;orignal_2&quot;))&#xA;&#xA;SubsetData(seuratObj, cells = rownames(subset_DR1), subset.name = seuratObj$orignal_1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I just started to work with seurat packages. I think I am missing something. &lt;/p&gt;&#xA;" OwnerUserId="4892" LastEditorUserId="4892" LastEditDate="2019-07-19T07:37:24.407" LastActivityDate="2019-08-02T18:50:11.307" Title="Subsetting of object existing of two samples" Tags="|r|seurat|subset|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9015" PostTypeId="1" AcceptedAnswerId="10639" CreationDate="2019-07-18T19:47:42.710" Score="8" ViewCount="2363" Body="&lt;p&gt;I have two pandas Dataframes, using python3.x:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;dict1 = {0:['chr1','chr1','chr1','chr1','chr2'], &#xA;    1:[1, 100, 150, 900, 1], 2:[100, 200, 500, 950, 100], &#xA;    3:['feature1', 'feature2', 'feature3', 'feature4', 'feature4'], &#xA;    4:[0, 0, 0, 0, 0], 5:['+','+','-','+','+']}&#xA;&#xA;df1 = pd.DataFrame(dict1)&#xA;&#xA;print(df1)&#xA;&#xA;##       0    1    2         3  4  5&#xA;## 0  chr1    1  100  feature1  0  +&#xA;## 1  chr1  100  200  feature2  0  +&#xA;## 2  chr1  150  500  feature3  0  -&#xA;## 3  chr1  900  950  feature4  0  +&#xA;## 4  chr2    1  100  feature4  0  +&#xA;&#xA;dict2 = {0:['chr1','chr1'], 1:[155, 800], 2:[200, 901], &#xA;    3:['feature5', 'feature6'], 4:[0, 0], 5:['-','+']}&#xA;&#xA;df2 = pd.DataFrame(dict2)&#xA;print(df2)&#xA;##       0    1    2         3  4  5&#xA;## 0  chr1  155  200  feature5  0  -&#xA;## 1  chr1  800  901  feature6  0  +&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The columns to focus on in these dataframes are the first three columns: location, start, and end. Each start:end value represents a distance on location (e.g. &lt;code&gt;chr1&lt;/code&gt;, &lt;code&gt;chr2&lt;/code&gt;, &lt;code&gt;chr3&lt;/code&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to output the intersection of &lt;code&gt;df1&lt;/code&gt; against &lt;code&gt;df2&lt;/code&gt;. Here is the correct output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    155 200 feature2    0   +&#xA;chr1    155 200 feature3    0   -&#xA;chr1    900 901 feature4    0   +&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;What is the most efficient (in terms of runtime and RAM) to find the intersections using pandas?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There are python wrappers around BEDTools: &lt;a href=&quot;https://daler.github.io/pybedtools/intersections.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://daler.github.io/pybedtools/intersections.html&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I was hoping there was a pandas only solution...&lt;/p&gt;&#xA;" OwnerUserId="1770" LastEditorUserId="3848" LastEditDate="2020-04-19T11:42:33.400" LastActivityDate="2021-01-09T21:21:33.833" Title="How to do `bedtools intersection` using pandas alone?" Tags="|python|bedtools|interval|pybedtools|pandas|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9016" PostTypeId="1" AcceptedAnswerId="9022" CreationDate="2019-07-18T22:20:17.550" Score="2" ViewCount="136" Body="&lt;p&gt;I am looking for a taxonomic dataset that would allow me to links tree genera (or species) to families. In particular, I am interested in the tropical species found in the Amazon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found a long but not exhaustive list on Wikipedia:&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_trees_and_shrubs_by_taxonomic_family&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/List_of_trees_and_shrubs_by_taxonomic_family&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After processing it, I realised it contains information about 404 genera, while the dataset I am dealing with has around 600. (in the end, I only got 23% of my data covered, very sad)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know if such a crosswalk file exists, in some scientific article or somewhere in the Internet?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, please tell me if there are other StackExchange websites which could be more appropriate, I wasn't sure of where to post this.&lt;/p&gt;&#xA;" OwnerUserId="4903" LastEditorUserId="57" LastEditDate="2019-07-21T12:45:03.253" LastActivityDate="2019-07-21T12:45:03.253" Title="Looking for a taxonomic dataset that links tree genera (or species) to families" Tags="|database|public-databases|taxonomy|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9017" PostTypeId="2" ParentId="9016" CreationDate="2019-07-19T00:33:16.833" Score="4" Body="&lt;p&gt;The NCBI taxonomy is a quite extensive resource: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/taxonomy&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/taxonomy&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would thus expect there is a good chance that it covers all of the taxa that you are interested in.  It officially claims that it is not an authoritative source, but it's still pretty darned good.&lt;/p&gt;&#xA;" OwnerUserId="4869" LastActivityDate="2019-07-19T00:33:16.833" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9018" PostTypeId="1" CreationDate="2019-07-19T03:24:11.543" Score="1" ViewCount="24" Body="&lt;p&gt;I was trying to estimate the minimum sequencing depth required to confidently detect somatic variant using Poisson distribution based variant caller (such as somatic variant caller or Pisces from illumina).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Power analysis is used which requires significance level (0.05), power / sensitivity (99%) and effect size to estimate sample size (sequencing depth). I am not sure how to calculate the effect size in such context. Can anyone share some insight? Thanks a lot.&lt;/p&gt;&#xA;" OwnerUserId="4905" LastActivityDate="2019-07-19T03:24:11.543" Title="Effect size in power analysis when dealing with Poisson based variant caller" Tags="|variant-calling|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9019" PostTypeId="2" ParentId="9015" CreationDate="2019-07-19T08:47:29.890" Score="2" Body="&lt;p&gt;I implemented pandas &quot;Intervals&quot; and  ... it should be a few lines, clearly there are limitations. For non-overlapping data it is very cool however. It will work for overlapping data, BUT if the data you are using as the interval data is overlapping, it falls over. It could work if an independent (non-overlapping) interval was constructed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, the point of the excercise was to keep it all in pandas,  &quot;start_stop&quot; should not be needed. This can't be done via Intervals, you've got to drop out, and that will slow it down and harder to parse. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The query contig &lt;code&gt;df3.iloc[1,]&lt;/code&gt;, is still a bug (hence not printed), should be [x,] where x is the value_counts hit, but this is just a pilot. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The script would be fine if it the df was wrangled using &lt;code&gt;groupby&lt;/code&gt; rather than &lt;code&gt;value_counts&lt;/code&gt; because &lt;code&gt;value_counts&lt;/code&gt; trashes important data, which is what I'm trying to recover via &lt;code&gt;df3.iloc[1,]&lt;/code&gt;. I like pandas elegant solutions, but you know another day another dollar. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I changed 200 to 199 otherwise there's too much overlap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Overlap Output&lt;/strong&gt; (not labeled very well)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                 start  stop&#xA;index                      &#xA;(1.0, 100.0]        0     0&#xA;(100.0, 199.0]      1     0&#xA;(150.0, 500.0]      1     1&#xA;(900.0, 950.0]      0     1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;def tableit (dict):&#xA;    df = pd.DataFrame(dict)&#xA;    table = pd.pivot_table(df, values=['start', 'stop'], columns=['chromosome'], index=['feature', 'misc', 'misc2'], fill_value = '-')&#xA;    return table&#xA;&#xA;def start_stop(point, table2, bin):&#xA;    df3 = pd.cut(table2[point,'chr1'], bin[0]).value_counts()&#xA;    df3 = df3.to_frame(point).reset_index()&#xA;    for x in bin:&#xA;            if not (x == bin[0]):&#xA;                df3_tmp = pd.cut(table2[point,'chr1'], x).value_counts()&#xA;                df3_tmp = df3_tmp.to_frame(point).reset_index()&#xA;                df3 = pd.concat([df3, df3_tmp]                                  &#xA;            else:&#xA;                continue&#xA;    df3.set_index('index', inplace=True)&#xA;    return df3, df3.iloc[1,] # [1,] is a bug and the query locus needs correctly reading from df3_tmp&#xA;&#xA;def main():&#xA;    dict1 = {'chromosome':['chr1','chr1','chr1','chr1','chr2'], 'start':[1, 100, 150, 900, 1], 'stop':[100, 199, 500, 950, 100], 'feature':['feature1', 'feature2', 'feature3', 'feature4', 'feature4'], 'misc':[0, 0, 0, 0, 0], 'misc2':['+','+','-','+','+']}&#xA;    dict2 = {'chromosome':['chr1','chr1'], 'start':[155, 800], 'stop':[200, 901], 'feature':['feature5', 'feature6'], 'misc':[0, 0], 'misc2':['-','+']}&#xA;&#xA;    table1 = tableit(dict1)&#xA;    table2 = tableit(dict2)&#xA;&#xA;    bin = [[i, j] for i, j in zip(table1['start', 'chr1'], table1['stop', 'chr1'])]&#xA;&#xA;    df_start, locus = start_stop('start', table2, bin)&#xA;# locus.name gets to the data&#xA;    df_stop, _ = start_stop('stop', table2, bin)&#xA;    df_merge = pd.merge(df_start, df_stop, on=&quot;index&quot;, how='outer')&#xA;    print (df_merge)&#xA;&#xA;if __name__ == &quot;__main__&quot;:&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The following would have been better,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;table_join = pd.concat([table1, table2]).fillna('-')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then sort them.. perhaps including a new column to specify the original dataframe and screen for the overlap with table_join.iloc[i, column] - table_join.iloc[1-i, other_column].&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key step in all approaches is to convert from &quot;long format&quot; to &quot;wide format&quot; and thats the most important thing I've done in all code and makes pandas work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output Table_join&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                     start        stop     &#xA;chromosome            chr1 chr2   chr1 chr2&#xA;feature  misc misc2                        &#xA;feature1 0    +        1.0    -  100.0    -&#xA;feature2 0    +      100.0    -  199.0    -&#xA;feature3 0    -      150.0    -  500.0    -&#xA;feature4 0    +      900.0    1  950.0  100&#xA;feature5 0    -      155.0    -  200.0    -&#xA;feature6 0    +      800.0    -  901.0    -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-07-21T11:24:12.680" LastActivityDate="2019-07-21T11:24:12.680" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9021" PostTypeId="1" AcceptedAnswerId="15278" CreationDate="2019-07-19T00:03:33.923" Score="1" ViewCount="605" Body="&lt;p&gt;I'm trying to align sequences using MUSCLE in my own computer. To compute Percent Identity Matrix I'm trying to use Clustal2.1 like it's done in &lt;a href=&quot;https://www.ebi.ac.uk/Tools/msa/muscle/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/Tools/msa/muscle/&lt;/a&gt;. Even though I'm doing the same thing, I get rounded up values in my computer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, this is the webtool output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#&#xA;#&#xA;#  Percent Identity  Matrix - created by Clustal2.1 &#xA;#&#xA;#&#xA;&#xA;     1: JX310755.1  100.00   92.79   65.29   63.98&#xA;     2: M97264.1     92.79  100.00   65.66   63.92&#xA;     3: AJ292230.1   65.29   65.66  100.00   66.88&#xA;     4: KM379144.1   63.98   63.92   66.88  100.00&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this is the output in my computer.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#&#xA;#&#xA;#  Percent Identity  Matrix - created by Clustal2.1 &#xA;#&#xA;#&#xA;&#xA;     1: JX310755.1     100      93      65      64&#xA;     2: M97264.1        93     100      66      64&#xA;     3: AJ292230.1      65      66     100      67&#xA;     4: KM379144.1      64      64      67     100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have both installed clustalw using &lt;code&gt;sudo apt install clustalw&lt;/code&gt; and by downloading the tarballs from &lt;a href=&quot;http://www.clustal.org/download/current/clustalw-2.1.tar.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.clustal.org/download/current/clustalw-2.1.tar.gz&lt;/a&gt;, with exact results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The commmand I'm using is &lt;code&gt;clustalw2 alignment -tree -pim&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any thoughts on why this could be? Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="10365" OwnerDisplayName="tokuretsu" LastActivityDate="2021-01-28T01:46:19.450" Title="Differences in webtool and standalone Clustal Omega Percent Identity Matrix" Tags="|sequence-alignment|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9022" PostTypeId="2" ParentId="9016" CreationDate="2019-07-19T10:07:46.943" Score="2" Body="&lt;p&gt;Probably the best currently available online database is RBG Kew's &lt;a href=&quot;http://www.plantsoftheworldonline.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Plants of the World Online&lt;/a&gt;. It organises taxonomic, morphological and distribution data from a range of databases. You can search for your taxa of interest and find their associated higher ranks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, there is &lt;a href=&quot;http://www.worldfloraonline.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;World Flora Online&lt;/a&gt;, replacing the now-defunct &lt;a href=&quot;http://www.theplantlist.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Plant List v1.1&lt;/a&gt;, which also integrates taxonomic data from a range of reliable sources and national lists. &lt;/p&gt;&#xA;" OwnerUserId="2841" LastActivityDate="2019-07-19T10:07:46.943" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="9024" PostTypeId="2" ParentId="9021" CreationDate="2019-07-19T12:00:42.817" Score="1" Body="&lt;p&gt;Try clustal omega, instead of clustalw because it is really old. It should report to 2 dp. Variations do occur depending on how indels are treated, but this is just rounding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could try &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;alignment -tree -pim -score&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The idea is to convert to absolute rather than percentage and might overcome the rounding error&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-07-19T12:00:42.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9025" PostTypeId="2" ParentId="9015" CreationDate="2019-07-19T12:42:10.350" Score="4" Body="&lt;p&gt;I don't think Pandas has this implemented functionality out-of-the-box. Even if it did, solutions not designed specifically for bioinformatics probably rarely handle intervals on different chromosomes correctly unless you split the intervals by chromosome first.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pandas does &lt;em&gt;handle&lt;/em&gt; intervals (see docs for the &lt;a href=&quot;https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Interval.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Interval&lt;/a&gt; and &lt;a href=&quot;https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.IntervalIndex.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;IntervalIndex&lt;/a&gt; classes), but I've never used these. I &lt;em&gt;have&lt;/em&gt; used the excellent &lt;a href=&quot;https://github.com/chaimleib/intervaltree/&quot; rel=&quot;nofollow noreferrer&quot;&gt;intervaltree&lt;/a&gt; package. Whatever approach and tools you decide to use, there's a good chance you'll have to write some code. Computing interval intersections with &lt;code&gt;intervaltree&lt;/code&gt; would look something like this.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; from collections import defaultdict&#xA;&amp;gt;&amp;gt;&amp;gt; from intervaltree import IntervalTree&#xA;&amp;gt;&amp;gt;&amp;gt; from pandas import DataFrame&#xA;&amp;gt;&amp;gt;&amp;gt; &#xA;&amp;gt;&amp;gt;&amp;gt; dict1 = {&#xA;...     0:['chr1','chr1','chr1','chr1','chr2'],&#xA;...     1:[1, 100, 150, 900, 1],&#xA;...     2:[100, 200, 500, 950, 100],&#xA;...     3:['feature1', 'feature2', 'feature3', 'feature4', 'feature4'],&#xA;...     4:[0, 0, 0, 0, 0],&#xA;...     5:['+','+','-','+','+']&#xA;... }&#xA;&amp;gt;&amp;gt;&amp;gt; dict2 = {&#xA;...     0:['chr1','chr1'],&#xA;...     1:[155, 800],&#xA;...     2:[200, 901],&#xA;...     3:['feature5', 'feature6'],&#xA;...     4:[0, 0],&#xA;...     5:['-','+']&#xA;... }&#xA;&amp;gt;&amp;gt;&amp;gt; df1 = DataFrame(dict1)&#xA;&amp;gt;&amp;gt;&amp;gt; df2 = DataFrame(dict2)&#xA;&amp;gt;&amp;gt;&amp;gt; &#xA;&amp;gt;&amp;gt;&amp;gt; &#xA;&amp;gt;&amp;gt;&amp;gt; # Construct an interval index from one of the dataframes,&#xA;&amp;gt;&amp;gt;&amp;gt; #   one tree per chromosome&#xA;&amp;gt;&amp;gt;&amp;gt; index = defaultdict(IntervalTree)&#xA;&amp;gt;&amp;gt;&amp;gt; for n, row in df2.iterrows():&#xA;...     index[row[0]].addi(row[1], row[2], row)&#xA;... &#xA;&amp;gt;&amp;gt;&amp;gt; &#xA;&amp;gt;&amp;gt;&amp;gt; # Query the index from the other dataframe to build a 3rd&#xA;&amp;gt;&amp;gt;&amp;gt; #   dataframe representing the intersection&#xA;&amp;gt;&amp;gt;&amp;gt; dict3 = { i: list() for i in range(6) }&#xA;&amp;gt;&amp;gt;&amp;gt; for n, row in df1.iterrows():&#xA;...     overlapping = index[row[0]].overlap(row[1], row[2])&#xA;...     for itvl in overlapping:&#xA;...         itsct_start = max(row[1], itvl.begin)&#xA;...         itsct_end = min(row[2], itvl.end)&#xA;...         dict3[0].append(row[0])&#xA;...         dict3[1].append(itsct_start)&#xA;...         dict3[2].append(itsct_end)&#xA;...         dict3[3].append(row[3])&#xA;...         dict3[4].append(row[4])&#xA;...         dict3[5].append(row[5])&#xA;... &#xA;&amp;gt;&amp;gt;&amp;gt; df3 = DataFrame(dict3)&#xA;&amp;gt;&amp;gt;&amp;gt; df3&#xA;      0    1    2         3  4  5&#xA;0  chr1  155  200  feature2  0  +&#xA;1  chr1  155  200  feature3  0  -&#xA;2  chr1  900  901  feature4  0  +&#xA;&amp;gt;&amp;gt;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-07-19T12:42:10.350" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9026" PostTypeId="1" CreationDate="2019-07-19T13:47:26.897" Score="1" ViewCount="434" Body="&lt;p&gt;Lemmatization is one of the most important tasks for text normalization in natural language processing (NLP), i.e. using deep learning to understand text data.&lt;/p&gt;&#xA;&lt;p&gt;One of the good lemmatizers I found for biomedical domain is &lt;a href=&quot;http://biolemmatizer.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioLemmatizer&lt;/a&gt;. However, it is developed in Java and is available from maven repo. I would like to use it from Python, but there seems to be no counterpart for BioLemmatizer in Python. Is there a way I can access BioLemmatizer from Python using some module like JPype. JPype, however, supports access only for the default classes in Java.&lt;/p&gt;&#xA;" OwnerUserId="3015" LastEditorUserId="3848" LastEditDate="2020-07-14T09:13:00.000" LastActivityDate="2024-03-25T20:07:25.973" Title="Is there a way to use BioLemmatizer from Python?" Tags="|python|text-processing|java|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9027" PostTypeId="1" CreationDate="2019-07-19T14:10:00.167" Score="1" ViewCount="351" Body="&lt;p&gt;I wish to normalize a scRNASeq dataset with respect to ERCC spike-ins, where &quot;for some of the samples, ERCC spike-in RNA was added to the lysis buffer&quot; and was wondering how to do so?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen similar questions posted on the forum, such as the following: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/52/normalization-methods-with-rna-seq-ercc-spike-in&quot;&gt;Normalization methods with RNA-Seq ERCC spike in?&lt;/a&gt; and have checked out the package RUVSeq in the top answer. However, in this package, it seems that spike-ins are taken as transcripts, whereas in the dataset I'm analyzing it seems as though 'spike-ins' are respective of the cell being sequenced rather than a transcript. In addition, in my research I stumbled upon the following tutorial for preprocessing of scRNASeq data: &lt;a href=&quot;https://f1000research.com/articles/5-2122/v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://f1000research.com/articles/5-2122/v2&lt;/a&gt; where also, as in the RUVSeq package, &quot;ERCC&quot; is &lt;code&gt;grep&lt;/code&gt;'d through the &lt;code&gt;rownames&lt;/code&gt; to identify ERCC transcripts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which leads me to my second question: what is a spike-in if it is not a transcript - and therefore &lt;em&gt;not&lt;/em&gt; within the &lt;code&gt;rownames&lt;/code&gt; of the dataset and rather within the samples? Please note that I am not from a biological background and, I have read the wikipedia article, along with many others, in my quest to understand what a spike-in is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, please note that I have a metadata file that states exactly which of the samples have these spike-ins and which dont. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any advice on how to proceed would be much appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="4001" LastActivityDate="2019-07-19T14:10:00.167" Title="Normalization of single cell RNASeq data with ERCC spike-ins" Tags="|r|scrnaseq|normalization|spike-in|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="9028" PostTypeId="1" CreationDate="2019-07-19T19:12:30.057" Score="4" ViewCount="890" Body="&lt;p&gt;I am working with paired end NGS miseq data from a viral genome with multiple timepoints.  I have filtered and trimmed this data for quality and adapter sequences.  I have then merged the filtered paired end reads and mapped them to a reference genome using bowtie2.  I then used samtools to filter for mapping quality and visualized the resulting filtered bam file on IGV.&lt;/p&gt;&#xA;&lt;p&gt;My goal is to now scan along the genome using various window sizes (e.g. 300 nt) across the reads that have mapped to the reference. I will then chop up the genome and build phylogenetic trees based on each window.&lt;/p&gt;&#xA;&lt;p&gt;How can I extract a multiple sequence alignment of these 300 nt windows from a sam or bam file using the coordinates of a reference viral genome?  I have tried using &lt;code&gt;samtools view&lt;/code&gt; but this does not actually provide the subsection or alignment within the window but rather all reads that overlap the window in some way.&lt;/p&gt;&#xA;&lt;p&gt;The virus is HIV1 and the immediate goal is to get an alignment of all reads that completely span a user specified coordinate range eg. 1-300 based on some reference sequence. Ideally reads that fit this criteria would be extracted and aligned. After ragged edges are removed, the resulting alignments subjected to phylogenetic analyses.&lt;/p&gt;&#xA;" OwnerUserId="4910" LastEditorUserId="73" LastEditDate="2021-05-05T13:01:40.083" LastActivityDate="2022-01-30T14:06:45.733" Title="How to get a MSA fasta from BAM/SAM?" Tags="|sam|samtools|covid-19|sars-cov-2|bowtie2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9029" PostTypeId="1" CreationDate="2019-07-20T12:56:30.057" Score="2" ViewCount="552" Body="&lt;p&gt;I have CLL samples as fastq files and I want to remove those reads which have a unique sequence with less than 10 read counts each. I tried this by following way using awk to make it faster:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for fq in &lt;span class=&quot;math-container&quot;&gt;$(ls fastq1/*.fq.gz); do&#xA;    basename=$&lt;/span&gt;(basename &quot;&lt;span class=&quot;math-container&quot;&gt;$fq&quot; | cut -f1 -d '.')&#xA;    zcat $&lt;/span&gt;fq | awk &quot;/^@/{getline; print}&quot; |sort | uniq --count | awk '{if(&lt;span class=&quot;math-container&quot;&gt;$1&amp;gt;= 10) print $&lt;/span&gt;2}' &amp;gt; tmp.txt; grep -B 1 -A 2 -f tmp.txt &lt;span class=&quot;math-container&quot;&gt;$fq | grep -v ^--'| pigz -p 10 &amp;gt; fastq2/$&lt;/span&gt;basename.fastq.gz&#xA;&#xA;done&#xA;&#xA;rm tmp.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But this is not working well as many reads with more than 10 read counts are also missing. I think it's OK to stick with bash/awk for fast fastq file processing. What am I missing here, or is this command wrong. Can anyone suggest me the right and fast way to do it??&lt;/p&gt;&#xA;" OwnerUserId="2141" LastActivityDate="2019-07-25T02:01:22.823" Title="Filter reads belonging to unique sequences with threshold" Tags="|fastq|bash|awk|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="9030" PostTypeId="2" ParentId="9029" CreationDate="2019-07-20T22:44:00.320" Score="1" Body="&lt;p&gt;Your algorithm generally works, you just had one minor issue, and a few stylistic changes that could be made.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd adjust your script to the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for fq in fastq1/*.fq.gz; do&#xA;    basename=&lt;span class=&quot;math-container&quot;&gt;$(basename &quot;$&lt;/span&gt;fq&quot; .fq.gz)&#xA;    zcat &quot;&lt;span class=&quot;math-container&quot;&gt;$fq&quot; | awk &quot;/^@/{getline; print}&quot; | sort | uniq --count | awk '{if($1&amp;gt;= 10) print $2}' &amp;gt; tmp.txt&#xA;	zcat | grep --no-group-separator -B 1 -A 2 -f tmp.txt &quot;$&lt;/span&gt;fq&quot; | pigz -p 10 &amp;gt; &quot;fastq2/$basename.fastq.gz&quot;&#xA;done&#xA;&#xA;rm tmp.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Instead of&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;for fq in $(ls fastq1/*.fq.gz)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;use this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;for fq in fastq1/*.fastq.gz&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Iterating over &lt;code&gt;ls&lt;/code&gt; is &lt;a href=&quot;http://mywiki.wooledge.org/ParsingLs&quot; rel=&quot;nofollow noreferrer&quot;&gt;generally not ideal&lt;/a&gt;, and can give unexpected results. Better to just rely on bash globbing.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Purely stylistic, but I'd simplify and use &lt;code&gt;basename&lt;/code&gt; to remove the suffix, instead of &lt;code&gt;cut&lt;/code&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Rather than removing the separators that &lt;code&gt;grep&lt;/code&gt; will introduce by filtering them out, use the following option to not create them:&#xA;&lt;code&gt;grep --no-group-separator&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;And, minor bug, as others have mentioned, you should use &lt;code&gt;zgrep&lt;/code&gt; or &lt;code&gt;zcat | grep&lt;/code&gt; since your files are compressed.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Oh - and quote your variables.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, while this does work, it may not be the most efficient method. The file is being decompressed twice, and may be better to decompress once to a temporary file that can be used and then deleted.&lt;/p&gt;&#xA;" OwnerUserId="92" LastEditorUserId="92" LastEditDate="2019-07-25T02:01:22.823" LastActivityDate="2019-07-25T02:01:22.823" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9032" PostTypeId="1" CreationDate="2019-07-21T12:33:14.677" Score="0" ViewCount="271" Body="&lt;p&gt;I have to analyse &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE114327&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSE114327&lt;/a&gt;. For that, I use GSE instruction based on &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM3139597&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSM3139597&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In that data processing section has written:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;FASTQ reads were subjected to adapter removal (cutadapt) per manufacturer's instructions (NEXTflex Small RNA Sequencing Kit v3).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, I use &lt;code&gt;cutadapt&lt;/code&gt; for trimming. Then &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The degenerate sequences (N4) at the 5' and 3' end of the reads were removed&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;based on the instruction. But I don’t understand that sentence and I don’t know how I can remove degenerate sequence (N4) at the 5' and 3' of the reads.&lt;/p&gt;&#xA;" OwnerUserId="4915" LastEditorUserId="48" LastEditDate="2019-07-22T13:52:58.207" LastActivityDate="2019-07-24T08:51:36.667" Title="How to remove degenerate sequence (N4) at the 5' and 3' of the reads?" Tags="|fastq|data-preprocessing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9033" PostTypeId="2" ParentId="9026" CreationDate="2019-07-21T14:31:27.363" Score="0" Body="&lt;p&gt;This area is huge in Python because of machine learning text processing, in fact O'Reilly have an entire book on it . However, we don't really deal with words in Bioinfo SE, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might want to look at the nltk (natural language toolkit) for Python, but for direct parallels with the Java package bio package (your question) I don't think it's something we would know.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;You could solve this via subprocessing into Java, and passing the result back to Python. I get the idea that the core ML/ DL used in natural language is set up for Python. Google for example have fantastic tensorflow stack for this.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-07-22T08:55:35.430" LastActivityDate="2019-07-22T08:55:35.430" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9034" PostTypeId="1" CreationDate="2019-07-21T23:02:56.703" Score="1" ViewCount="65" Body="&lt;p&gt;I have a set of cell line pairs and I want to know to what extent the pairs are similar based on their ontologies. &#xA;The problem I have is that I have found a Python library called Fastsemsim, but it has many similarity measurements. Or to be exact, there are actually many ontology similarity measurements. And I want to know which similarity measurement is best suited for my case (cell line ontologies similarities). There might be a couple of good ones that are good for cells ontologies. &#xA;These are a set of similarity measurements that are available in Fastsemsim.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Semantic Similarity&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;SemSim.TermSemSim&lt;/li&gt;&#xA;&lt;li&gt;SemSim.ObjSemSim&lt;/li&gt;&#xA;&lt;li&gt;SemSim.ObjSetSemSim&lt;/li&gt;&#xA;&lt;li&gt;SemSim.SetSemSim&lt;/li&gt;&#xA;&lt;li&gt;Specific Semantic Similarity measures&lt;/li&gt;&#xA;&lt;li&gt;SemSim.MixSemSim&lt;/li&gt;&#xA;&lt;li&gt;SemSim.avgSemSim&lt;/li&gt;&#xA;&lt;li&gt;SemSim.maxSemSim&lt;/li&gt;&#xA;&lt;li&gt;SemSim.BMASemSim&lt;/li&gt;&#xA;&lt;li&gt;SemSim.SemSimUtils&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Source:&#xA;&lt;a href=&quot;https://fastsemsim.readthedocs.io/en/latest/library.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://fastsemsim.readthedocs.io/en/latest/library.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4917" LastActivityDate="2019-07-22T12:42:44.230" Title="semantic similarity measurement for cell line ontologies" Tags="|python|single-cell|cell-line|ontology|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9035" PostTypeId="1" CreationDate="2019-07-22T01:36:57.110" Score="3" ViewCount="189" Body="&lt;p&gt;I have aligned a nanopore data set to a reference genome with graphmap, minimap2 and BLASR. The alignment results are stored in BAM files. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to do some concordance assessment, looking at the number of base pairs that are mapped to the same position by different aligner.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any tools or ways that allow me to do this effectively?&lt;/p&gt;&#xA;" OwnerUserId="4881" LastEditorUserId="48" LastEditDate="2019-07-22T13:49:31.557" LastActivityDate="2019-08-15T13:26:56.553" Title="Compare multiple alignment results' aligned bases" Tags="|sequence-alignment|sam|nanopore|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9036" PostTypeId="2" ParentId="9029" CreationDate="2019-07-22T05:03:07.670" Score="1" Body="&lt;p&gt;Checking for line start to find the read header is not the best option. Even if unlikely it is possible that the qualitys starts with a @. It's better to compare the current line number. Also you can use awk directly to count and print the result.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zcat &lt;span class=&quot;math-container&quot;&gt;$fq|awk 'NR%4==2 {count[$&lt;/span&gt;0]++} END { for(sequence in count) { if(count[sequence]&amp;gt;=10) print sequence }}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As the other statet out, the second command should be fine if you use &lt;code&gt;zcat&lt;/code&gt; instead of &lt;code&gt;cat&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1310" LastActivityDate="2019-07-22T05:03:07.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9037" PostTypeId="2" ParentId="9032" CreationDate="2019-07-22T12:27:36.743" Score="3" Body="&lt;p&gt;From the link you posted it looks as though they used &lt;a href=&quot;https://github.com/CGATOxford/UMI-tools&quot; rel=&quot;nofollow noreferrer&quot;&gt;UMI tools&lt;/a&gt; as per the section which says&#xA;&lt;em&gt;'were removed and added to the name of the read as a unique molecular identifier (UMI) using UMI tools'&lt;/em&gt;.  The link above has instructions for installation and use.&#xA;From this I would guess that this data has Unique Molecular Identifiers (UMIs) for each read.&lt;br&gt;&#xA;Looking up the protocol for the nextflex kit used &lt;a href=&quot;http://www.biooscientific.com/portals/0/manuals/ngs/5132-05-nextflex-small-rna-seq-v3.pdf?utm_source=copy+of+citation+dist+melanoma&amp;amp;utm_campaign=small+rna+v3+launch+dist&amp;amp;utm_medium=email&quot; rel=&quot;nofollow noreferrer&quot;&gt;(See Link)&lt;/a&gt; confirms this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Data Analysis&lt;/strong&gt;&#xA;  The 3' and 5' adapters included in this kit both contain 4 random bases that will appear immediately 5' and 3' to the insert in sequencing data. The presence of these random bases should be considered when choosing an alignment strategy. When using “end-to-end” alignment, we&#xA;  recommend processing data in the following manner: &#xA;  1. Clip the 3' adapter sequence (TGGAATTCTCGGGTGCCAAGG).&#xA;  2. Trim the first and last 4 bases from the adapter-clipped reads.&#xA;  3. Perform alignments as normal. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It seems that N4 is a 4 base random barcode at both the 5'&amp;amp; 3' end of the read - these are cut from the read and this metadata is added to the read name so that every read can be uniquely identified.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Edit:&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can install UMI tools via:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;conda install -c bioconda -c conda-forge umi_tools&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;pip install umi_tools&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usage instructions can be found at:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://umi-tools.readthedocs.io/en/latest/reference/extract.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://umi-tools.readthedocs.io/en/latest/reference/extract.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Usage:&#xA;  For single ended reads, the following reads from stdin and outputs to stdout:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;umi_tools extract --extract-method=string --bc-pattern=[PATTERN] -L extract.log [OPTIONS]&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;For paired end reads, the following reads end one from stdin and end two from FASTQIN and outputs end one to stdin and end two to FASTQOUT:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;umi_tools extract --extract-method=string --bc-pattern=[PATTERN] --bc-pattern2=[PATTERN] --read2-in=[FASTQIN] --read2-out=[FASTQOUT] -L extract.log [OPTIONS]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3100" LastEditorUserId="3100" LastEditDate="2019-07-24T08:51:36.667" LastActivityDate="2019-07-24T08:51:36.667" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9038" PostTypeId="2" ParentId="9034" CreationDate="2019-07-22T12:42:44.230" Score="1" Body="&lt;p&gt;About the metric you use, there are several reviews on the topic, if I recall well I think that the information content based are the best ones, of those, I only recognized the JiangConrathSemSim listed on the webpage. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you want to use a semantic similarity measure I would use the BMA method, which is more comprehensive than other methods. But the BMA method is only a way to aggregate several metrics of ontologies into a single numeric value. I am not confident that FastSemSim implements correctly those methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, that according to the &lt;a href=&quot;https://sourceforge.net/projects/fastsemsim/files/&quot; rel=&quot;nofollow noreferrer&quot;&gt;license&lt;/a&gt;, you should contact the authors before using it:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;You must contact us before using it or its results or any application/tool based on it in any published work.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I am not sure that the cell lines fit in a ontolgogy. Because ontologies are defined as directed acyclic graph (DAG) and there are some cycles that happens in cells like the induced pluripotent stem cells show. However they might be right if you just compare the cell lines and not the state of the cells on those lines (use it only if you want to compare the lineage of your cells lines).&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2019-07-22T12:42:44.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9039" PostTypeId="1" AcceptedAnswerId="9052" CreationDate="2019-07-22T13:48:55.797" Score="2" ViewCount="2972" Body="&lt;p&gt;I have my analysis on a cloud server. The data is quite large, so copying it locally is not really an option. I want to be able to host IGV as a server on that server, so that it can access those files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I installed IGV web (&lt;a href=&quot;https://github.com/igvteam/igv-webapp&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/igvteam/igv-webapp&lt;/a&gt;) on my server, and ran it, but the Tracks → Local File option opens a file browser on &lt;em&gt;my&lt;/em&gt; local machine, it doesn't let me select files on the server.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any easy way to host IGV web on a cloud server?&lt;/p&gt;&#xA;" OwnerUserId="4645" LastActivityDate="2020-04-17T04:11:21.827" Title="Hosting IGV web on a server" Tags="|sam|igv|cloud|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9040" PostTypeId="2" ParentId="9039" CreationDate="2019-07-22T14:02:50.597" Score="1" Body="&lt;p&gt;According to their discription &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;igv-webapp is a pure-client &quot;genome browser&quot; application based igv.js.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Pure-client, so it will be difficult to let it display tracks that are on the server itself. Perhaps you are looking for something more similar to the &lt;a href=&quot;https://genomearchitect.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Apollo Genome browser&lt;/a&gt;. This, while more difficult to install, allows for the display of server-side data tracks and allows for manual curating of annotations as a nice bonus :)&lt;/p&gt;&#xA;" OwnerUserId="4552" LastActivityDate="2019-07-22T14:02:50.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9041" PostTypeId="1" CreationDate="2019-07-22T18:29:22.867" Score="2" ViewCount="400" Body="&lt;p&gt;I am working on a sample which was sequenced using an amplicon kit and I would like to ensure that no variants are called in the PCR primer sequences. I know that the primer sequences are part of the genome and a variant there is no less valid than a variant anywhere else, but for reasons I cannot control, I need to ensure that no variants can be called in the primer regions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first thought was to use a soft-clipping tool like &lt;a href=&quot;https://github.com/tommyau/bamclipper&quot; rel=&quot;nofollow noreferrer&quot;&gt;BAMClipper&lt;/a&gt;. However, the variant caller I am using (&lt;a href=&quot;https://www.sentieon.com/products/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sentieon's implementation of GATK&lt;/a&gt;) preforms a local realignment when calling and if it finds that soft-clipped regions align perfectly against the genome, it will remove the soft clips. This makes perfect sense for cases where the softclips were added by the aligner, but sadly breaks my workflow since I need the softclipped primers to remain clipped.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would therefore like to hard clip, to completely remove the primer sequences from the bam file before variant calling. I have found one tool that seems to be able to do this, &lt;a href=&quot;http://fulcrumgenomics.github.io/fgbio/tools/latest/TrimPrimers.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;TrimPrimers&lt;/a&gt; by Fulcrum Genomics, but this fails on my input bam file with &lt;a href=&quot;https://pastebin.com/QRYK2kJY&quot; rel=&quot;nofollow noreferrer&quot;&gt;an error&lt;/a&gt; I can't really make much of. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any other tool that can take a list of regions (e.g. a bedpe file) and a BAM and hard-clip the regions from the BAM?&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-07-23T03:13:50.403" Title="Hard-clip primers in a bam file" Tags="|sam|primer|base-clipping|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9042" PostTypeId="5" CreationDate="2019-07-22T19:05:35.393" Score="0" Body="" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2019-07-22T19:05:35.393" LastActivityDate="2019-07-22T19:05:35.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9043" PostTypeId="4" CreationDate="2019-07-22T19:05:35.393" Score="0" Body="Used for questions regarding the &quot;Synthetic Biology Open Language&quot;, which can be used to describe genetic circuits, function, experimental data, etc." OwnerUserId="77" LastEditorUserId="77" LastEditDate="2019-07-22T19:05:35.393" LastActivityDate="2019-07-22T19:05:35.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9044" PostTypeId="2" ParentId="9041" CreationDate="2019-07-22T19:45:36.713" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I know that the primer sequences are part of the genome and a variant&#xA;  there is no less valid than a variant anywhere else,&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You won't see variants under PCR primers.  You get the primer sequence itself in the vast majority of reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wouldn't clip the bam file, I'd clip either the fastq, or clip within bcl2fastq while making the fastqs.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-07-22T19:45:36.713" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="9045" PostTypeId="1" AcceptedAnswerId="9046" CreationDate="2019-07-22T20:30:08.213" Score="1" ViewCount="566" Body="&lt;p&gt;I wish to conduct WGCNA on a single cell RNASeq dataset and, when choosing the optimal beta parameter after running &lt;code&gt;pickSoftThreshold&lt;/code&gt; am presented with a rather strange plot: &#xA;&lt;a href=&quot;https://i.stack.imgur.com/GkqGL.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GkqGL.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have normalized the data using &lt;code&gt;NormalizeData&lt;/code&gt; in &lt;code&gt;Seurat&lt;/code&gt; using the LogNormalize method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of my colleagues suggested using &lt;code&gt;hclust&lt;/code&gt; as a method for outlier detection to see potential cells/samples that could be excluded. I think this makes sense however I haven't found much information on such procedures besides the following paper: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5845381/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5845381/&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question would be, is this procedure just be as simple as running a hierarchical clustering on the data and seeing which samples cluster by themselves (and then removing those based on a minimum cluster size criteria) or is their more to it? Also, for those familiar with WGCNA, why does the scale-free topolgy plot generate such strange looking values, is this indicative of outliers being present in the data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your time.&lt;/p&gt;&#xA;" OwnerUserId="4001" LastActivityDate="2019-07-22T21:00:02.183" Title="Hierarchical clustering for outlier detection - single cell RNASeq &amp; WGCNA" Tags="|r|scrnaseq|seurat|clustering|wgcna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9046" PostTypeId="2" ParentId="9045" CreationDate="2019-07-22T21:00:02.183" Score="2" Body="&lt;p&gt;For outlier identification I suggest using the &lt;a href=&quot;https://bmcsystbiol.biomedcentral.com/articles/10.1186/1752-0509-6-63&quot; rel=&quot;nofollow noreferrer&quot;&gt;sample network approach&lt;/a&gt; developed by Oldham et al. It basically amounts to constructing a inter-sample connectivity (assuming &lt;code&gt;normalizedData&lt;/code&gt; contains the log-normalized data with genes in colunms and samples in rows)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;k = colSums(cor(t(normalizedData)))&lt;/code&gt;, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;scaling the connectivities &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Z.k = scale(k)&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and then removing samples with high negative Z.k (e.g. Z.k &amp;lt; -4). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A slightly different approach is based on euclidean distances rather than correlations, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Z.k = scale(as.matrix(dist(normalizedData)))&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and removing samples with high &lt;em&gt;positive&lt;/em&gt; Z.k (since distance is a measure of dissimilarity). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding the scale-free topology fit plot, I wouldn't worry about it too much, the fit index sometimes jumps around a bit. It certainly does not look like you have strong outliers or very heterogeneous data. But it is always a good idea to plot the sample clustering tree to check (1) that there are no strong clusters, and (2) that the outlier identification did something reasonable. &lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2019-07-22T21:00:02.183" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9047" PostTypeId="2" ParentId="9041" CreationDate="2019-07-23T03:13:50.403" Score="2" Body="&lt;p&gt;Another option would be to adjust the quality of the primer bases to be below the threshold that your variant caller uses.  In my case I have &quot;masked&quot; the primer bases by dropping the quality to 0.  This prevents the variants from being called based that are synthetic but if you have an overlapping amplicon it will call bases from those reads.&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2019-07-23T03:13:50.403" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9048" PostTypeId="1" CreationDate="2019-07-23T08:41:51.443" Score="1" ViewCount="232" Body="&lt;p&gt;I want to compare the results from functional enrichment (e.g GO term enrichment generated by gProfileR) between two gene lists. The idea is to see if these gene lists are enriched in the same terms or if they have specific terms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is easy to do at a glance when you have few terms but when the number of terms increases and due to redundancies in GO term annotation a manual comparison becomes unreliable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the questions is: &lt;strong&gt;Is there a tool for comparing the results from functional enrichment between different gene lists?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="678" LastActivityDate="2019-07-25T12:44:22.843" Title="Compare functional enrichment between genelists" Tags="|functional-annotation|go-enrichment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9049" PostTypeId="2" ParentId="9048" CreationDate="2019-07-23T08:56:28.323" Score="1" Body="&lt;p&gt;To compare gene list based on gene ontologies you can use &lt;a href=&quot;https://bioconductor.org/packages/GOSemSim/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GOSemSim&lt;/a&gt; (Note, I'm a contributor to the package). You can select the semantic similarity to compare them, that is how the redundancies and hierarchical relations are counted. The main result you can obtain is similarity matrix for each comparison between gene lists. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However if you perform a GO term enrichment and then you compare the gene list of each GO term by looking at which GO do they belong it won't be much useful. I would suggest you to use &lt;a href=&quot;https://bioconductor.org/packages/BioCor/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioCor&lt;/a&gt; (I'm the developer of it), which can use any pathway information (such as the ones of Reactome) to asses if they are related to the same pathways. &lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2019-07-25T12:44:22.843" LastActivityDate="2019-07-25T12:44:22.843" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9050" PostTypeId="2" ParentId="9039" CreationDate="2019-07-23T09:25:44.230" Score="1" Body="&lt;p&gt;One possibility might be to put the full IGV application on the server, and use the igvR package in R on the server to access the IGV API.  You could then use R plotting facilities etc to show your data, accessed via the IGV API.  There might be APIs for IGV in other languages, if you prefer, not sure.&lt;br&gt;&#xA;Edit:  I haven't tried this package, this is a speculative idea.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastEditorUserId="3828" LastEditDate="2019-07-23T09:33:19.100" LastActivityDate="2019-07-23T09:33:19.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9052" PostTypeId="2" ParentId="9039" CreationDate="2019-07-23T17:11:59.147" Score="7" Body="&lt;p&gt;You can use X11 forwarding to run IGV on the headless server where the data is located and simply forward the GUI to your client machine using the X11 protocol. This is a solution I use consistently for my research when working with 20+ tracks that I want to keep on the server.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Download and install IGV for linux from the IGV &lt;a href=&quot;https://software.broadinstitute.org/software/igv/download&quot; rel=&quot;noreferrer&quot;&gt;site&lt;/a&gt; on to your server. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Connect to the server using ssh with X11 forwarding enabled&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;ssh -X user@server&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Navigate to the recently downloaded and unzipped IGV folder. Execute the igv.sh script.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;./igv.sh&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The script launches the IGV session with 4gb of memory. Edit the script to launch with more, 32gb in this edited excerpt.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;exec java -showversion --module-path=&quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}/lib&quot; -Xmx32g \&#xA;@&quot;$&lt;/span&gt;{prefix}/igv.args&quot; \&#xA;-Dapple.laf.useScreenMenuBar=true \&#xA;-Djava.net.preferIPv4Stack=true \&#xA;--module=org.igv/org.broad.igv.ui.Main &quot;$@&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="181" LastActivityDate="2019-07-23T17:11:59.147" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9055" PostTypeId="1" CreationDate="2019-07-24T06:42:51.477" Score="3" ViewCount="84" Body="&lt;p&gt;The genomes are from same species. &#xA;Is it true that, in phylogenetic tree constructed from amino-acid based MSA (multiple sequence alignment) some information are lost, so for  phylogenetic reconstruction of closely related sequences nucleotide based MSA  should be used? &lt;/p&gt;&#xA;" OwnerUserId="3555" LastEditorUserId="3848" LastEditDate="2020-03-17T10:35:35.530" LastActivityDate="2020-03-17T10:35:35.530" Title="For phylogenetic tree construction from core-genome which one is preferable: amino-acid based MSA or nucleotide based MSA?" Tags="|sequence-alignment|fasta|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9056" PostTypeId="2" ParentId="9055" CreationDate="2019-07-24T07:16:44.980" Score="4" Body="&lt;p&gt;For a species complex nucleotide phylogenies all the way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason is neutral mutation, which is not observed at the amino acid level because these mutations are part of the degenerate code which don't induce amino acid substitutions. Such mutations are commonly referred to as the third codon wobble, although occasionally the first codon can mutate and not produce a change in amino acid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Neutral otherwise known as synonymous mutations occur at a much higher rate than mutations that cause a substitution of an amino acid. This is the reason why  you were told information is lost at the amino acid level because you're losing neutral mutations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are circumstances when the amino acid phylogenies are preferred to nucleoside trees, but these are bit pretty specialised areas.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-07-24T07:16:44.980" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9057" PostTypeId="1" AcceptedAnswerId="9058" CreationDate="2019-07-24T12:22:01.073" Score="3" ViewCount="1773" Body="&lt;p&gt;There are commercial sequencing kits/sequencers that allow for paired end sequencing in which the two reads obtained for each fragment are of different length?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example 150bp for R1 and 50bp for R2?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it at least possible in principle? There are some published speculations about that possibility?&lt;/p&gt;&#xA;" OwnerUserId="3772" LastActivityDate="2019-07-24T13:31:43.367" Title="Paired end sequencing with R1 and R2 of different length. Possible?" Tags="|ngs|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9058" PostTypeId="2" ParentId="9057" CreationDate="2019-07-24T13:31:43.367" Score="5" Body="&lt;p&gt;As indicated in the comments, yes, you can definitely tell standard Illumina sequencers to sequence mates in a pair to different lengths. This is quite common in single-cell RNA-seq, where R1 is often just a cell barcode and an UMI, so sequencing it to 150 bases would be a huge waste or reagent. One caveat to this is that you can't buy reagents sized for these custom length, just things like 150 base PE or other standards. So unless you end up saving and pooling reagents after such runs you end up paying the same for projects with one shorter read than if you'd just directly sequenced everything to the same length. Given that sequencer runs only take a day or two at the moment, this also doesn't result in that immense of a time savings.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-07-24T13:31:43.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9060" PostTypeId="2" ParentId="2580" CreationDate="2019-07-24T17:52:19.263" Score="2" Body="&lt;p&gt;This error came from the fact that  the output of the new fimo version has an additional 'motif_alt_id' column in second position. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;CRCmapper has been modified to take this into account. You can find the updated version here: &lt;a href=&quot;https://bitbucket.org/young_computation/crcmapper/src/master/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bitbucket.org/young_computation/crcmapper/src/master/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S.: sorry if we missed you request user1545 and thanks for using CRCmapper!&lt;/p&gt;&#xA;" OwnerUserId="4926" LastActivityDate="2019-07-24T17:52:19.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9061" PostTypeId="1" CreationDate="2019-07-24T20:24:23.450" Score="4" ViewCount="49" Body="&lt;p&gt;When representing a DNA sequence in SBOL as a &lt;code&gt;ComponentDefinition&lt;/code&gt;, you can mark things like promoters and coding sequences in two different ways:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;either as a &lt;code&gt;SequenceAnnotation&lt;/code&gt; in the &lt;code&gt;ComponentDefinition&lt;/code&gt;, or &lt;/li&gt;&#xA;&lt;li&gt;as a &lt;code&gt;Component&lt;/code&gt; linking to a &lt;code&gt;ComponentDefinition&lt;/code&gt; for the sub-comoponent&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;When should you choose one over the other?&lt;/p&gt;&#xA;" OwnerUserId="4869" LastEditorUserId="4942" LastEditDate="2019-07-29T10:29:33.993" LastActivityDate="2019-07-29T10:29:45.007" Title="In SBOL, when should I annotate a DNA sequence vs. making sub-components?" Tags="|sequence-annotation|sbol|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9062" PostTypeId="2" ParentId="9061" CreationDate="2019-07-25T00:15:54.680" Score="4" Body="&lt;p&gt;Typically you want to use a &lt;code&gt;ComponentDefinition&lt;/code&gt; when an sub-structure is something you might want to pull out and re-use in another genetic design, and a &lt;code&gt;SequenceAnnotation&lt;/code&gt; whenever it is not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Promoters, terminators, and protein coding sequences are typically best represented with their own &lt;code&gt;ComponentDefinition&lt;/code&gt;, since these are often pulled out and re-combined in new genetic designs.  Representing such an element with a &lt;code&gt;ComponentDefinition&lt;/code&gt; makes it easy to find the commonalities between designs.&lt;/li&gt;&#xA;&lt;li&gt;Assembly scars, binding sites, and cut sites are typically best represented with a &lt;code&gt;SequenceAnnotation&lt;/code&gt; because they are typically not very interesting except in the context of their surrounding genetic context. Representing such an element with a &lt;code&gt;SequenceAnnotation&lt;/code&gt; helps keep them associated tightly with that context.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Not every case is cut and dried, however.  For example, ribosome entry sites are often treated as separable components, but their performance is very tightly tied to the specifics of the coding sequence that they modulate.  Thus, in some cases it may make more sense to represent them separately as a &lt;code&gt;ComponentDefinition&lt;/code&gt; and in others to represent them as a &lt;code&gt;SequenceAnnotation&lt;/code&gt; on a joint RES/CDS &lt;code&gt;ComponentDefinition&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4869" LastEditorUserId="4942" LastEditDate="2019-07-29T10:29:45.007" LastActivityDate="2019-07-29T10:29:45.007" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9063" PostTypeId="1" CreationDate="2019-07-25T00:21:09.097" Score="1" ViewCount="107" Body="&lt;p&gt;I am trying to download a dataset from GEO using the GEOquery R package. Using this package, I can retrieve the GPL platform number(s) for the dataset, and I hope to use a CDF file from BrainArray on the dataset in order to undo any existing normalization performed, so that I can have multiple datasets that are normalized in a standard manner. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to do this, I need to know what the microarray platform name is. This information is usually provided on the website online (for example, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE15245&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, you can see that next to GPL96, it says HG-U133A, which is the CDF I would use from BrainArray).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I am wondering if there is any way to parse this platform name using the GEOquery package in R itself rather than having to manually check for every dataset and compare it to the BrainArray website? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is not possible to do this in the GEOquery package, is there any online tool for mapping GPL to platform name? I found &lt;a href=&quot;https://gist.github.com/seandavi/bc6b1b82dc65c47510c7&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; online which has the mapping for a decent amount of GPL numbers, but not all of them, meaning it sometimes crashes. I was wondering if there is a foolproof way to do what I am trying to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for the help in advance!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know I can access the highlighted portion in the first image below (the GPL platform number) through GEOquery. I'm wondering if I can access the name of the platform (highlighted in the second image). If not, my alternative option will probably be to scrape the web page to find this information, but if its possible to do so through GEOquery I'd rather do that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lseje.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lseje.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4929" LastEditorUserId="48" LastEditDate="2019-07-26T07:29:05.693" LastActivityDate="2019-07-26T07:29:05.693" Title="Mapping GPL Platform Number to Microarray Name" Tags="|r|bioconductor|geoquery|geo|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9064" PostTypeId="1" CreationDate="2019-07-25T00:33:33.503" Score="3" ViewCount="43" Body="&lt;p&gt;Do I need to include intermediate gene products like mRNA in order to express a gene regulatory relationship in SBOL?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, let's say that I'm representing transcriptional regulation of the pTet promoter by means of a coding sequence that expresses TetR. Do I need to represent the TetR transcript mRNA as well as the TetR protein?&lt;/p&gt;&#xA;" OwnerUserId="4869" LastEditorUserId="96" LastEditDate="2019-07-25T01:52:07.177" LastActivityDate="2019-07-26T00:28:07.773" Title="Does SBOL require representing intermediate products like mRNA?" Tags="|modelling|sbol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9065" PostTypeId="1" CreationDate="2019-07-25T12:26:56.407" Score="2" ViewCount="47" Body="&lt;p&gt;What is the point of having both a Component and a ComponentDefinition in SBOL? Shouldn't you be able to link straight from one component to its sub-components?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The same question goes for Module and ModuleDefinition.&lt;/p&gt;&#xA;" OwnerUserId="4869" LastActivityDate="2019-07-26T00:34:45.897" Title="Why does SBOL have both Component and ComponentDefinition?" Tags="|sbol|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9066" PostTypeId="1" CreationDate="2019-07-25T14:43:12.987" Score="1" ViewCount="184" Body="&lt;p&gt;I am trying to use DRIMseq for DTU with 2 treatments on two different strains of an animal model. &#xA;How do I select a proper design matrix to use in DRIMSeq ? How I know that one is the correct one to use? And how do I see which transcripts are differentially used among conditions after the pipeline was run? I suspect it is in the plotProportions function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, where do I see the logFoldChange between conditions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many thanks in advance as it would help me a lot in my work.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; sessionInfo()&#xA;R version 3.5.3 (2019-03-11)&#xA;Platform: x86_64-w64-mingw32/x64 (64-bit)&#xA;Running under: Windows &amp;gt;= 8 x64 (build 9200)&#xA;&#xA;Matrix products: default&#xA;&#xA;attached base packages:&#xA;[1] stats     graphics  grDevices utils     datasets  methods   base     &#xA;&#xA;other attached packages:&#xA;[1] sva_3.30.1          BiocParallel_1.16.6 genefilter_1.64.0   mgcv_1.8-28        &#xA;[5] nlme_3.1-137        edgeR_3.24.3        limma_3.38.3       &#xA;&#xA;loaded via a namespace (and not attached):&#xA; [1] Rcpp_1.0.1           compiler_3.5.3       pillar_1.4.2         bitops_1.0-6        &#xA; [5] tools_3.5.3          digest_0.6.20        zeallot_0.1.0        bit_1.1-14          &#xA; [9] annotate_1.60.1      RSQLite_2.1.1        memoise_1.1.0        tibble_2.1.3        &#xA;[13] lattice_0.20-38      pkgconfig_2.0.2      rlang_0.4.0          Matrix_1.2-17       &#xA;[17] DBI_1.0.0            rstudioapi_0.10      yaml_2.2.0           parallel_3.5.3      &#xA;[21] IRanges_2.16.0       S4Vectors_0.20.1     vctrs_0.2.0          locfit_1.5-9.1      &#xA;[25] stats4_3.5.3         bit64_0.9-7          grid_3.5.3           Biobase_2.42.0      &#xA;[29] AnnotationDbi_1.44.0 survival_2.44-1.1    XML_3.98-1.20        blob_1.2.0          &#xA;[33] matrixStats_0.54.0   backports_1.1.4      splines_3.5.3        BiocGenerics_0.28.0 &#xA;[37] xtable_1.8-4         RCurl_1.95-4.12      crayon_1.3.4  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user4936" LastEditorDisplayName="user4936" LastEditDate="2019-08-14T13:20:31.077" LastActivityDate="2019-08-14T13:20:31.077" Title="How do I select a proper design matrix to use in DRIMSeq?" Tags="|r|rna-seq|design|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9067" PostTypeId="2" ParentId="9035" CreationDate="2019-07-25T16:10:17.743" Score="1" Body="&lt;p&gt;I don't know of a single tool that can read multiple sets of BAM files and compare them. But you can use &lt;a href=&quot;https://github.com/KyleLevi/SAM_Reader#how-do-i-examine-the-matches-at-each-individual-position-in-the-genome&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Sam_Reader python3 module&lt;/a&gt; to create a csv file that summarizes each position in the genome for each of your 3 results and then compare them manually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should be installable with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pip3 install sam_reader&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then generate the CSV with &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from sam_reader import Sam_Reader&#xA;my_files = Sam_Reader('my_bam_file_or_folder/')&#xA;my_files.per_base_stats(write_file='new_file.csv')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;*but FYI this module requires &lt;a href=&quot;https://pysam.readthedocs.io/en/latest/installation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;pysam&lt;/a&gt;, which does not work on windows&lt;/p&gt;&#xA;" OwnerUserId="4937" LastActivityDate="2019-07-25T16:10:17.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9068" PostTypeId="2" ParentId="9066" CreationDate="2019-07-25T17:41:48.877" Score="2" Body="&lt;p&gt;From your question I understand that you have two treatments in two mouse strains.&lt;br&gt;&#xA;If you are interested in general treatment effects independet of the animal strain you would use the both variables in the design formula (similar to controlling for a batch effect).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;meta.data &amp;lt;- data.frame(treatment = c(&quot;a&quot;, &quot;a&quot;, &quot;b&quot;, &quot;b&quot;), strain = c(&quot;x&quot;, &quot;y&quot;, &quot;x&quot;, &quot;y&quot;))&#xA;design_combined &amp;lt;- model.matrix(~ treatment + strain, data = meta.data)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want to make the comparisons for each strain separately, the easiest way will be to split your data set by strain and do a set up just for treatment. Alternatively, you can also merge the two columns into a new one in case you also want to also compare between the strains.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;meta.data$combined &amp;lt;- paste0(meta.data$treatment, &quot;_&quot;, meta.data$strain)&#xA;# refactor to change the reference level&#xA;meta.data$combined &amp;lt;- factor(meta.data$combined, levels = &quot;bx&quot;, &quot;by&quot;, &quot;ax&quot;, &quot;ay&quot;)&#xA;design_combined &amp;lt;- model.matrix(~ combined, data = meta.data)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To get the fold changes, you can extract the proportions for each transcript using&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DRIMSeq::proportions(dataSet)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And from the proportions the log2 fold changes can be calculated with ease.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when I look at your plot something seems very off with your analysis right now. In all the examples I have seen and also in my own data the diamonds are representing the mean proportion of a transcript in a condition. See this example from the DRIMSeq manual on Bioconductor:&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/252dK.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/252dK.jpg&quot; alt=&quot;DRIMSeq example from the manual&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case there are several different means per group and if you look at the line to the very right the two treatments seem to be to be right on top of each other.&lt;br&gt;&#xA;So I think that treatment and strain are somehow mixed in your setup and this is causing this kind of merged plot. If you use a design as outlined above you should get much better results. Additionally, you could maybe check that the meta data is correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also in case you have not read it yet, the manual is a great resource:&lt;br&gt;&#xA;&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/DRIMSeq/inst/doc/DRIMSeq.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/vignettes/DRIMSeq/inst/doc/DRIMSeq.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2019-07-25T17:41:48.877" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9069" PostTypeId="2" ParentId="9035" CreationDate="2019-07-25T18:36:05.980" Score="1" Body="&lt;p&gt;I'm not aware of any tools that do this out-of-the-box. Assuming some programming experience, it would be fairly straightforward to implement this in Python using &lt;a href=&quot;https://pysam.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;pysam&lt;/a&gt; (or htslib bindings for your favorite scripting language). In particular, the pileup is a very useful data structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, you'll need to store an index of reference sequence positions by read name and position.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;read 1, offset 0 --&amp;gt; chrom 3, offset 248879&#xA;read 1, offset 1 --&amp;gt; chrom 3, offset 248880&#xA;read 1, offset 2 --&amp;gt; chrom 3, offset 248882&#xA;...&#xA;...&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Once you've done this for a pair of BAM files, computing the concordance statistics involves trivial set operations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are some important practical considerations though. If you implement the approach completely in memory, it will only be tractable for small-ish BAM files. It wouldn't take very many reads (relatively speaking) to overwhelm available RAM on a laptop or desktop. For large BAM files, you might even overwhelm the available RAM on a big-memory server. Alternatively, you could store the indexes in an on-disk database such as &lt;a href=&quot;https://docs.python.org/3/library/sqlite3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sqlite3&lt;/a&gt;. This would keep RAM requirements low, but would slow things down substantially.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the most reasonable approach would be to process the BAM files in batches: grab all the reads in a 100kb region (or whatever size makes sense), compute the index, store the congruence statistics, move on to the next 100kb region and repeat, and then aggregate statistics when all regions have been processed.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-07-25T18:36:05.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9070" PostTypeId="2" ParentId="9065" CreationDate="2019-07-25T19:56:11.403" Score="2" Body="&lt;p&gt;The key to understanding this is to think about a system with more than one copy of a genetic construct. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, let's say you have a construct that uses two copies of the pBAD promoter, each regulating a different coding sequence. Both of these promoters are pBAD promoters, with the same sequence and type information, so they should share the same &lt;code&gt;ComponentDefinition&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But there are two copies. In the context of the larger construct, we need to be able to talk about each copy separately. For example, we might need to say that the first comes after a MoClo cut site, while the second comes after a terminator. For that, we need the &lt;code&gt;Component&lt;/code&gt; object, to have a way to say &quot;this particular copy of pBAD&quot;, as opposed to &quot;pBAD in general&quot;, which is what the &lt;code&gt;ComponentDefinition&lt;/code&gt; tells us about.&lt;/p&gt;&#xA;" OwnerUserId="4869" LastActivityDate="2019-07-25T19:56:11.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9071" PostTypeId="2" ParentId="8849" CreationDate="2019-07-25T20:07:59.223" Score="2" Body="&lt;p&gt;This question is somewhat generic, so a generic answer is that ENCODE has a &lt;a href=&quot;https://www.encodeproject.org/chip-seq/transcription_factor/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Transcription Factor ChIP-seq Data Standards and Processing&lt;/a&gt; page that can give you a useful starting point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For TF ChIP-seq data with replicates, the Irreproducible Discovery Rate (IDR) method helps leverage replicates to produce higher confidence peak calls, producing both &quot;optimal&quot; and &quot;conservative&quot; peaks, defined by some IDR threshold. Here's &lt;a href=&quot;https://www.encodeproject.org/data-standards/terms/#concordance&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENCODE's description of the method&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A statistical procedure that operates on the replicated peak set and compares consistency of ranks of these peaks in individual replicate/pseudoreplicate peak sets. Peaks with high rank consistency are retained. IDR can operate on peaks across a pair of true replicates resulting in a “conservative” output peak set, or across a pair of pseudoreplicates resulting in an “optimal“ output peak set. Peaks in the conservative peak set can be interpreted as high confidence peaks, representing reproducible events across true biological replicates and accounting for true biological and technical noise. Peaks in the optimal set can be interpreted as high-confidence peaks, representing reproducible events and accounting for read sampling noise. The optimal set is more sensitive, especially when one of the replicates has lower data quality than the other.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The Python code to run IDR on peak sets can be found on &lt;a href=&quot;https://github.com/ENCODE-DCC/chip-seq-pipeline2/blob/master/src/encode_idr.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENCODE's GitHub Chip-seq pipeline repo&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2019-07-25T20:07:59.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9072" PostTypeId="1" AcceptedAnswerId="9074" CreationDate="2019-07-25T20:46:30.087" Score="1" ViewCount="988" Body="&lt;p&gt;Anyone here knows how to add a new gene (GFP) to customer reference by using cellranger? We use the Cre/loxp system by using R26RmTmG mice and want to use GFP to isolate specific cells. I have read the cellranger workflow but still, have no idea how to do it. Like it is said that I should update the GTF file as shown in the screenshot. &lt;a href=&quot;https://i.stack.imgur.com/VA1mD.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VA1mD.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3171" LastActivityDate="2019-07-26T20:53:54.237" Title="making a customer reference by using cell ranger" Tags="|single-cell|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9073" PostTypeId="2" ParentId="9072" CreationDate="2019-07-25T21:28:03.160" Score="0" Body="&lt;p&gt;Update your gtf file, update the genome fasta, run cellranger mkref.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-07-25T21:28:03.160" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9074" PostTypeId="2" ParentId="9072" CreationDate="2019-07-25T21:36:32.037" Score="1" Body="&lt;p&gt;Step 1: &#xA;Add the the GFP sequence to your fasta file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;GFP_name&#xA;ATG.....TAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Step 2:&#xA;Annotate the gtf so that it contains all the necessary information to annotate your information&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Add this line to your GTF file but modify where necessary:&#xA;&quot;Make sure that all columns are tab separated&quot;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GFP_name    AddedGene   exon    1   1966    .   +   1   gene_id &quot;GFP&quot;; transcript_id &quot;GFP&quot;; gene_name &quot;GFP&quot;; transcript_name &quot;GFP&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The first column has to correspond to the header of the fasta sequence &lt;/li&gt;&#xA;&lt;li&gt;start and end indicates the length of your sequence. I added a 1966 bp long sequence to my gtf&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Code example:&#xA;step 1:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat reference.fa gfp_seq.fa &amp;gt; new_reference.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;step 2:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat genes.gtf gfp.gtf &amp;gt; new_genes.gtf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;-  gfp_seq.fa is your custom gene in fasta file&#xA;-  gfp.gtf is your custom gtf line or lines if you want to add more&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also open your current reference fasta and gtf file in a text editor and copy paste the sequence and gtf line in there. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then run &lt;code&gt;cellranger mkref&lt;/code&gt; on your new_reference and new_genes file&lt;/p&gt;&#xA;" OwnerUserId="2675" LastEditorUserId="2675" LastEditDate="2019-07-26T20:53:54.237" LastActivityDate="2019-07-26T20:53:54.237" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="9075" PostTypeId="2" ParentId="9064" CreationDate="2019-07-26T00:28:07.773" Score="2" Body="&lt;p&gt;As far as I understand, the purpose of SBOL is not to create a comprehensive model, but to &lt;em&gt;communicate a design&lt;/em&gt;.  So the question you have to ask is &quot;is the mRNA part of my design, or is it an implicit part of the biology which does not help to convey what I'm trying to build?&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think your summary is a giveaway:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For example, let's say that I'm representing &lt;strong&gt;transcriptional regulation&lt;/strong&gt; of the &lt;strong&gt;pTet promoter&lt;/strong&gt; by means of a &lt;strong&gt;coding sequence that expresses TetR&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Notice that the mRNA was not mentioned?  I also daresay that if you drew this design as a diagram, the mRNA would be omitted - another indication that it is not helpful in communicating what you designed.&lt;/p&gt;&#xA;" OwnerUserId="4942" LastActivityDate="2019-07-26T00:28:07.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9076" PostTypeId="2" ParentId="9065" CreationDate="2019-07-26T00:34:45.897" Score="1" Body="&lt;p&gt;Essentially, a &lt;code&gt;ComponentDefinition&lt;/code&gt; represents &lt;em&gt;the component itself&lt;/em&gt;, while a &lt;code&gt;Component&lt;/code&gt; represents &lt;em&gt;the component being used as a sub-component&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is &lt;a href=&quot;https://github.com/SynBioDex/SEPs/blob/master/sep_015.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;a proposal&lt;/a&gt; within the SBOL community as to whether for SBOL version 3 &lt;code&gt;ComponentDefinition&lt;/code&gt; should be renamed to &lt;code&gt;Component&lt;/code&gt; and &lt;code&gt;Component&lt;/code&gt; should be renamed to &lt;code&gt;SubComponent&lt;/code&gt; to make this situation less confusing.&lt;/p&gt;&#xA;" OwnerUserId="4942" LastActivityDate="2019-07-26T00:34:45.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9077" PostTypeId="1" CreationDate="2019-07-26T09:45:08.633" Score="2" ViewCount="327" Body="&lt;p&gt;I am doing paired end RNA-seq analysis. I used STAR mapper and then StringTie for getting quantification of my data. After successfully running StringTie, I got &lt;code&gt;.gtf&lt;/code&gt; files which will give me information about genes, their coverage along with TPM and FPKM values. It also produced some more &lt;code&gt;.ctab&lt;/code&gt; files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to know how what percentage of my reads mapped to reference genome. I &lt;a href=&quot;https://www.biostars.org/p/5165/&quot; rel=&quot;nofollow noreferrer&quot;&gt;know&lt;/a&gt; this can be done with BAM or BED files produced after STAR alignment. However, I want to know if this can be done with output files generated by StringTie?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I just want to clarify my question after reading comments and answers I am getting. I want to know if this can be done with &lt;code&gt;.gtf&lt;/code&gt; files of StringTie. As I mentioned before, I already know how to achieve this with BAM files. This is little bit weird situation because, I have deleted my BAM files to get more space on my hard drive. So I could just run entire analysis again, but wanted to know if it is possible without redoing whole analysis. Apologies for the confusion. &lt;/p&gt;&#xA;" OwnerUserId="1155" LastEditorUserId="1155" LastEditDate="2019-07-26T19:22:43.593" LastActivityDate="2019-07-26T19:22:43.593" Title="Extract mapping coverage from GTF files" Tags="|coverage|star|stringtie|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9078" PostTypeId="2" ParentId="9077" CreationDate="2019-07-26T09:49:49.870" Score="1" Body="&lt;p&gt;So consider this, you know how many reads map to each of your genes (1). Does that tell you anything about (2) the number of reads that mapped against non-gene regions (3) the number of reads that did not map against the genome at all? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you only use the gtf files, you know the number of (1) but not of (2) and (3). To me it sounds like you would want to do something as described in &lt;a href=&quot;https://www.biostars.org/p/138116/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What it boils down to is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finding out how many (unique) reads are mapped, counting pairs once&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -F 0x4 foo.sorted.bam | cut -f 1 | sort | uniq | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Counting how many total reads you have. Since you have paired-end data the number of forward reads suffice:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat forward_reads.fastq | grep '@' | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The percentage is then simply &lt;code&gt;mapped_reads/total_reads&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="4552" LastEditorUserId="298" LastEditDate="2019-07-26T11:28:43.847" LastActivityDate="2019-07-26T11:28:43.847" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9079" PostTypeId="1" CreationDate="2019-07-26T11:06:25.133" Score="3" ViewCount="35" Body="&lt;p&gt;In SBOL, how do you represent small molecules like arabinose or chloramphenicol?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's a type for saying a component is a small molecule, but they don't really have sequences like DNA or proteins do, so what's the recommended way to provide information about it other than just the name?&lt;/p&gt;&#xA;" OwnerUserId="4869" LastEditorUserId="4869" LastEditDate="2019-07-26T15:00:08.420" LastActivityDate="2019-07-26T15:00:08.420" Title="How do you represent small molecules in SBOL?" Tags="|sbol|" AnswerCount="0" CommentCount="0" ClosedDate="2021-06-25T11:15:36.953" ContentLicense="CC BY-SA 4.0" />
  <row Id="9082" PostTypeId="2" ParentId="9077" CreationDate="2019-07-26T16:52:52.607" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I want to know how what percentage of my reads mapped to reference&#xA;  genome.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;And the Log.final.out file from STAR doesn't tell you that?&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-07-26T16:52:52.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9083" PostTypeId="1" AcceptedAnswerId="9085" CreationDate="2019-07-26T21:34:45.327" Score="2" ViewCount="518" Body="&lt;p&gt;I am having a single cell RNA seq data from ~500 Cells. I do understand this number is very low for present day analysis tools like Seurat. So, I want to know what if one have a such a small number of cells which tool works best?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4951" LastActivityDate="2019-07-27T17:39:27.517" Title="Single Cell RNA seq Analysis for low input Cells?" Tags="|rna-seq|scrnaseq|single-cell|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9084" PostTypeId="1" CreationDate="2019-07-27T06:23:44.723" Score="1" ViewCount="695" Body="&lt;p&gt;I have plotted a triangular plot,that three dimensions of which represent three herbal strategies. One dimension represents the strategy of C (competitive plant), the second dimension “S” (stress tolerant plants) and the third dimension”R” (ruderal plants), as you can see in the image below,&#xA;the points on it represent the plant species. Now I am trying to write name of  the plant species outside the triangle and connect with the arrow to the corresponding point. How do I do this work?&#xA;Many thanks in advance as it would help me a lot in my work. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TernaryPlot()&#xA;#Plot two stylised plots side by side, and plot data&#xA;par(mfrow=c(1, 1), mar=rep(0.3, 4))&#xA;TernaryPlot(atip='C%', btip='R%', ctip='S%',&#xA;            point='UP', lab.cex=0.8, grid.minor.lines=0,&#xA;            grid.lty='solid', col='#FFFFFF', grid.col='GREY', &#xA;            axis.col=rgb(0.1, 0.1, 0.1), ticks.col=rgb(0.1, 0.1, 0.1),&#xA;            padding=0.08)&#xA;data_points &amp;lt;- list(&quot;Bromus dantonia&quot; = c(47,   59, 149),&#xA;                    &quot;Calamagrosis psoudo phragmatis&quot; = c(90,    102,    63),&#xA;                    &quot;Carex diluta&quot; = c(109, 64, 82),&#xA;                    &quot;Carex divisa&quot; = c(96,  99, 59),&#xA;                    &quot;Carex pseudocyperus&quot; = c(130,  71, 54),&#xA;                    &quot;Carex stenophylla&quot; = c(97, 98, 59),&#xA;                    &quot;Catabrosa aquatica&quot; = c(100,   5,  150),&#xA;                    &quot;Centaurea iberica&quot; = c(124,    85, 46),&#xA;                    &quot;Cirsium hygrophilum&quot; = c(158,  42, 55),&#xA;                    &quot;Cladium mariscus&quot; = c(159, 96, 0),&#xA;                    &quot;cod2&quot; = c(54,  82, 119),&#xA;                    &quot;Cynodon dactylon&quot; = c(121, 54, 80),&#xA;                    &quot;Eleocharis palustri&quot; = c(124,  100,    31),&#xA;                    &quot;Epilobium parviflorum&quot; = c(67, 80, 107),&#xA;                    &quot;Eromopoa persica&quot; = c(83,  15, 157),&#xA;                    &quot;Funaria cf.microstoma&quot; = c(8,  0,  247),&#xA;                    &quot;Glaux maritime&quot; = c(4, 196,    55),&#xA;                    &quot;Hordeum brevisubulatum&quot; = c(76,    70, 109),&#xA;                    &quot;Hordeum glaucum&quot; = c(40,   79, 136),&#xA;                    &quot;Inula britannica&quot; = c(95,  108,    51),&#xA;                    &quot;Juncus articulatus&quot; = c(107,   79, 69),&#xA;                    &quot;Blysmus compressus&quot; = c(81,    127,    47),&#xA;                    &quot;Juncusinflexus&quot;= c(149,    106,    0),&#xA;                    &quot;Medicago polymorpha&quot; = c(60,   86, 109),&#xA;                    &quot;Mentha spicata&quot; = c(150,   23, 82),&#xA;                    &quot;Ononis spinosa&quot; = c(66,    112,    77),&#xA;                    &quot;Phragmites australis&quot; = c(234, 0,  21),&#xA;                    &quot;Plantago amplexicaulis&quot; = c(108,   83, 64),&#xA;                    &quot;Poa trivialis&quot; = c(90, 28, 138),&#xA;                    &quot;Polygonum paronychioides&quot; = c(20,  12, 223),&#xA;                    &quot;Potentila reptans&quot; = c(106,    41, 108),&#xA;                    &quot;Potentilla anserina&quot; = c(105,  58, 91),&#xA;                    &quot;Ranunculus grandiflorus&quot; = c(129,  25, 101),&#xA;                    &quot;Schoenus nigricans&quot; = c(143,   91, 21),&#xA;                    &quot;Setaria viridis&quot; = c(10,   7,  238),&#xA;                    &quot;Sonchus oleraceus&quot; = c(178,    0,  77),&#xA;                    &quot;Taraxacum officinale&quot; = c(117, 28, 110),&#xA;                    &quot;Trifolium repens&quot; = c(94,  4,  157),&#xA;                    &quot;Triglochin martima&quot; = c(63,    96, 95),&#xA;                    &quot;Veronica anagallis-aquatica&quot; = c(55,   37, 163)&#xA;)&#xA;AddToTernary(points, data_points, pch=21, cex=1.2, &#xA;             bg=vapply(data_points, &#xA;                       function (x) rgb(x[1], x[2], x[3], 128, maxColorValue=255),&#xA;                       character(1))&#xA;)&#xA;&#xA;AddToTernary(text, data_points, names(data_points), cex=0.8, font=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LyF7q.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LyF7q.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jOIfT.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jOIfT.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4952" LastEditorUserId="73" LastEditDate="2021-04-29T05:29:59.217" LastActivityDate="2021-04-29T05:29:59.217" Title="How do I add external labels and connecting lines to a Ternary Plot in R?" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9085" PostTypeId="2" ParentId="9083" CreationDate="2019-07-27T17:39:27.517" Score="3" Body="&lt;p&gt;You can perfectly use any of the single cell processing packages out there. Seurat, Monocle3, Scanpy, Kallisto, etc. Following some of the tutorials and vignettes will most likely help you figuring out how to proceed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These packages provide useful functions to explore single cell data, process the data and each of them has a series of advanced features to address specific questions. You don't need 10k cells to look at number of gene/cell, transcript/cells, % mitochondrial genes, etc. You can perfectly normalize and standardize 500 cells and run PCA, and analyze the complexity that describes your data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To address if 500 cells is enough for you pilot we need to know what question you want to address.&#xA;The following tool for example gives some intuition on how many cells to sample if you wish to detect n-cells per cluster. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://satijalab.org/howmanycells&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/howmanycells&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopes this helps&lt;/p&gt;&#xA;" OwnerUserId="2675" LastActivityDate="2019-07-27T17:39:27.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9086" PostTypeId="2" ParentId="9084" CreationDate="2019-07-28T11:42:19.353" Score="1" Body="&lt;p&gt;Hmmm I'm not familiar with AddtoTernary function, I can see that you've placed a function within the &quot;AddtoTernary&quot; function. The traditional ggplot2 way of doing this would simply be controlled by the theme &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggtern(data=x,aes(x1,x2,x3)) + &#xA;  geom_point(fill=&quot;red&quot;,shape=21,size=3) + &#xA;  theme_tern_bw()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will give an arrow for each axis, but additional code is required for different colours (I'd have to look up my past work). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think what you might want is a &lt;strong&gt;colour key&lt;/strong&gt;? In this case ggtern will have a &quot;key&quot; function because ggplot2 has one.Here, your colouring is based on species not on a colour gradient as to how far to the corner of a graph you are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively you can easily put names besides each point in ggplot2, but again this is traditional style code. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also you can add different shapes to the graph, which might be useful instead of colours.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Arrows and names together peronsally I've not seen on a ggplot2 graphic and I'd have not idea how to do it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;FYI I don't think we need all the data just a few points fine, although it shows why you prefer arrows - personally I would use colour coding. In &quot;traditional&quot; ggplot2 coding the graphing is built up in layers, so if additional functionality is required it is simply appended to the exisiting command. Thus it is quite easy to make modifications. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-07-28T12:20:23.443" LastActivityDate="2019-07-28T12:20:23.443" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9087" PostTypeId="1" AcceptedAnswerId="9089" CreationDate="2019-07-29T11:29:57.103" Score="1" ViewCount="803" Body="&lt;p&gt;In bioinformatics, a &lt;a href=&quot;https://en.wikipedia.org/wiki/Sequence_logo&quot; rel=&quot;nofollow noreferrer&quot;&gt;sequence logo&lt;/a&gt; is a graphical representation of the sequence conservation of nucleotides (in a strand of DNA/RNA) or amino acids (in protein sequences).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so, What is &quot;aligned sequences&quot; and &quot;consensus sequence&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;how can I get these sequences based on following data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/umTOV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/umTOV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Figure 2.5 from the book &quot;&lt;a href=&quot;http://people.cs.ubc.ca/~murphyk/MLbook/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning - A Probabilistic Perspective&lt;/a&gt;&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;more info about this figure at &lt;a href=&quot;https://stats.stackexchange.com/q/419575/247356&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4962" LastEditorUserId="104" LastEditDate="2019-08-01T09:49:44.143" LastActivityDate="2019-08-01T09:49:44.143" Title="What is &quot;aligned sequences&quot; and &quot;consensus sequence&quot; in the context of sequence logo? How to compute these?" Tags="|sequence-alignment|ngs|sequence-analysis|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9089" PostTypeId="2" ParentId="9087" CreationDate="2019-07-29T15:02:32.367" Score="5" Body="&lt;p&gt;Panel (a) shows a sequence gap-free alignment. Each row corresponds to a contiguous 15 base pair sequence of DNA (e.g. row 1 could be a human sequence, row 2 could be the equivalent mouse sequence etc.). Each column corresponds to a particular position/residue (e.g. column 1 is the 1st position of the alignment etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Panel (b) shows the sequence logo. As noted in the comments, this does not correspond to the alignment in (a). Instead, here is the corresponding logo:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3jR8j.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3jR8j.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From this you can intuitively understand that at the 1st sequence position you have only As and Ts and their frequency corresponds to the y-axis. For the 3rd position, you have only As, so only A is represented here in the logo etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The consensus sequence is the most frequent letter at each position in the alignment, i.e. &lt;code&gt;[AT]TAGC&lt;/code&gt; etc. where &lt;code&gt;[AT]&lt;/code&gt; means A or T since they are jointly high in frequency (5 each).&lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="104" LastEditDate="2019-07-29T15:31:12.983" LastActivityDate="2019-07-29T15:31:12.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9090" PostTypeId="1" CreationDate="2019-07-29T15:08:51.343" Score="2" ViewCount="400" Body="&lt;p&gt;I'm trying to get CSV files from snpEff -csvStats option and got stuck. I tried the code below and only got the vcf file with no contents. How can I use -csvStats option properly?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;snpEff -Xmx4g -csvStats -v hg19 08_FILTER/NA12878.Filtered.Variants.vcf &amp;gt; test.ann.vcf&#xA;&#xA;    00:00:00  SnpEff version SnpEff 4.3t (build 2017-11-24 10:18), by Pablo Cingolani&#xA;    00:00:00  Command: 'ann'&#xA;    00:00:00  Reading configuration file 'snpEff.config'. Genome: '08_FILTER/NA12878.Filtered.Variants.vcf'&#xA;    00:00:00  Reading config file: /home/ubuntu/01_NA12878/snpEff.config&#xA;    00:00:00  Reading config file: /home/ubuntu/miniconda2/share/snpeff-4.3.1t-2/snpEff.config&#xA;    java.lang.RuntimeException: Property: '08_FILTER/NA12878.Filtered.Variants.vcf.genome' not found&#xA;      at org.snpeff.interval.Genome.&amp;lt;init&amp;gt;(Genome.java:106)&#xA;      at org.snpeff.snpEffect.Config.readGenomeConfig(Config.java:681)&#xA;      at org.snpeff.snpEffect.Config.readConfig(Config.java:649)&#xA;      at org.snpeff.snpEffect.Config.init(Config.java:480)&#xA;      at org.snpeff.snpEffect.Config.&amp;lt;init&amp;gt;(Config.java:117)&#xA;      at org.snpeff.SnpEff.loadConfig(SnpEff.java:451)&#xA;      at org.snpeff.snpEffect.commandLine.SnpEffCmdEff.run(SnpEffCmdEff.java:1000)&#xA;      at org.snpeff.snpEffect.commandLine.SnpEffCmdEff.run(SnpEffCmdEff.java:984)&#xA;      at org.snpeff.SnpEff.run(SnpEff.java:1183)&#xA;      at org.snpeff.SnpEff.main(SnpEff.java:162)&#xA;    00:00:00  Logging&#xA;&#xA;00:00:01  Done.&#xA;&#xA; snpEff -Xmx4g -v hg19 -csvStats 08_FILTER/NA12878.Filtered.Variants.vcf &amp;gt; tes2.ann.vcf&#xA;&#xA;00:00:25  Genome stats :&#xA;#-----------------------------------------------&#xA;# Genome name                : 'Homo_sapiens (USCS)'&#xA;# Genome version             : 'hg19'&#xA;# Genome ID                  : 'hg19[0]'&#xA;#     'M' 16571   Vertebrate_Mitochondrial&#xA;#     'Un_gl000226'   15008   Standard&#xA;#     '18_gl000207_random'    4262    Standard&#xA;#     'MT'    1   Vertebrate_Mitochondrial&#xA;#-----------------------------------------------&#xA;&#xA;00:00:27  Predicting variants&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the snpEff.config file. I installed the snpEff at /miniconda2/share&#xA;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Databases are stored here&#xA;# E.g.: Information for 'hg19' is stored in data.dir/hg19/&#xA;#&#xA;# You can use tilde ('~') as first character to refer to your home directory. &#xA;# Also, a non-absolute path will be relative to config's file dir&#xA;# &#xA;#---&#xA;data.dir = ./data/&#xA;&#xA;#---&#xA;# Database repository: A URL to the server where you can download databases (command: 'snpEff download dbName')&#xA;#---&#xA;database.repository = http://downloads.sourceforge.net/project/snpeff/databases&#xA;&#xA;#---&#xA;# Latest version numbers. Check here if there is an update.&#xA;#---&#xA;versions.url = http://snpeff.sourceforge.net/versions.txt&#xA;&#xA;#-------------------------------------------------------------------------------&#xA;# Third party databases&#xA;#-------------------------------------------------------------------------------&#xA;&#xA;# Databases for human GRCh38 (hg38)&#xA;&#xA;database.clinvar.GRCh38                 = ./db/GRCh38/clinvar/clinvar-latest.vcf.gz&#xA;database.repository.clinvar             = ftp://ftp.ncbi.nlm.nih.gov/pub/clinvar/vcf_GRCh38/clinvar-latest.vcf.gz&#xA;&#xA;database.dbsnp.GRCh38                   = ./db/GRCh38/dbSnp/dbSnp.vcf.gz&#xA;database.repository.dbsnp.GRCH38        = ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606/VCF/00-All.vcf.gz&#xA;&#xA;database.dbnsfp.GRCH38                  = ./db/GRCh38/dbNSFP/dbNSFP.txt.gz&#xA;&#xA;# Databases for human GRCh37 (hg19)&#xA;database.clinvar.GRCh37                 = ./db/GRCh37/clinvar/clinvar-latest.vcf.gz&#xA;database.repository.clinvar.GRCh37      = ftp://ftp.ncbi.nlm.nih.gov/pub/clinvar/vcf_GRCh37/clinvar-latest.vcf.gz&#xA;&#xA;database.dbsnp.GRCh37                   = ./db/GRCh37/dbSnp/&#xA;database.repository.dbsnp.GRCh37        = ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606_b141_GRCh37p13/VCF/&#xA;&#xA;database.gwascatalog.GRCh37             = ./db/GRCh37/gwasCatalog/gwascatalog.txt&#xA;database.repository.gwascatalog.GRCh37  = http://www.genome.gov/admin/gwascatalog.txt&#xA;&#xA;database.dbnsfp.GRCh37                  = ./db/GRCh37/dbNSFP/dbNSFP.txt.gz&#xA;&#xA;#-------------------------------------------------------------------------------&#xA;# Loss of function (LOF) &#xA;#-------------------------------------------------------------------------------&#xA;&#xA;# It is assumed that even with a protein coding change at the &#xA;# last 5% of the protein, the protein could still be functional.&#xA;lof.ignoreProteinCodingAfter  : 0.95&#xA;&#xA;#  It is assumed that even with a protein coding change at the &#xA;#  first 5% of the protein: &#xA;#       &quot;..suggesting some disrupted transcripts are &#xA;#       rescued by transcriptional reinitiation at an &#xA;#       alternative start codon.&quot;&#xA;lof.ignoreProteinCodingBefore : 0.05&#xA;&#xA;# Larger deletions removing either the first exon or more than &#xA;# 50% of the protein-coding sequence of the affected transcript&#xA;lof.deleteProteinCodingBases : 0.50&#xA;&#xA;#-------------------------------------------------------------------------------&#xA;# Codon tables&#xA;#&#xA;# Format:   It's a comma separated &quot;codon/aminoAcid[+*]&quot; list &#xA;#           Where 'codon' is in uppper case, aminoAcid is a one letter &#xA;#           code, '+' denotes start codon and '*' denotes stop codon.&#xA;#&#xA;# References:   http://www.ncbi.nlm.nih.gov/Taxonomy/Utils/wprintgc.cgi&#xA;#               ftp://ftp.ncbi.nih.gov/entrez/misc/data/gc.prt&#xA;#-------------------------------------------------------------------------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4964" LastEditorUserId="4964" LastEditDate="2019-07-30T14:24:35.767" LastActivityDate="2023-12-09T04:11:21.393" Title="How to use -csvStats option?" Tags="|variation|gatk|snpeff|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9091" PostTypeId="1" AcceptedAnswerId="9094" CreationDate="2019-07-29T15:26:17.283" Score="2" ViewCount="463" Body="&lt;p&gt;Following &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/9087/what-is-aligned-sequences-and-consensus-sequence-in-the-context-of-sequence&quot;&gt;this question&lt;/a&gt;, I'm confused with the &lt;a href=&quot;https://en.wikipedia.org/wiki/Sequence_logo&quot; rel=&quot;nofollow noreferrer&quot;&gt;computation of sequence logo&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following data comes from the book &quot;&lt;a href=&quot;http://people.cs.ubc.ca/~murphyk/MLbook/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning - A Probabilistic Perspective&lt;/a&gt; (Figure_1)&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZoStG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZoStG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here is the corresponding &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/9089/4962&quot;&gt;sequence logo&lt;/a&gt; (Figure_2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3jR8j.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3jR8j.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ten rows represent sequences of DNA (e.g. row 1 could be a human sequence, row 2 could be the equivalent mouse sequence etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each column corresponds to a particular position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why do ten rows (Figure_1) correspond to 2 bits (Figure_2)?&lt;/p&gt;&#xA;" OwnerUserId="4962" LastEditorUserId="104" LastEditDate="2019-08-01T09:48:22.800" LastActivityDate="2019-08-01T10:01:38.627" Title="Why do ten rows (Figure_1) correspond to 2 bits (Figure_2) in a sequence logo?" Tags="|sequence-alignment|ngs|sequence-analysis|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9093" PostTypeId="1" CreationDate="2019-07-29T18:28:46.590" Score="1" ViewCount="1098" Body="&lt;p&gt;I have a list of proteins with their gene names/symbols. I would like to identify which of them are extracellular proteins or plasma membrane proteins. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can get this information in UniprotKB by searching the proteins one by one. Because I have hundreds of proteins, I wonder whether I can do a systematic search of the UniprotKB in RStudio. I tried the UniProt.ws select method. But I couldn't even get the example code in the package work. I downloaded the uniprot_sprot.dat file. But I don't know how to read this file correctly in RStudio. I used tab-delimited format. It turned out there is only one variable in the file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody tell me what I should do to solve the problem? I am pretty new to any computer languages. &lt;/p&gt;&#xA;" OwnerUserId="4967" LastEditorUserId="48" LastEditDate="2019-07-30T07:13:33.247" LastActivityDate="2019-12-27T21:02:25.313" Title="How to retrieve the subcellular location info from uniprot?" Tags="|r|sequence-annotation|uniprot|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9094" PostTypeId="2" ParentId="9091" CreationDate="2019-07-29T18:50:44.053" Score="6" Body="&lt;p&gt;Bits are not frequencies. If a position only contains an A (position 3 for example) then you would need 2 questions (bits) to derive that information without a priori knowledge.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is it a G or C? if no &gt; then it is a A or T&lt;/li&gt;&#xA;&lt;li&gt;Is it T? If no then it is an A&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Position 1 can be derived by 1 question only:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;is it a G or C?  If no then it is an A or T&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In this case the number of bits can be calculated using the frequenci)es. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;For 4 bases, A only (100%)&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4 * -0.25 * log2(0.25) - (-1 * log2(1)) = 2 (bits)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;For 8 letters, A only (100%)&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;8 * -1/8 * log(1/8)- (-1 * log2(1)) = 3 (bits)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Mixed base possibility (A/T 50% each) with 4 bases&lt;/strong&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4 * -0.25 * log2(0.25) - (-0.5 * log2(0.5)+-0.5 * log2(0.5)) = 1 (bit)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned in the comments, you can follow the wiki link for an example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or, an example with a little more explanation from the biology SE site.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://biology.stackexchange.com/questions/64368/how-to-determine-the-height-bits-in-a-sequence-logo&quot;&gt;sequence logo calcuation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopes this helps&lt;/p&gt;&#xA;" OwnerUserId="2675" LastEditorUserId="298" LastEditDate="2019-08-01T10:01:38.627" LastActivityDate="2019-08-01T10:01:38.627" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9095" PostTypeId="2" ParentId="5428" CreationDate="2019-07-29T23:19:55.283" Score="3" Body="&lt;p&gt;I received the following message from the author of another Drop-seq paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;After much conversation with the folks at the SRA, it appears the SRA&#xA;  compression algorithm automatically deletes certain subsets of bam&#xA;  tags and does so in cryptic way (in the sense of not alerting the&#xA;  uploader). Obviously this is going to be an issue for many single cell&#xA;  datasets encoded this way, and I hope the SRA will eventually change&#xA;  their system to accommodate.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you look at the &quot;Download&quot; tab of the &lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR6261587&quot; rel=&quot;nofollow noreferrer&quot;&gt;same page&lt;/a&gt;, there now exists a link to download the originally uploaded BAM file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternately, you can use the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra/docs/sra-aws-download/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SRA AWS tutorial&lt;/a&gt; to download the files on that platform. &lt;/p&gt;&#xA;" OwnerUserId="4968" LastActivityDate="2019-07-29T23:19:55.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9096" PostTypeId="5" CreationDate="2019-07-29T23:45:50.413" Score="0" Body="&lt;p&gt;Used for questions pertaining to the &lt;a href=&quot;http://sbml.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Systems Biology Markup Language (SBML)&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;h1&gt;SBML&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;SBML (the Systems Biology Markup Language) is an &lt;a href=&quot;https://en.wikipedia.org/wiki/XML&quot; rel=&quot;nofollow noreferrer&quot;&gt;XML&lt;/a&gt;-based format for communicating and storing representations of biological processes. Put more simply: &lt;em&gt;SBML is a machine-readable format for representing models in biology&lt;/em&gt;. It is suitable for models commonly found in research on a number of topics, including cell signaling pathways, metabolic pathways, biochemical reactions, gene regulation, and many others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SBML is not designed to be a universal language for representing quantitative models. It would be impossible to achieve a one-size-fits-all universal language. A more realistic alternative is to acknowledge the diversity of approaches and methods being explored in systems biology, and seek a common intermediate format—a lingua franca—enabling communication of the most essential aspects of the models. SBML is such a lingua franca.  Today, SBML is used by many &lt;a href=&quot;http://sbml.org/SBML_Software_Guide&quot; rel=&quot;nofollow noreferrer&quot;&gt;different software tools.&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Summary of the structure and features of SBML&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The core of SBML is focused on encoding models in which entities are located in containers and are acted upon by processes that modify, create or destroy entities. The containers do not need to correspond to physical structures; they can be conceptual or abstract. Additional constructs allow parameters, initial conditions, other variables, and other mathematical relationships to be defined. In the most common type of model, the &quot;entities&quot; are &lt;a href=&quot;https://en.wikipedia.org/wiki/Biomolecule&quot; rel=&quot;nofollow noreferrer&quot;&gt;biochemical substances&lt;/a&gt;, the &quot;containers&quot; are well-mixed and spatially homogeneous, and the &quot;processes&quot; are &lt;a href=&quot;https://en.wikipedia.org/wiki/Chemical_reaction#Biochemical_reactions&quot; rel=&quot;nofollow noreferrer&quot;&gt;biochemical reactions&lt;/a&gt; happening within or between the containers. This originally led to the SBML constructs being named (literally) &lt;em&gt;species&lt;/em&gt;, &lt;em&gt;compartments&lt;/em&gt;, and &lt;em&gt;reactions&lt;/em&gt;, respectively, but these names are historical artifacts and belie the generality of the underlying scheme. Software applications can map the names to other concepts to better suit their purposes. For instance, &lt;em&gt;species&lt;/em&gt; could be mapped to populations of molecules, cells, or even organisms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Values of model variables and their changes over time may be fixed or determined by mathematical expressions, either before or during simulation, continuously or in response to discrete events, with or without time delays.  Units of measurement can be specified for all entities and values; in addition to adding a layer of essential physical knowledge, information about units can be used to verify the relationships expressed in a model.  Units also facilitate reuse of models and components, interconnection of models, conversion of models between different frameworks, and integration of data with models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any element of an SBML model can be elaborated using machine-readable metadata (known as &lt;em&gt;annotations&lt;/em&gt; in SBML) as well as human-readable notes.  These can be used to express relationships between the entities in a given model and entities in external resources such as databases.  Software tools can also use annotations to encode tool-specific data in their own formats, thus providing a way to capture data that might otherwise be lost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SBML Level 3 introduced an extensible modular architecture consisting of a central set of fixed features (known as &lt;em&gt;SBML Level&amp;nbsp;3 Core&lt;/em&gt;) , and a scheme for adding &lt;em&gt;packages&lt;/em&gt; that can augment the &lt;em&gt;Core&lt;/em&gt; by extending existing elements, adding new elements, and adjusting the meaning or scope of elements.  SBML Packages offer support for such things as hierarchically-structured models (e.g., models containing submodels), qualitative models, flux balance constraints models, and more.  A model declares which packages it uses in order to guide its interpretation by software applications.  If a software tool detects the presence of packages that it does not support, it may inform users if it cannot work with the model.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Using SBML&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;A software tool can read an SBML model description and translate it into its own internal format for model analysis. For example, a tool might provide the ability to simulate the model by constructing a system of &lt;a href=&quot;https://en.wikipedia.org/wiki/Nonlinear_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;differential equations&lt;/a&gt; and then performing numerical time integration on the equations to explore the model's dynamic behavior. Or, alternatively, a tool might construct a discrete &lt;a href=&quot;https://en.wikipedia.org/wiki/Stochastic&quot; rel=&quot;nofollow noreferrer&quot;&gt;stochastic&lt;/a&gt; representation of the model and use a &lt;a href=&quot;https://en.wikipedia.org/wiki/Dynamic_Monte_Carlo_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monte Carlo simulation method&lt;/a&gt; such as the &lt;a href=&quot;https://en.wikipedia.org/wiki/Gillespie_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gillespie algorithm&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Resources&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;More information about SBML and its specifications can be found on &lt;a href=&quot;http://sbml.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;SBML.org&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SBML is an official &lt;a href=&quot;https://en.wikipedia.org/wiki/Internet_Engineering_Task_Force&quot; rel=&quot;nofollow noreferrer&quot;&gt;IETF&lt;/a&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/MIME&quot; rel=&quot;nofollow noreferrer&quot;&gt;MIME&lt;/a&gt; type, specified by &lt;a href=&quot;http://www.rfc-editor.org/rfc/rfc3823.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;RFC 3823&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4959" LastEditorUserId="4913" LastEditDate="2020-02-14T08:00:33.170" LastActivityDate="2020-02-14T08:00:33.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9097" PostTypeId="4" CreationDate="2019-07-29T23:45:50.413" Score="0" Body="Used for questions pertaining to the Systems Biology Markup Language (SBML)." OwnerUserId="4959" LastEditorUserId="4913" LastEditDate="2020-02-11T06:34:59.837" LastActivityDate="2020-02-11T06:34:59.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9098" PostTypeId="1" AcceptedAnswerId="9101" CreationDate="2019-07-30T03:59:22.820" Score="2" ViewCount="1611" Body="&lt;p&gt;I'm confused by the &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/9091/4962&quot;&gt;computation&lt;/a&gt; of sequence logo. &lt;a href=&quot;https://en.wikipedia.org/wiki/Sequence_logo#Logo_creation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt; gives a process about this without a concrete example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's just consider DNA, so there are 4 bases (nucleic acids).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following data comes from the book &quot;Machine Learning - A Probabilistic Perspective (Figure 1)&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XP7q1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XP7q1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The Shannon entropy of position &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;${\displaystyle H_{i}=-\sum _{b=a}^{t}f_{b,i}\times \log _{2}f_{b,i}}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where &lt;span class=&quot;math-container&quot;&gt;${\displaystyle f_{b,i}}$&lt;/span&gt; is the relative frequency of base&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/9094/4962&quot;&gt;post&lt;/a&gt; is computing position 3, where it seems that the relative frequency of &lt;code&gt;a&lt;/code&gt; is 100%, and the relative frequency of 3 other bases are 0%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, how do you calculate the Shannon entropy of position 3? It seems to be 0, which is apparently incorrect, where am I wrong?&lt;/p&gt;&#xA;" OwnerUserId="4962" LastEditorUserId="4645" LastEditDate="2019-08-01T18:49:43.257" LastActivityDate="2019-08-01T18:49:43.257" Title="How to compute the Shannon entropy for a strand of DNA?" Tags="|sequence-alignment|ngs|sequence-analysis|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9099" PostTypeId="1" CreationDate="2019-07-30T08:34:59.883" Score="1" ViewCount="209" Body="&lt;p&gt;I have a number of samples (from the 1000 genomes) that I want to analyse using Nextflow. However, I need to keep track of their metadata, fields like sample ID, analysis type etc. So I have a basic metadata object like this for each sample that I then pipe into a channel:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;            metadata = [&#xA;                sample: split[0],&#xA;                mapped: split[1],&#xA;                platform : split[2],&#xA;                alignment: split[3],&#xA;                tsi: split[4],&#xA;                type: split[5],&#xA;                date: split[6],&#xA;                filetype: split[7..-1],&#xA;                url: file(bam)&#xA;            ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, in the first process, I attempt to use this metadata object:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;process bamToFastq {&#xA;    input:&#xA;        val bam&#xA;&#xA;    container 'quay.io/biocontainers/samtools:1.8--4'&#xA;    cpus 4&#xA;    disk '50 GB'&#xA;&#xA;    output:&#xA;        file('r*.fastq') into fastq&#xA;&#xA;    &quot;&quot;&quot;&#xA;    samtools fastq -0 r0.fastq -1 r1.fastq -2 r2.fastq -@ &lt;span class=&quot;math-container&quot;&gt;${task.cpus} $&lt;/span&gt;bam.url&#xA;    &quot;&quot;&quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I already run into some issues:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Nextflow doesn't realise that &lt;code&gt;metadata.url&lt;/code&gt; is a file, since I declared the whole object using &lt;code&gt;val&lt;/code&gt;, so I'm guessing it won't be correctly localized (based on &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#input-of-files&quot; rel=&quot;nofollow noreferrer&quot;&gt;the documentation&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;li&gt;I want pass down a new metadata object for each of my generated FASTQs, that's the same as the input one, but I want to add in the FASTQ as a URL, and the read number of each FASTQ. Where/how can I do this? Nextflow doesn't provide any section of the &lt;code&gt;process&lt;/code&gt; for structuring the output data. If this were Python I would do the following:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;    metadata = bam.copy()&#xA;    metadata['fastq'] = file&#xA;    metadata['read_number'] = file.name[1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a good way to pass around structured metadata like this in nextflow? Will this solve either of my problems?&lt;/p&gt;&#xA;" OwnerUserId="4645" LastActivityDate="2019-07-30T08:34:59.883" Title="Passing around complex metadata in Nextflow" Tags="|nextflow|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9101" PostTypeId="2" ParentId="9098" CreationDate="2019-07-30T13:10:52.197" Score="5" Body="&lt;p&gt;Why do you think the entropy of 0 is incorrect? It intuitively makes sense, as there is no uncertainty about the base at position 3, and thus there is no entropy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, what is plotted in a sequence logo isn't the entropy, but rather a measure of the &quot;decrease in uncertainty&quot; as the sequence is aligned. This is calculated by taking the entropy at this position if we randomly aligned sequences (&lt;span class=&quot;math-container&quot;&gt;$H_g(L)$&lt;/span&gt;), and subtracting from it the entropy of the alignment (&lt;span class=&quot;math-container&quot;&gt;$H_g(s)$&lt;/span&gt;): &#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;R_{sequence}=H_g(L) - H_s(L)&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The entropy at position 3 based on a random alignment is calculated by assuming there are 4 equally likely events (one for each base), and thus:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{align*} &#xA;H_g(L) &amp;amp; = -((1/4 \times -2) + (1/4 \times -2) + (1/4 \times -2) + (1/4 \times -2)) \\&#xA;H_g(L) &amp;amp; = 2&#xA;\end{align*}&#xA;$$&lt;/span&gt;&#xA;Notably, &lt;span class=&quot;math-container&quot;&gt;$H_g(L)$&lt;/span&gt; will always be 2 when dealing with nucleotide sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in your example, we have:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{align*} &#xA;R_{sequence}&amp;amp;=H_g(L) - H_s(L) \\&#xA;R_{sequence}&amp;amp;=2 - 0 \\&#xA;R_{sequence}&amp;amp;=2 \text{ bits of entropy}&#xA;\end{align*}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, to work out the height of each base at each position in the logo, we multiply the frequency of that base by the overall information gain &lt;span class=&quot;math-container&quot;&gt;$R_{sequence}$&lt;/span&gt;. In this case the frequency of the base A at position 3 is of course 1:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin {align*}&#xA;H(b,l) &amp;amp;=f(b, l) \times R_{sequence} \\&#xA;H(A, 3) &amp;amp;= f(a, 3) \times 2 \\&#xA;&amp;amp;= 1 \times 2 \\&#xA;&amp;amp;= 2&#xA;\end{align*}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC332411/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sequence logos: a new way to display consensus sequences.&#xA; (Thomas D. Schneider, R. Michael Stephens, 1990)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I've excluded the correction factor &lt;span class=&quot;math-container&quot;&gt;$e(n)$&lt;/span&gt; from the overall information-gain calculation for the sake of simplicity. Refer to the paper above for an explanation of this.&lt;/p&gt;&#xA;" OwnerUserId="4645" LastEditorUserId="4645" LastEditDate="2019-07-31T15:12:19.477" LastActivityDate="2019-07-31T15:12:19.477" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="9102" PostTypeId="1" AcceptedAnswerId="9103" CreationDate="2019-07-30T13:13:04.547" Score="1" ViewCount="395" Body="&lt;p&gt;I need some help with GATK. I have a multisample vcf file containing 18125 samples. After performing PCA I have around 303 samples as outliers and I wanted to remove these samples from my multisample vcf file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried doing it using the GATK &lt;code&gt;SelectVariant&lt;/code&gt; tool. It does some stuff and generates a VCF file which is a bit smaller in size than the original file, but in the end the total count of samples remains the same. As I have no prior knowledge of GATK (working with it for the first time) I am just wondering if you guys can help me out in fixing this issue. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The command that I used was &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gatk SelectVariants -V filtered.vcf -xl-sn remove.txt -O test.vcf &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;remove.txt&lt;/code&gt; contains the sample names in each row (1 sample per row).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After running this command it generated a &lt;code&gt;test.vcf&lt;/code&gt; file, for which I counted samples using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools query -l test.vcf | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I got exactly same number of samples as in the original VCF file &lt;code&gt;filtered.vcf&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4970" LastEditorUserId="96" LastEditDate="2019-07-30T13:47:09.537" LastActivityDate="2019-07-30T13:47:09.537" Title="Subset a multisample VCF file" Tags="|vcf|quality-control|gatk|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9103" PostTypeId="2" ParentId="9102" CreationDate="2019-07-30T13:24:35.680" Score="3" Body="&lt;p&gt;Based on the way you've called the command (using &lt;code&gt;-xl-sn&lt;/code&gt;), I assume you are using GATK 4 and not GATK 3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is the case, the documentation for the &lt;code&gt;-xl-sn&lt;/code&gt; flag is available here: &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/tooldocs/current/org_broadinstitute_hellbender_tools_walkers_variantutils_SelectVariants.php#--exclude-sample-name&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://software.broadinstitute.org/gatk/documentation/tooldocs/current/org_broadinstitute_hellbender_tools_walkers_variantutils_SelectVariants.php#--exclude-sample-name&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By default, the &lt;code&gt;-xl-sn SAMPLE&lt;/code&gt; means &quot;exclude the sample called SAMPLE from my VCF&quot;. In your case, you're providing an entire list of samples, in a file, which is not the default behaviour. In order to tell GATK that you're providing a file and not a sample name, &lt;strong&gt;you need to ensure the file you provide has the extension &lt;code&gt;.args&lt;/code&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the documentation:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;File names must use the extension &quot;.args&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4645" LastActivityDate="2019-07-30T13:24:35.680" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9104" PostTypeId="1" CreationDate="2019-07-30T15:10:59.887" Score="3" ViewCount="1831" Body="&lt;h2&gt;The problem&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If I have a sample sheet that contains both single-indexed and dual-indexed samples, I can split it up into two sample sheets and then run bcl2fastq on each one. However, when doing this, large Undetermined fastq files are generated. E.g., when processing the single-indexed samples, all the dual-indexed samples go to Undetermined. And when processing the dual-indexed samples, the single-indexed samples go to Undetermined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, because they are being processed separately, if any single index A is part of a dual index A+B, then when processing the single indexes, an A+B may be mistaken for an A, so it would seem that they need to processed simultaneously to avoid this mis-assignment.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;The question&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Given a sample sheet and directories of BCL files, how can such a set of sequencing data be demultiplexed correctly, either using bcl2fastq or the Picard tools?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To put it another way, I want to demultiplex a single sequencing run that contains both single-indexed and dual-indexed samples. It can be assumed that the indexes are sufficiently distinct such that any sample's index configuration is different from any other. But assuming that the different index configurations are not segregated to particular lanes of the sequencer, the question is how to demultiplex the files correctly such that both the single-indexed and dual-indexed samples are recognized.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Attempts at a solution&lt;/h2&gt;&#xA;&#xA;&lt;h3&gt;Using bcl2fastq directly&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;If a sample sheet of a mix of indexes is given to bcl2fastq (v2.17) directly, it produces the error&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ERROR: bcl2fastq::common::Exception: Success (0): .../bcl2fastq2/src/cxx/lib/layout/BarcodeCollisionDetector.cpp(127): &#xA;Throw in function void bcl2fastq::layout::BarcodeCollisionDetector::validateNewBarcodeSizesAgainstExisting(const std::vector&amp;lt;long unsigned int&amp;gt;&amp;amp;) const&#xA;Dynamic exception type: boost::exception_detail::clone_impl&amp;lt;bcl2fastq::layout::BarcodeCollisionError&amp;gt;&#xA;std::exception::what: Barcodes have an unequal number of components.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Barcodes have an unequal number of components.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h3&gt;Using picard&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;It seems that it should be possible using the Picard tools, but I have not found a way to set up the inputs to ExtractIlluminaBarcodes and IlluminaBasecallsToFastq that processes this configuration correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The syntax for supporting multiple index configurations is not entirely clear. But when using various combinations of N and '*' on multiplex_params.tsv and barcodes.txt required by the Picard tools, a large Undetermined file is still produced and actual sample fastq files are tiny, indicating it is not processing them correctly.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Using multiple calls to bcl2fastq&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;As indicated in the discussion above, this suffers from the problem of having all the dual indexed samples going to the &quot;Undetermined&quot; file when processing the single-indexed samples, or vice versa, and creates two output directories of reports and stats, which must be merged.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Padding empty indexes with N&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;By padding, I mean converting ACGTACGT to ACGTACGT+NNNNNNNN so that the single-index samples in the sample sheet are &quot;dual&quot; as well. This strategy seems that it would work except there is a bug in bcl2fastq that it treats &quot;N&quot; literally instead of as a wildcard. &lt;a href=&quot;https://support.illumina.com/content/dam/illumina-support/documents/downloads/software/bcl2fastq/bcl2fastq2-2-20-0-release-notes-1000000039216-00.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;See the release notes for details.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4679" LastEditorUserId="4679" LastEditDate="2019-07-30T18:22:01.517" LastActivityDate="2021-05-26T19:58:00.250" Title="How to demultiplex a mix of single-indexed and dual-indexed samples" Tags="|illumina|picard|demultiplexing|indexing|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9105" PostTypeId="2" ParentId="9093" CreationDate="2019-07-30T15:42:38.377" Score="2" Body="&lt;p&gt;Since you are already on UniProtKB, the simplest way will be if you add this information to the output table and download it as a that you can import and filter in R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get additional annotations, click the button labelled &quot;Columns&quot; on top of the UniPortKB website. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/gQdsF.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gQdsF.jpg&quot; alt=&quot;UniProt button row&quot;&gt;&lt;/a&gt;&lt;br&gt;&#xA;This allows you to add a wide variety of annotations such as subcellular location to the output. Tick the checkmark for subcellular location and save on the top right.&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/HDAP1.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HDAP1.jpg&quot; alt=&quot;UniProtKB column selection&quot;&gt;&lt;/a&gt;&lt;br&gt;&#xA;Then you can download the proteome-wide annotation for example as a tab-separated list using the &quot;Download&quot; dialog next to the &quot;Columns&quot; selection. This list can be imported into R.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2019-07-30T15:42:38.377" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9106" PostTypeId="2" ParentId="9104" CreationDate="2019-07-30T16:15:38.150" Score="0" Body="&lt;p&gt;Have you just tried giving bcl2fastq one sample sheet with a mix of single and dual indices?  I don't think what you are trying to do is a problem.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;if any single index A is part of a dual index A+B,&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Well, it might be a problem if you did that.  That was poor planning.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-07-30T16:15:38.150" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="9107" PostTypeId="2" ParentId="9093" CreationDate="2019-07-30T20:31:44.323" Score="0" Body="&lt;p&gt;After my post of the question, I went to try the UniProt.ws package in the RStudio again. I realized that there is a mistake in the example code in the package. The up object is from a human database, but the keys the example provides are the ENTRZID from mouse proteins. When I tried human ENTRZID numbers, it worked. The only problem for the select method in the UniProt.ws is that it can't retrieve the protein information for a large batch of proteins. The maximum number that worked for me is about 80 proteins per time.&lt;/p&gt;&#xA;" OwnerUserId="4967" LastActivityDate="2019-07-30T20:31:44.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9108" PostTypeId="1" AcceptedAnswerId="9109" CreationDate="2019-07-30T22:56:10.960" Score="1" ViewCount="101" Body="&lt;p&gt;I am confused with &lt;a href=&quot;https://en.wikipedia.org/wiki/Sequence_alignment&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sequence alignment&lt;/a&gt;, which is&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;a way of arranging the sequences of DNA, RNA, or protein to&#xA;  identify regions of similarity that may be a consequence of&#xA;  functional, structural, or evolutionary relationships between the&#xA;  sequences.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;to understand this easily, I would like to use this &lt;a href=&quot;https://en.wikipedia.org/wiki/Nucleic_acid_sequence&quot; rel=&quot;nofollow noreferrer&quot;&gt;image&lt;/a&gt; (image_1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/f1Q8C.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/f1Q8C.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the &lt;a href=&quot;https://en.wikipedia.org/wiki/Nucleic_acid_sequence&quot; rel=&quot;nofollow noreferrer&quot;&gt;image&lt;/a&gt; above (image_1) is the primary part of nucleic acid structure (primary, secondary, tertiary, and quaternary), shown below (image_2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/w3oiI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/w3oiI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is the sequence gcgc... an aligned sequence? if yes, what does the sequence look like before alignment?&lt;/p&gt;&#xA;" OwnerUserId="4962" LastActivityDate="2019-07-31T00:03:48.923" Title="what does a sequence look like before alignment?" Tags="|sequence-alignment|ngs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9109" PostTypeId="2" ParentId="9108" CreationDate="2019-07-31T00:03:48.923" Score="4" Body="&lt;p&gt;Sequence alignment typically refers to the primary sequence. That is, we ignore the complex 3D structure of the molecule and its spatial arrangement in the cell. Instead, we use a symbolic representation composed of symbols like A, C, G, T, and U. Alignment is a conceptual tool that, as you pointed out, is used to find regions of similarity between sequences. Therefore, it only makes sense in the context of two or more similar sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By way of example, consider a simple case of two sequences X and Y. This is what they would look like before alignment. The similarities between the two sequences are unclear.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X: ATTGACATAGGCACTTCGACTA&#xA;Y: GACATCTTCGGCTAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is what they would look like after alignment. The similarities (and differences) are now much easier to see.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;X: ATTGACATAGGCACTTCCACTA-&#xA;      |||||     ||||  |||&#xA;Y: ---GACAT-----CTTCGGCTAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This arrangement identifies regions of similarity that &lt;em&gt;may&lt;/em&gt; be related through evolution. This doesn't always mean perfect agreement though. As this example shows, sometimes mismatches or gaps are allowed so that long stretches of identical matches are lined up correctly. Sometimes, for very closely related sequences, the optimal alignment doesn't require any gaps at all. This seems to be the case with some of your recent questions about sequence logos.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: Sequence alignment is a fundamental concept in bioinformatics, and there is a tremendous amount of research and development on the topic. It's hard to imagine doing any amount of serious DNA sequence analysis responsibly without a basic understanding of sequence alignment. I would recommend searching Google and/or Google Scholar for intro-level reviews of the topic.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-07-31T00:03:48.923" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9110" PostTypeId="1" AcceptedAnswerId="9120" CreationDate="2019-07-31T05:59:25.520" Score="0" ViewCount="2398" Body="&lt;p&gt;I have to intersect multiple data frame to find common elements. For visualization purpose i can use upset Plot but it doesn;t return a object containing the intersection object .I tried jvenn but at max it can take six sets to intersect .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Im able to do the interesction but i would like to know if i can do it more efficiently ,which apparently not sure how to do this is my code as of now where i read each file as data frame, covert them to list and pass them to do intersection .&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NewList &amp;lt;- split(GO0000122, f = seq(nrow(GO0000122)))&#xA;NewList1 &amp;lt;- split(GO0006351,f=seq(nrow(GO0006351)))&#xA;NewList2 &amp;lt;- split(GO0006355,f=seq(nrow(GO0006355)))&#xA;NewList3 &amp;lt;- split(GO0006357,f=seq(nrow(GO0006357)))&#xA;NewList4 &amp;lt;- split(GO0006366,f=seq(nrow(GO0006366)))&#xA;NewList5 &amp;lt;- split(GO0030154,f=seq(nrow(GO0030154)))&#xA;NewList6 &amp;lt;- split(GO0045892,f=seq(nrow(GO0045892)))&#xA;NewList8 &amp;lt;- split(GO0045893,f=seq(nrow(GO0045893)))&#xA;NewList7 &amp;lt;- split(GO0045944,f=seq(nrow(GO0045944)))&#xA;&#xA;a &amp;lt;- intersect_all(NewList,NewList1,NewList2,NewList3,NewList4,NewList5,NewList6,NewList7,NewList8)&#xA;&#xA;&#xA;b &amp;lt;- do.call(rbind.data.frame, a)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can i read the files and pass them into list and do the same that would help me from doing lot of things manually &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can read the files from the directory where i have my files but not sure how to pass them to make it as list.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;filenames &amp;lt;- gsub(&quot;\\.csv&lt;span class=&quot;math-container&quot;&gt;$&quot;,&quot;&quot;, list.files(pattern=&quot;\\.csv$&lt;/span&gt;&quot;))&#xA;for(i in filenames){&#xA;  assign(i, read.csv(paste(i, &quot;.csv&quot;, sep=&quot;&quot;)))&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The object filenames is &lt;code&gt;character&lt;/code&gt; here .It just returns me list of files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how to make my question bit more clear , i would like to read the list of file from the directory and then pass these to perform intersection. Any suggestion or help would be really appreciated .&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2019-07-31T21:14:50.477" Title="Intersecting multiple data frames" Tags="|r|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="9111" PostTypeId="1" CreationDate="2019-07-31T06:06:11.347" Score="2" ViewCount="89" Body="&lt;p&gt;I discovered that Phytozome's BLAST results look quite nice (please see below). &#xA;By any chance, does anyone know where to download this BLAST server in order to install it locally for organisms which are not part of Phytozome?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/c6WR7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/c6WR7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance,&lt;/p&gt;&#xA;" OwnerUserId="2477" LastEditorUserId="2477" LastEditDate="2019-07-31T12:51:26.263" LastActivityDate="2023-05-02T23:00:20.070" Title="BLAST server download" Tags="|sequence-alignment|blast|webservice|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9112" PostTypeId="1" CreationDate="2019-07-31T06:08:54.287" Score="2" ViewCount="120" Body="&lt;p&gt;I am trying to create a multiple coloured line graph based on six lines.&#xA;Essentially I have data that identifies different units of vegetation by the variable” NDVI (vegetation index)” I want to create a plot with differently coloured lines for each units.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following is a screen shot of the data structure and my code.&#xA;It is plotting all of the data. But the time data on X-axis is disrupted and not ordered. What am I missing? And also how do I get legend for my chart?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(ggplot2)&#xA;library(tidyverse)&#xA;&#xA;df2 &amp;lt;- data.frame(Time&amp;lt;-c(&quot;28Jan&quot;,&quot;13Feb&quot;,&quot;16Mar&quot;,&quot;1Apr&quot;,&quot;17Apr&quot;,&quot;3May&quot;,&quot;19May&quot;,&quot;4Jun&quot;,&quot;20Jun&quot;,&quot;6Jul&quot;,&quot;22Jul&quot;,&quot;7Aug&quot;,&quot;23Aug&quot;,&quot;8Sep&quot;,&quot;24Sep&quot;,&quot;10Oct&quot;,&quot;26Oct&quot;,&quot;11Nov&quot;,&quot;13Dec&quot;,&quot;29Dec&quot;),&#xA;                  A&amp;lt;-c(0.209,0.225,0.237,0.312,0.377,0.468,0.499,0.492,0.45,0.43,0.405,0.405,0.378,0.358,0.337,0.297,0.179,0.305,0.241,0.183),&#xA;                  B&amp;lt;-c(-0.472,-0.589,-0.422,-0.479,-0.371,-0.112,-0.03,-0.134,-0.132,-0.12,-0.1,-0.227,-0.137,-0.025,-0.02,-0.011,-0.024,-0.377,-0.494,-0.49),&#xA;                  C&amp;lt;-c(-0.278,-0.385,-0.31,-0.3,-0.279,-0.033,0.089,0.154,0.231,0.308,0.341,0.4,0.397, 0.383,0.37,0.309,0.248, 0.271,-0.212,-0.3),&#xA;                  D&amp;lt;-c(-0.544,-0.604,-0.505,-0.541,-0.475,-0.173,-0.054,-0.026,0.098,0.199,0.257,0.338,0.37,0.372,0.36,0.274,0.217,-0.007,-0.552,-0.528),&#xA;                  E&amp;lt;-c( 0.213,0.239,0.277,0.304,0.384,0.47,0.537,0.547,0.5,0.489,0.452,0.469,0.439,0.419,0.408,0.372,0.049,0.335,0.304,0.282),&#xA;                  F&amp;lt;-c(0.234,0.248,0.3,0.324,0.391,0.444,0.512,0.503,0.448,0.434,0.407,0.421,0.4,0.385,0.376,0.335,0.087,0.323,0.295,0.275))&#xA;head(df2)&#xA;&#xA;ggplot(df2, aes(x = Time, group = 1)) +&#xA;  geom_line(aes(y = A), colour=&quot;blue&quot;) +&#xA;  geom_line(aes(y = B), colour = &quot;red&quot;) +&#xA;  geom_line(aes(y = C), colour = &quot;green&quot;) +&#xA;  geom_line(aes(y = D), colour = &quot;black&quot;) +&#xA;  geom_line(aes(y = E), colour = &quot;yellow&quot;) +&#xA;  geom_line(aes(y = F), colour = &quot;brown&quot;) +&#xA;  ylab(label=&quot;NDVI&quot;) + xlab(&quot;Time&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/A8J8C.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/A8J8C.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4952" LastEditorUserId="678" LastEditDate="2019-07-31T08:53:07.830" LastActivityDate="2019-07-31T08:53:07.830" Title="Multiple line chart" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9113" PostTypeId="2" ParentId="9112" CreationDate="2019-07-31T07:34:25.177" Score="3" Body="&lt;p&gt;Couple of things regarding your code:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;It would be good to have your data frame in &lt;a href=&quot;https://rpubs.com/kaz_yos/ggplot-dataformat&quot; rel=&quot;nofollow noreferrer&quot;&gt;long format&lt;/a&gt; for ggplot2&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For having the x-axis ordered chronologically, you have to specify it manually, otherwise it is ordered alphabetically.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Then, here is my solution:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(ggplot2)&#xA;library(reshape2)&#xA;&#xA;# I have substituted your &amp;lt;- by = and changed the name of the groups&#xA;df2 &amp;lt;- data.frame(Time = c(&amp;quot;28Jan&amp;quot;,&amp;quot;13Feb&amp;quot;,&amp;quot;16Mar&amp;quot;,&amp;quot;1Apr&amp;quot;,&amp;quot;17Apr&amp;quot;,&amp;quot;3May&amp;quot;,&amp;quot;19May&amp;quot;,&amp;quot;4Jun&amp;quot;,&amp;quot;20Jun&amp;quot;,&amp;quot;6Jul&amp;quot;,&amp;quot;22Jul&amp;quot;,&amp;quot;7Aug&amp;quot;,&amp;quot;23Aug&amp;quot;,&amp;quot;8Sep&amp;quot;,&amp;quot;24Sep&amp;quot;,&amp;quot;10Oct&amp;quot;,&amp;quot;26Oct&amp;quot;,&amp;quot;11Nov&amp;quot;,&amp;quot;13Dec&amp;quot;,&amp;quot;29Dec&amp;quot;),&#xA;                  g_A = c(0.209,0.225,0.237,0.312,0.377,0.468,0.499,0.492,0.45,0.43,0.405,0.405,0.378,0.358,0.337,0.297,0.179,0.305,0.241,0.183),&#xA;                  g_B = c(-0.472,-0.589,-0.422,-0.479,-0.371,-0.112,-0.03,-0.134,-0.132,-0.12,-0.1,-0.227,-0.137,-0.025,-0.02,-0.011,-0.024,-0.377,-0.494,-0.49),&#xA;                  g_C = c(-0.278,-0.385,-0.31,-0.3,-0.279,-0.033,0.089,0.154,0.231,0.308,0.341,0.4,0.397, 0.383,0.37,0.309,0.248, 0.271,-0.212,-0.3),&#xA;                  g_D = c(-0.544,-0.604,-0.505,-0.541,-0.475,-0.173,-0.054,-0.026,0.098,0.199,0.257,0.338,0.37,0.372,0.36,0.274,0.217,-0.007,-0.552,-0.528),&#xA;                  g_E = c( 0.213,0.239,0.277,0.304,0.384,0.47,0.537,0.547,0.5,0.489,0.452,0.469,0.439,0.419,0.408,0.372,0.049,0.335,0.304,0.282),&#xA;                  g_F = c(0.234,0.248,0.3,0.324,0.391,0.444,0.512,0.503,0.448,0.434,0.407,0.421,0.4,0.385,0.376,0.335,0.087,0.323,0.295,0.275))&#xA;&#xA;# Convert to the proper format&#xA;df2.melted &amp;lt;- melt(df2)&#xA;&#xA;# Order of the factor in x-axis&#xA;df2.melted&lt;span class=&quot;math-container&quot;&gt;$Time &amp;lt;- factor(df2.melted$&lt;/span&gt;Time, levels = c(&amp;quot;28Jan&amp;quot;,&amp;quot;13Feb&amp;quot;,&amp;quot;16Mar&amp;quot;,&amp;quot;1Apr&amp;quot;,&amp;quot;17Apr&amp;quot;,&amp;quot;3May&amp;quot;,&amp;quot;19May&amp;quot;,&amp;quot;4Jun&amp;quot;,&amp;quot;20Jun&amp;quot;,&amp;quot;6Jul&amp;quot;,&amp;quot;22Jul&amp;quot;,&amp;quot;7Aug&amp;quot;,&amp;quot;23Aug&amp;quot;,&amp;quot;8Sep&amp;quot;,&amp;quot;24Sep&amp;quot;,&amp;quot;10Oct&amp;quot;,&amp;quot;26Oct&amp;quot;,&amp;quot;11Nov&amp;quot;,&amp;quot;13Dec&amp;quot;,&amp;quot;29Dec&amp;quot;))&#xA;&#xA;# Note group = variable to connect the points of the different groups&#xA;ggplot(df2.melted, aes(x = Time, y = value, colour = variable, group = variable)) + &#xA;  geom_line() + &#xA;  ylab(&amp;quot;NDVI&amp;quot;) + xlab(&amp;quot;Time&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/S8SVC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/S8SVC.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="678" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2019-07-31T07:34:25.177" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9114" PostTypeId="1" CreationDate="2019-07-31T08:45:09.640" Score="2" ViewCount="171" Body="&lt;p&gt;I was expecting something like in this image:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fPBKD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fPBKD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when I ran Mutect2, the FILTER field was empty in the output VCF file. How can I enable PASS filters?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    19063526    0   T   TAGCTTAGAATGGGATGTC 0   0   DP=1;ECNT=1;MBQ=0,34;MFRL=0,193;MMQ=60,52;MPOS=1;POPAF=7.30;TLOD=4.20   GT:AD:AF:DP:F1R2:F2R1:SB    0/1:0,1:0.667:1:0,0:0,1:0,0,0,1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Command used: &lt;code&gt;gatk Mutect2 -R hg38.fa -I test.bam -O test.vcf.gz&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GATK -v4.1.2.0&#xA;Mutect2 version 2.2&#xA;&#xA;Linux v4.15.0-1044&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4658" LastEditorUserId="298" LastEditDate="2019-07-31T11:05:38.100" LastActivityDate="2019-07-31T11:05:38.100" Title="Empty values in FILTER field after running Mutect2" Tags="|vcf|filtering|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="9115" PostTypeId="1" AcceptedAnswerId="13940" CreationDate="2019-07-31T09:39:21.363" Score="3" ViewCount="655" Body="&lt;p&gt;I am new to proteomics research and analyzing mass-spec data. I am trying to learn more about the theory/principle of TMT tagging (a type of isobaric labeling) coupled with LC-MS/MS spectrometry. I understand that each tag has a &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) protein binding group (amine reactive ONLY?) &#xA;2) reporter group (so we know the tag came from which sample?) &#xA;3) normalisation group (to ensure that each tag has same mass?) &#xA;4) cleavage group (so tag can be cleaved from the peptide?) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Did I get the purpose of all the four parts in the tag? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, what does it mean by 4-plex, 10-plex, n-plex tag? And why do the tags have different isotopes? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I could have some clarity on the workflow of TMT tagging and how it leads to analysis, it would be really helpful. &lt;/p&gt;&#xA;" OwnerUserId="4996" LastActivityDate="2020-08-19T21:00:40.067" Title="Principle of TMT Tags in Multiplex Proteomics" Tags="|proteins|proteomics|mass-spectrometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9117" PostTypeId="2" ParentId="9013" CreationDate="2019-07-31T10:10:17.087" Score="1" Body="&lt;p&gt;I figured out the issue by myself. It turned out that UCSC liftover doesn't like the character &quot;:&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I removed &quot;:&quot; and it worked&lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2019-07-31T10:10:17.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9118" PostTypeId="1" CreationDate="2019-07-31T10:56:42.850" Score="1" ViewCount="191" Body="&lt;p&gt;I am implementing hi-glass python on Jupyter, and upon running the test data code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from higlass.client import View, Track&#xA;from higlass.tilesets import cooler&#xA;&#xA;ts1 = cooler('../data/Dixon2012-J1-NcoI-R1-filtered.100kb.multires.cool')&#xA;tr1 = Track('heatmap', tileset=ts1)&#xA;view1 = View([tr1])&#xA;display, server, viewconf = higlass.display([view1])&#xA;&#xA;display&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get the error message stating:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error Retrieving tileset info: http://localhost:60501/spi/v1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Please advise on the correcting. The code above has been copied from the &lt;a href=&quot;https://docs.higlass.io/jupyter.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;HiGlass webpage&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4976" LastEditorUserId="57" LastEditDate="2019-07-31T12:05:53.330" LastActivityDate="2019-08-03T22:06:26.047" Title="Error Viewing file on HiGlass" Tags="|visualization|genomics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9119" PostTypeId="1" CreationDate="2019-07-31T11:24:03.960" Score="1" ViewCount="227" Body="&lt;p&gt;I found a good answer related to my question here:&#xA;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2503/how-to-get-a-list-of-genes-corresponding-to-the-list-of-snps-rs-ids&quot;&gt;How to get a list of genes corresponding to the list of SNPs (rs ids)?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But it says about small number of SNPs. I want to get gene names based on approximately 12000 to 19000 SNPs. NCBI says about 10 read per second with API key. Is there any efficient way to get gene names for thousands of SNPs? Is that database available to store locally? Any advice please?&lt;/p&gt;&#xA;" OwnerUserId="4790" LastEditorUserId="292" LastEditDate="2019-07-31T15:54:27.490" LastActivityDate="2019-10-03T17:01:55.390" Title="How to efficiently get human gene names from NCBI based on a large list of SNPs" Tags="|gene|snp|python|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9120" PostTypeId="2" ParentId="9110" CreationDate="2019-07-31T15:57:49.860" Score="1" Body="&lt;p&gt;Change dataframe to load all the files into a list:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;wd &amp;lt;- &quot;C:/all_your_dataframes_in_one_folder&quot;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;setwd(wd)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then you get the list of files you want from your folder&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;files &amp;lt;- list.files()&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Put into a list with a loop&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dflist &amp;lt;- list()&#xA;for(i in 1:length(files)){&#xA; dflist[[i]] &amp;lt;- read.csv(files[i])&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then use tidyverse solution &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;library(tidyverse)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;map(dflist, ~.$GeneID) %&amp;gt;% #creates a list just of the column of interest&#xA;    reduce(intersect)  #applies the intersect function cumulatively to the list&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="4001" LastEditDate="2019-07-31T21:14:50.477" LastActivityDate="2019-07-31T21:14:50.477" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="9122" PostTypeId="2" ParentId="9119" CreationDate="2019-07-31T21:07:35.133" Score="1" Body="&lt;p&gt;Modify SNPs and gene annotations for your genome assembly of interest. This example shows how one might do this for &lt;code&gt;hg19&lt;/code&gt;, for instance:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Download SNPs and convert to BED:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ wget -qO- ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606_b151_GRCh37p13/VCF/common_all_20180423.vcf.gz \&#xA;    | gunzip -c \&#xA;    | convert2bed --input=vcf --output=bed --sort-tmpdir=$&lt;/span&gt;{PWD} - \&#xA;    &amp;gt; hg19.snp151.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Filter this BED file for entries of interest:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -wFf snps-of-interest.txt hg19.snp151.bed | cut -f1-6 &amp;gt; hg19.snp151.filtered.bed6&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Download gene annotations and convert to BED:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ wget -O - ftp://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_19/gencode.v19.annotation.gtf.gz \&#xA;    | gunzip -c \&#xA;    | awk '($&lt;/span&gt;3 == &quot;gene&quot;)' \&#xA;    | gtf2bed \&#xA;    &amp;gt; gencode.v19.genes.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Note&lt;/strong&gt;: In the future, newer versions of Gencode annotations may be available for your reference assembly. Always check their site to confirm what is most recent.&lt;/em&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Map genes to SNPs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --echo --echo-map-id hg19.snp151.filtered.bed6 gencode.v19.genes.bed &amp;gt; snps_with_associated_gene_names.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;A list of genes that overlap a SNP of interest will be in the seventh column of the output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this must be done within Python, you could use something like &lt;code&gt;subprocess.call(...)&lt;/code&gt;, where &lt;code&gt;...&lt;/code&gt; includes commands wrapped in &lt;code&gt;bash -c&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't recommend this because of the need to escape quote marks and deal with Python's quirks in running command-line tasks, which makes a Python-based approach very difficult to set up and maintain. I'd suggest learning some basic &lt;code&gt;bash&lt;/code&gt; scripting to make this easy and fast.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2019-09-03T16:58:47.307" LastActivityDate="2019-09-03T16:58:47.307" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9123" PostTypeId="1" CreationDate="2019-07-31T23:14:54.420" Score="0" ViewCount="1063" Body="&lt;p&gt;I have a reference .fasta file and a raw .fastq file with chip-seq data. I am trying to create a bigwig track from the and .fastq and .fasta ref file of the raw signal. Then I would like to do some peak analysis for the chip-seq track.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone help me with googling the write things, I feel really lost while trying to google it and not sure where to start from. I was thinking to do &lt;code&gt;.fastq -&amp;gt; .bam -&amp;gt; wig -&amp;gt; bigwig&lt;/code&gt; and then somehow to do peak detection analysis. &lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2019-08-01T02:23:46.567" LastActivityDate="2019-08-01T08:54:34.413" Title="How to convert a .fastq chip-seq file into a wig/bigwig track and how to extract chip-seq peaks?" Tags="|fastq|chip-seq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9124" PostTypeId="2" ParentId="9123" CreationDate="2019-08-01T08:54:34.413" Score="1" Body="&lt;p&gt;Before you proceed, it is usual to do quality control on the .fastq data.  The FASTQC program is a good place to start.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have checked the quality, trimming is the typical next step to remove low-quality data.  There are options such as Trimomatic, trimGalore for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you are sure you have clean data, your first step is alignment - align the sequences in the .fastq file to the reference sequence in the .fasta file.  bowtie2 is one option for this process.  This gets you to the .bam stage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These steps will take a little time, some research, and may be somewhat frustrating, but they are the foundation for what you want to do. One option for somewhat automating these steps is to use the Galaxy platform (usegalaxy.org), otherwise these are mostly command-line tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The bam to wig and wig to bigwig steps are relatively simple conversions, and Biostars has answers for these.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-08-01T08:54:34.413" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9125" PostTypeId="1" AcceptedAnswerId="9127" CreationDate="2019-08-01T08:55:55.130" Score="0" ViewCount="589" Body="&lt;p&gt;In bioinformatics, a &lt;a href=&quot;https://en.wikipedia.org/wiki/Sequence_logo&quot; rel=&quot;nofollow noreferrer&quot;&gt;sequence logo&lt;/a&gt; is a graphical representation of the sequence conservation of nucleotides (in a strand of DNA/RNA) or amino acids (in protein sequences).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;wiki says&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The total height of the letters depicts the information content of the&#xA;  position, in bits.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and then gives this example&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A sequence logo showing the most conserved bases around the initiation&#xA;  codon from all human mRNAs (Kozak consensus sequence). Note that the&#xA;  initiation codon is not drawn to scale, otherwise the letters AUG&#xA;  would each have a height of 2 bits.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eA1L0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eA1L0.png&quot; alt=&quot;e&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;it seems that there is a 0.1 bit letter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;is this in the context of 2bits? is 2bits the highest bits in this sequence logo? Is there a convention or standard of the highest bits in a sequence logo?&lt;/p&gt;&#xA;" OwnerUserId="4983" LastActivityDate="2019-08-01T09:38:33.093" Title="Is there a convention or standard of the highest bits in a sequence logo?" Tags="|ngs|sequence-analysis|" AnswerCount="1" CommentCount="1" ClosedDate="2019-08-01T10:02:21.450" ContentLicense="CC BY-SA 4.0" />
  <row Id="9126" PostTypeId="2" ParentId="6672" CreationDate="2019-08-01T09:27:52.080" Score="1" Body="&lt;p&gt;There are specialized tools to deconvolute (separate) heterozygous InDels in a chromatogram trace using a reference genome. For instance, &lt;a href=&quot;https://www.gear-genomics.com/indigo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Indigo&lt;/a&gt; is a web application for this, the command-line application is called &lt;a href=&quot;https://github.com/gear-genomics/tracy&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tracy&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4984" LastEditorUserId="4984" LastEditDate="2019-09-24T11:47:37.817" LastActivityDate="2019-09-24T11:47:37.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9127" PostTypeId="2" ParentId="9125" CreationDate="2019-08-01T09:38:33.093" Score="1" Body="&lt;p&gt;This is explained in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/2172928&quot; rel=&quot;nofollow noreferrer&quot;&gt;the original paper&lt;/a&gt; [1] describing the logos (emphasis mine):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The importance of a particular position in a binding site is more clearly&#xA;  and consistently given by the information required to describe the pattern&#xA;  there [7, 8]. To choose one symbol or state from two equally likely possibilities&#xA;  requires one &quot;bit&quot; of information. For example, to communicate the result of&#xA;  a coin-ip to someone requires 1 bit of information because only one yes-no&#xA;  question needs to be answered: &quot;Is it heads?&quot;. &lt;strong&gt;If a position in a binding&#xA;  site always contains one base (e.g. G), then we need exactly two bits of&#xA;  information since two yes-no questions need to be answered: &quot;Is it A or G?&quot;&#xA;  (i.e. is it a purine?) and &quot;Is it A or C?&quot;. (If the answers to both questions&#xA;  are &quot;no&quot;, it must be a T.)&lt;/strong&gt; Furthermore, if a position contains two bases (e.g.&#xA;  sometimes A and sometimes G), only one question suffices since a two out of&#xA;  four choice is equivalent to a one out of two choice. Therefore, only one bit&#xA;  is needed to describe a position in a binding site that contains only purines,&#xA;  but two bits are needed to describe a position that always contains adenine.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So yes, 2bits is the highest value you will see on the y-axis of a &lt;strong&gt;&lt;em&gt;DNA&lt;/em&gt;&lt;/strong&gt; sequence logo (proteins are a different story since there are more possibilities). In the one you show, the scale is from 0 to 0.2, but it's the same principle. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you should think of logos as a useful visual aid, and not a way of getting rigorous mathematical information about a sequence. So don't put too much weight on the bit values. If you need that sort of data, calculate it from the alignment directly. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;[1] Schneider TD, Stephens RM., Sequence logos: a new way to display consensus sequences., &lt;em&gt;Nucleic Acids Res.&lt;/em&gt; 1990 Oct 25;18(&lt;strong&gt;20&lt;/strong&gt;):6097-100.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-08-01T09:38:33.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9128" PostTypeId="1" CreationDate="2019-08-01T10:25:50.107" Score="0" ViewCount="111" Body="&lt;p&gt;this is A &lt;a href=&quot;https://en.wikipedia.org/wiki/Kozak_consensus_sequence&quot; rel=&quot;nofollow noreferrer&quot;&gt;sequence logo&lt;/a&gt; showing the most conserved bases around the initiation codon from 10000 human mRNAs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Xq8l9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Xq8l9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to reproduce this sequence logo. How can I get the sequences? which database should I use? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;this search is apparently wrong, since I don't know the right way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/?term=10000+human+mRNAs&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/?term=10000+human+mRNAs&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4983" LastEditorUserId="4983" LastEditDate="2019-08-01T22:49:05.547" LastActivityDate="2019-08-02T16:03:40.330" Title="How to reproduce this specific sequence logo from 10 000 human mRNAs?" Tags="|sequence-alignment|ngs|sequence-analysis|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9129" PostTypeId="1" AcceptedAnswerId="9140" CreationDate="2019-08-01T15:42:12.820" Score="3" ViewCount="556" Body="&lt;p&gt;Snakemake provides access to a &lt;code&gt;workflow&lt;/code&gt; object within a Snakefile. This allows one to, for example, have dynamic programmatic access to the directory containing the Snakefile (via the &lt;code&gt;workflow.basedir&lt;/code&gt; attribute). Is there a similar way to access the working directory that is specified on the command line using &lt;code&gt;--directory&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Through a bit of exploration I was able to figure out that the &lt;code&gt;workflow&lt;/code&gt; object has a &lt;code&gt;workdir()&lt;/code&gt; method, but it appears this is used internally by Snakemake for registering working directories, not for access.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;: It looks like the working directory is stored internally as the &lt;code&gt;_workdir&lt;/code&gt; attribute of the &lt;code&gt;workflow&lt;/code&gt; object. However, this attribute is unset (default value &lt;code&gt;None&lt;/code&gt;) when I'm creating static/global objects in the Snakefile. Is it impossible to access the working directory prior to building the workflow DAG?&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2019-08-01T15:54:44.823" LastActivityDate="2019-08-02T18:21:44.650" Title="Access workdir defined on command line from within Snakefile" Tags="|python|snakemake|workflow-management|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9130" PostTypeId="2" ParentId="933" CreationDate="2019-08-01T19:03:21.757" Score="6" Body="&lt;p&gt;The new gnomAD site (as of August 2019) &lt;a href=&quot;https://gnomad.broadinstitute.org/faq#how-do-i-query-a-batch-of-variants-do-you-have-an-api&quot; rel=&quot;noreferrer&quot;&gt;says no, no API yet&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;How do I query a batch of variants? Do you have an API?&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;We currently do not have a way to submit batch queries on the browser, but we are actively working on developing an API for ExAC/gnomAD. If you would like to learn about GraphQL, which we will use to work with the API, an overview can be found at &lt;a href=&quot;https://graphql.org&quot; rel=&quot;noreferrer&quot;&gt;https://graphql.org&lt;/a&gt;.  You can also obtain information on all variants from the VCFs and Hail Tables available on our downloads page.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But, the web interface itself already makes POST requests to &lt;a href=&quot;https://gnomad.broadinstitute.org/api&quot; rel=&quot;noreferrer&quot;&gt;https://gnomad.broadinstitute.org/api&lt;/a&gt; to send and receive JSON/GraphQL.  So, you can make those same queries programmatically right now, even if it's not officially a public API.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gist.github.com/ressy/6fd7f6ee6401ac8e703dc2709399869e&quot; rel=&quot;noreferrer&quot;&gt;Here's an example in Python&lt;/a&gt; to get some basic info on variants for a particular gene.  This way you get simple nested Python objects to work with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  { 'consequence': 'intron_variant',&#xA;    'pos': 6928442,&#xA;    'rsid': 'rs782435448',&#xA;    'variant_id': '12-6928442-C-A'},&#xA;  { 'consequence': 'splice_region_variant',&#xA;    'pos': 6928462,&#xA;    'rsid': None,&#xA;    'variant_id': '12-6928462-C-A'},&#xA;  { 'consequence': 'splice_acceptor_variant',&#xA;    'pos': 6928464,&#xA;    'rsid': 'rs782577109',&#xA;    'variant_id': '12-6928464-G-A'},&#xA;  { 'consequence': 'missense_variant',&#xA;    'pos': 6928466,&#xA;    'rsid': 'rs782208003',&#xA;    'variant_id': '12-6928466-C-T'},&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(I found it useful to go this route because then the full metadata visible in the gnomAD web interface is then available, including the per-variant details like allele counts by population.  I couldn't find this information in the other APIs described here.)&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2019-08-01T19:03:21.757" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9131" PostTypeId="1" CreationDate="2019-08-01T20:34:23.313" Score="0" ViewCount="82" Body="&lt;p&gt;Is it possible to run cell ranger clustering using only a gene x cell matrix? Cellranger reanalyze performs clustering but requires a hd5f file. I don't want to run the whole pipeline (including alignment). I was trying to manually generate a hd5f file to input to cellranger reanalyze. Is this possible?&lt;/p&gt;&#xA;" OwnerUserId="4986" LastActivityDate="2019-08-01T21:43:56.413" Title="Clustering using cellranger" Tags="|clustering|cellranger|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9132" PostTypeId="2" ParentId="9014" CreationDate="2019-08-01T21:28:54.137" Score="2" Body="&lt;p&gt;I think your issue is simply not assigning the newly-subset object in to a new object.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming &lt;code&gt;rownames(subset_DR1)&lt;/code&gt; is indeed a list of cell names&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seuratObj_subset_dr1 &amp;lt;- SubsetData(seuratObj, cells=rownames(subset_DR1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;will work.&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="181" LastEditDate="2019-08-02T18:50:11.307" LastActivityDate="2019-08-02T18:50:11.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9133" PostTypeId="2" ParentId="9131" CreationDate="2019-08-01T21:43:56.413" Score="2" Body="&lt;p&gt;The direct answer is yes. If you have never worked with hdf5 you can start &lt;a href=&quot;https://portal.hdfgroup.org/display/HDF5/Introduction+to+HDF5&quot; rel=&quot;nofollow noreferrer&quot;&gt; here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But, cellranger outputs several hdf5 files. You can extract the count matrix from the  &quot;/outs/filtered_gene_bc_matrices_h5.h5&quot; file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They provide a lot of information on how to use the hdf5 format so their &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/advanced/h5_matrices&quot; rel=&quot;nofollow noreferrer&quot;&gt;their documentation&lt;/a&gt; is a good place to start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition: you can also load the data into Seurat, monocle or scanpy and follow their tutorials to analyze the data.&lt;/p&gt;&#xA;" OwnerUserId="2675" LastActivityDate="2019-08-01T21:43:56.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9134" PostTypeId="2" ParentId="9128" CreationDate="2019-08-01T23:55:44.227" Score="1" Body="&lt;p&gt;If you look at the Appendix from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC306349/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC306349/&lt;/a&gt;, there is a table of sequences that were likely used to construct the sequence logo, at least partially, from the 5' end to the stereotyped ATG subsequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's not in a nice format amenable to use directly with weblogo (like FASTA), but with some programming/elbow grease it might be rendered into that form, which you can then use.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, you might investigate the supplementary materials of these and related papers, to see if you might something directly usable.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2019-08-02T16:03:40.330" LastActivityDate="2019-08-02T16:03:40.330" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9136" PostTypeId="1" AcceptedAnswerId="9138" CreationDate="2019-08-02T08:34:37.737" Score="1" ViewCount="1078" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/AADACL3&quot; rel=&quot;nofollow noreferrer&quot;&gt;This figure&lt;/a&gt; comes from &lt;a href=&quot;https://en.wikipedia.org/wiki/AADACL3&quot; rel=&quot;nofollow noreferrer&quot;&gt;this wiki page&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0Lqh5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0Lqh5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I googled &quot;gene location figure&quot; and &quot;gene location plot&quot;, none gets similar results. I also tried search by image, none of the results is close.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so, what is the name of this type of figure?&lt;/p&gt;&#xA;" OwnerUserId="4983" LastEditorUserId="96" LastEditDate="2019-08-02T13:12:23.150" LastActivityDate="2019-08-02T13:12:23.150" Title="What is the name of this type of figure?" Tags="|ngs|visualization|genomics|figure-explanation|genetics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9137" PostTypeId="2" ParentId="9136" CreationDate="2019-08-02T08:46:23.427" Score="3" Body="&lt;p&gt;Such a figure is called ideogram, or chromosome ideogram.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2019-08-02T08:46:23.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9138" PostTypeId="2" ParentId="9136" CreationDate="2019-08-02T08:55:38.097" Score="6" Body="&lt;p&gt;This kind of visualization showing the banding patterns of chromosomes are called &quot;&lt;a href=&quot;https://www.dna-rainbow.org/ideograms&quot; rel=&quot;nofollow noreferrer&quot;&gt;chromosome ideograms&lt;/a&gt;&quot;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use for example &lt;a href=&quot;https://www.bioconductor.org/packages/IdeoViz&quot; rel=&quot;nofollow noreferrer&quot;&gt;IdeoViz&lt;/a&gt; in R to generate them.  &lt;/p&gt;&#xA;" OwnerUserId="4885" LastEditorUserId="48" LastEditDate="2019-08-02T10:36:41.377" LastActivityDate="2019-08-02T10:36:41.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9139" PostTypeId="1" AcceptedAnswerId="9142" CreationDate="2019-08-02T14:57:14.837" Score="1" ViewCount="48" Body="&lt;p&gt;I have been running a couple of blast searches and I am very suprised and somewhat frustrated by the huge variability in job completion time. The specifics of my blast search are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Running blast using webbrowser (chrome) via ncbi&lt;/li&gt;&#xA;&lt;li&gt;19 nucleotide query&lt;/li&gt;&#xA;&lt;li&gt;database: WGS of one particular sequencing project/bacterial strain (so database is VERY small, in total approx 4.5 MB divided over 60 contigs)&lt;/li&gt;&#xA;&lt;li&gt;parameters are mostly untouched and are automatically optimized for short queries by blast&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In one instance, I started the job more than an hour ago and it is still not done. Knowing from experience that this previously hasn't taken that much time, I started a new job in a different browser tab with exactly the same sequence, database and parameters, and now it finished within 20 seconds. What on earth is happening here?&lt;/p&gt;&#xA;" OwnerUserId="4993" LastEditorUserId="4993" LastEditDate="2019-08-03T09:22:10.643" LastActivityDate="2019-08-03T11:04:32.823" Title="Huge variability in blastn speed between same short sequence job" Tags="|blast|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9140" PostTypeId="2" ParentId="9129" CreationDate="2019-08-02T18:21:44.650" Score="2" Body="&lt;p&gt;You could query Python for the working directory within the Snakefile, since evidently Snakemake changes the actual working directory of the process to the one specified by &lt;code&gt;--directory&lt;/code&gt;.    For example, using a stub Snakefile with just these two lines:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;print(&quot;Current working directory: &quot; + os.getcwd())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And on the command line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ snakemake --directory working-directory&#xA;Creating specified working directory working-directory.&#xA;Current working directory: /home/jesse/dev/snakemake-wd/working-directory&#xA;Building DAG of jobs...&#xA;Nothing to be done.&#xA;Complete log: /home/jesse/dev/snakemake-wd/working-directory/.snakemake/log/2019-08-02T142009.349338.snakemake.log&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2019-08-02T18:21:44.650" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9142" PostTypeId="2" ParentId="9139" CreationDate="2019-08-03T11:04:32.823" Score="1" Body="&lt;p&gt;First of all, if you're using an online service like NCBI's BLAST, then the time to completion will depend on the load the service is experiencing. Your job is ut in a queue and will be started when it's its turn. If the service is under heavy load, then its turn might come later than you expect. Then, if the service is under heavy load, even after launching your job, it might take longer to finish because it won't have enough resources to run quickly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since you say that the same job finished quickly on a new tab, it sounds like either the first job was last in the queue or there was just a bug that caused it not be launched or something like that. We can't really help with things that resolve themselves like that. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-08-03T11:04:32.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9143" PostTypeId="1" CreationDate="2019-08-03T16:31:17.420" Score="1" ViewCount="93" Body="&lt;p&gt;After qblast, I wanted to print only details from the first 3 BLAST output. Yet, the code below printed the details of every BLAST output. Why can't I break out of the loop to achieve my desired output?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for alignment in blast_records.alignments:&#xA;    countResults = 0&#xA;    for hsp in alignment.hsps:  &#xA;        if countResults == maxResults:&#xA;            countResults = 0&#xA;        break&#xA;        else:&#xA;            print(alignment.title)&#xA;            print(alignment.length/ len(seq)*100)&#xA;            print(hsp.expect)&#xA;            print(hsp.identities)&#xA;            countResults += 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4999" LastEditorUserId="3848" LastEditDate="2019-08-05T12:47:36.493" LastActivityDate="2019-08-08T02:50:11.503" Title="Biopython Failure to break out of loop" Tags="|python|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9145" PostTypeId="2" ParentId="9118" CreationDate="2019-08-03T22:06:26.047" Score="1" Body="&lt;p&gt;I'm one of the authors of HiGlass. Sorry you're getting that error. I just tried it out in a fresh environment and it worked for me. Just one thing is missing in the docs which is that you need to do &lt;code&gt;import higlass&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's what I did to set up my environment:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conda create --name test-bi python=3.6&#xA;source activate test-bi&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The rest is from the docs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pip install jupyter higlass-python&#xA;jupyter nbextension install --py --sys-prefix --symlink higlass&#xA;jupyter nbextension enable --py --sys-prefix higlass&#xA;jupyter notebook&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It looks like either you or somebody else added a bug report here: &lt;a href=&quot;https://github.com/higlass/higlass-python/issues/29&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/higlass/higlass-python/issues/29&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll respond and continue the conversation there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, there's a repository full of examples from the HiGlass SciPy presentation here: &lt;a href=&quot;https://github.com/higlass/scipy19&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/higlass/scipy19&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2019-08-03T22:06:26.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9147" PostTypeId="2" ParentId="9143" CreationDate="2019-08-04T07:01:50.380" Score="2" Body="&lt;p&gt;Assuming that your actual code is this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for alignment in blast_records.alignments:&#xA;    countResults = 0&#xA;    for hsp in alignment.hsps:  &#xA;        if countResults == maxResults:&#xA;            countResults = 0&#xA;            break&#xA;        else:&#xA;            print(alignment.title)&#xA;            print(alignment.length/ len(seq)*100)&#xA;            print(hsp.expect)&#xA;            print(hsp.identities)&#xA;            countResults += 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here you have an outer loop, which is iterating over all the possible database matches, and an inner loop, iterating over the alignments within that database entry. You are currently limiting the inner loop, but not the outer loop, which you probably want to do. Try this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Take the first 3 database hits&#xA;for alignment in blast_records.alignments[:3]:&#xA;    for hsp in alignment.hsps:  &#xA;            print(alignment.title)&#xA;            print(alignment.length/ len(seq)*100)&#xA;            print(hsp.expect)&#xA;            print(hsp.identities)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4645" LastActivityDate="2019-08-04T07:01:50.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9148" PostTypeId="1" CreationDate="2019-08-04T07:47:39.280" Score="1" ViewCount="205" Body="&lt;p&gt;I received a large &lt;em&gt;sc-RNA-seq&lt;/em&gt; data matrix, as of the nature of &lt;em&gt;sc-RNA-seq&lt;/em&gt; compared to &lt;em&gt;bulk-RNA-seq&lt;/em&gt; the data matrix is very sparse.&#xA;Moreover, due to the fact that this is single cell data the cell progress stages become more dominant and might be dominant in finding the PC's.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to apply a dimensionality reduction method, such as PCA or tSNE, in order to enter it to some working algorithm to find principle curve for pathway analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ive encountered Bailey paper: &quot;&lt;a href=&quot;https://arxiv.org/abs/1208.4122&quot; rel=&quot;nofollow noreferrer&quot;&gt;Principal Component Analysis with Noisy and/or Missing Data&lt;/a&gt;&quot;, but not sure if it will help me in the case of &lt;em&gt;sc-RNA-seq&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would appreciate some reference link or other help&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="5003" LastEditorUserId="3848" LastEditDate="2019-08-25T08:53:11.393" LastActivityDate="2019-08-25T08:53:11.393" Title="PCA on large sparse matrix of single cell RNA-seq" Tags="|scrnaseq|pca|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9149" PostTypeId="2" ParentId="9143" CreationDate="2019-08-04T07:50:37.240" Score="2" Body="&lt;p&gt;There was an indentation bug (its a Python thing). Personally I'd use,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for alignment in blast_records.alignments:&#xA;    countResults = 0&#xA;    maxResults = 3&#xA;    for hsp in alignment.hsps:  &#xA;        if not countResults == maxResults:&#xA;            print(alignment.title)&#xA;            print(alignment.length/ len(seq)*100)&#xA;            print(hsp.expect)&#xA;            print(hsp.identities)&#xA;            countResults += 1&#xA;        else:&#xA;            countResults = 0 # I'd dedinitely remove this in your original code&#xA;            break&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The loop gives a bit more control if there is any subsequent filtering, species, percentage identity etc.. There are other ways to write it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an alternative&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for alignment in blast_records.alignments:&#xA;    maxResults = 3&#xA;    for countResults, hsp in enumerate(alignment.hsps):&#xA;        while countResults &amp;lt;= maxResults:                &#xA;            print(\&#xA;            alignment.title, &quot;\n&quot;,\&#xA;            alignment.length/ len(seq)*100, &quot;\n&quot;,\&#xA;            hsp.expect, &quot;\n&quot;, hsp.identities, &quot;\n&quot;\&#xA;            )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;BTW the enumerate function keeps track of the counting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternative print ... &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;           out = &quot;&quot;&quot; alignment.title,&#xA;           alignment.length/ len(seq)*100, &#xA;           hsp.expect, &#xA;           hsp.identities &quot;&quot;&quot;&#xA;           print (out)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Let me know if there are bugs&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-08-04T11:13:00.960" LastActivityDate="2019-08-04T11:13:00.960" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9150" PostTypeId="2" ParentId="2551" CreationDate="2019-08-04T08:35:15.290" Score="1" Body="&lt;p&gt;It is probably necessary to correct for the degree distribution of the network, I am not sure that a tool such as the Mantel test will quite do this (as suggested by user Michael G.). In naturally occurring networks degree distribution is a strong confounder.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then again I am not totally sure what shape your data is in or what the question is. You mention a background network, but I am not sure what this is or how you want to use it. Possibly your network reconstruction is a subset of your background network? In which case you just define the rule for how your subsample is drawn and then you compare it to permuted networks. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, here is some fairly crude but serviceable R code that implements a general purpose network rewiring test, using igraph. I am not using a background network as again I'm not sure what it is. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Python also has igraph so should be similar &lt;code&gt;rewire()&lt;/code&gt; function)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;compute_measure = function(network) {&#xA;  # i don't know what you are measuring, so you have to write this function.&#xA;  # this function takes a graph object as input &#xA;  # this function returns a single numeric value (whatever you're measuring) as output&#xA;}&#xA;&#xA;# &quot;net&quot; is a network of the graph type from igraph&#xA;&#xA;measure = compute_measure(net)&#xA;&#xA;num_perms = 1000&#xA;&#xA;perm_measures = c()&#xA;&#xA;for (i in 1:num_perms) {&#xA;  # permute the network by rewiring, rewire 10x as many times as edges&#xA;  # to ensure saturation&#xA;  perm_net = rewire(net, with = keeping_degseq(niter = gsize(net) * 10))&#xA;  perm_measures[i] = compute_measure(perm_net)&#xA;}&#xA;&#xA;# one-sided monte carlo p value is proportion of rewirings (plus one) greater than your measure&#xA;p_value = (1 + length(which(perm_measures &amp;gt;= measure))) / num_perms&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If on the other hand you are just trying to come up with a single permutation of the edges of your network to compare against, you can run &lt;code&gt;rewire()&lt;/code&gt; only once to get that permuted network. It depends on what question you are trying to answer, and what null hypothesis you're trying to get a p value for.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2019-08-04T08:35:15.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9151" PostTypeId="1" CreationDate="2019-08-04T18:39:28.507" Score="1" ViewCount="132" Body="&lt;p&gt;The format of the raw data that upload in GEO is different, like TXT, CEL or other form. How can I extract the &lt;strong&gt;list of genes name&lt;/strong&gt; from these formats? And then how can I normalize them with R?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be really glad if anyone write &lt;strong&gt;step by step&lt;/strong&gt; a complete set of codes for R with an example.&lt;/p&gt;&#xA;" OwnerUserId="4091" LastActivityDate="2024-03-12T13:08:21.000" Title="How can extract the list of genes name from the raw data in GEO?" Tags="|r|microarray|phylogenetics|data-download|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9152" PostTypeId="2" ParentId="6959" CreationDate="2019-08-04T22:23:33.943" Score="1" Body="&lt;p&gt;If the output is File, you can add this step &lt;a href=&quot;https://github.com/Barski-lab/workflows/blob/master/tools/rename.cwl&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Barski-lab/workflows/blob/master/tools/rename.cwl&lt;/a&gt; to rename the output file.&lt;/p&gt;&#xA;" OwnerUserId="5006" LastActivityDate="2019-08-04T22:23:33.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9153" PostTypeId="2" ParentId="5419" CreationDate="2019-08-04T22:44:57.280" Score="1" Body="&lt;p&gt;It is possible that the shell being used in the process is different from your standard command line shell. I am not familiar with the tool, and it's hard to say more without knowing your OSX version, but it looks like it is using zsh. I think that &lt;a href=&quot;https://www.theverge.com/2019/6/4/18651872/apple-macos-catalina-zsh-bash-shell-replacement-features&quot; rel=&quot;nofollow noreferrer&quot;&gt;zsh only fairly recently became the standard OSX shell&lt;/a&gt;. I don't know the tool or the shell very well, but it looks like you might be able to explicitly set up hgsql in your PATH in your .zshrc. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's also possible that the liftover script was not built to run via zsh. I notice that it is trying to run csh scripts as part of the workflow. Then again the localhost may insulate it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I think that my first step would be to sort out which shell I am using, whether it works with the liftover pipeline, and how it's running on the localhost. I notice that the ssh call running the csh script on the localhost is the step that actually fails. What shell is executing the csh script? Does that shell know about hgsql? It could be that the way that the error bubbles up it is hiding which shell it is actually using. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd recommend trying that ssh call in isolation from the rest of the pipeline, and diagnosing the error that way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2019-08-04T22:44:57.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9154" PostTypeId="1" CreationDate="2019-08-05T14:28:40.380" Score="2" ViewCount="64" Body="&lt;p&gt;What is the quickest way to calculate the number of chemical reactions present in each pathway for a given organism and which online database such as &lt;a href=&quot;https://www.google.co.uk/url?sa=t&amp;amp;source=web&amp;amp;rct=j&amp;amp;url=https://www.genome.jp/kegg/pathway.html&amp;amp;ved=2ahUKEwiI9L6GguXqAhXoQhUIHduBCAQQFjAAegQIAhAB&amp;amp;usg=AOvVaw2QyaydXTKesFiTDlhfFLeU&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kegg&lt;/a&gt;, &lt;a href=&quot;https://www.google.co.uk/url?sa=t&amp;amp;source=web&amp;amp;rct=j&amp;amp;url=https://metacyc.org/&amp;amp;ved=2ahUKEwj3ut-YguXqAhW4URUIHeufDZwQFjAAegQIAhAB&amp;amp;usg=AOvVaw1LIpPmxAfCLf4Ppt2OgKiw&quot; rel=&quot;nofollow noreferrer&quot;&gt;MetaCyc&lt;/a&gt;,&lt;a href=&quot;https://www.google.co.uk/url?sa=t&amp;amp;source=web&amp;amp;rct=j&amp;amp;url=https://reactome.org/&amp;amp;ved=2ahUKEwj348CtguXqAhVCXRUIHX24DhMQFjAAegQIChAD&amp;amp;usg=AOvVaw28zRp8yw0zw_85-RfQX1so&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reactome&lt;/a&gt; is more helpful/reliable for this type of statistics?&lt;/p&gt;&#xA;" OwnerUserId="5012" LastEditorUserId="3848" LastEditDate="2020-07-24T04:11:00.037" LastActivityDate="2020-07-24T04:11:00.037" Title="Number of reactions per metabolic pathway" Tags="|python|statistics|pathway|api|kegg|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9155" PostTypeId="1" AcceptedAnswerId="9158" CreationDate="2019-08-05T15:47:55.267" Score="5" ViewCount="102" Body="&lt;p&gt;I am not sure I am building the proper design formula for the question I want to answer&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the following samples with three factors; clone, the structure and the condition.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clone structure diabetic&#xA;1 07        2D      Dia&#xA;2 21        2D     Ctrl&#xA;3 23        2D      Dia&#xA;4 32        2D     Ctrl&#xA;5 34        2D      Dia&#xA;6 43        2D     Ctrl&#xA;7 07        3D      Dia&#xA;8 21        3D     Ctrl&#xA;9 23        3D      Dia&#xA;10 32        3D     Ctrl&#xA;11 34        3D      Dia&#xA;12 43        3D     Ctrl&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to pull the differentially expressed genes for structure (2D vs 3D) blocking by clone, and the one differentially expressed by condition &#xA; (regardless of the structure variable).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure how to build the design in one call, therefore I have produced the first object to pull 2D vs 3D blocking by clone using the following.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;des_structure &amp;lt;- DESeqDataSetFromHTSeqCount(sampleTable = samples,&#xA;                                       directory = &quot;../data/htseq_geneCounts/&quot;,&#xA;                                       design= ~ clone+structure)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;for the condition controlled by structure I have used&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;des_condidtion &amp;lt;- ddsHTSeq_diabetic &amp;lt;- DESeqDataSetFromHTSeqCount(sampleTable = samples,&#xA;                                                 directory = &quot;../data/htseq_geneCounts/&quot;,&#xA;                                                 design= ~ structure+diabetic)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am not sure whether this is the best way to build the designs.&lt;/p&gt;&#xA;" OwnerUserId="5013" LastEditorUserId="96" LastEditDate="2019-08-05T19:04:21.487" LastActivityDate="2019-08-05T20:42:37.723" Title="design formula question" Tags="|rna-seq|deseq2|design|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9156" PostTypeId="1" CreationDate="2019-08-05T17:34:01.660" Score="3" ViewCount="732" Body="&lt;p&gt;I have a matrix &lt;code&gt;data&lt;/code&gt; with cells as rows and samples as columns. Here I am giving the data with &lt;code&gt;dput&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dput(data)&#xA;&#xA;structure(c(0, 0, 0.0095, 0.0091, 0, 0.0195, 0, 0.006, 0.0023, &#xA;0.0035, 0, 0.0306, 0, 0.0859, 0, 0, 0.0059, 0.0229, 0, 5e-04, &#xA;0, 0.0116, 0.0173, 0.0076, 0.0133, 0.0344, 0, 0.1263, 0, 0, 0.0378, &#xA;0, 0, 0.0198, 0, 0.0268, 0, 0, 0.0044, 0.0607, 0, 0.0073, 0, &#xA;0.1343, 0.0016, 0.0864, 0, 0, 0, 0.0296, 0.0122, 0.0329, 0.0204, &#xA;0.0582, 0, 0, 0.003, 0.1011, 0, 0, 0, 0, 0.0124, 0.0013, 0, 0.0025, &#xA;0.0076, 0.0498, 0, 0.0103, 2e-04, 0.0562, 0.0673, 0.069, 0, 0.0297, &#xA;0, 0.0207, 0.0329, 0.0072, 0.0028, 0.0589, 0.0533, 0.0139, 0, &#xA;0.0791, 0.0361, 0.026, 0, 0.0036, 0, 0.0319, 0.0155, 0.0401, &#xA;0.064, 0.0743, 0.0026, 0.012, 0, 0.0089, 0.0468, 0.0994, 0, 0.0646, &#xA;0, 0.0391, 0.0209, 0.0051, 0.007, 0.0689, 0, 0.0218, 0, 0, 0.0157, &#xA;0, 0, 0.0119, 0, 0.0136, 0, 0.0036, 0.0016, 0.0583, 0, 0, 0.0031, &#xA;0.2211, 0.6257, 0, 4e-04, 0.173, 0.0557, 0.1104, 0.0288, 0.0474, &#xA;0, 0.0078, 0.0914, 0.0289, 0.0293, 0.1067, 0, 0.0121, 0, 0, 0.018, &#xA;0, 0.021, 0.039, 0.0146, 0.043, 0.0367, 0.0245, 0, 0, 0, 0, 0, &#xA;0.0097, 0, 0.003, 0.0072, 0.017, 0.0061, 0.0453, 0, 0.0587, 0, &#xA;0, 0.1009, 0.0231, 0, 0.0279, 0, 0.048, 0.01, 0, 0.0083, 0.0355, &#xA;0, 0.1058, 0, 0, 0.0634, 0.0138, 0, 0.0116, 0, 0.0683, 0, 0, &#xA;0.0229, 0.064, 0, 0, 0, 0, 0.0222, 0.005, 0, 0.0176, 0, 0.0291, &#xA;0.0017, 0.008, 0.003, 0.0693, 0, 0.0156, 0.0038, 0.0577, 0, 0, &#xA;0, 0, 0.0521, 0, 0, 0.0017, 0, 0.0218, 0.1336, 0, 0, 0, 0, 0, &#xA;0, 0.0398, 0, 0.0099, 9e-04, 0.0046, 0.0055, 0.0361, 0.0382, &#xA;0.0168), .Dim = c(14L, 17L), .Dimnames = list(c(&quot;Cell1&quot;, &quot;Cell2&quot;, &#xA;&quot;Cell3&quot;, &quot;Cell4&quot;, &quot;Cell5&quot;, &quot;Cell6&quot;, &quot;Cell7&quot;, &quot;Cell8&quot;, &quot;Cell9&quot;, &#xA;&quot;Cell10&quot;, &quot;Cell11&quot;, &quot;Cell12&quot;, &quot;Cell13&quot;, &quot;Cell14&quot;), c(&quot;Sample1&quot;, &#xA;&quot;Sample2&quot;, &quot;Sample3&quot;, &quot;Sample4&quot;, &quot;Sample5&quot;, &quot;Sample6&quot;, &quot;Sample7&quot;, &#xA;&quot;Sample8&quot;, &quot;Sample9&quot;, &quot;Sample10&quot;, &quot;Sample11&quot;, &quot;Sample12&quot;, &quot;Sample13&quot;, &#xA;&quot;Sample14&quot;, &quot;Sample15&quot;, &quot;Sample16&quot;, &quot;Sample17&quot;)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Annotation of samples is in a dataframe &lt;code&gt;df&lt;/code&gt; look likes this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dput(df)&#xA;&#xA;structure(list(Samples = c(&quot;Sample2&quot;, &quot;Sample6&quot;, &quot;Sample7&quot;, &quot;Sample8&quot;, &#xA;&quot;Sample9&quot;, &quot;Sample10&quot;, &quot;Sample11&quot;, &quot;Sample16&quot;, &quot;Sample17&quot;, &quot;Sample1&quot;, &#xA;&quot;Sample3&quot;, &quot;Sample4&quot;, &quot;Sample5&quot;, &quot;Sample12&quot;, &quot;Sample13&quot;, &quot;Sample14&quot;, &#xA;&quot;Sample15&quot;), Group = c(&quot;A&quot;, &quot;A&quot;, &quot;A&quot;, &quot;A&quot;, &quot;A&quot;, &quot;A&quot;, &quot;A&quot;, &quot;A&quot;, &#xA;&quot;A&quot;, &quot;B&quot;, &quot;B&quot;, &quot;B&quot;, &quot;B&quot;, &quot;B&quot;, &quot;B&quot;, &quot;B&quot;, &quot;B&quot;)), row.names = c(NA, &#xA;-17L), class = c(&quot;tbl_df&quot;, &quot;tbl&quot;, &quot;data.frame&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using the above data and annotation with below code I made a heatmap:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Associated libraries&#xA;library(ComplexHeatmap)&#xA;library(RcolorBrewer)&#xA;library(circlize)&#xA;&#xA;#Set annotation&#xA;ann &amp;lt;- data.frame(df$Group)&#xA;colnames(ann) &amp;lt;- c(&quot;Group&quot;)&#xA;colours &amp;lt;- list(&quot;Group&quot;=c(&quot;A&quot;=&quot;orange&quot;,&quot;B&quot;=&quot;darkgreen&quot;))&#xA;colAnn &amp;lt;- HeatmapAnnotation(df=ann, which=&quot;col&quot;, col=colours, annotation_width=unit(c(1, 4), &quot;cm&quot;), gap=unit(1, &quot;mm&quot;))&#xA;&#xA;myCol &amp;lt;- colorRampPalette(c(&quot;#ffffff&quot;,&quot;#0030dd&quot;, &quot;#3259e3&quot;, &quot;#7f97ee&quot;))(100)&#xA;myBreaks &amp;lt;- seq(0,1, length=100)&#xA;hmap &amp;lt;- Heatmap(data, name = &quot;xCell scores&quot;, col = colorRamp2(myBreaks, myCol),&#xA;                width = unit(100, &quot;mm&quot;),show_row_names = TRUE, show_column_names = FALSE, cluster_rows = FALSE,&#xA;                cluster_columns = FALSE, show_column_dend = FALSE, show_row_dend = FALSE,&#xA;                row_dend_reorder = FALSE, column_dend_reorder = FALSE, clustering_method_rows = &quot;ward.D2&quot;,&#xA;                top_annotation=colAnn)&#xA;draw(hmap,heatmap_legend_side=&quot;right&quot;, annotation_legend_side=&quot;right&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The heatmap looks like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pNuk5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pNuk5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I want to sort the rows (cell1 to cell14) keeping the cell4 (interested cell) on the top and also keep the samples with values (coloured in blue in heatmap) together and samples with 0 values (white coloured) following that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should look something like the below heatmap which is from the Figure 4a in this &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/29456550&quot; rel=&quot;nofollow noreferrer&quot;&gt;research paper&lt;/a&gt; &lt;a href=&quot;https://i.stack.imgur.com/HxvnJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HxvnJ.png&quot; alt=&quot;heatmap&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you see in this heatmap all the coloured samples are together. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This must be done with some sorting and using transform, but not able to make it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is appreciated. thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="1369" LastEditorUserId="48" LastEditDate="2020-06-04T07:38:37.193" LastActivityDate="2020-06-04T07:38:37.193" Title="How to transform and sort the matrix to make a heatmap showing signatures?" Tags="|r|rna-seq|heatmap|figure-reproduction|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9157" PostTypeId="1" AcceptedAnswerId="9181" CreationDate="2019-08-05T18:47:57.490" Score="0" ViewCount="38" Body="&lt;p&gt;Im quoting the methods which was used &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We performed a bioinformatics analysis to identify potential CpG&#xA;  islands in the genomic sequences of RANKL and OPG genes. The USCS&#xA;  Human Genome Browser public database (genome.ucsc.edu), the Primer&#xA;  Express software (available from Applied Biosystems), and CpG Island&#xA;  Explorer software (cpgie.sourceforge.net) were used to explore the&#xA;  nucleo- tide sequences surrounding de transcription start site (TSS)&#xA;  of RANKL isoforms I and II (NG_008990.1 Ref Seq Gene), and OPG&#xA;  (NG_012202.1 Ref Seq Gene). A (C + G)/total bases ratio&lt;/p&gt;&#xA;  &#xA;  &lt;blockquote&gt;&#xA;    &lt;p&gt;0.5 and CpG observed/CpG expected ratio &gt;0.6 were used as criteria.&lt;/p&gt;&#xA;  &lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Im looking for the region which they have used &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the paper &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;Role of DNA methylation in the regulation of the&#xA;RANKL-OPG system in human bone&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried using ucsc to check the region but didn't get much. The idea is to find the region to design the primer for the same&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can i get the region from ucsc ,the CpG island around the TSS of the above mentioned genes which are RANKL and OPG&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion or help would be highly appreciated &lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2019-08-12T14:20:55.387" Title="DNA bioinformatic search and methylation analysis by qMSP" Tags="|methylation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9158" PostTypeId="2" ParentId="9155" CreationDate="2019-08-05T20:42:37.723" Score="6" Body="&lt;p&gt;You can't (easily) use a single design, since clone is nested within diabetic status, so you were correct using two separate designs. Your designs are correct.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really wanted to use a single design for some reason, it'd be &lt;code&gt;~clone+structure&lt;/code&gt; and for the latter comparison you'd end up using a rather complicated contrast (something like &lt;code&gt;c(rep(c(1,-1), 3), 0)&lt;/code&gt;). I really can't recommend that unless you have a very compelling reason, since it's vastly easier to think in terms of factors for your experiment and you're not hurting anything by fitting twice.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-08-05T20:42:37.723" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9159" PostTypeId="1" CreationDate="2019-08-06T15:02:43.420" Score="1" ViewCount="150" Body="&lt;p&gt;I will be very grateful for any hint on how to overcome the error. I wish to deconvolve my bulk RNA seq data obtained from the lungs of mice using single cell RNA seq data. For practice, I am following this &lt;a href=&quot;https://shenorrlab.github.io/bseqsc/vignettes/bseq-sc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; using my own bulk seq data and the &lt;a href=&quot;https://shenorrlab.github.io/bseqsc/data/islet-eset.rds&quot; rel=&quot;nofollow noreferrer&quot;&gt;single cell rna seq data from the pancreas&lt;/a&gt; used in the &lt;a href=&quot;https://shenorrlab.github.io/bseqsc/vignettes/bseq-sc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt;. All seems to go well until when I try to obtain the cell type-specific differential expression by running the following code. From the beginning I thought the presence of factors(e.g. for the treatment conditions or phenoData slot) in my data set was the cause but after getting rid of the factors I still get the same error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;csfit &amp;lt;- bseqsc_csdiff(the_best_ilc_bulk_expr_set[genes, ] ~ phenoData| alpha + beta + ductal + acinar,  &#xA;                       verbose = 2, nperms = 5000, .rng = 12345)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I get this error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Groups: bleo=5L | bleo_ko_i=5L | bleo_wt_i=5L | Nacl=5L | NA0L&#xA;Cell type(s): 'alpha', 'beta', ..., 'acinar' (4 total)&#xA;Fitting mode: auto&#xA;Data (filtered): 1202 features x 20 samples&#xA;Model has factor effect with more than 2 levels: fitting lm interaction model&#xA;Fitting model with nonnegative effects&#xA;Model with more than 2 groups: switching to version 2&#xA; Fitting linear interaction model ... Error in dimnames(covmat.unscaled) &amp;lt;- list(xnames, xnames) : &#xA;  length of 'dimnames' [1] not equal to array extent&#xA;In addition: Warning messages:&#xA;1: In lsfit(D, G, intercept = FALSE) : 'X' matrix was collinear&#xA;2: In sqrt(((n - p) * stddevmat^2 - resids^2/(1 - hatdiag[good]))/(n -  :&#xA;  NaNs produced&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here are the details of my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# inspired from https://github.com/cozygene/bisque/issues/4&#xA;&#xA;# creating expression set from my bulk rna seq data&#xA;&#xA;library(openxlsx)&#xA;library(Biobase)&#xA;library(bseqsc)&#xA;library(tidyverse)&#xA;bseqsc_config('CIBERSORT.R')&#xA;data(pancreasMarkers)&#xA;&#xA;&#xA;# read in the raw data&#xA;&#xA;bulk &amp;lt;- read.xlsx(&quot;all_samples.xlsx&quot;,&#xA;                  sheet = &quot;rawCounts_unrepeated_genes&quot;)&#xA;head(bulk)&#xA;&#xA;bulk&lt;span class=&quot;math-container&quot;&gt;$Gene &amp;lt;- NULL&#xA;rownames(bulk) &amp;lt;- toupper(bulk$&lt;/span&gt;external_gene_name)&#xA;bulk$external_gene_name &amp;lt;- NULL&#xA;bulk &amp;lt;- as.matrix(bulk)&#xA;head(bulk)&#xA;&#xA;&#xA;# create expression matrix &#xA;&#xA;featureData &amp;lt;- as.character(rownames(bulk))&#xA;featureData &amp;lt;- as(data.frame(featureData, &#xA;                             stringsAsFactors = FALSE), &quot;AnnotatedDataFrame&quot;)&#xA;rownames(featureData) &amp;lt;- rownames(bulk)&#xA;phenoData &amp;lt;- c(rep(&quot;bleo&quot;, 5), rep(&quot;Nacl&quot;, 5), rep(&quot;bleo_wt_i&quot;, 5), rep(&quot;bleo_ko_i&quot;, 5))&#xA;phenoData &amp;lt;- as(as.data.frame(phenoData), &quot;AnnotatedDataFrame&quot;)&#xA;rownames(phenoData) &amp;lt;- colnames(bulk)&#xA;the_best_ilc_bulk_expr_set &amp;lt;- ExpressionSet(assayData = bulk, &#xA;                                            phenoData = phenoData, featureData = featureData)&#xA;&#xA;# read in the single cell rna seq data and fit the model&#xA;&#xA;eislet &amp;lt;- readRDS('islet-eset.rds')&#xA;B &amp;lt;- bseqsc_basis(eislet, pancreasMarkers, &#xA;                  clusters = 'cellType', samples = 'sampleID', ct.scale = TRUE)&#xA;fit &amp;lt;- bseqsc_proportions(the_best_ilc_bulk_expr_set, B, verbose = TRUE)&#xA;&#xA;&#xA;pData(the_best_ilc_bulk_expr_set) &amp;lt;- cbind(pData(the_best_ilc_bulk_expr_set), t(coef(fit)))&#xA;&#xA;fit_edger&amp;lt;- fitEdgeR(the_best_ilc_bulk_expr_set, ~phenoData, &#xA;                     coef = c(&quot;phenoDatableo_ko_i&quot;, &quot;phenoDatableo_wt_i&quot;, &quot;phenoDataNacl&quot;))&#xA;&#xA;# extended&#xA;&#xA;fit_edger_ext &amp;lt;- fitEdgeR(the_best_ilc_bulk_expr_set, ~ phenoData + beta + ductal + acinar +gamma,&#xA;                          coef = c(&quot;phenoDatableo_ko_i&quot;, &quot;phenoDatableo_wt_i&quot;,&#xA;                                   &quot;phenoDataNacl&quot;, &quot;beta&quot;, &quot;ductal&quot;, &#xA;                                   &quot;acinar&quot;, &quot;gamma&quot;))&#xA;&#xA;&#xA;fit_edger_ext$Symbol &amp;lt;- rownames(fit_edger_ext)&#xA;&#xA;# gather P-values from both models&#xA;&#xA;df_fit_edger &amp;lt;- as.data.frame(fit_edger, stringsAsFactors = FALSE)&#xA;df_fit_edger$Symbol &amp;lt;- rownames(df_fit_edger)&#xA;&#xA;req_df_fit_edger &amp;lt;- df_fit_edger[&quot;PValue&quot;]&#xA;colnames(req_df_fit_edger) &amp;lt;- &quot;Base&quot;&#xA;head(req_df_fit_edger)&#xA;req_df_fit_edger&lt;span class=&quot;math-container&quot;&gt;$Symbol &amp;lt;- rownames(req_df_fit_edger)&#xA;df_fit_edger_ext &amp;lt;- as.data.frame(fit_edger_ext, stringsAsFactors = FALSE)&#xA;df_fit_edger_ext$&lt;/span&gt;Symbol &amp;lt;- rownames(df_fit_edger_ext)&#xA;&#xA;head(df_fit_edger_ext)&#xA;&#xA;req_df_fit_edger_ext &amp;lt;- df_fit_edger_ext[&quot;PValue&quot;]&#xA;colnames(req_df_fit_edger_ext) &amp;lt;- &quot;Adjusted&quot;&#xA;req_df_fit_edger_ext$Symbol &amp;lt;- rownames(req_df_fit_edger_ext)&#xA;&#xA;&#xA;edger_pvals &amp;lt;- req_df_fit_edger_ext %&amp;gt;%&#xA;  inner_join(req_df_fit_edger)&#xA;head(edger_pvals)#&#xA;&#xA;rownames(edger_pvals) &amp;lt;- edger_pvals$Symbol&#xA;&#xA;&#xA;&#xA;edger_pvals &amp;lt;- mutate(edger_pvals, Regulated = Adjusted &amp;lt;= 0.01 &amp;amp; Adjusted &amp;lt;= Base)&#xA;&#xA;# plot Base vs Adjusted&#xA;pvalueScatter(Base ~ Adjusted, edger_pvals, pval.th = 0.01, label.th = 3.5)&#xA;&#xA;# ER genes&#xA;genes_ER &amp;lt;- c('HSPA5', 'MAFA', 'HERPUD1', 'DDIT3', 'UCN3', 'NEUROD1')&#xA;# Fit on ER stress genes and genes regulated beyond cell type proportion differences&#xA;genes &amp;lt;- union(genes_ER, subset(edger_pvals, Regulated)$Symbol)&#xA;&#xA;csfit &amp;lt;- bseqsc_csdiff(the_best_ilc_bulk_expr_set[genes, ] ~ phenoData| alpha + beta + ductal + acinar,  &#xA;                       verbose = 2, nperms = 5000, .rng = 12345)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I have the error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Groups: bleo=5L | bleo_ko_i=5L | bleo_wt_i=5L | Nacl=5L | NA0L&#xA;Cell type(s): 'alpha', 'beta', ..., 'acinar' (4 total)&#xA;Fitting mode: auto&#xA;Data (filtered): 1202 features x 20 samples&#xA;Model has factor effect with more than 2 levels: fitting lm interaction model&#xA;Fitting model with nonnegative effects&#xA;Model with more than 2 groups: switching to version 2&#xA; Fitting linear interaction model ... Error in dimnames(covmat.unscaled) &amp;lt;- list(xnames, xnames) : &#xA;  length of 'dimnames' [1] not equal to array extent&#xA;In addition: Warning messages:&#xA;1: In lsfit(D, G, intercept = FALSE) : 'X' matrix was collinear&#xA;2: In sqrt(((n - p) * stddevmat^2 - resids^2/(1 - hatdiag[good]))/(n -  :&#xA;  NaNs produced&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Session info:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; sessionInfo()&#xA;R version 3.6.0 (2019-04-26)&#xA;Platform: x86_64-pc-linux-gnu (64-bit)&#xA;Running under: Ubuntu 18.04.2 LTS&#xA;&#xA;Matrix products: default&#xA;BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1&#xA;LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1&#xA;&#xA;locale:&#xA; [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=de_DE.UTF-8        LC_COLLATE=en_US.UTF-8    &#xA; [5] LC_MONETARY=de_DE.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=de_DE.UTF-8       LC_NAME=C                 &#xA; [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C       &#xA;&#xA;attached base packages:&#xA;[1] parallel  stats     graphics  grDevices utils     datasets  methods   base     &#xA;&#xA;other attached packages:&#xA; [1] edgeR_3.26.6          limma_3.40.6          preprocessCore_1.46.0 e1071_1.7-2           forcats_0.4.0         stringr_1.4.0        &#xA; [7] dplyr_0.8.3           purrr_0.3.2           readr_1.3.1           tidyr_0.8.3           tibble_2.1.3          ggplot2_3.2.0        &#xA;[13] tidyverse_1.2.1       bseqsc_1.0            csSAM_1.4             Rcpp_1.0.2            Biobase_2.44.0        BiocGenerics_0.30.0  &#xA;[19] openxlsx_4.1.0.1     &#xA;&#xA;loaded via a namespace (and not attached):&#xA; [1] nlme_3.1-140         lubridate_1.7.4      bit64_0.9-7          doParallel_1.0.14    RColorBrewer_1.1-2   httr_1.4.0          &#xA; [7] tools_3.6.0          backports_1.1.4      R6_2.4.0             DBI_1.0.0            lazyeval_0.2.2       colorspace_1.4-1    &#xA;[13] withr_2.1.2          tidyselect_0.2.5     gridExtra_2.3        xbioc_0.1.17         bit_1.1-14           compiler_3.6.0      &#xA;[19] cli_1.1.0            rvest_0.3.4          xml2_1.2.1           pkgmaker_0.28        scales_1.0.0         NMF_0.22            &#xA;[25] digest_0.6.20        pkgconfig_2.0.2      bibtex_0.4.2         rlang_0.4.0          readxl_1.3.1         rstudioapi_0.10     &#xA;[31] RSQLite_2.1.2        generics_0.0.2       jsonlite_1.6         dendextend_1.12.0    zip_2.0.3            magrittr_1.5        &#xA;[37] Formula_1.2-3        munsell_0.5.0        S4Vectors_0.22.0     viridis_0.5.1        stringi_1.4.3        yaml_2.2.0          &#xA;[43] plyr_1.8.4           grid_3.6.0           blob_1.2.0           crayon_1.3.4         lattice_0.20-38      haven_2.1.1         &#xA;[49] splines_3.6.0        hms_0.5.0            locfit_1.5-9.1       zeallot_0.1.0        pillar_1.4.2         rngtools_1.4        &#xA;[55] reshape2_1.4.3       codetools_0.2-16     stats4_3.6.0         glue_1.3.1           BiocManager_1.30.4   modelr_0.1.4        &#xA;[61] vctrs_0.2.0          foreach_1.4.7        cellranger_1.1.0     gtable_0.3.0         assertthat_0.2.1     gridBase_0.4-7      &#xA;[67] xtable_1.8-4         broom_0.5.2          class_7.3-15         viridisLite_0.3.0    iterators_1.0.12     AnnotationDbi_1.46.0&#xA;[73] registry_0.5-1       memoise_1.1.0        IRanges_2.18.1       cluster_2.1.0   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3968" LastEditorUserId="48" LastEditDate="2019-08-07T11:33:04.923" LastActivityDate="2019-11-07T10:21:34.153" Title="Error using bseqsc" Tags="|r|rna-seq|single-cell|clustering|machine-learning|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9160" PostTypeId="1" CreationDate="2019-08-07T08:14:55.883" Score="0" ViewCount="63" Body="&lt;p&gt;I have performed GWAS and got some significant SNPs. How can I get candidates genes in the flanking regions of those SNPs and also their functional annotations (eg. KEGG). &lt;/p&gt;&#xA;" OwnerUserId="5023" LastEditorUserId="678" LastEditDate="2019-08-07T11:47:06.700" LastActivityDate="2019-08-07T11:47:06.700" Title="significant SNPs to annotated candidate genes" Tags="|snp|gwas|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9161" PostTypeId="2" ParentId="9160" CreationDate="2019-08-07T08:38:19.983" Score="2" Body="&lt;p&gt;You can use &lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/set-operations/closest-features.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS' closest-features&lt;/a&gt; tool. You only need to have the SNPs coordinates in .bed format and a .bed file with gene locations. BEDOPS will look for the closest genes upstream and downstream of your SNPs. The command will look something like this: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;closest-features yourSNPs.bed yourAnnotation.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, you can obtain your functional annotations through different tools such as biomart. &lt;/p&gt;&#xA;" OwnerUserId="678" LastActivityDate="2019-08-07T08:38:19.983" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9162" PostTypeId="2" ParentId="21" CreationDate="2019-08-07T10:19:05.123" Score="1" Body="&lt;p&gt;The UCSC Genome Browser Genes FAQ discusses this question in detail:&#xA;&lt;a href=&quot;https://genome.ucsc.edu/FAQ/FAQgenes.html#ens&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genome.ucsc.edu/FAQ/FAQgenes.html#ens&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Officially, the Ensembl and GENCODE gene models are the same. On the&#xA;  latest human and mouse genome assemblies (hg38 and mm10), the&#xA;  identifiers, transcript sequences, and exon coordinates are almost&#xA;  identical between equivalent Ensembl and GENCODE versions (excluding&#xA;  alternative sequences or fix sequences).&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;GENCODE uses the UCSC convention of prefixing chromosome names with&#xA;  &quot;chr&quot;, e.g. &quot;chr1&quot; and &quot;chrM&quot;, but Ensembl calls these &quot;1&quot; or &quot;MT&quot;. At&#xA;  the time of writing (Ensembl 89), a few transcripts differ due to&#xA;  conversion issues. In addition, around 160 PAR genes are duplicated in&#xA;  GENCODE but only once in Ensembl. The differences affect fewer than 1%&#xA;  of the transcripts. Apart from gene annotation itself, the links to&#xA;  external databases differ.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;The GENCODE Release History shows the release dates and can be linked&#xA;  to corresponding Ensembl releases. You can download the gene&#xA;  transcript models from the website &lt;a href=&quot;https://gencodegenes.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gencodegenes.org&lt;/a&gt; or from&#xA;  &lt;a href=&quot;http://ensembl.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ensembl.org&lt;/a&gt;. For most applications, the files distributed on&#xA;  the GENCODE website should be easier to use, as the third party&#xA;  database links are easier to parse and the sequence identifiers match&#xA;  the UCSC genome files, at least for the primary chromosomes.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Additional information on this question can be found on the GENCODE&#xA;  FAQ page.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="5024" LastActivityDate="2019-08-07T10:19:05.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9163" PostTypeId="1" AcceptedAnswerId="9169" CreationDate="2019-08-07T12:12:18.010" Score="0" ViewCount="113" Body="&lt;p&gt;As stated, which one is better for differential expression analysis? When I say normal pipeline I mean limma-voom, edgeR and DESeq2 pipeline for standard analysis. Kal's z test is mentioned in this &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/10359602&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; and I know some pre-made tools like the one on QIAgen uses this as well for single vs single comparison&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The situation I have is that I have RNA-Seq data for some patients with rare translocations in leukaemia. There are two of these translocation with almost 10 or even more patients so I have no problem with them. And then there are these translocation which mostly I have only 1, some with 2 or 3 patients, for 7 different translocations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Our hypothesis is that the two larger group can certainly form their own subtypes, and the others may fall into one of the three cases: (1) belongs to one of the two groups despite of the difference in translocation; (2) belongs to a separate group on their own i.e. complete outlier; (3) belongs to a separate group with other members in the &quot;others&quot; i.e. forming a third subtype. And this can be tested with clustering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For that, I need to do supervised feature selection to filter out useless genes. I can do that with those with multiple samples, but not those isolated cases. So I thought I can get some DGE analysis and use a certain adj.p and fold change as cut off, or adj.p as ranking matrix, and select those genes for clustering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my main questions are:&#xA;(1) when using limma/edgeR/DESeq2, should the model design include all the translocation labels in one go or should I just add the one contrast I want to make and build a new model everytime?&#xA;(2) Is Kal's z test useful for one vs all DGE analysis and is it better?&#xA;(3) Are there other outlier analysis working better under this situation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many thanks!&lt;/p&gt;&#xA;" OwnerUserId="3225" LastActivityDate="2019-08-08T07:53:41.533" Title="Single sample in group: normal pipeline or Kal's Z test" Tags="|differential-expression|deseq2|edger|limma|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9165" PostTypeId="1" CreationDate="2019-08-07T20:35:04.530" Score="1" ViewCount="20" Body="&lt;p&gt;I am looking for a way to use the &lt;a href=&quot;https://sparql.rhea-db.org/sparql&quot; rel=&quot;nofollow noreferrer&quot;&gt;SPARQL endpoint of RHEA&lt;/a&gt; to download all reactions of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Citric_acid_cycle&quot; rel=&quot;nofollow noreferrer&quot;&gt;TCA cycle&lt;/a&gt; in CSV or JSON format. &lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="1157" LastEditDate="2020-03-03T01:17:30.910" LastActivityDate="2020-03-03T01:17:30.910" Title="How to download all reactions of the TCA cycle from the RHEA database with the SPARQL endpoint?" Tags="|api|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9168" PostTypeId="2" ParentId="7334" CreationDate="2019-08-08T06:34:04.827" Score="1" Body="&lt;p&gt;The STRING download page is &lt;a href=&quot;https://string-db.org/cgi/download.pl?sessionId=Mi0vHkfUDOEu&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I am not sure that they provide any data in the form of an adjacency matrix, as the number of nodes (rows/columns) would be quite large, and for this reason they are likely to represent the data as lists of links, for compactness. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Their data will likely have to be cleaned for any purpose you are interested in. You will have to make some decisions on how to filter the data. For example, as suggested in the comments, you may want to filter on the quality of the evidence supporting interactions, or based on species of the protein, or other things.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's possible that you may have better luck going with another database that has done some of these steps already. For example, there are relatively compact human interactome datasets &lt;a href=&quot;http://interactome.baderlab.org/download&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, or yeast interactome datasets &lt;a href=&quot;http://interactome.dfci.harvard.edu/S_cerevisiae/index.php?page=download&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. In this case you will still need to convert from a list of links to an adjacency matrix, however this is not very difficult if you can use a scripting language. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notably, you may have to figure out how to distinguish missing data from tested non-interactions in these cases (or decide that missing data is ok). In most cases people will not have exhaustively tested every cell of the all-by-all interactome adjacency matrix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2019-08-08T06:34:04.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9169" PostTypeId="2" ParentId="9163" CreationDate="2019-08-08T07:53:41.533" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;You'll have a bit more power if you use all of the samples at once.&lt;/li&gt;&#xA;&lt;li&gt;Nothing is useful for this, don't waste your time with it.&lt;/li&gt;&#xA;&lt;li&gt;You're not looking for outliers, you're looking to see how samples cluster. You can just use correlation for that.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-08-08T07:53:41.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9170" PostTypeId="1" CreationDate="2019-08-08T16:59:57.170" Score="1" ViewCount="60" Body="&lt;p&gt;I checked EMBOSS but there is no tabular output for pairwise alignment. &#xA;I can write a script but would like to know if there is already a tool to summarize an alignment in &lt;a href=&quot;http://emboss.sourceforge.net/docs/themes/AlignFormats.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;srspair&lt;/a&gt; format in the form of a table (&lt;code&gt;merger&lt;/code&gt; from the EMBOSS collection gives a tabular summary of the differences but &lt;code&gt;needle&lt;/code&gt; doesn't).&lt;/p&gt;&#xA;" OwnerUserId="345" LastActivityDate="2019-08-08T16:59:57.170" Title="How to get a tabular summary from srspair alignment file?" Tags="|sequence-alignment|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9171" PostTypeId="1" AcceptedAnswerId="9174" CreationDate="2019-08-08T17:49:44.347" Score="0" ViewCount="256" Body="&lt;p&gt;I have a lists of DEGs identified by DESeq2, EdgeR and Limma.&#xA;I would like to correlate the the gene rankings in the lists to decide on a package to use in downstream analysis.&#xA;I am havig a few problems with this, would anyone be able to help?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="5036" LastActivityDate="2019-08-09T10:01:02.747" Title="Correlate DEGs from DESeq2, EdgeR and Limma results" Tags="|rna-seq|deseq2|edger|limma|correlation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9172" PostTypeId="1" AcceptedAnswerId="9173" CreationDate="2019-08-08T19:20:48.987" Score="2" ViewCount="633" Body="&lt;p&gt;I'm a novice trying to wrap my head around with fancy jargons. If I search residual, I understand that it represents the difference between the observed value of the dependent variable (y) and the predicted value (ŷ).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm unable to follow then, how are these residuals used as new phenotype?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found similar question in the forum:  &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2463/adjusting-phenotypes-by-regressing-out-covariates&quot;&gt;Adjusting phenotypes by regressing out covariates&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The origin of my question is predixcan pipeline, where residuals are used as the new gene expression values after regressing input values on covariates.  &lt;/p&gt;&#xA;" OwnerUserId="862" LastActivityDate="2019-08-09T05:35:43.093" Title="How are residuals used as new phenotype?" Tags="|statistics|linear-regression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9173" PostTypeId="2" ParentId="9172" CreationDate="2019-08-09T03:29:51.637" Score="1" Body="&lt;p&gt;The residual is the level of error in a regression model, the lower the residual the better the model. Residuals cannot equate to a phenotype, it is actually the opposite if your regression model is trying to investigate a given genotype/phenotype(s) model. The residual in this case is the amount of infomation which the genotype/phentype does not account for, i.e. there are other factors resulting in your observations which you have not captured in your model. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I assume is that the regression model you are referring  is  investigating whether the residual is lowered by examining different gene combinations. The objective of a regression model is to lower the residual and I suspect that it is comparing the new residual against the old one. Therefore it is not being used as an input into a regression model but an assessment of its robustness.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-08-09T05:35:43.093" LastActivityDate="2019-08-09T05:35:43.093" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9174" PostTypeId="2" ParentId="9171" CreationDate="2019-08-09T10:01:02.747" Score="1" Body="&lt;p&gt;I would use the p-value/FDR which each method returns to rank the gene list for that method in order from 'most likely to be DE' to least likely.  You would then have three ranked lists of genes - you would put the actual rankings in for each gene - 1 for the top ranked, 2 for the next, etc.  You could then carry out, say Spearman's rank correlation on the lists of rankings to assess agreement between the methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You could also decide to keep all 3 methods, and accept genes as DE if, for example, at least 2 out of 3 methods give an acceptable p-value/FDR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my experience, correlation should be reasonably high at the top of the list, where the methods tend to agree more on the rankings, and when the correlation gets lower, further down the list, would be where I would start to question whether the genes are really differentially expressed.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-08-09T10:01:02.747" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9175" PostTypeId="1" AcceptedAnswerId="16244" CreationDate="2019-08-09T12:03:56.320" Score="5" ViewCount="513" Body="&lt;p&gt;I'm currently analyzing two samples of eosinophil cells isolated from mouse lung and the samples are of very different quality.&lt;br /&gt;&#xA;According to the Cell Ranger summary 56% of the reads can be mapped to the transcriptome in the first sample and only 32% in the second sample. Also the number of cells identified in Cell Ranger are very different (5000 for sample 1 and 600 for sample 2).&lt;/p&gt;&#xA;&lt;p&gt;When I look at the raw data in FastQC I see that the second sample has a lot of poly-A sequences in the R2 read that obviously can't be mapped to the transcriptome.&lt;/p&gt;&#xA;&lt;p&gt;Sample 1:&lt;br /&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ElEFk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ElEFk.png&quot; alt=&quot;per base sequence content sample 1&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Sample 2:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/nogKK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nogKK.png&quot; alt=&quot;per base sequence content sample 2&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Also in many cases there is the following sequence in front of the poly-A:&lt;br /&gt;&#xA;&lt;code&gt;AAGCAGTGGTATCAACGCAGAGTACATGGG&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;To me it looks as if most of the reads do not contain an insert and are primer dimers. Is this connected to the lower cell count in the second sample?&lt;br /&gt;&#xA;Maybe some of you have already seen something like this and have a suggestion on how to avoid such a problem in the future.&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot&lt;/p&gt;&#xA;" OwnerUserId="4885" LastEditorUserId="4885" LastEditDate="2021-06-28T13:45:00.333" LastActivityDate="2021-06-28T13:48:20.520" Title="High percentage of poly A sequences in 10X chromium R2 read" Tags="|scrnaseq|ngs|10x-genomics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9176" PostTypeId="1" CreationDate="2019-08-09T19:43:04.423" Score="2" ViewCount="297" Body="&lt;p&gt;I have an optimization problem where I have a degenerate nucleotide sequence I want to align against subsets of a reference genome (exons, specifically, to make the problem more tractable). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sequence itself contains subregions of variable length.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sequence looks like this, containing degenerate bases:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;           L           C           R&#xA;5' - SSS......SSS|NNN.....NNN|WWW......WWW - 3'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Per IUPAC notation, &lt;code&gt;S&lt;/code&gt; denotes &lt;code&gt;C&lt;/code&gt; or &lt;code&gt;G&lt;/code&gt;, &lt;code&gt;N&lt;/code&gt; denotes any base, and &lt;code&gt;W&lt;/code&gt; denotes &lt;code&gt;A&lt;/code&gt; or &lt;code&gt;T&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am marking each block &lt;code&gt;L&lt;/code&gt;, &lt;code&gt;C&lt;/code&gt;, and &lt;code&gt;R&lt;/code&gt; to broadly denote &quot;left&quot;, &quot;center&quot;, and &quot;right&quot; regions, oriented from 5' to 3' (&quot;forward&quot;-strand oriented).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The widths of the &lt;code&gt;L&lt;/code&gt; and &lt;code&gt;R&lt;/code&gt; blocks can be from 20 to 25 bases long. The width of the &lt;code&gt;C&lt;/code&gt; block can be 14 to 16 bases long.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One programmatic way I could think to do this would be to brute-force a solution, generating all combinations of sequences of varied lengths, &lt;code&gt;blastn&lt;/code&gt;-aligning each of such combinations to the sequence for the exon, and then looking for the best alignments among such sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output would be the scores for regions from sequences that align, to be able to say, from point X to point Y of the genome, a sequence here aligns better than other candidate sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To avoid reinventing tools, and avoid brute-force methods, I'd like to know if there is a tool or even a wrapper around tools, which does this, in case it is already written, or another approach that is more elegant/efficient.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2019-08-12T08:48:41.763" Title="Appropriate tool or algorithm for sloppy alignment of degenerate bases" Tags="|sequence-alignment|blast|sequence|blastn|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9177" PostTypeId="1" AcceptedAnswerId="9179" CreationDate="2019-08-09T19:48:53.947" Score="2" ViewCount="548" Body="&lt;p&gt;I’m trying to develop a table from a series of lists generated using the Immunarch library to process TCR sequencing results. Each list is comprised of CDR3.aa (clone) information which are character strings and their count in a particular sample. The clones are short but vary between 7 and 20 (or more).  Each list has a header that identifies the sample sequenced. I have a list of 66 samples. Each sample can have several thousand clone strings. Not every clone is contained in every sample so the number of clones listed in the samples varies.  Here’s an example of the structure of a single sample list.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$PreAg_18_2&#xA;CDR3.aa           Count&#xA;CASSYGTAYTGELFF   1623&#xA;CASSRGDSDNSPLHF   1440&#xA;CASSREKAFF        1161&#xA;CSGMGALAKNIQYF     949&#xA;CSAYTGLSYEQYF      813&#xA;CASSLSLAVNSPLHF    634&#xA;CAIRDTPGSPQHF      574&#xA;CATGQVNTEAFF       555&#xA;CASSLKGQGGSPLHF    499&#xA;CASSYSRSPQPQHF     478&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to combine the results in a single table showing the clone counts with all the clone strings listed on the y-axis and the sample ID listed on the x-axis. For example:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                10_pep_10_1     preAg_10_2      Dec_2_18_1  …... &#xA;CASSYGTAYTGELFF    1623         234             0&#xA;CASSRGDSDNSPLHF    1440         522             28&#xA;CASSREKAFF         1161         445             50  &#xA;CSGMGALAKNIQYF      949         24              0&#xA;CASSYSRSPQPQHF      478         0               398&#xA;.&#xA;.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Currently, I'm working with R because that's what Immunarch is in, but I wonder if a) There's a better way to do this in Python, or b) If it's doable in r, then how do I generate the matrix matches.  The main problem is that not all the samples have the same number of clones, so I first have to separate each sample data before I can convert it to a data frame, and then how can I match the actual count with the actual clone.  I can't just merge the data frames.  Any suggestion is greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's what i have for code right now.&#xA;Code:&#xA;    library(immunarch)&#xA;    library(stringr)&#xA;    library(plyr)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;immdata = repLoad(&quot;/mnt/data/Development/Analysis_Scripts/input_files/&quot;)&#xA;&#xA;all &amp;lt;- immdata$data&#xA;&#xA;# Get list headers (names) and convert to df&#xA;sample.id &amp;lt;- names(all)&#xA;sample.id &amp;lt;- data.frame(sample.id)&#xA;&#xA;# Get list of clones and filter for unique clones per list.&#xA;for (i in 1:length(all)){&#xA;    all[[i]]$Sample.ID&amp;lt;-names(all)[i]&#xA;    all[[i]]&amp;lt;-all[[i]][,c(&quot;CDR3.aa&quot;, &quot;Clones&quot;)]&#xA;}&#xA;&#xA;# all is the variable that contains the list of samples and clones.&#xA;all.split &amp;lt;- split(all, )&#xA;&#xA;# make vector of all clones&#xA;all.clones &amp;lt;- unlist(all, use.names=FALSE)&#xA;&#xA;# Removes clone repeats&#xA;all.clones.u &amp;lt;- unique(all.clones)&#xA;&#xA;# convert list of clones to data frame&#xA;all.clones.u &amp;lt;- data.frame(all.clones.u)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;At this point I have both the list of sample ids and the list of clones.  My problem is now matching each count with respective sample and  clone in the table.  Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="2621" LastActivityDate="2023-01-25T21:45:14.377" Title="Creating a count table using R" Tags="|r|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9178" PostTypeId="2" ParentId="9176" CreationDate="2019-08-10T13:40:54.617" Score="1" Body="&lt;p&gt;I believe mafft can align degenerate sequences.  From the documentation for version 7 at: &lt;a href=&quot;https://mafft.cbrc.jp/alignment/software/anysymbol.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://mafft.cbrc.jp/alignment/software/anysymbol.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Acceptable nucleotide symbols in the default mode:&#xA;a, c, g, t, u&#xA;Ambiguous nucleotides (r, y, w, s, k, m, d, v, h, b; IUPAC-IUB codes) can be used and are scored as:&#xA;score(r,a) = ( score(a,a) + score(g,a) ) / 2 &#xA;score(r,g) = ( score(a,g) + score(g,g) ) / 2 &#xA;score(r,t) = ( score(a,t) + score(g,t) ) / 2 &#xA;score(r,c) = ( score(a,c) + score(g,c) ) / 2 &#xA;... &#xA;score(r,y) = ( score(r,t) + score(r,c) ) / 2 &#xA;... &#xA;score(r,r) = ( score(a,a) + score(g,g) ) / 2 &#xA;score(y,y) = ( score(c,c) + score(t,t) ) / 2 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;MAFFT is designed for multiple sequence alignment, so will attempt to find a local alignment of homologous segments (using FFT), and then attempt to make a global alignment between the input sequences.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastEditorUserId="3828" LastEditDate="2019-08-12T08:48:41.763" LastActivityDate="2019-08-12T08:48:41.763" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9179" PostTypeId="2" ParentId="9177" CreationDate="2019-08-10T20:35:54.313" Score="4" Body="&lt;p&gt;You could give a try with dplyr's &lt;code&gt;full_join()&lt;/code&gt; and purrr's &lt;code&gt;reduce()&lt;/code&gt;. &lt;code&gt;full_join()&lt;/code&gt; will make sure that you won't lose any data and &lt;code&gt;reduce()&lt;/code&gt; will merge tables in your list from left to right, left being the first table of the list in the first iteration but would become merged &lt;em&gt;table1+table2&lt;/em&gt; in the second iteration and so on. This left to right &quot;reduction&quot; will also be helpful when renaming the columns of the resulting table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Without your workspace I have attempted to replicate your data: There is a list of 3 data frames, each with 2 columns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(dplyr)&#xA;library(purrr)&#xA;&#xA;df1 &amp;lt;- data.frame(CDR3.aa = c(&quot;A&quot;, &quot;B&quot;, &quot;C&quot;),&#xA;                  Count = c(10,20,100))&#xA;&#xA;df2 &amp;lt;- data.frame(CDR3.aa = c(&quot;A&quot;, &quot;B&quot;, &quot;D&quot;),&#xA;                  Count = c(100,200,300))&#xA;&#xA;df3 &amp;lt;- data.frame(CDR3.aa = c(&quot;D&quot;, &quot;E&quot;, &quot;F&quot;),&#xA;                  Count = c(10,20,100))&#xA;&#xA;my_list &amp;lt;- list(df1 = df1, df2 = df2, df3 = df3)&#xA;&#xA;&amp;gt; my_list&#xA;$df1&#xA;  CDR3.aa Count&#xA;1       A    10&#xA;2       B    20&#xA;3       C   100&#xA;&#xA;$df2&#xA;  CDR3.aa Count&#xA;1       A   100&#xA;2       B   200&#xA;3       D   300&#xA;&#xA;$df3&#xA;  CDR3.aa Count&#xA;1       D    10&#xA;2       E    20&#xA;3       F   100&#xA;&#xA;# &quot;by&quot; argument specifies which column to use when merging&#xA;my_table &amp;lt;- reduce(my_list, full_join, by = &quot;CDR3.aa&quot;)&#xA;&#xA;names(my_table) &amp;lt;- c(&quot;CDR3.aa&quot;, names(my_list))&#xA;&#xA;&amp;gt; my_table&#xA;  CDR3.aa df1 df2 df3&#xA;1       A  10 100  NA&#xA;2       B  20 200  NA&#xA;3       C 100  NA  NA&#xA;4       D  NA 300  10&#xA;5       E  NA  NA  20&#xA;6       F  NA  NA 100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Clonotypes that are not present in table will be &quot;NA&quot;.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-08-10T20:40:57.357" LastActivityDate="2019-08-10T20:40:57.357" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9180" PostTypeId="2" ParentId="5176" CreationDate="2019-08-12T11:30:55.897" Score="1" Body="&lt;p&gt;To give some formal words for what you are doing, I would say this sounds like you are trying to reconstruct ancestral states.  There has been some work done on this, and a similar algorithm to yours was devised by Fitch in the 1970s: Fitch W: Toward defining the course of evolution: minimum change for a specific tree topology. Syst Zool 1971, 20: 406–416. 10.2307/2412116&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The missing part, from your approach, is that once Fitch gets to the root of the tree, he than retraverses back to the leaves, resolving ambiguities along the way (if two sister nodes have different states, how do you decide what the parental state was? Fitch's idea is that once you get to the root, you can come back down and use the parental state to set the child state in ambiguous cases).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a nice implementation of Fitch's algorithm in the R phangorn package, which makes the reconstruction a one-liner (given a rooted tree object, and a matrix of leaf node states):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;anc.mpr = ancestral.pars(rooted_tree, tree_samples_data_matrix.phyDat, &quot;MPR&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you don't want to use the R package itself, you could perhaps use it as pseudocode to port into python.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastEditorUserId="3828" LastEditDate="2019-08-12T14:41:22.503" LastActivityDate="2019-08-12T14:41:22.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9181" PostTypeId="2" ParentId="9157" CreationDate="2019-08-12T14:20:55.387" Score="1" Body="&lt;p&gt;It's likely that they accomplished this by using the &lt;a href=&quot;http://genome.ucsc.edu/cgi-bin/hgTables&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC Table Browser&lt;/a&gt; to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Find the loci of the two isoforms you mention&lt;/li&gt;&#xA;&lt;li&gt;Finding CpG islands (Group = Regulation, Track = CpG Islands) near those transcription start sites&lt;/li&gt;&#xA;&lt;li&gt;Design primers using Applied Biosystem's software&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Alternatively, they list the exact primers used in &lt;a href=&quot;https://www.tandfonline.com/action/downloadTable?id=T1&amp;amp;doi=10.4161%2Fepi.7.1.18753&amp;amp;downloadType=CSV&quot; rel=&quot;nofollow noreferrer&quot;&gt;Table 1&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They don't detail exactly how they did it, so if you have specific questions it might be worth it to email the corresponding author&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2019-08-12T14:20:55.387" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9183" PostTypeId="2" ParentId="5176" CreationDate="2019-08-13T00:57:07.850" Score="0" Body="&lt;p&gt;The OP could use the Rpy2 library to connect Python to R, but it is often better to use R directly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The alternative is to simply use MacClade 4.0, although you will need an old Mac. This is a GUI interface that will allow you to identfy the parsimonious character state for each node in the phylogeny. The API allows you to interactively explore the phylogeny by point and click moving the branches around the tree. The tree will then be assessed in terms of &quot;number character steps&quot; and automatically calculate this for each topology loaded. The output is very colourful so its easy to understand a large data set.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-08-13T00:57:07.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9184" PostTypeId="1" CreationDate="2019-08-13T04:03:01.060" Score="2" ViewCount="177" Body="&lt;p&gt;I have two .txt files and let's call it '&lt;strong&gt;File A&lt;/strong&gt;' and '&lt;strong&gt;File B&lt;/strong&gt;.Here are examples of what file A and file B looks like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;FILE A&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Chr    Start   End Ref Alt Phenotypes&#xA;1   1000    1000    A   T   Pheno A&#xA;2   2000    2000    T   G   Pheno F, Pheno G&#xA;3   3000    3000    C   G   Pheno L&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;FILE B&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Chr    Start   End Ref Alt Phenotypes&#xA;1   1000    1000    A   T   Pheno B&#xA;2   2000    2000    T   G   Pheno H, Pheno I&#xA;3   3000    3000    C   G   Pheno M&#xA;3   3000    3000    C   G   Pheno N&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's how I want the output to look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Chr    Start   End Ref Alt Phenotypes&#xA;1   1000    1000    A   T   Pheno A, Pheno B&#xA;2   2000    2000    T   G   Pheno F, Pheno G, Pheno H, Pheno I&#xA;3   3000    3000    C   G   Pheno L, Pheno M, Pheno N&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Both are tab-separated .txt files and I am trying to merge them together for SNP annotation and would love to hear suggestions on how to proceed this. &lt;/p&gt;&#xA;" OwnerUserId="5050" LastEditorUserId="298" LastEditDate="2019-08-14T18:24:32.003" LastActivityDate="2019-08-15T05:11:23.097" Title="Merging two .txt files" Tags="|sequence-annotation|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9185" PostTypeId="2" ParentId="9184" CreationDate="2019-08-13T06:48:18.363" Score="1" Body="&lt;p&gt;Convert your text files A and B to true, &lt;code&gt;sort-bed&lt;/code&gt;-sorted BED and then use &lt;code&gt;bedmap&lt;/code&gt; to map their associated IDs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk -vOFS=&quot;\t&quot; '($&lt;/span&gt;1!~/#/){ &lt;span class=&quot;math-container&quot;&gt;$3+=1; print $&lt;/span&gt;1,&lt;span class=&quot;math-container&quot;&gt;$2,$&lt;/span&gt;3,&lt;span class=&quot;math-container&quot;&gt;$4; print $&lt;/span&gt;1,&lt;span class=&quot;math-container&quot;&gt;$2,$&lt;/span&gt;3,&lt;span class=&quot;math-container&quot;&gt;$5; print $&lt;/span&gt;1,&lt;span class=&quot;math-container&quot;&gt;$2,$&lt;/span&gt;3,&lt;span class=&quot;math-container&quot;&gt;$6; }' A.txt | sort-bed - &amp;gt; A.bed&#xA;$&lt;/span&gt; awk -vOFS=&quot;\t&quot; '(&lt;span class=&quot;math-container&quot;&gt;$1!~/#/){ $3+=1; print $1,$2,$3,$4; print $1,$2,$3,$5; print $1,$2,$3,$&lt;/span&gt;6; }' B.txt | sort-bed - &amp;gt; B.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then map:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops -u A.bed B.bed | bedmap --echo --echo-map-id-uniq --delim '\t' --multidelim '\t' - &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you need to put the header back in:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops -u A.bed B.bed | bedmap --echo --echo-map-id-uniq --delim '\t' --multidelim '\t' - | cat &amp;lt;(echo -e &quot;#OUTPUT\n&quot;) &amp;lt;(awk '(NR==2)' A.bed) - &amp;gt; answer.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Some work might be needed with &lt;code&gt;awk&lt;/code&gt; to put reference and alternate alleles back into their own columns, but this should get you about 90% of the way there.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2019-08-13T06:48:18.363" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9186" PostTypeId="1" CreationDate="2019-08-13T11:06:24.303" Score="2" ViewCount="108" Body="&lt;p&gt;After having worked with Bayesian phylogenetic tree inference for some time, I am now trying out a method to infer network phylogenies. To get an idea of what it infers, I'd like to find a way of summarizing the results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output of the inference is a NEXUS file containing a stream of networks encoded in eXtended Newick as trees with duplicate node names, like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(((((((Kiraman:0.028228,Kui:0.028228)S1:0.0797301)#H4:0.35229)#H2:23.4168)#H1:15.8027,(Kamang:29.3316,#H1:5.45455[&amp;amp;&amp;amp;NHX:gamma=0.36825381342449554])S11:10.3482)S12:1.27331,((Klon:1.16727,((#H4:0.524833[&amp;amp;&amp;amp;NHX:gamma=0.33697036713230577],Kafoa:0.632791)S6:0.0615575)#H3:0.472921[&amp;amp;&amp;amp;NHX:gamma=0.6642547809404831])S2:35.6082,(#H2:22.2107[&amp;amp;&amp;amp;NHX:gamma=0.7544913491697585],((A-Petleng:0.421975,(A-Takalelang:0.168615,(A-Ulaga:0.145472,A-Atimelang:0.145472)S4:0.0231422)S5:0.253361)S7:0.902144,#H3:0.629771)S3:21.3468)S8:14.1045)S9:4.17761)S10:54.5301):0;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DYkkQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DYkkQ.png&quot; alt=&quot;The network described by the Newick string above&quot;&gt;&lt;/a&gt;￼&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no formal restriction on the shape of the network, and hybridization edges have a weight 0&amp;lt;γ&amp;lt;1. How do visually or formally summarize a stack of several thousands of these networks, and in particular the backbone tree structure plus a glimpse of the hybridization events they display?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have played with some tree summarizing approaches and tried to first infer a summary tree and then glue some cross-edges in there, but the way I came up with did not look reasonable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am now considering to calculate distances between leaves and using network visualization tools like SplitsTree or NeighborNet to get a visual representation of that, but how shoud I represent the uncertainty of the distances in even a single network (not to mention thousands of them) in the input for these algorithms?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any good approaches to summarize phylogenetic networks of the shape given above? Or to consistently simplify them to a shape that can be easily summarized?&lt;/p&gt;&#xA;" OwnerUserId="864" LastEditorUserId="3848" LastEditDate="2019-08-18T13:57:05.853" LastActivityDate="2019-08-18T13:57:05.853" Title="Visually summarizing phylogenetic networks" Tags="|visualization|phylogenetics|networks|phylogeny|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9187" PostTypeId="2" ParentId="9184" CreationDate="2019-08-13T11:40:02.090" Score="2" Body="&lt;p&gt;Whereas wouldn't be as fast as &lt;code&gt;awk&lt;/code&gt;, here is a rather dirty R solution:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(dplyr)&#xA;library(tidyr)&#xA;library(magrittr)&#xA;&#xA;# File 1&#xA;&#xA;# the &quot;fill&quot; parameter is used since not all the rows have the same number of cols&#xA;file_1 &amp;lt;- read.table(&quot;file_1.txt&quot;, skip = 1, fill = TRUE)&#xA;&#xA;# paste() function with argument collapse is applied to the &quot;phenotype&quot; &#xA;# columns of each row (these cols start at position 6 and goes until&#xA;# the number of cols: dim(x)[2] gives the number of cols of a table&#xA;phenotypes &amp;lt;- apply(file_1[, 6:dim(file_1)[2]], 1, function(x) paste(x, collapse = &quot;&quot;))&#xA;&#xA;# Cols referring to the genomic position, ref and alteration (cols 1:5)&#xA;# are combined with the newly created &quot;phenotypes&quot; column&#xA;file_1 &amp;lt;- data.frame(file_1[, 1:5], phenotypes)&#xA;&#xA;# colnames are added, could be avoided if dealt with at file reading step&#xA;names(file_1) &amp;lt;- c(&quot;Chr&quot;, &quot;Start&quot;, &quot;End&quot;, &quot;Ref&quot;, &quot;Alt&quot;, &quot;Phenotypes&quot;)&#xA;&#xA;# File 2&#xA;&#xA;file_2 &amp;lt;- read.table(&quot;file_2.txt&quot;, skip = 1, fill = TRUE)&#xA;&#xA;phenotypes &amp;lt;-apply(file_2[, 6:dim(file_2)[2]], 1, function(x) paste(x, collapse = &quot;&quot;))&#xA;&#xA;file_2 &amp;lt;- data.frame(file_2[, 1:5], phenotypes)&#xA;&#xA;names(file_2) &amp;lt;- c(&quot;Chr&quot;, &quot;Start&quot;, &quot;End&quot;, &quot;Ref&quot;, &quot;Alt&quot;, &quot;Phenotypes&quot;)&#xA;&#xA;# The two tables generated as above are merged using the appropriate&#xA;# columns as &quot;anchors&quot; in the &quot;by&quot; argument of full_join()&#xA;# The resulting Phenotypes.x and Phenotypes.y cols are concatenated&#xA;merged_tables &amp;lt;- full_join(file_1,&#xA;                           file_2,&#xA;                           by = c(&quot;Chr&quot;, &quot;Start&quot;, &quot;End&quot;, &quot;Ref&quot;, &quot;Alt&quot;))&#xA;merged_tables&lt;span class=&quot;math-container&quot;&gt;$Phenotypes &amp;lt;- paste(merged_tables$&lt;/span&gt;Phenotypes.x,&#xA;                                  merged_tables&lt;span class=&quot;math-container&quot;&gt;$Phenotypes.y,&#xA;                                  sep = &quot;,&quot;)&#xA;merged_tables$&lt;/span&gt;Phenotypes.x &amp;lt;- NULL&#xA;merged_tables$Phenotypes.y &amp;lt;- NULL&#xA;&#xA;# Duplicated rows (in terms of genomic pos, ref and alteration) in any &#xA;# of the files result in duplicated rows in the merged table and are&#xA;# &quot;handled&quot; by aggregating/summarizing the info at the phenotype cols&#xA;# by using paste()&#xA;# strsplit() is used to &quot;split&quot; the &quot;phenotypes&quot; on &quot;,&quot; in order to&#xA;# remove repeating phenotypes resulting from the full_join() call&#xA;&#xA;merged_tables %&amp;lt;&amp;gt;% group_by(Chr, Start, End, Ref, Alt) %&amp;gt;%&#xA;      summarize(Phenotypes = paste(Phenotypes, collapse = &quot;,&quot;)) %&amp;gt;%&#xA;      ungroup() %&amp;gt;%&#xA;      mutate(Phenotypes = strsplit(Phenotypes, &quot;,&quot;))&#xA;&#xA;merged_tables&lt;span class=&quot;math-container&quot;&gt;$Phenotypes &amp;lt;- lapply(merged_tables$&lt;/span&gt;Phenotypes, function(x) paste(unique(unlist(x)), collapse = &quot;,&quot;))&#xA;merged_tables&lt;span class=&quot;math-container&quot;&gt;$Phenotypes &amp;lt;- unlist(merged_tables$&lt;/span&gt;Phenotypes)&#xA;&#xA;# A tibble: 3 x 6&#xA;    Chr Start   End Ref   Alt   Phenotypes                 &#xA;  &amp;lt;int&amp;gt; &amp;lt;int&amp;gt; &amp;lt;int&amp;gt; &amp;lt;fct&amp;gt; &amp;lt;fct&amp;gt; &amp;lt;chr&amp;gt;                      &#xA;1     1  1000  1000 A     T     PhenoA,PhenoB              &#xA;2     2  2000  2000 T     G     PhenoF,PhenoG,PhenoH,PhenoI&#xA;3     3  3000  3000 C     G     PhenoL,PhenoM,PhenoN &#xA;&#xA;# The resulting table (tibble in this case) can be saved as a file&#xA;# using write.table() or write.xlsx()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;paste()&lt;/code&gt; in combination with its &lt;code&gt;collapse&lt;/code&gt; parameter is used because of the unequal numbers of columns in the files. &lt;code&gt;fread()&lt;/code&gt; of the &lt;code&gt;data.table&lt;/code&gt; package could be helpful for reading large files.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-08-13T17:36:38.063" LastActivityDate="2019-08-13T17:36:38.063" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9188" PostTypeId="2" ParentId="9184" CreationDate="2019-08-13T11:52:25.043" Score="0" Body="&lt;p&gt;If your files are small enough to fit in memory, here's one way, in Perl:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ perl -lane 'if(/^#/){$h=$_; next}; $&lt;/span&gt;k=join(&quot;\t&quot;,@F[0..4]); &lt;span class=&quot;math-container&quot;&gt;$v=join(&quot; &quot;,@F[5..$&lt;/span&gt;#F]); &lt;span class=&quot;math-container&quot;&gt;$d{$k} ? ($&lt;/span&gt;d{&lt;span class=&quot;math-container&quot;&gt;$k} .= &quot;, $&lt;/span&gt;v&quot;) : (&lt;span class=&quot;math-container&quot;&gt;$d{$k} = $&lt;/span&gt;v) ; }{ print &lt;span class=&quot;math-container&quot;&gt;$h;for(sort keys(%d)){print &quot;$_\t$d{$&lt;/span&gt;_}&quot;}  ' file*&#xA;#Chr Start End Ref Alt Phenotypes&#xA;1 1000 1000 A T Pheno A, Pheno B&#xA;2 2000 2000 T G Pheno F, Pheno G, Pheno H, Pheno I&#xA;3 3000 3000 C G Pheno L, Pheno M, Pheno N&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since that might be a little unclear, here's the same thing as a commented script and with informative variable names:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env perl&#xA;&#xA;## Iterate over each line of each input file. This is done by the -n switch.&#xA;while (&amp;lt;&amp;gt;) {&#xA;  ## remove trailing newlines. This is done by -l&#xA;  chomp;&#xA;  ## If this line starts with a #&#xA;  if(/^#/){&#xA;    ## Save it as a header. We do this for each header line, so only&#xA;    ## the last will be saved. but since the files both have the same header,&#xA;    ## that's not a problem.&#xA;    &lt;span class=&quot;math-container&quot;&gt;$header = $&lt;/span&gt;_;&#xA;    ## Skip to the next line. This is to avoid processing the headers.&#xA;    next&#xA;  }&#xA;  ## Split the line on whitespace into the @fields array. This is&#xA;  ## done by the -a switch which creates the @F array used above.&#xA;  @fields = split(/\s+/);&#xA;  ## The &lt;span class=&quot;math-container&quot;&gt;$key variable holds the concatenated string of the first 5 fields.&#xA;  $&lt;/span&gt;key=join(&quot;\t&quot;,@fields[0..4]);&#xA;  ## The &lt;span class=&quot;math-container&quot;&gt;$values variable holds the rest of the line.&#xA;  $&lt;/span&gt;values=join(&quot; &quot;,@fields[5..&lt;span class=&quot;math-container&quot;&gt;$#fields]);&#xA;  ## If we've seen this $&lt;/span&gt;key before&#xA;  if(&lt;span class=&quot;math-container&quot;&gt;$data{$key}){&#xA;    ## Append the current data to the value stored in the %data hash for this&#xA;    ## $key. &#xA;    $data{$key} .= &quot;, $values&quot;;&#xA;  }&#xA;  ## If we haven't seen this key before&#xA;  else{&#xA;    ## Save the values for this key in the %data hash.&#xA;    $data{$key}=$values&#xA;  }&#xA;}&#xA;## print the header&#xA;print &quot;$&lt;/span&gt;header\n&quot;;&#xA;## Now that all the data have been loaded, iterate over&#xA;## the sorted list of keys from the %data hash.&#xA;foreach my &lt;span class=&quot;math-container&quot;&gt;$key (sort keys(%data)){&#xA;  ##And print the output&#xA;  print &quot;$key\t$data{$&lt;/span&gt;key}\n&quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you save that as &lt;code&gt;foo.pl&lt;/code&gt;, you can run it like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ perl foo.pl fileA fileB&#xA;#Chr Start End Ref Alt Phenotypes&#xA;1 1000 1000 A T Pheno A, Pheno B&#xA;2 2000 2000 T G Pheno F, Pheno G, Pheno H, Pheno I&#xA;3 3000 3000 C G Pheno L, Pheno M, Pheno N&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Note that I am assuming the headers &lt;code&gt;#File A&lt;/code&gt; and &lt;code&gt;#FileB&lt;/code&gt; aren't actually part of your input and that you don't want &lt;code&gt;#OUTPUT&lt;/code&gt; included in the output file. If this isn't true, let me know and I can modify this.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-08-14T18:17:25.883" LastActivityDate="2019-08-14T18:17:25.883" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9189" PostTypeId="2" ParentId="9186" CreationDate="2019-08-13T12:14:46.303" Score="1" Body="&lt;p&gt;Firstly you treat this as two separate independent calculations.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The traditional approach is simply to produce a consensus phylogeny using your Bayesian method and assess the equivocal bootstraps/Bayesian probability.&lt;/li&gt;&#xA;&lt;li&gt;Feed your data set (not the trees) into NeighborNet within SplitsTrees. There are statistics that classify the level of departure from a bifurcating phylogeny.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Examine both outputs for orthogonal support for incongruence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; Phylogenetic network statistics are a relatively soft area of phylogeny (they are not used as hard evidence).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Networks from Bayesian trees&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final approach (which is what you are asking about), is to feed your Bayesian trees into &lt;a href=&quot;http://dendroscope.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dendroscope&lt;/a&gt;, this is a GUI that represents the uncertainty between different bifurcating phylogenies as networks and you can use it to understand the incongruence between different trees within your Bayesian MCMC, because the phylogenetic networks directly corresponds to the level of incongruence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dendroscope is a nice Java API and is very easy to use.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-08-13T12:14:46.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9191" PostTypeId="1" AcceptedAnswerId="9203" CreationDate="2019-08-13T14:20:06.583" Score="4" ViewCount="100" Body="&lt;p&gt;I'm searching for PDB code examples for &quot;healthy proteins&quot; and misfolded ones, so I can compare the 3D structure.&#xA;I'd prefer pairs of a &quot;protein misfolding disease&quot;, but could be any. Can I find such a list or how should I search for the particular codes?&#xA;I'm particularly interested in ones that also cointain SS-bonds.&lt;/p&gt;&#xA;" OwnerUserId="4449" LastEditorUserId="4449" LastEditDate="2019-08-13T18:44:25.577" LastActivityDate="2019-08-21T20:33:54.893" Title="PDB codes for misfolded proteins?" Tags="|protein-structure|pdb|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9193" PostTypeId="2" ParentId="9184" CreationDate="2019-08-13T18:36:30.950" Score="1" Body="&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; I could not get bedtools to produce exactly the right output. See note at the end.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First install bedtools. Then, in Bash:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(tail -n +2 FileA.txt; tail -n +2 FileB.txt) | sort -k1,1V -k2,2n -k3,3n - &amp;gt; FileC.txt&#xA;head -n 1 FileA.txt &amp;gt; FileD.txt&#xA;bedtools merge -i FileC.txt -c 4,5,6 -o distinct,distinct,distinct -delim &quot;, &quot; &amp;gt;&amp;gt; FileD.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;First line: &lt;code&gt;tail&lt;/code&gt; takes lines 2+ of the file, &lt;code&gt;sort&lt;/code&gt; sorts the files based on the chromosome, start, and stop coordinates, &lt;code&gt;-&lt;/code&gt; is the output from the previous command.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next line: &lt;code&gt;head&lt;/code&gt; takes the first 1 line of FileA (header), saves to FileD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Last line: &lt;code&gt;bedtools merge&lt;/code&gt; merges the intervals that overlap. Option &lt;code&gt;-c&lt;/code&gt; tells &lt;code&gt;merge&lt;/code&gt; what columns to do operations on when two intervals are merged. Option &lt;code&gt;-o&lt;/code&gt; tells merge what to do with those columns; in this case, it will make a comma-delimited list of distinct values. Option &lt;code&gt;-delim&lt;/code&gt; defines the delimiter for listing distinct info in columns 4-6.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Chr    Start   End Ref Alt Phenotypes&#xA;1   999     1001    A   T   Pheno A, Pheno B&#xA;2   1999    2001    T   G   Pheno F, Pheno G, Pheno H, Pheno I&#xA;3   2999    3001    C   G   Pheno L, Pheno M, Pheno N&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;PROBLEM:&lt;/strong&gt; This code will not produce the desired output b/c bedtools expands the interval &lt;code&gt;1    1000    1000&lt;/code&gt; to &lt;code&gt;1    999    1001&lt;/code&gt;. BED files are used for multi-nucleotide intervals whereas the VCF standard is better suited for single nucleotide intervals and &quot;ref/alt&quot; variant information. Bedtools might handle VCFs with 1-nt intervals appropriately, but but evidently not BED files with 1-nt intervals.&lt;/p&gt;&#xA;" OwnerUserId="3564" LastEditorUserId="3564" LastEditDate="2019-08-15T05:11:23.097" LastActivityDate="2019-08-15T05:11:23.097" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9194" PostTypeId="2" ParentId="5207" CreationDate="2019-08-13T19:11:36.263" Score="2" Body="&lt;p&gt;In case still needed, here is a piece of code that I have used for the same purpose (includes a lot of sanity checks). This is highly likely an overkill as each count file would probably have the same rows (as a result of being generated from the same workflow) and thus can easily be merged column-wise.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;count_file_dir &amp;lt;- &quot;../input/count_files/&quot;&#xA;count_file_names &amp;lt;- list.files(count_file_dir)&#xA;count_files &amp;lt;- paste0(count_file_dir, count_file_names)&#xA;&#xA;# read and merge count files, while removing last 5 lines that correspond to summary stats&#xA;count_table_list &amp;lt;- mclapply(count_files, function(x) {fread(x)[1:(.N-5)]})&#xA;names(count_table_list) &amp;lt;- sub(&quot;.count&quot;, &quot;&quot;, count_file_names)&#xA;&#xA;# check the dimensions of count files&#xA;t(sapply(count_table_list, dim))&#xA;&#xA;# check if the count files have the same gene ordering&#xA;# compare gene names of the first file with all&#xA;gene_names &amp;lt;- sapply(count_table_list, function(x) x$V1)&#xA;as.data.frame(apply(gene_names, 2, function(x) all(x == gene_names[,1])))&#xA;&#xA;# add file name as a unique column name for gene counts&#xA;for(i in names(count_table_list)) {&#xA;  names(count_table_list[[i]])[2] &amp;lt;- i&#xA;}&#xA;&#xA;# merge read counts and convert to a matrix&#xA;count_matrix &amp;lt;- Reduce(merge, count_table_list)&#xA;count_matrix_rownames &amp;lt;- count_matrix&lt;span class=&quot;math-container&quot;&gt;$V1&#xA;count_matrix$&lt;/span&gt;V1 &amp;lt;- NULL&#xA;count_matrix &amp;lt;- as.matrix(count_matrix)&#xA;rownames(count_matrix) &amp;lt;- count_matrix_rownames&#xA;colnames(count_matrix) &amp;lt;- sub(&quot;.count&quot;, &quot;&quot;, colnames(count_matrix))&#xA;&#xA;dim(count_matrix)&#xA;&#xA;dds &amp;lt;- DESeqDataSetFromMatrix(countData = count_matrix,&#xA;                              colData = ...,&#xA;                              design = ...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-08-13T19:26:05.493" LastActivityDate="2019-08-13T19:26:05.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9195" PostTypeId="1" CreationDate="2019-08-13T21:48:03.897" Score="2" ViewCount="47" Body="&lt;p&gt;I am building statistical models to analyse output from Illumina shotgun sequencing (HiSeq 4000) on stool samples (but RNA-seq data should behave similarily). The raw counts are a statistical sample of the total collection of DNA molecules used as input for the sequencing machine, and I think we can assume that the sequencing platform has limited capacity, such that detected reads are always a small fraction of the the original sequencing library. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider two different samples: one control sample and one treatment sample, which is identical apart from a single species (or gene) that has much higher absolute abundance relative what it in the control. As a result, the reads of this species will be much higher in the treatment sample, and also the 'real estate' of the remaining genes in that sample will be decreased. This is simply the nature of relative abundances.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, size factor estimation that uses the &quot;median ratio method&quot; (&lt;a href=&quot;http://dx.doi.org/10.1186/gb-2010-11-10-r106&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dx.doi.org/10.1186/gb-2010-11-10-r106&lt;/a&gt;) for instance, are based on &quot;experience with real data [...] [that] shows [that] a few highly and differentially expressed genes [or abundant species] may have strong influence on the total read count.&quot; (&lt;a href=&quot;http://dx.doi.org/10.1186/gb-2010-11-10-r106&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://dx.doi.org/10.1186/gb-2010-11-10-r106&lt;/a&gt;). I am curious to see experimental evidence for this last statement. Or does someone have at least an explanation of how total read count can be affected by the absolute abundance of a species/gene?&lt;/p&gt;&#xA;" OwnerUserId="5058" LastActivityDate="2019-08-14T07:56:18.397" Title="Total read count of shotgun affected by few highly abundant sequences" Tags="|statistics|illumina|coverage|read-correction|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9196" PostTypeId="1" CreationDate="2019-08-14T02:48:01.677" Score="2" ViewCount="126" Body="&lt;p&gt;Apologies if this question is previously asked. I have a &lt;code&gt;BAM&lt;/code&gt; file alignment to &lt;code&gt;hg19&lt;/code&gt; and miRNA gff3 file from &lt;code&gt;mirbase&lt;/code&gt; for annotation of BAM file. I want to extract information in the following format (e.g.):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;miRNA        chr    chr_start    chr_end    expression_count    sequence&#xA;hsa-mir-155  chr2       ...         ....         1020            ATGC....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I know that I can use htseq-count for BAM annotation and for expression counts but how can I get their chromosomal location. The reason for doing this specific way is that I want to show them in a CIRCOS plot.&lt;/p&gt;&#xA;&lt;p&gt;Just counting is not the goal, their chromosomal location also needs to be extracted. So just subread/htseq-count is not sufficient for extracting genomic location information.&lt;/p&gt;&#xA;&lt;p&gt;Previously, I had the same idea about processing the sam records. If I want to annotate chromosomal information with miRNA using .gff file, I have to check for each line in .sam records whether their chr_start and chr_end lie in the bandwidth of start and end of any miRNA in .gff file. This process can be very heavy and slow because .sam records are of more than 5 GB. That's why I want to know if there is any extremely fast and generalized way to do it.&lt;/p&gt;&#xA;&lt;p&gt;I want to avoid processing sam records in python because my experience with extremely heavy data like fastq or sam in python is not good. I would rather prefer awk/bash or any low-level language.&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="2141" LastEditorUserId="73" LastEditDate="2021-05-06T09:39:53.627" LastActivityDate="2023-05-31T11:09:16.187" Title="Chromosome information for miRNA" Tags="|sam|circos|chromosome-capture|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9197" PostTypeId="2" ParentId="9196" CreationDate="2019-08-14T04:46:13.873" Score="1" Body="&lt;p&gt;you can do &lt;code&gt;samtools view pathtobamfile&lt;/code&gt;, this will convert the file to &lt;code&gt;.sam&lt;/code&gt; format and send it to &lt;code&gt;stdout&lt;/code&gt;, the columns can then be parsed out which contain the information you are after.&lt;/p&gt;&#xA;" OwnerUserId="3381" LastEditorUserId="3381" LastEditDate="2021-05-11T06:02:26.083" LastActivityDate="2021-05-11T06:02:26.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9198" PostTypeId="2" ParentId="9195" CreationDate="2019-08-14T07:56:18.397" Score="2" Body="&lt;p&gt;That sentence in the Anders and Huber paper is a bit misleadingly worded (I don't think that was their intent). What they mean is that highly expressed and highly differentially expressed genes can out-compete the majority of the other genes, which thereby leads to them appearing to have lower expression if one uses total read counts for normalization (basically, what you said in the second paragraph).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An alternate interpretation of what they wrote (likely the one you understandable understood to have been their meaning) is that some highly expressed genes will inflate the total number of reads a sample actually gets. I have yet to see this in practice, but it certainly COULD happen. There are technical biases in sequencing such that some sequences (often due to length and GC content) are easier to sequence on common machines. A good example of this would be PhiX, which forms clusters much more easily on Illumina machines than real samples. I expect that this is relatively rare in practice, but using a more robust normalization would still help in protecting against it.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-08-14T07:56:18.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9199" PostTypeId="1" CreationDate="2019-08-14T10:52:51.840" Score="1" ViewCount="91" Body="&lt;p&gt;I am working with a gene cluster found in several species. The genes involved are pretty much the same, with the odd insertion, but they can be often found in different orientations. So, I can't just do a multiple alignment of the whole cluster. I've done phylogenies of individual genes and I've looked at cluster architecture (indels, rearrangements) and things make sense (eg. limited HGT), but..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;...Is there a simpler way to do this? A way to calculate / represent a phylogeny of the whole cluster? Like...some sort of web, maybe? Most of the software that does take rearrangements into consideration is meant for whole genomes, and is unreliable when it comes to fragments about 20-30kb long (like mine).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your help!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastEditorUserId="3848" LastEditDate="2020-04-18T04:16:06.627" LastActivityDate="2021-05-13T07:03:05.273" Title="Are there any resources for calculating / representing evolutionary relationships within gene clusters?" Tags="|fasta|phylogenetics|gene|phylogeny|pathway|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="9200" PostTypeId="2" ParentId="2634" CreationDate="2019-08-14T11:27:34.750" Score="2" Body="&lt;p&gt;The easiest and fastest method (which also doesn’t require internet access) is to use &lt;a href=&quot;http://bioconductor.org/packages/3.9/bioc/vignettes/AnnotationDbi/inst/doc/IntroToAnnotationPackages.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;annotation packages&lt;/a&gt;, e.g. &lt;a href=&quot;http://bioconductor.org/packages/3.9/data/annotation/html/org.Mm.eg.db.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;org.Mm.eg.db&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(AnnotationDbi)&#xA;library(org.Mm.eg.db)&#xA;&#xA;&#xA;# 1:1 mapping symbol→entrez&#xA;entrez_ids &amp;lt;- mapIds(org.Mm.eg.db, gene_symbols, 'ENTREZ', 'SYMBOL')&#xA;&#xA;# 1:n mapping symbol→…&#xA;print(columns(org.Mm.eg.db))  # check what you can query&#xA;df &amp;lt;- select(org.Mm.eg.db, gene_symbols, c(...), 'SYMBOL')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5060" LastActivityDate="2019-08-14T11:27:34.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9201" PostTypeId="2" ParentId="9191" CreationDate="2019-08-14T11:31:58.663" Score="2" Body="&lt;p&gt;Searching &quot;misfolded&quot; on the RCSB PDB turns up 64 hits. Most of these won't be relevant, but there are some interesting entries on tumor associated EGFR and misolded RNA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to experimentally obtain a structure of a protein it generally has to be in a stable conformation, i.e. at low energy. Being soluble also helps. This usually corresponds to the &quot;healthy&quot; form, as misfolding would lead to aggregation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That said, the structures of certain aggregates have been obtained. For example, see &lt;a href=&quot;https://www.pnas.org/content/116/23/11253&quot; rel=&quot;nofollow noreferrer&quot;&gt;this recent article&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also the concept of mutating a protein sequence to fold to a different structure, see for instance &lt;a href=&quot;https://www.nature.com/articles/nsb0998_778&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, though I guess you are looking for cases with the exact same sequence.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2019-08-14T11:31:58.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9202" PostTypeId="1" CreationDate="2019-08-14T14:47:20.797" Score="1" ViewCount="55" Body="&lt;p&gt;I need to identify differential genes across 8 groups. I know this can be done using DESeq2 or EdgeR as these are better suited and equipped.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I was thinking if something like this piece of code could be tweaked. Here it works for two groups each with 3 replicates and generates a heatmap. The code is in R &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pvals &amp;lt;- apply(dm, 1, function(x) wilcox.test(x~c(rep(0,3), rep(1,9)))$p.value)&#xA;heatmap(dm[pvals&amp;lt;0.05,])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thank you for your inputs and suggestions.&lt;/p&gt;&#xA;" OwnerUserId="5062" LastEditorUserId="292" LastEditDate="2019-08-15T08:09:12.843" LastActivityDate="2019-08-15T08:09:12.843" Title="Identification of differential genes across 8 groups" Tags="|r|rna-seq|differential-expression|" AnswerCount="0" CommentCount="3" ClosedDate="2019-08-31T16:10:25.130" ContentLicense="CC BY-SA 4.0" />
  <row Id="9203" PostTypeId="2" ParentId="9191" CreationDate="2019-08-14T14:52:32.017" Score="1" Body="&lt;p&gt;PDB &lt;code&gt;5E83&lt;/code&gt; sickle cell anaemia. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;All details for comparison to wildtype rbc are found in,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Oksenberg, D., Dufu, K., Patel, M.P., Chuang, C., Li, Z., Xu, Q., Silva-Garcia, A., Zhou, C., Hutchaleelaha, A., Patskovska, L., Patskovsky, Y., Almo, S.C., Sinha, U., Metcalf, B.W., Archer, D.R.&#xA;GBT440 increases haemoglobin oxygen affinity, reduces sickling and prolongs RBC half-life in a murine model of sickle cell disease.&#xA;(2016) Br.J.Haematol. 175: 141-153&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-08-14T14:52:32.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9204" PostTypeId="1" CreationDate="2019-08-14T21:40:47.240" Score="-1" ViewCount="292" Body="&lt;p&gt;I have a CLIP-Seq dataset I'm processing, which includes control samples and no inputs.  This is the second CLIP analysis I've performed to help out users of our genome core facility and the first one only had inputs. I've done a number of ChIP analyses in the past, which I loosely consider to be a similar type of analysis, and they all used inputs, so I've never had to look into the differences in using inputs versus controls when doing peak calling before.  Now that I'm trying to figure out what I need to do, I'm a little lost.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's what I've got:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Samples of cross-linked RNA/protein complex that went through a poly-A and then a FLAG pull-down (protein G beads with an antibody).  The complex was eluted, treated with proteinase K, and the RNA was sequenced.&lt;/li&gt;&#xA;&lt;li&gt;Controls went through the same processing as the samples except the control beads had no antibody&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;There are replicates in both cases.  (As a side-note, the RNA was treated with RNase III while bound to the beads and the RNA's trimmed off the complexes were also sequenced.  I'm not sure what to do with those samples yet.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was going to use one of the CLIP peak callers available on Galaxy, but whichever one I use, all the protocols I've found and the documentation I've looked at use inputs for the peak calling method and not controls.  So what do I need to do differently (if anything) to use the controls as the background in the peak calling?  Or do I need to suggest they also sequence the inputs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And just so I know (dumb question here)... Is &quot;input&quot; (as the name suggests) the sample as it was before the FLAG pulldown?  Why would one use one or the other (input versus control).&lt;/p&gt;&#xA;" OwnerUserId="5063" LastActivityDate="2021-09-04T14:06:40.687" Title="How to handle control samples in CLIP-Seq" Tags="|rna|peak-calling|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9205" PostTypeId="2" ParentId="9196" CreationDate="2019-08-14T22:58:55.043" Score="-1" Body="&lt;p&gt;The gff should have the coordinates for each feature.  I'd use featureCounts from subread to count up how many reads hit each feature.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-08-14T22:58:55.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9206" PostTypeId="1" AcceptedAnswerId="9220" CreationDate="2019-08-15T07:08:07.407" Score="2" ViewCount="177" Body="&lt;p&gt;I'm attempting to compare total RNA-seq with Ribo-Seq, to determine if changes in Ribo-Seq are due to changes in transcriptional expression (akin to the analysis performed by Anota2Seq). I am, however, struggling to understand how to accomplish this in DESeq2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My design is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cts &amp;lt;- read.delim(&quot;Interaction.txt&quot;, row.names=1)&#xA;coldata &amp;lt;- read.delim(&quot;Sample_Sheet.txt&quot;, row.names=1)&#xA;cts &amp;lt;- round(cts)&#xA;dds &amp;lt;- DESeqDataSetFromMatrix(countData = cts, colData = coldata, design = ~Genotype + Type + Genotype:Type)&#xA;dds &amp;lt;- DESeq(dds)&#xA;resultsNames(dds)&#xA;&#xA;&amp;gt; coldata&#xA;                Genotype Type&#xA;C_24h_Ribo_1     Control Ribo&#xA;C_24h_Ribo_2     Control Ribo&#xA;C_24h_Ribo_3     Control Ribo&#xA;CHIK_24h_Ribo_1    Viral Ribo&#xA;CHIK_24h_Ribo_2    Viral Ribo&#xA;CHIK_24h_Ribo_3    Viral Ribo&#xA;C_24h_RNA_1      Control  RNA&#xA;C_24h_RNA_2      Control  RNA&#xA;C_24h_RNA_3      Control  RNA&#xA;CHIK_24h_RNA_1     Viral  RNA&#xA;CHIK_24h_RNA_2     Viral  RNA&#xA;CHIK_24h_RNA_3     Viral  RNA&#xA;&#xA;&amp;gt; resultsNames(dds)&#xA;[1] &quot;Intercept&quot;                 &quot;Genotype_Viral_vs_Control&quot; &quot;Type_RNA_vs_Ribo&quot;          &quot;GenotypeViral.TypeRNA&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I do not understand which contrast is the one I need, or how to access the results i'm looking for (or interpret them). Any help?&lt;/p&gt;&#xA;" OwnerUserId="5065" LastEditorUserId="29" LastEditDate="2019-08-20T15:40:24.903" LastActivityDate="2019-08-20T15:40:24.903" Title="Comparing RNA-Seq and Ribo-Seq" Tags="|rna-seq|deseq2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9208" PostTypeId="2" ParentId="9035" CreationDate="2019-08-15T12:18:44.737" Score="0" Body="&lt;p&gt;You can use the &lt;a href=&quot;https://github.com/nanoporetech/wub/blob/master/scripts/bam_alignments_compare.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;bam_alignments_compare.py&lt;/a&gt; script from the &lt;a href=&quot;https://wub.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;wub&lt;/a&gt; Python package. &lt;/p&gt;&#xA;" OwnerUserId="4529" LastEditorUserId="4529" LastEditDate="2019-08-15T13:26:56.553" LastActivityDate="2019-08-15T13:26:56.553" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9209" PostTypeId="1" AcceptedAnswerId="9211" CreationDate="2019-08-15T13:19:05.420" Score="2" ViewCount="1233" Body="&lt;p&gt;I have eight fasta files, one per gene, with 106 fasta entries per file, like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;gene1.fasta:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;sample1_434564&#xA;GATCTCGTAGATGTGA&#xA;&amp;gt;sample2_2346764&#xA;GATCTCGTAGATGTGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gene2.fasta:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;sample1_532345&#xA;ATGCTGTGAAGTA&#xA;&amp;gt;sample2_6040103&#xA;ATGCTGTGAAGTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What I want to do is create one fasta file per sample so that I get all gene sequences for that sample in one file. As you can see, the sample ID is the same before the &lt;code&gt;_&lt;/code&gt; in the fasta headers, but the rest is unique for each fasta file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is  there any way to do this with &lt;code&gt;awk&lt;/code&gt; or other command-line tools? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Desired output:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sample1.fasta&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;gene1&#xA;GATCTCGTAGATGTGA&#xA;&amp;gt;gene2&#xA;ATGCTGTGAAGTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Sample2.fasta&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;gene1&#xA;GATCTCGTAGATGTGA&#xA;&amp;gt;gene2&#xA;ATGCTGTGAAGTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2239" LastEditorUserId="298" LastEditDate="2019-08-15T13:31:18.577" LastActivityDate="2021-10-03T12:35:03.877" Title="Concatenate multiple fasta files into one file per unique entry" Tags="|fasta|text-processing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9211" PostTypeId="2" ParentId="9209" CreationDate="2019-08-15T13:43:08.157" Score="6" Body="&lt;p&gt;You can do this simply enough in &lt;code&gt;awk&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk -F'_' '{&#xA;               split(FILENAME,n,/\./); &#xA;               if(/^&amp;gt;/){&#xA;                  sample=$1; &#xA;                  sub(/&amp;gt;/,&quot;&quot;,sample); &#xA;                  $&lt;/span&gt;0=&quot;&amp;gt;&quot;n[1]&#xA;               }&#xA;               print &amp;gt; sample&quot;.fasta&quot;&#xA;             }' gene1.fasta gene2.fasta ... geneN.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I run this on the two files you gave, I get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat sample1.fasta &#xA;&amp;gt;gene1&#xA;GATCTCGTAGATGTGA&#xA;&amp;gt;gene2&#xA;ATGCTGTGAAGTA&#xA;&#xA;$ cat sample2.fasta &#xA;&amp;gt;gene1&#xA;GATCTCGTAGATGTGA&#xA;&amp;gt;gene2&#xA;ATGCTGTGAAGTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Explanation&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;-F'_'&lt;/code&gt; : set the field separator to &lt;code&gt;_&lt;/code&gt; as an easy hack that lets us use &lt;code&gt;$1&lt;/code&gt; for the sample name (&lt;code&gt;sample_NNN&lt;/code&gt;, so &lt;code&gt;sample&lt;/code&gt; will be &lt;code&gt;$1&lt;/code&gt; and &lt;code&gt;NNN&lt;/code&gt; will be &lt;code&gt;$2&lt;/code&gt;).&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;split(FILENAME,n,/\./);&lt;/code&gt; : &lt;code&gt;FILENAME&lt;/code&gt; always holds the name of the file currently being processed. Here, we are splitting it on &lt;code&gt;.&lt;/code&gt; to get the gene's name. &lt;strong&gt;This assumes all of your files are named &lt;code&gt;$geneName.fasta&lt;/code&gt;.&lt;/strong&gt; The &lt;code&gt;split&lt;/code&gt; creates the array &lt;code&gt;n&lt;/code&gt; whose 1st element, &lt;code&gt;n[1]&lt;/code&gt; is the gene name.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;if(/^&amp;gt;/){&lt;/code&gt; : if this is a fasta header line.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;sample=$1&lt;/code&gt; : set the variable &lt;code&gt;sample&lt;/code&gt; to the sample's name. See the first note about &lt;code&gt;-F'_'&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;sub(/&amp;gt;/,&quot;&quot;,sample);&lt;/code&gt; : remove the &lt;code&gt;&amp;gt;&lt;/code&gt; from the &lt;code&gt;sample&lt;/code&gt; variable. We don't want that in the file name.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$0=&quot;&amp;gt;&quot;n[1]&lt;/code&gt; : set the current line to be &lt;code&gt;&amp;gt;&lt;/code&gt; and the gene name.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;print &amp;gt; sample&quot;.fasta&quot;&lt;/code&gt; : print each (modified) line into the appropriate file name.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-08-15T13:49:14.023" LastActivityDate="2019-08-15T13:49:14.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9212" PostTypeId="2" ParentId="4685" CreationDate="2019-08-15T16:54:58.700" Score="0" Body="&lt;p&gt;I think this is more to do with tree building than alignment construction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe the underlying issue is the 1) distribution of indels, this results in a blockwise deletion in the resulting phylogeny and 2) nucleotide saturation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Indels&lt;/strong&gt; Thus, for indels&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Archaeal &#xA;XXXX----YYYYY&#xA;bacterial&#xA;----ZZZZYYYYY&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The resulting phylogeny will be based on YYYYY and XXXX is ignored. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The alternative explanation is that the genetic distances involved have resulted in saturation of the nucleotide sequences. SSU has to be a nucleotide phylogeny and the level of reversion mutations over a vast amount of evolutionary time can become indistinguishable from random. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With regards the first point, you can replace the indels &quot;-&quot; with &quot;N&quot;. This is a cheat to trick a tree building program into not discarding the indel. Model based tree building programs have no ability to create a genetic distance between a base pair and an indel, so this data is essentially discarded from the tree. Worse it will remove the site from phylogenetic inference - which can be a massive loss of data. Two notes one this,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Some max. likelihood, Bayesian tree programs can avoid blockwise deletion, I don't know programmatically how they do this; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In PAML the distribution of indels was modelled, it doesn't appear to have been widely adopted however, but might be of value.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Alternative&lt;/strong&gt; The alternative method is really for protein genes and called &quot;RY&quot; coding, it might work on SSU because of the compensatory mutations involved in maintaining a secondary structure of the SSU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Word of warning&lt;/strong&gt;, the saturation may be so great, very little nucleotide point mutation is preserved. Essentially we are talking about preservation of phylogenetic signal across the origins of life ... and that is a very, very long time ago.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-08-15T16:54:58.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9213" PostTypeId="1" CreationDate="2019-08-15T20:16:03.350" Score="0" ViewCount="1082" Body="&lt;p&gt;it is said that the features.tsv from Cell Ranger v3 is analogous to the genes.tsv from Cell Ranger v2. So can I change the file name to genes.tsv for Seurat to read it? I know that Seurat has the update version 3 that can read the features.tsv, and I would like to use the old version. So if I change the file name, did it cause any problem?&lt;a href=&quot;https://i.stack.imgur.com/pRplq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pRplq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3171" LastActivityDate="2019-08-15T21:39:40.757" Title="Can I change the name of file features.tsv to genes.tsv" Tags="|single-cell|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9214" PostTypeId="2" ParentId="9213" CreationDate="2019-08-15T21:39:40.757" Score="1" Body="&lt;p&gt;You can follow the &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/output/matrices&quot; rel=&quot;nofollow noreferrer&quot;&gt;10X instructions&lt;/a&gt; to load your matrix into R:&#xA;the following is a direct quote from their webpage: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(Matrix)&#xA;matrix_dir = &quot;/opt/sample345/outs/filtered_feature_bc_matrix/&quot;&#xA;barcode.path &amp;lt;- paste0(matrix_dir, &quot;barcodes.tsv.gz&quot;)&#xA;features.path &amp;lt;- paste0(matrix_dir, &quot;features.tsv.gz&quot;)&#xA;matrix.path &amp;lt;- paste0(matrix_dir, &quot;matrix.mtx.gz&quot;)&#xA;mat &amp;lt;- readMM(file = matrix.path)&#xA;feature.names = read.delim(features.path, &#xA;                           header = FALSE,&#xA;                           stringsAsFactors = FALSE)&#xA;barcode.names = read.delim(barcode.path, &#xA;                           header = FALSE,&#xA;                           stringsAsFactors = FALSE)&#xA;colnames(mat) = barcode.names&lt;span class=&quot;math-container&quot;&gt;$V1&#xA;rownames(mat) = feature.names$&lt;/span&gt;V1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can then create a seurat object using mat&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;object &amp;lt;- CreateSeuratObject(mat, project=&quot;&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2675" LastActivityDate="2019-08-15T21:39:40.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9215" PostTypeId="1" CreationDate="2019-08-15T23:53:53.690" Score="1" ViewCount="142" Body="&lt;p&gt;I used a PSIBLAST (position specific scoring) to detect the similarities between specific proteins and the database of Clostridium. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The results did not reveal any similarities above the e value threshold of 0.001.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could I continue iterations of the second and third round of PSIBLAST?&lt;/p&gt;&#xA;" OwnerUserId="5073" LastEditorUserId="298" LastEditDate="2019-08-16T08:36:06.193" LastActivityDate="2021-09-05T23:00:25.107" Title="should I continue the iteration of psiblast if the blast step produced only below threshold sequences?" Tags="|psi-blast|blastp|" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9218" PostTypeId="2" ParentId="9204" CreationDate="2019-08-16T10:39:49.257" Score="0" Body="&lt;p&gt;I've not handled CLIP-seq before, but we have done iCLIP and in this we had controls, but no input. @DevonRyan is correct, these control samples are similar to IgG pulldowns in ChIP seq. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way we handled this was to call our peaks on isolated samples. The original peak calling alog from the first CLIP seq paper doesn't require input. Also I believe that clipper doesn't need input (although it will take it). We then removed any peaks from the sample that overlapped a peak in the control. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But most of our analyses we didn't do on peaks - we did lots of things like looking at metagenes, or classes of genes, or particular types of exon etc, and when we did this, because we were averaging over many sites, we could generate the pattern for the control as well as for the input, and often normalise the one to the other. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2019-08-16T10:39:49.257" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9219" PostTypeId="2" ParentId="9199" CreationDate="2019-08-17T05:24:29.353" Score="1" Body="&lt;p&gt;Firstly you wanted a webserver to build a tree, ok, I would use &lt;a href=&quot;http://www.atgc-montpellier.fr/phyml/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PhyML server&lt;/a&gt;. The good thing is that with only 22 taxa the calculations are easily doable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm assuming is you can map your rearrangements to produce a single homologous gene alignment for all taxa. So you appear to infer you can break your 20-30kb contig into individual aligments of homologous genes which has incorporated the rearrangement and enables easy production of an alignment file. This would leave you with around 20-30 gene alignment files. I hope that is correct, because that is how you get around the rearrangement problem and enables the data to be reassembled for the final tree (below).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing the tree is simple, because you can exploit the assumptions used in tree building regarding independence between mutations at each alignment position. The theoretical proof, the data is sound, is complicated. In summary,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;build a consensus phylogeny (one tree representative of lots of trees) using two different approaches&lt;/li&gt;&#xA;&lt;li&gt;combine the data into a single alignment, if the above looks good, or excluding the alignments that are affected recombination.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The theoretical issues with what you want to do are complicated because you need to distinguish bifurcating (clonal) homologues from recombination that could have occured in the reassortment events. The following is a  purists approach, you might not want to do this stuff, but it gives you some idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just before I do that ... I just want to check the alignments assumption is correct? and which bacteria?&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-08-17T16:25:52.337" LastActivityDate="2019-08-17T16:25:52.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9220" PostTypeId="2" ParentId="9206" CreationDate="2019-08-17T07:24:18.310" Score="3" Body="&lt;p&gt;Adding to what benn pointed out, at this line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData = cts, colData = coldata, design = ~Genotype + Type + Genotype:Type)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You specified that the counts, can be a explained by Genotype (being Control or Viral) or Type (RNAseq or Ribo) or an interaction effect &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you need is this coefficient, Genotype:Type. To use it, go&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;res &amp;lt;- results(dds,name=&quot;GenotypeViral.TypeRNA&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Based on your setup, you are always comparing RNA to Ribo (see &quot;Type_RNA_vs_Ribo&quot; in resultsNames). If changes to the abundance of a mRNA is similar in RNAseq or Ribo, then the coefficient of the interaction term will be zero and it will be zero in the log2foldchange column in results(res). A positive fold change indicates the log2foldchange between RNA and Ribo is higher in Viral.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would like to compare ribo versus and rna-seq, then set RNA as the reference by:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;coldata&lt;span class=&quot;math-container&quot;&gt;$Type = factor(coldata$&lt;/span&gt;Type,levels=c(&quot;RNA&quot;,&quot;Ribo&quot;))&#xA;# run deseq again&#xA;dds &amp;lt;- DESeqDataSetFromMatrix(countData = cts, colData = coldata, design = ~Genotype + Type + Genotype:Type)&#xA;dds &amp;lt;- DESeq(dds)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With this you will compare by default Ribo versus RNA, and the interaction term will tell you how much more enriched or depleted the ribo fraction is, in the Viral samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would recommend checking the vignette out &lt;a href=&quot;http://bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#contrasts&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#contrasts&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#contrasts&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5077" LastEditorUserId="3848" LastEditDate="2019-08-17T16:06:49.513" LastActivityDate="2019-08-17T16:06:49.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9221" PostTypeId="1" CreationDate="2019-08-17T10:28:42.873" Score="0" ViewCount="143" Body="&lt;p&gt;I have set of modules from various condition  ,with first column as Probe_ID name and I have an annotation file with Probe_ID and Gene_name.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have to map the Probe_ID to gene name in each file and write it back.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what im doing is as such &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;filelist = list.files(pattern = &quot;.*.txt&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get list of files &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then those list of files into list of dataframe&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;datalist = lapply(filelist, function(x)read.table(x, header=T)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now i read into the annotation file &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ANNOTATION_FILE &amp;lt;- read_delim(&quot;/run/media/punit/data1/Dev_bio_info/ANNOTATION_FILE.txt&quot;, &#xA;    &quot;\t&quot;, escape_double = FALSE, trim_ws = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The normal way i do is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I perform inner_join(ANNOTATION_FILE,my_file) for single file but as i have like 22 files in the module .I want to do it in loop and im not sure how to proceed further.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My data frame example &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how my module files looks like .&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dput(head(module_ivory[c(1:10)]))&#xA;structure(list(Gene = c(&quot;1556672_a_at&quot;, &quot;1557803_at&quot;, &quot;1558080_s_at&quot;, &#xA;&quot;1566465_at&quot;, &quot;213089_at&quot;, &quot;213229_at&quot;), GSM1304905 = c(5.977158746759, &#xA;6.83852532860115, 5.93931007942615, 6.11926533584282, 9.92182534751875, &#xA;10.2576150019626), GSM1304906 = c(7.51622774902859, 7.71268478229187, &#xA;7.32360342148457, 6.8928342169826, 9.12807784837542, 10.6767090567628&#xA;), GSM1304907 = c(6.58069278884036, 7.86890363253932, 6.9901005950279, &#xA;6.08614452085924, 10.1220627669628, 10.9203125158285), GSM1304908 = c(6.60119000844375, &#xA;6.71735654004629, 7.43132441110608, 5.48084261893207, 9.81689665710992, &#xA;10.7649566455486), GSM1304909 = c(5.95049879217177, 6.132975671606, &#xA;7.0902951305213, 5.59408027736444, 9.08474568290096, 9.84784203563294&#xA;), GSM1304910 = c(5.83616005353191, 5.94947517647125, 6.61469427098759, &#xA;4.31285839654272, 9.44446310110043, 9.99021299504538), GSM1304911 = c(5.92373056182016, &#xA;6.14502105445603, 6.2333758614231, 5.46455403294761, 9.55447963883665, &#xA;10.1810888944642), GSM1304912 = c(6.43107367158601, 6.36340779490876, &#xA;7.18374247647159, 5.71508717609646, 9.61355221684561, 10.3266656836978&#xA;), GSM1304913 = c(6.4010171640209, 7.31600259413326, 6.16969520557938, &#xA;5.40234538288784, 8.89481478840621, 10.290064262843)), class = c(&quot;tbl_df&quot;, &#xA;&quot;tbl&quot;, &quot;data.frame&quot;), row.names = c(NA, -6L))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My annotation file &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dput(head(ANNOTATION_FILE))&#xA;structure(list(Gene = c(&quot;1007_s_at&quot;, &quot;1053_at&quot;, &quot;117_at&quot;, &quot;121_at&quot;, &#xA;&quot;1255_g_at&quot;, &quot;1294_at&quot;), Gene_Symbol = c(&quot;DDR1 /// MIR4640&quot;, &#xA;&quot;RFC2&quot;, &quot;HSPA6&quot;, &quot;PAX8&quot;, &quot;GUCA1A&quot;, &quot;MIR5193 /// UBA7&quot;)), class = c(&quot;tbl_df&quot;, &#xA;&quot;tbl&quot;, &quot;data.frame&quot;), row.names = c(NA, -6L))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to do inner join on each file and write them back into the file .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help or suggestion would be really appreciated &lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2019-08-17T13:25:43.963" Title="Mapping probe ID to gene name" Tags="|r|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9222" PostTypeId="2" ParentId="9221" CreationDate="2019-08-17T11:27:44.303" Score="1" Body="&lt;p&gt;To end up with a single table made by merging all your tables, you can give a try with &lt;code&gt;reduce()&lt;/code&gt; of &lt;code&gt;purrr&lt;/code&gt;:  The first argument &lt;code&gt;.x&lt;/code&gt; would be the list of tables to merge (in your case the annotation file plus all the other files) and the &lt;code&gt;.y&lt;/code&gt; argument would be your function, i.e. &lt;code&gt;inner_join()&lt;/code&gt;. &lt;code&gt;reduce()&lt;/code&gt; would merge all the tables within the given list, left to right or right to left. &lt;code&gt;...&lt;/code&gt; would enable you to specify column(s) to use as &quot;anchors&quot; while merging.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would prefer to keep your tables separately then &lt;code&gt;lapply()&lt;/code&gt; is the answer: &lt;code&gt;lapply(list_of_files, function(x) inner_join(annotation_file, x))&lt;/code&gt;. This would give a list of merged tables.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-08-17T13:25:43.963" LastActivityDate="2019-08-17T13:25:43.963" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="9223" PostTypeId="1" CreationDate="2019-08-17T15:16:38.747" Score="3" ViewCount="1196" Body="&lt;p&gt;I have &lt;strong&gt;6 multi-fasta files&lt;/strong&gt;, every of them contains ca 1500 sequences like that: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Haladaptatus sp. R4&#xA;MQATVRDLNGEDADTVDLPDVFETTVRTDLIKRAVLAAQANRKQDYGTDPHAGMRTSAES&#xA;PGSGRGMAHVPQTNGRGARVPFTVGGRVAHPPKAEKDRSRSINKKERKLAVRSAIAATTD&#xA;AERVSERGHRFDEDTELPLVVSDDFEDLVKTQEVVSFLEAVGIDADIARAEDNKKVRAGR&#xA;GTTRGRKYKTPKSILFVTSEEPSRAARNLAGADVATAREVNTEDLAPGTQAGRLTVWTES&#xA;ALEEVADR&#xA;&amp;gt;Methanosphaera sp. WGK6&#xA;MAKVNVYSLKGDITEEIELPEIFEEEYRPDVIKRAVISTQTARIQPWGANPMAGKRTTAE&#xA;SFGSGRGAAMVPRVKGSSKAAFVPQAIGGRKAHPPRVNTIYHEKINKKERILAIRSAIAA&#xA;TANKEIVEQRGHAVANLEQVPFVVDDELETIKTTKETREIFKDLGIMDDILRAKKGRKIK&#xA;SGKGKLRGRKYRTPKGPLVVVGNDRGISLGARNHAGVEVVEVNNINAELLAPGTHAGRLT&#xA;IYTKSAVEKLADLFQQNRS&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and so on. &#xA;I need to create a new result file containing sequences that are common in these multi-fasta files. &#xA;Example:&#xA;If sequence&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Haladaptatus sp. R4&#xA;    MQATVRDLNGEDADTVDLPDVFETTVRTDLIKRAVLAAQANRKQDYGTDPHAGMRTSAES&#xA;    PGSGRGMAHVPQTNGRGARVPFTVGGRVAHPPKAEKDRSRSINKKERKLAVRSAIAATTD&#xA;    AERVSERGHRFDEDTELPLVVSDDFEDLVKTQEVVSFLEAVGIDADIARAEDNKKVRAGR&#xA;    GTTRGRKYKTPKSILFVTSEEPSRAARNLAGADVATAREVNTEDLAPGTQAGRLTVWTES&#xA;    ALEEVADR&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;occures in 1 &lt;strong&gt;and&lt;/strong&gt; 2 &lt;strong&gt;and&lt;/strong&gt; 3 &lt;strong&gt;and&lt;/strong&gt; 4 &lt;strong&gt;and&lt;/strong&gt; 5 &lt;strong&gt;and&lt;/strong&gt; 6 file it shoud be written to the result file. &#xA;May anoyone help?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edited: this is my Python script which should work on 2 files (not 6 for now to make it simplier):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for z in f1:&#xA;if z[0] == '&amp;gt;': &#xA;    znaleziony = False&#xA;    gatunek1 = z[1:len(z)]&#xA;    #print(gatunek1)&#xA;    for x in g1:&#xA;        #print(x)&#xA;        if x[0] == '&amp;gt;':&#xA;&#xA;            gatunek2 = x[1:len(x)]&#xA;            if gatunek1 == gatunek2:&#xA;                wynik.write('&amp;gt;')                    &#xA;                wynik.write(gatunek1)&#xA;                znaleziony = True&#xA;                continue&#xA;&#xA;&#xA;&#xA;elif znaleziony == True:&#xA;    wynik.write(z)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;wynik.close()&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately it repeats some identifiers of sequence and does compare only identifiers (not whole sequence).&lt;/p&gt;&#xA;" OwnerUserId="5081" LastEditorUserId="3848" LastEditDate="2020-04-24T14:53:01.093" LastActivityDate="2020-04-24T14:53:01.093" Title="How to find common sequences among 6 multi-fasta files" Tags="|python|fasta|ngs|phylogenetics|phylogeny|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="9224" PostTypeId="2" ParentId="4077" CreationDate="2019-08-18T14:07:14.413" Score="1" Body="&lt;p&gt;This is a haplotype map, where each node is proportional to the frequency of a given allele. The formal tool to do this is Network 5 found &lt;a href=&quot;http://www.fluxus-engineering.com/sharenet.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The problem with manaul haplotype maps is the area of the node should be proportional to the frequency, if this is not accurate it could be perceived to be misleading. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The approach for looking at each parsimonious state along a branch is a phylogenetics method which is performed via MacClade 4.0 ... requires an old Mac though.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-08-18T14:07:14.413" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9225" PostTypeId="1" AcceptedAnswerId="9227" CreationDate="2019-08-19T02:43:23.377" Score="2" ViewCount="15564" Body="&lt;p&gt;I would like to merge more than two sample in the Seurat, and the mergeseurat can only merge two sample. So what should I do now. The screenshot is my script.&lt;a href=&quot;https://i.stack.imgur.com/dN8oy.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dN8oy.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3171" LastEditorUserId="3848" LastEditDate="2023-07-17T12:24:09.503" LastActivityDate="2023-07-17T12:24:09.503" Title="how to merge more than two sample in Seurat?" Tags="|rna-seq|seurat|single-cell|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9226" PostTypeId="2" ParentId="9225" CreationDate="2019-08-19T07:00:08.353" Score="1" Body="&lt;p&gt;For now, Seurat does not have an option to merge more than two samples at a time. You can write a function to run the &lt;code&gt;MergeSeurat&lt;/code&gt; function in a loop. Following issue in Github has an example function for multiple samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/satijalab/seurat/issues/432&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/satijalab/seurat/issues/432&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="206" LastActivityDate="2019-08-19T07:00:08.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9227" PostTypeId="2" ParentId="9225" CreationDate="2019-08-19T07:11:10.853" Score="4" Body="&lt;p&gt;You have fed arguments to the &lt;code&gt;MergeSeurat()&lt;/code&gt; function that it does not expect. In terms of objects, &lt;code&gt;MergeSeurat()&lt;/code&gt; accepts only 2 arguments &lt;code&gt;object1&lt;/code&gt; and &lt;code&gt;object2&lt;/code&gt;, please do &lt;code&gt;?MergeSeurat&lt;/code&gt; and see yourself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seurat is probably the best documented single cell package, your exact question has its own dedicated vignette: &lt;a href=&quot;https://satijalab.org/seurat/v3.0/merge_vignette.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/v3.0/merge_vignette.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the vignette above is for Seurat v3 and in case you would like to use Seurat v2:&#xA;You can create a named list of Seurat objects to be merged and then use &lt;code&gt;reduce()&lt;/code&gt; of the &lt;code&gt;purrr&lt;/code&gt; package.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# create a named list of seurat objects to be merged&#xA;# code not shown&#xA;&#xA;# optional but probably a good idea&#xA;# rename cells using object names as prefix&#xA;for (i in names(seurat_object_list)) {&#xA;  seurat_object_list[[i]] &amp;lt;- RenameCells(seurat_object_list[[i]],&#xA;                                         add.cell.id = i)&#xA;}&#xA;&#xA;# merge all the objects in the list&#xA;merged_combined &amp;lt;- reduce(seurat_object_list,&#xA;                          MergeSeurat,&#xA;                          do.normalize = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-08-19T07:11:10.853" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="9228" PostTypeId="1" CreationDate="2019-08-19T07:43:42.493" Score="1" ViewCount="1075" Body="&lt;p&gt;I want to call peaks using MACS2 on my paired-end ATAC-seq BAM and I am interested in cutting sites. So I make use of the -BAM option (opposed to the -BAMPE) option. However this throws away all the mates, and all the reads of which the mate did not align (that's more than 50%!!!). My idea was to align paired-end, and then remove all the information from the BAM that it is paired-end. So my question is how do I:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Convert paired-end BAM into a single-end BAM and keep all the reads&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I made a start based on the &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAM-format specification &lt;/a&gt;, but when I run the result of this script through markduplicates I get the error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ERROR: Record 1, Read name SRR6245286.60336908., MRNM should not be set for unpaired read.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can not find what this MRNM is...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my code I used to generate the faulty &lt;em&gt;'single-end'&lt;/em&gt; bam, but I am open to suggestions (e.g. bash, preferably no R). &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pysam&#xA;&#xA;paired =         1&#xA;proper_pair =    2&#xA;mate_unmapped =  8&#xA;mate_reverse =   32&#xA;first_in_pair =  64&#xA;second_in_pair = 128&#xA;&#xA;bam_in = pysam.AlignmentFile(&quot;/vol/atacpipeline/bwa/GSM2837489-danRer7.samtools-coordinate.bam&quot;, &quot;rb&quot;)&#xA;bam_out = pysam.AlignmentFile(&quot;/vol/atacpipeline/bwa/GSM2837489-danRer7_SE.samtools-coordinate.bam&quot;, &quot;wb&quot;, template=bam_in)&#xA;for line in bam_in:&#xA;    if line.flag &amp;amp; second_in_pair:&#xA;        line.pos += line.template_length&#xA;    line.next_reference_id = 0&#xA;    line.next_reference_start = 0&#xA;    line.flag &amp;amp;= ~(paired + proper_pair + mate_unmapped + mate_reverse + first_in_pair + second_in_pair)&#xA;&#xA;    bam_out.write(line)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5084" LastActivityDate="2021-07-20T19:07:38.047" Title="Convert paired-end BAM into a single-end BAM and keep all the reads" Tags="|pysam|macs2|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9229" PostTypeId="1" AcceptedAnswerId="9238" CreationDate="2019-08-19T09:26:03.923" Score="4" ViewCount="1125" Body="&lt;p&gt;I have a VCF file that has deletions specified like this, with the REF allele containing only the deleted portion and the ALT allele just saying &lt;code&gt;-&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1 101 . AAA -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to combine this with another VCF in which deletions are specified like this, with REF and ALT both extended by 1bp to the left to ensure neither is empty:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1 100 . GAAA G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a tool to convert from one of these representations to the other? I have already tried &lt;code&gt;vt normalize&lt;/code&gt; and &lt;code&gt;bcftools norm&lt;/code&gt; but neither of them seem to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't really mind which representation I end up with although I think the second is the one more consistent with the spec.&lt;/p&gt;&#xA;" OwnerUserId="306" LastActivityDate="2023-01-25T11:30:21.040" Title="Fix dash/empty ALT alleles for deletions in VCF" Tags="|vcf|" AnswerCount="5" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9230" PostTypeId="2" ParentId="9229" CreationDate="2019-08-19T10:12:42.853" Score="1" Body="&lt;p&gt;Probably it is better to use dedicated tools, however, here is a quick R workaround. It is relatively straightforward to remove the extra base (first letter of column 4) in the second type of output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# just a workaround to encolose the second file in a data frame&#xA;# you can just read your file as a table using fread(), read.csv(), ...&#xA;test &amp;lt;- data.frame(&#xA;  col1 = &quot;chr1&quot;,&#xA;  col2 = 100,&#xA;  col3 = &quot;.&quot;,&#xA;  col4 = &quot;GAAA&quot;,&#xA;  col5 = &quot;G&quot;,&#xA;  stringsAsFactors = FALSE&#xA;)&#xA;&#xA;&amp;gt; test&#xA;  col1 col2 col3 col4 col5&#xA;1 chr1  100    . GAAA    G&#xA;&#xA;# remove the first letter in column 4&#xA;test&lt;span class=&quot;math-container&quot;&gt;$col4 &amp;lt;- substring(test$&lt;/span&gt;col4, 2)&#xA;# converting column 4 to &quot;-&quot;&#xA;test$col5 &amp;lt;- &quot;-&quot;&#xA;&#xA;&amp;gt; test&#xA;  col1 col2 col3 col4 col5&#xA;1 chr1  100    .  AAA    -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is how it can be done with &lt;code&gt;awk&lt;/code&gt; for speed and memory concerns: The logic is the same as above, remove the first letter in column 4 and then change column 5 to &quot;-&quot;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{&lt;span class=&quot;math-container&quot;&gt;$4 = substr($&lt;/span&gt;4,2); print}' infile &amp;gt; outfile&#xA;awk '{$5 = &quot;-&quot;; print}' outfile &amp;gt; outfile2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;infile&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1 100 . GAAA G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;outfile&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1 100 . AAA G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;outfile2&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1 100 . AAA -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-08-19T13:20:03.430" LastActivityDate="2019-08-19T13:20:03.430" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9231" PostTypeId="2" ParentId="9228" CreationDate="2019-08-19T10:29:23.910" Score="2" Body="&lt;p&gt;Like Devon pointed out, most likely you should sort out whether the files have been marked for duplicates correctly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use samtools rmdup too&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools rmdup &amp;lt;bamfile&amp;gt; &amp;lt;output&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also, I find it very odd that 50% of the reads are lost when you keep only properly paired reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest removing the duplicates and then converting the bam file to a bed file. Macs2 can take in the bed format. Any extremely crude python script that converts a bam into a bed:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pysam&#xA;samfile = pysam.AlignmentFile(&quot;test.bam&quot;, &quot;rb&quot;)&#xA;open(&quot;test.bed&quot;, 'w').close()&#xA;f = open(&quot;test.bed&quot;, &quot;a&quot;)&#xA;STRAND = [&quot;+&quot;,&quot;-&quot;]&#xA;for read in samfile.fetch():&#xA;    STR = STRAND[int(read.is_reverse)]&#xA;    BED =[read.reference_name,read.pos,read.reference_end,&quot;.&quot;,read.mapq,STR]&#xA;    f.write('\t'.join([str(i) for i in BED]))&#xA;    f.write('\n')&#xA;f.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can also try the conversion with bedtools or definitely refine that script above with pandas etc. &lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2019-08-19T10:29:23.910" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9232" PostTypeId="2" ParentId="9223" CreationDate="2019-08-19T12:35:49.787" Score="3" Body="&lt;p&gt;If both the ID lines and the sequences are duplicated, you can do this using the FastaToTbl and TblToFasta scripts &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2651/298&quot;&gt;I have posted before&lt;/a&gt; and just normal Linux utilities:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FastaToTbl file1.fa file2.fa ... file6.fa | &#xA;    sort | uniq -dc | awk '$1==6' | sed 's/^  *6 //' | TblToFasta &amp;gt; dupes.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To understand this, let's look at the output of each step. The &lt;code&gt;FastaToTbl&lt;/code&gt; script &#xA;just converts fasta to a tab-delimited format where the ID is the first field and the sequence is the second, all on one line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ FastaToTbl file*fa | head -n1&#xA;Haladaptatus sp. R4 MQATVRDLNGEDADTVDLPDVFETTVRTDLIKRAVLAAQANRKQDYGTDPHAGMRTSAESPGSGRGMAHVPQTNGRGARVPFTVGGRVAHPPKAEKDRSRSINKKERKLAVRSAIAATTDAERVSERGHRFDEDTELPLVVSDDFEDLVKTQEVVSFLEAVGIDADIARAEDNKKVRAGRGTTRGRKYKTPKSILFVTSEEPSRAARNLAGADVATAREVNTEDLAPGTQAGRLTVWTESALEEVADR&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, we pass this through &lt;code&gt;sort&lt;/code&gt; and &lt;code&gt;uniq -dc&lt;/code&gt; which prints out the number of times a line was repeated (only for duplicated lines):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ FastaToTbl file*fa | sort | uniq -dc&#xA;      4 some other sequence ALASOOPAKASKAPPPQQCDDREETAYSSLDAS&#xA;      6 Haladaptatus sp. R4 MQATVRDLNGEDADTVDLPDVFETTVRTDLIKRAVLAAQANRKQDYGTDPHAGMRTSAESPGSGRGMAHVPQTNGRGARVPFTVGGRVAHPPKAEKDRSRSINKKERKLAVRSAIAATTDAERVSERGHRFDEDTELPLVVSDDFEDLVKTQEVVSFLEAVGIDADIARAEDNKKVRAGRGTTRGRKYKTPKSILFVTSEEPSRAARNLAGADVATAREVNTEDLAPGTQAGRLTVWTESALEEVADR&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since we only want those that were present in all 6 files, we can use &lt;code&gt;awk&lt;/code&gt; telling it to only print lines whose first field was 6:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ FastaToTbl file*fa | sort | uniq -dc | awk '$&lt;/span&gt;1==6'&#xA;      6 Haladaptatus sp. R4 MQATVRDLNGEDADTVDLPDVFETTVRTDLIKRAVLAAQANRKQDYGTDPHAGMRTSAESPGSGRGMAHVPQTNGRGARVPFTVGGRVAHPPKAEKDRSRSINKKERKLAVRSAIAATTDAERVSERGHRFDEDTELPLVVSDDFEDLVKTQEVVSFLEAVGIDADIARAEDNKKVRAGRGTTRGRKYKTPKSILFVTSEEPSRAARNLAGADVATAREVNTEDLAPGTQAGRLTVWTESALEEVADR&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But then we don't want the 6 as part of the final output, so we need to remove it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ FastaToTbl file*fa | sort | uniq -dc | awk '$&lt;/span&gt;1==6' | sed 's/^  *6 //'&#xA;Haladaptatus sp. R4 MQATVRDLNGEDADTVDLPDVFETTVRTDLIKRAVLAAQANRKQDYGTDPHAGMRTSAESPGSGRGMAHVPQTNGRGARVPFTVGGRVAHPPKAEKDRSRSINKKERKLAVRSAIAATTDAERVSERGHRFDEDTELPLVVSDDFEDLVKTQEVVSFLEAVGIDADIARAEDNKKVRAGRGTTRGRKYKTPKSILFVTSEEPSRAARNLAGADVATAREVNTEDLAPGTQAGRLTVWTESALEEVADR&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally, convert to fasta again:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ FastaToTbl file*fa | sort | uniq -dc | awk '$&lt;/span&gt;1==6' | sed 's/^  *6 //' | TblToFasta &#xA;&amp;gt;Haladaptatus sp. R4 &#xA;MQATVRDLNGEDADTVDLPDVFETTVRTDLIKRAVLAAQANRKQDYGTDPHAGMRTSAES&#xA;PGSGRGMAHVPQTNGRGARVPFTVGGRVAHPPKAEKDRSRSINKKERKLAVRSAIAATTD&#xA;AERVSERGHRFDEDTELPLVVSDDFEDLVKTQEVVSFLEAVGIDADIARAEDNKKVRAGR&#xA;GTTRGRKYKTPKSILFVTSEEPSRAARNLAGADVATAREVNTEDLAPGTQAGRLTVWTES&#xA;ALEEVADR&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Alternatively, if your files are small enough to be loaded into memory, you could do this instead:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FastaToTbl file*fa | &#xA;    awk '{a[$0]++}END{for(s in a){if(a[s]==6){print s}}}' | &#xA;        TblToFasta &amp;gt; dupes.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or you could just do the whole thing in a single awk operation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{ &#xA;        if(/^&amp;gt;/){&#xA;            a[seq]++; &#xA;            seq=&lt;span class=&quot;math-container&quot;&gt;$0; &#xA;        }&#xA;        else{&#xA;            seq=seq&quot;\n&quot;$&lt;/span&gt;0&#xA;        }&#xA;      }&#xA;      END{&#xA;        a[seq]++;&#xA;        for(s in a){&#xA;            if(a[s]==6){&#xA;                print s&#xA;            }&#xA;        }&#xA;       }' file*.fa &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-08-19T12:44:47.937" LastActivityDate="2019-08-19T12:44:47.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9233" PostTypeId="1" AcceptedAnswerId="9236" CreationDate="2019-08-19T13:18:19.133" Score="0" ViewCount="170" Body="&lt;p&gt;After running a DRIMSeq pipeline and obtaining the genes that are differently used between the null and full models, how do I select the genes that are differently used in the different conditions? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/DRIMSeq/inst/doc/DRIMSeq.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the vignette&lt;/a&gt;, I would have to use the &lt;code&gt;plotProportions&lt;/code&gt; function and say in the &lt;code&gt;group_variable&lt;/code&gt; the variable of my interest. What if I want a list of the genes that are differently used specifically in the variables of my interest? Checking every gene manually sounds exhausting. Help would be much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="5086" LastEditorUserId="57" LastEditDate="2019-08-19T14:47:56.197" LastActivityDate="2019-08-19T15:00:42.887" Title="After running a DRIMSeq pipeline, how do I know which genes are upregulated in the different conditions?" Tags="|rna-seq|bioconductor|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9234" PostTypeId="2" ParentId="9225" CreationDate="2019-08-19T13:26:19.763" Score="2" Body="&lt;p&gt;The &lt;code&gt;MergeSeurat&lt;/code&gt; command is from Seurat v2.4 and only accepts two objects as parameters. If you need to merge more than one you can first merge two, then merge the combined object with the third and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the newer Seurat v3.0 this is replaced by the &lt;code&gt;merge&lt;/code&gt; command that can have a named list of Seurat objects as input&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# merge two objects&#xA;merge(x = pbmc_small, y = pbmc_small)&#xA;# to merge more than two objects, pass one to x and a list of objects to y&#xA;merge(x = pbmc_small, y = c(pbmc_small, pbmc_small))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want to perform batch correction at the same time as merging the data you should have a look at the &lt;code&gt;IntegrateData&lt;/code&gt; functionality of Seurat v3.0, explained in the corresponding vignettes:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://satijalab.org/seurat/v3.0/immune_alignment.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/v3.0/immune_alignment.html&lt;/a&gt;&lt;br&gt;&#xA;&lt;a href=&quot;https://satijalab.org/seurat/v3.0/integration.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/v3.0/integration.html&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A similar workflow for Seurat v2.4 can be cound here:  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://satijalab.org/seurat/v2.4/immune_alignment.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/v2.4/immune_alignment.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2019-08-19T13:26:19.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9235" PostTypeId="2" ParentId="9223" CreationDate="2019-08-19T13:29:28.540" Score="0" Body="&lt;p&gt;For pandas you simply load the fasta names and fasta sequences into separate arrays. The &quot;fasta names&quot; array becomes the index in a pandas Series, whilst the sequence are the values. The dataframe will correctly re-associated the names with each sequence. You load all the files into a single dataframe and use the duplicated() function then write out. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest way to do this is concatenate all your files into one via shell ...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat file*.fasta &amp;gt; concat_file.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Python script (biopython + pandas) ...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;import pandas as pd&#xA;&#xA;path = &quot;/Users/username/location/concat_file.fa&quot;&#xA;outpath = &quot;/Users/username/Desktop/duplicates.fasta&quot;&#xA;fasta_name = []&#xA;fasta_seq = []&#xA;&#xA;for record in SeqIO.parse(path, &quot;fasta&quot;):&#xA;    fasta_name.append('&amp;gt;' + record.id)&#xA;    fasta_seq.append(record.seq)&#xA;df = pd.Series(fasta_seq, index = fasta_name)&#xA;df_dups = df.index.duplicated()&#xA;mydups = df[df_dups]&#xA;mydups.to_csv(outpath, sep=&quot;\n&quot;, index=True, header=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I checked it on a ~4000 sequence fasta file of unique sequence ids (dengue), duplicated the last sequence id + sequence and the only sequence output was the very last sequence. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-08-20T00:08:57.383" LastActivityDate="2019-08-20T00:08:57.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9236" PostTypeId="2" ParentId="9233" CreationDate="2019-08-19T15:00:42.887" Score="0" Body="&lt;p&gt;After running the test for differential transcript usage you get a list with gene- and transcript-level p-values that you can use to filter which differences are significant. The gene-level p-value tells you if there is differential usage of transcripts within a gene and the transcript-level p-value tells you wich transcripts actually are changing.  Therefor you will want to select significant genes for further investigation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plots are mainly used to visualize top hits as detailed in the vignette.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;res &amp;lt;- results(d)&#xA;# The data is ordered by p-value&#xA;res &amp;lt;- res[order(res$pvalue, decreasing = FALSE), ]&#xA;# The top hit is selected&#xA;top_gene_id &amp;lt;- res$gene_id[1]&#xA;plotProportions(d, gene_id = top_gene_id, group_variable = &quot;group&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However you should consider running the two-stage test that is described on page 16 of the vignette you are using to reduce the number of false positive hits. From this you will get another list of adjusted p-values that can be used as above.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2019-08-19T15:00:42.887" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9238" PostTypeId="2" ParentId="9229" CreationDate="2019-08-20T04:29:47.187" Score="6" Body="&lt;p&gt;According to the &lt;a href=&quot;http://samtools.github.io/hts-specs/VCFv4.3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;specs&lt;/a&gt; the ALT is not allowed to contain &lt;code&gt;-&lt;/code&gt;. While technically supported, 'N' as the alternative allele is sometimes used unnecessarily and this script can also fix those cases if one exchanges the two &amp;quot;-&amp;quot; within the fix_record function for &amp;quot;N&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Here is a little python (&amp;gt;=3.6) script that is able to fix it. It makes use of the pysam module. You need a reference file and this fasta file must be indexed by &lt;code&gt;samtools faidx reference.fa&lt;/code&gt;. &lt;strike&gt;Please note that it doesn't support multiple values in the ALT column for records that contain a &lt;code&gt;-&lt;/code&gt;.&lt;/strike&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Usage:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;python fixvcf.py -f reference.fasta -o out.vcf in.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;The script:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python&#xA;# -*- coding: utf-8 -*-&#xA;import argparse&#xA;import sys&#xA;&#xA;import pysam&#xA;&#xA;&#xA;class Application:&#xA;    def __init__(self):&#xA;        self.args = Application.get_args()&#xA;        self.vcf = pysam.VariantFile(sys.stdin)&#xA;        self.fasta = pysam.FastaFile(self.args.fasta)&#xA;&#xA;    @staticmethod&#xA;    def get_args() -&amp;gt; argparse.Namespace:&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        Read command line arguments and set input and output sources.&#xA;&#xA;        :return: parsed command line arguments&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        parser = argparse.ArgumentParser(prog=&amp;quot;fixvcf.py&amp;quot;, description=&amp;quot;Fixes deletion represented by -&amp;quot;)&#xA;        parser.add_argument('--version', action='version', version='%(prog)s 0.1')&#xA;&#xA;        parser.add_argument(&amp;quot;input&amp;quot;, help=&amp;quot;input file, use - to read from stdin&amp;quot;)&#xA;        parser.add_argument(&amp;quot;-f&amp;quot;, &amp;quot;--fasta&amp;quot;, help=&amp;quot;fasta reference file&amp;quot;, required=True)&#xA;        parser.add_argument(&amp;quot;-o&amp;quot;, &amp;quot;--output&amp;quot;, help=&amp;quot;output file&amp;quot;)&#xA;&#xA;        if len(sys.argv) == 1:&#xA;            parser.print_help(sys.stderr)&#xA;            sys.exit(1)&#xA;&#xA;        args = parser.parse_args()&#xA;&#xA;        if args.output:&#xA;            sys.stdout = open(args.output, &amp;quot;w&amp;quot;)&#xA;&#xA;        if args.input != &amp;quot;-&amp;quot;:&#xA;            sys.stdin = open(args.input, &amp;quot;r&amp;quot;)&#xA;&#xA;        return args&#xA;&#xA;    def fix_record(self, record: pysam.libcbcf.VariantRecord) -&amp;gt; pysam.libcbcf.VariantRecord:&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        If the ALT column of the records contains a `-`, the variant position is shift one position to the left.&#xA;        For reaching this the reference base for the position is determined and prepended to the current REF bases.&#xA;        The ALT value `-` is replaced with the determined ref base on position before. In case there are multiple ALT&#xA;        values the determined ref base is prepended to them.&#xA;&#xA;        :param record: vcf file record&#xA;        :return: fixed vcf record&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        if &amp;quot;-&amp;quot; in record.alts:&#xA;            ref = self.fasta.fetch(region=f&amp;quot;{record.chrom}:{record.pos - 1}-{record.pos - 1}&amp;quot;)    &#xA;            record.pos = record.pos - 1&#xA;            record.ref = f&amp;quot;{ref}{record.ref}&amp;quot;&#xA;            record.alts = tuple(map(lambda alt: ref if alt == &amp;quot;-&amp;quot; else f&amp;quot;{ref}{alt}&amp;quot;, record.alts))&#xA;&#xA;        return record&#xA;&#xA;    def start(self):&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        Print the header of the vcf file. Afterwards the fix_record method is applied to each record in the vcf file.&#xA;&#xA;        For more information about python's map() see: https://docs.python.org/3.6/library/functions.html#map&#xA;&#xA;        For more information about the unpacking mechanism using `*`see:&#xA;        https://docs.python.org/3/tutorial/controlflow.html#tut-unpacking-arguments&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        print(self.vcf.header, end=&amp;quot;&amp;quot;)&#xA;        print(*map(self.fix_record, self.vcf), sep=&amp;quot;&amp;quot;, end=&amp;quot;&amp;quot;)&#xA;&#xA;&#xA;if __name__ == &amp;quot;__main__&amp;quot;:&#xA;    app = Application()&#xA;    app.start()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A lot of the code above is boilerplate for running a script on the command line. The critical code for the transformation is in the &lt;code&gt;fix_record&lt;/code&gt; function. The relevant code is below, expanded, with line numbers and a description of each command.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[1]   ref = self.fasta.fetch(&#xA;          region=f&amp;quot;{record.chrom}:{record.pos - 1}-{record.pos - 1}&amp;quot;&#xA;      )&#xA;[2]   record.pos = record.pos - 1&#xA;[3]   record.ref = f&amp;quot;{ref}{record.ref}&amp;quot;&#xA;[4]   record.alts = tuple(&#xA;          map(&#xA;              lambda alt: ref if alt == &amp;quot;-&amp;quot; else f&amp;quot;{ref}{alt}&amp;quot;,&#xA;              record.alts&#xA;          )&#xA;      )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;This grabs the base preceding the inserted/deleted sequence.&lt;/li&gt;&#xA;&lt;li&gt;This corrects the position of the variant to account for prepending another nucleotide.&lt;/li&gt;&#xA;&lt;li&gt;This defines the new corrected REF allele: concatenate the base grabbed in [1] with the original REF allele&lt;/li&gt;&#xA;&lt;li&gt;Apply an anonymous (lambda) function to each ALT allele; the function will replace &lt;code&gt;-&lt;/code&gt; with the base grabbed in [1], or preprend the base to the allele if it is not &lt;code&gt;-&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1310" LastEditorUserId="16819" LastEditDate="2023-01-25T11:30:21.040" LastActivityDate="2023-01-25T11:30:21.040" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9240" PostTypeId="1" CreationDate="2019-08-20T17:18:38.287" Score="1" ViewCount="130" Body="&lt;p&gt;I am using &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/ComplexHeatmap.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ComplexHeatmap&lt;/a&gt; package for transcriptomic analyses and all of a sudden,  &lt;code&gt;row_labels&lt;/code&gt; and &lt;code&gt;column_labels&lt;/code&gt; as well as &lt;code&gt;heatmap_width&lt;/code&gt; and &lt;code&gt;heatmap_height&lt;/code&gt; stopped being recognized as arguments of the function Heatmap:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;actual Heatmap&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Heatmap(matrix [RowFunctionOrder,ColumnImposedOrder],&#xA;        col = colgrad,&#xA;        show_column_names = F,&#xA;        cluster_columns = FALSE,&#xA;        cluster_rows = FALSE,&#xA;        row_names_gp = gpar(fontsize=7),&#xA;        heatmap_width=1,&#xA;        heatmap_height = 1, &#xA;        row_labels =  rnaseqMatrix$Protein.ID,&#xA;        bottom_annotation = Column_ha&#xA;        )&#xA;#creating a column annotation&#xA;&#xA;Column_ha = HeatmapAnnotation(condition = colnames(matrix),condition = runif(30), col = list(condition=Colcolour), show_legend = FALSE).&#xA;&#xA;&#xA;*Error in Heatmap(matrix[RowFunctionOrder, ColumnImposedOrder], col = colgrad,  : &#xA;  unused arguments (heatmap_width = 1, heatmap_height = 1, row_labels = rnaseqMatrix$Protein.ID)*.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;this code was perfectly working this morning, I do not understand why suddenly the arguments were not valid anymore. they also seem to have disappeared from the help menu. this event followed a bug where I was receiving messaged saying the &lt;code&gt;help(ComplexHeatmap) could't be update.&lt;/code&gt; this messages started appearing when I started typing arguments.&lt;/p&gt;&#xA;" OwnerUserId="5093" LastEditorUserId="57" LastEditDate="2019-08-21T13:35:45.563" LastActivityDate="2019-08-21T13:35:45.563" Title="Argument stopped working" Tags="|r|heatmap|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9241" PostTypeId="1" CreationDate="2019-08-21T01:47:57.677" Score="0" ViewCount="430" Body="&lt;p&gt;I have a list of genes and proteins extracted from literature. How can I convert them into standard gene names. &lt;/p&gt;&#xA;" OwnerUserId="5096" LastActivityDate="2019-10-27T22:46:46.450" Title="How to obtain the standard gene name" Tags="|database|proteins|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9242" PostTypeId="2" ParentId="9241" CreationDate="2019-08-21T02:08:35.263" Score="-2" Body="&lt;p&gt;bioDBnet might have what you're looking for if the gene name is in standard format&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://biodbnet-abcc.ncifcrf.gov/db/db2db.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biodbnet-abcc.ncifcrf.gov/db/db2db.php&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For proteins Uniprot can be used to look up a protein and find it's information&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.uniprot.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5074" LastActivityDate="2019-08-21T02:08:35.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9243" PostTypeId="1" CreationDate="2019-08-21T13:13:57.387" Score="1" ViewCount="838" Body="&lt;p&gt;When a rule fails in snakemake workflow, it returns a descriptive message in log file like following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in rule step1:&#xA;    jobid: 99&#xA;    output: results/foo/step1/sample5/outdir/output1.tsv, results/foo/step1/sample5/outdir/output2.tsv&#xA;    log: logs/step1/sample5_out.log (check log file(s) for error message)&#xA;    shell:&#xA;    ...&#xA;        (exited with non-zero exit code)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since I run workflow over many samples, I have defined a &lt;em&gt;postrun.sh&lt;/em&gt; bash script in &lt;em&gt;jobscript.sh&lt;/em&gt; directive that runs after snakemake &lt;code&gt;{exec_job}&lt;/code&gt; section and notifies me via userbot in slack channel when a step failed for one or more samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;jobscript.sh&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;        ########################### START SNAKEMAKE CMD ############################&#xA;    {exec_job}&#xA;        ############################ END SNAKEMAKE CMD #############################&#xA;    exitstat=$?&#xA;    export exitstat&#xA;&#xA;    echo &quot;END at &lt;span class=&quot;math-container&quot;&gt;$(date)&quot;&#xA;. &quot;$&lt;/span&gt;{{HOME}}&quot;/.smk_confs/postrun.sh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;postrun.sh&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;## execute after running each of snakemake job&#xA;echo 'Running postrun.sh'&#xA;ERRMSG=&quot;MYJOB ID: &lt;span class=&quot;math-container&quot;&gt;$PBS_JOBID failed at $&lt;/span&gt;(pwd) on &lt;span class=&quot;math-container&quot;&gt;$(hostname) for $&lt;/span&gt;{USER} with exit status: ${exitstat}.&quot;&#xA;&#xA;## ping to slack&#xA;ping_slack -m &quot;$ERRMSG&quot;&#xA;sleep 5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am looking for a snakemake exported environment variable that I can use in &lt;em&gt;postrun.sh&lt;/em&gt; to know which rule ID failed for one or more samples. For now, I can think of using job scheduler variable matching job name, e.g, PBS_JOBNAME, but better if I have a variable from snakemake API for rule ID or preferably, path to log file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="2978" LastActivityDate="2019-08-21T15:00:44.933" Title="snakemake get environment variable for rule name and log path" Tags="|snakemake|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9244" PostTypeId="2" ParentId="89" CreationDate="2019-08-21T14:58:38.243" Score="1" Body="&lt;p&gt;If the accessions have been interbred (conflicting with a tree-like phylogeny), you could try Splitstree, which tries to estimate a tree-like phylogeny with added hybridisation events.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-08-21T14:58:38.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9245" PostTypeId="2" ParentId="9243" CreationDate="2019-08-21T15:00:44.933" Score="1" Body="&lt;p&gt;You could add an exit trap to the beginning of the shell scripts inside your Snakefile, which is executed after each script is done and sends a message when there was an error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; shell:&#xA;    &quot;&quot;&quot;&#xA;    trap 'if [ $? -neq 0 ]; then ping_slack -m &quot;Error in rule xxx with input {input}&quot;; exit 1' EXIT&#xA;    ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1802" LastActivityDate="2019-08-21T15:00:44.933" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9246" PostTypeId="2" ParentId="89" CreationDate="2019-08-21T15:08:50.747" Score="0" Body="&lt;p&gt;Wait, you are right, but trees don't work here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the area of population genetics. You will generate allele frequencies for each locus. The allele frequenceies will be used to perform Hardy-Weinberg equilibrium, F statistics, Fst and Fis&lt;/p&gt;&#xA;&#xA;&lt;p&gt;H-W start by using Structure API to work out how many populations you have. This is vital information for the next two analyses:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Fis is particularly useful here because it is the inbreeding co-efficient, the nearer 1 it is the more inbreeding the data has been through. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The Fst distances can be used to generate a tree, but this is not a model of point mutations, rather a model of migration between populations. You can input the distance matrix in MEGA and make a tree from it, Fstat API (I think) will do it for you.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Pop gen is difficult to get your head around at first, but you'll get use to it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-08-21T15:08:50.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9247" PostTypeId="1" CreationDate="2019-08-21T16:37:42.883" Score="0" ViewCount="732" Body="&lt;p&gt;I have barcodes.tsv, genes.tsv and matrix.mtx file from my RNA seq data and I want to generate a scRNA counts files using R. When I tried using the &lt;code&gt;readMM&lt;/code&gt; function, it gave me the following error: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error: cannot allocate vector of size 185.1 Gb. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This happens even when I run it as a job. How can I generate a combined csv file using R?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="5098" LastEditorUserId="298" LastEditDate="2019-08-21T17:16:18.197" LastActivityDate="2019-10-25T21:02:50.340" Title="how to generate an scRNA counts file" Tags="|scrnaseq|seurat|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9248" PostTypeId="2" ParentId="9191" CreationDate="2019-08-21T20:33:54.893" Score="2" Body="&lt;p&gt;There are very few real structures of soluble misfolded proteins, for the reasons described by jgreener. An example with a disulfide bond is beta-2-microglobulin, eg PDB entries 2XKS vs 2XKU. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Atomic-resolution structures of amyloid fibrils may be what you're looking for, such as 6GK3 for beta-2-microglobulin amyloid (which has its disulfide intact). A search for &quot;amyloid fibril&quot; at rcsb.org will find many structures. But many amyloid precursor proteins are intrinsically disordered, or are fragments of native proteins.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There's no list that I know of, but here's a recent review: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/30237470&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pubmed/30237470&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5099" LastActivityDate="2019-08-21T20:33:54.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9249" PostTypeId="1" AcceptedAnswerId="9251" CreationDate="2019-08-22T00:51:17.850" Score="5" ViewCount="428" Body="&lt;p&gt;I run the command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools mpileup -O -s -q20-B -Q20 -f hg19.fa -r chr1:569929-569931 myFile.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and get: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    569929  G   7   ...,,., EEEEEEE NTTVTTU 53,48,42,60,30,29,27&#xA;chr1    569930  C   6   ...,.,  EEAEEE  NTTTTU  54,49,43,31,30,28&#xA;chr1    569931  G   6   ...,.,  EEEEEE  NTTTTU  55,50,44,32,31,29&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am interested in &lt;strong&gt;position 569930&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I inspect the bam with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view myFile.bam chr1:569929-569931&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I see:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NS500355:NS500355:HHYN5BGXB:2:21311:2303:9255   99  chr1    569869  0   76M =   569900  107 CTCCATAACGCTCCTCATACTAGGCCTACTAACCAACACACTAACCATATACCAATGATGGCGCGATGTAACACGA    AAAAAEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE&amp;lt;EEEEEEEAEEEEEEE    NM:i:2  MD:Z:5C3C66 MC:Z:76M    AS:i:66 XS:i:71 RG:Z:../H2O XA:Z:chrM,+9321,76M,1;&#xA;NS500355:NS500355:HHYN5BGXB:4:22603:25857:2799  99  chr1    569877  45  76M =   569896  95  CGCTCCTCATACTAGGCCTACTAACCAACACACTAACCATATACCAATGATGGCGCGATGTAACACGAGAAAGCAC    AAAAAEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE    NM:i:1  MD:Z:1C74   MC:Z:76M    AS:i:74 XS:i:71 RG:Z:../H2O XA:Z:chrM,+9329,76M,1;&#xA;NS500355:NS500355:HHYN5BGXB:1:12105:11868:12011 99  chr1    569882  51  76M =   569900  94  CTCATACTAGGCCTACTAACCAACACACTAACCATATACCAATGATGGCGCGATGTAACACGAGAAAGCACATACC    /AAAAEEEEEAEEEEEEEEEEEEEEEEEEEEEAEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEAEEEEEA    NM:i:0  MD:Z:76 MC:Z:76M    AS:i:76 XS:i:71 RG:Z:../H2O XA:Z:chrM,+9334,76M,1;&#xA;NS500355:NS500355:HHYN5BGXB:2:13301:2681:6114   99  chr1    569888  9   76M =   569952  140 CTAGGCATACTAACCAACACACTAACAATATACCAATGATGGAGCGATGTAACACGAGAAAGCACATACCAACGCC    AAAA/E/EE//AAEEEEE/EAAEAA//EEAEEE/E/EEEE/////A/EE/EE/EE/E//EE/EEEE/E/AEA####    NM:i:4  MD:Z:6C19C15C29G3   MC:Z:76M    AS:i:57 XS:i:52 RG:Z:../H2O XA:Z:chrM,+9340,76M,5;&#xA;NS500355:NS500355:HHYN5BGXB:2:23110:10125:9334  99  chr1    569888  51  76M =   569904  92  CTAGGCCTACTAACCAACACACTAACCATATACCAATGATGGCGCGATGTAACACGAGAAAGCACATACCAAGGCC    AAA/AEE6EEEE6E///E/A/AAE/E/AEEEEEEE/EE/EEEAEEEEAEE//EEEEEEAEEEAEEAE/AEE/AEE&amp;lt;    NM:i:0  MD:Z:76 MC:Z:76M    AS:i:76 XS:i:71 RG:Z:../H2O XA:Z:chrM,+9340,76M,1;&#xA;NS500355:NS500355:HHYN5BGXB:1:13209:10692:11045 83  chr1    569888  53  18S58M  =   569888  -58 CGTGTGCTCTTCCGATCTCTAGGCCTACTAACCAACACACTAACCATATACCAATGATGGCGCGATGTAACACGAG    EE&amp;lt;A/EE//AE/AA///AAE///EEEEE//6///E//EE/6EE/EEEEEEEA/A/A/E/E//AEAEEEEE6A6AAA    NM:i:0  MD:Z:58 MC:Z:59M17S AS:i:58 XS:i:53 RG:Z:../H2O XA:Z:chrM,-9340,18S58M,1;&#xA;NS500355:NS500355:HHYN5BGXB:1:21304:26480:15673 83  chr1    569890  11  76M =   569796  -170    AGGCCTACTAACCAACACACTAACCATATACCAATGATGGCGCGATGTAACACGAGAAAGCACATACCAAGGCCAC    AEEEEEEEAAEEEEEEEEEEEE/EEEEEEEEEEAEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEAAAAA    NM:i:0  MD:Z:76 MC:Z:76M    AS:i:76 XS:i:71 RG:Z:../H2O XA:Z:chrM,-9342,76M,1;&#xA;NS500355:NS500355:HHYN5BGXB:4:11602:21237:3028  83  chr1    569900  51  76M =   569900  -76 ACCAACACACTAACCATATACCAATGATGGCGCGATGTAACACGAGAAAGCACATACCAAGGCCACCACACACCAC    EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEAAAAA    NM:i:0  MD:Z:76 MC:Z:76M    AS:i:76 XS:i:71 RG:Z:../H2O XA:Z:chrM,-9352,76M,1;&#xA;NS500355:NS500355:HHYN5BGXB:3:13504:4369:8812   99  chr1    569901  51  76M =   569901  76  CCAACACACTAACCATATACCAATGATGGCGCGATGTAACACGAGAAAGCACATACCAAGGCCACCACACACCACC    AAAAAEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE6EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE    NM:i:0  MD:Z:76 MC:Z:76M    AS:i:76 XS:i:71 RG:Z:../H2O XA:Z:chrM,+9353,76M,1;&#xA;NS500355:NS500355:HHYN5BGXB:2:11203:19774:18704 99  chr1    569904  11  76M =   569952  124 ACACACTAACCATATACCAATGATGGCGCGATGTAACACGAGAAAGCACATACCAAGGCCACCACACACCACCTGT    6AAAAEEEEEEEEEEEEEAEEEEEEEEEEEEEEEE6EEEEAEEEAEEEEEEEEEEEEEEEEEEAE/E6EEAEEEEA    NM:i:0  MD:Z:76 MC:Z:76M    AS:i:76 XS:i:71 RG:Z:../H2O XA:Z:chrM,+9356,76M,1;&#xA;NS500355:NS500355:HHYN5BGXB:2:11309:13954:17567 99  chr1    569905  11  76M =   569972  143 CACACTAACCATATACCAATGATGGCGCGATGTAACACGAGAAAGCACATACCAAGGCCACCACACACCACCTGTC    AAAAAEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEAEEEEEEEEEEE    NM:i:0  MD:Z:76 MC:Z:76M    AS:i:76 XS:i:71 RG:Z:../H2O XA:Z:chrM,+9357,76M,1;&#xA;NS500355:NS500355:HHYN5BGXB:1:21202:13252:14049 99  chr1    569905  11  76M =   569927  98  CACACTAACCATATACCAATGATGGCGCGATGTAACACGAGAAAGCACATACCAAGGCCACCACACACCACCTGTC    AAAAAEE6EEEEEEEEEEEEEAEEEEEEEEEEEEEEEEEEE6EEEEEEEEEEEEEEEEE/EEEE&amp;lt;EEEEEEEEEEE    NM:i:0  MD:Z:76 MC:Z:76M    AS:i:76 XS:i:71 RG:Z:../H2O XA:Z:chrM,+9357,76M,1;&#xA;NS500355:NS500355:HHYN5BGXB:1:22311:21964:12072 83  chr1    569908  52  5S71M   =   569908  -71 GATCTACTAACCATATACCAATGATGGCGCGATGTAACACGAGAAAGCACATACCAAGGCCACCACACACCACCTG    &amp;lt;AEEEA6EEE6EEEEEEEEEEEEEE/EEEEEEEEEEEEEEEEE/EEEEEEEEEEEEEEEEEEAAEEEAEEEAA6AA    NM:i:0  MD:Z:71 MC:Z:71M5S  AS:i:71 XS:i:66 RG:Z:../H2O XA:Z:chrM,-9360,5S71M,1;&#xA;NS500355:NS500355:HHYN5BGXB:4:12602:7858:19418  99  chr1    569909  15  76M =   569917  84  CTAACCATATACCAATGATGGCGCGATGTAACACGAGAAAGCACATACCAAGGCCACCACACACCACCTGTCCAAA    AAAAAEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEAEE/AAEEEEEEEEEEEEEEE    NM:i:1  MD:Z:74G1   MC:Z:76M    AS:i:74 XS:i:71 RG:Z:../H2O XA:Z:chrM,+9361,76M,1;&#xA;NS500355:NS500355:HHYN5BGXB:2:12205:19456:7598  83  chr1    569927  0   76M =   569907  -96 TGGCGCGATGTAACACGAGAAAGCACATACCAAGGCCACCACACACCACCTGTCCAAAAAGGCCTTCGATACGGGA    EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEAA/EEEEEEEEEAAAAA    NM:i:1  MD:Z:56G19  MC:Z:76M    AS:i:71 XS:i:76 RG:Z:../H2O XA:Z:chrM,-9379,76M,0;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Based on mapq and positions (respect to 569930) I see that the read with &lt;code&gt;cigar=18S58M&lt;/code&gt; is skipped by mpileup.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;WHY?&lt;/strong&gt; The soft clip is way far from position chr1:569930 &lt;/p&gt;&#xA;" OwnerUserId="2970" LastActivityDate="2019-08-22T11:02:40.700" Title="samtools mpileup skipping read" Tags="|samtools|mpileup|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9250" PostTypeId="2" ParentId="9249" CreationDate="2019-08-22T04:20:27.143" Score="3" Body="&lt;p&gt;Why do you believe it's the read with &lt;code&gt;cigar=18S58M&lt;/code&gt;? The mapping qualities given in sam files are different to those in the mpileup output. I guess samtools is doing some recalculation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are using the parameters &lt;code&gt;-q20&lt;/code&gt; &lt;code&gt;-Q20&lt;/code&gt;. So you skip all reads with mapping quality &amp;lt;20. This will result in 7 reads got to mpileup. Furthermore you are skipping bases with base quality &amp;lt;20. Have a look at the base qualities for the read starting at position &lt;code&gt;569904&lt;/code&gt;. If I count it correct you end up with a quality of &lt;code&gt;/&lt;/code&gt; at position &lt;code&gt;569930&lt;/code&gt; which means &lt;code&gt;14&lt;/code&gt;. So I guess this read is dropped.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strike&gt;BTW: &lt;code&gt;samtools mpileup&lt;/code&gt; is deprecated&lt;/strike&gt; (only when creating bcf/vcf) &lt;strike&gt; and should be replaced by &lt;code&gt;bcftools mpileup&lt;/code&gt;.&lt;/strike&gt; &lt;/p&gt;&#xA;" OwnerUserId="1310" LastEditorUserId="298" LastEditDate="2019-08-22T11:02:40.700" LastActivityDate="2019-08-22T11:02:40.700" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9251" PostTypeId="2" ParentId="9249" CreationDate="2019-08-22T09:16:48.480" Score="3" Body="&lt;p&gt;Examining the mpileup output for a single read on its own is a good way to figure out why bases are not appearing as expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you suspect that the &lt;code&gt;cigar=18S58M&lt;/code&gt; read is the missing one (you are correct), prepare a SAM file containing just that read:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@SQ SN:chr1 LN:248956422&#xA;NS500355:NS500355:HHYN5BGXB:1:13209:10692:11045 83  chr1    569888  53  18S58M  =   569888  -58 CGTGTGCTCTTCCGATCTCTAGGCCTACTAACCAACACACTAACCATATACCAATGATGGCGCGATGTAACACGAG    EE&amp;lt;A/EE//AE/AA///AAE///EEEEE//6///E//EE/6EE/EEEEEEEA/A/A/E/E//AEAEEEEE6A6AAA    NM:i:0  MD:Z:58 MC:Z:59M17S AS:i:58 XS:i:53RG:Z:../H2O  XA:Z:chrM,-9340,18S58M,1;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and run it through &lt;code&gt;mpileup&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ samtools mpileup -Q0 -B oneread.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will show you that this read's base quality dips below 20 at the position you are interested in. Thus, as finswimmer suspected, your &lt;code&gt;-Q20&lt;/code&gt; is removing it at this position.&lt;/p&gt;&#xA;" OwnerUserId="134" LastActivityDate="2019-08-22T09:16:48.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9252" PostTypeId="2" ParentId="9247" CreationDate="2019-08-22T12:51:41.743" Score="0" Body="&lt;p&gt;You probably need to load the matrices part by part in order to avoid loading a large size vector. One way is to use features like &lt;code&gt;nlines&lt;/code&gt; or &lt;code&gt;skip&lt;/code&gt; of the &lt;a href=&quot;https://www.rdocumentation.org/packages/base/versions/3.6.1/topics/scan&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;scan()&lt;/code&gt;&lt;/a&gt; function. Also, this is suggested code by &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/output/matrices&quot; rel=&quot;nofollow noreferrer&quot;&gt;10xgenomics&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Matrix)&#xA;matrix_dir = &quot;/opt/sample345/outs/filtered_feature_bc_matrix/&quot;&#xA;barcode.path &amp;lt;- paste0(matrix_dir, &quot;barcodes.tsv.gz&quot;)&#xA;features.path &amp;lt;- paste0(matrix_dir, &quot;features.tsv.gz&quot;)&#xA;matrix.path &amp;lt;- paste0(matrix_dir, &quot;matrix.mtx.gz&quot;)&#xA;mat &amp;lt;- readMM(file = matrix.path)&#xA;feature.names = read.delim(features.path, &#xA;                           header = FALSE,&#xA;                           stringsAsFactors = FALSE)&#xA;barcode.names = read.delim(barcode.path, &#xA;                           header = FALSE,&#xA;                           stringsAsFactors = FALSE)&#xA;colnames(mat) = barcode.names&lt;span class=&quot;math-container&quot;&gt;$V1&#xA;rownames(mat) = feature.names$&lt;/span&gt;V1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5103" LastActivityDate="2019-08-22T12:51:41.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9253" PostTypeId="2" ParentId="8965" CreationDate="2019-08-22T13:03:43.063" Score="3" Body="&lt;p&gt;Your cluster labels come from graph clustering implemented in the &lt;code&gt;FindClusters()&lt;/code&gt; function. The resulting clusters are then visualised with a 2D tSNE plot (via &lt;code&gt;RunTSNE()&lt;/code&gt; and &lt;code&gt;TSNEPlot()&lt;/code&gt;). So, your cluster 13 is not split into three sub-clusters but cells within cluster 13 &lt;strong&gt;&lt;em&gt;look&lt;/em&gt;&lt;/strong&gt; somewhat &lt;strong&gt;&lt;em&gt;distant&lt;/em&gt;&lt;/strong&gt; from each other on the tSNE plot. Having mentioned distances on a tSNE plot, please see this SE Data Science entry &lt;a href=&quot;https://datascience.stackexchange.com/a/10820&quot;&gt;Can closer points be considered more similar in T-SNE visualization?&lt;/a&gt; and especially the comment &lt;strong&gt;&lt;em&gt;&quot;What I mean is that you can't just use distance in the lower space as a similarity criterion&quot;&lt;/em&gt;&lt;/strong&gt;. In your case &quot;the lower space&quot; would be your tSNE plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I can see from your code, you are following the widely-adopted &quot;Seurat workflow&quot; and all seem fine including your TSNE plot output. I could not spot the QC step though: apart from the one at object initiation, one usually applies filters for number of genes and % mitochondrial gene contribution (using &lt;code&gt;FilterCells()&lt;/code&gt; in Seurat v2 and &lt;code&gt;subset()&lt;/code&gt; in Seurat v3).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be a good idea to try UMAP instead of tSNE, please see &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/298430v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evaluation of UMAP as an alternative to t-SNE for single-cell data (Becht et al, 2019)&lt;/a&gt;. Seurat v3 vignettes are now using UMAP and not tSNE any more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, please see &lt;a href=&quot;https://distill.pub/2016/misread-tsne/&quot; rel=&quot;nofollow noreferrer&quot;&gt;How to Use t-SNE Effectively&lt;/a&gt; to learn about points to take into account when using/interpreting tSNE plots. You will see that you would get surprisingly dissimilar tSNE plots from the same underlying data with different parameters (some are hardcoded in the &lt;code&gt;RunTSNE()&lt;/code&gt; function).&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-08-22T13:03:43.063" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9254" PostTypeId="2" ParentId="4851" CreationDate="2019-08-22T21:16:46.600" Score="3" Body="&lt;p&gt;In case it is still helpful as the post is rather old, the code below would generate a heatmap with annotations thanks to the &lt;code&gt;ComplexHeatmap&lt;/code&gt; package.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But before that I would like to stress out the importance of example data, I bet a lot of experts could not help you just because of they did not have single cell data at their disposal.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using some scRNA-seq data stored within a &lt;code&gt;Seurat&lt;/code&gt; object. The expression data come from the &lt;code&gt;@scale.data&lt;/code&gt; slot and the annotations are from the &lt;code&gt;@meta.data&lt;/code&gt; slot. I am using a few general cell type markers as genes and 3 columns from the metadata slot (cell type, cell subtype, sample region). When using the &lt;code&gt;ComplexHeatmap&lt;/code&gt; package, the trick is to have the same order in the annotation and actual data tables and this is how you should control what (in your case single cells) is plotted in what order.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Seurat)&#xA;library(dplyr)&#xA;library(ComplexHeatmap)&#xA;library(circlize) # for the colorRamp2() function&#xA;&#xA;seurat_object &amp;lt;- readRDS(...)&#xA;&#xA;# Annotations&#xA;&#xA;meta_data &amp;lt;- seurat_object@meta.data&#xA;&#xA;# order of annotations/colors are defined here&#xA;ordered_meta_data &amp;lt;- meta_data[order(meta_data$cell_type), ]&#xA;&#xA;# OPTIONAL: YOU CAN PICK COLORS FOR EACH LEVEL OF ANNOTATION&#xA;# HERE I PROVIDE COLORS FOR ONLY TWO OF THE THREE LEVELS, COLORS FOR THE&#xA;# REMAINING LEVEL IS TAKEN CARE OF BY THE PACKAGE&#xA;annotation_colors &amp;lt;- list(&quot;cell_type&quot;= c(&quot;Epithelial&quot; = &quot;red&quot;,&#xA;                                         &quot;Immune&quot; = &quot;orange&quot;,&#xA;                                         &quot;Other&quot; = &quot;gray&quot;,&#xA;                                         &quot;Stroma&quot; = &quot;green&quot;),&#xA;                          &quot;sample_region&quot; = c(&quot;B&quot; = &quot;blue&quot;,&#xA;                                              &quot;C&quot; = &quot;yellow&quot;,&#xA;                                              &quot;N&quot; = &quot;maroon&quot;,&#xA;                                              &quot;T&quot; = &quot;darkblue&quot;))&#xA;ha = HeatmapAnnotation(df = ordered_meta_data,&#xA;                       show_annotation_name = TRUE,&#xA;                       col = annotation_colors)&#xA;&#xA;# Expression data&#xA;&#xA;genes_to_use &amp;lt;- c(&quot;EPCAM&quot;,&#xA;                  &quot;CD3D&quot;,&#xA;                  &quot;NCR1&quot;,&#xA;                  &quot;CD79A&quot;,&#xA;                  &quot;LYZ&quot;,&#xA;                  &quot;COL1A1&quot;,&#xA;                  &quot;CLDN5&quot;,&#xA;                  &quot;S100B&quot;,&#xA;                  &quot;KIT&quot;)&#xA;&#xA;seurat_object &amp;lt;- ScaleData(seurat_object,&#xA;                           genes.use = genes_to_use)&#xA;&#xA;my_data &amp;lt;- seurat_object@scale.data&#xA;&#xA;# COLUMN ORDER OF THE EXPRESSION DATA SHOULD MATCH THE ROW ORDER OF THE&#xA;# ANNOTATION TABLE&#xA;my_data &amp;lt;- my_data[, rownames(ordered_meta_data)]&#xA;&#xA;# Heatmap&#xA;&#xA;col_fun = colorRamp2(c(-2, 0, 2), c(&quot;blue&quot;, &quot;white&quot;, &quot;red&quot;))&#xA;png(filename = &quot;example_heatmap.png&quot;,&#xA;    width = 1000,&#xA;    height = 1000)&#xA;Heatmap(&#xA;  my_data,&#xA;  col = col_fun,&#xA;  cluster_rows = FALSE,&#xA;  cluster_columns = FALSE,&#xA;  column_order = NULL,&#xA;  show_row_dend = FALSE,&#xA;  show_column_dend = FALSE,&#xA;  show_row_names = TRUE,&#xA;  show_column_names = FALSE,&#xA;  use_raster = TRUE,&#xA;  raster_device = c(&quot;png&quot;),&#xA;  bottom_annotation = NULL,&#xA;  top_annotation = ha&#xA;)&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qBjYH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qBjYH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-08-22T21:16:46.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9255" PostTypeId="1" CreationDate="2019-08-23T06:39:13.557" Score="0" ViewCount="21" Body="&lt;p&gt;How to upload and process (de novo assembly pipe) a data set to Galaxy instance and retrieve the results programmatically? Are there any packages beyond BioBlend?&lt;/p&gt;&#xA;" OwnerUserId="5106" LastActivityDate="2019-08-23T07:20:58.937" Title="How to upload and process a data set to Galaxy instance and retrieve results programmatically?" Tags="|api|galaxy|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9257" PostTypeId="2" ParentId="9255" CreationDate="2019-08-23T07:20:58.937" Score="1" Body="&lt;p&gt;You best bet is bioblend, since it's a more convenient wrapper around the &lt;a href=&quot;https://galaxyproject.org/develop/api/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Galaxy API&lt;/a&gt;, but if you really want to you could use that directly. I should note that large files should probably still be uploaded via ftp or sftp rather than via the http(s) interface. &lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-08-23T07:20:58.937" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9258" PostTypeId="1" CreationDate="2019-08-23T07:46:25.190" Score="0" ViewCount="664" Body="&lt;p&gt;I write under code but I can't run it when I run by g++ . erro is about  strlen.&lt;br&gt;&#xA;errors: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a.cpp: In function ‘bool checker(char*, int)’:&#xA;a.cpp:185:26: error: ‘str’ is not a member of ‘std’&#xA; 185 |  for(int i = 0; i &amp;lt; std::str.length(word); i++)&#xA;  |                          ^~~&#xA;a.cpp: In function ‘int status(char*, int, int)’:&#xA;a.cpp:192:40: error: ‘str’ is not a member of ‘std’&#xA;   192 |  for(int i = 0; i &amp;lt; 4 &amp;amp;&amp;amp; nw*4+i &amp;lt; std::str.length(pmt[pn]); i++)&#xA;  |                                        ^~~&#xA;a.cpp:199:10: error: ‘str’ is not a member of ‘std’&#xA;  199 |  if(std::str.length(pmt[pn]) &amp;gt; nw*4)&#xA;  |          ^~~&#xA;a.cpp: In function ‘int main()’:&#xA;a.cpp:225:22: error: ‘str’ is not a member of ‘std’&#xA;  225 |     len_words = std::str.length(pmt[i])/2;&#xA;    |                      ^~~&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4456" LastEditorUserId="4456" LastEditDate="2019-08-23T19:17:29.517" LastActivityDate="2019-08-24T22:31:59.940" Title="c++ program to find length string an char*" Tags="|c++|" AnswerCount="1" CommentCount="7" ClosedDate="2019-08-25T12:17:09.600" ContentLicense="CC BY-SA 4.0" />
  <row Id="9261" PostTypeId="1" CreationDate="2019-08-23T14:06:33.360" Score="3" ViewCount="79" Body="&lt;p&gt;I want to use the single-cell data from the following expression profiling which concerns the RNA-seq of metastatic melanoma:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE72056&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE72056&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The data was also analyzed here,&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4944528/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4944528/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However I cannot find the mapping between patient and single-cell samples anywhere, neither in the paper nor in the supplementary materials. Anybody has an experience with this data?&lt;/p&gt;&#xA;&lt;p&gt;Note that in the Supp1 (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4944528/bin/NIHMS791837-supplement-Supp1.doc&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4944528/bin/NIHMS791837-supplement-Supp1.doc&lt;/a&gt;) from the paper, at Table S1 (almost at the end of the file) that represents the mapping between tumor and patient, which is a one-to-one mapping. But no mapping between samples and tumors found.&lt;/p&gt;&#xA;" OwnerUserId="5109" LastEditorUserId="3848" LastEditDate="2022-02-23T00:19:27.637" LastActivityDate="2022-02-23T00:19:27.637" Title="Patient-sample mapping in GSE72056 dataset" Tags="|rna-seq|read-mapping|cancer|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9262" PostTypeId="1" AcceptedAnswerId="9274" CreationDate="2019-08-23T17:54:23.443" Score="2" ViewCount="65" Body="&lt;p&gt;Given a set of bigBed files, each of which are disjoint to each other and in lexicographical (&quot;sort-bed&quot;) sort order, I would like to know if there is a way to concatenate these archives efficiently.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presuming I know the order in which to specify files (to constrain the order of intervals), is there a way to concatenate them directly, &lt;em&gt;i.e.,&lt;/em&gt; without first extracting them to bed, concatenating the extracted files, and remaking a bigBed file?&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2019-08-26T07:18:14.067" Title="Efficiently concatenating disjoint, ordered bigBed files" Tags="|ucsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9263" PostTypeId="1" AcceptedAnswerId="9265" CreationDate="2019-08-23T18:36:07.553" Score="1" ViewCount="58" Body="&lt;p&gt;Forgive the simple noob question&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have TPM data of ~50k genes (rows) across ~1k cell lines (columns). In R, I would like to output an &quot;intermediate expression&quot; gene list for each cell line, like:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt;head(finalExpressedGeneList, n=3)&#xA;&#xA;Celltype1    Celltype2    CelltypeN&#xA;gene1        gene5        gene3&#xA;gene3        gene6        gene6&#xA;etc          etc          etc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am defining intermediate expression as &quot;gene expression &gt;= median value within each cell line.&quot; I have already replaced all 0 values with NA using:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt;data[data == 0] &amp;lt;- NA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I then got medians for each column using:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt;colMed &amp;lt;- apply(data,2, FUN = median, na.rm = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now I need to somehow filter the rows from each column and copy the associated gene into a new matrix. I started building the code below:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt;if (data[,1]&amp;gt;=colMed [1]) &#xA;   {&#xA;   finalExpressedGeneList &amp;lt;- row.names(data)&#xA;   }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I realized that even if I got it working, I would still have to iterate over all of my columns. I'm sure that there is a cleaner way to do this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT (adding example data)&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;#create example data&#xA;&amp;gt;j = 5 &#xA;&amp;gt;M &amp;lt;- matrix(NA,j,j)&#xA;&amp;gt;M &amp;lt;- sapply(1:j, function(i) `length&amp;lt;-`(1:(j-i+1), j))&#xA;&amp;gt;rownames(M) &amp;lt;- c(&quot;gene1&quot;, &quot;gene2&quot;, &quot;gene3&quot;, &quot;gene4&quot;, &quot;gene5&quot;)&#xA;&amp;gt;colnames(M) &amp;lt;- c(&quot;cellLine1&quot;, &quot;cellLine2&quot;, &quot;cellLine3&quot;, &quot;cellLine4&quot;, &quot;cellLine5&quot;)&#xA;&amp;gt;view(M)&#xA;&#xA;       cellLine1   cellLine2   cellLine3   cellLine4   cellLine5&#xA;gene1          1           1           1           1           1&#xA;gene2          2           2           2           2          NA &#xA;gene3          3           3           3          NA          NA&#xA;gene4          4           4          NA          NA          NA&#xA;gene5          5          NA          NA          NA          NA&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to get an output of:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&#xA;cellLine1   cellLine2   cellLine3   cellLine4   cellLine5&#xA;    gene3       gene3       gene2       gene2       gene1&#xA;    gene4       gene4       gene3&#xA;    gene5&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;because the median of cellLine1 has three genes greater than or equal to the median of that column (3), cellLine2 has two genes that are greater than or equal to the median of that column (2.5), etc...&lt;/p&gt;&#xA;" OwnerUserId="5110" LastEditorUserId="5110" LastEditDate="2019-08-26T13:39:49.407" LastActivityDate="2019-08-26T14:48:14.043" Title="How do I create a filtered gene list using expression medians" Tags="|r|rna-seq|filtering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9265" PostTypeId="2" ParentId="9263" CreationDate="2019-08-24T07:02:03.533" Score="0" Body="&lt;p&gt;This below should work. It returns a list,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;## example data&#xA;j = 5 &#xA;M &amp;lt;- matrix(NA,j,j)&#xA;M &amp;lt;- sapply(1:j, function(i) `length&amp;lt;-`(1:(j-i+1), j))&#xA;rownames(M) &amp;lt;- c(&quot;gene1&quot;, &quot;gene2&quot;, &quot;gene3&quot;, &quot;gene4&quot;, &quot;gene5&quot;)&#xA;colnames(M) &amp;lt;- c(&quot;cellLine1&quot;, &quot;cellLine2&quot;, &quot;cellLine3&quot;, &quot;cellLine4&quot;, &quot;cellLine5&quot;)&#xA;#####&#xA;genelist = apply(M,2,function(i)rownames(M)[which(i&amp;gt;=median(i,na.rm=T))])&#xA;&#xA;####&#xA;#you can call the genes for cellline 1 using:&#xA;genelist$cellLine1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Hope I got it correct!&lt;/p&gt;&#xA;" OwnerUserId="5077" LastEditorUserId="5077" LastEditDate="2019-08-26T14:48:14.043" LastActivityDate="2019-08-26T14:48:14.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9266" PostTypeId="2" ParentId="9258" CreationDate="2019-08-24T22:31:59.940" Score="1" Body="&lt;p&gt;I think you're confusing C++ and C strings, which are very different. When you have a &lt;code&gt;char *&lt;/code&gt;, that's a C-style &quot;string&quot;. It's really just a bunch of bytes with a zero byte at the end, demarcating the end of the string. To find the length of a C string you include &lt;code&gt;&amp;lt;cstring&amp;gt;&lt;/code&gt; (or &lt;code&gt;&amp;lt;string.h&amp;gt;&lt;/code&gt; in C) and call &lt;code&gt;strlen&lt;/code&gt;, like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/** Print every character in the string */&#xA;void print_chars_in_string(char* word) {&#xA;    for (size_t i = 0; i &amp;lt; strlen(word); ++i) {&#xA;        putchar(word[i]);&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;strlen&lt;/code&gt; is actually pretty simple though, you can write your own version as an academic exercise like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;size_t deluxe_bespoke_strlen(char* str) {&#xA;    size_t length = 0;&#xA;&#xA;    while (*str++)&#xA;        ++length;&#xA;&#xA;    return length;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since the while-loop condition will be true until you hit the null terminator (the zero byte at the end), this is how you would implement the strlen function without a for-loop, otherwise you would need the strlen function for the strlen function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In C++, strings are actual objects, with methods you can call. To get the length of a C++ string simply do this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;std::string my_name = &quot;Jose&quot;;&#xA;std::cout &amp;lt;&amp;lt; my_name.length() &amp;lt;&amp;lt; std::endl;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you have a C string, you can convert it to a C++ string with the built-in constructor:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;const char* my_name = &quot;Jose&quot;;&#xA;std::string my_name_cppstring = std::string(my_name);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can also get the C version of a C++ string by calling the &lt;code&gt;c_str()&lt;/code&gt; member function like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;std::string sequence = &quot;GATTACA&quot;;&#xA;size_t length = strlen(sequence.c_str());&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My recommendation is to not mix C and C++. C is my favorite programming language, but that doesn't mean everyone should write everything in C. Since you're using C++ I'm guessing that's your preference, so I would suggest using C++ strings instead of C strings, just remember to include &lt;code&gt;&amp;lt;string&amp;gt;&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;hr /&gt;&#xA;&#xA;&lt;p&gt;As for your specific question, the compiler is giving you the correct message: &lt;code&gt;str&lt;/code&gt; is not a member of &lt;code&gt;std&lt;/code&gt;. You don't add the namespace to variables. This is how that works:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;cstdlib&amp;gt;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;&#xA;namespace CustomDeluxeBespokeStandardNamespace&#xA;{&#xA;    size_t custom_bespoke_strlen(char* str)&#xA;    {&#xA;        size_t length = 0;&#xA;&#xA;        while (*str++)&#xA;            ++length;&#xA;&#xA;        return length;&#xA;    }&#xA;}&#xA;&#xA;int main(int argc, char *argv[])&#xA;{&#xA;    if (argc == 1) {&#xA;        std::cerr &amp;lt;&amp;lt; &quot;No input(s).&quot; &amp;lt;&amp;lt; std::endl;&#xA;        std::exit(EXIT_FAILURE);&#xA;    }&#xA;&#xA;    while (*++argv) {&#xA;        std::cout &amp;lt;&amp;lt; *argv &amp;lt;&amp;lt; &quot;: &quot; &amp;lt;&amp;lt; CustomDeluxeBespokeStandardNamespace::custom_bespoke_strlen(*argv) &amp;lt;&amp;lt; std::endl;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So if you run it like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;testprogram horse house mouse cat&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You should get this output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;horse: 5&#xA;house: 5&#xA;mouse: 5&#xA;cat: 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So to recap, you were adding the namespace to your variables, and the compiler was giving you an error because it correctly could not find anything by that name in the &lt;code&gt;std&lt;/code&gt; namespace. Even if it had though, C style strings do not have member functions. There are no such things as objects in C, at least not in the sense that you can define member functions for an object, so that was the second error.&lt;/p&gt;&#xA;&#xA;&lt;hr /&gt;&#xA;&#xA;&lt;p&gt;As an aside, you will note all my examples using &lt;code&gt;strlen&lt;/code&gt; both custom and actual return &lt;code&gt;size_t&lt;/code&gt;. This is not an accident; in both C and C++ the return type of a call to &lt;code&gt;strlen&lt;/code&gt; is &lt;code&gt;size_t&lt;/code&gt;. You have to be careful because in your code you compare an &lt;code&gt;int&lt;/code&gt; to a &lt;code&gt;size_t&lt;/code&gt;. Comparing a signed value to an unsigned value is &lt;a href=&quot;https://stackoverflow.com/questions/5416414/signed-unsigned-comparisons&quot;&gt;Not&lt;/a&gt; &lt;a href=&quot;https://softwareengineering.stackexchange.com/questions/175253/why-does-an-unsigned-int-compared-with-a-signed-character-turn-out-with-an-unexp&quot;&gt;Good&lt;/a&gt; because it will work only if the signed value can be faithfully represented after a conversion to an unsigned value. If it can't though, you're in for a bad time.&lt;/p&gt;&#xA;" OwnerUserId="961" LastActivityDate="2019-08-24T22:31:59.940" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="9267" PostTypeId="1" AcceptedAnswerId="9268" CreationDate="2019-08-25T08:52:10.567" Score="2" ViewCount="2846" Body="&lt;p&gt;I’m trying to create a Group Chart like this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/oVbX3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oVbX3.png&quot; alt=&quot;aim somthing similar to this chart&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 7 classes for each of 4 species, not 5 like in the diagram. I want to build i diagram to show the distributions of the classes in each species&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here’s example provided online:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  library(ggplot2)&#xA;&#xA;    # create a dataset&#xA;    specie &amp;lt;- c(rep(&quot;sorgho&quot; , 3) , rep(&quot;poacee&quot; , 3) , rep(&quot;banana&quot; , 3) , rep(&quot;triticum&quot; , 3) )&#xA;    condition &amp;lt;- rep(c(&quot;normal&quot; , &quot;stress&quot; , &quot;Nitrogen&quot;) , 4)&#xA;    value &amp;lt;- abs(rnorm(12 , 0 , 15))&#xA;    data &amp;lt;- data.frame(specie,condition,value) &#xA;    # Grouped&#xA;    ggplot(data, aes(fill=condition, y=value, x=specie)) + &#xA;        geom_bar(position=&quot;dodge&quot;, stat=&quot;identity&quot;)&#xA;&#xA;&#xA;&#xA;------------------------------------------&#xA;    #Here’s code:&#xA;&#xA;&#xA;    &amp;gt; Species &amp;lt;- c(rep(&quot;Cames1&quot; , 5) , rep(&quot;Came2&quot; , 5) , rep(&quot;Cams3&quot; , 5) , rep(&quot;cams4&quot; , 5) )&#xA;    &amp;gt; Class &amp;lt;- rep(c(&quot;EVV&quot; , &quot;MaLR&quot; , &quot;psy&quot; , &quot;Nargo&quot; , &quot;Pao&quot; , &quot;Unknown&quot; , &quot;Undefiend&quot;) , 8)&#xA;    &amp;gt; value &amp;lt;- abs(rnorm(20 , 0 , 15 ))&#xA;    data &amp;lt;- read.table(text=&quot;Class Cames1 Came2 Cams3 cams4&#xA;    + EVV 2194 5435 2706 2902 &#xA;    + MaLR 503 108 184 231&#xA;    + psy 284 187 413 295&#xA;    + Nargo 274 1977 413 2195&#xA;    + Pao 66 29 78 97&#xA;    + Unknown 286 283 273 377&#xA;    + Undefined 76 77 80 95&quot;)&#xA;   &amp;gt; ggplot(data, aes(fill=Class, y=7, x=Species)) + geom_bar(position=&quot;dodge&quot;, stat=&quot;identity&quot;)&#xA;&#xA;Error: Aesthetics must be either length 1 or the same as the data (7): x, fill&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;i am able to build basics chart, but to group them all into one like the example provided challenging for me , could someone help out ? &lt;/p&gt;&#xA;" OwnerUserId="3332" LastEditorUserId="3332" LastEditDate="2019-08-25T09:26:04.337" LastActivityDate="2019-08-25T11:29:55.740" Title="Create a Grouped Barchart" Tags="|r|statistics|ggplot2|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9268" PostTypeId="2" ParentId="9267" CreationDate="2019-08-25T11:23:15.043" Score="4" Body="&lt;p&gt;I have noticed a few errors:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) You define a bunch of variables (Species, Class, ...) but then instead of creating your data frame data with these you create it by reading from text. I don't think this is good practice, you should do so like in your example code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) With the way you define, your data frame does not contain columns like Class or Species but columns V1 to V5, you should use argument &lt;code&gt;header = TRUE&lt;/code&gt; in your &lt;code&gt;read.table()&lt;/code&gt; call.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Even when you read your data as in step 2, you would need to transform your data table into &quot;long format&quot;, please see below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) In your &lt;code&gt;ggplot()&lt;/code&gt; call, you assign 7 to your y aesthetic, this does not make sense. You should either provide a vector of length equal to your other aesthetics (x and fill) or better a column name of values of interest.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# This is your code minus the redundant variables that you have created but are&#xA;# not used&#xA;&#xA;library(tidyr)&#xA;library(ggplot2)&#xA;&#xA;# Pay attention to the header argument&#xA;data &amp;lt;- read.table(&#xA;  text = &quot;Class Cames1 Came2 Cams3 cams4&#xA;     EVV 2194 5435 2706 2902&#xA;     MaLR 503 108 184 231&#xA;     psy 284 187 413 295&#xA;     Nargo 274 1977 413 2195&#xA;     Pao 66 29 78 97&#xA;     Unknown 286 283 273 377&#xA;     Undefined 76 77 80 95&quot;,&#xA;  header = TRUE&#xA;)&#xA;&#xA;# data in wide format&#xA;&amp;gt; data&#xA;      Class Cames1 Came2 Cams3 cams4&#xA;1       EVV   2194  5435  2706  2902&#xA;2      MaLR    503   108   184   231&#xA;3       psy    284   187   413   295&#xA;4     Nargo    274  1977   413  2195&#xA;5       Pao     66    29    78    97&#xA;6   Unknown    286   283   273   377&#xA;7 Undefined     76    77    80    95&#xA;&#xA;# Making the data wider: Keeping Class column as it is but condensing the data&#xA;# within the rest of the columns into two newly created columns, Species_ and&#xA;# Value_&#xA;&#xA;data_long &amp;lt;- gather(data,&#xA;                    key = &quot;Species_&quot;,&#xA;                    value = &quot;Value_&quot;,&#xA;                    -Class)&#xA;&#xA;# First few rows of your data in long format&#xA;&amp;gt; head(data_long)&#xA;    Class Species_ Value_&#xA;1     EVV   Cames1   2194&#xA;2    MaLR   Cames1    503&#xA;3     psy   Cames1    284&#xA;4   Nargo   Cames1    274&#xA;5     Pao   Cames1     66&#xA;6 Unknown   Cames1    286&#xA;&#xA;# x and y aesthetics are the newly created columns Species_ and Value_&#xA;ggplot(data_long, aes(fill=Class, y=Value_, x=Species_)) + geom_bar(position=&quot;dodge&quot;, stat=&quot;identity&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JXyOh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JXyOh.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-08-25T11:29:55.740" LastActivityDate="2019-08-25T11:29:55.740" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9269" PostTypeId="1" AcceptedAnswerId="9270" CreationDate="2019-08-25T18:19:33.970" Score="0" ViewCount="231" Body="&lt;p&gt;I am trying to infer a phylogenetic tree from a Blastn output and from what I have understood, what I should do is 1) extract the alignments and re-align them using Muscle, then 2) feed the .aln file into &lt;strong&gt;PhyML&lt;/strong&gt;. &#xA;Is this correct? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or should I be extracting the hsps from each alignment as suggested here &lt;a href=&quot;https://biopython.org/wiki/Phylo&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biopython.org/wiki/Phylo&lt;/a&gt;? I don't quite understand why the hsps are extracted here, shouldn't the whole sequences be compared in the multiple alignment?&lt;/p&gt;&#xA;" OwnerUserId="5114" LastEditorUserId="3848" LastEditDate="2019-08-25T22:35:39.137" LastActivityDate="2019-08-25T22:46:13.620" Title="Inferring a phylogenetic tree from BLASTn" Tags="|sequence-alignment|phylogenetics|phylogeny|blastn|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9270" PostTypeId="2" ParentId="9269" CreationDate="2019-08-25T18:55:17.497" Score="0" Body="&lt;p&gt;If you look into the &lt;code&gt;get_seqrecs()&lt;/code&gt; function, you will see that this function is not extracting hsps to use in the following steps but the hsps are used in order to filter out low score BLAST alignments. The function accepts a blast alignment result and a threshold, and if the hsp score is above the supplied threshold the whole sequence (not just the hsp) would be the output. Your intuition is correct, the whole sequence should go in the MSA step.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-08-25T19:50:37.030" LastActivityDate="2019-08-25T19:50:37.030" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9271" PostTypeId="1" CreationDate="2019-08-25T19:44:51.230" Score="1" ViewCount="903" Body="&lt;p&gt;To identify homologous genes, I performed tBLASTn analyses against my genome of interest resulting in sequences of a closely related species. Using this method, I found two (and in some cases up to five copies) of my different genes of interest in the target genome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;How can I determine if these gene copies correspond to true duplication events?&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I aligned the newly found sequences to the reference genes and I found good alignments in more than one region (20% - 50% identity), but I am unsure if this is enough to consider them as duplicates. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a &lt;strong&gt;threshold&lt;/strong&gt; to help me determine if they are indeed duplications? How can I distinguish between a true duplication event and a pseudogene or a paralogous gene? The target genome belongs to a polyploid organism, although the genome assembly type is listed as haploid. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help will be greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Thank you all very much for your responses, I am now beginning to understand how to find out if my genes are duplicated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I performed some of the analyses that you suggested and I would very much appreciate your input on the results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, I followed @terdon's advice and I took the best hit for one of the query (source) genes in the target genome and I used this sequence as a query against the query (source) genome, i.e., I performed a reciprocal BLAST analysis (in my case it was a reciprocal BLASTp analysis). The first hit was my original query gene, which had an E-value of 0.0. For other sequences the reciprocal BLASTp hits were more variable, ranging from 3e-42 to 2e-154, but the first hit was always the query sequence. I also looked at the protein domains and they are conserved, i.e., both the query and the target sequences have the same domains. As for the exonic structure, it is fairly similar, for e.g., the query has 12 exons while the target sequence has 10. Exons are similar in length. So, in summary, when the potential duplicates from my target genome are blasted against the original source genome, the best hit is the original query sequence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for @zhanf's suggestion, I did a basic phylogenetic analysis using Clustal Omega (neighbour-joining tree without distance corrections) using four sequences: the original query sequence and the three gene copies (seq_5-1, seq_5-2 and seq_5-3) that I found in the target genome. This is the result:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gM5yI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gM5yI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also compared each of the sequences against the original query:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;seq_5-1 vs query: 34% identity (query length 780 aa, seq_5-1 length 400 aa)&lt;/li&gt;&#xA;&lt;li&gt;seq_5-2 vs query: 42% identity (query length 780 aa, seq_5-2 length 580 aa)&lt;/li&gt;&#xA;&lt;li&gt;seq_5-3 vs query: 40% identity (query length 780 aa, seq_5-3 length 490 aa)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So in summary, these sequences would appear to be paralogues, right?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Again, thank you very much for your help, I greatly appreciate it!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Leah&lt;/p&gt;&#xA;" OwnerUserId="5115" LastEditorUserId="298" LastEditDate="2019-09-11T08:15:24.270" LastActivityDate="2019-09-11T12:09:42.090" Title="How can I find out if my gene of interest is duplicated?" Tags="|sequence-alignment|gene|phylogenetics|phylogeny|duplications|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="9272" PostTypeId="2" ParentId="9269" CreationDate="2019-08-25T22:33:09.407" Score="0" Body="&lt;p&gt;Firsly an .aln file feed into PhylML is fine, unless you can program in Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Precisely what level of tree building you need depends on its end purpose. Here's past example of a similar question &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/3377/how-to-find-build-the-evolutionary-history-of-a-protein-from-its-sequence/6928#6928&quot;&gt;here&lt;/a&gt; using NCBI Blast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Building trees (nj and parsimony) is one of the features of NCBI's Blast. Here's how to do it&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Go to Blast here, &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blast.ncbi.nlm.nih.gov/Blast.cgi&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Enter your sequence into the box (it doesn't accept PDB codes alone)&lt;/li&gt;&#xA;&lt;li&gt;Enter the protein database - when I first did this calculation I used SwissProt, thinking there would be alot of sequences - I then used &quot;nr&quot;&#xA;Under the algorithm parameters enter &quot;50&quot; (default is too many)&lt;/li&gt;&#xA;&lt;li&gt;Hit &quot;Blast&quot;&lt;/li&gt;&#xA;&lt;li&gt;Once the search is complete at the top of the page are the hyperlinks: &quot;Other reports: Search Summary [Taxonomy reports] [Distance tree of results]&quot;&lt;/li&gt;&#xA;&lt;li&gt;Click on &quot;Distance tree of results&quot;&lt;/li&gt;&#xA;&lt;li&gt;This will automatically aligning your sequences and produce, in this case a parsimony based tree, but there is also the option of a nj tree (recommended) &lt;/li&gt;&#xA;&lt;li&gt;Click &quot;Tool&quot;, &quot;Download&quot;, &quot;PDF&quot; ...&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you want probabilistic trees, PhylML is certainly one of them,  Blast remains quite central to building the alignment. &lt;/li&gt;&#xA;&lt;li&gt;With regards building partial genome trees, this will depend on what organism you are studing&lt;/li&gt;&#xA;&lt;li&gt;With regards Blast HSPs being treated individually, again it depends on what you question is and generally refers to advanced phylogenetics&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-08-25T22:46:13.620" LastActivityDate="2019-08-25T22:46:13.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9273" PostTypeId="2" ParentId="6852" CreationDate="2019-08-26T04:50:10.333" Score="3" Body="&lt;p&gt;As you mentioned, rdKit does not accept a chemical formula as input. The main reason for this are probably isomers. You could however use rdKit to initiate all the atoms, get their mass and add them together.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;from rdkit import Chem&#xA;&#xA;&#xA;def get_mass(formula):&#xA;&#xA;    parts = re.findall(&quot;[A-Z][a-z]?|[0-9]+&quot;, formula)&#xA;    mass = 0&#xA;&#xA;    for index in range(len(parts)):&#xA;        if parts[index].isnumeric():&#xA;            continue&#xA;&#xA;        atom = Chem.Atom(parts[index])&#xA;        multiplier = int(parts[index + 1]) if len(parts) &amp;gt; index + 1 and parts[index + 1].isnumeric() else 1&#xA;        mass += atom.GetMass() * multiplier&#xA;&#xA;    return mass&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; get_mass(&quot;C9H15N4O8P&quot;)&#xA;338.21299999999997&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5118" LastActivityDate="2019-08-26T04:50:10.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9274" PostTypeId="2" ParentId="9262" CreationDate="2019-08-26T07:18:14.067" Score="2" Body="&lt;p&gt;No, there's no simple way to do that without someone writing a program tailored toward it that handles the footer section and recreates the zoom levels that follow the compressed blocks of intervals. If you wanted to write such a program, modifying bigWigCat from &lt;a href=&quot;http://hgdownload.soe.ucsc.edu/admin/exe/linux.x86_64.v369/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kent's tools&lt;/a&gt; would be a reasonable tack.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-08-26T07:18:14.067" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9275" PostTypeId="2" ParentId="9271" CreationDate="2019-08-26T08:33:38.153" Score="2" Body="&lt;p&gt;You might want to have a look at &lt;a href=&quot;https://biology.stackexchange.com/q/4962/1306&quot;&gt;What is the difference between orthologs, paralogs and homologs?&lt;/a&gt;. Paralogs are duplicates by definition, and whether a gene is a pseudogene or still active has no bearing on whether it comes from a duplication. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In any case, to figure out what's going in in your target genome, you should use the target's genes. So, first get the best hit for each of your query genes in the target genome. Then, to investigate its evolutionary history, use that sequence (the one from the target genome) as  query against the target genome. That will let you assess how similar the target's hits are to one another and that's how you can attempt to figure out their evolutionary history. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that 20% identity is very low for closely related species. While there is no golden threshold, I would expect recent duplication events (and in closely related species, a duplication event that is in one genome and not the other will be recent) to produce much more similar sequences. In any case, you can't do this based on sequence identity alone. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are the protein domains of the gene products conserved? Is the exonic structure of the gene conserved? You should also try taking the putative duplicate of geneA from genome2 and blast it against genome1 (your original source genome). Is the best hit the geneA from genome1, the query you used to find the putative duplicate in genome2? Or is there a better hit for it against another gene? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are all questions to investigate and take into account when reconstructing the evolutionary history. You can't just look at identity and use some threshold to decide this sort of thing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you blast the potential dupicate from your target genome against your original source genome, is the best hit the duplicate? &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-08-26T08:33:38.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9279" PostTypeId="2" ParentId="8909" CreationDate="2019-08-27T02:35:55.860" Score="0" Body="&lt;p&gt;I don't know if you solved the problem. Howerver,I think it's caused by multiple data types.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For output from CellRanger &gt;= 3.0 with multiple data types&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data &amp;lt;- Read10X(data.dir = data_dir)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This step creates multiple matrices, depending on how many types of data there are.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seurat_object = CreateSeuratObject(counts = data$`Gene Expression`)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So (counts = data$&lt;code&gt;Gene Expression&lt;/code&gt;),You should also be able to choose the type of research you want to do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope my answer will help some people.&lt;/p&gt;&#xA;" OwnerUserId="5124" LastEditorUserId="1310" LastEditDate="2019-08-27T04:47:17.333" LastActivityDate="2019-08-27T04:47:17.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9280" PostTypeId="2" ParentId="8909" CreationDate="2019-08-27T07:25:27.870" Score="1" Body="&lt;p&gt;Given the fact that you have RNA and ADT data (probably on the same set of cells), some colnames (which represent individual cells) would be duplicated. Normally duplicated colnames are tolerated with matrices and sparse matrices, however, Seurat apparently does not do so, below is from the &lt;code&gt;?CreateAssayObject&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Details&#xA;Non-unique cell or feature names are not allowed. Please make unique before calling this function.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Possible solution to your problem: Seuart has a dedicated vignette for &lt;a href=&quot;https://satijalab.org/seurat/v3.0/multimodal_vignette.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;working with multimodal data&lt;/a&gt; and as you would see you will need to initiate your Seurat object with one matrix per assay: RNA and ADT. All you need to do is split your matrix into RNA and ADT, create your Seurat object with RNA data and then add the ADT data with:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;seurat_obj_with_rna_only[[&quot;ADT&quot;]] &amp;lt;- CreateAssayObject(counts = your_adt_matrix)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For efficiency, Seurat uses sparse matrices so don't forget to convert your data matrices to sparse.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-08-27T07:55:54.483" LastActivityDate="2019-08-27T07:55:54.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9281" PostTypeId="1" CreationDate="2019-08-27T08:15:21.350" Score="0" ViewCount="62" Body="&lt;p&gt;I've got a list of genes and I want to assess their association with drugs based on gene expression data sets. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A comprehensive tutorial would also be helpful.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="5125" LastEditorUserId="3848" LastEditDate="2019-08-27T12:50:37.373" LastActivityDate="2019-09-26T15:01:55.363" Title="Finding drugs based on gene expression data" Tags="|gene-expression|drugs|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9282" PostTypeId="1" CreationDate="2019-08-27T08:19:07.733" Score="1" ViewCount="90" Body="&lt;p&gt;I would like to ask if there are some known databases that allows to annotate non-coding mutations in a given region for the human genome?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Preferably, if the database is downloadable so I am able to automatize the processes of annotation.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2019-08-27T09:17:30.970" Title="Are there databases to annotate non-coding mutations?" Tags="|database|sequence-annotation|mutations|non-coding|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="9283" PostTypeId="2" ParentId="9282" CreationDate="2019-08-27T09:17:30.970" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://varsome.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;VarSome&lt;/a&gt; allows you to annotate any variant, including non coding ones. While ACMG classification will not be available if the variant isn't in a gene's transcript (the ACMG criteria are only applicable to variants in transcribed regions), non-coding variants in transcripts will be fully annotated. For example, if you look up variant &lt;a href=&quot;https://varso.me/JyHZ&quot; rel=&quot;nofollow noreferrer&quot;&gt;chr17-41197559-G-T&lt;/a&gt; which is a non-coding variant found in the UTR of one of the transcripts of BRCA1, you will get a &lt;em&gt;lot&lt;/em&gt; of information (probably best to click on the link, the image has a &lt;em&gt;lot&lt;/em&gt; of data):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AsaDw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AsaDw.png&quot; alt=&quot;VarSome screenshot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For non-coding variants that don't fall in any transcript, you can still get frequency data, when available, as well as DANN pathogenicity predictions, GERP score, an overview of the genomic regions and other variants and/or srtuctural variants that can be found around the region, and any community annotations that may have been submitted by other users. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://varsome.com/information/varsome-api-info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;There is also an API&lt;/a&gt; you can use to automate the annotation, and &lt;a href=&quot;https://varsome.com/editions/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VarSome Pro&lt;/a&gt; that lets you annotate an entire VCF file, but those are both paid services.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Disclaimer&lt;/strong&gt;: I work for the company behind VarSome. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-08-27T09:17:30.970" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9284" PostTypeId="2" ParentId="9281" CreationDate="2019-08-27T14:52:19.157" Score="1" Body="&lt;p&gt;It would be great if you can provide more information about the gene expression data set. If you have response to single drug, the general approach is to find the deferentially expressed genes with tools like &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/DESeq2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2&lt;/a&gt;, or make a heat map based on the expression of your listed genes to see the drug effect before and after the treatment. But, if you want to study the effects of combination of drugs it is necessary to consider the additive and synergistic effect of drug combinations. &lt;/p&gt;&#xA;" OwnerUserId="5103" LastActivityDate="2019-08-27T14:52:19.157" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9285" PostTypeId="2" ParentId="7500" CreationDate="2019-08-27T15:09:15.963" Score="2" Body="&lt;p&gt;Yes, you can do this with plink.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use the flag &lt;code&gt;--ld-snp &amp;lt;variant_id&amp;gt;&lt;/code&gt; to calculate LD between your SNP of interest and all other SNPs.&lt;br&gt;&#xA;If you have multiple SNPs of interest, and want each of those compared with every SNP in your dataset, use &lt;code&gt;--ld-snps &amp;lt;snp1-snp100,snp102,snp104&amp;gt;&lt;/code&gt;&lt;br&gt;&#xA;A final option to calculate LD between a specific pair of SNPs is &lt;code&gt;--ld &amp;lt;variant_id1&amp;gt; &amp;lt;variant_id2&amp;gt;&lt;/code&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.cog-genomics.org/plink/1.9/ld&quot; rel=&quot;nofollow noreferrer&quot;&gt;This the relevant plink manual page&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5087" LastActivityDate="2019-08-27T15:09:15.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9286" PostTypeId="1" CreationDate="2019-08-28T08:06:49.420" Score="1" ViewCount="440" Body="&lt;p&gt;I would like to use PHASTER (&lt;a href=&quot;http://phaster.ca&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://phaster.ca&lt;/a&gt;) from the University of Alberta, but it's currently down for maintenance, as is its predecessor, PHAST.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What alternatives to PHASTER are there for searching and annotating phage sequences within assembled genomes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen that PROKKA has some entries mentioning the possibility of using PHASER (currently discarded as an option) but also people using their own HMMs to annotate phage sequences using PROKKA, e.g.:&#xA;&lt;a href=&quot;https://github.com/tseemann/prokka/issues/375&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/tseemann/prokka/issues/375&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="180" LastEditDate="2019-08-28T08:12:12.487" LastActivityDate="2019-09-03T07:20:32.453" Title="alternatives to PHASTER phage annotation server" Tags="|sequence-annotation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9287" PostTypeId="1" AcceptedAnswerId="9288" CreationDate="2019-08-28T11:43:23.867" Score="3" ViewCount="215" Body="&lt;p&gt;I failed to add &lt;code&gt;Note=Gene description&lt;/code&gt; to mRNA attribute with the below code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/python3&#xA;import click&#xA;import gffutils&#xA;import gffutils.gffwriter as gffwriter&#xA;&#xA;@click.command()&#xA;@click.option('--gff3', help=&quot;Provide GFF3 file&quot;, required=True)&#xA;@click.option('--keep', help=&quot;Keep GFF3 file&quot;, required=True)&#xA;@click.option('--reject', help=&quot;Reject GFF3 file&quot;, required=True)&#xA;def run(gff3, keep, reject):&#xA;    #db = gffutils.create_db(gff3, dbfn='data/test.db', force=True, keep_order=True, merge_strategy='merge',&#xA;    #                        sort_attribute_values=True)&#xA;&#xA;    db = gffutils.FeatureDB('data/test.db', keep_order=True)&#xA;    for gene in db.features_of_type('gene', order_by='start'):&#xA;        print(gene)&#xA;        for i in db.children(gene, featuretype='mRNA', order_by='start'):&#xA;            print(i)&#xA;            print(i.id)&#xA;            print(i.attributes['Note']=[&quot;Gene description&quot;])&#xA;            db.update([i])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I got the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;File &quot;/projects/bioinf-scripts/mod_braker_gff3.py&quot;, line 20&#xA;    print(i.attributes['Note']=[&quot;Gene description&quot;])&#xA;         ^&#xA;SyntaxError: keyword can't be an expression&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What did I miss?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance&lt;/p&gt;&#xA;" OwnerUserId="2477" LastEditorUserId="298" LastEditDate="2019-08-28T12:22:21.777" LastActivityDate="2019-08-28T12:22:21.777" Title="Adding an attribute to GFF3 file" Tags="|python|sequence-annotation|gff3|gffutils|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="9288" PostTypeId="2" ParentId="9287" CreationDate="2019-08-28T12:21:03.660" Score="2" Body="&lt;p&gt;You are printing the attribute before setting it, that's why it's complaining. So instead of this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    print(i.attributes['Note']=[&quot;Gene description&quot;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Set it first and &lt;em&gt;then&lt;/em&gt; print:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;i.attributes['Note']=[&quot;Gene description&quot;]&#xA;print(i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-08-28T12:21:03.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9289" PostTypeId="1" CreationDate="2019-08-28T12:35:55.633" Score="2" ViewCount="494" Body="&lt;p&gt;I modified the GFF3 file with the below code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/python3&#xA;import click&#xA;import gffutils&#xA;import gffutils.gffwriter as gffwriter&#xA;&#xA;@click.command()&#xA;@click.option('--gff3', help=&quot;Provide GFF3 file&quot;, required=True)&#xA;@click.option('--keep', help=&quot;Keep GFF3 file&quot;, required=True)&#xA;@click.option('--reject', help=&quot;Reject GFF3 file&quot;, required=True)&#xA;def run(gff3, keep, reject):&#xA;    #db = gffutils.create_db(gff3, dbfn='data/test.db', force=True, keep_order=True, merge_strategy='merge',&#xA;    #                        sort_attribute_values=True)&#xA;&#xA;    out_keep = gffwriter.GFFWriter(keep)&#xA;    db = gffutils.FeatureDB('data/test.db', keep_order=True)&#xA;    for gene in db.features_of_type('gene', order_by='start'):&#xA;        print(gene)&#xA;        for i in db.children(gene, featuretype='mRNA', order_by='start'):&#xA;            print(i)&#xA;            i.attributes['Note'] = [&quot;Gene description&quot;]&#xA;            print(i)&#xA;            out_keep.write_rec(gene)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the above code saved only this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#GFF3 file (created by gffutils (v0.9) on 2019-08-28 22:14:13)&#xA;NbV1Ch08        AUGUSTUS        gene    7015    29794   0.01    -       .       ID=g1;&#xA;NbV1Ch08        AUGUSTUS        gene    60876   63944   0.03    +       .       ID=g2;&#xA;NbV1Ch08        AUGUSTUS        gene    64722   65524   0.32    -       .       ID=g3;&#xA;NbV1Ch08        AUGUSTUS        gene    66706   67484   0.12    -       .       ID=g4;&#xA;NbV1Ch08        AUGUSTUS        gene    115451  122754  0.16    -       .       ID=g5;&#xA;NbV1Ch08        AUGUSTUS        gene    147786  155025  0.24    +       .       ID=g6;&#xA;NbV1Ch08        AUGUSTUS        gene    155152  161104  0.28    -       .       ID=g7;&#xA;NbV1Ch08        AUGUSTUS        gene    237126  238098  0.05    +       .       ID=g8;&#xA;NbV1Ch08        AUGUSTUS        gene    265796  297830  0.82    +       .       ID=g9;&#xA;NbV1Ch08        AUGUSTUS        gene    265796  297830  0.82    +       .       ID=g9;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Rather than:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NbV1Ch08        AUGUSTUS        gene    7015    29794   0.01    -       .       ID=g1;&#xA;NbV1Ch08        AUGUSTUS        mRNA    7015    29794   0.01    -       .       ID=g1.t1;Parent=g1;Note=Gene description;&#xA;NbV1Ch08        AUGUSTUS        transcription_end_site  7015    7015    .       -       .       Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        three_prime_utr 7015    8531    0.2     -       .       ID=g1.t1.3UTR1;Parent=g1.t1&#xA;NbV1Ch08        AUGUSTUS        exon    7015    8747    .       -       .       ID=g1.t1.exon1;Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        stop_codon      8532    8534    .       -       0       Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        CDS     8532    8747    0.31    -       0       ID=g1.t1.CDS1;Parent=g1.t1&#xA;NbV1Ch08        AUGUSTUS        intron  8748    9191    0.49    -       .       Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        CDS     9192    9342    0.66    -       1       ID=g1.t1.CDS2;Parent=g1.t1&#xA;NbV1Ch08        AUGUSTUS        exon    9192    9342    .       -       .       ID=g1.t1.exon2;Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        intron  9343    9915    0.58    -       .       Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        CDS     9916    10006   0.71    -       2       ID=g1.t1.CDS3;Parent=g1.t1&#xA;NbV1Ch08        AUGUSTUS        exon    9916    10006   .       -       .       ID=g1.t1.exon3;Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        intron  10007   10101   0.74    -       .       Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        CDS     10102   10201   0.78    -       0       ID=g1.t1.CDS4;Parent=g1.t1&#xA;NbV1Ch08        AUGUSTUS        exon    10102   10201   .       -       .       ID=g1.t1.exon4;Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        intron  10202   10712   0.8     -       .       Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        CDS     10713   11107   0.11    -       2       ID=g1.t1.CDS5;Parent=g1.t1&#xA;NbV1Ch08        AUGUSTUS        exon    10713   11107   .       -       .       ID=g1.t1.exon5;Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        intron  11108   11569   0.07    -       .       Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        CDS     11570   12151   0.09    -       2       ID=g1.t1.CDS6;Parent=g1.t1&#xA;NbV1Ch08        AUGUSTUS        exon    11570   12151   .       -       .       ID=g1.t1.exon6;Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        intron  12152   12588   0.34    -       .       Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        CDS     12589   12717   0.39    -       2       ID=g1.t1.CDS7;Parent=g1.t1&#xA;NbV1Ch08        AUGUSTUS        exon    12589   12717   .       -       .       ID=g1.t1.exon7;Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        intron  12718   12789   0.42    -       .       Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        CDS     12790   13075   0.39    -       0       ID=g1.t1.CDS8;Parent=g1.t1&#xA;NbV1Ch08        AUGUSTUS        exon    12790   13075   .       -       .       ID=g1.t1.exon8;Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        intron  13076   14832   0.51    -       .       Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        CDS     14833   15009   0.39    -       0       ID=g1.t1.CDS9;Parent=g1.t1&#xA;NbV1Ch08        AUGUSTUS        exon    14833   15009   .       -       .       ID=g1.t1.exon9;Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        intron  15010   15278   0.59    -       .       Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        CDS     15279   15415   0.56    -       2       ID=g1.t1.CDS10;Parent=g1.t1&#xA;NbV1Ch08        AUGUSTUS        exon    15279   15415   .       -       .       ID=g1.t1.exon10;Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        intron  15416   15487   0.58    -       .       Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        CDS     15488   15612   0.96    -       1       ID=g1.t1.CDS11;Parent=g1.t1&#xA;NbV1Ch08        AUGUSTUS        exon    15488   15612   .       -       .       ID=g1.t1.exon11;Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        intron  15613   15706   0.96    -       .       Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        CDS     15707   15957   0.98    -       0       ID=g1.t1.CDS12;Parent=g1.t1&#xA;NbV1Ch08        AUGUSTUS        exon    15707   15958   .       -       .       ID=g1.t1.exon12;Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        start_codon     15955   15957   .       -       0       Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        five_prime_utr  15958   15958   0.99    -       .       ID=g1.t1.5UTR1;Parent=g1.t1&#xA;NbV1Ch08        AUGUSTUS        five_prime_utr  27458   28250   0.37    -       .       ID=g1.t1.5UTR2;Parent=g1.t1&#xA;NbV1Ch08        AUGUSTUS        exon    27458   28250   .       -       .       ID=g1.t1.exon13;Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        five_prime_utr  29272   29794   0.08    -       .       ID=g1.t1.5UTR3;Parent=g1.t1&#xA;NbV1Ch08        AUGUSTUS        exon    29272   29794   .       -       .       ID=g1.t1.exon14;Parent=g1.t1;&#xA;NbV1Ch08        AUGUSTUS        transcription_start_site        29794   29794   .       -       .       Parent=g1.t1;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What did I miss?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt; This is base on @daniel-standage code.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import click&#xA;import tag&#xA;def add_description(instream):&#xA;    for feature in instream:&#xA;        for mrna in tag.select.features(feature, type='mRNA', traverse=True):&#xA;            mrna.add_attribute('Note', 'Gene description')&#xA;&#xA;        yield feature&#xA;&#xA;&#xA;@click.command()&#xA;@click.option('--gff3', help=&quot;Provide GFF3 file&quot;, required=True)&#xA;@click.option('--keep', help=&quot;Keep GFF3 file&quot;, required=True)&#xA;@click.option('--reject', help=&quot;Reject GFF3 file&quot;, required=True)&#xA;def run(gff3, keep, reject):&#xA;    reader = tag.GFF3Reader(infilename=gff3)&#xA;    desc_stream = add_description(reader)&#xA;    writer = tag.GFF3Writer(desc_stream)&#xA;    writer.write()&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    run()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which caused this error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mod_braker_gff3.py --gff3 data/NbV1Ch08-augustus.hints_utr.gff3 --keep data/keep.gff3 --reject data/reject.gff3&#xA;Traceback (most recent call last):&#xA;  File &quot;/Users/lorencm/projects/bioinf-scripts/mod_braker_gff3.py&quot;, line 26, in &amp;lt;module&amp;gt;&#xA;    run()&#xA;  File &quot;/anaconda/envs/bioinf-scripts/lib/python3.7/site-packages/click/core.py&quot;, line 764, in __call__&#xA;    return self.main(*args, **kwargs)&#xA;  File &quot;/anaconda/envs/bioinf-scripts/lib/python3.7/site-packages/click/core.py&quot;, line 717, in main&#xA;    rv = self.invoke(ctx)&#xA;  File &quot;/anaconda/envs/bioinf-scripts/lib/python3.7/site-packages/click/core.py&quot;, line 956, in invoke&#xA;    return ctx.invoke(self.callback, **ctx.params)&#xA;  File &quot;/anaconda/envs/bioinf-scripts/lib/python3.7/site-packages/click/core.py&quot;, line 555, in invoke&#xA;    return callback(*args, **kwargs)&#xA;  File &quot;/Users/lorencm/projects/bioinf-scripts/mod_braker_gff3.py&quot;, line 22, in run&#xA;    writer.write()&#xA;  File &quot;/anaconda/envs/bioinf-scripts/lib/python3.7/site-packages/tag/writer.py&quot;, line 57, in write&#xA;    for entry in self._instream:&#xA;  File &quot;/Users/lorencm/projects/bioinf-scripts/mod_braker_gff3.py&quot;, line 8, in add_description&#xA;    for mrna in tag.select.features(feature, type='mRNA', traverse=True):&#xA;  File &quot;/anaconda/envs/bioinf-scripts/lib/python3.7/site-packages/tag/select.py&quot;, line 24, in features&#xA;    for feature in entry_type_filter(entrystream, tag.Feature):&#xA;  File &quot;/anaconda/envs/bioinf-scripts/lib/python3.7/site-packages/tag/select.py&quot;, line 99, in entry_type_filter&#xA;    for entry in entrystream:&#xA;TypeError: 'Directive' object is not iterable&#xA;&#xA;Process finished with exit code 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2477" LastEditorUserId="2477" LastEditDate="2019-08-28T23:57:44.920" LastActivityDate="2020-04-25T21:09:52.367" Title="Modifing a GFF3 file and writting to a new file" Tags="|python|sequence-annotation|gff3|gffutils|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9290" PostTypeId="2" ParentId="9289" CreationDate="2019-08-28T12:59:36.997" Score="1" Body="&lt;p&gt;You might want to check this line out:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;for gene in db.features_of_type('gene', order_by='start'):&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You are specifically filtering for &lt;code&gt;gene&lt;/code&gt;s hence other types of annotations are filtered out.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-08-28T13:14:59.073" LastActivityDate="2019-08-28T13:14:59.073" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="9291" PostTypeId="2" ParentId="9289" CreationDate="2019-08-28T13:16:57.707" Score="1" Body="&lt;p&gt;If you're willing to give another GFF3 processing library a shot, I'd recommend checking out the &lt;a href=&quot;https://tag.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;tag&lt;/code&gt; library&lt;/a&gt;. The following code should add the note attribute to mRNA records as you're attempting to do in you're original code, but will also print out all other records.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import sys&#xA;import tag&#xA;&#xA;def add_description(instream):&#xA;    for record in instream:&#xA;        if not isinstance(record, tag.Feature):&#xA;            yield record&#xA;            continue&#xA;        for mrna in tag.select.features(record, type='mRNA', traverse=True):&#xA;            mrna.add_attribute('Note', 'Gene description')&#xA;        yield feature&#xA;&#xA;reader = tag.GFF3Reader(instream=sys.stdin)&#xA;desc_stream = add_description(reader)&#xA;writer = tag.GFF3Writer(desc_stream)&#xA;writer.retainids = True&#xA;writer.write()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I didn't use click or argparse to put any CLI on the script, so unless you want to add those just use STDIN and STDOUT for I/O.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;$ python3 myscript.py &amp;lt; mydata.gff3 &amp;gt; mydatawithdescription.gff3&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;P.S. The &lt;code&gt;add_description&lt;/code&gt; can be simplified if you don't care about retaining comments, &lt;code&gt;##sequence-region&lt;/code&gt; pragmas, and so on.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;def add_description(instream):&#xA;    for feature in tag.select.features(instream):&#xA;        for mrna in tag.select.features(record, type='mRNA', traverse=True):&#xA;            mrna.add_attribute('Note', 'Gene description')&#xA;        yield feature&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2020-04-25T21:09:52.367" LastActivityDate="2020-04-25T21:09:52.367" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="9292" PostTypeId="2" ParentId="9289" CreationDate="2019-08-28T13:43:54.803" Score="2" Body="&lt;p&gt;I guess you want to take the features for a gene?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As @haci and @terdon pointed out, it's a problem with the selected. Below I included an example, with mouse_extra_comma.gff3 obtained from example file provided by the package gffutils.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#gff.py&#xA;#!/usr/bin/python3&#xA;&#xA;###&#xA;def addNote(x):&#xA;    x.attributes['Note'] = [&quot;Gene description&quot;]&#xA;    return(x)&#xA;&#xA;###&#xA;&#xA;import click&#xA;import gffutils&#xA;import gffutils.gffwriter as gffwriter&#xA;&#xA;fn = gffutils.example_filename('mouse_extra_comma.gff3')&#xA;db = gffutils.create_db(fn, dbfn='test.db', force=True, keep_order=True,&#xA;    merge_strategy='merge', sort_attribute_values=True)&#xA;&#xA;db = gffutils.FeatureDB('./test.db', keep_order=True)&#xA;&#xA;## here's to test so you can see how it works&#xA;gene = next(db.features_of_type('gene', order_by='start')) &#xA;items =[addNote(i) for i in db.children(gene, featuretype=['exon','CDS'],order_by='start')]&#xA;&#xA;items[0:5]&#xA;#[&amp;lt;Feature mRNA (chr17:6797760-6818159[+]) at 0x103659c50&amp;gt;,&#xA;# &amp;lt;Feature exon (chr17:6797760-6797769[+]) at 0x1037cee10&amp;gt;,&#xA;# &amp;lt;Feature five_prime_UTR (chr17:6797760-6797769[+]) at 0x103660898&amp;gt;,&#xA;# &amp;lt;Feature exon (chr17:6806513-6806553[+]) at 0x103659f98&amp;gt;,&#xA;# &amp;lt;Feature five_prime_UTR (chr17:6806513-6806526[+]) at 0x1036595f8&amp;gt;]&#xA;#####&#xA;## here's following your... code...&#xA;for gene in db.features_of_type('gene', order_by='start'):&#xA;    print(gene)&#xA;    for i in db.children(gene,order_by='start'):&#xA;        print(addNote(i))&#xA;&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Once you define the feature you want, should be ok&lt;/p&gt;&#xA;" OwnerUserId="5077" LastEditorUserId="5077" LastEditDate="2019-08-29T19:16:30.500" LastActivityDate="2019-08-29T19:16:30.500" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9293" PostTypeId="1" CreationDate="2019-08-28T19:25:24.807" Score="1" ViewCount="4183" Body="&lt;p&gt;I have a set of cells that I am performing Drop-seq on to look at cell expression.  Among my heat maps for gene expression I want to be able to graph them similar to the graph below:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/dZbSH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dZbSH.png&quot; alt=&quot;Heat map with clusters on left, genes on bottom, and genes grouped together&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where the cells are sorted by cluster on the left axis, and have the genes across the bottom.  However I also want to cluster the genes by the expression within the cluster (like how this graph does).  Is that possible in Seurat?&lt;/p&gt;&#xA;&lt;p&gt;Currently I have created a heat map that looks like this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/kU4ze.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kU4ze.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It has the clusters listed on the left axis, and small sub-selection of genes on the bottom.  However I want to be able to order those genes by their expression within those clusters, without having to manually order the genes when I insert them into the &amp;quot;feature&amp;quot; argument.&lt;/p&gt;&#xA;&lt;p&gt;Here is a sample of the code I am currently using to plot the graph:&lt;/p&gt;&#xA;&lt;p&gt;(Note: I have an object &amp;quot;Combined&amp;quot; that has the non-averaged cell expression, but for now I do want to use average expression)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;png(&amp;quot;ExpressionHeatmap%03d.png&amp;quot;, res=600, width=14, height=7, units=&amp;quot;in&amp;quot;)&#xA;top10 &amp;lt;- Combined.markers %&amp;gt;% group_by(cluster) %&amp;gt;% top_n(n = 10, wt = avg_logFC)    &#xA;avg &amp;lt;- AverageExpression(Combined, features = NULL, add.ident = NULL, return.seurat = TRUE, verbose = TRUE)&#xA;Combined.markers %&amp;gt;% group_by(cluster) %&amp;gt;% top_n(2000, avg_logFC) -&amp;gt; top&#xA;&#xA;cc.features &amp;lt;- readLines(con = &amp;quot;/home/alex/lab/drugres/DropSeqFiles/regev_lab_cell_cycle_genes.txt&amp;quot;)&#xA;s.features &amp;lt;- cc.features[1:43]&#xA;g2m.features &amp;lt;- cc.features[44:97]&#xA;CCS &amp;lt;- CellCycleScoring(object = Combined, s.features = s.features, g2m.features = g2m.features, set.ident = TRUE)&#xA;DoHeatmap(avg, features = c(&amp;quot;FTH1P3&amp;quot;, &amp;quot;MET&amp;quot;, &amp;quot;CYR61&amp;quot;, &amp;quot;NFKBIA&amp;quot;, &amp;quot;RIPK2&amp;quot;, &amp;quot;IFI16&amp;quot;, &amp;quot;PLK2&amp;quot;, &amp;quot;TNFSF10&amp;quot;, &amp;quot;CEBPD&amp;quot;, &amp;quot;PNRC1&amp;quot;), group.by = &amp;quot;ident&amp;quot;, size = 3, angle = 0, combine = TRUE, draw.lines = FALSE) + theme(axis.text.x = element_text(size = 7), axis.line = element_line(colour = &amp;quot;#ffffff&amp;quot;)) + scale_fill_gradient2(low = '#1000ff', mid = &amp;quot;#ffffff&amp;quot;, high = '#aa0101', space = &amp;quot;Lab&amp;quot;, na.value = &amp;quot;#ffffff&amp;quot;, midpoint = 0, guide = &amp;quot;colourbar&amp;quot;, aesthetics = &amp;quot;fill&amp;quot;) + coord_flip(expand = TRUE, clip = &amp;quot;on&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried to use the &amp;quot;FindAllMarkers&amp;quot;, &amp;quot;group_by&amp;quot; and &amp;quot;top_n&amp;quot; functions to sort the gene data, and while it helped it still didn't fully sort the genes the way I wanted.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;markers &amp;lt;- FindAllMarkers(Combined, features = c(genes-i-am-looking-for), only.pos = TRUE, min.pct = 0.25, thresh.use = 0.25, test.use = &amp;quot;biomod&amp;quot;)&#xA;geneorder &amp;lt;- markers %&amp;gt;% group_by(cluster) %&amp;gt;% top_n(n = number-of-genes, wt = avg_logFC)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would then replace the 'features' argument in the DoHeatMap function with the 'geneorder' object.&lt;/p&gt;&#xA;&lt;p&gt;I know that doing a &lt;code&gt;top10 &amp;lt;- combined.markers&lt;/code&gt; style function can get me the effect I'm looking for, but I want to do it, with specifically selected genes, rather than just the top 10 expressed per cluster.&lt;/p&gt;&#xA;&lt;p&gt;Note: this is for v3.1 of Seurat; the latest version of Seurat at the time of asking.&lt;/p&gt;&#xA;" OwnerUserId="5134" LastEditorUserId="73" LastEditDate="2022-06-21T10:33:05.993" LastActivityDate="2022-06-21T10:33:05.993" Title="Create heat map that groups genes by expression within cluster in Seurat" Tags="|seurat|single-cell|gene-expression|heatmap|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9294" PostTypeId="1" CreationDate="2019-08-29T09:03:05.837" Score="1" ViewCount="239" Body="&lt;p&gt;Please if anyone has experience with the use of the &lt;a href=&quot;https://shenorrlab.github.io/bseqsc/vignettes/bseq-sc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BSEQ-SC&lt;/a&gt; package for the deconvolution of bulk RNA sequencing data with single cell RNA sequencing data I will be very grateful for your suggestion.&lt;/p&gt;&#xA;&lt;p&gt;I wish to deconvolve my bulk RNA seq data with single cell RNA seq data from the same tissue. I run in to the following error when running the code line below and I do understand the error but not how I can overcome it. Can any one please provide any useful hints?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;csfit &amp;lt;- bseqsc_csdiff(ilc_bulk[genes, ] ~ treatment | fibroblast + macrophage, &#xA;                       verbose = 2, nperms = 100, .rng = 12345)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then comes the error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   Groups: a_nacl1=4L | a_nacl2=4L | a_nacl3=4L | a_nacl4=4L | a_nacl5=4L | NA0L&#xA;Cell type(s): 'fibroblast', 'macrophage' (2 total)&#xA;Fitting mode: auto&#xA;Data (filtered): 1684 features x 20 samples&#xA;Model has factor effect with more than 2 levels: fitting lm interaction model&#xA;Fitting model with nonnegative effects&#xA;Model with more than 2 groups: switching to version 2&#xA; Fitting linear interaction model ...  OK&#xA; Computing FDR using 100 permutations ... 101/100&#xA;  Alternative 'two.sided' ...   OK&#xA;  Alternative 'greater' ...   OK&#xA;  Alternative 'less' ...   OK&#xA; OK&#xA;Timing:&#xA;   user  system elapsed &#xA;  2.136   0.153   2.267 &#xA;Error in array(NA_real_, dim = c(nrow(b), ncol(cc), length(lev)), dimnames = c(rownames(b),  : &#xA;  length of 'dimnames' [1687] must match that of 'dims' [3]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;&amp;gt; sessionInfo()&#xA;R version 3.6.1 (2019-07-05)&#xA;Platform: x86_64-pc-linux-gnu (64-bit)&#xA;Running under: Ubuntu 18.04.3 LTS&#xA;&#xA;Matrix products: default&#xA;BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.7.1&#xA;LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.7.1&#xA;&#xA;locale:&#xA; [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=de_DE.UTF-8        LC_COLLATE=en_US.UTF-8    &#xA; [5] LC_MONETARY=de_DE.UTF-8    LC_MESSAGES=en_US.UTF-8    LC_PAPER=de_DE.UTF-8       LC_NAME=C                 &#xA; [9] LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=de_DE.UTF-8 LC_IDENTIFICATION=C       &#xA;&#xA;attached base packages:&#xA;[1] stats4    parallel  stats     graphics  grDevices utils     datasets  methods   base     &#xA;&#xA;other attached packages:&#xA; [1] edgeR_3.26.6                limma_3.40.6                forcats_0.4.0               stringr_1.4.0              &#xA; [5] purrr_0.3.2                 readr_1.3.1                 tidyr_0.8.3                 tibble_2.1.3               &#xA; [9] tidyverse_1.2.1             SoupX_0.3.1                 ggplot2_3.2.0               dplyr_0.8.3                &#xA;[13] DropletUtils_1.4.3          SingleCellExperiment_1.6.0  SummarizedExperiment_1.14.0 DelayedArray_0.10.0        &#xA;[17] BiocParallel_1.18.0         matrixStats_0.54.0          GenomicRanges_1.36.0        GenomeInfoDb_1.20.0        &#xA;[21] RColorBrewer_1.1-2          xbioc_0.1.17                AnnotationDbi_1.46.0        IRanges_2.18.1             &#xA;[25] S4Vectors_0.22.0            BisqueRNA_1.0               Seurat_3.0.2                preprocessCore_1.46.0      &#xA;[29] e1071_1.7-2                 bseqsc_1.0                  csSAM_1.4                   Rcpp_1.0.2                 &#xA;[33] openxlsx_4.1.0.1            Biobase_2.44.0              BiocGenerics_0.30.0        &#xA;&#xA;loaded via a namespace (and not attached):&#xA;  [1] reticulate_1.13        R.utils_2.9.0          tidyselect_0.2.5       RSQLite_2.1.2          htmlwidgets_1.3       &#xA;  [6] grid_3.6.1             Rtsne_0.15             devtools_2.1.0         munsell_0.5.0          codetools_0.2-16      &#xA; [11] ica_1.0-2              future_1.14.0          withr_2.1.2            colorspace_1.4-1       rstudioapi_0.10       &#xA; [16] ROCR_1.0-7             gbRd_0.4-11            listenv_0.7.0          NMF_0.22               Rdpack_0.11-0         &#xA; [21] labeling_0.3           GenomeInfoDbData_1.2.1 bit64_0.9-7            rhdf5_2.28.0           rprojroot_1.3-2       &#xA; [26] vctrs_0.2.0            generics_0.0.2         R6_2.4.0               doParallel_1.0.14      rsvd_1.0.2            &#xA; [31] locfit_1.5-9.1         bitops_1.0-6           assertthat_0.2.1       SDMTools_1.1-221.1     scales_1.0.0          &#xA; [36] gtable_0.3.0           npsurv_0.4-0           globals_0.12.4         processx_3.4.1         rlang_0.4.0           &#xA; [41] zeallot_0.1.0          splines_3.6.1          lazyeval_0.2.2         broom_0.5.2            modelr_0.1.4          &#xA; [46] BiocManager_1.30.4     yaml_2.2.0             reshape2_1.4.3         backports_1.1.4        tools_3.6.1           &#xA; [51] usethis_1.5.1          gridBase_0.4-7         gplots_3.0.1.1         sessioninfo_1.1.1      ggridges_0.5.1        &#xA; [56] plyr_1.8.4             zlibbioc_1.30.0        RCurl_1.95-4.12        ps_1.3.0               prettyunits_1.0.2     &#xA; [61] pbapply_1.4-1          viridis_0.5.1          cowplot_1.0.0          zoo_1.8-6              haven_2.1.1           &#xA; [66] ggrepel_0.8.1          cluster_2.1.0          fs_1.3.1               magrittr_1.5           data.table_1.12.2     &#xA; [71] lmtest_0.9-37          RANN_2.6.1             fitdistrplus_1.0-14    pkgload_1.0.2          hms_0.5.0             &#xA; [76] lsei_1.2-0             xtable_1.8-4           readxl_1.3.1           gridExtra_2.3          testthat_2.2.1        &#xA; [81] compiler_3.6.1         KernSmooth_2.23-15     crayon_1.3.4           R.oo_1.22.0            htmltools_0.3.6       &#xA; [86] Formula_1.2-3          lubridate_1.7.4        DBI_1.0.0              MASS_7.3-51.4          Matrix_1.2-17         &#xA; [91] cli_1.1.0              R.methodsS3_1.7.1      gdata_2.18.0           metap_1.1              igraph_1.2.4.1        &#xA; [96] pkgconfig_2.0.2        registry_0.5-1         plotly_4.9.0           xml2_1.2.1             foreach_1.4.7         &#xA;[101] dqrng_0.2.1            rngtools_1.4           pkgmaker_0.28          XVector_0.24.0         rvest_0.3.4           &#xA;[106] bibtex_0.4.2           callr_3.3.1            digest_0.6.20          sctransform_0.2.0      tsne_0.1-3            &#xA;[111] cellranger_1.1.0       dendextend_1.12.0      curl_4.0               gtools_3.8.1           nlme_3.1-141          &#xA;[116] jsonlite_1.6           Rhdf5lib_1.6.0         desc_1.2.0             viridisLite_0.3.0      pillar_1.4.2          &#xA;[121] lattice_0.20-38        httr_1.4.0             pkgbuild_1.0.3         survival_2.44-1.1      glue_1.3.1            &#xA;[126] remotes_2.1.0          zip_2.0.3              png_0.1-7              iterators_1.0.12       bit_1.1-14            &#xA;[131] class_7.3-15           stringi_1.4.3          HDF5Array_1.12.1       blob_1.2.0             caTools_1.17.1.2      &#xA;[136] memoise_1.1.0          irlba_2.3.3            future.apply_1.3.0     ape_5.3&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3968" LastEditorUserId="3694" LastEditDate="2022-07-21T16:48:01.933" LastActivityDate="2023-12-13T20:00:25.053" Title="length of 'dimnames' [1686] must match that of 'dims' [3]" Tags="|r|rna-seq|single-cell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9295" PostTypeId="2" ParentId="8803" CreationDate="2019-08-29T15:15:04.747" Score="0" Body="&lt;p&gt;What is proposed by @Jonathan works, just adding another possible solution here.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(GenomicRanges)&#xA;&#xA;test.gr = makeGRangesFromDataFrame(test,keep.extra.columns=TRUE)&#xA;# as long as the previous comp does not agree with current, we make it a new segment&#xA;test.gr&lt;span class=&quot;math-container&quot;&gt;$seg = c(0,cumsum(diff(as.numeric(test.gr$&lt;/span&gt;comp)) != 0))&#xA;&#xA;# simple function to collapse the range&#xA;fn = function(u){&#xA;  out = reduce(u)&#xA;  out&lt;span class=&quot;math-container&quot;&gt;$comp = unique(u$&lt;/span&gt;comp)&#xA;  out&lt;span class=&quot;math-container&quot;&gt;$mean = mean(u$&lt;/span&gt;PC1)&#xA;  out&#xA;}&#xA;&#xA;reduced_range = unlist(GRangesList(lapply(split(test.gr,test.gr$seg),fn))))&#xA;&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2019-08-29T15:15:04.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9296" PostTypeId="1" CreationDate="2019-08-29T19:42:10.583" Score="1" ViewCount="759" Body="&lt;p&gt;I'm using the R package vegan to perform canonical correspondence analysis (CCA). As input we have two matrices, one being (sites)x(species) and the other being (sites)x(conditions). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sample data (and source of plot) are &lt;a href=&quot;http://rfunctions.blogspot.com/2016/11/canonical-correspondence-analysis-cca.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Species loadings are easily accessed with &lt;code&gt;summary(cca_model)$species&lt;/code&gt;. What I'm trying to find is the loadings for the explanatory variables, the conditions. The only summary I can find is biplot scores. Looking through the documentation for &lt;code&gt;vegan&lt;/code&gt; I can't find any description of how they are computed. Can I sum them across CCA components to get an idea of how much they influence the data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a biplot of two CCA components. The scores are used as coordinates for the arrows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kQruf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kQruf.png&quot; alt=&quot;&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So two questions:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What are biplot scores in the context of CCA?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Can biplot scores be used to determine how much of an effect conditions have on the response variables?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="181" LastEditorUserId="181" LastEditDate="2019-08-31T17:56:26.313" LastActivityDate="2019-08-31T17:56:26.313" Title="Biplot scores from canonical correspondence analysis" Tags="|r|statistics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9300" PostTypeId="1" CreationDate="2019-08-30T19:14:49.137" Score="1" ViewCount="42" Body="&lt;p&gt;I have a vector of 200 million numbers. It has been running for over 24 hours and still not done. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x &amp;lt;- data   # the target sequence&#xA;k &amp;lt;- exp(-(-win:win)^2/(2*sig^2))/(sig*sqrt(2*pi))  # gaussian kernel&#xA;ptm &amp;lt;- proc.time()&#xA;y &amp;lt;- convolve(x,k,type=&quot;open&quot;)&#xA;proc.time()-ptm&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How to parallelize the &lt;a href=&quot;https://www.rdocumentation.org/packages/stats/versions/3.6.1/topics/convolve&quot; rel=&quot;nofollow noreferrer&quot;&gt;convolve&lt;/a&gt; function?&#xA;To parallelize, I tried splitting up the vector into pieces:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; x &amp;lt;- data[1:1e7]&#xA;&amp;gt; ptm &amp;lt;- proc.time()&#xA;&amp;gt; y &amp;lt;- convolve(x,k,type=&quot;open&quot;) # smoothed sequence&#xA;&amp;gt; proc.time()-ptm&#xA;   user  system elapsed &#xA; 40.743   0.206  40.940 &#xA;&amp;gt; x &amp;lt;- data[1e7+1:2e7]&#xA;&amp;gt; ptm &amp;lt;- proc.time()&#xA;&amp;gt; y &amp;lt;- convolve(x,k,type=&quot;open&quot;) # smoothed sequence&#xA;&amp;gt; proc.time()-ptm&#xA;   user  system elapsed &#xA;320.193   0.708 320.871 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, splitting it up and then adding back the convolutions will be tricky. &#xA;Does anyone have other ideas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This data is based on biological data. We want to cluster values in the vector based on nearby vector indices.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reproducible example with 20,000,000 random numbers should finish in about 5.5-6 minutes. But if you try running 200,000,000 locally or on HPC, it will take a long time (my HPC has limit 99 hours and it timed out at 99 hours unfinished).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;win &amp;lt;- 100        # window length of gaussian kernel (units acc to orig sequence)&#xA;sig &amp;lt;- 5        # std deviation of gaussian&#xA;k &amp;lt;- exp(-(-win:win)^2/(2*sig^2))/(sig*sqrt(2*pi))  # gaussian kernel&#xA;l &amp;lt;- 20000000&#xA;x &amp;lt;- round(runif(l, 0.0, 1.0),1)   # the target sequence with peaks&#xA;t &amp;lt;- seq(0,1-1/length(x),1/length(x))&#xA;ptm &amp;lt;- proc.time()&#xA;y &amp;lt;- convolve(x,k,type=&quot;open&quot;)[(win+1):(win+length(x))] # smoothed sequence&#xA;proc.time()-ptm&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5119" LastEditorUserId="5119" LastEditDate="2019-09-03T16:03:46.430" LastActivityDate="2019-09-03T16:03:46.430" Title="How to run function &quot;convolve&quot; in parallel in R?" Tags="|r|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9302" PostTypeId="2" ParentId="9296" CreationDate="2019-08-31T16:03:44.233" Score="-1" Body="&lt;p&gt;Canonical correlation analysis (CCA) not correspondence canonical analysis. Biplots are the same as in PCA: the representation in a new space of the samples and the relative position of the variables on the new space. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, CCA seeks the values of one block that maximize the correlation with the other block. Here you don't have an independent variable that you maximize from the variables you introduced on the CCA. So the interpretation is much trickier as there isn't a direction: WaterC and FallenTwigs show the opposite effect than BarSand and CoverMoss, but there is a group of variables that explain what is the effect of water on the plants(?)&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2019-08-31T16:03:44.233" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9304" PostTypeId="1" CreationDate="2019-09-03T03:14:40.120" Score="2" ViewCount="143" Body="&lt;p&gt;I used Kraken 2 to classify my 16S metagenomic data using both RDP and Greengenes database. As these are special databases the taxonomic ids assigned  do not match with the NCBI taxonomic ids. I found Krona uses NCBI taxonomy as its default. Krona github page has the following instruction regarding RDP database: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Taxonomic classifications of the Ribosomal Database Project 16S&#xA;  Classifier can be imported using ktImportRDP. To download&#xA;  classifications:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Run the classification tool.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Click &quot;show assignment detail&quot; for the root of the hierarchy.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Click &quot;download allrank result&quot; or &quot;download fixrank result&quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Problem is the online RDP classifier (&lt;a href=&quot;http://rdp.cme.msu.edu/classifier/classifier.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://rdp.cme.msu.edu/classifier/classifier.jsp&lt;/a&gt;) doesn't work and as far as I understood the above mentioned help applies only for web version of the tool. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And for Greengenes taxonomy id file found &lt;a href=&quot;https://greengenes.secondgenome.com/?prefix=downloads/greengenes_database/gg_13_5/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; , it is in different format than the one Krona uses. Krona taxonomy file looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   0   1   no rank root&#xA;2   2   131567  superkingdom    Bacteria&#xA;6   7   335928  genus   Azorhizobium&#xA;7   8   6   species Azorhizobium caulinodans&#xA;9   8   32199   species Buchnera aphidicola&#xA;10  7   1706371 genus   Cellvibrio&#xA;11  9   1707    species Cellulomonas gilvus&#xA;13  7   203488  genus   Dictyoglomus&#xA;14  8   13  species Dictyoglomus thermophilum&#xA;16  7   32011   genus   Methylophilus&#xA;17  8   16  species Methylophilus methylotrophus&#xA;18  8   213421  genus   Pelobacter&#xA;19  9   18  species Pelobacter carbinolicus&#xA;20  7   76892   genus   Phenylobacterium&#xA;21  8   20  species Phenylobacterium immobile&#xA;....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And Greengenes taxonomy file mentioned above looks like following: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;228054  k__Bacteria; p__Cyanobacteria; c__Synechococcophycideae; o__Synechococcales; f__Synechococcaceae; g__Synechococcus; s__&#xA;844608  k__Bacteria; p__Cyanobacteria; c__Synechococcophycideae; o__Synechococcales; f__Synechococcaceae; g__Synechococcus; s__&#xA;178780  k__Bacteria; p__Cyanobacteria; c__Synechococcophycideae; o__Synechococcales; f__Synechococcaceae; g__Synechococcus; s__&#xA;198479  k__Bacteria; p__Cyanobacteria; c__Synechococcophycideae; o__Synechococcales; f__Synechococcaceae; g__Synechococcus; s__&#xA;187280  k__Bacteria; p__Cyanobacteria; c__Synechococcophycideae; o__Synechococcales; f__Synechococcaceae; g__Synechococcus; s__&#xA;179180  k__Bacteria; p__Cyanobacteria; c__Synechococcophycideae; o__Synechococcales; f__Synechococcaceae; g__Synechococcus; s__&#xA;175058  k__Bacteria; p__Cyanobacteria; c__Synechococcophycideae; o__Synechococcales; f__Synechococcaceae; g__Synechococcus; s__&#xA;176884  k__Bacteria; p__Cyanobacteria; c__Synechococcophycideae; o__Synechococcales; f__Synechococcaceae; g__Synechococcus; s__&#xA;228057  k__Bacteria; p__Proteobacteria; c__Alphaproteobacteria; o__Rickettsiales; f__Pelagibacteraceae; g__; s__&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am stuck here. Where can I find taxonomy databases that use the format Krona uses? Is there any tool I can use to convert between these formats? &lt;/p&gt;&#xA;" OwnerUserId="3555" LastEditorUserId="3555" LastEditDate="2019-09-03T03:31:29.283" LastActivityDate="2019-09-03T03:31:29.283" Title="How to apply RDP, Greengenes and other special taxonomies in Krona?" Tags="|database|genomics|metagenome|taxonomy|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9306" PostTypeId="2" ParentId="9286" CreationDate="2019-09-03T07:20:32.453" Score="1" Body="&lt;p&gt;The answer is to use &lt;code&gt;phastaf&lt;/code&gt;, a recently developed software tool developed by Torsten Seemann (the same developer as prokka), which comes with a packaged phaster.ca db.  Get it &lt;a href=&quot;https://github.com/tseemann/phastaf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3381" LastActivityDate="2019-09-03T07:20:32.453" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9307" PostTypeId="2" ParentId="9119" CreationDate="2019-09-03T07:56:33.203" Score="0" Body="&lt;p&gt;Run them through the &lt;a href=&quot;http://www.ensembl.org/info/docs/tools/vep/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl VEP&lt;/a&gt;. You'll get the gene name and all the information about how the genes are affected by the variants.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2019-09-03T07:56:33.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9308" PostTypeId="1" AcceptedAnswerId="9310" CreationDate="2019-09-03T20:14:04.317" Score="1" ViewCount="94" Body="&lt;p&gt;I am trying to calculate unifrac distance for my OTU table, which has about 3000 taxa, and 300 samples. My phylogenetic tree only considers the most important 500 taxa. My questions are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Do I ignore the 2500 taxa that were not considered in the phylogenetic tree?&lt;/li&gt;&#xA;&lt;li&gt;Is their a package in R that deals with this kind of missing data?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="5158" LastEditorUserId="3848" LastEditDate="2020-05-02T19:21:47.197" LastActivityDate="2020-05-02T19:21:47.197" Title="Calculating Unifrac Distance and missing data" Tags="|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9309" PostTypeId="1" AcceptedAnswerId="9326" CreationDate="2019-09-04T03:10:55.073" Score="2" ViewCount="112" Body="&lt;p&gt;This is a follow up question to this post: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/9271/how-can-i-find-out-if-my-gene-of-interest-is-duplicated&quot;&gt;How can I find out if my gene of interest is duplicated?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Briefly, to identify homologous genes, I performed tBLASTn analyses against my genome of interest using various sequences of a closely related species as queries for the BLAST analyses. Using this method, I found two (and in some cases up to five copies) of my different genes of interest in the target genome. These genes are involved in different organelle functions (i.e., they are not surface antigens) and I want to find out whether these sequences represent duplication events or not. These analyses are being performed on protozoans parasites that belong to the same genus. The query genome is properly annotated, but the target genome is not (it is a whole genome shotgun project that has been assembled at the scaffold level).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following the advice of @Michael G., I re-analyzed my sequences and blasted them against the entirety of Genbank. If there are two sequences for the same protein (for e.g. protein 3-1 and protein 3-2) and I compare them among themselves they are very similar (over 70% identity at the protein level). When I blast each sequence against the whole protein RefSeq database, the first hit is my query and the rest of the hits correspond to the same protein in other organisms. I believe that these sequences represent duplication events and it is possible that they are paralogues, but more evidence is needed to classify them as such.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when there are more than two sequences for the same protein, I found something different. For e.g., say I have three sequences for the same protein: sequence 5-1, sequence 5-2 and sequence 5-3. Sequences 5-2 and 5-3 are very similar (78% identity at the protein level) but when I compare sequences 5-1 and 5-2 I find that they are overlap in the middle, i.e., they are part of the same sequence. Thus, it appears that I don't really have three sequences for this protein but two. Blastp analyses against the whole RefSeq database give the expected result (first hit is my query sequence, the rest of the hits are same sequence in other organisms). So, once again I have what appears to be a duplication event, but in this case one of the sequences appears to be split in two -- could this be because the target genome is not completely assembled? Why could this be happening?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much for your help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-Leah&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: The new results are posted in this thread: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/10365/duplicated-genes-and-genome-fragmentation-part-2&quot;&gt;Duplicated genes and genome fragmentation - Part 2&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5116" LastEditorUserId="5116" LastEditDate="2019-09-11T03:28:35.617" LastActivityDate="2019-09-11T03:28:35.617" Title="Duplicated genes and genome fragmentation" Tags="|phylogenetics|phylogeny|duplications|" AnswerCount="5" CommentCount="9" ClosedDate="2019-09-11T08:29:28.937" ContentLicense="CC BY-SA 4.0" />
  <row Id="9310" PostTypeId="2" ParentId="9308" CreationDate="2019-09-04T07:14:51.267" Score="1" Body="&lt;p&gt;To calculate the unifrac distance you need the phylogenetic tree. If you only have it for 500 taxa then you can only calculate it for those taxa. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is impossible to impute the phylogenetic tree without knowing the sequence on which the taxa is based. But if you know it then you can calculate the phylogenetic tree for the 3k taxa you have. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2019-09-04T07:14:51.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9311" PostTypeId="1" CreationDate="2019-09-04T12:19:53.167" Score="1" ViewCount="142" Body="&lt;p&gt;I have construct a repeat libraries using de-novo identification methods. All  results from different tools has been combined into one fasta file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I get the percentage of how much these sequences are found in the original genome? That has been used earlier for repeat detections.&lt;br&gt;&#xA;For example these identified candidates account for 30% from the total genome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used repeatmasker with a flag &lt;code&gt;-lib combinLibrary.fasta mygenome.fasta&lt;/code&gt; but its very slow now almost one week and still far to finish. &lt;/p&gt;&#xA;" OwnerUserId="3332" LastEditorUserId="3332" LastEditDate="2019-09-10T05:40:21.657" LastActivityDate="2019-09-10T05:40:21.657" Title="Calculate the percentage of the genome that is represented by some sequences" Tags="|database|repeat-elements|repeatmasker|repeat|" AnswerCount="0" CommentCount="7" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9312" PostTypeId="1" CreationDate="2019-09-05T08:32:09.950" Score="1" ViewCount="692" Body="&lt;p&gt;I have a list of scientific names of species. Is there a python module that can fetch &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Taxonomy/TaxIdentifier/tax_identifier.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI taxonomy IDs&lt;/a&gt;? &lt;/p&gt;&#xA;" OwnerUserId="3555" LastEditorUserId="3848" LastEditDate="2020-04-25T09:21:04.560" LastActivityDate="2022-03-11T17:26:51.983" Title="Python module for fetching NCBI id for a list of species" Tags="|python|ngs|genomics|metagenome|taxonomy|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9314" PostTypeId="2" ParentId="9309" CreationDate="2019-09-05T11:52:34.863" Score="1" Body="&lt;p&gt;The tBlastn looks good and looks like some sort of parology, although the hits you are getting need to be checked to make sure they are not psuedo-genes. The shot-gun genome however appears poorly annotated because it had misannotated 5-1 and 5-2. What I don't understand is why you haven't performed a blast back over the original highly annotated genome? You must have done this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few avenues to go down here, what I'd like to know which group of protists are you working with? I don't need to know the species, but I do need to know the family please.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-09-05T11:52:34.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9315" PostTypeId="2" ParentId="9312" CreationDate="2019-09-05T12:04:10.667" Score="1" Body="&lt;p&gt;Biopython has an efetch frontend.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez&#xA;Entrez.email = &amp;quot;mg@example.edu&amp;quot; &#xA;handle = Entrez.efetch(db=&amp;quot;nucleotide&amp;quot;, id=&amp;quot;KR052012.1&amp;quot;, rettype=&amp;quot;gb&amp;quot;, retmode=&amp;quot;text&amp;quot;)&#xA;print(handle.read())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;OUT&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;LOCUS       KR052012               10714 bp    RNA     linear   VRL 09-MAY-2015&#xA;DEFINITION  Dengue virus strain Hb33/CHN/2014, complete genome.&#xA;ACCESSION   KR052012&#xA;VERSION     KR052012.1&#xA;KEYWORDS&lt;br /&gt;&#xA;.... etc&lt;/p&gt;&#xA;&lt;p&gt;You will likely need to perform an EFetch search and then extract out the accession numbers for import (as done above)&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-03-11T17:26:51.983" LastActivityDate="2022-03-11T17:26:51.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9316" PostTypeId="2" ParentId="9312" CreationDate="2019-09-05T14:26:53.857" Score="0" Body="&lt;p&gt;If you want to perform a species search, here's the raw efetch code. I use this within an alias and the complex quotes are needed for complicated reasons. The reg-ex may need to be adapted to your search. $virus here is your species of interest, e.g. &lt;code&gt;set virus='some species'&lt;/code&gt; (csh or tcsh), `virus='some species' (bash). You'll just need to translate this to Biopython.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;alias esummary-acc \&#xA;'esearch -db protein -query &quot;&lt;span class=&quot;math-container&quot;&gt;$virus&quot; |\&#xA;efetch -format docsum |\&#xA;xtract -pattern DocumentSummary -element Title,AccessionVersion |\&#xA;grep &quot;$&lt;/span&gt;virus&quot; |\&#xA;perl -p -e '&quot;'&quot;'s/^.*\[.*\]\s+(\w+)/$1/'&quot;'&quot;''&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-09-05T14:26:53.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9321" PostTypeId="2" ParentId="9309" CreationDate="2019-09-06T03:47:12.020" Score="1" Body="&lt;p&gt;Many thanks for your response!&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What I don't understand is why you haven't performed a blast back over the original highly annotated genome? You must have done this?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, I have performed a blast back over the original highly annotated genome and, as expected, the first hit is the sequence that I used as query in the first blast analysis. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;There are a few avenues to go down here, what I'd like to know which group of protists are you working with? I don't need to know the species, but I do need to know the family please.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am working with Amoebozoans -- specifically, with members of the group Discosea - Centramoebida.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once again, thank you very much for your help.&lt;/p&gt;&#xA;" OwnerUserId="5116" LastActivityDate="2019-09-06T03:47:12.020" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9322" PostTypeId="2" ParentId="9156" CreationDate="2019-09-06T18:02:06.333" Score="1" Body="&lt;p&gt;You are almost there except for a few tricks:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want columns to be ordered using the values in Cell 4, you will need to provide a custom column order either with the &lt;code&gt;column_order&lt;/code&gt; argument of the &lt;code&gt;Heatmap()&lt;/code&gt; function or by providing ordered data. Below is the code to perform the latter. Most of the code is yours, I just tweaked it a little.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(tibble)&#xA;library(dplyr)&#xA;library(ComplexHeatmap)&#xA;library(RColorBrewer)&#xA;library(circlize)&#xA;&#xA;# Since you want to use cell4 as reference&#xA;cell4 &amp;lt;- as.data.frame(t(data[&quot;Cell4&quot;,]))&#xA;&#xA;df &amp;lt;- cbind(cell4,&#xA;            Group = as.character(ann$Group))&#xA;&#xA;# ordering by Group and by expression value, minus denotes decreasing order&#xA;df &amp;lt;- df[with(df, order(Group,-Cell4)),]&#xA;&#xA;# to make it compatible to your code in your post&#xA;df &amp;lt;- rownames_to_column(df, &quot;Samples&quot;)&#xA;&#xA;#Set annotation&#xA;ann &amp;lt;- data.frame(df$Group)&#xA;colnames(ann) &amp;lt;- c(&quot;Group&quot;)&#xA;colours &amp;lt;- list(&quot;Group&quot; = c(&quot;A&quot; = &quot;orange&quot;, &quot;B&quot; = &quot;darkgreen&quot;))&#xA;colAnn &amp;lt;-&#xA;  HeatmapAnnotation(&#xA;    df = ann,&#xA;    which = &quot;col&quot;,&#xA;    col = colours,&#xA;    annotation_width = unit(c(1, 4), &quot;cm&quot;),&#xA;    gap = unit(1, &quot;mm&quot;)&#xA;  )&#xA;&#xA;# rownames of the annotation df and the colnames of the data should be in&#xA;# the same order&#xA;data &amp;lt;- data[, df$Samples]&#xA;&#xA;myCol &amp;lt;-&#xA;  colorRampPalette(c(&quot;#ffffff&quot;, &quot;#0030dd&quot;, &quot;#3259e3&quot;, &quot;#7f97ee&quot;))(100)&#xA;myBreaks &amp;lt;- seq(0, 1, length = 100)&#xA;hmap &amp;lt;-&#xA;  Heatmap(&#xA;    as.matrix(data),&#xA;    name = &quot;xCell scores&quot;,&#xA;    col = colorRamp2(myBreaks, myCol),&#xA;    width = unit(100, &quot;mm&quot;),&#xA;    show_row_names = TRUE,&#xA;    show_column_names = FALSE,&#xA;    cluster_rows = TRUE, # you asked your cells to be ordered&#xA;    cluster_columns = FALSE, # column order is the same as the provided data&#xA;    show_column_dend = FALSE,&#xA;    show_row_dend = FALSE,&#xA;    row_dend_reorder = FALSE,&#xA;    column_dend_reorder = FALSE,&#xA;    clustering_method_rows = &quot;ward.D2&quot;,&#xA;    top_annotation = colAnn,&#xA;    column_order = NULL&#xA;  )&#xA;draw(hmap,&#xA;     heatmap_legend_side = &quot;right&quot;,&#xA;     annotation_legend_side = &quot;right&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The resulting heatmap, still not quite what you want:&lt;a href=&quot;https://i.stack.imgur.com/BrXMI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BrXMI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will realize that the values for Cell 4 are ordered from high to low for each of the two groups. To make Cell 4 be drawn at the top, you will need to reorder the rows. To do so, you will need to extract the row orders with the &lt;code&gt;row_order()&lt;/code&gt; function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; row_order(hmap)&#xA; [1]  3  2 14  4 12 11  9 10  7  1  5 13  6  8&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now you can re-draw the heatmap after altering the row order somehow. For this sample case, doing it manually.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# promoting Cell 4 to the first position&#xA;modified_row_order &amp;lt;- c(4, 3, 2, 14, 12, 11,  9, 10,  7,  1,  5, 13,  6,  8)&#xA;&#xA;hmap &amp;lt;-&#xA;  Heatmap(&#xA;    as.matrix(data),&#xA;    name = &quot;xCell scores&quot;,&#xA;    col = colorRamp2(myBreaks, myCol),&#xA;    width = unit(100, &quot;mm&quot;),&#xA;    show_row_names = TRUE,&#xA;    show_column_names = FALSE,&#xA;    cluster_rows = FALSE, # set to FALSE as a custom row order is supplied&#xA;    cluster_columns = FALSE,&#xA;    show_column_dend = FALSE,&#xA;    show_row_dend = FALSE,&#xA;    row_dend_reorder = FALSE,&#xA;    column_dend_reorder = FALSE,&#xA;    clustering_method_rows = &quot;ward.D2&quot;,&#xA;    top_annotation = colAnn,&#xA;    column_order = NULL,&#xA;    row_order = modified_row_order # argument to supply custom row order&#xA;  )&#xA;draw(hmap,&#xA;     heatmap_legend_side = &quot;right&quot;,&#xA;     annotation_legend_side = &quot;right&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FrBUr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FrBUr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I have misunderstood your question and you do not want columns to be ordered according to expression values in Cell4, forget about the first part with the ordered annotation data frame.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-09-06T18:02:06.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9323" PostTypeId="1" AcceptedAnswerId="9325" CreationDate="2019-09-06T19:16:09.083" Score="0" ViewCount="220" Body="&lt;p&gt;I've RNA seq, Human, Paired-end data, Sample size is &amp;lt;40. These are aligned using STAR, RSEM processed. With RSEM I've TPM and expected counts, that is two files columns as individual IDs and row as gene names.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm interested to normalize gene data. With edgeR tutorial (link in the end) and few other online resources I see that after following steps there's an R object that contains &lt;code&gt;norm.factors&lt;/code&gt; (Page 15) value for each individual.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm unable to wrap my head around it, now. If I'm interested to get normalized gene counts, can I go ahead and multiple each individual's norm.factor into its gene counts? For example, the expected count for &lt;code&gt;IND1&lt;/code&gt; for &lt;code&gt;Gene1&lt;/code&gt; is 100 and it's &lt;code&gt;norm.factor&lt;/code&gt; is &lt;code&gt;0.80&lt;/code&gt;, can I say that the normalized gene count is &lt;code&gt;100*0.80=80&lt;/code&gt;?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not interested to perform voomm, or differential expression analysis.     &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Follow up question: If I'm to sample few individuals' data from these RNA-seq should I&lt;br&gt;&#xA;a) normalize all together and extract interested individuals, or,&lt;br&gt;&#xA;b) extract interested individuals and then perform normalization.&lt;br&gt;&#xA;I think since normalization uses per person library size, each person's normalization is independent of the others.  Can someone please correct me if I'm wrong?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/vignettes/edgeR/inst/doc/edgeRUsersGuide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioconductor.org/packages/release/bioc/vignettes/edgeR/inst/doc/edgeRUsersGuide.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="862" LastActivityDate="2019-09-06T20:47:12.757" Title="RNA-seq: How to get new expression count after normalization" Tags="|rna-seq|normalization|edger|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9325" PostTypeId="2" ParentId="9323" CreationDate="2019-09-06T20:47:12.757" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If I'm interested to get normalized gene counts, can I go ahead and&#xA;  multiple each individual's norm.factor into its gene counts? For&#xA;  example, the expected count for IND1 for Gene1 is 100 and it's&#xA;  norm.factor is 0.80, can I say that the normalized gene count is&#xA;  100*0.80=80?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No; you divide, but if you poke around, you can probably find a way to get edgeR to return the normalized counts.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I think since normalization uses per person library size, each&#xA;  person's normalization is independent of the others.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Not true for the default normalization used in edgeR, TMM.  CPM normalization is independently determined for each sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, normalize everything together, unless you think that the algorithm assumptions might be violated by including all of them together.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-09-06T20:47:12.757" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9326" PostTypeId="2" ParentId="9309" CreationDate="2019-09-07T15:26:20.373" Score="1" Body="&lt;p&gt;The strategy from here is to determine duplication/parology through phylogeny, notably amino acid phylogeny using statistical support.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic result is whether the genes under investigation are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A. monophyletic to the given species under investigation&lt;/li&gt;&#xA;&lt;li&gt;B. Paraphyletic against protist species which are not under investigation,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Point A is compatible with paraology/duplication within the species. Point B infers the duplication is ancient. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The skill/difficulty is building a comprehensive alignment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before we start, &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Parology is usually underestimated within a genome because repetitive sequences are often collapsed in the assembly process. Thus a de novo assembly around the gene of interest in both annotated and shot-gun assemblies might yield better results, ie. more duplicates. You have already seen what appear to be N-terminal and C-terminal ends of a related gene fragmented in the assembly process. &lt;/li&gt;&#xA;&lt;li&gt;The tBlastn results looked quite cool and need to be included.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The core approach is to blastp your query sequence against the full NBCI database. Blast has an option to automatically download all hits (or all selected hits), e.g. in fasta format. This will provide a multi-species data set to understand the evolution of the protein of interest. This feature of blast is very cool and saves a lot of work. You want to combine the output of point 1 and point 2 in the fasta blastp list and align everything via ClustalO or Muscle (any alignment program). Then manually edit the AA alignment via an alignment editor (I use &lt;a href=&quot;https://bmcresnotes.biomedcentral.com/articles/10.1186/s13104-016-1927-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seqotron&lt;/a&gt; [Mac]) and thereon produce a very basic tree using neighbor-joining just to check everything looks ok. The NCBI blast trick might need a better explanation so let me know if you have trouble figuring this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you post the tree as a PHYLOGRAM (I don't need to see the sequence names - just the shape and branch lengths) we can then see whether it looks reasonable before commencing a heavy duty algorhitm to identify support for the competeting hypotheses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not ideal doing this via a message server, but its doable.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-09-07T15:26:20.373" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9327" PostTypeId="1" AcceptedAnswerId="9328" CreationDate="2019-09-07T16:59:48.517" Score="3" ViewCount="858" Body="&lt;p&gt;I automated a PDB download using a Pymol script (below)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;python&#xA;pdb_lists = ['3LZG', '6HJN'] # lots more pdbs&#xA;for x in pdb_lists:&#xA;    cmd.fetch(x)&#xA;    cmd.select(x)&#xA;    cmd.save(x + '.pdb', x)&#xA;    cmd.delete(x)&#xA;cmd.quit()&#xA;python end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When the script is run within Pymol it pulls down a collection of pdbs, however this flopped when I  plugged it through a Biopython PDB parser, but was okay when I manually downloaded the same pdb file from RCSB.org&lt;/p&gt;&#xA;&lt;p&gt;A manual downloaded PDB was perfect, meaning the amino acids which cannot be observed by X-ray crystalography (e.g. they not sufficiently stationary) were recognised as being absent from the parser (resulting in a protein ~390 amino acids), but the Pymol scripted download contained all amino acid residues, i.e. even those which couldn't be observed in the X-ray (resulting in a protein of ~440 amino acid residues). Any ideas why the script resulted in a pdb that flopped and how to correct it?&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-12-20T10:39:08.363" LastActivityDate="2020-12-20T10:39:25.147" Title="PDB file downloading: pymol automation vs. manual" Tags="|python|phylogenetics|phylogeny|pdb|pymol|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9328" PostTypeId="2" ParentId="9327" CreationDate="2019-09-07T20:21:02.967" Score="4" Body="&lt;p&gt;To expand on the comment by marcin:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;fetch&lt;/code&gt; downloads files in mmCIF format by default (&lt;a href=&quot;https://pymolwiki.org/index.php/Fetch&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pymolwiki.org/index.php/Fetch&lt;/a&gt;). Not all PDB entries have PDB format files, e.g. due to too many chains. Presumably this is why the change was made, though mmCIF files tend to be larger and hence download slower.&lt;/li&gt;&#xA;&lt;li&gt;When calling &lt;code&gt;save&lt;/code&gt; with the &lt;code&gt;.pdb&lt;/code&gt; file extension the structure is converted to PDB format. Perhaps the converter uses the SEQRES records to define the sequence, and hence gives bad files when there are some residues not resolved.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;To correct it you could try:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Saving as &lt;code&gt;.cif&lt;/code&gt; and reading in with the Biopython mmCIF parser.&lt;/li&gt;&#xA;&lt;li&gt;Providing the &lt;code&gt;type=pdb&lt;/code&gt; argument to &lt;code&gt;fetch&lt;/code&gt;, which will force PDB format downloads. Some PDB files may not be available.&lt;/li&gt;&#xA;&lt;li&gt;Don't do it in PyMol and instead use the Biopython structure downloader (&lt;a href=&quot;http://biopython.org/DIST/docs/tutorial/Tutorial.html#htoc187&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://biopython.org/DIST/docs/tutorial/Tutorial.html#htoc187&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Also, if download speed is limiting you then consider using the MMTF file format, which is a binary format and hence smaller. Biopython can read in and write out MMTF files.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2019-09-07T20:21:02.967" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9329" PostTypeId="2" ParentId="3121" CreationDate="2019-09-08T18:29:00.987" Score="0" Body="&lt;p&gt;SCmut is a method to detect cell-level mutation from single-cell RNA-sequencing, published by &lt;a href=&quot;https://academic.oup.com/bioinformatics/advance-article/doi/10.1093/bioinformatics/btz288/5480302&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vu et al. (2019)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/nghiavtr/SCmut&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nghiavtr/SCmut&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2019-09-08T18:29:00.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9330" PostTypeId="1" CreationDate="2019-09-09T02:56:09.370" Score="1" ViewCount="222" Body="&lt;p&gt;I'm trying to plot a simple distance tree of my blastn output with nj (like the tree view on NCBI). From what I understood, what I think I should do is &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;extract all the hsps from each alignment&lt;/li&gt;&#xA;&lt;li&gt;re-align them using MUSCLE &lt;/li&gt;&#xA;&lt;li&gt;generate a distance matrix from the msa file&lt;/li&gt;&#xA;&lt;li&gt;plot the tree using nj&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;However, there is a couple problems I am not sure about:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) When there are multiple hsps in one alignment then which one do I include in the tree?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) If two sequences map to two different parts of the query sequence and do not overlap, then what should I do?&lt;/p&gt;&#xA;" OwnerUserId="5114" LastEditorUserId="3848" LastEditDate="2020-04-24T18:14:02.987" LastActivityDate="2020-09-21T20:00:45.083" Title="Plotting distance tree from blastn output" Tags="|ngs|blast|blastn|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="9331" PostTypeId="2" ParentId="9309" CreationDate="2019-09-09T03:02:20.447" Score="1" Body="&lt;p&gt;So this is what I did to produce the phylogenetic tree:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I performed a BLASTp using the &lt;strong&gt;&lt;em&gt;query&lt;/em&gt;&lt;/strong&gt; sequence (lets call it sequence 5 from species &lt;strong&gt;&lt;em&gt;A&lt;/em&gt;&lt;/strong&gt;, which is thoroughly annotated) against full NCBI protein database (ref-seq database)&lt;/li&gt;&#xA;&lt;li&gt;I downloaded all the hits in fasta format (very cool feature indeed!)&lt;/li&gt;&#xA;&lt;li&gt;Here I was a bit confused, so what I did was to take the three hits I got in my &lt;strong&gt;&lt;em&gt;target&lt;/em&gt;&lt;/strong&gt; genome (sequences 5-1, 5-2 and 5-3 from species &lt;strong&gt;&lt;em&gt;B&lt;/em&gt;&lt;/strong&gt;, which is poorly annotated) and added them to the fasta blastp list&lt;/li&gt;&#xA;&lt;li&gt;I took the fasta blastp list and loaded it into Seqotron.  &lt;/li&gt;&#xA;&lt;li&gt;I aligned everything using the Compute - Alignment - MUSCLE option -- I didn't do any manual AA alignment (not too sure about this part)&lt;/li&gt;&#xA;&lt;li&gt;I produced the tree using Compute - Phylogenetic Tree (default parameters: distance matrix: K83, resampling: none). &lt;/li&gt;&#xA;&lt;li&gt;This is the tree I got. Sequences from the &lt;strong&gt;&lt;em&gt;target&lt;/em&gt;&lt;/strong&gt; genome (species &lt;strong&gt;B&lt;/strong&gt;) are shown with a red rectangle:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LeFrr.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LeFrr.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please let me know if I missed something or if I made a mistake somewhere. Many thanks!&#xA;-Leah&lt;/p&gt;&#xA;" OwnerUserId="5116" LastActivityDate="2019-09-09T03:02:20.447" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9332" PostTypeId="1" CreationDate="2019-09-09T04:55:52.333" Score="0" ViewCount="25" Body="&lt;p&gt;I have yeast growth data in various experimental conditions that are classified by the strain type. Can I use the GSEA tool from the Broad Institute to test for enrichment of specific groups of strains within the total population (i.e. gene sets)? If not, would something like a hypergeomtric test be useful for this kind of analysis?&lt;/p&gt;&#xA;" OwnerUserId="4266" LastActivityDate="2019-09-09T07:09:05.617" Title="Can I use GSEA to look for enrichment in data that is grouped into sets but are not necessarily genes?" Tags="|gsea|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9333" PostTypeId="2" ParentId="9332" CreationDate="2019-09-09T07:09:05.617" Score="1" Body="&lt;p&gt;Yes, if you have a ranked list, you can use GSEA to calculate the enrichment of the strains along the ranked list. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that hypergeometric test is for count data, so you would test if there are more strains of one type that fit on certain conditions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would recommend &lt;a href=&quot;https://doi.org/10.1371/journal.pcbi.1002375&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; comparing different enrichment methods to understand well what they do. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2019-09-09T07:09:05.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9334" PostTypeId="1" AcceptedAnswerId="9346" CreationDate="2019-09-09T12:08:01.370" Score="4" ViewCount="2214" Body="&lt;p&gt;I have read this question (&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/6914/running-snakemake-in-one-single-conda-env&quot;&gt;Running Snakemake in one single conda env&lt;/a&gt;) but still have some doubt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using the '--conda-prefix /some/dir' option and I have a rule in my snakemake file with the directive:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule xxx:&#xA;  ...&#xA;  conda:    &#xA;    &quot;envs/some.yaml&quot;&#xA;  ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first call of that rule will create the conda environment according to the yaml file in /some/dir/envs with some random name (in my case 'bf55a79e'). If I run the workflow in a different directory, the same conda environment (/some/dir/envs/bf55a79e) gets activated directly instead of being built again. So this is what I expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;is there some way of skipping the building and specifying that rule xxx should use an already existing conda environment somewhere? &lt;/li&gt;&#xA;&lt;li&gt;does anyone know where the mapping between rule and directory gets stored?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4733" LastActivityDate="2019-09-10T09:45:45.123" Title="snakemake with --conda-prefix, can I use a pre-built conda environment?" Tags="|snakemake|conda|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9335" PostTypeId="1" CreationDate="2019-09-09T12:13:08.367" Score="2" ViewCount="339" Body="&lt;p&gt;when mapping short reads with bwa mem, if a read has alternative mapping positions they are reported by bwa mem in the X0 and XA tags. &#xA;Now, let's say I want to compute the coverage of my bam file. How would I do for the alternative hits to be taken into accounts. What I mean is that, if a read QUERY has a selected best HIT position and 3 alternate HIT positions, I would like that QUERY read to be counted as 1 unit of coverage for the best HIT position, but also as 1 unit of coverage for each of the alternate HIT position. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding is that samtools depth will only consider the best hit (which makes sense in 99.99% of the cases). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess I could edit the bam file with custom scripts so that the alternative hits are also reported as primary hits but I was wondering if there was an already existing tool (as I am not that confident with my ability to mess around with sam files). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot.&lt;/p&gt;&#xA;" OwnerUserId="5176" LastActivityDate="2020-02-07T11:03:05.463" Title="How to take into account alternative bwa mem mapping when computing coverage" Tags="|samtools|bwa|coverage|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9336" PostTypeId="2" ParentId="9327" CreationDate="2019-09-09T13:23:22.250" Score="1" Body="&lt;p&gt;So this was the code I used and it worked (as far as I could see) and refers to point 2 of @jgreener's advice and was based on the comment by @marcin&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;python&#xA;pdb_lists = ['3LZG', '6HJN'] # lots more pdbs&#xA;for x in pdb_lists:&#xA;    cmd.fetch(x, type='pdb')&#xA;    cmd.select(x)&#xA;    cmd.save(x + '.pdb', x)&#xA;    cmd.delete(x)&#xA;cmd.quit()&#xA;python end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There are some Bio SE members who have serious skills in Pymol scripting, and constructive critique would be very welcome.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-12-20T10:39:25.147" LastActivityDate="2020-12-20T10:39:25.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9337" PostTypeId="2" ParentId="7106" CreationDate="2019-09-09T14:08:24.170" Score="1" Body="&lt;p&gt;You are trying to &quot;map&quot; mutation data to your heatmap, however, there is no mutation data in your code as far as I can see! You specify sample names within a data frame called &lt;code&gt;anno&lt;/code&gt; but this data frame has only one column, &lt;code&gt;Condition&lt;/code&gt;. All you have to do is add another column, &lt;code&gt;mutation&lt;/code&gt;, to the &lt;code&gt;anno&lt;/code&gt; data frame. Moreover, you have to pay attention to the order of your data and your meta data. The rownames of the meta data (&lt;code&gt;anno&lt;/code&gt; data frame in your case) should be the same as the colnames of your gene expression data frame.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also see that you create a row annotation object &lt;code&gt;RowAnn&lt;/code&gt;, which is not used in the &lt;code&gt;Heatmap()&lt;/code&gt; call. Similarly, there is a &lt;code&gt;categories&lt;/code&gt; data frame which is not used either. Your code would be easier to read without these unused variables.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-09-09T14:08:24.170" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9338" PostTypeId="1" AcceptedAnswerId="9356" CreationDate="2019-09-09T16:44:18.120" Score="1" ViewCount="2190" Body="&lt;p&gt;I have just started some single cell RNAseq analysis with 500 cells. I have used Seurat for analysis. Following the guideline I have generated few plots. Now I am trying to understand how to read those plots. Can the community provide their input in the following plots. To give a brief background of the experiment. This data was processed in MinION. Its from human brain sample from our collaborator. In their shortseq data they identified 15 clusters. However, both the libraries were processed independently. The main question I am trying to answer is celltype specific isoforms. but in my dataset I can find only 6 Cell clusters. So, I am trying to understand why I am getting such a small cluster? Is it because I stared with only 500 Cells? Also I want to know what these two PCA plot is indicating. Thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Lh9sn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Lh9sn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EqcAx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EqcAx.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/l8LFW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/l8LFW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4951" LastEditorUserId="48" LastEditDate="2019-09-10T07:59:46.983" LastActivityDate="2019-09-10T20:05:58.350" Title="Understanding Single Cell RNAseq Plots" Tags="|scrnaseq|seurat|figure-explanation|" AnswerCount="3" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9339" PostTypeId="1" AcceptedAnswerId="10364" CreationDate="2019-09-09T16:58:46.053" Score="0" ViewCount="290" Body="&lt;p&gt;I have a protein Seq. and I want to know&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;how to Know the longest ORF in this Protein?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;and if there is any tutorial for this process ?? &lt;/p&gt;&#xA;" OwnerUserId="4794" LastActivityDate="2019-10-11T03:02:25.277" Title="how to Know the longest ORF in Protein?" Tags="|proteins|orf|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9340" PostTypeId="2" ParentId="4131" CreationDate="2019-09-09T18:57:40.833" Score="1" Body="&lt;p&gt;In case an answer would still be relevant after a year and a half, below is a solution with the &lt;code&gt;Biostrings&lt;/code&gt; package.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Biostrings)&#xA;&#xA;# loading data.table for the fread() function.&#xA;# fread() is not only fast but is quite good at guessing how&#xA;# the file is formatted&#xA;library(data.table)&#xA;&#xA;# readDNAStringSet() function creates a &quot;DNAStringSet&quot; object&#xA;my_fasta &amp;lt;- readDNAStringSet(&quot;fasta.fa&quot;)&#xA;&#xA;&amp;gt; my_fasta&#xA;  A DNAStringSet instance of length 4&#xA;    width seq                                                                names               &#xA;[1]    61 TTTCACCGTAAATTAACTTTGAGAGGAGCTAATTCCTAAAAGAATTATACCGGCCTATTTG      rnd-5_family-5445...&#xA;[2]   485 AATCTGAGATTAGATTTATTTATCTGTTATCC...GGCCAATCATTCAAAAGTTATGAAAGTGATC rnd-5_family-2614...&#xA;[3]   215 TTTCTAGTTTGTATTTGAGGAAAGAAAATCCT...TAGAAATTGTAATTTACAAAATGAATTACTT rnd-5_family-2614...&#xA;[4]    87 ATGCTTAATTTGCTAAGCATTTAATAACCTTT...GTAGACATTAAAATAAACCTTATACTGCTAT rnd-4_family-798_...&#xA;&#xA;# the fasta header is stored at the @NAMES slot of the @ranges slot of the&#xA;# DNAStringSet object created above&#xA;&amp;gt; my_fasta@ranges@NAMES&#xA;[1] &quot;rnd-5_family-5445_Unspecified(-)&quot; &quot;rnd-5_family-2614_Unspecified(-)&quot;&#xA;[3] &quot;rnd-5_family-2614_Unspecified(-)&quot; &quot;rnd-4_family-798_Unspecified(-)&quot;&#xA;&#xA;# removing the trailing &quot;_Unpecified...&quot;&#xA;my_fasta@ranges@NAMES &amp;lt;- sub(&quot;_Unspecified.*&quot;,&#xA;                              &quot;&quot;,&#xA;                              my_fasta@ranges@NAMES)&#xA;&#xA;my_coord &amp;lt;- fread(&quot;repeat_masker.txt&quot;)&#xA;&amp;gt; my_coord&#xA;           V1   V2   V3                            V4 V5 V6&#xA;1: KB824701.1  417  478 rnd-5_family-5445_Unspecified  .  -&#xA;2: KB824701.1  587 1072 rnd-5_family-2614_Unspecified  .  -&#xA;3: KB824701.1  914 1129 rnd-5_family-2614_Unspecified  .  -&#xA;4: KB824701.1 1138 1225  rnd-4_family-798_Unspecified  .  -&#xA;#removing the &quot;&amp;gt;&quot; sign&#xA;&#xA;my_coord&lt;span class=&quot;math-container&quot;&gt;$V1 &amp;lt;- sub(&quot;&amp;gt;&quot;,&#xA;                   &quot;&quot;,&#xA;                   my_coord$&lt;/span&gt;V1)&#xA;&#xA;# appending first 3 columns of the coordinate file to &quot;NAMES&quot; as requested by the OP&#xA;my_fasta@ranges@NAMES &amp;lt;- paste(my_fasta@ranges@NAMES,&#xA;                               my_coord&lt;span class=&quot;math-container&quot;&gt;$V1,&#xA;                               my_coord$&lt;/span&gt;V2,&#xA;                               my_coord$V3,&#xA;                               sep = &quot;_&quot;)&#xA;&#xA;# writing the modified DNAStringSet object into a fasta file&#xA;writeXStringSet(my_fasta, &quot;edited_fasta.fa&quot;)&#xA;&#xA;&amp;gt;rnd-5_family-5445_KB824701.1_417_478&#xA;TTTCACCGTAAATTAACTTTGAGAGGAGCTAATTCCTAAAAGAATTATACCGGCCTATTTG&#xA;&amp;gt;rnd-5_family-2614_KB824701.1_587_1072&#xA;AATCTGAGATTAGATTTATTTATCTGTTATCCGCTCAGGTTGAAAAGTTTGCGCAATGTATAACTATAAAAATCTTTCTA&#xA;CCACTTTTGATTCATTTTTTAAATCTGGGGTCACATTTTATTCACGGTAGAAATTGTAATTTACAAAATGAATTACTTGA&#xA;AGGCAACACGAATCCAGAGTGATGCTTTACATAAATCTGCTTCTACCGATGCCAAAAATTGACGATATTCTATTATTTAA&#xA;TCTAAATGTTAGTCTTTACATACCCTCCCCTAATTGTTAGAATTTTATGAAATTTGATTTCAGGGGTCAGTTTAGCATGC&#xA;TAAATCTAATTCAATAGATTGATATTTTTCTTCAGGTAAAGAAAATTTTTGCGTCAAAGTAATCATATTCCTCCACGATT&#xA;GCATATAACTATGGTATATAATTTAAAAGATTACACTTTACGTAATGAAAAATCGGCCAATCATTCAAAAGTTATGAAAG&#xA;TGATC&#xA;&amp;gt;rnd-5_family-2614_KB824701.1_914_1129&#xA;TTTCTAGTTTGTATTTGAGGAAAGAAAATCCTACTATCACTTTTTAAAAAATTCACCAATCTGAGATTAGATTTATTTAT&#xA;CTGTTATCCGCTCAGGTTGAAAAGTTTGCGCAATGTATAACTATAAAAATCTTTCTACCACTTTTGATTCATTTTTTAAA&#xA;TCTGGGGTCACATTTTATTCACGGTAGAAATTGTAATTTACAAAATGAATTACTT&#xA;&amp;gt;rnd-4_family-798_KB824701.1_1138_1225&#xA;ATGCTTAATTTGCTAAGCATTTAATAACCTTTTTGGGTTTTGTGATAAAGGATGCTGTAGACATTAAAATAAACCTTATA&#xA;CTGCTAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-09-09T18:57:40.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9341" PostTypeId="2" ParentId="9309" CreationDate="2019-09-09T21:48:39.720" Score="1" Body="&lt;ul&gt;&#xA;&lt;li&gt;The first thing is the Sequence 5-2 and 5-3 are on very long branches. This is often indicative of sequence problems. You mentioned 5-2 and 5-3 are possibly two fragments of the same gene. My advice is to do a psuedo-annotation and reassess the phylogeny to see whether that super long branch length falls back in line&lt;/li&gt;&#xA;&lt;li&gt;The branches close to 5-1 are of interest and their genomes should be further investigated.&lt;/li&gt;&#xA;&lt;li&gt;You need to find the root of the phylogeny using external criteria. I can't do this because I've no idea was the species are.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Long branches are bad news in phylogeny, particularly if artefactual because of a phenomena known as 'long branch attraction'. Don't use your tree to understand duplication until this is resolved etc ... because the results can be distorted by LBA. Thus you must either delete them from the tree or perform a psuedo-annotation and redo the tree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was reluctant to forward the idea of phylogeny, because it is not just a bit of coding but an entire dialogue. Anyway by correcting the presumed misannotation you have sufficient results to move the investigation forward. You cant delineate your hypotheses until the root is correctly determined and then you set this as your outgroup under a more robust tree-building algorhithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think we are out of space on this thread (you're allowed 1 post, I'm allowed 2), but clearly things have moved forward. Again, please note once the long-branch is gone you can end up with a different tree.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-09-09T21:55:07.600" LastActivityDate="2019-09-09T21:55:07.600" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9342" PostTypeId="1" AcceptedAnswerId="9348" CreationDate="2019-09-09T22:30:32.867" Score="2" ViewCount="171" Body="&lt;p&gt;I encountered that many reads from single-cell RNA seq data were lost in the analysis because not assigned to any gene (genome: galgal6). I am trying to find an approach than could give me all the &quot;peaks&quot; in this data, that is to say regions where there is a high density of reads. Once I get there, I would extract &quot;peaks&quot; that are not assigned to any gene, and get the distance to the closest genes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am thinking of using first &lt;code&gt;bedtools genomecov&lt;/code&gt; to get the &quot;peaks&quot;, and then &lt;code&gt;bedtools intersect&lt;/code&gt; to get all reads assigned to a gene. Finally, I extract reads that are not in the output of &lt;code&gt;bedtools intersect&lt;/code&gt; to get a new file with all my reads from the peak, that are not assigned to any gene. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does it make sense this way ? Can you see any other way of doing ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are some tiny data: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; cat reads.bed&#xA;chr9    505479  505498&#xA;chr9    508014  508037&#xA;chr9    514603  514633&#xA;chr9    529519  529540&#xA;chr9    529519  529540&#xA;chr9    529519  529540&#xA;&#xA;&amp;gt; cat tiny.galGal6.chrom.sizes&#xA;chr9    24153086&#xA;&#xA;&amp;gt; bedtools genomecov -bg -split -i reads.bed -g tiny.galGal6.chrom.sizes&#xA;chr9    505479  505498  1&#xA;chr9    508014  508037  1&#xA;chr9    514603  514633  1&#xA;chr9    529519  529540  3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The last line would correspond to a peak. However, another difficulty is how should I define the thresholds ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be more than welcome.&lt;/p&gt;&#xA;" OwnerUserId="1531" LastEditorUserId="1531" LastEditDate="2019-09-09T22:37:57.517" LastActivityDate="2019-09-15T07:40:34.150" Title="How to best detect the &quot;peaks&quot; in RNA-seq data that are not assigned to any gene?" Tags="|rna-seq|scrnaseq|ngs|bedtools|coverage|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9343" PostTypeId="2" ParentId="9338" CreationDate="2019-09-10T06:27:29.547" Score="1" Body="&lt;p&gt;The first two plots are used in order to estimate the number of PCs to be used in later stages, for example clustering (this was split in to two functions in Seurat v3). The number of PCs selected would have an impact on the number of clusters obtained. The more PCs the more information for downstream applications. It would be a good idea to go over the genes within each PC, a trained eye can tell what kind of biology (or technical artefacts) contribute to the PCs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, &lt;code&gt;FindClusters()&lt;/code&gt; has a &lt;code&gt;resolution&lt;/code&gt; parameter, which enables the number of communities (clusters) that is generated. In Seurat v3, there is a wide selection of different clustering algorithms but recommending one over the other would be beyond my expertise.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a comparable dataset generated with some other technology/method, you can give a try with &quot;&lt;a href=&quot;https://satijalab.org/seurat/v3.1/integration.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;integrating&lt;/a&gt;&quot; the two datasets. An alternative non-Seurat approach that aims to do the same is &lt;code&gt;MNNCorrect&lt;/code&gt; from the &lt;code&gt;scran&lt;/code&gt; package.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-09-10T06:27:29.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9344" PostTypeId="2" ParentId="6871" CreationDate="2019-09-10T09:13:17.827" Score="1" Body="&lt;p&gt;I am using DMPfold to predict protein structure of a 382 residue protein sequence. All goes wel, and even the files are produced but during the &quot;seq2maps.csh&quot;, I get this error in between:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Running PSIPRED &amp;amp; SOLVPRED&#xA;Running PSI-BLAST with sequence c600m2-tail-before-lysin-sequence.temp.fasta ...&#xA;Segmentation fault (core dumped)&#xA;[blastpgp] WARNING: Unable to open BLOSUM62&#xA;[blastpgp] WARNING: BlastScoreBlkMatFill returned non-zero status&#xA;[blastpgp] WARNING: SetUpBlastSearch failed.&#xA;BLASTP 2.2.17 [Aug-26-2007]&#xA;&#xA;&#xA;Reference: Altschul, Stephen F., Thomas L. Madden, Alejandro A. Schaffer, &#xA;Jinghui Zhang, Zheng Zhang, Webb Miller, and David J. Lipman (1997), &#xA;&quot;Gapped BLAST and PSI-BLAST: a new generation of protein database search&#xA;programs&quot;,  Nucleic Acids Res. 25:3389-3402.&#xA;&#xA;&#xA;Reference for composition-based statistics:&#xA;Schaffer, Alejandro A., L. Aravind, Thomas L. Madden,&#xA;Sergei Shavirin, John L. Spouge, Yuri I. Wolf,  &#xA;Eugene V. Koonin, and Stephen F. Altschul (2001), &#xA;&quot;Improving the accuracy of PSI-BLAST protein database searches with &#xA;composition-based statistics and other refinements&quot;,  Nucleic Acids Res. 29:2994-3005.&#xA;&#xA;Query= C600M2_00100 tail protein [Salmonella phage BPS17S6]&#xA;         (377 letters)&#xA;&#xA;Database: c600m2-tail-before-lysin-sequence.a3m &#xA;           46,978 sequences; 11,323,713 total letters&#xA;&#xA;SearchingFATAL: Error whilst running blastpgp - script terminated!&#xA;Running PSICOV&#xA;Running FreeContact&#xA;Running CCMpred&#xA; _____ _____ _____               _ &#xA;|     |     |     |___ ___ ___ _| |&#xA;|   --|   --| | | | . |  _| -_| . |&#xA;|_____|_____|_|_|_|  _|_| |___|___|&#xA;                  |_|              &#xA;&#xA;using CPU (10 thread(s))&#xA;&#xA;Reweighted 30404 sequences with threshold 0.8 to Beff=22917.9 weight mean=0.753779, min=0.0232558, max=1&#xA;&#xA;Will optimize 62686429 32-bit variables&#xA;&#xA;iter    eval    f(x)        ║x║         ║g║         step&#xA;1       1       1.50338e+07 152357      3.021629e+11    1.55e-06&#xA;2       1       1.47369e+07 152356      2.1447226e+11   1.18e-06&#xA;3       1       1.44292e+07 152355      1.4646483e+11   1.22e-06&#xA;4       1       1.41268e+07 152353      1.0388224e+11   1.5e-06&#xA;5       1       1.38295e+07 152358      7.337667e+10    1.92e-06&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can you please let me know if I need to rerun seq2maps.csh because of this error? And, how this error can be solved? Thank you!&lt;/p&gt;&#xA;" OwnerUserId="5180" LastActivityDate="2019-09-10T09:13:17.827" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9345" PostTypeId="1" CreationDate="2019-09-10T09:31:06.030" Score="1" ViewCount="235" Body="&lt;p&gt;I am using DMPfold to predict protein structure of a 382 residue protein sequence. All goes wel, and even the files are produced but during the &quot;seq2maps.csh&quot;, I get this error in between:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Running PSIPRED &amp;amp; SOLVPRED&#xA;Running PSI-BLAST with sequence c600m2-tail-before-lysin-sequence.temp.fasta ...&#xA;Segmentation fault (core dumped)&#xA;[blastpgp] WARNING: Unable to open BLOSUM62&#xA;[blastpgp] WARNING: BlastScoreBlkMatFill returned non-zero status&#xA;[blastpgp] WARNING: SetUpBlastSearch failed.&#xA;BLASTP 2.2.17 [Aug-26-2007]&#xA;&#xA;&#xA;Reference: Altschul, Stephen F., Thomas L. Madden, Alejandro A. Schaffer, &#xA;Jinghui Zhang, Zheng Zhang, Webb Miller, and David J. Lipman (1997), &#xA;&quot;Gapped BLAST and PSI-BLAST: a new generation of protein database search&#xA;programs&quot;,  Nucleic Acids Res. 25:3389-3402.&#xA;&#xA;&#xA;Reference for composition-based statistics:&#xA;Schaffer, Alejandro A., L. Aravind, Thomas L. Madden,&#xA;Sergei Shavirin, John L. Spouge, Yuri I. Wolf,  &#xA;Eugene V. Koonin, and Stephen F. Altschul (2001), &#xA;&quot;Improving the accuracy of PSI-BLAST protein database searches with &#xA;composition-based statistics and other refinements&quot;,  Nucleic Acids Res. 29:2994-3005.&#xA;&#xA;Query= C600M2_00100 tail protein [Salmonella phage BPS17S6]&#xA;         (377 letters)&#xA;&#xA;Database: c600m2-tail-before-lysin-sequence.a3m &#xA;           46,978 sequences; 11,323,713 total letters&#xA;&#xA;SearchingFATAL: Error whilst running blastpgp - script terminated!&#xA;Running PSICOV&#xA;Running FreeContact&#xA;Running CCMpred&#xA; _____ _____ _____               _ &#xA;|     |     |     |___ ___ ___ _| |&#xA;|   --|   --| | | | . |  _| -_| . |&#xA;|_____|_____|_|_|_|  _|_| |___|___|&#xA;                  |_|              &#xA;&#xA;using CPU (10 thread(s))&#xA;&#xA;Reweighted 30404 sequences with threshold 0.8 to Beff=22917.9 weight mean=0.753779, min=0.0232558, max=1&#xA;&#xA;Will optimize 62686429 32-bit variables&#xA;&#xA;iter    eval    f(x)        ║x║         ║g║         step&#xA;1       1       1.50338e+07 152357      3.021629e+11    1.55e-06&#xA;2       1       1.47369e+07 152356      2.1447226e+11   1.18e-06&#xA;3       1       1.44292e+07 152355      1.4646483e+11   1.22e-06&#xA;4       1       1.41268e+07 152353      1.0388224e+11   1.5e-06&#xA;5       1       1.38295e+07 152358      7.337667e+10    1.92e-06&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can you please let me know if I need to rerun seq2maps.csh because of this error? And, how this error can be solved? Thank you!&lt;/p&gt;&#xA;" OwnerUserId="5180" LastActivityDate="2019-09-10T12:52:27.843" Title="DMPfold error while running protein structure prediction" Tags="|protein-structure|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9346" PostTypeId="2" ParentId="9334" CreationDate="2019-09-10T09:45:45.123" Score="3" Body="&lt;ol&gt;&#xA;&lt;li&gt;If the environment already exists it will simply be used. You cannot, however, tell it to use an environment with a normal name (e.g., &quot;my_env&quot;).&lt;/li&gt;&#xA;&lt;li&gt;I assume you mean, &quot;where the heck does the random looking conda environment name come from?!?&quot;, which is a very good question. The answer to that is that it's a hash of the &lt;code&gt;--conda-prefix&lt;/code&gt; setting and the contents of the environment yaml file you specify. The actual code for doing this (I'm &quot;stealing&quot; it &lt;a href=&quot;https://github.com/maxplanck-ie/snakepipes/blob/master/bin/snakePipes#L174&quot; rel=&quot;nofollow noreferrer&quot;&gt;from snakePipes&lt;/a&gt;) is:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import hashlib&#xA;md5hash = hashlib.md5()&#xA;md5hash.update(&quot;what you gave to --conda-prefix&quot;.encode())&#xA;f = open(&quot;your environment yaml file&quot;, 'rb')&#xA;md5hash.update(f.read())&#xA;f.close()&#xA;h = md5hash.hexdigest()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;h&lt;/code&gt; is then the directory name. There are actually two lengths of those, the shorter 8 character version that snakeMake uses by default or the full length version that the code above will produce. snakeMake will actually use either of these. If you're wondering, no I don't think any of this is documented anywhere, you have to dig through the snakeMake code to find it.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-09-10T09:45:45.123" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="9347" PostTypeId="2" ParentId="9335" CreationDate="2019-09-10T09:56:57.483" Score="1" Body="&lt;p&gt;You would want to instead use the &lt;code&gt;-a&lt;/code&gt; option with &lt;code&gt;bwa mem&lt;/code&gt; to output all alignments. You would be best off filtering these with a custom script to only keep those with a close enough score to the primary hit. &lt;code&gt;samtools depth&lt;/code&gt; can't be told to include secondary alignments (&lt;a href=&quot;https://github.com/samtools/samtools/blob/ffdddb6927827d805c603993cbc07a6faa9a63a6/bam2depth.c#L65&quot; rel=&quot;nofollow noreferrer&quot;&gt;it's hard-coded&lt;/a&gt;), so you'll need to use something else like &lt;code&gt;bamCoverage&lt;/code&gt; or &lt;code&gt;samtools mpileup&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-09-10T09:56:57.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9348" PostTypeId="2" ParentId="9342" CreationDate="2019-09-10T10:09:49.733" Score="5" Body="&lt;p&gt;Basically what you're discovering is that there are unannotated expressed features, so your task isn't really finding peaks, but rather finding novel expressed transcripts. For that, you can use &lt;a href=&quot;https://ccb.jhu.edu/software/stringtie/&quot; rel=&quot;noreferrer&quot;&gt;stringTie&lt;/a&gt; or similar programs. Ensure that the BAM file you give to stringTie have all of the cells together.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-09-10T10:09:49.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9349" PostTypeId="2" ParentId="9345" CreationDate="2019-09-10T11:37:43.657" Score="1" Body="&lt;p&gt;The comment by Devon Ryan is correct - looks like your &lt;code&gt;blastpgp&lt;/code&gt; command is not working properly. For a protein of that length it's more likely to be a problem with the BLAST executable running on your system than a memory issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try running the &lt;code&gt;blastpgp&lt;/code&gt; command (&lt;a href=&quot;https://github.com/psipred/DMPfold/blob/master/bin/runpsipredandsolvwithdb#L26&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/psipred/DMPfold/blob/master/bin/runpsipredandsolvwithdb#L26&lt;/a&gt;) on its own and see what happens:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;~/path/to/blastpgp -a 12 -b 0 -j 3 -h 0.001 -d yourfile.a3m -i yourfile.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2019-09-10T11:37:43.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9350" PostTypeId="2" ParentId="9345" CreationDate="2019-09-10T12:37:33.550" Score="0" Body="&lt;p&gt;Thanks, I did this but got this erro:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blastpgp -a 12 -b 0 -j 3 -h 0.001 -d c600m2-tail-before-lysin-sequence.a3m -i c600m2-tail-before-lysin-sequence.temp.fasta&#xA;[blastpgp] WARNING: Unable to open BLOSUM62&#xA;[blastpgp] WARNING: BlastScoreBlkMatFill returned non-zero status&#xA;[blastpgp] WARNING: SetUpBlastSearch failed.&#xA;BLASTP 2.2.26 [Sep-21-2011]&#xA;&#xA;&#xA;Reference: Altschul, Stephen F., Thomas L. Madden, Alejandro A. Schaffer, &#xA;Jinghui Zhang, Zheng Zhang, Webb Miller, and David J. Lipman (1997), &#xA;&quot;Gapped BLAST and PSI-BLAST: a new generation of protein database search&#xA;programs&quot;,  Nucleic Acids Res. 25:3389-3402.&#xA;&#xA;&#xA;Reference for compositional score matrix adjustment: Altschul, Stephen F., &#xA;John C. Wootton, E. Michael Gertz, Richa Agarwala, Aleksandr Morgulis,&#xA;Alejandro A. Schaffer, and Yi-Kuo Yu (2005) &quot;Protein database searches&#xA;using compositionally adjusted substitution matrices&quot;, FEBS J. 272:5101-5109.&#xA;&#xA;&#xA;Reference for composition-based statistics starting in round 2:&#xA;Schaffer, Alejandro A., L. Aravind, Thomas L. Madden,&#xA;Sergei Shavirin, John L. Spouge, Yuri I. Wolf,  &#xA;Eugene V. Koonin, and Stephen F. Altschul (2001), &#xA;&quot;Improving the accuracy of PSI-BLAST protein database searches with &#xA;composition-based statistics and other refinements&quot;,  Nucleic Acids Res. 29:2994-3005.&#xA;&#xA;Query= C600M2_00100 tail protein [Salmonella phage BPS17S6]&#xA;         (377 letters)&#xA;&#xA;Database: c600m2-tail-before-lysin-sequence.a3m &#xA;           46,978 sequences; 11,323,713 total letters&#xA;&#xA;SearchingSegmentation fault (core dumped)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Could you please let me know how this could be solved? Thanks&lt;/p&gt;&#xA;" OwnerUserId="5180" LastActivityDate="2019-09-10T12:37:33.550" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9352" PostTypeId="1" AcceptedAnswerId="9353" CreationDate="2019-09-10T14:43:36.547" Score="3" ViewCount="780" Body="&lt;p&gt;I have this fastq data from GEO:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zcat SRR1658526.fastq.gz | head -n 20&#xA;@SRR1658526.1 HWI-ST398:296:C1MP4ACXX:1:1101:1093:2094 length=102&#xA;GATCTCTATTACTTTTTGAAGGATTNNNNNNNNNNAANTTTTGAATCANNNNNNNNNNNNNNNNNNNNNNNNNNNNNTNNNNNNNNNNNNNNNNNNNNNNNN&#xA;+SRR1658526.1 HWI-ST398:296:C1MP4ACXX:1:1101:1093:2094 length=102&#xA;@&amp;lt;@FFDFDHHFHHIIIIGHBGGGIG##########10#0:BGDDHHII######################################################&#xA;@SRR1658526.2 HWI-ST398:296:C1MP4ACXX:1:1101:1167:2107 length=102&#xA;ATAATATTGTAGATATAAATGTTATCTAATCTTATCTGATCAGCTTGCTNNATANNNNNNNNNNNNNNNNNNACNTATGNNNNNNNNNNNNNNNNNNNNNCC&#xA;+SRR1658526.2 HWI-ST398:296:C1MP4ACXX:1:1101:1167:2107 length=102&#xA;CCCFFFFFHHFHHJJIIIIJIJJHJHJJJJIJJIJJJJIJHIJJJJGII#####################################################&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is supposed to be paired-end sequences. Is the prefix &lt;code&gt;@&lt;/code&gt; and &lt;code&gt;+&lt;/code&gt; are the R1 and R2? What's the convention here?&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2019-09-10T14:49:27.863" LastActivityDate="2019-09-12T15:58:08.807" Title="Does this FASTQ data contain single or paired end calls?" Tags="|fastq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9353" PostTypeId="2" ParentId="9352" CreationDate="2019-09-10T14:49:17.100" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/FASTQ_format&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entries in a fastq file&lt;/a&gt; occupy 4 lines each and R1 and R2 are typically in different files. Since that &lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR1658525&quot; rel=&quot;nofollow noreferrer&quot;&gt;SRA project is 2x51&lt;/a&gt; you seem to have run &lt;code&gt;fastq-dump&lt;/code&gt; without the &lt;code&gt;--split-3&lt;/code&gt; option, so both R1 and R2 are merged together. Make your life easier and never use SRA, but instead just download the individual files from ENA.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-09-10T14:49:17.100" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="9354" PostTypeId="1" AcceptedAnswerId="9358" CreationDate="2019-09-10T15:37:27.390" Score="2" ViewCount="336" Body="&lt;p&gt;I am using following command from MiXCR to both align, assemble and export my input files:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mixcr analyze amplicon \&#xA;    -s hsa \&#xA;    --starting-material rna \&#xA;    --5-end no-v-primers \&#xA;    --3-end c-primers \&#xA;    --adapters adapters-present \&#xA;    --export &quot;-aaFeature CDR1 -nFeature CDR1 -aaFeature CDR2 \&#xA;              -nFeature CDR2 -aaFeature CDR3 -nFeature CDR3 \&#xA;              -aaFeature VGene -cGenes -count&quot; \&#xA;    --assemble &quot;-OassemblingFeatures=[CDR1,CDR2,CDR3]&quot; \&#xA;    INPUT_FILE.fasta analysis_name&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, this creates multiple output file (analysis_name.clonotypes.ALL.txt, analysis_name.clonotypes.IGH.txt, etc.), which stands in contrast to the output of the &lt;code&gt;mixcr exportAlignments&lt;/code&gt; command.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I export a single txt file instead, while still using the &lt;code&gt;mixcr analyze&lt;/code&gt; command?&lt;/p&gt;&#xA;" OwnerUserId="5186" LastEditorUserId="5186" LastEditDate="2019-09-11T11:21:22.837" LastActivityDate="2023-01-24T15:15:05.500" Title="MiXCR: only create a single export file for all clonotypes" Tags="|sequence-alignment|fasta|bash|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9355" PostTypeId="2" ParentId="9352" CreationDate="2019-09-10T17:00:29.250" Score="2" Body="&lt;p&gt;The data shown in the fastq files are paired end data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fastQ files @ and + do not correspond to R1 and R2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/FASTQ_format&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia page&lt;/a&gt; for fastq files describe the following conventions:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A FASTQ file normally uses four lines per sequence.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Line 1 begins with a '@' character and is followed by a sequence identifier and an optional description (like a FASTA title line).&#xA;Line 2 is the raw sequence letters.&#xA;Line 3 begins with a '+' character and is optionally followed by the same sequence identifier (and any description) again.&#xA;Line 4 encodes the quality values for the sequence in Line 2, and must contain the same number of symbols as letters in the sequence.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;When using fastq-dump&#xA;&quot;.1&quot; and &quot;.2&quot; stand for read suffices assuming u used the -I flag.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case you can see that R1 (&quot;.1&quot;) and R2 (&quot;.2&quot;) are within the same file. &lt;/p&gt;&#xA;" OwnerUserId="2675" LastEditorUserId="2675" LastEditDate="2019-09-12T15:58:08.807" LastActivityDate="2019-09-12T15:58:08.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9356" PostTypeId="2" ParentId="9338" CreationDate="2019-09-10T17:10:05.350" Score="3" Body="&lt;p&gt;Identifying 15 clusters from 500 cells is challenging. It will depend on the relative abundance of each cell type within the tissue. You can use the cell type frequencies found in the shortSeq data set to see what to expect from a 500 cell dataset using the &lt;a href=&quot;https://satijalab.org/howmanycells&quot; rel=&quot;nofollow noreferrer&quot;&gt;howmanycells tool&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;About the pc plots: &#xA;The first is an elbowplot which explains how many variation is explained by each PC. The knee basically defines when components become less informative. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: I noticed that the y-axis of the elbowplot starts at 4 suggesting that the dimensionality of your data is most likely ~10 dimensions. But I expect many of the clusters that don't show up on your Umap plot to be present at very low frequencies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second plot is a jackstraw plot and provides a visualization of a statistical test that calculates the significance of your principal components. This one shows that all your components are significant. You can combine this with the elbowplot to estimate the dimensionality (10-15 dimensions) of your data and decide which components to use for downstream analysis. In general the more they dropoff to the dotted line the less significant they become. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While these plots are useful as a first impression, always rely on biological information and common sense to decide what information (here: components) to include in downstream analysis such as Umap, clustering, etc. For example: usually you don't want to include components mainly defined by ribosomal, mitochondrial or hsp genes. &lt;/p&gt;&#xA;" OwnerUserId="2675" LastEditorUserId="2675" LastEditDate="2019-09-10T20:05:58.350" LastActivityDate="2019-09-10T20:05:58.350" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="9357" PostTypeId="1" CreationDate="2019-09-10T18:45:03.130" Score="0" ViewCount="317" Body="&lt;p&gt;What is the difference between these two databases?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/blast/db/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ncbi.nlm.nih.gov/blast/db/&lt;/a&gt;&#xA; and&#xA;&lt;a href=&quot;ftp://ftp.ncbi.nih.gov/genbank/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ncbi.nih.gov/genbank/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that BLAST database has both nt for nucleotide and np for protein. But genbank is also for genomic database. So what is the difference between nt and Genbank databases?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="5188" LastEditorUserId="5188" LastEditDate="2019-09-10T18:57:28.900" LastActivityDate="2019-09-10T20:47:22.110" Title="blast nt vs Genbank" Tags="|blast|blastn|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9358" PostTypeId="2" ParentId="9354" CreationDate="2019-09-10T19:20:41.987" Score="3" Body="&lt;p&gt;This is from the &lt;a href=&quot;https://buildmedia.readthedocs.org/media/pdf/mixcr/develop/mixcr.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;MiXCR documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Under the hood analyze amplicon is equivalent to the execution of the following&#xA;MiXCR actions options:&#xA;&amp;gt; mixcr align -s hs -p kAligner2 input_R1.fastq input_R2.fastq alignments.vdjca&#xA;... Building alignments&#xA;&amp;gt; mixcr assemble alignments.vdjca clones.clns&#xA;... Assembling clones&#xA;&amp;gt; mixcr exportClones --chains IGH clones.clns clones.txt&#xA;... Exporting clones to tab-delimited file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So when using &lt;code&gt;analyze amplicon&lt;/code&gt;, you are using &lt;code&gt;mixcr exportClones&lt;/code&gt; under the hood and not &lt;code&gt;exportAlignments&lt;/code&gt;, hence the difference in output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe what you are experiencing results from a bug, please see this &lt;a href=&quot;https://github.com/milaboratory/mixcr/issues/454&quot; rel=&quot;nofollow noreferrer&quot;&gt;issue&lt;/a&gt;. If this is the case, all these files should be the same according to the bug report. Fortunately the developers have already addressed the bug in version 3.0.3.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-09-10T19:32:02.567" LastActivityDate="2019-09-10T19:32:02.567" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="9359" PostTypeId="2" ParentId="9338" CreationDate="2019-09-10T19:29:59.400" Score="2" Body="&lt;p&gt;The first two plots are described on the &lt;a href=&quot;https://satijalab.org/seurat/v3.1/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seurat PBMC tutorial&lt;/a&gt;. Specifically:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;‘Elbow plot’: a ranking of principle components based on the&#xA;  percentage of variance explained by each one (ElbowPlot function).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Based on the tutorial, you should use at least 10 PCs.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The JackStrawPlot function provides a visualization tool for comparing&#xA;  the distribution of p-values for each PC with a uniform distribution&#xA;  (dashed line). ‘Significant’ PCs will show a strong enrichment of&#xA;  features with low p-values (solid curve above the dashed line).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Based on the tutorial, you should use more than 15 PCs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No one can tell you the right number of clusters. If you already identified 15 clusters in a different dataset, then you should have cluster markers for those genes. You can check those genes in your dataset.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2019-09-10T19:29:59.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9360" PostTypeId="1" AcceptedAnswerId="10545" CreationDate="2019-09-10T20:08:27.867" Score="1" ViewCount="268" Body="&lt;p&gt;I have a list of around 500 protein targets (human). I only have their protein symbols. As an example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EGFR&#xA;AR&#xA;FOXA1&#xA;CXCR3&#xA;...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to get structural data (atomic coordinate) from PDB (Protein Data Bank) at once, either programmatically (Python or R) or with a web api. Do you know a clear step by step tutorial for this task or code example?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that is not available, at least I need to find the PDB IDs of my list of protein symbols at once.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks&lt;/p&gt;&#xA;" OwnerUserId="2536" LastEditorUserId="2536" LastEditDate="2019-10-08T16:47:48.093" LastActivityDate="2019-10-08T16:47:48.093" Title="I have a list of protein targets. How to get structural data (atomic coordinate) from PDB?" Tags="|bioconductor|python|protein-structure|pdb|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="9361" PostTypeId="2" ParentId="9357" CreationDate="2019-09-10T20:47:22.110" Score="2" Body="&lt;p&gt;I think the BLAST folder contains pre-formatted databases for BLAST whereas the GenBank folder contains sequences.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2019-09-10T20:47:22.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9363" PostTypeId="2" ParentId="783" CreationDate="2019-09-10T21:28:12.907" Score="0" Body="&lt;p&gt;Distance between C-alphas can also be found using the &lt;a href=&quot;https://github.com/BioJulia/BioStructures.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioStructures.jl&lt;/a&gt; package in Julia:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using BioStructures&#xA;struc = read(&quot;1AKE.pdb&quot;, PDB)&#xA;calphas = collectatoms(struc['A'], calphaselector)&#xA;dm = DistanceMap(calphas)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then &lt;code&gt;dm[5, 10]&lt;/code&gt; gives &lt;code&gt;12.19677604943208&lt;/code&gt; etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other variations also work, see the &lt;a href=&quot;https://biojulia.net/BioStructures.jl/stable/documentation.html#Spatial-calculations-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;docs&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2019-09-10T21:28:12.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9364" PostTypeId="2" ParentId="5507" CreationDate="2019-09-10T23:09:25.383" Score="-2" Body="&lt;p&gt;You can give a try to this tool: &lt;a href=&quot;https://www.dnaservic.es/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dnaservic.es&lt;/a&gt;&#xA;It's actually available experimentaly. It is a real local aligner, not just a search tool like BLAST. It really returns an alignment. It is usually slower than BLAST, but bot much slower in most cases. In contrast it usually returns alignments with better scores and not only partial alignments.&lt;/p&gt;&#xA;" OwnerUserId="5190" LastActivityDate="2019-09-10T23:09:25.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="9365" PostTypeId="1" CreationDate="2019-09-11T00:55:16.717" Score="0" ViewCount="142" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/398122/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I used macs2 to call peaks for atac-seq data. now my goal is to split the peaks into 50 bp windows with 25 bp steps and then calculate the Tn5 integration frequency in each window.&lt;/p&gt;&#xA;&lt;p&gt;How should I proceed with that?&lt;/p&gt;&#xA;" OwnerUserId="5191" LastEditorUserId="73" LastEditDate="2021-01-15T03:22:58.923" LastActivityDate="2021-06-14T06:06:10.470" Title="ATAC-seq macs2 peak splitting in sliding windows" Tags="|atac-seq|macs2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10364" PostTypeId="2" ParentId="9339" CreationDate="2019-09-11T02:04:28.250" Score="2" Body="&lt;p&gt;Proteins don't have ORFs, DNA sequences do.  What DNA sequences do you have?  If cDNA (i.e. transcript/mRNA) then an ORF finder will give multiple ORFs and the coordinates on the full-length input DNA which indicate it's length.&lt;/p&gt;&#xA;" OwnerUserId="2436" LastActivityDate="2019-09-11T02:04:28.250" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10365" PostTypeId="1" AcceptedAnswerId="10381" CreationDate="2019-09-11T03:26:07.423" Score="1" ViewCount="102" Body="&lt;p&gt;This is the second part of this thread:  &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/9309/duplicated-genes-and-genome-fragmentation?noredirect=1#comment12855_9309&quot;&gt;Duplicated genes and genome fragmentation&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As @Michael G. suggested, I did a pseudo-annotation of sequences 5-2 and 5-3 (i.e., I joined both sequences into one) and I did the phylogenetic analysis again. Here is the tree that I obtained: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rXaLi.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rXaLi.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both of my sequences are shown in a red rectangle. The length of the branch seems to have decreased a bit. Also, I looked at the branches near sequence 5-1. One of them is the original query sequence from species &lt;strong&gt;&lt;em&gt;A&lt;/em&gt;&lt;/strong&gt; and the other one is a hypothetical protein from the same species (probably related to the query sequence, but I couldn't find any domains because it is only a few amino acids long). I still need to find the root of the phylogeny but I am not sure how to do this... &lt;/p&gt;&#xA;" OwnerUserId="5116" LastEditorUserId="5116" LastEditDate="2019-09-12T00:37:01.747" LastActivityDate="2019-09-12T12:50:25.510" Title="How to make a robust amino acid phylogeny with adequate robustness support" Tags="|phylogenetics|phylogeny|duplications|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="10366" PostTypeId="1" AcceptedAnswerId="10368" CreationDate="2019-09-11T09:10:16.837" Score="0" ViewCount="54" Body="&lt;p&gt;I am new to R and biocondunctor. I have the normalized expression values for 20 samples for a disease and for 10 controls. I wanted to get the differential expressed values for each sample with all 10 controls.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How could I do that with limma. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help will be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: What I want to achieve is single-sample t-test between the control population and the single sample values to get DE values&lt;/p&gt;&#xA;" OwnerUserId="6192" LastEditorUserId="6192" LastEditDate="2019-09-11T09:42:45.977" LastActivityDate="2019-09-11T12:02:10.757" Title="How to perform DE analysis for each sample" Tags="|differential-expression|microarray|limma|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="10367" PostTypeId="2" ParentId="9354" CreationDate="2019-09-11T11:25:15.443" Score="2" Body="&lt;p&gt;After finding the correct entry in the MiXCR documentation (thanks to haci), I learned that the parameter &lt;code&gt;--receptor-type&lt;/code&gt; is responsible for which sequences will be returned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The default for this parameter when using mixcr analyze is &lt;code&gt;--receptor-type xcr&lt;/code&gt;, and stands for 'all chains', thus returning such a vast output. Changing this parameter to e.g. BCR solved my issue.&lt;/p&gt;&#xA;" OwnerUserId="5186" LastActivityDate="2019-09-11T11:25:15.443" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10368" PostTypeId="2" ParentId="10366" CreationDate="2019-09-11T12:02:10.757" Score="1" Body="&lt;p&gt;Treat your disease samples as individual groups and then follow the normal routine in limma to use contrasts to compare two group (the control group versus individual disease samples). Note that you need to think long and hard about what these results then mean. Normally we compare groups because then the results should generalize to other samples. That will not be the case when you look at individual disease samples, so I suggest you do some clustering of the samples to see if you perhaps have any coherent clustering of the disease samples into subsets that could then be used as more useful groups in DE testing.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-09-11T12:02:10.757" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10369" PostTypeId="2" ParentId="9271" CreationDate="2019-09-11T12:09:42.090" Score="-1" Body="&lt;p&gt;I don't have enough reputation to comment but if a sequence is duplicated then there has obviously been a duplication event. Doesn't matter how scientists call it now.It can be older or more recent but it's yet a duplication.&lt;/p&gt;&#xA;" OwnerUserId="5190" LastActivityDate="2019-09-11T12:09:42.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10371" PostTypeId="1" CreationDate="2019-09-11T12:14:08.713" Score="2" ViewCount="101" Body="&lt;p&gt;I am running DMPfold, &quot;seq2maps.sh&quot; ran successfully, but whrn running &quot;run_dmpfold.sh&quot;, it gives me this error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   Note: The following floating-point exceptions are signalling: IEEE_UNDERFLOW_FLAG IEEE_DENORMAL&#xA;Note: The following floating-point exceptions are signalling: IEEE_UNDERFLOW_FLAG IEEE_DENORMAL&#xA;seed =  30603&#xA;Note: The following floating-point exceptions are signalling: IEEE_UNDERFLOW_FLAG IEEE_DENORMAL&#xA;/media/Deepak/NPRP10/softwares/dmpfold/DMPfold-master/run_dmpfold.sh: line 177: c600m2-tail-before-lysin-sequence_[1-9]*.pdb: No such file or directory&#xA;No atoms read from PDB file&#xA;0 models read from PDB file&#xA;Outliers:&#xA;&#xA;&#xA;Traceback (most recent call last):&#xA;  File &quot;/media/Deepak/NPRP10/softwares/dmpfold/DMPfold-master/nn/dmp-softmax/pytorch_dmp_iterdistpred_cb.py&quot;, line 82, in &amp;lt;module&amp;gt;&#xA;    main()&#xA;  File &quot;/media/Deepak/NPRP10/softwares/dmpfold/DMPfold-master/nn/dmp-softmax/pytorch_dmp_iterdistpred_cb.py&quot;, line 55, in main&#xA;    with open(sys.argv[3], 'r') as refpdbfile:&#xA;FileNotFoundError: [Errno 2] No such file or directory: 'best_qdope.pdb'&#xA;Traceback (most recent call last):&#xA;  File &quot;/media/Deepak/NPRP10/softwares/dmpfold/DMPfold-master/nn/dmp-hb/pytorch_iterdmp_hbpred.py&quot;, line 75, in &amp;lt;module&amp;gt;&#xA;    main()&#xA;  File &quot;/media/Deepak/NPRP10/softwares/dmpfold/DMPfold-master/nn/dmp-hb/pytorch_iterdmp_hbpred.py&quot;, line 53, in main&#xA;    with open(sys.argv[3], 'r') as refpdbfile:&#xA;FileNotFoundError: [Errno 2] No such file or directory: 'best_qdope.pdb'&#xA;No valid current contacts.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can you please let me know how this can be solved? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;the comand line is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;run_dmpfold.sh sequence.fasta sequence.21c sequence.map ./dmpfold-lysin 10 500 &amp;amp;&amp;gt; dmpfold-log.log&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="5180" LastEditorUserId="5180" LastEditDate="2019-09-11T12:26:03.273" LastActivityDate="2020-06-08T12:06:55.710" Title="Dmpfold error &quot;No such file or directory: 'best_qdope.pdb'&quot; when running &quot;run_dmpfold.sh&quot;?" Tags="|protein-structure|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10373" PostTypeId="2" ParentId="10371" CreationDate="2019-09-11T22:14:48.747" Score="1" Body="&lt;p&gt;The &lt;code&gt;target&lt;/code&gt; variable, which is set to the first part of the sequence file name, should be &lt;code&gt;sequence&lt;/code&gt; according to your given command line. However the error message thinks it is &lt;code&gt;c600m2-tail-before-lysin-sequence&lt;/code&gt;. Have you edited the command line for simplicity?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a little worried about the floating point errors, which are almost certainly from CNS as there are 3 CNS calls and the error shows up 3 times. I would test your CNS installation to see if everything is okay - maybe try some examples from &lt;a href=&quot;https://www.mrc-lmb.cam.ac.uk/public/xtal/doc/cns/cns_1.3/getting_started/frame.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;their website&lt;/a&gt;. An error there would explain why no output is generated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can ignore any errors below line 5 of the above program output for now, as they are follow-on errors. In particular the question title isn't the 'real' error here, the real error is that no PDB files are generated on &lt;a href=&quot;https://github.com/psipred/DMPfold/blob/master/run_dmpfold.sh#L160&quot; rel=&quot;nofollow noreferrer&quot;&gt;line 160&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, I wouldn't use &lt;code&gt;10 500&lt;/code&gt; as parameters for iteration number and number of models. It will take ages to run. I'd just use the defaults (omit the arguments), which we find gives good results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, I'll point out that the &lt;a href=&quot;https://github.com/psipred/DMPfold&quot; rel=&quot;nofollow noreferrer&quot;&gt;DMPfold repository&lt;/a&gt; has an issue tracker where you can raise any future issues. Though we may as well finish debugging this one here since we've started!&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2019-09-11T22:14:48.747" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10374" PostTypeId="1" CreationDate="2019-09-11T23:41:28.503" Score="2" ViewCount="287" Body="&lt;p&gt;So this is related to CRISPR-CAS9. I am working with off-target predictions for my thesis and was looking at all scientific papers related to CRISPR. I found one and decided to use their datasets. The datasets were not available directly and the method used to obtain them was specified which I had mentioned below:&lt;/p&gt;&#xA;&lt;p&gt;They have around 30 sgRNAs which are each 23bp long and are aligning the sgRNAs to human reference genome with a tolerance of upto 6 mismatches using Bowtie2.&lt;/p&gt;&#xA;&lt;p&gt;In order to access the exact dataset, I thought I need to replicate the method they used. To do so, I collected the sgRNAs from their paper and tried to use Bowtie2 to align them against human genome with upto 6 mismatches.&lt;/p&gt;&#xA;&lt;p&gt;Since my laptop does not have the capacity to perform this operation (it has 8GB RAM), I am searching for interfaces that provide access to Bowtie2. I found &lt;a href=&quot;https://usegalaxy.eu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Galaxy&lt;/a&gt; to be useful. Does anyone know of any other tool that provides an option for Bowtie2?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="6198" LastEditorUserId="73" LastEditDate="2021-07-04T14:34:25.403" LastActivityDate="2024-03-20T19:04:15.513" Title="Read alignment using Bowtie2" Tags="|sequence-alignment|bowtie2|crispr|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10375" PostTypeId="1" CreationDate="2019-09-12T01:12:17.660" Score="3" ViewCount="66" Body="&lt;p&gt;There is a &lt;a href=&quot;https://bioconductor.org/packages/release/data/experiment/html/scRNAseq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scRNAseq&lt;/a&gt; Bioconductor package with a few different example scRNA-seq datasets. Are there any R packages that offer multiple modalities of single-cell data? For example, hashtags or ADTs or ATAC-seq. I am trying to find something that can be used for testing or for a quick tutorial. Preferably something that is on CRAN or Bioconductor.&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="35" LastEditDate="2019-09-12T16:01:57.607" LastActivityDate="2019-09-12T16:01:57.607" Title="public multi-modal single-cell data" Tags="|scrnaseq|single-cell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10377" PostTypeId="2" ParentId="10375" CreationDate="2019-09-12T02:24:28.540" Score="5" Body="&lt;p&gt;These tutorials on &lt;a href=&quot;https://satijalab.org/seurat/v3.1/multimodal_vignette.html&quot; rel=&quot;noreferrer&quot;&gt;Seurat multimodal&lt;/a&gt; data and the wrapper &lt;a href=&quot;https://github.com/satijalab/seurat-data&quot; rel=&quot;noreferrer&quot;&gt;Seurat data&lt;/a&gt; are easy ways to start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The wrapper has some cite-seq data preinstalled making it easy to work with benchmarked data sets&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using Seurat u can just as well have a look at &lt;a href=&quot;https://satijalab.org/signac/&quot; rel=&quot;noreferrer&quot;&gt;Signac&lt;/a&gt; to start working with some sc-atac seq&lt;/p&gt;&#xA;" OwnerUserId="2675" LastActivityDate="2019-09-12T02:24:28.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10378" PostTypeId="2" ParentId="10371" CreationDate="2019-09-12T08:15:58.890" Score="0" Body="&lt;p&gt;thanks! I figured out the solution, it was the cns. I downloaded the binaries instead of compiling from source and it works!&lt;/p&gt;&#xA;" OwnerUserId="5180" LastEditorUserId="298" LastEditDate="2019-09-12T10:12:16.683" LastActivityDate="2019-09-12T10:12:16.683" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10379" PostTypeId="1" AcceptedAnswerId="10380" CreationDate="2019-09-12T10:22:12.440" Score="2" ViewCount="53" Body="&lt;p&gt;I ran this command for 10 iteraions and 500 models.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;run_dmpfold.sh sequence.fasta sequence.21c sequence.map ./dmpfold-lysin 10 500 &amp;amp;&amp;gt; dmpfold-log.log&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am looking to generate 5000 dmpfold models to compare rosetta results with dmpfold; is it reasonable to do this? and, 5000 models can be generated by using &quot;10 5000&quot; parameter or &quot;10 500&quot; when running dmpfold as stated in the command line? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="5180" LastActivityDate="2019-09-12T10:47:49.207" Title="DMPfold: is it suitable to get 5000 models to compare with rosetta" Tags="|protein-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10380" PostTypeId="2" ParentId="10379" CreationDate="2019-09-12T10:47:49.207" Score="3" Body="&lt;p&gt;DMPfold tends to produce low-variability models - it produces a small number of distinct conformations, and either the right one is in there or it isn't. This arises from the way the models are generated from distance constraints. Rosetta, by contrast, produces many different conformations which is why 5000 or more models tend to be generated. This arises from Rosetta's method of assembling small fragments, which allows for many more combinations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You &lt;em&gt;could&lt;/em&gt; generate 5000 models with DMPfold. Running with &lt;code&gt;10 5000&lt;/code&gt; would give you 5000 models as output from the last iteration. Running with &lt;code&gt;10 500&lt;/code&gt; would give you 500 models from the last iteration but you could also use the 500 models from each of the previous iterations (saved as &lt;code&gt;ensemble.1.pdb&lt;/code&gt; etc.) to get 5000 models overall.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I wouldn't recommend it, it sounds like a waste of disk space and compute time. Just run with default parameters &lt;code&gt;3 50&lt;/code&gt; and use the final models saved as &lt;code&gt;final_*.pdb&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some numbers on how DMPfold compares to larger numbers of Rosetta models, and how much better DMPfold gets when you use more models, see Table 1 of &lt;a href=&quot;https://www.nature.com/articles/s41467-019-11994-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2019-09-12T10:47:49.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10381" PostTypeId="2" ParentId="10365" CreationDate="2019-09-12T12:50:25.510" Score="1" Body="&lt;p&gt;To finish you need bootstrap support using a model-based tree building algorhithm, I use RAxML, specifically raxmlHPC (easily downloadable and compiles on Linux and OSX). The codes are quite complicated and I've given them below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A robust maximum likelihood tree is,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;raxmlHPC -m PROTGAMMAILG -n output.tre -o outgroup-p 10000 -s alignment_file.phy&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To make a bootstrap tree,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;raxmlHPC -b 459 -m PROTGAMMAILG -n output.tre -o outgroup -p 10000 -s /path_to_alignment.phy -N 500&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Make a consensus tree of the bootstraps,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;raxmlHPC -f b -m PROTGAMMAILG -n output.tre -o outgroup -p 10000 -t /path_to_master_tree(best_tree) -z path_to_bootfile&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You require bootstrap support &gt;80% and please repeat this with and without 5-2/5-3 (it still looks long)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You need a fairly robust desktop, dedicated for around 24 hours to complete one calculation. You have enough results to draw a conclusion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Viewing the tree, FigTree is easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Root&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check the work of Cavalier-Smith, he heavily focused on basal protists to find a species which evolved, using external evidence, before any other member of your group. One of the ingroups may have sufficient evidence to say it evolved before the rest and that is your root, which you select as an outgroup. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-09-12T12:50:25.510" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="10382" PostTypeId="2" ParentId="9330" CreationDate="2019-09-12T13:04:33.280" Score="1" Body="&lt;p&gt;Specific questions&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;When there are multiple hsps in one alignment then which one do I include in the tree?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You can included them all because NJ is not computer intensive, so running loads is fine. You can remove identical sequences if you want either programmatically (these days I use pandas Python) or something like RAxML. Its honestly not critical for an NJ tree (and in fact the more the better) providing you can read the labels. A tree viewing program such as FigTree allows you to collapse clades to make the tree more readable to users.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;If two sequences map to two different parts of the query sequence and do not overlap, then what should I do?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Hmmmm.... good question. Do 3 alignments one with one e.g. 5' fragment, the other with the 3' fragment and attempt a joint alignment. Under other non-pairwise tree buidling algorithms this would be disasterous, NJ gets around the problem because it is pairwise. HOWEVER, ask yourself whether your feel the resulting tree is meaningful, I would argue not because there are no homologous sites between the 5' and 3' partial sequences and phylogeny models point mutations. NJ will give you a tree however and will even bootstrap it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generic questions, you can actually do this as one of the new features of NCBI's Blast&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Go to Blast here, &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blast.ncbi.nlm.nih.gov/Blast.cgi&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Enter your sequence into the box (it doesn't accept PDB codes alone)&lt;/li&gt;&#xA;&lt;li&gt;Enter the protein database - when I first did this calculation I used SwissProt, thinking there would be alot of sequences - I then used &quot;nr&quot;&lt;/li&gt;&#xA;&lt;li&gt;Under the algorithm parameters enter &quot;50&quot; (default is too many)&lt;/li&gt;&#xA;&lt;li&gt;Hit &quot;Blast&quot;&lt;/li&gt;&#xA;&lt;li&gt;Once the search is complete at the top of the page are the hyperlinks: &quot;Other reports:       &lt;/li&gt;&#xA;&lt;li&gt;Search Summary [Taxonomy reports] [Distance tree of results]&quot;&lt;/li&gt;&#xA;&lt;li&gt;Click on &quot;Distance tree of results&quot;&lt;/li&gt;&#xA;&lt;li&gt;The tree page will load automatically, automatically aligning your sequences and producing, in this case a parsimony based tree, but there is also the option of a nj tree (recommended) ....&lt;/li&gt;&#xA;&lt;li&gt;Click &quot;Tool&quot;, &quot;Download&quot;, &quot;PDF&quot; ...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;You can generate a distance and matrix and/or perform a neighbor-joining phylogeny in &lt;a href=&quot;https://www.megasoftware.net&quot; rel=&quot;nofollow noreferrer&quot;&gt;MEGA10&lt;/a&gt; and using Timura 3-parameter model is a good model. The bootstrap option is important. Its very easy to use, the format is a bit weird&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# MEGA&#xA;Title: Stuff&#xA;# Sequence1&#xA;ACTAGACGT&#xA;# Sequence2&#xA;ACCTTAGGA&#xA;&#xA;etc ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-09-12T13:18:30.180" LastActivityDate="2019-09-12T13:18:30.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10383" PostTypeId="2" ParentId="10374" CreationDate="2019-09-12T14:15:02.047" Score="0" Body="&lt;p&gt;One possible workaround would be to blast your sequences against the genome. Then, when you have found where they align, you can extract those regions only into a new fasta file (make sure to take a few kb around each target region) and then use that fasta file as the &quot;genome&quot; you pass to bowtie2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That would give bowtie a far smaller file to work with and should reduce the memory needed significantly.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-09-12T14:15:02.047" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10384" PostTypeId="1" AcceptedAnswerId="10385" CreationDate="2019-09-12T14:52:44.963" Score="0" ViewCount="36" Body="&lt;p&gt;I ran dmpfold using this command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;run_dmpfold.sh sequence.fasta sequence.21c sequence.map ./dmpfold-lysin 10 500 &amp;amp;&amp;gt; dmpfold-log.log&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It produced files like this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;sequence_sub_embed_1.pdb ..... sequence_sub_embed_500.pdb&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And, now I see some more files like this being produced:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;sequence_1.pdb .... sequence_2.pdb .....etc&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you please let me know what is dmpfold doing here? And can I stop dmpfold in between when desired number of models are generated? And, while it is running, can I move the previously made pdbs to new location becuase of disk space issues or all pdbs have to be in the &quot;dmpfold-lysin&quot; as stated in the command-line above? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="5180" LastEditorUserId="5180" LastEditDate="2019-09-12T15:29:28.860" LastActivityDate="2019-09-12T15:38:06.527" Title="DMPfold: Understand the run of DMPfold &quot;run_dmpfold.sh&quot;?" Tags="|protein-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10385" PostTypeId="2" ParentId="10384" CreationDate="2019-09-12T15:38:06.527" Score="1" Body="&lt;p&gt;From the &lt;a href=&quot;https://github.com/psipred/DMPfold&quot; rel=&quot;nofollow noreferrer&quot;&gt;docs&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The final model is &lt;code&gt;final_1.pdb&lt;/code&gt; and other structures may or may not be generated as &lt;code&gt;final_2.pdb&lt;/code&gt; to &lt;code&gt;final_5.pdb&lt;/code&gt; if they are significantly different. Many other files are generated totalling around 100 MB - these should be deleted to save disk space if you are running DMPfold on many sequences.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;These other files include intermediate CNS files (&lt;code&gt;sequence_sub_embed_1.pdb&lt;/code&gt; etc.), output files before clustering (&lt;code&gt;sequence_1.pdb&lt;/code&gt; etc.) and saved ensembles of previous iterations (&lt;code&gt;ensemble.1.pdb&lt;/code&gt; etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let the run finish or you won't get the final output files. If you want to use all the generated structures, use the &lt;code&gt;ensemble.n.pdb&lt;/code&gt; file from the last iteration (i.e. the highest number). You can also use the other ensemble files if you want to include models from previous iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would strongly recommend using default &quot;3 50&quot; parameters until you get things working. Don't move files during the run. You really don't get much benefit of going beyond 50 models per iteration, trust me. We tried it relatively extensively!&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2019-09-12T15:38:06.527" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="10386" PostTypeId="1" AcceptedAnswerId="10387" CreationDate="2019-09-12T15:45:09.103" Score="3" ViewCount="1135" Body="&lt;p&gt;There's a variety of formats for storing genomic features (e.g. Genbank, GFF, GTF, Bedfile) but all of the ones that I'm familiar with use either a custom format (Genbank) or a CSV with defined column conventions and space for additional data (GFF, GTF, Bedfile).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a feature format that is JSON based and adheres to a defined schema?&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2022-01-14T04:30:48.027" Title="Is there a JSON-based genomic feature format?" Tags="|features|genomefeatures|" AnswerCount="3" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="10387" PostTypeId="2" ParentId="10386" CreationDate="2019-09-12T20:15:27.040" Score="5" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/ekg/jvcf&quot; rel=&quot;nofollow noreferrer&quot;&gt;JVCF&lt;/a&gt; is a JSON schema for describing genetic variants. &lt;a href=&quot;https://jalview.github.io/biojson/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioJSON&lt;/a&gt; is a JSON schema for multiple sequence alignments. The output of &lt;a href=&quot;https://mygene.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;mygene.info&lt;/a&gt;'s gene annotation web service is JSON-formatted. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might build one of your own schemas from any or all of these, as inspiration for structure and content. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Consider that most genomics formats are historically tabular, because they are not hierarchical, because they originated prior to the advent of JavaScript as a common programming language (and subsequent use of JSON to serialize data), and because a lot of stream-oriented UNIX tools read in data and process it in ways that are natural to tabular structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some tabular results can be hierarchical, but use inefficiencies in field values to apply hierarchy. JSON is inefficient for storing tabular data, but it is absolutely wonderful for hierarchical data. This difference is perhaps worth considering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One could think of storing a gene as a master object containing various haplotypes, exons, introns, features, associated variants, associated TF binding sites, gene ontologies, etc. Ordering or prioritization of such items would be possible with the use of sorted lists within a gene object. The format is as extensible as it is a matter of adding a property, list, or object to the schema. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To gain adoption or &quot;buy-in&quot;, build tools which publish it to users, and tools that consume what users get in order to do things, along with conversion and validation utilities.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2019-09-14T09:40:10.650" LastActivityDate="2019-09-14T09:40:10.650" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10388" PostTypeId="2" ParentId="5507" CreationDate="2019-09-13T06:26:58.957" Score="0" Body="&lt;p&gt;All tools I’m aware of will be more-or-less as fast as blast. With kmer-based methods, the investment will be the up-front kmer database build.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Not sure if you’ve tried this, but NCBI has a 16S microbial blast database (lots of species, 16S rRNA only) just for this purpose. No point to search millions of nts for the 16S sequence every time if you know it’s SOME 16S sequence, so limit the search to only16S sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’d recommend you build a local blast database from &lt;a href=&quot;ftp://ftp.ncbi.nih.gov/blast/db/16SMicrobial.tar.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ncbi.nih.gov/blast/db/16SMicrobial.tar.gz&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, another answer says blast won’t give you the alignment. Blast actually will give you alignments, you just have to ask it for that output format.&lt;/p&gt;&#xA;" OwnerUserId="3564" LastActivityDate="2019-09-13T06:26:58.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10389" PostTypeId="2" ParentId="10386" CreationDate="2019-09-13T11:43:07.743" Score="1" Body="&lt;p&gt;BioJS has capacity to read and write GFF as streams, it might have something of interest such as this: &lt;a href=&quot;https://github.com/biojs-io/biojs-io-gff&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/biojs-io/biojs-io-gff&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3828" LastEditorUserId="3828" LastEditDate="2019-09-13T12:01:42.363" LastActivityDate="2019-09-13T12:01:42.363" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10390" PostTypeId="1" CreationDate="2019-09-13T22:44:02.653" Score="2" ViewCount="34" Body="&lt;p&gt;How do I translate the body of a function definition in SBML into infix using python libsbml?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example after loading the model using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;reader = libsbml.SBMLReader()&#xA;doc = reader.readSBMLFromString (r.getSBML())&#xA;model = doc.model&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4894" LastEditorUserId="4894" LastEditDate="2020-03-03T19:26:35.990" LastActivityDate="2020-03-03T19:26:35.990" Title="How to I get the infix expression for the body in a SBML function defintion using python libsbml?" Tags="|sbml|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10391" PostTypeId="2" ParentId="10390" CreationDate="2019-09-13T22:49:23.330" Score="1" Body="&lt;p&gt;To do the translation you need to access the function definition. First, make sure there are function definitions in your model by using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if model.getNumFunctionDefinitions() &amp;gt; 0:&#xA;   # continue&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To get the infix of the body for the ith function definition use the following code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f = model.getFunctionDeinfition(i)&#xA;body = f.getBody()&#xA;print (libsbml.formulaToL3String(body))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As a side note you can also get the jth argument for the function definition using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f = model.getFunctionDeinfition(i)&#xA;arg = f.getArgument(j)&#xA;print (libsbml.formulaToL3String(arg))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4894" LastActivityDate="2019-09-13T22:49:23.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10392" PostTypeId="1" CreationDate="2019-09-13T22:54:17.293" Score="2" ViewCount="180" Body="&lt;p&gt;I have gene expression data (RNA-seq) for 30 different time points (from 0 to 60 min each 2 min).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a set of 8 genes that behave similarly (although not identically) and I want to find the top X genes that behave most similarly to this set of genes in terms of their time-series expression profile.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried so far to do unsupervised clustering (&lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/Mfuzz.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;mfuzz&lt;/a&gt;) and check for genes that lie in the same cluster as my genes of interest. However, this approach relies on specifying a predefined number of clusters and since these genes are not exactly equal, sometimes they don't fall in the same cluster, depending on initial parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Because of this, I have thought about doing it the other way around and look for genes similar to my set of genes. However, I haven't found any method/resource for doing that so far. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So, is there a way to find genes with a time-series profile similar to a predefined set of genes?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="678" LastEditorUserId="298" LastEditDate="2019-09-14T10:53:36.333" LastActivityDate="2019-09-18T15:39:11.087" Title="Find most similar genes to a set of genes of interest in time series RNA-seq data" Tags="|rna-seq|clustering|gene-expression|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10394" PostTypeId="1" CreationDate="2019-09-15T02:59:01.153" Score="0" ViewCount="365" Body="&lt;p&gt;I'm curious about can we show the orig.ident in sprat windows by using Seurat? Here are my code and the fig I got. Is it possible that I can separate each orig.ident into a single panel? &lt;code&gt;DimPlot(whole,group.by = &quot;orig.ident&quot;, reduction = &quot;umap&quot;)&lt;/code&gt;&lt;a href=&quot;https://i.stack.imgur.com/oc7hY.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oc7hY.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3171" LastActivityDate="2021-06-07T01:04:49.733" Title="can we show the orig.ident in separate windows by using Seurat" Tags="|single-cell|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10395" PostTypeId="2" ParentId="10394" CreationDate="2019-09-15T07:23:46.827" Score="1" Body="&lt;p&gt;The &lt;code&gt;DimPlot()&lt;/code&gt; function of the new version of Seurat, &lt;code&gt;Seurat v3&lt;/code&gt; has a &lt;code&gt;split.by&lt;/code&gt; argument, which splits the plot based on the levels of the variable provided. You basically need to provide a column from the &lt;code&gt;@metadata&lt;/code&gt; slot:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;DimPlot(whole, split.by = &amp;quot;orig.ident&amp;quot;, reduction = &amp;quot;umap&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you are using &lt;code&gt;Seurat v2&lt;/code&gt;, you can have the &lt;code&gt;DimPlot()&lt;/code&gt; function return the &lt;code&gt;ggplot&lt;/code&gt; object created and then use &lt;code&gt;facet_wratp()&lt;/code&gt; or &lt;code&gt;facet_grid()&lt;/code&gt; to generate facets based on your variable of interest.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="73" LastEditDate="2021-06-07T01:04:49.733" LastActivityDate="2021-06-07T01:04:49.733" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10396" PostTypeId="2" ParentId="9342" CreationDate="2019-09-15T07:40:34.150" Score="2" Body="&lt;p&gt;Many pipelines have been set up for this such as software developed for the &lt;a href=&quot;http://fantom.gsc.riken.jp/software/&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Fantom Project&quot;&gt;FANTOM Project&lt;/a&gt; at RIKEN. The software for this is available from our website. These were set up for analysis of CAGE sequencing data for exactly this purpose of calling peaks of previously unannotated gene expression. You can also find data for many species (including chicken) from &lt;a href=&quot;http://fantom.gsc.riken.jp/5/&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;Fantom 5&quot;&gt;FANTOM5&lt;/a&gt; where this has already been done.&lt;/p&gt;&#xA;" OwnerUserId="1855" LastActivityDate="2019-09-15T07:40:34.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10397" PostTypeId="1" AcceptedAnswerId="10400" CreationDate="2019-09-15T08:51:52.080" Score="0" ViewCount="666" Body="&lt;p&gt;I have RNA-seq from two sequencing batches; Lab technician says that he has run the RNA expression quantification two times in bathes 1 and 2 for example &lt;code&gt;tumor 1 in batch 1 and tumor 1 in batch 2 , normal 2 in batch1 and normal 2 in batch 2&lt;/code&gt;. This is my design for DESeq2&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(mycols)&#xA;        condition batch&#xA;N_1_305         N     1&#xA;N_1_310         N     1&#xA;N_1_337         N     1&#xA;N_1_353         N     1&#xA;T_1_305         T     1&#xA;T_1_310         T     1&#xA;&amp;gt; tail(mycols)&#xA;        condition batch&#xA;T_2_337         T     2&#xA;T_2_338         T     2&#xA;T_2_344         T     2&#xA;T_2_346         T     2&#xA;T_2_349         T     2&#xA;T_2_353         T     2&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I got this PCA plot&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IYWxj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IYWxj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And this is biplot of samples&#xA;&lt;a href=&quot;https://i.stack.imgur.com/E2DOe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/E2DOe.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In PCA plot I am seeing for instance &#xA;, &lt;code&gt;T_1_337 (batch1) has been placed too close to T_2_337 (batch2)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I used svd for detecting hidden batch &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ab2uk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ab2uk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this mean that there is no big batch effect between experimental runs and I can concatenate the fastqs from both batches for each sample (technical replication) or collapse technical replicates afterwards?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please help me to interpret these&#xA;Thank you&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;EDITED&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Sorry @Devon, I have 4 lanes for each samples (paired end) in each of experimental runs; For concatenating fastq files can I do like this ?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat fastq1_lane1_batch1 fastq1_lane1_batch2 fastq1_lane2_batch1   fastq1_lane2_batch2  fastq1_lane3_batch1 fastq1_lane3_batch2 fastq1_lane4_batch1 fastq1_lane4_batch2  &amp;gt; fastq1  &#xA;&#xA;cat fastq2_lane1_batch1 fastq2_lane1_batch2 fastq2_lane2_batch1   fastq2_lane2_batch2  fastq2_lane3_batch1 fastq2_lane3_batch2 fastq2_lane4_batch1 fastq2_lane4_batch2 &amp;gt; fastq2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-09-15T23:45:33.963" LastActivityDate="2019-09-16T16:39:21.463" Title="Interpreting this PCA plot for RNA-seq" Tags="|r|rna-seq|deseq2|pca|batch-effects|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10398" PostTypeId="1" CreationDate="2019-09-15T12:28:12.947" Score="2" ViewCount="235" Body="&lt;p&gt;I was came across a problem during an exercise in a book and I don't really know how to solve it. I feel like something's missing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;coverage, c = &lt;span class=&quot;math-container&quot;&gt;$NL/G$&lt;/span&gt; (N=number of reads, L=read length, G=genome length)&#xA;total expected gap length = &lt;span class=&quot;math-container&quot;&gt;$G(e^{-c})$&lt;/span&gt;&#xA;total number of gaps = &lt;span class=&quot;math-container&quot;&gt;$N(e^{-c})$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question: You want to sequence a 4 Mb genome by the shotgun method, by assembling random fragments with read length 500. What coverage would you require, to expect no more than four gaps, assuming no complications arising from repetitive sequences or far-from equimolar base composition?&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer from the book is basically just: Solve &lt;span class=&quot;math-container&quot;&gt;$N(e^{-c})=4$&lt;/span&gt; for &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;, but the question doesn't give what &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is? &lt;/p&gt;&#xA;" OwnerUserId="6211" LastEditorUserId="73" LastEditDate="2019-09-15T22:41:10.843" LastActivityDate="2021-02-25T15:06:35.630" Title="Coverage required" Tags="|ngs|genome|assembly|coverage|homework|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10399" PostTypeId="1" CreationDate="2019-09-15T17:13:53.973" Score="1" ViewCount="311" Body="&lt;p&gt;I have done a WGCNA (weighted gene co-expression network analysis) analysis for various brain disorders. So along with control, I have 3 different disorders such as BPD, MDD and SCZ. Now I have to find out what sort of modules are preserved between the control and disease condition.&lt;/p&gt;&#xA;&lt;p&gt;I have 54 samples in control and 50 samples in MDD as one of the condition. The other conditions are similar, somewhere between 50-60 samples each.&lt;/p&gt;&#xA;&lt;p&gt;I have cited a paper where they have done module preservation analysis. Now I have checked the WGCNA or website they do have given example of mouse male and female liver module preservation.&lt;/p&gt;&#xA;&lt;p&gt;I am stuck at this do I need both the expression data together to do module preservation analysis? Or do I first have to run WCGNA on both the data sets and then look for?&lt;/p&gt;&#xA;&lt;p&gt;For example if I want to look at what are the modules preserved in CTRL and BPD condition.&lt;/p&gt;&#xA;&lt;p&gt;I have followed this &lt;a href=&quot;http://pages.stat.wisc.edu/%7Eyandell/statgen/ucla/WGCNA/wgcna.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; section &lt;strong&gt;12.5 Module preservation between female and male mice&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;where it reads both male and female data together which is bit confusing&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;maleData = read.csv(file.path(data.dir, &amp;quot;LiverMaleFromLiverFemale3600.csv&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have also followed this tutorial:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://horvath.genetics.ucla.edu/html/CoexpressionNetwork/ModulePreservation/Tutorials/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://horvath.genetics.ucla.edu/html/CoexpressionNetwork/ModulePreservation/Tutorials/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion would be really helpful.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://insight.jci.org/articles/view/94904/figure/2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Weighted gene network coexpression analysis (WGCNA) reveals stage-specific modules (gene network) in neonatal left and right ventricles&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="73" LastEditDate="2022-06-21T10:41:30.000" LastActivityDate="2022-06-21T10:41:30.000" Title="Module preservation analysis WGCNA" Tags="|r|rna-seq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10400" PostTypeId="2" ParentId="10397" CreationDate="2019-09-15T21:39:02.377" Score="3" Body="&lt;p&gt;Yes, you can safely concatenate the technical replicates. Odds are good that these are even the same libraries just sequenced twice, so even labeling them as replicates is a bit of a stretch. As an aside, it would be surprising if you actually had a batch effect in a situation like this. You will commonly see sequencing facilities just sequence a given sample a second time if it didn't get the number of reads requested by the client to begin with. They then just have the client concatenate the files.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-09-15T21:39:02.377" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10401" PostTypeId="1" CreationDate="2019-09-16T08:28:08.390" Score="0" ViewCount="142" Body="&lt;p&gt;I want to get the expression data that is &lt;strong&gt;&lt;em&gt;in-sample normalized&lt;/em&gt;&lt;/strong&gt; like FPKM and also &lt;strong&gt;&lt;em&gt;across samples normalized&lt;/em&gt;&lt;/strong&gt; as obtained using DESeq2 or else.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I am currently doing is that I first normalize the data across samples (using DESeq) and from the resultant expression I calculate the FPKM. Does it make sense or am I missing something here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how I am doing (not the exact code but the idea)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData, DataFrame(condition), ~ condition)&#xA;dds &amp;lt;- estimateSizeFactors(dds)&#xA;norm_data &amp;lt;- counts(dds, normalized=TRUE)&#xA;&#xA;foreach sample&#xA;{&#xA;    foreach transcript&#xA;    {&#xA;        FPKM = (Normalized read count * 10^9) / (transcript length * total mapped normalized read count)&#xA;    }&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1759" LastEditorUserId="1759" LastEditDate="2019-09-16T12:14:58.933" LastActivityDate="2019-09-16T14:16:20.810" Title="In-sample and across samples normalized expression" Tags="|rna-seq|normalization|reads|fpkm|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10402" PostTypeId="2" ParentId="10392" CreationDate="2019-09-16T09:23:06.990" Score="1" Body="&lt;p&gt;You can use &lt;a href=&quot;https://bioconductor.org/packages/GSVA/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSVA&lt;/a&gt; to summarize the information of your 8 genes and look for the time-serie change using limma or DESeq2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have enough samples for each time point you could also do a weighted gene co-expression network analysis (&lt;a href=&quot;https://horvath.genetics.ucla.edu/html/CoexpressionNetwork/Rpackages/WGCNA/Tutorials/&quot; rel=&quot;nofollow noreferrer&quot;&gt;WGCNA&lt;/a&gt;), and see if those clusters in each time point are co-expressed similarly (But it can be tricky and you will need to define a lower bound of the clusters to 8 or so). You could also divide your time series in two point at the beginning and at the end, and see if there are differences or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to get a list of genes similar to your set of genes you can use &lt;a href=&quot;https://bioconductor.org/packages/GOSemSim/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GOSemSim&lt;/a&gt; or &lt;a href=&quot;https://bioconductor.org/packages/BioCor/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioCor&lt;/a&gt; (Disclaimer I am a contributor to the first, and author of the second), which can calculate similarities between genes based on the gene ontology and on pathways respectively. For this approach you'll need to calculate the similarity between all the genes and then select those that are closer to your genes of interest. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not aware of any database/website with time-series profiles of genes (I assume that if there would be any would be on humans or some simple model organism).&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2019-09-17T07:10:11.613" LastActivityDate="2019-09-17T07:10:11.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10403" PostTypeId="1" CreationDate="2019-09-16T11:06:13.270" Score="2" ViewCount="714" Body="&lt;p&gt;I'm analyzing some Oxford Nanopore sequencing reads, with several articles stating the sequencing error is between 5% and 15%. Since its 1D and not 1D^2 sequencing i would assume the sequencing error is closer to 15%. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using Minimap2 aligner to align the reads to the reference genome, and then im choosing those reads with a mapQ of 60, to analyze.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where i know with the phred score mapQ of 60 means 99.9999% of the bases are called correctly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my question is, if the mapQ is 60, then does it really matter if my sequencing error is 15%? Or how is the sequencing error correlated with mapQ? Becuase i guess you could still obtain a good mapping quality with a few mismatches?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or am i completely off target.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your time and help.&lt;/p&gt;&#xA;" OwnerUserId="6214" LastActivityDate="2019-09-16T14:10:19.850" Title="Oxford Nanopore mapping Quality and sequencing error" Tags="|nanopore|read-mapping|quality-control|errors|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10404" PostTypeId="2" ParentId="10401" CreationDate="2019-09-16T12:05:39.250" Score="0" Body="&lt;p&gt;You might want to check the &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/vignettes/EDASeq/inst/doc/EDASeq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;EDASeq()&lt;/a&gt; package, which attempts to address both gene-specific and sample-specific effects (I believe this is what you are trying to achieve). From their vignette:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Following (Risso et al. 2011), we consider two main types of effects&#xA;  on gene-level counts: (1) within-lane gene-specific (and possibly&#xA;  lane-specific) effects, e.g., related to gene length or GC-content,&#xA;  and (2) effects related to between-lane distributional differences,&#xA;  e.g., sequencing depth. Accordingly, withinLaneNormalization and&#xA;  betweenLaneNormalization adjust for the first and second type of&#xA;  effects, respectively. &lt;strong&gt;We recommend to normalize for within-lane&#xA;  effects prior to between-lane normalization&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-09-16T12:05:39.250" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10405" PostTypeId="2" ParentId="10403" CreationDate="2019-09-16T14:10:19.850" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Where i know with the phred score mapQ of 60 means 99.9999% of the bases are called correctly.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That's not at all what the MAPQ means, it means that the sequence is very likely to have originated from that place in the genome, whether the underlying sequence is correct is another question all together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For long nanopore reads, sequence quality is not well correlated with MAPQ. MAPQ will tend to be either very low (0 or 1) or very high (60) since you tend to have kb+ reads and those will largely be uniquely assignable in the genome. The sequencing quality of nanopore reads is only an issue when you care about something at the base level, such as SNP calling or finding exact splice junctions. Even then, you mostly need sufficient coverage to get around this.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-09-16T14:10:19.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10406" PostTypeId="2" ParentId="10401" CreationDate="2019-09-16T14:16:20.810" Score="1" Body="&lt;p&gt;Yes, this is a standard way of obtaining RPKM/FPKM/CPM values for plotting. Not that you do not need to use a &lt;code&gt;for&lt;/code&gt; loop for any of the computations in R. You have a matrix of normalized values and things like transcript length are constant across samples (at least unless you're using something like salmon...although then you'd have TPMs to begin with).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-09-16T14:16:20.810" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10407" PostTypeId="1" CreationDate="2019-09-16T14:57:05.253" Score="2" ViewCount="2268" Body="&lt;p&gt;I want to see the expression of a gene in a group of patient amongst the entire cohort using my RNA-Seq data. While I can do a differential expression analysis with limma or DESeq2, I want to see how much each sample from my cohort expresses the gene. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The plan is to plot a waterfall plot (as defined in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4093310/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;). However, when I used TPM and rlog(CPM) they gave me very different looking graphs (Not just the shape, which is expected, but the ranking of expression). My concerns are, for TPM, the batch effect with two or three lots in one cohort, and, for rlog(CPM), the results don't match with prediction but is adjusted for batch effect. Which one would be a right or better choice?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Upper one is with plot with. Lower one is plot with voom transformed counts (log(CPM)). I didn't include all cases since there are too many. This look more or less what it is like in the one including every case. Target is the condition group in which I am interested. Differential expression analysis definitely showed that it is upregulated.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                     logFC       CI.L       CI.R    AveExpr         t      P.Value    adj.P.Val         B&#xA;GeneA  1.6576257  0.9143490  2.4009024  5.8713219  4.391796 1.646627e-05 0.0002581676  2.484413&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Working steps&#xA;TPM is basically from kallisto&lt;/p&gt;&#xA;&#xA;&lt;p&gt;log(CPM) calculation&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Import data from tximport&#xA;txi = tximport(files = read.hd5s,&#xA;                     type = &quot;kallisto&quot;,&#xA;                     tx2gene = tx2gene,&#xA;                     countsFromAbundance = &quot;lengthScaledTPM&quot;,&#xA;                     ignoreTxVersion = TRUE)&#xA;raw_data = DGEList(txi$counts)&#xA;# Filter low count genes&#xA;keep &amp;lt;- filterByExpr(raw_data)&#xA;raw_data &amp;lt;- raw_data[keep,,keep.lib.sizes=FALSE]&#xA;# Normalise counts&#xA;raw_data = calcNormFactors(raw_data)&#xA;# log(CPM) by voom&#xA;voom_data = voom(raw_data, design = NULL, normalize=&quot;none&quot;, plot = TRUE)&#xA;# Batch effect correction&#xA;voom_expr_psva = psva(as.matrix(voom_data$E),Batch)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I plot graph with TPM and log(CPM) respectively&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/f1t1V.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/f1t1V.png&quot; alt=&quot;Waterfall plot using TPM&quot;&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/jxRIp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jxRIp.png&quot; alt=&quot;Waterfall plot using voom&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3225" LastEditorUserId="3225" LastEditDate="2019-09-17T21:29:06.017" LastActivityDate="2019-10-17T22:02:25.317" Title="TPM or rlog(CPM) for comparing expression?" Tags="|rna-seq|limma|batch-effects|tpm|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10408" PostTypeId="1" CreationDate="2019-09-16T15:34:58.683" Score="1" ViewCount="172" Body="&lt;p&gt;Forgive me if this question is too trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a list of ~700 Ensembl ID. I need to extract the protein structures of each Ensembl id. How can I do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any script(python, R) so that we can download the structure from the website(like RCSB)?&lt;/p&gt;&#xA;" OwnerUserId="6216" LastActivityDate="2020-03-09T18:01:30.187" Title="From ensembl id to protein structure" Tags="|gene|proteins|python|protein-structure|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10409" PostTypeId="2" ParentId="10397" CreationDate="2019-09-16T16:39:21.463" Score="1" Body="&lt;p&gt;Prepping the RNA on different days, or making Illumina libraries on different days, or having different technicians handle different samples; that can lead to batch effects. &#xA; Running samples on two different days does not cause a significant batch effect, as you can plainly see in your PCA.  You should just combine the fastqs.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-09-16T16:39:21.463" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10410" PostTypeId="1" CreationDate="2019-09-16T17:40:58.217" Score="1" ViewCount="98" Body="&lt;p&gt;I have two sequences, say AAAGCTCGAGG and AAAGCGAGG. I need a convenient tool which shows me insertions or deletions between these, i.e. in this case something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AAAGCTCGAGG&#xA;|||||  ||||&#xA;AAAGC  GAGG&#xA;2bp deletion starting at base pair 5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and also insertions such as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AAAGCTC GAGG&#xA;||||||| ||||&#xA;AAAGCTCTGAGG&#xA;1bp insertion at base pair 7&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does something like that exist? Blastn finds the alignment of course, but only outputs the &lt;em&gt;number&lt;/em&gt; of gaps, not their &lt;em&gt;position&lt;/em&gt;. I guess I could parse a pileup, but do not want to create a bam file from one of the sequences first, so samtools seems to be out. Any further ideas?&lt;/p&gt;&#xA;" OwnerUserId="4145" LastActivityDate="2019-12-22T06:00:50.177" Title="Find indels between two short sequences" Tags="|samtools|indel|blastn|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10411" PostTypeId="2" ParentId="10410" CreationDate="2019-09-16T18:40:37.133" Score="0" Body="&lt;p&gt;Perhaps you could use something like &lt;code&gt;SearchIO&lt;/code&gt; to parse the output of BLAST, to get the desired alignment attributes: &lt;a href=&quot;https://biopython.org/DIST/docs/api/Bio.SearchIO.BlastIO-module.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biopython.org/DIST/docs/api/Bio.SearchIO.BlastIO-module.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2019-09-16T18:40:37.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10412" PostTypeId="1" CreationDate="2019-09-16T20:18:22.720" Score="0" ViewCount="83" Body="&lt;p&gt;I have a list of gene name, I want to download and save each protein 3D structure corresponding to each gene name. &lt;/p&gt;&#xA;" OwnerUserId="6216" LastActivityDate="2019-09-16T22:50:30.300" Title="Download protein structure from gene name" Tags="|gene|protein-structure|pdb|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10413" PostTypeId="2" ParentId="264" CreationDate="2019-09-16T20:19:02.353" Score="2" Body="&lt;p&gt;For structural variants  I think you can use &lt;a href=&quot;https://github.com/fritzsedlazeck/SURVIVOR&quot; rel=&quot;nofollow noreferrer&quot;&gt;SURVIVOR&lt;/a&gt; like &lt;code&gt;SURVIVOR stats&lt;/code&gt; which was design specifically for such purpose (statistics on SV file).&lt;/p&gt;&#xA;" OwnerUserId="418" LastActivityDate="2019-09-16T20:19:02.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10414" PostTypeId="2" ParentId="10410" CreationDate="2019-09-16T21:23:07.267" Score="0" Body="&lt;p&gt;blastn should be able to output this format of alignment result, using the &lt;code&gt;--outfmt 0&lt;/code&gt; option&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-09-16T21:23:07.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10415" PostTypeId="2" ParentId="10412" CreationDate="2019-09-16T22:50:30.300" Score="1" Body="&lt;p&gt;There may also be a misunderstanding, from your previous post. To reiterate, I don't use Ensembl, which is the source of your gene names.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On Genbank there is an entire &quot;Structural&quot; database for every structure that is on PDB. When Genbank hold the protein sequence that was used to generate the structure Genbank only use PDB codes. For example &quot;5LBV_A&quot; is both a PDB code and a Genbank code. So this means PDB entry 5LBV chain A. Thus its easy to download the crystal because you simply ask PDB for 5LBV. To prove it,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;efetch -db protein -format gpc -id &quot;5LBV_A&quot; | xtract -pattern INSDSeq -group INSDSeq -element INSDSeq_sequence&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;OUT &#xA;    ircigvsnrdfvegmsggtwvdvvlehggcvtvmaqdkptvdielvtttvsnmaevrsycyeasisdmasdsrcptqgeayldkqsdtqyvckrtlvdrgwgngcglfgkgslvtcakfacskkmtgksiqpenleyrimlsvhgsqhsgmivndtghetdenrakveitpnspraeatlggfgslgldceprtgldfsdlyyltmnnkhwlvhkewfhdiplpwhagadtgtphwnnkealvefkdahakrqtvvvlgsqegavhtalagaleaemdgakgrlssghlkcrlkmdklrlkgvsyslctaaftftkipaetlhgtvtvevqyagtdgpckvpaqmavdmqtltpvgrlitanpvitestenskmmleldppfgdsyivigvgekkithhwhrsgstiggpfeddddkagwshpqfekgggsgggsgggswshpqfek&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I cannot easily do, is go from a generic NCBI Genbank protein code to PDB code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, PDB uses Uniprot codes for proteins, so you may be able to use a Uniprot code to get hold of the PDB code, or Ensembl codes may, like Genbank, use PDB accessions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in summary perhaps you need to give more detail?&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-09-16T22:50:30.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10416" PostTypeId="2" ParentId="10407" CreationDate="2019-09-16T22:50:48.263" Score="3" Body="&lt;p&gt;For comparing the counts of different samples from DESeq2, Michael Love &lt;a href=&quot;https://support.bioconductor.org/p/120764/#120843&quot; rel=&quot;nofollow noreferrer&quot;&gt;recommends&lt;/a&gt; using the variance-stabilized transform. It'd be great if you could provide some specific code examples in your question, but without that here's something that should work with the DESeq2 workflow as mentioned in the package documentation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;## [assuming &quot;dds &amp;lt;- DESeq(dds)&quot; has been run]&#xA;dds.counts &amp;lt;- assay(vst(dds, blind=FALSE));&#xA;## [This makes the scaled counts resemble more closely the actual read counts]&#xA;dds.quantile99 &amp;lt;- quantile(dds.counts[dds.counts &amp;gt; min(dds.counts)], 0.99);&#xA;dds.counts &amp;lt;- ((dds.counts - min(dds.counts)) / (dds.quantile99-min(dds.counts))) * (dds.quantile99);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The matrix &lt;em&gt;dds.counts&lt;/em&gt; should now represent a model-normalised approximation of the log of counts, with column names as the sample names, and row names as the gene names. The default behaviour of the VST changes zero counts to an arbitrary positive number, which is why I've modified the output so that count values of zero will be changed to 0 in log space (i.e. a count of 1). To plot results for a single gene, subset based on the row name:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gene.VST &amp;lt;- sort(dds.counts[&quot;geneOfInterest&quot;,]);&#xA;plot(gene.VST);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2019-09-16T22:50:48.263" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10418" PostTypeId="1" AcceptedAnswerId="10421" CreationDate="2019-09-17T00:45:26.413" Score="0" ViewCount="1038" Body="&lt;p&gt;I am looking for homologous genes in various protozoan species using BLAST. The genome sequences of these species are deposited in NCBI's WGS database. NCBI’s webpage includes the global statistics for each WGS project and it indicates the assembly level (scaffold, contig, etc.), the assembly statistics (number of contigs, N50, L50, etc) and (if available) the genome coverage. Can any / some of these parameters be used to evaluate the genome assembly’s &quot;quality&quot;, especially if there is more than one genome assembly for the same species? Can these data be used to determine if the genome is very fragmented or not?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that quality is a relative term and that there are several options for assessing genome assemblies (QUAST, BUSCO, etc.) but I am just looking for a &quot;simple&quot; way to compare several genome assemblies (for e.g., this post &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/242/how-can-i-improve-a-long-read-assembly-with-a-repetitive-genome&quot;&gt;How can I improve a long-read assembly with a repetitive genome?&lt;/a&gt; states that the reference genome is highly fragmented, but there is no explanation regarding how the OP determined this). I believe that the N50 value can be helpful, but I need some context to understand what is &quot;good&quot; or &quot;bad&quot;.  I am at a loss of where to start, so any information will be greatly appreciated! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;-Leah&lt;/p&gt;&#xA;" OwnerUserId="5116" LastEditorUserId="5116" LastEditDate="2019-09-17T01:01:29.187" LastActivityDate="2019-09-18T15:33:27.857" Title="Determining the quality of a publicly available genome assembly" Tags="|genome|assembly|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10419" PostTypeId="2" ParentId="4065" CreationDate="2019-09-17T01:35:37.460" Score="1" Body="&lt;p&gt;I wrote a &lt;a href=&quot;https://github.com/sdmiller93/Public_Scripts/blob/master/up_down_flat.R&quot; rel=&quot;nofollow noreferrer&quot;&gt;script&lt;/a&gt; to create the binary data frame with my differential expression data. If it is of any help to anyone:&lt;/p&gt;&#xA;" OwnerUserId="6219" LastActivityDate="2019-09-17T01:35:37.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10421" PostTypeId="2" ParentId="10418" CreationDate="2019-09-17T06:37:09.037" Score="3" Body="&lt;p&gt;It is not so easy to determine the quality of an assembly - but sequence length, number of contigs, N50 and L50 can give you some guidance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How much does the sequence length deviate from the expected length (or how much variation is among assemblies of the same species). Same for number of contigs/scaffolds. The sequence length, and number of contigs together with N50 and L50 give you an idea about the fragmentation. Very small N50 compared to expected genome size and chromosomes is a likely indication of high fragmentation, same for high L50 compared to expected number of chromosomes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, these number can be misleading. High N50 and low L50 can occur witho some very long contigs, so you may want to compare N75/L75 or better the Quast statistics&lt;/p&gt;&#xA;" OwnerUserId="3252" LastEditorUserId="3252" LastEditDate="2019-09-17T08:33:18.487" LastActivityDate="2019-09-17T08:33:18.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10422" PostTypeId="1" AcceptedAnswerId="10424" CreationDate="2019-09-17T08:49:34.133" Score="0" ViewCount="14" Body="&lt;p&gt;I ran this command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;run_dmpfold.sh sequence.fasta sequence.21c sequence.map ./folder 10 500 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;From my previous discussion, I understand that I should run with default &quot;run_dmpfold.sh&quot; parameters i.e. &lt;code&gt;3 50&lt;/code&gt;; however, since I have already run the &lt;code&gt;10 500&lt;/code&gt; I have some questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see that after generating &quot;sequence_1.pdb to sequence_500.pdb&quot; files, these files are no more in the folder and, now I see &quot;ensemble_1.pdb&quot;. &#xA;Will there be more ensemble files like &quot;ensemble_500.pdb&quot; and &quot;sequence_n.pdb&quot; files? I mean when can I expect &quot;final_1.pdb' file? Also, I see files like &quot;CLUSTERS_001.pdb&quot;.&lt;/p&gt;&#xA;" OwnerUserId="5180" LastEditorUserId="48" LastEditDate="2019-09-17T12:56:29.110" LastActivityDate="2019-09-17T12:56:29.110" Title="DMPfold: number of models produced using &quot;10 500&quot; feature in &quot;run_dmpfold.sh&quot;?" Tags="|protein-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10423" PostTypeId="2" ParentId="10407" CreationDate="2019-09-17T10:46:43.697" Score="2" Body="&lt;p&gt;To add to what @gringer, when you do use TPM, the normalization done is for both library size and gene length. When you use rlog, the normalization is done via median normalization (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2864565/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2864565/&lt;/a&gt;). See below for a quick example about the rlog, and size factor normalization.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dds=makeExampleDESeqDataSet(betaSD=0.5,sizeFactors =runif(12))&#xA;rld = rlog(dds)&#xA;## you can see the data is variance stablized, i.e lower counts are converted&#xA;### see the size factor calculated &#xA;rld$sizeFactor&#xA;###&#xA;boxplot(assay(rld))&#xA;## look at unnormalized distributions&#xA;boxplot(log2(counts(dds)+1))&#xA;## now we normalize and see&#xA;dds = estimateSizeFactors(dds)&#xA;# the size factors should be similar&#xA;table(sizeFactors(dds) == rld$sizeFactor)&#xA;### here you can see the medians are similar but its a bit different from rlog&#xA;boxplot(log2(counts(dds,normalize=TRUE)+1))&#xA;###&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To answer your question specifically, the sizeFactor is unaware of any batch effect. It assumes median expression of all genes to be similar and normalize using that. If you think there is a batch effect, it has to be present across many genes, as opposed to one gene. I would check it with a PCA plot (&lt;a href=&quot;https://www.rdocumentation.org/packages/DESeq2/versions/1.12.3/topics/plotPCA&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rdocumentation.org/packages/DESeq2/versions/1.12.3/topics/plotPCA&lt;/a&gt;) and then test for differential expression by including the batch effect in the model.&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2019-09-17T10:46:43.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10424" PostTypeId="2" ParentId="10422" CreationDate="2019-09-17T11:13:42.957" Score="0" Body="&lt;p&gt;The &lt;code&gt;sequence_n.pdb&lt;/code&gt; files are temporary and are combined into an &lt;code&gt;ensemble.i.pdb&lt;/code&gt; file after each iteration &lt;code&gt;i&lt;/code&gt;. That file has many structures in. After the run is complete there will be a &lt;code&gt;final_1.pdb&lt;/code&gt; file. Don't worry about the cluster files. If you are interested in what they do, consult the &lt;code&gt;run_dmpfold.sh&lt;/code&gt; script and see what it is doing line by line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I have said multiple times elsewhere, you should run the script with default parameters and let it run &lt;em&gt;to completion&lt;/em&gt;. Then use the &lt;code&gt;final_1.pdb&lt;/code&gt; and any other &lt;code&gt;final_*.pdb&lt;/code&gt; files as the output. If you want more structures, use the &lt;code&gt;ensemble.i.pdb&lt;/code&gt; files.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2019-09-17T11:13:42.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10425" PostTypeId="2" ParentId="3444" CreationDate="2019-09-17T11:29:50.523" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://github.com/haddocking/pdb-tools/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pdb-tools&lt;/a&gt; suite has &lt;code&gt;pdb_delinsertion&lt;/code&gt; and &lt;code&gt;pdb_reres&lt;/code&gt; for this task. Something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pdb_delinsertion 1CTF.pdb | pdb_reres &amp;gt; out.pdb&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;should work.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2019-09-17T11:29:50.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10426" PostTypeId="1" AcceptedAnswerId="10464" CreationDate="2019-09-17T12:20:19.247" Score="0" ViewCount="92" Body="&lt;p&gt;I have RNA-seq from two sequencing batches; Lab technician says that he has run the RNA expression quantification two times in bathes 1 and 2 for example tumor 1 in batch 1 and tumor 1 in batch 2 , normal 2 in batch1 and normal 2 in batch 2. PCA shows however shows two experimental runs are too close to each other and can be merged&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[![enter image description here][1]][1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my question is can I compare only &lt;code&gt;a tumour sample with its own matched normal sample in one patient&lt;/code&gt; to get differentially expressed genes for this &lt;code&gt;patient&lt;/code&gt; by using experimental runs (technical replicates )? Design would be &lt;code&gt;Tumour1_batch1&lt;/code&gt; &lt;code&gt;Tumour1_batch2&lt;/code&gt; &lt;strong&gt;&lt;em&gt;versus&lt;/em&gt;&lt;/strong&gt; &lt;code&gt;Normal1_batch1&lt;/code&gt; &lt;code&gt;Normal1_bacth2&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(mycols)&#xA;              Condition&#xA;Normal1_batch1  N&#xA;Normal1_batch2  N     &#xA;Tumour1_batch1  T     &#xA;Tumour1_batch2  T     &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-09-24T11:33:34.250" LastActivityDate="2019-09-24T11:33:34.250" Title="How I deal with this kind of gene expression comparision" Tags="|r|rna-seq|differential-expression|deseq2|design|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10427" PostTypeId="1" CreationDate="2019-09-17T13:36:54.817" Score="2" ViewCount="202" Body="&lt;p&gt;I'm trying to parse a Genbank file to find a specific feature. I can pull it out if I know the feature type (e.g. repeat_region). E.g. if I'm looking for this feature:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; repeat_region   5623..5756&#xA;                 /label=5' ITR&#xA;                 /note=&quot;5' ITR&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I know that I can find it using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for feature in reference.features:&#xA;    if feature.type == &quot;repeat_region&quot;:&#xA;        print(feature.location)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I don't trust that it will always be a repeat_region. Instead, I'd like to look for it by label (5' ITR). I can seem to find a way to parse that from the feature object. Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="6224" LastEditorUserId="6244" LastEditDate="2019-09-23T09:00:51.133" LastActivityDate="2019-09-23T09:00:51.133" Title="Parsing specific features from Genbank by label?" Tags="|python|genbank|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10428" PostTypeId="1" AcceptedAnswerId="10430" CreationDate="2019-09-18T00:07:10.757" Score="1" ViewCount="100" Body="&lt;p&gt;I am trying to get a reference together for imputation using the 1000 genomes data set. The GRCh38 version of the data set was made available earlier this year, but seems to be missing a significant number of SNPs on the X chromosome (~96%).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any resource that can explain why? I have tried searching on their home page with no luck.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The data sets in question are &lt;a href=&quot;http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000_genomes_project/release/20190312_biallelic_SNV_and_INDEL/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GRCh38&lt;/a&gt;, &lt;a href=&quot;http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GRCh37&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6225" LastEditorUserId="48" LastEditDate="2019-09-18T14:06:03.407" LastActivityDate="2020-06-19T13:12:27.813" Title="1000 genomes missing SNPs on chr X in GRCh38" Tags="|snp|impute2|1000genomes|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10429" PostTypeId="2" ParentId="10428" CreationDate="2019-09-18T09:05:32.987" Score="2" Body="&lt;p&gt;I think these are &lt;a href=&quot;http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000_genomes_project/release/20190312_biallelic_SNV_and_INDEL/20190312_biallelic_SNV_and_INDEL_README.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;biallelic snps&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The set is restricted to biallelic SNVs and INDELs.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;If you check the &lt;a href=&quot;http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/GRCh38_positions/&quot; rel=&quot;nofollow noreferrer&quot;&gt;liftover version&lt;/a&gt;, the chromosome X is complete...&lt;/p&gt;&#xA;" OwnerUserId="5077" LastEditorUserId="48" LastEditDate="2019-09-18T14:07:05.533" LastActivityDate="2019-09-18T14:07:05.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10430" PostTypeId="2" ParentId="10428" CreationDate="2019-09-18T13:10:11.887" Score="3" Body="&lt;p&gt;As I told you on 1000 genomes helpdesk:&lt;br /&gt;&#xA;Because of the ploidy settings we used for our variant calling, we only called variants in the the pseudoautosomal region on the X chromosome. We are working on settings to correctly analyse X variants in both males and females and plan to analyse X properly soon.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastEditorUserId="57" LastEditDate="2020-06-19T13:12:27.813" LastActivityDate="2020-06-19T13:12:27.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10431" PostTypeId="2" ParentId="10418" CreationDate="2019-09-18T15:33:27.857" Score="1" Body="&lt;p&gt;Carambakaracho's answer is good.  A high N50 should usually be a good sign, as well as agreement between the overall genome assembly length and the expected genome size.  The number of 'N's in the assembly may also be important to check, as Ns are sometimes used to indicate single unknown bases, and sometimes to indicate unknown numbers of unknown bases, and Ns may be used to connect scaffolds,which may lead to artificially higher N50.  If you have access to a set of genome sequence reads (Illumina) for the same organism, then another more detailed check you can make is to compare the kmer spectrum of the reads with the kmer spectrum of the assembly, using the kat tool.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, if you want to compare several assemblies, you might align them against each other, to see how they compare qualitatively (e.g. using mauve).&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-09-18T15:33:27.857" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10432" PostTypeId="2" ParentId="7340" CreationDate="2019-09-18T15:35:52.823" Score="0" Body="&lt;p&gt;I just recently discovered the ampliCan R package from the Valen lab after asking this question. I highly recommend it. It does all of what I was expecting &amp;amp; more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To install: &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/amplican.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/html/amplican.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To run (&amp;amp; other explanations): &lt;a href=&quot;https://rdrr.io/bioc/amplican/f/vignettes/amplicanOverview.Rmd&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rdrr.io/bioc/amplican/f/vignettes/amplicanOverview.Rmd&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the original paper &amp;amp; Do not forget to credit the people!&#xA;&lt;a href=&quot;https://genome.cshlp.org/content/early/2019/03/08/gr.244293.118.abstract&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genome.cshlp.org/content/early/2019/03/08/gr.244293.118.abstract&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="868" LastActivityDate="2019-09-18T15:35:52.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10433" PostTypeId="2" ParentId="10392" CreationDate="2019-09-18T15:39:11.087" Score="1" Body="&lt;p&gt;DPGP (McDowell et al, 2018) uses a non-parametric clustering method, so should find clusters without you needing to specify cluster size or number of clusters.  It is more or less explicitly designed for clustering timeseries gene expression data to find clusters of co-expressed genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005896&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1005896&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SplineCluster (Heard et al. 2006) uses a different method, but for a similar purpose and in a similarly non-parametric way, and worked well when we used it a few years ago with microarray data.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-09-18T15:39:11.087" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10434" PostTypeId="1" AcceptedAnswerId="10446" CreationDate="2019-09-18T19:07:21.717" Score="1" ViewCount="71" Body="&lt;p&gt;I am parsing a csv file of annotated sequences and using Biopython to generate Genbank files for each. I want to add annotations of the sequence features.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My output file shows features listed without the correct line breaks. Other software is then unable to parse the names of the features&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;misc_feature    58..60  /note=&quot;Oxidation&quot;&#xA;misc_feature    100..102  /note=&quot;Oxidation&quot;&#xA;misc_feature    241..243  /note=&quot;Oxidation&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It should be formatted like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;misc_feature    58..60  &#xA;                /note=&quot;Oxidation&quot;&#xA;misc_feature    100..102  &#xA;                /note=&quot;Oxidation&quot;&#xA;misc_feature    241..243  &#xA;                /note=&quot;Oxidation&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My section of code adds a space between the location and the /note=&quot;&quot; but the Biopython writer doesn't add line breaks.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in range(len(start_list)):&#xA;    feature = Feature()&#xA;# Strict Genbank format calls non-standard things misc_feature&#xA;    feature.key = &quot;misc_feature&quot;&#xA;    liab_type = &quot;{}&quot;.format(liability).split(&quot;:&quot;)[1]&#xA;#need to add 1 to start list since csv counting is off&#xA;    feature.location = &quot;{}..{}&quot;.format(int(start_list[i]) +1, end_list[i]) + &quot;  &quot; + '/note=\&quot;{}\&quot;'.format(liab_type)&#xA;    container.features.append(feature)&#xA;    i +=1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The Biopython source is toggling the wrapping of the feature qualifiers on &lt;code&gt;no_space_key&lt;/code&gt; and &lt;code&gt;Bio.GenBank._BaseGenBankConsumer.remove_space_keys&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I enable wrapping of text to format output correctly?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    def __str__(self):&#xA;    &quot;&quot;&quot;Return feature as a GenBank format string.&quot;&quot;&quot;&#xA;    output = Record.INTERNAL_FEATURE_FORMAT % self.key&#xA;    output += _wrapped_genbank(self.location, Record.GB_FEATURE_INDENT,&#xA;                               split_char=',')&#xA;    for qualifier in self.qualifiers:&#xA;        output += &quot; &quot; * Record.GB_FEATURE_INDENT&#xA;&#xA;        # determine whether we can wrap on spaces&#xA;        space_wrap = 1&#xA;        for no_space_key in \&#xA;                Bio.GenBank._BaseGenBankConsumer.remove_space_keys:&#xA;            if no_space_key in qualifier.key:&#xA;                space_wrap = 0&#xA;&#xA;        output += _wrapped_genbank(qualifier.key + qualifier.value,&#xA;                                   Record.GB_FEATURE_INDENT, space_wrap)&#xA;    return output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6230" LastEditorUserId="48" LastEditDate="2019-09-21T16:42:21.013" LastActivityDate="2019-09-21T16:42:48.730" Title="Biopython Genbank writer not splitting long lines" Tags="|python|genbank|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10435" PostTypeId="1" AcceptedAnswerId="10436" CreationDate="2019-09-19T08:37:57.097" Score="0" ViewCount="65" Body="&lt;p&gt;I am trying to perform de novo structure prediction of a 1135 long protein sequence; can DMPfold and I-Tasser work well for de novo structure prediction of 1135 amino acid long sequence?&lt;/p&gt;&#xA;" OwnerUserId="5180" LastEditorUserId="5180" LastEditDate="2019-09-19T09:06:09.770" LastActivityDate="2019-09-26T15:53:45.230" Title="Can DMPfold and I-Tasser work well for de novo structure prediction of 1135 amino acid long sequence?" Tags="|protein-structure|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10436" PostTypeId="2" ParentId="10435" CreationDate="2019-09-19T10:00:54.817" Score="1" Body="&lt;p&gt;For DMPfold you might like to look at Figure 5B of &lt;a href=&quot;https://www.nature.com/articles/s41467-019-11994-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper&lt;/a&gt;, which implies that performance starts to drop off after ~600 residues. Only proteins up to 500 residues were used in the training set, for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A protein of that length is almost certainly multi-domain. I would consider doing domain prediction; we have a tool &lt;a href=&quot;http://bioinf.cs.ucl.ac.uk/psipred&quot; rel=&quot;nofollow noreferrer&quot;&gt;DomPred&lt;/a&gt;, but many others are available. Then run the sequences of the domains individually and recombine them later.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know about I-Tasser's ability to do this, perhaps see if they mention it in their papers.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2019-09-19T10:00:54.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10437" PostTypeId="1" CreationDate="2019-09-19T11:37:42.930" Score="0" ViewCount="1819" Body="&lt;p&gt;By EnhancedVolcano R package I have this plot with this code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    &amp;gt; head(res1)&#xA;         baseMean log2FoldChange     lfcSE        stat     pvalue      padj&#xA;SAMD11   237.7975     0.03584334 1.3244737  0.02706233 0.97841002 0.9971691&#xA;NOC2L   4751.0381    -0.62090622 0.6500115 -0.95522347 0.33946470 0.9239759&#xA;KLHL17   735.0849    -0.95126291 0.8417444 -1.13010892 0.25843033 0.8926461&#xA;PLEKHN1 1117.5598    -2.11321359 1.0411108 -2.02976816 0.04238011 0.6350143&#xA;PERM1    409.0061    -2.32444927 1.1255813 -2.06511000 0.03891259 0.6285916&#xA;HES4    1088.7549     0.28395931 0.9091259  0.31234320 0.75477971 0.9866881&#xA;&amp;gt;&#xA;p= EnhancedVolcano(res1,&#xA;    lab = rownames(res1),&#xA;    x = 'log2FoldChange',&#xA;    y = 'pvalue',&#xA;    xlim = c(-5, 8))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have stored my plot in an object&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; class(p)&#xA;[1] &quot;gg&quot;     &quot;ggplot&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/X32hE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/X32hE.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to limit the axis length in this plot in the other word shrining my plot more but I don't know how to do that. &lt;code&gt;For example y axis be from 0 to 10 instead of 20&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help please &lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="3541" LastEditDate="2019-09-21T14:39:01.260" LastActivityDate="2019-09-21T14:39:01.260" Title="Changing the axis limits of ggplot objects" Tags="|r|rna-seq|ggplot2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10438" PostTypeId="2" ParentId="10437" CreationDate="2019-09-19T12:18:06.333" Score="4" Body="&lt;p&gt;Axes for &lt;code&gt;ggplot&lt;/code&gt;s can be controlled via &lt;code&gt;xlim()&lt;/code&gt; and &lt;code&gt;ylim()&lt;/code&gt; functions. You can try:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;your_ggplot_object +&#xA;  ylim(-5,20) # assuming that you want the gene labels to be readabe, not clear from your post&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-09-19T14:46:12.947" LastActivityDate="2019-09-19T14:46:12.947" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10439" PostTypeId="1" AcceptedAnswerId="10440" CreationDate="2019-09-19T13:20:01.433" Score="3" ViewCount="106" Body="&lt;p&gt;How many models should be generated by rosetta for de novo protein structure prediction of a 382 long sequence? In &lt;a href=&quot;https://www.rosettacommons.org/demos/latest/tutorials/denovo_structure_prediction/Denovo_structure_prediction&quot; rel=&quot;nofollow noreferrer&quot;&gt;the tutorial&lt;/a&gt;, it says 50,000 models but practically it takes a lot of time (weeks) to generate this number. For instance, I am currently running rosetta on a 382 long sequence and in a week it has generated 1500 models; I am using 16 cpus with Openmpi.&lt;/p&gt;&#xA;" OwnerUserId="5180" LastEditorUserId="6322" LastEditDate="2020-07-01T09:11:50.733" LastActivityDate="2020-07-01T09:11:50.733" Title="How many models should be generated by rosetta for denovo protein structure prediction?" Tags="|protein-structure|computational-biochemistry|rosetta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10440" PostTypeId="2" ParentId="10439" CreationDate="2019-09-19T21:15:43.180" Score="2" Body="&lt;p&gt;Unfortunately you do in general increase your chance of success by going to the 50k range of models or higher. Projects like &lt;a href=&quot;https://boinc.bakerlab.org/rosetta&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rosetta@home&lt;/a&gt; exist because even large research groups can't access as much compute power as they'd like. In this case the long time arises from the combinatorial explosion of possible fragment combinations that need to be explored.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Success is rather protein dependent though - sometimes the right structure falls out after only a few models, other times your fragment library does not cover the protein adequately and you could run Rosetta forever without getting a good structure. Have a look at your 1,500 models and see how similar they look.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;382 residues is rather long for Rosetta. Another factor to consider is whether you are using predicted contacts as part of the folding process (e.g. in the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/25161237&quot; rel=&quot;nofollow noreferrer&quot;&gt;PconsFold&lt;/a&gt; protocol), which will greatly improve the chances of getting a good model and reduce the number of models you need. In fact the PconsFold group switched to CONFOLD for PconsFold2, I think mainly because it was much faster, so you are not alone in finding Rosetta runs long.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2019-09-19T21:15:43.180" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10441" PostTypeId="1" AcceptedAnswerId="19289" CreationDate="2019-09-20T09:56:44.257" Score="1" ViewCount="287" Body="&lt;p&gt;In &lt;a href=&quot;https://www.cell.com/neuron/pdf/S0896-6273(16)30074-5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; I am citing the lines below:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Intra-modular connectivity of resilient and susceptible network modules was compared to that of corresponding genes in control mice to identify gain, loss, or no change.Pie charts (lower panel) summarize module differential connectivity (MDC) analysis. Proportionally, very few resilient modules had significant MDC compared to more than half of all susceptible modules, which predominantly showed gain of connectivity&lt;/p&gt;&#xA;&lt;p&gt;A module differential connectivity (MDC) metric quantified differences in coexpression networks to identify phenotype-specific modules&#xA;(Zhang et al., 2013)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Need to clarify whether this &amp;quot;module differential connectivity (MDC)&amp;quot; analysis is a part of WGCNA or its an independent analysis. Because if I do a search for &lt;code&gt;module differential connectivity(MDC)analysis&lt;/code&gt; the result I get is this `DGCA: Differential Gene Correlation Analysis'. But no where in the paper they have mentioned &amp;quot;DGCA&amp;quot; library in the citation.&lt;/p&gt;&#xA;&lt;p&gt;It would be helpful if i get some suggestion regarding this analysis.&lt;/p&gt;&#xA;&lt;p&gt;This is the paper:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3677161/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3677161/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="73" LastEditDate="2022-06-21T10:48:57.310" LastActivityDate="2022-06-21T10:49:43.897" Title="module differential connectivity (MDC) analysis" Tags="|r|wgcna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10442" PostTypeId="1" AcceptedAnswerId="10514" CreationDate="2019-09-20T09:34:41.880" Score="3" ViewCount="1007" Body="&lt;p&gt;I'm new to PyMol (and StackExchange!) and working on my first project. I have the structure of a protein as an object, called PolyA-M, and the idea is that the residues of it are shown as spheres of differing sizes based on a calcualted conservation value. I want to label each sphere with it's amino acid, but to have the label size corresponding to sphere size. Here is a sample of the code that might be used to label one amino acid.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;alter ( resid 138 ), resn = &quot;Q135&quot;&#xA;alter ( name CB and resid 138 ), vdw = vdw * 0.8 * 0.679934640522875&#xA;set_color col138, [ 1.0, 0.181, 0.181 ]&#xA;show spheres, name CB and resi 138&#xA;show sticks, ( name CA or name CB ) and resid 138&#xA;set label_size, 14, PolyA-M&#xA;label PolyA-M and name CB and resid 138, &quot;Q&quot;&#xA;color col138, name CB and resi 138&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this case, residue 138 is medium sized sphere with a size 14 label Q that fits well. However, if later on in the code another residue is labelled as so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;alter ( resid 198 ), resn = &quot;A208&quot;&#xA;alter ( name CB and resid 198 ), vdw = vdw * 0.8 * 0.359803921568627&#xA;set_color col198, [ 1.0, 0.601, 0.601 ]&#xA;show spheres, name CB and resi 198&#xA;show sticks, ( name CA or name CB ) and resid 198&#xA;set label_size,  8, PolyA-M&#xA;label PolyA-M and name CB and resid 198, &quot;A&quot;&#xA;color col198, name CB and resi 198&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will label residue 198 with a small A to fit the small sphere. However, this will also change the earlier label to size 8 as well, making it too small for the sphere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way for me to prevent this, such that each residue keeps its own label size? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that one way is to create multiple identical objects and keep all labels of the same size restricted to the same object (For example, PolyA-M_10 contains all residues with label size 10, polyA-M_14 all residues with label size 14) but I was wondeirng if there is a more efficient way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for any help! This is my first Stack Exchange question so any feedback would be appreciated :) &lt;/p&gt;&#xA;" OwnerUserId="6239" OwnerDisplayName="A. King" LastActivityDate="2019-09-30T16:05:56.563" Title="Putting labels of different sizes on one PyMOl Object" Tags="|protein-structure|structural-biology|pymol|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10443" PostTypeId="2" ParentId="10442" CreationDate="2019-09-20T10:25:53.937" Score="2" Body="&lt;p&gt;'label_size' is a object-state-level setting, which means that you will have to rely on the 'create' command to create a new object for every different label size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It can certainly be automated, and for this I'd recommend getting into &lt;a href=&quot;https://pymolwiki.org/index.php/Simple_Scripting&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pyhton Scripting for PyMOL&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6240" LastActivityDate="2019-09-20T10:25:53.937" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10444" PostTypeId="2" ParentId="10427" CreationDate="2019-09-20T12:33:30.573" Score="1" Body="&lt;p&gt;The &lt;code&gt;/label=5'ITR&lt;/code&gt; is called a &lt;strong&gt;qualifier&lt;/strong&gt;. You can look for those qualifiers:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for feature in reference.features:&#xA;    for k, v in feature.qualifiers.items():&#xA;        if k == &quot;label&quot; and v == [&quot;5' ITR&quot;]:  # or &quot;5' ITR&quot; in v&#xA;            print(feature.location)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6244" LastEditorUserId="6244" LastEditDate="2019-09-21T12:23:30.507" LastActivityDate="2019-09-21T12:23:30.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10445" PostTypeId="1" CreationDate="2019-09-20T20:38:44.677" Score="0" ViewCount="382" Body="&lt;p&gt;Is it possible to create a Heatmap in Seurat that takes in and displays modules of genes as its features rather than individual genes?  So if I have a module of genes associated with a trait or phenotype I can compare the expression of that module across clusters against the expression of other modules of genes.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently the way that I am trying to do it is similar to the code below (In this example I am only using one module to test if it works):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Module_28 &amp;lt;- readLines(con = &quot;Path_to_file/module28.txt&quot;)&#xA;Module_28 &amp;lt;- list(Module_28)&#xA;Combined &amp;lt;- AddModuleScore(Combined, features = Module_28, ctrl = 5, name = 'Module_28')&#xA;names(x = Combined[[]])&#xA;avg &amp;lt;- AverageExpression(Combined, features = NULL, add.ident = NULL, return.seurat = TRUE, verbose = TRUE)&#xA;DoHeatmap(avg, features = c(&quot;Module_281&quot;), group.by = &quot;ident&quot;, size = 3, angle = 0, draw.lines = FALSE) + theme(axis.text.x = element_text(size = 7, angle = 45), axis.line = element_line(color = &quot;#ffffff&quot;)) + scale_fill_gradient2(low = '#1000ff', mid = &quot;#ffffff&quot;, hight = '#aa0101', space = &quot;Lab&quot;, na.value = &quot;#ffffff&quot;, midpoint = 0, guide = &quot;colourbar&quot;, aesthetics = &quot;fill&quot;) + coord_flip(expand = TRUE, clip = &quot;on&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Often When I try to run the DoHeatmap function I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in DoHeatmap(avg, features = c(&quot;Module_281&quot;), group.by = &quot;ident&quot;,  : &#xA;  No requested features found in the scale.data slot for the RNA assay.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5134" LastEditorUserId="5134" LastEditDate="2019-10-03T21:46:07.740" LastActivityDate="2019-10-03T21:46:07.740" Title="Create a Heatmap that uses modules of genes rather than individual genes in Seurat" Tags="|r|seurat|gene-expression|heatmap|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10446" PostTypeId="2" ParentId="10434" CreationDate="2019-09-21T16:28:36.447" Score="1" Body="&lt;p&gt;I read the Biopython Genbank record source code carefully and realized I wasn’t using the Qualifier class correctly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;After reading the source code again, I realized I was using the data structure incorrectly.  I need to use the Qualifier class to add a key and feature.  This is formatted correctly and solves the problem.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;feature = Feature()&#xA;# Strict Genbank format calls non-standard things misc_feature&#xA;feature.key = &quot;misc_feature&quot;&#xA;#need to add 1 to start list since csv counting is off&#xA;feature.location = &quot;{}..{}&quot;.format(int(start_list[i]) +1, end_list[i])&#xA;liability_qualifier = Qualifier()&#xA;liability_qualifier.key = '/note=&quot;'&#xA;liability_qualifier.value = '\&quot;{}\&quot;'.format(liability).split(&quot;:&quot;)[1]&#xA;feature.qualifiers.append(liability_qualifier)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6230" LastEditorUserId="48" LastEditDate="2019-09-21T16:42:48.730" LastActivityDate="2019-09-21T16:42:48.730" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10447" PostTypeId="1" CreationDate="2019-09-22T14:43:27.337" Score="2" ViewCount="1169" Body="&lt;p&gt;I need a Software or Python Program for Converting VCF(.vcf) to  FASTA(.fa) format with the help of reference (.fa)&lt;/p&gt;&#xA;" OwnerUserId="6249" LastEditorUserId="298" LastEditDate="2019-09-22T16:59:34.760" LastActivityDate="2019-09-22T20:11:09.713" Title="How to convert VCF format file into Fasta(.fa) with the Reference data" Tags="|fasta|sequence-analysis|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10448" PostTypeId="2" ParentId="10447" CreationDate="2019-09-22T15:58:25.653" Score="2" Body="&lt;p&gt;I used &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/tooldocs/3.8-0/org_broadinstitute_gatk_tools_walkers_fasta_FastaAlternateReferenceMaker.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;FastaAlternateReferenceMaker&lt;/a&gt; previously. You will need to download &lt;a href=&quot;https://software.broadinstitute.org/gatk/download/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATK&lt;/a&gt; first. Next you prepare your reference genome (reference.fasta) and your vcf file (input.vcf). You will call the GATK function like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; java -jar GenomeAnalysisTK.jar \&#xA;   -T FastaAlternateReferenceMaker \&#xA;   -R reference.fasta \&#xA;   -o output.fasta \&#xA;   -V input.vcf \&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;output.fasta will contain the new fasta with snps inserted at sites specific by the vcf file&lt;/p&gt;&#xA;" OwnerUserId="5077" LastEditorUserId="5077" LastEditDate="2019-09-22T20:11:09.713" LastActivityDate="2019-09-22T20:11:09.713" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10449" PostTypeId="1" CreationDate="2019-09-23T04:16:51.777" Score="2" ViewCount="1512" Body="&lt;p&gt;I want to do comparative analyses of my two individual single cell RNA-Seq datasets in Seurat. And I load and list my data as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Control.data &amp;lt;- Read10X(&quot;~/Documents/...  Control_filtered_feature_bc_matrix&quot;)&#xA;&#xA;Control &amp;lt;- CreateSeuratObject(counts  = Control.data, project = &quot;Control_summary&quot;, min.cells = 3, min.features = 200)&#xA;&#xA;Mut.data &amp;lt;- Read10X(&quot;~/Documents/... /Mut_filtered_feature_bc_matrix&quot;)&#xA;&#xA;Mut &amp;lt;- CreateSeuratObject(counts  = Mut.data, project = &quot;Mut_summary&quot;, min.cells = 3, min.features = 200)&#xA;&#xA;Object&amp;lt;- list(Control,Mut)&#xA;&#xA;names(Object) &amp;lt;- colnames(Object)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then the protocol &lt;a href=&quot;https://satijalab.org/seurat/v3.1/immune_alignment.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/v3.1/immune_alignment.html&lt;/a&gt; said, I need to use &lt;code&gt;SplitObject&lt;/code&gt; function, but what do I split by?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Object_list &amp;lt;- SplitObject(Object, split.by = &quot;?&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6252" LastEditorUserId="6252" LastEditDate="2019-09-23T16:40:13.830" LastActivityDate="2019-09-23T16:40:13.830" Title="How to load and split my two individual datasets when integrating datasets in Seurat?" Tags="|r|scrnaseq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10450" PostTypeId="2" ParentId="10449" CreationDate="2019-09-23T07:53:00.440" Score="4" Body="&lt;p&gt;&lt;code&gt;Seurat&lt;/code&gt; is expecting individual datasets to be normalized separately prior to data integration. In that respect lists of objects corresponding to different datasets are handy to manipulate each object/dataset individually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your case you do not need to do any splitting, you have an object corresponding to each of your datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would have started with a single count matrix of all datasets then you would need to split your data based on batches/experimental condition/...&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-09-23T07:53:00.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10451" PostTypeId="1" AcceptedAnswerId="10452" CreationDate="2019-09-23T11:07:50.927" Score="1" ViewCount="30" Body="&lt;p&gt;I am running dmpfold, &quot;seq2maps.sh&quot; and from a few days the program is at the &quot;PSICOV&quot; stage:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;psicov -z 15 -o -d 0.03 tail-fiber-sequence.aln&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&quot;ps aux&quot; gives me this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;daniel     26693  490  2.5 2546588 2514320 ?     Rl   سبت20 25116:58 psicov -z 15 -o -d 0.03 tail-fiber-sequence.aln&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any idea on what could be the matter? Yes, I am runinng &quot;rosetta&quot; too on my 32 CPUs, so all my CPUs are covered to max 100%; howver, RAM is 93Gb and only 32 GB has been used. Thanks&lt;/p&gt;&#xA;" OwnerUserId="5180" LastEditorUserId="5180" LastEditDate="2019-09-23T11:21:54.027" LastActivityDate="2019-09-23T11:55:57.763" Title="A program &quot;PSICOV&quot; &quot;stopped&quot; from a few days on Ubuntu 18.04?" Tags="|protein-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10452" PostTypeId="2" ParentId="10451" CreationDate="2019-09-23T11:50:22.370" Score="0" Body="&lt;p&gt;In summary just be patient. psicov doesn't  parrallelise so a tenfold increase in CPU will not hurry it along.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also psicov is not RAM intensive given the machine you are describing, so you haven't hit a RAM bottleneck. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know the ps output format you are using I'm use to seeing %CPU and %MEM from &lt;code&gt;ps -aux&lt;/code&gt; on Susie and OSX. You should also try,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;top -c -u username&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or for OSX &lt;code&gt;top -u&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should see the psicov happily chuntering away consuming ~100% of a single process. There is a UNIX/Linux guru specialist on BioSE who could almost certainly provide better diagnostics.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-09-23T11:55:57.763" LastActivityDate="2019-09-23T11:55:57.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10453" PostTypeId="1" CreationDate="2019-09-23T12:20:42.707" Score="1" ViewCount="133" Body="&lt;p&gt;I am running rosetta ab initio prediction using the command line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;mpiexec -np 16 /main/source/bin/AbinitioRelax.mpi.linuxgccrelease -database /main/database/ @options -mpi_tracer_to_file log &amp;gt; log1.log &amp;amp;&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&quot;Options&quot; file looks like this: &lt;a href=&quot;https://ibb.co/V9VrQHV&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ibb.co/V9VrQHV&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RMSD vs Energy plot is:&#xA;&lt;a href=&quot;https://ibb.co/1mjzwDj&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ibb.co/1mjzwDj&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is no native structure as this is an ab initio prediction. Could you please let me know about your comments on the RMSD being 0.00 for all 2000 models generated till now? Is this normal and will produce results after more runs? &lt;/p&gt;&#xA;" OwnerUserId="5180" LastActivityDate="2019-09-23T15:04:26.883" Title="Rosetta ab initio run gives 0.00 RMSD for generated 2000 models? (run still in progress)" Tags="|protein-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10454" PostTypeId="1" AcceptedAnswerId="10455" CreationDate="2019-09-23T12:42:29.467" Score="1" ViewCount="213" Body="&lt;p&gt;In &lt;code&gt;f1&lt;/code&gt; below I am trying to split &lt;code&gt;$1&lt;/code&gt; based on each line and create a bed file. If the line is a snp then the &lt;code&gt;:&lt;/code&gt; is spilt and the text is &lt;code&gt;$1&lt;/code&gt; the last digit is &lt;code&gt;$2&lt;/code&gt; -1 and &lt;code&gt;$3&lt;/code&gt; and the letter to the left of the &lt;code&gt;&amp;gt;&lt;/code&gt; is &lt;code&gt;$4&lt;/code&gt; and the letter to the right is &lt;code&gt;$5&lt;/code&gt;. (lines 1 and 2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the line has &lt;code&gt;dup&lt;/code&gt; then the &lt;code&gt;:&lt;/code&gt; is spilt and the text is &lt;code&gt;$1&lt;/code&gt; the last digit is &lt;code&gt;$2&lt;/code&gt; and &lt;code&gt;$3&lt;/code&gt; and the &lt;code&gt;$4&lt;/code&gt; is &lt;code&gt;-&lt;/code&gt; and the letter to the right is &lt;code&gt;$5&lt;/code&gt;. (line 4)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the line has &lt;code&gt;del&lt;/code&gt; with nothing else then the &lt;code&gt;:&lt;/code&gt; is spilt and the text is &lt;code&gt;$1&lt;/code&gt; the last digit is &lt;code&gt;$2&lt;/code&gt; and &lt;code&gt;$3&lt;/code&gt; and the letter to the left of the &lt;code&gt;&amp;gt;&lt;/code&gt; is &lt;code&gt;$4&lt;/code&gt; and &lt;code&gt;$5&lt;/code&gt; is &lt;code&gt;-&lt;/code&gt;. (line 5)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure how to format for line 3 (the complex indel)... maybe split &lt;code&gt;$1&lt;/code&gt; on the &lt;code&gt;:&lt;/code&gt; and the text is &lt;code&gt;$1&lt;/code&gt; the last digit is &lt;code&gt;$2&lt;/code&gt; and &lt;code&gt;$3&lt;/code&gt; and the letters to the left of the &lt;code&gt;&amp;gt;&lt;/code&gt; is &lt;code&gt;$4&lt;/code&gt; and the letters to the right is &lt;code&gt;$5&lt;/code&gt;. Thank you :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;f1&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr7:140453145A&amp;gt;T&#xA;chr7:140453136A&amp;gt;T &#xA;chr7:140453135_140453136delCAinsTT&#xA;chr20:31022287dupA&#xA;chr19:13054614delG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;desired&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr7    140453144   140453145   A   T &#xA;chr7    140453135   140453136   A   T &#xA;chr7    140453135   140453136   TT  CA&#xA;chr20   31022287    31022287    -   A&#xA;chr19   13054614    13054614    G   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;awk&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN{OFS=&quot;\t&quot;} {sub(/[^0-9]+&lt;span class=&quot;math-container&quot;&gt;$/, &quot;&quot;, $&lt;/span&gt;1)}' f1&#xA;&#xA;awk 'BEGIN { FS = &quot;[ -]&quot;; OFS=&quot;\t&quot; } NF==3 { print &lt;span class=&quot;math-container&quot;&gt;$1, $&lt;/span&gt;2 - 1, &lt;span class=&quot;math-container&quot;&gt;$2, $&lt;/span&gt;3 } NF==4 { print &lt;span class=&quot;math-container&quot;&gt;$1, $&lt;/span&gt;2, &lt;span class=&quot;math-container&quot;&gt;$3, $&lt;/span&gt;4 }' f1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4424" LastEditorUserId="4424" LastEditDate="2019-09-23T12:57:24.123" LastActivityDate="2019-09-24T20:03:03.917" Title="split 1 column input into 5 column bed file" Tags="|awk|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10455" PostTypeId="2" ParentId="10454" CreationDate="2019-09-23T13:44:53.047" Score="4" Body="&lt;p&gt;The following Python script seems to do the job.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;from __future__ import print_function&#xA;import re&#xA;import sys&#xA;&#xA;for line in sys.stdin:&#xA;    if '_' in line:&#xA;        match = re.search(r'(\S+):(\d+)_(\d+)del([ACGT]+)ins([ACGT]+)', line)&#xA;        print(match.group(1), match.group(2), match.group(3), match.group(5), match.group(4), sep='\t')&#xA;    elif 'dup' in line or 'del' in line:&#xA;        match = re.search(r'(\S+):(\d+)(del|dup)([ACGT]+)', line)&#xA;        if match.group(3) == 'del':&#xA;            print(match.group(1), match.group(2), match.group(2), match.group(4), '-', sep='\t')&#xA;        else:&#xA;            print(match.group(1), match.group(2), match.group(2), '-', match.group(4), sep='\t')&#xA;    else:&#xA;        match = re.search(r'(\S+):(\d+)([ACGT]+)&amp;gt;([ACGT]+)', line)&#xA;        coord = int(match.group(2))&#xA;        print(match.group(1), coord - 1, coord, match.group(3), match.group(4), sep='\t')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Invoked like so on the command line.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[standage@lappy ~] $ ./transform &amp;lt; f1&#xA;chr7 140453144 140453145 A T&#xA;chr7 140453135 140453136 A T&#xA;chr7    140453135   140453136   TT  CA&#xA;chr20   31022287    31022287    -   A&#xA;chr19   13054614    13054614    G   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2019-09-23T14:53:14.390" LastActivityDate="2019-09-23T14:53:14.390" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10456" PostTypeId="2" ParentId="10454" CreationDate="2019-09-23T14:47:03.910" Score="2" Body="&lt;p&gt;Here is an &lt;code&gt;R&lt;/code&gt; solution, would probably be slower than the suggested &lt;code&gt;Python&lt;/code&gt; solution though:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bed &amp;lt;- readLines(&quot;bed.txt&quot;)&#xA;&#xA;res &amp;lt;- data.frame()&#xA;&#xA;for(i in bed) {&#xA;&#xA;  temp &amp;lt;- strsplit(i, &quot;:&quot;) %&amp;gt;% unlist()&#xA;&#xA;  # case N&amp;gt;N&#xA;&#xA;  if(grepl(&quot;&amp;gt;&quot;, temp[[2]])) {&#xA;&#xA;    begin &amp;lt;- str_extract(temp[[2]], pattern = &quot;^[:digit:]+&quot;) %&amp;gt;% as.integer() - 1&#xA;    end &amp;lt;- begin + 1&#xA;    ref &amp;lt;- str_extract(temp[[2]], pattern = &quot;[A-Z]&quot;)&#xA;    alt &amp;lt;- str_extract(temp[[2]], pattern = &quot;[A-Z]$&quot;)&#xA;&#xA;    temp_res &amp;lt;- data.frame(temp[[1]], begin, end, ref, alt)&#xA;    res &amp;lt;- rbind(res, temp_res)&#xA;  }&#xA;&#xA;  # case del_ins&#xA;&#xA;  if(grepl(&quot;ins&quot;, temp[[2]])) {&#xA;&#xA;    begin &amp;lt;- str_extract(temp[[2]], pattern = &quot;^[:digit:]+&quot;) %&amp;gt;% as.integer()&#xA;    end &amp;lt;- str_extract(temp[[2]], pattern = &quot;_[:digit:]+&quot;)&#xA;    end &amp;lt;- substring(end, 2, nchar(end)) %&amp;gt;% as.integer()&#xA;    ref &amp;lt;- str_extract(temp[[2]], pattern = &quot;[A-Z]+$&quot;)&#xA;    alt &amp;lt;- str_extract(temp[[2]], pattern = &quot;[A-Z]+&quot;)&#xA;&#xA;    temp_res &amp;lt;- data.frame(temp[[1]], begin, end, ref, alt)&#xA;    res &amp;lt;- rbind(res, temp_res)&#xA;  }&#xA;&#xA;&#xA;  # case dup&#xA;&#xA;  if(grepl(&quot;dup&quot;, temp[[2]])) {&#xA;&#xA;    begin &amp;lt;- str_extract(temp[[2]], pattern = &quot;^[:digit:]+&quot;) %&amp;gt;% as.integer()&#xA;    end &amp;lt;- begin&#xA;    ref &amp;lt;- &quot;-&quot;&#xA;    alt &amp;lt;- str_extract(temp[[2]], pattern = &quot;[A-Z]$&quot;)&#xA;&#xA;    temp_res &amp;lt;- data.frame(temp[[1]], begin, end, ref, alt)&#xA;    res &amp;lt;- rbind(res, temp_res)&#xA;  }&#xA;&#xA;  # case del only&#xA;&#xA;  if(grepl(&quot;del&quot;, temp[[2]]) &amp;amp; !grepl(&quot;ins&quot;, temp[[2]])) {&#xA;&#xA;    begin &amp;lt;- str_extract(temp[[2]], pattern = &quot;^[:digit:]+&quot;) %&amp;gt;% as.integer()&#xA;    end &amp;lt;- begin&#xA;    ref &amp;lt;- str_extract(temp[[2]], pattern = &quot;[A-Z]$&quot;)&#xA;    alt &amp;lt;- &quot;-&quot;&#xA;&#xA;    temp_res &amp;lt;- data.frame(temp[[1]], begin, end, ref, alt)&#xA;    res &amp;lt;- rbind(res, temp_res)&#xA;  }&#xA;&#xA;}&#xA;&#xA;print(res)&#xA;&#xA;  temp..1..     begin       end ref alt&#xA;1      chr7 140453144 140453145   A   T&#xA;2      chr7 140453135 140453136   A   T&#xA;3      chr7 140453135 140453136  TT  CA&#xA;4     chr20  31022287  31022287   -   A&#xA;5     chr19  13054614  13054614   G   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-09-23T14:47:03.910" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10457" PostTypeId="2" ParentId="10453" CreationDate="2019-09-23T15:04:26.883" Score="1" Body="&lt;p&gt;RMSD is a measure of similarity between two structures. The relevant RMSD in protein structure prediction is between a model and the native structure. If you don't have a native structure you cannot have an RMSD in this sense. Ignore the fact it is 0.0, there is possibly some error that means you are calculating the RMSD between a model and itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And that's the difficulty of protein structure prediction... you don't know whether you are right or not! The structures you have generated are your results, and you'll need to find ways to filter or use them that are not reliant on RMSD. You could use estimation of model accuracy methods or clustering, for example.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2019-09-23T15:04:26.883" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10458" PostTypeId="2" ParentId="10454" CreationDate="2019-09-23T15:13:03.327" Score="3" Body="&lt;p&gt;The Python script in my first response is heavy on regex matching, which is pretty clunky in Python. I like Python much better than Perl overall, but a throwaway script like this will be clearer and more concise in Perl.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env perl&#xA;use strict;&#xA;&#xA;while(&amp;lt;STDIN&amp;gt;) {&#xA;    if (m/(\S+):(\d+)_(\d+)del([ACGT]+)ins([ACGT]+)/) {&#xA;        print(&quot;&lt;span class=&quot;math-container&quot;&gt;$1\t$&lt;/span&gt;2\t&lt;span class=&quot;math-container&quot;&gt;$3\t$&lt;/span&gt;5\t&lt;span class=&quot;math-container&quot;&gt;$4\n&quot;);&#xA;    }&#xA;    elsif (m/(\S+):(\d+)(del|dup)([ACGT]+)/) {&#xA;        if ($3 eq &quot;del&quot;) {&#xA;            print(&quot;$1\t$2\t$2\t$4\t-\n&quot;);&#xA;        }&#xA;        else {&#xA;            print(&quot;$1\t$2\t$2\t-\t$4\n&quot;);&#xA;        }&#xA;    }&#xA;    elsif (m/(\S+):(\d+)([ACGT]+)&amp;gt;([ACGT]+)/) {&#xA;        my $coord2 = $2;&#xA;        my $coord1 = $2 - 1;&#xA;        print(&quot;$1\t$coord1\t$coord2\t$3\t$&lt;/span&gt;4\n&quot;)&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It's been several years since I wrote Perl on a regular basis, so there are probably ways to make the script even more clear and concise. (Although I have a love/hate relationship with the default/hidden variables, which can make a script more concise but also harder for a newcomer to understand.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Invoked like so on the command line.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[standage@lappy ~] $ ./transform &amp;lt; f1&#xA;chr7 140453144 140453145 A T&#xA;chr7 140453135 140453136 A T&#xA;chr7    140453135   140453136   TT  CA&#xA;chr20   31022287    31022287    -   A&#xA;chr19   13054614    13054614    G   -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-09-23T15:13:03.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10459" PostTypeId="1" AcceptedAnswerId="10460" CreationDate="2019-09-23T19:00:29.233" Score="3" ViewCount="2225" Body="&lt;p&gt;I have a long list of variant IDs that I generated as a result of research done one GRCh37 genomes (e.g. &lt;code&gt;13_28025615_G_C_b37&lt;/code&gt;). I want to get their rsIDs and compare them to a newly-obtained list of GRCh38 rsIDs that have also been retrieved from similarly formatted variant IDs (e.g. &lt;code&gt;chr22_50473581_G_A_b38&lt;/code&gt;). How would I go about doing this? I've tried installing &lt;code&gt;biomaRt&lt;/code&gt; as a package but I can't quite make heads or tails of it and it's not clear to me that it would support both versions of genome builds. &lt;/p&gt;&#xA;" OwnerUserId="3442" LastEditorUserId="298" LastEditDate="2019-09-24T08:13:17.630" LastActivityDate="2020-08-22T17:44:50.183" Title="Find rsIDs for GRCh37 SNPs and rsIDs for GRCh38 SNPs and compare for overlap" Tags="|variation|biomart|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10460" PostTypeId="2" ParentId="10459" CreationDate="2019-09-23T19:15:55.477" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;it's not clear to me that it would support both versions of genome&#xA;  builds.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You can specify what version of the database you are querying.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd start with biomart on the ensembl website, see if you can get it to do what you want, then work on getting biomaRt to do what you want.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-09-23T19:15:55.477" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10461" PostTypeId="2" ParentId="10454" CreationDate="2019-09-23T22:42:47.663" Score="3" Body="&lt;p&gt;Quick Perl solution (following on from Daniel's answer saying that this would be &quot;clearer and more concise in Perl&quot;). I was a bit confused with your BED output because you've treated the start/end locations of a single base deletion and insertion different to a SNP. In my example here, the start and end locations for those cases are the same to distinguish from the two-base change for TT-&gt;CA [this can be changed]:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat 10454.txt | \&#xA;perl -ne 'if(/^(.*?):([0-9]+)(_([0-9]+))?(del|dup)?([ACGT]+)((ins|&amp;gt;)([ACGT]+))?/){&#xA;   print(join(&quot;\t&quot;,&#xA;         &lt;span class=&quot;math-container&quot;&gt;$1,&#xA;         $&lt;/span&gt;2,&#xA;         &lt;span class=&quot;math-container&quot;&gt;$4?$&lt;/span&gt;4:(&lt;span class=&quot;math-container&quot;&gt;$2),&#xA;         ($&lt;/span&gt;8 eq &quot;ins&quot;)?&lt;span class=&quot;math-container&quot;&gt;$9:($&lt;/span&gt;5 eq &quot;dup&quot;)?&quot;-&quot;:&lt;span class=&quot;math-container&quot;&gt;$6,&#xA;         (($&lt;/span&gt;5 eq &quot;dup&quot;) || (&lt;span class=&quot;math-container&quot;&gt;$8 eq &quot;ins&quot;))?$&lt;/span&gt;6:(&lt;span class=&quot;math-container&quot;&gt;$9)?$&lt;/span&gt;9:&quot;-&quot;).&quot;\n&quot;)&#xA;}';&#xA;chr7    140453145       140453145       A       T&#xA;chr7    140453136       140453136       A       T&#xA;chr7    140453135       140453136       TT      CA&#xA;chr20   31022287        31022287        -       A&#xA;chr19   13054614        13054614        G       -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Regular Expression&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This uses enclosing groups to describe the important bits of the sequence. The groups are numbered from 1 upwards, with each number corresponding to a left bracket (in order), capturing the entirety of the sub-expression until the paired right bracket. I'll describe the groups, which are referenced in the subsequent part of the script as &lt;code&gt;$&amp;lt;group_number&amp;gt;&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Any number of characters, up to (but not including) the first ':'&lt;/li&gt;&#xA;&lt;li&gt;Any number of the characters '0' to '9' [i.e. numbers]&lt;/li&gt;&#xA;&lt;li&gt;Optionally, a &quot;_&quot;, followed by any number of the characters '0' to '9'&lt;/li&gt;&#xA;&lt;li&gt;The numbers from Group &lt;strong&gt;[3]&lt;/strong&gt;, i.e. excluding the &quot;_&quot;&lt;/li&gt;&#xA;&lt;li&gt;Optionally, a &quot;dup&quot; or a &quot;del&quot;&lt;/li&gt;&#xA;&lt;li&gt;Any number of characters selected from &quot;A&quot;, &quot;C&quot;, &quot;G&quot;, &quot;T&quot; (i.e. bases)&lt;/li&gt;&#xA;&lt;li&gt;Optionally, an &quot;ins&quot; or &quot;&gt;&quot;, followed by any number of bases&lt;/li&gt;&#xA;&lt;li&gt;The &quot;ins&quot; or &quot;&gt;&quot; from Group &lt;strong&gt;[7]&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;The bases from Group &lt;strong&gt;[7]&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Print Code&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;print&lt;/code&gt; statement prints out a tab-separated list, finishing off with a line break. I'll describe each list element:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Group &lt;strong&gt;[1]&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Group &lt;strong&gt;[2]&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;If Group &lt;strong&gt;[4]&lt;/strong&gt; exists, then Group &lt;strong&gt;[4]&lt;/strong&gt;, otherwise Group &lt;strong&gt;[2]&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;If Group &lt;strong&gt;[8]&lt;/strong&gt; is &quot;ins&quot;, then Group &lt;strong&gt;[9]&lt;/strong&gt;; [otherwise]&lt;br&gt;&#xA;If Group &lt;strong&gt;[5]&lt;/strong&gt; is &quot;dup&quot;, then &quot;-&quot;, otherwise Group &lt;strong&gt;[6]&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;If Group &lt;strong&gt;[5]&lt;/strong&gt; is &quot;dup&quot;, or group &lt;strong&gt;[8]&lt;/strong&gt; is &quot;ins&quot;, then Group &lt;strong&gt;[6]&lt;/strong&gt;[otherwise]&lt;br&gt;&#xA;If Group &lt;strong&gt;[9]&lt;/strong&gt; exists, then Group &lt;strong&gt;[9]&lt;/strong&gt;, otherwise &quot;-&quot;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2019-09-24T20:03:03.917" LastActivityDate="2019-09-24T20:03:03.917" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10462" PostTypeId="1" AcceptedAnswerId="10463" CreationDate="2019-09-24T00:21:24.660" Score="1" ViewCount="42" Body="&lt;p&gt;I have a selected Arabidopsis Genome Initiative (AGI) list for RNA seq and proteomics data, how can I map them to metabolic pathway maps to vilualize(e.g. TCA cycle / FA / Photosynthesis) in KEGG or AraCyc or any freely available pathway maps?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone share an example if possible? &lt;/p&gt;&#xA;" OwnerUserId="2881" LastActivityDate="2019-09-24T08:13:11.293" Title="How to map selected genes to Metabolic pathway Maps" Tags="|r|rna-seq|pathway|proteomics|kegg|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10463" PostTypeId="2" ParentId="10462" CreationDate="2019-09-24T08:13:11.293" Score="1" Body="&lt;p&gt;KEGG has a webtool for this, &lt;a href=&quot;https://www.genome.jp/kegg/tool/map_pathway2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;KEGG Mapper&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can just copy-paste your gene ids in there and see how your pathways are colored.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2019-09-24T08:13:11.293" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10464" PostTypeId="2" ParentId="10426" CreationDate="2019-09-24T09:12:56.613" Score="3" Body="&lt;p&gt;No. If the only difference between Tumour_batch1 and Tumour_batch2 was the library prep/sequencing run, then all this will tell you is if the particular sample of RNA taken from the tumour is different from the particular sample of RNA taken from normal. It almost certainly is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Indeed if you did technical replicates of two RNA samples, both taken from normal, you would almost certainly find differences. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you want to find is the differences in RNA level between tumour in general and normal in general. To do that you need to make an estimate of the mean and variance of RNA levels in the tumour and in the normal. To do that you need multiple samples from each. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need differences within a single patient, it would be more honest just to compute the fold changes without any implication of statistical certainty. You would then need to think about alternate schemes for measuring the confidence in inferences drawn from the data when you consider the actual biological questions.&lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2019-09-24T09:12:56.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10465" PostTypeId="1" AcceptedAnswerId="10466" CreationDate="2019-09-24T09:48:35.677" Score="2" ViewCount="58" Body="&lt;p&gt;So, my runs have finished and got 5 final models. I predicted TM-score of one model to understand the result. The output of &quot;predict-tmscore.sh&quot; for &quot;final_1.pdb&quot; is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1 models read from PDB file&#xA;1 models read from PDB file&#xA;nres is 377&#xA;neff is 15423&#xA;dpscore is -300.599030&#xA;dpscorenorm is -0.082424&#xA;Probability of each TM-score bin (cols 1-10), estimated TM-score (col 11) and prob TM &amp;gt; 0.5 (col 12) written to sequence.ema.txt&#xA;Estimated TM-score of ../final_1.pdb is 0.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;could you please help me understand what does &quot;nres&quot;, &quot;neff&quot;, &quot;dpscore&quot;, and &quot;dpscorenorm&quot; below mean: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    nres is 377 &#xA;neff is 15423 &#xA;dpscore is -300.599030 &#xA;dpscorenorm is    -0.082424&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and it also generates a &quot;sequence.ema.txt&quot; file with below value:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;2.0308916646172293e-05 2.0682286958617624e-06 0.04564645141363144 0.14857041835784912 0.27177685499191284 0.31814685463905334 0.19150513410568237 0.022412151098251343 0.001872337656095624 4.743059980683029e-05 0.5036169363265117 0.5339840054512024&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Please let me know what these above values signify?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And,I would really appreciate your comments on what does &quot;rawdistpred.1&quot; file mean? There are total 9 such files. When predicitng TM-score, do I need to use &quot;rawdistpred.1&quot; file for all final models or I can use any &quot;rawdistpred.*&quot; file? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;thanks!   &lt;/p&gt;&#xA;" OwnerUserId="5180" LastActivityDate="2019-09-24T10:24:26.753" Title="What does final generated models with TM-score of DMPfold signify in &quot;sequence.ema.txt&quot; file?" Tags="|protein-structure|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10466" PostTypeId="2" ParentId="10465" CreationDate="2019-09-24T10:24:26.753" Score="2" Body="&lt;p&gt;This is described briefly in &lt;a href=&quot;https://www.nature.com/articles/s41467-019-11994-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper&lt;/a&gt;. The four quantities are inputs to the estimator of model accuracy:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;nres&lt;/code&gt; is the sequence length.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;neff&lt;/code&gt; is an effective sequence count, or how deep the alignment was.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;dpscore&lt;/code&gt; is how well the final model matches the initial distance prediction.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;dpscorenorm&lt;/code&gt; is a normalised version of the above.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Don't worry too much about directly interpreting these. They are only given for completion. The important line is &lt;code&gt;Estimated TM-score of ../final_1.pdb is 0.5&lt;/code&gt;, indicating that DMPfold just about thinks this is a decent model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &quot;sequence.ema.txt&quot; is more raw output giving the probability of the TM-score being in each bin. &quot;rawdistpred.1&quot; is the distance predictions from the first iteration. Always use &quot;rawdistpred.1&quot; as input to the TM-score predictor as this is how it was trained.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2019-09-24T10:24:26.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10467" PostTypeId="1" AcceptedAnswerId="10468" CreationDate="2019-09-24T11:32:24.980" Score="1" ViewCount="70" Body="&lt;p&gt;How do I perform protein model quality assessment?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will obtain predicted structures from DMPfold, I-Tasser, and Rosetta, what should be the next approach to select a structure from among this set of models?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, I have been looking at some, appropriate methods such as MetaMQAP, QA-RecobineIt, but their servers are not working. &lt;/p&gt;&#xA;" OwnerUserId="5180" LastEditorUserId="3848" LastEditDate="2020-05-01T23:44:51.620" LastActivityDate="2020-05-01T23:44:51.620" Title="Protein model quality assessment" Tags="|phylogenetics|phylogeny|protein-structure|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10468" PostTypeId="2" ParentId="10467" CreationDate="2019-09-24T13:21:42.880" Score="2" Body="&lt;p&gt;Have a read of the &lt;a href=&quot;https://onlinelibrary.wiley.com/doi/full/10.1002/prot.25767&quot; rel=&quot;nofollow noreferrer&quot;&gt;Estimation of model accuracy in CASP13&lt;/a&gt; paper to get the latest rankings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally you would run the model quality software on the models you have generated and choose the best. Also have a look at the structures in PyMol, you learn a lot that way. See, for instance, how much variety there is, and where the models vary. Is there a conserved core in many of the models? Or is there just a blob of &quot;spaghetti&quot; that doesn't look like an ordered protein?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have many models (e.g. from a Rosetta run) you might want to look into clustering the models first to reduce the number you have to work with.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2019-09-24T13:21:42.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10469" PostTypeId="1" CreationDate="2019-09-24T14:56:13.013" Score="0" ViewCount="422" Body="&lt;p&gt;I have my 24 samples analyzed by Hisat2&gt;Stringtie, from paired end sequenced data. I have got FPKM data and am planing to convert them to RPKM. to be able to compare my samples with another already published. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1- How can I convert from FPKM to RPKM?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2- Is it an acceptable method to compare samples using RPKM between two different studies or I need more or different procedures?    &lt;/p&gt;&#xA;" OwnerUserId="6256" LastActivityDate="2019-09-24T15:53:05.020" Title="Convert FPKM to RPKM" Tags="|rna-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10470" PostTypeId="2" ParentId="10469" CreationDate="2019-09-24T15:53:05.020" Score="4" Body="&lt;p&gt;I don't know if you have followed the discussions from the last years, but FPKM and RPKM are obsolete. Statisticians found that FPKM and RPKM are not the right values that can be used for proper statistics. It is a wrong way of normalizing RNA-seq data. In stead they suggest to use raw read counts, which can be used in tools like edgeR or DESeq2. Also, CPM would be better than FPKM or RPKM, for example you can use (log2) CPM values in limma with trend.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But to answer your questions.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You can't convert, you need to recalculate from raw data.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;No not acceptable. To compare studies you need to correct for batch, which you can do with statistical programs such as edgeR and DESeq2.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The best advice I can give is to start with raw data, also from the already published data that you would like to compare your own experiment with. And continue with edgeR or DESeq2 or the like.&lt;/p&gt;&#xA;" OwnerUserId="939" LastActivityDate="2019-09-24T15:53:05.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10471" PostTypeId="1" CreationDate="2019-09-24T20:31:30.817" Score="2" ViewCount="56" Body="&lt;p&gt;For 24 human stool samples I have metagenomic shotgun reads from an Illumina platform. The reads were filtered and mapped against a bacterial species library and specific maps to species were kept and counted. Importantly, before DNA extraction from any sample it was spiked with a solution of a fixed DNA concentration of &lt;em&gt;Thermus thermophilus&lt;/em&gt;, a species not occurring in the human microbiota. The reads from &lt;em&gt;T. thermophilus&lt;/em&gt; allow me to convert relative abundance estimates back to abundance in the stool sample, to absolute abundance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to test whether &lt;em&gt;T. thermophilus&lt;/em&gt; is indeed a good internal control that removes confounding by converting to absolute abundance. To test this, I compare between-species correlations on the relative abundance scale with their correlations on absolute abundance scale. I would expect less correlation in the latter. I first take the raw read counts of a random set of species, I divide counts by a sample-specific size factor estimate (total number of reads r in sample i / geometric mean of r's) to adjust for library size, then I do arcsinh transform (to stabilise variance) and then I look at between species correlation. Next I take again the raw read counts, correct by the size factor, then correct for an estimate of the coverage ( estimated proportion of &lt;em&gt;T. thermophilus&lt;/em&gt; in sample i ), and I do the acsinh transform. Again I look at between-species correlations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I compare species-pair correlation before and after &lt;em&gt;T. thermophilus&lt;/em&gt; correction in a scatterplot (see attached) to visualise the effect of the correction. I expected that the point cloud indicates a slope &amp;lt;1 reflecting that previously strong correlations have weakened on the absolute scale. But that is not what my result shows. Indeed, some weakly positive correlations have become stronger. How does this make sense? What other ways are there to test validity of an internal control? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vGRFY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vGRFY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5059" LastEditorUserId="48" LastEditDate="2019-09-25T10:56:00.413" LastActivityDate="2020-02-22T12:02:40.493" Title="Metagenomic shotgun data with internal control" Tags="|quality-control|metagenome|coverage|correlation|validation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10472" PostTypeId="2" ParentId="10467" CreationDate="2019-09-24T21:44:03.810" Score="0" Body="&lt;p&gt;Co-variance with convolution is an extremely hot method, so DMPFold via psicov. Checking the reality of the folding is obviously important, convolutions can be missed, inaccuracies can happen.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-09-24T21:44:03.810" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10473" PostTypeId="2" ParentId="7396" CreationDate="2019-09-25T06:58:13.700" Score="3" Body="&lt;p&gt;You can choose between a &lt;code&gt;window_size&lt;/code&gt; of 20 for sequences longer than 20 AAs, and the length of the sequence for those shorter::&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seq = &quot;CYGTGAVYLLGLGIGGFS&quot;&#xA;window_length = min(len(seq), 20)&#xA;&#xA;full_sequence_analysis = ProteinAnalysis(seq)&#xA;kyte_window = full_sequence_analysis.protein_scale(kyte, window_length, edge)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6244" LastActivityDate="2019-09-25T06:58:13.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10474" PostTypeId="2" ParentId="3380" CreationDate="2019-09-25T08:28:47.760" Score="2" Body="&lt;p&gt;Most of the solutions posted thus far are either slow, or can yield spurious results in certain cases. (I haven't tested Picard, which I assume will work as intended. But personally I tend to recoil when I have to set up a JVM!) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's approach the problem from a different angle:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Good hash functions have ~O(1) complexity for a lookup. If we can create a hash table of our QNAMEs, then searching a BAM file should be O(n) in the number of records. The Python developers are very proud of the hash function in cpython, so let's use that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;import sys&#xA;&#xA;with open(sys.argv[1], 'r') as indexfile:&#xA;    ids = set(l.rstrip('\r\n') for l in indexfile)&#xA;&#xA;for line in sys.stdin:&#xA;    qname, _ = line.split('\t', 1)&#xA;    if qname in ids:&#xA;        sys.stdout.write(line)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With a test set consisting of a ~3GB BAM file and a &lt;code&gt;qnames.txt&lt;/code&gt; with ~65K entries, running &lt;code&gt;samtools view input.bam | ./idfilter.py qnames.txt&lt;/code&gt; takes roughly &lt;strong&gt;6.5 seconds&lt;/strong&gt; on my test server. By comparison, piping the SAM to &lt;code&gt;fgrep -f qnames.txt&lt;/code&gt; (using GNU grep) takes about 8.5 seconds (but may yield spurious results).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Why might &lt;code&gt;grep&lt;/code&gt; yield spurious results? If you have a QNAME &lt;code&gt;foo&lt;/code&gt; in your &lt;code&gt;qnames.txt&lt;/code&gt;, but your BAM file also contains &lt;code&gt;fooX&lt;/code&gt; and &lt;code&gt;fooY&lt;/code&gt;, they will all be matched by &lt;code&gt;fgrep&lt;/code&gt;. The solution is to anchor each of your search patterns between the beginning of the line and a tab, e.g. &lt;code&gt;^foo\t&lt;/code&gt;, but then you have to use standard &lt;code&gt;grep&lt;/code&gt; which will have to construct an NFA for each pattern, and not &lt;code&gt;fgrep&lt;/code&gt; which implements Aho-Corasick, and you'll be orders of magnitude slower.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've rewritten my little Python script in Rust using &lt;code&gt;std::collections::HashMap&lt;/code&gt; for the hash function, and using the rust_htslib crate to read and write straight from BAM, and this is more than twice as fast as the Python, even when reading and writing BAM. However, my Rust isn't really fit for public consumption… &lt;/p&gt;&#xA;" OwnerUserId="6263" LastActivityDate="2019-09-25T08:28:47.760" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10475" PostTypeId="1" AcceptedAnswerId="10478" CreationDate="2019-09-25T08:50:59.587" Score="1" ViewCount="91" Body="&lt;p&gt;Very simple set-up: We have a PacBio long high-fidelity (HiFi) reads, genome sequencing, and some of those that contain a particular sequence. I want to find out which ones. The length of the query sequence is 7kb, though ideally I would use as query a sub-sequence of only 1.5kb.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output can the just the read ID or the full read sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We tried &lt;a href=&quot;https://github.com/thegenemyers/DAMAPPER&quot; rel=&quot;nofollow noreferrer&quot;&gt;DAMAPPER&lt;/a&gt; and it gives some decent results but it might not be appropriate to search for the shorter sub-sequence of 1.5kb. It also feels like an overkill for this purpose. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bear in mind that this is my first foray into long-read sequencing so my mindset is still very much one of short, low error-rate reads.&lt;/p&gt;&#xA;" OwnerUserId="108" LastEditorUserId="298" LastEditDate="2019-09-25T09:32:50.313" LastActivityDate="2019-09-25T13:38:15.323" Title="Best way to &quot;fish&quot; long reads that match a query sequence" Tags="|sequence-alignment|pacbio|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10476" PostTypeId="2" ParentId="10471" CreationDate="2019-09-25T11:28:20.627" Score="1" Body="&lt;p&gt;The problem with correlations in relative scale is know on the literature, it has been suggested to &lt;a href=&quot;https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1004075&quot; rel=&quot;nofollow noreferrer&quot;&gt;change to other metrics&lt;/a&gt;. So I wouldn't use it for internal control. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see on figure 3 of the linked article the correlation on the absolute scale tend to be higher. So I would say this is within expectations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As other way for internal controls are blanks (Just water to see if there are any contaminants), and technical replications, to make sure that the measure is correct not some problem with picking some amplifications instead of others...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also you don't mention it but the tool and library used to impute the microorganism is also important. You can check with other tools to see if the estimation is similar. &lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2019-09-25T11:28:20.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10478" PostTypeId="2" ParentId="10475" CreationDate="2019-09-25T13:38:15.323" Score="3" Body="&lt;p&gt;I have seen the blasr aligner typically used for aligning PacBio reads to a reference.  Is there a reason why blasr would not be a good choice? I guess the very short sequence you are trying to align to might be an issue.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Minimap2 is capable of mapping pacbio reads, and can also do pairwise mapping to see if reads overlap with each other, so this function might help you:&#xA;From &lt;a href=&quot;https://github.com/lh3/minimap2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/lh3/minimap2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 -x ava-pb  reads.fq query_seq.fq &amp;gt; ovlp.paf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-09-25T13:38:15.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10479" PostTypeId="1" CreationDate="2019-09-25T19:52:40.047" Score="2" ViewCount="1072" Body="&lt;p&gt;I need to do a bedtools intersect operation on a &lt;em&gt;very&lt;/em&gt; large .bam file.  When I use the standard bedtools intersect operation, the process consumes all the memory on my system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the bedtools documentation page, I find the &lt;code&gt;-sorted&lt;/code&gt; option to &lt;code&gt;bedtools intersect&lt;/code&gt;, which sounds like it is the perfect solution to the problem, but the documentation seems incomplete.  From the documentation, I need to ensure that the .bed file is sorted and I can use either &lt;code&gt;bedtools sort&lt;/code&gt; or the Linux &lt;code&gt;sort&lt;/code&gt; command to accomplish this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools sort -chrThenSizeA -i input.bed &amp;gt; sorted.bed&#xA;&#xA;sort -k1,1 -k2,2n input.bed &amp;gt; sorted.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I then execute the intersect operation on the .bam file, I get an error.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools intersect -sorted -c -a sorted.bed -b input.bam &amp;gt; intersect_linuxsorted.out&#xA;ERROR: chromomsome sort ordering for file input.bam is inconsistent with other files.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Tracking this down I find that &lt;code&gt;samtools sort&lt;/code&gt;, which was executed on the .bam file, &quot;sorts by the leftmost coordinate, or the read name if -n is used.&quot;  It would appear there is no way to put the .bam file in order by reference name, which is what bedtools instructs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a workaround, &lt;code&gt;bedtools sort&lt;/code&gt; does have the &lt;code&gt;-g&lt;/code&gt; option, which allows me to specify the ordering of the .bed file.  To get this ordering, I execute:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view input.bam | cut -f3 | unique &amp;gt; order.txt&#xA;bedtools sort -g order.txt &amp;gt; alt_sorted.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This puts the .bed file into the same order as the .bam file, and the &lt;code&gt;bedtools intersect -sorted&lt;/code&gt; command now works, but is it working correctly?  It seems that &lt;code&gt;bedtools intersect -sorted&lt;/code&gt; wants the files in consistent ordering, which can be done, but is that consistent with the intersect algorithm itself?  From the documentation's instructions to sort by the first 2 fields, chromosome name and left-most coordinate, it seems inconsistent to order it as the .bam file is ordered, by read name and not reference name.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure how to check this assumption.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:  Some extra details regarding sorting of the .bam file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the .bam file is sorted with &lt;code&gt;samtools sort input.bam &amp;gt; sorted.bam&lt;/code&gt; I get the error, 'ERROR: chromomsome sort ordering for file input.bam is inconsistent with other files`&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the .bam file is sorted with &lt;code&gt;samtools sort -n input.bam &amp;gt; sorted.bam&lt;/code&gt; I get this error, &lt;code&gt;Error: Sorted input specified, but the file sorted.bam has the following out of order&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would seem that if I sort without -n, it thinks the chromosomes are out of order, and if I sort with -n option it thinks the coordinate order is wrong. The only fix I have is forcing the order of the .bed file with the -g option, but again, I have no idea if that is consistent with the underlying algorithm's assumptions for the -sorted option.&lt;/p&gt;&#xA;" OwnerUserId="4491" LastEditorUserId="4491" LastEditDate="2019-09-25T23:28:12.027" LastActivityDate="2019-09-27T13:53:24.283" Title="bedtools intersect on very large .bam file - -sorted confusion" Tags="|samtools|bedtools|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10480" PostTypeId="2" ParentId="10479" CreationDate="2019-09-25T20:28:06.217" Score="4" Body="&lt;p&gt;It sounds like bedtools is behaving properly. The bam file is sorted by read name only if you used the &lt;code&gt;-n&lt;/code&gt; option in &lt;code&gt;samtools sort&lt;/code&gt;. Otherwise, if you used &lt;code&gt;samtools sort&lt;/code&gt; without the name-sort option, it is 'ordered' by reference name (which can be arbitrarily ordered, as I'll explain below) and then by position (which is numerically ordered). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think you are getting confused about 'ordering' the reference names. There are multiple ways to sort names; Some people might want numeric sorting (chr1, chr2, chr3...) and others might want lexicographic sorting (chr1, chr10, chr11...). Because of this, many tools allow you to specify the way you want to order the reference names. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There isn't necessary only one correct way to sort the reference names; what matters is that the order is the same between the files you're trying to intersect. This is exactly the purpose of &lt;code&gt;bedtools intersect -g&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to check the order of reference names in your bam, you can easily do so by checking the header via &lt;code&gt;samtools view -h input.bam&lt;/code&gt;. It will say something like this: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@HD     VN:1.3  SO:coordinate&#xA;@SQ     SN:chr1     LN:248956422&#xA;@SQ     SN:chr10    LN:133797422&#xA;@SQ     SN:chr11    LN:135086622&#xA;@SQ     SN:chr12    LN:133275309&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;@SQ&lt;/code&gt; lines tell you how your bam is sorted. My suspicion is that this order is different from the Linux &lt;code&gt;sort&lt;/code&gt; or &lt;code&gt;bedtools sort&lt;/code&gt; default, which is why your &lt;code&gt;bedtools intersect&lt;/code&gt; was failing. &lt;/p&gt;&#xA;" OwnerUserId="5088" LastEditorUserId="5088" LastEditDate="2019-09-25T21:34:53.497" LastActivityDate="2019-09-25T21:34:53.497" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10481" PostTypeId="2" ParentId="9247" CreationDate="2019-09-25T20:29:43.870" Score="2" Body="&lt;p&gt;I use the DropletUtils package from Bioconductor. Getting counts begins like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(DropletUtils)&#xA;filePath &amp;lt;- &quot;path_to_data/raw_feature_bc_matrix/&quot;&#xA;singleCellExperiment &amp;lt;- read10xCounts(filePath, col.names = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6232" LastActivityDate="2019-09-25T20:29:43.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10482" PostTypeId="1" AcceptedAnswerId="10487" CreationDate="2019-09-25T23:07:40.157" Score="1" ViewCount="74" Body="&lt;p&gt;I have downloaded some orthologues from ensembl website in fasta format. The problem is that it uses the ensembl sequence id as name for the sequences. I want to know the species name or any other information about the sequence (chromosome, location...) For example for this sequence id: ENSMODP00000018933. How can I do that? I have been looking at the REST API but the most I can find is an API call for getting the sequence passing the id as parameter.&#xA;&lt;a href=&quot;https://rest.ensembl.org/documentation/info/sequence_id&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rest.ensembl.org/documentation/info/sequence_id&lt;/a&gt;&#xA;I have also tried this API call but does not work for sequence ids:&#xA;&lt;a href=&quot;https://rest.ensembl.org/documentation/info/xref_id&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rest.ensembl.org/documentation/info/xref_id&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5190" LastActivityDate="2019-09-26T07:47:36.923" Title="Getting species from ensembl sequence id" Tags="|ensembl|api|orthologues|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10483" PostTypeId="1" AcceptedAnswerId="10488" CreationDate="2019-09-26T02:44:07.453" Score="4" ViewCount="233" Body="&lt;p&gt;I have two very large vcf files 2GB and 6GB I want to look for unique combinations of CHROM and POS and output the row that matches. However, because the files a so large my machine always hang and stop processing. Is there are a way to work around the problem of this large file? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using this command, taken from an answer to &lt;a href=&quot;https://unix.stackexchange.com/q/543633/22222&quot;&gt;another question&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk  '{ if(NR==FNR){a[&lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;2]=&lt;span class=&quot;math-container&quot;&gt;$0}else{if($1$&lt;/span&gt;2 in a){print}}}' file1.vcf file2.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;file 1&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT QUAL&#xA;1   10366   rs58108140  G   A   &#xA;1   10611   rs189107123 C   G   &#xA;1   51954   rs185832753 G   C   &#xA;1   13327   rs144762171 G   C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;file 2&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT QUAL    &#xA;1   10366   rs58108140  G   A   &#xA;1   51935   rs181754315 C   T   &#xA;1   51954   rs185832753 G   C   &#xA;1   52058   rs62637813  G   C   &#xA;1   52144   rs190291950 T   A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;output&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1 10366 rs58108140  G   A&#xA;1 51954  rs185832753 G   C &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6271" LastEditorUserId="298" LastEditDate="2019-09-26T08:35:48.530" LastActivityDate="2019-10-06T21:53:08.607" Title="awk working with large files" Tags="|vcf|awk|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10484" PostTypeId="2" ParentId="10483" CreationDate="2019-09-26T03:15:16.010" Score="0" Body="&lt;p&gt;Sort both files by chromosome and position and read each file one line by line. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Either you iterate to the next line in one file, or go the next in the other, or there's a match that you print out and then iterate both lines. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is not necessarily an awk approach, but is doable with Perl, Python, or any other scripting language.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In pseudocode:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sub getCommonLines()&#xA;  Sort FileA&#xA;  Sort FileB&#xA;  Get lineA from FileA&#xA;  Get lineB from FileB&#xA;  While not EOF in FileA or EOF in FileB:&#xA;    If lineA &amp;gt; lineB:&#xA;      Get lineB from FileB&#xA;    Elseif lineB &amp;gt; lineA:&#xA;      Get lineA from FileA&#xA;    Else:&#xA;      Print lineA or lineB&#xA;      Get lineA from FileA&#xA;      Get lineB from FileB&#xA;End Sub&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The use of the &lt;code&gt;&amp;gt;&lt;/code&gt; operator is based on lexicographical and numerical ordering provided from sorting files A and B, and comparison should be done only on the fields of interest. If you compare other fields (esp. from VCF files) you may get other fields that give you the wrong or unexpected comparison result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The term &lt;code&gt;EOF&lt;/code&gt; means end-of-file, which means just what it says — you're at the end of the file and there are no more lines to compare between files, so your subroutine is done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorting has a time cost, but memory overhead for modern sorting algorithms is very minimal. You can sort via process substitution, but sorting to an intermediate file can be useful if you have other things you want to do with them, where sorted input is valuable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This would be a good approach for very large files that you do not edit or update often (or at all), for which a hash table (or &quot;dictionary&quot;, or &quot;associative array&quot;, all the same terms for the same thing) is impractical because of its large memory overhead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use a tool like &lt;code&gt;comm&lt;/code&gt; with sorted files, as described in another answer/comment. This does basically what my answer describes without you having to understand or implement things. You must remember to provide sorted input, however, as with other Unix tools that do line comparison (&lt;code&gt;uniq&lt;/code&gt;, etc.). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite this answer being a really long &quot;comment&quot;, it is useful to have a basic understanding of how things work under the hood, so that when you use these tools, you know what's going on and why you need to provide inputs in a certain way. It's easy to get a million one-liners off of Stack Exchange sites, but understanding is valuable beyond gold.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2019-09-27T23:51:57.433" LastActivityDate="2019-09-27T23:51:57.433" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10486" PostTypeId="1" AcceptedAnswerId="10489" CreationDate="2019-09-26T05:10:50.473" Score="1" ViewCount="45" Body="&lt;p&gt;Are there databases that contain information about gene or protein expression before and after targeted therapy or chemotherapy? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am curious about different cancer cell lines or cancer types. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion on a set of resources I should look into?&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2019-09-29T18:37:43.437" LastActivityDate="2019-09-29T18:37:43.437" Title="Are there databases or resources that contain expression of proteins or genes before and after cancer treatment?" Tags="|public-databases|cancer|data-retrieval|public-dataset|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10487" PostTypeId="2" ParentId="10482" CreationDate="2019-09-26T07:47:36.923" Score="4" Body="&lt;p&gt;The &lt;a href=&quot;http://rest.ensembl.org/documentation/info/lookup&quot; rel=&quot;nofollow noreferrer&quot;&gt;lookup/id endpoint&lt;/a&gt; will get it for you. Of you can just look up the three letter species code, in this case MOD, &lt;a href=&quot;http://www.ensembl.org/info/genome/stable_ids/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;on the list in the documentation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2019-09-26T07:47:36.923" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10488" PostTypeId="2" ParentId="10483" CreationDate="2019-09-26T08:30:35.060" Score="3" Body="&lt;p&gt;Maybe &lt;a href=&quot;https://www.gnu.org/software/coreutils/manual/html_node/comm-invocation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;comm&lt;/a&gt;, which is part of GNU coreutils, is more efficient.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;comm writes to standard output lines that are common, and lines that are unique, to two input files; a file name of ‘-’ means standard input. Synopsis:&lt;/p&gt;&#xA;&lt;p&gt;comm [option]… file1 file2&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In your case you could sort both files based on the first 3 columns:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;file1=file1.vcf&#xA;file2=file2.vcf&#xA;&#xA;comm -12 &amp;lt;(awk '{print &lt;span class=&quot;math-container&quot;&gt;$1,$&lt;/span&gt;2,&lt;span class=&quot;math-container&quot;&gt;$3}' $&lt;/span&gt;file1 | sort) \&#xA;         &amp;lt;(awk '{print &lt;span class=&quot;math-container&quot;&gt;$1,$&lt;/span&gt;2,&lt;span class=&quot;math-container&quot;&gt;$3}' $&lt;/span&gt;file2 | sort) &amp;gt; matches&#xA;&#xA;&#xA;# result:&#xA;cat matches&#xA;1 10366 rs58108140&#xA;1 51954 rs185832753&#xA;#CHROM POS ID&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This could be of course polished by removing the header.&#xA;Also if you want the original lines as output, you can grep the matches to the original files:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -f matches file1&#xA;#CHROM POS ID REF ALT QUAL &#xA;1 10366 rs58108140 G A&#xA;1 51954 rs185832753 G C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Edit: In the first version, intermediate temp files were used, but as suggested by terdon in the comments, this can be avoided using &lt;a href=&quot;http://tldp.org/LDP/abs/html/process-sub.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Process Substitution&lt;/a&gt; allowing the &lt;code&gt;&amp;lt;(..)&lt;/code&gt; contstruct.&lt;/p&gt;&#xA;" OwnerUserId="1461" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2019-09-26T12:45:22.430" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="10489" PostTypeId="2" ParentId="10486" CreationDate="2019-09-26T11:46:41.417" Score="2" Body="&lt;p&gt;One such database/tool is &lt;code&gt;“[PharmacoGx][1]: an R package for analysis of large pharmacogenomic datasets.” Bioinformatics (Oxford, England)&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They brought together a large compendium of experimental data (cell lines and the like if I remember correctly from a talk), standardized these and made available within a special object class. I think there is genomic and transcriptomic data, not sure if there is proteomic data though.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-09-26T11:46:41.417" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10490" PostTypeId="2" ParentId="7195" CreationDate="2019-09-26T14:49:14.217" Score="0" Body="&lt;p&gt;That's certainly a lot of nodes and edges, however, there is a preference called viewThreshold that should prevent the creation of a view if the number of nodes+edges is larger than the set value.  I've just tested it on a large network I have (9000 nodes and 4,000,000 edges) and it doesn't appear to be working properly.  I'll mark this as a bug.  It does, however, render, but it's not really a useful network on Cytoscape 3.7.  It will actually be usable on Cytoscape 3.8 based on early testing, however.  If you want to make your network available somewhere, I'm happy to try to see why it's not loading.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- scooter&lt;/p&gt;&#xA;" OwnerUserId="6274" LastActivityDate="2019-09-26T14:49:14.217" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10491" PostTypeId="1" AcceptedAnswerId="10493" CreationDate="2019-09-26T15:23:58.583" Score="2" ViewCount="402" Body="&lt;p&gt;My directories are organized as follows: one main directory, in which I have multiple directories that end with a number ranging from 314 to 727, followed by .3   . For example, 'mgm4761314.3'. Within these directories, I have fasta files named after a similar pattern (for example, mgm4761314.3.299.screen.passed.fna is in the directory mgm4761314.3).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is only one such file (.fna) per directory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I've been trying to do is to write a loop that would go through all these folders and fasta files, and add the short number associated with the folder/file name (eg. 314, 315...727) to the headers. For example, for my first folder/fasta file, instead of:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;seq1&#xA;&amp;gt;seq2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;314_seq1&#xA;&amp;gt;314_seq2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output would be exported to a file called, for instance, 314.fna.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've sort of made it work for one number at a time, but I have plenty of files, so I'd appreciate if anyone could help me with the loop part! This is what I tried (and it didn't work). Using ubuntu.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in {314..727}; do sed &quot;s/&amp;gt;/&amp;gt;&lt;span class=&quot;math-container&quot;&gt;${i}_/&quot; [...]/*$&lt;/span&gt;{i}.3/mgm4761&lt;span class=&quot;math-container&quot;&gt;${i}.3.299.screen.passed.fna &amp;gt; [...]/*$&lt;/span&gt;{i}.3/${i}.fna; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4401" LastEditorUserId="4401" LastEditDate="2019-09-26T15:40:59.510" LastActivityDate="2019-09-26T16:57:49.220" Title="How to edit the headers of multiple fasta files from multiple folders?" Tags="|fasta|bash|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10492" PostTypeId="2" ParentId="10435" CreationDate="2019-09-26T15:40:09.697" Score="0" Body="&lt;p&gt;Generally, but not always the I-Tasser algorithm is not in the same league as a covariant mutation-based analysis (psicov) and don't think many people would disagree with this. I don't do I-Tasser but I'm aware it is the algorithm of choice for rough and ready predictions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Psicov based models are essentially de novo predictions, based on amino acid mutation patterns, which I assume are refined against known homologous structure in DMFold (inaccuracies can creep through). I suspect the starting point of I-Tasser is a known homologue and it's associated known structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said that you need alot of sequence data to make psicov covariance work (&gt;8000 aligned sequences) and (strictly in my opinion) it will also depend on the speed of mutation. Higher eukaryote  gene alignments could require much higher numbers of taxa in the alignment. The covariance value will give you the strength of the signal and values &gt;0.7 should be used. I personally consider&gt;0.5 to 0.7 borderline, which is fair enough because those are the generic thresholds in statistics. I don't know if you can access the underlying covariance values in DMFold, or moreover set the threshold.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I-Tasser could outperform DMFold in given situations, but the underlying output of covariance based approaches should make it clear how well the approach has performed. Obviously if it performs well I'd go with that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would follow@jgreener 's advice on the follow up program.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-09-26T15:53:45.230" LastActivityDate="2019-09-26T15:53:45.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10493" PostTypeId="2" ParentId="10491" CreationDate="2019-09-26T16:01:04.813" Score="3" Body="&lt;p&gt;You were so close! This should do the trick:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in {314..727}; do &#xA;    sed -i.bak &quot;s/^&amp;gt;/&amp;gt;&lt;span class=&quot;math-container&quot;&gt;${i}_/&quot; mgm4761$&lt;/span&gt;{i}.3/*.fna&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that I am using &lt;code&gt;-i&lt;/code&gt; which makes &lt;code&gt;sed&lt;/code&gt; edit the original file. Because I use &lt;code&gt;-i.bak&lt;/code&gt;, this will create a backup file with the original data. Once you are sure the command does what you need, you can delete all of these files with &lt;code&gt;rm */*fna.bak&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-09-26T16:57:49.220" LastActivityDate="2019-09-26T16:57:49.220" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="10494" PostTypeId="1" CreationDate="2019-09-26T18:32:58.403" Score="1" ViewCount="156" Body="&lt;p&gt;Normally, I use bwa-mem to map my fastq reads for a bunch of species using one protein-coding gene CDS from my reference species. This was fine when I had to try it for just a group of genes...I would obtain single CDS fastq files and use them as a reference.&#xA;However, we now want to obtain protein-coding sequences for ALL GENES in my exome datasets. If I have one file containing all the CDS from my reference species, can that be used in bwa-mem? The end of my custom script produces one multiple sequence alignment file when I ran it using one gene only...my concern is that bwa-mem will somehow replace certain CDS or would I just end up with over 20000+ MSAs?  Which is fine because I need them to be individual MSAs for each protein-coding gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use a simple python script at the end to extract base information from the mpileup files generated to create my MSAs. &lt;/p&gt;&#xA;" OwnerUserId="6275" LastActivityDate="2022-02-13T13:00:40.067" Title="How to map multiple CDS sequences to exome data using bwa mem?" Tags="|bwa|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10498" PostTypeId="2" ParentId="10494" CreationDate="2019-09-27T06:17:47.407" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If I have one file containing all the CDS from my reference species, can that be used in bwa-mem? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That's fine, normal usage of bwa-mem entails giving it a fasta file with all chromosomes/contigs/transcript/etc. in an organism, so your usage is no different. How your custom &quot;make an MSA&quot; script handles that is unknown of course.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-09-27T06:17:47.407" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10499" PostTypeId="1" CreationDate="2019-09-27T13:14:09.880" Score="0" ViewCount="50" Body="&lt;p&gt;I did illumina read mapping based on a particular gene sequence (reference sequence) using the following command,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bowtie2-build -f ref_gene.fasta ref_index &amp;amp;&amp;amp; bowtie2 -p 4 -N 1 -t -x ref_index -1 genome_R1.fastq -2 genome_R2.fastq -S tal.sam &amp;amp;&amp;amp; samtools view -bS tal.sam | samtools sort - -o tal.bam &amp;amp;&amp;amp; samtools mpileup -uf ref_gene.fasta tal.bam | bcftools call -c | vcfutils.pl vcf2fq &amp;gt; cns.fastq &amp;amp;&amp;amp; seqtk seq -aQ64 -q20 -n N cns.fastq &amp;gt; contig.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It maps the reads according to the reference sequence and yields, exactly same length of aligned/assembled sequences. But, I need obtain the unaligned overlapping read bases at the both end (reverse and forward end)  Please see the image for better understanding. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IsJOl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IsJOl.png&quot; alt=&quot;image of the reference gene and the extended read&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way to get the same?&lt;/p&gt;&#xA;" OwnerUserId="4712" LastEditorUserId="4712" LastEditDate="2019-09-27T14:59:27.957" LastActivityDate="2019-09-27T14:59:27.957" Title="How to obtain the extended unaligned bases of the read in reference based genome assembly?" Tags="|sam|read-mapping|assembly|reads|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10500" PostTypeId="2" ParentId="3380" CreationDate="2019-09-27T13:32:04.073" Score="3" Body="&lt;p&gt;Performing a grep on &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; alignments to &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; qnames would give you &lt;span class=&quot;math-container&quot;&gt;$O(mn)$&lt;/span&gt; operations. If you decide to sort your BAM and qnames first, you can reduce it to &lt;span class=&quot;math-container&quot;&gt;$O(\min\{m, n\})$&lt;/span&gt; by popping reads and qnames off their stacks. You also don't read the entirety of the BAM at once this way, limiting memory consumption. I'm pretty sure Picard does something like this, as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example in Python that I've written for this very purpose:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pysam&#xA;&#xA;def filter_qname(bamfile, idfile, outfile):&#xA;    '''&#xA;    Filter reads in a SAM/BAM file by their query names&#xA;&#xA;    Parameters&#xA;    ----------&#xA;    bamfile : str&#xA;        Sorted BAM file path&#xA;    idfile : str&#xA;        Text file path containing qnames to keep&#xA;    outfile : str&#xA;        Output file to write to&#xA;    '''&#xA;    # read in name-sorted BAM file&#xA;    bam = pysam.AlignmentFile(bamfile, &quot;rb&quot;)&#xA;    if outfile.endswith('.sam'):&#xA;        output = pysam.AlignmentFile(outputfile, &quot;w&quot;, template=bam)&#xA;    elif outfile.endswith('.bam'):&#xA;        output = pysam.AlignmentFile(outputfile, &quot;wb&quot;, template=bam)&#xA;    else:&#xA;        raise ValueError(&quot;Unknown output file format for `outfile`: {}&quot;.format(outfile))&#xA;    #  read in IDs to be removed (list(set(...)) to only keep unique IDs)&#xA;    ids = list(set([l.rstrip() for l in open(idfile, 'r').readlines()]))&#xA;    # sort IDs to match BAM for efficient processing (destructive function)&#xA;    ids.sort(key=str.lower, reverse=True)&#xA;    n_ids = len(ids)&#xA;    # variable for ensuring BAM is sorted by query name&#xA;    last_q = None&#xA;    reads = bam.fetch(until_eof=True)&#xA;    read = next(reads)&#xA;    while True:&#xA;        # if read name is greater than current top of stack&#xA;        if read.query_name &amp;gt; ids[-1]:&#xA;            # if this is the last ID&#xA;            if n_ids == 1:&#xA;                # write remaining reads to new file and break out of while loop&#xA;                output.write(read)&#xA;                for r in bam:&#xA;                    output.write(read)&#xA;                break&#xA;            # otherwise pop that ID, try again&#xA;            else:&#xA;                ids.pop()&#xA;                n_ids -= 1&#xA;        # skip reads that match the top of the stack&#xA;        elif read.query_name == ids[-1]:&#xA;            read = next(reads)&#xA;        # if read name is less that top of stack, write it and move on&#xA;        else:&#xA;            output.write(read)&#xA;            read = next(reads)&#xA;    output.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've implemented it in my own miscellaneous bioinformatics tools package &lt;a href=&quot;https://github.com/jrhawley/bio-jtools/blob/master/jtools/align/filter_qname.py#L23-L72&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; if you want to see it in action&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2019-09-27T13:32:04.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10501" PostTypeId="2" ParentId="10479" CreationDate="2019-09-27T13:53:24.283" Score="1" Body="&lt;p&gt;As per &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/10480/240&quot;&gt;Lynn's answer&lt;/a&gt;, sorting is dependent on how chromosomes/contigs are named, how you want them to be sorted, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;samtools sort&lt;/code&gt; or &lt;code&gt;sambamba sort&lt;/code&gt; will sort as per the &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAM specification&lt;/a&gt; by the order of the contigs in the header of the file. If you've aligned these sequences to a human reference genome, typically it will be &lt;code&gt;chr1&lt;/code&gt;, &lt;code&gt;chr2&lt;/code&gt;, ... &lt;code&gt;chr10&lt;/code&gt;, &lt;code&gt;chr11&lt;/code&gt;, ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To sort the BED file in the same way, you can specify the sort order with the &lt;code&gt;bedtools sort -g&lt;/code&gt; option and providing the explicit order.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, use &lt;code&gt;sort&lt;/code&gt; with the &lt;code&gt;-V, --version-sort&lt;/code&gt; option, that provides a &quot;natural sort of (version) numbers within text&quot;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sambamba sort -o output.bam input.bam&#xA;sort -k1,1 -V -k2,2n input.bed &amp;gt; sorted.bed&#xA;bedtools intersect -sorted -c -a sorted.bed -b input.bam &amp;gt; intersect_linuxsorted.out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="240" LastActivityDate="2019-09-27T13:53:24.283" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10502" PostTypeId="1" CreationDate="2019-09-27T16:09:40.790" Score="0" ViewCount="60" Body="&lt;p&gt;I got two lists of about 300 genes from my GO analysis, like the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Mtss1/Sox11/Fbn2/Mmp2/Tcf7l2/Six2/Ahr/Fat4/Lef1/Stc1/Acvr2a/Glis1/Nog/Igsf10/Twist2/Igf1/...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I convert the list to one element per row (for Venn diagram  on Venny 2.1), as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Mtss1&#xA;Sox11&#xA;Fbn2&#xA;Mmp2&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6252" LastEditorUserId="48" LastEditDate="2019-09-30T07:11:34.610" LastActivityDate="2019-09-30T08:46:58.107" Title="How to convert a list of genes to one element per row?" Tags="|go-enrichment|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10503" PostTypeId="2" ParentId="10502" CreationDate="2019-09-27T16:36:03.530" Score="2" Body="&lt;pre&gt;&lt;code&gt;$ echo &quot;Mtss1/Sox11/Fbn2/Mmp2/...&quot; | tr '/' '\n' &amp;gt; list.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For more information on &lt;code&gt;tr&lt;/code&gt;: &lt;a href=&quot;https://en.wikipedia.org/wiki/Tr_(Unix)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Tr_(Unix)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2019-09-27T16:36:03.530" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10504" PostTypeId="2" ParentId="10502" CreationDate="2019-09-27T16:51:33.170" Score="0" Body="&lt;p&gt;Many ways, assuming you're on a *nix system. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;tr&lt;/code&gt;. Alex &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/10503/298&quot;&gt;already gave you&lt;/a&gt; a fine &lt;code&gt;tr&lt;/code&gt; solution, but here's another approach assuming you have an input file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tr / '\n' &amp;lt; golist.txt &amp;gt; newlist.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;sed&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed 's|/|\n|g' golist.txt &amp;gt; newlist.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, to edit the original file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -i 's|/|\n|g' golist.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Perl&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl -pe 's|/|\n|g' golist.txt &amp;gt; newlist.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, to edit the original file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl -i -pe 's|/|\n|g' golist.txt &#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;grep&lt;/code&gt; (GNU &lt;code&gt;grep&lt;/code&gt;, the default on Linux systems)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -Eo '[^/]+' golist.txt &amp;gt; newlist.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-09-27T17:03:33.350" LastActivityDate="2019-09-27T17:03:33.350" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10505" PostTypeId="1" CreationDate="2019-09-29T08:53:58.257" Score="1" ViewCount="179" Body="&lt;p&gt;I am required to create a phylogenetic tree(s) of my genes of interest and am currently blasting (tblastn) my genes of interest from S. mediterranea against model systems before expanding my search.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt; I get something like the attached which does not look like right. What am I doing wrong? I'm aligning via MUSCLE, and running Maximum Likliehood Protein models and consequentially, using the correct model (JTT, for instance). &lt;a href=&quot;https://i.stack.imgur.com/0RAuH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0RAuH.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Greatly appreciate any help in advance.&lt;/p&gt;&#xA;" OwnerUserId="6285" LastEditorUserId="3848" LastEditDate="2020-07-27T06:47:10.683" LastActivityDate="2020-07-27T06:47:10.683" Title="Weird phylogenetic tree" Tags="|sequence-alignment|fasta|phylogenetics|phylogeny|systems-biology|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10506" PostTypeId="1" AcceptedAnswerId="10507" CreationDate="2019-09-29T10:50:57.743" Score="1" ViewCount="49" Body="&lt;p&gt;I have a file where i what to check for a pattern below and print out all the lines that matches the pattern but the output is empty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here is the pattern that i am looking for.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Father = 1/0 or 0/1 or 1|0 or 0|1&lt;/li&gt;&#xA;&lt;li&gt;Mother = 1/0 or 0/1 or 1|0 or 0|1&lt;/li&gt;&#xA;&lt;li&gt;Daughter 1 = 1|0 or 0|1 or 1/0 or 0/1&lt;/li&gt;&#xA;&lt;li&gt;Daughter 2 = 1|1 or 1/1&lt;/li&gt;&#xA;&lt;li&gt;Daughter 3 = 0|0 or 0/0&lt;/li&gt;&#xA;&lt;li&gt;Son 1 = 1|0 or 0|1 or 1/0 or 0/1&lt;/li&gt;&#xA;&lt;li&gt;Son 2 = 1|1 or 1/1&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;mysample file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM POS ID REF ALT FATHER MOTHER DAUGHTER1 DAUGHTER2 DAUGHTER3 SON1 SON2&#xA;1 736689 . T A 0/0:37:1.224:2.367:0.054:0.0:108:0.00:4.75:0.53:0.00:T,37,2.367:37,0 1/0:33:1.099:3.320:0.091:358.5:359:0.59:4.34:0.59:0.08:A,15,1.149,T,18,2.171:18,15 0|0:30:0.999:1.314:0.033:0.0:87:0.00:1.16:0.00:0.00:T,30,1.314:30,0 0|1:31:1.032:2.299:0.065:211.5:212:2.61:0.48:0.07:0.43:A,12,0.124,C,1,0.010,T,18,2.165:18,12 0|1:37:1.232:2.374:0.054:263.2:263:1.47:8.69:0.06:0.05:A,16,1.160,T,21,1.214:21,16 0|0:43:1.423:2.447:0.047:0.0:98:0.20:0.83:0.45:0.00:A,1,0.010,T,42,2.437:42,1 0|1:40:1.323:3.393:0.075:280.5:281:1.95:2.36:3.47:0.27:A,17,1.175,T,23,2.219:23,17&#xA;1 758663 . CGCC TTAG 0/0:27:0.893:0.296:0.000:0.0:90:5.15:0.85:3.67:0.01:27,0 0/1:21:0.699:0.234:0.000:2333.1:353:0.93:2.17:2.36:0.00:9,12 0|0:35:1.166:0.370:0.000:0.0:108:3.97:3.43:4.75:0.00:34,0 0|1:33:1.099:0.375:0.000:2820.8:447:0.07:8.69:0.53:0.00:17,15 0|1:23:0.766:0.232:0.000:1860.0:382:0.85:0.17:0.09:0.00:13,10 0|0:23:0.761:0.255:0.000:0.0:75:1.51:2.59:0.78:0.00:22,0 0|1:18:0.596:0.215:0.000:1686.8:275:0.00:2.17:0.10:0.00:9,9&#xA;1 762005 . G C 1/0:36:1.191:5.321:0.139:360.9:357:0.00:1.93:0.97:0.00:C,18,2.166,G,18,3.156:18,18 0/0:24:0.799:4.207:0.167:0.0:60:0.00:0.36:3.26:0.00:G,24,4.207:24,0 1|0:40:1.332:3.384:0.075:332.2:332:0.22:13.83:0.22:0.00:C,19,1.186,G,21,2.197:21,19 1|0:31:1.032:3.290:0.097:236.2:236:1.75:8.18:0.07:0.00:C,13,0.134,G,18,3.156:18,13 1|0:32:1.066:4.290:0.125:187.0:187:5.67:1.78:9.77:0.00:C,11,0.114,G,20,3.177,T,1,1.000:20,11 1|0:43:1.423:4.405:0.093:181.6:182:10.13:1.24:0.45:0.00:C,14,2.124,G,28,2.270,T,1,0.010:28,14 0|0:34:1.125:2.332:0.059:0.0:96:0.00:4.09:0.26:0.00:G,34,2.332:34,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried this, but the output is empty:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk  '&lt;span class=&quot;math-container&quot;&gt;$9==&quot;1|1&quot;  || $&lt;/span&gt;10==&quot;0|0&quot; || &lt;span class=&quot;math-container&quot;&gt;$12==&quot;1|1&quot; || $&lt;/span&gt;8==&quot;1|0&quot; || &lt;span class=&quot;math-container&quot;&gt;$11==&quot;1|0&quot; || $&lt;/span&gt;6==&quot;1/0&quot;||$7 ==&quot;1/0&quot; {print}' variants.vcf &amp;gt; test.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6271" LastEditorUserId="298" LastEditDate="2019-09-29T11:36:12.700" LastActivityDate="2019-10-09T12:59:49.617" Title="awk find Sickle cell anemia from father, mother, son, daughter" Tags="|awk|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10507" PostTypeId="2" ParentId="10506" CreationDate="2019-09-29T11:55:09.487" Score="1" Body="&lt;p&gt;The fields aren't only the genotype. They contain much more information. So &lt;code&gt;$9&lt;/code&gt; will never be exactly &lt;code&gt;1|1&lt;/code&gt;, what you are interested in is that the field &lt;em&gt;starts with&lt;/em&gt; &lt;code&gt;1|1&lt;/code&gt;. Also, if you need all of the conditions to match (meaning that at least one of the criteria for each sample is met), you need &lt;code&gt;&amp;amp;&amp;amp;&lt;/code&gt; not &lt;code&gt;||&lt;/code&gt;. So try this instead:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '(&lt;span class=&quot;math-container&quot;&gt;$6~/^1\/0/   || $&lt;/span&gt;6~/^0\/1/ || &lt;span class=&quot;math-container&quot;&gt;$6~/^1\\|0/   || $&lt;/span&gt;6~/^0\\|1/) &amp;amp;&amp;amp;&#xA;     (&lt;span class=&quot;math-container&quot;&gt;$7~/^1\/0/   || $&lt;/span&gt;7~/^0\/1/ || &lt;span class=&quot;math-container&quot;&gt;$7~/^1\\|0/   || $&lt;/span&gt;7~/^0\\|1/  ) &amp;amp;&amp;amp;&#xA;     (&lt;span class=&quot;math-container&quot;&gt;$8~/^1\/0/   || $&lt;/span&gt;8~/^0\/1/ || &lt;span class=&quot;math-container&quot;&gt;$8~/^1\\|0/   || $&lt;/span&gt;8~/^0\\|1/) &amp;amp;&amp;amp;&#xA;     (&lt;span class=&quot;math-container&quot;&gt;$9~/^1\\|1/  || $&lt;/span&gt;9~/^1\/1/ ) &amp;amp;&amp;amp;&#xA;     (&lt;span class=&quot;math-container&quot;&gt;$10~/^0\\|0/ || $&lt;/span&gt;10~/^0\/0/ ) &amp;amp;&amp;amp;&#xA;     (&lt;span class=&quot;math-container&quot;&gt;$11~/^1\/0/  || $&lt;/span&gt;11~/^0\/1/ || &lt;span class=&quot;math-container&quot;&gt;$11~/^1\\|0/ || $&lt;/span&gt;11~/^0\\|1/) &amp;amp;&amp;amp;&#xA;     (&lt;span class=&quot;math-container&quot;&gt;$12~/^1\\|1/ || $&lt;/span&gt;12 ~/^1\/1/)' foo.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Running that on your file returns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1 736689 . T A 0/0:37:1.224:2.367:0.054:0.0:108:0.00:4.75:0.53:0.00:T,37,2.367:37,0 1/0:33:1.099:3.320:0.091:358.5:359:0.59:4.34:0.59:0.08:A,15,1.149,T,18,2.171:18,15 0|0:30:0.999:1.314:0.033:0.0:87:0.00:1.16:0.00:0.00:T,30,1.314:30,0 0|1:31:1.032:2.299:0.065:211.5:212:2.61:0.48:0.07:0.43:A,12,0.124,C,1,0.010,T,18,2.165:18,12 0|1:37:1.232:2.374:0.054:263.2:263:1.47:8.69:0.06:0.05:A,16,1.160,T,21,1.214:21,16 0|0:43:1.423:2.447:0.047:0.0:98:0.20:0.83:0.45:0.00:A,1,0.010,T,42,2.437:42,1 0|1:40:1.323:3.393:0.075:280.5:281:1.95:2.36:3.47:0.27:A,17,1.175,T,23,2.219:23,17&#xA;1 758663 . CGCC TTAG 0/0:27:0.893:0.296:0.000:0.0:90:5.15:0.85:3.67:0.01:27,0 0/1:21:0.699:0.234:0.000:2333.1:353:0.93:2.17:2.36:0.00:9,12 0|0:35:1.166:0.370:0.000:0.0:108:3.97:3.43:4.75:0.00:34,0 0|1:33:1.099:0.375:0.000:2820.8:447:0.07:8.69:0.53:0.00:17,15 0|1:23:0.766:0.232:0.000:1860.0:382:0.85:0.17:0.09:0.00:13,10 0|0:23:0.761:0.255:0.000:0.0:75:1.51:2.59:0.78:0.00:22,0 0|1:18:0.596:0.215:0.000:1686.8:275:0.00:2.17:0.10:0.00:9,9&#xA;1 762005 . G C 1/0:36:1.191:5.321:0.139:360.9:357:0.00:1.93:0.97:0.00:C,18,2.166,G,18,3.156:18,18 0/0:24:0.799:4.207:0.167:0.0:60:0.00:0.36:3.26:0.00:G,24,4.207:24,0 1|0:40:1.332:3.384:0.075:332.2:332:0.22:13.83:0.22:0.00:C,19,1.186,G,21,2.197:21,19 1|0:31:1.032:3.290:0.097:236.2:236:1.75:8.18:0.07:0.00:C,13,0.134,G,18,3.156:18,13 1|0:32:1.066:4.290:0.125:187.0:187:5.67:1.78:9.77:0.00:C,11,0.114,G,20,3.177,T,1,1.000:20,11 1|0:43:1.423:4.405:0.093:181.6:182:10.13:1.24:0.45:0.00:C,14,2.124,G,28,2.270,T,1,0.010:28,14 0|0:34:1.125:2.332:0.059:0.0:96:0.00:4.09:0.26:0.00:G,34,2.332:34,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-10-09T12:59:49.617" LastActivityDate="2019-10-09T12:59:49.617" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10508" PostTypeId="2" ParentId="10505" CreationDate="2019-09-29T15:56:15.093" Score="2" Body="&lt;p&gt;Anyway the chances are this is gene duplication. It will be a housekeeping type gene or else histone that sort of thing. You simply map the different genes onto the various metazoan classes. In your case nematodes and arthropods are basal metazoan, so their is correct given the clade they form part of all all representatives is that gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is nothing weird you just need to read the deep phylogeneticists work like Tom Caveliar-Smith and Sandy Balfour, possibly Blair-Hedges&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are alot of eukaryotes prior to the invertebrates, just look at the Cambrian explosion&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-09-29T15:56:15.093" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10509" PostTypeId="1" AcceptedAnswerId="10510" CreationDate="2019-09-30T02:40:51.597" Score="0" ViewCount="51" Body="&lt;p&gt;Is it possible to filter out all the characters that come after the genotype, for example, I have a data &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM POS ID REF ALT FATHER MOTHER DAUGHTER1 DAUGHTER2 DAUGHTER3 SON1 SON2&#xA;1 736689 . T A 0/0:37:1.224:2.367:0.054:0.0:108:0.00:4.75:0.53:0.00:T,37,2.367:37,0 1/0:33:1.099:3.320:0.091:358.5:359:0.59:4.34:0.59:0.08:A,15,1.149,T,18,2.171:18,15 0|0:30:0.999:1.314:0.033:0.0:87:0.00:1.16:0.00:0.00:T,30,1.314:30,0 0|1:31:1.032:2.299:0.065:211.5:212:2.61:0.48:0.07:0.43:A,12,0.124,C,1,0.010,T,18,2.165:18,12 0|1:37:1.232:2.374:0.054:263.2:263:1.47:8.69:0.06:0.05:A,16,1.160,T,21,1.214:21,16 0|0:43:1.423:2.447:0.047:0.0:98:0.20:0.83:0.45:0.00:A,1,0.010,T,42,2.437:42,1 0|1:40:1.323:3.393:0.075:280.5:281:1.95:2.36:3.47:0.27:A,17,1.175,T,23,2.219:23,17&#xA;1 758663 . CGCC TTAG 0/0:27:0.893:0.296:0.000:0.0:90:5.15:0.85:3.67:0.01:27,0 0/1:21:0.699:0.234:0.000:2333.1:353:0.93:2.17:2.36:0.00:9,12 0|0:35:1.166:0.370:0.000:0.0:108:3.97:3.43:4.75:0.00:34,0 0|1:33:1.099:0.375:0.000:2820.8:447:0.07:8.69:0.53:0.00:17,15 0|1:23:0.766:0.232:0.000:1860.0:382:0.85:0.17:0.09:0.00:13,10 0|0:23:0.761:0.255:0.000:0.0:75:1.51:2.59:0.78:0.00:22,0 0|1:18:0.596:0.215:0.000:1686.8:275:0.00:2.17:0.10:0.00:9,9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;i want to filter out everything that comes after the genotype and have an output like below &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM POS ID REF ALT FATHER MOTHER DAUGHTER1 DAUGHTER2 DAUGHTER3 SON1 SON2&#xA;1 736689 . T A         0/0     1/0     0|0     0|1        0|1     0|0   0|1&#xA;1 758663 . CGCC TTAG   0/0     0/1     0|0     0|1        0|1     0|0   0|1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6271" LastEditorUserId="-1" LastEditDate="2019-09-30T06:20:57.403" LastActivityDate="2019-09-30T11:15:36.173" Title="Filter out other character after the genotype" Tags="|awk|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10510" PostTypeId="2" ParentId="10509" CreationDate="2019-09-30T06:24:33.317" Score="3" Body="&lt;p&gt;Using sed:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed 's,\([01][/|][01]\):[^ ]*,\1,g' file&#xA;#    ^  ^               ^     ^  ^   &#xA;#    |  |               |     |  *--- applies to all found pattern&#xA;#    |  |               |     *------ print the content inside the parenthesis&#xA;#    |  |               *------------ any character except a space&#xA;#    |  *---------------------------- the pattern to be kept&#xA;#    *------------------------------- substitution&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You also may want to format the output using &lt;code&gt;column&lt;/code&gt; command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed 's,\([01][/|][01]\):[^ ]*,\1,g' file | column -t&#xA;#CHROM  POS     ID  REF   ALT   FATHER  MOTHER  DAUGHTER1  DAUGHTER2  DAUGHTER3   SON1  SON2&#xA;1       736689  .   T     A     0/0     1/0     0|0        0|1        0|1        0|0   0|1&#xA;1       758663  .   CGCC  TTAG  0/0     0/1     0|0        0|1        0|1        0|0   0|1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note the comma separators in the sed command can replaced with other characters typically &quot;/&quot;&lt;/p&gt;&#xA;" OwnerUserId="6290" LastEditorUserId="3848" LastEditDate="2019-09-30T11:15:36.173" LastActivityDate="2019-09-30T11:15:36.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10511" PostTypeId="2" ParentId="10502" CreationDate="2019-09-30T08:26:09.477" Score="1" Body="&lt;p&gt;In R you can use &lt;code&gt;tidyr&lt;/code&gt;'s function &lt;code&gt;separate_rows&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;install.packages(&quot;tidyr&quot;) ## if not already installed&#xA;library(tidyr)&#xA;GO_results&amp;lt;-read.table(&quot;genes.txt&quot;)&#xA;separate_rows(GO_results, GO_results$V1, sep = &quot;/&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="939" LastEditorUserId="298" LastEditDate="2019-09-30T08:46:58.107" LastActivityDate="2019-09-30T08:46:58.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10512" PostTypeId="1" AcceptedAnswerId="10513" CreationDate="2019-09-30T09:09:28.777" Score="2" ViewCount="202" Body="&lt;p&gt;We send some samples to sequence and we got several (fastq.gz) files for each sample. The files are distributed at two or three folders with different dates (more than a week apart). The dates of the folders where the data is in:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;2019-08-12&#xA;2019-08-21&#xA;2019-08-21&#xA;2019-09-04&#xA;2019-09-14&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I asked the facility about why there are the same file in different folders, and they told us that:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;we have split sequencing of that project on different flowcells, so all&#xA;  files are correct and you have to merge the individual (duplicate) files&#xA;  to get the requested total read numbers.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How can different flow cells be from the same experiment and be run with more than a week of difference? (I thought that all the flow cells are run at the same time in each sequencing)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can be there some batch effect? How can I check if there is batch effect if we merge the files before trimming, mapping and counting?&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2019-09-30T10:08:44.863" LastActivityDate="2019-09-30T10:08:44.863" Title="Sequencing center returns data in several files" Tags="|ngs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10513" PostTypeId="2" ParentId="10512" CreationDate="2019-09-30T09:29:01.147" Score="5" Body="&lt;p&gt;Usually, a single sequencing library is created from each sample. Assuming you are using barcoded indexes to identify multiple samples, the index is attached to each library, and the libraries are pooled to make a library pool.  A library pool can be frozen, and is very stable.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A pool can be sequenced multiple times by pipetting a portion of the pool and supplying it to the sequencing process.  Each time, a sample of the molecules in the pool will be sequenced.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, it is true that sequencing runs are very repeatable, and that you will not suffer any batch effects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to check for batch effects you would need to analyse all of your sequence files separately, and check this first, or in parallel, to your main analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing you might want to check is whether the same library pool was sequenced multiple times, or whether separate pools or libraries were made for each sequencing run.  In the second case, there may be more likely to be batch effects.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastEditorUserId="48" LastEditDate="2019-09-30T10:08:28.593" LastActivityDate="2019-09-30T10:08:28.593" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10514" PostTypeId="2" ParentId="10442" CreationDate="2019-09-30T13:16:52.070" Score="1" Body="&lt;p&gt;I ended up fixing this by just putting this into the boilerplate code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;copy PolyA-MC_4, PolyA-MC&#xA;copy PolyA-MC_6, PolyA-MC&#xA;copy PolyA-MC_8, PolyA-MC&#xA;copy PolyA-MC_10, PolyA-MC&#xA;copy PolyA-MC_12, PolyA-MC&#xA;copy PolyA-MC_14, PolyA-MC&#xA;copy PolyA-MC_16, PolyA-MC&#xA;copy PolyA-MC_18, PolyA-MC&#xA;copy PolyA-MC_20, PolyA-MC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This copied my object such that I had one per size of label text, and then each label was placed on the object. As all labels on a given object are the same size, it worked! e.g. for one residue the labelling is now:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;set label_size,  4, PolyA-MC_4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Effectively, it's the same as what I was already doing, but instead of having ten copy filies in the directory only one is needed. Much easier for sending to people, and more efficient!&lt;/p&gt;&#xA;" OwnerUserId="6239" LastEditorUserId="6239" LastEditDate="2019-09-30T16:05:56.563" LastActivityDate="2019-09-30T16:05:56.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10515" PostTypeId="2" ParentId="8548" CreationDate="2019-10-01T10:15:25.140" Score="2" Body="&lt;p&gt;Here is how we do this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First break your gene model down into non-overlapping chunks, such that any possible gene model can be built from a combination of those chunks. E.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;transcript 1 |&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;|----------|&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;|---|&amp;gt;&amp;gt;&amp;gt;&amp;gt;|&#xA;transcript 2 |&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;|-------------|&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;|---|&amp;gt;&amp;gt;&amp;gt;&amp;gt;|&#xA;transcript 3 |&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;|-------------|&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;|&#xA;chunks       |+++1++|+2|+++++3++++|+++4++++|+5+|+6++|&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;this can be achieved using &lt;code&gt;cgat gtf2gtf --method=genes-to-unique-chunks&lt;/code&gt; from the &lt;code&gt;cgat-apps&lt;/code&gt; toolkit if you don't want to code it yourself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I call&lt;br&gt;&#xA;chunks 1, 4 and 6 constitutive exon&lt;br&gt;&#xA;chunk  3 constitutive intron&#xA;chunk  2 alternate intron&lt;br&gt;&#xA;chunk  5 annotated retained intron.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now use &lt;code&gt;featureCounts&lt;/code&gt; to count the number of reads in each chunk.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Test the chunk counts for usage using &lt;code&gt;DEXSeq&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Filter the results to just include chunks of type 2, 3 and 5 and recompute the FDRs.&lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2019-10-01T10:15:25.140" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10516" PostTypeId="1" CreationDate="2019-10-01T12:42:34.453" Score="2" ViewCount="115" Body="&lt;p&gt;I need advice on a suitable analysis of a dataset that has an akward design because we investigate a real-world situation. We  have two sites (A, B) where we can conduct the research. The hypothesis is: under-dimmed light, we catch less number of insects than under-fully lit street lights.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two sites each have ten street lights that were sampled during 16 nights of dimmed light and during 16 nights of full light. During these total of 32 nights, we sampled insects at each street-light pole (10 per site, 20 in total). For A and also at B, we also have temperature as a variable (nightly average, max, min) - a measure per site not per light pole.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to fit a model: number of  insects ~ dimming, temp_mean, temp_min, temp_max, site&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I  include &quot;site&quot; as categorical variable and would like to use the 32 nights (16 dimmed, 16 fully lit) as 32 unique samples (each night as one sample). I am not sure whether this is &quot;statistically sound&quot;. If not, does anybody have an idea on how to evaluate such data in order to prove the hypothesis? Would you go for t-test, PCA or something else?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;streetlight &amp;lt;- data.frame(site=c(rep(&quot;A&quot;,16),rep(&quot;B&quot;,16),rep(&quot;A&quot;,16),rep(&quot;B&quot;,16)), &#xA;                          dimming=c(rep(1,32),rep(0,32)),temp_mean=runif(32, min=20, max=24),t&#xA;                          emp_min=runif(32, min=12, max=19),temp_max=runif(32, min=25, max=28),&#xA;                          insects=round(runif(32, min=5, max=20),0))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6298" LastEditorUserId="48" LastEditDate="2020-03-16T10:41:38.120" LastActivityDate="2020-03-16T10:41:38.120" Title="How do I Investigate and test the hypothesis on the effect of street-light regime on insects?" Tags="|r|linear-regression|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10517" PostTypeId="2" ParentId="10516" CreationDate="2019-10-01T16:09:43.960" Score="0" Body="&lt;p&gt;Please avoid PCA. Go with logistic regression and work out the odds ratio. This is far and away the easiest most powerful method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Oh and you're likely to see a negative binomial distribution, very common for insect distributions. Your next step would be to get some DNA out of them&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-10-01T21:36:20.587" LastActivityDate="2019-10-01T21:36:20.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10518" PostTypeId="1" CreationDate="2019-10-02T06:02:26.640" Score="3" ViewCount="2364" Body="&lt;p&gt;I need to lightly verify a set of docking simulations from the reference RMSD of the outputs, and as we all know reference RMSD depends on the initial coordinates of the ligand set. What value of reference RMSD after docking from any starting coordinate would be considered outrageous to have?&lt;/p&gt;&#xA;" OwnerUserId="6299" LastActivityDate="2019-11-04T22:29:33.630" Title="What value of reference RMSD after docking simulation from any starting coordinate would be considered outrageous to have?" Tags="|proteins|docking|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10519" PostTypeId="1" CreationDate="2019-10-02T08:46:10.710" Score="0" ViewCount="261" Body="&lt;p&gt;This is an expansion of my previous question, &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/10491/how-to-edit-the-headers-of-multiple-fasta-files-from-multiple-folders&quot;&gt;How to edit the headers of multiple fasta files from multiple folders?&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My directories are organized as follows: one main directory, in which I have multiple project directories. Inside each project directory, there are multiple metagenome directories. Inside each metagenome directory, there is always a single .fna (fasta) file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Names: project directories start with 'mgp', metagenome directories start with 'mgm' and end with '.3'. They end in numbers, but the numbers are not related. (eg. project directory 'mgp83581' contains metagenome directory 'mgm4729322.3').&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want is a loop that would go through all these fasta files, and add to the headers codes corresponding to project and metagenome directory. For example, if the fasta file in project directory mgp83581, metagenome subdirectory mgm4729322.3 had the following headers...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&amp;gt;seq1&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&amp;gt;seq2&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;...then I would want to change it to:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&amp;gt;83581_322_seq1&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&amp;gt;83581_322_seq2&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, I want to add the number code associated with the project, and the last three digits of the 7-digit code associated with the metagenome (apart from the '.3' part of the file name).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is the answer that I got for my previous question, where I didn't have project directories, and where the 3-digit code for the metagenomes stayed in a manageable range. (thanks terdon!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;for i in {314..727}; do &#xA;    sed -i.bak &quot;s/^&amp;gt;/&amp;gt;${i}_/&quot; mgm4761${i}.3/*.fna&#xA;done&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using ubuntu. I would appreciate your help!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2019-10-02T12:20:53.717" Title="How to edit the headers of multiple fasta files from multiple folders? (recursively)" Tags="|fasta|bash|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10520" PostTypeId="2" ParentId="10519" CreationDate="2019-10-02T12:14:11.787" Score="1" Body="&lt;p&gt;You can edit your &lt;code&gt;for&lt;/code&gt; loop so as to include the paths to your files of interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The command below will give you the relative paths (relative to where you run the command, I think it is better to run this within your &lt;code&gt;main&lt;/code&gt; directory) of your files of interest:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ls mpg*/mgm*.3/*.fna&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can use the output of this command (your files of interest) to iterate over via your for loop. Your code will look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in {...}; do &#xA;    sed -i.bak &quot;s/^&amp;gt;/&amp;gt;${i}_/&quot; i&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-10-02T12:20:53.717" LastActivityDate="2019-10-02T12:20:53.717" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10521" PostTypeId="1" AcceptedAnswerId="10523" CreationDate="2019-10-03T02:06:32.153" Score="0" ViewCount="162" Body="&lt;p&gt;Is it possible to merge two files together using awk? I have two files with matched  Chrom and pos  column&lt;/p&gt;&#xA;&#xA;&lt;p&gt;file1 has the columns&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CHROM   POS ID  REF ALT QUAL    FILTER  INFO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;file2 has the columns &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CHROM POS ID REF ALT FATHER MOTHER DAUGHTER1 DAUGHTER2 DAUGHTER3 SON1 SON2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want the final merged file to have these columns&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CHROM   POS ID  REF ALT QUAL    FILTER  INFO FATHER MOTHER DAUGHTER1 DAUGHTER2 DAUGHTER3 SON1 SON2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6271" LastActivityDate="2019-10-04T15:29:14.577" Title="Merging two files together" Tags="|awk|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10522" PostTypeId="2" ParentId="10521" CreationDate="2019-10-03T02:43:05.083" Score="1" Body="&lt;p&gt;I did something like  below,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk 'FNR==NR{a[&lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;2]=&lt;span class=&quot;math-container&quot;&gt;$8 FS $&lt;/span&gt;12;next}{ print &lt;span class=&quot;math-container&quot;&gt;$0, a[$&lt;/span&gt;1 $2]}' file1.vcf file2.vcf &amp;gt; Final.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But then the output looks like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CHROM   POS ID  REF ALT QUAL    FILTER  FATHER MOTHER DAUGHTER1 DAUGHTER2 DAUGHTER3 SON1 SON2 INFO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is it possible to arrange the output to look like &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CHROM   POS ID  REF ALT QUAL    FILTER  INFO FATHER MOTHER DAUGHTER1 DAUGHTER2 DAUGHTER3 SON1 SON2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6271" LastEditorUserId="6271" LastEditDate="2019-10-03T05:54:29.673" LastActivityDate="2019-10-03T05:54:29.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10523" PostTypeId="2" ParentId="10521" CreationDate="2019-10-03T07:16:46.483" Score="2" Body="&lt;p&gt;If you just need to merge column together, you can use &lt;code&gt;paste&lt;/code&gt; command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;paste file1 &amp;lt;(cut -d\  -f6- file2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this case the whole &lt;code&gt;file1&lt;/code&gt; is taken and only part of &lt;code&gt;file2&lt;/code&gt; starting from column 6.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming your data are sorted, another alternative is to use &lt;code&gt;join&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;join -o '1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 2.6 2.7 2.8 2.9 2.10 2.11 2.12' file1 file2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;-o&lt;/code&gt; option allows to select column from either &lt;code&gt;file1&lt;/code&gt; or &lt;code&gt;file2&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6290" LastActivityDate="2019-10-03T07:16:46.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10525" PostTypeId="2" ParentId="10521" CreationDate="2019-10-03T16:50:23.510" Score="3" Body="&lt;p&gt;I'd recommend using R and either &lt;code&gt;merge&lt;/code&gt; or one of the &lt;code&gt;join&lt;/code&gt;s. Assuming your data is not too large, &lt;code&gt;data.frame&lt;/code&gt; + &lt;code&gt;read.delim&lt;/code&gt; should suffice to hold the two datasets. If not, try &lt;code&gt;data.table&lt;/code&gt; + &lt;code&gt;fread&lt;/code&gt;..&lt;/p&gt;&#xA;" OwnerUserId="650" LastEditorUserId="3848" LastEditDate="2019-10-04T15:29:14.577" LastActivityDate="2019-10-04T15:29:14.577" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10526" PostTypeId="2" ParentId="10521" CreationDate="2019-10-03T17:10:43.737" Score="1" Body="&lt;p&gt;It's not awk: but this is a fairly simple python/pandas script (assuming tab-delimited).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;file1&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CHROM   POS ID  REF ALT QUAL    FILTER  INFO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;file2&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CHROM POS ID REF ALT FATHER MOTHER DAUGHTER1 DAUGHTER2 DAUGHTER3 SON1 SON2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/python&#xA;import pandas as pd&#xA;&#xA;file1 = pd.read_csv('file1.txt', sep='\t')&#xA;file2 = pd.read_csv('file2.txt', sep='\t')&#xA;&#xA;# you can compare file1.shape vs. file2.shape to ensure they match&#xA;# you can also use fileN.sort_values(by=['POS', 'CHROM'], axis=1) to ensure sort matches&#xA;&#xA;file3 = pd.concat([file1, file2[['FATHER', 'MOTHER', 'DAUGHTER1', 'DAUGHTER2', 'DAUGHTER3', 'SON1', 'SON2']]], axis=1)&#xA;&#xA;file3.to_csv('merged_file.txt', sep='\t', index=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3384" LastEditorUserId="3384" LastEditDate="2019-10-03T17:17:52.217" LastActivityDate="2019-10-03T17:17:52.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10527" PostTypeId="1" CreationDate="2019-10-03T20:47:55.987" Score="1" ViewCount="321" Body="&lt;p&gt;I have some BED files that I have compressed with &lt;code&gt;bgzip&lt;/code&gt; and indexed with &lt;code&gt;tabix&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using version &lt;code&gt;1.9-392-gcfa32f7-dirty&lt;/code&gt; on the server side.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ /usr/local/bin/bgzip --version&#xA;bgzip (htslib) 1.9-392-gcfa32f7-dirty&#xA;Copyright (C) 2019 Genome Research Ltd.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The files are served from a subdirectory made available via &lt;code&gt;nginx&lt;/code&gt; on port 80 (http) and 443 (https).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is that, for some &lt;code&gt;bgzip&lt;/code&gt;-compressed files, I get a correct response to a &lt;code&gt;tabix&lt;/code&gt; query if I run it locally, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ tabix &quot;http://explore.altius.org/tabix/vB/hg19.15.all.KL.gz&quot; chr1:50669400-50669800&#xA;chr1    50669200    50669400    id:253347,qcat:[ [0,1], [0,2], [0,3], [0,4], [0,5], [0,6], [0,7], [0,8], [0,9], [0,10], [0,11], [0,12], [0,13], [0,14], [0.1764,15] ]&#xA;chr1    50669400    50669600    id:253348,qcat:[ [0,1], [0,2], [0,3], [0,4], [0,5], [0,6], [0,7], [0,8], [0,9], [0,10], [0,11], [0,12], [0,13], [0,14], [0.1764,15] ]&#xA;chr1    50669600    50669800    id:253349,qcat:[ [0,1], [0,2], [0,3], [0,4], [0,5], [0,6], [0,7], [0,8], [0,9], [0,10], [0,11], [0,12], [0,13], [0,14], [0.1764,15] ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, when I run this command from another host (such as my laptop), I get no result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ tabix &quot;http://explore.altius.org/tabix/vB/hg19.15.all.KL.gz&quot; chr1:50669400-50669800&#xA;$&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Running &lt;code&gt;echo $?&lt;/code&gt; gives a result of &lt;code&gt;0&lt;/code&gt; (no error). There is nothing in &lt;code&gt;nginx&lt;/code&gt; error logs to suggest an issue with http access. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other &lt;code&gt;bgzip&lt;/code&gt;-compressed files work, which further suggests a problem with either the compressed file or its &lt;code&gt;tbi&lt;/code&gt; index file, and not with the nginx configuration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From my laptop, for instance, the following file can be accessed:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ tabix &quot;http://explore.altius.org/tabix/vB/hg19.15.all.KLs.gz&quot; chr1:50669400-50669800&#xA;chr1    50669200    50669400    id:253347,qcat:[ [0,1], [0,2], [0,3], [0,4], [0,5], [0,6], [0,7], [0,8], [0,9], [0,10], [0,11], [0,12], [0,13], [0,14], [0.23635,15] ]&#xA;chr1    50669400    50669600    id:253348,qcat:[ [0,1], [0,2], [0,3], [0,4], [0,5], [0,6], [0,7], [0,8], [0,9], [0,10], [0,11], [0,12], [0,13], [0,14], [0.23635,15] ]&#xA;chr1    50669600    50669800    id:253349,qcat:[ [0,1], [0,2], [0,3], [0,4], [0,5], [0,6], [0,7], [0,8], [0,9], [0,10], [0,11], [0,12], [0,13], [0,14], [0.23635,15] ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've investigated the two test files and they are both sorted lexicographically by chromosome name and numerically by start and stop coordinates. I do not see any odd line endings. These two test files are both virtually identical, except for the data in the fourth column.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there anything I can do to further troubleshoot this and figure out why I cannot get a result from other hosts other than the server itself, and only for certain files? (Is there a debug version of htslib applications that would include verbose logging, say?)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In response to Jonathan Moore's comment, I &lt;code&gt;wget&lt;/code&gt; the bgzip-ed &lt;code&gt;gz&lt;/code&gt; archive and its associated &lt;code&gt;tbi&lt;/code&gt; index, and then try to query the file locally (not via http, but directly via the filesystem). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I get the following error message:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ tabix hg19.15.all.KL.gz chr1:50669400-50669800&#xA;[W::hts_idx_load2] The index file is older than the data file: hg19.15.all.KL.gz.tbi&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;tbi&lt;/code&gt; index file was made immediately after creating the &lt;code&gt;gz&lt;/code&gt; file, so I don't understand this error.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2019-10-04T20:04:21.083" LastActivityDate="2019-10-04T20:04:21.083" Title="How to troubleshoot access to tabix file via http or https?" Tags="|htslib|tabix|bgzip|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10528" PostTypeId="1" CreationDate="2019-10-04T05:37:25.740" Score="0" ViewCount="77" Body="&lt;p&gt;I have two files, &lt;code&gt;file1&lt;/code&gt; and &lt;code&gt;file2&lt;/code&gt;. I want to match &lt;code&gt;file1&lt;/code&gt; chrom with &lt;code&gt;file2&lt;/code&gt; chrom, and if there is a match then I want to check &lt;code&gt;file2&lt;/code&gt; if the position of the match chrom1 is within the exon region that is in &lt;code&gt;file1&lt;/code&gt;. i want to do this with the whole file and print out rows that match and the position is within the exonic regions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, &lt;code&gt;file1&lt;/code&gt; has a chrom 1 and POS 736689 which match chrom 1 in &lt;code&gt;file2&lt;/code&gt;, now I take &lt;code&gt;POS 736689&lt;/code&gt; from &lt;code&gt;file1&lt;/code&gt; and check if it is within the exon region &lt;code&gt;29553,30563,30975,   30039,30667,31097,...&lt;/code&gt; in &lt;code&gt;file2&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is within the region it will print out the file1 row that matches the pattern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The exon regions can be read as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;29553 to 30039&#xA;30563 to 30667&#xA;30975 to 31097&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;file1 look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CHROM   POS REF ALT FATHER  MOTHER  DAUGHTER1   DAUGHTER2   DAUGHTER3   SON1    SON2    INFO&#xA;1   736689  T   A   0/0 1/0 0|0 0|1 0|1 0|0 0|1 AN=2184;AC=6    &#xA;1   758663  CGCC    TTAG    0/0 0/1 0|0 0|1 0|1 0|0 0|1 AC=2;AN=2184    &#xA;1   762005  G   C   1/0 0/0 1|0 1|0 1|0 1|0 0|0 AN=2184;AC=13   &#xA;1   771950  C   T   0/0 0/1 0|1 0|0 0|0 0|1 0|0 AC=110;AN=2184  &#xA;1   772172  C   T   0/1 0/0 0|0 0|0 0|0 0|0 1|0 AN=2184;AC=8    &#xA;1   790590  C   T   0/1 0/1 0|0 0/1 0/1 0|0 1|1 AN=2184;AC=84   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;file2 look like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chrom   Start   End&#xA;chr1    29553,30563,30975,  30039,30667,31097,&#xA;chr1    53048,54829,    53067,54936,&#xA;chr1    69090,  70008,&#xA;chr1    160445,161313,  160690,161525,&#xA;chr1    324755,327551,  326514,328453,&#xA;chr1    367639, 368634,&#xA;chr1    459655,461153,461750,   459992,461288,461954,&#xA;chr1    523008,529614,  523081,530148,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6271" LastEditorUserId="6244" LastEditDate="2019-10-08T15:39:20.973" LastActivityDate="2019-10-08T15:39:20.973" Title="match two files and check if position of chrom is within exon region" Tags="|awk|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10529" PostTypeId="2" ParentId="10528" CreationDate="2019-10-04T06:44:30.987" Score="2" Body="&lt;p&gt;Despite an &lt;code&gt;awk&lt;/code&gt; solution is favored by the OP, here is an &lt;code&gt;R&lt;/code&gt; solution. (The task, especially dealing with exon coordinates in file2, is rather complex for a straightforward &lt;code&gt;awk&lt;/code&gt; solution, at least for me).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;res &amp;lt;- data.frame() # empty df for storing results&#xA;&#xA;for (i in 1:nrow(file1)) {&#xA;&#xA;  chr &amp;lt;- file1[i,1] # get chr&#xA;  pos &amp;lt;- file1[i,2] # get position of interest&#xA;  print(&quot;pos&quot;)&#xA;  print(pos)&#xA;&#xA;  if(chr %in% file2$chrom) {&#xA;&#xA;    file2_subset &amp;lt;- file2[file2$chrom == chr,] # subset file2&#xA;&#xA;    for(j in 1:nrow(file2_subset)) {&#xA;&#xA;      # create exon start - exon end pairs&#xA;&#xA;      exon_start &amp;lt;- file2_subset[j,2]&#xA;      exon_start &amp;lt;- as.numeric(unlist(strsplit(exon_start,&quot;,&quot;)))&#xA;      print(exon_start)&#xA;&#xA;      exon_end &amp;lt;- file2_subset[j,3]&#xA;      exon_end &amp;lt;- as.numeric(unlist(strsplit(exon_end,&quot;,&quot;)))&#xA;      print(exon_end)&#xA;&#xA;      # if position of interest lie within exons&#xA;      # need to be careful if start/end position are counted as exon&#xA;&#xA;      if(any(pos &amp;gt;= exon_start &amp;amp; pos &amp;lt;= exon_end)) {&#xA;        print(file1[i,])&#xA;        res &amp;lt;- rbind(res, file1[i,]) # write line i from file1 if satisfies above conds&#xA;      }&#xA;    }&#xA;  }&#xA;}&#xA;&#xA;&amp;gt; res&#xA;  CHROM    POS REF ALT FATHER MOTHER DAUGHTER1 DAUGHTER2 DAUGHTER3 SON1 SON2          INFO&#xA;6  chr1 790590   C   T    0/1    0/1       0|0       0/1       0/1  0|0  1|1 AN=2184;AC=84&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-10-04T12:39:49.010" LastActivityDate="2019-10-04T12:39:49.010" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10530" PostTypeId="1" CreationDate="2019-10-04T09:34:43.390" Score="0" ViewCount="241" Body="&lt;p&gt;I have RNA-seq data in response to treatment vs non response; By machine learning I selected three principle components likely can predict the response based on the gene expression. Now I have something like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(pca&lt;span class=&quot;math-container&quot;&gt;$rotation[1:10,1:6])&#xA;               PC1          PC2          PC3          PC4           PC5          PC6&#xA;ACTB   -0.01858031 -0.005810636  0.014939832 -0.009594126 -0.0118313644 -0.011534401&#xA;ATP5F1 -0.01822611  0.014851900 -0.016401920 -0.015616379  0.0254850111 -0.010225516&#xA;DDX5   -0.02477827 -0.020573978  0.002199966 -0.009296852  0.0006222345  0.014506614&#xA;EEF1G  -0.02803284  0.001126758 -0.010570521 -0.017035205  0.0007583815  0.011225639&#xA;GAPDH  -0.02691309  0.005503228  0.006446502 -0.012686738 -0.0289737909  0.007036631&#xA;NCL    -0.03019765 -0.003011778 -0.015534611 -0.002019556 -0.0126401245  0.013977142&#xA;&amp;gt; &#xA;&amp;gt; dim(pca$&lt;/span&gt;rotation)&#xA;[1] 3962   56&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Machine learning says PC 6, PC 21 and PC 39 are better; how I know from 3962 genes in each of these PCs which genes is more contribute to response to treatment?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2019-10-04T09:34:43.390" Title="Selecting genes with more contribution from PCA" Tags="|rna-seq|modelling|pca|features|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="10531" PostTypeId="2" ParentId="10398" CreationDate="2019-10-04T13:08:13.820" Score="1" Body="&lt;p&gt;According to the book you mentionned &lt;a href=&quot;https://www.academia.edu/12825649/Introduction_to_Genomics_Second_Edition-_Arthur_M_Lesk&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to Genomics SecondEdition by Arthur MLesk&lt;/a&gt; p.113&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(a) What fraction of a genome could you expect to assemble from eightfold coverage? &#xA;(b) What total gap length would you expect in an assembly of a 2 Mb target genome size from eightfold coverage? &#xA;(c) How many gaps would you expect in an assembly of a 2 Mb target genome size from an eightfold coverage of fragments with a read length of 500? &#xA;(d) You want to sequence a 4 Mb genome by the shotgun method, by assembling random fragments with read length 500. What coverage would you require, to expect no more than four gaps, assuming no complications arising from repetitive sequences or far-from-equimolar base composition?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm agreed that whitout knowing &lt;span class=&quot;math-container&quot;&gt;$ N $&lt;/span&gt; it's not obvious to answer this last question so perhaps they're a mistake in the statement of this exercise.&#xA;As Devon Ryan suggested, let's assume &lt;code&gt;eightfold coverage&lt;/code&gt; means the coverage &lt;span class=&quot;math-container&quot;&gt;$ C $&lt;/span&gt; = 8X&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So we can calculate the number of &lt;em&gt;reads&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ N = \frac{GC}{L} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ N = \frac{4000000\times{8}}{500} = 64000 $&lt;/span&gt; &lt;em&gt;reads&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then we can calculate the required coverage to expect no more than four gaps:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ N( e^{-C}) = 4 $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ e^{C} = \frac{N}{4} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ C = log{\frac{N}{4}} $&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ C $&lt;/span&gt; = 10X&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(PS: I would like to thank @LordOfRacoons to help answer this question.)&lt;/p&gt;&#xA;" OwnerUserId="4992" LastEditorUserId="4992" LastEditDate="2019-10-04T13:14:52.417" LastActivityDate="2019-10-04T13:14:52.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10532" PostTypeId="2" ParentId="10528" CreationDate="2019-10-04T16:04:36.343" Score="1" Body="&lt;p&gt;BEDTools already has a function to take a vcf and bedfile as input and look for intersections.  Most people would get the exon coordinates into bed format, and use that.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-10-04T16:04:36.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10533" PostTypeId="2" ParentId="10528" CreationDate="2019-10-04T20:27:10.517" Score="1" Body="&lt;p&gt;Make a sorted BED file called &lt;code&gt;exons.bed&lt;/code&gt; and run BEDOPS &lt;code&gt;bedmap&lt;/code&gt; to map variants to exons:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --echo --echo-map --delim '\t' --skip-unmapped exons.bed &amp;lt;(vcf2bed &amp;lt; variants.vcf) &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To make a sorted BED file from your exons, I'd suggest using &lt;code&gt;sort-bed&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would also suggest using &lt;code&gt;awk&lt;/code&gt; to make sure that the chromosome naming scheme is consistent between exons and variants, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ vcf2bed &amp;lt; variants.vcf | awk '{ print 'chr$&lt;/span&gt;0' }' &amp;gt; variants.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then use that result, instead, for mapping:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --echo --echo-map --delim '\t' --skip-unmapped exons.bed variants.bed &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If chromosome names are not consistent, mapping or other set operations with BEDOPS or other toolkits will not work.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2019-10-04T20:27:10.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10534" PostTypeId="1" AcceptedAnswerId="10536" CreationDate="2019-10-05T00:46:01.320" Score="1" ViewCount="821" Body="&lt;p&gt;I building a robust amino acid phylogeny with adequate robustness support (see previous post &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/10365/how-to-make-a-robust-amino-acid-phylogeny-with-adequate-robustness-support&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;This is a brief summary of what I have done:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I performed a BLASTp analysis using a protein sequence from species &lt;em&gt;&lt;strong&gt;A&lt;/strong&gt;&lt;/em&gt; as query against the full NCBI protein database,&lt;/li&gt;&#xA;&lt;li&gt;downloaded all the hits in fasta format and I added the sequences I identified with the query sequence in my &lt;em&gt;&lt;strong&gt;target&lt;/strong&gt;&lt;/em&gt; genome, which is closely related to species &lt;em&gt;&lt;strong&gt;A&lt;/strong&gt;&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;li&gt;I then aligned the sequences using the Compute - Alignment - MUSCLE option in Seqotron and I saved my alignment file in Phylip format.&lt;/li&gt;&#xA;&lt;li&gt;I tried to run this command line in RAxML to build a robust maximum likelihood tree:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;code&gt;raxmlHPC -m PROTGAMMAILG -n output.tre -o outgroup-p 10000 -s ~/Desktop/alignment_file.phy&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, the program showed the following error:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Error, the outgroup name &amp;quot;outgroup-p&amp;quot; you specified can not be found in the alignment, exiting ....&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; how can I build an outgroup for my phylogenetic tree, and how do I include it in my alignment?&lt;/p&gt;&#xA;&lt;p&gt;I appreciate your help and suggestions!&lt;/p&gt;&#xA;" OwnerUserId="5116" LastEditorUserId="3848" LastEditDate="2020-07-27T06:49:19.367" LastActivityDate="2020-07-27T06:49:19.367" Title="How to define an outgroup to build a robust amino acid tree" Tags="|sequence-alignment|phylogenetics|phylogeny|duplications|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10535" PostTypeId="2" ParentId="7048" CreationDate="2019-10-05T08:37:09.873" Score="0" Body="&lt;p&gt;InIGV you can visuals the alignments of individual reads to the reference as contained in the .bam file. I guess what you want to is get that information into a file containing all the (paired) read sequences aligned to a particular region, with padding for the missing data. That way you could inspect all SNPs or sequencing errors, check possible haplotypes across a few SNPs, use these for phylogenetic analysis etc.&#xA;I am also looking for such tool and best I could find so far is an &lt;a href=&quot;https://support.bioconductor.org/p/64748/&quot; rel=&quot;nofollow noreferrer&quot;&gt;answer on BioConductor&lt;/a&gt;. It refers to some code in R that can be used for this purpose:  &lt;code&gt;genomicalignments::stackStringsFromBam()&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6305" LastEditorUserId="57" LastEditDate="2019-10-08T15:41:11.857" LastActivityDate="2019-10-08T15:41:11.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10536" PostTypeId="2" ParentId="10534" CreationDate="2019-10-05T09:53:57.450" Score="2" Body="&lt;p&gt;I think you have a bug&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;raxmlHPC -m PROTGAMMAILG -n output.tre -o outgroup -p 10000 -s /Users)/username/Desktop/alignment_file.phy&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Try the above, where username is your username. I'll check my codes later. The most important bit is the space between the -p and word outgroup. You can omit the &lt;code&gt;-p 10000&lt;/code&gt; it is only needed  for very geeky maximum likelihood and sets the random number stream. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-10-05T16:30:20.227" LastActivityDate="2019-10-05T16:30:20.227" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="10537" PostTypeId="1" CreationDate="2019-10-06T06:51:09.633" Score="1" ViewCount="690" Body="&lt;p&gt;I want to calculate Fst by vcf tools and GATK. &#xA;I did this steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;for creating gvcf:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;gatk HaplotypeCaller -R ref.scf.fasta -I input.bam -ERC GVCF -O out.g.vcf&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;for combine:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;gatk CombineGVCFs -R ref.scf.fasta --variant 1.g.vcf --variant 2.g.vcf --variant 3.g.vcf --variant 4.g.vcf --variant 5.g.vcf --variant 6.g.vcf --variant 7.g.vcf --variant 8.g.vcf -O outt.vcf&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;for genotype:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;gatk GenotypeGVCFs -R ref.scf.fasta --variant outt -O outt.vcflist&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;for Fst:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;vcftools --vcf outt.vcflist --weir-fst-pop 1.txt --weir-fst-pop 2.txt --out 1_vs_2&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There was no error till step 3 and successfully done. But, I got error in step 4:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error: Require Genotypes in VCF file in order to output Fst statistics.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I used the last version (&lt;strong&gt;GATK/4.1.3.0-Java-1.8&lt;/strong&gt;) and the last version of vcf tools (&lt;strong&gt;VCFtools/0.1.16&lt;/strong&gt;), but I still have the same error!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would you please advise me?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best,&#xA;Razi&lt;/p&gt;&#xA;" OwnerUserId="6310" LastEditorUserId="3541" LastEditDate="2019-10-07T06:26:49.060" LastActivityDate="2019-10-07T06:26:49.060" Title="vcf tools Error: Require Genotypes in VCF file in order to output Fst statistics" Tags="|gatk|vcftools|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10538" PostTypeId="1" CreationDate="2019-10-06T21:16:09.873" Score="1" ViewCount="258" Body="&lt;p&gt;I have a fasta file, like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;TRINITY_DN100000_c1_g1::TRINITY_DN100000_c1_g1_i3::g.3039::m.3039 TRINITY_DN100000_c1_g1::TRINITY_DN100000_c1_g1_i3::g.3039  ORF type:complete len:100 (-) TRINITY_DN100000_c1_g1_i3:1027-1326(-)&#xA;MVWIKFRGLHRVLTSTPLVKSGKTPSQTWAFLDISVELIVFLFLNVHKSPMPHFKIYSEA&#xA;FSEEWSLLWLQYSRHLIQKPKPWQIKIELLHLCCCNRLC*&#xA;&amp;gt;TRINITY_DN100000_c1_g6::TRINITY_DN100000_c1_g6_i2::g.84365::m.84365 TRINITY_DN100000_c1_g6::TRINITY_DN100000_c1_g6_i2::g.84365  ORF type:complete len:112 (-) TRINITY_DN100000_c1_g6_i2:379-714(-)&#xA;MEMMQEIIPFAREMLSARPSKGTMKVYLVGGTFAVLGIVSGMVEAACSLFPEQEESTLTK&#xA;LMEDCLTVTAQNQEPQTFIPEDDEQDAEMEAKAKDLPMFRQRRMSFRAHAS*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;if I want to only keep the second header, like this (the amino acid sequence keep unchanged). which command could i USE?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;TRINITY_DN100000_c1_g1_i3&#xA;MVWIKFRGLHRVLTSTPLVKSGKTPSQTWAFLDISVELIVFLFLNVHKSPMPHFKIYSEA&#xA;FSEEWSLLWLQYSRHLIQKPKPWQIKIELLHLCCCNRLC*&#xA;&amp;gt;TRINITY_DN100000_c1_g6_i2&#xA;MEMMQEIIPFAREMLSARPSKGTMKVYLVGGTFAVLGIVSGMVEAACSLFPEQEESTLTK&#xA;LMEDCLTVTAQNQEPQTFIPEDDEQDAEMEAKAKDLPMFRQRRMSFRAHAS*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I know a command &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; sed 's/::.*//' input &amp;gt; output &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;can only keep the first header &lt;code&gt;&amp;gt;TRINITY_DN100000_c1_g1&lt;/code&gt; if I want to keep the second header with the isoform information &lt;code&gt;&amp;gt;TRINITY_DN100000_c1_g1_i3&lt;/code&gt;, how should I correct this command?&lt;/p&gt;&#xA;" OwnerUserId="6312" LastActivityDate="2019-10-07T15:56:40.913" Title="too long header for fasta file" Tags="|rna-seq|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10539" PostTypeId="2" ParentId="10538" CreationDate="2019-10-06T21:39:33.333" Score="1" Body="&lt;p&gt;A 2 step workaround is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Get rid of everything after the piece of info you want, that is anything starting with &lt;code&gt;::&lt;/code&gt; and goes on with &lt;code&gt;g&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;sed &quot;s/::g.*//&quot; fasta &amp;gt; fasta2&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&amp;gt;TRINITY_DN100000_c1_g1::TRINITY_DN100000_c1_g1_i3&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Get rid of the &quot;first header&quot; as you call it, after the first step, it is rather straightforward:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;sed &quot;s/&amp;gt;TRINITY_.*::/&amp;gt;/&quot; fasta2 &amp;gt; fasta3&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&amp;gt;TRINITY_DN100000_c1_g1_i3&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;@terdon showed that multiple &lt;code&gt;sed&lt;/code&gt; commands can be used when separated by &lt;code&gt;;&lt;/code&gt; in GNU &lt;code&gt;sed&lt;/code&gt; (default on Linux):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed &quot;s/::g.*//; s/&amp;gt;TRINITY_.*::/&amp;gt;/&quot; fasta &amp;gt; modified_fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="298" LastEditDate="2019-10-07T11:16:48.133" LastActivityDate="2019-10-07T11:16:48.133" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10540" PostTypeId="2" ParentId="10483" CreationDate="2019-10-06T21:53:08.607" Score="0" Body="&lt;p&gt;Use bedtools intersect. You can select the various combinations of outputs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/intersect.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedtools.readthedocs.io/en/latest/content/tools/intersect.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6314" LastActivityDate="2019-10-06T21:53:08.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10541" PostTypeId="1" AcceptedAnswerId="10542" CreationDate="2019-10-07T02:58:29.980" Score="1" ViewCount="1013" Body="&lt;p&gt;We are trying to use &lt;a href=&quot;https://deeptools.readthedocs.io/en/develop/content/tools/bamCompare.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;deeptools&lt;/a&gt; for analysis of ATAC Seq datasets. We have datasets with different sequencing depths and are wondering if bamCompare's SES based normalization is appropriate for comparing two ATACSeq datasets as opposed to ChipSeq datasets. If such normalization is okay to perform, does it make sense to compare difference in values across comparisons? Let's say I do bamCompare on A and B using SES, and then I do bamCompare (using SES) on A and C. Will it then make sense to compare the two comparisons (A-B &amp;amp; A-C)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If such normalization isn't okay, what are the best tools to normalize ATAC-Seq datasets with different sequencing depths so that one can perform comparisons like above? &#xA;Thanks for any suggestions or comments.&lt;/p&gt;&#xA;" OwnerUserId="809" LastActivityDate="2019-10-07T06:37:50.597" Title="Is it okay to use deeptools bamCompare (SES normalization) for comparisons across different ATAC-Seq datasets?" Tags="|normalization|atac-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10542" PostTypeId="2" ParentId="10541" CreationDate="2019-10-07T06:37:50.597" Score="1" Body="&lt;p&gt;I would be very hesitant to use SES normalization in a case like this. We recommend that people run plotFingerprint first and see if there is good separation between the samples and only then using SES. I would not expect there to be drastic separation between the samples for ATAC-seq data, so I don't think it will work well. Our standard ATAC-seq normalization is RPGC (aka, &quot;1x normalization&quot;). That should work well for your use-case as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an aside, if you really want to make statistically-backed comparisons between groups I can recommend the CSAW package.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-10-07T06:37:50.597" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="10543" PostTypeId="2" ParentId="10538" CreationDate="2019-10-07T09:27:53.767" Score="2" Body="&lt;p&gt;Here's one way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sed -E 's/&amp;gt;[^:]*::([^:]*)::.*/&amp;gt;\1/' file.fa &#xA;&amp;gt;TRINITY_DN100000_c1_g1_i3&#xA;MVWIKFRGLHRVLTSTPLVKSGKTPSQTWAFLDISVELIVFLFLNVHKSPMPHFKIYSEA&#xA;FSEEWSLLWLQYSRHLIQKPKPWQIKIELLHLCCCNRLC*&#xA;&amp;gt;TRINITY_DN100000_c1_g6_i2&#xA;MEMMQEIIPFAREMLSARPSKGTMKVYLVGGTFAVLGIVSGMVEAACSLFPEQEESTLTK&#xA;LMEDCLTVTAQNQEPQTFIPEDDEQDAEMEAKAKDLPMFRQRRMSFRAHAS*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The idea is to match a &lt;code&gt;&amp;gt;&lt;/code&gt;, then as many non&lt;code&gt;:&lt;/code&gt; as possible (&lt;code&gt;[^:]*&lt;/code&gt;), and then capture the string between the first &lt;code&gt;::&lt;/code&gt; and the second &lt;code&gt;::&lt;/code&gt;. Finally, you replace the entire line with the captured string. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you're satisfied this does what you need, you can edit the original file with &lt;code&gt;-i&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed -i -E 's/&amp;gt;[^:]*::([^:]*)::.*/&amp;gt;\1/' file.fa &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Alternatively, set &lt;code&gt;awk&lt;/code&gt;'s field separator to &lt;code&gt;::&lt;/code&gt; and print the second field only on lines that start with &lt;code&gt;&amp;gt;&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk -F'::' '/^&amp;gt;/ ? $&lt;/span&gt;0=&quot;&amp;gt;&quot;$2 : 1' file.fa &#xA;&amp;gt;TRINITY_DN100000_c1_g1_i3&#xA;MVWIKFRGLHRVLTSTPLVKSGKTPSQTWAFLDISVELIVFLFLNVHKSPMPHFKIYSEA&#xA;FSEEWSLLWLQYSRHLIQKPKPWQIKIELLHLCCCNRLC*&#xA;&amp;gt;TRINITY_DN100000_c1_g6_i2&#xA;MEMMQEIIPFAREMLSARPSKGTMKVYLVGGTFAVLGIVSGMVEAACSLFPEQEESTLTK&#xA;LMEDCLTVTAQNQEPQTFIPEDDEQDAEMEAKAKDLPMFRQRRMSFRAHAS*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-10-07T15:56:40.913" LastActivityDate="2019-10-07T15:56:40.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10545" PostTypeId="2" ParentId="9360" CreationDate="2019-10-08T14:17:19.303" Score="2" Body="&lt;p&gt;There is not a simple way and there are factor that you also need to consider before starting.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Are you using gene names or gene accession numbers/id (NCBI/Uniprot)?&lt;/li&gt;&#xA;&lt;li&gt;If not, are the names the current standard ones (&lt;em&gt;e.g.&lt;/em&gt; scraped names from old papers is a definite no)&lt;/li&gt;&#xA;&lt;li&gt;PDB residue index is rarely the same as protein index.&lt;/li&gt;&#xA;&lt;li&gt;Do you want a perfect match or do you want say 80% homology?&lt;/li&gt;&#xA;&lt;li&gt;Which PDBs do you want for a protein?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;h2&gt;Personal example&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For a website I have developed I have a route that gets all the PDBs for a given gene. (&lt;a href=&quot;https://michelanglo.sgc.ox.ac.uk/name&quot; rel=&quot;nofollow noreferrer&quot;&gt;enter link description here&lt;/a&gt;). The way I did it is that I parsed Uniprot using a &lt;a href=&quot;https://github.com/matteoferla/protein-module-for-VENUS&quot; rel=&quot;nofollow noreferrer&quot;&gt;custom parser&lt;/a&gt; and made a dictionary/DB for each species with many synonyms for a gene going to a Uniprot ID and then from there I have another dictionary/DB that goes from Uniprot to PDBs if any. I went for Uniprot as I needed other data too (e.g. features) to present to the user. Including a renumbering option.&#xA;All this is way overkill for what you need. However, if you specify the species I can easily share a JSON file online.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;SIFTs&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If point 3 is important and you have Uniprot IDs I would suggest going with &lt;a href=&quot;https://www.ebi.ac.uk/pdbe/docs/sifts/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SIFTs&lt;/a&gt;. It has the offsets and has mapping for Uniprot IDs (pdb_chain_uniprot.tsv).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Gene names&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In terms of mapping gene names to NCBI/Uniprot/Ensembl IDs, there are mapping tables everywhere. But if you may have obscure synonyms, using the NCBI API is a good option. &#xA;Uniprot data is better for protein and has PDB codes and chains, but the API gives a XML reply which is a bit annoying (I wrote a parser for it, but I think there was going to be one in Biopython).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Homologues&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If you would like homologues, say your organism is human and you are okay with mouse, you can either use the threaded SwissModel models or use the blast query in RCSB PDB (not the PDBe, which has better data normally) or NCBI set to PDB database.&#xA;Swissmodel actually contains also PDB structures, so that may be a good starting point if your organism is &lt;a href=&quot;https://swissmodel.expasy.org/repository&quot; rel=&quot;nofollow noreferrer&quot;&gt;one of those where precompiled exists&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Parts&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Some proteins, especially mammalian are formed by separate domains linked together, especially the docking scaffolds. In which case you may care for a specific model within the gene, hence the indexing issue and the potential getting features.&#xA;If you want a single PDB structure for each gene and do not care about the technical I have &lt;a href=&quot;https://github.com/matteoferla/protein_fuser&quot; rel=&quot;nofollow noreferrer&quot;&gt;some Pymol-using py3 code&lt;/a&gt; to fuse models together with the N and C termini aligned on line. But I strongly discourage that and would argue for protein feature based routes.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2019-10-08T14:17:19.303" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10546" PostTypeId="2" ParentId="8967" CreationDate="2019-10-08T16:25:18.120" Score="1" Body="&lt;p&gt;The first option. Using a pre-existing tool, such as PROKKA etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Diversity.&lt;/strong&gt; Bacterial diversity is staggering and species such as E. coli and Salmonella, which are closely related diverged when mammals and dinosaurs/birds diverged, so searching nucleotide sequences is a bad idea as you will get nothing. Especially as you are hoping to find distant sequences that may be missed by the commonly used annotation tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Operons.&lt;/strong&gt; Bacteria, especially thermophiles, often group their protein functionally in operons. Therefore, you can use the principle of &lt;code&gt;guilt by association&lt;/code&gt; to guess what a mysterious protein does. It is extremely powerful. Say you have Voet &amp;amp; Voet handy, a pathway with a hole and a mysterious protein you can guess easily what it does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, if you have a protein that you are particularly interested in, I would suggest making your own PSI PSSM file. The automated system cannot read the literature or do some quick checks to see when to stop, so may be overly conservative with the clusters.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2019-10-08T16:25:18.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10547" PostTypeId="1" CreationDate="2019-10-08T22:04:41.407" Score="3" ViewCount="226" Body="&lt;p&gt;Biopython provides a (bit unintuitive) API to access to the Kyoto Encyclopedia of Genes and Genomes (&lt;a href=&quot;https://www.genome.jp/kegg/&quot; rel=&quot;nofollow noreferrer&quot;&gt;KEGG&lt;/a&gt;). I am trying to make use of it, but the output is quite unhandy as a string. What is the best way to parse the data into a &lt;code&gt;pandas.DataFrame()&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.KEGG import REST as Kegg&#xA;import io&#xA;&#xA;pd.read_table(kegg.kegg_get('K02545'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="73" LastEditDate="2021-05-06T10:25:47.123" LastActivityDate="2021-05-06T10:25:50.830" Title="How to load output from the kegg API in biopython into a pandas dataframe?" Tags="|python|kegg|pandas|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10548" PostTypeId="1" AcceptedAnswerId="10551" CreationDate="2019-10-09T03:13:08.547" Score="0" ViewCount="74" Body="&lt;p&gt;I am trying to find this pattern on a large file but the output doesn't look good. The output doesn't seem to reflect the pattern.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Father = 1|0 or 1/0&#xA;&#xA;Mother = 110 or 1/0&#xA;&#xA;Daughter 1 = 1|0 or 1/0&#xA;&#xA;Daughter 2 = 1|1 or 1/1&#xA;&#xA;Daughter 3 = 0|0 or 0/0&#xA;&#xA;Son 1 = 1|0 or 1/0&#xA;&#xA;Son 2 = 1|1 or 1/1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The content of the file1 looks like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CHROM   POS REF ALT FATHER  MOTHER  DAUGHTER1   DAUGHTER2   DAUGHTER3   &#xA;&#xA;SON1    SON2    INFO    FREQUECY&#xA;1   1226852 G   C   1/0 0/0 1|0 1|0 1|0 1|0 0|0 AN=2184;AC=12   0.005494505&#xA;1   1847936 C   T   0/1 1/1 1|1 1|1 1|1 1|1 0|1 AC=1;AN=2184    0.000457875&#xA;1   2428427 C   G   0/1 0/0 0|0 0|0 0|0 0|0 1|0 AC=4;AN=2184    0.001831502&#xA;1   2515616 G   A   1/0 1/0 1/0 1|1 1|1 1/0 1/0 AC=1;AN=2184    0.000457875&#xA;1   3801895 T   C   1/0 1/0 0|0 1/0 1/0 0|0 1|1 AC=10;AN=2184   0.004578755&#xA;1   3833321 T   C   1/0 1/0 1|1 1/0 1/0 1|1 0|0 AN=2184;AC=801  0.366758242&#xA;1   9009352 A   G   0/1 0/1 1|1 0/1 0/1 1|1 0|0 AC=90;AN=2184   0.041208791&#xA;1   9328077 G   A   1/1 0/0 1|0 1|0 1|0 1|0 1|0 AC=1;AN=2184    0.000457875&#xA;1   10684412    C   T   0/1 0/0 1|0 1|0 1|0 1|0 0|0 AN=2184;AC=2    0.000915751&#xA;1   11782120    G   A   1/0 0/0 0|0 0|0 1|0 1|0 0|0 AC=3;AN=2184    0.001373626&#xA;1   12034879    A   T   0/1 1/1 0|1 0|1 0|1 0|1 1|1 AC=3;AN=2184    0.001373626&#xA;1   16064465    C   A   1/0 0/0 0|0 0|0 0|0 0|0 1|0 AN=2184;AC=4    0.001831502&#xA;1   16111170    G   A   1/0 1/0 1/0 0|0 0|0 1/0 1/0 AC=1;AN=2184    0.000457875&#xA;1   16200229    C   A   0/0 1/0 0|0 0|1 0|1 0|0 0|1 AC=9;AN=2184    0.004120879&#xA;1   16258118    C   T   0/1 0/0 1|0 1|0 1|0 1|0 0|0 AN=2184;AC=1    0.000457875&#xA;1   17966719    A   G   0/0 0/1 0|1 0|0 0|0 0|1 0|0 AC=0;AN=2184    0&#xA;1   19751079    CG  C   0/1 0/1 1|1 0|0 0|0 0/1 0/1 AN=2184;AC=1    0.000457875&#xA;1   19955045    G   A   0/0 1/0 0|0 0|1 0|1 0|0 0|1 AN=2184;AC=4    0.001831502&#xA;1   21060356    G   A   1/0 1/0 1/0 1/0 1/0 1|1 0|0 AN=2184;AC=541  0.247710623&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My awk script look like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '(&lt;span class=&quot;math-container&quot;&gt;$8~/1\/1/ || $&lt;/span&gt;8~/1\\|1/) &amp;amp;&amp;amp; (&lt;span class=&quot;math-container&quot;&gt;$9~/0\/0/ ||$&lt;/span&gt;9~/0\\|0/) &amp;amp;&amp;amp; (&lt;span class=&quot;math-container&quot;&gt;$7~/^1\\|0/||$&lt;/span&gt;7~/1\/0/) &amp;amp;&amp;amp; (&lt;span class=&quot;math-container&quot;&gt;$10~/1\\|0/||$&lt;/span&gt;7~/1\/0/) &amp;amp;&amp;amp; (&lt;span class=&quot;math-container&quot;&gt;$10~/0\/0/ ||  $&lt;/span&gt;10~/0\\|0/) &amp;amp;&amp;amp; (&lt;span class=&quot;math-container&quot;&gt;$5~/1\/0/|| $&lt;/span&gt;5~/1\\|0/) &amp;amp;&amp;amp; (&lt;span class=&quot;math-container&quot;&gt;$11~/0\/1/||$&lt;/span&gt;11~/0\\|1/)&amp;amp;&amp;amp; (&lt;span class=&quot;math-container&quot;&gt;$6~/1\\|0/||$&lt;/span&gt;6~/1\/0/) &amp;amp;&amp;amp; ($12 ~/1\/1/ ||12~/1\\|1/){print}' file1 &amp;gt;file2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6271" LastEditorUserId="6271" LastEditDate="2019-10-09T08:36:55.413" LastActivityDate="2019-10-09T11:23:50.410" Title="awk find pattern" Tags="|awk|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10549" PostTypeId="1" CreationDate="2019-10-09T09:48:08.663" Score="2" ViewCount="69" Body="&lt;p&gt;I have a list of about 900 E.coli genome ids (Genbank plus NCTC ids), e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NCTC8196_1&#xA;NCTC11105_1&#xA;CP027464.1&#xA;CP027459.1&#xA;CP027363.1&#xA;CP027388.1&#xA;CP027355.1&#xA;CP027675.1&#xA;CP027445.1&#xA;CP027325.1&#xA;CP027219.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I would like to find pathogenicity metadata for them so that I can classify the molecular data I found with the metadata.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to get a table of metadata from these genomes that I can use to associate with my molecular data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NOTES:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After some searching online, I found Enterobase (&lt;a href=&quot;http://enterobase.warwick.ac.uk&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://enterobase.warwick.ac.uk&lt;/a&gt;), but it seems the site is mostly dedicated to MLST typing, thus they use different naming conventions than the Genbank id. Maybe there is a way to link the info in Enterobase with Genbank ids somehow?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like Enterobase contains project ids, e.g. PRJNA218110. These project ids can also be found on the other side in the GenBank entries, for which we could associate the GenBank chromosomal ids.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Searching for PRJNA218110 in Enterobase gives a very long list of entries that seem to be from Illumina sequencing data, but in this list, the 'Path/Nonpath' column is always empty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Enterobase seems to contain lots of entries with useful information, such as: the strain is pathogenic, and produced diarrhea in humans. So a table with entries like the one below would be very useful:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Genbank_id: CPXXXXXXX.1&#xA;Path/Nonpath: Pathogenic&#xA;Disease: Diarrhea&#xA;&#xA;Genbank_id: CPYYYYYYY.1&#xA;Path/Nonpath: Nonpathogenic&#xA;Disease: NA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="180" LastEditorUserId="180" LastEditDate="2019-10-09T10:51:33.983" LastActivityDate="2019-11-09T11:01:20.270" Title="where to find pathogenicity metadata for E.coli genomes" Tags="|sequence-annotation|functional-annotation|microbial-genomics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10550" PostTypeId="2" ParentId="10534" CreationDate="2019-10-09T09:54:18.990" Score="2" Body="&lt;p&gt;To clear up what may be a possible misunderstanding about RAxML: RAxML (and most/all other maximum likelihood phylogenetic inference programs using &lt;strong&gt;reversible&lt;/strong&gt; models of sequence evolution) explicitly infer unrooted trees. This is because, for a reversible model of sequence evolution, changing the position of the root does not affect the likelihood of the tree. Therefore, as you will see in the RAxML manual, specifying an outgroup is explicitly a display option - it doesn't affect the inference in any way. This means that it's usually easier to just infer the tree in RAxML without specifying any outgroup, and then use any tree viewing software (e.g. &lt;a href=&quot;http://tree.bio.ed.ac.uk/software/figtree/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FigTree&lt;/a&gt;, &lt;a href=&quot;http://phylotree.hyphy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;phylotree.js&lt;/a&gt;, or equivalent) to reroot the tree as you need to.  &lt;/p&gt;&#xA;" OwnerUserId="2841" LastActivityDate="2019-10-09T09:54:18.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10551" PostTypeId="2" ParentId="10548" CreationDate="2019-10-09T11:10:00.610" Score="2" Body="&lt;p&gt;Example output of what you got and what you expected would be helpful, without these one can only guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all your input does not have a row satisfying the criteria you have given, so I added two dummy rows, one with &lt;code&gt;/&lt;/code&gt; and one with &lt;code&gt;|&lt;/code&gt; used as separator.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code I used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '(&lt;span class=&quot;math-container&quot;&gt;$5~/1\|0/ || $&lt;/span&gt;5~/1\/0/) &amp;amp;&amp;amp; (&lt;span class=&quot;math-container&quot;&gt;$6~/1\|0/ || $&lt;/span&gt;6~/1\/0/) &amp;amp;&amp;amp; (&lt;span class=&quot;math-container&quot;&gt;$7~/1\|0/ || $&lt;/span&gt;7~/1\/0/) &amp;amp;&amp;amp; (&lt;span class=&quot;math-container&quot;&gt;$8~/1\|1/ || $&lt;/span&gt;8~/1\/1/) &amp;amp;&amp;amp; (&lt;span class=&quot;math-container&quot;&gt;$9~/0\|0/ || $&lt;/span&gt;9~/0\/0/) &amp;amp;&amp;amp; (&lt;span class=&quot;math-container&quot;&gt;$10~/1\|0/ || $&lt;/span&gt;10~/1\/0/) &amp;amp;&amp;amp; (&lt;span class=&quot;math-container&quot;&gt;$11~/1\|1/ || $&lt;/span&gt;11~/1\/1/) {print}' test.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And its output (input is yours + 2 dummy rows):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Dummy  dummy D D 1/0 1/0 1/0 1/1 0/0 1/0 1/1 dummy dummy&#xA;Dummy  dummy D D 1|0 1|0 1|0 1|1 0|0 1|0 1|1 dummy dummy&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The code is basically yours, except for:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I escaped &lt;code&gt;|&lt;/code&gt; with one &lt;code&gt;\&lt;/code&gt; instead of two&lt;/li&gt;&#xA;&lt;li&gt;I changed this part: &lt;code&gt;($10~/1\\|0/||$7~/1\/0/)&lt;/code&gt;. I think &lt;code&gt;$7&lt;/code&gt; should have been &lt;code&gt;$10&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;I removed a &lt;code&gt;^&lt;/code&gt; from one of your conditions, I believe they all should start with &lt;code&gt;^&lt;/code&gt; though, per &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/10506/awk-find-sickle-cell-anemia-from-father-mother-son-daughter/10507#10507&quot;&gt;@terdon's answer on a related question&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;I believe the code above does what you want, but when I run code above with two backslashes to escape I get the following output and could not figure out why!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   3801895 T   C   1/0 1/0 0|0 1/0 1/0 0|0 1|1 AC=10;AN=2184   0.004578755&#xA;1   16200229    C   A   0/0 1/0 0|0 0|1 0|1 0|0 0|1 AC=9;AN=2184    0.004120879&#xA;1   19955045    G   A   0/0 1/0 0|0 0|1 0|1 0|0 0|1 AN=2184;AC=4    0.001831502&#xA;Dummy  dummy D D 1/0 1/0 1/0 1/1 0/0 1/0 1/1 dummy dummy&#xA;Dummy  dummy D D 1|0 1|0 1|0 1|1 0|0 1|0 1|1 dummy dummy&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-10-09T11:23:50.410" LastActivityDate="2019-10-09T11:23:50.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10552" PostTypeId="1" CreationDate="2019-10-09T12:03:48.943" Score="3" ViewCount="3009" Body="&lt;p&gt;I have a bunch of &lt;a href=&quot;https://github.com/lh3/miniasm/blob/master/PAF.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;PAF files&lt;/a&gt; resulting from the alignments of fastq files on a reference genome with &lt;a href=&quot;https://lh3.github.io/minimap2/minimap2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimap2&lt;/a&gt;. I would like to convert them into SAM/BAM format so I can use &lt;code&gt;samtools mpileup&lt;/code&gt; to recover the read coverage at filtered positions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I've looked at &lt;code&gt;paftools.js&lt;/code&gt; and found &lt;code&gt;sam2paf&lt;/code&gt; but no &lt;code&gt;paf2sam&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am open to suggestions but I'd like to avoid mapping the reads again. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have opened an issue on &lt;a href=&quot;https://github.com/lh3/minimap2/issues/493&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimap2's github&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Answer from Heng Li :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;You need to rerun alignment. No tools exist to convert paf to sam.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;why there is no paf2sam in paftools.js&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;It is technically possible, but someone needs to write the code.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="6273" LastEditorUserId="6273" LastEditDate="2019-10-10T11:59:58.307" LastActivityDate="2020-03-06T23:42:56.633" Title="Convert PAF format to SAM/BAM format" Tags="|file-formats|format-conversion|sam|minimap2|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10553" PostTypeId="1" CreationDate="2019-10-09T16:20:24.797" Score="1" ViewCount="302" Body="&lt;p&gt;I am currently using Seurat to do the scRNA-seq clustering analysis. After this, I am planning to use FateID to do the downstream analysis, could someone teach me how to convert or extract a Seurat object as the input for FateID?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="4150" LastActivityDate="2019-10-22T21:06:04.120" Title="Convert or extract Seurat object as the input of FateID" Tags="|single-cell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10554" PostTypeId="2" ParentId="10553" CreationDate="2019-10-09T17:00:45.683" Score="1" Body="&lt;p&gt;&lt;code&gt;Seurat&lt;/code&gt; objects have various &lt;code&gt;slot&lt;/code&gt;s where different types and forms of data are stored. For &lt;code&gt;FateID&lt;/code&gt; you would need an expression matrix (with cells as columns and genes as rows) and this can be extracted via:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seurat_object@assays$RNA@data&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here &lt;code&gt;RNA&lt;/code&gt; is the assay (you can have multiple assays per sample, RNA, protein, ...) and &lt;code&gt;data&lt;/code&gt; is normalized data. You can extract counts or scaled data instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using &lt;code&gt;Seurat v2&lt;/code&gt;, drop the &lt;code&gt;@assays$RNA&lt;/code&gt; part.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-10-22T21:06:04.120" LastActivityDate="2019-10-22T21:06:04.120" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10555" PostTypeId="1" AcceptedAnswerId="10556" CreationDate="2019-10-09T20:43:35.500" Score="0" ViewCount="989" Body="&lt;p&gt;I am using WGCNA to identify Consensus modules. I get a warning message when completing the following step:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Call the network topology analysis function for each set in turn&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Warning:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Warning message: executing %dopar% sequentially: no parallel backend registered&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does anyone no what this error code means as I cannot plot the scale free analysis, with soft thresholding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;powers = c(seq(4,10, by=1), seq(12,20, by=2))&#xA;powerTables = vector(mode= &quot;list&quot;, length = nSets)&#xA;for( set in 1:nSets)&#xA;powerTables[[set]] = list(data=pickSoftThreshold(multiExpr[[set]]$data, powerVector= powers,&#xA;verbose =2)[[2]])&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="4398" LastEditorUserId="4398" LastEditDate="2019-10-10T07:54:32.613" LastActivityDate="2019-10-10T07:54:32.613" Title="R WGCNA error code" Tags="|r|wgcna|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10556" PostTypeId="2" ParentId="10555" CreationDate="2019-10-09T21:09:17.287" Score="0" Body="&lt;p&gt;I think you forgot to load (or install) the &lt;code&gt;doParallel&lt;/code&gt; package as the &lt;strike&gt;error&lt;/strike&gt; warning message is complaining about the lack of a parallel backend. &lt;code&gt;library(&quot;doParallel&quot;)&lt;/code&gt; should do the trick.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-10-10T07:18:01.950" LastActivityDate="2019-10-10T07:18:01.950" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10558" PostTypeId="1" AcceptedAnswerId="10575" CreationDate="2019-10-09T22:10:06.837" Score="1" ViewCount="755" Body="&lt;p&gt;I have following table.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(n = c(29L, 11L, 10L, 10L), domain = c(1L, 5L, &#xA;6L, 32L)), row.names = c(NA, -4L), class = c(&quot;tbl_df&quot;, &quot;tbl&quot;, &#xA;&quot;data.frame&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I need to retrieve rows from another table that corresponds to the values of the 1st column of the above table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did use following code by manually entering the column values of the table.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;target &amp;lt;- c(&quot;1&quot;, &quot;5&quot;, &quot;6&quot;, &quot;32&quot;)&#xA;selected_domains = filter(gene_domains, domain %in% target)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gene_domains is the second table.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have other tables that have many rows and entering manually is impossible. So I will be grateful if someone can help me with this issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need, this is the second table (gene_domains).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(gene = c(&quot;ENSG00000230594 ENSG00000171155 ENSG00000224089 ENSG00000230347 ENSG00000236446 ENSG00000186471 ENSG00000101892 ENSG00000182890 ENSG00000232119 ENSG00000131721 ENSG00000101882 ENSG00000101883 ENSG00000242362 ENSG00000226685 ENSG00000125352 ENSG00000236126 ENSG00000237957 ENSG00000005893 ENSG00000125355 ENSG00000226600 ENSG00000125356 ENSG00000203989 ENSG00000226929 ENSG00000228517 ENSG00000177485 ENSG00000226023 ENSG00000236371 ENSG00000278646 ENSG00000158290&quot;, &#xA;&quot;ENSG00000176774 ENSG00000176746 ENSG00000232030 ENSG00000188408&quot;, &#xA;&quot;ENSG00000198205 ENSG00000215174 ENSG00000165591 ENSG00000198455 ENSG00000186787 ENSG00000204271 ENSG00000147059&quot;, &#xA;&quot;ENSG00000029993 ENSG00000166049 ENSG00000183862 ENSG00000102181 ENSG00000013619 ENSG00000130032 ENSG00000171100 ENSG00000160131 ENSG00000063601 ENSG00000147378 ENSG00000102195&quot;, &#xA;&quot;ENSG00000147099 ENSG00000067177 ENSG00000184388 ENSG00000225396 ENSG00000268994 ENSG00000198034 ENSG00000125931 ENSG00000269502 ENSG00000184911 ENSG00000275520&quot;, &#xA;&quot;ENSG00000102081 ENSG00000176988&quot;, &quot;ENSG00000050030 ENSG00000131263 ENSG00000187969 ENSG00000147100&quot;, &#xA;&quot;ENSG00000102174 ENSG00000184735 ENSG00000175809 ENSG00000165186&quot;, &#xA;&quot;ENSG00000185448&quot;, &quot;ENSG00000101977 ENSG00000101981&quot;, &quot;ENSG00000177504 ENSG00000205642&quot;, &#xA;&quot;ENSG00000130150 ENSG00000101958 ENSG00000181544&quot;, &quot;ENSG00000260802 ENSG00000077279 ENSG00000077274 ENSG00000101901&quot;, &#xA;&quot;ENSG00000062096 ENSG00000205667 ENSG00000101825&quot;, &quot;ENSG00000169933&quot;, &#xA;&quot;ENSG00000134597 ENSG00000102078 ENSG00000056277 ENSG00000147262&quot;, &#xA;&quot;ENSG00000147256 ENSG00000171054 ENSG00000147255&quot;, &quot;ENSG00000238210 ENSG00000169551 ENSG00000186376&quot;, &#xA;&quot;ENSG00000181191 ENSG00000130052 ENSG00000090776&quot;, &quot;ENSG00000102409 ENSG00000182916 ENSG00000166681 ENSG00000185222 ENSG00000180964 ENSG00000204065 ENSG00000133134&quot;, &#xA;&quot;ENSG00000197565 ENSG00000101844 ENSG00000101842 ENSG00000101843&quot;, &#xA;&quot;ENSG00000267978 ENSG00000010404 ENSG00000123584 ENSG00000241489 ENSG00000268738 ENSG00000171116 ENSG00000269556 ENSG00000197620 ENSG00000185247 ENSG00000156009&quot;, &#xA;&quot;ENSG00000169906 ENSG00000047230 ENSG00000182287 ENSG00000182798 ENSG00000169249 ENSG00000126010&quot;, &#xA;&quot;ENSG00000005889 ENSG00000130741 ENSG00000123130 ENSG00000184831 ENSG00000165182 ENSG00000130066 ENSG00000174010&quot;, &#xA;&quot;ENSG00000204086&quot;, &quot;ENSG00000001497 ENSG00000147065&quot;, &quot;ENSG00000076770 ENSG00000123728&quot;, &#xA;&quot;ENSG00000169084 ENSG00000214717&quot;, &quot;ENSG00000073464&quot;, &quot;ENSG00000132446&quot;, &#xA;&quot;ENSG00000126970&quot;, &quot;ENSG00000147246&quot;, &quot;ENSG00000173674 ENSG00000184368 ENSG00000173681&quot;, &#xA;&quot;ENSG00000046647 ENSG00000198759 ENSG00000046653 ENSG00000196459 ENSG00000176896 ENSG00000123595 ENSG00000046651&quot;, &#xA;&quot;ENSG00000123561 ENSG00000123572&quot;, &quot;ENSG00000183304 ENSG00000177138&quot;, &#xA;&quot;ENSG00000234469&quot;, &quot;ENSG00000196368 ENSG00000224109&quot;, &quot;ENSG00000004848&quot;, &#xA;&quot;ENSG00000174460 ENSG00000131724 ENSG00000175556&quot;, &quot;ENSG00000131725&quot;, &#xA;&quot;ENSG00000087842 ENSG00000165197&quot;, &quot;ENSG00000182220 ENSG00000234636 ENSG00000185753 ENSG00000238205 ENSG00000180182&quot;, &#xA;&quot;ENSG00000251192 ENSG00000147121 ENSG00000065923 ENSG00000102218 ENSG00000147119&quot;, &#xA;&quot;ENSG00000165175&quot;, &quot;ENSG00000158301 ENSG00000198932 ENSG00000125962&quot;, &#xA;&quot;ENSG00000196767&quot;, &quot;ENSG00000188158&quot;, &quot;ENSG00000198798 ENSG00000214107 ENSG00000099399 ENSG00000120289 ENSG00000169297&quot;, &#xA;&quot;ENSG00000156697 ENSG00000122121 ENSG00000188706 ENSG00000122122 ENSG00000085185&quot;, &#xA;&quot;ENSG00000102362 ENSG00000101811 ENSG00000174225 ENSG00000188917 ENSG00000182489 ENSG00000007952&quot;, &#xA;&quot;ENSG00000131831 ENSG00000177324&quot;, &quot;ENSG00000102172 ENSG00000230797&quot;, &#xA;&quot;ENSG00000215301 ENSG00000188937&quot;, &quot;ENSG00000147223 ENSG00000089682 ENSG00000133131 ENSG00000133138 ENSG00000165376&quot;, &#xA;&quot;ENSG00000002586&quot;, &quot;ENSG00000215113 ENSG00000147162 ENSG00000186810 ENSG00000215115 ENSG00000147174&quot;, &#xA;&quot;ENSG00000186288 ENSG00000186462&quot;, &quot;ENSG00000101888 ENSG00000176076&quot;, &#xA;&quot;ENSG00000102128 ENSG00000133169&quot;, &quot;ENSG00000157514 ENSG00000170935 ENSG00000147224 ENSG00000147234&quot;, &#xA;&quot;ENSG00000165194&quot;, &quot;ENSG00000079482&quot;, &quot;ENSG00000147003 ENSG00000130234 ENSG00000102010 ENSG00000165195&quot;, &#xA;&quot;ENSG00000146950 ENSG00000101850&quot;, &quot;ENSG00000183837 ENSG00000235961 ENSG00000198681&quot;, &#xA;&quot;ENSG00000158164 ENSG00000198908&quot;, &quot;ENSG00000008056 ENSG00000126756 ENSG00000126759 ENSG00000078061 ENSG00000126767 ENSG00000102265&quot;, &#xA;&quot;ENSG00000198814&quot;, &quot;ENSG00000268009 ENSG00000165584 ENSG00000269791&quot;, &#xA;&quot;ENSG00000173698&quot;, &quot;ENSG00000187268 ENSG00000205542 ENSG00000123594&quot;, &#xA;&quot;ENSG00000197976 ENSG00000196433&quot;, &quot;ENSG00000102230 ENSG00000067992&quot;, &#xA;&quot;ENSG00000005302&quot;, &quot;ENSG00000125363&quot;, &quot;ENSG00000189037&quot;, &quot;ENSG00000269405&quot;, &#xA;&quot;ENSG00000147394 ENSG00000147400 ENSG00000147383&quot;, &quot;ENSG00000184788 ENSG00000124429 ENSG00000147180&quot;, &#xA;&quot;ENSG00000072506 ENSG00000158423 ENSG00000072501&quot;, &quot;ENSG00000102226 ENSG00000102225 ENSG00000182872 ENSG00000130985 ENSG00000130988 ENSG00000147123&quot;, &#xA;&quot;ENSG00000122145&quot;, &quot;ENSG00000147117 ENSG00000147124&quot;, &quot;ENSG00000008086&quot;, &#xA;&quot;ENSG00000187808 ENSG00000198918&quot;, &quot;ENSG00000101940 ENSG00000068354 ENSG00000015285 ENSG00000101945 ENSG00000102317 ENSG00000102312 ENSG00000147155&quot;, &#xA;&quot;ENSG00000130054&quot;, &quot;ENSG00000273696 ENSG00000269586 ENSG00000270946 ENSG00000271449 ENSG00000181433 ENSG00000278085 ENSG00000169446 ENSG00000278289&quot;, &#xA;&quot;ENSG00000196664 ENSG00000101916&quot;, &quot;ENSG00000091482 ENSG00000185915&quot;, &#xA;&quot;ENSG00000102158 ENSG00000226784 ENSG00000131174&quot;, &quot;ENSG00000101846&quot;, &#xA;&quot;ENSG00000171659 ENSG00000171657&quot;, &quot;ENSG00000260548 ENSG00000133124&quot;, &#xA;&quot;ENSG00000147130 ENSG00000147166 ENSG00000169562 ENSG00000147140&quot;, &#xA;&quot;ENSG00000126012&quot;, &quot;ENSG00000130119&quot;, &quot;ENSG00000187516 ENSG00000165169&quot;, &#xA;&quot;ENSG00000102109 ENSG00000126768 ENSG00000102100 ENSG00000102145 ENSG00000094631 ENSG00000068308 ENSG00000187682 ENSG00000102103 ENSG00000102096&quot;, &#xA;&quot;ENSG00000120509 ENSG00000120500 ENSG00000147127 ENSG00000186912 ENSG00000204195&quot;, &#xA;&quot;ENSG00000006756 ENSG00000157399 ENSG00000056998&quot;, &quot;ENSG00000184083&quot;&#xA;), domain = c(1L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L, 11L, 12L, &#xA;13L, 14L, 15L, 17L, 18L, 20L, 21L, 28L, 30L, 31L, 32L, 33L, 34L, &#xA;35L, 37L, 38L, 40L, 42L, 43L, 44L, 45L, 47L, 48L, 49L, 51L, 53L, &#xA;54L, 56L, 57L, 59L, 60L, 61L, 62L, 63L, 64L, 65L, 66L, 67L, 68L, &#xA;69L, 70L, 71L, 72L, 73L, 75L, 77L, 78L, 79L, 81L, 82L, 83L, 84L, &#xA;85L, 89L, 90L, 91L, 96L, 99L, 102L, 105L, 108L, 109L, 111L, 113L, &#xA;116L, 117L, 119L, 122L, 125L, 129L, 130L, 131L, 132L, 133L, 136L, &#xA;142L, 143L, 144L, 145L, 150L, 161L, 165L, 168L, 169L, 174L, 175L, &#xA;183L, 184L, 187L, 190L, 191L, 194L)), row.names = c(NA, 103L), class = &quot;data.frame&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="856" LastActivityDate="2019-10-11T15:50:56.140" Title="How to retrieve values from one table that corresponds to the values in another table in R" Tags="|r|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10559" PostTypeId="1" CreationDate="2019-10-09T22:22:23.303" Score="1" ViewCount="1767" Body="&lt;p&gt;How could I extract reads from bam files by their read groups @RG ?, I've got one file containing all reads for 5 samples, the SM is appears NONE, So, I want to extract each sample reads by the @RG tags. &lt;/p&gt;&#xA;" OwnerUserId="6328" LastActivityDate="2020-04-23T15:34:43.573" Title="Extract reads from bam files by their @RG" Tags="|sam|ngs|reads|genome-sequencing|bacteria|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10560" PostTypeId="2" ParentId="10555" CreationDate="2019-10-09T23:00:21.903" Score="1" Body="&lt;p&gt;What you see is a warning, not an error. Your calculation will run fine, just slower. Unless you see other errors, you should be able to complete all steps of the analysis.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2019-10-09T23:00:21.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10561" PostTypeId="1" CreationDate="2019-10-10T00:22:40.407" Score="1" ViewCount="44" Body="&lt;p&gt;I have already asked this question in another forum but have not got an appropriate answer, so wondering if anyone here can help me?&#xA;I have a big file with over 3 million columns (SNPs genotypes) and 800 rows (one row for each individual). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first row is the SNPs_ID name with the SNP position on each chromosome (see the example below please), in total there are 17 chromosomes. I want to extract SNPs for each chromosome and store them in a separate file.&#xA;I have done it with &quot;R&quot;, but I am looking for a faster solution and I am wondering if anyone can help me to figure out another solution with bash, perl or python codes?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here is an example of my input file (SNPs on chromosome 1,2,3,4...,17)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;Ha412HOChr01:180159&quot; &quot;Ha412HOChr01:210724&quot; &quot;Ha412HOChr01:303270&quot; &quot;Ha412HOChr01:303280&quot;....... &quot;Ha412HOChr17:303402&quot;&#xA;0 1 0 0 ......0&#xA;0 1 0 0 ......0&#xA;0 1 0 0 ......0&#xA;0 2 0 0 ......0&#xA;0 1 1 1 ......1&#xA;0 2 0 0 ......0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and this is an example of my desired output for example for chromosome 1:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;out.chrom1&#xA;&quot;Ha412HOChr01:180159&quot; &quot;Ha412HOChr01:210724&quot; &quot;Ha412HOChr01:303270&quot; &quot;Ha412HOChr01:303280&quot; &#xA;0 1 0 0 &#xA;0 1 0 0&#xA;0 1 0 0 &#xA;0 2 0 0 &#xA;0 1 1 1 &#xA;0 2 0 0 &#xA;0 0 0 0 &#xA;0 2 0 0&#xA;0 1 2 2 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1749" LastEditorUserId="298" LastEditDate="2019-10-11T09:07:39.067" LastActivityDate="2019-10-11T09:07:39.067" Title="storing SNPs in the genotype file for each chromosome in a separate file" Tags="|python|perl|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10562" PostTypeId="2" ParentId="10561" CreationDate="2019-10-10T07:28:02.960" Score="2" Body="&lt;p&gt;I believe a &lt;code&gt;pandas&lt;/code&gt; approach would be fast enough (probably faster than any other Python approach without pandas).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After converting your table to a &lt;code&gt;pandas data frame&lt;/code&gt;, you can do something like &lt;code&gt;df.filter(regex=(&quot;.*Chr01&quot;))&lt;/code&gt; for the first chromosome then write the resulting data to file with the &lt;code&gt;to_csv()&lt;/code&gt; method.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;data = pd.read_csv('input.txt', delimiter = &quot;\t&quot;, index_col = False)&#xA;&#xA;for i in [&quot;Chr01&quot;, &quot;Chr02&quot;, &quot;Chr03&quot;]:&#xA;    regex_pattern = &quot;.*&quot; + i&#xA;    print(data.filter(regex=regex_pattern))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-10-10T21:44:46.147" LastActivityDate="2019-10-10T21:44:46.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10563" PostTypeId="1" AcceptedAnswerId="10567" CreationDate="2019-10-10T08:53:32.530" Score="2" ViewCount="218" Body="&lt;p&gt;I am studying ChIP-Seq data in HeLa cells and I've started using the RPGC normalisation of deepTool's bamCoverage. MACS2 also uses this normalisation for its peak calling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am seeing a large number of peaks (like, over 50% of my peaks) at the centromere of each chromosome, and the RPGC normalised BW files also show this increased signal at the centromere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does this mean that my protein really is at the centromere of each chromosome, or is it some normalisation bias I'm not aware of?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's the general aspect of my peak calling (in green). Notice the large increase at the centromere of each chromosome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uzO8q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uzO8q.png&quot; alt=&quot;General appearance of chromosomes&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here's a zoom on the centromere of chromosome 6. In blue at the top is raw BW, then there's a lane for input normalised BW but not RPGC, then green is the peak calling by MACS2, and black is RPGC normalised BW.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sBC9i.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sBC9i.png&quot; alt=&quot;Zoom onto Chr6 centromere&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6316" LastEditorUserId="6316" LastEditDate="2019-10-10T10:58:46.643" LastActivityDate="2023-03-11T15:28:26.407" Title="RPGC normalisation creates artefacts at centromere" Tags="|normalization|chip-seq|deeptools|centromere|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10564" PostTypeId="2" ParentId="10549" CreationDate="2019-10-10T10:50:15.683" Score="2" Body="&lt;p&gt;you can use BacDive. This is the largest database for bacterial metadata.:&#xA;&lt;a href=&quot;https://bacdive.dsmz.de/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bacdive.dsmz.de/&lt;/a&gt;&#xA;e.g. you can search for the sequence accession &quot;NCTC8196&quot; and you would get the metadata of the according strain:&#xA;&lt;a href=&quot;https://bacdive.dsmz.de/search?search=NCTC8196&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bacdive.dsmz.de/search?search=NCTC8196&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What kind of data is available, is highly dependent on the strain. Usually type strains are more rich on data that others.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For further questions don't hesitate to contact us:&#xA;contact@bacdive.de&lt;/p&gt;&#xA;" OwnerUserId="6331" LastActivityDate="2019-10-10T10:50:15.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10565" PostTypeId="1" AcceptedAnswerId="10566" CreationDate="2019-10-10T12:16:32.337" Score="2" ViewCount="106" Body="&lt;p&gt;I have a microarray gene expression matrix like this by weird gene IDs in rows. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(mat[1:10,1:5])&#xA;&#xA;&amp;gt; dim(mat)&#xA;[1] 39302    76&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There is matched gene symbol for each of these identifiers in another matrix &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; &#xA;&#xA;&amp;gt; dim(matched)&#xA;[1] 23107     1&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How I can have matched gene symbol with probe identifiers in the row names of my expression matrix please? The problem is, for one gene symbol we may &#xA;I tried &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; merged &amp;lt;- merge(mat, matched) &#xA;Error: cannot allocate vector of size 6.8 Gb&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;&lt;em&gt;EDITED&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tmp = paste(matched[rownames(mat)],rownames(mat),sep=&quot;_&quot;)&#xA;&amp;gt; rownames(mat) = tmp&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDITED&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my matrix after matching prob identifiers to gene symbol&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(array[,1:10,1:5])&#xA;               GSM482796 GSM482797 GSM482798 GSM482799&#xA;1         OR2T6    0.0171   -0.1100   -0.0394   -0.0141&#xA;2          EBF1    0.1890    0.0222    0.0832    0.0459&#xA;3 DKFZp686D0972    1.9400    0.2530    0.3770    0.8310&#xA;4        ATP8B4   -0.1490    0.0690   -0.0637   -0.0527&#xA;5      NOTCH2NL    0.1540   -0.3880    0.2160   -0.0812&#xA;6        SPIRE1    0.2920    0.1690    0.5500    0.1430&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but now for some genes I have several probes. For example for gene A I have several matched probes. So I have repetition for gene A. How I can take mean over the expression of repeated genes please and having an unique value?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-10-17T19:59:01.010" LastActivityDate="2019-10-17T19:59:01.010" Title="How I swap rownames in these data?" Tags="|r|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10566" PostTypeId="2" ParentId="10565" CreationDate="2019-10-10T12:45:12.537" Score="2" Body="&lt;p&gt;You need to think about whether to take the average for all probes belonging to one gene, or simply append the gene name to the probe id.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is simply adding a gene name, maybe try this below, note I am assuming that all your rownames of mat can be found in matched&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# check whether all rownames are in matched&#xA;table(rownames(mat) %in% rownames(matched))&#xA;# rename the rows&#xA;tmp = paste(matched[match(rownames(mat),rownames(matched)),1],rownames(mat),sep=&quot;_&quot;)&#xA;rownames(mat) = tmp&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastEditorUserId="5077" LastEditDate="2019-10-10T13:58:57.220" LastActivityDate="2019-10-10T13:58:57.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10567" PostTypeId="2" ParentId="10563" CreationDate="2019-10-10T13:04:24.893" Score="0" Body="&lt;h2&gt;UPDATE&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Here is some other protein ChIPs that I ran through the same normalisation:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/PyOMt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PyOMt.png&quot; alt=&quot;Centromere with other proteins&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, they don't seem to be enriched at the centromere. So unless anyone brings other contributions, I will accept this as answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;RPGC normalisation doesn't appear to create artefacts at centromere. The peaks observed here are as real as can be.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="6316" LastActivityDate="2019-10-10T13:04:24.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10568" PostTypeId="2" ParentId="10565" CreationDate="2019-10-10T13:12:40.827" Score="3" Body="&lt;p&gt;There are actually two questions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first one is about the memory issue. I believe &lt;code&gt;merge&lt;/code&gt; from &lt;code&gt;data.table&lt;/code&gt; would solve that issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second one would be aggregating or summarizing the identifiers. As @StupidWolf mentioned, you need to have a rule, summing them up, averaging them, ... For this second part one way is using &lt;code&gt;dplyr&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;merged_data %&amp;gt;%&#xA;  group_by(probe_colum) %&amp;gt;%&#xA;  summarise_all(mean or sum or ...) %&amp;gt;%&#xA;  mutate(new_column = paste(probe_column, gene_column, sep = &quot;_&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The same thing can be done faster with &lt;code&gt;data.table&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;merged_data[, lapply(.SD, mean or sum or ..., na.rm=TRUE), by = probe_column ] [, new_column := paste(probe_column, gene_column, sep = &quot;_&quot;),]  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;.SD&lt;/code&gt; means subset of data and your subsets are your groups based on your probes.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-10-10T13:26:31.607" LastActivityDate="2019-10-10T13:26:31.607" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10569" PostTypeId="1" AcceptedAnswerId="10574" CreationDate="2019-10-10T15:21:31.780" Score="1" ViewCount="202" Body="&lt;p&gt;My current understanding is:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;bases &lt;code&gt;A&lt;/code&gt; or &lt;code&gt;C&lt;/code&gt; can become methylated due to interaction with a methyltransferase&lt;/li&gt;&#xA;&lt;li&gt;these methylations only occur if the &lt;code&gt;A&lt;/code&gt; or &lt;code&gt;C&lt;/code&gt; is surrounded by a specific sequence of base pairs&lt;/li&gt;&#xA;&lt;li&gt;different types of DNA methylations occur in different species at different rates&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Is there a comprehensive list of which sequences of base pairs allow methylations to occur? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently aware of:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;dcm-methylation: &lt;code&gt;CCAGG&lt;/code&gt; or &lt;code&gt;CCTGG&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;dam-methylation: &lt;code&gt;GATC&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;EcoKI methylation: &lt;code&gt;AACNNNNNNGTGC&lt;/code&gt; or &lt;code&gt;GCACNNNNNNGTT&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;CpG methylation: &lt;code&gt;CG&lt;/code&gt; (or &lt;code&gt;CHG&lt;/code&gt; or &lt;code&gt;CHH&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6332" LastActivityDate="2019-10-11T15:24:09.967" Title="How many types of DNA methylation are there?" Tags="|phylogenetics|methylation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10570" PostTypeId="1" AcceptedAnswerId="10571" CreationDate="2019-10-10T15:58:09.490" Score="1" ViewCount="798" Body="&lt;p&gt;I have normalized expression values of some genes in some patients and the how many days they have survived from their diagnosis with cancer like below&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;&lt;em&gt;EDITED&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have taken mean of the expression values for each patient and I have how many days each patient survived since diagnosis as below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I done so but nothing I got&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; fit &amp;lt;- survfit(Surv(time) ~ gene, data = km)&#xA;&amp;gt; print(fit)&#xA;Call: survfit(formula = Surv(time) ~ gene, data = km)&#xA;&#xA;                  n events median 0.95LCL 0.95UCL&#xA;gene=-0.333140816 1      1    859      NA      NA&#xA;gene=-0.307846735 1      1    347      NA      NA&#xA;gene=-0.303559694 1      1   1339      NA      NA&#xA;gene=-0.290518776 1      1     61      NA      NA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The question is if these genes increase survival time or not but likely I am doing wrong&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;&lt;em&gt;By your help&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; fit &amp;lt;- coxph(Surv(time) ~ gene, data = km)&#xA;&amp;gt; print(fit)&#xA;Call:&#xA;coxph(formula = Surv(time) ~ gene, data = km)&#xA;&#xA;       coef exp(coef) se(coef)     z     p&#xA;gene 0.8664    2.3783   1.4520 0.597 0.551&#xA;&#xA;Likelihood ratio test=0.35  on 1 df, p=0.5543&#xA;n= 69, number of events= 69 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For plotting I guess I do need another information because&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; library(&quot;survminer&quot;)&#xA;&#xA;&amp;gt; ggsurvplot(fit, data = km)&#xA;Error in ggsurvplot(fit, data = a) : object 'ggsurv' not found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;&lt;em&gt;EDITED&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I  divided patients based on the median of survival days to Up and Down category like below but still I am failing to visualize that&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; fit &amp;lt;- survfit(Surv(Time, Status) ~ Gene, data = km)&#xA;&#xA;&#xA;&amp;gt; ggsurvplot(fit, data = km)&#xA;Error in data.frame(..., check.names = FALSE) : &#xA;  arguments imply differing number of rows: 70, 0, 140&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any help please?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-10-18T09:45:23.760" LastActivityDate="2019-10-18T09:45:23.760" Title="Doing plot with this data" Tags="|r|gene-expression|linear-regression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10571" PostTypeId="2" ParentId="10570" CreationDate="2019-10-10T17:09:23.877" Score="3" Body="&lt;p&gt;In general, survival analysis can be said to be composed of two steps; Cox regression, with which you calculate the &quot;hazard ratio&quot; based on your variables, and a &quot;Kaplan-Meier (KM) estimate&quot;, which is used to visuazlize the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a &lt;a href=&quot;http://www.sthda.com/english/wiki/survival-analysis-basics&quot; rel=&quot;nofollow noreferrer&quot;&gt;nice tutorial&lt;/a&gt; for doing survival analysis with the &lt;code&gt;survival&lt;/code&gt; and &lt;code&gt;survminer&lt;/code&gt; packages. The latter includes &lt;code&gt;ggplot2&lt;/code&gt; kind visualization of the KM curves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the Cox regression, you can use continuous(*) or discrete variables, in your case these would be &lt;em&gt;the normalized expression value of your gene&lt;/em&gt; and &lt;em&gt;categories of your gene&lt;/em&gt; respectively. You can categorize the expression level of a gene as &lt;em&gt;low vs high&lt;/em&gt;, or &lt;em&gt;low vs medium vs high&lt;/em&gt;, based on different quantiles.  Another approach to categorize your expression data would be to find an &quot;optimal&quot; cut point with &lt;a href=&quot;https://rpkgs.datanovia.com/survminer/reference/surv_cutpoint.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;maximally selected rank statistics&lt;/a&gt;; from the link: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This is an outcome-oriented methods providing a value of a cutpoint&#xA;  that correspond to the most significant relation with outcome (here,&#xA;  survival)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Basically you search for the cut-point where your data is split into two such that the two resulting parts of the data show the most separation/difference in terms of survival.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;* &lt;a href=&quot;https://stats.stackexchange.com/questions/228493/can-a-cox-proportional-hazards-model-be-built-only-with-continuous-predictors&quot;&gt;Can a Cox Proportional-Hazards Model be built only with continuous predictors?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-10-11T11:22:54.373" LastActivityDate="2019-10-11T11:22:54.373" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="10572" PostTypeId="2" ParentId="10558" CreationDate="2019-10-10T19:48:03.367" Score="3" Body="&lt;p&gt;You just need to learn how to do a &lt;a href=&quot;http://adv-r.had.co.nz/Subsetting.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;basic subset&lt;/a&gt; in R. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;table1 &amp;lt;- structure(list(n = c(29L, 11L, 10L, 10L), domain = c(1L, 5L, &#xA;6L, 32L)), row.names = c(NA, -4L), class = c(&quot;tbl_df&quot;, &quot;tbl&quot;, &#xA;&quot;data.frame&quot;))&#xA;&#xA;selected_domains &amp;lt;- filter(gene_domains, domain %in% table1$domain)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5088" LastActivityDate="2019-10-10T19:48:03.367" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10573" PostTypeId="1" CreationDate="2019-10-10T20:26:08.843" Score="1" ViewCount="33" Body="&lt;p&gt;&lt;a href=&quot;https://preview.data.humancellatlas.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Human Cell Atlas Preview Datasets&lt;/a&gt; have been available for a while now (there was some discussion about that &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/3541/human-cell-atlas-data-availability&quot;&gt;earlier&lt;/a&gt;). However, although the raw data is available, cell labels are not (for example, which cell type each cell belongs to). That seems like the most valuable part of this resource. I've heard of people using that type of information, but I have not been able to find it on the website. Is that available somewhere?&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2019-10-10T20:26:08.843" Title="Human Cell Atlas - cell annotations" Tags="|scrnaseq|public-databases|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10574" PostTypeId="2" ParentId="10569" CreationDate="2019-10-11T15:24:09.967" Score="5" Body="&lt;p&gt;If I'm understanding you correctly, by &quot;types of DNA methylation&quot; you mean &quot;nucleotide contexts where DNA methylation occurs&quot;.&#xA;This is going to be a function of the methyltransferase proteins involved in the methylation process, and this largely depends on the organism, and even cell type being studied.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, what has been observed is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The phenomenon of DNA methylation (methyl group addition to &lt;code&gt;A&lt;/code&gt;/&lt;code&gt;C&lt;/code&gt; nucleotides) has been observed in both eukaryotes and prokaryotes&lt;/li&gt;&#xA;&lt;li&gt;The nucleotide context of DNA methylation differs between the organism, and even the cell type being studied, due to the methyltransferase proteins involved in the methylation process&lt;/li&gt;&#xA;&lt;li&gt;Methylation of RNA, predominantly in N6 position &lt;code&gt;A&lt;/code&gt;s&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In humans:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;em&gt;DNMT1&lt;/em&gt; methyltransferase is referred to as the &quot;maintenance&quot; methyltransferase, due to its observation of binding to hemi-methylated &lt;code&gt;CpG&lt;/code&gt; sites (methylated &lt;code&gt;C&lt;/code&gt; on one strand and not the other)&lt;/li&gt;&#xA;&lt;li&gt;The &lt;em&gt;DNMT3A/B&lt;/em&gt; methyltransferases are referred to as the &quot;&lt;em&gt;de novo&lt;/em&gt;&quot; methyltransferases for their affinity of binding to naked DNA&lt;/li&gt;&#xA;&lt;li&gt;These proteins almost exclusively methylate &lt;code&gt;CpG&lt;/code&gt; dinucleotides in human cells&lt;/li&gt;&#xA;&lt;li&gt;There are cases of non-negligible methylation of cytosines in &lt;code&gt;CHG&lt;/code&gt; and &lt;code&gt;CHH&lt;/code&gt; nucleotide contexts (brain tissues, for example)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The proteins and their effects on DNA methylation are similar for mice, as well.&#xA;Plants tend to have similar nucleotide contexts of methylation, but have much more &lt;code&gt;CHG&lt;/code&gt; and &lt;code&gt;CHH&lt;/code&gt; methylation than mammals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In prokaryotes, again, the nucleotide context is dependent on the proteins.&#xA;Like you mentioned, we have that:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The Dam methyltransferase methylates the N6 position of the &lt;code&gt;A&lt;/code&gt; in &lt;code&gt;GATC&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;The Dcm methyltransferase methylates the C5 position of the second &lt;code&gt;C&lt;/code&gt; in &lt;code&gt;CCAGG&lt;/code&gt; and &lt;code&gt;CCTGG&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;The EcoKI methyltransferase methylates &lt;code&gt;A&lt;/code&gt;s in &lt;code&gt;AACNNNNNNGTGC&lt;/code&gt; and &lt;code&gt;GCACNNNNNNGTT&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;NEB has a good &lt;a href=&quot;https://international.neb.com/tools-and-resources/selection-charts/dam-dcm-and-cpg-methylation&quot; rel=&quot;noreferrer&quot;&gt;list of its restriciton enzymes&lt;/a&gt;, and how DNA methylation affects their binding/cutting, which also gives some info on the type of methylation in cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for a &quot;comprehensive list&quot;, you'll likely have to find some relevant review papers in your organism(s) of interest and compile on yourself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But also think about how you're phrasing your question.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is there a comprehensive list of which sequences of base pairs &lt;em&gt;&lt;strong&gt;allow&lt;/strong&gt;&lt;/em&gt; methylations to occur?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I don't think that many papers studying DNA methylation think of the nucleotide context &lt;strong&gt;allowing&lt;/strong&gt; methylation to occur so much as &lt;strong&gt;observing&lt;/strong&gt; the context in which methylation occurs and how it relates to proteins in those organisms.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2019-10-11T15:24:09.967" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10575" PostTypeId="2" ParentId="10558" CreationDate="2019-10-11T15:50:56.140" Score="0" Body="&lt;p&gt;I changed&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;target &amp;lt;- c(&quot;1&quot;, &quot;5&quot;, &quot;6&quot;, &quot;32&quot;)&#xA;selected_domains = filter(gene_domains, domain %in% target)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;into&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;target &amp;lt;- over10$domain&#xA;selected_domains = filter(gene_domains, domain %in% target)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and it's the same thing that @Lynn has suggested.&#xA;Thank you very much.&lt;/p&gt;&#xA;" OwnerUserId="856" LastActivityDate="2019-10-11T15:50:56.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10576" PostTypeId="2" ParentId="10408" CreationDate="2019-10-11T16:59:38.200" Score="1" Body="&lt;h2&gt;Ensembl to PDB&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;So the dataset you need is from &lt;a href=&quot;https://www.ebi.ac.uk/pdbe/docs/sifts/quick.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SIFT&lt;/a&gt;. There is one with Ensembl IDs. Majorly this also includes what chain is your peptide and what is the offset. About the latter, I actually wrote &lt;a href=&quot;https://blog.matteoferla.com/2019/09/pdb-numbering-rollercoaster.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;a blog post about it recently&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;PDB download&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Using Python, the &lt;a href=&quot;https://biopython.org/wiki/The_Biopython_Structural_Bioinformatics_FAQ&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biopython PDB module&lt;/a&gt; is very good. Although the PyMol module is more powerful. But as you want only coordinates (right?), you'd best use the former, which has a better documentation.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2019-10-11T16:59:38.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10577" PostTypeId="2" ParentId="10559" CreationDate="2019-10-12T01:55:38.107" Score="4" Body="&lt;p&gt;Typically I use &lt;a href=&quot;http://www.htslib.org/doc/samtools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools&lt;/a&gt; for operations like this.  Specifically I use &lt;code&gt;samtools view&lt;/code&gt; with either &lt;code&gt;-r&lt;/code&gt; or &lt;code&gt;-R&lt;/code&gt; flag depending on the use case.  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-r STR&#xA;Output alignments in read group STR [null]. Note that records with no RG tag will also be output when using this option. This behaviour may change in a future release.&#xA;&#xA;-R FILE&#xA;Output alignments in read groups listed in FILE [null]. Note that records with no RG tag will also be output when using this option. This behaviour may change in a future release.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="964" LastActivityDate="2019-10-12T01:55:38.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10578" PostTypeId="1" CreationDate="2019-10-12T10:41:45.487" Score="1" ViewCount="764" Body="&lt;p&gt;I have written a R code for Smith-Waterman (SW) algorithm. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to check the results of my code with online examples. In all cases -except one- I found the same alignment. So there is this one example -let's say example A*- for which I found a quite different alignment result: in the example they give three equivally good alignments whereas I found only one that corresponds to no none of those three alignments (we have the same score, though).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am now trying to understand whether example A is wrong or if my code is not working properly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, I have two requests:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does anyone have a dataset of the result of SW used on two sequences? &#xA;I could then compare these results with the output of my code.&lt;/li&gt;&#xA;&lt;li&gt;Does anyone have whatever way to compute SW algorithm? Then, I could give that person the example A, which could then try this example A, and give me the output.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;This would be very helpful if someone could help, I have already spent so much time on my code, without finding anything that could explain those different outputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Example A: Sequence A = &quot;TGCTCTGT&quot; and Sequence B = &quot;TACTATCAT&quot;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6340" LastEditorUserId="6341" LastEditDate="2019-10-13T15:14:08.237" LastActivityDate="2019-10-13T15:14:08.237" Title="Smith-Waterman Pairwise Local Alignment Algorithm" Tags="|sequence-alignment|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10579" PostTypeId="2" ParentId="10578" CreationDate="2019-10-12T20:52:31.400" Score="3" Body="&lt;p&gt;There are many implementations of the Smith-Waterman algorithm. However, they are not all guaranteed to give the same output because although it's an optimal local alignment algorithm, it still depends on the chosen scoring scheme (i.e. substitution matrix and gap penalties).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this in mind, you could try EMBOSS Water online:&#xA;&lt;a href=&quot;https://www.ebi.ac.uk/Tools/psa/emboss_water/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/Tools/psa/emboss_water/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or in Biopython, the &lt;code&gt;Align.PairwiseAligner()&lt;/code&gt; (which uses SW by default when &lt;code&gt;aligner.mode&lt;/code&gt; is set to &lt;code&gt;local&lt;/code&gt;):&#xA;&lt;a href=&quot;https://biopython.org/DIST/docs/api/Bio.Align.PairwiseAligner-class.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biopython.org/DIST/docs/api/Bio.Align.PairwiseAligner-class.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2019-10-12T20:52:31.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10580" PostTypeId="1" AcceptedAnswerId="10583" CreationDate="2019-10-12T22:53:44.207" Score="2" ViewCount="1373" Body="&lt;p&gt;I have a couple arrays of dN/dS scores, and I would like to calculate the confidence interval for each array of data. dN/dS scores are not normally distributed but are log-normally distrbuted, so I want some non-parametric approach for finding confidence intervals. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/questions/112829/how-do-i-calculate-confidence-intervals-for-a-non-normal-distribution&quot;&gt;An answer in the CrossValidated forum&lt;/a&gt; suggested that in R you can use a Wilcoxon test to find the median-based confidence interval, a function that I do not think the python scipy.stats.wilcoxon() package does. Is there anyway of doing something similar in Python for the confidence intervals of log-normally distributed data? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a example short array of dN/dS scores, here is the dN/dS scores of ATPases:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;array([0.02288081, 0.44170839, 0.10549733, 0.17515196, 0.09449279,&#xA;       0.07110412, 0.00893079, 0.23485109, 0.14533192, 0.05449631,&#xA;       0.10281173, 0.02113355, 0.05157087, 0.01705113, 0.02651948,&#xA;       0.09352947, 0.05828018, 0.20528157, 0.02873843, 0.0141598 ,&#xA;       0.11262881, 0.06337332, 0.13689815, 0.10557703, 0.04452507,&#xA;       0.05046484, 0.40241456, 0.05939199, 0.24423569, 0.05042892,&#xA;       0.164257  , 0.03408215, 0.04737262, 0.01037463, 0.01246448,&#xA;       0.02170375, 0.0241773 , 0.05136936, 0.02393366, 0.18913979,&#xA;       0.15781334, 0.06448557, 0.04355384, 0.02821125, 0.08015629,&#xA;       0.10985432, 0.06074574, 0.15775976, 0.05678278, 0.03749782,&#xA;       0.05518756, 0.00770479, 0.21248167, 0.08005044, 0.16307954,&#xA;       0.05783565, 0.05907416, 0.07044622, 0.13227131, 0.01627556,&#xA;       0.10859962, 0.08149819, 0.05600647, 0.16098728, 0.15183062,&#xA;       0.05202344, 0.01769589, 0.00789287, 0.07777749, 0.1324942 ,&#xA;       0.12734709, 0.17146938, 0.03890857, 0.1296019 , 0.085146  ,&#xA;       0.05602965, 0.02708089, 0.11807038, 0.07848828, 0.03291032,&#xA;       0.36302533, 0.07800343, 0.06551307, 0.04676282, 0.04765273,&#xA;       0.08060882, 0.06339636, 0.03349833, 0.01224308, 0.1481316 ,&#xA;       0.31738452, 0.15690855, 0.0693822 , 0.020425  , 0.02909208,&#xA;       0.03499225, 0.03019904, 0.13722717, 0.14403507, 0.01257245,&#xA;       0.02223452, 0.07068784, 0.0544813 , 0.08738558, 0.02884046,&#xA;       0.10549474, 0.06695546, 0.01341142, 0.09440411, 0.11840834,&#xA;       0.08558889, 0.2688645 , 0.28313546, 0.15127967, 0.01463191,&#xA;       0.1728421 ])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5029" LastActivityDate="2019-10-13T14:00:10.923" Title="Confidence Interval with Wilcoxon Test in Python for log-normal Distribution" Tags="|python|statistics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10582" PostTypeId="1" CreationDate="2019-10-13T12:45:10.133" Score="0" ViewCount="2593" Body="&lt;p&gt;I'm analyzing single cell rna-seq data and trying to compute the correlation score between different clusters. Wondering how to choose the correlation method(&quot;pearson&quot; (default), &quot;kendall&quot;, or &quot;spearman&quot;) of the Cor function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm a new comer in bioinformatics and know little about these arithmetics. Can anyone give me some advice or show me the code to decide which method to choose for my data matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is my gene expression matrix: rows are gene names,and columns are clusters.I use the default Peasrson method to do analysis,but I really don't know if it is right.&#xA;&lt;a href=&quot;https://user-images.githubusercontent.com/30619010/66716137-313b7600-edfd-11e9-810b-5bf7e58959ff.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/30619010/66716137-313b7600-edfd-11e9-810b-5bf7e58959ff.png&quot; alt=&quot;my data matrix&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CorOb.cor.exp &amp;lt;- as.data.frame(cor(CorOb.av.exp))&#xA;CorOb.cor.exp&lt;span class=&quot;math-container&quot;&gt;$x &amp;lt;- rownames(CorOb.cor.exp)&#xA;neupeptide.cor.df &amp;lt;- tidyr::gather(data = CorOb.cor.exp, y, correlation,CorOb.cor.exp$&lt;/span&gt;x)&#xA;a &amp;lt;- ggplot(neupeptide.cor.df, aes(x, y, fill = correlation)) + geom_raster()+scale_fill_gradientn(colors = c(&quot;blue&quot;, &quot;white&quot;, &quot;red&quot;))+ &#xA;  theme(axis.text.x = element_text(angle = 45, vjust = 1, &#xA;                                   size = 12, hjust = 1))+&#xA;  coord_fixed()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;a&lt;/p&gt;&#xA;" OwnerUserId="3851" LastEditorUserId="3851" LastEditDate="2019-10-13T13:09:05.617" LastActivityDate="2020-12-18T23:01:55.250" Title="Which correlation method to compute the correlation score between different clusters of Sc-RNAseq data?" Tags="|r|rna-seq|scrnaseq|seurat|single-cell|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10583" PostTypeId="2" ParentId="10580" CreationDate="2019-10-13T14:00:10.923" Score="2" Body="&lt;p&gt;I guess you mean confidence interval of the mean (or median). I looked at the array you provided, even after log it does not look normal-distributed. Use the bootstrap for non-parametric values. I modified the code by&lt;a href=&quot;http://www.jtrive.com/the-empirical-bootstrap-for-confidence-intervals-in-python.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;James Triveri&lt;/a&gt;, see example below, and you can use the mean or median. The textbook by Efron is a good reference for the method, which is also used in R package boot. There's also a&lt;a href=&quot;https://garstats.wordpress.com/2016/05/27/the-percentile-bootstrap/&quot; rel=&quot;nofollow noreferrer&quot;&gt;website&lt;/a&gt; which should be ok.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;import numpy as np&#xA;&#xA;def bootstrap(data, n=1000, func=np.mean,p=0.95):&#xA;&#xA;    sample_size = len(data)&#xA;    simulations = [func(np.random.choice(data, size=sample_size, replace=True)) for i in range(n)]&#xA;    simulations.sort()&#xA;    u_pval = (1+p)/2.&#xA;    l_pval = (1-u_pval)&#xA;    l_indx = int(np.floor(n*l_pval))&#xA;    u_indx = int(np.floor(n*u_pval))&#xA;    return(simulations[l_indx],simulations[u_indx])&#xA;&#xA;x = np.array([0.02288081, 0.44170839, 0.10549733, 0.17515196, 0.09449279,&#xA;       0.07110412, 0.00893079, 0.23485109, 0.14533192, 0.05449631,&#xA;       0.10281173, 0.02113355, 0.05157087, 0.01705113, 0.02651948,&#xA;       0.09352947, 0.05828018, 0.20528157, 0.02873843, 0.0141598 ,&#xA;       0.11262881, 0.06337332, 0.13689815, 0.10557703, 0.04452507,&#xA;       0.05046484, 0.40241456, 0.05939199, 0.24423569, 0.05042892,&#xA;       0.164257  , 0.03408215, 0.04737262, 0.01037463, 0.01246448,&#xA;       0.02170375, 0.0241773 , 0.05136936, 0.02393366, 0.18913979,&#xA;       0.15781334, 0.06448557, 0.04355384, 0.02821125, 0.08015629,&#xA;       0.10985432, 0.06074574, 0.15775976, 0.05678278, 0.03749782,&#xA;       0.05518756, 0.00770479, 0.21248167, 0.08005044, 0.16307954,&#xA;       0.05783565, 0.05907416, 0.07044622, 0.13227131, 0.01627556,&#xA;       0.10859962, 0.08149819, 0.05600647, 0.16098728, 0.15183062,&#xA;       0.05202344, 0.01769589, 0.00789287, 0.07777749, 0.1324942 ,&#xA;       0.12734709, 0.17146938, 0.03890857, 0.1296019 , 0.085146  ,&#xA;       0.05602965, 0.02708089, 0.11807038, 0.07848828, 0.03291032,&#xA;       0.36302533, 0.07800343, 0.06551307, 0.04676282, 0.04765273,&#xA;       0.08060882, 0.06339636, 0.03349833, 0.01224308, 0.1481316 ,&#xA;       0.31738452, 0.15690855, 0.0693822 , 0.020425  , 0.02909208,&#xA;       0.03499225, 0.03019904, 0.13722717, 0.14403507, 0.01257245,&#xA;       0.02223452, 0.07068784, 0.0544813 , 0.08738558, 0.02884046,&#xA;       0.10549474, 0.06695546, 0.01341142, 0.09440411, 0.11840834,&#xA;       0.08558889, 0.2688645 , 0.28313546, 0.15127967, 0.01463191,&#xA;       0.1728421 ])&#xA;&#xA;# for mean,95% confidence interval, do&#xA;bootstrap(x,1000,np.mean,0.95)&#xA;# for median, 95% confidence interval, do &#xA;bootstrap(x,1000,np.median,0.95)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2019-10-13T14:00:10.923" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10584" PostTypeId="2" ParentId="10578" CreationDate="2019-10-13T14:07:53.443" Score="2" Body="&lt;p&gt;In R, you can use &lt;a href=&quot;https://www.rdocumentation.org/packages/Biostrings/versions/2.40.2/topics/pairwiseAlignment&quot; rel=&quot;nofollow noreferrer&quot;&gt;pairwise Alignment in Biostrings&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;## Nucleotide global, local, and overlap alignments&#xA;library(Biostrings)&#xA;s1 &amp;lt;- DNAString(&quot;ACTTCACCAGCTCCCTGGCGGTAAGTTGAT&quot;)&#xA;s2 &amp;lt;- DNAString(&quot;GTTTCACTACTTCCTTTCGGGTAAGTAAAT&quot;)&#xA;# needleman&#xA;pairwiseAlignment(s1,s2,type=&quot;global&quot;)&#xA;# smith-waterman&#xA;pairwiseAlignment(s1,s2,type=&quot;local&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am not very sure how to implement it from scratch. If you face more problems with this example, I can have a go at it.&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2019-10-13T14:07:53.443" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10585" PostTypeId="1" AcceptedAnswerId="10597" CreationDate="2019-10-13T17:42:25.620" Score="0" ViewCount="90" Body="&lt;p&gt;I have read the content of a text file into pandas and needed some help matching the pattern.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the pattern, where &lt;code&gt;a&lt;/code&gt; can any number greater than zero and &lt;code&gt;b&lt;/code&gt; can be zero or any number greater than zero:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;father=a|0 || 0|a&#xA;mother=a|0 || 0|a&#xA;daughter1=b|0 || 0|b&#xA;daughter2=a|0 || 0|a&#xA;daughter3=b|0 || 0|b&#xA;son1=a|0 || 0|a&#xA;son2=b|0 || 0|b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the function that reads the content of text file to pandas&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;def read():&#xA;    varian=pd.read_csv('file.txt',sep = &quot;\t&quot;)&#xA;&#xA;father=varian[&quot;FATHER&quot;]&#xA;mother=varian[&quot;MOTHER&quot;]&#xA;daughter1=varian[&quot;DAUGHTER1&quot;]&#xA;daughter2=varian[&quot;DAUGHTER2&quot;]&#xA;daughter3=varian[&quot;DAUGHTER3&quot;]&#xA;son1=varian[&quot;SON1&quot;]&#xA;son2=varian[&quot;SON2&quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The content of the file looks like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;HROM    POS REF ALT FATHER  MOTHER  DAUGHTER1   DAUGHTER2   DAUGHTER3   SON1    SON2    INFO    FREQUECY&#xA;1   1226852 G   C   1/0 0/0 1|0 1|0 1|0 1|0 0|0 AN=2184;AC=12   0.005494505&#xA;1   1847936 C   T   0/1 1/1 1|1 1|1 1|1 1|1 0|1 AC=1;AN=2184    0.000457875&#xA;1   2428427 C   G   0/1 0/0 0|0 0|0 0|0 0|0 1|0 AC=4;AN=2184    0.001831502&#xA;1   2515616 G   A   1/0 1/0 1/0 1|1 1|1 1/0 1/0 AC=1;AN=2184    0.000457875&#xA;1   3801895 T   C   1/0 1/0 0|0 1/0 1/0 0|0 1|1 AC=10;AN=2184   0.004578755&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6271" LastEditorUserId="298" LastEditDate="2019-10-14T08:48:02.813" LastActivityDate="2019-10-18T08:11:39.677" Title="read and matching pattern with python" Tags="|python|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10586" PostTypeId="2" ParentId="10582" CreationDate="2019-10-13T18:51:35.317" Score="1" Body="&lt;p&gt;Disclaimer: This is not a complete answer but more a comment, posting as an answer because of character restrictions for comments. Also, I am not a statistican but a molecular biologist who did quite some reading/courses on stats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two scRNA-seq tools, namely &lt;code&gt;SingleR&lt;/code&gt; and &lt;code&gt;RCA&lt;/code&gt; (Reference Component Analysis, Li et. al, 2017) compute correlations of bulk transcriptomic data to single cells; for this they use &lt;code&gt;Spearman correlation&lt;/code&gt; and &lt;code&gt;Pearson correlation&lt;/code&gt; respectively (The two tools use the same bulk transcriptome reference). I am using these two tools for different purposes so I did not really compare their results, however, single cell clusters that I compute using the &lt;code&gt;reference components&lt;/code&gt; (calcuated with &lt;code&gt;RCA&lt;/code&gt;) is in good agreement with single cells labels computed with &lt;code&gt;RCA&lt;/code&gt; (adjusted rand index of 0.81).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition to that, please take a look at &lt;a href=&quot;https://academic.oup.com/bioinformatics/advance-article/doi/10.1093/bioinformatics/btz292/5480299&quot; rel=&quot;nofollow noreferrer&quot;&gt;scMatch: a single-cell gene expression profile annotation tool using reference datasets&lt;/a&gt;, they claim (scMatch computes bulk - single cell correlations like SingleR and RCA):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;From this comparison we found Spearman’s correlation coefficient&#xA;  outperformed Pearson’s correlation for all six libraries.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-10-13T19:19:37.490" LastActivityDate="2019-10-13T19:19:37.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10587" PostTypeId="2" ParentId="9241" CreationDate="2019-10-14T10:15:57.677" Score="1" Body="&lt;h2&gt;Gene name&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Using an Ensembl ID, Uniprot ID or NCBI ID it is much easier to look up your gene. So I would strongly reccomend using one of those as a key. Personally, I'd use Uniprot, but Ensembl is probably better if your genes are human.&#xA;One reason for this is also that different resources will use different names, especially for humans.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few considerations, so I will first say how this could be done properly...&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Custom Uniprot matcher with weights&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Some names are used by different protein, so which one is which would depend on context, co-occurrence with another gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your dataset might be extremely noisy. Therefore, you likely have assigned a confidence score to your matches, yes? It would be nice to give a score to the name based on whether it is the first name of the protein or the fourth synonym in the list.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore to do a conversion properly you really ought to make your own conversion system based upon context and Uniprot data. Uniprot contains the most data on synonyms and notes and binding partners etc. The dataset/API is in XML and Biopython does not have a parser, but I have some &lt;a href=&quot;https://blog.matteoferla.com/2019/02/uniprot-xml-and-python-elementtree.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;notes on the matter&lt;/a&gt; and can share some code if needed as I have done something similar (e.g. &lt;a href=&quot;https://github.com/matteoferla/Michelanglo-data/blob/master/dictionary/taxid9606-names2uniprot.json&quot; rel=&quot;nofollow noreferrer&quot;&gt;JSON for human gene/protein names to Uniprot IDs&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Lazy match&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The NCBI entrez API is rather flexible and good at presenting to you matches for a strange names. So I would suggest using that.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2019-10-14T10:15:57.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10588" PostTypeId="1" AcceptedAnswerId="10592" CreationDate="2019-10-14T13:28:06.280" Score="2" ViewCount="181" Body="&lt;p&gt;I know that telomeres are highly repeated sequences, but is there any way to retain any reads that map to these regions (on HG38)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recently managed to find some protein binding to centromeres, which are also mainly repeats. Therefore I wondered if there was any way to pick up signal on the telomeres.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My team and I are investigating binding of a protein complex to DNA and the experimental team reported that they saw it a bit on telomeres, so we were wondering if there was any way to measure this through bioinformatics. If not then it's no problem, we would just like to know.&lt;/p&gt;&#xA;" OwnerUserId="6316" LastEditorUserId="6316" LastEditDate="2019-10-14T15:41:51.747" LastActivityDate="2019-10-15T12:39:25.470" Title="Is there any way to align ChIP-seq reads to telomeres?" Tags="|sequence-alignment|genome|chip-seq|repeat-elements|telomere|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10589" PostTypeId="2" ParentId="10588" CreationDate="2019-10-14T15:38:17.443" Score="0" Body="&lt;p&gt;As far as I understand from this question and a related previous one, you have performed a ChIP-seq analysis and saw peaks at centromere regions. In that case, you can use &lt;code&gt;samtools&lt;/code&gt; to extract reads that are aligning to a given genomic region:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view input.bam &quot;Chr?:XX-YY&quot; &amp;gt; output.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-10-14T15:38:17.443" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10590" PostTypeId="2" ParentId="10518" CreationDate="2019-10-15T08:57:11.963" Score="2" Body="&lt;p&gt;Let's get some definitions out of the way:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;RMSD&lt;/strong&gt;, root mean square deviation, is a metric of distance of molecule A and molecule B. Think of it as average euclidean distance. It us casually called just 'deviation', but has nothing to do with standard deviation (the root of variance, 2nd moment). The units are &amp;Aring;ngstr&amp;ouml;m.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reference RMSD&lt;/strong&gt; is a term that appears in some manuals and is a the RMSD between the calculated pose and the starting pose. When you set up your docking you as a user define the starting position and how much it can deviate from that position. If you have a crystal structure with a ligand and you are docking the same or similar compound, then yes, it makes total sense to worry about this metric. &lt;strong&gt;In which case, RMSD greater than 2 are worrisome in a 2 &amp;Aring; resolution structure.&lt;/strong&gt; You especially see RMSD as a metric of accuracy when PDB structures are redocked, wherein the same ligand is docked and reference is the file from the PDB (or its PDB Redo derivative), so RMSD greater than 2 &amp;Aring; is bad as it would not match the density, while under 1 is really good &amp;mdash;note that in tests about 50% and 75% of re-docked ligands are docked less than 2 with AutoDock (free) and ICM-Dock (pay) respectively. But if you are doing it for an experiment and not for a test, you would probably have set the docking to be highly constrained in the first place (the grid size in AutoDock or position randomisation factor in most others, eg. Rosetta Ligand dock). If you actually care about reference RMSD the thing to manually look at is the top 3-4 poses in the electron density of your native structure. If you don't really know how your ligand docks please ignore this value as it is just a product of your starting pose. It is &lt;em&gt;not&lt;/em&gt; a measure of how &quot;happy&quot; the complex is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&amp;Delta;Gibbs&lt;/strong&gt; is a negative value in kcal/mol that is basically a metric of how &quot;happy&quot; the atoms are. It is actually an energy potential, but confer &lt;a href=&quot;https://en.wikipedia.org/wiki/Gibbs_free_energy&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia pages for the proper definition&lt;/a&gt;. The more negative it is the happier the atoms are. Force-field calculations solve this value and the difference between undocked and docked (&amp;Delta;&amp;Delta;G) is a good metric of how well your ligand docks. Please use this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Affinity&lt;/strong&gt; &amp;Delta;&amp;Delta;G is not a k_off or a k_on. That is a result of the ligand binding to nearby positions and how well it can get into the binding site (&lt;em&gt;e.g.&lt;/em&gt; in P450 enzymes the ligand orientation is dictated by how it squeezes into the entrance way). And how insoluble the ligand is, which is a major problem*. For a better metric, there is &lt;strong&gt;dynamic undocking&lt;/strong&gt; where a constrained MD simulation is done with the ligand getting pulled out of the pocket. But this is not trivial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&amp;lowast; As any comp-chemist can tell you it is mighty hard to calculate solubility. Some force-field calculations do not even take account of &amp;pi;-&amp;pi; or S-&amp;pi; interactions. Drug-like compounds are rigid aromatic and rigid and highly hydrophobic. Many programs use a hack called implicit waters, where the water is not simulated as individual molecules but as a flowing thing. This means that &amp;Delta;&amp;Delta;G benefits of constrained waters are ignored and alternative protontation states of water too. This is fine for protein (mostly), but has major errors for drugs. But in terms of thing to look out for in a docking algorithm, explicit waters is not the first as conformer sampling and side chain perturbation are more important.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2019-11-04T22:29:33.630" LastActivityDate="2019-11-04T22:29:33.630" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10591" PostTypeId="2" ParentId="9241" CreationDate="2019-10-15T11:35:23.743" Score="0" Body="&lt;p&gt;If you have a list of human gene names from various sources that may include old or alternative names, then I'd suggest using the HUGO database (www.genenames.org). From there you can get a table of approved names and symbols as well as alternative symbols* . With this you can search for your gene names in the table and assign the approved name. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wouldn't use  Ensembl ID as some genes have multiple IDs that indicate the haplotype of the source genome, which is probably not useful for most purposes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*Names are the full description of the protein, e.g. &quot;B-Raf proto-oncogene, serine/threonine kinase&quot; whereas the symbol is the short version you usually see, e.g &quot;BRAF&quot;&lt;/p&gt;&#xA;" OwnerUserId="6356" LastActivityDate="2019-10-15T11:35:23.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10592" PostTypeId="2" ParentId="10588" CreationDate="2019-10-15T12:39:25.470" Score="3" Body="&lt;p&gt;This paper &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/728519v1.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/10.1101/728519v1.full&lt;/a&gt; examines the mapping of some human telomeric sequences, and indicates that there is something like 130kb missing at the telomeres in the HG38 assembly.  One option might be to take the HG38 assembly, and add some missing telomere sequences as an additional pseudochromosome to the FASTA file.  Then if you used this modified reference to align your reads, you should soon get a feel for the level of telomeric reads in your data set.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-10-15T12:39:25.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10593" PostTypeId="2" ParentId="7504" CreationDate="2019-10-15T13:30:32.017" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What is the purpose of correlating docking score with catalytic efficiency (kcat/km)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This is specific to the enzymology of &lt;a href=&quot;https://www.nature.com/articles/ncomms12965&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper you are citing&lt;/a&gt;. If the solubility of your substrate is lower than your Michaelis constant (K_M) you cannot fit to a Michaelis-Menten curve and can only get the slope of the initial straight part of the curve, which is the catalytic efficiency (k_cat/K_M). Ideally K_M would be better. But in generally catalytic efficiency ought to correlate with well calculated ddGs of ligand binding.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I saw some papers are doing this, is it to validate the docking protocol?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Totally. Docking finds were ligands bind. Although the energetics don't match too well the experimental kinetics as of 2019.&#xA;The catalytic efficiency (k_cat/K_M) is the ratio of the turnover (k_cat) over Michaelis constant (K_M). The Michealis constant is in turn the ratio of on-rate over off-rate plus turnover. The &amp;Delta;&amp;Delta;Gibbs of the ligand ought to correlate with the ratio of the on-rate/off-rate, while the turnover is unrelated.&#xA;The turnover should correlate with the difference in energy between the substrate bound form and the transition state. This is the realm of theozyme design and requires correctly protonated ligands and protein, but when it is you get a nice transition state hill (e.g. &lt;a href=&quot;https://blog.matteoferla.com/2018/09/everything-you-wanted-to-know-about.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;personal example&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a few &quot;ought to&quot; in the above. Docking and force-field energy scores are not precise. The paper you refer to is doing very advance docking (akin to dynamic undocking) and, not accidentally, the ligands are very flexible and rather soluble.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Lets say the docking protocol is correct, the docking score will positively correlate with kcat/km?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A positive correlation coefficient just means that generally both the catalytic efficiency and docking scores increases or decrease together. If a metric &quot;worked backwards&quot;, such as &amp;Delta;&amp;Delta;Gibbs (more negative is better), you would want a negative correlation. What you want is a &lt;strong&gt;strong&lt;/strong&gt; correlation, with the sign a result of the nature of your metrics.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2019-10-15T13:30:32.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10594" PostTypeId="1" CreationDate="2019-10-15T15:04:01.677" Score="2" ViewCount="611" Body="&lt;p&gt;Heng Li's &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/34/18/3094/4994778&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper on minimap2&lt;/a&gt; is understandable to me right up until the moment that it gives this definition of gap cost:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;h3&gt;2.2.1 Alignment with 2-piece affine gap cost&lt;/h3&gt;&#xA;  &#xA;  &lt;p&gt;Minimap2 performs DP-based global alignment between adjacent anchors in a chain. It uses a 2-piece affine gap cost &lt;a href=&quot;http://dx.doi.org/10.1007/BF02458577&quot; rel=&quot;nofollow noreferrer&quot;&gt;(Gotoh, 1990)&lt;/a&gt;:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\gamma_a(l) = \min \{ q + |l| \cdot e, \tilde{q} + |l| \cdot \tilde{e} \}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Suddenly, out of nowhere, we've got &lt;em&gt;five&lt;/em&gt; variables used without prior introduction: &lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\tilde{q}$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$\tilde{e}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Heng doesn't state their meanings, and based on a quick skim and text search of the cited paper by Gotoh, that paper doesn't use these variable names anywhere either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does this equation mean?&lt;/p&gt;&#xA;" OwnerUserId="3144" LastEditorUserId="4719" LastEditDate="2019-10-20T10:15:33.157" LastActivityDate="2019-10-20T10:40:10.560" Title="What do the symbols mean in minimap2's gap cost equation?" Tags="|sequence-alignment|read-mapping|minimap2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10595" PostTypeId="1" AcceptedAnswerId="10599" CreationDate="2019-10-15T16:51:41.657" Score="1" ViewCount="1264" Body="&lt;p&gt;I have this code to filter some IDs from a fasta file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for line in file_open:&#xA;    line=line.rstrip()&#xA;    if '16S' in line:&#xA;        header=line&#xA;        header=header[0] + header[1:]&#xA;        seq=file_open.readline()&#xA;        seq=seq.strip()&#xA;        print(header + '\n'  + seq)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;however, this code doesn't get me all content of each ID sequence. I get this result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;ID1&#xA;AGTTTTGCCGAATTCCTTTAGGATTTTGTGAGGAACATTGTGTTAAATGTATTTATAGTTCTACTAATTC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I should get this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;ID1&#xA;AGTTTTGCCGAATTCCTTTAGGATTTTGTGAGGAACATTGTGTTAAATGTATTTATAGTTCTACTAATTC&#xA;TGCGAAGGTAGCATAATCATTAGCCTTTTGATTTGAGGCTGGAATGAATGGTTTGACGAGAGATGGTCTG&#xA;GGGCCGTGGTATACTGACCATGCGAAGGTAGCATAATCATTAGCCTCTTGATTTGAGGCTGGAATGAATG&#xA;GGGCCGTGGTATAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="2686" LastEditorUserId="298" LastEditDate="2019-10-15T18:24:52.147" LastActivityDate="2019-10-15T19:42:28.960" Title="Filtering fasta files by ID" Tags="|python|fasta|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10596" PostTypeId="1" AcceptedAnswerId="10598" CreationDate="2019-10-15T16:55:32.817" Score="0" ViewCount="56" Body="&lt;p&gt;I have two matrix identical in N° of rows and columns. In the first matrix rows indicate transposable elements and columns  indicate samples. the matrix show the degree of expression of each element in samples: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;M1 &amp;lt;-&#xA;&#xA;&#xA;            elements          ova       testes       retina   optic_lobe       suckers &#xA;1   LINES/3           1.473559e+03   962.953589   673.903449   558.879946   947.631992          &#xA;2   SINES             1.836210e+02   355.808323   577.807528   341.048786   99.902702          &#xA;3  DNA TRANSP         9.181051e+00   178.661200  4387.148414    53.907711   353.413028           &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In Second matrix rows indicate genes and columns indicate samples. the matrix show the degree of expression of each gene in samples: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; M2 &amp;lt;-&#xA;            elements          ova       testes       retina   optic_lobe  suckers &#xA;1   OCBIM_22028242mg 1.473559e+03   962.953589   673.903449   558.879946   947.631992          &#xA;2   OCBIM_22008718mg 1.836210e+02   355.808323   577.807528   341.048786   99.902702          &#xA;3   OCBIM_22009482mg 9.181051e+00   178.661200  4387.148414    53.907711   353.413028           &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have to make an element-gene correlation. I have to check, for example, if the element A expressed much in the tissue X has the corresponding gene A expressed very, little, or nothing in the tissue X&lt;/p&gt;&#xA;&#xA;&lt;p&gt;fOR example I'm expecting this result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;M1,2 &amp;lt;-&#xA;                elements          ova       testes       retina   optic_lobe  suckers &#xA;    1   TE/ gene1                  0.3          1          0.5         -1         0.3          &#xA;    2   TE/gene2                   0.1        0.6            0         -1         0.9      &#xA;    3   TE/gene3                     1          1          0.8        -0.2       -0.3            &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4475" LastEditorUserId="4475" LastEditDate="2019-10-15T18:08:10.760" LastActivityDate="2019-10-16T07:18:43.277" Title="correlation between transposon and gene" Tags="|r|gene|correlation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10597" PostTypeId="2" ParentId="10585" CreationDate="2019-10-15T19:00:44.843" Score="1" Body="&lt;p&gt;This is not a bioinformatics question, but a basic pandas question, so SO is a better place for this... But your odd looking VCF file can be parsed as follows without even using regex:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;def splitter(cell, key):&#xA;    parts = cell.split('||')&#xA;    if key in parts[0]:&#xA;         p = parts[0].split('|')&#xA;    else:&#xA;         p = parts[1].split('|')&#xA;    sp = [x.strip() for x in p if key not in x]&#xA;    return float(sp[0])&#xA;&#xA;def read():&#xA;    splitter_a = lambda v: v.split('||')[0].split('|')[1]) &#xA;    varian=pd.read_csv('file.txt',sep = &quot;\t&quot;)&#xA;    a_splitter = lambda c: splitter(c, 'a')&#xA;    b_splitter = lambda c: splitter(c, 'b')&#xA;    varian = varian.assign(father_a=varian.FATHER.apply(a_splitter)\&#xA;              .assign(father_a=varian.FATHER.apply(b_splitter)\&#xA;              .assign(mother_a=varian.MOTHER.apply(la_splitter)\&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;display(varian)&lt;/code&gt; or &lt;code&gt;print(varian.columns)&lt;/code&gt; will show that you have new columns called &lt;code&gt;father_a&lt;/code&gt; etc.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2019-10-15T19:20:09.827" LastActivityDate="2019-10-15T19:20:09.827" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10598" PostTypeId="2" ParentId="10596" CreationDate="2019-10-15T19:28:13.390" Score="1" Body="&lt;p&gt;Ok I think I roughly get what you want to get. Below I simulate something that's like your data because you provided two identical matrices.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#3 TE, 10 tissue, random values&#xA;# number of tissue&#xA;M1 = matrix(rnorm(30),3,10)&#xA;rownames(M1) = paste(&quot;TE&quot;,1:3,sep=&quot;_&quot;)&#xA;colnames(M1) = paste(&quot;Tissue&quot;,letters[1:10],sep=&quot;_&quot;)&#xA;## make gene matrix, , random values&#xA;M2 = matrix(rnorm(30),3,10)&#xA;rownames(M2) = paste(&quot;Gene&quot;,1:3,sep=&quot;_&quot;)&#xA;colnames(M2) = paste(&quot;Tissue&quot;,letters[1:10],sep=&quot;_&quot;)&#xA;## here we make TE1 highly correlated to Gene1&#xA;M2[1,] = M1[1,] + rnorm(10,0,0.3)&#xA;## here we make TE2 highly anti-correlated to Gene2&#xA;M2[2,] = -1*M1[2,] + rnorm(10,0,0.3)&#xA;#here we calculate for each TE, its correlation with each gene.&#xA;#credit to @haci&#xA;COR = apply(t(M1),t(M2))&#xA;# you can look at the correlation&#xA;COR&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What you need is the last three rows. Remember, you can only calculate correlation between the gene and the TE, and you do this across all tissue. For example, if gene1 and TE1 has a high correlation, this tells you TE1 expression can predict gene1 expression in most tissue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's meaningless to calculate correlation between one data point and another. You need more than 1 value. For example, if you do cor(10,5), it doesn't make sense. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this is what you are trying to do.. &lt;/p&gt;&#xA;" OwnerUserId="5077" LastEditorUserId="5077" LastEditDate="2019-10-16T07:18:43.277" LastActivityDate="2019-10-16T07:18:43.277" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10599" PostTypeId="2" ParentId="10595" CreationDate="2019-10-15T19:42:28.960" Score="8" Body="&lt;p&gt;You're assuming 2-line FASTA format, but the sequence can (and typically does) span multiple lines. You can use &lt;a href=&quot;https://biopython.org/&quot; rel=&quot;noreferrer&quot;&gt;Biopython&lt;/a&gt; to build a simple parser:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;for record in SeqIO.parse(&quot;file_name.fa&quot;, &quot;fasta&quot;):&#xA;    if &quot;16S&quot; in record.description:&#xA;        print(record.format(&quot;fasta&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastActivityDate="2019-10-15T19:42:28.960" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10601" PostTypeId="2" ParentId="7048" CreationDate="2019-10-16T23:21:23.177" Score="0" Body="&lt;p&gt;MSA fasta file can be generated from SAM,&#xA;but it is almost impractical to generate this HUGE file.&#xA;If you still insist to generate the file, then please check this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://sourceforge.net/projects/sam2fasta/files/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sourceforge.net/projects/sam2fasta/files/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6364" LastActivityDate="2019-10-16T23:21:23.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10602" PostTypeId="1" AcceptedAnswerId="10606" CreationDate="2019-10-17T13:33:09.583" Score="0" ViewCount="331" Body="&lt;pre&gt;&lt;code&gt;library(WGCNA);&#xA;# The following setting is important, do not omit.&#xA;library(flashClust)&#xA;enableWGCNAThreads()&#xA;options(stringsAsFactors = FALSE);&#xA;#Read in the female liver data set&#xA;femData = read.csv(&quot;CTRL_WGCNA.txt&quot;,sep = &quot;\t&quot;);&#xA;&#xA;head(femData)&#xA;# Read in the male liver data set&#xA;maleData = read.csv(&quot;BPD_WGCNA.txt&quot;,sep = &quot;\t&quot;);&#xA;# Take a quick look at what is in the data sets (caution, longish output):&#xA;dim(femData)&#xA;names(femData)&#xA;dim(maleData)&#xA;names(maleData)&#xA;&#xA;&#xA;#=====================================================================================&#xA;#&#xA;#  Code chunk 2&#xA;#&#xA;#=====================================================================================&#xA;&#xA;&#xA;# We work with two sets:&#xA;nSets = 2;&#xA;# For easier labeling of plots, create a vector holding descriptive names of the two sets.&#xA;setLabels = c(&quot;Female liver&quot;, &quot;Male liver&quot;)&#xA;shortLabels = c(&quot;Female&quot;, &quot;Male&quot;)&#xA;# Form multi-set expression data: columns starting from 9 contain actual expression data.&#xA;multiExpr = vector(mode = &quot;list&quot;, length = nSets)&#xA;&#xA;multiExpr[[1]] = list(data = as.data.frame(t(femData[-c(1)])));&#xA;names(multiExpr[[1]]&lt;span class=&quot;math-container&quot;&gt;$data) = femData$&lt;/span&gt;Gene;&#xA;rownames(multiExpr[[1]]&lt;span class=&quot;math-container&quot;&gt;$data) = names(femData)[-c(1)];&#xA;multiExpr[[2]] = list(data = as.data.frame(t(maleData[-c(1)])));&#xA;names(multiExpr[[2]]$&lt;/span&gt;data) = maleData&lt;span class=&quot;math-container&quot;&gt;$Gene;&#xA;rownames(multiExpr[[2]]$&lt;/span&gt;data) = names(maleData)[-c(1)];&#xA;# Check that the data has the correct format for many functions operating on multiple sets:&#xA;exprSize = checkSets(multiExpr)&#xA;&#xA;&#xA;#=====================================================================================&#xA;#&#xA;#  Code chunk 3&#xA;#&#xA;#=====================================================================================&#xA;&#xA;&#xA;# Check that all genes and samples have sufficiently low numbers of missing values.&#xA;gsg = goodSamplesGenesMS(multiExpr, verbose = 3);&#xA;gsg$allOK&#xA;&#xA;&#xA;#=====================================================================================&#xA;#&#xA;#  Code chunk 4&#xA;#&#xA;#=====================================================================================&#xA;&#xA;&#xA;if (!gsg&lt;span class=&quot;math-container&quot;&gt;$allOK)&#xA;{&#xA;  # Print information about the removed genes:&#xA;  if (sum(!gsg$goodGenes) &amp;gt; 0)&#xA;    printFlush(paste(&quot;Removing genes:&quot;, paste(names(multiExpr[[1]]$data)[!gsg$goodGenes], &#xA;                                              collapse = &quot;, &quot;)))&#xA;  for (set in 1:exprSize$nSets)&#xA;  {&#xA;    if (sum(!gsg$goodSamples[[set]]))&#xA;      printFlush(paste(&quot;In set&quot;, setLabels[set], &quot;removing samples&quot;,&#xA;                       paste(rownames(multiExpr[[set]]$data)[!gsg$goodSamples[[set]]], collapse = &quot;, &quot;)))&#xA;    # Remove the offending genes and samples&#xA;    multiExpr[[set]]$data = multiExpr[[set]]$data[gsg$goodSamples[[set]], gsg$&lt;/span&gt;goodGenes];&#xA;  }&#xA;  # Update exprSize&#xA;  exprSize = checkSets(multiExpr)&#xA;}&#xA;&#xA;&#xA;#=====================================================================================&#xA;#&#xA;#  Code chunk 5&#xA;#&#xA;#=====================================================================================&#xA;&#xA;&#xA;sampleTrees = list()&#xA;for (set in 1:nSets)&#xA;{&#xA;  sampleTrees[[set]] = hclust(dist(multiExpr[[set]]$data), method = &quot;average&quot;)&#xA;}&#xA;&#xA;&#xA;#=====================================================================================&#xA;#&#xA;#  Code chunk 6&#xA;#&#xA;#=====================================================================================&#xA;&#xA;&#xA;pdf(file = &quot;SampleClustering.pdf&quot;, width = 12, height = 12);&#xA;par(mfrow=c(2,1))&#xA;par(mar = c(0, 4, 2, 0))&#xA;for (set in 1:nSets)&#xA;  plot(sampleTrees[[set]], main = paste(&quot;Sample clustering on all genes in&quot;, setLabels[set]),&#xA;       xlab=&quot;&quot;, sub=&quot;&quot;, cex = 0.7);&#xA;dev.off();&#xA;&#xA;&#xA;  #=====================================================================================&#xA;#&#xA;#  Code chunk 7&#xA;#&#xA;#=====================================================================================&#xA;&#xA;&#xA;# Choose the &quot;base&quot; cut height for the female data set&#xA;baseHeight = 80&#xA;# Adjust the cut height for the male data set for the number of samples&#xA;cutHeights = c(80, 100*exprSize&lt;span class=&quot;math-container&quot;&gt;$nSamples[2]/exprSize$&lt;/span&gt;nSamples[1]);&#xA;# Re-plot the dendrograms including the cut lines&#xA;pdf(file = &quot;SampleClustering.pdf&quot;, width = 12, height = 12);&#xA;par(mfrow=c(2,1))&#xA;par(mar = c(0, 4, 2, 0))&#xA;for (set in 1:nSets)&#xA;{&#xA;  plot(sampleTrees[[set]], main = paste(&quot;Sample clustering on all genes in&quot;, setLabels[set]),&#xA;       xlab=&quot;&quot;, sub=&quot;&quot;, cex = 0.7);&#xA;  abline(h=cutHeights[set], col = &quot;red&quot;);&#xA;}&#xA;dev.off();&#xA;&#xA;&#xA;  #=====================================================================================&#xA;#&#xA;#  Code chunk 8&#xA;#&#xA;#=====================================================================================&#xA;&#xA;&#xA;for (set in 1:nSets)&#xA;{&#xA;  # Find clusters cut by the line&#xA;  labels = cutreeStatic(sampleTrees[[set]], cutHeight = cutHeights[set])&#xA;  # Keep the largest one (labeled by the number 1)&#xA;  keep = (labels==1)&#xA;  multiExpr[[set]]&lt;span class=&quot;math-container&quot;&gt;$data = multiExpr[[set]]$&lt;/span&gt;data[keep, ]&#xA;}&#xA;collectGarbage();&#xA; # Check the size of the leftover data&#xA;exprSize = checkSets(multiExpr)&#xA;exprSize&#xA;&#xA;&#xA;&#xA;&#xA;$nSets&#xA;[1] 2&#xA;&#xA;$nGenes&#xA;[1] 8213&#xA;&#xA;$nSamples&#xA;[1] 0 0&#xA;&#xA;$structureOK&#xA;[1] TRUE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think the issue is with the tree cut height because when i check the label all of them are zeroes i guess that is issue so far which is why im getting exprSize nsamples as &lt;code&gt;0 0&lt;/code&gt;instead of my actual sample size.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried it with different cut height but so far no success may its a pretty simple issue but im not able to figure it out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Data which im using here is the &lt;a href=&quot;https://drive.google.com/open?id=1q6j0UAGxy07TQBd1tbw0XWZAk8LvvieO&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion or help would be highly appreciated &lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2019-10-17T17:02:29.637" Title="Tree cut issue in WGCNA" Tags="|r|rna-seq|wgcna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10603" PostTypeId="1" CreationDate="2019-10-17T15:37:09.500" Score="1" ViewCount="35" Body="&lt;p&gt;Given a RNA-Seq data of a progeny between two inbred lines (for example an F4 plant which was derived from two inbred lines (parents) and the F1 selfed 3 times; however this is probably not crucial for the question).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We know the progeny should have only a few breakpoints in each chromosome, so one should expect long stretches (&gt;1MB) with those 3 states:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;homozygous for parent1 &lt;/li&gt;&#xA;&lt;li&gt;homozygous for parent2 &lt;/li&gt;&#xA;&lt;li&gt;heterozygous&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;In each state, most containing genes should therefore be reflecting this state, i.e. only expressing the parent1 allele in state 1) (i.e all SNPs support parent1) etc.&#xA;A caveat would be uni-allelic expression at a heterozygous state 3), but this surely will not be the case for all genes in a given block (100+ genes per block are expected).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both parental genomes and RNA-Seq libraries for both parents and F4 are available.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Naively I'd do SNP-calling (e.g. &lt;code&gt;GATK HaplotypeCaller&lt;/code&gt; or &lt;code&gt;bcftools call&lt;/code&gt;) of an F4 mapped against one parental reference and somehow combine the allele frequency in a sliding window and evaluate the likelihood of the given window to be in state 1-3. E.g. if all alleles support parent 1, it's probably state 1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I feel this must have been done before but can't quite find the right software or workflow software to use. But maybe I'm missing something.&#xA;Ideally, a workflow would have as input the F4 RNA-Seq data (possibly alos parental RNA-Seq data), the parental genome(s) and as output a list of segments per chromosome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know, we could do gDNA sequencing for this even better, but I think RNA-Seq might also do the job and might save effort since already available.&lt;/p&gt;&#xA;" OwnerUserId="1461" LastActivityDate="2019-10-17T15:37:09.500" Title="How to segment genome into homozygote/heterozygote blocks based on RNA-Seq data" Tags="|rna-seq|snp|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10604" PostTypeId="2" ParentId="4065" CreationDate="2019-10-17T15:38:46.100" Score="2" Body="&lt;p&gt;You need to generate a list of gene ID vectors to feed into &lt;code&gt;upset&lt;/code&gt;. In the example below, I am selecting observations (genes) which have an level above the median level. You could also choose only genes above the 3rd quartile, etc. You then need to convert these from a list of character vectors to a presence/absence matrix. UpSetR has a &lt;code&gt;fromList&lt;/code&gt; function that does just this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In future, be more specific about what you want to achieve and supply example data for people to work with.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(&quot;UpSetR&quot;)&#xA;&#xA;tmp &amp;lt;- mtcars&#xA;&#xA;tmp$car &amp;lt;- rownames(mtcars)&#xA;list &amp;lt;- lapply(colnames(mtcars), function(x) {&#xA;  tmp[tmp[[x]] &amp;gt; median(tmp[[x]]), &quot;car&quot;]&#xA;})&#xA;names(list) &amp;lt;- colnames(mtcars)&#xA;&#xA;upset(fromList(list))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/xwRHJ.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;Created on 2019-10-17 by the &lt;a href=&quot;https://reprex.tidyverse.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;reprex package&lt;/a&gt; (v0.3.0)&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="6363" LastActivityDate="2019-10-17T15:38:46.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10605" PostTypeId="2" ParentId="10582" CreationDate="2019-10-17T16:04:06.643" Score="1" Body="&lt;p&gt;This question does not have a definite answer. Rather, the answer depends heavily on the assumptions you wish to make regarding your data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a pair of random variables &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;, Pearson's rho has a value of&#xA;1 when there is a linear relationship between &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;; i.e. &lt;span class=&quot;math-container&quot;&gt;$y = x\beta + c$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; is a constant, and there is no noise. &#xA;Note that the slope, &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt;, and intercept, &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;, of this relationship&#xA;do not affect the correlation coefficient. With Spearman's rho, we first compute the rank&#xA;of each element in &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;. We then calculate the Pearson correlation between&#xA;these ranks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kendall's tau correlation coefficient also deals with the ranks of &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt;,&#xA;but is slightly more complex. The most relevant difference in this case is likely&#xA;that tau is better in the case of ties, and may be more robust for low sample sizes.&#xA;In most cases, Kendall's and Spearman's are likely to give very similar results&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you wish to assess only linear relationships between expression profiles, &#xA;Pearson's is the most appropriate. If you wish to allow for non-linear effects,&#xA;Spearman or Kendall is most appropriate (Kendall if the data contains ties).&lt;/p&gt;&#xA;" OwnerUserId="6363" LastActivityDate="2019-10-17T16:04:06.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10606" PostTypeId="2" ParentId="10602" CreationDate="2019-10-17T17:02:29.637" Score="2" Body="&lt;p&gt;It is likely that the cut heights set in the code (80 and 100*ratio of sample numbers) are too low. Look at the sample trees plotted into a pdf file (SampleClustering.pdf) and note the merging heights. You need to set the cut heights appropriately so that most samples go into a single cluster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Peter&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2019-10-17T17:02:29.637" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10607" PostTypeId="2" ParentId="10585" CreationDate="2019-10-18T07:16:37.620" Score="1" Body="&lt;p&gt;Its easy. You want to remove homozygotes and here is the solution for the FATHER class. Note the MOTHER class will comprise homozygotes in the answer because you are only searching for FATHER. To perform an intersection one approach is just to concatenate the mask.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;genedict = {\&#xA;            'FATHER': ['1/0', '1/1', '0/0', '1/0', '0/1'], \&#xA;            'MOTHER': ['1/1', '1/1', '1/0', '1/0', '0/0']}&#xA;&#xA;df = pd.DataFrame(genedict)    &#xA;mask = df.FATHER.str.contains(r'1[|/]0') + df.FATHER.str.contains(r'0[|/]1')&#xA;df[mask]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;OUTPUT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  FATHER MOTHER&#xA;0    1/0    1/1&#xA;3    1/0    1/0&#xA;4    0/1    0/0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note, this is not your original dataset, its a dictionary to demonstrate the principle and note all your other columns will carry across unless you want to 'drop' them. Finally, pandas has its own regex system, regardless the traditional r'' code is still needed&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-10-18T08:11:39.677" LastActivityDate="2019-10-18T08:11:39.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10609" PostTypeId="1" CreationDate="2019-10-18T08:56:26.923" Score="0" ViewCount="39" Body="&lt;p&gt;I've read that there are blood analysis devices that can automatically determine the data for each specified analyse. You can expect to get the results in less than a day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q: How and where do they store the information? Is there a database of the provider or the clinic? Are all the device providers saving the data in the same format?&lt;/p&gt;&#xA;" OwnerDisplayName="user6369" LastActivityDate="2020-04-16T16:52:17.847" Title="Blood analysis devices: how and where is the data saved" Tags="|database|data-download|data-management|data-preprocessing|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10610" PostTypeId="2" ParentId="10518" CreationDate="2019-10-18T12:31:44.180" Score="1" Body="&lt;p&gt;2A would be huge for docking, but agreed depends on the resolution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its 10A if you are looking at protein-protein bonds in a beta pleated sheet and your molecules need to be a lot smaller than that.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-10-18T12:31:44.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10611" PostTypeId="1" CreationDate="2019-10-18T15:39:58.257" Score="2" ViewCount="40" Body="&lt;p&gt;I want to fix a chromosome of around 50-100Mb (&lt;code&gt;chr.fasta&lt;/code&gt;) for a region that is missing a few kilobases of data. I have the missing sequence with a window upstream and downstream that matches perfectly to the reference (&lt;code&gt;contig.fasta&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What's the best way to produce a new chromosome, &lt;code&gt;fixed.fasta&lt;/code&gt; that contains the new data from &lt;code&gt;contig.fasta&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See depiction below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr.fasta    |---------------------------------------------------   -------------|&#xA;contig.fasta                                                   |-===-|&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Final intended result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fixed.fasta  |---------------------------------------------------===-------------|&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="180" LastEditDate="2019-10-21T09:41:13.890" LastActivityDate="2019-10-21T09:54:41.717" Title="fixing chromosome with new contig" Tags="|assembly|chromosomes|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10612" PostTypeId="1" AcceptedAnswerId="10613" CreationDate="2019-10-18T16:01:31.493" Score="0" ViewCount="405" Body="&lt;p&gt;I have case control studies,4 different condition where I have 1 control and against which I'm have 3 comparisons to make.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did WGCNA for individuals samples .Here I have the kept the number of genes equal in all the cases .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now my question is whether the number of modules which are obtained from WGCNA would be similar in number if I run module preservation between two condition .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lets say i get 15 modules in Control condition and 16 treated condition if I run module preservation the number of modules which would come would be 15+16= 31 or its independent .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The power I have kept same for both WGCNA and module preservation analysis. But Still I get different module numbers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion would be highly appreciated &lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2019-10-18T17:27:36.977" Title="WGCNA module preservation analysis" Tags="|wgcna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10613" PostTypeId="2" ParentId="10612" CreationDate="2019-10-18T17:27:36.977" Score="2" Body="&lt;p&gt;Your question is not clear, you seem to mix the relevant terminology... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, you cannot run WGCNA for individual samples, only for individual conditions (with or without WT, depending on what question you're trying to answer and assuming you have enough samples in each condition). If that's what you have done, fine. The number of modules in each independent analysis will generally be different, that's because each condition is different. The module identification built into WGCNA as the default approach does not allow the user to specify the number of modules beforehand, but if you insist on getting a pre-specified number of modules, you could run WGCNA using the &quot;manual&quot; way (for example, Tutorial I, section 2b) and substitute another clustering method (e.g., PAM) that allows you to specify the number of clusters. You may find the results inferior though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second, module preservation does not result in new modules. Module preservation evaluates whether modules found in data set A are preserved in a different data set B. To get new, common modules, you could run a consensus network analysis on pairs of conditions or all 3 conditions together. The number of modules you would find in a consensus analysis cannot be predicted from the number of modules you find in individual data sets. &lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2019-10-18T17:27:36.977" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10614" PostTypeId="1" CreationDate="2019-10-19T16:46:33.053" Score="0" ViewCount="63" Body="&lt;p&gt;I have a gene expression raw counts like below for 16 patients &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;         A  B    C      D   E   F&#xA;A2M     511 1623 665    208 553 469&#xA;AADAT   34  137 372     7   52  124&#xA;ABCB1   119 114 123    22   25  89&#xA;ABCB11  27  186 200    27   30  49&#xA;ABCC2   1   10  21     1    8   3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For each patients I also have some characteristics &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Patients                          IC          TCs&#xA;A1                                3            3&#xA;B1                                2            0&#xA;C1                                2            0&#xA;D1                                0            0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to find genes differential expressed in these patients regarding these characteristics but I don't know how to do that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help please?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-10-21T10:14:54.043" LastActivityDate="2019-10-21T10:14:54.043" Title="How I deal with this expression set?" Tags="|r|rna-seq|differential-expression|modelling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10615" PostTypeId="1" CreationDate="2019-10-19T20:55:01.703" Score="2" ViewCount="562" Body="&lt;p&gt;Is there a specification anywhere for how a Nanopore FASTQ header should be formatted?   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am creating &lt;a href=&quot;https://lab41.github.io/hypothesis-bio/api/fastq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;strategies for generating test FASTQ files&lt;/a&gt; and would like to ensure I am generating &quot;correct&quot; nanopore &lt;a href=&quot;https://lab41.github.io/hypothesis-bio/api/sequence_identifiers.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sequence identifiers&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="149" LastEditorUserId="149" LastEditDate="2019-10-26T03:56:55.823" LastActivityDate="2021-12-09T03:25:34.087" Title="Nanopore FASTQ header specifications" Tags="|nanopore|fastq|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10616" PostTypeId="2" ParentId="10594" CreationDate="2019-10-19T21:06:02.910" Score="5" Body="&lt;p&gt;Here is the translation:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$l$&lt;/span&gt; = gap length&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$q$&lt;/span&gt; = initial gap penalty for &lt;em&gt;shorter&lt;/em&gt; gaps&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$e$&lt;/span&gt; = &lt;em&gt;short&lt;/em&gt; gap extension penalty per every additional gap&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\tilde{q}$&lt;/span&gt; = initial gap penalty for &lt;em&gt;longer&lt;/em&gt; gaps&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\tilde{e}$&lt;/span&gt; = &lt;em&gt;long&lt;/em&gt; gap extension penalty per every additional gap&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To understand the first of the two expressions being minimized, see the Wikipedia section on &lt;a href=&quot;https://en.wikipedia.org/wiki/Gap_penalty#Affine&quot; rel=&quot;noreferrer&quot;&gt;affine gap penalties&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice the similar notation structure&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$A + B \cdot L$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; is the known gap penalty,&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; is the gap extension penalty, and&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$L$&lt;/span&gt; is the length of the gap&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To understand the other variables with the tildes on top, see the text below Equation 2 (emphasis added to key explanation):&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;On the condition that &lt;span class=&quot;math-container&quot;&gt;$e &amp;gt; \tilde{e}$&lt;/span&gt;⁠,&#xA;  it applies cost&#xA;  &lt;span class=&quot;math-container&quot;&gt;$q + |l| \cdot e$&lt;/span&gt;&#xA;  to gaps shorter than&#xA;  &lt;span class=&quot;math-container&quot;&gt;$[(\tilde{q} - q)/(e - \tilde{e})]$&lt;/span&gt;&#xA;  and applies&#xA;  &lt;span class=&quot;math-container&quot;&gt;$\tilde{q} + |l| \cdot \tilde{e}$&lt;/span&gt;&#xA;  to longer gaps. &lt;strong&gt;This scheme helps to recover longer insertions and deletions (INDELs)&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In other words, we make the assertion that we should penalize longer gaps differently than shorter gaps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a bonus, here's a visual understanding of why we use the minimization function on those two expressions using Python and some made-up (and probably not biologically useful) penalty terms. &lt;strong&gt;Note&lt;/strong&gt; the requirement that &lt;span class=&quot;math-container&quot;&gt;$e &amp;gt; \tilde{e}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;# Import libraries&#xA;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;# Setup&#xA;l = np.array(range(0, 15, 1))  &#xA;y_short = eval('10 + 5*l') # Q=10, E=5&#xA;y_long = eval('20 + 3*l')  # Q=20, E=3&#xA;&#xA;# Plot&#xA;plt.plot(l, y_short)&#xA;plt.plot(l, y_long)&#xA;plt.legend([&quot;Short Gap Penalty&quot;, &quot;Long Gap Penalty&quot;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/VZlcn.png&quot; alt=&quot;Plot example expressions for long and short gap penalty expressions&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using our example and the expression &lt;span class=&quot;math-container&quot;&gt;$[(\tilde{q} - q)/(e - \tilde{e})]$&lt;/span&gt;, it will apply the first expression in the minimization function (the one without the tildes) when the gap length is shorter than 5.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;[(\tilde{q} - q)/(e - \tilde{e})]&#xA;=&#xA;[20 - 10)/(5 - 3)]&#xA;=&#xA;10 / 2&#xA;= 5&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can see from our plot that the two lines intersect at 5, which we've verified through some arithmetic.&lt;/p&gt;&#xA;" OwnerUserId="4719" LastEditorUserId="4719" LastEditDate="2019-10-20T10:40:10.560" LastActivityDate="2019-10-20T10:40:10.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10617" PostTypeId="2" ParentId="10614" CreationDate="2019-10-20T05:55:49.413" Score="1" Body="&lt;p&gt;Look up a tutorial on DESeq2.  To keep things simple for your first time, I'd reduce your phenotypes to &quot;high&quot; and &quot;low&quot; bins instead of trying to use the cell count numbers as they are.  Then once you understand what the software is doing, you can get fancier.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-10-20T05:55:49.413" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10618" PostTypeId="1" AcceptedAnswerId="10621" CreationDate="2019-10-20T22:41:45.553" Score="2" ViewCount="128" Body="&lt;p&gt;I'm tring to do my M.Sc. research and I Have to do Docking to save some money as I cannot try all the compound that I'm working on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;what is the best Protein-Ligand Docking programme?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;And anyone has any tutorial for such programmes.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="4794" LastEditorUserId="6322" LastEditDate="2019-11-08T09:49:03.450" LastActivityDate="2020-03-25T11:43:49.670" Title="Asking for Docking programme" Tags="|docking|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10619" PostTypeId="2" ParentId="10611" CreationDate="2019-10-21T08:53:51.513" Score="1" Body="&lt;p&gt;I have not received any answer to the question, so I created a small tool to fix the file myself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am assuming as per the question that there are perfect matches upstream and downstream to the new sequence, and that anything in between can be deleted/updated by the new sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the first_n and last_n bases of the diagram in the question, and the entire patch sequence, we can create a single-line string of the reference with &lt;code&gt;seqtk seq -l9999999 -&lt;/code&gt; and then edit it with the line below (in Perl regex):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$chromosome_str =~ s/$&lt;/span&gt;first_n.+&lt;span class=&quot;math-container&quot;&gt;$last_n/$&lt;/span&gt;patch/i&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This creates the new string that can be used as the new fasta file.&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="180" LastEditDate="2019-10-21T09:40:47.163" LastActivityDate="2019-10-21T09:40:47.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10620" PostTypeId="2" ParentId="10611" CreationDate="2019-10-21T09:54:41.717" Score="0" Body="&lt;p&gt;You can give a try with &lt;code&gt;megamerger&lt;/code&gt; from the &lt;code&gt;emboss&lt;/code&gt; suite if you have (almost) perfect sequence match.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case the gap within your &lt;code&gt;chr.fasta&lt;/code&gt; complicates the sequence alignment and hence the resulting contig, you can generate a &quot;tiling path&quot; with: upstream of the gap from &lt;code&gt;chr.fasta&lt;/code&gt; + &lt;code&gt;contig.fasta&lt;/code&gt; + downstream of the gap from &lt;code&gt;chr.fasta&lt;/code&gt; (there should be sequence overlaps of course)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In a (more realistic) scenario where there is no perfect sequence match between the overlapping regions, you can fall back to &lt;code&gt;emboss merger&lt;/code&gt;, however, it is not designed to work on large sequences.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-10-21T09:54:41.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10621" PostTypeId="2" ParentId="10618" CreationDate="2019-10-21T11:01:57.943" Score="4" Body="&lt;h1&gt;It's case specific&lt;/h1&gt;&#xA;&lt;p&gt;Docking small compounds well is problematic and you need to consider whether:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;if a single conformer is okay&lt;/li&gt;&#xA;&lt;li&gt;if rigid sidechains are okay&lt;/li&gt;&#xA;&lt;li&gt;if implicit water is okay&lt;/li&gt;&#xA;&lt;li&gt;if a non-polarisable model is okay&lt;/li&gt;&#xA;&lt;li&gt;if a non-covalent interaction is okay&lt;/li&gt;&#xA;&lt;li&gt;if you are okay with a score that is a rough ΔΔGibbs approximation of the interaction (see &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/10518/what-value-of-reference-rmsd-after-docking-simulation-from-any-starting-coordina/10590#10590&quot;&gt;this SO answer&lt;/a&gt;) or do you want something more like a K_d&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Do note that for any experiment you do to have any validity you need to have controls. Do you know what can and cannot bind already? It is imperative that you score these too.&lt;/p&gt;&#xA;&lt;h1&gt;Free&lt;/h1&gt;&#xA;&lt;p&gt;I am assuming that you want to use a free program. Virtual compound screens (VCS) are a big business in pharma. Therefore, a lot of really good software is not free, such as ICM-Dock, which outscores AutoDock discuss below &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5425816/&quot; rel=&quot;nofollow noreferrer&quot;&gt;in most tests&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;AutoDock: simple cases&lt;/h2&gt;&#xA;&lt;p&gt;AutoDock Vina is easy to use and has a PyMOL plugin (I believe), but the peptide, in both its sidechains and its backbone, is rigid, which is not good. The ligand rotamers are generated for AutoDock at runtime, but there is a hardcoded low limit, it technically isn't a rigid ligand, but nearly.&#xA;It is the best-scoring free rigid peptide docking program.&lt;/p&gt;&#xA;&lt;p&gt;There are lots of tutorials on using Autodock, such as the &lt;a href=&quot;http://vina.scripps.edu/tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;official one&lt;/a&gt; or on &lt;a href=&quot;https://www.youtube.com/watch?v=blxSn3Lhdec&quot; rel=&quot;nofollow noreferrer&quot;&gt;YouTube&lt;/a&gt;. As shown by this &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/11471/protein-ligand-docking-how-to-convert-protein-pdb-to-protein-maps-fld/11479#11479&quot;&gt;Bioinformatics Stack Exchange question&lt;/a&gt; Autodock 4 (different score fxn) has several steps, each important.&lt;/p&gt;&#xA;&lt;h2&gt;Rosetta: very complicated cases&lt;/h2&gt;&#xA;&lt;p&gt;I prefer the harder to use Rosetta Ligand_dock which uses conformers (if the ligand is parameterised correctly, say they were generate via Open Babel conformers) and allows sidechains to repack.&lt;/p&gt;&#xA;&lt;h2&gt;Water&lt;/h2&gt;&#xA;&lt;p&gt;Both tools mentioned use implicit waters, which works badly for highly hydrophobic ligands.&#xA;That is the waters are simulated as a homogeneous field that affects the ligand and protein. But some solutions are possible, in the case of AutoDock WaterDock variant exists —although this is not polarisable water.&#xA;For more advanced cases there are MD methods such as dynamic undocking, which are completely non-trivial. All these methods mentioned use classical &lt;a href=&quot;https://en.wikipedia.org/wiki/Force_field_(chemistry)&quot; rel=&quot;nofollow noreferrer&quot;&gt;force-field calculations&lt;/a&gt; (Amber, Charmm, Talaris etc. are different models), but in MD noise (think of it as environmental heat) is present in the calculations, so the ligand comes unbound (and is actually pulled out). You can also get better results by refining your bound ligand with QM-MD (&lt;em&gt;e.g.&lt;/em&gt; gauss of gaussian).&lt;/p&gt;&#xA;&lt;h2&gt;Choices...&lt;/h2&gt;&#xA;&lt;p&gt;This is a lot of options. So a first question is are your ligands man-made drugs/fragments (i.e. obey &lt;a href=&quot;https://en.wikipedia.org/wiki/Lipinski%27s_rule_of_five&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lipinski's rule of 5&lt;/a&gt;) or metabolites? Generally the former are rigid and very hydrophobic (greasy) (and require better force fields), while the latter are flexible and hydrophilic.&lt;/p&gt;&#xA;&lt;h2&gt;Minor considerations&lt;/h2&gt;&#xA;&lt;p&gt;Also to consider is your entranceway to your protein as the Michaelis constant is dictated in some enzymes by the substrate going in, not by its binding (e.g. P450s).&lt;/p&gt;&#xA;&lt;p&gt;If the ligand have a shared backbone that is fixed there are some tricks you can do to cut your compute time and increase the reliability.&lt;/p&gt;&#xA;&lt;p&gt;If the ligands are bound not all methods are possible. Rosetta ligand_dock can but it is a bit fiddly to parameterize.&lt;/p&gt;&#xA;&lt;h2&gt;RDKit skills&lt;/h2&gt;&#xA;&lt;p&gt;One thing also to consider is your programming skills. Can you use RDKit? In most cases downloading the sdf off PubChem and running OBabel or similar to generate conformers is not possible and you have be familiar with a python package called RDKit, which is really powerful but non-trivial to use.&lt;/p&gt;&#xA;&lt;p&gt;Additionally, docking scores can be strongly augmented with ligand-only values (such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Partition_coefficient&quot; rel=&quot;nofollow noreferrer&quot;&gt;logP&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Polar_surface_area&quot; rel=&quot;nofollow noreferrer&quot;&gt;TPSA&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Molar_refractivity&quot; rel=&quot;nofollow noreferrer&quot;&gt;molar refractivity&lt;/a&gt;, molecular weight, QED, Bertz topology etc. etc) that are generated from RDKit.&lt;/p&gt;&#xA;&lt;p&gt;NB. The scoring function in Autodock Vina (not 4 though, as that is Amber FF) partially takes into account.&lt;/p&gt;&#xA;&lt;h2&gt;Ligand_dock details&lt;/h2&gt;&#xA;&lt;p&gt;Ligand_dock is an older application of the Rosetta suite and the current philosophy is to use a specific Rosetta script, however, it works fine and is actually quicker to implement. Here are some links:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.rosettacommons.org/docs/latest/application_documentation/docking/ligand-dock&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual of ligand_dock&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.rosettacommons.org/docs/latest/rosetta_basics/preparation/preparing-ligands&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial for ligand parameterisation&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pubchem.ncbi.nlm.nih.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PubChem for your SDF files&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://cactus.nci.nih.gov&quot; rel=&quot;nofollow noreferrer&quot;&gt;online tools for small molecule manipulation&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://blog.matteoferla.com/2019/10/rdkit-for-rosetta-plp-ligand-space-as.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;example of ligand creation for docking with RDKit&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2020-03-25T11:43:49.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10622" PostTypeId="2" ParentId="264" CreationDate="2019-10-21T23:01:05.413" Score="1" Body="&lt;p&gt;Let's say you have an INFO field called &lt;code&gt;SVTYPE&lt;/code&gt; to indicate the structure variant type.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is how you get the stats easily:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Install &lt;code&gt;vcfstats&lt;/code&gt;: &lt;code&gt;pip install vcfstats&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Define a macro to extract the &lt;code&gt;SVTYPE&lt;/code&gt; information, say in &lt;code&gt;svtype.py&lt;/code&gt;:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from vcfstats.macros import cat&#xA;@cat&#xA;def SVTYPE(variant):&#xA;    return variant.INFO['SVTYPE']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Generate the statistics:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;vcfstats --vcf &amp;lt;your vcf file&amp;gt; \&#xA;   --outdir &amp;lt;the output directory&amp;gt; \&#xA;   --macro svtype.py \&#xA;   --formula 'COUNT(1) ~ SVTYPE[DEL,INS,INV...]' \ # desired svtypes&#xA;   --title 'Number of SV types'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then in the output directory you will have the stats file named &lt;code&gt;Number_of_SV_Types.txt&lt;/code&gt; and a plot for it as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check the details at &lt;a href=&quot;https://github.com/pwwang/vcfstats&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/pwwang/vcfstats&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6383" LastActivityDate="2019-10-21T23:01:05.413" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10623" PostTypeId="1" CreationDate="2019-10-22T03:28:49.990" Score="3" ViewCount="1512" Body="&lt;p&gt;When reading &lt;a href=&quot;https://doi.org/10.1093/nar/gkt809&quot; rel=&quot;nofollow noreferrer&quot;&gt;a paper&lt;/a&gt;, I came across the following statement&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Standard vectors were constructed, flanked by pairs of homology sequences derived from yeast barcodes at the ends of each expression cassette. We reasoned that as these barcode sequences were designed to be &lt;strong&gt;orthogonal&lt;/strong&gt;, they could serve a dual purpose of reducing the probability of mis-annealing and dictating the assembly order of multiple cassettes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I am not sure what the &quot;orthogonal&quot; means here. I think it is about synthetic biology.&lt;/p&gt;&#xA;" OwnerUserId="6384" LastEditorUserId="6384" LastEditDate="2019-11-03T01:54:51.470" LastActivityDate="2020-04-01T02:01:50.193" Title="What are Orthogonal DNA Sequences?" Tags="|phylogenetics|" AnswerCount="1" CommentCount="6" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10624" PostTypeId="1" CreationDate="2019-10-22T06:47:12.840" Score="1" ViewCount="101" Body="&lt;p&gt;I am currently doing a Differential Gene Expression of RNAseq data in Lung Adenocarcinoma using TCGAbiolinks. In the data Preprocessing step TCGAanalyze_Normalisation, I am confused as to which method to select for normalization (geneLength or gcContent). What are the possible implications of both methods and how to decide which is the best suited model for the analysis?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better way to perform the DGE ?&lt;/p&gt;&#xA;" OwnerUserId="4121" LastEditorUserId="4121" LastEditDate="2019-10-22T08:56:53.683" LastActivityDate="2019-10-23T01:06:24.073" Title="RNAseq data analysis" Tags="|rna-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10625" PostTypeId="1" AcceptedAnswerId="10626" CreationDate="2019-10-22T08:24:04.293" Score="2" ViewCount="450" Body="&lt;p&gt;I am using a &lt;a href=&quot;https://community.nanoporetech.com/knowledge/bioinformatics/structural-variation-analy-5068/tutorial#_20_glossary_of_terms&quot; rel=&quot;nofollow noreferrer&quot;&gt;pipeline&lt;/a&gt; from nanopore to detect structural variants (SVs) in a human sample with long-reads sequencing. The first steps of the pipeline are:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;index the reference genome with &lt;code&gt;minimap2&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;map the long reads to a reference genome using same tool as above&lt;/li&gt;&#xA;&lt;li&gt;Get BED coordinates from the &lt;code&gt;.bam&lt;/code&gt; file.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;While the first two steps uses &lt;code&gt;minimap&lt;/code&gt;, the third one uses a python script written by the developers of the pipeline, which can be found &lt;a href=&quot;https://github.com/nanoporetech/pipeline-structural-variation/blob/a5fec9407f24362c759630b5d331e47a76a9f107/lib/pipeline_structural_variation/sniffles_bamref2bed.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After the pipeline calls &lt;code&gt;bamref2bed&lt;/code&gt; i.e. the python script above, my &lt;code&gt;target.bed&lt;/code&gt; looks like the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    0   249250621&#xA;chr2    0   243199373&#xA;chr3    0   198022430&#xA;chr4    0   191154276&#xA;chr5    0   180915260&#xA;chr6    0   171115067&#xA;chr7    0   159138663&#xA;chr8    0   146364022&#xA;chr9    0   141213431&#xA;chr10   0   135534747&#xA;chr11   0   135006516&#xA;chr12   0   133851895&#xA;chr13   0   115169878&#xA;chr14   0   107349540&#xA;chr15   0   102531392&#xA;chr16   0   90354753&#xA;chr17   0   81195210&#xA;chr18   0   78077248&#xA;chr19   0   59128983&#xA;chr20   0   63025520&#xA;chr21   0   48129895&#xA;chr22   0   51304566&#xA;chrX    0   155270560&#xA;chrY    0   59373566&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As far as I am concerned the first three columns of a bed file refer to (I) chr name, (II) start position (III) end position. The main thing I am wondering is how come all chromosomes have a start position of 0. Also, reading about &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/genomecov.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;converting bam to bed files&lt;/a&gt; I guess the most important feature would be the coverage across the whole chromosomes, which in this file is missing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But my main question would be: is it normal to have a start position of 0 for all chromosomes or is there something wrong here?&lt;/p&gt;&#xA;" OwnerUserId="3452" LastActivityDate="2019-10-22T09:02:28.150" Title="BED file from .bam alignment structure" Tags="|sequence-alignment|nanopore|bed|long-reads|structural-variation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10626" PostTypeId="2" ParentId="10625" CreationDate="2019-10-22T09:02:28.150" Score="4" Body="&lt;p&gt;I cannot access the pipeline from nanopore, but looking at the &lt;a href=&quot;https://github.com/nanoporetech/pipeline-structural-variation/blob/a5fec9407f24362c759630b5d331e47a76a9f107/lib/pipeline_structural_variation/sniffles_bamref2bed.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;python script&lt;/a&gt; you provided:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;with pysam.AlignmentFile(bam_file, &quot;r&quot;) as bam:&#xA;    for tid in range(0, bam.nreferences):&#xA;        ref_name = bam.get_reference_name(tid)&#xA;        if len(list(filter(lambda x: x in ref_name, filter_names))) &amp;gt; 0:&#xA;            continue&#xA;&#xA;        print(ref_name,&#xA;              &quot;0&quot;,&#xA;              str(bam.get_reference_length(ref_name)),&#xA;              sep='\t')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So the script gets the length of the chromosome and by default, outputs the start as 0.&lt;a href=&quot;http://genome.ucsc.edu/blog/the-ucsc-genome-browser-coordinate-counting-systems/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zero based coordinate system&lt;/a&gt; is sometimes used in UCSC systems, so I am only guessing the authors of the pipeline use this convention somewhere further down.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for the coverage you mentioned, that is usually output as a score in the 5th column (&lt;a href=&quot;https://genome.ucsc.edu/FAQ/FAQformat.html#format1&quot; rel=&quot;nofollow noreferrer&quot;&gt;bed format&lt;/a&gt;). In those situations, usually the coordinates of the alignments in the bam file are converted into bed format. In this pipeline, i think the authors are simply extracting the chromosome information out and putting it into a tabular form.&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2019-10-22T09:02:28.150" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10628" PostTypeId="1" CreationDate="2019-10-22T18:18:51.460" Score="1" ViewCount="287" Body="&lt;p&gt;We currently use DADA2 for picking ASVs and the &lt;code&gt;assignTaxonomy&lt;/code&gt; funciton for assingment to genera. Google does bring up various recent articles on species-level assignment with 16S. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone have an opinion on the current best-practice?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More simply, are there any current candidates for reasonably accurate species-level assignment?&lt;/p&gt;&#xA;" OwnerUserId="4758" LastEditorUserId="3848" LastEditDate="2020-04-26T05:06:29.547" LastActivityDate="2020-06-26T13:07:49.573" Title="16S species-level taxonomic assignment--what is the current state of the art?" Tags="|fasta|phylogenetics|phylogeny|machine-learning|bacteria|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10630" PostTypeId="1" CreationDate="2019-10-22T20:30:27.133" Score="0" ViewCount="177" Body="&lt;p&gt;I have a list of cell lines that should be compared against another list of cell lines as pairs. I am trying to find a way in order for me to compare two cells such as &lt;strong&gt;MCF7&lt;/strong&gt;    and &lt;strong&gt;10964c&lt;/strong&gt; cell lines. I am using python so, can anyone provide some steps so that I can do the comparison. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also have their ontologies in an OBO format as I need them to do the comparison. if there is a library or a certain procedure that can solve this problem will be a great help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The comparison I'm trying to do is to measure the similarity in different ways, there are many measurement methods such as the distance in the ontological hierarchy between two cells by calculating the shortest path. You can also measure the number of common ancestors. But also, there are libraries that perform complicated (and more accurate) similarity measurements like JiangConrath, Jaccard, or Resnik which is popular. The problem with theses libraries is that they are not oriented to calculate cell line ontologies and may arequire annotation corpus for some similarity measures. &lt;/p&gt;&#xA;" OwnerUserId="4917" LastEditorUserId="3541" LastEditDate="2019-10-25T22:39:50.000" LastActivityDate="2021-03-19T01:00:20.087" Title="Is there a way to measure cell line similarity using python?" Tags="|python|cell-line|ontology|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10631" PostTypeId="2" ParentId="10624" CreationDate="2019-10-23T01:06:24.073" Score="2" Body="&lt;p&gt;The &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/manuals/TCGAbiolinks/man/TCGAbiolinks.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;TCGAbiolinks reference manual&lt;/a&gt; provides a clear explanation of the two methods:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Normalization for RNA-Seq Numerical and graphical summaries of RNA-Seq&#xA;  read data. Withinlane normalization procedures to adjust for&#xA;  GC-content effect (or other gene-level effects) on read counts: loess&#xA;  robust local regression, global-scaling, and full-quantile&#xA;  normalization (Risso et al., 2011). Between-lane normalization&#xA;  procedures to adjust for distributional differences between lanes&#xA;  (e.g., sequencing depth): global-scaling and full-quantile&#xA;  normalization (Bullard et al., 2010).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Ideally, you would know if your samples spanned different lanes or not, which would indicate which you should use - &quot;geneLength&quot; if so, &quot;gcContent&quot; if not. If you're using TCGA data, that metadata is likely available somewhere.&lt;/p&gt;&#xA;" OwnerUserId="1269" LastActivityDate="2019-10-23T01:06:24.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10632" PostTypeId="2" ParentId="10410" CreationDate="2019-10-23T02:56:51.660" Score="1" Body="&lt;p&gt;Smith-Waterman is guaranteed to give you the best alignment given your choice of substitution matrix.  For a few such tiny sequences, it's what I would use.  Many implementations exist.&lt;/p&gt;&#xA;" OwnerUserId="2436" LastActivityDate="2019-10-23T02:56:51.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10633" PostTypeId="1" CreationDate="2019-10-23T08:45:56.943" Score="4" ViewCount="121" Body="&lt;p&gt;Can someone tell the way to extract the fasta sequences for the gene cytb of &lt;em&gt;Acetes japonicus&lt;/em&gt; (shrimp important to China and South Korea)?&lt;/p&gt;&#xA;&lt;p&gt;Can I extract them directly from NCBI nucleotide database (i.e. nuccore)?&lt;/p&gt;&#xA;&lt;p&gt;For instance, I'm trying to fetch the fasta data of &lt;em&gt;Acetes japonicus&lt;/em&gt; of cytb gene. So I'm using biopython, like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; handle = Entrez.esearch(db=&amp;quot;nucleotide&amp;quot;, term=&amp;quot;Acetes japonicus[Orgn] AND cytb[Gene]&amp;quot;)&#xA;record=Entrez.read(handle)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the output doesn't give the an ID, however when I search manually I get the result.&lt;/p&gt;&#xA;" OwnerUserId="2686" LastEditorUserId="3848" LastEditDate="2022-02-22T18:43:09.333" LastActivityDate="2024-03-14T02:52:03.013" Title="Extracting a cytochrome B sequence from NCBI's nucleotide database" Tags="|database|nucleotide-models|efetch|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10635" PostTypeId="2" ParentId="10628" CreationDate="2019-10-23T09:42:07.340" Score="0" Body="&lt;p&gt;My advice is to go for &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-018-2182-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fiannaca et al. 2018&lt;/a&gt; or similar and steer clear of the DADA2 classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My reasoning, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;... my criticism of the DADA2 assignments is the reliance on an old machine learning algorithm (naive Bayes). In its favour the taxonomic assignment looks broad. Modern ML will never rely on a single ML method (e.g. lasso regression, SVM, k-means), the only possible exception is &quot;Random Forests&quot;/decision trees, which are popular - but even then. Naïve Bayes (which is not to be confused with 'Bayesian') was popular about a decade ago, which makes sense with the publication cited. The problem is that ML methods give different results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In its favour it does look well trained, but you would need to understand the distribution used in the training. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Again my advice is to go for a deep learning solution, such as &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-018-2182-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fiannaca et al. 2018&lt;/a&gt; which is trained on bacterial 16s.  What looks good here is the CNN architecture. They also implement Deep Belief, which I've never heard of, but suspect it is a variant of RNN. There description suggests they don't much know about it either. Finally they do implement a naïve Bayes solution for comparison.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CNN is the top performing method, which is no surprise. Alot will depend on their filtering and striding, as well as how they trained it. I don't see their training graphs, nor their computer architecture (which are super important). We trust they have done them well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, classification down to the genus level is 85-90%, which for this stuff must be good (for Bioinformatics), but in reality its low.... in fact for DL 85% is not great and needs further attention (when I finally get my CNN working I might live to regret this statement). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Phylogeny (maximum likelihood) is the only error-free way of doing this. There is a bedrock of historic work here. If you want to get down to the species level then trees are your only way to get there. Obviously you'd have a mixed strategy towards that goal.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Final note&lt;/strong&gt;, whilst I am critical of DADA2 for classification their variant calling looks very cool, i.e. the robustness of the NGS data. I definitely get the impression the ML is just an add on for them.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-10-23T16:33:18.950" LastActivityDate="2019-10-23T16:33:18.950" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10636" PostTypeId="1" CreationDate="2019-10-23T09:51:19.393" Score="1" ViewCount="731" Body="&lt;p&gt;I m quoting this  &lt;a href=&quot;https://www.cell.com/cell-stem-cell/fulltext/S1934-5909(17)30429-0?innerTabgraphical_S1934590917304290=&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; &lt;a href=&quot;https://i.stack.imgur.com/9gJFl.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9gJFl.jpg&quot; alt=&quot;figure&quot;&gt;&lt;/a&gt;, figure B where they write &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;B) Violin plots for the normalized ATAC-seq &lt;strong&gt;&lt;em&gt;tag density&lt;/em&gt;&lt;/strong&gt; for all peaks within the indicated OC/CO groups at the indicated time points. Data were converted to a Z score based on the row-wise SD for each peak. ∗p &amp;lt; 0.01 Mann-Whitney U test.&quot; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is where is this tag density information coming from.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far for chipseq data what I have done is&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Aligned the data&lt;/li&gt;&#xA;&lt;li&gt;Then all required data processing post alignment &lt;/li&gt;&#xA;&lt;li&gt;Generated bed and bigwig files of respective samples required for comparison&lt;/li&gt;&#xA;&lt;li&gt;Made tag-directory using homer.&lt;/li&gt;&#xA;&lt;li&gt;Then peak-calling using homer,annotated peak using homer.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Now so far I am not able to figure out where is this tag-density information is .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help or suggestion would be really appreciated.&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2024-02-25T19:27:07.783" Title="Tag density plot form chip or atac seq data" Tags="|chip-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10638" PostTypeId="2" ParentId="10636" CreationDate="2019-10-23T10:23:35.753" Score="2" Body="&lt;p&gt;You can use &lt;code&gt;Homer&lt;/code&gt; to retrieve tag densities, quoting from their &lt;a href=&quot;http://homer.ucsd.edu/homer/ngs/quantification.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Calculating ChIP-Seq Tag Densities across different experiments&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;annotatePeaks.pl is [a] useful program for cross-referencing data from&#xA;  multiple experiments.  In order to count the number of tags from&#xA;  different sequencing experiments, you must first create tag&#xA;  directories for each of these experiments.  Once created, tag counts&#xA;  from these directories in the vicinity of your peaks can be added by&#xA;  specifying &quot;-d   ...&quot;.  You can&#xA;  specify as many tag directories as you like.  Tag totals for each&#xA;  directory will be placed in new columns starting on column 18.  For&#xA;  example:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;annotatePeaks.pl pu1peaks.txt mm8 -size 400 -d Macrophage-PU.1/&#xA;  Bcell-PU.1/ &gt; output.txt&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-10-23T10:54:35.197" LastActivityDate="2019-10-23T10:54:35.197" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10639" PostTypeId="2" ParentId="9015" CreationDate="2019-10-23T12:31:05.670" Score="8" Body="&lt;p&gt;Your setup:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;dict1 = {0:['chr1','chr1','chr1','chr1','chr2'], &#xA;    1:[1, 100, 150, 900, 1], 2:[100, 200, 500, 950, 100], &#xA;    3:['feature1', 'feature2', 'feature3', 'feature4', 'feature4'], &#xA;    4:[0, 0, 0, 0, 0], 5:['+','+','-','+','+']}&#xA;&#xA;df1 = pd.DataFrame(dict1)&#xA;&#xA;print(df1)&#xA;&#xA;##       0    1    2         3  4  5&#xA;## 0  chr1    1  100  feature1  0  +&#xA;## 1  chr1  100  200  feature2  0  +&#xA;## 2  chr1  150  500  feature3  0  -&#xA;## 3  chr1  900  950  feature4  0  +&#xA;## 4  chr2    1  100  feature4  0  +&#xA;&#xA;dict2 = {0:['chr1','chr1'], 1:[155, 800], 2:[200, 901], &#xA;    3:['feature5', 'feature6'], 4:[0, 0], 5:['-','+']}&#xA;&#xA;df2 = pd.DataFrame(dict2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now use pyranges (&lt;a href=&quot;https://github.com/biocore-ntnu/pyranges&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/biocore-ntnu/pyranges&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# pip install pyranges&#xA;# conda install -c bioconda pyranges&#xA;&#xA;import pyranges as pr&#xA;cols = &quot;Chromosome Start End Name Score Strand&quot;.split()&#xA;# Add column names to the dfs so PyRanges knows which &#xA;# represent Chromosomes, Starts, Ends and optionally Strand&#xA;&#xA;df1.columns = cols&#xA;df2.columns = cols&#xA;&#xA;# create PyRanges-objects from the dfs&#xA;gr1, gr2 = pr.PyRanges(df1), pr.PyRanges(df2)&#xA;&#xA;# intersect the two&#xA;gr = gr1.intersect(gr2)&#xA;&#xA;# get the result as a   df&#xA;df = gr.df&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print(gr)&#xA;# +--------------+-----------+-----------+------------+-----------+--------------+&#xA;# | Chromosome   |     Start |       End | Name       |     Score | Strand       |&#xA;# | (category)   |   (int32) |   (int32) | (object)   |   (int64) | (category)   |&#xA;# |--------------+-----------+-----------+------------+-----------+--------------|&#xA;# | chr1         |       155 |       200 | feature2   |         0 | +            |&#xA;# | chr1         |       900 |       901 | feature4   |         0 | +            |&#xA;# | chr1         |       155 |       200 | feature3   |         0 | -            |&#xA;# +--------------+-----------+-----------+------------+-----------+--------------+&#xA;# Stranded PyRanges object has 3 rows and 6 columns from 1 chromosomes.&#xA;# For printing, the PyRanges was sorted on Chromosome and Strand.&#xA;print(df)&#xA;#   Chromosome  Start  End      Name  Score Strand&#xA;# 0       chr1    155  200  feature2      0      +&#xA;# 1       chr1    900  901  feature4      0      +&#xA;# 2       chr1    155  200  feature3      0      -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;See the docs for more info: &lt;a href=&quot;https://biocore-ntnu.github.io/pyranges/&quot; rel=&quot;noreferrer&quot;&gt;https://biocore-ntnu.github.io/pyranges/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2688" LastEditorUserId="2688" LastEditDate="2019-10-24T07:00:45.290" LastActivityDate="2019-10-24T07:00:45.290" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10640" PostTypeId="2" ParentId="10630" CreationDate="2019-10-23T13:57:07.207" Score="1" Body="&lt;p&gt;At its current form, your question is not clear, for example which aspects of these cell lines are your trying to compare?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding your point on the ontologies, in &lt;code&gt;Python&lt;/code&gt; you can use the &lt;a href=&quot;https://pypi.org/project/pronto/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;pronto&lt;/code&gt;&lt;/a&gt; library to work with ontologies in OBO format.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-10-23T13:57:07.207" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10641" PostTypeId="1" CreationDate="2019-10-23T16:04:04.177" Score="2" ViewCount="1292" Body="&lt;p&gt;I was analysing a bad sequencing run of some RNA data using FastQC, I supplied it RNA-STAR-aligned bam files (and the human reference).  The output of MultiQC had many more &amp;quot;unique reads&amp;quot; in the sequence counts table than there were reads in the FastQ files that were fed to RNA-STAR.&lt;/p&gt;&#xA;&lt;p&gt;At first, I realized that all the FastQC stats were about &amp;quot;mappings&amp;quot;, not reads (as they were labeled), but &amp;quot;unique mappings&amp;quot; doesn't seem to make sense either, because even if I consider the possibility that segments of reads were mapping in different places, the numbers don't make sense.  Here is the worst case:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Max read length after trimming: 47 bases&lt;/li&gt;&#xA;&lt;li&gt;Number of reads in the original FastQ file that went into STAR: 59M&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Bam Stat Table Numbers:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Total records: 391 million&lt;/li&gt;&#xA;&lt;li&gt;Unique mappings: 3.1 million&lt;/li&gt;&#xA;&lt;li&gt;Non-primary Hits: 335 million&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;FastQC Sequence Counts Bar graph:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&amp;quot;Unique reads&amp;quot;: 185 million&lt;/li&gt;&#xA;&lt;li&gt;&amp;quot;Duplicate reads&amp;quot;: 206 million&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;While I found documentation stating that you can run FastQC on bam files, there were no caveats I could find about it needing to be unmapped reads.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone explain precisely why there can be 185 million &amp;quot;unique reads&amp;quot; when bam stats says there were 3 million unique mappings and the number of reads is 59M?&lt;/p&gt;&#xA;" OwnerUserId="5063" LastEditorUserId="5063" LastEditDate="2020-07-14T17:06:34.130" LastActivityDate="2020-07-14T17:06:34.130" Title="What pitfalls exist with running FastQC on a bam file?" Tags="|sam|rna-alignment|fastqc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10642" PostTypeId="1" CreationDate="2019-10-24T05:16:23.690" Score="0" ViewCount="36" Body="&lt;p&gt;We have done some RNA-Seq on Arabidopsis T-DNA lines and mapped the reads on to Arabidopsis genome. Now we wanted to confirm the T-DNA deletion of those lines by loading them on IGV. As a control we also wanted to load T-DNA lines as tracks on IGV to make sure that the T-DNA lines we chose were correct. I am wondering where can i get bam files for those confirmed T-DNA lines?&lt;/p&gt;&#xA;" OwnerUserId="2881" LastActivityDate="2019-10-24T05:16:23.690" Title="How to visualise /add Arabidopsis T-DNA Lines As Tracks On IGV" Tags="|sam|visualization|transcriptome|igv|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10643" PostTypeId="1" AcceptedAnswerId="10644" CreationDate="2019-10-24T10:13:30.567" Score="2" ViewCount="1901" Body="&lt;p&gt;After using files that I downloaded from the SRA with fasterq-dump, I realize I am not 100% sure that I have all the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I noticed in my downstream analysis that I seem to be missing the .1 and .2 numbers in the code associated with individual reads with the same spot ID. Then I noticed that my initial fastq file doesn't seem to have those either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I'm wondering if the -split-spot option that I used is doing what I want it do do, which is 'get all the sequence data', and how does it differ from --concatenate-reads. I am afraid I did not understand this from the program description. Please help?&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2021-03-23T19:20:57.160" Title="Fasterq-dump: --split-spot or -concatenate-reads?" Tags="|fastq|sratoolkit|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10644" PostTypeId="2" ParentId="10643" CreationDate="2019-10-24T14:26:48.720" Score="3" Body="&lt;p&gt;Fasterq comes from the latest version of sratools. So if you check the &lt;a href=&quot;https://github.com/ncbi/sra-tools/wiki/HowTo:-fasterq-dump&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt; , it says the equivalence is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;fastq-dump SRRXXXXXX --split-3 --skip-technical&lt;/p&gt;&#xA;&lt;p&gt;fasterq-dump SRRXXXXXX&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In older versions of sratoolkit, if you use fastq-dump without specifying --split-3 for paired-end reads, you get the format mentioned, spotID.1 for forward, spotID.2 for reverse:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fastq-dump SRR6462984.sra&#xA;more SRR6462984.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You get:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@SRR6462984.1 1 length=301&#xA;NATCTGCCCGTTCCACATAAACACCGTTAAACATCGATAGAGCGAAATAAAGCTGCTGAGTGATGTGATGATAAAGCTTTTCCGTTCTGACACGATCTTCCATCTCGTCTATCATTGCATCGAGAGCGTCAGAGATCGCAAGCAGTGCGGATATTGCCGTCCGTCTTGAAGCTTTGGCAGAGCCAGGCGGAATTTGTTTATCGGATAGTGTTTATGCGCAAATTTAGCATATGATTTCCGACAACTTTTGCTTCATCGGGCCGCAAAGCGTGAAAAACAGTGTCGTCACCGTTGATGTGTG&#xA;+SRR6462984.1 1 length=301&#xA;#/AAAEE/EEEA/EAAEEAEE/EE/EEA/EEEEE/AAE/EE/EAEAE//EEAEE/EEEEE&amp;lt;EEEEE/EE&amp;lt;EE/AEEEE/EAEEEEA/E&amp;lt;EEAEAEEEEEEAEEEE&amp;lt;EEAEEEEEAAAEEE/EEEEE/E//AEEEE6EA66EE&amp;lt;&amp;lt;&amp;lt;EAAA/AAAAAE/E6E/EA/AE/AEEEEEEAAEEEEA6EEEEEEEEE/EEE/AEEAAAEEEAE/E/EEEEEA//E//EEE/EEE/A&amp;lt;//AE/EE//6A///AA/E//E/E/EE//AE//AAEEE////E/A&amp;lt;AAE/EEAE//A//6/A&amp;lt;/&amp;lt;6/&amp;lt;/E/&#xA;@SRR6462984.2 2 length=302&#xA;NTCAGCAGGCGCCGTAACTTCAGCAAAACTGTAGCTATAACTTTTCCGAACTTTATACCTAACTTCGCTTTCCTAGCACTGTATAAAATGCCTAAAAGAAATCAGCAGGCGCCGTCACTTCAAAAAATCAGTATGGATCAATCAGACGCGGAGCTTATTAAAACAGAGCTTATTAAAAATTATGTCTTTCTTCTTCTTTCAGCAATTCGCAACGTAAAATTCAAAATTTTGGACGCCGTAATTGCGCAGGAACCAATTCAAAGAGGCGGCGGCCAATTAATTCCAACGTCTCGAAGAGTGGC&#xA;+SRR6462984.2 2 length=302&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With the latest version, it by defaults splits the paired end sra file into forward and reverse fastq (i.e --split-3):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fasterq-dump SRR6462984.sra &#xA;head -2 SRR6462984_*.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And you get&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;==&amp;gt; SRR6462984_1.fastq &amp;lt;==&#xA;@SRR6462984.1 1 length=150&#xA;NATCTGCCCGTTCCACATAAACACCGTTAAACATCGATAGAGCGAAATAAAGCTGCTGAGTGATGTGATGATAAAGCTTTTCCGTTCTGACACGATCTTCCATCTCGTCTATCATTGCATCGAGAGCGTCAGAGATCGCAAGCAGTGCGG&#xA;&#xA;==&amp;gt; SRR6462984_2.fastq &amp;lt;==&#xA;@SRR6462984.1 1 length=151&#xA;ATATTGCCGTCCGTCTTGAAGCTTTGGCAGAGCCAGGCGGAATTTGTTTATCGGATAGTGTTTATGCGCAAATTTAGCATATGATTTCCGACAACTTTTGCTTCATCGGGCCGCAAAGCGTGAAAAACAGTGTCGTCACCGTTGATGTGTG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So if you need to get back the .1 and .2 format, use fastq-dump from the latest installation.&lt;/p&gt;&#xA;&lt;p&gt;I would suggest sticking to the newest format, with 2 files, because fastq-dump will be deprecated in the future.&lt;/p&gt;&#xA;" OwnerUserId="5077" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2019-10-24T14:26:48.720" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10647" PostTypeId="1" AcceptedAnswerId="10648" CreationDate="2019-10-25T12:17:46.933" Score="1" ViewCount="117" Body="&lt;p&gt;I am completing the online tutorials for module preservation of human brain modules in chimpanzee brains and vice versa (III). If anyone is aware of this tutorial or practiced before wondering if can help with quick question please.&#xA;I completed sections 1a and 2a to upload the expression data and then proceeded to Appendix A to complete Network Analysis of expression data. Within the code; I get error: Object ConnectivityHuman or Error: Object ConnectivityChimp not found. I have searched for any Sections of tutorial missed but unable to find these? Anyone any advice on tutorial gratefully appreciated thank you &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ConnectivityHuman = ConnectivityHuman/max(ConnectivityHuman)&#xA;    ConnectivityChimp = ConnectivityChimp/max(ConnectivityChimp) &#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6227" LastActivityDate="2019-10-25T12:35:16.660" Title="WGCNA module preservation" Tags="|r|wgcna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10648" PostTypeId="2" ParentId="10647" CreationDate="2019-10-25T12:35:16.660" Score="1" Body="&lt;p&gt;Thanks for pointing this out. Looks like we forgot to include two important lines. Something along the lines of&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ConnectivityHuman = colSums(adjHuman)-1&#xA;ConnectivityChimp = colSums(adjChimp)-1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This should do it; I'll update the tutorial.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2019-10-25T12:35:16.660" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10650" PostTypeId="2" ParentId="4341" CreationDate="2019-10-28T06:58:02.533" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://www.biorxiv.org/content/10.1101/621805v1.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;Single-Cell_Signature_Explorer paper&lt;/a&gt; compared several such methods including: AUCell, Seurat CellCycleScore, and GSVA/ssGSEA.&lt;/p&gt;&#xA;" OwnerUserId="3612" LastActivityDate="2019-10-28T06:58:02.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10652" PostTypeId="1" AcceptedAnswerId="10653" CreationDate="2019-10-28T23:37:47.007" Score="0" ViewCount="3619" Body="&lt;p&gt;I was supposed to use the code below to get the number of cells per cluster. The code works with Seurat version 2, but while using version 3 I got the error &lt;code&gt;no slot of name &quot;ident&quot; for this object of class &quot;Seurat&quot;&lt;/code&gt;. Does anyone have any idea about how to deal with this?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Calculate number of cells per cluster from object@ident&#xA;&#xA;cell.num &amp;lt;- table(shox2crepos_mtmg@ident)&#xA;&#xA;# Add cell number per cluster to cluster labels&#xA;&#xA;ClusterLabels = paste(&quot;Cluster&quot;, names(cell.num), paste0(&quot;(n = &quot;, cell.num, &quot;)&quot;))&#xA;&#xA;# Order legend labels in plot in the same order as 'ClusterLabels'&#xA;&#xA;ClusterBreaks = names(cell.num)&#xA;&#xA;# Plot tSNE with new legend labels for clusters&#xA;&#xA;TSNEPlot(object = shox2crepos_mtmg, do.return = T) +&#xA;  scale_colour_discrete(breaks = ClusterBreaks,&#xA;                        labels = ClusterLabels) +&#xA;  labs(x = &quot;t-SNE 1&quot;,&#xA;       y = &quot;t-SNE 2&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3171" LastEditorUserId="3541" LastEditDate="2019-10-30T09:02:02.640" LastActivityDate="2019-10-30T09:02:02.640" Title="How to calculate the number of cells in different cell clusters with Seurat v3?" Tags="|scrnaseq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10653" PostTypeId="2" ParentId="10652" CreationDate="2019-10-29T08:27:32.547" Score="2" Body="&lt;p&gt;The &quot;identities&quot; can be accessed with the &lt;code&gt;Idents()&lt;/code&gt; function in Seurat v3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should change &lt;code&gt;cell.num &amp;lt;- table(shox2crepos_mtmg@ident)&lt;/code&gt; to &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;cell.num &amp;lt;- table(Idents(shox2crepos_mtmg))&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-10-29T15:03:11.893" LastActivityDate="2019-10-29T15:03:11.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10654" PostTypeId="1" AcceptedAnswerId="10655" CreationDate="2019-10-29T10:40:40.527" Score="0" ViewCount="98" Body="&lt;p&gt;I have two bed files where each column has a numeric score. I want to sum these numbers where there is overlap between the two bed files. How do I do that? Also, I am only interested in the score in specific genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is my setup:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;np.random.seed(0)&#xA;import pyranges as pr&#xA;import pandas as pd&#xA;&#xA;entries = int(1e6)&#xA;cpg1 = pr.random(entries)&#xA;cpg2 = pr.random(entries)&#xA;&#xA;cpg1.CPG = np.random.randint(100, size=entries)&#xA;cpg2.CPG = np.random.randint(100, size=entries)&#xA;&#xA;regulatory_elements = pr.random(int(1e5), length=int(1e4))&#xA;regulatory_elements.Gene = np.arange(len(regulatory_elements), dtype=int)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So my data looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cpg1&#xA;# +--------------+-----------+-----------+--------------+-----------+&#xA;# | Chromosome   | Start     | End       | Strand       | CPG       |&#xA;# | (category)   | (int32)   | (int32)   | (category)   | (int64)   |&#xA;# |--------------+-----------+-----------+--------------+-----------|&#xA;# | chr1         | 8830650   | 8830750   | +            | 72        |&#xA;# | chr1         | 9564361   | 9564461   | +            | 57        |&#xA;# | chr1         | 44977425  | 44977525  | +            | 99        |&#xA;# | chr1         | 239741543 | 239741643 | +            | 54        |&#xA;# | ...          | ...       | ...       | ...          | ...       |&#xA;# | chrY         | 29437476  | 29437576  | -            | 44        |&#xA;# | chrY         | 49995298  | 49995398  | -            | 43        |&#xA;# | chrY         | 50840129  | 50840229  | -            | 10        |&#xA;# | chrY         | 38069647  | 38069747  | -            | 89        |&#xA;# +--------------+-----------+-----------+--------------+-----------+&#xA;# Stranded PyRanges object has 1,000,000 rows and 5 columns from 25 chromosomes.&#xA;# For printing, the PyRanges was sorted on Chromosome and Strand.&#xA;&#xA;&#xA;regulatory_elements&#xA;# +--------------+-----------+-----------+--------------+-----------+&#xA;# | Chromosome   | Start     | End       | Strand       | Gene      |&#xA;# | (category)   | (int32)   | (int32)   | (category)   | (int64)   |&#xA;# |--------------+-----------+-----------+--------------+-----------|&#xA;# | chr1         | 217921634 | 217931634 | +            | 0         |&#xA;# | chr1         | 166226804 | 166236804 | +            | 1         |&#xA;# | chr1         | 170688210 | 170698210 | +            | 2         |&#xA;# | chr1         | 57958563  | 57968563  | +            | 3         |&#xA;# | ...          | ...       | ...       | ...          | ...       |&#xA;# | chrY         | 35870043  | 35880043  | -            | 99996     |&#xA;# | chrY         | 35057634  | 35067634  | -            | 99997     |&#xA;# | chrY         | 52718975  | 52728975  | -            | 99998     |&#xA;# | chrY         | 19196507  | 19206507  | -            | 99999     |&#xA;# +--------------+-----------+-----------+--------------+-----------+&#xA;# Stranded PyRanges object has 100,000 rows and 5 columns from 25 chromosomes.&#xA;# For printing, the PyRanges was sorted on Chromosome and Strand.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(This is a q from privateish correspondence).&lt;/p&gt;&#xA;" OwnerUserId="2688" LastEditorUserId="2688" LastEditDate="2019-11-01T10:13:13.133" LastActivityDate="2019-11-01T10:13:13.133" Title="Summing two columns where they share interval overlap" Tags="|python|pybedtools|pyranges|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10655" PostTypeId="2" ParentId="10654" CreationDate="2019-10-29T10:47:52.327" Score="0" Body="&lt;p&gt;If you want to sum two columns based on overlaps, converting them to RLEs is a good idea.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# get the cpg scores only where there is overlap with the genes you are interested in &#xA;cr1 = cpg1.join(regulatory_elements).new_position(&quot;intersection&quot;).drop(like=&quot;_(a|b)&quot;)&#xA;cr2 = cpg2.join(regulatory_elements).new_position(&quot;intersection&quot;).drop(like=&quot;_(a|b)&quot;)&#xA;&#xA;# convert these to run length encodings, using the score from the CPG column&#xA;rle1 = cr1.to_rle(&quot;CPG&quot;, rpm=False)&#xA;rle2 = cr2.to_rle(&quot;CPG&quot;, rpm=False)&#xA;&#xA;rle_sum = rle1 + rle2&#xA;&#xA;# convert the rles back to ranges&#xA;# (I will give the to_ranges function an option dtype eventually)&#xA;range_sum = rle_sum.to_ranges().apply(lambda df: df.astype({&quot;Start&quot;: np.int32, &quot;End&quot;: np.int32}))&#xA;&#xA;# add the gene names to the scores&#xA;gr = range_sum.join(regulatory_elements) # .drop(like=&quot;_b&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now the result is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print(gr)&#xA;# +--------------+-----------+-----------+-------------+------------+-----------+-----------+--------------+-----------+&#xA;# | Chromosome   | Start     | End       | Score       | Strand     | Start_b   | End_b     | Strand_b     | Gene      |&#xA;# | (object)     | (int32)   | (int32)   | (float64)   | (object)   | (int32)   | (int32)   | (category)   | (int64)   |&#xA;# |--------------+-----------+-----------+-------------+------------+-----------+-----------+--------------+-----------|&#xA;# | chr1         | 15623     | 15723     | 77.0        | +          | 9886      | 19886     | +            | 3714      |&#xA;# | chr1         | 16034     | 16134     | 84.0        | +          | 9886      | 19886     | +            | 3714      |&#xA;# | chr1         | 36588     | 36688     | 2.0         | +          | 36479     | 46479     | +            | 2871      |&#xA;# | chr1         | 37271     | 37371     | 30.0        | +          | 36479     | 46479     | +            | 2871      |&#xA;# | ...          | ...       | ...       | ...         | ...        | ...       | ...       | ...          | ...       |&#xA;# | chrY         | 59312851  | 59312951  | 28.0        | -          | 59308107  | 59318107  | +            | 98599     |&#xA;# | chrY         | 59332843  | 59332943  | 82.0        | -          | 59332664  | 59342664  | -            | 99215     |&#xA;# | chrY         | 59337592  | 59337692  | 41.0        | -          | 59332664  | 59342664  | -            | 99215     |&#xA;# | chrY         | 59341719  | 59341819  | 15.0        | -          | 59332664  | 59342664  | -            | 99215     |&#xA;# +--------------+-----------+-----------+-------------+------------+-----------+-----------+--------------+-----------+&#xA;# Stranded PyRanges object has 668,317 rows and 9 columns from 25 chromosomes.&#xA;# For printing, the PyRanges was sorted on Chromosome and Strand.&#xA;&#xA;print(gr[gr.Score &amp;gt; 100])&#xA;# +--------------+-----------+-----------+-------------+------------+-----------+-----------+--------------+-----------+&#xA;# | Chromosome   | Start     | End       | Score       | Strand     | Start_b   | End_b     | Strand_b     | Gene      |&#xA;# | (object)     | (int32)   | (int32)   | (float64)   | (object)   | (int32)   | (int32)   | (category)   | (int64)   |&#xA;# |--------------+-----------+-----------+-------------+------------+-----------+-----------+--------------+-----------|&#xA;# | chr1         | 96874     | 96974     | 116.0       | +          | 87668     | 97668     | -            | 7391      |&#xA;# | chr1         | 96874     | 96974     | 116.0       | +          | 89794     | 99794     | -            | 6357      |&#xA;# | chr1         | 749686    | 749697    | 180.0       | +          | 744041    | 754041    | +            | 1150      |&#xA;# | chr1         | 749686    | 749697    | 180.0       | +          | 749277    | 759277    | -            | 6395      |&#xA;# | ...          | ...       | ...       | ...         | ...        | ...       | ...       | ...          | ...       |&#xA;# | chrY         | 59060667  | 59060767  | 196.0       | -          | 59052942  | 59062942  | -            | 99957     |&#xA;# | chrY         | 59060667  | 59060767  | 196.0       | -          | 59059904  | 59069904  | +            | 98401     |&#xA;# | chrY         | 59063570  | 59063576  | 105.0       | -          | 59059904  | 59069904  | +            | 98401     |&#xA;# | chrY         | 59269010  | 59269085  | 152.0       | -          | 59264704  | 59274704  | -            | 99499     |&#xA;# +--------------+-----------+-----------+-------------+------------+-----------+-----------+--------------+-----------+&#xA;# Stranded PyRanges object has 106,643 rows and 9 columns from 25 chromosomes.&#xA;# For printing, the PyRanges was sorted on Chromosome and Strand.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2688" LastEditorUserId="2688" LastEditDate="2019-10-29T10:55:45.833" LastActivityDate="2019-10-29T10:55:45.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10656" PostTypeId="2" ParentId="838" CreationDate="2019-10-29T11:03:22.650" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/biocore-ntnu/pyranges&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyranges&lt;/a&gt; answer:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# pip install pyranges&#xA;# or &#xA;# conda install -c bioconda pyranges&#xA;&#xA;import pyranges as pr&#xA;s3 = pr.read_bed(&quot;s3.bed&quot;)&#xA;s4 = pr.read_bed(&quot;s4.bed&quot;)&#xA;s3.intersect(s4, how=&quot;containment&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Answer with setup (Edit):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;from io import StringIO&#xA;import pyranges as pr&#xA;&#xA;c1 = &quot;&quot;&quot;1   10  20&#xA;1   5   20&#xA;2   20  30&#xA;2   25  30&#xA;1   10  50&#xA;2   20  60&#xA;1   14  17&quot;&quot;&quot;&#xA;&#xA;c2 = &quot;&quot;&quot;1   10  20&#xA;2   20  30&quot;&quot;&quot;&#xA;&#xA;columns = &quot;Chromosome Start End&quot;.split()&#xA;&#xA;df1 = pd.read_table(StringIO(c1), sep=&quot;\s+&quot;, header=None, names=columns)&#xA;df2 = pd.read_table(StringIO(c2), sep=&quot;\s+&quot;, header=None, names=columns)&#xA;&#xA;gr1 = pr.PyRanges(df1)&#xA;gr2 = pr.PyRanges(df2)&#xA;&#xA;print(gr1.intersect(gr2, how=&quot;containment&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Result:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;+--------------+-----------+-----------+&#xA;|   Chromosome |     Start |       End |&#xA;|   (category) |   (int32) |   (int32) |&#xA;|--------------+-----------+-----------|&#xA;|            1 |        10 |        20 |&#xA;|            1 |        14 |        17 |&#xA;|            2 |        20 |        30 |&#xA;|            2 |        25 |        30 |&#xA;+--------------+-----------+-----------+&#xA;Unstranded PyRanges object has 4 rows and 3 columns from 2 chromosomes.&#xA;For printing, the PyRanges was sorted on Chromosome.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2688" LastEditorUserId="2688" LastEditDate="2019-10-30T08:15:23.480" LastActivityDate="2019-10-30T08:15:23.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10657" PostTypeId="1" CreationDate="2019-10-29T11:12:48.003" Score="1" ViewCount="582" Body="&lt;p&gt;I downloaded an RNAseq dataset from TCGA database in 3 formats: 1) HTSeq counts; 2) FPKM; 3) FPQM-upper quartile normalized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The complete dataset contains ~60,000 genes. All of my analysis will focus on a subset of ~2500 genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What confused me is the following statement on the TCGA documentation page:&#xA;&quot;RNA-Seq expression level read counts produced by HT-Seq are normalized using two similar methods: FPKM and FPKM-UQ. Normalized values should be used only within the context of the entire gene set. Users are encouraged to normalize raw read count values if a subset of genes is investigated.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I normalize the 2500 gene counts in isolation of the rest, I am guessing the normalization factor will be more unstable between samples than using the complete gene set. It means that if one patient has higher expression of the subset than another, normalization within subset will obscure that difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the normalization factor (either total mapped reads or upper quartile) becomes more stable the larger the number of genes considered. Less variation will be seen on a global scale because of biological differences, whilst technical differences will be normalized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In short, do you normalize the gene counts de novo if you analyse only a subset of genes? Why?&lt;/p&gt;&#xA;" OwnerUserId="6421" LastEditorUserId="6421" LastEditDate="2019-10-29T16:31:58.283" LastActivityDate="2019-10-30T16:29:37.887" Title="Is re-normalization of RNAseq data recommended for analysis of gene subsets?" Tags="|rna-seq|normalization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10658" PostTypeId="2" ParentId="9365" CreationDate="2019-10-29T11:20:41.730" Score="1" Body="&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;np.random.seed(0)&#xA;import pyranges as pr&#xA;&#xA;gr = pr.random()&#xA;gr.Score = np.random.randint(100, size=len(gr))&#xA;&#xA;gr = gr.slack(25) # make data wider for this example&#xA;&#xA;print(gr)&#xA;&#xA;&#xA;t1 = gr.tile(50)&#xA;&#xA;def increase_by_25(df):&#xA;    df = df.copy()&#xA;    df.Start += 25&#xA;    df.End += 25&#xA;    return df&#xA;&#xA;t2 = t1.apply(increase_by_25)&#xA;&#xA;tiled = pr.concat([t1, t2]).sort()&#xA;&#xA;print(tiled)&#xA;# +--------------+-----------+-----------+--------------+-----------+&#xA;# | Chromosome   | Start     | End       | Strand       | Score     |&#xA;# | (category)   | (int32)   | (int32)   | (category)   | (int64)   |&#xA;# |--------------+-----------+-----------+--------------+-----------|&#xA;# | chr1         | 5205300   | 5205350   | +            | 33        |&#xA;# | chr1         | 5205325   | 5205375   | +            | 33        |&#xA;# | chr1         | 5205350   | 5205400   | +            | 33        |&#xA;# | chr1         | 5205375   | 5205425   | +            | 33        |&#xA;# | ...          | ...       | ...       | ...          | ...       |&#xA;# | chrY         | 41326450  | 41326500  | -            | 3         |&#xA;# | chrY         | 41326475  | 41326525  | -            | 3         |&#xA;# | chrY         | 41326500  | 41326550  | -            | 3         |&#xA;# | chrY         | 41326525  | 41326575  | -            | 3         |&#xA;# +--------------+-----------+-----------+--------------+-----------+&#xA;# Stranded PyRanges object has 7,964 rows and 5 columns from 24 chromosomes.&#xA;# For printing, the PyRanges was sorted on Chromosome and Strand.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is as far as I can get without example data and a clearer explanation of what you need.&lt;/p&gt;&#xA;" OwnerUserId="2688" LastActivityDate="2019-10-29T11:20:41.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10660" PostTypeId="1" AcceptedAnswerId="10662" CreationDate="2019-10-29T18:28:06.307" Score="0" ViewCount="182" Body="&lt;p&gt;What is an .sf file.  What information does it contain?&lt;/p&gt;&#xA;" OwnerUserId="2621" LastActivityDate="2019-10-29T18:45:20.450" Title="What type of information does an .sf file contain?" Tags="|file-formats|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10661" PostTypeId="1" CreationDate="2019-10-29T18:29:51.053" Score="1" ViewCount="54" Body="&lt;p&gt;I have two files that look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ head ret&#xA;chr   pos     gene_id      pval_ret&#xA;chr1 10177 ENSG00000227232 0.381708&#xA;chr1 10352 ENSG00000227232 0.959523&#xA;chr1 11008 ENSG00000227232 0.218132&#xA;chr1 11012 ENSG00000227232 0.218132&#xA;chr1 13110 ENSG00000227232 0.998262&#xA;chr1 13116 ENSG00000227232 0.438572&#xA;chr1 13118 ENSG00000227232 0.438572&#xA;chr1 13273 ENSG00000227232 0.694523&#xA;chr1 14464 ENSG00000227232 0.825113&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ head lcls&#xA;chr  pos     gene_id       pval_nominal&#xA;chr1 13550 ENSG00000227232 0.375614&#xA;chr1 14671 ENSG00000227232 0.474708&#xA;chr1 14677 ENSG00000227232 0.699887&#xA;chr1 16841 ENSG00000227232 0.127895&#xA;chr1 16856 ENSG00000227232 0.627822&#xA;chr1 17005 ENSG00000227232 0.802803&#xA;chr1 17147 ENSG00000227232 0.443987&#xA;chr1 17407 ENSG00000227232 0.397521&#xA;chr1 17408 ENSG00000227232 0.615126&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I merge these two files, &lt;code&gt;ret&lt;/code&gt; and &lt;code&gt;lcls&lt;/code&gt;, on the first 3 columns: chr, pos, gene_id?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know how to merge them on the first two columns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk 'NR==FNR{a[&lt;span class=&quot;math-container&quot;&gt;$1,$&lt;/span&gt;2]=&lt;span class=&quot;math-container&quot;&gt;$3;next} ($&lt;/span&gt;1,&lt;span class=&quot;math-container&quot;&gt;$2) in a{print $0, a[$1,$&lt;/span&gt;2]}' ret lcls &amp;gt; overlap&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But how can I do it on the first 3 columns? In the output I want these 4 columns: &lt;code&gt;chr&lt;/code&gt;, &lt;code&gt;pos,gene_id&lt;/code&gt;, &lt;code&gt;pval_ret&lt;/code&gt;, &lt;code&gt;pval_nominal&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6428" LastEditorUserId="298" LastEditDate="2019-10-30T09:15:21.473" LastActivityDate="2019-11-12T11:48:41.900" Title="How to merge two files on the first 3 columns?" Tags="|text-processing|awk|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10662" PostTypeId="2" ParentId="10660" CreationDate="2019-10-29T18:45:20.450" Score="0" Body="&lt;p&gt;You should provide more context, otherwise one can only guess what is it you are looking for! In case you are asking about &lt;code&gt;Salmon&lt;/code&gt;'s output, below is from their &lt;a href=&quot;https://salmon.readthedocs.io/en/latest/file_formats.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Salmon’s main output is its quantification file. This file is a&#xA;  plain-text, tab-separated file with a single header line (which names&#xA;  all of the columns). This file is named quant.sf and appears at the&#xA;  top-level of Salmon’s output directory. The columns appear in the&#xA;  following order:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Name   Length   EffectiveLength   TPM   NumReads&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-10-29T18:45:20.450" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10663" PostTypeId="1" CreationDate="2019-10-29T19:40:38.027" Score="1" ViewCount="22" Body="&lt;p&gt;I was wondering if there is something I am doing wrong when trying to&#xA;calculate True positive rate, TPR with your function&#xA;this is what I am doing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(qq)&#xA;    chr   pos    gene_id       pval_nominal pval_ret  META&#xA;1: chr1 54490 ENSG00000227232     0.608495 0.783778 0.7733204&#xA;2: chr1 58814 ENSG00000227232     0.295211 0.897582 0.6970567&#xA;3: chr1 60351 ENSG00000227232     0.439788 0.867959 0.7525581&#xA;4: chr1 61920 ENSG00000227232     0.319528 0.601809 0.4407018&#xA;5: chr1 63671 ENSG00000227232     0.237739 0.988039 0.8626555&#xA;6: chr1 64931 ENSG00000227232     0.276679 0.907037 0.6971364&#xA;&#xA;library(qvalue)&#xA;pvals=qq&lt;span class=&quot;math-container&quot;&gt;$META&#xA;qval_obj=qvalue(pvals) #is false discovery rate&#xA;pi1=1-qval_obj$&lt;/span&gt;pi0 #TPR&#xA;&amp;gt; pi1&#xA;[1] 0.08827036&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It is very unlikely that TPR would be this small, can you please advise?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do have 59981 META p values in this data frame.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IS here pi1 representing TPR value?&lt;/p&gt;&#xA;" OwnerUserId="6428" LastEditorUserId="6428" LastEditDate="2019-10-29T19:48:16.613" LastActivityDate="2019-10-29T19:48:16.613" Title="Calculating True Positive Rate with qvalue function" Tags="|r|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10664" PostTypeId="1" CreationDate="2019-10-30T02:28:39.403" Score="2" ViewCount="87" Body="&lt;p&gt;Are there databases that relate genetic disorders with neurological disorders? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am thinking of something like COSMICS for cancer? AFAIK cancer usually develops due to a combination of 5-10 mutations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also if you can comment how genomics/bioinformatics in neurological contexts is compared to the how bioinformaticians study cancer? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related: &lt;a href=&quot;https://www.nimhgenetics.org/resources/psychencode&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nimhgenetics.org/resources/psychencode&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2019-11-19T13:13:50.923" LastActivityDate="2019-11-19T13:13:50.923" Title="Are there genomics databases for neurological and psychiatric disorders?" Tags="|database|genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10665" PostTypeId="2" ParentId="10664" CreationDate="2019-10-30T03:44:11.533" Score="3" Body="&lt;p&gt;Try with &lt;a href=&quot;http://www.psygenet.org/web/PsyGeNET/menu;jsessionid=1s5idap12defr159fq36eer9v3&quot; rel=&quot;nofollow noreferrer&quot;&gt;PsyGeNet&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;PsyGeNET (Psychiatric disorders Gene association NETwork) is a resource for the exploratory analysis of psychiatric diseases and their associated genes&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is based on literature mining and curated by experts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the publication if you want more details:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://academic.oup.com/bioinformatics/article/31/18/3075/240596&quot; rel=&quot;nofollow noreferrer&quot;&gt;PsyGeNET: a knowledge platform on psychiatric disorders and their genes&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and they even have a Bioconductor package for programatic queries:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://academic.oup.com/bioinformatics/article/33/24/4004/4083576&quot; rel=&quot;nofollow noreferrer&quot;&gt;psygenet2r: a R/Bioconductor package for the analysis of psychiatric disease genes&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="678" LastActivityDate="2019-10-30T03:44:11.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10666" PostTypeId="2" ParentId="8533" CreationDate="2019-10-30T05:41:52.877" Score="1" Body="&lt;p&gt;just stumbled on this.  I'm the developer of igv.js.  Please feel free to open issues and questions on our git forum,  I don't often have time to browse other forums.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Browsers will not allow loading of local files by file path in javascript, if they did it would be a gigantic security hole.  This restriction includes &quot;file://&quot; urls.  Local files can be loaded by user action (clicking a button),  that is supported in igv.js but there is no button for it as igv.js is designed as a component to embed in pages, not an application by itself.  For an example see igv.org/app (&lt;a href=&quot;https://github.com/igvteam/igv-webapp&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/igvteam/igv-webapp&lt;/a&gt;),  a minimal application wrapping igv.js.  You'll find &quot;local file&quot; options for genomes and tracks there. &lt;/p&gt;&#xA;" OwnerUserId="1681" LastActivityDate="2019-10-30T05:41:52.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10667" PostTypeId="2" ParentId="10661" CreationDate="2019-10-30T09:21:52.427" Score="2" Body="&lt;p&gt;You can use the same approach as for two fields, just add the third:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk 'NR==FNR{a[&lt;span class=&quot;math-container&quot;&gt;$1,$&lt;/span&gt;2,&lt;span class=&quot;math-container&quot;&gt;$3]=$&lt;/span&gt;4;next} (&lt;span class=&quot;math-container&quot;&gt;$1,$&lt;/span&gt;2,&lt;span class=&quot;math-container&quot;&gt;$3) in a{print $0, a[$1,$2,$&lt;/span&gt;3]}' ret lcls&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-10-30T09:21:52.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10668" PostTypeId="2" ParentId="10657" CreationDate="2019-10-30T16:29:37.887" Score="2" Body="&lt;p&gt;FPKM, RPKM are ways of normalising for both sequencing depth and gene length. You first calculate a scaling factor (&lt;code&gt;SF = #read/1Mil&lt;/code&gt;),you then divide the genes counts for the &lt;code&gt;SF&lt;/code&gt;(&lt;code&gt;RPM = # counts/SF&lt;/code&gt;), and finally you divide by the length of the gene (&lt;code&gt;RPM/length(gene)&lt;/code&gt;). Each sample has a different &lt;code&gt;SF&lt;/code&gt; meaning that you should not compare values from different samples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To fix this, you have to normalise across samples, by &lt;code&gt;upper quartile normalise&lt;/code&gt; using a subset of genes. In this case, the &lt;code&gt;SF&lt;/code&gt; is calculated based on the upper quartile (70%) of this shared and immutable subset, and it is therefore the same for all the samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you subset your genes, and you are using the normalised values for downstream analyses, then you need to re-scale your values. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I personally try to avoid &lt;code&gt;FPKM&lt;/code&gt;,&lt;code&gt;RPKM&lt;/code&gt;,&lt;code&gt;TPM&lt;/code&gt; (if I can) and always start from raw counts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest to have a look at the documentation of packages like &lt;code&gt;DESeq2&lt;/code&gt; and &lt;code&gt;edgeR&lt;/code&gt; for detailed and better explanations. &lt;/p&gt;&#xA;" OwnerUserId="3631" LastActivityDate="2019-10-30T16:29:37.887" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10669" PostTypeId="1" CreationDate="2019-10-30T22:13:10.600" Score="4" ViewCount="295" Body="&lt;p&gt;Our BAM files are created according to a &amp;quot;lossless&amp;quot; alignment procedure [&lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/article.php?id=6483&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;] from the Broad Institute &lt;code&gt;GATK&lt;/code&gt; documenation and involves re-adding the unaligned/unmapped reads into an aligned BAM,  using Picard's &lt;code&gt;MergeBamAlignment&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The BAM files are produced in the end contain both the mapped and the unmapped reads. These files are then sorted with &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360036510732-SortSam-Picard-&quot; rel=&quot;nofollow noreferrer&quot;&gt;SortSam&lt;/a&gt; [&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360036510732-SortSam-Picard-&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;]- so that the sort order in the header becomes:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;@HD VN:1.6 SO:coordinate&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;How does &lt;code&gt;MarkDuplicates&lt;/code&gt; handle the unmapped reads of a BAM file containing both unmapped and mapped?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; &lt;code&gt;MarkDuplicates&lt;/code&gt; seems to normally take the BAM's ordering into account, namely, it accepts arguments such as &lt;code&gt;--ASSUME_SORT_ORDER X&lt;/code&gt;.  However it's not specified whether reads without a position are ignored, or have to be compared with all other possible reads.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;em&gt;Disclaimer&lt;/em&gt;: I initially posted this question on the GATK forum [&lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/24557/&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;], but I'm reaching out to hopefully a broader audience.&lt;/p&gt;&#xA;&lt;p&gt;Citations:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1 &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/article.php?id=6483&quot; rel=&quot;nofollow noreferrer&quot;&gt;How to Map and clean up short read sequence data efficiently&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;2 &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360036510732-SortSam-Picard-&quot; rel=&quot;nofollow noreferrer&quot;&gt;SortSam (Picard)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;3 &lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/24557/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Original posting on the GATK forum&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3235" LastEditorUserId="3848" LastEditDate="2020-07-20T20:05:20.247" LastActivityDate="2020-07-20T22:58:27.170" Title="How does picard's MarkDuplicate handle unmapped reads?" Tags="|picard|markduplicate|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10670" PostTypeId="1" CreationDate="2019-10-31T00:51:59.467" Score="1" ViewCount="53" Body="&lt;p&gt;MSigDB recently added &lt;a href=&quot;http://www.gsea-msigdb.org/gsea/msigdb/supplementary_genesets.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;Supplementary Collections&lt;/a&gt;. The formatting is similar to the original MSigDB gene sets, but there is an additional &lt;code&gt;metadata.txt&lt;/code&gt; file (&lt;a href=&quot;http://software.broadinstitute.org/gsea/msigdb/supplemental/scsig.v1.0.metadata.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like (condensed for easier reading):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;STANDARD_NAME   Fan_Embryonic_CTX_Big_Groups_Cajal_Retzius&#xA;ORGANISM    Homo sapiens&#xA;ORGAN_SYSTEM    Central Nervous System&#xA;PMID    29867213&#xA;PUBLICATION_TITLE   Spatial transcriptomic survey of human embryonic cerebral cortex by single-cell RNA-seq analysis&#xA;AUTHORS Fan X,Dong J,Zhong S,Wei Y,Wu Q,Yan L,Yong J,Sun L,Wang X,Zhao Y,Wang W,Yan J,Wang X,Qiao J,Tang F,&#xA;GEOID   GSE103723&#xA;...&#xA;STANDARD_NAME   Fan_Embryonic_CTX_Big_Groups_Brain_Endothelial&#xA;ORGANISM    Homo sapiens&#xA;ORGAN_SYSTEM    Central Nervous System&#xA;PMID    29867213&#xA;PUBLICATION_TITLE   Spatial transcriptomic survey of human embryonic cerebral cortex by single-cell RNA-seq analysis&#xA;AUTHORS Fan X,Dong J,Zhong S,Wei Y,Wu Q,Yan L,Yong J,Sun L,Wang X,Zhao Y,Wang W,Yan J,Wang X,Qiao J,Tang F,&#xA;GEOID   GSE103723&#xA;...&#xA;STANDARD_NAME   Fan_Embryonic_CTX_Big_Groups_Excitatory_Neuron&#xA;ORGANISM    Homo sapiens&#xA;ORGAN_SYSTEM    Central Nervous System&#xA;PMID    29867213&#xA;PUBLICATION_TITLE   Spatial transcriptomic survey of human embryonic cerebral cortex by single-cell RNA-seq analysis&#xA;AUTHORS Fan X,Dong J,Zhong S,Wei Y,Wu Q,Yan L,Yong J,Sun L,Wang X,Zhao Y,Wang W,Yan J,Wang X,Qiao J,Tang F,&#xA;GEOID   GSE103723&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I could not find any description of the formatting. Is this a standard format? Is there a parser for it (preferably in R)? It looks like I can just transpose each record to get it in a table format, but I don't know if I can trust all records to be consistent.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2019-11-04T12:17:23.300" Title="Parsing MSigDB Supplementary Collections" Tags="|r|file-formats|format-conversion|pathway|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10671" PostTypeId="1" AcceptedAnswerId="10677" CreationDate="2019-10-31T01:00:02.787" Score="2" ViewCount="220" Body="&lt;p&gt;I have no experience in Bioinformatics and I need to understand what the annotations &lt;a href=&quot;https://figshare.com/articles/Supp_Tables1-4/8044556/1&quot; rel=&quot;nofollow noreferrer&quot;&gt;given here&lt;/a&gt; mean (I am including the first few lines, please see the link for more):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;contig_id                         feature_id              type       location                                 start   stop    strand&#xA;gi|1045318032|gb|MAZB01000333.1|  fig|252393.25.peg.1     peg        gi|1045318032|gb|MAZB01000333.1|_511_2   511     2       -&#xA;gi|1045318033|gb|MAZB01000332.1|  fig|252393.25.repeat.1  repeat     gi|1045318033|gb|MAZB01000332.1|_1_128   1       128     +&#xA;gi|1045318033|gb|MAZB01000332.1|  fig|252393.25.repeat.2  repeat     gi|1045318033|gb|MAZB01000332.1|_318_532 318     532     +&#xA;gi|1045318033|gb|MAZB01000332.1|  fig|252393.25.peg.2     peg        gi|1045318033|gb|MAZB01000332.1|_530_321 530     321     -&#xA;gi|1045318034|gb|MAZB01000331.1|  fig|252393.25.repeat.3  repeat     gi|1045318034|gb|MAZB01000331.1|_1_534   1       534     +&#xA;gi|1045318034|gb|MAZB01000331.1|  fig|252393.25.peg.3     peg        gi|1045318034|gb|MAZB01000331.1|_3_410   3       410     +&#xA;gi|1045318035|gb|MAZB01000330.1|  fig|252393.25.repeat.4  repeat     gi|1045318035|gb|MAZB01000330.1|_1_128   1       128     +&#xA;gi|1045318035|gb|MAZB01000330.1|  fig|252393.25.peg.4     peg        gi|1045318035|gb|MAZB01000330.1|_3_539   3       539     +&#xA;gi|1045318035|gb|MAZB01000330.1|  fig|252393.25.repeat.5  repeat     gi|1045318035|gb|MAZB01000330.1|_413_539 413     539     +&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Could someone help me to understand what these files and all annotations mean?&lt;/p&gt;&#xA;" OwnerUserId="2881" LastEditorUserId="1157" LastEditDate="2019-11-01T16:51:34.677" LastActivityDate="2019-11-01T16:51:34.677" Title="What do these files / annotations mean?" Tags="|file-formats|edger|identifiers|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10672" PostTypeId="2" ParentId="10671" CreationDate="2019-10-31T07:46:05.897" Score="1" Body="&lt;p&gt;gi looks like a Genbank Identifier.  You can search for the 10 digit number following gi| at the Genbank website.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-10-31T07:46:05.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10673" PostTypeId="1" CreationDate="2019-10-31T09:16:30.867" Score="0" ViewCount="148" Body="&lt;p&gt;I was running &lt;a href=&quot;https://cibersort.stanford.edu&quot; rel=&quot;nofollow noreferrer&quot;&gt;CIBERSORT&lt;/a&gt; web and saw this error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Exception in thread &quot;Thread-0&quot; java.lang.ArrayIndexOutOfBoundsException: 0&#xA;at launch.QN.doQN(QN.java:56)&#xA;at launch.Cibersort.Run_CIBERSORT(Cibersort.java:611)&#xA;at launch.Cibersort.run_all(Cibersort.java:179)&#xA;at launch.Cibersort$runCIBERSORT.run(Cibersort.java:1585)&#xA;at java.lang.Thread.run(Thread.java:745)&#xA;311&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does anyone know why?&lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2024-01-28T07:04:55.623" Title="CIBERSORT runtime error" Tags="|webservice|cibersort|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10674" PostTypeId="2" ParentId="10673" CreationDate="2019-10-31T10:14:29.747" Score="0" Body="&lt;p&gt;OK I found the bug. CIBERSORT seems to require tsv files instead of csv files. Plus it doesn't seem to like empty columns&lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2019-10-31T10:14:29.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10675" PostTypeId="1" CreationDate="2019-10-31T10:15:47.427" Score="0" ViewCount="358" Body="&lt;p&gt;I was running &lt;a href=&quot;https://cibersort.stanford.edu&quot; rel=&quot;nofollow noreferrer&quot;&gt;CIBERSORT&lt;/a&gt; but caught this error right after it started permutation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;org.rosuda.REngine.Rserve.RserveException: eval failed, request status: error code: 127&#xA;88&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1852" LastEditorUserId="1852" LastEditDate="2019-10-31T10:46:17.027" LastActivityDate="2022-03-19T18:00:00.723" Title="CIBERSORT runtime error eval failed" Tags="|rna-seq|cibersort|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10676" PostTypeId="1" AcceptedAnswerId="10686" CreationDate="2019-10-31T11:20:11.317" Score="0" ViewCount="405" Body="&lt;p&gt;I am trying to plot scatter plot of pie charts using ggplot2 and ggforce.&#xA;Following is my code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(data.table)&#xA;library(ggplot2)&#xA;library(ggforce)&#xA;library(dplyr)&#xA;&#xA;df &amp;lt;- fread(&quot;data_frame_pie_plot.csv&quot;, header = TRUE, &#xA;        data.table = FALSE, stringsAsFactors = FALSE)&#xA;&#xA;req_data &amp;lt;- apply(df, 1, function(x){&#xA;  geom_arc_bar(aes(x0 = x0, y0 = y0, r0 = 0, r = 0.45, &#xA;               amount = amount, fill = discrete_fill), stat = &quot;pie&quot;)})&#xA;&#xA;ggplot(df) + req_data + theme_no_axes()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;plot_obtained&lt;a href=&quot;https://i.stack.imgur.com/Y2m1Z.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Y2m1Z.jpg&quot; alt=&quot;plot_obtained&quot;&gt;&lt;/a&gt;&#xA;Link_to_data &lt;a href=&quot;https://drive.google.com/file/d/1cB8PJVXTVptuNxIrkcxaxsQ-mSWg9BqO/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Data_frame&lt;/a&gt;.&lt;br&gt;&#xA;Can I fill the colours manually (discrete(based on discrete_fill column) &#xA;and continous (based on conti_fill column))?&#xA;I have tried the following codes.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##================================================================================##&#xA;# Code 1&#xA;my_colors &amp;lt;- c(&quot;red&quot;,&quot;white&quot;,&quot;blue&quot;)&#xA;ggplot(df, fill = sample_mut_colour) + req_data +&#xA;  theme_no_axes() + scale_color_manual(values=my_colors)&#xA;&#xA;##================================================================================##&#xA;# Code 2&#xA;my_colors_1 &amp;lt;- c(&quot;red&quot;, &quot;white&quot;)&#xA;my_colors_2 &amp;lt;- c(&quot;blue&quot;, &quot;white&quot;)&#xA;&#xA;req_data &amp;lt;- apply(df, 1, function(x){&#xA;  geom_arc_bar(aes(x0 = x0, y0 = y0, r0 = 0, r = 0.45, &#xA;               amount = amount, &#xA;               fill = discrete_fill,&#xA;               color = case_when(discrete_fill == 1 ~ my_colors_1,&#xA;                                 discrete_fill == 2 ~ my_colors_2)), stat = &quot;pie&quot;)})&#xA;&#xA;ggplot(df) + req_data + theme_no_axes() &#xA;&#xA;##================================================================================##                  &#xA;# Code 3&#xA;req_data &amp;lt;- apply(df, 1, function(x){&#xA;  geom_arc_bar(aes(x0 = x0, y0 = y0, r0 = 0, r = 0.45, &#xA;               amount = amount, &#xA;               fill = case_when(discrete_fill == 1 ~ my_colors_1,&#xA;                                 discrete_fill == 2 ~ my_colors_2)), stat = &quot;pie&quot;)})&#xA;&#xA;ggplot(df) + req_data + theme_no_axes() &#xA;&#xA;##================================================================================##             &#xA;# Code 4&#xA;req_data &amp;lt;- apply(df, 1, function(x){&#xA;  geom_arc_bar(aes(x0 = x0, y0 = y0, r0 = 0, r = 0.45, &#xA;               amount = amount, &#xA;               fill = discrete_fill),&#xA;           palette = case_when(discrete_fill == 1 ~ &quot;red&quot;,&#xA;                               discrete_fill == 2 ~ &quot;black&quot;,&#xA;                               TRUE ~ &quot;white&quot;), stat = &quot;pie&quot;)})&#xA;&#xA;ggplot(df) + req_data + theme_no_axes()  &#xA;&#xA;##=================================================================================##&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But they do not seem to work. Any idea what is wrong in the code?&lt;/p&gt;&#xA;" OwnerUserId="2247" LastActivityDate="2019-10-31T18:11:25.763" Title="Customizing colors in geom_arc_bar (ggforce)" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10677" PostTypeId="2" ParentId="10671" CreationDate="2019-10-31T11:50:53.467" Score="5" Body="&lt;p&gt;You are looking at the supplementary data of a paper. That seems to have given you a list of features, and some information about those features. Specifically, you seem to have a list of two types of element:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&amp;quot;peg&amp;quot;. Based on the information there, I assume &amp;quot;peg&amp;quot; stands for &amp;quot;protein encoding gene&amp;quot;. Note that all peg lines have a peptide sequence (not seen in your question but present in the original link) and their &amp;quot;function&amp;quot; column is always some sort of protein function (e.g. &amp;quot;hypothetical protein&amp;quot; or &amp;quot;Transposase&amp;quot;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&amp;quot;repeat&amp;quot;. These are presumably &lt;a href=&quot;https://www.sciencedirect.com/topics/neuroscience/repeated-sequence-dna&quot; rel=&quot;nofollow noreferrer&quot;&gt;repetitive elements&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;For each line, you are given &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Sitemap/sequenceIDs.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;an identifier&lt;/a&gt; for the sequence in which this element can be found. For example, your first line is on &lt;code&gt;gi|1045318032|gb|MAZB01000333.1|&lt;/code&gt;. That is a GI (&lt;code&gt;gi|1045318032&lt;/code&gt;) and a GenBank identifier (&lt;code&gt;gb|MAZB01000333.1&lt;/code&gt;). If you go to the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI website&lt;/a&gt;, and search for &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/search/all/?term=1045318032&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;1045318032&lt;/code&gt;&lt;/a&gt;, you will find that it is the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/1045318032&quot; rel=&quot;nofollow noreferrer&quot;&gt;Candidatus Erwinia dacicola isolate Erw_SC contig_333, whole genome shotgun sequence&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So, your first line is telling you that the authors have identified a protein encoding gene on sequence &lt;code&gt;gi|1045318032&lt;/code&gt; and the gene starts at position 511 of that sequence and ends at position 2. This might sound odd, but that's because the gene is on the negative (&lt;code&gt;-&lt;/code&gt;) strand and positions are calculated with respect to the positive strand. So, if you were to count on the negative, the gene would start at 2 and end at 511.&lt;/p&gt;&#xA;&lt;p&gt;The same goes for all the other lines you have there. Just look up the GI number and you can know what sequences they are referring to.&lt;/p&gt;&#xA;&lt;p&gt;Finally, the second column, the &amp;quot;feature_id&amp;quot; looks like something specific to this figshare service that is hosting the data. I dont' know what it is, but I would guess it's an identifier for this particular feature. I don't think it has any relevance outside &lt;a href=&quot;http://figshare.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://figshare.com&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2019-10-31T15:19:19.193" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10678" PostTypeId="1" CreationDate="2019-10-31T12:38:43.737" Score="3" ViewCount="10109" Body="&lt;p&gt;I am learning about NGS analysis and im currently learning about QCing and removing adaptors. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working on SRR1972920_1.fastq file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When running fastqc tool on that file, adapter contamination is present in the form of Nextera Transposase adapters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Searching for that adapter sequence via google the sequence for at the 3' end to be removed is TCGTCGGCAGCGTCAGATGTGTATAAGAGACAG (obtained via illumina adapter sequence-Nature).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Running a very basic cutadapt command line to remove the adapter&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cutadapt -a TCGTCGGCAGCGTCAGATGTGTATAAGAGACAG -o &amp;lt;any output name&amp;gt; &amp;lt;my input file&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After running the fastqc tool again on the new output file from cutadapt, it seems nothing changed and the contamination still present.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, when running the same command above but changing the sequence to CTGTCTCTTATA, which is the sequence of transposase adaptors according to fastqc tool adapters_list.txt file all is good after running fastqc tool on the file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cutadapt -a CTGTCTCTTATA  -o &amp;lt;any output name&amp;gt; &amp;lt;my input file&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, why this is happening? which Nextera transposase sequence is the correct? or am I doing something wrong?&lt;/p&gt;&#xA;" OwnerUserId="6436" LastActivityDate="2021-04-10T21:19:47.317" Title="removing nextera transposase adapters, cutadapt" Tags="|adapter|cutadapt|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10679" PostTypeId="1" AcceptedAnswerId="10688" CreationDate="2019-10-31T12:45:45.343" Score="2" ViewCount="600" Body="&lt;p&gt;I am using &lt;a href=&quot;https://github.com/biocore-ntnu/pyranges&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyranges&lt;/a&gt; and I have two datasets that I want to compare in several genome-wide intervals based on a bed file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For every interval in the bed file, I want to get all overlapping positions from the two datasets and sum two columns. That gives me 4 numbers for a fisher exact test, but that's not the most important part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My current &lt;a href=&quot;https://github.com/wdecoster/methplotlib/blob/master/scripts/allele_specific_modification.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt; looks like below, and just iterates over the itertuples() of the bed file as pandas dataframe, and uses that to slice the pyranges object:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;import pandas as pd&#xA;import pyranges as pr&#xA;from pyranges import PyRanges&#xA;from scipy.stats import fisher_exact&#xA;import numpy as np&#xA;&#xA;&#xA;def main():&#xA;    colnames = [&quot;Chromosome&quot;, &quot;Start&quot;, &quot;End&quot;, &quot;num_motifs&quot;, &quot;calls&quot;, &quot;methylated&quot;, &quot;freq&quot;, &quot;seq&quot;]&#xA;    h1 = PyRanges(pd.read_csv(&quot;dataset1.tsv.gz&quot;, sep=&quot;\t&quot;, names=colnames, header=0))&#xA;    h2 = PyRanges(pd.read_csv(&quot;dataset2.tsv.gz&quot;, sep=&quot;\t&quot;, names=colnames, header=0))&#xA;    for chr, begin, end in pr.read_bed(&quot;chr21.bed.gz&quot;).merge().as_df().itertuples(index=False, name=None):&#xA;        odr, pv = fisher_exact([meth_counts(h1[chr, begin:end]),&#xA;                                meth_counts(h2[chr, begin:end])])&#xA;&#xA;&#xA;def meth_counts(interval):&#xA;    if not interval.empty:&#xA;        all, methylated = interval.as_df()[['calls', &quot;methylated&quot;]].sum()&#xA;        return [all - methylated, methylated]&#xA;    else:&#xA;        return [0, 0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This code works well, but I'm not sure if I use the &quot;full power&quot; of what pyranges could offer me for iterating and subsetting. I uploaded some example data on &lt;a href=&quot;https://drive.google.com/drive/folders/1N5SIcmeFFGaQgrQSuinOyrmCaLw1f-cX?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;google drive&lt;/a&gt; (250GB).&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2019-11-01T14:13:46.460" Title="Efficiently iterate over bed and subset pyranges objects" Tags="|python|bed|pyranges|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10680" PostTypeId="2" ParentId="10675" CreationDate="2019-10-31T12:48:32.307" Score="1" Body="&lt;p&gt;Ironically, I found that this problem was caused by the fact that the tables are too large!&#xA;They really should have designed the correct error messages!&lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2019-10-31T12:48:32.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10681" PostTypeId="2" ParentId="10678" CreationDate="2019-10-31T12:55:39.937" Score="3" Body="&lt;p&gt;I cannot quite help(*) on the problem you are having with &lt;code&gt;cutadapt&lt;/code&gt; but can point you out to &lt;code&gt;Trimmomatic&lt;/code&gt;, for which the developers have been granted permission to distribute  Illumina adapter sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you download &lt;code&gt;Trimmomatic&lt;/code&gt;, you will see a bunch of Illumina adapter sequence files in the &lt;code&gt;/trimmomatic-0.39/adapters/&lt;/code&gt; folder. Actually the adapter sequence you have found via googling is listed in the file &lt;code&gt;NexteraPE-PE.fa&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;* Regarding the undesired output with &lt;code&gt;cutadapt&lt;/code&gt;, you were probably removing one of the multiple Nextra adapter options but not the exact one used in your library preparation step. You won't have to worry about finding the exact adapter sequence used in the library prep step when using &lt;code&gt;Trimmomatic&lt;/code&gt; as it scans for the presence of all possible adapter sequences.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-10-31T12:55:39.937" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="10683" PostTypeId="2" ParentId="10670" CreationDate="2019-10-31T14:17:31.537" Score="2" Body="&lt;p&gt;If you want to separate each set, you can do something like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dat &amp;lt;- readLines(&quot;~/Desktop/scsig.v1.0.metadata.txt&quot;)&#xA;dat &amp;lt;- gsub(&quot;\t&quot;,&quot; &quot;,dat)&#xA;&#xA;# get the indeces of each type&#xA;standardName &amp;lt;- grep(&quot;STANDARD_NAME&quot;,dat)&#xA;organism &amp;lt;- grep(&quot;ORGANISM&quot;,dat)&#xA;organSystem &amp;lt;- grep(&quot;ORGAN_SYSTEM&quot;,dat)&#xA;pmid &amp;lt;- grep(&quot;PMID&quot;,dat)&#xA;&#xA;# return the values &#xA;standardName &amp;lt;- gsub(pattern = &quot;STANDARD_NAME&quot;,replacement = &quot;&quot;,dat[standardName])&#xA;organism &amp;lt;- gsub(pattern = &quot;ORGANISM&quot;,replacement = &quot;&quot;,dat[organism])&#xA;organSystem &amp;lt;- gsub(pattern = &quot;ORGAN_SYSTEM&quot;,replacement = &quot;&quot;,dat[organSystem])&#xA;pmid &amp;lt;- gsub(pattern = &quot;PMID&quot;,replacement = &quot;&quot;,dat[pmid])&#xA;&#xA;df &amp;lt;- data.frame(standardName,organism,organSystem,pmid,stringsAsFactors = FALSE)&#xA;&#xA; head(df)&#xA;                                     standardName      organism             organSystem      pmid&#xA;1      Fan_Embryonic_CTX_Big_Groups_Cajal_Retzius  Homo sapiens  Central Nervous System  29867213&#xA;2  Fan_Embryonic_CTX_Big_Groups_Brain_Endothelial  Homo sapiens  Central Nervous System  29867213&#xA;3  Fan_Embryonic_CTX_Big_Groups_Excitatory_Neuron  Homo sapiens  Central Nervous System  29867213&#xA;4              Fan_Embryonic_CTX_Big_Groups_Glial  Homo sapiens  Central Nervous System  29867213&#xA;5       Fan_Embryonic_CTX_Big_Groups_Brain_Immune  Homo sapiens  Central Nervous System  29867213&#xA;6          Fan_Embryonic_CTX_Big_Groups_Microglia  Homo sapiens  Central Nervous System  29867213&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This method is very verbose, but it's a start and it can be easily improved with a loop. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; #---- EDIT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Not knowing how the format works, there may be two issues:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) gene sets with missing values only&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;STANDARD_NAME Aizarani_Liver_C33_Stellate_cells_2&#xA;ORGANISM &#xA;ORGAN_SYSTEM &#xA;PMID 31292543&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;2) gene sets with missing value and type&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;STANDARD_NAME Aizarani_Liver_C33_Stellate_cells_2&#xA;PMID 31292543&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Case 1&lt;/code&gt; is not an issue, as the code above will simply add an empty cell and go to the next line.&#xA;For &lt;code&gt;Case 2&lt;/code&gt; we can use &lt;code&gt;unique&lt;/code&gt; to get a list of all define types, create an empty &lt;code&gt;data.frame&lt;/code&gt; with those &lt;code&gt;types&lt;/code&gt; and have a loop (or &lt;code&gt;sapply&lt;/code&gt;) to add the values in the right columns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can check the nature of &lt;code&gt;metadata.txt&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; types &amp;lt;- gsub(&quot;^(.*)\t.*$&quot;,&quot;\\1&quot;,dat)&#xA; sapply(unique(types), function(t) length(grep(t,types)))&#xA;&#xA;     STANDARD_NAME                ORGANISM            ORGAN_SYSTEM                    PMID       PUBLICATION_TITLE &#xA;                257                     257                     257                     257                     257 &#xA;            AUTHORS                   GEOID            EXACT_SOURCE    EXTERNAL_DETAILS_URL                    CHIP &#xA;                257                     257                     257                     257                     257 &#xA;      CATEGORY_CODE             CONTRIBUTOR         CONTRIBUTOR_ORG       DESCRIPTION_BRIEF RAW_PUBLICATION_MEMBERS &#xA;                257                     514                     257                     257                     257 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Your are lucky and you don't have to worry about shifting because of missing values. &#xA;There is a fixable issue with &lt;code&gt;CONTRIBUTORS&lt;/code&gt;, but it is related to the &lt;code&gt;regex&lt;/code&gt;, not the file format. &lt;/p&gt;&#xA;" OwnerUserId="3631" LastEditorUserId="3631" LastEditDate="2019-11-04T12:17:23.300" LastActivityDate="2019-11-04T12:17:23.300" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10684" PostTypeId="2" ParentId="10678" CreationDate="2019-10-31T15:05:17.717" Score="5" Body="&lt;p&gt;The adapter sequence you have googled is the sequence on the adapter primer. This works when you want to remove primer-dimers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With transposase adapters or ATAC seq, you have very short fragments and you sequence into the adapter. So what you is sequenced is actually the reverse complement. See below image from &lt;a href=&quot;http://www.usadellab.org/cms/uploads/supplementary/Trimmomatic/TrimmomaticManual_V0.32.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;trimmomatic manual&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TzdOb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TzdOb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So if you look at what worked, and the sequence you googled, they are reverse complement of one another.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also see this in the NexteraPE-PE.fa, Trans1 and Trans1_rc @Haci recommended:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;more NexteraPE-PE.fa &#xA;&amp;gt;PrefixNX/1&#xA;AGATGTGTATAAGAGACAG&#xA;&amp;gt;PrefixNX/2&#xA;AGATGTGTATAAGAGACAG&#xA;&amp;gt;Trans1&#xA;TCGTCGGCAGCGTCAGATGTGTATAAGAGACAG&#xA;&amp;gt;Trans1_rc&#xA;CTGTCTCTTATACACATCTGACGCTGCCGACGA&#xA;&amp;gt;Trans2&#xA;GTCTCGTGGGCTCGGAGATGTGTATAAGAGACAG&#xA;&amp;gt;Trans2_rc&#xA;CTGTCTCTTATACACATCTCCGAGCCCACGAGAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To answer your question, if most of your fragments are short, and there's not a lot of primer dimers, then yes you are doing the right thing.&lt;/p&gt;&#xA;" OwnerUserId="5077" LastEditorUserId="73" LastEditDate="2019-10-31T21:32:47.850" LastActivityDate="2019-10-31T21:32:47.850" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10685" PostTypeId="2" ParentId="10679" CreationDate="2019-10-31T15:48:44.357" Score="1" Body="&lt;p&gt;Unless you need to use Python and can't use &lt;code&gt;subprocess&lt;/code&gt;, here's a quick CLI one-liner which sums signal from sorted BED5 files, over the genomic space where they overlap:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --echo --sum --delim '\t' &amp;lt;(bedops --merge A.bed B.bed ... N.bed) &amp;lt;(bedops --everything A.bed B.bed ... N.bed) &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Requires &lt;code&gt;bash&lt;/code&gt; for process substitutions.)&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2019-10-31T15:48:44.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10686" PostTypeId="2" ParentId="10676" CreationDate="2019-10-31T18:11:25.763" Score="2" Body="&lt;p&gt;For a discrete scale your Code 1 example was nearly there.  Just use &lt;code&gt;scale_fill_manual&lt;/code&gt; instead of &lt;code&gt;scale_color_manual&lt;/code&gt;.  For more control over how the discrete values map to color values you can name the vector, too.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;my_colors &amp;lt;- c(&quot;red&quot;,&quot;white&quot;,&quot;blue&quot;)&#xA;ggplot(df) + req_data +&#xA;    theme_no_axes() + scale_fill_manual(values=my_colors)&#xA;# or, if you want consistent output independent of the order of&#xA;# df$discrete_fill:&#xA;my_colors &amp;lt;- c(&quot;1&quot; = &quot;red&quot;, &quot;2&quot; = &quot;white&quot;, &quot;dummy&quot; = &quot;blue&quot;)&#xA;ggplot(df) + req_data +&#xA;    theme_no_axes() + scale_fill_manual(values=my_colors)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can also just use a single call to geom_arc_bar for the whole data frame (rather than a list of call results from each row) and you'll get the same effect.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;ggplot(df) +&#xA;  geom_arc_bar(aes(x0 = x0, y0 = y0, r0 = 0, r = 0.45, &#xA;               amount = amount, fill = discrete_fill), stat = &quot;pie&quot;) +&#xA;    theme_no_axes() + scale_fill_manual(values=my_colors)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2019-10-31T18:11:25.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10687" PostTypeId="2" ParentId="3526" CreationDate="2019-11-01T02:45:43.000" Score="1" Body="&lt;ul&gt;&#xA;&lt;li&gt;@user172818 code can't count N at fa begining&lt;/li&gt;&#xA;&lt;li&gt;@terdon code get wrong bed position&lt;/li&gt;&#xA;&lt;li&gt;i change @terdon code like below; and it works for me&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{&#xA;    if (substr($1,1,1)==&quot;&amp;gt;&quot;)&#xA;        if (NR&amp;gt;1)&#xA;            printf &quot;\n%s,&quot;, substr($0,2,length($0)-1)&#xA;        else&#xA;            printf &quot;%s,&quot;, substr($0,2,length($0)-1)&#xA;    else&#xA;        printf &quot;%s&quot;, $0&#xA;}&#xA;END{printf &quot;\n&quot;}' test.fa | \&#xA;perl -F&quot;,&quot; -ane 'while($F[1] =~ /N+/ig){&#xA;                        for(0..$#-){                    &#xA;                            print &quot;$F[0]\t$-[$_]\t$+[$_]\n&quot;;&#xA;                        }&#xA;                  }'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3730" LastEditorUserId="3730" LastEditDate="2019-11-01T07:46:25.240" LastActivityDate="2019-11-01T07:46:25.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10688" PostTypeId="2" ParentId="10679" CreationDate="2019-11-01T10:01:40.383" Score="2" Body="&lt;p&gt;Setup:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;import pyranges as pr&#xA;from pyranges import PyRanges&#xA;from scipy.stats import fisher_exact&#xA;import numpy as np&#xA;&#xA;# ! zcat dataset1.tsv.gz | head -2&#xA;# chromosome    start   end num_motifs_in_group called_sites    called_sites_methylated methylated_frequency    group_sequence&#xA;# chr21 5010053 5010053 1   3   0   0.000   CACCACGTCCA&#xA;&#xA;# avoid using more columns than we need, especially seq which contains object&#xA;# is heavy to lug around&#xA;colnames = [&quot;Chromosome&quot;, &quot;Start&quot;, &quot;End&quot;, &quot;calls&quot;, &quot;methylated&quot;]&#xA;usecols = [0, 1, 2, 4, 5]&#xA;&#xA;h1 = PyRanges(pd.read_csv(&quot;dataset1.tsv.gz&quot;, sep=&quot;\t&quot;, names=colnames, header=0, usecols=usecols))&#xA;h2 = PyRanges(pd.read_csv(&quot;dataset2.tsv.gz&quot;, sep=&quot;\t&quot;, names=colnames, header=0, usecols=usecols))&#xA;&#xA;regions = pr.read_bed(&quot;chr21.bed.gz&quot;)&#xA;regions = regions[~regions.Chromosome.str.contains(&quot;_&quot;)].merge()&#xA;&#xA;regions.ID = list(range(len(regions)))&#xA;&#xA;regions&#xA;# +--------------+-----------+-----------+-----------+&#xA;# | Chromosome   | Start     | End       | ID        |&#xA;# | (category)   | (int32)   | (int32)   | (int64)   |&#xA;# |--------------+-----------+-----------+-----------|&#xA;# | chr21        | 5065660   | 5065810   | 0         |&#xA;# | chr21        | 5068140   | 5068290   | 1         |&#xA;# | chr21        | 5128400   | 5128550   | 2         |&#xA;# | chr21        | 5154780   | 5154930   | 3         |&#xA;# | ...          | ...       | ...       | ...       |&#xA;# | chr21        | 46637040  | 46637190  | 2377      |&#xA;# | chr21        | 46638480  | 46638630  | 2378      |&#xA;# | chr21        | 46639820  | 46639970  | 2379      |&#xA;# | chr21        | 46692680  | 46692830  | 2380      |&#xA;# +--------------+-----------+-----------+-----------+&#xA;# Unstranded PyRanges object has 2,381 rows and 4 columns from 1 chromosomes.&#xA;# For printing, the PyRanges was sorted on Chromosome.&#xA;&#xA;h1&#xA;# +--------------+-----------+-----------+-----------+--------------+&#xA;# | Chromosome   | Start     | End       | calls     | methylated   |&#xA;# | (category)   | (int32)   | (int32)   | (int64)   | (int64)      |&#xA;# |--------------+-----------+-----------+-----------+--------------|&#xA;# | chr21        | 5010053   | 5010053   | 3         | 0            |&#xA;# | chr21        | 5010215   | 5010215   | 5         | 1            |&#xA;# | chr21        | 5010331   | 5010335   | 6         | 0            |&#xA;# | chr21        | 5010525   | 5010525   | 2         | 2            |&#xA;# | ...          | ...       | ...       | ...       | ...          |&#xA;# | chr21        | 46699713  | 46699718  | 274       | 254          |&#xA;# | chr21        | 46699739  | 46699739  | 133       | 113          |&#xA;# | chr21        | 46699756  | 46699756  | 37        | 32           |&#xA;# | chr21        | 46699770  | 46699773  | 156       | 130          |&#xA;# +--------------+-----------+-----------+-----------+--------------+&#xA;# Unstranded PyRanges object has 331,373 rows and 5 columns from 1 chromosomes.&#xA;# For printing, the PyRanges was sorted on Chromosome.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Get total number of methylated per region:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def count_total_and_methylated(df):&#xA;    &quot;&quot;&quot;For each region in bed, get sum of calls and methylated&quot;&quot;&quot;&#xA;&#xA;    grpby = df.groupby(&quot;ID&quot;)&#xA;    total = grpby.calls.sum()&#xA;    methylated = grpby.methylated.sum()&#xA;&#xA;    chromosome = df.Chromosome.iloc[0]&#xA;    start = grpby.Start.first()&#xA;    end = grpby.End.first()&#xA;    region_id = grpby.ID.first()&#xA;&#xA;    return pd.DataFrame({&quot;Chromosome&quot;: chromosome, &quot;Start&quot;: start, &quot;End&quot;: end, &quot;ID&quot;: region_id,&#xA;                         &quot;calls&quot;: total, &quot;methylated&quot;: methylated})&#xA;&#xA;rh1 = regions.join(h1).drop(like=&quot;_b&quot;).apply(count_total_and_methylated)&#xA;rh2 = regions.join(h2).drop(like=&quot;_b&quot;).apply(count_total_and_methylated)&#xA;&#xA;rh1.File = 1&#xA;rh2.File = 2&#xA;&#xA;def add_missing(df, df2):&#xA;&#xA;    &quot;&quot;&quot;Since join is inner rh1 and rh2 might not have the exact same intervals. Fix this.&quot;&quot;&quot;&#xA;&#xA;    missing_in_first = np.setdiff1d(df2.ID, df.ID)&#xA;    first_missing = df2[df2.ID.isin(missing_in_first)].copy()&#xA;&#xA;    f = df.File.head().iloc[0]&#xA;&#xA;    first_missing.loc[:, &quot;File&quot;] = f&#xA;    first_missing.loc[:, &quot;calls&quot;] = 0&#xA;    first_missing.loc[:, &quot;methylated&quot;] = 0&#xA;&#xA;    return pd.concat([df, first_missing])&#xA;&#xA;rh1 = rh1.apply_pair(rh2, add_missing)&#xA;rh2 = rh2.apply_pair(rh1, add_missing)&#xA;&#xA;&#xA;j = rh1.join(rh2, suffix=2).drop(like=&quot;File*|ID*|(Start|End)2&quot;)&#xA;j&#xA;# +--------------+-----------+-----------+-----------+--------------+-----------+---------------+&#xA;# | Chromosome   | Start     | End       | calls     | methylated   | calls2    | methylated2   |&#xA;# | (object)     | (int32)   | (int32)   | (int64)   | (int64)      | (int64)   | (int64)       |&#xA;# |--------------+-----------+-----------+-----------+--------------+-----------+---------------|&#xA;# | chr21        | 5065660   | 5065810   | 188       | 8            | 70        | 0             |&#xA;# | chr21        | 5068140   | 5068290   | 168       | 166          | 74        | 72            |&#xA;# | chr21        | 5128400   | 5128550   | 901       | 14           | 143       | 1             |&#xA;# | chr21        | 5154780   | 5154930   | 653       | 0            | 593       | 0             |&#xA;# | ...          | ...       | ...       | ...       | ...          | ...       | ...           |&#xA;# | chr21        | 46637040  | 46637190  | 66        | 63           | 69        | 62            |&#xA;# | chr21        | 46638480  | 46638630  | 28        | 0            | 30        | 3             |&#xA;# | chr21        | 46639820  | 46639970  | 22        | 0            | 18        | 14            |&#xA;# | chr21        | 46692680  | 46692830  | 71        | 25           | 57        | 23            |&#xA;# +--------------+-----------+-----------+-----------+--------------+-----------+---------------+&#xA;# Unstranded PyRanges object has 1,922 rows and 7 columns from 1 chromosomes.&#xA;# For printing, the PyRanges was sorted on Chromosome.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Compute fisher exact using &lt;a href=&quot;https://github.com/brentp/fishers_exact_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;fisher&lt;/a&gt; instead of scipy:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def fe(df):&#xA;    from fisher import pvalue_npy&#xA;&#xA;    cols = &quot;calls methylated calls2 methylated2&quot;.split()&#xA;    df[cols] = df[cols].astype(np.uint)&#xA;    _, _, two_sided = pvalue_npy(df.calls.values, df.methylated.values, df.calls2.values, df.methylated2.values)&#xA;&#xA;    t1, t2, m1, m2 = df.calls, df.calls2, df.methylated, df.methylated2&#xA;&#xA;    _or =   (t1 * m2) / (t2 * m1)&#xA;&#xA;    df.insert(df.shape[1], &quot;PValue&quot;, two_sided)&#xA;    df.insert(df.shape[1], &quot;OR&quot;, _or)&#xA;&#xA;    return df&#xA;&#xA;result = j.apply(fe, nb_cpu=1)&#xA;result.drop(like=&quot;methylated*|calls*&quot;)&#xA;# +--------------+-----------+-----------+------------------------+--------------------+&#xA;# | Chromosome   | Start     | End       | PValue                 | OR                 |&#xA;# | (object)     | (int32)   | (int32)   | (float64)              | (float64)          |&#xA;# |--------------+-----------+-----------+------------------------+--------------------|&#xA;# | chr21        | 5065660   | 5065810   | 0.1152954430992567     | 0.0                |&#xA;# | chr21        | 5068140   | 5068290   | 0.9999999999995065     | 0.9846955389124064 |&#xA;# | chr21        | 5128400   | 5128550   | 0.7075919513120225     | 0.4500499500499501 |&#xA;# | chr21        | 5154780   | 5154930   | 1.0                    | nan                |&#xA;# | ...          | ...       | ...       | ...                    | ...                |&#xA;# | chr21        | 46638480  | 46638630  | 0.24262295081967644    | inf                |&#xA;# | chr21        | 46639820  | 46639970  | 0.00024909594620266284 | inf                |&#xA;# | chr21        | 46692680  | 46692830  | 0.7354565511951735     | 1.1459649122807019 |&#xA;# | chr21        | 36784100  | 36784250  | 1.0                    | nan                |&#xA;# +--------------+-----------+-----------+------------------------+--------------------+&#xA;# Unstranded PyRanges object has 1,932 rows and 5 columns from 1 chromosomes.&#xA;# For printing, the PyRanges was sorted on Chromosome.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2688" LastEditorUserId="2688" LastEditDate="2019-11-01T14:13:46.460" LastActivityDate="2019-11-01T14:13:46.460" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10689" PostTypeId="1" CreationDate="2019-11-01T10:35:59.327" Score="5" ViewCount="358" Body="&lt;p&gt;I have a collection of phylogenetic trees produced from clustering DNA sequences, mostly a few nodes in each cluster/tree, with several clusters of size 1. And I would like to visualize them all together to get an idea of the relative sizes of the trees in the collection of clusters (cluster set).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;E.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(ABC:0.0,('CL-118070':0.02010925,'CL-117615':0.00526463):0.00443474);&#xA;(DEF:0.0,('CL-117360':0.00967856,'CL-117181':0.00967856):0.00990975);&#xA;(GHI:0.0,('CL-123443':0.00123443,'CL-123443':0.00132446):0.00123435);&#xA;(JKL:0.0,('CL-134123':0.00123441,'CL-545443':0.00343446):0.00134335);&#xA;((((CL-114948:0.00908659,(CL-114989:0.00324875,CL-114959:0.00536289):0.00037038):0.00457230,(CL-114851:0.01291841,CL-114831:0.00590321):0.00077034):0.03411459,XYZ:0.00251364):0.00207594,(CL-114950:0.03074481,CL-114699:0.02256162):0.00733986,CL-116426:0.01643848);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am thinking there must be a tool where I give it an input file with a tree per line and shows them as all in one go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: looking at the options below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_phylogenetic_tree_visualization_software&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/List_of_phylogenetic_tree_visualization_software&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FigTree: in this tool, one can load several trees, one per line, and there is a 'Prev/Next' menu to iterate over them, but no apparent option to display all of them at once.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/clNVV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/clNVV.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Phylo.io: in this tool, one can load several trees, and display one at a time or two of them, with colouring options that compare the common nodes/branches. There isn't an apparent option to visualize many trees in one go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bpObp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bpObp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PhyBin: this tool (&lt;a href=&quot;https://cs.indiana.edu/~rrnewton/projects/phybin/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cs.indiana.edu/~rrnewton/projects/phybin/&lt;/a&gt;) seems to cluster the sets of trees, but not visualize all of them together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GrapeTree: this tool () can visualize only one tree at a time. It displays the last tree in the list if more than one are copy+pasted in the input text box.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JwjTw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JwjTw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is another group of tools under Graph drawing:&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Graph_drawing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Graph_drawing&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cytoscape: this tool is designed for network visualizations, which could be an option. Unsure if it visualizes newick trees, but there is are tools to convert trees to networks that could potentially work in combination with Cytoscape, e.g. &lt;a href=&quot;https://github.com/MonsantoCo/Newick2Network&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/MonsantoCo/Newick2Network&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="3848" LastEditDate="2020-02-22T14:59:05.763" LastActivityDate="2020-03-10T01:56:50.807" Title="tool to visualize a collection of phylogenetic trees" Tags="|phylogenetics|visualization|phylogeny|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10690" PostTypeId="2" ParentId="10689" CreationDate="2019-11-01T15:05:05.800" Score="1" Body="&lt;p&gt;One possible solution is to use ETE Toolkit Treeview, which given a set of Newick files, it will show all of them, albeit connected as shown in the screenshot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6Ng17.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6Ng17.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2019-11-01T15:05:05.800" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10691" PostTypeId="2" ParentId="10623" CreationDate="2019-11-01T17:37:05.783" Score="1" Body="&lt;p&gt;Orthogonal DNA sequences are the shortest possible single stranded nucleotide stretches constructed. These sequences are used as probes in DNA hybridization. Because of its length and construction it has least chance of binding with other probes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This technique improves the efficacy and specificity and is clinically used to detect genetic disorders in shortest time duration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Via:&#xA;&lt;a href=&quot;https://www.nature.com/articles/srep24650&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/srep24650&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.pnas.org/content/106/7/2289&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.pnas.org/content/106/7/2289&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6411" LastActivityDate="2019-11-01T17:37:05.783" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10692" PostTypeId="2" ParentId="10689" CreationDate="2019-11-01T18:06:36.743" Score="-1" Body="&lt;p&gt;Dendroscope is worth a look.  I would also look into the R phangorn package.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-11-01T18:06:36.743" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10693" PostTypeId="1" CreationDate="2019-11-01T20:20:01.387" Score="1" ViewCount="294" Body="&lt;p&gt;How do you calculate the trimming parameter when constructing an additive phylogeny?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it has a relationship with the length of the shortest hanging edge, but I can't figure out how this works.&lt;/p&gt;&#xA;" OwnerUserId="6442" LastEditorUserId="7340" LastEditDate="2020-06-17T05:28:31.990" LastActivityDate="2020-06-17T05:28:31.990" Title="Calculating the trimming parameter when constructing an additive phylogeny" Tags="|phylogenetics|phylogeny|trimming|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10694" PostTypeId="1" CreationDate="2019-11-02T00:48:31.333" Score="2" ViewCount="47" Body="&lt;p&gt;I know this can be very problem-specific -- but it seemed to me the standard approach to train an HMM to recognize a binding pattern of a protein is to use the multiple sequence alignment (MSA), instead of using Baum Welch. I suspect that this is the case because this is what I saw in the current version of HMMer3, which does not provide a baum-welch training procedure for unaligned sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing that I saw training with Baum Welch is that once a HMM is trained, using the Viterbi can give a pattern that contains a lot of insertions in the motif, which doesn't make biological sense. This could be mitigated by using Viterbi training, however, I don't see much literature discussing this as a viable approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It'd be appreciated if someone could share their experience on training a profile HMM to finding motifs.&lt;/p&gt;&#xA;" OwnerUserId="4761" LastEditorUserId="298" LastEditDate="2019-11-02T14:44:12.683" LastActivityDate="2019-11-02T14:44:12.683" Title="Profile HMM to find sequence motif using Baum-Welch instead of using a given multiple sequence alignment?" Tags="|modelling|motifs|hidden-markov-models|hmmer|transcription-factors|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10695" PostTypeId="2" ParentId="8552" CreationDate="2019-11-02T00:48:39.777" Score="0" Body="&lt;p&gt;Go to the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI homepage&lt;/a&gt; and search for 'human genome' (the dropdown menu on the left-hand side of the search box should be 'All Databases'). In the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/search/all/?term=human%20genome&quot; rel=&quot;nofollow noreferrer&quot;&gt;result page&lt;/a&gt;, click on the 'Download' button; choose 'RefSeq' as the 'Source database' and 'Feature table' as the 'File type' as shown in the screenshot below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/f4nwL.png&quot; width=&quot;400&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will download a &lt;code&gt;genome_assemblies.tar&lt;/code&gt; file which contains the feature table file, &lt;code&gt;GCF_000001405.39_GRCh38.p13_feature_table.txt.gz&lt;/code&gt;. This file has all of the annotated features in a tab-delimited format with the following fields: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;              # feature [  1]: gene&#xA;                  class [  2]: transcribed_pseudogene&#xA;               assembly [  3]: GCF_000001405.39&#xA;          assembly_unit [  4]: Primary Assembly&#xA;               seq_type [  5]: chromosome&#xA;             chromosome [  6]: 1&#xA;      genomic_accession [  7]: NC_000001.11&#xA;                  start [  8]: 11874&#xA;                    end [  9]: 14409&#xA;                 strand [ 10]: +&#xA;      product_accession [ 11]: &#xA;   non-redundant_refseq [ 12]: &#xA;      related_accession [ 13]: &#xA;                   name [ 14]: DEAD/H-box helicase 11 like 1&#xA;                 symbol [ 15]: DDX11L1&#xA;                 GeneID [ 16]: 100287102&#xA;              locus_tag [ 17]: &#xA;feature_interval_length [ 18]: 2536&#xA;         product_length [ 19]: &#xA;             attributes [ 20]: pseudo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can parse this file to obtain the CDS lengths of all annotated proteins. As an example, you can get a list of all annotated RefSeq proteins and their lengths as follows: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zcat GCF_000001405.39_GRCh38.p13_feature_table.txt.gz \&#xA;    | awk 'BEGIN{FS=&quot;\t&quot;;OFS=&quot;\t&quot;}(&lt;span class=&quot;math-container&quot;&gt;$1~/^#/ || $&lt;/span&gt;1==&quot;CDS&quot;)' \&#xA;    | cut -f6-11,13,15,16,18,19&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output of the above command will have the following fields: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;             chromosome [  1]: 1&#xA;      genomic_accession [  2]: NC_000001.11&#xA;                  start [  3]: 69091&#xA;                    end [  4]: 70008&#xA;                 strand [  5]: +&#xA;      product_accession [  6]: NP_001005484.1&#xA;      related_accession [  7]: NM_001005484.1&#xA;                 symbol [  8]: OR4F5&#xA;                 GeneID [  9]: 79501&#xA;feature_interval_length [ 10]: 918&#xA;         product_length [ 11]: 305&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2019-11-02T00:48:39.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10698" PostTypeId="1" AcceptedAnswerId="10699" CreationDate="2019-11-03T12:51:23.517" Score="0" ViewCount="451" Body="&lt;p&gt;I am generating 30 asymmetric units of a protein structure. Now, I want to select different combinations of 5 units which are 30 Angstroms away from the reference and save it as a different file. Here is the command I could work out till now :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fetch 3q2v&#xA;symexp sym,3q2v,(3q2v),30&#xA;select chains, bychain all within 30 of 3q2v &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How to loop in and select only 5 chains at a time obtained from the setup obtained by the above code, and save it in separate files?&lt;/p&gt;&#xA;" OwnerUserId="6445" LastActivityDate="2019-11-03T23:17:11.767" Title="pymol script to only select 5 chains within a distance from a reference and save the selection" Tags="|protein-structure|visualization|pymol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10699" PostTypeId="2" ParentId="10698" CreationDate="2019-11-03T23:17:11.767" Score="1" Body="&lt;p&gt;Firstly, I would preface that the PyMOL command line in the GUI is great, but &lt;strong&gt;you can run PyMOL as a regular Python library&lt;/strong&gt; (calling the appropriate python). This makes life so much simpler for more complex manipulations. You can do really complex manipulations with it. I cannot suggest it enough.&#xA;The API version of the commands works only marginally differently, but you can do API calls in the GUI &lt;code&gt;cmd.&lt;/code&gt;) and PyMOLWiki documentation is only slightly weaker for them (&lt;code&gt;help()&lt;/code&gt; fills the gaps).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you really want to use the GUI route...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's get the names of the models in the selection&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;onames = cmd.get_object_list('chains')&#xA;print onames&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, you want to save subsets of 5 chains (technically, objects or models). That is, you do not care about the order they are in and you don't want the same one twice (actually selections with 4 or less), so you want all combinations. Make sure to do the following&#xA;    print len(onames)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And google that number as &lt;code&gt;&amp;lt;n&amp;gt; choose 5&lt;/code&gt;. That is how many files you will get. I am pretty sure most of these sets will be meaningful and will be useless.&#xA;In a script a loop is easy, while in the gui you have to keep it simple or use a list coprehension hackishly.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; from itertools import combinations&#xA; for s in combinations(onames, 5): cmd.save('-'.join(s)+'.pdb', ' or '.join(s))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Obviously, you could alternatively use an enumerate method around the iterator and get an index for the file. But do note f-strings will not work.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; for i, s in enumerate(combinations(onames, 5)): cmd.save(str(i)+'.pdb', ' or '.join(s))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2019-11-03T23:17:11.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10701" PostTypeId="1" CreationDate="2019-11-04T08:53:34.200" Score="1" ViewCount="38" Body="&lt;p&gt;I need to analyze expression of ~ 400 genes with a certain function in the embryonic stem cells. All these genes are combined into one database with RefSeq IDs. What is a recommended workflow to quantify differential expression of these genes in RNA-Seq data of cells and visualize it?&lt;/p&gt;&#xA;&lt;p&gt;All genes for analysis concern Human E3 Ubiquitin Ligases originally located here, &lt;a href=&quot;https://hpcwebapps.cit.nih.gov/ESBL/Database/E3-ligases&quot; rel=&quot;nofollow noreferrer&quot;&gt;hpcwebapps.cit.nih.gov/ESBL/Database/E3-ligases&lt;/a&gt; and the updated database is here:&#xA;&lt;a href=&quot;https://esbl.nhlbi.nih.gov/Databases/KSBP2/Targets/Lists/E3-ligases/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://esbl.nhlbi.nih.gov/Databases/KSBP2/Targets/Lists/E3-ligases/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6448" LastEditorUserId="3848" LastEditDate="2022-07-06T13:33:26.913" LastActivityDate="2022-07-06T13:33:26.913" Title="How to analyze expression of certain group of genes in certain types of cell?" Tags="|rna-seq|sequence-analysis|transcriptome|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10702" PostTypeId="1" CreationDate="2019-11-04T09:02:26.667" Score="1" ViewCount="129" Body="&lt;p&gt;I am trying to summarize a list of QC-parameters and its threshold that should be applied for human WGS in general to avoid bad quality of data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which qc parameters for WGS are you using and why?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="6449" LastActivityDate="2020-04-03T17:00:40.263" Title="Common QC parameters and threshold in human WGS pipeline" Tags="|wgs|qc|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10703" PostTypeId="2" ParentId="10702" CreationDate="2019-11-04T10:04:46.443" Score="2" Body="&lt;p&gt;I would suggest you look at the website for the excellent MultiQC tool (&lt;a href=&quot;https://multiqc.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://multiqc.info/&lt;/a&gt;).  We use it in a clinical setting to collate all the QC statistics from our pipelines into a single report (It displays results from multiple QC programs such as FastQC, Picard, bcl2fastq, VerifyBAMid etc).  The homepage has a selection of example reports - clicking on &quot;Whole Genome Seq&quot; shows a report with the metrics most labs would expect from their WGS pipeline.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you dig around in their GitHub repo (&lt;a href=&quot;https://multiqc.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://multiqc.info/&lt;/a&gt;) you will get a good idea of the default thresholds set for flagging results - there should be a config file somewhere specifying all these values.&lt;/p&gt;&#xA;" OwnerUserId="3100" LastEditorUserId="3100" LastEditDate="2019-11-05T14:09:20.417" LastActivityDate="2019-11-05T14:09:20.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10704" PostTypeId="2" ParentId="10702" CreationDate="2019-11-04T10:21:49.393" Score="0" Body="&lt;p&gt;For your negative controls, you might want to check out QC Fail (&lt;a href=&quot;https://sequencing.qcfail.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sequencing.qcfail.com/&lt;/a&gt;).  There you can see examples of how things look in QC when things have gone wrong, such as loss of quality, position-specific errors on flowcells and contamination with primer dimers.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-11-04T10:21:49.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10705" PostTypeId="1" AcceptedAnswerId="10708" CreationDate="2019-11-04T12:57:25.150" Score="3" ViewCount="1314" Body="&lt;p&gt;The &lt;a href=&quot;http://software.broadinstitute.org/software/igv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IGV browser&lt;/a&gt; lets you export an alignment as an image (File =&gt; Save Image). However, this image only contains those reads that fit in the viewing window:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/s5Ma9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/s5Ma9.png&quot; alt=&quot;IGV screenshot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see in the image above by looking at the scroll bar on the right, this is only a small subset of aligned reads. Is there any way to export an image of the aligned reads that includes &lt;em&gt;all&lt;/em&gt; reads and not only those currently shown in the window?&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="6251" LastEditDate="2019-11-04T16:18:19.797" LastActivityDate="2019-11-04T16:51:47.083" Title="How can I export a full alignment from IGV as an image?" Tags="|sam|ngs|visualization|igv|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="10707" PostTypeId="1" CreationDate="2019-11-04T16:05:47.010" Score="0" ViewCount="554" Body="&lt;p&gt;I am trying to install SRA Toolkit on MacOS, following all the instructions from NCBI website. It is installed but fastq-dump cannot be executed because &quot;cannot execute binary files&quot;. I tried &quot;chmod +x&quot;, noah from home-brew to make binary executable on MacOS but fast-dump still cannot be executed. I attached the screenshot of my terminal window. Can anyone help? &lt;a href=&quot;https://i.stack.imgur.com/kfpqq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kfpqq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6448" LastActivityDate="2019-11-05T09:52:05.133" Title="SRA Toolkit execution problem" Tags="|fastq|sratoolkit|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10708" PostTypeId="2" ParentId="10705" CreationDate="2019-11-04T16:44:41.763" Score="4" Body="&lt;p&gt;Running an IGV batch script helps increase the number of reads per image, compared with &lt;code&gt;File | Save image&lt;/code&gt;, but it does not solve the issue by itself. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To run a script, first make a script file like so with 1 command per line:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;snapshotDirectory ./&#xA;goto chr2:28582094-28582790&#xA;maxPanelHeight 10000&#xA;snapshot chr2_28582094_28582790.png&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Run the script using &lt;code&gt;Tools | Run batch script&lt;/code&gt; menu. The importat trick is the &lt;code&gt;maxPanelHeight 10000&lt;/code&gt; which is what makes this include more reads than the default snapshot. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also convert this into a little shell script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;mkdir -p /tmp/igv&#xA;sleep 2&#xA;echo &quot;snapshotDirectory /tmp/igv&quot;&#xA;sleep 2&#xA;goto chr2:28582094-28582790&#xA;sleep 2&#xA;echo &quot;maxPanelHeight 10000&quot;&#xA;echo &quot;snapshot chr2_28582094_28582790.png&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, just open IGV, load the relevant BAM file and run the script like this to connect to IGV via telnet (&lt;a href=&quot;http://plindenbaum.blogspot.com/2011/07/controlling-igv-through-port-my.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;credit to Pierre Lindenbaum&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sh foo.sh |  telnet 127.0.0.1 60151&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6251" LastEditorUserId="298" LastEditDate="2019-11-04T16:51:47.083" LastActivityDate="2019-11-04T16:51:47.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10709" PostTypeId="2" ParentId="10707" CreationDate="2019-11-05T09:52:05.133" Score="1" Body="&lt;p&gt;You can't run Linux binaries on OSX (or vice versa). Download Mac OSX binaries.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-11-05T09:52:05.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10710" PostTypeId="1" AcceptedAnswerId="10711" CreationDate="2019-11-05T10:44:41.633" Score="2" ViewCount="74" Body="&lt;p&gt;I need to find the following pattern from a genomic sequence&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;TTCGB-N15-TTCGB&#xA;B represents C, G, or T&#xA;N15 represents A, T, G or C and these four base combination should have the length of 15 bases.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To do the same, I tried &lt;code&gt;grep&lt;/code&gt; command, but I could not specify &lt;code&gt;B&lt;/code&gt; and &lt;code&gt;N15&lt;/code&gt; in the &lt;code&gt;grep&lt;/code&gt; command line. Therefore, please help me to find the same.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I have a nucleotide sequences as follows&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;ATGCTTATTGCCCATTTCGTGCATGCATATGCGCATTCGCGATCGATTAGGGATAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Expected outcome,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;TTCGTGCATGCATATGCGCATTCGC&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.  &lt;/p&gt;&#xA;" OwnerUserId="4712" LastEditorUserId="4712" LastEditDate="2019-11-06T08:04:55.797" LastActivityDate="2019-11-06T08:04:55.797" Title="Pattern mining from a genomic sequence" Tags="|r|python|bash|perl|shell|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10711" PostTypeId="2" ParentId="10710" CreationDate="2019-11-05T11:09:23.707" Score="4" Body="&lt;pre&gt;&lt;code&gt;echo &quot;&amp;gt;ATGCTTATTGCCCATTTCGTGCATGCATATGCGCATTCGCGATCGATTAGGGATAT&quot; | grep -oP  'TTCG[CGT][ATGC]{1,15}TTCG[CGT]'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;[CGT]&lt;/code&gt; looks for one occurrence of one of the three.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;[ATGC]{1,15}&lt;/code&gt; looks for &quot;up to 15 letter long&quot; combinations of the four bases.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Moreover, &lt;code&gt;-P&lt;/code&gt; makes your regex call &lt;code&gt;Perl-like&lt;/code&gt; and is required to make this regex work. Different regex flavors behave (slightly) differently.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-11-05T11:15:26.517" LastActivityDate="2019-11-05T11:15:26.517" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="10714" PostTypeId="1" AcceptedAnswerId="10715" CreationDate="2019-11-06T09:51:59.740" Score="2" ViewCount="51" Body="&lt;p&gt;this is a very general question about diploid genome assembly. I am wondering how people deal with heterozygous region in a genome when assembling ?&#xA;Do they pick one of the haplotype to be in the referene and leave out the other ?&#xA;I am particularly thinking about situation similar to mammalian chromosome Y. &#xA;Do you put Y or X in the assembly ? An hybrid ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="6459" LastEditorUserId="6459" LastEditDate="2019-11-06T09:54:45.983" LastActivityDate="2019-11-06T09:54:45.983" Title="How do researchers account for heterozygosity in genome assembly?" Tags="|reference-genome|genome-sequencing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10715" PostTypeId="2" ParentId="10714" CreationDate="2019-11-06T09:54:41.963" Score="1" Body="&lt;p&gt;Generally a single allele is used. This may be arbitrary or match the major allele frequency of the species. For sex chromosomes, usually both/all are included, so for mammals both X and Y are part of the reference genome.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-11-06T09:54:41.963" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10716" PostTypeId="1" CreationDate="2019-11-06T10:14:39.350" Score="5" ViewCount="49" Body="&lt;p&gt;I have ChIP-seq data of RNAPII and two other factors which we think follow RNAPII during transcription in two different conditions, and I'd like to show that the genes that lose RNAPII also lose the other two factors, and same for gains.&lt;/p&gt;&#xA;&lt;p&gt;Do you know any methods that can calculate some sort of covariation coefficients?&lt;/p&gt;&#xA;&lt;p&gt;I know there are methods that calculate covariation of genes between two conditions, but none for different measures on the same gene.&lt;/p&gt;&#xA;" OwnerUserId="6316" LastEditorUserId="3848" LastEditDate="2020-07-09T13:18:42.003" LastActivityDate="2022-01-31T15:01:55.193" Title="Find covariation of different DNA-binding protein binding between two conditions" Tags="|statistics|chip-seq|transcriptome-regulation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10717" PostTypeId="1" AcceptedAnswerId="10719" CreationDate="2019-11-06T16:08:53.123" Score="12" ViewCount="22647" Body="&lt;p&gt;I have following accession numbers of the 10 chromosomes of &lt;em&gt;Theobroma cacao&lt;/em&gt; genome.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NC_030850.1&#xA;NC_030851.1&#xA;NC_030852.1&#xA;NC_030853.1&#xA;NC_030854.1&#xA;NC_030855.1&#xA;NC_030856.1&#xA;NC_030857.1&#xA;NC_030858.1&#xA;NC_030859.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I need to download these FASTA files using the terminal because I'm working on a server. I'll be grateful if someone can help me.&#xA;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="856" LastActivityDate="2020-02-24T19:45:19.593" Title="How to download FASTA sequences from NCBI using the terminal?" Tags="|fasta|phylogenetics|linux|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10718" PostTypeId="2" ParentId="10717" CreationDate="2019-11-06T16:21:23.373" Score="4" Body="&lt;p&gt;Just &lt;code&gt;wget&lt;/code&gt; or &lt;code&gt;curl&lt;/code&gt; each as &lt;code&gt;https://www.ncbi.nlm.nih.gov/nuccore/NC_030853.1?report=fasta&amp;amp;log$=seqview&amp;amp;format=text&lt;/code&gt;...&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2019-11-06T16:21:23.373" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10719" PostTypeId="2" ParentId="10717" CreationDate="2019-11-06T19:17:12.890" Score="11" Body="&lt;p&gt;Alternatively, you can use the &lt;strong&gt;NCBI Entrez Direct UNIX E-utilities&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, you have to download the install file here: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/bin/install-edirect.sh&quot; rel=&quot;noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/books/NBK179288/bin/install-edirect.sh&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the terminal, install it using: &lt;code&gt;source ./install-edirect.sh&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, you can download your sequence by doing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;esearch -db nucleotide -query &quot;NC_030850.1&quot; | efetch -format fasta &amp;gt; NC_030850.1.fasta&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And you should find your fasta sequence downloaded. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you have several sequences to download, I think it will be quite easy to add this command into a little bash script to process all of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More readings about that: &lt;a href=&quot;http://bioinformatics.cvr.ac.uk/blog/ncbi-entrez-direct-unix-e-utilities/&quot; rel=&quot;noreferrer&quot;&gt;http://bioinformatics.cvr.ac.uk/blog/ncbi-entrez-direct-unix-e-utilities/&lt;/a&gt; and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/books/NBK179288/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6437" LastActivityDate="2019-11-06T19:17:12.890" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10720" PostTypeId="1" CreationDate="2019-11-06T23:53:18.657" Score="2" ViewCount="182" Body="&lt;p&gt;I did Mendelian randomization using this software:&#xA;&lt;a href=&quot;https://cran.r-project.org/web/packages/MendelianRandomization/vignettes/Vignette_MR.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cran.r-project.org/web/packages/MendelianRandomization/vignettes/Vignette_MR.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(MendelianRandomization)&#xA;f=read.table(&quot;246LDout272Biobank_Retina.txt&quot;, header=T)&#xA;&amp;gt; head(f)&#xA;          rs exposure.beta exposure.se outcome.beta outcome.se&#xA;1  rs1029830      0.723525  0.03026430  0.066715400  0.0359278&#xA;2  rs1029832      0.723785  0.03029603  0.064105600  0.0359021&#xA;3 rs11078374     -0.411789  0.04189295 -0.000376929  0.0406439&#xA;4 rs11078382      0.882549  0.14275799 -0.197074000  0.1247720&#xA;5  rs1124961     -0.333763  0.05589377 -0.075468600  0.0576012&#xA;6  rs1135237     -0.316831  0.05552530 -0.074086200  0.0573111&#xA;&#xA;MRInputObject &amp;lt;- mr_input(bx = f$exposure.beta,bxse = f$exposure.se,by = f$outcome.beta,byse = f$outcome.se)&#xA;EggerObject &amp;lt;- mr_egger(MRInputObject,robust = FALSE,penalized = FALSE,correl = FALSE,distribution = &quot;normal&quot;,alpha = 0.05)&#xA;&#xA;MR-Egger method&#xA;(variants uncorrelated, random-effect model)&#xA;&#xA;Number of Variants =  246 &#xA;&#xA;------------------------------------------------------------------&#xA;  Method     Estimate Std Error  95% CI       p-value&#xA;  MR-Egger    0.115     0.016  0.084, 0.146   3.28e-13&#xA; (intercept) -0.010     0.008 -0.025, 0.006   0.226&#xA;------------------------------------------------------------------&#xA;Residual Standard Error :  1.106 &#xA;Heterogeneity test statistic = 298.4508 on 244 degrees of freedom,     (p-value = 0.0099)&#xA; I^2_GX statistic: 97.2%&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Let's say that my exposure is called Retina and my outcome is called Biobank. Can someone please help me interpret these results in terms of 'horizontal pleiotropy' as mentioned here: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/29771313&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pubmed/29771313&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="anamaria" LastEditorUserId="3848" LastEditDate="2023-07-25T14:40:55.673" LastActivityDate="2023-07-25T14:40:55.673" Title="How to interpret Mendelian randomization results?" Tags="|r|phylogenetics|phylogeny|statistics|population-genetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10721" PostTypeId="1" CreationDate="2019-11-07T09:31:29.857" Score="2" ViewCount="140" Body="&lt;p&gt;I'm working with ion torrent data where I apply the program cutadapt. &#xA;I'm analyzing ITS seq data, as well as Matk.&#xA;When I'm using cutadapt, I search the information for this genes, and built new files accordingly to his information. For instance, when I search for ITS, I generate a file about this gene, and the same applies to the Matk gene.&#xA;However, more recently I randomly picked a fasta sequence from ITS, blast it and received the information about the matk gene. Is this supposed to occur?&#xA;Thank you for the help and so sorry for the naive question.&lt;/p&gt;&#xA;" OwnerUserId="2686" LastEditorUserId="2686" LastEditDate="2019-11-11T12:16:38.880" LastActivityDate="2024-03-30T23:05:25.683" Title="working with cutadapt" Tags="|python|sequence-analysis|cutadapt|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10722" PostTypeId="1" CreationDate="2019-11-07T09:33:47.077" Score="2" ViewCount="47" Body="&lt;p&gt;CNV is usually detected using either read depth or SNP VAF. However, neither of these methods can detect CNV in case of genome doubling (assuming genome doubling is a kind of genome scale CNV comparing to normal tissue which is diplod). We thought of other platform such as SNP array and we did not think they can detect genome doubling as well. Can any one share some comments on the method for such event? Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="4905" LastEditorUserId="4905" LastEditDate="2019-11-11T03:27:16.937" LastActivityDate="2019-11-11T03:27:16.937" Title="Method to detect genome doubling" Tags="|cnv|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10724" PostTypeId="1" AcceptedAnswerId="10725" CreationDate="2019-11-07T13:35:22.793" Score="0" ViewCount="40" Body="&lt;p&gt;I have a .txt file; I have 139 unique sample IDs in sampleID column&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(mutations)&#xA;                                                                              sampleID chr     pos ref mut&#xA;4 LP2000107-DNA_A01_vs_LP2000102-DNA_A01__pv.1.6__rg.grch37_g1k__al.bwa_mem__.snp.pass   1  920552   G   A&#xA;5 LP2000107-DNA_A01_vs_LP2000102-DNA_A01__pv.1.6__rg.grch37_g1k__al.bwa_mem__.snp.pass   1 1350208   G   A&#xA;6 LP2000107-DNA_A01_vs_LP2000102-DNA_A01__pv.1.6__rg.grch37_g1k__al.bwa_mem__.snp.pass   1 1447367   T   G&#xA;7 LP2000107-DNA_A01_vs_LP2000102-DNA_A01__pv.1.6__rg.grch37_g1k__al.bwa_mem__.snp.pass   1 1909310   G   A&#xA;8 LP2000107-DNA_A01_vs_LP2000102-DNA_A01__pv.1.6__rg.grch37_g1k__al.bwa_mem__.snp.pass   1 2172675   G   C&#xA;9 LP2000107-DNA_A01_vs_LP2000102-DNA_A01__pv.1.6__rg.grch37_g1k__al.bwa_mem__.snp.pass   1 2846623   C   T&#xA;&amp;gt; &#xA;&#xA;&amp;gt; dim(mutations)&#xA;[1] 3574407       5&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to change name of sample from 1 to 139 in R or Linux terminal but I don't know how to do that, any help please?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks &lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-11-07T14:00:21.440" LastActivityDate="2019-11-07T14:06:16.403" Title="Replacing some strings in R or terminal" Tags="|r|linux|looper|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10725" PostTypeId="2" ParentId="10724" CreationDate="2019-11-07T13:51:12.747" Score="1" Body="&lt;p&gt;I believe that &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;uniqsampleID = 1:length(unique(mutations&lt;span class=&quot;math-container&quot;&gt;$sampleID))&#xA;names(uniqsampleID) = unique(mutations$&lt;/span&gt;sampleID)&#xA;mutations&lt;span class=&quot;math-container&quot;&gt;$sampleID = uniqsampleID[mutations$&lt;/span&gt;sampleID]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;should work, assuming &lt;code&gt;mutations&lt;/code&gt; is a data frame.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Explanation:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;1:N&lt;/code&gt; creates a vector of length N, containing 1, 2, ..., N. So using &lt;code&gt;1:length(X)&lt;/code&gt; will do this for each item in X, so &lt;code&gt;1:length(unique(mutations$sampleID))&lt;/code&gt; should give you a vector of &lt;code&gt;1:139&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then &lt;code&gt;names(uniqsampleID) = unique(mutations$sampleID)&lt;/code&gt; assigns to each of these numbers one of these sample IDs, which allows you to retrieve the sample number with &lt;code&gt;uniqsampleID[mutations$sampleID]&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6316" LastEditorUserId="6316" LastEditDate="2019-11-07T14:06:16.403" LastActivityDate="2019-11-07T14:06:16.403" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10726" PostTypeId="2" ParentId="10722" CreationDate="2019-11-07T14:27:51.150" Score="1" Body="&lt;p&gt;The following may work, assuming you have an Illumina genome sequence library with decent coverage, preferably amplification-free:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use kat to produce a kmer spectrum plot of the reads (kmer abundance vs. kmer frequency). When you look at, for example, the 21-mer spectrum of the reads, you would expect to see a peak in the spectrum where the kmer frequency corresponds with the estimated coverage of the genome.  For instance, if your reads would be expected to provide 10x coverage of the original diploid genome, then you would expect a peak in the kmer spectrum at a kmer frequency around 10.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the genome had been doubled, and each of the two copies had undergone a number of mutations, then you would also expect a second peak to be found with half the coverage of the first (at a kmer frequency of 5 in the example).  The relationship between the heights of the two peaks would reflect the degree to which diploidisation had occurred post-doubling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I seem to remember something like a pachytene chromosome spread being used with an optical microscope.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastEditorUserId="3828" LastEditDate="2019-11-07T14:36:12.903" LastActivityDate="2019-11-07T14:36:12.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10727" PostTypeId="2" ParentId="10717" CreationDate="2019-11-07T14:33:49.310" Score="2" Body="&lt;h2&gt;Downloading entire genomes&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The best way to download FASTA sequences for an entire genome is to search for the genome, for example &lt;code&gt;Theobroma cacao genome&lt;/code&gt; in the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Assembly&lt;/a&gt; portal and use the big blue Download button. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/VCa0e.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VCa0e.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Downloading individual chromosomes&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For a given assembly, if you want to download the FASTA sequences for a bunch of chromosomes, you can do that by going to the Genomes FTP path highlighted in the screenshot:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/xWl5Y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xWl5Y.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Once you are in the Genomes FTP path, you can navigate to the FASTA folder as follows: &#xA;&lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/403/535/GCA_000403535.1_Theobroma_cacao_20110822&quot; rel=&quot;nofollow noreferrer&quot;&gt;Theobroma FTP directory&lt;/a&gt; &gt;&gt; &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/403/535/GCA_000403535.1_Theobroma_cacao_20110822/GCA_000403535.1_Theobroma_cacao_20110822_assembly_structure/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Assembly Structure&lt;/a&gt; &gt;&gt; &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/403/535/GCA_000403535.1_Theobroma_cacao_20110822/GCA_000403535.1_Theobroma_cacao_20110822_assembly_structure/Primary_Assembly/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Primary Assembly&lt;/a&gt; &gt;&gt; &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/403/535/GCA_000403535.1_Theobroma_cacao_20110822/GCA_000403535.1_Theobroma_cacao_20110822_assembly_structure/Primary_Assembly/assembled_chromosomes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Assembled Chromosomes&lt;/a&gt; &gt;&gt; &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/403/535/GCA_000403535.1_Theobroma_cacao_20110822/GCA_000403535.1_Theobroma_cacao_20110822_assembly_structure/Primary_Assembly/assembled_chromosomes/FASTA/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FASTA&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Downloading a few sequences&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;For this, you can use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez Direct&lt;/a&gt; as mentioned by @dc37. Why not always use Entrez Direct? While it is fine for a small number of sequences, it can be slow to download a large number of sequences. Entrez Direct by default will download uncompressed data so you will end up spending more time downloading a larger file instead of downloading a smaller, compressed file from FTP more quickly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you were to use Entrez Direct for this purpose, I'd not bother with a bash script and use &lt;code&gt;epost&lt;/code&gt; to first post the entire list of accessions and then pipe it to &lt;code&gt;efetch&lt;/code&gt; as shown below: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;epost -db nuccore -input &amp;lt;file_with_accs&amp;gt; -format acc \&#xA;  | efetch -format fasta &amp;gt; sequences.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2019-11-07T14:33:49.310" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10728" PostTypeId="1" AcceptedAnswerId="10729" CreationDate="2019-11-07T15:25:54.127" Score="2" ViewCount="213" Body="&lt;p&gt;I have 139 samples; In R for each sample I run below codes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for example for &lt;strong&gt;&lt;em&gt;sample 1&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sample_1 = whichSignatures(tumor.ref = sigs.input, &#xA;                           signatures.ref = signatures.nature2013, &#xA;                           sample.id = 1, &#xA;                           contexts.needed = TRUE,&#xA;                           tri.counts.method  = 'genome')&#xA;&#xA;a1=(sample_1[[1]])&#xA;&#xA;&#xA;&#xA;&#xA;&amp;gt; class(sample_1)&#xA;[1] &quot;list&quot;&#xA;&#xA;&amp;gt; a1=(sample_1[[1]])&#xA;&#xA;&amp;gt; head(a1)&#xA;  Signature.1A Signature.1B Signature.2 Signature.3 Signature.4 Signature.5 Signature.6 Signature.7 Signature.8&#xA;1    0.5370653            0           0           0           0           0           0           0           0&#xA;  Signature.9 Signature.10 Signature.11 Signature.12 Signature.13 Signature.14 Signature.15 Signature.16 Signature.17&#xA;1           0            0            0            0            0            0            0            0    0.3383947&#xA;  Signature.18 Signature.19 Signature.20 Signature.21 Signature.R1 Signature.R2 Signature.R3 Signature.U1 Signature.U2&#xA;1    0.1123409            0            0            0            0            0            0            0            0&#xA;&amp;gt; dim(a1)&#xA;[1]  1 27&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and for &lt;strong&gt;&lt;em&gt;sample 2&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sample_2 = whichSignatures(tumor.ref = sigs.input, &#xA;                           signatures.ref = signatures.nature2013, &#xA;                           sample.id = 2, &#xA;                           contexts.needed = TRUE,&#xA;                           tri.counts.method  = 'genome')&#xA;&#xA;a2=(sample_2[[1]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And finally binding &lt;code&gt;a1,a2,...a139&lt;/code&gt; by&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sig=rbind(a1,a2)&#xA;&#xA;&#xA;&amp;gt; print(sig)&#xA;  Signature.1A Signature.1B Signature.2 Signature.3 Signature.4 Signature.5 Signature.6 Signature.7 Signature.8&#xA;1    0.5370653            0           0           0           0           0           0           0           0&#xA;2    0.7162292            0           0           0           0           0           0           0           0&#xA;  Signature.9 Signature.10 Signature.11 Signature.12 Signature.13 Signature.14 Signature.15 Signature.16 Signature.17&#xA;1           0            0            0            0            0            0            0            0    0.3383947&#xA;2           0            0            0            0            0            0            0            0    0.2837708&#xA;  Signature.18 Signature.19 Signature.20 Signature.21 Signature.R1 Signature.R2 Signature.R3 Signature.U1 Signature.U2&#xA;1    0.1123409            0            0            0            0            0            0            0            0&#xA;2    0.0000000            0            0            0            0            0            0            0            0&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;sigs.input&lt;/code&gt; is a common R object here&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing these manually is too painful; Can you please help me how I do that automatically in R? I am not sure but something like looping or whatever you guess possible &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="131" LastEditDate="2020-01-08T10:50:41.803" LastActivityDate="2020-01-08T10:50:41.803" Title="How I loop over this code" Tags="|r|loop|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10729" PostTypeId="2" ParentId="10728" CreationDate="2019-11-07T15:38:58.203" Score="6" Body="&lt;pre&gt;&lt;code&gt;list.of.results &amp;lt;- lapply(seq(INSERT_TOTAL_NUMER_OF_SAMPLES_HERE),function(sample.id){&#xA;whichSignatures(tumor.ref = sigs.input, &#xA;                           signatures.ref = signatures.nature2013, &#xA;                           sample.id = sample.id, &#xA;                           contexts.needed = TRUE,&#xA;                           tri.counts.method  = 'genome')[[1]]&#xA;})&#xA;&#xA;sig &amp;lt;- do.call(rbind,list.of.results)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="383" LastActivityDate="2019-11-07T15:38:58.203" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10731" PostTypeId="1" AcceptedAnswerId="10912" CreationDate="2019-11-07T16:54:06.550" Score="1" ViewCount="237" Body="&lt;p&gt;I'm looking for a way to simulate phenotypes against a real SNP data source, such as the 1000 Genomes. It must be free for commercial purpose (Eg.: MIT license). Any recommendation? &#xA;I'm trying to use the &lt;a href=&quot;http://cnsgenomics.com/software/gcta/#GWASSimulation&quot; rel=&quot;nofollow noreferrer&quot;&gt;GCTA64&lt;/a&gt;, but I couldn't get it working. The documentation doesn't help much.&#xA;At the end of the day, I want:&#xA;1 - Simulate Case/Control and/or quantitative phenotype&#xA;2 - Link it with a real SNP dataset (eg: 1000 Genomes)&#xA;3 - Conduct GWAS analysis using Plink and/or Hail.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking Hail's and Plink's tutorials for GWAS, I realised both use simulated phenotypes from real SNP data sources (1000 Genomes and HapMap, respectively), but how they created the datasets are beyond of the scope of the tutorials, thus not reported.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As mentioned before, I've tried &lt;code&gt;gcta64&lt;/code&gt;, but no success. Here's what I've tried:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1 - Downloaded 1000 Genome sample from Plink page:&#xA;&lt;a href=&quot;https://www.cog-genomics.org/plink/1.9/resources&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entire dataset as a single .tar.gz (1.12 GB) (A2 allele major, not ref, on chr3 before 15 Oct 2017)&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2 - Tried to generate the simulate data by:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./gcta64  --bfile 1kGenomesP1/1kg_phase1_all/1kg_phase1_all --simu-qt  --simu-causal-loci causal.snplist  --simu-hsq 0.5 --simu-rep 3  --keep test&#xA;.indi.list --out 1kg_phase1_all&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Error: &lt;code&gt;Error: --keep test.indi.list not found&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question: What should be the files:&lt;code&gt;causal.snplist&lt;/code&gt;, &lt;code&gt;test.indi.list&lt;/code&gt;, &lt;code&gt;teest&lt;/code&gt;? I know the property name suggests what should be, but there's no example of file structure nowhere... Any practical example or tutorial? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Btw - I apologise in advance if this is a too trivial question. I'm quite new at it. Appreciate your patience and help :)&lt;/p&gt;&#xA;" OwnerUserId="6464" LastEditorUserId="6464" LastEditDate="2019-12-01T16:22:25.517" LastActivityDate="2023-01-07T01:01:10.520" Title="Simulating phenotype with the 1000 Genomes Project" Tags="|snp|gwas|1000genomes|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10733" PostTypeId="2" ParentId="10717" CreationDate="2019-11-07T21:04:30.853" Score="0" Body="&lt;p&gt;(Sorry, I can't post a comment with my reputation score). &#xA;@vkkodali, interesting command I did not know about it ;). However, your command is downloading all sequences from the input file into a single fasta file. I believe that if you have a big bunch of sequences, it could be a little bit tricky after to manipulate that kind of file. Just by curiosity, do you have a trick to create one sequence per file using &lt;code&gt;efetch&lt;/code&gt; ?&lt;/p&gt;&#xA;" OwnerUserId="6437" LastActivityDate="2019-11-07T21:04:30.853" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10734" PostTypeId="1" AcceptedAnswerId="10737" CreationDate="2019-11-07T23:19:53.153" Score="2" ViewCount="2018" Body="&lt;p&gt;Running BWA Mem on a number of paired end fastq files using process substitution on the inputs results in this error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[mem_sam_pe] paired reads have different names: &quot;HWI-D00360:5:H814YADXX:1:2111:20284:33068&quot;, &quot;HWI-D00360:5:H814YADXX:1:1213:17580:34847&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The command: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bwa mem -M ref.fa &quot;&amp;lt;(cat U0a_CGATGT_L001_R1_004.fastq U0a_CGATGT_L001_R1_005.fastq U0a_CGATGT_L002_R1_001.fastq U0a_CGATGT_L002_R1_002.fastq U0a_CGATGT_L002_R1_003.fastq U0a_CGATGT_L001_R1_001.fastq U0a_CGATGT_L001_R1_002.fastq U0a_CGATGT_L002_R1_004.fastq U0a_CGATGT_L001_R1_003.fastq U0a_CGATGT_L002_R1_005.fastq)&quot; &quot;&amp;lt;(cat U0a_CGATGT_L001_R2_003.fastq U0a_CGATGT_L002_R2_005.fastq U0a_CGATGT_L001_R2_004.fastq U0a_CGATGT_L001_R2_005.fastq U0a_CGATGT_L002_R2_001.fastq U0a_CGATGT_L002_R2_002.fastq U0a_CGATGT_L001_R2_001.fastq U0a_CGATGT_L002_R2_003.fastq U0a_CGATGT_L001_R2_002.fastq U0a_CGATGT_L002_R2_004.fastq)&quot; &amp;gt; out.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However the read (there were several) that threw this error does seem to be properly paired: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep 'HWI-D00360:5:H814YADXX:1:2111:20284:33068' *.fastq&#xA;U0a_CGATGT_L001_R1_004.fastq:@HWI-D00360:5:H814YADXX:1:2111:20284:33068 1:N:0:CGATGT&#xA;U0a_CGATGT_L001_R2_004.fastq:@HWI-D00360:5:H814YADXX:1:2111:20284:33068 2:N:0:CGATGT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can see that the concatenation of the input files is happening in a different order, so this is the next thing I'm going to try to fix, but has anyone else run into this?  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I probably just answered my own question but I'll post it anyway, that may or may not be the issue) &lt;/p&gt;&#xA;" OwnerUserId="460" LastActivityDate="2019-11-11T19:31:36.617" Title="Error with BWA Mem input having multiple fastq files using cat and process substitution" Tags="|sequence-alignment|read-mapping|fastq|bwa|bash|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10736" PostTypeId="2" ParentId="10722" CreationDate="2019-11-08T02:53:37.737" Score="3" Body="&lt;p&gt;In a tissue or cell line that used to be diploid and has undergone genome wide doubling, you expect more variants having observed allele frequencies that would also occur in tetraploid organism. That is, observed VAF distribution will have (possibly small) peaks at 0.25 and 0.75, in addition to the diploid peaks at 0, 0.5 and 1. Compare the distributions in the tissue or cell line of interest to those of the control, that is, known diploid. Also if it is a doubled male genome, the sex chromosomes will show a new peak at AF 0.5, that is, heterozygous X or Y variants. This is just an idea, I have never tried to look for such events. &lt;/p&gt;&#xA;" OwnerUserId="6251" LastActivityDate="2019-11-08T02:53:37.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10737" PostTypeId="2" ParentId="10734" CreationDate="2019-11-08T05:24:21.500" Score="3" Body="&lt;p&gt;You are correct that the concatenation of input read files will confuse BWA. BWA depends on the input reads being in the same order in R1 and R2 files for paired-end alignment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;http://bio-bwa.sourceforge.net/bwa.shtml#3&quot; rel=&quot;nofollow noreferrer&quot;&gt;bwa docs&lt;/a&gt;: &lt;code&gt;If mates.fq is present, this command assumes the i-th read in reads.fq and the i-th read in mates.fq constitute a read pair.&lt;/code&gt; BWA will not search through FASTQ files trying to find matching mate IDs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will notice that your initial error message is reporting two different read IDs that are improperly paired. Presumably the mispaired read corresponds to the first read in the out-of-order file. &lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2019-11-08T05:24:21.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10738" PostTypeId="1" CreationDate="2019-11-08T08:16:57.873" Score="1" ViewCount="6088" Body="&lt;p&gt;I want to use the DotPlot function from Seurat v3 to visualise the expression of some genes across clusters. However when the expression of a gene is zero or very low, the dot size is so small that it is not clearly visible when printed on paper. Please is there a possibility to increase the minimum dot size in the DotPlot function to make the dot sizes more visible when printed?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I would like to have a minimum dot size set to be like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aQfL6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aQfL6.png&quot; alt=&quot;in this image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5746044/figure/F11/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this publication&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rather than:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KqEwE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KqEwE.png&quot; alt=&quot;this image&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;from &lt;a href=&quot;https://www.nature.com/articles/s41598-018-24725-0/figures/1#&quot; rel=&quot;nofollow noreferrer&quot;&gt;this other publication&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for your helpful hint.&lt;/p&gt;&#xA;" OwnerUserId="3968" LastActivityDate="2021-11-18T07:35:18.393" Title="How do I increase the minimum dot size in Seurat's DotPlot function?" Tags="|seurat|ggplot2|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10739" PostTypeId="1" AcceptedAnswerId="10740" CreationDate="2019-11-08T09:31:32.483" Score="1" ViewCount="174" Body="&lt;p&gt;I'm trying to download the 1000 Genomes (phase 3) through FTP, but it's taking ages.&#xA;I saw &lt;a href=&quot;https://www.internationalgenome.org/faq/how-download-files-using-aspera&quot; rel=&quot;nofollow noreferrer&quot;&gt;the documentation&lt;/a&gt; about using Aspera (fasp protocol), but the instructions don't work: &lt;code&gt;ERR [ascp] SSH authentication failed&lt;/code&gt;.&#xA;Eg: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ascp -i /home/ibmuser/.aspera/connect/etc/asperaweb_id_dsa.putty -Tr -Q -l 100M \&#xA;     -P33001 -L- fasp-g1k@fasp.1000genomes.ebi.ac.uk:vol1/ftp/release/20130502/ALL.chr1.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz.tbi ./&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Via Aspera Desktop client I got:&#xA;&lt;code&gt;SSH_MSG_DISCONNECT: 2 Too many authentication failures&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="6464" LastEditorUserId="298" LastEditDate="2019-11-08T09:45:02.797" LastActivityDate="2019-11-08T09:56:18.180" Title="There's a faster way to download 1000 Genomes phase 3?" Tags="|1000genomes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10740" PostTypeId="2" ParentId="10739" CreationDate="2019-11-08T09:56:18.180" Score="0" Body="&lt;p&gt;You are using the command for older versions. In the page you link to, they give two commands, one for the newer and one for the older versions of aspera. You are using the one for the older. The only difference is the keyfile, but what you want is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ascp -i /home/ibmuser/.aspera/connect/etc/asperaweb_id_dsa.openssh -Tr -Q -l 100M \&#xA; -P33001 -L- fasp-g1k@fasp.1000genomes.ebi.ac.uk:vol1/ftp/release/20130502/ALL.chr1.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz.tbi ./&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;At least, I just tried this a second ago with the aspera bundle I downloaded today and that's the command that worked: &lt;code&gt;asperaweb_id_dsa.openssh&lt;/code&gt; and not &lt;code&gt;asperaweb_id_dsa.putty&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-11-08T09:56:18.180" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10741" PostTypeId="1" AcceptedAnswerId="10742" CreationDate="2019-11-08T11:22:32.970" Score="0" ViewCount="235" Body="&lt;p&gt;I want to compute the nearest neighbour distances using a for loop. I can do this with a random subset by doing this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df &amp;lt;-  read.table(&quot;tmp.txt&quot;, header=F)&#xA;colnames(df) &amp;lt;- c(&quot;gene&quot;, &quot;seqnames&quot;, &quot;start&quot;, &quot;stop&quot;, &quot;LOCID&quot;)&#xA;gr &amp;lt;- makeGRangesFromDataFrame(df, keep.extra.columns=TRUE)&#xA;&#xA;# genes of interest in a list&#xA;gene.list &amp;lt;- read.table(&quot;careplastic_diffs.txt&quot;)&#xA;&#xA;# filters the GenomicRanges object with genes of interest&#xA;sub.gr &amp;lt;- gr[(elementMetadata(gr)[, &quot;gene&quot;] %in% gene.list$V1)]&#xA;&#xA;# compute distances&#xA;dists &amp;lt;- distanceToNearest(x = sub.gr)&#xA;&#xA;mcols(dists)[,1] # contains a list of distances&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I now want to essentially do this 5 times, using the same logic as above:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;## create a rand dataframe, each column is a random list of 50 genes&#xA;rand.df &amp;lt;- data.frame(replicate(5, sample(gene.list$V1, 50)))&#xA;&#xA;# loop through columns and gets distances and prints to a file&#xA;for(i in 1:ncol(rand.df)){&#xA;  rand.gr[i] &amp;lt;- gr[(elementMetadata(gr)[,&quot;gene&quot;] %in% rand.df[,i])]&#xA;  dists[i] &amp;lt;- distanceToNearest(x = rand.gr[i])&#xA;  output &amp;lt;- print(mcols(dists[i][,1]))&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I can't seem to make this work.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error: subscript contains out-of-bounds indices In addition: Warning&#xA;  message: In recycleSingleBracketReplacementValue(value, x, nsbs) :&lt;br&gt;&#xA;  number of values supplied is not a sub-multiple of the number of&#xA;  values to be replaced&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It's probably something really silly but any help would be appreciated ... &lt;/p&gt;&#xA;" OwnerUserId="4834" LastActivityDate="2019-11-12T23:29:49.513" Title="GenomicRanges: get nearest neighbor distances for random genes using a for loop" Tags="|r|bioconductor|genomicranges|genomefeatures|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10742" PostTypeId="2" ParentId="10741" CreationDate="2019-11-08T11:47:10.547" Score="1" Body="&lt;p&gt;Is it because rand.gr has not been initialised, so when you ask to set rand.gr[i] it can't because there isn't one yet?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you set i to a value (e.g. 1), and run the lines inside your loop, which of the lines produces the error?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't need to retain all the random genomic ranges at the end, you could just leave out some of the [i]s, something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for(i in 1:ncol(rand.df)){&#xA;  rand.gr &amp;lt;- gr[(elementMetadata(gr)[,&quot;gene&quot;] %in% rand.df[,i])]&#xA;  dists &amp;lt;- distanceToNearest(x = rand.gr)&#xA;  output &amp;lt;- print(mcols(dists[,1]))&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3828" LastEditorUserId="3828" LastEditDate="2019-11-08T12:17:14.480" LastActivityDate="2019-11-08T12:17:14.480" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10743" PostTypeId="1" AcceptedAnswerId="10751" CreationDate="2019-11-08T14:45:22.433" Score="0" ViewCount="87" Body="&lt;p&gt;I'm looking for highly variable genes from a data set. I have calculated the coefficient of variation of the expressed genes of my dataset and inserted them in a histogram graph. Now I should select only the highly variable genes. Should I select genes with a coefficient of variation &gt;1, or how?&#xA;thank you for your help,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;regards&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PECPs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PECPs.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4475" LastActivityDate="2019-11-09T05:56:11.460" Title="HVG and variation coefficient" Tags="|r|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10744" PostTypeId="2" ParentId="10734" CreationDate="2019-11-08T14:55:26.733" Score="3" Body="&lt;p&gt;AS both the OP and @MaximilianPress suggested, &lt;strong&gt;the paired-end reads should be provided as 2 separate arguments to bwa&lt;/strong&gt;. This is true whether the reads are in files or streamed. For the sake of completeness, I provide the corrected command here.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;bwa mem -M ref_file.fa &amp;lt;(cat A.read1.fq B.read1.fq ...) &amp;lt;(cat A.read2.fq B.read2.fq ...)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The order of the files is important&lt;/strong&gt;: if the order is A, B for read1, then it should be A, B and &lt;em&gt;not&lt;/em&gt; B, A for read2.&lt;/p&gt;&#xA;&lt;h1&gt;EDIT:&lt;/h1&gt;&#xA;&lt;p&gt;&lt;em&gt;(Thanks to @JohnMarshall for pointing this out.)&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Avoid double quotes around process substitution. Quote the file names (if needed).&lt;/strong&gt; The quotes around process substitution, as apparently intended, work only on some systems/versions&lt;sup&gt;1&lt;/sup&gt;, and seem like an undocumented feature to me.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;DON'T:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;bwa mem -M ref_file.fa &amp;quot;&amp;lt;(cat A.read1.fq B.read1.fq ...)&amp;quot; &amp;quot;&amp;lt;(cat A.read2.fq B.read2.fq ...)&amp;quot;&lt;/code&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;sup&gt;1&lt;/sup&gt;&#xA;Oddly enough, process substitution inside double quotes works at least in the case I tested before posting: &lt;code&gt;bwa mem -M test1.fa &amp;quot;&amp;lt;(cat test_assembler_tiny_1.1.fastq)&amp;quot; &amp;quot;&amp;lt;(cat test_assembler_tiny_1.2.fastq)&amp;quot;&lt;/code&gt; This contradicts the answer here: &lt;a href=&quot;https://unix.stackexchange.com/q/486019&quot;&gt;https://unix.stackexchange.com/q/486019&lt;/a&gt; . I used bwa 0.7.17-r1188, with both GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin17) and zsh 5.7.1 (x86_64-apple-darwin17.7.0), on macOS High Sierra 10.13.6. From OP's error msg, it seems it worked for @glyph too.&lt;/p&gt;&#xA;" OwnerUserId="6251" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2019-11-11T19:31:36.617" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10745" PostTypeId="1" CreationDate="2019-11-08T16:17:07.863" Score="2" ViewCount="4095" Body="&lt;p&gt;I have a bioinformatics exam coming up. I can understand the difference between the global algorithm and the local algorithm, but I have a problem with gap opening penalty and gap extension penalty. What is the difference between gap opening penalty and gap extension? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anybody help me in working out how to use the gap opening penalty and gap extension penalty for blosum62?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are a lot of questions like the one below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Compute the score of the following sequence alignment given the BLOSUM62 matrix below and gap opening penalty (GOP) = 12, and gap extension penalty (GEP) = 2.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SDRVIKAAIFDPIQPDF---G-----PVYFGLGHVH&#xA;RDLVERLFILDMI-PGLIKAGDSFPIPVALMINHIF&#xA;&#xA;   A  R  N  D  C  Q  E  G  H  I  L  K  M  F  P  S  T  W  Y  V&#xA;A  4 -1 -2 -2  0 -1 -1  0 -2 -1 -1 -1 -1 -2 -1  1  0 -3 -2  0&#xA;R -1  5  0 -2 -3  1  0 -2  0 -3 -2  2 -1 -3 -2 -1 -1 -3 -2 -3&#xA;N -2  0  6  1 -3  0  0  0  1 -3 -3  0 -2 -3 -2  1  0 -4 -2 -3&#xA;D -2 -2  1  6 -3  0  2 -1 -1 -3 -4 -1 -3 -3 -1  0 -1 -4 -3 -3&#xA;C  0 -3 -3 -3  9 -3 -4 -3 -3 -1 -1 -3 -1 -2 -3 -1 -1 -2 -2 -1&#xA;Q -1  1  0  0 -3  5  2 -2  0 -3 -2  1  0 -3 -1  0 -1 -2 -1 -2&#xA;E -1  0  0  2 -4  2  5 -2  0 -3 -3  1 -2 -3 -1  0 -1 -3 -2 -2&#xA;G  0 -2  0 -1 -3 -2 -2  6 -2 -4 -4 -2 -3 -3 -2  0 -2 -2 -3 -3&#xA;H -2  0  1 -1 -3  0  0 -2  8 -3 -3 -1 -2 -1 -2 -1 -2 -2  2 -3&#xA;I -1 -3 -3 -3 -1 -3 -3 -4 -3  4  2 -3  1  0 -3 -2 -1 -3 -1  3&#xA;L -1 -2 -3 -4 -1 -2 -3 -4 -3  2  4 -2  2  0 -3 -2 -1 -2 -1  1&#xA;K -1  2  0 -1 -3  1  1 -2 -1 -3 -2  5 -1 -3 -1  0 -1 -3 -2 -2&#xA;M -1 -1 -2 -3 -1  0 -2 -3 -2  1  2 -1  5  0 -2 -1 -1 -1 -1  1&#xA;F -2 -3 -3 -3 -2 -3 -3 -3 -1  0  0 -3  0  6 -4 -2 -2  1  3 -1&#xA;P -1 -2 -2 -1 -3 -1 -1 -2 -2 -3 -3 -1 -2 -4  7 -1 -1 -4 -3 -2&#xA;S  1 -1  1  0 -1  0  0  0 -1 -2 -2  0 -1 -2 -1  4  1 -3 -2 -2&#xA;T  0 -1  0 -1 -1 -1 -1 -2 -2 -1 -1 -1 -1 -2 -1  1  5 -2 -2  0&#xA;W -3 -3 -4 -4 -2 -2 -3 -2 -2 -3 -2 -3 -1  1 -4 -3 -2 11  2 -3&#xA;Y -2 -2 -2 -3 -2 -1 -2 -3  2 -1 -1 -2 -1  3 -3 -2 -2  2  7 -1&#xA;V  0 -3 -3 -3 -1 -2 -2 -3 -3  3  1 -2  1 -1 -2 -2  0 -3 -1  4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6469" LastEditorUserId="73" LastEditDate="2019-11-09T02:23:04.363" LastActivityDate="2020-12-05T10:07:35.767" Title="how to calculate Pairwise Alignment Scores for blosum62" Tags="|sequence-alignment|homework|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10746" PostTypeId="2" ParentId="10743" CreationDate="2019-11-08T16:44:25.320" Score="1" Body="&lt;p&gt;There seems to be a dividing line around CoV=0.9.  Perhaps this is a good place to start for a cutoff.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-11-08T16:44:25.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10747" PostTypeId="1" AcceptedAnswerId="10769" CreationDate="2019-11-08T18:07:09.233" Score="4" ViewCount="609" Body="&lt;p&gt;The samtools documentation for &lt;a href=&quot;http://www.htslib.org/doc/samtools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;mpileup&lt;/a&gt; states:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;At this column, a dot stands for a match to the reference base on the forward strand, a comma for a match on the reverse strand, a '&gt;' or '&amp;lt;' for a reference skip&#xA;  ...&#xA;  Similarly, a pattern `-[0-9]+[ACGTNacgtn]+' represents a deletion from the reference.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;What is a reference skip? &lt;/p&gt;&#xA;" OwnerUserId="272" LastEditorUserId="73" LastEditDate="2019-11-09T02:27:12.213" LastActivityDate="2019-11-11T12:03:15.920" Title="What is a samtools mpileup reference skip?" Tags="|samtools|mpileup|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10748" PostTypeId="1" CreationDate="2019-11-08T18:35:19.780" Score="1" ViewCount="60" Body="&lt;p&gt;I am currently doing a project involving gene rank correlations.  How we rank our genes is based on the absolute values of the log2 fold change.  I.e. a gene which is doubled with respect to the control condition is listed as log2FC=1 a gene at 4xbaseline would be log2FC=2 and a gene at half baseline would be log2FcC=-1.  Since we are more interested simply in if there is an effect i.e. we are hypothesis free about whether or not gene under expression or over expression should be considered important we took the absolute values of these genes so hypothetically a gene at half the expression level of baseline are ranked the same as genes at double the expression level of baseline.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My concern is that this list is across all gene types, i.e. we are lumping kinases in with ion channels and axonal binding etc etc.  Would certain gene ontologies have different log fold changes simply due to a smaller range of raw expression?  And therefore should be normalized within their ontology?&lt;/p&gt;&#xA;" OwnerUserId="2482" LastActivityDate="2019-11-08T18:35:19.780" Title="Gene Ontology Specific Fold Changes" Tags="|microarray|gene-ontology|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10749" PostTypeId="2" ParentId="10747" CreationDate="2019-11-09T02:26:41.753" Score="1" Body="&lt;p&gt;Skipped references are similar to deletions, but the different symbols used are indicating that it's an &lt;em&gt;expected&lt;/em&gt; deletion. The most common use for a skipped reference is when excluding intronic sequence from cDNA reads that are being matched to a genome reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the skipped reference bases are added by the mapping program, which generates the SAM/BAM/CRAM format and decides on whether to call a deletion or a skipped base.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2019-11-09T18:09:50.773" LastActivityDate="2019-11-09T18:09:50.773" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="10750" PostTypeId="1" AcceptedAnswerId="10754" CreationDate="2019-11-09T02:46:05.333" Score="1" ViewCount="356" Body="&lt;p&gt;I am trying to find the subcellular localization of my 10,000 proteins using UniProt.ws package for R. However, I am unable to find all the columns available for query.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used another package named Bioconductor which had easy way to lookup all the columns available using COLUMNS(org.Hs.eg.db)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried reading the documentation and couldn't find anything which can output subcellular localization. I am new to this field, any help is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="6471" LastActivityDate="2019-11-09T14:06:24.417" Title="How can I find all the columns available in UniProt.ws package for R?" Tags="|r|bioconductor|uniprot|rstudio|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10751" PostTypeId="2" ParentId="10743" CreationDate="2019-11-09T05:56:11.460" Score="1" Body="&lt;p&gt;Heuristically, I agree with @Jonathan Moore's answer. However, you could try to take a more disciplined approach and model the distribution as a mixture of two distributions, and infer the parameters of the two distributions from that, which will allow you to make somewhat more informed decisions. A basic intro to mixture modeling in R is &lt;a href=&quot;https://towardsdatascience.com/mixture-modelling-from-scratch-in-r-5ab7bfc83eef&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, a worry of mine when I see data such as this is that you have some confounder that leads to that spike at CV ~ 1. I would actually plot the mean against the SD of the genes in question, and make sure that it is not due to observations of e.g. one transcript across samples when in most samples there are zero transcripts (see last section &lt;a href=&quot;https://stats.idre.ucla.edu/other/mult-pkg/faq/general/faq-what-is-the-coefficient-of-variation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). This would almost certainly be noise, and then what you would be enriching for is genes with very low expression, leading to a very small denominator for the CV calculation. If you are pseudocounting the expression (for example adding 1 to all values to avoid dividing by zero), I would try a few different values for the pseudocount to ensure that the value of the pseudocount is not significantly affecting your CV distribution that you show here. That would be a major warning sign of artifacts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore I would also suggest setting some sort of cutoff for minimal expression of genes in that distribution. Maybe an average of 1 transcript per observation (sample [bulk RNA-seq] or cell [scRNA-seq]) might be a reasonable cutoff for genes to consider.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would do this data cleaning and filtering before making any decisions about this CV distribution. &lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2019-11-09T05:56:11.460" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10752" PostTypeId="2" ParentId="10750" CreationDate="2019-11-09T07:43:07.087" Score="1" Body="&lt;p&gt;Do you have a small sample of the proteins you are looking for and what you have tried so far in R using Uniprot.ws ? (including output of it), it could be helpful to identify your problem and how to solve it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally, I would rather consider Bioconductor as a platform for providing R packages dedicated to bioinformatics than a package to analyze data (but it's my opinion, maybe I'm wrong).&lt;/p&gt;&#xA;" OwnerUserId="6437" LastActivityDate="2019-11-09T07:43:07.087" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10754" PostTypeId="2" ParentId="10750" CreationDate="2019-11-09T14:06:24.417" Score="1" Body="&lt;p&gt;From the &lt;code&gt;UniProt.ws&lt;/code&gt; &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/manuals/UniProt.ws/man/UniProt.ws.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;columns&lt;/code&gt; shows which kinds of data can be returned for the&#xA;  &lt;code&gt;UniProt.ws&lt;/code&gt; object.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, you will need to go over the output of &lt;code&gt;columns(your_uniprotws_object)&lt;/code&gt; if it includes the data type you are looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/9093/how-to-retrieve-the-subcellular-location-info-from-uniprot&quot;&gt;here&lt;/a&gt; is a related question.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-11-09T14:06:24.417" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10755" PostTypeId="1" AcceptedAnswerId="10816" CreationDate="2019-11-09T21:24:46.033" Score="2" ViewCount="2055" Body="&lt;p&gt;I am looking for a python package that can convert InChi keys (e.g.SGNXVBOIDPPRJJ-UHFFFAOYSA-N) to molecular structure or SMILES strings. Can I do that with RDKit? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key comes from: &lt;a href=&quot;https://www.metabolomicsworkbench.org/databases/refmet/refmet_latest.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.metabolomicsworkbench.org/databases/refmet/refmet_latest.txt&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2019-11-20T08:42:32.597" Title="Is there a Python package to convert InChi to molecular structures?" Tags="|structural-biology|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10756" PostTypeId="1" AcceptedAnswerId="10764" CreationDate="2019-11-09T21:42:07.110" Score="1" ViewCount="2314" Body="&lt;p&gt;I have 200 .vcf files in a folder with long names like &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;LP6005409-DNA_E03_vs_LP6005408-DNA_E03.snp.pass.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I change the name of each file to, for example &lt;code&gt;A1.vcf ....A200.vcf&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Renaming these files manually takes ages, help please.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; file.rename(list.files(pattern = &quot;vcf&quot;), paste0(&quot;A&quot;,1:length(list.files(pattern = &quot;.vcf&quot;)),&quot;.vcf&quot;))&#xA;Error in file.rename(list.files(pattern = &quot;vcf&quot;), paste0(&quot;A&quot;, 1:length(list.files(pattern = &quot;.vcf&quot;)),  : &#xA;  'from' and 'to' are of different lengths&#xA;&#xA;&#xA;[fi1d18@cyan01 new]&lt;span class=&quot;math-container&quot;&gt;$ for i in *.vcf; do echo mv $&lt;/span&gt;i A&lt;span class=&quot;math-container&quot;&gt;${k}.vcf ; k=`echo &quot;1+$&lt;/span&gt;{k}&quot; | bc`; done&#xA;mv LP2000325-DNA_A01_vs_LP2000317-DNA_A01.snp.pass.vcf A.vcf&#xA;(standard_in) 2: syntax error&#xA;mv LP6008031-DNA_A04_vs_LP6008032-DNA_D03.snp.pass.vcf A.vcf&#xA;(standard_in) 2: syntax error&#xA;[fi1d18@cyan01 new]$ for i in *.vcf; do mv $i A${k}.vcf ; k=`echo &quot;1+${k}&quot; | bc`; done&#xA;(standard_in) 2: syntax error&#xA;mv: try to overwrite `A.vcf', overriding mode 0500 (r-x------)?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-11-12T10:30:20.277" LastActivityDate="2021-05-23T13:34:31.763" Title="How I can change the name of multiple files at once in R or terminal?" Tags="|r|perl|linux|shell|awk|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10757" PostTypeId="2" ParentId="10756" CreationDate="2019-11-09T22:27:16.563" Score="3" Body="&lt;p&gt;If your filenames do not contain blank spaces, you can do it with a for-loop in &lt;code&gt;bash&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Make a test run with &lt;code&gt;echo&lt;/code&gt; so that the command is only printed but not yet executed:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in *.vcf; do echo mv &lt;span class=&quot;math-container&quot;&gt;$i A$&lt;/span&gt;{k}.vcf ; let k++ ; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And if the result looks good remove the &lt;code&gt;echo&lt;/code&gt; statement to execute the &lt;code&gt;mv&lt;/code&gt; commands:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in *.vcf; do mv &lt;span class=&quot;math-container&quot;&gt;$i A$&lt;/span&gt;{k}.vcf ; let k++ ; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2019-11-15T20:42:39.477" LastActivityDate="2019-11-15T20:42:39.477" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10758" PostTypeId="2" ParentId="10756" CreationDate="2019-11-09T22:30:55.957" Score="3" Body="&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(Based on the comments by @terdon, with minor changes)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try this Perl one-liner. It is recommended for general use, even if your file names contain &quot;unexpected&quot; characters such as newlines.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl -e 'for ( 0..&lt;span class=&quot;math-container&quot;&gt;$#ARGV ) { rename $ARGV[$_] =&amp;gt; &quot;A@{[ $&lt;/span&gt;_ + 1 ]}.vcf&quot;; } ' *pass.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here, &lt;code&gt;-e&lt;/code&gt; tells &lt;code&gt;perl&lt;/code&gt; to look for code on the command line instead of a file with the script. &lt;code&gt;@ARGV&lt;/code&gt; is the array of the command line arguments, here, files with old names. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;for ( 0..$#ARGV ) { ... }&lt;/code&gt; loop goes through the indexes of elements of this array, assigning each implicitly to &lt;code&gt;$_&lt;/code&gt;. The first argument to &lt;code&gt;rename&lt;/code&gt; is the old file name: &lt;code&gt;$ARGV[$_]&lt;/code&gt;. Instead of the comma separator of the method arguments I use &lt;code&gt;=&amp;gt;&lt;/code&gt; synonym for clarity in this method call. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The new file name is the second argument. &lt;code&gt;@{[ ... ]}&lt;/code&gt; inside the double quotes is the Perl idiom to evaluate the expression inside, which is &lt;code&gt;$_ + 1&lt;/code&gt;, to make the file numbering start with 1. If you don't care about this, it is easier to make it start with 0: just use &lt;code&gt;&quot;A$_.vcf&quot;&lt;/code&gt;, or, better, &lt;code&gt;&quot;A${_}.vcf&quot;&lt;/code&gt;. The last form needs to be used for cases like this: &lt;code&gt;&quot;A${_}pass.vcf&quot;&lt;/code&gt; to prevent Perl from treating &lt;code&gt;$_pass&lt;/code&gt; as a variable, where &lt;code&gt;$_&lt;/code&gt; is intended.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;DEPRECATED:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use the one-liner below when you are sure that the old file names do not have &quot;unexpected&quot; characters such as newlines. It is not for general use, see: &lt;a href=&quot;https://mywiki.wooledge.org/ParsingLs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Why you shouldn't parse the output of ls&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It will work with the example shown by the OP. It also works with blanks and tabs in the file names, but it &lt;strong&gt;breaks when file names have newlines&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The (deprecated) Perl one-liner:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ls *.pass.vcf | perl -lne 'rename &lt;span class=&quot;math-container&quot;&gt;$_, &quot;A$&lt;/span&gt;..vcf&quot;;'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here, &lt;code&gt;ls |&lt;/code&gt; lists all files with old names, one per line and passes the stdout into stdin of the next command, which is the Perl one line script. &lt;code&gt;-l&lt;/code&gt; removes the new lines from the input. &lt;code&gt;-n&lt;/code&gt; reads the input one line at a time. These are all commonly used command line Perl options. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the one liner script, &lt;code&gt;$_&lt;/code&gt; is the line read from stdin, and &lt;code&gt;$.&lt;/code&gt; is the line number.&lt;/p&gt;&#xA;" OwnerUserId="6251" LastEditorUserId="6251" LastEditDate="2019-11-11T17:00:13.367" LastActivityDate="2019-11-11T17:00:13.367" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10759" PostTypeId="2" ParentId="10618" CreationDate="2019-11-10T04:05:35.433" Score="2" Body="&lt;p&gt;&lt;code&gt;Autodock-Vina&lt;/code&gt; is a free molecular docking solution. You need to generate different input conformations for each ligand though, e.g. with RDKit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a link to &lt;a href=&quot;http://vina.scripps.edu/tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2019-11-10T04:05:35.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10760" PostTypeId="2" ParentId="7447" CreationDate="2019-11-10T04:09:45.020" Score="2" Body="&lt;p&gt;You can run a Molecular Dynamics simulation to see if the pose is stable and whether the protein changes/adapts it’s conformation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can calculate the ligand efficiency, the docking score divided by the number if heavy atoms, and rank the compounds accordingly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also compare the scores of known ligands to the experimental values and see if the docking the scores are predicting known ligands or correlate with the experimental ones.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At some point you have to make a selection and purchase or synthesize the compounds. How many compounds can you test experimentally? How many can you purchase? What is the purpose of these ligands going to be? Drugs? Did you screen the correct set of ligands e.g. a library of lead compounds that are ready to purchase for cheap, that would be a good start.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2019-11-18T20:10:38.673" LastActivityDate="2019-11-18T20:10:38.673" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10761" PostTypeId="2" ParentId="7336" CreationDate="2019-11-10T04:14:29.770" Score="0" Body="&lt;p&gt;The molecular docking programs usually tread the target structure as rigid. It does not know which atom belongs to the receptor and which is bulk water. So, if you keep the water in the structure the program cannot properly place the ligand there. &lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2019-11-10T04:14:29.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10762" PostTypeId="1" CreationDate="2019-11-10T12:38:15.270" Score="1" ViewCount="400" Body="&lt;p&gt;We have created a model to integrate several OMICs data, but we realized that the maximum TPM values of RNA-Seq data were so big that had unexpected effects on our results. We hypothesized that this might be due to the negative binomial distribution inherent to this OMIC's layer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To tackle this circumstance, we proposed to introduce as input the values obtained after using &lt;a href=&quot;https://rdrr.io/bioc/limma/man/voom.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;voom()&lt;/a&gt; (&lt;a href=&quot;http://www.statsci.org/smyth/pubs/VoomPreprint.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;) in our count data. The reason behind that decision was that voom would transform the input data to be more accurately introduced to a linear model. There are, though, several issues/arguments against doing so:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The results obtained through voom() contain negative values, which do not make sense in our model. I tried to correct that by adding to all the results from a specific sample the absolute minimum value of that sample, but I am not confident whether that is a good way forward.&lt;/li&gt;&#xA;&lt;li&gt;Original 0 values (non-expressed) turn into different negative values across samples (-6 in Sample 1, -4 in Sample 2), creating a difference that did not exist to begin with. &lt;/li&gt;&#xA;&lt;li&gt;Some sources do not recommend to log-transform count data &lt;a href=&quot;https://www.r-bloggers.com/do-not-log-transform-count-data-bitches/&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1]&lt;/a&gt;, including data from negative binomial distributions &lt;a href=&quot;https://besjournals.onlinelibrary.wiley.com/doi/full/10.1111/j.2041-210X.2010.00021.x&quot; rel=&quot;nofollow noreferrer&quot;&gt;[2]&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I am not really versed into distribution transformations, so any comment would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="6477" LastActivityDate="2019-11-10T12:38:15.270" Title="Should I use log2-CPM values (voom-limma) as input for my model?" Tags="|r|rna-seq|statistics|normalization|multi-omics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10764" PostTypeId="2" ParentId="10756" CreationDate="2019-11-11T05:47:33.830" Score="5" Body="&lt;p&gt;just to complete your question, you can do it in R in one line.&lt;/p&gt;&#xA;&lt;p&gt;After setting the directory as your working directory, just type the following:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;file.rename(list.files(pattern = &amp;quot;vcf&amp;quot;), paste0(&amp;quot;A&amp;quot;,1:length(list.files(pattern = &amp;quot;.vcf&amp;quot;)),&amp;quot;.vcf&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Maybe you can try &lt;code&gt;file.rename(list.files(pattern = &amp;quot;.vcf&amp;quot;), paste0(&amp;quot;A&amp;quot;,1:length(list.files(pattern = &amp;quot;.vcf&amp;quot;)),&amp;quot;.vcf&amp;quot;))&lt;/code&gt;. Also, if there is still an error, can you post the output of &lt;code&gt;list.files(pattern = &amp;quot;.vcf&amp;quot;)&lt;/code&gt; just to see the structure of your files in folder (maybe one file has a weird name not detected by the pattern).&lt;/p&gt;&#xA;" OwnerUserId="6437" LastEditorUserId="73" LastEditDate="2021-05-23T13:34:31.763" LastActivityDate="2021-05-23T13:34:31.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10765" PostTypeId="1" AcceptedAnswerId="10768" CreationDate="2019-11-11T07:15:24.030" Score="0" ViewCount="648" Body="&lt;p&gt;I need to extract sequences one after another consecutively from a large fasta files (multiple fasta files) and each extracted files to be saved in new fasta file (I mean the first sequence extracted from multiple fasta file to be saved as gene1 and second sequence extracted from multiple fasta file to be saved in another file and so on). For that, I tried a script like this, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for file in *.fasta do cat &quot;&lt;span class=&quot;math-container&quot;&gt;$file&quot; \&#xA;  | awk '/^&amp;gt;/ {printf(&quot;%s%s\t&quot;,(N&amp;gt;0?&quot;\n&quot;:&quot;&quot;),$0);N++;next;} \&#xA;          {printf(&quot;%s&quot;,$0);} \&#xA;     END {printf(&quot;\n&quot;);}' \&#xA;  | awk '{if ( NR==1 ) print $0;}' \&#xA;  | awk '{print $1&quot;\n&quot;$2}' | uniq &amp;gt; &quot;new_$&lt;/span&gt;{file}&quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But, its not serving my purpose. Firstly, I could not loop my sequence extraction procedure from multiple fasta file and save as gene1 and gene2 and so on.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;For example,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have fasta files as follows,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;org1.fasta&#xA;&amp;gt;locus1 &#xA;ATGCGTAGAG&#xA;&amp;gt;LG12 &#xA;TATGCATGAT&#xA;&amp;gt;bpl&#xA;TAGATGGAT&#xA;&#xA;org2.fasta&#xA;&amp;gt;ysr&#xA;ATGTAGCGA&#xA;&amp;gt;inc&#xA;TGATCTATG&#xA;&amp;gt;pym&#xA;ATGCATAGA&#xA;&#xA;org3.fasta&#xA;&amp;gt;siv&#xA;TAGTAGTAT&#xA;&amp;gt;tdp&#xA;ATGAGTCGA&#xA;&amp;gt;tem&#xA;ATGCTHATG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I need to extract first positioned sequences from all the &lt;code&gt;org1.fasta org2.fasta and org3.fasta&lt;/code&gt; files and save in a new file &lt;code&gt;gene1.fasta&lt;/code&gt; and second positioned sequences as &lt;code&gt;gene2.fasta&lt;/code&gt;and third positioned sequences to be saved as &lt;code&gt;gene3.fasta&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Expected output,&#xA;&#xA;gene1.fasta&#xA;&amp;gt;locus1 &#xA;ATGCGTAGAG&#xA;&amp;gt;ysr&#xA;ATGTAGCGA&#xA;&amp;gt;siv&#xA;TAGTAGTAT&#xA;&#xA;gene2.fasta&#xA;&amp;gt;LG12 &#xA;TATGCATGAT&#xA;&amp;gt;inc&#xA;TGATCTATG&#xA;&amp;gt;tdp&#xA;ATGAGTCGA&#xA;&#xA;gene3.fasta&#xA;&amp;gt;bpl&#xA;TAGATGGAT&#xA;&amp;gt;pym&#xA;ATGCATAGA&#xA;&amp;gt;tem&#xA;ATGCTHATG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Please help me to do the same. &#xA;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="4712" LastActivityDate="2019-11-11T14:13:53.920" Title="How to loop multiple function in shell script?" Tags="|r|python|bash|perl|shell|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10766" PostTypeId="2" ParentId="10745" CreationDate="2019-11-11T08:44:32.010" Score="1" Body="&lt;p&gt;Your alignment is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;&#xA;SDRVIKAAIFDPIQPDF---G-----PVYFGLGHVH&#xA;RDLVERLFILDMI-PGLIKAGDSFPIPVALMINHIF&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For each position in the alignment you calculate the score for that alignment. For position 1 we'd look up S vs R in the matrix and find a score of -1. You continue doing this until you hit the first -, which is not in the matrix. The first - is a gapopening, each consequent - in a series of -'s counts as a gap extension, instead of an opening.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;---G--&#xA;IKAGDS&#xA;&lt;/code&gt;&#xA;gives&#xA;&lt;code&gt;-12 -2 -2 6 -12 -2&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3818" LastActivityDate="2019-11-11T08:44:32.010" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10767" PostTypeId="2" ParentId="10756" CreationDate="2019-11-11T10:14:18.437" Score="9" Body="&lt;p&gt;Use &lt;code&gt;perl-rename&lt;/code&gt;. This is usually called &lt;code&gt;rename&lt;/code&gt; on Debian-based systems like Ubuntu or Mint, and &lt;code&gt;perl-rename&lt;/code&gt; or &lt;code&gt;prename&lt;/code&gt; on others. Assuming you have it as &lt;code&gt;rename&lt;/code&gt;, you can simply do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rename -n 's/.*\.vcf/&quot;A&quot; . ++$c . &quot;.vcf&quot;/e' *snp.pass.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;-n&lt;/code&gt; causes &lt;code&gt;rename&lt;/code&gt; to only print what it would do, without doing anything. So once you make sure it does what you want, run the command again without the &lt;code&gt;-n&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-11-14T13:25:34.770" LastActivityDate="2019-11-14T13:25:34.770" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10768" PostTypeId="2" ParentId="10765" CreationDate="2019-11-11T11:16:34.000" Score="3" Body="&lt;p&gt;You don't need a loop. You can do the whole thing with a simple awk one-liner:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -vRS='&amp;gt;' 'FNR&amp;gt;1{ printf &quot;&amp;gt;%s&quot;,$0 &amp;gt; &quot;gene&quot;FNR-1&quot;.fasta&quot;}' org*fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I ran this on your files and got:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ for file in gene*; do echo &quot;=== File: $&lt;/span&gt;file ===&quot;; cat $file;  done&#xA;=== File: gene1.fasta ===&#xA;&amp;gt;locus1 &#xA;ATGCGTAGAG&#xA;&amp;gt;ysr&#xA;ATGTAGCGA&#xA;&amp;gt;siv&#xA;TAGTAGTAT&#xA;=== File: gene2.fasta ===&#xA;&amp;gt;LG12 &#xA;TATGCATGAT&#xA;&amp;gt;inc&#xA;TGATCTATG&#xA;&amp;gt;tdp&#xA;ATGAGTCGA&#xA;=== File: gene3.fasta ===&#xA;&amp;gt;bpl&#xA;TAGATGGAT&#xA;&amp;gt;pym&#xA;ATGCATAGA&#xA;&amp;gt;tem&#xA;ATGCTHATG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Explanation&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;-vRS='&amp;gt;'&lt;/code&gt; : set the record separator to &lt;code&gt;&amp;gt;&lt;/code&gt;. This will make &lt;code&gt;awk&lt;/code&gt; consider every fasta sequence as a single &quot;line&quot;. &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;FNR&amp;gt;1{foo}&lt;/code&gt; : &lt;code&gt;FNR&lt;/code&gt; is the line number of the current file. Because &lt;code&gt;&amp;gt;&lt;/code&gt; is the first character of the entry, the first &quot;line&quot; will always be empty, so it needs t obe skipped.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;printf &quot;&amp;gt;%s&quot;,$0 &amp;gt; &quot;gene&quot;FNR-1&quot;.fasta&quot;&lt;/code&gt; : this does the work. It prints a &lt;code&gt;&amp;gt;&lt;/code&gt; followed by the current &quot;line&quot; (&lt;code&gt;$0&lt;/code&gt;, this is the entire sequence + the header) and redirects it into a file called &lt;code&gt;gene&lt;/code&gt;, then the current line number -1 (-1 is needed because of the issue mentioned above) and a &lt;code&gt;.fasta&lt;/code&gt; extension.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;=====&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;code&gt;org*fasta&lt;/code&gt; will be expanded with the default collation order. This means that &lt;code&gt;org10.fasta&lt;/code&gt; will be processed before &lt;code&gt;org1.fasta&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ printf &quot;org1.fasta\norg10.fasta\n&quot; | sort &#xA;org10.fasta&#xA;org1.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To avoid this and have the files in numerical order, you can use brace expansion instead of the &lt;code&gt;*&lt;/code&gt;. So, if you know your files are all &lt;code&gt;orgN.fasta&lt;/code&gt; where &lt;code&gt;N&lt;/code&gt; can take values from 1 to 20, do this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -vRS='&amp;gt;' 'FNR&amp;gt;1{ printf &quot;&amp;gt;%s&quot;,$0 &amp;gt; &quot;gene&quot;FNR-1&quot;.fasta&quot;}' org{1..20}.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2019-11-11T14:13:53.920" LastActivityDate="2019-11-11T14:13:53.920" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10769" PostTypeId="2" ParentId="10747" CreationDate="2019-11-11T11:37:19.250" Score="5" Body="&lt;p&gt;That documentation has been expanded recently (see &lt;a href=&quot;https://github.com/samtools/samtools/pull/1055&quot; rel=&quot;nofollow noreferrer&quot;&gt;PR #1055&lt;/a&gt;) and now describes the characters seen in this column in more detail:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Forward   Reverse                    Meaning&#xA;---------------------------------------------------------------&#xA; . dot    , comma   Base matches the reference base&#xA; ACGTN     acgtn    Base is a mismatch to the reference base&#xA;   &amp;gt;         &amp;lt;      Reference skip (due to CIGAR &quot;N&quot;)&#xA;   *         *      Deletion of the reference base (CIGAR &quot;D&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(And so on — see the pull request or &lt;em&gt;doc/samtools-mpileup.1&lt;/em&gt; on the &lt;a href=&quot;https://github.com/samtools/samtools&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;develop&lt;/strong&gt; branch&lt;/a&gt; for the complete pileup format description. The documentation on that web page will be updated when this is released in a new samtools release.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As noted in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/421/134&quot;&gt;https://bioinformatics.stackexchange.com/a/421/134&lt;/a&gt;, SAM has always had separate &lt;code&gt;D&lt;/code&gt; (deletion from the reference) and &lt;code&gt;N&lt;/code&gt; (skipped region from the reference) operators. These are treated similarly computationally but emitted intentionally by aligners to distinguish introns and the like from other deletions.&lt;/p&gt;&#xA;" OwnerUserId="134" LastEditorUserId="134" LastEditDate="2019-11-11T12:03:15.920" LastActivityDate="2019-11-11T12:03:15.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10770" PostTypeId="2" ParentId="10765" CreationDate="2019-11-11T11:45:19.230" Score="1" Body="&lt;p&gt;The title of your question indicates a &lt;code&gt;shell&lt;/code&gt; solution, however you have added &lt;code&gt;R&lt;/code&gt; and &lt;code&gt;Python&lt;/code&gt; as tags. Here is an &lt;code&gt;R&lt;/code&gt; solution making use of &lt;code&gt;Biostrings&lt;/code&gt; pacakge (using two input files for simplicity):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Biostrings)&#xA;&#xA;# create Biostring objects from each multifasta file&#xA;org1 &amp;lt;- readDNAStringSet(&quot;org1.fasta&quot;)&#xA;org2 &amp;lt;- readDNAStringSet(&quot;org2.fasta&quot;)&#xA;&#xA;# loop over the Biostring objects&#xA;for(i in 1:length(org1)){&#xA;&#xA;  output_file &amp;lt;- paste0(&quot;gene_&quot;, i, &quot;.fasta&quot;)&#xA;&#xA;  # create a Biostring object for each index&#xA;  genes_at_index_i &amp;lt;- c(org1[i], org2[i])&#xA;  assign(gene_name, genes_at_index_i)&#xA;&#xA;  # save Biostring object as a multifasta file&#xA;  writeXStringSet(genes_at_index_i, output_file)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output, gene 1:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;locus1 &#xA;ATGCGTAGAG&#xA;&amp;gt;ysr&#xA;ATGTAGCGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output, gene 2:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;LG12 &#xA;TATGCATGAT&#xA;&amp;gt;inc&#xA;TGATCTATG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Warning: The code above won't work if you don't have the same number of fasta entries in each multifasta file.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-11-11T11:45:19.230" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10771" PostTypeId="1" AcceptedAnswerId="10772" CreationDate="2019-11-11T14:14:14.863" Score="0" ViewCount="67" Body="&lt;p&gt;I am trying module preservation with WGCNA using my own data. I am plotting the summary statistics including Zsummary, medianRank, observed IGP and IGP permutation p-value for two comparison data sets.&lt;/p&gt;&#xA;&lt;p&gt;When plotting the scatterplots for IGP &amp;amp; p-value vs. module size I get following error code. I have re-run code and done practice data with no problems but unsure how to correct for this error? Does anyone have any experience with this and know how to resolve? Thank you kindly&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt; plotData[[2]][, 3] = cr[[ref]][[test]]&lt;span class=&quot;math-container&quot;&gt;$Acutal.IGP&#xA;  plotData[[2]],[, 4] = -log(cr[[ref]][[test]]$&lt;/span&gt;p.value + 1e -4)&#xA;&#xA;errror in plotData[[2]][, 3] = cr[[ref]][[test]]$Acutal.IGP&#xA;number of items to replace is not a multiple of replacement length&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6227" LastEditorUserId="650" LastEditDate="2024-01-31T16:36:53.320" LastActivityDate="2024-01-31T16:36:53.320" Title="WGCNA module preservation error code with plotting summary statistics" Tags="|r|wgcna|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10772" PostTypeId="2" ParentId="10771" CreationDate="2019-11-11T14:34:18.900" Score="1" Body="&lt;p&gt;My first guess is that there is a typo in your code, and Acutal.IGP should be Actual.IGP. I checked my tutorials and the word Acutal is in none of them, so I would assume you mistyped the variable name.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2019-11-11T14:34:18.900" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10773" PostTypeId="1" CreationDate="2019-11-11T19:01:05.640" Score="0" ViewCount="38" Body="&lt;p&gt;How could I turn on/off the smooth potential in AutoDock4? I need to do two docking. The first one is soft docking with smooth potential and the second is re-docking without the smooth potential.&lt;/p&gt;&#xA;" OwnerUserId="6481" LastActivityDate="2019-11-11T21:23:53.570" Title="AutoDock 4 -soft docking" Tags="|proteins|software-usage|docking|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10774" PostTypeId="2" ParentId="10773" CreationDate="2019-11-11T21:23:53.570" Score="1" Body="&lt;p&gt;The &lt;code&gt;smooth&lt;/code&gt; parameter is an optional keyword and command that changes the smoothing potential. If not provided, the default is 0.5 angstroms by default. Try passing 0 with this keyword in the AutoGrid Parameter File to turn off smoothing potential.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the &lt;a href=&quot;http://autodock.scripps.edu/faqs-help/manual/autodock-4-2-user-guide/AutoDock4.2_UserGuide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;AutoDock 4.2.5 documentation&lt;/a&gt; for more details.&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2019-11-11T21:23:53.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10776" PostTypeId="1" AcceptedAnswerId="10789" CreationDate="2019-11-12T09:50:54.630" Score="0" ViewCount="449" Body="&lt;p&gt;I have a total of 78 ATAC seq samples from which Im trying to do a peak call .I tried this &lt;code&gt;batchParallel.pl findPeaks peaks&lt;/code&gt; function but i couldn't find the output what it makes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I tried using loop but again the peak file is not written due to incorrect argument usage I guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are my sample directories where i have all the tags&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SRR2920564  SRR2920565&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I used homer to make tag directories and now i using it for peakcalling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;here is my command.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for d in */ ; do findPeaks &lt;span class=&quot;math-container&quot;&gt;$d -style factor -o &quot;$&lt;/span&gt;{d}.txt&quot; ; done &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What i need is all the peakfile should be written as the directory name.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Such as if directory is &quot;&lt;code&gt;SRR2920564&lt;/code&gt;&quot; then my peak file should be written in the same directory as &lt;code&gt;SRR2920564.txt&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do run the loop it runs successfully but no output written. May be a mundane mistake am not able to figure it out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion or help would be highly appreciated .&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2019-11-13T10:35:52.663" Title="Peak-calling using homer" Tags="|shell|atac-seq|peak-calling|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10777" PostTypeId="1" CreationDate="2019-11-12T10:30:16.660" Score="1" ViewCount="930" Body="&lt;p&gt;I want to analyse data from a CRISPR/Cas9 screen (control vs. treatment) and I'm using Mageck (&lt;a href=&quot;https://sourceforge.net/projects/mageck/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sourceforge.net/projects/mageck/&lt;/a&gt;). The problem is that I'm working with paired-end sequencing data, do I have to analyse R1 and R2 fastq files separately or is possible to do it at same time with Mageck?&lt;/p&gt;&#xA;" OwnerUserId="4681" LastActivityDate="2021-04-08T18:54:40.870" Title="CRISPR/Cas9 screen analysis with Mageck: paired-end sequencing" Tags="|ngs|crispr|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10778" PostTypeId="1" CreationDate="2019-11-12T11:14:07.320" Score="-1" ViewCount="88" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;EDITED&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have list of my samples in first column and corresponding file name for my .vcf files in second column of a file name &lt;code&gt;clin_name.txt&lt;/code&gt; like below. For example name of one vcf is &lt;code&gt;LP6008031-DNA_A04_vs_LP6008032-DNA_D03.snp.pass.vcf&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;OC/AH/120/E/T3  LP6005409-DNA_F01&#xA;OC/AH/126/E/T1  LP2000325-DNA_A01&#xA;OC/AH/136/E/T3  LP6005409-DNA_E02&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the link of my file &lt;a href=&quot;https://www.dropbox.com/s/mvzf47feiqe7x69/clin_name.txt.txt?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/mvzf47feiqe7x69/clin_name.txt.txt?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the second file name &lt;code&gt;pre_post.list&lt;/code&gt; I have information of pre or post treatment for each sample like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pre-treated post-treated&#xA;OC/AH/120   OC/AH/065&#xA;OC/AH/126   OC/AH/069&#xA;OC/AH/136   OC/AH/074&#xA;OC/AH/160   OC/AH/084&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is link of this &lt;a href=&quot;https://www.dropbox.com/s/ikrl1h8yoc0sngx/pre_post.list.xlsx?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/ikrl1h8yoc0sngx/pre_post.list.xlsx?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you suggested I typed &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat pre_post.list | awk '{print &lt;span class=&quot;math-container&quot;&gt;$2}' | sed 's@.*/@@'  &amp;gt; post_list.txt&#xA;cat pre_post.list | awk '{print $&lt;/span&gt;1}' | sed 's@.*/@@'  &amp;gt; pre_list.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I have &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pre-treated&#xA;120&#xA;126&#xA;136&#xA;160&#xA;182&#xA;&#xA;post-treated&#xA;065&#xA;069&#xA;074&#xA;084&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;in I typed &lt;code&gt;./test.sh&lt;/code&gt; in which I wrote&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for item in  &lt;span class=&quot;math-container&quot;&gt;$(cat  pre_list.txt | grep -Eo '[0-9]{1,4}');&#xA;    do if grep -q $&lt;/span&gt;item clin_name.txt ;&#xA;    then out=&lt;span class=&quot;math-container&quot;&gt;$(grep $&lt;/span&gt;item clin_name.txt | awk  '{print &lt;span class=&quot;math-container&quot;&gt;$1}');&#xA;         cp *$&lt;/span&gt;out* /temp/hgig/fi1d18/TRG45/snp/&#xA;    else echo post_list $item;  fi;  done;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That returns&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cp: cannot stat `*OC/AH/069/S/T3*': No such file or directory&#xA;[fi1d18@cyan01 snp]$&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or I run &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for item in  &lt;span class=&quot;math-container&quot;&gt;$(cat  post_list.txt | grep -Eo '[0-9]{1,4}');&#xA;    do if grep -q $&lt;/span&gt;item clin_name.txt ;&#xA;    then out=&lt;span class=&quot;math-container&quot;&gt;$(grep $&lt;/span&gt;item clin_name.txt | awk  '{print &lt;span class=&quot;math-container&quot;&gt;$1}');&#xA;         cp *$&lt;/span&gt;out* /temp/hgig/fi1d18/TRG45/snp/pre/&#xA;    else cp *$out* /temp/hgig/fi1d18/TRG45/snp/post/;  fi;  done;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That returns&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cp: cannot stat `OC/ST/026/S/T1*': No such file or directory&#xA;[fi1d18@cyan01 snp]$&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I really don't know how to solve this :(&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-12-05T16:52:40.743" LastActivityDate="2019-12-05T16:52:40.743" Title="How I move these files based on their names in R or terminal" Tags="|r|bash|looper|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10779" PostTypeId="2" ParentId="10661" CreationDate="2019-11-12T11:48:41.900" Score="0" Body="&lt;p&gt;With the 'paste' command you can merge two files. Example :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;paste ret lcls | awk '{print &lt;span class=&quot;math-container&quot;&gt;$1&quot; &quot;$&lt;/span&gt;2&quot; &quot;&lt;span class=&quot;math-container&quot;&gt;$3&quot; &quot;$&lt;/span&gt;6&quot; &quot;&lt;span class=&quot;math-container&quot;&gt;$4&quot; &quot;$&lt;/span&gt;8}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Do you want 4 or 5 columns ?&#xA;A sample of the result you want could help understand your needs.&lt;/p&gt;&#xA;" OwnerUserId="6484" LastActivityDate="2019-11-12T11:48:41.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10780" PostTypeId="1" CreationDate="2019-11-12T13:33:43.350" Score="1" ViewCount="57" Body="&lt;p&gt;Let's say I've run the PCR and I now have a certain amount of the amplicon product. Is there any model that infers how much of it was there before the PCR started? I suspect it should be a probabilistic model (a distribution) parameters (shape) of which will strongly depend on the details of the PCR protocol and the characteristics of the amplicon...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, then may be there are forward models that predict the amount of the amplified result from the initial content?&lt;/p&gt;&#xA;" OwnerUserId="6483" LastActivityDate="2019-11-13T03:52:23.917" Title="In silico PCR: any models that can quantitatively predict the initial content from the end amplified result?" Tags="|pcr|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10781" PostTypeId="1" CreationDate="2019-11-12T15:34:44.060" Score="1" ViewCount="577" Body="&lt;p&gt;I am using bedtools coverage to compute the sequencing depth at every positions of a chromosome but it didn't work as I expected.&#xA;Instead it reported 0 coverage at every positions.&lt;/p&gt;&#xA;&lt;p&gt;This is how I did it :&lt;/p&gt;&#xA;&lt;p&gt;I made a &lt;code&gt;chr1.bed&lt;/code&gt; bedfile :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr start   end &#xA;1   1   158534110&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I ran this command  :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;coverageBed -a -d chr1.bed -b aln.bam -s &amp;gt; coverage.perposition_chr1.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However I don't find any position in the output that is actually covered :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr start end 1-pos depth&#xA;1   1   158534110   1   0&#xA;1   1   158534110   2   0&#xA;1   1   158534110   3   0&#xA;1   1   158534110   4   0&#xA;1   1   158534110   5   0&#xA;1   1   158534110   6   0&#xA;1   1   158534110   7   0&#xA;1   1   158534110   8   0&#xA;(...)&#xA;1   1   158534110   158534109   0&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I must do something wrong because I already computed depth at specific pattern positions using the same &lt;code&gt;aln.bam&lt;/code&gt; and it was working fine...&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Suggestions and tries :&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Swapping &lt;code&gt;-a&lt;/code&gt; &amp;amp; &lt;code&gt;-d&lt;/code&gt; parameters produces the same results&lt;/li&gt;&#xA;&lt;li&gt;Adding a strand column to the bedfile still gives 0 coverage.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;code&gt;chr1.bed&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1       1       158534110       .       0       +&#xA;1       1       158534110       .       0       -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;output&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;1       1       158534110       .       0       +       1       0&#xA;1       1       158534110       .       0       +       2       0&#xA;(...)&#xA;1       1       158534110       .       0       -       158534108       0&#xA;1       1       158534110       .       0       -       158534109       0&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6273" LastEditorUserId="3848" LastEditDate="2020-07-17T19:26:08.787" LastActivityDate="2022-11-17T01:05:42.210" Title="bedtools coverage - Report the depth at each position in each A feature" Tags="|sequence-alignment|bedtools|coverage|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10782" PostTypeId="1" CreationDate="2019-11-12T15:59:57.090" Score="1" ViewCount="170" Body="&lt;p&gt;This question is related to a normal score transformation which is performed in the paper by&#xA;RUAN, Quansong, et al. Local similarity analysis reveals unique associations among marine bacterioplankton species and environmental factors. Bioinformatics, 2006, 22.20: 2532-2538.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They claim that given a variable X with observations &lt;span class=&quot;math-container&quot;&gt;$x= (x_1,\ldots,x_n)$&lt;/span&gt; that are not normally distributed, normal score transformation is performed by first creating a rank vector observations &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$R^x = (R_1^x,\ldots, R_n^x)$&lt;/span&gt; and then using and then transforming to &lt;span class=&quot;math-container&quot;&gt;$x'=(x_1',\ldots,x_n)$&lt;/span&gt;, by &lt;span class=&quot;math-container&quot;&gt;$x_i\ = \phi_{-1} (\frac{R_i^x}{n+1} )$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$\phi_{-1}$&lt;/span&gt; is the inverse of cummulative normal distribution. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's intuitively clear (but maybe a bit harder to prove?) that resulting variable &lt;span class=&quot;math-container&quot;&gt;$X'$&lt;/span&gt; has observations normally distributed, but I have a question about time series dependence. If we had that observations &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; were correlated, say, over 4 adjacent observations, , i.e. &lt;span class=&quot;math-container&quot;&gt;$x_{n+1} = f(x_{n},x_{n-1},x_{n-2})$&lt;/span&gt;, how does this change after the normal transformation? Can we still find a function g, so that &lt;span class=&quot;math-container&quot;&gt;$x'_{n+1} = g(x'_{n},x'_{n-1},x'_{n-2})$&lt;/span&gt; would hold? Since we're transforming based on the rank of data, I expect this to be not possible?&lt;/p&gt;&#xA;" OwnerUserId="6485" LastActivityDate="2020-04-11T22:37:28.397" Title="Normal score transformation" Tags="|statistics|normalization|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10783" PostTypeId="1" CreationDate="2019-11-12T17:03:42.310" Score="0" ViewCount="269" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kFC32.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kFC32.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; As it is shown in the picture that we can get the pseudotime plot by using Monocle. I was wondering whether we can use some function like the Seurat FindMarker to find out the differential expression genes between the Double-KO and WT in the  bottom right corner?&lt;/p&gt;&#xA;" OwnerUserId="3171" LastActivityDate="2019-11-13T02:14:52.810" Title="different gene expression in the monocle" Tags="|seurat|single-cell|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10784" PostTypeId="2" ParentId="10783" CreationDate="2019-11-12T22:32:15.940" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://www.rdocumentation.org/packages/Seurat/versions/3.1.1/topics/FindMarkers&quot; rel=&quot;nofollow noreferrer&quot; &gt; Seurat FindMarker Doc &lt;/a&gt;&#xA;Yes based upon the R documentation here it seems possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This &lt;a href=&quot;https://satijalab.org/seurat/v3.1/de_vignette.html&quot; rel=&quot;nofollow noreferrer&quot; &gt; Differential expression article &lt;/a&gt; seems to show this is how.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Seurat)&#xA;pbmc &amp;lt;- readRDS(file = &quot;../data/pbmc3k_final.rds&quot;)&#xA;# list options for groups to perform differential expression on&#xA;levels(pbmc)&#xA;# Find differentially expressed features between CD14+ and FCGR3A+ Monocytes&#xA;monocyte.de.markers &amp;lt;- FindMarkers(pbmc, ident.1 = &quot;CD14+ Mono&quot;, ident.2 = &quot;FCGR3A+ Mono&quot;)&#xA;# view results&#xA;head(monocyte.de.markers)&#xA;# Find differentially expressed features between CD14+ Monocytes and all other cells, only&#xA;# search for positive markers&#xA;monocyte.de.markers &amp;lt;- FindMarkers(pbmc, ident.1 = &quot;CD14+ Mono&quot;, ident.2 = NULL, only.pos = TRUE)&#xA;# view results&#xA;head(monocyte.de.markers)&#xA;# Pre-filter features that are detected at &amp;lt;50% frequency in either CD14+ Monocytes or FCGR3A+&#xA;# Monocytes&#xA;head(FindMarkers(pbmc, ident.1 = &quot;CD14+ Mono&quot;, ident.2 = &quot;FCGR3A+ Mono&quot;, min.pct = 0.5))&#xA;#...&#xA;# Test for DE features using the MAST package&#xA;head(FindMarkers(pbmc, ident.1 = &quot;CD14+ Mono&quot;, ident.2 = &quot;FCGR3A+ Mono&quot;, test.use = &quot;MAST&quot;))&#xA;head(FindMarkers(pbmc, ident.1 = &quot;CD14+ Mono&quot;, ident.2 = &quot;FCGR3A+ Mono&quot;, test.use = &quot;DESeq2&quot;, max.cells.per.ident = 50))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Maybe something like this would work for you. &lt;a href=&quot;https://rdrr.io/bioc/monocle/man/exportCDS.html&quot; rel=&quot;nofollow noreferrer&quot; &gt; Monocle export &lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  #replace the monocle_cds with your monocle&#xA;  seurat &amp;lt;-exportCDS(monocle_cds, export_to = c(&quot;Seurat&quot;, &quot;Scater&quot;))&#xA;  #This bellow will list the options for ident.1 and ident.2&#xA;  levels(seurat) &#xA;  # insert name from levels(seurat) command in parentheses&#xA;  head(FindMarkers(seurat, ident.1 = &quot;Double-KO+ Mono&quot;, ident.2 = &quot;WT+ Mono&quot;, test.use = &quot;DESeq2&quot;, max.cells.per.ident = 50))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Hopefully you are able to separate the two groups Double-KO and WT into ident.1 and ident.2 so you can use the function on them. Good luck. As long as the exportCDS function works on your data type you input for cds.&lt;/p&gt;&#xA;" OwnerUserId="6487" LastEditorUserId="6487" LastEditDate="2019-11-13T02:14:52.810" LastActivityDate="2019-11-13T02:14:52.810" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10786" PostTypeId="2" ParentId="10741" CreationDate="2019-11-12T23:29:49.513" Score="0" Body="&lt;p&gt;You could do this much more easily and much faster on the command line, starting with an unsorted BED file of genes called &lt;code&gt;genes.unsorted.bed&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sort-bed genes.unsorted.bed | shuf - | head -5 - | sort-bed - | closest-features --closest --dist --no-overlaps - &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Replace the &lt;code&gt;5&lt;/code&gt; in &lt;code&gt;head -5&lt;/code&gt; with however many random genes you want. Remove &lt;code&gt;--no-overlaps&lt;/code&gt; if you want to allow consideration of overlapping genes (which returns a distance of zero).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Output columns are delimited with &lt;code&gt;|&lt;/code&gt;. This can be replaced with another delimiter, but the output can be brought into R easily with &lt;code&gt;read.table&lt;/code&gt; or similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ref: &lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/set-operations/closest-features.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedops.readthedocs.io/en/latest/content/reference/set-operations/closest-features.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2019-11-12T23:29:49.513" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10787" PostTypeId="2" ParentId="10780" CreationDate="2019-11-13T03:52:23.917" Score="1" Body="&lt;p&gt;The basic equation for PCR kinetics (Equation 1) states that the amount of amplicon after c cycles (Nc) is the starting concentration of the amplicon (N0) times the amplification efficiency (E) to the power c. The PCR efficiency in this equation is a number between 1 and 2 (2 indicates 100% efficiency). &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2665230/&quot; rel=&quot;nofollow noreferrer&quot; &gt; NCBI &lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this &lt;a href=&quot;https://www.researchgate.net/post/How_can_I_calculate_the_expected_size_for_amplification_pcr_product_in_a_pcr_reaction&quot; rel=&quot;nofollow noreferrer&quot; &gt; article &lt;/a&gt; on research gate they said &lt;a href=&quot;https://inp.univ-amu.fr/en/amplifx-manage-test-and-design-your-primers-for-pcr&quot; rel=&quot;nofollow noreferrer&quot; &gt; amplifx &lt;/a&gt; was the software you were looking for to, calculate &quot;(i.e. expected efficiency in PCR reaction)&quot;. The amplifx even mentions &quot;Testing primers in in silico PCR&quot;.  &lt;/p&gt;&#xA;" OwnerUserId="6487" LastActivityDate="2019-11-13T03:52:23.917" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10788" PostTypeId="2" ParentId="10778" CreationDate="2019-11-13T06:32:11.597" Score="1" Body="&lt;p&gt;Assuming the names of all the files from one group are in file named list.  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat pre_post.list | awk '{print &lt;span class=&quot;math-container&quot;&gt;$2}' | sed 's@.*/@@'  &amp;gt; post_list.txt&#xA;cat pre_post.list | awk '{print $&lt;/span&gt;1}' | sed 's@.*/@@'  &amp;gt; pre_list.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now you have two number list&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    for item in  &lt;span class=&quot;math-container&quot;&gt;$(cat  pre_list.txt | grep -Eo '[0-9]{1,4}');&#xA;    do if grep -q $&lt;/span&gt;item clin_name.txt ;&#xA;    then out=&lt;span class=&quot;math-container&quot;&gt;$(grep $&lt;/span&gt;item clin_name.txt | awk  '{print &lt;span class=&quot;math-container&quot;&gt;$1}');&#xA;         cp *$&lt;/span&gt;out* /dir/for_pre_files/&#xA;    else echo post_list $item;  fi;  done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;for post&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    for item in  &lt;span class=&quot;math-container&quot;&gt;$(cat  post_list.txt | grep -Eo '[0-9]{1,4}');&#xA;    do if grep -q $&lt;/span&gt;item clin_name.txt ;&#xA;    then out=&lt;span class=&quot;math-container&quot;&gt;$(grep $&lt;/span&gt;item clin_name.txt | awk  '{print &lt;span class=&quot;math-container&quot;&gt;$1}');&#xA;         cp *$&lt;/span&gt;out* /dir/for_post_file/&#xA;    else echo Prelist $item;  fi;  done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or you could do this if one that isn't in post belongs in pre&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    for item in  &lt;span class=&quot;math-container&quot;&gt;$(cat  post_list.txt | grep -Eo '[0-9]{1,4}');&#xA;    do if grep -q $&lt;/span&gt;item clin_name.txt ;&#xA;    then out=&lt;span class=&quot;math-container&quot;&gt;$(grep $&lt;/span&gt;item clin_name.txt | awk  '{print &lt;span class=&quot;math-container&quot;&gt;$1}');&#xA;         cp *$&lt;/span&gt;out* /dir/for_post_file/&#xA;    else cp *$out* /dir/for_pre_file/;  fi;  done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6487" LastActivityDate="2019-11-13T06:32:11.597" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="10789" PostTypeId="2" ParentId="10776" CreationDate="2019-11-13T10:35:52.663" Score="1" Body="&lt;p&gt;You just need to strip the &lt;code&gt;/&lt;/code&gt; from &lt;code&gt;$d&lt;/code&gt;, for which there are a number of options:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for d in */ ; do &#xA;    dname=`basename $d`&#xA;    findPeaks &lt;span class=&quot;math-container&quot;&gt;$d -style factor -o $&lt;/span&gt;{dname}.txt&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for d in */ ; do&#xA;    dname=&lt;span class=&quot;math-container&quot;&gt;${d%%/}&#xA;    findPeaks $&lt;/span&gt;d -style factor -o ${dname}.txt&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are likely other ways one could go about this. Having said that, since you're not running these in parallel you could simply move the default output file (&lt;code&gt;peak.txt&lt;/code&gt;) to a file name of your choice (using one of the strategies above).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-11-13T10:35:52.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10790" PostTypeId="2" ParentId="10782" CreationDate="2019-11-13T11:03:14.763" Score="1" Body="&lt;p&gt;As you suspected, correlations will be heavily changed by such a transformation. The reason for this is also exactly as you described, namely that using ranks rather than values for the transformation eliminates all of the information needed to actually preserve these relationships. As an example, suppose we have a single observation with rank 1 across all observations that none the less shows a decrease in absolute value across them (i.e., there's a correlation with a negative slope over time). Such a transformation will result in a correlation of 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One might be able to find a different transformation that preserves this relationship, or might be able to simply pool all of the time points together before computing the ranks (this would then at least partially preserve some correlations, though depending on what's needed to properly normalize the data it may cause other problems), but in general there's no guarantee that this is possible.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-11-13T11:03:14.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10791" PostTypeId="1" CreationDate="2019-11-13T11:15:30.923" Score="0" ViewCount="72" Body="&lt;p&gt;Greetings,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I’m a newbie in bioinformatics working currently on my first project regarding detection of variants (SNPs and Indels) in mammal assembly genome. I used the following method to detect variants for non-model organism (Camelus dromedarius), and then predict variants effects:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1- Downloading two runs of raw sequencing reads in paired end mode using fastq-dump (version 2.9.3). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2- Raw reads were quality checked, then trimmed using Trimmomatic (version 0.36).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3- Clean reads were mapped to reference genome using BWA-MEM (version 0.7.15). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;4- Merging SAM files to one big file, converting SAM to BAM, sorting reads by coordinate, and marking duplicate reads using Picard (version 2.20.1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5- Create realignment targets using RealignTargetCreator, realign around those targets using IndelRealigner, calling variants for the Base Quality Scoring Recalibration BQSR process using HaplotypeCaller, apply BQSR using two steps of BaseRecalibrator then PrintReads, calling variants using HaplotypeCaller, previous processes were within GATK (version 3.8).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;6- Calling variants again using BCFtools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;7- Overlapping the two VCF files from HaplotypeCaller and BCFtools mpileup using BEDtools. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;8- Filtering variants using BCFtools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;9- Select variants SNPs and Indels in two separated files using GATK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;10- Predicting effects using SnpEff.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;11- SnpEff database was built using reference genome + GTF file which was converted from reference genome GFF version 3 file using gffread.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kindly, I would like to share my pipeline with those whom has an experience in related field, to review the above steps conducted on my task, whether it's accurate or still required additional steps, or rearrangement of the pipeline sequence? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, I would like to share my results with you to make an assessment whether my results are suitable to be reported to my supervisor or not? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your response is highly appreciated,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kind regards,&lt;/p&gt;&#xA;" OwnerUserId="6489" LastActivityDate="2020-04-12T09:17:12.760" Title="Questions regarding the detection of variants (SNPs and Indels) in mammal" Tags="|snp|variant-calling|snpeff|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10792" PostTypeId="2" ParentId="10791" CreationDate="2019-11-13T22:12:53.023" Score="0" Body="&lt;p&gt;It looks like you mostly followed GATK?  It looks reasonable, though a lot will depend on how good your genome and gtf file are.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-11-13T22:12:53.023" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10793" PostTypeId="1" AcceptedAnswerId="10794" CreationDate="2019-11-14T10:20:16.493" Score="1" ViewCount="448" Body="&lt;p&gt;I have 7 different cell types, 4 normal cell types and 3 disease cell types. &#xA;I have done until peak calling. Now I want to do peak annotation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The sample I have is all biological replicates. Should I merge the peak file and then do peak annotation, or should I be doing individual peak annotation? &lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="7340" LastEditDate="2020-06-17T05:28:11.727" LastActivityDate="2020-06-17T05:28:11.727" Title="ATAC-seq peak annotation and downstream analysis" Tags="|atac-seq|homer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10794" PostTypeId="2" ParentId="10793" CreationDate="2019-11-14T11:35:02.673" Score="1" Body="&lt;p&gt;The way we do this (and I think its quite common?), is to merge all the files of the same condition (taking the same number of reads from each), and then calling peaks on the merged sample - so you would have two peak sets - one from normal and one from disease. You then do the union of the two peak sets. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This makes annotation much easier.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For downstream quantitative analysis, you can then quantitate each individual sample against the merged peak set. &lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2019-11-14T11:35:02.673" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="10795" PostTypeId="1" AcceptedAnswerId="10796" CreationDate="2019-11-14T11:39:00.203" Score="1" ViewCount="186" Body="&lt;p&gt;Hello I made a &lt;code&gt;blastX research&lt;/code&gt;  (A query genome in nucleotide format translated into protein in 6 reading frames against a protein dabatase) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here is a head of the result : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;IDBA_scaffold_7517  493 YP_009316100    0.360   61  27  1   150 4   126 186 6.775E-03   37&#xA;IDBA_scaffold_29149 519 AIG51500    0.428   35  18  1   2792    2896    392 424 2.712E-03   40&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have 2 hits: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first one is the query &lt;code&gt;IDBA_scaffold_7517&lt;/code&gt; that have a hit with a protein sequence &lt;code&gt;YP_009316100&lt;/code&gt; in the &lt;code&gt;(-)&lt;/code&gt; strand because the start begins by &lt;code&gt;150&lt;/code&gt; and ends with &lt;code&gt;4&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The seconde one is the query &lt;code&gt;IDBA_scaffold_29149&lt;/code&gt; that have a hit with a protein sequence &lt;code&gt;AIG51500&lt;/code&gt; in the &lt;code&gt;(+)&lt;/code&gt; strand because the start &lt;code&gt;begins 2794&lt;/code&gt; and &lt;code&gt;ends by 2893&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so with that I actually created a bed file in order to extract the fasta sequences from my genome ( I assume that since there is a hit with a protein in the Genomes the coordinates of this hit should be in a good ORF (without stop codon). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the &lt;code&gt;bed_file&lt;/code&gt; I made : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;IDBA_scaffold_7517  4   150 Species 0   -&#xA;IDBA_scaffold_29149 2794    2893    Species 0   +&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then in order to extract the sequences from the coordinates in the Genome I used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools getfasta -fi Genome.fa -bed bed_file -s -fo test&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with &lt;code&gt;-s&lt;/code&gt; to force strandedness. If the feature occupies the antisense, strand, the sequence will be reverse complemented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when I do that the sequences extracted from the query are not in the good ORF. &#xA;I guess it is because I should take into account the fact that the coordinates are from nuc translated to portein but how could I do that? Should I change the coordinates manually from the bed file according to the fact that it is a negative of positif strand ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your help. &lt;/p&gt;&#xA;" OwnerUserId="2328" LastActivityDate="2019-11-14T12:55:16.113" Title="Bedtools get fasta and ORF from a blastX run" Tags="|fasta|bedtools|orf|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10796" PostTypeId="2" ParentId="10795" CreationDate="2019-11-14T12:55:16.113" Score="2" Body="&lt;p&gt;Blast uses a 1-based coordinate system. Bed is 0-based half open. Your code needs to convert appropriately. See for example: &lt;a href=&quot;https://www.biostars.org/p/317112/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/317112/&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="6251" LastActivityDate="2019-11-14T12:55:16.113" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10797" PostTypeId="2" ParentId="653" CreationDate="2019-11-14T13:57:17.357" Score="1" Body="&lt;p&gt;Several have mentioned the excellent Cytoscape desktop package for visualising interactions as a graph.  If you want to build your own web-based visualisations (rather than using a pre-built website to visualise them) then the &lt;a href=&quot;https://js.cytoscape.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cytoscape javascript library&lt;/a&gt; might fulfil your needs.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-11-14T13:57:17.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10798" PostTypeId="2" ParentId="7336" CreationDate="2019-11-14T14:46:50.513" Score="4" Body="&lt;p&gt;My answer is mainly based on molecular dynamics,but in this context it shouldn't make much of a difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main reason has to do with the &lt;code&gt;force-field&lt;/code&gt;-&lt;code&gt;water&lt;/code&gt; model used during the simulation. Examples of water models are &lt;code&gt;SPC&lt;/code&gt;, &lt;code&gt;TIP3P&lt;/code&gt; or &lt;code&gt;TIP4P&lt;/code&gt;; same molecule (water) but different topology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &lt;code&gt;pdb&lt;/code&gt; usually comes from different sources and it can contain molecules that are not standard and therefore not recognised by the model of your simulation. Usually, once the structure is loaded, your system is &lt;code&gt;re-built&lt;/code&gt; so that it conforms to the model topology. There are exception though, for example if your water is part of a functional active site. In that case you can keep it, but its structures (topology of the water molecule) has to be recognised by the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hydrogens are usually added to &quot;saturate&quot; and stabilise the bonds. &lt;/p&gt;&#xA;" OwnerUserId="3631" LastActivityDate="2019-11-14T14:46:50.513" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10800" PostTypeId="1" AcceptedAnswerId="10806" CreationDate="2019-11-14T19:49:00.123" Score="1" ViewCount="5097" Body="&lt;p&gt;I'm working with some large Seurat objects (MOCA, MCA, Tabula Muris) studying gene coexpression, and I'm running into memory issues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to remove all genes in a Seurat object that are not in a specified gene array? Ideally I'd like to be working with a small R matrix with say, 250k rows (cells) and 10-50 columns (gene expression data) which will enable fast processing and comprehensive analysis using other packages.&lt;/p&gt;&#xA;" OwnerUserId="6498" LastEditorUserId="3694" LastEditDate="2022-11-27T09:14:28.060" LastActivityDate="2022-11-27T09:14:28.060" Title="Trim Seurat object to contain expression info only for selected genes" Tags="|r|scrnaseq|seurat|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10801" PostTypeId="1" AcceptedAnswerId="10857" CreationDate="2019-11-14T19:56:42.333" Score="2" ViewCount="8098" Body="&lt;p&gt;I apologize for this very basic question. With @Michael G.'s help, I have been learning how to use RAxML to build a phylogenetic tree (&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/10534/how-to-define-an-outgroup-to-build-a-robust-amino-acid-tree&quot;&gt;How to define an outgroup to build a robust amino acid tree&lt;/a&gt;) and after running my analysis I obtained several results files. I have both a &lt;em&gt;RAxML_bestTree.result&lt;/em&gt; file and a &lt;em&gt;RAxML_bootstrap.result&lt;/em&gt; file. How do I &quot;combine&quot; these files so that my tree shows the bootstrap values? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help will be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="5116" LastEditorUserId="3848" LastEditDate="2020-04-27T15:29:16.443" LastActivityDate="2020-04-27T15:29:16.443" Title="How to show bootstrap values on a phylogenetic tree constructed with RAxML" Tags="|phylogenetics|phylogeny|software-usage|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10802" PostTypeId="1" AcceptedAnswerId="10803" CreationDate="2019-11-15T01:09:40.620" Score="0" ViewCount="60" Body="&lt;p&gt;I have a matrix in which sample IDs are in rows and genes are in column like below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(data)&#xA;                                 ETV7    PLAUR  SLC11A1&#xA;TCGA.JY.A6FB.01A.11R.A336.31 1.791167 6.241607 3.709848&#xA;TCGA.JY.A938.01A.11R.A37I.31 5.389175 7.048782 3.124435&#xA;                                BIRC3 TNFRSF1B&#xA;TCGA.JY.A6FB.01A.11R.A336.31 6.726991 5.758026&#xA;TCGA.JY.A938.01A.11R.A37I.31 8.097783 6.621900&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In another matrix I have sample IDs, months to death and status while the order of sample IDs is not the same with my previous matrix like below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Patient_ids OS_time  OS_event&#xA;TCGA-IG-A4QS    3.933333333 1&#xA;TCGA-JY-A6FB    61.23333333 0&#xA;TCGA-JY-A938    35.33333333 0&#xA;TCGA-JY-A939    22  0&#xA;TCGA-JY-A93C    23.5    0&#xA;TCGA-JY-A93E    25.56666667 0&#xA;TCGA-L5-A4OJ    21.3    0&#xA;TCGA-L5-A4OS    59.4    0&#xA;TCGA-L5-A8NM    7.866666667 1&#xA;TCGA-R6-A6KZ    5.133333333 1&#xA;TCGA-R6-A6XG    38.93333333 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How I can bind OS_time and OS_event columns from second matrix to the first matrix keeping the right sample order?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;EDITED&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; # Reproducible example:&#xA;&amp;gt; patient_id = c(&quot;TCGA.JY.A6FB.01A.11R.A336.31&quot;,&quot;TCGA.JY.A938.01A.11R.A37I.31&quot;,&quot;TCGA.L5.A4OS.01A.11R.A212.31&quot;,&quot;TCGA.AA.A400.01A.11R.A212.31&quot;)&#xA;&amp;gt; ETV7 = c(1.79,5.38,5,6)&#xA;&amp;gt; PLAUR = c(6.24,7.05,5,6)&#xA;&amp;gt; SLC11A1 = c(3.7,3.12,5,6)&#xA;&amp;gt; df = data.frame(ETV7,PLAUR, SLC11A1)&#xA;&amp;gt; rownames(df) = patient_id&#xA;&amp;gt; # OS data&#xA;&amp;gt; Patient_ids = c(&quot;TCGA-IG-A4QS&quot;,&quot;TCGA-JY-A6FB&quot;,&quot;TCGA-JY-A938&quot;,&quot;TCGA-JY-A939&quot;,&quot;TCGA-JY-A93C&quot;,&quot;TCGA-JY-A93E&quot;,&quot;TCGA-L5-A4OJ&quot;,&quot;TCGA-L5-A4OS&quot;)&#xA;&amp;gt; OS_time = c(3.93,61.23,35.33,22,23.5,25.56,21.3,59.4)&#xA;&amp;gt; os_data = data.frame(Patient_ids,OS_time)&#xA;&amp;gt; library(rebus.base)&#xA;&amp;gt; df&lt;span class=&quot;math-container&quot;&gt;$Patient_id = gsub(DOT,&quot;-&quot;,rownames(df))&#xA;&amp;gt; position = sapply(os_data$&lt;/span&gt;Patient_ids,function(x){grep(x,df&lt;span class=&quot;math-container&quot;&gt;$Patient_id)})&#xA;&amp;gt; position = lapply(position,function(x){c(x,rep(NA,1-length(x)))})&#xA;&amp;gt; os_data$&lt;/span&gt;Position_matched = unlist(position)&#xA;&amp;gt; m = data.frame(matrix(ncol = length(df),nrow = nrow(os_data)))&#xA;&amp;gt; colnames(m) = colnames(df)&#xA;&amp;gt; for(i in 1:length(position))&#xA;+ {&#xA;+   if(is.na(position[i])){}&#xA;+   else{&#xA;+     m[i,] = df[position[i],]&#xA;+   }&#xA;+ }&#xA;Error in xj[i] : invalid subscript type 'list'&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-11-17T20:44:26.610" LastActivityDate="2019-11-17T20:48:37.737" Title="Binding columns with matched elements" Tags="|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10803" PostTypeId="2" ParentId="10802" CreationDate="2019-11-15T05:48:23.923" Score="2" Body="&lt;p&gt;Here a little way to do it. I don't think it is a perfect solution but it should work.&#xA;The trick here is to compare &lt;code&gt;patients_id&lt;/code&gt; that are wrote with a &lt;code&gt;.&lt;/code&gt; or with a &lt;code&gt;-&lt;/code&gt;.&#xA;So, here is how I do (I modified your example in order to reflect several configurations):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# Reproducible example:&#xA;patient_id = c(&quot;TCGA.JY.A6FB.01A.11R.A336.31&quot;,&quot;TCGA.JY.A938.01A.11R.A37I.31&quot;,&quot;TCGA.L5.A4OS.01A.11R.A212.31&quot;,&quot;TCGA.AA.A400.01A.11R.A212.31&quot;)&#xA;ETV7 = c(1.79,5.38,5,6)&#xA;PLAUR = c(6.24,7.05,5,6)&#xA;SLC11A1 = c(3.7,3.12,5,6)&#xA;&#xA;df = data.frame(ETV7,PLAUR, SLC11A1)&#xA;rownames(df) = patient_id&#xA;&#xA;# OS data&#xA;Patient_ids = c(&quot;TCGA-IG-A4QS&quot;,&quot;TCGA-JY-A6FB&quot;,&quot;TCGA-JY-A938&quot;,&quot;TCGA-JY-A939&quot;,&quot;TCGA-JY-A93C&quot;,&quot;TCGA-JY-A93E&quot;,&quot;TCGA-L5-A4OJ&quot;,&quot;TCGA-L5-A4OS&quot;)&#xA;OS_time = c(3.93,61.23,35.33,22,23.5,25.56,21.3,59.4)&#xA;os_data = data.frame(Patient_ids,OS_time)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, I create a new column in &lt;code&gt;df&lt;/code&gt; that will contains &lt;code&gt;patient_id&lt;/code&gt; wrote with a &lt;code&gt;-&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(rebus.base)&#xA;df$Patient_id = gsub(DOT,&quot;-&quot;,rownames(df))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, I'm looking for common pattern between &lt;code&gt;df$Patient_id&lt;/code&gt; and &lt;code&gt;os_data$Patient_ids&lt;/code&gt;, find their matching row position and extract these lines:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;position = sapply(os_data$Patient_ids,function(x){grep(x,df$Patient_id)})&#xA;position = lapply(position,function(x){c(x,rep(NA,1-length(x)))})&#xA;position = unlist(position)&#xA;os_data$Position_matched = position&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, we are creating a matrix &lt;code&gt;m&lt;/code&gt; that we will fill with matching values&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;m = data.frame(matrix(ncol = length(df),nrow = nrow(os_data)))&#xA;colnames(m) = colnames(df)&#xA;&#xA;for(i in 1:length(position))&#xA;{&#xA;  if(is.na(position[i])){}&#xA;  else{&#xA;    m[i,] = df[position[i],]&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And now, we can bind both &lt;code&gt;os_data&lt;/code&gt; and &lt;code&gt;m&lt;/code&gt;: &lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;New_df = cbind(os_data,m)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And here the final output:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; New_df&#xA;   Patient_ids OS_time Position_matched ETV7 PLAUR SLC11A1                   Patient_id&#xA;1 TCGA-IG-A4QS    3.93               NA   NA    NA      NA                         &amp;lt;NA&amp;gt;&#xA;2 TCGA-JY-A6FB   61.23                1 1.79  6.24    3.70 TCGA-JY-A6FB-01A-11R-A336-31&#xA;3 TCGA-JY-A938   35.33                2 5.38  7.05    3.12 TCGA-JY-A938-01A-11R-A37I-31&#xA;4 TCGA-JY-A939   22.00               NA   NA    NA      NA                         &amp;lt;NA&amp;gt;&#xA;5 TCGA-JY-A93C   23.50               NA   NA    NA      NA                         &amp;lt;NA&amp;gt;&#xA;6 TCGA-JY-A93E   25.56               NA   NA    NA      NA                         &amp;lt;NA&amp;gt;&#xA;7 TCGA-L5-A4OJ   21.30               NA   NA    NA      NA                         &amp;lt;NA&amp;gt;&#xA;8 TCGA-L5-A4OS   59.40                3 5.00  5.00    5.00 TCGA-L5-A4OS-01A-11R-A212-31&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Warning:&lt;/strong&gt; This script should work for some simple case like yours but take extra cautious to the &lt;code&gt;grep&lt;/code&gt; output as the short &lt;code&gt;patient_id&lt;/code&gt; term of the &lt;code&gt;os_data&lt;/code&gt; could eventually generate several hits instead of one.&lt;/p&gt;&#xA;" OwnerUserId="6437" LastEditorUserId="6437" LastEditDate="2019-11-17T20:48:37.737" LastActivityDate="2019-11-17T20:48:37.737" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="10805" PostTypeId="1" AcceptedAnswerId="10815" CreationDate="2019-11-15T15:01:39.823" Score="0" ViewCount="269" Body="&lt;p&gt;I am running foldx position scan for GFPmut2 (PDB: 6GO9) and I use a config file with the following template:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;command=PositionScan&#xA;pdb=6GO9.pdb&#xA;pdb-dir= &amp;lt;pdb location&amp;gt;&#xA;output-dir= &amp;lt;output directory&amp;gt;&#xA;output-file= &amp;lt;file name&amp;gt;&#xA;positions= &amp;lt;list of positions, comma separated: QA204R,KA107R...&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I had created three different config files with different position list. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some of the positions were present in more than one files. Foldx produced different estimates of &amp;Delta;&amp;Delta;G for these positions from these different config files. Example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;File1-LYSA41K   0&#xA;File1:LYSA41R   -0.353758&#xA;--&#xA;File2-LYSA41K   0&#xA;File2:LYSA41R   -0.0356187&#xA;--&#xA;File3-LYSA41K   0&#xA;File3:LYSA41R   -0.0031362&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Another example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;File1-LYSA52K   0&#xA;File1:LYSA52R   0.0057333&#xA;--&#xA;File2-LYSA52K   0&#xA;File2:LYSA52R   -0.00809613&#xA;--&#xA;File3-LYSA52K   0&#xA;File3:LYSA52R   0.0077189&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I can certainly run foldx with a union of the three list of positions but I don't understand how this difference can even occur. Shouldn't each position scan be independent? &lt;/p&gt;&#xA;" OwnerUserId="345" LastActivityDate="2019-11-20T08:48:23.827" Title="Can foldx position-scan produce different result with different lists of mutations?" Tags="|protein-structure|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10806" PostTypeId="2" ParentId="10800" CreationDate="2019-11-15T15:02:41.757" Score="1" Body="&lt;p&gt;Is your memory usage low enough that you can initially read everything in and then filter later? If so you could read the Seurat object in like normal and then use &lt;code&gt;subset&lt;/code&gt; against a vector of your gene names.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;subsample &amp;lt;- subset(seurat_obj, features = my_genes)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="252" LastEditorUserId="252" LastEditDate="2019-11-15T15:08:36.317" LastActivityDate="2019-11-15T15:08:36.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10807" PostTypeId="1" AcceptedAnswerId="10818" CreationDate="2019-11-15T20:18:57.057" Score="0" ViewCount="1016" Body="&lt;p&gt;How to convert mzXML to mzML format (mass spectrometry files) preferably under Linux and/or Python?&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2019-11-17T20:38:21.230" Title="How to convert mzXML to mzML format (mass spectrometry files)?" Tags="|python|linux|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10808" PostTypeId="1" CreationDate="2019-11-16T09:50:28.107" Score="0" ViewCount="327" Body="&lt;p&gt;I would like to know how to convert a pileup file for an adna y chromosome to vcf with Hg19 alignment.&lt;/p&gt;&#xA;" OwnerUserId="6505" LastEditorUserId="96" LastEditDate="2019-11-18T16:01:39.530" LastActivityDate="2020-08-14T19:03:00.303" Title="How to convert a Pileup file to VCF format with Hg19 alignment" Tags="|samtools|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10810" PostTypeId="1" AcceptedAnswerId="10813" CreationDate="2019-11-16T16:39:14.930" Score="2" ViewCount="108" Body="&lt;p&gt;Recently I've done some analysis on human mitochondrial DNA and now I want to run the same analysis on only hypervariable regions of mitochondrial DNA to see whether the changes that I have observed are linked to only hypervariable regions. To do that, I'll need to make new fasta files from the mitochondrial DNA entries that I have and I want to know how to extract those. Any leads will be appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/MK248274.1?report=fasta&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link to one of the reference fasta files that I have&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2883" LastEditorUserId="3848" LastEditDate="2020-05-01T09:06:07.320" LastActivityDate="2020-05-01T09:06:07.320" Title="How to extract hypervariable region of mitochondrial dna from its fasta file?" Tags="|fasta|phylogenetics|database|phylogeny|genomics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10811" PostTypeId="1" AcceptedAnswerId="10817" CreationDate="2019-11-16T16:53:05.113" Score="1" ViewCount="6283" Body="&lt;p&gt;Seurat VlnPlots are most commonly used to visualize differences in any given gene expression across multiple clusters or cell types. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;VlnPlot(object = MouseCellAtlas, idents = c(&quot;T cell&quot;,&quot;Neutrophil&quot;,&quot;Erythroblast&quot;,&quot;Monocyte&quot;,&quot;Macrophage&quot;), features = 'Fzd9')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ppD8p.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ppD8p.png&quot; alt=&quot;Seurat VlnPlot for a single gene in multiple cluster idents&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But suppose I want to view the expression of several genes in just &lt;code&gt;idents = &quot;T cell&quot;&lt;/code&gt;. I can do this to generate three separate plots:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;VlnPlot(object = mca, idents = 'T cell', features = c('Fzd9','Ctnnb1','Apc'), combine = TRUE, ncol = 3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mMivT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mMivT.png&quot; alt=&quot;Seurat plot for multiple genes in a single cluster ident&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But ideally I'd like this second plot to look just like the first plot, except with only a single tissue on the legend (&quot;T cell&quot;) and multiple genes in the various colored violins. Unfortunately, I am not aware of any simple way to combine these violins into a single plot area.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This doesn't do what I need:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;VlnPlot(object = mca, idents = 'T cell', features = c('Fzd9','Ctnnb1','Apc'), combine = TRUE, ncol = 3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there built-in functionality in Seurat for generating a VlnPlot for multiple genes in a single tissue in a single plot? Or can anyone suggest a workaround?&lt;/p&gt;&#xA;" OwnerUserId="6498" LastEditorUserId="3694" LastEditDate="2023-09-07T11:50:03.570" LastActivityDate="2023-09-07T11:50:03.570" Title="Seurat VlnPlot presenting expression of multiple genes in a single cluster" Tags="|r|scrnaseq|seurat|ggplot2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10812" PostTypeId="1" CreationDate="2019-11-16T17:38:16.963" Score="1" ViewCount="277" Body="&lt;p&gt;I am interested in using RDP classifier and RDPTools at the command line.  In following the docs from RDP, I ran into &lt;a href=&quot;https://github.com/rdpstaff/RDPTools/issues/22&quot; rel=&quot;nofollow noreferrer&quot;&gt;this issue&lt;/a&gt; during the build, after running &lt;code&gt;make&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BUILD FAILED&#xA;/home/denir/bin/RDPTools/Clustering/nbproject/build-impl.xml:896: The following error occurred while executing this line:&#xA;/home/denir/bin/RDPTools/Clustering/nbproject/build-impl.xml:1470: The following error occurred while executing this line:&#xA;/home/denir/bin/RDPTools/AlignmentTools/nbproject/build-impl.xml:896: The following error occurred while executing this line:&#xA;/home/denir/bin/RDPTools/AlignmentTools/nbproject/build-impl.xml:1414: The following error occurred while executing this line:&#xA;/home/denir/bin/RDPTools/ReadSeq/nbproject/build-impl.xml:920: The following error occurred while executing this line:&#xA;/home/denir/bin/RDPTools/ReadSeq/nbproject/build-impl.xml:300: Compile failed; see the compiler error output for details.&#xA;&#xA;Total time: 1 second&#xA;Makefile:15: recipe for target 'Clustering/dist/Clustering.jar' failed&#xA;make: *** [Clustering/dist/Clustering.jar] Error 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As one user suggested, I have tried downgrading from ant 1.10 to ant 1.9, this didn't work for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would rather not downgrade to JDK 7 as another user suggested if possible so I am looking for alternatives. One user was able to install rdp classifier with bioconda. I was able to do that OK. However, I'm not sure where to go from here? The tutorial provided with commands like: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -Xmx2g -jar ~/RDPTools/Clustering.jar derep -m '#=GC_RF' -o derep.fa all_seqs.ids all_seqs.samples alignment/*.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Depend on the build created using the install method recommended by RDP.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone else use rdp classifier installed via conda?  Is there a quickstart guide out there that I could refer to?  Thanks so much for your help!&lt;/p&gt;&#xA;" OwnerUserId="6507" LastEditorUserId="298" LastEditDate="2019-11-16T17:58:29.770" LastActivityDate="2019-11-16T19:18:35.707" Title="How to Installing &amp; Using RDP Classifier through bioconda?" Tags="|bioconda|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10813" PostTypeId="2" ParentId="10810" CreationDate="2019-11-16T17:53:56.977" Score="2" Body="&lt;p&gt;You can use &lt;a href=&quot;https://www.mitomap.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mitomap&lt;/a&gt; to get the coordinates of the hypervariable regions in the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/251831106&quot; rel=&quot;nofollow noreferrer&quot;&gt;rCRS mitochondrial sequence&lt;/a&gt; (the standard MT sequence in the field). I checked the &lt;a href=&quot;https://www.mitomap.org/foswiki/bin/view/MITOMAP/GenomeLoci&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genome Loci&lt;/a&gt; section of Mitomap and saw that it lists three:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Map Locus   Starting    Ending  Shorthand   Description&#xA;MT-HV1      16024       16383   CR:HVS1/HV1 Hypervariable segment 1 &#xA;                                            [classic:16024-16365]&#xA;MT-HV2      57          372     CR:HVS2     Hypervariable segment 2&#xA;                                            [classic:73-340 ]   &#xA;MT-HV3      438         574     CR:HVS3     Hypervariable segment 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That gives you the coordinates on the rCRS sequence. What you can do now, is align all of your sequences and the rCRS together and see if you can infer where the hypervariable regions fall in your target sequences. Since, presumably, the rest of the sequence will be more conserved since it isn't hypervariable, you should be able to identify the syntenic blocks and locate the hypervariable regions in your sequences. I would expect to see a very high level of sequence identity in the non-hypervariable regions across all of your sequences (since they're all human mitochondria), so you should be able to find the hypervariable ones as the regions of low sequence similarity which also coincide with the known coordinates of the rCRS.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-11-16T17:53:56.977" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10814" PostTypeId="2" ParentId="10812" CreationDate="2019-11-16T19:18:35.707" Score="1" Body="&lt;p&gt;The benefit of using the conda package is that you no longer need to know or care exactly where the jar files are (that many java-based packages require this is TERRIBLE design). The example &lt;code&gt;java -jar ...&lt;/code&gt; command in your post becomes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Clustering derep -m '#=GC_RF' -o derep.fa all_seqs.ids all_seqs.samples alignment/*.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;in the bioconda version. In bioconda packages, there are wrapper (usually python) scripts meant to provide somewhat friendlier interfaces to java programs that otherwise lack executables. In this case, RDP Classfier has the following executables in bioconda:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;AbundanceStats&lt;/li&gt;&#xA;&lt;li&gt;AlignmentTools&lt;/li&gt;&#xA;&lt;li&gt;Clustering&lt;/li&gt;&#xA;&lt;li&gt;FrameBot&lt;/li&gt;&#xA;&lt;li&gt;KmerFilter&lt;/li&gt;&#xA;&lt;li&gt;ProbeMatch&lt;/li&gt;&#xA;&lt;li&gt;ReadSeq&lt;/li&gt;&#xA;&lt;li&gt;SeqFilters&lt;/li&gt;&#xA;&lt;li&gt;SequenceMatch&lt;/li&gt;&#xA;&lt;li&gt;hmmgs&lt;/li&gt;&#xA;&lt;li&gt;classifier&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you really want to know, the .jar file is under &lt;code&gt;$PREFIX/share/rdptools-$PKG_VERSION-$PKG_BUILDNUM/&lt;/code&gt;, where &lt;code&gt;$PREFIX&lt;/code&gt; is where ever the environment is and the other two variables indicate the package version and build number (at the moment, &lt;code&gt;rdptools-2.0.2-1&lt;/code&gt; would be the likely folder name).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-11-16T19:18:35.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10815" PostTypeId="2" ParentId="10805" CreationDate="2019-11-16T22:26:55.287" Score="1" Body="&lt;p&gt;&lt;strong&gt;Minute values.&lt;/strong&gt;&#xA;So just a reminder that the ∆∆G units is a kcal/mol. Water bounces around and collides with different energies following a Maxwell-Boltzmann distribution, where the average collision at 25°C is 0.59 kcal/mol (thermal energy constant).&#xA;A hydrogen bond reduced the ∆G by 1–3 kcal/mol. Other interactions such as &amp;pi;-&amp;pi;, &amp;pi;-S are on the 1–2 mark. So the difference you are seeing is totally negligible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Heuristic.&lt;/strong&gt;&#xA;The reason why you see the differences is that the force field calculations are heuristic and are not testing all perturbations —it is not only sampling a limited amount of conformers of side chains (kind of like PyMOL would be if it were FF based), but neighbour perturbations. So sometimes you hit a wee energy minima. Hence why in some protocols replicates are required —not mutation scanning mind you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Bonus: Protein pI.&lt;/strong&gt;&#xA;Lastly, a caveat about pI of protein, you probably already know this, but best be safe. Your scores are all low, so I am guessing these surface mutations? Improving thermal stability is great for increasing the T_M measured by DSC or catalysis if you are scanning a transition state as done for theozyme design, but it does not mean you get better solubility. In this case, you are introducing positive charges, which will alter the pI of the protein upwards and most protein have a pI less than 7 and the closer it is to pH 7 the more they crash out. If you are designing a high pI GFP, &lt;code&gt;Rosetta supercharge&lt;/code&gt; does FF calculations to alter the pI of protein, which might be worth a gander.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2019-11-20T08:48:23.827" LastActivityDate="2019-11-20T08:48:23.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10816" PostTypeId="2" ParentId="10755" CreationDate="2019-11-17T09:11:44.730" Score="3" Body="&lt;p&gt;Yes, RDKit can be used, however, if you installed it with &lt;code&gt;conda&lt;/code&gt; it will not work out of the box for inChi key fetching. You can either spend some time installing the missing bit or use something else. It is not like Mol2 support in RDKit, which shouldn't be used, it is just not worth the bother.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest &lt;a href=&quot;https://pubchemdocs.ncbi.nlm.nih.gov/pug-rest&quot; rel=&quot;nofollow noreferrer&quot;&gt;PubChem's RESTFul API&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; import requests&#xA; ethanol = 'LFQSCWFLJHTTHZ-UHFFFAOYSA-N'&#xA; r = requests.get(f'https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/inchikey/{ethanol}/property/CanonicalSMILES/JSON').json()&#xA; smiles = r['PropertyTable']['Properties'][0]['CanonicalSMILES']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2019-11-20T08:42:32.597" LastActivityDate="2019-11-20T08:42:32.597" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10817" PostTypeId="2" ParentId="10811" CreationDate="2019-11-17T10:22:25.797" Score="2" Body="&lt;p&gt;To do so one workaround it to have your data in &quot;long format&quot; and then use the column that holds the &quot;gene names&quot; as the &lt;code&gt;x&lt;/code&gt; variable while plotting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use &lt;code&gt;FetchData()&lt;/code&gt; to extract data from a &lt;code&gt;Seurat&lt;/code&gt; object. &lt;code&gt;VlnPlot&lt;/code&gt;'s default is the &lt;code&gt;data&lt;/code&gt; slot (of the active assay if using Seurat v3 I suppose). And you can specify which cells and genes to retrieve.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;selected_cells &amp;lt;- names(panc8&lt;span class=&quot;math-container&quot;&gt;$celltype[panc8$&lt;/span&gt;celltype == &quot;gamma&quot;])&#xA;data &amp;lt;- FetchData(panc8,&#xA;                    vars = c('FZD9','CTNNB1','APC'),&#xA;                    cells = selected_cells ,&#xA;                    slot = &quot;data&quot;)&#xA;&amp;gt; head(data)&#xA;           FZD9   CTNNB1      APC&#xA;D101_5        0 0.000000 0.000000&#xA;D101_43       0 2.007853 1.001958&#xA;D101_93       0 0.000000 0.000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Melt()&lt;/code&gt; would transform your data into &quot;long format&quot;. By not specifying any arguments, we push for all of the info in the three variables to be gathered in two columns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;long_data &amp;lt;- melt(data)&#xA;No id variables; using all as measure variables&#xA;&amp;gt; head(long_data)&#xA;  variable value&#xA;1     FZD9     0&#xA;2     FZD9     0&#xA;3     FZD9     0&#xA;&#xA;&amp;gt; tail(long_data)&#xA;     variable value&#xA;1870      APC     0&#xA;1871      APC     0&#xA;1872      APC     0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;ggplot2&lt;/code&gt; is used to add &quot;violin&quot; and &quot;jitter&quot; layers. You can customize the output to look (exactly) like the &lt;code&gt;VlnPlot()&lt;/code&gt; output.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(long_data,&#xA;       aes(x = variable, y = value)) +&#xA;  geom_violin() +&#xA;  geom_jitter(size = 0.1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kDnuf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kDnuf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And here is the same graph generated by &lt;code&gt;VlnPlot()&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3iZyv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3iZyv.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are no &quot;violins&quot; as the counts are almost entirely zeros. And see how different the ranges are in the y axes of the &lt;code&gt;VlnPlot&lt;/code&gt;. So the &quot;tweak&quot; I have presented here would only work for genes that are expressed at similar levels / similar ranges. &lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-11-17T14:51:59.400" LastActivityDate="2019-11-17T14:51:59.400" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10818" PostTypeId="2" ParentId="10807" CreationDate="2019-11-17T12:14:13.250" Score="4" Body="&lt;p&gt;&lt;a href=&quot;http://proteowizard.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;ProteoWizard&lt;/code&gt;&lt;/a&gt; offers this functionality:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;msconvert your_mzXML_file --mzML&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;FileConverter&lt;/code&gt; of &lt;code&gt;OpenMS&lt;/code&gt; (I believe using the integrated ProteoWizard) also does &lt;a href=&quot;https://abibuilder.informatik.uni-tuebingen.de/archive/openms/Documentation/release/latest/html/TOPP_FileConverter.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-11-17T12:22:16.613" LastActivityDate="2019-11-17T12:22:16.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10820" PostTypeId="1" CreationDate="2019-11-17T20:38:21.480" Score="0" ViewCount="239" Body="&lt;p&gt;I am using RStudio Cloud for running velocyto R package. So I have tried to import velocyto file from &lt;a href=&quot;http://pklab.med.harvard.edu/velocyto/mouseBM/SCG71.loom&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://pklab.med.harvard.edu/velocyto/mouseBM/SCG71.loom&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to upload that like so&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rsi7N.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rsi7N.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But saying &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; SCG71 &amp;lt;- read.table(&quot;/cloud/project/SCG71.loom&quot;, header=TRUE, quote=&quot;\&quot;&quot;)&#xA;Error in make.names(col.names, unique = TRUE) : &#xA;  invalid multibyte string at '&amp;lt;89&amp;gt;HDF'&#xA;In addition: There were 19 warnings (use warnings() to see them)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can you help me in very initial of velocyto R?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-11-17T21:21:03.853" LastActivityDate="2019-11-18T01:21:48.963" Title="Uploading files from local computer or web in rstudio cloud" Tags="|r|single-cell|rstudio|cloud|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10821" PostTypeId="2" ParentId="10820" CreationDate="2019-11-17T21:38:38.757" Score="1" Body="&lt;p&gt;This seems to be an encoding problem (certain characters not being recognized), see &lt;a href=&quot;https://stackoverflow.com/questions/14363085/invalid-multibyte-string-in-read-csv/29586283&quot;&gt;this&lt;/a&gt;, &lt;a href=&quot;https://stackoverflow.com/questions/4993837/r-invalid-multibyte-string&quot;&gt;this&lt;/a&gt; and also &lt;a href=&quot;https://www.r-bloggers.com/web-scraping-and-invalid-multibyte-string/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. Probably the file &lt;code&gt;&quot;/cloud/project/SCG71.loom&quot;&lt;/code&gt; is encoded with a format that is not included in your &quot;locale&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check the output of &lt;code&gt;Sys.getlocale()&lt;/code&gt; and see if your input file's encoding is listed. If not, you will need to try something like &lt;code&gt;Sys.setlocale(&quot;LC_CTYPE&quot;, &quot;en_US.UTF-8&quot;)&lt;/code&gt; or &lt;code&gt;Sys.setlocale(&quot;LC_ALL&quot;, &quot;English&quot;)&lt;/code&gt; (see the third link for details).&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-11-17T21:38:38.757" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10822" PostTypeId="2" ParentId="10820" CreationDate="2019-11-18T01:21:48.963" Score="1" Body="&lt;p&gt;&lt;strong&gt;Velocyto.R&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure that your issues is from RStudio cloud but more about how you are trying to open the file. The file that you are trying to import is not a &lt;code&gt;velocyto&lt;/code&gt; file but a &lt;code&gt;.loom&lt;/code&gt; file that can be open using the &lt;code&gt;velocyto.R&lt;/code&gt; package. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not very familiar with this kind of of &lt;code&gt;.loom&lt;/code&gt; file but on the &lt;a href=&quot;http://pklab.med.harvard.edu/velocyto/notebooks/R/DG1.nb.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;velocyto.R&lt;/code&gt;&lt;/a&gt; tutorial, it is specified to do the following: &lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(velocyto.R)&#xA;ldat &amp;lt;- read.loom.matrices(&quot;SCG71.loom&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And you have the following file structures:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; str(ldat)&#xA;List of 3&#xA; $ spliced  :Formal class 'dgCMatrix' [package &quot;Matrix&quot;] with 6 slots&#xA;  .. ..@ i       : int [1:2822201] 74 130 178 192 202 223 225 258 353 418 ...&#xA;  .. ..@ p       : int [1:6668] 0 464 786 1469 2502 3290 3760 4974 5388 5659 ...&#xA;  .. ..@ Dim     : int [1:2] 24421 6667&#xA;  .. ..@ Dimnames:List of 2&#xA;  .. .. ..$ : chr [1:24421] &quot;Xkr4&quot; &quot;Rp1&quot; &quot;Sox17&quot; &quot;Mrpl15&quot; ...&#xA;  .. .. ..$ : chr [1:6667] &quot;SCG71:AAACGATCCTACTTGT&quot; &quot;SCG71:AAACACTACGTAGAAG&quot; &quot;SCG71:AAACACGGCATCGCAG&quot; &quot;SCG71:AAACAAACAGCAGAAC&quot; ...&#xA;  .. ..@ x       : num [1:2822201] 1 1 1 1 1 1 1 1 1 4 ...&#xA;  .. ..@ factors : list()&#xA; $ unspliced:Formal class 'dgCMatrix' [package &quot;Matrix&quot;] with 6 slots&#xA;  .. ..@ i       : int [1:528048] 61 100 339 352 377 445 465 528 617 758 ...&#xA;  .. ..@ p       : int [1:6668] 0 115 188 283 431 580 696 994 1127 1232 ...&#xA;  .. ..@ Dim     : int [1:2] 24421 6667&#xA;  .. ..@ Dimnames:List of 2&#xA;  .. .. ..$ : chr [1:24421] &quot;Xkr4&quot; &quot;Rp1&quot; &quot;Sox17&quot; &quot;Mrpl15&quot; ...&#xA;  .. .. ..$ : chr [1:6667] &quot;SCG71:AAACGATCCTACTTGT&quot; &quot;SCG71:AAACACTACGTAGAAG&quot; &quot;SCG71:AAACACGGCATCGCAG&quot; &quot;SCG71:AAACAAACAGCAGAAC&quot; ...&#xA;  .. ..@ x       : num [1:528048] 1 3 1 1 1 1 1 1 1 1 ...&#xA;  .. ..@ factors : list()&#xA; $ ambiguous:Formal class 'dgCMatrix' [package &quot;Matrix&quot;] with 6 slots&#xA;  .. ..@ i       : int [1:47846] 11915 13649 13830 15638 18068 19585 23490 24345 8142 8767 ...&#xA;  .. ..@ p       : int [1:6668] 0 8 13 25 42 56 64 87 97 100 ...&#xA;  .. ..@ Dim     : int [1:2] 24421 6667&#xA;  .. ..@ Dimnames:List of 2&#xA;  .. .. ..$ : chr [1:24421] &quot;Xkr4&quot; &quot;Rp1&quot; &quot;Sox17&quot; &quot;Mrpl15&quot; ...&#xA;  .. .. ..$ : chr [1:6667] &quot;SCG71:AAACGATCCTACTTGT&quot; &quot;SCG71:AAACACTACGTAGAAG&quot; &quot;SCG71:AAACACGGCATCGCAG&quot; &quot;SCG71:AAACAAACAGCAGAAC&quot; ...&#xA;  .. ..@ x       : num [1:47846] 2 1 1 1 1 2 1 2 1 1 ...&#xA;  .. ..@ factors : list()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Alternative 1 - LoomExperiment&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On bioconductor, you can also find &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/LoomExperiment/inst/doc/LoomExperiment.html#available-methods-for-the-loomexperiment&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;LoomExperiment&lt;/code&gt;&lt;/a&gt;. On my computer, once I get it install, I was able to load your &lt;code&gt;SCG71.loom&lt;/code&gt; file using:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(LoomExperiment)&#xA;scle &amp;lt;- import(&quot;SCG71.loom&quot;,type = &quot;LoomExperiment&quot;, rownames_attr = NULL, colnames_attr = NULL)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, you can find the output here:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; head(scle)&#xA;class: LoomExperiment &#xA;dim: 6 6667 &#xA;metadata(4): chunks creation_date velocyto.__version__ velocyto.logic&#xA;assays(4): matrix ambiguous spliced unspliced&#xA;rownames: NULL&#xA;rowData names(6): Accession Chromosome ... Start Strand&#xA;colnames: NULL&#xA;colData names(1): CellID&#xA;rowGraphs(0): NULL&#xA;colGraphs(0): NULL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can then load some data&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; scle$CellID[c(1:5)]&#xA;[1] &quot;SCG71:AAACGATCCTACTTGT&quot; &quot;SCG71:AAACACTACGTAGAAG&quot; &quot;SCG71:AAACACGGCATCGCAG&quot;&#xA;[4] &quot;SCG71:AAACAAACAGCAGAAC&quot; &quot;SCG71:AAACAAACTTTGGGAG&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After that, I don't know what to do with that but I think your issue of importing a datafile is because you don't have the right package to open it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Alternative - using LoomR&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you can also install the package &lt;a href=&quot;https://satijalab.org/loomR/loomR_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;loomR&lt;/code&gt;&lt;/a&gt; and then enter:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(loomR)&#xA;lfile = connect(filename = &quot;SCG71.loom&quot;, mode = &quot;r+&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And you get the output:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; lfile&#xA;Class: loom&#xA;Filename: /***my_directory***/SCG71.loom&#xA;Access type: H5F_ACC_RDWR&#xA;Attributes: creation_date, chunks, velocyto.__version__, velocyto.logic, version&#xA;Listing:&#xA;       name    obj_type dataset.dims dataset.type_class&#xA;  col_attrs   H5I_GROUP         &amp;lt;NA&amp;gt;               &amp;lt;NA&amp;gt;&#xA; col_graphs   H5I_GROUP         &amp;lt;NA&amp;gt;               &amp;lt;NA&amp;gt;&#xA;     layers   H5I_GROUP         &amp;lt;NA&amp;gt;               &amp;lt;NA&amp;gt;&#xA;     matrix H5I_DATASET 6667 x 24421          H5T_FLOAT&#xA;  row_attrs   H5I_GROUP         &amp;lt;NA&amp;gt;               &amp;lt;NA&amp;gt;&#xA; row_graphs   H5I_GROUP         &amp;lt;NA&amp;gt;               &amp;lt;NA&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, for manipulating these files, you should refer to their tutorial (it's beyond my skills and the scope of your question).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps you to figure the solution of your question&lt;/p&gt;&#xA;" OwnerUserId="6437" LastActivityDate="2019-11-18T01:21:48.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10823" PostTypeId="1" AcceptedAnswerId="10824" CreationDate="2019-11-18T03:50:49.400" Score="0" ViewCount="48" Body="&lt;p&gt;When I enter a list of gene names into common gene ontology (GO) term enricher (e.g. MsigDB or ToppGene) only a small fraction of genes is actually used for the enrichment:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4gnnA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4gnnA.png&quot; alt=&quot;msigdb gene restriction&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I am struggling to understand why precisely only such a small number of genes is mapped here.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My dataset is mapped with the GENCODE annotation. What annotation is the base of gene ontology? (The picture mentions entrez genes)&lt;/p&gt;&#xA;" OwnerUserId="3575" LastActivityDate="2019-11-18T08:32:30.330" Title="Why are my genes filtered for Gene Ontology term enrichment?" Tags="|gene-ontology|functional-annotation|go-enrichment|sequence-annotation|gencode|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10824" PostTypeId="2" ParentId="10823" CreationDate="2019-11-18T08:32:30.330" Score="1" Body="&lt;p&gt;What is &quot;click here for details&quot; saying? That link should contain an explanation of what happened.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;GSEA usually requires &lt;code&gt;entrez_id&lt;/code&gt; to run. If your list contains &lt;code&gt;gene_symbols&lt;/code&gt;, then before enrichment, you need to convert the genes names. This can and is (always) tricky as genes are known with different names. For example, by googling I couldn't find any &lt;code&gt;AASS3&lt;/code&gt; gene, but I found &lt;code&gt;AASS&lt;/code&gt;, which is also named &lt;code&gt;LKRSDH&lt;/code&gt;,&lt;code&gt;LORSDH&lt;/code&gt;, and &lt;code&gt;LKR/SDH&lt;/code&gt;. Depending on the software you are using for conversion, and the genome version used some genes will be missed. I usually run (in R) a combination of &lt;code&gt;geneSynonym&lt;/code&gt; and &lt;code&gt;biomaRt&lt;/code&gt; to fill the gaps in the lists.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Even better, would be to use &lt;code&gt;ensembl_id&lt;/code&gt; or even better &lt;code&gt;entrez_id&lt;/code&gt; directly&lt;/p&gt;&#xA;" OwnerUserId="3631" LastActivityDate="2019-11-18T08:32:30.330" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10825" PostTypeId="1" CreationDate="2019-11-18T13:07:19.683" Score="3" ViewCount="168" Body="&lt;p&gt;I am a newbie in this world (just finished my master's on bioinformatics. As a practice, a researcher gave me 4 fastq files from a small RNAseq experiment, to see if I am able to reproduce their results but with an updated pipeline and tools (they used bowtie in 2014). &#xA;I used fastqc for the quality check and I removed the adapters properly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first intention was to use Salmon for pseudo-align them, but after do some reading it seems not to be a good idea for such small RNAs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone provide me with some light on this? Which alignment tool use and which parameters, and etc. Ideally a proper protocol/pipeline would be amazing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PD: I was thinking to use the HISAT2 with the full genome (or the pre-builded index from HISAT2 website). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kind regards, &#xA;and thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="6515" LastActivityDate="2021-10-11T09:31:39.667" Title="Pipeline to analyse small RNAseq and tRNAs?" Tags="|trna|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10826" PostTypeId="2" ParentId="10808" CreationDate="2019-11-18T16:20:47.083" Score="2" Body="&lt;h2&gt;Short answer&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;You'll have better luck searching for the term &lt;em&gt;&lt;strong&gt;variant calling&lt;/strong&gt;&lt;/em&gt; than with the term &lt;em&gt;&lt;strong&gt;converting&lt;/strong&gt;&lt;/em&gt;. A Google search for &quot;variant calling&quot; gives several beginner-level tutorials on the topic. Commonly used tools for variant calling include &lt;a href=&quot;http://www.htslib.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools&lt;/a&gt;, &lt;a href=&quot;https://software.broadinstitute.org/gatk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATK&lt;/a&gt;, and &lt;a href=&quot;https://github.com/ekg/freebayes&quot; rel=&quot;nofollow noreferrer&quot;&gt;FreeBayes&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Long answer&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Converting data files from one format to another is a common task in bioinformatics. But that is not the task you want to perform here—the term &lt;em&gt;&lt;strong&gt;convert&lt;/strong&gt;&lt;/em&gt; is misleading.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A pileup file contains reads aligned to a reference sequence. A VCF file contains a description of putative or validated genetic variant. The process of creating a VCF from alignments is commonly called &lt;em&gt;variant discovery&lt;/em&gt;, &lt;em&gt;variant prediction&lt;/em&gt;, or &lt;em&gt;variant calling&lt;/em&gt;. This is a broad and active area of research interest in bioinformatics, with many available algorithms and tools, each with a different set of parameters and assumptions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This forum is not well designed to give you a good introduction to this topic. We are much better equipped to answer specific questions you have about how a particular tool or algorithm works, or why you are observing a particular unexpected result, and so on...assuming you have put in some time to &lt;a href=&quot;https://bioinformatics.stackexchange.com/help/how-to-ask&quot;&gt;research the question yourself&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2019-11-18T16:20:47.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10827" PostTypeId="2" ParentId="10738" CreationDate="2019-11-18T18:07:24.520" Score="2" Body="&lt;p&gt;Did you try to use &lt;code&gt;DotPlot(..., scale.by = &amp;quot;size&amp;quot;)&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;In contrast to the default &lt;code&gt;scale.by= &amp;quot;radius&amp;quot;&lt;/code&gt;, this will link the &lt;em&gt;area&lt;/em&gt; (&lt;code&gt;==2*pi*r^2&lt;/code&gt;), not the radius, of the circles to the fraction of cells expressing the feature.&#xA;This corresponds much better to our perception of size and will make differences in low values easier to see.&lt;/p&gt;&#xA;&lt;p&gt;Reading &lt;code&gt;?Seurat::DotPlot&lt;/code&gt; the &lt;code&gt;scale.min&lt;/code&gt; parameter looked promising but looking at &lt;a href=&quot;https://github.com/satijalab/seurat/blob/c9f2660ee6b88189acc3ff4aa6088a46a2558dd3/R/visualization.R#L1980&quot; rel=&quot;nofollow noreferrer&quot;&gt;the code&lt;/a&gt; it seems to censor the data as well.&lt;/p&gt;&#xA;&lt;p&gt;Since Seurat's plotting functionality is based on &lt;code&gt;ggplot2&lt;/code&gt; you can also adjust the color scale by simply adding &lt;code&gt;scale_fill_viridis()&lt;/code&gt; etc. to the returned plot. This might also work for &lt;code&gt;size&lt;/code&gt;. Try something like:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;DotPlot(...) + scale_size(range = c(5, 10)) # will likely warn about supplying the same scale twice&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Apart from this, Seurat's plotting system is not very hackable and I find it much easier to extract the relevant data and plot them myself with &lt;code&gt;ggplot2&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Yet another comment: Your plot with the strong differences looks much more convincing to me wrt. to the marker property of these genes than the cited plot.&lt;/p&gt;&#xA;" OwnerUserId="3773" LastEditorUserId="3773" LastEditDate="2021-11-18T07:35:18.393" LastActivityDate="2021-11-18T07:35:18.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10828" PostTypeId="1" AcceptedAnswerId="10829" CreationDate="2019-11-18T22:00:10.563" Score="2" ViewCount="169" Body="&lt;p&gt;I have a bunch of &lt;code&gt;.vcf&lt;/code&gt; files in a folder and I want to run command below on all of them but doing that one by one manually is really painful. I am seeking for a way in terminal to do that simultaneously. The output would be a &lt;code&gt;.txt&lt;/code&gt; file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -v &quot;##&quot; /file.vcf | awk '{print &lt;span class=&quot;math-container&quot;&gt;$1&quot;\t&quot;$&lt;/span&gt;2&quot;\t&quot;&lt;span class=&quot;math-container&quot;&gt;$2&quot;\t&quot;$&lt;/span&gt;4&quot;\t&quot;&lt;span class=&quot;math-container&quot;&gt;$5&quot;\t&quot;$&lt;/span&gt;6&quot;\t&quot;&lt;span class=&quot;math-container&quot;&gt;$7&quot;\t&quot;$&lt;/span&gt;8&quot;\t&quot;&lt;span class=&quot;math-container&quot;&gt;$9&quot;\t&quot;$&lt;/span&gt;10&quot;\t&quot;$11}' &amp;gt; /file.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can you help please?&lt;/p&gt;&#xA;" OwnerDisplayName="user6517" LastEditorUserId="131" LastEditDate="2020-01-08T11:19:32.487" LastActivityDate="2020-01-08T11:19:32.487" Title="How can I run a command for multiple files?" Tags="|bash|linux|loop|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10829" PostTypeId="2" ParentId="10828" CreationDate="2019-11-18T22:06:45.583" Score="5" Body="&lt;p&gt;Assuming all your vcfs are in the same folder:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for file in *.vcf; do&#xA;    grep -v &quot;##&quot; &lt;span class=&quot;math-container&quot;&gt;${file} | awk '{print $1&quot;\t&quot;$2&quot;\t&quot;$2&quot;\t&quot;$4&quot;\t&quot;$5&quot;\t&quot;$6&quot;\t&quot;$7&quot;\t&quot;$8&quot;\t&quot;$9&quot;\t&quot;$10&quot;\t&quot;$11}' &amp;gt; $&lt;/span&gt;{file}.txt&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="73" LastEditDate="2019-11-19T00:07:15.627" LastActivityDate="2019-11-19T00:07:15.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10831" PostTypeId="1" CreationDate="2019-11-18T22:41:36.887" Score="0" ViewCount="133" Body="&lt;p&gt;I am annotating some .txt files by Annovar software by this code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nnovar]$ module load annovar/2016Feb01&#xA;&#xA;[cyan01 annovar]$ table_annovar.pl &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I really got confused to do that by &lt;code&gt;awk&lt;/code&gt; for all of my files in the path; Can somebody help me?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many thanks&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;EDITED&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I found this but not working &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;module load annovar/20&#xA;&#xA;for file in *.vcf; do&#xA;    table_annovar.pl example/a.txt humandb/ -buildver hg19 -out file -remove -protocol refGene,cytoBand,exac03,avsnp147,dbnsfp30a -operation gx,r,f,f,f &quot;&lt;span class=&quot;math-container&quot;&gt;$f&quot; &amp;gt; &quot;$&lt;/span&gt;{f%.*}.txt&quot;&#xA;done;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user6517" LastEditorDisplayName="user6517" LastEditDate="2020-03-21T09:37:40.410" LastActivityDate="2020-03-21T09:50:00.143" Title="How I can run this code on my files?" Tags="|bash|linux|shell|looper|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10832" PostTypeId="2" ParentId="10831" CreationDate="2019-11-19T00:05:30.240" Score="1" Body="&lt;p&gt;The basic outline for loops is &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;for sample in *; do mycommand &quot;$sample&quot; ; done&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might need some way of incorporating the name of &lt;code&gt;$sample&lt;/code&gt; into the output file name, so you don't keep overwriting the same output file over and over again (or do &lt;code&gt;mkdir&lt;/code&gt; with each sample name and write everything for each sample in its own folder).&lt;/p&gt;&#xA;" OwnerUserId="2388" LastEditorUserId="298" LastEditDate="2019-11-19T15:57:08.277" LastActivityDate="2019-11-19T15:57:08.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10833" PostTypeId="2" ParentId="10825" CreationDate="2019-11-19T07:34:22.610" Score="1" Body="&lt;p&gt;I have heard that the &lt;a href=&quot;http://srna-workbench.cmp.uea.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Small RNA Workbench&lt;/a&gt; from UEA can be an easy solution for a full small RNA pipeline.  It is java based, has a GUI, and is supposed to be easy to install and run.  I am not an expert in small RNA analysis, but I think this pipeline will certainly given you an idea about the steps needed for a small RNA analysis, it has been put together by experts.&lt;/p&gt;&#xA;" OwnerUserId="3828" LastActivityDate="2019-11-19T07:34:22.610" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10834" PostTypeId="2" ParentId="6750" CreationDate="2019-11-19T08:08:37.110" Score="1" Body="&lt;p&gt;Denote the strips in your permutation as &lt;span class=&quot;math-container&quot;&gt;$\langle S_1, S_2, \cdots, S_n \rangle$&lt;/span&gt;. Suppose &lt;span class=&quot;math-container&quot;&gt;$k-1$&lt;/span&gt; is in segment &lt;span class=&quot;math-container&quot;&gt;$S_i$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is in segment &lt;span class=&quot;math-container&quot;&gt;$S_j$&lt;/span&gt;, for some &lt;span class=&quot;math-container&quot;&gt;$i, j \in [n], i \neq j$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recall by definition of &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; that &lt;span class=&quot;math-container&quot;&gt;$k-1$&lt;/span&gt; must be the maximum and therefore rightmost element in an increasing strip, and &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; the minimum and therefore also rightmost in a decreasing strip. Consider the two cases:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$i &amp;lt; j$&lt;/span&gt;. The increasing strip comes before the decreasing. &lt;span class=&quot;math-container&quot;&gt;$k-1$&lt;/span&gt; is on the 'right' end of &lt;span class=&quot;math-container&quot;&gt;$S_i$&lt;/span&gt;, but &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is on the 'right' end of &lt;span class=&quot;math-container&quot;&gt;$S_j$&lt;/span&gt; also. In order for &lt;span class=&quot;math-container&quot;&gt;$k-1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; to be adjacent, reverse the segments &lt;span class=&quot;math-container&quot;&gt;$S_{i+1}, \cdots, S_j$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$j &amp;lt; i$&lt;/span&gt;. The decreasing strip comes before the increasing. &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is on the 'right' end of &lt;span class=&quot;math-container&quot;&gt;$S_j$&lt;/span&gt;, but &lt;span class=&quot;math-container&quot;&gt;$k-1$&lt;/span&gt; is on the 'right' end of &lt;span class=&quot;math-container&quot;&gt;$S_i$&lt;/span&gt; also. Again, we reverse &lt;span class=&quot;math-container&quot;&gt;$S_{i+1}, \cdots, S_j$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;So, &quot;reversing the segment between &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$k-1$&lt;/span&gt;&quot; means to reverse all the strips in-between the two strips containing &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$k-1$&lt;/span&gt;, excluding the 'leftmost' strip, but including the 'rightmost' strip.&lt;/p&gt;&#xA;" OwnerUserId="6520" LastActivityDate="2019-11-19T08:08:37.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10835" PostTypeId="2" ParentId="10800" CreationDate="2019-11-19T08:25:59.550" Score="2" Body="&lt;p&gt;A different approach if you are using &lt;code&gt;Seurat3&lt;/code&gt;, is &lt;code&gt;DietSeurat()&lt;/code&gt;. It allows you to diet the object by removing the components that you don't need. For example you can keep the normalised/scaled matrix and remove the raw counts. This approach could reduce space and memory usage, while keeping all your genes in place.&lt;/p&gt;&#xA;" OwnerUserId="3631" LastActivityDate="2019-11-19T08:25:59.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10836" PostTypeId="2" ParentId="10801" CreationDate="2019-11-19T10:05:13.327" Score="2" Body="&lt;p&gt;Depending on the search parameter settings RAxML produces several output files among which you normally find:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;RAxML_bipartitions.result&lt;/strong&gt;: If you used the ­-f b option, this file&#xA;  will contain the input tree with confidence values from 0 to 100 drawn&#xA;  on its nodes! It is also printed when -f a ­x have been specified, at&#xA;  the end of the analysis the program will draw the BS support values on&#xA;  the best tree found during the ML search.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&lt;strong&gt;RAxML_bipartitionsBranchLabels.result&lt;/strong&gt;: Contains the same information&#xA;  as the file above, but support values are correctly displayed as&#xA;  Newick branch labels and not node labels! Support values always refer&#xA;  to branches/splits of trees and never to nodes of the tree. Note that,&#xA;  some tree viewers have problems displaying branch labels, they are&#xA;  however part of the standard Newick format.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;from &lt;a href=&quot;https://cme.h-its.org/exelixis/resource/download/NewManual.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cme.h-its.org/exelixis/resource/download/NewManual.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't find the files in your output directory you did not specify the above parameters. If a rerun is not feasible you could you use the multinewick file RAxML_bootstrap.result with raxml or as input for the program splitstree to generate a consenus tree with different rules (majority rule, strict, etc.). However, these options don't guarantee to generate the topology of the maximum likelihood tree.&lt;/p&gt;&#xA;" OwnerUserId="257" LastActivityDate="2019-11-19T10:05:13.327" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10837" PostTypeId="1" AcceptedAnswerId="10838" CreationDate="2019-11-19T10:23:29.327" Score="1" ViewCount="460" Body="&lt;p&gt;I would like to calculate what is the number of region which is lost or gained between two condition,such as Stem cell vs Leukemic stemcell, atac seq data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I get the information after peak calling such as what tools should i be using ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As shown in the paper figure 1c I would like to do the same. How do i calculate the gain and loss of peak relative to a condition &#xA;&lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-015-0840-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2020-02-18T06:18:59.140" LastActivityDate="2020-02-18T06:18:59.140" Title="Loss and gain of region in ATAC seq" Tags="|atac-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10838" PostTypeId="2" ParentId="10837" CreationDate="2019-11-19T11:24:08.090" Score="3" Body="&lt;p&gt;What you are looking for is &quot;Differential accessibility analysis&quot;, you are using ATAC-seq, hence &quot;accessibility&quot;. You will need to &quot;call&quot; and then &lt;strike&gt;&quot;normalize&quot;&lt;/strike&gt; &quot;count&quot;(*) peaks, after these, you can use &lt;code&gt;DESeq2&lt;/code&gt; or &lt;code&gt;edgeR&lt;/code&gt; for the &quot;differential&quot; part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/nf-core/atacseq&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is a nice Nextflow workflow&lt;/a&gt; that addresses all of the steps of a differential accessibility analysis of ATAC-Seq data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;* &lt;code&gt;featureCounts&lt;/code&gt; can be used the identify peaks, see &lt;a href=&quot;https://support.bioconductor.org/p/103059/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://support.bioconductor.org/p/120013/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2020-02-17T19:23:12.040" LastActivityDate="2020-02-17T19:23:12.040" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="10839" PostTypeId="2" ParentId="10831" CreationDate="2019-11-19T11:56:27.860" Score="4" Body="&lt;p&gt;&lt;code&gt;$f&lt;/code&gt; in your command is not pointing anywhere since your for loop was defined as &lt;code&gt;for file in *.vcf;&lt;/code&gt;. You should use &lt;code&gt;$file&lt;/code&gt; instead of &lt;code&gt;$f&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT AS THE OP HAS EDITED THE QUESTION:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, please copy and paste the error messages you are getting when seeking answers from the community, most of the time the error messages are clear enough to point to the right direction.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In you case, the problem is that you are trying to use a variable that you did not declare in the first place. It is explained above in my original answer but here I go again:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With this bit &lt;code&gt;for file in *.vcf;&lt;/code&gt;, you are creating a variable &lt;code&gt;file&lt;/code&gt; that you will using throughout the iteration. But then within the iteration, the next line (&lt;code&gt;... -operation gx,r,f,f,f &quot;$f&quot; &amp;gt; ...&lt;/code&gt;, you try to use another variable &lt;code&gt;file&lt;/code&gt;, that has not been declared yet (as far as I can see from your code bit).&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2020-03-21T09:50:00.143" LastActivityDate="2020-03-21T09:50:00.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10840" PostTypeId="1" CreationDate="2019-11-19T14:39:11.210" Score="1" ViewCount="62" Body="&lt;p&gt;Im trying to write a script where an optional input of -X flag can be used, or if that info is not available download all reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;my script as follow:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# usage : bash dump_qc.sh &amp;lt;run id&amp;gt; &amp;lt;number of reads&amp;gt;&#xA;&#xA;#run id variable.&#xA;SRR=$1&#xA;&#xA;#maximum reads variable&#xA;X=$2&#xA;&#xA;fastq-dump --split-3 -X &lt;span class=&quot;math-container&quot;&gt;${X} $&lt;/span&gt;{SRR}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, if I run the script&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bash dump_qc.sh SRR1234 100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Will download 100 reads from the specified SRR file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I would like to do is to make the second argument optional, if not specified download the whole data set!&lt;/p&gt;&#xA;" OwnerUserId="6436" LastActivityDate="2019-11-19T15:45:29.380" Title="Fastq-dump script download X spots or all" Tags="|fastq|bash|shell|sratoolkit|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10841" PostTypeId="2" ParentId="10840" CreationDate="2019-11-19T15:45:29.380" Score="1" Body="&lt;p&gt;You don't need to copy your arguments into variables, you can just use &lt;code&gt;$1&lt;/code&gt; directly. Anyway, this should be enough:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;# usage : bash dump_qc.sh &amp;lt;run id&amp;gt; &amp;lt;number of reads&amp;gt;&#xA;&#xA;if [ -z $1 ]; then&#xA;  echo &quot;ERROR: at least one argument need to be given!&quot; &amp;gt;&amp;amp;2&#xA;  exit 1&#xA;fi&#xA;&#xA;if [ ! -z &lt;span class=&quot;math-container&quot;&gt;$2 ]; then&#xA;    x=&quot;-X '$&lt;/span&gt;2'&quot;&#xA;fi&#xA;&#xA;fastq-dump --split-3 &quot;&lt;span class=&quot;math-container&quot;&gt;$x&quot; &quot;$&lt;/span&gt;1&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-11-19T15:45:29.380" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10842" PostTypeId="1" CreationDate="2019-11-19T16:02:50.117" Score="0" ViewCount="2063" Body="&lt;p&gt;I am going to adjust Seurat dimplot in a way avoiding some cells so both my dimplot and heatmap look nice&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you please consider this picture, you would see some cells are far from the clusters so I want to avoid them in dimplot and of course for heatmap (coming from finding markers).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;[![enter image description here][1]][1]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you suggest me a piece of code doing that?&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-11-22T16:51:22.853" LastActivityDate="2020-08-18T18:00:25.043" Title="Putting cells in Seurat dimplot selectively" Tags="|r|seurat|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10844" PostTypeId="2" ParentId="10755" CreationDate="2019-11-20T00:14:46.267" Score="2" Body="&lt;p&gt;Slightly modified Matteos answer:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;def get_smiles_from_inchikey(inchikey):&#xA;    r = requests.get(f'https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/inchikey/{inchikey}/property/CanonicalSMILES/JSON').json()&#xA;    return r['PropertyTable']['Properties'][0]['CanonicalSMILES']&#xA;&#xA;inchikey = 'SGNXVBOIDPPRJJ-UHFFFAOYSA-N'&#xA;get_smiles_from_inchikey(inchikey)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2019-11-20T00:14:46.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10845" PostTypeId="1" CreationDate="2019-11-20T08:36:59.177" Score="1" ViewCount="578" Body="&lt;p&gt;I was thinking about the best method for normalization, which takes gene length into account (in order to compare genes)...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you think I can do that? :&#xA;- taking raw counts and dividing each gene by its length&#xA;- using the function rlog (DESeq2) on these counts divided by gene length (I would modify the rlog function to allow it to be used on decimal data).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if it would be ok to do that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much for your help&lt;/p&gt;&#xA;" OwnerUserId="6525" LastActivityDate="2021-03-09T12:06:00.517" Title="RNASeq: Normalization, stabilization, gene length and rlog" Tags="|rna-seq|deseq2|normalization|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10846" PostTypeId="2" ParentId="10845" CreationDate="2019-11-20T08:50:08.993" Score="1" Body="&lt;p&gt;I am not sure what you mean by &quot;decimal data&quot;, however, if you really want to use &quot;gene length&quot; information for normalization, take a look at &lt;code&gt;EDASeq&lt;/code&gt;. Unlike &lt;code&gt;DESeq&lt;/code&gt; or &lt;code&gt;EdgeR&lt;/code&gt;, its normalization step takes gene length into account to produce &quot;normalized counts&quot;, which you can feed into &lt;code&gt;DESeq&lt;/code&gt; or &lt;code&gt;EdgeR&lt;/code&gt; according to their &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/EDASeq/inst/doc/EDASeq.html#differential-expression-analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is from the said manual: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The normalized counts (or the original counts and the offset) obtained&#xA;  using the EDASeq package can be supplied to packages such as edgeR&#xA;  (Robinson, McCarthy, and Smyth 2010) or DESeq (Anders and Huber 2010)&#xA;  to find differentially expressed genes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-11-20T08:50:08.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10847" PostTypeId="1" CreationDate="2019-11-20T11:05:34.927" Score="1" ViewCount="23" Body="&lt;p&gt;Calculation of coverage involves mapping reads with a reference and depending on the choice of reference we can get very different result. If a species has a genome that has very low size variation then the choice of reference doesn't have big impact on the result. On the other hand, if the genome size variation is high then the choice of reference is critical to the accuracy of calculated coverage. My question is , how to choose reference in such cases? &lt;/p&gt;&#xA;" OwnerUserId="3555" LastActivityDate="2019-11-20T11:05:34.927" Title="How to choose reference to calculate coverage of WGS of organism that has high intra-species variation?" Tags="|read-mapping|wgs|coverage|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10848" PostTypeId="2" ParentId="10842" CreationDate="2019-11-20T15:00:06.993" Score="1" Body="&lt;p&gt;If you're using a GUI you could select the cells interactively:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plot &amp;lt;- DimPlot(seurat_obj, reduction = &quot;umap&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then select the cells by clicking around them&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;select.cells &amp;lt;- CellSelector(plot = plot)&#xA;Idents(seurat_obj, cells = select.cells) &amp;lt;- &quot;SubCells&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and subset based on these cells&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sub_cells &amp;lt;- WhichCells(seurat_obj, idents = &quot;SubCells&quot;)&#xA;sub_obj &amp;lt;- subset(seurat_obj, cells = sub_cells)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You could then use subobject in your downstream dimplot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://satijalab.org/seurat/v3.0/visualization_vignette.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for some of the interactive options. I guess you could outright subset on UMAP coordinates, but that may not always be easy if your boundary isn't linear.&lt;/p&gt;&#xA;" OwnerUserId="252" LastEditorUserId="252" LastEditDate="2019-11-20T16:19:55.570" LastActivityDate="2019-11-20T16:19:55.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10849" PostTypeId="1" CreationDate="2019-11-20T18:06:03.127" Score="0" ViewCount="268" Body="&lt;p&gt;I want to test for significance between differences in cell count between conditions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 2 conditions (N, T), 3 donors (samples) each, and four cell types (A-D) as shown in the example below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe this problem is analogous to the differential expression test performed with edgeR or DEseq.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Count   Sample  Celltype    Condition&#xA;555 P1  D   T&#xA;3829    P1  C   T&#xA;945 P1  A   T&#xA;222 P1  B   T&#xA;795 P2  D   T&#xA;5464    P2  C   T&#xA;839 P2  A   T&#xA;438 P2  B   T&#xA;629 P3  D   T&#xA;6426    P3  C   T&#xA;647 P3  A   T&#xA;457 P3  B   T&#xA;8   P1  D   N&#xA;5122    P1  C   N&#xA;3204    P1  A   N&#xA;1   P1  B   N&#xA;20  P2  D   N&#xA;6111    P2  C   N&#xA;2524    P2  A   N&#xA;26  P2  B   N&#xA;66  P3  D   N&#xA;7490    P3  C   N&#xA;5272    P3  A   N&#xA;3   P3  B   N&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="208" LastActivityDate="2019-11-20T18:15:51.140" Title="Calculating celltype proportion changes between conditions in single-cell data" Tags="|scrnaseq|single-cell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10850" PostTypeId="2" ParentId="10849" CreationDate="2019-11-20T18:15:51.140" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/s41586-018-0393-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;This publication&lt;/a&gt; used a negative binomial regression model of the counts of cells at each time-point.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the methods section: &lt;em&gt;&quot;For each cell type, we model the number of lineage-labelled cells detected in each analysed mouse as a random count variable using a negative binomial distribution. The frequency of detection is modelled by using the natural log of the total number of cells of that type profiled in a given mouse as an offset. The time point of each mouse (0, 30 or 60 days post tamoxifen) is provided as a covariate. The negative binomial model was fit using the R command ‘glm.nb’ from the ‘MASS’ package. The P value for the significance of the change in labelled fraction size between time-points was assessed using a likelihood-ratio test, computing using the R function ‘anova’.&quot;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The method is available &lt;a href=&quot;https://github.com/adamh-broad/single_cell_airway/blob/d7741835738ed58a9f93f2febb4dd302d9a9455b/PulseSeq.md#compute-the-p-values-for-increasing-labeled-fraction-using-negative-binomial-regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; as an R script.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input &lt;code&gt;x&lt;/code&gt; is a dataframe with the following columns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x&lt;span class=&quot;math-container&quot;&gt;$variable # celltype&#xA;x$&lt;/span&gt;value # cell count&#xA;x&lt;span class=&quot;math-container&quot;&gt;$batch # condition+donor&#xA;x$&lt;/span&gt;timepoint # condition (e.g. adult, fetal)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Montoro, D.T., Haber, A.L., Biton, M. et al. A revised airway epithelial hierarchy includes CFTR-expressing ionocytes. Nature 560, 319–324 (2018) doi:10.1038/s41586-018-0393-7&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2019-11-20T18:15:51.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10851" PostTypeId="1" CreationDate="2019-11-20T18:19:19.440" Score="1" ViewCount="106" Body="&lt;p&gt;Burrows wheeler transformation based aligner like BWA or bowtie seems a standard alignment tool used many area. I was just wondering if there is a kind of alignment task in which BWT algorithm is not suitable.&lt;/p&gt;&#xA;" OwnerUserId="6530" LastActivityDate="2019-11-21T18:25:16.990" Title="Is BWT based aligner suitable for any types of alignment task?" Tags="|sequence-alignment|bwa|bowtie2|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10852" PostTypeId="2" ParentId="10851" CreationDate="2019-11-20T18:43:37.180" Score="3" Body="&lt;p&gt;BWT based algorithm is just a component of a full-pledge aligner. To that end, it is applicable to most alignment tasks. However, for long-read alignment and metagenomes, we don't often use BWT based algorithms as other methods are faster.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2019-11-20T18:43:37.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10853" PostTypeId="1" AcceptedAnswerId="10855" CreationDate="2019-11-20T19:10:16.657" Score="0" ViewCount="258" Body="&lt;p&gt;I am correcting a data frame of sample comparison using p value. However in some columns I have different sample numbers. Within the data frame I have 4 comparisons e.g B_R, R_C, B_C, C_C. They each have different sample numbers in the comparisons. I need to create a loop or change the code so I can vary the nSamples numbers for each of the comparisons. I have tried to do this for each comparison separately but to no avail. Does anyone have any suggestions? Thank you. I have tried.. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nGenes = ncol(cells)&#xA;nSamples = nrow(cells)&#xA;&#xA;MEs0 = moduleEigengenes(cells, moduleColors)$eigengenes&#xA;MEs = orderMEs(MEs0)&#xA;moduleTraitCor= cor(MEs, status, use=&quot;p&quot;)&#xA;moduleTraitPvalue = corPvalueStudent(moduleTraitCor, nSamples)&#xA;&#xA;sizeGrWindow(10,6)&#xA;textMatrix = paste(signif(moduleTraitCor, 2), &quot;\n(&quot;,&#xA;                   signif(moduleTraitPvalue, 1), &quot;)&quot;, sep = &quot;&quot;)  ``````````&#xA;&#xA;I have tried to code for the varying Sample numbers using:&#xA;&#xA;MEs0 = moduleEigengenes(cell, moduleColors)&lt;span class=&quot;math-container&quot;&gt;$eigengenes&#xA;MEs = orderMEs(MEs0)&#xA;moduleTraitCor= cor(MEs, NR_status, use=&quot;p&quot;)&#xA;moduleTraitPvalue$&lt;/span&gt;B_R = corPvalueStudent(moduleTraitCor, 32)&#xA;moduleTraitPvalue&lt;span class=&quot;math-container&quot;&gt;$R_C = corPvalueStudent(moduleTraitCor, 31)&#xA;moduleTraitPvalue$&lt;/span&gt;B_C = corPvalueStudent(moduleTraitCor, 47)&#xA;moduleTraitPvalue$C_C = corPvalueStudent(moduleTraitCor, 31)&#xA;&#xA;sizeGrWindow(10,6)&#xA;textMatrix = paste(signif(moduleTraitCor, 2), &quot;\n(&quot;,&#xA;                   signif(moduleTraitPvalue&lt;span class=&quot;math-container&quot;&gt;$NR_R,&#xA;                      moduleTraitPvalue$&lt;/span&gt;R_C,&#xA;                          moduleTraitPvalue&lt;span class=&quot;math-container&quot;&gt;$C_C,&#xA;                      moduleTraitPvalue$&lt;/span&gt;NR_C, 1), &quot;)&quot;, sep = &quot;&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6227" LastActivityDate="2019-11-20T20:52:51.853" Title="WGCNA correcting p value for varying sample numbers in each of the trait columns" Tags="|r|wgcna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10854" PostTypeId="2" ParentId="10851" CreationDate="2019-11-20T20:34:17.043" Score="0" Body="&lt;p&gt;Surely you know that neither bwa nor bowtie by itself is splice-aware?&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2019-11-20T20:34:17.043" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10855" PostTypeId="2" ParentId="10853" CreationDate="2019-11-20T20:52:51.853" Score="0" Body="&lt;p&gt;You can use the function &lt;code&gt;corAndPvalue&lt;/code&gt;. Simply replace the lines&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;moduleTraitCor= cor(MEs, status, use=&quot;p&quot;)&#xA;moduleTraitPvalue = corPvalueStudent(moduleTraitCor, nSamples)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;by something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cp = corAndPvalue(MEs, status)&#xA;moduleTraitCor = cp&lt;span class=&quot;math-container&quot;&gt;$cor&#xA;moduleTraitPvalue = cp$&lt;/span&gt;p&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;corAndPvalue&lt;/code&gt; takes automatically into account the number of valid samples for each element of the resulting correlation matrix.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2019-11-20T20:52:51.853" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10856" PostTypeId="1" AcceptedAnswerId="10860" CreationDate="2019-11-20T23:38:07.857" Score="0" ViewCount="62" Body="&lt;p&gt;I am trying to map a RNA-seq dataset using ensembl genome for medaka fish. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From here &lt;a href=&quot;http://uswest.ensembl.org/Oryzias_latipes/Info/Index&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://uswest.ensembl.org/Oryzias_latipes/Info/Index&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;when I click on a &lt;code&gt;Download DNA sequence (FASTA)&lt;/code&gt; I am forwarded here &lt;a href=&quot;ftp://ftp.ensembl.org/pub/release-98/fasta/oryzias_latipes/dna/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ensembl.org/pub/release-98/fasta/oryzias_latipes/dna/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where I have a huge number of &lt;code&gt;.fa.gz&lt;/code&gt; files and I am just looking for a single file, that I can use. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While when I am downloading medaka genome from NCBI (and using this technique) &lt;a href=&quot;http://www.metagenomics.wiki/tools/fastq/ncbi-ftp-genome-download&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.metagenomics.wiki/tools/fastq/ncbi-ftp-genome-download&lt;/a&gt; I have my ncbi genome in one file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I get an ensembl genome in one file? &lt;/p&gt;&#xA;" OwnerUserId="6470" LastEditorUserId="6273" LastEditDate="2019-12-05T00:21:13.687" LastActivityDate="2019-12-05T00:21:13.687" Title="Where can I get ensembl Medaka genome for RNAseq" Tags="|rna-seq|ensembl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10857" PostTypeId="2" ParentId="10801" CreationDate="2019-11-21T02:24:45.583" Score="3" Body="&lt;p&gt;Apologies for the delay. The calculation is a majority consensus phylogeny of a maximum likelihood bootstrap, which is superimposed into a maximum likelihood phylogram. That's the tech. speak over ... the command is,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;raxmlHPC -f b -m PROTGAMMAILG -n output_bootstrap.tre -t RAxML_bestTree* -z RAxML_bootstrap.result&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;RAxML_bestTree* = your maximum likelihood tree without bootstraps (file is probably 1 line long)&lt;/li&gt;&#xA;&lt;li&gt;RAxML_bootstrap.result = your complete list of bootstraps (file is 500 lines long [or the number of bootstraps you set])&lt;/li&gt;&#xA;&lt;li&gt;output_bootstap.tre = final result tree,  i.e. majority consensus. This file is one line long but will have with all bootstraps attached to each branch.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The calculation will take less than one minute. The code is C and very slick. Note the &quot;RAxML_bestTree*&quot; is probably better to write the file name in full, not truncated via the glob (*). BTW you must specify the model.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Visualising the tree&lt;/strong&gt; You use a Mac ... so the cool tree program is &lt;a href=&quot;http://tree.bio.ed.ac.uk/software/figtree/&quot; rel=&quot;nofollow noreferrer&quot;&gt;figtree&lt;/a&gt; . Make sure when you load it you click the branch option and set it to see your bootstraps, otherwise it will not visualise them by default. You might need to re-root the tree, from memory your command specified an outgroup. It might not be the right one though and figtree will easily re-root it for you. Bootstrap values above 75% are considered robust. If you get stuck, can chat but I'm not around until this time next week (bereavement leave and loads of meetings).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-11-21T03:20:01.510" LastActivityDate="2019-11-21T03:20:01.510" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10860" PostTypeId="2" ParentId="10856" CreationDate="2019-11-21T13:13:12.653" Score="3" Body="&lt;p&gt;Download the ones without a chromosome number in the filename. Those files are the whole genome.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastEditorUserId="1332" LastEditDate="2019-11-21T16:19:59.927" LastActivityDate="2019-11-21T16:19:59.927" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10861" PostTypeId="1" AcceptedAnswerId="10862" CreationDate="2019-11-21T16:17:38.103" Score="3" ViewCount="2688" Body="&lt;p&gt;I have 7 biological replicate of a normal sample after peak-calling I converted it into bed files and now I want to find what are the common region between those .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So i tried with bedtool intersect &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;my command is this &#xA;bedtools intersect -a SRR2920506.filtered.bed -b SRR2920531.filtered.bed SRR2920466.filtered.bed SRR2920478.filtered.bed SRR2920507.filtered.bed SRR2920532.filtered.bed  &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;it says &quot; the intersect tool can detect overlaps between a single -a file and multiple -b files (instead of just one previously). One simply provides multiple -b files on the command line.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My doubt is the file that goes to &quot;a&quot; argument is not query which i have to detect overlaps against &quot;b&quot; files as these are all biological replicate. Am i doing the right way?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly I tried with &lt;code&gt;bedops&lt;/code&gt; but the number of overlaps which is reported is really low when i compare the result which i get from bedtools intersect function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion or help regarding how to intersect multiple would be really appreciated &lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2019-11-21T16:33:25.133" Title="bedtool intersect multiple bed files" Tags="|bedtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10862" PostTypeId="2" ParentId="10861" CreationDate="2019-11-21T16:26:59.547" Score="5" Body="&lt;p&gt;If you want to find the intersection between ALL your bedfiles, you can try &lt;code&gt;multiIntersectBed&lt;/code&gt; (available since bedtools 2.14.3).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should work like this :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;multiIntersectBed -i SRR2920506.filtered.bed SRR2920531.filtered.bed SRR2920466.filtered.bed SRR2920478.filtered.bed SRR2920507.filtered.bed SRR2920532.filtered.bed  &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To sort your bedfiles by chromosome then by start :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sort -k 1,1 -k2,2n myfile.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6273" LastEditorUserId="6273" LastEditDate="2019-11-21T16:33:25.133" LastActivityDate="2019-11-21T16:33:25.133" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10863" PostTypeId="2" ParentId="10851" CreationDate="2019-11-21T18:25:16.990" Score="1" Body="&lt;p&gt;Not sure about the algorithm itself but Heng Li wrote an article about the future of BWA when releasing minimap2. It gives helpful insights about the topic : &lt;a href=&quot;https://lh3.github.io/2018/04/02/minimap2-and-the-future-of-bwa&quot; rel=&quot;nofollow noreferrer&quot;&gt;Heng Li's blog&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6273" LastActivityDate="2019-11-21T18:25:16.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10864" PostTypeId="1" CreationDate="2019-11-21T18:42:32.730" Score="1" ViewCount="451" Body="&lt;p&gt;&lt;em&gt;Note: this question was also asked on &lt;a href=&quot;https://www.biostars.org/p/409111/#409142&quot; rel=&quot;nofollow noreferrer&quot;&gt;biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;are you aware of any tool that is able to perform error-tolerant pattern-matching search on protein FASTA files?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, I want to know, which proteins in my fasta file to match&lt;code&gt;ADNG..C.G&lt;/code&gt; regexp (which represents &lt;code&gt;ADNGCG&lt;/code&gt; pattern). However, I want to be tolerant to matching errors, meaning that I'm good with any protein that differs in any 1 letter in the motif: &lt;code&gt;ADNG..C.D&lt;/code&gt;, &lt;code&gt;ADNG..D.G&lt;/code&gt;, &lt;code&gt;MDNG..C.G&lt;/code&gt;, etc are all good. Running all possible variants through &lt;code&gt;grep&lt;/code&gt; is possible, but exponentially long for longer patterns (I usually have &lt;code&gt;15-20&lt;/code&gt; letters in pattern, and scan &lt;code&gt;~10e6-7&lt;/code&gt; sequences).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am aware of the tool &lt;code&gt;agrep&lt;/code&gt; (&lt;a href=&quot;https://en.wikipedia.org/wiki/Agrep&quot; rel=&quot;nofollow noreferrer&quot;&gt;docs&lt;/a&gt;. As far as I know, it's not supported anymore, and also does not distinguish letters in which it's allowed to make error. Also, it does not support long enough patterns (with more than 9 errors -- and yes, I tried to recompile it with myself from &lt;a href=&quot;https://github.com/Wikinaut/agrep&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and could not get it work). Also, it's not designed for proteins specifically.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am also aware of &lt;a href=&quot;https://prosite.expasy.org/scanprosite/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ScanProsite&lt;/a&gt; and &lt;a href=&quot;https://www.bioinformatics.org/sms2/protein_pattern.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Protein Pattern Find&lt;/a&gt; services. The first one is not error-prone, the second one is seems to be a web interface for &lt;code&gt;grep&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6537" LastEditorUserId="73" LastEditDate="2019-11-22T05:20:49.250" LastActivityDate="2020-09-01T16:08:10.890" Title="Is there any tool for fuzzy sequence matching?" Tags="|proteins|python|blast|software-recommendation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10866" PostTypeId="1" CreationDate="2019-11-22T03:51:41.300" Score="0" ViewCount="93" Body="&lt;p&gt;what is the most optimal keyword search query to input into  NCBI In order to find all the available proteins accession numbers for a specific family? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, how would I find all the reviewed accession numbers for all currently available human ALDP (AdrenoLeukoDystrophy Protein) family of proteins that are in NCBI? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, is there a specific database within NCBI that is optimized for my task? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="6539" LastActivityDate="2019-11-22T11:12:50.207" Title="how to search NCBI to find all available reviewed accession numbers for a protein family" Tags="|phylogenetics|literature-search|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10867" PostTypeId="2" ParentId="10410" CreationDate="2019-11-22T05:33:41.767" Score="0" Body="&lt;p&gt;Is there any particular reason why you don't want a BAM output format? If the sequences are sufficiently long, then the CIGAR field from the SAM output of minimap2 could be used:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 -a seq1.fa seq2.fa | samtools view | awk '{print $6}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2019-11-22T05:33:41.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10868" PostTypeId="1" CreationDate="2019-11-22T06:08:31.920" Score="1" ViewCount="219" Body="&lt;p&gt;I have a GSE data file in csv file format containing fields such as: ID, adj.P.Val, P.Value, t, B, logFC, Gene.symbol, Gene.title. In which adj.P.Val, P.Value, t, B, logFC fields being numeric. What are the factors I need to consider if I want to cluster the data only on logFC using K-Means clustering algorithm ? And first of all is it feasible to perform clustering on GSE data files ? If yes, what should be the approach ? If not, what different kinds of analysis can be performed on such kind of datasets ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following is snapshot of the data: &lt;a href=&quot;https://i.stack.imgur.com/GdPhj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GdPhj.png&quot; alt=&quot;GSE Data snapshot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6541" LastActivityDate="2023-08-04T20:03:00.350" Title="What kind of analysis is practically done on GSE data files?" Tags="|bioconductor|python|clustering|machine-learning|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10869" PostTypeId="2" ParentId="10866" CreationDate="2019-11-22T10:31:03.853" Score="2" Body="&lt;p&gt;A simple approach is to search for the gene name in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene/?term=ALDP[name]&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Gene portal&lt;/a&gt; but this can miss a few cases and often include genes that are unrelated because the gene symbol is used by other unrelated genes as well. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a cleaner dataset, you can use the NCBI Orthologs as follows. Search for &lt;code&gt;human ALDP&lt;/code&gt; in the NCBI homepage with &lt;code&gt;All Databases&lt;/code&gt; selected in the dropdown menu as shown below: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GGcbA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GGcbA.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After you click the 'Search' button, you will be directed to a &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/search/all/?term=human%20ALDP&quot; rel=&quot;nofollow noreferrer&quot;&gt;result page&lt;/a&gt; that looks like this: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3dJvD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3dJvD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Click on the 'Orthologs' button and you will be directed to a &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene/215/ortholog/?scope=117570&quot; rel=&quot;nofollow noreferrer&quot;&gt;page&lt;/a&gt; with the list of proteins that are similar to the one you have started from. Note, these may not be orthologs in the truest sense; see this &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/kis/info/how-are-orthologs-calculated/&quot; rel=&quot;nofollow noreferrer&quot;&gt;description&lt;/a&gt; about how they are calculated. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can search for the taxonomic group that you are interested in, pick and choose the organisms of your interest and download the table in CSV format. It will have the NCBI GeneID, RNA and protein accessions. &lt;/p&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2019-11-22T10:31:03.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10870" PostTypeId="2" ParentId="10866" CreationDate="2019-11-22T11:12:50.207" Score="0" Body="&lt;p&gt;&lt;em&gt;This is more of a comment, but due to the lengthiness I am posting it as an answer, which it isn't.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If by protein family you mean more than the 3-5 similar protein in bony fish and tetrapods resulting from the x4 genome duplication after the split with lampreys, the solution by @vkkodali is perfect for you —and I think that is the case. Doing a Blast of the gene with &lt;code&gt;human&lt;/code&gt; and &lt;code&gt;Petromyzon marinus&lt;/code&gt; as an outgroup also works well especially if you want to infer some relationship between them. The huge ABC transporter family is one of the exceptions, so this trick won't help as it duplicate many many times across the tree of life.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you meant &lt;em&gt;domain fold&lt;/em&gt; family, &lt;em&gt;i.e.&lt;/em&gt; in the structural sense, like TIM barrels, Rossmann folds, &amp;beta;-propellers (WD40 repeat), SH3 and so forth, you are after &lt;a href=&quot;https://pfam.xfam.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PFam families&lt;/a&gt;, for which Uniprot is a much better database than NCBI, albeit trickier to parse programmatically: &lt;a href=&quot;https://www.uniprot.org/uniprot/P33897&quot; rel=&quot;nofollow noreferrer&quot;&gt;uniprot: P33897&lt;/a&gt; gives &lt;a href=&quot;http://pfam.xfam.org/family/PF00005&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pfam: PF00005&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2019-11-22T11:12:50.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10872" PostTypeId="1" AcceptedAnswerId="10908" CreationDate="2019-11-22T20:10:33.707" Score="3" ViewCount="443" Body="&lt;p&gt;I have recently been tasked with analyzing a heterogeneous microbial sample composed of 16S data generated on a MinION sequencer, with the goal of assigning taxonomies and calculating respective abundance. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is my understanding that traditional preprocessing methods such as Dada2, Unoise3, and Deblur would likely not be appropriate for this kind of data given the poorer read quality inherent with nanopore sequencing. To give an example, the average read quality of this current experiment is a Phred score of 15 and an average read length of 1500bp. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, it would seem best to try and utilize a classifier that makes use of a full length 16S reference, rather than just the V3 region, to try and offset the higher error rate. But are there any classifiers that do this out of the box?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am very interested to hear of any advice or input the community has on this. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="6548" LastActivityDate="2020-07-08T06:39:57.620" Title="Microbial Classification with 16S Nanopore Data" Tags="|nanopore|minion|microbiology|" AnswerCount="2" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10873" PostTypeId="2" ParentId="10868" CreationDate="2019-11-22T20:20:02.970" Score="1" Body="&lt;p&gt;Have you ever used &lt;code&gt;GEO2R&lt;/code&gt;? You can use &lt;code&gt;GEO2R&lt;/code&gt; to analyze the data dividing it in subgroups (clusters) that interest you.&lt;/p&gt;&#xA;" OwnerUserId="6509" LastEditorUserId="3541" LastEditDate="2019-11-23T09:00:51.560" LastActivityDate="2019-11-23T09:00:51.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10874" PostTypeId="2" ParentId="10868" CreationDate="2019-11-23T11:35:58.157" Score="1" Body="&lt;p&gt;You have multiple questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First of all, here is the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/info/overview.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;description&lt;/a&gt; of &lt;code&gt;GSE files&lt;/code&gt; or &lt;code&gt;GEO Series&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Series records are supplied by submitters&lt;/strong&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;A Series record links together a group of related Samples and provides&#xA;  a focal point and description of the whole study. Series records may&#xA;  also contain tables describing extracted data, summary conclusions, or&#xA;  analyses. Each Series record is assigned a unique and stable GEO&#xA;  accession number (GSExxx).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Apparently the authors have provided their (differential expression) analysis results (p-values, adjusted p-values, logFC, ...). And the table you are sharing in your question does not look like &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE3541&quot; rel=&quot;nofollow noreferrer&quot;&gt;a typical GSE profile&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On your question on the feasibility of clustering based on GSE data: that would depend on the features (columns) you are provided with, you should have an understanding of each feature that you have. In your case, you can cluster your genes based on p-value, adjusted p-value and logFC, but would that make sense?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding your question using k-means on only one variable, logFC: &lt;a href=&quot;https://stats.stackexchange.com/questions/13781/clustering-1d-data&quot;&gt;This link&lt;/a&gt; points to a related discussion. Short answer is &quot;technically yes&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your last question depends on the &quot;research question&quot; you have, if you are asking about the table in your question: pathway analysis, gene set enrichment analysis, checking for co-expressed modules, ...&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-11-23T11:35:58.157" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10875" PostTypeId="1" AcceptedAnswerId="10886" CreationDate="2019-11-23T21:30:32.313" Score="0" ViewCount="765" Body="&lt;p&gt;I need help interpreting a kernal density plot and a histogram of this data. I have provided some context below let me know if you need more information&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A co-worker that I am collaborating with has a list of genes ranked by expression fold change.  He says most of the genes in one dataset are noise with respect to his analysis so he is trying to filter this out.  He has sent me a document of his methods for generating a new list of genes which he think are relevant.  he has run a permutation test on the ranked genes I have provided a quote here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;To determine which genes contributed more strongly in real data, as compared by permuted data, the rankings achieved for each gene in real data were compared to the shuffled rankings. For each gene, and each permutation type, a p-value was calculated by counting how many times that gene was ranked higher in shuffled data than in real data, and dividing it by the number of comparisons (16 * 256 = 4096). To achieve p&amp;lt;0.05 a gene’s real ranking would have to be higher than 95% of its shuffled rankings.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following this he provides kernel density plots of the genes under different rankings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Below are kernel density estimation (kde) plots, overlaid on histograms, of the rankings of two genes randomly selected from the few genes with p&amp;lt;0.05 vs all three permutations.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/011cw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/011cw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Below are kde plots of the rankings of two genes randomly selected from the many genes with p&gt;0.05 vs all three permutations.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LBAEa.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LBAEa.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I looking at?  What does this tell me about the genes ranking statistics.&lt;/p&gt;&#xA;" OwnerUserId="2482" LastActivityDate="2019-11-25T18:15:31.457" Title="Interpreting a kernel density function plot of gene expression" Tags="|statistics|gene-expression|gene-ontology|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="10877" PostTypeId="1" CreationDate="2019-11-24T11:30:19.150" Score="0" ViewCount="27" Body="&lt;p&gt;It is almost common sense that local amplification of gene increase the its mRNA expression . I recently heard that large scale SCNA (such as arm-level amplification) does not increase the mRNA expression level of gene within the region. I did some analysis (SCNA is determined by SNP array and NGS, mRNA expression is measured by RNA-Seq) and found that this is indeed the case. I further did some research and did not found any explanations. it seems like a common acknowledgement. Is there any technical bias or biological explanation that I am missing here? Or is it something that has not have a answer yet?&lt;/p&gt;&#xA;&lt;p&gt;I have no clue about this at all. Can you recommend this papers on this?&lt;/p&gt;&#xA;" OwnerUserId="4905" LastEditorUserId="73" LastEditDate="2021-07-14T11:54:52.590" LastActivityDate="2021-07-14T11:56:24.953" Title="Arm-level SCNA does not change mRNA expression level?" Tags="|rna|cnv|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10880" PostTypeId="2" ParentId="10582" CreationDate="2019-11-24T22:00:04.947" Score="0" Body="&lt;p&gt;Pearson correlation evaluates two continuous variables. Spearman correlation is often used to evaluate ordinal variables. Kendall correlation is a nonparametric test for two continuous variables.&lt;/p&gt;&#xA;" OwnerUserId="6509" LastActivityDate="2019-11-24T22:00:04.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10881" PostTypeId="1" CreationDate="2019-11-25T03:42:38.007" Score="0" ViewCount="414" Body="&lt;p&gt;I have tried to export the monocle data into Seurat and got the error&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;Error in Seurat::CreateSeuratObject(raw.data = data, normalization.method = &quot;LogNormalize&quot;,  :&#xA;  unused arguments (raw.data = data, normalization.method = &quot;LogNormalize&quot;, do.scale = TRUE, do.center = TRUE, is.expr = monocle_cds@lowerDetectionLimit)&#xA;Calls: exportCDS&#xA;In addition: Warning message:&#xA;In if (export_to == &quot;Seurat&quot;) { :&#xA;  the condition has length &amp;gt; 1 and only the first element will be used&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is my code &lt;code&gt;seurat_SAN_cds_subset &amp;lt;- exportCDS(SAN_cds_subset)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is anyone familiar with this?&lt;/p&gt;&#xA;" OwnerUserId="3171" LastEditorUserId="77" LastEditDate="2020-02-20T09:36:25.420" LastActivityDate="2020-03-21T10:01:30.260" Title="how to export the monocle data into Seurat" Tags="|seurat|single-cell|monocle|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10882" PostTypeId="1" AcceptedAnswerId="10883" CreationDate="2019-11-25T05:54:39.003" Score="0" ViewCount="85" Body="&lt;p&gt;I was looking how to take readcount from ATAC seq data , this &lt;a href=&quot;https://www.biostars.org/p/320394/#409383&quot; rel=&quot;nofollow noreferrer&quot;&gt;post&lt;/a&gt; which suggested to use bedtools and featurecounts to take out readcounts .But multiBamSummary seems to be pretty straight forward. Now I have my count file but only issue is there are no gene names. So now is it possible to annotate these chromosome regions? if so how ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion or help would be highly appreciated &lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2019-11-25T08:28:24.343" Title="multiBamSummary output chromosome region annotate issue" Tags="|deeptools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10883" PostTypeId="2" ParentId="10882" CreationDate="2019-11-25T08:28:24.343" Score="1" Body="&lt;p&gt;Assuming you used a BED file as input then the output is in the same order, so just merge the two files together.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-11-25T08:28:24.343" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="10885" PostTypeId="1" CreationDate="2019-11-25T14:37:41.177" Score="4" ViewCount="91" Body="&lt;p&gt;I am looking for a project or tool that allows programmatic pairwise alignments of proteins but that takes care with transmembrane regions of proteins. TM regions are traditionally too information poor for many tools to align accurately &lt;a href=&quot;https://pdfs.semanticscholar.org/f02a/f2722752e6897e85a2e7fba01deea39ec372.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;(Wong et al, 2010)&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.nature.com/articles/s41598-017-13083-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;TM-aligner&lt;/a&gt; doesn't appear to have a program to download, and is only available via webserver. On the otherhand,&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1160151/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PRALINE&lt;/a&gt; has some optimisation for TM regions, however was released in 2008 and since then there has been a boom in TM knowledge from structures so does not benefit from that.Currently I am using PSI-BLAST. However,&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/BLAST/tutorial/Altschul-2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PSI-Blast uses SEG to mask low complexity regions&lt;/a&gt;. This will most likely mean that TMHs are masked making the alignment of those regions fairly unreliable.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Primarily I am interesting in sequence alignments. But if anyone knows of a TM structural alignment, that could be good to know.&lt;/p&gt;&#xA;&lt;p&gt;I think generally that counting idea works for an ideal helix, but a vast majority of TMHs break convention. I wouldn't expect to see a pattern based aligner that was that general. I'd be interested if anyone knows of a tool that juggles that though!&lt;/p&gt;&#xA;&lt;p&gt;If I understand correctly, Matteo was saying that TMHs follow a simple patter (HHP/H,HHP/H...~7 times). This is not the case. There are too many examples of TMHs that do not match this pattern to assume this pattern would accurately align TMHs. As @M__ suggests, the number of TMP structures has not caught up to their soluble counterparts in the database. I've updated the question to explain why PSI-Blast might not be the best option.&lt;/p&gt;&#xA;" OwnerUserId="747" LastEditorUserId="3848" LastEditDate="2022-03-03T17:30:01.687" LastActivityDate="2022-03-03T17:30:01.687" Title="Is there a modern alignment tool tailored for transmembrane regions?" Tags="|sequence-alignment|proteins|covid-19|membrane-protein|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10886" PostTypeId="2" ParentId="10875" CreationDate="2019-11-25T18:15:31.457" Score="2" Body="&lt;p&gt;There are two topics here to be discussed. Gene Set Enrichment Analysis and a kernel Density plot.&lt;/p&gt;&#xA;&lt;h2&gt;GSEA&lt;/h2&gt;&#xA;&lt;p&gt;The Wikipedia page about &lt;a href=&quot;https://en.wikipedia.org/wiki/Gene_set_enrichment_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gene Set Enrichment Analysis&lt;/a&gt; covers well the principle and the tools that can be used to calculate it. To cite it:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The general steps are summarized below:&lt;/p&gt;&#xA;&lt;p&gt;Calculate the enrichment score (ES) that represents the amount to which the genes in the set are over-represented at either the top or bottom of the list. This score is a Kolmogorov–Smirnov-like statistic [a ranking based statistic].&lt;/p&gt;&#xA;&lt;p&gt;Estimate the statistical significance of the ES. This calculation is done by a phenotypic-based permutation test in order to produce a null distribution for the ES. The P value is determined by comparison to the null distribution.&lt;/p&gt;&#xA;&lt;p&gt;Calculating significance this way tests for the dependence of the gene set on the diagnostic/phenotypic labels&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Basically, you want to test if the ranking is random or not. Therefore you make a random dataset and test whether your set is like that. One assumption underlying this is that the output of the gene whatever it may be is &lt;em&gt;not&lt;/em&gt; normally distributed (parametric, follows a bell curve), hence the use of ranking, which removes this (you lose signal but you lose biases). I am assuming the non-parametric nature of your ouput values was tested or is a known non-parametric value, such as RPKM or log2 fold change (by virtue of being a ratio). In other words, the magnitude difference between your samples is lost, but their order is not.&lt;/p&gt;&#xA;&lt;p&gt;There are several tools, some online, that do this enrichment testing for you depending on the task, some even work with gene ontologies.&lt;/p&gt;&#xA;&lt;h2&gt;KDE&lt;/h2&gt;&#xA;&lt;p&gt;A kernel density plot is a like a histogram, but smoothed, albeit not in a moving average way. In a histogram you divide your x-axis into bins: it is discreet and you get an integer count per bin. In a kernel density plot the data is fitted to &amp;quot;probability density function&amp;quot;, an equation which given x will give y, where the integral of the curve is one, hence why y is density not counts. The x axis will be your bins-like value, in this case rank. The binning window metric for a KS function is called bandwidth.&lt;/p&gt;&#xA;&lt;p&gt;See &lt;a href=&quot;https://uk.mathworks.com/help/stats/kernel-distribution.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matlab documentation&lt;/a&gt; for a fairly non-opaque explanation.&lt;/p&gt;&#xA;&lt;p&gt;However, if you want to compare histograms, I strongly suggest violin plots, not because they are strongly in vogue, but because you can compare histograms without worrying about overlaying them.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2019-11-25T18:15:31.457" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10891" PostTypeId="1" CreationDate="2019-11-26T12:47:40.123" Score="0" ViewCount="91" Body="&lt;p&gt;I am not sure how to tackle this one, so I’ll explain the general idea I have in mind. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given results of a CRISPR knockout screening experiment &lt;a href=&quot;https://www.nature.com/articles/s41467-018-06676-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;(like in this paper)&lt;/a&gt; with two types of assays: control vs. treated with 5 time points per each of the assays. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to use the screening data to perform systems biology &lt;a href=&quot;https://www.ebi.ac.uk/training/online/course/network-analysis-protein-interaction-data-introduction/network-analysis-biology-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;network analysis&lt;/a&gt;? Something which is related to signal transduction, transcriptional regulatory, or metabolic networks? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe something which uses Bayesian inference? Or something similar to &lt;a href=&quot;https://www.mathworks.com/help/bioinfo/ug/identifying-differentially-expressed-genes-from-rna-seq-data.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;differentially expressed representation&lt;/a&gt; of the CRISPR screening? Alternatively, a GWAS analysis similar to &lt;a href=&quot;https://www.nature.com/articles/s41467-019-13585-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2020-01-14T04:13:45.337" LastActivityDate="2021-08-18T10:28:04.043" Title="CRISPR screening and systems biology" Tags="|genomics|gwas|crispr|systems-biology|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10892" PostTypeId="2" ParentId="10891" CreationDate="2019-11-27T06:16:23.947" Score="1" Body="&lt;p&gt;What do you call screening results?&#xA;Do you have a kind of array with protein or RNA data?&#xA;If so, you can use several open-source tools available on the internet, like &lt;a href=&quot;https://amp.pharm.mssm.edu/Enrichr&quot; rel=&quot;nofollow noreferrer&quot;&gt;Enrichr&lt;/a&gt; or &lt;a href=&quot;https://www.networkanalyst.ca&quot; rel=&quot;nofollow noreferrer&quot;&gt;network analyst&lt;/a&gt; to better understand the pathways affected by your treatment.&lt;/p&gt;&#xA;" OwnerUserId="6509" LastEditorUserId="6509" LastEditDate="2019-11-27T16:13:27.610" LastActivityDate="2019-11-27T16:13:27.610" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10893" PostTypeId="1" CreationDate="2019-11-27T06:21:50.110" Score="1" ViewCount="494" Body="&lt;p&gt;I was running tblastn using standalone blast2.9.0+ through Ubuntu Linux, to get the alignment result of a list of protein sequences for same species, against its complete genome file. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The command I was using: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;~/softwares/ncbi-blast-2.9.0+/bin/tblastn -query TBLASTNstring/882.Tstring -subject ../com_gen/*882*.fasta -outfmt &quot;7 acc qacc pident qcovs evalue sstart send sframe&quot; -max_hsps 1 -num_alignments 1 -out 882.blastout&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Info update: The query file has more than 1 sequence but all are in fasta format; the subject file has only 1 sequence; there is only 1 file with &quot;882&quot; named in the folder)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I received error from tblastn as follow:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error:  (CArgException::eSynopsis) Too many positional arguments (1), the offending value: ../com_gen/DP_882_DeVul_Hilde.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;All the sequences in both my query and subject sequence files are of two lines, the header line started with &quot;&gt;&quot;, followed with sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have checked both query and subject sequence files using &lt;code&gt;sed -n '0~2p' | grep -v -E '[[:alpha:]]'&lt;/code&gt; to see if there exist special character or space within the sequence line, but the output of the results return nothing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS: the complete genome file contained of nucleotide sequences with only ATGC (I have checked using &lt;code&gt;grep -v &quot;ATGC&quot;&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran the same command for other species as well and it worked fine. &#xA;&lt;a href=&quot;https://i.stack.imgur.com/jM9fz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jM9fz.png&quot; alt=&quot;Example of result&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Info Update: The screenshot above was example of the result for species with TaxID 196627.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wondered what is the thing that went wrong here? And what else I can check on them?&lt;/p&gt;&#xA;" OwnerUserId="5054" LastEditorUserId="5054" LastEditDate="2019-11-28T03:29:50.133" LastActivityDate="2019-11-28T03:29:50.133" Title="TBLASTN error: Too many positional arguments" Tags="|blast|linux|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="10894" PostTypeId="2" ParentId="694" CreationDate="2019-11-27T12:46:55.427" Score="1" Body="&lt;p&gt;This &lt;a href=&quot;https://gist.github.com/sbliven/b7cc2c5305aa3652a75a580ae7c6ce33&quot; rel=&quot;nofollow noreferrer&quot;&gt;python script&lt;/a&gt; uses &lt;a href=&quot;https://biopython.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biopython&lt;/a&gt; and does the job well even on large structures. For example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;python cif2pdb.py 4ckh-assembly-1.cif 4ckh-assembly-1.pdb&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This generates the structure below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wWiJG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wWiJG.png&quot; alt=&quot;A large protein structure&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="747" LastActivityDate="2019-11-27T12:46:55.427" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10895" PostTypeId="2" ParentId="10891" CreationDate="2019-11-27T14:52:52.340" Score="1" Body="&lt;p&gt;To investigate genes through a network based approach you can use several tools. I don't know which level of programming you have or which tool you prefer but here is a brief overview of the possibilities you have.&lt;/p&gt;&#xA;&lt;p&gt;You can look into established databases containing information about interactions like, for instance, &lt;a href=&quot;https://string-db.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;STRING&lt;/a&gt;. Or, you can build networks using the &lt;a href=&quot;https://horvath.genetics.ucla.edu/html/CoexpressionNetwork/Rpackages/WGCNA/&quot; rel=&quot;nofollow noreferrer&quot;&gt;wgcna&lt;/a&gt; method with R.&lt;/p&gt;&#xA;&lt;p&gt;Then, once you have your network you can analyse its properties and here you also have several tools available such as standalone, java-base software like &lt;a href=&quot;https://cytoscape.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;cytoscape&lt;/a&gt;, R libraries such as &lt;a href=&quot;https://igraph.org/r/&quot; rel=&quot;nofollow noreferrer&quot;&gt;igraph&lt;/a&gt;, python libraries such as &lt;a href=&quot;https://networkx.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;networkX&lt;/a&gt; online tools like &lt;a href=&quot;http://www.centiserver.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;centiserver&lt;/a&gt;...&lt;/p&gt;&#xA;&lt;p&gt;Also you can perform gene set enrichment analsysis, which means adding a further layer of biological information to you set of genes like metabolic pathways (from KEGG, for example) and molecular functions (from GeneOntology, for instance) (gsea) as &lt;a href=&quot;http://www.gsea-msigdb.org/gsea/doc/GSEAUserGuideFrame.html?_Interpreting_GSEA_Results&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;http://www.bioconductor.org/packages/release/bioc/vignettes/fgsea/inst/doc/fgsea-tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, and &lt;a href=&quot;https://bioinformatics-core-shared-training.github.io/cruk-summer-school-2018/RNASeq2018/html/06_Gene_set_testing.nb.html#gsea-analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here there is also a nice &lt;a href=&quot;http://kateto.net/networks-r-igraph&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; about network analysis with R.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;" OwnerUserId="3119" LastEditorUserId="3119" LastEditDate="2021-08-18T10:28:04.043" LastActivityDate="2021-08-18T10:28:04.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10896" PostTypeId="1" CreationDate="2019-11-27T16:09:07.000" Score="4" ViewCount="2375" Body="&lt;p&gt;There is already &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/694/converter-between-pdb-or-mmcif-and-mmtf&quot;&gt;a question on PDB/CIF to MMTF&lt;/a&gt;, however what is a robust way to programmatically go between PDB and CIF files?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example I can use a python script from this &lt;a href=&quot;https://gist.github.com/sbliven/b7cc2c5305aa3652a75a580ae7c6ce33&quot; rel=&quot;nofollow noreferrer&quot;&gt;gist&lt;/a&gt; that relies on Biopython.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;id=&quot;4ckh&quot;&#xA;wget http://www.ebi.ac.uk/pdbe/static/entry/download/&lt;span class=&quot;math-container&quot;&gt;$id-assembly-1.cif.gz&#xA;gunzip $&lt;/span&gt;id-assembly-1.cif.gz&#xA;python cif2pdb.py &lt;span class=&quot;math-container&quot;&gt;$id-assembly-1.cif $&lt;/span&gt;id.pdb&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This process yields a pdb structure as shown below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/wWiJG.png&quot; alt=&quot;A large protein structure&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issues are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Requires python/Biopython to be installed.&lt;/li&gt;&#xA;&lt;li&gt;The output isn't perfectly clean.&lt;/li&gt;&#xA;&lt;li&gt;The gist is relatively undeveloped and hasn't got much evidence of testing.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="747" LastEditorUserId="747" LastEditDate="2019-11-28T14:46:29.070" LastActivityDate="2019-11-30T00:45:37.400" Title="How to quickly and robustly convert between mmCIF and PDB?" Tags="|file-formats|protein-structure|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10897" PostTypeId="1" CreationDate="2019-11-27T18:27:08.900" Score="1" ViewCount="157" Body="&lt;p&gt;Is there a best practices for mapping Oxford Nanopore files to a reference?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a tool that can take a tarball of fast5 files and map them directly or do they need to be converted to fastq first?&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2019-11-29T00:37:42.327" Title="How do you map nanopore fast5 files?" Tags="|read-mapping|nanopore|fast5|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10898" PostTypeId="1" CreationDate="2019-11-28T09:04:02.537" Score="1" ViewCount="374" Body="&lt;p&gt;I have adapted my pipeline to the new filtering strategies implemented in gatk 4.1.4.0 and while looking at its output I noticed that the stat file generated by mutect2 has a negative number:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;callable -1.538687311E9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I did not find the documentation about this file content but in the gatk forum it is mentioned that is should represent the number of bases above a certain coverage (10 by default). &#xA;This happened for a WGS sample, while for WES I obtain a more sensible value (well, at least positive and integer, even if maybe a tad high):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;callable        8.6884303E7&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Has this happened to anyone else?&#xA;I compared the resulting calls obtained for the WGS with a previous version of mutect2 and they are comparable-ish (98% of the new &quot;ones&quot; are in common with the &quot;old&quot; ones), but I am still worried that something fishy is happening, expecially considering that the filtering step of the pipeline uses this stats file.&lt;/p&gt;&#xA;" OwnerUserId="6570" LastEditorUserId="6570" LastEditDate="2019-11-28T14:38:04.727" LastActivityDate="2020-09-05T16:06:05.563" Title="GATK 4.1.4.0 Mutect stats output" Tags="|wgs|gatk|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10899" PostTypeId="2" ParentId="5229" CreationDate="2019-11-28T17:12:10.733" Score="6" Body="&lt;p&gt;I tried several R packages (&lt;code&gt;mygene&lt;/code&gt;, &lt;code&gt;org.Hs.eg.db&lt;/code&gt;, &lt;code&gt;biomaRt&lt;/code&gt;, &lt;code&gt;EnsDb.Hsapiens.v79&lt;/code&gt;) to convert Ensembl.gene to gene.symbol, and found that the &lt;code&gt;EnsDb.Hsapiens.v79&lt;/code&gt; package / gene database provides the best conversion quality (in terms of being able to convert most of Ensembl.gene to gene.symbol).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Install the package if you have not installed by running this command: &#xA;BiocManager::install(&quot;EnsDb.Hsapiens.v79&quot;)&#xA;&#xA;library(EnsDb.Hsapiens.v79)&#xA;&#xA;# 1. Convert from ensembl.gene to gene.symbol&#xA;ensembl.genes &amp;lt;- c(&quot;ENSG00000150676&quot;, &quot;ENSG00000099308&quot;, &quot;ENSG00000142676&quot;, &quot;ENSG00000180776&quot;, &quot;ENSG00000108848&quot;, &quot;ENSG00000277370&quot;, &quot;ENSG00000103811&quot;, &quot;ENSG00000101473&quot;)&#xA;&#xA;geneIDs1 &amp;lt;- ensembldb::select(EnsDb.Hsapiens.v79, keys= ensembl.genes, keytype = &quot;GENEID&quot;, columns = c(&quot;SYMBOL&quot;,&quot;GENEID&quot;))&#xA;&#xA;# 2. Convert from gene.symbol to ensembl.gene&#xA;geneSymbols &amp;lt;-  c('DDX26B','CCDC83',  'MAST3', 'RPL11', 'ZDHHC20',  'LUC7L3',  'SNORD49A',  'CTSH', 'ACOT8')&#xA;&#xA;geneIDs2 &amp;lt;- ensembldb::select(EnsDb.Hsapiens.v79, keys= geneSymbols, keytype = &quot;SYMBOL&quot;, columns = c(&quot;SYMBOL&quot;,&quot;GENEID&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Other available R packages / gene database for the conversion can refer to &lt;a href=&quot;https://shiring.github.io/genome/2016/10/23/AnnotationDbi&quot; rel=&quot;noreferrer&quot;&gt;this GitHub page&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6572" LastEditorUserId="6572" LastEditDate="2019-11-29T16:15:43.367" LastActivityDate="2019-11-29T16:15:43.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10900" PostTypeId="1" CreationDate="2019-11-28T19:42:42.353" Score="1" ViewCount="37" Body="&lt;p&gt;I did not find a documentation. Can Maven (mass spectrometry software) read mzML files?&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2024-04-01T02:03:55.397" Title="Can Maven (mass spectrometry software) read mzML files?" Tags="|mass-spectrometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10901" PostTypeId="2" ParentId="10897" CreationDate="2019-11-29T00:37:42.327" Score="2" Body="&lt;p&gt;Fast5 files need to be converted to fastq first prior to mapping. There are currently no tools available to map the entire raw signal directly to a reference, although a few tools are available for identifying short sequences within the raw signal.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2019-11-29T00:37:42.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10902" PostTypeId="1" AcceptedAnswerId="10904" CreationDate="2019-11-29T08:46:31.753" Score="1" ViewCount="81" Body="&lt;p&gt;How can one do the reverse of bcftools merge?&#xA;To get one sample back out of the merged one.&#xA;Without holding reference sites.&lt;/p&gt;&#xA;" OwnerUserId="6573" LastActivityDate="2019-11-29T12:37:15.610" Title="How to do the reverse of bcf merge?" Tags="|bcftools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10903" PostTypeId="1" AcceptedAnswerId="10905" CreationDate="2019-11-29T11:09:22.997" Score="0" ViewCount="26" Body="&lt;p&gt;I am trying to extract data from this:&#xA;&lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR6245218&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR6245218&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I installed SRA Toolkit, downloaded the SRR6245218 file and executed this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fastq-dump -X 5 SRR6245218 --fasta -O ./&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result is a fasta file 900 bytes long, which does not contain the mitochondrial data that I need.&#xA;What am I doing wrong?&lt;/p&gt;&#xA;" OwnerUserId="5190" LastActivityDate="2019-11-29T14:40:11.260" Title="SRA Toolkit and lebanese data" Tags="|fasta|phylogenetics|sratoolkit|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10904" PostTypeId="2" ParentId="10902" CreationDate="2019-11-29T12:37:15.610" Score="1" Body="&lt;p&gt;&lt;code&gt;bcftools&lt;/code&gt; has the &lt;code&gt;view&lt;/code&gt; command which can do this. Create a text file (&lt;code&gt;samples&lt;/code&gt;) which has the names of the samples you want, one per line, and then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view -S samples merged.vcf.gz &amp;gt; new.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, if you're running a shell that supports it, you can use input redirection for a single sample name:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view -S &amp;lt;(echo &quot;sample name&quot;) merged.vcf.gz &amp;gt; new.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-11-29T12:37:15.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10905" PostTypeId="2" ParentId="10903" CreationDate="2019-11-29T14:40:11.260" Score="1" Body="&lt;p&gt;Your command is fine. It is working as expected. Did you intend to include &lt;code&gt;-X 5&lt;/code&gt; in your command? That restricts the number of spots to 5. If you drop it, you will download the entire set of 2563898 spots. &lt;/p&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2019-11-29T14:40:11.260" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10906" PostTypeId="2" ParentId="9090" CreationDate="2019-11-29T21:14:21.040" Score="0" Body="&lt;p&gt;Had the same issue, but found that I need to provide a filename to use the csvstats option&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;snpEff -Xmx4g -csvStats stats.csv -v hg19 08_FILTER/NA12878.Filtered.Variants.vcf &amp;gt; test.ann.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6580" LastActivityDate="2019-11-29T21:14:21.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10907" PostTypeId="2" ParentId="10896" CreationDate="2019-11-30T00:45:37.400" Score="5" Body="&lt;p&gt;Converting a PDB file to some version of an mmCIF file is always possible provided the PDB file doesn't have any &quot;problems&quot;. However grouping identical molecules into the same entity is non-trivial so the Biopython and BioJulia mmCIF writers (which I wrote) do not try and do this, simply assigning new entites for each molecule. These files will also be missing other things like the metadata parts of the mmCIF dictionary not involving the atoms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Biopython:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio.PDB import PDBParser, MMCIFIO&#xA;p = PDBParser()&#xA;struc = p.get_structure(&quot;&quot;, &quot;file.pdb&quot;)&#xA;io = MMCIFIO()&#xA;io.set_structure(struc)&#xA;io.save(&quot;file.cif&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In BioJulia:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using BioStructures&#xA;struc = read(&quot;file.pdb&quot;, PDB)&#xA;writemmcif(&quot;file.cif&quot;, struc)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Going the other way is, however, not always possible. You can run out of atom serials or chain IDs to use in the output PDB file (this is one reason for the switch to mmCIF files in general). The gist you have linked is a way to get round this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If this is not a problem for your structure you can invert the above snippets to get a decent PDB file - see the relevant docs. Note that in both cases it will use the &lt;code&gt;auth_seq_id&lt;/code&gt; and &lt;code&gt;auth_asym_id&lt;/code&gt; fields to get the residue number and chain ID rather than the relabelled &lt;code&gt;label&lt;/code&gt; versions. The test suites of Biopython and BioJulia do test these conversions pretty thoroughly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is also &lt;a href=&quot;https://github.com/haddocking/pdb-tools&quot; rel=&quot;noreferrer&quot;&gt;pdb-tools&lt;/a&gt; if you do not want to install Biopython, though both involve a simple &lt;code&gt;pip install&lt;/code&gt; so it's not much difference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For some speed benchmarks of various software see &lt;a href=&quot;https://github.com/jgreener64/pdb-benchmarks&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://github.com/project-gemmi/mmcif-benchmark&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2019-11-30T00:45:37.400" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10908" PostTypeId="2" ParentId="10872" CreationDate="2019-11-30T10:12:13.967" Score="3" Body="&lt;p&gt;Use &lt;a href=&quot;https://ccb.jhu.edu/software/kraken2/index.shtml?t=downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;kraken2&lt;/a&gt; with one of the 16S databases. Kraken2 has worked well for me with nanopore sequences. &lt;a href=&quot;https://ccb.jhu.edu/software/pavian/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pavian&lt;/a&gt; can be used to inspect the Kraken2 report files visually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using uncorrected called reads has worked for me. Any errors from nanopore sequencing should be inconsistent enough (i.e. not always lead towards the same wrong organism) that the noise doesn't matter.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: if you're using the MiniKraken database instead of a comprehensive 16S database, there might be some false positive species assignments depending on which kmers get pruned out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another possibility is &lt;a href=&quot;https://ccb.jhu.edu/software/centrifuge/&quot; rel=&quot;nofollow noreferrer&quot;&gt;centrifuge&lt;/a&gt;, but I'm not sure if that works for 16S amplicon sequencing; it's primarily for metagenomic analysis.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2019-12-10T22:38:14.970" LastActivityDate="2019-12-10T22:38:14.970" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10909" PostTypeId="2" ParentId="10828" CreationDate="2019-11-30T23:04:29.353" Score="2" Body="&lt;p&gt;If additional CPUs are available:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CPU_COUNT=5&#xA;for file in *.vcf; do &#xA;     (grep -v &quot;##&quot; ${file} | awk '{print $1&quot;\t&quot;$2&quot;\t&quot;$2&quot;\t&quot;$4&quot;\t&quot;$5&quot;\t&quot;$6&quot;\t&quot;$7&quot;\t&quot;$8&quot;\t&quot;$9&quot;\t&quot;$10&quot;\t&quot;$11}' &amp;gt; ${file%.vcf}.txt) &amp;amp; let count+=1&#xA;     [[ $((count%CPU_COUNT)) -eq 0 ]] &amp;amp;&amp;amp; wait&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;one-liner:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CPU_COUNT=5; for file in *.vcf; do (grep -v &quot;##&quot; ${file} | awk '{print $1&quot;\t&quot;$2&quot;\t&quot;$2&quot;\t&quot;$4&quot;\t&quot;$5&quot;\t&quot;$6&quot;\t&quot;$7&quot;\t&quot;$8&quot;\t&quot;$9&quot;\t&quot;$10&quot;\t&quot;$11}' &amp;gt; ${file%.vcf}.txt) &amp;amp; let count+=1; [[ $((count%CPU_COUNT)) -eq 0 ]] &amp;amp;&amp;amp; wait; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6343" LastEditorUserId="6343" LastEditDate="2019-12-02T02:34:30.667" LastActivityDate="2019-12-02T02:34:30.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10911" PostTypeId="1" AcceptedAnswerId="10950" CreationDate="2019-12-01T16:18:26.370" Score="3" ViewCount="107" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/410466/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to understand what is the best algorithm for GWAS nowadays.&#xA;I know we have many tools available like Plink and Hail, but currently, what is the best algorithm if I won't use any them? Let's say, write down a script in R or Python from scratch. Which statistical algorithm should I use?&#xA;Is it Linear mixed models (LMMs)? I'm confused as we can have binary phenotypes (case/control) or quantitative phenotypes. LMM seems to address quantitative ones, but can it be used for case/control as well? &#xA;Actually, what is the state of the art for both/each of them? Pair-reviewed papers as references will be appreciated.&#xA;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="6464" LastEditorUserId="73" LastEditDate="2019-12-08T03:33:00.140" LastActivityDate="2020-01-08T15:16:39.133" Title="What is the state of the art for GWAS in terms of the statistical algorithm for either Case/control and Quantitative traits?" Tags="|statistics|gwas|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10912" PostTypeId="2" ParentId="10731" CreationDate="2019-12-01T16:28:56.313" Score="1" Body="&lt;p&gt;I managed to get it done without the gcta64.&#xA;I have written a Jupyter Notebook with the steps: &lt;a href=&quot;https://github.com/bambrozio/bioinformatics/blob/master/utils/1k-genomes-phenotype-simulator.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/bambrozio/bioinformatics/blob/master/utils/1k-genomes-phenotype-simulator.ipynb&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6464" LastActivityDate="2019-12-01T16:28:56.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10914" PostTypeId="1" AcceptedAnswerId="10919" CreationDate="2019-12-02T05:42:56.297" Score="0" ViewCount="3514" Body="&lt;p&gt;I used &lt;code&gt;ComBat()&lt;/code&gt; for batch effect correction in my expression data. basically, that function inputs are expression data, Batch covariate, and Model matrix for the outcome of interest and other covariates besides batch. So, I prepared all inputs based on the instruction of &lt;code&gt;ComBat()&lt;/code&gt; function in 'sva' package. &quot;Pheno_LMS&quot; is my clinical data inclus batchId of my samples and &quot;TCGA_expr_log&quot; is my expression data. my code is below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Pheno_LMS structure:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sample        batchId   age  . . . &#xA;GSM71019.CEL    396     63&#xA;GSM71020.CEL    396     58&#xA;GSM71021.CEL    410     85&#xA;GSM71020.CEL    411     58&#xA;GSM71021.CEL    410     40&#xA;.&#xA;.&#xA;.&#xA;dim(Pheno_LMS)&#xA;[1] 74 37&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;TCGA_expr_log structure:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                   GSM71019.CEL 1      GSM71019.CEL 2        GSM71019.CEL 3  . . . &#xA;    Gene id          &#xA;  ENSG00000000003     10.115170           8.628044                  8.779235  &#xA;  ENSG00000000005     5.345168            5.063598                   5.113116&#xA;  ENSG00000000009     6.348024            6.663625                   6.465892     &#xA;    .&#xA;    .&#xA;    .&#xA;dim(TCGA_expr_log)&#xA;[1] 42817    74&#xA;&#xA;batch = Pheno_LMS$batchId&#xA;modcombat = model.matrix(~1, data=Pheno_LMS)&#xA;TCGA_expr_Co &amp;lt;-  ComBat(as.matrix(TCGA_expr_log),batch = batch,mod = modcombat, par.prior = TRUE,mean.only = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but when I run that function, I got below error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in apply(dat[, batch == batch_level], 1, function(x) { : &#xA;  dim(X) must have a positive length&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, I need anybody comments about my problem. thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S: when i call batch I see:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; batch&#xA; [1] 13 13 13 14 14 14 14 14 14 14 14 6  6  7  7  7  7  7  7  7  7  7  7  7  7  7  7  7 &#xA;[29] 7  8  7  8  11 11 11 14 12 1  6  4  2  3  3  3  3  6  6  1  3  9  8  6  11 7  3  5 &#xA;[57] 4  5  5  4  4  11 8  8  12 12 12 12 12 10 10 10 11 13&#xA;Levels: 1 2 3 4 5 6 7 8 9 10 11 12 13 14&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4915" LastEditorUserId="4915" LastEditDate="2019-12-02T15:00:17.240" LastActivityDate="2019-12-22T15:57:24.660" Title="how to solve “dim(X) must have a positive length” at running ComBat function in R" Tags="|r|batch-effects|combat|sva|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10915" PostTypeId="2" ParentId="10914" CreationDate="2019-12-02T09:30:21.817" Score="0" Body="&lt;p&gt;as suggested in the comment you should verify if the command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dat[, batch == batch_level]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;returns an empty object or not. Apparently, &lt;em&gt;batch&lt;/em&gt; and &lt;em&gt;batch_level&lt;/em&gt; do not have anyting that match and the function tries to run over an empty dataframe.&lt;/p&gt;&#xA;" OwnerUserId="3119" LastActivityDate="2019-12-02T09:30:21.817" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10916" PostTypeId="1" CreationDate="2019-12-02T17:41:46.947" Score="0" ViewCount="72" Body="&lt;p&gt;I am trying to include a new tool to the galaxy main menu, following &lt;a href=&quot;https://galaxyproject.org/admin/tools/add-tool-tutorial/&quot; rel=&quot;nofollow noreferrer&quot;&gt;these&lt;/a&gt; instructions.&lt;br&gt;&#xA;&lt;code&gt;tool_conf.xml&lt;/code&gt; doesn't exists only &lt;code&gt;tool_conf.xml.main&lt;/code&gt; and &lt;code&gt;tool_conf.xml.sample&lt;/code&gt; and none of them seem to be working. I make changes and nothing happens when I run galaxy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I missing?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="2970" LastActivityDate="2019-12-03T08:13:29.000" Title="Add new tool to galaxy" Tags="|ngs|galaxy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10917" PostTypeId="1" CreationDate="2019-12-02T20:35:30.200" Score="5" ViewCount="342" Body="&lt;p&gt;“The Iceman” was a man who lived 5300 years ago and whose body was recovered from an Alpine glacier in 1991.  Some fungal material was recovered from his clothing and sequenced.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ice man : found as a frozen corpse in the Val Senales Glacier in Italy. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thought to be carrying &lt;em&gt;Fomitopsis betulina&lt;/em&gt; (formerly known as Piptoporus betulinus) commonly known as birch polypore, birch bracket, or razor strop) is a common Basidiomycota brown rot macrofungus growing on decaying birch wood.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Other species known : &lt;em&gt;Fomes fomentarius&lt;/em&gt; (commonly known as the tinder fungus, false tinder fungus, hoof fungus,] tinder conk, tinder polypore or ice man fungus)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given two DNA fragments from NCBI genbank, try to find what species these DNA fragments belongs to or what species these fragments are most closely related to. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Z54169.1 Iceman fungal clone 'Sim C-a1' 18S rRNA gene (partial)&#xA;GTATATTAAAGTTGTTGCAGTTAAAAAGCTCGTAGTTGAAACTTGGGCCTGGCTGGCAGGTCCGCCTCAC&#xA;CGCGTGTACTTGTCCGGCCGGGCCTTTCCTTCTGGAGAACCTCATGCCCTTCACTGGGTGTGTTGGGGAA&#xA;CCAGGACTTTTACTTTGAAAAAATTAGAGTGTTCAAAGCAGGCCTTTGCTCGAATACGTTAGCATGGAAT&#xA;AATAGAATAGGACGTGCGGTCCTATTTTGTTGGTTTCTAGGACCGCCGTAATGATTAATAGGGACAGTCG&#xA;GGGGCATCAGTATTCAATTGTCAGAGGTGAAATTCTTGGATTTATTGAAGACTAACTACTGCGAAAGCAT&#xA;TTGCCAAGGATGTTTTCATTAATCAGTGAACGAAAGTTAGGGGATCGAAGACGATCAGATACCGTCGTAG&#xA;TCTTAACCGTAAACTATGCCGACTAGGGATCGGGCGATGTTCTTTTTCTGACTCGCTCGGCACCTTACGA&#xA;GAAATCAAAGTTTTTGG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Z54155.1 Iceman fungal clone 'Sim C-a2' 18S rRNA gene (partial)&#xA;GTATATTAAAGTTGTTGCAGTTAAAAAGCTCGTAGTTGAACCTTGGGTCTGGCTGGCAGGTCCGCCTCAC&#xA;CGCGTGTACTTGTCCGGCCGGGCCTTCCTTCTGGAGAACCTCATGTCTTTTATTAGGCGTGTTGGGGACC&#xA;AGAACTTTTACTTTGAAAAAATTAGAGTGTTCAAAGCAGGCCTTTGCTCGAATACGTTAGCATGGAATAA&#xA;TAGAATAGGACGTGCGGTCCTATTTTGTTGGTTTCTAGGACCGCCGTAATGATTAATAGGGACAGTCGGG&#xA;GGCATCAGTATTCAATTGTCAGAGGTGAAATTCTTGGATTTATTGAAGACTAACTACTGCGAAAGCATTT&#xA;GCCAAGGATGTTTTCATTAATCAGTGAACGAAAGTTAGGGGATCGAAGACGATCAGATACCGTCGTAGTC&#xA;TTAACCGTAAACTATGCCGACTAGGGATCGGGCGATGTTCTTTTTCTGACTCGCTCGGCACCTTACGAGA&#xA;AATCAAAGTTTTTGG&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now I know I can just google what species of mushrooms (which is where the above background info is from). But say that I was unable to and had to just go off the DNA fragments in order to find the species. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried &lt;code&gt;BLASTn&lt;/code&gt;, but I'm not sure if it is the right tool or if I need to properly calibrate the parameters to use it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;so far, I've tried:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Under “Choose Search Set:&#xA;Database : Non-redundant protein sequences (nr)&#xA;Organisms excluded: Animalia (taxid:33208), Plantae, Prokaryota  (taxid:2), Protozoa &#xA;Organisms included: fungi  (taxid:4751) &#xA;Included: Models (XM/XP),   Uncultured/environmental sample sequences&#xA;Under Program Selection:  Optimized for Highly similar sequences (megablast)&#xA;&#xA;Algorithm Parameters (General Parameters, Scoring Parameters, Filters and Masking, &#xA;PS/PHI?DELTA BLAST) &#xA;&#xA;are all kept default. &#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6539" LastEditorUserId="3848" LastEditDate="2020-04-29T19:19:10.167" LastActivityDate="2020-09-26T21:02:35.267" Title="which NCBI tool is optimized to identify a species from a DNA fragment?" Tags="|blast|phylogeny|phylogenetics|genbank|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10918" PostTypeId="1" CreationDate="2019-12-02T20:38:33.137" Score="1" ViewCount="238" Body="&lt;p&gt;I looking for software to conduct a haplotype network using cytochrome oxidase (CO1) mitochondrial sequence data to study population structure in Anopheles mosquitoes. &lt;/p&gt;&#xA;" OwnerDisplayName="user55763" LastEditorUserId="3848" LastEditDate="2020-04-30T09:53:59.527" LastActivityDate="2020-05-30T10:02:15.307" Title="What software can I use to develop a population haplotype network?" Tags="|phylogeny|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10919" PostTypeId="2" ParentId="10914" CreationDate="2019-12-03T08:10:40.947" Score="1" Body="&lt;p&gt;&lt;code&gt;batch = Pheno_LMS$batchId&lt;/code&gt; should be &lt;code&gt;batch = Pheno_LMS$BatchId&lt;/code&gt;, if you fix that then at least the code you showed works just fine.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-12-03T08:10:40.947" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10920" PostTypeId="2" ParentId="10916" CreationDate="2019-12-03T08:13:29.000" Score="1" Body="&lt;p&gt;The .main and .sample file files usually come with default installations and may have been removed by whoever setup this Galaxy instance. You can look at the originals online and just base your own file on those: &lt;a href=&quot;https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/config/sample/tool_conf.xml.sample&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/galaxyproject/galaxy/blob/dev/lib/galaxy/config/sample/tool_conf.xml.sample&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-12-03T08:13:29.000" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10921" PostTypeId="1" CreationDate="2019-12-03T17:48:20.740" Score="1" ViewCount="184" Body="&lt;p&gt;I'm new on working with genetics data and I'm just wondering if I can use the &lt;code&gt;lm&lt;/code&gt; function when I'm building my models or do I have to use &lt;code&gt;lmFit&lt;/code&gt; function in limma package to build the models and what is the difference between the two packages in terms of performing linear regression models &lt;/p&gt;&#xA;" OwnerUserId="6590" LastActivityDate="2020-05-02T08:07:10.640" Title="Can I use a regular liner regression model when I'm working with DNA methylation data?" Tags="|limma|linear-regression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10924" PostTypeId="2" ParentId="10921" CreationDate="2019-12-04T07:06:01.633" Score="2" Body="&lt;p&gt;You can use either, but &lt;code&gt;lmFit&lt;/code&gt; has the benefit of returning an object that can be used with &lt;code&gt;eBayes()&lt;/code&gt; so you can pool information across genes/probes/whatever. &lt;code&gt;lm()&lt;/code&gt; is a base R function applicable basically everywhere. &lt;code&gt;lmFit()&lt;/code&gt; if from the limma package, so originally intended for microarray data, though these days pretty much everything omics is analyzed with it since sample sizes tend to be small so everyone want's to pool information for more statistical power.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, if you have methylation data then you'll want to fit data in logit space.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-12-04T07:06:01.633" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10925" PostTypeId="1" AcceptedAnswerId="10930" CreationDate="2019-12-04T10:11:34.360" Score="-1" ViewCount="573" Body="&lt;p&gt;The WGCNA analysis &lt;a href=&quot;https://avesis.yildiz.edu.tr/resume/downloadfile/zeyneb?key=0936b44a-e367-4ed1-8f8b-cdf3cb4db7ae&quot; rel=&quot;nofollow noreferrer&quot;&gt;page48 of that link&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this analysis they are comparing the Preservation of modules between human and chimpanzee brain networks where i see the number of human modules and chimp modules are exact same in number as well as the color they are labeled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So is it necessary that two different condition where, Im comparing need to have equal number of modules to check the module preservation? Because the power would be different,then the downstream output would be different for two set of condition ,let say if Im trying to see what are the module conserved in my Control vs a disease condition ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 4 different condition such as CONTROL,BPD,SCZ and MDD all[micro-array expression data] brain disorder with samples number of 50,54,54,48.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what im trying to see is what are the modules are getting preseved in each diseased state when they are compared to Control condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What i have done is I ran WGCNA on each condition separately and found what are the modules are there in different condition. So here I see the number of modules that are observed in each individual condition varies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now if I do module preservation is it necessary to have similar number of modules from each condition ? Or its fine to have different module number from different condition and can still run the module preservation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;These are gene expression network.&#xA;Modules are obtained as explained in the WGCNA tutorial.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Do you mean &quot;conservation&quot; or &quot;preservation&quot;?&lt;/strong&gt; I mean module preservation &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suggestion would be highly appreciated &lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2019-12-04T16:30:53.293" LastActivityDate="2019-12-04T17:43:22.817" Title="WGCNA module preservation doubt" Tags="|wgcna|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10926" PostTypeId="1" AcceptedAnswerId="10931" CreationDate="2019-12-04T12:42:55.357" Score="1" ViewCount="888" Body="&lt;p&gt;I have a dataset (view/download) &lt;a href=&quot;https://drive.google.com/file/d/1uqgWchDCwiwfzSRajw3bmSRQXjxOtmDR/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I need to run PCA on this data set and need to illustrate a 3D visualisation of the three main PCs using plot3d() package.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have looked for more information before coming here on which package to use, some people use princomp and some us prcomp (in university we have personally been using prcomp). The confusing part is were to move forward with this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The full question is &quot;Illustrate a 3D visualisation of the three main PCs using plot3d() package in R. Use the following colours for samples (data points) belonging to the various immune subgroups/subtypes (C1: red, C2: yellow, C3: green, C4: cyan, C5: blue, C6: purple)&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After reading in the CSV and loading rgl library&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the current state I have removed the rows with the subgroups: see code below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Remove last row from dataset&#xA;dataMinusLastRow = data[1:(nrow(data)-1),]&#xA;&#xA;# Convert dataset to numeric and apply&#xA;coltemp &amp;lt;- colnames(dataMinusLastRow)&#xA;pcaData = t(apply(dataMinusLastRow, 1, as.numeric))&#xA;colnames(pcaData) &amp;lt;- coltemp&#xA;&#xA;# PCA&#xA;pc = prcomp(t(pcaData), scale=T, center=T)&#xA;summary(pc) # From this we can see that PC184 accounts for 95% of cumulative proportion&#xA;&#xA;# Getting the cumulative proportion in numeric dataset&#xA;vars &amp;lt;- apply(pc$x, 2, var)  &#xA;props &amp;lt;- vars / sum(vars)&#xA;pc.cs &amp;lt;- cumsum(props) # Cumulative Proportion&#xA;pc.index&amp;lt;-min(which(pc.cs&amp;gt;0.95)) # Index of which cumulative proportion &amp;gt; 95%&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think the main part which confuses me is that it asks for the three main PC's which I am not sure what they are and have had not much support from university with information regarding this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks to anyone who can help in any shape/form!&lt;/p&gt;&#xA;" OwnerUserId="6595" LastActivityDate="2019-12-04T17:44:36.183" Title="Illustrate a 3D visualisation of the three main PCs using plot3d() package in R?" Tags="|r|pca|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10927" PostTypeId="1" AcceptedAnswerId="10928" CreationDate="2019-12-04T16:44:33.743" Score="1" ViewCount="101" Body="&lt;p&gt;I have 200 &lt;code&gt;.txt&lt;/code&gt; files on a Linux system. In each file, I have the reference allele in the &lt;code&gt;REF&lt;/code&gt; column and INDELs in the &lt;code&gt;ALT&lt;/code&gt; column.&#xA;Let's say I have &lt;code&gt;A&lt;/code&gt; in the REF column and &lt;code&gt;AC&lt;/code&gt; in the ALT. I want to remove  the &lt;code&gt;A&lt;/code&gt; from the ALT column because that is a repetition of the &lt;code&gt;A&lt;/code&gt; reference allele. For example, if I have &lt;code&gt;C&lt;/code&gt; in the REF column and &lt;code&gt;CCAT&lt;/code&gt; in the ALT column, I only want CAT to remain in the ALT column.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what my files look like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CHROM   REF ALT&#xA;chr1    C   CT&#xA;chr1    C   CT&#xA;chr1    C   CA&#xA;chr1    C   CT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can someone help me please?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solution from @Terdon&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my original file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT&#xA;chr4    5708160 .   G   GATAAC&#xA;chr4    30280292    .   C   CACGTAT&#xA;chr6    166253422   .   C   CCTTTCT&#xA;chr11   21931340    .   A   AACACACGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is what happened after running your kindly suggestion &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[fi1d18@cyan01 indel]&lt;span class=&quot;math-container&quot;&gt;$ awk -F&quot;\t&quot; -vOFS=&quot;\t&quot; '{ if(substr($5,1,1) == $4){$5=substr($&lt;/span&gt;5,4); }}1' indels.vcf &amp;gt; out.vcf&#xA;&#xA;#CHROM  POS ID  REF ALT&#xA;chr4    5708160 .   G   AAC&#xA;chr4    30280292    .   C   GTAT&#xA;chr6    166253422   .   C   TTCT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think more than one allele being removed from ALT column, is not it?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="298" LastEditDate="2019-12-04T17:19:41.103" LastActivityDate="2021-05-23T13:40:06.183" Title="Remove repeated ALT allele from the REF field of a vcf-like file" Tags="|vcf|text-processing|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10928" PostTypeId="2" ParentId="10927" CreationDate="2019-12-04T16:51:04.373" Score="4" Body="&lt;p&gt;I am making a couple of assumptions here: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the delimiter is tab&lt;/li&gt;&#xA;&lt;li&gt;the reference allele is always present in the ALT column&lt;/li&gt;&#xA;&lt;li&gt;the reference allele in the ALT column is always in the first position&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;With those in mind, you can do this using &lt;code&gt;awk&lt;/code&gt; as follows: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN{FS=&quot;\t&quot;;OFS=&quot;\t&quot;}{$3=substr($3,2)}{print $0}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Update 1: Using the new VCF style file that @Angel provided, here's how I would do it. This is not significantly different from what @terdon proposed below. The code below will work even if the reference allele is &gt;1 nt.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat test.vcf&#xA;#CHROM  POS     ID      REF     ALT&#xA;chr4    5708160 .       G       GATAAC&#xA;chr4    30280292        .       C       CACGTAT&#xA;chr6    166253422       .       C       CCTTTCT&#xA;chr11   21931340        .       A       AACACACGC&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;$ awk 'BEGIN{FS=&quot;\t&quot;;OFS=&quot;\t&quot;}{l=length($4); if ($4==substr($5,1,l)) {$5=substr($5,l+1); print $0} else {print $&lt;/span&gt;0}}' test.vcf&#xA;#CHROM  POS     ID      REF     ALT&#xA;chr4    5708160 .       G       ATAAC&#xA;chr4    30280292        .       C       ACGTAT&#xA;chr6    166253422       .       C       CTTTCT&#xA;chr11   21931340        .       A       ACACACGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3577" LastEditorUserId="3577" LastEditDate="2019-12-04T17:41:49.570" LastActivityDate="2019-12-04T17:41:49.570" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10929" PostTypeId="2" ParentId="10927" CreationDate="2019-12-04T16:56:01.950" Score="2" Body="&lt;p&gt;Another awk approach. This should work on a normal VCF file (it will print the headers) as well as your example and only changes the ALT field if its first character is the same as the 2nd field:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk -F&quot;\t&quot; -vOFS=&quot;\t&quot; '{ if(substr($5,1,1) == $4){$5=substr($&lt;/span&gt;5,2); }}1' ff.vcf &#xA;#CHROM  POS ID  REF ALT&#xA;chr4    5708160 .   G   ATAAC&#xA;chr4    30280292    .   C   ACGTAT&#xA;chr6    166253422   .   C   CTTTCT&#xA;chr11   21931340    .   A   ACACACGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To run this on all of your files, assuming they are the only files in the current directory with a &lt;code&gt;.txt&lt;/code&gt; extension, you can do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for file in *txt; do&#xA;    awk -F&amp;quot;\t&amp;quot; -vOFS=&amp;quot;\t&amp;quot; '{ if(substr(&lt;span class=&quot;math-container&quot;&gt;$5,1,1) == $&lt;/span&gt;4){&lt;span class=&quot;math-container&quot;&gt;$5=substr($&lt;/span&gt;5,2); }}1' &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$file&quot; &amp;gt; &quot;fixed.$&lt;/span&gt;file&amp;quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will loop over all &lt;code&gt;*.txt&lt;/code&gt; files in the current directory. For each &lt;code&gt;foo.txt&lt;/code&gt;, it will create &lt;code&gt;fixed.foo.txt&lt;/code&gt; with the modified data.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="73" LastEditDate="2021-05-23T13:40:06.183" LastActivityDate="2021-05-23T13:40:06.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10930" PostTypeId="2" ParentId="10925" CreationDate="2019-12-04T17:43:22.817" Score="1" Body="&lt;p&gt;The networks you want to compare do not have to have the same number of modules. (The human and chimp networks just happen to have the same number of modules.) In fact, you only need modules in the reference network; modules from the test network are not used at all and need not be defined. You may want to read the original article describing module preservation, it would help you understand how the method works, what it needs and what it does not.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2019-12-04T17:43:22.817" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10931" PostTypeId="2" ParentId="10926" CreationDate="2019-12-04T17:44:36.183" Score="1" Body="&lt;p&gt;Regarding your question on how to decide which PCs to choose for plotting, &lt;code&gt;prcomp&lt;/code&gt; is ordering PCs by their proportions in the variance. For example, using the dataset &lt;code&gt;iris&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;iris_pc &amp;lt;- prcomp(iris[,1:4], scale  = TRUE, center = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and you get &lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; summary(iris_pc)&#xA;Importance of components:&#xA;                          PC1    PC2     PC3     PC4&#xA;Standard deviation     1.7084 0.9560 0.38309 0.14393&#xA;Proportion of Variance 0.7296 0.2285 0.03669 0.00518&#xA;Cumulative Proportion  0.7296 0.9581 0.99482 1.00000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;BTW, using the &lt;code&gt;princomp&lt;/code&gt; instead of &lt;code&gt;prcomp&lt;/code&gt; gives similar results:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;iris_pc2 &amp;lt;- princomp(iris[,1:4], cor=TRUE, scores=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; summary(iris_pc)&#xA;Importance of components:&#xA;                          Comp.1    Comp.2     Comp.3      Comp.4&#xA;Standard deviation     1.7083611 0.9560494 0.38308860 0.143926497&#xA;Proportion of Variance 0.7296245 0.2285076 0.03668922 0.005178709&#xA;Cumulative Proportion  0.7296245 0.9581321 0.99482129 1.000000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, this means if you want to represent the main 3 PCs, you have to pick the first three PCs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, based on your dataset, if you take a look at the output of &lt;code&gt;prcomp&lt;/code&gt;, you will see:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;pc = prcomp(t(pcaData), scale=T, center=T)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; summary(pc)&#xA;Importance of components:&#xA;                          PC1    PC2     PC3     PC4     PC5     PC6     PC7    PC8     PC9    PC10   PC11    PC12&#xA;Standard deviation     9.3653 7.9806 5.46230 5.02379 3.95681 3.36495 3.21719 3.1042 2.98449 2.73437 2.6529 2.56841&#xA;Proportion of Variance 0.1993 0.1447 0.06781 0.05736 0.03558 0.02573 0.02352 0.0219 0.02024 0.01699 0.0160 0.01499&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can see that the first three PCs count for only 41% of the total variance. This can be explain by the diversity of your samples (coming from different cancers for example). So it means that your graphic representation won't be perfect to separate clusters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, the graphic representation of the cumulative proportion of variances accross PC:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;plot(pc, type = &quot;l&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XZtpc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XZtpc.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next for the plotting and color attribution, I get a lot of trouble to extract the values from your &lt;code&gt;Subgroups&lt;/code&gt; row.... for some reasons, I was not able to convert them into a character vector... weird ... anyway, I re-defined groups based on the code you put in patients ids:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;patients = as.character(colnames(data))&#xA;grp = c(rep(&quot;C1&quot;,length(grep(&quot;.C1_&quot;,patients))),&#xA;        rep(&quot;C2&quot;,length(grep(&quot;.C2_&quot;,patients))),&#xA;        rep(&quot;C3&quot;,length(grep(&quot;.C3_&quot;,patients))),&#xA;        rep(&quot;C4&quot;,length(grep(&quot;.C4_&quot;,patients))),&#xA;        rep(&quot;C5&quot;,length(grep(&quot;.C5_&quot;,patients))),&#xA;        rep(&quot;C6&quot;,length(grep(&quot;.C6_&quot;,patients))))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For plotting results of PCA, you can use plot3d from &lt;code&gt;rgl&lt;/code&gt; package for example. It gives you an interactive plot. &lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(rgl)&#xA;colors = c(&quot;red&quot;,&quot;yellow&quot;, &quot;green&quot;,&quot;cyan&quot;, &quot;blue&quot;,&quot;purple&quot;)&#xA;plot3d(pc$x[,1:3], col =colors[as.factor(grp)], main = &quot;pc / prcomp&quot;)&#xA;legend3d(&quot;topright&quot;,legend = unique(grp),col = colors, pch = 16, cex = 2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And here the snapshot of your plot:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mTM9M.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mTM9M.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that it helps you to figure it out the solution of your problem&lt;/p&gt;&#xA;" OwnerUserId="6437" LastActivityDate="2019-12-04T17:44:36.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10932" PostTypeId="1" AcceptedAnswerId="10934" CreationDate="2019-12-04T18:35:36.773" Score="-1" ViewCount="258" Body="&lt;p&gt;I have 200 .vcf files like this on a Linux system:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT&#xA;chr4    5708160 .   G   GATAAC&#xA;chr4    30280292    .   C   CACGTAT&#xA;chr6    166253422   .   C   CCTTTCT&#xA;chr11   21931340    .   A   AACACACGC&#xA;chr12   118404422   .   G   GCACACA&#xA;chr15   22672047    .   T   TAAGTAGATTTAAG&#xA;chr16   32656293    .   ATGATT  A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to modify my .vcf files in this way&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance if the reference allele in &lt;code&gt;REF column&lt;/code&gt; is &lt;code&gt;TCTC&lt;/code&gt; and alteration allele in &lt;code&gt;ALT column&lt;/code&gt; is &lt;code&gt;T&lt;/code&gt;, because T is the first common nucleotide in both reference and alteration so we remove &lt;code&gt;T&lt;/code&gt; from &lt;code&gt;ALT column&lt;/code&gt; so REF would remain as &lt;code&gt;TCTC&lt;/code&gt; and ALT would be &lt;code&gt;-&lt;/code&gt; . In total I mean removing whatever in ALT which is common with the first allele in REF.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help please?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I got by @haci's code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(var.annovar.maf2[1:6,1:13])&#xA;   Tumor_Sample_Barcode Chromosome Start_Position End_Position Reference_Allele Tumor_Seq_Allele2 Hugo_Symbol Variant_Classification&#xA;1:                  new       chr1       89616151     89616151                G                 T        GBP7      Missense_Mutation&#xA;2:                  new      chr16       20556547     20556547                C                 G      ACSM2B      Missense_Mutation&#xA;3:                  new      chr17       66992092     66992092                C                 A       ABCA9      Missense_Mutation&#xA;4:                  new      chr20       13251339     13251339                C                 T        ISM1                 Silent&#xA;5:                  new       chr1          56987        56987               CA                 -           .                   &amp;lt;NA&amp;gt;&#xA;6:                  new       chr1        4133415      4133415               AG                 -           .                   &amp;lt;NA&amp;gt;&#xA;             tx   exon txChange aaChange Variant_Type&#xA;1:    NM_207398  exon6  c.C733A  p.L245I          SNP&#xA;2: NM_001105069 exon10 c.G1213C  p.G405R          SNP&#xA;3:    NM_080283 exon26 c.G3499T p.G1167C          SNP&#xA;4:    NM_080826  exon2  c.C327T  p.N109N          SNP&#xA;5:         &amp;lt;NA&amp;gt;   &amp;lt;NA&amp;gt;     &amp;lt;NA&amp;gt;     &amp;lt;NA&amp;gt;          DEL&#xA;6:         &amp;lt;NA&amp;gt;   &amp;lt;NA&amp;gt;     &amp;lt;NA&amp;gt;     &amp;lt;NA&amp;gt;          DEL&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-12-05T13:30:06.873" LastActivityDate="2019-12-05T13:30:06.873" Title="Modifying .vcf files" Tags="|vcf|text-processing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10933" PostTypeId="1" CreationDate="2019-12-04T18:41:26.590" Score="2" ViewCount="181" Body="&lt;p&gt;I am writing a Python script to automate a 5PL Standard Curve for ELISA data, using an XLSX file template. I am using &lt;code&gt;pandas&lt;/code&gt; dataframes to hold the 96 wells as different rows, then in a separate dataframe, holding Standard Curve data (not all wells need a column for known concentration). I am also using &lt;code&gt;scipy&lt;/code&gt; and &lt;code&gt;numpy&lt;/code&gt; for analysis, then using &lt;code&gt;matplotlib&lt;/code&gt;.pyplot for plotting the curve (to eventually be output to the XLSX), though I am open to suggestions.&lt;/p&gt;&#xA;&lt;p&gt;So far, the program correctly takes the input, generates the aforementioned dataframes, and makes initial guesses for the 5PL's parameters. I was attempting to modify the 4PL code given &lt;a href=&quot;https://people.duke.edu/%7Eccc14/pcfb/analysis.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; to work with my standard curve, using all actual standard replicates to optimize the curve, and obviously using 5PL instead of 4PL. However, when I use &lt;code&gt;scipy&lt;/code&gt;'s least_squares function for parameter optimization, I receive the following errors/traceback:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;C:\Users\Jake\Anaconda3\lib\site-packages\ipykernel_launcher.py:6: RuntimeWarning: overflow encountered in power&#xA;  &#xA;&#xA;---------------------------------------------------------------------------&#xA;ValueError                                Traceback (most recent call last)&#xA;&amp;lt;ipython-input-6-70b754f01d56&amp;gt; in &amp;lt;module&amp;gt;&#xA;      4 &#xA;      5 # Fit equation using least squares optimization&#xA;----&amp;gt; 6 plsq = least_squares(fun = residuals, jac = 'cs', x0 = p0, bounds = (0.0001, 100000), args=(y_meas, x))&#xA;&#xA;~\Anaconda3\lib\site-packages\scipy\optimize\_lsq\least_squares.py in least_squares(fun, x0, jac, bounds, method, ftol, xtol, gtol, x_scale, loss, f_scale, diff_step, tr_solver, tr_options, jac_sparsity, max_nfev, verbose, args, kwargs)&#xA;    793 &#xA;    794     if not in_bounds(x0, lb, ub):&#xA;--&amp;gt; 795         raise ValueError(&amp;quot;`x0` is infeasible.&amp;quot;)&#xA;    796 &#xA;    797     x_scale = check_x_scale(x_scale, x0)&#xA;&#xA;ValueError: `x0` is infeasible.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I was initially receiving these errors with the aforementioned article's suggested scipy.least_sq function, but I assumed it was minimizing exponents to 0 and encountering an overflow, so I swapped to scipy.least_squares instead due to its bounds argument. Code in question is below. Any help would be greatly appreciated! Thank you so much! And if this question is better suited to stats.SE, just let me know!&lt;/p&gt;&#xA;&lt;p&gt;(I did find &lt;a href=&quot;https://github.com/lmfit/lmfit-py&quot; rel=&quot;nofollow noreferrer&quot;&gt;lmfit-py&lt;/a&gt;, though I was hoping to complete this project on my own if possible to get a better understanding of the regression. Still open to using it instead if that seems to be a better option.)&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;from scipy.optimize import least_squares&#xA;&#xA;def logistic5(x, A, B, C, D, E):&#xA;    &amp;quot;&amp;quot;&amp;quot;5PL logistic equation&amp;quot;&amp;quot;&amp;quot;&#xA;    return D + ((A-D)/(np.power((1 + np.power((x / C), B)), E)))&#xA;&#xA;def residuals(p, y, x):&#xA;    &amp;quot;&amp;quot;&amp;quot;Deviations of data from fitted 5PL Curve&amp;quot;&amp;quot;&amp;quot;&#xA;    A, B, C, D, E = p&#xA;    err = y - logistic5(x, A, B, C, D, E)&#xA;    return err&#xA;&#xA;def peval(x, p):&#xA;    &amp;quot;&amp;quot;&amp;quot;Evaluated value at x with current parameters&amp;quot;&amp;quot;&amp;quot;&#xA;    A, B, C, D, E = p&#xA;    return logistic5(x, A, B, C, D, E)&#xA;&#xA;x = []&#xA;y_meas = []&#xA;for ind, row in std_df.iterrows(): # Iterate through rows in std dataframe.&#xA;    for rep in std_df_rep_cols: # Iterate through replicate columns.&#xA;        if not pd.isnull(row[rep]): # If there is a value in the cell...&#xA;            x.append(row['conc']) # Add an iteration of the respective conc. to the x list.&#xA;            y_meas.append(row[rep]) # Add the cell's value to the y_meas list.&#xA;&#xA;x = np.asarray(x)&#xA;y_meas = np.asarray(y_meas)&#xA;            &#xA;print(len(x))&#xA;print(len(y_meas))&#xA;&#xA;print(x)&#xA;print(y_meas)&#xA;&#xA;# Initial approximations of parameters&#xA;A = np.amax([np.amin(minBlk_ser)] + [0]) # Min asymptote&#xA;D = np.amax(y_meas) # Max asymptote&#xA;B = (D - A) / np.amax(x) # Steepness&#xA;C = np.amax(x) / 2 # Inflection Point, conc at which y = (D - A) / 2&#xA;E = 1 # Asymmetry factor&#xA;print(A, B, C, D, E)&#xA;&#xA;p0 = [A, B, C, D, E] # List containing initial guesses as arg for least_squares&#xA;y_true = logistic5(x, A, B, C, D, E) # Assumed &amp;quot;true&amp;quot; curve based on initial params&#xA;&#xA;# Fit equation using least squares optimization&#xA;plsq = least_squares(fun = residuals, x0 = p0, bounds = (0.0001, 100000), args=(y_meas, x))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6597" LastEditorUserId="3848" LastEditDate="2020-07-09T20:09:38.250" LastActivityDate="2020-07-09T20:09:38.250" Title="Trouble optimizing Five Parameter Logistic (5PL) Standard Curve for ELISA data using Python" Tags="|python|statistics|linear-regression|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10934" PostTypeId="2" ParentId="10932" CreationDate="2019-12-04T23:56:33.340" Score="1" Body="&lt;p&gt;I am pretty sure that there is a reason that the first nucleotide of the &lt;code&gt;ALT&lt;/code&gt; is the same as that of &lt;code&gt;REF&lt;/code&gt; in &lt;code&gt;VCF&lt;/code&gt; files, however I was not able to find a reference for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really need to do this for some reason, you will need to compare the alt and ref entries base by base starting from base 1 of each and until a mismatch. Here is some R code doing that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(data.table)&#xA;my_csv &amp;lt;- fread(&quot;example_vcf.txt&quot;)&#xA;print(my_csv)&#xA;&#xA;# loop over each row&#xA;for(i in 1:nrow(my_csv)) {&#xA;&#xA;  # make &quot;strings&quot; from the ref and alt columns, make &quot;arrays&quot; of letters out of these strings&#xA;  ref &amp;lt;- as.character(my_csv[i, &quot;REF&quot;])&#xA;  ref_array &amp;lt;- unlist(strsplit(as.character(my_csv[i, &quot;REF&quot;]), split = &quot;&quot;))&#xA;&#xA;  alt &amp;lt;- as.character(my_csv[i, &quot;ALT&quot;])&#xA;  alt_array &amp;lt;- unlist(strsplit(as.character(my_csv[i, &quot;ALT&quot;]), split = &quot;&quot;))&#xA;&#xA;  # which of the ref alt-pair shorter and how long&#xA;  shortest &amp;lt;- c(nchar(ref), nchar(alt))[which.min(c(nchar(ref), nchar(alt)))]&#xA;&#xA;  # compare ref and alt base by base, remove from alt if same nucleotide present in both, stop at first mismatch&#xA;  index &amp;lt;- 1&#xA;  new_alt &amp;lt;- alt_array&#xA;  while(index &amp;lt;= shortest) {&#xA;    if (ref_array[index] == alt_array[index]) {&#xA;      new_alt &amp;lt;- new_alt[-1]&#xA;    } else {&#xA;      break()&#xA;    }&#xA;    index &amp;lt;- index + 1&#xA;  }&#xA;&#xA;  my_csv[i,&quot;ALT&quot;] &amp;lt;- paste(new_alt, collapse = &quot;&quot;)&#xA;&#xA;  # use &quot;-&quot; if alt is empty&#xA;  if (my_csv[i,&quot;ALT&quot;] == &quot;&quot;) {&#xA;    my_csv[i,&quot;ALT&quot;] &amp;lt;- &quot;-&quot;&#xA;  }&#xA;}&#xA;&#xA;print(my_csv)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have added some test cases at the end of the sample piece in your sample and the code is adding another column so that the changes made to the &lt;code&gt;ALT&lt;/code&gt; are easier to see:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; print(my_csv)&#xA;    #CHROM       POS ID    REF            ALT       NEW_ALT&#xA; 1:   chr4   5708160  .      G         GATAAC         ATAAC&#xA; 2:   chr4  30280292  .      C        CACGTAT        ACGTAT&#xA; 3:   chr6 166253422  .      C        CCTTTCT        CTTTCT&#xA; 4:  chr11  21931340  .      A      AACACACGC      ACACACGC&#xA; 5:  chr12 118404422  .      G        GCACACA        CACACA&#xA; 6:  chr15  22672047  .      T TAAGTAGATTTAAG AAGTAGATTTAAG&#xA; 7:  chr16  32656293  . ATGATT              A             -&#xA; 8: chrXXX       XXX  .   TCTC              T             -&#xA; 9: chrXXX       XXX  .   TCTC           TCTC             -&#xA;10: chrXXX       XXX  .   TCTC             TA             A&#xA;11: chrXXX       XXX  .   TCTC            TAT            AT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-12-04T23:56:33.340" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10935" PostTypeId="2" ParentId="10917" CreationDate="2019-12-05T01:15:09.547" Score="1" Body="&lt;p&gt;My conclusion from the tree is the fungi appears to be genuine and not an extant contaminant (which is very easily done). It is not Fomitopsis betulina because this is division Basidiomycota. The fungi here is division Ascomycota and these are very distinct from Basidiomycota. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;There have been many examples of contamination in ancient DNA work, but this doesn't appear to be one of them. The rationale is given below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mw4No.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mw4No.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first fragment was not interesting and described 100% identity to ascomycetes. The tree above is a simple distance tree from the second fragment of 18S which shows it is an outgroup to the ascomycetes (green terminal taxa). The ancient fungi contains 3 (I think) unique nucleotide residues that have not been sequenced in any extant ascomycetes to date and sequences are unique to any known species. The location of the 3 unique nucleotides is tightly clustered within the locus with 100% identities either side. This is significant because it demonstrates it is not a PCR mosaic artefact (yep its been done in ancient DNA work and subsequently proven artifactual). In the latter PCR mosaic artefact I'm referring to (which I wouldn't cite), was a fungi/bacterial mosaic from memory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the homology of 18S in the extant ascomycetes I would say this appears to be a new genus, however you would need to be sure this was not sequencing error and this is very old DNA, which is subject to degradation even when frozen. The implication of the tree is quite unusual, it implies widespread extinction of this group of fungi in recent history (5000 years is slight in evolutionary terms) BUT it could be a sampling bias, i.e. no one has sequenced those fungi and NCBI is not representative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume that you have performed phenotypic/morphological analysis on the spore to believe it is Basidiomycota.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fragment in yellow is the query and the sequence next to it was the sequence already deposited on Genbank of this ancient isolate. You would need to look up the biology and ecology of ascomycete to understand its significance, but it is in a lineage of its own, so maybe difficult to interpret by comparison to extant species. You could concatenate the two alignments based on each fragment, but the problem with 18S is that is a hard to model and you'd need to perform secondary RNA structure modelling to assess whether these mutations would be viable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In conclusion, it definitely looks genuine but it is not the species you think it is, in fact not even close.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-12-05T17:41:37.910" LastActivityDate="2019-12-05T17:41:37.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10936" PostTypeId="2" ParentId="10918" CreationDate="2019-12-05T01:57:50.220" Score="1" Body="&lt;p&gt;The software you are looking for is Network 5.0.0.3&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Phylogenetic Network Software . Network © Copyright Fluxus Technology Ltd 1999–2018. Retrieved from &lt;a href=&quot;https://www.fluxus-engineering.com/sharenet.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.fluxus-engineering.com/sharenet.htm&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whilst its old code it is still regularly used in pop gen for example there guys &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6305635/#mgg3499-bib-0022&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; recently used it. It will do the job, although personally I think Google Fusion tables are very cool, but that is complicated to use properly. R has some cool stuff in this context BTW, but in context I'd keep it simple (i.e. Network 5)&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-12-05T01:57:50.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10937" PostTypeId="1" AcceptedAnswerId="10940" CreationDate="2019-12-05T06:17:51.130" Score="0" ViewCount="46" Body="&lt;p&gt;Did anyone know whether the GO terms can include more detail information? Like I can get the DotPlot of the GO terms as below:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/B1iWS.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/B1iWS.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;The problem is that some of the genes, in the GO terms, is more about sinoatrial-node (SAN). So whether there is any method that we can do to dig deeper into the GO terms?&lt;/p&gt;&#xA;" OwnerUserId="3171" LastActivityDate="2019-12-05T10:03:59.290" Title="about GO terms's name" Tags="|rna-seq|go-enrichment|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10938" PostTypeId="2" ParentId="10937" CreationDate="2019-12-05T08:16:28.100" Score="0" Body="&lt;p&gt;This ends up not being a question of what method to use, but rather how detailed the GO term database you're using is. GO terms are maintained by the &lt;a href=&quot;https://github.com/bioconda/bioconda-recipes/pull/18701&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gene Ontology Consortium&lt;/a&gt; and are relatively broad for the most part, with a few more specific molecular signaling pathways included. If you &lt;a href=&quot;http://amigo.geneontology.org/amigo/search/ontology?q=atrial&quot; rel=&quot;nofollow noreferrer&quot;&gt;search for the ontologies for &quot;atrial&quot;&lt;/a&gt; you'll find a number of relevant GO terms. The question mostly becomes whether those are sufficient for your needs. If they are, make sure you include &quot;biological process&quot; in your GO analysis. If not, you might have to make your own group of relevant genes.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-12-05T08:16:28.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10939" PostTypeId="1" CreationDate="2019-12-05T09:27:08.533" Score="0" ViewCount="428" Body="&lt;p&gt;I have a raw counts data-set of 20,502 genes and 137 samples. I want to find out Principal Components which best explain variation between samples in different stages of tumor.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am new to Machine learning and would like some help in selecting a criteria to reduce the number of genes before doing a PCA. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried doing a DESeq2 analysis on the data-set but I am confused as to what to do next.&lt;/p&gt;&#xA;" OwnerUserId="4121" LastActivityDate="2020-06-08T22:38:04.400" Title="Pre-filtering genes for Principal Component Analysis" Tags="|rna-seq|pca|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10940" PostTypeId="2" ParentId="10937" CreationDate="2019-12-05T10:03:59.290" Score="1" Body="&lt;p&gt;The various Gene Ontologies are arranged as Direct Acyclical Graphs. This means that each term can have children and parents. When a gene is annotated with a GO term, it also inherits all of that term's parents. Therefore, when you run a GO enrichment analysis, it is very common that you find the broader terms enriched since those are the ones shared by most genes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the Biological Process (BP) GO term &lt;a href=&quot;http://amigo.geneontology.org/amigo/term/GO:0003163&quot; rel=&quot;nofollow noreferrer&quot;&gt;GO:0003163 sinoatrial node development&lt;/a&gt; has the following tree:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;￼ GO:0008150 biological_process&#xA;     ￼ GO:0048856 anatomical structure development&#xA;     ￼ GO:0032501 multicellular organismal process&#xA;         ￼ GO:0032502 developmental process&#xA;         ￼ GO:0007275 multicellular organism development&#xA;             ￼ GO:0048731 system development&#xA;             ￼ GO:0009888 tissue development&#xA;                 ￼ GO:0048513 animal organ development&#xA;                 ￼ GO:0072359 circulatory system development&#xA;                 ￼ GO:0060537 muscle tissue development&#xA;                     ￼ GO:0048738 cardiac muscle tissue development&#xA;                     ￼ GO:0007507 heart development&#xA;                     ￼ GO:0014706 striated muscle tissue development&#xA;                         ￼ GO:0003161 cardiac conduction system development&#xA;                             ￼ GO:0003163 sinoatrial node development&#xA;                                 ￼ GO:0060921 sinoatrial node cell differentiation&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This means that any gene annotated with &lt;code&gt;GO:0003163&lt;/code&gt; will also be annotated with every one of the terms above it in the tree, all the way to the top of the graph. All genes with at least one BP GO term will also be annotated with GO:0008150, the root of the tree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, your &quot;muscle tissue development (GO:0060537)&quot; is also there and is a parent of &quot;sinoatrial node development&quot;. However, if you just ran an enrichment analysis (I ave no idea what you did since you haven't actually told us anything), then what will have happened is that there are many genes annotated with various different child terms of GO:0060537, but since all child terms will inherit the parent annotations, what you see is the parent and not the specific term you are looking for.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So yes, there are far more specific terms that what you show, but whether or not they appear in your results will depend on what you are doing. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2019-12-05T10:03:59.290" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10941" PostTypeId="2" ParentId="10939" CreationDate="2019-12-05T11:23:34.857" Score="3" Body="&lt;p&gt;You can filter out genes that do not show variation across samples, they would not differentiate samples anyway. For the specifics on how to do so, please see the &lt;code&gt;Subpopulation Analysis&lt;/code&gt; section of a nice &lt;a href=&quot;http://pklab.med.harvard.edu/scw2014/&quot; rel=&quot;nofollow noreferrer&quot;&gt;single cell RNA-seq workflow&lt;/a&gt; from the &lt;a href=&quot;http://pklab.med.harvard.edu&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kharchenko Lab&lt;/a&gt;. I believe there won't be fundamental differences when applying this approach to bulk RNA-seq.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2020-05-31T19:17:53.313" LastActivityDate="2020-05-31T19:17:53.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10943" PostTypeId="1" CreationDate="2019-12-05T17:05:14.303" Score="1" ViewCount="63" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;EDITED&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have list of my samples in first column and corresponding file name for my .vcf files in second column of a file name &lt;code&gt;clin_name.txt&lt;/code&gt; like below. For example name of one vcf is &lt;code&gt;LP6008031-DNA_A04_vs_LP6008032-DNA_D03.snp.pass.vcf&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;OC/AH/120/E/T3  LP6005409-DNA_F01&#xA;OC/AH/126/E/T1  LP2000325-DNA_A01&#xA;OC/AH/136/E/T3  LP6005409-DNA_E02&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the link of my file &lt;a href=&quot;https://www.dropbox.com/s/mvzf47feiqe7x69/clin_name.txt.txt?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/mvzf47feiqe7x69/clin_name.txt.txt?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the second file name &lt;code&gt;pre_post.list&lt;/code&gt; I have information of pre or post treatment for each sample like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pre-treated post-treated&#xA;OC/AH/120   OC/AH/065&#xA;OC/AH/126   OC/AH/069&#xA;OC/AH/136   OC/AH/074&#xA;OC/AH/160   OC/AH/084&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is link of this &lt;a href=&quot;https://www.dropbox.com/s/nhv3odjkdjdp7hz/pre_post.list.txt?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/nhv3odjkdjdp7hz/pre_post.list.txt?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For moving different .vcf files based on the pre or post treatment to separate folders I typed &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat pre_post.list | awk '{print &lt;span class=&quot;math-container&quot;&gt;$2}' | sed 's@.*/@@'  &amp;gt; post_list.txt&#xA;cat pre_post.list | awk '{print $&lt;/span&gt;1}' | sed 's@.*/@@'  &amp;gt; pre_list.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I have &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pre-treated&#xA;120&#xA;126&#xA;136&#xA;160&#xA;182&#xA;&#xA;post-treated&#xA;065&#xA;069&#xA;074&#xA;084&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I then typed &lt;code&gt;./test.sh&lt;/code&gt; in which I wrote&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for item in  &lt;span class=&quot;math-container&quot;&gt;$(cat  pre_list.txt | grep -Eo '[0-9]{1,4}');&#xA;    do if grep -q $&lt;/span&gt;item clin_name.txt ;&#xA;    then out=&lt;span class=&quot;math-container&quot;&gt;$(grep $&lt;/span&gt;item clin_name.txt | awk  '{print &lt;span class=&quot;math-container&quot;&gt;$1}');&#xA;         cp *$&lt;/span&gt;out* /temp/hgig/fi1d18/TRG45/snp/&#xA;    else echo post_list $item;  fi;  done;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That returns&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cp: cannot stat `*OC/AH/069/S/T3*': No such file or directory&#xA;[fi1d18@cyan01 snp]$&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or I run &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for item in  &lt;span class=&quot;math-container&quot;&gt;$(cat  post_list.txt | grep -Eo '[0-9]{1,4}');&#xA;    do if grep -q $&lt;/span&gt;item clin_name.txt ;&#xA;    then out=&lt;span class=&quot;math-container&quot;&gt;$(grep $&lt;/span&gt;item clin_name.txt | awk  '{print &lt;span class=&quot;math-container&quot;&gt;$1}');&#xA;         cp *$&lt;/span&gt;out* /temp/hgig/fi1d18/TRG45/snp/pre/&#xA;    else cp *$out* /temp/hgig/fi1d18/TRG45/snp/post/;  fi;  done;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That returns&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cp: cannot stat `OC/ST/026/S/T1*': No such file or directory&#xA;[fi1d18@cyan01 snp]$&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I really don't know how to solve this :(&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-12-06T09:59:57.757" LastActivityDate="2019-12-06T17:52:49.117" Title="move files of a different group to separate directory" Tags="|shell|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10944" PostTypeId="1" CreationDate="2019-12-05T19:59:59.607" Score="1" ViewCount="51" Body="&lt;p&gt;How can I get more specific GO terms when using  clusterProfiler? I got my Dotplot by:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cluster0_all_gene_ego&amp;lt;-enrichGO(cluster0_all_gene,keyType = 'SYMBOL', &#xA;                                OrgDb = org.Mm.eg.db, &#xA;                                ont = &amp;quot;BP&amp;quot;, &#xA;                                pAdjustMethod = &amp;quot;BH&amp;quot;,&#xA;                                pvalueCutoff = 1,&#xA;                                qvalueCutoff = 1)&#xA;dotplot(cluster0_all_gene_ego)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;as below:&lt;a href=&quot;https://i.stack.imgur.com/sGoUi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sGoUi.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It seems that I can only get the parent terms as some one tell me that there can have more specific child terms like below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GO:0008150 biological_process&#xA;     ￼ GO:0048856 anatomical structure development&#xA;     ￼ GO:0032501 multicellular organismal process&#xA;         ￼ GO:0032502 developmental process&#xA;         ￼ GO:0007275 multicellular organism development&#xA;             ￼ GO:0048731 system development&#xA;             ￼ GO:0009888 tissue development&#xA;                 ￼ GO:0048513 animal organ development&#xA;                 ￼ GO:0072359 circulatory system development&#xA;                 ￼ GO:0060537 muscle tissue development&#xA;                     ￼ GO:0048738 cardiac muscle tissue development&#xA;                     ￼ GO:0007507 heart development&#xA;                     ￼ GO:0014706 striated muscle tissue development&#xA;                         ￼ GO:0003161 cardiac conduction system development&#xA;                             ￼ GO:0003163 sinoatrial node development&#xA;                                 ￼ GO:0060921 sinoatrial node cell differentiation&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to have some terms like the sinoatrial node, so did anyone know how to modify the code?&lt;/p&gt;&#xA;&lt;p&gt;I have got the GO term's result and find most of the gene is expressed in the sinoatrial node, but there are no sinoatrial node terms.&lt;/p&gt;&#xA;" OwnerUserId="3171" LastEditorUserId="73" LastEditDate="2021-07-14T11:53:18.853" LastActivityDate="2021-07-14T11:53:18.853" Title="How to detail the specific GO terms" Tags="|rna-seq|go-enrichment|clusterprofiler|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10945" PostTypeId="1" AcceptedAnswerId="10954" CreationDate="2019-12-05T22:28:38.233" Score="2" ViewCount="57" Body="&lt;p&gt;At our institution we have had some debate about this topic. So let me give some background: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Qiime2 produces a OTU table which shows the number of sequences that matched to a specific OTU. It also produces an Taxonomy table which links all the OTUs to their corresponding taxonomic identity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So what we were discussing was whether to use the number of sequence hits per OTU for alpha and beta diversity, as well as abundance graphs. Or should we use the Taxonomy table and count the number of occurrences for each taxa.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of this would be something like the following: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are two OTUs:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;D_0__Bacteria;D_1__Cyanobacteria;D_2__Oxyphotobacteria;D_3__Chloroplast;D_4__Virgulinella fragilis;D_5__Virgulinella fragilis;D_6__Virgulinella fragilis&lt;/code&gt; and &lt;code&gt;D_0__Bacteria;D_1__Cyanobacteria;D_2__Oxyphotobacteria;D_3__Nostocales;D_4__Oscillatoriaceae;D_5__Oscillatoria PCC-10802;Ambiguous_taxa&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we used the first approach then there would be, for example, 3000 sequence matches for the first OTU and 1500 for the second OTU. However, they represent two different species from the same Class. So abundance wise, the first OTU could be represented twice as much as the second. With the second approach they would share the same abundance percentage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the correct approach for these types of analyses according to the bioinformatics community?&lt;/p&gt;&#xA;" OwnerUserId="2968" LastActivityDate="2019-12-08T18:49:35.407" Title="Which taxonomy data type to use for abundance graphs and statistical analyses?" Tags="|taxonomy|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10946" PostTypeId="1" CreationDate="2019-12-06T02:35:45.740" Score="1" ViewCount="51" Body="&lt;p&gt;This is the first time that I used &lt;code&gt;lmfit&lt;/code&gt; function from the &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/limma.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;limma&lt;/a&gt; package&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I'm a little bit confused on how to interpret the result.&lt;/li&gt;&#xA;&lt;li&gt;is there a way to get the estimate or the beta parameter of &lt;code&gt;lmfit&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6590" LastEditorUserId="3848" LastEditDate="2020-11-20T14:11:29.553" LastActivityDate="2020-11-20T14:11:29.553" Title="How can I get the estimate of lmFit function form limma package" Tags="|phylogenetics|limma|methylation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10947" PostTypeId="2" ParentId="10864" CreationDate="2019-12-06T14:50:35.203" Score="2" Body="&lt;p&gt;Python solution:&lt;br&gt;&#xA;e&amp;lt;=2 will allow up to 2 insertions, 2 deletions, or 2 SNPs in any 2 combinations&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import regex&#xA;seq_list=[&quot;GGGAGGG&quot;, &quot;GGGAAGGG&quot;, &quot;GGGAAAGGG&quot;, &quot;TTTATTT&quot;, &quot;TTTAATTT&quot;, &quot;TTTAAATTT&quot;, &quot;GGGG&quot;, &quot;GGG&quot;]&#xA;for sequence in seq_list:&#xA;    if regex.search(&quot;(GGGGGG|TTTTTT){e&amp;lt;=2}&quot;, sequence):&#xA;        print(sequence)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;GGGAGGG&lt;br&gt;&#xA;GGGAAGGG&lt;br&gt;&#xA;TTTATTT&lt;br&gt;&#xA;TTTAATTT&lt;br&gt;&#xA;GGGG  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can also include a wildcard&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seq_list=[&quot;GGGAAAAAGGG&quot;, &quot;GGGAAAAATGG&quot;, &quot;GGGAAAAATTG&quot;, &quot;GGGAAAAATTT&quot;]&#xA;for sequence in seq_list:&#xA;    if regex.search(&quot;(GGG.*GGG|TTTTTT){e&amp;lt;=2}&quot;, sequence):&#xA;        print(sequence)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;GGGAAAAAGGG&lt;br&gt;&#xA;GGGAAAAATGG&lt;br&gt;&#xA;GGGAAAAATTG&lt;/p&gt;&#xA;" OwnerUserId="6343" LastActivityDate="2019-12-06T14:50:35.203" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10948" PostTypeId="1" AcceptedAnswerId="10955" CreationDate="2019-12-06T15:07:29.013" Score="2" ViewCount="391" Body="&lt;p&gt;I ran a calculation in RAxML to determine the majority consensus phylogeny of a maximum likelihood bootstrap (&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/10801/how-to-show-bootstrap-values-on-a-phylogenetic-tree-constructed-with-raxml&quot;&gt;How to show bootstrap values on a phylogenetic tree constructed with RAxML&lt;/a&gt;), and I got three output files: &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;&lt;em&gt;RAxML_bipartitions.output_bootstrap.tre&lt;/em&gt;&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;&lt;em&gt;RAxML_bipartitionsBranchLabels.output_bootstrap.tre&lt;/em&gt;&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;&lt;em&gt;RAxML_info.output_bootstrap.tre&lt;/em&gt;&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What is the difference between the &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;&lt;em&gt;RAxML_bipartitions.output_bootstrap.tre&lt;/em&gt;&lt;/strong&gt; file and &lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;&lt;em&gt;RAxML_bipartitionsBranchLabels.output_bootstrap.tre&lt;/em&gt;&lt;/strong&gt;? &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5116" LastEditorUserId="3848" LastEditDate="2020-05-01T08:59:41.080" LastActivityDate="2020-05-01T08:59:41.080" Title="Interpreting quantitive outputs from maximum likelihood phylogenetic trees" Tags="|phylogenetics|phylogeny|software-usage|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10949" PostTypeId="2" ParentId="10943" CreationDate="2019-12-06T17:46:02.277" Score="1" Body="&lt;p&gt;You need to bring together the information in the two tables, I prefer &lt;code&gt;R&lt;/code&gt; for such actions. The common columns between the two tables do not quite contain the same information hence additional data cleaning was required. Moreover the table with treatment information had to be changed to long format with &lt;code&gt;melt&lt;/code&gt;. And I have filtered the &lt;code&gt;merged_data&lt;/code&gt; so as to conation only &quot;pretreatment&quot; info but you can add a if/else statement to the for loop to control how/where you will move your files.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(data.table)&#xA;&#xA;clin_name &amp;lt;- fread(&quot;clin_name.txt&quot;, header = TRUE)&#xA;clin_name&lt;span class=&quot;math-container&quot;&gt;$sample_id &amp;lt;- sub(&quot;/E/T.&quot;, &quot;&quot;, clin_name$&lt;/span&gt;sample_id)&#xA;&#xA;pre_post &amp;lt;- fread(&quot;pre_post.list&quot;, header = TRUE)&#xA;print(pre_post)&#xA;pre_post &amp;lt;- melt(pre_post,&#xA;     measure.vars = c(&quot;pre-treated&quot;, &quot;post-treated&quot;),&#xA;     variable.name = &quot;treatment&quot;,&#xA;     value.name = &quot;sample_id&quot;)&#xA;&#xA;&amp;gt; print(pre_post)&#xA;      treatment sample_id&#xA;1:  pre-treated OC/AH/120&#xA;2:  pre-treated OC/AH/126&#xA;3:  pre-treated OC/AH/136&#xA;4:  pre-treated OC/AH/160&#xA;5: post-treated OC/AH/065&#xA;6: post-treated OC/AH/069&#xA;7: post-treated OC/AH/074&#xA;8: post-treated OC/AH/084&#xA;&#xA;&#xA;merged_data &amp;lt;- merge(clin_name, pre_post)&#xA;print(merged_data)&#xA;&#xA;pre &amp;lt;- merged_data[merged_data$treatment == &quot;pre-treated&quot;, &quot;file_name&quot;]&#xA;&#xA;&amp;gt; print(merged_data)&#xA;   sample_id         file_name   treatment&#xA;1: OC/AH/120 LP6005409-DNA_F01 pre-treated&#xA;2: OC/AH/126 LP2000325-DNA_A01 pre-treated&#xA;3: OC/AH/136 LP6005409-DNA_E02 pre-treated&#xA;&#xA;# invoking a terminal command to move each file to a directory called &quot;pre&quot;&#xA;for (i in pre$file_name) {&#xA;  command &amp;lt;- paste0(&quot;mv &quot;, i, &quot; pre/&quot;)&#xA;  system(command)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2019-12-06T17:52:49.117" LastActivityDate="2019-12-06T17:52:49.117" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10950" PostTypeId="2" ParentId="10911" CreationDate="2019-12-07T14:44:42.057" Score="2" Body="&lt;p&gt;I've got the answer to my question at &lt;a href=&quot;https://www.biostars.org/p/410466/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/410466/&lt;/a&gt; by &lt;a href=&quot;https://www.biostars.org/u/9575/&quot; rel=&quot;nofollow noreferrer&quot;&gt;chrchang522&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;The main regression executed by Plink was introduced by EIGENSTRAT in ~2006; see &lt;a href=&quot;https://www.nature.com/articles/ng1847&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/ng1847&lt;/a&gt; . This is actually straightforward to write in R/Python from scratch; the harder part is optimizing the implementation for large datasets.&lt;/p&gt;&#xA;&lt;p&gt;The Firth regression added to Plink 2.0 to improve handling of rare variants and imbalanced binary phenotypes was motivated by &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4049324/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4049324/&lt;/a&gt; .&lt;/p&gt;&#xA;&lt;p&gt;Mixed linear models provide better statistical power when you have lots of close relatives in your dataset, but are much trickier to solve; actually, this is still a significant research area. Two tools covering parts of the current state-of-the-art are SAIGE (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6119127/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6119127/&lt;/a&gt; ; handles imbalanced binary phenotypes, but relatively slow) and fastGWA (&lt;a href=&quot;https://www.nature.com/articles/s41588-019-0530-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41588-019-0530-8&lt;/a&gt; ; great speed, but doesn't support dosage data yet and uses a misspecified model for binary phenotypes).&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="6464" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2020-01-08T15:16:39.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10951" PostTypeId="1" AcceptedAnswerId="10953" CreationDate="2019-12-07T19:14:40.493" Score="1" ViewCount="43" Body="&lt;p&gt;I am a MSc student in Applied Biochemistry. For my Masters Thesis, I will be doing some research about finding new potent inhibitors for a specific enzyme of my choice using molecular docking (virtual screening).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have selected 5 enzymes that have all been approved by my advisor and now the choice is for me to make. I am very interested in neurological diseases as you can see:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kynurenine 3-monooxygenase (KMO)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Glycogen synthase kinase-3 (GSK3)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dopa decarboxylase (DDC)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;HMG CoA reductase&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Phospholipase A2 (PLA2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Knowing that I plan to publish my work. I would be really really grateful, if you guys could help pick :)&lt;/p&gt;&#xA;" OwnerUserId="4921" LastEditorUserId="4921" LastEditDate="2019-12-08T06:58:30.957" LastActivityDate="2020-01-13T15:01:10.173" Title="Which of these enzymes is a better target for virtual screening?" Tags="|docking|molecular-genetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10952" PostTypeId="1" CreationDate="2019-12-08T12:15:53.597" Score="0" ViewCount="598" Body="&lt;p&gt;I study on RNA-seq expression dataset about one cancer in TCGA. I downloaded FPKM dataset and removed batch effect by ComBat() function. Now, I used &lt;code&gt;pvclust&lt;/code&gt; for clustering 72 samples and the result of that can access in this &lt;a href=&quot;https://www.dropbox.com/s/emk425f1xyi3olv/fit_LMS10k_72Samples.pdf?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;.my code for running pvclust is as below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fit_LMS &amp;lt;-pvclust(datForpvclust,method.hclust = &quot;average&quot;,&#xA;                          method.dist = &quot;euclidean&quot;,nboot=10000, parallel=TRUE ) &#xA;pvrect(fit_LMS, alpha=0.95)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you see, by alpha=0.95 just 3 clusters are significant although in right cluster a few clusters have been included. really I can't analysis this result.&#xA;So, I have 2 questions:&#xA;1-  What is the interpretation of &lt;code&gt;pvclust&lt;/code&gt; dendrogram? Does my dataset has meaning full clusters?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2-  I am interested in the height of tree cut in the dendrogram, which height is better for this dataset based on pvclust result? H=105 or H=110 or another height?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I appreciate it if anybody shares his/her comment with me.&lt;/p&gt;&#xA;" OwnerUserId="4915" LastEditorUserId="4915" LastEditDate="2019-12-08T19:34:30.923" LastActivityDate="2019-12-08T22:24:10.333" Title="how to interpret of &quot;pvclust&quot; dendrogram and finding height for cutting dendrogram?" Tags="|r|rna-seq|clustering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10953" PostTypeId="2" ParentId="10951" CreationDate="2019-12-08T17:53:16.823" Score="1" Body="&lt;p&gt;The ultimate choice stems from your individual interests and not a scientific reason... but having said that some pointer of issues may be handy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am assuming that you want to make an inhibitor that messes with the activity and not a ligand that binds somewhere and you then plan to add a moiety to make it a protac to get it degraded or to make it fluorescent to mark it. If not, ignore any talk of activity.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Crystal&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;is a crystal structure available?&lt;/li&gt;&#xA;&lt;li&gt;does the structure have the cofactor if any?&lt;/li&gt;&#xA;&lt;li&gt;is a structure available (itself or homologue) with the native metabolite bound?&lt;/li&gt;&#xA;&lt;li&gt;how is the active site? In the case of many cytochrome P450 enzyme the entranceway is really deep and is guarded by a flappy gate, both of which confer a great deal of specificity regardless of the residues in the active site, which are often just water coordinating. Your first enzyme, is that a NADPH/FAD or a cytochrome enzyme?&lt;/li&gt;&#xA;&lt;li&gt;What is the mechanism and is there a classic type of inhibitor chemistry? I.e. catalytic triad with a cysteine nucleophile is a nice target.(1)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Methodology&lt;/h2&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Wet lab: A problem with computational chemistry/biochemistry is that someone needs to make the protein and physically test your compounds. Are these easy to express? This may sound like not your problem, but a lot of issues can come from this latter stage.&lt;/li&gt;&#xA;&lt;li&gt;is there a close homologue you want to avoid targetting? If so, you'll have to do twice the docking (not a big deal and potentially more interesting). (2)&lt;/li&gt;&#xA;&lt;li&gt;is the method you plan to use okay with covalently bound ligands? Dopa decarboxylase sounds like a PLP enzyme. If it is, making an inhibitor using classic functional groups that nucleophilically attack the pyridine ring is very straighforward, but not all docking methods are okay with covalent ligands.&lt;/li&gt;&#xA;&lt;li&gt;Are any hits likely to follow Lipinski's rule of five? If you have a very charged active site, your hits will be metabolite-like (polar) and not drug-like (horrendously insoluble).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;A minor thing is to make sure that nobody else is working on it or you may be left behind. Industry is all cloak and dagger, but the academic SGC has a list of [target enabling packages)[&lt;a href=&quot;https://www.thesgc.org/tep]&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.thesgc.org/tep]&lt;/a&gt; and their future ones (nominations).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Physiology&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This was most likely in your lit search. But I am double checking.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You have chosen these enzymes because their unchecked activity is disease-causing, right?&lt;/li&gt;&#xA;&lt;li&gt;And they do not have any other roles in other tissues?&lt;/li&gt;&#xA;&lt;li&gt;Abolition of activity does what you want?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Other hits&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Lastly, but not least. See which of these have many paper about barely active hits: this should mean it is hard. For example, bacterial alanine racemase has been a holy grail for a while, but progress is slow as it has been described as a &quot;ship in a bottle&quot; due to its long and unforgiving entranceway.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Addenda&lt;/h2&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A catalytic triad is acid-base-nucleophile: if you have an electrophile with a leaving group (including internal ones, such as epoxydes) you can virtually screen more complex small molecules that resemble your ligand, but with the difference that there is a leaving group, resulting in an irreversible inhibition. Whereas competitive inhibitors require nanomolar affinity to outcompete the natural ligand, an irreversible inhibition can even slightly worse and will work.&lt;/li&gt;&#xA;&lt;li&gt;The problem of homologues.&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Paralogues (homologous genes in the same organism): you want to target protein 1, but you do not want to target protein 2. This is a common problem with kinases. You therefore need to find a hit that works better in one than in the other, which with a big of rational inference is easy.&lt;/li&gt;&#xA;&lt;li&gt;Homologues (homologous genes in different organism). You want to kill species A (say a fly), but not B (say a human). Same thing.&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Twice the docking. It is more of a challenge, but makes for a better spin on the story. There is an explosion of people doing virtual screens that do not go anyway afterwards, so it is getting harder and harder to publish studies without extensive empirical backing. Hence the need for an interesting angle.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2019-12-14T14:14:09.147" LastActivityDate="2019-12-14T14:14:09.147" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10954" PostTypeId="2" ParentId="10945" CreationDate="2019-12-08T18:49:35.407" Score="0" Body="&lt;p&gt;The fundamental answer to this question is the role of allele frequency, i.e. population genetics, within taxonomy. The answer of course is that population genetics and population dynamics should be a integral part of taxonomic designation PROVIDING it is not subject to unreported sample bias. Thus it has to be option 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, your definitions I'm not so sure about because the counts per OTU using your literal definition means every SNP within a species. I don't believe that is what you mean, but that is what you have said. Obviously at a taxonomic focus of a single SNP this isn't  meaningful because there would be as many OTUs as there are sequences and the species definition would be meaningless. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both approaches suffer from lacking a clear definition to make a species assignment and this is controversial. If you are using a tree to assign species membership, e.g. 16S at 5% identity, then that would generally be acceptable, but some would question it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can of course use deep learning algorithms to do this, but this is only as accurate your training and at present (check Bioinformatics journal) the accuracy is at best 90%, which is very low by comparison to phylogeny.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus I think it is a non-question, because allelic diversity is extremely important in vast swathes of biological analysis.  Taxonomy is not a disapline in its own right, but a foundational step biologists use to understand the ecotope, ecoregion etc... The only  justification for pursing approach 2 is if there is persistent concern in the bias of the sampling process, to the point where the alleleic (species) frequency becomes meaningless.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-12-08T18:49:35.407" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10955" PostTypeId="2" ParentId="10948" CreationDate="2019-12-08T20:03:48.433" Score="1" Body="&lt;p&gt;In summary,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;RAxML_bipartitions.output_bootstrap.tre&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is the only file of interest. The reason this is true in this context is really complicated and you have to understand the statistics of likelihood and how they are interpreted within phylogeny to understand why. This file is simply the final output of a non-parametric bootstrap analysis performed by maximum likelihood.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What on earth is a non-parametric boostrap?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A non-parametric bootstrap is resampling each alignment position with replacement. Thus if we have alignment positions 1,2,3,4,5&#xA;A bootstrap resample for 2 replicates might be,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Replicate 1&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1,1,3,5,2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Replicate 2&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;4,2,5,2,1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The ML algorithm will make trees of replicates 1 and 2 and find the consensus between them. If you think about it in any other context a bootstrap replicate is pretty meaningless because it no longer reflects the true biological sequence. Thus information on how the consensus was derived, are not really of interest to us providing we are confident this has been done correctly, viz. RAxML_bipartitionsBranchLabels.output_bootstrap.tre and RAxML_bipartitionsBranchLabels.output_bootstrap.tre&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So why is this output of limited use?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are situations to some investigators this information is useful, but assess the robustness of a tree topology its not needed. The only thing we want is a phylogram (bestTree) with the bootstrap values superimposed on them. We really don't need complicated stuff such as the tree to be represented for example as a polytomy (non-bifurcating tree) because we can just read the bootstraps to make that deduction (values &gt;&gt; 75%). In addition, there is not perfect consensus what boostrap value constitutes robustness, but generally most agree &gt;80% is robust.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What output files have useful information in them?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The information that is important are the files associated with &quot;bestTree&quot;, that was the single maximum likelihood tree performed on the intact native sequence. The &quot;info&quot; file for this contains 3 really important parameters:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;-lnL ... very important!! &lt;/li&gt;&#xA;&lt;li&gt;Gamma distribution parameter &quot;alpha&quot;,&lt;/li&gt;&#xA;&lt;li&gt;PINVAR, proportion of invariant sites, &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;-lnL&lt;/strong&gt; is the highest log-likelihood (probability) of the phylogeny. It is usually a very small number for which where is an enormous amount of theory over it. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Alpha parameter of the gamma distribution&lt;/strong&gt; this is the shape parameter of the mutation rate, if it is very low (&amp;lt;1) the distribution of mutations across the alignment is very tight clustered and approximates to a negative binomial distribution. This means some sites don't mutate at all and a small number of sites mutate alot. If it is very large &gt;200 (which is never observed) it approximates to the Poisson distribution, meaning the mutation distribution is randomised across the alignment. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;PINVAR&lt;/strong&gt; this is a straight percentage/frequency and simply means the sites that don't mutate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How are they calculated?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PINVAR and alpha are not emperically calculated, i.e. if you look at an alignment and say 'no mutations at that position', PINVAR would of course agree but may consider other invariant depending on the phylogeny. These parameters are calculated by maximum likelihood and you can begin to see why the calculation takes so long ... alpha and PINVAR affect the tree topology (which affects -lnL), but the topology affects alpha and PINVAR. Thus, is a multidimensional search of tree and parameter space.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So what stuff do I report in my Results?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway reporting -lnL is good technique and shows the reader you've done maximum likelihood, citing PINVAR and alpha from gamma distribution helps ('Methods' parameters were calculated reiteratively under maximum likelihood). This is only useful for bestTree. The -lnL, PINVAR and gamma's alpha are also calculated for every single bootstrap replicate, but these values are of limited to use, because we have resampled the data, only the consensus tree counts... Obviously presenting the bootstrapped phylogram is extremely important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Welcome to the technical world of phylogeny! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The amino acid matrix you used BTW .. LG is in vogue right now. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How do I do it?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I do this stuff its via Biopython and ETE3, I capture the values within the pipeline and don't examine the output files of RAxML because I generate my own.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-12-09T20:00:41.457" LastActivityDate="2019-12-09T20:00:41.457" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10956" PostTypeId="2" ParentId="10952" CreationDate="2019-12-08T22:24:10.333" Score="1" Body="&lt;ul&gt;&#xA;&lt;li&gt;You do not need to give links to your images, you can embed images in your questions using the &quot;image&quot; icon.&lt;/li&gt;&#xA;&lt;li&gt;FPKM (and RPKM) might not be the best RNA-seq normalization method, see &lt;a href=&quot;https://www.rna-seqblog.com/rpkm-fpkm-and-tpm-clearly-explained/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://hbctraining.github.io/DGE_workshop/lessons/02_DGE_count_normalization.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;pvclust&lt;/code&gt; providing p-values for the clusters computed by &lt;code&gt;hclust&lt;/code&gt;. To do so, bootstrap resampling is used. From &lt;a href=&quot;https://github.com/shimo-lab/pvclust&quot; rel=&quot;nofollow noreferrer&quot;&gt;their documentation&lt;/a&gt;: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Values on the edges of the clustering are p-values (%). Red values are&#xA;  AU p-values, and green values are BP values. Clusters with AU larger&#xA;  than 95% are highlighted by rectangles, which are strongly supported&#xA;  by data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Basically you are getting a clustering pattern and a statistic associated with each of the clusters giving you an idea about the &quot;statistical significance&quot; of the clusters you have obtained (i.e. clusters &lt;em&gt;most probably&lt;/em&gt; not being a result of strong batch effetcs, ...).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be able to comment on the resulting clusters making sense or not, you will need to consult somebody with &lt;em&gt;domain knowledge&lt;/em&gt;, it is not a number or two that would determine the quality of the output but also how good is the fit between your result and the (expected) biology. For example in your case with TCGA data:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Do the samples cluster by batch?&lt;/li&gt;&#xA;&lt;li&gt;Do tumor and adjacent normal tissue samples aggregate into different clusters?&lt;/li&gt;&#xA;&lt;li&gt;Do the samples in the same cluster share the same mutation/methylation pattern/disease subgroup?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-12-08T22:24:10.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10957" PostTypeId="1" CreationDate="2019-12-09T09:44:51.623" Score="1" ViewCount="50" Body="&lt;p&gt;&lt;em&gt;after comments on my original &lt;a href=&quot;https://biology.stackexchange.com/questions/89133/detect-differentially-expressing-cells?noredirect=1#comment158173_89133&quot;&gt;post&lt;/a&gt;, I will ask my question again here&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have data (RNA expression values, obtained with multi-channel in situ hybridization) collected from 1mio human cells. For each cell, I have the expression value of a negative control (non-human RNA) and single-cell measures for different RNA species. I know that the expression level of the negative control correlate in a linear fashion with all the other RNA species when there is no real expression (non-expressing cells). For example, when a cell has a high expression value of the negative control, the signal for all other RNA species will also be high. I now want to detect the cells that express an RNA without having a high negative signal, e.g. the &quot;real&quot; expressing cells. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing to have in mind is that some RNA species are abundantly expressed in many cells, other are very rare and have a low expression value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to do this with a linear model (for each of the RNA species, negative control vs. single RNA), computing z-scores, infer p-values and correct for multiple testing. For some RNA species, it works just fine. As soon as I have many cells expressing a certain RNA, the thing gets more tricky because the slope of the linear model doesn't reflect the linear relationship of negative control vs. RNA species in non-expressing cells because it is more shifted towards the &quot;real&quot; expressing cells. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To visualize the problem, I attached two plots. One plot showing high RNA expression many cells (left plot) and one from an RNA species which is expressed on a very low level in few cells (right plot). Blue colored dots are considered as expressing cells according to the linear model. As you can see, for the scenario shown on the left side, many &quot;real&quot; expressing cells (low expression level) are not detected. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NdzZ6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NdzZ6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any ideas which statistical model would better fit to detect those cells? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your suggestions!&lt;/p&gt;&#xA;" OwnerUserId="6611" LastEditorUserId="6611" LastEditDate="2019-12-09T11:27:47.247" LastActivityDate="2019-12-09T11:27:47.247" Title="Detect differentially expressing cells" Tags="|statistics|single-cell|modelling|linear-regression|multi-omics|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10958" PostTypeId="1" CreationDate="2019-12-09T13:27:39.497" Score="0" ViewCount="60" Body="&lt;p&gt;I have tried different combination of tree cut heights but im getting the exprSize as one sample as full the other one as empty .&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Choose the &quot;base&quot; cut height for the female data set&#xA;baseHeight = 135&#xA;# Adjust the cut height for the male data set for the number of samples&#xA;cutHeights = c(135, 145*exprSize&lt;span class=&quot;math-container&quot;&gt;$nSamples[2]/exprSize$&lt;/span&gt;nSamples[1]);&#xA;# Re-plot the dendrograms including the cut lines&#xA;pdf(file = &quot;SampleClustering.pdf&quot;, width = 12, height = 12);&#xA;par(mfrow=c(2,1))&#xA;par(mar = c(0, 4, 2, 0))&#xA;for (set in 1:nSets)&#xA;{&#xA;  plot(sampleTrees[[set]], main = paste(&quot;Sample clustering on all genes in&quot;, setLabels[set]),&#xA;       xlab=&quot;&quot;, sub=&quot;&quot;, cex = 0.7);&#xA;  abline(h=cutHeights[set], col = &quot;red&quot;);&#xA;}&#xA;dev.off();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My figure &lt;a href=&quot;https://i.stack.imgur.com/Pi7QD.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Pi7QD.jpg&quot; alt=&quot;tree cut&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When i see the cutHeights and exprSize i do see its higher than 120 but still i get only one set of sample which is 55 whereas the my other set which is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;exprSize&#xA;$nSets&#xA;[1] 2&#xA;&#xA;$nGenes&#xA;[1] 8213&#xA;&#xA;$nSamples&#xA;[1] 55 47&#xA;&#xA;$structureOK&#xA;[1] TRUE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is after tree cut &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;exprSize&#xA;$nSets&#xA;[1] 2&#xA;&#xA;$nGenes&#xA;[1] 8213&#xA;&#xA;$nSamples&#xA;[1] 55  0&#xA;&#xA;$structureOK&#xA;[1] TRUE&#xA;&#xA;&amp;gt; cutHeights&#xA;[1] 135.0000 123.9091&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Im not able to figure out what is wrong &lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2019-12-09T17:47:58.457" Title="Tree cut height for Consensus data input not working" Tags="|wgcna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10959" PostTypeId="2" ParentId="10958" CreationDate="2019-12-09T17:47:58.457" Score="1" Body="&lt;p&gt;Your data don't have obvious outliers so you don't have to worry about removing them. The problem is that both sets show prominent sample clusters. I would try to figure out what drives those clusters (is it technical or biological), and whether the variation should be removed. See the comments in &lt;a href=&quot;https://horvath.genetics.ucla.edu/html/CoexpressionNetwork/Rpackages/WGCNA/faq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;WGCNA FAQ&lt;/a&gt; under point 5 (My data are heterogeneous. Can I still use WGCNA?). This looks like data from GEO, so look into the sample annotation for these data sets and see if any of the recorded sample characteristics align with the clusters you see in the sample trees.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2019-12-09T17:47:58.457" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10960" PostTypeId="1" AcceptedAnswerId="10962" CreationDate="2019-12-09T18:12:50.803" Score="0" ViewCount="93" Body="&lt;p&gt;I have a very big matrix like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(mat[1:4,1:4])&#xA;                     Tumor_Sample_Barcode Chromosome Start_Position End_Position&#xA;1: LP6005334-DNA_H01_vs_LP6005333-DNA_H01       chr1        1026918      1026918&#xA;2: LP6005334-DNA_H01_vs_LP6005333-DNA_H01       chr1       15772155     15772155&#xA;3: LP6005334-DNA_H01_vs_LP6005333-DNA_H01       chr1       16054813     16054813&#xA;4: LP6005334-DNA_H01_vs_LP6005333-DNA_H01       chr1       24120222     24120222&#xA;&#xA;&amp;gt; dim(mat)&#xA;[1] 450754     53&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In &lt;code&gt;Tumor_Sample_Barcode&lt;/code&gt; column I have the name of samples so that&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; unique(mat$Tumor_Sample_Barcode)&#xA; [1] &quot;LP6005334-DNA_H01_vs_LP6005333-DNA_H01&quot;     &quot;LP6005500-DNA_D03_vs_LP6005499-DNA_D03&quot;    &#xA; [3] &quot;LP6005935-DNA_G02_vs_LP6005979-DNA_C01&quot;     &quot;LP6007600_vs_LP6007599&quot;                    &#xA; [5] &quot;LP6008202-DNA_B03_vs_LP6008201-DNA_B03&quot;     &quot;LP6008269-DNA_F06_vs_LP6008268-DNA_G06&quot;    &#xA; [7] &quot;LP6008334-DNA_A03_vs_LP6008335-DNA_E02&quot;     &quot;LP6008334-DNA_A04_vs_LP6008335-DNA_E03&quot;    &#xA; [9] &quot;LP6008334-DNA_B02_vs_LP6008335-DNA_F01&quot;     &quot;LP6008334-DNA_C02_vs_LP6008335-DNA_G01&quot;    &#xA;[11] &quot;LP6008334-DNA_D02_vs_LP6008335-DNA_H01&quot;     &quot;LP6008336-DNA_F02_vs_LP6008333-DNA_F02&quot;    &#xA;[13] &quot;LP6008336-DNA_G01_vs_LP6008333-DNA_G01&quot;     &quot;LP6008336-DNA_H01_vs_LP6008333-DNA_H01&quot;    &#xA;[15] &quot;LP6008336-DNA_H02_vs_LP6008333-DNA_H02&quot;     &quot;LP6008337-DNA_A07_vs_LP6008338-DNA_E06&quot;    &#xA;[17] &quot;LP6008337-DNA_H06_vs_LP6008338-DNA_D06&quot;     &quot;LP6008460-DNA_A04_vs_LP6008340-DNA_D05__pv&quot;&#xA;[19] &quot;LP6008460-DNA_D01_vs_LP6008340-DNA_E02__pv&quot; &quot;LP6008460-DNA_F02_vs_LP6008340-DNA_E05&quot;    &#xA;[21] &quot;LP6008460-DNA_G03_vs_LP6008340-DNA_C05__pv&quot;&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to change the name of these samples to A1... A21 but I don't know how&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you help?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="181" LastEditDate="2019-12-10T10:18:02.213" LastActivityDate="2020-01-08T10:56:49.523" Title="Changing the name of elements in a column" Tags="|r|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10961" PostTypeId="2" ParentId="10960" CreationDate="2019-12-09T18:30:44.147" Score="1" Body="&lt;p&gt;Using &lt;code&gt;sub&lt;/code&gt; in &lt;code&gt;base&lt;/code&gt; R, you can:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) define the replacement vector:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;vector = paste0(&quot;A&quot;,seq(1:21))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;2) and loop over each levels of your first columns to replace them individually:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;for(i in 1:length(levels(mat&lt;span class=&quot;math-container&quot;&gt;$Tumor_Sample_Barcode)))&#xA;{&#xA;  levels(mat$Tumor_Sample_Barcode)[i] = sub(levels(mat$Tumor_Sample_Barcode)[i],vector[i], levels(mat$&lt;/span&gt;Tumor_Sample_Barcode)[i])&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Example:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here, a small example to show you:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;df = data.frame(vect = factor(rep(c(&quot;a&quot;,&quot;b&quot;,&quot;c&quot;,&quot;d&quot;),5)),&#xA;                       value = rnorm(20))&#xA;&#xA;&amp;gt; head(df)       &#xA;  vect        value&#xA;1    a  0.418186287&#xA;2    b -0.232936122&#xA;3    c  1.563349091&#xA;4    d -0.001453912&#xA;5    a -0.453995163&#xA;6    b -0.481360070&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;vector = paste0(&quot;A&quot;,seq(1,4))&#xA;for(i in 1:length(levels(df$vect)))&#xA;{&#xA;  levels(df$vect)[i] = sub(levels(df$vect)[i],vector[i], levels(df$vect)[i]) &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And you get:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; head(df)&#xA;  vect        value&#xA;1   A1  0.418186287&#xA;2   A2 -0.232936122&#xA;3   A3  1.563349091&#xA;4   A4 -0.001453912&#xA;5   A1 -0.453995163&#xA;6   A2 -0.481360070&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does it answer your question ?&lt;/p&gt;&#xA;" OwnerUserId="6437" LastEditorUserId="6437" LastEditDate="2019-12-09T19:50:38.223" LastActivityDate="2019-12-09T19:50:38.223" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10962" PostTypeId="2" ParentId="10960" CreationDate="2019-12-09T18:53:56.640" Score="3" Body="&lt;p&gt;If your data is in a dataframe you can make use of &lt;code&gt;plyr&lt;/code&gt;. Using &lt;code&gt;mapvalues()&lt;/code&gt; from this library, you can rename provide a list of keys and a list of values. Instead of explicitly listing the keys, we can just use &lt;code&gt;unique(mat$Tumor_Sample_Barcode)&lt;/code&gt; [credit to @dc37]. I use the &lt;code&gt;paste0&lt;/code&gt; function here and the colon operator to generate the vector of &lt;code&gt;A1,A2,...,A21&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assign the vector produced by &lt;code&gt;mapvalues()&lt;/code&gt; to the column and you're off!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;install.packages(&quot;plyr&quot;)&#xA;mat$Tumor_Sample_Barcode &amp;lt;-&#xA;plyr::mapvalues(x=mat$Tumor_Sample_Barcode,&#xA;                from=unique(mat$Tumor_Sample_Barcode),&#xA;                to=paste0(&quot;A&quot;, 1:21))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'd recommend keeping this column and adding a new column with your simplified sample names, but this is not what you requested.&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="181" LastEditDate="2019-12-10T20:54:46.133" LastActivityDate="2019-12-10T20:54:46.133" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="10964" PostTypeId="1" CreationDate="2019-12-11T09:36:34.167" Score="0" ViewCount="38" Body="&lt;p&gt;In large Volunteer computing for molecular docking projects such as FightAIDS@home, do they use the original crystal structure of ligands in docking or unconfirmed structures of ligand?   &lt;/p&gt;&#xA;" OwnerUserId="6299" LastActivityDate="2020-05-10T09:04:35.573" Title="Are crystal structure of Ligands used in volunteer computational docking like FightAIDS@home?" Tags="|docking|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10965" PostTypeId="2" ParentId="10964" CreationDate="2019-12-11T10:58:35.707" Score="1" Body="&lt;p&gt;I don't know that project's details, so cannot say officially, but undoubtedly it will be &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;crystal structures of protein or models of point mutants based on structures of protein.&lt;/li&gt;&#xA;&lt;li&gt;Computer structure of the ligands based on force-field calculation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Protein structures&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Firstly, HIV virus has been extensively studied by structural biologists, so most components have structures.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, docking on models is generally done only for threaded models based on closely related templates, say a human model of a mouse structure or a recently diverged pair (i.e. conserved proteins that split with the genome double duplication in bony fish). Whereas a given virus protein will likely have poor homology to a known structure and not be a good candidate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A corner case is core point mutants... If you have a mutation in a protein it is likely going to be on the surface and not core: a model of a point mutant is likely very faithful to an empirical version &amp;mdash;and generally you'd want to design drugs to a region that is not variable anyway as you'd want it to work universally. If you have a change in the active site to confer resistance say (i.e. in the core), there is generally a fitness loss to the organism (say lowered catalysis or thermostability) which needs to be compensated with other mutations (epistasis). As this is rarer and very interesting scientifically and medically, these resistant variants will most likely have been crystallised as the conditions for crystallization will most likely be the same.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Small molecule structures&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Force field calculations for small molecules are most often valid (there computer Vs empirical RMSD is minute). The partial charges may be off, but that's not something crystals give anyway: In fact a ligand used in docking is (generally/ought to be) energy minimised beforehand. For docking getting charges wrong is the killer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Crystallization of small molecules is used as validation of a synthesis route, so some may have structures, but they will not be used in a pipeline as a SMILES string is all you need.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2019-12-12T08:43:08.543" LastActivityDate="2019-12-12T08:43:08.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10966" PostTypeId="2" ParentId="10872" CreationDate="2019-12-11T13:00:57.183" Score="2" Body="&lt;p&gt;I would second the answer gringer gave recommending Kraken2.  I would suggest using &lt;a href=&quot;https://ccb.jhu.edu/software.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;bracken&lt;/a&gt; to process the results from Kraken2 if you want to estimate species abundance (the raw Kraken2 results should not be used for this).  I would recommend Pavian for visualising the results, it produces beautiful Sankey plots in a fairly pain-free manner.  If you want to save yourself some work the 16S tool provided by Oxford Nanopore as part of epi2me produces good results for limited effort - although this is a bit of a proprietary black box.&lt;/p&gt;&#xA;" OwnerUserId="3100" LastActivityDate="2019-12-11T13:00:57.183" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10967" PostTypeId="1" CreationDate="2019-12-11T13:21:57.163" Score="0" ViewCount="470" Body="&lt;p&gt;Seurat V2 had a option to find clustering information saved in object: PrintFindClustersParams(object = pbmc). How can I get the same clustering parameters from objects in Seurat3? Do I need to manually save clustering parameters or they are saved automatically?&lt;/p&gt;&#xA;&lt;p&gt;Update: the command line &lt;code&gt;object@commands&lt;/code&gt; works perfectly for &lt;code&gt;...$FindClusters&lt;/code&gt;. However, if I would like to find information about dimensions used there is an error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;...$FindNeighbors.SCT.pca Error in [[.Seurat(x, i, drop = TRUE) : Cannot find 'FindNeighbors.SCT.pca' in this Seurat object&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6621" LastEditorUserId="73" LastEditDate="2021-05-05T06:32:18.840" LastActivityDate="2021-05-05T06:32:18.840" Title="Clustering information saved in Seurat object" Tags="|seurat|single-cell|clustering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10968" PostTypeId="2" ParentId="10967" CreationDate="2019-12-11T13:58:37.977" Score="1" Body="&lt;p&gt;I am not sure of the output of &lt;code&gt;PrintFindClustersParams&lt;/code&gt;, as I haven't use it before, but try with &lt;code&gt;object@commands&lt;/code&gt;. It returns a list containing every command and parameters used.&lt;/p&gt;&#xA;&lt;p&gt;For example&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;$FindNeighbors.SCT.pca&#xA;Command: FindNeighbors(seurObj, dims = 1:20, verbose = vrb)&#xA;Time: 2019-12-11 12:41:12&#xA;reduction : pca &#xA;dims : 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 &#xA;assay : SCT &#xA;k.param : 20 &#xA;compute.SNN : TRUE &#xA;prune.SNN : 0.06666667 &#xA;nn.method : rann &#xA;annoy.metric : euclidean &#xA;nn.eps : 0 &#xA;verbose : FALSE &#xA;force.recalc : FALSE &#xA;do.plot : FALSE &#xA;graph.name : SCT_nn SCT_snn &#xA;&#xA;$FindClusters&#xA;Command: FindClusters(seurObj, verbose = vrb)&#xA;Time: 2019-12-11 12:41:17&#xA;graph.name : SCT_snn &#xA;modularity.fxn : 1 &#xA;resolution : 0.8 &#xA;algorithm : 1 &#xA;n.start : 10 &#xA;n.iter : 10 &#xA;random.seed : 0 &#xA;group.singletons : TRUE &#xA;verbose : FALSE &#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Re: update&lt;/p&gt;&#xA;&lt;p&gt;I think it depends on how you built the object. &lt;code&gt;FindNeighbors.SCT.pca&lt;/code&gt; should be result of running &lt;code&gt;sctransform normalization&lt;/code&gt;. If you use the &lt;em&gt;standard way&lt;/em&gt;, you may have other names (e.g.&lt;code&gt;FindNeighbors.pca&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="3631" LastEditorUserId="73" LastEditDate="2021-05-05T06:31:19.620" LastActivityDate="2021-05-05T06:31:19.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10970" PostTypeId="1" CreationDate="2019-12-11T21:13:00.733" Score="3" ViewCount="543" Body="&lt;p&gt;Is it faster to use the PySam package to run a python script on a bam&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for read in samfile.fetch('chr1', 100, 120):&#xA; print read&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;compared to using a pipe and reading line by line from stdin in a python script. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;samtools view example.bam | python script.py.&#xA;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for line in tqdm(sys.stdin, total=number_of_lines)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6625" LastEditorUserId="6251" LastEditDate="2019-12-15T05:16:53.063" LastActivityDate="2019-12-15T05:16:53.063" Title="pysam or piping samtools view to a python script" Tags="|python|sam|samtools|pysam|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10971" PostTypeId="2" ParentId="10970" CreationDate="2019-12-11T21:51:34.613" Score="5" Body="&lt;p&gt;It's a bit hard to say with certainty, though I would suspect that offloading the BAM decompression by using a pipe will be very slightly faster. Note that decompressing and parsing the BAM file will not be the bottleneck in your processing, rather the python script itself will be.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-12-11T21:51:34.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10972" PostTypeId="1" CreationDate="2019-12-12T10:42:41.420" Score="1" ViewCount="33" Body="&lt;p&gt;&quot;As a first step, we sought to systematically establish the complete set of genes implicated in the physiological response to SCI. We conducted a systematic analysis of the SCI literature, reviewing over 500 papers, in order to reveal a set of 695 unique human genes associated with the response to SCI by small-scale experiments (Supplementary file 1). Of these genes, 559 were upregulated following SCI, 213 were downregulated, and the protein products of 8 were differentially phosphorylated. Among all genes, 151 were associated with the response to SCI by more than one study (Figure 2A). The complete set includes genes that have been associated with SCI in a wide range of experimental models of SCI, in addition to human injuries (Figure 2—figure supplement 1A); in multiple species, including human as well as rat, mouse, and rabbit (Figure 2—figure supplement 1B); using a range of experimental techniques (Figure 2B); and at a variety of time points, from 1 hr to 6 months post-injury (Figure 2—figure supplement 1C).&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://elifesciences.org/articles/39188&quot; rel=&quot;nofollow noreferrer&quot;&gt;Paper&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know what kind of tools or specifically R library can be used to this kind of literature mining .&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2019-12-12T10:42:41.420" Title="Literature mining tools and methods" Tags="|r|literature-search|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10974" PostTypeId="1" CreationDate="2019-12-12T16:46:38.310" Score="3" ViewCount="380" Body="&lt;p&gt;I am going to start work on the Materials and Methods for a paper I contributed to a lot in the bioinformatics processing part. I have seen &lt;a href=&quot;https://www.researchgate.net/figure/Flowchart-of-the-main-steps-and-bioinformatics-tools-required-for-pathway-reconstruction_fig1_233751111&quot; rel=&quot;nofollow noreferrer&quot;&gt;some papers&lt;/a&gt; nowadays tend to first present the processing steps that they followed as a flow chart, to show the reader what happened generally before going into details in the written M&amp;amp;M section.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any specific guidelines for creating this flowchart? Do you have any tips or advice from personal experience?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;(I am aware that this may be off-topic. I have a meta question about it &lt;a href=&quot;https://bioinformatics.meta.stackexchange.com/questions/219/is-asking-about-publishing-in-bioinformatics-on-topic&quot;&gt;here&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="6316" LastActivityDate="2019-12-12T17:14:09.020" Title="Bioinformatics flowchart for Materials and Methods" Tags="|literature|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10975" PostTypeId="2" ParentId="10974" CreationDate="2019-12-12T17:14:09.020" Score="4" Body="&lt;p&gt;If you're using a workflow management system for pipelining like &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;snakemake&lt;/a&gt; or &lt;a href=&quot;http://www.ruffus.org.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ruffus&lt;/a&gt;, you can actually build a flowchart based on the rules/functions in your code:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/getting_started/examples.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Snakemake example&lt;/a&gt; with the &lt;code&gt;--dag&lt;/code&gt; flag:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FSAD4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FSAD4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ruffus.org.uk/gallery.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ruffus example&lt;/a&gt; using &lt;code&gt;pipeline_printout_graph&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uTGfJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uTGfJ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You'll probably still need edit the image aftwards in software like inkscape or illustrator though.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2019-12-12T17:14:09.020" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="10977" PostTypeId="1" CreationDate="2019-12-13T12:32:26.427" Score="2" ViewCount="1165" Body="&lt;p&gt;The Python statsmodels library grants access to many R-datasets which can be downloaded e.g. with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import statsmodels.api as sm&#xA;df = sm.datasets.get_rdataset(dataname='iris', package='datasets').data&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But for this you have to know the id or the name of the dataset.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sm.datasets.get_rdataset??&#xA;Signature: sm.datasets.get_rdataset(dataname, package='datasets', cache=False)&#xA;Source:   &#xA;def get_rdataset(dataname, package=&quot;datasets&quot;, cache=False):&#xA;    &quot;&quot;&quot;download and return R dataset&#xA;&#xA;    Parameters&#xA;    ----------&#xA;    dataname : str&#xA;        The name of the dataset you want to download&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There is a list of available datasets &lt;a href=&quot;https://www.statsmodels.org/devel/datasets/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, but this list does not contain the identifiers. Also not the &lt;a href=&quot;https://www.statsmodels.org/devel/datasets/generated/co2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;detailed view&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an overview of all the available datasets that contains the ids e.g. &quot;iris&quot;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is an &lt;a href=&quot;https://vincentarelbundock.github.io/Rdatasets/datasets.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;index&lt;/a&gt; that is supposed to list all the datasets, but e.g. 'rice' as a name does not work.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2019-12-13T12:42:29.730" LastActivityDate="2022-07-07T14:42:48.887" Title="How to retrieve dataset names from python statsmodels?" Tags="|python|statsmodels|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10979" PostTypeId="2" ParentId="7336" CreationDate="2019-12-14T15:59:40.537" Score="4" Body="&lt;h2&gt;Hydrogens&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Most crystallographic structures are X-ray and the hydrogens are absent. In the case of neutron diffraction models, there may be many missing high b-factor ones (&lt;a href=&quot;https://michelanglo.sgc.ox.ac.uk/data/e5590567-3c91-4149-8ac1-275f64f1060c&quot; rel=&quot;nofollow noreferrer&quot;&gt;example in PDB:5MOP&lt;/a&gt;). As a consequence models are reprotonated. Also missing carbons and missing loop residues are added for some in silico applications too (not docking).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, it is always good to manually tweak the protonation. In the case of enzymes with protanatable side chains in the active site, some reading is required to get the charges right: by default aspartates will be unprotonated, but they may be protonated (glutamic acid) in the active site and it is important to get it right (&lt;em&gt;cf.&lt;/em&gt; &lt;a href=&quot;https://blog.matteoferla.com/2018/09/everything-you-wanted-to-know-about.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post about it&lt;/a&gt;). Some times its the ligand parameterisation may give the physiologically incorrect protonation (&lt;em&gt;cf.&lt;/em&gt; &lt;a href=&quot;https://blog.matteoferla.com/2019/10/rdkit-for-rosetta-plp-ligand-space-as.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Waters&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Crystallographic waters can be important catalytically and structurally, but can be pushed out and actually it is a good thing: you have a hit that take its place. Polar bonds are stronger than hydrophobic ones and most types of &amp;pi; bonding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Waters in most docking software is &lt;code&gt;implicit&lt;/code&gt; and is modelled as a parameter in the force-fields —think of it as a ghostly watery haze. So a left behind water, not only will get in the way of the docked small molecule, it may for some programs cause issues as they will try to dock it. A notable exception is WaterDock mod for AutoDock, but the jury is out on whether it is better. Water is polarisable in reality in a very complex way (protonated (OH3+) or deprotonated (OH-) with very different charges) and most water models do not cut it and instead do not account for this.&#xA;The way implicit waters are computed differs in some applications, which is also something to look into.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2019-12-14T15:59:40.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10980" PostTypeId="1" CreationDate="2019-12-14T21:07:36.613" Score="0" ViewCount="427" Body="&lt;p&gt;I wanna install UBCG.JAR in ubuntu 18.04 but I cannot run the java File UBCG.JAR.&#xA;when I type the code &lt;code&gt;java -jar UBCG.JAR&lt;/code&gt; an error message appears.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Exception in thread &amp;quot;main&amp;quot; java.lang.NoClassDefFoundError: org/json/simple/parser/ParseException&#xA;at java.base/java.lang.Class.forName0(Native Method)&#xA;at java.base/java.lang.Class.forName(Class.java:398)&#xA;at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:56)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Caused by: java.lang.ClassNotFoundException: org.json.simple.parser.ParseException&#xA;at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)&#xA;at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)&#xA;at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)&#xA;... 3 more&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ java -version&#xA;Openjdk version &amp;quot;11.0.7&amp;quot; 2020-04-14&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6328" LastEditorUserId="73" LastEditDate="2022-03-21T21:26:36.853" LastActivityDate="2023-12-12T02:02:05.210" Title="Java error when trying to run UBCG jar (java.lang.ClassNotFoundException)" Tags="|java|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="10984" PostTypeId="1" AcceptedAnswerId="10994" CreationDate="2019-12-15T07:37:09.093" Score="1" ViewCount="36" Body="&lt;p&gt;I wrote a &amp;quot;script&amp;quot; to pull out reads from a huge fastq file in an iterative manner, by finding homology to the previous sequence.&lt;/p&gt;&#xA;&lt;p&gt;It should be relatively easy to overlap them and assemble the gene, however traditional assemblers like SPAdes fail due to &lt;em&gt;obviously&lt;/em&gt; low coverage.&lt;/p&gt;&#xA;&lt;p&gt;I am hesitant to just feed it more faked reads, when I know I have all the data necessary to just overlap them into a single sequence.&lt;/p&gt;&#xA;&lt;p&gt;The reads are set up like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GATCAAACATC&#xA;     AACATCAGTTAG&#xA;         TCAGTTAG&#xA;              TAGAGGATAGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What kind of assembler is out there where I can feed it this kind of terribly-formatted data?&lt;/p&gt;&#xA;" OwnerUserId="3811" LastEditorUserId="3811" LastEditDate="2020-10-19T04:24:34.187" LastActivityDate="2020-10-19T09:35:30.610" Title="Finding simple sequence from reads with significant overlap" Tags="|sequence-alignment|assembly|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10985" PostTypeId="1" AcceptedAnswerId="10992" CreationDate="2019-12-15T14:22:15.610" Score="-1" ViewCount="81" Body="&lt;p&gt;In the below &lt;code&gt;bash loop for strelka2&lt;/code&gt;, 3 bam files are being stored in a variable. The two questions are:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this proper way to loop through multi bam and wait for workflow to be created for &lt;code&gt;strelka2&lt;/code&gt; processing,&#xA;and can you create a workflow in a directory without the bam in it. Thank you :).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;contents of dir&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;s1.bam&#xA;s2.bam&#xA;s3.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;bash&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dir=/path/to/bam&#xA;data=/path/to/processed/data&#xA;&#xA;cd &lt;span class=&quot;math-container&quot;&gt;$dir&#xA;	for bam in $&lt;/span&gt;{dir}/*.bam; do sample_basename=&lt;span class=&quot;math-container&quot;&gt;$(basename &quot;$&lt;/span&gt;{bam}&quot;); done # loop through dir and get bam&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;${path_to_strelka}/bin/configureStrelkaGermlineWorkflow.py \&#xA;  --bam $&lt;/span&gt;{dir}/&lt;span class=&quot;math-container&quot;&gt;${sample_basename} \&#xA;  --referenceFasta &amp;lt;fasta&amp;gt; \&#xA;  --runDir $&lt;/span&gt;{data}&#xA;  wait  # wait until workflow in data is created before executing below  in data&#xA;${data}/runWorkflow.py -m local -j 20&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4424" LastEditorUserId="4424" LastEditDate="2019-12-15T14:30:34.163" LastActivityDate="2019-12-16T20:40:37.840" Title="bash loop for strelka2 processing for multiple files from one directory" Tags="|bash|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10986" PostTypeId="1" AcceptedAnswerId="12799" CreationDate="2019-12-15T16:19:59.877" Score="3" ViewCount="122" Body="&lt;p&gt;I want to compare two secondary structures of aligned proteins. I do not know exactly how to do it well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;EEEEEEEEEEETTTTTTHCTTTEEECTTTEEECTTECTCCCHHHHHHHHHCHCCTHHEEEETTTEEECTTTEEECTTTEEC&#xA;&#xA;EEEHHHHHHHTTTTTTTTCTTHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHHEEEECCCCCCCCCCEEEEEEEEEEEEEE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Example of matching:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;EEEEEEEEEEETTTTTTHCTTTEEECTTTEEECTTECTCCCHHHHHHHHHCHCCTHHEEEETTTEEECTTTEEECTTTEEC&#xA;|||        |||||| |||                    |||||||||     ||||||      |   |||    ||&#xA;EEEHHHHHHHTTTTTTTTCTTHHHHHHHHHHHHHHHHHHHHHHHHHHHHHH----HHEEEECCCCCCCCCCEEEEEEEEEE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Hypothetically: 50% aligned - &gt; score 0.5 - &gt;  thats scoring is wrong:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;EEEEEEEEEEETTTTTTHCTTTEEECTTTEEECTTECTCCCHHHHHHHHHCHCCTHHEEEETTTEEECTTTEEECTTTEEC&#xA;||||||||||||||||||||||||||||||||||||||||||&#xA;EEEEEEEEEEETTTTTTHCTTTEEECTTTEEECTTECTCCCHEEEEEEEEEEEEEEEHHHHHHHHHHHHHHHHHHHHHHHH&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This case will have similar score... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a puzzle for me, I don't know how to approach it technically.&lt;/p&gt;&#xA;" OwnerUserId="1224" LastEditorUserId="1224" LastEditDate="2019-12-15T16:40:24.893" LastActivityDate="2020-04-01T15:48:18.407" Title="protein secondary structure matching/comparison scoring" Tags="|proteins|python|statistics|secondary-structure|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="10988" PostTypeId="1" CreationDate="2019-12-16T10:11:32.343" Score="1" ViewCount="24" Body="&lt;p&gt;I want to look at violation of the strict molecular clock and compare dn/ds ratio to test differences in evolutionary rates in a set of genes. Some of these genes are really short (30AA), is it likely to prevent the detection of rate changes ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="6459" LastEditorUserId="6459" LastEditDate="2019-12-16T13:26:30.403" LastActivityDate="2019-12-16T13:26:30.403" Title="Gene size and power of detection in sequence divergence analyses" Tags="|gene|positive-selection|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="10990" PostTypeId="1" AcceptedAnswerId="10993" CreationDate="2019-12-16T17:26:15.863" Score="7" ViewCount="2187" Body="&lt;p&gt;Doubt regarding density plot what is the scale being plotted in the Y axis.Is it possible to scale it to 1?.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;w &amp;lt;- read.csv(&quot;Normal_Myeloid_Dev_stages/Myeloid_non_coding_non_CDS_CORRVALUE.txt&quot;,header = TRUE)&#xA;head(w)&#xA;#names(w)[1] = &quot;Sample&quot;&#xA;head(w)&#xA;&#xA;#w.plot &amp;lt;- melt(w) &#xA;&#xA;df &amp;lt;- w&#xA;&#xA;df_melt=melt(df,id.vars=&quot;Sample&quot;)&#xA;#head(df_melt)&#xA;tail(df_melt)&#xA;df_melt&lt;span class=&quot;math-container&quot;&gt;$Group &amp;lt;- gsub('[0-9]', '',df_melt$&lt;/span&gt;variable)&#xA;head(df_melt)&#xA;&#xA;p1 &amp;lt;- ggplot(aes(x=value, colour=Group), data=df_melt)&#xA;p1 + geom_density()&#xA;&#xA;ggplot(df_melt, aes(x = value,y=..density.., fill = Group)) + geom_density(alpha = 0.5)+&#xA;  theme_bw(base_size=30)+&#xA;  theme(axis.text.x=element_text(angle = 45, size=45, face=&quot;bold&quot;, hjust = 1), &#xA;        axis.text.y=element_text(angle=0, size=50, face=&quot;bold&quot;, vjust=0.5),&#xA;        plot.title = element_text(size=40, face=&quot;bold&quot;), &#xA;        legend.title=element_blank(), &#xA;        legend.key.size=unit(1, &quot;cm&quot;),      #Sets overall area/size of the legend&#xA;        legend.text=element_text(size=30)) &#xA;&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qeOij.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qeOij.jpg&quot; alt=&quot;figure&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Second question regarding finding the overlapping area between various condition let say I have four condition and i want to find out what is the overlapping or intersecting area between them .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried on of the &lt;a href=&quot;https://stackoverflow.com/questions/41914257/calculate-area-of-overlapping-density-plot-by-ggplot-using-r?noredirect=1&amp;amp;lq=1&quot;&gt;stack solution&lt;/a&gt; firstly is it the right way to do it for finding overlap? if yes can i try finding all the condition overlap such as I have HSC,CMP,GMP and Mono.  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FDensity = approxfun(density(df_melt&lt;span class=&quot;math-container&quot;&gt;$value[df_melt$&lt;/span&gt;Group==&quot;CMP&quot;], from=.80, to=1))&#xA;MDensity = approxfun(density(df_melt&lt;span class=&quot;math-container&quot;&gt;$value[df_melt$&lt;/span&gt;Group==&quot;Mono&quot;], from=.80, to=1))&#xA;plot(FDensity, xlim=c(.80,1), ylab=&quot;Density&quot;)&#xA;curve(MDensity, add=TRUE)&#xA;&#xA;FminusM = function(x) { FDensity(x) - MDensity(x) }&#xA;Intersect = uniroot(FminusM, c(.8, 1))$root&#xA;points(Intersect, FDensity(Intersect), pch=20, col=&quot;red&quot;)&#xA;&#xA;integrate(MDensity, .80,Intersect)&lt;span class=&quot;math-container&quot;&gt;$value + &#xA;  integrate(FDensity, Intersect, 1)$&lt;/span&gt;value&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any suggestion or help would be really appreciated &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My dataframe small subset&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dput(head(df))&#xA;structure(list(Sample = structure(c(9L, 10L, 11L, 12L, 1L, 2L&#xA;), .Label = c(&quot;CMP1&quot;, &quot;CMP2&quot;, &quot;CMP3&quot;, &quot;CMP4&quot;, &quot;GMP1&quot;, &quot;GMP2&quot;, &#xA;&quot;GMP3&quot;, &quot;GMP4&quot;, &quot;HSC1&quot;, &quot;HSC2&quot;, &quot;HSC3&quot;, &quot;HSC4&quot;, &quot;Mono1&quot;, &quot;Mono2&quot;, &#xA;&quot;Mono3&quot;, &quot;Mono4&quot;), class = &quot;factor&quot;), HSC1 = c(1, 0.901758000194052, &#xA;0.880971458703505, 0.900098712568466, 0.873507054094022, 0.906315987662777&#xA;), HSC2 = c(0.901758000194052, 1, 0.945122894369186, 0.955453795442752, &#xA;0.844777154811663, 0.960260413763721), HSC3 = c(0.880971458703505, &#xA;0.945122894369186, 1, 0.931891528389177, 0.829521543809914, 0.93636611655036&#xA;), HSC4 = c(0.900098712568466, 0.955453795442752, 0.931891528389177, &#xA;1, 0.852155096935692, 0.947192034704188), CMP1 = c(0.873507054094022, &#xA;0.844777154811663, 0.829521543809914, 0.852155096935692, 1, 0.88406635204624&#xA;), CMP2 = c(0.906315987662777, 0.960260413763721, 0.93636611655036, &#xA;0.947192034704188, 0.88406635204624, 1), CMP3 = c(0.883085447599108, &#xA;0.928929790213059, 0.937278860333014, 0.923139729112196, 0.870595169861622, &#xA;0.958330253497026), CMP4 = c(0.903500691840647, 0.931015449611016, &#xA;0.915510499719426, 0.949501343959892, 0.89151869084162, 0.960206335031192&#xA;), GMP1 = c(0.859320159223793, 0.833454007185579, 0.821479398953591, &#xA;0.837605565046685, 0.856892209387879, 0.8612685942226), GMP2 = c(0.894680165774456, &#xA;0.944129985346879, 0.92461599550388, 0.937108867595306, 0.875930650641323, &#xA;0.966512850930959), GMP3 = c(0.881416749993221, 0.925001567707159, &#xA;0.923015420025178, 0.920758302664238, 0.863998116268766, 0.95069438729762&#xA;), GMP4 = c(0.887230702792805, 0.911614382740949, 0.898704725226052, &#xA;0.938310378760322, 0.875956580458754, 0.93865124308053), Mono1 = c(0.80834274546097, &#xA;0.803706049167148, 0.785535924609134, 0.794438848049474, 0.794402126200663, &#xA;0.810382054101301), Mono2 = c(0.829928035079747, 0.868021129791583, &#xA;0.848929312833536, 0.852953184941131, 0.818074585324121, 0.880490870699251&#xA;), Mono3 = c(0.834028389107401, 0.859818911060746, 0.852694253707916, &#xA;0.85039434449057, 0.82251614206182, 0.872740655210769), Mono4 = c(0.832524159512094, &#xA;0.858444209302146, 0.84397886802844, 0.862850406374668, 0.818436260136434, &#xA;0.870293804485003)), row.names = c(NA, 6L), class = &quot;data.frame&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="285" LastEditorUserId="131" LastEditDate="2020-01-08T11:19:25.727" LastActivityDate="2020-01-08T11:19:25.727" Title="Density plot, scale it to 0-1" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10992" PostTypeId="2" ParentId="10985" CreationDate="2019-12-16T20:40:37.840" Score="1" Body="&lt;p&gt;The loop you're using does not work, as it overwrites the value of &lt;code&gt;sample_basename&lt;/code&gt; each time. Plus, given that you're passing in the full path to the bam (using &lt;code&gt;$dir/$sample_basename&lt;/code&gt;) anyway, I don't see the reason to use the &lt;code&gt;sample_basename&lt;/code&gt; variable at all.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dir=/path/to/bam&#xA;data=/path/to/processed/data&#xA;ref_fa=/path/to/ref/fasta&#xA;&#xA;cd $dir&#xA;&#xA;for bam_file in *.bam&#xA;do&#xA;  &lt;span class=&quot;math-container&quot;&gt;${path_to_strelka}/bin/configureStrelkaGermlineWorkflow.py \&#xA;    --bam $&lt;/span&gt;{bam_file} \&#xA;    --referenceFasta &lt;span class=&quot;math-container&quot;&gt;${ref_fa} \&#xA;    --runDir $&lt;/span&gt;{data}&#xA;  ${data}/runWorkflow.py -m local -j 20 #I'm assuming this goes inside the loop. If not, move it outside the loop. You don't need a `wait` in a shell script that's working line-by-line already.&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastActivityDate="2019-12-16T20:40:37.840" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10993" PostTypeId="2" ParentId="10990" CreationDate="2019-12-17T00:53:12.803" Score="6" Body="&lt;p&gt;I'm not fully to have understood your second questions, but first, if you want to scale the density to 1, you can use &lt;code&gt;y= ..scaled..&lt;/code&gt; in your &lt;code&gt;aes&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(tidyverse)&#xA;df %&amp;gt;% pivot_longer(., everything(), names_to = &amp;quot;Variable&amp;quot;, values_to = &amp;quot;Value&amp;quot;) %&amp;gt;%&#xA;  ggplot(., aes(x = Value, y = ..scaled.., fill = Variable)) + geom_density(alpha = 0.4)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/89meV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/89meV.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now regarding the second question, if you are looking for the overlap between each of your density, you can use the package &lt;code&gt;overlapping&lt;/code&gt; as specified in the post you are referring: &lt;a href=&quot;https://stackoverflow.com/questions/41914257/calculate-area-of-overlapping-density-plot-by-ggplot-using-r?noredirect=1&amp;amp;lq=1&quot;&gt;https://stackoverflow.com/questions/41914257/calculate-area-of-overlapping-density-plot-by-ggplot-using-r?noredirect=1&amp;amp;lq=1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(overlapping)&#xA;library(lattice)&#xA;x &amp;lt;- list(D1 = D1, D2 = D2, D3 = D3, D4 = D4)&#xA;overlap(x, plot = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6tLdW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6tLdW.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And if you want to extract the value of overlap, you can get it:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;OV &amp;lt;- overlap(x)$OV&#xA;&#xA;&amp;gt; OV&#xA;    D1-D2     D1-D3     D1-D4     D2-D3     D2-D4     D3-D4 &#xA;0.5971760 0.3171644 0.7566528 0.4926057 0.6683083 0.3756224 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is it what you are looking for ? Or did I misunderstood your question ?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Dummy Dataset&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# Dummy example:&#xA;D1 = rnorm(100, mean =1, sd = 2)&#xA;D2 = rnorm(100, mean = 2, sd = 2)&#xA;D3 = rnorm(100, mean = 3, sd = 1)&#xA;D4 = rnorm(100,mean = 1.5, sd = 1.5)&#xA;&#xA;df = data.frame(D1,D2,D3,D4)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT: Using reproducible example provided in the question&lt;/strong&gt;&#xA;For plotting &lt;code&gt;geom_density&lt;/code&gt; of your data:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(tidyverse)&#xA;df %&amp;gt;% pivot_longer(., -Sample, names_to = &amp;quot;Variable&amp;quot;, values_to = &amp;quot;Value&amp;quot;) %&amp;gt;% &#xA;  mutate(New_Var = gsub(&amp;quot;\\d&amp;quot;,&amp;quot;&amp;quot;,Variable)) %&amp;gt;%&#xA;  ggplot(., aes(x = Value, y = ..scaled.., fill = New_Var))+&#xA;  geom_density(alpha = 0.4)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EE97t.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EE97t.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And to find the overlap:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;d&amp;lt;- df %&amp;gt;% pivot_longer(., -Sample, names_to = &amp;quot;Variable&amp;quot;, values_to = &amp;quot;Value&amp;quot;) %&amp;gt;% mutate(New_Var = gsub(&amp;quot;\\d&amp;quot;,&amp;quot;&amp;quot;,Variable))&#xA;a &amp;lt;- split(d&lt;span class=&quot;math-container&quot;&gt;$Value, unique(d$&lt;/span&gt;New_Var))&#xA;&#xA;g &amp;lt;- overlap(a, plot = TRUE)$OV&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cIm2x.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cIm2x.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I don't think there is an option in the &lt;code&gt;overlap&lt;/code&gt; function to scaled each overlap plot to 1 but these plot are using &lt;code&gt;ggplot&lt;/code&gt; and so, I imagine you can extract their points and scaled them by plotting them separately in &lt;code&gt;ggplot2&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Hope it answers your question&lt;/p&gt;&#xA;" OwnerUserId="6437" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2019-12-17T22:20:12.943" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="10994" PostTypeId="2" ParentId="10984" CreationDate="2019-12-17T01:38:23.023" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/lh3/miniasm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Miniasm&lt;/a&gt; will probably work for this; it's designed to be an ultra-fast, greedy assembler, which has been used for people doing real-time assemblies from MinION data. Because of its greedy nature, it can misassemble regions where the overlapping fragments are ambiguous, so it's worth using some other method in addition to miniasm to confirm that the assembly is correct.&lt;/p&gt;&#xA;&lt;p&gt;By default it looks for overlap regions that have a coverage of 3 reads when joining (option &lt;code&gt;-c&lt;/code&gt;), but I assume that could be reduced down to 2 reads, as in your example (which has a minimum overlapping coverage of 2).&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-10-19T09:35:30.610" LastActivityDate="2020-10-19T09:35:30.610" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="10995" PostTypeId="1" CreationDate="2019-12-17T11:08:30.933" Score="2" ViewCount="1373" Body="&lt;p&gt;What score value is considered acceptable when using Vina Autodock tool for molecular docking simulations?&lt;/p&gt;&#xA;" OwnerUserId="6299" LastEditorUserId="964" LastEditDate="2019-12-17T15:32:11.470" LastActivityDate="2020-06-30T15:49:58.143" Title="What score value is considered acceptable when using Vina Autodock tool for molecular docking simulations?" Tags="|docking|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10996" PostTypeId="2" ParentId="10995" CreationDate="2019-12-17T14:32:50.577" Score="1" Body="&lt;p&gt;The answer is &amp;quot;Depends&amp;quot;. But the best thing is always to look at what you ended up with and do some triaging before buying/synthesising anything. By triaging I mean filtering by its properties (&lt;em&gt;.e.g.&lt;/em&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Lipinski%27s_rule_of_five&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lipinski&lt;/a&gt;, reactivity, synthetic feasibility &lt;em&gt;etc.&lt;/em&gt; —discussed in &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/12971/generate-ligands-candidates-based-on-protein-shape/12975&quot;&gt;this Bioinfo SE question&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;h2&gt;Score&lt;/h2&gt;&#xA;&lt;p&gt;Regardless of docking software used, a forcefield free-energy score is a result of the scoring function used and of the properties of the molecule —including VENA which uses a hybrid score function.&lt;/p&gt;&#xA;&lt;p&gt;This score is &lt;em&gt;not&lt;/em&gt;, but generally is set up to try to match, the actual phys-chem one.&#xA;&lt;a href=&quot;https://en.wikipedia.org/wiki/Gibbs_free_energy&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gibbs free energy&lt;/a&gt; is measured in kcal/mol (or less commonly in kJ/mol). A hydrogen bond is about 2 kcal/mol, a π-π bond 1 kcal/mol and a salt bridge 3. The average energy of a collision of RT water molecules is 0.6 kcal/mol.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, if you dock multiple compounds (virtual library screen, VLS) you sort by free energy to see the strongest binders and consider the top ones.&#xA;However, a catch is that this will be biased by MW.&lt;/p&gt;&#xA;&lt;p&gt;Also, it is useful to score the hits with a different program (say Gold or ICM-dock).&lt;/p&gt;&#xA;&lt;h2&gt;Ligand Efficiency&lt;/h2&gt;&#xA;&lt;p&gt;A variant metric that is employed when your hits vary in size and when they are over a given size (&lt;em&gt;e.g.&lt;/em&gt; not fragments under 200 Da) is &lt;a href=&quot;https://en.wikipedia.org/wiki/Ligand_efficiency&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ligand Efficiency (LE&lt;/a&gt;), which is the ratio of your score over the number of heavy atoms (&lt;a href=&quot;https://www.nature.com/articles/nrd4163&quot; rel=&quot;nofollow noreferrer&quot;&gt;ref&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;h2&gt;Single compound&lt;/h2&gt;&#xA;&lt;p&gt;If you have a single compound and want to know if it binds acceptably, then this is a length-of-a-piece-of-string question as every compound has an affinity constant to a given protein, except most may be beyond their solubility. Initial hits will have µM affinity, while a lead will have nM affinity. However, a KD is not a ∆G and depends on your protein.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-06-30T15:49:58.143" LastActivityDate="2020-06-30T15:49:58.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10997" PostTypeId="1" CreationDate="2019-12-17T22:24:50.510" Score="1" ViewCount="413" Body="&lt;p&gt;I am trying to establish pipeline in snakePipes and while running DNA mapping I got next IndexError error.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(snakePipes) ~ $ DNA-mapping -i /Volumes/HD2/snakePipes_CHIP/ChiPseq -o /Volumes/HD2/snakePipes_CHIP/ChiPseq1 --mapq 5 -j 10 --dedup hg38&#xA;Building DAG of jobs...&#xA;Using shell: /bin/bash&#xA;Provided cluster nodes: 10&#xA;Job counts:&#xA;    count   jobs&#xA;    5   Bowtie2&#xA;    5   FASTQ1&#xA;    5   FASTQ2&#xA;    &#xA;&#xA;rule origFASTQ2:&#xA;    input: /Volumes/HD2/snakePipes_CHIP/ChiPseq/SRR6761499_R2.fastq.gz&#xA;    output: originalFASTQ/SRR6761499_R2.fastq.gz&#xA;    jobid: 75&#xA;    wildcards: sample=SRR6761499&#xA;&#xA;RuleException in line 9 of /Users/v/miniconda3/envs/snakePipes/lib/python3.6/site-packages/snakePipes/shared/rules/FASTQ.snakefile:&#xA;IndexError: list index out of range&#xA;Error: snakemake returned an error code of 1, so processing is incomplete!&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I am looking in FASTQ.snakefile:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule origFASTQ1:&#xA;      input:&#xA;          indir+&amp;quot;/{sample}&amp;quot;+reads[0]+ext&#xA;      output:&#xA;          &amp;quot;originalFASTQ/{sample}&amp;quot;+reads[0]+&amp;quot;.fastq.gz&amp;quot;&#xA;      shell:&#xA;          &amp;quot;( [ -f {output} ] || ln -s -r {input} {output} )&amp;quot;&#xA;&#xA;line9: rule origFASTQ2:&#xA;      input:&#xA;          indir+&amp;quot;/{sample}&amp;quot;+reads[1]+ext&#xA;      output:&#xA;          &amp;quot;originalFASTQ/{sample}&amp;quot;+reads[1]+&amp;quot;.fastq.gz&amp;quot;&#xA;      shell:&#xA;          &amp;quot;( [ -f {output} ] || ln -s -r {input} {output} )&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I do not understand why I see this error. I hope somebody can point me on the source of the error.&lt;/p&gt;&#xA;&lt;p&gt;When I use --local parameter I do not see this error, however I get another error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ln: illegal option -- r&#xA;&#xA;usage: ln [-Ffhinsv] source_file [target_file]&#xA;       ln [-Ffhinsv] source_file ... target_dir&#xA;       link source_file target_file&#xA;    output: originalFASTQ/SRR6761495_R1.fastq.gz&#xA;    shell:&#xA;        ( [ -f originalFASTQ/SRR6761495_R1.fastq.gz ] || ln -s -r /Volumes/HD2/snakePipes_CHIP/ChiPseq/SRR6761495_R1.fastq.gz originalFASTQ/SRR6761495_R1.fastq.gz )&#xA;        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;as far as I understand -r option creates symbolic links relative to link location&lt;/p&gt;&#xA;&lt;p&gt;in &lt;code&gt;/Volumes/HD2/snakePipes_CHIP/ChiPseq&lt;/code&gt; I have:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Bowtie2         SRR6761495_R1.fastq.gz  SRR6761495_R2.fastq.gz&#xA;command.sh      SRR6761499_R1.fastq.gz  SRR6761499_R2.fastq.gz&#xA;ChIPdict.yaml   SRR6761497_R1.fastq.gz  SRR6761497_R2.fastq.gz&#xA;deepTools_qc    SRR6761498_R1.fastq.gz  SRR6761498_R2.fastq.gz  &#xA;fetch.sh        SRR6761502_R1.fastq.gz  SRR6761502_R2.fastq.gz&#xA;filtered_bam&#xA;Sambamba&#xA;rename_files.sh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am working on macOS Catalina in bash shell and ln -r is valid option (I installed coreutils)&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="6661" LastEditorUserId="73" LastEditDate="2021-07-14T11:52:21.603" LastActivityDate="2021-07-14T11:52:21.603" Title="“IndexError: list index out of range” in snakemake" Tags="|snakemake|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="10999" PostTypeId="1" CreationDate="2019-12-18T12:34:53.900" Score="0" ViewCount="765" Body="&lt;p&gt;Where can I find information about the basecaller used in a specific nanopore run (we have a Gridion machine)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know already that we have &lt;code&gt;Guppy v. 3.0.3&lt;/code&gt;, which I found out by clicking on the &lt;code&gt;about&lt;/code&gt; tab on the MinKNOW GUI).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering, though, if additional information about the basecaller is outputted in any of the files generated when the run is finished? I checked already the &lt;code&gt;final_summary.txt&lt;/code&gt; and it does not contain such information. I also checked the head of the &lt;code&gt;fastq&lt;/code&gt; files and no further details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My ultimate goal for the moment is to run &lt;a href=&quot;https://github.com/nanoporetech/medaka&quot; rel=&quot;nofollow noreferrer&quot;&gt;medaka&lt;/a&gt; on a draft genome assembly. But for that I'd need to specify the &lt;code&gt;medaka model&lt;/code&gt;. From &lt;code&gt;medaka_consensus -h&lt;/code&gt; I see that the options available and defaults are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;-m  medaka model, (default: r941_min_high_g330).&#xA;        Available: r941_min_fast_g303, r941_min_high_g303, r941_min_high_g330, r941_prom_fast_g303, r941_prom_high_g303, r941_prom_high_g330, r10_min_high_g303, r10_min_high_g340, r941_prom_diploid_snp, r941_min_high_g340_rle.&#xA;        Alternatively a .hdf file from 'medaka train'.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Therefore I already know that my option should contain &lt;code&gt;min&lt;/code&gt; and &lt;code&gt;g303&lt;/code&gt; (format of the models available described &lt;a href=&quot;https://nanoporetech.github.io/medaka/installation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). But know I'd need to find out the &lt;code&gt;{pore}&lt;/code&gt; and &lt;code&gt;{caller variant}&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3452" LastActivityDate="2019-12-19T02:54:25.487" Title="nanopore - where to retrieve information from the basecaller used" Tags="|nanopore|assembly|long-reads|base-calling|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11000" PostTypeId="1" AcceptedAnswerId="11001" CreationDate="2019-12-18T14:17:50.747" Score="1" ViewCount="92" Body="&lt;p&gt;I have two files like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(a)&#xA;  chr position&#xA;1   1   136962&#xA;2   1   562020&#xA;3   1   672948&#xA;&amp;gt; class(a)&#xA;[1] &quot;data.frame&quot;&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dim(b)&#xA;[1] 1855235      10&#xA;&amp;gt; head(b[,1:3])&#xA;            SNP chr position&#xA;1:  rs62635286          1    13116&#xA;2:  rs75454623          1    14930&#xA;3:    rs806731          1    30923&#xA;4: rs200943160          1    49298&#xA;5: rs116400033          1    51479&#xA;6: rs141149254          1    54490&#xA;&amp;gt; class(b)&#xA;[1] &quot;data.table&quot; &quot;data.frame&quot;&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to find corresponding SNP in file b matched to position in file a so I did like this that gives memory allocation error. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; unique(merge(a,b,x.by=a&lt;span class=&quot;math-container&quot;&gt;$position,y.by=b$&lt;/span&gt;position)) &#xA;Error: cannot allocate vector of size 137.2 Gb&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also tried rstudio cloud but uploading the bigger file never completes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you help me with task please?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The expected output would be something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rs62635286  1   13116&#xA;rs376723915 1   91515&#xA;rs147061536 1   92858&#xA;rs62642131  1   135982&#xA;rs371474651 1   158006&#xA;rs201293782 1   665401&#xA;rs138476838 1   668374&#xA;rs1401137   1   2063094&#xA;rs3128293   1   2065339&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;EDITED&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; setDT(a)&#xA;&amp;gt; setkey(a, chr, position)&#xA;&amp;gt; setkey(b, chr, position)&#xA;Error in setkeyv(x, cols, verbose = verbose, physical = physical) : &#xA;  some columns are not in the data.table: chr,position&#xA;&amp;gt; setkey(b, Chromosome, Position)&#xA;&amp;gt; a[b, nomatch=0]&#xA;Error in bmerge(i, x, leftcols, rightcols, roll, rollends, nomatch, mult,  : &#xA;  Incompatible join types: x.chr (factor) and i.Chromosome (integer). Factor columns must join to factor or character columns.&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-12-18T16:40:49.047" LastActivityDate="2019-12-18T18:24:08.743" Title="Merging files gives memory allocation error" Tags="|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11001" PostTypeId="2" ParentId="11000" CreationDate="2019-12-18T16:11:37.850" Score="2" Body="&lt;p&gt;You could try something like this :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First you should work with the same class of object : &lt;code&gt;data.table&lt;/code&gt; for fast merging operation&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Turning a to a data.table&#xA;setDT(a)&#xA;# b is already a data.table &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you might have to set the keys of both table : &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# I added chr in case you need to match the same position on a different chromosome.&#xA;setkey(a, chr, position)&#xA;setkey(b, chr, position)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Finally perform the joining operation :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a[b, nomatch=0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Because you didn't provide the expected output, it might need some tuning !&lt;/p&gt;&#xA;" OwnerUserId="6273" LastEditorUserId="6273" LastEditDate="2019-12-18T18:24:08.743" LastActivityDate="2019-12-18T18:24:08.743" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11002" PostTypeId="2" ParentId="4091" CreationDate="2019-12-18T17:03:40.273" Score="2" Body="&lt;p&gt;The Ensembl Variant Effect Predictor will annotate your A. thaliana variants:&#xA;&lt;a href=&quot;http://plants.ensembl.org/Oryza_sativa/Tools/VEP&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://plants.ensembl.org/Oryza_sativa/Tools/VEP&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The web based tool will use the reference genome assembly and gene annotation used in the Ensembl browser:&#xA;&lt;a href=&quot;http://plants.ensembl.org/Arabidopsis_thaliana/Info/Annotation/#assembly&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://plants.ensembl.org/Arabidopsis_thaliana/Info/Annotation/#assembly&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you use the command-line version of VEP, you can provide your own reference genome assembly and gene annotation files to be used as a reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can find all of the documentation here:&#xA;&lt;a href=&quot;http://www.ensembl.org/info/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ensembl.org/info/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best wishes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ben&#xA;Ensembl Helpdesk&lt;/p&gt;&#xA;" OwnerUserId="6500" LastActivityDate="2019-12-18T17:03:40.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11003" PostTypeId="1" CreationDate="2019-12-18T22:06:45.690" Score="1" ViewCount="973" Body="&lt;p&gt;I would like to know how to change the PC use in the dimplot and featureplot by using Seurat. for we can get the x-axis and the y-axis like PC-1 and PC-2, if I want to use PC-4 and PC-5. How to modify the code?&lt;/p&gt;&#xA;" OwnerUserId="3171" LastActivityDate="2019-12-19T21:55:40.407" Title="how to change the PC use in the dimplot and feature plot" Tags="|seurat|single-cell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11004" PostTypeId="2" ParentId="11003" CreationDate="2019-12-18T22:26:53.203" Score="1" Body="&lt;p&gt;You can choose which PC dims to plot by specifying the &lt;code&gt;dims&lt;/code&gt; argument within the functions. To use PC-4 and PC-5,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DimPlot(object = pbmc_small, dims = c(4, 5), reduction = &quot;pca&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FeaturePlot(object = pbmc_small, features = &quot;Xist&quot;, dims = c(4, 5), reduction = &quot;pca&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Seurat documentation is available &lt;a href=&quot;https://cran.r-project.org/web/packages/Seurat/Seurat.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="181" LastEditorUserId="181" LastEditDate="2019-12-19T21:55:40.407" LastActivityDate="2019-12-19T21:55:40.407" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11005" PostTypeId="2" ParentId="10999" CreationDate="2019-12-19T02:54:25.487" Score="2" Body="&lt;p&gt;I basecall separately with guppy. In the output folder specified by &lt;code&gt;--save_path&lt;/code&gt; or &lt;code&gt;-s&lt;/code&gt; there are a whole bunch of .log files.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ls -l *.log | head&#xA;-rw-r--r-- 1 tom tom 5242714 Dec  3 11:04 guppy_basecaller_log-2019-12-02_22-02-36.log&#xA;-rw-r--r-- 1 tom tom 5242718 Dec  3 11:06 guppy_basecaller_log-2019-12-02_22-04-38.log&#xA;-rw-r--r-- 1 tom tom 5242730 Dec  3 11:08 guppy_basecaller_log-2019-12-02_22-06-41.log&#xA;-rw-r--r-- 1 tom tom 5242879 Dec  3 11:11 guppy_basecaller_log-2019-12-02_22-08-50.log&#xA;-rw-r--r-- 1 tom tom 5242811 Dec  3 11:13 guppy_basecaller_log-2019-12-02_22-11-06.log&#xA;-rw-r--r-- 1 tom tom 5242817 Dec  3 11:15 guppy_basecaller_log-2019-12-02_22-13-28.log&#xA;-rw-r--r-- 1 tom tom 5242880 Dec  3 11:18 guppy_basecaller_log-2019-12-02_22-15-47.log&#xA;-rw-r--r-- 1 tom tom 5242816 Dec  3 11:20 guppy_basecaller_log-2019-12-02_22-18-09.log&#xA;-rw-r--r-- 1 tom tom 5242777 Dec  3 11:22 guppy_basecaller_log-2019-12-02_22-20-30.log&#xA;-rw-r--r-- 1 tom tom 5242742 Dec  3 11:25 guppy_basecaller_log-2019-12-02_22-22-47.log&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The information you need for &lt;code&gt;medaka&lt;/code&gt; is at the top of the &lt;strong&gt;first&lt;/strong&gt; log file (alphabetically, and by creation date).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ head -n 13 guppy_basecaller_log-2019-12-02_22-02-36.log &#xA;2019-12-02 22:02:36.255793 [guppy/message] ONT Guppy basecalling software version 3.4.1+ad4f8b9&#xA;config file:        /opt/ont/guppy/data/dna_r9.4.1_450bps_hac.cfg&#xA;model file:         /opt/ont/guppy/data/template_r9.4.1_450bps_hac.jsn&#xA;input path:         raw/minion&#xA;save path:          test_341/minion&#xA;chunk size:         1000&#xA;chunks per runner:  512&#xA;minimum qscore:     7&#xA;records per file:   4000&#xA;num basecallers:    1&#xA;gpu device:         cuda:0&#xA;kernel path:        &#xA;runners per device: 4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you ran live basecalling and you don't have those log files you could also try a recursive grep from the output folder, e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -nr &quot;^model file&quot; .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1021" LastActivityDate="2019-12-19T02:54:25.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11006" PostTypeId="1" CreationDate="2019-12-19T20:03:34.910" Score="1" ViewCount="628" Body="&lt;p&gt;I have merged two &lt;strong&gt;huge&lt;/strong&gt; &lt;code&gt;.gen&lt;/code&gt; files with roughly 500k samples in total - around 3 TB. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish to run an IBD (identity by descent) check to identify if there are identical participants in the two cohorts before doing any further analyses. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the past I have simply used the &lt;code&gt;--genome&lt;/code&gt; argument in PLINK and it would work fine however, in this instance the file is so large that doing such would cause the following error to appear: &lt;em&gt;Error: Out of memory.  The --memory flag may be helpful.&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then opted to use PLINK v2 as it has a &lt;code&gt;--parallel&lt;/code&gt; argument that, apparently, can be used alongside &lt;code&gt;--make-king&lt;/code&gt; and/or &lt;code&gt;--make-king-table&lt;/code&gt; to conduct this IBD check. I have had some trouble understanding the documentation surrounding these aforementioned arguments and would like some clarification as to how to ideally use them. Please assume that I am running on a multi-node, 24 core cluster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the current code I am running:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plink2 --gen testo.gen \&#xA;--sample testo.sample \&#xA;--make-king \&#xA;--make-king-table \&#xA;--parallel 1 24&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And here is what the PLINK documentation says about the arguments for this command:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;--parallel &amp;lt;1-based current job index&amp;gt; &amp;lt;total job pieces&amp;gt;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I have failed to understand what the first argument does, even after playing around with the parameters a little and extensively searching the web for an answer. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would changing this &quot;1-based current job index&quot; do? What does it mean? And for &quot;total job pieces&quot;, will the adjustment of this parameter alter how many cores the jobs are distributed to or is it just an identifier as to how many compartmentalized sets are being made?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, is there any way to specify which nodes you would like to run a job on and specify the number of cores as well? If yes, could you please provide an example? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="4001" LastActivityDate="2019-12-19T20:03:34.910" Title="Running IBD using PLINK --parallel (v2)" Tags="|snp|plink|genotyping|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11007" PostTypeId="2" ParentId="10977" CreationDate="2019-12-20T10:08:24.933" Score="1" Body="&lt;p&gt;Here is the website of the creator of rdatasets I guess with the info you are looking for:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://vincentarelbundock.github.io/Rdatasets/datasets.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://vincentarelbundock.github.io/Rdatasets/datasets.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a short look at it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Package     Item    Title   Rows    Cols    n_binary    n_character     n_factor    n_logical   n_numeric   CSV     Doc&#xA;boot    acme    Monthly Excess Returns  60  3   0   1   0   0   2   CSV     DOC&#xA;boot    aids    Delay in AIDS Reporting in England and Wales    570     6   1   0   0   0   6   CSV     DOC&#xA;boot    aircondit   Failures of Air-conditioning Equipment  12  1   0   0   0   0   1   CSV     DOC&#xA;boot    aircondit7  Failures of Air-conditioning Equipment  24  1   0   0   0   0   1   CSV     DOC&#xA;boot    amis    Car Speeding and Warning Signs  8437    4   1   0   0   0   4   CSV     DOC&#xA;boot    aml     Remission Times for Acute Myelogenous Leukaemia     23  3   2   0   0   0   3   CSV     DOC &#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I assume the Item column are the identifiers.&lt;/p&gt;&#xA;" OwnerUserId="1786" LastActivityDate="2019-12-20T10:08:24.933" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11008" PostTypeId="1" CreationDate="2019-12-20T12:16:08.427" Score="0" ViewCount="22" Body="&lt;p&gt;I'm using blast locally against NT databases and I'm excluding the uncultured and environmental samples by the taxonomy ID. Is there another way to exclude them entirely, without using the taxonomy ID?&lt;/p&gt;&#xA;" OwnerUserId="2686" LastActivityDate="2019-12-20T12:16:08.427" Title="exclude uncultured and environmental samples from NT databases" Tags="|blastn|nt|" AnswerCount="0" CommentCount="2" ClosedDate="2020-01-06T17:20:19.210" ContentLicense="CC BY-SA 4.0" />
  <row Id="11009" PostTypeId="1" CreationDate="2019-12-20T16:30:28.273" Score="0" ViewCount="158" Body="&lt;p&gt;Given a &lt;code&gt;.gb&lt;/code&gt; file and a specific locus in the genome - how can I parse the relevant annotations in Python (i.e., annotations that include that locus)?&lt;/p&gt;&#xA;&lt;br/&gt;&#xA;&lt;p&gt;I import the features using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SeqIO.read('my_gb_file.gb', 'gb').features&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then wish scan/filter sequence meta data.&lt;/p&gt;&#xA;&lt;br/&gt;&#xA;&lt;p&gt;Is there a function or method in &lt;code&gt;Biopython&lt;/code&gt; that does this?&lt;br/&gt;&lt;/p&gt;&#xA;" OwnerUserId="4868" LastEditorUserId="3848" LastEditDate="2022-07-06T13:37:43.607" LastActivityDate="2022-07-06T13:37:43.607" Title="Given a .gb file and a locus - how to get relevant annotations in Python?" Tags="|sequence-annotation|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11010" PostTypeId="1" CreationDate="2019-12-20T17:13:37.633" Score="0" ViewCount="64" Body="&lt;p&gt;I have sample information (sampleinfo) in this format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SampleID    condition   type&#xA;MP_1    A_1 Paired-End&#xA;MP_2    A_1 Paired-End&#xA;MP_3    A_1 Paired-End&#xA;MN_1    A_2 Paired-End&#xA;MN_2    A_2 Paired-End&#xA;MN_3    A_2 Paired-End&#xA;WP_1    B_1 Paired-End&#xA;WP_2    B_1 Paired-End&#xA;WP_3    B_1 Paired-End&#xA;WN_1    B_2 Paired-End&#xA;WN_2    B_2 Paired-End&#xA;WN_3    B_2 Paired-End&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to compare A_1 with A_2, B_1 with B_2 ans also want to compare (A_1+A_2) with (B_1+B_2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is how I used DESeqDataSetFromMatrix&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ddsMat &amp;lt;- DESeqDataSetFromMatrix(countData = seqdata[,rownames(sampleinfo)], colData =  sampleinfo, design = ~ condition)&#xA;ddsMat &amp;lt;- DESeq(ddsMat)&#xA;&#xA;res.05_FC0&amp;lt;-results(ddsMat,contrast = c(&quot;condition&quot;,&quot;A_1&quot;,&quot;A_2&quot;),alpha=0.05,lfcThreshold = 0)&#xA;res.05_FC0&amp;lt;-results(ddsMat,contrast = c(&quot;condition&quot;,&quot;B_1&quot;,&quot;B_2&quot;),alpha=0.05,lfcThreshold = 0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I compare (A_1+A_2) with (B_1+B_2) using the contrast argument in results function?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course I can make a new sampleinfo table, but I want to use the same sampleinfo as above. I tried to read about numeric contrast, but I am unable to use it correctly.&lt;/p&gt;&#xA;" OwnerUserId="2040" LastActivityDate="2019-12-25T11:28:14.030" Title="Comparing factor levels in deseq2" Tags="|deseq2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11011" PostTypeId="1" AcceptedAnswerId="14021" CreationDate="2019-12-20T18:53:39.857" Score="0" ViewCount="90" Body="&lt;p&gt;I need to obtain sample data from modern humans in fasta format. I just need some megabytes of data from every individual. I actually use a script that obtains the cram file from here (ftp.1000genomes.ebi.ac.uk) and then processes it to obtain the fasta file. &#xA;&lt;strong&gt;The problem is that cram files are large, slow to download and slow to process. It takes days to get the samples.&#xA;Is there a better way to get these samples in fasta format?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The script already makes use of samtools to retrieve only the part of the bam file it needs but doesn't help much. Cram files are still gigabytes large for only a few megabytes of data that I need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the same problem with data from the 1000 genomes project.&lt;/p&gt;&#xA;" OwnerUserId="5190" LastActivityDate="2020-07-27T10:03:45.767" Title="Obtaining HGDP project data in fasta format" Tags="|fasta|samtools|human-genome|1000genomes|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11012" PostTypeId="2" ParentId="11009" CreationDate="2019-12-21T13:20:58.357" Score="1" Body="&lt;p&gt;Biopython is the main package for this. It's only a few lines, so it is not reinventing the wheel.&lt;/p&gt;&#xA;&lt;p&gt;So you want to iterate across the features table (a list in Biopython) of a record and find the case where the qualifier['locus'][0] matches your query.&lt;/p&gt;&#xA;&lt;p&gt;The things to watch out for are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;filter by &lt;code&gt;type&lt;/code&gt; (CDS?)&lt;/li&gt;&#xA;&lt;li&gt;entries where there is not key among the qualifiers, so add error catching (try: ... except Exception: pass&lt;/li&gt;&#xA;&lt;li&gt;the values of the qualifiers of a feature are a list. so add a [0]&lt;/li&gt;&#xA;&lt;li&gt;multithreading.Pool might help if speed is a worry (which it really shouldn't). Asyncio and thread are not the way (both single core).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;A nice handy way to do operations on the &lt;code&gt;features&lt;/code&gt; is to convert it to a pandas dataframe.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.SeqFeature import SeqFeature&#xA;from Bio import SeqIO&#xA;from Bio.Seq import Seq&#xA;import pandas as pd&#xA;&#xA;        def convert(feat: SeqFeature, flatten:bool=True):&#xA;    if flatten:&#xA;        clean = lambda v: v[0] if isinstance(v, list) else v&#xA;        qualifiers = {k: clean(v) for k, v in feat.qualifiers.items()}&#xA;    else:&#xA;        qualifiers = feat.qualifiers&#xA;    return {'id': feat.id,&#xA;            'type': feat.type,&#xA;            'location_strand': feat.location.strand, &#xA;            'location_start': feat.location.start,&#xA;            'location_end': feat.location.end,&#xA;            **qualifiers}&#xA;&#xA;transcript = SeqIO.read('.gb', 'genbank')&#xA;feats = pd.DataFrame([convert(feat) for feat in transcript.features])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This means that in a jupyter notebook one can display the table to look at it (&lt;code&gt;feats&lt;/code&gt; as last line in a cell or &lt;code&gt;IPython.display.display(feats)&lt;/code&gt;), export it to a csv (&lt;code&gt;feats.to_cvs('.csv')&lt;/code&gt;) or subset it:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;subfeats = feats.loc[(feats['type'] == 'exon') &amp;amp; (feats['location_start']  &amp;gt; 100)&amp;amp; (feats['location_end']  &amp;lt; 500)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2021-09-13T17:13:20.563" LastActivityDate="2021-09-13T17:13:20.563" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11015" PostTypeId="1" CreationDate="2019-12-23T13:04:45.867" Score="0" ViewCount="176" Body="&lt;p&gt;I am new here so forgive me if I don't provide all the details in the first go. There is a very similar question (&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/10914/how-to-solve-dimx-must-have-a-positive-length-at-running-combat-function-in&quot;&gt;how to solve “dim(X) must have a positive length” at running ComBat function in R&lt;/a&gt;) but the solution did not work for me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am getting &lt;code&gt;Error in apply(dat[, batch == batch_level], 1, function(x) { :    dim(X) must have a positive length&lt;/code&gt; when I use the &lt;code&gt;sva&lt;/code&gt; library's &lt;code&gt;comBat()&lt;/code&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It works when I use the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dim(log_proteins_norm_sorted)&#xA;[1] 5415   66&#xA;&#xA;&amp;gt; log_proteins_norm_sorted[1:5,1:5]&#xA;      1027     1075     1116     1119    11211&#xA;1 22.50513 20.64312 21.56182 22.01455 23.17497&#xA;2 21.67946 20.90994 21.61928 21.92025 21.98503&#xA;3 21.06812 20.52647 20.72076 20.79732 21.38957&#xA;4 18.95485 19.70027 18.39504 20.20218 19.79887&#xA;5 22.75966 22.32306 22.32936 22.39271 22.84204&#xA;&#xA;&amp;gt; dim(samples_pheno)&#xA;[1] 66 24&#xA;&#xA;&amp;gt; samples_pheno[1:3,1:3]&#xA;     Trial Number  Trial Sample&#xA;1027      222       A   1027&#xA;1075      223       A   1075&#xA;1116      224       A   1116&#xA;&#xA;&amp;gt; proteins_combat = ComBat(dat= as.matrix(log_proteins_norm_sorted), batch=samples_pheno$Batch ,  par.prior = T)&#xA;Found10batches&#xA;Adjusting for0covariate(s) or covariate level(s)&#xA;Standardizing Data across genes&#xA;Fitting L/S model and finding priors&#xA;Finding parametric adjustments&#xA;Adjusting the Data&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So as you can see, this works. However, when I subset some of the samples and reduce the sample number from 66 to 55 and run comBat, I get the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dim(log_proteins_MRD_sorted)&#xA;[1] 5415   55&#xA;&amp;gt; log_proteins_MRD_sorted[1:5,1:5]&#xA;      1027     1075     1116     1119     1217&#xA;1 22.55492 20.69292 21.61162 22.06435 21.86900&#xA;2 21.72926 20.95974 21.66907 21.97005 21.35203&#xA;3 21.11791 20.57627 20.77055 20.84712 21.01896&#xA;4 19.00465 19.75006 18.44483 20.25198 18.95107&#xA;5 22.80946 22.37285 22.37915 22.44250 22.50524&#xA;&#xA;&amp;gt; dim(samples_pheno3)&#xA;[1] 55 24&#xA;&amp;gt; samples_pheno3[1:3,1:3]&#xA;     Trial Number  Trial Sample&#xA;1027      222        A   1027&#xA;1075      223        A   1075&#xA;1116      224        A   1116&#xA;&#xA;&amp;gt; proteins_combat_MRD = ComBat(dat= as.matrix(log_proteins_MRD_sorted), batch=samples_pheno3$Batch ,  par.prior = T)&#xA;Error in apply(dat[, batch == batch_level], 1, function(x) { : &#xA;  dim(X) must have a positive length&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In case you wanted to know, I am working with the same original file for &lt;code&gt;log_proteins_MRD_sorted&lt;/code&gt; and &lt;code&gt;log_proteins_norm_sorted&lt;/code&gt;. I just removed the 11 samples from the 66 samples and then normalised the data again (hence slight difference in values between the two objects. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For &lt;code&gt;samples_pheno3&lt;/code&gt;, I run this code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; #Subset the phenotype data to those samples with only MRD data&#xA;&amp;gt; samples_pheno2 &amp;lt;- samples_pheno[-c(5,6,11,16,21, 26, 29,30,32, 45,56),]&#xA;&amp;gt; MRD_NA1 &amp;lt;- rownames(samples_pheno2)[!is.na(samples_pheno2$MRD)]&#xA;&amp;gt; samples_pheno3 &amp;lt;- samples_pheno2[MRD_NA1,]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also this is out my data looks like for the batches:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; samples_pheno3$Batch&#xA; [1] 7  5  6  9  7  7  7  7  8  8  8  8  9  9  9  9  10 10 10 10 11 11 11 2  7  2  2  2  9  3  3  3  3  3  3  3  7 &#xA;[38] 10 3  4  10 4  4  8  4  4  4  5  5  5  8  11 5  5  5 &#xA;Levels: 2 3 4 5 6 7 8 9 10 11&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So just by subsetting the data to 55 samples, it doesn't seem to work for some reason. Is it the way I am subsetting the data which is confusing the code?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be grateful for your advice. &lt;/p&gt;&#xA;" OwnerUserId="6672" LastEditorUserId="3541" LastEditDate="2019-12-23T20:05:48.943" LastActivityDate="2019-12-23T21:21:27.803" Title="Error in apply(dat[, batch == batch_level], 1, function(x) { : dim(X) must have a positive length" Tags="|combat|sva|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11016" PostTypeId="2" ParentId="11015" CreationDate="2019-12-23T21:21:27.803" Score="1" Body="&lt;p&gt;With the help of my very capable local bioinformatician, I have found a solution. Essentially, the problem was that when I reduced the sample size from 66 to 55, there was a batch number which only had 1 sample in it. When I removed that sample from the analysis, and ensured that each batch had at least 2 samples in it, the &lt;code&gt;ComBat()&lt;/code&gt; then worked nicely!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this helps future googlers.&lt;/p&gt;&#xA;" OwnerUserId="6672" LastActivityDate="2019-12-23T21:21:27.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11017" PostTypeId="1" CreationDate="2019-12-23T21:56:40.187" Score="0" ViewCount="3702" Body="&lt;p&gt;I would like to know how to change the UMAP used in &lt;code&gt;Dimplot&lt;/code&gt; and &lt;code&gt;FeaturePlot&lt;/code&gt; from Seurat: how we can get the x-axis and the y-axis like UMAP-1 and UMAP-2 if I want to use UMAP-4 and UMAP-5. How to modify the code? It is easy to change the PC by using &lt;code&gt;DimPlot(object = pbmc_small, dims = c(4, 5), reduction = &quot;PCA&quot;)&lt;/code&gt; but if I changed to &lt;code&gt;reduction = &quot;UMAP&quot;&lt;/code&gt;, I got the error &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&quot;Error in Embeddings(object = object[[reduction]])[cells, dims] : subscript out of bounds Calls: DimPlot Execution halted&quot;. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does anyone have any idea how to solve this problem? &lt;/p&gt;&#xA;" OwnerUserId="3171" LastEditorUserId="8777" LastEditDate="2020-05-14T16:20:53.713" LastActivityDate="2020-05-14T16:20:53.713" Title="how to change the UMAP use in the dimplot and feature plot" Tags="|seurat|single-cell|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11018" PostTypeId="1" AcceptedAnswerId="11022" CreationDate="2019-12-24T01:31:45.683" Score="0" ViewCount="132" Body="&lt;p&gt;WGCNA carefully takes advantage of server parallelization in several functions, and I can switch this on with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;enableWGCNAThreads(nThreads=28)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That speeds up some functions significantly, but the two bottleneck functions in the pipeline (in my experience) are part of the &quot;Projective k-means&quot; operation, primarily &lt;code&gt;&quot;..k-means clustering..&quot;&lt;/code&gt; and &lt;code&gt;&quot;..merging smaller clusters..&quot;&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, parallelization does not work for those two functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Am I missing something?  How can I parallelize these functions?  Otherwise I'm wasting HPC resources and cluster time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For reference, I'm working on a 28-processor HPC node with 96GB RAM, and I'm trying to run analysis on a 15,000-square correlation matrix.&lt;/p&gt;&#xA;" OwnerUserId="6498" LastEditorUserId="6498" LastEditDate="2019-12-24T13:07:44.480" LastActivityDate="2019-12-24T13:07:44.480" Title="Parallelizing WGCNA k-means clustering and merging smaller clusters" Tags="|r|wgcna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11019" PostTypeId="1" CreationDate="2019-12-24T04:08:03.790" Score="3" ViewCount="6188" Body="&lt;p&gt;A very basic question: What exactly is a read count in shotgun sequencing and how is it calculated? I'm particularly interested in understanding the definition of the read count in the context of shotgun metagenomic sequencing though I assume the same concept applies to other types of shotgun sequencing as well. Surprisingly, a Google search did not results in any answers!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context of metagenomics, the taxonomic profiling of metagenomic data provides a list of microbial taxa and their &quot;counts&quot;. These counts are often normalized to account for genome size or are normalized to convert counts to proportions. Understanding the concept of a read count may help me better understand all these. &#xA;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="6680" LastEditorUserId="6680" LastEditDate="2019-12-24T05:31:07.670" LastActivityDate="2019-12-24T14:54:02.577" Title="What is a read count?" Tags="|ngs|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11020" PostTypeId="1" CreationDate="2019-12-24T04:18:04.023" Score="2" ViewCount="581" Body="&lt;p&gt;I'm fairly new to the field of bioinformatics and ran into a question while reading a paper I found on bioRxiv. The overall setting of the paper is using multi-task deep neural networks for kinase classification. The paper that I'm referring to is &lt;a href=&quot;https://doi.org/10.1101/512459&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Kinome-Wide Activity Classification of Small Molecules by Deep Learning (Allen et al., 2019)&lt;/em&gt;&lt;/a&gt; for anyone who's curious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll get straight into the specific figure that I'm having trouble understanding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dCU89.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dCU89.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that questions like these are a bit discouraged on SE, but I'm having trouble on how to start regarding the interpretation of this figure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on my understanding, the point seems to be that multitaks DNN's have the highest overall EF/EFmax ratio amounts, but what does this ratio signify?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did some research and according to Wikipedia an &quot;enrichment factor&quot; in terms of bioinformatics is a concept &quot;to measure the added value of a search tool over another one or over the homogeneous distribution in the genome population.&quot; Does this mean that multitask DNN's are &quot;better&quot; than the other models, due to its higher number of EF/EFmax points?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if we look at the curves on the right, it seems to say that for fewer number of compounds, the ratio for multitask DNN's is lower...&lt;/p&gt;&#xA;" OwnerUserId="6679" LastEditorUserId="73" LastEditDate="2019-12-27T03:47:12.963" LastActivityDate="2019-12-31T20:16:13.920" Title="What is the significance of enrichment factor regarding machine learning methods?" Tags="|machine-learning|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11021" PostTypeId="2" ParentId="11019" CreationDate="2019-12-24T04:34:12.000" Score="2" Body="&lt;p&gt;Typically read count is the total number of reads going into the analysis. It could be based off single or multiple sequencing libraries.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also it can be used to describe the number of reads that align to a region of the reference.  Depth or coverage are also terms used in this case. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will need to be more specific in your use case for an accurate definition. &lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2019-12-24T04:34:12.000" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11022" PostTypeId="2" ParentId="11018" CreationDate="2019-12-24T06:17:43.733" Score="2" Body="&lt;p&gt;The bad news is that, indeed, projectiveKMeans is not parallelized and I am not sure how much of it is (easily) parallelizable. The good news is that with 15k features (genes) and 96GB RAM you don't need to run preclustering at all. Just analyze the whole data set in one block.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you for some reason insist on splitting the data into blocks, make sure the BLAS R uses on the cluster is a fast BLAS, ideally one that can parallelize matrix operations. A lot of time in merging of small clusters is spent calculating singular value decompositions, which becomes much faster using a good BLAS. HPC clusters will often have Intel KML available, or the admins could install OpenBLAS.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2019-12-24T06:17:43.733" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11023" PostTypeId="2" ParentId="11017" CreationDate="2019-12-24T09:04:21.300" Score="0" Body="&lt;p&gt;Change &lt;code&gt;n.components&lt;/code&gt; / &lt;code&gt;max.dims&lt;/code&gt; (depending on your seurat version) to 5 when you call &lt;code&gt;runUMAP&lt;/code&gt; and you should have dim 4 and 5 available.&lt;/p&gt;&#xA;" OwnerUserId="3818" LastActivityDate="2019-12-24T09:04:21.300" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11024" PostTypeId="2" ParentId="11019" CreationDate="2019-12-24T14:54:02.577" Score="5" Body="&lt;p&gt;To add to the answer from Bioathlete, &quot;reads&quot; in the traditional genomic sense are the string of nucleotide data inferred from the sequencer from a given fragment of genetic material (could be Illumina, Nanopore, PacBio, 10X, etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So read count first applies for all of the &quot;reads&quot; defined above generated by the sequencer. From a NovaSeq, the total number of bases from a single flowcell is up to 3 trillion bases at 150bp each read for the highest throughput flowcell (so a read count of 3,000,000,000,000/150 = 20B total read count). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The read count for a single Nanopore flowcell is roughly (30 billion bases/avg read length), but the read lengths can be SUPER long (100kb compared to 150bp is substantial). Long read technologies are generally used for different analyses than short reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So all of the above reads would be defined as &quot;raw reads&quot;, meaning they are reads not yet analysed in any way to be used for data analysis. So using illumina as an example, for a single Whole Genome sequencing sample we might have 800,000,000 raw reads. We then use an aligner like bwa mem to map the reads to a single reference genome (or in your case you would use a different aligner to map to multiple genomes). From here we still have the concept of raw reads, but we also now have the total number of mapped reads. A typical good illumina library for a single reference genome will have &gt;90% mapping rate for a human sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can then say we have 800,000,000 raw reads, and 720,000,000 mapped reads with a mapping to a reference genome of 90%. So we can compare &quot;raw read count&quot; to &quot;mapped read count&quot; to get an idea of the quality of our library. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of metagenomics, you would consider your &quot;raw read count&quot; to still be the total number of reads you got off the sequencer, but the &quot;mapped read count&quot; would be the sum of the reads mapped to each individual reference genome to get a total quantity of &quot;mapped reads&quot; which could give you an indication of library quality. I'm not sure what a good metagenomics library &quot;mapped read count&quot; is, but I'm sure you can find that information. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, we want to get an idea of the depth of coverage for each mapped read to the genome. This will tell us how well our mapped reads actually covered the reference genome or genomes we are interested in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope this was helpful!&lt;/p&gt;&#xA;" OwnerUserId="3669" LastActivityDate="2019-12-24T14:54:02.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11026" PostTypeId="1" CreationDate="2019-12-25T09:19:36.807" Score="1" ViewCount="196" Body="&lt;p&gt;I have a strong mathematical and computer background (also with machine learning) and would like to switch to bioinformatics. I know a fair amount of biology. But I find it hard to find real world practical training in this field. So I would like to try to replicate some bioinformatics papers (with their provided data). Good beginner papers for me would be shorter ones where the biological problem is well stated and the problem is also fitted in to the bigger scope (or is just easily understandable). Do you have any papers to recommend?&lt;/p&gt;&#xA;" OwnerUserId="4803" LastActivityDate="2019-12-27T01:14:37.647" Title="Replicate papers (beginner)" Tags="|literature-search|reproducibility|literature|" AnswerCount="1" CommentCount="1" ClosedDate="2019-12-27T03:02:42.527" ContentLicense="CC BY-SA 4.0" />
  <row Id="11027" PostTypeId="2" ParentId="11010" CreationDate="2019-12-25T11:28:14.030" Score="1" Body="&lt;p&gt;You can use &lt;code&gt;contrast = c(0.5, 0.5, -0.5, -0.5)&lt;/code&gt;, which means &quot;the average of the first two coefficients minus the average of the second two.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-12-25T11:28:14.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11028" PostTypeId="1" AcceptedAnswerId="11029" CreationDate="2019-12-25T14:38:50.560" Score="0" ViewCount="145" Body="&lt;p&gt;May I know what's the use of &lt;code&gt;Electronic Phenotyping using EHR data&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did refer this &lt;a href=&quot;https://currentprotocols.onlinelibrary.wiley.com/doi/full/10.1002/cphg.80&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt; but have few questions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand that &lt;code&gt;Phenotype&lt;/code&gt; is a set of criteria that you apply on your EHR data to select patients of interest from EHR database.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However what I am trying to understand is the difference between ordinary SQL query and electronic Phenotype&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If both are same, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) what I would like to know is the use of &lt;code&gt;electronic phenotyping&lt;/code&gt;? How it is used? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Why do we have &lt;code&gt;pheKB&lt;/code&gt; repository and other repositories? What's the use in validating them in multiple sites?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3) Is it like identifying patients real time? I am not able to get the usefulness of this. I understand that we select patients of our interest (cohort), but i can just write an ordinary SQL and send it to my EHR database to identify patients. but what's special about Electronic Phenotype?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) I also see that they use multiple conditions (Lab tests, Medications, Diagnosis etc) to identify patients. What am I missing here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone help me by giving an simple example? I am new to healthcare and trying to learn. Would really be helpful&lt;/p&gt;&#xA;" OwnerUserId="6380" LastActivityDate="2019-12-25T20:29:03.333" Title="Use of Electronic Phenotype in EHR" Tags="|sequence-alignment|database|genome|statistics|machine-learning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11029" PostTypeId="2" ParentId="11028" CreationDate="2019-12-25T20:29:03.333" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I understand that Phenotype is a set of criteria that you apply on your EHR data to select patients of interest from EHR database.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;No, a phenotype is a way of behavior or other observed characteristic of a person, resulting from a combination on its underlying biology and the environment it's in. Think things like, height, running speed, strength, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the context in which you're interested, this would be more properly, &quot;the clinical presentation of a disease or disorder&quot;.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Step one of diagnosing a patient is defining exactly how they present (i.e., exactly what their maladies, aberrant electrolyte levels, genetic mutations, etc. are).&lt;/li&gt;&#xA;&lt;li&gt;An algorithm that only works on the test population is useless. Google &quot;model overfitting&quot;.&lt;/li&gt;&#xA;&lt;li&gt;The point of an algorithm is to take the EHR data and progress to a diagnosis. This isn't real time, you'd just run a batch of algorithms as test results come in.&lt;/li&gt;&#xA;&lt;li&gt;You need patients in your test/validation sets to have the required inputs for your algorithm.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;FYI, I don't know of any regulars on this site that deal with EHR data. If there's a community of such people on the web they must be elsewhere.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2019-12-25T20:29:03.333" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11030" PostTypeId="1" CreationDate="2019-12-25T21:33:16.343" Score="3" ViewCount="455" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rlI7l.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rlI7l.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hi, I was wondering whether we can find the differentially expressed genes between the &lt;code&gt;Double-KO&lt;/code&gt; and the &lt;code&gt;Shox2-KO&lt;/code&gt;. For the &lt;code&gt;FindMarker()&lt;/code&gt; function, only see the differential expressed gene between different identities. I don't know whether we can find the differential expressed gene between only these two samples.&lt;/p&gt;&#xA;" OwnerUserId="3171" LastEditorUserId="3541" LastEditDate="2019-12-28T12:53:40.283" LastActivityDate="2022-05-16T19:01:15.090" Title="Finding differentially expressed genes between two samples" Tags="|seurat|single-cell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11031" PostTypeId="1" AcceptedAnswerId="11032" CreationDate="2019-12-25T10:28:19.357" Score="4" ViewCount="959" Body="&lt;p&gt;When I was drawing conformation of Autodock result, I noticed that Autodock generated pdbqt from dlg has some structural problem (e.g. Benzene ring missing), and wants to correct this. However I noticed that the atom order of pdbqt doesn't match with the original mol2's, and caused some problems when I tried to fix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So how could I reorder pdbqt according to mol2's atom coordinate?&lt;/p&gt;&#xA;" OwnerUserId="6684" OwnerDisplayName="march_happy" LastEditorUserId="6322" LastEditDate="2019-12-27T00:57:59.980" LastActivityDate="2019-12-27T00:57:59.980" Title="How could I match atom orders between a .mol2 and a .pdb?" Tags="|proteins|format-conversion|docking|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11032" PostTypeId="2" ParentId="11031" CreationDate="2019-12-26T15:32:29.717" Score="5" Body="&lt;p&gt;Atom reordering is a common problem in compchemistry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rdkit (a python package) can do this, but it is limited by the formats it can read and mol2 files are a bit hit or miss. It works really well with SMILES, SMARTS and mol (sdf) files. But the writing may cause problems with Brookhaven pdb and mol2 files.&#xA;So the formats you have are both problematic. One is a pdbqt (a PDB file with partial charges (Q, generally Gasteiger calculated) and atom type (T) and the other a mol2. Let's pretend first that they are not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also the hydrogens might be made implicit depending on your mol2 file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from rdkit import Chem&#xA;ref = Chem.MolFromMol2File('ligand.mol2')&#xA;target = Chem.MolFromPDBFile('ligand.pdb')&#xA;atoms = target.GetSubstructMatches(ref) #: List[int]&#xA;new = Chem.RenumberAtoms(target, atoms)&#xA;Chem.MolToMolFile(new, 'neoligand.mol')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Extra steps would be required for the protons and if your starting ligand was not from a file, but a SMILES.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mol = Chem.AddHs(mol) #protonate explicitly&#xA;Chem.GetSSSR(mol) #resonance fixing&#xA;AllChem.EmbedMolecule(mol) #initialise for 3d.&#xA;AllChem.UFFOptimizeMolecule(mol, maxIters=2000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So this is all nice, but the formats may not work. Luckily openbabel can deal with this problem.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;obabel -i mol2 input.mol2 -o pdbqt -O output.pdbqt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, one thing to keep an eye out is what will be recalculated. Say, sdf files do not have partial charges or what PDB calls &quot;atom names&quot; and Rdkit calls &quot;atom labels&quot; (About the latter, I actually wrote &lt;a href=&quot;https://github.com/matteoferla/AtomicRenamer&quot; rel=&quot;noreferrer&quot;&gt;a script to relabel atom labels in pdbs from mol files&lt;/a&gt;), while mol2 and PDBQT have.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2019-12-26T15:32:29.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11034" PostTypeId="2" ParentId="11026" CreationDate="2019-12-27T01:14:37.647" Score="1" Body="&lt;p&gt;Bioinformatics is a big and very heterogeneous field with a lot of variation, so it's hard to recommend something that covers everything. That said, I think the best you'll do is to build familiarity with some of the common tools with some canned analyses. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One possible option would be some of the scRNA-seq workflows from the Pachter group, they seem to be fairly conscientious about documentation. &lt;a href=&quot;https://www.kallistobus.tools/getting_started&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is one such canned analysis&lt;/a&gt;, &lt;a href=&quot;https://academic.oup.com/bioinformatics/article-abstract/35/21/4472/5487510?redirectedFrom=fulltext&quot; rel=&quot;nofollow noreferrer&quot;&gt;here is the paper&lt;/a&gt;, &lt;a href=&quot;https://liorpachter.wordpress.com/2019/06/21/near-optimal-single-cell-rna-seq-pre-processing/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here is a blog post about it that suggests next steps and discusses at a high level&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;scRNA-seq is just what's been trendy for a few years, next year it will be something different. But it at least gets you familiar with some of the file formats and concepts such as alignment, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2019-12-27T01:14:37.647" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11035" PostTypeId="2" ParentId="11020" CreationDate="2019-12-27T03:46:40.930" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;I did some research and according to Wikipedia an &quot;enrichment factor&quot; in terms of bioinformatics is...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Where possible, it's best to use explanations provided in the original source. This is because definitions could be different, depending on who is writing. Here's some descriptive information from the paper that may be more informative:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The enrichment factor (EF) is another metric we use and is very&#xA;  popular for use in evaluation of virtual screening performance. EF&#xA;  denotes the quotient of true actives among a subset of predicted&#xA;  actives and the overall fraction of actives.... To evaluate the&#xA;  maximum achievable enrichment factor (EFmax), the maximum number of&#xA;  actives among the percentage of tested compounds is divided by the&#xA;  fraction of actives in the entire dataset. To more consistently&#xA;  quantify enrichment at 0.1% of all compounds tested, we report the&#xA;  ratio of EF/ EFmax. This normalized metric is useful because EF values&#xA;  are not directly comparable across different datasets due to the&#xA;  maximum possible EF being constrained by the ratio of total to active&#xA;  compounds and the evaluated fraction as shown in the equation above.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So for the purpose of this paper, they're saying that this ratio is an appropriately normalised substitute for a plain enrichment factor. High enrichment factor == good; low enrichment factor == bad.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given the information you've provided, I'd agree that the curves in this figure appear to suggest that the multitask DNN performs worse with smaller numbers of active compounds (where Naive Bayes or Logistic regression would be a better choice). It's hard to tell from the graph (and legend) what the curves represent, and the clipping at EF/EFmax = 0 doesn't help.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The curves have a strange shape, suggesting the number of active compounds is dependent on the EF/EFmax ratio, rather than the other way round (which would make more sense to me). If I consider the curves to be something like a smoothed mean of the active compound number, then any place where there are fewer dots (i.e. low EF/EFmax for Multitask DNN) will have more associated error. For Multitask DNN, I see a large gap between 0.0 and 0.2, with lots of outliers on the 0.0 line (for all methods), so presumably the 0.0 outliers are skewing the curves.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that this is a preprint, the generation of the curves isn't specified, and the other figures (especially Figure 3) seem to tell a different story (i.e. that multi-task DNN performs better at all scales), I'd recommend contacting the authors to ask / clarify what the curves represent, or ignoring the curves entirely.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2019-12-27T04:10:15.980" LastActivityDate="2019-12-27T04:10:15.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11036" PostTypeId="1" CreationDate="2019-12-27T17:18:41.910" Score="0" ViewCount="3984" Body="&lt;p&gt;Hello brilliant researchers!&#xA;After virtual screening, I performed MD Simulation via GROMACS to check the stability of the protein-top hit complex by following this tutorial &lt;a href=&quot;http://www.mdtutorials.com/gmx/complex/09_analysis.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.mdtutorials.com/gmx/complex/09_analysis.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that, I facing some difficulties in interpreting the resulted RMSD graph as it is quite different from most of the published RMSD graph. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on the graph attached, how to interpret it? Is AChE-dexketoprofen complex (Red) more stable (or conserved) than BuChE-dexketoprofen complex (Black)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/H5Sip.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/H5Sip.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3760" LastActivityDate="2020-01-28T11:01:15.127" Title="How to interpret the GROMACS MD Simulation RMSD Graph?" Tags="|proteins|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11037" PostTypeId="1" AcceptedAnswerId="11038" CreationDate="2019-12-28T01:30:35.137" Score="8" ViewCount="2693" Body="&lt;p&gt;When a DNA sequence is sequenced, I've only ever dealt with A,T,C,G and N which indicates un-identifiable bases. However, I came across a 'k' recently and I had asked another researcher who gave me an answer for what 'k' represents but I don't quite recall. It can't just be an anomaly in that one sequence file. Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="2519" LastEditorUserId="3848" LastEditDate="2020-07-31T10:46:41.470" LastActivityDate="2020-07-31T10:46:41.470" Title="What is 'k' in sequencing?" Tags="|sequence-alignment|ngs|phylogeny|phylogenetics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11038" PostTypeId="2" ParentId="11037" CreationDate="2019-12-28T01:52:09.000" Score="16" Body="&lt;p&gt;See &lt;a href=&quot;https://www.bioinformatics.org/sms/iupac.html&quot; rel=&quot;noreferrer&quot;&gt;IUPAC codes&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5uFOM.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5uFOM.png&quot; alt=&quot;IUPAC codes&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, as you can see above, &lt;code&gt;K&lt;/code&gt; means &quot;Either G or T&quot;.&lt;/p&gt;&#xA;" OwnerUserId="6690" LastEditorUserId="298" LastEditDate="2019-12-28T15:57:38.753" LastActivityDate="2019-12-28T15:57:38.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11039" PostTypeId="2" ParentId="11037" CreationDate="2019-12-28T04:12:47.557" Score="4" Body="&lt;p&gt;It is recommended you learn the degenerate nucleotide code. In sequencing it can signify poor quality sequence data, but in primer design it is useful. R (mutation within purines) and Y (mutation within pyrimidines) are common. K, a purine to pyrimadine or pyrimadine to purine mutation is, in my opinion, rare. I would treat a K mutation with caution and consider the triplet codon around it, i.e. if it is part of a protein gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most phylogeneitcs programs will work with the degenerate nucleotide code, so in theory you can still obtain useful information with it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-12-28T15:04:40.540" LastActivityDate="2019-12-28T15:04:40.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11041" PostTypeId="2" ParentId="11030" CreationDate="2019-12-28T12:52:30.557" Score="1" Body="&lt;p&gt;If I understand your question correctly, you can simply use &lt;code&gt;SetIdent()&lt;/code&gt; to change the &quot;default identity&quot; to your samples and then use &lt;code&gt;FindMarker()&lt;/code&gt; with the &lt;code&gt;ident.1 = &quot;Double-KO&quot;&lt;/code&gt; and &lt;code&gt;ident.2 = &quot;Shox2-KO&quot;&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you can &quot;collapse&quot; or &quot;summarize&quot; your single cell data into a pseudo-bulk and then use one of the bulk RNA-seq packages.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-12-28T12:52:30.557" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11042" PostTypeId="1" CreationDate="2019-12-29T03:38:30.243" Score="7" ViewCount="10477" Body="&lt;p&gt;My annotation file is in .gff format. I would like to convert it to .gtf format or to know if there is a way to directly download the annotation file in .gtf format?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working on sequences from the P Falicparum 3D7.&lt;/p&gt;&#xA;" OwnerUserId="6694" LastEditorUserId="6955" LastEditDate="2020-01-30T23:03:54.960" LastActivityDate="2021-12-23T20:11:09.687" Title="How to convert gff to gtf?" Tags="|rna-seq|gtf|gff|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11043" PostTypeId="1" AcceptedAnswerId="11635" CreationDate="2019-12-29T06:15:41.397" Score="3" ViewCount="160" Body="&lt;p&gt;I am working on a clinical data analysis project and am from IT domain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I am analyzing the data, I came to know that though some of the variables are not statistically significant, they are clinically significant.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also read that difference in median values between case and control group for variables is used as a measure to indicate clinical significance&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone help me understand this with a simple example as to why difference in median between groups is considered for clinical significance?&lt;/p&gt;&#xA;" OwnerUserId="6380" LastEditorUserId="7374" LastEditDate="2020-03-12T20:51:20.593" LastActivityDate="2024-02-09T15:31:29.483" Title="Difference between clinical significance and statistical significance" Tags="|database|errors|validation|p-values|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11044" PostTypeId="2" ParentId="11042" CreationDate="2019-12-29T09:46:01.213" Score="5" Body="&lt;p&gt;You can use &lt;code&gt;gffread&lt;/code&gt; to convert &lt;code&gt;gff&lt;/code&gt; to &lt;code&gt;gtf2&lt;/code&gt;, below is from &lt;a href=&quot;http://ccb.jhu.edu/software/stringtie/gff.shtml&quot; rel=&quot;noreferrer&quot;&gt;the manual&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;In order to see the GTF2 version of the same transcripts, the -T&#xA;  option should be added:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gffread -E annotation.gff -T -o- | more&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;The examples above also show that gffread can be used to convert a&#xA;  file between GTF2 and GFF3 file formats.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-12-29T09:46:01.213" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11045" PostTypeId="2" ParentId="11036" CreationDate="2019-12-29T10:32:15.677" Score="1" Body="&lt;p&gt;The opposite-ish.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your graph is showing only the ligand, but RMSD is generally used for structures as a whole —so I'll mention about whole protein for future readers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;RMSD is a measure of how much does a structure deviate in position from a reference. Your ligand in the protein in red &quot;wiggles&quot; more than the one in black.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Structure RSMD&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If this were RMSD of the protein (and not the index group for the ligand), it would not really mean that the black protein is more stable. Stability is to do with force-field energetics not position. Some protein have two state positions, say MBP is a flexing dumbbell. This would be seen as a jump in the RSMD graph, similar to that seen for the black trace.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the command you used was based on &lt;code&gt;gmx rms -s em.tpr -f md_0_10_center.xtc -n index.ndx -tu ns -o rmsd_jz4.xvg&lt;/code&gt; from the page you linked. Key here is the index &lt;code&gt;-n index.ndx&lt;/code&gt;, which limits it to the ligand. Before this happened &lt;code&gt;gmx trjconv&lt;/code&gt; aligned the protein in the trajectory, so that common mistake was not done.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Ligand RSMD&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In terms of ligand RMSD, lower deviation/less wobble correlates with more stable binding, but energetics (&lt;code&gt;gmx energy&lt;/code&gt;) will tell you how strongly does it bind.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The black trace has a jump, so your ligand in the black trace (which wobbles less) has an alternative binding mode. This is why you are looking at RSMD.&#xA;To explore this further you can:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;look at the trajectory&lt;/li&gt;&#xA;&lt;li&gt;in PyMOL using the command &lt;code&gt;split_states&lt;/code&gt; twice, once for the first and then the last state, followed by &lt;code&gt;align&lt;/code&gt; of the two&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;If it were the whole protein&lt;/em&gt;, looking at the per bond matrix &lt;code&gt;-bm&lt;/code&gt; option in Gromacs rms and spotting the change.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;3Å??&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I strongly suggest checking the trajectory because both have a deviation of 3Å, which for a ligand is not good.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Did the energy minimisation converge properly?&lt;/li&gt;&#xA;&lt;li&gt;If it is because the proteins themselves wobble substantially, revisit &lt;code&gt;gmx trjconv&lt;/code&gt; with a focus on the active site minus the ligand.&lt;/li&gt;&#xA;&lt;li&gt;Is the ligand binding but leaving one substituent group to wobble?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h2&gt;Energetics&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;As with docking, the &quot;score&quot; is free energy difference (in kcal/mol). This is not the same as k_off and k_on and their ratio K_d (Dissociation constant). For those you need to do steered MD or have a way longer simulation (not recommended). The reason being is that the ligand has to make its way into the active site. However, you have a secondary binding mode in black, so have two free energy scores, this means that there is a stronger k_on.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2019-12-29T10:32:15.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11046" PostTypeId="1" CreationDate="2019-12-30T11:51:25.710" Score="1" ViewCount="96" Body="&lt;p&gt;I am working on clinical study where I am focusing on binary classification problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are few variables which has outliers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to know which would be the right value to replace outliers in a clinical study?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it &lt;code&gt;mean&lt;/code&gt;, &lt;code&gt;median&lt;/code&gt; &lt;code&gt;min&lt;/code&gt;,&lt;code&gt;max&lt;/code&gt; or &lt;code&gt;mode&lt;/code&gt; of a column of interest?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently I use &lt;code&gt;min&lt;/code&gt; and &lt;code&gt;max&lt;/code&gt; for values below and above 1.5*IQR threshold&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use &lt;code&gt;IQR&lt;/code&gt; approach to identify outliers but would like to know how it is done usually in clinical studies?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone help me with this? &lt;/p&gt;&#xA;" OwnerUserId="6380" LastActivityDate="2019-12-31T14:35:46.613" Title="Appropriate value to replace outliers in clinical studies" Tags="|public-databases|literature-search|data-management|data-preprocessing|public-dataset|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11047" PostTypeId="1" AcceptedAnswerId="11048" CreationDate="2019-12-30T16:00:19.727" Score="0" ViewCount="32" Body="&lt;p&gt;I have a dataframe like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dim(anno_maf)&#xA;[1] 32786   129&#xA;&amp;gt; head(anno_maf[1:2,1:4])&#xA;   Tumor_Sample_Barcode Chromosome Start_Position&#xA;1:                    a       chr1       27105930&#xA;2:                    a       chr1        3342771&#xA;   End_Position&#xA;1:     27105930&#xA;2:      3342771&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have a list of genes like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gene name&#xA;PTEN&#xA;PTEN&#xA;MAP3K1&#xA;MALAT1&#xA;MED12&#xA;RUNX1&#xA;PIK3CA&#xA;FBXW7&#xA;ARID1A&#xA;MED12&#xA;TAB1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to add a column in my dataframe named COSMIC, if a gene in gene list exists in my dataframe &lt;strong&gt;&lt;em&gt;Hugo_Symbol&lt;/em&gt;&lt;/strong&gt; column, put a &lt;strong&gt;yes&lt;/strong&gt; for this gene if not put a &lt;strong&gt;&lt;em&gt;no&lt;/em&gt;&lt;/strong&gt;; Finally my data frame has an extra column as &lt;strong&gt;&lt;em&gt;COSMIC&lt;/em&gt;&lt;/strong&gt; in which I have &lt;strong&gt;&lt;em&gt;yes&lt;/em&gt;&lt;/strong&gt; or &lt;strong&gt;&lt;em&gt;no&lt;/em&gt;&lt;/strong&gt; in &lt;strong&gt;&lt;em&gt;Hugo_Symbol&lt;/em&gt;&lt;/strong&gt; column based on the existence of genes in my gene list&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you help me please?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2019-12-30T18:04:13.253" Title="Adding new columns to this data frame" Tags="|r|looper|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11048" PostTypeId="2" ParentId="11047" CreationDate="2019-12-30T18:04:13.253" Score="2" Body="&lt;p&gt;You can use &lt;code&gt;ifelse()&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;anno_maf&lt;span class=&quot;math-container&quot;&gt;$COSMIC &amp;lt;- ifelse(anno_maf$&lt;/span&gt;Hugo_Symbol %in% your_gene_list,&#xA;                          yes = &quot;yes&quot;,&#xA;                          no = &quot;no&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-12-30T18:04:13.253" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11049" PostTypeId="1" AcceptedAnswerId="11050" CreationDate="2019-12-30T20:32:55.570" Score="1" ViewCount="42" Body="&lt;p&gt;I am working on cancer genomics failed to follow some terms even by googling &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference of &lt;strong&gt;&lt;em&gt;focal genome amplifications&lt;/em&gt;&lt;/strong&gt; versus &lt;strong&gt;&lt;em&gt;convergent amplification&lt;/em&gt;&lt;/strong&gt; of given region of genome for instance fibroblast growth factors region?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for any clarification &lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2019-12-30T22:04:19.187" Title="What does these terms mean?" Tags="|genomics|wgs|copy-number|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11050" PostTypeId="2" ParentId="11049" CreationDate="2019-12-30T22:04:19.187" Score="1" Body="&lt;p&gt;I think these definitions still have a degree of subjectivity and I don't think they are mutually exclusive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Convergent amplification basically means that &quot;similar&quot; cancers/neoplasms have &quot;similar&quot; amplification patterns, i.e. at around the same loci (see &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/16751803&quot; rel=&quot;nofollow noreferrer&quot;&gt;Myllykangas S et al.,2006&lt;/a&gt; for an example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Focal genome amplification is defined as &quot;CNA [copy number alteration] of limited size, frequently enriched for cancer driver genes&quot; (&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0167488914003000?via%3Dihub&quot; rel=&quot;nofollow noreferrer&quot;&gt;Krijgsmanthis et al., 2014&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-12-30T22:04:19.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11051" PostTypeId="1" CreationDate="2019-12-30T22:29:24.730" Score="0" ViewCount="60" Body="&lt;p&gt;I'm familiar with the sQTL and eQTL GTEx data that can be downloaded from the&#xA;&lt;a href=&quot;https://gtexportal.org/home/datasets&quot; rel=&quot;nofollow noreferrer&quot;&gt;GTEx Portal&lt;/a&gt;, but I'm interested in gathering at least two other types of molecular QTL datasets:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Larger eQTL and/or sQTL datasets (greater overall sample size, even if performed only for blood cells)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Other functional molecular QTLs such as methylation QTLs, histone QTLs, DNA accessibility QTLs, etc.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="177" LastActivityDate="2022-04-13T02:08:30.790" Title="What are the largest fully publicly available molecular QTL datasets?" Tags="|genomics|public-dataset|qtls|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11052" PostTypeId="1" AcceptedAnswerId="11056" CreationDate="2019-12-30T22:51:12.003" Score="1" ViewCount="168" Body="&lt;p&gt;I have a file in Stockholm format and I am trying to read it using the Biopython package. The code is as follows :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from Bio import AlignIO&#xA;&#xA;alignments = AlignIO.parse(&quot;Rfam.seed&quot;, &quot;stockholm&quot;)&#xA;for alignment in alignments:&#xA;    print(alignment)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It gets read for some alignments and then throws up following error :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &quot;test.py&quot;, line 5, in &amp;lt;module&amp;gt;&#xA;    for alignment in alignments:&#xA;  File &quot;/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/Bio/AlignIO/__init__.py&quot;, line 394, in parse&#xA;    for a in i:&#xA;  File &quot;/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/Bio/AlignIO/StockholmIO.py&quot;, line 408, in __next__&#xA;    line = handle.readline()&#xA;  File &quot;/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/codecs.py&quot;, line 322, in decode&#xA;    (result, consumed) = self._buffer_decode(data, self.errors, final)&#xA;UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe1 in position 2468: invalid continuation byte&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How to resolve this error?&lt;/p&gt;&#xA;" OwnerUserId="6445" LastActivityDate="2020-01-30T11:00:26.377" Title="Unable to read all alignments in stockholm format : UnicodeDecodeError: 'utf-8'" Tags="|python|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11054" PostTypeId="2" ParentId="11046" CreationDate="2019-12-31T03:36:12.570" Score="3" Body="&lt;p&gt;In my experience, it is more common to use all &lt;em&gt;clinical&lt;/em&gt; data as-is for clinical studies. And if data is missing, either omit the sample or omit the variable with missing data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your classifier can't handle the wide variation commonly seen in clinical studies then you may want to use a classifier which is less impacted by outliers.&lt;/p&gt;&#xA;" OwnerUserId="39" LastActivityDate="2019-12-31T03:36:12.570" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11056" PostTypeId="2" ParentId="11052" CreationDate="2019-12-31T10:44:27.967" Score="2" Body="&lt;p&gt;The last line indicates that this is an encoding problem. Try to see if the function you are using to read the data has an &quot;encoding&quot; parameter that you can set to whatever encoding your file is using. If that doesn't work, you can change the encoding of your file by:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;with &lt;code&gt;save as&lt;/code&gt; using a text editor of your choice and selecting &quot;UTF8&quot; as the encoding. That would be a quick work-around if you have a small number of files.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You can use the &lt;code&gt;codecs&lt;/code&gt; library to check and convert file encodings within your code. &lt;a href=&quot;https://stackoverflow.com/a/191403/7236472&quot;&gt;Here is an example&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Warning: I haven't used &lt;code&gt;Biopython&lt;/code&gt; in ages but encountered similar problems with Python in the past.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2019-12-31T10:44:27.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11057" PostTypeId="2" ParentId="11046" CreationDate="2019-12-31T14:35:46.613" Score="1" Body="&lt;p&gt;Removing outliers is common practice in statistical modeling and perfectly acceptable. However, with regards 1.5 IQR I am far from certain about this approach. Normally, if you want to be conservative then 3 standard deviations (SD) denote an outlier, which is more stringent than IQR. Some use 2 SD. If the value is lower than 2 SD from the group mean it isn't an outlier, that's because 1.96 SD is the normal distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Normally you would delete the outlier and it takes no further part in the calculation, assigning it to 'na' should work. The subsequent calculation will recognise an 'na' value, I think in python it is np('na') ... Np = numpy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this case you appear to be deleting the value and treating it as missing data. Treatment of missing data is complicated I agree 100%. Mean, median are possibilities for missing data I can't see this working for binary data. One approach is regression and in python there is a command specifically to do this (which I forget), R will have the same command and it will assign a value  for the missing data. However, you can't do this for outliers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The other routes to remove outliers are via transformation, log transformation is the most common. Transformation is a big subject.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2019-12-31T14:35:46.613" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11058" PostTypeId="2" ParentId="11043" CreationDate="2019-12-31T14:48:21.540" Score="3" Body="&lt;p&gt;This is non-parametric statistics, the mean requires conformation to the normal distribution (but there are exceptions), or perhaps better put it requires a fixed relationship between mean and standard deviation. Non-parametric stats always use the median and are distribution free. Wilcoxon test/ Mann-Whitney U test  are used to replace t-test. The Kruskal-Wallis test replaces the 1-way ANOVA&#xA;Finally the Friedman test replaces 2-way ANOVA but has its critiques.  The t- and ANOVA tests require normality, F-max is one approach to determine normality.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="650" LastEditDate="2024-02-09T15:31:29.483" LastActivityDate="2024-02-09T15:31:29.483" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11059" PostTypeId="2" ParentId="11020" CreationDate="2019-12-31T20:10:29.813" Score="0" Body="&lt;p&gt;Ok I'll be brief, given the ms is likely under review.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; in silico drug discovery has been part of deep learning for a long time 2015(?), and tensorflow was deployed. They worked out how to vectorise a small molecule and thereafter the entire field was.wide open to advanced ML.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Ratio&lt;/strong&gt; The ratio is an accuracy index of the machine learning algorithms. The denominator is the total number of biological observations, the numerator is the predicted number. This 90% would be an accuracy index, i.e. the algorithm is correct 9 out of 10 times.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Result&lt;/strong&gt; The plot shows, as the data set size increases the accuracy of the prediction increases most notably for the DNN and that conforms to all the theory. DNN/ANN only become powerful for 'big data', standard ML, i.e. random forest, naive Bayes, LR work well on small data sets, are fast and produce a classification which isn't wrapped up in a 3-4 Gig dump, thereafter performance saturates for routine ML methods.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The multi task DNN (MT-DNN) means that more than one loss function is being used, which is unusual in generic deep learning, but has been applied to natural language and drug discovery previously. Why this is being deployed is a long explanation, so I'll leave it there.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2019-12-31T20:16:13.920" LastActivityDate="2019-12-31T20:16:13.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11060" PostTypeId="1" AcceptedAnswerId="11062" CreationDate="2019-12-31T20:12:24.047" Score="1" ViewCount="75" Body="&lt;p&gt;I have a big data frame like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dim(data1)&#xA;[1] 474303    128&#xA;&amp;gt; head(data1[1:2,1:6])&#xA;   Tumor_Sample_Barcode Chromosome Position&#xA;1:                   A1       chr1       15772155&#xA;2:                   A1       chr1       16054813&#xA;   End_Position Reference_Allele Tumor_Seq_Allele2&#xA;1:     15772155                G                 A&#xA;2:     16054813                G                 A&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have another dataframe like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Chrom   Position    Ref Alt VAF&#xA;chr1    2716062 C   A   0.323&#xA;chr1    3095112 T   G   0.365&#xA;chr1    3342771 C   G   0.123&#xA;chr1    3526376 C   T   0.365&#xA;chr1    4121415 T   G   0.225&#xA;chr1    4215706 G   A   0.125&#xA;chr1    4304014 T   G   0.286&#xA;chr1    4304016 A   C   0.302&#xA;chr1    4388809 A   C   0.135&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to create another column named &lt;strong&gt;&lt;em&gt;i_TumorVAF_WU&lt;/em&gt;&lt;/strong&gt; in my big data frame in which VAF column comes from small data frame. Please notice that VAF values should be matched with Position in big data because small data coming from big data&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you help please?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    &amp;gt; unique(merge(data1,data,x.by=data1&lt;span class=&quot;math-container&quot;&gt;$Position,y.by=data$&lt;/span&gt;Position))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Returns a weird output with losing a big part of big data frame. I just want to add corresponding VAF values to the big data by keeping all the information in big data file&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2019-12-31T22:27:53.583" LastActivityDate="2020-01-08T10:33:08.700" Title="Merging matched parts of two dataframes" Tags="|r|wgs|maf|snv|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11061" PostTypeId="1" AcceptedAnswerId="11067" CreationDate="2019-12-31T20:38:53.733" Score="0" ViewCount="197" Body="&lt;p&gt;I have a bash script which extracts some information from a .vcf file but one .vcf file at each time. How I can change this script in a way to work on a bunch of .vcf files and the output is a .txt file named by the name of original .vcf&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my script &lt;/p&gt;&#xA;&#xA;&lt;p&gt;help please&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;EDITED&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; #!/usr/bash&#xA;&#xA;#outfilename&#xA;outname=$&quot;.parsed.txt&quot;&#xA;&#xA;#Header for output file&#xA;echo -e &quot;&lt;span class=&quot;math-container&quot;&gt;$outname\t$&lt;/span&gt;chrom\t&lt;span class=&quot;math-container&quot;&gt;$Pos\t$&lt;/span&gt;Ref\t&lt;span class=&quot;math-container&quot;&gt;$Alt\t$&lt;/span&gt;ReadCount\t&lt;span class=&quot;math-container&quot;&gt;$VariantAlleleCount\t$&lt;/span&gt;ReferenceAlleleCount\t&lt;span class=&quot;math-container&quot;&gt;$ReadCountControl\t$&lt;/span&gt;VariantAlleleCountControl\t&lt;span class=&quot;math-container&quot;&gt;$ReferenceAlleleCountControl\t$&lt;/span&gt;VAF&quot; &amp;gt;&amp;gt; $outname ;&#xA;while read -r line ; &#xA;do&#xA;&#xA;#Basic information&#xA;chrom=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;$&lt;/span&gt;line&quot; | sed 's/ /\t/g' | cut -f 1) #&amp;amp;&amp;amp; echo &lt;span class=&quot;math-container&quot;&gt;$chrom;&#xA;Pos=$&lt;/span&gt;(echo &quot;&lt;span class=&quot;math-container&quot;&gt;$line&quot; | sed 's/ /\t/g' |  cut -f 2) #&amp;amp;&amp;amp; echo $&lt;/span&gt;Pos;&#xA;Ref=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;$&lt;/span&gt;line&quot; | sed 's/ /\t/g' | cut -f 4)&#xA;Alt=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;$&lt;/span&gt;line&quot; | sed 's/ /\t/g' | cut -f 5)&#xA;&#xA;#Tumor sample read, variant and reference information&#xA;ReadCount=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;$&lt;/span&gt;line&quot; | cut -f 8 | sed 's/;/\t/g' | cut -f 13 | sed 's/ReadCount=//' )&#xA;VariantAlleleCount=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;$&lt;/span&gt;line&quot; | cut -f 8 | sed 's/;/\t/g' | cut -f 26| sed 's/VariantAlleleCount=//')&#xA;ReferenceAlleleCount=&lt;span class=&quot;math-container&quot;&gt;$(echo $&lt;/span&gt; line | awk -v rc=&quot;&lt;span class=&quot;math-container&quot;&gt;$ReadCount&quot; -v vac=&quot;$&lt;/span&gt;VariantAlleleCount&quot; '{print rc-vac}')&#xA;&#xA;&#xA;#Control or Normal read, variant, reference information&#xA;ReadCountControl=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;$&lt;/span&gt;line&quot; | cut -f 8 | sed 's/;/\t/g' | cut -f 14 | sed 's/ReadCountControl=//')&#xA;VariantAlleleCountControl=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;$&lt;/span&gt;line&quot; | cut -f 8 | sed 's/;/\t/g' | cut -f 27 | sed 's/VariantAlleleCountControl=//')   &#xA;&#xA;ReferenceAlleleCountControl=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;$&lt;/span&gt;line&quot; | awk -v rcc=&quot;&lt;span class=&quot;math-container&quot;&gt;$ReadCountControl&quot; -v vacc=&quot;$&lt;/span&gt;VariantAlleleCountControl&quot; '{print rcc-vacc}')&#xA;&#xA;&#xA;VAF=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;$&lt;/span&gt;line&quot; | cut -f 8 | sed 's/;/\t/g' | cut -f 28 | sed 's/VariantAlleleFrequency=//')&#xA;#Print output&#xA;echo -e &quot;&lt;span class=&quot;math-container&quot;&gt;$outname\t&quot;&quot;$&lt;/span&gt;chrom&quot;'\t'&quot;&lt;span class=&quot;math-container&quot;&gt;$Pos&quot;'\t'&quot;$&lt;/span&gt;Ref&quot;'\t'&quot;&lt;span class=&quot;math-container&quot;&gt;$Alt&quot;'\t'&quot;$&lt;/span&gt;ReadCount&quot;'\t'&quot;&lt;span class=&quot;math-container&quot;&gt;$VariantAlleleCount&quot;'\t'&quot;$&lt;/span&gt;ReferenceAlleleCount&quot;'\t'&quot;&lt;span class=&quot;math-container&quot;&gt;$ReadCountControl&quot;'\t'&quot;$&lt;/span&gt;VariantAlleleCountControl&quot;'\t'&quot;&lt;span class=&quot;math-container&quot;&gt;$ReferenceAlleleCountControl&quot;'\t'&quot;$&lt;/span&gt;VAF&quot; &amp;gt;&amp;gt; $outname ;&#xA;&#xA;#Remove info tags from VCF&#xA;done; &amp;lt; &amp;lt;( egrep -v '#' &quot;$1&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;And the error I obtained&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[fi1d18@cyan01 new]&lt;span class=&quot;math-container&quot;&gt;$ bash vcf_parasing.sh&#xA;vcf_parasing.sh: line 7: syntax error near unexpected token `echo'&#xA;vcf_parasing.sh: line 7: `echo -e $&lt;/span&gt;outname'\t' &quot;Chrom&quot;'\t'&quot;Position&quot;'\t'&quot;Ref&quot;'\t'&quot;Alt&quot;'\t'&quot;TumorReadCount&quot;'\t'&quot;TumorVariantAlleleCount&quot;'\t'&quot;TumorReferenceAlleleCount&quot;'\t'&quot;NormalReadCount&quot;'\t'&quot;NormalVariantAlleleCount&quot;'\t'&quot;NormalReferenceAlleleCount&quot;'\t'&quot;VAF&quot; &amp;gt; &lt;span class=&quot;math-container&quot;&gt;$outname'&#xA;[fi1d18@cyan01 new]$&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My script&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/di49mij2a43i43s/vcf_parasing.sh?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/di49mij2a43i43s/vcf_parasing.sh?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two samples of .vcf&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/ryk7q0aybeo5cy2/LP6008202-DNA_B03_vs_LP6008201-DNA_B03.snp.pass.vcf?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/ryk7q0aybeo5cy2/LP6008202-DNA_B03_vs_LP6008201-DNA_B03.snp.pass.vcf?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/u44s6d54r5d57bu/LP6008336-DNA_H02_vs_LP6008333-DNA_H02.snp.pass.vcf?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/u44s6d54r5d57bu/LP6008336-DNA_H02_vs_LP6008333-DNA_H02.snp.pass.vcf?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-01-01T15:54:23.640" LastActivityDate="2020-01-01T17:03:10.967" Title="Changing this code in a way to work for my files" Tags="|vcf|bash|linux|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11062" PostTypeId="2" ParentId="11060" CreationDate="2020-01-01T02:05:33.300" Score="3" Body="&lt;p&gt;If you have issue with memory and dealing with large object, maybe &lt;code&gt;data.table&lt;/code&gt; is the way to go (&lt;a href=&quot;https://github.com/Rdatatable/data.table/wiki&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Rdatatable/data.table/wiki&lt;/a&gt;): &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's take two dummy data.frame:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;dfA &amp;lt;- data.frame(Chrom = rep(&quot;chr1&quot;,150),&#xA;                  Pos = 1:150,&#xA;                  ref = rep(&quot;A&quot;,150))&#xA;dfB &amp;lt;- data.frame(Chrom = rep(&quot;chr1&quot;,10),&#xA;                  Pos = 11:20,&#xA;                  Vaf = rep(&quot;VAF&quot;, 10))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The big data.frame &lt;code&gt;dfA&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; head(dfA)&#xA;  Chrom Pos ref&#xA;1  chr1   1   A&#xA;2  chr1   2   A&#xA;3  chr1   3   A&#xA;4  chr1   4   A&#xA;5  chr1   5   A&#xA;6  chr1   6   A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the small dataset:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; dfB&#xA;   Chrom Pos Vaf&#xA;1   chr1  11 VAF&#xA;2   chr1  12 VAF&#xA;3   chr1  13 VAF&#xA;4   chr1  14 VAF&#xA;5   chr1  15 VAF&#xA;6   chr1  16 VAF&#xA;7   chr1  17 VAF&#xA;8   chr1  18 VAF&#xA;9   chr1  19 VAF&#xA;10  chr1  20 VAF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, using &lt;code&gt;data.table&lt;/code&gt; that is more efficient and less memory consuming for the manipulation of dataframe, you can do:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(data.table)&#xA;setDT(dfA)&#xA;setDT(dfB)&#xA;dfA[dfB, on = c(&quot;Chrom&quot;,&quot;Pos&quot;), vaf := i.Vaf]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And now, your data.frame &lt;code&gt;dfA&lt;/code&gt; looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; dfA[5:25]&#xA;    Chrom Pos ref  vaf&#xA; 1:  chr1   5   A &amp;lt;NA&amp;gt;&#xA; 2:  chr1   6   A &amp;lt;NA&amp;gt;&#xA; 3:  chr1   7   A &amp;lt;NA&amp;gt;&#xA; 4:  chr1   8   A &amp;lt;NA&amp;gt;&#xA; 5:  chr1   9   A &amp;lt;NA&amp;gt;&#xA; 6:  chr1  10   A &amp;lt;NA&amp;gt;&#xA; 7:  chr1  11   A  VAF&#xA; 8:  chr1  12   A  VAF&#xA; 9:  chr1  13   A  VAF&#xA;10:  chr1  14   A  VAF&#xA;11:  chr1  15   A  VAF&#xA;12:  chr1  16   A  VAF&#xA;13:  chr1  17   A  VAF&#xA;14:  chr1  18   A  VAF&#xA;15:  chr1  19   A  VAF&#xA;16:  chr1  20   A  VAF&#xA;17:  chr1  21   A &amp;lt;NA&amp;gt;&#xA;18:  chr1  22   A &amp;lt;NA&amp;gt;&#xA;19:  chr1  23   A &amp;lt;NA&amp;gt;&#xA;20:  chr1  24   A &amp;lt;NA&amp;gt;&#xA;21:  chr1  25   A &amp;lt;NA&amp;gt;&#xA;    Chrom Pos ref  vaf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you see, we incorporate the column &lt;code&gt;Vaf&lt;/code&gt; from the small dataframe into the big dataframe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With your example, I will first rename the colnames of your small dataframe in order to have the perfect match with the big dataframe:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;colnames(data) &amp;lt;- gsub(&quot;Chrom&quot;, &quot;Chromosome&quot;, colnames(data))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then,&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;setDT(data1)&#xA;setDT(data)&#xA;data1[data, on = c(&quot;Chromosome&quot;,&quot;Position&quot;), i_TumorVAF_WU := i.VAF]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does it look what you are trying to obtain ?&lt;/p&gt;&#xA;" OwnerUserId="6437" LastEditorUserId="6437" LastEditDate="2020-01-01T02:11:24.293" LastActivityDate="2020-01-01T02:11:24.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11064" PostTypeId="2" ParentId="11061" CreationDate="2020-01-01T11:06:24.120" Score="1" Body="&lt;p&gt;Here is a quick and kind of dirty solution with minimum intervention to the code:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Add &lt;code&gt;$outname'\t'&lt;/code&gt; right after &lt;code&gt;echo -e&lt;/code&gt; to two two lines, the lines after &lt;code&gt;#Header for output file&lt;/code&gt; and &lt;code&gt;#Print output&lt;/code&gt;. The former will alter your header and the latter your actual output: you will have a first column with file name information.&lt;/li&gt;&#xA;&lt;li&gt;Then you can run this script in a for loop to generate multiple output files which then can be merged into a single file with &lt;code&gt;cat&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Note that with this workaround, you will have multiple headers in the merged file. If you need a clean output, you will need to put your while loop in a for loop.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-01-01T11:06:24.120" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11067" PostTypeId="2" ParentId="11061" CreationDate="2020-01-01T17:03:10.967" Score="5" Body="&lt;p&gt;Using shell loops for text practice &lt;a href=&quot;https://unix.stackexchange.com/q/169716/22222&quot;&gt;is considered bad practice&lt;/a&gt;. It is exceedingly slow, the syntax is complicated so it's very easy to get it wrong and it's just painful. The shell isn't designed as a proper scripting language, so while it can be (ab)used that way, you really should avoid it whenever possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a Perl script that should do exactly what your bash script does but will be several times faster and can deal with multiple input files:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env perl&#xA;&#xA;## Iterate over all files given at the command line&#xA;foreach my &lt;span class=&quot;math-container&quot;&gt;$inFile (@ARGV) {&#xA;  print STDERR &quot;Processing $inFile...\n&quot;;&#xA;  my $outFile = &quot;$inFile.parsed.txt&quot;;&#xA;  $outFile =~ m|([^/]+)$|;&#xA;  #/#Useless comment to fix SE syntax highlighting&#xA;  my $outFileName = $1;&#xA;  open(my $outFileHandle, '&amp;gt;', $outFile) or&#xA;      die(&quot;Couldn't open '$outFile' for writing: $!&quot;);&#xA;  open(my $inFileHandle, '&amp;lt;', $inFile) or&#xA;      die(&quot;Couldn't open '$inFile' for writing: $&lt;/span&gt;!&quot;);&#xA;&#xA;  ## Print header&#xA;  print &lt;span class=&quot;math-container&quot;&gt;$outFileHandle &quot;$&lt;/span&gt;outFileName\tChrom\tPosition\tRef\tAlt\tTumorReadCount\t&quot; .&#xA;      &quot;TumorVariantAlleleCount\tTumorReferenceAlleleCount\tNormalReadCount&quot; .&#xA;          &quot;\tNormalVariantAlleleCount\tNormalReferenceAlleleCount\tVAF\n&quot;;&#xA;  ## Read all lines of the input file&#xA;  while (&amp;lt;&lt;span class=&quot;math-container&quot;&gt;$inFileHandle&amp;gt;) {&#xA;    ## Skip headers&#xA;    next if /^#/;&#xA;    ## Get VCF fields&#xA;    my ($chrom, $pos, $name, $ref, $alt, $qual,&#xA;        $filter, $info, $format, $values) = split(/\t/);&#xA;    my @infoFields = split(/;/, $info);&#xA;    my $readCount = $infoFields[12];&#xA;    $&lt;/span&gt;readCount =~ s/ReadCount=//;&#xA;    #/#Useless comment to fix SE syntax highlighting&#xA;&#xA;    my &lt;span class=&quot;math-container&quot;&gt;$variantAlleleCount = $&lt;/span&gt;infoFields[25];&#xA;    &lt;span class=&quot;math-container&quot;&gt;$variantAlleleCount =~ s/VariantAlleleCount=//;&#xA;    #/#Useless comment to fix SE syntax highlighting&#xA;    my $&lt;/span&gt;referenceAlleleCount= &lt;span class=&quot;math-container&quot;&gt;$readCount - $&lt;/span&gt;variantAlleleCount;&#xA;&#xA;    my &lt;span class=&quot;math-container&quot;&gt;$readCountControl = $&lt;/span&gt;infoFields[13];&#xA;    $readCountControl =~ s/ReadCountControl=//;&#xA;    #/#Useless comment to fix SE syntax highlighting&#xA;&#xA;    my &lt;span class=&quot;math-container&quot;&gt;$variantAlleleCountControl = $&lt;/span&gt;infoFields[26];&#xA;    $variantAlleleCountControl =~ s/VariantAlleleCountControl=//;&#xA;    #/#Useless comment to fix SE syntax highlighting&#xA;&#xA;    my &lt;span class=&quot;math-container&quot;&gt;$referenceAlleleCountControl = $&lt;/span&gt;readCountControl - &lt;span class=&quot;math-container&quot;&gt;$variantAlleleCountControl;&#xA;    my $&lt;/span&gt;VAF = &lt;span class=&quot;math-container&quot;&gt;$infoFields[27];&#xA;    $&lt;/span&gt;VAF =~ s/VariantAlleleFrequency=//;&#xA;    #/#Useless comment to fix SE syntax highlighting&#xA;&#xA;    my &lt;span class=&quot;math-container&quot;&gt;$outString = join(&quot;\t&quot;, $&lt;/span&gt;outFileName, &lt;span class=&quot;math-container&quot;&gt;$chrom, $&lt;/span&gt;pos, &lt;span class=&quot;math-container&quot;&gt;$ref, $&lt;/span&gt;alt, &lt;span class=&quot;math-container&quot;&gt;$readCount,&#xA;                         $&lt;/span&gt;variantAlleleCount, &lt;span class=&quot;math-container&quot;&gt;$referenceAlleleCount, $&lt;/span&gt;readCountControl,&#xA;                         &lt;span class=&quot;math-container&quot;&gt;$variantAlleleCountControl, $&lt;/span&gt;referenceAlleleCountControl, &lt;span class=&quot;math-container&quot;&gt;$VAF);&#xA;    print $&lt;/span&gt;outFileHandle &quot;&lt;span class=&quot;math-container&quot;&gt;$outString\n&quot;;&#xA;  }&#xA;  close($&lt;/span&gt;outFileHandle);&#xA;  close($inFileHandle);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Save that file as &lt;code&gt;parser.pl&lt;/code&gt; and then run it on all your vcf files with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl parser.pl /path/to/vcfs/*vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That should create a &lt;code&gt;.parsed.txt&lt;/code&gt; file for each input file in the target directory. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If you just have to use the shell for this, you can try with this improved version of your script:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env bash&#xA;&#xA;for file in &quot;&lt;span class=&quot;math-container&quot;&gt;$@&quot;; do&#xA;  outFile=&quot;$&lt;/span&gt;{file}.parsed.txt&quot;&#xA;  outFileName=&lt;span class=&quot;math-container&quot;&gt;$(basename &quot;$&lt;/span&gt;outFile&quot;)&#xA;  (&#xA;    printf '%s\tChrom\tPosition\tRef\tAlt\tTumorReadCount\t' &quot;&lt;span class=&quot;math-container&quot;&gt;$outFileName&quot;&#xA;    printf 'TumorVariantAlleleCount\tTumorReferenceAlleleCount\tNormalReadCount\t'&#xA;    printf 'NormalVariantAlleleCount\tNormalReferenceAlleleCount\tVAF\n'&#xA;  ) &amp;gt; &quot;$&lt;/span&gt;outFile&quot;&#xA;&#xA;  echo &quot;Reading &lt;span class=&quot;math-container&quot;&gt;$file&quot;&#xA;  grep -v '^#' &quot;$&lt;/span&gt;file&quot; |&#xA;    while read -r line; do&#xA;      read chrom Pos Ref Alt rest &amp;lt; &amp;lt;(awk -F&quot;\t&quot; '{print &lt;span class=&quot;math-container&quot;&gt;$1,$&lt;/span&gt;2,&lt;span class=&quot;math-container&quot;&gt;$4,$&lt;/span&gt;5}' &amp;lt;&amp;lt;&amp;lt;&quot;&lt;span class=&quot;math-container&quot;&gt;$line&quot;)&#xA;      ReadCount=$&lt;/span&gt;(echo &quot;&lt;span class=&quot;math-container&quot;&gt;$line&quot; | cut -f 8 | sed 's/;/\t/g' | cut -f 13 | sed 's/ReadCount=//' )&#xA;      VariantAlleleCount=$&lt;/span&gt;(echo &quot;&lt;span class=&quot;math-container&quot;&gt;$line&quot; | cut -f 8 | sed 's/;/\t/g' | cut -f 26| sed 's/VariantAlleleCount=//')&#xA;      ReferenceAlleleCount=$&lt;/span&gt;((&lt;span class=&quot;math-container&quot;&gt;$ReadCount - $&lt;/span&gt;VariantAlleleCount))&#xA;&#xA;      ReadCountControl=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;$&lt;/span&gt;line&quot; | cut -f 8 | sed 's/;/\t/g' | cut -f 14 | sed 's/ReadCountControl=//')&#xA;      VariantAlleleCountControl=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;$&lt;/span&gt;line&quot; | cut -f 8 | sed 's/;/\t/g' | cut -f 27 | sed 's/VariantAlleleCountControl=//')   &#xA;&#xA;      ReferenceAlleleCountControl=&lt;span class=&quot;math-container&quot;&gt;$(($&lt;/span&gt;ReadCountControl - $VariantAlleleCountControl))&#xA;&#xA;      VAF=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;$&lt;/span&gt;line&quot; | cut -f 8 | sed 's/;/\t/g' | cut -f 28 | sed 's/VariantAlleleFrequency=//')&#xA;      printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' &quot;&lt;span class=&quot;math-container&quot;&gt;$outFileName&quot; &quot;$&lt;/span&gt;chrom&quot; &quot;&lt;span class=&quot;math-container&quot;&gt;$Pos&quot; &quot;$&lt;/span&gt;Ref&quot; &quot;&lt;span class=&quot;math-container&quot;&gt;$Alt&quot; \&#xA;             &quot;$&lt;/span&gt;ReadCount&quot; &quot;&lt;span class=&quot;math-container&quot;&gt;$VariantAlleleCount&quot; &quot;$&lt;/span&gt;ReferenceAlleleCount&quot; &quot;&lt;span class=&quot;math-container&quot;&gt;$ReadCountControl&quot; &quot;$&lt;/span&gt;VariantAlleleCountControl&quot; \&#xA;             &quot;&lt;span class=&quot;math-container&quot;&gt;$ReferenceAlleleCountControl&quot; &quot;$&lt;/span&gt;VAF&quot; &amp;gt;&amp;gt; &quot;$outFile&quot; &#xA;    done&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Just to illustrate what I mean about the shell being slow, I ran both scripts above on the example files you provided:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ time foo.pl *vcf 2&amp;gt;/dev/null&#xA;&#xA;real    0m0.051s&#xA;user    0m0.047s&#xA;sys     0m0.003s&#xA;&#xA;$ time bar.sh *vcf 2&amp;gt;/dev/null&#xA;&#xA;real    2m28.842s&#xA;user    3m33.648s&#xA;sys     1m12.456s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see above, the perl script took less than a second, while the bash script took almost two and a half minutes!&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Finally, if you find the perl script too long, here's the same thing in a more condensed style:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env perl&#xA;&#xA;## Iterate over all files given at the command line&#xA;foreach my &lt;span class=&quot;math-container&quot;&gt;$inFile (@ARGV) {&#xA;  print STDERR &quot;Processing $inFile...\n&quot;;&#xA;  my $outFile = &quot;$inFile.parsed.txt&quot;;&#xA;  $outFile =~ m|([^/]+)$|;&#xA;  #/#Useless comment to fix SE syntax highlighting&#xA;  my $outFileName = $1;&#xA;  open(my $outFileHandle, '&amp;gt;', $outFile);&#xA;  open(my $inFileHandle, '&amp;lt;', $&lt;/span&gt;inFile);&#xA;&#xA;  ## Print header&#xA;  print &lt;span class=&quot;math-container&quot;&gt;$outFileHandle &quot;$&lt;/span&gt;outFileName\tChrom\tPosition\tRef\tAlt\tTumorReadCount\t&quot; .&#xA;      &quot;TumorVariantAlleleCount\tTumorReferenceAlleleCount\tNormalReadCount&quot; .&#xA;          &quot;\tNormalVariantAlleleCount\tNormalReferenceAlleleCount\tVAF\n&quot;;&#xA;  ## Read all lines of the input file&#xA;  while (&amp;lt;&lt;span class=&quot;math-container&quot;&gt;$inFileHandle&amp;gt;) {&#xA;    ## Skip headers&#xA;    next if /^#/;&#xA;    ## Get VCF fields&#xA;    my ($chrom, $pos, $name, $ref, $alt, $qual,&#xA;        $filter, $info, $format, $values) = split(/\t/);&#xA;    $info=~s/[^;]+=//g;&#xA;    my @infoFields = split(/;/, $info);&#xA;    print $outFileHandle (join &quot;\t&quot;, $outFileName, $chrom, $pos, $ref, $alt,&#xA;                               $infoFields[12],$infoFields[25],&#xA;                               $infoFields[12]-$infoFields[25],&#xA;                               $infoFields[13],$infoFields[26],&#xA;                               $infoFields[13]-$infoFields[26],&#xA;                               $&lt;/span&gt;infoFields[27]) . &quot;\n&quot;;&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2020-01-01T17:03:10.967" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11068" PostTypeId="1" CreationDate="2020-01-01T17:55:04.727" Score="2" ViewCount="523" Body="&lt;p&gt;I am using IGV 2.5.2 and would not like to see the insertions  in my aligned reads. How can I do that? I have managed to remove mismatched bases because there is an option to do so when I right-click on the window (&quot;Show mismatched bases&quot;). Now I would like to do the same with these purple insertions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hxliq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hxliq.png&quot; alt=&quot;IGV screenshot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;3rd January 2020 update&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/avy5K.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/avy5K.png&quot; alt=&quot;IGV preferences&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks to a member's response, I found some parameter in the preferences that might be helpful to hide the indels. However, I have tried different numbers from 0 to 1000 and the indels are still shown. What numbers should I put to the parameters &quot;Show indel threshold (bases)&quot; and should I click the box or not?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="1075" LastEditDate="2020-01-03T18:04:22.677" LastActivityDate="2020-01-03T18:04:22.677" Title="How can I not show insertions in the Integrative Genome Viewer (IGV)?" Tags="|sequence-alignment|visualization|igv|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11070" PostTypeId="2" ParentId="11068" CreationDate="2020-01-02T22:10:34.813" Score="3" Body="&lt;p&gt;Preferences &lt;code&gt;-&amp;gt;&lt;/code&gt; Alignments &lt;code&gt;-&amp;gt;&lt;/code&gt; Hide indels &amp;lt; 'x' bases.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Credit to &lt;a href=&quot;https://www.biostars.org/p/339007/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pierre Lindenbaum&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2020-01-02T22:10:34.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11071" PostTypeId="2" ParentId="3500" CreationDate="2020-01-03T22:39:24.113" Score="1" Body="&lt;p&gt;Use this line of code:    &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print(analysed_seq.protein_scale(analysed_seq.get_amino_acids_percent(), window=9, edge=0.4))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6714" LastActivityDate="2020-01-03T22:39:24.113" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11072" PostTypeId="1" CreationDate="2020-01-04T00:20:18.280" Score="5" ViewCount="37" Body="&lt;p&gt;I found that there are a couple of projects that allow the user to simulate the neural system of &lt;em&gt;C elegans&lt;/em&gt;, for example OpenWorm one update being &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6158223/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I don't know how to navigate sources in bioinformatics and thus struggle to evaluate their quality.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; Are there any fundamental / most widely accepted data sources for raw data of C Elegans connectome or electron microscope scans of this organism, similar to Human Genome Project? If not, what are the ways to derive the data?&lt;/p&gt;&#xA;" OwnerUserId="6715" LastEditorUserId="3848" LastEditDate="2022-02-21T23:34:38.280" LastActivityDate="2022-07-22T14:01:25.117" Title="Are there any &quot;canonical&quot; sources of C. elegans connectome?" Tags="|public-databases|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11074" PostTypeId="1" AcceptedAnswerId="11085" CreationDate="2020-01-04T16:39:12.990" Score="0" ViewCount="211" Body="&lt;p&gt;I have a bash script:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am wondering how I can change this script to loop over a bunch of .vcf files and give output .txt with the name of corresponding .vcf &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried changes done in similar script in  &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/11061/changing-this-code-in-a-way-to-work-for-my-files/11067?noredirect=1#comment15031_11067&quot;&gt;Changing this code in a way to work for my files&lt;/a&gt; but I just  messed everything up&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-01-07T12:10:26.760" LastActivityDate="2020-01-07T15:28:39.057" Title="Looping over several files in bash" Tags="|vcf|bash|linux|wgs|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11075" PostTypeId="1" AcceptedAnswerId="11099" CreationDate="2020-01-04T22:07:38.867" Score="2" ViewCount="94" Body="&lt;h2&gt;&lt;a href=&quot;https://i.stack.imgur.com/cnOA0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cnOA0.png&quot; alt=&quot;Subchains view.&quot;&gt;&lt;/a&gt;&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I'm trying to programmatically construct a name for each protein subchain in any ribosome from Uniprot in accordance with Ban et. al's 2014 proposal (excerpt given below) using PDB's and Uniprot's APIs.&#xA;This is to be a part of a bigger ribosomal analysis suite released next year and we would like to standardize the names as much as possible.&#xA;Ban et. al's proposal: [&lt;a href=&quot;https://bangroup.ethz.ch/research/nomenclature-of-ribosomal-proteins.html]&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bangroup.ethz.ch/research/nomenclature-of-ribosomal-proteins.html]&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I'm however not remotely a biologist (I work in compilers) and &lt;strong&gt;&lt;em&gt;would really appreciate some clarification on the way nomenclature works across domains of life and how homology plays into it.&lt;/em&gt;&lt;/strong&gt; &#xA;This is my &lt;em&gt;naive&lt;/em&gt; approach thus far: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I get a ribosome from PDB let's say (&lt;em&gt;3J9M&lt;/em&gt; or &lt;em&gt;5MYJ&lt;/em&gt; or any other for that matter) and split it into protein subchains. So on the order of 20-80 proteins.&lt;/li&gt;&#xA;&lt;li&gt;For each protein subchain, I can get all the information in the world on it from Uniprot: names, sequence, organisms containing it, publications, synonymous names, you name it...&lt;/li&gt;&#xA;&lt;li&gt;Right now (again, naively) for those proteins that &lt;em&gt;don't already have a new name assigned to them&lt;/em&gt;, I take name-synonyms that PDB has plenty of (from various publications, I assume) and scan them with regexes (ex.&lt;code&gt;/[LS]\d{1,2}/g&lt;/code&gt;) for things like &lt;code&gt;&quot;L8&quot;&lt;/code&gt; or &lt;code&gt;&quot;S15&quot;&lt;/code&gt;. &lt;/li&gt;&#xA;&lt;li&gt;Thus, for each subchain, I end up with something like &lt;code&gt;&quot;L15&quot;&lt;/code&gt; or &lt;code&gt;&quot;S28&quot;&lt;/code&gt; in multiple copies (depending on the abundance of synonymous entries in PDB).&lt;/li&gt;&#xA;&lt;li&gt;The rut i'm in right now is this: having something like &lt;code&gt;&quot;L15&quot;&lt;/code&gt; to match it to a new &quot;Ban-nomenclature&quot; name: they provide the nomenclature look-up tables in terms of these three &quot;historical&quot; naming conventions(i.e &quot;human&quot;, &quot;yeast&quot;, &quot;bacteria&quot;) and a taxonomic range while all the I have is the ribosomal structure(ex `` from which I got the subchain. I'm sure there's plenty of information that I can get &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Could somebody elucidate to me which assumptions I can and cannot make in this field and how exactly to use the &lt;em&gt;taxonomic range&lt;/em&gt;? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say, (i) can I assume that all archean names are the same as bacterial homologs?&#xA;If I draw a eukaryotic cytoplasmic ribosome, do I match its subchains against yeast column or human column (since both are eukaryotic)?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Somebody suggested that I start grouping these subchains based on sequence, not on a domain and I do have access to each protein's sequence, but I am not sure how to go about it. I was thinking to get each of Ban's new names' &lt;strong&gt;UniRef90&lt;/strong&gt; cluster, let's say, and then start checking which cluster a given subchain belongs to. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I'm obviously guessing at this point. Any advice or pointers are very appreciated. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt; &lt;a href=&quot;https://i.stack.imgur.com/TlcEE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TlcEE.png&quot; alt=&quot;Excerpt from Ban&quot;&gt;&lt;/a&gt;&lt;/h2&gt;&#xA;" OwnerUserId="6511" LastEditorUserId="6511" LastEditDate="2020-01-06T06:09:29.777" LastActivityDate="2020-01-07T21:44:58.037" Title="Infer the new (Ban et. al) ribosomal nomenlature (ex. uL53 ) from the ribosomal protein's sequence" Tags="|python|uniprot|ribosomal|nomenclature|sequence|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11077" PostTypeId="1" CreationDate="2020-01-05T08:58:12.143" Score="4" ViewCount="983" Body="&lt;p&gt;we performed a combined gene expression and CiteSeq experiment with the 10x VDJ kit and 20 conjugated antibodies and sequenced on hiseq. I used cellranger to process the sequencing output. The cellranger summary shows overall good values except for a low percentage of &quot;Antibody Reads Usable&quot; and a low fraction of &quot;Antibody Reads in Cells&quot;. How is it possible that so few antibody reads are usable? Does this mean that most antibodies were not bound to the cell surface?  &lt;a href=&quot;https://i.stack.imgur.com/cHkxg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cHkxg.png&quot; alt=&quot;cellranger metrics&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="6718" LastActivityDate="2022-02-18T14:47:45.790" Title="Low Fraction of usable antibody reads in CiteSeq" Tags="|scrnaseq|ngs|single-cell|10x-genomics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11078" PostTypeId="2" ParentId="11074" CreationDate="2020-01-05T11:11:41.790" Score="1" Body="&lt;p&gt;Error means that quotations are missing. I am not going to review this chunk but at first glance check &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;printf 'TumorAltAlleleCount\tTumorAltAlleleFrequency\t'\n'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are three quotation marks so one is not matched. I suggest you develop code within a good IDE that will help you debug. Please google around for potential solutions. I personally do R and Shell scripting in RStudio. It differently colors quoted and unquoted code so you immediately see if you missed a closing quotation somewhere.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-01-05T11:11:41.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11079" PostTypeId="1" AcceptedAnswerId="11080" CreationDate="2020-01-05T17:20:38.883" Score="1" ViewCount="63" Body="&lt;p&gt;I have a list of vcf files; I also have a list of names in a txt file like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;LP6005409-DNA_F01&#xA;LP2000325-DNA_A01&#xA;LP6005409-DNA_E02&#xA;LP6005500-DNA_C03&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What I have in this txt files are part of name of each vcf files&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to move .vcf files having these in their name to a separate folder&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I done&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[fi1d18@cyan01 TRG]&lt;span class=&quot;math-container&quot;&gt;$ for i in *.vcf&#xA;&amp;gt; do if grep -q $&lt;/span&gt;i 1.txt;&#xA;&amp;gt; cp *&lt;span class=&quot;math-container&quot;&gt;$out* /temp/hgig/fi1d18/TRG45/snp/snp/TRG/pre/&#xA;&amp;gt; done&#xA;-bash: syntax error near unexpected token `done'&#xA;[fi1d18@cyan01 TRG]$&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How I can complete this task?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="298" LastEditDate="2020-01-06T09:43:15.410" LastActivityDate="2020-01-21T19:29:07.280" Title="Moving file based on their names" Tags="|bash|linux|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11080" PostTypeId="2" ParentId="11079" CreationDate="2020-01-05T19:34:42.530" Score="4" Body="&lt;p&gt;First of all you are getting an error because you are missing a &lt;code&gt;then&lt;/code&gt; in the third line of your code and then a &lt;code&gt;fi&lt;/code&gt; to close the &lt;code&gt;if&lt;/code&gt;. It should be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for i in *.vcf&#xA;do if grep -q &lt;span class=&quot;math-container&quot;&gt;$i 1.txt; then&#xA;    cp *$&lt;/span&gt;out* /temp/hgig/fi1d18/TRG45/snp/snp/TRG/pre/&#xA;fi&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I guess the one-liner below achieves what you are trying to do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;while IFS= read -r line; do mv &quot;$line&quot;*.vcf some-directory; done &amp;lt; file-list.txt &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="298" LastEditDate="2020-01-06T09:38:19.013" LastActivityDate="2020-01-06T09:38:19.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11081" PostTypeId="1" CreationDate="2020-01-05T19:47:10.463" Score="0" ViewCount="88" Body="&lt;p&gt;I'm studying on a plant that has no reference genome and only has one scaffold assembly and one gff3 annotation file. Can I create an index with the same assembly and gff3 in STAR and do the mapping? If I want to do de novo assembly using the Galaxy Can I import the links of 20 fastq files from the RNA-seq of the samples together to construct the Reference Genome with Trinity? If this is possible, please help me import the links of these files together into Galaxy and make the reference genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot&lt;/p&gt;&#xA;" OwnerUserId="6719" LastActivityDate="2020-01-05T20:43:29.183" Title="mapping and de novo assembly of plant without reference genome" Tags="|rna-seq|assembly|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11082" PostTypeId="2" ParentId="11081" CreationDate="2020-01-05T20:43:29.183" Score="1" Body="&lt;p&gt;You have multiple questions, I will try to answer one by one:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;On creating an index with &lt;code&gt;STAR&lt;/code&gt;: As long as you provide a fasta file and its corresponding gff, you will be able to generate an index.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;On creating a reference with &lt;code&gt;Trinity&lt;/code&gt;: You have mentioned &quot;to construct the Reference Genome with Trinity&quot;, however, with your RNA-seq data and Trinity, you can construct a de novo transcriptome and not a genome.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Uploading files to &lt;code&gt;Galaxy&lt;/code&gt;: I suggest to do so via FTP using &lt;code&gt;FileZilla&lt;/code&gt; or something similar. The transfer can take some time if your files are large though.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If I understand your questions correctly, you are performing RNA-seq and you are concerned about the completeness of the genome of your model organism. In that case, I would advise to bring together a de novo transcriptome from your RNA-seq data (with Trinity), use one of the quasi-mapping methods (with a tool like &lt;code&gt;Salmon&lt;/code&gt; or &lt;code&gt;kallisto&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-01-05T20:43:29.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11083" PostTypeId="1" CreationDate="2020-01-06T01:49:45.790" Score="0" ViewCount="473" Body="&lt;p&gt;I have a &lt;code&gt;.vcf&lt;/code&gt; file generated using &lt;code&gt;samtools&lt;/code&gt;. I want to know mutations in specific genes by comparing &lt;code&gt;.vcf&lt;/code&gt; files with a reference genome. I cannot use the Ensembl VEP tool as my reference genome is not listed there. Can you please help?&lt;/p&gt;&#xA;" OwnerUserId="6720" LastEditorUserId="3541" LastEditDate="2020-01-06T07:46:03.750" LastActivityDate="2020-02-05T10:01:50.360" Title="bedtool intersect to compare .vcf file to reference genome" Tags="|vcf|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11084" PostTypeId="2" ParentId="11083" CreationDate="2020-01-06T09:59:52.677" Score="1" Body="&lt;p&gt;If you want to filter a VCF file, you might want to give &lt;strong&gt;VCFtools&lt;/strong&gt; a try.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the documentation : &lt;a href=&quot;http://vcftools.sourceforge.net/man_latest.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;VCFtools Manual&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not a specialist of this tool but you could do as the following. First build a bedfile with the chromosome names and positions of your genes (start and stop) such as this :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr   start   stop   (name)   (score)   (strand)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then it should work with this command line :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;vcftools --vcf file.vcf --bed positions.bed --out filtered.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output will be a VCF filtered on the provided regions&lt;/p&gt;&#xA;" OwnerUserId="6273" LastActivityDate="2020-01-06T09:59:52.677" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11085" PostTypeId="2" ParentId="11074" CreationDate="2020-01-06T11:36:28.780" Score="2" Body="&lt;p&gt;If we save the script pasted in the main post as a sh file and we have some .vcf files in a folder, by this line we can iterate over vcf files to extract what mentioned in the script returning .txt output by the name of corresponding vcf&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;for file in *.vcf ; do bash indel_vcf_parasing.sh $file ; done&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-01-07T15:28:39.057" LastActivityDate="2020-01-07T15:28:39.057" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11087" PostTypeId="1" CreationDate="2020-01-06T14:37:48.307" Score="0" ViewCount="376" Body="&lt;p&gt;I am analyzing a single-cell sequencing dataset from the website 10xgenomics, with 2000 cells. It is a BAM file and I am trying to obtain the individual cells per sample. I used the command&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view mtdnaAsorted.bam | grep CB:Z: | sed 's/.CB:Z:([ACGT]).*/\1/' | sort | uniq -c &amp;gt; reads_per_barcode_mtdnaA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to get a list of the individual bar codes and their reads, but there are 100,000 unique barcodes even with only 2000 cells. Does anyone know what could be the problem?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know that there could be noise, but this is a problem for me because I'm trying to analyze coverage per cell, and if there are a lot of cells that contribute towards the total number of reads, the net coverage per cell is going to be really low&lt;/p&gt;&#xA;" OwnerUserId="6722" LastEditorUserId="964" LastEditDate="2020-01-06T14:51:37.723" LastActivityDate="2020-01-06T15:37:37.463" Title="Single-cell sequencing dataset has too many barcodes" Tags="|samtools|single-cell|10x-genomics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11088" PostTypeId="2" ParentId="11087" CreationDate="2020-01-06T15:37:37.463" Score="4" Body="&lt;p&gt;The BAM file you downloaded has all of the detected 10X barcodes instead of those that represent cells.  In a given 10X experiment the number of input barcodes vastly outnumbers the input cell count.  After the reaction some of barcoded beads are invariably sequenced because they are encapsulated with ambient RNA, dead cells etc. After processing the BAM file Cell Ranger infers which cells likely represent cells and generates gene expression matrices from these files.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would need to download one of the output files that contains the list of passed barcodes such as the &lt;code&gt;Gene / cell matrix (filtered)&lt;/code&gt; and then use the list of passed barocode sequences to filter your BAM file. Alternatively you could approximately select the same cells by selecting the top 2,000 barcodes with the highest numbers of reads.&lt;/p&gt;&#xA;" OwnerUserId="64" LastActivityDate="2020-01-06T15:37:37.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11090" PostTypeId="1" CreationDate="2020-01-06T21:12:25.983" Score="1" ViewCount="70" Body="&lt;p&gt;I'm wondering if there are any tools out there that perform statistical phasing of genetic data (e.g. Eagle) that take read-based phasing from GATK into account.  I've found &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/35/14/i242/5529122&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; which sounds like what I am looking for, but that tool was developed specifically for Hi-C data, and I only have exomes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="59" LastActivityDate="2020-10-06T21:02:45.400" Title="Read-informed statistical phasing" Tags="|exome|genetics|phasing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11093" PostTypeId="1" AcceptedAnswerId="11130" CreationDate="2020-01-07T10:32:50.877" Score="1" ViewCount="90" Body="&lt;p&gt;&lt;strong&gt;Background:&lt;/strong&gt;&#xA;I am looking for some advice on the use of SNP variants versus INDEL variants for determining Genetic Genealogy, specifically for determining accurately the &quot;START&quot; and &quot;END&quot; of shared DNA segments (shared between two DNA testers' results). The test data used is RAW DNA from direct to consumer DNA testing companies such as AncestryDNA and 23andMe.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&#xA;For the purpose of Genetic Genealogy, are the INDEL variants as useful as the SNP variants? Should the INDEL variants be ignored? Or, alternatively, do the INDEL variants hold a greater significance than the SNP variants?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I hope that using INDEL variants would increase the data in the comparison algorithm, thus increasing the certainty and resolution when determining each shared segment.&lt;/p&gt;&#xA;" OwnerUserId="6728" LastEditorUserId="3233" LastEditDate="2020-01-13T15:51:43.230" LastActivityDate="2020-01-13T15:51:43.230" Title="Genetic Genealogy related to SNP variants and INDEL variants" Tags="|genetics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11094" PostTypeId="1" AcceptedAnswerId="11095" CreationDate="2020-01-07T13:27:10.803" Score="2" ViewCount="395" Body="&lt;p&gt;I got NGS reads back from sequencing platform. I check for adaptors and trimmed them. But I realized only a fraction (eg 30%) have adaptors... why not all of them?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="6459" LastEditorUserId="181" LastEditDate="2020-01-07T16:59:20.903" LastActivityDate="2020-01-07T16:59:20.903" Title="Why don't all reads have adaptors?" Tags="|ngs|adapter|cutadapt|fastqc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11095" PostTypeId="2" ParentId="11094" CreationDate="2020-01-07T13:39:04.447" Score="8" Body="&lt;p&gt;Things like this might depend on your specific library prep, but in general: sequencing starts at the end of the adapter, not before it. You will only see adapters if you sequence through the entire fragment into the adapters on the other side. If your fragment is long enough you won't see any adapter, and of course, that's the desirable outcome of your library prep.&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2020-01-07T13:39:04.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11096" PostTypeId="2" ParentId="5454" CreationDate="2020-01-07T20:55:28.797" Score="0" Body="&lt;p&gt;For me it works best to combine &lt;code&gt;cat&lt;/code&gt;, &lt;code&gt;while&lt;/code&gt; and official download tool &lt;code&gt;fasterq-dump&lt;/code&gt;.   &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat your_accession_list.txt | while read i&#xA;do  &#xA;fasterq-dump ${i} &amp;lt;options&amp;gt;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As commented above, &lt;code&gt;your_accession_list.txt&lt;/code&gt; should have Linux endings. Link to &lt;code&gt;fasterq-dump&lt;/code&gt; is &lt;a href=&quot;https://github.com/ncbi/sra-tools/wiki/HowTo:-fasterq-dump&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2268" LastActivityDate="2020-01-07T20:55:28.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11097" PostTypeId="2" ParentId="7027" CreationDate="2020-01-07T21:00:57.383" Score="0" Body="&lt;p&gt;Not a command-line tool but also useful is redesigned &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Traces/study/?&quot; rel=&quot;nofollow noreferrer&quot;&gt;SRA Run Selector&lt;/a&gt;. Insert comma separated accessions and you can filter/download based on metadata.&lt;/p&gt;&#xA;" OwnerUserId="2268" LastActivityDate="2020-01-07T21:00:57.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11098" PostTypeId="2" ParentId="411" CreationDate="2020-01-07T21:06:08.390" Score="1" Body="&lt;p&gt;Another possibility in 2020 is to use &lt;a href=&quot;https://github.com/ncbi/sra-tools/wiki/HowTo:-fasterq-dump&quot; rel=&quot;nofollow noreferrer&quot;&gt;fasterq-dump&lt;/a&gt; which is promoted to be faster than previous &lt;code&gt;fastq-dump&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2268" LastActivityDate="2020-01-07T21:06:08.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11099" PostTypeId="2" ParentId="11075" CreationDate="2020-01-07T21:44:58.037" Score="1" Body="&lt;p&gt;&lt;em&gt;This is a copy-paste nearly verbatim of comments so that the question has an answer&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you are trying to do is find what genes are in what homology cluster. This is common problem and there are many solution each with some issues. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Uniprot90 is indeed a cluster of homologues but it is too limited. Whereas you require clusters that span all the domains (universal).&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;COG&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;The nearly original grouping, is COG (cluster of orthologous genes). It is really handy for this but aren't really available from Uniprot and NCBI (its curator) has been phasing COGs out. In Uniprot records, they are &lt;em&gt;patchily&lt;/em&gt; present as its derivative eggNOG (which has varying degrees of &quot;scope&quot;, LUCA being the universal one):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;dbReference type=&quot;eggNOG&quot; id=&quot;COG0451&quot;&amp;gt;&#xA;&amp;lt;property type=&quot;taxonomic scope&quot; value=&quot;LUCA&quot;/&amp;gt;&#xA;&amp;lt;/dbReference&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Pfam&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Another option that is well mapped and always present when possible in Uniprot entries is &lt;a href=&quot;https://pfam.xfam.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pfam&lt;/a&gt; ids. These are domain family annotations and are also good option. It is a &lt;em&gt;domain&lt;/em&gt; fold group. So domains within protein with homologous folds are grouped into one. If a protein has two known domains, it will have two Pfam entries —repeat proteins have loads, but most protein have one domain. In Uniprot XML you will find these are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;dbReference type=&quot;Pfam&quot; id=&quot;PF01370&quot;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Similarly there is InterPro, which is more narrow than Pfam, but generally overlaps.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;DIY&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Then there's always the option of making your own psi-Blast motif on the NCBI website (setting high number of hits, setting the database to refseq or even PDB and doing a few iterations), saving the motif pattern (PSSM) and using local blast with it.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Caveats&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;In general you may however encounter these pitfalls:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;you may have two protein clusters that share a single id&lt;/li&gt;&#xA;&lt;li&gt;you have gene fusions (one protein, 2 domains of interest). In Pfam the protein will have two ids, with COGs or similar, you'd have only an anamalous length to go by.&lt;/li&gt;&#xA;&lt;li&gt;you may have wished to find analogues (i.e. two protein that arose independently that have the same role (a lot of ribosomal protein are simply structural). No homology clustering scheme can find these.  Say a bacterium has evolved to have a different protein with that role, yet as it has never been studied it is a domain of unknown function (guilt by association is the common strategy, but that is beyond your scope here).&lt;/li&gt;&#xA;&lt;li&gt;you have a very small protein which is poorly conserved&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-01-07T21:44:58.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11100" PostTypeId="1" CreationDate="2020-01-07T23:27:34.313" Score="2" ViewCount="164" Body="&lt;p&gt;Is there any user friendly way to find rare mutations in the individual human whole genome sequencing raw data? (from Dante, 30x coverage).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be more specific, I want to find mutations from this paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://docs.google.com/document/d/1EkRMuD6J0-zyMY3MegKhz7Th0hzhzVXg6DWg2Hm6l2Y/edit?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.google.com/document/d/1EkRMuD6J0-zyMY3MegKhz7Th0hzhzVXg6DWg2Hm6l2Y/edit?usp=sharing&lt;/a&gt;&#xA;(it's very short, less than one page).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I'm confused. In their paper they just talk about genes (RAD21, B3GAT2, SMC3, SCN11A , SCN5A, SCN9A, SCN10A, SCN11A, TRPA1), but not mutations? Can we find that diseased-mutations which they talking about in genes, which their list? Or is there not enough data in the paper/study for this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And if there is not enough data, which data I need to request from authors?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or those genes have kind a &quot;gold standard sequences&quot; and if that sequence different from standard - there is &quot;diseased&quot; gene?&#xA;(I have only very basic genetics and bioinformatics knowledges)&lt;/p&gt;&#xA;" OwnerUserId="6733" LastEditorUserId="6733" LastEditDate="2020-01-07T23:36:13.367" LastActivityDate="2021-02-06T17:04:50.470" Title="Any user friendly way to find rare mutations in whole genome raw?" Tags="|gene|ngs|genome|sequence-analysis|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11101" PostTypeId="1" CreationDate="2020-01-08T10:20:54.703" Score="1" ViewCount="165" Body="&lt;p&gt;I am trying to code a shiny app for RNA-Seq data analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to include glimma interactive plots in it. &#xA;However, in my current interface, clicking the action button &lt;code&gt;Glimma plot&lt;/code&gt; opens the glimma interface in a new page of my web browser and not in the shiny UI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use &lt;code&gt;shiny&lt;/code&gt;, &lt;code&gt;glimma&lt;/code&gt; and &lt;code&gt;EdgeR&lt;/code&gt;. Here is my code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ui &amp;lt;- fluidPage(&#xA;  actionButton(inputId=&quot;run.glimma&quot;, label=&quot;Glimma plot&quot;),&#xA;  htmlOutput(&quot;glimma&quot;)&#xA;&#xA;&#xA;server &amp;lt;- function(input, output, session){&#xA;  observeEvent(input&lt;span class=&quot;math-container&quot;&gt;$run.glimma, {&#xA;    glMDSPlot(dgeObj.norm(), labels=rownames(dgeObj.norm()$samples), groups=dgeObj.norm()$samples[,as.character(input$&lt;/span&gt;mds.grouping.feature)])&#xA;  }),&#xA;&#xA;  output$glimma &amp;lt;- renderUI({&#xA;      includeHTML(&quot;glimma-plots/MDS-Plot.html&quot;)&#xA;    })&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I only indicated the part of the code that I use to generate the &lt;code&gt;glimma&lt;/code&gt; MDS plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to display the &lt;code&gt;glimma&lt;/code&gt; plots in the shiny app UI and not in a new webpage ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="6734" LastActivityDate="2022-09-28T16:27:52.547" Title="include a glimma interface in a shiny app" Tags="|r|rna-seq|bioconductor|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11102" PostTypeId="2" ParentId="11060" CreationDate="2020-01-08T10:33:08.700" Score="1" Body="&lt;p&gt;Read the manuals for &lt;code&gt;data.table::merge&lt;/code&gt; and &lt;code&gt;base::merge&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This (wrong syntax):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;unique(merge(data1,data, x.by = data1&lt;span class=&quot;math-container&quot;&gt;$Position, y.by = data$&lt;/span&gt;Position))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Should work fine as below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;merge(data1, data, by = &quot;Position&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Regarding: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;losing a big part of big data frame&quot;`&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;It is only returning rows when there is a match on &lt;em&gt;Position&lt;/em&gt;, if you need to return the full dataset, then use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;merge(data1, data, by = &quot;Position&quot;, all.x = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="131" LastActivityDate="2020-01-08T10:33:08.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11103" PostTypeId="5" CreationDate="2020-01-08T10:42:27.667" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Looper is a pipeline submitting engine. Looper deploys any command-line pipeline for each sample in a project organized in standard PEP format. You can think of looper as providing a single user interface to running, summarizing, monitoring, and otherwise managing all of your sample-intensive research projects the same way, regardless of data type or pipeline used.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/pepkit/looper&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://looper.databio.org/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Website&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="131" LastEditorUserId="131" LastEditDate="2020-01-08T11:19:11.060" LastActivityDate="2020-01-08T11:19:11.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11104" PostTypeId="4" CreationDate="2020-01-08T10:42:27.667" Score="0" Body="A job submitter that uses Portable Encapsulated Projects" OwnerUserId="131" LastEditorUserId="131" LastEditDate="2020-01-08T11:19:06.460" LastActivityDate="2020-01-08T11:19:06.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11105" PostTypeId="2" ParentId="10960" CreationDate="2020-01-08T10:50:10.843" Score="2" Body="&lt;p&gt;Change to &lt;em&gt;factor&lt;/em&gt; then to &lt;em&gt;numeric&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mat&lt;span class=&quot;math-container&quot;&gt;$Tumor_Sample_Barcode &amp;lt;- as.numeric(as.factor(mat$&lt;/span&gt;Tumor_Sample_Barcode))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using @dc37 example data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;as.numeric(as.factor(df$vect))&#xA;# [1] 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If we need to prefix with &quot;A&quot;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;paste0(&quot;A&quot;, as.numeric(as.factor(df$vect)))&#xA;# [1] &quot;A1&quot; &quot;A2&quot; &quot;A3&quot; &quot;A4&quot; &quot;A1&quot; &quot;A2&quot; &quot;A3&quot; &quot;A4&quot; &quot;A1&quot; &quot;A2&quot; &quot;A3&quot; &quot;A4&quot; &quot;A1&quot; &quot;A2&quot; &quot;A3&quot; &quot;A4&quot; &quot;A1&quot; &quot;A2&quot; &quot;A3&quot; &quot;A4&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or we can keep them as &lt;em&gt;factors&lt;/em&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;x &amp;lt;- sort(unique(df&lt;span class=&quot;math-container&quot;&gt;$vect))&#xA;factor(df$&lt;/span&gt;vect, levels = x, labels = paste0(&quot;A&quot;, seq_along(x)))&#xA;# [1] A1 A2 A3 A4 A1 A2 A3 A4 A1 A2 A3 A4 A1 A2 A3 A4 A1 A2 A3 A4&#xA;# Levels: A1 A2 A3 A4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="131" LastEditorUserId="131" LastEditDate="2020-01-08T10:56:49.523" LastActivityDate="2020-01-08T10:56:49.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11106" PostTypeId="1" AcceptedAnswerId="11107" CreationDate="2020-01-08T13:24:29.093" Score="1" ViewCount="263" Body="&lt;p&gt;I am running the &lt;code&gt;STAR-RSEM&lt;/code&gt; pipeline, I have a simple shell script(*), which I use to loop over fasta files corresponding to different samples. From the output, I am interested in the &lt;code&gt;*.genes.results&lt;/code&gt; files, for which the columns are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; names(my_df1)&#xA;[1] &quot;gene_id&quot; &quot;transcript_id(s)&quot; &quot;length&quot; &quot;effective_length&quot; &quot;expected_count&quot; &quot;TPM&quot; &quot;FPKM&quot;            &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To my surprise, whereas the &lt;code&gt;gene_id&lt;/code&gt; and &lt;code&gt;transcript_id(s)&lt;/code&gt; columns are the same, I see different &lt;code&gt;length&lt;/code&gt; and &lt;code&gt;effective_length&lt;/code&gt; values for the same genes in different sample results (omitting the &lt;code&gt;transcript_id(s)&lt;/code&gt; column for formatting purposes):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; my_df1 &amp;lt;- fread(paste0(&quot;alignment/&quot;, my_files[1]))&#xA;&amp;gt; head(my_df1[1:5,c(1,3:6)])&#xA;              gene_id  length effective_length expected_count   TPM&#xA;1: ENSG00000000003.14 2292.23          2242.23            978 26.69&#xA;2:  ENSG00000000005.5 1339.00          1289.00             11  0.52&#xA;3: ENSG00000000419.12 1024.45           974.45            469 29.45&#xA;4: ENSG00000000457.13 4204.00          4154.00            241  3.55&#xA;5: ENSG00000000460.16 2953.40          2903.40             62  1.31&#xA;&amp;gt; &#xA;&amp;gt; my_df2 &amp;lt;- fread(paste0(&quot;alignment/&quot;, my_files[2]))&#xA;&amp;gt; head(my_df2[1:5,c(1,3:6)])&#xA;              gene_id  length effective_length expected_count    TPM&#xA;1: ENSG00000000003.14 2260.38          2210.38       10052.00 117.78&#xA;2:  ENSG00000000005.5 1269.22          1219.22         132.00   2.80&#xA;3: ENSG00000000419.12 1058.13          1008.13        4386.00 112.68&#xA;4: ENSG00000000457.13 4175.05          4125.05         873.00   5.48&#xA;5: ENSG00000000460.16 2783.92          2733.92         780.01   7.39&#xA;&amp;gt; &#xA;&amp;gt; my_df3 &amp;lt;- fread(paste0(&quot;alignment/&quot;, my_files[3]))&#xA;&amp;gt; head(my_df3[1:5,c(1,3:6)])&#xA;              gene_id  length effective_length expected_count  TPM&#xA;1: ENSG00000000003.14 2662.59          2612.59             49 2.90&#xA;2:  ENSG00000000005.5  940.50           890.50              0 0.00&#xA;3: ENSG00000000419.12  956.87           906.87             58 9.89&#xA;4: ENSG00000000457.13 6364.00          6314.00             66 1.62&#xA;5: ENSG00000000460.16 2661.00          2611.00             15 0.89&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;length&lt;/code&gt; and &lt;code&gt;effective_length&lt;/code&gt; are defined per gene (and its transcripts) and should not differ across samples. Below is from the &lt;a href=&quot;http://deweylab.biostat.wisc.edu/rsem/rsem-calculate-expression.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A gene's 'length' and 'effective_length' are defined as the weighted&#xA;  average of its transcripts' lengths and effective lengths (weighted by&#xA;  'IsoPct')&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I would appreciate any hints pointing out what I am missing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;* The script I am using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for file in ...; \&#xA;do rsem-calculate-expression \&#xA;-p 8 \&#xA;--star \&#xA;# --output-genome-bam \&#xA;--star-path path-to-star-executable \&#xA;--star-gzipped-read-file \&#xA;--forward-prob=0 \&#xA;&lt;span class=&quot;math-container&quot;&gt;$file \&#xA;path-to-genome \ # should end with the string used for the &quot;reference_name&quot; parameter of &quot;rsem-prepare-reference&quot;&#xA;$&lt;/span&gt;{file:11:7}; \ # sample prefix&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;EDIT: For those who will be interested, &lt;a href=&quot;https://groups.google.com/d/msg/rsem-users/gDh0OBLK6_4/z-3ZoETxn5sJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;this entry&lt;/a&gt; from the RSEM Users Google Group demonstrates how to calculate gene length from weighted transcript length.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2020-01-08T14:19:46.073" LastActivityDate="2020-01-08T14:19:46.073" Title="Why I am getting different gene lengths for the same genes in different samples with rsem-calculate-expression?" Tags="|star|rsem|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11107" PostTypeId="2" ParentId="11106" CreationDate="2020-01-08T13:27:32.387" Score="2" Body="&lt;p&gt;If there are differences in the expression of individual transcripts between samples then both the length and effective length will differ between samples. It is commonly the case that the ratio of longer and short transcript expression varies a bit between samples, which gives rise to this.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-01-08T13:27:32.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11108" PostTypeId="1" AcceptedAnswerId="13052" CreationDate="2020-01-08T15:18:24.273" Score="4" ViewCount="282" Body="&lt;p&gt;I am mapping kmers back to a few bacterial genomes using &lt;code&gt;bwa fastmap&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bwa fastmap -l 9 ref.fasta kmers.fasta &amp;gt; out.fastmap                                           &#xA;[M::bwa_idx_load_from_disk] read 0 ALT contigs                                                                                                                                                                     &#xA;[main] Version: 0.7.17-r1188                                                                                                                                                                                       &#xA;[main] CMD: bwa fastmap -l 9 ref.fasta kmers.fasta                                                                                                                   &#xA;[main] Real time: 0.168 sec; CPU: 0.157 sec&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SQ      AAAAGTAGTAAGCAGGAAGACAACACGGTTG 31&#xA;EM      0       11      1       contig1:+2041368&#xA;EM      3       13      2       contig1:+1875252        contig1:-2474744&#xA;EM      4       15      1       contig1:+3779779&#xA;EM      5       16      1       contig1:-3253348&#xA;EM      6       17      2       contig1:+938710 contig1:+1066682&#xA;EM      7       20      1       contig1:-4912797&#xA;EM      10      21      4       contig1:+4803473        contig1:-2830252        contig1:+4495150        contig1:-4907283&#xA;EM      11      23      1       contig1:-3148132&#xA;EM      13      24      1       contig1:-4651172&#xA;EM      14      25      3       contig1:-2142223        contig1:+4994474        contig1:+873066&#xA;EM      16      26      4       contig1:+156775 contig1:+27749  contig1:+2207492        contig1:-1340811&#xA;EM      17      29      1       contig1:+3523989&#xA;EM      19      30      24      *&#xA;&#xA;EM      20      31      3       contig1:-2533354        contig1:-208660 contig1:-1080177&#xA;//&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question is: what is the meaning of the &lt;code&gt;*&lt;/code&gt; char in the row before last? Why is &lt;code&gt;bwa fastmap&lt;/code&gt; not reporting all the hits as in the other lines? I could not find a page that explains this output anywhere.&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2020-04-23T21:29:10.223" Title="What is the meaning of the &quot;*&quot; character in bwa fastmap's output?" Tags="|sequence-alignment|bwa|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11109" PostTypeId="2" ParentId="11108" CreationDate="2020-01-09T00:25:34.623" Score="3" Body="&lt;p&gt;It means the read is unmapped. No origin in the reference could be found that was sufficiently similar to the read to call it a proper alignment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the &lt;a href=&quot;http://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAM specification&lt;/a&gt;, &lt;code&gt;*&lt;/code&gt; is always used if the information for that field is not available. Therefore, if no mapping information is available (=unmapped) one sets &lt;code&gt;*&lt;/code&gt;. &lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorUserId="298" LastEditDate="2020-01-09T10:05:18.677" LastActivityDate="2020-01-09T10:05:18.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11110" PostTypeId="1" AcceptedAnswerId="11111" CreationDate="2020-01-09T12:30:53.240" Score="-2" ViewCount="186" Body="&lt;p&gt;I have two dataframes&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(a[1:4,1:4])&#xA;   Tumor_Sample_Barcode Chromosome Position End_Position&#xA;1:                   A1       chr4 90169866     90169866&#xA;2:                   A1      chr11 60235747     60235747&#xA;3:                   A1       chr1   983023       983023&#xA;4:                   A1       chr1 11346060     11346060&#xA;&amp;gt; &#xA;&#xA;&amp;gt; head(c)&#xA;   Chromosome Position      VAF&#xA;1:       chrM     6691 0.610284&#xA;2:       chrM    14503 0.693325&#xA;3:       chr1 31412236 0.645161&#xA;4:       chr1 55693305 0.602941&#xA;5:       chr1 69963412 0.709302&#xA;6:       chr1 72608266 0.720000&#xA;&amp;gt;&#xA;&#xA;&amp;gt; str(c)&#xA;Classes ‘data.table’ and 'data.frame':  44175 obs. of  3 variables:&#xA; &lt;span class=&quot;math-container&quot;&gt;$ Chromosome: chr  &quot;chrM&quot; &quot;chrM&quot; &quot;chr1&quot; &quot;chr1&quot; ...&#xA; $&lt;/span&gt; Position  : num  6691 14503 31412236 55693305 69963412 ...&#xA; $ VAF       : num  0.61 0.693 0.645 0.603 0.709 ...&#xA; - attr(*, &quot;.internal.selfref&quot;)=&amp;lt;externalptr&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to merge them like this but complaining about not being integer&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; a=a[c, on = c(&quot;Chromosome&quot;,&quot;Position&quot;), VAF := i.VAF]&#xA;Error in bmerge(i, x, leftcols, rightcols, roll, rollends, nomatch, mult,  : &#xA;  Incompatible join types: x.Position (character) and i.Position (integer)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have tried something like these but not working&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; c=c[!is.na(as.numeric(as.character(c&lt;span class=&quot;math-container&quot;&gt;$Position))), ]&#xA;&amp;gt; &#xA;&amp;gt; c=c[!is.na(as.numeric(as.character(c$&lt;/span&gt;VAF))), ]&#xA;&amp;gt; &#xA;&amp;gt; c&lt;span class=&quot;math-container&quot;&gt;$VAF &amp;lt;- as.numeric(as.character(c$&lt;/span&gt;VAF)) &#xA;&#xA;c&lt;span class=&quot;math-container&quot;&gt;$Position &amp;lt;- as.numeric(as.character(c$&lt;/span&gt;Position)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which not working&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2020-01-09T14:25:50.377" Title="Making some columns if a dataframe numeric" Tags="|r|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11111" PostTypeId="2" ParentId="11110" CreationDate="2020-01-09T14:25:50.377" Score="2" Body="&lt;p&gt;Your issue seems to be related to the column &lt;code&gt;Position&lt;/code&gt; being a character column in &lt;code&gt;a&lt;/code&gt;, you should try:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(data.table)&#xA;a$Position &amp;lt;- as.numeric(a$Position)&#xA;a[c, on = c(&quot;Chromosome&quot;,&quot;Position&quot;), VAF := i.VAF]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6437" LastActivityDate="2020-01-09T14:25:50.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11113" PostTypeId="1" AcceptedAnswerId="11116" CreationDate="2020-01-09T14:47:55.620" Score="1" ViewCount="659" Body="&lt;p&gt;I have generated a list of canonical markers for cluster 0 using the following command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cluster0_canonical &amp;lt;- FindMarkers(project, ident.1=0,  ident.2=c(1,2,3,4,5,6,7,8,9,10,11,12,13,14), grouping.var = &quot;status&quot;, &#xA;min.pct = 0.25, print.bar = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I got the following output, but I do not understand why the canonically expressed &lt;strong&gt;ribosomal&lt;/strong&gt; markers seem to be expressed at high levels in both the cells within the cluster AND cells that are NOT in the cluster (i.e. both pct.1 and pct.2 are close to 1, but P value is highly significant). We would not expect this for canonical markers - we would instead expect the genes to be expressed at high levels in the cells within the cluster but at low levels in cells not within the cluster. I don't seem to have such an issue with non-ribosomal genes within cluster 0 and cluster 0 is the only cluster where I get a high number of ribosomal genes as canonical markers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gtJs4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gtJs4.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4621" LastEditorUserId="181" LastEditDate="2020-01-09T17:05:16.613" LastActivityDate="2020-01-09T17:05:16.613" Title="Seurat FindMarkers() output, percentage" Tags="|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11115" PostTypeId="1" CreationDate="2020-01-09T16:20:47.123" Score="4" ViewCount="8314" Body="&lt;p&gt;I want to convert my gff3 annotation files to genbank format for use in Mauve. I found the seqret tool here &lt;a href=&quot;https://www.ebi.ac.uk/Tools/sfc/emboss_seqret/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/Tools/sfc/emboss_seqret/&lt;/a&gt; which can perform this task, but my files (bacterial genomes) are too big.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest a simple method to make this conversion?&lt;/p&gt;&#xA;" OwnerUserId="6738" LastActivityDate="2022-07-16T11:23:30.180" Title="Existing tool for converting gff3 to genbank (gbk)" Tags="|format-conversion|gff3|genbank|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11116" PostTypeId="2" ParentId="11113" CreationDate="2020-01-09T16:57:56.690" Score="3" Body="&lt;p&gt;Your interpretation of &lt;code&gt;pct.1&lt;/code&gt; and &lt;code&gt;pct.2&lt;/code&gt; is misguided. If &lt;code&gt;pct.1 == 1&lt;/code&gt;, the given gene was detected in all of the cells. This carries no information about high expression or low expression in the cell, just that it was detected. &lt;code&gt;avg_logFC&lt;/code&gt; is the indication of expression &lt;strong&gt;levels&lt;/strong&gt;. So your expectation, that &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;we would instead expect the genes to be expressed at high levels in the cells within the cluster but at low levels in cells not within the cluster&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;holds true, as indicated by the positive &lt;code&gt;avg_logFC&lt;/code&gt; and significant p-values of the listed RP genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Seurat use nautral log, so the FC of RPS6 in cluster 0 vs. all other clusters indicated is 2.718281828459^.55947=1.750.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Seurat &lt;code&gt;FindMarkers()&lt;/code&gt; &lt;a href=&quot;https://www.rdocumentation.org/packages/Seurat/versions/3.1.2/topics/FindMarkers&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="181" LastActivityDate="2020-01-09T16:57:56.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11117" PostTypeId="1" AcceptedAnswerId="11119" CreationDate="2020-01-09T18:36:47.323" Score="0" ViewCount="409" Body="&lt;p&gt;I have a data frame like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(a[1:2,1:4])&#xA;   Tumor_Sample_Barcode Chromosome Position End_Position&#xA;1:                   A1       chr4 90169866     90169866&#xA;2:                   A1      chr11 60235747     60235747&#xA;    &amp;gt; dim(a)&#xA;[1] 753655    134&#xA;&amp;gt;&#xA;&amp;gt; unique(a$Tumor_Sample_Barcode)&#xA; [1] &quot;A1&quot;  &quot;A10&quot; &quot;A11&quot; &quot;A12&quot; &quot;A13&quot; &quot;A14&quot; &quot;A15&quot; &quot;A16&quot; &quot;A17&quot; &quot;A18&quot; &quot;A19&quot; &quot;A2&quot;  &quot;A20&quot; &quot;A21&quot; &quot;A22&quot; &quot;A23&quot; &quot;A24&quot; &quot;A25&quot; &quot;A26&quot; &quot;A27&quot; &quot;A28&quot;&#xA;[22] &quot;A29&quot; &quot;A3&quot;  &quot;A30&quot; &quot;A31&quot; &quot;A32&quot; &quot;A33&quot; &quot;A4&quot;  &quot;A5&quot;  &quot;A6&quot;  &quot;A7&quot;  &quot;A8&quot;  &quot;A9&quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to change &lt;code&gt;a$Tumor_Sample_Barcode&lt;/code&gt;  from &lt;strong&gt;&lt;em&gt;A1:A21&lt;/em&gt;&lt;/strong&gt; to &lt;strong&gt;&lt;em&gt;res1:res21&lt;/em&gt;&lt;/strong&gt; but I don't how to do that&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="131" LastEditDate="2020-01-10T15:52:22.353" LastActivityDate="2020-01-10T15:52:22.353" Title="Replace a prefix of a string" Tags="|r|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11118" PostTypeId="1" CreationDate="2020-01-09T19:12:21.117" Score="2" ViewCount="47" Body="&lt;p&gt;I'm building a somatic variant calling pipeline with &lt;code&gt;snakemake&lt;/code&gt;. The pipeline is hosted on Github, and I'd like to use Travis CI for continuous integration testing. Due to privacy concerns, I am looking for a publicly available tumor-normal dataset similar to Genome in a Bottle's NA12878 dataset that I can place online without privacy concerns and use for integration testing. Any sources to find tumor-normal FASTQ data that is publicly available?&lt;/p&gt;&#xA;" OwnerUserId="6674" LastActivityDate="2020-01-09T19:12:21.117" Title="NA12878 equivalent for somatic variant calling" Tags="|variant-calling|snakemake|gatk|public-dataset|" AnswerCount="0" CommentCount="3" ClosedDate="2020-01-14T08:08:12.453" ContentLicense="CC BY-SA 4.0" />
  <row Id="11119" PostTypeId="2" ParentId="11117" CreationDate="2020-01-09T20:01:30.590" Score="8" Body="&lt;p&gt;I think you are looking for &lt;code&gt;gsub&lt;/code&gt; function:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;a$Tumor_Sample_Barcode &amp;lt;- gsub(&quot;A&quot;,&quot;res&quot;,a$Tumor_Sample_Barcode)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you prefer, you can use the &lt;code&gt;str_replace_all&lt;/code&gt; function from &lt;code&gt;stringr&lt;/code&gt; package. But it tooks a little bit more time than &lt;code&gt;gsub&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Dummy data&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;seq = as.factor(paste0(&quot;A&quot;,1:21))&#xA;library(data.table)&#xA;dt &amp;lt;- data.table(Barcode = rep(seq,10000),&#xA;                 Value = rnorm(210000))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Test of &lt;code&gt;gsub&lt;/code&gt; vs &lt;code&gt;str_replace_all&lt;/code&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; system.time(dt&lt;span class=&quot;math-container&quot;&gt;$Barcode &amp;lt;- gsub(&quot;A&quot;,&quot;res&quot;,dt$&lt;/span&gt;Barcode))&#xA;   user  system elapsed &#xA;  0.077   0.000   0.077 &#xA;&#xA;&amp;gt; system.time(dt&lt;span class=&quot;math-container&quot;&gt;$Barcode &amp;lt;- str_replace_all(dt$&lt;/span&gt;Barcode,&quot;A&quot;,&quot;res&quot;))&#xA;   user  system elapsed &#xA;  0.123   0.004   0.126 &#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6437" LastEditorUserId="6437" LastEditDate="2020-01-09T20:26:55.773" LastActivityDate="2020-01-09T20:26:55.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11120" PostTypeId="2" ParentId="11117" CreationDate="2020-01-10T10:04:06.367" Score="7" Body="&lt;p&gt;Other options:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Drop 1st character and prefix with &lt;code&gt;&quot;res&quot;&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;We get 5x speed up if we set &lt;code&gt;fixed = TRUE&lt;/code&gt; for &lt;em&gt;gsub&lt;/em&gt;.   &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;microbenchmark::microbenchmark(&#xA;  # dc37 answer&#xA;  gsub(&quot;A&quot;,&quot;res&quot;, dt&lt;span class=&quot;math-container&quot;&gt;$Barcode),&#xA;  stringr::str_replace_all(dt$&lt;/span&gt;Barcode, &quot;A&quot;, &quot;res&quot;),&#xA;  # other options&#xA;  paste0(&quot;res&quot;, substring(dt&lt;span class=&quot;math-container&quot;&gt;$Barcode, 2)),&#xA;  gsub(&quot;A&quot;,&quot;res&quot;,dt$&lt;/span&gt;Barcode, fixed = TRUE),&#xA;&#xA;  # bench set up&#xA;  check = &quot;identical&quot;, &#xA;  unit = &quot;relative&quot;&#xA;  )&#xA;&#xA;Unit: relative&#xA;                                             expr      min       lq     mean   median       uq      max neval&#xA;                     gsub(&quot;A&quot;, &quot;res&quot;, dt&lt;span class=&quot;math-container&quot;&gt;$Barcode) 5.171682 5.132816 4.283014 5.033968 3.177772 2.545173   100&#xA; stringr::str_replace_all(dt$&lt;/span&gt;Barcode, &quot;A&quot;, &quot;res&quot;) 6.914598 6.870849 5.747396 6.728126 4.809257 3.518024   100&#xA;          paste0(&quot;res&quot;, substring(dt&lt;span class=&quot;math-container&quot;&gt;$Barcode, 2)) 1.983757 1.991876 1.779557 1.968488 1.480892 3.415619   100&#xA;       gsub(&quot;A&quot;, &quot;res&quot;, dt$&lt;/span&gt;Barcode, fixed = TRUE) 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000   100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="131" LastActivityDate="2020-01-10T10:04:06.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11121" PostTypeId="1" AcceptedAnswerId="11123" CreationDate="2020-01-10T14:26:01.717" Score="-1" ViewCount="80" Body="&lt;p&gt;I ran module preservation analysis between two condition and on the basis of preservation z summary i would to go for downstream analysis.So I m interested in looking for hub-genes of those modules which have lowest z summary. But so far in the tutorial i didn't find a section in the WGCNA tutorial where how to find hubgenes after module preservation analysis was mentioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion would be really appreciated &lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2020-01-10T19:57:48.983" Title="Module preservation and hub genes finding" Tags="|networks|wgcna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11122" PostTypeId="2" ParentId="11090" CreationDate="2020-01-10T18:04:13.090" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://odelaneau.github.io/shapeit4/&quot; rel=&quot;nofollow noreferrer&quot;&gt;shapeit4&lt;/a&gt; is currently (probably) the most accurate and quickest phasing algorithm which can account for sequencing reads in the model. They suggest you extract the phase information from the reads using &lt;a href=&quot;https://whatshap.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;WhatsHap&lt;/a&gt; first however. The guide for WhatsHap can be found &lt;a href=&quot;https://whatshap.readthedocs.io/en/latest/guide.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and has some info on using data processed using GATK. &lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2020-01-10T18:04:13.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11123" PostTypeId="2" ParentId="11121" CreationDate="2020-01-10T19:57:48.983" Score="1" Body="&lt;p&gt;Simply select the module(s) you are interested in and look for hub genes using the standard calculations. Having done a module preservation analysis does not change the procedure in the slightest.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2020-01-10T19:57:48.983" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11125" PostTypeId="2" ParentId="11093" CreationDate="2020-01-11T19:02:35.177" Score="0" Body="&lt;p&gt;first of all there's no way to accurately determine the start/end position of shared DNA segments when using microarray data from DTC's like 23andMe and Ancestry.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;INDEL's are indeed important when it comes to identify shared segments though I'm not sure how often the change over the course of several generations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My mum's sister has an INDEL that both my mum and her brother didn't inherit, it will be inherited by her children (my 1st cousins).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately the microarray from 23andMe doesn't give you any detailed information of than DI, II and II but not how long the deletion or insert is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To determine such it requires to use WGS data.&lt;/p&gt;&#xA;" OwnerUserId="6743" LastActivityDate="2020-01-11T19:02:35.177" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11126" PostTypeId="2" ParentId="933" CreationDate="2020-01-12T18:14:07.807" Score="2" Body="&lt;p&gt;I faced same issue recently, I found those link and python script:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;gnomAD GraphQL api&#xA;&lt;a href=&quot;https://gnomad.broadinstitute.org/api&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gnomad.broadinstitute.org/api&lt;/a&gt;&#xA;It works great but it is a kind of different query language. &#xA;Please check here for the docs: &lt;a href=&quot;https://graphql.org/learn/queries/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://graphql.org/learn/queries/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;gnomAD Python Api&#xA;&lt;a href=&quot;https://github.com/furkanmtorun/gnomad_python_api&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/furkanmtorun/gnomad_python_api&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6754" LastActivityDate="2020-01-12T18:14:07.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11127" PostTypeId="1" CreationDate="2020-01-13T13:50:41.243" Score="3" ViewCount="178" Body="&lt;p&gt;A given sequencing machine assigns a 'certainty' to each base call in the form of a per-base quality score (FASTQ format).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have reads from several machines aligned to the corresponding reference (BAM format or similar). Ignoring SNPs and misalignments (for simplicity), mismatches should represent sequencing errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to parse through the FASTQ and BAM file and evaluate how closely the per-base quality score reflects 'true' sequencing errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there an existing efficient, user-friendly, open source, well documented tool to do this? Failing that, is there a paper doing this that claims to have made the tool freely available?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another method would be to look at bases corrected using kmer analysis or similar. Again, the per-base quality score can be compared to the 'true' sequencing errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any tool to do that? I guess it could be baked into an existing error correction tool.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do short read aligners actually make use of per-base quality score?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many thanks,&lt;/p&gt;&#xA;" OwnerUserId="3233" LastActivityDate="2020-01-13T20:56:02.023" Title="Tool for calibrating base quality scores?" Tags="|sam|fastq|quality-control|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11128" PostTypeId="2" ParentId="11115" CreationDate="2020-01-13T13:55:52.120" Score="1" Body="&lt;p&gt;You can use BioPerl to do this. Read in the GFF and make gene objects and write out those gene objects in Genbank format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps this tool (doing the opposite) will get you started:&#xA;&lt;a href=&quot;https://metacpan.org/pod/bp_genbank2gff3.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://metacpan.org/pod/bp_genbank2gff3.pl&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a script doing what you ask but it doesn't use any libraries (so is probably buggy):&#xA;&lt;a href=&quot;https://gist.github.com/avrilcoghlan/5386810&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gist.github.com/avrilcoghlan/5386810&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps,&#xA;Dan.&lt;/p&gt;&#xA;" OwnerUserId="3233" LastActivityDate="2020-01-13T13:55:52.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11129" PostTypeId="2" ParentId="11100" CreationDate="2020-01-13T14:02:56.187" Score="1" Body="&lt;p&gt;You need to run a SNP finding pipeline, filter for SNPs in those genes and then run a SNP annotation pipeline. (You could annotate before you filter but it will take longer).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;SNP finding usually has three steps, 1) read alignment to reference, 2) SNP calling and 3) SNP filtering (by quality).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You then bring in gene annotation data to filter SNPs to just those genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You then run SNP annotation using tools like VEP, SIFT, PolyPhen, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;People use tools like GATK to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hopefully that's enough information to get you started.&lt;/p&gt;&#xA;" OwnerUserId="3233" LastActivityDate="2020-01-13T14:02:56.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11130" PostTypeId="2" ParentId="11093" CreationDate="2020-01-13T14:16:45.127" Score="1" Body="&lt;p&gt;InDels are just as informative, but are less reliable to call. Errors should be averaged out in this kind of analysis though (so long as they don't predominate).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Both SNPs and InDels are valid data points to consider when comparing triads, but identifying IBD segments will only ever be probabilistic.&lt;/p&gt;&#xA;" OwnerUserId="3233" LastActivityDate="2020-01-13T14:16:45.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11131" PostTypeId="1" AcceptedAnswerId="11148" CreationDate="2020-01-13T16:53:02.027" Score="2" ViewCount="395" Body="&lt;p&gt;I would like to create a simple QC check of RNA-Seq data that simply maps the data to rRNA and then counts the number of reads that map.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've done this manually for human using sequences for 5S, 5.8S, 18S, and 28S rRNA (rDNA, actually).  I gathered those sequences manually to create a small fasta file to use as a reference and it worked well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to have a more definitive source for this reference though, and I don't necessarily want to use a species-specific database, so I thought that using a fasta file downloaded from &lt;a href=&quot;https://www.arb-silva.de/no_cache/download/archive/release_138/Exports/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SILVA&lt;/a&gt; would be great, but I don't know if I'm using the wrong fasta file or what.  I'm not very familiar with SILVA, but I noticed that when I tested it using the same sequencing data, I got unexpectedly many fewer hits than I did with my manually created 4-sequence reference than with the SILVA NR99 fasta file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I blasted my 4 rDNA sequences against the SILVA NR99 data and only the 18S was found.  The others (5S, 5.8S, and 28S) don't appear to be there.  Am I misunderstanding the nature of the SILVA project or could I be doing something wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;UPDATE&lt;/em&gt;: Please note, this question is about &lt;strong&gt;where to find rRNA and/or rDNA sequences to use as a reference&lt;/strong&gt;, not about tools that can be employed in quantifying rRNA read content.  While I appreciate hearing about tools that can generate nice reports about rRNA reads in an RNA-Seq run, the question is not answered unless it addresses the missing rRNA/rDNA sequences that are necessary for comprehensive quantification.  Any tool can count rRNA/rDNA mappings, but the results will only be as good as the rRNA reference data supplied (unless it is a reference-less tool).  And it's the lack of 5S, 18S, and 28S ribosomal subunits in SILVA that is my main concern.  It's the difference between 1k reads and 500k reads mapping to rRNA/rDNA sequences.&lt;/p&gt;&#xA;" OwnerUserId="5063" LastEditorUserId="5063" LastEditDate="2020-01-15T22:17:07.970" LastActivityDate="2020-01-16T21:02:05.700" Title="Ribosomal RNA QC quantification using SILVA" Tags="|ribosomal|" AnswerCount="3" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="11132" PostTypeId="1" AcceptedAnswerId="11136" CreationDate="2020-01-13T19:57:38.370" Score="0" ViewCount="105" Body="&lt;p&gt;Before I do this myself...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any compiled batch-corrected dataset of the major scRNA-seq atlases (Mouse Cell Atlas, Mouse Organogenesis Atlas, Mouse Gastrulation Atlas, Tabula Muris, Tabula Muris Senis, etc.) with processed spliced AND unspliced count matrices?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would be surprised if there is, but in the event there isn't, are there new pipelines in development that will make processing of raw data easier for studying RNA velocity?&lt;/p&gt;&#xA;" OwnerUserId="6498" LastActivityDate="2020-01-14T03:45:55.333" Title="Processed spliced and unspliced count matrices for existing scRNA-seq atlases" Tags="|r|python|scrnaseq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11133" PostTypeId="2" ParentId="11127" CreationDate="2020-01-13T20:56:02.023" Score="1" Body="&lt;p&gt;I think what you are looking for is GATK's Base Quality Score Recalibration.  Their &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035890531-Base-Quality-Score-Recalibration-BQSR-&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; is great so I won't rewrite it here.  &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;BQSR stands for Base Quality Score Recalibration. In a nutshell, it is&#xA;  a data pre-processing step that detects systematic errors made by the&#xA;  sequencing machine when it estimates the accuracy of each base call.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="964" LastActivityDate="2020-01-13T20:56:02.023" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11134" PostTypeId="5" CreationDate="2020-01-13T21:05:48.940" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2020-01-13T21:05:48.940" LastActivityDate="2020-01-13T21:05:48.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11135" PostTypeId="4" CreationDate="2020-01-13T21:05:48.940" Score="0" Body="pybedtools wraps and extends BEDTools and offers feature-level manipulations from within Python. " OwnerUserId="964" LastEditorUserId="964" LastEditDate="2020-01-22T14:40:32.600" LastActivityDate="2020-01-22T14:40:32.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11136" PostTypeId="2" ParentId="11132" CreationDate="2020-01-14T03:45:55.333" Score="1" Body="&lt;p&gt;I am not sure if there are preprocessed datasets available, however, &lt;a href=&quot;https://www.kallistobus.tools/velocity_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kallisto + Bustools&lt;/a&gt; was designed specifically for the rapid quantification of spliced and unspliced transcripts. &lt;/p&gt;&#xA;" OwnerUserId="64" LastActivityDate="2020-01-14T03:45:55.333" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11137" PostTypeId="2" ParentId="6804" CreationDate="2020-01-14T10:54:25.777" Score="0" Body="&lt;p&gt;Although @JamesHawley answer was informative, I don't think it solved the initial issue requested by the OP. (&lt;strong&gt;edit&lt;/strong&gt;: or maybe it partially did, I'm not sure as I got a bit confused with the wildcards on the initial question/answer. I'm anyway leaving this answer here, in case another way of explaining it is useful for others)&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Solution&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;I think you only need to change the output of your &lt;code&gt;rule multiqc&lt;/code&gt; from:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;os.path.join(qc_dir, 'multiqc_report.html')&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;to&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;os.path.join(qc_dir, '{sample}.html')&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This seemed counter-intuitive to me initially, but I think what's happening is that &lt;code&gt;snakemake&lt;/code&gt; will look at &lt;code&gt;rule all&lt;/code&gt; and see that you want an output called &quot;qc/multiqc_report.html&quot;. And the only place where it can find something of that sort is in &lt;code&gt;rule multiqc&lt;/code&gt;, by replacing the wildcard &lt;code&gt;{sample}&lt;/code&gt; by the string &lt;code&gt;multiqc_report&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It felt confusing to me, because the same &lt;code&gt;{sample}&lt;/code&gt; wildcard is used in other rules (namely the rules you use to generate the outputs in your &lt;code&gt;analysed_dir&lt;/code&gt;). But &lt;code&gt;snakemake&lt;/code&gt; will be smart to replace &lt;code&gt;{sample}&lt;/code&gt; with whatever it needs to recreate the output files you're requesting. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;h3&gt;Longer explanation&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Here is a (hopefully) reproducible example, if it helps.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First create some fake directory structure with some files:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;mkdir inputs&#xA;mkdir analysed_dir&#xA;mkdir qc_dir&#xA;&#xA;touch inputs/file1.txt&#xA;touch inputs/file2.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, this would be the snakefile:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;# Define your variables, fetch things from config, or csv, etc...&#xA;input_dir = &quot;inputs&quot;&#xA;output_dir = &quot;outputs&quot;&#xA;samples = [&quot;file1.txt&quot;, &quot;file2.txt&quot;]&#xA;&#xA;&#xA;# Rule all defining all the output files required&#xA;rule all:&#xA;  input:&#xA;    expand(&quot;analysed_dir/{samples}&quot;, samples = samples),&#xA;    &quot;qc_dir/multi_qc_report.html&quot;&#xA;&#xA;# this rule is equivalent to whichever rules process your data individually&#xA;rule analyse_each_file:&#xA;  input:&#xA;    &quot;inputs/{sample}.txt&quot;&#xA;  output:&#xA;    &quot;analysed_dir/{sample}.txt&quot;&#xA;  shell:&#xA;    &quot;cat {input} &amp;gt; {output}&quot;&#xA;&#xA;# This rule is equivalent to your multiqc step where multiple files result in a single output&#xA;rule multiqc:&#xA;  input:&#xA;    expand(&quot;analysed_dir/{samples}&quot;, samples = samples)&#xA;  output:&#xA;    &quot;qc_dir/{sample}.html&quot;&#xA;  shell:&#xA;    &quot;cat {input} &amp;gt; {output}&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to generate this pipeline:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9LnDy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9LnDy.png&quot; alt=&quot;dag&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And then issuing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;snakemake --cluster &quot;qsub -N {wildcards.sample}-{rule}&quot; --jobs 10&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generates the following job names:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;file1-analyse_each_file&lt;/li&gt;&#xA;&lt;li&gt;file2-analyse_each_file&lt;/li&gt;&#xA;&lt;li&gt;multi_qc_report-multiqc&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit with further explanation:&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Notice in the DAG how the &lt;code&gt;sample&lt;/code&gt; wildcard takes different values for the different rules. I think the snakemake logic is as follows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Starting with &lt;strong&gt;&lt;code&gt;rule all&lt;/code&gt;&lt;/strong&gt; I see that you want three files as input (the first two come from the &lt;code&gt;expand()&lt;/code&gt; call): &#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;analysed_dir/file1.txt&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;analysed_dir/file2.txt&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;qc_dir/multi_qc_report.html&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&lt;/li&gt;&#xA;&lt;li&gt;Let me start with the first one &lt;code&gt;analysed_dir/file1.txt&lt;/code&gt;. The file doesn't exist, so let me look in the other rules and see if I can find it in their output. Actually in &lt;strong&gt;&lt;code&gt;rule analyse_each_file&lt;/code&gt;&lt;/strong&gt; I see the output is set to &lt;code&gt;&quot;analysed_dir/{sample}.txt&quot;&lt;/code&gt;, so if I replace &lt;code&gt;{sample}&lt;/code&gt; by &lt;code&gt;file1.txt&lt;/code&gt; I can recreate your file from it. Therefore, the input for that rule will be &lt;code&gt;&quot;inputs/file1.txt&quot;&lt;/code&gt; (and this file already exists, so I can issue this job).&lt;/li&gt;&#xA;&lt;li&gt;For the second file &lt;code&gt;analysed_dir/file2.txt&lt;/code&gt;, well it's a similar case as with the previous one, so I'll issue a job with the same &lt;strong&gt;&lt;code&gt;rule analyse_each_file&lt;/code&gt;&lt;/strong&gt;, but this time &lt;code&gt;{sample}&lt;/code&gt; = &lt;code&gt;file2.txt&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Finally, you want a file called &lt;code&gt;qc_dir/multi_qc_report.html&lt;/code&gt;. Where can I create such a file from? Well, the output of &lt;strong&gt;&lt;code&gt;rule multiqc&lt;/code&gt;&lt;/strong&gt; is set to &lt;code&gt;&quot;qc_dir/{sample}.html&quot;&lt;/code&gt;, so if I make &lt;code&gt;{sample}&lt;/code&gt; = &lt;code&gt;multi_qc_report&lt;/code&gt;, then I can issue the job and produce your file of interest. In this case, the rule takes as input two files &lt;code&gt;&quot;analysed_dir/file1.txt&quot;&lt;/code&gt; and &lt;code&gt;&quot;analysed_dir/file2.txt&quot;&lt;/code&gt;, which I already know how to create from another rule. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The key thing to realise is that &lt;code&gt;snakemake&lt;/code&gt; &lt;strong&gt;defines the wildcard &lt;code&gt;{sample}&lt;/code&gt; for each rule separately&lt;/strong&gt;. So, in this example, &lt;code&gt;wildcard.sample&lt;/code&gt; in &lt;code&gt;rule analyse_each_file&lt;/code&gt; does not take on the same values as in &lt;code&gt;rule multiqc&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6760" LastEditorUserId="6760" LastEditDate="2020-01-16T11:23:18.193" LastActivityDate="2020-01-16T11:23:18.193" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11138" PostTypeId="2" ParentId="7456" CreationDate="2020-01-14T13:12:20.430" Score="2" Body="&lt;p&gt;You said that you want to compare with other microbial genome data. Have you already annotated your genome? If not, you might consider a pipeline I've developed, I can gladly help if you face some problems using it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is called &lt;a href=&quot;https://github.com/fmalmeida/bacannot&quot; rel=&quot;nofollow noreferrer&quot;&gt;bacannot&lt;/a&gt;, it does a complete annotation using Prokka, Kofam, CARD, VFDB, and other resources. It might help you to have a first scratch of your data since it also collapses annotation information in a genome browser&lt;/p&gt;&#xA;" OwnerUserId="6767" LastActivityDate="2020-01-14T13:12:20.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11139" PostTypeId="2" ParentId="4367" CreationDate="2020-01-14T17:02:01.170" Score="1" Body="&lt;p&gt;&lt;strong&gt;singleCellNet&lt;/strong&gt; is a computational method using supervised machine learning for &lt;strong&gt;quantitatively&lt;/strong&gt; &lt;strong&gt;cell type annotation&lt;/strong&gt;. singleCellNet also enables &lt;strong&gt;cross-platform&lt;/strong&gt; and &lt;strong&gt;cross-species&lt;/strong&gt; comparison. It is also actively-supported. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the github &lt;a href=&quot;https://github.com/pcahan1/singleCellNet&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/pcahan1/singleCellNet&lt;/a&gt;&#xA;, and the vignette &lt;a href=&quot;https://pcahan1.github.io/singleCellNet/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pcahan1.github.io/singleCellNet/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2993" LastActivityDate="2020-01-14T17:02:01.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11140" PostTypeId="2" ParentId="11115" CreationDate="2020-01-14T17:07:14.633" Score="3" Body="&lt;p&gt;I believe &lt;code&gt;seqret&lt;/code&gt; is already the simplest approach. If you install the emboss command line utility, genome (file) sizes will not be a problem. You can even install it with conda, see &lt;a href=&quot;https://anaconda.org/bioconda/emboss&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://anaconda.org/bioconda/emboss&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After that, you can use it as the example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seqret -sequence {genome file} -feature -fformat gff -fopenfile {gff file} -osformat genbank -osname_outseq {output prefix} -ofdirectory_outseq gbk_file -auto&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Hope it helps&lt;/p&gt;&#xA;" OwnerUserId="6767" LastActivityDate="2020-01-14T17:07:14.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11141" PostTypeId="1" AcceptedAnswerId="11142" CreationDate="2020-01-14T23:17:51.827" Score="3" ViewCount="700" Body="&lt;p&gt;&lt;a href=&quot;http://www.htslib.org/doc/samtools-phase.1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools phase&lt;/a&gt; is an easy-to-use tool to phase variants from a bam file and a reference.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For a sequence that has two haplotypes, like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;   position: 123456789&#xA;haplotype 0: GACCATATT&#xA;       SNPs:  **    *&#xA;haplotype 1: GTACATACT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CC  M?   chr    PS     pos  allele0 allele1 hetIndex #supports0 #errors0 #supp1 #err1&#xA;CC&#xA;CC&#xA;PS      frame-1   2    9&#xA;M1      frame-1  60    2      A       T       1       3       0       23      0&#xA;M1      frame-1  60    3      C       A       2       35      1       24      0&#xA;M1      frame-1  60    8      T       C       3       46      0       30      0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I take the reference fasta file and the output of &lt;code&gt;samtools phase&lt;/code&gt; to generate one &lt;code&gt;fasta&lt;/code&gt; file for each of the haplotypes?&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2020-01-14T23:25:04.617" LastActivityDate="2020-01-20T12:51:38.150" Title="How to generate two fasta files from samtools phase output?" Tags="|sam|fasta|samtools|phase|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11142" PostTypeId="2" ParentId="11141" CreationDate="2020-01-14T23:17:51.827" Score="3" Body="&lt;p&gt;&lt;strong&gt;Answer January 20th, 2020.&lt;/strong&gt; If you run &lt;code&gt;samtools phase&lt;/code&gt; with the option &lt;code&gt;-b my_prefix&lt;/code&gt; it outputs a bam file for both haplotypes, named &lt;code&gt;my_prefix.0.bam&lt;/code&gt; and &lt;code&gt;my_prefix.1.bam&lt;/code&gt;. If the sequence has been correctly phased, use these bam files to correct the reference using &lt;a href=&quot;https://github.com/broadinstitute/pilon&quot; rel=&quot;nofollow noreferrer&quot;&gt;pilon&lt;/a&gt;. This is preferable to the method described in the &lt;strong&gt;January 14th, 2020&lt;/strong&gt; answer because pilon will correct SNPs and indels, whereas the python program below will only correct the specific SNPs that &lt;code&gt;samtools phase&lt;/code&gt; found. &lt;code&gt;samtools phase&lt;/code&gt; does not find indels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Answer January 14th, 2020.&lt;/strong&gt; This python program below takes a fasta file as an argument and the output of samtools phase piped in, then prints out a fasta file of the two haplotypes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Run the program like this: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;samtools phase reads.bam | phase2fasta.py ref.fa &amp;gt; haplotypes.fasta&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code of &lt;code&gt;phase2fasta.py&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python&#xA;&#xA;import os&#xA;import sys&#xA;&#xA;if (len(sys.argv) == 1) or not os.path.exists(sys.argv[1]):&#xA;    raise Exception(&quot;The first and only argument is the reference fasta.&quot;)&#xA;allele0 = []&#xA;seqname = &quot;&quot;&#xA;with open(sys.argv[1]) as f:&#xA;    for line in f:&#xA;        if line[0] == &quot;&amp;gt;&quot;:&#xA;            seqname = line[1:].split(&quot; &quot;)[0].strip()&#xA;        else:&#xA;            for c in line.strip().upper():&#xA;                allele0.append(c)&#xA;allele1 = [x for x in allele0]&#xA;for line in sys.stdin:&#xA;    if line[0:2] == &quot;M1&quot;:&#xA;        fields = line.strip().split()&#xA;        if fields[1] == seqname:&#xA;            pos = int(fields[3]) - 1&#xA;            allele0[pos] = fields[4]&#xA;            allele1[pos] = fields[5]&#xA;print(&quot;&amp;gt;{}_allele0&quot;.format(seqname))&#xA;print(&quot;&quot;.join(allele0))&#xA;print(&quot;&amp;gt;{}_allele1&quot;.format(seqname))&#xA;print(&quot;&quot;.join(allele1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2020-01-20T12:51:38.150" LastActivityDate="2020-01-20T12:51:38.150" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11143" PostTypeId="1" CreationDate="2020-01-15T05:40:41.560" Score="1" ViewCount="628" Body="&lt;p&gt;I am trying to use ComplexHeatmap to plot heatmap with annotation. &#xA;The code that I  have tried &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(ComplexHeatmap)&#xA;library(circlize)&#xA;&#xA;set.seed(123)&#xA;mat = matrix(rnorm(25), 5)&#xA;rownames(mat) = paste0(&quot;R&quot;, 1:5)&#xA;colnames(mat) = paste0(&quot;C&quot;, 1:5)&#xA;&#xA;col_fun = colorRamp2(c(0, 2, 5), c(&quot;white&quot;, &quot;blue&quot;, &quot;red&quot;))&#xA;data_foo = c(0.1, 3, 1.6, 2.8, 4)&#xA;&#xA;ha = HeatmapAnnotation(which = &quot;row&quot;, foo = data_foo, col = list(foo = col_fun))&#xA;Heatmap(mat, name = &quot;mat&quot;, right_annotation = ha)&#xA;&#xA;ha1 = HeatmapAnnotation(which = &quot;row&quot;, foo = anno_text(data_foo))&#xA;Heatmap(mat, name = &quot;mat&quot;, right_annotation = ha1)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can I obtain the annotation with colors varying as per the annotation values and also display the values in the annotation? i.e is it possible to combine the annotation in both figures?&#xA;&lt;a href=&quot;https://i.stack.imgur.com/y1DSx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/y1DSx.png&quot; alt=&quot;HeatMap with color annotation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0qBYQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0qBYQ.png&quot; alt=&quot;HeatMap with values as annotation&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2247" LastEditorUserId="6437" LastEditDate="2020-01-15T15:28:32.700" LastActivityDate="2024-03-31T01:05:55.727" Title="Heatmap annotation with value and colour gradient" Tags="|r|heatmap|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11144" PostTypeId="1" AcceptedAnswerId="11145" CreationDate="2020-01-15T16:28:44.180" Score="0" ViewCount="2009" Body="&lt;p&gt;I am aware of this question &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/4249/manually-define-clusters-in-seurat-and-determine-marker-genes&quot;&gt;Manually define clusters in Seurat and determine marker genes&lt;/a&gt; that is similar but I couldn't make tit work for my use case.&lt;/p&gt;&#xA;&lt;p&gt;So I have a single cell experiments and the clustering id not great I have a small groups of 6 cells (I know it is extremely small, but nonetheless I would like to make the most of it) that are clearly isolate in UMAP and display marker that I am interresting in.&lt;/p&gt;&#xA;&lt;p&gt;I managed to get those cells names in barcode (e.g.: AGCCAGCTCGCTTTAT).&lt;/p&gt;&#xA;&lt;p&gt;I would like to compute differential expression of those cells against all others groups (using FindMarkers or FindAllMarkers).&lt;/p&gt;&#xA;&lt;p&gt;I tried to manually define the cluster in:&#xA;Seurat_obj@meta.data$seurat_clusters&#xA;But using FindMarkers (it says this groups don't exist)&lt;/p&gt;&#xA;&lt;p&gt;then I try to modify&#xA;s_19270BC@active.ident&lt;/p&gt;&#xA;&lt;p&gt;but then it could not found metadata attached to my newly define group.&lt;/p&gt;&#xA;&lt;p&gt;I am kind of stuck and any help is warmly welcome&lt;/p&gt;&#xA;&lt;h1&gt;Code&lt;/h1&gt;&#xA;&lt;p&gt;My R is quite rusty...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;names_ # a vector with the names of the cell I want as a cluster&#xA;&#xA;# my seurat object have 5 cluster 0..4. So I want to add a 6th clusters with id 5&#xA;&#xA;#First attempt manually modify the vector active.ident and cluster&#xA;# they are both factor so we need to add a factor level&#xA;levels(seurat_obj@active.ident) = c(levels(seurat_obj@active.ident), 5)&#xA;levels(seurat_obj@meta.data&lt;span class=&quot;math-container&quot;&gt;$seurat_clusters) = c(levels(seurat_obj@meta.data$&lt;/span&gt;seurat_clusters), 5)&#xA;# then modify them :&#xA;seurat_obj@active.ident[which(row.names(seurat_obj@meta.data) %in%  names_)] = 5&#xA;seurat_obj@meta.data$seurat_clusters[which(row.names(seurat_obj@meta.data) %in%  names_)] = 5&#xA;&#xA;FindMarkers(s_19270BC, ident.1 = 5)&#xA; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So This morning it works... I don't know what I have done yesterday...&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&#xA;I added the Code I used&lt;/p&gt;&#xA;" OwnerUserId="6655" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2020-02-15T09:02:45.337" Title="Seurat define manually a cluster and find markers" Tags="|r|seurat|differential-expression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11145" PostTypeId="2" ParentId="11144" CreationDate="2020-01-15T16:49:54.260" Score="2" Body="&lt;p&gt;It would have been better if you have shared your code as well as the error messages you got format these as such.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As far as I can understand, your problem is that you failed to make your cluster as default ids. For this you can use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seurat_obj &amp;lt;- SetIdent(seurat_obj, id=seurat_obj@meta.data$column_of_interest).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The cluster identities are stored in the &lt;code&gt;@meta.data&lt;/code&gt; slot of your object.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-01-15T16:49:54.260" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11148" PostTypeId="2" ParentId="11131" CreationDate="2020-01-15T19:57:30.487" Score="1" Body="&lt;p&gt;SILVA is intended for metagenomic samples and thus does not contain the human ribosomal subunits.  While the 5.8S subunit is in the ensembl, ncbi, and ucsc annotations, and there are numerous rRNA &quot;biotype&quot; (or &quot;biomol&quot;) entries, the 5S, 18S, and 28S subunit sequences are absent.  Even downloading rRNA entries in the rmsk table from ucsc will not return all 4 major human subunits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can however (as I discovered) download all rDNA gene sequences (including those 4 subunits) from the nucleotide db on ncbi using the search term &quot;biomol_rrna[PROP]&quot;.  This returns 240,240 coding sequences (or 30,775 gene sequences).  You can then use this as a reference for mapping, blasting, or as input to rRNA QC tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following e-Utilities commands from ncbi will download the gene sequences:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db nuccore -query &quot;biomol_rRNA[prop]&quot; | efetch -format fasta &amp;gt; all_rrnas.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is not a common analysis.  Usually, rRNA reads can just be ignored and analyses are unaffected, which is likely why there is a paucity of records for these sequences in annotation files, but mapping to just these sequences could make for a simple QC measure as a gauge for rRNA contamination.  While the sequences may be present in the genome, they're not in the annotations and may even be masked as repetitive, since there are so many copies.  So you have to be careful to count unique reads and not mappings.&lt;/p&gt;&#xA;" OwnerUserId="5063" LastActivityDate="2020-01-15T19:57:30.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11149" PostTypeId="2" ParentId="11131" CreationDate="2020-01-15T21:57:58.117" Score="0" Body="&lt;p&gt;I'd recommend using &lt;a href=&quot;https://ccb.jhu.edu/software/kraken2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;kraken2&lt;/a&gt; for this. There's a pre-built kraken2 database for SILVA (16S only) which can be downloaded from here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ccb.jhu.edu/software/kraken2/downloads.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ccb.jhu.edu/software/kraken2/downloads.shtml&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For other ribosomal subunits, it's necessary to create a kraken2 database from scratch. 16S/18S subunits are in the SSU database, while 23S/28S are in the LSU database:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.arb-silva.de/download/arb-files/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.arb-silva.de/download/arb-files/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that SILVA is a database of collected rRNA sequences. As far as I'm aware there's no canonical &quot;human&quot; 18S or 28S sequence in SILVA, but it includes a collection of multiple 18S &amp;amp; 28S sequences that have been annotated with the human taxon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kraken2 has a command-line interface for creating reports:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;kraken2 --db &amp;lt;location_of_database&amp;gt; --gzip-compressed input.fastq.gz --report output.kraken.report.txt &amp;gt; output.kraken.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ccb.jhu.edu/software/pavian/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pavian&lt;/a&gt; can be used for visualising the results and creating proportion tables.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-01-16T04:37:25.810" LastActivityDate="2020-01-16T04:37:25.810" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11150" PostTypeId="1" CreationDate="2020-01-15T21:58:11.953" Score="1" ViewCount="11965" Body="&lt;p&gt;I am using Seurat to analyze my single cell data. I have 2 conditions, treated and untreated.  I am trying to create a stacked bar graph in order to show the differences in cell types for each condition but need to collect the percentages of each cluster for the specific cell types. How to I put together a sheet that contains the percentages of each cluster and the make a stacked bar graph of comparison of each condition. &lt;/p&gt;&#xA;" OwnerUserId="6774" LastActivityDate="2022-01-15T14:07:15.660" Title="Percentage of each cluster in Seurat" Tags="|r|scrnaseq|seurat|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11154" PostTypeId="1" CreationDate="2020-01-16T13:29:45.850" Score="1" ViewCount="331" Body="&lt;p&gt;In Strelka vcf in INFO column we have these&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Depth=&quot;The number of reads covering the variant position including duplicates, supplementary records and reads that fall below minimum base and mapping quality thresholds.&quot;&#xA;&#xA;ReadCount=&quot;The number of reads covering the variant position excluding duplicates, supplementary records and reads that fall below minimum base and mapping quality thresholds.&quot;&amp;gt;&#xA;ReadCountControl=&quot;The number of reads covering the variant position in the control excluding duplicates, supplementary records and reads that fall below minimum base and mapping quality thresholds.&quot;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And in FORMAT column we have &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AU=&quot;Number of 'A' alleles used in tiers 1,2&quot;&amp;gt;&#xA;CU=&quot;Number of 'C' alleles used in tiers 1,2&quot;&amp;gt;&#xA;DP=&quot;Read depth for tier1 (used+filtered)&quot;&amp;gt;&#xA;GU=&quot;Number of 'G' alleles used in tiers 1,2&quot;&amp;gt;&#xA;TU=&quot;Number of 'T' alleles used in tiers 1,2&quot;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What is the difference of &lt;strong&gt;ReadCount&lt;/strong&gt; in INFO column and &lt;strong&gt;DP&lt;/strong&gt; in FORMAT column?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is the difference of &lt;strong&gt;number of allele&lt;/strong&gt; versus n&lt;strong&gt;umber of reads covering the variant position&lt;/strong&gt; ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please help for clarification if you have any intuition on this definitions&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="2776" LastEditDate="2020-04-24T18:30:42.730" LastActivityDate="2020-04-24T18:30:42.730" Title="Help with the definitions of fields in a VCF output by Strelka" Tags="|vcf|variant-calling|wgs|snv|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11155" PostTypeId="1" CreationDate="2020-01-16T15:45:02.100" Score="-1" ViewCount="38" Body="&lt;p&gt;I have a list of files in which I have this information&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Chrom   Position        Ref Alt TumorVAF    NormalVAF&#xA;chr1    56987           CA  C   0.222222    0&#xA;chr1    4133415         AG  A   0.425       0&#xA;chr1    4611030         A   AC  0.36        0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the name of one of my files&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;LP6005334-DNA_H01_vs_LP6005333-DNA_H01.passed.somatic.indels.vcf.parsed.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have another matched file which has too much information&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the name of these big files&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;LP6005334-DNA_H01_vs_LP6005333-DNA_H01.passed.somatic.indels.vcf_fixed_vcf.txt.hg19_multianno.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you are seeing, most of the name of txt files are matched&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to cut the 4th and 5th columns from my small files and add these columns to the matched big files&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Manually this is possible but time consuming for a lot of files&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is these any way to do that by coding?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="181" LastEditDate="2020-01-16T16:35:33.157" LastActivityDate="2020-01-16T17:13:04.857" Title="Appending some columns from one tabular file to another" Tags="|r|bash|loop|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11156" PostTypeId="1" AcceptedAnswerId="11174" CreationDate="2020-01-16T16:37:30.237" Score="0" ViewCount="114" Body="&lt;p&gt;I have R code takes a vcf file returns a parsed .txt output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have 21 vcf files in a folder &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I named my R code parse_vcf_alt1 function&lt;/p&gt;&#xA;" OwnerDisplayName="user3962" LastEditorDisplayName="user3962" LastEditDate="2020-01-23T11:53:51.477" LastActivityDate="2020-01-23T11:53:51.477" Title="How I can push a function to act on my files" Tags="|r|loop|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11157" PostTypeId="2" ParentId="11155" CreationDate="2020-01-16T16:56:10.367" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;As you are seeing, most of the name of txt files are matched&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Our eyes and brains are smarter than computers. We can see patterns that can be difficult to describe. For a computer to be useful in this task, the difference between the two file names must be explicitly defined.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In your example (which seems to be an ANNOVAR-generated output file with a prefix specified during annovar run), the common part seems to be the output of &lt;code&gt;$(basename LP6005334-DNA_H01_vs_LP6005333-DNA_H01.passed.somatic.indels.vcf.parsed.txt .parsed.txt)&lt;/code&gt;. If this is true for all files, you could automate that using a loop such as:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for f in *.parsed.txt&#xA;do&#xA;    other_file=&lt;span class=&quot;math-container&quot;&gt;$(basename &quot;$&lt;/span&gt;f&quot; .parsed.txt)&#xA;    # Rest of your logic goes here&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You could then &lt;code&gt;paste&lt;/code&gt; the columns from &lt;code&gt;$f&lt;/code&gt; to &lt;code&gt;$other_file&lt;/code&gt; like so (not tested):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;paste &quot;&lt;span class=&quot;math-container&quot;&gt;$other_file&quot; &amp;lt;(cut -f3,4 $&lt;/span&gt;f)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is an outline to get to your solution. You should be able to build off this.&lt;/p&gt;&#xA;" OwnerUserId="650" LastEditorUserId="298" LastEditDate="2020-01-16T17:13:04.857" LastActivityDate="2020-01-16T17:13:04.857" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11158" PostTypeId="1" CreationDate="2020-01-16T18:03:08.230" Score="2" ViewCount="251" Body="&lt;p&gt;What assembler is appropriate for High-Fidelity PacBio reads?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, canu is good for high-error PacBio reads. But what algorithm to use for HiFi reads? Would it be OK to use canu without the error-correction step?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastEditorUserId="57" LastEditDate="2020-01-17T14:46:12.283" LastActivityDate="2021-09-10T18:36:48.957" Title="What assembler is appropriate for High-Fidelity PacBio reads" Tags="|assembly|software-recommendation|long-reads|pacbio|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11159" PostTypeId="2" ParentId="11158" CreationDate="2020-01-16T18:29:03.687" Score="3" Body="&lt;p&gt;As recently &lt;a href=&quot;https://twitter.com/aphillippy/status/1191821832891949056&quot; rel=&quot;nofollow noreferrer&quot;&gt;tweeted by A. Phillippy&lt;/a&gt;, Canu v1.9 now supports HiFi reads. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore you should just read their manual and look for the implementation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;:)&lt;/p&gt;&#xA;" OwnerUserId="6767" LastEditorUserId="57" LastEditDate="2020-01-17T14:47:56.043" LastActivityDate="2020-01-17T14:47:56.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11160" PostTypeId="2" ParentId="11156" CreationDate="2020-01-16T19:50:15.470" Score="0" Body="&lt;p&gt;Why don't you try using &lt;a href=&quot;https://github.com/docopt/docopt.R&quot; rel=&quot;nofollow noreferrer&quot;&gt;docopt&lt;/a&gt; which can make your R script &quot;understand&quot;  command line parameters.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing that, you can create a simple bash loop to execute the script for all vcf files, such as:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;for files in &quot;*.vcf&quot; ; do script.R -i $files -o {output} ; done&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See this example &lt;a href=&quot;https://github.com/fmalmeida/rscripts/blob/master/parse_sql.R&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps,&#xA;Felipe.&lt;/p&gt;&#xA;" OwnerUserId="6767" LastEditorUserId="6767" LastEditDate="2020-01-16T19:56:10.693" LastActivityDate="2020-01-16T19:56:10.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11161" PostTypeId="2" ParentId="11156" CreationDate="2020-01-16T20:18:53.620" Score="0" Body="&lt;p&gt;The upper part, where you try to use &lt;code&gt;lapply()&lt;/code&gt; over your list of files with your custom function does not quite make sense to me: You declare your &lt;code&gt;files&lt;/code&gt; but you do not use this in your &lt;code&gt;lapply()&lt;/code&gt; call. I would expect to see something like `lapply(files, your_custom_function).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, the error you get is results from &lt;code&gt;dplyr::rename()&lt;/code&gt; not being able to find &lt;code&gt;#CHROM&lt;/code&gt; because &lt;code&gt;readr::read_tsv()&lt;/code&gt; could not read the header line properly. In my case the vcf was read as an n x 1 data frame! You could use &lt;code&gt;readr::read_delim()&lt;/code&gt; to make sure you are reading the data right.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would, however, would not try to read a &lt;code&gt;VCF&lt;/code&gt; file as an ordinary data frame, there are packages for that, for example &lt;code&gt;[vcfR][1]&lt;/code&gt;. Once you create your vcfR object, you can check the different slots:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The vcfR object is an S4 class object with three slots containing the&#xA;  metadata, the fixed data and the genotype data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-01-16T20:18:53.620" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11162" PostTypeId="1" AcceptedAnswerId="11180" CreationDate="2020-01-16T20:55:32.733" Score="2" ViewCount="119" Body="&lt;p&gt;What algorithms for linear Pseudotime trajectory construction (diffusion-based) are the most scalable to large datasets?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently using Slingshot based on the recommendation in this manuscript: &lt;a href=&quot;https://www.biorxiv.org/content/biorxiv/early/2018/03/05/276907.full.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/biorxiv/early/2018/03/05/276907.full.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Slingshot is nice and easy to use, but it turns out it is terrible at scaling to big datasets. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For 2000 highest variable genes and a random sampling of a variable number of cells in my ~80% sparse expression matrix I get the following computation times in R (96 GB RAM):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#cells / Time (min)&#xA;100 / 0.7&#xA;1000 / 4.3&#xA;2000 / 12.3&#xA;5000 / 71.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;At this rate it would take nearly 1.5 days to process 200,000 cells and 4.5 months to process 2 million cells based on 2,000 HVG.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel like Slingshot is limited by a fundamentally unnecessary comparison of diffusion for all cells against all cells, rather than preclustering and then finer resolution of pseudotime across cluster edges.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What algorithms are better, but still implement a similar realization of pseudotime ordering?  I just need a single linear trajectory.&lt;/p&gt;&#xA;" OwnerUserId="6498" LastActivityDate="2020-07-01T00:51:16.427" Title="Most scalable pseudotime ordering algorithm" Tags="|r|scrnaseq|statistics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11163" PostTypeId="2" ParentId="11131" CreationDate="2020-01-16T21:02:05.700" Score="0" Body="&lt;p&gt;SortMeRNA has fasta files from RFAM and Silva for 5S, 5.8S, 18S, and 28S.  You can download them from &lt;a href=&quot;https://github.com/biocore/sortmerna/tree/master/data/rRNA_databases&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6780" LastActivityDate="2020-01-16T21:02:05.700" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11164" PostTypeId="1" CreationDate="2020-01-16T23:05:27.923" Score="0" ViewCount="87" Body="&lt;p&gt;I am new to this. I have an SNP array composed of digits [0,1,2] for chromosome 21. How do I find the start:end locations for the region that is &quot;21q22.11&quot; step by step?&lt;/p&gt;&#xA;" OwnerUserId="6782" LastActivityDate="2020-01-17T08:07:17.153" Title="How to Convert From Gene Location to SNP Location" Tags="|gene|snp|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11165" PostTypeId="2" ParentId="4470" CreationDate="2020-01-17T00:37:41.667" Score="0" Body="&lt;p&gt;This updates the answer provided by Mack123456 to Seurat 3 and works in 3.1.2. There are some minor difference with colouring, but it is otherwise similar. The HoverLocator plot throws a warning, all about the plot colours and title, but the plot is fine and labels appear as expected.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;load(file= &quot;~/seuset_16.RData&quot;)&#xA;&#xA;seuset_h16 &amp;lt;- UpdateSeuratObject(seuset_h16)&#xA;&#xA;high &amp;lt;- c(&quot;s1.1&quot;,&quot;s1.4&quot;, &quot;s1.80&quot;)&#xA;low &amp;lt;- setdiff(rownames(seuset_h16@meta.data), high)&#xA;Idents(seuset_h16, cells = high) &amp;lt;- &quot;high&quot;&#xA;Idents(seuset_h16, cells = low) &amp;lt;- &quot;low&quot;&#xA;seuset_h16$expr &amp;lt;- Idents(seuset_h16)&#xA;Idents(object = seuset_h16) &amp;lt;- &quot;res.1&quot;&#xA;&#xA;t &amp;lt;- DimPlot(seuset_h16, pt.size = 2, shape.by = &quot;expr&quot;)&#xA;t&#xA;&#xA;t2 &amp;lt;- DimPlot(seuset_h16, pt.size = 2, cols = c(&quot;red&quot;, &quot;orange&quot;,&quot;green&quot;,&quot;blue&quot;))&#xA;t2 &amp;lt;- HoverLocator(plot = t2, information = FetchData(object = seuset_h16, vars = c(&quot;ident&quot;, &quot;res.1&quot;)))&#xA;t2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6783" LastActivityDate="2020-01-17T00:37:41.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11166" PostTypeId="1" CreationDate="2020-01-17T05:45:21.197" Score="0" ViewCount="70" Body="&lt;p&gt;Trying to download colorspace data from SRA, but getting an error  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;abi-dump -A SRR1657115.sra&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;abi-dump.2.9.6 err: item not found while constructing within virtual database module - the path 'SRR1657115.sra' cannot be opened as database or table.&lt;/p&gt;&#xA;" OwnerUserId="4121" LastActivityDate="2020-02-18T02:01:45.187" Title="Downloading dataset from SRA (SOLiD Platform)" Tags="|ngs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11167" PostTypeId="2" ParentId="11164" CreationDate="2020-01-17T08:07:17.153" Score="2" Body="&lt;p&gt;You first need to get the coordinates of the &quot;cytoband&quot; 21q22.11, which I would search for using the &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTables?hgsid=791314831_aE2GlksiE79iyiT0FHCRXDZcFwPC&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC table browser&lt;/a&gt;. You need the following settings: &#xA;Clade: Mammal&#xA;genome: Human&#xA;assembly: &#xA;group: Mapping and Sequencing&#xA;track: Chromosome Band&#xA;table: cytoBand&#xA;region: position chr21&#xA;output format: BED&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(you can leave other options as default)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That will give you a bed file of cytobands on chr21. You'll need to search (e.g. grep) for your cytoband: q22.11.&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2020-01-17T08:07:17.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11168" PostTypeId="2" ParentId="11158" CreationDate="2020-01-17T08:08:41.390" Score="2" Body="&lt;p&gt;From the group of Heng Li there is &lt;a href=&quot;https://github.com/chhylp123/hifiasm&quot; rel=&quot;nofollow noreferrer&quot;&gt;hifiasm&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2020-01-17T08:08:41.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11170" PostTypeId="1" CreationDate="2020-01-17T14:37:10.567" Score="2" ViewCount="1381" Body="&lt;p&gt;I know that I don't have the correct format in my .ped file for PLINK but I don't know how to convert it.&#xA;My actual file is something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BSCAN BSCAN20016 0 0 2 0 0 1 1 1....&#xA;BSCAN BSCAN20017 0 0 2 0 0 1 1 1....&#xA;BSCAN BSCAN20018 0 0 2 0 0 1 1 1....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where column 1 is FamilyID, column 2 is ID, column 3 is FatherID, column 4 MotherID, column 5 Sex and then the genotype of each individual.&#xA;I only need to create binary files, so I tried to run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plink-1.9-rc --file admixed --make-bed --no-pheno --cow --out admixed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but the program shows: Error: half-missing call in .ped file at variant 1, line 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyone know how can I change the format or recode it to make a correct input?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="6786" LastActivityDate="2023-05-31T23:05:30.670" Title="Wrong format .ped in PLINK" Tags="|file-formats|plink|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11171" PostTypeId="2" ParentId="11170" CreationDate="2020-01-17T14:44:03.237" Score="1" Body="&lt;p&gt;So, columns 7 and 8 should be the genotype of variant 1 (each column is one of the alleles). You have them as 0 1, which means absent and major allele.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think you're just missing a column. From the doc:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Column 6 is Phenotype value ('1' = control, '2' = case, '-9'/'0'/non-numeric = missing data if case/control)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally you would know which SNPs are heterozygous based on your SNP calling and have such SNPs as 1 2, but if you have a homozygous organism or if you're working with data that's been flattened you can simply generate the .ped file with the same number twice for each SNP.&lt;/p&gt;&#xA;" OwnerUserId="6771" LastEditorUserId="6771" LastEditDate="2020-01-17T14:54:12.167" LastActivityDate="2020-01-17T14:54:12.167" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="11172" PostTypeId="2" ParentId="7371" CreationDate="2020-01-17T14:45:05.060" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://busco.ezlab.org/busco_userguide.html#conda-package&quot; rel=&quot;nofollow noreferrer&quot;&gt;The new version of busco (v4)&lt;/a&gt; has the user entry point called &lt;code&gt;busco&lt;/code&gt; (it's still a python script). BUSCO v4 is &lt;a href=&quot;https://anaconda.org/bioconda/busco&quot; rel=&quot;nofollow noreferrer&quot;&gt;available at conda&lt;/a&gt;, note it's a very fresh update.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2020-01-17T14:45:05.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11174" PostTypeId="2" ParentId="11156" CreationDate="2020-01-17T15:27:51.177" Score="0" Body="&lt;p&gt;By adding this to the script &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;args=(commandArgs(TRUE))&#xA;&#xA;file_list &amp;lt;- list.files(path=&quot;/path/to/vcf/files/&quot;)&#xA;for (i in 1:length(file_list))&#xA;{&#xA;  x=file_list[i]&#xA;  fn=paste('test',x,sep = &quot;&quot;)&#xA;  parse_vcf_alt1(x,fn)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But Also adding &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df &amp;lt;- apply(vcf,2,as.character)&#xA;  fn=paste(&quot;C:/Users/44736/Videos/Captures/New folder/&quot;,fn,sep=&quot;&quot;)&#xA;  write.csv(df,fn)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;to the main function I could get this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem is I don't know how to have a tab delimited file rather than csv also removing quote from the written output&lt;/p&gt;&#xA;" OwnerDisplayName="user3962" LastActivityDate="2020-01-17T15:27:51.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11175" PostTypeId="1" AcceptedAnswerId="11178" CreationDate="2020-01-17T17:03:44.400" Score="1" ViewCount="121" Body="&lt;p&gt;I need to find peptides/proteins with a certain sequence of amino acids in a certain type of secondary structure. For example, &quot;ALA PHE GLY&quot; in alpha-helix. Is there any tool that does that?&lt;/p&gt;&#xA;" OwnerUserId="6483" LastActivityDate="2020-01-28T17:04:46.340" Title="Searching for proteins by primary and secondary structure" Tags="|protein-structure|secondary-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11178" PostTypeId="2" ParentId="11175" CreationDate="2020-01-18T10:11:58.690" Score="3" Body="&lt;p&gt;Your request is rather specific, so there is no tool...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is a straightforwards request though. I'd impose the requirement that the structures be solved and no predicted secondary structure. You can do it two ways if using Python. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;PDB&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;After downloading all &lt;a href=&quot;ftp://ftp.wwpdb.org/pub/pdb/data/structures/all/pdb/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB structures&lt;/a&gt;, in python with PyMOL (installable with conda, tricky without) you'd do something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pymol2, os&#xA;from multiprocessing import Pool&#xA;&#xA;def find_motif(file):&#xA;    motif = 'AFG'&#xA;    try:&#xA;        with pymol2.PyMOL() as pymol&#xA;            pymol.cmd.load(file)&#xA;            seq = pymol.cmd.get_fastastr()&#xA;            if motif not in seq:&#xA;                return None&#xA;            d = {x: []}&#xA;            # str.find will cause issues if there is a gap also structure may not start at one.&#xA;            pymol.cmd.iterate('resn ALA+PHE+GLY', 'x.append((resi, resn, ss))', space=d)&#xA;            sequence = -1&#xA;            i = 0&#xA;            for residue in d['x']:&#xA;                if residue['resn'] = 'ALA':&#xA;                    sequence = 0 # lazy enum&#xA;                    i = int(residue['resi'])&#xA;                elif sequence != -1 :&#xA;                    if int(residue['resi']) == i+1 and residue['resn'] = 'PHE':&#xA;                        sequence = 1&#xA;                    elif int(residue['resi']) == i+2 and residue['resn'] = 'GLY':&#xA;                        return (file, i)&#xA;                    else:&#xA;                        sequence = -1&#xA;    finally:&#xA;        return (file, None)&#xA;&#xA;if __name__ == '__main__':&#xA;    p = Pool(12)&#xA;    structure_files = [f for f in os.listdir('pdb') if os.path.splitext(f)[1] == '.pdb']&#xA;    verdicts = p.map(finder, structure_files)&#xA;    print([(f,v) for f,v in verdicts if v is not None])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The end result will be a list of PDB with where in the sequence (in that PDB code).&#xA;(I wrote this in the reply box, so there may be a typo or two).&#xA;PyMOL has a selection algebra called &lt;code&gt;pepseq&lt;/code&gt;, which matches missing atoms, but I was a whimp and played it safe by being more verbose. Also, a for loop can be used instead of an iterate. But I think it is slower.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;atoms = pymol.cmd.get_model(f&quot;pepseq {target} and name CA&quot;)&#xA;for atom in atoms.atom:&#xA;      if atom.ss == ' ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;Uniprot&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In Uniprot, if a protein is crystallised, there will be the feature entries corresponding to the secondary structure. It is sheet or helix. No turns, pi-helices or 3_10 helices. There is no Biopython method for parsing Uniprot XML. I wrote myself a parser and it is not too horrid thing to do —although I &lt;a href=&quot;https://blog.matteoferla.com/2019/02/uniprot-xml-and-python-elementtree.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;monkeypatched elementtree&lt;/a&gt;. Alternatively, &lt;code&gt;xmlschema.XMLSchema.to_dict&lt;/code&gt; can be used to convert the XML to a dictionary. The advantage of Uniprot is that the residues ids will not require correcting. The disadvantage is parsing XML.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My parser is specific for my application and it is poorly documented (so would not recommend it) but but... I can share &lt;a href=&quot;https://github.com/matteoferla/MichelaNGLo-protein-module&quot; rel=&quot;nofollow noreferrer&quot;&gt;it&lt;/a&gt; and &lt;a href=&quot;https://github.com/matteoferla/MichelaNGLo-human-protein-data&quot; rel=&quot;nofollow noreferrer&quot;&gt;human data&lt;/a&gt;. I'd be used like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; import os&#xA; from michelanglo_protein import ProteinCore&#xA; ProteinCore.settings.startup(data_folder='protein-data')&#xA; p = ProteinCore().gload(file=os.path.join('gpickle','Q86V25.gpz'))&#xA; #print(p.asdict())&#xA; if 'helix' in p.features:&#xA;        n = p.sequence.find( ...&#xA;        for ... p.features['helix'] #('helix', 'turn', 'strand')&#xA;        if x &amp;lt; n &amp;lt; y ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-01-28T17:04:46.340" LastActivityDate="2020-01-28T17:04:46.340" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11179" PostTypeId="2" ParentId="11166" CreationDate="2020-01-19T01:17:42.297" Score="1" Body="&lt;p&gt;You have to use the accession name without the &lt;code&gt;.sra&lt;/code&gt; suffix.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;abi-dump -A SRR1657115&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-01-19T01:17:42.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11180" PostTypeId="2" ParentId="11162" CreationDate="2020-01-19T14:24:33.303" Score="1" Body="&lt;p&gt;As it turns out, pseudotime algorithms are exceedingly difficult to parallelize. Pseudotime does not scale linearly to a subset of data compared to the full dataset, and as such a course-grained approach to parallelizing pseudotime calculations on random bins of the data, followed by bin merging, does not work. These values will never converge on the actual full-dataset solution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I put in a pretty good effort trying to parallelize slingshot, but with no success. I realize my code may be of limited utility without more explanation, but I'm not putting more time into this. FWIW:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This code:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Finds computationally optimal bin size for slingshot based on a number of genes and features&lt;/li&gt;&#xA;&lt;li&gt;Randomly splits the SingleCellExperiment object into that number of bins and runs slingshot on each bin&lt;/li&gt;&#xA;&lt;li&gt;Orients each bin so that a cell with a pseudotime of 1 in the first bin is similar to a pseudotime of 1 in every bin. Pseudotime can get flipped depending on the sampled subset.&lt;/li&gt;&#xA;&lt;li&gt;Merges all bins into a single matrix so each cell has a assigned pseudotime based on randomly sampled bins.&lt;/li&gt;&#xA;&lt;li&gt;Repeats steps #2 to #5 with a different random seed for binning every time.&lt;/li&gt;&#xA;&lt;li&gt;Measures convergence towards actual solution at three pseudotime values, where convergence is defined as correlation of cell indices along the trajectory.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I abandoned the project after observing that more iterations to not change the convergence towards the actual solution. While pseudotime values close to 0 recapitulated the actual solution quite well, pseudotime values towards 100 failed to correlate at all with the actual solution. There was no convergence with more iterations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe this will be helpful to somebody, which is why I'm posting.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Parallel Iterative Sampling with Slingshot&#xA;# pissshot is a course-grained estimator of the actual slingshot solution, and iteratively converges towards the actual solution&#xA;# pissshot randomly partitions cells in a large database into bins, bin size is determined by finding the computationally optimal number of cells&#xA;# pissshot is run independently on each bin, pseudotime vectors are aligned across all bins, and then all bins are merged&#xA;# pissshot repeats this process and the average of pseudotime assignments for each cell is taken at the end of each iteration. &#xA;# The convergence of the blingshot model at the end of each iteration is calculated by running slingshot on a small sample of adjacent cells at a random pseudotime interval and measuring correlation with the predicted model&#xA;# When satisfactory convergence is reached, blingshot returns a singlecellobject, just as slingshot would&#xA;&#xA;pissshot &amp;lt;- function(sce, num_iterations = 20){&#xA;    cat(&quot;Running blingshot on&quot;, dim(sce)[2],&quot;cells and&quot;,dim(sce)[1],&quot;features\n&quot;)&#xA;    cat(&quot;  Step 1/5: Finding computationally optimal increment sizes\n   &quot;)&#xA;#    increment_size &amp;lt;- FindOptimalIncrementSize(sce)&#xA;    increment_size &amp;lt;- 688&#xA;    num_increments &amp;lt;- ceiling(dim(sce)[2]/increment_size)-1&#xA;    cat(&quot;\n  ...Increment size of&quot;,increment_size,&quot;cells is computationally optimal\n&quot;)&#xA;    cat(&quot;  Step 2/5: Course-grained pseudotime assignment across&quot;,num_increments,&quot;increments&quot;)&#xA;    ps &amp;lt;- list()&#xA;    convergenceArr &amp;lt;- c()&#xA;    for(iteration in seq(from=1,to=num_iterations,by=1)){&#xA;        cat(&quot;\n   ... iteration&quot;,iteration,&quot;...\n&quot;)&#xA;        cat(&quot;         randomly assigning cells to&quot;,num_increments,&quot;bins\n         0%.&quot;)&#xA;        bins &amp;lt;- PartitionBins(sce, increment_size = increment_size, seed = 7*iteration)&#xA;        cat(&quot;100%\n         running slingshot on each bin\n         0%&quot;)&#xA;        cell_pseudotimes &amp;lt;- c()&#xA;        for(b in seq(from=1,to=length(bins),by=1)){&#xA;            bin_with_pseudotime &amp;lt;- suppressMessages(slingshot(bins[[b]]))&#xA;            # Find right cell and left cell in pseudotime array (cell with highest/lowest pseudotime values)&#xA;            pcells &amp;lt;- as.data.frame(colData(bin_with_pseudotime)&lt;span class=&quot;math-container&quot;&gt;$slingPseudotime_1)&#xA;            rownames(pcells) &amp;lt;- colnames(bin_with_pseudotime)&#xA;            pcells &amp;lt;- data.frame(lapply(pcells, function(x) as.numeric(as.character(x))), check.names = F, row.names = rownames(pcells))&#xA;            pcells.sorted &amp;lt;- pcells[order(-pcells[,1]), , drop = FALSE]&#xA;            right_cell_ID &amp;lt;- rownames(pcells.sorted)[dim(pcells.sorted)[1]]&#xA;            left_cell_ID &amp;lt;- rownames(pcells.sorted)[1]&#xA;            right_cell_pos &amp;lt;- match(right_cell_ID,colnames(bins[[b]]))&#xA;            left_cell_pos &amp;lt;- match(left_cell_ID,colnames(bins[[b]]))&#xA;            right_cell_logcounts &amp;lt;- assays(bins[[b]])$&lt;/span&gt;logcounts[,right_cell_pos]&#xA;            left_cell_logcounts &amp;lt;- assays(bins[[b]])&lt;span class=&quot;math-container&quot;&gt;$logcounts[,left_cell_pos]&#xA;            if(b==1&amp;amp;&amp;amp;iteration==1){&#xA;                ref_right_cell_logcounts &amp;lt;- right_cell_logcounts&#xA;                ref_left_cell_logcounts &amp;lt;- left_cell_logcounts&#xA;            }&#xA;            # Figure out whether this pseudotime in this bin has flipped relative to reference (first bin), and if so, flip pseudotime values&#xA;            cor_opposite_ends &amp;lt;- mean(cor(right_cell_logcounts,ref_left_cell_logcounts),cor(left_cell_logcounts,ref_right_cell_logcounts))&#xA;            cor_same_ends &amp;lt;- mean(cor(right_cell_logcounts,ref_right_cell_logcounts),cor(left_cell_logcounts,ref_left_cell_logcounts))&#xA;            flipped &amp;lt;- FALSE&#xA;            if(cor_opposite_ends &amp;gt; cor_same_ends){&#xA;                flipped &amp;lt;- TRUE&#xA;            }&#xA;            if(flipped==TRUE){&#xA;                # find max pseudotime in colData(bin_with_pseudotime)$slingPseudotime&#xA;                # Recalculate pseudotime in bin_with_pseudotime as max-value&#xA;                pseudotimes &amp;lt;- colData(bin_with_pseudotime)$slingPseudotime_1&#xA;                colData(bin_with_pseudotime)$slingPseudotime_1 &amp;lt;- max(pseudotimes)-pseudotimes &#xA;            }&#xA;            cells_with_pseudotime &amp;lt;- as.data.frame(colData(bin_with_pseudotime)$&lt;/span&gt;slingPseudotime_1,colnames(bin_with_pseudotime))&#xA;            cell_pseudotimes &amp;lt;- rbind(cell_pseudotimes, cells_with_pseudotime)&#xA;            cat(&quot;.&quot;)&#xA;        }&#xA;        cat(&quot;100%\n&quot;)&#xA;&#xA;        ps[[iteration]] &amp;lt;- cell_pseudotimes[order(row.names(cell_pseudotimes)), , drop = FALSE]&#xA;&#xA;        pss &amp;lt;- data.frame(ps)&#xA;        if(iteration&amp;gt;1){&#xA;            convergence10 &amp;lt;- measureConvergence(pss, sce = sce, ptime = 10, increment_size = increment_size)&#xA;            convergence50 &amp;lt;- measureConvergence(pss, sce = sce, ptime = 50, increment_size = increment_size)&#xA;            convergence99 &amp;lt;- measureConvergence(pss, sce = sce, ptime = 99, increment_size = increment_size)&#xA;            cat(&quot;   convergence at 10:&quot;,convergence10,&quot;... at 50:&quot;,convergence50,&quot;...at 100:&quot;,convergence99,&quot;\n&quot;)&#xA;            convergenceArr &amp;lt;- rbind(convergenceArr,c(iteration,convergence10,convergence50,convergence99))&#xA;        }&#xA;        # Iterate until the model converges towards the actual slingshot solution, within an indicated fraction (i.e. convergence = 0.02)&#xA;        # The convergence measure compares how similar the model is to the actual slingshot solution, i.e. a convergence of 0.02 means that cell pseudotime values are within 2% of the actual solution&#xA;        # Assess this by selecting a pseudotime at random, analyzing the following increment_size cells, and comparing accuracy of pseudotime measurement across that trajectory with the composite of the course-grained analysis&#xA;&#xA;        # need to study how convergence changes over pseudotime&#xA;    }&#xA;&#xA;    # remove cells with really high standard deviations&#xA;&#xA;    return(pss)&#xA;#    return(convergenceArr)&#xA;}&#xA;&#xA;&#xA;measureConvergence &amp;lt;- function(mat, sce = sce, ptime = 45, increment_size = 500){&#xA;    # this line is throwing an error&#xA;    mat[,&quot;avg&quot;] &amp;lt;- apply(mat[,1:dim(mat)[2]],1,mean)&#xA;    mat &amp;lt;- mat[order(mat[,&quot;avg&quot;]), , drop = FALSE]&#xA;    # select a random pseudotime increment&#xA;    mat &amp;lt;- mat[mat[, &quot;avg&quot;] &amp;gt; ptime,]&#xA;    mat &amp;lt;- mat[1:increment_size,]&#xA;    cell_IDs &amp;lt;- rownames(mat)&#xA;    # pull out this list of cell_IDs from the sce object and run slingshot&#xA;    sce2 &amp;lt;- suppressMessages(slingshot(sce[,cell_IDs]))&#xA;    cpcells &amp;lt;- as.data.frame(colData(sce2)$slingPseudotime_1)&#xA;    rownames(cpcells) &amp;lt;- colnames(sce2)&#xA;    cpcells &amp;lt;- data.frame(lapply(cpcells, function(x) as.numeric(as.character(x))), check.names = F, row.names = rownames(cpcells))&#xA;    # sort cpcells by increasing pseudotime&#xA;    cpcells &amp;lt;- cpcells[order(cpcells[,1]), , drop = FALSE]&#xA;    ordered_cells &amp;lt;- cbind(rownames(mat),rownames(cpcells))&#xA;    # get the index of each cell in the dataframe, run a correlation on how well the indices line up&#xA;    cell_indices &amp;lt;- c()&#xA;    for(cell in cell_IDs){&#xA;        new_indices &amp;lt;- c(which(ordered_cells[,1] == cell), which(ordered_cells[,2] == cell))&#xA;        cell_indices &amp;lt;- rbind(cell_indices,new_indices)&#xA;    }&#xA;    convergence_measure &amp;lt;- cor(cell_indices[,1],cell_indices[,2])&#xA;    plot(cell_indices[,1],cell_indices[,2])&#xA;    return(convergence_measure)&#xA;}&#xA;&#xA;PartitionBins &amp;lt;- function(sce, increment_size = 500, seed = 123){&#xA;    bins &amp;lt;- c()&#xA;    while(dim(sce)[2]&amp;gt;=increment_size){&#xA;        if(dim(sce)[2]&amp;lt;increment_size*2){&#xA;            bins &amp;lt;- c(bins, sce)&#xA;            sce &amp;lt;- sce[,1]&#xA;        }&#xA;        else{&#xA;            set.seed(seed)&#xA;            train_ind &amp;lt;- sample(seq_len(ncol(sce)), size = increment_size)&#xA;            bins &amp;lt;- c(bins, sce[,train_ind])&#xA;            sce &amp;lt;- sce[,-train_ind]&#xA;            cat(&quot;.&quot;)&#xA;        }&#xA;    }&#xA;    return(bins)&#xA;}&#xA;&#xA;FindOptimalIncrementSize &amp;lt;- function(sce){&#xA;    # Find the computationally optimal increment_size for grainy alignments using 5 increment points (100, 500, 1000, 2500)&#xA;    res &amp;lt;- c(100,250,500,750,1000,1500,2500,5000)&#xA;    runtimes &amp;lt;- c()&#xA;    for(i in res){&#xA;        runtimes &amp;lt;- c(runtimes, suppressMessages(system.time(slingshot(sce[,0:i])))[&quot;elapsed&quot;])&#xA;    }&#xA;    mat &amp;lt;- as.data.frame(t(rbind(res,runtimes)))&#xA;    c &amp;lt;- summary(lm(mat&lt;span class=&quot;math-container&quot;&gt;$runtimes ~ poly(mat$&lt;/span&gt;res, 2, raw=TRUE)))$coefficients[,&quot;Estimate&quot;]&#xA;    predicted_runtimes &amp;lt;- c()&#xA;    for(i in seq(from=100, to=5000, by=1)){&#xA;        predicted_runtime &amp;lt;- (c[1] + c[2]*i + c[3]*i^2)*(dim(sce)[2]/i)&#xA;        predicted_runtimes &amp;lt;- rbind(predicted_runtimes,c(i,predicted_runtime))&#xA;        cat(&quot;.&quot;)&#xA;    }&#xA;    increment_size &amp;lt;- predicted_runtimes[which.min(predicted_runtimes[,2]),1]&#xA;    return(increment_size)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6498" LastActivityDate="2020-01-19T14:24:33.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11181" PostTypeId="1" AcceptedAnswerId="11182" CreationDate="2020-01-19T14:59:30.530" Score="-2" ViewCount="43" Body="&lt;p&gt;I have a big data frame&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dim(anno_maf)&#xA;[1] 449892    132&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In chromosome column I have chr 1 to 22 and another contigs like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; unique(anno_maf$Chromosome)&#xA; [1] &quot;chr1&quot;                  &quot;chr6&quot;                 &#xA; [3] &quot;chr16&quot;                 &quot;chr17&quot;                &#xA; [5] &quot;chr20&quot;                 &quot;chr2&quot;                 &#xA; [7] &quot;chr3&quot;                  &quot;chr4&quot;                 &#xA; [9] &quot;chr14&quot;                 &quot;chr19&quot;                &#xA;[11] &quot;chr5&quot;                  &quot;chr10&quot;                &#xA;[13] &quot;chr9&quot;                  &quot;chr12&quot;                &#xA;[15] &quot;chr13&quot;                 &quot;chr11&quot;                &#xA;[17] &quot;chr22&quot;                 &quot;chr7&quot;                 &#xA;[19] &quot;chr15&quot;                 &quot;chr8&quot;                 &#xA;[21] &quot;chr18&quot;                 &quot;chr21&quot;                &#xA;[23] &quot;chrX&quot;                  &quot;chrY&quot;                 &#xA;[25] &quot;chr4_gl000194_random&quot;  &quot;chr17_gl000205_random&quot;&#xA;[27] &quot;chrUn_gl000241&quot;        &quot;hs37d5&quot;               &#xA;[29] &quot;chrUn_gl000219&quot;        &quot;chrUn_gl000234&quot;       &#xA;[31] &quot;chr1_gl000191_random&quot;  &quot;chrUn_gl000211&quot;       &#xA;[33] &quot;chrUn_gl000224&quot;        &quot;chrUn_gl000225&quot;       &#xA;[35] &quot;chr17_gl000203_random&quot; &quot;chrUn_gl000212&quot;       &#xA;[37] &quot;chrUn_gl000243&quot;        &quot;chrUn_gl000214&quot;       &#xA;[39] &quot;chrM&quot;                  &quot;chr1_gl000192_random&quot; &#xA;[41] &quot;chr7_gl000195_random&quot;  &quot;chrUn_gl000232&quot;       &#xA;[43] &quot;chr4_gl000193_random&quot;  &quot;chr19_gl000208_random&quot;&#xA;[45] &quot;chrUn_gl000226&quot;        &quot;chrUn_gl000218&quot;       &#xA;[47] &quot;chr9_gl000199_random&quot;  &quot;chrUn_gl000217&quot;       &#xA;[49] &quot;chrUn_gl000229&quot;        &quot;chrUn_gl000216&quot;       &#xA;[51] &quot;chrUn_gl000231&quot;        &quot;chr9_gl000198_random&quot; &#xA;[53] &quot;chr17_gl000204_random&quot; &quot;chrUn_gl000220&quot;       &#xA;[55] &quot;chrUn_gl000235&quot;        &quot;chr11_gl000202_random&quot;&#xA;[57] &quot;chrUn_gl000222&quot;        &quot;chrUn_gl000240&quot;       &#xA;[59] &quot;chrUn_gl000233&quot;        &quot;chrUn_gl000230&quot;       &#xA;[61] &quot;chrUn_gl000213&quot;        &quot;chrUn_gl000238&quot;       &#xA;[63] &quot;chr19_gl000209_random&quot; &quot;chrUn_gl000237&quot;       &#xA;[65] &quot;#CHROM&quot; &#xA;&#xA;  How I can only keep rows having chr1 to chr22 and removing the rest?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2020-01-19T15:06:35.853" Title="Removing rows containing some strings" Tags="|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11182" PostTypeId="2" ParentId="11181" CreationDate="2020-01-19T15:06:35.853" Score="1" Body="&lt;p&gt;You can create a vector consisting of values that you would like to keep (or discard):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;to_keep &amp;lt;- paste0(&quot;chr&quot;, seq(1,22))&#xA;&#xA;&amp;gt; to_keep&#xA; [1] &quot;chr1&quot;  &quot;chr2&quot;  &quot;chr3&quot;  &quot;chr4&quot;  &quot;chr5&quot;  &quot;chr6&quot;  &quot;chr7&quot;  &quot;chr8&quot;  &quot;chr9&quot;  &quot;chr10&quot; &quot;chr11&quot;&#xA;[12] &quot;chr12&quot; &quot;chr13&quot; &quot;chr14&quot; &quot;chr15&quot; &quot;chr16&quot; &quot;chr17&quot; &quot;chr18&quot; &quot;chr19&quot; &quot;chr20&quot; &quot;chr21&quot; &quot;chr22&quot;&#xA;&#xA;your_df[your_df$Chromosome %in% to_keep,]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;data.table&lt;/code&gt; way, which should be faster for large data, would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;your_df[Chromosome %in% to_keep]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't know what your goal is but you might want to keep chromosomes X and Y as well.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-01-19T15:06:35.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11183" PostTypeId="1" AcceptedAnswerId="11185" CreationDate="2020-01-19T21:38:29.317" Score="-5" ViewCount="55" Body="&lt;p&gt;I have a data frame like with more samples like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(a[1:2,1:4])&#xA;             ID               ids pre.or.post.treatment&#xA;1 OCCAMS/AH/120 LP6005409-DNA_F01                   pre&#xA;2 OCCAMS/AH/126 LP2000325-DNA_A01                   pre&#xA;            TRG&#xA;1 non-responder&#xA;2 non-responder&#xA;&amp;gt; dim(a)&#xA;[1] 160   4&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Another data frame with part of my samples but more characteristics for each sample like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dim(b)&#xA;[1] 60 50&#xA;&#xA;&amp;gt; head(b[1:2,1:6])&#xA;             ID       TRG pre.or.post.treatment          ID.1&#xA;1 OCCAMS/AH/212 responder                   pre OCCAMS/AH/212&#xA;2 OCCAMS/AH/279 responder                   pre OCCAMS/AH/279&#xA;  DiagnosisDate FE.LastSeenDate&#xA;1      8/8/2013        6/6/2014&#xA;2     6/10/2014       11/8/2018&#xA;&amp;gt;  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How I can  merge these data frame getting a new data frame with my all 160 samples columns of both a and b&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2020-01-20T04:18:49.043" Title="Merging two data frames" Tags="|r|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11184" PostTypeId="2" ParentId="2817" CreationDate="2020-01-19T23:17:45.590" Score="1" Body="&lt;p&gt;If you have a very big fasta file(60 GB) why use SeqIO.parse and write the output file using biopython?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why not use SimpleFastaParser instead (you get a tuple(name, seq)) and you dont need to wait so long as using the parser and using Seqrecords.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What do you think about it guys?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Paulo&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kind of:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.SeqIO.FastaIO import SimpleFastaParser&#xA;from collections import defaultdict&#xA;&#xA;dedup_records = defaultdict(list)&#xA;for (name, seq) in SimpleFastaParser(filename):&#xA;    # Use the sequence as the key and then have a list of id's as the value&#xA;    dedup_records[seq)].append(name)&#xA;with open(filename, 'w') as output:&#xA;    for seq, ids in dedup_records.items():&#xA;        # Join the ids and write them out as the fasta&#xA;        output.write(&quot;&amp;gt;{}\n&quot;.format('|'.join(ids)))&#xA;        output.write(seq + &quot;\n&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4222" LastActivityDate="2020-01-19T23:17:45.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11185" PostTypeId="2" ParentId="11183" CreationDate="2020-01-20T03:11:31.550" Score="2" Body="&lt;p&gt;Assuming your column ID in both dataframes &lt;code&gt;a&lt;/code&gt; and &lt;code&gt;b&lt;/code&gt; shared same IDs, and as your dataframe are not particularly heavy, you can have the use of the function &lt;code&gt;left_join&lt;/code&gt; from &lt;code&gt;dplyr&lt;/code&gt; package as this:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(dplyr)&#xA;DF &amp;lt;- left_join(a,b, by = &quot;ID&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6437" LastActivityDate="2020-01-20T03:11:31.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11186" PostTypeId="2" ParentId="11183" CreationDate="2020-01-20T04:18:49.043" Score="3" Body="&lt;p&gt;@dc37's dplyr solution is quite elegant, but you can also use base R's merge:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;merge(x = df_a, y = df_b, by = &quot;ID&quot;, all.x = TRUE, all.y = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can tweak various parameters, such as use &lt;code&gt;by.x&lt;/code&gt; and &lt;code&gt;by.y&lt;/code&gt; instead of just &lt;code&gt;by&lt;/code&gt;, change &lt;code&gt;all.x&lt;/code&gt; and &lt;code&gt;all.y&lt;/code&gt; based on requirements to get your results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a pretty common R question by the way, and if you had googled &quot;merging data frames&quot;, you'd see a lot of options.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2020-01-20T04:18:49.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11187" PostTypeId="1" CreationDate="2020-01-20T09:16:13.010" Score="0" ViewCount="72" Body="&lt;p&gt;working on lung cancer project and need to find out which genes are deferentially expressed and which are not. for that i a have downloaded miRNA data set from GEO id is GSE110907. And the file i got has to variable gene_id and read counts. from that i have to derive p value from that data/file.&lt;a href=&quot;https://i.stack.imgur.com/osaNe.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/osaNe.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user3659" LastActivityDate="2020-01-20T09:37:12.140" Title="how can i calculate p value of miRNA raw reads counts using R?" Tags="|r|p-values|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11188" PostTypeId="2" ParentId="11187" CreationDate="2020-01-20T09:37:12.140" Score="1" Body="&lt;p&gt;I would start by re-formulating the question in this post: Your end-goal seems to the differential expression analysis (DEA) of your samples (and you have to have to levels for this like treatment-control, mutant-WT, etc ...). P-values are context-dependent and should be calculated based on your research question/hypothesis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DEA has been addressed by many researchers and &lt;code&gt;R&lt;/code&gt; has a lot of packages to help you with this; &lt;code&gt;DESeq2&lt;/code&gt;, &lt;code&gt;edgeR&lt;/code&gt;, &lt;code&gt;limma&lt;/code&gt;, etc ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would go to the &lt;code&gt;Biconductor&lt;/code&gt; website, search for one of the three above, and go through their rich &lt;strong&gt;vignettes&lt;/strong&gt; and would come back here for &lt;strong&gt;specific&lt;/strong&gt; questions.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-01-20T09:37:12.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11190" PostTypeId="1" AcceptedAnswerId="11191" CreationDate="2020-01-20T13:25:31.967" Score="-2" ViewCount="50" Body="&lt;p&gt;I have a too big data frame&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dim(anno_maf)&#xA;[1] 2635299     132&#xA;&amp;gt;&#xA;&#xA;&amp;gt; head(anno_maf[1:3,1:4])&#xA;                     Tumor_Sample_Barcode Chromosome Start_Position End_Position&#xA;1: LP2000107-DNA_A01_vs_LP2000102-DNA_A01       chr5      137536800    137536803&#xA;2: LP2000330-DNA_A01_vs_LP2000316-DNA_A01      chr11       64012229     64012231&#xA;3: LP2000330-DNA_A01_vs_LP2000316-DNA_A01      chr17       41720917     41720917&#xA;&amp;gt; &#xA;&#xA;&amp;gt; unique(anno_maf$Tumor_Sample_Barcode)&#xA;  [1] &quot;LP2000107-DNA_A01_vs_LP2000102-DNA_A01&quot; &quot;LP2000330-DNA_A01_vs_LP2000316-DNA_A01&quot; &quot;LP2000332-DNA_A01_vs_LP2000323-DNA_A01&quot;&#xA;  [4] &quot;LP2000333-DNA_A01_vs_LP2000324-DNA_A01&quot; &quot;LP6005334-DNA_A02_vs_LP6005333-DNA_A02&quot; &quot;LP6005334-DNA_C01_vs_LP6005333-DNA_C01&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to remove everything thing from &lt;code&gt;vs&lt;/code&gt; onward in &lt;code&gt;anno_maf$Tumor_Sample_Barcode&lt;/code&gt;, for example leaving &lt;code&gt;LP2000107-DNA_A01&lt;/code&gt; from &lt;code&gt;LP2000107-DNA_A01_vs_LP2000102-DNA_A01&lt;/code&gt; but I don't know how to do that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2020-01-20T13:32:21.160" Title="Chaning string in rows" Tags="|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11191" PostTypeId="2" ParentId="11190" CreationDate="2020-01-20T13:32:21.160" Score="3" Body="&lt;p&gt;&lt;code&gt;sub()&lt;/code&gt; is the way to go.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your pattern is your regular expression: Check for &quot;_vs_&quot; followed by any character (provided with &lt;code&gt;.&lt;/code&gt;) for an undefined length (provided with &lt;code&gt;*&lt;/code&gt;).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;my_text &amp;lt;- &quot;LP2000107-DNA_A01_vs_LP2000102-DNA_A01&quot;&#xA;&#xA;sub(pattern = &quot;_vs_.*&quot;,&#xA;    replacement = &quot;&quot;,&#xA;    x = my_text)&#xA;&#xA;[1] &quot;LP2000107-DNA_A01&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-01-20T13:32:21.160" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11192" PostTypeId="1" AcceptedAnswerId="11193" CreationDate="2020-01-20T14:15:13.417" Score="7" ViewCount="5304" Body="&lt;p&gt;I have a FASTA file which I would like to convert into FASTQ format as the tool I want to use my data in requires it in FASTQ format. So dummy quality scores are fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I am not using any functionality of the tool that will require the quality score.&lt;/p&gt;&#xA;" OwnerUserId="149" LastEditorUserId="149" LastEditDate="2020-01-23T15:09:49.373" LastActivityDate="2022-03-17T01:06:24.740" Title="Convert FASTA to FASTQ with dummy quality scores" Tags="|fasta|fastq|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11193" PostTypeId="2" ParentId="11192" CreationDate="2020-01-20T14:15:13.417" Score="4" Body="&lt;p&gt;One way of doing this is with two subcommands from the &lt;a href=&quot;https://github.com/sanger-pathogens/Fastaq&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;pyfastaq&lt;/code&gt;&lt;/a&gt; suite.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;fastaq to_fake_qual in.fasta - | fastaq fasta_to_fastq in.fasta - out.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first tool, &lt;code&gt;to_fake_qual&lt;/code&gt;, creates fake quality scores (default 40) for each base and the &lt;code&gt;-&lt;/code&gt; sends that file (&lt;code&gt;.qual&lt;/code&gt;) to stdout. The second tool, &lt;code&gt;fasta_to_fastq&lt;/code&gt;, consumes both the original fasta and the quality scores coming from stdin and turns these into a fastq file.&lt;/p&gt;&#xA;" OwnerUserId="149" LastActivityDate="2020-01-20T14:15:13.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11194" PostTypeId="2" ParentId="11192" CreationDate="2020-01-20T20:59:28.290" Score="6" Body="&lt;p&gt;A simple Biopython solution:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;for r in SeqIO.parse(&amp;quot;myfile.fa&amp;quot;, &amp;quot;fasta&amp;quot;):&#xA;    r.letter_annotations[&amp;quot;solexa_quality&amp;quot;] = [40] * len(r)&#xA;    print(r.format(&amp;quot;fastq&amp;quot;), end='')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ cat &amp;lt;&amp;lt; EOF &amp;gt; myfile.fa &#xA;&amp;gt;1&#xA;ATG&#xA;&amp;gt;2&#xA;ATGTAGA&#xA;EOF&#xA;$&lt;/span&gt; python3 myscript.py &#xA;@1&#xA;ATG&#xA;+&#xA;III&#xA;@2&#xA;ATGTAGA&#xA;+&#xA;IIIIIII&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastEditorUserId="-1" LastEditDate="2021-04-30T12:26:16.180" LastActivityDate="2021-04-30T12:26:16.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11195" PostTypeId="2" ParentId="3206" CreationDate="2020-01-20T21:18:30.180" Score="2" Body="&lt;p&gt;The &lt;a href=&quot;https://rdrr.io/bioc/scran/man/quickCluster.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;scran::quickCluster&lt;/code&gt; method&lt;/a&gt; has a &lt;code&gt;BPPARAM&lt;/code&gt; (i.e., &lt;code&gt;BiocParallelParam&lt;/code&gt;), so if one provides something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clusters &amp;lt;- quickCluster(sce, BPPARAM=BiocParallel::MulticoreParam())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;it will use all available cores (a specific number of cores can also be given). See &lt;a href=&quot;https://bioconductor.org/packages/devel/bioc/vignettes/BiocParallel/inst/doc/Introduction_To_BiocParallel.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the intro doc to &lt;code&gt;BiocParallel&lt;/code&gt;&lt;/a&gt; for info on other options.&lt;/p&gt;&#xA;" OwnerUserId="900" LastActivityDate="2020-01-20T21:18:30.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11196" PostTypeId="1" AcceptedAnswerId="11260" CreationDate="2020-01-20T22:52:42.333" Score="0" ViewCount="178" Body="&lt;p&gt;Newbie R user here. I want to create a plot with boxplots of read length distribution on the Y-axis, and samples on the X-axis -- like this image from doi: 10.1038/srep16498 &lt;a href=&quot;https://i.stack.imgur.com/rHloO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rHloO.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I'm not sure how to plot my data in &lt;code&gt;ggplot2&lt;/code&gt; to make this. My data is currently in this format:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;Length Sample1 Sample2&#xA;20 10 5&#xA;30 15 223&#xA;40 16 4923&#xA;50 293 239&#xA;60 3290 23&#xA;70 248 6&#xA;80 23 1&#xA;90 21 0&#xA;100 5 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Length is the read length bin, with the sample columns containing the number of reads of that length. I've tried using &lt;code&gt;ggplot2&lt;/code&gt;, but don't think I can do it with the data in its current format.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Here are the number of reads for each individual read length (converted to long format):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&quot;Readlength&quot; &quot;name&quot; &quot;value&quot;&#xA;&quot;1&quot; 25 &quot;Sample1&quot; 457&#xA;&quot;2&quot; 25 &quot;Sample2&quot; 580&#xA;&quot;3&quot; 26 &quot;Sample1&quot; 603&#xA;&quot;4&quot; 26 &quot;Sample2&quot; 648&#xA;&quot;5&quot; 27 &quot;Sample1&quot; 599&#xA;&quot;6&quot; 27 &quot;Sample2&quot; 715&#xA;&quot;7&quot; 28 &quot;Sample1&quot; 686&#xA;&quot;8&quot; 28 &quot;Sample2&quot; 914&#xA;&quot;9&quot; 29 &quot;Sample1&quot; 796&#xA;&quot;10&quot; 29 &quot;Sample2&quot; 1018&#xA;&quot;11&quot; 30 &quot;Sample1&quot; 1009&#xA;&quot;12&quot; 30 &quot;Sample2&quot; 1366&#xA;&quot;13&quot; 31 &quot;Sample1&quot; 1225&#xA;&quot;14&quot; 31 &quot;Sample2&quot; 1653&#xA;&quot;15&quot; 32 &quot;Sample1&quot; 1579&#xA;&quot;16&quot; 32 &quot;Sample2&quot; 2135&#xA;&quot;17&quot; 33 &quot;Sample1&quot; 1817&#xA;&quot;18&quot; 33 &quot;Sample2&quot; 2678&#xA;&quot;19&quot; 34 &quot;Sample1&quot; 2322&#xA;&quot;20&quot; 34 &quot;Sample2&quot; 3424&#xA;&quot;21&quot; 35 &quot;Sample1&quot; 2861&#xA;&quot;22&quot; 35 &quot;Sample2&quot; 4167&#xA;&quot;23&quot; 36 &quot;Sample1&quot; 3460&#xA;&quot;24&quot; 36 &quot;Sample2&quot; 5220&#xA;&quot;25&quot; 37 &quot;Sample1&quot; 4161&#xA;&quot;26&quot; 37 &quot;Sample2&quot; 6088&#xA;&quot;27&quot; 38 &quot;Sample1&quot; 4930&#xA;&quot;28&quot; 38 &quot;Sample2&quot; 7609&#xA;&quot;29&quot; 39 &quot;Sample1&quot; 5910&#xA;&quot;30&quot; 39 &quot;Sample2&quot; 9308&#xA;&quot;31&quot; 40 &quot;Sample1&quot; 7007&#xA;&quot;32&quot; 40 &quot;Sample2&quot; 11062&#xA;&quot;33&quot; 41 &quot;Sample1&quot; 8339&#xA;&quot;34&quot; 41 &quot;Sample2&quot; 13341&#xA;&quot;35&quot; 42 &quot;Sample1&quot; 10105&#xA;&quot;36&quot; 42 &quot;Sample2&quot; 16288&#xA;&quot;37&quot; 43 &quot;Sample1&quot; 11720&#xA;&quot;38&quot; 43 &quot;Sample2&quot; 19023&#xA;&quot;39&quot; 44 &quot;Sample1&quot; 13312&#xA;&quot;40&quot; 44 &quot;Sample2&quot; 22152&#xA;&quot;41&quot; 45 &quot;Sample1&quot; 15430&#xA;&quot;42&quot; 45 &quot;Sample2&quot; 25360&#xA;&quot;43&quot; 46 &quot;Sample1&quot; 17515&#xA;&quot;44&quot; 46 &quot;Sample2&quot; 28789&#xA;&quot;45&quot; 47 &quot;Sample1&quot; 20022&#xA;&quot;46&quot; 47 &quot;Sample2&quot; 32985&#xA;&quot;47&quot; 48 &quot;Sample1&quot; 22825&#xA;&quot;48&quot; 48 &quot;Sample2&quot; 38019&#xA;&quot;49&quot; 49 &quot;Sample1&quot; 26150&#xA;&quot;50&quot; 49 &quot;Sample2&quot; 42972&#xA;&quot;51&quot; 50 &quot;Sample1&quot; 28998&#xA;&quot;52&quot; 50 &quot;Sample2&quot; 49293&#xA;&quot;53&quot; 51 &quot;Sample1&quot; 33236&#xA;&quot;54&quot; 51 &quot;Sample2&quot; 56239&#xA;&quot;55&quot; 52 &quot;Sample1&quot; 37302&#xA;&quot;56&quot; 52 &quot;Sample2&quot; 62791&#xA;&quot;57&quot; 53 &quot;Sample1&quot; 41212&#xA;&quot;58&quot; 53 &quot;Sample2&quot; 69865&#xA;&quot;59&quot; 54 &quot;Sample1&quot; 45683&#xA;&quot;60&quot; 54 &quot;Sample2&quot; 76922&#xA;&quot;61&quot; 55 &quot;Sample1&quot; 49844&#xA;&quot;62&quot; 55 &quot;Sample2&quot; 82906&#xA;&quot;63&quot; 56 &quot;Sample1&quot; 53833&#xA;&quot;64&quot; 56 &quot;Sample2&quot; 90092&#xA;&quot;65&quot; 57 &quot;Sample1&quot; 58486&#xA;&quot;66&quot; 57 &quot;Sample2&quot; 98194&#xA;&quot;67&quot; 58 &quot;Sample1&quot; 64650&#xA;&quot;68&quot; 58 &quot;Sample2&quot; 105602&#xA;&quot;69&quot; 59 &quot;Sample1&quot; 69514&#xA;&quot;70&quot; 59 &quot;Sample2&quot; 114319&#xA;&quot;71&quot; 60 &quot;Sample1&quot; 76202&#xA;&quot;72&quot; 60 &quot;Sample2&quot; 123807&#xA;&quot;73&quot; 61 &quot;Sample1&quot; 83273&#xA;&quot;74&quot; 61 &quot;Sample2&quot; 133289&#xA;&quot;75&quot; 62 &quot;Sample1&quot; 90783&#xA;&quot;76&quot; 62 &quot;Sample2&quot; 146387&#xA;&quot;77&quot; 63 &quot;Sample1&quot; 98805&#xA;&quot;78&quot; 63 &quot;Sample2&quot; 159302&#xA;&quot;79&quot; 64 &quot;Sample1&quot; 107247&#xA;&quot;80&quot; 64 &quot;Sample2&quot; 168856&#xA;&quot;81&quot; 65 &quot;Sample1&quot; 115200&#xA;&quot;82&quot; 65 &quot;Sample2&quot; 180408&#xA;&quot;83&quot; 66 &quot;Sample1&quot; 123359&#xA;&quot;84&quot; 66 &quot;Sample2&quot; 191083&#xA;&quot;85&quot; 67 &quot;Sample1&quot; 134258&#xA;&quot;86&quot; 67 &quot;Sample2&quot; 200708&#xA;&quot;87&quot; 68 &quot;Sample1&quot; 142179&#xA;&quot;88&quot; 68 &quot;Sample2&quot; 212830&#xA;&quot;89&quot; 69 &quot;Sample1&quot; 155549&#xA;&quot;90&quot; 69 &quot;Sample2&quot; 226788&#xA;&quot;91&quot; 70 &quot;Sample1&quot; 165425&#xA;&quot;92&quot; 70 &quot;Sample2&quot; 239063&#xA;&quot;93&quot; 71 &quot;Sample1&quot; 178681&#xA;&quot;94&quot; 71 &quot;Sample2&quot; 255338&#xA;&quot;95&quot; 72 &quot;Sample1&quot; 194293&#xA;&quot;96&quot; 72 &quot;Sample2&quot; 272725&#xA;&quot;97&quot; 73 &quot;Sample1&quot; 211417&#xA;&quot;98&quot; 73 &quot;Sample2&quot; 288512&#xA;&quot;99&quot; 74 &quot;Sample1&quot; 225639&#xA;&quot;100&quot; 74 &quot;Sample2&quot; 307752&#xA;&quot;101&quot; 75 &quot;Sample1&quot; 242618&#xA;&quot;102&quot; 75 &quot;Sample2&quot; 324919&#xA;&quot;103&quot; 76 &quot;Sample1&quot; 259391&#xA;&quot;104&quot; 76 &quot;Sample2&quot; 338429&#xA;&quot;105&quot; 77 &quot;Sample1&quot; 274041&#xA;&quot;106&quot; 77 &quot;Sample2&quot; 354866&#xA;&quot;107&quot; 78 &quot;Sample1&quot; 295365&#xA;&quot;108&quot; 78 &quot;Sample2&quot; 373456&#xA;&quot;109&quot; 79 &quot;Sample1&quot; 315802&#xA;&quot;110&quot; 79 &quot;Sample2&quot; 389945&#xA;&quot;111&quot; 80 &quot;Sample1&quot; 337826&#xA;&quot;112&quot; 80 &quot;Sample2&quot; 411033&#xA;&quot;113&quot; 81 &quot;Sample1&quot; 363906&#xA;&quot;114&quot; 81 &quot;Sample2&quot; 437069&#xA;&quot;115&quot; 82 &quot;Sample1&quot; 388803&#xA;&quot;116&quot; 82 &quot;Sample2&quot; 460419&#xA;&quot;117&quot; 83 &quot;Sample1&quot; 416728&#xA;&quot;118&quot; 83 &quot;Sample2&quot; 487959&#xA;&quot;119&quot; 84 &quot;Sample1&quot; 446797&#xA;&quot;120&quot; 84 &quot;Sample2&quot; 518295&#xA;&quot;121&quot; 85 &quot;Sample1&quot; 474343&#xA;&quot;122&quot; 85 &quot;Sample2&quot; 538191&#xA;&quot;123&quot; 86 &quot;Sample1&quot; 496615&#xA;&quot;124&quot; 86 &quot;Sample2&quot; 558646&#xA;&quot;125&quot; 87 &quot;Sample1&quot; 524080&#xA;&quot;126&quot; 87 &quot;Sample2&quot; 583043&#xA;&quot;127&quot; 88 &quot;Sample1&quot; 549720&#xA;&quot;128&quot; 88 &quot;Sample2&quot; 601107&#xA;&quot;129&quot; 89 &quot;Sample1&quot; 573864&#xA;&quot;130&quot; 89 &quot;Sample2&quot; 623715&#xA;&quot;131&quot; 90 &quot;Sample1&quot; 604559&#xA;&quot;132&quot; 90 &quot;Sample2&quot; 648209&#xA;&quot;133&quot; 91 &quot;Sample1&quot; 632324&#xA;&quot;134&quot; 91 &quot;Sample2&quot; 668883&#xA;&quot;135&quot; 92 &quot;Sample1&quot; 657924&#xA;&quot;136&quot; 92 &quot;Sample2&quot; 698291&#xA;&quot;137&quot; 93 &quot;Sample1&quot; 686875&#xA;&quot;138&quot; 93 &quot;Sample2&quot; 727313&#xA;&quot;139&quot; 94 &quot;Sample1&quot; 712778&#xA;&quot;140&quot; 94 &quot;Sample2&quot; 745380&#xA;&quot;141&quot; 95 &quot;Sample1&quot; 732786&#xA;&quot;142&quot; 95 &quot;Sample2&quot; 766930&#xA;&quot;143&quot; 96 &quot;Sample1&quot; 755081&#xA;&quot;144&quot; 96 &quot;Sample2&quot; 784337&#xA;&quot;145&quot; 97 &quot;Sample1&quot; 772734&#xA;&quot;146&quot; 97 &quot;Sample2&quot; 793358&#xA;&quot;147&quot; 98 &quot;Sample1&quot; 787861&#xA;&quot;148&quot; 98 &quot;Sample2&quot; 804737&#xA;&quot;149&quot; 99 &quot;Sample1&quot; 807004&#xA;&quot;150&quot; 99 &quot;Sample2&quot; 820443&#xA;&quot;151&quot; 100 &quot;Sample1&quot; 823211&#xA;&quot;152&quot; 100 &quot;Sample2&quot; 826114&#xA;&quot;153&quot; 101 &quot;Sample1&quot; 836362&#xA;&quot;154&quot; 101 &quot;Sample2&quot; 839164&#xA;&quot;155&quot; 102 &quot;Sample1&quot; 853124&#xA;&quot;156&quot; 102 &quot;Sample2&quot; 853144&#xA;&quot;157&quot; 103 &quot;Sample1&quot; 865378&#xA;&quot;158&quot; 103 &quot;Sample2&quot; 861670&#xA;&quot;159&quot; 104 &quot;Sample1&quot; 874578&#xA;&quot;160&quot; 104 &quot;Sample2&quot; 875069&#xA;&quot;161&quot; 105 &quot;Sample1&quot; 887322&#xA;&quot;162&quot; 105 &quot;Sample2&quot; 883285&#xA;&quot;163&quot; 106 &quot;Sample1&quot; 888977&#xA;&quot;164&quot; 106 &quot;Sample2&quot; 875065&#xA;&quot;165&quot; 107 &quot;Sample1&quot; 889700&#xA;&quot;166&quot; 107 &quot;Sample2&quot; 877648&#xA;&quot;167&quot; 108 &quot;Sample1&quot; 893138&#xA;&quot;168&quot; 108 &quot;Sample2&quot; 875875&#xA;&quot;169&quot; 109 &quot;Sample1&quot; 887592&#xA;&quot;170&quot; 109 &quot;Sample2&quot; 865221&#xA;&quot;171&quot; 110 &quot;Sample1&quot; 888684&#xA;&quot;172&quot; 110 &quot;Sample2&quot; 863268&#xA;&quot;173&quot; 111 &quot;Sample1&quot; 887037&#xA;&quot;174&quot; 111 &quot;Sample2&quot; 863513&#xA;&quot;175&quot; 112 &quot;Sample1&quot; 887121&#xA;&quot;176&quot; 112 &quot;Sample2&quot; 855519&#xA;&quot;177&quot; 113 &quot;Sample1&quot; 883085&#xA;&quot;178&quot; 113 &quot;Sample2&quot; 854312&#xA;&quot;179&quot; 114 &quot;Sample1&quot; 886536&#xA;&quot;180&quot; 114 &quot;Sample2&quot; 855445&#xA;&quot;181&quot; 115 &quot;Sample1&quot; 875703&#xA;&quot;182&quot; 115 &quot;Sample2&quot; 844795&#xA;&quot;183&quot; 116 &quot;Sample1&quot; 870749&#xA;&quot;184&quot; 116 &quot;Sample2&quot; 837992&#xA;&quot;185&quot; 117 &quot;Sample1&quot; 863969&#xA;&quot;186&quot; 117 &quot;Sample2&quot; 830157&#xA;&quot;187&quot; 118 &quot;Sample1&quot; 849188&#xA;&quot;188&quot; 118 &quot;Sample2&quot; 809476&#xA;&quot;189&quot; 119 &quot;Sample1&quot; 840690&#xA;&quot;190&quot; 119 &quot;Sample2&quot; 798965&#xA;&quot;191&quot; 120 &quot;Sample1&quot; 834472&#xA;&quot;192&quot; 120 &quot;Sample2&quot; 791200&#xA;&quot;193&quot; 121 &quot;Sample1&quot; 822558&#xA;&quot;194&quot; 121 &quot;Sample2&quot; 776191&#xA;&quot;195&quot; 122 &quot;Sample1&quot; 811864&#xA;&quot;196&quot; 122 &quot;Sample2&quot; 765913&#xA;&quot;197&quot; 123 &quot;Sample1&quot; 805300&#xA;&quot;198&quot; 123 &quot;Sample2&quot; 761200&#xA;&quot;199&quot; 124 &quot;Sample1&quot; 794602&#xA;&quot;200&quot; 124 &quot;Sample2&quot; 747483&#xA;&quot;201&quot; 125 &quot;Sample1&quot; 784650&#xA;&quot;202&quot; 125 &quot;Sample2&quot; 739639&#xA;&quot;203&quot; 126 &quot;Sample1&quot; 775107&#xA;&quot;204&quot; 126 &quot;Sample2&quot; 730152&#xA;&quot;205&quot; 127 &quot;Sample1&quot; 763282&#xA;&quot;206&quot; 127 &quot;Sample2&quot; 713285&#xA;&quot;207&quot; 128 &quot;Sample1&quot; 748652&#xA;&quot;208&quot; 128 &quot;Sample2&quot; 699446&#xA;&quot;209&quot; 129 &quot;Sample1&quot; 736764&#xA;&quot;210&quot; 129 &quot;Sample2&quot; 686165&#xA;&quot;211&quot; 130 &quot;Sample1&quot; 723134&#xA;&quot;212&quot; 130 &quot;Sample2&quot; 671281&#xA;&quot;213&quot; 131 &quot;Sample1&quot; 710686&#xA;&quot;214&quot; 131 &quot;Sample2&quot; 658314&#xA;&quot;215&quot; 132 &quot;Sample1&quot; 701834&#xA;&quot;216&quot; 132 &quot;Sample2&quot; 650564&#xA;&quot;217&quot; 133 &quot;Sample1&quot; 689531&#xA;&quot;218&quot; 133 &quot;Sample2&quot; 635331&#xA;&quot;219&quot; 134 &quot;Sample1&quot; 678070&#xA;&quot;220&quot; 134 &quot;Sample2&quot; 628475&#xA;&quot;221&quot; 135 &quot;Sample1&quot; 668757&#xA;&quot;222&quot; 135 &quot;Sample2&quot; 617210&#xA;&quot;223&quot; 136 &quot;Sample1&quot; 654506&#xA;&quot;224&quot; 136 &quot;Sample2&quot; 601336&#xA;&quot;225&quot; 137 &quot;Sample1&quot; 644720&#xA;&quot;226&quot; 137 &quot;Sample2&quot; 592700&#xA;&quot;227&quot; 138 &quot;Sample1&quot; 630305&#xA;&quot;228&quot; 138 &quot;Sample2&quot; 579707&#xA;&quot;229&quot; 139 &quot;Sample1&quot; 615571&#xA;&quot;230&quot; 139 &quot;Sample2&quot; 562022&#xA;&quot;231&quot; 140 &quot;Sample1&quot; 603896&#xA;&quot;232&quot; 140 &quot;Sample2&quot; 550812&#xA;&quot;233&quot; 141 &quot;Sample1&quot; 597816&#xA;&quot;234&quot; 141 &quot;Sample2&quot; 540016&#xA;&quot;235&quot; 142 &quot;Sample1&quot; 584121&#xA;&quot;236&quot; 142 &quot;Sample2&quot; 526066&#xA;&quot;237&quot; 143 &quot;Sample1&quot; 571689&#xA;&quot;238&quot; 143 &quot;Sample2&quot; 516774&#xA;&quot;239&quot; 144 &quot;Sample1&quot; 564142&#xA;&quot;240&quot; 144 &quot;Sample2&quot; 509033&#xA;&quot;241&quot; 145 &quot;Sample1&quot; 552281&#xA;&quot;242&quot; 145 &quot;Sample2&quot; 496389&#xA;&quot;243&quot; 146 &quot;Sample1&quot; 543184&#xA;&quot;244&quot; 146 &quot;Sample2&quot; 487283&#xA;&quot;245&quot; 147 &quot;Sample1&quot; 532683&#xA;&quot;246&quot; 147 &quot;Sample2&quot; 477035&#xA;&quot;247&quot; 148 &quot;Sample1&quot; 520019&#xA;&quot;248&quot; 148 &quot;Sample2&quot; 461698&#xA;&quot;249&quot; 149 &quot;Sample1&quot; 510484&#xA;&quot;250&quot; 149 &quot;Sample2&quot; 451095&#xA;&quot;251&quot; 150 &quot;Sample1&quot; 507832&#xA;&quot;252&quot; 150 &quot;Sample2&quot; 446390&#xA;&quot;253&quot; 151 &quot;Sample1&quot; 495343&#xA;&quot;254&quot; 151 &quot;Sample2&quot; 431015&#xA;&quot;255&quot; 152 &quot;Sample1&quot; 484169&#xA;&quot;256&quot; 152 &quot;Sample2&quot; 417551&#xA;&quot;257&quot; 153 &quot;Sample1&quot; 471422&#xA;&quot;258&quot; 153 &quot;Sample2&quot; 404674&#xA;&quot;259&quot; 154 &quot;Sample1&quot; 460944&#xA;&quot;260&quot; 154 &quot;Sample2&quot; 390280&#xA;&quot;261&quot; 155 &quot;Sample1&quot; 448227&#xA;&quot;262&quot; 155 &quot;Sample2&quot; 380134&#xA;&quot;263&quot; 156 &quot;Sample1&quot; 437738&#xA;&quot;264&quot; 156 &quot;Sample2&quot; 369703&#xA;&quot;265&quot; 157 &quot;Sample1&quot; 428359&#xA;&quot;266&quot; 157 &quot;Sample2&quot; 354902&#xA;&quot;267&quot; 158 &quot;Sample1&quot; 417445&#xA;&quot;268&quot; 158 &quot;Sample2&quot; 343343&#xA;&quot;269&quot; 159 &quot;Sample1&quot; 408065&#xA;&quot;270&quot; 159 &quot;Sample2&quot; 331193&#xA;&quot;271&quot; 160 &quot;Sample1&quot; 395244&#xA;&quot;272&quot; 160 &quot;Sample2&quot; 315775&#xA;&quot;273&quot; 161 &quot;Sample1&quot; 386510&#xA;&quot;274&quot; 161 &quot;Sample2&quot; 305048&#xA;&quot;275&quot; 162 &quot;Sample1&quot; 377028&#xA;&quot;276&quot; 162 &quot;Sample2&quot; 292336&#xA;&quot;277&quot; 163 &quot;Sample1&quot; 366563&#xA;&quot;278&quot; 163 &quot;Sample2&quot; 277876&#xA;&quot;279&quot; 164 &quot;Sample1&quot; 357659&#xA;&quot;280&quot; 164 &quot;Sample2&quot; 267642&#xA;&quot;281&quot; 165 &quot;Sample1&quot; 348242&#xA;&quot;282&quot; 165 &quot;Sample2&quot; 255871&#xA;&quot;283&quot; 166 &quot;Sample1&quot; 337553&#xA;&quot;284&quot; 166 &quot;Sample2&quot; 242636&#xA;&quot;285&quot; 167 &quot;Sample1&quot; 327824&#xA;&quot;286&quot; 167 &quot;Sample2&quot; 231054&#xA;&quot;287&quot; 168 &quot;Sample1&quot; 317584&#xA;&quot;288&quot; 168 &quot;Sample2&quot; 219066&#xA;&quot;289&quot; 169 &quot;Sample1&quot; 306454&#xA;&quot;290&quot; 169 &quot;Sample2&quot; 204829&#xA;&quot;291&quot; 170 &quot;Sample1&quot; 295255&#xA;&quot;292&quot; 170 &quot;Sample2&quot; 192423&#xA;&quot;293&quot; 171 &quot;Sample1&quot; 284770&#xA;&quot;294&quot; 171 &quot;Sample2&quot; 180440&#xA;&quot;295&quot; 172 &quot;Sample1&quot; 274444&#xA;&quot;296&quot; 172 &quot;Sample2&quot; 166529&#xA;&quot;297&quot; 173 &quot;Sample1&quot; 264469&#xA;&quot;298&quot; 173 &quot;Sample2&quot; 155605&#xA;&quot;299&quot; 174 &quot;Sample1&quot; 253923&#xA;&quot;300&quot; 174 &quot;Sample2&quot; 145166&#xA;&quot;301&quot; 175 &quot;Sample1&quot; 243462&#xA;&quot;302&quot; 175 &quot;Sample2&quot; 134932&#xA;&quot;303&quot; 176 &quot;Sample1&quot; 233021&#xA;&quot;304&quot; 176 &quot;Sample2&quot; 124474&#xA;&quot;305&quot; 177 &quot;Sample1&quot; 223360&#xA;&quot;306&quot; 177 &quot;Sample2&quot; 114975&#xA;&quot;307&quot; 178 &quot;Sample1&quot; 213341&#xA;&quot;308&quot; 178 &quot;Sample2&quot; 105267&#xA;&quot;309&quot; 179 &quot;Sample1&quot; 202191&#xA;&quot;310&quot; 179 &quot;Sample2&quot; 95632&#xA;&quot;311&quot; 180 &quot;Sample1&quot; 193325&#xA;&quot;312&quot; 180 &quot;Sample2&quot; 87027&#xA;&quot;313&quot; 181 &quot;Sample1&quot; 182022&#xA;&quot;314&quot; 181 &quot;Sample2&quot; 78802&#xA;&quot;315&quot; 182 &quot;Sample1&quot; 173011&#xA;&quot;316&quot; 182 &quot;Sample2&quot; 71116&#xA;&quot;317&quot; 183 &quot;Sample1&quot; 164207&#xA;&quot;318&quot; 183 &quot;Sample2&quot; 64758&#xA;&quot;319&quot; 184 &quot;Sample1&quot; 154985&#xA;&quot;320&quot; 184 &quot;Sample2&quot; 57082&#xA;&quot;321&quot; 185 &quot;Sample1&quot; 147070&#xA;&quot;322&quot; 185 &quot;Sample2&quot; 51728&#xA;&quot;323&quot; 186 &quot;Sample1&quot; 138927&#xA;&quot;324&quot; 186 &quot;Sample2&quot; 46295&#xA;&quot;325&quot; 187 &quot;Sample1&quot; 130203&#xA;&quot;326&quot; 187 &quot;Sample2&quot; 40908&#xA;&quot;327&quot; 188 &quot;Sample1&quot; 122935&#xA;&quot;328&quot; 188 &quot;Sample2&quot; 36275&#xA;&quot;329&quot; 189 &quot;Sample1&quot; 115565&#xA;&quot;330&quot; 189 &quot;Sample2&quot; 31825&#xA;&quot;331&quot; 190 &quot;Sample1&quot; 107969&#xA;&quot;332&quot; 190 &quot;Sample2&quot; 28028&#xA;&quot;333&quot; 191 &quot;Sample1&quot; 100995&#xA;&quot;334&quot; 191 &quot;Sample2&quot; 24447&#xA;&quot;335&quot; 192 &quot;Sample1&quot; 94510&#xA;&quot;336&quot; 192 &quot;Sample2&quot; 21125&#xA;&quot;337&quot; 193 &quot;Sample1&quot; 88365&#xA;&quot;338&quot; 193 &quot;Sample2&quot; 18489&#xA;&quot;339&quot; 194 &quot;Sample1&quot; 82247&#xA;&quot;340&quot; 194 &quot;Sample2&quot; 15878&#xA;&quot;341&quot; 195 &quot;Sample1&quot; 77315&#xA;&quot;342&quot; 195 &quot;Sample2&quot; 13881&#xA;&quot;343&quot; 196 &quot;Sample1&quot; 71657&#xA;&quot;344&quot; 196 &quot;Sample2&quot; 12183&#xA;&quot;345&quot; 197 &quot;Sample1&quot; 66920&#xA;&quot;346&quot; 197 &quot;Sample2&quot; 10305&#xA;&quot;347&quot; 198 &quot;Sample1&quot; 61298&#xA;&quot;348&quot; 198 &quot;Sample2&quot; 9096&#xA;&quot;349&quot; 199 &quot;Sample1&quot; 57765&#xA;&quot;350&quot; 199 &quot;Sample2&quot; 7759&#xA;&quot;351&quot; 200 &quot;Sample1&quot; 53811&#xA;&quot;352&quot; 200 &quot;Sample2&quot; 6645&#xA;&quot;353&quot; 201 &quot;Sample1&quot; 48906&#xA;&quot;354&quot; 201 &quot;Sample2&quot; 5532&#xA;&quot;355&quot; 202 &quot;Sample1&quot; 44847&#xA;&quot;356&quot; 202 &quot;Sample2&quot; 4642&#xA;&quot;357&quot; 203 &quot;Sample1&quot; 41546&#xA;&quot;358&quot; 203 &quot;Sample2&quot; 3997&#xA;&quot;359&quot; 204 &quot;Sample1&quot; 38125&#xA;&quot;360&quot; 204 &quot;Sample2&quot; 3360&#xA;&quot;361&quot; 205 &quot;Sample1&quot; 35039&#xA;&quot;362&quot; 205 &quot;Sample2&quot; 2858&#xA;&quot;363&quot; 206 &quot;Sample1&quot; 31902&#xA;&quot;364&quot; 206 &quot;Sample2&quot; 2447&#xA;&quot;365&quot; 207 &quot;Sample1&quot; 29076&#xA;&quot;366&quot; 207 &quot;Sample2&quot; 2185&#xA;&quot;367&quot; 208 &quot;Sample1&quot; 26271&#xA;&quot;368&quot; 208 &quot;Sample2&quot; 1811&#xA;&quot;369&quot; 209 &quot;Sample1&quot; 24037&#xA;&quot;370&quot; 209 &quot;Sample2&quot; 1494&#xA;&quot;371&quot; 210 &quot;Sample1&quot; 21487&#xA;&quot;372&quot; 210 &quot;Sample2&quot; 1313&#xA;&quot;373&quot; 211 &quot;Sample1&quot; 19187&#xA;&quot;374&quot; 211 &quot;Sample2&quot; 1028&#xA;&quot;375&quot; 212 &quot;Sample1&quot; 17317&#xA;&quot;376&quot; 212 &quot;Sample2&quot; 954&#xA;&quot;377&quot; 213 &quot;Sample1&quot; 15741&#xA;&quot;378&quot; 213 &quot;Sample2&quot; 819&#xA;&quot;379&quot; 214 &quot;Sample1&quot; 14000&#xA;&quot;380&quot; 214 &quot;Sample2&quot; 693&#xA;&quot;381&quot; 215 &quot;Sample1&quot; 12494&#xA;&quot;382&quot; 215 &quot;Sample2&quot; 614&#xA;&quot;383&quot; 216 &quot;Sample1&quot; 10963&#xA;&quot;384&quot; 216 &quot;Sample2&quot; 556&#xA;&quot;385&quot; 217 &quot;Sample1&quot; 9928&#xA;&quot;386&quot; 217 &quot;Sample2&quot; 549&#xA;&quot;387&quot; 218 &quot;Sample1&quot; 8652&#xA;&quot;388&quot; 218 &quot;Sample2&quot; 421&#xA;&quot;389&quot; 219 &quot;Sample1&quot; 7604&#xA;&quot;390&quot; 219 &quot;Sample2&quot; 404&#xA;&quot;391&quot; 220 &quot;Sample1&quot; 6515&#xA;&quot;392&quot; 220 &quot;Sample2&quot; 394&#xA;&quot;393&quot; 221 &quot;Sample1&quot; 5735&#xA;&quot;394&quot; 221 &quot;Sample2&quot; 325&#xA;&quot;395&quot; 222 &quot;Sample1&quot; 4783&#xA;&quot;396&quot; 222 &quot;Sample2&quot; 301&#xA;&quot;397&quot; 223 &quot;Sample1&quot; 4108&#xA;&quot;398&quot; 223 &quot;Sample2&quot; 308&#xA;&quot;399&quot; 224 &quot;Sample1&quot; 3611&#xA;&quot;400&quot; 224 &quot;Sample2&quot; 290&#xA;&quot;401&quot; 225 &quot;Sample1&quot; 3224&#xA;&quot;402&quot; 225 &quot;Sample2&quot; 307&#xA;&quot;403&quot; 226 &quot;Sample1&quot; 2676&#xA;&quot;404&quot; 226 &quot;Sample2&quot; 304&#xA;&quot;405&quot; 227 &quot;Sample1&quot; 2261&#xA;&quot;406&quot; 227 &quot;Sample2&quot; 273&#xA;&quot;407&quot; 228 &quot;Sample1&quot; 1936&#xA;&quot;408&quot; 228 &quot;Sample2&quot; 303&#xA;&quot;409&quot; 229 &quot;Sample1&quot; 1664&#xA;&quot;410&quot; 229 &quot;Sample2&quot; 302&#xA;&quot;411&quot; 230 &quot;Sample1&quot; 1529&#xA;&quot;412&quot; 230 &quot;Sample2&quot; 291&#xA;&quot;413&quot; 231 &quot;Sample1&quot; 1180&#xA;&quot;414&quot; 231 &quot;Sample2&quot; 313&#xA;&quot;415&quot; 232 &quot;Sample1&quot; 975&#xA;&quot;416&quot; 232 &quot;Sample2&quot; 288&#xA;&quot;417&quot; 233 &quot;Sample1&quot; 897&#xA;&quot;418&quot; 233 &quot;Sample2&quot; 274&#xA;&quot;419&quot; 234 &quot;Sample1&quot; 812&#xA;&quot;420&quot; 234 &quot;Sample2&quot; 281&#xA;&quot;421&quot; 235 &quot;Sample1&quot; 712&#xA;&quot;422&quot; 235 &quot;Sample2&quot; 318&#xA;&quot;423&quot; 236 &quot;Sample1&quot; 587&#xA;&quot;424&quot; 236 &quot;Sample2&quot; 288&#xA;&quot;425&quot; 237 &quot;Sample1&quot; 555&#xA;&quot;426&quot; 237 &quot;Sample2&quot; 299&#xA;&quot;427&quot; 238 &quot;Sample1&quot; 524&#xA;&quot;428&quot; 238 &quot;Sample2&quot; 314&#xA;&quot;429&quot; 239 &quot;Sample1&quot; 443&#xA;&quot;430&quot; 239 &quot;Sample2&quot; 336&#xA;&quot;431&quot; 240 &quot;Sample1&quot; 411&#xA;&quot;432&quot; 240 &quot;Sample2&quot; 354&#xA;&quot;433&quot; 241 &quot;Sample1&quot; 382&#xA;&quot;434&quot; 241 &quot;Sample2&quot; 339&#xA;&quot;435&quot; 242 &quot;Sample1&quot; 344&#xA;&quot;436&quot; 242 &quot;Sample2&quot; 342&#xA;&quot;437&quot; 243 &quot;Sample1&quot; 340&#xA;&quot;438&quot; 243 &quot;Sample2&quot; 366&#xA;&quot;439&quot; 244 &quot;Sample1&quot; 362&#xA;&quot;440&quot; 244 &quot;Sample2&quot; 353&#xA;&quot;441&quot; 245 &quot;Sample1&quot; 325&#xA;&quot;442&quot; 245 &quot;Sample2&quot; 373&#xA;&quot;443&quot; 246 &quot;Sample1&quot; 290&#xA;&quot;444&quot; 246 &quot;Sample2&quot; 340&#xA;&quot;445&quot; 247 &quot;Sample1&quot; 319&#xA;&quot;446&quot; 247 &quot;Sample2&quot; 345&#xA;&quot;447&quot; 248 &quot;Sample1&quot; 293&#xA;&quot;448&quot; 248 &quot;Sample2&quot; 341&#xA;&quot;449&quot; 249 &quot;Sample1&quot; 304&#xA;&quot;450&quot; 249 &quot;Sample2&quot; 365&#xA;&quot;451&quot; 250 &quot;Sample1&quot; 354&#xA;&quot;452&quot; 250 &quot;Sample2&quot; 381&#xA;&quot;453&quot; 251 &quot;Sample1&quot; 296&#xA;&quot;454&quot; 251 &quot;Sample2&quot; 366&#xA;&quot;455&quot; 252 &quot;Sample1&quot; 315&#xA;&quot;456&quot; 252 &quot;Sample2&quot; 415&#xA;&quot;457&quot; 253 &quot;Sample1&quot; 315&#xA;&quot;458&quot; 253 &quot;Sample2&quot; 399&#xA;&quot;459&quot; 254 &quot;Sample1&quot; 331&#xA;&quot;460&quot; 254 &quot;Sample2&quot; 377&#xA;&quot;461&quot; 255 &quot;Sample1&quot; 321&#xA;&quot;462&quot; 255 &quot;Sample2&quot; 410&#xA;&quot;463&quot; 256 &quot;Sample1&quot; 326&#xA;&quot;464&quot; 256 &quot;Sample2&quot; 421&#xA;&quot;465&quot; 257 &quot;Sample1&quot; 349&#xA;&quot;466&quot; 257 &quot;Sample2&quot; 433&#xA;&quot;467&quot; 258 &quot;Sample1&quot; 370&#xA;&quot;468&quot; 258 &quot;Sample2&quot; 429&#xA;&quot;469&quot; 259 &quot;Sample1&quot; 318&#xA;&quot;470&quot; 259 &quot;Sample2&quot; 439&#xA;&quot;471&quot; 260 &quot;Sample1&quot; 344&#xA;&quot;472&quot; 260 &quot;Sample2&quot; 433&#xA;&quot;473&quot; 261 &quot;Sample1&quot; 346&#xA;&quot;474&quot; 261 &quot;Sample2&quot; 400&#xA;&quot;475&quot; 262 &quot;Sample1&quot; 341&#xA;&quot;476&quot; 262 &quot;Sample2&quot; 419&#xA;&quot;477&quot; 263 &quot;Sample1&quot; 345&#xA;&quot;478&quot; 263 &quot;Sample2&quot; 499&#xA;&quot;479&quot; 264 &quot;Sample1&quot; 348&#xA;&quot;480&quot; 264 &quot;Sample2&quot; 418&#xA;&quot;481&quot; 265 &quot;Sample1&quot; 371&#xA;&quot;482&quot; 265 &quot;Sample2&quot; 454&#xA;&quot;483&quot; 266 &quot;Sample1&quot; 405&#xA;&quot;484&quot; 266 &quot;Sample2&quot; 441&#xA;&quot;485&quot; 267 &quot;Sample1&quot; 380&#xA;&quot;486&quot; 267 &quot;Sample2&quot; 476&#xA;&quot;487&quot; 268 &quot;Sample1&quot; 365&#xA;&quot;488&quot; 268 &quot;Sample2&quot; 464&#xA;&quot;489&quot; 269 &quot;Sample1&quot; 366&#xA;&quot;490&quot; 269 &quot;Sample2&quot; 461&#xA;&quot;491&quot; 270 &quot;Sample1&quot; 351&#xA;&quot;492&quot; 270 &quot;Sample2&quot; 483&#xA;&quot;493&quot; 271 &quot;Sample1&quot; 431&#xA;&quot;494&quot; 271 &quot;Sample2&quot; 481&#xA;&quot;495&quot; 272 &quot;Sample1&quot; 340&#xA;&quot;496&quot; 272 &quot;Sample2&quot; 503&#xA;&quot;497&quot; 273 &quot;Sample1&quot; 405&#xA;&quot;498&quot; 273 &quot;Sample2&quot; 493&#xA;&quot;499&quot; 274 &quot;Sample1&quot; 370&#xA;&quot;500&quot; 274 &quot;Sample2&quot; 529&#xA;&quot;501&quot; 275 &quot;Sample1&quot; 374&#xA;&quot;502&quot; 275 &quot;Sample2&quot; 469&#xA;&quot;503&quot; 276 &quot;Sample1&quot; 385&#xA;&quot;504&quot; 276 &quot;Sample2&quot; 503&#xA;&quot;505&quot; 277 &quot;Sample1&quot; 361&#xA;&quot;506&quot; 277 &quot;Sample2&quot; 496&#xA;&quot;507&quot; 278 &quot;Sample1&quot; 410&#xA;&quot;508&quot; 278 &quot;Sample2&quot; 491&#xA;&quot;509&quot; 279 &quot;Sample1&quot; 410&#xA;&quot;510&quot; 279 &quot;Sample2&quot; 521&#xA;&quot;511&quot; 280 &quot;Sample1&quot; 368&#xA;&quot;512&quot; 280 &quot;Sample2&quot; 514&#xA;&quot;513&quot; 281 &quot;Sample1&quot; 397&#xA;&quot;514&quot; 281 &quot;Sample2&quot; 501&#xA;&quot;515&quot; 282 &quot;Sample1&quot; 379&#xA;&quot;516&quot; 282 &quot;Sample2&quot; 546&#xA;&quot;517&quot; 283 &quot;Sample1&quot; 384&#xA;&quot;518&quot; 283 &quot;Sample2&quot; 502&#xA;&quot;519&quot; 284 &quot;Sample1&quot; 389&#xA;&quot;520&quot; 284 &quot;Sample2&quot; 513&#xA;&quot;521&quot; 285 &quot;Sample1&quot; 419&#xA;&quot;522&quot; 285 &quot;Sample2&quot; 571&#xA;&quot;523&quot; 286 &quot;Sample1&quot; 403&#xA;&quot;524&quot; 286 &quot;Sample2&quot; 519&#xA;&quot;525&quot; 287 &quot;Sample1&quot; 423&#xA;&quot;526&quot; 287 &quot;Sample2&quot; 536&#xA;&quot;527&quot; 288 &quot;Sample1&quot; 480&#xA;&quot;528&quot; 288 &quot;Sample2&quot; 653&#xA;&quot;529&quot; 289 &quot;Sample1&quot; 478&#xA;&quot;530&quot; 289 &quot;Sample2&quot; 564&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6797" LastEditorUserId="73" LastEditDate="2020-01-21T22:42:57.427" LastActivityDate="2020-01-28T15:13:07.623" Title="Boxplot of read length distributions from multiple samples" Tags="|r|ggplot2|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11197" PostTypeId="1" CreationDate="2020-01-20T23:17:09.230" Score="0" ViewCount="608" Body="&lt;p&gt;I want to create a stacked bar-graph with different cell cycles for each cell type within each condition. I have uploaded the file for it. Don't know how to go about it as now I have another condition (cell cycle). I was able to do the bar graph comparing cell cycle per condition but now I want to compare cell type and cell cycle in each condition&lt;/p&gt;&#xA;&#xA;&lt;p&gt;output of dput(head(all.combined@metadata))&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(orig.ident = c(&quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &#xA;    &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;), nCount_RNA = c(1892, 307, 1348, &#xA;    3699, 4205, 4468), nFeature_RNA = c(960L, 243L, 765L, 1612L, &#xA;    1341L, 1644L), percent.mt = c(0.211416490486258, 1.62866449511401, &#xA;    4.45103857566766, 4.4065963773993, 0.0713436385255648, 3.87197851387645&#xA;    ), RNA_snn_res.0.5 = structure(c(11L, 11L, 5L, 6L, 11L, 13L), .Label = c(&quot;0&quot;, &quot;1&quot;, &quot;2&quot;, &quot;3&quot;, &quot;4&quot;, &quot;5&quot;, &quot;6&quot;, &quot;7&quot;, &quot;8&quot;, &quot;9&quot;, &quot;10&quot;, &quot;11&quot;, &quot;12&quot;, &#xA;    &quot;13&quot;, &quot;14&quot;, &quot;15&quot;, &quot;16&quot;, &quot;17&quot;, &quot;18&quot;, &quot;19&quot;), class = &quot;factor&quot;), seurat_clusters = structure(c(11L, 11L, 5L, 6L, 11L, 13L), .Label = c(&quot;0&quot;, &quot;1&quot;, &quot;2&quot;, &quot;3&quot;, &quot;4&quot;, &quot;5&quot;, &quot;6&quot;, &quot;7&quot;, &quot;8&quot;, &quot;9&quot;, &quot;10&quot;, &quot;11&quot;, &quot;12&quot;, &quot;13&quot;, &quot;14&quot;, &quot;15&quot;, &quot;16&quot;, &quot;17&quot;, &quot;18&quot;, &quot;19&quot;), class = &quot;factor&quot;), S.Score = c(0.476893835992198, -0.0200784617568548, -0.0335915198305002, -0.0247184276246385, 0.010785196602457, 0.0190008903712199), G2M.Score = c(0.204441469200986, 0.173804859670862, -0.0313235510969097, -0.0376796363661889, -0.0559526905696905, -0.0122031631356698), Phase = structure(c(3L, 2L, 1L, 1L, 3L, 3L), .Label = c(&quot;G1&quot;, &quot;G2M&quot;, &quot;S&quot;), class = &quot;factor&quot;), old.ident = structure(c(7L,7L, 1L, 4L, 7L, 9L), .Label = c(&quot;Fibroblast&quot;, &quot;T cell&quot;, &quot;Macrophage&quot;, &quot;Stellate&quot;, &quot;Acinar&quot;, &quot;Endothelial&quot;, &quot;Tumor&quot;, &quot;B cell&quot;, &quot;Mast cell&quot;, &quot;Ductal&quot;, &quot;Islets of Langerhans&quot;), class = &quot;factor&quot;)), row.names = c(&quot;treated_AAACGCTAGCGGGTTA-1&quot;, &quot;treated_AAAGGTAAGTACAGAT-1&quot;, &quot;treated_AAAGTGAGTTTGATCG-1&quot;, &quot;treated_AAATGGACAAAGTGTA-1&quot;, &#xA;    &quot;treated_AACAAAGGTCGACTTA-1&quot;, &quot;treated_AACAGGGTCCTAGCCT-1&quot;), class = &quot;data.frame&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;output of dput(tail(all.combined@metadata))&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(orig.ident = c(&quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &#xA;&quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;), nCount_RNA = c(901, 823, &#xA;1184, 1835, 1147, 1407), nFeature_RNA = c(482L, 479L, 649L, 1043L, &#xA;604L, 709L), percent.mt = c(1.77580466148724, 2.91616038882138, &#xA;4.22297297297297, 3.86920980926431, 2.0052310374891, 4.05117270788913&#xA;), RNA_snn_res.0.5 = structure(c(7L, 7L, 7L, 14L, 7L, 7L), .Label = c(&quot;0&quot;, &#xA;&quot;1&quot;, &quot;2&quot;, &quot;3&quot;, &quot;4&quot;, &quot;5&quot;, &quot;6&quot;, &quot;7&quot;, &quot;8&quot;, &quot;9&quot;, &quot;10&quot;, &quot;11&quot;, &quot;12&quot;, &#xA;&quot;13&quot;, &quot;14&quot;, &quot;15&quot;, &quot;16&quot;, &quot;17&quot;, &quot;18&quot;, &quot;19&quot;), class = &quot;factor&quot;), &#xA;    seurat_clusters = structure(c(7L, 7L, 7L, 14L, 7L, 7L), .Label = c(&quot;0&quot;, &#xA;    &quot;1&quot;, &quot;2&quot;, &quot;3&quot;, &quot;4&quot;, &quot;5&quot;, &quot;6&quot;, &quot;7&quot;, &quot;8&quot;, &quot;9&quot;, &quot;10&quot;, &quot;11&quot;, &#xA;    &quot;12&quot;, &quot;13&quot;, &quot;14&quot;, &quot;15&quot;, &quot;16&quot;, &quot;17&quot;, &quot;18&quot;, &quot;19&quot;), class = &quot;factor&quot;), &#xA;    S.Score = c(-0.0320858200243315, 0.0304725660342869, 0.0215996091745327, &#xA;    0.0384166213301423, 0.144956251122548, -0.0242770509986111&#xA;    ), G2M.Score = c(0.0904224391544142, 0.050148242050667, -0.0178041670730754, &#xA;    -0.0112596867977946, -0.0519554524339088, -0.0136533184257381&#xA;    ), Phase = structure(c(2L, 2L, 3L, 3L, 3L, 1L), .Label = c(&quot;G1&quot;, &#xA;    &quot;G2M&quot;, &quot;S&quot;), class = &quot;factor&quot;), old.ident = structure(c(5L, &#xA;    5L, 5L, 5L, 5L, 5L), .Label = c(&quot;Fibroblast&quot;, &quot;T cell&quot;, &quot;Macrophage&quot;, &#xA;    &quot;Stellate&quot;, &quot;Acinar&quot;, &quot;Endothelial&quot;, &quot;Tumor&quot;, &quot;B cell&quot;, &quot;Mast cell&quot;, &#xA;    &quot;Ductal&quot;, &quot;Islets of Langerhans&quot;), class = &quot;factor&quot;)), row.names = c(&quot;untreated_TTTGGTTGTCTAATCG-18&quot;, &#xA;&quot;untreated_TTTGGTTTCCCGAGGT-18&quot;, &quot;untreated_TTTGTTGAGAACTGAT-18&quot;, &#xA;&quot;untreated_TTTGTTGAGCTCGGCT-18&quot;, &quot;untreated_TTTGTTGAGTGCCTCG-18&quot;, &#xA;&quot;untreated_TTTGTTGCACGGTGCT-18&quot;), class = &quot;data.frame&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Been using this code to generate the previous graph.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(CC, aes(x = Condition, y = Percent, fill = Cell_Cycle))+&#xA;  geom_bar(stat = &quot;identity&quot;)+&#xA;  geom_text(aes(label = paste(round(Percent,2),&quot;%&quot;)), position = position_stack(vjust =  0.5))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fkecc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fkecc.png&quot; alt=&quot;cell cycle by condition&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6774" LastEditorUserId="6774" LastEditDate="2020-01-20T23:47:46.657" LastActivityDate="2020-01-21T00:11:07.753" Title="Stacked barplot for single cell analysis" Tags="|single-cell|ggplot2|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11198" PostTypeId="2" ParentId="11196" CreationDate="2020-01-20T23:29:28.737" Score="1" Body="&lt;p&gt;My first recommendation would be to use (i.e. ask for) the indidividual-level data, which is what ggplot2 expects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Binned Data&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If that's not possible, binned data can be shoehorned into a ggplot2 boxplot by forcing it back into individual values. My example here adds new unbinned rows to the table with a count of -1, then removes rows with a positive count:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse)&#xA;library(ggplot2)&#xA;&#xA;read_table2(&quot;11196_input.txt&quot;) %&amp;gt;%&#xA;    pivot_longer(cols=c(&quot;Sample1&quot;, &quot;Sample2&quot;),&#xA;                 names_to=&quot;Sample&quot;, values_to=&quot;Count&quot;) %&amp;gt;%&#xA;    bind_rows(tibble(Length=rep(.&lt;span class=&quot;math-container&quot;&gt;$Length, .$&lt;/span&gt;Count),&#xA;                     Sample=rep(.&lt;span class=&quot;math-container&quot;&gt;$Sample, .$&lt;/span&gt;Count),&#xA;                     Count=rep(.&lt;span class=&quot;math-container&quot;&gt;$Length * 0 - 1, .$&lt;/span&gt;Count))) %&amp;gt;%&#xA;    filter(Count &amp;lt; 0) %&amp;gt;%&#xA;    select(-Count) -&amp;gt; data.tbl&#xA;&#xA;data.tbl %&amp;gt;%&#xA;    ggplot() +&#xA;    aes(x = Sample, y = Length) +&#xA;    geom_boxplot()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iIgah.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iIgah.png&quot; alt=&quot;boxplot of binned data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would be possible to make the boxplot look more normal by modifying the &lt;code&gt;rep&lt;/code&gt; functions to create values based on a distribution (e.g. using &lt;code&gt;runif&lt;/code&gt; or &lt;code&gt;rnorm&lt;/code&gt;), but that gets quite tricky because (for example) binned normal data would have different probability distributions for each bin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Individual Values&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If, on the other hand, individual values are available in long format, creating a boxplot becomes much easier because the data fits the format expected by ggplot2:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse)&#xA;library(ggplot2)&#xA;&#xA;read.table(&quot;11196_input2.txt&quot;) -&amp;gt; data.tbl&#xA;&#xA;data.tbl %&amp;gt;%&#xA;    ggplot() +&#xA;    aes(x = name, y = Readlength) +&#xA;    geom_boxplot()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FeYEH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FeYEH.png&quot; alt=&quot;boxplot of raw data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-01-21T22:42:19.643" LastActivityDate="2020-01-21T22:42:19.643" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11199" PostTypeId="2" ParentId="11196" CreationDate="2020-01-20T23:33:52.690" Score="1" Body="&lt;p&gt;The plot you pasted in your question has two layers of information: i) fragment (or read) length, ii) sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To be able to draw the same plot with your data using &lt;code&gt;ggplot2&lt;/code&gt;, you will need your data to be in the &quot;long format&quot; (your data as you presented now is in the &quot;wide format&quot;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(readr)&#xA;library(tidyr)&#xA;library(ggplot2)&#xA;&#xA;my_data &amp;lt;- &quot;&#xA;Length Sample1 Sample2&#xA;20 10 5&#xA;30 15 223&#xA;40 16 4923&#xA;50 293 239&#xA;60 3290 23&#xA;70 248 6&#xA;80 23 1&#xA;90 21 0&#xA;100 5 0&#xA;&quot;&#xA;&#xA;my_data &amp;lt;- read_delim(my_data, delim = &quot; &quot;)&#xA;&#xA;my_data_long &amp;lt;- pivot_longer(my_data, cols = c(&quot;Sample1&quot;, &quot;Sample2&quot;))&#xA;&#xA;&amp;gt; my_data_long&#xA;# A tibble: 18 x 3&#xA;   Length name    value&#xA;    &amp;lt;dbl&amp;gt; &amp;lt;chr&amp;gt;   &amp;lt;dbl&amp;gt;&#xA; 1     20 Sample1    10&#xA; 2     20 Sample2     5&#xA; 3     30 Sample1    15&#xA; 4     30 Sample2   223&#xA; 5     40 Sample1    16&#xA; 6     40 Sample2  4923&#xA; 7     50 Sample1   293&#xA; 8     50 Sample2   239&#xA; 9     60 Sample1  3290&#xA;10     60 Sample2    23&#xA;11     70 Sample1   248&#xA;12     70 Sample2     6&#xA;13     80 Sample1    23&#xA;14     80 Sample2     1&#xA;15     90 Sample1    21&#xA;16     90 Sample2     0&#xA;17    100 Sample1     5&#xA;18    100 Sample2     0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Moreover, boxplots are meaningful when you would like to visualize distributions, in your case you don't have all of your data points but only a summary of those, counts of read lengths. The best I could think of was using a third aesthetic to reflect the counts (after removing read lengths that were not observed):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# remove read lengths that are not observed&#xA;my_data_long &amp;lt;- my_data_long[my_data_long$value != 0,]&#xA;&#xA;ggplot(my_data_long,&#xA;       aes(x = name, y = Length)) +&#xA;  geom_boxplot() +&#xA;  geom_point(aes(size = value))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/McA5k.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/McA5k.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: In a boxplot, the quantiles reflect the number of data points but the way I draw the boxplot (using &lt;code&gt;Length&lt;/code&gt; and &lt;code&gt;name&lt;/code&gt; columns) do not make use of such information. A better way of visualising the data would be the following (and is not that different from what @gringer suggested):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(my_data_long,&#xA;       aes(x = name, y = Length)) +&#xA;  geom_point(aes(size = value))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8HbzT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8HbzT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2020-01-20T23:50:39.763" LastActivityDate="2020-01-20T23:50:39.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11200" PostTypeId="2" ParentId="11197" CreationDate="2020-01-21T00:11:07.753" Score="1" Body="&lt;p&gt;Based on your example, you can use count for each phase of cell cycle for each cell types in each condition. Here, I'm using &lt;code&gt;dplyr&lt;/code&gt; package to do that but most likely, you could have the same output using various methods:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;DF &amp;lt;- rbind(dfhead, dftail)&#xA;&#xA;library(dplyr)&#xA;DF_Count &amp;lt;- DF %&amp;gt;%group_by(orig.ident,Phase,old.ident) %&amp;gt;%&#xA;  count() %&amp;gt;%&#xA;  ungroup() %&amp;gt;%&#xA;  group_by(orig.ident,old.ident) %&amp;gt;%&#xA;  mutate(Freq = n/sum(n)*100)&#xA;&#xA;# A tibble: 8 x 5&#xA;# Groups:   orig.ident, old.ident [5]&#xA;  orig.ident Phase old.ident      n  Freq&#xA;  &amp;lt;chr&amp;gt;      &amp;lt;fct&amp;gt; &amp;lt;fct&amp;gt;      &amp;lt;int&amp;gt; &amp;lt;dbl&amp;gt;&#xA;1 treated    G1    Fibroblast     1 100  &#xA;2 treated    G1    Stellate       1 100  &#xA;3 treated    G2M   Tumor          1  33.3&#xA;4 treated    S     Tumor          2  66.7&#xA;5 treated    S     Mast cell      1 100  &#xA;6 untreated  G1    Acinar         1  16.7&#xA;7 untreated  G2M   Acinar         2  33.3&#xA;8 untreated  S     Acinar         3  50  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see, &lt;code&gt;DF_Count&lt;/code&gt; has the frequency for each cell types of each phase of the cell cycle in function of the condition. We can use &lt;code&gt;DF_Count&lt;/code&gt; to get the following plot. Using &lt;code&gt;facet_wrap&lt;/code&gt;, you can create two panels based on the condition column and thus represent the cell cycle of each cell types in function of the treatment condition:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;ggplot(DF_Count, aes(x = old.ident, y = Freq, fill = Phase))+&#xA;  geom_col()+&#xA;  geom_text(aes(label = paste(round(Freq, 2),&amp;quot;%&amp;quot;)),position = position_stack(vjust = 0.5))+&#xA;  facet_wrap(~orig.ident)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hcjcz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hcjcz.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Does it look what you are expecting ?&lt;/p&gt;&#xA;&lt;p&gt;NB: &lt;code&gt;DF&lt;/code&gt; here is the addition of your head and tail of your &lt;code&gt;all.combined&lt;/code&gt; dataframe.&lt;/p&gt;&#xA;" OwnerUserId="6437" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2020-01-21T00:11:07.753" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11201" PostTypeId="1" AcceptedAnswerId="11204" CreationDate="2020-01-21T09:11:14.040" Score="0" ViewCount="979" Body="&lt;p&gt;I know that with GRCh38, it is possible to extract the matching RefSeq transcripts for Ensembl genes/transcripts/proteins using Biomart from Ensembl. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, for GRCh37, this option does not exists. Using the following MySQL command returns Ensembl transcript IDs for matching RefSeq transcripts:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mysql -u anonymous -h ensembldb.ensembl.org -A -P 3337 -D homo_sapiens_core_99_37 -e &quot;SELECT transcript.stable_id, xref.display_label FROM transcript, object_xref, xref,external_db WHERE transcript.transcript_id = object_xref.ensembl_id AND object_xref.ensembl_object_type = 'Transcript' AND object_xref.xref_id = xref.xref_id AND xref.external_db_id = external_db.external_db_id AND external_db.db_name = 'RefSeq_mRNA';&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also, I was trying to play around with the &lt;a href=&quot;https://www.ensembl.org/info/docs/api/core/diagrams/Core.svg&quot; rel=&quot;nofollow noreferrer&quot;&gt;databases schema&lt;/a&gt; but I am not able to find any solution to problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ultimately, I wish to obtain a table like the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Gene stable ID      Transcript stable ID    Protein stable ID   RefSeq match transcript&#xA;ENSG00000120329     ENST00000239451         ENSP00000239451     NM_031947.4&#xA;ENSG00000164219     ENST00000419445         ENSP00000404676     NM_005023.4&#xA;ENSG00000150594     ENST00000280155         ENSP00000280155     NM_000681.4&#xA;ENSG00000113578     ENST00000337706         ENSP00000338548     NM_000800.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2020-01-22T20:47:01.650" Title="Is it possible to obtain RefSeq transcripts from Ensembl database for GRCh37 to match Ensembl Genes?" Tags="|ensembl|refseq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11203" PostTypeId="2" ParentId="11192" CreationDate="2020-01-21T10:39:06.400" Score="2" Body="&lt;p&gt;I think the python package &lt;a href=&quot;https://bioconvert.readthedocs.io/en/master/user_guide.html#explicit-conversion&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioconvert&lt;/a&gt; does what you want if you don't provide qualities: &lt;a href=&quot;https://github.com/bioconvert/bioconvert/blob/master/bioconvert/fasta2fastq.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/bioconvert/bioconvert/blob/master/bioconvert/fasta2fastq.py&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The following outputs lots of warnings for me, but does generate a fastq file &lt;code&gt;test.fq&lt;/code&gt; from an existing fasta file &lt;code&gt;test.fa&lt;/code&gt; (using dummy quality &quot;I&quot;, and adding &quot;None&quot; to sequence headers that do not have a comment part in the original fasta):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bioconvert fasta2fastq test.fa test.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(This requires installing bioconvert using &lt;code&gt;pip&lt;/code&gt; beforehand: &lt;code&gt;pip install bioconvert&lt;/code&gt;.)&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2020-01-22T11:45:44.253" LastActivityDate="2020-01-22T11:45:44.253" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11204" PostTypeId="2" ParentId="11201" CreationDate="2020-01-21T15:44:20.883" Score="4" Body="&lt;p&gt;The RefSeq match option in BioMart is from the &lt;a href=&quot;http://www.ensembl.org/info/genome/genebuild/mane.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matched Annotation from NCBI and EBI (MANE)&lt;/a&gt; collaboration between RefSeq and Ensembl. It has only been calculated for the up-to-date gene annotation on GRCh38 so cannot be obtained on GRCh37. You can get mapping from Ensembl to RefSeq transcripts through BioMart as RefSeq mRNA ID (refseq_mrna in R) but this is not a perfect match like the MANE, it is a mapping based on sequence similarity and similar genomic location, and there can be mismatches between them.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2020-01-21T15:44:20.883" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11205" PostTypeId="2" ParentId="11079" CreationDate="2020-01-21T19:29:07.280" Score="1" Body="&lt;p&gt;&lt;code&gt;for i in $(cat text.txt); do cp -v *${i}*vcf mydir; done&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="6343" LastActivityDate="2020-01-21T19:29:07.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11206" PostTypeId="1" CreationDate="2020-01-22T03:46:06.183" Score="2" ViewCount="505" Body="&lt;p&gt;I wrote a script which should changes IDs in a GFF3 file. Unfortunately, the below script has two problems.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;It attaches the new ID to &lt;code&gt;Parent&lt;/code&gt; which leads that &lt;code&gt;Parent&lt;/code&gt; contains the old and new id. How is it possible to keep only the new one?&lt;/li&gt;&#xA;&lt;li&gt;How is it possible to access the chromosome name?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This how to run the script &lt;code&gt;python renameIDgff3.py --gff3 braker_utr.gff3 --prefix ACTG --output braker_utr-newID.gff3&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/python3&#xA;import click&#xA;from BCBio.GFF import GFFExaminer&#xA;from BCBio import GFF&#xA;&#xA;@click.command()&#xA;@click.option('--gff3', help=&quot;Provide GFF3 file&quot;, required=True)&#xA;@click.option('--prefix', help=&quot;e.g. ASSCTG&quot;, required=True)&#xA;@click.option('--output', help=&quot;Keep GFF3 file&quot;, required=True)&#xA;def run(gff3, prefix, output):&#xA;    print(&quot;Hello&quot;)&#xA;    with open(output, &quot;w&quot;) as out_handle:&#xA;        for rec in GFF.parse(gff3):&#xA;&#xA;            for count, feature in enumerate(rec.features):&#xA;                print(&quot;count&quot;, count)&#xA;                print(feature)&#xA;                print(feature.qualifiers.get(&quot;Name&quot;))&#xA;                print(feature.sub_features)&#xA;                print(feature.sub_features[0].qualifiers.get(&quot;Name&quot;))&#xA;                print(&quot;!!!!change&quot;)&#xA;                feature.qualifiers[&quot;ID&quot;] = prefix + str(count).zfill(6)&#xA;&#xA;&#xA;                print(feature.sub_features[0].qualifiers[&quot;ID&quot;])&#xA;                id_extension =  feature.sub_features[0].qualifiers[&quot;ID&quot;][0].split('.')[1]&#xA;                feature.sub_features[0].qualifiers[&quot;ID&quot;] = prefix + str(count).zfill(6) + '.' + id_extension&#xA;                print(feature.sub_features[0].qualifiers[&quot;Parent&quot;])&#xA;                print(&quot;-----------&quot;)&#xA;&#xA;            GFF.write([rec], out_handle)&#xA;&#xA;if __name__ == '__main__':&#xA;    run()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Input file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NbV1Ch08    AUGUSTUS    gene    7015    29794   0.01    -   .   ID=g1;&#xA;NbV1Ch08    AUGUSTUS    mRNA    7015    29794   0.01    -   .   ID=g1.t1;Parent=g1&#xA;NbV1Ch08    AUGUSTUS    transcription_end_site  7015    7015    .   -   .   Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    three_prime_utr 7015    8531    0.2 -   .   ID=g1.t1.3UTR1;Parent=g1.t1&#xA;NbV1Ch08    AUGUSTUS    exon    7015    8747    .   -   .   ID=g1.t1.exon1;Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    stop_codon  8532    8534    .   -   0   Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    CDS 8532    8747    0.31    -   0   ID=g1.t1.CDS1;Parent=g1.t1&#xA;NbV1Ch08    AUGUSTUS    intron  8748    9191    0.49    -   .   Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    CDS 9192    9342    0.66    -   1   ID=g1.t1.CDS2;Parent=g1.t1&#xA;NbV1Ch08    AUGUSTUS    exon    9192    9342    .   -   .   ID=g1.t1.exon2;Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    intron  9343    9915    0.58    -   .   Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    CDS 9916    10006   0.71    -   2   ID=g1.t1.CDS3;Parent=g1.t1&#xA;NbV1Ch08    AUGUSTUS    exon    9916    10006   .   -   .   ID=g1.t1.exon3;Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    intron  10007   10101   0.74    -   .   Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    CDS 10102   10201   0.78    -   0   ID=g1.t1.CDS4;Parent=g1.t1&#xA;NbV1Ch08    AUGUSTUS    exon    10102   10201   .   -   .   ID=g1.t1.exon4;Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    intron  10202   10712   0.8 -   .   Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    CDS 10713   11107   0.11    -   2   ID=g1.t1.CDS5;Parent=g1.t1&#xA;NbV1Ch08    AUGUSTUS    exon    10713   11107   .   -   .   ID=g1.t1.exon5;Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    intron  11108   11569   0.07    -   .   Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    CDS 11570   12151   0.09    -   2   ID=g1.t1.CDS6;Parent=g1.t1&#xA;NbV1Ch08    AUGUSTUS    exon    11570   12151   .   -   .   ID=g1.t1.exon6;Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    intron  12152   12588   0.34    -   .   Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    CDS 12589   12717   0.39    -   2   ID=g1.t1.CDS7;Parent=g1.t1&#xA;NbV1Ch08    AUGUSTUS    exon    12589   12717   .   -   .   ID=g1.t1.exon7;Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    intron  12718   12789   0.42    -   .   Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    CDS 12790   13075   0.39    -   0   ID=g1.t1.CDS8;Parent=g1.t1&#xA;NbV1Ch08    AUGUSTUS    exon    12790   13075   .   -   .   ID=g1.t1.exon8;Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    intron  13076   14832   0.51    -   .   Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    CDS 14833   15009   0.39    -   0   ID=g1.t1.CDS9;Parent=g1.t1&#xA;NbV1Ch08    AUGUSTUS    exon    14833   15009   .   -   .   ID=g1.t1.exon9;Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    intron  15010   15278   0.59    -   .   Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    CDS 15279   15415   0.56    -   2   ID=g1.t1.CDS10;Parent=g1.t1&#xA;NbV1Ch08    AUGUSTUS    exon    15279   15415   .   -   .   ID=g1.t1.exon10;Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    intron  15416   15487   0.58    -   .   Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    CDS 15488   15612   0.96    -   1   ID=g1.t1.CDS11;Parent=g1.t1&#xA;NbV1Ch08    AUGUSTUS    exon    15488   15612   .   -   .   ID=g1.t1.exon11;Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    intron  15613   15706   0.96    -   .   Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    CDS 15707   15957   0.98    -   0   ID=g1.t1.CDS12;Parent=g1.t1&#xA;NbV1Ch08    AUGUSTUS    exon    15707   15958   .   -   .   ID=g1.t1.exon12;Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    start_codon 15955   15957   .   -   0   Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    five_prime_utr  15958   15958   0.99    -   .   ID=g1.t1.5UTR1;Parent=g1.t1&#xA;NbV1Ch08    AUGUSTUS    five_prime_utr  27458   28250   0.37    -   .   ID=g1.t1.5UTR2;Parent=g1.t1&#xA;NbV1Ch08    AUGUSTUS    exon    27458   28250   .   -   .   ID=g1.t1.exon13;Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    five_prime_utr  29272   29794   0.08    -   .   ID=g1.t1.5UTR3;Parent=g1.t1&#xA;NbV1Ch08    AUGUSTUS    exon    29272   29794   .   -   .   ID=g1.t1.exon14;Parent=g1.t1;&#xA;NbV1Ch08    AUGUSTUS    transcription_start_site    29794   29794   .   -   .   Parent=g1.t1;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##gff-version 3&#xA;##sequence-region NbV1Ch08 1 129222376&#xA;NbV1Ch08    AUGUSTUS    gene    7015    29794   0.01    -   .   ID=ACTG000000&#xA;NbV1Ch08    AUGUSTUS    mRNA    7015    29794   0.01    -   .   ID=ACTG000000.t1;Parent=g1,ACTG000000&#xA;NbV1Ch08    AUGUSTUS    transcription_end_site  7015    7015    .   -   .   Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    three_prime_utr 7015    8531    0.2 -   .   ID=g1.t1.3UTR1;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    exon    7015    8747    .   -   .   ID=g1.t1.exon1;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    stop_codon  8532    8534    .   -   0   Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    CDS 8532    8747    0.31    -   0   ID=g1.t1.CDS1;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    intron  8748    9191    0.49    -   .   Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    CDS 9192    9342    0.66    -   1   ID=g1.t1.CDS2;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    exon    9192    9342    .   -   .   ID=g1.t1.exon2;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    intron  9343    9915    0.58    -   .   Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    CDS 9916    10006   0.71    -   2   ID=g1.t1.CDS3;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    exon    9916    10006   .   -   .   ID=g1.t1.exon3;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    intron  10007   10101   0.74    -   .   Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    CDS 10102   10201   0.78    -   0   ID=g1.t1.CDS4;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    exon    10102   10201   .   -   .   ID=g1.t1.exon4;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    intron  10202   10712   0.8 -   .   Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    CDS 10713   11107   0.11    -   2   ID=g1.t1.CDS5;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    exon    10713   11107   .   -   .   ID=g1.t1.exon5;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    intron  11108   11569   0.07    -   .   Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    CDS 11570   12151   0.09    -   2   ID=g1.t1.CDS6;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    exon    11570   12151   .   -   .   ID=g1.t1.exon6;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    intron  12152   12588   0.34    -   .   Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    CDS 12589   12717   0.39    -   2   ID=g1.t1.CDS7;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    exon    12589   12717   .   -   .   ID=g1.t1.exon7;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    intron  12718   12789   0.42    -   .   Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    CDS 12790   13075   0.39    -   0   ID=g1.t1.CDS8;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    exon    12790   13075   .   -   .   ID=g1.t1.exon8;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    intron  13076   14832   0.51    -   .   Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    CDS 14833   15009   0.39    -   0   ID=g1.t1.CDS9;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    exon    14833   15009   .   -   .   ID=g1.t1.exon9;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    intron  15010   15278   0.59    -   .   Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    CDS 15279   15415   0.56    -   2   ID=g1.t1.CDS10;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    exon    15279   15415   .   -   .   ID=g1.t1.exon10;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    intron  15416   15487   0.58    -   .   Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    CDS 15488   15612   0.96    -   1   ID=g1.t1.CDS11;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    exon    15488   15612   .   -   .   ID=g1.t1.exon11;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    intron  15613   15706   0.96    -   .   Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    CDS 15707   15957   0.98    -   0   ID=g1.t1.CDS12;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    exon    15707   15958   .   -   .   ID=g1.t1.exon12;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    start_codon 15955   15957   .   -   0   Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    five_prime_utr  15958   15958   0.99    -   .   ID=g1.t1.5UTR1;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    five_prime_utr  27458   28250   0.37    -   .   ID=g1.t1.5UTR2;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    exon    27458   28250   .   -   .   ID=g1.t1.exon13;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    five_prime_utr  29272   29794   0.08    -   .   ID=g1.t1.5UTR3;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    exon    29272   29794   .   -   .   ID=g1.t1.exon14;Parent=g1.t1,ACTG000000.t1&#xA;NbV1Ch08    AUGUSTUS    transcription_start_site    29794   29794   .   -   .   Parent=g1.t1,ACTG000000.t1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance&lt;/p&gt;&#xA;" OwnerUserId="2477" LastActivityDate="2020-07-05T04:02:55.240" Title="renaming IDs in gff3 with BCBio.GFF" Tags="|sequence-annotation|python|gff3|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11207" PostTypeId="2" ParentId="11201" CreationDate="2020-01-22T04:48:40.857" Score="1" Body="&lt;p&gt;Try the GRCh37 version of BioMart, and the External reference &quot;RefSeq mRNA ID&quot; / &quot;RefSeq ncRNA ID&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://grch37.ensembl.org/biomart&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://grch37.ensembl.org/biomart&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the Perl code provided by BioMart, it looks like the field name should be &quot;refseq_mrna&quot; or &quot;refseq_ncrna&quot;. I'm not sure how that translates to the SQL code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel compelled to add a caution to this, though. GRCh37 is a very old build, having been &lt;em&gt;replaced&lt;/em&gt; by GRCh38 in &lt;a href=&quot;https://en.wikipedia.org/wiki/Reference_genome#Human_reference_genome&quot; rel=&quot;nofollow noreferrer&quot;&gt;December 2013&lt;/a&gt;. If possible, please find a way to use GRCh38 for your analysis instead, and don't create any &lt;em&gt;new&lt;/em&gt; software that depends on GRCh37. For example, gnomAD v3 supports GRCh38 (see &lt;a href=&quot;https://bioconductor.org/packages/release/data/annotation/html/MafDb.gnomAD.r3.0.GRCh38.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioconductor&lt;/a&gt;, or &lt;a href=&quot;https://macarthurlab.org/2019/10/16/gnomad-v3-0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-01-22T20:47:01.650" LastActivityDate="2020-01-22T20:47:01.650" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11208" PostTypeId="2" ParentId="321" CreationDate="2020-01-22T10:32:17.390" Score="0" Body="&lt;p&gt;In case BQSR is not an option (i.e. non-model organisms) it would be best to use some internal control sequence such as PhiX for illumina platform. Although this is supposed to be common practice some facilities ignore it. In principle, the machines should use these sequences as reference so that the scoring would be more accurate. In my experience the first 10-15 bases of the illumina reads always had lower quality. This can easily be seen in the nucleotide distributions. I would advise trimming the first 10-15 bases and quality based end-trimmingiIf the quality of the individual reads are important, such as low coverage resequencing or de-novo genome assembly applications.&lt;/p&gt;&#xA;" OwnerUserId="6802" LastEditorUserId="6802" LastEditDate="2020-01-22T11:10:16.963" LastActivityDate="2020-01-22T11:10:16.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11209" PostTypeId="1" AcceptedAnswerId="11211" CreationDate="2020-01-22T11:16:01.023" Score="-1" ViewCount="543" Body="&lt;p&gt;I have 3 data frames for treatment of 3 different drugs and in each of them I have the number of mutation types like insertion, deletion, SNP and total of mutations for each patients. In each group I have different number of patients like below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(dat1)&#xA;                patient  DEL  INS   SNP MNP total&#xA;1:    LP6008337-DNA_H06  927  773 40756   0 42456&#xA;2:    LP6008334-DNA_D02 1049  799 31009   0 32857&#xA;&amp;gt; dim(dat1)&#xA;[1] 21  6&#xA;&amp;gt; &#xA;&#xA; head(dat2)&#xA;                Patient   DEL  INS   SNP MNP total&#xA;1:    LP6008031-DNA_E01 13552 3374 62105   0 79031&#xA;2:    LP6005500-DNA_G01   539  500 43451   0 44490&#xA;&amp;gt; dim(dat2)&#xA;[1] 33  6&#xA;&amp;gt; &#xA;&#xA;&amp;gt; head(dat3)&#xA;                Patient   DEL   INS    SNP MNP  total&#xA;1:    LP6005935-DNA_F03 39168 16739  58095   0 114002&#xA;2:    LP6008269-DNA_D08   849   910 103501  11 105271&#xA;&#xA;&amp;gt; dim(dat3)&#xA;[1] 106   6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to show if the number of each mutational category and total is different for each drug by boxplot, violinplot or any type of visualization&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example this picture is very nice &lt;a href=&quot;https://i.stack.imgur.com/Kss8C.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Kss8C.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you help me?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="6723" LastEditDate="2020-01-23T11:02:31.247" LastActivityDate="2020-01-23T11:09:04.033" Title="Making a boxplot or violinplot for several dataframe with different number of samples" Tags="|r|visualization|ggplot2|wgs|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11210" PostTypeId="2" ParentId="11209" CreationDate="2020-01-22T12:47:52.560" Score="1" Body="&lt;p&gt;If you use a violin plot instead and ggplot2 to make it then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;geom_violin(...not relevant stuff..., scale=&quot;count&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;will scale each violin plot according to the number of actual entries in it. I often find this nice so small groups look super skinny and people don't focus on random changes in them so much.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-01-22T12:47:52.560" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11211" PostTypeId="2" ParentId="11209" CreationDate="2020-01-22T13:13:08.300" Score="2" Body="&lt;p&gt;To help us help you, please make your data available with &lt;code&gt;dput()&lt;/code&gt; next time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To achieve your goal with &lt;code&gt;ggplot2&lt;/code&gt;, you would need all of your data in one data frame and in the &quot;long format&quot;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dat1 &amp;lt;- &quot;&#xA;Patient DEL INS SNP MNP total&#xA;LP6008337-DNA_H06 927 773 40756 0 42456&#xA;LP6008334-DNA_D02 1049 799 31009 0 32857&#xA;&quot;&#xA;&#xA;dat2 &amp;lt;- &quot;&#xA;Patient DEL INS SNP MNP total&#xA;LP6008031-DNA_E01 13552 3374 62105 0 79031&#xA;LP6005500-DNA_G01 539 500 43451 0 44490&#xA;&quot;&#xA;&#xA;dat3 &amp;lt;- &quot;&#xA;Patient DEL INS SNP MNP total&#xA;LP6005935-DNA_F03 39168 16739 58095 0 114002&#xA;LP6008269-DNA_D08 849 910 103501 11 105271&#xA;&quot;&#xA;&#xA;# read data and add a column indicating data frame of origin (drug in this case)&#xA;dat1 &amp;lt;- readr::read_delim(dat1, delim = &quot; &quot;)&#xA;dat1&lt;span class=&quot;math-container&quot;&gt;$drug &amp;lt;- &quot;drug1&quot;&#xA;dat2 &amp;lt;- readr::read_delim(dat2, delim = &quot; &quot;)&#xA;dat2$&lt;/span&gt;drug &amp;lt;- &quot;drug2&quot;&#xA;dat3 &amp;lt;- readr::read_delim(dat3, delim = &quot; &quot;)&#xA;dat3$drug &amp;lt;- &quot;drug3&quot;&#xA;&#xA;# bind rows&#xA;dat &amp;lt;- rbind(dat1, dat2, dat3)&#xA;&#xA;&amp;gt; dat&#xA;# A tibble: 6 x 7&#xA;  Patient             DEL   INS    SNP   MNP  total drug &#xA;  &amp;lt;chr&amp;gt;             &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt; &amp;lt;chr&amp;gt;&#xA;1 LP6008337-DNA_H06   927   773  40756     0  42456 drug1&#xA;2 LP6008334-DNA_D02  1049   799  31009     0  32857 drug1&#xA;3 LP6008031-DNA_E01 13552  3374  62105     0  79031 drug2&#xA;4 LP6005500-DNA_G01   539   500  43451     0  44490 drug2&#xA;5 LP6005935-DNA_F03 39168 16739  58095     0 114002 drug3&#xA;6 LP6008269-DNA_D08   849   910 103501    11 105271 drug3&#xA;&#xA;# convert data to long format, ggplot() expects that&#xA;dat_long &amp;lt;- tidyr::pivot_longer(dat, cols = 2:6)&#xA;&#xA;&amp;gt; dat_long&#xA;# A tibble: 30 x 4&#xA;   Patient           drug  name  value&#xA;   &amp;lt;chr&amp;gt;             &amp;lt;chr&amp;gt; &amp;lt;chr&amp;gt; &amp;lt;dbl&amp;gt;&#xA; 1 LP6008337-DNA_H06 drug1 DEL     927&#xA; 2 LP6008337-DNA_H06 drug1 INS     773&#xA; 3 LP6008337-DNA_H06 drug1 SNP   40756&#xA; 4 LP6008337-DNA_H06 drug1 MNP       0&#xA; 5 LP6008337-DNA_H06 drug1 total 42456&#xA; 6 LP6008334-DNA_D02 drug1 DEL    1049&#xA;&#xA;ggplot(dat_long,&#xA;                aes(x = name, y = value)) +&#xA;  geom_boxplot() +&#xA;  facet_wrap(~drug)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xURLG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xURLG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: setting &lt;code&gt;scales&lt;/code&gt; argument of &lt;code&gt;facet_wrap()&lt;/code&gt; to &quot;free&quot; could be helpful&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(dat_long,&#xA;       aes(x = name, y = value)) +&#xA;  geom_boxplot() +&#xA;  facet_wrap(~drug, scales = &quot;free&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SVenb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SVenb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2020-01-23T11:09:04.033" LastActivityDate="2020-01-23T11:09:04.033" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11212" PostTypeId="2" ParentId="2378" CreationDate="2020-01-22T13:26:27.567" Score="0" Body="&lt;p&gt;I've got the RIP-seq analysis outputs. We used an isotype control antibody VS antibody against an RBP to pull down corresponding RNAs! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did the following pipeline: Hisat2 (mapping the reads after QC) -&gt; htseq-count (getting read counts) -&gt; DESeq2 (getting enriched RNAs in test VS isotype control) -&gt; volcano plot shows up-regulated and down-regulated genes in the test sample. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Logically, I should have seen just enriched RNAs (up-regulated genes), but surprisingly there are some RNAs down-regulated in the test sample (or up-regulated in the isotype control sample)!!!!!&#xA;How can we explain and interpret this???? it seems some genes are pull down in isotype control sample as well. Are these background? Is my computational pipeline right? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW, it's worth to mention that all enriched RNA (called up-regulated genes in DESeq2 pipeline) are significantly relevant and meaningful to my experiment. However, I am not sure how much confident I can be on this output? And how should I explain down-regulated RNAs? because normally we just expect enriched RNAs in RIP-seq experiment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any thoughts?&lt;/p&gt;&#xA;" OwnerUserId="6804" LastActivityDate="2020-01-22T13:26:27.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11213" PostTypeId="2" ParentId="11192" CreationDate="2020-01-22T14:42:30.503" Score="2" Body="&lt;p&gt;Here's a quick one-liner with no dependencies other than the UNIX core utils and Perl.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat seqs.fasta | paste - - | perl -ne 'chomp; s/^&amp;gt;/@/; @v = split /\t/; printf(&quot;%s\n%s\n+\n%s\n&quot;, $v[0], $v[1], &quot;B&quot;x length($v[1]))' &amp;gt; seqs.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Caveat: assumes your Fasta sequences are not wrapped, i.e. two lines per record.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2020-01-22T14:42:30.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11214" PostTypeId="1" CreationDate="2020-01-22T17:12:39.807" Score="1" ViewCount="69" Body="&lt;p&gt;I searched the Python documentation for libSBML but couldn't find a staightforward answer to this question. For a given SBML model, how can I use Python libSBML to retrieve the values of the &lt;code&gt;id&lt;/code&gt; and &lt;code&gt;volume&lt;/code&gt; attributes for a given compartment?&lt;/p&gt;&#xA;" OwnerUserId="4894" LastEditorUserId="4913" LastEditDate="2020-02-11T06:35:22.087" LastActivityDate="2020-02-11T06:35:22.087" Title="How do I get the compartment Id and volume from SBML using libSBML Python?" Tags="|sbml|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11215" PostTypeId="2" ParentId="11214" CreationDate="2020-01-22T17:22:44.707" Score="1" Body="&lt;p&gt;There aren't any examples in the python docs on how to do this. The best bet is to search the C/C++ docs which are more extensive. There is a python example in the docs which shows how to retrieve the number of compartments in a model but it doesn't go further. The code below shows how you can retrieve the information requested, assuming you have a model called testmodel.xml:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import libsbml&#xA;&#xA;document = libsbml.readSBML(&quot;testmodel.xml&quot;)&#xA;&#xA;if document.getNumErrors() &amp;gt; 0:&#xA;   print(&quot;Encountered the following SBML errors:&quot; + &quot;\n&quot;)&#xA;   document.printErrors()&#xA;else:       &#xA;   model = document.getModel()&#xA;&#xA;   nCompartments = model.getNumCompartments()&#xA;&#xA;   print (&quot;Number of compartments: &quot;, nCompartments)&#xA;   for i in range (nCompartments):&#xA;       comp = model.getCompartment(i)&#xA;&#xA;       print (&quot;Id = &quot;, comp.getId())&#xA;       print (&quot;Volume = &quot;, comp.getVolume())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4894" LastActivityDate="2020-01-22T17:22:44.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11216" PostTypeId="2" ParentId="8908" CreationDate="2020-01-22T18:31:42.317" Score="2" Body="&lt;p&gt;What I wanted to do is called &quot;Consensus Sequence&quot;. Two steps were needed:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools mpileup -Ou -f ref.fa input.bam | bcftools call -Ou -mv | bcftools norm -f ref.fa Oz -o output.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After that you can create the consensus:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools consensus -f ref.fa output.vcf.gz &amp;gt; out.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4362" LastActivityDate="2020-01-22T18:31:42.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11217" PostTypeId="2" ParentId="7307" CreationDate="2020-01-22T22:58:48.517" Score="1" Body="&lt;p&gt;The nomenclature is no longer being followed as NCBI is moving all the files to S3/Google cloud. &lt;a href=&quot;https://github.com/saketkc/pysradb&quot; rel=&quot;nofollow noreferrer&quot;&gt;pysradb&lt;/a&gt; allows you to fetch the metadata and get the URLs for each individual SRR. See examples in &lt;a href=&quot;https://colab.research.google.com/drive/1tOVBg7vPnhTYNjyFQTvN0K9jGT3MyrVb&quot; rel=&quot;nofollow noreferrer&quot;&gt;this notebook&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also download an entire project by doing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pysradb download -p &amp;lt;SRP&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="161" LastActivityDate="2020-01-22T22:58:48.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11218" PostTypeId="2" ParentId="7027" CreationDate="2020-01-22T23:00:28.657" Score="0" Body="&lt;p&gt;You can get metadata for an entire project using &lt;a href=&quot;https://github.com/saketkc/pysradb&quot; rel=&quot;nofollow noreferrer&quot;&gt;pysradb&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; pysradb metadata SRP063732 --detailed&#xA;&#xA;study_accession experiment_accession experiment_title                                                       experiment_desc                                                        organism_taxid  organism_name   library_strategy library_source library_selection sample_accession sample_title instrument           total_spots total_size   run_accession run_total_spots run_total_bases run_alias             sra_url                                                                                 experiment_alias isolate                 dev_stage sex   tissue             treatment     BioSampleModel           &#xA; SRP063732       SRX1249861           Bisulfite-seq of rhesus macaque 26168_HeavyDrinker: Nucleus accumbens  Bisulfite-seq of rhesus macaque 26168_HeavyDrinker: Nucleus accumbens  9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071080                    Illumina HiSeq 2000  89051187    11830170462  SRR2420581    89051187        17988339774     26168_NaC_HD_files    https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420581/SRR2420581.1                   26168_NucleusAccumbens  Adult     male  Nucleus accumbens  HeavyEthanol  Model organism or animal&#xA; SRP063732       SRX1249860           Bisulfite-seq of rhesus macaque 26148_HeavyDrinker: Nucleus accumbens  Bisulfite-seq of rhesus macaque 26148_HeavyDrinker: Nucleus accumbens  9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071081                    Illumina HiSeq 2000  87335402    11763248173  SRR2420580    87335402        17641751204     26148_NaC_HD_files    https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420580/SRR2420580.1                   26148_NucleusAccumbens  Adult     male  Nucleus accumbens  HeavyEthanol  Model organism or animal&#xA; SRP063732       SRX1249859           Bisulfite-seq of rhesus macaque 25822_HeavyDrinker: Nucleus accumbens  Bisulfite-seq of rhesus macaque 25822_HeavyDrinker: Nucleus accumbens  9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071082                    Illumina HiSeq 2000  83544130    9481342835   SRR2420579    83544130        13784781450     25822_NaC_HD_files    https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420579/SRR2420579.1                   25822_NucleusAccumbens  Adult     male  Nucleus accumbens  HeavyEthanol  Model organism or animal&#xA; SRP063732       SRX1249858           Bisulfite-seq of rhesus macaque 25154_HeavyDrinker: Nucleus accumbens  Bisulfite-seq of rhesus macaque 25154_HeavyDrinker: Nucleus accumbens  9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071083                    Illumina HiSeq 2000  115476876   13003165573  SRR2420578    115476876       19053684540     25154_NaC_HD_files    https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420578/SRR2420578.1                   25154_NucleusAccumbens  Adult     male  Nucleus accumbens  HeavyEthanol  Model organism or animal&#xA; SRP063732       SRX1249857           Bisulfite-seq of rhesus macaque 23784_HeavyDrinker: Nucleus accumbens  Bisulfite-seq of rhesus macaque 23784_HeavyDrinker: Nucleus accumbens  9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071084                    Illumina HiSeq 2000  101539017   13697691917  SRR2420577    101539017       20510881434     23784_NaC_HD_files    https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420577/SRR2420577.1                   23784_NucleusAccumbens  Adult     male  Nucleus accumbens  HeavyEthanol  Model organism or animal&#xA; SRP063732       SRX1249856           Bisulfite-seq of rhesus macaque 23779_HeavyDrinker: Nucleus accumbens  Bisulfite-seq of rhesus macaque 23779_HeavyDrinker: Nucleus accumbens  9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071085                    Illumina HiSeq 2000  98394582    13326784419  SRR2420576    98394582        19875705564     23779_NaC_HD_files    https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420576/SRR2420576.1                   23779_NucleusAccumbens  Adult     male  Nucleus accumbens  HeavyEthanol  Model organism or animal&#xA; SRP063732       SRX1249855           Bisulfite-seq of rhesus macaque 23773_HeavyDrinker: Nucleus accumbens  Bisulfite-seq of rhesus macaque 23773_HeavyDrinker: Nucleus accumbens  9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071086                    Illumina HiSeq 2000  84442355    11218164664  SRR2420575    84442355        17057355710     23773_NaC_HD_files    https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420575/SRR2420575.1                   23773_NucleusAccumbens  Adult     male  Nucleus accumbens  HeavyEthanol  Model organism or animal&#xA; SRP063732       SRX1249854           Bisulfite-seq of rhesus macaque 23764_HeavyDrinker: Nucleus accumbens  Bisulfite-seq of rhesus macaque 23764_HeavyDrinker: Nucleus accumbens  9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071087                    Illumina HiSeq 2000  87197006    11291883405  SRR2420574    87197006        17613795212     23764_NaC_HD_files    https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420574/SRR2420574.1                   23764_NucleusAccumbens  Adult     male  Nucleus accumbens  HeavyEthanol  Model organism or animal&#xA; SRP063732       SRX1249853           Bisulfite-seq of rhesus macaque 23762_HeavyDrinker: Nucleus accumbens  Bisulfite-seq of rhesus macaque 23762_HeavyDrinker: Nucleus accumbens  9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071088                    Illumina HiSeq 2000  89455608    11917710767  SRR2420573    89455608        18070032816     23762_NaC_HD_files    https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420573/SRR2420573.1                   23762_NucleusAccumbens  Adult     male  Nucleus accumbens  HeavyEthanol  Model organism or animal&#xA; SRP063732       SRX1249852           Bisulfite-seq of rhesus macaque 25811_LowDrinker: Nucleus accumbens    Bisulfite-seq of rhesus macaque 25811_LowDrinker: Nucleus accumbens    9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071089                    Illumina HiSeq 2000  88216999    10285962574  SRR2420572    88216999        14555804835     25811_NaC_LMD _files  https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420572/SRR2420572.1                   25811_NucleusAccumbens  Adult     male  Nucleus accumbens  LightEthanol  Model organism or animal&#xA; SRP063732       SRX1249851           Bisulfite-seq of rhesus macaque 25790_LowDrinker: Nucleus accumbens    Bisulfite-seq of rhesus macaque 25790_LowDrinker: Nucleus accumbens    9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071090                    Illumina HiSeq 2000  94731241    10748979142  SRR2420571    94731241        15630654765     25790_NaC_LMD _files  https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420571/SRR2420571.1                   25790_NucleusAccumbens  Adult     male  Nucleus accumbens  LightEthanol  Model organism or animal&#xA; SRP063732       SRX1249850           Bisulfite-seq of rhesus macaque 25742_LowDrinker: Nucleus accumbens    Bisulfite-seq of rhesus macaque 25742_LowDrinker: Nucleus accumbens    9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071091                    Illumina HiSeq 2000  113512952   17943763654  SRR2420570    113512952       22929616304     25742_NaC_LMD _files  https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420570/SRR2420570.1                   25742_NucleusAccumbens  Adult     male  Nucleus accumbens  LightEthanol  Model organism or animal&#xA; SRP063732       SRX1249849           Bisulfite-seq of rhesus macaque 25240_LowDrinker: Nucleus accumbens    Bisulfite-seq of rhesus macaque 25240_LowDrinker: Nucleus accumbens    9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071092                    Illumina HiSeq 2000  81002663    9232986327   SRR2420569    81002663        13365439395     25240_NaC_LMD _files  https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420569/SRR2420569.1                   25240_NucleusAccumbens  Adult     male  Nucleus accumbens  LightEthanol  Model organism or animal&#xA; SRP063732       SRX1249848           Bisulfite-seq of rhesus macaque 25184_LowDrinker: Nucleus accumbens    Bisulfite-seq of rhesus macaque 25184_LowDrinker: Nucleus accumbens    9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071093                    Illumina HiSeq 2000  110228135   12594727479  SRR2420568    110228135       18187642275     25184_NaC_LMD _files  https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420568/SRR2420568.1                   25184_NucleusAccumbens  Adult     male  Nucleus accumbens  LightEthanol  Model organism or animal&#xA; SRP063732       SRX1249847           Bisulfite-seq of rhesus macaque 25157_LowDrinker: Nucleus accumbens    Bisulfite-seq of rhesus macaque 25157_LowDrinker: Nucleus accumbens    9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071094                    Illumina HiSeq 2000  97249804    11026452312  SRR2420567    97249804        16046217660     25157_NaC_LMD _files  https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420567/SRR2420567.1                   25157_NucleusAccumbens  Adult     male  Nucleus accumbens  LightEthanol  Model organism or animal&#xA; SRP063732       SRX1249846           Bisulfite-seq of rhesus macaque 21119_LowDrinker: Nucleus accumbens    Bisulfite-seq of rhesus macaque 21119_LowDrinker: Nucleus accumbens    9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071095                    Illumina HiSeq 2000  68899204    9241244635   SRR2420566    68899204        13917639208     21119_NaC_LMD _files  https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420566/SRR2420566.1                   21119_NucleusAccumbens  Adult     male  Nucleus accumbens  LightEthanol  Model organism or animal&#xA; SRP063732       SRX1249845           Bisulfite-seq of rhesus macaque 26104_Control: Nucleus accumbens       Bisulfite-seq of rhesus macaque 26104_Control: Nucleus accumbens       9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071096                    Illumina HiSeq 2000  84859783    9604170375   SRR2420565    84859783        14001864195     26104_NaC_Ctrl_files  https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420565/SRR2420565.1                   26104_NucleusAccumbens  Adult     male  Nucleus accumbens  Control       Model organism or animal&#xA; SRP063732       SRX1249844           Bisulfite-seq of rhesus macaque 26082_Control: Nucleus accumbens       Bisulfite-seq of rhesus macaque 26082_Control: Nucleus accumbens       9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071097                    Illumina HiSeq 2000  73346886    8248399265   SRR2420564    73346886        12102236190     26082_NaC_Ctrl_files  https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420564/SRR2420564.1                   26082_NucleusAccumbens  Adult     male  Nucleus accumbens  Control       Model organism or animal&#xA; SRP063732       SRX1249843           Bisulfite-seq of rhesus macaque 25207_Control: Nucleus accumbens       Bisulfite-seq of rhesus macaque 25207_Control: Nucleus accumbens       9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071098                    Illumina HiSeq 2000  77306973    12235827725  SRR2420563    77306973        15616008546     25207_NaC_Ctrl_files  https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420563/SRR2420563.1                   25207_NucleusAccumbens  Adult     male  Nucleus accumbens  Control       Model organism or animal&#xA; SRP063732       SRX1249842           Bisulfite-seq of rhesus macaque 25407_Control: Nucleus accumbens       Bisulfite-seq of rhesus macaque 25407_Control: Nucleus accumbens       9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071099                    Illumina HiSeq 2000  83985968    13355795218  SRR2420562    83985968        16965165536     25407_NaC_Ctrl_files  https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420562/SRR2420562.1                   25407_NucleusAccumbens  Adult     male  Nucleus accumbens  Control       Model organism or animal&#xA; SRP063732       SRX1249841           Bisulfite-seq of rhesus macaque 24818_Control: Nucleus accumbens       Bisulfite-seq of rhesus macaque 24818_Control: Nucleus accumbens       9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071100                    Illumina HiSeq 2000  72782928    8484613123   SRR2420561    72782928        12009183120     24818_NaC_Ctrl_files  https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420561/SRR2420561.1                   24818_NucleusAccumbens  Adult     male  Nucleus accumbens  Control       Model organism or animal&#xA; SRP063732       SRX1249840           Bisulfite-seq of rhesus macaque 24698_Control: Nucleus accumbens       Bisulfite-seq of rhesus macaque 24698_Control: Nucleus accumbens       9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071101                    Illumina HiSeq 2000  79414016    12627204820  SRR2420560    79414016        16041631232     24698_NaC_Ctrl_files  https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420560/SRR2420560.1                   24698_NucleusAccumbens  Adult     male  Nucleus accumbens  Control       Model organism or animal&#xA; SRP063732       SRX1249839           Bisulfite-seq of rhesus macaque 24340_Control: Nucleus accumbens       Bisulfite-seq of rhesus macaque 24340_Control: Nucleus accumbens       9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071102                    Illumina HiSeq 2000  84772576    13436053136  SRR2420559    84772576        17124060352     24340_NaC_Ctrl_files  https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420559/SRR2420559.1                   24340_NucleusAccumbens  Adult     male  Nucleus accumbens  Control       Model organism or animal&#xA; SRP063732       SRX1249838           Bisulfite-seq of rhesus macaque 22770_Control: Nucleus accumbens       Bisulfite-seq of rhesus macaque 22770_Control: Nucleus accumbens       9544            Macaca mulatta  Bisulfite-Seq    GENOMIC        other             SRS1071103                    Illumina HiSeq 2000  83310630    13224887149  SRR2420558    83310630        16828747260     22770_NaC_Ctrl_files  https://sra-download.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR2420558/SRR2420558.1                   22770_NucleusAccumbens  Adult     male  Nucleus accumbens  Control       Model organism or animal&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="161" LastEditorUserId="161" LastEditDate="2020-01-22T23:06:16.333" LastActivityDate="2020-01-22T23:06:16.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11220" PostTypeId="1" AcceptedAnswerId="11221" CreationDate="2020-01-23T14:03:12.697" Score="-2" ViewCount="44" Body="&lt;p&gt;I have this expression matrix&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(dat[1:8,1:4])&#xA;  Gene.Symbol        p1        p2         p3&#xA;1       STAT1     7.6159     7.6784     7.4430&#xA;2       STAT1     7.4435     7.2850     7.1250&#xA;3       STAT1     7.8925     7.9269     7.4956&#xA;4       STAT1     7.5867     7.9285     7.6741&#xA;5       GAPDH     8.4861     8.3189     9.0913&#xA;6       GAPDH     9.0206     8.0813     9.2360&#xA;&amp;gt; dim(dat)&#xA;[1] 58324   276&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you are seeing, I have repeated genes, so how I take average over expression values for redundant genes finishing with one unique expression value for each gene for each sample in columns? &lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-01-23T14:09:32.170" LastActivityDate="2020-01-23T14:22:18.627" Title="Calculating mean accross rows with repeated entries in R" Tags="|r|microarray|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11221" PostTypeId="2" ParentId="11220" CreationDate="2020-01-23T14:16:15.140" Score="3" Body="&lt;p&gt;&lt;strong&gt;Using &lt;code&gt;group_by&lt;/code&gt; from &lt;code&gt;dplyr&lt;/code&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use &lt;code&gt;group_by&lt;/code&gt; function from &lt;code&gt;dplyr&lt;/code&gt; to calculate the mean of each gene symbol in each samples:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(dplyr)&#xA;dat %&amp;gt;% &#xA;  group_by(Gene.Symbol) %&amp;gt;% &#xA;  summarise_at(vars(contains(&quot;p&quot;)), funs(mean(., na.rm = TRUE)))&#xA;&#xA;# A tibble: 2 x 4&#xA;  Gene.Symbol    p1    p2    p3&#xA;  &amp;lt;chr&amp;gt;       &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;&#xA;1 GAPDH        8.75  8.20  9.16&#xA;2 STAT1        7.63  7.70  7.43&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Using &lt;code&gt;data.table&lt;/code&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using &lt;code&gt;data.table&lt;/code&gt; to speed up the calculation, you can do:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(data.table)&#xA;setDT(dat)&#xA;dat[, lapply(.SD, mean), by = Gene.Symbol]&#xA;&#xA;   Gene.Symbol      p1     p2       p3&#xA;1:       STAT1 7.63465 7.7047 7.434425&#xA;2:       GAPDH 8.75335 8.2001 9.163650&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6437" LastEditorUserId="6437" LastEditDate="2020-01-23T14:22:18.627" LastActivityDate="2020-01-23T14:22:18.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11222" PostTypeId="1" CreationDate="2020-01-23T16:37:35.753" Score="0" ViewCount="9297" Body="&lt;p&gt;I'm attempting to plot a stacked barplot with ggplot2 with this code:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;ggplot(CC, aes(x = Condition, y = Percent, fill = Cell_Cycle))+&#xA;  geom_bar(stat = &quot;identity&quot;)+&#xA;  geom_text(aes(label = paste(round(Percent,2),&quot;%&quot;)), position = position_stack(vjust =  0.5))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YTjJJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YTjJJ.png&quot; alt=&quot;stacked bar graph&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Question: How do I go about adding error bars to the graph?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried this code:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;ggplot(DF, aes(x = Condition, y = Percent, fill = Cell_Type))+&#xA;     geom_bar(stat = &quot;identity&quot;)+&#xA;     geom_text(aes(label = paste(round(Percent,2),&quot;%&quot;)), position = position_stack(vjust =  0.5) + geom_errorbar(aes(ymin=Condition, ymax=Condition), width = .2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But got this error: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error: Cannot add ggproto objects together.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;output of dput(DF)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(Cell_Type = structure(c(1L, 2L, 3L, 4L, 5L, 6L, &#xA;7L, 1L, 2L, 3L, 4L, 5L, 6L, 7L), .Label = c(&quot;Fibroblast&quot;, &quot;T cell&quot;, &#xA;&quot;Macrophage&quot;, &quot;Tumor&quot;, &quot;Islets of Langerhans&quot;, &quot;Endothelial&quot;, &#xA;&quot;B cell&quot;), class = &quot;factor&quot;), Condition = structure(c(1L, 1L, &#xA;1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L), .Label = c(&quot;treated&quot;, &#xA;&quot;untreated&quot;), class = &quot;factor&quot;), Freq = c(6051L, 1892L, 1133L, &#xA;657L, 116L, 868L, 832L, 5331L, 3757L, 1802L, 835L, 287L, 704L, &#xA;256L), Percent = c(52.3941466793662, 16.3823707680319, 9.8103731924842, &#xA;5.68880422547407, 1.00441596675037, 7.51580223395965, 7.20408693393367, &#xA;41.0962072155412, 28.9623805118717, 13.8914585260561, 6.43694110391613, &#xA;2.21245760098674, 5.42707369719396, 1.97348134443417)), class = c(&quot;grouped_df&quot;, &#xA;&quot;tbl_df&quot;, &quot;tbl&quot;, &quot;data.frame&quot;), row.names = c(NA, -14L), groups = structure(list(&#xA;    Condition = structure(1:2, .Label = c(&quot;treated&quot;, &quot;untreated&quot;&#xA;    ), class = &quot;factor&quot;), .rows = list(1:7, 8:14)), row.names = c(NA, &#xA;-2L), class = c(&quot;tbl_df&quot;, &quot;tbl&quot;, &quot;data.frame&quot;), .drop = TRUE))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6810" LastEditorUserId="6810" LastEditDate="2020-01-23T19:17:47.933" LastActivityDate="2020-01-23T20:00:13.053" Title="stacked bargraph with error bars" Tags="|ggplot2|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="11223" PostTypeId="1" AcceptedAnswerId="11252" CreationDate="2020-01-23T18:51:25.313" Score="0" ViewCount="384" Body="&lt;p&gt;I'm using GATK4 to build a somatic variant calling pipeline. The pipeline uses MultiQC to aggregate quality control data, and one of the QC measures reported is base quality score recalibration from GATK4's &lt;code&gt;BaseRecalibraton&lt;/code&gt;. The description in MultiQC says&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This plot shows the distribution of base quality scores in each sample before and after base quality score recalibration (BQSR). Applying BQSR should broaden the distribution of base quality scores.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rxcP5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rxcP5.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The black line is for my tumor sample and blue line is my normal sample. I don't understand how the before and after distributions are &lt;strong&gt;both&lt;/strong&gt; shown in the plot as I only see 1 distribution per sample. Am I missing something, or should there be a &quot;Post-calibration Count&quot; plot that isn't shown? If nothing is missing, how should I be interpreting the plot?&lt;/p&gt;&#xA;" OwnerUserId="6674" LastEditorUserId="6674" LastEditDate="2020-01-23T19:07:32.927" LastActivityDate="2020-01-27T21:20:53.797" Title="Interpreting MultiQC plot of GATK BaseRecalibration data?" Tags="|fasta|ngs|variant-calling|gatk|qc|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11224" PostTypeId="2" ParentId="11222" CreationDate="2020-01-23T19:52:00.803" Score="1" Body="&lt;p&gt;If you want to indicate the error or the uncertainty for each cell type, you need to replicate your experiments, ccount each cell type in each replicate, calculate the average percentage of each cell type in each condition and the associated standard deviation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question about how to plot a stacked bargraph with error bar, first this option is not integrated into &lt;code&gt;ggplot2&lt;/code&gt; (see this discussion from the creator of ggplot2: &lt;a href=&quot;https://github.com/tidyverse/ggplot2/issues/1079&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/tidyverse/ggplot2/issues/1079&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, you need to calculate the position of each error bar by hand. You can do that by using &lt;code&gt;dplyr&lt;/code&gt;. Here, I set the same error bar for everyone as you don't know what is the standard deviation in your data but at least you can observe the structure of the dataframe:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(dplyr)&#xA;library(ggplot2)&#xA;DF %&amp;gt;% mutate(SD = 2) %&amp;gt;%&#xA;  mutate(Cell_Type = factor(Cell_Type, levels = c(&quot;B cell&quot;,&quot;Endothelial&quot;,&quot;Islets of Langerhans&quot;,&quot;Tumor&quot;,&quot;Macrophage&quot;,&quot;T cell&quot;,&quot;Fibroblast&quot;))) %&amp;gt;%&#xA;  group_by(Condition) %&amp;gt;%&#xA;  mutate(SDPos = cumsum(Percent)) %&amp;gt;%&#xA;  ggplot(aes(x = Condition, y = Percent, fill = Cell_Type))+&#xA;  geom_bar(stat = &quot;identity&quot;)+&#xA;  geom_text(aes(label = paste(round(Percent,2),&quot;%&quot;)), position = position_stack(vjust =  0.5))+&#xA;  geom_errorbar(aes(ymin = SDPos-SD, ymax = SDPos+SD), width = 0.3, position = &quot;identity&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Bn98R.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Bn98R.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see, it's pretty ugly... so, if you need to add error bar, I will rather advised you to do a dodged barchart as the following:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(dplyr)&#xA;library(ggplot2)&#xA;DF %&amp;gt;% mutate(SD = 2) %&amp;gt;%&#xA;  ggplot(aes(x = Cell_Type, y = Percent, fill = Condition))+&#xA;  geom_bar(stat = &quot;identity&quot;, position = position_dodge())+&#xA;  geom_text(aes(label = paste(round(Percent,2),&quot;%&quot;)), position = position_dodge(0.9), vjust =-2, size = 3)+&#xA;  geom_errorbar(aes(ymin = Percent-SD, ymax = Percent+SD), width = 0.3, position = position_dodge(0.9))+&#xA;  theme(axis.text.x = element_text(angle = 45, hjust = 1))+&#xA;  scale_y_continuous(limits = c(-2,60))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/M22VR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/M22VR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To my opinion, it makes easier the comparison of the distribution of each cell type between conditions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that it answers your question.&lt;/p&gt;&#xA;" OwnerUserId="6437" LastEditorUserId="6437" LastEditDate="2020-01-23T20:00:13.053" LastActivityDate="2020-01-23T20:00:13.053" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11225" PostTypeId="1" CreationDate="2020-01-24T12:10:22.327" Score="1" ViewCount="54" Body="&lt;p&gt;I am working with an in silico model to measure the distances between drugs targets and disease proteins of a disease. I would want to compare these distances with the distances of random networks but I don't know which is the best way to do the comparison. For instance, if I have 300 drug targets and 500 disease proteins mapped in the interactome and I want to measure the distance between them and then compare to random networks with the same features. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I generate these random networks? Choosing the same number of drug targets (300) and disease genes (500) randomly in my referral network? Choosing the same number of nodes (300 and 500) and the same number of links?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd appreciate your help!&lt;/p&gt;&#xA;" OwnerUserId="6815" LastActivityDate="2020-11-01T22:04:20.433" Title="Random networks in PPi" Tags="|networks|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11226" PostTypeId="2" ParentId="7007" CreationDate="2020-01-24T14:22:49.057" Score="1" Body="&lt;p&gt;This one uses JQ for parsing and querying conda generated json output (&lt;a href=&quot;https://stedolan.github.io/jq/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://stedolan.github.io/jq/&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conda list | awk '{if(NR&amp;gt;3) printf(&quot;%s=%s\n&quot;, &lt;span class=&quot;math-container&quot;&gt;$1, $&lt;/span&gt;2)}' | xargs -I{} conda search --info --json &quot;{}&quot; | jq --raw-output '.[][0] | &quot;\(.name)\t\(.version)\t\(.license)&quot;'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;builds the list, asks conda search for all info on package as json, feeds json to JQ, which selects the first item and outputs its name, version and license as tsv.&lt;/p&gt;&#xA;" OwnerUserId="6816" LastActivityDate="2020-01-24T14:22:49.057" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11227" PostTypeId="1" AcceptedAnswerId="11229" CreationDate="2020-01-25T00:55:19.673" Score="191" ViewCount="123866" Body="&lt;p&gt;The SARS-Cov2 coronavirus's genome was released, and is now available on Genbank.  Looking at it...&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;    1 attaaaggtt tataccttcc caggtaacaa accaaccaac tttcgatctc ttgtagatct&#xA;   61 gttctctaaa cgaactttaa aatctgtgtg gctgtcactc ggctgcatgc ttagtgcact&#xA;  121 cacgcagtat aattaataac taattactgt cgttgacagg acacgagtaa ctcgtctatc&#xA;  ...&#xA;29761 acagtgaaca atgctaggga gagctgccta tatggaagag ccctaatgtg taaaattaat&#xA;29821 tttagtagtg ctatccccat gtgattttaa tagcttctta ggagaatgac aaaaaaaaaa&#xA;29881 aaaaaaaaaa aaaaaaaaaa aaa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;&lt;sub&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/MN908947&quot; rel=&quot;noreferrer&quot;&gt;Wuhan seafood market pneumonia virus isolate Wuhan-Hu-1, complete genome&lt;/a&gt;, Genbank&lt;/sub&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Geeze, that's a lot of &lt;a href=&quot;https://en.wikipedia.org/wiki/Adenine&quot; rel=&quot;noreferrer&quot;&gt;a&lt;/a&gt; nucleotides---I don't think that's just random.  I would guess that it's either an artifact of the sequencing process, or there is some underlying biological reason.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;: Why does the SARS-Cov2 coronavirus genome end in 33 a's?&lt;/p&gt;&#xA;" OwnerUserId="1451" LastEditorUserId="3848" LastEditDate="2023-05-05T23:46:03.763" LastActivityDate="2023-05-05T23:46:03.763" Title="Why does the SARS-Cov2 coronavirus genome end in aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa (33 a's)?" Tags="|phylogenetics|sequence-alignment|fasta|phylogeny|covid-19|" AnswerCount="4" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="11228" PostTypeId="2" ParentId="11227" CreationDate="2020-01-25T02:09:15.500" Score="13" Body="&lt;p&gt;Not an expert, but some searching on eukaryotic positive-strand RNA viruses seems to show that polyadenylation is not uncommon.  For example, &lt;a href=&quot;https://jvi.asm.org/content/84/6/2843&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;Steil, et al., 2010&lt;/em&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="900" LastActivityDate="2020-01-25T02:09:15.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11229" PostTypeId="2" ParentId="11227" CreationDate="2020-01-25T11:48:59.720" Score="163" Body="&lt;p&gt;Good observation! The 3' poly(A) tail is actually a very common feature of positive-strand RNA viruses, including coronaviruses and picornaviruses.&lt;/p&gt;&#xA;&lt;p&gt;For coronaviruses in particular, we know that the poly(A) tail is required for replication, functioning in conjunction with the 3' untranslated region (UTR) as a cis-acting signal for negative strand synthesis and attachment to the ribosome during translation. Mutants lacking the poly(A) tail are severely compromised in replication. Jeannie Spagnolo and Brenda Hogue report:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The 3′ poly (A) tail plays an important, but as yet undefined role in Coronavirus genome replication. To further examine the requirement for the Coronavirus poly(A) tail, we created truncated poly(A) mutant defective interfering (DI) RNAs and observed the effects on replication. Bovine Coronavirus (BCV) and mouse hepatitis Coronavirus A59 (MHV-A59) DI RNAs with tails of 5 or 10 A residues were replicated, albeit at delayed kinetics as compared to DI RNAs with wild type tail lengths (&amp;gt;50 A residues). A BCV DI RNA lacking a poly(A) tail was unable to replicate; however, a MHV DI lacking a tail did replicate following multiple virus passages. Poly(A) tail extension/repair was concurrent with robust replication of the tail mutants. Binding of the host factor poly(A)- binding protein (PABP) appeared to correlate with the ability of DI RNAs to be replicated. Poly(A) tail mutants that were compromised for replication, or that were unable to replicate at all exhibited less in vitro PABP interaction. The data support the importance of the poly(A) tail in Coronavirus replication and further delineate the minimal requirements for viral genome propagation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-1-4615-1325-4_68&quot; rel=&quot;noreferrer&quot;&gt;Spagnolo J.F., Hogue B.G. (2001) Requirement of the Poly(A) Tail in Coronavirus Genome Replication. In: Lavi E., Weiss S.R., Hingley S.T. (eds) The Nidoviruses. Advances in Experimental Medicine and Biology, vol 494. Springer, Boston, MA&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yu-Hui Peng &lt;em&gt;et al.&lt;/em&gt; also report that the length of the poly(A) tail is regulated during infection:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Similar to eukaryotic mRNA, the positive-strand coronavirus genome of ~30 kilobases is 5’-capped and 3’-polyadenylated. It has been demonstrated that the length of the coronaviral poly(A) tail is not static but regulated during infection; however, little is known regarding the factors involved in coronaviral polyadenylation and its regulation. Here, we show that during infection, the level of coronavirus poly(A) tail lengthening depends on the initial length upon infection and that the minimum length to initiate lengthening may lie between 5 and 9 nucleotides. By mutagenesis analysis, it was found that (i) the hexamer AGUAAA and poly(A) tail are two important elements responsible for synthesis of the coronavirus poly(A) tail and may function in concert to accomplish polyadenylation and (ii) the function of the hexamer AGUAAA in coronaviral polyadenylation is position dependent. Based on these findings, we propose a process for how the coronaviral poly(A) tail is synthesized and undergoes variation. Our results provide the first genetic evidence to gain insight into coronaviral polyadenylation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0165077&quot; rel=&quot;noreferrer&quot;&gt;Peng Y-H, Lin C-H, Lin C-N, Lo C-Y, Tsai T-L, Wu H-Y (2016) Characterization of the Role of Hexamer AGUAAA and Poly(A) Tail in Coronavirus Polyadenylation. PLoS ONE 11(10): e0165077&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This builds upon prior work by Hung-Yi Wu &lt;em&gt;et al&lt;/em&gt;, which showed that the coronaviral 3' poly(A) tail is approximately 65 nucleotides in length in both genomic and sgmRNAs at peak viral RNA synthesis, and also observed that the precise length varied throughout infection. Most interestingly, they report:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Functional analyses of poly(A) tail length on specific viral RNA species, furthermore, revealed that translation, in vivo, of RNAs with the longer poly(A) tail was enhanced over those with the shorter poly(A). Although the mechanisms by which the tail lengths vary is unknown, experimental results together suggest that the length of the poly(A) and poly(U) tails is regulated. One potential function of regulated poly(A) tail length might be that for the coronavirus genome a longer poly(A) favors translation. The regulation of coronavirus translation by poly(A) tail length resembles that during embryonal development suggesting there may be mechanistic parallels.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3726627/&quot; rel=&quot;noreferrer&quot;&gt;Wu HY, Ke TY, Liao WY, Chang NY. Regulation of coronaviral poly(A) tail length during infection. PLoS One. 2013;8(7):e70548. Published 2013 Jul 29. doi:10.1371/journal.pone.0070548&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It's also worth pointing out that poly(A) tails at the 3' end of RNA are not an unusual feature of viruses. Eukaryotic mRNA almost always contains poly(A) tails, which are added post-transcriptionally in a process known as polyadenylation. It should not therefore be surprising that positive-strand RNA viruses would have poly(A) tails as well. In eukaryotic mRNA, the central sequence motif for identifying a polyadenylation region is AAUAAA, identified way back in the 1970s, with more recent research confirming its ubiquity. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3175714/&quot; rel=&quot;noreferrer&quot;&gt;Proudfoot 2011&lt;/a&gt; is a nice review article on poly(A) signals in eukaryotic mRNA.&lt;/p&gt;&#xA;" OwnerUserId="6823" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2020-01-25T11:48:59.720" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="11230" PostTypeId="2" ParentId="11227" CreationDate="2020-01-25T19:33:51.023" Score="32" Body="&lt;p&gt;This question is quite general, so I'm going to attempt to tie it back to bioinformatics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&#xA;The tree for the current coronavirus is &lt;a href=&quot;https://www.ecohealthalliance.org/2020/01/phylogenetic-analysis-shows-novel-wuhan-coronavirus-clusters-with-sars&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, showing it is closely related to bat-coronavirus and in particular SARS. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&#xA;The bioinformatics question for the current coronavirus is why this virus appears to be able to infect humans and transmit to human.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Genome size&lt;/strong&gt;&#xA;Firstly, you said that 30kb was large ... this is a standard size for a coronavirus genome, albeit it is unusual in that the family Coronaviridae are the largest genomes for a single stranded RNA virus, for example flaviviruses are 10kb. Thus, all coronaviruses are all approximately 30Kb. Some coronaviruses don't infect humans (zero symptoms), some cause very mild symptoms, others are MERS and SARS with 40-60% and 10% mortality rates, respectively. So, genome size is of little bioinformatics interest in my opinion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Polyadenylation&lt;/strong&gt;&#xA;Polyadenylation and capping (5' methylation) enable the RNA to be trafficked and transcribed by ribosomes and the mechanism is widely used by viruses. Methylation would also prevent the innate immune response from the shredding the vRNA. &lt;a href=&quot;https://www.pnas.org/content/107/8/3283#ref-1&quot; rel=&quot;noreferrer&quot;&gt;Koonin and Moss (2010)&lt;/a&gt;, interpreted a given capping mechanism as being common to the Mononegavirales - a viral Order including measles, mumps, Ebolavirus. Its a big statement, but regardless poly-A and capping are simply mimicking the host mRNA which a lot of viruses use. Poly-A and capping per se are not really interesting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/11250/ancestry-of-the-coronavirus-2019-ncov-wuhan-city-china&quot;&gt;Evolution and SARS&lt;/a&gt;&lt;/strong&gt; A more detailed examination the evolution of 2019-nCov and its epidemiology in relation to SARS can be found &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/11250/ancestry-of-the-coronavirus-2019-ncov-wuhan-city-china&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&#xA;The bioinformatics question is the genome size wierd - no, its standard for a coronavirus, is the poly-A weird - no its generic amongst lots of viruses as is capping. Is the length of the poly-A excessive (33 As), it looks odd  but a human genecist/bioinformaticist needs to answer that ... so is it (potentially) linked with its epidemiology/clinical symptoms?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't think 33 poly-As are linked with anything bioinformatically interesting. This is because it will likely vary dramatically between genomes (not simply epidemic vs. non-epidemic strains). I don't know the mechanism for poly-adenylation, but I think slippage is a likely mutation resulting in large variations between individual genomes, particularly for poly-A - which notorious for slippage. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So ultimately could poly-As be linked with the ability of the new coronavirus to infect/transmit and could we therefore explore that bioinformatically? I personally think slippage mutations would prevent a clonal lineage emerging, i.e. that the size of the poly-As is not stable between genomes, but that would assume a given given mechanism of polyadenylation. Thus as a bioinformatics question I wouldn't pursue it, because I don't think there is sufficient biological rationale. I agree weird stuff should be questioned and that bit of the genome jumps out ... but I doubt it would go anywhere.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Slippage&lt;/strong&gt;&#xA;The definition of a slippage mutation is &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Slipped_strand_mispairing&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, but basically it means this genome has 33 poly-As, however another isolate from the same epidemic could say have 30 poly-As (just an example), another might have 25 poly-As and so on.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Just my 2 cents&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-01-30T14:21:35.930" LastActivityDate="2020-01-30T14:21:35.930" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11231" PostTypeId="1" AcceptedAnswerId="11232" CreationDate="2020-01-25T19:35:34.567" Score="2" ViewCount="2927" Body="&lt;p&gt;I have generated a volcano plot with a differential expression file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code for inputing file:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;macrophage_list &amp;lt;- read.table(&quot;differential_expression_macrophage.csv&quot;, header = T, sep = &quot;,&quot;)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyr)&#xA;final_df &amp;lt;- df %&amp;gt;% pivot_longer(., -c(Feature.ID, Feature.Name), names_to = c(&quot;set&quot;,&quot;.value&quot;), names_pattern = &quot;(.+)_(.+)&quot;)&#xA;&#xA;# A tibble: 80 x 6&#xA;   Feature.ID Feature.Name set       Mean.Counts Log2.fold.change Adjusted.p.value&#xA;   &amp;lt;fct&amp;gt;      &amp;lt;fct&amp;gt;        &amp;lt;chr&amp;gt;           &amp;lt;dbl&amp;gt;            &amp;lt;dbl&amp;gt;            &amp;lt;dbl&amp;gt;&#xA; 1 a          A            Cluster.1    0.000961            0.292           1     &#xA; 2 a          A            Cluster.2    0.000902            0.793           1     &#xA; 3 a          A            Cluster.3    0.00181             1.46            0.758 &#xA; 4 a          A            Cluster.4    0.000642            0.269           1     &#xA; 5 b          B            Cluster.1    0.000320            1.95            0.910 &#xA; 6 b          B            Cluster.2    0.00180             4.77            0.154 &#xA; 7 b          B            Cluster.3    0                   2.19            1     &#xA; 8 b          B            Cluster.4    0                   1.66            1     &#xA; 9 c          C            Cluster.1    0.00128            -2.01            0.0467&#xA;10 c          C            Cluster.2    0.00632             0.352           1     &#xA;# … with 70 more rows&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;output of head(final_tumor)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# A tibble: 6 x 6&#xA;  Feature.ID        Feature.Name set       Mean.Counts Log2.fold.change Adjusted.p.value&#xA;  &amp;lt;fct&amp;gt;             &amp;lt;fct&amp;gt;        &amp;lt;chr&amp;gt;           &amp;lt;dbl&amp;gt;            &amp;lt;dbl&amp;gt;            &amp;lt;dbl&amp;gt;&#xA;1 ENSG00000227232.5 WASH7P       Cluster.1     0                   1.50            1    &#xA;2 ENSG00000227232.5 WASH7P       Cluster.2     0                   1.73            1    &#xA;3 ENSG00000227232.5 WASH7P       Cluster.3     0                   1.77            1    &#xA;4 ENSG00000227232.5 WASH7P       Cluster.4     0.00114             4.30            0.293&#xA;5 ENSG00000227232.5 WASH7P       Cluster.5     0                   2.15            1    &#xA;6 ENSG00000227232.5 WASH7P       Cluster.6     0                   1.22            1 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;output of tail(final_tumor)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# A tibble: 6 x 6&#xA;  Feature.ID        Feature.Name set        Mean.Counts Log2.fold.change Adjusted.p.value&#xA;  &amp;lt;fct&amp;gt;             &amp;lt;fct&amp;gt;        &amp;lt;chr&amp;gt;            &amp;lt;dbl&amp;gt;            &amp;lt;dbl&amp;gt;            &amp;lt;dbl&amp;gt;&#xA;1 ENSG00000210196.2 MT-TP        Cluster.6       0.0699          -0.202           0.790  &#xA;2 ENSG00000210196.2 MT-TP        Cluster.7       0.0801           0.0386          1      &#xA;3 ENSG00000210196.2 MT-TP        Cluster.8       0.0711           0.0875          1      &#xA;4 ENSG00000210196.2 MT-TP        Cluster.9       0.0152          -2.31            0.00127&#xA;5 ENSG00000210196.2 MT-TP        Cluster.10      0.0147          -2.30            0.00612&#xA;6 ENSG00000210196.2 MT-TP        Cluster.11      0.122            0.762           1  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Code for generating volcano plot:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(ggplot2)&#xA;library(ggrepel)&#xA;ggplot(final_tumor, aes(x = Log2.fold.change,y = -log10(Adjusted.p.value), label = Feature.Name))+ &#xA;  geom_point()+&#xA;  geom_text_repel(data = subset(final_tumor, Adjusted.p.value &amp;lt; 0.05), &#xA;                  aes(label = Feature.Name))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, I want to pull out a certain gene, Casp14, from the list and box it on the plot. How do I do that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PCi7U.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PCi7U.png&quot; alt=&quot;ggplot of rep_final&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6827" LastEditorUserId="6827" LastEditDate="2020-01-26T15:05:41.927" LastActivityDate="2020-01-26T15:05:41.927" Title="Pulling out a certain gene in a volcano plot" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11232" PostTypeId="2" ParentId="11231" CreationDate="2020-01-25T20:12:20.550" Score="4" Body="&lt;p&gt;As you did for labeling genes with an adjusted p value below 0.05, you can subset your dataset for keeping only rows corresponding to &quot;Casp14&quot;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(ggplot2)&#xA;library(ggrepel)&#xA;ggplot(final_tumor, aes(x = Log2.fold.change,y = -log10(Adjusted.p.value), label = Feature.Name))+ &#xA;  geom_point()+&#xA;  geom_text_repel(data = subset(final_tumor, Adjusted.p.value &amp;lt; 0.05), &#xA;                  aes(label = Feature.Name))+&#xA;  geom_text_repel(data = subset(final_tumor, Feature.Name == &quot;Casp14&quot;), &#xA;                  aes(label = Feature.Name), color = &quot;red&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With this code, you should see now the labeling of the gene of interest (Casp14) in red on your volcano plot. &lt;/p&gt;&#xA;" OwnerUserId="6437" LastActivityDate="2020-01-25T20:12:20.550" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="11235" PostTypeId="1" CreationDate="2020-01-26T15:37:10.033" Score="0" ViewCount="34" Body="&lt;p&gt;I am reading &quot;&lt;code&gt;Statistical methods in bioinformatics&lt;/code&gt;&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I came across this formula:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/eSebp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eSebp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As explained in the book &lt;strong&gt;O&lt;/strong&gt; are the observations:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/DnIgr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DnIgr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is why the author only specified &lt;span class=&quot;math-container&quot;&gt;$prob(O)$&lt;/span&gt; for a sequence of length=1 ?, I assume he is only talking about one observation, but generally we have a sequence &lt;span class=&quot;math-container&quot;&gt;$O_1 ,O_2,O_3,...$&lt;/span&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume that &lt;span class=&quot;math-container&quot;&gt;$Prob(sequence)= P(O_1).P(O_2)...$&lt;/span&gt; but I am not sure.&lt;/p&gt;&#xA;" OwnerUserId="6829" LastActivityDate="2020-08-25T20:35:40.233" Title="probability of an entire sequence, not just one value" Tags="|ngs|hidden-markov-models|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11236" PostTypeId="1" AcceptedAnswerId="11239" CreationDate="2020-01-26T16:54:25.403" Score="0" ViewCount="60" Body="&lt;p&gt;I want to create a dodged barchart with different cell cycles for each cell type within each condition. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;dput(DF_count)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(orig.ident = c(&quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &#xA;&quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &#xA;&quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &#xA;&quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &#xA;&quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &#xA;&quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &#xA;&quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &#xA;&quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &#xA;&quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &#xA;&quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &#xA;&quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &#xA;&quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &#xA;&quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;), Phase = structure(c(1L, &#xA;1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, &#xA;2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, &#xA;1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, &#xA;2L, 2L, 2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, &#xA;3L), .Label = c(&quot;G1&quot;, &quot;G2M&quot;, &quot;S&quot;), class = &quot;factor&quot;), old.ident = structure(c(1L, &#xA;2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L, 11L, 1L, 2L, 3L, 4L, 5L, &#xA;6L, 7L, 8L, 9L, 10L, 11L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, &#xA;10L, 11L, 1L, 2L, 3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L, 11L, 1L, 2L, &#xA;3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L, 11L, 1L, 2L, 3L, 4L, 5L, 6L, &#xA;7L, 8L, 9L, 10L, 11L), .Label = c(&quot;Fibroblast&quot;, &quot;T cell&quot;, &quot;Macrophage&quot;, &#xA;&quot;Stellate&quot;, &quot;Acinar&quot;, &quot;Endothelial&quot;, &quot;Tumor&quot;, &quot;B cell&quot;, &quot;Mast cell&quot;, &#xA;&quot;Ductal&quot;, &quot;Islets of Langerhans&quot;), class = &quot;factor&quot;), n = c(3257L, &#xA;666L, 560L, 446L, 91L, 464L, 192L, 506L, 162L, 207L, 90L, 603L, &#xA;608L, 238L, 98L, 22L, 110L, 215L, 95L, 63L, 97L, 10L, 2191L, &#xA;618L, 335L, 349L, 35L, 294L, 250L, 231L, 142L, 153L, 16L, 2910L, &#xA;1259L, 853L, 559L, 1134L, 348L, 306L, 133L, 116L, 12L, 192L, &#xA;706L, 1186L, 374L, 112L, 616L, 106L, 277L, 32L, 47L, 6L, 27L, &#xA;1715L, 1312L, 575L, 390L, 1036L, 250L, 252L, 91L, 66L, 8L, 68L&#xA;), Freq = c(53.8258139150554, 35.2008456659619, 49.4263018534863, &#xA;49.9440089585666, 61.4864864864865, 53.4562211981567, 29.2237442922374, &#xA;60.8173076923077, 44.141689373297, 45.2954048140044, 77.5862068965517, &#xA;9.96529499256321, 32.1353065539112, 21.0061782877317, 10.9742441209406, &#xA;14.8648648648649, 12.6728110599078, 32.7245053272451, 11.4182692307692, &#xA;17.1662125340599, 21.2253829321663, 8.62068965517241, 36.2088910923814, &#xA;32.6638477801269, 29.567519858782, 39.0817469204927, 23.6486486486486, &#xA;33.8709677419355, 38.0517503805175, 27.7644230769231, 38.6920980926431, &#xA;33.4792122538293, 13.7931034482759, 54.5863815419246, 33.5107798775619, &#xA;47.3362930077692, 52.6861451460886, 40.7035175879397, 49.4318181818182, &#xA;36.6467065868263, 51.953125, 50.6550218340611, 46.1538461538462, &#xA;66.8989547038328, 13.2432939410992, 31.5677402182592, 20.7547169811321, &#xA;10.5560791705938, 22.1105527638191, 15.0568181818182, 33.1736526946108, &#xA;12.5, 20.5240174672489, 23.0769230769231, 9.40766550522648, 32.1703245169762, &#xA;34.9214799041789, 31.9089900110988, 36.7577756833176, 37.1859296482412, &#xA;35.5113636363636, 30.1796407185629, 35.546875, 28.82096069869, &#xA;30.7692307692308, 23.6933797909408)), class = c(&quot;grouped_df&quot;, &#xA;&quot;tbl_df&quot;, &quot;tbl&quot;, &quot;data.frame&quot;), row.names = c(NA, -66L), groups = structure(list(&#xA;    orig.ident = c(&quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &#xA;    &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &quot;treated&quot;, &#xA;    &quot;treated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &#xA;    &quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &quot;untreated&quot;, &#xA;    &quot;untreated&quot;, &quot;untreated&quot;), old.ident = structure(c(1L, 2L, &#xA;    3L, 4L, 5L, 6L, 7L, 8L, 9L, 10L, 11L, 1L, 2L, 3L, 4L, 5L, &#xA;    6L, 7L, 8L, 9L, 10L, 11L), .Label = c(&quot;Fibroblast&quot;, &quot;T cell&quot;, &#xA;    &quot;Macrophage&quot;, &quot;Stellate&quot;, &quot;Acinar&quot;, &quot;Endothelial&quot;, &quot;Tumor&quot;, &#xA;    &quot;B cell&quot;, &quot;Mast cell&quot;, &quot;Ductal&quot;, &quot;Islets of Langerhans&quot;), class = &quot;factor&quot;), &#xA;    .rows = list(c(1L, 12L, 23L), c(2L, 13L, 24L), c(3L, 14L, &#xA;    25L), c(4L, 15L, 26L), c(5L, 16L, 27L), c(6L, 17L, 28L), &#xA;        c(7L, 18L, 29L), c(8L, 19L, 30L), c(9L, 20L, 31L), c(10L, &#xA;        21L, 32L), c(11L, 22L, 33L), c(34L, 45L, 56L), c(35L, &#xA;        46L, 57L), c(36L, 47L, 58L), c(37L, 48L, 59L), c(38L, &#xA;        49L, 60L), c(39L, 50L, 61L), c(40L, 51L, 62L), c(41L, &#xA;        52L, 63L), c(42L, 53L, 64L), c(43L, 54L, 65L), c(44L, &#xA;        55L, 66L))), row.names = c(NA, -22L), class = c(&quot;tbl_df&quot;, &#xA;&quot;tbl&quot;, &quot;data.frame&quot;), .drop = TRUE))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Used this code to generate a stacked bar graph: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(DF_Count, aes(x = old.ident, y = Freq, fill = Phase))+&#xA;  geom_col()+&#xA;  geom_text(aes(label = paste(round(Freq, 2),&quot;%&quot;)),position = position_stack(vjust = 0.5))+&#xA;  facet_wrap(~orig.ident)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But want the 2 conditions to be next to each other on the same graph. How do I use this dataset and create a dodged bar chart to have both conditions together?&lt;/p&gt;&#xA;" OwnerUserId="6827" LastActivityDate="2020-01-26T19:39:44.060" Title="Dodged bar chart" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11237" PostTypeId="1" AcceptedAnswerId="11240" CreationDate="2020-01-26T17:37:58.763" Score="0" ViewCount="161" Body="&lt;p&gt;I am trying to plot a box plot/stacked bar graph with error bars.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;output of dput(DF)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;`structure(list(Cell_Type = structure(c(1L, 2L, 3L, 4L, 5L, 6L, &#xA;7L, 1L, 2L, 3L, 4L, 5L, 6L, 7L), .Label = c(&quot;Fibroblast&quot;, &quot;T cell&quot;, &#xA;&quot;Macrophage&quot;, &quot;Tumor&quot;, &quot;Islets of Langerhans&quot;, &quot;Endothelial&quot;, &#xA;&quot;B cell&quot;), class = &quot;factor&quot;), Condition = structure(c(1L, 1L, &#xA;1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L), .Label = c(&quot;treated&quot;, &#xA;&quot;untreated&quot;), class = &quot;factor&quot;), Freq = c(6051L, 1892L, 1133L, &#xA;657L, 116L, 868L, 832L, 5331L, 3757L, 1802L, 835L, 287L, 704L, &#xA;256L), Percent = c(52.3941466793662, 16.3823707680319, 9.8103731924842, &#xA;5.68880422547407, 1.00441596675037, 7.51580223395965, 7.20408693393367, &#xA;41.0962072155412, 28.9623805118717, 13.8914585260561, 6.43694110391613, &#xA;2.21245760098674, 5.42707369719396, 1.97348134443417)), class = c(&quot;grouped_df&quot;, &#xA;&quot;tbl_df&quot;, &quot;tbl&quot;, &quot;data.frame&quot;), row.names = c(NA, -14L), groups = structure(list(&#xA;    Condition = structure(1:2, .Label = c(&quot;treated&quot;, &quot;untreated&quot;&#xA;    ), class = &quot;factor&quot;), .rows = list(1:7, 8:14)), row.names = c(NA, &#xA;-2L), class = c(&quot;tbl_df&quot;, &quot;tbl&quot;, &quot;data.frame&quot;), .drop = TRUE))`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Gives me the following dataset.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;       Var1      Var2 Freq&#xA;1   Fibroblast   treated 6051&#xA;2       T cell   treated 1892&#xA;3   Macrophage   treated 1133&#xA;4     Stellate   treated  893&#xA;5       Acinar   treated  148&#xA;6  Endothelial   treated  868&#xA;7   Fibroblast untreated 5331&#xA;8       T cell untreated 3757&#xA;9   Macrophage untreated 1802&#xA;10    Stellate untreated 1061&#xA;11      Acinar untreated 2786&#xA;12 Endothelial untreated  704&#xA;&#xA;library(dplyr)&#xA;DF &amp;lt;- df %&amp;gt;% rename(Cell_Type = Var1, Condition = Var2) %&amp;gt;%&#xA;  group_by(Condition) %&amp;gt;% &#xA;  mutate(Percent = Freq / sum(Freq)*100)&#xA;&#xA;# A tibble: 12 x 4&#xA;# Groups:   Condition [2]&#xA;   Cell_Type   Condition  Freq Percent&#xA;   &amp;lt;fct&amp;gt;       &amp;lt;fct&amp;gt;     &amp;lt;int&amp;gt;   &amp;lt;dbl&amp;gt;&#xA; 1 Fibroblast  treated    6051   55.1 &#xA; 2 T cell      treated    1892   17.2 &#xA; 3 Macrophage  treated    1133   10.3 &#xA; 4 Stellate    treated     893    8.13&#xA; 5 Acinar      treated     148    1.35&#xA; 6 Endothelial treated     868    7.90&#xA; 7 Fibroblast  untreated  5331   34.5 &#xA; 8 T cell      untreated  3757   24.3 &#xA; 9 Macrophage  untreated  1802   11.7 &#xA;10 Stellate    untreated  1061    6.87&#xA;11 Acinar      untreated  2786   18.0 &#xA;12 Endothelial untreated   704    4.56&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Calculated SD/Mean with this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;my_sum_1 &amp;lt;- DF %&amp;gt;%&#xA;    group_by(Cell_Type) %&amp;gt;%&#xA;    summarise( &#xA;        n=n(),&#xA;        mean=mean(Freq),&#xA;        sd=sd(Freq)&#xA;    ) %&amp;gt;%&#xA;    mutate( se=sd/sqrt(n))  %&amp;gt;%&#xA;    mutate( ic=se * qt((1-0.05)/2 + .5, n-1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then plotted with this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(my_sum_1) +&#xA;    geom_bar( aes(x=Cell_Type, y=mean), stat=&quot;identity&quot;, fill=&quot;forestgreen&quot;, alpha=0.5) +&#xA;    geom_errorbar( aes(x=Cell_Type, ymin=mean-ic, ymax=mean+ic), width=0.4, colour=&quot;orange&quot;, alpha=0.9, size=1.5) +&#xA;    ggtitle(&quot;Cell Types&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to plot based on condition. Not sure how I would do that. I tried the &lt;code&gt;fill&lt;/code&gt; command but doesn't seem to work as the object is not found, I can't seem to add it back to the data set. How do I include the condition in the bar chart?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I'm calculating sd, I think it's combining the cell type value into one, rather than taking it individually. How do I change that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;dput(head(all@meta.data))&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(orig.ident = c(&quot;PDAC&quot;, &quot;PDAC&quot;, &quot;PDAC&quot;, &quot;PDAC&quot;, &#xA;&quot;PDAC&quot;, &quot;PDAC&quot;), nCount_RNA = c(7945, 7616, 7849, 5499, 853, &#xA;1039), nFeature_RNA = c(2497L, 2272L, 2303L, 2229L, 509L, 588L&#xA;), percent.mt = c(2.63058527375708, 3.7421218487395, 4.9433048796025, &#xA;0.490998363338789, 0.234466588511137, 0.192492781520693), RNA_snn_res.0.5 = structure(c(5L, &#xA;5L, 5L, 5L, 1L, 6L), .Label = c(&quot;0&quot;, &quot;1&quot;, &quot;2&quot;, &quot;3&quot;, &quot;4&quot;, &quot;5&quot;, &#xA;&quot;6&quot;, &quot;7&quot;, &quot;8&quot;, &quot;9&quot;, &quot;10&quot;, &quot;11&quot;, &quot;12&quot;, &quot;13&quot;, &quot;14&quot;, &quot;15&quot;, &quot;16&quot;, &#xA;&quot;17&quot;, &quot;18&quot;, &quot;19&quot;), class = &quot;factor&quot;), seurat_clusters = structure(c(5L, &#xA;5L, 5L, 5L, 1L, 6L), .Label = c(&quot;0&quot;, &quot;1&quot;, &quot;2&quot;, &quot;3&quot;, &quot;4&quot;, &quot;5&quot;, &#xA;&quot;6&quot;, &quot;7&quot;, &quot;8&quot;, &quot;9&quot;, &quot;10&quot;, &quot;11&quot;, &quot;12&quot;, &quot;13&quot;, &quot;14&quot;, &quot;15&quot;, &quot;16&quot;, &#xA;&quot;17&quot;, &quot;18&quot;, &quot;19&quot;), class = &quot;factor&quot;), S.Score = c(0.0171539298241524, &#xA;-0.00599651468836135, 0.011058574403656, -0.0424413740104151, &#xA;0.03549521163095, -0.0243347616753686), G2M.Score = c(-0.0591387992008088, &#xA;0.0253275642795205, -0.0512402869839816, -0.0239076248512967, &#xA;-0.0263164126515597, -0.0339086517371782), Phase = structure(c(3L, &#xA;2L, 3L, 1L, 3L, 1L), .Label = c(&quot;G1&quot;, &quot;G2M&quot;, &quot;S&quot;), class = &quot;factor&quot;), &#xA;    old.ident = structure(c(1L, 1L, 1L, 1L, 1L, 3L), .Label = c(&quot;Fibroblast&quot;, &#xA;    &quot;T Cell&quot;, &quot;Endothelial&quot;, &quot;Tumor&quot;, &quot;Stellate&quot;, &quot;Macrophage&quot;, &#xA;    &quot;B Cell&quot;, &quot;Mast Cell&quot;, &quot;Acinar&quot;, &quot;Endocrine&quot;, &quot;Exocrine&quot;), class = &quot;factor&quot;)), row.names = c(&quot;PDAC_AAACCCAGTCGGCTAC-1&quot;, &#xA;&quot;PDAC_AAACGAAGTCCAGGTC-1&quot;, &quot;PDAC_AAAGAACAGCAAGTGC-1&quot;, &quot;PDAC_AAAGAACAGGATGAGA-1&quot;, &#xA;&quot;PDAC_AAAGTGACATCCGTTC-1&quot;, &quot;PDAC_AACAACCAGGAAGTAG-1&quot;), class = &quot;data.frame&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6827" LastEditorUserId="6827" LastEditDate="2020-01-27T23:25:38.250" LastActivityDate="2020-01-27T23:25:38.250" Title="Bar plot with error bars" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11238" PostTypeId="1" CreationDate="2020-01-26T18:24:18.040" Score="0" ViewCount="1243" Body="&lt;p&gt;I have generated a volcano plot with a differential expression file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code for inputing file:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(tidyr)&#xA;final_tumor &amp;lt;- df %&amp;gt;% pivot_longer(., -c(Feature.ID, Feature.Name), names_to = c(&quot;set&quot;,&quot;.value&quot;), names_pattern = &quot;(.+)_(.+)&quot;)&#xA;&#xA;# A tibble: 80 x 6&#xA;Feature.ID Feature.Name set       Mean.Counts Log2.fold.change Adjusted.p.value&#xA;&amp;lt;fct&amp;gt;      &amp;lt;fct&amp;gt;        &amp;lt;chr&amp;gt;           &amp;lt;dbl&amp;gt;            &amp;lt;dbl&amp;gt;            &amp;lt;dbl&amp;gt;&#xA;  1 a          A            Cluster.1    0.000961            0.292           1     &#xA;2 a          A            Cluster.2    0.000902            0.793           1     &#xA;3 a          A            Cluster.3    0.00181             1.46            0.758 &#xA;4 a          A            Cluster.4    0.000642            0.269           1     &#xA;5 b          B            Cluster.1    0.000320            1.95            0.910 &#xA;6 b          B            Cluster.2    0.00180             4.77            0.154 &#xA;7 b          B            Cluster.3    0                   2.19            1     &#xA;8 b          B            Cluster.4    0                   1.66            1     &#xA;9 c          C            Cluster.1    0.00128            -2.01            0.0467&#xA;10 c          C            Cluster.2    0.00632             0.352           1     &#xA;# … with 70 more rows&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here the output of head(final_tumor)&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# A tibble: 6 x 6&#xA;Feature.ID        Feature.Name set       Mean.Counts Log2.fold.change Adjusted.p.value&#xA;&amp;lt;fct&amp;gt;             &amp;lt;fct&amp;gt;        &amp;lt;chr&amp;gt;           &amp;lt;dbl&amp;gt;            &amp;lt;dbl&amp;gt;            &amp;lt;dbl&amp;gt;&#xA;  1 ENSG00000227232.5 WASH7P       Cluster.1     0                   1.50            1    &#xA;2 ENSG00000227232.5 WASH7P       Cluster.2     0                   1.73            1    &#xA;3 ENSG00000227232.5 WASH7P       Cluster.3     0                   1.77            1    &#xA;4 ENSG00000227232.5 WASH7P       Cluster.4     0.00114             4.30            0.293&#xA;5 ENSG00000227232.5 WASH7P       Cluster.5     0                   2.15            1    &#xA;6 ENSG00000227232.5 WASH7P       Cluster.6     0                   1.22            1 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And here is the output of tail(final_tumor):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# A tibble: 6 x 6&#xA;Feature.ID        Feature.Name set        Mean.Counts Log2.fold.change Adjusted.p.value&#xA;&amp;lt;fct&amp;gt;             &amp;lt;fct&amp;gt;        &amp;lt;chr&amp;gt;            &amp;lt;dbl&amp;gt;            &amp;lt;dbl&amp;gt;            &amp;lt;dbl&amp;gt;&#xA;  1 ENSG00000210196.2 MT-TP        Cluster.6       0.0699          -0.202           0.790  &#xA;2 ENSG00000210196.2 MT-TP        Cluster.7       0.0801           0.0386          1      &#xA;3 ENSG00000210196.2 MT-TP        Cluster.8       0.0711           0.0875          1      &#xA;4 ENSG00000210196.2 MT-TP        Cluster.9       0.0152          -2.31            0.00127&#xA;5 ENSG00000210196.2 MT-TP        Cluster.10      0.0147          -2.30            0.00612&#xA;6 ENSG00000210196.2 MT-TP        Cluster.11      0.122            0.762           1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the code for generating volcano plot:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(ggplot2)&#xA;library(ggrepel)&#xA;ggplot(final_tumor, aes(x = Log2.fold.change,y = -log10(Adjusted.p.value), label = Feature.Name))+ &#xA;  geom_point()+&#xA;  geom_text_repel(data = subset(final_tumor, Adjusted.p.value &amp;lt; 0.05), &#xA;                  aes(label = Feature.Name))+&#xA;  geom_text_repel(data = subset(final_tumor, Feature.Name == &quot;Casp14&quot;), &#xA;                  aes(label = Feature.Name), color = &quot;red&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Generating the ggplot with all the data was taking forever, it was suggested to select random rows of the dataframe: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rep_final = final_tumor[sample(1:nrow(final_tumor), size = 1000), ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nt36a.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nt36a.png&quot; alt=&quot;plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I did:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;ggplot(rep_final, aes(x = Log2.fold.change,y = -log10(Adjusted.p.value), label = Feature.Name))+ &#xA;  geom_point()+ &#xA;  geom_text_repel(data = subset(rep_final, Adjusted.p.value &amp;lt; 0.05), aes(label = Feature.Name))+ &#xA;  geom_text_repel(data = subset(final_tumor, Feature.Name == &quot;Casp14&quot;), aes(label = Feature.Name), color = &quot;red&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Everything is very tight and cluttered. I want to show the top differentially expressed genes. How do I go about simplifying the plot? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to do this on EnhancedVolcano, pulling out a certain gene?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;EnhancedVolcano(final_tumor, lab = as.character(final_tumor$FeatureName), x = 'Log2.fold.change', y = 'Adjusted.p.value', xlim = c(-8,8), title = 'Tumor', pCutoff = 10e-5, FCcutoff = 1.5, pointSize = 3.0, labSize = 3.0) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6827" LastEditorUserId="6437" LastEditDate="2020-01-27T11:40:06.850" LastActivityDate="2024-02-27T16:29:26.343" Title="Making a volcano plot look less cluttered" Tags="|r|bioconductor|ggplot2|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11239" PostTypeId="2" ParentId="11236" CreationDate="2020-01-26T19:39:44.060" Score="1" Body="&lt;p&gt;Here is a rather straight-forward work-around for your problem:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(DF_Count, aes(x = orig.ident, y = Freq, fill = Phase))+&#xA;  geom_col()+&#xA;  geom_text(aes(label = paste(round(Freq, 2),&quot;%&quot;)),position = position_stack(vjust = 0.5))+&#xA;  facet_wrap(~old.ident)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3lpqw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3lpqw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It does not give a single plot or a dodged bar chart representing the two conditions you would like to visualize. For an exact solution to your problem, see &lt;a href=&quot;https://stackoverflow.com/a/46597859/7236472&quot;&gt;this&lt;/a&gt; (actually the answer below the linked answer is analogous to what I have suggested above).&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-01-26T19:39:44.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11240" PostTypeId="2" ParentId="11237" CreationDate="2020-01-27T04:11:58.247" Score="0" Body="&lt;p&gt;Your issue is coming by the fact that when you calculate &lt;code&gt;mean&lt;/code&gt;, &lt;code&gt;se&lt;/code&gt;, ... you removed the &lt;code&gt;Condition&lt;/code&gt; column, here is your &lt;code&gt;my_sum_1&lt;/code&gt; results:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# A tibble: 7 x 6&#xA;  Cell_Type                n  mean    sd    se     ic&#xA;  &amp;lt;fct&amp;gt;                &amp;lt;int&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;&#xA;1 Fibroblast               2 5691   509. 360    4574.&#xA;2 T cell                   2 2824. 1319. 932.  11849.&#xA;3 Macrophage               2 1468.  473. 334.   4250.&#xA;4 Tumor                    2  746   126.  89.0  1131.&#xA;5 Islets of Langerhans     2  202.  121.  85.5  1086.&#xA;6 Endothelial              2  786   116.  82    1042.&#xA;7 B cell                   2  544   407. 288    3659.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To calculate the &lt;code&gt;mean&lt;/code&gt;, &lt;code&gt;sd&lt;/code&gt;, ... from each cell types in each conditions, you need to group by &lt;code&gt;Cell_Type&lt;/code&gt; and &lt;code&gt;Condition&lt;/code&gt; by writing:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;my_sum_1 &amp;lt;- DF %&amp;gt;%&#xA;  group_by(Cell_Type, Condition) %&amp;gt;%&#xA;  summarise( &#xA;    n=n(),&#xA;    mean=mean(Freq),&#xA;    sd=sd(Freq)&#xA;  ) %&amp;gt;%&#xA;  mutate( se=sd/sqrt(n))  %&amp;gt;%&#xA;  mutate( ic=se * qt((1-0.05)/2 + .5, n-1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But now, your table looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# A tibble: 14 x 7&#xA;# Groups:   Cell_Type [7]&#xA;   Cell_Type            Condition     n  mean    sd    se    ic&#xA;   &amp;lt;fct&amp;gt;                &amp;lt;fct&amp;gt;     &amp;lt;int&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;&#xA; 1 Fibroblast           treated       1  6051    NA    NA    NA&#xA; 2 Fibroblast           untreated     1  5331    NA    NA    NA&#xA; 3 T cell               treated       1  1892    NA    NA    NA&#xA; 4 T cell               untreated     1  3757    NA    NA    NA&#xA; 5 Macrophage           treated       1  1133    NA    NA    NA&#xA; 6 Macrophage           untreated     1  1802    NA    NA    NA&#xA; 7 Tumor                treated       1   657    NA    NA    NA&#xA; 8 Tumor                untreated     1   835    NA    NA    NA&#xA; 9 Islets of Langerhans treated       1   116    NA    NA    NA&#xA;10 Islets of Langerhans untreated     1   287    NA    NA    NA&#xA;11 Endothelial          treated       1   868    NA    NA    NA&#xA;12 Endothelial          untreated     1   704    NA    NA    NA&#xA;13 B cell               treated       1   832    NA    NA    NA&#xA;14 B cell               untreated     1   256    NA    NA    NA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Because in your initial dataset, you only have one value for each cell type in each condition, so you can calculate &lt;code&gt;sd&lt;/code&gt;, &lt;code&gt;sem&lt;/code&gt;. You need to first merge your replicates into your dataframe in order to be able to calculate your &lt;code&gt;sem&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, you will be able to do:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(ggplot2)&#xA;ggplot(my_sum_1, aes(x=Cell_Type, y=mean, fill = Condition)) +&#xA;  geom_col(alpha=0.5) +&#xA;  geom_errorbar( aes(ymin=mean-ic, ymax=mean+ic), width=0.4, colour=&quot;orange&quot;, alpha=0.9, size=1.5) +&#xA;  ggtitle(&quot;Cell Types&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Hope it helps you to figure it out what is your issue with your current data.&lt;/p&gt;&#xA;" OwnerUserId="6437" LastActivityDate="2020-01-27T04:11:58.247" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11241" PostTypeId="2" ParentId="11238" CreationDate="2020-01-27T04:33:43.677" Score="1" Body="&lt;p&gt;As we discussed in comments in your previous post: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/11231/pulling-out-a-certain-gene-in-a-volcano-plot/11232#11232&quot;&gt;Pulling out a certain gene in a volcano plot&lt;/a&gt;, in order to have less labeled genes on your volcano plot, you can specify more stringent cutoff.&lt;/p&gt;&#xA;&lt;p&gt;For example, you can decide to display only genes with an absolute fold change superior to 2 by selecting the appropriate subset of your data:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;ggplot(rep_final, aes(x = Log2.fold.change,y = -log10(Adjusted.p.value), label = Feature.Name))+ &#xA;  geom_point()+ &#xA;  geom_text_repel(data = subset(rep_final, abs(Log2.fold.change) &amp;gt; 2), aes(label = Feature.Name))+ &#xA;  geom_text_repel(data = subset(final_tumor, Feature.Name == &amp;quot;Casp14&amp;quot;), aes(label = Feature.Name), &#xA;                  color = &amp;quot;red&amp;quot;, size = 5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alternatively, you can maybe decide that a p value inferior to 0.05 is not enough stringent and decide to labeled only genes with an adjusted p value inferior to 0.00001 by doing:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;ggplot(rep_final, aes(x = Log2.fold.change,y = -log10(Adjusted.p.value), label = Feature.Name))+ &#xA;  geom_point()+ &#xA;  geom_text_repel(data = subset(rep_final, Adjusted.p.value &amp;lt; 0.00001), aes(label = Feature.Name))+ &#xA;  geom_text_repel(data = subset(final_tumor, Feature.Name == &amp;quot;Casp14&amp;quot;), aes(label = Feature.Name), &#xA;                  color = &amp;quot;red&amp;quot;, size = 5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or, you can want to do a combination of both criteria by doing:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;ggplot(rep_final, aes(x = Log2.fold.change,y = -log10(Adjusted.p.value), label = Feature.Name))+ &#xA;  geom_point()+ &#xA;  geom_text_repel(data = subset(rep_final, abs(Log2.fold.change) &amp;gt; 2 &amp;amp; Adjusted.p.value &amp;lt; 0.00001), aes(label = Feature.Name))+ &#xA;  geom_text_repel(data = subset(final_tumor, Feature.Name == &amp;quot;Casp14&amp;quot;), aes(label = Feature.Name), &#xA;                  color = &amp;quot;red&amp;quot;, size = 5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;At the end, I think it depends of what you are trying to demonstrate by presenting this volcano plot. Do you want to highlight some particular genes? Or do you want to display genes that are the most differentially expressed? If so, how many?&lt;/p&gt;&#xA;&lt;p&gt;All of this depends of the scientific question underlying your experiments and why you came to make this volcano plot.&lt;/p&gt;&#xA;&lt;p&gt;Hope this couple of example helps you to figure it out how to customize your volcano plot.&lt;/p&gt;&#xA;" OwnerUserId="6437" LastEditorUserId="650" LastEditDate="2024-02-27T16:29:26.343" LastActivityDate="2024-02-27T16:29:26.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11242" PostTypeId="2" ParentId="11227" CreationDate="2020-01-27T09:37:05.457" Score="21" Body="&lt;p&gt;Some of the other answers here seem quite good; at the same time I think the core answer to the OP's question is maybe a bit hard to tease out of them, so I'd like to try to state it more plainly. It's worth noting that a truly complete answer to this question seems to be beyond current research, but any kind of &quot;Why?&quot; is inevitably a hard or even impossible sort of question to answer fully in biology. We have some ideas about it though.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;mRNA is used as a template for protein synthesis within a cell. A single mRNA is used repeatedly, but is eventually &quot;used up&quot; and taken apart. In eukaryotes, poly(A) tails are almost always found on mRNAs produced in the nucleus. The poly(A) tail is ultimately shortened during the transcription process, and this shortening contributes to the mRNA being degraded. (See &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK6413/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; for more.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Coronaviruses also have a poly(A) tail, similarly to eukaryote mRNA. The precise mechanical functions of this poly(A) tail and the means of its synthesis are objects of ongoing research, but research has shown that &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-1-4615-1325-4_68&quot; rel=&quot;noreferrer&quot;&gt;its presence greatly increases the degree to which Coronavirus RNA is replicated by the host cell&lt;/a&gt;. Research has also shown that &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3726627/&quot; rel=&quot;noreferrer&quot;&gt;longer tails increase replication compared to shorter tails&lt;/a&gt;. It's quite likely that the presence of the tail assists in recruiting the cell's protein synthesis machinery and allows the RNA to last longer within the host cell, just as it does in the cell's own mRNA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interestingly, the pattern in which Coronavirus poly(A) tail length is regulated during infection, in which it starts out shorter, gets longer, then gets much shorter, resembles &lt;a href=&quot;https://elifesciences.org/articles/16955&quot; rel=&quot;noreferrer&quot;&gt;poly(A) tail length regulation of mRNA during eukaryote embryogenesis&lt;/a&gt;, suggesting parallels (see the paper in the &quot;longer tails&quot; link for more on this as well). Longer poly(A) tail length is closely tied to greater translational efficiency in that context.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There has been some speculation in the comments as to whether or not the Coronavirus poly(A) tail resembles a NOP sled in computer programming. I think the resemblance is mostly coincidental. NOP sleds are used in exploits because a processor, encountering a NOP, moves to the next instruction without taking any other actions. A long chain of NOPs, if entered by the processor at any point within it, will lead it to the instructions at the &quot;bottom,&quot; after the NOPs. This is advantageous to use if you can't get the processor to go exactly where you want but you know it will end up somewhere close by, because it increases your chances of having your payload executed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's unusual to see a lengthy NOP sled in legitimate code, to the point that people writing them usually have to disguise their function in order to avoid automatic detection. (see pg. 183 &lt;a href=&quot;https://link.springer.com/content/pdf/10.1007/s10994-009-5143-5.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;) In contrast, a poly(A) tail is &lt;a href=&quot;https://www.cell.com/fulltext/S0092-8674(02)01137-6&quot; rel=&quot;noreferrer&quot;&gt;almost universally found on nuclear eukaryote mRNA (and on some mRNAs of almost all organisms to some capacity, even mitochondria)&lt;/a&gt;. Furthermore, the functions of the poly(A) tail are complex enough that it's still an object of ongoing research decades after its initial discovery, whereas a NOP sled does one very mechanical thing. Since the environment inside a cell is so different from the environment of a processor interacting with memory, I think it's hard to make comparisons that are so granular as to deal with a specific set of machine instructions, at least in this kind of context—a processor is a very straightforward kind of machine compared to a cell.&lt;/p&gt;&#xA;" OwnerUserId="6928" LastActivityDate="2020-01-27T09:37:05.457" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11243" PostTypeId="1" AcceptedAnswerId="11247" CreationDate="2020-01-27T12:04:57.213" Score="1" ViewCount="1920" Body="&lt;p&gt;I have 3 data frames for three groups of patients and in each of them I have the number of mutation types like insertion, deletion, SNP and total of mutations for each patients. In each group I have different number of patients like below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(dat1)&#xA;                patient  DEL  INS   SNP   total&#xA;1:    LP6008337-DNA_H06  927  773 40756   42456&#xA;2:    LP6008334-DNA_D02 1049  799 31009   32857&#xA;&amp;gt; dim(dat1)&#xA;[1] 21  6&#xA;&amp;gt; &#xA;&#xA; head(dat2)&#xA;                Patient   DEL  INS   SNP total&#xA;1:    LP6008031-DNA_E01 13552 3374 62105   79031&#xA;2:    LP6005500-DNA_G01   539  500 43451   44490&#xA;&amp;gt; dim(dat2)&#xA;[1] 33  6&#xA;&amp;gt; &#xA;&#xA;&amp;gt; head(dat3)&#xA;                Patient   DEL   INS    SNP    total&#xA;1:    LP6005935-DNA_F03 39168 16739  58095   114002&#xA;2:    LP6008269-DNA_D08   849   910 103501  105271&#xA;&#xA;&amp;gt; dim(dat3)&#xA;[1] 106   6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to show if the number of each mutational category and total is different between these three groups by chi-square test&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actual question is if total number of mutations, SNP, DEL and INS are statistically significant among groups. I used pairwise t test but I afraid this test is not a suitable test for the distribution of data moreover I don't know how to visualize the p-value&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Imagining the comparison of total number of mutations between three groups, this picture is a good example&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ABR04.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ABR04.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or in this plot they compare several features but only two groups &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Sy4cn.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Sy4cn.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you help ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks &lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-01-27T14:46:17.413" LastActivityDate="2020-01-27T15:36:10.237" Title="Visualizing the difference among three groups" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11244" PostTypeId="2" ParentId="11238" CreationDate="2020-01-27T13:59:56.470" Score="2" Body="&lt;p&gt;&lt;strong&gt;The element that is clustering your plot the most is the ggrepel labels&lt;/strong&gt;, so that's what you need to work on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To get to a readable number of labels, &lt;strong&gt;I would not remove random genes&lt;/strong&gt;: you risk losing important info, be it genes that could be of interest to you or some of the most regulated. Incidentally that seems to happen on your example: you try plotting in red the label for Casp14, but it does not show on the plot, probably because the gene was removed in your sampling.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you can do however is, as dc37 suggests, &lt;strong&gt;increase the thresholds for the labels&lt;/strong&gt; (you can probably play around with the thresholds to see when it does become readable). This should also help with the plotting: you mention that generating the plot was taking a long time, but with the number of genes you are looking at (~10k-30k?), the geom_points() should not be a real problem (maybe a few seconds), however the ggrepel is the most computationnally expensive so that's what you should try to reduce.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This should get you started:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(final_tumor, aes(x = Log2.fold.change,y = -log10(Adjusted.p.value), label = Feature.Name))+ &#xA;  geom_point()+ &#xA;  geom_text_repel(data = subset(final_tumor, Adjusted.p.value &amp;lt; 0.001 &amp;amp; abs(Log2.fold.change) &amp;gt; 1 ), aes(label = Feature.Name))+ &#xA;  geom_text_repel(data = subset(final_tumor, Feature.Name == &quot;Casp14&quot;), aes(label = Feature.Name), color = &quot;red&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Notice everything plots the &quot;final_tumor&quot; object and not the &quot;rep_final&quot;. Feel free to be even more stringent with the cutoffs if it's still taking too long to plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A bit outside of the score of your question, but I also wanted to mention that you should never be trying to label too many genes on a volcano plot: this type of graph is useful to visualize your data's p-value/FC distribution and highlight a subset of genes in this context (most regulated, genes regulated in another experiment, pathway of interest...), but if you label too many of them it will never be readable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you really want to be able to indentify all the genes on your graph, I would try an interactive visualization (like with &lt;a href=&quot;https://plot.ly/r/&quot; rel=&quot;nofollow noreferrer&quot;&gt;plotly&lt;/a&gt;). I'm not going to detail the code here but you could look in that direction! (However, as with ggrepel, you might need to remove some genes so that it does not &quot;take forever&quot;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Have fun :)&lt;/p&gt;&#xA;" OwnerUserId="1452" LastActivityDate="2020-01-27T13:59:56.470" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11245" PostTypeId="1" CreationDate="2020-01-27T14:54:59.273" Score="1" ViewCount="59" Body="&lt;p&gt;I have a RNA aligned file without any header info. &#xA;I need to calculate the coverage of each mapped base for per chromosome. Is it possible to do it with perl ? &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;NS500377:28:H16A7BGXX:2:21205:16038:7658        99      chr2   3000482 255     46M     =       3000502 56      GTCCTACAGTGTGCATTTCTCGTTTTCCACGTTTCT    AAAA)FFFFFFFFFAFFFFFFFFFFF.F.FFFFFAF    XA:i:2  MD:Z:21T4T7T1   NM:i:3&#xA;NS500377:28:H16A7BGXX:3:11503:24953:10427       99      chr2   3000482 255     50M     =       3000513 71      GTCCTACAGTGTGCATTTCTCATTTTTCAAGTTTTTCAGT        AAAAAFFFFFFFFFFFFAFFFFFF7AAFFFFFFFFFFFFF&#xA;        XA:i:1  MD:Z:21T7C10    NM:i:2&#xA;NS500377:28:H16A7BGXX:3:12404:23929:9392        163     chr2   3000482 255     49M     =       3000483 40      GTCCTACAGTGTGCATTTCCCTTTTTTCACGTTTTTTAG AAFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF XA:i:1  MD:Z:19T16C2    NM:i:2&#xA;NS500377:28:H16A7BGXX:3:12404:23929:9392        83      chr2   3000483 255     49M     =       3000482 -40     TCCTACAGTGTGCATTTCCCTTTTTTCACGTTTTTTAGT AFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF XA:i:2  MD:Z:18T16C3    NM:i:2&#xA;NS500377:28:H16A7BGXX:3:21606:15675:1639        83      chr2   3000483 255     50M     =       3000480 -43     TCTTACAGTGTGCATTTCTCTTTTTTCACGTTTTTCAGTG        AFFFFFFFFFAFFFFFFFFFFFFFFFFFFFFFFFFAAAAA&#xA;        XA:i:0  MD:Z:2C37       NM:i:1&#xA;NS500377:28:H16A7BGXX:4:12605:18505:11088       83      chr2   3000486 255     50M     =       3000481 -45     TACAATGTGCATTTCTCTTTTTTCACGTTTTTCAGTATTT        FFFFFFFAFFF)FFFFFFFFF&amp;lt;FFFF&amp;lt;AFFFFFFFAA&amp;lt;AA&#xA;        XA:i:1  MD:Z:4G31G3     NM:i:2&#xA;NS500377:28:H16A7BGXX:2:13303:20634:12085       99      chr2   3000487 255     49M     =       3000521 73      ACAGTGTACATTTCTCATTTTTCACGTTTTTCAGTGTTA AAAAAFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF. XA:i:2  MD:Z:7G8T21T0   NM:i:3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6958" LastEditorUserId="6958" LastEditDate="2020-01-27T15:10:25.740" LastActivityDate="2020-01-27T17:39:13.583" Title="how to calculate coverage of each base of mapped reads?" Tags="|rna|coverage|awk|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11247" PostTypeId="2" ParentId="11243" CreationDate="2020-01-27T15:36:10.237" Score="3" Body="&lt;p&gt;Here a way to do it is to start first by creating a dataset containing your three different group, based on the few lines you display:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;dat1&lt;span class=&quot;math-container&quot;&gt;$Condition = &quot;dat1&quot;&#xA;dat2$&lt;/span&gt;Condition = &quot;dat2&quot;&#xA;dat3$Condition = &quot;dat3&quot;&#xA;colnames(dat1)[1] = &quot;Patient&quot;&#xA;DF &amp;lt;- rbind(dat1,dat2,dat3)&#xA;&#xA;             Patient   DEL   INS    SNP  total Condition&#xA;1: LP6008337-DNA_H06   927   773  40756  42456      dat1&#xA;2: LP6008334-DNA_D02  1049   799  31009  32857      dat1&#xA;3: LP6008031-DNA_E01 13552  3374  62105  79031      dat2&#xA;4: LP6005500-DNA_G01   539   500  43451  44490      dat2&#xA;5: LP6005935-DNA_F03 39168 16739  58095 114002      dat3&#xA;6: LP6008269-DNA_D08   849   910 103501 105271      dat3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, you can reshape &lt;code&gt;DF&lt;/code&gt; in order to put all values DEL, INS, SNP and Total into a single column and create a categorical column. Doing this will be helpful for later producing the facets of your plot. To do that, you can use &lt;code&gt;pivot_longer&lt;/code&gt; from &lt;code&gt;tidyr&lt;/code&gt; package:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(tidyr)&#xA;DF &amp;lt;- DF %&amp;gt;% pivot_longer(.,cols = c(DEL,INS,SNP, total), names_to = &quot;var&quot;, values_to = &quot;val&quot;)&#xA;&#xA;# A tibble: 24 x 4&#xA;   Patient           Condition var     val&#xA;   &amp;lt;chr&amp;gt;             &amp;lt;chr&amp;gt;     &amp;lt;chr&amp;gt; &amp;lt;int&amp;gt;&#xA; 1 LP6008337-DNA_H06 dat1      DEL     927&#xA; 2 LP6008337-DNA_H06 dat1      INS     773&#xA; 3 LP6008337-DNA_H06 dat1      SNP   40756&#xA; 4 LP6008337-DNA_H06 dat1      total 42456&#xA; 5 LP6008334-DNA_D02 dat1      DEL    1049&#xA; 6 LP6008334-DNA_D02 dat1      INS     799&#xA; 7 LP6008334-DNA_D02 dat1      SNP   31009&#xA; 8 LP6008334-DNA_D02 dat1      total 32857&#xA; 9 LP6008031-DNA_E01 dat2      DEL   13552&#xA;10 LP6008031-DNA_E01 dat2      INS    3374&#xA;# … with 14 more rows&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, you can plot your data and use the function &lt;code&gt;stat_compare_means&lt;/code&gt; from &lt;code&gt;ggpubr&lt;/code&gt; package to get the significance display on each tab. Based on the size of your group, you may want to adjust for the type of statistical test to be used (take a look at &lt;code&gt;?stat_compare_means&lt;/code&gt; to find the statistical test adequate to your dataset). You can pass various conditions to test as a list by using &lt;code&gt;comparisons&lt;/code&gt; argument. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For facetting your graph, you can use &lt;code&gt;facet_grid&lt;/code&gt; (or&lt;code&gt;facet_wrap&lt;/code&gt;, try the one you preferred). Altogether, you can get the plot by writing:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(ggplot2)&#xA;library(ggpubr)&#xA;ggplot(DF, aes(x = Condition, y = val, fill = Condition))+&#xA;  geom_boxplot()+&#xA;  stat_compare_means(comparisons = list(c(&quot;dat1&quot;,&quot;dat2&quot;), c(&quot;dat1&quot;,&quot;dat3&quot;), c(&quot;dat2&quot;,&quot;dat3&quot;)))+&#xA;  stat_compare_means()+&#xA;  facet_grid(var~., scales = &quot;free&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LAuWd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LAuWd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps you to figure it out how to represent your data.&lt;/p&gt;&#xA;" OwnerUserId="6437" LastActivityDate="2020-01-27T15:36:10.237" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11249" PostTypeId="2" ParentId="11245" CreationDate="2020-01-27T17:39:13.583" Score="1" Body="&lt;p&gt;You are looking for &lt;code&gt;bedtools genomecov&lt;/code&gt;, specifically &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/genomecov.html#d-reporting-per-base-genome-coverage&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;-d&lt;/code&gt;&lt;/a&gt; option. See &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/genomecov.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedtools.readthedocs.io/en/latest/content/tools/genomecov.html&lt;/a&gt; , for example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools genomecov -i A.bed -g my.genome -d&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;I recommend not reinventing the wheel: use &lt;code&gt;bedtools&lt;/code&gt; instead of reimplementing the same functionality in Perl.&lt;/p&gt;&#xA;" OwnerUserId="6251" LastActivityDate="2020-01-27T17:39:13.583" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11250" PostTypeId="1" AcceptedAnswerId="11268" CreationDate="2020-01-27T18:38:00.190" Score="5" ViewCount="898" Body="&lt;p&gt;In one of the answers to &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/11227/why-does-the-wuhan-coronavirus-genome-end-in-aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa&quot;&gt;another question about the corona virus&lt;/a&gt; a link was given to this &lt;a href=&quot;https://www.ecohealthalliance.org/2020/01/phylogenetic-analysis-shows-novel-wuhan-coronavirus-clusters-with-sars&quot; rel=&quot;nofollow noreferrer&quot;&gt;phylogenetic analysis&lt;/a&gt; of the virus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can somebody assist a non-bio type here? It seems to show that the current corona virus split from a virus in bats. And that the same ancestral virus is also ancestral to several other viruses, some in humans and some in bats. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is my understanding basically correct?&lt;/p&gt;&#xA;" OwnerUserId="6971" LastEditorUserId="3848" LastEditDate="2020-07-23T22:02:57.100" LastActivityDate="2020-07-23T22:02:57.100" Title="Ancestry of the coronavirus 2019-nCov, WuHan city, China" Tags="|sequence-alignment|fasta|phylogenetics|phylogeny|covid-19|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11251" PostTypeId="1" CreationDate="2020-01-27T21:17:59.723" Score="4" ViewCount="471" Body="&lt;p&gt;This is the BLAST tree of the latest coronavirus out of China (from Wuhan Institute of Virology, China). It seems strange that there is so much divergence from all the other coronaviruses. Is this expected of new diseases?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zIawo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zIawo.png&quot; alt=&quot;Wuhan BLAST cornoavirus tree&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6985" LastEditorUserId="3848" LastEditDate="2020-11-30T22:29:22.750" LastActivityDate="2020-12-01T09:13:45.420" Title="Interpretion of my coronavirus 2019-nCov, Wuhan, China BLAST tree?" Tags="|sequence-alignment|phylogenetics|phylogeny|covid-19|sars-cov-2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11252" PostTypeId="2" ParentId="11223" CreationDate="2020-01-27T21:20:53.797" Score="1" Body="&lt;p&gt;Turns out two recalibration tables are needed to show pre-recalibration and post-recalibration plots. I previously only had generated the pre-calibration table by running &lt;code&gt;BaseRecalibrator&lt;/code&gt; on the BAM that hadn't undergone &lt;code&gt;ApplyBQSR&lt;/code&gt; to adjust base scores. Here are the steps to show pre and post recalibration plots in &lt;code&gt;MultiQC&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Run &lt;code&gt;BaseRecalibrator&lt;/code&gt; on your initial input BAM. This will create a recalibration table with the original base scores. &lt;/li&gt;&#xA;&lt;li&gt;Run &lt;code&gt;ApplyBQSR&lt;/code&gt; with the recalibration table from step 1 to create a new BAM with the adjusted base quality scores.&lt;/li&gt;&#xA;&lt;li&gt;Run &lt;code&gt;BaseRecalibrator&lt;/code&gt; on the adjusted BAM generated in step 2 to create a new recalibration table with the adjusted base quality scores. &lt;/li&gt;&#xA;&lt;li&gt;Use both recalibration tables as inputs to &lt;code&gt;MultiQC&lt;/code&gt;. This should create both plots. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="6674" LastActivityDate="2020-01-27T21:20:53.797" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11253" PostTypeId="1" AcceptedAnswerId="11267" CreationDate="2020-01-27T23:35:10.380" Score="0" ViewCount="93" Body="&lt;p&gt;I want to create a column plot for my single cell analysis that has 2 sample types, a normal and PDAC. I want to show, the percentage of different cell types per sample type.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I go about determining the percent of each cell type in each condition and then go about plotting the column plot? Also, is it possible to determine the SD from the dataset and add it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;dput(head(all@meta.data))&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(orig.ident = c(&quot;PDAC&quot;, &quot;PDAC&quot;, &quot;PDAC&quot;, &quot;PDAC&quot;, &#xA;&quot;PDAC&quot;, &quot;PDAC&quot;), nCount_RNA = c(7945, 7616, 7849, 5499, 853, &#xA;1039), nFeature_RNA = c(2497L, 2272L, 2303L, 2229L, 509L, 588L&#xA;), percent.mt = c(2.63058527375708, 3.7421218487395, 4.9433048796025, &#xA;0.490998363338789, 0.234466588511137, 0.192492781520693), RNA_snn_res.0.5 = structure(c(5L, &#xA;5L, 5L, 5L, 1L, 6L), .Label = c(&quot;0&quot;, &quot;1&quot;, &quot;2&quot;, &quot;3&quot;, &quot;4&quot;, &quot;5&quot;, &#xA;&quot;6&quot;, &quot;7&quot;, &quot;8&quot;, &quot;9&quot;, &quot;10&quot;, &quot;11&quot;, &quot;12&quot;, &quot;13&quot;, &quot;14&quot;, &quot;15&quot;, &quot;16&quot;, &#xA;&quot;17&quot;, &quot;18&quot;, &quot;19&quot;), class = &quot;factor&quot;), seurat_clusters = structure(c(5L, &#xA;5L, 5L, 5L, 1L, 6L), .Label = c(&quot;0&quot;, &quot;1&quot;, &quot;2&quot;, &quot;3&quot;, &quot;4&quot;, &quot;5&quot;, &#xA;&quot;6&quot;, &quot;7&quot;, &quot;8&quot;, &quot;9&quot;, &quot;10&quot;, &quot;11&quot;, &quot;12&quot;, &quot;13&quot;, &quot;14&quot;, &quot;15&quot;, &quot;16&quot;, &#xA;&quot;17&quot;, &quot;18&quot;, &quot;19&quot;), class = &quot;factor&quot;), S.Score = c(0.0171539298241524, &#xA;-0.00599651468836135, 0.011058574403656, -0.0424413740104151, &#xA;0.03549521163095, -0.0243347616753686), G2M.Score = c(-0.0591387992008088, &#xA;0.0253275642795205, -0.0512402869839816, -0.0239076248512967, &#xA;-0.0263164126515597, -0.0339086517371782), Phase = structure(c(3L, &#xA;2L, 3L, 1L, 3L, 1L), .Label = c(&quot;G1&quot;, &quot;G2M&quot;, &quot;S&quot;), class = &quot;factor&quot;), &#xA;    old.ident = structure(c(1L, 1L, 1L, 1L, 1L, 3L), .Label = c(&quot;Fibroblast&quot;, &#xA;    &quot;T Cell&quot;, &quot;Endothelial&quot;, &quot;Tumor&quot;, &quot;Stellate&quot;, &quot;Macrophage&quot;, &#xA;    &quot;B Cell&quot;, &quot;Mast Cell&quot;, &quot;Acinar&quot;, &quot;Endocrine&quot;, &quot;Exocrine&quot;), class = &quot;factor&quot;)), row.names = c(&quot;PDAC_AAACCCAGTCGGCTAC-1&quot;, &#xA;&quot;PDAC_AAACGAAGTCCAGGTC-1&quot;, &quot;PDAC_AAAGAACAGCAAGTGC-1&quot;, &quot;PDAC_AAAGAACAGGATGAGA-1&quot;, &#xA;&quot;PDAC_AAAGTGACATCCGTTC-1&quot;, &quot;PDAC_AACAACCAGGAAGTAG-1&quot;), class = &quot;data.frame&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6827" LastActivityDate="2020-02-03T11:11:19.227" Title="Clustered column chart for single cell data" Tags="|r|ggplot2|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11254" PostTypeId="1" CreationDate="2020-01-28T03:02:39.093" Score="0" ViewCount="141" Body="&lt;p&gt;I would like to construct a mathematical graph: &lt;span class=&quot;math-container&quot;&gt;$G = (E,V, w(E))$&lt;/span&gt; a graph with vertices, edges with weight on them &lt;a href=&quot;https://www.biostars.org/p/85763/&quot; rel=&quot;nofollow noreferrer&quot;&gt;based on a KEGG pathway&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to do that with existing tools? If not I will start to implement a Python module for that. My main question is what is the mathematical graph representation of pathways? Assuming the possible node types and the different connections? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to transform a pathway to a graph because then one can run min-cut-max-flow algorithms and stuff like that as part of my &lt;a href=&quot;https://en.wikipedia.org/wiki/Pathway_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;pathway analysis&lt;/a&gt;. &lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="7340" LastEditDate="2020-06-17T02:42:32.793" LastActivityDate="2020-06-17T02:42:32.793" Title="How to represent pathways as mathematical graphs?" Tags="|python|pathway|kegg|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11255" PostTypeId="1" CreationDate="2020-01-28T06:38:06.180" Score="0" ViewCount="350" Body="&lt;p&gt;Although the Kallisto software was designed for RNA, it has also been very reliable in counting selected FASTA sequences in DNA sequencing. For example, given a  WGS sample and my FASTA file. Kallisto is able to accurately return the counts for each sequence in my FASTA file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, I'd like to do something similar on Nanopore long reads. Is that a similar software?&lt;/p&gt;&#xA;" OwnerUserId="174" LastActivityDate="2020-01-28T07:14:56.147" Title="Kallisto for Nanopore?" Tags="|nanopore|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11256" PostTypeId="1" CreationDate="2020-01-28T07:01:13.693" Score="2" ViewCount="237" Body="&lt;p&gt;How can we retrieve the file offsets of the zipped-chunks in a block-gzipped file that contain records for a region/window of interest (e.g. Chr01:100-Chr02:145)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems like the CLI of &lt;code&gt;samtools&lt;/code&gt;, or &lt;code&gt;tabix&lt;/code&gt; should support that, but I can't find a way to just get the offsets in the datafile, without having the accompanying datafile at hand. I couldn't find a formal specification for those index files either.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Perhaps one can recommend a library that can parse those index files, in python, C, or golang? It's hard to accept that there isn't a reusable block-gzipped index parser somewhere, other than the one buried deep in samtools/htslib/gatk code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;edit&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd initially given a small motivating example making use of file offsets to access a subportion of file on a remote server. Commenters were quick to point out that samtools supported a series of protocols for doing exactly that: ftp, http, and s3 (which requires https, which requires that samtools be compiled with libcurl support) Thanks! That example was detracting, so I removed it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;samtools view&lt;/code&gt; does fetch a subportion of a file (potentially from a remote location), but it does a few extra things, like decompressing the chunks retrieved. With this question, I'm interested in manipulating the gz chunks directly.&lt;/p&gt;&#xA;" OwnerUserId="3235" LastEditorUserId="3235" LastEditDate="2020-01-28T11:29:16.240" LastActivityDate="2020-02-29T03:47:50.877" Title="How to obtain file offsets corresponding to range query in block-gzipped file" Tags="|sam|samtools|indexing|tabix|bgzip|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11257" PostTypeId="2" ParentId="11255" CreationDate="2020-01-28T07:14:56.147" Score="1" Body="&lt;p&gt;It seems &lt;a href=&quot;https://github.com/a-slide/NanoCount&quot; rel=&quot;nofollow noreferrer&quot;&gt;NanoCount&lt;/a&gt; can do what you are looking for.:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;EM based transcript abundance from nanopore reads mapped to a&#xA;  transcriptome with minimap2 Python package adapted from&#xA;  &lt;a href=&quot;https://github.com/jts/nanopore-rna-analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/jts/nanopore-rna-analysis&lt;/a&gt; by Jared Simpson&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;NanoCount estimates transcript abundance from ONT direct-RNA&#xA;  Sequencing reads mapped to a transcriptome. It uses an&#xA;  expectation-maximization approach like RSEM, Kallisto, salmon, etc to&#xA;  handle multi-mapping reads. The reads must be mapped to the transcript&#xA;  set using minimap2.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="681" LastActivityDate="2020-01-28T07:14:56.147" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11258" PostTypeId="2" ParentId="11251" CreationDate="2020-01-28T10:57:49.717" Score="4" Body="&lt;p&gt;The  evolutionary related group (clade) of betacoronaviruses you have identified share an amino acid homology of 85% and include SARS. I know this from the underlying tree published on BioRxiv of a broader group of betacoronaviruses, i.e. your data is a defined subset of the betacoronaviruses which all share a unique, single common ancester.&lt;/p&gt;&#xA;&lt;p&gt;Lets call this group the SARS clade.&lt;/p&gt;&#xA;&lt;p&gt;You have performed a nucleotide blast and asked NCBI to produce a NJ tree using 2019-nCov as the reference. I can tell this by the colour coding and the scale bar at the bottom right hand corner shows the genetic distance involved is reasonably larger than 15% divergence by amino acid data. The scale bar represents the number of mutations per nucleotide.&lt;/p&gt;&#xA;&lt;p&gt;In summary, your tree is essentially a subset of the broader genetic diversity of the one published, but there is a rooting issue.&lt;/p&gt;&#xA;&lt;p&gt;In your tree, the majority of sequences are from the 2002 SARS epidemic and the virtually zero genetic distance between them is simply because it is a rapidly transmitted outbreak. &lt;em&gt;I didn't realize that SARS has two independent origins both initially from bats. This is quite scary.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;2019-nCov is an outgroup within the 'SARS clade' hence it appears at the other side of the tree, i.e. they share a more distant common ancestor&lt;/li&gt;&#xA;&lt;li&gt;However 2019-nCov is not the most distant common ancester within this subset of the betacoronaviruses, i.e. SARS clade, this belongs to the two bat viruses ZC45 and ZX21. The program has made a likely rooting error (below).&lt;/li&gt;&#xA;&lt;li&gt;Again the blast omitted the majority of the betacoronaviruses, for example MERS&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Rooting issue&lt;/strong&gt; the reason I suggest there is a rooting error is because the BioRxiv tree using a broad sample of the betacoronaviruses placed the bat strains ZC45 and ZX21 as outgroups to the SARS clade and 2019-nCOV was immediately within that, so in this definition 2019-nCOV is an 'ingroup' within the SARS clade, whereas your tree it is an 'outgroup'. It not a huge issue, but the location of the &amp;quot;root&amp;quot; is determined by the common ancester above it (sequences with &amp;lt;85% homology) and in your tree those have been omitted.&lt;/p&gt;&#xA;&lt;p&gt;Generally, I like the analysis otherwise and gives an insight into SARS that I hadn't previosly been aware of.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="9497" LastEditDate="2020-12-01T09:13:45.420" LastActivityDate="2020-12-01T09:13:45.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11259" PostTypeId="2" ParentId="11256" CreationDate="2020-01-28T11:03:40.973" Score="2" Body="&lt;p&gt;So if the purpose of obtaining file offsets is to access subportions of remote files via protocols that samtools supports, as mentioned in &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/11256/how-to-obtain-file-offsets-corresponding-to-range-query-in-block-gzipped-file#comment15279_11256&quot;&gt;this comment&lt;/a&gt;, then you can make your life easier and use samtools/tabix directly on the cli, prefixing your datafile with the corresponding scheme. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One side effect of using the CLI however, is that the blocks retrieved will be decompressed (e.g. samtools view). You'd have to re-compress them if what you're trying to do is create a smaller &quot;bam&quot; file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keep in mind that if you retrieve all zipped chunks which cover your range of interest, then you might be extracting more data than you asked for. The first and last chunk may contain records outside your region, just because their neighboring records happened to be compressed in the same chunk. You'd have to be careful to filter those downstream.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Specification for the tabix format&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you absolutely need a custom solution, the tabix data structure format, authored by Heng Li, is on the samtools website: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://samtools.github.io/hts-specs/tabix.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://samtools.github.io/hts-specs/tabix.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more in depth discussion of how the data structure works can be found here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3042176/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3042176/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The tree-shaped binning index is a bit hard to understand from the text alone, but the tabix paper mentions that their approach is almost identical to the binning algorithm in this paper: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC186604/pdf/X6.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC186604/pdf/X6.pdf&lt;/a&gt; (page 1003). That paper has a nice illustration which makes things clearer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Operation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: This is from my understanding of how the indices interoperate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't need the data file to retrieve chunk offsets, you can do the work with just the tabix file. But it's complicated, and probably error prone if you're implementing this yourself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's assume the query interval is &lt;code&gt;Chr01:100-Chr02:2000&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each sequence (Chr01, Chr02, etc) has its own pair of indices (binning index + linear index) in the tabix file. So you would break the query into two queries: &lt;code&gt;Chr01:100-999999999999&lt;/code&gt; and &lt;code&gt;Chr02:0-2000&lt;/code&gt;, and concatenate the two results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Within one sequence, to find all the blocks for open interval &lt;code&gt;[X,Y)&lt;/code&gt;, you would use the binning index to retrieve all the bins overlapping this interval (There is C code to do this in the data structure pdf). Then, you need to go through all the possible bins selected, and filter chunks. Not all chunks in a given bin will be overlapping with &lt;code&gt;X, Y&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To filter chunks within a bin, you need the help of the linear index. There is one entry in the linear index per 16kbp of sequence positions. Each entry of the linear index contains the virtual offset of the leftmost record overlapping that 16kb window. I don't know what the entry contains if there is no record overlapping that 16kb window, presumably 0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would find the linear entry in which position X falls (X/16Kb), take that offset, call it O_x. You find the linear entry corresponding to position Y, and take that offset, call it O_y. When you go through the selected bins, filter out any chunk whose offset is &amp;lt; O_x, or whose offset &gt;= O_y.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming your datafile is sorted, the chunk range you're interested in is contained between the minimum offset and the maximum offset selected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The virtual offsets are 64bit unsigned integers. The higher 48 bits of the virtual chunk offset corresponds to the file offset of the datafile (the other 16 you don't need. They are the offset of the record within that decompressed chunk).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Software&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The software I have found typically mimics closely the hts_lib API. And hts_lib doesn't export a function which loads an index without also requiring a datafile. As far as I know.&lt;/p&gt;&#xA;" OwnerUserId="3235" LastEditorUserId="3235" LastEditDate="2020-02-29T03:47:50.877" LastActivityDate="2020-02-29T03:47:50.877" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11260" PostTypeId="2" ParentId="11196" CreationDate="2020-01-28T15:13:07.623" Score="1" Body="&lt;p&gt;Here's my attempt.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse)&#xA;library(S4Vectors)&#xA;&#xA;# This would probably be read in using `read_tsv()` or something&#xA;rawData &amp;lt;- structure(&#xA;    list(&#xA;        Readlength = c(25, 26, 27, 28, 29, 30, 31, 32, &#xA;                       33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, &#xA;                       49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, &#xA;                       65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, &#xA;                       81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, &#xA;                       97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, &#xA;                       110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, &#xA;                       123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, &#xA;                       136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, &#xA;                       149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, &#xA;                       162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, &#xA;                       175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, &#xA;                       188, 189, 190, 191, 192, 193, 194, 195, 196, 197, 198, 199, 200, &#xA;                       201, 202, 203, 204, 205, 206, 207, 208, 209, 210, 211, 212, 213, &#xA;                       214, 215, 216, 217, 218, 219, 220, 221, 222, 223, 224, 225, 226, &#xA;                       227, 228, 229, 230, 231, 232, 233, 234, 235, 236, 237, 238, 239, &#xA;                       240, 241, 242, 243, 244, 245, 246, 247, 248, 249, 250, 251, 252, &#xA;                       253, 254, 255, 256, 257, 258, 259, 260, 261, 262, 263, 264, 265, &#xA;                       266, 267, 268, 269, 270, 271, 272, 273, 274, 275, 276, 277, 278, &#xA;                       279, 280, 281, 282, 283, 284, 285, 286, 287, 288, 289), &#xA;        Sample1 = c(457, 603, 599, 686, 796, 1009, 1225, 1579, 1817, 2322, 2861, 3460, &#xA;                    4161, 4930, 5910, 7007, 8339, 10105, 11720, 13312, 15430, 17515, &#xA;                    20022, 22825, 26150, 28998, 33236, 37302, 41212, 45683, 49844, &#xA;                    53833, 58486, 64650, 69514, 76202, 83273, 90783, 98805, 107247, &#xA;                    115200, 123359, 134258, 142179, 155549, 165425, 178681, 194293, &#xA;                    211417, 225639, 242618, 259391, 274041, 295365, 315802, 337826, &#xA;                    363906, 388803, 416728, 446797, 474343, 496615, 524080, 549720, &#xA;                    573864, 604559, 632324, 657924, 686875, 712778, 732786, 755081, &#xA;                    772734, 787861, 807004, 823211, 836362, 853124, 865378, 874578, &#xA;                    887322, 888977, 889700, 893138, 887592, 888684, 887037, 887121, &#xA;                    883085, 886536, 875703, 870749, 863969, 849188, 840690, 834472, &#xA;                    822558, 811864, 805300, 794602, 784650, 775107, 763282, 748652, &#xA;                    736764, 723134, 710686, 701834, 689531, 678070, 668757, 654506, &#xA;                    644720, 630305, 615571, 603896, 597816, 584121, 571689, 564142, &#xA;                    552281, 543184, 532683, 520019, 510484, 507832, 495343, 484169, &#xA;                    471422, 460944, 448227, 437738, 428359, 417445, 408065, 395244, &#xA;                    386510, 377028, 366563, 357659, 348242, 337553, 327824, 317584, &#xA;                    306454, 295255, 284770, 274444, 264469, 253923, 243462, 233021, &#xA;                    223360, 213341, 202191, 193325, 182022, 173011, 164207, 154985, &#xA;                    147070, 138927, 130203, 122935, 115565, 107969, 100995, 94510, &#xA;                    88365, 82247, 77315, 71657, 66920, 61298, 57765, 53811, 48906, &#xA;                    44847, 41546, 38125, 35039, 31902, 29076, 26271, 24037, 21487, &#xA;                    19187, 17317, 15741, 14000, 12494, 10963, 9928, 8652, 7604, 6515, &#xA;                    5735, 4783, 4108, 3611, 3224, 2676, 2261, 1936, 1664, 1529, 1180, &#xA;                    975, 897, 812, 712, 587, 555, 524, 443, 411, 382, 344, 340, 362, &#xA;                    325, 290, 319, 293, 304, 354, 296, 315, 315, 331, 321, 326, 349, &#xA;                    370, 318, 344, 346, 341, 345, 348, 371, 405, 380, 365, 366, 351, &#xA;                    431, 340, 405, 370, 374, 385, 361, 410, 410, 368, 397, 379, 384, &#xA;                    389, 419, 403, 423, 480, 478), &#xA;        Sample2 = c(580, 648, 715, 914, &#xA;                    1018, 1366, 1653, 2135, 2678, 3424, 4167, 5220, 6088, 7609, 9308, &#xA;                    11062, 13341, 16288, 19023, 22152, 25360, 28789, 32985, 38019, &#xA;                    42972, 49293, 56239, 62791, 69865, 76922, 82906, 90092, 98194, &#xA;                    105602, 114319, 123807, 133289, 146387, 159302, 168856, 180408, &#xA;                    191083, 200708, 212830, 226788, 239063, 255338, 272725, 288512, &#xA;                    307752, 324919, 338429, 354866, 373456, 389945, 411033, 437069, &#xA;                    460419, 487959, 518295, 538191, 558646, 583043, 601107, 623715, &#xA;                    648209, 668883, 698291, 727313, 745380, 766930, 784337, 793358, &#xA;                    804737, 820443, 826114, 839164, 853144, 861670, 875069, 883285, &#xA;                    875065, 877648, 875875, 865221, 863268, 863513, 855519, 854312, &#xA;                    855445, 844795, 837992, 830157, 809476, 798965, 791200, 776191, &#xA;                    765913, 761200, 747483, 739639, 730152, 713285, 699446, 686165, &#xA;                    671281, 658314, 650564, 635331, 628475, 617210, 601336, 592700, &#xA;                    579707, 562022, 550812, 540016, 526066, 516774, 509033, 496389, &#xA;                    487283, 477035, 461698, 451095, 446390, 431015, 417551, 404674, &#xA;                    390280, 380134, 369703, 354902, 343343, 331193, 315775, 305048, &#xA;                    292336, 277876, 267642, 255871, 242636, 231054, 219066, 204829, &#xA;                    192423, 180440, 166529, 155605, 145166, 134932, 124474, 114975, &#xA;                    105267, 95632, 87027, 78802, 71116, 64758, 57082, 51728, 46295, &#xA;                    40908, 36275, 31825, 28028, 24447, 21125, 18489, 15878, 13881, &#xA;                    12183, 10305, 9096, 7759, 6645, 5532, 4642, 3997, 3360, 2858, &#xA;                    2447, 2185, 1811, 1494, 1313, 1028, 954, 819, 693, 614, 556, &#xA;                    549, 421, 404, 394, 325, 301, 308, 290, 307, 304, 273, 303, 302, &#xA;                    291, 313, 288, 274, 281, 318, 288, 299, 314, 336, 354, 339, 342, &#xA;                    366, 353, 373, 340, 345, 341, 365, 381, 366, 415, 399, 377, 410, &#xA;                    421, 433, 429, 439, 433, 400, 419, 499, 418, 454, 441, 476, 464, &#xA;                    461, 483, 481, 503, 493, 529, 469, 503, 496, 491, 521, 514, 501, &#xA;                    546, 502, 513, 571, 519, 536, 653, 564)),&#xA;    class = c(&quot;spec_tbl_df&quot;, &quot;tbl_df&quot;, &quot;tbl&quot;, &quot;data.frame&quot;),&#xA;    row.names = c(NA, -265L), &#xA;    spec = structure(&#xA;        list(&#xA;            cols = list(&#xA;                Readlength = structure(&#xA;                    list(), class = c(&quot;collector_double&quot;, &quot;collector&quot;)&#xA;                    ), &#xA;                Sample1 = structure(&#xA;                    list(), class = c(&quot;collector_double&quot;, &quot;collector&quot;)&#xA;                ), &#xA;                Sample2 = structure(&#xA;                    list(), class = c(&quot;collector_double&quot;, &quot;collector&quot;))&#xA;                ), &#xA;            default = structure(&#xA;                list(), class = c(&quot;collector_guess&quot;, &quot;collector&quot;)&#xA;            ), &#xA;            skip = 1), &#xA;        class = &quot;col_spec&quot;)&#xA;    )&#xA;&#xA;rlData &amp;lt;- rawData %&amp;gt;%&#xA;    pivot_longer( # Put all the counts in a single column&#xA;        cols = contains(&quot;Sample&quot;),&#xA;        names_to = &quot;Sample&quot;,&#xA;        values_to = &quot;Count&quot;&#xA;        ) %&amp;gt;%&#xA;    split(f = .&lt;span class=&quot;math-container&quot;&gt;$Sample) %&amp;gt;% # Separate into each sample&#xA;lapply(function(x){&#xA;    # Now create a run-length encoded vector&#xA;    # This encodes millions of repetitive values into 2 numbers&#xA;    # 1) the value &amp;amp; 2) how many&#xA;    Rle(values = x$Readlength, lengths = x$&lt;/span&gt;Count)&#xA;        # They can be a bit difficult to plot though&#xA;    }) &#xA;&#xA;# I would create the quantiles for the boxplot separately now&#xA;q &amp;lt;- c(min = 0, q25 = 0.25, median = 0.5, q75 = 0.75, max = 1)&#xA;# Collect the stats&#xA;bxp &amp;lt;- rlData %&amp;gt;%&#xA;    lapply(quantile, probs = q) %&amp;gt;% # for every Rle, get the stats &#xA;    as_tibble() %&amp;gt;%&#xA;    mutate(stat = names(q)) %&amp;gt;% # This just collates the output&#xA;    pivot_longer(&#xA;        # Make it long&#xA;        cols = contains(&quot;Sample&quot;),&#xA;        names_to = &quot;Sample&quot;,&#xA;        values_to = &quot;value&quot;&#xA;    ) %&amp;gt;%&#xA;    pivot_wider(&#xA;        # Now spread it out for each stat/column&#xA;        names_from = stat,&#xA;        values_from = value&#xA;    ) %&amp;gt;%&#xA;    mutate(&#xA;        # set a few values for easier plotting&#xA;        IQR = q75 - q25,&#xA;        Sample = as.factor(Sample),&#xA;        x = as.integer(Sample)&#xA;    )&#xA;&#xA;w &amp;lt;- 0.4 # Box width&#xA;bxp %&amp;gt;%&#xA;    mutate(&#xA;        # Set the width of the boxes&#xA;        xmin = x - w,&#xA;        xmax = x + w,&#xA;        # Set the values for the whiskers. &#xA;        # These usually extend 1.5*IQR. &#xA;        # Any points beyond are considered outliers&#xA;        boxMax = case_when(&#xA;            max &amp;gt; q75 + 1.5*IQR ~ q75 + 1.5*IQR,&#xA;            max &amp;gt; q75 + 1.5*IQR ~ max&#xA;        ),&#xA;        boxMin = case_when(&#xA;            min &amp;lt; q25 - 1.5*IQR ~ q25 - 1.5*IQR,&#xA;            min &amp;gt; q25 - 1.5*IQR ~ min&#xA;        )&#xA;    ) %&amp;gt;%&#xA;    ggplot(aes(x = x, y = median)) +&#xA;    geom_rect(&#xA;        # Draw the box&#xA;        aes(xmin = xmin, xmax = xmax, ymin = q25, ymax = q75),&#xA;        fill = &quot;grey80&quot;,&#xA;        colour = &quot;black&quot;&#xA;    ) +&#xA;    geom_segment(&#xA;        # Add the median&#xA;        aes(x = xmin, xend = xmax, yend = median),&#xA;        colour = &quot;black&quot;&#xA;    ) +&#xA;    geom_segment(&#xA;        # The top whiskers&#xA;        aes(x = x, xend = x, y = q75, yend = boxMax)&#xA;    ) +&#xA;    geom_segment(&#xA;        # The bottom whiskers&#xA;        aes(x = x, xend = x, y = q25, yend = boxMin)&#xA;    ) +&#xA;    geom_point(aes(y = max)) + # Add some symbolic outliers if you want&#xA;    geom_point(aes(y = min)) +&#xA;    scale_x_continuous(&#xA;        # Tidy up the x-axis ticks&#xA;        breaks = bxp&lt;span class=&quot;math-container&quot;&gt;$x,&#xA;    labels = bxp$&lt;/span&gt;Sample&#xA;    ) +&#xA;    labs(&#xA;        # Tidy up the labels&#xA;        x = &quot;Library&quot;,&#xA;        y = &quot;Read Length&quot;&#xA;    ) +&#xA;    theme_bw() # Get rid of the stupid grey background&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That gives me this figure&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/r3Ice.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/r3Ice.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that solves it for you. I probably went the long way around though :)&lt;/p&gt;&#xA;" OwnerUserId="7020" LastActivityDate="2020-01-28T15:13:07.623" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11261" PostTypeId="2" ParentId="11254" CreationDate="2020-01-29T07:21:52.870" Score="1" Body="&lt;p&gt;Pathways are generally directed graphs, which can be implemented in python using the &lt;code&gt;networkx&lt;/code&gt; package. Note that there will be cycles in graphs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that there are large numbers of pathways analysis packages that already exist (both python and R packages but also just webservers), so I strongly recommend searching to see if something already exists before wasting time and implementing it yourself.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-01-29T07:21:52.870" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11262" PostTypeId="1" AcceptedAnswerId="11263" CreationDate="2020-01-29T10:38:07.387" Score="-2" ViewCount="100" Body="&lt;p&gt;I have a data frame&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(custom.cn.data)&#xA;      Gene       Sample_name         CN&#xA;1: DDX11L1          F02         0.6788844&#xA;2: DDX11L1          C02        -0.9924524&#xA;3: DDX11L1          A07        -2.3833559&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to convert values in CN column &amp;lt; 0 to Del and &gt;0 to Amp&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I am getting error&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; a %&amp;gt;% mutate(q = replace(CN, which(CN&amp;lt;0), Del))&#xA;Error in replace(CN, which(CN &amp;lt; 0), Del) : object 'Del' not found&#xA;&#xA;&amp;gt; a&lt;span class=&quot;math-container&quot;&gt;$CN &amp;lt;- replace(a$&lt;/span&gt;CN, which(a&lt;span class=&quot;math-container&quot;&gt;$CN &amp;lt; 0), Del)&#xA;Error in replace(a$&lt;/span&gt;CN, which(a$CN &amp;lt; 0), Del) : object 'Del' not found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any help?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2020-01-30T11:15:32.617" Title="Converting values to string" Tags="|r|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11263" PostTypeId="2" ParentId="11262" CreationDate="2020-01-29T10:42:12.237" Score="6" Body="&lt;pre&gt;&lt;code&gt;custom.cn.data&lt;span class=&quot;math-container&quot;&gt;$new_column &amp;lt;- ifelse(custom.cn.data$&lt;/span&gt;CN &amp;lt; 0, &quot;Del&quot;, &quot;Amp&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this scenario &lt;code&gt;CN == 0&lt;/code&gt; would be &lt;code&gt;Amp&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-01-29T10:42:12.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11264" PostTypeId="2" ParentId="11262" CreationDate="2020-01-29T10:48:09.770" Score="4" Body="&lt;p&gt;The following code would create a new column with the desired values, 0 values will have &lt;code&gt;NA&lt;/code&gt; in this new column.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;custom.cn.data[custom.cn.data$CN &amp;lt; 0, &quot;CN_string&quot;] &amp;lt;- &quot;Del&quot; &#xA;custom.cn.data[custom.cn.data$CN &amp;gt; 0, &quot;CN_string&quot;] &amp;lt;- &quot;Amp&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6955" LastActivityDate="2020-01-29T10:48:09.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11265" PostTypeId="1" CreationDate="2020-01-29T13:37:56.677" Score="1" ViewCount="149" Body="&lt;p&gt;I have a metagenomic dataset across multiple samples that has been grouped and binned (we did this due to poor sequence quallity).&lt;/p&gt;&#xA;&lt;p&gt;It is several samples taken from termite guts of differing castes, the samples have been grouped to make binning easier due to poor sequence suitability. They have been binned via ASVs, assembling reads from all samples to generate the putative bacterial species. So I have fasta files of putative species that have been annotated via prodigal. I want to now calculate the abundance of each bin in a given sample. I think the best way to do this would be at the gene level by looking at the number of reads that map to a gene in any given sample. I am just unsure how to do this.&lt;/p&gt;&#xA;&lt;p&gt;I have annotated the bins and now want to calculate the abundance of each bin in a sample but looking at the number of reads from a sample that map to each gene in a bin.&lt;/p&gt;&#xA;&lt;p&gt;Is this a viable method? If so how would I do it and if not what else could I do?&lt;/p&gt;&#xA;" OwnerUserId="7031" LastEditorUserId="73" LastEditDate="2021-07-14T11:41:38.010" LastActivityDate="2021-09-21T09:51:40.440" Title="How to calculate coverage depth of genes in an annotated sequence?" Tags="|genome|metagenome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11267" PostTypeId="2" ParentId="11253" CreationDate="2020-01-30T09:21:41.813" Score="0" Body="&lt;p&gt;You can make use of &lt;code&gt;table()&lt;/code&gt; and &lt;code&gt;prop.table()&lt;/code&gt; to calculate the number and proportions of your cell types with respect to tissue of origin.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;prop.table()&lt;/code&gt; requires a table as input so you can start with something like &lt;code&gt;prop.table(table(object@meta.data$cell_type, object@meta.data$tissue)&lt;/code&gt;. This will create an objet of class &lt;code&gt;table&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Next step is to create a data frame with &lt;code&gt;as.data.frame(your_table_object)&lt;/code&gt; and then to convert it to &quot;long format&quot;, for example with &lt;code&gt;tidyr::pivot_longer()&lt;/code&gt;. You can feed these data in long format to &lt;code&gt;ggplot()&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-01-30T09:21:41.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11268" PostTypeId="2" ParentId="11250" CreationDate="2020-01-30T10:55:19.290" Score="5" Body="&lt;p&gt;&lt;strong&gt;Overview&lt;/strong&gt;&#xA;The central focus of the tree is to highlight the key biological concern of the new coronavirus, 2019-nCov. The key concern is the genetic similarities to SARS epidemic, and relates to the SARS receptor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;SARS background&lt;/strong&gt; SARS is endemic in bats (your &lt;a href=&quot;https://www.ecohealthalliance.org/2020/01/phylogenetic-analysis-shows-novel-wuhan-coronavirus-clusters-with-sars&quot; rel=&quot;noreferrer&quot;&gt;BioRxiv tree&lt;/a&gt; partly shows that and &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/11251/how-to-interpret-this-blast-tree&quot;&gt;this tree&lt;/a&gt; definitely shows it) and in the 2002 epidemic infected civet cats which then infected humans. More importantly it is then transmitted from one human to another. This is a consequence of the SARS receptor been able to exploit the cellular receptors in the respiratory tract of bats, civet cats and humans and use this tissue as a site of replication. The concern is exacerbated because the tree here shows SARS independently infected humans on two separate occasions, suggesting the cross-over has an underlying genetic basis, which is part of the &quot;SARS lineage&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://www.ecohealthalliance.org/2020/01/phylogenetic-analysis-shows-novel-wuhan-coronavirus-clusters-with-sars&quot; rel=&quot;noreferrer&quot;&gt;BioRxiv 2019-nCov tree&lt;/a&gt;&lt;/strong&gt; Your tree shows the 2019-nCov has a recent common ancestor with SARS in comparison to the rest of the betacoronaviruses. This therefore provides some circumstantial evidence the receptor mechanism and ability to frequently crossover from bats and ultimately transmit human to human could be shared with SARS. At an evolutionary level (which is what ANY tree is supposed to reconstruct) it raises the question which common ancestor of SARS did the &quot;SARS receptor&quot; originate. I'd need to draw a diagram to better demonstrate this point, but I hope you get the idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;The &quot;influenza receptor&quot;&lt;/strong&gt; The analogy is with influenza virus and the entire epidemiology of influenza hinges around is sialic acid receptor of influenza and its ability to bind on to the cellular receptors in the upper/lower respiratory tract of birds, pigs and humans. The hypothesised mechanism is called the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2702078/&quot; rel=&quot;noreferrer&quot;&gt;mixing vessel theory&lt;/a&gt; and is the classic epidemiological understanding of how new pathogenic influenza pandemics occur. If you replace &quot;birds and pigs&quot; with e.g. &quot;bats, civet cats and humans&quot; you get the idea why this could be scary. We don't the intemediate host of 2019-nCov ... but I speculate there must be one, unless eating Cov-infected bats is common in China.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Technical details of the BioRxiv tree&lt;/strong&gt; The tree is a nicely diverse selection of the beta-coronaviruses. The authors have rooted the tree using the outgroups of delacoronaviruses and gammacoronaviruses, so it is a good robust selection of outgroups which can be used to correctly identify the direct of evolution of betacoronavirus divergence. In tree theory (its formal name is phylogenetics theory), extensive rooting is good and minimises artefacts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the earliest members of the betacoronaviruses to diverge is MERS (Middle Eastern respiratory syndrome), which form a single &quot;clade&quot; (all viruses share a unique common ancestor) and these represent around 50% amino acid divergence from SARS/2019-nCov. The selection of MERS in the tree from both camels and humans looks good. The selection of other betacoronaviruses looks great, I wasn't aware of the &quot;ruminant clade&quot;  at all involving buffalo, cow etc .. infections and there was an associated human infection. There are loads of bat isolates of the betacoronaviruses throughout the tree, but we eventually arrive at the &quot;SARS clade&quot;. The authors show that 2019-nCov is an outgroup to the SARS clade and shows a close relationship to one (BioRxiv tree) or two (this tree) bat isolates. Looking at the precise SARS clade (better shown in this tree) we can see loads of bat virus associating with SARS lineages. We therefore assume the reservoir to SARS and likely 2019-nCov is bat, moreover that the single ancestor to both viruses was a bat (it is called a parsimonious hypothesis). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The one thing the BioRXiv tree omits, for example with regards &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/11251/interpretion-of-my-coronavirus-2019-ncov-wuhan-china-blast-tree&quot;&gt;this tree&lt;/a&gt;, is the diversity of SARS and in particularly the two independent origins of SARS, which is a weakness of their analysis, particular if this fed into downstream analysis. It is not to say the authors were wrong, but it was uncool. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Word of caution&lt;/strong&gt; my understanding is the divergence between SARS and 2010-nCov is around 15% and this is a quite a large amount of genetic divergence, even if both viruses share a most recent common-ancestor, form part of the same clade and receptor peptide motifs. Nevertheless there is sufficient amino acid divergence to generate notable differences in epidemiology, clinical symptom and transmission. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;MERS, SARS and 2019-nCov mortality rates&lt;/strong&gt; It is worth noting that within the betacoronaviruses the mortality rates between difference clinically important viruses is very different. MERS has a mortality rate of 40-60%, SARS is around 10% but 2019-nCov is &amp;lt;2.5%. The 2019-nCov mortality rate is still important given it has infected more and could potentially infect alot more people than SARS.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-01-30T13:01:51.960" LastActivityDate="2020-01-30T13:01:51.960" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="11269" PostTypeId="2" ParentId="11262" CreationDate="2020-01-30T11:15:32.617" Score="2" Body="&lt;p&gt;Try to stay within one package (paradigm) for one problem. This will make the code more readable, easier to debug.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment you have data as a &lt;em&gt;data.table&lt;/em&gt; object from &lt;strong&gt;data.table&lt;/strong&gt; package, then trying to add a new column using &lt;em&gt;mutate&lt;/em&gt; from &lt;strong&gt;dplyr&lt;/strong&gt; package, and within &lt;em&gt;mutate&lt;/em&gt; using &lt;strong&gt;base&lt;/strong&gt; &lt;em&gt;replace&lt;/em&gt; function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Solutions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;data.table&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# update by row index&#xA;custom.cn.data[ CN &amp;lt; 0, NewColumn := &quot;Del&quot; ]&#xA;custom.cn.data[ CN &amp;gt;= 0, NewColumn := &quot;Amp&quot; ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or use &lt;em&gt;ifelse&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;custom.cn.data[, NewColumn := ifelse(CN &amp;lt; 0, &quot;Del&quot;, &quot;Amp&quot;) ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;base&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;see &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/11264/131&quot;&gt;@DavyCat's&lt;/a&gt; and &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/11263/131&quot;&gt;@haci's&lt;/a&gt; answers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;dplyr&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, this will change the &lt;em&gt;data.table&lt;/em&gt; into &lt;em&gt;data.frame&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# ifelse&#xA;custom.cn.data &amp;lt;- custom.cn.data %&amp;gt;%&#xA;  mutate(NewColumn = ifelse(CN &amp;lt; 0, &quot;Del&quot;, &quot;Amp&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or use &lt;em&gt;case_when&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;custom.cn.data &amp;lt;- custom.cn.data %&amp;gt;%&#xA;  mutate(NewColumn = case_when(&#xA;    CN &amp;lt; 0 ~ &quot;Del&quot;,&#xA;    CN &amp;gt;= 0 ~ &quot;Amp&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="131" LastActivityDate="2020-01-30T11:15:32.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11271" PostTypeId="1" AcceptedAnswerId="11272" CreationDate="2020-01-30T13:50:18.507" Score="1" ViewCount="140" Body="&lt;p&gt;I have been working with several tools on bam files recently and I am not sure how I should interpret some of the outputs.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools idxstats mapped.sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Chr1    15096   0   0&#xA;Chr2    33397   0   0&#xA;Chr3    43888   41  0&#xA;Chr4    20819   1   0&#xA;*   0   0   34&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Documentation says third and fourth column give &quot;mapped reads&quot; and &quot;unmapped reads&quot;. So I interpreted the last line (&lt;code&gt;*&lt;/code&gt;) as the number of unmapped reads as this is the only line with a non 0 value at this field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I used &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bam splitChromosome --in mapped.sorted.bam --out test/aln.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Reference Name: chr6 has 6 records&#xA;Reference Name: Chr7 has 1 records&#xA;Reference Name: Chr3 has 41 records&#xA;Reference Name: Chr4 has 1 records&#xA;Reference Name: * has 34 records&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And looking at the bams outputted (one per reference sequence) in &lt;code&gt;test/&lt;/code&gt; I find an &lt;code&gt;aln.UnknownChrom.bam&lt;/code&gt; that I am not sure where it comes from. I actually found 97 unique reads ID in it, where I was waiting for 34 or less. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So my &lt;strong&gt;first question&lt;/strong&gt; is : Does &quot;*&quot; actually means &quot;all reads that didn't mapp&quot; and this information is stored in way I didn't know about in BAM files ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And my &lt;strong&gt;second question&lt;/strong&gt; would be : Anybody knows where this &lt;code&gt;aln.UnknownChrom.bam&lt;/code&gt; comes from when using &lt;code&gt;splitChromosome&lt;/code&gt; ? It is a source of error in a pipeline I am building and I'd like to get rid of it, if these are not actual mapping informations.&lt;/p&gt;&#xA;" OwnerUserId="6273" LastEditorUserId="6273" LastEditDate="2020-10-13T09:04:24.127" LastActivityDate="2020-10-13T09:04:24.127" Title="samtools / bamUtil | Meaning of &lt;wildcard&gt; as Reference Name" Tags="|sam|samtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11272" PostTypeId="2" ParentId="11271" CreationDate="2020-01-30T14:07:22.740" Score="2" Body="&lt;p&gt;As per the SAM/BAM format specifications:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&quot;RNAME: Reference sequence NAME of the alignment. If @SQheader lines are present, RNAME(if not ‘*’) must be present in one of the SQ-SN tag. &lt;strong&gt;An unmapped segment without coordinate has a ‘*’ at this field.&lt;/strong&gt;  However, an unmapped segment may also have an ordinary coordinate such that it can be placed at a desired position after sorting.  If RNAME is ‘*’, no assumptions can be made about POS and CIGAR.&quot; -- &lt;a href=&quot;http://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://samtools.github.io/hts-specs/SAMv1.pdf&lt;/a&gt;, emphasis mine&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So, yes, these are the unmapped reads. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't used this &lt;code&gt;splitChromosome&lt;/code&gt; utility before myself, but as it splits the BAM file into one BAM file per chromosome, I suspect it simply also creates a BAM file for the unknown (&lt;code&gt;*&lt;/code&gt;) chromosome. Why the numbers don't match I can't tell.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to deal with that file in your pipeline will depend on what the pipeline is meant to do. But if you aren't interested in these reads, I suppose you could simply remove it before continuing to the next step.&lt;/p&gt;&#xA;" OwnerUserId="6955" LastEditorUserId="6955" LastEditDate="2020-01-30T14:58:36.630" LastActivityDate="2020-01-30T14:58:36.630" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11273" PostTypeId="1" CreationDate="2020-01-30T17:00:43.330" Score="1" ViewCount="68" Body="&lt;p&gt;I have cfDNA samples which were sequenced (at about 300-1000X) on a 30-gene panel via a capture-seq protocol for the purposes of calling variants on circulating tumour DNA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the resulting variant calls (from both varscan and Mutect2) are dominated by transversions from C&gt;A and G&gt;T at low allele frequencies, which are characteristic of DNA oxidation of guanine to 8-oxoguanine, and which I suspect are artefacts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have matched baseline tumour samples from the same patients, sequenced in the same way, (and there are a few C&gt;A/G&gt;T transversions at low AFs in these samples that I can filter out by a direct allele fraction filter).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is this, what is the best way of filtering out the false positive mutations derived from the 8-oxoG formation and retaining the true positives (which will be at low allelic fraction)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some background: I do have serial (cfDNA) time points for some patients, so I am loath to just look at mutations called in the baseline tumour samples, as any (true) novel variants appearing later in the time points would be missed.&lt;/p&gt;&#xA;" OwnerUserId="2244" LastActivityDate="2020-01-30T17:00:43.330" Title="Filtering out 8-oxoG-derived artefacts in variant calls from cfDNA" Tags="|variants|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11276" PostTypeId="1" CreationDate="2020-01-31T12:12:49.253" Score="2" ViewCount="224" Body="&lt;p&gt;I've aligned reads onto an haploid reference genome. I'd like to have a consensus sequence of all my aligned reads that doesn't take into account the reference genome I used (i.e. I just want the nucleotides from my reads at each base (taking into account frequency and quality) but if there is no read a one position I rather have N/? than the reference nucleotide. &#xA;Is there a way to do this ? Thanks&lt;/p&gt;&#xA;" OwnerUserId="6459" LastActivityDate="2020-02-13T12:50:28.870" Title="Make a fasta out of mapped reads without taking into account the reference" Tags="|sequence-alignment|sam|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11277" PostTypeId="1" CreationDate="2020-01-31T14:11:23.757" Score="-1" ViewCount="260" Body="&lt;p&gt;I have a big clinical data file with 96 columns like age, gender, BMI, etc&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to see which of these clinical characteristics respond to chemotherapy. Response to chemotherapy divides patients to two groups as &lt;code&gt;yes&lt;/code&gt; and &lt;code&gt;no&lt;/code&gt;. Among these clinical characteristics, some are based on continuous data and some reflect `categorical data.  Is it possible to write a function to loop over these characteristics and test for relationship between response to chemotherapy? Something like Wilcox test for continuous values and chi square for categorical characteristics. By hand, it takes too much to do for all one by one !&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my data&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ID  response_to_chemo   BMI Chemo   Predictor   DJANGO  gender&#xA;AH/155  no  50  1   1   0   M&#xA;RS/022  no  67  1   1   0   M&#xA;RS/027  no  80  1   1   1   F&#xA;ST/023  no  65  1   1   0   M&#xA;SH/051  yes 47  1   1   0   M&#xA;AH/075  yes 90  0   1   0   M&#xA;RS/047  no  67  0   1   0   F&#xA;ST/029  no  61  1   1   0   F&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For instance &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;myTable &amp;lt;- table(clin&lt;span class=&quot;math-container&quot;&gt;$gender, clin$&lt;/span&gt;response_to_chemo)&#xA;&#xA;chisq.test(myTable)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Gives p-value for testing the relationship of gender (CATEGORICAL) in response to chemotherapy&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;t.test(clin&lt;span class=&quot;math-container&quot;&gt;$BMI ~ clin$&lt;/span&gt;response_to_chemo)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;is for BMI&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I meant a function calculating these p-values for clinical characteristics one by one &lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="7374" LastEditDate="2020-03-24T03:11:41.627" LastActivityDate="2023-04-12T20:40:51.597" Title="Is it possible to do this in R?" Tags="|r|database|statistics|software-recommendation|correlation|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11278" PostTypeId="1" CreationDate="2020-01-31T16:30:12.643" Score="1" ViewCount="41" Body="&lt;p&gt;How can I obtain a BED file that describes the splicing sites of all genes for GRCh38 human genome ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2020-01-31T16:30:12.643" Title="How to get the splicing regions (BED) of all genes for Human GRCh38?" Tags="|bed|human|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11280" PostTypeId="2" ParentId="11276" CreationDate="2020-01-31T18:44:38.540" Score="0" Body="&lt;p&gt;Have you googled &lt;em&gt;de novo assembly&lt;/em&gt;?&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-01-31T18:44:38.540" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11281" PostTypeId="1" CreationDate="2020-01-31T19:13:22.650" Score="1" ViewCount="69" Body="&lt;p&gt;I am trying to make a paired matrix of gene-gene correlation. Considering that I have a huge matrix (13000 genes and 900 samples) and for some reasons I don't want to decrease the number of my genes, my gene-correlation matrix would be 13000*13000 and my paired matrix will become 169 million *4 (Column 1: Gene 1; Column 2 : Gene 2; Column 3: Correlations; Column 4: P-values) . In this case, I have to exclude unnecessary calculations as much as I can. I have excluded the situation that Gene 1 = Gene 2. But I couldn't find a way to exclude the condition that &quot;Column 1: Gene 1 ; Column 2: Gene 2 = Column 1: Gene 2; Column 2: Gene 1 &quot;. To make a long story short, correlation between G1 and G2 is equal to G2 and G1. It is like calculating just lower section of diag in a symmetric matrix. I would be grateful if anybody help me in this case. I have enclosed my python codes here:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;import numpy as np&#xA;import scipy&#xA;import math&#xA;import openpyxl&#xA;from openpyxl import Workbook&#xA;from scipy.stats import spearmanr&#xA;&#xA;&#xA;din=pd.read_csv('m_test.csv', index_col=0)&#xA;&#xA;out=pd.DataFrame()&#xA;outdf=pd.DataFrame()&#xA;&#xA;for g1 in din.index:&#xA;    for g2 in din.index:&#xA;        temp=din.loc[[g1, g2]]&#xA;        if g1==g2:&#xA;            next&#xA;        else: &#xA;            spR, spP=spearmanr(temp.loc[g1], temp.loc[g2])&#xA;            frame={'g1':[g1], 'g2':[g2], 'spR':[spR], 'spP':[spP]}&#xA;&#xA;            out=pd.DataFrame(frame)&#xA;&#xA;            outdf=pd.concat([outdf, out])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6670" LastActivityDate="2020-03-14T10:01:05.123" Title="How to exclude the repetition of gene-gene correlation calculation in python?" Tags="|python|statistics|gene-expression|correlation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11282" PostTypeId="2" ParentId="11277" CreationDate="2020-01-31T19:35:12.220" Score="3" Body="&lt;p&gt;The general approach would be to loop through every column starting at column 2. You can use numeric indexes to do that.&lt;/p&gt;&#xA;&lt;p&gt;For each column, check its type. If it is a factor, use your &lt;code&gt;chisq.test&lt;/code&gt; method. If it is of numeric type, use your &lt;code&gt;t.test&lt;/code&gt; approach. Write the results to a list so you can parse them later.&lt;/p&gt;&#xA;" OwnerUserId="650" LastEditorUserId="73" LastEditDate="2023-04-12T20:40:51.597" LastActivityDate="2023-04-12T20:40:51.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11283" PostTypeId="1" AcceptedAnswerId="11285" CreationDate="2020-01-31T20:37:11.530" Score="9" ViewCount="2149" Body="&lt;p&gt;Quote:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;We found 4 insertions in the spike glycoprotein (S) which are unique to the 2019-nCoV and are not present in other coronaviruses. Importantly, amino acid residues in all the 4 inserts have identity or similarity to those in the HIV-1 gp120 or HIV-1 Gag. Interestingly, despite the inserts being discontinuous on the primary amino acid sequence, 3D-modelling of the 2019-nCoV suggests that they converge to constitute the receptor binding site. The finding of &lt;strong&gt;4 unique inserts&lt;/strong&gt; &lt;strong&gt;in the 2019-nCoV, all of which have identity/similarity to amino acid residues in key structural proteins of HIV-1 is unlikely to be fortuitous in nature&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Source: &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2020.01.30.927871v1&quot; rel=&quot;noreferrer&quot;&gt;https://www.biorxiv.org/content/10.1101/2020.01.30.927871v1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CzZTD.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CzZTD.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know this is a preprint paper and it's not yet peer reviewed, but can someone tell me what the implications are if this is true? Does this mean that the virus is artificially constructed?&lt;/p&gt;&#xA;" OwnerUserId="6920" LastEditorUserId="3848" LastEditDate="2020-07-23T08:35:43.337" LastActivityDate="2020-10-26T14:54:17.663" Title="A new paper suggests the Corona Virus has &quot;Uncanny similarity of unique inserts in the 2019-nCoV spike protein to HIV-1&quot; - What does this mean?" Tags="|sequence-alignment|phylogenetics|ngs|phylogeny|covid-19|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11284" PostTypeId="1" CreationDate="2020-01-31T17:30:30.037" Score="0" ViewCount="103" Body="&lt;p&gt;I have multiple ligands (in pdb format) which I want to try and dock to various proteins.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But for each ligand I have only a single conformation (by which I meant the spatial relationship between it's atoms, not its position relative to the target protein).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I generate other possible conformations?&#xA;Is there a generally used tool for that? &lt;/p&gt;&#xA;" OwnerUserId="7057" OwnerDisplayName="user2182857" LastActivityDate="2020-02-03T09:40:40.880" Title="How to generate conformations for a ligand for in silico screening?" Tags="|pdb|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11285" PostTypeId="2" ParentId="11283" CreationDate="2020-01-31T23:10:57.760" Score="11" Body="&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt; The article has now been withdrawn with the following note:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This paper has been withdrawn by its authors. They intend to revise it&#xA;  in response to comments received from the research community on their&#xA;  technical approach and their interpretation of the results. If you&#xA;  have any questions, please contact the corresponding author.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;This is very odd, and will require a rigorous investigation, but my initial reaction is one of scepticism. Considering just the 1st insert, the insert sequence is &lt;code&gt;GTNGTKR&lt;/code&gt;, short at just 7 amino acids. A simple BLASTP vs NR did not find perfect matches to HIV sequences, but did reveal 100% identity across the full sequence length to &gt;50 other short protein sequences, which could be spurious 'chance' hits of course. Many of the top 100 hits are against eukaryotic sequences, for example one is against &lt;em&gt;Pristionchus pacificus&lt;/em&gt;, a type of nematode worm (see alignment below). Someone needs to do a proper peer review of this preprint before any conclusions are drawn from it.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;tank-1 [Pristionchus pacificus]&#xA;Sequence ID: PDM74036.1 Length: 2481 &#xA;Range 1: 1474 to 1480&#xA;&#xA;Score:24.0 bits(49), Expect:2477, &#xA;Method:, &#xA;Identities:7/7(100%), Positives:7/7(100%), Gaps:0/7(0%)&#xA;&#xA;Query  1     GTNGTKR  7&#xA;             GTNGTKR&#xA;Sbjct  1474  GTNGTKR  1480&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE 1:&lt;/strong&gt; Things are moving fast. There are now 10 comments under the preprint, all of them critical of the idea that these inserts are meaningfully similar to HIV-1 sequences for the reason I outlined above (i.e. short sequences with many hits against organisms from across the tree of life) and additional points, such as small insertions being quite normal evolution for RNA viruses. There are also similar &lt;a href=&quot;https://twitter.com/trvrb/status/1223337991168380928?s=20&quot; rel=&quot;nofollow noreferrer&quot;&gt;critiques on twitter&lt;/a&gt; and biorxiv has added the following disclaimer header to their website:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;bioRxiv is receiving many new papers on coronavirus 2019-nCoV.   A&#xA;  reminder: these are preliminary reports that have not been&#xA;  peer-reviewed. They should not be regarded as conclusive, guide&#xA;  clinical practice/health-related behavior, or be reported in news&#xA;  media as established information.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have never seen such rapid (and unanimous) post-publication peer review before. It is therefore already (just 1 day after publication) fairly clear that at least some claims in this study are completely false and there is no good evidence to support the claim that the 2019-nCoV have acquired sequences from HIV or evidence to suggest the virus is engineered.&lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="77" LastEditDate="2020-02-05T07:49:50.970" LastActivityDate="2020-02-05T07:49:50.970" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11286" PostTypeId="2" ParentId="7415" CreationDate="2020-02-01T00:23:46.150" Score="2" Body="&lt;p&gt;The UCSC Genome Browser downloads page has old human genome assembly versions way back to the first human genome ever published (by UCSC in 2000).&#xA;&lt;a href=&quot;http://hgdownload.soe.ucsc.edu/downloads.html#human&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://hgdownload.soe.ucsc.edu/downloads.html#human&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We also have a variety of archived dbSNP data up to dbSNP141 on hg38, dbSNP 131 on hg37, dbSNP126 on hg18:&#xA;&lt;a href=&quot;http://hgdownload.soe.ucsc.edu/goldenPath/hg38/database/snp141.txt.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://hgdownload.soe.ucsc.edu/goldenPath/hg38/database/snp141.txt.gz&lt;/a&gt;&#xA;&lt;a href=&quot;http://hgdownload.soe.ucsc.edu/goldenPath/hg19/database/snp138.txt.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://hgdownload.soe.ucsc.edu/goldenPath/hg19/database/snp138.txt.gz&lt;/a&gt;&#xA;&lt;a href=&quot;http://hgdownload.soe.ucsc.edu/goldenPath/hg18/database/snp126.txt.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://hgdownload.soe.ucsc.edu/goldenPath/hg18/database/snp126.txt.gz&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;They're in a custom table format, which can be turned into VCF manually.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also get all of this data from Table Browser and add filters, etc.:&#xA;&lt;a href=&quot;http://genome.ucsc.edu/cgi-bin/hgTables&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genome.ucsc.edu/cgi-bin/hgTables&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7067" LastActivityDate="2020-02-01T00:23:46.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11287" PostTypeId="1" CreationDate="2020-02-01T00:35:14.790" Score="3" ViewCount="276" Body="&lt;p&gt;I have a BAM file produced using BWA-MEM and I need to calculate the coverage of each exon in this sequencing. I tried using this command:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;bedtools coverage -a exome_sorted.bed -b exome_alignment.bam -sorted&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;And the result was the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1    12595   12802   ref|DDX11L1,ref|NR_046018,ens|ENST00000518655,ens|ENST00000450305,ens|ENST00000456328,ens|ENST00000515242  .       ref|DDX11L1     0       0       207     0.0000000&#xA;chr1    13163   13658   ref|DDX11L1,ref|NR_046018,ens|ENST00000518655,ens|ENST00000450305,ens|ENST00000456328,ens|ENST00000515242  .       ref|DDX11L1     0       0       495     0.0000000&#xA;chr1    14620   15015   ref|WASH7P,ref|NR_024540,ens|ENST00000538476,ens|ENST00000438504,ens|ENST00000488147,ens|ENST00000541675,ens|ENST00000423562       .       ref|WASH7P      0       0       395     0.0000000&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That is, zero coverage for all exons. But there are reads mapping to the exons, as I can see by opening my BAM file on IGV and looking at some exonic regions:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TdnLY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TdnLY.png&quot; alt=&quot;IGV view of the exonic region chr1:12,595-12,802 , clearly showing many reads mapping.&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have already tried sorting the BED file using &lt;code&gt;sort -k1,1 -k2,2n&lt;/code&gt; (&lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/coverage.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;as recommended on the Bedtools documentation&lt;/a&gt;) and using &lt;code&gt;bedtools sort&lt;/code&gt; with the &lt;code&gt;-faidx&lt;/code&gt; option, but neither option solved my issue.&lt;/p&gt;&#xA;&lt;p&gt;if I skip the &lt;code&gt;-sorted&lt;/code&gt; option, the memory usage becomes way too high.&lt;/p&gt;&#xA;&lt;p&gt;I have also checked if the chromosome names are the same in the BAM file and in the BED file, using &lt;code&gt;samtools view -H&lt;/code&gt;. Both use &lt;code&gt;chr1,chr2,...,chr22,chrX,chrY&lt;/code&gt;, although BAM has &lt;code&gt;chrM&lt;/code&gt; and stuff like &lt;code&gt;chrUn_gl000214&lt;/code&gt; which are not on the BED.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&lt;p&gt;@ATpoint pointed out that all visible reads in this region have MAPQ0. I looked at another exon that did not have this issue, but Bedtools still outputs zero coverage. IGV picture below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qJvWz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qJvWz.png&quot; alt=&quot;IGV view of the exonic region chr1:12,595-12,802 , clearly showing many reads mapping.&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3389" LastEditorUserId="73" LastEditDate="2021-06-17T10:30:42.703" LastActivityDate="2021-06-17T10:30:42.703" Title="Bedtools wrongly indicates zero coverage" Tags="|bedtools|coverage|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11289" PostTypeId="2" ParentId="11283" CreationDate="2020-02-01T15:50:02.980" Score="7" Body="&lt;p&gt;Normally &quot;inserts&quot; used in the manuscript are &quot;indels&quot; in protein alignments, short for insertions and deletions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I think has happened is a group investigating indels in HIV env noticed indels in 2019-nCov. Essentially I think the correlation is spurious - but I haven't test it, but the area of research in understanding indels is certainly valid and important.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is certain is that indels induce a large structural change to a protein structure and any Gibbs free-energy style calculation will identify this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Vaccine&lt;/strong&gt;&#xA;The spike protein will be the primary candidate to make a 2019-nCov vaccine and that is a very important reason why the sequence was rapidly released. So it is an important protein and the structural changes indels induce mean that a SARS vaccine will probably not provide much protection against 2019-Cov, even apart from the amino acid divergence (below). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Differences 2019-nCov vs HIV&lt;/strong&gt;&#xA;In summary, alot. HIV env and particularly HIV gag are very different from coronaviruses, both in the mechanism of genome replication, coronavirus never leaves the cytoplasm, clinical outcomes, tissue tropism and duration of infection.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Similarities&lt;/strong&gt;&#xA;HIV env and the glycoprotein spike of coronaviruses are the receptor binding protein to gain entry into a cell. They are called &lt;strong&gt;structural&lt;/strong&gt; proteins. Entry to a cell can be blocked by antibodies and these antibodies are called &quot;neutralizing antibodies&quot;. Neutralizing antibodies are catastrophic for a virus. Other antibody responses can be effective, such as IgM, but to clear an infection just using antibodies, you need neutralising antibodies. Both HIV env and the coronavirus spike protein are subject to neutralising antibodies. HIV gag has nothing to do with HIV env, in terms of function or antibody exposure. This is why the spike protein will be the primary vaccine candidate for a subunit vaccine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Coincidence, law of chance&lt;/strong&gt;&#xA;There is large variation of indels in HIV env within HIV and what the authors are inferring is there is a resemblence to that between SARS and 2019-nCov. In my opinion this is a coincidence, because they are comparing a large repertoire of HIV varients against a single indel pattern in the coronaviruses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Why coronavirus indels?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is a very good question. Generically indels in viral surface antigen genes are common, much more common in other proteins - such as those involved in virus replication (non-structural proteins). The amino acid identity between SARS and 2019-nCov is 80%, and in any virus, such as flaviviruses 80% identity means indels will be present in surface antigens between the viruses. The answer is it is not unusual in any RNA virus to see indels at a comparatively large amino acid divergence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What function could they serve&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've briefly looked at indel bioinformatics between flaviviruses (Zika virus, yellow fever virus etc..) notably using envelope (E) protein sequences, and they also occur between African Zika viruses in the E-protein. E-protein being the equivalent of coronavirus spike protein, the receptor-binding protein. No-one has ascribed a function to them and that is the problem with this manuscript.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hypthoses&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;One theory is that a structural change in the protein will occur to stop antibody binding.&lt;/li&gt;&#xA;&lt;li&gt;Another theory is they have functional differences, such as cell tropism&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Bioinformatically separating the two theories is extremely hard without wetlab experimentation.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-02-01T18:31:01.017" LastActivityDate="2020-02-01T18:31:01.017" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11290" PostTypeId="2" ParentId="11284" CreationDate="2020-02-01T16:34:43.037" Score="1" Body="&lt;p&gt;One option I found was Frog2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://mobyle.rpbs.univ-paris-diderot.fr/cgi-bin/portal.py#forms::Frog2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://mobyle.rpbs.univ-paris-diderot.fr/cgi-bin/portal.py#forms::Frog2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/tuffery/Frog2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/tuffery/Frog2&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From an input of a &lt;code&gt;mol2&lt;/code&gt; or &lt;code&gt;sdf&lt;/code&gt; file it creates a plethora of conformers, which can be outputted as a &lt;code&gt;pdb&lt;/code&gt; file&lt;/p&gt;&#xA;" OwnerUserId="7057" LastActivityDate="2020-02-01T16:34:43.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11291" PostTypeId="1" CreationDate="2020-02-01T21:36:07.770" Score="1" ViewCount="347" Body="&lt;p&gt;I am looking for a way to download all pdb files that detail a protein-drug complex.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I could download the entirety of the pdb and search manually (or maybe write a script to discard all proteins that do not have an HETATM entry that details a non-water, non-salt molecule), that sounds effort intensive and error prone.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems reasonable that this subset already exist, but I was not able to find it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does such a list/database exists?  &lt;/p&gt;&#xA;" OwnerUserId="7057" LastActivityDate="2020-03-05T08:17:13.430" Title="Looking for a way to download all pdb files that detail a protein-drug complex" Tags="|pdb|interactions|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11292" PostTypeId="1" AcceptedAnswerId="11294" CreationDate="2020-02-03T05:36:49.583" Score="1" ViewCount="239" Body="&lt;p&gt;This is my data &#xA; dput(METADATA)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(Run = c(&quot;SRR2920466&quot;, &quot;SRR2920477&quot;, &quot;SRR2920478&quot;, &#xA;&quot;SRR2920506&quot;, &quot;SRR2920507&quot;, &quot;SRR2920532&quot;, &quot;SRR2920531&quot;, &quot;SRR2920469&quot;, &#xA;&quot;SRR2920470&quot;, &quot;SRR2920481&quot;, &quot;SRR2920482&quot;, &quot;SRR2920500&quot;, &quot;SRR2920501&quot;, &#xA;&quot;SRR2920536&quot;, &quot;SRR2920537&quot;, &quot;SRR2920471&quot;, &quot;SRR2920472&quot;, &quot;SRR2920483&quot;, &#xA;&quot;SRR2920484&quot;, &quot;SRR2920505&quot;, &quot;SRR2920538&quot;, &quot;SRR2920539&quot;, &quot;SRR2920475&quot;, &#xA;&quot;SRR2920476&quot;, &quot;SRR2920487&quot;, &quot;SRR2920488&quot;, &quot;SRR2920542&quot;, &quot;SRR2920543&quot;, &#xA;&quot;SRR2920555&quot;, &quot;SRR2920559&quot;, &quot;SRR2920570&quot;, &quot;SRR2920573&quot;, &quot;SRR2920578&quot;, &#xA;&quot;SRR2920583&quot;, &quot;SRR2920586&quot;, &quot;SRR2920594&quot;, &quot;SRR2920546&quot;, &quot;SRR2920547&quot;, &#xA;&quot;SRR2920548&quot;, &quot;SRR2920549&quot;, &quot;SRR2920550&quot;, &quot;SRR2920551&quot;, &quot;SRR2920553&quot;, &#xA;&quot;SRR2920558&quot;, &quot;SRR2920563&quot;, &quot;SRR2920565&quot;, &quot;SRR2920567&quot;, &quot;SRR2920569&quot;, &#xA;&quot;SRR2920572&quot;, &quot;SRR2920575&quot;, &quot;SRR2920577&quot;, &quot;SRR2920580&quot;, &quot;SRR2920582&quot;, &#xA;&quot;SRR2920585&quot;, &quot;SRR2920589&quot;, &quot;SRR2920590&quot;, &quot;SRR2920591&quot;, &quot;SRR2920593&quot;, &#xA;&quot;SRR2920554&quot;, &quot;SRR2920552&quot;, &quot;SRR2920556&quot;, &quot;SRR2920557&quot;, &quot;SRR2920560&quot;, &#xA;&quot;SRR2920561&quot;, &quot;SRR2920562&quot;, &quot;SRR2920564&quot;, &quot;SRR2920566&quot;, &quot;SRR2920568&quot;, &#xA;&quot;SRR2920571&quot;, &quot;SRR2920574&quot;, &quot;SRR2920576&quot;, &quot;SRR2920581&quot;, &quot;SRR2920584&quot;, &#xA;&quot;SRR2920587&quot;, &quot;SRR2920588&quot;, &quot;SRR2920592&quot;, &quot;SRR2920595&quot;, &quot;SRR2920579&quot;&#xA;), source_name = c(&quot;HSC&quot;, &quot;HSC&quot;, &quot;HSC&quot;, &quot;HSC&quot;, &quot;HSC&quot;, &quot;HSC&quot;, &#xA;&quot;HSC&quot;, &quot;CMP&quot;, &quot;CMP&quot;, &quot;CMP&quot;, &quot;CMP&quot;, &quot;CMP&quot;, &quot;CMP&quot;, &quot;CMP&quot;, &quot;CMP&quot;, &#xA;&quot;GMP&quot;, &quot;GMP&quot;, &quot;GMP&quot;, &quot;GMP&quot;, &quot;GMP&quot;, &quot;GMP&quot;, &quot;GMP&quot;, &quot;Mono&quot;, &quot;Mono&quot;, &#xA;&quot;Mono&quot;, &quot;Mono&quot;, &quot;Mono&quot;, &quot;Mono&quot;, &quot;LSC&quot;, &quot;LSC&quot;, &quot;LSC&quot;, &quot;LSC&quot;, &quot;LSC&quot;, &#xA;&quot;LSC&quot;, &quot;LSC&quot;, &quot;LSC&quot;, &quot;Blast&quot;, &quot;Blast&quot;, &quot;Blast&quot;, &quot;Blast&quot;, &quot;Blast&quot;, &#xA;&quot;Blast&quot;, &quot;Blast&quot;, &quot;Blast&quot;, &quot;Blast&quot;, &quot;Blast&quot;, &quot;Blast&quot;, &quot;Blast&quot;, &#xA;&quot;Blast&quot;, &quot;Blast&quot;, &quot;Blast&quot;, &quot;Blast&quot;, &quot;Blast&quot;, &quot;Blast&quot;, &quot;Blast&quot;, &#xA;&quot;Blast&quot;, &quot;Blast&quot;, &quot;Blast&quot;, &quot;Blast&quot;, &quot;pHSC&quot;, &quot;pHSC&quot;, &quot;pHSC&quot;, &quot;pHSC&quot;, &#xA;&quot;pHSC&quot;, &quot;pHSC&quot;, &quot;pHSC&quot;, &quot;pHSC&quot;, &quot;pHSC&quot;, &quot;pHSC&quot;, &quot;pHSC&quot;, &quot;pHSC&quot;, &#xA;&quot;pHSC&quot;, &quot;pHSC&quot;, &quot;pHSC&quot;, &quot;pHSC&quot;, &quot;pHSC&quot;, &quot;pHSC&quot;, &quot;pHSC&quot;)), class = c(&quot;spec_tbl_df&quot;, &#xA;&quot;tbl_df&quot;, &quot;tbl&quot;, &quot;data.frame&quot;), row.names = c(NA, -78L), spec = structure(list(&#xA;    cols = list(Run = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), `Assay Type` = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), AvgSpotLen = structure(list(), class = c(&quot;collector_double&quot;, &#xA;    &quot;collector&quot;)), BioProject = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), BioSample = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), Cell_type = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), `Center Name` = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), Consent = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), `DATASTORE filetype` = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), `DATASTORE provider` = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), `DATASTORE region` = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), donorid = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), Experiment = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), GEO_Accession = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), Instrument = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), LibraryLayout = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), LibrarySelection = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), LibrarySource = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), MBases = structure(list(), class = c(&quot;collector_double&quot;, &#xA;    &quot;collector&quot;)), MBytes = structure(list(), class = c(&quot;collector_double&quot;, &#xA;    &quot;collector&quot;)), Organism = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), Platform = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), ReleaseDate = structure(list(format = &quot;&quot;), class = c(&quot;collector_datetime&quot;, &#xA;    &quot;collector&quot;)), sample_acc = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), `Sample Name` = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), sample_type = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), source_name = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;)), `SRA Study` = structure(list(), class = c(&quot;collector_character&quot;, &#xA;    &quot;collector&quot;))), default = structure(list(), class = c(&quot;collector_guess&quot;, &#xA;    &quot;collector&quot;)), skip = 1), class = &quot;col_spec&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to have the below output to be in my final dataframe which i want to use for renaming the original file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; Run        source_name&#xA;   &amp;lt;chr&amp;gt;      &amp;lt;chr&amp;gt;      &#xA; 1 SRR2920466 HSC1        &#xA; 2 SRR2920477 HSC2       &#xA; 3 SRR2920478 HSC3        &#xA; 4 SRR2920506 HSC4        &#xA; 5 SRR2920507 HSC5        &#xA; 6 SRR2920532 HSC6        &#xA; 7 SRR2920531 HSC7        &#xA; 8 SRR2920469 CMP1        &#xA; 9 SRR2920470 CMP2        &#xA;10 SRR2920481 CMP3    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Found a solution but I have to apply it to each string in the last column&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;i1 &amp;lt;- !sapply(df1, is.numeric)&#xA;df1[i1] &amp;lt;- lapply(df1[i1], function(x) factor(replace(as.character(x),&#xA;                                                      x == &quot;HSC&quot;, paste0(x[x==&quot;HSC&quot;], seq_len(sum(x == &quot;HSC&quot;))))))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Right now I am able to match repeated string in the column and add increment.How to apply it for all the other string .&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2020-02-03T06:44:51.120" LastActivityDate="2020-02-08T12:34:45.967" Title="How to add an increment to a repeated strings" Tags="|r|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11293" PostTypeId="2" ParentId="11292" CreationDate="2020-02-03T07:07:09.150" Score="1" Body="&lt;p&gt;EDIT: Wheres the OP accepted this answer, @zx8754's answer below, as well as the ones in the link they gave, are much more elegant and straightforward.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Here is a work-around with &lt;code&gt;table()&lt;/code&gt;, &lt;code&gt;lapply()&lt;/code&gt; and &lt;code&gt;seq_len()&lt;/code&gt; (I have basically used your &lt;code&gt;lapply()&lt;/code&gt; call on the output of a &lt;code&gt;table()&lt;/code&gt; call):    &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;my_counts &amp;lt;- table(df1$source_name)&#xA;&#xA;&amp;gt; my_counts&#xA;&#xA;Blast   CMP   GMP   HSC   LSC  Mono  pHSC &#xA;   23     8     7     7     8     6    19 &#xA;&#xA;df1 &amp;lt;- df1[order(df1$source_name),]&#xA;&#xA;&amp;gt; head(df1)&#xA;          Run source_name&#xA;37 SRR2920546       Blast&#xA;38 SRR2920547       Blast&#xA;39 SRR2920548       Blast&#xA;40 SRR2920549       Blast&#xA;41 SRR2920550       Blast&#xA;42 SRR2920551       Blast&#xA;&#xA;&amp;gt; names(unlist(lapply(my_counts, seq_len)))&#xA; [1] &quot;Blast1&quot;  &quot;Blast2&quot;  &quot;Blast3&quot;  &quot;Blast4&quot;  &quot;Blast5&quot;  &quot;Blast6&quot;  &quot;Blast7&quot;  &quot;Blast8&quot;  &quot;Blast9&quot;  &quot;Blast10&quot; &quot;Blast11&quot; &quot;Blast12&quot; &quot;Blast13&quot;&#xA;[14] &quot;Blast14&quot; &quot;Blast15&quot; &quot;Blast16&quot; &quot;Blast17&quot; &quot;Blast18&quot; &quot;Blast19&quot; &quot;Blast20&quot; &quot;Blast21&quot; &quot;Blast22&quot; &quot;Blast23&quot; &quot;CMP1&quot;    &quot;CMP2&quot;    &quot;CMP3&quot;   &#xA;[27] &quot;CMP4&quot;    &quot;CMP5&quot;    &quot;CMP6&quot;    &quot;CMP7&quot;    &quot;CMP8&quot;    &quot;GMP1&quot;    &quot;GMP2&quot;    &quot;GMP3&quot;    &quot;GMP4&quot;    &quot;GMP5&quot;    &quot;GMP6&quot;    &quot;GMP7&quot;    &quot;HSC1&quot;   &#xA;[40] &quot;HSC2&quot;    &quot;HSC3&quot;    &quot;HSC4&quot;    &quot;HSC5&quot;    &quot;HSC6&quot;    &quot;HSC7&quot;    &quot;LSC1&quot;    &quot;LSC2&quot;    &quot;LSC3&quot;    &quot;LSC4&quot;    &quot;LSC5&quot;    &quot;LSC6&quot;    &quot;LSC7&quot;   &#xA;[53] &quot;LSC8&quot;    &quot;Mono1&quot;   &quot;Mono2&quot;   &quot;Mono3&quot;   &quot;Mono4&quot;   &quot;Mono5&quot;   &quot;Mono6&quot;   &quot;pHSC1&quot;   &quot;pHSC2&quot;   &quot;pHSC3&quot;   &quot;pHSC4&quot;   &quot;pHSC5&quot;   &quot;pHSC6&quot;  &#xA;[66] &quot;pHSC7&quot;   &quot;pHSC8&quot;   &quot;pHSC9&quot;   &quot;pHSC10&quot;  &quot;pHSC11&quot;  &quot;pHSC12&quot;  &quot;pHSC13&quot;  &quot;pHSC14&quot;  &quot;pHSC15&quot;  &quot;pHSC16&quot;  &quot;pHSC17&quot;  &quot;pHSC18&quot;  &quot;pHSC19&quot;&#xA;&#xA;df1$new_col &amp;lt;- names(unlist(lapply(my_counts, seq_len)))&#xA;&#xA;&amp;gt; head(df1)&#xA;          Run source_name new_col&#xA;37 SRR2920546       Blast  Blast1&#xA;38 SRR2920547       Blast  Blast2&#xA;39 SRR2920548       Blast  Blast3&#xA;40 SRR2920549       Blast  Blast4&#xA;41 SRR2920550       Blast  Blast5&#xA;42 SRR2920551       Blast  Blast6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can store the initial row order somewhere to have resulting data frame in the same order the initial one.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2020-02-08T12:34:45.967" LastActivityDate="2020-02-08T12:34:45.967" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11294" PostTypeId="2" ParentId="11292" CreationDate="2020-02-03T07:41:22.420" Score="3" Body="&lt;p&gt;We need to group by &lt;em&gt;source_name&lt;/em&gt; and add row id, here is using &lt;em&gt;data.table&lt;/em&gt; package:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(data.table)&#xA;&#xA;# convert to data.table&#xA;setDT(df1)&#xA;&#xA;# group by source_name, and add rowid&#xA;df1[, myNewColumn := paste0(source_name, rowid(source_name))]&#xA;&#xA;df1&#xA;#            Run source_name myNewColumn&#xA;#  1: SRR2920466         HSC        HSC1&#xA;#  2: SRR2920477         HSC        HSC2&#xA;#  3: SRR2920478         HSC        HSC3&#xA;#  4: SRR2920506         HSC        HSC4&#xA;#  5: SRR2920507         HSC        HSC5&#xA;#  6: SRR2920532         HSC        HSC6&#xA;#  7: SRR2920531         HSC        HSC7&#xA;#  8: SRR2920469         CMP        CMP1&#xA;#  9: SRR2920470         CMP        CMP2&#xA;# 10: SRR2920481         CMP        CMP3&#xA;# ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://stackoverflow.com/q/12925063/680068&quot;&gt;here&lt;/a&gt; for more solutions.&lt;/p&gt;&#xA;" OwnerUserId="131" LastActivityDate="2020-02-03T07:41:22.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11295" PostTypeId="2" ParentId="11284" CreationDate="2020-02-03T09:40:40.880" Score="1" Body="&lt;h2&gt;Open babel&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The most commonly used tool is &lt;code&gt;open-babel&lt;/code&gt; (obabel). It seems annoying that you have to specify the number of conformers, but this is because if you get something like retinol, the number of conformers is huge. The command from the &lt;a href=&quot;https://open-babel.readthedocs.io/en/latest/3DStructureGen/multipleconformers.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;obabel startingConformer.mol -O ga_conformers.sdf --conformer --nconf 30&#xA;   --score rmsd --writeconformers&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;An alternative argument is &lt;code&gt;--confab&lt;/code&gt; which will pick each representative of a RMSD cutoff cluster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do note that 2D representations and un-protonated molecules will need extra command line arguments.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;RDKit&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;You can also generate conformers in RDkit, but I think this is beyond your requirements, but I thought it may be good to mention the essential compchem python library.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-02-03T09:40:40.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11296" PostTypeId="2" ParentId="11253" CreationDate="2020-02-03T11:11:19.227" Score="0" Body="&lt;p&gt;If I understand your question correctly and you have &lt;code&gt;dplyr&lt;/code&gt; and &lt;code&gt;ggplot&lt;/code&gt; installed, you could try something like (if a is your data.frame)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a %&amp;gt;% &#xA;dplyr::group_by(orig.ident, old.ident) %&amp;gt;% &#xA;dplyr::summarise(count  = n()) %&amp;gt;% &#xA;ggplot(aes(x=orig.ident, y=count)) + geom_bar(stat='identity', aes(group=old.ident), position='dodge')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2020-02-03T11:11:19.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11297" PostTypeId="1" AcceptedAnswerId="11298" CreationDate="2020-02-03T16:00:28.557" Score="4" ViewCount="98" Body="&lt;p&gt;In the fastq format every 4k+4-th line contains the positionwise qualityscore (ascii encoded):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@SEQ_ID&#xA;GATTTGGGGTTCAAAGCAGTATCGATCAAATAGTAAATCCATTTGTTCAACTCACAGTTT&#xA;+&#xA;!''*((((***+))%%%++)(%%%%).1***-+*''))**55CCF&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;CCCCCCC65 &#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;which program uses this information besides trimming tools (cutadapt) or quality control (fastqc)?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I heard that segemehl (mapping) abandons this quality string in favor of running time (maybe other mapping tools too?)&#xA;But what about abyss, trinity, ...? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So i atrifically generated a fastq file containing 2 identical reads that perfectly match a sequence in some genome (C.fna). &#xA;The two reads have the most extreme phred scores (only '!!'=highest sanger phread and only 'II'=lowest sanger phred)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now using segemehl the mapping quality is identical (60)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@HD VN:1.0&#xA;@SQ SN:C_fna    LN:15440&#xA;@RG ID:A1   SM:sample1  LB:library1 PU:unit1    PL:illumina&#xA;@PG ID:segemehl VN:0.3.4    CL:segemehl -i C2.fna.idx -d C2.fna -q test.fq&#xA;A   0   C_fna   261 60  220=    *   0   0   ACAGGAAACACAGAAAAAAGCCCGCACCTGACAGTGCGGGCTTTTTTTTTCGACCAAAGGTAACGAGGTAACAACCATGCGAGTGTTGAAGTTCGGCGGTACATCAGTGGCAAATGCAGAACGTTTTCTGCGTGTTGCCGATATTCTGGAAAGCAATGCCAGGCAGGGGCAGGTGGCCACCGTCCTCTCTGCCCCCGCCAAAATCACCAACCACCTGGTG    !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!    HI:i:0  NH:i:1  NM:i:0  MD:Z:220    RG:Z:A1 YZ:Z:0&#xA;B   0   C_fna   261 60  220=    *   0   0   ACAGGAAACACAGAAAAAAGCCCGCACCTGACAGTGCGGGCTTTTTTTTTCGACCAAAGGTAACGAGGTAACAACCATGCGAGTGTTGAAGTTCGGCGGTACATCAGTGGCAAATGCAGAACGTTTTCTGCGTGTTGCCGATATTCTGGAAAGCAATGCCAGGCAGGGGCAGGTGGCCACCGTCCTCTCTGCCCCCGCCAAAATCACCAACCACCTGGTG    IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII    HI:i:0  NH:i:1  NM:i:0  MD:Z:220    RG:Z:A1 YZ:Z:0&#xA;[SEGEMEHL] Thu Feb  6 08:35:15 2020: matching w/ suffixarray has taken 0.000000 seconds.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So segemehl does NOT use the read quality phred (basically only using the read sequence for mapping)&lt;/p&gt;&#xA;" OwnerUserId="2766" LastEditorUserId="2766" LastEditDate="2020-02-06T07:48:17.467" LastActivityDate="2020-02-10T16:04:46.210" Title="Is the quality score of fastq used somewhere besides trimming/fastqc?" Tags="|rna-seq|fastq|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11298" PostTypeId="2" ParentId="11297" CreationDate="2020-02-03T16:20:38.773" Score="4" Body="&lt;p&gt;I think almost all alignment programs would use base quality as a factor to calculate mapping quality (See &lt;a href=&quot;http://maq.sourceforge.net/qual.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). These would also be used in Variant Calling to pick reads where high quality bases confidently align as either the reference base or one of the alternate bases.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2020-02-03T16:20:38.773" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11299" PostTypeId="1" CreationDate="2020-02-03T17:50:13.917" Score="3" ViewCount="397" Body="&lt;p&gt;I am analysing a human single cell RNA seq experiment, where we have 4 groups, four samples each. Data has been analysed using Seurat, with the canonical workflow. I have tried DE using various methods (Wilcox, MAST, LR), and for each of them the top genes (biggest fold change, lowest adjusted p) are genes over expressed by specific samples, e.g.:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/PCyTH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PCyTH.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at the average expression for these genes per each subject, and plotting on a Boxplot, it does not seem to be a genuine difference (e.g. for the gene above):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/l0041.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/l0041.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Besides the subject, there is no other parameter (e.g. age, sex) that is biased across groups and that shows up as differentially represented in the PCA plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a DE method that takes into account this behaviour and corrects for it?&#xA;How can these results be presented? &#xA;Besides increasing number of subjects (not really feasible), is there a solution to analyse these data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="7113" LastEditorUserId="7113" LastEditDate="2020-02-04T10:41:55.230" LastActivityDate="2020-05-11T11:42:01.077" Title="Differential gene expression bias due to effect of an individual sample" Tags="|rna-seq|scrnaseq|seurat|differential-expression|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11300" PostTypeId="2" ParentId="11281" CreationDate="2020-02-03T21:44:12.630" Score="1" Body="&lt;p&gt;You don't need &lt;code&gt;g2&lt;/code&gt; to go from &lt;code&gt;0&lt;/code&gt; to &lt;code&gt;din.index&lt;/code&gt;, just from &lt;code&gt;0&lt;/code&gt; to &lt;code&gt;g1 - 1&lt;/code&gt;. This way, you'll end up calculating just for the &quot;lower triangle&quot;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;g1-&amp;gt; 0   1   2   3&#xA;g2&#xA;0                    #(0,0 to 0,1-1 = nothing)&#xA;1    X               #(1,0 to 1,1-1 = 1,0)&#xA;2    X   X           #(2,0 to 2,2-1 = 2,0; 2,1)&#xA;3    X   X   X       #(3,0 to 3,3-1 = 3,0; 3,1 and 3,2)&#xA;4    X   X   X   X   #(4,0 to 4,4-1 = 4,0; 4,1; 4,2 and 4,3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastActivityDate="2020-02-03T21:44:12.630" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11301" PostTypeId="1" AcceptedAnswerId="11306" CreationDate="2020-02-04T01:54:06.940" Score="19" ViewCount="1726" Body="&lt;p&gt;I have heard several conspiracy theories regarding the origin of the new coronavirus, 2019-nCov. For example that the virus and/or SARS were produced in a laboratory or were some variant of Middle Eastern respiratory syndrome (MERS), shipped via laboratory workers. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am well aware bioinformatics has debunked many conspiracy theories involving infectious diseases, an important one being polio vaccination programs in Africa were the origin of HIV, for example &lt;a href=&quot;https://www.historyofvaccines.org/content/articles/debunked-polio-vaccine-and-hiv-link&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. Likewise, numerous conspiracy theories on the man-made origins of HIV were similarly overturned using bioinformatic studies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have bioinformatics evidence that would debunk the current conspiracists about coronaviruses?&lt;/p&gt;&#xA;" OwnerUserId="7122" LastEditorUserId="298" LastEditDate="2022-03-16T11:03:52.480" LastActivityDate="2022-03-16T11:03:52.480" Title="Is it possible for coronavirus or SARS to be synthetic?" Tags="|phylogenetics|phylogeny|ngs|covid-19|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11302" PostTypeId="2" ParentId="11277" CreationDate="2020-02-04T05:46:36.093" Score="6" Body="&lt;p&gt;R supports logistic regression, which would seem to be the most efficient method for tackling this question. Assuming the &quot;Chemo&quot; variable is the type of chemo the  code would be something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;glm( (response_to_chemo == &quot;yes&quot;) ~  BMI + Chemo  + Predictor +  DJANGO + gender, &#xA;        family=&quot;binomial&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;EDIT: Corrected a typo (added a missing double quote)&lt;/p&gt;&#xA;" OwnerUserId="6668" LastEditorUserId="650" LastEditDate="2020-02-10T17:58:50.190" LastActivityDate="2020-02-10T17:58:50.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11303" PostTypeId="1" CreationDate="2020-02-04T14:37:00.590" Score="0" ViewCount="419" Body="&lt;p&gt;In order to obtain orthologs for human genes, I am working with &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2808972/&quot; rel=&quot;nofollow noreferrer&quot;&gt;InParanoid version 7.0&lt;/a&gt;. This version of the database uses Ensembl Protein Identifiers (&lt;strong&gt;ENSP&lt;/strong&gt;) from ENSEMBL version v54, which is based on &lt;strong&gt;NCBI 36&lt;/strong&gt; assembly of the human genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For my project, I need to have the Ensembl Gene Identifiers (&lt;strong&gt;ENSG&lt;/strong&gt;) of these proteins, but in the last version of ENSEMBL, which is based on &lt;strong&gt;GRCh38&lt;/strong&gt; assembly. I need to use this version of InParanoid for consistency with previous analyzes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus, my question is, &lt;strong&gt;could I just use biomaRt to map ENSP (from NCBI36) to ENSG (actual version)&lt;/strong&gt; or this approach is inherently wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to &quot;convert&quot; ENSP identifiers from NCBI36 to GRCh38 and then do the map to ENSG?. I am aware of &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgLiftOver&quot; rel=&quot;nofollow noreferrer&quot;&gt;liftOver&lt;/a&gt; but I am not sure it would work here.&lt;/p&gt;&#xA;" OwnerUserId="678" LastActivityDate="2020-02-04T15:27:04.863" Title="Ensembl protein identifiers from different assemblies" Tags="|sequence-annotation|ensembl|liftover|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11304" PostTypeId="2" ParentId="11303" CreationDate="2020-02-04T14:43:08.397" Score="4" Body="&lt;p&gt;Those IDs are elderly! Ensembl 54 was 2009!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would recommend using BioMart combined with the &lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Tools/IDMapper&quot; rel=&quot;nofollow noreferrer&quot;&gt;ID history converter&lt;/a&gt;. The ID history converter will convert old IDs to new, and BioMart will convert ENSPs to ENSGs. You can either use the ID history converter with the ENSPs first, then the &lt;a href=&quot;http://www.ensembl.org/biomart/martview&quot; rel=&quot;nofollow noreferrer&quot;&gt;current BioMart&lt;/a&gt; to get the ENSGs. Or you can use &lt;a href=&quot;http://may2009.archive.ensembl.org/biomart/martview&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl 54 BioMart&lt;/a&gt; to convert the old ENSPs to old ENSGs, then update the ENSGs with the ID history converter. I would probably do both and compare results.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastEditorUserId="1332" LastEditDate="2020-02-04T15:27:04.863" LastActivityDate="2020-02-04T15:27:04.863" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11305" PostTypeId="2" ParentId="11287" CreationDate="2020-02-04T16:25:06.760" Score="2" Body="&lt;p&gt;If this is your output :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    12595   12802   ref|DDX11L1,ref|NR_046018,ens|ENST00000518655,ens|ENST00000450305,ens|ENST00000456328,ens|ENST00000515242  .       ref|DDX11L1     0       0       207     0.0000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I guess your bedfile looks like this (the first 6 fields) :&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;chrom : chr1&lt;/li&gt;&#xA;&lt;li&gt;FeatureStart : 12595&lt;/li&gt;&#xA;&lt;li&gt;FeatureEnd : 12802&lt;/li&gt;&#xA;&lt;li&gt;name : ref|DDX11L1,ref|NR_046018,ens|ENST00000518655,ens|ENST00000450305,ens|ENST00000456328,ens|ENST00000515242&lt;/li&gt;&#xA;&lt;li&gt;score : .&lt;/li&gt;&#xA;&lt;li&gt;strand : ref|DDX11L1&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The issue I see here is that you are giving the wrong information to the strand field of your bedfile. It should be either &lt;code&gt;+&lt;/code&gt;, either &lt;code&gt;-&lt;/code&gt; or &lt;code&gt;.&lt;/code&gt; if you don't want to consider the strand information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope it helps !&lt;/p&gt;&#xA;" OwnerUserId="6273" LastActivityDate="2020-02-04T16:25:06.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11306" PostTypeId="2" ParentId="11301" CreationDate="2020-02-04T16:30:11.957" Score="24" Body="&lt;p&gt;The scenarios are impossible and would be laughable if they were not so serious. The evidence is in the phylogenetic trees. Its a bit like a crime scene when the forensics team investigate. We've done enough crime-scenes often going to the site, collecting the pathogen, sequencing and then analysis - (usually neglected diseases) without any associated conspiracy theories. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key technical issue is coronaviruses are &lt;strong&gt;zoonoses&lt;/strong&gt; , pathogens spread to humans from animal reservoirs and phylogenetic tree really helps understand the  how the virus is transmitted.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Trees&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The key thing about all the trees are &lt;strong&gt;bats&lt;/strong&gt;. Bats lineages are present at every single point of the betacoronavirus phylogeny (tree), both as paraphyletic and monophyletic lineages, one example is this tree of betacoronaviruses &lt;a href=&quot;https://www.ecohealthalliance.org/2020/01/phylogenetic-analysis-shows-novel-wuhan-coronavirus-clusters-with-sars&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. Meaning the nodes connecting the branches of the tree to the &quot;master-branch&quot;, represent common ancestors and these were almost certainly bat-borne cornaviruses. This is especially true for SARS and - &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/11251/interpretion-of-my-coronavirus-2019-ncov-wuhan-china-blast-tree&quot;&gt;here&lt;/a&gt; bat-viruses are EVERYWHERE. &lt;/li&gt;&#xA;&lt;li&gt;The tree &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/11251/interpretion-of-my-coronavirus-2019-ncov-wuhan-china-blast-tree&quot;&gt;here&lt;/a&gt; also shows that SARS arose on independently on two occassions, again surrounded by bat lineages and 2019-nCov has emerged separately at least once, again associated with bats. &lt;/li&gt;&#xA;&lt;li&gt;Finally the tree below is a figure from BioRxiv Zhou et al (2020) &quot;Discovery of a novel coronavirus associated with the recent pneumonia outbreak in humans and its potential bat origin&quot; shows the 2019-nCov lineage is a direct descendent of a very closely virus isolated from a bat (RaTG13*). This is a really conclusive finding BTW.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/55HtA.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/55HtA.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;, I don't normally present inline images, but it is such a nice finding (hint to reviewers) and BioRxiv is open access.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conspiracy theory 1&lt;/strong&gt;: &lt;em&gt;laboratory made virus&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Literally it would require someone passaging a new virus, with unknown human pathogenicity, and independently introducing all the earlier passages enmass across bat populations of China. They would then hope each lineage becomes an indepedent virus population before, then introducing the virus to humans. Thus when field teams of scientists go around using mist nets to trap the bats, buy them from markets, isolate the virus and sequence it they would find a beautiful, array of natural variation in the bat populations leading up to the human epidemics, that perfectly matches vast numbers of other viral zoonoses. Moreover, this would have to have happen substancially prior to SARS and 2019-nCov, because the bat betacoronaviruses have been known about prior both epidemics, viz. its simply not feasible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Biological explanation&lt;/strong&gt;&#xA;&lt;em&gt;General&lt;/em&gt; Bats are a reservoir host to vast numbers of pathogens, particularly viruses, including many alphaviruses, flaviviruses, rabies virus and beleived to be important in ebolavirus (I don't know about this) and even important to several eukaryotic parasites. It makes sense, they are mammals, so evolutionary much closer to us than birds for example, with large dispersal potential and roost in 'overcrowded' areas enable rapid transmission between bats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Technical&lt;/em&gt;&#xA;The trees show bats are the common ancestor of betacoronaviruses in particular for the lineage leading into the emergence of 2019-nCov and SARS, this is seen in this &lt;a href=&quot;https://www.ecohealthalliance.org/2020/01/phylogenetic-analysis-shows-novel-wuhan-coronavirus-clusters-with-sars&quot; rel=&quot;noreferrer&quot;&gt;tree&lt;/a&gt;, this &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/11251/interpretion-of-my-coronavirus-2019-ncov-wuhan-china-blast-tree&quot;&gt;one&lt;/a&gt; and the tree above. The obvious explanation is the virus circulates endemically in bats and has jumped into humans. For SARS the intermediate host, or possible &quot;vector&quot; was civet cats. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The theory and the observations fit into a seamless biological answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conspiracy theory 2&lt;/strong&gt;: &lt;em&gt;Middle Eastern connection&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I heard a very weird conspiracy theory attempting to connect MERS with 2019-nCov. The theory was elaborate and I don't think it is productive to describe here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Biological explanation&lt;/strong&gt;&#xA;All the trees of betacoronaviruses show MERS was one of the earliest viruses to diverge and is very distant from 2019-nCov, to the extent the theory is completely implausible. The homology between these viruses is 50%, so its either MERS or 2019-nCov. Its more extreme than mixing up yellow fever virus (mortality 40-80%) with West Nile virus (mortality &amp;lt;&amp;lt;0.1%), the two viruses are completely different at every level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What about errors?&lt;/strong&gt; Phylogeneticists can spot it a mile off. There are tell-tale phylogenetic signatures we pick up, but also we do this to assess 'rare' genetic phenomina. There is nothing 'rare' about the coronaviruses. The only anomaly is variation in the poly-A tail and that is the natural variation from in vitro time-series experiments. Basically we've looked at enough virses/parasites through trees, that have no conspiracy theories at all (often neglected diseases), and understand how natural variation operates - so a phylogenecist can shift the wheat from the chaff without really thinking about it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Opinion&lt;/strong&gt;&#xA;The conspiracy theories are deeply misplaced, and the only connection I can imagine is its China. However, the Chinese have loads of viruses, influenza in particular which causes major pandemics, but that is a consequence of their natural ecology (small-holder farming) allowing the virus to move between reservoir hosts. I've not visited small-hold farms in China, but I have in other parts of the world and when you see them, you get it. The pigs, chickens (ducks .. China), dogs, horses and humans all living within 10 meters of each other. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&#xA;Shipping large numbers of bats to market, bat soup, raw meat from arboreal (tree-living) mammals such as civets that are sympatric to bats. Then consider the classical epidemiology in light of the phylogenetic data, which is very consistent, a single picture emerges that coronavirus is one of many zoonoses which has managed to transmit between patients. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; The fundamental point is the bioinformatics fit into the classical epidemiology of a zoonose.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*, &lt;strong&gt;Note&lt;/strong&gt; The bat coronavirus RaTG13 predates the 2019-nCov outbreak by 7 years. It is not even clear whether the virus has been isolated, i.e. could just be a RNA sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;They have found some 500 novel coronaviruses, about 50 of which fall relatively close to the SARS virus on the family tree, including RaTG13—it was fished out of a bat fecal sample they collected in 2013 from a cave in Moglang in Yunnan province.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cohen, &quot;Mining coronavirus genomes for clues to the outbreak’s origins&quot; Feb, 2020 Science Magazine,&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-02-04T21:02:48.577" LastActivityDate="2020-02-04T21:02:48.577" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11307" PostTypeId="1" CreationDate="2020-02-04T16:34:08.517" Score="0" ViewCount="152" Body="&lt;p&gt;I'm trying to show cell cycle regression and input sample metadata to my umap. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've been following Satija Lab's tutorials and have generated my umap but now want to specify which sample corresponds to each cluster, if one sample is driving a certain cluster, etc. I also, would like yo show cell cycle regression in each cluster. I following the cell cycle regression tutorial but that only tells me the board overview of the cells in each cycle. I would want to highlight it in the umap. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would I go about each of these tasks?&lt;/p&gt;&#xA;" OwnerUserId="6827" LastActivityDate="2021-02-28T22:06:55.697" Title="Seurat clusters" Tags="|scrnaseq|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11308" PostTypeId="1" AcceptedAnswerId="11311" CreationDate="2020-02-04T16:40:40.257" Score="2" ViewCount="149" Body="&lt;p&gt;I wrote some throwaway code for ingesting paired-end reads from a Fastq file into memory.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;def ingest(stream1, stream2):&#xA;    while True:&#xA;        record1, record2 = list(), list()&#xA;        for _ in range(4):&#xA;            record1.append(next(stream1))&#xA;            record2.append(next(stream2))&#xA;        yield record1, record2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I had initially put the &lt;code&gt;for&lt;/code&gt; loop in a &lt;code&gt;try/Except&lt;/code&gt; block, anticipating I'd need to explicitly break out of the endless &lt;code&gt;while&lt;/code&gt; loop. However, the code seems to work correctly even after I removed it. I even &quot;corrupted&quot; the final read in one of the paired files, and the &lt;code&gt;ingest&lt;/code&gt; function performs reasonably (the last read pair is never &lt;code&gt;yield&lt;/code&gt;ed).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is...&lt;strong&gt;why does this code work&lt;/strong&gt;, since I never explicitly break out of the loop?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example of how the code is invoked.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; with open('seqR1.fastq', 'r') as fh1, open('seqR2.fastq', 'r') as fh2:&#xA;...     counter = 0&#xA;...     for r1, r2 in interleave(fh1, fh2):&#xA;...         counter += 1&#xA;&amp;gt;&amp;gt;&amp;gt; counter&#xA;5000&#xA;&amp;gt;&amp;gt;&amp;gt; r1&#xA;['@read5000/1\n', 'AGGTCTGGGGACAGGTGTTTTACAGGCTCCTTTAGGGGTGGCAAGTCTCCCGGAAACATGATTAGTGAAGACACGTTAACAGCGAAACTGACAGAGAACCTCAACATTCTCTGGTAGGAAAAGCCAGGATCCTGGCAGAGGAAGCAACGATTCGAGCAGGGAACTGTGCAGGACAGAGAACAATGCGGCCCCAATTTTGATCCAGGGGTAGTTAGCTTAGCTAAGGAGTTCCCCAGTCCCTAAGTGGAGAGGGGATCCAGAACCCTACAATTTATTATTATATCTTAGCGGATATATCTAT\n', '+\n', 'CCCCCGGGEGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG@GGGGGGGFAGGGCGGGGGFGGGGGGGGGCGGGGGDGGGGFGGGGGGGGGCGGGGGGGGGGGGGGGCGGGGGGGGGGGGGGGGG@G&amp;lt;GGGGGGGGGGGGFGGFGFF:DGGFFCGGGGCGGFEGGGGGGEGGGFB:GGGDG=FGGGG,GAGGGDGGFGGGCD3GFG+G?G*FGGG&amp;lt;GCGGGGC@CD&amp;lt;FFGF*:+G=FE&amp;lt;;GG,8C7GF=;9D=G&amp;lt;C5C66G6C&amp;gt;7)FD*6G7D)9;*C:)4D5+9&amp;gt;@G*=0):)*;1*504\n']&#xA;&amp;gt;&amp;gt;&amp;gt; r2&#xA;['@read5000/2\n', 'TGTGGGGTTCTGGATCCCCTCTCCACTTAGGGACTGGGGAACTCCTTAGCTAAGCTAATTACCCCTGGATCAAAATTGGGGCCGCATTGTTCTCTGTCCTGCACAGTTCCCTGCTCGAATCGTTGCTTCCTCTGCCAGGATCCTGGCTTTTCCTACCAGAGAATGTTGAGGTTCTCTGTCATTTTCGCTGCTAATGTGTCTTCACTAATCGTGTTTCCGGGAGACTTGCCACCCCTAAAGGAGCTTGTTGAACACTTGTCACCAGACCTAAGTCCTTAGGGATGAACCTGTGTCTCGGCAC\n', '+\n', 'CCCCBGC9FGCD-GGGFGGGEFEFF@CCGFGGGGFF+DG6,F69+FGG+FGF&amp;lt;FCC;GGEBFGGEDECFGGG6GDG7BFG9GCG9G9GDGGGG&amp;lt;GDFFFGEGC,&amp;lt;GC=GFG:,:A:EE+GD&amp;lt;:E,FGF=FCAGEFF++G,G,&amp;lt;EGGFGFFA&amp;gt;*FG9EGFCFCGC,B&amp;gt;43GFF@E@2&amp;gt;9ED@,568D=A;C+=5@E/F53D&amp;gt;,6F*9@+,E,C=*0+*@56&amp;lt;*9@:/F,1,:*)*14*0CC=*1)*0*,*)0;+)*))/**)**;0;//9)1+)10.()5/)070*)7*C**3/)(2*****\n']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="96" LastActivityDate="2020-02-05T15:46:30.730" Title="Why does this naive Python paired Fastq reader work?" Tags="|python|fastq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11309" PostTypeId="2" ParentId="11307" CreationDate="2020-02-04T19:30:58.680" Score="1" Body="&lt;p&gt;You can use the &lt;code&gt;group.by&lt;/code&gt; argument of the &lt;code&gt;DimPlot()&lt;/code&gt; function. &lt;code&gt;group.by&lt;/code&gt; accepts column names present in the &lt;code&gt;meta.data&lt;/code&gt; column of the Seurat objects.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-02-04T19:30:58.680" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11310" PostTypeId="2" ParentId="11308" CreationDate="2020-02-04T21:42:24.807" Score="2" Body="&lt;p&gt;EDIT: &lt;strong&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/a/11311/4980&quot;&gt;Chris_Rands's answer&lt;/a&gt; is more authoritative and clarifies that this behavior is essentially a bug.&lt;/strong&gt;  (I had only tested my example on Python 3.6.8 and hadn't seen PEP 479.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When &lt;code&gt;next&lt;/code&gt; runs out of lines, it raises &lt;code&gt;StopIteration&lt;/code&gt;, and apparently that stops the while loop similarly to calling &lt;code&gt;break&lt;/code&gt; without you needing to catch the exception.  I couldn't see a definitive source on that but you can see it in action if you catch and re-raise the exception:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def ingest2(stream1, stream2):&#xA;    while True:&#xA;        try:&#xA;            record1, record2 = list(), list()&#xA;            for _ in range(4):&#xA;                record1.append(next(stream1))&#xA;                record2.append(next(stream2))&#xA;            yield record1, record2&#xA;        except StopIteration:&#xA;            print(&quot;StopIteration raised&quot;)&#xA;            raise&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can also try to catch it outside the loop, but it doesn't get re-raised at that point.  (If you try raising one yourself in a while loop, though, it &lt;em&gt;doesn't&lt;/em&gt; catch it in the same way, so there must be some extra Python magic going on.  I can't see any difference between the two exception objects so I'd be curious what's going on there.)  Neat example.  You could also make a pair of FASTQ parsers using Biopython's &lt;a href=&quot;https://biopython.org/wiki/SeqIO#sequence-input&quot; rel=&quot;nofollow noreferrer&quot;&gt;SeqIO.parse&lt;/a&gt; for more complex/permanent situations if needed.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastEditorUserId="4980" LastEditDate="2020-02-05T15:46:30.730" LastActivityDate="2020-02-05T15:46:30.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11311" PostTypeId="2" ParentId="11308" CreationDate="2020-02-04T21:54:48.543" Score="4" Body="&lt;p&gt;Your code only runs prior to Python 3.7, now it raises &lt;code&gt;RuntimeError: generator raised StopIteration&lt;/code&gt;. This is essentially a bug fix described in &lt;a href=&quot;https://www.python.org/dev/peps/pep-0479/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PEP 479 -- Change StopIteration handling inside generators&lt;/a&gt;. You can enable the new behaviour from Python 3.5 via &lt;code&gt;from __future__ import generator_stop&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note, it's not related to file-handles per se, you can reproduce it with a smaller example with a simple iterator:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;it = iter([1,2])&#xA;&#xA;def f(it):&#xA;    while True:&#xA;        yield next(it)&#xA;&#xA;print(list(f(it)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You should not write code relying on the bug&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2020-02-04T21:54:48.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11313" PostTypeId="1" AcceptedAnswerId="11314" CreationDate="2020-02-05T01:48:39.167" Score="13" ViewCount="2824" Body="&lt;p&gt;My colleague and I have developed a software tool intend to release it open-source. This tool is specifically for tasks in bioinformatics but we think it would be helpful for the wider community. Our institution will permit us to release it provided we get appropriate credit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus we wish to publish it in peer-review. Is peer-review publication of bioinformatics software available? If so what is required to publish it?&lt;/p&gt;&#xA;" OwnerUserId="1855" LastActivityDate="2020-02-05T12:25:23.877" Title="Can open-source software be peer-reviewed and published?" Tags="|publishing|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11314" PostTypeId="2" ParentId="11313" CreationDate="2020-02-05T07:36:47.217" Score="13" Body="&lt;p&gt;There are whole journals based primarily around publishing open source tools. The primary example of that is &quot;Bioinformatics&quot;, where a lot of the open-source tools are published. We've also had luck publishing in the Nucleic Acids Research yearly special webservers issue, since we make Galaxy wrappers around our tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also publish on tools in venues like Nature Methods, but it's vastly more difficult to get in there if your paper is purely on a tool and it's not something game-changing like &lt;code&gt;salmon&lt;/code&gt; or &lt;code&gt;kallisto&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding what's actually needed to publish a tool, you will typically need the following:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Decent documentation. I ask for this as a reviewer and am asked about it when I submit things.&lt;/li&gt;&#xA;&lt;li&gt;Easy installation. No one wants to spend two hours trying to compile your tool, so make sure there's a conda package or at least a docker container for it.&lt;/li&gt;&#xA;&lt;li&gt;Test data. Without this your reviewers and users will likely not be happy since they won't be able to try it out on something small.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For the actual submission you'll tend to need comparisons to other tools. I'm generally not a big fan of this, but you will find that some reviewers will demand it (assuming there are other tools that do something similarish to yours).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-02-05T07:36:47.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11315" PostTypeId="2" ParentId="11313" CreationDate="2020-02-05T12:25:23.877" Score="9" Body="&lt;p&gt;Yes, there is even a journal dedicated to this: &lt;a href=&quot;https://joss.theoj.org&quot; rel=&quot;noreferrer&quot;&gt;The Journal of Open Source Software&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here the software itself is peer reviewed and a decent documentation and tests are required. Since the submission and review process is based on GitHub, publishing there is straightforward if your code is on GitHub already.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I am not affiliated with this journal, I just published two software packages there a while ago.&lt;/p&gt;&#xA;" OwnerUserId="668" LastActivityDate="2020-02-05T12:25:23.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11316" PostTypeId="2" ParentId="11276" CreationDate="2020-02-05T19:58:07.297" Score="1" Body="&lt;p&gt;This is similar to &lt;a href=&quot;https://www.biostars.org/p/303439/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this question&lt;/a&gt;, which links to &lt;a href=&quot;https://samtools.github.io/bcftools/howtos/consensus-sequence.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this solution&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The difference here is that you want to go through and replace low coverage (maybe zero-coverage?) positions with Ns rather than using the reference. You can likely accomplish this &lt;a href=&quot;https://www.htslib.org/doc/samtools-depth.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;using&lt;/a&gt; &lt;code&gt;samtools depth&lt;/code&gt; to measure coverage and then writing a script that will go through each position in your reference, check its coverage, and string-replace with an N. Probably a bash one-liner could do this if someone very devoted to bash one-liners came up with one (I don't care for them, but other people are more creative than I). Very quick and dirty pseudocode might look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;### Someone could come up with something better if they spent &amp;gt;3min on it.&#xA;### should be easy to get it working if you use biopython&#xA;# read_and_preprocess() stands for whatever you need to do to get data in the right shape&#xA;coverage_data = read_and_preprocess('samtools_coverage.txt')&#xA;# reference_with_variants.fasta comes from the solution linked above&#xA;reference_fasta = read_and_preprocess('reference_with_variants.fasta')&#xA;&#xA;threshold = 0&#xA;fasta_string = ''&#xA;for position in reference_fasta:&#xA;  if coverage_data[position] &amp;lt;= threshold:&#xA;    fasta_string += &quot;N&quot;&#xA;  else:&#xA;    fasta_string += reference_fasta[position]&#xA;&#xA;# nonexistent fn that writes a fasta. &#xA;write_as_fasta(fasta_string)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am a little skeptical of this approach because it seems like &lt;strong&gt;if you do not trust the reference sequence to stand in for zero-coverage areas, then you probably shouldn't trust it to align to and call variants against&lt;/strong&gt;. Read mapping can be a rather noisy process, especially when you are using a more distantly related reference. I think this is related to the @swbarnes2 comment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest instead indicating the regions of low/zero coverage by putting them in lowercase &lt;code&gt;acgt&lt;/code&gt; with your covered regions uppercase &lt;code&gt;ACGT&lt;/code&gt;, which preserves the coverage information that you care about but still allows you access to the reference information. It's not a perfect solution, but I think it makes the most out of the marginal data you're working with, and it is trivial to change the above example code to do it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps-&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2020-02-05T21:09:02.633" LastActivityDate="2020-02-05T21:09:02.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11318" PostTypeId="2" ParentId="11225" CreationDate="2020-02-05T20:35:36.013" Score="1" Body="&lt;p&gt;If your goal is only to account for degree distribution (i.e. Matteo Ferla's comment), a simulation approach you could use is:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;measure the degree (number of links) of each protein in your PPI&lt;/li&gt;&#xA;&lt;li&gt;find a large set of proteins that are equal or similar in degree to each of your proteins of interest (you may have to use a range of degrees to get a large enough set for comparison, ideally &gt;=1000). We'll call these &quot;comparable proteins&quot; for each of your proteins of interest.&lt;/li&gt;&#xA;&lt;li&gt;For each protein of interest, make all of your measurements also for its set of comparable proteins&lt;/li&gt;&#xA;&lt;li&gt;Compare your measurement (shortest distance?) for the protein of interest to that for its comparable proteins. You could compute Z-scores from the parameters of the comparable protein distribution, or just empirical Monte Carlo p-values from where it falls in the distribution.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;This will give you at least a heuristic null distribution for each protein. It should be relatively easy to code up in whatever your favorite language is.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps-&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-02-05T20:35:36.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11319" PostTypeId="1" CreationDate="2020-02-05T22:01:04.287" Score="1" ViewCount="116" Body="&lt;p&gt;I am very new to using python3 and am hoping to use it to reduce the time I need to spend running Infernal to look for a variety of RNAs in a large collection of bacterial genomes. I am hoping to write a short module in python that will let me sequentially run infernal using a single covariance model on a list of genome files. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;At the moment I am trying to use subprocess.run() or subprocess.Popen() to do so. &lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;&#xA;import subprocess&#xA;import os &#xA;import glob&#xA;&#xA;genomes = list(glob.glob(os.path.join(&quot;/path/to/genomes/&quot;, &quot;*.fa&quot;)))&#xA;&#xA;for x in files:&#xA;    subprocess.Popen(args=[&quot;cmsearch&quot;, &quot;/path/to/covariancemodel/&quot;, &quot;genomes&quot;], shell=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;my issue is I do not know a way to have &quot;genomes&quot; passed as the components of the list within the command. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this a way off approach to trying to automate this? If so is there a better way to do it or is there a way to simply pass each file in the list to the cmsearch command? &lt;/p&gt;&#xA;" OwnerUserId="7137" LastActivityDate="2020-02-06T01:39:30.860" Title="Writing a python module to run Infernal on a list of fasta files" Tags="|python|rna|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11320" PostTypeId="2" ParentId="411" CreationDate="2020-02-06T00:21:28.930" Score="2" Body="&lt;p&gt;NCBI is moving slowly to cloud (AWS/Google) for hosting. As such the tradiational FTP urls do not always work. My solution involves using &lt;a href=&quot;https://github.com/saketkc/pysradb&quot; rel=&quot;nofollow noreferrer&quot;&gt;pysradb&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An entire project can be downloaded by:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pysradb download -p &amp;lt;SRP_ID&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A Google Colab notebook with metadata retrieval commands is &lt;a href=&quot;https://colab.research.google.com/drive/1C60V-jkcNZiaCra_V5iEyFs318jgVoUR&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="161" LastActivityDate="2020-02-06T00:21:28.930" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11321" PostTypeId="2" ParentId="11319" CreationDate="2020-02-06T01:39:30.860" Score="1" Body="&lt;p&gt;I think what you want to do is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import subprocess&#xA;import os &#xA;import glob&#xA;&#xA;genomes = glob.glob(os.path.join(&quot;/path/to/genomes/&quot;, &quot;*.fa&quot;))&#xA;&#xA;for gfile in genomes:&#xA;    subprocess.Popen(args=[&quot;cmsearch&quot;, &quot;/path/to/covariancemodel/&quot;, gfile], shell=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;each pass through the for loop will take a one of the genome files and pass it into the cmsearch command. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) glob.glob returns a list so no need to cast it again - &lt;a href=&quot;https://docs.python.org/2/library/glob.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.python.org/2/library/glob.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2020-02-06T01:39:30.860" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11322" PostTypeId="2" ParentId="11206" CreationDate="2020-02-06T02:08:25.840" Score="1" Body="&lt;p&gt;Removing the old id from 'Parent`:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for subfeature in feature.sub_features:&#xA;    subfeature.qualifiers[&quot;Parent&quot;].pop()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Chromosome name:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; for rec in GFF.parse(gff3):&#xA;            chrID = rec.id &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2477" LastActivityDate="2020-02-06T02:08:25.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11323" PostTypeId="2" ParentId="11299" CreationDate="2020-02-06T06:59:33.553" Score="3" Body="&lt;p&gt;Leave-one-out cross validation is an iterative method that leaves out one sample until each sample has been left out once. You can then identify the intersection of the differentially expressed genes from the sample space of repeated analyses to produce a set of genes that are consistently differentially expressed independent of individual sample biases.&#xA;more info here:&#xA;&lt;a href=&quot;http://efavdb.com/leave-one-out-cross-validation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://efavdb.com/leave-one-out-cross-validation/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4507" LastActivityDate="2020-02-06T06:59:33.553" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11328" PostTypeId="1" AcceptedAnswerId="11383" CreationDate="2020-02-06T20:59:47.763" Score="2" ViewCount="675" Body="&lt;p&gt;The question says it all. I want to calculate lipophilicity of proteins to screen the compounds out for antigen-antibody interactions.&lt;/p&gt;&#xA;" OwnerUserId="7150" LastActivityDate="2020-03-03T21:15:15.950" Title="How to calculate lipophilicity of proteins?" Tags="|protein-protein-interaction|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11329" PostTypeId="1" AcceptedAnswerId="11330" CreationDate="2020-02-06T21:01:10.527" Score="0" ViewCount="381" Body="&lt;p&gt;I'm trying to run the following Rosetta command.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rosetta_scripts.linuxgccrelease -s data/binding/dataset/test/protein.pdb \ &#xA;    -parser:protocol data/binding/test/scripts/pack.xml \&#xA;    -out:path:all data/binding/test/output/ \&#xA;    -resfile data/binding/test/output/resfile.txt \&#xA;    -out:prefix RosettaOutput_ \&#xA;    -nstruct 1 -ignore_zero_occupancy false -overwrite&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I keep getting the error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;core.pack.task.operation.TaskOperations: {0} [ ERROR ] File:resfile.txt not found!&#xA;&#xA;ERROR: Cannot open file resfile.txt&#xA;ERROR:: Exit from: /home/sean/src/rosetta_src_2019.35.60890_bundle/main/source/src/core/pack/task/operation/TaskOperations.cc line: 1645&#xA;Error: {0} [ ERROR ] ERROR: Exception caught by JobDistributor while trying to get pose from job 'RosettaOutput_2MWY_0001'&#xA;Error: {0} [ ERROR ] Treating failure as bad input; canceling similar jobs&#xA;&#xA;[ ERROR ]: Caught exception:&#xA;&#xA;&#xA;File: /home/sean/src/rosetta_src_2019.35.60890_bundle/main/source/src/core/pack/task/operation/TaskOperations.cc:1645&#xA;[ ERROR ] UtilityExitException&#xA;ERROR: Cannot open file resfile.txt&#xA;&#xA;&#xA;&#xA;&#xA;AN INTERNAL ERROR HAS OCCURED. PLEASE SEE THE CONTENTS OF ROSETTA_CRASH.log FOR DETAILS.&#xA;&#xA;&#xA;protocols.jd2.FileSystemJobDistributor: {0} job failed, reporting bad input; other jobs of same input will be canceled: RosettaOutput_2MWY_0001&#xA;protocols.jd2.JobDistributor: {0} no more batches to process... &#xA;protocols.jd2.JobDistributor: {0} 1 jobs considered, 1 jobs attempted in 13 seconds&#xA;&#xA;[ ERROR ]: Caught exception:&#xA;&#xA;&#xA;File: /home/sean/src/rosetta_src_2019.35.60890_bundle/main/source/src/protocols/jd2/JobDistributor.cc:327&#xA;1 jobs failed; check output for error messages&#xA;&#xA;&#xA;AN INTERNAL ERROR HAS OCCURED. PLEASE SEE THE CONTENTS OF ROSETTA_CRASH.log FOR DETAILS.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The error resolves itself if I move &lt;code&gt;resfile.txt&lt;/code&gt; to the current working directory. Why can't &lt;code&gt;resfile.txt&lt;/code&gt; be found?&lt;/p&gt;&#xA;" OwnerUserId="6966" LastEditorUserId="6322" LastEditDate="2020-07-01T09:10:19.233" LastActivityDate="2020-07-01T09:10:19.233" Title="How to specify input resfile for Rosetta?" Tags="|protein-structure|computational-biochemistry|rosetta|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11330" PostTypeId="2" ParentId="11329" CreationDate="2020-02-06T21:01:10.527" Score="2" Body="&lt;p&gt;The problem is my &lt;code&gt;pack.xml&lt;/code&gt; specified a &lt;code&gt;resfile.txt&lt;/code&gt; location which over-wrote my command-line option.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    &amp;lt;TASKOPERATIONS&amp;gt;&#xA;       &amp;lt;ReadResfile name=&quot;Design_Resfile&quot; filename=&quot;resfile.txt&quot;/&amp;gt;&#xA;    &amp;lt;/TASKOPERATIONS&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you remove the &lt;code&gt;filename&lt;/code&gt; option in the XML file, you are then able to specify the path.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    &amp;lt;TASKOPERATIONS&amp;gt;&#xA;       &amp;lt;ReadResfile name=&quot;Design_Resfile&quot;/&amp;gt;&#xA;    &amp;lt;/TASKOPERATIONS&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6966" LastActivityDate="2020-02-06T21:01:10.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11331" PostTypeId="1" CreationDate="2020-02-06T21:22:43.683" Score="2" ViewCount="70" Body="&lt;p&gt;There is a place in my pipeline where I call PDB's API with a ribosomal id to get all the subchains of that molecule programmatically&lt;/p&gt;&#xA;&lt;p&gt;I'd like to separate subchains that are proteins from nucleic acids in the response, or at least know how to get the chain-ids of the nucleic acids so that i can exclude them from all manually.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NDLUe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NDLUe.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Wondering if there is some query parameter like &lt;em&gt;is_nucleic_acid&lt;/em&gt; in the PDB's ADL, but that's just a hunch. I'd be happy to learn if there is a more idiomatic way to do it( from Uniprot mappings maybe? ).&lt;/p&gt;&#xA;&lt;p&gt;My query as of now looks like when i want to get, say &lt;em&gt;uniprot acccessions&lt;/em&gt; for subchains(in &lt;strong&gt;fl&lt;/strong&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;const PDBSearchSelect = &amp;quot;https://ww.ebi.ac.uk/pdbe/search/select?&amp;quot;&amp;quot;;&#xA;const params = { &#xA;// this is of the form &amp;quot;3j9m&amp;quot;, &amp;quot;5afi&amp;quot;, etc...   &#xA;q: `${input.toLowerCase()}`,        &#xA;//wondering if there is an &amp;quot;fl&amp;quot; parameter to get all nucleic acids&#xA;fl: &amp;quot;molecule_name, uniprot_id, uniprot_accession, struct_asym_id,...other&amp;quot;, &#xA;//&#xA;rows: &amp;quot;200&amp;quot;&#xA;};&#xA;&#xA;axios.get(PDBSearchSelect, {params}).... =&amp;gt; ..&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would actually be super grateful if somebody could elucidate what's the difference between&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.ebi.ac.uk/pdbe/search/pdb&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/pdbe/api&lt;/a&gt;/... and&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;`&lt;a href=&quot;https://www.ebi.ac.uk/pdbe/search/pdb&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/pdbe/search/pdb&lt;/a&gt;/select?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I've been looking at sides of the PDB API for a few weeks now and it still seems like complete babylon in terms of naming. I realize that it grew organically over years.&lt;/p&gt;&#xA;&lt;p&gt;Thanks a ton.&lt;/p&gt;&#xA;" OwnerUserId="6511" LastEditorUserId="3848" LastEditDate="2020-11-19T20:39:34.867" LastActivityDate="2020-11-19T20:39:34.867" Title="Get nucleic acids' chain names from PDB API" Tags="|pdb|uniprot|api|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11334" PostTypeId="1" AcceptedAnswerId="11337" CreationDate="2020-02-07T05:08:41.877" Score="1" ViewCount="678" Body="&lt;p&gt;When calculating how lethal a virus outbreak is, I've noticed that most sources use &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CONFIRMED_DEATHS/CONFIRMED_INFECTIONS&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, given that there are only 2 possible outcomes; death or recovery, wouldn't it be more logical to use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CONFIRMED_DEATHS/(CONFIRMED_DEATHS + CONFIRMED_RECOVERIES)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which formula is applicable to which situations?&lt;/p&gt;&#xA;" OwnerUserId="7156" LastEditorUserId="3848" LastEditDate="2020-12-04T15:37:00.183" LastActivityDate="2020-12-04T15:37:00.183" Title="Calculating the mortality rate of a pandemic, e.g. coronavirus 2019-nCov?" Tags="|statistics|covid-19|sars-cov-2|epidemiology|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11335" PostTypeId="1" AcceptedAnswerId="11340" CreationDate="2020-02-07T11:10:43.583" Score="0" ViewCount="2325" Body="&lt;p&gt;I have 25 tumor samples with counts data. Initially, I filtered out low expressed genes and then converted counts to &lt;code&gt;varianceStabilizingTransformation&lt;/code&gt; using &lt;code&gt;deseq2&lt;/code&gt; package. With this data I started using &lt;code&gt;WGCNA&lt;/code&gt; for co-expression network analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For selecting the soft threshold I see very strange plot. R2 cutoff is 0.8 and I see that none of the scale free topology model fit is above that. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the code I used:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;df&lt;/code&gt; is a dataframe with genes as rows and 25 samples as columns with counts data.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(&quot;DESeq2&quot;)&#xA;filtered.counts &amp;lt;- df[rowSums(df==0)&amp;lt;5, ]&#xA;&#xA;U3 &amp;lt;- as.matrix(filtered.counts)&#xA;vsd &amp;lt;- vst(U3, blind=FALSE)&#xA;&#xA;oed &amp;lt;- vsd&#xA;&#xA;gene.names=rownames(oed)&#xA;trans.oed=t(oed)&#xA;dim(trans.oed)&#xA;&#xA;n=16462;&#xA;datExpr=trans.oed[,1:n]&#xA;dim(datExpr)&#xA;&#xA;SubGeneNames=gene.names[1:n]&#xA;&#xA;library(WGCNA)&#xA;options(stringsAsFactors = FALSE);&#xA;allowWGCNAThreads()&#xA;&#xA;powers = c(c(1:10), seq(from = 12, to=20, by=2));&#xA;sft=pickSoftThreshold(datExpr,dataIsExpr = TRUE,&#xA;                      powerVector = powers,corFnc = cor,&#xA;                      corOptions = list(use = 'p'),networkType = &quot;unsigned&quot;)&#xA;&#xA;   Power SFT.R.sq  slope truncated.R.sq mean.k. median.k. max.k.&#xA;1      1 0.000273  0.041          0.786 4140.00  4000.000 6730.0&#xA;2      2 0.428000 -1.130          0.852 1570.00  1400.000 3770.0&#xA;3      3 0.673000 -1.540          0.882  729.00   579.000 2410.0&#xA;4      4 0.737000 -1.720          0.891  383.00   268.000 1670.0&#xA;5      5 0.745000 -1.830          0.886  220.00   134.000 1210.0&#xA;6      6 0.704000 -1.990          0.860  134.00    71.700  909.0&#xA;7      7 0.737000 -1.980          0.890   85.90    40.300  701.0&#xA;8      8 0.742000 -2.020          0.903   57.30    23.500  551.0&#xA;9      9 0.733000 -2.090          0.917   39.40    14.200  441.0&#xA;10    10 0.750000 -2.080          0.934   27.80     8.810  357.0&#xA;11    12 0.770000 -2.080          0.952   14.80     3.670  243.0&#xA;12    14 0.775000 -2.090          0.951    8.43     1.660  171.0&#xA;13    16 0.397000 -2.820          0.459    5.06     0.801  124.0&#xA;14    18 0.409000 -2.770          0.474    3.18     0.406   91.4&#xA;15    20 0.421000 -2.720          0.490    2.06     0.215   68.8&#xA;&#xA;&#xA;# Plot the results&#xA;sizeGrWindow(9, 5)&#xA;par(mfrow = c(1,2));&#xA;cex1 = 0.9;&#xA;&#xA;# Scale-free topology fit index as a function of the soft-thresholding power&#xA;plot(sft&lt;span class=&quot;math-container&quot;&gt;$fitIndices[,1], -sign(sft$&lt;/span&gt;fitIndices[,3])*sft&lt;span class=&quot;math-container&quot;&gt;$fitIndices[,2],xlab=&quot;Soft Threshold (power)&quot;,ylab=&quot;Scale Free Topology Model Fit, signed R^2&quot;,type=&quot;n&quot;, main = paste(&quot;Scale independence&quot;));&#xA;text(sft$&lt;/span&gt;fitIndices[,1], -sign(sft&lt;span class=&quot;math-container&quot;&gt;$fitIndices[,3])*sft$&lt;/span&gt;fitIndices[,2],labels=powers,cex=cex1,col=&quot;red&quot;);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MJBpD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MJBpD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this the right way? Which soft threshold power should I select? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is appreciated. thanq.&lt;/p&gt;&#xA;" OwnerUserId="1369" LastActivityDate="2020-02-07T18:02:07.940" Title="WGCNA: Problem with selecting soft threshold" Tags="|r|rna-seq|networks|rna|wgcna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11336" PostTypeId="1" CreationDate="2020-02-07T11:44:52.787" Score="2" ViewCount="70" Body="&lt;p&gt;My impression is that small InDel (a couple of bp) is identified through cigar string in BAM and typical CNV (at least thousands of bp) is detected through read depth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What about InDel or CNV with size these them, say hundreds of bp? These CNV is too long to be covered by read and too short to be detected by statistically comparing the difference of read depth.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do we have consensus that this kind of CNV is hard to detect? Or is there a common strategy of detection for them? Do we generally think them having impact on protein function (especially in oncology background) ?&lt;/p&gt;&#xA;" OwnerUserId="4905" LastEditorUserId="96" LastEditDate="2020-02-07T16:07:40.437" LastActivityDate="2021-11-04T15:05:15.530" Title="Detection of CNV(InDel) of intermediate size" Tags="|variant-calling|human-genome|cnv|indel|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11337" PostTypeId="2" ParentId="11334" CreationDate="2020-02-07T15:45:34.467" Score="4" Body="&lt;p&gt;This may not strike most as a bioinformatics, but getting the key clinical outcome is essential in understanding the molecular basis of pathogenicity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the mortality rate is over-reported. This is not to say the situation of 2019-nCov is not serious - it is very serious.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two essential factors missing in your equation for 2019-nCov are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Age. Mortality is likely skewed towards the old and very young&lt;/li&gt;&#xA;&lt;li&gt;Seroprevalence. To identify the number of asymptomatic patients.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Old people have a weakened immune system (e.g. reduced T-cell response). The very young have an underdeveloped immune system. The problem is hospital cases are skewed towards those with severe infection, which is likely the old and very young. In 2019-nCov the first infecteds were middle aged men, but likely to be restricted to those exposed to the &quot;Wuhan meat market&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt; &#xA;Asymptomatic &quot;spreaders&quot; are known in influenza and are suspected in 2019-nCov. Asymptomatics will inflate the mortality rate (because we're not detecting them).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Solution(s)&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;One solution is to conduct seroprevalence surveys. Serological surveys screening for antibodies (IgG) against 2019-nCov will identify asymptomatic patients, using the first formula you stated. However, would you seriously go around the general population in a 2019-nCov epidemic taking blood samples? Errhhh no.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;strong&gt;&lt;em&gt;bio-statistics&lt;/em&gt;&lt;/strong&gt; approach is to use a &lt;strong&gt;catalytic model&lt;/strong&gt; and critical to this is the age of the patient. Catalytic models are a priori models that incorporates the age-prevelance profile of infection. By introducing the incidence and age of hospitalisation of e.g. the old and the young, the rest of the distribution, i.e. asymptomatics can be calculated by regression. This estimates the total population of infecteds and again you use equation 1 to calculate the mortality rate recorded by the hospital.&lt;/li&gt;&#xA;&lt;li&gt;Mathematical models based on transmission dynamics. The problem here is the rate of transmission is unknown.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I would assume the hospital is the place virus-induced mortality is reasonably accurate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second formula you described works very well for Ebolavirus, no-one has ever heard of asymptomatic Ebolavirus. It would also work well for another betacoronavirus Middle Eastern respiratory syndrome. The key problem with 2019-nCov is we don't know the number of asymptomatic patients or patients with such mild infections they never report illness. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&#xA;In summary, I think the mortality is being over-reported if it is based on your equation 2. Equation 1, however, will also over-report the mortality rate.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-02-07T16:28:58.377" LastActivityDate="2020-02-07T16:28:58.377" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11338" PostTypeId="2" ParentId="11336" CreationDate="2020-02-07T16:07:19.757" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;My impression is that small InDel (a couple of bp) is identified through cigar string in BAM...&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Well, yes, by gaps in the alignment, easily detected by examining the CIGAR string.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;...and typical CNV (at least thousands of bp) is detected through read depth.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Also by deviations from expected read pair distance or orientation.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;What about InDel or CNV with size these them, say hundreds of bp? These CNV is too long to be covered by read and too short to be detected by statistically comparing the difference of read depth. Do we have consensus that this kind of CNV is hard to detect?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Yes, what you describe has certainly been recognized and acknowledged in the literature. It was one of the primary motivations for &lt;a href=&quot;https://doi.org/10.1016/j.isci.2019.07.032&quot; rel=&quot;nofollow noreferrer&quot;&gt;a recently published paper I contributed to&lt;/a&gt; on &lt;em&gt;de novo&lt;/em&gt; (germline) variant discovery. Here's an excerpt from that paper:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;In a reference-mapping context, calling indels with confidence requires accurate mapping of each read spanning the indel, with all gaps arranged consistently. This is possible only for short indels and tends to be prone to error and misalignment. Thus prediction of indels with length 10 bp has proved to be very challenging and accompanied by high false-positive and false-negative rates. Furthermore, the prediction of SVs via read mapping is only possible through indirect signatures such as alterations in read depth or read-pair signatures. These signatures can be quite noisy and result in high rate of false-negative and false-positive prediction.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;As for your question:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;is there a common strategy of detection for them?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Numerous mapping-free k-mer based methods for variant detection are coming up in the literature, and all seem to perform pretty well. The trick is how to define which k-mers are interesting—then you grab those k-mers (or the reads containing them) and assemble them into variant-spanning contigs. A few examples:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In the &lt;a href=&quot;https://doi.org/10.1016/j.isci.2019.07.032&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kevlar paper&lt;/a&gt;, we defined  &quot;interesting&quot; k-mers as those with high abundance in an individual of interest but zero (or very low) abundance in the individual's parents.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;https://doi.org/10.7554/eLife.32920&quot; rel=&quot;nofollow noreferrer&quot;&gt;HAWK paper&lt;/a&gt;, which describes a k-mer based GWAS-like method, defines &quot;significant&quot; k-mers as those significantly over- or under-represented in one class of samples versus a second class. &lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;https://doi.org/10.1038/nmeth.4084&quot; rel=&quot;nofollow noreferrer&quot;&gt;NovoBreak&lt;/a&gt; paper compares paired tumor and normal WGS data to find k-mers that span mutatino breakpoints in cancer genomes.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;Do we generally think them having impact on protein function (especially in oncology background)?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is definitely &lt;em&gt;potential&lt;/em&gt; that indels can have a substantial impact on protein function. But since their detection has been so difficult for so long, a lot remains unknown about their &lt;em&gt;actual&lt;/em&gt; collective impact on function.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2020-02-07T16:07:19.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11340" PostTypeId="2" ParentId="11335" CreationDate="2020-02-07T18:02:07.940" Score="2" Body="&lt;p&gt;Don't worry about it too much. I would go with power 8 based on my general experience (also reflected in WGCNA FAQ) and on the mean connectivity around 50 and median around 20, which seems reasonable to me.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2020-02-07T18:02:07.940" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11342" PostTypeId="2" ParentId="11291" CreationDate="2020-02-08T15:26:30.383" Score="2" Body="&lt;p&gt;Proving a negative is hard, but to my knowledge I do not know a database that has &lt;em&gt;all&lt;/em&gt; the structure of &lt;em&gt;drug&lt;/em&gt; bound protein. I can give some close matches.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First a few terms to be on the same page. A &lt;strong&gt;ligand&lt;/strong&gt; is something that binds, which can be physiological, such as a &lt;strong&gt;cofactor&lt;/strong&gt; or &lt;strong&gt;substrate&lt;/strong&gt;, or not (especially man-made), such as a &lt;strong&gt;drug&lt;/strong&gt;, i.e. a compound that has a physiological effect, or a &lt;strong&gt;hit&lt;/strong&gt;, a small fragment from a fragment screen that will not bind with nanomolar affinity. Generally, ligands are small molecules, but may include oligopeptides and oligonucleotides. In the case of crystal structures, you also have &lt;strong&gt;crystallisation additives&lt;/strong&gt;, which are not ligands, but are hetatoms in PDBs. Also, the definition of drug may vary. &lt;a href=&quot;https://www.drugbank.ca/&quot; rel=&quot;nofollow noreferrer&quot;&gt;drugbank.ca&lt;/a&gt; has the following categories of drugs &quot;approved&quot;, &quot;experimental&quot;, &quot;withdrawn&quot; for pharmaceuticals, &quot;illicit&quot; for narcotics and &quot;investigational&quot; for probes and counts macromolecules as drugs.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.rcsb.org/pdb/ligand/drugMapping.do&quot; rel=&quot;nofollow noreferrer&quot;&gt;RCSB PDB: drug mapping table&lt;/a&gt; is a simple searchable table with ligands that are clinical drugs only —experimental compounds (=probes) will be absent.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.bindingdb.org/bind/surflex_entry.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;bindingdb.org&lt;/a&gt; contains many table, including a '3D' one. This table contains all the PDBs with ligands and their affinities. This contains crystallisation additives and natural ligands. Peter Curran's repo for the HotSpots algorithm (first written by Chris Radoux) has a &lt;a href=&quot;https://github.com/prcurran/hotspots/blob/master/hotspots/excluded_het_id.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;blacklist&lt;/a&gt;, but you are bound to get stuff missed.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;http://bindingmoad.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;bindingmoad.org&lt;/a&gt; is a database with all the protein-ligand structures from PDB up until 2018, but it includes natural ligands, like cofactors and substrates as above.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://klifs.vu-compmedchem.nl/index.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://klifs.vu-compmedchem.nl&lt;/a&gt; a well curated database of kinases, which has a lot of members and many ligands, but is not at all diverse. As a result it is a clean dataset often used as a testbed for novel methodologies.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I mentioned in the comments the &lt;a href=&quot;https://www.thesgc.org/tep&quot; rel=&quot;nofollow noreferrer&quot;&gt;SGC Target Enabling Packages&lt;/a&gt; as a small subset of proteins with many hits each. But I realise that there is not a handy PDB list and the targets are not drugs, but probes.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-03-05T08:17:13.430" LastActivityDate="2020-03-05T08:17:13.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11343" PostTypeId="1" AcceptedAnswerId="11344" CreationDate="2020-02-08T15:44:32.220" Score="0" ViewCount="39" Body="&lt;p&gt;I am new to bioinformatics. While reading Momand's '&lt;em&gt;Concepts in Bioinformatics and Genomics&lt;/em&gt;' , I understood clearly that each codon translates to a specific amino acid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But then, he talks about the &lt;em&gt;TP53&lt;/em&gt; gene, and then without explanation talks about P53... Is that a naming convention? It's not explicitly explained... Does that applies for any case? ie &lt;em&gt;TP90&lt;/em&gt; gene encodes protein P90?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't understand the naming: correct me if wrong, but they named the gene after the protein's weight, which is 53 kilodaltons? &#xA;(It confuses me with the location of the gene).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly I do not understand: if there is a mutation in the DNA, the mutation shall appear in only one cell I guess, how then the others cells' DNA start to mutate? or is it that all DNA in all cells are mutated instantly?&lt;/p&gt;&#xA;" OwnerUserId="6829" LastActivityDate="2020-02-08T17:35:46.460" Title="tp53 example in Momand's book" Tags="|gene|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11344" PostTypeId="2" ParentId="11343" CreationDate="2020-02-08T17:35:46.460" Score="3" Body="&lt;p&gt;This is possibly more of a molecular genetics question appropriate to biology.stackexchange.com, also note that it's usually better on stackexchange sites to &lt;a href=&quot;https://meta.stackexchange.com/help/how-to-ask&quot;&gt;restrict your post to a single question&lt;/a&gt; and do some preliminary research via google or your search engine of choice. But I don't see any harm in answering it here nonetheless. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The real answer is that naming conventions for genes/proteins are incredibly inconsistent and counterintuitive. You can't really draw any conclusions about a gene/protein based on its name alone. For example, some proteins such as p53 are named for biochemical features, others are named for phenotypes observed when their gene is mutated, such as for instance the gene &lt;em&gt;decapentaplegic&lt;/em&gt; in fruit flies. Often they involve little jokes, such as the gene called &lt;em&gt;mothers against decapentaplegic&lt;/em&gt; which represses the &lt;em&gt;decapentaplegic&lt;/em&gt; mutant phenotype. So it's all a bit of a mess and people kind of assume that you'll follow along, which can be confusing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a little rarer that gene names differ substantially from the names of the proteins they encode, but not unheard-of. This is indeed the case with your &lt;em&gt;TP53&lt;/em&gt;/p53 example. See a short explainer &lt;a href=&quot;https://ghr.nlm.nih.gov/gene/TP53&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; or a longer one &lt;a href=&quot;https://en.wikipedia.org/wiki/P53&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. What you'll notice is that there are in fact multiple different names for the same gene/protein. What you'll often see is that the shorter name (p53) is the one that people tend to use colloquially.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your second question regarding how mutations are propagated, the short answer is that mutations are inherited from mother to daughter cells. Cells don't generally &quot;infect&quot; other cells with mutations (though in biology there are always exceptions). I think that what you are interested in is &lt;a href=&quot;https://www.google.com/search?q=somatic%20and%20germline%20mutations&amp;amp;rlz=1C5CHFA_enUS695US695&amp;amp;oq=somatic%20and%20germ&amp;amp;aqs=chrome.0.0j69i57j0l6.3541j0j4&amp;amp;sourceid=chrome&amp;amp;ie=UTF-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;the difference between &quot;somatic&quot; and &quot;germ-line&quot; mutation&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-02-08T17:35:46.460" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11345" PostTypeId="1" CreationDate="2020-02-08T20:33:52.533" Score="5" ViewCount="4439" Body="&lt;p&gt;I want to convert a list of fasta ( protein sequences) in a .text file into corresponding nucleotide sequences. A Google search gives me result of DNA to protein conversion but not vice versa. Also, I came across How do I find the nucleotide sequence of a protein using Biopython?, but this is what I am not looking for. Is there any possible way to do it using python.Moreover, I would like to solve it using python programming. I am sure there must be some way to do it rather than writing a code from scratch. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="6794" LastActivityDate="2020-11-23T16:01:58.590" Title="How to translate amino acid sequences to Nucleotide sequences" Tags="|proteins|python|phylogenetics|sequence|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11346" PostTypeId="1" CreationDate="2020-02-09T04:31:03.800" Score="0" ViewCount="72" Body="&lt;p&gt;I am working with the &lt;code&gt;createMAE()&lt;/code&gt; function of the ELMER Bioconductor package. While executing the &lt;code&gt;createMAE()&lt;/code&gt; function using TCGA LIHC DNA Methylation 450K and RNASeq HTSeq - Counts data, I am getting this error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in.rowNamesDF &amp;lt;-(x, value = value) :&#xA;&#xA;duplicate 'row.names' are not allowed&#xA;&#xA;In addition: Warning message:&#xA;&#xA;non-unique values when setting 'row.names': 'TCGA-DD-AACA-02A', 'TCGA-DD-AACA-02B'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can anyone tell me how to get rid of such error?&lt;/p&gt;&#xA;" OwnerUserId="4498" LastEditorUserId="3541" LastEditDate="2020-02-09T06:35:52.713" LastActivityDate="2020-03-10T07:00:40.403" Title="Error in createMAE function: non-unique values when setting 'row.names' in TCGA LIHC Data" Tags="|r|rna-seq|bioconductor|methylation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11347" PostTypeId="2" ParentId="11346" CreationDate="2020-02-09T06:39:31.330" Score="1" Body="&lt;p&gt;The error message is pretty clear: R requires unique row names on data frames and some of the &quot;values&quot; that you are trying to set as row names are non-unique.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will need to provide a vector i) of length the same as your row number, ii) that is composed of non-unique values.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-02-09T06:39:31.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11350" PostTypeId="2" ParentId="11345" CreationDate="2020-02-09T09:57:14.263" Score="1" Body="&lt;p&gt;You can't do this because there is redundancy in the genetic code and the &lt;em&gt;same&lt;/em&gt; protein sequences can be encoded by &lt;em&gt;different&lt;/em&gt; nucleotide sequences. There are 64 codons and ~20 amino acids, e.g. &lt;code&gt;GCT&lt;/code&gt;, &lt;code&gt;GCC&lt;/code&gt;, &lt;code&gt;GCA&lt;/code&gt; and &lt;code&gt;GCG&lt;/code&gt; all encode Alanine.&lt;/p&gt;&#xA;&#xA;&#xA;" OwnerUserId="104" LastEditorUserId="7340" LastEditDate="2020-06-16T12:15:28.983" LastActivityDate="2020-06-16T12:15:28.983" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11351" PostTypeId="1" AcceptedAnswerId="11375" CreationDate="2020-02-09T10:05:54.883" Score="1" ViewCount="224" Body="&lt;p&gt;I am trying to obtain the sequence from PDB file, but no output is given when I run the following code :  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;with open(&quot;1a9l.pdb&quot;, &quot;r&quot;) as handle:&#xA;    for record in SeqIO.parse(handle, &quot;pdb-atom&quot;):&#xA;        print(record.seq)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6445" LastEditorUserId="4622" LastEditDate="2020-02-10T16:20:42.637" LastActivityDate="2020-02-11T13:12:11.250" Title="No output when trying to obtain protein sequence from PDB file" Tags="|python|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11352" PostTypeId="2" ParentId="11345" CreationDate="2020-02-09T16:57:30.287" Score="5" Body="&lt;p&gt;Although there is not a unique nucleotide sequence that translates to a given protein, one can list all the possible DNA sequences that do translate to that protein.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An online tool that does just that is &lt;a href=&quot;https://www.ebi.ac.uk/Tools/st/emboss_backtranambig/&quot; rel=&quot;noreferrer&quot;&gt;Backtranambig&lt;/a&gt;, from EMBOSS. It produces a DNA sequence representing all the nucleotide sequences matching the input protein, using &lt;a href=&quot;https://droog.gs.washington.edu/parc/images/iupac.html&quot; rel=&quot;noreferrer&quot;&gt;IUPAC ambiguity codes&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3389" LastActivityDate="2020-02-09T16:57:30.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11353" PostTypeId="1" AcceptedAnswerId="11358" CreationDate="2020-02-09T17:13:16.303" Score="34" ViewCount="9228" Body="&lt;p&gt;I'm looking at &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/MN988713.1?report=fasta&quot; rel=&quot;noreferrer&quot;&gt;a genome sequence for 2019-nCoV on NCBI&lt;/a&gt;. The FASTA sequence looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;MN988713.1 Wuhan seafood market pneumonia virus isolate 2019-nCoV/USA-IL1/2020, complete genome&#xA;ATTAAAGGTTTATACCTTCCCAGGTAACAAACCAACCAACTTTCGATCTCTTGTAGATCTGTTCTCTAAA&#xA;CGAACTTTAAAATCTGTGTGGCTGTCACTCGGCTGCATGCTTAGTGCACTCACGCAGTATAATTAATAAC&#xA;TAATTACTGTCGTTGACAGGACACGAGTAACTCGTCTATCTTCTGCAGGCTGCTTACGGTTTCGTCCGTG&#xA;...  &#xA;...&#xA;TTAATCAGTGTGTAACATTAGGGAGGACTTGAAAGAGCCACCACATTTTCACCGAGGCCACGCGGAGTAC&#xA;GATCGAGTGTACAGTGAACAATGCTAGGGAGAGCTGCCTATATGGAAGAGCCCTAATGTGTAAAATTAAT&#xA;TTTAGTAGTGCTATCCCCATGTGATTTTAATAGCTTCTTAGGAGAATGACAAAAAAAAAAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Coronavirus is an RNA virus, so I was expecting the sequence to consist of &lt;code&gt;AUGC&lt;/code&gt; characters. But the letters here are &lt;code&gt;ATGC&lt;/code&gt;, which looks like DNA!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found a possible answer, that this is the sequence of a &lt;a href=&quot;https://en.wikipedia.org/wiki/Complementary_DNA&quot; rel=&quot;noreferrer&quot;&gt;&quot;complementary DNA&quot;&lt;/a&gt;. I read that&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The term &lt;em&gt;cDNA&lt;/em&gt; is also used, typically in a bioinformatics context, to refer to an mRNA transcript's sequence, expressed as DNA bases (GCAT) rather than RNA bases (GCAU).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;However, I don't believe this theory that I'm looking at a cDNA. If this were true, the end of the true mRNA sequence would be &lt;code&gt;...UCUUACUGUUUUUUUUUUUU&lt;/code&gt;, or a &quot;poly(U)&quot; tail. But I believe the coronavirus has a &lt;a href=&quot;https://en.wikipedia.org/wiki/Polyadenylation&quot; rel=&quot;noreferrer&quot;&gt;poly(A) tail&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also found that the start of all highlighted genes begin with the sequence &lt;code&gt;ATG&lt;/code&gt;. This is the DNA equivalent of &lt;a href=&quot;https://en.wikipedia.org/wiki/Start_codon&quot; rel=&quot;noreferrer&quot;&gt;the RNA start codon &lt;code&gt;AUG&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I believe what I'm looking at is the true mRNA, in 5'→3' direction, but with all &lt;code&gt;U&lt;/code&gt; converted to &lt;code&gt;T&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;So, is this really what I'm looking at? Is this some formatting/representation issue? Or does 2019-nCoV really contain DNA, rather than RNA?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="6910" LastEditorUserId="3848" LastEditDate="2020-07-21T19:00:14.157" LastActivityDate="2020-07-21T19:00:14.157" Title="Why does the FASTA sequence for coronavirus look like DNA, not RNA?" Tags="|fasta|phylogenetics|ngs|phylogeny|covid-19|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11354" PostTypeId="1" CreationDate="2020-02-09T17:14:05.957" Score="1" ViewCount="528" Body="&lt;p&gt;I've got some doubts on the hisat2 &lt;code&gt;--rna-strandness&lt;/code&gt; option and its output for downstream analysis. Is it expected to see a difference in alignment and counting results given the default usage of &lt;code&gt;--rna-strandness&lt;/code&gt; in hisat2 followed by &lt;code&gt;htseq-count -s reverse&lt;/code&gt; on a strand-specific assay?&lt;/p&gt;&#xA;&lt;p&gt;Please see below.&lt;/p&gt;&#xA;&lt;p&gt;I understand that the &lt;code&gt;--rna-strandness&lt;/code&gt; option produces an XS tag to indicate where a transcript is from (on the + or - strand) for downstream transcriptome assembly analysis. I have a paired-end stranded sequencing library that was aligned to the genome using hisat2 without specifying the &lt;code&gt;--rna-strandness&lt;/code&gt; (in other words, the default unstranded was the usage). Following this, the reads were assigned to genes using htseq-count and this time &lt;code&gt;-s reverse&lt;/code&gt; was specified given the strand-specific sequencing assay type.&lt;/p&gt;&#xA;&lt;p&gt;Since &lt;code&gt;--rna-strandness&lt;/code&gt; is for transcriptome assembly using the XS tags generated and htseq does not use XS tags for counting, I presume there should be no practical impact from the above. Could you also shed light on this? in case I may have been overlooking other facts of the usages of the tools.&lt;/p&gt;&#xA;&lt;p&gt;To help verify the above, I re-aligned and counted the reads from 2 samples by switching on &lt;code&gt;--rna-strandness RF&lt;/code&gt; in hisat2. I attach the alignment and count features info. below for assessment.&lt;/p&gt;&#xA;&lt;p&gt;Overall alignment rate of Sample 1:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;94.52% (--rna-strandness RF)&#xA;94.12% (--rna-strandness unstranded)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Overall alignment rate of Sample 2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;94.57% (--rna-strandness RF)&#xA;94.15% (--rna-strandness unstranded)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Feature counts of Sample 1 (following --rna-strandness RF + -s reverse):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;__no_feature        6327294&#xA;__ambiguous     2954776&#xA;__too_low_aQual     3784481&#xA;__not_aligned       688856&#xA;__alignment_not_unique      4858182&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Feature counts of Sample 1 (following --rna-strandness unstranded + -s reverse):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;__no_feature        6291151&#xA;__ambiguous     2911298&#xA;__too_low_aQual     4075017&#xA;__not_aligned       754400&#xA;__alignment_not_unique      16136045&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Feature counts of Sample 2 (following --rna-strandness RF + -s reverse):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;__no_feature        5417882&#xA;__ambiguous     1708510&#xA;__too_low_aQual     3532352&#xA;__not_aligned       564596&#xA;__alignment_not_unique      2859501&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Feature counts of Sample 2 (following --rna-strandness unstranded + -s reverse):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;__no_feature        5359434&#xA;__ambiguous     1676091&#xA;__too_low_aQual     3813344&#xA;__not_aligned       623122&#xA;__alignment_not_unique      2891792&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These results look comparable to me across pipelines.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&#xA;Guan&lt;/p&gt;&#xA;" OwnerUserId="7167" LastEditorUserId="73" LastEditDate="2021-07-06T01:32:20.487" LastActivityDate="2022-11-29T13:08:29.237" Title="hisat2 --rna-strandness option and downstream htseq-count analysis" Tags="|rna-seq|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11356" PostTypeId="1" AcceptedAnswerId="11359" CreationDate="2020-02-09T17:39:14.907" Score="-1" ViewCount="173" Body="&lt;p&gt;I have a long format data frame like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(df)&#xA;     gene &#xA;&#xA;1:  CCL28   2.030674       2.023333 1.921508    1.952007&#xA;2:  CXCL2   2.204847       2.275296 2.175790    2.241554&#xA;3:  CXCR4   2.121186       2.147181 1.950065    1.982639&#xA;4: IFNGR1   2.311390       2.324111 2.220727    2.255556&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And by this code &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(df, aes(variable,value)) +&#xA;  stat_boxplot(geom=&quot;errorbar&quot;, width=.5)+&#xA;  geom_boxplot(aes(fill=variable))+&#xA;  theme_bw()+&#xA;  theme(axis.title.x=element_blank(), axis.title.y=element_blank())+&#xA;  stat_summary(fun.y=median, colour=&quot;red&quot;, geom=&quot;line&quot;, aes(group = 1))+&#xA;  geom_jitter(position = position_jitter(0.2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have this box&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have added this line to my code where I got &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;geom_text(data = df_melt, aes(x = variable, y = value, label = gene), size = 5) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;[![enter image description here][2]][2]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to show the name of genes in box plot but I don't how, cam you help?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-02-10T10:47:13.820" LastActivityDate="2020-02-10T10:47:13.820" Title="Showing name of genes in boxplot" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11357" PostTypeId="2" ParentId="11353" CreationDate="2020-02-09T19:03:37.820" Score="17" Body="&lt;p&gt;Most sequencing experiments, be it Illumina-based next-generation-sequencing or Sanger sequencing uses DNA as template, not RNA. Even if this virus is RNA-based it would be reverse-transcribed prior to any sequencing experiment. Therefore the output is DNA and this is what NCBI provides here.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2020-02-09T19:10:59.477" LastActivityDate="2020-02-09T19:10:59.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11358" PostTypeId="2" ParentId="11353" CreationDate="2020-02-09T19:10:08.697" Score="49" Body="&lt;p&gt;That is the correct sequence for 2019-nCov. Coronavirus is of course an RNA virus and in fact, to my knowledge, every RNA virus in Genbank is present as cDNA (AGCT, i.e. thydmine) and not RNA (AGCU, i.e. uracil).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason is simple, we never sequence directly from RNA because RNA is too unstable and easily degraded by RNase. Instead the genome is reverse transcribed, either by targeted reverse transcription or random amplification and thus converted to cDNA. cDNA is stable and is essentially reverse transcribed RNA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cDNA is either sequenced directly or further amplified by PCR and then sequenced. Hence the sequence we observe is the cDNA rather than RNA, thus we observe thymine rather than uracil and that is how it is reported. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-02-09T19:15:37.033" LastActivityDate="2020-02-09T19:15:37.033" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11359" PostTypeId="2" ParentId="11356" CreationDate="2020-02-09T19:13:38.297" Score="1" Body="&lt;p&gt;In addition, to have several errors in your code for plotting (for example, you are using &lt;code&gt;df&lt;/code&gt; for plotting and then &lt;code&gt;df_melt&lt;/code&gt; for &lt;code&gt;geom_text&lt;/code&gt; but you didnot mention what is it). Based on your graph, it seems that you have issues with the melting part. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, these are quite secondary issues that are probably be fixed with the code below. with your example, there is one trick because you are using &lt;code&gt;jitter&lt;/code&gt; to plot your points (that is not addressed by the link that @ATpoint provided in comments). This will randomly position the point at the right y value but with different x position in order to not overlap points. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, when you will add your text label, the labeling won't fit to the position of points. But you can make the jitter reproducible by adding a &lt;code&gt;seed&lt;/code&gt; (see this github discussion: &lt;a href=&quot;https://github.com/tidyverse/ggplot2/pull/1996&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/tidyverse/ggplot2/pull/1996&lt;/a&gt;). However, &lt;code&gt;geom_jitter&lt;/code&gt; function does not have the argument &lt;code&gt;seed&lt;/code&gt;, so you can use &lt;code&gt;geom_point&lt;/code&gt; function and the argument &lt;code&gt;position_jitter&lt;/code&gt; instead.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, using the few genes that you provided as an example. I have to reshape it (a step you probably did but you did not provide in your question), here I'm using &lt;code&gt;pivot_longer&lt;/code&gt; function from &lt;code&gt;tidyr&lt;/code&gt; package:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(dplyr)&#xA;library(tidyr)&#xA;DF &amp;lt;- df %&amp;gt;% pivot_longer(., -gene, names_to = &quot;variable&quot;, values_to = &quot;value&quot;)&#xA;&#xA;# A tibble: 16 x 3&#xA;   gene   variable       value&#xA;   &amp;lt;chr&amp;gt;  &amp;lt;chr&amp;gt;          &amp;lt;dbl&amp;gt;&#xA; 1 CCL28  Res_immune      2.03&#xA; 2 CCL28  Non_res_immune  2.02&#xA; 3 CCL28  Res_OBP         1.92&#xA; 4 CCL28  Non_res_OBP     1.95&#xA; 5 CXCL2  Res_immune      2.20&#xA; 6 CXCL2  Non_res_immune  2.28&#xA; 7 CXCL2  Res_OBP         2.18&#xA; 8 CXCL2  Non_res_OBP     2.24&#xA; 9 CXCR4  Res_immune      2.12&#xA;10 CXCR4  Non_res_immune  2.15&#xA;11 CXCR4  Res_OBP         1.95&#xA;12 CXCR4  Non_res_OBP     1.98&#xA;13 IFNGR1 Res_immune      2.31&#xA;14 IFNGR1 Non_res_immune  2.32&#xA;15 IFNGR1 Res_OBP         2.22&#xA;16 IFNGR1 Non_res_OBP     2.26&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then, for the plotting part, you can use &lt;code&gt;geom_text_repel&lt;/code&gt; function of the &lt;code&gt;ggrepel&lt;/code&gt; package to add labeling without overlapping your points:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(ggplot2)&#xA;library(ggrepel)&#xA;ggplot(DF, aes(x = variable, y = value, fill = variable, label = gene))+&#xA;  geom_boxplot(alpha = 0.2)+&#xA;  geom_point(position = position_jitter(width = 0.2, seed = 2), show.legend = FALSE)+&#xA;  geom_text_repel(position = position_jitter(0.2, seed = 2), show.legend = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KW09u.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KW09u.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Warning&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a long list of genes, it will be pointless to show all of them on the boxplot. It will makes things really illisible. If you want to show some particular genes of interest, make a subset of the original dataframe. A lot of posts on SO and here have addressed this subsetting.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reproducible data example&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;structure(list(gene = c(&quot;CCL28&quot;, &quot;CXCL2&quot;, &quot;CXCR4&quot;, &quot;IFNGR1&quot;), &#xA;    Res_immune = c(2.030674, 2.204847, 2.121186, 2.31139), Non_res_immune = c(2.023333, &#xA;    2.275296, 2.147181, 2.324111), Res_OBP = c(1.921508, 2.17579, &#xA;    1.950065, 2.220727), Non_res_OBP = c(1.952007, 2.241554, &#xA;    1.982639, 2.255556)), row.names = c(NA, -4L), class = c(&quot;data.table&quot;, &#xA;&quot;data.frame&quot;), .internal.selfref = &amp;lt;pointer: 0x563d91b65350&amp;gt;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6437" LastEditorUserId="6437" LastEditDate="2020-02-09T19:37:56.163" LastActivityDate="2020-02-09T19:37:56.163" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11362" PostTypeId="1" AcceptedAnswerId="11368" CreationDate="2020-02-10T13:46:52.977" Score="4" ViewCount="136" Body="&lt;p&gt;I have written a bioinformatics package in R that I want to publish in a bioinformatics Journal. Presently, I am maintaining a local repo of that package and I want to put in the Bioconductor repository (after publication or at least after submission to the journal), is it possible? or I must submit it to Bioconductor first?&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="7177" LastActivityDate="2020-02-10T18:14:54.630" Title="Can I submit a R package to Bioconductor or CRAN if I have already published it a journal?" Tags="|bioconductor|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11363" PostTypeId="2" ParentId="11351" CreationDate="2020-02-10T14:03:09.313" Score="3" Body="&lt;p&gt;That &lt;a href=&quot;http://www.rcsb.org/structure/1A9L&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB entry&lt;/a&gt; is an RNA molecule, so there is no protein sequence to extract.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it extracted the RNA sequence by default here you can see there could be an ambiguity as to whether it was an RNA sequence or a protein sequence, as the alphabets share characters.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2020-02-10T14:03:09.313" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11364" PostTypeId="2" ParentId="11297" CreationDate="2020-02-10T16:04:46.210" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://csb5.github.io/lofreq/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LoFreq&lt;/a&gt; is a variant calling tool that specifically leverages per-base quality scores. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For long read technologies (PacBio SMRT, ONT), I think quality score usage is mostly limited to qc steps as well, such as filtering out poor reads either by mean quality score of the whole read (as in ONT's basecalling softwares and &lt;a href=&quot;https://github.com/rrwick/Filtlong&quot; rel=&quot;nofollow noreferrer&quot;&gt;Filtlong&lt;/a&gt;) or by the lowest quality score within a window in a read (&lt;a href=&quot;https://github.com/rrwick/Filtlong&quot; rel=&quot;nofollow noreferrer&quot;&gt;Filtlong&lt;/a&gt;).  I think most long read mappers ignore these values (for example minimap2 and ngmlr do and can take fasta for read inputs).&lt;/p&gt;&#xA;" OwnerUserId="3908" LastActivityDate="2020-02-10T16:04:46.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11365" PostTypeId="2" ParentId="11353" CreationDate="2020-02-10T16:22:43.560" Score="5" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If this were [cDNA], the end of the true mRNA sequence would be ...UCUUACUGUUUUUUUUUUUU, or a &quot;poly(U)&quot; tail.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;A cDNA sequence, maybe confusingly, refers to the &lt;em&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Coding_strand&quot; rel=&quot;noreferrer&quot;&gt;coding strand&lt;/a&gt;&lt;/em&gt; of the cDNA (despite being called “complementary”). So while cDNA is the result of reverse transcribing RNA into DNA, by convention it has the same strandedness as the original RNA. That’s why what you’re seeing is read in 5′→3′ direction and contains a visible poly(A) tail. Having a single conventional reading direction for all archived sequences vastly simplifies data handling, and reduces errors.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, since cDNA is double-stranded, there is no a priori reason why a computer-stored cDNA sequence should refer to the template strand (i.e. the opposite strand, which is synthesised from the RNA during reverse transcription).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Complementary_DNA#Synthesis&quot; rel=&quot;noreferrer&quot;&gt;The whole (simplified) synthesis process of cDNA is as follows&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A primer hybridises to the template RNA molecule.&lt;/li&gt;&#xA;&lt;li&gt;The RNA template is reverse transcribed into DNA using reverse transcriptase.&lt;/li&gt;&#xA;&lt;li&gt;The RNA template is removed.&lt;/li&gt;&#xA;&lt;li&gt;A complementary strand is transcribed along the (currently) single-stranded cDNA, resulting in a double-stranded cDNA product.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="29" LastActivityDate="2020-02-10T16:22:43.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11366" PostTypeId="1" AcceptedAnswerId="11367" CreationDate="2020-02-10T16:59:40.240" Score="0" ViewCount="67" Body="&lt;p&gt;FlyBase uses systematic naming of genes, transcripts and proteins in a format &lt;code&gt;FBtypeNUMBER&lt;/code&gt;. However, the numbers are not the same for genes and their products. For example gene &lt;code&gt;FBgn0259101&lt;/code&gt; transcribes to &lt;code&gt;FBtr0299513&lt;/code&gt; and translates to &lt;code&gt;FBpp0288787&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;On this &lt;a href=&quot;https://flybase.org/convert/id&quot; rel=&quot;nofollow noreferrer&quot;&gt;webpage&lt;/a&gt; users can conver individual IDs, but what If I need to convert practically whole transcriptome? Where can I get the full translation table?&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2020-02-10T16:59:40.240" Title="how to get FlyBase ID conversion table" Tags="|identifiers|conversion|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11367" PostTypeId="2" ParentId="11366" CreationDate="2020-02-10T16:59:40.240" Score="0" Body="&lt;p&gt;Indeed the web interface does not seem to link the full translation table, but it is available on their &lt;a href=&quot;ftp://ftp.flybase.net&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp server&lt;/a&gt; at &lt;code&gt;releases -&amp;gt; release_ID -&amp;gt; precomputed_files -&amp;gt; genes -&amp;gt; fbgn_fbtr_fbpp_fb_2019_06.tsv.gz&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, &lt;a href=&quot;ftp://ftp.flybase.net/releases/FB2019_06/precomputed_files/genes/fbgn_fbtr_fbpp_fb_2019_06.tsv.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; is the latest (&lt;code&gt;FB2019_06&lt;/code&gt;) translation table available. &lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2020-02-10T16:59:40.240" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11368" PostTypeId="2" ParentId="11362" CreationDate="2020-02-10T17:10:42.920" Score="5" Body="&lt;p&gt;You can submit a package to a repo after the publication, but I would say that you really should do it before.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.bioconductor.org/developers/package-submission/&quot; rel=&quot;noreferrer&quot;&gt;Bioconductor&lt;/a&gt; accept only packages that are not published on CRAN, however, academic publications are fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Conversely, Bioinformatics does not enforce any particular platform for sharing the code, but you have to make it available somehow (e.g. GitHub repo, university webpage, bioconductor). From the &lt;a href=&quot;https://academic.oup.com/bioinformatics/pages/instructions_for_authors&quot; rel=&quot;noreferrer&quot;&gt;instructions for authors&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If the manuscript describes new software tools or the implementation&#xA;  of novel algorithms the software must be freely available to&#xA;  non-commercial users at the time of submission, and appropriate test&#xA;  data should be made available.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;However, you as a developer should make all the effort to make your software available and easily accessible. Also, sooner you make it available, sooner people start to cite you, which is a great advantage for you. Finally, you really need users to make sure that your package really works (i.e. debugging all the corner cases).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could not emphasise more, the correct order is 1. make it available 2. write a publication&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2020-02-10T18:14:54.630" LastActivityDate="2020-02-10T18:14:54.630" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11369" PostTypeId="2" ParentId="11353" CreationDate="2020-02-10T19:05:23.027" Score="3" Body="&lt;p&gt;It's not common to sequence directly from RNA because most sequencing platforms don't have that as an option. Nanopore sequencers do allow this, but I'm not aware yet of any 2019-nCov preprints involving nanopore RNA sequencing. I expect that will change in the next month or so.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Commercial kits exist; there are no insurmountable technical issues with it. Direct RNA sequencing can be done locally, on-site near the point of discovery without sample transfer or culture on a USB-powered device that fits in a pocket (RNA preparation takes &lt;a href=&quot;https://store.nanoporetech.com/catalog/product/view/id/297/s/direct-rna-sequencing-kit/category/28/&quot; rel=&quot;nofollow noreferrer&quot;&gt;about 2 hours&lt;/a&gt;). Flow cells that have potentially-infectious RNA inside them can be disposed as biohazard waste. However, the ease at which RNA can be quickly converted to more stable cDNA then amplified to create a much higher concentration DNA sample (which is quicker / more efficient to get results from) means that cDNA is generally preferred for sequencing unless the native RNA is needed (e.g. for looking at RNA base modifications that are destroyed when converting to cDNA).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is a paper on coronavirus direct RNA sequencing with nanopore &lt;a href=&quot;https://doi.org/10.1101/483693&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;; I would expect that 2019-nCoV would have a similar difficulty. The zika virus has an extremely low viral load in human blood, but has also been sequenced via direct RNA sequencing of [carefully] cultured cells (see &lt;a href=&quot;https://doi.org/10.4167/jbv.2019.49.3.115&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regardless of whether or not RNA sequencing has actually been carried out, most genetic data analysis programs will only work with A/C/G/T sequences, so it's conventional to replace any U parts of an RNA sequence with T for data storage. There's no loss of information by doing this, as T replaces all Us in the RNA sequence.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-02-10T23:06:46.277" LastActivityDate="2020-02-10T23:06:46.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11370" PostTypeId="1" AcceptedAnswerId="11372" CreationDate="2020-02-10T21:52:19.800" Score="0" ViewCount="80" Body="&lt;p&gt;I have this data frame&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;head(d)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# A tibble: 6 x 3&#xA;  gene  variable       value&#xA;1 CCT5  Res_immune     10.6 &#xA;2 CCT5  Non_res_immune 10.5 &#xA;3 CDK1  Res_immune      9.25&#xA;4 CXCL5 Non_res_immune  9.82&#xA;5 GBP1  Res_immune      9.04&#xA;6 GBP1  Non_res_immune  9.43&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(ggplot2)&#xA;library(ggrepel)&#xA;ggplot(d, aes(x = variable, y = value, fill = variable, label = gene))+&#xA;  geom_boxplot(alpha = 0.2)+&#xA;  geom_point(position = position_jitter(width = 0.2, seed = 2), show.legend = FALSE)+&#xA;  geom_text_repel(position = position_jitter(0.2, seed = 2), show.legend = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The genes in first column are involved in cytokine signaling pathway in two groups of patients&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my plot&#xA;&lt;a href=&quot;https://i.stack.imgur.com/2ciYI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2ciYI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have another data frame for genes in IL17 signaling in the sample patients like&lt;/p&gt;&#xA;&#xA;&lt;p&gt;head(d)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# A tibble: 6 x 3&#xA;  gene  variable       value&#xA;  &amp;lt;chr&amp;gt; &amp;lt;chr&amp;gt;          &amp;lt;dbl&amp;gt;&#xA;1 CDK6  Res_immune     17.6 &#xA;2 IL6  Non_res_immune 14.5 &#xA;3 CXCL8  Res_immune      1.25&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How I can put plots from both data in a grade like below where the name of pathways shown instead of DEL, INS, SNP and total like below picture?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/aRH0K.png&quot; alt=&quot;&quot;&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="6437" LastEditDate="2020-02-11T06:34:52.317" LastActivityDate="2020-02-11T06:34:52.317" Title="Griding ggplot object in r" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11371" PostTypeId="1" AcceptedAnswerId="11380" CreationDate="2020-02-10T21:57:50.637" Score="1" ViewCount="270" Body="&lt;p&gt;I am having a tough time calling LiftoverVcf in GATK. I have concatenated all of the Vcfs of interest and removed rows with problematic values but now when I run LiftoverVCF I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Intel; Inflater: Intel; Provider GCS is available; Picard version: Version:4.0.12.0&#xA;INFO 2020-02-10 14:36:03 LiftoverVcf Loading up the target reference genome.&#xA;INFO 2020-02-10 14:36:16 LiftoverVcf Lifting variants over and sorting (not yet writing the output file.)&#xA;&#xA;[Mon Feb 10 14:59:31 EST 2020] picard.vcf.LiftoverVcf done. Elapsed time: 23.50 minutes.&#xA;Runtime.totalMemory()=7414480896&#xA;To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp&#xA;java.lang.IllegalStateException: Key . found in VariantContext field INFO at 1:121387974 but this key isn't defined in the VCFHeader. We require all VCFs to have complete VCF headers by default.&#xA;at htsjdk.variant.vcf.VCFEncoder.fieldIsMissingFromHeaderError(VCFEncoder.java:202)&#xA;at htsjdk.variant.vcf.VCFEncoder.write(VCFEncoder.java:141)&#xA;at htsjdk.variant.variantcontext.writer.VCFWriter.add(VCFWriter.java:248)&#xA;at picard.vcf.LiftoverVcf.rejectVariant(LiftoverVcf.java:456)&#xA;at picard.vcf.LiftoverVcf.doWork(LiftoverVcf.java:357)&#xA;at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:295)&#xA;at org.broadinstitute.hellbender.cmdline.PicardCommandLineProgramExecutor.instanceMain(PicardCommandLineProgramExecutor.java:25)&#xA;at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)&#xA;at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)&#xA;at org.broadinstitute.hellbender.Main.main(Main.java:289)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Not sure why I am getting this error other than there may be a &lt;code&gt;.&lt;/code&gt; in one of the info fields. Here is what the first several rows of my VCF looks like minus the header fields:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM POS ID REF ALT QUAL FILTER INFO FORMAT AltaiNea&#xA;1 10001 . T . 51.05 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=6.32;MQ0=242 GT:DP:GQ:PL:A:C:G:T:IR 0/0:249:21.05:0,21,265:1,2:0,1:1,0:116,129:2&#xA;1 10002 . A . 36.01 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=3.31;MQ0=248 GT:DP:GQ:PL:A:C:G:T:IR 0/0:250:6.01:0,6,69:171,184:0,9:0,0:0,2:0&#xA;1 10003 . A . 36.01 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=3.31;MQ0=248 GT:DP:GQ:PL:A:C:G:T:IR 0/0:250:6.02:0,6,72:222,201:1,0:0,0:0,8:0&#xA;1 10004 . C . 39.01 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=3.61;MQ0=247 GT:DP:GQ:PL:A:C:G:T:IR 0/0:250:9.02:0,9,97:0,1:268,211:0,0:0,1:0&#xA;1 10005 . C . 41.99 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=4.31;MQ0=246 GT:DP:GQ:PL:A:C:G:T:IR 0/0:250:12:0,12,119:0,0:341,287:0,0:0,0:0&#xA;1 10006 . C . 36.01 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=3.31;MQ0=248 GT:DP:GQ:PL:A:C:G:T:IR 0/0:249:6.02:0,6,72:0,1:408,347:0,0:2,0:0&#xA;1 10007 . T . 39.01 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=4.21;MQ0=246 GT:DP:GQ:PL:A:C:G:T:IR 0/0:250:9.02:0,9,97:0,0:0,0:0,0:506,451:0&#xA;1 10008 . A . 33.01 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=4.05;MQ0=247 GT:DP:GQ:PL:A:C:G:T:IR 0/0:250:3.01:0,3,40:547,507:0,0:0,0:0,0:0&#xA;1 10009 . A . 36.01 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=3.31;MQ0=248 GT:DP:GQ:PL:A:C:G:T:IR 0/0:250:6.02:0,6,70:600,534:0,1:0,0:0,1:0&#xA;1 10010 . C . 42.01 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=4.72;MQ0=244 GT:DP:GQ:PL:A:C:G:T:IR 0/0:249:12.02:0,12,125:0,0:649,547:0,1:1,0:0&#xA;1 10011 . C . 51.02 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=5.12;MQ0=242 GT:DP:GQ:PL:A:C:G:T:IR 0/0:249:21.03:0,21,209:0,2:732,612:0,1:0,0:0&#xA;1 10012 . C . 44.99 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=3.94;MQ0=243 GT:DP:GQ:PL:A:C:G:T:IR 0/0:249:14.99:0,15,141:0,3:804,657:0,0:6,1:0&#xA;1 10013 . T . 57.04 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=7.20;MQ0=238 GT:DP:GQ:PL:A:C:G:T:IR 0/0:249:27.04:0,27,285:0,0:1,0:0,0:904,760:0&#xA;1 10015 . A . 60.04 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=7.19;MQ0=238 GT:DP:GQ:PL:A:C:G:T:IR 0/0:249:30.04:0,30,301:985,850:0,0:1,0:0,0:0&#xA;1 10016 . C . 72.03 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=7.99;MQ0=235 GT:DP:GQ:PL:A:C:G:T:IR 0/0:249:42.04:0,42,427:2,1:1024,852:0,0:1,0:0&#xA;1 10017 . C . 84.05 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=8.37;MQ0=232 GT:DP:GQ:PL:A:C:G:T:IR 0/0:250:54.06:0,54,534:0,2:1100,912:0,0:0,0:0&#xA;1 10018 . C . 108.02 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=8.87;MQ0=224 GT:DP:GQ:PL:A:C:G:T:IR 0/0:250:78.03:0,78,757:0,0:1167,978:0,1:2,0:0&#xA;1 10019 . T . 96.05 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=10.24;MQ0=219 GT:DP:GQ:PL:A:C:G:T:IR 0/0:250:66.05:0,66,646:1,0:1,0:1,0:1258,1090:0&#xA;1 10020 . A . 65.97 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=9.74;MQ0=221 GT:DP:GQ:PL:A:C:G:T:IR 0/0:250:35.97:0,36,331:1309,1150:0,0:1,1:0,0:0&#xA;1 10021 . A . 65.34 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=8.14;MQ0=228 GT:DP:GQ:PL:A:C:G:T:IR 0/0:250:35.35:0,35,489:1339,1179:3,0:0,0:1,0:0&#xA;1 10022 . C . 92.98 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=7.79;MQ0=229 GT:DP:GQ:PL:A:C:G:T:IR 0/0:250:62.99:0,63,590:0,1:1389,1188:0,1:0,0:0&#xA;1 10023 . C . 98.93 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=7.69;MQ0=227 GT:DP:GQ:PL:A:C:G:T:IR 0/0:250:68.94:0,69,636:2,2:1453,1242:0,0:1,2:0&#xA;1 10024 . C . 98.95 . AC=0;AF=0.00;AN=2;DP=250;DS;MQ=7.46;MQ0=227 GT:DP:GQ:PL:A:C:G:T:IR 0/0:250:68.96:0,69,620:1,3:1497,1302:1,0:3,2:0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a way for me to add &lt;code&gt;.&lt;/code&gt; into the VCF header as a tag for all of the appropriate fields so that I can bypass this problem? I know from reading online that the &lt;code&gt;.&lt;/code&gt; means that value is unknown. I would like to figure out how to instruct GATK to simply accept it regardless of the fact that it cannot be found in the VCF header.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, I am open to other ways I can liftover this genome to hg38. Below is the command I am using to do so using GATK.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;java -jar ~/work/progs/gatk-4.0.12.0/gatk-package-4.0.12.0-local.jar LiftoverVcf -I AltaiNea.hg19_1000g.concat.LowQualRemoved.vcf -O AltaiNea.hg38_1000g.concat.LowQualRemoved.vcf -C hg19ToHg38.over.chain -R hg38.fa --REJECT rejectedRecords.vcf&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3442" LastActivityDate="2020-03-12T21:01:05.330" Title="VCF's unknown ('.') values causing problems for GATK's Liftover Vcf" Tags="|vcf|vcftools|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11372" PostTypeId="2" ParentId="11370" CreationDate="2020-02-10T22:20:47.523" Score="4" Body="&lt;p&gt;What you are looking for is &lt;code&gt;facetting&lt;/code&gt;. Using this key word on any search engines you will find dozen of answers describing its use: &lt;a href=&quot;https://ggplot2.tidyverse.org/reference/facet_grid.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ggplot2.tidyverse.org/reference/facet_grid.html&lt;/a&gt; and &lt;a href=&quot;https://ggplot2.tidyverse.org/reference/facet_wrap.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ggplot2.tidyverse.org/reference/facet_wrap.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To prepare the faceting in &lt;code&gt;ggplot2&lt;/code&gt;, a possible way is to bind both dataframes together by first specifying which group they belong such as:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;cytokine&lt;span class=&quot;math-container&quot;&gt;$Group = &quot;Cytokine&quot;&#xA;il17$&lt;/span&gt;Group = &quot;IL17&quot;&#xA;DF &amp;lt;- rbind(cytokine,il17)&#xA;DF&#xA;&#xA;    gene       variable value    Group&#xA;1:  CCT5     Res_immune 10.60 Cytokine&#xA;2:  CCT5 Non_res_immune 10.50 Cytokine&#xA;3:  CDK1     Res_immune  9.25 Cytokine&#xA;4: CXCL5 Non_res_immune  9.82 Cytokine&#xA;5:  GBP1     Res_immune  9.04 Cytokine&#xA;6:  GBP1 Non_res_immune  9.43 Cytokine&#xA;7:  CDK6     Res_immune 17.60     IL17&#xA;8:   IL6 Non_res_immune 14.50     IL17&#xA;9: CXCL8     Res_immune  1.25     IL17&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, use &lt;code&gt;facet_grid&lt;/code&gt; function of &lt;code&gt;ggplot2&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(ggplot2)&#xA;library(ggrepel)&#xA;ggplot(DF, aes(x = variable, y = value, fill = variable, label = gene))+&#xA;  geom_boxplot(alpha = 0.2)+&#xA;  geom_point(position = position_jitter(width = 0.2, seed = 2), show.legend = FALSE)+&#xA;  geom_text_repel(position = position_jitter(0.2, seed = 2), show.legend = FALSE)+&#xA;  facet_grid(Group~., scales = &quot;free&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/P9c3f.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/P9c3f.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you can use &lt;code&gt;facet_grid&lt;/code&gt; (or &lt;code&gt;facet_wrap&lt;/code&gt;) to have your plot in a single row:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(ggplot2)&#xA;library(ggrepel)&#xA;ggplot(DF, aes(x = variable, y = value, fill = variable, label = gene))+&#xA;  geom_boxplot(alpha = 0.2)+&#xA;  geom_point(position = position_jitter(width = 0.2, seed = 2), show.legend = FALSE)+&#xA;  geom_text_repel(position = position_jitter(0.2, seed = 2), show.legend = FALSE)+&#xA;  facet_wrap(.~Group, scales = &quot;free&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AohiU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AohiU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Reproducible data example&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;structure(list(gene = c(&quot;CCT5&quot;, &quot;CCT5&quot;, &quot;CDK1&quot;, &quot;CXCL5&quot;, &quot;GBP1&quot;, &#xA;&quot;GBP1&quot;), variable = c(&quot;Res_immune&quot;, &quot;Non_res_immune&quot;, &quot;Res_immune&quot;, &#xA;&quot;Non_res_immune&quot;, &quot;Res_immune&quot;, &quot;Non_res_immune&quot;), value = c(10.6, &#xA;10.5, 9.25, 9.82, 9.04, 9.43), Group = c(&quot;Cytokine&quot;, &quot;Cytokine&quot;, &#xA;&quot;Cytokine&quot;, &quot;Cytokine&quot;, &quot;Cytokine&quot;, &quot;Cytokine&quot;)), row.names = c(NA, &#xA;-6L), class = c(&quot;data.table&quot;, &quot;data.frame&quot;), .internal.selfref = &amp;lt;pointer: 0x55bfc0db9350&amp;gt;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;structure(list(gene = c(&quot;CDK6&quot;, &quot;IL6&quot;, &quot;CXCL8&quot;), variable = c(&quot;Res_immune&quot;, &#xA;&quot;Non_res_immune&quot;, &quot;Res_immune&quot;), value = c(17.6, 14.5, 1.25), &#xA;    Group = c(&quot;IL17&quot;, &quot;IL17&quot;, &quot;IL17&quot;)), row.names = c(NA, -3L&#xA;), class = c(&quot;data.table&quot;, &quot;data.frame&quot;), .internal.selfref = &amp;lt;pointer: 0x55bfc0db9350&amp;gt;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6437" LastEditorUserId="6437" LastEditDate="2020-02-10T22:28:07.117" LastActivityDate="2020-02-10T22:28:07.117" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11373" PostTypeId="1" AcceptedAnswerId="11385" CreationDate="2020-02-11T09:57:11.437" Score="1" ViewCount="67" Body="&lt;p&gt;&lt;em&gt;Note: this question can also be found on &lt;a href=&quot;https://www.biostars.org/p/421485/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to perform a stress test in a GWAS tool and the duty demands a dataset (plink format) having 100 thousand samples, having 40 million SNPs in a proportion of case:control=30:70. I'm performing the command:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;plink1.9 --simulate ds1.sim --make-bed --out ds1 --simulate-ncases 30000 --simulate-ncontrols 70000&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But after about 3 hours execution, I'm facing &lt;code&gt;Error: File write failure.&lt;/code&gt;, even though my environment has 256T of disk. (It's a container with a S3 (Object store) mounted through s3fs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if there's a clever way to do that? Perhaps breaking it into small chunks and merging up later somehow? It would prevent me to be waiting hours till an error happens and I loose either time and the sample generated so far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="6464" LastEditorUserId="73" LastEditDate="2020-02-12T10:13:00.927" LastActivityDate="2020-02-12T10:57:33.907" Title="Question: How to simulate 100k samples having 40 million SNPs in a proportion of case:control=30:70?" Tags="|gwas|plink|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11375" PostTypeId="2" ParentId="11351" CreationDate="2020-02-11T13:12:11.250" Score="2" Body="&lt;p&gt;If you are using a conda version of python you can use Pymol module for this, which is actually more robust in some aspects than &lt;code&gt;bio.PDB&lt;/code&gt;, although errors may cause segmentation faults. The following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pymol2&#xA;## paralellisable version.&#xA;with pymol2.PyMOL() as pymol:&#xA;    pymol.cmd.fetch('1a9l')&#xA;    print(pymol.cmd.get_fastastr('polymer.nucleic'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Gives me:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1a9l_A&#xA;GGGUGACUCCAGAGGUCGAGAGACCGGAGAUAUCACCC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Alternatively, you can use PDBe's API.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;&#xA;response = requests.get('https://www.ebi.ac.uk/pdbe/api/pdb/entry/molecules/1a9l').json()['1a9l']&#xA;&#xA;for entity in response:&#xA;    if entity['molecule_type'] == 'polyribonucleotide':&#xA;        print(entity['sequence'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Do note that missing 'residues' (residues, small molecules and nucleotides) will cause issues. PDB files do not have missing 'residues', while mmCIF files have a dictionary that does. The API will give you the latter.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-02-11T13:12:11.250" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="11376" PostTypeId="1" CreationDate="2020-02-11T14:15:13.523" Score="1" ViewCount="36" Body="&lt;p&gt;What is the highest data compression ratio achieved on a population of genomes measured to include the compressing algorithm itself?  The size of the compressed database approximates its &lt;a href=&quot;http://www.scholarpedia.org/article/Algorithmic_information_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;algorithmic information&lt;/a&gt; content*.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've seen &lt;a href=&quot;http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1021.1196&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;a claim of on the order of 1000:1 compression&lt;/a&gt; but it appears dependent on an external &quot;reference&quot; database.  In my opinion, a reference data must be included in the size of the compressing algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*The Universal Turing Machine notion of Minimum Description Length is also called the Kolmogorov Complexity or Algorithmic Information measure of the data.  I'm using the term &quot;Algorithmic Information&quot; as the most concise and accessible descriptive phrase for this measure.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The relevance of this measure is that according to the closely related theory of &lt;a href=&quot;http://www.scholarpedia.org/article/Algorithmic_probability&quot; rel=&quot;nofollow noreferrer&quot;&gt;algorithmic probability&lt;/a&gt;, the algorithm that maximally compresses a dataset also provides the most optimal data-driven predictions.&lt;/p&gt;&#xA;" OwnerUserId="7194" LastEditorUserId="3848" LastEditDate="2020-04-30T16:01:51.020" LastActivityDate="2020-04-30T16:01:51.020" Title="Algorithmic Information Measurement of Human Genome Databases?" Tags="|phylogeny|phylogenetics|data-management|correlation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11377" PostTypeId="2" ParentId="4977" CreationDate="2020-02-11T15:45:13.437" Score="0" Body="&lt;p&gt;While I agree with not submitting snakemake to the cluster, it's not a great solution to make your pilot program run interactively - automation via batch script should not be dependent on an open shell. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How about allocating a whole node with many cores and working within the GNU parallel abilities of snakemake.&lt;/p&gt;&#xA;" OwnerUserId="7196" LastActivityDate="2020-02-11T15:45:13.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11378" PostTypeId="1" AcceptedAnswerId="11379" CreationDate="2020-02-11T17:53:25.040" Score="2" ViewCount="317" Body="&lt;p&gt;I tried to find a homolog that has a PDB structure available so I can use this PDB file for comparative modeling. I ran a BLAST search but none of the search results seemed to have a known structure. Is there any way to find the results I want? &lt;/p&gt;&#xA;" OwnerUserId="7197" LastEditorUserId="4622" LastEditDate="2020-02-14T08:00:21.950" LastActivityDate="2020-02-14T08:00:21.950" Title="How to find a homolog that has a PDB structure available" Tags="|blast|pdb|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11379" PostTypeId="2" ParentId="11378" CreationDate="2020-02-11T19:20:45.000" Score="1" Body="&lt;p&gt;If you are using NCBI Blast and blastp against the pdb database gives you nothing, you can do a psiblast against the normal refseq database, do a few rounds, save the PSSM matrix and submit a new job with the PSSM matrix but against the PDB database.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might need to limit the species of you get lots of too close homologues —say blasting E. coli protein for which you'd want to exclude Enterobacteriales.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-02-11T19:20:45.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11380" PostTypeId="2" ParentId="11371" CreationDate="2020-02-11T20:59:47.507" Score="1" Body="&lt;p&gt;My problem was solved using &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360037226512-LiftoverVcf-Picard-#--ALLOW_MISSING_FIELDS_IN_HEADER&quot; rel=&quot;nofollow noreferrer&quot;&gt;--ALLOW-MISSING-FIELDS-IN-HEADER&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3442" LastActivityDate="2020-02-11T20:59:47.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11381" PostTypeId="2" ParentId="11354" CreationDate="2020-02-11T21:53:08.760" Score="1" Body="&lt;p&gt;If you reran the command with the correct settings, just leave it at that.  (It is not at all clear to me that strandedness rf is correct) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want people to tell you if you ran the commands right, you need to put down what commands you used.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-02-11T21:53:08.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11382" PostTypeId="2" ParentId="11378" CreationDate="2020-02-11T22:43:59.427" Score="1" Body="&lt;p&gt;To expand the first answer by @MatteoFurla: re-run the P-blast and under database option, which is set by default to non-redundant database (nr) select Protein databank (PDB), the third option from the bottom of the menu. All the Blast results will be from PDB and thus all results will have associated structures. Also note that the Genbank codes will be PDB codes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have zero hits then consider PSI-Blast, how I suspect you are running P-Blast via the nr database.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-02-11T22:51:23.033" LastActivityDate="2020-02-11T22:51:23.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11383" PostTypeId="2" ParentId="11328" CreationDate="2020-02-12T00:35:39.963" Score="2" Body="&lt;h2&gt;Oligopeptide&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The lipophilicity of an oligopeptide or a small molecule can be calculated with RDKit, the classic compChem python toolkit. The measure is &lt;a href=&quot;https://en.wikipedia.org/wiki/Partition_coefficient&quot; rel=&quot;nofollow noreferrer&quot;&gt;logP or partition coefficient&lt;/a&gt;, namely the magnitude of the ratio of the concentration in octanol over than in water, so the higher it is, the more it partitions in the octanol phase. However, with a compound of several dalton it may be good to see what the contribution of each atom to this score are. This measure is the Crippen contribution to logP, which can be handily plotted on the molecule. Note that these are may be at first glance look similar to Gastaiger partial changes, but are not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Okay, first the imports&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from rdkit import Chem&#xA;from rdkit.Chem import AllChem, Descriptors&#xA;from rdkit.Chem.Draw import SimilarityMaps&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now let's make the molecules. Let's start with something we can simply use a SMILES string of the web: &lt;a href=&quot;https://en.wikipedia.org/wiki/Bortezomib&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bortezomib&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;name = 'Bortezomib'&#xA;mol = Chem.MolFromSmiles('O=C(N[C@H](C(=O)N[C@H](B(O)O)CC(C)C)Cc1ccccc1)c2nccnc2')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now we can do all the relevant calculations on the &lt;code&gt;mol&lt;/code&gt; object:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;print('logP', Descriptors.MolLogP(mol))&#xA;AllChem.Compute2DCoords(mol)&#xA;contribs = Chem.rdMolDescriptors._CalcCrippenContribs(mol)&#xA;fig = SimilarityMaps.GetSimilarityMapFromWeights(mol, [x for x,y in contribs], colorMap='BuPu', contourLines=10)&#xA;fig.savefig(name+'_crippen.png', bbox_inches='tight')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;LogP=0.3&#xA;&lt;a href=&quot;https://i.stack.imgur.com/M6qEH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/M6qEH.png&quot; alt=&quot;Bortezomib&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;colorMap='BuPu'&lt;/code&gt; is the matplotlib colorscheme. I have no idea which work best, I have spent way too much time trying and I do not know. But with BuPu, white is hydrophilic, purple is hydrophobic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Say the oligopeptide does not have a SMILES or we don't want to do it that way. Let's take &lt;a href=&quot;https://en.wikipedia.org/wiki/Casomorphin&quot; rel=&quot;nofollow noreferrer&quot;&gt;β-Casomorphins 1–3&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;name = 'β-Casomorphins 1–3'&#xA;mol = Chem.MolFromFASTA('YPF')&#xA;mol = AllChem.ReplaceSubstructs(mol,&#xA;                                Chem.MolFromSmiles('N'),&#xA;                                Chem.MolFromSmiles('[NH][OH]'),&#xA;                                replacementConnectionPoint=0)[0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The result with the last block from before gives:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;logP=1&#xA;&lt;a href=&quot;https://i.stack.imgur.com/1JHCC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1JHCC.png&quot; alt=&quot;casomorphin&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Proteins&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;With protein, it is a lot harder. Membrane affinity of protein is dependent on their structure. In PyMOL you can use the Adaptive Poisson-Boltzmann Solver (APBS) to show the surface by charge. For transmembrane protein there is clear ring of hydrophobicity —CHARMM-GUI membrane builder and a variety of tools for example can guess where it is. For a membrane associated protein only one side will be hydrophobic with a tell-tell charged ring around it (e.g. DHR1 domain) and there are no hard and tested methods for that.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-03-03T21:15:15.950" LastActivityDate="2020-03-03T21:15:15.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11384" PostTypeId="1" AcceptedAnswerId="11387" CreationDate="2020-02-12T10:25:28.073" Score="0" ViewCount="63" Body="&lt;p&gt;Good morning, &#xA;I'm aligning with blast some species, against all nt database to analyse the vegetables. However, in some cases we have species of animals mixed with vegetables and want only to detect the vegetable species. Normally, we would only expect to detect the vegetables only. But this is normal since we are using the all nt. So now, could anyone point me out, the way to detect only vegetables? Thanks for the help.&lt;/p&gt;&#xA;" OwnerUserId="2686" LastEditorUserId="2686" LastEditDate="2020-02-12T15:53:12.663" LastActivityDate="2020-02-12T15:53:12.663" Title="Vegetable databases usage" Tags="|database|nt|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11385" PostTypeId="2" ParentId="11373" CreationDate="2020-02-12T10:57:33.907" Score="1" Body="&lt;p&gt;First comment: you should be using real data, rather than simulated data, as we are not very good at modelling genetic variation and recombination at a genome-wide scale. A way that this can be done with GCTA is mentioned in the &lt;a href=&quot;https://www.cog-genomics.org/plink/1.9/input#random&quot; rel=&quot;nofollow noreferrer&quot;&gt;PLINK manual&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'll do some calculations to estimate the size of data you need to store:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assume that this plink-formatted file is in tped format (one row per SNP), and the entire genome is diploid. That means that a single SNP/individual pair will take up three bytes. You want 30,000 cases and 70,000 controls, so each line from each sample will take up 300kB (plus a bit of change for the header). With 1 thousand SNPs, that's 300MB per sample; with 1 million SNPs, that's 300GB per sample; with 10 million SNPs, that's 3TB per sample; with 40 million SNPs, that's 12 TB for one sample.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Reading the plink manual, I see that the BED format is a two-bit compression for each SNP/individual pair, so one byte per 4 individuals, or about 12x compression (i.e. 1TB for one sample).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Okay, that's a lot. It's well within a 256TB storage limit, but might hit file size limits depending on the file system.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is a data generation issue with PLINK, it might be worth trying (as you suggest) splitting the input for simulation then merging afterwards. My guess is that doing the simulation on fewer SNPs (rather than fewer individuals) would work better. Plink has a &lt;code&gt;--bmerge&lt;/code&gt; &lt;a href=&quot;https://www.cog-genomics.org/plink/1.9/data#merge&quot; rel=&quot;nofollow noreferrer&quot;&gt;option&lt;/a&gt; for merging the compressed binary file formats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I usually approach problems like these by working my way up from something that I know works. Try first merging two datasets of 1000 SNPs. If that works, try larger datasets. Rinse and repeat until the computer crashes, you get bored, or you reach your goal.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2020-02-12T10:57:33.907" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11386" PostTypeId="1" CreationDate="2020-02-12T11:03:18.040" Score="3" ViewCount="913" Body="&lt;p&gt;I have  very large sizes tab-delimited .vcf files and want to match these two / or 3 files based on their position and print to a new .csv file  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;File structures:&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;File_1: tab-delimited file (.vcf) and its as column names as follows&#xA;&lt;code&gt;&#xA;(line number 3439) #CHROM     POS     ID       REF      ALT      QUAL    FILTER &#xA; INFO&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;File 2: same as file_1 column names&lt;/strong&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;(line number 3407) #CHROM     POS     ID      REF     ALT     QUAL    FILTER &#xA; INFO  FORMAT  SAMPLE_1  SAMPLE_2 .....&lt;/code&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In file_2, column 7 (INFO) contain many substrings like&#xA;&lt;code&gt;AC=46,2;AF=0.958,0.042;AN=48;DP=269;ExcessHet=3.0103;FS=0.000;InbreedingCoeff=0.5411;MLEAC=92,4;MLEAF=1.00,0.083;QD=25.36;SOR=2.488  from these strings , have to print  only information of AC=, AF=, AN=, DP=&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Desired output files to generate:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;position matched in two of files: common_position_matched.csv  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;if I have more than 3 files also, the output should be one file and important thing, if a position column (1-POS) only matched only in 2 files and 1st file line should be NA NA NA   &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;file_1  CHROM  POS    REF  ALT     file_2 #CHROM  POS      REF  ALT  INFO&#xA;&#xA;1       22    10511521  C     T         1    22  10511521   C    T   AC=46,2 AF=0.958,0.042 AN=48 DP=269  &#xA;2       22   10510544   G     A         2    22  10510544   G    A   AC=49,2 AF=0.958,0.042 AN=89 DP=536  &#xA;3       22   10515068  AGAT,T AGAT,AT   3    22  10515068   AAA  AAAGG,A,GAA AC=100  AF=0.958,0.042 AN=62 DP=123  &#xA;4       22   10515118  A G,   TAA       4    22  10515118   AG,  TAA AC=32   AF=0.958,0.042 AN=45 DP=500  &#xA;5       22   10515118  AAAG   A         5    22  10515118   AATG A   AC=50   AF=0.958,0.042 AN=49 DP=129&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;note:&lt;/strong&gt; while doing matching, not removing the duplicates, because in the same position there may be an addition or sometimes it may be deletion.      &lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;unique position of each file, in tab-delimited  &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;output: File1_unique.csv and File2_unique.csv etc.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;so far was able to read the file and match them according to position and print the output, but was not able to write efficient code   &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;df1 = pd.read_csv(&quot;File1_3.vcf&quot;,sep='\t',usecols = ['POS']) ## Reading file1&#xA;df2 = pd.read_csv(&quot;file2_3.vcf&quot;, sep=&quot;\t&quot;, usecols = ['POS']) ## Reading file2&#xA;df3 = pd.concat([df1,df2], sort=True) ## Combining both the dataframes&#xA;df4 = df3.drop_duplicates(keep=False) ## Dropping the duplicates (intersect)&#xA;df4.to_csv(&quot;c3-UniquePosition_of_bothData.csv&quot;, sep=&quot;\t&quot;, index=False, header=True) ## Writing the unique to both&#xA;df1_Uni_file1_c3Posi = pd.merge(df4, df1, on='POS', how='inner') ## Identifying the unique position of File1 &#xA;df2_Uni_File2_c3Posi = pd.merge(df4, df2, on='POS', how='inner') ## Identifying the unique position of File2&#xA;df_File1_File2_common_c3Posi = pd.merge(df1, df2, on='POS', how='inner') # Identifying the common chr-position of File1 and File2```  &#xA;&#xA;Program 2: (giving original file without editting)&#xA;import pandas as pd&#xA;df1= pd.read_csv(&quot;File1_22.vcf.gz&quot;, sep=&quot;\t&quot;, skiprows=3438, usecols = [0,1,2,3,4])&#xA;df2 = pd.read_csv(&quot;File2_22.vcf.gz&quot;, sep=&quot;\t&quot;, skiprows=3406, usecols = [0,1,3,4,7])&#xA;#writing the output files&#xA;#df1.to_csv(&quot;File1_c22.csv&quot;)&#xA;#df2.to_csv(&quot;File2_c22.csv&quot;)&#xA;#mergeing&#xA;df3 = pd.merge(df1, df2, on='POS', how='inner', sort=True)&#xA;df3.to_csv(&quot;common_position.csv&quot;, sep=&quot;,&quot;, index=False, header=True)&#xA;#df3 = pd.concat([df1,df2], axis=1).to_csv('check1.csv') # this command join multiple output to single output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Could any one give efficient python pandas script, to do this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks All&lt;/p&gt;&#xA;" OwnerUserId="1162" LastEditorUserId="1162" LastEditDate="2020-02-14T06:01:08.760" LastActivityDate="2024-04-03T12:00:50.097" Title="Finding common and unique data set by comparing two files based on their column and to split the columns multiple strings to print in output" Tags="|python|perl|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11387" PostTypeId="2" ParentId="11384" CreationDate="2020-02-12T11:22:34.040" Score="1" Body="&lt;p&gt;Using the blast command line tool, you can limit your search to a specific taxon using the &lt;code&gt;–taxids&lt;/code&gt; option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, in your case &lt;code&gt;-taxids 3193&lt;/code&gt; should limit your search to plants.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;If you are using the ncbi blast webinterface, you can restrict your search by entering &quot;plants&quot; into the Organism field in the &quot;Choose Search Set&quot; section&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MwD75.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MwD75.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="668" LastEditorUserId="668" LastEditDate="2020-02-12T12:44:31.533" LastActivityDate="2020-02-12T12:44:31.533" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11388" PostTypeId="1" AcceptedAnswerId="11391" CreationDate="2020-02-12T15:09:10.333" Score="1" ViewCount="705" Body="&lt;p&gt;kindly your comment highly appreciated &lt;/p&gt;&#xA;&#xA;&lt;p&gt;have more than 4k header sequences look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;LTR_retrotransposon100_Gypsy? &#xA;cgtgcaccccaatgttcatagcagcactatttacaatagccaagacatggaaacaaccta aatgtcctttgacaggtgactggataaagaagctgtggtatatttatacaatggaatgct actcagccataaaaaagaataagataatgccatttacagcaacatggatagacctggaga atgttattctaagtgaagtaagccagaaagaaaaagaaaaataccatatgctatcactta&#xA;&#xA;&amp;gt;LTR_retrotransposon10011_ERVK &#xA;taatttgaaaagatacatgcaacccaatgttcatagcagcattatttacaattgccaaga tatggaagcaatctaagtgtccatcaacagatgaatggagagagaagatgtggtatatat acaataaaatactattctgtcataaaaatgaataaaattctgccatttgcaacaacatgg atagacctggagggtattatgctatgtgaaataagtcagacagagaaagacatatactat&#xA;&#xA;&amp;gt;LTR_retrotransposon10011_ERVL-MaLR &#xA;    taatttgaaaagatacatgcaacccaatgttcatagcagcattatttacaattgccaaga tatggaagcaatctaagtgtccatcaacagatgaatggagagagaagatgtggtatatat acaataaaatactattctgtcataaaaatgaataaaattctgccatttgcaacaacatgg atagacctggagggtattatgctatgtgaaataagtcagacagagaaagacatatactat&#xA;&#xA;&amp;gt;LTR_retrotransposon100_Copia &#xA;    cgtgcaccccaatgttcatagcagcactatttacaatagccaagacatggaaacaaccta aatgtcctttgacaggtgactggataaagaagctgtggtatatttatacaatggaatgct actcagccataaaaaagaataagataatgccatttacagcaacatggatagacctggaga atgttattctaagtgaagtaagccagaaagaaaaagaaaaataccatatgctatcactta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to use awk or sed to remove everything between the underscore, including the underscore and replace it with / &#xA;plus add sequential RNA followed by #.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;and the target output could be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;RNA1#LTR/Gypsy?&#xA;ctcagcagcactatttacaatagccaagacatggaaacaacctaaatgtcttatcaatag atgactggataaaggagctgtggtatatctatacaatggaataccatcagccataaaaaa gaataaaatattgccatttgcagcaacatggatggacctggagattatcattctaaggga agtaagccagaaagagaaagaaaaataccatatgatatcacttatatgtggaggtaaaaa aaaaaaaaaagacacaaatcaatttatttgcaaaacatacaTGGActttcagacatagaa&#xA;&#xA;&amp;gt;RNA2#LTR/ERVL-MaLR&#xA;taatttgaaaagatacatgcaacccaatgttcatagcagcattatttacaattgccaaga tatggaagcaatctaagtgtccatcaacagatgaatggagagagaagatgtggtatatat acaataaaatactattctgtcataaaaatgaataaaattctgccatttgcaacaacatgg atagacctggagggtattatgctatgtgaaataagtcagacagagaaagacatatactat&#xA;&#xA;&#xA;&#xA;&amp;gt;RNA3#LTR/ERVK&#xA;    taatttgaaaagatacatgcaacccaatgttcatagcagcattatttacaattgccaaga tatggaagcaatctaagtgtccatcaacagatgaatggagagagaagatgtggtatatat acaataaaatactattctgtcataaaaatgaataaaattctgccatttgcaacaacatgg atagacctggagggtattatgctatgtgaaataagtcagacagagaaagacatatactat&#xA;&#xA;&amp;gt;RNA4#LTR/Copia &#xA;    cgtgcaccccaatgttcatagcagcactatttacaatagccaagacatggaaacaaccta aatgtcctttgacaggtgactggataaagaagctgtggtatatttatacaatggaatgct actcagccataaaaaagaataagataatgccatttacagcaacatggatagacctggaga atgttattctaagtgaagtaagccagaaagaaaaagaaaaataccatatgctatcactta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3332" LastEditorUserId="3848" LastEditDate="2020-02-20T15:33:23.277" LastActivityDate="2020-04-09T08:36:17.453" Title="Edit all the fasta headers using awk" Tags="|fasta|phylogenetics|filtering|data-preprocessing|awk|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11389" PostTypeId="2" ParentId="11388" CreationDate="2020-02-12T16:29:01.507" Score="2" Body="&lt;pre&gt;&lt;code&gt;sed -e 's:^\(.*\)_.*_\(.*\)$:\1/\2:' &amp;lt; input &amp;gt; output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3332" LastActivityDate="2020-02-12T16:29:01.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11390" PostTypeId="1" CreationDate="2020-02-12T17:10:48.107" Score="1" ViewCount="50" Body="&lt;p&gt;I need to use tophat2 to align some sequences, and I wish to use the docker container. These are large sequences so it will take a long time, plus I'm working on a university server so the chances of being kicked off the server are very high. How can I use nohup and &amp;amp; to run tophat2 through docker, if I need some user interaction?&lt;/p&gt;&#xA;" OwnerUserId="7209" LastActivityDate="2020-02-13T07:10:03.690" Title="Running tophat dockerfile in background" Tags="|sequence-alignment|bash|docker|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11391" PostTypeId="2" ParentId="11388" CreationDate="2020-02-12T17:27:27.617" Score="4" Body="&lt;p&gt;&lt;strong&gt;EDIT after OP's update:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Try this Perl one-liner:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl -pe 'BEGIN { &lt;span class=&quot;math-container&quot;&gt;$i = 1 } $&lt;/span&gt;i += s{&amp;gt;([^_]+)_.*_}{&amp;gt;RNA&lt;span class=&quot;math-container&quot;&gt;${i}#$&lt;/span&gt;{1}/}' input_file &amp;gt; output_file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here, the command line flags are:&lt;br&gt;&#xA;&lt;code&gt;-e&lt;/code&gt;: tells the perl interpreter to look for the code inline, rather than in a file with the script,&lt;br&gt;&#xA;&lt;code&gt;-p&lt;/code&gt;: loop over the input one line at a time, execute the code in the one-liner for each iteration of the loop, and &lt;code&gt;print&lt;/code&gt; the result into &lt;code&gt;STDOUT&lt;/code&gt;.&lt;br&gt;&#xA;The code has regex replacement (see &lt;a href=&quot;https://perldoc.perl.org/perlre.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;perlre&lt;/code&gt;&lt;/a&gt; for more details):&lt;br&gt;&#xA;&lt;code&gt;s{&amp;gt;([^_]+)_.*_}{&amp;gt;RNA${i}#${1}/}&lt;/code&gt;:&lt;br&gt;&#xA;replace an underscore, followed by zero or more occurrences (&lt;code&gt;*&lt;/code&gt; quantifier) of any character (&lt;code&gt;.&lt;/code&gt;), followed by an underscore, with slash (&lt;code&gt;/&lt;/code&gt;).&lt;br&gt;&#xA;Keep &lt;code&gt;&amp;gt;&lt;/code&gt; and the original header part before the underscore, captured by parens into &lt;code&gt;$1&lt;/code&gt; variable.&lt;br&gt;&#xA;Add string &lt;code&gt;RNA${i}&lt;/code&gt;, where &lt;code&gt;$i&lt;/code&gt; is the counter of the fasta headers. The counter is incremented by 1 when the substitution occurs, that is, when the next header occurs.&lt;br&gt;&#xA;The variables are used like so in the substitution: &lt;code&gt;${i}&lt;/code&gt;, &lt;code&gt;${1}&lt;/code&gt; to prevent wrong interpolation.&lt;br&gt;&#xA;I am also using &lt;code&gt;s{}{}&lt;/code&gt; as delimiters for readability (as in most cases) instead of the default &lt;code&gt;s///&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EXAMPLE:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;echo &quot;&amp;gt;LTR_retrotransposon100_Gypsy?\nACGT\n&amp;gt;LTR_retrotransposon10011_ERVK\nTCGA&quot; | perl -pe 'BEGIN { &lt;span class=&quot;math-container&quot;&gt;$i = 1 } $&lt;/span&gt;i += s{&amp;gt;([^_]+)_.*_}{&amp;gt;RNA&lt;span class=&quot;math-container&quot;&gt;${i}#$&lt;/span&gt;{1}/}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Prints:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;RNA1#LTR/Gypsy?&#xA;ACGT&#xA;&amp;gt;RNA2#LTR/ERVK&#xA;TCGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6251" LastEditorUserId="6251" LastEditDate="2020-02-18T22:08:28.993" LastActivityDate="2020-02-18T22:08:28.993" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11392" PostTypeId="2" ParentId="11334" CreationDate="2020-02-12T18:25:34.603" Score="2" Body="&lt;p&gt;To add to Michael G.'s answer, there is a further wrinkle in the case of 2019-nCov that is due to the exponential growth in confirmed infections daily.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That growth affects your second formula &lt;code&gt;CONFIRMED_DEATHS/(CONFIRMED_DEATHS + CONFIRMED_RECOVERIES&lt;/code&gt; since deaths probably occur earlier on in the disease trajectory than recoveries do. Since the number of confirmed cases is growing exponentially on a daily basis, the number of deaths will presumably be inflated relative to the number of recoveries at this point. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This feature may have the opposite effect on your first formula &lt;code&gt;CONFIRMED DEATHS/CONFIRMED INFECTIONS&lt;/code&gt;, depending on when the infections are confirmed. If the infections are generally confirmed several days before death occurs, then a large portion of confirmed infections are still too early to have ended in death and &lt;code&gt;CONFIRMED DEATHS/CONFIRMED INFECTIONS&lt;/code&gt; actually underestimates the actual number of fatalities. &lt;/p&gt;&#xA;" OwnerUserId="7210" LastActivityDate="2020-02-12T18:25:34.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11393" PostTypeId="1" CreationDate="2020-02-12T18:35:44.547" Score="0" ViewCount="87" Body="&lt;p&gt;I have few 100 raw fastq files from whole-genome sequencing data and I would like to map these files to a set of genes only (and not whole genome) so as to find SNP's associated with them. Can anyone tell me what snp analysis pipeline is best for targeted variation and not whole genome?&#xA;I have this in mind:&#xA;Mapping the reads with bwa and then using varscan&#xA;Hope to hear for more options&lt;/p&gt;&#xA;" OwnerUserId="2040" LastEditorUserId="57" LastEditDate="2020-02-12T20:24:48.210" LastActivityDate="2020-02-13T20:11:27.487" Title="Variant calling for a subset of genes using whole gneome sequencing data" Tags="|snp|variant-calling|bwa|wgs|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11394" PostTypeId="1" AcceptedAnswerId="11395" CreationDate="2020-02-12T20:42:15.877" Score="1" ViewCount="1731" Body="&lt;p&gt;I have called peaks using MACS2. Then I got a narrowPeak file like this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   4491713 4491913 testONE_peak_1  37  .   2.86003 3.72580 1.66717 80&#xA;1   4492782 4493153 testONE_peak_2  27  .   2.22933 2.71107 0.82442 242&#xA;1   4571554 4572026 testONE_peak_3  72  .   4.16005 7.21000 4.85200 282&#xA;1   4739486 4739879 testONE_peak_4  64  .   3.63128 6.42948 4.14394 275&#xA;1   4784848 4785061 testONE_peak_5  31  .   2.24218 3.19160 1.21569 133&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I tried to visualize it in UCSC Genome browser and I got following error.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error File 'testONE_peaks.narrowPeak' - Unrecognized format line 1 of file: 1 4491713 4491913 testONE_peak_1  37  .   2.86003 3.72580 1.66717 80 (note: chrom names are case sensitive, e.g.: correct: 'chr1', incorrect: 'Chr1', incorrect: '1')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So I added chr to the chromosome column instead of 1 (Chromosome number). &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    4491713 4491913 testONE_peak_1  37  .   2.86003 3.7258  1.66717 80&#xA;chr1    4492782 4493153 testONE_peak_2  27  .   2.22933 2.71107 0.82442 242&#xA;chr1    4571554 4572026 testONE_peak_3  72  .   4.16005 7.21    4.852   282&#xA;chr1    4739486 4739879 testONE_peak_4  64  .   3.63128 6.42948 4.14394 275&#xA;chr1    4784848 4785061 testONE_peak_5  31  .   2.24218 3.1916  1.21569 133&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I tried to load the file again I got following error.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error File 'testONE_peaks_NEW2.narrowPeak' - Error line 1 of custom track: thickStart out of range (chromStart to chromEnd, or 0 if no CDS)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then I added following line to the top and I could load the file to UCSC Genome browser.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;track type=narrowPeak name=&quot;Somite narrowPeak&quot;&#xA;chr1    4491713 4491913 testONE_peak_1  37  .   2.86003 3.7258  1.66717 80&#xA;chr1    4492782 4493153 testONE_peak_2  27  .   2.22933 2.71107 0.82442 242&#xA;chr1    4571554 4572026 testONE_peak_3  72  .   4.16005 7.21    4.852   282&#xA;chr1    4739486 4739879 testONE_peak_4  64  .   3.63128 6.42948 4.14394 275&#xA;chr1    4784848 4785061 testONE_peak_5  31  .   2.24218 3.1916  1.21569 133&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But the problem is I can not see anything in the Genome browser. Only the name is there. But nothing else. Do you know what I should do?&lt;/p&gt;&#xA;" OwnerUserId="856" LastActivityDate="2020-10-26T07:53:27.043" Title="How to visualize called narrowPeak files in UCSC Genome browser or IGV?" Tags="|rna-seq|chip-seq|ucsc|igv|macs2|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11395" PostTypeId="2" ParentId="11394" CreationDate="2020-02-12T21:34:53.253" Score="2" Body="&lt;p&gt;I think you might have changed the separator (or at least have some kind of inconsistency from the &lt;a href=&quot;https://genome.ucsc.edu/FAQ/FAQformat.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;required format&lt;/a&gt;) for your file. Note that peak output files from MACS2 are variants of BED files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems you &lt;strong&gt;need to have a &lt;em&gt;tab&lt;/em&gt; separator for this type of file.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I copied your example file and then ran &lt;code&gt;awk -v OFS=&quot;\t&quot; '$1=$1' your_peaks.narrowPeaks &amp;gt; fixed_your_peaks.narrowPeaks&lt;/code&gt; on it. This produced a result that I was able to see in both the UCSC genome browser and IGV.&lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="383" LastEditDate="2020-02-12T21:51:33.843" LastActivityDate="2020-02-12T21:51:33.843" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11396" PostTypeId="2" ParentId="11393" CreationDate="2020-02-12T22:43:51.447" Score="3" Body="&lt;p&gt;Your approach is not recommended. Always align against the entire genome. If you align against a subset you might accumulate false mappings. The aligner will always try to find the best match for every read. If the true origin of the read is not in the reference then the aligner will still try to find an acceptable mapping position. Therefore, align against the full reference and then extract the regions you are interested in, e.g. with the region option of &lt;code&gt;samtools view&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;samtools view [options] &amp;lt;in.bam&amp;gt;|&amp;lt;in.sam&amp;gt;|&amp;lt;in.cram&amp;gt; [region ...]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Check its manual for details.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While there is basically nothign wrong with &lt;code&gt;VarScan&lt;/code&gt; it is quite old and not maintained anymore. I suggest you give &lt;code&gt;bcftools&lt;/code&gt; a try (check its manual). Other maintained alternatives can be (among others) &lt;code&gt;freebayes&lt;/code&gt;, strelka2, VarDict or the GATK from the Broad institute.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-02-12T22:43:51.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11397" PostTypeId="2" ParentId="11390" CreationDate="2020-02-13T07:10:03.690" Score="2" Body="&lt;p&gt;Assuming your user is in the group permitted to use docker:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;docker pull quay.io/biocontainers/tophat:2.1.1--py27_3&#xA;nohup docker run -v /some/path:/data quay.io/biocontainers/tophat:2.1.1--py27_3 tophat2 ... &amp;amp;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The only thing to remember is to mount everything you'll need into the docker container (&lt;code&gt;/data&lt;/code&gt; in the example above). If you need to debug things then you can add the &lt;code&gt;-it&lt;/code&gt; flag to &lt;code&gt;docker run&lt;/code&gt; and use &lt;code&gt;bash&lt;/code&gt; as the command.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an aside, you might find using &lt;code&gt;screen&lt;/code&gt; or &lt;code&gt;tmux&lt;/code&gt; preferable to &lt;code&gt;nohup ... &amp;amp;&lt;/code&gt;, since you can reconnect to sessions with them.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-02-13T07:10:03.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11398" PostTypeId="2" ParentId="11281" CreationDate="2020-02-13T09:47:33.470" Score="1" Body="&lt;p&gt;Actually I found an easy and efficient way.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;import numpy as np&#xA;import scipy&#xA;import math&#xA;import openpyxl&#xA;from openpyxl import Workbook&#xA;from scipy.stats import pearsonr&#xA;from scipy.stats import spearmanr&#xA;from scipy.stats import kendalltau&#xA;&#xA;pro=pd.read_csv('m_test.csv', index_col=0)&#xA;&#xA;xInds = []&#xA;yInds = []&#xA;for i in range(len(pro.index)):&#xA;    for j in range(i+1, len(pro.index)):&#xA;        xInds.append(i)&#xA;        yInds.append(j)&#xA;&#xA;z=0&#xA;Rlist_sp = []&#xA;Plist_sp = []&#xA;Rlist_pe = []&#xA;Plist_pe = []&#xA;while z &amp;lt; len(xInds):&#xA;    b=xInds[z]&#xA;    c=yInds[z]&#xA;&#xA;    spR, spP = spearmanr(pro.iloc[b].values, pro.iloc[c].values)&#xA;    peR, peP = pearsonr(pro.iloc[b].values, pro.iloc[c].values) &#xA;&#xA;    Rlist_sp.append(spR)&#xA;    Plist_sp.append(spP)&#xA;    Rlist_pe.append(peR)&#xA;    Plist_pe.append(peP)    &#xA;    z=z+1 &#xA;&#xA;G1=pd.DataFrame(xInds)&#xA;G2=pd.DataFrame(yInds)&#xA;R_sp=pd.DataFrame(Rlist_sp)&#xA;P_sp=pd.DataFrame(Plist_sp)&#xA;R_pe=pd.DataFrame(Rlist_pe)&#xA;P_pe=pd.DataFrame(Plist_pe)        &#xA;&#xA;Final=pd.concat([G1, G2, R_sp, P_sp, R_pe, P_pe], axis=1)&#xA;np.savetxt('Final_gg_A549.txt', Final.values, fmt='%s', delimiter='\t')  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6670" LastActivityDate="2020-02-13T09:47:33.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11400" PostTypeId="2" ParentId="11378" CreationDate="2020-02-13T10:59:09.257" Score="2" Body="&lt;p&gt;You could use HMM-based comparison, for example with &lt;a href=&quot;https://toolkit.tuebingen.mpg.de/#/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HHpred&lt;/a&gt;. This is generally the best way to find structural templates because structure is more conserved than sequence, and comparing HMMs will turn up hits of the same structure but minimal sequence overlap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If HHpred turns up nothing you are in the region of de novo modelling, which is a different problem. However, it has advanced in recent years to the point where it may be useful to you. For example, you could try our method &lt;a href=&quot;https://www.nature.com/articles/s41467-019-11994-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;DMPfold&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2020-02-13T10:59:09.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11401" PostTypeId="2" ParentId="11276" CreationDate="2020-02-13T12:50:28.870" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/USDA-VS/vSNP&quot; rel=&quot;nofollow noreferrer&quot;&gt;vSNP&lt;/a&gt; outputs a VCF file that includes positions with no coverage, represented as &quot;N&quot;.  Then there is a tool to merge a VCF file into the FASTA reference  &lt;code&gt;vsnp_merge_vcf_into_fasta.py&lt;/code&gt;.  It will merge those &quot;N&quot; regions into the FASTA reference. There are options for considering frequency and quality.&#xA; It's limited to just SNPs and those regions along the reference with no coverage.  It does not merge small indels or unmapped reads.  &lt;a href=&quot;https://github.com/broadinstitute/pilon/wiki/Methods-of-Operation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pilon&lt;/a&gt; is a better tool for applying indels, but I'm not aware of it letting regions with no coverage be represented as &quot;N&quot;.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;vSNP_step1.py -r1 *_R1*fastq.gz -r2 *_R2*fastq.gz -r ref.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Use &lt;code&gt;*zc.vcf&lt;/code&gt; output to merge into FASTA&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;vsnp_merge_vcf_into_fasta.py -f ref.fasta -v *_zc.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6343" LastActivityDate="2020-02-13T12:50:28.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11403" PostTypeId="1" AcceptedAnswerId="11404" CreationDate="2020-02-13T16:14:29.463" Score="6" ViewCount="2653" Body="&lt;p&gt;(How) can you index a BAM file using &lt;code&gt;pysam&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I tried the intuitive &lt;code&gt;pysam.index&lt;/code&gt; I got:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pysam&#xA;my_bam = pysam.AlignmentFile(&quot;regular_bwamem_mapping.bam&quot;, &quot;rb&quot;)&#xA;pysam.index(my_bam)&#xA;&#xA;---------------------------------------------------------------------------&#xA;TypeError                                 Traceback (most recent call last)&#xA;&amp;lt;ipython-input-5-a0b3a04ecb2c&amp;gt; in &amp;lt;module&amp;gt;&#xA;----&amp;gt; 1 pysam.index(L_bam)&#xA;&#xA;~/miniconda3/lib/python3.7/site-packages/pysam/utils.py in __call__(self, *args, **kwargs)&#xA;     59             args,&#xA;     60             catch_stdout=kwargs.get(&quot;catch_stdout&quot;, True),&#xA;---&amp;gt; 61             save_stdout=kwargs.get(&quot;save_stdout&quot;, None))&#xA;     62&#xA;     63         if kwargs.get(&quot;split_lines&quot;, False):&#xA;&#xA;pysam/libcutils.pyx in pysam.libcutils._pysam_dispatch()&#xA;&#xA;TypeError: object of type 'pysam.libcalignmentfile.AlignmentFile' has no len()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="57" LastEditorUserId="1775" LastEditDate="2020-02-14T08:00:27.927" LastActivityDate="2023-11-26T02:28:43.727" Title="Index a BAM file using pysam" Tags="|sam|indexing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11404" PostTypeId="2" ParentId="11403" CreationDate="2020-02-13T16:14:29.463" Score="11" Body="&lt;p&gt;Oh you silly sausage, &lt;code&gt;pysam.index&lt;/code&gt; takes a bam file name, not a python object.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pysam&#xA;pysam.index(&amp;quot;regular_bwamem_mapping.bam&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;will index your &lt;code&gt;.bam&lt;/code&gt; file.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2021-05-17T10:58:45.903" LastActivityDate="2021-05-17T10:58:45.903" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11406" PostTypeId="2" ParentId="11393" CreationDate="2020-02-13T20:11:27.487" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/USDA-VS/vSNP&quot; rel=&quot;nofollow noreferrer&quot;&gt;vSNP&lt;/a&gt; can be used to target specific regions of a genome. Pipeline is a 2 step process:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;create VCF files (via Freebayes)&lt;/li&gt;&#xA;&lt;li&gt;then in your case run all 100 VCF files to create SNP table&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;A filter file can be supplied to step 2.  By filtering all position ranges other than the genes of interest, step 2 will output a SNP table (.xlsx) with only your genes of interest.  Also, table will include annotation if GBK is provide. &lt;/p&gt;&#xA;" OwnerUserId="6343" LastActivityDate="2020-02-13T20:11:27.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11407" PostTypeId="1" AcceptedAnswerId="11408" CreationDate="2020-02-13T22:07:36.073" Score="3" ViewCount="70" Body="&lt;p&gt;I recently downloaded a genome from NCBI for chimpanzee and was surprised to see no Y chromosome.  I added in one from a separate accession and carried on, but then today went to do the same for gorilla and saw the same thing, no Y.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For Gorilla gorilla (western gorilla): &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genome/2156&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/genome/2156&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;For Pan troglodytes (chimpanzee): &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genome/202&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/genome/202&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Why would this be?  I can see a separate entry for a Y chromosome for that same exact chimp used for the representative genome (Clint) here, but I don't see why it's not in the representative genome listing itself:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/DP000054.3&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/DP000054.3&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2020-02-14T05:06:33.640" Title="Why do these NCBI representative genomes for ape species have no Y chromosome?" Tags="|reference-genome|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11408" PostTypeId="2" ParentId="11407" CreationDate="2020-02-14T05:06:33.640" Score="8" Body="&lt;p&gt;tl;dr: technical difficulties, or sex&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the gorilla, that genome is from a female gorilla (&lt;a href=&quot;https://www.nature.com/news/gorilla-joins-the-genome-club-1.10185&quot; rel=&quot;noreferrer&quot;&gt;Kamilah the gorilla&lt;/a&gt;) so she doesn't have a Y chromosome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the chimp link, there is a visualization of the chromosomes that you can click on that takes you to the genome browser for the Y chromosome (see image). Notably, there is a &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly/GCA_000001515.5&quot; rel=&quot;noreferrer&quot;&gt;more recent genome build&lt;/a&gt; that does have a Y chromosome sequence assembly that is directly linked, they just haven't updated the main reference sequence yet.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kw5so.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kw5so.png&quot; alt=&quot;chimp entry&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To directly address the question, the Y chromosome is &lt;a href=&quot;https://www.cell.com/trends/genetics/pdf/S0168-9525(17)30019-7.pdf&quot; rel=&quot;noreferrer&quot;&gt;very hard to assemble&lt;/a&gt;. The easy part of it to assemble is already on the X chromosome anyways (the pseudoautosomal region), and the hard part of it is often a mess of repeats, especially in mammals. Many groups will decide to just assemble the X as well as they can and call it good, due to the difficulties associated with assembling the Y. If they are working with a male individual the Y sequence is still there, it just may not be anchored to a chromosome sequence, so it's only in the unscaffolded/unanchored assembly (&quot;master WGS&quot; I believe, usually). It is also probably highly fragmented (it isn't a whole chromosome of sequence stitched together, but rather a lot of little pieces of sequence).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In some cases they will focus all sequencing efforts on female individuals specifically to make their assembly job easier, because in females the X is at the same copy number as the autosomes and doesn't have interference from homologous regions on the Y.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-02-14T05:06:33.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11410" PostTypeId="2" ParentId="6852" CreationDate="2020-02-14T14:12:53.523" Score="3" Body="&lt;p&gt;I tried to adapt this approach within rdKit to calculate exact mass (for mass spectrometry), but was not successful.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This library (&lt;a href=&quot;https://pypi.org/project/molmass/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pypi.org/project/molmass/&lt;/a&gt;) appears to be able to do this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from molmass import Formula&#xA;f = Formula('C7H11N3O2')&#xA;f.formula&#xA;f.isotope.mass&#xA;'169.08512660696'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7223" LastActivityDate="2020-02-14T14:12:53.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11412" PostTypeId="1" CreationDate="2020-02-14T16:11:43.883" Score="2" ViewCount="105" Body="&lt;p&gt;Is there a methodology for detecting whether a genome has been tampered with by a biologist?  For instance, say there has been an ebola outbreak such as in &quot;The Andromeda Strain&quot;.  Is there a way a researcher can tell by just looking at the ebola genetic data and determine it is a genetically engineered strain?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some ideas I have:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;organism is abnormally close to unrelated organisms, i.e. some of the genetic code from the other organisms has been inserted into the first&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;subsequences are very different from the rest of the genome.  If there is a consistent rate of mutation applied to the entire genome, then a subsequence that is abnormally noise free could be inserted.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;combination of 1 and 2: finding clear instances of unrelated organism genomes inserted into the genome being examined.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Additionally, are there any publicly accessible reference genomes that are known to be genetically engineered, that I could use as examples to test such ideas?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;UPDATE: the wikipedia article on detecting genetically modified organism is not quite what I am looking for, since it claims current methods cannot detect unknown genetic modification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Detection_of_genetically_modified_organisms&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Detection_of_genetically_modified_organisms&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Currently, it is highly unlikely that the presence of unexpected or&#xA;  even unknown GMOs will be detected, since either the DNA sequence of&#xA;  the transgene or its product, the protein, must be known for&#xA;  detection.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4806" LastEditorUserId="3848" LastEditDate="2020-11-28T15:29:12.123" LastActivityDate="2020-11-30T17:21:27.120" Title="Detecting whether genome is genetically engineered?" Tags="|phylogenetics|phylogeny|covid-19|genetics|sars-cov-2|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11413" PostTypeId="1" CreationDate="2020-02-14T19:26:31.683" Score="1" ViewCount="121" Body="&lt;p&gt;&amp;quot;Aliases&amp;quot; or &amp;quot;Synonyms&amp;quot; should represent the same gene but with different names. But when I try to find the sequence of one alias and try to match with another alias they are significantly different.&lt;/p&gt;&#xA;&lt;p&gt;During bioinformatics analysis, I need to convert NAT5 alias ID to its gene symbol ID (NAA20, NAA50). When I tried to match the sequences of NAA20 and NAA50, they are very different but still NAT5 is an alias of both NAA20 and NAA50.&lt;/p&gt;&#xA;&lt;p&gt;I have to convert one id to another id (ENSEMBL). These gene symbol and alias is so confusing.&lt;/p&gt;&#xA;&lt;p&gt;I am looking for NAT5 and when I search for NAT5 in ENSEMBL, it shows both NAA20 and NAA50.&lt;/p&gt;&#xA;&lt;p&gt;From UniProt the canonical sequence of both the proteins:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.uniprot.org/uniprot/P61599#sequences&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uniprot/P61599#sequences&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.uniprot.org/uniprot/Q9GZZ1#sequences&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uniprot/Q9GZZ1#sequences&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am confused which one to chose?&lt;/p&gt;&#xA;" OwnerUserId="6471" LastEditorUserId="73" LastEditDate="2021-07-14T11:37:22.253" LastActivityDate="2021-07-14T11:57:48.040" Title="Why the gene symbol aliases have significantly different sequences?" Tags="|ensembl|identifiers|entrez|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11415" PostTypeId="1" CreationDate="2020-02-14T23:58:11.373" Score="2" ViewCount="298" Body="&lt;p&gt;I would like to retrieve a list of all the human genes and their proteins’ functional classification. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;Gene type&lt;/code&gt; attribute on &lt;a href=&quot;http://useast.ensembl.org/biomart/martview/440eaadc1b7770036788e1a8d7787471&quot; rel=&quot;nofollow noreferrer&quot;&gt;ensembl.org&lt;/a&gt; is not specific enough, because it assign the label &lt;code&gt;coding protein&lt;/code&gt; to all the proteins for which I am trying to classify. Also the &lt;code&gt;KEGG Pathway and Enzyme ID&lt;/code&gt; option doesn't seem to help. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have translated all the gene names to their GO representation, but now I am not sure how to query for their cellular function, biological process, and cellular component.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there such a platform that can retrieve this info from PANTHER?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sadly, the &lt;a href=&quot;http://www.ebi.ac.uk/reference_proteomes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;reference proteome&lt;/a&gt; of humans doesn't contain a functional classification either:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; zcat UP000005640_9606.fasta.gz | grep -i mYOSIN-2&#xA;&amp;gt;sp|Q9UKX2|MYH2_HUMAN Myosin-2 OS=Homo sapiens OX=9606 GN=MYH2 PE=1 SV=1&#xA;&#xA;# &amp;gt;db|UniqueIdentifier|EntryName ProteinName OS=OrganismName OX=OrganismIdentifier [GN=GeneName ]PE=ProteinExistence SV=SequenceVersion&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there any way to use &lt;a href=&quot;https://bgee.org/?page=gene&amp;amp;gene_id=ENSG00000125414&quot; rel=&quot;nofollow noreferrer&quot;&gt;bgee&lt;/a&gt;, &lt;a href=&quot;https://www.ebi.ac.uk/QuickGO/annotations?geneProductId=Q9UKX2&quot; rel=&quot;nofollow noreferrer&quot;&gt;QuickGO&lt;/a&gt;, &lt;a href=&quot;https://tools.dice-database.org/GOnet/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GONet&lt;/a&gt;, or &lt;a href=&quot;https://www.uniprot.org/uniprot/Q9UKX2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Uniport&lt;/a&gt; to cluster genes using the &lt;a href=&quot;https://www.uniprot.org/uniprot/?query=proteome:UP000005640#customize-columns&quot; rel=&quot;nofollow noreferrer&quot;&gt;customize columns&lt;/a&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that getting the most specific (&lt;code&gt;GO term name&lt;/code&gt;) and least specific (&lt;code&gt;GO domain&lt;/code&gt;) gene's GO values can be retrieved from ensmebl:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QrELf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QrELf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2020-05-09T15:27:25.400" LastActivityDate="2020-05-09T15:27:25.400" Title="How to get the GO information for all the human genes?" Tags="|python|gene|proteins|go-enrichment|functional-annotation|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11416" PostTypeId="1" CreationDate="2020-02-15T15:29:39.667" Score="1" ViewCount="20" Body="&lt;p&gt;I want to find genes of fundamental metabolic processes in the &lt;em&gt;Phytophthora cinnamomi&lt;/em&gt; genome. I'm using the genome sequence deposited in the NCBI and I'm searching ORF by ORF, using the ORF finder (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/orffinder/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/orffinder/&lt;/a&gt;). Next, I use SMARTBLAST (&lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/smartblast/?LINK_LOC=BlastHomeLink&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blast.ncbi.nlm.nih.gov/smartblast/?LINK_LOC=BlastHomeLink&lt;/a&gt;) to find similar organisms. &#xA;How do I know that the gene that I'm searching expresses proteins of fundamental metabolic processes, and if the organism is similar to mine?&#xA;Can the best hits in SMARTBLAST tell me something about that?&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="7228" LastEditorUserId="6509" LastEditDate="2020-02-17T19:25:31.357" LastActivityDate="2020-02-17T19:25:31.357" Title="How can I find genes of fundamental metabolic processes for an organism?" Tags="|sequence-alignment|gene|blast|phylogenetics|sequence-homology|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11417" PostTypeId="1" AcceptedAnswerId="11426" CreationDate="2020-02-15T20:28:43.300" Score="0" ViewCount="151" Body="&lt;p&gt;Plink documentation about &lt;a href=&quot;https://www.cog-genomics.org/plink/1.9/formats#bim&quot; rel=&quot;nofollow noreferrer&quot;&gt;.bim&lt;/a&gt; says the columns 5 and 6 are the Allele 1 and Allele 2 respectively. &#xA;The documentation about the &lt;a href=&quot;https://www.cog-genomics.org/plink/1.9/formats#ped&quot; rel=&quot;nofollow noreferrer&quot;&gt;.ped&lt;/a&gt; in turn, says &quot;...The first six fields are the same as those in a .fam file. The seventh and eighth fields are allele calls for the first variant in the .map file ('0' = no call); the 9th and 10th are allele calls for the second variant; and so on..&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've simulated a dataset as per:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ plink1.9 --simulate ds1.sim acgt --out ds1&#xA;...&#xA;$&lt;/span&gt; head ds1.bim &#xA;1       null_0  0       1       C       T&#xA;1       null_1  0       2       A       C&#xA;1       null_2  0       3       C       A&#xA;1       null_3  0       4       T       A&#xA;1       null_4  0       5       G       C&#xA;1       null_5  0       6       T       C&#xA;1       null_6  0       7       T       C&#xA;1       null_7  0       8       G       C&#xA;1       null_8  0       9       C       G&#xA;1       null_9  0       10      T       A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Than I converted it to plain PED and MAP: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ plink1.9 --bfile ds1 --recode --out ds1&#xA;...&#xA;$&lt;/span&gt; head ds1.ped &amp;gt; ds1.head.ped&#xA;$ vim ds1.head.ped&#xA;per0 per0 0 0 2 2 T T A C A A T A C C T C T C C C....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Given the data above, why my .bim file shows &quot;C and T&quot; for the first variant while my .ped file shows &quot;T and T&quot;?&#xA;Apparently they don't match as per described in the documentation (or more likely, I'm misinterpreting it)...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, how do those 2 files related to each other? E.g.: How do I know that my Sample 1 has the SNP's null_3, null_4 or null_5, etc?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to clarify those doubt because I want to fit a LightGBM model with the Genomic Data (can be plink or vcf). Not sure yet how to do. I was trying to convert the plink files to a CSV-style somehow, to be able to use in my model. Not sure if this is the way to go though... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="6464" LastActivityDate="2020-02-16T19:14:15.807" Title="Why my bim file doesn't match to my ped file as the Plink documentation suggests?" Tags="|machine-learning|plink|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11418" PostTypeId="1" CreationDate="2020-02-15T21:08:02.107" Score="0" ViewCount="73" Body="&lt;p&gt;Phasing a genome is the process of determining which variants lie on which copy of each chromosome. For example, position 5,430,500 might have an A on the paternal chromosome, but a T base on the maternal chromosome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A haplotype block is a section of the genome for which we know which haplotype has what variants. A completely phased genome has phase blocks that are the full length of each chromosome. Every variant on every chromosome can be assigned specifically to the maternal to or the paternal haplotype.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question is&lt;/strong&gt;, what can we learn from having completely phased genomes that we cannot from an unphased genome, or from a phased genome with short haplotype blocks? &lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2020-02-19T04:20:52.253" Title="Q: What analyses can I perform with a completely phased genome assembly?" Tags="|vcf|snp|phasing|variants|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11420" PostTypeId="2" ParentId="9229" CreationDate="2020-02-16T02:07:04.343" Score="1" Body="&lt;p&gt;One more thing. If empty ALT allele is not &quot;-&quot;, you can change it such as &quot;*&quot;, as follows.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    if &quot;*&quot; in record.alts:&#xA;        ref = self.fasta.fetch(reference=record.chrom, start=record.pos - 1, end=record.pos)&#xA;        record.pos = record.pos - 1&#xA;        record.ref = f&quot;{ref}{record.ref}&quot;&#xA;        record.alts = tuple(map(lambda alt: ref if alt == &quot;*&quot; else f&quot;{ref}{alt}&quot;, record.alts))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7230" LastActivityDate="2020-02-16T02:07:04.343" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11422" PostTypeId="1" AcceptedAnswerId="11423" CreationDate="2020-02-16T05:48:22.880" Score="1" ViewCount="309" Body="&lt;p&gt;For easier management of a database of dna profiles, I'm creating a tool to perform one-to-one matching on two profiles with snp genotype information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, something that works similar to GEDMatch's one-to-one matching feature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Determining the length of runs of half or fully matched snp's, including some allowance for a genotyping error rate, is the easy part.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is about converting an snp run length to &lt;a href=&quot;https://en.wikipedia.org/wiki/Centimorgan&quot; rel=&quot;nofollow noreferrer&quot;&gt;centimorgans&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I figure that the length needs to be adjusted by the likelihood of a change in the particular region of the chromosome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since my dna profiles and the positions of snps are based on Genome Reference Consortium Human Build 37, I figure that his information could be in the reference build, directly or derivable in some way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly/GCF_000001405.13/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GRCh37 download files&lt;/a&gt; and the definitions of their formats, I'm unable to find such information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this information present in GRCh37, or can it be downloaded from another reference source, and from which field in the dataset can it be retrieved or derived?&lt;/p&gt;&#xA;" OwnerUserId="7226" LastActivityDate="2020-02-16T11:41:47.583" Title="How to calculate Centimorgan lengths from run lengths in one-to-one dna profile comparison" Tags="|snp|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11423" PostTypeId="2" ParentId="11422" CreationDate="2020-02-16T11:34:40.897" Score="3" Body="&lt;p&gt;Try having a look at this link:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://bochet.gcc.biostat.washington.edu/beagle/genetic_maps/plink.GRCh37.map.zip&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://bochet.gcc.biostat.washington.edu/beagle/genetic_maps/plink.GRCh37.map.zip&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you unzip the file, there is a file for each chromosome (1:22) and then 3 for the x chromosome. The first column of each file details which chromosome the file corresponds to, the 3rd column gives you the cumulative genetic distance in cM and the 4th column gives you the genomic coordinate (position) in bp. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then calculate the distance in cM between two positions from this data by subtracting the cM positions between the two positions you are interested in. &lt;/p&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="3694" LastEditDate="2020-02-16T11:41:47.583" LastActivityDate="2020-02-16T11:41:47.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11424" PostTypeId="2" ParentId="11388" CreationDate="2020-02-16T14:44:26.863" Score="3" Body="&lt;p&gt;Your &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/11389/57&quot;&gt;own answer&lt;/a&gt; doesn't seem to do what you ask as it doesn't add the sequential identifier to each sequence. Maybe try this?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -v rec=0 '{&#xA;    if(&lt;span class=&quot;math-container&quot;&gt;$0 ~ &quot;^&amp;gt;&quot;){&#xA;        rec++&#xA;        sub(&quot;&amp;gt;&quot;, &quot;&amp;gt;RNA&quot; rec &quot;#&quot;, $0)&#xA;        sub(&quot;_.*_&quot;, &quot;/&quot;, $0);&#xA;    } &#xA;    print $&lt;/span&gt;0&#xA;}' yourfile.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="339" LastEditorUserId="57" LastEditDate="2020-04-09T08:36:17.453" LastActivityDate="2020-04-09T08:36:17.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11425" PostTypeId="2" ParentId="11412" CreationDate="2020-02-16T17:02:25.473" Score="2" Body="&lt;p&gt;The definition of a GMO for example UK GMO regulatory bodies (HSE) is strictly the artificial introduction of foreign genetic material using targeted genetic engineering, such as splicing,  genome integration etc... Attenuation for example by serial passage the basis of the yellow fever vaccine, does not count as a GMO. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The simple answer is look at phylogenetic trees between the GMO strain against the total distribution of isolates. If large regions have been spliced the 'silent mutation rate' looks very different,.because the GMO parent is from a very different environment to the recipient.  The switch in replication rate affects the numbers of 'silent mutation', which is fast in RNA viruses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the GMO is a point mutation, ie limited amino acid change then I've actually done this with dengue virus and it is fairly easy because the novel mutation make the GMO 'stick out'.of the phylogeny against all natural mutation patterns. At this point the outlier is tracked back to Genbank where the relavant publication defines the engineered virus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are inferring a conspiracy theory that is a different process and a different discussion, particularly if it is Ebolavirus which is a Category 4 pathogen. The investigation is again within the phylogeny, but in this case the answer is always the isolate is a natural zoonose from a sylvatic (wild) transmission cycle which has spilled over into humans.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-02-16T20:42:21.050" LastActivityDate="2020-02-16T20:42:21.050" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11426" PostTypeId="2" ParentId="11417" CreationDate="2020-02-16T19:14:15.807" Score="2" Body="&lt;p&gt;Responded in the plink2-users group (&lt;a href=&quot;https://groups.google.com/forum/#!topic/plink2-users/zgJxdXxvdLo&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://groups.google.com/forum/#!topic/plink2-users/zgJxdXxvdLo&lt;/a&gt; ).&lt;/p&gt;&#xA;" OwnerUserId="472" LastActivityDate="2020-02-16T19:14:15.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11427" PostTypeId="1" AcceptedAnswerId="11467" CreationDate="2020-02-17T09:38:11.897" Score="3" ViewCount="605" Body="&lt;p&gt;I want to compare RNA-Seq datasets obtained from the TCGA to investigate how my gene of interest is implicated in different types of cancer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to download the data from the GDC Data Portal (&lt;a href=&quot;https://portal.gdc.cancer.gov&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://portal.gdc.cancer.gov&lt;/a&gt;), but it is not user-friendly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can somebody give me some initial directions? I found a bioconductor package that facilitates access to the TCGA (&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/TCGAbiolinks.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/html/TCGAbiolinks.html&lt;/a&gt;), but first of all, I would like to understand how the files are organized, the nomenclature and how to inspect them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;thank you very much, Fabiano&lt;/p&gt;&#xA;" OwnerUserId="6509" LastEditorUserId="6509" LastEditDate="2020-02-17T10:17:41.787" LastActivityDate="2021-01-29T23:42:51.390" Title="How to initiate RNA-Seq analysis of TCGA files?" Tags="|rna-seq|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11428" PostTypeId="1" CreationDate="2020-02-17T10:10:35.277" Score="2" ViewCount="276" Body="&lt;p&gt;I'm trying to use cellranger to &lt;code&gt;mkfastq&lt;/code&gt; and then &lt;code&gt;count&lt;/code&gt; and &lt;code&gt;aggregate&lt;/code&gt; single-cell data. This data was not generated using 10x Genomics RNASeq. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I run &lt;code&gt;count&lt;/code&gt;, cellranger fails to auto id the chemistry and crashes out. Is it possible to wrangle the data into the cellranger method or should I look elsewhere. I'd ideally like to use the same method for all my single-cell data but not all of it is 10x.&lt;/p&gt;&#xA;" OwnerUserId="7232" LastActivityDate="2020-02-17T20:35:40.230" Title="Using cellranger for non-10x data" Tags="|single-cell|cellranger|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11429" PostTypeId="2" ParentId="11418" CreationDate="2020-02-17T11:52:51.137" Score="2" Body="&lt;p&gt;Some population genetic inference methods require the data to be phased because they look for long stretches of matching alleles between individuals, rather than just matching genotypes. Here, 'genotype' data is un-phased, and 'allele' data is phased. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is because there is a degree of ambiguity when it comes to matching heterozygous positions - if the data is un-phased, you don't know if 2 individuals share mutations on the same chromosome (here, I mean different copies of the same chromosome - like either copy of chromosome 1, for example). This is important, because the fact that the mutation is shared on the same chromosome means that they are much more likely to share recent common ancestry with one another. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's say we have genotype data for two individuals (where the number corresponds to the number of non-reference alleles present):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Ind 1&#xA;&#xA;0 ----- 1 ----- 2 ----- 0&#xA;&#xA;Ind 2&#xA;&#xA;0 ----- 1 ----- 2 ----- 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If were looking at unphased genotype data only, we know the alleles match exactly in the 1st, 3rd and 4th positions, because they are homozgyous (either 0 or 2 copies of the non-referencee allele), but because the second position is heterozygous, we can't be sure which chromosome the non-reference allele is on. This is because the haplotype configurations for individual 1 could be either &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0 ----- 0 ----- 1 ----- 0 Chromosome1a&#xA;0 ----- 1 ----- 1 ----- 0 Chromosome1b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0 ----- 1 ----- 1 ----- 0 Chromosome1a&#xA;0 ----- 0 ----- 1 ----- 0 Chromosome1b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;i.e., we don't know which chromosome the non-reference allele at position 2 is on. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If we then phased the data, and found that the non-reference allele was on chromosome 1a for Individual 1, and chromosome 1b for Individual 2 there is less evidence of recent shared ancestry than otherwise. I think this is more or less the approach that 23andme takes when they are inferring your relationship to other individuals in the dataset. After writing this, I realised there is a much better description of what I just said on the &lt;a href=&quot;https://www.23andme.com/en-gb/ancestry-composition-guide/&quot; rel=&quot;nofollow noreferrer&quot;&gt;23andme website&lt;/a&gt;! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, the above applies to the &lt;a href=&quot;https://journals.plos.org/plosgenetics/article?id=10.1371/journal.pgen.1002453&quot; rel=&quot;nofollow noreferrer&quot;&gt;ChromoPainter/fineSTRUCTURE&lt;/a&gt; algorithms. This means that they are able to detect more subtle genetic differences between individuals than if you were to use non-phased data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is a slightly niche example, and there occasions where you need phased data, but this is the one I am most familiar with :). Off the top of my head, methods such as &lt;a href=&quot;https://www.nature.com/articles/ng.3015&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multiply Sequential Markovian Coalescent&lt;/a&gt; (MSMC), &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=6&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwivhv-gwtjnAhVXTxUIHUutCcUQFjAFegQIBRAB&amp;amp;url=https%3A%2F%2Fwww.nature.com%2Farticles%2Fs41588-019-0484-x&amp;amp;usg=AOvVaw1y_rNgh_j-Wi_My5iXUrp4&quot; rel=&quot;nofollow noreferrer&quot;&gt;RELATE&lt;/a&gt;, and &lt;a href=&quot;https://www.nature.com/articles/nature01140&quot; rel=&quot;nofollow noreferrer&quot;&gt;Extended Haplotype Homozygosity&lt;/a&gt; all require the data to be phased. &lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2020-02-17T11:52:51.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11430" PostTypeId="1" CreationDate="2020-02-17T13:48:28.077" Score="1" ViewCount="73" Body="&lt;p&gt;I want to remove variant including in copy number changed area by maftools but I am permanently get this error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; tcga.ab.3009.het = inferHeterogeneity(maf = laml_res, tsb = tsb, segFile = seg, vafCol = 'T_VAF')&#xA;CN data for following samples not found. Ignoring them ..&#xA; [1] &amp;quot;LP6008337-DNA_A07&amp;quot; &amp;quot;LP6008460-DNA_F02&amp;quot; &amp;quot;LP6008460-DNA_G03&amp;quot; &amp;quot;LP6008460-DNA_D01&amp;quot; &amp;quot;LP6008202-DNA_B03&amp;quot; &amp;quot;LP6008460-DNA_A04&amp;quot;&#xA; [7] &amp;quot;LP6005935-DNA_G02&amp;quot; &amp;quot;LP6008334-DNA_B02&amp;quot; &amp;quot;LP6008269-DNA_F06&amp;quot; &amp;quot;LP6008336-DNA_F02&amp;quot; &amp;quot;LP6005334-DNA_H01&amp;quot; &amp;quot;LP6005500-DNA_D03&amp;quot;&#xA;[13] &amp;quot;LP6008334-DNA_A04&amp;quot; &amp;quot;LP6008336-DNA_G01&amp;quot; &amp;quot;LP6008336-DNA_H02&amp;quot; &amp;quot;LP6007600&amp;quot;        &#xA;Processing LP6008334-DNA_D02..&#xA;Error in data.table::foverlaps(x = tsb.dat, y = seg, by.x = c(&amp;quot;Chromosome&amp;quot;,  : &#xA;  The last two columns in by.x should correspond to the 'start' and 'end' intervals in data.table 'x' and must be integer/numeric type.&#xA;&#xA;&#xA;&amp;gt; head(dat.tsb[1:3,1:6])&#xA;   Tumor_Sample_Barcode Chromosome Start_Position End_Position Reference_Allele Tumor_Seq_Allele2&#xA;1:    LP6008334-DNA_A03       chr5       70858261     70858261                -                 A&#xA;2:    LP6008334-DNA_A03      chr10      115987869    115987869                A                 -&#xA;3:    LP6008334-DNA_A03      chr16       29859285     29859287              ACC                 -&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I do this?&lt;/p&gt;&#xA;" OwnerDisplayName="user3962" LastEditorUserId="131" LastEditDate="2020-07-16T11:56:47.937" LastActivityDate="2020-07-16T11:56:47.937" Title="Permanent error with maftools" Tags="|r|maftools|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11431" PostTypeId="1" AcceptedAnswerId="11455" CreationDate="2020-02-17T15:14:13.103" Score="2" ViewCount="576" Body="&lt;p&gt;I collected my own denovo library using many different tools Pipeline (structure-based ) and I updated all the headers corresponding to RepeatMasker format, to obtain Repeat coverage in the genome and .tbl annotation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;when I run repeatmasker the annotation results assume to be Homosapien, where Homo annotation table ( .tbl) is limited as shown below Table (1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actually my denovo repeat library consists of more than what is in the table above, example, rolling circles, Copia, Gypsy etc.. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;RepeatMasker -Lib myOwnLibrary.fa Mygenome.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Table (1)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2S9Dq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2S9Dq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how can I run RepeatMasker with my own library using -lib Flag and result like the table (2) below, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;the table below assume to be vertebrates genome, this is my aim. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;RepeatMasker -Species Vertebrates Mygenome.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Table (2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/b97Gw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/b97Gw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards&lt;/p&gt;&#xA;" OwnerUserId="3332" LastEditorUserId="3332" LastEditDate="2020-02-19T15:51:37.950" LastActivityDate="2020-05-14T17:57:11.007" Title="RepeatMasker annotation" Tags="|sequence-annotation|repeat-elements|repeatmasker|repeat|" AnswerCount="2" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11432" PostTypeId="1" AcceptedAnswerId="11444" CreationDate="2020-02-17T19:22:17.310" Score="2" ViewCount="52" Body="&lt;p&gt;I have a file with Ensembl gene and Transcript IDs :&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Dg9J0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Dg9J0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I want to retain only the first entires from both the column so that the output looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ENSG00000003137 ENST00000001146&#xA;ENSG00000003402 ENST00000309955&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and ignore the rest. &#xA;Have tried awk &lt;code&gt;awk '!a[$1$2]++'&lt;/code&gt; and versions of it. Nothing works.&#xA;Kindly suggest some way to get the desired output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;regards.&lt;/p&gt;&#xA;" OwnerUserId="7236" LastEditorUserId="3848" LastEditDate="2020-03-19T21:08:43.577" LastActivityDate="2020-03-19T21:08:43.577" Title="select unique names while removing the duplicates from a column" Tags="|ensembl|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11433" PostTypeId="2" ParentId="11428" CreationDate="2020-02-17T20:35:40.230" Score="1" Body="&lt;p&gt;Possible?  Sure.  Your biggest problem is that 10x is expecting barcodes from its whitelist.  I used to alter the python scripts to accept a 'new' kind of chemistry where the barcodes and umi lengths matched what I had, but I found that my modifications stopped working when the version updated,  and I couldn't figure it out, so I started modifying fastqs instead.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-02-17T20:35:40.230" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11434" PostTypeId="1" CreationDate="2020-02-18T09:09:26.197" Score="1" ViewCount="162" Body="&lt;p&gt;I am running kraken 2.0.8 beta to do taxonomic classification of micorbiota data coming from a MinION run based on the 16S gene. To do so I am using the pre-bult kraken SILVA  database. I managed to get the standard kraken output format but now I would like to replace the taxonomic ID column with  scientific names. According to the manual (&lt;a href=&quot;https://ccb.jhu.edu/software/kraken2/index.shtml?t=manual&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ccb.jhu.edu/software/kraken2/index.shtml?t=manual&lt;/a&gt;) I should run the command: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;kraken2 --use-names&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but it gives the error: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;command kraken2 not found&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried variants such as: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;./kraken 2 --use-names&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but still it gives an error.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried without kraken2 and thus, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;--use-names &lt;/p&gt;&#xA;&#xA;&lt;p&gt;but it gives this error: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;--use-names command not found &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any idea how I can proceed? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the picture you can see that everything works to obtain the standard kraken output for barcode 76. But if I try to implement the --use-names command then it does not work  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hWvgY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hWvgY.png&quot; alt=&quot;ERROR THAT I GET IF I TRY TO USE THE --USE-NAMES COMMAND&quot;&gt;&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I solved the problem: it was a matter of the position of the --use-names command. By changing position (see picture) it worked.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/vnCGH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vnCGH.png&quot; alt=&quot;problem solved picture&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7203" LastEditorUserId="7203" LastEditDate="2020-02-18T16:48:39.640" LastActivityDate="2020-02-18T16:48:39.640" Title="Scientific names of taxids with kraken 2.0.8 beta, command kraken2 --use-names error" Tags="|database|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11435" PostTypeId="1" AcceptedAnswerId="11438" CreationDate="2020-02-18T09:52:42.540" Score="2" ViewCount="1442" Body="&lt;p&gt;I will use GATK for SNP calling (&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035531412-HaplotypeCaller-in-a-nutshell&quot; rel=&quot;nofollow noreferrer&quot;&gt;HaplotypeCaller&lt;/a&gt;). I need to feed the interval file in the command, otherwise I get errors (even though I want to use the whole genome, not a subset  - btw it's not the point of the question, so will not elaborate). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/1319/collected-faqs-about-interval-lists&quot; rel=&quot;nofollow noreferrer&quot;&gt;interval file&lt;/a&gt; for GATK can be for example a &lt;code&gt;bed&lt;/code&gt; file, with 3 columns: &lt;code&gt;chr_name  chr_start  chr_end&lt;/code&gt;. I don't have this file, but have the genome and the reads. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To obtain the intervals per chr/scaf, I proceeded this way: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;converted the alignment to interval file, using &lt;code&gt;bam2bed&lt;/code&gt;    (&lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/file-management/conversion/bam2bed.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;extract the columns $1,2,3 (using &lt;code&gt;cut&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;get the min &amp;amp; max coordinates per chr/scaff with a small R script:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bd &amp;lt;- read.table('all.bed', h=FALSE)&#xA;bd_min &amp;lt;- aggregate( bd$V3 ~ bd$V1 , bd, function(x) min(x))&#xA;bd_max &amp;lt;- aggregate( bd$V2 ~ bd$V1 , bd, function(x) max(x))&#xA;bd_coord &amp;lt;- merge(bd_min, bd_max, by='bd$V1')&#xA;write.table(bd_coord, file='coords.bed', sep='\t', quote=FALSE, col.names=FALSE, row.names=FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These should thus represent the mapped intervals, conceptually same as not feeding the intervals at all (= considering all the sequences).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The ideas for this:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;in the bed file: &lt;code&gt;$1&lt;/code&gt; is be the feature, &lt;code&gt;$2&lt;/code&gt; the start position, &lt;code&gt;$3&lt;/code&gt; the end position&lt;/li&gt;&#xA;&lt;li&gt;so, I need to get the min of &lt;code&gt;$2&lt;/code&gt; for each unique item in &lt;code&gt;$1&lt;/code&gt;(for the start coord) and the max of &lt;code&gt;$3&lt;/code&gt; for each item in &lt;code&gt;$1&lt;/code&gt; (for the end coord)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The output looks plausible:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat coords.bed | head -3&#xA;Bla_chrm1   678 43860826&#xA;Bla_chrm10  181 20381540&#xA;Bla_chrm11  343 20367560&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question here:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;is this a correct way to proceed?&lt;/li&gt;&#xA;&lt;li&gt;is there a standardized way to perform this?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;My main concern is that I will have to re-build this &quot;coordinate&quot; files for every GATK run, because I think the coordinates per chr/scaf will shift, even slightly, at each GATK run with different datasets.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My other minor concern, is that this approach is a bit slow (given the 160M lines in the current bed file), so an unix-tools solution will also be accepted. I tried to compute myself the max/min using awk (e.g. &lt;code&gt;awk '$3&amp;gt;max[$1]{max[$1]=$3; row[$1]=$0} END{for (i in row) print row[i]}'&lt;/code&gt;), but I get different results than with the R approach... &lt;/p&gt;&#xA;" OwnerUserId="294" LastEditorUserId="294" LastEditDate="2020-02-19T05:46:29.210" LastActivityDate="2020-04-07T02:47:47.947" Title="Get start and end coordinates per chromosome" Tags="|file-formats|bed|gatk|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11436" PostTypeId="2" ParentId="11432" CreationDate="2020-02-18T09:59:51.153" Score="1" Body="&lt;p&gt;That sound like a job for a perl one-liner:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl -ane 'unless(exists &lt;span class=&quot;math-container&quot;&gt;$seen{$F[0]}){print $_; $seen{$&lt;/span&gt;F[0]}++}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Explanation:&lt;br/&gt; &#xA;The input is read line-wise and printed back to the command line if the ID in the first column has not been seen before. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;perl -ane '&lt;/code&gt; reads input line-wise and splits fields to array @F&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;unless(exists $seen{$F[0]}){&lt;/code&gt; checks if ID exists in hash %seen and continues if that is not the case&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;print $_;&lt;/code&gt; prints the original input to STDOUT &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$seen{$F[0]}++}'&lt;/code&gt; adds the ID to the hash %seen &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="668" LastActivityDate="2020-02-18T09:59:51.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11437" PostTypeId="1" CreationDate="2020-02-18T10:05:34.167" Score="0" ViewCount="1699" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/420333/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am going to use ultrametric tree for downstream analysis. I would like to see use the ultrametric tree for Ranger-DTL for infer gene family evolution by speciation, gene duplication, horizontal gene transfer, and gene loss.&lt;/p&gt;&#xA;&lt;p&gt;I have generated rooted species tree with 1000 bootstraps by RAxML. Further, I need to generate ultrametric tree for the same. Therefore, Please suggest me how to generate ultrametric tree from the rooted species tree.&lt;/p&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="4712" LastEditorUserId="73" LastEditDate="2021-06-07T02:08:10.963" LastActivityDate="2021-06-07T02:08:10.963" Title="How to generate ultrametric phylogenetic trees?" Tags="|python|phylogenetics|phylogeny|linux|perl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11438" PostTypeId="2" ParentId="11435" CreationDate="2020-02-18T13:25:38.093" Score="0" Body="&lt;p&gt;You can try this, since you have the bam file, all of them are aligned to the same reference genome (I hope)(and apologies for the awful awk.. been a while since I used it):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -H some.bam | awk '{if(&lt;span class=&quot;math-container&quot;&gt;$1~/@SQ/){print $2&quot;\t1\t&quot;$&lt;/span&gt;3}}' | sed 's/[SL]N://g' &amp;gt; coords.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For the human genome I got this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    1   195471971&#xA;chr2    1   182113224&#xA;chr3    1   160039680&#xA;chr4    1   156508116&#xA;chr5    1   151834684&#xA;chr6    1   149736546&#xA;chr7    1   145441459&#xA;chr8    1   129401213&#xA;chr9    1   124595110&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What I did is to get the chromosome length, and make every line start at 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you use R, you can do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Rsamtools)&#xA;CL = scanBamHeader(&amp;lt;bam file&amp;gt;)[[1]]$targets&#xA;write.table(data.frame(names(CL),start=1,end=CL),&#xA;&quot;coords.bed&quot;,row.names=FALSE,quote=F,col.names=FALSE,sep=&quot;\t&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-02-18T13:25:38.093" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11439" PostTypeId="2" ParentId="11435" CreationDate="2020-02-18T13:58:32.173" Score="2" Body="&lt;p&gt;Not sure if I completely understood your question, but if you want simply an interval of your entire genome  you could do instead:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools faidx MyGenome.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Obviously that means that you need access to that, not sure if you have.&#xA;The resulting fai file contains a format similar to that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;000000F 33203223        94      60      61&#xA;000001F 28828106        33756799        60      61&#xA;000002F 27810542        63065468        60      61&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where the first column is your chromosome and the second one your length.&#xA;I guess you could then just do a &lt;code&gt;awk '{OFS=FS=&quot;\t&quot;}{print $1,&quot;0&quot;,$2-1}'&lt;/code&gt; to get your file.&lt;/p&gt;&#xA;" OwnerUserId="1709" LastActivityDate="2020-02-18T13:58:32.173" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11440" PostTypeId="2" ParentId="11437" CreationDate="2020-02-18T16:16:24.727" Score="1" Body="&lt;p&gt;An ultrametric tree is simply a global molecular clock and can be easily generate in PAML using a single evolutionary rate. You take you RAxML tree in phylip format (PAML might do nexus format know), and click the &quot;clock function&quot;. PAML will do the rest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can alternatively generate it de novo via UPGMA neighbor-joining tree which is found in the MEGAX package (which is probably easier and considered a technical definition of an ultrametric tree). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A global molecular clock (as opposed to a local molecular clock) is quite a specific request, but if you performed a likelihood ratio test, it would identify &quot;clock-like&quot; behaviour.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-02-18T16:58:39.830" LastActivityDate="2020-02-18T16:58:39.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11442" PostTypeId="1" CreationDate="2020-02-18T16:42:00.117" Score="1" ViewCount="26" Body="&lt;p&gt;I have three types of transcripts for &lt;em&gt;Rosa chinensis&lt;/em&gt; &quot;Old Blush&quot; homozygous genome v2.0 from GDR genome browser.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;RcHm_v2.0_CPg0501991&#xA;RcHm_v2.0_Chr0c12g0499591  or RcHm_v2.0_Chr1g0313011&#xA;RcHm_v2.0_MTg0498341&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Does anyone know what this means. At least &quot;CPg&quot; and &quot;Mtg&quot;. I know &quot;Chr&quot; means chromosome. And there is &quot;1&quot; at each and every transcript. But it is not separated using anything, like people do to mention the transcript number. I will be grateful if someone can help.&#xA;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="856" LastActivityDate="2020-02-18T16:42:00.117" Title="What are the meanings of these transcript ids?" Tags="|proteins|transcriptome|rna|phylogenetics|nomenclature|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11444" PostTypeId="2" ParentId="11432" CreationDate="2020-02-18T18:02:08.557" Score="1" Body="&lt;p&gt;Keep it simple:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '!a[$1]++' file.txt &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you have a different command line separator you need to specify with the &lt;code&gt;-F&lt;/code&gt; option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Space&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -F' ' '!a[$1]++' text.txt &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Tab&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -F'\t' '!a[$1]++' text.txt &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you don't have the first column in order maybe should be a good idea sort first. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sort text.txt | awk -F' ' '!a[$1]++'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The name of the array in this case is &lt;code&gt;a&lt;/code&gt;. There is a better explanation in &lt;a href=&quot;https://superuser.com/a/1107659&quot;&gt;this post&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2020-02-18T18:02:08.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11445" PostTypeId="1" CreationDate="2020-02-18T18:00:16.417" Score="1" ViewCount="1332" Body="&lt;p&gt;I'm new here. I need to construct a phylogenetic tree from 16s Sequence data (merged 151 bp) and color the branches/nodes based on my experimental conditions. From &lt;a href=&quot;https://towardsdatascience.com/molecular-phylogenetics-using-bio-phylo-57ce27492ee9&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog&lt;/a&gt;, it looks like I need a .phy file to make the tree. I want to use Biopython for this since I'm reasonably comfortable with python.&lt;/p&gt;&#xA;&lt;p&gt;I have a csv file with the 16s sequences, the phylogenetic information, and the &amp;quot;name&amp;quot; of the sequence, and the experimental conditions. How can I make a .phy file from this?&lt;/p&gt;&#xA;&lt;p&gt;I would appreciate any help.&#xA;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="7244" OwnerDisplayName="Kaumz" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2020-02-20T13:17:09.977" Title="How to create a .phy file for constructing a phylogenetic tree?" Tags="|phylogenetics|python|phylogeny|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11446" PostTypeId="2" ParentId="11445" CreationDate="2020-02-18T19:39:14.560" Score="1" Body="&lt;p&gt;You simply use Biopython's SeqIO and AlignIO.&#xA;The documentation is &lt;a href=&quot;https://biopython.org/wiki/AlignIO&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biopython.org/wiki/AlignIO&lt;/a&gt; &#xA;etc ...&#xA;The format you seek is phylip&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-02-18T19:39:14.560" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11448" PostTypeId="2" ParentId="11445" CreationDate="2020-02-19T00:29:43.177" Score="1" Body="&lt;p&gt;Python works great for tying everything together and file manipulations.  I find stepping out of Python, using system calls, best for alignment and tree building.  For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;os.system(&quot;mafft --auto {} &amp;gt; {}&quot; .format(&quot;unaligned_16s_seq_in_file.fasta&quot;, &quot;alignment_out_file.fasta&quot;))&#xA;&#xA;os.system(&quot;raxml -f a -s {} -p 12345 -x 12345 -# 100 -m GTRCAT -n {}&quot; .format(&quot;alignment_out_file.fasta&quot;, &quot;tree_out_file.tre&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6343" LastActivityDate="2020-02-19T00:29:43.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11449" PostTypeId="1" CreationDate="2020-02-19T01:24:37.053" Score="1" ViewCount="87" Body="&lt;p&gt;I'm reading a virology paper where the authors made a classification for the patients as to whether they had a specific disease outcome or not against the underlying sequence ...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The research focused on two different class of DNA which are Hepatitis B Virus&#xA;(HBV) Genotype B nucleotide sequence and Non-HBV nucleotide sequences`&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I get the data set or how can search to find it ?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How can data sets of sequences date classify between illness and asymptomatic patients?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7248" LastEditorUserId="3848" LastEditDate="2020-11-28T17:39:12.683" LastActivityDate="2020-11-28T17:39:12.683" Title="How can i get data set of illness and not illness people?" Tags="|phylogenetics|phylogeny|covid-19|nucleotide-models|sars-cov-2|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11450" PostTypeId="1" CreationDate="2020-02-19T01:37:53.527" Score="1" ViewCount="68" Body="&lt;p&gt;I found &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/3510/57&quot;&gt;a very good link&lt;/a&gt; to generate bwt suffix array in BW based tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;@user172818 provides a nice code. could anyone tell me how to store SA elements' original position in the original string?  &lt;/p&gt;&#xA;" OwnerUserId="7246" LastEditorUserId="57" LastEditDate="2020-02-19T10:18:12.010" LastActivityDate="2020-02-19T10:18:12.010" Title="bwt suffix array construction" Tags="|bwa|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11451" PostTypeId="2" ParentId="10689" CreationDate="2020-02-19T03:51:06.657" Score="2" Body="&lt;p&gt;I recommend using &lt;code&gt;densitree&lt;/code&gt;: &lt;a href=&quot;https://www.cs.auckland.ac.nz/~remco/DensiTree/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cs.auckland.ac.nz/~remco/DensiTree/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And &lt;code&gt;rwty&lt;/code&gt;: &lt;a href=&quot;https://github.com/danlwarren/RWTY&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/danlwarren/RWTY&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Densitree plots the superimposed set of trees in a heatmap-like format: &lt;a href=&quot;https://i.stack.imgur.com/6LcOe.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6LcOe.jpg&quot; alt=&quot;densitree&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And &lt;code&gt;rwty&lt;/code&gt;, a very powerful &lt;code&gt;R&lt;/code&gt; package that allows visualisation of treespace in an NMDS format (among other methods): &lt;a href=&quot;https://i.stack.imgur.com/S2fsR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/S2fsR.png&quot; alt=&quot;rwty&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3381" LastEditorUserId="3381" LastEditDate="2020-03-10T01:56:50.807" LastActivityDate="2020-03-10T01:56:50.807" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11452" PostTypeId="2" ParentId="11418" CreationDate="2020-02-19T04:20:52.253" Score="0" Body="&lt;p&gt;A situation where having phasing data is useful is with allele-specific protein binding.&#xA;Some individuals will have a heterozygous mutation within the binding site of a particular protein.&#xA;If you perform a standard chromatin immunoprecipitation (ChIP) sequencing assay, you'll find sequencing reads around the protein's binding site, but you won't know which allele they'll come from since sequencing is an ensemble measurement.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if you have phasing information, you can measure whether this specific mutation has an effect on that protein's binding, and get some insight into the function of that mutation in the context of that individual's cells.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2020-02-19T04:20:52.253" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11453" PostTypeId="1" CreationDate="2020-02-19T05:16:37.840" Score="1" ViewCount="16" Body="&lt;p&gt;How is it possible to get the SwissProt Proteins into my Jbrowser in the same way like shown below?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jbxR0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jbxR0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance,&lt;/p&gt;&#xA;" OwnerUserId="2477" LastActivityDate="2020-02-19T05:16:37.840" Title="SwissProt Proteins into my Jbrowser" Tags="|functional-annotation|sequence-annotation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11454" PostTypeId="1" CreationDate="2020-02-19T11:03:25.273" Score="5" ViewCount="348" Body="&lt;p&gt;I am an avid user of Snakemake. Recently we have been refreshing our pipelines and I saw that a cluster.json file is no longer the recommended way to store the cluster configuration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used to start my pipelines like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;snakemake --cluster-config cluster.json --cluster &quot;qsub -W umask=002 -d cluster_outputs  -l nodes={cluster.nodes}:ppn={cluster.ppn} -N {rule}&quot; --jobs 245 --use-conda --rerun-incomplete --keep-going --latency-wait 60&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With the cluster.json looking like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;    &quot;__default__&quot; :&#xA;    {&#xA;        &quot;cluster&quot;   : &quot;qsub&quot;,&#xA;        &quot;jobs&quot;      : 70 ,&#xA;        &quot;nodes&quot;     : 1,&#xA;        &quot;ppn&quot;       : 1&#xA;    },&#xA;    &quot;rsem_quantify&quot; :&#xA;    {&#xA;        &quot;ppn&quot;       : 8&#xA;    },&#xA;    &quot;star_map_human&quot; :&#xA;    {&#xA;        &quot;ppn&quot;       : 4&#xA;    },&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, I understand that some of this goes in to the new profiles.yaml like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cluster: qsub&#xA;jobs: 70&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But how do I set the ppn values per rule? And how do I set the other configuration options into profile.yaml (like: snakemake &quot;qsub -W umask=002 -d cluster_outputs  -l nodes={cluster.nodes}:ppn={cluster.ppn} -N {rule}&quot;, etc.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any examples are appreciated, &lt;a href=&quot;https://github.com/snakemake-profiles&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/snakemake-profiles&lt;/a&gt; is not really helping&lt;/p&gt;&#xA;" OwnerUserId="3964" LastActivityDate="2020-02-19T19:38:12.100" Title="Snakemake: Migrating from deprecated cluster.json to new profiles.yaml" Tags="|snakemake|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11455" PostTypeId="2" ParentId="11431" CreationDate="2020-02-19T15:05:52.753" Score="1" Body="&lt;p&gt;After long years of just conceptual knowledge of what the software does, I was dealing with &lt;code&gt;RepearMasker&lt;/code&gt; this week. So, I am not entirely sure I am correct, but this is how I understand it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The two flags you explored serve for different scenarios:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;-species&lt;/code&gt; - if the taxa/species is present in the repeat database (&lt;code&gt;repBase&lt;/code&gt; or &lt;code&gt;Dfam&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;-lib&lt;/code&gt; - if you want to give repeatModeler something different.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In your case it seems like your library is completely missing all LINEs, SINEs and a few others. What you can do is to get from the database all repetitions of the corresponding taxa and catinate it with the repeats you are interested in (the one you supplied in the &lt;code&gt;-lib&lt;/code&gt; argument in the first command).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;util/queryRepeatDatabase.pl -species Vertebrates &amp;gt; flavoured_vertebrate_repetitions.lib&#xA;cat myOwnLibrary.fa &amp;gt;&amp;gt; flavoured_vertebrate_repetitions.lib&#xA;RepeatMasker -lib flavoured_vertebrate_repetitions.lib Mygenome.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This should produce annotation similar to the bottom case, but also using the repetitions you manually added.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. &lt;code&gt;util/queryRepeatDatabase.pl&lt;/code&gt; is a script to query the database of repeats and it is shipped together with &lt;code&gt;repeatModeler&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- edit --&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The TE classification in the summary file of repeatMasker is based on headers of the repetitive elements in the fasta file you specify in to &lt;code&gt;-lib&lt;/code&gt; argument. So if you want to get the second-table-like, you need to somehow reclassify the repeat library to contan repBase-like classification of TEs. The explanation of lib argument from the &lt;a href=&quot;https://www.animalgenome.org/bioinfo/resources/manuals/RepeatMasker.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The recommended format for IDs in a custom library is:&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&gt;repeatname#class/subclass or simply&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;&gt;repeatname#class&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;In this format, the data will be processed (overlapping repeats are&#xA;  merged etc), alternative output (.ace or .gff) can be created and an&#xA;  overview .tbl file will be created. Classes that will be displayed in&#xA;  the .tbl file are 'SINE', 'LINE', 'LTR', 'DNA', 'Satellite', anything&#xA;  with 'RNA' in it, 'Simple_repeat', and 'Other' or 'Unknown' (the&#xA;  latter defaults when class is missing). Subclasses are plentiful. They&#xA;  are not all tabulated in the .tbl file or necessarily spelled&#xA;  identically as in the repeat files, so check the RepeatMasker.embl&#xA;  file for names that can be parsed into the .tbl file.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2020-02-26T16:54:24.577" LastActivityDate="2020-02-26T16:54:24.577" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11456" PostTypeId="2" ParentId="11454" CreationDate="2020-02-19T19:38:12.100" Score="1" Body="&lt;p&gt;It looks like you can just bundle your cluster config in its own YAML file, and reference that from within a profile YAML.  Here's a clumsy little example using a fake job submission script: &lt;a href=&quot;https://github.com/ressy/example-snakemake-profiles&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ressy/example-snakemake-profiles&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/snakefiles/configuration.html#cluster-configuration-deprecated&quot; rel=&quot;nofollow noreferrer&quot;&gt;current documentation&lt;/a&gt; makes it sound like cluster configuration support is going away and being replaced by profiles:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;While still being possible, cluster configuration has been deprecated by the introduction of Profiles.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;...but from those example profiles, that might be misleading; here's an example template for a profile YAML that references a separate cluster-config YAML:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/Snakemake-Profiles/generic/blob/master/%7B%7Bcookiecutter.profile_name%7D%7D/config.yaml&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Snakemake-Profiles/generic/blob/master/%7B%7Bcookiecutter.profile_name%7D%7D/config.yaml&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;cluster-config: &quot;$((INSTALDIR))/overwrite_cluster_param.yaml&quot; #abs path&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That cluster-config YAML has the same default and per-rule entries just your JSON version does:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/Snakemake-Profiles/generic/blob/master/%7B%7Bcookiecutter.profile_name%7D%7D/overwrite_cluster_param.yaml&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Snakemake-Profiles/generic/blob/master/%7B%7Bcookiecutter.profile_name%7D%7D/overwrite_cluster_param.yaml&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;# specific parameters for certain rules, which need more time/memory&#xA;&#xA;#run_assembler:&#xA;#  queue: bigmem&#xA;#  time: 100&#xA;#   threads and memory definen in config file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So I think you can just set up a pair of YAML files accordingly including options for your rsem_quantify and star_map_human rules.  From my own test it looks like it handles the cluster argument templating just as it did before when using a profile.  (If this is correct we should nag them to make the new documentation more explicit though.)&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2020-02-19T19:38:12.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11457" PostTypeId="1" CreationDate="2020-02-19T22:32:03.520" Score="2" ViewCount="109" Body="&lt;p&gt;This isn't quite my speciality, but I've been tasked with finding any evidence of Mobile Genetic Elements (like transposable elements, retrotransposons, bacteriophages, etc.) within some WGS samples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a &quot;definitive&quot; database used with sequences? How &quot;species-specific&quot; would this be?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, are there standard algorithms used to find these MGEs? I suspect we don't have a comprehensive database of all possible MGEs, but it may be possible to query against the sequences we do indeed have (or detect any repetitive elements within the genome). &lt;/p&gt;&#xA;" OwnerUserId="1770" LastActivityDate="2020-02-21T22:59:02.440" Title="Is there a &quot;definitive&quot; database for Mobile Genetic Elements?" Tags="|blast|repeat-elements|transposable-elements|bacteria|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11458" PostTypeId="2" ParentId="11449" CreationDate="2020-02-20T05:00:26.107" Score="2" Body="&lt;p&gt;The data set will be available on &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genbank&lt;/a&gt;, you simply search for the &quot;authors surname and HBV&quot; and their entire data set should be revealed. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The HBV genome is 5Kb, the smallest viral genome so its not alot of data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am confused by &quot;Genotype B nucleotide sequence and non-HBV nucleotide sequences&quot;. What I suspect is the investigators looked at HBV genotype B in comparison to HBV genotypes A, C to I. Essentially they wanted to assess the relationship between genetic variation and pathogenesis, which is an attempt to assess virus-induced cancer. Obviously I don't know the paper but I'm aware of the basic research front in this area. HBV has a very wierd evolution and I personally wouldn't want to investigate it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-02-20T05:00:26.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11459" PostTypeId="2" ParentId="10881" CreationDate="2020-02-20T09:37:27.830" Score="1" Body="&lt;p&gt;There's no default for the &lt;code&gt;export_to&lt;/code&gt; option, so you just need to specify &lt;code&gt;export_to=&quot;Seurat&quot;&lt;/code&gt;. Note that monocle2 doesn't seem to be compatible with the newest Seurat (3.1) releases.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-02-20T09:37:27.830" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11460" PostTypeId="1" CreationDate="2020-02-20T12:15:34.050" Score="-1" ViewCount="92" Body="&lt;p&gt;There is a function does need an input as selected genomic site across genomic regions. But my genomic region bed file already is a subset of interesting site (like promoters). I want this function to work not complaining about genomic site&#xA;However when I leave genomic site says &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; prepare_elements_regions_tf &amp;lt;- prepare_elements(&quot;regions.bed&quot;,&#xA;+                                                 window_size = 50000)&#xA;Successfully read elements&#xA; Error in utils::read.delim(sites_file, stringsAsFactors = FALSE, header = FALSE) : &#xA;  argument &quot;sites_file&quot; is missing, with no default &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The whole function is here &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/wenmm/ActiveDriverWGS/blob/master/R/ActiveDriverWGS.R&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/wenmm/ActiveDriverWGS/blob/master/R/ActiveDriverWGS.R&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But part of function that nags is this &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;prepare_elements = function(elements_file, sites_file, window_size = 50000, mc.cores = 1, recovery_dir = NA) {&#xA;&#xA;    try_error = &quot;try-error&quot;&#xA;&#xA;&#xA;&#xA;    element_coords_filename = paste0(recovery_dir, &quot;/ADWGS_elements_coords_recovery_file.rsav&quot;)&#xA;&#xA;    element_coords_1 = NULL&#xA;&#xA;    load_result = suppressWarnings(try(load(element_coords_filename), silent = TRUE))&#xA;&#xA;    if (class(load_result) == try_error) {&#xA;&#xA;        element_coords_1 = prepare_element_coords_from_BED(elements_file)&#xA;&#xA;        if (!is.na(recovery_dir)) {&#xA;&#xA;            save(element_coords_1, file = element_coords_filename)&#xA;&#xA;        }&#xA;&#xA;        cat(&quot;Successfully read elements\n&quot;)&#xA;&#xA;    } else {&#xA;&#xA;        cat(&quot;Recovered elements\n&quot;)&#xA;&#xA;    }&#xA;&#xA;&#xA;&#xA;    element_coords = coords_sanity(element_coords_1)&#xA;&#xA;&#xA;&#xA;    element_3n_filename = paste0(recovery_dir, &quot;/ADWGS_element_3n_recovery_file.rsav&quot;)&#xA;&#xA;    element_3n = NULL&#xA;&#xA;    load_result = suppressWarnings(try(load(element_3n_filename), silent = TRUE))&#xA;&#xA;    if (class(load_result) == try_error) {&#xA;&#xA;        element_3n = add_trinucs_by_elements(get_3n(element_coords))&#xA;&#xA;        if (!is.na(recovery_dir)) {&#xA;&#xA;            save(element_3n, file = element_3n_filename)&#xA;&#xA;        }&#xA;&#xA;    }&#xA;&#xA;&#xA;&#xA;    window_coords_filename = paste0(recovery_dir, &quot;/ADWGS_window_coords_recovery_file.rsav&quot;)&#xA;&#xA;    window_coords = NULL&#xA;&#xA;    load_result = suppressWarnings(try(load(window_coords_filename), silent = TRUE))&#xA;&#xA;    if (class(load_result) == try_error) {&#xA;&#xA;        window_coords = coords_sanity(get_window_coords(element_coords, window_size))&#xA;&#xA;        if (!is.na(recovery_dir)) {&#xA;&#xA;            save(window_coords, file = window_coords_filename)&#xA;&#xA;        }&#xA;&#xA;    }&#xA;&#xA;&#xA;&#xA;    window_3n_filename = paste0(recovery_dir, &quot;/ADWGS_window_3n_recovery_file.rsav&quot;)&#xA;&#xA;    window_3n = NULL&#xA;&#xA;    load_result = suppressWarnings(try(load(window_3n_filename), silent = TRUE))&#xA;&#xA;    if (class(load_result) == try_error) {&#xA;&#xA;        window_3n = add_trinucs_by_elements(get_3n(window_coords))&#xA;&#xA;        if (!is.na(recovery_dir)) {&#xA;&#xA;            save(window_3n, file = window_3n_filename)&#xA;&#xA;        }&#xA;&#xA;    }&#xA;&#xA;&#xA;&#xA;    raw_site_coords_filename = paste0(recovery_dir, &quot;/ADWGS_raw_site_coords_recovery_file.rsav&quot;)&#xA;&#xA;    site_coords_filename = paste0(recovery_dir, &quot;/ADWGS_site_coords_recovery_file.rsav&quot;)&#xA;&#xA;    raw_site_coords = NULL&#xA;&#xA;    site_coords = NULL&#xA;&#xA;    sites_load_result = suppressWarnings(try(load(site_coords_filename), silent = TRUE))&#xA;&#xA;    if (class(sites_load_result) == try_error) {&#xA;&#xA;        load_result = suppressWarnings(try(load(raw_site_coords_filename), silent = TRUE))&#xA;&#xA;        if (class(load_result) == try_error) {&#xA;&#xA;            raw_site_coords = utils::read.delim(sites_file, stringsAsFactors = FALSE, header = FALSE)&#xA;&#xA;            colnames(raw_site_coords) = c(&quot;chr&quot;, &quot;starts&quot;, &quot;ends&quot;, &quot;site_info&quot;)&#xA;&#xA;            raw_site_coords&lt;span class=&quot;math-container&quot;&gt;$chr = gsub(&quot;chr&quot;, &quot;&quot;, raw_site_coords$&lt;/span&gt;chr, ignore.case = TRUE)&#xA;&#xA;            raw_site_coords&lt;span class=&quot;math-container&quot;&gt;$chr = paste0(&quot;chr&quot;, raw_site_coords$&lt;/span&gt;chr)&#xA;&#xA;            if (!is.na(recovery_dir)) {&#xA;&#xA;                save(raw_site_coords, file = raw_site_coords_filename)&#xA;&#xA;            }&#xA;&#xA;            cat(&quot;Successfully read active sites\n&quot;)&#xA;&#xA;        } else {&#xA;&#xA;            cat(&quot;Recovered active sites\n&quot;)&#xA;&#xA;        }&#xA;&#xA;&#xA;&#xA;        if (!is.null(raw_site_coords[[1]])) {&#xA;&#xA;            # first only keep sites that overlap with element coordinates&#xA;&#xA;            site_gr = GenomicRanges::GRanges(raw_site_coords$chr,&#xA;&#xA;                IRanges::IRanges(start=raw_site_coords&lt;span class=&quot;math-container&quot;&gt;$starts, end=raw_site_coords$&lt;/span&gt;ends))&#xA;&#xA;            element_gr = GenomicRanges::GRanges(element_coords$chr, &#xA;&#xA;                IRanges::IRanges(start=element_coords&lt;span class=&quot;math-container&quot;&gt;$starts, end=element_coords$&lt;/span&gt;ends))&#xA;&#xA;&#xA;&#xA;            rm(raw_site_coords)&#xA;&#xA;&#xA;&#xA;            overlaps_found = GenomicRanges::findOverlaps(element_gr, site_gr)&#xA;&#xA;            site_here_gr = site_gr[S4Vectors::subjectHits(overlaps_found)]&#xA;&#xA;&#xA;&#xA;            rm(site_gr, element_gr, overlaps_found)&#xA;&#xA;&#xA;&#xA;            site_coords = do.call(rbind, parallel::mclapply(1:nrow(element_coords), &#xA;&#xA;                get_sites_per_element, element_coords, site_here_gr, mc.cores=mc.cores))&#xA;&#xA;            rm(site_here_gr)&#xA;&#xA;            site_coords = coords_sanity(site_coords)&#xA;&#xA;        }&#xA;&#xA;        if (!is.na(recovery_dir)) {&#xA;&#xA;            save(site_coords, file = site_coords_filename)&#xA;&#xA;        }&#xA;&#xA;    } else {&#xA;&#xA;        cat(&quot;Recovered active sites\n&quot;)&#xA;&#xA;    }&#xA;&#xA;    rm(sites_load_result, element_coords)&#xA;&#xA;&#xA;&#xA;    site_3n_filename = paste0(recovery_dir, &quot;/ADWGS_site_3n_recovery_file.rsav&quot;)&#xA;&#xA;    site_3n = NULL&#xA;&#xA;    load_result = suppressWarnings(try(load(site_3n_filename), silent = TRUE))&#xA;&#xA;    if (class(load_result) == try_error) {&#xA;&#xA;        site_3n = get_3n(site_coords)&#xA;&#xA;        if (!is.na(recovery_dir)) {&#xA;&#xA;            save(site_3n, file = site_3n_filename)&#xA;&#xA;        }&#xA;&#xA;    }&#xA;&#xA;&#xA;&#xA;    list(element_coords=element_coords_1, element_3n=element_3n, window_coords=window_coords, window_3n=window_3n, site_coords=site_coords, site_3n=site_3n)&#xA;&#xA;&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How I get rid of nagging part?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2021-01-03T09:04:45.433" Title="How I change a function not to need an input" Tags="|r|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11461" PostTypeId="2" ParentId="11457" CreationDate="2020-02-20T14:52:27.747" Score="1" Body="&lt;p&gt;Back in the Day™ &lt;a href=&quot;http://www.repeatmasker.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RepeatMasker&lt;/a&gt;&lt;sup&gt;[1]&lt;/sup&gt; was the go-to tool everyone used for TE annotation in eukaryotes. At the time, the &lt;a href=&quot;https://www.girinst.org/repbase/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RepBase database from GIRI&lt;/a&gt;&lt;sup&gt;[2]&lt;/sup&gt; (license required) was the &lt;em&gt;de facto&lt;/em&gt; database recommended for RepeatMasker users. I know &lt;em&gt;de facto&lt;/em&gt; isn't necessarily &lt;em&gt;definitive&lt;/em&gt;, but it may have been the only public database supported for some time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like &lt;a href=&quot;https://github.com/rmhubley/RepeatMasker#libraries-overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;more recent efforts&lt;/a&gt; have focused on developing the &lt;a href=&quot;https://dfam.org/home&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dfam database&lt;/a&gt;, mirrored after the design of databases like Pfam and Rfam.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;[1]&lt;/sup&gt;Smit AFA, Hubley R, Green P (2013-2015) RepeatMasker Open-4.0, &lt;a href=&quot;http://www.repeatmasker.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.repeatmasker.org&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;[2]&lt;/sup&gt;Jurka J, Kapitonov VV, Pavlicek A, Klonowski P, Kohany O, Walichiewicz J (2005) Repbase Update, a database of eukaryotic repetitive elements. &lt;em&gt;Cytogentic and Genome Research&lt;/em&gt; 110:462-467, &lt;a href=&quot;https://doi.org/10.1159/000084979&quot; rel=&quot;nofollow noreferrer&quot;&gt;doi:10.1159/000084979&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2020-02-20T14:52:27.747" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11462" PostTypeId="2" ParentId="11460" CreationDate="2020-02-20T17:34:03.507" Score="1" Body="&lt;p&gt;Think about it this way - your &lt;code&gt;elements_file&lt;/code&gt; already has only the sites of interest and hence does not need to be subject to a subset operation. In other words, if you were to have a &lt;code&gt;sites_file&lt;/code&gt;, it would have the same content as the &lt;code&gt;elements_file&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you see where I am going with this?&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2020-02-20T17:34:03.507" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11463" PostTypeId="1" AcceptedAnswerId="11468" CreationDate="2020-02-20T17:48:40.397" Score="0" ViewCount="232" Body="&lt;p&gt;My current settings are &lt;code&gt;megahit -r input.fq --num-cpu-threads 32 --min-contig-len 300 --presets meta-large -o output&lt;/code&gt;. I picked the 'presets meta-large' because I am dealing with a quite large and diverse metagenome out of which I only need some genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've got a contig of interest ~19kb, with a depth of 16. Could I make this contig larger with more relaxed settings? Help would be appreciated!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2020-02-20T22:03:54.463" Title="Can I get longer contigs by changing MEGAHIT settings?" Tags="|assembly|metagenome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11464" PostTypeId="2" ParentId="11457" CreationDate="2020-02-20T18:38:58.330" Score="2" Body="&lt;p&gt;Would advise doing Denovo identification method structure-based. Then use these public databases (  RepBase, Dfam ) to look for high similarity or homology exploration, this possibility if your species, the repeats not yet discovered, or deposited in the databases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the species well-investigated earlier and the mobile element recognised, I assume it will be a straightforward task using species-specific search via Repeatmasker tool with the -species flag to obtain the MobileElement annotation in the genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me know if you need any data from Repbase related to your species, i have an entree. &lt;/p&gt;&#xA;" OwnerUserId="3332" LastEditorUserId="3332" LastEditDate="2020-02-21T22:59:02.440" LastActivityDate="2020-02-21T22:59:02.440" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11465" PostTypeId="1" AcceptedAnswerId="11466" CreationDate="2020-02-20T19:12:39.510" Score="2" ViewCount="3490" Body="&lt;p&gt;I have a large BAM file and I want to filter it by just what's defined in a bed file, and write it to a new file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to do this with samtools, and if so, how? Is there a better tool I could use to do this?&lt;/p&gt;&#xA;" OwnerUserId="7259" LastActivityDate="2023-05-23T08:32:04.360" Title="How to filter a SAM file by a bed file?" Tags="|sam|samtools|pysam|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11466" PostTypeId="2" ParentId="11465" CreationDate="2020-02-20T21:06:20.237" Score="5" Body="&lt;p&gt;Thanks to the helpful comments I figured it out:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -b -h -L bedfile.bed originalbam.bam &amp;gt; newbam.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;-b = Output as bam file&lt;br /&gt;&#xA;-h = include header&lt;br /&gt;&#xA;-L = only output alignments overlapping the input BED file&lt;/p&gt;&#xA;" OwnerUserId="7259" LastEditorUserId="490" LastEditDate="2023-05-09T02:53:27.677" LastActivityDate="2023-05-09T02:53:27.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11467" PostTypeId="2" ParentId="11427" CreationDate="2020-02-20T21:26:17.393" Score="3" Body="&lt;p&gt;The GDC API user guide (&lt;a href=&quot;https://docs.gdc.cancer.gov/API/PDF/API_UG.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.gdc.cancer.gov/API/PDF/API_UG.pdf&lt;/a&gt;) provides some guidance on the structure of GDC data. So for getting RNA-Seq data from GDC, you would provide this filter to get the 551 text files with workflow_type set as &quot;HTSeq - FPKM-UQ&quot; and &quot;TCGA-LUSC&quot; for the project. There are other workflow types as well:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cases.project.program.name in [&quot;TCGA&quot;]&#xA;and cases.project.project_id in [&quot;TCGA-LUSC&quot;]&#xA;and files.access in [&quot;open&quot;]&#xA;and files.analysis.workflow_type in [&quot;HTSeq - FPKM-UQ&quot;]&#xA;and files.data_format in [&quot;txt&quot;]&#xA;and files.data_type in [&quot;Gene Expression Quantification&quot;]&#xA;and files.experimental_strategy in [&quot;RNA-Seq&quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(You don’t need all those filters to get the 551 files; I included them to show what is available.) The txt files are returned as gzipped tar files that hold text files with rows of the form:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ENSG00000158486.12  5341.21601153&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The above filters provide all the FPKM-UQ files for TCGA-LUSC. If you wanted to just get the data for normal samples as controls, that would require filtering on sample type as well, adding in the additional filter:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;and cases.samples.sample_type in [“solid tissue normal”]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That will return 49 files. If you instead wished to get only the tumor files, the additional filter would instead be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;and cases.samples.sample_type in [“primary tumor”] &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and that returns 502 files. Note that there are several different sample types for tumor and normal tissues; those just happen to be the ones used here.&#xA;(Note that if you are using the GDC Repository portal, adding that additional filter requires you to select “Add a Case/Biospecimen Filter” on the “Cases” filter tab.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since GDC is part of the NCI Cancer Research Data Commons (CRDC), instead of downloading the data, you can actually work with it in the cloud using the various CRDC Cloud Resources such as the ISB-CGC: &lt;a href=&quot;https://www.isb-cgc.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.isb-cgc.org/&lt;/a&gt;. In fact, all the RNA-Seq data for TCGA is in an ISB-CGC Google BigQuery table: (&lt;a href=&quot;https://console.cloud.google.com/bigquery?p=isb-cgc&amp;amp;d=TCGA_hg38_data_v0&amp;amp;t=RNAseq_Gene_Expression&amp;amp;page=table&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://console.cloud.google.com/bigquery?p=isb-cgc&amp;amp;d=TCGA_hg38_data_v0&amp;amp;t=RNAseq_Gene_Expression&amp;amp;page=table&lt;/a&gt;)&#xA;And the above count data can be pulled using the query:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SELECT HTSeq__FPKM_UQ&#xA;FROM `isb-cgc.TCGA_hg38_data_v0.RNAseq_Gene_Expression`&#xA;WHERE Ensembl_gene_id_v = &quot;ENSG00000158486.12&quot; AND project_short_name = &quot;TCGA-LUSC&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you want to discriminate between normal and tumor expression levels, the ISB-CGC BQ tables make it pretty easy to tag the expression values with the type of tissue involved. The sample type can be assigned by joining the previous query with the biospecimen table:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;WITH bio as (&#xA;SELECT sample_barcode, sample_type_name&#xA;FROM `isb-cgc.TCGA_bioclin_v0.Biospecimen`&#xA;),&#xA;expr as (SELECT sample_barcode, HTSeq__FPKM_UQ&#xA;FROM `isb-cgc.TCGA_hg38_data_v0.RNAseq_Gene_Expression`&#xA;WHERE Ensembl_gene_id_v = &quot;ENSG00000158486.12&quot; AND project_short_name = &quot;TCGA-LUSC&quot;)&#xA;SELECT bio.sample_type_name, expr.HTSeq__FPKM_UQ&#xA;FROM expr&#xA;JOIN bio&#xA;ON bio.sample_barcode = expr.sample_barcode&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The GDC has listed some of the available sample types on this page: &lt;a href=&quot;https://gdc.cancer.gov/resources-tcga-users/tcga-code-tables/sample-type-codes&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gdc.cancer.gov/resources-tcga-users/tcga-code-tables/sample-type-codes&lt;/a&gt;&#xA;Also, BigQuery has a user interface that makes exploring data sets and writing queries easy, but you need to log in with a Google ID, so the ISB-CGC has a BigQuery Table Search (&lt;a href=&quot;https://isb-cgc.appspot.com/bq_meta_search/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://isb-cgc.appspot.com/bq_meta_search/&lt;/a&gt;) that allows you to explore the available tables without logging in. If you want to run queries, Google offers a free tier that you can use to try things out and run small jobs (&lt;a href=&quot;https://cloud.google.com/free/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cloud.google.com/free/&lt;/a&gt;) and ISB-CGC offers free credits too. ISB-CGC also has examples of working in R with BigQuery within the Community Notebook Repository (&lt;a href=&quot;https://isb-cancer-genomics-cloud.readthedocs.io/en/latest/sections/HowTos.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://isb-cancer-genomics-cloud.readthedocs.io/en/latest/sections/HowTos.html&lt;/a&gt;). A few useful notebooks would be 'How do I create cohorts of patients?', 'How do I plot a heatmap using data in BigQuery?', and 'How do I get started fast?'.&lt;/p&gt;&#xA;" OwnerUserId="6566" LastEditorUserId="6566" LastEditDate="2020-02-29T01:21:18.073" LastActivityDate="2020-02-29T01:21:18.073" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11468" PostTypeId="2" ParentId="11463" CreationDate="2020-02-20T22:03:54.463" Score="1" Body="&lt;p&gt;You can use the option &lt;code&gt;--k-step 10&lt;/code&gt; and &lt;code&gt;--kmin-1pass&lt;/code&gt; for low-coverage datasets.&lt;/p&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2020-02-20T22:03:54.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11469" PostTypeId="1" CreationDate="2020-02-20T22:30:54.330" Score="1" ViewCount="205" Body="&lt;p&gt;I'm currently using &lt;a href=&quot;https://www.ebi.ac.uk/thornton-srv/software/PROCHECK/download.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Procheck&lt;/a&gt;* to check a protein backbone. The process is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Determine backbone phi, psi, and omega angles&lt;/li&gt;&#xA;&lt;li&gt;Compare to known angle distributions to identify outliers, and angles in disallowed regions&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Is &lt;a href=&quot;https://servicesn.mbi.ucla.edu/WHATCHECK/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Whatcheck&lt;/a&gt;** equivalent for this task?&lt;/p&gt;&#xA;&lt;p&gt;*, checks the stereochemical quality of a protein structure, producing a number of PostScript plots analysing its overall and residue-by-residue geometry. It includes PROCHECK-NMR for checking the quality of structures solved by NMR.&lt;/p&gt;&#xA;&lt;p&gt;**, Derived from a subset of protein verification tools from the WHATIF program (Vriend, 1990), this does extensive checking of many sterochemical parameters of the residues in the model.&lt;/p&gt;&#xA;" OwnerUserId="6966" LastEditorUserId="3848" LastEditDate="2020-11-19T20:33:20.327" LastActivityDate="2020-11-19T20:33:20.327" Title="What's the difference between Procheck and Whatcheck?" Tags="|protein-structure|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11470" PostTypeId="1" AcceptedAnswerId="11472" CreationDate="2020-02-20T23:37:57.450" Score="2" ViewCount="5036" Body="&lt;p&gt;I recently discovered &lt;a href=&quot;https://github.com/snakemake-workflows/rna-seq-star-deseq2&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Snakemake pipeline&lt;/a&gt; for RNASeq that uses STAR's quantMode to quantify gene expression for DESeq2 differential expression analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the past I've always seen the workflow as:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;STAR alignment&lt;/li&gt;&#xA;&lt;li&gt;RSEM/HTSeqCount quantification&lt;/li&gt;&#xA;&lt;li&gt;DESeq2 analysis&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;More recently I've also seen:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Kallisto/Salmon pseudo-alignment quantification&lt;/li&gt;&#xA;&lt;li&gt;DESeq2/Sleuth analysis&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I tried searching online for differences between STAR quantMode vs other quantification algorithms but couldn't find many details. What are the benefits and drawbacks of using STAR quantMode vs RSEM/Kallisto/Salmon?&lt;/p&gt;&#xA;" OwnerUserId="6674" LastActivityDate="2020-02-21T19:53:43.243" Title="STAR quantMode vs RSEM vs Kallisto" Tags="|rna-seq|differential-expression|deseq2|rsem|kallisto|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11471" PostTypeId="1" CreationDate="2020-02-21T02:31:57.870" Score="1" ViewCount="1449" Body="&lt;p&gt;Hello I'm helping to develop a cloud docking tool for screening compounds, similar to Swissdock but with mass throughput and GPU optimizations.&lt;/p&gt;&#xA;&lt;p&gt;Specifically helping screen existing drugs against coronavirus proteins (couple structures solved, rest TASSER).&lt;/p&gt;&#xA;&lt;p&gt;I'm planning to use GPU optimized AutoDock-GPU , which takes in .maps.fld&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I know you can use autogrid to select the bounding box and generate the .maps.fld, but I've been unable to figure out the workflow.&lt;/li&gt;&#xA;&lt;li&gt;for preliminary screening I want to search the whole protein without specifying a bounding box.&lt;/li&gt;&#xA;&lt;li&gt;Is there a script for converting protein.pdb to .maps.fld?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Collaborators welcome.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="7263" LastEditorUserId="3848" LastEditDate="2020-12-04T15:35:55.053" LastActivityDate="2022-10-27T15:41:41.493" Title="Protein ligand docking: how to convert &lt;protein&gt;.pdb to &lt;protein&gt;.maps.fld?" Tags="|protein-structure|covid-19|sars-cov-2|docking|structural-biology|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11472" PostTypeId="2" ParentId="11470" CreationDate="2020-02-21T09:01:16.057" Score="3" Body="&lt;p&gt;STAR quantMode (GeneCounts) essentially provides the same output as HTSeq-Count would, ie. number of reads that cover a given gene. This is the most simple measure of expression you could get from RNA-seq data. Kallisto and Salmon utilize pseudo-alignment to determine expression measures of transcripts (as opposed to genes). RSEM uses some algorithm to determine isoform fractions, but still based on alignments. (Note that I don't have any personal experience using RSEM.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It ultimately depends on the research question which method will be most appropriate for you.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;read counts&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;pros:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;relatively easy to understand concept-wise&lt;/li&gt;&#xA;&lt;li&gt;can be used to generate many different expression measures (FPKMs, TPM, CPM, etc.) and is, thus, more versatile in down-stream analyses&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;cons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;you can't distinguish isoforms (from the counts themselves)&lt;/li&gt;&#xA;&lt;li&gt;requires alignment and, thus, more time to obtain&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h1&gt;salmon/kallisto:&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;pros:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;faster (due to lack of alignment step)&lt;/li&gt;&#xA;&lt;li&gt;quantifies (known) isoforms&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;cons:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;expression values are more abstract, less easy to explain&lt;/li&gt;&#xA;&lt;li&gt;gene-level analyses will require some extra work&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;h1&gt;RSEM&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;Likely lies somewhere in between, still requiring alignments will make it slower. But it does give you expression values for isoforms. &lt;/p&gt;&#xA;" OwnerUserId="6955" LastActivityDate="2020-02-21T09:01:16.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11473" PostTypeId="1" CreationDate="2020-02-21T11:57:01.360" Score="1" ViewCount="733" Body="&lt;p&gt;EDIT: I posted this as a bug, perhaps wait for the result of that? &lt;a href=&quot;https://github.com/snakemake/snakemake/issues/240&quot; rel=&quot;nofollow noreferrer&quot;&gt;github&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using Snakemake to build a pipeline. To define input and output names, I constructed them based on a filename base (from the samples table), a read indicator (like R1, R2) and the fastq file extension, like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samples = pd.read_csv('samples.tsv', sep='\t').set_index('samples', drop=False)&#xA;reads = ['R1', 'R2']&#xA;fastq_file_extension = _001.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule fastqc:&#xA;    input:&#xA;        os.path.join(config['Directories']['fastq_raw_dir'], '{sample}{read}' + fastq_file_extension)&#xA;    output:&#xA;        html=os.path.join(config['Directories']['fastqc_dir'], '{sample}{read}_fastqc.html'),&#xA;        zip=os.path.join(config['Directories']['fastqc_dir'], '{sample}{read}_fastqc.zip')&#xA;    wrapper:&#xA;        &quot;0.49.0/bio/fastqc&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, during pipeline execution already I can see strange things happening, for this rule it reports:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[Fri Feb 21 12:39:57 2020]&#xA;rule fastqc:&#xA;input: fastq_raw/0053_P2017BB3S20R_S2_R1_001.fastq.gz&#xA;output: qc/fastqc/0053_P2017BB3S20R_S2_R1_fastqc.html, qc/fastqc/0053_P2017BB3S20R_S2_R1_fastqc.zip&#xA;jobid: 10&#xA;wildcards: sample=0053_P2017BB3S20R_S2_R, read=1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As you can see, Snakemake pulled the R part of the read to the sample and set the read to 1 (should be R1). But this still works because it produces output files as Snakemake expects.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But the next rule, trim_galore_pe:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule trim_galore_pe:&#xA;    input:&#xA;        [os.path.join(config['Directories']['fastq_raw_dir'], '{sample}' + read + fastq_file_extension) for read in reads]&#xA;    output:&#xA;        fastq_read1 = os.path.join(config['Directories']['fastq_trimmed_dir'], '{sample}.1_val_1.fq.gz'),&#xA;        fastq_read2 = os.path.join(config['Directories']['fastq_trimmed_dir'], '{sample}.2_val_2.fq.gz'),&#xA;        report_read1 = os.path.join(config['Directories']['trim_galore_qc_dir'], '{sample}.1.fastq.gz_trimming_report.txt'),&#xA;        report_read2 = os.path.join(config['Directories']['trim_galore_qc_dir'], '{sample}.2.fastq.gz_trimming_report.txt')&#xA;    params:&#xA;        extra=&quot;--retain_unpaired&quot;&#xA;    wrapper:&#xA;        &quot;0.49.0/bio/trim_galore/pe&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This goes wrong, during pipeline execution Snakemake reports:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[Fri Feb 21 12:40:37 2020]&#xA;Error in rule trim_galore_pe:&#xA;jobid: 7&#xA;output: fastq_trimmed/0053_P2017BB3S20R_S2_.1_val_1.fq.gz, fastq_trimmed/0053_P2017BB3S20R_S2_.2_val_2.fq.gz, qc/trim_galore/0053_P2017BB3S20R_S2_.1.fastq.gz_trimming_report.txt, qc/trim_galore/0053_P2017BB3S20R_S2_.2.fastq.gz_trimming_report..txt&#xA;conda-env: /home/nlv24077/projects/snaqs/snakemake_envs/1b94430b&#xA;cluster_jobid: 11904662.osios&#xA;Error executing rule trim_galore_pe on cluster (jobid: 7, external: 11904662.osios, jobscript: /home/nlv24077/temp/RNASeqPairedEnd/.snakemake/tmp.ooypuklc/snakejob.trim_galore_pe.7.sh).&#xA;For error details see the cluster log and the log files of the involved rule(s).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is understandeble because the file fastq_trimmed/0053_P2017BB3S20R_S2_.1_val_1.fq.gz does not exist, it should be fastq_trimmed/0053_P2017BB3S20R_S2.1_val_1.fq.gz, somehow Snakemake added an extra underscore, which you can see in the execution output, under &quot;wildcards&quot; (0053_P2017BB3S20R_S2_).&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[Fri Feb 21 12:39:57 2020]&#xA;rule trim_galore_pe:&#xA;input: fastq_raw/0053_P2017BB3S20R_S2_R1_001.fastq.gz, fastq_raw/0053_P2017BB3S20R_S2_R2_001.fastq.gz&#xA;output: fastq_trimmed/0053_P2017BB3S20R_S2_.1_val_1.fq.gz, fastq_trimmed/0053_P2017BB3S20R_S2_.2_val_2.fq.gz, qc/trim_galore/0053_P2017BB3S20R_S2_.1.fastq.gz_trimming_report.txt, qc/trim_galore/0053_P2017BB3S20R_S2_.2.fastq.gz_trimming_report..txt&#xA;jobid: 7&#xA;wildcards: sample=0053_P2017BB3S20R_S2_&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So how can I prevent Snakemake from changing the samples and read wildcard in seemingly arbitrary ways?&lt;/p&gt;&#xA;" OwnerUserId="3964" LastEditorUserId="3964" LastEditDate="2020-02-21T12:45:05.590" LastActivityDate="2021-07-18T14:07:50.707" Title="Prevent Snakemake from changing wildcards: sample for every rule" Tags="|snakemake|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11474" PostTypeId="2" ParentId="11473" CreationDate="2020-02-21T14:33:17.483" Score="1" Body="&lt;p&gt;Placing two wildcards directly behind each other will make it impossible for snakemake to determine where one wildcard ends and the other begins. So it is simply taking the longest possible match for the first wildcard (&lt;code&gt;sample&lt;/code&gt;) and then one character for the second (&lt;code&gt;read&lt;/code&gt;). This will also cause the &lt;code&gt;_&lt;/code&gt; to be included in the wildcard value &lt;code&gt;sample&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm assuming that this same pattern is used for the inputs of another rule to refer to the trim_galore_pe output. Hence why the &lt;code&gt;_&lt;/code&gt; is included in the wildcards there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You might be able to fix this, simply by adding a &lt;code&gt;_&lt;/code&gt; in-between &lt;code&gt;{sample}&lt;/code&gt; and &lt;code&gt;{readgroup}&lt;/code&gt; in your first rule (and any other rules using this same pattern):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule fastqc:&#xA;    input:&#xA;        os.path.join(config['Directories']['fastq_raw_dir'], '{sample}_{read}' + fastq_file_extension)&#xA;    output:&#xA;        html=os.path.join(config['Directories']['fastqc_dir'], '{sample}{read}_fastqc.html'),&#xA;        zip=os.path.join(config['Directories']['fastqc_dir'], '{sample}{read}_fastqc.zip')&#xA;    wrapper:&#xA;        &quot;0.49.0/bio/fastqc&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6955" LastActivityDate="2020-02-21T14:33:17.483" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11475" PostTypeId="1" CreationDate="2020-02-21T14:41:39.810" Score="2" ViewCount="189" Body="&lt;p&gt;&lt;strong&gt;I am interested in identifying gene pairs (or better: sets of genes) whose expression is mutually exclusive.&lt;/strong&gt;&#xA;Ideally, both genes (or gene sets) would be widely expressed but I am also interested in cases that are very cell type specific.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Familiar examples of the former would include some of the cyclin genes, in humans e.g. cyclin E and cyfclin B, &lt;a href=&quot;https://en.wikipedia.org/wiki/File:Cyclin_Expression.svg&quot; rel=&quot;nofollow noreferrer&quot;&gt;which are expressed during S-phase and M-phase, respectively.&lt;/a&gt; Another example would be some of the clock-genes, where &lt;a href=&quot;https://www.researchgate.net/publication/232233128_Early_Chronotype_and_Tissue-Specific_Alterations_of_Circadian_Clock_Function_in_Spontaneously_Hypertensive_Rats&quot; rel=&quot;nofollow noreferrer&quot;&gt;Per1 and Per2 are often highly expressed in the suprachiasmatic nucleus (SCN) before midday, and whereas Bmal1 is expressed after midday&lt;/a&gt;.&#xA;However, such genes need not be cyclically expressed, they can simply be very specific markers for certain cell types. For example, GAD65 and GAD67 are very specific for interneurons, whereas CamKIIa is very specific for excitatory glutamatergic cells.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been able to find a single paper so far that systematically tries to find such genes from gene expression data, namely: &lt;a href=&quot;https://www.researchgate.net/publication/221239044_Mining_for_Mutually_Exclusive_Gene_Expressions&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tzanis &amp;amp; Vlahavas (2010) Mining for Mutually Exclusive Gene Expressions&lt;/a&gt;. According to Google Scholar, this paper is cited by a single other paper that does not seem to be relevant. In Tzanis &amp;amp; Vlahavas (2010), they use a &lt;a href=&quot;http://mlkd.csd.auth.gr/mutex/datasets.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAGE library&lt;/a&gt; (from 2002) of short expression sequence tags to find these strong negative associations. By today's standards, the data set is pretty small (basically, it contains 90 samples). The paper itself is hence anything but a canonical source for such negative interactions.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Questions:&lt;/h3&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there a curated database of mutually exclusive gene pairs or gene sets?&lt;/li&gt;&#xA;&lt;li&gt;If not, is there a curated database that would allow me to find such genes?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Otherwise, my current plan would be to use the human or mouse cell atlas single cell data to define a signed, weighted gene expression network.&#xA;I would then search for pairs of clusters/modules, where genes have predominantly positive correlations within a cluster but predominantly negative correlations between clusters.&#xA;So essentially, signed, weighted gene correlation network analysis (WGCNA) with extra steps.&lt;/p&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Has somebody published such an analysis already (the WGCNA literature is pretty substantial)?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there other (recent) related work that I am unaware off?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I am fairly new to bioinformatics, so the lack of papers and resources that I am able to find is likely mostly a reflection of my lack of ability to find the correct search term.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;Addendum&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Similar biostars &lt;a href=&quot;https://www.biostars.org/p/397493/&quot; rel=&quot;nofollow noreferrer&quot;&gt;thread&lt;/a&gt; from 5 months ago, also without an answer.&lt;/p&gt;&#xA;" OwnerUserId="7266" LastEditorUserId="7266" LastEditDate="2020-02-24T15:22:34.453" LastActivityDate="2020-03-04T16:04:32.047" Title="Identifying mutually **exclusive** gene sets" Tags="|scrnaseq|networks|wgcna|correlation|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="11477" PostTypeId="1" CreationDate="2020-02-21T19:30:48.607" Score="1" ViewCount="256" Body="&lt;p&gt;I have called peaks using MACS2 and now I'm trying to run ChIPQC bioconductor package. Can someone please tell me should I use sorted BAM files or unsorted BAM files? I have created sorted BAM files and from that indexes for BAM files. When I was trying run ChIPQC earlier, I got an error. Then I figured out that is because I don't have index files. Then I created indexes after sorting BAM files.&lt;/p&gt;&#xA;" OwnerUserId="856" LastEditorUserId="5077" LastEditDate="2020-09-03T21:55:06.613" LastActivityDate="2020-09-03T21:55:06.613" Title="When running ChIPQC bioconductor package after peak calling, should I use sorted BAM files or unsorted BAM files?" Tags="|r|bioconductor|chip-seq|macs2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11478" PostTypeId="2" ParentId="11470" CreationDate="2020-02-21T19:53:43.243" Score="0" Body="&lt;p&gt;A shorter version...STAR's is convenient, in that it gets done right alongside the alignment, but RSEM and Kallisto are smarter about dealing with ambiguous reads.  So I'd say your results from either of those are superior.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally, I use STAR for alignment, then give the bam to RSEM.  STAR's gene count is good for a reality check to make sure that I got the strandedness right, and RSEM's gene level expected counts are good for DESeq once they are rounded.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-02-21T19:53:43.243" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11479" PostTypeId="2" ParentId="11471" CreationDate="2020-02-22T09:43:38.137" Score="2" Body="&lt;p&gt;There are a few steps in between &lt;code&gt;pdb&lt;/code&gt; and &lt;code&gt;.maps.fld&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here is the list of several scripts that can do tasks for you that you downloaded with autodock MGLTools: &lt;a href=&quot;http://autodock.scripps.edu/faqs-help/faq/where-can-i-find-the-python-scripts-for-preparing-and-analysing-autodock-dockings&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://autodock.scripps.edu/faqs-help/faq/where-can-i-find-the-python-scripts-for-preparing-and-analysing-autodock-dockings&lt;/a&gt;. Look at the &lt;code&gt;prepare_&lt;/code&gt; files. Also note that the scripts come with a pre-bundled python, but you can install these scripts with conda.&#xA;Here are the basic order of things:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;prepare_receptor4.py -r protein.pdb&#xA;prepare_ligand4.py -l ligand.mol2&#xA;prepare_gpf4.py -l ligand.pdbqt -r protein.pdbqt -y&#xA;autogrid4 -p protein.gpf&#xA;prepare_dpf4.py -l ligand.pdbqt -r protein.pdbqt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;partial charge&lt;/h2&gt;&#xA;&lt;p&gt;Your PDB needs partial charges so first you convert it to &lt;code&gt;PDBQT&lt;/code&gt; file.&#xA;This is true for your protein and your ligand. That is no complicated parameterisation for your ligand —which is not necessarily a good thing!&#xA;In this step it is essential to correct protonation if absent each have their own flags.&lt;/p&gt;&#xA;&lt;h2&gt;grid&lt;/h2&gt;&#xA;&lt;p&gt;The you make a grid file (&lt;code&gt;gpf&lt;/code&gt;), which contains your grid. To make a box that spans the whole protein simply use the unit cell dimensions form the PDB. For an mmCIF, if you open the header dictionary you need &lt;code&gt;_pdbx_struct_oper_list.matrix&lt;/code&gt; using &lt;code&gt;MMCIF2Dict&lt;/code&gt; from biopython's PDB submodule. For a PDB file, just search for the &lt;code&gt;CRYST1&lt;/code&gt; line (cf. &lt;a href=&quot;http://www.wwpdb.org/documentation/file-format-content/format33/sect8.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;format&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Once you have have the grid you can run autogrid. This step will create the &lt;code&gt;map.fld&lt;/code&gt;&lt;/p&gt;&#xA;&lt;h3&gt;Why a big box is bad for a VS&lt;/h3&gt;&#xA;&lt;p&gt;However, all those who take this approach opt for a box that is huge: as there is no solvent having a huge bounding has little penalty. However, this is considered a very bad strategy for a virtual screen. This not because it uses up computer resources as these are dirty cheap. Structural knowledge is important and targeting the active site of an enzyme stops it, while binding to the surface does nothing unless it's an interface. So manually reading what the stuff are and choosing the box wisely will save you a lot of time downstream in analysis. There is after all the saying &amp;quot;one week in the lab will save you an hour in the library&amp;quot;...&lt;/p&gt;&#xA;&lt;h2&gt;Protein and ligand&lt;/h2&gt;&#xA;&lt;p&gt;Lastly, merge the protein with the small molecule in a &lt;code&gt;dpf&lt;/code&gt; file.&lt;/p&gt;&#xA;&lt;h2&gt;Caveat against docking with models&lt;/h2&gt;&#xA;&lt;p&gt;It is not a good idea dock against models. Swissmodel (unless virtually identical), I-Tasser, Phyre or EVFold models. Docking is very sensitive to small structural difference that may have resulted from the modelling, so docking to models is highly discouraged. For Coronavirus protein, dock against SARS, which is highly similar. Or at the very least pay strong attention to the I-Tasser C-scores and discard the bottom 2/3 of their models.&lt;/p&gt;&#xA;&lt;h2&gt;Note about coronavirus&lt;/h2&gt;&#xA;&lt;p&gt;Your objective of docking coronavirus may be a bit too late however.&#xA;I did &lt;a href=&quot;https://michelanglo.sgc.ox.ac.uk/r/coronavirus&quot; rel=&quot;nofollow noreferrer&quot;&gt;a wee interactive summary of the literature&lt;/a&gt; about the solved structure which is the protease. Proteases are very easy to rationally design drugs for.&#xA;There are already:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://postera.ai/covid&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Covid Moonshot project&lt;/a&gt; is an open collaboration between a fragment-screening X-ray facility (Diamond XChem) and many researchers from various disciplines (myself included).&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1197287/&quot; rel=&quot;nofollow noreferrer&quot;&gt;a paper&lt;/a&gt; that has an empirically validated coronavirus specific ligand&lt;/li&gt;&#xA;&lt;li&gt;many many manuscripts about virtual screens in bioarxiv/chemarxiv.&lt;/li&gt;&#xA;&lt;li&gt;licensed HIV protease inhibitors lopinavir and ritonavir that were initially suspected to be effective —&lt;a href=&quot;https://clinicaltrials.gov/ct2/show/NCT04252885&quot; rel=&quot;nofollow noreferrer&quot;&gt;a clinical trial&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Also you may want to check out &lt;a href=&quot;https://galaxyproject.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Galaxy project&lt;/a&gt;, a tool to run pipelines for genomics, but have started moving into biochemistry from genomics. Eg. &lt;a href=&quot;https://covid19.galaxyproject.org/cheminformatics/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://covid19.galaxyproject.org/cheminformatics/&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-06-20T11:53:12.170" LastActivityDate="2020-06-20T11:53:12.170" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11480" PostTypeId="1" AcceptedAnswerId="11500" CreationDate="2020-02-22T16:38:47.847" Score="2" ViewCount="52" Body="&lt;p&gt;This is a link to the UCSC Genome Browser showing a portion of Ch 17 containing TP53 &lt;a href=&quot;http://genome.ucsc.edu/cgi-bin/hgTracks?db=hg19&amp;amp;lastVirtModeType=default&amp;amp;lastVirtModeExtraState=&amp;amp;virtModeType=default&amp;amp;virtMode=0&amp;amp;nonVirtPosition=&amp;amp;position=chr17%3A7570000%2D7593868&amp;amp;hgsid=802661817_ENKSAs3qbWfOxWUAXXfwDPZXbC8L&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several representations of the gene of varying length and number of exons. I presume these are equivalent to alternate splicings.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming this presumption is correct(?), how can I determine which result in RNAs that are experimentally verified?&lt;/p&gt;&#xA;" OwnerDisplayName="user7271" LastEditorUserId="3848" LastEditDate="2020-02-22T23:12:17.733" LastActivityDate="2020-02-24T20:00:09.833" Title="Distinguishing Splicings that Produce Functional Transcripts Amongst Alternates Shown" Tags="|genome-browser|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11481" PostTypeId="2" ParentId="11427" CreationDate="2020-02-22T21:07:52.063" Score="0" Body="&lt;p&gt;I find UCSC Xena to be well-organized and easy to use: &lt;a href=&quot;https://xenabrowser.net/datapages/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://xenabrowser.net/datapages/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Plain text files that are clearly labeled for all TCGA data (that is not protected).&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="35" LastEditDate="2020-02-22T21:14:26.100" LastActivityDate="2020-02-22T21:14:26.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11482" PostTypeId="2" ParentId="9229" CreationDate="2020-02-22T23:45:42.910" Score="1" Body="&lt;p&gt;This is my first post and I will add some explanations. When I tried to use this script, &lt;code&gt;print(*map(self.fix_record, self.vcf), sep=&amp;quot;&amp;quot;, end=&amp;quot;&amp;quot;)&lt;/code&gt;, it will increase memory usage because it will load everything into memory and then start to apply fix_record function. So, I just used two lines &lt;code&gt;for line in self.vcf: print(self.fix_record(line), sep=&amp;quot;&amp;quot;, end=&amp;quot;&amp;quot;)&lt;/code&gt;. Because &lt;code&gt;pysam&lt;/code&gt; uses generator by default, if we process one by one by pysam's &lt;code&gt;VariantFile&lt;/code&gt; function, it will consume very little memory. In my case, the VCF file was more than 200GB, so I had to fix this script for my VCF file.&lt;/p&gt;&#xA;&lt;p&gt;Second my VCF file used &amp;quot;*&amp;quot; for empty ALT, so I fixed two lines (1) &lt;code&gt;if &amp;quot;*&amp;quot; in record.alts:&lt;/code&gt; (2) &lt;code&gt;record.alts = tuple(map(lambda alt: ref if alt == &amp;quot;*&amp;quot; else f&amp;quot;{ref}{alt}&amp;quot;, record.alts))&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;# -*- coding: utf-8 -*-&#xA;import argparse&#xA;import sys&#xA;&#xA;import pysam&#xA;&#xA;&#xA;class Application:&#xA;    def __init__(self):&#xA;        self.args = Application.get_args()&#xA;        self.vcf = pysam.VariantFile(sys.stdin)&#xA;        self.fasta = pysam.FastaFile(self.args.fasta)&#xA;&#xA;    @staticmethod&#xA;    def get_args() -&amp;gt; argparse.Namespace:&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        Read command line arguments and set input and output sources.&#xA;&#xA;        :return: parsed command line arguments&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;        parser = argparse.ArgumentParser(prog=&amp;quot;fixvcf.py&amp;quot;, description=&amp;quot;Fixes deletion represented by -&amp;quot;)&#xA;        parser.add_argument('--version', action='version', version='%(prog)s 0.1')&#xA;&#xA;        parser.add_argument(&amp;quot;input&amp;quot;, help=&amp;quot;input file, use - to read from stdin&amp;quot;)&#xA;        parser.add_argument(&amp;quot;-f&amp;quot;, &amp;quot;--fasta&amp;quot;, help=&amp;quot;fasta reference file&amp;quot;, required=True)&#xA;        parser.add_argument(&amp;quot;-o&amp;quot;, &amp;quot;--output&amp;quot;, help=&amp;quot;output file&amp;quot;)&#xA;&#xA;        if len(sys.argv) == 1:&#xA;            parser.print_help(sys.stderr)&#xA;            sys.exit(1)&#xA;&#xA;        args = parser.parse_args()&#xA;&#xA;        if args.output:&#xA;            sys.stdout = open(args.output, &amp;quot;w&amp;quot;)&#xA;&#xA;        if args.input != &amp;quot;-&amp;quot;:&#xA;            sys.stdin = open(args.input, &amp;quot;r&amp;quot;)&#xA;&#xA;        return args&#xA;&#xA;    def fix_record(self, record: pysam.libcbcf.VariantRecord) -&amp;gt;     pysam.libcbcf.VariantRecord:&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        If the ALT column of the records contains a `-`, the variant position is shift one position to the left.&#xA;        For reaching this the reference base for the position is determined and prepended to the current REF bases.&#xA;        The ALT value `-` is replaced with the determined ref base on position before. In case there are multiple ALT&#xA;        values the determined ref base is prepended to them.&#xA;&#xA;        :param record: vcf file record&#xA;        :return: fixed vcf record&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;        if &amp;quot;*&amp;quot; in record.alts:&#xA;            ref = self.fasta.fetch(reference=record.chrom, start=record.pos - 1, end=record.pos)&#xA;&#xA;            record.pos = record.pos - 1&#xA;            record.ref = f&amp;quot;{ref}{record.ref}&amp;quot;&#xA;            record.alts = tuple(map(lambda alt: ref if alt == &amp;quot;*&amp;quot; else f&amp;quot;{ref}{alt}&amp;quot;, record.alts))&#xA;&#xA;        return record&#xA;&#xA;    def start(self):&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        Print the header of the vcf file. Afterwards the fix_record method is applied to each record in the vcf file.&#xA;&#xA;        For more information about python's map() see: &#xA;https://docs.python.org/3.6/library/functions.html#map&#xA;&#xA;        For more information about the unpacking mechanism using `*`see:&#xA;        https://docs.python.org/3/tutorial/controlflow.html#tut-unpacking-arguments&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;        print(self.vcf.header, end=&amp;quot;&amp;quot;)&#xA;        #print(*map(self.fix_record, self.vcf), sep=&amp;quot;&amp;quot;, end=&amp;quot;&amp;quot;)&#xA;        for line in self.vcf:&#xA;            print(self.fix_record(line), sep=&amp;quot;&amp;quot;, end=&amp;quot;&amp;quot;)&#xA;&#xA;if __name__ == &amp;quot;__main__&amp;quot;:&#xA;    app = Application()&#xA;    app.start()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7230" LastEditorUserId="3541" LastEditDate="2020-07-09T18:19:35.523" LastActivityDate="2020-07-09T18:19:35.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11483" PostTypeId="1" CreationDate="2020-02-23T01:26:52.720" Score="1" ViewCount="3557" Body="&lt;p&gt;I have multiple single cell samples to analyze and I'm following the instructions in Satija Lab's website. I want to merge all the count files from all the samples at once, and associate the metadata to each sample. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the &lt;code&gt;Read10x&lt;/code&gt; command, is there a way to read multiple files at once, such as merging all the filter_featire_bc_matrix files?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have all the samples saved in a directory on my desktop, is it possible to call the directory within the script for it to read all the raw counts?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what I have thus far:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# load the required libraries&#xA;library(dplyr)&#xA;library(Seurat) &#xA;library(ggplot2)&#xA;&#xA;-----------------------------------------------&#xA;&#xA;# set up the working directory&#xA;&#xA;wd = (&quot;~/Seurat&quot;) setwd(wd)&#xA;&#xA;# read all metadata&#xA;&#xA;metadata = read.table(&quot;metadata_Jan2020.csv&quot;, sep=';', header=T, stringsAsFactors=F)&#xA;rownames(metadata) = metadata$LibraryID&#xA;&#xA;#metadata&#xA;#mf = metadata&#xA;&#xA;mf = metadata %&amp;gt;% filter(Type %in% c(&quot;PDAC&quot;, &quot;NORMAL&quot;))&#xA;mf = mf %&amp;gt;% filter(!(ID %in% c(&quot;PDAC-HTB2854&quot;)))`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6827" LastEditorUserId="48" LastEditDate="2020-02-26T15:27:55.577" LastActivityDate="2020-02-26T15:27:55.577" Title="Reading multiple raw files in Seurat" Tags="|r|scrnaseq|seurat|10x-genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11484" PostTypeId="2" ParentId="11483" CreationDate="2020-02-23T06:18:15.340" Score="2" Body="&lt;p&gt;Seurat has &lt;a href=&quot;https://satijalab.org/seurat/v3.1/merge_vignette.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;a vignette specifically for combining multiple 10x libraries&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2020-02-23T06:18:15.340" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="11485" PostTypeId="1" AcceptedAnswerId="11487" CreationDate="2020-02-23T06:24:35.260" Score="4" ViewCount="276" Body="&lt;p&gt;I know of the common green glow gene but I forgot the name and I also know that some algae glow blue. There are so many types of bioluminesent organisms, so I am wondering what species have which genes and what other genes are associated with their metabolism and the systems that regulate these genes.&lt;/p&gt;&#xA;" OwnerUserId="7272" LastEditorUserId="3848" LastEditDate="2023-04-15T15:57:08.443" LastActivityDate="2023-04-15T15:57:08.443" Title="What are the different kinds of bioluminescent genes?" Tags="|phylogenetics|phylogeny|gene|blast|systems-biology|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11486" PostTypeId="2" ParentId="11485" CreationDate="2020-02-23T08:32:00.970" Score="2" Body="&lt;p&gt;The green protein is called GFP of 238 amino acid residues (26.9 kDa) and produced by:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the jellyfish Aequorea victoria, It has a major excitation peak at a wavelength of 395 nm. &lt;/li&gt;&#xA;&lt;li&gt;the sea pansy, Renilla reniformis of 498 nm. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;However, the protein is widespread and not restricted to these species.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-02-23T08:32:00.970" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11487" PostTypeId="2" ParentId="11485" CreationDate="2020-02-23T09:37:44.257" Score="6" Body="&lt;p&gt;There are three processes wherein light is emitted.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Bioluminescence&lt;/strong&gt;: a chemical reaction releases light. The enzyme that does this is luciferase, while its substrate is luciferin, a small molecule. There are half a dozen organisms that have analogous (evolved independently) enzymes and totally different substrates, such as fireflies, Alivibrio bacteria, diatomes, deep sea fish, some mushrooms etc. The &lt;a href=&quot;https://en.wikipedia.org/wiki/Luciferin&quot; rel=&quot;noreferrer&quot;&gt;wiki article&lt;/a&gt; has a list of these and their small molecules.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Fluorescence&lt;/strong&gt;: light is absorbed by the molecule and is emitted at a lower energy (wider wavelength, redshifted). Green fluorescent protein of Aequorea victoria jellyfish is the classic example of this. It is excited with UV and emits green light. Here is &lt;a href=&quot;https://michelanglo.sgc.ox.ac.uk/data/5bb431b4-e2b6-49ac-aef8-71370bec04e8&quot; rel=&quot;noreferrer&quot;&gt;an interactive model&lt;/a&gt;, I threw together. Variants exists for biotech reasons that span the whole of the visible spectrum. Other species have a homologous beta barrel, such as corals (red).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Phosphorescence&lt;/strong&gt;: light is absorbed by the molecule and emitted later.  Ignoring the picosecond delay in fluorescent small molecules (cf. fluorescent lifetime microscopy), this is exclusively a properties of lanthanide elements (e.g. Europium). Lathanide binding domains have been designed, but do no exist in nature.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-02-23T09:37:44.257" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11488" PostTypeId="1" AcceptedAnswerId="11490" CreationDate="2020-02-23T13:26:03.780" Score="1" ViewCount="64" Body="&lt;p&gt;I have a copy number segment file with this by SCAT R package&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Chromosome  Start   End&#xA;1           10583   120983&#xA;1           121009  245455&#xA;1           404187  5719467&#xA;1           5719471 5735807&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to count the number of markers in these locations&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I asked somebody how to do this, he replied:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;We created marker file by dividing the genome into 10 kb bins, so the markers file contains the start and stop positions for each of these 10kb segments for each chromosome along with ASCAT segments start and end positions. Then, for each ASCAT copy number segment, we count how many markers fall within that region for total probes.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can somebody helps to understand what he mean by dividing the genome &lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2020-02-23T16:27:04.087" Title="What does this statement mean?" Tags="|r|wgs|cnv|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11489" PostTypeId="1" CreationDate="2020-02-23T13:36:28.380" Score="0" ViewCount="1052" Body="&lt;p&gt;I have hundreds of bacterial genomes and I would like to know which is the best informatic tool in which I can run all genomes on one go and get a 16S rRNA gene sequence of each genome as output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anyone give me some advice?&lt;/p&gt;&#xA;" OwnerUserId="7405" LastActivityDate="2022-08-11T09:55:37.173" Title="Looking for a tool to find 16S rRNA in hundreds of genomes" Tags="|genome|bacteria|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11490" PostTypeId="2" ParentId="11488" CreationDate="2020-02-23T16:27:04.087" Score="4" Body="&lt;p&gt;They created 10kb bins, which could be called &quot;dividing the genome&quot;. These are just 10kb long contiguous stretches (so position 1-10000,then 10001-20000, etc.). This is, of course, a nonsensical way of going about things, because &quot;counting those up&quot; in the regions you posted is the same as taking the width of the regions, dividing by 10000, and rounding a bit.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Presumably you have your markers in something like BED format, so just use bedtools to intersect them and do the counting for you.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-02-23T16:27:04.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11491" PostTypeId="2" ParentId="11485" CreationDate="2020-02-23T16:32:28.027" Score="1" Body="&lt;p&gt;There appears to be a dinoflagellate with Luciferase and the diversity of their genes was studied in the Baltic, found described in an article here: &lt;a href=&quot;https://www.tandfonline.com/doi/full/10.1080/09670262.2016.1160441&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tandfonline.com/doi/full/10.1080/09670262.2016.1160441&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This article by Smithsonian: &lt;a href=&quot;https://ocean.si.edu/ocean-life/fish/bioluminescence&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ocean.si.edu/ocean-life/fish/bioluminescence&lt;/a&gt; describes a lot of Luciferin metabolizing species in the oceans, saying that different species independently evolved the forms they use. It describes some of the systems like endosymbiotic bacteria, special tissue, and photo-proteins. &lt;/p&gt;&#xA;" OwnerUserId="7272" LastActivityDate="2020-02-23T16:32:28.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11492" PostTypeId="1" AcceptedAnswerId="13145" CreationDate="2020-02-23T19:21:53.437" Score="2" ViewCount="64" Body="&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;: I have low-coverage sequencing data from humans in vcf format. For each sample, I have PL scores / preliminary genotype calls from the software ATLAS (it's a bit like GATK for those who don't know). Because the samples are low coverage (mean 1x, with lots of missing data), I am looking to enhance the genotype calls by using a reference panel and genetic map, before go on to phase the samples. Briefly, the reference panel and genetic map can be used as a kind of prior to inform one about genotypes when there is little data present.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I've done so far&lt;/strong&gt;: So far I have been using Beagle4 with a command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;beagle4 gl=lowCoverageSamples.vcf ref=highCoverageReference.vcf map=geneticMap out=lowCoverageSamples.newCalls.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, Beagle4 is extremely slow, very memory inefficient and quite old. There have been many advances in phasing methods, which are now much more accurate, faster and memory efficient than the beagle4 model. However, I haven't come across any similar improved methods for the genotype re-estimation step.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone knew of any newer pieces of software which do such a thing, which may more faster and more accurate than Beagle4. It needs to have 2 things in particular:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Accept data input in PL score format, as to allow the appropriate representation of uncertainty in low coverage samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Produce a genotype call / genotype probability for each SNP in each individual as output.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2020-04-29T20:27:28.003" Title="Can someone recommend software for reference-based genotype-estimation?" Tags="|vcf|genotyping|phasing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11493" PostTypeId="1" AcceptedAnswerId="11496" CreationDate="2020-02-24T11:02:26.147" Score="2" ViewCount="47" Body="&lt;p&gt;I am seeking virological databases focussing on:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;vaccines&lt;/li&gt;&#xA;&lt;li&gt;antibody interactions&lt;/li&gt;&#xA;&lt;li&gt;host receptors.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Preferably the data base will host multiple strains/samples of a given virus samples.&lt;/p&gt;&#xA;&lt;p&gt;Any assistance would be welcome.&lt;/p&gt;&#xA;" OwnerUserId="7219" LastEditorUserId="3848" LastEditDate="2022-03-01T16:06:40.360" LastActivityDate="2022-03-01T16:06:40.360" Title="Database with antigens and current vaccines" Tags="|phylogenetics|phylogeny|database|covid-19|sars-cov-2|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11494" PostTypeId="2" ParentId="11473" CreationDate="2020-02-24T11:45:11.663" Score="1" Body="&lt;p&gt;I think you want to prevent snakemake from interpreting your wildcard strings as regular expressions. For this, escape each wildcard value using &lt;code&gt;wildcard_constraints&lt;/code&gt;. E.g., towards the top of your Snakefile add:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;wildcard_constraints:&#xA;    read= '|'.join([re.escape(x) for x in reads]),&#xA;    sample= '|'.join([re.escape(x) for x in samples])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Personally, I use the &lt;code&gt;wildcard_constraints&lt;/code&gt; directive as above quite liberally to constrain every wildcard)&lt;/p&gt;&#xA;" OwnerUserId="339" LastActivityDate="2020-02-24T11:45:11.663" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11495" PostTypeId="2" ParentId="11489" CreationDate="2020-02-24T12:18:17.070" Score="2" Body="&lt;p&gt;You can use &lt;a href=&quot;https://github.com/tseemann/barrnap&quot; rel=&quot;nofollow noreferrer&quot;&gt;Barrnap&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;barrnap -k bac --threads &quot;$(nproc)&quot; \&#xA;        -o output_rrna.fna &amp;lt; input_file.fna \&#xA;        &amp;gt; output_rrna.gff 2&amp;gt; /dev/null&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Put the file in one line of sequence per fasta id and get sequence with 16S annotation.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl -pe 'if(/\&amp;gt;/){s/\n/\t/}; s/\n//; s/\&amp;gt;/\n\&amp;gt;/' output_rrna.fa | \&#xA;    perl -pe 's/\t/\n/' | tail -n+2 | cut -d\: -f1 | \&#xA;    grep -A1 16S &amp;gt; output_16S.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Check the &lt;code&gt;output_rrna.gff&lt;/code&gt; and make a BLAST of the sequence to be sure about the predicted 16S.&lt;/p&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2020-02-24T12:18:17.070" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11496" PostTypeId="2" ParentId="11493" CreationDate="2020-02-24T13:36:36.447" Score="1" Body="&lt;p&gt;Given &quot;any virus will do&quot;, the viruses that are &quot;head and shoulders&quot; above the rest in terms of volumes of data are HIV and influenza.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The influenza database is &lt;a href=&quot;https://www.fludb.org/brc/home.spg?decorator=influenza&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The influenza viral spike that attaches to the cell receptor is the HA protein – hemagglutinin. The cell receptor is sialic acid – a small sugar that is attached to many different proteins on the cell surface. You can of course simply search PDB for HA or else Genbank.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-02-24T14:43:36.787" LastActivityDate="2020-02-24T14:43:36.787" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11497" PostTypeId="1" AcceptedAnswerId="11515" CreationDate="2020-02-24T16:05:39.103" Score="1" ViewCount="1003" Body="&lt;p&gt;Is there some tool that will convert a vcf/bcf file to some gene annotation, based on the fact that I have the gene annotation of the already exsisting reference genome. Or some tool that will give me the genes based on the most probable chance.&lt;/p&gt;&#xA;" OwnerUserId="6573" LastActivityDate="2020-02-26T16:37:19.170" Title="vcf file to gene annotations and or genes" Tags="|vcf|gene|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="11498" PostTypeId="1" AcceptedAnswerId="11502" CreationDate="2020-02-24T17:14:46.217" Score="2" ViewCount="313" Body="&lt;p&gt;I'm looking for a way to download multiple proteomes at once from one clade, as fasta. Probably from the NCBI, because it looks more user-friendly than Ensembl, but Ensembl is okay too. In the best case scenario, I would like to constrain the download to a specific clade (eg. Actinobacteria, Chloroflexi etc.), maybe with a specific RefSeq Category (eg. reference/representative genome) or even depending on the genome assembly level. Command line would probably be the best option, but if there's a direct download option it would be nice to know it as well.&#xA;Does anything like this exist? Help would be much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2020-02-26T15:02:06.597" Title="How to download multiple proteomes at once?" Tags="|phylogenetics|ensembl|data-download|proteomics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11499" PostTypeId="2" ParentId="10717" CreationDate="2020-02-24T19:45:19.593" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;Go to &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NC_030853.1?report=fasta&amp;amp;log$=seqview&amp;amp;format=text...&quot; rel=&quot;nofollow noreferrer&quot;&gt;the main FASTA sequence page&lt;/a&gt;, view source, and get the &quot;ncbi_uidlist&quot; value.  In this case, it's &lt;code&gt;1061361601&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use the ID in the query string to download directly:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sviewer/viewer.fcgi?id=&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/sviewer/viewer.fcgi?id=&lt;/a&gt;&lt;strong&gt;1061361601&lt;/strong&gt;&amp;amp;db=nuccore&amp;amp;report=fasta&amp;amp;retmode=text&amp;amp;withmarkup=on&amp;amp;tool=portal&amp;amp;log$=seqview&amp;amp;maxdownloadsize=100000000&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;You can also get this link directly, by using Chrome's developer tools (F12), viewing the Network tab, then loading the page in (1).&lt;/p&gt;&#xA;" OwnerUserId="7279" LastActivityDate="2020-02-24T19:45:19.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11500" PostTypeId="2" ParentId="11480" CreationDate="2020-02-24T20:00:09.833" Score="0" Body="&lt;p&gt;There are two resources worth checking: &lt;a href=&quot;https://gtexportal.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GTEx portal&lt;/a&gt; and &lt;a href=&quot;https://www.ensembl.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl&lt;/a&gt;.&#xA;Uniprot and NCBI will call &quot;canonical&quot; the longest sequence comprising valid exons. These are not necessarily the biologically valid ones. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to Ensembl (&lt;a href=&quot;https://www.ensembl.org/Homo_sapiens/Gene/TranscriptComparison?db=core;g=ENSG00000141510;r=17:7661779-7687550&quot; rel=&quot;nofollow noreferrer&quot;&gt;TP53&lt;/a&gt;, there are a bunch that are experimentally validated and are all class 1. Note the Uniprot column being nearly homologeous: to match the transcripts one has to look at the length, in this case TP53-201 is 393aa long, which is class 1. As mentioned this is not always the case.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So which are more expressed in tissue X?&#xA;In &lt;a href=&quot;https://gtexportal.org/home/gene/ENSG00000141510&quot; rel=&quot;nofollow noreferrer&quot;&gt;GTEx&lt;/a&gt;, under the collapsed div labelled &quot;Exon Expression&quot; press the &quot;isoform Expression&quot; tab. The most expressed is ENST00000620739.4 (354aa long) join with ENST00000269305.8 (393aa).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the Exon Expression you will find out what exons are the issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now. to get the sequence of a transcript or covert a variant Ensembl drops the ball. There is no clear web tab that I know of. Nor an API route.&#xA;&lt;a href=&quot;ftp://ftp.ensembl.org/pub/release-99/fasta/homo_sapiens/pep/Homo_sapiens.GRCh38.pep.all.fa.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ensembl.org/pub/release-99/fasta/homo_sapiens/pep/Homo_sapiens.GRCh38.pep.all.fa.gz&lt;/a&gt; contains all the sequences however. For a project of mine (&lt;code&gt;ENSTMapper&lt;/code&gt; in &lt;a href=&quot;https://github.com/matteoferla/MichelaNGLo-app/blob/master/michelanglo_app/views/transcript.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt;) to map an amino acid mutation on one transcript to another I ended up write my own short converter, although there are a few similar tools out there.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-02-24T20:00:09.833" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11501" PostTypeId="1" AcceptedAnswerId="11503" CreationDate="2020-02-25T00:06:49.247" Score="3" ViewCount="1170" Body="&lt;p&gt;I have stranded, paired end RNASeq reads that I have aligned using STAR. I plan do conduct a differential expression analysis with DESeq2. After running quality control checks, a good portion of my reads are aligned but I'm a little concerned about the proportion of properly paired reads&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5YUeX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5YUeX.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the inner distance plot for the samples&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hLB1a.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hLB1a.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For human RNA sequencing, does anyone have a rule of thumb that they'd consider is an appropriate proportion of properly paired reads? Are there any good papers that talk about this issue? As MultiQC marks the column as yellow, I'm concerned the proper pair percentage may not be satisfactory. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How important are properly paired reads for differential expression analyses?&lt;/p&gt;&#xA;" OwnerUserId="6674" LastActivityDate="2020-02-26T16:34:36.737" Title="Importance of Proper Pairs vs Aligned Reads for RNASeq data" Tags="|rna-seq|ngs|transcriptome|quality-control|star|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11502" PostTypeId="2" ParentId="11498" CreationDate="2020-02-25T09:01:52.020" Score="1" Body="&lt;p&gt;You could try &lt;a href=&quot;https://github.com/pmenzel/download-refseq-genomes&quot; rel=&quot;nofollow noreferrer&quot;&gt;download-refseq-genomes&lt;/a&gt;. It will fetch all genomes from the NCBI FTP server that are in a specific subtree of the phylogeny. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, downloading the amino acid sequences from all proteins in each genome assembly for all species belonging to Chloroflexi (NCBI taxon ID = &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?mode=Tree&amp;amp;id=200795&amp;amp;lvl=3&amp;amp;keep=1&amp;amp;srchmode=1&amp;amp;unlock&quot; rel=&quot;nofollow noreferrer&quot;&gt;200795&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;download-refseq-genomes.pl -t faa 200795&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will download all assemblies with status=Complete. Use option &lt;code&gt;-a&lt;/code&gt; to download all assemblies with any status.&lt;/p&gt;&#xA;" OwnerUserId="1802" LastActivityDate="2020-02-25T09:01:52.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11503" PostTypeId="2" ParentId="11501" CreationDate="2020-02-25T09:16:37.897" Score="1" Body="&lt;p&gt;Differential gene analysis sees only the read counts. So the proportion of not properly paired reads will have an effect on the counts, if the programme / software used, counts fragments / read pairs instead of single reads. For example HTSeq will count&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;From &lt;a href=&quot;https://htseq.readthedocs.io/en/release_0.11.1/count.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://htseq.readthedocs.io/en/release_0.11.1/count.html&lt;/a&gt;&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;For paired-end data, does htseq-count count reads or read pairs? Read&#xA;  pairs. The script is designed to count “units of evidence” for gene&#xA;  expression. If both mates map to the same gene, this still only shows&#xA;  that one cDNA fragment originated from that gene. Hence, it should be&#xA;  counted only once.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In Rsubread, featureCounts you set &lt;code&gt;requireBothEndsMapped=TRUE&lt;/code&gt; to count fragments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Short anaswer is if the sequencing is deep enough, you should still have enough reads. In your situation, I estimated that you lose about 40% of your reads if you use only properly paired reads, which means you lose quite a bit of counts.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The larger question is what reads are you missing? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I see that you have about 20-30% of reads that are non-uniquely mapped, I suppose part of this will contribute towards the not properly paired reads. I would check first whether most of the not properly paired comes from multiple aligned. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then if not, you can check where these not properly paired reads are falling, if they are some repetitive regions or say rRNA etc, then most likely thats a problem with the library prep..&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-02-25T09:16:37.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11504" PostTypeId="2" ParentId="10689" CreationDate="2020-02-25T11:35:12.153" Score="0" Body="&lt;p&gt;For comparing a large number of trees side-by-side - Dendroscope.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For combining the trees into a network - Splitstree.&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2020-02-25T11:35:12.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11505" PostTypeId="1" CreationDate="2020-02-25T15:51:58.903" Score="2" ViewCount="53" Body="&lt;p&gt;I was wondering if anyone could help with my little issue in manipulating tsv files in order to calculate estimated coverage across many tsv files. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tsv files that look something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ID        Info    Info2  x&#xA;M80428    info    info   1000 &#xA;T83241    info    info   8400&#xA;E81292    info    info   10200&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also have another tsv file with all possible IDs and a corresponding number like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ID        y&#xA;M80428    27311&#xA;E11123    46531&#xA;P38163    14213&#xA;A34197    12892&#xA;T83241    37416&#xA;Q21345    27321&#xA;E81292    13101&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;x in the first tsv are estimated read counts and the y in the second is the genome length of each ID.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to divide the &quot;x&quot; by the &quot;y&quot; in the second tsv that has the corresponding ID in order to get the estimated coverage of each hit. e.g. the first row would be 1000/27311, second row 8400/37416. I would then want this to be written to the fifth column of the tsv. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am working in bash shell mostly here, and think that the way I would probably do this is with awk and by making the second tsv an associative array but I can't work out how exactly to do so. The python package dict seems like another option but my Python skills are practically zero. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for the help in advance!&lt;/p&gt;&#xA;" OwnerUserId="7287" LastEditorUserId="7287" LastEditDate="2020-02-25T18:56:05.433" LastActivityDate="2020-02-26T12:40:23.703" Title="Trying to divide number in fourth TSV column by a certain number according to its corresponding identifier in the first column" Tags="|python|bash|awk|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11506" PostTypeId="2" ParentId="11505" CreationDate="2020-02-25T17:20:03.553" Score="1" Body="&lt;p&gt;I would do it in R:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tab1 &amp;lt;- read.table('file1.tsv', header = T, stringsAsFactors = F)&#xA;tab2 &amp;lt;- read.table('file2.tsv', header = T, stringsAsFactors = F)&#xA;merged_tav &amp;lt;- merge(reads_tab, genome_len_tab)&#xA;merged_tav&lt;span class=&quot;math-container&quot;&gt;$z &amp;lt;- merged_tav$&lt;/span&gt;x / merged_tav$y&#xA;merged_tav&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;returns&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;      ID Info Info2     x     y          z&#xA;1 E81292 info  info 10200 13101 0.77856652&#xA;2 M80428 info  info  1000 27311 0.03661528&#xA;3 T83241 info  info  8400 37416 0.22450289&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;note that only IDs that are in both tables will be preserved.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2020-02-26T12:40:23.703" LastActivityDate="2020-02-26T12:40:23.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11507" PostTypeId="1" CreationDate="2020-02-26T04:20:18.240" Score="0" ViewCount="123" Body="&lt;p&gt;Hope all of you are fine. I am wondering if exists a program easy to use that can bring me Basic Statistics of Quality Control from raw data made by MinION device (nanopore).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have read about FASTAQC and Porechop. My sequencing raw data was a result of a metagenomics analysis (16s and 18s). My main objective is only to filter the sequences and obtain a basic statistic about the lenght and the number of sequences ordered by the lenght. Thank you ladies and gentlemen&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PD Sorry for my poor english. Day to day i'm trying to improve it.&lt;/p&gt;&#xA;" OwnerUserId="7291" LastActivityDate="2020-03-27T08:03:00.547" Title="Program Quality control for data sequencing (nanopore)" Tags="|nanopore|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11508" PostTypeId="2" ParentId="11507" CreationDate="2020-02-26T07:06:53.893" Score="1" Body="&lt;p&gt;A simple start would be to use &lt;a href=&quot;https://github.com/roblanf/minion_qc&quot; rel=&quot;nofollow noreferrer&quot;&gt;MinIONQC&lt;/a&gt;, which is also available via bioconda under the name &lt;code&gt;r-minionqc&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1802" LastActivityDate="2020-02-26T07:06:53.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11509" PostTypeId="2" ParentId="11507" CreationDate="2020-02-26T07:23:06.887" Score="1" Body="&lt;p&gt;I assume raw data is in Fastq format, but ideally you should also use the sequencing_summary.txt file generated by the basecaller. Other alternatives for QC are &lt;a href=&quot;https://github.com/a-slide/pycoQC&quot; rel=&quot;nofollow noreferrer&quot;&gt;PycoQC&lt;/a&gt; and &lt;a href=&quot;https://github.com/wdecoster/NanoPlot&quot; rel=&quot;nofollow noreferrer&quot;&gt;NanoPlot&lt;/a&gt;. I developed and maintain NanoPlot, so let me know if there is something I can help you with.&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2020-02-26T07:23:06.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11510" PostTypeId="2" ParentId="11505" CreationDate="2020-02-26T10:32:27.757" Score="3" Body="&lt;p&gt;Python dictionaries are indeed a good tool for that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a script that takes 2 arguments (the file containing the counts and the file containing the genome length info), and prints the input data, with an added column containing coverage:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;import sys&#xA;&#xA;counts_fname = sys.argv[1]&#xA;genome_lengths_fname = sys.argv[2]&#xA;&#xA;genome_length_dict = {}&#xA;with open(genome_lengths_fname) as genome_lengths_f:&#xA;    # Checking the header line, for extra safety&#xA;    header = genome_lengths_f.readline().strip().split(&quot;\t&quot;)&#xA;    assert header[0] == &quot;ID&quot;&#xA;    assert header[1] == &quot;y&quot;&#xA;    # Reading the data lines&#xA;    for line in genome_lengths_f:&#xA;        [genome_id, genome_length] = line.strip().split(&quot;\t&quot;)&#xA;        genome_length_dict[genome_id] = int(genome_length)&#xA;&#xA;with open(counts_fname) as counts_f:&#xA;    print(&quot;ID\tInfo\tInfo2\tx\tx/y&quot;)&#xA;    # Checking the header line, for extra safety&#xA;    header = counts_f.readline().strip().split(&quot;\t&quot;)&#xA;    assert header[0] == &quot;ID&quot;&#xA;    assert header[1] == &quot;Info&quot;&#xA;    assert header[2] == &quot;Info2&quot;&#xA;    assert header[3] == &quot;x&quot;&#xA;    # Reading the data lines&#xA;    for line in counts_f:&#xA;        [genome_id, info1, info2, counts] = line.strip().split(&quot;\t&quot;)&#xA;        # /!\ It would be different in python 2:&#xA;        # different print and division behaviours&#xA;        print(&#xA;            genome_id, info1, info2, int(counts),&#xA;            int(counts) / genome_length_dict[genome_id],&#xA;            sep=&quot;\t&quot;)&#xA;&#xA;sys.exit(0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Testing it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ./coverage.py x.tsv y.tsv&#xA;ID  Info    Info2   x   x/y&#xA;M80428  info    info    1000    0.0366152832192157&#xA;T83241  info    info    8400    0.22450288646568314&#xA;E81292  info    info    10200   0.7785665216395695&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2020-02-26T10:37:34.387" LastActivityDate="2020-02-26T10:37:34.387" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11511" PostTypeId="1" AcceptedAnswerId="11523" CreationDate="2020-02-26T13:47:34.340" Score="0" ViewCount="121" Body="&lt;p&gt;I'm downloading translated viral genomes initially via Blast - which had shortcomings - and now by &lt;code&gt;efetch&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I want to do is obtain a complete list of all translated protein ids from a nucleotide Genbank entry.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;efetch -db nucleotide -format gpc -id MT039887 | xtract -insd ....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The protein id and the translated gene sequences are within the &lt;code&gt;efetch -db nucleotide -format gpc -id MT039887&lt;/code&gt; output ... but how do I use &lt;code&gt;xtract&lt;/code&gt; to parse them?&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-04-29T11:42:10.073" LastActivityDate="2020-04-29T13:37:06.470" Title="efetch nucleotide -&gt; protein ids" Tags="|genome|phylogeny|phylogenetics|efetch|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11512" PostTypeId="1" CreationDate="2020-02-26T13:59:05.083" Score="4" ViewCount="78" Body="&lt;p&gt;I have two sets of PacBio reads from genomic DNA of an Aspergillus species that were made from separate preps of  the culture. One of them has two additional peaks at 38% and 60% in the percent GC histogram produced by FastQC. Do these additional peaks indicate that there is contamination in that sample?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IdvnD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IdvnD.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1454" LastActivityDate="2023-12-27T22:39:05.647" Title="Do additional peaks in percent GC of PacBio gDNA reads indicate contamination?" Tags="|genome|pacbio|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11513" PostTypeId="2" ParentId="11498" CreationDate="2020-02-26T14:30:56.180" Score="0" Body="&lt;p&gt;You can use NCBI resources to download these data directly. For the entire proteome, you can download the sequences of all proteins annotated on the genome assembly in FASTA format. In the NCBI FTP directories, these files have the names with the suffix &lt;code&gt;_protein.faa.gz&lt;/code&gt;. More details about the files in the FTP directories are described in the &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/genomes/all/README.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;README&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Command-line method using EDirect&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;This method uses &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez Direct&lt;/a&gt; to first query the Assembly database for all available genomes, parses the output to extract the ftp path and downloads data using wget command. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db assembly -query 'txid200795[organism] AND lastest_refseq[filter]' \&#xA;  | esummary \&#xA;  | xtract -pattern DocumentSummary -element FtpPath_RefSeq \&#xA;  | while read -r url ; do &#xA;    path=$(echo $url | perl -pe 's/(GC[FA]_\d+.*)/\1\/\1_protein.faa.gz/g') ; &#xA;    wget -q --show-progress &quot;$path&quot; -P protein_data ; &#xA;  done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here, I am restricting the results to latest RefSeq data by using the &lt;code&gt;latest_refseq[filter]&lt;/code&gt; but you can use any other Entrez filters to further narrow down your results. Along the same lines, you can modify the code minimally as shown below to download multiple file types: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db assembly -query 'txid200795[organism] AND latest_refseq[filter]' \&#xA;  | esummary \&#xA;  | xtract -pattern DocumentSummary -element FtpPath_RefSeq \&#xA;  | while read -r url ; do &#xA;    base_path=$(echo $url | perl -pe 's/(GCF_\d+.*)/\1\/\1_/g') ; &#xA;    for suf in genomic.fna.gz protein.faa.gz ; do&#xA;      wget -q --show-progress &quot;$base_path$suf&quot; -P ncbi_data ; &#xA;    done ;&#xA;  done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;GUI method&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;You can use the same query as shown above in the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Assembly&lt;/a&gt; portal and then use the 'Download Assemblies' button to pick and choose the type of files you would like to download. By default, if you do not select any assemblies in the results, this interface downloads files for all results.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ZbQY9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZbQY9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3577" LastEditorUserId="3577" LastEditDate="2020-02-26T15:02:06.597" LastActivityDate="2020-02-26T15:02:06.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11514" PostTypeId="2" ParentId="11501" CreationDate="2020-02-26T16:34:36.737" Score="2" Body="&lt;p&gt;If you check the &lt;a href=&quot;http://rseqc.sourceforge.net/#inner-distance-py&quot; rel=&quot;nofollow noreferrer&quot;&gt;RSeQC docs&lt;/a&gt; you can get an explanation of how the inner distance stats work. By the look of your inner-distance MultiQC plot I guess that your sequencing is 2x100bp reads. This plot shows that your insert sizes on the pairs is pretty short, meaning that a lot of the paired reads are overlapping significantly. This will be affecting the alignments a little, resulting in the low proportion of properly paired reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The good news is that for differential expression, this probably doesn't really matter too much. What is important is getting fragment counts, so as long as at least one of the pair mates is mapping properly, then you will will be getting sensible counts - even if read 2 is not mapping properly. You should be able to have confidence in your results from multiple replicates if in doubt - you have more than just &lt;code&gt;-rep1&lt;/code&gt; right? ;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that the yellow background in the MultiQC column is not passing judgement with categories like pass/warn/fail in FastQC - it just happens that the colour scheme ended up there for your samples. It's a continuous gradient from red to yellow to green for those values (0-100%).&lt;/p&gt;&#xA;" OwnerUserId="224" LastActivityDate="2020-02-26T16:34:36.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11515" PostTypeId="2" ParentId="11497" CreationDate="2020-02-26T16:37:19.170" Score="2" Body="&lt;p&gt;Creating a FASTA file by applying variants found in a VCF file is called &quot;creating a consensus FASTA file&quot;. You can use &lt;a href=&quot;https://samtools.github.io/bcftools/bcftools.html#consensus&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;bcftools consensus&lt;/code&gt;&lt;/a&gt;. Unfortunately, the tools only works genome-wide (at the FASTA level), so you might need to use a different tool to extract a gene sequence from the FASTA file using the gene's co-ordinates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the VCF file is annotated with HGVS notations for the variants, there might be a way to use those notations to apply the edit to the reference gene/transcript sequence directly. I know &lt;a href=&quot;https://hgvs.readthedocs.io/en/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the hgvs python module&lt;/a&gt; works well with the HGVS notation, but I don't see a way to &quot;apply&quot; the variant to the reference sequence.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2020-02-26T16:37:19.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11516" PostTypeId="1" CreationDate="2020-02-26T17:15:15.317" Score="2" ViewCount="363" Body="&lt;p&gt;I have a &lt;code&gt;data.frame&lt;/code&gt; with results from a CRISPR screen.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to calculate the % enrichment of a variable of interest &lt;code&gt;var&lt;/code&gt; from baseline to treatment group.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the distribution of &lt;code&gt;var&lt;/code&gt; in linear:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/cDOXS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cDOXS.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can do scale &lt;code&gt;var&lt;/code&gt; with log transformation&#xA;&lt;a href=&quot;https://i.stack.imgur.com/tzXXW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tzXXW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But many rows with &lt;code&gt;var&lt;/code&gt; = 0 will be lost.&#xA;Moreover, I cannot compare % enrichment by dividing &lt;code&gt;treatmentgroup::var&lt;/code&gt; to &lt;code&gt;Control/baseline::var&lt;/code&gt; because I will run into the problem of dividing by zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've seen people circumvent this problem by adding a small value ex)0.01 to the entire column of &lt;code&gt;var&lt;/code&gt;.&#xA;But I'm wondering if there's a better way of circumventing this problem. For example, a simple transformation that can deal with zeros and negative values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Appreciate any input!&lt;/p&gt;&#xA;" OwnerUserId="7056" LastActivityDate="2020-04-18T03:42:07.023" Title="How to deal with zeros in the denominator when calculating percent enrichment?" Tags="|r|statistics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11517" PostTypeId="1" CreationDate="2020-02-26T19:13:17.757" Score="0" ViewCount="90" Body="&lt;p&gt;I have had a segmentation file (copy number) like below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  Sample      Chromosome  Start      End          Segment_Mean&#xA;LP6008334-DNA_C02   1     10583      62271      -0.992452416&#xA;LP6008334-DNA_C02   1   17201069    17230953    -0.992452416&#xA;LP6008334-DNA_C02   1   17235034    17235196     0.592510084&#xA;LP6008334-DNA_C02   1   142781738   142967188   -0.992452416&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I was using bedtools to count the number of markers in above genomic range (Start, End) when I got as below&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1   10583   62271   6&#xA;1   10583   62271   6&#xA;1   10583   750153  75&#xA;1   14907   1062025 106&#xA;1   121009  245455  13&#xA;1   404187  5719467 532&#xA;1   732809  5726842 500&#xA;1   821926  1120377 31&#xA;1   952428  1179983 23&#xA;1   1181751 1469134 29&#xA;1   3458681 143542402   14010&#xA;1   5726844 5736229 2&#xA;1   5735836 16832446    1111&#xA;1   5736682 8081290 236&#xA;1   8353020 8476455 13&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically the same files but the second one has extra last column for the number of marker but sorted one I then I can not matched this with the above sample names and Segment_Mean&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How I can add the corresponding sample names and Segment_Mean to my second file?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried this where I am getting this mess&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DF &amp;lt;- left_join(df_res1,df, by = &quot;Start&quot;)&#xA;&#xA;&amp;gt; head(DF)&#xA;           Sample.x Chromosome.x     Start     End.x Num_Probes          Sample.y Chromosome.y     End.y&#xA;1 LP6008334-DNA_C02            1     10583     62271          9 LP6008334-DNA_C02            1     62271&#xA;2 LP6008334-DNA_C02            1     10583     62271          9 LP6008337-DNA_A07            1     62271&#xA;3 LP6008334-DNA_C02            1     10583     62271          9 LP6008460-DNA_F02            1    750153&#xA;4 LP6008334-DNA_C02            1  17201069  17230953         20 LP6008334-DNA_C02            1  17230953&#xA;5 LP6008334-DNA_C02            1  17235034  17235196          1 LP6008334-DNA_C02            1  17235196&#xA;6 LP6008334-DNA_C02            1 142781738 142967188         40 LP6008334-DNA_C02            1 142967188&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also with this I have some overlaps like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;300 segment overlaps detected in file '/temp/hgig/fi1d18/man/examplefiles/res1.txt'.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;First overlap detected between segments at lines 1 and 9.&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-02-27T09:43:12.013" LastActivityDate="2020-02-27T09:43:12.013" Title="Joining columns to a sorted file" Tags="|r|bedtools|wgs|cnv|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11519" PostTypeId="2" ParentId="11517" CreationDate="2020-02-27T00:09:09.077" Score="2" Body="&lt;p&gt;Rearrange your text file into a sorted BED file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ tail -n+2 segmentation.txt | awk -vFS=&quot;\t&quot; -vOFS=&quot;\t&quot; '{ print $2, $3, $4, $&lt;/span&gt;1 }' | sort-bed - &amp;gt; segmentation.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Use BEDOPS &lt;code&gt;bedmap&lt;/code&gt; with &lt;code&gt;--echo-map-id-uniq&lt;/code&gt; and &lt;code&gt;--count&lt;/code&gt; operators on merged regions:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --delim '\t' --echo --count --echo-map-id-uniq --fraction-map 1 &amp;lt;(bedops --merge segmentation.bed) segmentation.bed &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The file &lt;code&gt;answer.bed&lt;/code&gt; will contain merged intervals from &lt;code&gt;segmentation.bed&lt;/code&gt; in the first three columns, the number of intervals from &lt;code&gt;segmentation.bed&lt;/code&gt; which are contained entirely within the merged interval in the fourth column, and a unique listing of IDs of those contained intervals in the fifth column.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2020-02-27T00:09:09.077" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11520" PostTypeId="1" CreationDate="2020-02-27T02:54:34.880" Score="2" ViewCount="86" Body="&lt;p&gt;This is my first post here. I can think of no way of giving an easily reproducible example, as per the stack ethos. SO apologies in advance and any feedback on question format appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have called samtools mpileup (Version: 1.9 (using htslib 1.9)) on some oxford nanopore data (mapped to GRCh38).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am seeing some instances like below (grouped with pythons Counter), here reference is T at this position:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; Counter({',': 34, '.': 33, 'c': 9, 'g': 6, 'a': 4, 't': 2, '-:ACTC': 1, '-:ACTCGG': 1, '-:5': 1, '$': 1, '*': 1})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Why is 't': 2' in there?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;flags used: -A -B -Q 1 -R -x&lt;/p&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2020-07-26T18:05:45.627" Title="Why does samtools mpileup sometimes include ref bases (other than ',' or '.')?" Tags="|python|samtools|long-reads|mpileup|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11522" PostTypeId="1" AcceptedAnswerId="11524" CreationDate="2020-02-27T15:31:33.137" Score="1" ViewCount="146" Body="&lt;p&gt;I know for example &lt;code&gt;PolyX&lt;/code&gt; from Pierre's &lt;a href=&quot;http://lindenb.github.io/jvarkit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;Jvarkit&lt;/code&gt;&lt;/a&gt; to find the number of repeated REF bases (homopolymers) around a position in a VCF file as described &lt;a href=&quot;https://www.biostars.org/p/140069/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; exists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to know however if there tools that describes tandem repeats around a reported variant (TR) up to 6 repeated bases. I can imagine 3 different scenarios:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Scenario 1:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chrK    posY    C     G&#xA;&#xA;TR of (ATC)&#xA;____________v&#xA;ATCATCATCATCGATGATAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Scenario 2:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chrK    posY    A     T&#xA;&#xA;             TR of (AACT)&#xA;            v____________&#xA;CAGTCGATTGGATAACTAACTAACT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Scenario 3:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chrK    posY    C     A&#xA;&#xA;TR of (AG)    TR of (AG)&#xA;____________v__________&#xA;AGAGAGAGAGAGCAGAGAGAGAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you know any tools in mind that can be used for VCFs, I'd really appreciate citing them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="4413" LastEditDate="2020-02-27T15:43:13.550" LastActivityDate="2020-02-27T20:35:01.080" Title="Are there any tools for discovering tandem repeats around a variant in a VCF file?" Tags="|vcf|repeat-elements|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11523" PostTypeId="2" ParentId="11511" CreationDate="2020-02-27T15:51:05.097" Score="1" Body="&lt;p&gt;If you want the cds in fasta format, try:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db nucleotide -query MT039887 | efetch -format fasta_cds_na&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And for the proteins sequence in fasta format:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;elink -db nucleotide -target protein -id MT039887 | efetch -format fasta &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2020-02-27T15:51:05.097" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11524" PostTypeId="2" ParentId="11522" CreationDate="2020-02-27T15:58:14.937" Score="1" Body="&lt;p&gt;After researching a bit, I found a couple of tools that take as input BAM files. I did not test them yet:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/gymreklab/GangSTR&quot; rel=&quot;nofollow noreferrer&quot;&gt;GangSTR&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/mgymrek/lobstr-code&quot; rel=&quot;nofollow noreferrer&quot;&gt;lobSTR&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/tfwillems/HipSTR&quot; rel=&quot;nofollow noreferrer&quot;&gt;hipSTR&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://tandem.bu.edu/trf/trf.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tandem Repeat Finder&lt;/a&gt; (thanks to Michael G.'s answer)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If I find more tools, I will update this post. After testing them, I will come back to share my feedback.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="4413" LastEditDate="2020-02-27T20:35:01.080" LastActivityDate="2020-02-27T20:35:01.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11525" PostTypeId="2" ParentId="11520" CreationDate="2020-02-27T16:47:40.930" Score="-1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Why is 't': 2' in there?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;How else do you think the pileup should show you that two reads (in the reverse direction) have a T in that position?&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-02-27T16:47:40.930" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11526" PostTypeId="2" ParentId="11522" CreationDate="2020-02-27T20:05:25.973" Score="1" Body="&lt;p&gt;The old program for doing this last updated in 2016 is Tandem Repeat Finder (TRF) &lt;a href=&quot;https://tandem.bu.edu/trf/trf.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. This will not find a microsatellite sequence at a given variant, but instead find all tandem repeats in the genome and their position. It has been around for a long time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You would then simply screen this output against the position of your variant. TRF may have been modified however to perform a search within given nucleotide boundaries.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-02-27T20:05:25.973" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11528" PostTypeId="1" CreationDate="2020-02-28T17:58:40.910" Score="1" ViewCount="146" Body="&lt;p&gt;I have the data below. I need to use a clustering method to classify them and into categories of &quot;Heterozygotote, Allele 1, Allele 2 and No Call. The values in RFU1 and RFU2 are used to determine the call variable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any values in RFU1 and RFU2 that are negative should automatically be classified as No Call. I want to develop my own validation check to check the PCR program calls.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Data&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(Well = structure(1:96, .Label = c(&quot;A01&quot;, &quot;A02&quot;, &#xA;&quot;A03&quot;, &quot;A04&quot;, &quot;A05&quot;, &quot;A06&quot;, &quot;A07&quot;, &quot;A08&quot;, &quot;A09&quot;, &quot;A10&quot;, &quot;A11&quot;, &#xA;&quot;A12&quot;, &quot;B01&quot;, &quot;B02&quot;, &quot;B03&quot;, &quot;B04&quot;, &quot;B05&quot;, &quot;B06&quot;, &quot;B07&quot;, &quot;B08&quot;, &#xA;&quot;B09&quot;, &quot;B10&quot;, &quot;B11&quot;, &quot;B12&quot;, &quot;C01&quot;, &quot;C02&quot;, &quot;C03&quot;, &quot;C04&quot;, &quot;C05&quot;, &#xA;&quot;C06&quot;, &quot;C07&quot;, &quot;C08&quot;, &quot;C09&quot;, &quot;C10&quot;, &quot;C11&quot;, &quot;C12&quot;, &quot;D01&quot;, &quot;D02&quot;, &#xA;&quot;D03&quot;, &quot;D04&quot;, &quot;D05&quot;, &quot;D06&quot;, &quot;D07&quot;, &quot;D08&quot;, &quot;D09&quot;, &quot;D10&quot;, &quot;D11&quot;, &#xA;&quot;D12&quot;, &quot;E01&quot;, &quot;E02&quot;, &quot;E03&quot;, &quot;E04&quot;, &quot;E05&quot;, &quot;E06&quot;, &quot;E07&quot;, &quot;E08&quot;, &#xA;&quot;E09&quot;, &quot;E10&quot;, &quot;E11&quot;, &quot;E12&quot;, &quot;F01&quot;, &quot;F02&quot;, &quot;F03&quot;, &quot;F04&quot;, &quot;F05&quot;, &#xA;&quot;F06&quot;, &quot;F07&quot;, &quot;F08&quot;, &quot;F09&quot;, &quot;F10&quot;, &quot;F11&quot;, &quot;F12&quot;, &quot;G01&quot;, &quot;G02&quot;, &#xA;&quot;G03&quot;, &quot;G04&quot;, &quot;G05&quot;, &quot;G06&quot;, &quot;G07&quot;, &quot;G08&quot;, &quot;G09&quot;, &quot;G10&quot;, &quot;G11&quot;, &#xA;&quot;G12&quot;, &quot;H01&quot;, &quot;H02&quot;, &quot;H03&quot;, &quot;H04&quot;, &quot;H05&quot;, &quot;H06&quot;, &quot;H07&quot;, &quot;H08&quot;, &#xA;&quot;H09&quot;, &quot;H10&quot;, &quot;H11&quot;, &quot;H12&quot;), class = &quot;factor&quot;), Sample = c(NA, &#xA;NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, &#xA;NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, &#xA;NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, &#xA;NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, &#xA;NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, &#xA;NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA, NA), &#xA;    Call = structure(c(4L, 4L, 1L, 1L, 1L, 1L, 3L, 3L, 1L, 1L, &#xA;    3L, 3L, 4L, 4L, 1L, 1L, 1L, 1L, 1L, 1L, 3L, 3L, 1L, 1L, 4L, &#xA;    4L, 1L, 1L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 4L, 4L, 1L, 1L, &#xA;    3L, 3L, 3L, 3L, 1L, 1L, 3L, 3L, 3L, 3L, 1L, 1L, 1L, 1L, 1L, &#xA;    1L, 2L, 2L, 1L, 1L, 3L, 3L, 1L, 1L, 4L, 1L, 1L, 1L, 1L, 1L, &#xA;    1L, 1L, 1L, 1L, 2L, 2L, 3L, 3L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, &#xA;    2L, 3L, 3L, 1L, 1L, 1L, 1L, 1L, 1L, 3L, 3L), .Label = c(&quot;Allele 1&quot;, &#xA;    &quot;Allele 2&quot;, &quot;Heterozygote&quot;, &quot;No Call&quot;), class = &quot;factor&quot;), &#xA;    Type = structure(c(1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, &#xA;    1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, &#xA;    1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, &#xA;    1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, &#xA;    1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, &#xA;    1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, &#xA;    1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L), .Label = &quot;Auto&quot;, class = &quot;factor&quot;), &#xA;    RFU1 = c(-0.295502405, 0.964070798, 3381.332182, 3532.769062, &#xA;    3431.836843, 3242.966511, 2104.791167, 2220.008503, 3548.252161, &#xA;    3506.51418, 2290.273178, 2281.587684, -5.64819475, -11.73109864, &#xA;    3784.914039, 3619.00781, 3618.211608, 3248.106466, 3394.650325, &#xA;    3339.870196, 2449.202902, 2426.835174, 3432.153478, 2761.392304, &#xA;    -9.267907504, -7.365704356, 3743.092314, 3787.241702, 2172.027787, &#xA;    2096.845649, 2135.649551, 2149.145547, 2293.757257, 2348.099108, &#xA;    2321.019045, 2022.168867, -17.93532331, -12.59832941, 3805.416768, &#xA;    3498.998138, 2304.597239, 2509.63987, 2181.11547, 2261.011876, &#xA;    3432.453036, 3662.758933, 2371.11049, 3068.827061, 2612.107589, &#xA;    2687.824075, 3179.315918, 3688.525218, 3465.327523, 3405.154043, &#xA;    2535.514915, 2452.200079, 374.435299, 423.6015308, 3742.515563, &#xA;    3578.777925, 2634.955017, 2527.514043, 3817.579252, 3550.999412, &#xA;    -10.72035816, 3294.486334, 3352.40368, 3463.150507, 3472.576514, &#xA;    3741.898759, 3571.369947, 3720.645869, 3739.569593, 3855.583168, &#xA;    418.6837047, 49.47548241, 2171.034284, 2155.314713, 3432.363384, &#xA;    3582.508917, 3425.415274, 3487.203299, 3505.23909, 3413.342096, &#xA;    113.5100691, 128.6414053, 2454.588175, 2323.061591, 3188.705702, &#xA;    3376.950546, 3291.072437, 3181.001961, 3195.013863, 3776.919197, &#xA;    2284.22659, 2277.338631), RFU2 = c(-8.346468029, 235.4058561, &#xA;    637.9218251, 650.3759507, 617.4161748, 604.0792911, 4270.310727, &#xA;    4199.615749, 689.863543, 712.6144338, 4274.287194, 4541.168491, &#xA;    -1.626221758, -2.437395631, 802.0941252, 730.5998997, 686.9037384, &#xA;    625.8245403, 644.3586836, 642.8833044, 4937.691887, 5159.479928, &#xA;    725.4449756, 573.3910899, -4.006398006, 213.2859144, 739.7910786, &#xA;    731.0150586, 4408.81923, 4767.533969, 4302.641493, 4325.913445, &#xA;    4597.47663, 4666.904418, 4800.357526, 4142.535329, -17.23239968, &#xA;    178.5311942, 778.305843, 743.1438168, 4214.507094, 4553.703511, &#xA;    4629.339014, 4459.697405, 661.7299014, 727.1054982, 4553.170272, &#xA;    5482.231486, 4520.517999, 4737.802036, 643.3599887, 726.4314715, &#xA;    696.5968338, 697.6099599, 411.8118071, 409.4943424, 5687.32635, &#xA;    5757.51512, 766.4240193, 779.2403225, 4745.055632, 4582.267792, &#xA;    749.5679421, 675.8747055, -7.254521898, 628.3467565, 631.116767, &#xA;    672.7064514, 687.2642132, 718.1192917, 731.785499, 668.3686048, &#xA;    784.8055727, 791.3155894, 4471.047168, 4501.597841, 4504.670332, &#xA;    4442.621066, 682.0632225, 706.6204595, 680.5242182, 683.9558692, &#xA;    684.2909706, 618.6535251, 5727.684954, 6098.485474, 5099.952926, &#xA;    4779.742057, 571.4303822, 614.9258218, 602.9830491, 651.2847695, &#xA;    591.8833499, 742.2387568, 4443.376841, 4716.792177)), class = &quot;data.frame&quot;, row.names = c(NA, &#xA;-96L))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;What I have tried so far&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(cluster)&#xA;library(factoextra)&#xA;library(formattable)&#xA;&#xA;df &amp;lt;- df[,c(1,5,6)]&#xA;df&lt;span class=&quot;math-container&quot;&gt;$RFU1[df$&lt;/span&gt;RFU1 &amp;lt; 0] &amp;lt;- 0&#xA;df&lt;span class=&quot;math-container&quot;&gt;$RFU2[df$&lt;/span&gt;RFU2 &amp;lt; 0] &amp;lt;- 0&#xA;df&lt;span class=&quot;math-container&quot;&gt;$RFU1 &amp;lt;- formattable(df$&lt;/span&gt;RFU1, digits = 2, format = &quot;f&quot;)&#xA;df&lt;span class=&quot;math-container&quot;&gt;$RFU2 &amp;lt;- formattable(df$&lt;/span&gt;RFU2, digits = 2, format = &quot;f&quot;)&#xA;df&lt;span class=&quot;math-container&quot;&gt;$Well &amp;lt;- as.numeric(df$&lt;/span&gt;Well)&#xA;&#xA;&#xA;clusters &amp;lt;- kmeans(df, centers = 4)&#xA;Kmeans_plot &amp;lt;- fviz_cluster(clusters, data = df)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;This is the plot generated&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Rr86P.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Rr86P.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Points in the top right 57,58,75,76,85,86 should be in another cluster (classed as allele 2 but placed in cluster 1 (rest of points in this cluster are heterozygote)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also points 24, 55, 56 should be clustered with cluster 2 as all these points are classed as allele 1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;I need an algorithm that can detect optimal cluster center, some PCR runs may only have 3 different call variables instead of 4&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Programs output - Software company stated their algorithm isn't sophisticated&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/D69v9.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D69v9.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7305" LastEditorUserId="7305" LastEditDate="2020-02-29T00:16:14.107" LastActivityDate="2020-02-29T00:16:14.107" Title="K means clustering, would PCA be a better option?" Tags="|r|clustering|k-mer|pca|pcr|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11529" PostTypeId="2" ParentId="11528" CreationDate="2020-02-28T18:34:25.293" Score="0" Body="&lt;p&gt;What happens when &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;clusters &amp;lt;- kmeans(df, centers = 3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Anyway K means is pretty good and hasn't done a bad job of your data, although I don't know what you mean by the &quot;cluster centre&quot;. There are methods to establish 'centroids'.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With K-means you need to vary the group size or have a very good a priori why it is 4, rather than any other number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The analysis to perform is tSNE ... either by itself or else following PCA. This will provide a clear a priori of the number of groups in your analysis. I don't know R, but it is present in scikit-learn in Python.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Your observation that some of points are in the wrong cluster ... increase the number of groups in k-means, or switch to PCA, tSNE or PCA-tSNE. If the points remain in the &quot;wrong groups&quot; in all analyses at different group sizes for k-means, then basically thats the result. K-means is good, but not perfect, and misclassification will happen. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another route is the ML repertoire of lasso regression, ridge regression, decision trees, random forests etc ... they all do the same basic job but I'd look at K-means, PCA, tSNE first and if ok ... last. Regression style calculations will give different answers. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;t-SNE is a dimensionality reduction approach to produce a bivariate plot and the fondness of this approach is purely observational rather than empirical. It works and also works very well following PCA. Its the method used in Seurat. K-means is used in ML, and certainly has a fan base, but it is not to say it will give you a correct answer.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-02-28T20:23:44.057" LastActivityDate="2020-02-28T20:23:44.057" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11530" PostTypeId="2" ParentId="11528" CreationDate="2020-02-28T18:54:22.057" Score="1" Body="&lt;p&gt;I would stay away from using k-means, and instead use a method that doesn't &lt;em&gt;a priori&lt;/em&gt; define a number of clusters to detect. It also looks as though your clusters aren't exactly spherical, which is an assumption of k-means. I personally am a fan of &lt;code&gt;dbscan&lt;/code&gt;, which is available in the R package of the same name. The other poster recommended t-SNE (available in the &lt;code&gt;Rtsne&lt;/code&gt; package) but wasn't exactly clear on how / why to use it. t-SNE is a nonlinear dimension reduction technique used specifically for visualization, so you could cluster your points in &lt;em&gt;n&lt;/em&gt;-dimensional PCA space and then plot the results in two-dimensional t-SNE space. I think the other poster was recommending plotting your data in t-SNE space before clustering to get an idea of how many true clusters are available, which is another option you could pursue.  &lt;/p&gt;&#xA;" OwnerUserId="6232" LastActivityDate="2020-02-28T18:54:22.057" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11532" PostTypeId="2" ParentId="11528" CreationDate="2020-02-28T21:46:06.040" Score="0" Body="&lt;p&gt;I don't think clustering algorithms are working here, it looks like you can do this much simpler by looking at whatever determines the X axis in your plot.  The samples split cleanly when looking in just one dimension.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-02-28T21:46:06.040" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11533" PostTypeId="1" CreationDate="2020-02-28T22:56:38.310" Score="0" ViewCount="175" Body="&lt;p&gt;I have 2 dataframe with same genes both different data sets&lt;/p&gt;&#xA;&#xA;&lt;p&gt;like &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(heat1[1:3,1:4])&#xA;             A10         A11        A12        A2&#xA;ATP5F1 2.1163092  0.32044094 -0.5661269 0.7978236&#xA;DDX5   0.6277182 -1.67055853 -0.0123082 0.8895472&#xA;EEF1G  1.6459212  0.02923401 -1.2737676 0.5842162&#xA;&amp;gt; &#xA;&#xA;&amp;gt; head(heat2[1:3,1:4])&#xA;              A10        A12          A3          A4&#xA;A2M    -0.6031969  0.6378864 0.000573381 -0.07769313&#xA;ABCB11  0.5600164  0.3087120 0.171636836  1.25681512&#xA;ABCC2   0.2093133 -0.5608016 0.782278709  0.16618683&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to plot them side by side with the same order of genes &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; ht1 = Heatmap(heat1)&#xA;&amp;gt; ht2 = Heatmap(heat2)&#xA;&amp;gt; ht_list = ht1 + ht2&#xA;&amp;gt; draw(ht_list, ht_gap = unit(1, &quot;cm&quot;))&#xA;Warning message:&#xA;Row names of heatmap 2 is not consistent as the main&#xA;heatmap (1) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I getting not the same order of genes&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4Lojp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4Lojp.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you help me with this?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-02-29T00:14:26.820" LastActivityDate="2020-02-29T02:04:28.177" Title="Drawing 2 heatmaps with the same gene order" Tags="|r|complexheatmap|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11534" PostTypeId="2" ParentId="11475" CreationDate="2020-02-28T23:15:55.373" Score="1" Body="&lt;p&gt;&quot;Mutually exclusive&quot; is not a precise statistical term because it could be seen as independence. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway if truly mutually exclusive AND their behaviour follows periodicity, i.e. sine waves, which it might do in a cell cycle, i.e. the cells divide at periodic intervals, you can do this via asynchronicity. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the gene expression can be described under periodicity analysis you simply sum the amplitude between two genes. When the amplitude is zero ... they are expression is antogonistic - when one is expressed the other is silent. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Amplitude simply means the height of the period.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Periodicity analysis is quite simple if you have uniform time measurements. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have 'missing data', i.e. you time intervals are not uniform, for example you skipped a reading then this is called missing data in periodicity analysis. Missing data periodicity analysis is complicated, but its doable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in this context its quite simple, you would identify the periodicity of all genes and construct a 2x2 matrix subtracting their respective amplitudes. It would be kinda juicy to do. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The above is cool if the genes are periodic, however covariance is a better solution in general terms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you want is &lt;strong&gt;negative co-variance&lt;/strong&gt;. If two genes had a covariance of -1 ... that will mutually exclusive. As above 2x2 matrix and the co-variance across the grid. You do need to be careful that the program you are using doesn't remove negative values of the covariance.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Covariance = 1, meaning two genes are co-expressed&#xA;Covariance = 0, random distribution&#xA;Covariance = -1 mutually exclusive expression&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-03-04T16:04:32.047" LastActivityDate="2020-03-04T16:04:32.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11535" PostTypeId="2" ParentId="11533" CreationDate="2020-02-29T02:04:28.177" Score="3" Body="&lt;p&gt;Please read through the &lt;a href=&quot;https://jokergoo.github.io/ComplexHeatmap-reference/book/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ComplexHeatmap manual&lt;/a&gt;, especially the section on row and column orders: &lt;a href=&quot;https://jokergoo.github.io/ComplexHeatmap-reference/book/a-single-heatmap.html#get-orders-and-dendrograms-from-heatmap&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://jokergoo.github.io/ComplexHeatmap-reference/book/a-single-heatmap.html#get-orders-and-dendrograms-from-heatmap&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use &lt;code&gt;row_order&lt;/code&gt; and &lt;code&gt;column_order&lt;/code&gt; following a &lt;code&gt;draw()&lt;/code&gt; (which you can nest between a &lt;code&gt;pdf(null)&lt;/code&gt; and &lt;code&gt;dev.off()&lt;/code&gt; to disable drawing it on any device). That way, you'll be sure to get the right order from the first heatmap to use with the second.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By nesting, I mean like so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;h1 &amp;lt;- Heatmap(...)&#xA;pdf(NULL)&#xA;h1d &amp;lt;- draw(h1)&#xA;dev.off()&#xA;ro1 &amp;lt;- row_order(h1d)&#xA;co1 &amp;lt;- column_order(h1d)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Use whichever (row/column) order is appropriate to gene order.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2020-02-29T02:04:28.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11536" PostTypeId="1" CreationDate="2020-02-29T21:42:36.700" Score="1" ViewCount="767" Body="&lt;p&gt;I have two FASTAs/assemblies of the same species, but the bases are somewhat different. I would like to explore this. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What tools/methods exist to compare two FASTAs and see the difference in structure between them? Are there visualization tools available, something like IGV? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think users would need to do a multiple-sequence alignment between these FASTAs, and then check for the differences between them. However, I don't know if a &quot;standard approach&quot; exists to do this. &lt;/p&gt;&#xA;" OwnerUserId="1770" LastActivityDate="2021-01-02T22:48:36.153" Title="Tools for comparing/visualizing FASTAs?" Tags="|sequence-alignment|fasta|assembly|visualization|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11537" PostTypeId="2" ParentId="11536" CreationDate="2020-02-29T23:16:13.777" Score="2" Body="&lt;p&gt;You can try &lt;a href=&quot;http://darlinglab.org/mauve/mauve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mauve&lt;/a&gt; multiple genome aligner.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tWfoR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tWfoR.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2020-02-29T23:16:13.777" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11538" PostTypeId="1" AcceptedAnswerId="11539" CreationDate="2020-03-01T00:11:16.100" Score="-1" ViewCount="62" Body="&lt;p&gt;I have two groups of patients; for each patient I have an output file (RNA-seq) contains this information&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                  readcounts_union  readcounts_intersectionNotEmpt  genelength  R/FPKM_union    R/FPKM_intersectionNotEmpt&#xA;ENSG00000000003.10             8                      8               3688     1.06120392730222        1.05722142116966&#xA;&#xA;ENSG00000000419.8             31                     31               4032     3.76132572546529               3.74721015127372&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;People say&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;readcounts_union&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;reads that overlap a single feature are included in the count for that feature; reads that overlap more than one feature are not included in the counts&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;readcounts_intersectionNotEmpty&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;regions that are shared between features are excluded; reads that overlap a remaining region are included in the count for the feature that includes that region; reads that overlap regions from more than one feature are not included in the counts&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R/FPKM_union&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;readcounts_union normalised for gene length and sequencing depth&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R/FPKM_intersectionNotEmpty&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;readcounts_intersectionNotEmpty normalised for gene length and sequencing depth&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to do differential expression analysis between groups but I don't know which column I should use for DESeq2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anybody familiar with such a format?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="3541" LastEditDate="2020-03-01T08:05:25.037" LastActivityDate="2020-03-01T08:35:05.233" Title="How are these definitions related to differential expression?" Tags="|r|rna-seq|differential-expression|deseq2|normalization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11539" PostTypeId="2" ParentId="11538" CreationDate="2020-03-01T08:35:05.233" Score="2" Body="&lt;p&gt;Either of the first two columns are fine to use, though I personally prefer the &quot;intersectionNotEmpty&quot; one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an aside, that looks like a custom format file produced from post-processing htseq-count output.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-03-01T08:35:05.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11542" PostTypeId="2" ParentId="11536" CreationDate="2020-03-01T21:44:21.447" Score="2" Body="&lt;p&gt;Depending on how long the sequences are, you could use the LAST Web Service:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://lastweb.cbrc.jp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://lastweb.cbrc.jp/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It has an HTML view that shows base-level differences with colour-coding.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;LAST [download / command-line version] can also be used for producing whole-genome comparison dot-plots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another thing I do for comparing assemblies is to chop one assembly into smaller pieces (e.g. 10kb), map the pieces to other other assembly to generate BAM output, then look at the BAM pileup file in a BAM viewer (e.g. &lt;a href=&quot;https://ics.hutton.ac.uk/tablet/download-tablet/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tablet&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2020-03-01T21:44:21.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11543" PostTypeId="1" AcceptedAnswerId="12855" CreationDate="2020-03-01T23:28:44.973" Score="0" ViewCount="505" Body="&lt;p&gt;I'm working on calling somatic variants from solid tissue tumors. I have one normal and one tumor sample for each patient. I plan to use Mutect2 to call somatic variants after preprocessing the data with the GATK Best Practices pipeline. Mutect2 recommends the use of a &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035890631-Panel-of-Normals-PON-&quot; rel=&quot;nofollow noreferrer&quot;&gt;Panel of Normals (PON)&lt;/a&gt;, but I can't figure out what samples should be used to built the PON. The guide says&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Additionally, the samples should come from subjects that were young and healthy to minimize the chance of using as normal a sample from someone who has an undiagnosed tumor. &lt;strong&gt;Normals are typically derived from blood samples.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;My normal samples are from solid tissue, not blood. &#xA;Can I use the normal samples from each patient to build the panel of normals? Or would this cause problems because I'm using reusing the same normals for both the PON and as the normal sample for somatic variant calling? My first thought is that most somatic variants in individual normal samples should be filtered out by pooling all the normals together to build the PON but I wasn't sure if there are other reasons to avoid using non-blood based normal samples to build the PON&lt;/p&gt;&#xA;" OwnerUserId="6674" LastActivityDate="2020-04-07T02:39:54.530" Title="What samples can be used for a Mutect2 Panel of Normals" Tags="|ngs|variant-calling|quality-control|gatk|somatic|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11544" PostTypeId="1" AcceptedAnswerId="11546" CreationDate="2020-03-02T10:38:59.947" Score="0" ViewCount="96" Body="&lt;p&gt;These are the list of files &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Control_Input_sorted.bam_rem.bam  &#xA;Control_H2BUb_sorted.bam_rem.bam  &#xA;Control_IgG_sorted.bam_rem.bam &#xA;PTPN6_g2_6_H2Bub_sorted.bam_rem.bam  &#xA;PTPN6_g2_6_Input_sorted.bam_rem.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So to get unique files i do this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; ls -1 *bam | sort | sed -r 's/_sorted.bam_rem.bam//g' | sort | uniq &#xA;Control_Input&#xA;Control_H2BUb&#xA;Control_IgG&#xA;PTPN6_g2_6_H2Bub&#xA;PTPN6_g2_6_Input&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now i have to run each sample against input and IgG for peak-calling.&#xA;Like  &lt;code&gt;Control_H2BUb_sorted.bam_rem.bam&lt;/code&gt; against &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Contol_Input_sorted.bam_rem.bam &amp;amp; Control_IgG_sorted.bam_rem.bam and &#xA;&#xA;&#xA;PTPN6_g2_6_H2Bub_sorted.bam_rem.bam against PTPN6_g2_6_Input_sorted.bam_rem.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So how do I parse and execute using gnu parallel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I came across this &lt;a href=&quot;http://crazyhottommy.blogspot.com/2015/09/macs2-parallel-peak-calling.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; where it does something like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat sample_names.txt | parallel --max-procs=12 'macs2 callpeak -t {}-A-NC.sorted.bam \&#xA; -c {}-G-NC.sorted.bam -g hs -n {}-A-NC-sharp-model -q 0.01 --outdir {}-A-NC-sharp-model-peaks 2&amp;gt; {}-A-NC-sharp-model.stderr'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now im not sure how to pass the file names as argument for IP and Input or IgG.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If i use logic a little bit i have to fix Control and PTPN6_g2_6 as constant naming!.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion or help would be really appreciated.&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2020-03-02T10:57:47.847" LastActivityDate="2020-03-02T13:47:48.827" Title="gnu parallel for macs peak calling" Tags="|shell|macs2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11545" PostTypeId="1" AcceptedAnswerId="11563" CreationDate="2020-03-02T11:33:07.297" Score="2" ViewCount="93" Body="&lt;p&gt;I want to align a short sequence to a reference, and both are encoded in the &lt;a href=&quot;https://en.wikipedia.org/wiki/Nucleic_acid_notation&quot; rel=&quot;nofollow noreferrer&quot;&gt;extended IUPAC notation&lt;/a&gt; (i.e. 16 letters).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What alignment software can I use for that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, in &lt;a href=&quot;http://bowtie-bio.sourceforge.net/bowtie2/news.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bowtie&lt;/a&gt; &lt;em&gt;&quot;all non-A/C/G/T characters in reads become Ns&quot;,&lt;/em&gt; so it ignores which extended letters are matching.&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2020-03-04T09:24:25.143" Title="Sequence alignment of extended IUPAC sequences" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11546" PostTypeId="2" ParentId="11544" CreationDate="2020-03-02T13:47:48.827" Score="1" Body="&lt;p&gt;not gnuparallel but &lt;strong&gt;nextflow&lt;/strong&gt;. I don't know macs and what is 'first.bam' and 'second.bam' is not clear not me but I hope you'll get the idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The nextflow:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Channel.fromPath(params.bams).splitCsv(header: false,sep:'\t',strip:true).map{T-&amp;gt;file(T[0])}.into{bams1;bams2}&#xA;&#xA;boolean isControl(file) {&#xA;    return file.name.endsWith(&quot;_Input_sorted.bam_rem.bam&quot;) || file.name.endsWith(&quot;_IgG_sorted.bam_rem.bam &quot;);&#xA;    }&#xA;&#xA;bams1.filter{T-&amp;gt;isControl(T)}.set{controls}&#xA;bams2.filter{T-&amp;gt;!isControl(T)}.set{experiment}&#xA;&#xA;process mac {&#xA;    maxForks 14&#xA;    tag &quot;&lt;span class=&quot;math-container&quot;&gt;${bam1} vs $&lt;/span&gt;{bam2}&quot;&#xA;    input:&#xA;        set bam1 , bam2  from experiment.combine(controls)&#xA;    output:&#xA;        file(&quot;output.peaks&quot;) into mac_peak&#xA;    script:&#xA;    &quot;&quot;&quot;&#xA;    # check syntax please&#xA;    macs2 callpeak -t {bam1.toRealPath()}  -c ${bam2.toRealPath()} -g hs -n out-model -q 0.01 --outdir output.peaks&#xA;    &quot;&quot;&quot;&#xA;    }     &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;invoke:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;find /dir1/dir2 -type f -name &quot;*.bam&quot; &amp;gt; input.list&#xA;&#xA;nextflow run --bams input.list workflow.nf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastActivityDate="2020-03-02T13:47:48.827" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11547" PostTypeId="1" CreationDate="2020-03-02T14:47:36.357" Score="5" ViewCount="233" Body="&lt;p&gt;I am wondering about an article in the Economist here:&#xA;&lt;a href=&quot;https://www.economist.com/briefing/2020/02/29/covid-19-is-now-in-50-countries-and-things-will-get-worse&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.economist.com/briefing/2020/02/29/covid-19-is-now-in-50-countries-and-things-will-get-worse&lt;/a&gt;&#xA;There is a graph there&#xA;&lt;a href=&quot;https://i.stack.imgur.com/O4X8w.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/O4X8w.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The explanation is as follows&#xA;The course of an epidemic is shaped by a variable called the reproductive rate, or R. It represents, in effect, the number of further cases each new case will give rise to. If R is high, the number of newly infected people climbs quickly to a peak before, for want of new people to infect, starting to fall back again (see chart 2). If R is low the curve rises and falls more slowly, never reaching the same heights. With sars-cov-2 now spread around the world, the aim of public-health policy, whether at the city, national or global scale, is to flatten the curve, spreading the infections out over time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am having trouble with this. There are 2 curves here. First of all, I am not sure of the different processes that determine them. It says one is without measures. The other is with measures. But it also says one is with high R and the other with low R. Is R a fixed constant for a disease? Alternatively, is there some variability in R and does that degree of variation affect transmission? This is not explained or stated in the article and I haven't heard R described in this way. At best it is not written clearly in my opinion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In addition, does without measures necessarily imply high R? Is that what they are getting at? Also, it looks to me as though the blue graph goes to zero very quickly, basically when t is equal to the mean of the orange curve. That means that without measures the epidemic will end quickly (because it exhausts all the possible patients). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally the areas under the two curves is about the same. This means that the number of patients is about the same with or without measures. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Anyway, can anyone clear up any of this?&lt;/p&gt;&#xA;" OwnerUserId="7317" LastEditorUserId="3848" LastEditDate="2020-11-30T22:28:59.527" LastActivityDate="2020-11-30T22:28:59.527" Title="Economist article on coronavirus" Tags="|statistics|covid-19|sars-cov-2|epidemiology|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11548" PostTypeId="1" AcceptedAnswerId="14879" CreationDate="2020-03-02T16:07:02.210" Score="1" ViewCount="23" Body="&lt;p&gt;A colleague has uploaded about a dozen candidate disease genes to GeneMatcher, described &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4833888/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://www.nature.com/articles/s41436-018-0383-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and located &lt;a href=&quot;https://genematcher.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;,  with excellent success. However last year and again recently he has encountered the case wherein a case was uploaded and a cohort formed, but a few months down the line one of the dropped out collaborators published an n=1 paper in a low tier journal, basically using GeneMatcher to check whether the disease link of the gene was legitimate, to assess the phenotypic range and see what test are worth doing.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there documented precedence of this behaviour (preferably citable)? If not,&lt;/li&gt;&#xA;&lt;li&gt;does anyone have personal experience of this and what is the ratio of incidences over total submissions (I know that is hearsay, but it would be very helpful)?&lt;/li&gt;&#xA;&lt;li&gt;And also has anyone successfully taken action (say getting user barred)?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="3848" LastEditDate="2020-11-19T20:29:39.590" LastActivityDate="2020-11-21T18:15:55.703" Title="Scooping and GeneMatcher" Tags="|gene|mutations|publishing|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11549" PostTypeId="1" AcceptedAnswerId="11552" CreationDate="2020-03-02T16:46:18.850" Score="0" ViewCount="588" Body="&lt;p&gt;I am using several cellranger tools/pipelines to analyze some data, specifically, mkgtf, mkref, count, and aggr. I am using mkgtf and mkref to include a PI-synthesized protein to my reference. I have finished the analysis but am not seeing the protein in my output, so I am going back to the start to verify the process. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unlike the documentation (&lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/advanced/references&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/advanced/references&lt;/a&gt;), my GTF file was not downloaded from ENSEMBL or UCSC, but NCBI (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/AY678269.1?report=genbank&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/AY678269.1?report=genbank&lt;/a&gt;). This GFF file does not have the same key nomenclature as seen in ENSEMBL or USCS.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did use the mkgtf tool, but the created file is identical to the input. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cellranger mkgtf custom_ref/AY678269_genes.gtf reference_data/AY678269_mkgtf_genes.gtf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I went ahead and used cat to add the fasta and gtf file from the synthesized protein to the respective reference fasta and gene files, then ran mkref. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The s.protein is in the fasta file: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;lcl|AY678269.1_cds_AAV52169.1_1 [protein=tandem-dimer red fluorescent protein] [protein_id=AAV52169.1] [location=1..1431] [gbkey=CDS] &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;as well as the gene list &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##sequence-region AY678269.1 1 1431&#xA;##species https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?id=32630&#xA;AY678269.1      Genbank region  1       1431    .       +       .&#xA;AY678269.1      Genbank CDS     1       1431    .       +       0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However this protein is not found in the output...surely I must have made an error. Does anyone have any ideas about what I could be doing incorrectly?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do I need to change the entire nomenclature of the GFF keys to match what I see from ENSEMBL (e.g., gene_id &quot;&quot;; gene_version &quot;&quot;; transcript_id, etc...)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any assistance is greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2934" LastActivityDate="2020-03-02T18:58:16.967" Title="Using cellranger mkgtf w/ NCBI GFF3 files" Tags="|cellranger|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11550" PostTypeId="1" AcceptedAnswerId="11551" CreationDate="2020-03-02T16:53:01.377" Score="0" ViewCount="539" Body="&lt;p&gt;I have to filter the raw sequencing reads based on the following criteria:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Remove reads containing adapters&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Remove reads containing N &gt; 10% (N represents base that could not be determined)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Remove reads where the Qscore (Quality value) of over 50% bases of the read is &amp;lt;= 5&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The reads are paired-end fastq files. I have been able to do the first part by using trimmomatic:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;trimmomatic \&#xA;  PE in_1.fq.gz in_2.fq.gz \&#xA;  pair_out_1 unpair_out_1 pair_out_2 unpair_out_2 \&#xA;  ILLUMINACLIP:adapter.fa:2:30:5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The other 2 steps are a bit more challenging since I have not found a tool that perform exactly what is required. I could write some custom code of course but I would like to use the existing tools if possible. No need to reinvent the wheel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any suggestions on how to proceed?&lt;/p&gt;&#xA;" OwnerUserId="6410" LastActivityDate="2020-03-02T18:25:07.700" Title="Filtering raw sequencing reads" Tags="|fastq|quality-control|filtering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11551" PostTypeId="2" ParentId="11550" CreationDate="2020-03-02T18:25:07.700" Score="2" Body="&lt;p&gt;I think you can come close to this with &lt;code&gt;cutadapt&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cutadapt \&#xA;  -a &lt;span class=&quot;math-container&quot;&gt;${FWD_ADAPTER} -A $&lt;/span&gt;{REV_ADAPTER} \         ## adapter&#xA;  --max-n 0.1 \                                 ## do not allow &amp;gt; 10% Ns&#xA;  -q 5 \                                        ## remove bases with basequal &amp;lt; 5&#xA;  --minimum-length (readlength/2) \             ## minimum length after trimming&#xA;  -o read1_trimmed.fq -p read2_trimmed.fq \     ## outputs&#xA;  read1.fq read2.fq                             ## inputs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will remove reads with more than 10% of bases being &lt;code&gt;N&lt;/code&gt; and remove bases with base quality below 5. If you then set &lt;code&gt;--minimum-length&lt;/code&gt; to half the read length, this should also remove reads where more than half of the bases had a base quality below 5 and therefore got trimmed away. Poor base quality in Illumina reads typically happens towards the 3' end so I think this little workaround is a sufficient proxy to achieve your goal. A base quality of 5 is very low in any case, so if sequencing went normal the typical quality should be notably higher. I would be surprised if this third filter is really necessary and will eliminate a notable number of reads. &lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-03-02T18:25:07.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11552" PostTypeId="2" ParentId="11549" CreationDate="2020-03-02T18:52:17.277" Score="1" Body="&lt;p&gt;Regarding your comment:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;This GFF file does not have the same key nomenclature as seen in&#xA;  ENSEMBL or USCS&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The link you have provided corresponds to an entry in &lt;code&gt;Genbank&lt;/code&gt; format and not to &lt;code&gt;GFF&lt;/code&gt; format. I believe you need a file in &lt;code&gt;GTF&lt;/code&gt; format in particular.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Nevertheless, given the fact that you have manually added the corresponding entries to the reference &lt;code&gt;fasta&lt;/code&gt; and &lt;code&gt;gtf&lt;/code&gt; files prior to &lt;code&gt;mkref&lt;/code&gt;, you should have seen your gene of interest in the output. I would double check one of the original fasta headers and original gtf entries in the reference file and match these for the new entry.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2020-03-02T18:58:16.967" LastActivityDate="2020-03-02T18:58:16.967" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11553" PostTypeId="2" ParentId="11547" CreationDate="2020-03-02T19:49:41.870" Score="5" Body="&lt;p&gt;R is a reference to R0, otherwise known as the basic reproductive rate, and means the number of new cases from a single patient infection. R0 has to be above 1 for a disease to persist. It is a classic equation which account for the probability of transmission against immunity. The y-axis is the number of infecteds, very poorly presented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that R0 will be lower if there is containment (less people to infect because you can't move around), but will last longer, possibly because the number of naive patients (not encountered the virus) will be larger over a longer period of time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That particular graph appears to be saying that everyone at risk of infection will get infected with or without control measures (the surface area under the graphs is approximately the same*) and the only thing that will halt the virus is acquired immunity. If this is mainstream economic thinking then I beg to differ, albeit I note the source is CDC (again see note*). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, all other respiratory viral infections are seasonal. If COVID-19  is seasonal containment is the right choice, because the window of transmission is limited.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*, Given this is of CDC origin I think either its a mistake, or at the very least was taken out of context, because the idea is that overall there is a lower number of people infected as a result of &quot;social distancing&quot; and that does not appear to be what the two curves show.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-03-09T18:20:40.817" LastActivityDate="2020-03-09T18:20:40.817" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11554" PostTypeId="1" CreationDate="2020-03-03T03:45:37.533" Score="0" ViewCount="555" Body="&lt;p&gt;Sorry if this has been asked before, I've done a quick search and I don't think there's an easy explanation for me to understand.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm really new to bioinformatics/RNASEQ analysis, having only taken up an attempt to process my RNASEQ data last week. I have 0 background and some rudimentary knowledge of command line. So far I've managed to get access to my universities HPC and perform a fastqc on my fastq data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I'm stuck at at the moment is attempting to trim my RNASEQ data and remove adapters. My data had been run through Novaseq and I'm not exactly sure how to trim my current RNASEQ data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've attempted to run the following script using trimmomatic&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# go to working folder&#xA;cd /project/CHR_RNASEQ/reads&#xA;&#xA;# run trimmomatic&#xA;trimmomatic SE -threads 8 -phred33 SLIDINGWINDOW:4:20 ILLUMINACLIP:$TRIM/adapters/TruSeq3-SE-2.fa:2:30:10 MINLEN:36 1_R1.fastq.gz 1_R1_trimmed.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It did log a job in the HPC, but I had no output at the end. I didn't receive an error either. This is unfortunately just some code I had lifted and trialed and errored until something finally ran, but I'm very sure I'm very wrong and any help with understanding this would be really appreciated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cheers&lt;/p&gt;&#xA;" OwnerUserId="7322" LastEditorUserId="3541" LastEditDate="2020-03-03T05:18:55.240" LastActivityDate="2020-03-03T05:35:58.857" Title="How to run trimmomatic in HPC" Tags="|rna-seq|data-preprocessing|trimming|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11556" PostTypeId="2" ParentId="11554" CreationDate="2020-03-03T05:35:58.857" Score="1" Body="&lt;p&gt;Depending on the cluster management tool, you might have received e-mails when the &quot;job&quot; begins and ends. If so, you can check the &quot;Exit status&quot; of the job. For example, in the case of our HPC the relevant lines from the &quot;job completion e-mail&quot; would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Execution terminated&#xA;Exit_status=0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;BTW, at first I was confused with the phrase &quot;Execution terminated&quot;, however, in the scheduler used in our system, it means &quot;the run has been successfully or unsuccessfully completed&quot;. &lt;code&gt;Exit_status=0&lt;/code&gt; int the next line means a successful run and for the unsuccessful runs there is a list with the error codes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Secondly, if your run was successful, you would see new fastq files containing &quot;_trimmed&quot; in the filename (reading this from your code). These fastqs should be more or less similar in size to the original depending on how aggressive the trimming settings are.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can provide more information, i.e. the type of cluster management tool used, the log file, ..., you can get more help and precise answers from the community.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-03-03T05:35:58.857" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11557" PostTypeId="1" CreationDate="2020-03-03T08:10:45.590" Score="2" ViewCount="306" Body="&lt;p&gt;Is there any tool available to predict the melting temperature for rna/lna oligos with its perfect complement ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know this amazing website from qiagen/geneglobe that estimates the tm value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But is there any command line tool that does this? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can estimate the normal tm value with RNAcofold and the -T option but there is no lna option. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I searched for a while but i can only find online services. &lt;/p&gt;&#xA;" OwnerUserId="2766" LastActivityDate="2022-04-23T12:06:25.860" Title="Tool for rna/lna melting temperature prediction" Tags="|rna|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11558" PostTypeId="1" AcceptedAnswerId="11559" CreationDate="2020-03-03T09:34:22.880" Score="1" ViewCount="50" Body="&lt;p&gt;I'm trying to build a genomic database for DNA alignments.&#xA;I started with NCBI accessions, but the data is very multiplicative, so I want to use subset of [max] N different strains for each specie.&#xA;my question is: I can i obtain the minimal N that the samples will be diverse enough and will cover the specie good?&lt;/p&gt;&#xA;" OwnerUserId="7326" LastActivityDate="2020-03-03T14:27:02.457" Title="choosing a good representative genome subset" Tags="|sequence-alignment|clustering|subset|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11559" PostTypeId="2" ParentId="11558" CreationDate="2020-03-03T14:27:02.457" Score="0" Body="&lt;p&gt;Check out the progenomes database: &lt;a href=&quot;http://progenomes.embl.de/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://progenomes.embl.de/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1802" LastActivityDate="2020-03-03T14:27:02.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11560" PostTypeId="1" CreationDate="2020-03-03T18:39:14.103" Score="0" ViewCount="85" Body="&lt;p&gt;I'm sorry if this is a naive question, but I get an odd thing going on.&#xA;I'm performing blast remotely in this way: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;'blastn -task megablast -outfmt &quot;6 std ssciname&quot; -reward 1 -penalty -3  -db nt_v5 -max_target_seqs 100 -word_size 22 -query ' + final_file + &quot; -taxidlist &quot; + Plants_DB + &quot; -out &quot; + blastfile&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, when I see my remote results, I get a species whose seqid gives me &lt;em&gt;Vaccinium oxycoccos&lt;/em&gt;. And when I put this seqid in blast online, the species is totally different. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(With this blast command, I perform a blast search against a specific list of IDS.) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can this happen?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="2686" LastEditorUserId="2686" LastEditDate="2020-03-04T15:24:39.743" LastActivityDate="2020-03-04T15:24:39.743" Title="Wrong blast results" Tags="|blastn|windows|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="11561" PostTypeId="2" ParentId="11557" CreationDate="2020-03-03T22:47:57.333" Score="1" Body="&lt;p&gt;There are several programs for simulating/predicting DNA melting but you could try them for RNA.&#xA;You can look here &lt;a href=&quot;http://bioinformatics.org/meltsim/wiki/Main/HomePage&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://bioinformatics.org/meltsim/wiki/Main/HomePage&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6518" LastActivityDate="2020-03-03T22:47:57.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11562" PostTypeId="1" CreationDate="2020-03-04T07:00:17.617" Score="1" ViewCount="92" Body="&lt;p&gt;I have an MSA from MAFFT (as a FASTA file; there are several sequences in the MSA), and I would like to quantify the &amp;quot;variants&amp;quot; between these sequences.&lt;/p&gt;&#xA;&lt;p&gt;What is the standard tool to do something like this?&lt;/p&gt;&#xA;&lt;p&gt;I would prefer to know the non-consensus variants in the the entire MSA, but that may be too tricky, so I would like to figure out the non-consensus variants between those select sequences.&lt;/p&gt;&#xA;&lt;p&gt;I'm familiar with calculating an MSA, and then visualizing it---but I would like an output file with all of the different bases.&lt;/p&gt;&#xA;" OwnerUserId="1770" LastEditorUserId="73" LastEditDate="2021-07-14T11:29:48.413" LastActivityDate="2021-10-20T18:03:39.557" Title="Given a multiple sequence alignment, how do I output all of the non-consensus characters by location?" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11563" PostTypeId="2" ParentId="11545" CreationDate="2020-03-04T09:24:25.143" Score="1" Body="&lt;p&gt;You can try using Biostrings, depends on how large is your reference, if it is small, you can just pairwiseAlignment with the nucleotideSubstitutionMatrix which supposedly :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Biostrings)&#xA;data(phiX174Phage)&#xA;&#xA;x = DNAString(&quot;TCCAWRSCGCAGAAGT&quot;)&#xA;&#xA;lapply(phiX174Phage,function(y)pairwiseAlignment(x,y,&#xA;substitutionMatrix = nucleotideSubstitutionMatrix()))&#xA;&#xA;$Genbank&#xA;Global PairwiseAlignmentsSingleSubject (1 of 1)&#xA;pattern: -------------TCCAWRSCGCAGAAGT...----------------------------&#xA;subject: GAGTTTTATCGCTTCCATGACGCAGAAGT...TAAAAATGATTGGCGTATCCAACCTGCA&#xA;score: -21491.92 &#xA;&#xA;$RF70s&#xA;Global PairwiseAlignmentsSingleSubject (1 of 1)&#xA;pattern: -------------TCCAWRSCGCAGAAGT...----------------------------&#xA;subject: GAGTTTTATCGCTTCCATGACGCAGAAGT...TAAAAATGATTGGCGTATCCAACCTGCA&#xA;score: -21491.92 &#xA;&#xA;$SS78&#xA;Global PairwiseAlignmentsSingleSubject (1 of 1)&#xA;pattern: -------------TCCAWRSCGCAGAAGT...----------------------------&#xA;subject: GAGTTTTATCGCTTCCATGACGCAGAAGT...TAAAAATGATTGGCGTATCCAACCTGCA&#xA;score: -21491.92 &#xA;&#xA;$Bull&#xA;Global PairwiseAlignmentsSingleSubject (1 of 1)&#xA;pattern: -------------TCCAWRSCGCAGAAGT...----------------------------&#xA;subject: GAGTTTTATCGCTTCCATGACGCAGAAGT...TAAAAATGATTGGCGTATCCAACCTGCA&#xA;score: -21491.92 &#xA;&#xA;$G97&#xA;Global PairwiseAlignmentsSingleSubject (1 of 1)&#xA;pattern: -------------TCCAWRSCGCAGAAGT...----------------------------&#xA;subject: GAGTTTTATCGCTTCCATGACGCAGAAGT...TAAAAATGATTGGCGTATCCAACCTGCA&#xA;score: -21491.92 &#xA;&#xA;$NEB03&#xA;Global PairwiseAlignmentsSingleSubject (1 of 1)&#xA;pattern: -------------TCCAWRSCGCAGAAGT...----------------------------&#xA;subject: GAGTTTTATCGCTTCCATGACGCAGAAGT...TAAAAATGATTGGCGTATCCAACCTGCA&#xA;score: -21491.92 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If it is much larger, you can convert the sequence of interest to a pwm and find it in the reference fasta, like shown in &lt;a href=&quot;https://www.bioconductor.org/help/course-materials/2009/SeattleNov09/Biostrings/MatchAlign.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this example&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-03-04T09:24:25.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11564" PostTypeId="2" ParentId="4942" CreationDate="2020-03-04T13:12:37.890" Score="0" Body="&lt;p&gt;If you are an R user, then you can easily do so with &lt;code&gt;biomaRt&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(&quot;biomaRt&quot;)&#xA;library(&quot;dplyr&quot;)&#xA;&#xA;genes &amp;lt;- read.delim(&quot;genes.file.txt&quot;, header=T)&#xA;&#xA;head(genes)&#xA;          gene_id&#xA;1 ENSG00000000003&#xA;2 ENSG00000000005&#xA;3 ENSG00000000419&#xA;4 ENSG00000000457&#xA;5 ENSG00000000460&#xA;6 ENSG00000000938&#xA;&#xA;## Build a biomart query &#xA;# In the example below, I use the human gene annotation from Ensembl release 82 located on &quot;sep2015.archive.ensembl.org&quot;, More about the ensembl_build can be found on &quot;http://www.ensembl.org/info/website/archives/index.html&quot;&#xA;&#xA;dataset=&quot;hsapiens_gene_ensembl&quot;&#xA;mart = biomaRt::useMart(&quot;ENSEMBL_MART_ENSEMBL&quot;, dataset = dataset, &#xA;host = paste0(&quot;sep2015&quot;, &quot;.archive.ensembl.org&quot;), path = &quot;/biomart/martservice&quot;, archive = FALSE)&#xA;&#xA;annotations &amp;lt;- biomaRt::getBM(mart = mart, attributes=c(&quot;ensembl_gene_id&quot;, &quot;external_gene_name&quot;, &quot;start_position&quot;, &quot;end_position&quot;))&#xA;annotations &amp;lt;- dplyr::transmute(annotations, ensembl_gene_id, external_gene_name, gene_length = end_position - start_position)&#xA;&#xA;# Filter and re-order gene.annotations to match the order in your input genes list&#xA;final.genes &amp;lt;- annotations %&amp;gt;% dplyr::filter(ensembl_gene_id %in% genes&lt;span class=&quot;math-container&quot;&gt;$gene_id)&#xA;final.genes &amp;lt;- final.genes[order(match(final.genes$&lt;/span&gt;ensembl_gene_id, genes$gene_id)),]; rownames(final.genes) &amp;lt;-NULL&#xA;&#xA;head(final.genes)&#xA;  ensembl_gene_id external_gene_name gene_length&#xA;1 ENSG00000000003             TSPAN6       12882&#xA;2 ENSG00000000005               TNMD       15083&#xA;3 ENSG00000000419               DPM1       23688&#xA;4 ENSG00000000457              SCYL3       44636&#xA;5 ENSG00000000460           C1orf112      192073&#xA;6 ENSG00000000938                FGR       23213&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4305" LastActivityDate="2020-03-04T13:12:37.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11565" PostTypeId="1" AcceptedAnswerId="11566" CreationDate="2020-03-04T13:58:52.667" Score="1" ViewCount="2014" Body="&lt;p&gt;I have been using minimap2 for a few month now and I just found a script that follows these steps:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 [-x preset] -d target.mmi target.fa&#xA;minimap2 -a [-x preset] target.mmi query.fa &amp;gt; output.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have never seen this &lt;code&gt;.mmi&lt;/code&gt; index before. I always used minimap2 directly like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 -a [-x preset] target.fa query.fastq &amp;gt; output.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So my question is why should I index my reference genome before mapping? Is it only for speed up? Will it up speed my code if I do it? Or it is already done without specifying it?&lt;/p&gt;&#xA;" OwnerUserId="6273" LastEditorUserId="3694" LastEditDate="2022-07-22T10:31:33.933" LastActivityDate="2022-07-22T10:31:33.933" Title="Why index a sequence file with minimap2?" Tags="|minimap2|genome-index|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11566" PostTypeId="2" ParentId="11565" CreationDate="2020-03-04T15:50:46.837" Score="6" Body="&lt;p&gt;If you don't provide an index, minimap2 will just generate the index on the fly (so, yes, it's done without specifying it). As such the mapping step will take longer. However, index generation is supposed to be very fast in minimap2 (a few minutes for the human genome), so it probably won't matter much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Doing it separately will therefore also not speed up your code unless you end up using the pre-generated index multiple times.&lt;/p&gt;&#xA;" OwnerUserId="6955" LastActivityDate="2020-03-04T15:50:46.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11567" PostTypeId="1" CreationDate="2020-03-04T17:42:02.873" Score="0" ViewCount="88" Body="&lt;p&gt;I have been trying to assemble my genome via canu. Canu gave me about 200 contigs using default parameters. My genomesize is ~9Mbp. Can you recommend some parameters I can tweak to decrease the contigs number? &lt;/p&gt;&#xA;" OwnerUserId="7336" LastActivityDate="2020-03-04T18:35:06.430" Title="Optimum parameters for genome assembly via canu" Tags="|assembly|canu|microbial-genomics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11568" PostTypeId="2" ParentId="11567" CreationDate="2020-03-04T18:35:06.430" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://canu.readthedocs.io/en/latest/parameter-reference.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;canu parameter reference guide&lt;/a&gt; is always a good place to start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It would help to know a little bit more about what your assembly looks like. E.g. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;what is your n50, n90...? &lt;/li&gt;&#xA;&lt;li&gt;What is your dataset coverage?&lt;/li&gt;&#xA;&lt;li&gt;Have you run any tools like &lt;a href=&quot;http://quast.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;quast&lt;/a&gt; to evaluate quality? &lt;/li&gt;&#xA;&lt;li&gt;What is your input data type (probably PacBio)? Read length distribution?&lt;/li&gt;&#xA;&lt;li&gt;What is your expected chromosome/replicon number? &lt;/li&gt;&#xA;&lt;li&gt;Have you compared the assembly to any other data, e.g. genomes of close relatives? (quast can do this)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Basically, &lt;strong&gt;why do you think that 200 contigs is too many contigs&lt;/strong&gt;? Sometimes you just don't have enough coverage to get more contiguous/&quot;nice&quot; assemblies. That's not to say it's not possible, of course, just that we don't have enough information to evaluate.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's possible that some of those contigs are small and artefactual, so you might get rid of them with just length/coverage filtering. You can do some of this in canu itself (see &quot;Output Filtering&quot; section).&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-03-04T18:35:06.430" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11571" PostTypeId="1" CreationDate="2020-03-05T05:58:44.933" Score="2" ViewCount="129" Body="&lt;p&gt;please would you let me know, is there a package in R/python/C that has implemented &lt;strong&gt;randomization/permutation tests for joint genomic analyses&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(an example of joint genomic analyses -- when jointly considering both GENE EXPRESSION and PROTEIN BINDING along the DNA).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The context of my question is the following :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;let's consider &lt;strong&gt;1000 UP_REGULATED genes with increased PROTEIN X and with a HISTONE MARK Y :&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;in order to show that PROTEIN X is related to HISTONE MARK Y for 1000 UP-REGULATED genes, what &quot;controls&quot; would you use for the comparison :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- 1000 RANDOM GENES (and multiple randomization tests)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- 1000 UP-REG GENES with NO PROTEIN X, NO HISTONE MARK Y&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- 1000 UP-REG GENES with PROTEIN X, and NO HISTONE MARK Y&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- 1000 UP-REG GENES with NO PROTEIN X, and with HISTONE MARK Y&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- 1000 NOT-UP-REG GENES with NO PROTEIN X, NO HISTONE MARK Y&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- 1000 NOT-UP-REG GENES with PROTEIN X, and NO HISTONE MARK Y&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- 1000 NOT-UP-REG GENES with NO PROTEIN X, and with HISTONE MARK Y&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-- anything else ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;any suggestions are very welcome. thanks a lot,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;bogdan&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ps : 've posted a similar question on BioC website : &lt;a href=&quot;https://support.bioconductor.org/p/128828/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.bioconductor.org/p/128828/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6772" LastEditorUserId="3848" LastEditDate="2020-04-19T10:55:33.800" LastActivityDate="2020-05-19T11:06:15.727" Title="a package with randomization/permutation tests for genomic analyses" Tags="|phylogenetics|statistics|phylogeny|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11572" PostTypeId="1" AcceptedAnswerId="11596" CreationDate="2020-03-05T10:43:33.020" Score="1" ViewCount="3026" Body="&lt;p&gt;I'm trying to run MaxQuant on a linux laptop, hower I'm constantly running into problems with MaxQuant crashing at different stages&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to use&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;mono 6.8&lt;/li&gt;&#xA;&lt;li&gt;mono 5.4.1&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;on &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;centos&lt;/li&gt;&#xA;&lt;li&gt;ubuntu&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;each with MaxQuant&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1.6.3&lt;/li&gt;&#xA;&lt;li&gt;1.6.7&lt;/li&gt;&#xA;&lt;li&gt;1.6.11&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;(everything using docker), but I had no success so far.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried to use the ahead of time compilation of mono, also without sucess.&#xA;MQ mostly stalls at the &quot;preparing searches&quot; stage.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone managed to get MQ running properly on linux and can tell me, which linux distro/version, mono version and MQ version you are using?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any other help is of course also welcome&lt;/p&gt;&#xA;" OwnerUserId="7339" LastActivityDate="2021-09-02T22:18:19.690" Title="Running MaxQuant on Linux" Tags="|linux|proteomics|mass-spectrometry|maxquant|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11573" PostTypeId="1" AcceptedAnswerId="16388" CreationDate="2020-03-05T11:04:06.110" Score="0" ViewCount="84" Body="&lt;p&gt;I would like to know how to perform this global Pol II occupancy which is shown here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/puBbr.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/puBbr.jpg&quot; alt=&quot;Pol II occupancy analysis&quot; /&gt;&lt;/a&gt; &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4679144/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Paper&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Figure 1G&lt;/p&gt;&#xA;&lt;p&gt;So in this figure they are analysing the Pol II occupancy in Control,PAF1 KD and LEO1.&lt;/p&gt;&#xA;&lt;p&gt;I have similar data.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In order to determine whether PAF1 has a direct effect on the regulation of Pol II at the promoter-proximal region, ChIP-seq was performed. Heatmaps ranked by decreasing Pol II occupancy show a positive correlation with PAF1 and Pol II (Figure 1G). To corroborate these findings, we also performed ChIP-seq with LEO1, as this was the only other antibody that we tested that gave any significant ChIP-seq signal. Like PAF1, LEO1 is found throughout gene bodies with particular enrichment around the TSS (Figure 1G). Interestingly, LEO1 was not broadly lost from chromatin upon PAF1 knockdown, but the distribution was altered, including reduced occupancy in the promoter-proximal region&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So what I have done so far in terms of analysis is I did peak-calling using homer for my control and KD condition along with their Pol II chip.I was going for this differential binding analysis but this is now how i Pol II data is done after reading the paper I got. In &lt;a href=&quot;http://homer.ucsd.edu/homer/ngs/analyzeRNA.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;homer&lt;/a&gt; there is this function analyzeRepeats.pl which is provided in this section &lt;strong&gt;Measuring Gene Expression in Exons vs. Gene Bodies.&lt;/strong&gt; telling genes &lt;em&gt;(default) - Counts tags on the full gene body (TSS to TTS).This is useful for GRO-Seq where we expect coverage across the entire transcript.Can also be used to quantify H3K36me3 or PolII ChIP-Seq.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;How do I proceed and what tool should I use to show the Pol II occupancy at different condition and as of now I don't have respective rna seq data, do i need expression data of genes to perform occupancy analysis.&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="73" LastEditDate="2021-07-14T11:17:26.930" LastActivityDate="2021-07-14T11:27:19.127" Title="Pol_II genome wide occupancy analysis" Tags="|chip-seq|deeptools|homer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11574" PostTypeId="1" AcceptedAnswerId="13530" CreationDate="2020-03-05T10:50:56.963" Score="1" ViewCount="286" Body="&lt;p&gt;Basically a GenBank file consists of gene entries (announced by 'gene') followed by its corresponding 'CDS' entry (only one per gene) like the two shown here below. I would like to extract part of the data from the input file shown below according to the following rules and print it in the terminal. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;There are two blocks of gene data shown below. I want to extract part of both blocks. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The extracted text for each block starts with a line that contains spaces at the beginning of the line followed by &lt;code&gt;gene&lt;/code&gt; &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The extracted text for each block ends with a line that contains &lt;code&gt;/db_xref=&quot;GeneID&lt;/code&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The perl and awk tags are just suggestions. An answer can use a different program(s).&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Input file (source: &lt;a href=&quot;https://stackoverflow.com/q/21888945&quot;&gt;biotech&lt;/a&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;     gene            complement(8972..9094)&#xA;                     /locus_tag=&quot;HAPS_0004&quot;&#xA;                     /db_xref=&quot;GeneID:7278619&quot;&#xA;     CDS             complement(8972..9094)&#xA;                     /locus_tag=&quot;HAPS_0004&quot;&#xA;                     /codon_start=1&#xA;                     /transl_table=11&#xA;                     /product=&quot;hypothetical protein&quot;&#xA;                     /protein_id=&quot;YP_002474657.1&quot;&#xA;                     /db_xref=&quot;GI:219870282&quot;&#xA;                     /db_xref=&quot;GeneID:7278619&quot;&#xA;                     /translation=&quot;MYYKALAHFLPTLSTMQNILSKSPLSLDFRLLFLAFIDKR&quot;&#xA;     gene            9632..11416&#xA;                     /gene=&quot;frdA&quot;&#xA;                     /locus_tag=&quot;HAPS_0005&quot;&#xA;                     /db_xref=&quot;GeneID:7278620&quot;&#xA;     CDS             9632..11416&#xA;                     /gene=&quot;frdA&quot;&#xA;                     /locus_tag=&quot;HAPS_0005&quot;&#xA;                     /note=&quot;part of four member fumarate reductase enzyme&#xA;                     complex FrdABCD which catalyzes the reduction of fumarate&#xA;                     to succinate during anaerobic respiration; FrdAB are the&#xA;                     catalytic subcomplex consisting of a flavoprotein subunit&#xA;                     and an iron-sulfur subunit, respectively; FrdCD are the&#xA;                     membrane components which interact with quinone and are&#xA;                     involved in electron transfer; the catalytic subunits are&#xA;                     similar to succinate dehydrogenase SdhAB&quot;&#xA;                     /codon_start=1&#xA;                     /transl_table=11&#xA;                     /product=&quot;fumarate reductase flavoprotein subunit&quot;&#xA;                     /protein_id=&quot;YP_002474658.1&quot;&#xA;                     /db_xref=&quot;GI:219870283&quot;&#xA;                     /db_xref=&quot;GeneID:7278620&quot;&#xA;                     /translation=&quot;MQTVNVDVAIVGAGGGGLRAAIAAAEANPNLKIALISKVYPMRS&#xA;                     HTVAAEGGAAAVAKEEDSYDKHFHDTVAGGDWLCEQDVVEYFVEHSPVEMTQLERWGC&#xA;                     PWSRKADGDVNVRRFGGMKIERTWFAADKTGFHLLHTLFQTSIKYPQIIRFDEHFVVD&#xA;                     ILVDDGQVRGCVAMNMMEGTFVQINANAVVIATGGGCRAYRFNTNGGIVTGDGLSMAY&#xA;                     RHGVPLRDMEFVQYHPTGLPNTGILMTEGCRGEGGILVNKDGYRYLQDYGLGPETPVG&#xA;                     KPENKYMELGPRDKVSQAFWQEWRKGNTLKTAKGVDVVHLDLRHLGEKYLHERLPFIC&#xA;                     ELAQAYEGVDPAKAPIPVRPVVHYTMGGIEVDQHAETCIKGLFAVGECASSGLHGANR&#xA;                     LGSNSLAELVVFGKVAGEMAAKRAVEATARNQAVIDAQAKDVLERVYALARQEGEESW&#xA;                     SQIRNEMGDSMEEGCGIYRTQESMEKTVAKIAELKERYKRIKVKDSSSVFNTDLLYKI&#xA;                     ELGYILDVAQSISSSAVERKESRGAHQRLDYVERDDVNYLKHTLAFYNADGTPTIKYS&#xA;                     DVKITKSQPAKRVYGAEAEAQEAAAKKE&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Desired output:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;     gene            complement(8972..9094)&#xA;                     /locus_tag=&quot;HAPS_0004&quot;&#xA;                     /db_xref=&quot;GeneID:7278619&quot;&#xA;     CDS             complement(8972..9094)&#xA;                     /locus_tag=&quot;HAPS_0004&quot;&#xA;                     /codon_start=1&#xA;                     /transl_table=11&#xA;                     /product=&quot;hypothetical protein&quot;&#xA;                     /protein_id=&quot;YP_002474657.1&quot;&#xA;                     /db_xref=&quot;GI:219870282&quot;&#xA;                     /db_xref=&quot;GeneID:7278619&quot;&#xA;     gene            9632..11416&#xA;                     /gene=&quot;frdA&quot;&#xA;                     /locus_tag=&quot;HAPS_0005&quot;&#xA;                     /db_xref=&quot;GeneID:7278620&quot;&#xA;     CDS             9632..11416&#xA;                     /gene=&quot;frdA&quot;&#xA;                     /locus_tag=&quot;HAPS_0005&quot;&#xA;                     /note=&quot;part of four member fumarate reductase enzyme&#xA;                     complex FrdABCD which catalyzes the reduction of fumarate&#xA;                     to succinate during anaerobic respiration; FrdAB are the&#xA;                     catalytic subcomplex consisting of a flavoprotein subunit&#xA;                     and an iron-sulfur subunit, respectively; FrdCD are the&#xA;                     membrane components which interact with quinone and are&#xA;                     involved in electron transfer; the catalytic subunits are&#xA;                     similar to succinate dehydrogenase SdhAB&quot;&#xA;                     /codon_start=1&#xA;                     /transl_table=11&#xA;                     /product=&quot;fumarate reductase flavoprotein subunit&quot;&#xA;                     /protein_id=&quot;YP_002474658.1&quot;&#xA;                     /db_xref=&quot;GI:219870283&quot;&#xA;                     /db_xref=&quot;GeneID:7278620&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried using &lt;code&gt;pcregrep --multiline .*'START-SEARCH-TERM.*(\n|.)*END-SEARCH-TERM' path/to/SOURCE-FILE&lt;/code&gt;. I installed pcregrep (grep utility that uses Perl-style regexps) in Ubuntu with &lt;code&gt;sudo apt install pcregrep&lt;/code&gt;. I also installed Biopython with &lt;code&gt;sudo apt install python3-biopython&lt;/code&gt; and ran the Simple GenBank parsing example from &lt;em&gt;Biopython Tutorial and Cookbook&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="7340" OwnerDisplayName="karel" LastEditorUserId="7340" LastEditDate="2020-03-06T03:29:57.950" LastActivityDate="2020-06-14T13:58:48.740" Title="Parsing a GenBank file with multiple gene entries" Tags="|python|text-processing|genbank|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11575" PostTypeId="1" CreationDate="2020-03-05T12:53:55.493" Score="-2" ViewCount="53" Body="&lt;p&gt;I have a .csv file like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pair,S17A,S2..APOBEC.,S1..age.,S18.like,S3..BRCA.,S17B&#xA;LP2000104-DNA_A01_vs_LP2000101-DNA_A01,0.448027080410302,0,0,0,0,0.410602648202438&#xA;LP2000325-DNA_A01_vs_LP2000317-DNA_A01,0.343096214261112,0,0.111190317814775,0.171937435363758,0.187395051556141,0.156209735124657&#xA;LP2000327-DNA_A01_vs_LP2000319-DNA_A01,0.436249785689912,0,0.066118729867486,0.077392575088635,0.116384601626772,0.263251836199875&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have a list of samples which are a part of first column for example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;LP6005334-DNA_H01&#xA;LP6008334-DNA_A03&#xA;LP6008334-DNA_A04&#xA;LP6008334-DNA_B02&#xA;LP6008334-DNA_C02&#xA;LP6008334-DNA_D02&#xA;LP6008336-DNA_H01&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to extract .csv for this list of samples&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can not think of intersect function as these samples are a part of sample name not full name&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you help?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2020-03-05T13:03:09.640" Title="Extracting part of a csv by a list" Tags="|r|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11576" PostTypeId="2" ParentId="11575" CreationDate="2020-03-05T13:03:09.640" Score="1" Body="&lt;p&gt;You can use &lt;code&gt;strsplit&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ciao &amp;lt;- &quot;LP2000104-DNA_A01_vs_LP2000101-DNA_A01&quot;&#xA;sampleIneed &amp;lt;- &quot;LP6008334-DNA_A03&quot;   &#xA;sampleIneed %in% strsplit(ciao,&quot;_vs_&quot;)[[1]]&#xA;[1] FALSE&#xA;&#xA;&quot;LP2000104-DNA_A01&quot; %in% strsplit(ciao,&quot;_vs_&quot;)[[1]]&#xA;[1] TRUE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can loop (e.g. &lt;code&gt;sapply&lt;/code&gt;or &lt;code&gt;lapply&lt;/code&gt;) through the element of the first column to extract only the rows you need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also simply use &lt;code&gt;grep&lt;/code&gt; on the first column&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ciao &amp;lt;_ your_csv&#xA;grep(&quot;LP2000104-DNA_A01&quot;, ciao$pair)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3631" LastActivityDate="2020-03-05T13:03:09.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11578" PostTypeId="1" CreationDate="2020-03-05T17:47:10.793" Score="1" ViewCount="48" Body="&lt;p&gt;I want to collect all isoforms of all genes (Fasta/Fastq file of nucleotide and protein sequences) Arabidopsis Col-0. I am wondering if there is a straightforward way to download the file from any database.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am new to sequence analysis, so I apologize if asking a very basic question. &lt;/p&gt;&#xA;" OwnerUserId="7347" LastEditorUserId="3848" LastEditDate="2020-04-26T05:05:07.530" LastActivityDate="2020-04-26T12:02:00.313" Title="Download sequences of isoforms" Tags="|rna-seq|phylogenetics|sequence-analysis|phylogeny|isoform|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11579" PostTypeId="1" AcceptedAnswerId="11580" CreationDate="2020-03-05T20:40:28.887" Score="3" ViewCount="4448" Body="&lt;p&gt;I have several pdb files and each of them corresponds to a domain on a protein. I want to merge these files into one single pdb for the whole protein, and the approach I'm looking for should allow me to order these domains according to the order by which they're located on the protein (for example, merging domain A, B, and C by the order CBA). It would be preferable if I can do the merging with VMD but any suggestion would be appreciated. &lt;/p&gt;&#xA;" OwnerUserId="7197" LastActivityDate="2020-03-05T22:59:16.650" Title="Merge multiple PDB files into a single file" Tags="|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11580" PostTypeId="2" ParentId="11579" CreationDate="2020-03-05T22:59:16.650" Score="1" Body="&lt;p&gt;PyMOL is a rather flexible tool for doing this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The command &lt;code&gt;create combo, domA or domB or domC&lt;/code&gt; in the PyMOL app will create a single object. However, they will need to be the same chain and segment (&lt;code&gt;segi&lt;/code&gt;). Also you will need to fix the numbering to match what you expect them to be.&#xA;To fix this you need to do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; alter domA, chain='A'&#xA; alter domA, resv+=0&#xA; alter domB, chain='A'&#xA; alter domA, resv+=100&#xA; alter domC, chain='A'&#xA; alter domA, resv+=200&#xA; alter all, segi=''&#xA; sort&#xA; create combo, domA or domB or domC&#xA; save combo.pdb, combo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;PyMOL can be installed as a Python 3 module, which works in the same way but with &lt;code&gt;pymol.cmd&lt;/code&gt; and is a lot more replicable. But for starting out the App is fine.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pymol2&#xA;with pymol2.PyMOL() as pymol:&#xA;     pymol.cmd.alter('domA','chain=&quot;A&quot;')&#xA;     etc.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now. This assumed that the coordinates are where we want them to be. Which is highly unlikely and that there are zero overlaps.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If there are overlaps, aligning the overlapping residues (&lt;code&gt;align domB and resi 80-100, domA and resi 80-100&lt;/code&gt;) can be done. Just remember that the first element is the mobile object, the second the target, hence why I wrote it backwards.&lt;/li&gt;&#xA;&lt;li&gt;If do not have overlaps and want the domains docked together you can use the &lt;a href=&quot;https://zhanglab.ccmb.med.umich.edu/DEMO&quot; rel=&quot;nofollow noreferrer&quot;&gt;DEMO tool&lt;/a&gt; by the I-Tasser people.&lt;/li&gt;&#xA;&lt;li&gt;If you want your domains as &quot;balls on a string&quot; aligned on the x-axis for a nice picture and you do not care if it may not be biological, you may want to check out a &lt;a href=&quot;https://github.com/matteoferla/protein_fuser&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github repo&lt;/a&gt; of mine for doing just so (it offsets, aligns if possible and spaces them out, but does not add linkers).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If you want to add linkers, you can use submit your sequence to Phyre or I-Tasser with your missing density model as a reference, or you can add them with rosetta remodel (tricky) or make them up in Coot or PyMOL (bad idea, but if you are in a hurry... here is &lt;a href=&quot;http://blog.matteoferla.com/2017/10/hacking-pdbs-for-fusion-protein.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;a tutorial&lt;/a&gt; of mine).&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-03-05T22:59:16.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11581" PostTypeId="2" ParentId="11571" CreationDate="2020-03-06T02:17:31.220" Score="1" Body="&lt;p&gt;I would simply look at co-variance across the entire dataset and examine values &gt;0.8 in the first instance rather than select 1000 genes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With your approach sampling 1000 genes can be criticised for selective sampling and the way to get around this is to perform replicates, with (bootstraps) or without replacement. You could produce e.g. 1000 replicates of 1000 genes and perform co-variance on each replicate to examine Y = X. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Python has a specific permutation/combinations module call itertools, which is different to random resampling and doesn't need replicates&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from itertools import combinations&#xA;X = [1,2,3]&#xA;Y = [1,2,3]&#xA;print (list(combinations(iterable, 1000) + X + Y))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The standard way is to sample is,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from random import sample &#xA;mygenes = [1, 2, 3, 4, 5]  &#xA;&#xA;print(sample(mygenes,3)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[2, 4, 5]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There is nuance and further funcationality ... I cannot remember is &quot;sample&quot; is with or without replacement (its quite a big deal)&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-03-06T11:00:46.777" LastActivityDate="2020-03-06T11:00:46.777" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11582" PostTypeId="1" CreationDate="2020-03-06T05:04:35.623" Score="0" ViewCount="214" Body="&lt;p&gt;(Ported from stack overflow)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to summarize gene count using htseq-count; and it's returning 0 counts at every gene. I'm not sure what I'm doing wrong; I think it has to do with the gene flag I'm using. &#xA;I've tried using the GTF for Arabidopsis TAIR 10: which I got from: &lt;a href=&quot;ftp://ftp.ensemblgenomes.org/pub/release-46/plants/gtf/arabidopsis_thaliana/Arabidopsis_thaliana.TAIR10.46.gtf.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ensemblgenomes.org/pub/release-46/plants/gtf/arabidopsis_thaliana/Arabidopsis_thaliana.TAIR10.46.gtf.gz&lt;/a&gt; as well as the GFF file available on Araport.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using various BAM files I know to have gene expression values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried both locally and on galaxy&#xA;The local commands I've tried so far are:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python3 -m HTSeq.scripts.count --idattr=gene_id #bam #gtf &amp;gt; summary.txt&#xA;python3 -m HTSeq.scripts.count -i gene_id #bam #gff &amp;gt; summary.txt&#xA;python3 -m HTSeq.scripts.count -i locus_tag #bam #gff &amp;gt; summary.txt&#xA;python3 -m HTSeq.scripts.count -i ID #bam #gff &amp;gt; summary.txt&#xA;python3 -m HTSeq.scripts.count -i SeqID #bam #gff &amp;gt; summary.txt&#xA;python3 -m HTSeq.scripts.count -i Parent #bam #gff &amp;gt; summary.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I've done the same on galaxy; inputing the -i value in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help as to where I'm going wrong would be great.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="7331" LastActivityDate="2020-03-06T10:11:13.900" Title="HTSeq-count is returning 0 for every gene, instead of expression value" Tags="|sam|gene|gene-expression|gff|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11583" PostTypeId="1" CreationDate="2020-03-06T06:25:43.903" Score="0" ViewCount="149" Body="&lt;p&gt;I have tried to incorporate blast in my application using biopython.i could extract the blast results but now i want to extract the structure file corresponding matches. My code is :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.SeqIO import parse &#xA;from Bio.SeqRecord import SeqRecord &#xA;from Bio.Seq import Seq &#xA;&#xA;file = open(&quot;prats.fasta&quot;) &#xA;&#xA;records = parse(file,&quot;fasta&quot;) &#xA;for record in records:   &#xA;    print(&quot;Id: %s&quot; % record.id) &#xA;    print(&quot;Name: %s&quot; % record.name) &#xA;    print(&quot;Description: %s&quot; % record.description) &#xA;    print(&quot;Annotations: %s&quot; % record.annotations) &#xA;    print(&quot;Sequence Data: %s&quot; % record.seq) &#xA;    print(&quot;Sequence Alphabet: %s&quot; % record.seq.alphabet)&#xA;&#xA;from Bio.Blast import NCBIWWW&#xA;sequence_data = open(&quot;prats.fasta&quot;).read() &#xA;sequence_data &#xA;&#xA;result_handle = NCBIWWW.qblast(&quot;blastn&quot;, &quot;nt&quot;, sequence_data) &#xA;result_handle&#xA;&#xA;with open('results.xml', 'w') as save_file:&#xA;    blast_results = result_handle.read()&#xA;    print(blast_results)&#xA;    save_file.write(blast_results)&#xA;&#xA;from Bio.Blast import NCBIXML&#xA;E_VALUE_THRESH = 1e-20 &#xA;for record in NCBIXML.parse(open(&quot;results.xml&quot;)): &#xA;    if record.alignments: &#xA;        print(&quot;\n&quot;) &#xA;        print(&quot;query: %s&quot; % record.query[:100]) &#xA;        for align in record.alignments: &#xA;            for hsp in align.hsps: &#xA;                if hsp.expect &amp;lt; E_VALUE_THRESH: &#xA;                    print(&quot;match: %s &quot; % align.title[:100])&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;from here i want to generate the xml files and then fetch structure of each nucleotides results&lt;/p&gt;&#xA;" OwnerUserId="7353" LastActivityDate="2020-04-05T08:00:28.490" Title="how to generate structure file from blast results of an input sequence" Tags="|python|blast|pdb|nucleotide-models|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11584" PostTypeId="1" AcceptedAnswerId="11588" CreationDate="2020-03-06T06:48:15.150" Score="2" ViewCount="592" Body="&lt;p&gt;I have gene expression data from RNAseq, specifically: log2(x+1) transformed RSEM normalized counts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I convert these data from multiple samples to determine fold change in gene expression?&lt;/p&gt;&#xA;" OwnerUserId="7354" LastActivityDate="2020-03-07T09:55:21.437" Title="How to calculate fold change in gene expression from RSEM (RNAseq)" Tags="|rna-seq|gene-expression|rsem|fold-change|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11585" PostTypeId="2" ParentId="11583" CreationDate="2020-03-06T07:32:51.883" Score="1" Body="&lt;h2&gt;If you the DNA/RNA crystal structure:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Simply change the &lt;code&gt;nt&lt;/code&gt; in &lt;code&gt;NCBIWWW.qblast(&quot;blastn&quot;, &quot;nt&quot;, sequence_data)&lt;/code&gt; to &lt;code&gt;pdbnt&lt;/code&gt; to get the nucleotide sequence in a chain.&#xA;The PDB code will be the accession ID with an underscore plus chain letter.&#xA;Do note that the PDB structure may be offset.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;If you want the crystal structures of the encoded protein:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;blastx&lt;/code&gt; (translation) on the &lt;code&gt;pdbaa&lt;/code&gt; database. As above the PDB code will be as above.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;If you want a model of the encoded protein:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Check Swissmodel for a pre-existing close threaded model, else check out I-Tasser (best), Phyre (fast).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;If you want a model of the DNA/RNA:&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;You are running blast without special settings —its not a short motif—, so these things may be long, so do be weary of any result. There are a few RNA folding prediction tools online (e.g. RNAFold).&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-03-06T07:32:51.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11586" PostTypeId="1" CreationDate="2020-03-06T08:53:17.340" Score="3" ViewCount="72" Body="&lt;p&gt;I have spent some time studying RNA structures. While there is a range of interesting phenomena and functions that certainly deserve understanding from the scientific point of view, I have never encountered a practical use of this knowledge (in terms of disease diagnostics, medications, etc.) Have you? If a practical use of RNA structures is still in the future, what could that be? &lt;/p&gt;&#xA;" OwnerUserId="1813" LastEditorUserId="3848" LastEditDate="2023-09-08T23:00:58.820" LastActivityDate="2023-09-08T23:00:58.820" Title="Practical use of RNA structure" Tags="|phylogenetics|phylogeny|covid-19|rna-splicing|rna-structure|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11587" PostTypeId="1" CreationDate="2020-03-06T09:09:16.367" Score="4" ViewCount="2121" Body="&lt;p&gt;In PDB file we always see insertion codes (ic) in a few proteins. Could anybody please tell me why it has been added in structure and what is the aim of that ??? &lt;/p&gt;&#xA;" OwnerUserId="7355" LastEditorUserId="266" LastEditDate="2020-03-10T14:36:53.037" LastActivityDate="2020-03-10T14:36:53.037" Title="What is the aim of insertion codes in the pdb file format?" Tags="|pdb|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11588" PostTypeId="2" ParentId="11584" CreationDate="2020-03-06T09:13:47.197" Score="3" Body="&lt;p&gt;You should use a proper statistical framework for RNA-seq dfferential analysis (which includes FC calculation). Standard tools for this are (among others) &lt;code&gt;edgeR&lt;/code&gt; or &lt;code&gt;DESeq2&lt;/code&gt;. You could use &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/tximport/inst/doc/tximport.html#rsem&quot; rel=&quot;nofollow noreferrer&quot;&gt;tximport&lt;/a&gt; to import RSEM outputs into R and then use its output for e.g. &lt;code&gt;DESeq2&lt;/code&gt;. The linked manual provides example code for this. Please read the manuals of &lt;code&gt;edgeR&lt;/code&gt; or &lt;code&gt;DESeq2&lt;/code&gt; towards the respective downstream analysis. Both are extensive and compreshensive.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2020-03-07T09:55:21.437" LastActivityDate="2020-03-07T09:55:21.437" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11589" PostTypeId="2" ParentId="11582" CreationDate="2020-03-06T10:11:13.900" Score="1" Body="&lt;p&gt;Maybe the contig names in your BAM files don't match those in the GFF file? Ensembl uses contig names like &lt;code&gt;1&lt;/code&gt;, some other reference sources use &lt;code&gt;chr1&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, for ensembl GFF files, HTSeq's default value for idattr should be fine though.&lt;/p&gt;&#xA;" OwnerUserId="6955" LastActivityDate="2020-03-06T10:11:13.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11590" PostTypeId="2" ParentId="11587" CreationDate="2020-03-06T10:16:06.650" Score="7" Body="&lt;p&gt;When I wondered it myself years ago and I found a good explanation in a mailing list archive. I think it was &lt;a href=&quot;https://lists.sdsc.edu/pipermail/pdb-l/2004-March/001513.html&quot; rel=&quot;noreferrer&quot;&gt;this one&lt;/a&gt;, by Frances C. Bernstein:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The problem with numbering started when people wanted to compare the&#xA;  'same' proteins from different species.  They found that there were&#xA;  the following possibilities that gave rise to differences:&lt;/p&gt;&#xA;  &#xA;  &lt;ol&gt;&#xA;  &lt;li&gt;More or fewer residues at either end.&lt;/li&gt;&#xA;  &lt;li&gt;Extra residues at various places within the chain.&lt;/li&gt;&#xA;  &lt;li&gt;Fewer residues at various places within the chain.&lt;/li&gt;&#xA;  &lt;li&gt;Different amino acids at the same place.&lt;/li&gt;&#xA;  &lt;/ol&gt;&#xA;  &#xA;  &lt;p&gt;Now imagine that residue PHE 195 is very important for the activity of&#xA;  the protein in species A.  But in species B it is residue PHE 197 and&#xA;  in species C it is PHE 212, because species B and C are not the same&#xA;  length as species A.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Because people felt it was important to preserve the amino acid numbering for 'important' residues and to be able to readily discuss&#xA;  and compare the structures from different species, various people&#xA;  decided to try to number the proteins from species B, C, etc. to match&#xA;  the numbering used for species A.  In doing this, one must have gaps&#xA;  (missing numbers) where a sequence is shorter.  But what should one do&#xA;  when a sequence is longer?  This is the case where it is necessary to&#xA;  insert extra numbers and this is done by using insertion codes.&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;Thus the insertion code is an integral part of the residue number and it is improper to ignore that field when using a PDB entry.  You&#xA;  must also allow for 'missing' numbers when using a PDB entry.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;In the PDB format the insertion code is in column 27, following the sequence number (columns 23-26):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                      nnnnI&#xA;ATOM  11918  CZ  PHE D 100      -6.852  76.356 -23.289  1.00107.94           C&#xA;ATOM  11919  N   ARG D 100A     -9.676  74.726 -19.958  1.00105.71           N&#xA;...&#xA;ATOM  11970  CE  MET D 100H     -8.264  83.348 -19.494  1.00107.93           C&#xA;ATOM  11971  N   ASP D 101     -11.329  81.237 -14.804  1.00107.41           N&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Together, columns 23-27 make a sequence ID, which is a mostly-numeric ID of the&#xA;residue position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: It's not a full ID of the residue. When you have a point&#xA;mutation (a.k.a. microheterogeneity) you have 2+ residues with partial occupancy at the same position. To fully identify a residue both the sequence ID and residue name are needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As of 2020, the PDB (organization) uses PDBx/mmCIF as the primary format. The PDB format is only a legacy one. The presentation of the insertion codes (and many other things) in the mmCIF format are rather confusing, so it's worth explaining.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The mmCIF format introduced double identifiers for chains, residues and atoms: author's ID and &quot;label&quot; ID. Both look the same and are easy to confuse. For example: two chains named B and A by the author are renamed as, respectively, A and B by the wwPDB; both names are used in parallel: one chain is B(A), the other A(B). Poor design choice at every step. The same with the sequence ID. Originally, we had two tags:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;_atom_site.auth_seq_id&lt;/code&gt; -- author's sequence ID, possibly with an insertion code (100A),&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;_atom_site.label_seq_id&lt;/code&gt; -- numeric-only sequence ID, numbered with respect to the sequence of this molecule (SEQRES).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Then, as I heard, numeric insertion codes were spotted in the wild. (In a database big enough any mistake that is not automatically checked for will happen). This made a problem with translation from the PDB format to mmCIF. The sequence number 100 with IC 1 would be interpreted as sequence number 1001. The numeric ICs must have been remediated in the meantime, because they absent as of 2020. But at that time (10-20 years ago?), instead of correcting the entries, the decision was made to move the insertion code to a separate column (pdbx_PDB_ins_code).&#xA;The documentation was never updated, so the auth_seq_id is still documented to store the sequence ID, not just the sequence number.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is an example from 3B9F:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                             !!  I                                      nn&#xA;ATOM   770  C CD2 . LEU B 2  46  ? -23.416 21.041  -27.401 1.00 19.94 ? 60   LEU H CD2 1 &#xA;ATOM   771  N N   . TYR B 2  47  A -21.191 16.717  -28.119 1.00 18.73 ? 60   TYR H N   1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The sequence IDs here are 60 and 60A. Since the meaning of the insertion code is not widely known, it landed next to label_seq_id which has nothing to do with it. So again: the insertion code is just a part of the author's sequence ID (originally, in auth_seq_id), and the letter A above (10th field) is simply an extension to the number 60 (17th field).&lt;/p&gt;&#xA;" OwnerUserId="266" LastEditorUserId="266" LastEditDate="2020-03-06T12:30:54.503" LastActivityDate="2020-03-06T12:30:54.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11592" PostTypeId="1" CreationDate="2020-03-06T12:14:41.743" Score="-1" ViewCount="156" Body="&lt;p&gt;I wish to investigate how a virus could be attenuated through targeted mutagenesis by a priori prediction of attenuating amino acid mutations.&lt;/p&gt;&#xA;&lt;p&gt;In particular, I am seeking bioinformatic solutions where the protein sequence of an exisiting live vaccine strais has been compared against it wild-type counterparts to identify key attenuating mutations and the key attenuating mutation(s) can be deduced without wet-lab site directed mutagenesis.&lt;/p&gt;&#xA;&lt;p&gt;I presume surface antigens, such as the spike protein in COVID-10 would be the proteins of interest, but I'm open to suggestions.&lt;/p&gt;&#xA;&lt;p&gt;I am a computer scientist looking to investigate the mutational parameter space of viral protein sequence data. A key approach I am considering is to use a mutational correlation/covariance matrix and then remove convolution artefacts. I hope this will identify compensatory mutations that will maintain a local interaction for example within a fold, providing a starting point for identifying attenuating mutations.&lt;/p&gt;&#xA;&lt;p&gt;This may have application for example to COVID-19 and the goal is to produce an efficacious vaccine.&lt;/p&gt;&#xA;" OwnerUserId="7346" LastEditorUserId="3848" LastEditDate="2020-12-08T20:21:58.290" LastActivityDate="2020-12-08T20:21:58.290" Title="Algorithmic investigation of viral proteins for vaccine design" Tags="|sequence-alignment|phylogenetics|phylogeny|covid-19|sars-cov-2|" AnswerCount="1" CommentCount="1" ClosedDate="2020-03-09T13:58:18.430" ContentLicense="CC BY-SA 4.0" />
  <row Id="11593" PostTypeId="1" CreationDate="2020-03-06T13:54:02.403" Score="1" ViewCount="259" Body="&lt;p&gt;so I'm fairly new to using linux and blast - have encountered an error in my work. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have created a fasta database with the results in this directory path: &#xA;&lt;code&gt;~/class/hw/db&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm currently trying to run a search against my created database against query sequences in a file called &lt;code&gt;seqs.fas&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code is as follows: &#xA;&lt;code&gt;blastn -db $db -query seqs.fas -out q1results.txt&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I keep getting this error: &#xA;&lt;code&gt;Error: Too many positional arguments (1), the offending value: seqs.fas&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Error:  (CArgException::eSynopsis) Too many positional arguments (1), the offending value: seqs.fas&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure what I'm doing wrong here, and would appreciate any advice to fix this, thanks!&lt;/p&gt;&#xA;" OwnerUserId="7295" LastActivityDate="2020-03-06T14:06:48.040" Title="using blastn to search [error]" Tags="|blastn|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11594" PostTypeId="2" ParentId="11593" CreationDate="2020-03-06T14:06:48.040" Score="2" Body="&lt;p&gt;I remember having this problem in the past, it was encoding related. Rewriting the &lt;code&gt;-&lt;/code&gt;s manually (the dash sign) should fix the issue, please see &lt;a href=&quot;http://www.metagenomics.wiki/tools/blast/error-too-many-positional-arguments&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Problem is a copy &amp;amp; past from the BLAST documentation, all dashes&#xA;  ('–') need to be replaced by hand at the command line.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-03-06T14:06:48.040" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11595" PostTypeId="1" CreationDate="2020-03-06T15:39:25.923" Score="1" ViewCount="97" Body="&lt;p&gt;Basecalling on nanopore requires a machine learning neural network model, for example &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-020-3459-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;A popular software from Nanopore for it is &lt;a href=&quot;https://github.com/nanoporetech/taiyaki&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nanoporetech/taiyaki&lt;/a&gt;, which has many people forking it.&lt;/p&gt;&#xA;&lt;p&gt;However, the project requires an existing neural model to get started. I don't seem to be able to find a project for training a brand new model for Nanopore.&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="3848" LastEditDate="2020-11-20T14:09:31.077" LastActivityDate="2020-11-20T14:09:31.077" Title="Train a new basecalling model for nanopore?" Tags="|nanopore|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11596" PostTypeId="2" ParentId="11572" CreationDate="2020-03-06T17:16:38.493" Score="4" Body="&lt;p&gt;There is a &lt;a href=&quot;https://anaconda.org/bioconda/maxquant&quot; rel=&quot;nofollow noreferrer&quot;&gt;conda package&lt;/a&gt; for maxquant that should work on most platforms (I've used it on CentOS and I think it's been used on Ubuntu as well). There's also a docker container version of that that we (Bioconda) have posted to &lt;a href=&quot;https://quay.io/repository/biocontainers/maxquant?tab=tags&quot; rel=&quot;nofollow noreferrer&quot;&gt;the biocontainers project&lt;/a&gt; that should also work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note that large parts of a maxquant analysis are single threaded, regardless of what you specify. We've also had single analysis runs lasting over a month (even when using 90 threads), so the program can be very slow if your search space is large.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-03-06T17:16:38.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11597" PostTypeId="2" ParentId="10552" CreationDate="2020-03-06T23:42:56.633" Score="2" Body="&lt;p&gt;The paf format does not have the CIGAR string and base quality information that are typically present in a BAM/SAM file. While it is in principle possible to write a paf2sam converter, it's going to be very difficult to regenerate the CIGAR string with all the indel information (and maybe it's probably impossible to do so) that you'll need in a BAM/SAM file. If your motivation is to get coverage information, it might make more sense to convert the paf file into a bed file, before using a tool like bedtools to convert it into a coverage track.&lt;/p&gt;&#xA;" OwnerUserId="7349" LastActivityDate="2020-03-06T23:42:56.633" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11598" PostTypeId="2" ParentId="2289" CreationDate="2020-03-07T01:51:45.223" Score="1" Body="&lt;p&gt;This dbsnp github repo seems to offer that information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/ncbi/dbsnp/tree/master/Build%20Announcements/151&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ncbi/dbsnp/tree/master/Build%20Announcements/151&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7349" LastActivityDate="2020-03-07T01:51:45.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11599" PostTypeId="1" AcceptedAnswerId="11600" CreationDate="2020-03-07T05:51:12.923" Score="1" ViewCount="86" Body="&lt;p&gt;I am confused about how to construct the phylogenetic tree of the streptomyces genus. The taxonomy of the streptomyces genus is mostly based on the MLST method, but now I need to use the gene data from NCBI to construct the phylogenetic tree of the streptomyces genus. Should I use the whole gene of the MLST's housekeeping gene to construct the tree? Or I should just use the loci to build the tree? Or the other method? I am a junior in university, have little knowledge of the filed., but my tutor asked me to do that. plus: Sorry for my awkward expression~&lt;/p&gt;&#xA;" OwnerUserId="7365" LastEditorUserId="3848" LastEditDate="2020-03-07T15:55:11.727" LastActivityDate="2020-03-07T19:55:33.930" Title="How to construct the phylogenetic tree of the whole genus?" Tags="|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11600" PostTypeId="2" ParentId="11599" CreationDate="2020-03-07T15:21:45.593" Score="1" Body="&lt;p&gt;The MLST genes are not used to contruct a tree, because each pattern of alleles determines a predetermined genotype, e.g.  Staphlococcus aureus ST 22 is the MLST type concerning the MRSA epidemic. You could build a tree, but its only purpose is to visualise the ST, so its a bit pointless but okay at least you will not mistype something.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Performing a whole genome tree on Strep for the genus is complicated but doable and there are missing genes and stuff and if you just want typing then its not needed.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;So if typing is your goal I would submit your MLST loci to the Steptomyces MLST database,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://pubmlst.org/bigsdb?db=pubmlst_streptomyces_seqdef&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pubmlst.org/bigsdb?db=pubmlst_streptomyces_seqdef&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-03-07T19:55:33.930" LastActivityDate="2020-03-07T19:55:33.930" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11601" PostTypeId="2" ParentId="11592" CreationDate="2020-03-07T15:31:50.127" Score="5" Body="&lt;p&gt;There isn't a vaccine for any coronavirus, and your question is generally about targeted attentuation, which is a complex area. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic building blocks for any vaccine development is virological understanding of the proteins involved in pathogenesis. I will focus on covid-19 as an example here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The majority of bioinformatics work is based around the spike protein as is wetlab vaccine development. This gene is immediately after the 20 kb of ORF1ab, including the RNA dependent RNA polymerase, its around 3.5 kb but it might be a bit bigger. The spike protein is a major B-cell (antibody) antigenic determinant and the cellular receptor sites are located within it. Your bioinformatics could ultimately focus on targeting these specific sites, in particular using antigenic (antibody) prediction algorithms. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;First you need some basic biology of spike and hydrophobicity prediction is a good starting point such as TMHMM &lt;a href=&quot;http://www.cbs.dtu.dk/services/TMHMM/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and consider the SARS spike protein structure for example &lt;a href=&quot;http://www.rcsb.org/structure/5WRG&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Once you've a handle on those structures antigenicity scanning would be a next phase analysis and possibly homology modelling. This is because COVID-19 and SARS are closely related, but there are key differences. You finally consider the full range of amino acid mutations in current epidemic via NCBI.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;At this point you would target a specific vaccine design strategy and have a clearer understanding on the approach you wish to take. Good luck!&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-03-11T08:42:14.430" LastActivityDate="2020-03-11T08:42:14.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11602" PostTypeId="1" CreationDate="2020-03-07T20:51:12.920" Score="8" ViewCount="144" Body="&lt;p&gt;Is there anything known about the RNA structures of coronaviruses? More specifically - do they have any interesting known structures in the translatable region, like RRE of HIV or the double loops in flaviviruses?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;br&gt;&#xA;&lt;a href=&quot;https://syncedreview.com/2020/02/04/baidu-open-sources-rna-prediction-algorithm-for-2019-novel-coronavirus/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is a recent development&lt;/a&gt; on the side of structure prediction.&lt;/p&gt;&#xA;" OwnerUserId="1813" LastEditorUserId="3848" LastEditDate="2020-11-30T20:27:09.823" LastActivityDate="2020-11-30T20:27:09.823" Title="Coronavirus RNA structures?" Tags="|rna|covid-19|rna-splicing|sars-cov-2|rna-structure|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11603" PostTypeId="2" ParentId="11586" CreationDate="2020-03-07T23:23:47.000" Score="3" Body="&lt;p&gt;Please look up flavivirus 'double loops' as you described them previously (post for &quot;Coronavirus RNA') and associated RNA secondary structure anomalies for dengue virus and associated vaccine (Butantan) and the yellow fever virus and its vaccine (17D). If you are aware of &quot;double loops&quot;, we must be aware of the association of RNA secondardy structure and attenuated strains, i.e. live vaccines. The point is that modification of the RNA secondary structure is a key strategy in flavivirus attenuation. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-03-08T03:26:11.380" LastActivityDate="2020-03-08T03:26:11.380" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11604" PostTypeId="1" AcceptedAnswerId="11605" CreationDate="2020-03-08T12:15:17.580" Score="1" ViewCount="42" Body="&lt;p&gt;I've calculated the ANI of the streptomyces genus, but I'm confused about how to separate them into a different group, could anybody kindly tell me how to do that? Thanks!!!&lt;/p&gt;&#xA;" OwnerUserId="7365" LastEditorUserId="3848" LastEditDate="2020-03-08T13:19:19.987" LastActivityDate="2020-03-08T13:27:44.743" Title="How to seperate the genus into different group based on th ANI calculation?" Tags="|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11605" PostTypeId="2" ParentId="11604" CreationDate="2020-03-08T13:27:44.743" Score="1" Body="&lt;p&gt;ANI = average nucleotide identity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will give you a 2x2 matrix. You reconstruct a phylogeny using clustering, the most important being neighbor-joining, UPGMA is very occassionally used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The statistic is clustering algorithm and its done by pairwise reconstruction of sister groups. Once a sister-group is defined (usually on nearest-neighor both in neighbor-joining and UPGMA) the genetic divergence from the hypothetical node is established either using the mean (UPGMA) or additive criteria (neighbor-joining). Okay enough theory ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You get Mega X and find out the import format for a 2x2 distance matric. Mega X however will calculate this for you. You then go to the build tree menu heading and click &quot;neighbor-joining&quot; (or &quot;distance&quot;). The tree will very quickly automatically appear.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-03-08T13:27:44.743" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="11606" PostTypeId="1" CreationDate="2020-03-08T13:51:22.240" Score="1" ViewCount="300" Body="&lt;p&gt;I have DNA sample from 5 pools, having 25 fastq files each. I am running cutadapt to remove the primers using this command &#xA;&lt;code&gt;cutadapt -g ACTTAAGTGTATGTAAACTTCCGACTTCAACTG A7_S7_R1_001.fastq -o sb_A7_S7_R1_001.fastq --discard-untrimmed&lt;/code&gt;&#xA;However, this is too time consuming.  I tried my hand at writing shell script&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for file in /dir/*&#xA;do&#xA;  cmd cutadapt -g ACTTAAGTGTATGTAAACTTCCGACTTCAACTG &quot;$file&quot; &amp;gt;&amp;gt; results.out&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But it is not running . &#xA;Can someone help.&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="7236" LastActivityDate="2020-03-12T14:20:38.600" Title="processing multiple fastq files with cutadapt" Tags="|fastq|shell|cutadapt|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11607" PostTypeId="1" AcceptedAnswerId="11608" CreationDate="2020-03-09T01:56:11.057" Score="1" ViewCount="519" Body="&lt;p&gt;What does the yellow mean in this image of a SARS (coronavirus) protease from the Virus Pathogen resource database? It's from &lt;a href=&quot;https://www.viprbrc.org/brc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.viprbrc.org/brc/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nE84E.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nE84E.jpg&quot; alt=&quot;It's from https://www.viprbrc.org/brc/&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7371" LastEditorUserId="3848" LastEditDate="2022-03-06T04:20:24.363" LastActivityDate="2022-03-06T04:20:24.363" Title="What does the yellow mean in this image from Virus Pathogen Resource?" Tags="|proteins|protein-structure|covid-19|sars-cov-2|molecular-genetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11608" PostTypeId="2" ParentId="11607" CreationDate="2020-03-09T08:07:59.933" Score="7" Body="&lt;p&gt;Sulfur atoms are shown in yellow.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The molecular viewer that you use is JSMol (JMol ported to the web).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Atoms are colored by element: grey C, blue N, red O and yellow S.&lt;br&gt;&#xA;If you wonder how other atoms would be colored, see JMol's &lt;a href=&quot;http://jmol.sourceforge.net/jscolors/&quot; rel=&quot;noreferrer&quot;&gt;Default element colors, by periodic table&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2020-03-09T08:07:59.933" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11609" PostTypeId="1" CreationDate="2020-03-09T09:28:08.353" Score="1" ViewCount="188" Body="&lt;p&gt;I ran the orthomcl program with genomes of the whole streptomyces genus(from ncbi) as input, but I only got 22 orthologous groups(streptomyces's genome size vary from 2M to 15M, which shows the great difference among them). And the analyst from the bioinformatics company told me that my result wasn't reasonable if I try to construct the phylogenetic tree with the 22 orthologous groups I got. He also told me, I'd better use almost 1000 orthologous groups to build the tree. My question is, how to know my orthologous groups are enough to construct a tree?&lt;/p&gt;&#xA;&lt;p&gt;The program was run during my summer vacation(a long time ago...), but I still can't realize why &amp;quot;1000 orthologous groups are needed&amp;quot;. And the analyst form company just tell me that it was the experience number. But when I read some article about comparative genomics, I see some papers are used just about 10 single-copy genes to build the tree, why???&lt;/p&gt;&#xA;" OwnerUserId="7365" LastEditorUserId="73" LastEditDate="2021-08-11T01:48:38.340" LastActivityDate="2022-05-09T20:03:15.310" Title="How many single copy gene are needed to construct the phylogenetic tree?" Tags="|phylogenetics|phylogeny|bacteria|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11610" PostTypeId="2" ParentId="11536" CreationDate="2020-03-09T10:17:20.820" Score="1" Body="&lt;p&gt;For whole-genome align:&#xA;Maybe you could use mummer? Mummer is good to align the whole genome of species, and it also has so many other tools to modify the result. (It can show the SNP.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For short sequence:&#xA;You could try mega X&lt;/p&gt;&#xA;" OwnerUserId="7365" LastActivityDate="2020-03-09T10:17:20.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11611" PostTypeId="1" CreationDate="2020-03-09T10:24:03.570" Score="4" ViewCount="671" Body="&lt;p&gt;Let's say we have a metagenomic sample that is paired-end FASTQ files including 10,000,000 DNA reads collected using shotgun sequencing. &#xA;How would one make a random subsample of the mentioned metagenomic sample with for example 1,000,000 reads? I know about the &lt;code&gt;seqtk&lt;/code&gt; function that could be used in such cases, but my question is more about whether it is meaningful to represent a metagenomic sample with a small subsample generated using this way or it has to satisfy certain criteria, e.g. in terms of the number of reads and diversity to be a valid representative of the metagenomic sample? &#xA;Knowing that some species might be very rare in the sample, would it ever be proper to do subsampling on metagenomic samples? &lt;/p&gt;&#xA;" OwnerUserId="6593" LastActivityDate="2020-03-13T04:49:07.583" Title="What is a proper way for random subsampling of metagenomic data?" Tags="|fastq|taxonomy|microbiology|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11612" PostTypeId="1" CreationDate="2020-03-09T11:42:36.860" Score="-2" ViewCount="217" Body="&lt;p&gt;Does the protein structure shown in the JSmol link have any Sulfur Dioxide molecules?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QTds1.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QTds1.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the picture, the Sulfur element is yellow and Oxygen is red (see &lt;a href=&quot;http://jmol.sourceforge.net/jscolors/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.viprbrc.org/brc/structureOperation.spg?accession=1P9S&amp;amp;decorator=corona&amp;amp;context=1583712525175&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.viprbrc.org/brc/structureOperation.spg?accession=1P9S&amp;amp;decorator=corona&amp;amp;context=1583712525175&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Link to mmcif file (I have the VS Code plugin installed):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://files.rcsb.org/view/1P9S.cif&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://files.rcsb.org/view/1P9S.cif&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7371" LastEditorUserId="73" LastEditDate="2021-06-07T02:36:13.487" LastActivityDate="2021-06-07T02:36:13.487" Title="Are there any Sulfur Dioxide molecules in this structure?" Tags="|protein-structure|molecular-genetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11613" PostTypeId="1" CreationDate="2020-03-09T15:40:10.977" Score="2" ViewCount="71" Body="&lt;p&gt;I have a 120-concatenated protein sequence in a fasta file. I would like to split by proteins.&#xA;How can I identify each protein?&lt;/p&gt;&#xA;" OwnerUserId="7405" LastActivityDate="2020-03-13T04:33:44.560" Title="Identifying proteins in a concatenated protein sequences" Tags="|sequence-alignment|fasta|proteins|" AnswerCount="1" CommentCount="6" ClosedDate="2020-03-15T16:50:18.053" ContentLicense="CC BY-SA 4.0" />
  <row Id="11614" PostTypeId="2" ParentId="11277" CreationDate="2020-03-09T17:15:12.610" Score="2" Body="&lt;p&gt;You have category-based variables as well as variables based on continuous data. The best-option could be linear Mixed- effects model for your research purpose. Chi-square test and t-test may be used to cross-validate or re-examine the outcome of LME  model.&lt;/p&gt;&#xA;" OwnerUserId="7374" LastActivityDate="2020-03-09T17:15:12.610" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11615" PostTypeId="2" ParentId="11602" CreationDate="2020-03-10T09:49:00.783" Score="3" Body="&lt;p&gt;To add a more complete answer: the current coronavirus is closely related to the SARS virus that caused the outbreak in 2004, and on which much research has been done.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.mdpi.com/1999-4915/11/1/59/htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is a general review&lt;/a&gt; of the coronavirus epidemiology, life cycle etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I haven't found yet any materials about the RNA structures in the translatable region, however the structures in the 3' and 5' terminal regions are well studied and are known to be crucial for initiating viral replication and for its packaging.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/29223446&quot; rel=&quot;nofollow noreferrer&quot;&gt;This recent article&lt;/a&gt; addresses in details the RNA structures in 5' terminal of human coronaviruses, whereas &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/25736566&quot; rel=&quot;nofollow noreferrer&quot;&gt;this older article&lt;/a&gt; describes the structures both in 3' and 5' terminals.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, the interesting algorithmic developments for the link that I added to the question are described &lt;a href=&quot;https://syncedreview.com/2020/02/04/baidu-open-sources-rna-prediction-algorithm-for-2019-novel-coronavirus/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1813" LastActivityDate="2020-03-10T09:49:00.783" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11616" PostTypeId="2" ParentId="11612" CreationDate="2020-03-10T12:17:51.303" Score="2" Body="&lt;p&gt;The answer, as provided in the comments by gilleain is No. The commenter wanted to know why I asked this. The reason is I heard sulfur dioxide as silent speech when thinking about antibodies that’s why I asked.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I searched the file &lt;a href=&quot;https://files.rcsb.org/view/1P9S.cif&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://files.rcsb.org/view/1P9S.cif&lt;/a&gt; for the SO2 molecule and didn't find it. I got confirmation in the comments of the question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would anyone be interested in an app that takes a .cif file of a molecule and allows you to search the file for any type of element or molecule in it? Would that help anyone with their research?&lt;/p&gt;&#xA;" OwnerUserId="7371" LastActivityDate="2020-03-10T12:17:51.303" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11617" PostTypeId="1" AcceptedAnswerId="11618" CreationDate="2020-03-10T16:07:17.827" Score="1" ViewCount="882" Body="&lt;p&gt;I have following 5 narrow peak files after peak calling.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;K14_peaks.narrowPeak&#xA;K15_peaks.narrowPeak&#xA;K16_peaks.narrowPeak&#xA;K3_peaks.narrowPeak&#xA;K8_peaks.narrowPeak&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I need to get the intersection of peaks into a new file (The peaks that represent all these files). I used following code but I got higher number of peaks not a lower number.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedtools intersect -a K14_peaks.narrowPeak -b K15_peaks.narrowPeak K16_peaks.narrowPeak K3_peaks.narrowPeak K8_peaks.narrowPeak -wo &amp;gt; intersected.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can someone help me?&#xA;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="856" LastActivityDate="2020-03-10T16:25:39.120" Title="How to get the intersection of peaks after peak calling using MACS2?" Tags="|ngs|bedtools|chip-seq|peak-calling|macs2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11618" PostTypeId="2" ParentId="11617" CreationDate="2020-03-10T16:25:39.120" Score="1" Body="&lt;p&gt;&lt;code&gt;bedtools intersect&lt;/code&gt; with multiple files after &lt;code&gt;-b&lt;/code&gt; performs pairwise intersections between &lt;code&gt;-a&lt;/code&gt; and each file in &lt;code&gt;-b&lt;/code&gt;.&#xA;It does not intersect all files simultaneously.&#xA;See here, from the &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/intersect.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bedtools documentation&lt;/a&gt;. The green section only contains intervals from &lt;code&gt;-a&lt;/code&gt; that also intersect at least one interval in any &lt;code&gt;-b&lt;/code&gt; file.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/5Z8CE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5Z8CE.png&quot; alt=&quot;Bedtools intersect&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I think you're looking for is &lt;code&gt;bedtools multiinter&lt;/code&gt; (documentation &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/multiinter.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is unfortunately blank).&#xA;&lt;code&gt;bedtools multiinter -i input [input...]&lt;/code&gt; will produce a BED file that covers every position listed in your input files, followed by how many and which of your input samples overlap that given position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can use this BED file and create something like an &lt;a href=&quot;https://cran.r-project.org/web/packages/UpSetR/vignettes/basic.usage.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;UpSet plot&lt;/a&gt; to visualize how many bases/peaks are shared between any combination of your samples.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2020-03-10T16:25:39.120" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11619" PostTypeId="1" CreationDate="2020-03-10T18:04:02.537" Score="2" ViewCount="38" Body="&lt;pre&gt;&lt;code&gt;#Series Matrix File(s) &#xA;&amp;gt;data_matrix&amp;lt;- getGEO(filename=&quot;GSE115167_series_matrix.txt.gz&quot;)&#xA;&amp;gt;dim(exprs(data_matrix))&#xA;[1] 49395    12&#xA;&#xA;#SOFT formatted family file(s)&#xA;&amp;gt;data_soft&amp;lt;-getGEO(filename=&quot;GSE115167_family.soft.gz&quot;)&#xA;&amp;gt;probesets &amp;lt;- Table(GPLList(data_soft)[[1]])&lt;span class=&quot;math-container&quot;&gt;$ID&#xA;&amp;gt;data_soft1&amp;lt;- do.call(&quot;cbind&quot;, lapply(GSMList(data_soft), function(x) {&#xA; tab &amp;lt;- Table(x)&#xA; mymatch &amp;lt;- match(probesets, tab$ID_REF)&#xA; return(tab$&lt;/span&gt;VALUE[mymatch])}))&#xA;&amp;gt;data_soft1 &amp;lt;- apply(data_soft1, 2, function(x) {&#xA; as.numeric(as.character(x))})&#xA;&amp;gt;rownames(data_soft1) &amp;lt;- probesets&#xA;&amp;gt;colnames(data_soft1) &amp;lt;- names(GSMList(data_soft))&#xA;&amp;gt;dim(data_soft1)&#xA;[1] 49395    12&#xA;&#xA;#Raw data (GSE115167_RAW) .CEL file&#xA;&amp;gt;data_raw &amp;lt;- ReadAffy(widget=TRUE)&#xA;&amp;gt;data_raw&amp;lt;-rma(data_raw)&#xA;&amp;gt;dim(exprs(data_raw))&#xA;[1] 49495    12&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There are 100 extra probe ID in raw data.Those 100 extra probe ID are not found in the&lt;br&gt;&#xA;GPL15207 platform. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; &amp;gt;tail(exprs(data_matrix), 2)&#xA; #shows probe ID&#xA; AFFX-TrpnX-5_at&#xA; AFFX-TrpnX-M_at&#xA;&#xA;&amp;gt;tail(exprs(data_raw),2)&#xA;#shows probe ID&#xA;ERCC-00172-01_at&#xA;ERCC-00176-01_at&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;ERCC-00172-01_at, ERCC-00176-01_at&lt;/strong&gt; probe ID are not found in GPL15207 platform. I found there are 100 of them in raw data. Can anyone tell what are those probe IDs?&lt;/p&gt;&#xA;" OwnerDisplayName="Paul" LastActivityDate="2020-03-10T18:28:30.223" Title="Why there is extra probeID in microarry Raw data (CEL) as compare to data in Series Matrix File &amp; SOFT formatted family file?" Tags="|microarray|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11620" PostTypeId="2" ParentId="11606" CreationDate="2020-03-10T21:33:46.890" Score="2" Body="&lt;p&gt;Try this one&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for file in /dir/*&#xA;do&#xA;  cutadapt -g ACTTAAGTGTATGTAAACTTCCGACTTCAACTG &quot;$file&quot; &amp;gt;&amp;gt; results.out&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also you have omitted &lt;code&gt;--discard-untrimmed&lt;/code&gt; from the original command. If you want that as well along with everything to run in parallel:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for file in /dir/*&#xA;do&#xA;  cutadapt -g ACTTAAGTGTATGTAAACTTCCGACTTCAACTG --discard-untrimmed &quot;&lt;span class=&quot;math-container&quot;&gt;$file&quot; -o &quot;`basename -s .fastq $&lt;/span&gt;file`_out.fastq&quot; &amp;amp;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="175" LastEditorUserId="175" LastEditDate="2020-03-12T14:20:38.600" LastActivityDate="2020-03-12T14:20:38.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11621" PostTypeId="2" ParentId="10720" CreationDate="2020-03-11T06:52:38.757" Score="1" Body="&lt;p&gt;It looks like you have linkage as opposed to linkage disequilibrium, therefore horizontal pleiotropy is an explanation of the calculation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The calculation appears to be a GLMM around the beta distribution, the residual looks low (which is good). The heterogeneity statistic is significant, you would need to assess whether this is an index of heterozygosity, but it will relate to expectation of Mendellian inhertiance based around the Hardy Weinberg equilibrium.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-03-11T06:52:38.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11622" PostTypeId="2" ParentId="2507" CreationDate="2020-03-11T08:18:24.903" Score="1" Body="&lt;p&gt;Hedge's g measures the difference between two groups here experimental and  control in terms of combined (weighted) standard deviation. A higher g implies greater difference and lower g means lesser difference between  two groups.&#xA;It depends  on the study how the scoring of questions is proceeded with : the effect-size will be same. For example, answer to question item may be scored 1 to 10 depending on whether question item is written in positive  sense (the apple is good for health) or negative sense (the apple is not good for health). Positive Hedge's g does not indicate larger effect for the treatment group. THE negative Hedges' g for the other two studies indicates positive effect for the treatment group if the difference has been computed by  deducting experimental group mean from the control group mean (which is less than exp. Group Mean). Overall, I find that a knowledge of questionnaire scaling and scoring of scales and question items could be very helpful in general and in doing a meta-analysis effectively.&lt;/p&gt;&#xA;" OwnerUserId="7374" LastActivityDate="2020-03-11T08:18:24.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11623" PostTypeId="1" AcceptedAnswerId="11626" CreationDate="2020-03-11T10:34:50.960" Score="3" ViewCount="278" Body="&lt;p&gt;I have Pol_II data for Ctrl vs Knockout condition with two replicates each. I want to calculate the pausing index for the Ctrl vs KO condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here I have given tag directory as input to Homer.Now what I see is as such,for each biological replicate it calculates the pausing ratio.How can I calculate as a whole like 2 Ctrl vs 2 KO replicate? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm giving a sample of the output.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help or suggestion would be really appreciated. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dput(head(CTRL_PTPEM_Pol_II_pausing))&#xA;structure(list(`Transcript/RepeatID (cmd=analyzeRepeats.pl rna hg38 -strand both -count pausing -condenseGenes -d Control_Pol_II_new_set/ Control_Pol_II_set2/ PTPN6_KO_Pol_II_PTPEN/ PTPN6_KO_Pol_II_set2/ -rpkm)` = c(&quot;NM_173803&quot;, &#xA;&quot;NM_001301039&quot;, &quot;NR_132775&quot;, &quot;NM_022114&quot;, &quot;NM_001011666&quot;, &quot;NR_160728&quot;&#xA;), chr = c(&quot;chr16&quot;, &quot;chr1&quot;, &quot;chr3&quot;, &quot;chr1&quot;, &quot;chr7&quot;, &quot;chr19&quot;), &#xA;    start = c(15395704, 150272234, 13618331, 3069153, 28686083, &#xA;    22527485), end = c(15400754, 150277284, 13623381, 3074203, &#xA;    28691133, 22532535), strand = c(&quot;+&quot;, &quot;+&quot;, &quot;+&quot;, &quot;+&quot;, &quot;+&quot;, &#xA;    &quot;-&quot;), Length = c(250, 250, 250, 250, 250, 250), Copies = c(2, &#xA;    1, 1, 1, 1, 1), `Annotation/Divergence` = c(&quot;MPV17L|M-LPH|MLPH1|MLPH2|MPV17L1|-|16p13.11|protein-coding&quot;, &#xA;    &quot;C1orf54|-|-|1q21.2|protein-coding&quot;, &quot;SNORA93|-|-|3p25.1|snoRNA&quot;, &#xA;    &quot;PRDM16|CMD1LL|KMT8F|LVNC8|MEL1|PFM13|-|1p36.32|protein-coding&quot;, &#xA;    &quot;CREB5|CRE-BPA|CREB-5|CREBPA|-|7p15.1|protein-coding&quot;, &quot;LOC105376917|-|-|19p12|ncRNA&quot;&#xA;    ), `Control_Pol_II_new_set/ Pausing Ratio` = c(0.982, 1.832, &#xA;    1.114, 1.247, 3.476, 0.632), `Control_Pol_II_new_set/ Promoter Reads (-50,200)` = c(0.125, &#xA;    0.748, 0.249, 0.873, 0.499, 0), `Control_Pol_II_new_set/ GeneBody Reads (200,5000)` = c(0.13, &#xA;    0.338, 0.208, 0.669, 0.032, 0.091), `Control_Pol_II_set2/ Pausing Ratio` = c(0.686, &#xA;    1.131, 1.054, 0.816, 2.205, 1.005), `Control_Pol_II_set2/ Promoter Reads (-50,200)` = c(0.1, &#xA;    0.7, 0.1, 0.3, 0.3, 0.1), `Control_Pol_II_set2/ GeneBody Reads (200,5000)` = c(0.203, &#xA;    0.604, 0.089, 0.396, 0.068, 0.099), `PTPN6_KO_Pol_II_PTPEN/ Pausing Ratio` = c(1.08, &#xA;    1.362, 1.108, 1.224, 1.943, 1.2), `PTPN6_KO_Pol_II_PTPEN/ Promoter Reads (-50,200)` = c(0.151, &#xA;    0.602, 0.151, 0.301, 0.452, 0.151), `PTPN6_KO_Pol_II_PTPEN/ GeneBody Reads (200,5000)` = c(0.125, &#xA;    0.392, 0.118, 0.212, 0.141, 0.094), `PTPN6_KO_Pol_II_set2/ Pausing Ratio` = c(1.652, &#xA;    0.909, 1.551, 1.395, 1.774, 0.584), `PTPN6_KO_Pol_II_set2/ Promoter Reads (-50,200)` = c(0.429, &#xA;    0.536, 0.429, 0.536, 0.321, 0.107), `PTPN6_KO_Pol_II_set2/ GeneBody Reads (200,5000)` = c(0.206, &#xA;    0.603, 0.229, 0.346, 0.123, 0.279)), class = c(&quot;tbl_df&quot;, &#xA;&quot;tbl&quot;, &quot;data.frame&quot;), row.names = c(NA, -6L))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="285" LastActivityDate="2020-03-11T16:06:35.393" Title="Pausing index calculation" Tags="|homer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11624" PostTypeId="1" CreationDate="2020-03-11T10:58:15.093" Score="3" ViewCount="103" Body="&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/s41564-020-0695-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;This new article&lt;/a&gt; presents the phylogeny of the coronaviruses and the placement of the new coronavirus in it. This made me wonder of what we know about the coronavirus evolution, particularly where it comes to evading human immune defenses: it is not mutating as fast as HIV - indeed, the genomes responsible for outbreaks in various countries are &lt;a href=&quot;https://nextstrain.org/ncov&quot; rel=&quot;nofollow noreferrer&quot;&gt;almost identical&lt;/a&gt;. In the same time it is unlikely to undergo gene reshuffling typical of influenza, since coronaviruses have single RNA genome.&lt;/p&gt;&#xA;" OwnerUserId="1813" LastEditorUserId="3848" LastEditDate="2020-12-04T15:34:28.737" LastActivityDate="2020-12-04T15:34:28.737" Title="Coronavirus phylogeny and evolution" Tags="|phylogenetics|genome|phylogeny|covid-19|sars-cov-2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11625" PostTypeId="1" CreationDate="2020-03-11T13:51:40.643" Score="1" ViewCount="35" Body="&lt;p&gt;I have been asked by my supervisor to conduct an RNAseq without experimentally depleting the samples. However I am quite unsure regarding how much sequencing depth per sample I would require in order to map a decent percentage of the ORFs if I were to proceed this way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Has anyone ever done RNAseq w/o depletion and with what results? Would you go on with it or should I make the case for depleting even though it's gonna be more expensive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="7384" LastEditorUserId="77" LastEditDate="2020-03-11T21:15:55.890" LastActivityDate="2020-03-11T21:15:55.890" Title="RNAseq without depletion?" Tags="|rna-seq|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11626" PostTypeId="2" ParentId="11623" CreationDate="2020-03-11T14:40:03.070" Score="1" Body="&lt;p&gt;but I would say the pausing index is something that is invented, commonly used, but with no clear indication of what is the expected distribution or how it can be subjected to test like you've mentioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume you would like to get your work published, so going by the papers with IF &gt; 5 such as &lt;a href=&quot;http://genesdev.cshlp.org/content/genesdev/32/1/26/F6.expansion.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;, &lt;a href=&quot;http://genesdev.cshlp.org/content/25/7/742.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;and this&lt;/a&gt;, &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S1097276515000957&quot; rel=&quot;nofollow noreferrer&quot;&gt;and this&lt;/a&gt;, the clear strategy is use a boxplot to show the difference between conditions, and apply a statistical test such as mann whitney u or t.test to say, there is a difference. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let's try this, sorry not very good in R,using your dataframe which i named x:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(dplyr)&#xA;library(tidyr)&#xA;library(ggplot2)&#xA;library(ggpubr)&#xA;&#xA;df = x %&amp;gt;% select(contains(&quot;pausing&quot;) | contains(&quot;Transcript&quot;)) %&amp;gt;% &#xA;rename_at(vars(1),substr,1,10) %&amp;gt;% &#xA;pivot_longer(-Transcript,names_to = &quot;condition&quot;) %&amp;gt;% &#xA;mutate(condition=case_when(&#xA;grepl(&quot;Control&quot;,condition) ~ &quot;Control&quot;,&#xA;grepl(&quot;PTPN6&quot;,condition) ~ &quot;PTPN6_KO&quot;&#xA;)) %&amp;gt;%&#xA;group_by(Transcript,condition) %&amp;gt;% summarize_all(mean) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the data frame above, I have averaged the pausing index between replicates (not very good but then again, most papers have only 1 replicate), and group them by condition. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# A tibble: 12 x 3&#xA;# Groups:   Transcript [6]&#xA;   Transcript   condition value&#xA;   &amp;lt;chr&amp;gt;        &amp;lt;chr&amp;gt;     &amp;lt;dbl&amp;gt;&#xA; 1 NM_001011666 Control   2.84 &#xA; 2 NM_001011666 PTPN6_KO  1.86 &#xA; 3 NM_001301039 Control   1.48 &#xA; 4 NM_001301039 PTPN6_KO  1.14 &#xA; 5 NM_022114    Control   1.03 &#xA; 6 NM_022114    PTPN6_KO  1.31 &#xA; 7 NM_173803    Control   0.834&#xA; 8 NM_173803    PTPN6_KO  1.37 &#xA; 9 NR_132775    Control   1.08 &#xA;10 NR_132775    PTPN6_KO  1.33 &#xA;11 NR_160728    Control   0.818&#xA;12 NR_160728    PTPN6_KO  0.892&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then we plot:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p &amp;lt;- ggboxplot(df, x = &quot;condition&quot;, y = &quot;value&quot;,&#xA;                color = &quot;condition&quot;, palette =c(&quot;#c72e29&quot;,&quot;#016392&quot;,&quot;#be9c2e&quot;,&quot;#098154&quot;),&#xA;                add = &quot;jitter&quot;)&#xA;p+stat_compare_means(comparisons=list( c(&quot;Control&quot;, &quot;PTPN6_KO&quot;)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fZ6MB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fZ6MB.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5077" LastEditorUserId="5077" LastEditDate="2020-03-11T16:06:35.393" LastActivityDate="2020-03-11T16:06:35.393" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="11627" PostTypeId="1" AcceptedAnswerId="11671" CreationDate="2020-03-11T15:14:18.823" Score="1" ViewCount="65" Body="&lt;p&gt;I have been trying to use a pipeline for predicting gene expression for a target gene (Although you can do it for many) - the name of this method is epiXcan. Here is a link to the paper: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/s41467-019-11874-7#Sec22&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41467-019-11874-7#Sec22&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the very first steps in the pipeline is obtaining SNP 'annotations' from an epigenome reference consortium known as the REMC. In the methods paper they say: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;We first prepare eQTL statistics (computed with MatrixEQTL4) and SNP annotations (extracted from REMC &lt;a href=&quot;https://egg2.wustl.edu/roadmap/web_portal/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://egg2.wustl.edu/roadmap/web_portal/&lt;/a&gt;). For each eQTL tissue, we use the matched REMC tissue to&#xA;extract the corresponding annotations&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have completed the first step of obtaining eQTL statistics. However obtaining SNP annotations is where I am lost. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my case, I am only trying to predict the expression of one gene (DRD4) and so my region of interest is the pre-frontal cortex. If you visit the website they provide though, it doesn't seem as though annotating a large list of SNPs - at least in the way they are presenting - is possible. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone provide some insight or direction on how to proceed? How does one obtain SNP annotations from the REMC? I have tried emailing the authors of the paper and they have not been responsive.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you,&lt;/p&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="4001" LastEditDate="2020-03-17T15:22:00.627" LastActivityDate="2020-12-12T17:01:15.070" Title="REMC SNP annotations for transcriptome prediction - epiXcan" Tags="|sequence-annotation|snp|transcriptome|imputation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11628" PostTypeId="1" AcceptedAnswerId="11629" CreationDate="2020-03-11T16:41:45.780" Score="2" ViewCount="67" Body="&lt;p&gt;I am using a few clustering algorithms as well as my own tool for protein sequences. Also, I have benchmarked clustering results (i.e. ground truth) to compare with the results. Is there any tool/software/script to generate evaluation metrics for the clustering results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The clustering results/benchmarked are in the following format&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;1  Name1&#xA;1  Name2&#xA;2  Name3&#xA;2  Name4&#xA;2  Name5&#xA;.....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;UPDATE: I am evaluating the clustering performance using &lt;a href=&quot;https://scikit-learn.org/stable/modules/clustering.html#clustering-performance-evaluation&quot; rel=&quot;nofollow noreferrer&quot;&gt;scikit&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2594" LastEditorUserId="2594" LastEditDate="2020-03-12T20:37:00.083" LastActivityDate="2020-03-12T20:37:00.083" Title="Tools or softwares for clustering evaluation" Tags="|sequence-analysis|clustering|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11629" PostTypeId="2" ParentId="11628" CreationDate="2020-03-11T18:22:32.003" Score="1" Body="&lt;p&gt;The definitive answer is no, that is why they are not so popular. There are metrics that can be used:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;you can examine &quot;minimum distance&quot; as a criterion under the assumption that the lowest distance between taxa represents the largest homology (e.g. if it is sequence data), which would be translate to other biological scenarios.&lt;/li&gt;&#xA;&lt;li&gt;resampling, e.g. bootstrapping, jack-knifing and examine the tree structure of each node against the biological plausibility of the answer being correct. &lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For both points a biological a priori, i.e. biological plausibility of assessing the tree structure is required to confirm/reject the clustering method.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Clustering algorithms are not like the probabilistic methods, where the highest likelihood is the singular criteria and of course they do give different answers. Its a brief explanation I know but hope it makes sense.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-03-11T18:22:32.003" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11630" PostTypeId="1" AcceptedAnswerId="11637" CreationDate="2020-03-11T22:10:52.023" Score="3" ViewCount="173" Body="&lt;p&gt;A previous thread on Bioinformatics StackExchange citing an article by the Economist which references a cartoon of predictive outcomes by the CDC (&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/11547/economist-article-on-coronavirus&quot;&gt;Economist article on coronavirus&lt;/a&gt;) suggests a model in epidemiology using the &quot;basic reproductive rate&quot;, commonly known as R0. Although I'm no bioinformatician, I imagine that one could tweak a model based on similar infections like SARS and HDN1. Yet, I found no models predicting or giving confidence intervals for the spread of coronavirus. I wonder if the the data is still too sparse, e.g. we don't know if it's seasonal (mentioned in the above thread) and &lt;a href=&quot;https://biology.stackexchange.com/questions/90636/is-covid-19-claimed-to-get-less-deadly-over-time-if-so-why&quot;&gt;we cannot infer the virulence&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would such a model be reliable? If not, how many months of data would we need to reliably use a model predicting the spread?&lt;/p&gt;&#xA;" OwnerUserId="7388" LastEditorUserId="3848" LastEditDate="2020-12-04T15:34:03.863" LastActivityDate="2020-12-04T15:34:03.863" Title="Prediction model for spread of coronavirus" Tags="|statistics|covid-19|sars-cov-2|coronavirus|epidemiology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11631" PostTypeId="2" ParentId="11624" CreationDate="2020-03-12T02:33:13.903" Score="1" Body="&lt;p&gt;FYI coronaviruses have proof reading activity, unlike HIV, its a huge genome, unlike HIV. Thus the point mutation rate is unlikely to be as rapid as HIV.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The phylogenies in this paper are not really about the evolution or epidemiology of the virus. The trees used for viral taxonomy and form part of the International Committee for the Taxonomy of Viruses (ICTV) assessment in officially naming the virus. The ICTV is the authority in naming a virus. The ICTV essentially called it SARS2 as opposed to the WHO who used the name COVID-19. Thats the point of this paper and will be massively cited. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The broader evolutionary epidemiology of &quot;SARS2&quot; is for further studies. The Lancet paper on 22nd February gave some indication of this (and I suspect you know about that paper).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-03-12T02:33:13.903" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11632" PostTypeId="1" CreationDate="2020-03-12T03:34:48.297" Score="2" ViewCount="124" Body="&lt;p&gt;I need to construct the phylogenetic tree of the whole genus of streptomyces. And &lt;strong&gt;my goal&lt;/strong&gt; is to &lt;strong&gt;find out how my target strep species distributed in the whole strep genus and obtain the distribution pattern&lt;/strong&gt;. Strep is a big genus, has 658 species(65% of them have the genome info), and my target species is about 25. &lt;strong&gt;But the traditional 16S method seems not helping when it comes to strep is a genetic recombination genus&lt;/strong&gt;. And still lack of MLST scheme to distinguish my target species. I want to get the phylogeny for the whole genus, but I lack related genetics knowledge.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is something I did before try to get the tree.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I ran the orthomcl program for 400 strep genomes which I got from NCBI, and the result is: I got 22 orthologues&lt;/li&gt;&#xA;&lt;li&gt;I ran the fastani program for 200 strep genomes(type material) and got the ani result.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;What can I do to achieve my goal? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="7365" LastEditorUserId="3848" LastEditDate="2020-04-10T06:15:27.560" LastActivityDate="2021-01-05T08:03:05.250" Title="How to construct the phylogenetic tree of the &quot;High Recombination&quot; genus streptomyces?" Tags="|fasta|phylogenetics|phylogeny|bacteria|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11633" PostTypeId="1" CreationDate="2020-03-12T04:31:35.223" Score="2" ViewCount="32" Body="&lt;p&gt;The UCSC Genome Browser Team is &lt;a href=&quot;http://genome.ucsc.edu/blog/the-ucsc-genome-browser-coordinate-counting-systems/&quot; rel=&quot;nofollow noreferrer&quot;&gt;clear&lt;/a&gt; about that the genome browser GUI indexing which is 1-based, closed interval. Genomic sequence retrieval via the UCSC REST API is 1-based, closed interval. All UCSC tables are indexed at 0-based, half-open intervals.&lt;/p&gt;&#xA;&lt;p&gt;However, I can't seem to find a list of how resources are indexed at &lt;a href=&quot;http://bioinfo.hpc.cam.ac.uk/cellbase/webservices/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cellbase&lt;/a&gt;. I know genomic sequence queries are 1-based and closed-interval, but are all the other tables? I could come up with tests for each resource I use at Cellbase, but I'd need a known true value for each test, which is impossible or impractical for most resources I need.&lt;/p&gt;&#xA;&lt;p&gt;It's such a fundamental thing to know that there's probably a description somewhere, but I can't seem to find it. Or maybe there is a method to easily know, but I don't know that method. I've looked at:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the Cellbase &lt;a href=&quot;http://bioinfo.hpc.cam.ac.uk/cellbase/webservices/&quot; rel=&quot;nofollow noreferrer&quot;&gt;website&lt;/a&gt;,&lt;/li&gt;&#xA;&lt;li&gt;the Cellbase &lt;a href=&quot;http://docs.bioinfo.cipf.es/projects/cellbase/wiki&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki&lt;/a&gt; (down for maintenance as of 3/11/20), and&lt;/li&gt;&#xA;&lt;li&gt;I've read both the original &lt;a href=&quot;https://academic.oup.com/nar/article/40/W1/W609/1752209&quot; rel=&quot;nofollow noreferrer&quot;&gt;NAR paper&lt;/a&gt; and&lt;/li&gt;&#xA;&lt;li&gt;the recent &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-019-2726-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;BMC Bioinformatics&lt;/a&gt; paper on the pycellbase library (which I'm super-excited to try).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;There is no mention of how nucleotides are counted. I'm sure I'm missing something; any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="3564" LastEditorUserId="3564" LastEditDate="2020-11-13T06:34:33.977" LastActivityDate="2020-11-13T06:34:33.977" Title="What Cellbase resources are 0-indexed, and which are 1-indexed?" Tags="|genome|genomics|interval|genomicranges|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11634" PostTypeId="1" CreationDate="2020-03-12T05:45:06.413" Score="2" ViewCount="68" Body="&lt;p&gt;I am trying to assemble some genomes from pacbio reads using HGAP3 followed by quiver polishing. However, even after multiple rounds of quiver polishing, I cannot attain quality value 50 (QV50) for a genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;QV50 seems to be recommended and I am able to get upto QV30 only. I have done polishing for almost 15 times already. I have tried polishing with different parameter changes like subread length cutoff and blasr maximum divergence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone suggest me on how I can achieve an assembled genome of QV50 value?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="7336" LastActivityDate="2021-12-04T21:00:50.133" Title="polishing assembled genome to QV50 value" Tags="|assembly|pacbio|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11635" PostTypeId="2" ParentId="11043" CreationDate="2020-03-12T12:52:22.393" Score="2" Body="&lt;p&gt;Another viewpoint could be : statistically significant implies use of sampling theory to establish presence of significant difference  between  case and control groups. The &quot;clinical &quot; significance seems to test for correctness or validity of measures of variables. Apparenty, the measurement theory is meant for checking validity of data - the difference  could arise on account of incorrect measurements. The non-parameteric tests -  it seems - take care of measurement errors and hence, produce perhaps more reliable results.&lt;/p&gt;&#xA;" OwnerUserId="7374" LastActivityDate="2020-03-12T12:52:22.393" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11636" PostTypeId="1" AcceptedAnswerId="11641" CreationDate="2020-03-12T13:30:44.340" Score="-2" ViewCount="322" Body="&lt;p&gt;I have 15 .vcf files. I need to remove `common variants in the 1000 genomes database' appearing in at least 0.5% of the population&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know from where I may start?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you so much&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2020-03-13T13:51:18.597" Title="Removing common variants in the 1000 genomes database from .vcf" Tags="|vcf|wgs|1000genomes|" AnswerCount="2" CommentCount="0" ClosedDate="2020-03-16T18:21:43.087" ContentLicense="CC BY-SA 4.0" />
  <row Id="11637" PostTypeId="2" ParentId="11630" CreationDate="2020-03-12T14:51:50.300" Score="2" Body="&lt;p&gt;The original R0 calculation was developed in the early 1900s to model malaria transmission and is known as the Ross-MacDonald model so your idea is over 100 years old. The model was very famous for identifying the optimal strategy for malarial control by targeting the adult mosquito rather than the mosquito larvae. What it said was reducing the mosquito population and reducing malaria transmission were two different issues. So the bednet strategy is based in this idea and particularly spraying the inside of houses with insecticide.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;R0 was famously used in the 1980s to predict the HIV pandemic when WHO thought it would remain localised in Africa. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The problem with R0 and Coronaviruses is we don't know the asymptomatic population who could also transmit the virus. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This population is potentially very large. Epidemiologists will attempt extrapolation from cruise ship quarrentine, eg the Diamond Princess where everyone is tested to understand this transmission parameter and project this onto the rest of the planet. Clearly this could have limitations or inaccuracies. The other issue I am baffled by is seasonal transmission and whether this will occur with COVID-19. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One thing R0 of COVID-19 will identify is the latent period transmission which occurs before the symptoms, i.e. the incubation period. Essentially, people with cough/mild symptoms before they self isolate and become really ill with a lung infection will transmit the virus. In this context R0 will be/was very influential for understanding when to move between quarrentine and containment phases of the pandemic.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-03-12T20:52:33.497" LastActivityDate="2020-03-12T20:52:33.497" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="11638" PostTypeId="1" AcceptedAnswerId="11667" CreationDate="2020-03-12T14:52:38.257" Score="5" ViewCount="361" Body="&lt;p&gt;I have recently ran in R four geneset enrichment analysis in the same database (TCGA:breast cancer) comparing two intrinsic subtypes. The methods I used were:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;  &lt;li&gt;MIGSA, that imports mGSZ package and combines it with a SEA algorithm. Using RNA-seq TMM normalized counts as input, because the software supports it. All the parameters are as default.&lt;/li&gt;&#xA;  &lt;li&gt;mGSZ with standard parameters, using RNA-seq TMM normalized log2 cpm. All parameters as default&lt;/li&gt;&#xA;  &lt;li&gt;FGSEA preranked, using log fold change as ranking metric, obtained from limma deg from RNA-seq TMM normalized counts. All the parameters are as default.&lt;/li&gt;&#xA;  &lt;li&gt;GSVA + limma: extracted by-sample enrichment score from RNA-seq TMM normalized log2 cpm, then ran a differential expression analysis with geneset enrichment scores as if they were simple gene expression values. I know this is not a formal approach but I wanted to compare. All the parameters are as default.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;The genesets tested are the full combined list of KEGG, GO:BP, GO:MF and GO:CC. In all cases I trimmed the genesets to genesets of length &amp;lt;500, which leaves a total of 21723 genesets tested.&lt;br&gt;&#xA;Sadly, results are confusing. For starters mGSZ for some reason is cropping the output and giving results for only around 9700 of those genesets -I've already sent an email to the package mainteiner asking about that. But that isn't the biggest problem, the most shocking thing is that none of the analyses performedt has a good correlation to the results of any of the other analyses. Here I plotted the -log10(adjusted p-value + 0.0001) of each analysis agaisnt every other analysis, with lines indicating -log10(0.05)=1.3 and -log10(0.01)=2.&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/VEGsz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VEGsz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;I was expecting the results to be somehow different, but here I can't see even something remotely close to a correlation between the different results. It is also evident that MIGSA and GSVA+limma output a lot more significatively enriched genesets than the other two methods.&lt;br&gt;&#xA;I have gone through the code time and again, and I can't find anything errors. Also I have searched for studies doing this kind of direct comparison between methods, without luck so far. Has anybody experienced this kind of inconsistencies between gene set enrichment analysis results with different tools? What could I possible be doing wrong?.&lt;/p&gt;&#xA;" OwnerUserId="7389" LastEditorUserId="7389" LastEditDate="2020-03-13T12:39:31.490" LastActivityDate="2020-11-04T16:19:31.187" Title="are GSEA and other geneset enrichment analysis supposed to yield extremely different results between them?" Tags="|r|gsea|go-enrichment|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11639" PostTypeId="2" ParentId="11578" CreationDate="2020-03-12T16:31:51.310" Score="1" Body="&lt;p&gt;Col-0 is, to my knowledge, the standard reference Arabidopsis thaliana ecotype, so the easiest way to get the sequences for all genes is to just download them from Ensembl. The nucleotide sequences are &lt;a href=&quot;ftp://ftp.ensemblgenomes.org/pub/release-46/plants/fasta/arabidopsis_thaliana/cdna/&quot; rel=&quot;nofollow noreferrer&quot;&gt;available here&lt;/a&gt; and the protein sequences &lt;a href=&quot;ftp://ftp.ensemblgenomes.org/pub/release-46/plants/fasta/arabidopsis_thaliana/pep/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-03-12T16:31:51.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11640" PostTypeId="2" ParentId="11628" CreationDate="2020-03-12T17:58:41.570" Score="1" Body="&lt;p&gt;I am evaluating the clustering performance using &lt;a href=&quot;https://scikit-learn.org/stable/modules/clustering.html#clustering-performance-evaluation&quot; rel=&quot;nofollow noreferrer&quot;&gt;scikit&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2594" LastActivityDate="2020-03-12T17:58:41.570" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11641" PostTypeId="2" ParentId="11636" CreationDate="2020-03-12T19:33:50.283" Score="4" Body="&lt;p&gt;Use VEP, ANNOVAR or snpEff to annotate your VCF file (I'd recommend combining your VCFs into a single file if they're single sample VCFs or are all comprised of samples from the same experiment/cohort). You can annotate the VCF file with 1000g (among a ton of other annotation sources). Once done, you can use &lt;code&gt;bcftools view&lt;/code&gt; to subset the VCF as required.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or, you can subset the standalone 1000g VCF and pick regions with your frequency threshold, then you can use that regions file to subset your VCF file(s).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Remember to left-align and normalize (and split multi-allelic records in) your VCF files so all comparisons can be done accounting for chr, pos, ref and alt fields.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2020-03-12T19:33:50.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11643" PostTypeId="2" ParentId="10516" CreationDate="2020-03-13T03:52:43.270" Score="1" Body="&lt;p&gt;The hypothesis can be examined by t-test. You can group the data (number of catches) into  dimmed light, and fully lit street lights and proceed with t-test to assess whether dimmed light results  in less en-catchment or higher en-catchment or equivalent encatchment.  Alternatively, a linear model can be applied subject to certain conditions  here. PCA is not  appropriate here.&lt;/p&gt;&#xA;" OwnerUserId="7374" LastEditorUserId="7374" LastEditDate="2020-03-13T05:20:07.527" LastActivityDate="2020-03-13T05:20:07.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11644" PostTypeId="2" ParentId="11613" CreationDate="2020-03-13T04:33:44.560" Score="1" Body="&lt;p&gt;The only way I can think is to break the protein apart on methionine residues, blast each amino acid locus and concatenate any locus that does not score a query protein position of '1'. The theory is simple every protein starts with a methionine residue, however there are plenty of M residues within the protein that don't represent the beginning of the gene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Breaking the protein apart on M would be easy, but then automating and parsing the blast would be fiddly. Obvioulsy you will need to code it and if its a one off would it be worth it?&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-03-13T04:33:44.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11645" PostTypeId="2" ParentId="11611" CreationDate="2020-03-13T04:49:07.583" Score="1" Body="&lt;p&gt;I am not NGS expert, I known ML. Essentially you are performing data augmentation which is essential for ML. What I question is whether a replicate of e6 from a population of e7 would really be sufficient for ML because you need sample sizes (replicates in this case) of &gt;100000. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The answer is really simple you bootstrap the data (sample with replacement), to obtain your replicates. Numpy has a random module that will do this,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;p.random.seed(345)&#xA;pop = np.random.randint(0,10000000 , size=1000000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Use &lt;code&gt;pop&lt;/code&gt; to select from fastaq. The other way in is to define each fastq sequence within pop then simply,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sample = np.random.choice(pop, size=1000000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or a combination of the two. If you can do ML you can do Python coding.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-03-13T04:49:07.583" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11646" PostTypeId="5" CreationDate="2020-03-13T05:23:23.537" Score="0" Body="&lt;p&gt;Phylogeny shares a close relationship with phylogenetics. A molecular phylogeny is the same thing as a phylogenetic tree. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A phylogeny doesn't need to be built using genetic data, whilst phylogenetics exclusively uses genetic data thus phylogenetics is a subset of phylogeny. A phylogeny could be used for example to classify insects based on morphological criteria or describe antibody variation between patients and may not related to the underlying genetic variation of the pathogen or the patient but the history of exposure to different pathogens.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Parimony, distance matrices, maximum likelihood and Bayesian estimation are all methods to produce a phylogeny.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-03-17T04:31:32.973" LastActivityDate="2020-03-17T04:31:32.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11647" PostTypeId="4" CreationDate="2020-03-13T05:23:23.537" Score="0" Body="Phylogeny is a statistical approach for reconstructing evolutionary relationships using trees " OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-03-17T04:31:29.817" LastActivityDate="2020-03-17T04:31:29.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11648" PostTypeId="2" ParentId="6833" CreationDate="2020-03-13T07:02:54.177" Score="1" Body="&lt;p&gt;I have not seen the papers but I know the technique SVM. This is a machine learning approach its pretty good, but has been around this the 1980s. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its quite simple known ORIs are given with the associated genetic trait, e.g. GC skew, SVM will then regress the GC skew of a known ORI against the GC content of the genome, there will be a training, parameter optimization and test split in the data, typically 80:20:20. De novo data will then be feed into the trained algorithm, in the same way the test data was, and a prediction will be generated. There are weaknesses, the authors have used only one ML technique, typically numerous will be used e.g. SVM, random forest, lasso regression etc... In addition, the prediction accuracy for GC is low (75%), in addition 500 genomes isn't great for ML, you need 1000s. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-03-13T07:02:54.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11650" PostTypeId="2" ParentId="11042" CreationDate="2020-03-13T13:05:39.923" Score="6" Body="&lt;p&gt;I would suggest to use &lt;code&gt;agat_convert_sp_gff2gtf.pl&lt;/code&gt; from &lt;a href=&quot;https://github.com/NBISweden/AGAT&quot; rel=&quot;noreferrer&quot;&gt;AGAT&lt;/a&gt; because you loose information with &lt;code&gt;gffread&lt;/code&gt;.&lt;br&gt;&#xA;e.g here a gff example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##gff-version 3&#xA;scaffold625 maker   gene    337818  343277  .   +   .   ID=CLUHARG00000005458;Name=TUBB3_2&#xA;scaffold625 maker   transcript  337818  343277  .   +   .   ID=CLUHART00000008717;Parent=CLUHARG00000005458&#xA;scaffold625 maker   CDS 337915  337971  .   +   0   ID=CLUHART00000008717:cds;Parent=CLUHART00000008717&#xA;scaffold625 maker   CDS 340733  340841  .   +   0   ID=CLUHART00000008717:cds;Parent=CLUHART00000008717&#xA;scaffold625 maker   CDS 341518  341628  .   +   2   ID=CLUHART00000008717:cds;Parent=CLUHART00000008717&#xA;scaffold625 maker   CDS 341964  343033  .   +   2   ID=CLUHART00000008717:cds;Parent=CLUHART00000008717&#xA;scaffold625 maker   exon    337818  337971  .   +   0   ID=CLUHART00000008717:exon1;Parent=CLUHART00000008717&#xA;scaffold625 maker   exon    340733  340841  .   +   0   ID=CLUHART00000008717:exon2;Parent=CLUHART00000008717&#xA;scaffold625 maker   exon    341518  341628  .   +   2   ID=CLUHART00000008717:exon3;Parent=CLUHART00000008717&#xA;scaffold625 maker   exon    341964  343277  .   +   2   ID=CLUHART00000008717:exon4;Parent=CLUHART00000008717&#xA;scaffold625 maker   five_prime_UTR  337818  337914  .   +   .   ID=CLUHART00000008717:five_prime_utr;Parent=CLUHART00000008717&#xA;scaffold625 maker   three_prime_UTR 343034  343277  .   +   .   ID=CLUHART00000008717:three_prime_utr;Parent=CLUHART00000008717&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;with gffread you get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;scaffold625 maker   transcript  337818  343277  .   +   .   transcript_id &quot;CLUHART00000008717&quot;; gene_id &quot;CLUHARG00000005458&quot;;&#xA;scaffold625 maker   exon    337818  337971  .   +   .   transcript_id &quot;CLUHART00000008717&quot;; gene_id &quot;CLUHARG00000005458&quot;;&#xA;scaffold625 maker   exon    340733  340841  .   +   .   transcript_id &quot;CLUHART00000008717&quot;; gene_id &quot;CLUHARG00000005458&quot;;&#xA;scaffold625 maker   exon    341518  341628  .   +   .   transcript_id &quot;CLUHART00000008717&quot;; gene_id &quot;CLUHARG00000005458&quot;;&#xA;scaffold625 maker   exon    341964  343277  .   +   .   transcript_id &quot;CLUHART00000008717&quot;; gene_id &quot;CLUHARG00000005458&quot;;&#xA;scaffold625 maker   CDS 337915  337971  .   +   0   transcript_id &quot;CLUHART00000008717&quot;; gene_id &quot;CLUHARG00000005458&quot;;&#xA;scaffold625 maker   CDS 340733  340841  .   +   0   transcript_id &quot;CLUHART00000008717&quot;; gene_id &quot;CLUHARG00000005458&quot;;&#xA;scaffold625 maker   CDS 341518  341628  .   +   2   transcript_id &quot;CLUHART00000008717&quot;; gene_id &quot;CLUHARG00000005458&quot;;&#xA;scaffold625 maker   CDS 341964  343033  .   +   2   transcript_id &quot;CLUHART00000008717&quot;; gene_id &quot;CLUHARG00000005458&quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;while with AGAT you get&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;scaffold625 maker   gene    337818  343277  .   +   .   ID CLUHARG00000005458 ; Name TUBB3_2 ; gene_id CLUHARG00000005458&#xA;scaffold625 maker   mRNA    337818  343277  .   +   .   ID CLUHART00000008717 ; Parent CLUHARG00000005458 ; gene_id CLUHARG00000005458 ; transcript_id CLUHART00000008717&#xA;scaffold625 maker   exon    337818  337971  .   +   0   ID &quot;CLUHART00000008717:exon1&quot;  ; Parent CLUHART00000008717 ; gene_id CLUHARG00000005458 ; transcript_id CLUHART00000008717&#xA;scaffold625 maker   exon    340733  340841  .   +   0   ID &quot;CLUHART00000008717:exon2&quot;  ; Parent CLUHART00000008717 ; gene_id CLUHARG00000005458 ; transcript_id CLUHART00000008717&#xA;scaffold625 maker   exon    341518  341628  .   +   2   ID &quot;CLUHART00000008717:exon3&quot;  ; Parent CLUHART00000008717 ; gene_id CLUHARG00000005458 ; transcript_id CLUHART00000008717&#xA;scaffold625 maker   exon    341964  343277  .   +   2   ID &quot;CLUHART00000008717:exon4&quot;  ; Parent CLUHART00000008717 ; gene_id CLUHARG00000005458 ; transcript_id CLUHART00000008717&#xA;scaffold625 maker   CDS 337915  337971  .   +   0   ID &quot;CLUHART00000008717:cds&quot;  ; Parent CLUHART00000008717 ; gene_id CLUHARG00000005458 ; transcript_id CLUHART00000008717&#xA;scaffold625 maker   CDS 340733  340841  .   +   0   ID &quot;CLUHART00000008717:cds&quot;  ; Parent CLUHART00000008717 ; gene_id CLUHARG00000005458 ; transcript_id CLUHART00000008717&#xA;scaffold625 maker   CDS 341518  341628  .   +   2   ID &quot;CLUHART00000008717:cds&quot;  ; Parent CLUHART00000008717 ; gene_id CLUHARG00000005458 ; transcript_id CLUHART00000008717&#xA;scaffold625 maker   CDS 341964  343033  .   +   2   ID &quot;CLUHART00000008717:cds&quot;  ; Parent CLUHART00000008717 ; gene_id CLUHARG00000005458 ; transcript_id CLUHART00000008717&#xA;scaffold625 maker   five_prime_UTR  337818  337914  .   +   .   ID &quot;CLUHART00000008717:five_prime_utr&quot;  ; Parent CLUHART00000008717 ; gene_id CLUHARG00000005458 ; transcript_id CLUHART00000008717&#xA;scaffold625 maker   three_prime_UTR 343034  343277  .   +   .   ID &quot;CLUHART00000008717:three_prime_utr&quot;  ; Parent CLUHART00000008717 ; gene_id CLUHARG00000005458 ; transcript_id CLUHART00000008717&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So you can see as example that &lt;code&gt;Name=TUBB3_2&lt;/code&gt; from the gene feature disappeared with &lt;code&gt;gffread&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2486" LastEditorUserId="2486" LastEditDate="2020-03-17T16:00:46.463" LastActivityDate="2020-03-17T16:00:46.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11652" PostTypeId="2" ParentId="11636" CreationDate="2020-03-13T13:51:18.597" Score="-3" Body="&lt;p&gt;Think removing everything &amp;lt; 0.05 from &lt;code&gt;1000g2015aug_all&lt;/code&gt; column of ANNOVAR annotation file helps with this&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2020-03-13T13:51:18.597" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11653" PostTypeId="1" AcceptedAnswerId="11656" CreationDate="2020-03-13T23:33:26.680" Score="0" ViewCount="67" Body="&lt;p&gt;I am trying to instll sigproextractor package on my Mac&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pc-133-235:~ fi1d18&lt;span class=&quot;math-container&quot;&gt;$ python --version&#xA;Python 2.7.10&#xA;pc-133-235:~ fi1d18$&lt;/span&gt; python3 --version&#xA;Python 3.8.2&#xA;pc-133-235:~ fi1d18&lt;span class=&quot;math-container&quot;&gt;$ pip --version&#xA;-bash: pip: command not found&#xA;pc-133-235:~ fi1d18$&lt;/span&gt; curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But whatever I am trying says that can't find pip &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2020-03-14T06:46:21.450" Title="Installing a package on Mac" Tags="|python|conda|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11654" PostTypeId="2" ParentId="11653" CreationDate="2020-03-14T01:46:57.273" Score="2" Body="&lt;p&gt;At a guess you don't have python3 installed and hence pip3&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/17271319/how-do-i-install-pip-on-macos-or-os-x&quot;&gt;https://stackoverflow.com/questions/17271319/how-do-i-install-pip-on-macos-or-os-x&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally I would install via Anaconda3 and skip a direct Python3 installation altogether. The answer above recommends using homebrew, again personally I wouldn't touch it and go straight for a full Anaconda3 installation. You might be able to install via&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c XXX sigproextractor&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where XXX is the channel, but its certainly on github. Anaconda3 is very easy to install.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-03-14T01:46:57.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11655" PostTypeId="1" CreationDate="2020-03-14T05:57:13.027" Score="2" ViewCount="72" Body="&lt;p&gt;I have a sequence below from MYC gene about which I need to translate the sequence in all possible frames AND identify (for each frame) which codons are actually used in MYC&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AAAGCAATAATACAATTTAAAACCTGGGTCTCTAGAGGTGTTAGGACGTG&#xA;GTGTTGGGTAGGCGCAGGCAGGGGAAAAGGGAGGCGAGGATGTGTCCGAT&#xA;TCTCCTGGAATCGTTGACTTGGAAAAACCAGGGCGAATCTCCGCACCCAG&#xA;CCCTGACTCCCCTGCCGCGGCCGCCCTCGGGTGTCCTCGCGCCCGAGATG&#xA;CGGAGGAACTGCGAGGAGCGGGGCTCTGGGCGGTTCCAGAACAGCTGCTA&#xA;CCCTTGGTGGGGTGGCTCCGGGGGAGGTATCGCAGCGGGGTCTCTGGCGC&#xA;AGTTGCATCTCCGTATTGAGTGCGAAGGGAGGTGCCCCTATTATTATTTG&#xA;ACACCCCCCTTGTATTTATGGAGGGGTGTTAAAGCCCGCGGCTGAGCTCG&#xA;CCACTCCAGCCGGCGAGAGAAAGAAGAAAAGCTGGCAAAAGGAGTGTTGG&#xA;ACGGGGGCGGTACTGGGGGTGGGGACGGGGGCGGTGGAGAGGGAAGGTTG&#xA;GGAGGGGCTGCGGTGCCGGCGGGGGTAGGAGAGCGGCTAGGGCGCGAGTG&#xA;GGAACAGCCGCAGCGGAGGGGCCCCGGCGCGGAGCGGGGTTCACGCAGCC&#xA;GCTAGCGCCCAGGCGCCTCTCGCCTTCTCCTTCAGGTGGCGCAAAACTTT&#xA;GTGCCTTGGATTTTGGCAAATTGTTTTCCTCACCGCCACCTCCCGCGGCT&#xA;TCTTAAGGGCGCCAGGGCCGATTTCGATTCCTCTGCCGCTGCGGGGCCGA&#xA;CTCCCGGGCTTTGCGCTCCGGGCTCCCGGGGGAGCGGGGGCTCGGCGGGC&#xA;ACCAAGCCGCTGGTTCACTAAGTGCGTCTCCGAGATAGCAGGGGACTGTC&#xA;CAAAGGGGGTGAAAGGGTGCTCCCTTTATTCCCCCACCAAGACCACCCAG&#xA;CCGCTTTAGGGGATAGCTCTGCAAGGGGAGAGGTTCGGGACTGTGGCGCG&#xA;CACTGCGCGCTGCGCCAGGTTTCCGCACCAAGACCCCTTTAACTCAAGAC&#xA;TGCCTCCCGCTTTGTGTGCCCCGCTCCAGCAGCCTCCCGCGACGATGCCC&#xA;CTCAACGTTAGCTTCACCAACAGGAACTATGACCTCGACTACGACTCGGT&#xA;GCAGCCGTATTTCTACTGCGACGAGGAGGAGAACTTCTACCAGCAGCAGC&#xA;AGCAGAGCGAGCTGCAGCCCCCGGCGCCCAGCGAGGATATCTGGAAGAAA&#xA;TTCGAGCTGCTGCCCACCCCGCCCCTGTCCCCTAGCCGCCGCTCCGGGCT&#xA;CTGCTCGCCCTCCTACGTTGCGGTCACACCCTTCTCCCTTCGGGGAGACA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where should I look for all 4 pieces of information easily? Please guide.&lt;/p&gt;&#xA;" OwnerUserId="4996" LastEditorUserId="3848" LastEditDate="2020-03-15T10:35:20.627" LastActivityDate="2021-04-09T13:04:45.477" Title="Translating a genome sequence into possible frames" Tags="|sequence-alignment|ngs|proteins|blast|phylogenetics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11656" PostTypeId="2" ParentId="11653" CreationDate="2020-03-14T06:46:21.450" Score="4" Body="&lt;p&gt;maybe you could try &lt;code&gt;python3 -m pip install package&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="7365" LastActivityDate="2020-03-14T06:46:21.450" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11657" PostTypeId="2" ParentId="2913" CreationDate="2020-03-14T14:03:20.063" Score="1" Body="&lt;p&gt;As of the most recent snakemake (v5.10.0) the recommended way of defining resources is through a &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/executing/cli.html#profiles&quot; rel=&quot;nofollow noreferrer&quot;&gt;profile&lt;/a&gt;.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems you are trying to submit a job on an LSF cluster system - there is a snakemake &lt;a href=&quot;https://github.com/Snakemake-Profiles/snakemake-lsf&quot; rel=&quot;nofollow noreferrer&quot;&gt;profile for LSF&lt;/a&gt; to save you having to do this work yourself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It reduces your snakemake submission to something like&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;snakemake --profile &quot;lsf&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="149" LastActivityDate="2020-03-14T14:03:20.063" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11658" PostTypeId="2" ParentId="11634" CreationDate="2020-03-14T16:40:26.160" Score="1" Body="&lt;p&gt;15 times is too many times. If you were going to get quiver to get you good QVs it would have happened after 2-3 rounds probably. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why are you using quiver? Are your reads from RSII chemistry? Arrow is recommended for Sequel reads.&lt;/li&gt;&#xA;&lt;li&gt;What is your dataset coverage? If you do not have very high coverage (I think &gt;30X is usually a minimum for a eukaryotic genome) it may be that you are just not going to get exactly the values you want because assembling low coverage data is hard.&lt;/li&gt;&#xA;&lt;li&gt;Have you tried other polishing tools? They might be able to get you better value. I am not an expert but some people like to use Racon or Apollo.&lt;/li&gt;&#xA;&lt;li&gt;While I'm not an expert, looking into how the QVs are calculated can be helpful. I looked at the &lt;a href=&quot;https://manpages.debian.org/testing/pbgenomicconsensus/genomicconsensus.7&quot; rel=&quot;nofollow noreferrer&quot;&gt;quiver docs&lt;/a&gt; and it looks like if the reads map ambiguously then QV is low. So if you for example have lots of haplotigs, then mapping might be ambiguous for a lot of reads, and therefore this will depress your QV values. This is therefore a problem that can come from the starting assembly and not just the polishing. &lt;strong&gt;You may want to go back and take another look at your starting assembly&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-03-14T16:40:26.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11659" PostTypeId="2" ParentId="11655" CreationDate="2020-03-14T19:51:13.707" Score="1" Body="&lt;p&gt;The 6-frame translation  results are below you will need to do the rest. It is translated 5' to 3' but appears to contain two splice sites. You will need to use blast to resolve the start/stop of the splicing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All 6-codon translation is available the Expasy server &lt;a href=&quot;https://web.expasy.org/translate/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. You will need to blast to resolve the splicing.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;5'3' Frame 1&#xA;KAIIQFKTWVSRGVRTWCWVGAGRGKGRRGCVRFSWNR-LGKTRANLRTQP-LPCRGRPRVSSRPRCGGTARSGALGGSRTAATLGGVAPGEVSQRGLWRSCISVLSAKGGAPIII-HPPCIYGGVLKPAAELATPAGERKKKSWQKECWTGAVLGVGTGAVEREGWEGLRCRRG-ESG-GASGNSRSGGAPARSGVHAAASAQAPLAFSFRWRKTLCLGFWQIVFLTATSRGFLRAPGPISIPLPLRGRLPGFALRAPGGAGARRAPSRWFTKCVSEIAGDCPKGVKGCSLYSPTKTTQPL-GIALQGERFGTVARTARCARFPHQDPFNSRLPPALCAPLQQPPATMPLNVSFTNRNYDLDYDSVQPYFYCDEEENFYQQQQQSELQPPAPSEDIWKKFELLPTPPLSPSRRSGLCSPSYVAVTPFSLRGD&#xA;5'3' Frame 2&#xA;KQ-YNLKPGSLEVLGRGVG-AQAGEKGGEDVSDSPGIVDLEKPGRISAPSPDSPAAAALGCPRARDAEELRGAGLWAVPEQLLPLVGWLRGRYRSGVSGAVASPY-VRREVPLLLFDTPLVFMEGC-SPRLSSPLQPARERRKAGKRSVGRGRYWGWGRGRWRGKVGRGCGAGGGRRAARARVGTAAAEGPRRGAGFTQPLAPRRLSPSPSGGAKLCALDFGKLFSSPPPPAAS-GRQGRFRFLCRCGADSRALRSGLPGERGLGGHQAAGSLSASPR-QGTVQRG-KGAPFIPPPRPPSRFRG-LCKGRGSGLWRALRAAPGFRTKTPLTQDCLPLCVPRSSSLPRRCPSTLASPTGTMTSTTTRCSRISTATRRRTSTSSSSRASCSPRRPARISGRNSSCCPPRPCPLAAAPGSARPPTLRSHPSPFGET&#xA;5'3' Frame 3&#xA;SNNTI-NLGL-RC-DVVLGRRRQGKREARMCPILLESLTWKNQGESPHPALTPLPRPPSGVLAPEMRRNCEERGSGRFQNSCYPWWGGSGGGIAAGSLAQLHLRIECEGRCPYYYLTPPLYLWRGVKARG-ARHSSRREKEEKLAKGVLDGGGTGGGDGGGGEGRLGGAAVPAGVGERLGREWEQPQRRGPGAERGSRSR-RPGASRLLLQVAQNFVPWILANCFPHRHLPRLLKGARADFDSSAAAGPTPGLCAPGSRGSGGSAGTKPLVH-VRLRDSRGLSKGGERVLPLFPHQDHPAALGDSSARGEVRDCGAHCALRQVSAPRPL-LKTASRFVCPAPAASRDDAPQR-LHQQEL-PRLRLGAAVFLLRRGGELLPAAAAERAAAPGAQRGYLEEIRAAAHPAPVP-PPLRALLALLRCGHTLLPSGR&#xA;3'5' Frame 1&#xA;CLPEGRRV-PQRRRASRARSGG-GTGAGWAAARISSRYPRWAPGAAARSAAAAGRSSPPRRSRNTAAPSRSRGHSSCW-S-R-GASSREAAGAGHTKREAVLS-RGLGAETWRSAQCAPQSRTSPLAELSPKAAGWSWWGNKGSTLSPPLDSPLLSRRRT--TSGLVPAEPPLPREPGAQSPGVGPAAAEESKSALAPLRSRGRWR-GKQFAKIQGTKFCAT-RRRREAPGR-RLREPRSAPGPLRCGCSHSRPSRSPTPAGTAAPPNLPSPPPPSPPPVPPPSNTPFASFSSFSRRLEWRAQPRALTPLHKYKGGVK---GHLPSHSIRRCNCARDPAAIPPPEPPHQG-QLFWNRPEPRSSQFLRISGARTPEGGRGRGVRAGCGDSPWFFQVNDSRRIGHILASLFPCLRLPNTTS-HL-RPRF-IVLLL&#xA;3'5' Frame 2&#xA;VSPKGEGCDRNVGGRAEPGAAARGQGRGGQQLEFLPDILAGRRGLQLALLLLLVEVLLLVAVEIRLHRVVVEVIVPVGEANVEGHRRGRLLERGTQSGRQS-VKGVLVRKPGAARSARHSPEPLPLQSYPLKRLGGLGGGIKGAPFHPLWTVPCYLGDALSEPAAWCPPSPRSPGSPERKARESAPQRQRNRNRPWRP-EAAGGGGEENNLPKSKAQSFAPPEGEGERRLGASGCVNPAPRRGPSAAAVPTRALAALLPPPAPQPLPTFPLHRPRPHPQYRPRPTLLLPAFLLSLAGWSGELSRGL-HPSINTRGVSNNNRGTSLRTQYGDATAPETPLRYLPRSHPTKGSSCSGTAQSPAPRSSSASRARGHPRAAAAGESGLGAEIRPGFSKSTIPGESDTSSPPFSPACAYPTPRPNTSRDPGFKLYYCF&#xA;3'5' Frame 3&#xA;SPRREKGVTAT-EGEQSPERRLGDRGGVGSSSNFFQISSLGAGGCSSLCCCCW-KFSSSSQ-KYGCTES-SRS-FLLVKLTLRGIVAGGCWSGAHKAGGSLELKGSWCGNLAQRAVRATVPNLSPCRAIP-SGWVVLVGE-REHPFTPFGQSPAISETHLVNQRLGARRAPAPPGARSAKPGSRPRSGRGIEIGPGALKKPREVAVRKTICQNPRHKVLRHLKEKARGAWALAAA-TPLRAGAPPLRLFPLAP-PLSYPRRHRSPSQPSLSTAPVPTPSTAPVQHSFCQLFFFLSPAGVASSAAGFNTPP-IQGGCQIIIGAPPFALNTEMQLRQRPRCDTSPGATPPRVAAVLEPPRAPLLAVPPHLGREDTRGRPRQGSQGWVRRFALVFPSQRFQENRTHPRLPFPLPAPTQHHVLTPLETQVLNCIIA&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-03-15T10:36:13.517" LastActivityDate="2020-03-15T10:36:13.517" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11660" PostTypeId="2" ParentId="11632" CreationDate="2020-03-14T20:18:52.947" Score="1" Body="&lt;p&gt;You need bootstrap support using a model-based tree building algorhithm, via maximum likelihood (a few people use Bayes). The file format is relaxed phylip format (please submit a separate question if you have difficulties here - its a bit tricky).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use standard RAxML &lt;a href=&quot;https://github.com/stamatak/standard-RAxML&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, specifically raxmlHPC (easily downloadable and compiles on Linux and OSX). The codes are quite complicated and I've given them below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A robust maximum likelihood tree is,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;raxmlHPC -m GTRGAMMA -n output.tre -o outgroup-p 10000 -s alignment_file.phy&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To make a bootstrap tree,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;raxmlHPC -b 459 -m GTRGAMMA -n output.tre -o outgroup -p 10000 -s /path_to_alignment.phy -N 500&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This tree will bootstrap for 500 replications, however to start I would use 100 replications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Make a consensus tree of the bootstraps,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;raxmlHPC -f b -m GTRGAMMA -n output.tre -o outgroup -p 10000 -t /path_to_master_tree(best_tree) -z path_to_bootfile&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You require bootstrap support &gt;80% and please repeat this with and without 5-2/5-3 (it still looks long)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The cluster you have access to is fine for the calculation, it will take aound 24 hours to complete one bootstrap calculation for one data set and obviously you need to parallelise you calculation across the 22 contigs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Viewing the tree, FigTree (for Mac OSX) is easy.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Root&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Rooting can be complicated because I don't really know your bacteria.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The recombination issue is more complicated, but I would construct 22 trees from your contigs and assess them for congruence. Panmixia is a concerned, which means too much recombination&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-03-15T10:52:28.207" LastActivityDate="2020-03-15T10:52:28.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11661" PostTypeId="1" CreationDate="2020-03-15T10:53:13.627" Score="1" ViewCount="78" Body="&lt;p&gt;I have a fasta file with several 120-concatenation protein sequences. I also have a data frame with 120 names of proteins in column 1 and their length in column 2. Using this dataframe I want to create a new fasta file per sequence in which I get each protein independent with the name of the protein and the bacteria.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here my fasta file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Bacteria_1&#xA;Seq('GRRKAAVARAYMTAGKGNVTVNKKDYKDFFPVPTLQYKITQPFAITETLGQYDV...IEE', SingleLetterAlphabet())&#xA;&#xA;Bacteria_2&#xA;Seq('GRRKDSVARVYLRPGKGEIVVNKRPVEDYFGRETLKMVLRQPLELTESLDKFDI...YK-', SingleLetterAlphabet())&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here my dataframe&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;      Names  LEN&#xA;0     Ribosomal_S9:  121&#xA;1     Ribosomal_S8:  129&#xA;2    Ribosomal_L10:  100&#xA;3             GrpE:  166&#xA;4           DUF150:  141&#xA;..              ...  ...&#xA;115      TIGR03632:  117&#xA;116      TIGR03654:  175&#xA;117      TIGR03723:  314&#xA;118      TIGR03725:  212&#xA;119      TIGR03953:  188&#xA;&#xA;[120 rows x 2 columns]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The name of the genes and their length is the same for all my bacteria so this df is applicable for all sequences in my fasta file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am trying to get new fasta files like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Bacteria_1 Ribosomal_S9:&#xA;GRRKAAVARAYMTAGKGNVTVNKKDYKDFFPVPTLQYKITQPFAITETLGQYDV...&#xA;&#xA;&amp;gt;Bacteria_1 Ribosomal_S8:&#xA;KDYKDFFPVPTLAAVARAYMTAGKGNLGQYDVGRRKAAVARAYMTAGKGN...&#xA;...&#xA;&#xA;&amp;gt;Bacteria_1 TIGR03953:&#xA;AJEFISUBEVFCVALSKDMCFPADNSCF0AISCIABNSCNAISCN...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The second fasta file&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Bacteria_2 Ribosomal_S9:&#xA;GRRKAAVARAYMTAGKGNVTVNKKDYKDFFPVPTLQYKITQPFAITETLGQYDV...&#xA;&#xA;&amp;gt;Bacteria_2 Ribosomal_S8:&#xA;KDYKDFFPVPTLAAVARAYMTAGKGNLGQYDVGRRKAAVARAYMTAGKGN...&#xA;...&#xA;&#xA;&amp;gt;Bacteria_2 TIGR03953:&#xA;AJEFISUBEVFCVALSKDMCFPADNSCF0AISCIABNSCNAISCN...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7405" LastEditorUserId="3848" LastEditDate="2020-04-23T17:50:31.693" LastActivityDate="2020-04-23T17:50:31.693" Title="Cutting the sequence into several sequences with the information of a dataframe" Tags="|fasta|ngs|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11662" PostTypeId="2" ParentId="11661" CreationDate="2020-03-15T11:22:15.693" Score="1" Body="&lt;p&gt;We wouldn't write the code for you from scratch (thats your job), but its an easy problem because in Python any string is an &quot;array&quot; or &quot;dictionary&quot;. Thus &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bacteria1 = 'GRRKAAVARAYMTAGKGNVTVNKKDYKDFFPVPTLQYKITQPFAITETLGQYDVIEEXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'&#xA;print (bacteria1[0:120])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will define the first gene in your list, remembering the array starts at 0 whilst your gene starts at position 1 (so you need to subtract 1). You simply parse the input list so the gene lengths form a dictionary and loop through the dictionary of gene lengths ensuring the last length is added to the current one and then issuing a write command. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You don't really need biopython in this instance, unless it will parse the gene length file you need to open (I'm not aware it can).&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;BTW the format you supplied isn't fasta it looks like a Biopython statement, again in this problem I wouldn't bother because Python is perfectly capable of splicing a string.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-03-15T11:29:00.727" LastActivityDate="2020-03-15T11:29:00.727" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11663" PostTypeId="2" ParentId="11609" CreationDate="2020-03-15T11:46:16.553" Score="1" Body="&lt;p&gt;The one thing that strikes me is that OrthoMCL was written under David Roos supervision, which really means it was written with Toxoplasma gondii in mind, i.e. a eukaryotic parasite with notable levels of gene duplication. They tested it against Plasmodium falciparum, but that isn't surprising because both are members of the Apicomplexa. Personally I would suggest looking at a tool more orientated towards bacteria, albeit I'd like to see the output of OrthoMCL. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Prokka is a tool that might give better results, &lt;a href=&quot;https://kbase.us/applist/apps/ProkkaAnnotation/annotate_contigs/release?gclid=EAIaIQobChMIttzYjbOc6AIVibHtCh258QmaEAAYASAAEgJ5xvD_BwE&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, which is designed for bacteria, but personally I would extract the orthologes/homologues directly if the primary purpose is to contruct a phylogeny, i.e. I would parse annotated genes which are not &quot;hypothetical&quot;. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-03-15T11:46:16.553" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11664" PostTypeId="1" AcceptedAnswerId="11670" CreationDate="2020-03-15T18:06:18.067" Score="1" ViewCount="65" Body="&lt;p&gt;The BUSCO &lt;code&gt;_full_table.tsv&lt;/code&gt; output has scores per gene (an example bellow).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/31/19/3210/211866&quot; rel=&quot;nofollow noreferrer&quot;&gt;the original publication&lt;/a&gt;, authors just mention a condition to satisfy a bitscore, but I could not find what the bitscore actually is. In the &lt;a href=&quot;https://busco.ezlab.org/busco_userguide.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;use guide&lt;/a&gt; they say&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;If found to be complete, whether single-copy or duplicated, the BUSCO&#xA;  matches have scored within the expected range of scores ...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;but again, I don't understand what the score actually represents. I am even not sure if greater the better or visa reverse. Does anybody know what the score is?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# BUSCO version is: 4.0.2 &#xA;# The lineage dataset is: insecta_odb10 (Creation date: 2019-11-20, number of species: 75, number of BUSCOs: 1367)&#xA;# Busco id      Status  Sequence        Gene Start      Gene End        Score   Length&#xA;47at50557       Complete        scf1:20448-42284    20448   42284   4607.1  3554    https://www.orthodb.org/v10?query=47at50557     Immunoglobulin-like domain&#xA;547at50557      Complete        scf2:9780-23590    9780    23590   2743.8  1555    https://www.orthodb.org/v10?query=547at50557    protein unc-80 homolog&#xA;755at50557      Complete        scf3:9036-28083    9036    28083   3149.9  2206    https://www.orthodb.org/v10?query=755at50557    SRCR-like domain&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="57" LastActivityDate="2020-03-16T14:01:25.773" Title="What is the per-gene score in BUSCO output?" Tags="|busco|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="11665" PostTypeId="1" CreationDate="2020-03-16T03:56:34.163" Score="4" ViewCount="195" Body="&lt;p&gt;I'm a linguist interested in phylogenetic tree inference using language data. I'm posting here because I'm using Bayesian phylogenetic methods in my work (probably using BEAST and/or RevBayes). For the purposes of this question, please accept the assumption that phylogenetic trees are generally applicable to languages (I'm aware this premise may be subject to dispute). My question is intended to be methodological and not necessarily domain specific. You could imagine I'm constructing biological species trees using morphological data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two datasets: A set of binary traits (&lt;span class=&quot;math-container&quot;&gt;$D_{b}$&lt;/span&gt;) and a set of continuous traits (&lt;span class=&quot;math-container&quot;&gt;$D_{c}$&lt;/span&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to compare the results of phylogenetic tree inference with &lt;strong&gt;(a)&lt;/strong&gt; binary data alone vs. &lt;strong&gt;(b)&lt;/strong&gt; binary data and continuous data together. Model parameters are the same, except of course &lt;strong&gt;(b)&lt;/strong&gt; requires some extra parameters to model the evolution of continuous traits. In other words, both the data and model of &lt;strong&gt;(a)&lt;/strong&gt; is a subset of &lt;strong&gt;(b)&lt;/strong&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know this probably sounds like a strange research question. The reason it's of interest to me is that the binary data represent a data type that is well-established in historical linguistics and often used to infer language trees. The continuous data come from different language structures and haven't been used to infer language trees before. I want to know if adding this new kind of data can help aid tree inference significantly or if it has a neutral effect (or even just introduces more noise).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm familiar with phylogenetic model comparison using Bayes factors. But in the standard use case I'm familiar with, the dataset stays exactly the same and Bayes factors are used to evaluate the fit of different evolutionary models.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My question: Is there a standard way of evaluating the effect of adding more data to a phylogenetic tree model? Can marginal likelihoods be compared still even though the data differs in each model?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not, would it make any sense to reframe the question as one of model comparison? I'm imagining, for example, running both &lt;strong&gt;(a)&lt;/strong&gt; and &lt;strong&gt;(b)&lt;/strong&gt; with identical datasets (&lt;span class=&quot;math-container&quot;&gt;$D_b + D_c$&lt;/span&gt; for both). In &lt;strong&gt;(a)&lt;/strong&gt;, there would be a trivial model of trait evolution that doesn't aid tree inference at all—something like a model where traits can jump from any value to any other value with equal probability(?). In &lt;strong&gt;(b)&lt;/strong&gt;, there would be a more standard model of continuous trait evolution, e.g. Brownian motion.&lt;/p&gt;&#xA;" OwnerUserId="7407" LastEditorUserId="7407" LastEditDate="2020-03-18T03:05:39.013" LastActivityDate="2023-04-02T09:07:35.877" Title="Comparing phylogenetic models with different datasets" Tags="|phylogenetics|statistics|phylogeny|modelling|beast|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11666" PostTypeId="1" CreationDate="2020-03-16T10:28:03.303" Score="-2" ViewCount="2072" Body="&lt;p&gt;I want to install &lt;code&gt;Signature Tools Lib R package&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I typed&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; devtools::install()&#xA;Error: Could not find package root, is your working directory inside a package?&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Packages does need this list of sotware&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;R package dependencies:&#xA;    VariantAnnotation,&#xA;    BSgenome.Hsapiens.UCSC.hg38,&#xA;    BSgenome.Hsapiens.1000genomes.hs37d5,&#xA;    BSgenome.Mmusculus.UCSC.mm10,&#xA;    BSgenome.Cfamiliaris.UCSC.canFam3,&#xA;    SummarizedExperiment,&#xA;    BiocGenerics,&#xA;    GenomeInfoDb,&#xA;    NMF,&#xA;    foreach,&#xA;    doParallel,&#xA;    doMC,&#xA;    lpSolve,&#xA;    ggplot2,&#xA;    methods,&#xA;    cluster,&#xA;    stats,&#xA;    NNLM,&#xA;    nnls,&#xA;    GenSA,&#xA;    gmp,&#xA;    plyr,&#xA;    RCircos,&#xA;    scales,&#xA;    GenomicRanges,&#xA;    IRanges,&#xA;    BSgenome&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have installed all of these one by one but again when trying things like the below, I get error&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in tabToSNVcatalogue(subs = tmpSNVtab, genome.v = &quot;hg19&quot;) : &#xA;  could not find function &quot;tabToSNVcatalogue&quot;&#xA;&amp;gt; devtools::test()&#xA;Error: Could not find package root, is your working directory inside a package?&#xA;&amp;gt; &#xA;&amp;gt; library(signature.tools.lib)&#xA;Error in library(signature.tools.lib) : &#xA;  there is no package called ‘signature.tools.lib’&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-03-16T11:00:58.043" LastActivityDate="2020-03-16T12:23:58.857" Title="Error in ruuning a package in R" Tags="|r|" AnswerCount="1" CommentCount="1" ClosedDate="2020-03-16T21:12:37.820" ContentLicense="CC BY-SA 4.0" />
  <row Id="11667" PostTypeId="2" ParentId="11638" CreationDate="2020-03-16T10:44:10.953" Score="2" Body="&lt;p&gt;Each of these methods do something different, so it is reasonable to expect different results. The bottom line is that there isn't a single question you can ask when you do an &amp;quot;enrichment test&amp;quot;. For example: you can test if a set of genes are more expressed than this other list of genes or they are above the mean levels of any other gene.&lt;/p&gt;&#xA;&lt;p&gt;Even more, you use the gene ontology, which has an underlying structure (directed acyclic graph) that no method you used takes into account but could modify the results if taken into account. However, the KEGG pathways do not have this structure. So when you want to apply a method mixing both data sources you need to decide if you want to use it or not.&lt;/p&gt;&#xA;&lt;p&gt;This is also complicated because for most of these methods you can only compare the p-value which depends on the underlying assumptions of the methods (uniform, normal, binomial distribution?)&lt;/p&gt;&#xA;&lt;p&gt;The latest comprehensive study comparing enrichment methods including some of the methods you used and more is this one (that I know of): &lt;a href=&quot;https://doi.org/10.1093/bib/bbz158&quot; rel=&quot;nofollow noreferrer&quot;&gt;Toward a gold standard for benchmarking gene set enrichment analysis&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Last, what would a correlation tell you here? It is know that there is little agreement between methods. You couldn't say that one method is equal to the other unless you tested with several datasets and you had always a correspondence on the p-value.&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2020-11-04T16:19:31.187" LastActivityDate="2020-11-04T16:19:31.187" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11668" PostTypeId="2" ParentId="11666" CreationDate="2020-03-16T12:23:58.857" Score="4" Body="&lt;p&gt;If you do &lt;code&gt;?devtools::install&lt;/code&gt;, you will see that the &lt;code&gt;install()&lt;/code&gt; function from the &lt;code&gt;devtools&lt;/code&gt; package, not surprisingly, requires a package to be specified:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;install(pkg = &quot;.&quot;, reload = TRUE, quick = FALSE, build = !quick,&#xA;  args = getOption(&quot;devtools.install.args&quot;), quiet = FALSE,&#xA;  dependencies = NA, upgrade = &quot;ask&quot;, build_vignettes = FALSE,&#xA;  keep_source = getOption(&quot;keep.source.pkgs&quot;), force = FALSE, ...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So it is absolutely normal that &lt;code&gt;devtools::install()&lt;/code&gt; does not install anything. Just specify the packages(s) with the &lt;code&gt;pkg&lt;/code&gt; argument.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And this is how you install &lt;code&gt;Signature Tools&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;install.packages(&quot;remotes&quot;)&#xA;remotes::install_github(&quot;Nik-Zainal-Group/signature.tools.lib&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-03-16T12:23:58.857" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11669" PostTypeId="2" ParentId="11665" CreationDate="2020-03-16T13:32:24.030" Score="1" Body="&lt;p&gt;The key issue is the data is subject to reticulation, languages are seldom clean breaks. In nucleotide data this can be identified due to deviations from physical linkage and call it recombination. Linkage doesn't apply to languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fundamental assumption is one tree - the same tree - describes all the data, with reticulation that may not be a good assumption. Thus English is a hybrid language, and lots of English words find their way back into French and German 'parents'. Or perhaps a better example is Gaelic mingles with English over time. There are methods used to monitor reticulation in languages.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus id keep it simple. The branch length is not only directly comparable it provides linguistical information that is useful to understand. This is the amount of 'evolutionaey' change per word (it that is what you binary data is describing) and the relationship to the node ('breakpoiint') is of primary interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second step is to simply draw a tree of data set A and compare it to data set A+B and data set B and the probabilities describing the sister groups. Are the trees congruent, i.e. are they statistically the same?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The area you describing is more given to parametric bootstrapping, using custom resampling, but you need to be certain that the model being used is appropriate for the data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The final point is whether a reversible or directional matrix is being used, which impacts on the specific result.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-03-17T13:43:09.193" LastActivityDate="2020-03-17T13:43:09.193" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11670" PostTypeId="2" ParentId="11664" CreationDate="2020-03-16T13:44:46.417" Score="3" Body="&lt;p&gt;You are correct, it is the score of the candidate protein against the BUSCO profile hmm assessed by hmmer. If it passes a threshold, it is validated as being that BUSCO gene. The threshold is defined for each profile based on the score of the species used to produce the profile. It can be found in &lt;code&gt;name_odb10/scores_cutoff&lt;/code&gt;. And yes, the higher the better.&lt;/p&gt;&#xA;" OwnerUserId="7413" LastEditorUserId="3541" LastEditDate="2020-03-16T14:01:25.773" LastActivityDate="2020-03-16T14:01:25.773" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11671" PostTypeId="2" ParentId="11627" CreationDate="2020-03-17T14:17:50.473" Score="0" Body="&lt;p&gt;After much research I found out how to obtain the annotations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You must go to the REMC website: &lt;a href=&quot;https://egg2.wustl.edu/roadmap/web_portal/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://egg2.wustl.edu/roadmap/web_portal/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Go to the chromatin-state-learning tab, and choose the n-chromatin state model desired - the core has 15 states, then 18, then 25 (imputed states). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following this, you must go to the download URL link and, you will find a lot of files, primarily with the following coding information: E0*.. This corresponds to a particular tissue/cell type and so you must choose the one relevant to your research question. The way they are correspond to each tissue/cell type can be found in the following paper:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/s41467-019-11874-7#Sec22&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41467-019-11874-7#Sec22&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="4001" LastEditDate="2020-03-17T15:20:12.513" LastActivityDate="2020-03-17T15:20:12.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11672" PostTypeId="2" ParentId="11042" CreationDate="2020-03-17T16:03:41.023" Score="1" Body="&lt;p&gt;Actually I started a mini-review about the tools to do such conversion.&lt;br&gt;&#xA;You can find it here: &#xA;&lt;a href=&quot;https://github.com/NBISweden/GAAS/blob/master/annotation/knowledge/gff_to_gtf.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/NBISweden/GAAS/blob/master/annotation/knowledge/gff_to_gtf.md&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I mentioned earlier they do not behave all the same way...&lt;/p&gt;&#xA;" OwnerUserId="2486" LastActivityDate="2020-03-17T16:03:41.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11673" PostTypeId="1" CreationDate="2020-03-17T16:35:52.027" Score="1" ViewCount="248" Body="&lt;p&gt;I've been using MEGAHIT to assemble metagenomes, with particular focus on specific genomic areas.&lt;/p&gt;&#xA;&lt;p&gt;Sometimes all I get is gene fragments or pathway fragments (eg. if I know that genes A, B, C, D and E should be together, I only get A, B, C in one contig, and maybe D and E in another). That is understandable when the sequencing depth is low, and I'm assume that it's the best I can get.&lt;/p&gt;&#xA;&lt;p&gt;However. In two of my metagenomes, the genes of interest seem to be abundant (depth of 100-300+ listed for MEGAHIT contigs). And I still have the same problem. I assume that these 'breaks' between contigs are due to natural sequence variation which breaks the assembly process. (while sequencing errors are also possible, I've used bbduk to clean my data prior to assembly)&lt;/p&gt;&#xA;&lt;p&gt;I'd like to have larger contigs because I'm curious about the gene order. Things I've tried (did not work): visualizing assembly graphs with Bandage around Blast hits, starting with a smaller k-min (21 as opposed to 27).&lt;/p&gt;&#xA;&lt;p&gt;So I guess I'm wondering if using even smaller k-mers might make a difference. Or if someone has another suggestion, I'm happy to listen! Thank you.&lt;/p&gt;&#xA;&lt;p&gt;Megahit setting used:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;megahit -r input_files.fastq --num-cpu-threads 32 --min-contig-len 300 --presets meta-large -o output&#xA;megahit -r input_files.fastq --num-cpu-threads 32 --min-contig-len 1500 --presets meta-sensitive -o output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Meta-large starts from minimum kmer size 27; meta-sensitive from 21.&lt;/p&gt;&#xA;" OwnerUserId="4401" LastEditorUserId="57" LastEditDate="2021-08-09T14:54:53.063" LastActivityDate="2024-01-13T23:01:20.163" Title="Will using smaller kmers help get larger contigs? If not, then what?" Tags="|gene|assembly|pathway|k-mer|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11674" PostTypeId="1" CreationDate="2020-03-17T21:45:35.147" Score="2" ViewCount="3049" Body="&lt;p&gt;I want to merge/concatenate BLAST databases as suggested &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/7386/merging-concatentating-blast-databases&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is my example script that explains what I'm trying to achieve better than words can:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;for file in FAM1079.ffn FAM3228.ffn FAM6161.ffn FAM19036.ffn&#xA;do&#xA;    echo &quot;## Make DB for &lt;span class=&quot;math-container&quot;&gt;$file&quot;&#xA;	makeblastdb -in $&lt;/span&gt;file -dbtype nucl &amp;amp;&amp;gt;/dev/null&#xA;    blastn -db $file  -query query_single_nucl.fasta -outfmt 6&#xA;    echo&#xA;done&#xA;&#xA;echo &quot;## Create testdb.nal...&quot;&#xA;blastdb_aliastool -dblist &quot;FAM1079.ffn FAM3228.ffn FAM6161.ffn FAM19036.ffn&quot; -dbtype nucl -title test -out testdb&#xA;&#xA;echo&#xA;echo &quot;## Test testdb.nal...&quot;&#xA;blastn -db testdb.nal -query query_single_nucl.fasta -outfmt 6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;## Make DB for FAM1079.ffn&#xA;test_query  FAM1079_00001   100.000 1266    0   0   1   1266    1   1266    0.0 2338&#xA;&#xA;## Make DB for FAM3228.ffn&#xA;test_query  FAM3228_000001  100.000 1266    0   0   1   1266    1   1266    0.0 2338&#xA;&#xA;## Make DB for FAM6161.ffn&#xA;test_query  FAM6161_000001  100.000 1266    0   0   1   1266    1   1266    0.0 2338&#xA;&#xA;## Make DB for FAM19036.ffn&#xA;test_query  FAM19036p_000001    100.000 1266    0   0   1   1266    1   1266    0.0 2338&#xA;&#xA;## Create testdb.nal...&#xA;Created nucleotide BLAST (alias) database testdb with 12692 sequences&#xA;&#xA;## Test testdb.nal...&#xA;BLAST Database error: No alias or index file found for nucleotide database [testdb.nal] in search path [/path/to/files::]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;While the individual databases work, the concatenated databases don't.&lt;/p&gt;&#xA;" OwnerUserId="3628" LastActivityDate="2020-03-18T09:32:53.393" Title="Cannot blast database created with blastdb_aliastool: BLAST Database error: No alias or index file found" Tags="|blast|makeblastdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11675" PostTypeId="1" CreationDate="2020-03-18T09:11:36.223" Score="2" ViewCount="122" Body="&lt;p&gt;I am looking for a program/GUI app/package that would help me do simulations for cell growth. I have a microscopic mathematical model, and the scenario is basically the following: I start with a rod-shaped cell characterized my its center and direction angle, it grows until it reaches a certain length, it divides into two identical cells whose angles change randomly and the process continues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any program or package that I can use for such a situation? I have found several on GitHub, but they were mostly for very complicated biological processes. I use Python/Julia/Matlab and can code, but I was wondering whether there are any tools for such simpler scenarios.&lt;/p&gt;&#xA;" OwnerUserId="7425" LastActivityDate="2020-03-19T11:07:00.953" Title="Tools for modelling and visualizing growth of cells" Tags="|python|modelling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11676" PostTypeId="2" ParentId="11674" CreationDate="2020-03-18T09:32:53.393" Score="2" Body="&lt;p&gt;The problem was that the database must be specified without suffix, i.e. &lt;code&gt;blastn -db testdb&lt;/code&gt; as opposed to &lt;code&gt;blastn -db testdb.nal&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way, it produces exactly the same output as&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;blastn -db &quot;FAM1079.ffn FAM3228.ffn FAM6161.ffn FAM19036.ffn&quot; -query query_single_nucl.fasta -outfmt 6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and the performance is not better.&lt;/p&gt;&#xA;" OwnerUserId="3628" LastActivityDate="2020-03-18T09:32:53.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11677" PostTypeId="1" CreationDate="2020-03-18T16:06:15.787" Score="1" ViewCount="72" Body="&lt;p&gt;I model the following events: a rod-shaped cell with center &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; grows symmetrically until it reaches a maximum length/age, it divides into two identical cells, and the process continues. At division, the direction angle &lt;span class=&quot;math-container&quot;&gt;$\varphi$&lt;/span&gt; of the daughter cells changes randomly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the growth rate is &lt;span class=&quot;math-container&quot;&gt;$l=\frac{l_{max}}{2}+\frac{a}{a_{max}}\frac{l_{max}}{2}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any way of naively visualizing this? I found different cell modellers on GitHub but they were very complicated biologically speaking. I have some basic Python knowledge, but I have never done simulations. &lt;/p&gt;&#xA;" OwnerUserId="7426" LastActivityDate="2020-04-18T21:59:05.157" Title="Visualizing cell growth" Tags="|python|modelling|bacteria|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11678" PostTypeId="1" CreationDate="2020-03-18T18:51:35.000" Score="1" ViewCount="79" Body="&lt;p&gt;I was trying to download some data from NCBI SRA &lt;strong&gt;(SRA059451)&lt;/strong&gt;. There are 27 samples available for &lt;strong&gt;SRA059451&lt;/strong&gt;. But i am unable to understand which samples are &lt;strong&gt;'control'&lt;/strong&gt; and &lt;strong&gt;'test'&lt;/strong&gt; samples. Please help me regarding how to differentiate the samples.&lt;/p&gt;&#xA;" OwnerUserId="7427" LastActivityDate="2020-03-19T16:24:49.283" Title="NCBI SRA database sample: control vs test" Tags="|sequence-analysis|phylogenetics|metagenome|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11680" PostTypeId="1" CreationDate="2020-03-19T00:19:21.523" Score="1" ViewCount="47" Body="&lt;p&gt;Could someone please confirm that I understand this correctly? I have the res and qual columns from mpileup and I would like to match them to get the qual per base. It seems that the indel initiation (+2tt) has no qual so if I remove that I can get the phred with pythons ord().&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;res_with_indel = 'T*,t**,***T**t***,+2tt**,TT,.**.***.,'&#xA;res_without_indel= 'T*,t**,***T**t***,**,TT,.**.***.,'&#xA;qual = &quot;:'21(:,838&amp;gt;&amp;lt;*'915%2-;=@=99'*9'A$(&quot;&#xA;len(res_with_indel), len(res_without_indel), len(qual)&#xA;&#xA;(37, 33, 33)&#xA;&#xA;for base, quality in zip(res_without_indel, qual):&#xA;    print(f'{base} : {ord(quality)}')&#xA;&#xA;T : 58&#xA;* : 39&#xA;, : 50&#xA;t : 49&#xA;*  : 40&#xA;* : 58&#xA;, : 44&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To get the indel qualities then I need to match up each of the *'s across their respective indels lengths?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a library that does that? Biopython maybe?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2020-03-19T00:19:21.523" Title="Indel quality from mpileup" Tags="|python|samtools|indel|mpileup|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11681" PostTypeId="2" ParentId="11675" CreationDate="2020-03-19T11:07:00.953" Score="1" Body="&lt;p&gt;If the scenario is simpler, and you can't find anything by searching, I would consider just writing your own.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The modelling can be done in any language but if compute time is an issue - for example you want to run a simulation with many cells or many simulations - then I would use a language that can support that such as Julia or NumPy in Python. In fact Julia is a nice language for this type of thing as you end up with readable code that runs fast.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For visualisation in Julia specifically you could use the animation functionality of &lt;a href=&quot;http://docs.juliaplots.org/latest/animations&quot; rel=&quot;nofollow noreferrer&quot;&gt;Plots.jl&lt;/a&gt;, which is basic but there, or could try &lt;a href=&quot;http://makie.juliaplots.org/dev/animation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Makie.jl&lt;/a&gt;. See the &lt;a href=&quot;https://jgreener64.github.io/Molly.jl/dev/docs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Molly.jl documentation&lt;/a&gt; for some examples of visualising particle-type processes using Plots.jl.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2020-03-19T11:07:00.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11682" PostTypeId="2" ParentId="11678" CreationDate="2020-03-19T16:24:49.283" Score="3" Body="&lt;p&gt;You can download the information using this &lt;a href=&quot;https://www.ebi.ac.uk/ena/data/warehouse/filereport?accession=SRA059451&amp;amp;result=read_run&amp;amp;fields=run_accession,experiment_title,sra_ftp,sample_alias&amp;amp;download=txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did it in R:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tab = read.delim(&quot;https://www.ebi.ac.uk/ena/data/warehouse/filereport?accession=SRA059451&amp;amp;result=read_run&amp;amp;fields=run_accession,experiment_title,sra_ftp,sample_alias&quot;)&#xA;&#xA;head(tab)&#xA;  run_accession&#xA;1     SRR585697&#xA;2     SRR585700&#xA;3     SRR585726&#xA;4     SRR585727&#xA;5     SRR585728&#xA;6     SRR585729&#xA;                                                                experiment_title&#xA;1 Illumina HiSeq 2000 sequencing; Symptomatic atherosclerosis gut metagenome 171&#xA;2 Illumina HiSeq 2000 sequencing; Symptomatic atherosclerosis gut metagenome 238&#xA;3 Illumina HiSeq 2000 sequencing; Symptomatic atherosclerosis gut metagenome 241&#xA;4 Illumina HiSeq 2000 sequencing; Symptomatic atherosclerosis gut metagenome 242&#xA;5 Illumina HiSeq 2000 sequencing; Symptomatic atherosclerosis gut metagenome 243&#xA;6 Illumina HiSeq 2000 sequencing; Symptomatic atherosclerosis gut metagenome 245&#xA;                                      sra_ftp&#xA;1 ftp.sra.ebi.ac.uk/vol1/srr/SRR585/SRR585697&#xA;2 ftp.sra.ebi.ac.uk/vol1/srr/SRR585/SRR585700&#xA;3 ftp.sra.ebi.ac.uk/vol1/srr/SRR585/SRR585726&#xA;4 ftp.sra.ebi.ac.uk/vol1/srr/SRR585/SRR585727&#xA;5 ftp.sra.ebi.ac.uk/vol1/srr/SRR585/SRR585728&#xA;6 ftp.sra.ebi.ac.uk/vol1/srr/SRR585/SRR585729&#xA;                                    sample_alias&#xA;1     Symptomatic atherosclerosis gut metagenome&#xA;2 Symptomatic atherosclerosis gut metagenome 238&#xA;3 Symptomatic atherosclerosis gut metagenome 241&#xA;4 Symptomatic atherosclerosis gut metagenome 242&#xA;5 Symptomatic atherosclerosis gut metagenome 243&#xA;6 Symptomatic atherosclerosis gut metagenome 245&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Normally the information will be here, but I cannot find what does metagenome 238 etc refers to. No table in the paper that can link sample to phenotype.. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Most likely you need to contact the authors. This list might be useful though.&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-03-19T16:24:49.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11683" PostTypeId="1" CreationDate="2020-03-19T18:28:20.040" Score="1" ViewCount="396" Body="&lt;p&gt;I have a PCA plot from DESeq2's &lt;code&gt;plotPCA(vsd, intgroup=c(&quot;conditions&quot;))&lt;/code&gt; function. I have 9 samples distributed in to 3 groups of 3 biological replicates each. My reason for trying to visualise all on the same PCA plot is to see that samples from the same group (condition) cluster together and to see a separation between the groups. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The PCA plot I have is does not appear as I expect:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1-Can I just remove a sample which I deem is the outlier and perform a reclustering of the rest on the PCA?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2-What parameter can I use to declare that a point is an outlier rather than an in-group Biological variation?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;3-I also wish to find out if it is allowed to plot more than two groups of sample at a time on a PCA plot.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for your kind hints. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YWIiS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YWIiS.png&quot; alt=&quot;PCA plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3968" LastEditorUserId="3968" LastEditDate="2020-03-20T06:30:03.117" LastActivityDate="2020-03-23T18:49:55.090" Title="How do I differentiate outliers from in-group variations in a DESeq2 PCA plot of 9 samples distributed into 3 conditions?" Tags="|rna-seq|deseq2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11684" PostTypeId="1" AcceptedAnswerId="11709" CreationDate="2020-03-19T19:30:44.797" Score="1" ViewCount="108" Body="&lt;p&gt;I have a single-cell RNASeq sample, in which I'd like to identify latent variables (e.g. response to stress) that I think might be affecting the clustering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The approach I was planning to use is to reduce the dimensionality of the dataset using one algorithm among:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;PCA&lt;/li&gt;&#xA;&lt;li&gt;ICA&lt;/li&gt;&#xA;&lt;li&gt;NMF&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Then identify the most important biological processes for every component (e.g. in each Principal Component), and then finally regress out the processes that might not be relevant to the analysis or that might affect the downstream clustering.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a few questions:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Are there issues in this approach?&lt;/li&gt;&#xA;&lt;li&gt;Is there a collection of gene sets (similar to MSigDB) that can be used to identify noise latent variables?&lt;/li&gt;&#xA;&lt;li&gt;Which algorithm (PCA, ICA, NMF, or others) is more suited to this kind of analysis?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2020-03-23T19:09:27.177" Title="How to identify latent variables in single-cell RNA-Seq data" Tags="|scrnaseq|clustering|pca|gene-ontology|noise|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="11685" PostTypeId="1" CreationDate="2020-03-19T21:40:43.117" Score="-1" ViewCount="352" Body="&lt;p&gt;On Macconkey agar, lactose fermenting bacteria turn the agar bright red. Why does non-lactose fermenting bacteria turn the MaCconkey agar yellow?. Does it have to do with the alkalinity of waste products? &lt;/p&gt;&#xA;" OwnerUserId="7434" LastActivityDate="2020-03-20T10:49:05.730" Title="MaCconkey Agar colour change in non-lactose fermentors" Tags="|microbiology|" AnswerCount="1" CommentCount="1" ClosedDate="2020-03-20T15:24:22.510" ContentLicense="CC BY-SA 4.0" />
  <row Id="11686" PostTypeId="1" CreationDate="2020-03-20T08:20:08.387" Score="2" ViewCount="75" Body="&lt;p&gt;I can understand two-color and one-color(Channel) gene chip; but there is another type of gene-Chip called &lt;a href=&quot;https://www.rdocumentation.org/packages/Biobase/versions/2.32.0/topics/NChannelSet-class&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;N-channel chips&quot;&lt;/a&gt; ; I do not know how can several (&gt; 2) samples be hybridize in one chip. I cannot find some reference for the a material for this concept. See the following code that it has two channels &quot;E“ and &quot;Eb&quot;; what does that channel mean? The expression matrices seem to have the same dimension. In my understanding, the two-color or two-channel microarray is to combine two samples (treated vs. control) in a chip, and the log(ratio) is used to express the differential expression.   &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AE1548 = getAE(&quot;E-MTAB-1548&quot;, &#xA;               path=&quot;/Users/zhang/Documents/2020/GEOsepsis/Data&quot;,&#xA;               local = TRUE, sourcedir = &quot;/Users/zhang/Documents/2020/GEOsepsis/Data&quot;,&#xA;               type = 'full')&#xA;AE1548raw = ae2bioc(mageFiles = AE1548)#established from raw data; multi-channel data&#xA;AE1548expr1 = assayDataElement(AE1548raw,&quot;E&quot;)&#xA;AE1548expr2 = assayDataElement(AE1548raw,&quot;Eb&quot;)&#xA;  AE1548expr1[1:5,1:3]&#xA;  SIRS_18exp2 SIRS_19exp2 SIRS_23exp1&#xA;1    273812.0     97733.5      205330&#xA;2        41.0        26.5          50&#xA;3        42.5        25.0          48&#xA;4        40.0        23.5          49&#xA;5        38.5        24.0          53&#xA;  AE1548expr2[1:5,1:3]&#xA;  SIRS_18exp2 SIRS_19exp2 SIRS_23exp1&#xA;1          25        18.0          36&#xA;2          25        18.0          37&#xA;3          25        18.0          38&#xA;4          25        19.0          37&#xA;5          24        18.5          37&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7435" LastEditorUserId="7435" LastEditDate="2020-03-22T22:57:54.990" LastActivityDate="2024-04-01T23:02:40.403" Title="what is the concept of N-channel geneChip?" Tags="|gene|chip-seq|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11687" PostTypeId="2" ParentId="11685" CreationDate="2020-03-20T10:01:38.547" Score="1" Body="&lt;p&gt;McConkey agar is already red. It's trick is that it has two carbon sources. The bacteria that cannot use the more energetically useful lactose use the amino acids (peptone) generating ammonia which changes the pH and the crystal violet goes yellow. The ones that can use lactose acidify the agar a bit so go a bit more red —but not much.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This being the SE for bioinformatics, it should be said that if the genome of the organism in question is sequenced and deposited in NCBI you can find if the organism possesses the enzyme that allows the hydrolysis of lactose (beta-galactosidase) by doing &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PAGE=Proteins&amp;amp;PROGRAM=blastp&amp;amp;BLAST_PROGRAMS=blastp&amp;amp;QUERY=NP_414878.1&amp;amp;LINK_LOC=protein&amp;amp;PAGE_TYPE=BlastSearch&quot; rel=&quot;nofollow noreferrer&quot;&gt;a protein BLAST&lt;/a&gt;. The lacZ gene encoded protein in E. coli is NP_414878 in NCBI. This goes in the accession number input box. The species in question goes in the organism box. If you get something with good coverage (80%) with more than 20% homology the organism is likely lac+.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-03-20T10:49:05.730" LastActivityDate="2020-03-20T10:49:05.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11688" PostTypeId="1" AcceptedAnswerId="11689" CreationDate="2020-03-21T00:53:35.063" Score="1" ViewCount="990" Body="&lt;p&gt;I am trying to find sequence homology between viral sequences and my protein of interest. I have the sequences of their epitopes which varies from 5 to 500 amino acids long. For shorter sequences, it is recommended to use &lt;code&gt;blastp-short&lt;/code&gt; but I am confused until which length it is okay to run &lt;code&gt;blastp&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For shorter sequences, what are the parameters needs to be added for effective results?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;blastp -task blastp-short -query POI.fasta -db virus.fasta -out output.txt -outfmt 6 -evalue 20000&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="6471" LastEditorUserId="3848" LastEditDate="2020-12-11T00:22:56.777" LastActivityDate="2020-12-11T00:22:56.777" Title="How to do BLASTP for short sequences (&lt;20 aa) effectively?" Tags="|sequence-alignment|blast|covid-19|sequence-homology|blastp|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11689" PostTypeId="2" ParentId="11688" CreationDate="2020-03-21T02:27:26.683" Score="1" Body="&lt;p&gt;You should increase the &lt;code&gt;-evalue 10&lt;/code&gt; default option to a higher value because this value it's adjusted to the sequence database size. Check this table, maybe this can help you to decide which parameters are the best option for you.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    OPTION      TASK          DEFAUL    DESCRIPTION&#xA;    word_size   blastp        3         Word size of initial match.&#xA;    word_size   blastp-short  2         Word size of initial match.&#xA;    gapopen     blastp        1         Cost to open a gap.&#xA;    gapopen     blastp-short  9         Cost to open a gap.&#xA;    gapextend   blastp        1         Cost to extend a gap.&#xA;    gapextend   blastp-short  1         Cost to extend a gap.&#xA;    matrix      blastp        BLOSUM62  Scoring matrix name.&#xA;    matrix      blastp-short  PAM30     Scoring matrix name.&#xA;    threshold   blastp        11        Minimum score to add a word to the BLAST lookup table.&#xA;    threshold   blastp-short  16        Minimum score to add a word to the BLAST lookup table.&#xA;    window_size blastp        40        Multiple hits window size, use 0 to specify 1-hit algorithm.&#xA;    window_size blastp-short  15        Multiple hits window size, use 0 to specify 1-hit algorithm.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2020-03-21T02:27:26.683" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11690" PostTypeId="1" CreationDate="2020-03-21T02:50:11.157" Score="1" ViewCount="79" Body="&lt;p&gt;I have mutational catalogues of 4 samples like below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(out)&#xA;        Responders1 Nonresponders1 Responders2 Nonresponders2&#xA;A[C&amp;gt;A]A       11744          13546        2897           3655&#xA;A[C&amp;gt;A]C        5144           7172        2295           2163&#xA;A[C&amp;gt;A]G         939           1257         290            279&#xA;A[C&amp;gt;A]T        6065           7997        2078           2088&#xA;C[C&amp;gt;A]A        8969          11155        2055           2582&#xA;C[C&amp;gt;A]C        4050           6657        1367           1173&#xA;&amp;gt;&#xA;&#xA; &amp;gt; dim(mut)&#xA;[1] 96  4&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;Note: A[C&amp;gt;A]A - Single nucleotide base substitution&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want to look if mutational substitutions are different between samples&lt;/p&gt;&#xA;&lt;p&gt;I have tried&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;wilcox.test(mut&lt;span class=&quot;math-container&quot;&gt;$Responders2,mut$&lt;/span&gt;Nonresponders2)&#xA;&#xA;t.test(mut&lt;span class=&quot;math-container&quot;&gt;$Responders2,mut$&lt;/span&gt;Nonresponders2)&#xA;&#xA;var.test(mut&lt;span class=&quot;math-container&quot;&gt;$Responders2,mut$&lt;/span&gt;Nonresponders2)&#xA;fisher.test(mut&lt;span class=&quot;math-container&quot;&gt;$Responders2,mut$&lt;/span&gt;Nonresponders2,simulate.p.value=TRUE)&#xA;&#xA;kruskal.test(mut&lt;span class=&quot;math-container&quot;&gt;$Responders2,mut$&lt;/span&gt;Nonresponders2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To find a difference but I think you may know a better test to find any difference here&lt;/p&gt;&#xA;&lt;p&gt;This is base shift (mutation, substitution across genome of each patient). I guess mutations happen randomly but I don't know if mutational caller algorithm also pick them by randomness. I expect the shift of A (Adenine) to C (Cytosine) in a group of these patients be higher than the other group. This higher rate likely should be due to some background than randomness picked by algorithm. So I think null hypothesis is that rate of base shifting is not the same in both group. I also don't expect a normalised distribution of base shifting rather a poison distribution.&lt;/p&gt;&#xA;&lt;p&gt;Can you help please?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="73" LastEditDate="2021-07-14T11:33:00.750" LastActivityDate="2023-08-03T17:01:20.177" Title="Finding difference between groups" Tags="|r|wgs|mutations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11691" PostTypeId="1" AcceptedAnswerId="11694" CreationDate="2020-03-21T10:21:13.677" Score="2" ViewCount="162" Body="&lt;p&gt;Is there a possibility, using biopython to save output table of blastn in memory, not in file on hard drive, for process it using pandas for example and then delete it from memory?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I mean, is there a possibility to redirect output of &quot;out&quot; argument of this function into memory? In pandas DataFrame for example.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;NcbiblastnCommandline(query=&quot;fasta1.fasta&quot;, subject=&quot;fasta2.fasta&quot;, out=&quot;output_table.csv&quot;, outfmt=6)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I always saved output tables in file before, but now I need to align one set of small fasta files against another set of small fasta files and I think that the best way is avoid writing of many small tables in hard drive because after recieving one table I have to read and process this table immediately and after that I don't need this table anymore.&lt;/p&gt;&#xA;" OwnerUserId="7439" LastEditorUserId="3848" LastEditDate="2020-03-22T11:29:13.437" LastActivityDate="2020-03-22T11:29:13.437" Title="Possibility to save output blastn table in memory using biopython" Tags="|python|blast|pandas|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11693" PostTypeId="1" CreationDate="2020-03-21T13:01:31.270" Score="-3" ViewCount="50" Body="&lt;p&gt;I have 48 files in which I have&lt;/p&gt;&#xA;&#xA;&lt;p&gt;r1&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Organ-specific cancer signature Contribution&#xA;Esophagus_A 2948.114&#xA;Esophagus_B 815.6483&#xA;Esophagus_C 1278.381&#xA;Esophagus_D 883.5358&#xA;Esophagus_E 1446.103&#xA;Esophagus_F 880.0416&#xA;Esophagus_G 1262.907&#xA;Esophagus_H 743.2682&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;r2&#xA;..&#xA;r15&lt;/p&gt;&#xA;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Non_res1&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Organ-specific cancer signature Contribution&#xA;Esophagus_A 2660.722&#xA;Esophagus_B 1471.017&#xA;Esophagus_C 1639.904&#xA;Esophagus_D 0&#xA;Esophagus_E 2580.775&#xA;Esophagus_F 946.3675&#xA;Esophagus_G 2013.654&#xA;Esophagus_H 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Non_res33&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want a unified file in column one is Organ-specific cancer and 48 more columns are the second column of each separate file&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How I can do that?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2020-03-24T08:28:58.913" Title="Joinning columns from different files" Tags="|r|bash|awk|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11694" PostTypeId="2" ParentId="11691" CreationDate="2020-03-21T14:10:30.410" Score="6" Body="&lt;p&gt;You can specify send to stdout using &lt;code&gt;out='-'&lt;/code&gt; in the Biopython wrapper.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.Blast.Applications import NcbiblastnCommandline&#xA;import pandas as pd&#xA;&#xA;cline = NcbiblastnCommandline(query='seq.fna', subject='seq2.fna', outfmt=6, out='-')&#xA;output = cline()[0].strip()&#xA;&#xA;rows = [line.split() for line in output.splitlines()]&#xA;cols = ['qseqid', 'sseqid', 'pident', 'length',&#xA;        'mismatch', 'gapopen', 'qstart', 'qend',&#xA;        'sstart', 'send', 'evalue', 'bitscore']&#xA;&#xA;data_types = {'pident': float, 'length': int, 'mismatch': int,&#xA;              'gapopen': int, 'qstart': int, 'qend': int,&#xA;              'sstart': int, 'send': int, 'evalue': float,&#xA;              'bitscore': float}&#xA;&#xA;df = pd.DataFrame(rows, columns=cols).astype(data_types)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2020-03-21T14:10:30.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11695" PostTypeId="1" CreationDate="2020-03-22T03:01:22.270" Score="2" ViewCount="863" Body="&lt;p&gt;I am trying to understand how to use &lt;code&gt;BuildClusterTree&lt;/code&gt; of Seurat to understand the relationship between clusters. Being from neither a bioinformatics or statistical background, I am not understating how to use it. I am have the following Clustertree plot. Can someone explain how to use this function to &lt;code&gt;merge cell type clusters&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tU92R.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tU92R.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4951" LastEditorUserId="3848" LastEditDate="2020-11-15T12:51:07.430" LastActivityDate="2020-11-15T12:51:07.430" Title="Understanding BuildClusterTree of Seurat" Tags="|seurat|single-cell|clustering|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11696" PostTypeId="1" AcceptedAnswerId="11702" CreationDate="2020-03-22T14:27:10.027" Score="0" ViewCount="471" Body="&lt;p&gt;I'm trying to simulate binary phenotypes from the &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/10731/simulating-phenotype-with-the-1000-genomes-project&quot;&gt;1000 Genome Phase 3 datasets&lt;/a&gt; using &lt;a href=&quot;https://www.biostars.org/p/428314/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gcta64 --simu-cc&lt;/a&gt;, but no success. &#xA;Everything seems to be going well, but in the end I get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error: can not open the file [] to read.&#xA;An error occurs, please check the options or data&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the log shows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Accepted options:&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's the commands I'm using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Convert the VCF to plink format:&#xA;$ ./plink2 --vcf ../../ALL.phase3.biallelic-only.vcf.gz.10kSNPs.vcf.gz --make-bed --out ALL.phase3.biallelic-only.vcf.gz.10kSNPs&#xA;&#xA;# Try to simulate the phenotype:&#xA;$ ./gcta64 --bfile ALL.phase3.biallelic-only.vcf.gz.10kSNPs --simu-cc 500 500 --simu-hsq 0.5 --simu-k 0.1 --simu-rep 3 --out ALL.phase3.biallelic-only.vcf.gz.10kSNPs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here's the whole steps with the outputs:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ls&#xA;gcta64  plink2&#xA;&#xA;$ ./plink2 --vcf ../../ALL.phase3.biallelic-only.vcf.gz.10kSNPs.vcf.gz --make-bed --out ALL.phase3.biallelic-only.vcf.gz.10kSNPs&#xA;PLINK v2.00a2.3 64-bit (24 Jan 2020)           www.cog-genomics.org/plink/2.0/&#xA;(C) 2005-2020 Shaun Purcell, Christopher Chang   GNU General Public License v3&#xA;Logging to ALL.phase3.biallelic-only.vcf.gz.10kSNPs.log.&#xA;Options in effect:&#xA;  --make-bed&#xA;  --out ALL.phase3.biallelic-only.vcf.gz.10kSNPs&#xA;  --vcf ../../ALL.phase3.biallelic-only.vcf.gz.10kSNPs.vcf.gz&#xA;&#xA;Start time: Sun Mar 22 11:56:23 2020&#xA;16384 MiB RAM detected; reserving 8192 MiB for main workspace.&#xA;Using up to 8 compute threads.&#xA;--vcf: 220000 variants scanned.&#xA;--vcf: ALL.phase3.biallelic-only.vcf.gz.10kSNPs-temporary.pgen +&#xA;ALL.phase3.biallelic-only.vcf.gz.10kSNPs-temporary.pvar +&#xA;ALL.phase3.biallelic-only.vcf.gz.10kSNPs-temporary.psam written.&#xA;2504 samples (0 females, 0 males, 2504 ambiguous; 2504 founders) loaded from&#xA;ALL.phase3.biallelic-only.vcf.gz.10kSNPs-temporary.psam.&#xA;220000 variants loaded from&#xA;ALL.phase3.biallelic-only.vcf.gz.10kSNPs-temporary.pvar.&#xA;Note: No phenotype data present.&#xA;Writing ALL.phase3.biallelic-only.vcf.gz.10kSNPs.fam ... done.&#xA;Writing ALL.phase3.biallelic-only.vcf.gz.10kSNPs.bim ... done.&#xA;Writing ALL.phase3.biallelic-only.vcf.gz.10kSNPs.bed ... done.&#xA;End time: Sun Mar 22 11:56:28 2020&#xA;&#xA;$ ls&#xA;ALL.phase3.biallelic-only.vcf.gz.10kSNPs.bed    ALL.phase3.biallelic-only.vcf.gz.10kSNPs.fam    gcta64&#xA;ALL.phase3.biallelic-only.vcf.gz.10kSNPs.bim    ALL.phase3.biallelic-only.vcf.gz.10kSNPs.log    plink2&#xA;&#xA;$ ./gcta64 --bfile ALL.phase3.biallelic-only.vcf.gz.10kSNPs --simu-cc 500 500 --simu-hsq 0.5 --simu-k 0.1 --simu-rep 3 --out ALL.phase3.biallelic-only.vcf.gz.10kSNPs&#xA;*******************************************************************&#xA;* Genome-wide Complex Trait Analysis (GCTA)&#xA;* version 1.93.0 beta Mac&#xA;* (C) 2010-2019, The University of Queensland&#xA;* Please report bugs to Jian Yang &amp;lt;jian.yang@uq.edu.au&amp;gt;&#xA;*******************************************************************&#xA;Analysis started at 11:59:16 GMT on Sun Mar 22 2020.&#xA;Hostname: Brunos-MBP&#xA;&#xA;Accepted options:&#xA;--bfile ALL.phase3.biallelic-only.vcf.gz.10kSNPs&#xA;--simu-cc 500 500&#xA;--simu-hsq 0.5&#xA;--simu-k 0.1&#xA;--simu-rep 3&#xA;--out ALL.phase3.biallelic-only.vcf.gz.10kSNPs&#xA;&#xA;&#xA;Reading PLINK FAM file from [ALL.phase3.biallelic-only.vcf.gz.10kSNPs.fam].&#xA;2504 individuals to be included from [ALL.phase3.biallelic-only.vcf.gz.10kSNPs.fam].&#xA;Reading PLINK BIM file from [ALL.phase3.biallelic-only.vcf.gz.10kSNPs.bim].&#xA;220000 SNPs to be included from [ALL.phase3.biallelic-only.vcf.gz.10kSNPs.bim].&#xA;Warning: Duplicated SNP ID &quot;rs145607083&quot; has been changed to &quot;rs145607083_5264&quot;&#xA;.Warning: Duplicated SNP ID &quot;rs145607083&quot; has been changed to &quot;rs145607083_5265&quot;&#xA;.Warning: Duplicated SNP ID &quot;rs71955229&quot; has been changed to &quot;rs71955229_27061&quot;&#xA;.Warning: Duplicated SNP ID &quot;rs71955229&quot; has been changed to &quot;rs71955229_27062&quot;&#xA;.Warning: Duplicated SNP ID &quot;rs71589472&quot; has been changed to &quot;rs71589472_42505&quot;&#xA;.Warning: Duplicated SNP ID &quot;rs563156514&quot; has been changed to &quot;rs563156514_49111&quot;&#xA;.Warning: Duplicated SNP ID &quot;rs563156514&quot; has been changed to &quot;rs563156514_49112&quot;&#xA;.Warning: Duplicated SNP ID &quot;rs539504239&quot; has been changed to &quot;rs539504239_79196&quot;&#xA;.Warning: Duplicated SNP ID &quot;rs35739849&quot; has been changed to &quot;rs35739849_105514&quot;&#xA;.Warning: Duplicated SNP ID &quot;rs148795567&quot; has been changed to &quot;rs148795567_123134&quot;&#xA;.Warning: Duplicated SNP ID &quot;rs143101359&quot; has been changed to &quot;rs143101359_201815&quot;&#xA;.Reading PLINK BED file from [ALL.phase3.biallelic-only.vcf.gz.10kSNPs.bed] in SNP-major format ...&#xA;Genotype data for 2504 individuals and 220000 SNPs to be included from [ALL.phase3.biallelic-only.vcf.gz.10kSNPs.bed].&#xA;Simulation parameters:&#xA;Number of simulation replicate(s) = 3 (Default = 1)&#xA;Heritability of liability = 0.5 (Default = 0.1)&#xA;Disease prevalence = 0.1 (Default = 0.1)&#xA;Number of cases = 500&#xA;Number of controls = 500&#xA;&#xA;Error: can not open the file [] to read.&#xA;An error occurs, please check the options or data&#xA;&#xA;$ head ALL.phase3.biallelic-only.vcf.gz.10kSNPs.log &#xA;*******************************************************************&#xA;* Genome-wide Complex Trait Analysis (GCTA)&#xA;* version 1.93.0 beta Mac&#xA;* (C) 2010-2019, The University of Queensland&#xA;* Please report bugs to Jian Yang &amp;lt;jian.yang@uq.edu.au&amp;gt;&#xA;*******************************************************************&#xA;Analysis started at 11:59:16 GMT on Sun Mar 22 2020.&#xA;Hostname: Brunos-MBP&#xA;&#xA;Accepted options:&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm opened to use other tools if you have any recommendation. Not sure, but looks like Plink doesn't do that (you can simulate phenotype as long as you also simulate the genetic data as well...)&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;PS 1: I managed to simulate it before as answered &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/10731/simulating-phenotype-with-the-1000-genomes-project&quot;&gt;here&lt;/a&gt;, but I've just realised it doesn't generate reliable ratios (although the logic in place, phenotypes are still too aleatory assigned - too low precision, recall and f1 scores).&lt;/li&gt;&#xA;&lt;li&gt;PS 2: Also asked in the &lt;a href=&quot;https://www.biostars.org/p/428314/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioStars&lt;/a&gt; forum&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6464" LastEditorUserId="6464" LastEditDate="2020-03-22T14:45:11.513" LastActivityDate="2020-04-23T08:07:30.827" Title="How to simulate phenotype from real genetic data for GWAS purpose?" Tags="|plink|simulated-data|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11697" PostTypeId="1" CreationDate="2020-03-22T15:22:26.640" Score="1" ViewCount="71" Body="&lt;p&gt;Trying to download run data of Run accession &lt;strong&gt;SRR2155174&lt;/strong&gt;. Here's the link: &lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR2155174&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR2155174&lt;/a&gt;&#xA;Please go to the link and go to the &quot;Data Access&quot; tab. Hereunder header &quot;Name&quot; you can find two links:&#xA;&lt;a href=&quot;https://sra-downloadb.be-md.ncbi.nlm.nih.gov/sos1/sra-pub-run-5/SRR2155174/SRR2155174.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sra-downloadb.be-md.ncbi.nlm.nih.gov/sos1/sra-pub-run-5/SRR2155174/SRR2155174.1&lt;/a&gt;&#xA;&lt;a href=&quot;https://sra-downloadb.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-6/SRR2155174/SRR2155174.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sra-downloadb.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-6/SRR2155174/SRR2155174.1&lt;/a&gt;&#xA;&lt;strong&gt;Do these two links download the same data or they are for different purposes?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="7427" LastActivityDate="2020-03-22T15:53:02.190" Title="SRA run data access" Tags="|sequence-analysis|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11698" PostTypeId="2" ParentId="11697" CreationDate="2020-03-22T15:53:02.190" Score="2" Body="&lt;p&gt;It seems just like two different mirrors contains the same SRA archive. If you still doubt, you can download both (its have relatively small size) and compare their md5sum.&lt;/p&gt;&#xA;" OwnerUserId="7439" LastActivityDate="2020-03-22T15:53:02.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11700" PostTypeId="1" CreationDate="2020-03-23T05:45:05.627" Score="1" ViewCount="804" Body="&lt;p&gt;I have two resources of .bam files. One is generated by our lab (1 sample = 1 bam). One is downloaded online (again 1 sample = 1 bam). For the downloaded samples the chromosomes are labelled: chr1, chr2, chr3 etc For our lab samples, the chromosomes are labelled: 1, 2, 3 etc. I want to generate a single VCF file of variants across all samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using bcftools:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools mpileup -Ov -f ref.fasta -b samples.txt | bcftools call -mv -o bamMge.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, I get no calls and the repeated error: &lt;strong&gt;[E::faidx_adjust_position] The sequence &quot;1&quot; was not found&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two questions:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is my strategy correct (i.e. bcftools mpileup)? Do I need to incorporate extra steps (NB I also have matching gVCF files)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I either (i) alter the chromosome labeling of one of the subsets of bam files, or, (ii) use a mapping file to match chromosome labels during the mpileup run?&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="7443" LastEditorUserId="3848" LastEditDate="2020-03-23T08:47:36.210" LastActivityDate="2020-04-22T23:24:52.333" Title="Generate VCF from different .bam files with different chromosome names" Tags="|vcf|ngs|mpileup|bcftools|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11701" PostTypeId="1" AcceptedAnswerId="11705" CreationDate="2020-03-23T09:20:08.200" Score="1" ViewCount="505" Body="&lt;p&gt;I have a question regarding &quot;Per Base Sequence Content&quot; plot for &quot;fastqc&quot;:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the fastqc documentation, it is written: &quot;In a random library you would expect that there would be little to no difference between the different bases of a sequence run, so the lines in this plot should run parallel with each other.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I don't understand why different bases in a read should follow the same pattern of allele frequency (&quot;A/T/C/G&quot;). I mean they are different positions in the genome and it is normal that each position has different allele.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate it if someone could help me, please.&lt;/p&gt;&#xA;" OwnerUserId="7444" LastActivityDate="2020-03-23T14:04:19.337" Title="Per Base Sequence Content in fastqc" Tags="|ngs|genome|fastqc|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11702" PostTypeId="2" ParentId="11696" CreationDate="2020-03-23T11:13:22.623" Score="1" Body="&lt;p&gt;The question has been answered in the bioStars forum: &lt;a href=&quot;https://www.biostars.org/p/428314/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/428314/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It was missing the param &lt;code&gt;--simu-causal-loci&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I created the &lt;code&gt;causal.snplist&lt;/code&gt; file by finding out which SNP's are known causals of the treat I want to simulate.&#xA;In my case, I want to simulate a Diabetes phenotype, thus I:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Got the known Diabetes causal SNP's from &lt;a href=&quot;https://www.snpedia.com/index.php/Type-2_diabetes&quot; rel=&quot;nofollow noreferrer&quot;&gt;snpedia&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Filtered out whichcausal SNP's also exist in my genomic data:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat chr3_6_10.biallelic-only.bim | grep &quot;rs4655595\|rs6718526\|rs4402960...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Created the &lt;code&gt;causal.snplist&lt;/code&gt; with the causal SNP's as per &lt;a href=&quot;https://cnsgenomics.com/software/gcta/#GWASSimulation&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ head causal.snplist&#xA;rs1111875&#xA;rs12255372&#xA;rs1801282&#xA;rs237025&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;Run gcta-simu again:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ./gcta64 --bfile chr3_6_10.biallelic-only --simu-cc 500 500 --simu-causal-loci causal.snplist --simu-hsq 0.5 --simu-k 0.1 --simu-rep 3 --out chr3_6_10.biallelic-only.gcta-simu&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ol start=&quot;5&quot;&gt;&#xA;&lt;li&gt;Got the expected results:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ head chr3_6_10.biallelic-only.gcta-simu.phen &#xA;0 HG00096 1 1 1 &#xA;0 HG00100 1 1 1 &#xA;0 HG00101 2 1 2 &#xA;0 HG00102 1 1 2 &#xA;0 HG00103 1 1 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The last 3 columns are phenotypes. 3 phenotypes were generated due the param &lt;code&gt;--simu-rep 3&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="6464" LastEditorUserId="6464" LastEditDate="2020-03-24T08:01:13.890" LastActivityDate="2020-03-24T08:01:13.890" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11703" PostTypeId="1" CreationDate="2020-03-23T12:24:21.840" Score="1" ViewCount="137" Body="&lt;p&gt;I want to analyze some viral sequences. I want to use shanon entropy as a measure to analyze the sequences. However I do  not understand the concept of shannon entropy. Would anyone explain please how it works or advise on some resources i can use to understand shannon entropy.&lt;/p&gt;&#xA;" OwnerUserId="7446" LastEditorUserId="3848" LastEditDate="2023-08-04T19:08:44.217" LastActivityDate="2023-08-04T19:08:44.217" Title="How to compute Shanon entropy" Tags="|phylogenetics|phylogeny|statistics|genomics|population-genetics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11704" PostTypeId="2" ParentId="11703" CreationDate="2020-03-23T13:36:00.467" Score="2" Body="&lt;p&gt;I think there needs to be clarification of the question. Shannon entropy is not the same thing as the Shannon-Wiener diversity index (H), commonly referred to as the Shannon index.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason I brought it up is because you have the tag &quot;population genetics&quot;. Shannon diversity index is commonly used in population genetics to compare species richness (the number of species in a given area) and their relative abundance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Shannon entropy is more to do protein structure, which isn't really population genetics, but can relate to how stable a given mutation might be. That is not really population genetics however, population genetics is about abundance of a given allele in this context. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am pretty sure that on this site most responders will recommend alternative free energy approaches to perform this style of analysis.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-03-23T13:36:00.467" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11705" PostTypeId="2" ParentId="11701" CreationDate="2020-03-23T14:04:19.337" Score="2" Body="&lt;p&gt;The positions you see in the x-axis of this plot are positions on your &lt;em&gt;reads&lt;/em&gt;, not on the reference genome. Since each read comes from a random position on the genome, the frequency of A/T/C/G on each base position should reflect the base composition of the entire genome. Therefore, the percentage of each base should be approximately constant across the read.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope that helps!&lt;/p&gt;&#xA;" OwnerUserId="3389" LastActivityDate="2020-03-23T14:04:19.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11706" PostTypeId="2" ParentId="11686" CreationDate="2020-03-23T16:31:31.873" Score="0" Body="&lt;p&gt;&lt;strong&gt;This is a mere long comment, which may help, but I do not know the details required for the answer.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Several (&gt; 2) samples hybridized in one chip&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;A traditional microarray has two parts, the mobile target, which is the labelled sample, and fixed to the filter paper, glass or doped silicon is the probe, which is not labelled and is in vast excess. You can label the target fluorescently with one of a variety of fluorophores, or with different radioisotopes, which have different energies. Nobody uses radioisotope microscopy for this except for the most obscure method, say nuclear run off assays, but I mention this just to cover all bases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whereas you can buy whatever fluorophore you wish, the detecting machine needs to be set up for it with lasers, beam splitters and filters. Most florescent confocal microscopes use a LED lamp, a splitter and a loads of cubes of filters. In the case of flow cytometry, you can get over 16 fluorescent channels at the same time. However, deconvolution becomes necessary when one channel &quot;bleeds&quot; into an other —I am not sure how one deals with it with a microarray, I suspect before the step to remove regional biases (the step with LOESS regression or similar).&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Affymetrix Gene chip (R)&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Now, in the case of an Affymetrix Gene chip, I do not know if there is a multichannel machine, but it could be custom build. The spots on an Affymetrix chip are 3-5 µm, so regular microscopy optics still work. Although the silicon base is not transparent like a home made slide or an Agilent microarray (if these are still made). Hopefully, someone else can answer about Affymetrix machinery!&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Statistics&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;I am not familiar with why or how you would analyse a 3+ channel experiment. One guess could be to do a dose-response experiment. But there will be some problems:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The dye flip correction would be an issue for greater than 2 samples&lt;/li&gt;&#xA;&lt;li&gt;Channel deconvolution may be required for dyes with overlapping spectra.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-03-23T16:31:31.873" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11707" PostTypeId="2" ParentId="11683" CreationDate="2020-03-23T18:49:55.090" Score="1" Body="&lt;p&gt;Since you have so few points here, it is hard to tell which are outliers by just looking at the PCA plot.&#xA;I suggest you use correlation heatmap for your gene expression and check whether there are obvious outliers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for using more than one groups when plotting PCA, I think you can just try passing a vector with many groups to the argument &lt;code&gt;intgroup&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-03-23T18:49:55.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11708" PostTypeId="2" ParentId="11051" CreationDate="2020-03-23T18:52:37.947" Score="1" Body="&lt;p&gt;The largest eQTL datasets I can think of is probably &lt;a href=&quot;https://eqtlgen.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;eQTLGen&lt;/a&gt;. This is a meta-analysis of many studies using &gt;30,000 samples from whole blood. This might be less ideal if you want cell-type-specific QTL information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For other molecular QTLs, I do not think there are large-scale datasets yet.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastEditorUserId="4766" LastEditDate="2020-03-23T22:47:22.530" LastActivityDate="2020-03-23T22:47:22.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11709" PostTypeId="2" ParentId="11684" CreationDate="2020-03-23T19:09:27.177" Score="1" Body="&lt;p&gt;I think to identify latent variables, PCA is probably not going to work. NMF might be worth trying. You might want to check out a method called consensus NMF (cNMF) (&lt;a href=&quot;https://elifesciences.org/articles/43803&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://elifesciences.org/articles/43803&lt;/a&gt;) published in eLife. The author described some latent variables corresponds to cell types, while others corresponds to cell-cycle or metabolic states of cells.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-03-23T19:09:27.177" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11710" PostTypeId="2" ParentId="11150" CreationDate="2020-03-23T20:00:09.653" Score="2" Body="&lt;pre&gt;&lt;code&gt;pt &amp;lt;- table(Idents(myobj), myobj&lt;span class=&quot;math-container&quot;&gt;$orig.ident)&#xA;pt &amp;lt;- as.data.frame(pt)&#xA;pt$&lt;/span&gt;Var1 &amp;lt;- as.character(pt$Var1)&#xA;&#xA;ggplot(pt, aes(x = Var2, y = Freq, fill = Var1)) +&#xA;  theme_bw(base_size = 15) +&#xA;  geom_col(position = &quot;fill&quot;, width = 0.5) +&#xA;  xlab(&quot;Sample&quot;) +&#xA;  ylab(&quot;Proportion&quot;) +&#xA;  scale_fill_manual(values = brewer.pal(12, &quot;Paired&quot;)) +&#xA;  theme(legend.title = element_blank())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This shall work. &lt;code&gt;myobj&lt;/code&gt; is your seurat object. This is my old code so if I remember correct, &lt;code&gt;orig.ident&lt;/code&gt; is your treated and untreated conditions. &lt;code&gt;Idents&lt;/code&gt; should give you cell clusters.&#xA;If this does not work, please let me know&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-03-23T20:00:09.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11711" PostTypeId="1" CreationDate="2020-03-23T22:12:56.333" Score="1" ViewCount="48" Body="&lt;p&gt;I have some raw read count data from a crispr screen.&#xA;The library used was addgene's&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Mouse Improved Genome-wide Knockout CRISPR Library v2&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I'm following an analysis pipeline that requires counts from &lt;strong&gt;non target gRNA&lt;/strong&gt; in my raw read count file.&lt;/p&gt;&#xA;&lt;p&gt;But the gRNA library does not contain &lt;strong&gt;non target gRNA&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I'm wondering what genes are commonly used as phenotypic controls in CRISPR screens?&lt;/p&gt;&#xA;&lt;p&gt;In other words, what &lt;code&gt;Genes&lt;/code&gt;, when mutated, does not affect the cell's fitness, thus can be used as a reference when comparing gRNA counts?&lt;/p&gt;&#xA;" OwnerUserId="7056" LastEditorUserId="3848" LastEditDate="2020-11-18T20:52:48.393" LastActivityDate="2020-11-18T20:52:48.393" Title="CRISPR Screen Control - What genes do not affect phenotype when knocked out?" Tags="|crispr|genetics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11713" PostTypeId="2" ParentId="11700" CreationDate="2020-03-23T22:54:14.047" Score="1" Body="&lt;p&gt;I am not familiar with getting variants from bam files, so I will just answer the second part. From the error message, it seems that it is necessary to have consistent chromosome labels across datasets. It is also a good idea to do so because this keeps your datasets consistent and makes things a lot easier. To change the chromosomes labels, you can write your own scripts (I use Python, for example). You can also find several solutions on BioStars (&lt;a href=&quot;https://www.biostars.org/p/98582/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/98582/&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-03-23T22:54:14.047" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11715" PostTypeId="1" CreationDate="2020-03-24T06:33:47.600" Score="0" ViewCount="136" Body="&lt;p&gt;The terms in LME i.e. fixed and random create confusion? What is the genesis that can distinguish between the two ?&lt;/p&gt;&#xA;" OwnerUserId="7374" LastActivityDate="2020-04-02T08:55:39.513" Title="What is the difference between fixed effects and random effects in the context of Linear-mixed models?" Tags="|statistics|linear-regression|structural-biology|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="11716" PostTypeId="2" ParentId="11693" CreationDate="2020-03-24T08:28:58.913" Score="1" Body="&lt;p&gt;If the first column is always the same:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/a/33073145/680068&quot;&gt;R cbind in a loop&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;bash &lt;code&gt;paste file*.txt&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If the first columns in not same:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/q/8091303/680068&quot;&gt;R merge in a loop&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/q/13710876/680068&quot;&gt;bash Merge multiple files by common field&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="131" LastActivityDate="2020-03-24T08:28:58.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11717" PostTypeId="1" CreationDate="2020-03-24T10:20:30.017" Score="1" ViewCount="87" Body="&lt;p&gt;I am checking the quality control of my sequences using the Fastqc tool. For some steps, like &quot;Per base GC content&quot;, etc., I received a warning. So, I was wondering whether I Should also take care of warning messages in the Fastqc tool or I should only care about errors! &#xA;I would appreciate it if someone could help me with this, please.&lt;/p&gt;&#xA;" OwnerUserId="7444" LastActivityDate="2020-03-24T12:18:47.480" Title="Warning in fastqc" Tags="|ngs|genome|sequence-analysis|genome-sequencing|fastqc|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11720" PostTypeId="2" ParentId="11717" CreationDate="2020-03-24T12:18:47.480" Score="1" Body="&lt;p&gt;Generally yes, but it depends the type of study. Remember that Fastqc was developed for bacterial genomes, mainly, so an unusually shaped distribution in that particular dataset could indicate a contaminated library. But, for example, in an amplicon 16S amplicon dataset it's expected that other modules fail (Duplicate Sequences, Overrepresented Sequences, etc.) not only GC.&lt;/p&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2020-03-24T12:18:47.480" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11721" PostTypeId="1" CreationDate="2020-03-24T14:41:29.067" Score="21" ViewCount="5039" Body="&lt;p&gt;I'm confused about the difference between genome and DNA. Is it correct to say that the same type of bacteria has the same DNA? But my understanding is that it is not correct to say that the same type of human has the same DNA, since every human has a different DNA. What am I missing here?&lt;/p&gt;&#xA;" OwnerUserId="7450" LastActivityDate="2020-03-27T11:16:05.757" Title="Since every human has a different DNA (different combinations of C, G, A, T) what does it mean to have the genome done?" Tags="|genome|phylogenetics|" AnswerCount="8" CommentCount="7" ClosedDate="2020-03-27T11:38:07.770" ContentLicense="CC BY-SA 4.0" />
  <row Id="11722" PostTypeId="2" ParentId="11721" CreationDate="2020-03-24T15:32:40.130" Score="4" Body="&lt;p&gt;DNA is the product you get if you paste the nucleotides A, T, C and G together. This is DNA and all genomes consist of these basic building blocks like a book consists of letters. Still, the genome, or the content of the book is the specific combination of these letters plus the proteins that bind to the chromosomes, the folding of the DNA and all the epigenetic modifications that can happen. So yes, we all have the same DNA in terms of the same building blocks, but the content is different.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2020-03-24T22:46:48.617" LastActivityDate="2020-03-24T22:46:48.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11723" PostTypeId="1" CreationDate="2020-03-24T16:10:31.483" Score="2" ViewCount="39" Body="&lt;p&gt;I have the counts matrix of an RNA sequencing dataset of fibroblasts and I wish to identify isoforms of a particular gene of interest in it. Can anyone please hint me on a bioinformatics method to achieve this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In literature, 9 different isoforms of my gene of interest have been reported. My colleague who does the wet lab experiments has identified 3 of those isoforms through western blotting in the fibroblasts that later got sequenced. There might be more isoforms in these fibroblasts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I will be grateful for any computational hint to identify gene isoforms from my dataset, irrespective if they are differentially expressed or not.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance for your kind help.&lt;/p&gt;&#xA;" OwnerUserId="3968" LastActivityDate="2020-03-25T08:42:39.180" Title="Is there a computational tool or possibility to identify mRNA isoforms from the count matrix of a bulk RNA sequencing dataset?" Tags="|rna-seq|deseq2|rna|rna-alignment|rna-splicing|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11724" PostTypeId="2" ParentId="11721" CreationDate="2020-03-24T17:04:51.857" Score="8" Body="&lt;p&gt;DNA is only the substance that makes up the genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;DNA can come with many forms. For example, the fragments you get by PCR in the lab are DNA. A bacterium also has DNA molecules in it. You may also find DNA fragments in many other places, like in our blood, or at a crime scene.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Genome, however, is a very specific term. It means all the heritable substances in an individual or an organism. Most times, when we talk about the genome in eukaryotes, we refer to the nucleus genome, that is, the DNA in the nucleus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that genomes do not have to be made of DNA. RNA viruses, for example, only have RNA as their genomes. Yet we still refer to that as a genome, because that is the heritable substances for those viruses.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-03-24T17:04:51.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11725" PostTypeId="2" ParentId="11715" CreationDate="2020-03-24T17:12:38.467" Score="1" Body="&lt;p&gt;I do not know much about statistics but I will try my best to explain.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;First, random effects are defined as the factors (categories) in the population that we are not aware of (not observed), so we are randomly sampling levels of those factors when we sample the population.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Practically speaking, random effects can be found when there are hierarchical structures in the data. For example, we want to compare the relationship between students' SAT score and their college admission rates. You might just want to run a simple linear regression. But there might be some structure in the data that is hidden. For example, the students can be grouped into classes, and then schools. You can also group students by the social-economic status of their parents.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The idea is that students belonging to different categories may have different college admission rates even if they have the same SAT score. Therefore, it is important to consider some of these &quot;random effects&quot; when building your model.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-03-24T17:12:38.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11726" PostTypeId="2" ParentId="11386" CreationDate="2020-03-25T00:00:56.073" Score="0" Body="&lt;p&gt;One way to do this is to use a dictionary. You can read the first file line by line, and for each line, use &quot;chr:position&quot; as a key, and the corresponding line as the value. If you want to handle duplicates, you can make the value a list and store multiple lines with duplicated &quot;chr:position&quot; as elements in that list.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After constructing this dictionary, read the second file line by line. For each line, set &quot;chr:position&quot; as the so-called name of that line. Then using this &quot;chr:position&quot;, you can easily find the corresponding line in the dictionary. Append the line from the second file to that line from the first file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After finishing reading the second file, you are ready to write the new file to output. To do so, iterate over the keys in the dictionary, and write the value to each key to a new file. Remember that each value in that dictionary is now a new line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I choose to use a dictionary because they are basically a hash table, meaning that using the key you can instantly read that value in the dictionary, instead of iterating every element in a list to find that element. This is very handy when you have a large dictionary.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-03-25T00:00:56.073" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11728" PostTypeId="2" ParentId="11721" CreationDate="2020-03-25T03:36:32.873" Score="21" Body="&lt;p&gt;All humans have some differences in their DNA, but there's far more that is shared. On average the difference between humans is only about &lt;a href=&quot;http://book.bionumbers.org/how-genetically-similar-are-two-random-people/&quot; rel=&quot;nofollow noreferrer&quot;&gt;one thousandth&lt;/a&gt; of their full DNA, which means we're about 99.9% the same. These differences aren't distributed fully randomly, but are often because of specific gene alternatives. (Random mutations do occur, but they are also often fatal, so the random mutations we see of living adults are much more restrictive than all the random mutations that occur.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To identify the human genome is to study many people's DNA and to label the parts that are shared between everyone, the parts with two or three variations, and the parts with even more variations. Even though every person has different DNA, we can still say they fit the pattern, just like every T-shirt may be unique but they all fit the T-shirt pattern and not the trousers pattern.&lt;/p&gt;&#xA;" OwnerUserId="1797" LastEditorUserId="7491" LastEditDate="2020-03-27T00:33:22.437" LastActivityDate="2020-03-27T00:33:22.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11729" PostTypeId="2" ParentId="11723" CreationDate="2020-03-25T03:38:30.847" Score="1" Body="&lt;p&gt;If literally all you have is the count info by gene, you can't magic up the isoform breakdown.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-03-25T03:38:30.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11730" PostTypeId="1" CreationDate="2020-03-25T07:51:13.300" Score="1" ViewCount="55" Body="&lt;p&gt;I have questions regarding sequencing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are paired-end reads from different stands?&#xA;What about single reads, are they coming from both strands?&lt;/p&gt;&#xA;" OwnerUserId="7444" LastActivityDate="2020-03-25T21:55:57.633" Title="Location of Reads in Sequencing" Tags="|sequence-alignment|ngs|genome|sequence-analysis|genome-sequencing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11731" PostTypeId="2" ParentId="11723" CreationDate="2020-03-25T08:42:39.180" Score="2" Body="&lt;p&gt;You can back to your bam file and use cuffdiff tools it is very able to do your job follow this to liks&#xA;&lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/manual/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cole-trapnell-lab.github.io/cufflinks/manual/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/Jeanielmj/bioinformatics-workshop/wiki/Differential-Analysis-with-Cuffdiff&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Jeanielmj/bioinformatics-workshop/wiki/Differential-Analysis-with-Cuffdiff&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My best&lt;/p&gt;&#xA;" OwnerUserId="6694" LastActivityDate="2020-03-25T08:42:39.180" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11733" PostTypeId="1" CreationDate="2020-03-25T09:31:01.770" Score="1" ViewCount="233" Body="&lt;p&gt;I've detected homology between targets of ligands in drugbank and proteins in the proteome of a pathogen. I've parsed the output very rudimentary and calculated my query coverage. This exists in an excel file (see attached)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'd like to further filter these hits. For query coverages &gt; 70% alone I have 1500+ hits. Some redundancy exists and some hits are for drugbank targets for which no data has been deposited in the PDB. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So there are ways of further filtering these hits, but I'd like to do pairwise alignments first before I do anything else, as I think this is the most rational filter I can think of. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do not want to filter based on lack of PDB data necessarily, as when I check my high query coverage hits, I'm getting predictions that are actually backed up by in vitro data. This includes high coverage hits lacking x-ray data&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have CLUSTALO installed on my desktop: How do programmatically send it jobs if I have the Drugbank sequences and the pathogen sequences in the same file? I would also like to parse out the pathogen IDs from the excel file. There’s 1500 and they are a bit scattered throughout the columns. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;0.980634         -                    572         sp|P06672|PDC_ZYMMO                         -                     568                               1.2e-263             874.5          0.0            1        1      6.6e-267  1.4e-263   874.2      0.0       5         562    15     570    12     572    0.98   Uncharacterized                         protein                   OS=Lomentospora            prolificans        OX=41688             GN=jhhlp_005678  PE=3             SV=1&#xA;&#xA;0.950704         -                    580         sp|P06672|PDC_ZYMMO                         -                     568                               1.2e-258             858.0          0.0            1        1      6.8e-262  1.5e-258   857.7      0.0       6         546    10     560    7      578    0.98   Uncharacterized                         protein                   OS=Lomentospora            prolificans        OX=41688             GN=jhhlp_005014  PE=3             SV=1&#xA;&#xA;0.950704         -                    691         sp|P06672|PDC_ZYMMO                         -                     568                               8.2e-162             538.3          0.1            1        1      2.4e-164  5.1e-161   535.6      0.1       1         541    89     650    89     658    0.89   Acetolactate                            synthase                  OS=Lomentospora            prolificans        OX=41688             GN=jhhlp_006830  PE=3             SV=1&#xA;&#xA;0.492958         -                    585         sp|P06672|PDC_ZYMMO                         -                     568                               8e-87                290.7          0.0            1        2      3.3e-83   7e-80      267.8      0.0       3         283    2      287    1      292    0.94   Uncharacterized                         protein                   OS=Lomentospora            prolificans        OX=41688             GN=jhhlp_001897  PE=3             SV=1&#xA;&#xA;0.107394         -                    585         sp|P06672|PDC_ZYMMO                         -                     568                               8e-87                290.7          0.0            2        2      1.7e-08   3.6e-05    21.1       0.0       413       474    436    505    376    583    0.72   Uncharacterized                         protein                   OS=Lomentospora            prolificans        OX=41688             GN=jhhlp_001897  PE=3             SV=1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7459" LastEditorUserId="3848" LastEditDate="2020-04-23T20:22:44.890" LastActivityDate="2020-04-24T07:17:35.613" Title="Can I automate CLUSTALO and output alignment sequence identity?" Tags="|ngs|python|protein-structure|pdb|protein-protein-interaction|" AnswerCount="2" CommentCount="10" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11734" PostTypeId="1" CreationDate="2020-03-25T09:51:22.813" Score="3" ViewCount="162" Body="&lt;p&gt;Regarding Sequencing:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do/can the paired-end reads have overlaps?&lt;/p&gt;&#xA;" OwnerUserId="7444" LastActivityDate="2020-03-25T10:14:32.627" Title="Overlap in Paired-end Reads for Sequencing?" Tags="|sequence-alignment|ngs|genome|sequence-analysis|genome-sequencing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11735" PostTypeId="2" ParentId="11734" CreationDate="2020-03-25T10:14:32.627" Score="4" Body="&lt;p&gt;Yes, if the insert size is smaller than the read sizes, this would happen. For some applications, for example SNP detection for molecular diagnostics purposes, this approach is used.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-03-25T10:14:32.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11737" PostTypeId="1" CreationDate="2020-03-25T15:47:46.997" Score="2" ViewCount="60" Body="&lt;p&gt;Please, can you help me with a confusion?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In whole genome sequencing, if&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Total copy number (major + minor allele copy number by SCAT R package)&lt;/code&gt; for SMAD4 in a sample = &lt;code&gt;4&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Th estimated &lt;code&gt;ploidy&lt;/code&gt; by SCAT for this sample = &lt;code&gt;4.043696&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, the &lt;code&gt;segment mean&lt;/code&gt; (required for GISTIC) for SMAD4 for this given position of genome in the given sample = log2(4&lt;code&gt;/&lt;/code&gt;4.043696) =  &lt;code&gt;-0.01567461&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We can consider this as a &lt;code&gt;loss of copy number&lt;/code&gt; for SMAD4&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But with &lt;code&gt;GISTIC threshold -td = 0.1&lt;/code&gt; this is neutral copy number change (if I am not wrong)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, If I want to ask GISTIC to give me &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;amplification: copy number ≥2 × average ploidy​&#xA;gain:  copy number &amp;gt;1.25 × average ploidy&#xA;loss: copy number &amp;lt;0.75 ×average ploidy&#xA;deletion: copy number 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If I put &lt;code&gt;-ta = 1.25&lt;/code&gt; and &lt;code&gt;-td = 0.75&lt;/code&gt; , do I achieve the above?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am seriously struggling with this intuition of relationship between segment mean and GISTIC copy number threshold&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for any help&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2020-03-25T15:47:46.997" Title="Confusion about GISTIC threshold" Tags="|wgs|cnv|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11739" PostTypeId="2" ParentId="11721" CreationDate="2020-03-25T16:27:13.573" Score="1" Body="&lt;p&gt;Might have misunderstood you - but &quot;we&quot; have sampled and &quot;decoded&quot; the DNA of a few humans (often volunteers who surrendered their body to science after their death) - but this gives of course only the genetic code of those few individuals. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When scientists say we decoded the genome they mean they deciphered which parts of the DNA &quot;do&quot; specific things - the actual &quot;genes&quot; or gene-sequences - usually done when looking at the DNA molecule packed up into the chromosomes to give easier &quot;orientation&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And by my understanding the first task was accomplished - supported by use of supercomputers, but the deciphering of the gene-sequences still has a long way to go for scientists to state &quot;complete.&quot; But there is growing success - And as usual scientists will not stop until complete.&lt;/p&gt;&#xA;" OwnerUserId="7467" LastActivityDate="2020-03-25T16:27:13.573" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="11740" PostTypeId="2" ParentId="11721" CreationDate="2020-03-25T19:19:09.663" Score="0" Body="&lt;p&gt;&quot;To have the genome done&quot; means that we have sequenced all the DNA in particular reference individuals. You are correct that every human has different DNA, so there is no such thing as a single &quot;human genome&quot;, as (almost) everyone's is unique. But the vast majority of everyone's DNA is identical - that's what makes us human. Having the full sequence of a few &quot;normal&quot; individuals who don't have glaring genetic defects is extremely useful for learning about the genome of &lt;em&gt;all&lt;/em&gt; individuals, since so much of it is the same from person to person.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've used the term &quot;reference&quot; a few times, since we use a few individuals' genomes as an expectation of what a human genome looks like. The process of sequencing DNA involves breaking it into many bits, and then recombining them using a computer. You can put the pieces together without the reference, but it is much more difficult and time consuming. Even if there are differences from the reference, having an expectation of what &gt;99% of the genome should be makes the problem much simpler. In a way, it's like the difference between doing a jigsaw puzzle without knowing what it's supposed to look like, and using the picture on the box as a reference. Even if your exact jigsaw puzzle has a few different pieces, the box reference is  mostly the same and immensely useful.&lt;/p&gt;&#xA;" OwnerUserId="1930" LastActivityDate="2020-03-25T19:19:09.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11741" PostTypeId="1" CreationDate="2020-03-25T19:29:08.867" Score="0" ViewCount="54" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/428956/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have run &lt;em&gt;&lt;strong&gt;cuffdiff&lt;/strong&gt;&lt;/em&gt; (with statistics turned ON) to compare two groups of samples: &lt;em&gt;Control group&lt;/em&gt; and &lt;em&gt;Late AD group&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This is the command I ran to be precise:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cuffdiff -L Control,AD_Late_Braak -p 8 --total-hits-norm --frag-bias-correct ../References/ensembl.GRCh38.99.fa --multi-read-correct --library-norm-method quartile ../References/Homo_sapiens.GRCh38.99.chr.gtf Early_Braak_Control_1,Early_Braak_Control_2,Early_Braak_Control_3 Late_Braak_Sample_1,Late_Braak_Sample_2,Late_Braak_Sample_3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm looking at the output from &lt;em&gt;cuffdiff&lt;/em&gt; and I see a &lt;em&gt;&lt;strong&gt;gene_exp.diff&lt;/strong&gt;&lt;/em&gt; file which contains the results of the differential expression testing. I want to know what is the best way to filter the results of this gene_exp.diff file so as to restrict the number of genes that are up-regulated and down-regulated to a list between ~50-850.&lt;/p&gt;&#xA;&lt;p&gt;P.S My thoughts are to tweak the &lt;strong&gt;p-values&lt;/strong&gt; and &lt;strong&gt;log2 fold change&lt;/strong&gt; values, but seems like a trail-and-error method, so I was wondering if there was a more &amp;quot;formal&amp;quot; method/approach?&lt;/p&gt;&#xA;" OwnerUserId="7471" LastEditorUserId="73" LastEditDate="2021-06-07T02:07:10.693" LastActivityDate="2021-06-07T02:07:10.693" Title="Filtering genes from cuffdiff results" Tags="|rna-seq|differential-expression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11742" PostTypeId="2" ParentId="11730" CreationDate="2020-03-25T21:55:57.633" Score="2" Body="&lt;p&gt;Mates in a pair are giving signal for a single strand, though this is only apparent in bisulfite-treated samples. The reason for this is that you are sequencing the ends of fragments of material and it's random which strand of this (or both in different clusters) will get sequenced. Different pairs of reads will then randomly sequence one strand or the other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Single-end reads are the same as paired-end reads, the second read in the pair is just not made.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-03-25T21:55:57.633" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11743" PostTypeId="2" ParentId="11733" CreationDate="2020-03-25T23:59:47.137" Score="0" Body="&lt;p&gt;The question is still in numerous parts,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;parsing a blast output&lt;/li&gt;&#xA;&lt;li&gt;obtaining sequences &lt;/li&gt;&#xA;&lt;li&gt;feeding this as input into clustalo&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;If I was automating this I would parse the Blast genbank codes [regex either Perl one-liner (scripts aren't trendy ;-( ), or Pythons &quot;re&quot;] and feed them into NCBI's &lt;code&gt;efetch&lt;/code&gt; to hook out the sequence data. Then I would create a subprocess that feed the files into clustalo. In Perl you simply issue the linux command wrapped in &quot;`&quot; either side, in Python use &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;os.system('clustalo -in file.in -out file.out')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Ensuing your 'bin' is correctly configured to execute clustalo&lt;/p&gt;&#xA;&#xA;&lt;p&gt;or &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import subprocess # use popen&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To my knowledge there is no wrapper for &lt;code&gt;clustalo&lt;/code&gt; unlike several other phylogenetics programs in BioPython, which enormously simplifies pipelines. Others might recommend using shell rather than e.g. Python.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BTW I personally use &lt;code&gt;Muscle&lt;/code&gt; having been a long term user of clustalo, thats a long discussion. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To be honest you should look at the power of NBCI's online blast utility. The download facilities are certainly powerful particularly if you are pulling down e.g. 1500 sequences from blast hits, because it automatically can splice the hits of interest from their genes/genomes. However, its context specific.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-03-26T00:05:44.343" LastActivityDate="2020-03-26T00:05:44.343" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11745" PostTypeId="1" AcceptedAnswerId="11747" CreationDate="2020-03-26T09:47:15.030" Score="2" ViewCount="1285" Body="&lt;p&gt;I am trying to identify the best library prep method for noninvasive prenatal test samples, to be sequenced with Illumina Novaseq.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One metric that I am evaluating is the number/ percentage of PCR or optical duplicates. In order to achieve that I have done the following steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;aligned the fastq reads against the reference genome with &lt;code&gt;bwa mem&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;the &lt;code&gt;.sam&lt;/code&gt; output from &lt;code&gt;bwa mem&lt;/code&gt; was converted to &lt;code&gt;bam&lt;/code&gt;, sorted and indexed&lt;/li&gt;&#xA;&lt;li&gt;the duplicates were marked with the command &lt;code&gt;MarkDuplicates&lt;/code&gt; from picard&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Then if I call &lt;code&gt;samtools flagstat&lt;/code&gt; on the sorted &lt;code&gt;bam&lt;/code&gt; file which had the duplicates marked with picard I get:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;26595942 + 0 in total (QC-passed reads + QC-failed reads)&#xA;0 + 0 secondary&#xA;466 + 0 supplementary&#xA;1636809 + 0 duplicates&#xA;24969064 + 0 mapped (93.88% : N/A)&#xA;26595476 + 0 paired in sequencing&#xA;13297738 + 0 read1&#xA;13297738 + 0 read2&#xA;21102678 + 0 properly paired (79.35% : N/A)&#xA;24049752 + 0 with itself and mate mapped&#xA;918846 + 0 singletons (3.45% : N/A)&#xA;462212 + 0 with mate mapped to a different chr&#xA;182138 + 0 with mate mapped to a different chr (mapQ&amp;gt;=5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which sounds a bit strange to me, as no duplicates were found.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the &lt;code&gt;MarkDuplicates&lt;/code&gt; command also outputted a metrics file (&lt;code&gt;dups/dupsMetrics.txt&lt;/code&gt;) that did identify some duplicates. This was achieved with the following command:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar picard.jar MarkDuplicates I=alignment/sample.sorted.bam O=alignment/sample_markDup.bam M=dups/dupsMetrics.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I see that the value under the column &lt;code&gt;PERCENT_DUPLICATION&lt;/code&gt; in &lt;code&gt;dups/dupsMetrics.txt&lt;/code&gt; is 0.065555 (given the column name, I would expect that the value is indeed 0.06%, and not 6.55%)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What would be the explanation between the discrepancy in the output of &lt;code&gt;samtools flagstat&lt;/code&gt; and &lt;code&gt;MarkDuplicates&lt;/code&gt;. Is it reasonable to expect that the value displayed by &lt;code&gt;samtools flagstat&lt;/code&gt; is &lt;strong&gt;rounded&lt;/strong&gt; to zero, given the low percentage which was observed by &lt;code&gt;MarkDuplicates&lt;/code&gt; (0.06%)?&lt;/p&gt;&#xA;" OwnerUserId="3452" LastEditorUserId="2776" LastEditDate="2020-04-24T06:03:16.180" LastActivityDate="2020-04-24T06:03:16.180" Title="Marking optical or PCR duplicates with picard vs. samtools flagstat" Tags="|sam|ngs|samtools|gatk|picard|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11746" PostTypeId="1" AcceptedAnswerId="11761" CreationDate="2020-03-26T12:18:36.220" Score="3" ViewCount="581" Body="&lt;p&gt;I don't have any background in genetics and bioinformatics, so I ask you if you think that the arguments provided in the article &lt;a href=&quot;https://www.nature.com/articles/s41591-020-0820-9?fbclid=IwAR3KyrFty1XSXDsNrYcFfchr8ijK6qZeCria_y5VsAj-Oxdiei--fLOiviY&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;The proximal origin of SARS-CoV-2&lt;/em&gt;&lt;/a&gt; by Andersen &lt;em&gt;et al.&lt;/em&gt; are convincing. In particular:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;While the analyses above suggest that SARS-CoV-2 may bind human ACE2 with high affinity, computational analyses predict that the interaction is not ideal and that the RBD sequence is different from those shown in SARS-CoV to be optimal for receptor binding. Thus, the high-affinity binding of the SARS-CoV-2 spike protein to human ACE2 is most likely the result of natural selection...&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I interpret this passage as &quot;if the virus had been engineered, they would have done a better job&quot;. Is this interpretation correct? And if it is, does it sound convincing to you as a proof that the virus developed in nature?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The second motivation given in the article is the following:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;Furthermore, if genetic manipulation had been performed, one of the several reverse-genetic systems available for betacoronaviruses would probably have been used&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Again, do you think that this exclude the possibility of a human intervention in the creation of the virus?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your opinions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: I am a fan of Occam razor, I know that the scenario in which the virus originated in the wild is by far more likely than the human engineered scenario. I just want to know if, given our current knowledge in genetics, would have been possible for some high skilled researchers to engineered COVID19 (without implying that they did this with any bad purpose)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit 2: I share the concerns of the author of this post, that is, that the findings of the article are merely opinions and some of arguments are misleading. Anyone having a solid technical background can comment on this? &lt;a href=&quot;https://harvardtothebighouse.com/2020/03/19/china-owns-nature-magazines-ass-debunking-the-proximal-origin-of-sars-cov-2-claiming-covid-19-wasnt-from-a-lab/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;China owns Nature magazine’s ass – Debunking “The proximal origin of SARS-CoV-2” claiming COVID-19 definitely wasn’t from a lab&lt;/em&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7480" LastEditorUserId="3848" LastEditDate="2020-11-30T22:30:13.227" LastActivityDate="2020-11-30T22:30:13.227" Title="Are the conclusions in &quot;The proximal origin of SARS-CoV-2&quot; legit?" Tags="|sequence-alignment|phylogenetics|phylogeny|covid-19|sars-cov-2|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="11747" PostTypeId="2" ParentId="11745" CreationDate="2020-03-26T12:20:40.610" Score="5" Body="&lt;p&gt;There are duplicates, in this line:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1636809 + 0 duplicates, gives 1636809/26595942 = 0.06154356&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to &lt;a href=&quot;http://www.htslib.org/doc/samtools-flagstat.html&quot; rel=&quot;noreferrer&quot;&gt;samtools documentation for flagstat&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Provides counts for each of 13 categories based primarily on bit flags&#xA;  in the FLAG field. Each category in the output is broken down into QC&#xA;  pass and QC fail. In the default output format, these are presented as&#xA;  &quot;#PASS + #FAIL&quot; followed by a description of the category.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So you can usually take whatever is in the first column&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-03-26T12:20:40.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11748" PostTypeId="1" CreationDate="2020-03-26T14:37:03.930" Score="0" ViewCount="36" Body="&lt;p&gt;I want to compare the protein dynamics &#xA;) pH 7 versus pH 3, or &#xA;) wild type versus mutant&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The protein will have slightly different number of atoms at each condition, due to protonation or mutation, which is bad for machine learning. The alternative is to only consider the alpha-carbon atoms, but this will lose significant amount of information (e.g. side chain, salt bridge).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way in molecular dynamics to extract features from ensembles of different number of atoms?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I have used common methods to decipher the difference between two conditions, including RMSD, RMSF, Rg, secondary structure, salt bridge, SASA, etc. They can explain the difference to some extent. However, I feel the selection of properties is &quot;subjective&quot;, and there must be some hidden features not known. So I am seeking a machine learning method, that can &quot;automatically&quot; capture the most essential properties based on the coordinates as inputs.&lt;/p&gt;&#xA;" OwnerUserId="1806" LastEditorUserId="1806" LastEditDate="2020-03-27T02:11:14.770" LastActivityDate="2020-03-27T09:54:20.510" Title="Feature extraction methods that can handle inconsistent numbers of atoms for molecular dynamics" Tags="|machine-learning|features|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11749" PostTypeId="1" AcceptedAnswerId="11750" CreationDate="2020-03-26T15:51:06.277" Score="3" ViewCount="226" Body="&lt;p&gt;I started reading some papers on X-wide association studies, where X can be metabolome, epigenome, etc... The authors usually describe which are the dependent and which are the independent variables of their models. Taking as an example a metabolome-wide association study (MWAS), the authors use as model the following: &lt;code&gt;metabolite&lt;/code&gt; ~ &lt;code&gt;condition&lt;/code&gt;. That is, each metabolite is considered the dependent variable and the condition (e.g., smoker vs. non-smoker) is considered the independent variable. I was a little bit confused, so I looked into an example study using GWAS: in this case the model can be written as &lt;code&gt;condition&lt;/code&gt; ~ &lt;code&gt;SNPs&lt;/code&gt;, which is what I would expect (knowing nothing about GWAS but something about Data Science) and quite the opposite from the MWAS described above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does the choice of the dependent variable depend on the questions asked in the study, or does it exist some sort of &lt;em&gt;agreement&lt;/em&gt;? Moreover, regarding the GWAS model described above, I read that usually researchers fit a model for each SNP: what's wrong with fitting a model which consists of several independent variables (i.e., &lt;code&gt;y ~ SNP_1 + SNP_2 + ... + SNP_N&lt;/code&gt;)?&lt;/p&gt;&#xA;" OwnerUserId="2862" LastActivityDate="2020-03-26T16:34:50.567" Title="GWAS, MWAS, EWAS: what are the (in)dependent variables?" Tags="|gwas|linear-regression|logistic-regression|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="11750" PostTypeId="2" ParentId="11749" CreationDate="2020-03-26T16:34:50.567" Score="3" Body="&lt;p&gt;To answer the first part of your question, the dependent and independent variable of X-WAS is kind of arbitrary and dependent on the question you asked. But it gradually becomes a convention in the field after the initial name and concept are accepted by the community. For example, GWAS from the very beginning is written as &lt;code&gt;condition ~ SNP&lt;/code&gt;, and there is no doubt about it. Later, people did something like &lt;code&gt;condition ~ gene expression&lt;/code&gt; and called it TWAS, which is also widely accepted now. In your case, MWAS is indeed &lt;code&gt;metabolite ~ condition&lt;/code&gt;, and the only important thing is to know the convention in the research community so people are on the same page when they talk to each other.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for why running GWAS using single SNP. In a GWAS, it is typical to use 500K to 7M SNPs. So this is a large number. It is obvious that we cannot include all the SNP in one single model, then the question becomes how many SNPs to include in each model, and how to choose those SNPs. This is very hard to decide. Also, due to link disequilibrium, many SNPs are correlated, which means co-linearity, a thing you want to avoid in linear models. It is not clear how to correctly interpret the effect sizes in a model with many SNPs as well. Therefore, due to feasibility, simplicity, and interpretability, it might be best to perform regression on every single SNP.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now since we are running GWAS in an SNP-by-SNP fashion, there are a lot of work that needs to be done after GWAS in order to find the true causal SNP. Many fine-mapping methods have been developed to tackle this problem.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Interestingly, there are some multi-variate GWASs methods, but they do not come in the flavor of simple linear models. Instead, sometimes complex Bayesian statistics are used, for instance.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-03-26T16:34:50.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11751" PostTypeId="2" ParentId="11721" CreationDate="2020-03-26T16:54:29.793" Score="0" Body="&lt;p&gt;Think of it as a special book. Everyone's book is different, but they all have the same number of chapters and each chapter covers the same topic in each individual book (sorta). The problem is we can only read a few words at a time. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the genome is &quot;done&quot;, it means that someone has read a bunch of these sentence fragments from one person's book and then assembled all the words in the correct order to make a complete book. Then we can get a good idea of how many chapters it has and what those chapters are about.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the human genome wouldn't be a novel, it would be more like a set of encyclopedias.&lt;/p&gt;&#xA;" OwnerUserId="7485" LastActivityDate="2020-03-26T16:54:29.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11753" PostTypeId="2" ParentId="11721" CreationDate="2020-03-26T19:07:30.850" Score="0" Body="&lt;p&gt;When people say &quot;the genome&quot; for a species, they mean a &lt;a href=&quot;https://en.wikipedia.org/wiki/Reference_genome&quot; rel=&quot;nofollow noreferrer&quot;&gt;reference genome&lt;/a&gt;. To quote wikipedia,&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;A reference genome (also known as a reference assembly) is a digital nucleic acid sequence database, assembled by scientists as a representative example of a species' set of genes. As they are often assembled from the sequencing of DNA from a number of donors, reference genomes do not accurately represent the set of genes of any single person.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The genome is updated over time as more is learned, the most recent human reference is hg38.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For individualized results, the patient is sequenced and the data is compared to the reference genome. Then, the differences are presented as a list of &lt;a href=&quot;https://en.wikipedia.org/wiki/Single-nucleotide_polymorphism&quot; rel=&quot;nofollow noreferrer&quot;&gt;genetic variants&lt;/a&gt;. This is more efficient than looking at the entire genome for each patient, because on average greater than 99.9% of sites match the reference.&lt;/p&gt;&#xA;" OwnerUserId="4782" LastActivityDate="2020-03-26T19:07:30.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11754" PostTypeId="2" ParentId="11334" CreationDate="2020-03-26T20:32:33.623" Score="2" Body="&lt;p&gt;I was facing this problem in January when coming up with a methodology to forecast on the UrbanSurvival.com site.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My solution - using low data collection was to use a daily rolling rate of infections and divide that into the 5-day previous reported death counts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This was based on the assumption that, for a given cohort, the people who get sick today will &quot;get dead&quot; in about 10-days time.&lt;br&gt;&#xA;So half of this (5 days) ought to get me into the ballpark.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Feb, when CDC was reporting 3.4% mort. rate, we'd been reporting that for a couple of weeks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a formula it's not unlike deriving an exponent.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is:&#xA;Cases Today (T1) / Deaths 5-Days Ago (T2)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, this is presently running a bit higher mortality rate now (4.5%) but I would expect on the backside of the expansion of cases it will decline.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's hopin'...&lt;/p&gt;&#xA;" OwnerUserId="7489" LastActivityDate="2020-03-26T20:32:33.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11755" PostTypeId="2" ParentId="11748" CreationDate="2020-03-26T22:54:31.533" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;This answer is no longer on topic. The question is about what is the best machine learning algorithm to use to analyse atomic coordinates from an MD trajectory in order to infer a novel properties.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&amp;nbsp;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Also, it should be preface for any confused reader that unfortunately &quot;feature&quot; is a technical term in both genetics and statistics. For the former &lt;a href=&quot;https://en.wikipedia.org/wiki/General_feature_format&quot; rel=&quot;nofollow noreferrer&quot;&gt;a feature&lt;/a&gt; is a region of DNA, RNA or protein that has a given property, such as open reading frame or PFAM domain. While in the latter &lt;a href=&quot;https://en.wikipedia.org/wiki/Feature_(machine_learning)&quot; rel=&quot;nofollow noreferrer&quot;&gt;a feature&lt;/a&gt; is a preferably independent property that forms part of a feature vector for different types of machine learning.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;h2&gt;Heavy atoms are common&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Assuming the pH difference only results in protonations, the protons differ between the two conditions, but not the heavy atoms. These will retain all the side chain information.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Merit of backbone only&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;(Section no longer relevant due to OP edit)&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The common metric used to compare two structures is root mean square deviation (RMSD) of the alpha carbons to assess how much does the backbone differ and with analysis of MD simulations it is used to assess whether one is more dynamic than the other (e.g. this &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/11036/how-to-interpret-the-gromacs-md-simulation-rmsd-graph/11045&quot;&gt;Bioinformatics SE question&lt;/a&gt;). The reason why the RMSD is done with the alpha carbons and not the minimum common subset of atoms is that backbone changes are large scale, while the side chain changes only affect locally —so their inclusion would needlessly dilute the score. The C prime and backbone nitrogens behave like the C alpha so there is no need for these.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Extraction of coordinates&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Assuming by features you mean atomic coordinates, there are loads of ways. Using Pymol module for Python, one could do something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pymol2&#xA;with pymol2.PyMOL() as pymol: #can be parallelised&#xA;    pymol.cmd.load('trajectory.gro', 'mytraj')&#xA;    pymol.cmd.remove('h.') #remove hydrogens&#xA;    pymol.cmd.intra_fit(&quot;name CA&quot;, 1) #align all steps in the trajectory&#xA;    for i in range(1, pymol.cmd.count_states('mytraj')):&#xA;        for atom in pymol.cmd.get_model(f'state {i}').atom:&#xA;            atom.coord #List[int int int]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;One could change &lt;code&gt;pymol.cmd.get_model(f'state {i}')&lt;/code&gt; to select &lt;code&gt;state {i} and name CA&lt;/code&gt; for C&amp;alpha;s only.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Edit: Mutated residues&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;In the case of mutated residues or residue differences due to the two proteins being close homologues, &quot;mutating&quot; the differing residues to alanine, i.e. by stripping all heavy atom beyond the C&amp;beta; (keeping C&amp;prime;, C&amp;alpha;, H, N, O &amp;amp; C&amp;beta;) in the MD trajectory —or glycine if one is glycine. If they are homologues with different lengths, this approach would mean skipping residues that match up with gaps in the MSA. Even though not uncommon for statistically analyses, it does pose some issues statistically. It may unbury hidden residues, but length variant generally happens with loops.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the snippet above, the pymol mutagenesis wizard could be used, but that is overkill, does not work in parallel and the approach does not port to other PDB reading modules in whatever language one wants. So say you have a list &lt;code&gt;to_be_trimmed&lt;/code&gt; of integers of PDB residue indices (not pose residue indices):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if atom.resi not in to_be_trimmed or atom.name.strip() in ('C', 'CA', 'H', 'N', 'O', 'CB'):&#xA;      do whatever with atom.coord &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In acidic environments, you get some funky things happening like exposed aspartate-glycine residue pairs forming &lt;a href=&quot;https://en.wikipedia.org/wiki/Isoaspartate&quot; rel=&quot;nofollow noreferrer&quot;&gt;isoaspartate&lt;/a&gt; or exposed aspartate-proline resulting in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3149706/&quot; rel=&quot;nofollow noreferrer&quot;&gt;a cleavage&lt;/a&gt;, etc. For these cases, which are nearly never modelled, ignoring these values is the best option.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Edit: small comment on SASA&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The investigated property list is very extensive and some reader may adopt them. So I thought I should mention that per residue SASA is generally converted to relative surface area, which is residue independent, by dividing it by the maximum SASA for that residue (values from Tien et al. 2013: &lt;code&gt;'A': 121.0, 'R': 265.0, 'N': 187.0, 'D': 187.0, 'C': 148.0, 'E': 214.0, 'Q': 214.0, 'G': 97.0, 'H': 216.0, 'I': 195.0, 'L': 191.0, 'K': 230.0, 'M': 203.0, 'F': 228.0, 'P': 154.0, 'S': 143.0, 'T': 163.0, 'W': 264.0, 'Y': 255.0, 'V': 165.0}&lt;/code&gt;), where RSA &gt; 0.2 is an exposed residue.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-03-27T09:54:20.510" LastActivityDate="2020-03-27T09:54:20.510" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="11757" PostTypeId="1" AcceptedAnswerId="12793" CreationDate="2020-03-27T10:33:24.287" Score="1" ViewCount="119" Body="&lt;p&gt;I have hundreds of GBs metagenomic 16S rRNA gene sequence data. I want to do &lt;strong&gt;microbiome composition profiling&lt;/strong&gt; (with relative abundance) from the data. Also after that, I will do functional profiling (&lt;strong&gt;gene and metabolome level&lt;/strong&gt;). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you please suggest me the &lt;strong&gt;best software(s)&lt;/strong&gt; for these two tasks?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank You&lt;/p&gt;&#xA;" OwnerUserId="7427" LastEditorUserId="7427" LastEditDate="2020-03-27T10:40:10.940" LastActivityDate="2023-05-25T13:20:53.277" Title="Software for microbial profiling from 16S rRNA gene sequence" Tags="|software-recommendation|metagenome|multi-omics|systems-biology|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11758" PostTypeId="1" CreationDate="2020-03-27T10:39:15.003" Score="2" ViewCount="62" Body="&lt;p&gt;Using NCBI Gene database, I retrieved list of 7015 protein which are implicated in autism pathogenesis. This list includes tax-ID, organism name, symbol, aliases, map location, chromosome, genomic nucleotide accession version, start position on genomic accession, end position on genomic accession, orientation, exon count and OMIM). Among these 7015, 1144 proteins are involved in neurodevelopmental functions. I want to know that how many of these 1144 proteins are functionally similar to 7015 proteins? Is there a method to query 1144 proteins against 7015 proteins and group them based on structural similarity/ function similarity/ gene family?. Since I have large set of proteins, I am trying to understand if sub-set of proteins share similar function with this large set of proteins.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Priya&lt;/p&gt;&#xA;" OwnerUserId="7424" LastEditorUserId="7424" LastEditDate="2020-03-27T13:02:35.987" LastActivityDate="2020-03-28T13:08:48.700" Title="Analyzing proteins based on sequence similarity" Tags="|structural-variation|gene-ontology|sequence-homology|multi-omics|paralogs|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="11760" PostTypeId="2" ParentId="11721" CreationDate="2020-03-27T11:16:05.757" Score="0" Body="&lt;p&gt;Yes you are right.  99.9% of our genome remains same, it is only the 0.1% which makes each individual unique from the rest of us. Single nucleotide polymorphisms (SNPs) contributes to the 0.1% variation in our genome. SNPs explain the variation in eye color, susceptibility towards disease conditions etc&lt;/p&gt;&#xA;" OwnerUserId="7424" LastActivityDate="2020-03-27T11:16:05.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="11761" PostTypeId="2" ParentId="11746" CreationDate="2020-03-27T13:34:23.503" Score="4" Body="&lt;p&gt;In summary, the authors are saying the complete opposite of &quot;human intervention&quot;.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;While the analyses above suggest that SARS-CoV-2 may bind human ACE2&#xA;  with high affinity, computational analyses predict that the&#xA;  interaction is not ideal and that the RBD sequence is different from&#xA;  those shown in SARS-CoV to be optimal for receptor binding.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The interaction with ACE2 is taking place on the Spike protein (S) in both COVID-19 and SARS-Cov. The S protein is very different between COVID-19 and SARS both in terms of sequence and COVID-19 includes an additional furin cleavage site, meaning structurally it is quite distinct. The &quot;computational analysis predicts&quot; is difficult because there is no crystal structure for COVID-19 S protein therefore modelling ligand binding is essentially guess work, particularly as it comprises an additional (furin) cleaved domain. What the authors appear to conclude is that they cannot map ACE2-Spike binding of COVID-19 using an established SARS S protein structure. I assume they are homology modelling SARS-CoV onto COVID-19 S protein sequence data.  In my personal opinion there is no way to homology model an additional furin protease cleavage site (furin is a host protease) so it would be difficult for &quot;good fit&quot; to occur, but it is just my opinion and the authors may have established methods to overcome this notable limitation, e.g. similar phenomena occur in influenza. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Thus, the high-affinity binding of the SARS-CoV-2 spike protein to&#xA;  human ACE2 is most likely the result of natural selection..&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Their conclusion of &quot;natural selection&quot; is not strictly accurate and should be &quot;Darwinian positive selection&quot;, but its a small issue. What they are saying is Darwinian adaptation has occurred as opposed to &quot;purifying selection, i.e. conservation. Thus they are definitely excluding human intervention because that is not &quot;natural selection&quot;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Furthermore, if genetic manipulation had been performed, one of the&#xA;  several reverse-genetic systems available for betacoronaviruses would&#xA;  probably have been used&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;They appear to be saying that if a synthetic virus was constructed it would be created using an established reverse genetics system, and all those produced to date look nothing like COVID-19.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The phylogenetics argument is always a stronger argument than either of the above (although my opinion isn't entirely objective).  Phylogeneticists have had a long history in opposing conspiracy theories, e.g. polio vaccination, HIV and the list goes on, so it is much more natural territory. An example of such an argument is &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/11306&quot;&gt;here&lt;/a&gt; which @terdon helpfully pointed out.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;To address the questions of @Hans&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 1&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Purifying selection = any evolutionary change is deleterious, so the virus is less able to transmit between humans. The vast majority of mutations are deleterious in classical Darwinian thinking*&lt;/li&gt;&#xA;&lt;li&gt;Adaptation = the amino acid change occurs to ensure the virus is better able to transmit between humans. In phylogenetics this is called 'positive selection' which is detected at a nucleotide level, this is however a very stringest test.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;*, There is something called nearly neutral theory, but thats just complicated&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Question 2&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What we are talking about is a virus infectious clone system. For a 30kb virus this would not be trival at all, but I agree for 10kb viruses it much easier. However, the phylogenetics are opposed to this, because SARS-CoV-2 is 5% divergent from a virus (RaTG13) isolated a long time before the current epidemic. You can't just engineer a virus that is 5% divergent from a bat virus across the entire genome and never been seen before. It would be a work of genius the likes we have never seen using bioinformatics as yet unheard of, because what would happen is you would encounter endless deleterious mutations. Even if you got over all those, how do you know what fitness traits you are aiming for?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Serial passage&lt;/strong&gt; &#xA;This level of change is doable by serial passage of the virus, but this leads to attenuation of virus (less dangerous) not increased fitness. The yellow fever virus vaccine was created in this way. Even if a virus had been serially passaged we could tell, because we know the mutational patterns of other serially passaged viruses.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-04-30T09:51:57.353" LastActivityDate="2020-04-30T09:51:57.353" CommentCount="27" ContentLicense="CC BY-SA 4.0" />
  <row Id="11764" PostTypeId="2" ParentId="11758" CreationDate="2020-03-28T13:08:48.700" Score="2" Body="&lt;p&gt;I suggest you give &lt;a href=&quot;https://david.ncifcrf.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DAVID&lt;/a&gt; a try. Specifically, their &lt;a href=&quot;https://david.ncifcrf.gov/summary.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;Functional Annotation&lt;/a&gt; tool. Just enter your list of protein IDs, and it will return groups of proteins where particular GO functions are overrepresented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kkbI7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kkbI7.png&quot; alt=&quot;screenshot of the DAVID page&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This should at least help you start categorizing your list into functional groups. &lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2020-03-28T13:08:48.700" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12764" PostTypeId="1" CreationDate="2020-03-28T22:25:12.610" Score="2" ViewCount="544" Body="&lt;p&gt;I need to merge my all transcripts.gtf from cufflinks output follow this command line :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cuffmerge -o merged_gtf_output -p 15 -s ref.fasta -g anot.gtf assembly.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I have this warning :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[bam_header_read] EOF marker is absent. The input is probably truncated.&#xA;[bam_header_read] invalid BAM binary header (this is not a BAM file).&#xA;File merge_B3//tmp/mergeSam_fileOaNffV doesn't appear to be a valid BAM file, trying SAM...&#xA;[21:40:04] Loading reference annotation.&#xA;[21:40:05] Inspecting reads and determining fragment length distribution.&#xA;Processed 3921 loci.                        &#xA;&amp;gt; Map Properties:&#xA;&amp;gt;       Normalized Map Mass: 33242.00&#xA;&amp;gt;       Raw Map Mass: 33242.00&#xA;&amp;gt;       Fragment Length Distribution: Truncated Gaussian (default)&#xA;&amp;gt;                     Default Mean: 200&#xA;&amp;gt;                  Default Std Dev: 80&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can someone tell me what this warning means and how to fix it?&lt;/p&gt;&#xA;" OwnerUserId="6694" LastEditorUserId="48" LastEditDate="2020-09-01T12:29:49.287" LastActivityDate="2021-09-26T16:06:25.483" Title="Cuffmerge: EOF marker is absent. The input is probably truncated" Tags="|rna-seq|samtools|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12765" PostTypeId="2" ParentId="4248" CreationDate="2020-03-28T22:25:58.493" Score="1" Body="&lt;p&gt;I read a paper where the authors used different combinations of parameters to iteratively perform LSI until some criteria. Maybe a similar strategy can be applied to Louvain clustering as well?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper can be found here: &lt;a href=&quot;https://www.nature.com/articles/s41587-019-0332-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41587-019-0332-7&lt;/a&gt;.&#xA;You can also check their code to see how they implemented it.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-03-28T22:25:58.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12767" PostTypeId="1" CreationDate="2020-03-29T00:38:40.653" Score="1" ViewCount="32" Body="&lt;p&gt;I am downloading sample slides (in svs format) from the &lt;a href=&quot;https://portal.gdc.cancer.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GDC Portal&lt;/a&gt; for a convolutional neural network model, but I also need normal tissue slides to perform classification.&lt;/p&gt;&#xA;&lt;p&gt;There are some normal tissue slides that are adjacent to the tumors, but I'm looking for other easy to use portals on the internet where I can download different types of normal tissue (bone, connective, etc).&lt;/p&gt;&#xA;&lt;p&gt;Can anyone recommend similar type portals where I can download svs files (or equivalent) of normal tissue samples?&lt;/p&gt;&#xA;" OwnerUserId="8508" LastEditorUserId="3848" LastEditDate="2020-11-18T20:50:14.377" LastActivityDate="2020-11-18T20:50:14.377" Title="Portals besides the National Cancer Institute GDC Data Portal for downloading histology slides" Tags="|cancer|machine-learning|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12768" PostTypeId="1" CreationDate="2020-03-29T06:21:34.073" Score="1" ViewCount="77" Body="&lt;p&gt;I am trying to create an integrated callset with coding regions from phased exomes and noncoding regions from a genotyping array.  I would like to merge the data together in the correct phase.  Since the exomes are already phased with read informed phasing, I don't want to rephase them because some of the phasing changes after doing this.  Is there a way to statistically phase genetic data that has already been partially phased?&lt;/p&gt;&#xA;&lt;p&gt;My data is in VCF format.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="59" LastEditorUserId="73" LastEditDate="2021-07-14T11:13:24.013" LastActivityDate="2023-08-03T18:08:05.997" Title="Phasing partially phased genomic data" Tags="|genomics|genotyping|exome|phasing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12769" PostTypeId="1" AcceptedAnswerId="12772" CreationDate="2020-03-29T07:18:41.983" Score="2" ViewCount="385" Body="&lt;p&gt;I am new to the field of Bioinformatics. &lt;a href=&quot;https://gtexportal.org/home/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GTEx&lt;/a&gt; database is great; I know how to check whether a SNP is eQtl &lt;strong&gt;with the online tools&lt;/strong&gt; of GTEx. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;But my question is how to download the right data to draw a boxplot on my own. like this &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5567283/&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/zNkIm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zNkIm.png&quot; alt=&quot;Figure 1 in this paper&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please help me . I have been stuck on this for many days.&lt;/p&gt;&#xA;" OwnerUserId="6519" LastActivityDate="2020-03-30T02:46:44.063" Title="How can I download the SNP genotyping information from GTEX?" Tags="|snp|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="12770" PostTypeId="1" CreationDate="2020-03-29T16:56:07.090" Score="4" ViewCount="313" Body="&lt;p&gt;I'm a beginner to structural biology and for fun, calculated the tortional angles of some 100, 000 proteins. Here is my Ramachandran plot:  &lt;a href=&quot;https://i.stack.imgur.com/UxBRk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UxBRk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I went to look for &quot;canonical&quot; Ramachandran plots, I discovered most researchers overlay a &quot;contour&quot; plot of allowed torsional angles (see below). My two questions are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How are the distributions (ie. &lt;strong&gt;countours&lt;/strong&gt;) of protein secondary structures calculated/derived? (ie. How do I &quot;cluster&quot; these into beta-sheets, alpha helices?)&lt;/li&gt;&#xA;&lt;li&gt;How can I derive them myself?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uWdUk.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uWdUk.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6610" LastActivityDate="2020-04-01T09:55:45.447" Title="How did researchers derive the Ramachandran &quot;validation&quot; contours?" Tags="|structural-biology|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12771" PostTypeId="1" AcceptedAnswerId="12774" CreationDate="2020-03-29T18:11:45.243" Score="1" ViewCount="89" Body="&lt;p&gt;I have a simple task which seems very complicated now &#xA;Two data-frame I have One contains the Chromosome coordinates of distal enhancer elements and other one rlog values of accessibility which I want to map.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The distal coordinates &lt;a href=&quot;https://drive.google.com/file/d/1LWJIZuLv2Uatt24KtmmTRZCBhpjBV-FN/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;data&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dput(head(a))&#xA;structure(list(Chr = c(&quot;chr6&quot;, &quot;chr11&quot;, &quot;chr20&quot;, &quot;chr20&quot;, &quot;chr10&quot;, &#xA;&quot;chrX&quot;), Start = c(154986726, 70314780, 15895696, 40706992, 29132014, &#xA;129525939), End = c(154987062, 70315059, 15895987, 40707264, &#xA;29132266, 129526265)), class = c(&quot;tbl_df&quot;, &quot;tbl&quot;, &quot;data.frame&quot;&#xA;), row.names = c(NA, -6L))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The Rlog_ATAC_location &lt;a href=&quot;https://drive.google.com/file/d/1SA4dm-q40F9tnpGHg_RLZt9QagCAOe92/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;data&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Chr    Start    End  CMP1  CMP2  CMP3  CMP4  CMP5  CMP6  GMP1  GMP2  GMP3  GMP4  GMP5  GMP6  HSC1  HSC2  HSC3  HSC4  HSC5  HSC6&#xA;  &amp;lt;chr&amp;gt;  &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;&#xA; chr1   16103  16348 2.70  2.99  2.87  1.88   2.06  1.94 2.63   2.37  2.61  1.81  1.31  1.21 1.89  1.67  1.95  1.09  2.17  0.861&#xA; chr1   96493  96796 0.976 0.325 0.943 0.216  1.33  1.77 1.66   1.33  1.47  2.39  1.76  2.37 0.939 0.198 1.37  0.439 0.868 0.756&#xA; chr1  271114 271432 1.95  2.62  2.04  2.29   2.49  2.86 2.12   2.89  2.30  2.25  3.16  3.50 0.695 1.03  0.491 0.785 1.15  1.00 &#xA; chr1  273026 273333 0.740 1.88  1.48  1.71   2.50  2.88 2.81   2.66  2.89  2.07  3.02  3.61 1.24  1.40  0.984 1.35  0.651 1.53 &#xA; chr1  274265 274599 0.954 1.59  2.06  0.227  2.05  1.19 0.906  1.60  1.62  1.99  2.05  2.47 0.920 0.211 1.06  0.426 1.99  0.529&#xA; chr1  402478 402771 2.98  3.26  3.01  3.07   2.51  2.76 3.29   2.65  2.34  3.03  3.00  2.99 2.54  3.04  2.88  2.96  2.94  2.68 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I tried inner join ,merge etc but i can map the chromosome cooridnates but the expression values comes empty&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tmp &amp;lt;- merge(a,Rlog_ATAC_location, by=c(&quot;Chr&quot;,&quot;Start&quot; ,&quot;End&quot;), all.x=TRUE)&#xA;&#xA;Chr  Start    End CMP1 CMP2 CMP3 CMP4 CMP5 CMP6 GMP1 GMP2 GMP3 GMP4 GMP5 GMP6 HSC1 HSC2 HSC3 HSC4 HSC5 HSC6 Mono1 Mono2 Mono3&#xA; chr1  16104  16348   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA    NA    NA    NA&#xA; chr1  96494  96796   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA    NA    NA    NA&#xA; chr1 271115 271432   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA    NA    NA    NA&#xA; chr1 273027 273333   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA    NA    NA    NA&#xA; chr1 274266 274599   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA    NA    NA    NA&#xA; chr1 402479 402771   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA   NA    NA    NA    NA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Tried left join as well i get same result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to map the coordinates as well as the samples columns which contains the expression&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion or help would be really appreciated &lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2020-03-30T04:55:52.203" LastActivityDate="2020-03-30T13:56:14.843" Title="Merging based on Chromosome coordinates" Tags="|r|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12772" PostTypeId="2" ParentId="12769" CreationDate="2020-03-29T19:59:10.770" Score="1" Body="&lt;p&gt;There are two types of data you can get from GTEx. The first is summary statistics, basically the slopes and p-values you see in the boxplot. You can also get expression levels for all genes measured in GTEx.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, in order to plot boxplot by yourself, you also need genotype data. This is not publicly available because of privacy issues. In order to get it, you have to apply to it on dbGaP. I think you can find relevant information on the GTEx website.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastEditorUserId="4766" LastEditDate="2020-03-30T02:46:44.063" LastActivityDate="2020-03-30T02:46:44.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12774" PostTypeId="2" ParentId="12771" CreationDate="2020-03-30T13:56:14.843" Score="2" Body="&lt;p&gt;In your ATAC data frame, you have &lt;code&gt;chr1   16103  16348&lt;/code&gt;. But in your merged data frame, you have &lt;code&gt;chr1   16104  16348&lt;/code&gt;. Is it possible that you &lt;code&gt;a&lt;/code&gt; is 1-based whereas your ATAC data frame is 0-based? This can happen in bioinformatics and can be annoying. This might be why you have a lot of NA in your merged data frame. You should subtract 1 from the start position of your &lt;code&gt;a&lt;/code&gt; data frame while keeping the end position unchanged. Then try to merge again.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should always be aware of your data format to know whether it is 1-based or 0-based.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-03-30T13:56:14.843" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12775" PostTypeId="2" ParentId="884" CreationDate="2020-03-30T17:29:18.263" Score="1" Body="&lt;p&gt;&lt;strong&gt;Use any REMARK number not reserved for something else.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The REMARK field was initially for general remarks. In the format description from 1992 only the numbers 1 to 3 were reserved for specific things, with the note:&#xA;&lt;em&gt;Other general commentary is given in higher numbered REMARKs&lt;/em&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, in the next years, other remark numbers got reserved (and sometimes retired) for various specific annotations. The content of remarks was getting more and more formalized, to make it easier to parse by computers. The &lt;a href=&quot;http://www.wwpdb.org/documentation/file-format&quot; rel=&quot;nofollow noreferrer&quot;&gt;specifications&lt;/a&gt; from 1996 and 1998 recommended:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Non-standard remark annotations, or those with no clearly-defined&#xA;  topic or assigned remark number, appear with remark number 6 or&#xA;  greater, but less than remark number 100.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'd still follow this advice.&#xA;It is generally understood that when parsing PDB files one may have custom REMARK records.&#xA;I've seen also programs that output remarks without numbers - and this also doesn't cause problems with any reasonable PDB reader.&#xA;On the other hand, using remarks with numbers reserved for very specific things&#xA;(such as remark 250 in the most upvoted answer) can cause confusion.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The current spec (from 2012) is a precise description of files served by the wwPDB. It doesn't attempt to define a format for other uses, such as passing data between programs. In the section about &lt;a href=&quot;https://www.wwpdb.org/documentation/file-format-content/format33/remarks1.html#REMARKs%206%20-%2099&quot; rel=&quot;nofollow noreferrer&quot;&gt;REMARKs 6-99&lt;/a&gt; it has the last bullet point saying: &quot;REMARKs 6-99 are no longer for use of free text annotation&quot;, because these remarks were removed from the old PDB entries. But it doesn't prevent from using it outside of the wwPDB.&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2020-03-30T17:29:18.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12776" PostTypeId="1" CreationDate="2020-03-30T18:06:08.533" Score="8" ViewCount="197" Body="&lt;p&gt;I am a researcher in high-performance computing (with very little bioinformatics background), and I am trying to understand what are the current biggest computational bottlenecks of software used for research on the topic of Covid-19 (testing, vaccine development, drug identification, etc...). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know this is a very broad question, but I am looking for the biggest common denominators. Said differently, if you could pick a few operations / algorithms / data structures to speed up Covid-19 research and they would magically run 1000 times faster, what would they be?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: links to books, papers, and scientific literature are very welcomed.&lt;/p&gt;&#xA;" OwnerUserId="7408" LastEditorUserId="3848" LastEditDate="2020-11-30T22:28:27.080" LastActivityDate="2020-11-30T22:28:27.080" Title="Understanding some of the computational bottlenecks of Covid-19 research" Tags="|sequence-alignment|sequence-analysis|software-recommendation|covid-19|sars-cov-2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12777" PostTypeId="2" ParentId="12776" CreationDate="2020-03-30T20:23:28.343" Score="5" Body="&lt;p&gt;I can only speak of drug design (and even then I am terrible at turning down the jargon).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the case of drug design, this is pretty much plan C. Namely, none of compounds that entered clinical trial at the start of the year work (let's call this plan A) and none of the vaccines that are entering now clinical trial work (let's call this plan B although vaccination prevents so is a better solution). Clinical trials take a lot time as they need to be thorough (cf. the &lt;a href=&quot;https://en.wikipedia.org/wiki/Thalidomide&quot; rel=&quot;noreferrer&quot;&gt;infamous thalidomide&lt;/a&gt; from the days with more lax testing)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are several bottlenecks in drug design. These are technical based on our imperfect understanding of things, which results in less than ideal models. CPU time is not really an issue. Empirical validation is required at many steps along the way. This is greatly increased by automation. The XChem team at Diamond light source have done &lt;a href=&quot;https://www.diamond.ac.uk/covid-19/for-scientists/Main-protease-structure-and-XChem.html&quot; rel=&quot;noreferrer&quot;&gt;a fragment-screening against the SARS-CoV-2 main protease&lt;/a&gt; at an insane pace: things are changing fast!&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Drug screen&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;There are many ways to make a drug. In one way, a test system was set up (a reporter assay) that gives you a measurable read-out, say mm of clearance of bacteria on a Petri dish (viability) around a cellulose dish soaked in different separated natural extracts —and you might find that &lt;em&gt;Penicillium chrysogenum&lt;/em&gt; has a compound of interest. What you are doing is drug screen. Screening a large panel of compounds (of varying purity) does not necessarily require knowing the protein target.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Rational drug design&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Fast forward to this decade and it becomes possible to predict the free energy release upon binding of a small molecule against a protein —mostly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This requires a solved 3D structure of the protein by X-ray crystallography. Crystallising a protein is a tricky business, as a pure protein sample has to come out of solution in a folded form to form a neat lattice as opposed to just making unfolded gunk. What salt conditions work? Anyone's guess. Some have done deep learning on it, but the best solution is to have a robot try a very large range of combinations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have a protein structure you can predict what fits into the part that you want to target, say an active site or an interface. This is not done by simple cartesian geometric shape fitting like LEGO bricks as there are many molecular forces involved. So a mathematical model of these forces is made, called a &lt;a href=&quot;https://en.wikipedia.org/wiki/Force_field_(chemistry)&quot; rel=&quot;noreferrer&quot;&gt;force field&lt;/a&gt;. Many have been made, a lot of study has gone into these, but they are not perfect. It is all made worse by the fact that we cannot really model water properly —most models use a proxy (implicit solvent) or use non-polarisable water.&#xA;This is all while made worse by the fact that a protein is not static but wobbles (following the same mathematical models). So a ligand does not slot in really like a key in a lock, but has an induced fit. Modelling all this is accurately is problematic, but one can still try: calculating where exactly a ligand binds to a protein is called for which one gets energy difference upon binding. And one can try an inordinate amount of small molecules —&lt;a href=&quot;https://zinc.docking.org/&quot; rel=&quot;noreferrer&quot;&gt;Zinc DB&lt;/a&gt; contains 230 million purchasable compounds. This is called a virtual screen. One can even start with the natural substrate and &quot;weaponise it&quot; by adding chemical groups that do a chemical reaction that breaks the protein. However, a metabolite is very different from a drug (cf. &lt;a href=&quot;https://en.wikipedia.org/wiki/Lipinski%27s_rule_of_five&quot; rel=&quot;noreferrer&quot;&gt;Lipinski's rule of five&lt;/a&gt;) as the latter must cross membranes etc. But generally the accuracy is not great without multiple steps of empirical validation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To improve the results from a virtual screen the most common approach is to do a fragment screen to guide it. This is to automatedly soak protein crystals with a large library of small compounds and see which &lt;a href=&quot;https://www.diamond.ac.uk/covid-19/for-scientists/Main-protease-structure-and-XChem/Interactive-views.html&quot; rel=&quot;noreferrer&quot;&gt;crystal contains a bound hit&lt;/a&gt;. The hits can be expanded into bigger fragments in a virtual screens and the short list of compounds tested (these are not cheap) by soaking and crystallising again.&#xA;In the case of Covid19 virus, the cool thing is that this whole process is open in the aforementioned current campaign (Diamond is a government owned synchrotron and not a big pharma). What is really nice about this is that for the second cycle comp-chemists from everywhere have done their own guided virtual screen and have submit compounds (&lt;a href=&quot;https://covid.postera.ai/covid/submissions&quot; rel=&quot;noreferrer&quot;&gt;which you can see publicly&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should be said that drug screens that are not rational design are still a thing. The recent antibiotic-discovering deep-learning paper in Cell]&lt;a href=&quot;https://linkinghub.elsevier.com/retrieve/pii/S0092-8674(20)30102-1&quot; rel=&quot;noreferrer&quot;&gt;6&lt;/a&gt; is such an example, although it should be said that it was a feat not only in AI but also in automated empirical testing.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Future&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Automation&lt;/strong&gt; via better and open software will definitely improve. With a small drive possibly from the dissemination of easy to use and to code off-the-shelf electronics, such as Rasperry Pis or Arduinos. But mainly with more complex machines such as the &lt;a href=&quot;https://opentrons.com/&quot; rel=&quot;noreferrer&quot;&gt;Opentron system&lt;/a&gt; (a python-controlled robot).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But indubitably where the greatest improvement can come is &lt;strong&gt;better mathematical models&lt;/strong&gt;. Possibly even with deep learning. After all researches were amused last year-ish (2018 competition) when DeepMind &quot;won&quot; the protein 3D structure (fold) modelling competition (CASP13) and there have definitely been some really nice results in the field of docking with hybrid models. So deep-learning may still surprise us yet...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Lastly, &lt;strong&gt;open science&lt;/strong&gt; will be a strong driver: using premade packages or tools on GitHub and code snippets from StackOverflow etc. will accelerate these.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-03-30T20:45:41.750" LastActivityDate="2020-03-30T20:45:41.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12778" PostTypeId="2" ParentId="12776" CreationDate="2020-03-30T22:36:15.000" Score="1" Body="&lt;p&gt;&lt;strong&gt;Drug screening&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I understand the idea behind this, particularly with deep learning playing such an important role in computation (see GPU). Moreover, molecular dynamics is famous for sumping computer power, so yeah this is a bottleneck. However, this is not trivial because the wet-lab work is such a long way off. The first crystal/cyro/EM/NMR structure that will be resolved is &lt;strong&gt;Spike&lt;/strong&gt; (S) and this would not be a 'traditional' drug target, its a vaccine candidate (okay influenza has surface antigen drug target, Tamiflu). Expressing S is a priority (vaccines) and solving the structure is a by-product and its not that big, plus the majority of it is water-soluable so it will cystalise nicely (its a surface antigen). Obtaining the structure for a more 'traditional drug target' which would be ORFa and ORFb (RNA dependent RNA polymerase ... and other replication associated proteins), will be difficult, because they are huge (ORFa + ORFb are 20 Kb of RNA, which for an RNA virus is twice as big as the genomes for most RNA viruses) and for crystals ... solubility(?). Ultimately, how do you cystalise a protein that 4kb of amino acids - well you'd have to do it in subdomains, you'd probably need NMR/ really hot EM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;General&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;HPC is a big field and it highlights the importance of one area most struggle with which is GPU (graphic processing unit) for deep learning (DL) applications. HPC's role molecular dynamics/drug discovery) and a mixture of DL and MD. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Traditional CPU 'farms' most of use have access to, is less critical because whether it takes 2 days per calculation or 2 hours isn't a huge issue (but its very seriously annoying indeed)..&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most pressing need for computational power will be epidemiological modelling, particularly if someone is pursing a microsimulation. The firepower would enable them to explore a large number of scenarios and parameter explorations. Apart from a vaccines then epi modelling is the hot-topic of the moment.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-03-31T10:32:05.317" LastActivityDate="2020-03-31T10:32:05.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12781" PostTypeId="1" CreationDate="2020-03-31T10:25:03.357" Score="0" ViewCount="74" Body="&lt;p&gt;Is there an amino-acid shift that will 99% of the time end up resulting the same biochemical function/structure and implications? Like Alanine replaced by Leucine will code in 99% of the cases(at any position that first has Alanine) towards proteins that will function the same without big percentual changes in how good it binds or catalyses.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I do have a big amount of mutations. I'd like to filter them out on chance of biological/phenotypical impact, and since some amino acids look like good substitutes by eye (if I look at the molecular structure)&lt;/p&gt;&#xA;" OwnerUserId="6573" LastEditorUserId="6573" LastEditDate="2020-04-07T08:32:51.690" LastActivityDate="2020-04-07T08:32:51.690" Title="Is there a aminoacid mutation that forms similar protein?" Tags="|mutations|" AnswerCount="1" CommentCount="3" ClosedDate="2020-04-04T13:11:09.743" ContentLicense="CC BY-SA 4.0" />
  <row Id="12782" PostTypeId="2" ParentId="388" CreationDate="2020-03-31T11:26:12.363" Score="2" Body="&lt;p&gt;BioJulia maintains &lt;a href=&quot;https://github.com/BioJulia/BioFmtSpecimens&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioFmtSpecimens&lt;/a&gt;. From the README:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Bioinformatics is rife with formats and parsers for those formats. These parsers don't always agree on the definitions of these formats, since many lack any sort of formal standard.&lt;/p&gt;&#xA;&lt;p&gt;This repository aims to consolidate a collection of format specimens to create a unified set of data with which to test software against. Testing against the same cases is a first step towards agreeing on the details and edge cases of a format.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4622" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2020-03-31T11:26:12.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12783" PostTypeId="2" ParentId="694" CreationDate="2020-03-31T11:31:48.683" Score="2" Body="&lt;p&gt;You can do this with &lt;a href=&quot;https://github.com/BioJulia/BioStructures.jl&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioStructures.jl&lt;/a&gt; in Julia. All the 6 transformations between PDB/mmCIF/MMTF are possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, PDB to MMTF:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using BioStructures&#xA;struc = read(in_filepath, PDB)&#xA;writemmtf(out_filepath, struc)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;mmCIF to MMTF:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;using BioStructures&#xA;struc = read(in_filepath, MMCIF)&#xA;writemmtf(out_filepath, struc)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2020-03-31T11:31:48.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12785" PostTypeId="1" AcceptedAnswerId="12791" CreationDate="2020-03-31T14:12:24.610" Score="3" ViewCount="303" Body="&lt;p&gt;All,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As the first step in my Snakemake pipeline, I generate small fastq files from all input fastq.gz files. This is to run a quick round of &quot;infer_experiment.py&quot; (from RSeQC, to get the kit strandedness). However, Snakemake says this step fails. I think I know why, it is because a command like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zcat fastq_raw/20190011-043_S43_R1_001.fastq.gz | head -n 400000 &amp;gt; infer_strandedness/20190011-043_S43_subset_R1.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Ends with an error: &lt;code&gt;gzip: stdout: Broken pipe&lt;/code&gt; and Snakemake uses Bash strict mode. (Snakemake reports: &lt;code&gt;(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)&lt;/code&gt;). So my question is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to avoid this, or, are there other truncating methods that do finish cleanly? Preferebly they are also efficient (seqtk offers subsampling but this parses the entire fastq file!).&lt;/p&gt;&#xA;" OwnerUserId="3964" LastEditorUserId="3964" LastEditDate="2020-03-31T14:36:10.390" LastActivityDate="2020-03-31T22:10:12.203" Title="truncate fastq.gz file without error in Snakemake" Tags="|snakemake|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12787" PostTypeId="2" ParentId="12764" CreationDate="2020-03-31T16:50:46.180" Score="1" Body="&lt;p&gt;There is a &lt;a href=&quot;http://seqanswers.com/forums/showthread.php?t=39749&quot; rel=&quot;nofollow noreferrer&quot;&gt;post about this on seqanswers&lt;/a&gt;. The poster says the following about cuffmerge behavior:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;I kept the temporary files while running cuffmerge. It builgs a SAM file from transcripts.gtf files. Then it seems first it reads it as BAM since I read samtools take the files as BAM default. Then it finds it is not a BAM so it tries SAM. Then it can read it and resume the task. &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So it sounds like the BAM file is generated from GTF as a kludge inside cuffmerge, not the other way around. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Did you change any of the intermediate files or delete anything? Is there maybe a different cuffmerge run that it might be colliding with?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Did you get any output? it sounds like this error is frequently observed when actually the command completes fine, e.g. &lt;a href=&quot;https://www.biostars.org/p/189139/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/189139/&lt;/a&gt;. See other posts by googling the problem maybe, e.g. &lt;a href=&quot;https://www.biostars.org/p/344439/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/344439/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I know this isn't a proper answer but it was too involved for a comment.)&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-03-31T16:50:46.180" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12788" PostTypeId="2" ParentId="12781" CreationDate="2020-03-31T21:30:41.193" Score="4" Body="&lt;p&gt;A mutation is most often likely to be either neutral (no effect) or destabiling (protein misfolds). Changes in activity require changes at the active site or the entranceway to it —which are often only a handful of residues, so are very uncommon.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the answer to your question is most mutations keep the activity, although a large fraction break it. While to get a new activity the process is more fluid and is not boolean.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NB. Not all proteins are enzymes, but for simplicity I will concentrate on enzymes.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Neutrality and core vs. surface&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;The outer residues of a protein, the surface residues, are very tolerant to mutations. Unless they are part of an binding surface (interface) or the change results in a charge change that is. The core residues on the other hand are more sensitive to mutations.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Destabilisation&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;One can do a scan &lt;em&gt;in silico&lt;/em&gt; or &lt;em&gt;in vitro&lt;/em&gt; or &lt;em&gt;in vivo&lt;/em&gt; to determine the mutational landscape of a protein, which can take various forms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This being Bioinformatics stack overflow I'll give an in silico example. The following for example is a heatmap I did some time back with &lt;a href=&quot;https://www.rosettacommons.org/docs/latest/application_documentation/design/pmut-scan-parallel&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rosetta pmut_scan&lt;/a&gt;, which is a classic tool for this and shows in the y axis the residue it was mutated to and on the x axis the position. A larger change in &lt;a href=&quot;https://en.wikipedia.org/wiki/Protein_folding&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gibbs free energy of folding&lt;/a&gt;) relative to wild type (∆∆G) means it is destabiling, but anything less than 5 kcal/mol is neutral. As you can see many are neutral, but some half are destabiling —with a caveat for proline, which require a backbone change.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kZMGf.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kZMGf.jpg&quot; alt=&quot;landscape&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the in vitro setting one speaks of melting temperature (DSC assays) or turnover number over Michaelis constant (enzyme specific assay). In the in vivo setting one speaks of a fitness cost of a mutation, i.e. the protein is less active so the organism grows less.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Drug resistance&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Drugs inhibit enzymes by binding the active site. Therefore resistance is often costly. There are some nice papers that look into the fitness cost and mutational landscape of HIV protease or &amp;beta;-lactamase. In this situation a strong epistasis is seen. That is, a first mutation that confers resistance to the drug greatly destabilises the protein, while a second compensates for the first mutation.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Change in function&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;An enzyme can catalyse a reaction well on a given substrate, because it was evolved this way (technically the encoding gene evolved, but shh). This is the physiological activity. However, enzymes can also catalyse side activities on similar substrates with a lower turnover and higher Michaelis constant. This is especially true for substrates it was not selected against. This is called &quot;enzyme promiscuity&quot;. The tolerance of the physiological activity to mutation is called &quot;robustness&quot;, while the ability to increase promiscuous activities is &quot;plasticity&quot;. Different enzymes have a different balance of the two.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So a mutation may change the strength of the physiological activity, but may result in a new promiscuous activity that can be used as a stepping stone for selection to increase this activity. Therefore, as you can see it is not a binary switch, except for a few very specific case.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Best amino acid for changes?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Proline is a very strange amino acid as it is a secondary amino acid. It lacks a hydrogen on the backbone nitrogen so it cannot form the required hydrogen bond for an &amp;alpha;-helix. And can adopt a cis-conformation with minimal penalty (whereas it is about 5 kcal/mol for other chiral amino acids). So changes to this amino acid are often very destabilising.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Protein cores tend to be hydrophobic so changes to the hydrophobic residues are often deleterious, especially when going from a small residue to a larger one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The remaining residues are frequently found on the surface so may be neutral. However, some are found in the active site and may be able to shift the parameters of the ligand space for that enzyme. A change in charge may result in an increased specificity for a different set of substrates for example. A size change from a larger amino acid to a smaller one may result in a broader range of promiscuous activities —mutations to alanine for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So ironically, polar and charged residues as they are frequently found on surfaces are the residues which are most often neutral, but in a small amount of cases they may change activity.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-03-31T22:55:03.153" LastActivityDate="2020-03-31T22:55:03.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12789" PostTypeId="1" AcceptedAnswerId="12790" CreationDate="2020-03-31T21:35:46.073" Score="1" ViewCount="189" Body="&lt;p&gt;main.nf is &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env nextflow&#xA;&#xA;params.input = ''&#xA;tchan = Channel.fromFile(params.input)&#xA;&#xA;tchan.println()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have a.bam, b.bam and c.bam  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I run: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nextflow run main.nf --input *bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But it only prints &lt;code&gt;a.bam&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;According to the &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#frompath&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;, wildcard character it is interpreted as a glob path matcher&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What am I doing wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you &lt;/p&gt;&#xA;" OwnerUserId="2970" LastActivityDate="2020-03-31T21:47:11.337" Title="nextflow `fromPath` not passing all files with *" Tags="|nextflow|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12790" PostTypeId="2" ParentId="12789" CreationDate="2020-03-31T21:47:11.337" Score="2" Body="&lt;p&gt;got it!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I should run:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nextflow run main.nf --input &quot;*bam&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;instead of &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;nextflow run main.nf --input *bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2970" LastActivityDate="2020-03-31T21:47:11.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12791" PostTypeId="2" ParentId="12785" CreationDate="2020-03-31T22:10:12.203" Score="1" Body="&lt;p&gt;Snakemake is using bash strict mode to ensure best practice error reporting in shell commands. Unfortunately some tools violate bash strict mode.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More details of the error, &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/project_info/faq.html#my-shell-command-fails-with-exit-code-0-from-within-a-pipe-what-s-wrong&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Alternatively, you can use if you don't want disable the strict mode.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;zcat file.fastq.gz  | awk '(NR&amp;lt;=400000)' &amp;gt; subset.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2020-03-31T22:10:12.203" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12793" PostTypeId="2" ParentId="11757" CreationDate="2020-04-01T06:32:55.897" Score="2" Body="&lt;p&gt;I tried &lt;a href=&quot;https://benjjneb.github.io/dada2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dada2&lt;/a&gt; and is not bad (if you know R).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://qiime2.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;QIIME2&lt;/a&gt; is also an option.&#xA;Many other are available, the choice might also depend on your sample and your exact question.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For functional profiling you may try &lt;a href=&quot;https://github.com/picrust/picrust2/wiki&quot; rel=&quot;nofollow noreferrer&quot;&gt;picrust2&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1140" LastActivityDate="2020-04-01T06:32:55.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12794" PostTypeId="2" ParentId="12770" CreationDate="2020-04-01T09:55:45.447" Score="3" Body="&lt;p&gt;To expand on Matteo Ferla's comment as an answer, there are two approaches to getting these &quot;allowed&quot; contours:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;From probabilities found in observed crystal structures. You could imagine plotting a Ramachandran plot for the whole PDB and colouring the points by assigned secondary structure from &lt;a href=&quot;https://swift.cmbi.umcn.nl/gv/dssp&quot; rel=&quot;nofollow noreferrer&quot;&gt;DSSP&lt;/a&gt;. The contours would then be regions that encompass e.g. 99% of alpha-helical residues, or 99% of all residues. In doing this you would notice for example that glycine can adopt more regions of the Ramachandran space that other amino acids. &lt;a href=&quot;https://www.ebi.ac.uk/thornton-srv/software/PROCHECK&quot; rel=&quot;nofollow noreferrer&quot;&gt;PROCHECK&lt;/a&gt; is software that scores a protein based partly on the likelihood of the phi/psi angles of each residue.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;From reasoning about the chemistry of the secondary structure of proteins. Only certain phi/psi combinations allow the i -&gt; i + 4 pattern in an alpha-helix or the alternating up/down sidechain pattern in a beta-strand.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2020-04-01T09:55:45.447" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12795" PostTypeId="1" AcceptedAnswerId="12796" CreationDate="2020-04-01T10:54:55.097" Score="-2" ViewCount="160" Body="&lt;p&gt;I am trying to run a script in python but I getting this error&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;loan-mac-13:vcf2maf fi1d18$ python&#xA;Python 2.7.10 (default, Feb 22 2019, 21:55:15) &#xA;[GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14)] on darwin&#xA;Type &quot;help&quot;, &quot;copyright&quot;, &quot;credits&quot; or &quot;license&quot; for more information.&#xA;&amp;gt;&amp;gt;&amp;gt; import os&#xA;&amp;gt;&amp;gt;&amp;gt; os.chdir(&quot;/Users/fi1d18/Downloads/vcf2maf-master/vcf2maf&quot;)&#xA;&amp;gt;&amp;gt;&amp;gt; &#xA;&amp;gt;&amp;gt;&amp;gt; &#xA;&amp;gt;&amp;gt;&amp;gt; os.getcwd()&#xA;'/Users/fi1d18/Downloads/vcf2maf-master/vcf2maf'&#xA;&amp;gt;&amp;gt;&amp;gt; vcf2maf.py -h&#xA;Traceback (most recent call last):&#xA;  File &quot;&amp;lt;stdin&amp;gt;&quot;, line 1, in &amp;lt;module&amp;gt;&#xA;NameError: name 'vcf2maf' is not defined&#xA;&amp;gt;&amp;gt;&amp;gt; python vcf2maf.py -h&#xA;  File &quot;&amp;lt;stdin&amp;gt;&quot;, line 1&#xA;    python vcf2maf.py -h&#xA;                 ^&#xA;SyntaxError: invalid syntax&#xA;&amp;gt;&amp;gt;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where I am doing wrong?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for any help&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By your comments I run&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sorry says that Last login: Wed Apr  1 11:50:14 on ttys000&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pc-133-235:~ fi1d18&lt;span class=&quot;math-container&quot;&gt;$ pwd&#xA;/Users/fi1d18&#xA;pc-133-235:~ fi1d18$&lt;/span&gt; &#xA;pc-133-235:~ fi1d18&lt;span class=&quot;math-container&quot;&gt;$ cd Downloads&#xA;pc-133-235:Downloads fi1d18$&lt;/span&gt; python vcf2maf.py -h&#xA;Traceback (most recent call last):&#xA;  File &quot;vcf2maf.py&quot;, line 19, in &amp;lt;module&amp;gt;&#xA;    import vcf&#xA;ImportError: No module named vcf&#xA;pc-133-235:Downloads fi1d18$&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-04-01T12:04:35.700" LastActivityDate="2020-04-01T12:04:35.700" Title="Running a script in python" Tags="|python|vcf|software-usage|" AnswerCount="1" CommentCount="3" ClosedDate="2020-04-04T13:23:27.087" ContentLicense="CC BY-SA 4.0" />
  <row Id="12796" PostTypeId="2" ParentId="12795" CreationDate="2020-04-01T11:53:52.413" Score="4" Body="&lt;p&gt;You are trying to run a script file in the python console. That doesn't make sense. The console is for running commands manually, interactively. A script is a collection of commands. You can't just run that by name in the console. If you want to use it, copy the &lt;em&gt;contents&lt;/em&gt; of your script and try to run them in the console. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The way you run a python &lt;em&gt;script file&lt;/em&gt; is simply:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python vcf2maf.py -h&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, assuming you have made the file executable and it has the right &lt;a href=&quot;https://en.wikipedia.org/wiki/Shebang_(Unix)&quot; rel=&quot;nofollow noreferrer&quot;&gt;shebang line&lt;/a&gt;, just:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;./vcf2maf.py -h&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The specific error you are getting is because &lt;code&gt;foo.bar&lt;/code&gt; in Python means &quot;run the &lt;code&gt;bar&lt;/code&gt; method from the object &lt;code&gt;foo&lt;/code&gt;. So &lt;code&gt;vcf2maf.py&lt;/code&gt; means run &lt;code&gt;py&lt;/code&gt; from &lt;code&gt;vcf2maf&lt;/code&gt;. Since there is no object defined with the name &lt;code&gt;vcf2maf&lt;/code&gt;, you get the &lt;code&gt;name 'vcf2maf' is not defined&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2020-04-01T11:53:52.413" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12797" PostTypeId="1" AcceptedAnswerId="12798" CreationDate="2020-04-01T12:18:58.890" Score="2" ViewCount="417" Body="&lt;p&gt;I ran freebayes on bacterial data and got a vcf file. Some of the SNPs found have &lt;code&gt;AC=0&lt;/code&gt;, but in the bam file I can see that there are alternative alleles, so in my understanding it supposed to be &lt;code&gt;AC&amp;gt;0&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;example:&#xA;in vcf file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CHROM  POS     ID      REF     ALT     QUAL    FILTER  INFO    FORMAT  55509   55508   55506   55507&#xA;scaffold1       78      .       A       C       9.27683e-13     .       AB=0;ABP=0;AC=0;AF=0;AN=3;AO=466;CIGAR=1X;DP=2254;DPB=2254;DPRA=0;EPP=20.9226;EPPR=29.3274;GTI=0;LEN=1;MEANALT=2;MQM=50.3412;MQMR=45.5379;NS=3;NUMALT=1;ODDS=2080.75;PAIRED=0.899142;PAIREDR=0.533932;PAO=0;PQA=0;PQR=0;PRO=0;QA=16347;QR=63107;RO=1783;RPL=203;RPP=19.7856;RPPR=291.853;RPR=263;RUN=1;SAF=465;SAP=1006.25;SAR=1;SRF=894;SRP=3.04075;SRR=889;TYPE=snp       GT:GQ:DP:AD:RO:QR:AO:QA:GL      .       0:126.704:776:626,149:626:22118:149:5255:0,-1217.7      0:126.704:834:663,170:663:23671:170:6004:0,-1224.79     0:126.704:644:494,147:494:17318:147:5088:0,-770.853&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but with samtools tview:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aS63N.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aS63N.png&quot; alt=&quot;samtools tview screenshot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can someone tell me if this makes sense and why it happens?&lt;/p&gt;&#xA;" OwnerUserId="7326" LastEditorUserId="3541" LastEditDate="2020-04-01T15:26:16.080" LastActivityDate="2020-04-01T15:26:16.080" Title="Allele count is zero" Tags="|vcf|snp|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12798" PostTypeId="2" ParentId="12797" CreationDate="2020-04-01T14:42:24.680" Score="3" Body="&lt;p&gt;The AC is the number of alternate alleles from called genotypes, you can see this in the vcf header:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;##INFO=&amp;lt;ID=AC,Number=A,Type=Integer,Description=&quot;Total number of &#xA;alternate alleles in called genotypes&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Assuming diploid, if we have 3 individuals, 0/0 , 0/1, 0/0, AC will be 1, because it detects 1 count of alternate allele.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you look at your genotype calls, all are 0, so AC = 0&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-04-01T14:42:24.680" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12799" PostTypeId="2" ParentId="10986" CreationDate="2020-04-01T15:48:18.407" Score="1" Body="&lt;p&gt;Hey i solve this simply by use of SOV alghoritm. &lt;a href=&quot;https://www.researchgate.net/publication/319526145_A_modified_definition_of_Sov_a_segment-based_measure_for_protein_secondary_structure_prediction_assessment&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.researchgate.net/publication/319526145_A_modified_definition_of_Sov_a_segment-based_measure_for_protein_secondary_structure_prediction_assessment&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;cheers&lt;/p&gt;&#xA;" OwnerUserId="1224" LastActivityDate="2020-04-01T15:48:18.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12801" PostTypeId="2" ParentId="2083" CreationDate="2020-04-01T22:29:24.910" Score="0" Body="&lt;p&gt;I was trying the code from @Alex Reynolds, however, I got retrieved an error: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedmap --count --exact SRR800856_*.bed | awk '{s+=$1}END{print s;}' &amp;gt; exact_counts.txt&#xA;May use bedmap --help for more help.&#xA;&#xA;Error: Option SRR800856_ngs_te_mapper_nonredundant.bed does not start with '--'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But when I tried with just two bed files: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bedmap --count --exact SRR800856_ngs_te_mapper_nonredundant.bed SRR800856_popoolationte_nonredundant.bed &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't get any error. &#xA;Could you tell me what I should add. &#xA;Thank in advance for your time. &#xA;P.D. As I am new here I couldn't get to comment on the specific thread. &lt;/p&gt;&#xA;" OwnerUserId="8531" LastActivityDate="2020-04-01T22:29:24.910" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12802" PostTypeId="1" CreationDate="2020-04-02T00:46:04.253" Score="2" ViewCount="564" Body="&lt;p&gt;Between the &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/vignettes/qvalue/inst/doc/qvalue.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;qvalue&lt;/a&gt; package and the &lt;a href=&quot;https://stat.ethz.ch/R-manual/R-patched/library/stats/html/p.adjust.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;p.adjust&lt;/a&gt; function, which is more appropriate to use when trying to calculate the q-values of a dataset? According to the manual for the q-value package, the q-value calculated is not an adjusted p-value which is what the p.adjust function would return. There seems to be an equal amount of people that use one or the other, but I wanted to see other opinions that could help clarify what would be most appropriate to use. &lt;/p&gt;&#xA;" OwnerUserId="7036" LastActivityDate="2024-01-28T05:08:16.047" Title="qvalue and p.adjust functions" Tags="|r|bioconductor|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12803" PostTypeId="2" ParentId="12802" CreationDate="2020-04-02T02:43:59.273" Score="0" Body="&lt;p&gt;&lt;code&gt;p.adjust&lt;/code&gt; can calculate adjusted p-values with different methods. I think you can choose Bonferroni correction and BH correction, for instance. BH correction gives you False Discovery Rate (FDR), which is one of the most commonly used I guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Q-value is different and cannot be calculated by &lt;code&gt;p.adjust&lt;/code&gt;. The idea is that when you have many p-values to adjust, they form a distribution. If all p-values come from the null hypothesis, the p-value distribution is going to be uniform on [0, 1]. But usually, our p-values are a mixture of the null distribution and alternative distribution. The parameter &lt;span class=&quot;math-container&quot;&gt;$\pi_0$&lt;/span&gt; describes the proportion of p-values (or tests) that comes from the null hypothesis. To calculated q-values, the algorithm first estimates &lt;span class=&quot;math-container&quot;&gt;$\pi_0$&lt;/span&gt;, then calculates the q-value.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that if you set &lt;span class=&quot;math-container&quot;&gt;$\pi_0=1$&lt;/span&gt;, which means that you believe all the p-values come from the null distribution, then q-value is equal to FDR.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think most people do not know about the meaning and details of q-values and it might be sufficient to use BH procedure in most cases.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-04-02T02:43:59.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12804" PostTypeId="2" ParentId="7026" CreationDate="2020-04-02T06:49:41.693" Score="1" Body="&lt;p&gt;Also potentially useful is the solution proposed here: &lt;a href=&quot;https://github.com/satijalab/seurat/issues/1888&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/satijalab/seurat/issues/1888&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, save DotPlot to an object, and your percentages are found within. &lt;/p&gt;&#xA;" OwnerUserId="8532" LastActivityDate="2020-04-02T06:49:41.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12805" PostTypeId="2" ParentId="11715" CreationDate="2020-04-02T07:29:32.640" Score="0" Body="&lt;p&gt;Linear regression hypothesis is that  effect of a&#xA; variable is comprised as a constant component plus variability. In other  words,  the independent variable (s) is deemed to reflect a fixed (constant) part - determined as population effect size - and a variable part - random effect. Further, the term - &#xA;Linear Mixed(regression)model  is used to reflect use of continuous variables and nominal variables acting as independent variables that influence  the dependent  variable.  THE CONTINUOUS VARIABLE  is  comprised  of population effect size - and a variable part - random effect . The nominal or categories based variable effect  is probabilistic and we use the term probabilistic variable for such variables. Sometimes academics mistakenly call its effect on dependent  variable as random effect.&#xA;Finally, this may be noted that  principles of ANOVA and probability  have been combined to formulate linear Mixed effects models.&lt;/p&gt;&#xA;" OwnerUserId="7374" LastEditorUserId="7374" LastEditDate="2020-04-02T08:55:39.513" LastActivityDate="2020-04-02T08:55:39.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12806" PostTypeId="1" AcceptedAnswerId="12815" CreationDate="2020-04-02T11:39:18.893" Score="-1" ViewCount="72" Body="&lt;p&gt;I have a big data frame like this for copy number (exome seq)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    &amp;gt; head(df)&#xA;  Chromosome    Start      End&#xA;1 1             64613  5707515&#xA;2 1           5712940  5732322&#xA;3 1           5732399 16383682&#xA;4 1          16383742 16389288&#xA;5 1          16390813 16830026&#xA;6 1          16830201 17278112&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also have coordinate of human exams like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Exone   Choromosome     Start_of_exone  End_of_exone &#xA;uc001aaa.3  chr1    11873   12227&#xA;uc001aaa.3  chr1    12612   12721&#xA;uc001aaa.3  chr1    13220   14409&#xA;uc010nxr.1  chr1    11873   12227&#xA;uc010nxr.1  chr1    12645   12697&#xA;uc010nxr.1  chr1    13220   14409&#xA;uc010nxq.1  chr1    11873   12227&#xA;uc010nxq.1  chr1    12594   12721&#xA;uc010nxq.1  chr1    13402   14409&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Actually I need to interset my copy number coordinates and human exome coordinates to see there are how many exomes in these coordinates&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used this code but all returns 0&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;snp_table &amp;lt;- expanse coordinate file&#xA;genomic_ranges &amp;lt;- df&#xA;&#xA;how_many_in_range &amp;lt;- function(coords){&#xA;    coords = as.vector(coords)&#xA;    sum(snp_table&lt;span class=&quot;math-container&quot;&gt;$Chromosome == coords[1] &amp;amp; snp_table$&lt;/span&gt;Position &amp;gt; as.numeric(coords[2]) &amp;amp; snp_table$Position &amp;lt; as.numeric(coords[3]))&#xA;}&#xA;&#xA;genomic_ranges$number_of_snps &amp;lt;- apply(genomic_ranges, 1, how_many_in_range)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can you help me?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-04-02T19:11:27.980" LastActivityDate="2020-04-03T09:51:40.077" Title="Calculating the number of probes for a given genomic range" Tags="|r|bash|cnv|exome|genomicranges|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12808" PostTypeId="1" AcceptedAnswerId="12826" CreationDate="2020-04-02T20:44:02.457" Score="0" ViewCount="147" Body="&lt;p&gt;Can anyone offer advice on what the below error code relates to. I am using co-loc R package with 2 GWAS datasets. Using coloc.abf function with pvalues, MAF, sample size and case/control cohort. Thank you &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  results &amp;lt;- coloc.abf(  dataset1=list(snp=as.character(1_subset[,2]),pvalues=1_subset[,11],N=1_data_subset&lt;span class=&quot;math-container&quot;&gt;$sample_size, type=&quot;cc&quot;,s=0.327,MAF=1_data_subset[,12]),   dataset2=list(snp=as.character(2_subset[,2]),pvalues=2_subset[,8],N=2_subset$&lt;/span&gt;sample_size,s=0.4319,MAF=1_subset[,12], type=&quot;cc&quot;),p12=1e-06)&#xA;&#xA;&#xA;error code: Error in data.frame(V, z, r, lABF) : &#xA;  arguments imply differing number of rows: 1994, 1992&#xA;In addition: Warning messages:&#xA;1: In 2 * N * f :&#xA;  longer object length is not a multiple of shorter object length&#xA;2: In 2 * N * f * (1 - f) :&#xA;  longer object length is not a multiple of shorter object length&#xA;3: In r * z^2 :&#xA;  longer object length is not a multiple of shorter object length&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6227" LastEditorUserId="6227" LastEditDate="2020-04-03T08:47:33.207" LastActivityDate="2020-04-04T13:16:17.567" Title="co-localisation error code using co-loc" Tags="|r|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12810" PostTypeId="1" CreationDate="2020-04-02T22:39:45.930" Score="1" ViewCount="164" Body="&lt;p&gt;I'm trying to create a umap for single cell data from human samples and ptx samples. I can get the umap to where it shows the umap with the different clusters but I want to show where the ptx samples are and where the human samples are. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code is as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#create the Seurat object&#xA;&#xA;OD_10K_HUMAN &amp;lt;- CreateSeuratObject(counts = HUMAN_OD_10K.data, min.cells = 0, project = &quot;human&quot;)&#xA;SD_5K_HUMAN &amp;lt;- CreateSeuratObject(counts = HUMAN_SD_5K.data, min.cells = 0, project = &quot;human&quot;)&#xA;BNL.5K &amp;lt;- CreateSeuratObject(counts = SD_BNL_5K.data, min.cells = 0, project = &quot;ptx&quot;)&#xA;BNL.10K &amp;lt;- CreateSeuratObject(counts = OD_BNL_10K.data, min.cells = 0, project = &quot;ptx&quot;)&#xA;BNM.10K &amp;lt;- CreateSeuratObject(counts = OD_BNM_10K.data, min.cells = 0, project = &quot;ptx&quot;)&#xA;BNM.5K &amp;lt;- CreateSeuratObject(counts = SD_BNM_5K.data, min.cells = 0, project = &quot;ptx&quot;)&#xA;&#xA;#merge data&#xA;scData &amp;lt;- merge(BNL.10K, y = c(BNL.5K, BNM.10K, BNM.5K, SD_5K_HUMAN, OD_10K_HUMAN), add.cell.ids = c(&quot;A&quot;, &quot;B&quot;, &quot;C&quot;, &quot;D&quot;, &quot;E&quot;, &quot;F&quot;), project = &quot;HTB2876&quot;)&#xA;&#xA;mark the mito genes&#xA;mito.genes &amp;lt;- grep(pattern = &quot;^MT-&quot;, x = rownames(x = scData), value = TRUE)&#xA;length(mito.genes)&#xA;scData[[&quot;percent.mt&quot;]] &amp;lt;- PercentageFeatureSet(scData, pattern = &quot;^MT-&quot;)&#xA;&#xA;scData[[&quot;log_nCount_RNA&quot;]] &amp;lt;- log2(scData[[&quot;nCount_RNA&quot;]]+1)&#xA;&#xA;# remove cells with &amp;lt;200 RNA molecules, or &amp;gt;6000 molecules, or &amp;gt;30% mito&#xA;scData &amp;lt;- subset(scData, subset = nFeature_RNA &amp;gt; 200 &amp;amp; nFeature_RNA &amp;lt; 6000 &amp;amp; percent.mt &amp;lt; 30)&#xA;scData &amp;lt;- NormalizeData(scData, normalization.method = &quot;LogNormalize&quot;, scale.factor = 10000)&#xA;scData &amp;lt;- FindVariableFeatures(scData, selection.method = &quot;vst&quot;, nfeatures = 2000)&#xA;&#xA;all.genes &amp;lt;- rownames(scData)&#xA;scData &amp;lt;- ScaleData(scData, features = VariableFeatures(object = scData), vars.to.regress = c(&quot;nCount_RNA&quot;))&#xA;scData &amp;lt;- RunPCA(scData, features = VariableFeatures(object = scData))&#xA;#DimPlot(scData, reduction = &quot;pca&quot;)&#xA;&#xA;numPC = 20&#xA;scData &amp;lt;- FindNeighbors(scData, dims = 1:numPC)&#xA;scData &amp;lt;- FindClusters(scData, resolution = 0.4)&#xA;scData &amp;lt;- RunUMAP(scData, dims = 1:numPC)&#xA;&#xA;DimPlot(scData, reduction = &quot;umap&quot;, label = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6827" LastEditorUserId="6827" LastEditDate="2020-04-02T22:58:35.630" LastActivityDate="2020-10-19T09:08:25.897" Title="umap highlighting two different models" Tags="|scrnaseq|seurat|umap|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12811" PostTypeId="2" ParentId="12806" CreationDate="2020-04-03T01:41:23.490" Score="1" Body="&lt;p&gt;Regardless of what your specific data is, the goal of your analysis is really to find overlaps between two data frames of genomic coordinations. My favorite package for doing this is &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/GenomicRanges.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GenomicRanges&lt;/a&gt;. It is very fast and can take care of different strands (if your data contains strand information).&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-04-03T01:41:23.490" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12812" PostTypeId="1" CreationDate="2020-04-03T01:47:14.053" Score="1" ViewCount="35" Body="&lt;p&gt;I performed several BLAST searches and identified several genes of interest in a Whole Genome Shotgun (wgs) project. I know that gene &lt;em&gt;&lt;strong&gt;X&lt;/strong&gt;&lt;/em&gt; is located on scaffold 1234, from nucleotide 1 - 2250, while gene &lt;em&gt;&lt;strong&gt;Y&lt;/strong&gt;&lt;/em&gt; is located on scaffold 2571, from nucleotide 3380 - 5790.&lt;/p&gt;&#xA;&lt;p&gt;Given this information is it possible to determine if these genes are close to each other in the genome?&lt;/p&gt;&#xA;&lt;p&gt;Many thanks for your help,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Leah&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5116" LastEditorUserId="3848" LastEditDate="2020-11-15T12:53:26.543" LastActivityDate="2020-11-15T12:53:26.543" Title="Is it possible to determine the genomic context of a gene in a Whole Genome Shotgun project?" Tags="|genome|wgs|scaffold|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12813" PostTypeId="1" CreationDate="2020-04-03T05:11:42.033" Score="1" ViewCount="5290" Body="&lt;p&gt;I am trying to merge two bed files (more in future) to one. my bed files are something like : &#xA;&lt;a href=&quot;https://i.stack.imgur.com/dcAAi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dcAAi.png&quot; alt=&quot;result of macs pea calling&quot;&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I need to merge them in a way to have the shared chromosome location. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to do that ?&lt;/p&gt;&#xA;" OwnerUserId="8543" LastActivityDate="2020-04-04T15:17:45.307" Title="merging two/or more bed file into one bed file" Tags="|sequence-alignment|samtools|sequence-analysis|bedtools|macs2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12814" PostTypeId="2" ParentId="12813" CreationDate="2020-04-03T07:33:53.447" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;Convert each of your Excel spreadsheets to tab-delimited text files (&lt;a href=&quot;https://www.howtogeek.com/79991/convert-an-excel-spreadsheet-to-a-tab-delimited-text-file/&quot; rel=&quot;nofollow noreferrer&quot;&gt;how-to&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://cygwin.com/install.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Install and run Cygwin&lt;/a&gt;, if using Windows. If you are using OS X, open up the Terminal app.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/installation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Install the BEDOPS toolkit&lt;/a&gt; to get sorting and set operation tools (if you're not averse to using them).&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Strip out the Microsoft line endings, strip the header, and sort each of the &lt;em&gt;N&lt;/em&gt; files:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ tr -d '\r' &amp;lt; fileN.tsv | tail -n+2 - | sort-bed - &amp;gt; fileN.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Merge the files:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops --merge file1.bed file2.bed ... fileN.bed &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The file &lt;code&gt;answer.bed&lt;/code&gt; will contain all merged regions from files &lt;code&gt;file1.bed&lt;/code&gt; through &lt;code&gt;fileN.bed&lt;/code&gt;. Merging will be done on intervals for each unique chromosome name.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2020-04-03T07:33:53.447" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12815" PostTypeId="2" ParentId="12806" CreationDate="2020-04-03T09:51:40.077" Score="0" Body="&lt;p&gt;This command gave me the number of axons&#xA;I hope this is correct&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please correct me if I am wrong&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;how_many_in_range &amp;lt;- function(coords){&#xA;    coords = as.vector(coords)&#xA;    sum(human_exome_coordinates&lt;span class=&quot;math-container&quot;&gt;$Choromosome == coords[1] &amp;amp; human_exome_coordinates$&lt;/span&gt;Position &amp;gt; as.numeric(coords[2]) &amp;amp; human_exome_coordinates$Position &amp;lt; as.numeric(coords[3]))&#xA;}&#xA;&#xA;genomic_ranges$number_of_exon &amp;lt;- apply(segment_mean_table, 1, how_many_in_range)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2020-04-03T09:51:40.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12816" PostTypeId="1" CreationDate="2020-04-03T11:31:12.323" Score="3" ViewCount="745" Body="&lt;p&gt;As far as I understood, for most assembly programs, the scaffolding step takes into consideration paired-end information in order to get from contigs (contiguous sequences) to scaffolds (longer sequences that might have some N-filled gaps).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My assembly software of choice, MEGAHIT, uses paired-end information to build the contigs, but it does not output a standard scaffold. So I am wondering the following, related things:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-is it meaningful to run a scaffold program on the output of MEGAHIT? I imagine there might be some instances in which paired-end information could span a gap.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-Which software would you recommend for it? (I've tried soapdenovo2 and SSPACE, but they appear not to be actively maintained so I have the issue of 'it doesn't work and I can't do anything about it')&lt;/p&gt;&#xA;&#xA;&lt;p&gt;-Could relevant information (eg. two contigs being connected by pair-end information) be recovered by alternative and perhaps more user-friendly means, such as exploring the assembly graph with Bandage?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your time!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2021-05-29T11:03:25.323" Title="Is there any value in scaffolding the output contigs of MEGAHIT assembler given a metagenomic dataset?" Tags="|assembly|metagenome|scaffold|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12817" PostTypeId="1" CreationDate="2020-04-03T16:28:03.567" Score="2" ViewCount="730" Body="&lt;p&gt;I am having issues with some prokaryote reference genome databases (exact names : ref_prok_rep_genomes.*), that I downloaded from the NCBI website : &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/blast/db/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nlm.nih.gov/blast/db/&lt;/a&gt;. Files in this database go from &lt;code&gt;ref_prok_rep_genomes.00&lt;/code&gt; to &lt;code&gt;ref_prok_rep_genomes.12&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Linux command line, I was able to launch successfully the bash command  &lt;code&gt;blastn -query (my_query) -db ref_prok_re_.genomes.* -out (my_output)&lt;/code&gt; with indexes from &lt;code&gt;00&lt;/code&gt; to &lt;code&gt;05&lt;/code&gt;, but this does not work from indexes &lt;code&gt;06&lt;/code&gt; to &lt;code&gt;12&lt;/code&gt; (with the same query and output files). The exact same thing occurs with &lt;code&gt;blastdbcmd -info -db ref_prok_rep_genomes.*&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each time the error displayed is the same: &#xA;&lt;code&gt;BLAST Database error: Input db vol does not match lmdb vol&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All the &lt;code&gt;ref_prok_rep_genomes&lt;/code&gt; files are in the same folder. I downloaded them using &lt;code&gt;wget&lt;/code&gt; and extracted them using &lt;code&gt;tar -zxvf&lt;/code&gt;. All &lt;code&gt;ref_prok_rep_genomes.*&lt;/code&gt; databases (indexes &lt;code&gt;00&lt;/code&gt; to &lt;code&gt;12&lt;/code&gt;) have &lt;code&gt;nhr&lt;/code&gt;, &lt;code&gt;nin&lt;/code&gt;, &lt;code&gt;nnd&lt;/code&gt;, &lt;code&gt;nni&lt;/code&gt;, &lt;code&gt;nog&lt;/code&gt; and &lt;code&gt;nsq&lt;/code&gt; files. The folder also contains &lt;code&gt;ref_prok_rep_genomes&lt;/code&gt; files (without numbers), with  &lt;code&gt;nal&lt;/code&gt;, &lt;code&gt;ndb&lt;/code&gt;, &lt;code&gt;nos&lt;/code&gt;, &lt;code&gt;not&lt;/code&gt;, &lt;code&gt;ntf&lt;/code&gt; and &lt;code&gt;nto&lt;/code&gt; extensions respectively, as well as &lt;code&gt;taxdb.btd&lt;/code&gt; and &lt;code&gt;taxdb.bti&lt;/code&gt; files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you guys have an idea of what's going on?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you very much for your time :)&lt;/p&gt;&#xA;" OwnerUserId="8547" LastActivityDate="2020-04-03T16:28:03.567" Title="Cannot blast against specific NCBI databases" Tags="|database|blast|bash|phylogenetics|blastn|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12818" PostTypeId="2" ParentId="11757" CreationDate="2020-04-03T18:16:35.323" Score="0" Body="&lt;p&gt;The best thing to do is a phylogenetic tree in my opinion. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The trendy approach and the one deemed acceptable in publication is deep learning for precisely this application, such as the pipeline published &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-018-2182-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. This approach is attractive in 16S NGS metagenomics classification because it uses advanced approaches in deep learning, notably convolution neural networks (CNN) and deep belief networks (DBN). I don't really understand DBN, but I suspect it is a variant of recurrent neural networks (RNN) such as long-short term memory (LSTM). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said all that the accuracy is low, viz. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For instance, at the genus level, both CNN and DBN reached 91.3% of&#xA;  accuracy with AMP short-reads, whereas RDP classifier obtained 83.8%&#xA;  with the same data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;91% accuarcy for a deep learning neural network is minimal, but it will be quick and will do precisely what you want with little fuss. A phylogenetic tree on the otherhand can take several hundred hours of total processor time. The thing about a 16S tree is that it is 100% accurate, because you can manually identify &quot;grey zones&quot; to assign their classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can retrain CNN models and in future this will likely resolve the accuracy issue. However, I don't know about about DBN and I don't think RSS can retrained, more specifically I don't know how to do it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-04-03T18:28:01.597" LastActivityDate="2020-04-03T18:28:01.597" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12819" PostTypeId="2" ParentId="12816" CreationDate="2020-04-03T19:12:05.267" Score="1" Body="&lt;p&gt;&lt;strong&gt;Update 2: It looks like your approach &lt;a href=&quot;https://github.com/voutcn/megahit/issues/24&quot; rel=&quot;nofollow noreferrer&quot;&gt;has actually been suggested here&lt;/a&gt; as one way to use the PE information. I guess megahit may not be really using the PE information anyways. I still believe that it's kind of weird but other people do suggest it, so maybe it's worth trying what Torsten suggests.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that scaffolding with PE information from the reads used as input to Megahit is somewhat sketchy. You could still try it, but I'd be worried about artifacts, just because de novo assemblers are so heuristic.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I think that it is perfectly ok to scaffold using &lt;em&gt;orthogonal&lt;/em&gt; data. Here are some examples of orthogonal data:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A reference assembly (weird for metagenomics but still possibly ok).&lt;/li&gt;&#xA;&lt;li&gt;long reads, e.g. ONT or PacBio (low coverage probably fine).&lt;/li&gt;&#xA;&lt;li&gt;Proximity ligation data, e.g. Hi-C reads.&lt;/li&gt;&#xA;&lt;li&gt;Mate-pair reads (also lead to artifacts at a somewhat high rate, but technically valid).&lt;/li&gt;&#xA;&lt;li&gt;Linked-reads (e.g. 10X).&lt;/li&gt;&#xA;&lt;li&gt;Probably other methods as well.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The tools used in each case would be somewhat different. For a little more information about how you might use this, &lt;a href=&quot;https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1006994&quot; rel=&quot;nofollow noreferrer&quot;&gt;here is a recent review of tech&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Full disclosure: I work for a company that sells Hi-C kits for such applications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: Realized that I missed one part of the Q. I think that visually exploring the assembly using e.g. bandage is always a good idea. Quite possibly you can make some scaffolding decisions that way, but to me it sounds somewhat painful to do, especially in a metagenome where there are going to be a lot of multiple-branching collapsed regions. &lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2020-04-03T19:17:48.863" LastActivityDate="2020-04-03T19:17:48.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12820" PostTypeId="1" CreationDate="2020-04-03T20:55:09.770" Score="1" ViewCount="198" Body="&lt;p&gt;Multiple sources (see for instance &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5958914/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; or &lt;a href=&quot;https://spectrum.ieee.org/computing/software/the-desperate-quest-for-genomic-compression-algorithms&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;) discuss how genetic data will have scalability problems, given the huge file size of the human genome. The most straightforward encoding (see &lt;a href=&quot;https://stackoverflow.com/questions/8954571/how-much-storage-would-be-required-to-store-a-human-genome&quot;&gt;here&lt;/a&gt;) of the human genome requires about 700 Mb. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I came across this &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/25/2/274/218156&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; claiming to be able to store the human genome in about 4 Mb, having a reference genome and exploiting the fact that all human genomes are mostly equal. This paper is much older than the the other references discussing the scalability problems. Why is this technique not widely used? &lt;/p&gt;&#xA;" OwnerUserId="8544" LastActivityDate="2021-09-12T14:02:45.280" Title="Compressing the human genome to few megabytes" Tags="|phylogenetics|human-genome|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12821" PostTypeId="1" AcceptedAnswerId="12828" CreationDate="2020-04-03T22:09:58.730" Score="0" ViewCount="100" Body="&lt;p&gt;I am developing a wrapper for a bioinformatics tool writen in python3.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The native Galaxy-project runs on python2.7.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How can I the tool that requires python3 in a galaxy-project instance? &lt;/p&gt;&#xA;" OwnerUserId="2970" LastEditorUserId="2970" LastEditDate="2020-04-04T22:27:07.723" LastActivityDate="2020-04-04T22:27:07.723" Title="run a tool with python&gt;=3.6 on galaxy-project instance that runs on python2.7" Tags="|python|galaxy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12822" PostTypeId="1" AcceptedAnswerId="12823" CreationDate="2020-04-04T00:58:15.597" Score="1" ViewCount="56" Body="&lt;p&gt;I have 2 groups of patients; responders to chemotherapy and non-responders&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have calculated cancer cell fraction (CCF) of a set of driver genes (by variant allele frequency) for each group and I have &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(dat)&#xA;    Response       CCF&#xA;1 Responders 1.0000000&#xA;2 Responders 0.5413323&#xA;3 Responders 1.0000000&#xA;4 Responders 1.0000000&#xA;5 Responders 1.0000000&#xA;6 Responders 1.0000000&#xA;&#xA;unique(dat$Response)&#xA;&#xA;&#xA;[1] Responders    Nonresponders&#xA;Levels: Nonresponders Responders&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If CCF &gt; 0.95 , the mutation is clonal otherwise sub clonal&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to show how many clone and subclone are in these two groups by box plot or something similar like below&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Q3smM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Q3smM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried this which was nonsense &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(dat, aes(x=Response, y=CCF)) + &#xA;    geom_boxplot()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can you help me?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2020-04-04T03:43:50.257" Title="A box plot of qualitative variables" Tags="|r|ggplot2|software-usage|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12823" PostTypeId="2" ParentId="12822" CreationDate="2020-04-04T03:43:50.257" Score="1" Body="&lt;p&gt;I'm not sure that boxplot will be the more appropriate representation, as you will end with two numbers (count of Clonal and SubClonal) per groups of patients.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One solution will be to first create a new categorical variable based on CCF values using for example &lt;code&gt;ifelse&lt;/code&gt; statement for example by writing:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;df$Clonal &amp;lt;- ifelse(df$CCF &amp;gt; 0.95,&quot;Clone&quot;,&quot;SubClonal&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then you can get the count of each Clone and subclone for each responders and non responders by using &lt;code&gt;table&lt;/code&gt;: &lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;DF &amp;lt;- as.data.frame(table(df[,c(&quot;Response&quot;,&quot;Clonal&quot;)]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can finally convert these new values as percentage by grouping according to the Response. For example, using &lt;code&gt;dplyr&lt;/code&gt;, you can do something like that:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(dplyr)&#xA;&#xA;DF %&amp;gt;% group_by(Response) %&amp;gt;% mutate(Freq_percent = Freq / sum(Freq))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And finally, you can plot it as a barchart or single point in &lt;code&gt;ggplot2&lt;/code&gt;.&#xA;Hope it helps you to figure it out how to deal with your data. &lt;/p&gt;&#xA;" OwnerUserId="6437" LastActivityDate="2020-04-04T03:43:50.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12824" PostTypeId="1" CreationDate="2020-04-04T09:18:03.407" Score="6" ViewCount="474" Body="&lt;p&gt;How small can a compressed file containing the human genome be?&#xA;I'm aware that this question cannot actually be answered, since it is asking for the &lt;a href=&quot;https://en.wikipedia.org/wiki/Kolmogorov_complexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kolmogorov complexity&lt;/a&gt; of the human genome, which is not computable. So reformulating:&#xA;What is the largest known lower bound that we can put on the size of a file containing the human genome?&lt;/p&gt;&#xA;&lt;p&gt;An example answer could be: on average we know that the human genome contain at least this amount X of randomness, therefore it is not possible to do better than Y bytes.&lt;/p&gt;&#xA;&lt;p&gt;I'm agnostic on which human genome, but I would say without polymorphisms to make it simpler.&lt;/p&gt;&#xA;&lt;p&gt;The compressed sequence would need to be stand alone, without using a reference (I asked a question on that topic here: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/12820/compressing-the-human-genome-to-few-megabytes&quot;&gt;Compressing the human genome to few megabytes&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;There should be a program able to compress and decompress the genome in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Lossless_compression&quot; rel=&quot;nofollow noreferrer&quot;&gt;lossless way&lt;/a&gt;. Apart from that, any kind of modelling and coding is allowed.&lt;/p&gt;&#xA;" OwnerUserId="8544" LastEditorUserId="73" LastEditDate="2021-05-26T05:03:32.890" LastActivityDate="2022-05-27T15:43:05.637" Title="Theoretical limit of human genome compression" Tags="|phylogenetics|human-genome|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12826" PostTypeId="2" ParentId="12808" CreationDate="2020-04-04T13:16:17.567" Score="0" Body="&lt;p&gt;In your &lt;code&gt;dataset2 = list()&lt;/code&gt;, the MAF argument should be &lt;code&gt;MAF=2_subset[,12]&lt;/code&gt; instead of &lt;code&gt;MAF=1_subset[,12]&lt;/code&gt;. Since your two datasets have different numbers of lines, this is what causes the error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am not sure whether this is a typo in your code, or it is because you do not have MAF information in the second dataset. What I like to do before COLOC is to filter two datasets so they have the same number of lines, and then feed those filtered datasets to the coloc function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(Also just out of curiosity, is &lt;code&gt;1_subset&lt;/code&gt; a data frame for your GWAS results? I think it is not recommended to have variable names starting with a number.)&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-04-04T13:16:17.567" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12827" PostTypeId="2" ParentId="12813" CreationDate="2020-04-04T15:17:45.307" Score="2" Body="&lt;p&gt;In my opinion, the easiest way to merge bed files is to use &lt;code&gt;bedtools merge&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-04-04T15:17:45.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12828" PostTypeId="2" ParentId="12821" CreationDate="2020-04-04T16:25:36.397" Score="2" Body="&lt;p&gt;Each Galaxy tool uses its own (typically conda-based) environment, which will normally not use the same python as Galaxy itself (note that Galaxy can run under python 3). An example of this is deepTools, which is python 3 only but will happily run under a python 2 Galaxy since conda is being used to create a python 3 environment for it. It's best to distribute your tool via bioconda and then Galaxy will be easily able to install it.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-04-04T16:25:36.397" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12829" PostTypeId="1" CreationDate="2020-04-04T18:46:37.747" Score="1" ViewCount="799" Body="&lt;p&gt;I have a vcf file that contains SNPs from chromosome 2 for 2000 samples. I want to create a genotype matrix from this vcf file.&lt;/p&gt;&#xA;&lt;p&gt;The file is in zipped format as well so that I can use &lt;code&gt;bcftools&lt;/code&gt;. I want to filter snips with&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;MAF&amp;gt;= 1%&lt;/li&gt;&#xA;&lt;li&gt;more than 2 alleles.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I'm not quite sure why I have to filter for more than 2 alleles, is it common to do this if I'm trying to perform PCA analysis?&lt;/p&gt;&#xA;&lt;p&gt;The instruction I'm using right now is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools query -e'AF&amp;lt;0.01' -e'AF&amp;gt;0.99' chr2.vcf.gz | \&#xA;  sed 's/0|0/0/g' | sed 's/0|1/1/g' | sed 's/1|0/1/g' | sed 's/1|1/2/g' | \&#xA;  grep -v &amp;quot;|&amp;quot; &amp;gt; extract.tab&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How do I only extract the multi-allelic cases?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I have a way to do it using bcftools view -m2 chr2.vcf.gz but the results seem fishy.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How do I correct this?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2519" LastEditorUserId="3848" LastEditDate="2020-11-18T20:48:11.097" LastActivityDate="2020-11-18T20:48:11.097" Title="Create a genotype matrix from VCF file using bcftools" Tags="|vcf|software-usage|bcftools|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12830" PostTypeId="1" CreationDate="2020-04-04T20:57:06.053" Score="1" ViewCount="278" Body="&lt;p&gt;I have several mab files from medip-seq. they supposed to be replicates and I have to draw their correlation. First I used multiBamSummary from deeptools to merge all the bam files based on bi/bed. Now when I am using the plotCorrelation I do not get any file. the argument is :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plotCorrelation --corData myMultiBamSumary.npz --corMethod spearman --whatToPlot scatterplot --skipZeros --removeOutliers --plotFile Myplot --plotFileFormat pdf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is that the correct strategy to pot the correlation? Is there any suggestion?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks alot&lt;/p&gt;&#xA;" OwnerUserId="8543" LastEditorUserId="77" LastEditDate="2020-04-04T21:06:50.203" LastActivityDate="2020-04-04T22:54:28.787" Title="plot correlation between several bam files" Tags="|sam|sequence-alignment|sequence-analysis|chip-seq|deeptools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12831" PostTypeId="2" ParentId="12830" CreationDate="2020-04-04T21:09:02.700" Score="1" Body="&lt;p&gt;That's a completely normal way to plot correlation between them. Spearman correlation was a wise choice on your part, since it's rank based and you don't have to care as much about normalization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: It turns out that matplotlib is broken and I need to have a new deepTools release to work around that. That should be available sometime Sunday.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2020-04-04T22:54:28.787" LastActivityDate="2020-04-04T22:54:28.787" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="12832" PostTypeId="1" CreationDate="2020-04-04T21:17:34.070" Score="-4" ViewCount="78" Body="&lt;p&gt;I have a data frame&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Response TO CHEMOTHERAPY    Gene    CCF&#xA;Responders                  POLQ    1&#xA;Responders                 CDKN2A   1&#xA;Responders                  TP53    1&#xA;Responders              CCDC102B    0.906729075&#xA;Non-responders           CCDC102B   1&#xA;Non-responders           CCDC102B   1&#xA;Non-responders           CCDC102B   1&#xA;Non-responders             TSHZ3    1&#xA;Non-responders             PIK3CA   1&#xA;Responders                 SLIT2    1&#xA;Non-responders              SLIT2   1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have &lt;code&gt;two groups of patients&lt;/code&gt;; &lt;code&gt;Responders to chemotherapy and non-responders&lt;/code&gt;. For each group I have calculated &lt;code&gt;cancer cell faction (CCF)&lt;/code&gt; for a fixed set of genes so that an individual gene has several reads in each group&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;code&gt;CCF = 1&lt;/code&gt; the gene clonal and if CCF &amp;lt; 1 gene is sub clonal, for a given gene for each group, I therefore would have the number of clonal and sub clonal for this gene, for example 2 clonal 7 sub clonal which would be 2/7 something like the below heat map&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6DzE8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6DzE8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By your kindly code I produced this&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GW0pY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GW0pY.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which is not well informative&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is another potential example heat map &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HeabT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HeabT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which show the clonal frequency for each group&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By the way thanks for your kindness &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know how to use &lt;code&gt;dput()&lt;/code&gt; but this is my complete data&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/ol7m4nlkuyuy85a/c1.txt?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/ol7m4nlkuyuy85a/c1.txt?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-04-08T12:23:30.430" LastActivityDate="2020-04-08T12:23:30.430" Title="Attempting to generate a heat map like in this figure" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="2" ClosedDate="2020-04-07T08:54:15.750" ContentLicense="CC BY-SA 4.0" />
  <row Id="12833" PostTypeId="1" AcceptedAnswerId="12835" CreationDate="2020-04-05T00:35:18.850" Score="-1" ViewCount="32" Body="&lt;p&gt;I have a very big clinical data like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Date.Death  Date Of Operation&#xA;02/08/2015  27/11/2012&#xA;02/08/2015  27/11/2012&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to know how many weeks each patient has lived after operation&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then tried &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; t1 &amp;lt;- as.POSIXct(&quot;27/11/2012&quot;)&#xA;&amp;gt; t2 &amp;lt;- as.POSIXct(&quot;02/08/2015&quot;)&#xA;&amp;gt; as.double(difftime(t2,t1,unit=&quot;weeks&quot;))&#xA;[1] -1317.571&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; span &amp;lt;- interval(as.POSIXct(&quot;27/11/2012&quot;), as.POSIXct(&quot;02/08/2015&quot;))&#xA;&amp;gt; span&#xA;[1] 0027-11-20 LMT--0002-08-20 LMT&#xA;&amp;gt; t &amp;lt;- as.period(span, unit=&quot;day&quot;)&#xA;&amp;gt; t&#xA;[1] &quot;-9223d 0H 0M 0S&quot;&#xA;&amp;gt; t / as.period(dweeks(1))&#xA;estimate only: convert to intervals for accuracy&#xA;[1] -1317.571&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Both give wrong number of weeks&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Manually calculating one by one is painful and error prone &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You know anyway to get the number of weeks?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2020-04-05T04:13:27.917" Title="Getting the number of weeks from a time span" Tags="|r|" AnswerCount="1" CommentCount="2" ClosedDate="2020-04-07T08:53:52.143" ContentLicense="CC BY-SA 4.0" />
  <row Id="12835" PostTypeId="2" ParentId="12833" CreationDate="2020-04-05T04:13:27.917" Score="2" Body="&lt;p&gt;I think your error is located in your use of &lt;code&gt;as.POSIXct&lt;/code&gt; which gives you wrong date formats because you did not provide a format in your function:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; as.POSIXct(&quot;27/11/2012&quot;)&#xA;[1] &quot;27-11-20 LMT&quot;&#xA;&#xA;&amp;gt; as.POSIXct(&quot;27/11/2012&quot;, format = &quot;%d/%m/%Y&quot;)&#xA;[1] &quot;2012-11-27 PST&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So, you need to first convert your date in the appropriate date format and then you will be able to calculate the time interval. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Personally, I prefer to manipulate dates with &lt;code&gt;lubridate&lt;/code&gt; package because the name of their function is a little bit more intuitive to me. For example, if you have a date to the format &quot;Day/Month/Year&quot;, you can use &lt;code&gt;dmy&lt;/code&gt; function. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;With your example, you can do:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(lubridate)&#xA;library(dplyr)&#xA;&#xA;df &amp;lt;- data.frame(Date.Death = c(&quot;02/08/2015&quot;,&quot;02/05/2015&quot;),&#xA;                 Date.Operation = c(&quot;27/11/2012&quot;,&quot;27/11/2012&quot;))&#xA;&#xA;df %&amp;gt;% mutate_all(~dmy(.)) %&amp;gt;% &#xA;  mutate(Diff_Weeks = interval(Date.Operation,Date.Death) / weeks(1),&#xA;                                      Diff_days = Date.Death - Date.Operation)&#xA;&#xA;  Date.Death Date.Operation Diff_Weeks Diff_days&#xA;1 2015-08-02     2012-11-27   139.7143  978 days&#xA;2 2015-05-02     2012-11-27   126.5714  886 days&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6437" LastActivityDate="2020-04-05T04:13:27.917" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12838" PostTypeId="1" CreationDate="2020-04-05T11:47:40.113" Score="4" ViewCount="1049" Body="&lt;p&gt;On &lt;a href=&quot;https://www.gisaid.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GISAID&lt;/a&gt; they classified the coronavirus using 4 clades(S, G, V, Other).&lt;br /&gt;&#xA;I downloaded around 1,000 complete genomes of the coronavirus from &lt;a href=&quot;https://www.gisaid.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GISAID&lt;/a&gt; and I would like to classify each one as belonging to one of the 4 clades(S, G, V, Other).&lt;/p&gt;&#xA;&lt;p&gt;On the top left of the image below is the classification of the 3 clades(S, G, V).&lt;br /&gt;&#xA;More specifically, the three strains were classified based on three specific variants(This &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/12840/8554&quot;&gt;answer&lt;/a&gt; was taken from my previous question):&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Strain S, variant ORF8-L84S: a variant in the gene &amp;quot;ORF8&amp;quot; which changes the leucine (L) residue at position 84 of the gene's protein product to a serine (S).&lt;/li&gt;&#xA;&lt;li&gt;Strain G, variant S-D614G: a variant in the gene &amp;quot;S&amp;quot; which changes the aspartic acid (D) residue at position 614 of the gene's protein product to a glycine (G).&lt;/li&gt;&#xA;&lt;li&gt;Strain V, variant NS3-G251V: a variant in the gene &amp;quot;NS3&amp;quot; which changes the glycine (G) residue at position 251 of the  gene's protein product   to a valine (V).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My problem is that using this method of classification for the coronavirus genome I am seeking information on the protein products of the genes in the genome.&lt;/p&gt;&#xA;&lt;p&gt;So to be more specific here are my concise questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How can I classify a genome as &amp;quot;Strain S&amp;quot; using only the complete genome assembly, without any information about its proteins?&lt;/li&gt;&#xA;&lt;li&gt;How can I classify a genome as &amp;quot;Strain G&amp;quot; using only the complete genome assembly, without any information about its proteins?&lt;/li&gt;&#xA;&lt;li&gt;How can I classify a genome as &amp;quot;Strain V&amp;quot; using only the complete genome assembly, without any information about its proteins?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rxDYn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rxDYn.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This picture was taken from this &lt;a href=&quot;https://sars-cov-2.netlify.com/images/GISAID_hCoV-19_Analysis_Update_2020-04-04_1500UTC.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDF&lt;/a&gt; on &lt;a href=&quot;https://www.gisaid.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GISAID&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8554" LastEditorUserId="3848" LastEditDate="2023-04-07T17:16:03.017" LastActivityDate="2023-04-07T17:16:03.017" Title="How can I classify the 3 clades(S, G, V) of the coronavirus without using protein data?" Tags="|rna-seq|phylogenetics|phylogeny|covid-19|sars-cov-2|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12840" PostTypeId="2" ParentId="12838" CreationDate="2020-04-05T14:46:00.660" Score="8" Body="&lt;p&gt;It seems to be explained right there in the image you posted:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nsmf9.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nsmf9.png&quot; alt=&quot;detail from the OP's image&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, the three strains were classified based on three specific variants:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Strain S, variant ORF8-L84S: a variant in the gene &amp;quot;ORF8&amp;quot; which changes the leucine (L) residue at position 84 of the gene's protein product to a serine (S).&lt;/li&gt;&#xA;&lt;li&gt;Strain G, variant S-D614G: a variant in the gene &amp;quot;S&amp;quot; which changes the aspartic acid (D) residue at position 614 of the gene's protein product to a glycine (G).&lt;/li&gt;&#xA;&lt;li&gt;Strain V, variant NS3-G251V: a variant in the gene &amp;quot;NS3&amp;quot; which changes the glycine (G) residue at position 251 of the  gene's protein product   to a valine (V).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="298" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2020-04-05T15:05:31.120" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="12841" PostTypeId="2" ParentId="12838" CreationDate="2020-04-05T16:01:50.023" Score="2" Body="&lt;p&gt;The most important thing to say is we are grateful this information has been released. It is important to note these are very large genomes of around 9000 amino acids in total.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The clades are assigned to single amino acid mutations. The tree is built using nucleotide data therefore synonymous (silent) mutations are resulting in phylogenetic resolution and the authors have superimposed amino acid mutations over this tree structure. They have represented the tree as a circular phylogram.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the reason why there is loads of branches (of a phylogenetic tree) of different sizes is because there are different numbers of silent (mostly 3rd codon wobble) mutations between the isolates. RNA viruses mutate rapidly. There are only 4+ amino acid changes across the entire epidemic according to this analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Nomenclature&lt;/strong&gt;&#xA;The clades are named after the amino acid mutation that arises within that group. Its a bit weird, but its okay. Hence S clade means a serine mutation, G clade a glycine mutation etc.... The thing is that S also equals Spike, which is a very important protein (below), so it is confusing because the S clade refers to ORF8. Which gene the amino acid mutation is occurring in is more important than the specific mutation. Secondly the &quot;other&quot; clade is dying out - which is not that informative, but others are holding their course.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Mutation scheme and protein&lt;/strong&gt;&#xA;So @terdon is right &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;L-&gt;S at position 84 of ORF8 - Hmmmm I am not hugely excited by that because it is a very weird gene and is is not a radical mutation (the amino acids are related) which is absent in some closely related coronaviruses altogether. In other words some closesly related viruses can remove this gene altogether and remain funcational. &lt;/li&gt;&#xA;&lt;li&gt;D-&gt;G (aspartic acid is D) in the Spike protein at position 614. That is an interesting mutation because S protein contains the cellular receptor and that is an unusual mutation. S protein is critical for vaccination.&lt;/li&gt;&#xA;&lt;li&gt;G-&gt;V in the NS3 ... I dunno because glycine to valine means it will be a bit more hydrophobic. NS3 is in the structural protein region of the genome (including spike, capid, membrane and another structural protein) but I'm not sure precisely what is does&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Epidemiological interpretation&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Epidemiological interpretation is difficult because it doesn't really explain the possible differences in mortality rates between different countries, for example USA appears to have a low mortality rate in comparison to Italy but there is lots of mixing. It is also not easy to read the tree because it is a circular phylogram and I am struggling to read each &quot;clade&quot; (which has a precise phylogenetic diffinition) I have to take their word for it. Secondly is the colour scheme the 'red' swamps every other colour and gives a disproportionate red colour. This is an old epidemiologists trick in heat maps, there precise colour scheme albeit that is relative, appears to convey a &quot;danger&quot; message to the reader, even though mathematically its simply a consequence of the colour scheme.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fundamentally it is not really clear whether the mutations confer phenotypic changes and these are huge genomes for an RNA virus. However, the G clade needs further investigation.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&#xA;The battle continues as to whether data collection skews, social skews, age-prevelence skews, climatic skews result in variation of mortality rates between countries? Versus is their a genetic difference in mortality. On this data I'd go for the non-genetic factors.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The answer to your second question is simple, you just translate the triplet codon into amino acids and identify which amino acid is present at the above 3 positions. You then simply denote the clade within the taxon ID of your nucleotide data. MEGA X has an alignment editor that will allow this translation.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-04-06T09:53:58.693" LastActivityDate="2020-04-06T09:53:58.693" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12842" PostTypeId="1" CreationDate="2020-04-05T16:25:04.363" Score="0" ViewCount="60" Body="&lt;p&gt;If continuous-time Markov models contain a parameter q that denotes the instantaneous rate of change and a global/strict clock model is the overall rate of evolution, how are these two parameters related? That is, for a given substitution model (e.g., JC69, F81, GTR), is there a deterministic relationship between the rate parameter(s) of these substitution models and the rate of the clock model? For instance, should the individual instantaneous rates of change (e.g., A -&gt; C, T -&gt; C, etc.) sum up to the global clock rate? If this is not the case, how should one understand the relationship between the rate of a continuous-time Markov model and the overall evolutionary rate?&lt;/p&gt;&#xA;" OwnerUserId="2961" LastEditorUserId="3848" LastEditDate="2020-04-25T10:23:59.703" LastActivityDate="2020-09-22T11:07:25.803" Title="Relationship between the instantaneous rate of change of a continuous time Markov model and the rate of a global clock model?" Tags="|fasta|phylogenetics|phylogeny|molecular-genetics|molecular-clock|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12843" PostTypeId="1" CreationDate="2020-04-05T16:53:33.930" Score="2" ViewCount="336" Body="&lt;p&gt;I am trying to subset SNPs from 32 cultivars. I have the following exclusion criteria:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&gt;20% missing data&lt;/li&gt;&#xA;&lt;li&gt;SNPs present in less than 50% of genotypes&lt;/li&gt;&#xA;&lt;li&gt;Each genotype must have at least 50% of included SNPs&lt;/li&gt;&#xA;&lt;li&gt;&gt;=60% heterozygous calls&lt;/li&gt;&#xA;&lt;li&gt;Bi-allelic SNPs only&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;No very rare variants (i.e. MAF&amp;lt;0.05)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view --output-type u --min-alleles 2 --max-alleles 2 \ &#xA;  --types snps --exclude MAF[0]&amp;lt;0.05 --exclude ** \&#xA;    --known variants_raw_sorted.bcf &amp;gt; SNP_filtered_sorted_kkf.bcf&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Where the &lt;code&gt;**&lt;/code&gt; is, I am trying to implement the remainder my exclusion criteria, (criteria 1-5), however I am having a great deal of difficulty understanding how to use the expression in bcftools. I am looking to understand how to use the information in the info fields (AN,AC, AF/MAF, NS) to implement my exclusion criteria. Any help is very much appreciated.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Further information : This is the upstream code that led to the bcf file.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fastp -i ACDC_R1.fastq.gz -I ACDC_R2.fastq.gz \ &#xA;  -o fastp_ACDC_R1P.fastq.gz fastp_ACDC_R2P.fastq.gz \ &#xA;  -R &quot;ACDC_fastp_QC&quot; -h &quot;Fastp_ACDC&quot; -j &quot;ACDC_fastp&quot; \ &#xA;  -e 20 -c -a &quot;auto&quot; -L -r \ &#xA;  -m --merged_out &quot;ACDC_fastp_Merged&quot; --out1 &quot;ACDC_R1_fastp_Unmerged&quot; \&#xA;  --out2 &quot;ACDC_R2_fastp_Unmerged&quot; \ &#xA;  --unpaired1 &quot;ACDC_R1Pass_R2Fail&quot; --unpaired2 &quot;ACDC_R2Pass_R1Fail&quot;&#xA;&#xA;bowtie2 -p 20 -x cs10 -U ACDC_fastp_Merged.fastq.gz -S ACDC_aligned.sam&#xA;&#xA;samtools view -bS ACDC.sam &amp;gt; ACDC.bam&#xA;&#xA;bcftools mpileup --output-type u -f cs10.fasta -b bam_list.txt | &#xA;    bcftools call -vmO z -o all_raw_condition_1.vcf.gz --threads=20&#xA;&#xA;bcftools view --output-type u --output-file variants_raw.bcf variants_raw.vcf.gz&#xA;&#xA;bcftools sort --max-mem 14000 --output-type u --output-file variants_raw_sorted.bcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7267" LastEditorUserId="7267" LastEditDate="2020-04-06T18:33:25.783" LastActivityDate="2020-04-06T18:33:25.783" Title="Subsetting SNPs with specific exclusion criteria using bcftools" Tags="|snp|bcftools|subset|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12844" PostTypeId="1" AcceptedAnswerId="12850" CreationDate="2020-04-05T19:13:42.507" Score="1" ViewCount="123" Body="&lt;p&gt;I have a bunch of .hic files of different species. But, I don't know which reference genome assembly (e.g. GRCh37 or GRCh38 in the case of human genome) the genomic co-ordinates of the .hic file correspond to. So I wonder if there is any way to figure out the version of the reference genome assembly from a .hic file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Therefore, my question is&lt;/strong&gt;: Does .hic file contain metadata such as the version of reference genome assembly? If yes, then how could I access it? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(I could obtain the information I need by digging into the methods sections of each paper corresponding to each .hic file. But it would be much better if I could get that right-away from the .hic file.).&lt;/p&gt;&#xA;" OwnerUserId="274" LastEditorUserId="274" LastEditDate="2020-04-06T00:31:44.123" LastActivityDate="2020-05-06T15:01:05.077" Title="How to identify the reference genome assembly used to create a .hic file?" Tags="|reference-genome|chromosome-capture|hi-c|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12847" PostTypeId="1" CreationDate="2020-04-06T06:15:05.290" Score="1" ViewCount="146" Body="&lt;p&gt;I have genes differentially expressed between two groups (case and control). I would like to annotate them by classifying them according to their biological functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I work on parasites of Plasmodium Falciparum type Dd2 which are divided into two groups (case and control). I carried out a Differential expression analysis with cuffdiff and CummeRbund then I annotated my genes on plasmodb.org.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Example of id genes: pfDd2_14005040&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I would like to do a pathway analysis to see which biological pathways these genes are involved in.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;StupidWolf,&#xA;I resumed my annotation with the 3D7 strain of plasmoduim Falciparum then I created my geneList as you indicated but when I make the dotplot nothing is displayed&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;setwd(&quot;C:/cygwin64/home/DIANGO/EXCELL&quot;)&#xA;library(org.Pf.plasmo.db)&#xA;library(clusterProfiler)&#xA;&#xA;gene &amp;lt;- c (&quot;PF3D7_0936800&quot;,&quot;PF3D7_1478900&quot;,&quot;PF3D7_1009700&quot;,&quot;PF3D7_0508500&quot;)&#xA;geneList = keys(org.Pf.plasmo.db)&#xA;enr &amp;lt;- enrichGO(gene,universe=geneList,OrgDb='org.Pf.plasmo.db',keyType =&quot;SYMBOL&quot;,ont=&quot;BP&quot;)&#xA;head(enr@result,readtable=TRUE)&#xA;dotplot(enr)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output gives a white rectangle&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                   ID                           Description GeneRatio  BgRatio     pvalue&#xA;GO:0006323 GO:0006323                         DNA packaging       1/2  21/2615 0.01599974&#xA;GO:0071103 GO:0071103               DNA conformation change       1/2  30/2615 0.02281728&#xA;GO:0051276 GO:0051276               chromosome organization       1/2  58/2615 0.04387582&#xA;GO:0006996 GO:0006996                organelle organization       1/2 144/2615 0.10712138&#xA;GO:0016043 GO:0016043       cellular component organization       1/2 203/2615 0.14925925&#xA;GO:0006464 GO:0006464 cellular protein modification process       1/2 265/2615 0.19244223&#xA;            p.adjust     qvalue        geneID Count&#xA;GO:0006323 0.1026777 0.07205456 PF3D7_0508500     1&#xA;GO:0071103 0.1026777 0.07205456 PF3D7_0508500     1&#xA;GO:0051276 0.1316275 0.09237015 PF3D7_0508500     1&#xA;GO:0006996 0.2203602 0.15463872 PF3D7_0508500     1&#xA;GO:0016043 0.2203602 0.15463872 PF3D7_0508500     1&#xA;GO:0006464 0.2203602 0.15463872 PF3D7_1009700     1&#xA;&amp;gt; dotplot(enr)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3PMyb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3PMyb.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once again I need your help. &lt;/p&gt;&#xA;" OwnerUserId="6694" LastEditorUserId="77" LastEditDate="2020-04-30T23:22:36.273" LastActivityDate="2020-04-30T23:22:36.273" Title="Differential Expression" Tags="|rna-seq|differential-expression|functional-annotation|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12848" PostTypeId="1" CreationDate="2020-04-06T06:41:01.237" Score="1" ViewCount="104" Body="&lt;p&gt;The matrix of transition probabilities for a substitution model Q over time t is found as follows: &lt;span class=&quot;math-container&quot;&gt;$e^{Qt}$&lt;/span&gt;. Since Q and t only show up as a product, we cannot distinguish time and rate of change. So we multiple Q by a constant so that the average rate of substitution is 1. Time t is then measured by the expected number of substitutions per site.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But why does forcing the average rate of substitution to equal 1 have this effect? &lt;/p&gt;&#xA;" OwnerUserId="2961" LastEditorUserId="3848" LastEditDate="2020-04-28T15:36:06.757" LastActivityDate="2020-04-28T15:36:06.757" Title="Why do we scale the rate matrix of a substitution model to make the average rate of substitution equal 1?" Tags="|phylogeny|phylogenetics|molecular-genetics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12850" PostTypeId="2" ParentId="12844" CreationDate="2020-04-06T14:54:06.203" Score="1" Body="&lt;p&gt;Turns out HiC file does store such metadata.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;straw&lt;/code&gt; tool from Aiden Lab contains a script (&lt;a href=&quot;https://github.com/aidenlab/straw/blob/master/python/read_hic_header.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;read_hic_header.py&lt;/a&gt;) that prints out the metadata from a given HiC file. Usage: &lt;code&gt;read_hic_header.py &amp;lt;hic file or URL&amp;gt; [verbose]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In order to obtain the metadata in the form of a dictionary, one may consider using a function of the same tool that is currently available in my &lt;a href=&quot;https://github.com/rraadd88/straw.git&quot; rel=&quot;nofollow noreferrer&quot;&gt;fork&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from straw import straw_module&#xA;straw_module.read_metadata(infile,verbose=False)&#xA;&#xA;Args&#xA;    infile: str, path to the HiC file &#xA;    verbose: bool, print the metadata if True else return the metadata&#xA;Returns&#xA;    metadata: dict, the metadata is returned if verbose is False. &#xA;                Keys of the metadata: &#xA;                HiC version, &#xA;                Master index, &#xA;                Genome ID (str), &#xA;                Attribute dictionary (dict), &#xA;                Chromosomes (dict), &#xA;                chromosomes2length (dict), &#xA;                Base pair-delimited resolutions (list), &#xA;                Fragment-delimited resolutions (list). &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="274" LastActivityDate="2020-04-06T14:54:06.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12851" PostTypeId="1" AcceptedAnswerId="12863" CreationDate="2020-04-06T18:19:06.663" Score="3" ViewCount="200" Body="&lt;p&gt;I am trying to reproduce a machine learning model that has been developed &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/842336v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. As one of the datasets, they use single-domain proteins with full length 3D-structures solved. Since I don't have a background in biology, I am not sure how to filter the &lt;a href=&quot;https://www.rcsb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB&lt;/a&gt; for these kind of proteins. Any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="8561" LastEditorUserId="4622" LastEditDate="2020-04-07T19:11:01.640" LastActivityDate="2020-04-07T19:11:01.640" Title="How to filter the PDB for single-domain proteins with full length 3D-structures solved" Tags="|proteins|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12853" PostTypeId="2" ParentId="12847" CreationDate="2020-04-06T23:37:04.703" Score="1" Body="&lt;p&gt;I guess plasmodium doesn't have that many online tools dedicated to it so you can use the annotation packages from bioconductor and try it. I used clusterProfiler for analysis and org.Pf.plasmo.db for annotation below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From &lt;a href=&quot;https://bmcgenomics.biomedcentral.com/articles/10.1186/s12864-018-5257-x#MOESM4&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, grabbed table 1, a table of supposed genes involved in something: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;genes = c(&quot;PF3D7_1302100&quot;, &quot;PF3D7_0501300&quot;, &quot;PF3D7_0302500*&quot;, &quot;PF3D7_1030200&quot;, &#xA;&quot;PF3D7_0302200*&quot;, &quot;PF3D7_1327300&quot;, &quot;PF3D7_0410000&quot;, &quot;PF3D7_0104300&quot;, &#xA;&quot;PF3D7_0420300&quot;, &quot;PF3D7_0810000*&quot;, &quot;PF3D7_0522300&quot;, &quot;PF3D7_0935700*&quot;, &#xA;&quot;PF3D7_0424700*&quot;, &quot;PF3D7_1477700*&quot;, &quot;PF3D7_0202000*&quot;, &quot;PF3D7_1371600&quot;, &#xA;&quot;PF3D7_1036300&quot;, &quot;PF3D7_0422900&quot;, &quot;PF3D7_0501400&quot;, &quot;PF3D7_0221700&quot;, &#xA;&quot;PF3D7_0301000&quot;, &quot;PF3D7_1454100&quot;, &quot;PF3D7_0831800&quot;, &quot;PF3D7_0424900&quot;, &#xA;&quot;PF3D7_1447500&quot;, &quot;PF3D7_1477400&quot;, &quot;PF3D7_0424800&quot;, &quot;PF3D7_0823700&quot;, &#xA;&quot;PF3D7_1477300&quot;, &quot;PF3D7_0814100&quot;, &quot;PF3D7_1372100&quot;, &quot;PF3D7_0201500&quot;, &#xA;&quot;PF3D7_0825300&quot;, &quot;PF3D7_0201700&quot;, &quot;PF3D7_1470000&quot;, &quot;PF3D7_1401000&quot;, &#xA;&quot;PF3D7_0724900&quot;, &quot;PF3D7_0601700&quot;, &quot;PF3D7_1410000&quot;, &quot;PF3D7_0109300&quot;, &#xA;&quot;PF3D7_1347300&quot;, &quot;PF3D7_0201800&quot;, &quot;PF3D7_1242100&quot;, &quot;PF3D7_0403300&quot;, &#xA;&quot;PF3D7_0114500&quot;, &quot;PF3D7_1301200&quot;, &quot;PF3D7_0931200&quot;, &quot;PF3D7_0832200.1&quot;, &#xA;&quot;PF3D7_0305900&quot;, &quot;PF3D7_0608310&quot;, &quot;PF3D7_0221800&quot;, &quot;PF3D7_0511100&quot;, &#xA;&quot;PF3D7_1001300&quot;, &quot;PF3D7_1225900&quot;, &quot;PF3D7_1444600&quot;, &quot;PF3D7_1349900&quot;, &#xA;&quot;PF3D7_1008300&quot;, &quot;PF3D7_0919600&quot;, &quot;PF3D7_1210300&quot;, &quot;PF3D7_0929700&quot;, &#xA;&quot;PF3D7_0701600&quot;, &quot;PF3D7_0201600&quot;, &quot;PF3D7_0801600&quot;, &quot;PF3D7_1463600&quot;, &#xA;&quot;PF3D7_0201900&quot;, &quot;PF3D7_0905000&quot;, &quot;PF3D7_1310000&quot;, &quot;PF3D7_0315200&quot;, &#xA;&quot;PF3D7_1139200&quot;, &quot;PF3D7_1023500&quot;, &quot;PF3D7_1111300&quot;, &quot;PF3D7_0930700&quot;, &#xA;&quot;PF3D7_1350800&quot;, &quot;PF3D7_0412300&quot;, &quot;PF3D7_0831900&quot;, &quot;PF3D7_0406600&quot;, &#xA;&quot;PF3D7_1448100&quot;, &quot;PF3D7_0114300&quot;, &quot;PF3D7_0526100&quot;, &quot;PF3D7_1243200&quot;, &#xA;&quot;PF3D7_0921100&quot;, &quot;PF3D7_0301900&quot;, &quot;PF3D7_0622700&quot;, &quot;PF3D7_0811800&quot;, &#xA;&quot;PF3D7_0114200&quot;, &quot;PF3D7_0221500&quot;, &quot;PF3D7_1129850&quot;, &quot;PF3D7_1142200&quot;, &#xA;&quot;PF3D7_0114100&quot;, &quot;PF3D7_0512900&quot;, &quot;PF3D7_1346200&quot;, &quot;PF3D7_1369400&quot;, &#xA;&quot;PF3D7_0314300&quot;, &quot;PF3D7_0324100&quot;, &quot;PF3D7_0107700&quot;, &quot;PF3D7_1122000&quot;, &#xA;&quot;PF3D7_0618800&quot;, &quot;PF3D7_0601200&quot;, &quot;PF3D7_1360300&quot;, &quot;PF3D7_1318500&quot;, &#xA;&quot;PF3D7_1111700&quot;, &quot;PF3D7_0901900&quot;, &quot;PF3D7_1313900&quot;, &quot;PF3D7_1101700&quot;, &#xA;&quot;PF3D7_1313200&quot;, &quot;PF3D7_1039700&quot;, &quot;PF3D7_0222100&quot;, &quot;PF3D7_1371500&quot;, &#xA;&quot;PF3D7_1344700&quot;, &quot;PF3D7_1408300&quot;, &quot;PF3D7_0929800&quot;, &quot;PF3D7_0115000&quot;, &#xA;&quot;PF3D7_1462900&quot;, &quot;PF3D7_1352600&quot;, &quot;PF3D7_0626200&quot;, &quot;PF3D7_0831750&quot;, &#xA;&quot;PF3D7_0602300&quot;, &quot;PF3D7_0521100&quot;, &quot;PF3D7_1452100&quot;, &quot;PF3D7_0718900&quot;, &#xA;&quot;PF3D7_0101300&quot;, &quot;PF3D7_0815700&quot;, &quot;PF3D7_0113600&quot;, &quot;PF3D7_0937200&quot;, &#xA;&quot;PF3D7_1108800&quot;, &quot;PF3D7_0221100&quot;, &quot;PF3D7_0921500&quot;, &quot;PF3D7_1038300&quot;, &#xA;&quot;PF3D7_0201400&quot;, &quot;PF3D7_0822200&quot;, &quot;PF3D7_0937000&quot;, &quot;PF3D7_0301100&quot;&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We can for example using clusterProfiler:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(org.Pf.plasmo.db)&#xA;library(clusterProfiler)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Besides the list of interest, you need to get a &quot;universe&quot; of all genes tested, in your situation it will be genes that you tested for differential gene expression, below I used all the genes annotated:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;geneList = keys(org.Pf.plasmo.db)&#xA;&#xA;enr &amp;lt;- enrichGO(genes,universe=geneList, OrgDb='org.Pf.plasmo.db', &#xA;keyType =&quot;SYMBOL&quot;,ont=&quot;BP&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Above is a GO term enrichment, biological processes, you can look at the result like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;head(enr@result,readtable=TRUE)&#xA;                       ID&#xA;    GO:0042000 GO:0042000&#xA;    GO:0044417 GO:0044417&#xA;    GO:0044766 GO:0044766&#xA;    GO:0051808 GO:0051808&#xA;    GO:0051836 GO:0051836&#xA;    GO:1902579 GO:1902579&#xA;                                                                                               Description&#xA;    GO:0042000                                             translocation of peptides or proteins into host&#xA;    GO:0044417                                                        translocation of molecules into host&#xA;    GO:0044766                                                                    multi-organism transport&#xA;    GO:0051808 translocation of peptides or proteins into other organism involved in symbiotic interaction&#xA;    GO:0051836            translocation of molecules into other organism involved in symbiotic interaction&#xA;    GO:1902579                                                                 multi-organism localization&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And you can plot it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dotplot(enr)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jSfQr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jSfQr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can read more about the analysis you can do at the package &lt;a href=&quot;https://yulab-smu.github.io/clusterProfiler-book/&quot; rel=&quot;nofollow noreferrer&quot;&gt;website&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-04-06T23:37:04.703" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12854" PostTypeId="2" ParentId="11745" CreationDate="2020-04-07T02:26:59.310" Score="2" Body="&lt;p&gt;@StupidWolf's answer is correct -- that first number in the flagstat output is what you want to look at to see the number of reads marked as duplicates. I wanted to add that the number given in the Picard metrics file is in fact 6%, not 0.06%. This is due to an ambiguity that is widespread in Picard metrics files; in many places the program emits a fraction even when the column name is &quot;percent of something&quot;. Soo Hee Lee wrote a very short GATK blog post about this way back in the day, see the archived version &lt;a href=&quot;https://github.com/broadinstitute/gatk-docs/blob/master/blog-2012-to-2019/2016-03-18-Metrics_say_percent_%E2%80%93_Double-check_those_decimals_%E2%80%93_Fractions_everywhere.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2776" LastActivityDate="2020-04-07T02:26:59.310" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12855" PostTypeId="2" ParentId="11543" CreationDate="2020-04-07T02:39:54.530" Score="2" Body="&lt;p&gt;As suggested by @burger in their comment, you can indeed use solid tissue samples as normals for your PON. You are clearly aware, but I want to spell this out for others, that there are a few caveats that go with the practice of using solid tissue normals rather than blood normals. Mainly, if your normals are &quot;tissue-adjacent&quot;, ie sampled from an area of tissue adjacent to the tumor, there is a higher risk of including contaminating tumor cells compared to blood normals. As a result you might have real somatic variants that end up filtered because they are seen in the normal. Though that typically acts more at the level of the direct Tumor-Normal comparison than at the level of the PON, since the PON draws from many normal samples and the chance of seeing the same contaminating somatic mutation in many such samples should be quite low. So ultimately that should be fine. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It's a very common practice to use whatever matched normals you have available to build your PON, and there doesn't seem to be any drawback to doing that as long as you have enough samples (at least a few dozen, ideally a few hundred or more). If you don't have enough samples in your own study, then you should seek to use other samples from young and healthy people that have been sequenced using the same technology (same library prep kits/protocols, same chemistry version etc) since the goal of the PON is to weed out recurrent technical artifacts.&lt;/p&gt;&#xA;" OwnerUserId="2776" LastActivityDate="2020-04-07T02:39:54.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12856" PostTypeId="2" ParentId="11435" CreationDate="2020-04-07T02:47:47.947" Score="0" Body="&lt;p&gt;This probably comes too late, but for the record GATK supports multiple kinds of interval file format. BED is the most common, and there's the Picard format (.interval_list) which is BED-like with a few tweaks. Both require coordinates. &lt;strong&gt;However, GATK also supports a simpler format (.list or .intervals) that allows you to just give the contig names (eg chr1) without any coordinates. Sounds like that would satisfy your need.&lt;/strong&gt; Incidentally, GATK also supports using VCF files as interval file if you want to re-genotype specific regions based on variants located there (ideally paired with the interval padding arguments).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;See the GATK doc article for more details:&#xA;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035531852-Intervals-and-interval-lists&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360035531852-Intervals-and-interval-lists&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(The doc has a few formatting issues but I just notified the team so they'll fix it)&lt;/p&gt;&#xA;" OwnerUserId="2776" LastActivityDate="2020-04-07T02:47:47.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12858" PostTypeId="2" ParentId="10898" CreationDate="2020-04-07T03:12:11.520" Score="1" Body="&lt;p&gt;This was answered by the OP -- it was a bug that has been fixed in a subsequent release, 4.1.5.0. See the release notes here: &lt;a href=&quot;https://github.com/broadinstitute/gatk/releases/tag/4.1.5.0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/broadinstitute/gatk/releases/tag/4.1.5.0&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Made callable sites a Long to avoid integer overflow (#6303)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Issue ticket: &lt;a href=&quot;https://github.com/broadinstitute/gatk/issues/6302&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/broadinstitute/gatk/issues/6302&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2776" LastEditorUserId="2776" LastEditDate="2020-04-08T14:42:48.793" LastActivityDate="2020-04-08T14:42:48.793" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12859" PostTypeId="1" AcceptedAnswerId="12860" CreationDate="2020-04-07T05:05:08.957" Score="1" ViewCount="1548" Body="&lt;p&gt;I am relatively new to Bioinformatics and scRNA-seq data analysis. I am using Seurat V3 to analyze a scRNA-seq dataset in R. Currently, I have merged three scRNA-seq samples from the same donor into one Seurat object, &lt;em&gt;All_Samples&lt;/em&gt;. We'll call them:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Uninfected&lt;/li&gt;&#xA;&lt;li&gt;Virus1&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Virus2&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;All_Samples &amp;lt;- merge(x = Uninfected, y = c(Virus1, Virus2))&#xA;&lt;/code&gt;&lt;/pre&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Uninfected cells did not receive virus (i.e. express no viral genes), the other two samples were infected with a virus introducing new genes into the cells. I would like to accomplish two things. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;First&lt;/strong&gt;: I want to create a new Ident for Virus1 and Virus2 samples based on expression of a viral gene  of interest: &quot;GeneA&quot;. All cells from Virus1 and Virus2 samples that have &quot;GeneA&quot; expression &gt; 0.5 would be labeled &quot;Pos&quot;, those with &quot;GeneA&quot; &amp;lt; 0.5 would be labeled &quot;Neg&quot; in a  the new Column called GeneA. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am able to subset the objects based on GeneA expression that applies to all samples in the object. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;All_Samples_GeneA_Pos &amp;lt;- subset(All_Samples, subset = GeneA &amp;gt; 0.5)&#xA;All_Samples_GeneA_Neg &amp;lt;- subset(All_Samples, subset = GeneA &amp;lt; 0.5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I assume I would have to modify All_Samples@meta.data based on &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/4249/manually-define-clusters-in-seurat-and-determine-marker-genes&quot;&gt;this&lt;/a&gt; post and &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/8658/how-to-set-new-identities-for-new-clusters-based-on-a-gene-expression-in-seurat&quot;&gt;this&lt;/a&gt; post, but I admit I'm not entirely sure how to implement the suggested answers into my data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Second&lt;/strong&gt;: Once I have separated the data from Virus1 and Virus2 into &quot;Pos&quot; and &quot;Neg&quot; cells for &quot;GeneA&quot;, I want to look for differentially expressed genes between all Uninfected cells and &quot;Pos&quot; cells from Virus1 or Virus2. I have been using something like this to compare samples:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; FindMarkers(object = All_Samples, group.by = 'virus', ident.1 = &quot;1&quot;, ident.2 = &quot;2&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But how would I do something equivalent to this: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; FindMarkers(object = All_Samples, ident.1 = &quot;Uninfected&quot;, ident.2 = &quot;Virus1_GeneA_Pos&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or maybe&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; FindMarkers(object = All_Samples, ident.1 = &quot;Uninfected_GeneA_Neg&quot;, ident.2 = &quot;Virus1_GeneA_Pos&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Let me know if I can clarify any points. I appreciate any help!&lt;/p&gt;&#xA;" OwnerUserId="8563" LastEditorUserId="3848" LastEditDate="2020-11-28T14:22:43.350" LastActivityDate="2020-11-28T14:22:43.350" Title="Set new Idents based on gene expression in Seurat and mix n match identities to compare using FindAllMarkers" Tags="|r|scrnaseq|seurat|covid-19|sars-cov-2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12860" PostTypeId="2" ParentId="12859" CreationDate="2020-04-07T07:57:51.713" Score="1" Body="&lt;p&gt;For the first question, you can use &lt;code&gt;ifelse()&lt;/code&gt; to create a new column in the &lt;code&gt;meta.data&lt;/code&gt; slot:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;All_Samples@meta.data$new_column &amp;lt;-&#xA;  ifelse(&#xA;    rownames(All_Samples@meta.data) %in% colnames(All_Samples_GeneA_Pos),&#xA;    &quot;GeneA_Pos&quot;,&#xA;    &quot;GeneB_Pos&quot;&#xA;  )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;colnames(seurat_object)&lt;/code&gt; provides a vector of cell names in a given &lt;code&gt;Seurat&lt;/code&gt; object. Here whatever cell that is in the &lt;code&gt;All_Samples_GeneA_Pos&lt;/code&gt; object would be &lt;code&gt;GeneA_Pos&lt;/code&gt; and whatever is not &lt;code&gt;GeneB_Pos&lt;/code&gt;. To better control the behavior, you can use a &quot;nested&quot; &lt;code&gt;ifelse()&lt;/code&gt;; you can have another &lt;code&gt;ifelse()&lt;/code&gt; instead of the &lt;code&gt;&quot;GeneB_Pos&quot;&lt;/code&gt; bit above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the second question, the rationale is the same as above, you can define an extra column with the &quot;identities of interest&quot;, then use these identities with &lt;code&gt;SetIdent()&lt;/code&gt; and then in turn using two identities of interest in the differential expression call.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-04-07T07:57:51.713" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12861" PostTypeId="2" ParentId="12832" CreationDate="2020-04-07T08:47:05.910" Score="1" Body="&lt;p&gt;Given the lack of effort from the OP, probably I shouldn't have wasted my time on this question with a lot of problems:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;i) The question is not reproducible, the OP should have supplied their data with &lt;code&gt;dput()&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ii) The data as pasted above does not quite make sense to me, why would you have three readings from the same gene and even in such a case how should these replicates be represented in a heatmap?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;iii) The question is not specific enough and should not go like &quot;how can I generate a heatmap&quot; and should be more like &quot;in an attempt to generate a heatmap like in this figure I have tried this piece of code but ...&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Having said these, and after downvoting the question, here is a solution leveraging the &lt;code&gt;ComplexHeatmap&lt;/code&gt; package:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(data.table)&#xA;library(ComplexHeatmap)&#xA;&#xA;my_data &amp;lt;-&#xA;&quot;&#xA;Sample  Gene    CCF&#xA;Sample1 POLQ    1&#xA;Sample1 CDKN2A  1&#xA;Sample1 TP53    1&#xA;Sample1 CCDC102B    0.906729075&#xA;Sample1 CCDC102B    1&#xA;Sample1 CCDC102B    1&#xA;Sample1 CCDC102B    1&#xA;Sample2 TSHZ3   1&#xA;Sample2 PIK3CA  1&#xA;Sample2 SLIT2   1&#xA;Sample2 SLIT2   1&#xA;&quot;&#xA;&#xA;my_data &amp;lt;- fread(my_data)&#xA;&#xA;my_data &amp;lt;- dcast.data.table(my_data, &#xA;                            formula = Gene ~ Sample,&#xA;                            value.var = &quot;CCF&quot;,&#xA;                            fun=mean)&#xA;&#xA;# dcast generates NA values as the provided data does not have values for each gene&#xA;my_data[is.na(my_data)] &amp;lt;- 0&#xA;&#xA;# Heatmap() expects a matrix input&#xA;my_matrix &amp;lt;- as.matrix(my_data[,c(&quot;Sample1&quot;, &quot;Sample2&quot;)])&#xA;rownames(my_matrix) &amp;lt;- my_data$Gene&#xA;&#xA;Heatmap(my_matrix,&#xA;        cell_fun = function(j, i, x, y, width, height, fill) {&#xA;          grid.text(my_matrix[i, j], x, y, gp = gpar(fontsize = 10))})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is how the data look after &lt;code&gt;dcast&lt;/code&gt;, in which multiple entries per gene (my point ii above) is summarized with &lt;code&gt;mean()&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(my_matrix)&#xA;           Sample1 Sample2&#xA;CCDC102B 0.9766823       0&#xA;CDKN2A   1.0000000       0&#xA;PIK3CA   0.0000000       1&#xA;POLQ     1.0000000       0&#xA;SLIT2    0.0000000       1&#xA;TP53     1.0000000       0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is the resulting heatmap:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sMGAM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sMGAM.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-04-07T08:47:05.910" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12862" PostTypeId="1" AcceptedAnswerId="12888" CreationDate="2020-04-07T10:38:03.820" Score="1" ViewCount="265" Body="&lt;p&gt;I have 2 crystal structures, one for my protein and one for my ligand (I have several protein-ligand pairs). I am using AutoDock Vina to simulate docking, which returns another file of the ligand with updated 3D coordinates. Everything is in pdbqt format, but I can use OpenBabel to convert them if needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a programmatic way (mainly python, but other languages work as well) to load both the protein and the docked ligand, then extract only the acting atoms which connect the 2 structures together (and the bonds between them).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If a direct answer is too complex, I would be happy if someone could suggest a tool/framework I could use to manipulate this sort of data easily so I can implement a solution on my own.&lt;/p&gt;&#xA;" OwnerUserId="5118" LastActivityDate="2020-04-11T16:27:21.323" Title="Extract autodocked protein-ligand connections programatically" Tags="|python|protein-structure|docking|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12863" PostTypeId="2" ParentId="12851" CreationDate="2020-04-07T11:12:40.630" Score="3" Body="&lt;p&gt;If you want to reproduce the model in that paper, you should email the corresponding author asking for the exact set of PDB IDs/chain IDs/residue ranges they used. There is too much flexibility in the criteria as given to get the exact set any other way.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to assemble a similar set for your own work, the RCSB PDB advanced search is very powerful and would let you filter e.g. by max sequence length or for only monomers.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However the PDB doesn't deal with domains (regions that evolve and fold independently) directly. A PDB chain may contain part of a domain, one domain, or multiple domains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want a dataset of protein domains you could use &lt;a href=&quot;http://www.cathdb.info&quot; rel=&quot;nofollow noreferrer&quot;&gt;CATH&lt;/a&gt;, &lt;a href=&quot;http://scop.berkeley.edu&quot; rel=&quot;nofollow noreferrer&quot;&gt;SCOP&lt;/a&gt; or &lt;a href=&quot;http://prodata.swmed.edu/ecod&quot; rel=&quot;nofollow noreferrer&quot;&gt;ECOD&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As a side issue, please remember to remove overlap of homologous proteins between the training and test sets. People entering the field often neglect to do that. See for example &lt;a href=&quot;https://www.nature.com/articles/s41580-019-0176-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;this comment piece&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2020-04-07T11:12:40.630" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12864" PostTypeId="2" ParentId="12842" CreationDate="2020-04-07T11:23:58.410" Score="1" Body="&lt;p&gt;Essentially you are comparing a &quot;local clock&quot; with a global clock and whether the instance rate relates to the global it depends on the model and in MCMC clocks in most cases the answer is no. A global clock is a very specific and very limiting model (below). Everything below is a comparison between taxa not between alignment positions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&quot;Local clock&quot; in the context of MCMC clocks is in inverted commas because they are using population models. I will use the nomenclature here for convenience to describe rate heterogeneity between branches, not within a branch (last example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A &quot;local clock&quot; has mutation rate heterogeniety across the tree whilst a global clock is completely homogenious. You can't say how is a &quot;local clock&quot; related to global clock because there are numerous different models. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Global clocks never hold for biological data because there are always some taxa evolving at a faster rate than others. This is particularly true in infectious disease.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One historic model used in traditional local clocks (no inverted commas) was autocorrelation, where a local clock would relate to a global clock on a &quot;clade by clade&quot; basis. That is clades are correlated by their mutation rate, so some clades evolve faster and other clades slower. Statastically an autocorrelated clock always outperforms a global clock under a likelihood ratio test.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I remember autocorrelation clearly because I implemented a variant of this approach, so I needed to understand how they did it, rather than click a few buttons on a GUI, which is what happens in Beast. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beast has at least 3 population models and off-hand I cannot remember the population scenarios. I do remember the population scenarios made sense and secondly they can give 10-fold differences in rate in the data set I looked at, You need to read on the background theory for each model and assess what is more biologically appropriate for a given context. One model (from dim memories) reminded me of a classic population growth equation, i.e. an exponential. Population growth and collapse in phylodynamics would be informative of the rate, but again you need to go to the underlying theory. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The molecular clock theory that the rate of evolution towards the tips of trees is much faster than at the deeper branches is the sort of model that could be implemented in a MCMC clock. In the latter case the relationship between a global clock and the rate heterogeneity clock (which is a better description in that model than &quot;local clock&quot;) is quite different from global vs. &quot;local clock&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The fundamental issue with clocks is exemplified in the 10-fold difference issue. The lesson that is important to me is very easy to get a &quot;wrong answer&quot;. The lesson important to you is if two MCMC models can't agree the same rate how can both obtain the same answer as a global clock? I am personally a fan of maximum heterogeneity, but in this day and age they will ask you to perform a Beast model to support it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-04-07T12:03:21.383" LastActivityDate="2020-04-07T12:03:21.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12866" PostTypeId="1" CreationDate="2020-04-07T13:28:34.183" Score="1" ViewCount="3343" Body="&lt;p&gt;I will have some scRNA-seq data. The goal of the experiment will be to see if there is any difference in gene expression between treatment groups using the package Seurat from R.&lt;/p&gt;&#xA;&lt;p&gt;I have read a tutorial how to do the analyze, but this tutorial does not explain how to import data.&#xA;I am a total beginner with R, I searched the internet,but the majority of the explanations related to csv/excel/txt... common files.&#xA;Data are from Cell ranger and spread in 3 files with following file extensions : .tsv and .mtx (barcodes.tsv, genes.tsv and matrix.mtx).&lt;/p&gt;&#xA;&lt;p&gt;Collaborators ran Cell Ranger and gave these cell ranger output files : &lt;code&gt;barcodes.tsv&lt;/code&gt;, &lt;code&gt;genes.tsv&lt;/code&gt; and &lt;code&gt;matrix.mtx&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Can someone give me the code to import these kind of data to R ?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;&lt;p&gt;update: By &amp;quot;directory&amp;quot; you means &amp;quot;folder&amp;quot; on my computer for example ? So I need to save the 3 files under one folder and then to put name of the folder into brackets after the Read10x function ? I am a total beginner with R (I have used to R commander for statistical tests but that's it...).&lt;/p&gt;&#xA;&lt;p&gt;update 2: I successed to import data into Seurat with the explanation of StupidWolf. I do not know C language ( I do not know almost any language ..). My system crashed a little further in the analyze after importing, not enough RAM... so now I am learning Linux commands to switch on the cluster.&lt;/p&gt;&#xA;" OwnerUserId="8566" LastEditorUserId="73" LastEditDate="2021-05-07T09:59:52.800" LastActivityDate="2022-06-01T12:07:30.863" Title="How to import data from cell ranger to R (Seurat)?" Tags="|r|scrnaseq|seurat|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12867" PostTypeId="1" CreationDate="2020-04-07T14:27:19.047" Score="1" ViewCount="435" Body="&lt;h1&gt;Problem:&lt;/h1&gt;&#xA;&lt;p&gt;When I run my Snakefile on our cluster, all the processes complete successfully. I can see from the logs that my shell scripts are being run correctly and are claiming to complete successfully. About half-way through when analysis steps begins, the workflow crashes due to empty input files (which is normal for &lt;a href=&quot;https://www.cog-genomics.org/plink/1.9/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PLINK-1.9&lt;/a&gt;, what's not normal is that the input files are empty). I went back and checked, most of the data-containing files (&lt;code&gt;*.map&lt;/code&gt; and &lt;code&gt;*.ped&lt;/code&gt;, in some cases/processes Plinks binary file set of &lt;code&gt;*.bed&lt;/code&gt;, &lt;code&gt;*.bim&lt;/code&gt; and &lt;code&gt;*.fam&lt;/code&gt;) are completely empty since the beginning of the workflow? (Logs still claim processes all completed and created output files successfully?)&lt;/p&gt;&#xA;&lt;h1&gt;Nuances:&lt;/h1&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;When I was playing around, I accidentally messed up the dynamic naming variables (escaped the variable interpret syntax, causing literal usage) and caused creation of single file with actual variable names, not dataset names. When I did this however, the file was not empty?&lt;/li&gt;&#xA;&lt;li&gt;Some 'side' files produced by the software (E.g. PLINK-1.9 produces a &lt;code&gt;*.nosex&lt;/code&gt; file indicating samples with missing sex information) are not empty but the actuall data-containing output is empty?&lt;/li&gt;&#xA;&lt;li&gt;Had a problem earlier using &lt;code&gt;cut -f 4 {params.prefix}.unlifted | sed '/^#/d' &amp;gt;&amp;gt; {params.prefix3}&lt;/code&gt; &lt;em&gt;(Intention was to collate list of entries ID's to exclude in next workflow step)&lt;/em&gt;, where the use of &amp;gt;&amp;gt; with Snakemake on cluster environment simply would not be allowed (Permission denied error despite it being created line before by the same script?). Had to fix by adding in &lt;code&gt;chmod a+rwx {params.prefix3}&lt;/code&gt; just before. Might this be a symptom of the same problem?&lt;/li&gt;&#xA;&lt;li&gt;This is being executed on a PBS/Torque cluster environment where the scripts are run as user who submitted them so permissions are not &lt;em&gt;supposed&lt;/em&gt; to be an issue. Cluster profile was built for Snakemake using &lt;a href=&quot;https://github.com/snakemake-profiles/pbs-torque&quot; rel=&quot;nofollow noreferrer&quot;&gt;this PBS/Torque Profile&lt;/a&gt;. &lt;em&gt;(I just stripped out dependency flags because they were breaking things)&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h1&gt;Code:&lt;/h1&gt;&#xA;&lt;p&gt;Here is the first three processes. So far, it crashes at the third &lt;code&gt;CLEAN&lt;/code&gt; rule as the software PLINK-1.9 does not play nice with empty inputs.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule TRIM:&#xA;    input:&#xA;        expand(&amp;quot;rawData/{{sample}}.{extension}&amp;quot;, extension=['bed', 'bim', 'fam'])&#xA;&#xA;    output:&#xA;        expand(&amp;quot;Results/{{sample}}_{{location}}_TRIMMED.{fileExtensions}&amp;quot;, fileExtensions=[&amp;quot;map&amp;quot;, &amp;quot;ped&amp;quot;])&#xA;&#xA;    params:&#xA;        fromBP = lambda wildcards: config[&amp;quot;locations&amp;quot;][wildcards.location][&amp;quot;GRCh37&amp;quot;][&amp;quot;from&amp;quot;],&#xA;        toBP = lambda wildcards: config[&amp;quot;locations&amp;quot;][wildcards.location][&amp;quot;GRCh37&amp;quot;][&amp;quot;to&amp;quot;],&#xA;        chr = lambda wildcards: config[&amp;quot;locations&amp;quot;][wildcards.location][&amp;quot;GRCh37&amp;quot;][&amp;quot;chromosome&amp;quot;]&#xA;&#xA;    shell:&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        module load plink-1.9&#xA;        plink --bfile rawData/{wildcards.sample} --chr {params.chr} --set-missing-var-ids @_# --recode --keep-allele-order --from-bp {params.fromBP} --to-bp {params.toBP} --out Results/{wildcards.sample}_{wildcards.location}_TRIMMED&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;&#xA;#/********* LiftOverPlink (LIFTOVER) *********/&#xA;&#xA;rule LIFTOVER:&#xA;    input:&#xA;        expand(&amp;quot;Results/{{sample}}_{{location}}_TRIMMED.{fileExtensions}&amp;quot;, fileExtensions=[&amp;quot;map&amp;quot;, &amp;quot;ped&amp;quot;])&#xA;&#xA;    output:&#xA;        outMap=&amp;quot;Results/{sample}_{location}_LIFTED.map&amp;quot;,&#xA;        outPed=&amp;quot;Results/{sample}_{location}_LIFTED.ped&amp;quot;,&#xA;        exclusion=&amp;quot;Results/{sample}_{location}_EXCLUDE.dat&amp;quot;&#xA;&#xA;    params:&#xA;        prefix=lambda wildcards: &amp;quot;Results/{}_{}_PRE_FILTER&amp;quot;.format(wildcards.sample, wildcards.location),&#xA;        prefix2=lambda wildcards: &amp;quot;Results/{}_{}_POST_FILTER&amp;quot;.format(wildcards.sample, wildcards.location),&#xA;        prefix3=lambda wildcards: &amp;quot;Results/{}_{}_EXCLUDE.dat&amp;quot;.format(wildcards.sample, wildcards.location),&#xA;        chainFile=&amp;quot;Binaries/hg19ToHg38.over.chain&amp;quot;,&#xA;        LiftOver=&amp;quot;Binaries/liftOverPlink.py&amp;quot;,&#xA;        rmBadLifts=&amp;quot;Binaries/rmBadLifts.py&amp;quot;,&#xA;        sexes=&amp;quot;rawData/1000g.sexes&amp;quot;&#xA;&#xA;    run:&#xA;        print(&amp;quot;Determining Liftover requirements now...&amp;quot;)&#xA;        if config['samples'][wildcards.sample]['refGenome'] != &amp;quot;GRCh38&amp;quot;:&#xA;            print(&amp;quot;Liftover required. Dataset {} is mapped to {}&amp;quot;.format(wildcards.sample, config['samples'][wildcards.sample]['refGenome'])),&#xA;            shell(&amp;quot;module load liftover&amp;quot;),&#xA;            if config['samples'][wildcards.sample]['refGenome'] == &amp;quot;GRCh37&amp;quot; or config['samples'][wildcards.sample]['refGenome'] == &amp;quot;Hg19&amp;quot;:&#xA;                print(&amp;quot;Lifting from GRCh37 to GRCh38.&amp;quot;),&#xA;                shell(&amp;quot;python {params.LiftOver} -m Results/{wildcards.sample}_{wildcards.location}_TRIMMED.map -p Results/{wildcards.sample}_{wildcards.location}_TRIMMED.ped -o {params.prefix} -c {params.chainFile}&amp;quot;),&#xA;                shell(&amp;quot;ls Results&amp;quot;),&#xA;                shell(&amp;quot;python {params.rmBadLifts} --map {params.prefix}.map --out {output.outMap} --log {params.prefix2}.log&amp;quot;),&#xA;                shell(&amp;quot;cut -f 2 {params.prefix2}.log &amp;gt; {params.prefix3}&amp;quot;),&#xA;                shell(&amp;quot;chmod a+rwx {params.prefix3}&amp;quot;),&#xA;                shell(&amp;quot;cut -f 4 {params.prefix}.unlifted | sed '/^#/d' &amp;gt;&amp;gt; {params.prefix3}&amp;quot;),&#xA;                shell(&amp;quot;less {params.prefix3} | {output.exclusion}&amp;quot;),&#xA;                shell(&amp;quot;awk 'NR==FNR{{a[&lt;span class=&quot;math-container&quot;&gt;$1]=$&lt;/span&gt;4;next}}{{ if (a[&lt;span class=&quot;math-container&quot;&gt;$1]!= 'NULL') {{ if (a[$1] == 'female') {{ $5=2; print }} else if (a[$1] == 'male') {{ $5=1; print }} else {{$&lt;/span&gt;5=0; print}}}}}}' {params.sexes} {params.prefix}.ped &amp;gt; Results/{wildcards.sample}_{wildcards.location}_LIFTED.ped&amp;quot;)&#xA;                # ToDo: Add conditionals for other human reference genome builds&#xA;        else:&#xA;            print(&amp;quot;No liftover required. Dataset {} is already mapped to GRCh38.&amp;quot;.format(wildcards.sample)),&#xA;            shell(&amp;quot;cp Results/{wildcards.sample}_{wildcards.location}_TRIMMED.map Results/{wildcards.sample}_{wildcards.location}_LIFTED.map&amp;quot;),&#xA;            shell(&amp;quot;cp Results/{wildcards.sample}_{wildcards.location}_TRIMMED.ped Results/{wildcards.sample}_{wildcards.location}_LIFTED.ped&amp;quot;),&#xA;            shell(&amp;quot;touch Resuts/{wildcards.sample}_{wildcards.location}_EXCLUDE.dat&amp;quot;)&#xA;&#xA;&#xA;rule CLEAN:&#xA;    input:&#xA;        mapFile=&amp;quot;Results/{sample}_{location}_LIFTED.map&amp;quot;,&#xA;        pedFile=&amp;quot;Results/{sample}_{location}_LIFTED.ped&amp;quot;,&#xA;        exclude=&amp;quot;Results/{sample}_{location}_EXCLUDE.dat&amp;quot;&#xA;&#xA;    output:&#xA;        expand(&amp;quot;Results/{{sample}}_{{location}}_CLEANED.{extension}&amp;quot;, extension=['bed', 'bim', 'fam'])&#xA;&#xA;    shell:&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        module load plink-1.9&#xA;        plink --map {input.mapFile} --ped {input.pedFile} --out Results/{wildcards.sample}_{wildcards.location}_CLEANED --make-bed --keep-allele-order --exclude {input.exclude}&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8546" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2020-04-08T12:18:11.087" Title="Snakemake on cluster produces empty output files" Tags="|snakemake|workflow-management|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="12868" PostTypeId="1" AcceptedAnswerId="12897" CreationDate="2020-04-07T18:44:39.463" Score="6" ViewCount="5270" Body="&lt;p&gt;I have a BAM file from a single cell sequencing experiment. Each read has had the cell barcode annotated in the &lt;code&gt;CB&lt;/code&gt; tag. Some reads do not have a &lt;code&gt;CB&lt;/code&gt; tag. For each of an arbitrary set of barcodes, I would like to split out the reads for each of those barcodes into a separate file. Is there a tool that does this?&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2022-08-09T22:45:04.413" Title="Is there a command line tool to split a SAM/BAM file by CB (cell barcode) tag?" Tags="|sam|single-cell|data-preprocessing|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12869" PostTypeId="2" ParentId="12868" CreationDate="2020-04-07T18:44:39.463" Score="4" Body="&lt;p&gt;I wrote a &lt;a href=&quot;https://gist.github.com/winni2k/978b33d62fee5e3484ec757de1a00412&quot; rel=&quot;nofollow noreferrer&quot;&gt;python script&lt;/a&gt; based on pysam that is free for anyone to use:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;&quot;&quot;&quot;&#xA;MIT License&#xA;Copyright (c) 2020 Warren W. Kretzschmar&#xA;Permission is hereby granted, free of charge, to any person obtaining a copy&#xA;of this software and associated documentation files (the &quot;Software&quot;), to deal&#xA;in the Software without restriction, including without limitation the rights&#xA;to use, copy, modify, merge, publish, distribute, sublicense, and/or sell&#xA;copies of the Software, and to permit persons to whom the Software is&#xA;furnished to do so, subject to the following conditions:&#xA;The above copyright notice and this permission notice shall be included in all&#xA;copies or substantial portions of the Software.&#xA;THE SOFTWARE IS PROVIDED &quot;AS IS&quot;, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR&#xA;IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,&#xA;FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE&#xA;AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER&#xA;LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,&#xA;OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE&#xA;SOFTWARE.&#xA;&quot;&quot;&quot;&#xA;&#xA;from pathlib import Path&#xA;&#xA;import pysam&#xA;&#xA;&#xA;class BamWriter:&#xA;    def __init__(self, alignment, barcodes, prefix):&#xA;        self.alignment = alignment&#xA;        self.prefix = prefix&#xA;        self.barcodes = set(barcodes)&#xA;        self._out_files = {}&#xA;&#xA;    def write_record_to_barcode(self, rec, barcode):&#xA;        if barcode not in self.barcodes:&#xA;            return&#xA;        if barcode not in self._out_files:&#xA;            self._open_file_for_barcode(barcode)&#xA;        self._out_files[barcode].write(rec)&#xA;&#xA;    def _open_file_for_barcode(self, barcode):&#xA;        self._out_files[barcode] = pysam.AlignmentFile(&#xA;            f&quot;{self.prefix}_{barcode}.bam&quot;, &quot;wb&quot;, template=self.alignment&#xA;        )&#xA;&#xA;&#xA;def main(input_bam, barcodes_file, output_prefix, contigs):&#xA;    &quot;&quot;&quot;Split a 10x barcoded sequencing file into barcode-specific BAMs&#xA;&#xA;    input:&#xA;    barcodes_file: a file containing barcodes or a single barcode&#xA;    contigs: '.' for all contigs, 'chr1' for the contig 'chr1',&#xA;    or '1-5' for chromosomes 1, 2, 3, 4, and 5&#xA;    &quot;&quot;&quot;&#xA;    alignment = pysam.AlignmentFile(input_bam)&#xA;    if Path(barcodes_file).is_file():&#xA;        with open(barcodes_file, &quot;r&quot;) as fh:&#xA;            barcodes = [l.rstrip() for l in fh.readlines()]&#xA;    else:&#xA;        barcodes = [barcodes_file]&#xA;        print(f&quot;Extracting single barcode: {barcodes}&quot;)&#xA;    writer = BamWriter(alignment=alignment, barcodes=barcodes, prefix=output_prefix)&#xA;    if contigs == &quot;.&quot;:&#xA;        print(&quot;Extracting reads from all contigs&quot;)&#xA;        recs = alignment.fetch()&#xA;    else:&#xA;        if &quot;-&quot; in contigs:&#xA;            start, end = contigs.split(&quot;-&quot;)&#xA;            print(f&quot;Extracting reads from contigs {start} to {end}&quot;)&#xA;            recs = (alignment.fetch(str(contig)) for contig in range(start, end + 1))&#xA;        elif &quot;,&quot; in contigs:&#xA;            contigs = contigs.split(&quot;,&quot;)&#xA;            print(f&quot;Extracting reads from contigs {contigs}&quot;)&#xA;            recs = (alignment.fetch(str(contig)) for contig in contigs)&#xA;        else:&#xA;            print(&quot;Extracting reads for one contig: {contigs}&quot;)&#xA;            recs = (alignment.fetch(c) for c in [contigs])&#xA;&#xA;    for rec in recs:&#xA;        try:&#xA;            barcode = rec.get_tag(&quot;CB&quot;)&#xA;            writer.write_record_to_barcode(rec=rec, barcode=barcode)&#xA;        except KeyError:&#xA;            pass&#xA;&#xA;&#xA;if __name__ == &quot;__main__&quot;:&#xA;    import sys&#xA;&#xA;    main(&#xA;        input_bam=sys.argv[1],&#xA;        barcodes_file=sys.argv[2],&#xA;        output_prefix=sys.argv[3],&#xA;        contigs=sys.argv[4],&#xA;    )&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="492" LastEditorUserId="492" LastEditDate="2020-04-08T09:03:09.250" LastActivityDate="2020-04-08T09:03:09.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12870" PostTypeId="1" CreationDate="2020-04-08T00:36:31.087" Score="1" ViewCount="278" Body="&lt;p&gt;I am using &lt;a href=&quot;https://www.ensembl.org/info/docs/tools/vep/script/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;VEP&lt;/a&gt; from Ensembl to annotate my VCF files with the extra plugins of &lt;a href=&quot;https://www.ensembl.org/info/docs/tools/vep/script/vep_plugins.html#genesplicer&quot; rel=&quot;nofollow noreferrer&quot;&gt;GeneSplicer&lt;/a&gt;, &lt;a href=&quot;https://www.ensembl.org/info/docs/tools/vep/script/vep_plugins.html#maxentscan&quot; rel=&quot;nofollow noreferrer&quot;&gt;MaxEntScan&lt;/a&gt; and &lt;a href=&quot;https://www.ensembl.org/info/docs/tools/vep/script/vep_plugins.html#spliceregion&quot; rel=&quot;nofollow noreferrer&quot;&gt;SpliceRegion&lt;/a&gt;. However, I don't fully understand the output of these scores.&lt;/p&gt;&#xA;&lt;p&gt;I know that they are used for splicing predictions on trained sets.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What are the range of these values ?&lt;/li&gt;&#xA;&lt;li&gt;What does a high or low value mean ?&lt;/li&gt;&#xA;&lt;li&gt;What is the threshold ?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Can someone please explain them please ?&lt;/p&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="3848" LastEditDate="2020-11-18T20:44:49.890" LastActivityDate="2020-11-18T20:44:49.890" Title="How are the scores of GeneSplicer, MaxEntScan and SpliceRegion interpreted from VEP annotations?" Tags="|machine-learning|ensembl|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12871" PostTypeId="2" ParentId="8712" CreationDate="2020-04-08T00:54:31.753" Score="3" Body="&lt;p&gt;Here's an example I wrote recently:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-java prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env nextflow&#xA;&#xA;params.requestedCPU = 80&#xA;params.requestedMem = 50&#xA;&#xA;maxcpus = Runtime.runtime.availableProcessors()&#xA;requestedCpus = Math.round((params.requestedCPU * maxcpus) / 100)&#xA;&#xA;maxmem = Runtime.runtime.totalMemory() / 10241024&#xA;requestedMem = (Math.round((params.requestedMem * maxmem) / 100))&#xA;&#xA;process foo {&#xA;    echo true&#xA;    cpus = requestedCpus&#xA;    memory = requestedMem.GB&#xA;&#xA;    script:&#xA;        &quot;&quot;&quot;&#xA;        echo &lt;span class=&quot;math-container&quot;&gt;$requestedCpus&#xA;	echo $&lt;/span&gt;requestedMem&#xA;        &quot;&quot;&quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can pass the number of CPUs or Memory directly on the command line in percentage, e.g. &lt;code&gt;nextflow run main.nf --requestedCPU 100&lt;/code&gt; to have all CPU usages. Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2020-04-08T00:54:31.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12872" PostTypeId="1" CreationDate="2020-04-08T08:27:18.557" Score="2" ViewCount="342" Body="&lt;p&gt;I am using Platanus-allee 2.2.2 for heterozygous genome (~500mb) assembly with Illumina short reads and PacBio reads input data.&lt;/p&gt;&#xA;&lt;p&gt;I have the file contigs.fa from short reads but phasing step with PacBio reads gives the following error after running for couple of hours&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error(13): Error, SolveDBG exception!! platanus_allee solve_DBG&#xA;command failed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here is the script used for contig assembly and phasing&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;num_threads=20 &#xA;tmp_dir=/path-to-/tem/platanus_allee &#xA;memory_GB=100 &#xA;out_prefix=out&#xA;minimap2_path=minimap2&#xA;&#xA;# Input reads&#xA;&#xA;PE=trimmed_L1_*.fq&#xA;PacBio=pacbio???.fastq.gz&#xA;&#xA;# Contig assembly commands&#xA;platanus_allee assemble -t &lt;span class=&quot;math-container&quot;&gt;${num_threads} -tmp $&lt;/span&gt;{tmp_dir} -m &lt;span class=&quot;math-container&quot;&gt;${memory_GB} -o $&lt;/span&gt;{out_prefix} \&#xA;-f &lt;span class=&quot;math-container&quot;&gt;$PE \&#xA;2&amp;gt;$&lt;/span&gt;{out_prefix}.assembleLog&#xA;&#xA;# phasing commands &#xA;platanus_allee phase -t &lt;span class=&quot;math-container&quot;&gt;${num_threads} -tmp $&lt;/span&gt;{tmp_dir} -o &lt;span class=&quot;math-container&quot;&gt;${out_prefix} -mapper $&lt;/span&gt;{minimap2_path} \&#xA;-c out_contig.fa \&#xA;-IP1 &lt;span class=&quot;math-container&quot;&gt;$PE -p $&lt;/span&gt;PacBio \ &#xA;2&amp;gt;${out_prefix}.phaseLog&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;cat out.phaseLog&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;platanus_allee version: 2.2.2&#xA;platanus_allee phase -t 20 -tmp /path-to-/tem/platanus_allee -o out -mapper minimap2 -c out_contig.fa -IP1 trimmed_L1_1.fq trimmed_L1_2.fq -p /path-to-/pacbio*.fastq.gz &#xA;&#xA;Error(13): Error, SolveDBG exception!!&#xA;platanus_allee solve_DBG command failed.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;tail -n 15 path-to-/out_intermediateResults/round1/out.solveDBGLog&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dividing nodes based on bubbles ...&#xA;dividing scaffolds to adjust bubble-breakpoints...&#xA;dividing indonsistent bubble-ends...&#xA;pairing bubble nodes...&#xA;dividing scaffolds to adjust bubble-breakpoints distiguishing strands ...&#xA;NUM_DIVISION = 1832&#xA;dividing scaffolds to adjust bubble-breakpoints...&#xA;dividing indonsistent bubble-ends...&#xA;pairing bubble nodes...&#xA;dividing scaffolds to adjust bubble-breakpoints distiguishing strands ...&#xA;NUM_DIVISION = 0&#xA;TOTAL_NUM_DIVISIONS =1832&#xA;&#xA;dividing erroneous scaffolds based on base-level coverages ...&#xA;Segmentation fault (core dumped)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A bug causing the segmentation fault seems issue on &lt;a href=&quot;http://platanus.bio.titech.ac.jp/platanus2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Platanus-allee v2.0.0 and earlier&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8574" LastEditorUserId="3848" LastEditDate="2020-07-17T19:22:37.690" LastActivityDate="2020-07-17T19:22:37.690" Title="Platanus-allee phasing fail: Error(13): Error, SolveDBG exception!" Tags="|sequence-alignment|assembly|software-recommendation|long-reads|pacbio|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12873" PostTypeId="1" CreationDate="2020-04-08T10:46:14.810" Score="1" ViewCount="549" Body="&lt;p&gt;I got a genome (&lt;code&gt;data/genome/genome.fasta&lt;/code&gt;) and braker-based genome annotation (&lt;code&gt;data/genome/annotation.gff3&lt;/code&gt;), now I would like to get sequences of one transcript per gene (let's say the longest one).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cufflinks has a &lt;code&gt;gffread&lt;/code&gt; program to get the transcriptome out of the genome and the annotation. Something like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gffread data/genome/annotation.gff3 -g data/genome/genome.fasta -w data/genome/transcripts.fasta -y data/genome/proteins.faa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Will do the job. However, it is pulling multiple transcripts per gene. I read the help page twice, found parameters that could potentially do what I would like to (collapsing alternative transcripts into single one), but even when I specify my candidate parameters &lt;code&gt;-M -K&lt;/code&gt; I get still &lt;code&gt;.t1&lt;/code&gt; and &lt;code&gt;.t2&lt;/code&gt; sequences for some genes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The task is surprisingly simple, and I got already a custom script solution, but it just feels wrong. There must be a better way! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What is a reasonable/elegant way to get nucleotide sequences of genes out of a genome and annotation? Am I missing something with cufflinks?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Related, not exactly identical: &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/5585/how-to-get-the-longest-fasta-sequence-including-all-possible-switching-isoforms&quot;&gt;How to get the longest fasta sequence including all possible switching isoforms of a gene out of isoforms&lt;/a&gt; (dealing more with terminology of Trinity)&lt;/li&gt;&#xA;&lt;li&gt;When I used Kallisto/Salmon for differential expression analysis, I collapsed transcripts into genes while importing to R via &lt;code&gt;tximport&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2020-04-09T09:10:22.233" LastActivityDate="2020-04-09T09:10:22.233" Title="Getting sequences of one transcript per gene out of annotation and genome" Tags="|genome|sequence-annotation|isoform|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12874" PostTypeId="1" CreationDate="2020-04-08T11:53:05.030" Score="0" ViewCount="46" Body="&lt;p&gt;Hope everyone is well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So i was appointed with a task, and i need some guidance. Attention, the one that gave me the task may not have explained this very clearly. I will write what he exactly asked of me, and what he gave me as primary data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So he gave me 4 bed files (human genome) with chr and coordinates(start/stop), and asked me to do: &quot;sequencing/coordinates conservation between diverse species&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is that doable with coordinates only or do i need sequences in fasta format (phast uses fasta format)? I just dont understand how i will find &quot;conservation between diverse species&quot; only with coordinates. It seems weird to me.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found these programs: TargetOrtho2, Phast and BUSCO(?). Are they any good? Should i continue with one of them that you may have experience with?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="7172" LastActivityDate="2020-04-20T16:50:12.580" Title="Phylogenetics analysis between diverse species" Tags="|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12875" PostTypeId="2" ParentId="12867" CreationDate="2020-04-08T12:18:11.087" Score="1" Body="&lt;p&gt;Ok, so I feel like a right PawPaw. This is &lt;strong&gt;NOT a snakemake problem&lt;/strong&gt;. The issue was multi-part but it was me, python and good ol' PBS/Torque Cluster environments.&lt;/p&gt;&#xA;&lt;h3&gt;Empty input files:&lt;/h3&gt;&#xA;&lt;p&gt;I was using a wrapper python script by the name of &lt;a href=&quot;https://github.com/sritchie73/liftOverPlink&quot; rel=&quot;nofollow noreferrer&quot;&gt;liftOverPlink.py&lt;/a&gt; to convert my files from PLINK-1.9's binary file format into the format needed by the software liftOver by UCSC. In my infinite wisdom, I did make some custom alterations. Turns out in python, there is a big difference between &lt;code&gt;&amp;quot;w+&amp;quot;&lt;/code&gt; and &lt;code&gt;&amp;quot;a+&amp;quot;&lt;/code&gt; for the &lt;code&gt;open()&lt;/code&gt; function &lt;em&gt;(&lt;code&gt;&amp;quot;w+&amp;quot;&lt;/code&gt; = write mode and create if needed, &lt;code&gt;&amp;quot;a+&amp;quot;&lt;/code&gt; = append mode and create if needed)&lt;/em&gt;. &lt;code&gt;&amp;quot;w+&amp;quot;&lt;/code&gt; opens or creates a file but places the cursor at the beginning of the file, effectively wiping the file if it already had content inside, wheras &lt;code&gt;&amp;quot;a+&amp;quot;&lt;/code&gt; puts the cursor at the end of the files content (which is beginning of file if it had to create file). I used &lt;code&gt;&amp;quot;w+&amp;quot;&lt;/code&gt; which was effectively erasing my input files content...&lt;/p&gt;&#xA;&lt;h3&gt;Why alter the script in the first place?&lt;/h3&gt;&#xA;&lt;p&gt;As it turns out, our PBS/Torque cluster was not properly serving up the software at runtime (software in question being liftOver). From what I have been able to gather, normally liftOverPlink would create intermediate files, liftOver would run and create its files where necessary, and liftOverPlink would then step in and perform format conversion back into PLINK-1.9 format. Since liftOver was not firing, the files did not exist and liftOverPlink kicked up errors since it was never designed to handle that (functionality to detect missing files would have been redundant). I attempted to ugly-fix it by using&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;filename = open(filename, &amp;quot;w+&amp;quot;)&#xA;filename.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to open and close the file, effectively forcefully creating it if needed. Unfortunately that let to the above problem. Providing an absolute path to the liftOver executable fixed runtime issues and my custom code was no-longer necessary.&lt;/p&gt;&#xA;" OwnerUserId="8546" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2020-04-08T12:18:11.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12876" PostTypeId="1" CreationDate="2020-04-08T13:00:16.627" Score="0" ViewCount="118" Body="&lt;p&gt;I know this question may seem strange.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm using Spearman correlation between gene expression profiles for various reasons (I won't go into details here). As a result, I often compare RNA-Seq and Microarray samples.&#xA;For preliminary analysis, I usually grab what version of the data is easily accessible (RPKM, FPKM,...), but I'd like to dig a bit more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Intuitively, I'd think a value similar to RPKM or such would make more sense than raw counts, which is why I usually convert raw counts to RPKM (&lt;strong&gt;I know they're considered obsolete for statistical analysis of the data&lt;/strong&gt;, what I'm doing relies on the correlation)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I ask what RNA-Seq gene expression value would be closest to a microarray equivalent ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;i.e.&lt;/em&gt; What would (theoretically) maximize correlation between gene expression of the  same sample profiled with RNA-seq and microarray. RPKM ? FPKM ? TPM ? etc.&lt;/p&gt;&#xA;" OwnerUserId="8575" LastActivityDate="2020-04-12T08:17:03.600" Title="What RNA-Seq expression value would be closest to Microarray equivalent?" Tags="|rna-seq|transcriptome|microarray|fpkm|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12877" PostTypeId="1" AcceptedAnswerId="12879" CreationDate="2020-04-08T13:01:21.570" Score="-4" ViewCount="65" Body="&lt;p&gt;I have a data frame&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Response_TO_CHEMOTHERAPY    Gene    CCF&#xA;Responders                  POLQ    1&#xA;Responders                 CDKN2A   1&#xA;Responders                  TP53    1&#xA;Responders              CCDC102B    0.906729075&#xA;Non-responders           CCDC102B   1&#xA;Non-responders           CCDC102B   1&#xA;Non-responders           CCDC102B   1&#xA;Non-responders             TSHZ3    1&#xA;Non-responders             PIK3CA   1&#xA;Responders                 SLIT2    1&#xA;Non-responders              SLIT2   1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have &lt;code&gt;two groups of patients&lt;/code&gt;; &lt;code&gt;Responders to chemotherapy and non-responders&lt;/code&gt;. For each group I have calculated &lt;code&gt;cancer cell faction (CCF)&lt;/code&gt; for a fixed set of genes so that an individual gene has several reads in each group&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &lt;code&gt;CCF = 1&lt;/code&gt; the gene clonal and if CCF &amp;lt; 1 gene is sub clonal, for a given gene for each group, I therefore would have the number of clonal and sub clonal for this gene, for example 2 clonal 7 sub clonal which would be 2/7 something like the below figure&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6DzE8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6DzE8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is another potential example which reveals what is in my mind&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HeabT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HeabT.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Which show the clonal frequency for each group&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know how to use &lt;code&gt;dput()&lt;/code&gt; but this is my complete data&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/ol7m4nlkuyuy85a/c1.txt?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/ol7m4nlkuyuy85a/c1.txt?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-04-08T16:04:45.180" LastActivityDate="2020-04-08T16:30:29.813" Title="How to visualise the proportion of clonal genes to sub clonal genes" Tags="|r|ggplot2|cnv|heatmap|complexheatmap|" AnswerCount="1" CommentCount="1" ClosedDate="2020-04-09T09:05:03.490" ContentLicense="CC BY-SA 4.0" />
  <row Id="12879" PostTypeId="2" ParentId="12877" CreationDate="2020-04-08T13:46:06.977" Score="2" Body="&lt;p&gt;I've used some dummy data for simplicity. It's fairly easy to do this with ggplot, but you have to format your matrix-like data for ggplot input.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;eg :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# make data&#xA;set.seed(1)&#xA;ngenes &amp;lt;- 20&#xA;nsamps &amp;lt;- 10&#xA;my_dat &amp;lt;- matrix(rnorm(ngenes*nsamps), ncol = nsamps, nrow = ngenes)&#xA;colnames(my_dat) &amp;lt;- paste(&quot;samp&quot;, seq_len(nsamps))&#xA;rownames(my_dat) &amp;lt;- paste(&quot;gene&quot;, seq_len(ngenes))&#xA;&#xA;library(ggplot2)&#xA;library(reshape2) # for melt&#xA;&#xA;# format data for plotting&#xA;dat_toplot &amp;lt;- melt(my_dat)&#xA;&#xA;#plot&#xA;ggplot(dat_toplot, aes(x = Var1, y = Var2, fill = value)) + geom_tile()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;plots&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uNLW0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uNLW0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you need specifics on plot tweaking, google's your friend. There are plenty of tutorials on how to use ggplot for heatmaps. (eg. &lt;a href=&quot;https://learnr.wordpress.com/2010/01/26/ggplot2-quick-heatmap-plotting/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, or &lt;a href=&quot;https://jcoliver.github.io/learn-r/006-heatmaps.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;EDIT&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;I loaded your data into the &lt;code&gt;dat&lt;/code&gt; object.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse)&#xA;dat &amp;lt;- as_tibble(dat)&#xA;&#xA;dat_counts &amp;lt;- dat %&amp;gt;% group_by(Sample, Gene) %&amp;gt;% summarise(count_subclonal = sum(CCF&amp;lt;1), count_clonal = sum(CCF&amp;gt;=1))&#xA;dat_counts&lt;span class=&quot;math-container&quot;&gt;$ratio &amp;lt;- dat_counts$&lt;/span&gt;count_subclonal/dat_counts$count_clonal&#xA;&#xA;# set infinite values (/0) to NA&#xA;dat_counts&lt;span class=&quot;math-container&quot;&gt;$ratio[is.infinite(dat_counts$&lt;/span&gt;ratio)] &amp;lt;- NA&#xA;&#xA;&#xA;ggplot(dat_counts, aes(x = Sample, y = Gene, fill = ratio)) + geom_tile() + &#xA;  geom_text(aes(label = paste(count_subclonal, &quot;/&quot;, count_clonal)), col = 'white') # add ratio text&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Outputs :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fjKie.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fjKie.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8575" LastEditorUserId="8575" LastEditDate="2020-04-08T16:30:29.813" LastActivityDate="2020-04-08T16:30:29.813" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12880" PostTypeId="1" CreationDate="2020-04-08T14:13:26.143" Score="1" ViewCount="32" Body="&lt;p&gt;I have a Human genome sequenced in a BAM file (along with other files with the indels, snps, cnvs).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to find every mutation with regards to the reference Human genome. However the majority of &quot;variants&quot; are noise/read errors. I am therefore required to filter the variants via a minimum threshold based on allele freq (eg. 0,5).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using a reference (eg Hg19), how can I find the position of every mutation where the allele frequency is greater than the threshold?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS. I am just starting, so I don't know which are the best (command line?) tools for the job nor how to build a pipeline for this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Help would be greatly appreciated&lt;/p&gt;&#xA;" OwnerUserId="8577" LastEditorUserId="3848" LastEditDate="2020-04-09T17:46:51.223" LastActivityDate="2020-04-09T17:46:51.223" Title="How can I find the position of every mutation where the Allele Frequency is greater than X, in regards to a reference such as Hg19?" Tags="|genome|ngs|variant-calling|allele-frequency|hgp|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12881" PostTypeId="1" CreationDate="2020-04-08T14:30:38.760" Score="1" ViewCount="136" Body="&lt;p&gt;Dear Researches community &lt;/p&gt;&#xA;&#xA;&lt;p&gt;A question concerning the current outbreak, recently we attempting to contribute to the study of the Covid19, with a tight submission time ( 4 weeks).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Inquiring help because I'm not a virologist with an insufficient biological background. However, I assume able to do in silico analysis or ( dry lab). &#xA;Moreover, &lt;strong&gt;new sample&lt;/strong&gt; sequences from local &lt;strong&gt;strain&lt;/strong&gt; will be provided. Assembly and annotation could be done at the first stage. Then perhaps a comparative genome investigation between the old SARS and the Cuvid19.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, The virus estimation length that causes all this is only around 29,829 base pairs of RNA. The genome contains at least 10 predicted open reading frames (ORFs): ORF1a,  ORF1b, S,  3,  4a, 4b,  5,  E,  M  and  N,  with sixteen predicted nonstructural proteins being encoded by ORF1a/b. &#xA;I am wondering if someone can assist by considering a particular section to focus on or a study suggests. Perhaps we can contribute at least 0.001 % to the communities.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been reading articles around, and seems are thousand of them every day.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;so I have come up with starting list:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Genome assemblies, annotation&lt;/li&gt;&#xA;&lt;li&gt;epitome studies&#xA;Here you predict the protein sequences based on the genome that will sequence, then there are a number of tools that analyze the immunogenicity of different proteins&#xA;That can help you identify parts of the COVID-19 proteins that are likely to be detected by the immune system and they, of course, can be targeted by vaccine developers.&lt;/li&gt;&#xA;&lt;li&gt;comparison between old saras Vs newCoVID&lt;/li&gt;&#xA;&lt;li&gt;The M protein is a good target. It is extremely abundant in the nucleocapsid and is the protein responsible for giving the virus it's the shape, Almost everyone is looking at the S protein, which is justified. But there is not as much focus on the M protein.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Your response is highly appreciated if you could attach the any reference related to the list above.&lt;/p&gt;&#xA;" OwnerUserId="3332" LastEditorUserId="3848" LastEditDate="2020-11-24T13:43:43.587" LastActivityDate="2020-11-24T13:43:43.587" Title="Concerning the current outbreak, using bioinformatics" Tags="|covid-19|literature-search|" AnswerCount="2" CommentCount="5" ClosedDate="2020-04-11T18:12:22.453" ContentLicense="CC BY-SA 4.0" />
  <row Id="12882" PostTypeId="2" ParentId="12881" CreationDate="2020-04-08T14:54:15.687" Score="4" Body="&lt;p&gt;My personal approach to something like this is to first try to reproduce existing work in order to learn to use relevant tools and understand the concepts involved. Then as you get more comfortable with the subject matter, you may spot an opportunity to add value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you'd like to get started with the phylogenomics side of things, you can use this public workspace developed by the viral genomics group at the Broad Institute, which includes viral genome sequences from NCBI SRA as well as some workflows for doing genome assembly and quality control: &lt;a href=&quot;https://app.terra.bio/#workspaces/pathogen-genomic-surveillance/COVID-19&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://app.terra.bio/#workspaces/pathogen-genomic-surveillance/COVID-19&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2776" LastEditorUserId="2776" LastEditDate="2020-04-08T17:52:42.407" LastActivityDate="2020-04-08T17:52:42.407" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12883" PostTypeId="2" ParentId="12873" CreationDate="2020-04-08T14:57:21.883" Score="1" Body="&lt;p&gt;Here is the custom python script solution:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;from collections import defaultdict&#xA;from Bio import SeqIO&#xA;&#xA;# read file into dictionary of transcrpt-&amp;gt;gene&#xA;# jg25690   jg25690.t1&#xA;# jg25690   jg25690.t2&#xA;# jg25691   jg25691.t1&#xA;transcripts2genes = dict()&#xA;gene2transcript_mapfilename = 'data/genome/transcripts2genes.map'&#xA;with open(gene2transcript_mapfilename, 'r') as g2t_file:&#xA;    for line in g2t_file:&#xA;        gene, trainscript = line.rstrip('\n').split('\t')&#xA;        transcripts2genes[trainscript] = gene&#xA;&#xA;# construct a dictionary gene-&amp;gt;list of transcripts &#xA;# this fasta file is output of default gffread -w&#xA;genes2transcript_seuqneces = defaultdict(list)&#xA;transcripts_fasta_filename = 'data/genome/transcripts.fasta'&#xA;transcripts_fasta_file = SeqIO.parse(transcripts_fasta_filename, 'fasta')&#xA;for seq_record in transcripts_fasta_file:&#xA;    gene = transcripts2genes[seq_record.name]&#xA;    genes2transcript_seuqneces[gene].append(seq_record)&#xA;&#xA;genes_fasta_filename = 'data/genome/genes.fasta'&#xA;with open(genes_fasta_filename, 'w') as out_file:&#xA;    # iterate through genes&#xA;    for gene in genes2transcript_seuqneces.keys():&#xA;        # get all the associated transcripts&#xA;        transcripts = genes2transcript_seuqneces[gene]&#xA;        # find the longest, by default the one with index 0&#xA;        lengest = 0&#xA;        for index, tr in enumerate(transcripts):&#xA;            if len(transcripts[lengest].seq) &amp;gt; len(tr.seq):&#xA;                lengest = index&#xA;        # this would be nicer with Bio interface (with wrapping of sequence lines)&#xA;        # but I was lazy and blast does not care, only thing I do is converting sequence to uppercase (not even sure if necessary)&#xA;        fasta_string = '&amp;gt;' + gene + '\n' + str(transcripts[lengest].upper().seq) + '\n'&#xA;        out_file.write(fasta_string)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="57" LastActivityDate="2020-04-08T14:57:21.883" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12884" PostTypeId="1" CreationDate="2020-04-08T18:47:27.973" Score="0" ViewCount="214" Body="&lt;p&gt;I want to use RDKit with a python script in PyMOL. RDKit requires a conda environment, so I think I need to be able to specify that the instance of python that I want to use would be in that environment, instead of the default for PyMOL, so that I can run it through PyMOL. The script I made is working for me through PyCharm when I test it, by using the RDkit environment &quot;rdkit-env&quot; as my python interpreter, with all of the additional modules I need installed there as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to run it through PyMOL so I can add some PyMOL specific functionality as well. I've tried using shebang to specify the RDkit environment that I made, but that does not seem to be working for me with Windows.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone please help me figure out what other approach might work for this? I'm new to this area, so I would appreciate any insights. Thank you!&lt;/p&gt;&#xA;" OwnerUserId="8579" LastActivityDate="2020-04-09T07:24:36.437" Title="Use a specific python.exe instance with PyMOL/Windows?" Tags="|python|pymol|conda|windows|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12885" PostTypeId="2" ParentId="12881" CreationDate="2020-04-08T20:37:45.110" Score="2" Body="&lt;p&gt;Please note that Cuvid19 is COVID-19, the WHO nomenclature. SARS2 is specifically called SARS-CoV-2 and is the virologists name for COVID-19 because that is the nomenclature of the International Committee on the Taxonomy of Viruses (ICTV) for the same virus following formal taxonomic investigation. It is important not to get the names confused.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Technically you are asking to compare COVID-19 with COVID-19. Whilst that is a standard investigation this would be technically rewritten &quot;to investigate the molecular epidemiology of COVID-19&quot;.  If you are working within the virological community then the virus is termed SARS2-CoV-2. Phylogeny is a key approach, rather than &quot;comparative genomics&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wish you luck with your study. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-04-08T20:45:03.127" LastActivityDate="2020-04-08T20:45:03.127" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12886" PostTypeId="2" ParentId="12884" CreationDate="2020-04-09T07:24:36.437" Score="1" Body="&lt;p&gt;So it is really hard to make a given python instance run a library from a different instance. It is not just one &lt;code&gt;dist-packages&lt;/code&gt; library folder, but multiple that you would need to link, and all that assuming they use the same version of python.&#xA;So the best bet is to install a new instance of pymol the python module.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The module can be installed in a python that uses conda with&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c schrodinger pymol&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the incentive version of PyMOL 2 and does not require a licence.&#xA;You installed rdkit, so I am assuming you used conda as &lt;code&gt;conda install -c rdkit rdkit&lt;/code&gt;. But just in case you went through the pain of compiling rdkit for a given python instance without conda and want the same for PyMOL. There are guides of instructions required to compile open source. Such as &lt;a href=&quot;https://pymolwiki.org/index.php/Windows_Install&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pymolwiki.org/index.php/Windows_Install&lt;/a&gt;. On a barebone OS like a Pi's Raspsian, I had to do a lot more than claimed say &lt;code&gt;sudo apt-get install python3-dev libglm-dev freeglut3-dev libglew-dev libpng12-dev libfreetype6-dev build-essential libxml++3.6-dev&lt;/code&gt; and link a few things manually. So just like for rdkit, conda make life a breeze.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PS. RDKit and PyMOL? Add PyRosetta and you just my favourite modules!&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-04-09T07:24:36.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12887" PostTypeId="2" ParentId="12866" CreationDate="2020-04-09T07:26:02.033" Score="-2" Body="&lt;p&gt;To convert data from one format into another, you may find some software tool sometimes. But mostly no such tool can be found to satisfy all the requirements that we want.&lt;/p&gt;&#xA;&lt;p&gt;In such situations, I recommend that you write a C program that reads data from the input file and writes it into the output file. In this way, you have complete flexibility so that you can determine the exact output format and what part of data to ignore or include. To do that, you should be familiar with some concepts in C language such as arrays, loops, strings, and file operations.&lt;/p&gt;&#xA;&lt;p&gt;For an example of such conversions, you can get ideas from the following page:&#xA;&lt;a href=&quot;https://stackoverflow.com/questions/55347322/c-program-to-convert-a-text-file-into-a-csv-file&quot;&gt;https://stackoverflow.com/questions/55347322/c-program-to-convert-a-text-file-into-a-csv-file&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Of course, such a converter program can be written in other languages too such as C# and Python.&lt;/p&gt;&#xA;&lt;p&gt;The program can be developed in R language too. You need to be familiar with import functions, loops, strings, and arrays in this language.&lt;/p&gt;&#xA;&lt;p&gt;We cannot expect that custom tools exist for all our file convertions. Every bioinformatician must be skilled in a standard programming language such as C, C#, Java, Python, or R. Then, he/she will be able to develop any custom program that reads an input file and generates a custom output file.&lt;/p&gt;&#xA;" OwnerUserId="8573" LastEditorUserId="73" LastEditDate="2021-05-07T10:04:32.803" LastActivityDate="2021-05-07T10:04:32.803" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12888" PostTypeId="2" ParentId="12862" CreationDate="2020-04-09T07:37:42.307" Score="4" Body="&lt;p&gt;With pymol as a python module (&lt;code&gt;conda install -c schrodinger pymol&lt;/code&gt;, not the GUI one), it is very easy.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pymol2&#xA;with pymol2.PyMOL() as pymol:&#xA;     pymol.cmd.load('protein.pdbqt', 'protein') # the second is the name within pymol instance.&#xA;     pymol.cmd.load('ligand.sdf', 'ligand')&#xA;     pymol.cmd.save('neighbours.pdb','ligand expand 3') # get atoms 3 Å around&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Done!&#xA;However, if you are doing stuff in python in the first place you might want the last line as a string: &lt;code&gt;pymol.cmd.get_pdbstr('ligand expand 3')&lt;/code&gt;. Changing the selection &lt;code&gt;'ligand expand 3'&lt;/code&gt; to &lt;code&gt;'byres ligand expand 3'&lt;/code&gt; will save the full residues.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively you can get the atom coordinates themselves etc. with it. To see what attributes &lt;code&gt;Atom&lt;/code&gt; has you have you can do this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;for atom in pymol.cmd.get_model('ligand expand 3').atom:&#xA;    print(dir(atom))&#xA;    break&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which will give a large amount of attributes, such as &lt;code&gt;atom.coord&lt;/code&gt;, which is a 3-element list.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-04-11T16:27:21.323" LastActivityDate="2020-04-11T16:27:21.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12889" PostTypeId="2" ParentId="12880" CreationDate="2020-04-09T08:19:25.367" Score="2" Body="&lt;p&gt;Hello and welcome to the world of bioinformatics,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;BAM files are compressed formats of SAM and you can read more about it &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To answer your question, I presume that your BAM is generated from a Whole Genome library preparation. Calling mutations on a Whole Genome takes a longer time than a Whole Exome since you are looking at every position instead of the exons of your genome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you are looking for is a special format of files called &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Variant Caller Format (VCF)&lt;/a&gt; generated by Variant Callers tools listed below in comparison studies. There isn't a tool better than another, they all have their advantages and disadvantages. I suggest that you read about them as they also describe their pipeline approaches.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.nature.com/articles/s41598-018-36177-7.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Comparison of three variant callers for human whole genome sequencing&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/track/pdf/10.1186/s12859-018-2440-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;Comparing the performance of selected variant callers using synthetic data and genome segmentation&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/track/pdf/10.1186/s12859-019-2928-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;Performance assessment of variant calling pipelines using human whole exome sequencing and simulated data&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.nature.com/articles/s41598-019-45835-3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Systematic comparison of germline variant calling pipelines cross multiple next-generation sequencers&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3785481/pdf/pone.0075619.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Variant Callers for Next-Generation Sequencing Data: A Comparison Study&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Once the VCF is generated, you will also have a column named &lt;code&gt;AF&lt;/code&gt; which corresponds to the allele frequency also computed when using these tools. There are tools that allow you to filter VCF files to extract the needed information with a wished criteria like &lt;a href=&quot;http://snpeff.sourceforge.net/SnpSift.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SnpSift&lt;/a&gt;,  &lt;a href=&quot;https://vcftools.github.io/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;vcftools&lt;/a&gt;, &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360037434691-VariantFiltration&quot; rel=&quot;nofollow noreferrer&quot;&gt;VariantFiltration&lt;/a&gt;. So, I suggest reading the above articles as a start.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As for building pipelines, it depends on what language you feel at ease at. However, if I could suggest, &lt;a href=&quot;https://www.nextflow.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nextflow&lt;/a&gt; is a good start. You can also use &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Snakemake&lt;/a&gt; if you like Python. Or maybe simply &lt;code&gt;bash&lt;/code&gt; ? In any case, there are many pipelines already made to help you through your needs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(If this helps, don't forget to either accept the answer, vote the answer or both).&lt;/p&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2020-04-09T08:19:25.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12890" PostTypeId="1" CreationDate="2020-04-09T08:56:29.457" Score="1" ViewCount="61" Body="&lt;p&gt;I want to do some HLA typing, most of the tools require the bam file is aligned to primary genome without alternative read handling. From the International Genome Sample Resource project, I can get the high coverage file, but they are all aligned to primary genome with hla, decoy and alternative reads. I want to make the alternative reads into unmapped reads.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think about unmapping the whole file to fastq, and then realigning according to different assembly, but it's time-consuming and also costs a lot of storage, so I don't want to follow that approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried &lt;a href=&quot;https://github.com/humanlongevity/HLA/blob/master/bin/get-reads-alt-unmap.sh&quot; rel=&quot;nofollow noreferrer&quot;&gt;this script&lt;/a&gt;, but so far it's not working (I have all requirement packages installed).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help, tool or insight for this problem is warmly welcome. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="4134" LastEditorUserId="57" LastEditDate="2020-04-09T10:53:18.310" LastActivityDate="2020-04-09T10:53:18.310" Title="Unmapping Alternative reads in BAM file" Tags="|sequence-alignment|phylogenetics|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12891" PostTypeId="2" ParentId="12876" CreationDate="2020-04-09T14:32:05.217" Score="2" Body="&lt;p&gt;I think it is very hard to say which are the closest because they are not really comparable. But since you are using Spearman correlation, I guess RPKM, FPKM, and TPM do not change the order of gene expression levels. You might also want to normalize RNA-seq and microarray data so that they are more comparable.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-04-09T14:32:05.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12892" PostTypeId="1" CreationDate="2020-04-09T15:10:55.903" Score="6" ViewCount="1092" Body="&lt;p&gt;I hope you'll indulge a question from a computer scientist with limited bioinformatics knowledge.   I've been working with the Google tool for language modeling called BERT.   It's generally regarded as state of the art when encoding language.   It allows you to say that the sentences &quot;I went to the bank to cash my check.&quot; and &quot;My wife is going to the Credit Union to pay the mortgage&quot; are more similar than if compared to &quot;he was fishing and fell off the bank.&quot;   In short, BERT takes sentences and encodes them so that related &lt;strong&gt;concepts&lt;/strong&gt; are &lt;em&gt;near&lt;/em&gt; each other and unrelated concepts are &lt;em&gt;farther&lt;/em&gt; apart.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;BERT is an amazing tool and I wondered if it could successfully model the language of the genome if it were encoded into &quot;sentences&quot; (I chose 7-mer words and sentences of length 64, arbitrarily).  Below is the result of encoding the full genomes of human, bonobo, gorilla, chimp, and orangutan.  There are about 270K &quot;sentences&quot; here.   You can see there are four major blobs, with indications of small tight clusters.   BERT has decided that the points indicated in red are somehow semantically related.  I think is is significant, or at the very least interesting.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/O6yiC.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/O6yiC.png&quot; alt=&quot;BERT UMAP visualization&quot;&gt;&lt;/a&gt;  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My first thought was that clustering was trivial degenerate cases.  Maybe all the &quot;AAAAAAA...&quot; were put near each other, and all the &quot;TTTTTTTTT....&quot; etc.  I printed out samples from the clusters and visually that is not the case.  Also I've made histograms of the average Levenshtein distance in groups.  Within a tight group, the LD is statistically much lower (but not trivially low) as compared to unclustered regions (the big blob is all unclustered).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iM4tG.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iM4tG.png&quot; alt=&quot;Two histograms of Levenshtein distance&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If there is a biological basis for the clustering, then new sequences could be similarly transformed, and classified according to cluster membership (and related biological function).  This exhausts my biological knowledge.  I've taken sequences from clusters and used BLAST to try and get some insight but I'm &lt;strong&gt;way&lt;/strong&gt; out of my comfort zone with that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My primary question is, given a set of sequences (in my case len=448), how do you figure out what they all have in common (given my hypothesis is that BERT says they are related based on proximity)?    And my backup question is, do you think this is interesting or a waste of time?&lt;/p&gt;&#xA;" OwnerUserId="8590" LastEditorUserId="8590" LastEditDate="2020-04-09T15:39:48.843" LastActivityDate="2020-04-30T22:47:07.113" Title="BERT Language Model and Gene Sequences - How Do I Relate Clusters of Sequences?" Tags="|genome|blast|machine-learning|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12893" PostTypeId="2" ParentId="12874" CreationDate="2020-04-09T21:06:32.623" Score="1" Body="&lt;p&gt;I second commenter Devon Ryan's suggestion, I'm just going to give a little more detail about how to do that. You can plug the BED files into the UCSC browser following these instructions: &lt;a href=&quot;https://genome.ucsc.edu/goldenPath/help/customTrack.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genome.ucsc.edu/goldenPath/help/customTrack.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you can look at the overlap between your bed tracks (black boxes I believe) and conservation (red box).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Fi3oz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Fi3oz.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that is all you want to do, that should do it. If you want to do further data analysis or processing, you can download the conservation track as a &lt;code&gt;.wib&lt;/code&gt; file from UCSC. You can convert this file to BED as described &lt;a href=&quot;http://genomewiki.cse.ucsc.edu/index.php/Using_hgWiggle_without_a_database&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, and use &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDTools&lt;/a&gt; to analyze it together with your BED of regions of interest.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2020-04-20T16:50:12.580" LastActivityDate="2020-04-20T16:50:12.580" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12894" PostTypeId="2" ParentId="12848" CreationDate="2020-04-09T21:55:55.893" Score="1" Body="&lt;p&gt;@Michael's comment is correct but possibly a little difficult to understand, so I'm going to rephrase it here with an example (please let me know if I'm misrepresenting): &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This matrix (pretending for a moment that we care only about purine/pyrimidine transitions and thus only two states) is directly interpretable in terms of the number of transitions per unit time:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ Q = \begin{matrix}&#xA;0 &amp;amp; 1.5 \\&#xA;0.5 &amp;amp; 0 \\&#xA;\end{matrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Whereas this matrix is harder to think about because it is talking about the number of transitions per (I think?) &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{0.37}$&lt;/span&gt; units of time:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ Q = \begin{matrix}&#xA;0 &amp;amp; 0.37 \\&#xA;0.37 &amp;amp; 0 \\&#xA;\end{matrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You will have to rescale any result anyways to talk about it in terms of &quot;per unit time&quot;, so you might as well do it in the matrix itself so that at each point we are dealing with a more or less intuitively comprehensible numbers, given that the scaling doesn't actually change anything because time is dimensionless as you point out. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, if you have a time-calibrated phylogeny then time is not dimensionless, so it might make sense to use a different scaling, such that the average rate in &lt;span class=&quot;math-container&quot;&gt;$Q$&lt;/span&gt; is instead &quot;per year&quot; or &quot;per million years&quot; or whatever works within the allowances of floating-point precision and brains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it's not that rescaling has any particular effect on the math, but rather that rescaling makes it easier for human brains to think about, because we like to represent time in units.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-04-09T21:55:55.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12897" PostTypeId="2" ParentId="12868" CreationDate="2020-04-10T16:11:50.047" Score="1" Body="&lt;p&gt;I wrote a package called sinto (&lt;a href=&quot;https://github.com/timoast/sinto&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/timoast/sinto&lt;/a&gt;) that contains a function to split a BAM file into different files based on the cell barcodes.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://timoast.github.io/sinto/installation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;installation&lt;/a&gt; can be done via anaconda:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda create -p conda/envs/sinto python=3.8 sinto&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And a &lt;a href=&quot;https://timoast.github.io/sinto/basic_usage.html#filter-cell-barcodes-from-bam-file&quot; rel=&quot;nofollow noreferrer&quot;&gt;usage example from the documentation&lt;/a&gt; is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sinto filterbarcodes -b my_bam.bam -c my_barcodes.txt --barcodetag &amp;quot;CB&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Where the &lt;code&gt;my_barcodes.txt&lt;/code&gt; file contains cluster names in a tab separated list format:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TGGCAATGTTGAAGCG-1  A&#xA;GACCAATCACCATTCC-1  A&#xA;CAGGATTCAGAACTTC-1  B&#xA;GAACCTAAGAGAGGTA-1  B,A&#xA;..&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3415" LastEditorUserId="3575" LastEditDate="2022-08-09T22:45:04.413" LastActivityDate="2022-08-09T22:45:04.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12898" PostTypeId="1" CreationDate="2020-04-11T04:39:11.873" Score="0" ViewCount="105" Body="&lt;p&gt;This question comes up in my bioinfo class.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a csv file containing gene names and the cluster they belong to. I want to run GESA on each individual cluster. So I would like to extract the list of gene names corresponding to each cluster. So I wrote code like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;gene_list = pd.read_csv('marker_genes.csv')&#xA;for i in range(12):&#xA;    &quot;cluster_&quot;+str(i) = gene_list.gene[gene_list.cluster==i]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I know this raises syntax error, so what is the correct way of doing it. &#xA;In the end, I would want &lt;code&gt;cluster_0&lt;/code&gt; to &lt;code&gt;cluster_12&lt;/code&gt; each being a list of gene names.&lt;/p&gt;&#xA;" OwnerUserId="3345" LastEditorUserId="57" LastEditDate="2020-04-12T11:55:53.650" LastActivityDate="2020-04-12T11:55:53.650" Title="how do i change a variable with iterator under a for loop in python" Tags="|python|pandas|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12899" PostTypeId="1" AcceptedAnswerId="13061" CreationDate="2020-04-11T07:24:06.993" Score="1" ViewCount="252" Body="&lt;p&gt;I have used DOCK software to dock 10 ligands into the protein structure.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to generate a pdb file that contains all 10 binding modes of the ligands together with the protein. So all together in one pdb file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now, what I want to know is when I give this file to LIGPLOT, is it possible to generate something like:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;%polar interactions XX&#xA;%non-polar interactions XX&#xA;%hydrogen interactions XX&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to do it with LIGPLOT, if now is there some other software that can be recommended for this type of assessment?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been having hard time, trying to find a way or software that can perform similar type of calculation and give me similar type of output that I want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best&lt;/p&gt;&#xA;" OwnerUserId="8598" LastActivityDate="2020-04-24T08:25:23.957" Title="Visualizing protein-ligand interactions with LIGPLOT or any other suitable software?" Tags="|proteins|protein-structure|docking|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12901" PostTypeId="1" AcceptedAnswerId="12903" CreationDate="2020-04-11T12:59:49.790" Score="2" ViewCount="1370" Body="&lt;p&gt;I would like to download RNAseq gene expression data from GTEx from the liver only. What is the best way to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried a few things. I downloaded several files from the GTEx website (&lt;a href=&quot;https://www.gtexportal.org/home/datasets&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.gtexportal.org/home/datasets&lt;/a&gt;, under &quot;RNA-Seq Data&quot;), but the problems I run into are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Not sure what is the best file: &quot;Gene read counts&quot; or &quot;Gene TPMs&quot;? What is the difference?&lt;/li&gt;&#xA;&lt;li&gt;I would like 1 value per sample per gene. There are many duplicate genes in these 2 files. How can I collapse them to 1 value per gene?&lt;/li&gt;&#xA;&lt;li&gt;How can I download the data for liver tissue only, instead of ALL data for ALL tissues?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Thank you for suggestions.&lt;/p&gt;&#xA;" OwnerUserId="7354" LastActivityDate="2020-04-11T15:15:59.487" Title="How to download RNAseq gene expression data from GTEx" Tags="|rna-seq|public-databases|gene-expression|public-dataset|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12902" PostTypeId="2" ParentId="10782" CreationDate="2020-04-11T15:13:37.313" Score="0" Body="&lt;p&gt;It appears the original observations were not a timeseries and the OP is specifically asking about application to timeseries data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In my experience these are two separate areas of analysis because a temporal skew may result in aggregate behaviour which violates the original transformation. For example hot weather versus cold weather. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What the OP appears to be doing is normalising each point in the time series, albeit I don't know what X represents. Personally I have doubts because the aggregate correlation is across all time points and one set of aggregrates, e.g. Ecoregion 1 is compared with another e.g. Ecoregion 2. This for example would be done via linear modelling and assessement of fit performed through many approaches, e.g. the residual is just one approach.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If data has large temporal skews your approach is unlikely to work, ultimately you don't know the variation in the time series which could far more heterogeneous than the agrregrate data. Seasonality is likely a major factor in your data and I would consider this in the first instance. There are lots of ways to transform a time series, standardisation is a classic approach. Again my personal experience is these are two separate areas of statistics and directly applying one to the other is something that should be assessed with care.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;So just to try and explain in this scenario terms, when its cold if the variation in bacterioplankton is much greater than when its hot (smaller population sizes increases variance), i.e. at different times of year then your approach can't work because normalisation is about a fixed (constant) relationship between mean and standard deviation.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-04-11T22:37:28.397" LastActivityDate="2020-04-11T22:37:28.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12903" PostTypeId="2" ParentId="12901" CreationDate="2020-04-11T15:15:59.487" Score="1" Body="&lt;ul&gt;&#xA;&lt;li&gt;Gene read counts are the number of reads or fragments that are mapped to each gene. So it is an integer and the values are not directly comparable between samples due to differences in sequencing depth. TPM stands transcripts per million. It is the normalized gene expression level. Basically, you first normalize read counts by gene length and then normalize the library size to 1e6 in all libraries. As for which one should you use, it depends on what analysis you want to do and what tool you are going to use.&lt;/li&gt;&#xA;&lt;li&gt;I have not noticed that there are duplicated genes in those files, can you show an example?&lt;/li&gt;&#xA;&lt;li&gt;I do not think you can download expression from one tissue, but you can always subset. The whole dataset is not very large.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-04-11T15:15:59.487" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12905" PostTypeId="1" CreationDate="2020-04-11T18:16:16.033" Score="5" ViewCount="388" Body="&lt;p&gt;I would be very grateful if somebody could sketch out the methods &lt;a href=&quot;http://pfam.xfam.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pfam&lt;/a&gt; and &lt;a href=&quot;http://www.pantherdb.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;PANTHER&lt;/a&gt; use to assign a family to a given protein and how they are different. My (cursory) understanding is that InterpProp pools databases like Pfam and PANTHER and provides a meta-classification based on some combination of their data.&lt;/p&gt;&#xA;&lt;p&gt;In particular, I wish to understand where the discrepancy comes from described in the picture below?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/M4a4S.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/M4a4S.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm fairly new to the ecosystem, so feel free to throw any bone this way.&lt;/p&gt;&#xA;" OwnerUserId="6511" LastEditorUserId="3848" LastEditDate="2022-03-24T14:57:54.660" LastActivityDate="2022-03-24T14:57:54.660" Title="What exactly does each of InterPro, PANTHER, Pfam bring to the table individually in classifying a protein?" Tags="|database|proteins|public-databases|hidden-markov-models|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12906" PostTypeId="2" ParentId="12898" CreationDate="2020-04-11T20:52:56.490" Score="1" Body="&lt;p&gt;cluster_0 to cluster_12 are just strings. they cannot be the names of arrays or matrices. Strings can only be used in print statements and string processing statements.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The correct way to define an array or matrix is as follows: call the arrays as: cluster[0] to cluster[12]&lt;/p&gt;&#xA;&#xA;&lt;p&gt;then, assign:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  for i in range(12):&#xA;        cluster[i] = gene_list.gene[gene_list.cluster==i]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;then whenever you need to print the names of these arrays, you use:  &quot;cluster_&quot;+str(i)&lt;/p&gt;&#xA;" OwnerUserId="8573" LastEditorUserId="8573" LastEditDate="2020-04-11T21:22:04.270" LastActivityDate="2020-04-11T21:22:04.270" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12907" PostTypeId="1" AcceptedAnswerId="12908" CreationDate="2020-04-11T23:05:53.133" Score="1" ViewCount="147" Body="&lt;p&gt;I am looking for guidance to load all SwissProt PPI data for Homo sapiens into a graph (from NetworkX). In particular, the part that I'm struggling with is gathering the data from SwissProt for each human protein. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;The data that I'm looking to extract is protein-protein interaction. This is not included in the files found &lt;a href=&quot;ftp://ftp.uniprot.org/pub/databases/uniprot/current_release/knowledgebase/idmapping/by_organism/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, which @rtviii (answer below) suggested to use. Rather, when say you click on an entry in SwissProt, for each entry, there is a section devoted to &quot;Interaction&quot; as shown in the image attached below. I would like to extract that information for each human protein, and load it into a graph data structure using NetworkX&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VKTPS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VKTPS.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;EDIT 2:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How I ended up doing it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;    query = &quot;reviewed:yes+AND+organism:9606&quot;&#xA;    args = {'query': query, 'format': 'xls', 'columns':'id,entry name,interactor'}&#xA;&#xA;    uniprot_url = &quot;https://www.uniprot.org/uniprot/?{}&quot;.format(urllib.parse.urlencode(args))&#xA;&#xA;    with open(&amp;lt;DESIRED FILE PATH&amp;gt;, &quot;wb&quot;) as target:&#xA;          target.write(requests.get(url).content)&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="3848" LastEditDate="2020-04-15T17:49:46.343" LastActivityDate="2020-04-16T02:56:54.693" Title="Loading all PPI information from Uniprot into a graph in NetworkX" Tags="|python|uniprot|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12908" PostTypeId="2" ParentId="12907" CreationDate="2020-04-11T23:36:33.343" Score="2" Body="&lt;p&gt;The query link for all interactors would be thus : &lt;code&gt;https://www.uniprot.org/uniprot/?query=organism%3A9606+AND+interactor%3A*&amp;amp;sort=score&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;There is always an option of downloading the static version of the service if you find it incovnient to deal with their API for your use case &lt;a href=&quot;ftp://ftp.uniprot.org/pub/databases/uniprot/&quot; rel=&quot;nofollow noreferrer&quot;&gt;from here&lt;/a&gt; and have it all to yourself, pipe data into your dataframes locally. I find it useful to just broswe their repositories sometimes instead of reading the specifications.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, you can get a variety of datashapes  from uniprot that you cannot get from their static database-image by specifying the appropriate parameters in the request. You can browse and explore parameters by expanding their GUI's &lt;strong&gt;search fields&lt;/strong&gt; as described in &lt;a href=&quot;http://insideuniprot.blogspot.com/2015_03_01_archive.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this little guide&lt;/a&gt;. Then you can request your data like so, in python for example. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;URL = &quot;https://www.uniprot.org/uniprot/&quot;&#xA;PARAMS = {'organism': 9606, 'format': 'tab', ....#Additional parameters#...}&#xA;response = requests.get(url=URL, params=PARAMS)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6511" LastEditorUserId="6511" LastEditDate="2020-04-16T02:56:54.693" LastActivityDate="2020-04-16T02:56:54.693" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12909" PostTypeId="1" CreationDate="2020-04-12T08:13:41.370" Score="1" ViewCount="144" Body="&lt;p&gt;First of all, I'd like to apologize for any spelling or grammar mistakes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm having a problem using R for a classification problem. My dataset contains ~300.000 genomic data, and the features are DNA-related features (number of dinucleotides, number of trinucleotides, the CG Content, and some more). In conclusion, I have a dataset of 300.000 rows and 84 columns (columns= features). The 84th feature is basically the classification variable (there are two classes: class 1 and class 2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I split the dataset in train set and test set and fit the train set to create a model using different methods, like Random Forest, XGBoost, Adaboost, etc. and check the accuracy using the test set.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In every model, the accuracy of the prediction on the test set is always around 86-87%. What I noticed is that, in the test set, for EVERY MODEL, almost the same number of rows (data) is predicted right (the model predict its true class), and always almost the same data of rows (data) is predicted wrong (the model predict wrong class. For example its true class is 1 but the model predicts 2).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I take the set of data that are predicted right for every model I've created (250.000 data from the 300.000), and build a new model out of them (splitting them again in a new train and test set), I have an accuracy of 98%.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That means that the other 50.000 data (from the 300.000), add an error at every model I'm trying to create.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But I can't just ignore them. I thought that it would be a way to create two differents models, but doesn't know how to figure this out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then I thought that there could be a way to find which variables from the two subsets have strong differences, but even if I figure this out, I don't know what's the way for example to scale the original dataset to prevent those differences from keeping the accuracy in 87% in every model (using the original dataset of the 300.000 rows).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In conclusion, I have a dataset that pretends like &quot;two datasets&quot;. It consists of two subsets: the first one is always predicted right from any model (machine learning algorithm) I'm trying, and the second one is always predicted wrong. How could I build a strong model taking into account the differences?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also took a look at the box plots of the features. The most important feature for every model is called &quot;CG&quot; (number of dinucleotide of cytosines and guanines in DNA). In the following image you can see the boxplot of CG feature separating the original dataset in two datasets (T = for the dataset that is predicted right in every model) and (F = for the dataset that is predicted wrong in every model). 1 stands for data of class 1 and 2 stands for data of class 2. So, T1 means data from the right predicted dataset of class 1, T2 means data from the right predicted dataset of classes 2, T means data from the right predicted dataset for both class 1 and 2, etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZmeEN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZmeEN.png&quot; alt=&quot;box plot CG&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As I'm a new beginner in R and in machine learning, I hope you do understand my problem. Thanks sincerely.&lt;/p&gt;&#xA;" OwnerUserId="8600" LastActivityDate="2023-12-23T16:03:15.360" Title="Problem with classification model of genomic data: every machine learing model predicts wrongly almost always the same subset of dataset" Tags="|r|genomics|phylogenetics|machine-learning|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="12910" PostTypeId="2" ParentId="12876" CreationDate="2020-04-12T08:17:03.600" Score="2" Body="&lt;p&gt;I did a comparison of cDNA count data against microarray data that was &lt;a href=&quot;https://doi.org/10.1084/jem.20160470&quot; rel=&quot;nofollow noreferrer&quot;&gt;published&lt;/a&gt; a few years ago:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;For comparisons to published data (Fig. S2; Miller et al., 2012), a generalized linear model was fitted to the relationship between log-transformed microarray and VSTPk expression levels obtained from the ImmGen Project database, and was used to transform the microarray data into values comparable to our VSTPks.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I found that the Variance-Stabilizing Transformation that was carried out by &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#variance-stabilizing-transformation&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2&lt;/a&gt; was close to what I wanted, but there still seemed to be a length-based bias to the reads. I corrected this by dividing by the length of the longest gene isoform in kilobases (creating something that I called &lt;em&gt;VSTPk&lt;/em&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After doing this, there were range differences between the microarray and cDNA data, so I did an additional linear transformation to get the data fitting as close as possible.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2020-04-12T08:17:03.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12911" PostTypeId="2" ParentId="12909" CreationDate="2020-04-12T08:24:51.463" Score="0" Body="&lt;p&gt;You could be seeing the effect of SNPs on haplotype-driven conditions (i.e. where a combination of multiple variants predicts the class, rather than a single variant). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you have any method to determine the age of particular variants (e.g. via an LD plot to find inconsistencies between variants and the surrounding block structure), so that you can see if that correlates with the likelihood of error?&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2020-04-12T08:24:51.463" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12912" PostTypeId="2" ParentId="10791" CreationDate="2020-04-12T09:17:12.760" Score="1" Body="&lt;p&gt;This is close to what many people do. May I add:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;between 4 and 5 you might want to add read groups using Picard. This will allow GATK to work. For this purpose (in case picard gives you issues), some time ago I made &lt;a href=&quot;https://github.com/MatteoSchiavinato/Utilities/blob/master/add-read-groups.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;filtering with BCFtools is somewhat uncomfortable (for me at least). I would suggest you to plot the distribution of every variable in the INFO and GENOTYPE fields of the VCF file and understand their distribution and meaning. It could vary a lot and there is no one-size-fits-all threshold.&#xA;For that purpose, some time ago I made &lt;a href=&quot;https://github.com/MatteoSchiavinato/Utilities/blob/master/SNP-dogg&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://github.com/MatteoSchiavinato/Utilities/blob/master/SNP-dogg-v2.0&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3378" LastActivityDate="2020-04-12T09:17:12.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12913" PostTypeId="1" CreationDate="2020-04-12T19:24:06.600" Score="1" ViewCount="63" Body="&lt;p&gt;I don't have any theoretical background in biology, so forgive me if my question is a bit.. well..dumb. I'm trying to use a Monte Carlo approach to find the discriminating position of a given sequence alignment column. So essentially, given a sequence alignment, I want to find the position at which we can distinguish active and inactive conformations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, here is my code for the Monte Carlo method:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def MonteCarlo(S, max_iter, threshold):&#xA;    &quot;&quot;&quot;&#xA;    Computes the discriminating position for a sequence column&#xA;&#xA;    Input: S = sequence column, max_iter = max iterations, threshold = threshold of acceptance for boltzmann distribution (between 0 and 1)&#xA;    Output: 2 classes&#xA;    &quot;&quot;&quot;&#xA;    n = len(S)&#xA;    S_copy = S.copy()&#xA;    num_class1 = int(0.30*n) #number in class1&#xA;    print(num_class1)&#xA;    for _ in range(num_class1): #intialisation&#xA;        rand_position = random.randint(0,len(S)-1)&#xA;        if S_copy[rand_position] == 'X':&#xA;            rand_position = random.randint(0,len(S)) #reselect if used position&#xA;        S_copy[rand_position] = 'X' &#xA;&#xA;    #Monte Carlo with Insertions and Deletions&#xA;&#xA;    s_entropy = entropy(S)&#xA;    n_iter = 0 #counter for iterations&#xA;    class_1 = dict() #class 1 determined by X's&#xA;    class_2 = dict() #class 2 is rest of S&#xA;    for i in range(len(S)):&#xA;        if S_copy[i] == 'X':&#xA;            class_1[i] =S[i]&#xA;        else:&#xA;            class_2[i] =S[i]&#xA;&#xA;    res = score(S, class_1, class_2)&#xA;&#xA;    while n_iter &amp;lt; max_iter:&#xA;        r = random.random()&#xA;&#xA;        if r &amp;lt; 0.25: #25% chance to do an insertion&#xA;          (c1,c2,pos) = Insertion(S, class_1, class_2)&#xA;&#xA;        elif r &amp;lt; 0.5: #25% chance to do a deletion&#xA;          (c1,c2,pos) = Deletion(S, class_1, class_2)&#xA;&#xA;        else: # 50% chance to do a swap&#xA;          (c1,c2,pos) = Swap(S, class_1, class_2)&#xA;&#xA;        res2 = score(S, c1, c2)&#xA;&#xA;        #Boltzmann Distribution coefficient &#xA;        delta_S = abs(res2 - res)&#xA;        lambda_, N = 0.2, 19&#xA;        size = 6000&#xA;        ind1 = random.randint(0,size - 1)&#xA;        rv = boltzmann.rvs(lambda_, N, size=size)/N &#xA;        beta = rv[ind1] #pick float between 0 and 1 with boltzmann distribution&#xA;        if  (res2 &amp;gt; res) or (beta*delta_S &amp;gt; threshold): #if better score i.e. higher entropy gain&#xA;            res = res2&#xA;            class_1 = c1&#xA;            class_2 = c2&#xA;&#xA;        n_iter += 1&#xA;&#xA;&#xA;    return class_1, class_2, pos&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here I'm using the Boltzmann distribution as a probability measure. All the other functions (swap, insertion, deletion, entropy) calculate pretty much what the name says (if you need more clarification on my side, I'd be happy to provide it!)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me explain what my code is trying to do: Given a sequence alignment (as a list) say ['M','M','M',...,'L','L','L'] I first set a bunch of random positions and mark them out as 'X' (the number of Xs are determined by num_class1). Next, in the while loop I do swaps,insertions, and deletions with the goal of obtaining a better score. This meansI'm adding, removing, or swapping positions between class1 and class2. Eventually, my code returns two classes and the discriminating position.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While my code works, I don't think it's giving the right result. When I use a base case example of a list with two letters 'M' and 'L with a clear discriminating position, I get random results. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone explain theoretically (or via code) how one can find the discriminating position using a Monte Carlo method as above?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="8603" LastActivityDate="2020-09-10T00:01:40.187" Title="Finding discriminating positions for a sequence alignment column" Tags="|python|sequence-alignment|proteins|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12914" PostTypeId="2" ParentId="12913" CreationDate="2020-04-12T22:41:14.647" Score="1" Body="&lt;p&gt;Without asking further questions, my suspicion would be how indels (gaps) are counted in your code against the aligment. Its the classic place for alignment mis-position errors. In other words, based on my experience your calculation probably performs fine, but linking the calculated alignment position against the actual observed alignment position is classic area of logical errors and needs careful monitoring of variables using code breakpoints within your loops.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-04-12T22:41:14.647" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12917" PostTypeId="1" CreationDate="2020-04-13T18:42:24.073" Score="0" ViewCount="118" Body="&lt;p&gt;I am interested in 20 different SNPs that all are either As or Gs, and they all occur on the same chromosome. How can I assess the co-occurrence of these SNPs? In other words, I want to know, if SNP1 is &quot;G&quot;, does that make SNP2 to be more likely to be a &quot;G&quot; as well?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I know this is broad and I'm not looking for a script or anything so specific, just a point in the right direction. I'm working with a non-model organism, so any organism-specific application won't be useful without being able to learn the methodology.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Googling thus far has only brought up GWAS and similar methods of correlating a genotype and a phenotype, but not within a genotype.&lt;/p&gt;&#xA;" OwnerUserId="2727" LastEditorUserId="3541" LastEditDate="2020-04-13T19:17:11.977" LastActivityDate="2020-04-15T14:06:32.783" Title="How to analyze co-occurrence of multiple SNPs?" Tags="|snp|gwas|correlation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12918" PostTypeId="1" AcceptedAnswerId="13208" CreationDate="2020-04-13T20:52:06.170" Score="3" ViewCount="6288" Body="&lt;p&gt;I am new to R and computational biology. I am trying to look through a published data set to check for gene expression for my own project. I am having trouble finding materials to teach me how to analyze single cell RNA sequencing data on R from the GEO NCBI database.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the data that I am trying to access &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE138651&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE138651&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would appreciate any directions or suggestions for tutorials.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="8610" LastActivityDate="2020-05-07T12:44:55.373" Title="How do I pull singe cell RNA sequencing data from GEO database?" Tags="|r|scrnaseq|phylogenetics|geoquery|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12919" PostTypeId="1" CreationDate="2020-04-13T21:05:57.993" Score="-2" ViewCount="128" Body="&lt;p&gt;I wish to reproduce a very informative plot in google &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qQ9IF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qQ9IF.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given a group of samples and a list of genes for which we have data how each genes has mutated and whether the alleles are homozygous of heterozygous. These are expressed as boolean values&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Genes enriched in 2 pathways like cell cycle and p53 as shown in this figure&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Map of functional alterations for a group of patients. Genes (rows) encoding components p53–DNA repair pathway;  are affected by selected functional events (percent of samples altered and types of alteration are represented by colored squares) across group of samples in column. Alterations of the pathway are observed stacked green bar plots at bottom&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have the boolean matrix of mutated genes per sample would be grateful for assistence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example of my copy number data&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dataset_gistic$annotations&#xA;     type                event     &#xA;G2   &quot;Homozygous Deletion&quot;   &quot;ACVR2A&quot;  &#xA;G13  &quot;Homozygous Deletion&quot;   &quot;HIST1H3B&quot;&#xA;G73  &quot;Heterozygous Deletion&quot; &quot;CCNE1&quot;   &#xA;G74  &quot;Heterozygous Deletion&quot; &quot;TSHZ3&quot;   &#xA;G78  &quot;Low-level Amplification&quot;    &quot;CTNNB1&quot;&#xA;G110 &quot;Low-level Amplification&quot;    &quot;CCNE1&quot;   &#xA;G111 &quot;Low-level Amplification&quot;    &quot;TSHZ3&quot;   &#xA;G119 &quot;High-level Amplification&quot;   &quot;PIK3CA&quot;  &#xA;G121 &quot;High-level Amplification&quot;   &quot;NIPBL&quot;     &#xA;G147 &quot;High-level Amplification&quot;   &quot;CCNE1&quot;   &#xA;G148 &quot;High-level Amplification&quot;   &quot;TSHZ3&quot;   &#xA;&amp;gt;&#xA;&#xA;&amp;gt; dput(dataset_gistic$annotations)&#xA;structure(c(&quot;Homozygous Loss&quot;, &quot;Homozygous Loss&quot;, &quot;Homozygous Loss&quot;, &#xA;&quot;Homozygous Loss&quot;, &quot;Homozygous Loss&quot;, &quot;Homozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &#xA;&quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &#xA;&quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &#xA;&quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &#xA;&quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &#xA;&quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &#xA;&quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &#xA;&quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &#xA;&quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &#xA;&quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &#xA;&quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &#xA;&quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &quot;Heterozygous Loss&quot;, &#xA;&quot;Heterozygous Loss&quot;, &quot;Low-level Gain&quot;, &quot;Low-level Gain&quot;, &quot;Low-level Gain&quot;, &#xA;&quot;Low-level Gain&quot;, &quot;Low-level Gain&quot;, &quot;Low-level Gain&quot;, &quot;Low-level Gain&quot;, &#xA;&quot;Low-level Gain&quot;, &quot;Low-level Gain&quot;, &quot;Low-level Gain&quot;, &quot;Low-level Gain&quot;, &#xA;&quot;Low-level Gain&quot;, &quot;Low-level Gain&quot;, &quot;Low-level Gain&quot;, &quot;Low-level Gain&quot;, &#xA;&quot;Low-level Gain&quot;, &quot;Low-level Gain&quot;, &quot;Low-level Gain&quot;, &quot;Low-level Gain&quot;, &#xA;&quot;Low-level Gain&quot;, &quot;Low-level Gain&quot;, &quot;Low-level Gain&quot;, &quot;Low-level Gain&quot;, &#xA;&quot;Low-level Gain&quot;, &quot;High-level Gain&quot;, &quot;High-level Gain&quot;, &quot;High-level Gain&quot;, &#xA;&quot;High-level Gain&quot;, &quot;High-level Gain&quot;, &quot;High-level Gain&quot;, &quot;High-level Gain&quot;, &#xA;&quot;High-level Gain&quot;, &quot;High-level Gain&quot;, &quot;High-level Gain&quot;, &quot;High-level Gain&quot;, &#xA;&quot;High-level Gain&quot;, &quot;ACVR2A&quot;, &quot;HIST1H3B&quot;, &quot;CDKN2A&quot;, &quot;PTEN&quot;, &quot;NAV3&quot;, &#xA;&quot;LIN7A&quot;, &quot;ARID1A&quot;, &quot;ACVR2A&quot;, &quot;SCN3A&quot;, &quot;CTNNB1&quot;, &quot;PBRM1&quot;, &quot;EPHA3&quot;, &#xA;&quot;POLQ&quot;, &quot;PIK3CA&quot;, &quot;SLIT2&quot;, &quot;NIPBL&quot;, &quot;MSH3&quot;, &quot;APC&quot;, &quot;HIST1H3B&quot;, &#xA;&quot;ARID1B&quot;, &quot;CDKN2A&quot;, &quot;NOTCH1&quot;, &quot;PTEN&quot;, &quot;CCND1&quot;, &quot;KRAS&quot;, &quot;LRRK2&quot;, &#xA;&quot;ARID2&quot;, &quot;NAV3&quot;, &quot;LIN7A&quot;, &quot;SIN3A&quot;, &quot;TP53&quot;, &quot;ERBB2&quot;, &quot;GPATCH8&quot;, &#xA;&quot;RNF43&quot;, &quot;GATA6&quot;, &quot;SMAD4&quot;, &quot;CCDC102B&quot;, &quot;STK11&quot;, &quot;SMARCA4&quot;, &quot;CCNE1&quot;, &#xA;&quot;TSHZ3&quot;, &quot;CTNNB1&quot;, &quot;PIK3CA&quot;, &quot;SLIT2&quot;, &quot;NIPBL&quot;, &quot;HIST1H3B&quot;, &quot;ARID1B&quot;, &#xA;&quot;EGFR&quot;, &quot;KCNQ3&quot;, &quot;NOTCH1&quot;, &quot;PTEN&quot;, &quot;CCND1&quot;, &quot;KRAS&quot;, &quot;LRRK2&quot;, &#xA;&quot;ARID2&quot;, &quot;SIN3A&quot;, &quot;ERBB2&quot;, &quot;GPATCH8&quot;, &quot;RNF43&quot;, &quot;GATA6&quot;, &quot;CCDC102B&quot;, &#xA;&quot;STK11&quot;, &quot;SMARCA4&quot;, &quot;CCNE1&quot;, &quot;TSHZ3&quot;, &quot;PIK3CA&quot;, &quot;NIPBL&quot;, &quot;EGFR&quot;, &#xA;&quot;KCNQ3&quot;, &quot;CCND1&quot;, &quot;KRAS&quot;, &quot;ERBB2&quot;, &quot;GPATCH8&quot;, &quot;RNF43&quot;, &quot;GATA6&quot;, &#xA;&quot;CCNE1&quot;, &quot;TSHZ3&quot;), .Dim = c(77L, 2L), .Dimnames = list(c(&quot;G2&quot;, &#xA;&quot;G13&quot;, &quot;G17&quot;, &quot;G19&quot;, &quot;G24&quot;, &quot;G25&quot;, &quot;G38&quot;, &quot;G39&quot;, &quot;G40&quot;, &quot;G41&quot;, &#xA;&quot;G42&quot;, &quot;G43&quot;, &quot;G44&quot;, &quot;G45&quot;, &quot;G46&quot;, &quot;G47&quot;, &quot;G48&quot;, &quot;G49&quot;, &quot;G50&quot;, &#xA;&quot;G51&quot;, &quot;G54&quot;, &quot;G55&quot;, &quot;G56&quot;, &quot;G57&quot;, &quot;G58&quot;, &quot;G59&quot;, &quot;G60&quot;, &quot;G61&quot;, &#xA;&quot;G62&quot;, &quot;G63&quot;, &quot;G64&quot;, &quot;G65&quot;, &quot;G66&quot;, &quot;G67&quot;, &quot;G68&quot;, &quot;G69&quot;, &quot;G70&quot;, &#xA;&quot;G71&quot;, &quot;G72&quot;, &quot;G73&quot;, &quot;G74&quot;, &quot;G78&quot;, &quot;G82&quot;, &quot;G83&quot;, &quot;G84&quot;, &quot;G87&quot;, &#xA;&quot;G88&quot;, &quot;G89&quot;, &quot;G90&quot;, &quot;G92&quot;, &quot;G93&quot;, &quot;G94&quot;, &quot;G95&quot;, &quot;G96&quot;, &quot;G97&quot;, &#xA;&quot;G100&quot;, &quot;G102&quot;, &quot;G103&quot;, &quot;G104&quot;, &quot;G105&quot;, &quot;G107&quot;, &quot;G108&quot;, &quot;G109&quot;, &#xA;&quot;G110&quot;, &quot;G111&quot;, &quot;G119&quot;, &quot;G121&quot;, &quot;G126&quot;, &quot;G127&quot;, &quot;G131&quot;, &quot;G132&quot;, &#xA;&quot;G139&quot;, &quot;G140&quot;, &quot;G141&quot;, &quot;G142&quot;, &quot;G147&quot;, &quot;G148&quot;), c(&quot;type&quot;, &quot;event&quot;&#xA;)))&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this is sample of my mutational data the number of mutations for each gene in each sample&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    om$numericMatrix&#xA;       LP6005500-DNA_D03 LP6008336-DNA_G01 LP6008460-DNA_D01 LP6005334-DNA_H01&#xA;TP53                   2                 2                 2                 2&#xA;CDKN2A                 3                 1                 2                 0&#xA;       LP6007600 LP6008334-DNA_A03 LP6008334-DNA_A04 LP6008334-DNA_B02&#xA;TP53           2                 2                 2                 2&#xA;CDKN2A         0                 0                 0                 0&#xA;       LP6008334-DNA_C02 LP6008334-DNA_D02 LP6008336-DNA_H01 LP6008337-DNA_A07&#xA;TP53                   2                 1                 2                 2&#xA;CDKN2A                 0                 0                 0                 0&#xA;       LP6008337-DNA_H06 LP6008460-DNA_A04&#xA;TP53                   2                 2&#xA;CDKN2A                 0                 0&#xA;&amp;gt; &#xA;&#xA;&amp;gt; dput(om$numericMatrix)&#xA;structure(c(2, 3, 2, 1, 2, 2, 2, 0, 2, 0, 2, 0, 2, 0, 2, 0, 2, &#xA;0, 1, 0, 2, 0, 2, 0, 2, 0, 2, 0), .Dim = c(2L, 14L), .Dimnames = list(&#xA;    c(&quot;TP53&quot;, &quot;CDKN2A&quot;), c(&quot;LP6005500-DNA_D03&quot;, &quot;LP6008336-DNA_G01&quot;, &#xA;    &quot;LP6008460-DNA_D01&quot;, &quot;LP6005334-DNA_H01&quot;, &quot;LP6007600&quot;, &quot;LP6008334-DNA_A03&quot;, &#xA;    &quot;LP6008334-DNA_A04&quot;, &quot;LP6008334-DNA_B02&quot;, &quot;LP6008334-DNA_C02&quot;, &#xA;    &quot;LP6008334-DNA_D02&quot;, &quot;LP6008336-DNA_H01&quot;, &quot;LP6008337-DNA_A07&quot;, &#xA;    &quot;LP6008337-DNA_H06&quot;, &quot;LP6008460-DNA_A04&quot;)))&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="3848" LastEditDate="2020-04-15T12:16:03.533" LastActivityDate="2020-04-15T12:16:03.533" Title="Is it possible to get such a plot" Tags="|r|heatmap|figure-reproduction|" AnswerCount="1" CommentCount="3" ClosedDate="2020-04-18T21:29:20.590" ContentLicense="CC BY-SA 4.0" />
  <row Id="12920" PostTypeId="2" ParentId="12918" CreationDate="2020-04-13T21:21:15.620" Score="5" Body="&lt;p&gt;The simplest would be using a count matrix (at the end of the link you have shared, section &quot;Supplementary file&quot;). For example &lt;code&gt;GSE138651_Vagal_WholeNodoseSeq_raw.mtx.gz&lt;/code&gt; corresponds to &quot;raw&quot; count matrices obtained after mapping and counting the reads. This file, along with the &lt;code&gt;GSE138651_barcodes.tsv.gz&lt;/code&gt; and &lt;code&gt;GSE138651_genes.tsv.gz&lt;/code&gt; can be used to &quot;read&quot; the data into &lt;code&gt;R&lt;/code&gt;; to that end there are many options in the form of packages, &lt;a href=&quot;https://satijalab.org/seurat/&quot; rel=&quot;noreferrer&quot;&gt;Seurat&lt;/a&gt; is one of the best documented and hence is beginner-friendly.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-04-13T21:21:15.620" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12921" PostTypeId="1" AcceptedAnswerId="12923" CreationDate="2020-04-14T08:21:46.017" Score="1" ViewCount="477" Body="&lt;p&gt;I am learning bioinformatics and i was solving a problem on &lt;a href=&quot;http://rosalind.info/problems/gc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;rosalind.info&lt;/a&gt;.&#xA;Input is a string consist of multiple DNA strand names and the DNA sequences, and Output is the name of the sequence that has the highest GC percentage and the GC percentage.&#xA;this is the input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Rosalind_6404&#xA;CCTGCGGAAGATCGGCACTAGAATAGCCAGAACCGTTTCTCTGAGGCTTCCGGCCTTCCC&#xA;TCCCACTAATAATTCTGAGG&#xA;&amp;gt;Rosalind_5959&#xA;CCATCGGTAGCGCATCCTTAGTCCAATTAAGTCCCTATCCAGGCGCTCCGCCGAAGGTCT&#xA;ATATCCATTTGTCAGCAGACACGC&#xA;&amp;gt;Rosalind_0808&#xA;CCACCCTCGTGGTATGGCTAGGCATTCAGGAACCGGAGAACGCTTCAGACCAGCCCGGAC&#xA;TGGGAACCTGCGGGCAGTAGGTGGAAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;this is the correct output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Rosalind_0808&#xA;60.919540&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is my code (sorry for this filthy mess):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DNA = input()&#xA;wholeRosalind_DNA = DNA.splitlines()&#xA;Rosalind = []&#xA;Rosalind_DNA = []&#xA;for i in wholeRosalind_DNA:&#xA;    if &quot;Rosalind&quot; in i:&#xA;        Rosalind.append(i)&#xA;Countstring = []&#xA;maxGCcontent = []&#xA;GCcontent = []&#xA;def eachRosalind_DNA(DNA):&#xA;    for i in range(0,len(Rosalind)):&#xA;        if Rosalind[i] in wholeRosalind_DNA:&#xA;            a = wholeRosalind_DNA.index(Rosalind[i])&#xA;            if a == wholeRosalind_DNA.index(Rosalind[-1]):  &#xA;                b = len(wholeRosalind_DNA)+1&#xA;            else:&#xA;                b = wholeRosalind_DNA.index(Rosalind[i+1])&#xA;            Rosalind_DNA.append(wholeRosalind_DNA[a:b])&#xA;    for i in Rosalind_DNA:&#xA;        Countstring.append(''.join(i))&#xA;    for i in Countstring:&#xA;        bases = len(i)-14&#xA;        CG = i.count(&quot;C&quot;) + i.count(&quot;G&quot;)&#xA;        GCcontent.append([i[1:14],(100*(CG/bases))])&#xA;    for i in GCcontent:&#xA;        maxGCcontent.append(i[1])&#xA;    result = max(maxGCcontent)&#xA;    for i in GCcontent:&#xA;        if i[1] == result:&#xA;            return (i[0]),(round(i[1],6))&#xA;print(eachRosalind_DNA(DNA)[0])&#xA;print(eachRosalind_DNA(DNA)[1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but i get this instead:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  File &quot;GC_Content.py&quot;, line 26, in eachRosalind_DNA&#xA;    GCcontent.append([i[1:14],(100*(CG/bases))])&#xA;ZeroDivisionError: division by zero&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;i will get the desired output if i just paste the input into the variabe &quot;DNA&quot; without using user input. Please help me fix the code and distinguish those two ways of getting input&lt;/p&gt;&#xA;" OwnerUserId="8611" LastActivityDate="2020-04-14T14:07:28.923" Title="Output when compute GC content is different between using user input and using string" Tags="|python|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12922" PostTypeId="2" ParentId="12921" CreationDate="2020-04-14T10:49:45.777" Score="0" Body="&lt;p&gt;The error is that &lt;code&gt;bases = 0&lt;/code&gt;. Briefly looking at the code carriage returns in your original code will trigger this, as will confusing a faster header with sequence. The latter is because the fasta headers are 14 characters long and subtracting 14 from bases which is of course zero. The former will likely result in 'nan' because it would be 14 subtracted from ''.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've a simpler solution, download Biopython (using &lt;code&gt;conda&lt;/code&gt; or last resort &lt;code&gt;pip&lt;/code&gt;), you might want to use SeqIO (Biopython), which would be cooler because it allows you to call Rosalind_0808 directly. Then:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.SeqUtils import GC&#xA;seq = 'CCACCCTCGTGGTATGGCTAGGCATTCAGGAACCGGAGAACGCTTCAGACCAGCCCGGACTGGGAACCTGCGGGCAGTAGGTGGAAT'&#xA;print (GC(seq))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Done. If you want to splice off the last 14 characters its &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GC(seq[:-14])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-04-14T10:55:36.003" LastActivityDate="2020-04-14T10:55:36.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12923" PostTypeId="2" ParentId="12921" CreationDate="2020-04-14T10:57:05.947" Score="3" Body="&lt;p&gt;Is it cheating to use Biopython(?), will a solution might be helpful anyway. This assumes you have your sequences stored in the seq.fa FASTA file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;from Bio.SeqUtils import GC&#xA;&#xA;print(max(((r.id, GC(r.seq)) for r in SeqIO.parse(&quot;seq.fa&quot;, &quot;fasta&quot;)),&#xA;          key=lambda x: x[1]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;('Rosalind_0808', 60.91954022988506)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastActivityDate="2020-04-14T10:57:05.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12924" PostTypeId="2" ParentId="12919" CreationDate="2020-04-14T11:57:45.027" Score="6" Body="&lt;p&gt;While it was easy to get your data after your edit, the data was not quite useful, the second data frame/matrix has 2 columns only!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had to create a fake data set, that looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;my_df &amp;lt;- data.frame(&#xA;  gene_class = rep(letters[1:4],3),&#xA;  some_value = rep(1:3*10,4)&#xA;)&#xA;rownames(my_df) &amp;lt;- paste(&quot;Gene-&quot;,rep(LETTERS[1:4],each = 3),1:4,sep=&quot;&quot;)&#xA;&#xA;&amp;gt; my_df&#xA;        gene_class some_value&#xA;Gene-A1          a         10&#xA;Gene-A2          b         20&#xA;Gene-A3          c         30&#xA;Gene-B4          d         10&#xA;Gene-B1          a         20&#xA;Gene-B2          b         30&#xA;Gene-C3          c         10&#xA;Gene-C4          d         20&#xA;Gene-C1          a         30&#xA;Gene-D2          b         10&#xA;Gene-D3          c         20&#xA;Gene-D4          d         30&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Using the &lt;code&gt;ComplexHeatmap&lt;/code&gt; package, I got something similar to the figure you have posted.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# data frame for annotation&#xA;my_annot &amp;lt;- my_df[,&quot;gene_class&quot;,drop=FALSE]&#xA;&#xA;# the Heatmap() function requires a matrix&#xA;my_matrix &amp;lt;- as.matrix(my_df[,&quot;some_value&quot;,drop=FALSE])&#xA;&#xA;dev.off() # clear the plotting area in case not empty&#xA;ha = rowAnnotation(gene_class = my_annot$gene_class)&#xA;&#xA;ht &amp;lt;-  Heatmap(my_matrix,&#xA;               left_annotation = ha, # gene_class&#xA;               row_names_side = &quot;left&quot;, # position of row (gene) names&#xA;               heatmap_width = unit(0.3, &quot;cm&quot;)*nrow(my_matrix), # cell width&#xA;               cluster_rows = FALSE, # no clustering&#xA;               cluster_columns = FALSE, # no clustering&#xA;               cell_fun = function(j, i, x, y, w, h, col) { # add text (&quot;some value column&quot;) to each grid&#xA;                 grid.text(my_matrix[i, j], x, y)&#xA;               })&#xA;draw(ht, heatmap_legend_side = &quot;bottom&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;ComplexHeatmap&lt;/code&gt; package offers enormous flexibility for customization. I strongly recommend devoting some time to it. The way I use it, I try to put all of my data/information in a single data frame/table, which I subsequently subset to create data frames and matrices for annotations and the actual heatmap(s) respectively.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8md20.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8md20.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-04-14T11:57:45.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12925" PostTypeId="1" CreationDate="2020-04-14T13:46:27.337" Score="1" ViewCount="152" Body="&lt;p&gt;I would like to ask if anyone knows or has experience in representing BigWig signals in a circular plot? I mean like IGV but in circular plot not linear. Or any library that can be used in plotting BigWig signals especially in Javascript?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I appreciate any suggestion,&#xA;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="8614" LastActivityDate="2021-03-12T21:35:53.807" Title="BigWig circular plot representation" Tags="|bigwig|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12926" PostTypeId="2" ParentId="12921" CreationDate="2020-04-14T14:07:28.923" Score="0" Body="&lt;p&gt;I think the hard thing in this problem is not the calculation of GC content, it is the processing of DNA strings because they are separated on different lines. I would suggest reading the file line by line to generate complete DNA strings for each Rosalind entry. (I am not able to spot what is causing the error in the code).&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;seqDic = {}&#xA;with open(&quot;YouInFile&quot;, &quot;r&quot;) as inFile:&#xA;    for ln in inFile:&#xA;        ln = ln.rstrip()&#xA;        if ln.startswith(&quot;&amp;gt;&quot;):&#xA;            ln.name = ln[1:]&#xA;            seq = inFile.readline().rstrip() + inFile.readline().rstrip()&#xA;            seqDic[ln.name] = seq&#xA;        else&#xA;            continue&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically, whenever you encounter a &quot;&gt;&quot; in your file, you write down the Rosalind name, and then you read the next two lines and combine them together. You put the name and the sequence in a dictionary, and iterate over the dictionary to calculate GC content.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that I did not debug my code. Let me know if it is not working.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-04-14T14:07:28.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12927" PostTypeId="2" ParentId="12925" CreationDate="2020-04-15T00:05:45.387" Score="2" Body="&lt;p&gt;circosJS should probably work well, based on your requirements: &lt;a href=&quot;https://github.com/nicgirault/circosJS&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nicgirault/circosJS&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I posted a short Observable notebook that shows what calls could be used to bring data from remote bigWig files into a circosJS plot:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://observablehq.com/@alexpreynolds/rendering-bigwig-file-data-in-a-circular-representation&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://observablehq.com/@alexpreynolds/rendering-bigwig-file-data-in-a-circular-representation&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is less intended as an explicit how-to in Javascript, than as a starting point for the end user with some of the function calls and data scale issues involved. Hopefully it can be useful to the reader for exploratory purposes.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2021-03-12T21:35:53.807" LastActivityDate="2021-03-12T21:35:53.807" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12929" PostTypeId="1" CreationDate="2020-04-15T13:35:21.123" Score="1" ViewCount="49" Body="&lt;p&gt;I need to obtain publicly available RNA-seq + SNP genotyping data for a cohort - ideally one where the RNA-seq data has been sequenced from brain tissue. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have found CMC and GTEx to be sufficient for my needs, however the procedure to obtaining data from both these sources are extremely long and they request us to share our data with them as well; which we can not do because it is not ours to share. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was wondering if anyone knows of another resource that can be used for obtaining such data?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4001" LastActivityDate="2020-04-19T21:08:27.507" Title="Database of RNA-Seq and SNP genotyping data" Tags="|rna-seq|database|snp|genotyping|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12930" PostTypeId="2" ParentId="12917" CreationDate="2020-04-15T14:06:32.783" Score="1" Body="&lt;p&gt;I found the &lt;a href=&quot;https://cran.r-project.org/web/packages/LDheatmap/vignettes/LDheatmap.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;LDheatmap&lt;/a&gt; package in R very helpful at calculating linkage disequilibrium. You can make plots such as this:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kexIy.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kexIy.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2727" LastActivityDate="2020-04-15T14:06:32.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12931" PostTypeId="1" AcceptedAnswerId="12932" CreationDate="2020-04-15T14:08:47.217" Score="-5" ViewCount="107" Body="&lt;p&gt;I have this data all is one line even gene from first sample joined the second one while I want this data column wise each field separated like I have provided &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#Uploaded variation   #Uploaded_variation 1 883477 G T 1_883477_G/TPRDM16 1 3125744 A G 1_3125744_A/GPRDM16&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I want&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;         V1     V2  V3      V4  V5  V6&#xA;1_883477_G/T    1   883477  G   T   PRDM16&#xA;1_3125744_A/G   1   3125744 A   G   PRDM16&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2020-04-16T08:40:20.993" Title="How I reformat this data" Tags="|awk|" AnswerCount="1" CommentCount="5" ClosedDate="2020-04-16T16:49:31.413" ContentLicense="CC BY-SA 4.0" />
  <row Id="12932" PostTypeId="2" ParentId="12931" CreationDate="2020-04-15T20:37:34.917" Score="1" Body="&lt;p&gt;I am really curious what software or piece of code gave you this one-line data. I would think twice before using a tool giving such output. Anyway, I think regular expressions is the way to go in this case (this solution uses &lt;code&gt;R&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(stringr)&#xA;&#xA;text &amp;lt;- &quot;#Uploaded variation   #Uploaded_variation 1 883477 G T 1_883477_G/TPRDM16 1 3125744 A G 1_3125744_A/GPRDM16&quot;&#xA;&#xA;my_pattern &amp;lt;- &quot;[0-9]+\\s[0-9]+\\s[ATGC]+\\s[ATGC]+\\s[0-9]+_[0-9]+_[ATGC]+\\/[0-9A-Z]+&quot;&#xA;&#xA;info_as_list &amp;lt;- str_extract_all(text,&#xA;                                pattern = my_pattern) %&amp;gt;% unlist()&#xA;&#xA;info_as_list &amp;lt;- sub(&quot;/&quot;,&#xA;                    &quot; &quot;,&#xA;                    info_as_list)&#xA;&#xA;info_as_list &amp;lt;- str_split(info_as_list, &quot; &quot;)&#xA;&#xA;as.data.frame(Reduce(rbind, info_as_list))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is the output, I guess the rowname &quot;init&quot; comes from &lt;code&gt;Reduce()&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;     V1      V2 V3 V4          V5      V6&#xA;init  1  883477  G  T  1_883477_G TPRDM16&#xA;      1 3125744  A  G 1_3125744_A GPRDM16&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here is what the regular expression does, you might need to alter this based on your data:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[0-9]+\\s       :   start position&#xA;[0-9]+\\s       :   end position&#xA;[ATGC]+\\s      :   ref nucleotide(s)&#xA;[ATGC]+\\s      :   alternate nucleotide(s)&#xA;[0-9]+          :   start&#xA;_               :   underscore&#xA;[0-9]+          :   end&#xA;_               :   underscore&#xA;[ATGC]+         :   ref&#xA;\\/             :   slash&#xA;[0-9A-Z]+       :   gene symbol&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;+&lt;/code&gt; means one or more of the preceding and &lt;code&gt;R&lt;/code&gt; uses two backslashes, &lt;code&gt;\\&lt;/code&gt;, as escape characters.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2020-04-16T08:40:20.993" LastActivityDate="2020-04-16T08:40:20.993" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12933" PostTypeId="1" CreationDate="2020-04-16T00:42:09.893" Score="2" ViewCount="271" Body="&lt;p&gt;I have extracted microarray gene expression data from GEO using the &lt;code&gt;GEOquery&lt;/code&gt; package in R. There are multiple probes per gene. Is there a way to select the best probe? If so, what would be the criteria and how can you do this? Maybe use &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-12-474&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jetset probes&lt;/a&gt; (=best quality probes) for this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Otherwise, would you use the average expression value of all probes for each gene? If so, how can you do this in R, please?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit after comment below:&#xA;I like the suggestion proposed here: &lt;a href=&quot;https://support.bioconductor.org/p/70133/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.bioconductor.org/p/70133/&lt;/a&gt;, using this code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; Probesets=paste(&quot;a&quot;,1:200,sep=&quot;&quot;) # &quot;fake probesets&quot;&#xA; Genes=sample(letters,200,replace=T)&#xA; Value=rnorm(200)&#xA; X=data.frame(Probesets,Genes,Value) # I edited this line, as it seemed to contain an error&#xA; X=X[order(X&lt;span class=&quot;math-container&quot;&gt;$Value,decreasing=T),]&#xA; Y=X[which(!duplicated(X$&lt;/span&gt;Genes)),]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But, I do not understand it. How is the value for each gene in Y calculated? It was commented that this is the MAD, but calculating the Median Absolute Deviation (MAD) by hand results in a different value. Could anyone help, please? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="7354" LastEditorUserId="7354" LastEditDate="2020-04-16T10:53:49.250" LastActivityDate="2020-04-16T10:53:49.250" Title="How can I collapse probes to single value per gene using GEOquery?" Tags="|r|microarray|gene-expression|geoquery|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12934" PostTypeId="1" CreationDate="2020-04-16T08:07:58.717" Score="2" ViewCount="554" Body="&lt;p&gt;What's the best/easiest way for accessing .bam/.cram files from S3?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have .bam/.bai .cram/.crai and .bed + .gff files that I want to make available easily to others so they can browse in IGV and other solutions from their computers. There seem to be different solutions after cursory Googling, what would you recommend?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is what I've found so far:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;IGV:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://twitter.com/igvteam/status/1187766079457644544&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://twitter.com/igvteam/status/1187766079457644544&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://umccr.org/blog/igv-amazon-backend-setup/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://umccr.org/blog/igv-amazon-backend-setup/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Samtools: samtools directly supports AWS&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.htslib.org/doc/htslib-s3-plugin.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.htslib.org/doc/htslib-s3-plugin.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;The full URL format is:&#xA;&#xA;    s3[+SCHEME]://[ID[:SECRET[:TOKEN]]@]BUCKET/PATH&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="180" LastEditorUserId="180" LastEditDate="2020-04-16T08:21:47.107" LastActivityDate="2022-04-10T16:56:02.450" Title="Accessing .bam/.cram files from AWS S3?" Tags="|sam|samtools|igv|cram|aws|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12935" PostTypeId="1" CreationDate="2020-04-16T10:10:36.367" Score="2" ViewCount="403" Body="&lt;p&gt;Failing to build my own Arabidopsis thaliana reference with Mt using 'codon.Mitochondrial' and Pt using 'codon.Bacterial_and_Plant_Plastid' but the same commands works well with both lines with codon tables removed below.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The snpeff commands &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -Xms24g -Xmx24g -jar /opt/biotools/snpEff/snpEff.jar build -gff3 -v Arabidopsis_thaliana.TAIR10.46 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;returns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java.lang.RuntimeException: Error parsing property 'Arabidopsis_thaliana.TAIR10.46.Pt.codonTable'. No such codon table 'codon.Bacterial_and_Plant_Plastid'&#xA;        at org.snpeff.snpEffect.Config.createCodonTables(Config.java:169)&#xA;        at org.snpeff.snpEffect.Config.readConfig(Config.java:650)&#xA;        at org.snpeff.snpEffect.Config.init(Config.java:480)&#xA;        at org.snpeff.snpEffect.Config.&amp;lt;init&amp;gt;(Config.java:117)&#xA;        at org.snpeff.SnpEff.loadConfig(SnpEff.java:451)&#xA;        at org.snpeff.snpEffect.commandLine.SnpEffCmdEff.run(SnpEffCmdEff.java:1000)&#xA;        at org.snpeff.snpEffect.commandLine.SnpEffCmdEff.run(SnpEffCmdEff.java:984)&#xA;        at org.snpeff.SnpEff.run(SnpEff.java:1183)&#xA;        at org.snpeff.SnpEff.main(SnpEff.java:162)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(same error for the Mt when removing Pt)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;my fasta has the two chromosomes&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1 dna:chromosome chromosome:TAIR10:1:1:30427671:1 REF&#xA;&amp;gt;2 dna:chromosome chromosome:TAIR10:2:1:19698289:1 REF&#xA;&amp;gt;3 dna:chromosome chromosome:TAIR10:3:1:23459830:1 REF&#xA;&amp;gt;4 dna:chromosome chromosome:TAIR10:4:1:18585056:1 REF&#xA;&amp;gt;5 dna:chromosome chromosome:TAIR10:5:1:26975502:1 REF&#xA;&amp;gt;Mt dna:chromosome chromosome:TAIR10:Mt:1:366924:1 REF&#xA;&amp;gt;Pt dna:chromosome chromosome:TAIR10:Pt:1:154478:1 REF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;my data folder is OK&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data/Arabidopsis_thaliana.TAIR10.46/&#xA;drwxr-xr-x   3 u0002316 domain users 4.0K Apr 15 13:33 .&#xA;drwxr-xr-x 244 u0002316 domain users  16K Apr 10 09:39 ..&#xA;-rw-r--r--   1 u0002316 domain users 106M Apr 10 09:40 genes.gff&#xA;-rw-r--r--   1 u0002316 domain users 117M Apr 10 09:40 sequences.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;my snpeff.config contains:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Arabidopsis_thaliana.TAIR10.46.genome : Arabidopsis_thaliana.TAIR10.46&#xA;Arabidopsis_thaliana.TAIR10.46.Mt.codonTable : codon.Mitochondrial&#xA;Arabidopsis_thaliana.TAIR10.46.Pt.codonTable : codon.Bacterial_and_Plant_Plastid&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;the two codon tables are present by default in the snpeff.config&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;codon.Mitochondrial                         : TTT/F, TTC/F, TTA/L+, TTG/L+, TCT/S, TCC/S, TCA/S, TCG/S, TAT/Y, TAC/Y, TAA/*, TAG/*, TGT/C, TGC/C, TGA/W, TGG/W, CTT/L, CTC/L, CTA/L, CTG/L+, CCT/P, CCC/P, CCA/P, CCG/P, CAT/H, CAC/H, CAA/Q, CAG/Q, CGT/R, CGC/R, CGA/R, CGG/R, ATT/I+, ATC/I+, ATA/I+, ATG/M+, ACT/T, ACC/T, ACA/T, ACG/T, AAT/N, AAC/N, AAA/K, AAG/K, AGT/S, AGC/S, AGA/R, AGG/R, GTT/V, GTC/V, GTA/V, GTG/V+, GCT/A, GCC/A, GCA/A, GCG/A, GAT/D, GAC/D, GAA/E, GAG/E, GGT/G, GGC/G, GGA/G, GGG/G&#xA;&#xA;codon.Bacterial_and_Plant_Plastid           : TTT/F, TTC/F, TTA/L, TTG/L+, TCT/S, TCC/S, TCA/S, TCG/S, TAT/Y, TAC/Y, TAA/*, TAG/*, TGT/C, TGC/C, TGA/*, TGG/W, CTT/L, CTC/L, CTA/L, CTG/L+, CCT/P, CCC/P, CCA/P, CCG/P, CAT/H, CAC/H, CAA/Q, CAG/Q, CGT/R, CGC/R, CGA/R, CGG/R, ATT/I+, ATC/I+, ATA/I+, ATG/M+, ACT/T, ACC/T, ACA/T, ACG/T, AAT/N, AAC/N, AAA/K, AAG/K, AGT/S, AGC/S, AGA/R, AGG/R, GTT/V, GTC/V, GTA/V, GTG/V+, GCT/A, GCC/A, GCA/A, GCG/A, GAT/D, GAC/D, GAA/E, GAG/E, GGT/G, GGC/G, GGA/G, GGG/G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Anybody ever succeeded to build a custom genome with specific codon tables?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="2205" LastActivityDate="2020-04-17T14:38:04.673" Title="SNPEFF genome build with specific codon usage" Tags="|sequence-annotation|variants|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12937" PostTypeId="1" CreationDate="2020-04-16T19:15:44.643" Score="3" ViewCount="2203" Body="&lt;p&gt;I want to download the original BAM files that the authors had uploaded to SRA. Normally, I would just use &lt;code&gt;sam-dump&lt;/code&gt;, but the files are having issues that seem related to &lt;a href=&quot;https://github.com/ncbi/sra-tools/issues/23&quot; rel=&quot;nofollow noreferrer&quot;&gt;this issue&lt;/a&gt;. Since according to the entry, AWS S3 also hosts the original BAM files, I thought I could download these directly.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Example SRA Entry&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/LP8Y7.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;NCBI documentation implies that I can't download this directly, but I can freely copy to other AWS locations within the region. To this end, I created my own S3 bucket (&lt;code&gt;mm-mneuron&lt;/code&gt;) and am now trying to copy from the SRA bucket to mine.  Here's what I try:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import boto3&#xA;import botocore&#xA;&#xA;s3 = boto3.resource('s3')&#xA;&#xA;bam_file = {&#xA;  'Bucket': 'sra-pub-src-6',&#xA;  'Key': 'SRR5253957/RPI25_0.bam'&#xA;}&#xA;&#xA;my_bucket = s3.Bucket('mm-mneuron')&#xA;&#xA;my_bucket.copy(bam_file, 'RPI25_0.bam')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This fails with:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;botocore.exceptions.ClientError: An error occurred (403) when calling&#xA;  the HeadObject operation: Forbidden&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That is, it sounds like I can't access the SRA bucket. I've tested downloading and uploading to my bucket, so I know I have write permissions. Not sure what else to try here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;How can I access the SRA data on S3?&lt;/strong&gt; &lt;/p&gt;&#xA;" OwnerUserId="900" LastActivityDate="2020-09-14T23:07:55.737" Title="Downloading SRA Files from AWS" Tags="|rna-seq|python|phylogenetics|aws|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12938" PostTypeId="1" AcceptedAnswerId="12939" CreationDate="2020-04-16T19:46:46.923" Score="2" ViewCount="243" Body="&lt;p&gt;I am analyzing some SARS-CoV-2 sequencing runs abd often find read alignments like the one in the image.&lt;a href=&quot;https://i.stack.imgur.com/mrCwf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mrCwf.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GACCCGGTGGGTTTTAC---ACTTAAAAACACAGTCTGTAC&#xA;GACCCTGTGGGTTTTAC---ACTTAAAAACACAGTCTGTAC&#xA;          GTTTCAC---ACTTAAAAACACAGTCTGTAC&#xA;          GGTTTAC---ACTTAAAAGCACAGTCTGTAC&#xA;CCACACTCTCCTAGCAC---ACTTAAAAACACAGTCTGTAC&#xA;CCACACTCTCCTAGCACCATACTTAAAAACACAGTCTGTAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It basically says there are reads that share part of a sequence and part of another one. The first 4 reads are the same sequence and it's a sequence that you can effectively find in the official SARS-CoV-2 assembly but the other two reads are a mix of this one with another sequence (and both are the same mix) but with only a small (3bps) indel of difference.&lt;/p&gt;&#xA;&lt;p&gt;The sequence of the last two reads does not appear anywhere in the official SARS-CoV-2 sequence.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What does it mean?&lt;/li&gt;&#xA;&lt;li&gt;Are these sequencing errors?&lt;/li&gt;&#xA;&lt;li&gt;If that is the case how is it that the same mistake is repeated with similar but not exactly the same data?&lt;/li&gt;&#xA;&lt;li&gt;If these are real RNA sequences present in the probe, why are these not used anywhere in the final assembly and how did they arrive there?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Is there any other option that I am missing?&lt;/p&gt;&#xA;" OwnerUserId="5190" LastEditorUserId="3848" LastEditDate="2020-12-05T10:10:51.080" LastActivityDate="2020-12-05T10:10:51.080" Title="What is the meaning of these misaligned reads in a sequencing run?" Tags="|sequence-alignment|ngs|assembly|covid-19|sars-cov-2|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12939" PostTypeId="2" ParentId="12938" CreationDate="2020-04-16T20:11:27.537" Score="2" Body="&lt;p&gt;The unaligned parts of the bottom two sequences are from elsewhere in the sars-cov-2 genome if you blast them. Likely causes of this are library prep. artifacts (not terribly common, but you'll find a few cases here in there once you have millions of reads) where fragments end up ligating together or (probably more likely) rearrangements of the viral sequence. My guess would be that the latter is what occurred, given how rapidly viruses reproduce. Whether that arrangement can lead to viable viruses I don't know (that's too far out of my field).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assemblies represent at best a snapshot of a single example of a species at a single moment in time. Actual individuals in the wild can vary quite substantially, especially when they don't need to be viable.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-04-16T20:11:27.537" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="12940" PostTypeId="1" AcceptedAnswerId="12949" CreationDate="2020-04-16T21:19:41.203" Score="1" ViewCount="46" Body="&lt;p&gt;After predicting and annotating all the transposable elements. For four related species.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now I would like to find out these predicted candidates found in my costume library, either vertical or horizontal transfers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;are there any methodologies to suggest? &lt;/p&gt;&#xA;" OwnerUserId="3332" LastEditorUserId="3332" LastEditDate="2020-04-16T21:40:32.407" LastActivityDate="2020-04-21T12:14:23.903" Title="TEs vertical or horizontal transfers?" Tags="|transposable-elements|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12941" PostTypeId="1" CreationDate="2020-04-16T22:41:10.100" Score="2" ViewCount="62" Body="&lt;p&gt;I am analyzing the reads for the SARS-CoV-2 assembly with id SRR11140748. Apparently these reads were obtained with parallel sequencing by Illumina and Oxford Nanopore Technologies.&lt;/p&gt;&#xA;&lt;p&gt;I have found these reads&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GGTAAGTATGTACAAATACCTACAACTTGTGCTAATGACCCTGTGGGTTTTACACTTAAAAACACAGTCTGTACCGTCTGCGGTATGTGGAAAGGTTATGGCTGTAGTTGTGATCAACTCCGCGAACCCATGCTTCAGTCAGCTGATGCACAATCGTTTTTAAACGGGTTTGCGGTGTAAGTGCAGCCCGTCTTACACCGTGCGGCACAGGCACTAGTACTGATGTCGTATACAGGGCTTTTGACATCAGTATGTACAAATACCTACAACTTGTGCTAATGACCCTGTGGGTTTTACACTTAAAAACACAGTCTGTACCGTCTGCGGTATGTGGAAAGGTTATGGCTGTAGTTGTGATCAACTCCGCGAACCCATGCTTCAGTCAGCTGATGCACAATCGTTTTTAAACGAGTATGTACAAATACCTACAACTTGTGCT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see, the second one has an exact copy at the tail of its beginning:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;AGTATGTACAAATACCTACAACTTGTGCT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have found other issues apparently caused by rearrangements of the virus genome, but this case seems to be different. It seems unlikely that the virus mutated in such a way and then a read was cut exactly this way. What is the most likely explanation for this read?&lt;/p&gt;&#xA;&lt;p&gt;It's not the only one displaying the same or similar behavior. for example this other one has a copy at the tail but from bps not exactly at the beginning (CACCGGAAGCCAATATGGATC):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CAGTTACACCGGAAGCCAATATGGATCAAGAATCCTTTGGTGGTGCATCGTGTTGTCTGTACTGCCGTTGCCACATAGATCATCCAAATCCTAAAGGATTTTGTGACTTAAAAGGTAAGTATGTACAAATACCTACAACTTGTGCTAATGACCCTGTGGGTTTTACACTTAAAAACACAGTCTGTACCGTCTGCGGTATGTGGAAAGGTTATGGCTGTAGTTGTTACACCGGAAGCCAATATGGATC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5190" LastEditorUserId="3848" LastEditDate="2020-11-30T22:30:45.677" LastActivityDate="2020-11-30T22:30:45.677" Title="Assembly reads having a copy of their beginning in their tail" Tags="|ngs|covid-19|illumina|genome-sequencing|sars-cov-2|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12943" PostTypeId="1" CreationDate="2020-04-17T03:49:06.287" Score="0" ViewCount="853" Body="&lt;p&gt;pdb code: &lt;code&gt;2obt&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The pdb file has a coordinates of a monomer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have used following commands to generate an assembly, or in other words to upload one more biological unit so that the protein would be a dimer:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;set assembly, &quot;&quot;&#xA;fetch 2obt, asu, async=0&#xA;&#xA;set assembly, &quot;1&quot;&#xA;fetch 2obt, assembly1, async=0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After generating the functional dimer in  pymol under name &lt;code&gt;assembly1&lt;/code&gt;, I have an important issue that I cannot save a dimer structure in pdb file with dimer coordinates. Does anybody has an idea how to go around it or is it even possible to do that?&lt;/p&gt;&#xA;" OwnerUserId="8598" LastActivityDate="2020-04-17T05:06:14.317" Title="Is there a way to save a full pdb file in PYMOL after uploading biological unit?" Tags="|pymol|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12944" PostTypeId="2" ParentId="9039" CreationDate="2020-04-17T04:11:21.827" Score="1" Body="&lt;p&gt;obviously I'm late but if this is still an open question open an issue in one of our github projects,&lt;br&gt;&#xA;github.com/igvteam/igv (Java desktop), &#xA;github.com/igvteam/igv.js (pure javascript component), &#xA;github.com/igvteam/igv-webapp (html/javascript application, 100% client side)&#xA;as appropriate.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As already mentioned the web igvs are 100% client, data is fetched from servers using range requests.   It scales very well,  as long as your data is indexed.   If you really want to generate server side images a better choice might be the UCSC genome browser which is designed for that.&lt;/p&gt;&#xA;" OwnerUserId="1681" LastActivityDate="2020-04-17T04:11:21.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12945" PostTypeId="2" ParentId="12943" CreationDate="2020-04-17T05:06:14.317" Score="4" Body="&lt;p&gt;So on the GUI form of PyMOL on the right hand side you have a list of what is ambiguously called &lt;code&gt;objects&lt;/code&gt; in PyMOL (models elsewhere), which can be enabled or disabled by clicking on their name.&#xA;The name also has a &lt;code&gt;1/1&lt;/code&gt; in green or similar. This indicates the state. In your case you have 2 states.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To split a multistate object use&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;split_state prot_A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To combine two objects one uses the command&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;create new_name, prot_A or prot_B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now, this has two caveats which if violated result in a nasty piece of modern art:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;if the chains, segment ids and residues are the same&lt;/li&gt;&#xA;&lt;li&gt;if you had two overlapping macromolecules&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;To deal with the first, change the chain id on one of them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;alter prot_B, chain='B'&#xA;   sort&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Segment identifier is a weird thing that most comp-biochem tools dislike, so I like to remove that&lt;/p&gt;&#xA;&#xA;&lt;p&gt;alter prot_B, segi=''&#xA;   sort&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now when you go to save your &lt;code&gt;new_name&lt;/code&gt; object, in the popup of the GUI you get a dropdown menu defaulted to enabled (visible objects that have not be clicked to be disabled). There you can choose &lt;code&gt;new_name&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Aligning two&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If someone that is in not the OP author has a template protein and needs to make a dimer of another based on that, the process is the same.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;create twin, prot_B&#xA;align twin, prot_A and chain B&#xA;alter twin, chain='B'&#xA;sort&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h2&gt;SDF files&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If someone else has a multicompound sdf where the state is the header name, I strongly advise not to use &lt;code&gt;split_state&lt;/code&gt;. Yes, the python pymol module (not the app) handles states in a really weird way, but states can be selected with &lt;code&gt;state 1&lt;/code&gt; selection algebra and &lt;code&gt;state&lt;/code&gt; is an argument for most commands that do not accept &lt;code&gt;state&lt;/code&gt; selector. &lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-04-17T05:06:14.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12946" PostTypeId="2" ParentId="12941" CreationDate="2020-04-17T08:37:29.543" Score="2" Body="&lt;p&gt;If you google AGTATGTACAAATACCTACAACTTGTGCT you'll find it's a primer sequence: &lt;a href=&quot;https://artic.network/resources/ncov/ncov-amplicon-v3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://artic.network/resources/ncov/ncov-amplicon-v3.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2020-04-17T08:37:29.543" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12947" PostTypeId="1" AcceptedAnswerId="12948" CreationDate="2020-04-17T08:51:38.230" Score="4" ViewCount="311" Body="&lt;p&gt;I would like to use the &lt;code&gt;--default-resources&lt;/code&gt; parameter for profiles which is available in later versions of &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;snakemake&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To install snakemake, I created a new conda environment:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;conda create -n snakemake -c conda-forge -c bioconda snakemake&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The currently installed version is listed as 5.14:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ conda activate snakemake&#xA;(snakemake)$ conda list | grep snakemake&#xA;# packages in environment at /home/uvi/be/hde/.conda/envs/snakemake:&#xA;snakemake                 5.14.0                        0    bioconda&#xA;snakemake-minimal         5.14.0                     py_1    bioconda&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, command line tool says it is on version 5.1.4:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(snakemake)$ snakemake --version&#xA;5.1.4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This version does not know the new parameter &lt;code&gt;--default-resources&lt;/code&gt; yet. So it gives an error when using it&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;snakemake: error: unrecognized arguments: --default-resources partition = &quot;thin-shared&quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I already checked that I'm running the executable from inside the environment, rather than a different, external one:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;(snakemake)&lt;span class=&quot;math-container&quot;&gt;$ which snakemake&#xA;~/.conda/envs/snakemake/bin/snakemake&#xA;(snakemake)$&lt;/span&gt; $(which snakemake) --version&#xA;5.1.4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8629" LastEditorUserId="8629" LastEditDate="2020-04-17T09:23:50.207" LastActivityDate="2020-04-17T09:23:50.207" Title="conda doesn't install latest version of snakemake" Tags="|snakemake|conda|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12948" PostTypeId="2" ParentId="12947" CreationDate="2020-04-17T09:15:43.370" Score="4" Body="&lt;p&gt;Python (whether installed with conda or not) will prefer packages installed in your home directory unless you tell it not to. This is a little known secret about using a virtualenv, actually, since internally those tell python to ignore everything in your home directory. Within conda, there's long been a debate about whether python (and also R) should continue looking for packages in the normal places or only within a conda environment. Since it would likely break many people's workflows, it's been decided to allow the normal language features of looking for packages not just within conda envs but also in the normal system and user locations. For python in particular you can disable this as follows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;PYTHONNOUSERSITE=True snakemake ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then only the packages installed within your conda environment will be used.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-04-17T09:15:43.370" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12949" PostTypeId="2" ParentId="12940" CreationDate="2020-04-17T10:51:03.283" Score="1" Body="&lt;p&gt;You make a phylogenetic tree of the &quot;host species&quot; and compare it to the phylogeny of the TE's. Homology is not a good idea for evolutionary trees.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two snags:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Concerted evolution, the TE's will risk recombining with each other within a host genome and that can mess up the phylogeneic signal (the theory as why is quite complicated)&lt;/li&gt;&#xA;&lt;li&gt;If host species phylogeny are higher eukaryotes thats a lot of genetic material, so you'd need a short-cut. You can either use someones species tree, or focus on TE's within a subsection of the host genome and produce a tree of this subsection.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;With point number 2 &quot;a subsection of the genome&quot; you can then assess synteny, i.e. has the TE jumped between position between species. If the TE hasn't moved then its an orthogonal proof of putative vertical transmission. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;What you are looking for is incongruence between host phylogeny and TE phylogeny as evidence of horizontal transmission and congruence to call vertical transmission. However, incongruence could occur due to the complexity of concerted evolution, so synteny would assist again as an orthogonal proof. Congruence is assessed as a &quot;mirror tree&quot;, there is quite a lot of theory on this looking at host-parasite coevolution&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For making trees RAxML or PhyML are flavours of the month particularly for TEs. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;A state of the art-of-the-art review is &lt;a href=&quot;https://academic.oup.com/cz/article/62/4/393/1745416&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, but it goes into significant detail into precise proofs, which in my opinion arn't needed&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-04-21T12:14:23.903" LastActivityDate="2020-04-21T12:14:23.903" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12950" PostTypeId="1" CreationDate="2020-04-17T11:18:05.990" Score="1" ViewCount="793" Body="&lt;p&gt;I was wondering how could I obtain the end position of an alignment using its start position and its CIGAR String.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If an alignment present some soft clipping, for example: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;    20S55M&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, would the start position indicated in the SAM file be the first soft clipped base, or the first correctly mapped base?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="8631" LastEditorUserId="3848" LastEditDate="2020-04-17T15:45:17.927" LastActivityDate="2023-08-10T03:44:09.750" Title="Extract end position from CIGAR" Tags="|r|sequence-alignment|sam|cigar|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12951" PostTypeId="1" AcceptedAnswerId="12956" CreationDate="2020-04-17T11:23:59.560" Score="3" ViewCount="293" Body="&lt;p&gt;I have several long-read sequences, which when aligned to a human reference genome, aligns to multiple chromosomal fragment as a result of chromothripsis shattering the linear DNA and randomly piecing them together. This creates for a single primary alignment multiple supplementary alignments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With a single example of a read (only showing parts of all the bamfile information) being&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;cdaf1fcb-27bf-4890-8ac7-a097dfa12c38    16      chr11   15151487        60&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With the primary alignment starting at chr11 at 15151487&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then the supplementary alignment can be seen as follows &#xA;&lt;code&gt;SA:Z:chr13,49827969,+,1839S956M25D1584S,60,53;chr11,15152459,-,833S749M22D2797S,60,72;chrX,138324656,-,2540S404M2D1435S,60,20;chr11,15152066,+,278M11D4101S,60,20;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where the alignments are to chr13, 11, X and 11. With both regions on chr11 being close to the initial region from the primary alignment, making the regions on chromosome 11 the same (I know I haven't calculated the full chromosomal interval using the Cigar string for this example).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume the order of that supplementary information above is in the actual order. But the problem is that my DNA should be circular which doesn't fit with the order of the alignments since the circle would then be chr 11 (primary, start of circle) -&gt; chr 13 -&gt; chr11 (same as primary) -&gt; chrX -&gt; chr11 (back to the start of circle) &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is:&#xA;Q) So is the order of the supplementary information just pieces together randomly or is it true that I can assume the order is the true order of the sequencing read? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So thanks a lot for your help.&lt;/p&gt;&#xA;" OwnerUserId="8630" LastActivityDate="2020-04-17T16:12:18.713" Title="Chromosomal order of supplementary alignment in BAM file &quot;SA&quot; tag" Tags="|sequence-alignment|sam|long-reads|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12952" PostTypeId="2" ParentId="12950" CreationDate="2020-04-17T11:35:09.920" Score="2" Body="&lt;p&gt;The match is starting with the first mapped base ignoring any softclipped parts of the read.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example this are two reads from a bam file of mine:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;start   cigar   sequence &#xA;11710   17S83M  NNNNNNNNNNNNNNNNNGGCCTGGCCATGTGTATTTTTTTAAATTTCCACTGATGATTTTGCTGCATGGCCGGTGTTGAGAATGACTGCGCAAATTTGCC   &#xA;11713   100M    CTGGCCATGTGTATTTTTTTAAATTTCCACTGATGATTTTGCTGCATGGCCGGTGTTGAGAATGACTGCGCAAATTTGCCGGATTTCCTTTGCTGTTCCT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4885" LastEditorUserId="4885" LastEditDate="2020-04-17T11:42:56.453" LastActivityDate="2020-04-17T11:42:56.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12953" PostTypeId="1" CreationDate="2020-04-17T13:18:09.217" Score="0" ViewCount="58" Body="&lt;p&gt;I have 2000 lines of wheat genotyped with a 15K illumina chip. &#xA;I also have the name of the SNPs. &#xA;I would like to know how could I find out which SNPs are in each chromosomes.&lt;/p&gt;&#xA;" OwnerUserId="8632" LastActivityDate="2024-03-31T02:01:55.270" Title="General question: How to assign my SNP (Wheat, Illumina 15K) to specific chromosomes?" Tags="|snp|chromosomes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12954" PostTypeId="2" ParentId="12950" CreationDate="2020-04-17T14:19:08.043" Score="3" Body="&lt;p&gt;The start position is the first correctly mapped base. One way to do it, is to use GenomicAlignments in R to read in the alignments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example the sam file looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cat test.sam &#xA;@HD VN:1.3  SO:coordinate&#xA;@SQ SN:chrI LN:228861&#xA;D00689:415:CD4LUANXX:3:1316:7447:98083#CGGCTATGCAGGACGT 0   chrI    769 0   42M8S   *   0   0   CCATAGCCCGGTGGGGCCTGTGCTGCCACAGAGGGTGTGTCTAGATCGGA  B/B//FFFFFFBFFFF//7FF/&amp;lt;F/FFBFFFF//&amp;lt;F///&amp;lt;F/BFB//&amp;lt;F/  NM:i:1  MD:Z:21A20  AS:i:37 XS:i:37&#xA;D00689:415:CD4LUANXX:3:2308:9597:30445#CGGCTATGCAGGACGT 0   chrI    3965    0   3S47M   *   0   0   CCACACACCCACACACCACACCCACACCACACCCACACCACACCCACACC  BBBBBFFFFFFFFFFFFFFFFFFFFFFFFFFFFBFFFFFBF&amp;lt;FFFFF&amp;lt;FF  NM:i:0  MD:Z:47 AS:i:47 XS:i:47&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We convert to bam:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -bS test.sam &amp;gt; test.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And read using R:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(GenomicAlignments)&#xA;gr = readGAlignments(&quot;test.bam&quot;)&#xA;gr&#xA;&#xA;GAlignments object with 2 alignments and 0 metadata columns:&#xA;      seqnames strand       cigar    qwidth     start       end     width&#xA;         &amp;lt;Rle&amp;gt;  &amp;lt;Rle&amp;gt; &amp;lt;character&amp;gt; &amp;lt;integer&amp;gt; &amp;lt;integer&amp;gt; &amp;lt;integer&amp;gt; &amp;lt;integer&amp;gt;&#xA;  [1]     chrI      +       42M8S        50       769       810        42&#xA;  [2]     chrI      +       3S47M        50      3965      4011        47&#xA;          njunc&#xA;      &amp;lt;integer&amp;gt;&#xA;  [1]         0&#xA;  [2]         0&#xA;  -------&#xA;  seqinfo: 1 sequence from an unspecified genome&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The start and the end can be retrieved using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;start(gr)&#xA;[1]  769 3965&#xA;end(gr)&#xA;[1]  810 4011&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-04-17T14:19:08.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12955" PostTypeId="2" ParentId="12935" CreationDate="2020-04-17T14:38:04.673" Score="2" Body="&lt;p&gt;&lt;strong&gt;codon table issue&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I solved the issue with codon tables, the correct way to specify a table is the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Arabidopsis_thaliana.TAIR10.46.genome : Arabidopsis_thaliana.TAIR10.46&#xA;Arabidopsis_thaliana.TAIR10.46.Mt.codonTable : Mitochondrial&#xA;Arabidopsis_thaliana.TAIR10.46.Pt.codonTable : Bacterial_and_Plant_Plastid&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Note&lt;/em&gt; that one should remove the leading prefix '&lt;strong&gt;codon.&lt;/strong&gt;' from the name of the tables chosen from the config file&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Concerning accessory files:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;data to reproduce this issue:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;baseurl=&quot;ftp://ftp.ensemblgenomes.org/pub/plants/release-46&quot;&#xA;wget &lt;span class=&quot;math-container&quot;&gt;${baseurl}/fasta/arabidopsis_thaliana/dna/Arabidopsis_thaliana.TAIR10.dna.toplevel.fa.gz&#xA;wget $&lt;/span&gt;{baseurl}/fasta/arabidopsis_thaliana/cds/Arabidopsis_thaliana.TAIR10.cds.all.fa.gz&#xA;wget &lt;span class=&quot;math-container&quot;&gt;${baseurl}/fasta/arabidopsis_thaliana/pep/Arabidopsis_thaliana.TAIR10.pep.all.fa.gz&#xA;wget $&lt;/span&gt;{baseurl}/gff3/arabidopsis_thaliana/Arabidopsis_thaliana.TAIR10.46.gff3.gz&#xA;wget ${baseurl}/variation/vcf/arabidopsis_thaliana/arabidopsis_thaliana_incl_consequences.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I suspect that the format used by Ensembl is not 100% OK for SnpEff; I therefore simplified the fasta headers to keep only the first argument (name) and remove the remaining of the line&lt;/p&gt;&#xA;&#xA;&lt;p&gt;eg.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;AT3G05780.1 pep chromosome:TAIR10:3:1714941:1719608:-1 gene:AT3G05780 transcript:AT3G05780.1 gene_biotype:protein_coding transcript_biotype:protein_coding gene_symbol:LON3 description:Lon protease homolog 3, mitochondrial [Source:UniProtKB/Swiss-Prot;Acc:Q9M9L8]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;becomes&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;AT3G05780.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Despite this modification, I still have an issue with&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;java -jar /opt/biotools/snpEff/SnpEff.jar build -gff3 -v Arabidopsis_thaliana_TAIR10_46 2&amp;gt;&amp;amp;1 | tee -a log.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a LOT of warning about not being able to add UTR while it seems that UTR's are well present in the GFF3 file&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Reading GFF3 data file  :&#xA;  '/opt/biotools/snpEff_v4.4/./data/Arabidopsis_thaliana_TAIR10_46/genes.gff'&#xA;  WARNING: Cannot find transcript 'transcript:AT1G01010.1'. File&#xA;  '/opt/biotools/snpEff_v4.4/./data/Arabidopsis_thaliana_TAIR10_46/genes.gff'&#xA;  line&#xA;  18    '1  araport11   five_prime_UTR  3631    3759    .   +   .   Parent=transcript:AT1G01010.1'&#xA;  WARNING: Could not add UTR. File&#xA;  '/opt/biotools/snpEff_v4.4/./data/Arabidopsis_thaliana_TAIR10_46/genes.gff'&#xA;  line&#xA;  18    '1  araport11   five_prime_UTR  3631    3759    .   +   .   Parent=transcript:AT1G01010.1'&#xA;  WARNING: Cannot find transcript 'transcript:AT1G01020.3'. File&#xA;  '/opt/biotools/snpEff_v4.4/./data/Arabidopsis_thaliana_TAIR10_46/genes.gff'&#xA;  line&#xA;  69    '1  araport11   three_prime_UTR 6788    6914    .   -   .   Parent=transcript:AT1G01020.3'&#xA;  WARNING: Could not add UTR. File&#xA;  '/opt/biotools/snpEff_v4.4/./data/Arabidopsis_thaliana_TAIR10_46/genes.gff'&#xA;  line&#xA;  69    '1  araport11   three_prime_UTR 6788    6914    .   -   .   Parent=transcript:AT1G01020.3'&#xA;  WARNING: Cannot find transcript 'transcript:AT1G01020.5'. File&#xA;  '/opt/biotools/snpEff_v4.4/./data/Arabidopsis_thaliana_TAIR10_46/genes.gff'&#xA;  line&#xA;  88    '1  araport11   three_prime_UTR 6788    6914    .   -   .   Parent=transcript:AT1G01020.5'&#xA;  WARNING: Could not add UTR. File&#xA;  '/opt/biotools/snpEff_v4.4/./data/Arabidopsis_thaliana_TAIR10_46/genes.gff'&#xA;  line&#xA;  88    '1  araport11   three_prime_UTR 6788    6914    .   -   .   Parent=transcript:AT1G01020.5'&#xA;  WARNING: Cannot find transcript 'transcript:AT1G01020.4'. File&#xA;  '/opt/biotools/snpEff_v4.4/./data/Arabidopsis_thaliana_TAIR10_46/genes.gff'&#xA;  line&#xA;  109   '1  araport11   three_prime_UTR 6788    6914    .   -   .   Parent=transcript:AT1G01020.4'&#xA;  WARNING: Could not add UTR. File&#xA;  '/opt/biotools/snpEff_v4.4/./data/Arabidopsis_thaliana_TAIR10_46/genes.gff'&#xA;  line&#xA;  109   '1  araport11   three_prime_UTR 6788    6914    .   -   .   Parent=transcript:AT1G01020.4'&#xA;  WARNING: Cannot find transcript 'transcript:AT1G01020.1'. File&#xA;  '/opt/biotools/snpEff_v4.4/./data/Arabidopsis_thaliana_TAIR10_46/genes.gff'&#xA;  line&#xA;  128   '1  araport11   three_prime_UTR 6788    6914    .   -   .   Parent=transcript:AT1G01020.1'&#xA;  WARNING: Could not add UTR. File&#xA;  '/opt/biotools/snpEff_v4.4/./data/Arabidopsis_thaliana_TAIR10_46/genes.gff'&#xA;  line&#xA;  128   '1  araport11   three_prime_UTR 6788    6914    .   -   .   Parent=transcript:AT1G01020.1'&#xA;  WARNING: Cannot find transcript 'transcript:AT1G01030.1'. File&#xA;  '/opt/biotools/snpEff_v4.4/./data/Arabidopsis_thaliana_TAIR10_46/genes.gff'&#xA;  line 155  '1  araport11   three_prime_UTR 11649   11863   .   -   .   Parent&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;nothing good coming out of the cds.fa input although I treated it with the same name fix as the pep file&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;00:00:19  CDS check (FASTA file):&#xA;  '/opt/biotools/snpEff_v4.4/./data/Arabidopsis_thaliana_TAIR10_46/cds.fa'&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;00:00:19  Checking database using CDS sequences 00:00:19  Reading CDSs&#xA;  from file&#xA;  '/opt/biotools/snpEff_v4.4/./data/Arabidopsis_thaliana_TAIR10_46/cds.fa'...&#xA;  00:00:20  done (75976 CDSs). 00:00:20 Comparing CDS...    Labels:         '+' :&#xA;  OK        '.' : Missing       '*' : Error&#xA;    ....................................................................................................&#xA;    ....................................................................................................&#xA;    ....................................................................................................&lt;/p&gt;&#xA;  &#xA;  &lt;p&gt;many more lines with dots&#xA;    ....................................................................................................&#xA;    ....................................................................................................&#xA;    ....................................................................................................&#xA;    ...............................     CDS&#xA;  check:    Arabidopsis_thaliana_TAIR10_46  OK: 0   Warnings: 0 Not found:&#xA;  104147    Errors: 0   Error percentage: NaN% Fatal error: No CDS checked.&#xA;  This is might be caused by differences in FASTA file transcript IDs&#xA;  respect to database's transcript's IDs. Transcript IDs from database&#xA;  (sample):     'transcript:AT3G12250.7'    'transcript:AT5G18570.1'&#xA;    'AT3G27810.1'   'AT3G27810.2'   'transcript:AT2G16810.1'&#xA;    'transcript:AT3G01170.1'    'AT4G37030.1'   'transcript:AT1G79940.4'&#xA;    'transcript:AT2G44850.1'    'AT1G49540.1'   'AT1G49540.2'&#xA;    'transcript:AT4G35290.5'    'transcript:AT4G15840.2'&#xA;    'transcript:AT4G27500.1'    'AT4G37022.1'   'transcript:AT2G40008.1'&#xA;    'AT3G27820.1'   'transcript:AT4G24265.1'    'transcript:AT2G20618.1'&#xA;    'transcript:AT4G18375.2'    'AT4G37020.2'   'AT4G37020.1' Transcript IDs&#xA;  from database (fasta file):   'AT4G02541'     'AT1G12410.1'   'AT4G26510'&#xA;    'AT4G02540'     'AT3G04500.1'   'AT3G04500.2'   'AT3G04500.3'   'AT2G36250'&#xA;    'AT5G63350'     'AT2G36255'     'AT2G12280'     'AT1G03970.1'   'AT5G18390.1'&#xA;    'AT3G04490.4'   'AT5G38386'     'AT3G04490.2'   'AT3G04490.3'&#xA;    'AT3G04490.1'   'AT5G38380'     'AT4G02530'     'AT4G26520'     'AT1G69280.2'&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;link to the full stdout log after running the command without cds.fa&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/gxa4e533p64wxm9/SnpEff_build-log.txt?dl=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/gxa4e533p64wxm9/SnpEff_build-log.txt?dl=1&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2205" LastActivityDate="2020-04-17T14:38:04.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12956" PostTypeId="2" ParentId="12951" CreationDate="2020-04-17T16:12:18.713" Score="0" Body="&lt;p&gt;There is nothing in the &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMtags.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAMtags&lt;/a&gt; specification that dictates the order of the entries in an SA tag.  It may be entirely dependent on the aligner that you used.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking at my own data that was aligned with &lt;a href=&quot;https://github.com/lh3/bwa&quot; rel=&quot;nofollow noreferrer&quot;&gt;bwa&lt;/a&gt;, I cannot see a definite pattern.  Longer matches tend to be first, but not always.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your aligner documentation may have more information on how it uses SA tags.&lt;/p&gt;&#xA;" OwnerUserId="8636" LastActivityDate="2020-04-17T16:12:18.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12958" PostTypeId="1" AcceptedAnswerId="12960" CreationDate="2020-04-17T19:17:23.383" Score="1" ViewCount="104" Body="&lt;p&gt;I'm getting started in biology and bioinformatics with sequencing the SARS-Cov-2 or Coronavirus genome.  I'm interested in this code which identifies k-mers in the genome:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from skbio import DNA &#xA;DNA(str(seq.seq)).kmer_frequencies(5)&#xA;kmers = DNA(str(seq1.seq)).kmer_frequencies(5)&#xA;kmers&#xA;{'ATTAA': 60, 'TTAAA': 95, 'TAAAG': 64, 'AAAGG': 45, 'AAGGT': 46, 'AGGTT': 51, 'GGTTT': 55, &#xA;'GTTTA': 69, 'TTTAT': 75,...}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The author of the code wants to use k-mer breakdowns to vectorize the data for machine learning purposes.  What I'm curious about is why the k-mer in this case is five nucleotides?  What factors should go into my selection of the length of the k-mer for analysis?&lt;/p&gt;&#xA;" OwnerUserId="8640" LastEditorUserId="3848" LastEditDate="2020-12-01T20:25:43.323" LastActivityDate="2020-12-01T20:25:43.323" Title="Significance of k-mer length in COVID-19 sequence analysis?" Tags="|python|sequence-analysis|covid-19|k-mer|sars-cov-2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12959" PostTypeId="2" ParentId="10825" CreationDate="2020-04-17T19:27:53.277" Score="1" Body="&lt;p&gt;piPipes is biased for piRNAs but should be a good starting point:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/bowhan/piPipes/wiki/smallRNA-seq&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/bowhan/piPipes/wiki/smallRNA-seq&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well documented and with many features.&lt;/p&gt;&#xA;" OwnerUserId="108" LastActivityDate="2020-04-17T19:27:53.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12960" PostTypeId="2" ParentId="12958" CreationDate="2020-04-17T19:37:49.467" Score="0" Body="&lt;p&gt;5bp produces a high resolution analysis, providing its sufficiently robust.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Its a sliding window of 5 bp wide and 1 bp step length which runs along the length of each genome. They are performing SNP analysis/classification. The original sequence they were examining was,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ATTAAAGGTTTAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In any sliding window analysis the window size is arbitary towards the resolution and robustness of the analysis. The author believes that 5bp provides sufficiently robust information for a very high resolution classification/training analysis in their example. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The proof is the &lt;code&gt;accuracy&lt;/code&gt; of the classification. I assume they have good results from 5bp. In ML the rationale is the general laws of ML (over/under training) and assessed on the test split in the data (probably 20%).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A key reason for the 1bp step is that this is the way ML works, you need a lot of data and by doing 5 bp sliding window at 1 bp step that will generate ~30 000 pieces of data for each sequence. So that's 3 million data points for 100 sequences. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-04-18T09:14:31.297" LastActivityDate="2020-04-18T09:14:31.297" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12961" PostTypeId="2" ParentId="12929" CreationDate="2020-04-17T20:10:25.200" Score="1" Body="&lt;p&gt;Human RNA-seq data, like genetic data is very sensitive data as it can be used to identify people (and their relatives), so people will likely not distribute it without entering a formal agreement that ensures that people can be held accountable if the data is misused, stolen or made public. Depending on the consent form signed by the patients (or their relatives), they may not be allowed to share it at all. One possibility would be to start a collaboration with a group that has access to such data and ask them to run your analyses on their data. You can then have a shared publication and it's a win for everybody.&lt;/p&gt;&#xA;" OwnerUserId="1366" LastEditorUserId="1366" LastEditDate="2020-04-19T21:08:27.507" LastActivityDate="2020-04-19T21:08:27.507" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="12962" PostTypeId="2" ParentId="12937" CreationDate="2020-04-17T22:52:52.590" Score="2" Body="&lt;p&gt;A member of the SRA submission staff pointed out that using&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;prefetch --type all SRR5253957&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;will download the original files. In this case, it means running the above within an EC2 instance colocated with the S3 bucket (so, &lt;strong&gt;us-east-1&lt;/strong&gt;) and having installed and configured SRA Toolkit to work from AWS (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra/docs/sra-aws-download/&quot; rel=&quot;nofollow noreferrer&quot;&gt;as per this documentation&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, the particular files I am concerned with are not currently accessible, but this should generically work in most situations.&lt;/p&gt;&#xA;" OwnerUserId="900" LastActivityDate="2020-04-17T22:52:52.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12963" PostTypeId="2" ParentId="11516" CreationDate="2020-04-17T23:44:48.660" Score="1" Body="&lt;p&gt;Cube root transformation is another way to deal with zeros without dropping them.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;var^(1/3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; as Michael pointed out, square root transformation is another--perhaps more common--transformation that leaves zeros as zeros.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sqrt(var)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8641" LastEditorUserId="8641" LastEditDate="2020-04-18T03:42:07.023" LastActivityDate="2020-04-18T03:42:07.023" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12964" PostTypeId="2" ParentId="2700" CreationDate="2020-04-17T23:50:13.537" Score="1" Body="&lt;p&gt;Check out this file on SIFT.&#xA;ftp.ebi.ac.uk/pub/databases/msd/sifts/csv/pdb_chain_uniprot.csv&#xA;It contains residue-wise mapping of PDB chains with uniprot ids.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# 2020/04/12 - 21:03 | PDB: 15.20 | UniProt: 2020.02&#xA;PDB,CHAIN,SP_PRIMARY,RES_BEG,RES_END,PDB_BEG,PDB_END,SP_BEG,SP_END&#xA;101m,A,P02185,1,154,0,153,1,154&#xA;102l,A,P00720,1,40,1,40,1,40&#xA;102l,A,P00720,42,165,41,None,41,164&#xA;.&#xA;.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="274" LastActivityDate="2020-04-17T23:50:13.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12965" PostTypeId="1" CreationDate="2020-04-18T04:44:15.063" Score="0" ViewCount="53" Body="&lt;p&gt;Is there a way to programmatically retrieve graph features (a networkx graph) from Cytoscape in Python 3? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: To add more details, I had a dictionary of proteins, each protein was associated with a list of proteins that it interacts with. I wanted to create a protein-protein interaction graph so that I could extract graph features from it, including things like &lt;a href=&quot;https://en.wikipedia.org/wiki/Degeneracy_(graph_theory)&quot; rel=&quot;nofollow noreferrer&quot;&gt;k-core&lt;/a&gt; value of each node. &lt;/p&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="8601" LastEditDate="2020-04-21T04:03:09.737" LastActivityDate="2020-04-21T04:10:05.470" Title="Programmatically retrieve graph features from Cytoscape" Tags="|python|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12966" PostTypeId="1" CreationDate="2020-04-18T08:01:53.577" Score="2" ViewCount="59" Body="&lt;p&gt;I am looking for literature on protein and RNA mis-folding. I am on the computational/biophysics side, and what interests me are the practical applications: diseases caused by misfolding, drug discovery, etc. &lt;/p&gt;&#xA;" OwnerUserId="1813" LastEditorUserId="8653" LastEditDate="2020-04-23T21:17:21.940" LastActivityDate="2020-04-23T21:17:21.940" Title="Protein misfolding" Tags="|proteins|protein-structure|rna|literature-search|rna-structure|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12967" PostTypeId="1" CreationDate="2020-04-18T09:29:43.283" Score="1" ViewCount="782" Body="&lt;p&gt;I am starting to use Cromwell WDL scripts for running GATK workflows on a slurm cluster. When using the paired-fastq-to-unmapped-bam script I need to define the fastq sepcifics in the appropriate json file. This needs to be written separately for each pair of fastqs. My question is: &lt;strong&gt;Is there a way of generating the json files on a batch of fastq-files by scripting?&lt;/strong&gt; Of course, I could start writing scripts myself, but since I assume that GATK is used in a batch environment, I have the hope that something already exists.&lt;/p&gt;&#xA;" OwnerUserId="8644" LastEditorUserId="2776" LastEditDate="2020-04-24T18:30:34.913" LastActivityDate="2020-04-24T18:30:34.913" Title="How to generate the json files for Cromwell workflow execution?" Tags="|gatk|wdl|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12968" PostTypeId="2" ParentId="2871" CreationDate="2020-04-18T10:19:54.737" Score="0" Body="&lt;p&gt;&quot;If pilon was installed by condo, edit the file &quot;/Users/user/anaconda/bin/pilon&quot; by changing the maximum memory limit from 1G to higher numbers like &quot;14G&quot;. It works. &quot; &#xA;check the thread: &lt;a href=&quot;https://github.com/broadinstitute/pilon/issues/26&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/broadinstitute/pilon/issues/26&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8645" LastActivityDate="2020-04-18T10:19:54.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12969" PostTypeId="2" ParentId="12934" CreationDate="2020-04-18T10:21:10.137" Score="3" Body="&lt;p&gt;You could set up a &lt;a href=&quot;https://aws.amazon.com/premiumsupport/knowledge-center/read-access-objects-s3-bucket/&quot; rel=&quot;nofollow noreferrer&quot;&gt;public read-only&lt;/a&gt; bucket in an S3 account and put your files there. URLs would look like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;https://&lt;span class=&quot;math-container&quot;&gt;${username}.s3.amazonaws.com/public/$&lt;/span&gt;{filename}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For samtools, the request for a BAM file's index should be built off that URL. For IGV, I think you can specify separate, custom URLs for the data and the index resources.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Public buckets are available to all, without need for credentials. So that might be an issue to think about, with respect to &lt;a href=&quot;https://aws.amazon.com/s3/pricing/&quot; rel=&quot;nofollow noreferrer&quot;&gt;billing for web requests and data transfer&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm still not 100% certain that samtools/tabix support https requests reliably (I have a prior question on here about that), but that might be one approach.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2020-04-24T21:59:39.770" LastActivityDate="2020-04-24T21:59:39.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12970" PostTypeId="1" CreationDate="2020-04-18T13:35:40.937" Score="2" ViewCount="234" Body="&lt;p&gt;The data is present in the PLINK format .bed (individual genotypes), .bim (genetic markers) and .fam (sample IDs and disease phenotype file). I have also separate text files containing information on subjects related to two populations. (i.e. European and Hispanic)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NDGdV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NDGdV.png&quot; alt=&quot;Question related PLINK data&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8646" LastEditorUserId="3694" LastEditDate="2020-04-18T14:59:34.747" LastActivityDate="2021-09-04T11:07:00.620" Title="How to calculate average minor allele frequency (MAF) difference between two populations (i.e. European, Hispanic)" Tags="|bioconductor|plink|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12971" PostTypeId="1" AcceptedAnswerId="12975" CreationDate="2020-04-18T16:05:52.370" Score="2" ViewCount="164" Body="&lt;p&gt;Recent approaches to novel drug design using machine learning (ML) and deep learning, often involve generating hundreds of potential ligands which are later tested by docking with a target protein and recording the resulting binding affinity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it possible to go the other way around by first constraining the ligand generation process to the shape of the protein thus limiting the search space, as opposed to generating ligands of random shapes?&lt;/p&gt;&#xA;" OwnerUserId="8648" LastEditorUserId="3848" LastEditDate="2020-04-19T10:28:00.470" LastActivityDate="2022-08-08T09:19:01.427" Title="Generate ligands candidates based on protein shape" Tags="|protein-structure|machine-learning|docking|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12972" PostTypeId="1" CreationDate="2020-04-18T16:56:49.370" Score="1" ViewCount="441" Body="&lt;p&gt;I have removed duplicates from my bam file (not secondary, nor suplementary alignments), and then I have looked for some repeated &lt;code&gt;qnames&lt;/code&gt; (to assess the failure of a program) and I've found a lot of them. All of them has &lt;code&gt;TLEN&lt;/code&gt; equal to 0. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example: this is part of a bed file that a generate from the bam.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CHR  Start   End       QNAME                                 PNEXT  TLEN&#xA;12 86945793 86945944 A05325:240:SS348DSZX:2:5105:12246:3790 90044835 0&#xA;4 168461445 168461485 A05325:240:SS348DSZX:2:5105:12246:3790 90044835 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And those are these reads in the original bam:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A05325:240:SS348DSZX:2:5105:12246:3790  2177    4       168461445       0       111H40M 11      90044835        0       ATAAAATTAAAAAAATAAAAAAATTAAAAAAAAAAAAAAA        :,F,F,,FF:FF,F,,F,FFFF,,F,F,:,:,FFFFFFFF     NM:i:2  MD:Z:28T8G2     MC:Z:151M       AS:i:32 XS:i:31 RG:Z:HM327DSXX.3        SA:Z:12,86945793,+,20S37M94S,1,1;&#xA;A05325:240:SS348DSZX:2:5105:12246:3790  129     12      86945793        1       20S37M94S       11      90044835        0       ATGGTGAAAAAACAAATTATTTAAAAAAATCAAAAAAAAAAAAAAAACCAAAAAACATAAAAAAAAGGGCAGTAATTTTATTCATGGGGTCAAACAAAAAAAAAGCATAATATAAAATTAAAAAAATAAAAAAATTAAAAAAAAAAAAAAA      FF,,,:,F,F,,,,F,,,,,,,F,FFFF:,:FFFF:FFF,FF::F,:,,FFFF,,,F,FF::,,F,::,,,,F,,F:FF,,,,,,,F::F,F,,,,F,FFFFF:,F,,F,,:,F,F,,FF:FF,F,,F,FFFF,,F,F,:,:,FFFFFFFF      NM:i:1  MD:Z:9G27       MC:Z:151M       AS:i:32 XS:i:29 RG:Z:HM327DSXX.3        SA:Z:4,168461445,+,111S40M,0,2;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This are the commands used to remove duplicates:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -F 1024 -bo filtered.bam original.bam&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The commands used to generate this bed were:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools view sample.bam | awk '{ print $3, $4, $4+length($10), $1, $8, $9}' &amp;gt; sample.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Why do I present those alignments?&lt;/p&gt;&#xA;" OwnerUserId="8631" LastEditorUserId="8631" LastEditDate="2020-04-19T11:31:52.983" LastActivityDate="2020-04-20T17:36:25.230" Title="What's the meaning of having different alignments with the same qname in a bam with no duplicates?" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12973" PostTypeId="1" CreationDate="2020-04-18T21:21:01.110" Score="1" ViewCount="92" Body="&lt;p&gt;I have 2 data frames&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(df1)&#xA;   Chromosome Start_Position End_Position       Gene&#xA;1:      chr12       46254706     46254706       ARID2&#xA;2:      chr17        7579849      7579849        TP53&#xA;3:       chr9       21974777     21974780      CDKN2A&#xA;4:       chr1       27105930     27105930      ARID1A&#xA;5:       chr1      201982973    201982973        ELF3&#xA;6:       chr3       89480459     89480459       EPHA3&#xA;&amp;gt; head(df2)&#xA;  Chromosome Start_Position End_Position&#xA;1          1          10583       863511&#xA;2          1       12841835     12854479&#xA;3          1       17231299     17232877&#xA;4          1      120380739    142540462&#xA;5          1      142540484    142684781&#xA;6          1      142685823    143462598&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How I can intersect genes from df1 which are in the range of df2? Because both data frames have chromosome number, start and end but I don't know how to identify genes from df1 based on df2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have tried&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;df1[df1&lt;span class=&quot;math-container&quot;&gt;$Start_Position%in%df2$&lt;/span&gt;Start_Position,]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I obtain one gene&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; new_dataset &amp;lt;- df1 %&amp;gt;% right_join(df2, by=c(&quot;Chromosome&quot;,&quot;Start_Position&quot;))&#xA;Error: Can't join on 'Start_Position' x 'Start_Position' because of incompatible types (character / integer)&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-04-18T21:44:28.533" LastActivityDate="2020-04-28T14:01:44.110" Title="Intersecting a data frame based on the multiple columns" Tags="|r|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12974" PostTypeId="2" ParentId="11677" CreationDate="2020-04-18T21:59:05.157" Score="1" Body="&lt;p&gt;I would recommend looking into Python's plotting utilities, for example &lt;a href=&quot;https://matplotlib.org/tutorials/introductory/pyplot.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would recommend specifically looking at the examples that plot mathematical functions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The basic idea is that you create a function that computes your equation based on the different parameters as inputs (&lt;span class=&quot;math-container&quot;&gt;$a, a_{max}, l_{max}$&lt;/span&gt;) and then you can plug in a range of values for each parameter and plot e.g. with &lt;span class=&quot;math-container&quot;&gt;$a_{max}$&lt;/span&gt; on the x axis, then different lines showing a range of values for &lt;span class=&quot;math-container&quot;&gt;$l_{max}$&lt;/span&gt;, or whatever makes sense for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import matplotlib.pyplot as plt&#xA;&#xA;def fn(a, amax, lmax):&#xA;  return lmax/2 + (a/amax) * (lmax/2)&#xA;&#xA;plt.figure()&#xA;amax = np.arange(0.01, 1, .01)&#xA;plt.plot(amax, fn(.005, amax, .1), &quot;ro&quot;) ## red points&#xA;plt.plot(amax, fn(.005, amax, .2), &quot;go&quot;) ## green point&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have no idea if these parameter ranges are meaningful for you, you will have to figure that out for yourself. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Nc6FU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Nc6FU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-04-18T21:59:05.157" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12975" PostTypeId="2" ParentId="12971" CreationDate="2020-04-19T09:25:58.867" Score="7" Body="&lt;p&gt;TL;DR: docking is much slower than any ML approach, but the ML approach can be constrained by pharmacophores dictated by the active site.&lt;/p&gt;&#xA;&lt;h2&gt;Side note: Scale&lt;/h2&gt;&#xA;&lt;p&gt;The scale for ligand space exploration is generally several orders of magnitude higher than &amp;quot;hundreds&amp;quot;: &lt;a href=&quot;https://zinc.docking.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zinc DB&lt;/a&gt; lists 750 million enumerated compounds, &lt;a href=&quot;http://gdb.unibe.ch/downloads/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GDB13&lt;/a&gt; has enumerate all possible compounds up to 13 atoms (970 million), &lt;a href=&quot;https://enamine.net/library-synthesis/real-compounds/real-database&quot; rel=&quot;nofollow noreferrer&quot;&gt;Enamine Real&lt;/a&gt; is 1.2 billion etc. In an actual experiment, one would use a more restrictive set. As an example of a real project, in &lt;a href=&quot;https://covid.postera.ai/covid&quot; rel=&quot;nofollow noreferrer&quot;&gt;Covid19 moonshot project&lt;/a&gt; there are 2,800 user suggested follow-ups.&lt;/p&gt;&#xA;&lt;h2&gt;Side note: More than just binding&lt;/h2&gt;&#xA;&lt;p&gt;There are a few requirements for a drug to be effective, and binding is only one of them:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Must bind...&lt;/li&gt;&#xA;&lt;li&gt;Must follow &lt;a href=&quot;https://en.wikipedia.org/wiki/Lipinski%27s_rule_of_five&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lipinksi's rule of five&lt;/a&gt;, i.e. it must be membrane permeable and small.&lt;/li&gt;&#xA;&lt;li&gt;Must be synthesisable. Whereas the experience of someone learnèd is by far the best way to tell how an expansion can be done, there are several machine leaning approaches that can filter a huge dataset. IBM actually have a &lt;a href=&quot;https://rxn.res.ibm.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;fun online tool&lt;/a&gt; —.&lt;/li&gt;&#xA;&lt;li&gt;Must not have side effects&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So pre-filtering the list of compounds, regardless of docked poses, will happened, be it manually, by cut-offs or by advanced machine learning.&lt;/p&gt;&#xA;&lt;h2&gt;CPU time: docking vs. machine learning&lt;/h2&gt;&#xA;&lt;p&gt;These various steps are generally followed in that order.&#xA;Calculating properties (MW, logP etc.) for a given SMILES string to filter based on Lipinski's rule takes a fraction of a second, so that is generally always a first step.&lt;/p&gt;&#xA;&lt;p&gt;Whether the compound can synthesised or bought is a general left last and done manually. Pharmacology is complicated so is ignored.&lt;/p&gt;&#xA;&lt;p&gt;Docking a ligand takes time and different approaches have different time scales. A decent algorithm that uses implicit solvent model takes about a minute per core. A short MD simulation takes an 1 hour per core or more. There are docking programs for screening that can be faster but these dock only one conformer of the small molecule against a rigid protein and the results are not of much value. Also setting up docking properly is not easy (cf. &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/11471/protein-ligand-docking-how-to-convert-protein-pdb-to-protein-maps-fld?rq=1&quot;&gt;steps required for Autodock 4&lt;/a&gt;, each with their own caveats). So to recap what needs to be considered that results in the slowness:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Hundreds of conformers of the ligand&lt;/li&gt;&#xA;&lt;li&gt;Some flexibility of the protein, side-chains repacking only or even backbone changes.&lt;/li&gt;&#xA;&lt;li&gt;thousands of poses explored.&lt;/li&gt;&#xA;&lt;li&gt;Optionally. Implicit water is not as good as a TIP3 water model&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Machine learning have a much smaller time requirement and is way more attractive for publications. Generally these work by using ligand only information, such as properties, splitting it into pharmacophores, colour etc. This requires a decent dataset, such as empirical results, say from a fragment screen, what compounds bound and which didn't. Often there is a lot of hype with machine learning and the result is something close to something already seen, as well put in &lt;a href=&quot;http://practicalcheminformatics.blogspot.com/2019/09/dissecting-hype-with-cheminformatics.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog post by Pat Walters&lt;/a&gt;. This is a side effect of the starting data. The &lt;a href=&quot;https://www.cell.com/cell/pdf/S0092-8674(20)30102-1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;recent Stokes et al. 2020 Cell paper&lt;/a&gt; uses AI perfectly because of their staggering automated empirical data collection pipeline.&lt;/p&gt;&#xA;&lt;h2&gt;ML after docking&lt;/h2&gt;&#xA;&lt;p&gt;However, ML can actually be used after docking to improve the quality of the score.&#xA;The &lt;a href=&quot;https://academic.oup.com/bioinformatics/article-abstract/36/3/758/5554651&quot; rel=&quot;nofollow noreferrer&quot;&gt;Boyles, Deane, Morris 2019 Bioinformatics paper&lt;/a&gt; investigated ligand-only machine learning to see why do ML methods that rely solely on ligand somehow often work. It is a good paper from a sceptical computational biochemistry viewpoint.&#xA;The scorefunction used in docking by the sampler can be classified as forcefield-based, empirical or hybrid. In some cases empirical cases, many terms are ligand based properties (/features) that would normally go into a ML search (say one trained on failed hits and successful hits).&#xA;For example Autodock Vina (not 4) has regression derived factors in the final weights of the score&lt;/p&gt;&#xA;&lt;h2&gt;Structural features in machine learning: HotSpots&lt;/h2&gt;&#xA;&lt;p&gt;Information of the protein active site can be used in ranking compounds in a non-docking machine learning approach. Namely, finding out what kind of chemical groups bind preferentially to different parts of the protein's active site, &lt;em&gt;i.e.&lt;/em&gt; making a &amp;quot;fragment hotspot map&amp;quot;. Specifically, a hotspot map allows the determination of a pharmacophore, which is an abstract generalisation of a group of molecules with both both electronic and steric features —a coarse-grain model, if you will. This approach is also used to detect where the active site actually is in the extreme cases where it is not know (primarily a problem for pipelines, not humans). There are several implementations as described &lt;a href=&quot;https://pubs.acs.org/doi/10.1021/acs.jmedchem.5b01980&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, the first being FTMap (&lt;a href=&quot;http://ftmap.bu.edu/login.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;site&lt;/a&gt;, &lt;a href=&quot;https://www.nature.com/articles/nprot.2015.043&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;), which docks 16 different types of compounds. Waters can play a role and severely complicate matters, hence approaches can be taken to counter that. However, unlike docking protein rigidity is not a big nuisance: this is because a hotspot map is most often used for filtering/weighing by the &amp;quot;color&amp;quot; not the shape of the molecules. But shape based searching is possible.&lt;/p&gt;&#xA;&lt;h3&gt;Shape based scoring&lt;/h3&gt;&#xA;&lt;p&gt;In terms of shape-based searching, this is actually a question of having pre-computed conformers, which can be problematic (but can and is often done on the fly), and shape based scoring —for which there are different metrics, such as the classical RMSD to more complexly weighted scores such as SuCOS.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2022-08-08T09:19:01.427" LastActivityDate="2022-08-08T09:19:01.427" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="12976" PostTypeId="1" AcceptedAnswerId="12986" CreationDate="2020-04-19T17:03:44.453" Score="1" ViewCount="102" Body="&lt;p&gt;I have a boolean matrix of mutated or non mutated genes across some samples&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(muts)&#xA;                  TP53 CDKN2A TSHZ3 ARID1A EPHA3 NOTCH1 ACVR2A ARID1B ARID2&#xA;LP6008336-DNA_G01    1      1     1      0     0      0      0      0     0&#xA;LP6008460-DNA_D01    1      1     0      1     0      0      0      0     0&#xA;LP6005500-DNA_D03    1      1     0      0     0      0      0      0     0&#xA;LP6008334-DNA_B02    1      0     1      0     0      0      0      0     0&#xA;LP6008334-DNA_A04    1      0     1      0     0      0      0      0     0&#xA;LP6005334-DNA_H01    1      0     0      0     1      0      0      1     0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;By this code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pw_rt = data.frame(&#xA;  gene = c('PIK3CA', 'EGFR', 'ERBB2', 'PTEN'),&#xA;  pw = c('RTK'),&#xA;  stringsAsFactors = F&#xA;)&#xA;&#xA;pw_cc = data.frame(&#xA;  gene = c('CDKN2A', 'CCNE1'),&#xA;  pw = c('Cell Cycle'),&#xA;  stringsAsFactors = F&#xA;)&#xA;&#xA;pw = rbind(pw_rt, pw_cc)&#xA;&#xA;pw_col = RColorBrewer::brewer.pal(n = 3, name = 'Set1')[1:2]&#xA;names(pw_col) = unique(pw$pw)&#xA;&#xA;require(dplyr)&#xA;&#xA;lmuts = reshape2::melt(muts) %&amp;gt;% as_tibble() &#xA;colnames(lmuts) = c('sample', 'gene', 'value')&#xA;&#xA;pw_n = pw&lt;span class=&quot;math-container&quot;&gt;$pw&#xA;names(pw_n) = pw$&lt;/span&gt;gene&#xA;&#xA;Np = nrow(muts)&#xA;&#xA;lmuts %&amp;gt;% &#xA;  mutate(gene = paste(gene), PW = pw_n[gene]) %&amp;gt;%&#xA;  group_by(gene, PW) %&amp;gt;%&#xA;  summarise(N = sum(value == 1)) %&amp;gt;%&#xA;  ungroup() %&amp;gt;%&#xA;  mutate(&#xA;    PW = ifelse(is.na(PW), &quot;None&quot;, PW), &#xA;    N = paste0(N, ' (', round(N/Np * 100, 1), '%)')&#xA;    ) %&amp;gt;%&#xA;  ggplot(aes(x = PW, y = gene, fill = PW)) +&#xA;  geom_tile() +&#xA;  geom_text(aes(label = N)) +&#xA;  theme_light() +&#xA;  theme(legend.position = 'bottom') +&#xA;  guides(fill = guide_legend('Pathway')) +&#xA;  scale_fill_manual(values = c(pw_col, `None` = 'gainsboro')) +&#xA;  scale_x_discrete(limits = c(names(pw_col), 'None')) +&#xA;  labs(&#xA;    x = &quot;&quot;,&#xA;    y = 'Gene',&#xA;    title = &quot;Occurrence of mutations&quot;&#xA;  )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am able to get proportion of altered pathway by these genes like &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kqZ7O.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kqZ7O.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I want to highlight oncogenes and tumour suppressors within this plot &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance TP53 is a tumour suppressor and CCNE1 is an oncogene&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I don't know how to manipulate the above code to highlight or partition genes to tumour suppressor or oncogene&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you help?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDITED&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have add extra column to lmuts like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(lmuts)&#xA;             sample gene value gene_class&#xA;1 LP6005690-DNA_H02 TP53     1        TSG&#xA;2 LP2000333-DNA_A01 TP53     1        TSG&#xA;3 LP6005409-DNA_D03 TP53     1        TSG&#xA;4 LP6008141-DNA_H02 TP53     1        TSG&#xA;5 LP6008336-DNA_E02 TP53     1        TSG&#xA;6 LP6008269-DNA_B06 TP53     1        TSG&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; lmuts %&amp;gt;% &#xA;+     mutate(gene = paste(gene), PW = pw_n[gene]) %&amp;gt;%&#xA;+     group_by(gene, PW) %&amp;gt;%&#xA;+     summarise(N = sum(value == 1)) %&amp;gt;%&#xA;+     ungroup() %&amp;gt;%&#xA;+     mutate(&#xA;+         PW = ifelse(is.na(PW), &quot;None&quot;, PW), &#xA;+         N = paste0(N, ' (', round(N/Np * 100, 1), '%)')&#xA;+     ) %&amp;gt;%&#xA;+     ggplot(aes(x = PW, y = gene, fill = PW)) +&#xA;+     geom_tile() +&#xA;+     geom_text(aes(label = N)) +&#xA;+     theme_light() +&#xA;+     theme(legend.position = 'bottom') +&#xA;+     guides(fill = guide_legend('Pathway')) +&#xA;+     scale_fill_manual(values = c(pw_col, `None` = 'gainsboro')) +&#xA;+     scale_x_discrete(limits = c(names(pw_col), 'None')) +&#xA;+     labs(&#xA;+         x = &quot;&quot;,&#xA;+         y = 'Gene',&#xA;+         title = &quot;Occurrence of mutations&quot;&#xA;+     )+facet_wrap(~gene_class, ncol=1)&#xA;Error: At least one layer must contain all faceting variables: `gene_class`.&#xA;* Plot is missing `gene_class`&#xA;* Layer 1 is missing `gene_class`&#xA;* Layer 2 is missing `gene_class`&#xA;Run `rlang::last_error()` to see where the error occurred.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-04-20T07:36:56.163" LastActivityDate="2020-04-20T07:36:56.163" Title="Grouping a set of genes in a ggplot object" Tags="|r|ggplot2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12977" PostTypeId="2" ParentId="12966" CreationDate="2020-04-19T18:25:24.823" Score="2" Body="&lt;p&gt;Excellent question! I would recommend that you start with an NIH resource at the National Library of Medicine (NLM). PubMed (&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pubmed.ncbi.nlm.nih.gov/&lt;/a&gt;) provides search capabilities across the biomedical literature. You can also create an account and use it to manage your searches of the literature. An example search below for &quot;protein misfolding disease&quot; in PubMed retrieved almost 50,000 hits. You can filter to make the results more specific to your interests. You can view the citations, including the abstracts, and often links to the full text. Since you mention that you are on the &quot;computational/biophysics side,&quot; you might be interested in the PubMed API (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/tools/developers/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/tools/developers/&lt;/a&gt;) and python modules that use the API, such as Biopython. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;E-utilities:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK25501/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/books/NBK25501/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Biopython:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://biopython.org/DIST/docs/api/Bio.Entrez-module.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biopython.org/DIST/docs/api/Bio.Entrez-module.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;PubMed Search:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/?term=protein%20misfolding%20disease&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pubmed.ncbi.nlm.nih.gov/?term=protein%20misfolding%20disease&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Specific Articles: &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;a href=&quot;https://www.nature.com/scitable/topicpage/protein-misfolding-and-degenerative-diseases-14434929/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/scitable/topicpage/protein-misfolding-and-degenerative-diseases-14434929/&lt;/a&gt;&#xA;  &lt;a href=&quot;https://www.nature.com/articles/nrn1007&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/nrn1007&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="8653" LastActivityDate="2020-04-19T18:25:24.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12978" PostTypeId="2" ParentId="4748" CreationDate="2020-04-19T18:34:18.857" Score="1" Body="&lt;p&gt;I have recently come across this submission from &lt;a href=&quot;https://arxiv.org/abs/2003.11817&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wang et al. (2020)&lt;/a&gt;, which is a method using the tool &lt;a href=&quot;https://github.com/fanagislab/kmerfreq&quot; rel=&quot;nofollow noreferrer&quot;&gt;kmerfreq&lt;/a&gt; on error-corrected long reads (e.g. generated using Canu, FLAS, or LoRMA). It looks to be a promising approach.&lt;/p&gt;&#xA;" OwnerUserId="2893" LastActivityDate="2020-04-19T18:34:18.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12979" PostTypeId="1" AcceptedAnswerId="12983" CreationDate="2020-04-19T20:09:06.997" Score="0" ViewCount="452" Body="&lt;p&gt;I am trying to build a HISAT2 index using Pertea et al. procedure (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/27560171&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pubmed/27560171&lt;/a&gt;) using their &lt;code&gt;extract_splice_sites.py&lt;/code&gt; and &lt;code&gt;extract_exons.py&lt;/code&gt; pieces of code. For that, I need a &lt;code&gt;.gtf&lt;/code&gt; file of Human Genome HG38. I have obtained this file using recommendations described here &lt;a href=&quot;http://www.metagenomics.wiki/tools/fastq/ncbi-ftp-genome-download&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.metagenomics.wiki/tools/fastq/ncbi-ftp-genome-download&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The file I have downloaded is &lt;code&gt;GCF_000001405.39_GRCh38.p13_genomic.gff&lt;/code&gt; (1,1 Gb) and when I use on it this command &lt;code&gt;python3 extract_splice_sites.py GCF_000001405.39_GRCh38.p13_genomic.gff &amp;gt; test.ss&lt;/code&gt; my &lt;code&gt;test.ss&lt;/code&gt; file is 0 Kb! I wonder if there's another &lt;code&gt;.gff&lt;/code&gt; file that I must use and where I can get it? This is my first time working with the human genome so I am not sure if I am doing the procedure the right way.&lt;/p&gt;&#xA;" OwnerUserId="6948" LastActivityDate="2020-04-21T20:58:25.367" Title="A .gtf file for HG38 for HISAT2 indexing" Tags="|python|gtf|human-genome|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12981" PostTypeId="1" CreationDate="2020-04-19T21:29:40.910" Score="0" ViewCount="2560" Body="&lt;p&gt;I am interested in the analysis of single-cell RNA-sequencing data.&#xA;I installed Cell Ranger according to the instruction (&lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/tutorial_in#download&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/tutorial_in#download&lt;/a&gt;).&#xA;I know, it should be easy, but I found an error occurring after test running as below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[error] Pipestance failed. Error log at:&#xA;tiny/SC_RNA_COUNTER_CS/SC_RNA_COUNTER/_BASIC_SC_RNA_COUNTER/COUNT_GENES/fork0/chnk0-u6c389ca928/_errors&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Within that file, it shows the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Job failed in stage code. signal: killed.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, what do you think about this problem?&lt;/p&gt;&#xA;&lt;p&gt;Additional information:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a checksum test (to check the integrity of the downloaded files) was successful&lt;/li&gt;&#xA;&lt;li&gt;I performed a test run on a personal laptop, mac.&lt;/li&gt;&#xA;&lt;li&gt;I used SSH command to connect the server, but we are sharing the server with our lab members.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I found an error as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;WARNING: configured to use 112GB of local memory, but only 80.5GB is currently available.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, at this time with unknown reasons, the installation is successfully completed, in spite of the warning. Do you have idea?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="8657" LastEditorUserId="73" LastEditDate="2021-07-14T11:00:47.370" LastActivityDate="2022-06-12T23:03:20.370" Title="Cell Ranger) Error during performing a test run" Tags="|scrnaseq|software-installation|cellranger|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12983" PostTypeId="2" ParentId="12979" CreationDate="2020-04-20T00:50:16.947" Score="0" Body="&lt;p&gt;You can download a GTF file from NCBI now. For hg38 (also known as GRCh38), just go to &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Assembly&lt;/a&gt; and search for 'GRCh38'. Click on the first result to go to &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly/GCF_000001405.39&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt; and use the blue 'Download Assemblies' button to download data. Here, after you choose 'RefSeq' as your source of data, you will see a GTF file to download from the drop-down list, as shown in the screenshot below: &#xA;&lt;a href=&quot;https://i.stack.imgur.com/iye4v.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iye4v.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2020-04-20T00:50:16.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12984" PostTypeId="1" AcceptedAnswerId="12987" CreationDate="2020-04-20T02:16:35.427" Score="2" ViewCount="47" Body="&lt;p&gt;I'm browsing through ribosomes which have been modeled with Cryoem on PDB and am pretty confused by the fact that for some, PDB, or rather the authors of the deposition probably, put multiple organisms from different domains of life... How does one interpret that?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far as i understand &lt;em&gt;bos taurus&lt;/em&gt; is a cow and &lt;em&gt;e.coli&lt;/em&gt; is a bacterium. So eukarya and bacteria.. Thank you.&lt;a href=&quot;https://i.stack.imgur.com/tWXD0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tWXD0.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6511" LastActivityDate="2020-04-20T08:46:34.590" Title="What does it mean when PDB lists multiple organisms for the same structure?" Tags="|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12985" PostTypeId="2" ParentId="12976" CreationDate="2020-04-20T03:47:06.113" Score="0" Body="&lt;p&gt;I can't reproduce your graph, because you don't make &lt;code&gt;muts&lt;/code&gt; available, but it appears your problem is having &lt;code&gt;x = PW&lt;/code&gt; in your ggplot call. Don't you want to that to be whatever your variable name for sample is (ie, the rownames in your &lt;code&gt;but&lt;/code&gt; dataframe you have at the beginning). &lt;/p&gt;&#xA;" OwnerUserId="1044" LastActivityDate="2020-04-20T03:47:06.113" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12986" PostTypeId="2" ParentId="12976" CreationDate="2020-04-20T05:05:44.477" Score="3" Body="&lt;p&gt;If you had the classification into oncogene/TSG for each gene in a column (let's call it &lt;code&gt;gene_class&lt;/code&gt;) in &lt;code&gt;lmuts&lt;/code&gt;, you could split both groups into facets within the plot by adding:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;+ facet_wrap(~gene_class, ncol=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8629" LastActivityDate="2020-04-20T05:05:44.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12987" PostTypeId="2" ParentId="12984" CreationDate="2020-04-20T08:46:34.590" Score="3" Body="&lt;p&gt;Since this is a 40S ribosomal protein, and therefore eukaryotic-specific (as the description says) the listing of &lt;em&gt;E. coli&lt;/em&gt; is not intuitive as you note. If you go down to &quot;Macromolecules&quot; and click on &quot;Nucleic Acids/Hybrids&quot; you can see the listed &lt;em&gt;E. coli&lt;/em&gt; sequence is just 28bp, and all &lt;code&gt;U&lt;/code&gt;s (poly(U) tail?). So, you can safely ignore the bacterial annotation I think.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2020-04-20T08:46:34.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12988" PostTypeId="1" AcceptedAnswerId="13003" CreationDate="2020-04-20T12:22:42.163" Score="2" ViewCount="82" Body="&lt;p&gt;I want to pass a series of files (e.g., &lt;code&gt;a&lt;/code&gt;, &lt;code&gt;b&lt;/code&gt;) to a program &lt;code&gt;prog&lt;/code&gt; like: &lt;code&gt;prog a,b&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to do this in Snakemake?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My attempt is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule run_prog:&#xA;  input: &quot;a&quot;, &quot;b&quot;&#xA;  output: &quot;output&quot;&#xA;  run: &quot;prog {input}&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But the actual command Snakemake generated is &lt;code&gt;prog a b&lt;/code&gt; instead of &lt;code&gt;prog a,b&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any solution to pass a comma concatenated file names to Snakemake?&lt;/p&gt;&#xA;" OwnerUserId="8660" LastActivityDate="2020-04-22T09:28:53.430" Title="How to concatename file names in Snakemake?" Tags="|snakemake|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12989" PostTypeId="2" ParentId="12970" CreationDate="2020-04-20T12:26:25.167" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;You need to report allele frequencies using &lt;code&gt;plink&lt;/code&gt; for some European samples&lt;/li&gt;&#xA;&lt;li&gt;Do the same for Hispanic samples&lt;/li&gt;&#xA;&lt;li&gt;Use a software to subtract the two allele frequencies obtained above and obtain the result.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="8660" LastActivityDate="2020-04-20T12:26:25.167" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="12990" PostTypeId="1" AcceptedAnswerId="12991" CreationDate="2020-04-20T13:30:54.617" Score="1" ViewCount="439" Body="&lt;p&gt;To call minority variants in my Mtb sequences I'm using a pipeline of &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fastq -&amp;gt; &#xA;    BWA MEM for mapping -&amp;gt; &#xA;    picard MarkDuplicates -&amp;gt; &#xA;    GATK3 for indel realignment and base recalibration -&amp;gt;  &#xA;    lofreq for variant calling (https://csb5.github.io/lofreq/)   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However the resulting VCF files don't have an MQ tag for me to filter on. Where in this process would mapping quality information be generated or lost so that it doesn't end up in the VCF file?&lt;/p&gt;&#xA;" OwnerUserId="1152" LastEditorUserId="3848" LastEditDate="2020-04-22T21:35:25.023" LastActivityDate="2020-04-22T21:35:25.023" Title="No MQ tags in VCF files" Tags="|ngs|variant-calling|bwa|gatk|bowtie2|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12991" PostTypeId="2" ParentId="12990" CreationDate="2020-04-20T15:23:14.877" Score="2" Body="&lt;p&gt;As @ATpoint explains in his comment, the mapping quality (&lt;code&gt;MQ&lt;/code&gt;) applies to read mappings and, hence, is contained in BAM files, but not VCFs. The &lt;code&gt;MQ&lt;/code&gt; values carry important information that ideally the variant caller should use when determining the variant quality (&lt;code&gt;QUAL&lt;/code&gt; in the VCF) of individual variant loci.&lt;/p&gt;&#xA;&lt;p&gt;In the &lt;a href=&quot;https://csb5.github.io/lofreq/faq/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LoFreq documentation&lt;/a&gt; it says that&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;h1&gt;Choice of mapper: Bowtie and BWA-SW&lt;/h1&gt;&#xA;&lt;p&gt;BWA-SW assigns very low mapping qualities to mapped reads. &lt;strong&gt;LoFreq incorporates mapping quality by default into its variant calling model&lt;/strong&gt;, which doesn’t help in this scenario and will reduce its sensitivity. Consider disabling the use of mapping quality for &lt;code&gt;lofreq call&lt;/code&gt; with &lt;code&gt;-N&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So LoFreq is already aware of the &lt;code&gt;MQ&lt;/code&gt; values and takes them into consideration during variant calling.&lt;/p&gt;&#xA;&lt;p&gt;Since you are using BWA-MEM (and not BWA-SW) you should probably not be concerned about the issue they mention and trust their quality filtering.&lt;/p&gt;&#xA;&lt;p&gt;If you are sceptic about the quality of mappings, though, you could still filter them out directly after mapping, e.g. using &lt;code&gt;samtools view -q 20 ...&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="8629" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2020-04-20T15:34:01.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12992" PostTypeId="2" ParentId="12965" CreationDate="2020-04-20T16:50:48.877" Score="1" Body="&lt;p&gt;I was able to extract the graph features that I wanted just using networkx, without going through the intermediate step of Cytoscape. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: Initially, I planned to import the data into Cytoscape and extract graph features from Cytoscape. However, I noticed that a much easier solution would be to just use the methods of the networkx library. I initially had a few graph features in mind that I wanted to extract like k-core value of each node, and I was able to do this using methods from &lt;a href=&quot;https://networkx.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;networkX&lt;/a&gt;. To calculate k-core for instance, I used the &lt;a href=&quot;https://networkx.github.io/documentation/networkx-1.7/reference/generated/networkx.algorithms.core.k_core.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;k_core&lt;/a&gt; method. For more information on networkX methods, check the &lt;a href=&quot;https://networkx.github.io/documentation/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;docs&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="8601" LastEditDate="2020-04-21T04:10:05.470" LastActivityDate="2020-04-21T04:10:05.470" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="12994" PostTypeId="2" ParentId="12972" CreationDate="2020-04-20T17:36:25.230" Score="2" Body="&lt;p&gt;I think that you are dealing with two different phenomena:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1) TLEN == 0 reads&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the BAM spec:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;TLEN: signed observed Template LENgth. If all segments are mapped to the same reference sequence, the absolute value of TLEN equals the distance between the mapped end of the template and the mapped start of the template, inclusively (i.e., end − start + 1).14 Note that mapped base is defined to be one that aligns to the reference as described by CIGAR, hence excludes soft-clipped bases. The TLEN field is positive for the leftmost segment of the template, negative for the rightmost, and the sign for any middle segment is undefined. If segments cover the same coordinates then the choice of which is leftmost and rightmost is arbitrary, but the two ends must still have differing signs. &lt;strong&gt;It is set as 0 for a single-segment template or when the information is unavailable (e.g., when the first or last segment of a multi-segment template is unmapped or when the two are mapped to different reference sequences).&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Note that in each case the mate read appears to be mapped to a different chromosome/contig. So, the mapper can't estimate insert size in these cases, so it just says zero.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2. Multiple reads with same qname&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are various reasons why the same qname can appear multiple times. Again from the BAM spec, one such explanation is that it is a chimeric alignment:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;strong&gt;Chimeric alignment:&lt;/strong&gt; An alignment of a read that cannot be represented as a linear alignment. A chimeric alignment is represented as a set of linear alignments that do not have large overlaps. Typically, one of the linear alignments in a chimeric alignment is considered the “representative” alignment, and the others are called “supplementary” and are distinguished by the supplementary alignment flag. All the SAM records in a chimeric alignment have the same QNAME and the same values for 0x40 and 0x80 flags (see Section 1.4). The decision regarding which linear alignment is representative is arbitrary.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;You may have to do some digging and looking at the different flags to decide what exactly is going on, but my bet is that they are supplementary alignments.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-04-20T17:36:25.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12995" PostTypeId="1" CreationDate="2020-04-20T21:25:34.913" Score="0" ViewCount="2689" Body="&lt;p&gt;I want to upload an excel file sheet that has certain barcodes that I would like to show on my umap. How do I go about adding the file and linking it to the metadata?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Below is my following code.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# set up the working directory&#xA;wd = &quot;/home/PTX_AAC656&#xA;setwd(wd)&#xA;&#xA;# load counts&#xA;# SD-52-JZ33 patient sample but aligned to both mouse and human&#xA;SD_52_JZ33.data &amp;lt;- Read10X(data.dir = &quot;../PTX_AAC656/metadata/patient_sample/SD-52-JZ33/outs/filtered_feature_bc_matrix/&quot;)&#xA;BNM_human_only.data &amp;lt;- Read10X(data.dir = &quot;../PTX_AAC656/metadata/human_only_alignment/SD-52-JZ33_BNM_5K/outs/filtered_feature_bc_matrix/&quot;)&#xA;BNL_human_only.data &amp;lt;- Read10X(data.dir = &quot;../PTX_AAC656/metadata/human_only_alignment/SD-52-JZ33_BNL_5K/outs/filtered_feature_bc_matrix/&quot;)&#xA;BNL_mouse_only.data &amp;lt;- Read10X(data.dir = &quot;../PTX_AAC656/metadata/mouse_only_alignment/SD-52-JZ33_BNL_5K/outs/filtered_feature_bc_matrix/&quot;)&#xA;BNM_mouse_only.data &amp;lt;- Read10X(data.dir = &quot;../PTX_AAC656/metadata/mouse_only_alignment/SD-52-JZ33_BNM_5K/outs/filtered_feature_bc_matrix/&quot;)&#xA;&#xA;&#xA;   BNL_both.data &amp;lt;- Read10X(data.dir = &quot;../AAC656/SD-52-JZ33_BNL_5K/outs/filtered_feature_bc_matrix/&quot;)&#xA;    BNM_both.data &amp;lt;- Read10X(data.dir = &quot;../AAC656/SD-52-JZ33_BNM_5K/outs/filtered_feature_bc_matrix/&quot;)&#xA;&#xA;setwd(wd)&#xA;&#xA;# create the Seurat objects&#xA;SD_52_JZ33 &amp;lt;- CreateSeuratObject(counts = SD_52_JZ33.data, min.cells = 0, project = &quot;human&quot;)&#xA;BNL_human_only &amp;lt;- CreateSeuratObject(counts = BNL_human_only.data, min.cells = 0, project = &quot;human_alignment_ptx&quot;)&#xA;BNM_human_only &amp;lt;- CreateSeuratObject(counts = BNM_human_only.data, min.cells = 0, project = &quot;human_alignment_ptx&quot;)&#xA;BNM_both_ptx &amp;lt;- CreateSeuratObject(counts = BNM_both.data, min.cells = 0, project = &quot;human_mouse_alignment_ptx&quot;)&#xA;BNL_both_ptx &amp;lt;- CreateSeuratObject(counts = BNL_both.data, min.cells = 0, project = &quot;human_mouse_alignment_ptx&quot;)&#xA;BNL_both_ptx&lt;span class=&quot;math-container&quot;&gt;$model = &quot;mouse_human_alignment_ptx&quot;&#xA;BNM_both_ptx$&lt;/span&gt;model = &quot;mouse_human_alignment_ptx&quot;&#xA;SD_52_JZ33&lt;span class=&quot;math-container&quot;&gt;$model = &quot;patient_aligned_mouse_human&quot;&#xA;BNM_human_only$&lt;/span&gt;model = &quot;human_alignment_only_ptx&quot;&#xA;BNL_human_only&lt;span class=&quot;math-container&quot;&gt;$model = &quot;human_alignment_only_ptx&quot;&#xA;BNL_both_ptx$&lt;/span&gt;model = &quot;mouse_human_alignment_ptx&quot;&#xA;BNM_both_ptx&lt;span class=&quot;math-container&quot;&gt;$model = &quot;mouse_human_alignment_ptx&quot;&#xA;BNM_mouse_only &amp;lt;- CreateSeuratObject(counts = BNM_mouse_only.data, min.cells = 0, project = &quot;mouse_alignment_ptx&quot;)&#xA;BNL_mouse_only &amp;lt;- CreateSeuratObject(counts = BNL_mouse_only.data, min.cells = 0, project = &quot;mouse_alignment_ptx&quot;)&#xA;BNL_mouse_only$&lt;/span&gt;model = &quot;mouse_alignment_only_ptx&quot;&#xA;BNM_mouse_only$model = &quot;mouse_alignment_only_ptx&quot;&#xA;&#xA;# merge samples&#xA;mouse_human_patient &amp;lt;- merge(BNL_mouse_only, y = c(BNM_mouse_only, SD_52_JZ33), add.cell.ids = c(&quot;A&quot;, &quot;B&quot;, &quot;C&quot;), project = &quot;HTB2876&quot;)&#xA;human_ptx_human_patient &amp;lt;- merge(BNL_human_only, y = c(BNM_human_only, SD_52_JZ33), add.cell.ids = c(&quot;D&quot;, &quot;E&quot;, &quot;F&quot;), project = &quot;HTB2876&quot;)&#xA;ptx_human_patient.1 &amp;lt;- merge(BNL_both_ptx, y = c(BNM_both_ptx, SD_52_JZ33), add.cell.ids = c(&quot;G&quot;, &quot;H&quot;, &quot;I&quot;), project = &quot;HTB2876&quot;)&#xA;&#xA;# mouse ptx model with patient&#xA;# mark the mito genes&#xA;mito.genes &amp;lt;- grep(pattern = &quot;^MT-&quot;, x = rownames(x = mouse_human_patient), value = TRUE)&#xA;mouse_human_patient[[&quot;percent.mt&quot;]] &amp;lt;- PercentageFeatureSet(mouse_human_patient, pattern = &quot;^MT-&quot;)&#xA;mouse_human_patient[[&quot;log_nCount_RNA&quot;]] &amp;lt;- log2(mouse_human_patient[[&quot;nCount_RNA&quot;]]+1)&#xA;&#xA;# remove cells with &amp;lt;200 RNA molecules, or &amp;gt;6000 molecules, or &amp;gt;30% mito&#xA;mouse_human_patient &amp;lt;- subset(mouse_human_patient, subset = nFeature_RNA &amp;gt; 200 &amp;amp; nFeature_RNA &amp;lt; 6000 &amp;amp; percent.mt &amp;lt; 30)&#xA;mouse_human_patient &amp;lt;- NormalizeData(mouse_human_patient, normalization.method = &quot;LogNormalize&quot;, scale.factor = 10000)&#xA;mouse_human_patient &amp;lt;- FindVariableFeatures(mouse_human_patient, selection.method = &quot;vst&quot;, nfeatures = 2000)&#xA;all.genes.mouse &amp;lt;- rownames(mouse_human_patient)&#xA;mouse_human_patient &amp;lt;- ScaleData(mouse_human_patient, features = VariableFeatures(object = mouse_human_patient), vars.to.regress = c(&quot;nCount_RNA&quot;))&#xA;mouse_human_patient &amp;lt;- RunPCA(mouse_human_patient, features = VariableFeatures(object = mouse_human_patient))&#xA;mouse_human_patient &amp;lt;- FindNeighbors(mouse_human_patient, dims = 1:20)&#xA;mouse_human_patient &amp;lt;- FindClusters(mouse_human_patient, resolution = 0.4)&#xA;mouse_human_patient &amp;lt;- RunUMAP(mouse_human_patient, dims = 1:20)&#xA;DimPlot(mouse_human_patient, reduction = &quot;umap&quot;, group.by = &quot;model&quot;)&#xA;&#xA;# human only alignment pix with patient &#xA;# mark the mito genes&#xA;mito.genes &amp;lt;- grep(pattern = &quot;^MT-&quot;, x = rownames(x = human_ptx_human_patient), value = TRUE)&#xA;human_ptx_human_patient[[&quot;percent.mt&quot;]] &amp;lt;- PercentageFeatureSet(human_ptx_human_patient, pattern = &quot;^MT-&quot;)&#xA;human_ptx_human_patient[[&quot;log_nCount_RNA&quot;]] &amp;lt;- log2(human_ptx_human_patient[[&quot;nCount_RNA&quot;]]+1)&#xA;human_ptx_human_patient &amp;lt;- subset(human_ptx_human_patient, subset = nFeature_RNA &amp;gt; 200 &amp;amp; nFeature_RNA &amp;lt; 6000 &amp;amp; percent.mt &amp;lt; 30)&#xA;&#xA;# remove cells with &amp;lt;200 RNA molecules, or &amp;gt;6000 molecules, or &amp;gt;30% mito&#xA;human_ptx_human_patient &amp;lt;- NormalizeData(human_ptx_human_patient, normalization.method = &quot;LogNormalize&quot;, scale.factor = 10000)&#xA;human_ptx_human_patient &amp;lt;- FindVariableFeatures(human_ptx_human_patient, selection.method = &quot;vst&quot;, nfeatures = 2000)&#xA;all.genes.human.ptx &amp;lt;- rownames(human_ptx_human_patient)&#xA;human_ptx_human_patient &amp;lt;- ScaleData(human_ptx_human_patient, features = VariableFeatures(object = human_ptx_human_patient), vars.to.regress = c(&quot;nCount_RNA&quot;))&#xA;human_ptx_human_patient &amp;lt;- RunPCA(human_ptx_human_patient, features = VariableFeatures(object = human_ptx_human_patient))&#xA;DimPlot(human_ptx_human_patient, reduction = &quot;pca&quot;)&#xA;human_ptx_human_patient &amp;lt;- FindNeighbors(human_ptx_human_patient, dims = 1:20)&#xA;human_ptx_human_patient &amp;lt;- FindClusters(human_ptx_human_patient, resolution = 0.4)&#xA;human_ptx_human_patient &amp;lt;- RunUMAP(human_ptx_human_patient, dims = 1:20)&#xA;DimPlot(human_ptx_human_patient, reduction = &quot;umap&quot;, group.by = &quot;model&quot;)&#xA;&#xA;# ptx and human&#xA;# mark the mito genes&#xA;mito.genes.ptx_patient &amp;lt;- grep(pattern = &quot;^MT-&quot;, x = rownames(x = ptx_human_patient.1), value = TRUE)&#xA;ptx_human_patient.1[[&quot;percent.mt&quot;]] &amp;lt;- PercentageFeatureSet(ptx_human_patient.1, pattern = &quot;^MT-&quot;)&#xA;ptx_human_patient.1[[&quot;log_nCount_RNA&quot;]] &amp;lt;- log2(ptx_human_patient.1[[&quot;nCount_RNA&quot;]]+1)&#xA;&#xA;# remove cells with &amp;lt;200 RNA molecules, or &amp;gt;6000 molecules, or &amp;gt;30% mito&#xA;ptx_human_patient.1 &amp;lt;- subset(ptx_human_patient.1, subset = nFeature_RNA &amp;gt; 200 &amp;amp; nFeature_RNA &amp;lt; 6000 &amp;amp; percent.mt &amp;lt; 30)&#xA;ptx_human_patient.1 &amp;lt;- NormalizeData(ptx_human_patient.1, normalization.method = &quot;LogNormalize&quot;, scale.factor = 10000)&#xA;ptx_human_patient.1 &amp;lt;- FindVariableFeatures(ptx_human_patient.1, selection.method = &quot;vst&quot;, nfeatures = 2000)&#xA;all.genes.both.patient.1 &amp;lt;- rownames(ptx_human_patient.1)&#xA;ptx_human_patient.1 &amp;lt;- ScaleData(ptx_human_patient.1, features = VariableFeatures(object = ptx_human_patient.1), vars.to.regress = c(&quot;nCount_RNA&quot;))&#xA;ptx_human_patient.1 &amp;lt;- RunPCA(ptx_human_patient.1, features = VariableFeatures(object = ptx_human_patient.1))&#xA;ptx_human_patient.1 &amp;lt;- FindNeighbors(ptx_human_patient.1, dims = 1:20)&#xA;ptx_human_patient.1 &amp;lt;- FindClusters(ptx_human_patient.1, resolution = 0.4)&#xA;ptx_human_patient.1 &amp;lt;- RunUMAP(ptx_human_patient.1, dims = 1:20)&#xA;&#xA;#run plots&#xA;DimPlot(ptx_human_patient.1, reduction = &quot;umap&quot;, group.by = &quot;model&quot;)&#xA;DimPlot(ptx_human_patient.1, reduction = &quot;umap&quot;, label = &quot;TRUE&quot;)&#xA;saveRDS(ptx_human_patient.1, file = &quot;derived_data/PTX_human_2876_res04_6000g.rds&quot;)&#xA;DimPlot(ptx_human_patient.1, reduction = &quot;umap&quot;, group.by = &quot;model&quot;)&#xA;DimPlot(ptx_human_patient.1, reduction = &quot;umap&quot;, label = TRUE, pt.size = 0.5) + NoLegend()&#xA;ptx_human_patient.1$clustering.idents &amp;lt;- Idents(object = ptx_human_patient.1)&#xA;Idents(ptx_human_patient.1) &amp;lt;- &quot;clustering.idents&quot;&#xA;new.cluster.ids &amp;lt;- c(&quot;Human PTX Tumor&quot;, &quot;PTX Fibroblast&quot;, &quot;Human Fibroblast&quot;, &quot;PTX Macrophage&quot;, &quot;Human PTX Tumor&quot;, &quot;Human Mitochondrial&quot;, &quot;Human Macrophage&quot;, &quot;Human Endothelial&quot;, &quot;Human T-cells&quot;, &quot;Human PTX Tumor&quot;,&quot;Human Fibroblast&quot;, &quot;PTX Fibroblast&quot;, &quot;Human Tumor Ductal&quot;, &quot;Human Acinar&quot;, &quot;PTX Macrophage&quot;, &quot;PTX Stellate&quot;, &quot;PTX Endothelial&quot;)&#xA;names(new.cluster.ids) &amp;lt;- levels(ptx_human_patient.1)&#xA;ptx_human_patient.1 &amp;lt;- RenameIdents(ptx_human_patient.1, new.cluster.ids)&#xA;saveRDS(ptx_human_patient.1, file = &quot;derived_data/PTX_human_2876_res04_6000g.rds&quot;)&#xA;p1 = DimPlot(ptx_human_patient.1, reduction = &quot;umap&quot;, label = TRUE, pt.size = 0.5,  group.by = &quot;clustering.idents&quot;) + NoLegend()&#xA;p2 = DimPlot(ptx_human_patient.1, reduction = &quot;umap&quot;, label = TRUE, pt.size = 0.5) + NoLegend()&#xA;print(CombinePlots(plots = list(p1, p2)))&#xA;markers2 &amp;lt;- FindAllMarkers(ptx_human_patient.1, min.pct = 0.25, logfc.threshold = 0.25)&#xA;markers2 %&amp;gt;% group_by(cluster) %&amp;gt;% top_n(n = 2, wt = avg_logFC)&#xA;saveRDS(markers, file=&quot;derived_data/markers_HTB2876_PC20_res04_6000g.rds&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Part of my excel file &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;barcode muTaTionsTaTus&#xA;GAGATGGCACTAGGCC-1  G12V&#xA;CTGAGCGAGACTCAAA-1  G12V&#xA;TACCGGGGTCCGAAGA-1  G12V&#xA;TATACCTCATAGGCGA-1  G12V&#xA;CTGCCATGTATTGGCT-1  G12V&#xA;AGTGTTGTCAGGAGAC-1  G12V&#xA;AATGCCAAGCAGTACG-1  G12V&#xA;TTTGTTGAGGCTTCCG-1  G12V&#xA;ACAGAAATCCATTCGC-1  G12V&#xA;GTCATTTCAACTAGAA-1  G12V&#xA;GAGTGTTGTGAGATTA-1  G12V&#xA;TCTTGCGCATAACGGG-1  G12V&#xA;ACATCGATCGGCATCG-1  G12V&#xA;CATCCCACACACAGCC-1  G12V&#xA;GACAGCCGTAGCTGTT-1  G12V&#xA;CACAGATGTTAGGCCC-1  G12V&#xA;AACGGGATCACAACCA-1  G12V&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6827" LastActivityDate="2020-04-23T16:31:19.980" Title="Adding certain barcodes to metadata in Seurat" Tags="|seurat|single-cell|umap|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="12996" PostTypeId="2" ParentId="12995" CreationDate="2020-04-21T00:50:18.410" Score="0" Body="&lt;p&gt;From my understanding, you are just trying to add some new information to the metadata of your Seurat object. To do so you can just add the column to &lt;code&gt;meta.data&lt;/code&gt; slots in your Seurat object. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;(&lt;a href=&quot;https://github.com/satijalab/seurat/wiki/Seurat#slots&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/satijalab/seurat/wiki/Seurat#slots&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;myBarcode = rownames(seurat_obj@meta.data) #get barcode from seurat&#xA;mutation.df = mutation.df[match(myBarcode, mutation.df$barcode), ] #match the order of seurat barcode with you data&#xA;seurat_obj$mutation = mutation.df$muTaTionsTaTus #put mutation into metadata&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4766" LastEditorUserId="4766" LastEditDate="2020-04-22T16:09:02.853" LastActivityDate="2020-04-22T16:09:02.853" CommentCount="15" ContentLicense="CC BY-SA 4.0" />
  <row Id="12997" PostTypeId="1" AcceptedAnswerId="13000" CreationDate="2020-04-21T01:52:22.630" Score="1" ViewCount="237" Body="&lt;p&gt;I am using gene expression data from the The Genotype-Tissue Expression (GTEx) project, available here: &lt;a href=&quot;http://gtexportal.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://gtexportal.org&lt;/a&gt;. I would also like to use phenotype data for each sample (especially age, sex, race, body mass index, smoking history).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Access to such information is controlled by the NIH, so you need to apply for access.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before I go through that probably lengthy process, I would like to know if it is worth it. &lt;strong&gt;Are any of the phenotype data age, sex, race, body mass index, smoking history available on a per-sample level, if you have access?&lt;/strong&gt; I could not find this information anywhere.&lt;/p&gt;&#xA;" OwnerUserId="7354" LastActivityDate="2020-04-21T03:56:48.277" Title="Available phenotype data from GTEx" Tags="|gene|public-databases|gene-expression|public-dataset|gtex|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13000" PostTypeId="2" ParentId="12997" CreationDate="2020-04-21T03:56:48.277" Score="3" Body="&lt;p&gt;If helpful for anyone else, I found that all and much more are available. See below 2 links.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/projects/gap/cgi-bin/dataset.cgi?study_id=phs000424.v8.p2&amp;amp;phv=169091&amp;amp;phd=3910&amp;amp;pha=&amp;amp;pht=2742&amp;amp;phvf=&amp;amp;phdf=&amp;amp;phaf=&amp;amp;phtf=&amp;amp;dssp=1&amp;amp;consent=&amp;amp;temp=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/projects/gap/cgi-bin/dataset.cgi?study_id=phs000424.v8.p2&amp;amp;phv=169091&amp;amp;phd=3910&amp;amp;pha=&amp;amp;pht=2742&amp;amp;phvf=&amp;amp;phdf=&amp;amp;phaf=&amp;amp;phtf=&amp;amp;dssp=1&amp;amp;consent=&amp;amp;temp=1&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/dbgap/studies/phs000424/phs000424.v8.p2/pheno_variable_summaries/phs000424.v8.pht002742.v8.p2.GTEx_Subject_Phenotypes.var_report.xml&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nlm.nih.gov/dbgap/studies/phs000424/phs000424.v8.p2/pheno_variable_summaries/phs000424.v8.pht002742.v8.p2.GTEx_Subject_Phenotypes.var_report.xml&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7354" LastActivityDate="2020-04-21T03:56:48.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13001" PostTypeId="1" AcceptedAnswerId="13002" CreationDate="2020-04-21T06:32:01.403" Score="0" ViewCount="78" Body="&lt;p&gt;I'm trying to add &lt;a href=&quot;https://github.com/ncbi/amr&quot; rel=&quot;nofollow noreferrer&quot;&gt;amrfinder&lt;/a&gt; running within conda to a snakemake workflow, working on a slurm HPC.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first time you run &lt;code&gt;amrfinder&lt;/code&gt;, you have to run &lt;code&gt;amrfinder -u&lt;/code&gt; to update/download the database of genes/proteins which it searches for. Since I'm using the &lt;code&gt;--use-conda&lt;/code&gt; flag of &lt;code&gt;snakemake&lt;/code&gt;, the first time I run the workflow, it creates a new env for the &lt;code&gt;conda&lt;/code&gt; package, so this command has to be run before that copy of &lt;code&gt;amrfinder&lt;/code&gt; will work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is: how do I run &lt;code&gt;amrfinder -u&lt;/code&gt; within &lt;code&gt;snakemake&lt;/code&gt;? If I do it as part of the standard &lt;code&gt;amrfinder&lt;/code&gt; command, it runs as many times as there are samples, and as all the jobs are looking at the same database, when it updates for one job, it breaks the other jobs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't figure out how to run this command in a 'snakemake-y' way, as it has no input, it's just literally &lt;code&gt;amrfinder -u&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course, I can make it work with a hack/workaround, but I'm interested in a snakemake way of doing it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the main snakemake rule for running amr_finder_plus, the new rule, to run &lt;code&gt;amrfinder -u&lt;/code&gt; needs to run before this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule amr_finder_plus:&#xA;    input:&#xA;        assembly = 'my_assembly.fasta'&#xA;    output:&#xA;        amr_finder_plus_results = 'my_amr_finder_plus_results.tsv'&#xA;    conda:&#xA;        '../../envs/amrfinderplus.yaml'&#xA;    shell:&#xA;        'amrfinder -n {input.assembly} -O Salmonella --output {output.amr_finder_plus_results} --threads 4'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="993" LastActivityDate="2020-04-21T09:50:39.957" Title="How to run a one-time 'setup' step for amrfinder in snakemake using conda" Tags="|snakemake|conda|hpc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13002" PostTypeId="2" ParentId="13001" CreationDate="2020-04-21T06:51:36.260" Score="1" Body="&lt;p&gt;There are two approaches you can take:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;There's a &lt;code&gt;--conda-create-envs-only&lt;/code&gt; option that you can use to exit after creating the envs. You should get information about where the envs are so you can run &lt;code&gt;amrfinder -u&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Add a new rule called &quot;amr_setup&quot; or something like that that runs:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;amrfinder -u&#xA;touch amrfinder.setup&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then put &lt;code&gt;amrfinder.setup&lt;/code&gt; into your snakemake workflow as an input file into the amr_finder_plus rule. Either of these will work&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2020-04-21T09:50:39.957" LastActivityDate="2020-04-21T09:50:39.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13003" PostTypeId="2" ParentId="12988" CreationDate="2020-04-21T09:01:09.900" Score="1" Body="&lt;p&gt;You can use join to convert the list of input file into a string. &#xA;I think this would work as expected.:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;rule run_prog:&#xA;    input: &quot;a&quot;, &quot;b&quot;&#xA;    output: &quot;output&quot;&#xA;    run: f&quot;prog {','.join(input)}&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="439" LastEditorUserId="439" LastEditDate="2020-04-22T09:28:53.430" LastActivityDate="2020-04-22T09:28:53.430" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13004" PostTypeId="1" CreationDate="2020-04-21T11:03:22.357" Score="0" ViewCount="64" Body="&lt;p&gt;hope everyone is ok. I used Trinity to do a de novo transcriptome assembly, then blastp/blastx and then used Blast2GO software to do KEGG and GO analysis. So i got some txt files with header :&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for GO&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Sequence name Sequence desc. Sequence length Hit desc. Hit ACC E-Value Similarity Bit-Score Alignment length Positives&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;for KEGG&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Seq Pathway Pathway ID #Enzs in Pathway Enzyme&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All well so far. Then i was asked to do 3 tasks. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Tables with terpene synthases, (monoterpenes, sesquiterpenes, diterpenes and triterpenes) and corresponding phylogenetic trees including other known terpene synthases from all categories from other plant species. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Tables with Cytochrome P450s enzymes implicated in terpene biosynthesis (categorization per clades and clans) and corresponding phylogenetic trees including other known terpene CYPs from all clades and clans from other plant species. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;From the graphs corresponding to the different KEGG and GO ontology analyses, export the data for the GO analysis for all three parameters (categories, names, blasts, total number of contigs and percentage) to an excel file to be able to transform them in other formats for future publication.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I have no idea how to proceed. Any guidance? Thanks&lt;/p&gt;&#xA;" OwnerUserId="7172" LastEditorUserId="3848" LastEditDate="2020-04-28T08:05:21.537" LastActivityDate="2020-09-25T09:03:15.323" Title="After KEGG and GO analysis, how to make tables+phylogenetic trees" Tags="|phylogenetics|assembly|phylogeny|kegg|go|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13008" PostTypeId="2" ParentId="12995" CreationDate="2020-04-21T15:21:03.667" Score="2" Body="&lt;p&gt;If you want to add metadata to a Seurat object the new metadata needs to have the same length as the old metadata (== number of cells).  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For an example, if you have a 8 cells &lt;code&gt;c(&quot;a&quot;, &quot;b&quot;, &quot;c&quot;, &quot;d&quot;, &quot;e&quot;, &quot;f&quot;, &quot;g&quot;)&lt;/code&gt; and you want to be able to highlight the 4 cells with the &quot;barcodes&quot; &lt;code&gt;c(&quot;a&quot;, &quot;c&quot;, &quot;f&quot;, &quot;g&quot;)&lt;/code&gt; in a umap plot you have to add a new grouping column:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# The Seurat object is called &quot;ExampleData&quot; and columns can be directly adressed using &quot;$&quot;&#xA;# The barcodes of all cells are stored as the rownames of the meta.data slot &#xA;ExampleData$cells_of_interest &amp;lt;- ifelse(rownames(ExampleData@meta.data) %in% c(&quot;a&quot;, &quot;c&quot;, &quot;f&quot;, &quot;g&quot;), &quot;interesting&quot;, &quot;boring&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;From this you will get a new column in the meta data with the values:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;interesing, boring, interesting, boring, boring, interesting, interesting&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This way you can use this column in &lt;code&gt;DimPlot&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DimPlot(ExampleData, reduction = &quot;umap&quot;, group.by = &quot;cells_of_interest&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;With two colors in the plot one called &quot;interesting&quot; and one &quot;boring&quot;&lt;/p&gt;&#xA;" OwnerUserId="4885" LastEditorUserId="4885" LastEditDate="2020-04-23T16:31:19.980" LastActivityDate="2020-04-23T16:31:19.980" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="13009" PostTypeId="2" ParentId="11547" CreationDate="2020-04-21T15:42:17.150" Score="4" Body="&lt;p&gt;R, the reproductive number, relates to the average number of (new) people that will get sick (infected) per person that is already sick.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For instance, if &lt;span class=&quot;math-container&quot;&gt;$R=2$&lt;/span&gt; and you start with a single infected person, then the next generation will be 2 people, those 2 people will make 4 people sick, those 4 people will make 8 people sick, and so on.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Is R a fixed constant for a disease?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;This reproduction number, how many people the disease is passed on to, depends a lot on the circumstances. For instance, &lt;strong&gt;how frequent&lt;/strong&gt; are contacts and how likely is &lt;strong&gt;transmission per contact&lt;/strong&gt;. The probability of transmission per contact will depend on the type of disease (and from that stems those &lt;a href=&quot;https://informationisbeautiful.net/visualizations/the-microbescope-infectious-diseases-in-context/&quot; rel=&quot;nofollow noreferrer&quot;&gt;comparisons of different disease&lt;/a&gt;), but can also be influenced by measures like reducing frequency and intensity of contacts (social distancing).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The coronavirus spreads in droplets from breath, coughs and sneezes, and will reach other people mostly during close contact (Although there may be events of long-distance spreading if the virus is spread in aerosols, which is claimed to be not the case but it may still exist to a small extend, like with SARS in 2003 explained &lt;a href=&quot;https://www.nejm.org/doi/full/10.1056/nejmoa031349&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://www.nejm.org/doi/full/10.1056/NEJMoa032867&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). So, the reproduction number, R, will likely be influenced by measures like preventing close contact, what they also call social distancing, as well as proper hygiene (washing hands to prevent transmission from virus picked up from fomites).&#xA;A clear example is the case of a sexually transmittable disease, where condom use, or not having sex at all, will reduce the spreading of the disease.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Alternatively, is there some variability in R and does that degree of variation affect transmission? &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;There is a &lt;em&gt;lot&lt;/em&gt; of variability in transmission rate. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The spread of the covid-19 virus has likely been accelerated by celebration of New Year's in China and skiing holidays and carnaval in Europe (as well as large scale meetings during soccer matches and church services).  (this is why large events are bad)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, the disease will at some point satiate the surrounding. Once people have been infected they are likely immune for some while and less likely to be infected again. So once the entire household, classroom, or workplace is infected then new infections will be less likely. (this is why long distance travelling is bad because it brings the virus quickly into untouched area's)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is wrong to speak about &lt;em&gt;the&lt;/em&gt; reproduction number. The reproduction occurs heterogeneously and it is wrong to speak of a single number. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The average reproduction might be 1 or less, but there can &lt;em&gt;still&lt;/em&gt; be effectively exponential growth within a sub-population with relatively higher transmission rates. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Especially when this sub-population is strongly connected network on a large scale. (the virus is like hopping from cluster to cluster in a large network, the transmission occurs mostly in dense clusters of the network and some area's remain relatively little affected, this is why Sweden does not need to take though measures, their network is relatively sparse and safe)&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Finally the areas under the two curves is about the same. This means that the number of patients is about the same with or without measures.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;That sketch in the economist article is just a simplistic sketch. The 'flattening the curve' will reduce the rate of growth, but it should &lt;em&gt;also&lt;/em&gt; reduce the total number of cases.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An example is the image below from this SE post:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://stats.stackexchange.com/questions/461319/using-different-nodes-in-a-networked-compartmental-model-sir-for-different-reg&quot;&gt;https://stats.stackexchange.com/questions/461319/using-different-nodes-in-a-networked-compartmental-model-sir-for-different-reg&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the image you see the effect of different social distancing measures (more details about it in that post). On the left the cumulative cases are plotted (this relates to the area under the curves) and you see a reduction of the total cases. The curves are not &lt;em&gt;only&lt;/em&gt; flattened, but also reduced in volume. (Of course, this does require that the measures are continuously followed for the entire period. That image is an idealistic situation)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/uHIht.png&quot; alt=&quot;example&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Also, it looks to me as though the blue graph goes to zero very quickly&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The endemic will eventually halt when a lot of people have already gotten sick (it may also continue for a long time, and become endemic, because the population is not static, and births introduce new people that have not been sick before, also immunity may become lost over time). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;em&gt;basic&lt;/em&gt; reproduction value, often denoted &lt;span class=&quot;math-container&quot;&gt;$R_0$&lt;/span&gt;, is the number of people that the infection is passed on to &lt;em&gt;when nobody in the surrounding is immune&lt;/em&gt;. But if a lot of people have gotten sick already, or when there is a vaccine, then a lot of people are immune, and less new people will acquire the disease. Effectively the reproduction will be less.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When the disease spreads faster, then this point of satiation/immunity will also be reached more quickly. Currently, strategies are being developed to make the disease spread as fast as possible (to get over with it quickly) but not too fast (to not overload the intensive care units and health care system). This may take a long time (there is still a lot of uncertainty about it) and possibly not economically viable. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you see that the spreading might be over more quickly when we do nothing about it, then it could be better to just let the disease continue without measures, accept the high amount of deaths, and be done with it quickly instead of letting it linger for a long time. (But not many would dare to suggest these things seriously, because of the complicated ethical issues, and a stigmatization of an utilitarian approach to health and preventing deaths. )&lt;/p&gt;&#xA;" OwnerUserId="8671" LastEditorUserId="8671" LastEditDate="2020-04-22T08:55:29.163" LastActivityDate="2020-04-22T08:55:29.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13010" PostTypeId="1" AcceptedAnswerId="13116" CreationDate="2020-04-21T15:57:52.027" Score="1" ViewCount="1377" Body="&lt;p&gt;I have some snakemake pipelines that have rules which run in different environments. Is it possible to run the pipeline without having to rebuild the environments from scratch every time when run in a different directory?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is the way I would normally run snakemake:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;snakemake --directory &amp;lt;my_output_dir&amp;gt; --use-conda&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My snakefile might have rules that need to use different environments:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule A:&#xA;   ....&#xA;   conda: envA.yml&#xA;   ....&#xA;&#xA;rule B:&#xA;   ....&#xA;   conda: envB.yml&#xA;   ....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So if I wanted to do two different analyses with the same pipeline, is there anyway that I can point them to the same environments without having to install everything separately in each output directory?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The snakemake documentation makes it seem like they don't currently support this. You can either use 1) --conda-prefix when running snakemake and then point to a single env or 2) use the --use-conda directive and the env.yml files by rule, in which case it will create new envs any time you run it in a new output directory. So my case doesn't seem to be covered, but does anyone have a workaround to suggest? I often find conflicts between some of the R packages I use and python packages, so I tend to keep them in their own environments to avoid headache.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm starting to get neck-deep in conda environments at this point which is taking up space from my disk quota, but also I am bothered by the fact that I may actually want to run analysis of data from different projects with the same environments. If the environments are rebuilt at each execution, I do not know that they are the same, short of specifying the exact version info for everything I use in the env.yaml files (and even then, they could be slightly different if the repo has changed).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have looked at this similar question: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/6914/running-snakemake-in-one-single-conda-env/6921#6921&quot;&gt;Running Snakemake in one single conda env&lt;/a&gt;&#xA;in which it is suggested there is a way to do such a thing, but I'm afraid I don't understand how to implement it without an example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="8670" LastActivityDate="2020-04-28T06:11:28.257" Title="Running snakemake with multiple conda environments without rebuilding environments every time" Tags="|snakemake|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13013" PostTypeId="1" CreationDate="2020-04-21T18:23:31.283" Score="1" ViewCount="1187" Body="&lt;p&gt;I recently tried to load library(Seurat) after installing Seurat package. It caused my R studio to have a fatal error and abort my session. I have successfully downloaded it before. I have uninstalled R and R studio and reinstalled, restarted my computer. I am new to R too, so I'm not sure what I'm doing. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am following the directions provided on the satijalab.org website on how to install Seurat. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a screenshot of the error message. &lt;a href=&quot;https://i.stack.imgur.com/bOj4f.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bOj4f.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran install.packages(&quot;Seurat&quot;) in R. R is also crashing as soon as I run library(Seurat). Here's a screenshot right before I hit run, and when it crashes, R application just closes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zmtHg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zmtHg.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8610" LastEditorUserId="8610" LastEditDate="2020-04-21T20:16:14.447" LastActivityDate="2022-08-10T19:19:28.507" Title="Why is library(Seurat) causing my R session to have a fatal error?" Tags="|r|seurat|rstudio|" AnswerCount="2" CommentCount="14" ContentLicense="CC BY-SA 4.0" />
  <row Id="13014" PostTypeId="1" CreationDate="2020-04-21T18:50:55.067" Score="1" ViewCount="1791" Body="&lt;p&gt;I successfully installed cellranger and performed a test run. Even, I installed bcl2fastq. Everything looked perfect. Thus, I ran &lt;code&gt;cellranger count&lt;/code&gt;. I ran into an error during the &lt;code&gt;cellranger count&lt;/code&gt; run. I put command as suggested in the tutorial as follows (I double-checked any typo that can cause error).&lt;/p&gt;&#xA;&lt;p&gt;tutorial (&lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/tutorial_ct&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/tutorial_ct&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cellranger count --id=run_count_1kpbmcs \&#xA;--fastqs=/mnt/home/user.name/yard/run_cellranger_count/pbmc_1k_v3_fastqs \&#xA;--sample=pbmc_1k_v3 \&#xA;--transcriptome=/mnt/home/user.name/yard/run_cellranger_count/refdata-cellranger-GRCh38-3.0.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Initially, the process went smoothly, but it suddenly found error as follows. It's not likely due to memory issues.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;pre&gt;&lt;code&gt;[error] Pipestance failed. Error log at:&#xA;run_count_1kpbmcs/SC_RNA_COUNTER_CS/SC_RNA_COUNTER/_BASIC_SC_RNA_COUNTER/ALIGN_READS/fork0/chnk00-u27879f31e3/_errors&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Log message:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &amp;quot;/ufrc/zhangw/my.kim/yard/apps/cellranger-3.1.0/martian-cs/v3.2.3/adapters/python/martian_shell.py&amp;quot;, line 589, in _main&#xA;    stage.main()&#xA;  File &amp;quot;/ufrc/zhangw/my.kim/yard/apps/cellranger-3.1.0/martian-cs/v3.2.3/adapters/python/martian_shell.py&amp;quot;, line 554, in main&#xA;    self._run(lambda: self._module.main(args, outs))&#xA;  File &amp;quot;/ufrc/zhangw/my.kim/yard/apps/cellranger-3.1.0/martian-cs/v3.2.3/adapters/python/martian_shell.py&amp;quot;, line 523, in _run&#xA;    cmd()&#xA;  File &amp;quot;/ufrc/zhangw/my.kim/yard/apps/cellranger-3.1.0/martian-cs/v3.2.3/adapters/python/martian_shell.py&amp;quot;, line 554, in &amp;lt;lambda&amp;gt;&#xA;    self._run(lambda: self._module.main(args, outs))&#xA;  File &amp;quot;/ufrc/zhangw/my.kim/yard/apps/cellranger-3.1.0/cellranger-cs/3.1.0/mro/stages/counter/align_reads/__init__.py&amp;quot;, line 128, in main&#xA;    align_reads(args, outs)&#xA;  File &amp;quot;/ufrc/zhangw/my.kim/yard/apps/cellranger-3.1.0/cellranger-cs/3.1.0/mro/stages/counter/align_reads/__init__.py&amp;quot;, line 67, in align_reads&#xA;    read_group_tags=tk_bam.make_star_rg_header(args.read_group))&#xA;  File &amp;quot;/ufrc/zhangw/my.kim/yard/apps/cellranger-3.1.0/cellranger-cs/3.1.0/lib/python/cellranger/reference.py&amp;quot;, line 616, in align&#xA;    raise cr_io.CRCalledProcessError(e.msg + ' Check STAR logs for errors: %s .' % star_log)&#xA;CRCalledProcessError: Process returned error code -9: STAR . Check STAR logs for errors: /ufrc/zhangw/my.kim/yard/run_cellranger_count/run_count_1kpbmcs/SC_RNA_COUNTER_CS/SC_RNA_COUNTER/_BASIC_SC_RNA_COUNTER/ALIGN_READS/fork0/chnk00-u27879f31e3/files/Log.out .&#xA;&#xA;&#xA;Waiting 6 seconds for UI to do final refresh.&#xA;Pipestance failed. Use --noexit option to keep UI running after failure.&#xA;&#xA;2020-04-21 13:48:37 Shutting down.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h2 id=&quot;meanwhile-i-checked-star-logs-for-errors-according-to-haci-said-and-i-found-these&quot;&gt;Meanwhile, I checked STAR logs for errors according to Haci said, and I found these,&lt;/h2&gt;&#xA;&lt;pre&gt;&lt;code&gt;STAR version=                 STAR_2.5.1b&#xA;STAR compilation time,server,dir=Wed May 8 16:13:25 PDT 2019 :/mnt/home/jenkins/workspace/module-STAR-5dda596---latest/sake/modules/STAR/5dda596/source&#xA;##### DEFAULT parameters:&#xA;versionSTAR                       20201&#xA;versionGenome                     20101   20200   &#xA;parametersFiles                   -   &#xA;sysShell                          -&#xA;runMode                           alignReads&#xA;runThreadN                        1&#xA;runDirPerm                        User_RWX&#xA;runRNGseed                        777&#xA;genomeDir                         ./GenomeDir/&#xA;genomeLoad                        NoSharedMemory&#xA;genomeFastaFiles                  -   &#xA;genomeSAindexNbases               14&#xA;genomeChrBinNbits                 18&#xA;genomeSAsparseD                   1&#xA;genomeSuffixLengthMax             18446744073709551615&#xA;readFilesIn                       Read1   Read2   &#xA;readFilesCommand                  -   &#xA;readMatesLengthsIn                NotEqual&#xA;readMapNumber                     18446744073709551615&#xA;readNameSeparator                 /   &#xA;inputBAMfile                      -&#xA;bamRemoveDuplicatesType           -&#xA;bamRemoveDuplicatesMate2basesN    0&#xA;limitGenomeGenerateRAM            31000000000&#xA;limitIObufferSize                 150000000&#xA;limitOutSAMoneReadBytes           100000&#xA;limitOutSJcollapsed               1000000&#xA;limitOutSJoneRead                 1000&#xA;limitBAMsortRAM                   0&#xA;limitSjdbInsertNsj                1000000&#xA;outFileNamePrefix                 ./&#xA;outTmpDir                         -&#xA;outReadsUnmapped                  None&#xA;outQSconversionAdd                0&#xA;outMultimapperOrder               Old_2.4&#xA;outSAMtype                        SAM   &#xA;outSAMmode                        Full&#xA;outSAMstrandField                 None&#xA;outSAMattributes                  Standard   &#xA;outSAMunmapped                    None   &#xA;outSAMorder                       Paired&#xA;outSAMprimaryFlag                 OneBestScore&#xA;outSAMreadID                      Standard&#xA;outSAMmapqUnique                  255&#xA;outSAMflagOR                      0&#xA;outSAMflagAND                     65535&#xA;outSAMattrRGline                  -   &#xA;outSAMheaderHD                    -   &#xA;outSAMheaderPG                    -   &#xA;outSAMheaderCommentFile           -&#xA;outBAMcompression                 1&#xA;outBAMsortingThreadN              0&#xA;outSAMfilter                      None   &#xA;outSAMmultNmax                    18446744073709551615&#xA;outSAMattrIHstart                 1&#xA;outSJfilterReads                  All&#xA;outSJfilterCountUniqueMin         3   1   1   1   &#xA;outSJfilterCountTotalMin          3   1   1   1   &#xA;outSJfilterOverhangMin            30   12   12   12   &#xA;outSJfilterDistToOtherSJmin       10   0   5   10   &#xA;outSJfilterIntronMaxVsReadN       50000   100000   200000   &#xA;outWigType                        None   &#xA;outWigStrand                      Stranded   &#xA;outWigReferencesPrefix            -&#xA;outWigNorm                        RPM   &#xA;outFilterType                     Normal&#xA;outFilterMultimapNmax             10&#xA;outFilterMultimapScoreRange       1&#xA;outFilterScoreMin                 0&#xA;outFilterScoreMinOverLread        0.66&#xA;outFilterMatchNmin                0&#xA;outFilterMatchNminOverLread       0.66&#xA;outFilterMismatchNmax             10&#xA;outFilterMismatchNoverLmax        0.3&#xA;outFilterMismatchNoverReadLmax    1&#xA;outFilterIntronMotifs             None&#xA;clip5pNbases                      0   &#xA;clip3pNbases                      0   &#xA;clip3pAfterAdapterNbases          0   &#xA;clip3pAdapterSeq                  -   &#xA;clip3pAdapterMMp                  0.1   &#xA;winBinNbits                       16&#xA;winAnchorDistNbins                9&#xA;winFlankNbins                     4&#xA;winAnchorMultimapNmax             50&#xA;scoreGap                          0&#xA;scoreGapNoncan                    -8&#xA;scoreGapGCAG                      -4&#xA;scoreGapATAC                      -8&#xA;scoreStitchSJshift                1&#xA;scoreGenomicLengthLog2scale       -0.25&#xA;scoreDelBase                      -2&#xA;scoreDelOpen                      -2&#xA;scoreInsOpen                      -2&#xA;scoreInsBase                      -2&#xA;seedSearchLmax                    0&#xA;seedSearchStartLmax               50&#xA;seedSearchStartLmaxOverLread      1&#xA;seedPerReadNmax                   1000&#xA;seedPerWindowNmax                 50&#xA;seedNoneLociPerWindow             10&#xA;seedMultimapNmax                  10000&#xA;alignIntronMin                    21&#xA;alignIntronMax                    0&#xA;alignMatesGapMax                  0&#xA;alignTranscriptsPerReadNmax       10000&#xA;alignSJoverhangMin                5&#xA;alignSJDBoverhangMin              3&#xA;alignSJstitchMismatchNmax         0   -1   0   0   &#xA;alignSplicedMateMapLmin           0&#xA;alignSplicedMateMapLminOverLmate    0.66&#xA;alignWindowsPerReadNmax           10000&#xA;alignTranscriptsPerWindowNmax     100&#xA;alignEndsType                     Local&#xA;alignSoftClipAtReferenceEnds      Yes&#xA;chimSegmentMin                    0&#xA;chimScoreMin                      0&#xA;chimScoreDropMax                  20&#xA;chimScoreSeparation               10&#xA;chimScoreJunctionNonGTAG          -1&#xA;chimJunctionOverhangMin           20&#xA;chimOutType                       SeparateSAMold&#xA;chimFilter                        banGenomicN   &#xA;chimSegmentReadGapMax             0&#xA;sjdbFileChrStartEnd               -   &#xA;sjdbGTFfile                       -&#xA;sjdbGTFchrPrefix                  -&#xA;sjdbGTFfeatureExon                exon&#xA;sjdbGTFtagExonParentTranscript    transcript_id&#xA;sjdbGTFtagExonParentGene          gene_id&#xA;sjdbOverhang                      100&#xA;sjdbScore                         2&#xA;sjdbInsertSave                    Basic&#xA;quantMode                         -   &#xA;quantTranscriptomeBAMcompression    1&#xA;quantTranscriptomeBan             IndelSoftclipSingleend&#xA;twopass1readsN                    18446744073709551615&#xA;twopassMode                       None&#xA;##### Command Line:&#xA;STAR --genomeDir /ufrc/zhangw/my.kim/yard/run_cellranger_count/refdata-cellranger-GRCh38-3.0.0/star --outSAMmultNmax -1 --runThreadN 4 --readNameSeparator space --outSAMunmapped Within KeepPairs --outSAMtype SAM --outStd SAM --outSAMorder PairedKeepInputOrder --outSAMattrRGline ID:run_count_1kpbmcs:0:1:HFWFVDMXX:1 SM:run_count_1kpbmcs LB:0.1 PU:run_count_1kpbmcs:0:1:HFWFVDMXX:1 PL:ILLUMINA --readFilesIn /dev/fd/63&#xA;##### Initial USER parameters from Command Line:&#xA;outStd                            SAM&#xA;###### All USER parameters from Command Line:&#xA;genomeDir                     /ufrc/zhangw/my.kim/yard/run_cellranger_count/refdata-cellranger-GRCh38-3.0.0/star     ~RE-DEFINED&#xA;outSAMmultNmax                18446744073709551615     ~RE-DEFINED&#xA;runThreadN                    4     ~RE-DEFINED&#xA;readNameSeparator             space        ~RE-DEFINED&#xA;outSAMunmapped                Within   KeepPairs        ~RE-DEFINED&#xA;outSAMtype                    SAM        ~RE-DEFINED&#xA;outStd                        SAM     ~RE-DEFINED&#xA;outSAMorder                   PairedKeepInputOrder     ~RE-DEFINED&#xA;outSAMattrRGline              ID:run_count_1kpbmcs:0:1:HFWFVDMXX:1   SM:run_count_1kpbmcs   LB:0.1   PU:run_count_1kpbmcs:0:1:HFWFVDMXX:1   PL:ILLUMINA        ~RE-DEFINED&#xA;readFilesIn                   /dev/fd/63        ~RE-DEFINED&#xA;##### Finished reading parameters from all sources&#xA;&#xA;##### Final user re-defined parameters-----------------:&#xA;runThreadN                        4&#xA;genomeDir                         /ufrc/zhangw/my.kim/yard/run_cellranger_count/refdata-cellranger-GRCh38-3.0.0/star&#xA;readFilesIn                       /dev/fd/63   &#xA;readNameSeparator                 space   &#xA;outStd                            SAM&#xA;outSAMtype                        SAM   &#xA;outSAMunmapped                    Within   KeepPairs   &#xA;outSAMorder                       PairedKeepInputOrder&#xA;outSAMattrRGline                  ID:run_count_1kpbmcs:0:1:HFWFVDMXX:1   SM:run_count_1kpbmcs   LB:0.1   PU:run_count_1kpbmcs:0:1:HFWFVDMXX:1   PL:ILLUMINA   &#xA;outSAMmultNmax                    18446744073709551615&#xA;&#xA;-------------------------------&#xA;##### Final effective command line:&#xA;STAR   --runThreadN 4   --genomeDir /ufrc/zhangw/my.kim/yard/run_cellranger_count/refdata-cellranger-GRCh38-3.0.0/star   --readFilesIn /dev/fd/63      --readNameSeparator space      --outStd SAM   --outSAMtype SAM      --outSAMunmapped Within   KeepPairs      --outSAMorder PairedKeepInputOrder   --outSAMattrRGline ID:run_count_1kpbmcs:0:1:HFWFVDMXX:1   SM:run_count_1kpbmcs   LB:0.1   PU:run_count_1kpbmcs:0:1:HFWFVDMXX:1   PL:ILLUMINA      --outSAMmultNmax 18446744073709551615&#xA;&#xA;##### Final parameters after user input--------------------------------:&#xA;versionSTAR                       20201&#xA;versionGenome                     20101   20200   &#xA;parametersFiles                   -   &#xA;sysShell                          -&#xA;runMode                           alignReads&#xA;runThreadN                        4&#xA;runDirPerm                        User_RWX&#xA;runRNGseed                        777&#xA;genomeDir                         /ufrc/zhangw/my.kim/yard/run_cellranger_count/refdata-cellranger-GRCh38-3.0.0/star&#xA;genomeLoad                        NoSharedMemory&#xA;genomeFastaFiles                  -   &#xA;genomeSAindexNbases               14&#xA;genomeChrBinNbits                 18&#xA;genomeSAsparseD                   1&#xA;genomeSuffixLengthMax             18446744073709551615&#xA;readFilesIn                       /dev/fd/63   &#xA;readFilesCommand                  -   &#xA;readMatesLengthsIn                NotEqual&#xA;readMapNumber                     18446744073709551615&#xA;readNameSeparator                 space   &#xA;inputBAMfile                      -&#xA;bamRemoveDuplicatesType           -&#xA;bamRemoveDuplicatesMate2basesN    0&#xA;limitGenomeGenerateRAM            31000000000&#xA;limitIObufferSize                 150000000&#xA;limitOutSAMoneReadBytes           100000&#xA;limitOutSJcollapsed               1000000&#xA;limitOutSJoneRead                 1000&#xA;limitBAMsortRAM                   0&#xA;limitSjdbInsertNsj                1000000&#xA;outFileNamePrefix                 ./&#xA;outTmpDir                         -&#xA;outStd                            SAM&#xA;outReadsUnmapped                  None&#xA;outQSconversionAdd                0&#xA;outMultimapperOrder               Old_2.4&#xA;outSAMtype                        SAM   &#xA;outSAMmode                        Full&#xA;outSAMstrandField                 None&#xA;outSAMattributes                  Standard   &#xA;outSAMunmapped                    Within   KeepPairs   &#xA;outSAMorder                       PairedKeepInputOrder&#xA;outSAMprimaryFlag                 OneBestScore&#xA;outSAMreadID                      Standard&#xA;outSAMmapqUnique                  255&#xA;outSAMflagOR                      0&#xA;outSAMflagAND                     65535&#xA;outSAMattrRGline                  ID:run_count_1kpbmcs:0:1:HFWFVDMXX:1   SM:run_count_1kpbmcs   LB:0.1   PU:run_count_1kpbmcs:0:1:HFWFVDMXX:1   PL:ILLUMINA   &#xA;outSAMheaderHD                    -   &#xA;outSAMheaderPG                    -   &#xA;outSAMheaderCommentFile           -&#xA;outBAMcompression                 1&#xA;outBAMsortingThreadN              0&#xA;outSAMfilter                      None   &#xA;outSAMmultNmax                    18446744073709551615&#xA;outSAMattrIHstart                 1&#xA;outSJfilterReads                  All&#xA;outSJfilterCountUniqueMin         3   1   1   1   &#xA;outSJfilterCountTotalMin          3   1   1   1   &#xA;outSJfilterOverhangMin            30   12   12   12   &#xA;outSJfilterDistToOtherSJmin       10   0   5   10   &#xA;outSJfilterIntronMaxVsReadN       50000   100000   200000   &#xA;outWigType                        None   &#xA;outWigStrand                      Stranded   &#xA;outWigReferencesPrefix            -&#xA;outWigNorm                        RPM   &#xA;outFilterType                     Normal&#xA;outFilterMultimapNmax             10&#xA;outFilterMultimapScoreRange       1&#xA;outFilterScoreMin                 0&#xA;outFilterScoreMinOverLread        0.66&#xA;outFilterMatchNmin                0&#xA;outFilterMatchNminOverLread       0.66&#xA;outFilterMismatchNmax             10&#xA;outFilterMismatchNoverLmax        0.3&#xA;outFilterMismatchNoverReadLmax    1&#xA;outFilterIntronMotifs             None&#xA;clip5pNbases                      0   &#xA;clip3pNbases                      0   &#xA;clip3pAfterAdapterNbases          0   &#xA;clip3pAdapterSeq                  -   &#xA;clip3pAdapterMMp                  0.1   &#xA;winBinNbits                       16&#xA;winAnchorDistNbins                9&#xA;winFlankNbins                     4&#xA;winAnchorMultimapNmax             50&#xA;scoreGap                          0&#xA;scoreGapNoncan                    -8&#xA;scoreGapGCAG                      -4&#xA;scoreGapATAC                      -8&#xA;scoreStitchSJshift                1&#xA;scoreGenomicLengthLog2scale       -0.25&#xA;scoreDelBase                      -2&#xA;scoreDelOpen                      -2&#xA;scoreInsOpen                      -2&#xA;scoreInsBase                      -2&#xA;seedSearchLmax                    0&#xA;seedSearchStartLmax               50&#xA;seedSearchStartLmaxOverLread      1&#xA;seedPerReadNmax                   1000&#xA;seedPerWindowNmax                 50&#xA;seedNoneLociPerWindow             10&#xA;seedMultimapNmax                  10000&#xA;alignIntronMin                    21&#xA;alignIntronMax                    0&#xA;alignMatesGapMax                  0&#xA;alignTranscriptsPerReadNmax       10000&#xA;alignSJoverhangMin                5&#xA;alignSJDBoverhangMin              3&#xA;alignSJstitchMismatchNmax         0   -1   0   0   &#xA;alignSplicedMateMapLmin           0&#xA;alignSplicedMateMapLminOverLmate    0.66&#xA;alignWindowsPerReadNmax           10000&#xA;alignTranscriptsPerWindowNmax     100&#xA;alignEndsType                     Local&#xA;alignSoftClipAtReferenceEnds      Yes&#xA;chimSegmentMin                    0&#xA;chimScoreMin                      0&#xA;chimScoreDropMax                  20&#xA;chimScoreSeparation               10&#xA;chimScoreJunctionNonGTAG          -1&#xA;chimJunctionOverhangMin           20&#xA;chimOutType                       SeparateSAMold&#xA;chimFilter                        banGenomicN   &#xA;chimSegmentReadGapMax             0&#xA;sjdbFileChrStartEnd               -   &#xA;sjdbGTFfile                       -&#xA;sjdbGTFchrPrefix                  -&#xA;sjdbGTFfeatureExon                exon&#xA;sjdbGTFtagExonParentTranscript    transcript_id&#xA;sjdbGTFtagExonParentGene          gene_id&#xA;sjdbOverhang                      100&#xA;sjdbScore                         2&#xA;sjdbInsertSave                    Basic&#xA;quantMode                         -   &#xA;quantTranscriptomeBAMcompression    1&#xA;quantTranscriptomeBan             IndelSoftclipSingleend&#xA;twopass1readsN                    18446744073709551615&#xA;twopassMode                       None&#xA;----------------------------------------&#xA;&#xA;WARNING --outSAMattrRG defines a read group, therefore STAR will output RG attribute&#xA;Finished loading and checking parameters&#xA;Reading genome generation parameters:&#xA;versionGenome                 20201        ~RE-DEFINED&#xA;genomeFastaFiles              /mnt/yard2/pat/cr/new-refs/reference_buildscripts2/GRCh38/fasta/genome.fa        ~RE-DEFINED&#xA;genomeSAindexNbases           14     ~RE-DEFINED&#xA;genomeChrBinNbits             18     ~RE-DEFINED&#xA;genomeSAsparseD               3     ~RE-DEFINED&#xA;sjdbOverhang                  100     ~RE-DEFINED&#xA;sjdbFileChrStartEnd           -        ~RE-DEFINED&#xA;sjdbGTFfile                   /mnt/yard2/pat/cr/new-refs/reference_buildscripts2/GRCh38/genes/genes.gtf     ~RE-DEFINED&#xA;sjdbGTFchrPrefix              -     ~RE-DEFINED&#xA;sjdbGTFfeatureExon            exon     ~RE-DEFINED&#xA;sjdbGTFtagExonParentTranscripttranscript_id     ~RE-DEFINED&#xA;sjdbGTFtagExonParentGene      gene_id     ~RE-DEFINED&#xA;sjdbInsertSave                Basic     ~RE-DEFINED&#xA;Genome version is compatible with current STAR version&#xA;Number of real (reference) chromosmes= 194&#xA;1   1   248956422   0&#xA;2   10  133797422   249036800&#xA;3   11  135086622   382992384&#xA;4   12  133275309   518258688&#xA;5   13  114364328   651689984&#xA;6   14  107043718   766246912&#xA;7   15  101991189   873463808&#xA;8   16  90338345    975699968&#xA;9   17  83257441    1066139648&#xA;10  18  80373285    1149501440&#xA;11  19  58617616    1229979648&#xA;12  2   242193529   1288699904&#xA;13  20  64444167    1530920960&#xA;14  21  46709983    1595408384&#xA;15  22  50818468    1642332160&#xA;16  3   198295559   1693188096&#xA;17  4   190214555   1891631104&#xA;18  5   181538259   2081947648&#xA;19  6   170805979   2263613440&#xA;20  7   159345973   2434531328&#xA;21  8   145138636   2593914880&#xA;22  9   138394717   2739142656&#xA;23  MT  16569   2877554688&#xA;24  X   156040895   2877816832&#xA;25  Y   57227415    3034054656&#xA;26  KI270728.1  1872759 3091464192&#xA;27  KI270727.1  448248  3093561344&#xA;28  KI270442.1  392061  3094085632&#xA;29  KI270729.1  280839  3094609920&#xA;30  GL000225.1  211173  3095134208&#xA;31  KI270743.1  210658  3095396352&#xA;32  GL000008.2  209709  3095658496&#xA;33  GL000009.2  201709  3095920640&#xA;34  KI270747.1  198735  3096182784&#xA;35  KI270722.1  194050  3096444928&#xA;36  GL000194.1  191469  3096707072&#xA;37  KI270742.1  186739  3096969216&#xA;38  GL000205.2  185591  3097231360&#xA;39  GL000195.1  182896  3097493504&#xA;40  KI270736.1  181920  3097755648&#xA;41  KI270733.1  179772  3098017792&#xA;42  GL000224.1  179693  3098279936&#xA;43  GL000219.1  179198  3098542080&#xA;44  KI270719.1  176845  3098804224&#xA;45  GL000216.2  176608  3099066368&#xA;46  KI270712.1  176043  3099328512&#xA;47  KI270706.1  175055  3099590656&#xA;48  KI270725.1  172810  3099852800&#xA;49  KI270744.1  168472  3100114944&#xA;50  KI270734.1  165050  3100377088&#xA;51  GL000213.1  164239  3100639232&#xA;52  GL000220.1  161802  3100901376&#xA;53  KI270715.1  161471  3101163520&#xA;54  GL000218.1  161147  3101425664&#xA;55  KI270749.1  158759  3101687808&#xA;56  KI270741.1  157432  3101949952&#xA;57  GL000221.1  155397  3102212096&#xA;58  KI270716.1  153799  3102474240&#xA;59  KI270731.1  150754  3102736384&#xA;60  KI270751.1  150742  3102998528&#xA;61  KI270750.1  148850  3103260672&#xA;62  KI270519.1  138126  3103522816&#xA;63  GL000214.1  137718  3103784960&#xA;64  KI270708.1  127682  3104047104&#xA;65  KI270730.1  112551  3104309248&#xA;66  KI270438.1  112505  3104571392&#xA;67  KI270737.1  103838  3104833536&#xA;68  KI270721.1  100316  3105095680&#xA;69  KI270738.1  99375   3105357824&#xA;70  KI270748.1  93321   3105619968&#xA;71  KI270435.1  92983   3105882112&#xA;72  GL000208.1  92689   3106144256&#xA;73  KI270538.1  91309   3106406400&#xA;74  KI270756.1  79590   3106668544&#xA;75  KI270739.1  73985   3106930688&#xA;76  KI270757.1  71251   3107192832&#xA;77  KI270709.1  66860   3107454976&#xA;78  KI270746.1  66486   3107717120&#xA;79  KI270753.1  62944   3107979264&#xA;80  KI270589.1  44474   3108241408&#xA;81  KI270726.1  43739   3108503552&#xA;82  KI270735.1  42811   3108765696&#xA;83  KI270711.1  42210   3109027840&#xA;84  KI270745.1  41891   3109289984&#xA;85  KI270714.1  41717   3109552128&#xA;86  KI270732.1  41543   3109814272&#xA;87  KI270713.1  40745   3110076416&#xA;88  KI270754.1  40191   3110338560&#xA;89  KI270710.1  40176   3110600704&#xA;90  KI270717.1  40062   3110862848&#xA;91  KI270724.1  39555   3111124992&#xA;92  KI270720.1  39050   3111387136&#xA;93  KI270723.1  38115   3111649280&#xA;94  KI270718.1  38054   3111911424&#xA;95  KI270317.1  37690   3112173568&#xA;96  KI270740.1  37240   3112435712&#xA;97  KI270755.1  36723   3112697856&#xA;98  KI270707.1  32032   3112960000&#xA;99  KI270579.1  31033   3113222144&#xA;100 KI270752.1  27745   3113484288&#xA;101 KI270512.1  22689   3113746432&#xA;102 KI270322.1  21476   3114008576&#xA;103 GL000226.1  15008   3114270720&#xA;104 KI270311.1  12399   3114532864&#xA;105 KI270366.1  8320    3114795008&#xA;106 KI270511.1  8127    3115057152&#xA;107 KI270448.1  7992    3115319296&#xA;108 KI270521.1  7642    3115581440&#xA;109 KI270581.1  7046    3115843584&#xA;110 KI270582.1  6504    3116105728&#xA;111 KI270515.1  6361    3116367872&#xA;112 KI270588.1  6158    3116630016&#xA;113 KI270591.1  5796    3116892160&#xA;114 KI270522.1  5674    3117154304&#xA;115 KI270507.1  5353    3117416448&#xA;116 KI270590.1  4685    3117678592&#xA;117 KI270584.1  4513    3117940736&#xA;118 KI270320.1  4416    3118202880&#xA;119 KI270382.1  4215    3118465024&#xA;120 KI270468.1  4055    3118727168&#xA;121 KI270467.1  3920    3118989312&#xA;122 KI270362.1  3530    3119251456&#xA;123 KI270517.1  3253    3119513600&#xA;124 KI270593.1  3041    3119775744&#xA;125 KI270528.1  2983    3120037888&#xA;126 KI270587.1  2969    3120300032&#xA;127 KI270364.1  2855    3120562176&#xA;128 KI270371.1  2805    3120824320&#xA;129 KI270333.1  2699    3121086464&#xA;130 KI270374.1  2656    3121348608&#xA;131 KI270411.1  2646    3121610752&#xA;132 KI270414.1  2489    3121872896&#xA;133 KI270510.1  2415    3122135040&#xA;134 KI270390.1  2387    3122397184&#xA;135 KI270375.1  2378    3122659328&#xA;136 KI270420.1  2321    3122921472&#xA;137 KI270509.1  2318    3123183616&#xA;138 KI270315.1  2276    3123445760&#xA;139 KI270302.1  2274    3123707904&#xA;140 KI270518.1  2186    3123970048&#xA;141 KI270530.1  2168    3124232192&#xA;142 KI270304.1  2165    3124494336&#xA;143 KI270418.1  2145    3124756480&#xA;144 KI270424.1  2140    3125018624&#xA;145 KI270417.1  2043    3125280768&#xA;146 KI270508.1  1951    3125542912&#xA;147 KI270303.1  1942    3125805056&#xA;148 KI270381.1  1930    3126067200&#xA;149 KI270529.1  1899    3126329344&#xA;150 KI270425.1  1884    3126591488&#xA;151 KI270396.1  1880    3126853632&#xA;152 KI270363.1  1803    3127115776&#xA;153 KI270386.1  1788    3127377920&#xA;154 KI270465.1  1774    3127640064&#xA;155 KI270383.1  1750    3127902208&#xA;156 KI270384.1  1658    3128164352&#xA;157 KI270330.1  1652    3128426496&#xA;158 KI270372.1  1650    3128688640&#xA;159 KI270548.1  1599    3128950784&#xA;160 KI270580.1  1553    3129212928&#xA;161 KI270387.1  1537    3129475072&#xA;162 KI270391.1  1484    3129737216&#xA;163 KI270305.1  1472    3129999360&#xA;164 KI270373.1  1451    3130261504&#xA;165 KI270422.1  1445    3130523648&#xA;166 KI270316.1  1444    3130785792&#xA;167 KI270340.1  1428    3131047936&#xA;168 KI270338.1  1428    3131310080&#xA;169 KI270583.1  1400    3131572224&#xA;170 KI270334.1  1368    3131834368&#xA;171 KI270429.1  1361    3132096512&#xA;172 KI270393.1  1308    3132358656&#xA;173 KI270516.1  1300    3132620800&#xA;174 KI270389.1  1298    3132882944&#xA;175 KI270466.1  1233    3133145088&#xA;176 KI270388.1  1216    3133407232&#xA;177 KI270544.1  1202    3133669376&#xA;178 KI270310.1  1201    3133931520&#xA;179 KI270412.1  1179    3134193664&#xA;180 KI270395.1  1143    3134455808&#xA;181 KI270376.1  1136    3134717952&#xA;182 KI270337.1  1121    3134980096&#xA;183 KI270335.1  1048    3135242240&#xA;184 KI270378.1  1048    3135504384&#xA;185 KI270379.1  1045    3135766528&#xA;186 KI270329.1  1040    3136028672&#xA;187 KI270419.1  1029    3136290816&#xA;188 KI270336.1  1026    3136552960&#xA;189 KI270312.1  998 3136815104&#xA;190 KI270539.1  993 3137077248&#xA;191 KI270385.1  990 3137339392&#xA;192 KI270423.1  981 3137601536&#xA;193 KI270392.1  971 3137863680&#xA;194 KI270394.1  970 3138125824&#xA;--sjdbOverhang = 100 taken from the generated genome&#xA;Started loading the genome: Tue Apr 21 13:48:20 2020&#xA;&#xA;checking Genome sizefile size: 3205385489 bytes; state: good=1 eof=0 fail=0 bad=0&#xA;checking SA sizefile size: 8651064546 bytes; state: good=1 eof=0 fail=0 bad=0&#xA;checking /SAindex sizefile size: 1565873619 bytes; state: good=1 eof=0 fail=0 bad=0&#xA;Read from SAindex: genomeSAindexNbases=14  nSAi=357913940&#xA;nGenome=3205385489;  nSAbyte=8651064546&#xA;GstrandBit=32   SA number of indices=2097227768&#xA;Shared memory is not used for genomes. Allocated a private copy of the genome.&#xA;Genome file size: 3205385489 bytes; state: good=1 eof=0 fail=0 bad=0&#xA;Loading Genome ... cat: .: 디렉터리입니다&#xA;-----------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I googled it but it failed to solve the issue. What should I do next? Do you have any idea?&lt;/p&gt;&#xA;&lt;p&gt;Thank you for helping me, toddler.&lt;/p&gt;&#xA;&lt;p&gt;Additional information:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the issue seemed to be related to annotation, but MD5 for both fastq and reference file is correct.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Here are the errors related to my problem:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; (1) WARNING --outSAMattrRG defines a read group, therefore STAR will output RG attribute&#xA; Finished loading and checking parameters&#xA; Shared memory is not used for genomes&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8657" LastEditorUserId="73" LastEditDate="2021-07-14T10:52:17.270" LastActivityDate="2021-07-14T10:52:17.270" Title="10X cellranger error during count" Tags="|cellranger|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13015" PostTypeId="2" ParentId="13013" CreationDate="2020-04-21T18:52:30.203" Score="1" Body="&lt;p&gt;If you haven't already done so I would make a complete installation of Anaconda, particularly good with bash. The I would either install using Anaconda Navigator, or ...&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Create an R environment and activate it&#xA;conda create -n r_env r-essentials r-base python=3.7&#xA;conda activate r_env&#xA;&#xA;# install Reurat&#xA;conda install -c bioconda r-seurat&#xA;&#xA;# installing standard packages&#xA;conda install -c r r-randomforest --yes&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you have a Mac the instructions are &lt;a href=&quot;https://www.guru99.com/download-install-r-rstudio.html#4&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. In some systems you need to just &lt;code&gt;activate r_env&lt;/code&gt; . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Please note if you've installed e.g. &lt;code&gt;python3&lt;/code&gt; with &lt;strong&gt;&lt;code&gt;pip&lt;/code&gt;&lt;/strong&gt; a conflict occurs with the Anaconda package. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-04-21T18:59:02.023" LastActivityDate="2020-04-21T18:59:02.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13017" PostTypeId="1" AcceptedAnswerId="13022" CreationDate="2020-04-21T20:53:46.003" Score="2" ViewCount="561" Body="&lt;p&gt;I'm trying to use Seurat for the first time and am learning about single-cell analysis for the first time, and I'm doing so with split-seq data. (Full disclosure: I'm also a lightweight when it comes to R.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When Read10X didn't work on the mtx and associated files output by the split-seq-pipeline no matter how much I tried to make it look exactly like a 10x dataset (based on numerous examples and descriptions), I tried converting my sparse market matrix format into a dense tsv format (see update below for a sample of the file) and I read it in using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dge_data &amp;lt;- as.matrix(read.table(&quot;DGE.tsv&quot;, sep=&quot;\t&quot;, header=TRUE))&#xA;pbmc &amp;lt;- CreateSeuratObject(counts = dge_data, min.cells = 3, min.features  = 200, project = &quot;SPLITSEQ_TT&quot;, assay = &quot;RNA&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I started following chapter 9 of the scRNA-Seq course I found &lt;a href=&quot;https://scrnaseq-course.cog.sanger.ac.uk/website/seurat-chapter.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I ran into some issues which I was able to resolve by tweaking my mtx to tsv conversion script, such as including the chromosome in the row names (so that the mitochondrial QC step would work), but I ran into some errors that I assume have to do with the format of the gene names/IDs and possibly some issues stemming from cell names/IDs (which I've thus far been able to work through I think).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, given errors like the following, and the fact that I can't seem to find specifications on these sorts of row/column name requirements (e.g. no commas and must include at least the &quot;MT&quot; chromosome name for QC steps using mitochondrial genes), I was wondering if I could be directed to somewhere where those specifications are laid bare. For the most part, the gene names have not been a problem, but I run afoul in this step (where it appears to complain about a comma):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; FeatureScatter(object = pbmc, feature1 = &quot;hg38-4,ENSG00000074966,TXK&quot;, feature2 = &quot;PC_1&quot;)&#xA;Error in parse(text = x) : &amp;lt;text&amp;gt;:1:7: unexpected ','&#xA;1: hg38.4,&#xA;          ^&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also, when I look at how CellsByIdentities behaves (while I was working through other errors/warnings), I realized I'm not sure that the column names are being processed correctly due to the underscores (followed by a number (e.g. &lt;code&gt;_0&lt;/code&gt;)), because my understanding is that the &lt;code&gt;_#&lt;/code&gt; portion of the split-seq cell barcode indicates the first well barcode, which would be needed to uniquely identify the cells in the table. I had retrieved the cell barcode IDs from the cell_metadata.csv file output by split-seq. Should I modify these column names? Because I infer from this output from CellsByIdentities that it's only using the sequence portion of the cell barcodes to uniquely identify cells, which I don't think is correct:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; CellsByIdentities(object=pbmc, cells=colnames(pbmc))&#xA;$AAACATCGAAACATCG&#xA;[1] &quot;AAACATCGAAACATCG_0&quot; &quot;AAACATCGAAACATCG_1&quot;&#xA;&#xA;$AAACATCGAACGTGAT&#xA;[1] &quot;AAACATCGAACGTGAT_0&quot;&#xA;&#xA;$AACGTGATAAACATCG&#xA;[1] &quot;AACGTGATAAACATCG_0&quot; &quot;AACGTGATAAACATCG_1&quot;&#xA;&#xA;$AACGTGATAACGTGAT&#xA;&#xA;[1] &quot;AACGTGATAACGTGAT_0&quot; &quot;AACGTGATAACGTGAT_1&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The other assumptions have to do with data size. I was running a small set of test data, and I had to adjust some parameters. So I would like to know if there's a hard lower limit on the data. And do I need to restructure my data to separate counts of genes from different species (e.g. mouse &amp;amp; human)? I.e. Should I create separate matrices that have only the human-identified cells and gene annotations to use it with Seurat?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't seem to find these specifications, presumably because Seurat was designed exclusively around 10x data and that the tool wasn't tested on data from other sources.  Perhaps I'm worrying too much, but without knowing what may rely on column/row name format assumptions, I can't be sure my data is being analyzed properly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is the head of the tsv file I originally tried to load when I ran into various errors that I suspect were due to the formatting of the row and column names (note, there are only 7 cells total, and it's a small but real test dataset from our core facility):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AAACATCGAAACATCG_0  AAACATCGAAACATCG_1  AAACATCGAACGTGAT_0  AACGTGATAAACATCG_0  AACGTGATAAACATCG_1  AACGTGATAACGTGAT_0  AACGTGATAACGTGAT_1&#xA;hg38_X,ENSG00000000003,TSPAN6   8   2   2   26  15  17  13&#xA;hg38_X,ENSG00000000005,TNMD 0   0   0   0   0   2   0&#xA;hg38_20,ENSG00000000419,DPM1    22  26  27  73  43  50  28&#xA;hg38_1,ENSG00000000457,SCYL3    6   6   5   12  7   19  8&#xA;hg38_1,ENSG00000000460,C1orf112 23  10  24  59  34  49  21&#xA;hg38_1,ENSG00000000971,CFH  101 44  129 331 151 270 122&#xA;hg38_6,ENSG00000001036,FUCA2    33  28  24  82  58  74  36&#xA;hg38_6,ENSG00000001084,GCLC 30  16  14  72  27  54  42&#xA;hg38_6,ENSG00000001167,NFYA 14  9   17  41  28  31  12&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Yesterday, I did end up trying to change the column and row names to the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;AAACATCGAAACATCG-0  AAACATCGAAACATCG-1  AAACATCGAACGTGAT-0  AACGTGATAAACATCG-0  AACGTGATAAACATCG-1  AACGTGATAACGTGAT-0  AACGTGATAACGTGAT-1&#xA;hg38-X.ENSG00000000003.TSPAN6   8   2   2   26  15  17  13&#xA;hg38-X.ENSG00000000005.TNMD 0   0   0   0   0   2   0&#xA;hg38-20.ENSG00000000419.DPM1    22  26  27  73  43  50  28&#xA;hg38-1.ENSG00000000457.SCYL3    6   6   5   12  7   19  8&#xA;hg38-1.ENSG00000000460.C1orf112 23  10  24  59  34  49  21&#xA;hg38-1.ENSG00000000971.CFH  101 44  129 331 151 270 122&#xA;hg38-6.ENSG00000001036.FUCA2    33  28  24  82  58  74  36&#xA;hg38-6.ENSG00000001084.GCLC 30  16  14  72  27  54  42&#xA;hg38-6.ENSG00000001167.NFYA 14  9   17  41  28  31  12&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And it seems to have fixed most of the issues.  The violin plots now have curvy brown swaths down the center that were not there before.  And in particular, when I run CellsByIdentities manually on the column names now, I get what looks correct (as opposed to what I pasted yesterday above (which appeared to group the 7 cells in 3 pairs and one individual):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CellsByIdentities(object = pbmc, cells = colnames(pbmc))&#xA;$SPLITSEQ_TT&#xA;[1] &quot;AAACATCGAAACATCG.0&quot; &quot;AAACATCGAAACATCG.1&quot; &quot;AAACATCGAACGTGAT.0&quot; &quot;AACGTGATAAACATCG.0&quot; &quot;AACGTGATAAACATCG.1&quot; &quot;AACGTGATAACGTGAT.0&quot;&#xA;[7] &quot;AACGTGATAACGTGAT.1&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think that the remaining issues are likely due to the small data size.  For reference as to the data size, this is the first data line from the original mtx file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;7 38526 194539&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Please note also that I'm not looking for how to get this particular dataset to work.  I am (as a member of the bioinformatics team that supports users of our core sequencing facility) trying to establish the requirements for the conversion script so that when we deliver SPLiT-Seq results, our output can be directly used with Seurat by the users without issues or need for further conversion.  So I specifically need to know what makes row or column names valid or invalid, as opposed to what to change WRT this specific dataset.  We may be supplied with annotations from various sources that may have different characters in the gene symbols, or accession numbers, or even chromosome names.&lt;/p&gt;&#xA;" OwnerUserId="5063" LastEditorUserId="5063" LastEditDate="2020-04-22T14:38:48.707" LastActivityDate="2023-09-04T20:02:20.297" Title="What column and row naming requirements exist with Seurat (context: when loading SPLiT-Seq data)" Tags="|scrnaseq|seurat|splitseq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13018" PostTypeId="2" ParentId="12979" CreationDate="2020-04-21T20:58:25.367" Score="1" Body="&lt;p&gt;I had the same problem and I was able to solve it with that&#xA;&lt;a href=&quot;https://github.com/NBISweden/GAAS/blob/master/annotation/knowledge/gff_to_gtf.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/NBISweden/GAAS/blob/master/annotation/knowledge/gff_to_gtf.md&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6694" LastActivityDate="2020-04-21T20:58:25.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13020" PostTypeId="2" ParentId="11741" CreationDate="2020-04-22T01:00:35.967" Score="1" Body="&lt;p&gt;To filter your gene_exp.diff either you export it to excel and then filter because it is columns separated by tabs or use the command (awk and head -n) to filter and have the number of lines you want if you master Linux&lt;/p&gt;&#xA;" OwnerUserId="6694" LastActivityDate="2020-04-22T01:00:35.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13022" PostTypeId="2" ParentId="13017" CreationDate="2020-04-22T07:06:36.160" Score="1" Body="&lt;p&gt;&lt;code&gt;Read10X()&lt;/code&gt; works on &lt;code&gt;mtx&lt;/code&gt; files, that is not the root of your problem. The problem is that you are pushing non-10x-data to look and work like 10x-data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can just use &lt;code&gt;CreateSeuratObject()&lt;/code&gt; and feed your count matrix to this function. I am not sure if this particular function accepts &lt;code&gt;mtx&lt;/code&gt; files, could not find this in the documentation, but in the worst case you will just need to supply a plain count matrix, count matrix being genes as rows and cells as columns.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And that could be a good idea to show the first few rows and columns of your &lt;code&gt;SPLIT-Seq&lt;/code&gt; output so that we can have and idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regarding gene identifiers, I would stick with gene symbols, to help you in cluster annotation but also using the &quot;MT&quot; prefix for mitochondrial genes for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like you are providing somewhat obscure identifiers as gene identifiers, a mix of genome build, ensemble gene id and gene symbol and apparently &lt;code&gt;Seurat&lt;/code&gt; does not appreciate the &lt;code&gt;,&lt;/code&gt;. While I cannot give you a definitive recipe on what can and cannot be set as gene identifiers when using Seurat, I can tell that I have worked both with gene symbols and Ensembl identifiers, separately, and both was fine with Seurat. &lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2020-04-22T11:38:12.477" LastActivityDate="2020-04-22T11:38:12.477" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="13023" PostTypeId="1" AcceptedAnswerId="13062" CreationDate="2020-04-22T08:10:14.683" Score="1" ViewCount="113" Body="&lt;p&gt;I've got a DNA sequence that I'd like to make synonymous mutations throughout, thereby preserving the amino acid sequence. Does anyone know of a tool to achieve this&lt;/p&gt;&#xA;" OwnerUserId="8682" LastEditorUserId="3848" LastEditDate="2020-04-24T13:05:17.020" LastActivityDate="2020-11-27T14:54:18.717" Title="Is there a tool for synonymous recoding of DNA sequences?" Tags="|r|bioconductor|mutations|synonymous-mutations|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13025" PostTypeId="2" ParentId="11283" CreationDate="2020-04-22T10:50:35.817" Score="0" Body="&lt;p&gt;SurpriseDog - I suspect the wording &quot;unlikely to be fortuitous in nature&quot; is what led you to ask &quot;Does this mean the virus is artificially constructed?&quot; - No!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the writing here is imprecise and confusing. There is no implication that these are human-generated (non-natural/engineered) insertions but rather I think they are implying that these mutations are likely to have functional implications. Whether there is any relation to HIV is questionable.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There has been some political finger pointing and threats of full-scale investigation into whether the virus escaped from a lab. Hard to say, but current thinking is that SARS-Cov-2 is a natural isolate, not engineered (see &lt;a href=&quot;https://www.nature.com/articles/s41591-020-0820-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41591-020-0820-9&lt;/a&gt;). It was probably transmitted from bats. &lt;/p&gt;&#xA;" OwnerUserId="8685" LastActivityDate="2020-04-22T10:50:35.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13029" PostTypeId="1" AcceptedAnswerId="13037" CreationDate="2020-04-22T16:53:05.570" Score="1" ViewCount="575" Body="&lt;p&gt;I will be docking a small library of molecules that I have downloaded from &lt;strong&gt;PubChem&lt;/strong&gt; using &lt;strong&gt;AutoDock&lt;/strong&gt;. I thought why not &lt;strong&gt;minimize their energy&lt;/strong&gt; before docking using &lt;strong&gt;ChemDraw&lt;/strong&gt;. I am thinking; what is the best method of &lt;strong&gt;optimization/ energy minimization&lt;/strong&gt; of small molecules. &lt;strong&gt;MM2&lt;/strong&gt;, &lt;strong&gt;MMF94&lt;/strong&gt; or other? And why?&lt;/p&gt;&#xA;" OwnerUserId="4921" LastActivityDate="2020-04-23T12:10:41.410" Title="What is the perferred method of optimization or energy minimization of small molecules downloaded from PubChem?" Tags="|protein-structure|docking|software-usage|3d-structure|structural-biology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13030" PostTypeId="1" CreationDate="2020-04-22T17:53:51.583" Score="2" ViewCount="111" Body="&lt;p&gt;so I'm having a problem using &lt;a href=&quot;http://www.htslib.org/doc/samtools-phase.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools phase&lt;/a&gt;. I'm trying to phase my .bam archives that where generated by aligning my samples with my reference genome. Following the documentation I tried the command: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;samtools phase -A  my_referenceGenome.fasta   my_sample_toBePhased.bam&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The -A flag stands for dropping reads with ambiguous phase. The &lt;strong&gt;error&lt;/strong&gt; I get is: &lt;code&gt;[W::sam_read1] Parse error at line 1&lt;/code&gt;. I generated .bam files with and without the SAM header but the error persists.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I feel like I am missing something basic but that isn't clearly stated anywere. Does the &lt;code&gt;my_referenceGenome.fasta&lt;/code&gt; archive needs to be previously phased? If so, how can I do it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you so much!&lt;/p&gt;&#xA;" OwnerUserId="8665" LastActivityDate="2020-04-22T23:31:54.473" Title="Phasing problem using &quot;samtools phase&quot;" Tags="|sam|fasta|samtools|phase|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13031" PostTypeId="2" ParentId="3956" CreationDate="2020-04-22T20:10:31.257" Score="3" Body="&lt;p&gt;If you're interested in variant calling, I just co-authored an O'Reilly book called &lt;a href=&quot;https://oreil.ly/genomics-cloud&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genomics in the Cloud&lt;/a&gt; that covers the GATK Best Practices from a scientific standpoint (germline and somatic short variants + somatic copy number alterations), as well as technical considerations like how to run it efficiently at scale (that's where the cloud part comes in). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can read more about it on the GATK blog; there's an &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360042408831&quot; rel=&quot;nofollow noreferrer&quot;&gt;announcement&lt;/a&gt; and a second &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360042106212&quot; rel=&quot;nofollow noreferrer&quot;&gt;more personal blog post&lt;/a&gt; that I wrote to explain how I approached the topic. Happy to discuss more if you're interested.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Oh and you can preview the first few chapters for free &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/1491975199&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;on Amazon&lt;/a&gt; if you like. &lt;/p&gt;&#xA;" OwnerUserId="2776" LastActivityDate="2020-04-22T20:10:31.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13032" PostTypeId="2" ParentId="11700" CreationDate="2020-04-22T23:24:52.333" Score="3" Body="&lt;p&gt;The main issue here is that your bam files have different chromosome labels. (i.e. 1,2,3 vs. chr1,chr2,chr3) as you mentioned.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This suggests that the data was mapped to a slightly different version of the reference genome. As a first layer, you might want to double check that these reference chromosomes are cross compatible (e.g. both correspond to hg19 for instance). To find more information on the reference genome used, you can check the header information using &quot;samtools view -H &quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Assuming that the reference genomes are cross compatible, you can try converting it to sam, adding the missing &quot;chr&quot; label in a pipe, and converting it back to a bam file. However, this will also require modification of the header file which can be challenging in itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The easiest (and more rigorous) approach is probably just to convert the batch of bam files with the wrong chromosome labels back into fastq files. You will then want to remap them using the exact same reference genome as the other batch of bam files. After which, you should be able to perform variant calling on the combined batch of bam files.&lt;/p&gt;&#xA;" OwnerUserId="7349" LastActivityDate="2020-04-22T23:24:52.333" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13033" PostTypeId="2" ParentId="13030" CreationDate="2020-04-22T23:31:54.473" Score="2" Body="&lt;p&gt;This is what I see in my samtools phase (v1.10)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Usage:   samtools phase [options] &amp;lt;in.bam&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;samtools phase only takes a bam file as an input. But you are specifying a .fasta file as an input. If you want to specify the reference genome, you should be using the argument &quot;--reference&quot; as described here&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;      --reference FILE&#xA;           Reference sequence FASTA FILE [null]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7349" LastActivityDate="2020-04-22T23:31:54.473" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13034" PostTypeId="1" AcceptedAnswerId="13035" CreationDate="2020-04-23T02:24:01.370" Score="1" ViewCount="668" Body="&lt;p&gt;i have a question when i want to extract cds sequence using gene id. but cds file is not just start with &gt;gene is, it has many other annotation. the only same  is star with gene:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;cds fasta:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Zm002 cds gene:Zm1d035916 gene_biotype:protein_coding &#xA;ATCGGCAT&#xA;&amp;gt;Zm001 cds RefGen_v4:9:153880862:153883850:-1 gene:Zm1d048 gene_biotype:protein_coding&#xA;ATGCGGCA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;gene_list&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Zm1d035916&#xA;Zm1d048 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;how to get result like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Zm1d035916&#xA;ATCGGCAT&#xA;&amp;gt;Zm1d048 &#xA;ATGCGGCA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8692" LastActivityDate="2020-04-23T11:16:44.747" Title="how to use list of gene id to get cds sequence(cds fasta file have many annotation, only gene id: is same to query id)" Tags="|python|genome|sequence-analysis|perl|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13035" PostTypeId="2" ParentId="13034" CreationDate="2020-04-23T07:09:12.713" Score="2" Body="&lt;p&gt;If you can use &lt;code&gt;awk&lt;/code&gt;, you could use this simple script:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '&#xA;BEGIN {p=0}&#xA;NR==FNR {gid[$1];next}&#xA;/^&amp;gt;/{&#xA;  p=0;&#xA;  for(g in gid){&#xA;    re=&quot;\\sgene:&quot;g&quot;\\s&quot;&#xA;    if($0~re){p=1;break}&#xA;  }&#xA;}p' gene_list cds.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can also put the code inside a script file:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;extract_genes.awk&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/awk&#xA;BEGIN {&#xA;  p = 0; # p indicates whether to print a line or not&#xA;}&#xA;# while scanning the first file, store gene ids in an array&#xA;NR==FNR {&#xA;  gid[$1];&#xA;  next&#xA;}&#xA;# check FASTA headers&#xA;/^&amp;gt;/ {&#xA;  p = 0;&#xA;  for(g in gid) {&#xA;    # construct regular expression to match gene id in FASTA header&#xA;    re = &quot;\\sgene:&quot;g&quot;\\s&quot;;&#xA;    if ($0~re) { # a match was found!&#xA;      p = 1;&#xA;      break;&#xA;    }&#xA;  }&#xA;}&#xA;p # print line or not?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And call it like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk -f extract_genes.awk gene_list cds.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8629" LastActivityDate="2020-04-23T07:09:12.713" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="13037" PostTypeId="2" ParentId="13029" CreationDate="2020-04-23T11:09:10.200" Score="2" Body="&lt;p&gt;&lt;strong&gt;The difference between different force-fields is not going to be major, it is the side steps which are.&lt;/strong&gt; &lt;/p&gt;&#xA;&#xA;&lt;h2&gt;When&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;If you are starting from a SMILES string, optimisation is a must obviously.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are using a 3D conformer from PubChem or even an actual sub-1 Å crystal structure from CSD, optimisation is nice for consistency.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Which&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;MMF94 is a solid choice.&lt;/strong&gt; RDkit offers MMF94 (&lt;a href=&quot;https://en.wikipedia.org/wiki/Merck_molecular_force_field&quot; rel=&quot;nofollow noreferrer&quot;&gt;Merck force field 1994&lt;/a&gt;) and UFF (&lt;a href=&quot;https://en.wikipedia.org/wiki/Universal_force_field&quot; rel=&quot;nofollow noreferrer&quot;&gt;universal force field&lt;/a&gt;). The latter has some extra features, such as not kicking the bucket at atoms that aren't your usual suspects. ChemDraw offers MM2 and MMF94 IIRC, the former is very old. g16 in Gaussian (paid software) uses Gaussian Electrostatic Model, which is polarisable and will solve partial charges in a nice way (which I'll get to in a minute). OpenBabel uses MMF94 under the hood. You can even optimise with macromolecular forcefields, such as AMBER and CHARMM.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Given that the AutoDock uses a flexible ligand it should not matter too much anyway.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;What else?&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Autodock does not require conformers by virtue of the flexible ligand business (the sampler does rototorsions). Other programs might if they use rigid ligands (e.g. Fred) or they use a mixed approach and use a conformer library for speed (e.g. Rosetta ligand_dock). Conformer diversity is actually a bigger issue as conformers that are only slightly contorted, often even less than kB (0.6 kcal/mol), may not be sampled.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing that messes things is &lt;strong&gt;incorrect protonation&lt;/strong&gt;. OpenBabel can protonate at a given pH, RDKit cannot. Not an issue for ligands with drug screens (cf. Lipinski rule), but is for natural substrates. However, for the protein it is essential to make sure acidic and basic residue are correctly protonated and that histidine have the correct tautomer —there are tools, c.f. MOE, that can do it but by far the best approach is reading up on the mechanism. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The next thing is &lt;strong&gt;dodgy Gasteiger Marsili partial charges&lt;/strong&gt; —for Autodock 4 (and most other full/hybrid physics-based docking programs), partial charges are important: after all PDBQT has a Q in it for charge! For &lt;code&gt;prepare_ligand4.py&lt;/code&gt; you give it a mol2 file. A major consideration is what format you are using for your molecules. This sounds a very silly thing to point out, but you'd be surprised how many trip up. An mol file (or an sdf file, which is the same but multi-entry) does not have an official partial charge column (although they are often added as comments after &lt;code&gt;M END&lt;/code&gt;). So whatever force-field you chose make sure that the program you use gave you partial charges are not formal charges. This does not quite apply to AutoDock Vina, which is charge independent and more reliant on machine learning derived factors.&lt;/p&gt;&#xA;&#xA;&lt;h2&gt;Verdict&lt;/h2&gt;&#xA;&#xA;&lt;p&gt;Open babel is nice in that it can start with a SMILES, handles multiple formats including mol2 and PDBQT, can protonate at pH 7, uses MMF94 and does Gasteiger Marsili charges (and its confab algorithm is very good, not need here though).&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-04-23T12:10:41.410" LastActivityDate="2020-04-23T12:10:41.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13038" PostTypeId="2" ParentId="13034" CreationDate="2020-04-23T11:16:44.747" Score="1" Body="&lt;p&gt;If you read it in python, they would take care of that:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO                                                                 &#xA;records = list(SeqIO.parse(&quot;test.fa&quot;,&quot;fasta&quot;)) &#xA;&#xA;records&#xA;&#xA;[SeqRecord(seq=Seq('ATCGGCAT', SingleLetterAlphabet()), id='Zm002', name='Zm002', description='Zm002 cds gene:Zm1d035916 gene_biotype:protein_coding', dbxrefs=[]),&#xA; SeqRecord(seq=Seq('ATGCGGCA', SingleLetterAlphabet()), id='Zm001', name='Zm001', description='Zm001 cds RefGen_v4:9:153880862:153883850:-1 gene:Zm1d048 gene_biotype:protein_coding', dbxrefs=[])]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Get the gene and sequence.. maybe there's a easier to do this in biopython, this is like my 10th time using it:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def format(rec):&#xA;    des = rec.description.split()&#xA;    gene = [entry[5:] for entry in des if &quot;gene:&quot; in entry][0]&#xA;    return(&quot;&amp;gt;&quot;+gene+&quot;\n&quot;+str(rec.seq))&#xA;&#xA;[format(rec) for rec in records]&#xA;&#xA;['&amp;gt;Zm1d035916\nATCGGCAT', '&amp;gt;Zm1d048\nATGCGGCA']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Write it out:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f=open('example.fasta','w')&#xA;f.writelines(&quot;\n&quot;.join([format(rec) for rec in records]))&#xA;f.close()&#xA;&#xA;list(SeqIO.parse(&quot;example.fasta&quot;,&quot;fasta&quot;))                                           &#xA;[SeqRecord(seq=Seq('ATCGGCAT', SingleLetterAlphabet()), id='Zm1d035916', name='Zm1d035916', description='Zm1d035916', dbxrefs=[]),&#xA; SeqRecord(seq=Seq('ATGCGGCA', SingleLetterAlphabet()), id='Zm1d048', name='Zm1d048', description='Zm1d048', dbxrefs=[])]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-04-23T11:16:44.747" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13039" PostTypeId="2" ParentId="10559" CreationDate="2020-04-23T15:34:43.573" Score="2" Body="&lt;p&gt;This sounds more like a job for &lt;code&gt;samtools split&lt;/code&gt; if you want to split out all the read groups into separate bams in one go. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.htslib.org/doc/samtools-split.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.htslib.org/doc/samtools-split.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;samtools split [options] merged.sam|merged.bam|merged.cram&#xA;&#xA;Splits a file by read group.&#xA;&#xA;Options:&#xA;&#xA;-u FILE1&#xA;Put reads with no RG tag or an unrecognised RG tag into FILE1&#xA;&#xA;-u FILE1:FILE2&#xA;As above, but assigns an RG tag as given in the header of FILE2&#xA;&#xA;-f STRING&#xA;Output filename format string (see below) [&quot;%*_%#.%.&quot;]&#xA;&#xA;-v&#xA;Verbose output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Alternatively if you do want to proceed one read group at a time, in addition to &lt;code&gt;samtools view&lt;/code&gt;, you can also do this with GATK using the PrintReads tool with the ReadGroupReadFilter as documented here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360041415172-ReadGroupReadFilter&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360041415172-ReadGroupReadFilter&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The command line would look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gatk PrintReads \&#xA;   -I input.bam \&#xA;   --read-filter ReadGroupReadFilter \&#xA;   --keep-read-group &amp;lt;RG:readgroup&amp;gt; \&#xA;   -O output.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;There's also a filter called ReadGroupBlackListReadFilter that allows you to exclude a list of readgroups.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360041851151-ReadGroupBlackListReadFilter&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360041851151-ReadGroupBlackListReadFilter&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The PrintReads tool doc is here: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360041849911-PrintReads&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360041849911-PrintReads&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2776" LastActivityDate="2020-04-23T15:34:43.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13041" PostTypeId="2" ParentId="3401" CreationDate="2020-04-23T15:56:08.440" Score="3" Body="&lt;p&gt;You can do this more easily with the GATK tool SelectVariants, which allows you to subset variants based on various criteria. Here's the doc page:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360041849831-SelectVariants&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360041849831-SelectVariants&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this case (subsetting by chromosome) you would simply do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gatk SelectVariants \&#xA;     -R reference.fasta \&#xA;     -V input.vcf \&#xA;     -L chr2 \&#xA;     -O output.chr2.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you wanted to do this for several chromosomes, or regions within chromosomes, you would give the tool a file of intervals with &lt;code&gt;-L&lt;/code&gt; instead of the chromosome name. Suitable interval file formats are described here: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035531852-Intervals-and-interval-lists&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360035531852-Intervals-and-interval-lists&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2776" LastActivityDate="2020-04-23T15:56:08.440" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13042" PostTypeId="2" ParentId="654" CreationDate="2020-04-23T17:17:06.153" Score="5" Body="&lt;p&gt;The GATK is probably the most widely used program for this. There are several workflows you can use depending on what types of variants you're interested in (germline or somatic, short variants, copy number, structural). The most common use case is Germline short variant discovery (SNPs and Indels), for which the overall workflow is described here: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035535932&quot; rel=&quot;noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360035535932&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The GATK team at the Broad Institute now provides workflow scripts written in WDL (Worfklow Description Language) that can be run on pretty much any platform with the Cromwell engine. The workflows include data pre-processing, including mapping (alignment) and some important cleanup operations, plus variant calling and filtering. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;One of the advantages of using those workflows is that the Broad uses them in production for all the data they sequence, which nowadays amounts to abut a new human whole genome every five minutes. That means there's a lot of pressure to make sure the tools are reliable and efficient because any interruption of the pipelining operations would be a big problem. They're also highly standardized so that the outputs will be compatible with other datasets and tools. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Disclosure: I work at the Broad Institute and wrote a book about using GATK (&lt;a href=&quot;https://oreil.ly/genomics-cloud&quot; rel=&quot;noreferrer&quot;&gt;https://oreil.ly/genomics-cloud&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="2776" LastActivityDate="2020-04-23T17:17:06.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13044" PostTypeId="2" ParentId="4575" CreationDate="2020-04-23T17:49:55.777" Score="0" Body="&lt;p&gt;For your second question, I don't think that variant matches any of the examples shown in Figure 1c. Note that that panel's caption says it shows four examples of arbitration. Per you last sub-question, I don't think this variant was subjected to arbitration. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your second question, I believe the subtlety is in the field descriptions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;ADALL = Net allele depths across all datasets&lt;/p&gt;&#xA;&#xA;&lt;p&gt;AD = Net allele depths across all unfiltered datasets &#xA;&lt;strong&gt;with called genotype&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My understanding is that ADALL is a count of allele depths that includes samples where the variant was filtered out or the genotype was not concordant; something to that effect. So generally I would expect ADALL &gt; AD. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm honestly not sure exactly what they mean by &quot;unfiltered datasets&quot;. That could mean &quot;counting all reads without filtering on quality&quot; or &quot;callsets to which variant filtering was not yet applied&quot;. Maybe there's another subtlety there that explains why the two variants have opposite AD vs ADALL proportions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For your additional questions:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Why are these two variants a PASS if there are 3 “callable” datasets in regions of low coverage?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;According to the paper, having the variant in low-coverage regions isn't disqualifying if the genotype calls are concordant, which makes sense to me. Low coverage makes me more suspicious, but if multiple callsets are in agreement, I'm still willing to consider the call good (unless of course they all exhibit the same error mode, which is the main potential weakness of the consensus approach used by GiaB).&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Did the variants follow an arbitration process? I guess not, if the INFO flag “arbitrated” is missing.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Right, if that flag is not present, it means arbitration was not needed/applied. &lt;/p&gt;&#xA;" OwnerUserId="2776" LastActivityDate="2020-04-23T17:49:55.777" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13045" PostTypeId="2" ParentId="3307" CreationDate="2020-04-23T17:54:52.713" Score="0" Body="&lt;p&gt;The variant record in the VCF produced by GATK should include information about depth -- both total read depth (DP) and allelic depth (AD), ie depth per allele. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Keep in mind however that the caller makes its decision based on more than just allelic depth. There are some additional statistics involved, including the quality of the individual bases. Also, if any filtering was applied, your variant may have been excluded due to other statistics, like for example if there is evidence of strand bias -- eg if all the reads supporting your variant are in the same orientation, that is a classic red flag suggesting it's an artifact.&lt;/p&gt;&#xA;" OwnerUserId="2776" LastActivityDate="2020-04-23T17:54:52.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13047" PostTypeId="2" ParentId="5450" CreationDate="2020-04-23T18:59:03.943" Score="1" Body="&lt;p&gt;At the Broad Institute, we do something similar to what is described by user172818. We use uBAM (unmapped BAM) to store unmapped reads, because as you say the SAM format has a more appropriate way to encode metadata. However, we use a Picard tool called SamToFastq to do the initial conversion then pipe the output of that to BWA mem. Then we use an additional step to reattach metadata, read group info and so on, using a Picard tool called MergeBamAlignments. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see the full implementation here (starts as line 71): &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/gatk-workflows/gatk4-genome-processing-pipeline/blob/master/tasks/Alignment.wdl&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/gatk-workflows/gatk4-genome-processing-pipeline/blob/master/tasks/Alignment.wdl&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2776" LastActivityDate="2020-04-23T18:59:03.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13048" PostTypeId="2" ParentId="7220" CreationDate="2020-04-23T19:37:24.063" Score="0" Body="&lt;p&gt;We've seen this error before on the GATK forum; if you google &quot;bin field of BAM record does not equal value computed based on alignment start and end&quot; you will find this forum thread: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gatkforums.broadinstitute.org/gatk/discussion/4290/sam-bin-field-error-for-the-gatk-run&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatkforums.broadinstitute.org/gatk/discussion/4290/sam-bin-field-error-for-the-gatk-run&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This part of the last post should help you solve this:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;The only way I can think of to fix this is to fix the BAM. In order to&#xA;  do that, you would need to convert to SAM format, which does not have&#xA;  a field for indexing bin, and then back to BAM. That would be the&#xA;  safest thing, followed by recreating the BAI. So you can do this with&#xA;  GATK tools using the following commands.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gatk SamFormatConverter - to convert Bam to Sam&#xA;&#xA;gatk SamFormatConverter - to convert Sam back to Bam&#xA;&#xA;gatk BuildBamIndex - to create the Bam index file.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;  &#xA;  &lt;p&gt;Also, please make sure your Picardtools and GATK versions are up to&#xA;  date to the most recent versions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2776" LastActivityDate="2020-04-23T19:37:24.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13049" PostTypeId="2" ParentId="11154" CreationDate="2020-04-23T20:00:51.990" Score="2" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;What is the difference of ReadCount in INFO column and DP in FORMAT column?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I'm not super familiar with Strelka but I believe it follows the VCF spec, so DP in the FORMAT column should give the depth of coverage (ie read count) per sample. ReadCount in INFO is not in the VCF spec but based on the definition it sounds like the depth of coverage aggregated over all samples. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;What is the difference of number of allele versus number of reads covering the variant position ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I believe the &quot;number of alleles used&quot; refers to counts of each base that are used in the indel error modeling process as described in the preprint here: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biorxiv.org/content/10.1101/192872v2.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/10.1101/192872v2.full&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it sounds like subsets of the number of reads covering the variant position.&lt;/p&gt;&#xA;" OwnerUserId="2776" LastActivityDate="2020-04-23T20:00:51.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13050" PostTypeId="2" ParentId="7448" CreationDate="2020-04-23T20:19:53.937" Score="1" Body="&lt;p&gt;You can do the extraction part with the GATK tool VariantsToTable, as described here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360041414592-VariantsToTable&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360041414592-VariantsToTable&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The usage example from that doc: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gatk VariantsToTable \&#xA;    -V input.vcf \&#xA;    -F CHROM -F POS -F TYPE -GF AD \&#xA;    -O output.table&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;would produce a file that looks like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt; CHROM  POS        TYPE   HSCX1010N.AD  HSCX1010T.AD&#xA; 1      31782997   SNP    77,0          53,4&#xA; 1      40125052   SNP    97,0          92,7&#xA; 1      65068538   SNP    49,0          35,4&#xA; 1      111146235  SNP    69,1          77,4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So you might still need to reorder columns etc but that should allow you to at least get the values out in a tabular format that will be easier to work with.&lt;/p&gt;&#xA;" OwnerUserId="2776" LastActivityDate="2020-04-23T20:19:53.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13051" PostTypeId="1" AcceptedAnswerId="13058" CreationDate="2020-04-23T21:28:07.903" Score="3" ViewCount="1847" Body="&lt;p&gt;I have 40 samples and these are into two groups. I would like to perform a differential analysis between two groups. The library size of the samples is very low. But there are two samples (GroupA_12 and GroupB_2) in which the number of counts is too large compared to other samples. Please check the library sizes below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eI2m8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eI2m8.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I wanted to downsample these two cases and then would like to perform differential analysis. May I know how to downsample those two samples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help is appreciated. thanq.&lt;/p&gt;&#xA;" OwnerUserId="1377" LastActivityDate="2020-04-24T07:07:45.570" Title="How to downsample some of the samples in RNA-seq data?" Tags="|r|rna-seq|differential-expression|gene-expression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13052" PostTypeId="2" ParentId="11108" CreationDate="2020-04-23T21:29:10.223" Score="3" Body="&lt;p&gt;Just starting to work with bwa fastmap and indeed the documentation is very sparse...&#xA;This is what we get from running the program without arguments:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bwa fastmap&#xA;&#xA;Usage:   bwa fastmap [options] &amp;lt;idxbase&amp;gt; &amp;lt;in.fq&amp;gt;&#xA;&#xA;Options: -l INT    min SMEM length to output [17]&#xA;         -w INT    max interval size to find coordiantes [20]&#xA;         -i INT    min SMEM interval size [1]&#xA;         -L INT    max MEM length [2147483647]&#xA;         -I INT    stop if MEM is longer than -l with a size less than INT [0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Which is a bit cryptic as is...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output does not really follow SAM specification and I'm pretty sure the '*' means does not mean unmapped (in fact an unmapped Exact Match doesn't make any sense does it?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In fact, the '*' means that there are too many matches to print the reference id and position.&#xA;Looking quickly into the source code this is controlled by the -w arguments.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It defaults to 20, which is why you don't get an output for your 24 matches.&lt;/p&gt;&#xA;" OwnerUserId="8703" LastActivityDate="2020-04-23T21:29:10.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13053" PostTypeId="5" CreationDate="2020-04-23T21:57:40.977" Score="0" Body="&lt;p&gt;The Picard toolkit is available as a standalone package from &lt;a href=&quot;https://github.com/broadinstitute/picard&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/broadinstitute/picard&lt;/a&gt; with tool documentation at &lt;a href=&quot;https://broadinstitute.github.io/picard/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://broadinstitute.github.io/picard/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;All Picard tools are also bundled in GATK versions 4.0 and later, with tool documentation included in the GATK documentation at &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/categories/360002369672&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/categories/360002369672&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Support for Picard tools is provided on the GATK support forum at &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/community/topics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/community/topics&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2776" LastEditorUserId="2776" LastEditDate="2020-04-24T06:03:25.917" LastActivityDate="2020-04-24T06:03:25.917" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13054" PostTypeId="4" CreationDate="2020-04-23T21:57:40.977" Score="0" Body="for questions about Picard; popular open-source toolkit focused primarily on sequence and variant data manipulation." OwnerUserId="2776" LastEditorUserId="57" LastEditDate="2020-04-23T22:59:44.563" LastActivityDate="2020-04-23T22:59:44.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13055" PostTypeId="2" ParentId="2644" CreationDate="2020-04-23T22:06:45.480" Score="1" Body="&lt;p&gt;I was able to find a solution for this using &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez Direct&lt;/a&gt; and SRA toolkit :) If you have the project number or &lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/?study=SRP022054&quot; rel=&quot;nofollow noreferrer&quot;&gt;SRA project number&lt;/a&gt;, yours would be SRP022054 in this case for the 36 SRAs, you can use esearch to make a query like so and pipe it into SRA toolkit with this one liner:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;esearch -db sra -query SRP022054  | efetch --format runinfo | cut -d ',' -f 1 | grep SRR | head -5 | xargs fastq-dump  --skip-technical  --readids --read-filter pass --dumpbase --split-3&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="8702" LastActivityDate="2020-04-23T22:06:45.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13057" PostTypeId="2" ParentId="2644" CreationDate="2020-04-24T03:36:28.627" Score="0" Body="&lt;p&gt;You can download an entire project using &lt;a href=&quot;https://github.com/saketkc/pysradb&quot; rel=&quot;nofollow noreferrer&quot;&gt;pysradb&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pysradb download -p SRP022054&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It retains the same schema as SRA.&lt;/p&gt;&#xA;" OwnerUserId="161" LastActivityDate="2020-04-24T03:36:28.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13058" PostTypeId="2" ParentId="13051" CreationDate="2020-04-24T07:07:45.570" Score="4" Body="&lt;p&gt;You have a few options:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Downsample the fastq files and rerun the entire analysis. You can do this with &lt;code&gt;seqtk sample&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Downsample the BAM files, which you can do with &lt;code&gt;samtools view -s&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Divide all of the counts in the counts files by some factor and round that to an integer.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I personally prefer option 2, since it's quick and doesn't usually have any side-effects unless you use two-pass alignment or did the quantification with something like salmon. Option 3 is always tempting, but I have to say I've never been convinced that this consistently produces similar enough results to options 1 or 2. &lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-04-24T07:07:45.570" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="13059" PostTypeId="2" ParentId="11733" CreationDate="2020-04-24T07:17:35.613" Score="1" Body="&lt;p&gt;So, as the first reply said, there's sort of multiple questions here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For filtering things out from a table, I would absolutely use pandas. &lt;a href=&quot;https://swcarpentry.github.io/python-novice-gapminder/08-data-frames/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://swcarpentry.github.io/python-novice-gapminder/08-data-frames/&lt;/a&gt; Pandas is a way of managing dataframes in python. It looks like what you want from your table is to remove certain rows, if the content of those rows at a certain column matches certain IDs from a list (pathogen IDs). That's the sort of thing the pandas library is good at.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For managing workflows, I use either bash or snakemake. Snakemake is a python thing. It has some advantages, such as being able to start from an intermediary file (for example, if a workflow is &quot;sequences -&gt; multiple alignment -&gt; phylogenetic tree&quot;, it can easily start from the multiple alignment). Also deals with branched workflows quite well. Can be easily installed with conda (anaconda is absolutely a lifesaver for bioinformatics work imo). I still use bash for simple workflows, because it's slightly simpler/faster.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For potential filtering - If you want to keep the top hits as they are, but filter the lower-scoring hits based on whether they have a PDB ID or not, I'd personally look into Entrez queries. They're a way of making your search more specific. Here they're shown for WebBlast, but they work quite well for command line blast too. &lt;a href=&quot;https://www.youtube.com/watch?v=bxx5uaKjMa8&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=bxx5uaKjMa8&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2020-04-24T07:17:35.613" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13060" PostTypeId="1" CreationDate="2020-04-24T07:55:40.907" Score="2" ViewCount="41" Body="&lt;p&gt;I have a lot of unassembled sequencing data and I want to build phylogeny for some genes from these data. I can retrieved the sequene by mapping reads to a reference sequence but if the two species are too divergent, reads will not match and I won't get the full CDS. Is there a better way to deal with that? Maybe I can do that to retrieve the well conserved part and then extend this region with reads that overlap (as if it was gap-filling)? &#xA;What do you think of this approach? Do you know a software for such extension process? &#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="6459" LastEditorUserId="6459" LastEditDate="2020-04-24T15:59:26.657" LastActivityDate="2020-04-24T15:59:26.657" Title="Assemble reads for a specific gene" Tags="|gene|assembly|read-mapping|phylogeny|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13061" PostTypeId="2" ParentId="12899" CreationDate="2020-04-24T08:25:23.957" Score="1" Body="&lt;p&gt;You can visualize with Discovery Studio Visualizer and Schrodinger.&#xA;Arpeggio and PLIP can generate output files of the different interactions.&lt;/p&gt;&#xA;" OwnerUserId="3861" LastActivityDate="2020-04-24T08:25:23.957" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13062" PostTypeId="2" ParentId="13023" CreationDate="2020-04-24T12:20:46.033" Score="1" Body="&lt;p&gt;There seems to be a bioconductor package named &lt;code&gt;synmut&lt;/code&gt; which does just that.&#xA;Apparently, it can take codon usage into account when generating synonymous mutations.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Copy pasting an example that matches your use case from &lt;a href=&quot;https://bioconductor.org/packages/3.11/bioc/vignettes/SynMut/inst/doc/SynMut.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the official doc&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(&quot;SynMut&quot;)&#xA;filepath.fasta &amp;lt;- system.file(&quot;extdata&quot;, &quot;example.fasta&quot;, package = &quot;SynMut&quot;)&#xA;filepath.csv &amp;lt;- system.file(&quot;extdata&quot;, &quot;target_regions.csv&quot;, package = &quot;SynMut&quot;)&#xA;region &amp;lt;- read.csv(filepath.csv)&#xA;rgd.seq &amp;lt;- input_seq(filepath.fasta, region)&#xA;mut.seq &amp;lt;- codon_random(rgd.seq, keep = TRUE)&#xA;Biostrings::writeXStringSet(get_dna(mut.seq), &quot;rgd.fasta&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="439" LastActivityDate="2020-04-24T12:20:46.033" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13063" PostTypeId="5" CreationDate="2020-04-24T12:51:10.047" Score="0" Body="&lt;p&gt;The WDL language is stewarded by the OpenWDL community as described at &lt;a href=&quot;https://openwdl.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://openwdl.org&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Core WDL goals:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Make it straightforward to define analysis tasks, chain them together in workflows, and parallelize their execution.   &lt;/li&gt;&#xA;&lt;li&gt;Achieve portability not only across execution platforms, but also different types of users. Whether one is an analyst, a programmer, an operator of a production system, or any other sort of user, WDL should be accessible and understandable.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;OpenWDL resources:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;WDL specification: &lt;a href=&quot;https://github.com/openwdl/wdl&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/openwdl/wdl&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Educational materials: &lt;a href=&quot;https://github.com/openwdl/learn-wdl&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/openwdl/learn-wdl&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;Slack workspace: &lt;a href=&quot;https://openwdl.slack.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://openwdl.slack.com&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Mailing list:  &lt;a href=&quot;https://groups.google.com/a/openwdl.org/forum/#!forum/community&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://groups.google.com/a/openwdl.org/forum/#!forum/community&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;WDL workflows published in Dockstore:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://dockstore.org/search?descriptorType=WDL&amp;amp;searchMode=files&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://dockstore.org/search?descriptorType=WDL&amp;amp;searchMode=files&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2776" LastEditorUserId="2776" LastEditDate="2020-04-24T13:35:15.907" LastActivityDate="2020-04-24T13:35:15.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13064" PostTypeId="4" CreationDate="2020-04-24T12:51:10.047" Score="0" Body="Workflow Description Language; used to specify data processing and analysis workflows with a human-readable and -writeable syntax. " OwnerUserId="2776" LastEditorUserId="2776" LastEditDate="2020-04-24T13:35:26.507" LastActivityDate="2020-04-24T13:35:26.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13065" PostTypeId="1" AcceptedAnswerId="13070" CreationDate="2020-04-24T13:49:11.797" Score="0" ViewCount="2035" Body="&lt;p&gt;I am trying to convert a list of gene names into Ensembl IDs but I'm having a bit of trouble.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've tried using the EnsDb.Hsapiens.v86 package but I always get LRG-ID values returned as well, even when using the GeneIdFilter &quot;startsWith&quot; thing.&lt;br&gt;&#xA;My code looks like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;targets &amp;lt;- as.character(targetCSV[, 2])&#xA;&#xA;&#xA;targetEnsemble &amp;lt;- ensembldb::select(&#xA;      EnsDb.Hsapiens.v86, &#xA;      filter = GeneIdFilter(&quot;ENS&quot;, &quot;startsWith&quot;),&#xA;      keys = targets, &#xA;      keytype = &quot;SYMBOL&quot;, &#xA;      columns = c(&quot;SYMBOL&quot;,&quot;GENEID&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And for example I get back:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    SYMBOL          GENEID&#xA;1     CYR61 ENSG00000142871&#xA;2      CTGF ENSG00000118523&#xA;3    AMOTL2 ENSG00000114019&#xA;4    ANKRD1 ENSG00000148677&#xA;5    ANKRD1         LRG_379&#xA;6    IGFBP3 ENSG00000146674&#xA;7        F3 ENSG00000117525&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also tried using org.Hs.eg.db package but get a different problem where some of my genes return NA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My code:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  mapIds(x = org.Hs.eg.db, &#xA;     keys = targets, &#xA;     &quot;ENSEMBL&quot;, &#xA;     &quot;SYMBOL&quot;,&#xA;     fuzzy = TRUE,&#xA;     multiVals = first)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this works on some but not all the genes for some reason.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    $CYR61&#xA;[1] NA&#xA;&#xA;$CTGF&#xA;[1] NA&#xA;&#xA;$AMOTL2&#xA;[1] &quot;ENSG00000114019&quot;&#xA;&#xA;$ANKRD1&#xA;[1] &quot;ENSG00000148677&quot;&#xA;&#xA;$IGFBP3&#xA;[1] &quot;ENSG00000146674&quot;&#xA;&#xA;$F3&#xA;[1] &quot;ENSG00000117525&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If anyone can solve either of these problems that would be great, thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="8706" LastActivityDate="2020-05-24T18:03:35.380" Title="How to map gene names to ensembl Ids in R" Tags="|r|gene|ensembl|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13066" PostTypeId="2" ParentId="13060" CreationDate="2020-04-24T13:53:44.040" Score="1" Body="&lt;p&gt;That's a good question. I remember seeing a poster about making trees directly from reads, but unfortunately, I can not find any other record than this &lt;a href=&quot;https://twitter.com/ISBSIB/status/1011987705507516416/photo/1&quot; rel=&quot;nofollow noreferrer&quot;&gt;tweet&lt;/a&gt;. Perhaps you can try to email David, if he would be keen to share the implementation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;While searching for any information about David's work, I run into a GitHub repository called &lt;a href=&quot;https://github.com/kheyduk/reads2trees&quot; rel=&quot;nofollow noreferrer&quot;&gt;reads2trees&lt;/a&gt;. It's quite old and it's not super popular, so hard to say if it can be useful. But the name matches the problem description quite well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also remember there were some attempts (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5224691/&quot; rel=&quot;nofollow noreferrer&quot;&gt;An example paper&lt;/a&gt;) to get phylogeny directly from kmers, but I have never tried that on my own. I also think these approached were focused on microbial phylogienies. So again, not sure if it's going to be useful for you.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;That's all I got.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2020-04-24T13:53:44.040" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13067" PostTypeId="1" AcceptedAnswerId="13071" CreationDate="2020-04-24T14:04:50.433" Score="2" ViewCount="337" Body="&lt;p&gt;This might be a really stupid question, but I can't figure out how to do this. I've read through the DESeq2 vignette and manual pages but couldn't find an answer.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a bunch of samples split up into different conditions (eg. celltypes and disease state). I would like to make a comparison between the two possible values within one condition, but only for the samples with a specific value of the other condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, with the following samplesheet: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;patient | phenotype | type&#xA;--------+-----------+-----&#xA;1       | healthy   | A&#xA;1       | healthy   | B&#xA;1       | sick      | A&#xA;1       | sick      | B&#xA;2       | healthy   | A&#xA;2       | sick      | A&#xA;2       | sick      | B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to compare &quot;healthy&quot; vs &quot;sick&quot;, but only for type &quot;A&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Currently, I have the following code, but this will also include the type &quot;B&quot; sample:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData=counts, colData=design, design = ~ patient + phenotype + type)&#xA;keep &amp;lt;- rowSums(counts(dds)) &amp;gt;= 10&#xA;dds &amp;lt;- dds[keep,]&#xA;dds &amp;lt;- DESeq(dds)&#xA;res &amp;lt;- results(dds, contrast=c(&quot;phenotype&quot;, &quot;healthy&quot;, &quot;sick&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Any idea how to accomplish this?&lt;/p&gt;&#xA;" OwnerUserId="6955" LastActivityDate="2020-04-24T16:24:14.990" Title="DESeq2 compare within a condition" Tags="|differential-expression|deseq2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13068" PostTypeId="2" ParentId="12967" CreationDate="2020-04-24T15:54:27.630" Score="2" Body="&lt;p&gt;Unfortunately generating inputs for Cromwell is not something that is handled by Cromwell itself or by GATK, and generally needs to be scripted to fit your custom needs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Despite this being a bit arduous, there is a very good reason for this. WDL itself is a highly polymorphic workflow language, where inputs can be vastly different between each workflow. Cromwell &lt;em&gt;ONLY&lt;/em&gt; understands how to parse the workflow and the inputs, but does not know the context of what it is running (ie this is a paired mapping job), nor does GATK know that it is running within a WDL. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Additionally, I've encountered many different cases where there are not just 2 fastq files but, due to older sequencing technologies, 10 or more for the same sample. Unless GATK/Cromwell knows how your specific samples are arranged there is no easy way for them to generate your inputs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are tools that would possibly help you simplify scripting better. &lt;a href=&quot;https://cromwell.readthedocs.io/en/stable/WOMtool/&quot; rel=&quot;nofollow noreferrer&quot;&gt;WOMTool&lt;/a&gt; developed by the Broad has an &lt;code&gt;inputs&lt;/code&gt; command which will take an  arbitrary workflow and generate a &lt;code&gt;JSON&lt;/code&gt; template of the inputs, which you can then use in your own scripts. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, imagine you have this WDL workflow in a file called &lt;code&gt;my.wdl&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;version 1.0&#xA;workflow foo {&#xA;  input {&#xA;    Array[Pair[File,File]] fastqFiles&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When you run WOMtool's &lt;code&gt;inputs&lt;/code&gt; command, it produces a template JSON listing all inputs and specifying the variable types expected for each:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;$ java -jar womtool.jar inputs my.wdl&#xA;{&#xA;  &quot;foo.fastqFiles&quot;: &quot;Array[Pair[File, File]]&quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can use this template to produce actual JSONs for each of your samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that you can also get the template written to a file as follows: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ java -jar womtool.jar inputs my.wdl &amp;gt; my.inputs.json&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7253" LastEditorUserId="2776" LastEditDate="2020-04-24T17:26:18.813" LastActivityDate="2020-04-24T17:26:18.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13070" PostTypeId="2" ParentId="13065" CreationDate="2020-04-24T16:13:47.947" Score="3" Body="&lt;p&gt;You can pull a full list of Ensembl gene IDs and MGI symbols from biomaRt in R. From there it should be easy enough to filter for the genes you want:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(biomaRt)&#xA;hsapiens_genes &amp;lt;- getBM(attributes = c(&quot;ensembl_gene_id&quot;, &#xA;                                       &quot;hgnc_symbol&quot;),&#xA;                        mart = useMart(&quot;ensembl&quot;, dataset = &quot;hsapiens_gene_ensembl&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You do not find some genes in your example because you were not using the official gene name. For example CTGF is offocially called CCN2 and with this name you will find the appropriate Ensembl gene ID in the biomaRt table. This is why I personally prefer to use gene name that are like &lt;code&gt;ENSG00000118523|CCN2&lt;/code&gt; in my work. That way you always have a unique identifier (the Ensembl ID) that is unambiguous and you do not run into trouble that for one gene many more or less official gene names exist.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2020-04-24T16:18:57.513" LastActivityDate="2020-04-24T16:18:57.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13071" PostTypeId="2" ParentId="13067" CreationDate="2020-04-24T16:24:14.990" Score="1" Body="&lt;p&gt;Welcome to the world of all possible pairwise comparisons in DESeq2 ! &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So the easiest way is to create another group in your data frame, first I simulate some sensible counts:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;counts = counts(makeExampleDESeqDataSet(m=48))&#xA;design = expand.grid(id=1:3,&#xA;phenotype=rep(c(&quot;sick&quot;,&quot;healthy&quot;),each=2),&#xA;type = rep(c(&quot;A&quot;,&quot;B&quot;),2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And you make a group that is a combination of phenotype and type:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;design&lt;span class=&quot;math-container&quot;&gt;$group = paste(design$&lt;/span&gt;phenotype,design$type,sep=&quot;_&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Run DESeq2, including id:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData=counts, colData=design, &#xA;design = ~ id + group)&#xA;keep &amp;lt;- rowSums(counts(dds)) &amp;gt;= 10&#xA;dds &amp;lt;- dds[keep,]&#xA;dds &amp;lt;- DESeq(dds)&#xA;res &amp;lt;- results(dds, contrast=c(&quot;group&quot;, &quot;healthy_A&quot;, &quot;sick_A&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So this compares only healthy_A with sick_A.. hopefully this is what you need... The other option (normal statistical) is to fit a model with interaction, but I think this is easier to explain and interpret. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another question you might have is why not do two separate analysis for A and B, I think it's sometimes better if you model the variance with all available data, given the limited number of replicates.&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-04-24T16:24:14.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13073" PostTypeId="2" ParentId="12934" CreationDate="2020-04-24T18:20:07.027" Score="1" Body="&lt;p&gt;Doo you have your heart set on AWS, or are you open to using Google Cloud Platform (GCP)? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're open to using GCP, there are some advantages. For example GATK can read and write directly to and from Google storage buckets, and IGV can use your login credentials to access data in Google storage as described here: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://googlegenomics.readthedocs.io/en/latest/use_cases/browse_genomic_data/igv.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://googlegenomics.readthedocs.io/en/latest/use_cases/browse_genomic_data/igv.html&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that if you make the data public, by default you're going to get billed if people download it. Not sure how that's handled on AWS, but on GCP you can set the bucket (or just the file) to be &quot;requester-pays&quot;, ie the person who downloads is the person who pays for it: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://cloud.google.com/storage/docs/requester-pays&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cloud.google.com/storage/docs/requester-pays&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general there are several ways to interact with Google buckets. &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;web-based file browser (example is a public bucket I manage): &lt;a href=&quot;https://console.cloud.google.com/storage/browser/genomics-in-the-cloud&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://console.cloud.google.com/storage/browser/genomics-in-the-cloud&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;command-line access through the cloud sdk toolkit &lt;code&gt;gsutil&lt;/code&gt;: &lt;a href=&quot;https://cloud.google.com/storage/docs/gsutil&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cloud.google.com/storage/docs/gsutil&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Disclosure: I work at the Broad Institute, which uses a lot of GCP infrastructure; I do not get any compensation from Google Cloud, but I recently wrote a book about Genomics in the Cloud that uses GCP.&lt;/em&gt; &lt;/p&gt;&#xA;" OwnerUserId="2776" LastActivityDate="2020-04-24T18:20:07.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13075" PostTypeId="1" CreationDate="2020-04-24T22:02:10.127" Score="1" ViewCount="75" Body="&lt;p&gt;I'm trying to learn about what the state of technology is at presently.   It seems like we clearly can go from nucleotide to digitally stored sequence, but can we transcribe something from the digital sequence?&lt;/p&gt;&#xA;" OwnerUserId="8712" LastEditorUserId="2108" LastEditDate="2020-04-25T12:05:54.590" LastActivityDate="2020-04-28T12:47:02.037" Title="Is it possible to go from a digital sequence to an actual nucleotide?" Tags="|fasta|sequence-analysis|phylogenetics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13076" PostTypeId="2" ParentId="13075" CreationDate="2020-04-24T22:52:04.203" Score="3" Body="&lt;p&gt;Oligonucleotide synthesis is a &lt;a href=&quot;https://www.idtdna.com/pages/products/custom-dna-rna/dna-oligos/custom-dna-oligos?utm_source=google&amp;amp;utm_medium=cpc&amp;amp;utm_campaign=ga_oligos&amp;amp;utm_content=ad_group_oligos&amp;amp;gclid=CjwKCAjwnIr1BRAWEiwA6GpwNX13eq8hHXX5D8cVVApGc0hzAcsaIEd2c29PYdHrxeFp8B4dpijrlhoC7wgQAvD_BwE&quot; rel=&quot;nofollow noreferrer&quot;&gt;service offered by many providers such as IDT&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-04-24T22:52:04.203" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13078" PostTypeId="2" ParentId="13004" CreationDate="2020-04-24T23:15:34.987" Score="1" Body="&lt;p&gt;I'm going to answer 1+2 because 3 is hard to understand and maybe trivial?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Each of 1+2 seems to be have these sub-tasks:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;identify the contigs in your assembly that have the genes of interest.&lt;/li&gt;&#xA;&lt;li&gt;pull out the gene sequences from the contigs (you should have intermediate tblastx output somewhere with coordinates, if you don't you can just re-blast those contigs).&lt;/li&gt;&#xA;&lt;li&gt;possibly translate those in silico to protein sequences.&lt;/li&gt;&#xA;&lt;li&gt;tblastx (gene) or blastp (protein) those gene/protein sequences against whatever db is relevant.&lt;/li&gt;&#xA;&lt;li&gt;pull sequences for the hits above some threshold for each gene from the db.&lt;/li&gt;&#xA;&lt;li&gt;perform a multiple sequence alignment for each gene against the hits.&lt;/li&gt;&#xA;&lt;li&gt;plug the MSA into whatever phylogenetic tool you like. I used phyml a lot back in the day, don't know what the kids like now.&lt;/li&gt;&#xA;&lt;li&gt;make a visualization of the tree, probably?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-04-24T23:15:34.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13079" PostTypeId="1" AcceptedAnswerId="13081" CreationDate="2020-04-25T09:55:59.787" Score="4" ViewCount="595" Body="&lt;p&gt;I would like to merge together several ccp4 formatted density maps (and do a few minor things).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So ideally I would like to open the ccp4/mrc files as numpy arrays in Python 3 and save the array as a ccp4 file. CCTBX can open ccp4 maps as numpy arrays, but I really would like to avoid using Python 2.7 for various reasons.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is to be used for constraining energy minimisation in PyRosetta using the &lt;code&gt;pyrosetta.rosetta.protocols.electron_density.SetupForDensityScoringMover&lt;/code&gt; mover/sampler. But its &lt;code&gt;pyrosetta.rosetta.core.scoring.electron_density.getDensityMap&lt;/code&gt; data loader only likes ccp4/mrc files (density maps) and not mtz files (structure factor maps).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From what I can tell there is no way to fudge a ccp4 map from a mtz map, correct? The fact that only ePDB provides ccp4 maps makes me want to use mtz files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;CCP4's Blend can merge datasets but uses only mtz files...&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-04-25T11:44:23.370" Title="CCP4 file to a Python 3 numpy array or similar workaround" Tags="|proteins|protein-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13080" PostTypeId="1" CreationDate="2020-04-25T10:37:03.630" Score="2" ViewCount="48" Body="&lt;p&gt;i have created a phmm in hmmer from a multiple alignment focused on E. coli produced in tcoffee from sequences taken from blast similar to my query sequence.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I would like to ask how i can compare the results from HMMER for a pHMM against UNIPROT (i used the phmm that i created and i searched in hmmer search against uniprot) and the results from BLAST.&lt;/li&gt;&#xA;&lt;li&gt;I would like to ask if i can search a pHMM in BLAST somehow, or if blast accepts only fasta one sequence.&lt;/li&gt;&#xA;&lt;li&gt;Why e-values of hmmer and blast are not comparable??Why e-value in hmmer is never 0?&#xA;Thank you so much!!&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="8713" LastEditorUserId="3848" LastEditDate="2020-06-24T19:59:16.183" LastActivityDate="2020-06-24T19:59:16.183" Title="phmm blast hmmer" Tags="|blast|hidden-markov-models|hmmer|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13081" PostTypeId="2" ParentId="13079" CreationDate="2020-04-25T11:44:23.370" Score="4" Body="&lt;p&gt;You can use &lt;a href=&quot;https://gemmi.readthedocs.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gemmi&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import gemmi&#xA;ccp4_map = gemmi.read_ccp4_map('my.map')&#xA;ccp4_map.setup()  # optional&#xA;np_array = numpy.array(ccp4_map.grid, copy=False)&#xA;# ...&#xA;ccp4_map.write_ccp4_map('out.ccp4')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The setup() call expands data to the whole unit cell.&lt;br&gt;&#xA;The numpy array above accesses the data through Python's &lt;em&gt;Buffer Protocol&lt;/em&gt; (it does not copy the data). So changing values in numpy array changes gemmi.Ccp4Map.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can also use gemmi to read mtz file and transform map coefficients to a real-space map, see &lt;a href=&quot;https://gemmi.readthedocs.io/en/latest/hkl.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gemmi.readthedocs.io/en/latest/hkl.html&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2020-04-25T11:44:23.370" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13082" PostTypeId="1" AcceptedAnswerId="13083" CreationDate="2020-04-25T12:08:29.523" Score="0" ViewCount="156" Body="&lt;p&gt;i need to search for one protein against UNIPROT in BLAST for a selected organism.&#xA;How can I do it? (SWSISSPROT+TREMBL)&#xA;THANKS!!&lt;/p&gt;&#xA;" OwnerUserId="8713" LastActivityDate="2020-10-23T15:52:04.287" Title="how can i search in p-blast against UNIPROT(not only swissprot)" Tags="|blast|uniprot|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13083" PostTypeId="2" ParentId="13082" CreationDate="2020-04-25T15:31:23.230" Score="0" Body="&lt;p&gt;I don't know of any online tool enabling a search against TrEMBL. You probably will have to set up your own local BLAST environment, roughly following these steps:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&amp;amp;PAGE_TYPE=BlastDocs&amp;amp;DOC_TYPE=Download&quot; rel=&quot;nofollow noreferrer&quot;&gt;download blast+&lt;/a&gt; - but there might be an easier way to install the software, depending on your operating system (e.g. on Ubuntu you can just &lt;code&gt;sudo apt install ncbi-blast+&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.uniprot.org/downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;download target databases&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK279688/&quot; rel=&quot;nofollow noreferrer&quot;&gt;build local database&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;I hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="8629" LastActivityDate="2020-04-25T15:31:23.230" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13084" PostTypeId="1" AcceptedAnswerId="13115" CreationDate="2020-04-25T16:05:37.947" Score="0" ViewCount="1269" Body="&lt;p&gt;I am trying to perform a &lt;code&gt;HISAT2&lt;/code&gt; indexing using a &lt;code&gt;GCF_000001405.39_GRCh38.p13_genomic.gtf&lt;/code&gt; and &lt;code&gt;GCF_000001405.39_GRCh38.p13_genomic.fna&lt;/code&gt; files downloaded from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly/GCF_000001405.39&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/assembly/GCF_000001405.39&lt;/a&gt; using the procedure described in  Pertea et al. (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/27560171&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pubmed/27560171&lt;/a&gt;), e.g. first I create &lt;code&gt;extract_splice_sites.py&lt;/code&gt; and &lt;code&gt;extract_exons.py&lt;/code&gt; and then my code looks like this&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;#SBATCH --time=02:30:00&#xA;#SBATCH --account=def-myusername&#xA;#SBATCH --mem=160000M&#xA;#SBATCH --mail-user=mymail@gmail.com&#xA;#SBATCH --mail-type=All&#xA;&#xA;module load nixpkgs/16.09  intel/2018.3&#xA;module load hisat2/2.1.0&#xA;&#xA;hisat2-build --ss imam.ss --exon imam.exon GCF_000001405.39_GRCh38.p13_genomic.fna imam002&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am trying to receive a set of &lt;code&gt;imam002*.ht2&lt;/code&gt; files but every time the procedure ends with a &lt;code&gt;Ran out of memory&lt;/code&gt; message. From here &lt;a href=&quot;https://github.com/griffithlab/rnaseq_tutorial/wiki/Indexing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/griffithlab/rnaseq_tutorial/wiki/Indexing&lt;/a&gt; I learned that I must allocate 160Gb of RAM if I want to index the entire genome. Even when I do that and submit my script via sbatch on a HPC I still got this &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;slurmstepd: error: Detected 1 oom-kill event(s) in step 30827017.batch cgroup. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Just in case here's the window that I have when I don't sbatch my script, I still have an error&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Inactive Modules:&#xA;  1) openmpi/2.1.1&#xA;&#xA;The following have been reloaded with a version change:&#xA;  1) gcccore/.5.4.0 =&amp;gt; gcccore/.7.3.0     2) icc/.2016.4.258 =&amp;gt; icc/.2018.3.222     3) ifort/.2016.4.258 =&amp;gt; ifort/.2018.3.222     4) imkl/11.3.4.258 =&amp;gt; imkl/2018.3.222     5) intel/2016.4 =&amp;gt; intel/2018.3&#xA;&#xA;Settings:&#xA;  Output files: &quot;imam002.*.ht2&quot;&#xA;  Line rate: 7 (line is 128 bytes)&#xA;  Lines per side: 1 (side is 128 bytes)&#xA;  Offset rate: 4 (one in 16)&#xA;  FTable chars: 10&#xA;  Strings: unpacked&#xA;  Local offset rate: 3 (one in 8)&#xA;  Local fTable chars: 6&#xA;  Local sequence length: 57344&#xA;  Local sequence overlap between two consecutive indexes: 1024&#xA;  Endianness: little&#xA;  Actual local endianness: little&#xA;  Sanity checking: disabled&#xA;  Assertions: disabled&#xA;  Random seed: 0&#xA;  Sizeofs: void*:8, int:4, long:8, size_t:8&#xA;Input files DNA, FASTA:&#xA;  GCF_000001405.39_GRCh38.p13_genomic.fna&#xA;Reading reference sizes&#xA;  Time reading reference sizes: 00:00:23&#xA;Calculating joined length&#xA;Writing header&#xA;Reserving space for joined string&#xA;Joining reference sequences&#xA;  Time to join reference sequences: 00:00:13&#xA;  Time to read SNPs and splice sites: 00:00:15&#xA;  Ran out of memory; automatically trying more memory-economical parameters.&#xA;./hisat2_index.sh: line 24:   943 Killed                  hisat2-build --ss imam.ss --exon imam.exon GCF_000001405.39_GRCh38.p13_genomic.fna imam002&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I use the same script for another genome (not human) - I don't have this problem. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it really a problem of RAM and, if it is, how is usually human genome gets indexed by HISAT2? Is it even possible?&lt;/p&gt;&#xA;" OwnerUserId="6948" LastActivityDate="2020-04-28T05:25:09.927" Title="How can I bypass 'out of memory' problem using a HISAT2 for human genome indexing?" Tags="|rna-seq|human-genome|genome-index|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13085" PostTypeId="1" CreationDate="2020-04-25T16:54:58.910" Score="1" ViewCount="1514" Body="&lt;p&gt;I'm using &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/tools/primer-blast/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Primer-Blast&lt;/a&gt; to compare a PCR template against a custom database of reference sequences, using Primer-Blast's &lt;em&gt;Custom&lt;/em&gt; database functionality.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However whenever I try to upload a FASTA file as the custom database and then run Primer-Blast, I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Exception error: Fasta Reader: sequence id ends with 80 valid nucleotide characters. Was the sequence accidentally placed in the definition line?&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;No matter how I try to format my FASTA file, I get this error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone know how to properly format a FASTA file to have it accepted as a custom database of reference sequences to compare against the PCR template? Or is this a bug in the software?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's an example FASTA file, to upload both for the template and the custom database.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;test1&#xA;ATTAAAGGTTTATACCTTCCCAGGTAACAAACCAACCAACTTTCGATCTCTTGTAGATCTGTTCTCTAAACGAACTTTAA&#xA;AATCTGTGTGGCTGTCACTCGGCTGCATGCTTAGTGCACTCACGCAGTATAATTAATAACTAATTACTGTCGTTGACAGG&#xA;ACACGAGTAACTCGTCTATCTTCTGCAGGCTGCTTACGGTTTCGTCCGTGTTGCAGCCGATCATCAGCACATCTAGGTTT&#xA;CGTCCGGGTGTGACCGAAAGGTAAGATGGAGAGCCTTGTCCCTGGTTTCAACGAGAAAACACACGTCCAACTCAGTTTGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1456" LastEditorUserId="1456" LastEditDate="2020-04-25T17:01:33.227" LastActivityDate="2020-04-25T18:31:29.817" Title="Primer-Blast Custom Database [error]" Tags="|blast|phylogenetics|primer|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13086" PostTypeId="2" ParentId="13085" CreationDate="2020-04-25T17:24:37.067" Score="1" Body="&lt;p&gt;I just copied your FASTA sequence into a file (on Mac OS) and uploaded it to Primer BLAST, the search did not give an error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the error is caused by the line endings, maybe you can fix your downloaded files with a tool like &lt;a href=&quot;http://macappstore.org/dos2unix/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dos2unix&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;h1&gt;Update:&lt;/h1&gt;&#xA;&#xA;&lt;p&gt;On a whim, I tried adding a description to the FASTA header:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;test1 description&#xA;ATTAAAGGTTTATACCTTCCCAGGTAACAAACCAACCAACTTTCGATCTCTTGTAGATCTGTTCTCTAAACGAACTTTAA&#xA;AATCTGTGTGGCTGTCACTCGGCTGCATGCTTAGTGCACTCACGCAGTATAATTAATAACTAATTACTGTCGTTGACAGG&#xA;ACACGAGTAACTCGTCTATCTTCTGCAGGCTGCTTACGGTTTCGTCCGTGTTGCAGCCGATCATCAGCACATCTAGGTTT&#xA;CGTCCGGGTGTGACCGAAAGGTAAGATGGAGAGCCTTGTCCCTGGTTTCAACGAGAAAACACACGTCCAACTCAGTTTG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This time, there was no error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does that work for you as well?&lt;/p&gt;&#xA;" OwnerUserId="8629" LastEditorUserId="8629" LastEditDate="2020-04-25T17:57:45.047" LastActivityDate="2020-04-25T17:57:45.047" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13087" PostTypeId="2" ParentId="13085" CreationDate="2020-04-25T18:05:38.843" Score="0" Body="&lt;p&gt;I did it by copy and paste. Here's the top line of the output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    Primer pair 1&#xA;Sequence (5'-&amp;gt;3')   Template strand Length  Start   Stop    Tm  GC% Self complementarity    Self 3' complementarity&#xA;Forward primer  TGTCGTTGACAGGACACGAG    Plus    20  148 167 59.97   55.00   5.00    2.00&#xA;Reverse primer  TTACCTTTCGGTCACACCCG    Minus   20  264 245 59.97   55.00   3.00    2.00&#xA;Product length  117&#xA;Primer pair 2&#xA;Sequence (5'-&amp;gt;3')   Template strand Length  Start   Stop    Tm  GC% Self complementarity    Self 3' complementarity&#xA;Forward primer  ATCTGTGTGGCTGTCACTCG    Plus    20  82  101 60.04   55.00   5.00    3.00&#xA;Reverse primer  ACTCGTGTCCTGTCAACGAC    Minus   20  168 149 59.97   55.00   4.00    3.00&#xA;Product length  87&#xA;Primer pair 3&#xA;Sequence (5'-&amp;gt;3')   Template strand Length  Start   Stop    Tm  GC% Self complementarity    Self 3' complementarity&#xA;Forward primer  TACTGTCGTTGACAGGACACG   Plus    21  145 165 60.00   52.38   7.00    3.00&#xA;Reverse primer  ACACCCGGACGAAACCTAGA    Minus   20  251 232 60.54   55.00   4.00    2.00&#xA;Product length  107&#xA;Primer pair 4&#xA;Sequence (5'-&amp;gt;3')   Template strand Length  Start   Stop    Tm  GC% Self complementarity    Self 3' complementarity&#xA;Forward primer  GCTTACGGTTTCGTCCGTGT    Plus    20  192 211 60.94   55.00   4.00    0.00&#xA;Reverse primer  GTTGAAACCAGGGACAAGGC    Minus   20  292 273 59.33   55.00   4.00    2.00&#xA;Product length  101&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A command line solution would work, based on the bug reported by the OP. However there's nothing in NCBIs e-utilities and Biopython doesn't do Primer-Blast, but will do Blast. You could do it Perl via the LWP Agent, e.g. &lt;a href=&quot;http://www.bioinformatics-made-simple.com/2012/02/perl-script-4-how-to-blast-multiple.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-04-25T18:31:29.817" LastActivityDate="2020-04-25T18:31:29.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13089" PostTypeId="2" ParentId="9312" CreationDate="2020-04-25T21:37:03.490" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;This isn't a Python library, but it's easily invoked from Python code, easy to install, and has a lot of useful features.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I just discovered the amazing &lt;a href=&quot;https://github.com/shenwei356/taxonkit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;taxonkit library&lt;/a&gt; this week. In particular, the &lt;a href=&quot;https://bioinf.shenwei.me/taxonkit/usage/#name2taxid&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;taxonkit name2taxid&lt;/code&gt; command&lt;/a&gt; seems to be what you're looking for.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat NAMES&#xA;Polistes dominula&#xA;Sinorhizobium meliloti&#xA;Gossypium hirsutum&#xA;$ cat NAMES | taxonkit name2taxid&#xA;Polistes dominula       743375&#xA;Sinorhizobium meliloti  382&#xA;Gossypium hirsutum      3635&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Easily installed using &lt;code&gt;conda install taxonkit&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2020-04-25T21:37:03.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13091" PostTypeId="2" ParentId="11578" CreationDate="2020-04-26T12:02:00.313" Score="0" Body="&lt;p&gt;For RefSeq data, go to &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI homepage&lt;/a&gt; and search for &lt;code&gt;Arabidopsis Col-0 genome&lt;/code&gt;. Click on the 'Get Data' button in the result box and you will be directed to the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets/genomes/?acc=GCF_000001735.4&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Datasets&lt;/a&gt; portal with the latest RefSeq assembly selected. Click on the blue 'Download' button, uncheck the genome and GFF3 files from list, check transcript and protein FASTA files from the list and download data. &lt;/p&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2020-04-26T12:02:00.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13092" PostTypeId="1" AcceptedAnswerId="13103" CreationDate="2020-04-26T15:49:39.550" Score="0" ViewCount="82" Body="&lt;p&gt;Currently, I am facing such a question: how to predict one connected protein structures from two separate known structures?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It should be different from the protein-protein docking method, which is non-covalent interactions. My question is that the two proteins should &quot;contently&quot; link together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestions are appreciated.&lt;/p&gt;&#xA;" OwnerUserId="8722" LastEditorUserId="8722" LastEditDate="2020-05-07T18:47:49.343" LastActivityDate="2020-07-04T20:54:41.587" Title="Methods to predict one connected structure from two known separate structures" Tags="|proteins|protein-structure|assembly|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13093" PostTypeId="1" CreationDate="2020-04-26T15:55:13.573" Score="0" ViewCount="49" Body="&lt;p&gt;I have made a search in p-blast.&#xA;I would like to search for the protein results in uniprot. how can I do it directly?&#xA;I can go to GENBANK but i would like to go to UNIPROT.&#xA;Thank you!!!&lt;/p&gt;&#xA;" OwnerUserId="8713" LastActivityDate="2021-07-20T18:05:30.580" Title="how could i find uniprot ac in blast?" Tags="|blast|uniprot|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13096" PostTypeId="1" AcceptedAnswerId="13106" CreationDate="2020-04-27T06:38:11.217" Score="1" ViewCount="318" Body="&lt;p&gt;I'm trying to implement a polygenic score pipeline in WDL and I'm quite new to this pipeline management.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first step of the pipeline takes a file of GWAS summary statistics and splits it by chromosome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The corresponding wdl task:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;task split {&#xA;    input {&#xA;        File gwas&#xA;        String output_prefix&#xA;    }&#xA;&#xA;    command {&#xA;        ./splitGwas -i &lt;span class=&quot;math-container&quot;&gt;${gwas} -o $&lt;/span&gt;{output_prefix}&#xA;    }&#xA;&#xA;    output {&#xA;        Array [File] gwas_by_chr = glob(&quot;${output_prefix}_*.assoc&quot;)&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The next part of the process is to compute posterior effects of SNPs by chromosome. The inputs to this task are going to be the files split by chromosome in the previous step and an LD matrix for the chromosome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The corresponding wdl call would be something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Array [File] ld_matrices = read_lines(file_of_ld_matrices_by_chr)&#xA;&#xA;scatter(pair in zip(split.gwas_by_chr, ld_matrices)) {&#xA;    call sbayes.run {&#xA;        input:&#xA;            GCTB = gctb_executable_path,&#xA;            gwas = pair.left,&#xA;            ld_matrix = pair.right,&#xA;            output_prefix = out&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Since I'm grabbing the split files using glob in the previous step, I do not particularly know which element in the array corresponds to which chromosome. So I would be making a mistake if I zip the summary stats for one chromosome with the LD matrix corresponding to a different chromosome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The programmatic way would be to iterate the scatter over chromosome name but then I would lose the ability to specify the dependency between the two tasks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a better way to do this?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Cross posted on biostars (&lt;a href=&quot;https://www.biostars.org/p/434582/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/434582/&lt;/a&gt;) so pardon me if that is discouraged.&lt;/p&gt;&#xA;" OwnerUserId="601" LastActivityDate="2020-04-27T14:59:49.550" Title="Matching files by chromosome during wdl scatter gather" Tags="|wdl|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13097" PostTypeId="1" AcceptedAnswerId="13099" CreationDate="2020-04-27T07:09:59.667" Score="0" ViewCount="45" Body="&lt;p&gt;I am trying to write a research paper on PRIMER-BLAST. I have read something about it on &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/22708584/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pubmed/22708584/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but still, it is not enough. So can you please tell me more about it or suggest some resources.&lt;/p&gt;&#xA;" OwnerUserId="8729" LastActivityDate="2020-04-27T08:53:15.613" Title="Can you give me a detailed explanation about PrimeBlast?" Tags="|blast|primer|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13098" PostTypeId="1" CreationDate="2020-04-27T08:20:35.523" Score="1" ViewCount="35" Body="&lt;p&gt;I have created a model with SVM classifier and evaluated it with 5-fold cross-validation. The dataset was sequence data containing lysine modified sites. Now I want to build a web server where anyone can input any sequence in FASTA format. Apparently, I have no label information at this point. So how can I will be able to extract features or use any feature selection method which requires label information? How can I extract or create the test set to predict? Suppose I have used f-score feature selection method in my model and selected 50 features from the given dataset. But for web server, having no label information, how can I select 50 features from the sequence that are provided by an user and then test it with my previously trained model?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As being a novice in the world of bioinformatics, I am seeking the answer which might be improper for others. But it will be very much helpful for me.&#xA;I am giving some link of some papers and its corresponding web servers.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;iPTM-mLys: identifying multiple lysine PTM sites and their different type (&lt;a href=&quot;https://academic.oup.com/bioinformatics/article/32/20/3116/2196525&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://academic.oup.com/bioinformatics/article/32/20/3116/2196525&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Web server link : &lt;a href=&quot;http://www.jci-bioinfo.cn/iPTM-mLys&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.jci-bioinfo.cn/iPTM-mLys&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;iHyd-PseCp: Identify hydroxyproline and hydroxylysine in proteins by incorporating sequence-coupled effects into general PseAAC &#xA;(&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5190098/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5190098/&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Web server link : &lt;a href=&quot;http://www.jci-bioinfo.cn/iHyd-PseCp&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.jci-bioinfo.cn/iHyd-PseCp&lt;/a&gt;&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="8656" LastActivityDate="2021-01-22T10:07:50.833" Title="After completing the evaluation, how to predict PTM lysine or any other modification with unlabeled data for making a web server?" Tags="|fasta|sequence-analysis|proteomics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13099" PostTypeId="2" ParentId="13097" CreationDate="2020-04-27T08:53:15.613" Score="0" Body="&lt;p&gt;I would start at the Primer3 website &lt;a href=&quot;http://bioinfo.ut.ee/primer3-0.4.0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you understand all the variables on the website via the manual &lt;a href=&quot;https://primer3.org/manual.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and the associated publications Koressaar et al (2007) and Untergasser et al (2012) you will understand the gist of the algorithm used in primer design. This gives you optimised primers for a single strand of DNA/RNA&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Primer-Blast looks in addition at maximising sequence conservation within an alignment of different sequences. The alignment is created using a Blast (I think its Mega-Blast. Thus Primer-Blast looks at one more parameter accounting for mutation heterogeneity which might be encountered in the PCR/RT-PCR and the number of primers it recommends is therefore reduced over say PRimer3. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Prove it for yourself take a given sequence and run it through Primer3 and then Primer-Blast and assess the differences and similarities. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;References&lt;/em&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Untergasser A, Cutcutache I, Koressaar T, Ye J, Faircloth BC, Remm M, Rozen SG (2012) Primer3 - new capabilities and interfaces. Nucleic Acids Research 40(15):e115&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Koressaar T, Remm M (2007) Enhancements and modifications of primer design program Primer3 Bioinformatics 23(10):1289-91&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-04-27T08:53:15.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13100" PostTypeId="1" AcceptedAnswerId="13130" CreationDate="2020-04-27T09:18:27.593" Score="1" ViewCount="908" Body="&lt;p&gt;I'm trying to install the latest version of snakemake from conda into a conda env, following the instructions &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/getting_started/installation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I.e.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;conda create -n snakemake -c conda-forge -c bioconda snakemake&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, this seems to only give me the option to install snakemake v5.3.0, while the latest &lt;code&gt;no-arch&lt;/code&gt; version on conda is v5.15.0.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help greatly appreciated.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Solving environment: done&#xA;&#xA;## Package Plan ##&#xA;&#xA;  environment location: /home/ubuntu/anaconda3/envs/snakemake&#xA;&#xA;  added / updated specs:&#xA;    - snakemake&#xA;&#xA;&#xA;The following packages will be downloaded:&#xA;&#xA;    package                    |            build&#xA;    ---------------------------|-----------------&#xA;    aioeasywebdav-2.4.0        |        py36_1000          18 KB  conda-forge&#xA;    aiohttp-3.6.2              |   py36h7b6447c_0         544 KB&#xA;    appdirs-1.4.3              |   py36h28b3542_0          15 KB&#xA;    async-timeout-3.0.1        |           py36_0          12 KB&#xA;    attrs-19.3.0               |             py_0          39 KB&#xA;    bcrypt-3.1.7               |   py36h7b6447c_0          40 KB&#xA;    blas-2.14                  |         openblas          10 KB  conda-forge&#xA;    boto3-1.9.66               |           py36_0         105 KB&#xA;    botocore-1.12.189          |             py_0         3.4 MB&#xA;    datrie-0.8                 |   py36h7b6447c_0         147 KB&#xA;    docutils-0.16              |           py36_0         668 KB&#xA;    dropbox-9.4.0              |           py36_0         756 KB&#xA;    filechunkio-1.6            |           py36_0           7 KB  bioconda&#xA;    ftputil-3.2                |           py36_0          85 KB  bioconda&#xA;    google-api-core-1.16.0     |           py36_1          86 KB&#xA;    googleapis-common-protos-1.51.0|           py36_2          72 KB&#xA;    idna_ssl-1.1.0             |           py36_0           7 KB&#xA;    importlib_metadata-1.5.0   |           py36_0          48 KB&#xA;    jsonschema-3.2.0           |           py36_0          95 KB&#xA;    liblapacke-3.8.0           |      14_openblas          10 KB  conda-forge&#xA;    multidict-4.7.3            |   py36h7b6447c_0          68 KB&#xA;    networkx-2.4               |             py_0         1.2 MB&#xA;    numpy-1.17.0               |   py36h99e49ec_0          24 KB  r&#xA;    numpy-base-1.17.0          |   py36h2f8d375_0         5.2 MB  r&#xA;    pandas-1.0.3               |   py36h0573a6f_0         8.6 MB&#xA;    protobuf-3.11.4            |   py36he6710b0_0         635 KB&#xA;    psutil-5.7.0               |   py36h7b6447c_0         315 KB&#xA;    pygraphviz-1.3.1           |           py36_0         205 KB  bioconda&#xA;    pynacl-1.3.0               |   py36h7b6447c_0         1.1 MB&#xA;    pyrsistent-0.16.0          |   py36h7b6447c_0          93 KB&#xA;    pysftp-0.2.9               |           py36_0          31 KB  bioconda&#xA;    ratelimiter-1.2.0          |        py36_1000          12 KB  conda-forge&#xA;    rsa-3.1.4                  |           py36_0          87 KB  bioconda&#xA;    s3transfer-0.1.13          |           py36_0          79 KB&#xA;    snakemake-5.3.0            |           py36_1           4 KB  bioconda&#xA;    snakemake-minimal-5.3.0    |           py36_1         280 KB  bioconda&#xA;    sqlite-3.31.1              |       h62c20be_1         2.0 MB&#xA;    typing_extensions-3.7.4.1  |           py36_0          40 KB&#xA;    wrapt-1.12.1               |   py36h7b6447c_1          49 KB&#xA;    yarl-1.4.2                 |   py36h7b6447c_0         132 KB&#xA;    zipp-2.2.0                 |             py_0          12 KB&#xA;    ------------------------------------------------------------&#xA;                                           Total:        26.2 MB&#xA;&#xA;The following NEW packages will be INSTALLED:&#xA;&#xA;  _libgcc_mutex      pkgs/main/linux-64::_libgcc_mutex-0.1-main&#xA;  aioeasywebdav      conda-forge/linux-64::aioeasywebdav-2.4.0-py36_1000&#xA;  aiohttp            pkgs/main/linux-64::aiohttp-3.6.2-py36h7b6447c_0&#xA;  appdirs            pkgs/main/linux-64::appdirs-1.4.3-py36h28b3542_0&#xA;  asn1crypto         pkgs/main/linux-64::asn1crypto-1.3.0-py36_0&#xA;  async-timeout      pkgs/main/linux-64::async-timeout-3.0.1-py36_0&#xA;  attrs              pkgs/main/noarch::attrs-19.3.0-py_0&#xA;  bcrypt             pkgs/main/linux-64::bcrypt-3.1.7-py36h7b6447c_0&#xA;  blas               conda-forge/linux-64::blas-2.14-openblas&#xA;  boto3              pkgs/main/linux-64::boto3-1.9.66-py36_0&#xA;  botocore           pkgs/main/noarch::botocore-1.12.189-py_0&#xA;  ca-certificates    pkgs/main/linux-64::ca-certificates-2020.1.1-0&#xA;  cachetools         pkgs/main/noarch::cachetools-3.1.1-py_0&#xA;  cairo              pkgs/main/linux-64::cairo-1.14.12-h8948797_3&#xA;  certifi            pkgs/main/linux-64::certifi-2020.4.5.1-py36_0&#xA;  cffi               pkgs/main/linux-64::cffi-1.14.0-py36h2e261b9_0&#xA;  chardet            pkgs/main/linux-64::chardet-3.0.4-py36_1003&#xA;  configargparse     pkgs/main/noarch::configargparse-1.1-py_0&#xA;  cryptography       pkgs/main/linux-64::cryptography-2.8-py36h1ba5d50_0&#xA;  datrie             pkgs/main/linux-64::datrie-0.8-py36h7b6447c_0&#xA;  decorator          pkgs/main/noarch::decorator-4.4.2-py_0&#xA;  docutils           pkgs/main/linux-64::docutils-0.16-py36_0&#xA;  dropbox            pkgs/main/linux-64::dropbox-9.4.0-py36_0&#xA;  expat              pkgs/main/linux-64::expat-2.2.6-he6710b0_0&#xA;  filechunkio        bioconda/linux-64::filechunkio-1.6-py36_0&#xA;  fontconfig         pkgs/main/linux-64::fontconfig-2.13.0-h9420a91_0&#xA;  freetype           pkgs/main/linux-64::freetype-2.9.1-h8a8886c_1&#xA;  fribidi            pkgs/main/linux-64::fribidi-1.0.5-h7b6447c_0&#xA;  ftputil            bioconda/linux-64::ftputil-3.2-py36_0&#xA;  gitdb              pkgs/main/noarch::gitdb-4.0.2-py_0&#xA;  gitpython          pkgs/main/noarch::gitpython-3.1.1-py_1&#xA;  glib               pkgs/main/linux-64::glib-2.63.1-h5a9c865_0&#xA;  google-api-core    pkgs/main/linux-64::google-api-core-1.16.0-py36_1&#xA;  google-auth        pkgs/main/noarch::google-auth-1.13.1-py_0&#xA;  google-cloud-core  pkgs/main/noarch::google-cloud-core-1.3.0-py_0&#xA;  google-cloud-stor~ pkgs/main/noarch::google-cloud-storage-1.27.0-py_0&#xA;  google-resumable-~ pkgs/main/noarch::google-resumable-media-0.5.0-py_1&#xA;  googleapis-common~ pkgs/main/linux-64::googleapis-common-protos-1.51.0-py36_2&#xA;  graphite2          pkgs/main/linux-64::graphite2-1.3.13-h23475e2_0&#xA;  graphviz           pkgs/main/linux-64::graphviz-2.40.1-h21bd128_2&#xA;  harfbuzz           pkgs/main/linux-64::harfbuzz-1.8.8-hffaf4a1_0&#xA;  icu                pkgs/main/linux-64::icu-58.2-h9c2bf20_1&#xA;  idna               pkgs/main/noarch::idna-2.9-py_1&#xA;  idna_ssl           pkgs/main/linux-64::idna_ssl-1.1.0-py36_0&#xA;  importlib_metadata pkgs/main/linux-64::importlib_metadata-1.5.0-py36_0&#xA;  jinja2             pkgs/main/noarch::jinja2-2.11.1-py_0&#xA;  jmespath           pkgs/main/noarch::jmespath-0.9.4-py_0&#xA;  jpeg               pkgs/main/linux-64::jpeg-9b-h024ee3a_2&#xA;  jsonschema         pkgs/main/linux-64::jsonschema-3.2.0-py36_0&#xA;  ld_impl_linux-64   pkgs/main/linux-64::ld_impl_linux-64-2.33.1-h53a641e_7&#xA;  libblas            conda-forge/linux-64::libblas-3.8.0-14_openblas&#xA;  libcblas           conda-forge/linux-64::libcblas-3.8.0-14_openblas&#xA;  libedit            pkgs/main/linux-64::libedit-3.1.20181209-hc058e9b_0&#xA;  libffi             pkgs/main/linux-64::libffi-3.2.1-hd88cf55_4&#xA;  libgcc-ng          pkgs/main/linux-64::libgcc-ng-9.1.0-hdf63c60_0&#xA;  libgfortran-ng     pkgs/main/linux-64::libgfortran-ng-7.3.0-hdf63c60_0&#xA;  liblapack          conda-forge/linux-64::liblapack-3.8.0-14_openblas&#xA;  liblapacke         conda-forge/linux-64::liblapacke-3.8.0-14_openblas&#xA;  libopenblas        conda-forge/linux-64::libopenblas-0.3.7-h5ec1e0e_6&#xA;  libpng             pkgs/main/linux-64::libpng-1.6.37-hbc83047_0&#xA;  libprotobuf        pkgs/main/linux-64::libprotobuf-3.11.4-hd408876_0&#xA;  libsodium          pkgs/main/linux-64::libsodium-1.0.16-h1bed415_0&#xA;  libstdcxx-ng       pkgs/main/linux-64::libstdcxx-ng-9.1.0-hdf63c60_0&#xA;  libtiff            pkgs/main/linux-64::libtiff-4.1.0-h2733197_0&#xA;  libuuid            pkgs/main/linux-64::libuuid-1.0.3-h1bed415_2&#xA;  libxcb             pkgs/main/linux-64::libxcb-1.13-h1bed415_1&#xA;  libxml2            pkgs/main/linux-64::libxml2-2.9.9-hea5a465_1&#xA;  markupsafe         pkgs/main/linux-64::markupsafe-1.1.1-py36h7b6447c_0&#xA;  multidict          pkgs/main/linux-64::multidict-4.7.3-py36h7b6447c_0&#xA;  ncurses            pkgs/main/linux-64::ncurses-6.2-he6710b0_0&#xA;  networkx           pkgs/main/noarch::networkx-2.4-py_0&#xA;  numpy              r/linux-64::numpy-1.17.0-py36h99e49ec_0&#xA;  numpy-base         r/linux-64::numpy-base-1.17.0-py36h2f8d375_0&#xA;  openssl            pkgs/main/linux-64::openssl-1.1.1g-h7b6447c_0&#xA;  pandas             pkgs/main/linux-64::pandas-1.0.3-py36h0573a6f_0&#xA;  pango              pkgs/main/linux-64::pango-1.42.4-h049681c_0&#xA;  paramiko           pkgs/main/noarch::paramiko-2.7.1-py_0&#xA;  pcre               pkgs/main/linux-64::pcre-8.43-he6710b0_0&#xA;  pip                pkgs/main/linux-64::pip-20.0.2-py36_1&#xA;  pixman             pkgs/main/linux-64::pixman-0.38.0-h7b6447c_0&#xA;  prettytable        conda-forge/noarch::prettytable-0.7.2-py_3&#xA;  protobuf           pkgs/main/linux-64::protobuf-3.11.4-py36he6710b0_0&#xA;  psutil             pkgs/main/linux-64::psutil-5.7.0-py36h7b6447c_0&#xA;  pyasn1             pkgs/main/noarch::pyasn1-0.4.8-py_0&#xA;  pyasn1-modules     pkgs/main/noarch::pyasn1-modules-0.2.7-py_0&#xA;  pycparser          pkgs/main/noarch::pycparser-2.20-py_0&#xA;  pygraphviz         bioconda/linux-64::pygraphviz-1.3.1-py36_0&#xA;  pynacl             pkgs/main/linux-64::pynacl-1.3.0-py36h7b6447c_0&#xA;  pyopenssl          pkgs/main/linux-64::pyopenssl-19.1.0-py36_0&#xA;  pyrsistent         pkgs/main/linux-64::pyrsistent-0.16.0-py36h7b6447c_0&#xA;  pysftp             bioconda/linux-64::pysftp-0.2.9-py36_0&#xA;  pysocks            pkgs/main/linux-64::pysocks-1.7.1-py36_0&#xA;  python             pkgs/main/linux-64::python-3.6.10-hcf32534_1&#xA;  python-dateutil    pkgs/main/noarch::python-dateutil-2.8.1-py_0&#xA;  python-irodsclient conda-forge/noarch::python-irodsclient-0.8.2-py_0&#xA;  pytz               pkgs/main/noarch::pytz-2019.3-py_0&#xA;  pyyaml             pkgs/main/linux-64::pyyaml-5.3.1-py36h7b6447c_0&#xA;  ratelimiter        conda-forge/linux-64::ratelimiter-1.2.0-py36_1000&#xA;  readline           pkgs/main/linux-64::readline-8.0-h7b6447c_0&#xA;  requests           pkgs/main/linux-64::requests-2.23.0-py36_0&#xA;  rsa                bioconda/linux-64::rsa-3.1.4-py36_0&#xA;  s3transfer         pkgs/main/linux-64::s3transfer-0.1.13-py36_0&#xA;  setuptools         pkgs/main/linux-64::setuptools-46.1.3-py36_0&#xA;  six                pkgs/main/linux-64::six-1.14.0-py36_0&#xA;  smmap              pkgs/main/noarch::smmap-3.0.2-py_0&#xA;  snakemake          bioconda/linux-64::snakemake-5.3.0-py36_1&#xA;  snakemake-minimal  bioconda/linux-64::snakemake-minimal-5.3.0-py36_1&#xA;  sqlite             pkgs/main/linux-64::sqlite-3.31.1-h62c20be_1&#xA;  tk                 pkgs/main/linux-64::tk-8.6.8-hbc83047_0&#xA;  typing_extensions  pkgs/main/linux-64::typing_extensions-3.7.4.1-py36_0&#xA;  urllib3            pkgs/main/linux-64::urllib3-1.25.8-py36_0&#xA;  wheel              pkgs/main/linux-64::wheel-0.34.2-py36_0&#xA;  wrapt              pkgs/main/linux-64::wrapt-1.12.1-py36h7b6447c_1&#xA;  xmlrunner          conda-forge/noarch::xmlrunner-1.7.7-py_0&#xA;  xz                 pkgs/main/linux-64::xz-5.2.5-h7b6447c_0&#xA;  yaml               pkgs/main/linux-64::yaml-0.1.7-had09818_2&#xA;  yarl               pkgs/main/linux-64::yarl-1.4.2-py36h7b6447c_0&#xA;  zipp               pkgs/main/noarch::zipp-2.2.0-py_0&#xA;  zlib               pkgs/main/linux-64::zlib-1.2.11-h7b6447c_3&#xA;  zstd               pkgs/main/linux-64::zstd-1.3.7-h0b5b093_0&#xA;&#xA;&#xA;Proceed ([y]/n)?&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="993" LastActivityDate="2020-05-02T10:33:04.037" Title="Can't install snakemake v5.15.0 from conda" Tags="|snakemake|conda|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13102" PostTypeId="2" ParentId="13098" CreationDate="2020-04-27T09:43:50.717" Score="1" Body="&lt;p&gt;Interesting application. I would suggest using Python's Flask system, which is described in a few pages in O'Reilly's Programming PyTorch, chapter 8. PyTorch in Production. Django is not really recommended (don't say that on Stackoverflow however). O'Reilly do an entire book on Flask, but thats to make the site pretty.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously, the explanation here is Deep Learning rather than ML but the basic gist will hold. You should also think about building a docker for distribution which is easy (same chapter). We're in lockdown so I don't have this book with me. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;strong&gt;difficult part&lt;/strong&gt; will be web architecture because you shouldn't have the CPU/GPU on the same server as the quiry interface, otherwise you are prone to a denial of service attack or just sytem overload. To do this you need to consider  Representational state transfer (RESTful) and that is not easy at all (at least for me). Flask facilitates RESTful, but thats like systems admin stuff. I guess before getting into RESTful I'd see how popular your website is, but if its the same server as your institute website you'd need to be really careful because an external user could bring your institutional website down (you wouldn't be popular).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-04-27T09:49:04.530" LastActivityDate="2020-04-27T09:49:04.530" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13103" PostTypeId="2" ParentId="13092" CreationDate="2020-04-27T09:46:22.767" Score="1" Body="&lt;p&gt;There are two approaches here.&lt;/p&gt;&#xA;&lt;h2&gt;Physiological&lt;/h2&gt;&#xA;&lt;p&gt;If domain A and B of your protein interact with each other physiologically, then you need to protein-protein docking. This is not too straightforward as protein are flexible.&lt;/p&gt;&#xA;&lt;p&gt;Several online servers exists such as &lt;a href=&quot;http://www.bonvinlab.org/software/haddock2.2/haddock-start/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Haddock&lt;/a&gt; and include &lt;a href=&quot;https://cluspro.org/login.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;ClusPro&lt;/a&gt;, which use more traditional approaches. While the online server &lt;a href=&quot;https://zhanglab.ccmb.med.umich.edu/DEMO/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DEMO (I-Tasser group)&lt;/a&gt; does a structural homology based search and a final MD based repacking to better minimise the structures (no protein-protein docking).&lt;/p&gt;&#xA;&lt;p&gt;Many of these work by magic as far as the user is concerned. But understanding what is going on is really important, even if it is overly laborious. When you don't know where a protein goes, it's called blind or global docking, which is computational expensive and so imprecise that the effort required to prove it is correct for peer-review is too steep for it to be useful. If you are doing local docking it is okay, which may be the case here depending on the linker. Specifically, you need to generate more 'decoys' (potential poses) the more unsure you are: tens of thousands or more. Then you have find clusters and see how each scores following more precise refinement. As an example of a tediously and manually set up run, Rosetta has &lt;a href=&quot;https://www.rosettacommons.org/demos/latest/tutorials/Protein-Protein-Docking/Protein-Protein-Docking&quot; rel=&quot;nofollow noreferrer&quot;&gt;several docking protocols&lt;/a&gt;, which explain the steps required —of note are the many judgement calls, such as in the local refinement, where one has to judge whether setting &lt;code&gt;-dock_pert 3 8&lt;/code&gt; (3 Å and 8° perturbation) is too strict etc.&lt;/p&gt;&#xA;&lt;h2&gt;Pretty picture&lt;/h2&gt;&#xA;&lt;p&gt;The protein-protein docking approach will result in a globular protein, which may be incorrect and majorly is not great for pictures.&lt;/p&gt;&#xA;&lt;p&gt;If you want a pretty picture of two or more domains most likely you would want them placed like beads on a string. If you have a linker that is not solved the recommended distance to space them is &lt;span class=&quot;math-container&quot;&gt;$(3.5 N_{missing})^{0.5}$&lt;/span&gt; Å —unfortunately I do not know where this lore comes from. If it helps I wrote a &lt;a href=&quot;https://github.com/matteoferla/protein_fuser&quot; rel=&quot;nofollow noreferrer&quot;&gt;script&lt;/a&gt; that does this in automatic but manual placement works best.&lt;/p&gt;&#xA;&lt;h2&gt;Linkers&lt;/h2&gt;&#xA;&lt;p&gt;In the situation you have a good confidence about how the two parts fit together (or have placed them like beads on a string), you may want to add linkers. If you absolutely don't care about correct bond topology &lt;a href=&quot;https://blog.matteoferla.com/2017/10/hacking-pdbs-for-fusion-protein.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here is a blog post on how to join bits (shoddily) with PyMOL&lt;/a&gt; and &lt;a href=&quot;http://blog.matteoferla.com/2020/07/filling-missing-loops-proper-way.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here is one to do it properly with Rosetta or Pyrosetta&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If one wanted to do a better job, one tool to use is &lt;a href=&quot;https://www.rosettacommons.org/docs/latest/application_documentation/design/Remodel&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rosetta remodel&lt;/a&gt;, which has a bit of a learning curve, but is very powerful for protein design. This requires the PDB to be on a single chain (pymol &lt;code&gt;alter&lt;/code&gt; command, followed by &lt;code&gt;create&lt;/code&gt; will do the trick: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/11579/merge-multiple-pdb-files-into-a-single-file/11580#11580&quot;&gt;more here&lt;/a&gt;) and actually the amino acids to not have gaps —PDB number being the same as the pose numbering. Rosetta gives better results if the poses are energy minimised, but for this it does not matter too much. Then a &amp;quot;blueprint file&amp;quot; formatted as discussed in the documentation. Briefly, in a way that residues left alone are in the form &lt;code&gt;1 M .&lt;/code&gt; (where residue 1 is a methionine), while the residues before and after the insertion are like &lt;code&gt;100 D L PIKAA D&lt;/code&gt; (meaning D100 is changed to a Asp in loop SS; not actual numbers or residue), while between entries are added &lt;code&gt;0 X L PIKAA G&lt;/code&gt; etc. To model loops, &lt;code&gt;-generic_aa G&lt;/code&gt; is a good option as this tells the default rough fitting (centroid mode) to use glycine and not valine as generic residue.&#xA;The top pose will actually not have a superb score, so a 3 (regular) or more cycles (15 thorough) with Rosetta Relax application either of the whole protein or with a movemap specific to the linker, may be done.&lt;/p&gt;&#xA;&lt;p&gt;There actually is an option in Rosetta remodel to have a domain insertion, i.e. it would swing around like a medieval flail. However, it will not repack and therefore properly dock so this option is a bad idea for this specific application.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-07-04T20:54:41.587" LastActivityDate="2020-07-04T20:54:41.587" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13104" PostTypeId="1" CreationDate="2020-04-27T11:32:26.863" Score="1" ViewCount="180" Body="&lt;p&gt;I would like to ask how I can interpret objectively multiple alignment results. I have used JALVIEW  and TCOFFEE.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I would like to ask if consistency score plays any role in the interpretation?&lt;/li&gt;&#xA;&lt;li&gt;I would like to ask for a region of a protein to be considered conserved, all the amino acids should have similar properties? So even if only one protein has a different amino acid (among the proteins that are included in the multiple alignment), then there is no conservation? for example, in this picture,&lt;/li&gt;&#xA;&lt;li&gt;I am curious why there is no dot beneath some columns with almost identical amino acids (expect one sequence for example).&lt;a href=&quot;https://i.stack.imgur.com/CRb7F.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CRb7F.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="8713" LastEditorUserId="3848" LastEditDate="2020-06-24T20:18:55.623" LastActivityDate="2020-06-24T20:18:55.623" Title="How can I interpret multiple alignment results?" Tags="|sequence-alignment|proteins|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13105" PostTypeId="1" AcceptedAnswerId="13117" CreationDate="2020-04-27T12:40:09.607" Score="0" ViewCount="397" Body="&lt;p&gt;I want to use &lt;code&gt;MEGAHIT&lt;/code&gt; (git: &lt;a href=&quot;https://github.com/voutcn/megahit&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/voutcn/megahit&lt;/a&gt;) to assemble metagenome shotgun sequencing reads (illumina). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I install the package with the command &lt;code&gt;conda install -c bioconda megahit&lt;/code&gt; as described in on the github page which yields: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Solving environment: done&#xA;&#xA;## Package Plan ##&#xA;&#xA;  environment location: /home/datura-workstation/miniconda3&#xA;&#xA;  added / updated specs:&#xA;    - megahit&#xA;&#xA;&#xA;The following NEW packages will be INSTALLED:&#xA;&#xA;  megahit            bioconda/linux-64::megahit-1.2.9-h8b12597_0&#xA;&#xA;&#xA;Proceed ([y]/n)?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where I click yes and the packages is installed. However, after installation I am not able to use the command &lt;code&gt;megahit&lt;/code&gt; as this results in the error: &lt;code&gt;megahit: command not found&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use Ubuntu 18.04.3 LTS and have installed &lt;code&gt;miniconda3&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I reproduced the installation of &lt;code&gt;megahit&lt;/code&gt; on a windows computer with a virtual machine using the same Ubuntu version. On my virtual Ubuntu I CAN use the &lt;code&gt;megahit&lt;/code&gt; command.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could anyone please help me solve this problem? I searched the internet but cannot find a good solution on the issue (also not on the github issue page of this package).&lt;/p&gt;&#xA;" OwnerUserId="7476" LastActivityDate="2020-04-28T06:41:31.707" Title="&quot;megahit: command not found&quot; after correct installation with miniconda" Tags="|assembly|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="13106" PostTypeId="2" ParentId="13096" CreationDate="2020-04-27T14:59:49.550" Score="1" Body="&lt;p&gt;You could use the &lt;a href=&quot;https://github.com/openwdl/wdl/blob/master/versions/1.0/SPEC.md#string-substring-string-string&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;sub&lt;/code&gt;&lt;/a&gt; function to obtain the chromosome from the globbed file names and then use that to retrieve the correct LD matrices from a Map (see &lt;a href=&quot;https://github.com/openwdl/wdl/blob/master/versions/1.0/SPEC.md#mapstring-string-read_mapstringfile&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;read_map&lt;/code&gt;&lt;/a&gt; or &lt;a href=&quot;https://github.com/openwdl/wdl/blob/master/versions/1.0/SPEC.md#mixed-read_jsonstringfile&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;read_json&lt;/code&gt;&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;eg.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Map[String, String] ld_matrices = read_json(file_of_ld_matrices_by_chr)&#xA;&#xA;scatter(gwas_for_chr in split.gwas_by_chr) {&#xA;    String chr = sub(sub(gwas_for_chr, prefix, &quot;&quot;), &quot;\.assoc$&quot;, &quot;&quot;)&#xA;&#xA;    call sbayes.run {&#xA;        input:&#xA;            GCTB = gctb_executable_path,&#xA;            gwas = gwas_for_chr,&#xA;            ld_matrix = ld_matrices[chr],&#xA;            output_prefix = out&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;file_of_ld_matrices_by_chr&lt;/code&gt; would have to look like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;  &quot;chr1&quot;: &quot;/path/to/file1&quot;,&#xA;  &quot;chr2&quot;: &quot;/path/to/file2&quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6955" LastActivityDate="2020-04-27T14:59:49.550" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13107" PostTypeId="1" CreationDate="2020-04-27T16:58:36.877" Score="0" ViewCount="81" Body="&lt;p&gt;I am struggling a bit to model the following problem. Basically, I would like to model tumor mutation burden (mutations per megabase, a continuous) as a function of treatment (categorical). The following is an example of how data looks like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;patient-id therapy-1 therapy-2 therapy-3 cancer-type TMB&#xA;PID-1      1         NA        1         CT-1        1.3&#xA;PID-2      0         0         1         CT-1        0.14 &#xA;PID-3      NA        0         1         CT-2        5&#xA;PID-4      1.        1         NA        CT-3        2.3&#xA;PID-5      1         1         NA        CT-3        10&#xA;...&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;1 - treated; 0 - not treated; NA - No information&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was trying multiple linear regression, but in most cases one patient got multiple therapies. This is where I am confused on how to interpret the results of linear regression. So far, I haven't included the &lt;code&gt;cancer-type&lt;/code&gt; information into the model. However, IMO, including cancer-type into model will yield specific treatment effects in specific cancer type? Or is it over-expectation of what we can achieve from linear model? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Also, is there a better way to address the problem to get reasonable associations between treatment and TMB in this case?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for the help. Stay safe!&lt;/p&gt;&#xA;" OwnerUserId="8731" LastActivityDate="2024-01-08T03:09:06.090" Title="Model the effect of treatment on mutation burden" Tags="|r|python|mutations|linear-regression|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13108" PostTypeId="1" CreationDate="2020-04-27T17:21:19.877" Score="0" ViewCount="434" Body="&lt;p&gt;I have two outputs of macs2 peak calling for two of my data sets. I wanted to plot the Venn diagram to see how many peaks are shared. I mainly work on Unix/Linux. Do you know any way that I can have those diagrams plotted?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Much Appreciated&lt;/p&gt;&#xA;" OwnerUserId="8543" LastActivityDate="2020-05-27T22:26:12.213" Title="How to make Venn diagram for the Macs peak calling output of two data sets?" Tags="|sequence-alignment|sequence-analysis|chip-seq|peak-calling|macs2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13109" PostTypeId="2" ParentId="13108" CreationDate="2020-04-27T17:28:53.647" Score="1" Body="&lt;p&gt;Check out the tool &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-017-1708-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;Intervene&lt;/code&gt;&lt;/a&gt;. It's exactly designed for this. &lt;a href=&quot;https://github.com/asntech/intervene&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github link&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="8731" LastEditorUserId="3848" LastEditDate="2020-05-27T22:26:12.213" LastActivityDate="2020-05-27T22:26:12.213" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13110" PostTypeId="1" CreationDate="2020-04-27T19:38:02.967" Score="0" ViewCount="125" Body="&lt;p&gt;I have a file that contains species name, chromosome, start and end locations like the following format&#xA;&lt;code&gt;hg19,chr8,77778733,77779026&lt;/code&gt;. I would like to know some information about the regions &lt;code&gt;77778733-77779026&lt;/code&gt;. How can I quickly find out if it lies in an intron or exon of a gene? Or it is in the upstream or downstream regions of a gene? Is there a tool or web service from where I can get this information? &lt;/p&gt;&#xA;" OwnerUserId="2594" LastActivityDate="2020-04-29T10:22:58.930" Title="Gene information from chromosome location" Tags="|chromosomes|hg19|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13111" PostTypeId="2" ParentId="13107" CreationDate="2020-04-27T19:43:07.907" Score="0" Body="&lt;p&gt;Depends how you consider &quot;linear regression&quot;, if you mean Pearsons, Spearman's analysis then general linear modelling (GLM) is a far better approach if you have continuous data. I don't really understand TMB, but that is a critical output then GLM will work. Logistic regression is an alternative approach data type depending. You have to perform a Q-Q plot and the residual against against any transformations you make (you might need to transform TMB). The lower the residual the better, the straight the Q-Q plot the better.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To explain how GLM works long hair and height correlate in a straight regression analysis. The implication is if you want to be tall - cut your hair! Of once you account for gender and send it through a GLM then gender and hairlength will correlate far stronger than hair length and height.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mac OSX has a very easy to use pointy clicky, but highly professional, GLM package on its AppStore called Wizzard (bit expensive). It doesn't do general linear mixed modelling however. Everything does GLM of course, its just this package is really easy to use and you can figure out GLM by trial and error. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-04-27T19:43:07.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13112" PostTypeId="2" ParentId="13010" CreationDate="2020-04-27T20:29:16.153" Score="1" Body="&lt;p&gt;@vantom, while I am not very familiar with snakemake on a whole, I do have extensive knowledge writing workflows in &lt;a href=&quot;https://openwdl.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;WDL&lt;/a&gt; as well as CWL. The approach both of these languages take to overcome a similar issue that you are describing, is to use containerization through ie &lt;a href=&quot;https://docs.docker.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;docker&lt;/a&gt;. This approach allows you to bundle the specific tools you need for a specific task, making different images for different tasks. There are several large repositories of images online ie: &lt;a href=&quot;https://hub.docker.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;Docker Hub&lt;/a&gt;, or more biologically focused: &lt;a href=&quot;https://biocontainers.pro/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biocontainers&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Looking through the documentation, it does appear that snakemake supports  &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/snakefiles/deployment.html#running-jobs-in-containers&quot; rel=&quot;nofollow noreferrer&quot;&gt;running your rule from within the context of a docker image&lt;/a&gt;. Without knowing how your compute environment is setup, I am not sure if there is any additional configuration you would need, however the documentation seems to suggest that it should be as easy as describing the name of the container. As long as you can run docker, (or an alternative like singularity) you should be able to avoid the current headache you are facing.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-yaml prettyprint-override&quot;&gt;&lt;code&gt;rule NAME:&#xA;    input:&#xA;        &quot;table.txt&quot;&#xA;    output:&#xA;        &quot;plots/myplot.pdf&quot;&#xA;    containers:&#xA;        &quot;docker://joseespinosa/docker-r-ggplot2&quot;&#xA;    script:&#xA;        &quot;scripts/plot-stuff.R&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7253" LastActivityDate="2020-04-27T20:29:16.153" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13113" PostTypeId="2" ParentId="13110" CreationDate="2020-04-28T01:16:46.377" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;How can I quickly find out if it lies in an intron or exon of a gene?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I have an answer on Biostars that might help: &lt;a href=&quot;https://www.biostars.org/p/124515/#124522&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/124515/#124522&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;Or it is in the upstream or downstream regions of a gene?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;One option is to build a table of annotated bins and then map your regions-of-interest to those annotations. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These annotations would consist of whether the bin is inside a gene, or, after picking the nearest gene by strand, locate the bin to an upstream region of the gene TSS or to an downstream region of the gene TES.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To start, one could download stranded gene annotations:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ GENCODE_URL=ftp://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_28/gencode.v28.basic.annotation.gff3.gz&#xA;$ GENCODE_GENES=${PWD}/../data/gencode.v28.genes.bed&#xA;$ wget -qO- &quot;${GENCODE_URL}&quot; | gunzip -c | awk '($3==&quot;gene&quot;)' | gff2bed - | awk -v FS=&quot;\t&quot; -v OFS=&quot;\t&quot; '{ n=split($10,a,&quot;;&quot;); p=a[4]; n=split(p,a,&quot;=&quot;); $4=a[2]; print $0; }' | cut -f1-6 &amp;gt; ${GENCODE_GENES}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Chop up nuclear chromosomes of the genome of interest into 1kb bins (or whatever gradation of bin size you prefer):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ ASSEMBLY=hg38&#xA;$ GENOME=${PWD}/../data/${ASSEMBLY}.bed&#xA;$ CHOP_SIZE=1000&#xA;$ CHOPPED_GENOME=${PWD}/../data/chopped.${ASSEMBLY}.bed&#xA;$ fetchChromSizes ${ASSEMBLY} | awk -v FS=&quot;\t&quot; -v OFS=&quot;\t&quot; '{ print $1, &quot;0&quot;, $2 }' | grep -ve &quot;\.*_\.*&quot; | grep -v &quot;chrM&quot; | sort-bed - &amp;gt; ${GENOME}&#xA;$ bedops --chop ${CHOP_SIZE} ${GENOME} &amp;gt; ${CHOPPED_GENOME}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then assign the midpoints of bins inside a gene, or to the nearest of any particular set of intergenic categories. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here's a script I wrote which uses the output of &lt;code&gt;closest-features&lt;/code&gt; to do that. This could be easily modified for any set of distance categories:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;&#xA;import sys&#xA;import numpy as np&#xA;&#xA;'''&#xA;Input should be bin midpoints run through closest-elements, against some set of BED6+ (stranded) gene annotations&#xA;'''&#xA;&#xA;def assign_to_category(dist):&#xA;  if dist &amp;lt; -50000:&#xA;    return '-50kb_to_-Inf'&#xA;  elif dist &amp;lt; -20000 and dist &amp;gt;= -50000:&#xA;    return '-20kb_to_-50kb'&#xA;  elif dist &amp;lt; -10000 and dist &amp;gt;= -20000:&#xA;    return '-10kb_to_-20kb'&#xA;  elif dist &amp;lt; -5000 and dist &amp;gt;= -10000:&#xA;    return '-5kb_to_-10kb'&#xA;  elif dist &amp;lt; -1000 and dist &amp;gt;= -5000:&#xA;    return '-1kb_to_-5kb'&#xA;  elif dist &amp;lt; 0 and dist &amp;gt;= -1000:&#xA;    return 'TSS_to_-1kb'&#xA;  elif dist == 0:&#xA;    return 'Gene'&#xA;  elif dist &amp;gt; 0 and dist &amp;lt;= 1000:&#xA;    return 'TES_to_+1kb'&#xA;  elif dist &amp;gt; 1000 and dist &amp;lt;= 5000:&#xA;    return '+1kb_to_+5kb'&#xA;  elif dist &amp;gt; 5000 and dist &amp;lt;= 10000:&#xA;    return '+5kb_to_+10kb'&#xA;  elif dist &amp;gt; 10000 and dist &amp;lt;= 20000:&#xA;    return '+10kb_to_+20kb'&#xA;  elif dist &amp;gt; 20000 and dist &amp;lt;= 50000:&#xA;    return '+20kb_to_+50kb'&#xA;  elif dist &amp;gt; 50000:&#xA;    return '+50kb_to_+Inf'&#xA;&#xA;for in_line in sys.stdin:&#xA;  (bin_str, upstream_gene_str, upstream_gene_dist, downstream_gene_str, downstream_gene_dist) = in_line.rstrip().split('|')&#xA;  try:&#xA;    upstream_gene_dist = int(upstream_gene_dist)&#xA;  except ValueError:&#xA;    upstream_gene_dist = np.nan&#xA;  try:&#xA;    downstream_gene_dist = int(downstream_gene_dist)&#xA;  except ValueError:&#xA;    downstream_gene_dist = np.nan&#xA;  upstream_gene_elems = upstream_gene_str.split('\t')&#xA;  downstream_gene_elems = downstream_gene_str.split('\t')&#xA;&#xA;  '''&#xA;  We have six possible scenarios, depending on the minimum distance and strandedness of closest elements&#xA;  '''&#xA;&#xA;  '''&#xA;  1) If either distance value is NaN, the bin is assigned to the non-NaN distance element&#xA;  '''&#xA;  if upstream_gene_dist is np.nan and downstream_gene_dist is np.nan:&#xA;    raise SystemError(&quot;Bad NaN input on:\n[{}]\n&quot;.format(in_line.rstrip()))&#xA;  elif upstream_gene_dist is np.nan:&#xA;    gene_id = downstream_gene_elems[3]&#xA;    if downstream_gene_elems[5] == '+':&#xA;      # bin is closer to downstream TSS&#xA;      sys.stdout.write('{}\t{}\t{}\n'.format(bin_str, assign_to_category(-(abs(downstream_gene_dist))), gene_id))&#xA;    else:&#xA;      # bin is closer to downstream TES&#xA;      sys.stdout.write('{}\t{}\t\n'.format(bin_str, assign_to_category(abs(downstream_gene_dist)), gene_id))&#xA;    continue&#xA;  elif downstream_gene_dist is np.nan:&#xA;    gene_id = upstream_gene_elems[3]&#xA;    if upstream_gene_elems[5] == '+':&#xA;      # bin is closer to upstream TES&#xA;      sys.stdout.write('{}\t{}\t{}\n'.format(bin_str, assign_to_category(abs(upstream_gene_dist)), gene_id))&#xA;    else:&#xA;      # bin is closer to upstream TSS&#xA;      sys.stdout.write('{}\t{}\t{}\n'.format(bin_str, assign_to_category(-(abs(upstream_gene_dist))), gene_id))&#xA;    continue&#xA;&#xA;  '''&#xA;  2) If either distance value is zero, the bin is assigned to &quot;Gene&quot; to state overlap with a gene&#xA;  '''&#xA;  if upstream_gene_dist == 0:&#xA;    gene_id = upstream_gene_elems[3]&#xA;    sys.stdout.write('{}\t{}\t{}\n'.format(bin_str, assign_to_category(0), gene_id))&#xA;    continue&#xA;&#xA;  if downstream_gene_dist == 0:&#xA;    gene_id = downstream_gene_elems[3]&#xA;    sys.stdout.write('{}\t{}\t{}\n'.format(bin_str, assign_to_category(0), gene_id))&#xA;    continue&#xA;&#xA;  '''&#xA;  3) If the upstream strand is positive, if the downstream strand is positive, then: &#xA;&#xA;   -- the upstream distance is a signed (-) distance to the upstream gene's TES &#xA;   -- the downstream distance is a signed (+) distance to the downstream gene's TSS&#xA;&#xA;  '''&#xA;  if upstream_gene_elems[5] == '+' and downstream_gene_elems[5] == '+':&#xA;    if abs(upstream_gene_dist) &amp;lt; abs(downstream_gene_dist):&#xA;      # bin is closer to upstream TES&#xA;      gene_id = upstream_gene_elems[3]&#xA;      sys.stdout.write('{}\t{}\t{}\n'.format(bin_str, assign_to_category(abs(upstream_gene_dist)), gene_id))&#xA;    else:&#xA;      # bin is closer to downstream TSS&#xA;      gene_id = downstream_gene_elems[3]&#xA;      sys.stdout.write('{}\t{}\t{}\n'.format(bin_str, assign_to_category(-(abs(downstream_gene_dist))), gene_id))&#xA;    continue&#xA;&#xA;  '''&#xA;  4) If the upstream strand is negative, if the downstream strand is positive, then: &#xA;&#xA;   -- the upstream distance is a signed (-) distance to the upstream gene's TSS &#xA;   -- the downstream distance is a signed (+) distance to the downstream gene's TSS&#xA;&#xA;  '''&#xA;  if upstream_gene_elems[5] == '-' and downstream_gene_elems[5] == '+':&#xA;    if abs(upstream_gene_dist) &amp;lt; abs(downstream_gene_dist):&#xA;      # bin is closer to upstream TSS&#xA;      gene_id = upstream_gene_elems[3]&#xA;      sys.stdout.write('{}\t{}\t{}\n'.format(bin_str, assign_to_category(-(abs(upstream_gene_dist))), gene_id))&#xA;    else:&#xA;      # bin is closer to downstream TSS&#xA;      gene_id = downstream_gene_elems[3]&#xA;      sys.stdout.write('{}\t{}\t{}\n'.format(bin_str, assign_to_category(-(abs(downstream_gene_dist))), gene_id))&#xA;    continue&#xA;&#xA;  '''&#xA;  5) If the upstream strand is positive, if the downstream strand is negative, then: &#xA;&#xA;   -- the upstream distance is a signed (-) distance to the upstream gene's TES &#xA;   -- the downstream distance is a signed (+) distance to the downstream gene's TES&#xA;&#xA;  '''&#xA;  if upstream_gene_elems[5] == '+' and downstream_gene_elems[5] == '-':&#xA;    if abs(upstream_gene_dist) &amp;lt; abs(downstream_gene_dist):&#xA;      # bin is closer to upstream TES&#xA;      gene_id = upstream_gene_elems[3]&#xA;      sys.stdout.write('{}\t{}\t{}\n'.format(bin_str, assign_to_category(abs(upstream_gene_dist)), gene_id))&#xA;    else:&#xA;      # bin is closer to downstream TES&#xA;      gene_id = downstream_gene_elems[3]&#xA;      sys.stdout.write('{}\t{}\t{}\n'.format(bin_str, assign_to_category(abs(downstream_gene_dist)), gene_id))&#xA;    continue&#xA;&#xA;  '''&#xA;  6) If the upstream strand is negative, if the downstream strand is negative, then: &#xA;&#xA;   -- the upstream distance is a signed (-) distance to the upstream gene's TES &#xA;   -- the downstream distance is a signed (+) distance to the downstream gene's TES&#xA;&#xA;  '''&#xA;  if upstream_gene_elems[5] == '-' and downstream_gene_elems[5] == '-':&#xA;    if abs(upstream_gene_dist) &amp;lt; abs(downstream_gene_dist):&#xA;      # bin is closer to upstream TSS&#xA;      gene_id = upstream_gene_elems[3]&#xA;      sys.stdout.write('{}\t{}\t{}\n'.format(bin_str, assign_to_category(-(abs(upstream_gene_dist))), gene_id))&#xA;    else:&#xA;      # bin is closer to downstream TES&#xA;      gene_id = downstream_gene_elems[3]&#xA;      sys.stdout.write('{}\t{}\t{}\n'.format(bin_str, assign_to_category(abs(downstream_gene_dist)), gene_id))&#xA;    continue&#xA;&#xA;  '''&#xA;  Sanity check: If we get here, something went wrong&#xA;  '''&#xA;  raise SystemError(&quot;Bad logic on:\n[{}]\n&quot;.format(in_line.rstrip()))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To use it, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ MP_LEFT=500&#xA;$ MP_RIGHT=-499 &#xA;$ MP_LEFT_UNDO=-500&#xA;$ MP_RIGHT_UNDO=499&#xA;$ CATEGORIZED_BINS=${PWD}/../data/categorized_bins.${ASSEMBLY}.bed&#xA;$ ./assign_chops.py &amp;lt; &amp;lt;(bedops --range ${MP_LEFT}:${MP_RIGHT} --everything ${CHOPPED_GENOME} | closest-features --dist - ${GENCODE_GENES}) | sort-bed - | bedops --range ${MP_LEFT_UNDO}:${MP_RIGHT_UNDO} --everything - &amp;gt; ${CATEGORIZED_BINS}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output from this file annotates bins with the closest feature to a gene. It looks something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ more categorized_bins.hg38.bed&#xA;...&#xA;chr1  32000 33000 TES_to_+1kb RP11-34P13.3&#xA;chr1  33000 34000 TES_to_+1kb FAM138A&#xA;chr1  34000 35000 Gene  FAM138A&#xA;chr1  35000 36000 Gene  FAM138A&#xA;chr1  36000 37000 Gene  FAM138A&#xA;chr1  37000 38000 TSS_to_-1kb FAM138A&#xA;chr1  38000 39000 -1kb_to_-5kb  FAM138A&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For example, the region &lt;code&gt;chr1:33000-34000&lt;/code&gt; is closest to the 1kb window downstream of the TES of &lt;code&gt;FAM138A&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you have this file, you can quickly map it to your sorted regions-of-interest file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --echo --echo-map --delim '\t' regions-of-interest.bed categorized_bins.hg38.bed &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The file &lt;code&gt;answer.bed&lt;/code&gt; will contain your regions of interest and the annotated bins which overlap them.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2020-04-28T01:16:46.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13115" PostTypeId="2" ParentId="13084" CreationDate="2020-04-28T05:25:09.927" Score="1" Body="&lt;p&gt;You haven't allocated the requisite 160GB of RAM required. Or rather, you allocated 160GB of memory, which is what you were told to do, but they most likely meant 160&lt;strong&gt;GiB&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Computers use binary, not decimal, so a gigabyte is 1024 megabytes. This is the same annoying mix-up people run into when their hard drives don't have the space the marketing department says it does. By requesting 160000M in your job script, you actually requested a few gigabytes less than what your source says you need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;160000 \text{ megabytes} \times \frac{1 \text{ gigabyte}}{1024 \text{ megabytes}} = 156.25 \text{ gigabytes}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;160 \text{ gigabytes} \times \frac{1024 \text{ megabytes}}{1 \text{ gigabyte}} = 163840 \text{ megabytes}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If &quot;160GB&quot; is what you need, then &lt;code&gt;163840M&lt;/code&gt; is the correct value for the job script.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would honestly suggest &lt;a href=&quot;https://en.wikipedia.org/wiki/Gordian_Knot&quot; rel=&quot;nofollow noreferrer&quot;&gt;cutting the Gordian knot&lt;/a&gt; and not dealing with this &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Gigabyte&quot; rel=&quot;nofollow noreferrer&quot;&gt;gigabytes&lt;/a&gt; vs. &lt;a href=&quot;https://en.wikipedia.org/wiki/Gordian_Knot&quot; rel=&quot;nofollow noreferrer&quot;&gt;gibibytes&lt;/a&gt;&quot; ordeal by simply using the &lt;code&gt;G&lt;/code&gt; suffix.&lt;sup&gt;&lt;a href=&quot;https://slurm.schedmd.com/sbatch.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#SBATCH --mem=160G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="961" LastActivityDate="2020-04-28T05:25:09.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13116" PostTypeId="2" ParentId="13010" CreationDate="2020-04-28T06:11:28.257" Score="3" Body="&lt;p&gt;Each YAML file will always create its own environment. The packages in each aren't downloaded and installed each time, if they're already installed elsewhere they're just hard-linked in. You'll want to use both &lt;code&gt;--use-conda&lt;/code&gt; and &lt;code&gt;--conda-prefix&lt;/code&gt;. The latter will point to the base directory in which all conda environments will be stored. This has the benefit that running the same workflow in different places won't result in a new env being created (by default snakemake will do so inside a .snakemake directory otherwise).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-04-28T06:11:28.257" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13117" PostTypeId="2" ParentId="13105" CreationDate="2020-04-28T06:41:31.707" Score="2" Body="&lt;p&gt;type &lt;code&gt;conda info --envs&lt;/code&gt;.  Look at your list of environments.  Say it's &lt;code&gt;base&lt;/code&gt; that has &lt;code&gt;megahit&lt;/code&gt; in it, do &lt;code&gt;conda activate base&lt;/code&gt;, then just do &lt;code&gt;megahit&lt;/code&gt;. &lt;/p&gt;&#xA;" OwnerUserId="3381" LastActivityDate="2020-04-28T06:41:31.707" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13118" PostTypeId="1" CreationDate="2020-04-28T08:33:19.820" Score="3" ViewCount="85" Body="&lt;p&gt;I am wondering if I am doing something wrong here.&lt;/p&gt;&#xA;&lt;p&gt;I ran &lt;code&gt;Pindel&lt;/code&gt; (alongside two other structural variant [SV] callers) on four WGS samples. The two other callers report approximately 50000 SVs and have a decent running time.&lt;/p&gt;&#xA;&lt;p&gt;However, Pindel takes long to run and its output files are very large (VCF files of 9 GB, more than a million detected SVs). I never used Pindel before, but I am wondering if I am doing something wrong here.&lt;/p&gt;&#xA;&lt;p&gt;My command to call Pindel (from Python to cmd) is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pindel -f {} -i {} -o {} -T {}.format(reference, config_file, output_path, threads)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;PS: I use Pindel version 0.2.5b9&lt;/p&gt;&#xA;&lt;p&gt;I am mainly confused how Pindel reports so many more calls in comparison to the other two callers. I appreciate any suggestions or remarks!&lt;/p&gt;&#xA;" OwnerUserId="8739" LastEditorUserId="3848" LastEditDate="2020-07-15T11:50:01.900" LastActivityDate="2020-07-15T11:50:01.900" Title="Pindel has an extremely large output" Tags="|vcf|variant-calling|structural-variation|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13119" PostTypeId="2" ParentId="13100" CreationDate="2020-04-28T08:39:30.913" Score="0" Body="&lt;p&gt;I don't know if it means anything but...&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;It seems you are using anaconda instead of miniconda as required by the snakemake &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/getting_started/installation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;docs&lt;/a&gt;. Maybe reinstall conda using miniconda3.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;conda is installing python 3.6 instead of the more recent 3.8... odd&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="339" LastActivityDate="2020-04-28T08:39:30.913" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13120" PostTypeId="2" ParentId="12973" CreationDate="2020-04-28T11:11:04.520" Score="1" Body="&lt;p&gt;reformat df2 so that you have &lt;code&gt;chr1&lt;/code&gt; is instead of &lt;code&gt;1&lt;/code&gt;, then you basically have two bed files.  Then use &lt;code&gt;bedtools intersect&lt;/code&gt;.  &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/intersect.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedtools.readthedocs.io/en/latest/content/tools/intersect.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3381" LastActivityDate="2020-04-28T11:11:04.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13121" PostTypeId="1" AcceptedAnswerId="13122" CreationDate="2020-04-28T11:20:48.167" Score="0" ViewCount="848" Body="&lt;p&gt;My population is divided into 10 groups. I want to run the cox-proportional hazard model separately for each of these 10 groups so that in the end i can compare which groups has highest hazard ratio depending uopn the covariates.&#xA;How can i do this in R? Earlier i was using this formula.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Model_1 &amp;lt;- coxph(serve_object ~ GROUP + age+ Smoking+ Diabetes, data = DF&#xA;summary(Model_1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But the output is very confusing. it gives hazard ratio of 9 out of 10 groups. I think it dealt group as covariate. &lt;/p&gt;&#xA;" OwnerUserId="8654" LastActivityDate="2020-04-28T12:28:00.133" Title="How to run Cox proportional Hazard model for each group in R?" Tags="|statistics|statsmodels|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13122" PostTypeId="2" ParentId="13121" CreationDate="2020-04-28T12:14:06.410" Score="2" Body="&lt;p&gt;The fact that you supplied &lt;code&gt;GROUP&lt;/code&gt; as a covariate, you are getting 9 coefficients as one of the ten levels is used as the reference level.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would like to fit one model per group (and in this scenario you would not be correcting for &lt;code&gt;GROUP&lt;/code&gt;), you should first split your data frame and fit your models on the resulting list's elements:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;lapply(split(DF, DF$GROUP),&#xA;       function(x) coxph(serve_object ~ age + Smoking + Diabetes, data = DF))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Example with the &lt;code&gt;mtcars&lt;/code&gt; data set when grouped by number of cylinders, &lt;code&gt;cyl&lt;/code&gt; having values of 4, 6 or 8:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; lapply(split(mtcars, mtcars&lt;span class=&quot;math-container&quot;&gt;$cyl),&#xA;+        function(x) lm(disp ~ hp, data = x))&#xA;$&lt;/span&gt;`4`&#xA;&#xA;Call:&#xA;lm(formula = disp ~ hp, data = x)&#xA;&#xA;Coefficients:&#xA;(Intercept)           hp  &#xA;    59.0369       0.5579  &#xA;&#xA;&#xA;$`6`&#xA;&#xA;Call:&#xA;lm(formula = disp ~ hp, data = x)&#xA;&#xA;Coefficients:&#xA;(Intercept)           hp  &#xA;   290.9178      -0.8799  &#xA;&#xA;&#xA;$`8`&#xA;&#xA;Call:&#xA;lm(formula = disp ~ hp, data = x)&#xA;&#xA;Coefficients:&#xA;(Intercept)           hp  &#xA;   320.2083       0.1572  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2020-04-28T12:28:00.133" LastActivityDate="2020-04-28T12:28:00.133" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="13123" PostTypeId="2" ParentId="13075" CreationDate="2020-04-28T12:47:02.037" Score="0" Body="&lt;p&gt;To add to &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/13076/57&quot;&gt;@Maximilian Press' answer&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There is quite a lot of research on how to store arbitrary digital data in DNA. Nick Goldman describes his research on the topic in his &lt;a href=&quot;https://www.youtube.com/watch?v=a4PiGWNsIEU&quot; rel=&quot;nofollow noreferrer&quot;&gt;TEDx talk&lt;/a&gt;. The last progress in the field I have noticed was &lt;a href=&quot;http://dnafountain.teamerlich.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DNA fountain&lt;/a&gt;, a tranformation of bites in DNA with maximal efficiency of data storage in DNA (information / nt) and reliable recovery when sequenced afterwards.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2020-04-28T12:47:02.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13124" PostTypeId="2" ParentId="12973" CreationDate="2020-04-28T14:01:44.110" Score="0" Body="&lt;p&gt;In R:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; write.table(df1, 'df1.txt', quote=F, sep=&quot;\t&quot;, row.names=F, col.names=F)&#xA;&amp;gt; write.table(df2, 'df2.txt', quote=F, sep=&quot;\t&quot;, row.names=F, col.names=F)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Convert both files to sorted BED:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ sort-bed df1.txt &amp;gt; df1.bed&#xA;$ awk -v FS=&quot;\t&quot; -v OFS=&quot;\t&quot; '{ print &quot;chr&quot;$1 }' df2.txt | sort-bed - &amp;gt; df2.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then map &lt;code&gt;df1&lt;/code&gt; gene names to &lt;code&gt;df2&lt;/code&gt; coordinates:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bedmap --echo --echo-map-id-uniq --delim '\t' df2.bed df1.bed &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To bring back into R, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dfMap &amp;lt;- read.table('answer.bed', sep='\t', col.names=c('Chromosome', 'Start_Position', 'End_Position', 'Genes'), colClasses=c(&quot;factor&quot;, &quot;numeric&quot;, &quot;numeric&quot;, &quot;character&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="776" LastActivityDate="2020-04-28T14:01:44.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13125" PostTypeId="2" ParentId="13110" CreationDate="2020-04-28T14:27:49.283" Score="0" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;How can I quickly find out if it lies in an intron or exon of a gene?&#xA;  Or it is in the upstream or downstream regions of a gene? Is there a&#xA;  tool or web service from where I can get this information?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Since you emphasise speed and suggest a web service, I'm guessing you just want to check a few individual loci. In that case, just use a genome browser like those from UCSC or Ensembl. If you look up those coordinates in &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTracks?db=hg19&amp;amp;lastVirtModeType=default&amp;amp;lastVirtModeExtraState=&amp;amp;virtModeType=default&amp;amp;virtMode=0&amp;amp;nonVirtPosition=&amp;amp;position=chr8%3A77778732%2D77779026&amp;amp;hgsid=829268769_MD3OdRZimVWXMLwJzD0n3Bo8mah5&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC&lt;/a&gt; you can see the sequence falls within the last exon of the ZFHX4 protein coding gene and other information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/r8dpB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/r8dpB.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2020-04-28T14:27:49.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13126" PostTypeId="1" CreationDate="2020-04-28T14:29:36.693" Score="2" ViewCount="1329" Body="&lt;p&gt;I have a file with 16S sequences. some headers contain species information. For my purposes I would like to exclude a number of species from the file, therefore I would like to do a pattern matching within a 16S header (that contains other characters beyond the species name) to then remove that header and the following sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, I have:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;NCBI_ID1;high_rank_taxonomy;...;species_name1&#xA;ATCCTT&#xA;&amp;gt;NCBI_ID2;high_rank_taxonomy;...;species_name2&#xA;ATTTTCCC&#xA;&amp;gt;NCBI_ID3;high_rank_taxonomy;...;species_name3&#xA;TTTCCCCAAAA&#xA;&amp;gt;NCBI_ID4;high_rank_taxonomy;...;species_name4&#xA;CCCTTTAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I would like to exclude, say, species 1,2 and 4 to retain:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;NCBI_ID3;high_rank_taxonomy;...;species_name3&#xA;TTTCCCCAAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I found this solution in an adjacent question: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN{while((getline&amp;lt;&quot;ids.txt&quot;)&amp;gt;0)l[&quot;&amp;gt;&quot;$1]=1}/^&amp;gt;/{f=!l[$1]}f' seq.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;where I have my species names in the file ids.txt and sequences in file seq.fa&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but that would work if the whole of my header was just the species name (contents of each line in ids.txt), which is not the case.&#xA;I thought I could be smart and use &lt;code&gt;*species_name1*&lt;/code&gt; instead of &lt;code&gt;species_name1&lt;/code&gt; as contents of ids.txt but that apparently doesn't work.&#xA;Could someone help me with modifying the current solution to accommodate the existence of additinal strings in the headers?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many thanks!&#xA;A&lt;/p&gt;&#xA;" OwnerUserId="8741" LastActivityDate="2020-04-29T09:29:18.733" Title="remove sequences from fasta file matching a string in the header" Tags="|fasta|bash|shell|awk|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13127" PostTypeId="1" CreationDate="2020-04-28T15:19:22.243" Score="0" ViewCount="90" Body="&lt;p&gt;Is there a software that can recognise the protein PDB/CIF files, and fold multiple lines of a same section into one line, so that it is easier to handle? I need to understand the format in CIF so as to create my own CIF file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, when &lt;code&gt;Notepad++&lt;/code&gt; is used to read a &lt;code&gt;.py&lt;/code&gt; file, it can fold the same section into one line. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;So for a PDB/CIF file, e.g. all the atom lines for a same chain can be folded into one line, and I can expand it if needed.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Like this picture, assuming &quot;This PC&quot; is the whole protein, while the other objects are the different chains.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/akVRW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/akVRW.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1806" LastEditorUserId="1806" LastEditDate="2020-04-29T00:49:24.803" LastActivityDate="2020-04-29T00:49:24.803" Title="Protein PDB/CIF text file reader that can fold a same section into one line" Tags="|pdb|text-processing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13128" PostTypeId="2" ParentId="13127" CreationDate="2020-04-28T16:07:00.403" Score="1" Body="&lt;p&gt;Programs that work with PDB and mmCIF files may present the structure as a tree, for example:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cfCVA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cfCVA.png&quot; alt=&quot;ccp4MG screenshot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;(this is a Find dialog from ccp4MG)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but I'd be surprised if a text editor had such a feature. Normally, you don't edit PDB files in a text editor.&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2020-04-28T16:07:00.403" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13130" PostTypeId="2" ParentId="13100" CreationDate="2020-04-28T20:51:57.527" Score="4" Body="&lt;p&gt;This is an example of the conda solver behaving incorrectly. In this case, it's choosing to pick a version that minimizes the number of dependencies required. Version 5.3.0 has 25 fewer dependencies, so if you let conda choose the version that's what it will pick. There's nothing you can really do about this other than specifying the version of snakemake that you want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: It's very likely that &lt;a href=&quot;https://github.com/QuantStack/mamba&quot; rel=&quot;nofollow noreferrer&quot;&gt;mamba&lt;/a&gt; will correctly do what you want by default. Though I haven't used it, it's supposed to be a drop in replacement for conda here.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2020-05-02T10:33:04.037" LastActivityDate="2020-05-02T10:33:04.037" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13133" PostTypeId="1" CreationDate="2020-04-29T06:49:43.197" Score="0" ViewCount="778" Body="&lt;p&gt;I am using &lt;code&gt;BioEdit&lt;/code&gt;. It can perform an align of multiple sequences. Can this sequence similarity be outputted as a value, e.g. 70%?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QDL9y.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QDL9y.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1806" LastEditorUserId="3848" LastEditDate="2020-04-29T11:39:12.840" LastActivityDate="2021-01-24T13:03:55.370" Title="Can I use BioEdit to quantify the sequence similarity?" Tags="|sequence-alignment|phylogeny|bioedit|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13134" PostTypeId="1" AcceptedAnswerId="13138" CreationDate="2020-04-29T08:08:39.727" Score="1" ViewCount="158" Body="&lt;p&gt;I am performing a haplotype and nucleotide diversity analysis of my Sanger sequences (from Watermelon mosaic virus coat protein region) using the Arlequin program. My aim is to see the probability that two randomly sampled alleles are different, and the expansion and genetic variation of the virus respectively.&lt;/p&gt;&#xA;&lt;p&gt;Some of my sequences have gaps (-) because the PCR reaction could not cover all my region of interest. After making the alignment in Mega and export in a Nexus file to DnaSp6 I saw that there are many options to export to the Arlequin program after DnaSP6. I do not know what is the best option to generate the output from DNAsp6 to Arlequin because I must choose one option for the sites with the alignment gaps option and another option for the invariable sites option.&lt;/p&gt;&#xA;&lt;p&gt;These are the options:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Sites with Alignment Gaps options:&#xA;&#xA;Not considered: These sites are ignored (complete deletion).&#xA;Considered: Gaps are considered just like another nucleotide variant (fifth state).&#xA;Only gaps are considered: Only gaps information is considered to build haplotypes.&#xA;&#xA;Invariable Sites options:&#xA;&#xA;Removed: Invariable (monomorphic) sites will not be included in the output file.&#xA;Included: Invariable (monomorphic) sites will be included in the output file.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From my point of view I think that it is more certain for my analysis (above explained) to use the Not consider gaps option (because they can buffer the information of the other sequences) and include invariable sites option because I want to analyse these parameters of my dataset.&lt;/p&gt;&#xA;&lt;p&gt;Is it sensible? I would like to know what is the best option for my dataset.&lt;/p&gt;&#xA;" OwnerUserId="7430" LastEditorUserId="4941" LastEditDate="2021-12-04T06:03:54.463" LastActivityDate="2021-12-04T06:03:54.463" Title="Generate an appropriate output from DNaSP6 to Arlequin program" Tags="|phylogeny|phylogenetics|haplotypes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13135" PostTypeId="2" ParentId="7458" CreationDate="2020-04-29T09:02:25.250" Score="2" Body="&lt;p&gt;You may either install the emboss package (&lt;a href=&quot;http://emboss.sourceforge.net/download/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://emboss.sourceforge.net/download/&lt;/a&gt;) and use the revseq program.&#xA;It contains many useful command-line programs such as the pairwise alignment tool needle.&lt;/p&gt;&#xA;" OwnerUserId="8749" LastActivityDate="2020-04-29T09:02:25.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13136" PostTypeId="2" ParentId="13126" CreationDate="2020-04-29T09:19:11.780" Score="2" Body="&lt;p&gt;you may use simple R commands, using the 'seqinr' package&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#load package&#xA;library(seqinr)&#xA;#load file containing sequences&#xA;data&amp;lt;-read.fasta(&quot;filename.fa&quot;)&#xA;#create a vector containing species names: these are the last part of the string&#xA;vec.names&amp;lt;-unlist(lapply(strsplit(names(data), &quot;;&quot;), function(x)x[length(x)]))&#xA;#find names to keep: indices which are not in the species to remove&#xA;species.to.remove&amp;lt;-c(&quot;species_name1&quot;, &quot;species_name2&quot;, &quot;species_name4&quot;)&#xA;vec.tokeep&amp;lt;-which(! vec.names %in%  species.to.remove)&#xA;#write the final output&#xA;write.fasta(sequences=data[vec.tokeep], names=names(data)[vec.tokeep], file.out=&quot;output.fa&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8749" LastEditorUserId="8749" LastEditDate="2020-04-29T09:29:18.733" LastActivityDate="2020-04-29T09:29:18.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13137" PostTypeId="2" ParentId="13110" CreationDate="2020-04-29T10:22:58.930" Score="0" Body="&lt;p&gt;There is already some very useful answers out there but in my experience &lt;a href=&quot;http://homer.ucsd.edu/homer/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Homer&lt;/a&gt; does a very efficient job at annotating bed files. This gives you a lot of details about nearest transcription start sites or exonic regions and more.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Be sure to convert your data to &lt;a href=&quot;https://genome.ucsc.edu/FAQ/FAQformat.html#format1&quot; rel=&quot;nofollow noreferrer&quot;&gt;bed format&lt;/a&gt; first&lt;/p&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2020-04-29T10:22:58.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13138" PostTypeId="2" ParentId="13134" CreationDate="2020-04-29T11:29:22.197" Score="1" Body="&lt;p&gt;Yes, your question makes alot of sense and its a good question because hits at the heart of differences between population genetic analysis and phylogenetic analysis. However, when you speak about gaps you are referring to 'indels' (insertion/deletion).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Indels&lt;/strong&gt; The conservative approach is to simply delete sites with gaps. Arlequin uses a lot of maximum likelihood solutions and gaps would automatically be deleted in an Arlequin max. likelihood analysis, because it is the output per alignment per matrix. If the matrix doesn't have a 'gap'. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Indels and pop gen&lt;/strong&gt; Thats tricky, it is better to remove them. HOWEVER the indel may contain underlying population signal, if its a maximum likelihood analysis (pop gen does this too) I suspect they will be ignored. If its pairwise, such Fst including indels might give you a better answer because there is more polymorphisms to assess population structure. So if you're doing a pairwise analysis, I'd do it both ways and then report the methodology correctly if there are differences in results and simply explain the theory.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Invariable sites&lt;/strong&gt; This is different from an indel. An invariable site is a site which carries no phylogenetic signal, it carries information but a phylogenetic analysis could draw any conclusions because all the nucleotides/amino acids are identical at a give alignment position. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The output you obtain in allele frequency will be different and here's the rub ...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Phylogenetics analysis&lt;/strong&gt; Arlequin is really pop gen, but it will contain some phylogenetics programs. If you see anything with 'Jukes Cantor correction', 'Kimura 2 (or 3) parameter model', 'HGW model', '&quot;REV&quot; model' (eek I can't remember its name any its a reverible 6-parameter model of nucleotides),  you are dealing with phylogenetics. Keeping invariable sites is normal here because the rate matrix is calculation &quot;per alignment site&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Population genetics analysis&lt;/strong&gt; Fst, Fis, linkage disequilibrium statistics, REMOVE THEM. Why? Because you want SNPs (single nucleotide polymorphisms). The invariable sites will get in the way of the statistics. For example is the locus recombines and you want to assess a linkage disequilibrium statistic (recombination), invariable sites will recombine, but the algorithm can't see this because the nucleotides are all the same. The only thing the algorithm can see and analyse are SNPs, which 'representative' of all the surrounding invariant sites. Thus removing invariants is essential. Arelquin might do this automatically (but I don't know the preparatory algorithms), but if you submit multiple loci Arlquin will want to know which are different loci. It will then recode the loci automatically if a single locus comprises more than one SNP. We are going into the underlying theory of linkage disequilibrium here and I'm not sure if say Fst/Fis is particularly bothered with this or is just concerned with the populations of genetic diversity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; So gaps ... if they are genuine indels and not just 'N', ie indeterminant nucleotide calls, remove them for phylogenetics. If its pop gen do the calculation both ways and assess the output, if there is no difference report the 'gap/indel removal' result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Invariant sites .. for pop gen you remove them but Arlequin might do this automatically to prepare a data set for analysis (I don't know). For phylogenetics you never remove invariant sites.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Goodluck&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-04-29T11:29:22.197" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13139" PostTypeId="2" ParentId="13133" CreationDate="2020-04-29T11:37:14.987" Score="1" Body="&lt;p&gt;No Bioedit is an alignment editor only. It is not in anyway an analysis program.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Dump the format in fasta or mega (its the option is available) and import it into Mega X &lt;a href=&quot;https://www.megasoftware.net/dload_mac_beta&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. You can quantify the diversity as a pairwise matrix, population analysis (if you have a clear a priori that you can subdivide your sequences into different groups) or phylogeny ... which provides diversity as mutations per alignment position. Its pointy and clicky and easy to use. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-04-29T11:37:14.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13141" PostTypeId="2" ParentId="11511" CreationDate="2020-04-29T13:37:06.470" Score="1" Body="&lt;p&gt;In case you are still interested in just getting the protein ids and not sequence, you can use &lt;code&gt;xtract&lt;/code&gt; like this: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;efetch -db nucleotide -format gpc -id MT039887 | xtract  -insd CDS gene product protein_id &#xA;MT039887.1  orf1ab  orf1ab polyprotein           QHZ00388.1&#xA;MT039887.1  S       surface glycoprotein         QHZ00389.1&#xA;MT039887.1  orf3a   orf3a protein                QHZ00390.1&#xA;MT039887.1  E       envelope protein             QHZ00391.1&#xA;MT039887.1  M       membrane glycoprotein        QHZ00392.1&#xA;MT039887.1  orf6    orf6 protein                 QHZ00393.1&#xA;MT039887.1  orf7a   orf7a protein                QHZ00394.1&#xA;MT039887.1  orf8    orf8 protein                 QHZ00395.1&#xA;MT039887.1  N       nucleocapsid phosphoprotein  QHZ00396.1&#xA;MT039887.1  orf10   orf10 protein                QHZ00397.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2020-04-29T13:37:06.470" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13145" PostTypeId="2" ParentId="11492" CreationDate="2020-04-29T20:27:28.003" Score="1" Body="&lt;p&gt;In case anyone was interested, a new piece of software for rapidly imputing and phasing low coverage sequence data from genotype likelihoods has just been released. It is called &lt;a href=&quot;https://github.com/odelaneau/GLIMPSE/tree/master/phase&quot; rel=&quot;nofollow noreferrer&quot;&gt;GLIMSPE&lt;/a&gt; and is exactly what I was after. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks to be much faster (1200x speedup) compared to Beagle 4.0, and is more accurate at imputing low frequency variants. It is also more amenable to use with larger reference panels (such as the Haplotype Reference Consortium panel). &lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2020-04-29T20:27:28.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13146" PostTypeId="1" CreationDate="2020-04-30T02:48:04.313" Score="1" ViewCount="62" Body="&lt;p&gt;Or do I need to find individual studies and obtain data the long way!&lt;/p&gt;&#xA;" OwnerUserId="8756" LastActivityDate="2020-04-30T13:58:58.290" Title="Is there a single cell RNAseq equivalent of GTEx or TCGA?" Tags="|rna-seq|scrnaseq|single-cell|gtex|" AnswerCount="1" CommentCount="1" ClosedDate="2020-05-01T17:08:25.007" ContentLicense="CC BY-SA 4.0" />
  <row Id="13147" PostTypeId="1" CreationDate="2020-04-30T12:29:16.467" Score="1" ViewCount="59" Body="&lt;p&gt;I want to map some gene ids to their proteins structures if they exist.&#xA;The genes are rows of this &lt;a href=&quot;https://science.sciencemag.org/highwire/filestream/709675/field_highwire_adjunct_files/5/aap7847_Zhang_SM_Table-S5.xlsx&quot; rel=&quot;nofollow noreferrer&quot;&gt;CSV&lt;/a&gt;. There are other information that may help also such as Product description but that may lead to anbiguities. I am looking for a programmatic way, biopython or pdb API?&lt;/p&gt;&#xA;" OwnerUserId="3861" LastActivityDate="2020-04-30T20:18:28.013" Title="Map gene IDs to protein structure if exist" Tags="|gene|protein-structure|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13148" PostTypeId="2" ParentId="13146" CreationDate="2020-04-30T13:58:58.290" Score="2" Body="&lt;p&gt;I think the most well-known scRNA dataset that is similar to GTEx in the Human Cell Atlas (&lt;a href=&quot;https://www.humancellatlas.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.humancellatlas.org&lt;/a&gt;). It is pretty much still ongoing but there are some preliminary data.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-04-30T13:58:58.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13149" PostTypeId="2" ParentId="12892" CreationDate="2020-04-30T16:15:09.317" Score="2" Body="&lt;p&gt;The UMAP results look very pretty :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I were to try to make sense of this blob, I'd try to do the following.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Pull out Gene Ontology (GO) terms for a list of candidate genes (you can try to subset Uniprot / Swissprot for some examples). Then I feed through some genes through your fragmented BERT kmer approach to see where they land. I'd be particularly interested in seeing where transmembrane proteins and enzymes land here.&lt;/li&gt;&#xA;&lt;li&gt;I'd try to color this by introns vs exons (you should be able to extract this sort of information from your genome files). There is a good chance that the difference between introns / exons could explain some of these differences.&lt;/li&gt;&#xA;&lt;li&gt;It maybe worth looking into genes that only encode RNAs vs genes that encode proteins. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="8761" LastActivityDate="2020-04-30T16:15:09.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13150" PostTypeId="2" ParentId="12892" CreationDate="2020-04-30T16:54:40.220" Score="1" Body="&lt;p&gt;In truth deep neural networks quite mainstream now, albeit they are a pain because of their reliance on very expensive graphics cards to process the data (GPU) rather than traditional CPU.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are two issues,&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The algorhithms need an application to a biological phenomena and this is dependent on the model. DNA applications (usually) use a different model to language applications (below) &lt;/li&gt;&#xA;&lt;li&gt;The model is only as good what it is trained against. In languages this would be people saying stuff, like in your example. In biology it is a phenotype. Thus you need to feed it the right answer and all variants of the right answer. It will then spot the pattern and recognise this pattern in de novo inputs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;What we are talking is deep learning using a neural network  yes we use them, but again the models are (usually)  different. Convolution neural networks (CNN) are common in DNA/genomics because it assess the physical linkage of DNA. CNN is also used in image recognition, so like when you type 'dog' into google and see dog images, chances are the search is a consequence of CNN classification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In languages recurrent neural networks particularly long short term memory (LSTM) models are common because they need to learn the patterns and idioms of a language. Google of course will only use their development called TensorFlow to build their models, which is very geeky, for example they have a programming construct called a &quot;placeholder&quot;. In my view a placeholder is something found in Adobe PhotoShop rather than a programming language etc... TensorFlow exponents outside Google tend to use the Keras interface. In Python there is Natural Language Toolkit — NLTK 3.5 to do this.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; So, to answer your question, by dumping DNA sequence into a language RNN its fun, but it wouldn't mean very much because the evolution of language, or construction of a given language (i.e. the training set) is a very different data to mechanism(s) involved in the evolution of DNA. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Fundamentally, we don't speak DNA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think it would be fun dumping whale songs and dolphin squeaks into these programs to see whether there is any convergence with our own speech.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;My advice is to use an unsupervised training approach to identify your clusters. This would be model-free and training-free and would help further analysis in &quot;supervised-learning&quot; OR just stick with the clusters it generates.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-04-30T22:47:07.113" LastActivityDate="2020-04-30T22:47:07.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13152" PostTypeId="1" CreationDate="2020-04-30T20:06:08.497" Score="1" ViewCount="587" Body="&lt;p&gt;I am new to the area of bioinformatics, so apologies if this is too obvious of a query.&lt;/p&gt;&#xA;&lt;p&gt;I need to analyze the RNA Seq data from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE98455&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSE98455&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The dataset is of the following format:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;|---------------------|------------------|&#xA;|    Some Id          |      Counts      | &#xA;|---------------------|------------------|&#xA;|    13101.t00001     |       392        |&#xA;|---------------------|------------------|&#xA;|    13101.t00002     |        20        |&#xA;|---------------------|------------------|&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to extract the counts for certain genes only. So how do I match the ID column with gene names? Is there a reference table or something I need to look at?&lt;/p&gt;&#xA;&lt;p&gt;Side Note: Usually for microarray data, I refer to the GPL files which provide a reference between accession ID and Gene Names. I was not able to do that for this RNA Seq Data set.&lt;/p&gt;&#xA;&lt;p&gt;Update May 2, 2020: I reached out to the authors earlier today. I looked at the paper associated with this dataset, it seems the annotation belongs to &lt;em&gt;Oryza sativa japonica&lt;/em&gt; reference genome v 6. I can't find anything related to it on the web. If I get any response from the authors, I will update it here.&lt;/p&gt;&#xA;" OwnerUserId="8754" LastEditorUserId="3541" LastEditDate="2021-07-14T10:37:51.907" LastActivityDate="2021-07-14T10:37:51.907" Title="How to convert Illumina IDs to Gene Names?" Tags="|rna-seq|gene-expression|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13153" PostTypeId="2" ParentId="13147" CreationDate="2020-04-30T20:18:28.013" Score="2" Body="&lt;p&gt;Not sure if you want the structure exactly or a handle of some sort, but the best way i know to get ALL your 5000 identifiers to map to uniprot/pdb is uniprot's &lt;a href=&quot;https://www.uniprot.org/uploadlists/&quot; rel=&quot;nofollow noreferrer&quot;&gt;mapping service&lt;/a&gt;.&#xA;You can either copy your ids straight from excel or parse with python ex.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;import os&#xA;&#xA;file_name = 'aap7847_Zhang_SM_Table-S5.xlsx'&#xA;sheetname = 'Table S5'&#xA;df = pd.read_excel(file_name, sheet_name=sheetname, skiprows=1)&#xA;ids = list(df['Gene_ID'])&#xA;&#xA;name_of_file_to_save_to = &quot;geneIds.txt&quot;&#xA;with open(name_of_file_to_save_to, &quot;w&quot;) as newf:&#xA;    for id in ids:&#xA;        print(&quot;{}\n&quot;.format(id), file=newf)&#xA;    newf.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can then save the Uniprot's response and either add it as a column to your file or map to other parts of Elixir ecosystem with SIFTS, for example, as @Matteo Ferla says.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/MK1Qq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MK1Qq.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;In the case that you maybe need a single id's information, you could hit their API too, although this wouldn't work for bulk or at least didn't work for me. You can map the uniprot accession you get from this to PDB or whatever else you need!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;def get_uniprot_accession(geneid, format='list'):&#xA;    resp = requests.get(&#xA;        &quot;https://www.uniprot.org/uniprot/?query={}&amp;amp;sort=score&amp;amp;format={}&quot;.format(geneid, format)).text&#xA;    if format == 'list':&#xA;        resp = resp.strip()&#xA;    return resp&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6511" LastActivityDate="2020-04-30T20:18:28.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13155" PostTypeId="1" CreationDate="2020-04-30T23:45:06.000" Score="1" ViewCount="351" Body="&lt;p&gt;I have used both to transform my leukaemia RNA-Seq data for subsequent hierarchical clustering. The result is quite different. Some subtypes of leukaemia only form a cluster (or at least sit closer) in one of the transformed data set. While I understand both can be used to transform counts into gene expression matrix that can be use for clustering, I want to know more about the difference. From my really poor statistic background I assume voom() is very similar to rlog(CPM) and can generate a microarray-like data set. On the other hand, the name of vst() is quite self-explanatory and is described to be better at handling varying size factors than rlog() does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What are the actual difference and is there one that you think is better at handling jobs like clustering analysis?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am hoping this to be a more educational question than technical one. If it violates any rules please delete it.&lt;/p&gt;&#xA;" OwnerUserId="3225" LastActivityDate="2020-04-30T23:45:06.000" Title="vst() from DESeq2 vs voom() from limma" Tags="|rna-seq|deseq2|clustering|limma|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13156" PostTypeId="1" CreationDate="2020-05-01T07:02:02.673" Score="7" ViewCount="164" Body="&lt;p&gt;I am aligning 4 genomes of &lt;em&gt;Psuedomonas chlororaphis&lt;/em&gt; using progressive mauve software. Command used:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;progressiveMauve --output=outputfile --disable-backbone chl189.fas chl17415.fas chl50083.fas chlTAMOak81.fas&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;At the end, the resulting aligned genomes are not of the same length. What could be the potential problems?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sfV99.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sfV99.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is a global alignment. Which is why im surprised that they aren't all the same length after alignment?&#xA;Also, while analysing the subsequent genomes, I noticed that some region which were present in only 2/4 genomes were misaligned in the end result. How do I proceed troubleshooting this problem?&lt;/p&gt;&#xA;&lt;p&gt;I am a student and new to this field, could you please tell me how to proceed with manually checking to see whats wrong?&lt;/p&gt;&#xA;" OwnerUserId="8767" LastEditorUserId="73" LastEditDate="2021-05-10T06:08:12.303" LastActivityDate="2023-10-14T14:06:40.713" Title="What does it mean if aligned genomes are not of the same length?" Tags="|sequence-alignment|genome|mauve|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13157" PostTypeId="1" AcceptedAnswerId="13159" CreationDate="2020-05-01T14:24:29.507" Score="1" ViewCount="49" Body="&lt;p&gt;I have made some architectures of neural networks in order to classify proteins in three categories. I have calculated true positive, true negative, false negative, false positive, Mathews, sensitivity, specificity, accuracy for each category. My question is what is the right way to evaluate the architecture. Except for the c-index, could I calculate a cumulative sensitivity, accuracy etc?or should I evaluate separately for each category? Thank you!&lt;/p&gt;&#xA;" OwnerUserId="8713" LastActivityDate="2020-05-01T20:18:35.213" Title="how to evaluate neural networks" Tags="|proteins|networks|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13158" PostTypeId="1" CreationDate="2020-05-01T15:03:59.117" Score="4" ViewCount="783" Body="&lt;p&gt;I have metabarcoding sequence data (COI) from bulk animal samples (including arthropoda, nematoda, annelida, mollusca) and I want to BLAST all of these sequences. I used following command to do this: &lt;code&gt;blastn -remote -db nt -query COI_all.fasta -num_alignments 2 -out COI_blasted.txt&lt;/code&gt;. However this results in errors similar to this post: &lt;a href=&quot;https://www.biostars.org/p/359971/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/359971/&lt;/a&gt; . &lt;/p&gt;&#xA;&#xA;&lt;p&gt;These errors probably appear due to the number of sequences in my file (around 700) and &#xA;the remote connection is thus interrupted. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found that a solution would be to use &lt;code&gt;blastn&lt;/code&gt; with a local database and since the samples are so diverse, I would like to download ALL animal COI sequences from BOLD (or gen bank). It would not be a problem if non-animal (e.g. plant) sequences would also be included.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think the BOLD database would be great to BLAST my sequences to. However, I'm currently struggling to find a good way to download all animal &lt;code&gt;COI&lt;/code&gt; sequences from BOLD.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When entering &lt;code&gt;COI-5P&lt;/code&gt; as search term on &lt;a href=&quot;http://v4.boldsystems.org/index.php/Public_SearchTerms&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://v4.boldsystems.org/index.php/Public_SearchTerms&lt;/a&gt; I receive error: &lt;code&gt;Your search terms resulted in too many matching terms. Please try again with more specific search criteria.&lt;/code&gt;. I could likely download the sequences from all the &lt;code&gt;phyla&lt;/code&gt; etc seperately and merge them, but I'd rather just download 1 file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also tried to use the API by running: &lt;code&gt;wget http://v4.boldsystems.org/index.php/API_Public/sequence?marker=COI-5P&lt;/code&gt;. A download starts but around 3.7 MB download, it is stuck and the file I receive only contains ~5000 sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE: I've contacted BOLD about the stalling behavior and this is their reply: &quot;This issue is because of the large API request that retrieves millions of records, which our system does not handle. Please break up the search by smaller groups, such as classes.&quot;&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Does anyone have a solution to download all COI sequences from BOLD in one file?&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could also download COI sequences from gen bank using the &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/blast/db/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ncbi.nlm.nih.gov/blast/db/&lt;/a&gt; URL, but I'm not sure which exact files I need. For &lt;code&gt;16S, 18S,..&lt;/code&gt; it is obvious, but not for COI. Any suggestions?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for the help.&lt;/p&gt;&#xA;" OwnerUserId="7476" LastEditorUserId="7476" LastEditDate="2020-05-06T07:37:33.673" LastActivityDate="2020-05-06T07:37:33.673" Title="Downloading all COI sequences from BOLD fails" Tags="|public-databases|data-download|blastn|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13159" PostTypeId="2" ParentId="13157" CreationDate="2020-05-01T16:17:12.603" Score="0" Body="&lt;p&gt;Wow, I would just go for accuarcy as the singular most important measure, but be careful of sampling bias. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the classification training had an unbiased sample* and you have an accuracy of &gt;90%, in general ANN that wouldn't be great, but in bioinformatcs thats really good. If your training comprises sampling bias that gets tricky and you either need to accomodate that in your accuarcy ROC etc ... or else retrain to eliminate the sample bias.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can account for all of them of course by wrapping everything into a single analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;*, by this I mean that each of the 3 classifications comprises 33% in each training set. If this is badly skewed, this requires consideration and gets complicated. If it is precisely 1/3 each then 'accurary' is the key stat.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-05-01T18:44:28.037" LastActivityDate="2020-05-01T18:44:28.037" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="13160" PostTypeId="1" AcceptedAnswerId="16074" CreationDate="2020-05-01T16:58:41.423" Score="1" ViewCount="614" Body="&lt;p&gt;I am filtering a VEP annotated vcf, trying to maintain just those variants classified as deleterious by &lt;code&gt;SIFT&lt;/code&gt;and as damaging (probably or possibly included) by &lt;code&gt;PolyPhen&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;filter_vep -i &quot;$input&quot; -o &quot;$output&quot; -filter &quot;SIFT = deleterious and PolyPhen match damaging&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I also tried:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;filter_vep -i &quot;$input&quot; -o &quot;$output&quot; -filter &quot;SIFT match deleterious and PolyPhen match damaging&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The output contains just variants with some value in these two fields, but some of the variants are classified as tolerated by &lt;code&gt;SIFT&lt;/code&gt; and as benign by &lt;code&gt;PolyPhen&lt;/code&gt;, so do you know why the filter is not working?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example: this two variants shouldn't exist in my filtered VCF, because the first one is classified as benign by PolyPhen and the second is classified as benign by both fields:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    CHROM  POS  REF  ALT  FILTER  INFO&#xA;    1       1934441 .       G       T       .       PASS    deleterious(0.01)|benign(0.138)&#xA;    1       207085100       .       G       A       .       PASS   tolerated(0.16)|benign(0.037)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;As an example of a variant that has pass properly those filters:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    CHROM  POS  REF  ALT  FILTER  INFO&#xA;    2       3938449 .       T       A       .       PASS    deleterious(0.01)|probably damaging(0.67)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Those are 3 variants among those that I have in my filtered VCF, but I should be getting just those variants which are classified as deleterious and damaging by both SIFT and PolyPhen, as the third variant I've shown.&lt;/p&gt;&#xA;" OwnerUserId="8631" LastEditorUserId="8631" LastEditDate="2020-05-12T08:00:00.917" LastActivityDate="2021-06-12T12:24:08.290" Title="Filtering VEP annotation file" Tags="|sequence-annotation|variant-calling|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13161" PostTypeId="2" ParentId="997" CreationDate="2020-05-01T16:59:03.360" Score="2" Body="&lt;p&gt;Virulign does this for virus sequences, the publication is available &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/30295730&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and github address &lt;a href=&quot;https://github.com/rega-cev/virulign&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8772" LastEditorUserId="3848" LastEditDate="2020-05-01T17:25:22.953" LastActivityDate="2020-05-01T17:25:22.953" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13162" PostTypeId="1" CreationDate="2020-05-01T17:52:00.537" Score="2" ViewCount="424" Body="&lt;p&gt;I am very new to bioinformatics (and python in general), but I would like to use python to more efficiently analyse enzymes both in terms of structure and functio, using Jupiter notebook. I would like to ask what is the best program/source code for multiple sequence alignments (amino acids) to identify conserved binding sites etc. I see that &lt;a href=&quot;https://biopython.org/docs/1.74/api/index.html#&quot; rel=&quot;nofollow noreferrer&quot;&gt;biopython&lt;/a&gt; has a few ways of creating alignments, but I have the impression that it is more focused on nucleotide sequences?&lt;/p&gt;&#xA;" OwnerUserId="8744" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2021-02-07T12:06:00.500" Title="MSA (protein) with biopython or something else?" Tags="|sequence-alignment|proteins|python|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13163" PostTypeId="1" CreationDate="2020-05-01T19:06:54.200" Score="1" ViewCount="75" Body="&lt;p&gt;&lt;code&gt;Entrez.efetch&lt;/code&gt; helpfully gives me the feature table of a nuccore entry.&lt;br&gt;&#xA;If I understand correctly, &lt;code&gt;efetch&lt;/code&gt; can only return the feature table in a textual format (e.g., as mentioned &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK25499/table/chapter4.T._valid_values_of__retmode_and/?report=objectonly&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any built-in Python/Biopython function that parses this textual format of a feature table?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a code snippet that uses &lt;code&gt;efetch&lt;/code&gt; and writes the feature table to a file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez&#xA;Entrez.email = &quot;orenmn@gmail.com&quot;&#xA;with Entrez.efetch(db='nuccore', id='U00096.3', rettype='ft') as handle:&#xA;    feature_table = handle.read()&#xA;&#xA;with open('feature_table.txt', 'w') as f:&#xA;    f.write(feature_table)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here are the first 20 lines of &lt;code&gt;feature_table.txt&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Feature gb|U00096.3|&#xA;190 255 gene&#xA;            gene    thrL&#xA;            gene_syn    ECK0001&#xA;            locus_tag   b0001&#xA;            db_xref ASAP:ABE-0000006&#xA;            db_xref ECOCYC:EG11277&#xA;            db_xref EcoGene:EG11277&#xA;190 255 CDS&#xA;            product thr operon leader peptide&#xA;            transl_table    11&#xA;            protein_id  gb|AAC73112.1|&#xA;            transcript_id   gnl|b0001|mrna.b0001&#xA;            db_xref UniProtKB/Swiss-Prot:P0AD86&#xA;337 2799    gene&#xA;            gene    thrA&#xA;            gene_syn    ECK0002&#xA;            gene_syn    Hs&#xA;            gene_syn    thrA1&#xA;            gene_syn    thrA2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4868" LastEditorUserId="4868" LastEditDate="2020-05-02T09:57:39.067" LastActivityDate="2020-05-02T09:57:39.067" Title="Is there a built-in parser for the textual feature table that Entrez.efetch returns?" Tags="|database|python|sequence-annotation|public-databases|phylogenetics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13167" PostTypeId="2" ParentId="7344" CreationDate="2020-05-02T09:43:24.900" Score="1" Body="&lt;p&gt;You can easily do it via Notepad++ or even Excel. Just don't forget to save the output file as a &lt;code&gt;.csv&lt;/code&gt; file (comma-separated value). &#xA;The info inside should look like (my working example):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;id,state&#xA;lar_control141,control&#xA;lar_control142,control&#xA;lar_control143,control&#xA;lar_probiotic142,probiotic&#xA;lar_probiotic143,probiotic&#xA;lar_probiotic144,probiotic&#xA;lar_probiotic145,probiotic&#xA;lar_probiotic146,probiotic&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;That should work&lt;/p&gt;&#xA;" OwnerUserId="6948" LastActivityDate="2020-05-02T09:43:24.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13168" PostTypeId="1" CreationDate="2020-05-02T09:50:51.807" Score="2" ViewCount="269" Body="&lt;p&gt;I am trying to create a ballgown object, and I would like to set a phenodata information.&#xA;For that I do &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;phenodata &amp;lt;- read.csv('l002.csv')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and my l002.csv file contains the following info&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;id,state&#xA;RNA-CCC_8N_S5_L002,control&#xA;RNA-CCC_8T1_S6_L002,tube&#xA;RNA-CCC_12N_S7_L002,control&#xA;RNA-CCC_12T_S8_L002,tube&#xA;RNA-CCC_13N_S105_L002,control&#xA;RNA-CCC_13T_S106_L002,tube&#xA;RNA-CCC_23N_S107_L002,control&#xA;RNA-CCC_23T_S108_L002,tube&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;A ballgown folder contains folders that have exactly the same names, e.g. RNA-CCC_8N_S5_L002, RNA-CCC_8T1_S6_L002, etc...&#xA;Then, I use this piece of code for bg object creation&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bg = ballgown(dataDir = &quot;ballgown&quot;, samplePattern = &quot;RNA-CCC_&quot;, pData = phenodata )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;but every time I run it, I have the same error every time:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in ballgown(dataDir = &quot;ballgown&quot;, samplePattern = &quot;RNA-CCC_&quot;, pData = phenodata) : &#xA;  first column of pData does not match the names of the folders containing the ballgown data.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;What can be a problem since first column of pData matches exactly the names of the folders containing the ballgown data.. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If I get rid of phenodata ballgown object creation works, but I need this phenodata for further manipulations.&lt;/p&gt;&#xA;" OwnerUserId="6948" LastActivityDate="2020-05-02T20:27:58.393" Title="Why this ballgown error (first column of pData does not match the names of the folders) comes out" Tags="|rna-seq|stringtie|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13169" PostTypeId="1" CreationDate="2020-05-02T10:01:01.990" Score="1" ViewCount="153" Body="&lt;p&gt;I wish to efficiently search &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore&quot; rel=&quot;nofollow noreferrer&quot;&gt;nuccore&lt;/a&gt; to retrieve only annotated sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;By &quot;annotated&quot; I mean that the nuccore entry contains annotation information.&lt;br&gt;&#xA;An equivalent definition (if I understand correctly) is that the entry's &lt;code&gt;Feature Table&lt;/code&gt; isn't empty.&lt;br&gt; The table can be downloaded by clicking &lt;code&gt;Send to&lt;/code&gt; in the web page of the nuccore entry.&lt;br&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Z0tKq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Z0tKq.png&quot; alt=&quot;download_feature_table&quot;&gt;&lt;/a&gt;&lt;br&gt;&#xA;Alternatively, the table can be downloaded using Biopython: &lt;code&gt;Entrez.efetch(db='nuccore', id=nuccore_id, rettype='ft')&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This definition suggests one answer to my question: Search nuccore normally, then for each entry download its &lt;code&gt;Feature Table&lt;/code&gt;, and discard each nuccore entry whose table is empty.&lt;br/&gt;&#xA;The problem is that this would be very inefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My best idea thus far is to add &lt;code&gt;gene[Feature key]&lt;/code&gt; to my search term.&lt;br&gt;&#xA;E.g., instead of &lt;code&gt;&quot;escherichia coli&quot;[Primary Organism]&lt;/code&gt;, use &lt;code&gt;(&quot;escherichia coli&quot;[Primary Organism]) AND gene[Feature key]&lt;/code&gt;.&lt;br&gt;&#xA;This seems to work, but also feels like a quite dirty hack.&lt;br&gt;&#xA;Is this a good idea? Or am I missing something crucial here?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Another idea I had was using (Biopython) &lt;code&gt;Entrez.esummary(db='nuccore', id=nuccore_id)&lt;/code&gt;, and hopefully finding there a &lt;code&gt;is_sequence_annotated&lt;/code&gt; flag or something similarly wonderful. One of the fields &lt;code&gt;Entrez.esummary&lt;/code&gt; returns is &lt;code&gt;Flags&lt;/code&gt;, but I have no idea how to parse it. I asked about it in &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/13166/4868&quot;&gt;another question&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4868" LastActivityDate="2020-05-02T10:01:01.990" Title="How to (efficiently) search nuccore to retrieve only annotated sequences?" Tags="|database|python|sequence-annotation|public-databases|phylogenetics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13170" PostTypeId="1" CreationDate="2020-05-02T10:42:11.417" Score="1" ViewCount="77" Body="&lt;p&gt;I am analyzing BLAST results from total RNA NGS, pair ends. This BLAST results was performed against the ssRNA virus database download from NCBI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I filtered those read data whose paired ends uniquely matched a single virus and I have both (removing single reads).&#xA;Moreover, based on the &lt;code&gt;sstart&lt;/code&gt; and &lt;code&gt;send&lt;/code&gt; values from a unique virus in each library I calculated the difference (ssdif) of the lowest (sstart) of the highest (send) value that I got in the BLAST results, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;for example:&lt;/strong&gt; If I got 4 reads that match against one virus in one library (2 reads with pair ends) I got the lowest &lt;code&gt;sstart&lt;/code&gt; and the highest &lt;code&gt;send&lt;/code&gt; for these four hits.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, I calculated the coverage using the ssdif; &lt;code&gt;coverage=(ssdif/genome length for that virus) * 100&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After I calculate the coverage for each virus in each library based of the BLAST results I calculated the mean coverage for each genus and for each family of all viruses detected in my analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Does it make sense the way that I calculated the coverage? I would like to filter my data for those viruses that have similar or higher coverage to the genus or family (previously calculated)&#xA;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Based on what taxonomic level does it make more sense filter them?&lt;/li&gt;&#xA;&lt;/ol&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="7430" LastEditorUserId="3848" LastEditDate="2020-12-01T20:26:07.577" LastActivityDate="2020-12-01T20:26:07.577" Title="Analyzing a blast result of NGS data" Tags="|blast|covid-19|coverage|sars-cov-2|taxonomy|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13172" PostTypeId="2" ParentId="13168" CreationDate="2020-05-02T20:27:58.393" Score="2" Body="&lt;p&gt;Well, it looks like if you're using Windows for managing files - that's not a good idea - it sorts names of your ballgown folders in alphabetical order while you need it differently. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;That is why the one should use Linux - when you're &lt;code&gt;ls -ltr&lt;/code&gt; your ballgown folder in Linux it gives you the &quot;right&quot; sequence of folder names to be put inside a ballgown object using pData. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This is what helped me to get rid of this error:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In Linux enter your ballgown folder and do &lt;code&gt;ls -ltr | awk '{print$9}'&lt;/code&gt; and you will have the &lt;strong&gt;right&lt;/strong&gt; sequence that you have to put inside you &lt;code&gt;.csv&lt;/code&gt; file using Notepad++, indicating what's what. Below's my example compared to the above mentioned initial example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;id,state&#xA;RNA-CCC_12N_S7_L002,control&#xA;RNA-CCC_12T_S8_L002,tube&#xA;RNA-CCC_13N_S105_L002,control&#xA;RNA-CCC_13T_S106_L002,tube&#xA;RNA-CCC_23N_S107_L002,control&#xA;RNA-CCC_23T_S108_L002,tube&#xA;RNA-CCC_8N_S5_L002,control&#xA;RNA-CCC_8T1_S6_L002,tube&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't have my error anymore and after &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;phenodata &amp;lt;- read.csv('my_phenodata_file.csv')&#xA;bg = ballgown(dataDir = &quot;ballgown&quot;, samplePattern = &quot;RNA-CCC_&quot;, pData = phenodata) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;my results are nicely wrapped&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Sat May 02 22:11:57 2020&#xA;Sat May 02 22:11:57 2020: Reading linking tables&#xA;Sat May 02 22:11:59 2020: Reading intron data files&#xA;Sat May 02 22:12:04 2020: Merging intron data&#xA;Sat May 02 22:12:06 2020: Reading exon data files&#xA;Sat May 02 22:12:14 2020: Merging exon data&#xA;Sat May 02 22:12:16 2020: Reading transcript data files&#xA;Sat May 02 22:12:20 2020: Merging transcript data&#xA;Wrapping up the results&#xA;Sat May 02 22:12:22 2020&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The problem is explained here as well&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/228389/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/228389/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Hope this will help anyone&lt;/p&gt;&#xA;" OwnerUserId="6948" LastActivityDate="2020-05-02T20:27:58.393" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13173" PostTypeId="1" CreationDate="2020-05-03T03:40:56.053" Score="1" ViewCount="209" Body="&lt;p&gt;I have several matrix from computematrix of deeptools. I need to merge two of them using &quot;computeMatrixOperations cbind -m input1.mat.gz input2.mat.gz -o output.mat.gz&quot;&#xA;but I am running to error &quot;/computeMatrixOperations.py&quot;, line 521, in cbindMatrices&#xA;    hm.matrix.matrix = np.hstack((hm.matrix.matrix, np.empty(hm2.matrix.matrix.shape)))&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any other way to merge the matrixes? &lt;/p&gt;&#xA;" OwnerUserId="8543" LastEditorUserId="77" LastEditDate="2021-01-29T09:13:12.943" LastActivityDate="2021-01-29T09:13:12.943" Title="merge the matrix from computematrix of deeptools" Tags="|sequence-alignment|ngs|chip-seq|deeptools|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13174" PostTypeId="1" AcceptedAnswerId="13178" CreationDate="2020-05-03T08:46:19.650" Score="2" ViewCount="182" Body="&lt;p&gt;I have a folder named quant. Inside this folder I have 27 folders named SRR8068516_quant to SRR8068543_quant. I want to edit a file inside these folders titled quant.sf and produce an output. I attempted writing a shell script to do so:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;    #!/bin/bash&#xA;for fn in /home/usr/Downloads/salmon-1.2.1_linux_x86_64/quants/SRR/quant.sf{8068516..8068543};&#xA;do&#xA;samp=`basename ${fn}`&#xA;echo &quot;Processing sample &lt;span class=&quot;math-container&quot;&gt;${samp}&quot;&#xA;awk 'BEGIN { OFS=FS=&quot;\t&quot; } { sub(&quot;\\..*&quot;, &quot;&quot;, $&lt;/span&gt;1); print }' quant.sf &amp;gt; equant.sf&#xA;done &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I run the script it gives me an error saying : &lt;strong&gt;Processing sample quant.sf8068516&#xA;awk: cannot open quant.sf (No such file or directory).&lt;/strong&gt;&#xA;What is wrong that I am doing.&#xA;Kindly help.&lt;/p&gt;&#xA;" OwnerUserId="7236" LastActivityDate="2020-05-03T15:53:41.187" Title="awk shell script" Tags="|shell|awk|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13176" PostTypeId="1" AcceptedAnswerId="13196" CreationDate="2020-05-03T11:40:19.957" Score="4" ViewCount="285" Body="&lt;p&gt;I would like to perform a &lt;code&gt;samtools mpileup&lt;/code&gt; from a single file that contains thousands of read groups with different SM tags. I could split the bam by read group using &lt;code&gt;samtools split&lt;/code&gt;, and then perform the mpileup, but splitting into thousands of files and then performing an mpileup from thousands of files is crazy slow on my machine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I could imagine an implementation using pysam, but I was hoping someone has a solution already.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bcftools mpileup already has this feature, but the feature was never implemented for the original mpileup format, which is what I need.&lt;/p&gt;&#xA;&#xA;&lt;h3&gt;An example&lt;/h3&gt;&#xA;&#xA;&lt;p&gt;Contents of &lt;code&gt;input.sam&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@HD VN:1.6  SO:unknown&#xA;@SQ SN:chr1 LN:1000000&#xA;@RG ID:0    SM:sample_0&#xA;@RG ID:1    SM:sample_1&#xA;r0  0   chr1    24  0   1M  *   0   0   G   I   RG:Z:0&#xA;r1  0   chr1    24  0   1M  *   0   0   G   I   RG:Z:1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I run samtools mpileup I get this behavior:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; samtools mpileup input.sam&#xA;[mpileup] 2 samples in 1 input files&#xA;chr1    24  N   2   ^!G$^!G$    II&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Whereas, what I actually want is this output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;[mpileup] 2 samples in 1 input files&#xA;chr1    24  N   1   ^!G$    I   1   ^!G$    I&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For a long time I thought this was a bug in samtools mpileup, but it appears that it is a feature that was described/hinted at in the samtools documentation, but was never implemented: &lt;a href=&quot;https://github.com/samtools/samtools/issues/599#issuecomment-604941884&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/samtools/samtools/issues/599#issuecomment-604941884&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="492" LastEditDate="2020-05-04T08:00:10.540" LastActivityDate="2021-11-13T13:10:26.187" Title="Is there a tool that can perform a read-group-aware mpileup from a single file?" Tags="|sam|samtools|pysam|mpileup|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13177" PostTypeId="1" AcceptedAnswerId="13193" CreationDate="2020-05-03T14:14:00.543" Score="7" ViewCount="2420" Body="&lt;p&gt;My goal is to process two samples, &lt;code&gt;S1&lt;/code&gt; and &lt;code&gt;S2&lt;/code&gt;, using two rules, &lt;code&gt;step1&lt;/code&gt;, &lt;code&gt;step2&lt;/code&gt;.&#xA;The Snakemake file is like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SAMPLES = ['S1', 'S2']&#xA;&#xA;rule all:&#xA;  input: expand(&quot;{sample}.done&quot;, sample = SAMPLES)&#xA;&#xA;rule step1:&#xA;  input: &quot;{sample}&quot;&#xA;  output: &quot;{sample}.step1.done&quot;&#xA;  shell: &quot;touch {output}&quot;&#xA;&#xA;rule step2:&#xA;  input: &quot;{sample}.step1.done&quot;&#xA;  output: &quot;{sample}.done&quot;&#xA;  shell: &quot;touch {output}&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The current execution order is: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;step1 on S1&#xA;step1 on S2&#xA;step2 on S1&#xA;step2 on S2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there a way to change the order to:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;step1 on S1&#xA;step2 on S1&#xA;step1 on S2&#xA;step2 on S2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically, I want to complete &lt;code&gt;S1&lt;/code&gt;, then &lt;code&gt;S2&lt;/code&gt;. &#xA;Is there a way to specify the order of executions in Snakemake?&lt;/p&gt;&#xA;" OwnerUserId="8660" LastEditorUserId="8660" LastEditDate="2020-05-03T14:20:40.827" LastActivityDate="2022-10-14T22:48:52.220" Title="How to modify execution orders for Snakemake?" Tags="|snakemake|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13178" PostTypeId="2" ParentId="13174" CreationDate="2020-05-03T14:50:44.673" Score="2" Body="&lt;p&gt;You are running the command on the same file, &lt;code&gt;quant.sf&lt;/code&gt; each time. You need to use your &lt;code&gt;$fn&lt;/code&gt; variable instead:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;#!/bin/bash&#xA;for fn in /home/usr/Downloads/salmon-1.2.1_linux_x86_64/quants/SRR/quant.sf{8068516..8068543};&#xA;do&#xA;    samp=`basename ${fn}`&#xA;    echo &quot;Processing sample ${samp}&quot;&#xA;    awk 'BEGIN { OFS=FS=&quot;\t&quot; } { sub(&quot;\\..*&quot;, &quot;&quot;, $1); print }' &quot;$fn&quot; &amp;gt; equant.sf&#xA;done &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, that would also fail since it will be simply overwriting a file called &lt;code&gt;equant.sf&lt;/code&gt; in your current directory with the results of processing each file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I can't be sure, since you haven't actually shown us your file structure, but I suspect what you're looking for is actually this:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;#!/bin/bash&#xA;for fn in /home/usr/Downloads/salmon-1.2.1_linux_x86_64/quants/SRR/quant.sf{8068516..8068543};&#xA;do&#xA;    samp=`basename ${fn}`&#xA;    outName=&quot;${fn/quant.sf/equant.sf}&quot;&#xA;    echo &quot;Processing sample ${samp}&quot;&#xA;    awk 'BEGIN { OFS=FS=&quot;\t&quot; } { sub(&quot;\\..*&quot;, &quot;&quot;, $1); print }' &quot;$fn&quot; &amp;gt; &quot;$outName&quot;&#xA;done &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;outName=&quot;${fn/quant.sf/equant.sf}&quot;&lt;/code&gt; replaces the first occurrence of &lt;code&gt;quant.sf&lt;/code&gt; in the variable &lt;code&gt;$fn&lt;/code&gt; with &lt;code&gt;equant.sf&lt;/code&gt;, keeping the path and thereby ensuring that the output of each file is stored in the same directory with the name &lt;code&gt;equant.sfNNNNNN&lt;/code&gt;. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ fn=/home/usr/Downloads/salmon-1.2.1_linux_x86_64/quants/SRR/quant.sf8068516&#xA;$&lt;/span&gt; echo ${fn/quant.sf/equant.sf}&#xA;/home/usr/Downloads/salmon-1.2.1_linux_x86_64/quants/SRR/equant.sf8068516&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This means that your awk command will become:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN { OFS=FS=&quot;\t&quot; } { sub(&quot;\\..*&quot;, &quot;&quot;, $1); print }' /home/usr/Downloads/salmon-1.2.1_linux_x86_64/quants/SRR/quant.sf8068516 &amp;gt; /home/usr/Downloads/salmon-1.2.1_linux_x86_64/quants/SRR/equant.sf8068516&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2020-05-03T15:53:41.187" LastActivityDate="2020-05-03T15:53:41.187" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13179" PostTypeId="1" CreationDate="2020-05-03T19:19:53.160" Score="2" ViewCount="35" Body="&lt;p&gt;I have used HMMER search and I have created a phmm. But I would like to visualize it, not as a logo but as a phmm. I do not have Linux, so I cannot use build hmmer. Is there any web browser that I could use?Thank you!&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://hmmer.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;HMMER&lt;/a&gt; searches databases for protein homologs. This can contribute to making sequence alignments. HMMER uses an advanced statistical approach (profile hidden Markov models).&lt;/p&gt;&#xA;&lt;p&gt;PHMM is a profile hidden Markov models, i.e. an output of HMMER which would be used to represent a protein family.&lt;/p&gt;&#xA;" OwnerUserId="8713" LastEditorUserId="3848" LastEditDate="2022-03-24T14:53:29.197" LastActivityDate="2022-03-24T14:53:29.197" Title="phmm visualization" Tags="|hidden-markov-models|hmmer|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13180" PostTypeId="2" ParentId="13173" CreationDate="2020-05-03T20:20:47.587" Score="1" Body="&lt;p&gt;You're trying to cbind two matrices of different sizes, so computeMatrix is throwing an error since it's unclear which row goes with which. You need to ensure that matrices have exactly the same number of rows and that the Nth row in each match (i.e., they should be bound together with cbind).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-05-03T20:20:47.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13182" PostTypeId="1" AcceptedAnswerId="13183" CreationDate="2020-05-04T08:13:33.233" Score="4" ViewCount="2220" Body="&lt;p&gt;Let's take the example &lt;code&gt;p.Arg452Pro&lt;/code&gt; that I got from an annotated VCF file that contains an &lt;code&gt;HGVSp&lt;/code&gt; column.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What does &lt;code&gt;452&lt;/code&gt; mean in this case ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Theoretically, is it possible to have the same &lt;code&gt;HGVSp&lt;/code&gt; on the same gene but on a different transcript level ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Chromosome      Position        Ensembl_Gene        Ensembl_transcript      HGVSp&#xA;4               4613173         ENSG00052342        ENST00141456            p.Arg452Pro&#xA;4               4613176         ENSG00052342        ENST00141458            p.Arg452Pro&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2021-09-20T09:44:13.073" Title="What does the number mean in an HGVSp annotation?" Tags="|vcf|sequence-annotation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13183" PostTypeId="2" ParentId="13182" CreationDate="2020-05-04T09:36:14.273" Score="4" Body="&lt;p&gt;It's the position in the protein. &lt;code&gt;p.Arg452Pro&lt;/code&gt; means that there is a variant that changes an &lt;strong&gt;Arg&lt;/strong&gt;inine residue at position 452 of the protein into a &lt;strong&gt;Pro&lt;/strong&gt;line. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And yes, it is absolutely possible to have the same HGVS p. notation on the same gene but on different transcripts. More than possible, it is quite common. Your particular example isn't actually a real variant since that position on chromosome 4 is intronic in both hg19 and hg38, but &lt;a href=&quot;https://varso.me/oi&quot; rel=&quot;nofollow noreferrer&quot;&gt;TP53:p.R175L&lt;/a&gt;&lt;sup&gt;1&lt;/sup&gt; is a real example of what you describe:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IBHKo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IBHKo.png&quot; alt=&quot;screenshot of varsome.com page for TP53:p.R175L&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you can see in the image or by visiting &lt;a href=&quot;https://varso.me/oi&quot; rel=&quot;nofollow noreferrer&quot;&gt;the link&lt;/a&gt;, this variant on TP53 has the same HGVS p. notation on 6 transcripts. It also has a different p. notation on two more transcripts that are not shown in the image as well as no p. notation for a few other transcripts in which it doesn't affect the protein product.&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;&lt;sup&gt;1&lt;/sup&gt;&lt;sub&gt;Disclaimer: The link goes to &lt;a href=&quot;https://varsome.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://varsome.com&lt;/a&gt;, a free human variant search engine that is developed by the company I work for. &lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2020-05-04T16:58:55.350" LastActivityDate="2020-05-04T16:58:55.350" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13184" PostTypeId="2" ParentId="12816" CreationDate="2020-05-04T09:37:56.003" Score="0" Body="&lt;p&gt;You can use SOAPdenovo-Fusion to scaffold contigs produced by MEGAHIT as suggested by one of the developers: &lt;a href=&quot;https://github.com/aquaskyline/SOAPdenovo2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/aquaskyline/SOAPdenovo2&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7476" LastActivityDate="2020-05-04T09:37:56.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13185" PostTypeId="2" ParentId="13158" CreationDate="2020-05-04T09:39:05.277" Score="1" Body="&lt;p&gt;A straightforward way of getting the sequences from GenBank is from their website using this URL:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore?term=COI%5BGene%20Name%5D&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore?term=COI[Gene Name]&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There you can click on &quot;Send to&quot; -&gt; select &quot;Gene Features&quot; and &quot;FASTA Nucleotide&quot; -&gt; Click &quot;Create File&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are &gt;3 mio hits, so the download could take a while.&lt;/p&gt;&#xA;" OwnerUserId="8629" LastActivityDate="2020-05-04T09:39:05.277" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="13186" PostTypeId="1" CreationDate="2020-05-04T12:42:52.010" Score="2" ViewCount="90" Body="&lt;p&gt;I know I can run my PyMOL scripts if I specify a valid path from the current working directory and also using absolute paths, but can I have a central script repository, so I don't always need to specify the full path or copy my scripts around from project to project?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Environment info:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Kernel: Linux 5.6.8-1-MANJARO x86_64&#xA;Python: 3.8.2&#xA;Python2: 2.7.18&#xA;PyMOL: 2.3.0 Open-Source&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8786" LastEditorUserId="8786" LastEditDate="2020-05-04T14:09:10.480" LastActivityDate="2020-05-04T14:25:44.920" Title="What is PyMOLs search path for scripts?" Tags="|pymol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13187" PostTypeId="2" ParentId="13158" CreationDate="2020-05-04T12:58:06.717" Score="2" Body="&lt;p&gt;I couldn't help but looking a bit deeper into this. There is actually an &lt;a href=&quot;https://cran.r-project.org/web/packages/bold/&quot; rel=&quot;nofollow noreferrer&quot;&gt;R package that uses the BOLD API&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, they also say that there will be timeouts when downloading large sets of data. Luckily, they suggest a &lt;a href=&quot;https://docs.ropensci.org/bold/#large-data&quot; rel=&quot;nofollow noreferrer&quot;&gt;workaround&lt;/a&gt; (downloading sequences separately for each taxonomic Class).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So you should be able to download the marker sequences using the suggested steps:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# install packages (if necessary)&#xA;install.packages(&quot;bold&quot;)&#xA;install.packages(&quot;taxize&quot;)&#xA;install.packages(&quot;seqinr&quot;)&#xA;# load packages&#xA;library(tidyverse)&#xA;library(bold)    # API interface to BOLD&#xA;library(taxize)  # for NCBI taxonomy lookup&#xA;library(seqinr)  # for FASTA output&#xA;&#xA;# get class-level taxa within &quot;Animalia&quot; from NCBI taxonomy&#xA;taxa &amp;lt;- downstream(&quot;Animalia&quot;, db = &quot;ncbi&quot;, downto = &quot;class&quot;)&#xA;# check if taxa present in BOLD&#xA;checks &amp;lt;- bold_tax_name(taxa&lt;span class=&quot;math-container&quot;&gt;$Animalia$&lt;/span&gt;childtaxa_name)&#xA;taxa_bold &amp;lt;- checks[!is.na(checks&lt;span class=&quot;math-container&quot;&gt;$taxon),]$&lt;/span&gt;taxon&#xA;&#xA;# download sequences from BOLD for each class-level taxon&#xA;sequences &amp;lt;- map(taxa_bold[1:3], bold_seq, marker = 'COI-5P') %&amp;gt;%&#xA;  flatten() %&amp;gt;%&#xA;  bind_rows()&#xA;&#xA;# write sequences to file&#xA;write.fasta(&#xA;  sequences = as.list(sequences&lt;span class=&quot;math-container&quot;&gt;$sequence), &#xA;  names = as.list(sequences$&lt;/span&gt;id), &#xA;  nbchar = 80, &#xA;  file.out = 'coi5p.fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note that I selected the first tree taxa (&lt;code&gt;taxa_bold[1:3]&lt;/code&gt;) for testing, so that it runs quicker. If you want to get all taxa, just remove the &lt;code&gt;[1:3]&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One last importand caveat: In the &lt;a href=&quot;https://cran.r-project.org/web/packages/bold/bold.pdf#Rfn.bold.Rul.seq&quot; rel=&quot;nofollow noreferrer&quot;&gt;help for &lt;code&gt;bold_seq&lt;/code&gt;&lt;/a&gt; it says that&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Notes from BOLD on the &lt;code&gt;marker&lt;/code&gt; param:  &quot;All markers for a specimen matching the search stringwill be returned. ie. A record with COI-5P and ITS will return sequence data for both markers evenif only COI-5P was specified.&quot;You will likely end up with data with markers that you did not request - just be sure to filter thoseout as needed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;So you might want to double-check that the downloaded sequences are actually COI-5P. But if you are using them as a BLAST database, you might also be happy with COI-5P &lt;strong&gt;and&lt;/strong&gt; ITS markers being present in the database (it affects the e-values, though).&lt;/p&gt;&#xA;" OwnerUserId="8629" LastActivityDate="2020-05-04T12:58:06.717" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13188" PostTypeId="2" ParentId="13186" CreationDate="2020-05-04T14:25:44.920" Score="1" Body="&lt;p&gt;You can use &lt;a href=&quot;https://pymolwiki.org/index.php/Pymolrc&quot; rel=&quot;nofollow noreferrer&quot;&gt;pymolrc&lt;/a&gt; and create aliases to your scripts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your script is located e.g. in&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/Users/tillyboy/pymolscripts/myscript1.pml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Just enter this line to your pymolrc:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;alias myscript1, run /Users/tillyboy/pymolscripts/script1.pml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Now myscript1 is available for all your Pymol sessions independent of your current working directory.&lt;/p&gt;&#xA;" OwnerUserId="6240" LastActivityDate="2020-05-04T14:25:44.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13189" PostTypeId="2" ParentId="13177" CreationDate="2020-05-04T15:24:01.677" Score="0" Body="&lt;p&gt;The problem that you are encountering, is that when snakemake determines the order to run a set of rules it first builds a &lt;a href=&quot;https://en.wikipedia.org/wiki/Directed_acyclic_graph&quot; rel=&quot;nofollow noreferrer&quot;&gt;Directed Acyclic Graph&lt;/a&gt; (DAG) of all the rules. This shows which rules are are dependant on each other, and what order they can be safely executed in, without breaking downstream dependencies. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, if you want tasks to execute one after another which do not share an explicit file dependency, you can use a &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#flag-files&quot; rel=&quot;nofollow noreferrer&quot;&gt;Flag File&lt;/a&gt; as an ouput for &lt;code&gt;rule 1&lt;/code&gt; then reference it as an input in &lt;code&gt;rule 2&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Unfortunately, I do not think this will help you in your case. You are expanding on the &lt;code&gt;SAMPLES&lt;/code&gt; list and creating a wildcard. in the &lt;code&gt;DAG&lt;/code&gt; this creates two competely separate execution branches which have no dependencies with each other. &lt;code&gt;step1&lt;/code&gt; and &lt;code&gt;step2&lt;/code&gt; are guaranteed to occur in order for each sample, however there is no guarantees between samples. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a specific reason why you need all of &lt;code&gt;S2&lt;/code&gt; tasks to be run after &lt;code&gt;S1&lt;/code&gt;? If so, then your best bet would be to create explicit rules for sample 1 and sample 2, using the &lt;code&gt;Flag File&lt;/code&gt; to incidate when &lt;code&gt;step2&lt;/code&gt; for &lt;code&gt;S1&lt;/code&gt; is done.&lt;/p&gt;&#xA;" OwnerUserId="7253" LastActivityDate="2020-05-04T15:24:01.677" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13191" PostTypeId="1" AcceptedAnswerId="13192" CreationDate="2020-05-04T23:58:11.300" Score="0" ViewCount="54" Body="&lt;p&gt;everyone. &#xA;I am a postdoc who just begins to analyze single-cell RNA-seq data.&#xA;Nowadays, I found a really interesting paper (&lt;a href=&quot;https://doi.org/10.1016/j.celrep.2018.10.047&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1016/j.celrep.2018.10.047&lt;/a&gt;, PMID: 30404002). So, I decided to study bioinformatics by drawing the same graphs presented in the paper during the corona outbreak season. However, it was turned out that I can't get any access to the website in which code is deposited. Here is the address for the website. &#xA;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you guys know how to get the code to learn bioinformatics and make graphs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="8657" LastActivityDate="2020-05-05T00:24:34.877" Title="Is there a way to get the code from &quot;github.mit.edu&quot;?" Tags="|scrnaseq|sequence-analysis|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13192" PostTypeId="2" ParentId="13191" CreationDate="2020-05-05T00:24:34.877" Score="0" Body="&lt;p&gt;It is very strange that they put their code somewhere cannot be easily accessed. If they just put it under a normal Github repo, there would not be such a problem. One way to get the code is simply to email the corresponding author. People are usually responsive if you are interested in their work.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-05-05T00:24:34.877" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13193" PostTypeId="2" ParentId="13177" CreationDate="2020-05-05T07:53:13.990" Score="5" Body="&lt;p&gt;I think you should use the &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#priorities&quot; rel=&quot;noreferrer&quot;&gt;priority&lt;/a&gt; directive to give precedence to downstream rule(s).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here below I give higher priority to &lt;code&gt;step2&lt;/code&gt; so when the first sample has completed &lt;code&gt;step1&lt;/code&gt; snakemake will run step2 on that sample rather than submitting another sample to step1. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SAMPLES = ['S1', 'S2', 'S3']&#xA;&#xA;wildcard_constraints:&#xA;    sample= '|'.join([re.escape(x) for x in SAMPLES])&#xA;&#xA;rule all:&#xA;    input: expand(&quot;{sample}.done&quot;, sample = SAMPLES)&#xA;&#xA;rule step1:&#xA;    priority: 1&#xA;    output: &quot;{sample}.step1.done&quot;&#xA;    shell: &#xA;        r&quot;&quot;&quot;&#xA;        sleep 5&#xA;        touch {output}&#xA;        &quot;&quot;&quot;&#xA;&#xA;rule step2:&#xA;    priority: 10&#xA;    input: &quot;{sample}.step1.done&quot;&#xA;    output: &quot;{sample}.done&quot;&#xA;    shell: &#xA;        r&quot;&quot;&quot;&#xA;        sleep 5&#xA;        touch {output}&#xA;        &quot;&quot;&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(Note that without &lt;code&gt;wildcard_constraints&lt;/code&gt; your/this code throws &lt;code&gt;AmbiguousRuleException&lt;/code&gt;)&lt;/p&gt;&#xA;" OwnerUserId="339" LastActivityDate="2020-05-05T07:53:13.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13194" PostTypeId="1" AcceptedAnswerId="14466" CreationDate="2020-05-05T11:10:15.880" Score="3" ViewCount="150" Body="&lt;p&gt;I cannot seem to find a way to query nucleotide database (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore&lt;/a&gt;) by a specific qualifier of the feature. For example, by /host under the feature &quot;source&quot;.&#xA;For example, below is a part of one record, where the qualifier /host contains &quot;Bemisia tabaci B&quot;.&#xA;How could I search for all sequences that contain the word &quot;Bemisia&quot; in the qualifier \host?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;FEATURES             Location/Qualifiers&#xA; source          1..351658&#xA;                 /organism=&quot;Candidatus Portiera aleyrodidarum BT-B-HRs&quot;&#xA;                 /mol_type=&quot;genomic DNA&quot;&#xA;                 /strain=&quot;BT-B&quot;&#xA;                 /host=&quot;Bemisia tabaci B&quot;&#xA;                 /db_xref=&quot;taxon:1206109&quot;&#xA; gene            1..1926&#xA;                 /gene=&quot;dnaK&quot;&#xA;                 /locus_tag=&quot;C530_RS00005&quot;&#xA;                 /old_locus_tag=&quot;C530_001&quot;&#xA; CDS             1..1926&#xA;                 /gene=&quot;dnaK&quot;&#xA;                 /locus_tag=&quot;C530_RS00005&quot;&#xA;                 /old_locus_tag=&quot;C530_001&quot;&#xA;                 /inference=&quot;COORDINATES: similar to AA&#xA;                 sequence:RefSeq:WP_014894920.1&quot;&#xA;                 /note=&quot;Derived by automated computational analysis using&#xA;                 gene prediction method: Protein Homology.&quot;&#xA;                 /codon_start=1&#xA;                 /transl_table=11&#xA;                 /product=&quot;molecular chaperone DnaK&quot;&#xA;                 /protein_id=&quot;WP_014894920.1&quot;&#xA;                 /translation=&quot;MIRIIGIDLGTTNSCVAVLENGKSRVIENSEGARTTPSIIAYTD&#xA;                 DNEILIGQTAKRQAVTNPSNTLYAIKRLIGRKFNDDIVQKDIKMVPYKIVSAENGDAW&#xA;                 VQVKNRKLAPPQISAEILKKMKKTAEDYLGEKITEAVITVPAYFNDSQRQATKDAGRI&#xA;                 AGLNVKRIINEPTAAALAYGMDKNRGDKKIAVYDLGGGTFDISIIEIADVDGETQFEV&#xA;                 MATNGDTFLGGEDFDLKLINYIVKQFKLESGVDLSGDSLAMQRLKEAAEKGKIELSSS&#xA;                 QQTDINLPYITADKTGPKHLTLKITRAKLESLVEDLIKRSLLPCKKAIDDAKLTTKEI&#xA;                 DDIILVGGQTRMPLVQKKVAEFFGKEARKDVNPDEAVAMGAAIQGGVLGGDVKDVLLL&#xA;                 DVTPLTLGIETMGGIMTPLIEKNTTIPTKKTQIFSTAEDNQTAVTIHALQGERKKALQ&#xA;                 NKSLGRFDLTDIPPAPRGTPQIEVAFDLDANGILNITAKDKATGKEQSIIIKSSGGLS&#xA;                 EEEINKMIQDAETNKEEDKKFEELVQTRNQADGMIHTIRKTLKEIKNEDEKNKLESLI&#xA;                 NNLEQTLKSDDIKTIKEDLNKLTEEYSTIYQKIYSEKKPQNHNSEPAQKNAEKSSSKK&#xA;                 EGDVVDAEYEEVNNKTK&quot;&#xA;                 ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Best,&#xA;Milan&lt;/p&gt;&#xA;" OwnerUserId="8792" LastActivityDate="2020-10-01T15:48:15.690" Title="How to query NCBI (Nucleotide database) by a feature qualifier?" Tags="|database|phylogenetics|entrez|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13195" PostTypeId="1" CreationDate="2020-05-05T13:10:33.800" Score="1" ViewCount="206" Body="&lt;p&gt;I did a GWAS analysis in the past for antibiotic resistance of E. Coli and the results were interesting (matching the literature). I did a new GWAS analysis for some new samples, but the results are not in agreement with the previous GWAS and the literature. So, I was wondering if the combination of them would be in agreement with the literature, as having more samples to support an argument is always a good idea.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, my question is how I can combine two GWAS study and what quality control steps do I need to follow?&lt;/p&gt;&#xA;" OwnerUserId="7444" LastEditorUserId="7444" LastEditDate="2020-05-05T14:59:25.203" LastActivityDate="2020-05-06T12:13:33.190" Title="How to combine two Genome-wide Association Study (GWAS)?" Tags="|sequence-alignment|ngs|sequence-analysis|genomics|gwas|" AnswerCount="2" CommentCount="1" ClosedDate="2020-05-07T20:38:16.110" ContentLicense="CC BY-SA 4.0" />
  <row Id="13196" PostTypeId="2" ParentId="13176" CreationDate="2020-05-05T13:15:20.237" Score="2" Body="&lt;p&gt;I broke down and finally scratched this itch. I have implemented a tool that performs read-group-aware text-pileup from a single SAM/BAM file. The tool is called &lt;a href=&quot;https://pypi.org/project/streaming-pileupy/&quot; rel=&quot;nofollow noreferrer&quot;&gt;streaming_pileupy&lt;/a&gt; and it's available from pypi for installation with pip. After installation the command would be:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;spileup input.bam sample_names.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The tool is pre-alpha and missing some features. Contributions are welcome.&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="492" LastEditDate="2021-11-13T13:10:26.187" LastActivityDate="2021-11-13T13:10:26.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13197" PostTypeId="2" ParentId="13195" CreationDate="2020-05-05T15:44:41.660" Score="0" Body="&lt;p&gt;The first way is to combine two samples and re-run GWAS. A second way is to do a meta-analysis using GWAS results.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-05-05T15:44:41.660" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13198" PostTypeId="1" AcceptedAnswerId="13227" CreationDate="2020-05-05T16:13:50.457" Score="1" ViewCount="103" Body="&lt;p&gt;The example code provided by the &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/snakefiles/remote_files.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Snakemake documentation&lt;/a&gt; for remote downloading from the EGA archive fails for me. I have defined all required environment variables, but an error comes up stating &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;{&quot;error&quot;:&quot;invalid_client&quot;,&quot;error_description&quot;:&quot;Bad client credentials&quot;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I guess I am not defining the &lt;code&gt;EGA_CLIENT_ID&lt;/code&gt; variable correctly? I am using Snakemake 5.16 and Python 3.7.2. Thanks a lot in advance for your advice. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Code: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import snakemake.remote.EGA as EGA&#xA;&#xA;ega = EGA.RemoteProvider()&#xA;os.environ[&quot;EGA_USERNAME&quot;] = &quot;XXX&quot;&#xA;os.environ[&quot;EGA_PASSWORD&quot;] = &quot;XXX&quot;&#xA;os.environ[&quot;EGA_CLIENT_SECRET&quot;] = &quot;XXX&quot;&#xA;os.environ[&quot;EGA_CLIENT_ID&quot;] = &quot;pyega3&quot;&#xA;rule a:&#xA;  input:&#xA;      ega.remote(&quot;ega/EGAD00001002142/COLO_829_EPleasance_TGENPipe.bam.bai&quot;)&#xA;  output:&#xA;      &quot;data/COLO_829BL_BCGSC_IlluminaPipe.bam.bai&quot;&#xA;  shell:&#xA;      &quot;cp {input} {output}&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5046" LastActivityDate="2020-05-08T16:53:44.123" Title="EGA remote download via snakemake fails due to EGA_CLIENT_ID specification" Tags="|snakemake|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13199" PostTypeId="1" CreationDate="2020-05-05T19:59:00.267" Score="4" ViewCount="231" Body="&lt;p&gt;I'm looking at running the PAM50 classifier on RNA-Seq from 138 breast cancer samples. However, the R package (&lt;code&gt;genefu&lt;/code&gt;) that's useful for this does not have a particularly helpful &lt;code&gt;vignette&lt;/code&gt; when it comes to processing the RNAseq counts because it uses pre-processed publicly-available RNAseq data.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Starting from counts, is there a way to pre-process the data for running &lt;code&gt;intrinsic.cluster.predict&lt;/code&gt; in &lt;code&gt;R/genefu&lt;/code&gt; using the built-in signatures? (I probably won't have enough power to split the set into a training/test set).&lt;/li&gt;&#xA;&lt;li&gt;The best choices for the parameters in &lt;code&gt;intrinsic.cluster.predict&lt;/code&gt; would also be very helpful.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;For differential expression (I have pre and post surgery pairs), I have used conditional quantile normalisation to get normalised log2(RPKM) values for running through &lt;code&gt;R/EdgeR&lt;/code&gt;, so I also have these available if that is the best way to go.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;EDIT: looking at the &lt;code&gt;genefu&lt;/code&gt; vignette, this is all based on microarray data (which is essentially log2(expression)+offset), so will adding an offset to the log2(RPKM) values improve prediction for PAM50?&lt;/p&gt;&#xA;" OwnerUserId="2244" LastEditorUserId="3848" LastEditDate="2020-06-26T13:20:40.033" LastActivityDate="2020-06-26T13:20:40.033" Title="PAM50 gene expression classification" Tags="|r|rna-seq|" AnswerCount="0" CommentCount="0" FavoriteCount="0" ClosedDate="2021-06-25T11:06:19.083" ContentLicense="CC BY-SA 4.0" />
  <row Id="13201" PostTypeId="1" CreationDate="2020-05-06T03:13:21.010" Score="4" ViewCount="50" Body="&lt;p&gt;I am passing a custom distance matrix to T_Coffee (written in C), as per the docs, but can't seem to get it to work.&lt;/p&gt;&#xA;&lt;p&gt;The documentation says to pass a matrix like this using BLAST format&#xA;&lt;a href=&quot;https://tcoffee.readthedocs.io/en/latest/tcoffee_technical_documentation.html#blast-format-recommended&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://tcoffee.readthedocs.io/en/latest/tcoffee_technical_documentation.html#blast-format-recommended&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# BLAST_MATRIX FORMAT&#xA;# ALPHABET=AGCT&#xA;A G C T&#xA;A 0 1 2 3&#xA;G 0 2 3 4&#xA;C 1 1 2 3&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've tried all combinations (with and without the leading # comments, and with and without the trailing) ..., but no matter what I keep getting this error:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Parsing_error when reading blast_matrix doolittle: A C 0&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here, my distance matrix file is titled doolittle because it is the difference between the hydropathy indices of amino acids based on the Kyte-Doolittle scale * 10 to make them all integers. I've pasted the full matrix below.&lt;/p&gt;&#xA;&lt;p&gt;I am calling the command from linux 18.08 ubuntu terminal (although I've tried most other combinations.)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;t_coffee -in=rhomboids.fasta -matrix doolittle&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any help would be appreciated. Thank you!&lt;/p&gt;&#xA;&lt;p&gt;doolittle&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# BLAST_MATRIX FORMAT&#xA;# ALPHABET=ACDEFGHIKLMNPQRSTVWY&#xA;A C D E F G H I K L M N P Q R S T V W Y&#xA;A 0 7 53 53 10 22 50 27 57 20 1 53 34 53 58 26 25 24 27 31&#xA;C 7 0 60 60 3 29 57 20 64 13 6 60 41 60 65 33 32 17 34 38&#xA;D 53 60 0 0 63 31 3 80 4 73 54 0 19 0 5 27 28 77 26 22&#xA;E 53 60 0 0 63 31 3 80 4 73 54 0 19 0 5 27 28 77 26 22&#xA;F 10 3 63 63 0 32 60 17 67 10 9 63 44 63 68 36 35 14 37 41&#xA;G 22 29 31 31 32 0 28 49 35 42 23 31 12 31 36 4 3 46 5 9&#xA;H 50 57 3 3 60 28 0 77 7 70 51 3 16 3 8 24 25 74 23 19&#xA;I 27 20 80 80 17 49 77 0 84 7 26 80 61 80 85 53 52 3 54 58&#xA;K 57 64 4 4 67 35 7 84 0 77 58 4 23 4 1 31 32 81 30 26&#xA;L 20 13 73 73 10 42 70 7 77 0 19 73 54 73 78 46 45 4 47 51&#xA;M 1 6 54 54 9 23 51 26 58 19 0 54 35 54 59 27 26 23 28 32&#xA;N 53 60 0 0 63 31 3 80 4 73 54 0 19 0 5 27 28 77 26 22&#xA;P 34 41 19 19 44 12 16 61 23 54 35 19 0 19 24 8 9 58 7 3&#xA;Q 53 60 0 0 63 31 3 80 4 73 54 0 19 0 5 27 28 77 26 22&#xA;R 58 65 5 5 68 36 8 85 1 78 59 5 24 5 0 32 33 82 31 27&#xA;S 26 33 27 27 36 4 24 53 31 46 27 27 8 27 32 0 1 50 1 5&#xA;T 25 32 28 28 35 3 25 52 32 45 26 28 9 28 33 1 0 49 2 6&#xA;V 24 17 77 77 14 46 74 3 81 4 23 77 58 77 82 50 49 0 51 55&#xA;W 27 34 26 26 37 5 23 54 30 47 28 26 7 26 31 1 2 51 0 4&#xA;Y 31 38 22 22 41 9 19 58 26 51 32 22 3 22 27 5 6 55 4 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8795" LastEditorUserId="3848" LastEditDate="2022-03-01T20:05:46.840" LastActivityDate="2022-03-01T20:05:46.840" Title="Passing custom distance matrix to T_Coffee" Tags="|sequence-alignment|covid-19|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13202" PostTypeId="1" CreationDate="2020-05-06T11:56:09.367" Score="2" ViewCount="98" Body="&lt;p&gt;I have 5000 protein sequences in one multifasta file. I found more reads have gaps as X in their reads. So, want to eliminate those reads completely (Whole protein seq) from the file. I am keeping filter criteria as if a read contains morethan 2 X ( continuesly or anywhere in the read) should be removed. Thanks in advance for your help. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The input sequence looks like this &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Prot1 &#xA;ANSTVKKKKLLLYYYSSSEERXFGHYFGHYFGHFYVHFGFYVHCEDYHF&#xA;&amp;gt;Prot2&#xA;ANSTVKKKKLLLYYYSSSEERXXXXXXXXXXXFGHYFGHYFGHFYVHFGFYVHCEDYHF&#xA;&amp;gt;Prot3&#xA;ANSTVKKKKLLLYYYSSSEERFGHYFGHYFGHFYVHFGFYVHCEDYHF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want output Like this &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Prot3&#xA;ANSTVKKKKLLLYYYSSSEERFGHYFGHYFGHFYVHFGFYVHCEDYHF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8798" LastEditorUserId="3848" LastEditDate="2020-05-06T12:08:36.003" LastActivityDate="2020-05-08T12:03:30.973" Title="How to remove sequence reads contains more than 2 X from multifasta file?" Tags="|sequence-alignment|ngs|parsing|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13203" PostTypeId="2" ParentId="13195" CreationDate="2020-05-06T12:13:33.190" Score="0" Body="&lt;p&gt;I'd recommend first having a think about how you would go about combining two GWAS with the &lt;em&gt;same&lt;/em&gt; sample set (e.g. split the samples randomly in two). Even with the same source population, you'll get different associations coming up by sampling a different sub-set of that population.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once you can do that multiple times, you can attach a level of confidence to each SNP association within each study, which can be treated as a prior probability for the purpose of combining multiple independent studies.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It sounds like you're in a great position to be able to do this, because you've done the analysis on both groups and [presumably] have all the genotype-level data from both studies.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2020-05-06T12:13:33.190" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13204" PostTypeId="2" ParentId="13202" CreationDate="2020-05-06T13:38:07.007" Score="1" Body="&lt;p&gt;A Biopytnon solution that works for &lt;code&gt;.gz&lt;/code&gt; files too :-). The script make a script &lt;code&gt;filter_faa.py&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;import sys&#xA;from Bio import SeqIO&#xA;from mimetypes import guess_type&#xA;import gzip&#xA;&#xA;filename = sys.argv[1]&#xA;&#xA;encoding = guess_type(filename)[1]&#xA;_open = partial(gzip.open, mode='rt') if encoding == 'gzip' else open&#xA;&#xA;with _open(filename) as ffile:&#xA;    for seq_record in SeqIO.parse(ffile, 'fasta'):&#xA;        if seq_record.seq.count('X') &amp;lt; 3:&#xA;            print(seq_record.format(&quot;fasta&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then usage is&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python3 filter_faa.py my.faa &amp;gt; my_filtered.faa &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;my_filtered.faa&lt;/code&gt; then will contain&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Prot1&#xA;ANSTVKKKKLLLYYYSSSEERXFGHYFGHYFGHFYVHFGFYVHCEDYHF&#xA;&#xA;&amp;gt;Prot3&#xA;ANSTVKKKKLLLYYYSSSEERFGHYFGHYFGHFYVHFGFYVHCEDYHF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The filtering condition is hard-coded. You could potentially parametrise it if you feel like later you might want to filter by a different number of unknown aa.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2020-05-07T12:26:49.433" LastActivityDate="2020-05-07T12:26:49.433" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13205" PostTypeId="2" ParentId="12918" CreationDate="2020-05-06T19:00:40.243" Score="-1" Body="&lt;p&gt;You might want to check out the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/GEOquery.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioConductor &lt;em&gt;&lt;code&gt;GEOquery&lt;/code&gt;&lt;/em&gt; package&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Install using &lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;if (!requireNamespace(&quot;BiocManager&quot;, quietly = TRUE))&#xA;    install.packages(&quot;BiocManager&quot;)&#xA;&#xA;BiocManager::install(&quot;GEOquery&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;More information on how to use this package and read in data from GEO can be found in the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/GEOquery/inst/doc/GEOquery.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;vignette&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8777" LastActivityDate="2020-05-06T19:00:40.243" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13206" PostTypeId="1" AcceptedAnswerId="13217" CreationDate="2020-05-06T19:01:50.513" Score="2" ViewCount="44" Body="&lt;p&gt;Hello to the experts in analyzing chromosome structure data in HiC format,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a very basic question. I have a HiC file (specifics are mentioned below), using which I would like to obtain contact probability between a pair of genomic loci.&#xA;I obtained contact probabilities between the two loci at different resolutions available in the HiC file. However, rather than having a vector of contact probabilities, I would like to have a single number that appropriately represents the contact probability between the loci.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My guess is that there could be two ways to get a single number as the contact probability between two given genomic loci. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Should I take the highest contact probability?&#xA;or,&lt;/li&gt;&#xA;&lt;li&gt;Depending on the lengths of the genomics loci of interest, may be the resolution that is close to that could be selected as a single number that represents the contact probability between the loci. Eg. with this strategy, if genomic loci of interest are ~5kb in length the contact probability at 5kb resolution is selected as a single number. Then, the issue is that the length of the genomic loci of interest could be lower than the lowest length of resolution available in the HiC file.&lt;br&gt;&#xA;or,  &lt;/li&gt;&#xA;&lt;li&gt;Contact probabilities at different resolutions could be aggregated. Then I wonder what would be the appropriate way to aggregate contact probabilities, e.g. by multiplication or median or something else.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Just to reiterate, above these are my guess and I am not sure whats the best way.&#xA;So basically my question is: &lt;strong&gt;how can I get a single number that would represent the contact probability between a pair of genomic loci using HiC data&lt;/strong&gt;?&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Some specifics that might be useful in answering my question.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;source of the HiC file: &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/geo/series/GSE63nnn/GSE63525/suppl/GSE63525_K562_combined_30.hic&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nlm.nih.gov/geo/series/GSE63nnn/GSE63525/suppl/GSE63525_K562_combined_30.hic&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;organism: human (K562 cell line)&lt;/li&gt;&#xA;&lt;li&gt;Resolutions in HiC data (bp): [2500000, 1000000, 500000, 250000, 100000, 50000, 25000, 10000, 5000]&lt;/li&gt;&#xA;&lt;li&gt;Type of genomic loci: upstream non-coding regions of the genes&lt;/li&gt;&#xA;&lt;li&gt;lengths of genomic loci of interest: average ~1000bp (range: 500-5000bp).&lt;/li&gt;&#xA;&lt;li&gt;software I am using to &quot;extract&quot; data from HiC file: &lt;a href=&quot;https://github.com/aidenlab/straw&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/aidenlab/straw&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="274" LastEditorUserId="274" LastEditDate="2020-05-07T14:28:36.433" LastActivityDate="2022-12-21T14:20:17.077" Title="Get a single number representing the contact probability between a pair of genomic loci using HiC data" Tags="|genome|hi-c|chromosome-capture|chromosomes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13207" PostTypeId="2" ParentId="5536" CreationDate="2020-05-07T08:21:18.323" Score="0" Body="&lt;p&gt;you may use the OD-seq algorithm to filter outliers in a multiple alignment.&#xA;&lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-015-0702-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-015-0702-1&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8749" LastActivityDate="2020-05-07T08:21:18.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13208" PostTypeId="2" ParentId="12918" CreationDate="2020-05-07T12:44:55.373" Score="-1" Body="&lt;p&gt;Install package&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;if (!requireNamespace(&quot;BiocManager&quot;, quietly = TRUE))&#xA;    install.packages(&quot;BiocManager&quot;)&#xA;&#xA;BiocManager::install(&quot;GEOquery&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Use function getGEO to download the files.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;expressionSet &amp;lt;- getGEO(&quot;GSE138651&quot;)&#xA;expressionSet &amp;lt;- expressionSet[[1]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can then look at the info of the samples:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pData(expressionSet)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the assay data itself&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;exprs(expressionSet)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7056" LastActivityDate="2020-05-07T12:44:55.373" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13209" PostTypeId="2" ParentId="13202" CreationDate="2020-05-07T12:58:36.777" Score="0" Body="&lt;p&gt;&lt;strong&gt;Updated&lt;/strong&gt;: as pointed out, this will not work for a multifasta as asked for in your original question. However it would work fine for a single line fasta&lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;You could accomplish this with a single line of bash code if you would like (this assumes your fasta is in gzip format, if not just use cat):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;zcat prot.fasta.gz | grep -E -v -B1 '(^&amp;gt;.*|^(.*X.*){3}$)' | grep -E -v '^--$'  - | gzip &amp;gt; prot.filtered.fasta.gzip&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;-v&lt;/code&gt; option of grep inverts the regex, and the &lt;code&gt;-B1&lt;/code&gt; will print out the line occuring before the current context. The &lt;code&gt;-B1&lt;/code&gt; introduces &lt;code&gt;--&lt;/code&gt; between the matches, which is why the second grep is there. Then you can immediately pipe it back into a zipped file &lt;/p&gt;&#xA;" OwnerUserId="7253" LastEditorUserId="7253" LastEditDate="2020-05-08T12:03:30.973" LastActivityDate="2020-05-08T12:03:30.973" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13211" PostTypeId="1" CreationDate="2020-05-07T14:59:12.250" Score="1" ViewCount="41" Body="&lt;p&gt;I have a question about finding differentially covered regions (coverage represents methylation level which goes from 0 to several thousands). I'm using enrichment based method which can be summarized with coverage per gene:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data &amp;lt;- matrix(sample(80), 20)&#xA;# Genes in rows&#xA;rownames(data) &amp;lt;- letters[1:20]&#xA;colnames(data) &amp;lt;- c(&quot;group_A_tr1&quot;, &quot;group_A_tr2&quot;, &quot;group_B_tr1&quot;, &quot;group_B_tr2&quot;)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In &lt;code&gt;data&lt;/code&gt; matrix each row represents a gene and each column represents a sample. There are two sample groups (&lt;em&gt;A&lt;/em&gt; and &lt;em&gt;B&lt;/em&gt;) with two technical replicates per each group. &lt;strong&gt;Problem is that we do not have any biological replicates.&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My goal is to identify genes that are differentially methylated between two groups. I know that &lt;code&gt;limma&lt;/code&gt;, &lt;code&gt;edgeR&lt;/code&gt;, &lt;code&gt;DESeq2&lt;/code&gt; can be used in analysis like this, however I don't have enough samples. Basically I'll need to compare only two columns (after averaging technical replicates).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What method would be appropriate to work with data like this? Is it possible to treat technical replicates as biological ones?&lt;/p&gt;&#xA;" OwnerUserId="186" LastEditorUserId="186" LastEditDate="2020-05-07T15:04:31.147" LastActivityDate="2020-05-12T12:36:19.373" Title="Identify differentially covered genes only between two samples" Tags="|bioconductor|deseq2|limma|edger|methylation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13212" PostTypeId="1" AcceptedAnswerId="13214" CreationDate="2020-05-07T15:20:16.213" Score="0" ViewCount="334" Body="&lt;p&gt;What exactly is the difference between protein-protein interaction (PPI) networks and gene co-expression networks? Based on my rudimentary understanding, PPIs are constructed from gene co-expression networks, and so represent processed data. But could someone elaborate more on the differences?&lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2020-05-07T16:44:46.010" Title="Difference between PPI networks and Gene Co-expression networks" Tags="|gene-expression|protein-protein-interaction|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13213" PostTypeId="2" ParentId="13212" CreationDate="2020-05-07T15:53:00.230" Score="0" Body="&lt;p&gt;Co-expression networks are based on gene expression data. Genes are compared using their expression profiles over multiple samples. A common distance measurement is the Pearson Correlation Coefficient, assuming that co-expressed genes (genes expressed simultaneously in similar conditions) will have a high correlation coefficient.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In its simple form, a co-expression network is done like this. An expression matrix containing n genes x p samples is used in input (from microarrays or RNA-seq) to calculate a distance matrix of size n x n. Then the distance matrix is thresholded to keep only the most significant relationship (eg correlation &gt;0.8). Gene pairs are connected if their correlation coefficient is above this threshold. Then all connections are visualized in a network, which can then be subjected to community detection algorithms to find more densely connected set of genes. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;PPI are usually built on in vitro assays such as Yeast-2-Hybrid. But connections can also be inferred from literature data and expression data. You may have a look at the StringDB for example to see which kind of data can be used to connect proteins.&lt;/p&gt;&#xA;" OwnerUserId="8749" LastEditorUserId="298" LastEditDate="2020-05-07T16:44:46.010" LastActivityDate="2020-05-07T16:44:46.010" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13214" PostTypeId="2" ParentId="13212" CreationDate="2020-05-07T16:44:20.767" Score="1" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;Based on my rudimentary understanding, PPIs are constructed from gene co-expression networks&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Not really, no. PPI stands for protein-protein interaction. In PPI networks, each node is a protein and each edge (line) connecting it to another node represents an interaction. A very common example is that an edge represents a physical interaction between these two proteins. This means that there have been experiments to indicate that these two proteins come into contact in the cell and interact. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, they might form part of a protein complex, so their interaction was seen using chromatin immunoprecipitation, where the two proteins were captured together. Or, they may have been seen together using &lt;a href=&quot;https://en.wikipedia.org/wiki/F%C3%B6rster_resonance_energy_transfer#Biology&quot; rel=&quot;nofollow noreferrer&quot;&gt;FRET&lt;/a&gt;, a technique that lets you detect when two molecules are in very close proximity.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, some interactions can be theoretical, where &lt;em&gt;in silico&lt;/em&gt; modelling has suggested that these two proteins can interact, even though we don't have any direct evidence that they do so. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally, yes, it is also possible to make a PPI network based on gene co-expression data, under the assumption that if two genes are co-expressed, then their protein products will be present in the same tissue. However, that doesn't mean they interact in any way, so it isn't really a very common way of building PPI networks, as far as I know. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In co-expression networks, on the other hand, each node is a gene and not its protein product. And here, the edges indicate the coexpression. Often the level of correlation between the two genes' expression data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The most important difference between these two kinds of network, apart from the obvious mentioned above, is that they are used in different ways. A gene coexpression network lets you identify genes that are regulated in similar ways. It is a tool you can use to investigate the regulatory network of the cell, to see what genes are &quot;turned on&quot; or &quot;off&quot; together. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The PPI network, on the other hand, is a way of understanding the result of this gene co-regulation. You are not looking at how the gene network is regulated any more, but how the products of the genes interact with one another. So it is a very different focus.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can maybe think of this in terms of school children and their parents. You could make a network where each node represents a pair of parents, and then draw lines between them if their children were born in the same year. That would give you a general overview of when this group of people procreated. However, that two children were born on the same day doesn't necessarily imply that the children themselves will ever come into contact. This could be the equivalent of the gene co-expression network. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then, make another network where each node is a child and draw a line between any children who know each other. This would be the equivalent of the PPI network. You would likely find some correlations between the two, with children who know each other being more likely to have been born at around the same time, but the two networks give you very different information.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2020-05-07T16:44:20.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13215" PostTypeId="1" AcceptedAnswerId="13225" CreationDate="2020-05-07T19:27:38.247" Score="1" ViewCount="1657" Body="&lt;p&gt;I downloaded a few FASTQs from the SRA that were paired-end 76 bp.&#xA;When I look at the FASTQs, I get something like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@SRR7702334.1 1 length=152&#xA;AAGATNTTTTTCAGTATCAATCAGANGNNANANNCNNCTGCCCCAAATCCTGCAAAGAGCATGCAACGTCCTTAGAAGACAACGTCTTGCTCTGTCTCACCCAGGCTGGAGCATAGGGGCATGATCGATCATGACAAGGAGTTTACATTTTG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I forgot to add the &lt;code&gt;--split-3&lt;/code&gt; option to my &lt;code&gt;fastq-dump&lt;/code&gt; command.&#xA;Is there an easy and fast way, without re-downloading the files, to split the reads in half and write them to two output FASTQs?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried the following using Biopython, and it'll work, but it's going to take a long time to finish.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from os.path import basename&#xA;from Bio import SeqIO&#xA;from Bio.Seq import Seq&#xA;from Bio.SeqRecord import SeqRecord&#xA;from Bio.Alphabet import IUPAC&#xA;from gzip import open as gzopen&#xA;import argparse&#xA;from tqdm import tqdm&#xA;&#xA;&#xA;def split_read(read, pos=152 // 2):&#xA;    &quot;&quot;&quot;&#xA;    Split read in half to produce 2 reads&#xA;&#xA;    Parameters&#xA;    ----------&#xA;    read_id : Bio.SeqRecord.SeqRecord&#xA;        Read to be split&#xA;    &quot;&quot;&quot;&#xA;    return {&#xA;        # R1/R2 as keys&#xA;        &quot;R&quot; + str(i + 1): SeqRecord(&#xA;            # first and second halves of sequence string&#xA;            seq=Seq(str(read.seq[(pos * i):(pos * (i + 1))]), IUPAC.ambiguous_dna),&#xA;            id=read.id,&#xA;            description=&quot;&quot;,&#xA;            letter_annotations={&quot;phred_quality&quot;: read.letter_annotations[&quot;phred_quality&quot;][(pos * i):(pos * (i + 1))]}&#xA;        ) for i in range(2)&#xA;    }&#xA;&#xA;def split_fastq(fastq, prefix=&quot;split&quot;):&#xA;    &quot;&quot;&quot;&#xA;    Split reads from a FASTQ&#xA;&#xA;    Parameters&#xA;    ----------&#xA;    fastq : str&#xA;        Path to input FASTQ file&#xA;    prefix : str&#xA;        Prefix for output FASTQ files (`_R1.fastq.gz` and `_R2.fastq.gz` are appended)&#xA;    &quot;&quot;&quot;&#xA;    # check for file format&#xA;    if fastq.endswith(&quot;.gz&quot;):&#xA;        fq_handle = gzopen(fastq, &quot;rt&quot;)&#xA;    else:&#xA;        fq_handle = open(fastq, &quot;r&quot;)&#xA;    out_handles = {r: gzopen(prefix + &quot;_&quot; + r + &quot;.fastq.gz&quot;, &quot;wt&quot;) for r in [&quot;R1&quot;, &quot;R2&quot;]}&#xA;    # load reads for random access&#xA;    records = SeqIO.parse(fq_handle, &quot;fastq&quot;)&#xA;    for read in tqdm(records):&#xA;        # split read into each mate and fix header information&#xA;        mates = split_read(read)&#xA;        # write to each output file&#xA;        for m in mates:&#xA;            out_handles[m].write(mates[m].format(&quot;fastq&quot;))&#xA;    fq_handle.close()&#xA;    for m in mates:&#xA;        out_handle[m].close()&#xA;&#xA;if __name__ == &quot;__main__&quot;:&#xA;    # parse command line arguments&#xA;    PARSER = argparse.ArgumentParser(description=&quot;Split a FASTQ into 2 FASTQs by splitting each read in half&quot;)&#xA;    PARSER.add_argument(&#xA;        &quot;fq&quot;,&#xA;        type=str,&#xA;        help=&quot;Input FASTQ to split&quot;&#xA;    )&#xA;    PARSER.add_argument(&#xA;        &quot;-p&quot;,&#xA;        &quot;--prefix&quot;,&#xA;        type=str,&#xA;        help=&quot;Prefix for output FASTQ files (`_R1.fastq.gz` and `_R2.fastq.gz` are appended)&quot;,&#xA;        default=&quot;split&quot;&#xA;    )&#xA;    ARGS = PARSER.parse_args()&#xA;    # run splitting function&#xA;    split_fastq(ARGS.fq, ARGS.prefix)&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="240" LastActivityDate="2021-07-05T04:29:54.200" Title="How to split FASTQ reads without re-running `fastq-dump`?" Tags="|fastq|sratoolkit|fastq-dump|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13216" PostTypeId="2" ParentId="13201" CreationDate="2020-05-07T19:40:13.340" Score="3" Body="&lt;p&gt;So I didn't find an answer,but I did find a way around if it anyone else has this problem. It does involve modifying the source code, but even if you don't understand c (I barely do) its still easy to do. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) Download source code from github &#xA;&lt;a href=&quot;https://github.com/cbcrg/tcoffee&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/cbcrg/tcoffee&lt;/a&gt;&#xA;Source code is here, view the dependency list - you will need all these libraries to compile the code for the change to take effect. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) Add your custom matrix to source code.&#xA;Search &lt;code&gt;matrices.h&lt;/code&gt;. In this version it is &lt;code&gt;t_coffee/lib/matrices.h&lt;/code&gt;&#xA;You will see how they are defining their default matrices. Just add your matrix with the same format and give it a name. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int customMT[]={&#xA;  4,&#xA;  0,  5,&#xA; -3, -2, 17,&#xA;  0,  5, -3,  9,&#xA;  0,  0,  1,  1,  6,&#xA; -2, -3, -3, -5, -4, 10,&#xA;  0,  0, -4, -1, -2, -3,  8,&#xA; -2, -2, -5, -2,  0, -3, -3, 14,&#xA;  0, -2, -2, -4, -3,  0, -1, -2,  6,&#xA;  0,  0, -3,  0,  2, -1, -1, -2, -2,  4,&#xA; -1, -1,  0, -1, -1,  2, -2, -1,  2, -2,  4,&#xA;  1, -2, -2, -3, -1, -2, -2,  2,  1,  2,  2,  6,&#xA;  0,  4, -1,  1, -1, -1,  0, -1,  0,  0, -2,  0,  8,&#xA; -1, -2, -3, -1,  1, -4, -1,  1, -3,  1, -3, -4, -3, 11,&#xA;  1, -1, -2, -1,  2, -3, -2,  0, -2,  0, -2, -1, -1,  0,  8,&#xA; -1, -2, -2, -1, -1, -1, -2, -1, -3,  1, -2,  0, -2, -1,  3,  8,&#xA;  1,  0, -2,  0,  0, -1,  0, -1, -1,  0, -2, -2,  0, -1, -1, -1,  4,&#xA;  1,  0, -2, -1, -2, -2, -2, -2,  0, -1,  0,  0,  1,  0,  0, -3,  2,  5,&#xA;  1, -2, -2, -2, -3,  1, -3, -3,  4, -2,  1,  0, -2, -4, -3, -1, -1,  1,  5,&#xA; -5, -5, -2, -4, -1,  1,  1, -5, -3, -2, -2, -3, -7, -3, -1,  0, -3, -5, -3, 20,&#xA;  0, -1, -2, -1, -1, -1, -1, -1,  0,  0,  0,  0,  0, -1,  0, -1,  0,  0,  0, -2, -1,&#xA; -4, -3, -6, -1, -2,  3, -3,  0, -1, -1,  3, -1, -4, -2, -1,  0, -2, -1,  1,  5, -1,  9,&#xA;  0,  0,  0,  0,  5, -4, -2,  0, -3,  1, -1, -1, -1,  0,  4,  0, -1, -1, -3, -1,  0, -2,  4};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, search for file &lt;code&gt;tcoffee/lib/io_lib/io_func.c&lt;/code&gt; Find the function &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;int ** read_matrices( char *mat_name_in)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You will see a long list of if else clauses, add the name of your custom matrix using the same pattern as they do. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/*Read Header Matrices*/&#xA;if (strm(mat_name, &quot;pam250mt&quot;))vector=pam250mt;&#xA;else if (strm(mat_name, &quot;idmat&quot;))vector=idmat;&#xA;else if (strm(mat_name, &quot;dna_idmat&quot;))vector=idmat;&#xA;&#xA;/*add custom matrix here*/&#xA;else if (strm(mat_name, &quot;customMT&quot;))vector=customMT;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;3) Save the files, move into tcoffee/src/. Rebuild the files using the make file as described on the github installation. It is here you need make, g++, perl, and whatever else on the list as dependencies&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;make t_coffee&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;4) Now, align your file! The only thing here is to use the t_coffee of the source code you just edited, and not t_coffee that might already be on your path. So change t_coffee/src/t_coffee to whatever the relative path to this folder is. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tcoffee/src/t_coffee rhomboids.fasta -matrix customMT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And that's it!&lt;/p&gt;&#xA;" OwnerUserId="8795" LastEditorUserId="8795" LastEditDate="2020-05-08T00:14:46.237" LastActivityDate="2020-05-08T00:14:46.237" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13217" PostTypeId="2" ParentId="13206" CreationDate="2020-05-07T19:43:40.410" Score="2" Body="&lt;p&gt;The fundamental unit of a Hi-C contact is the nearest restriction enzyme cut site.&#xA;This is where each mate in a read pair from a Hi-C library comes from and is mapped to when converting sequencing data into a contact matrix.&lt;/p&gt;&#xA;&lt;p&gt;Because the distance to the nearest cut site is variable at different genomic positions, and depth of coverage of any single locus is typically low, contacts are aggregated into bins and &amp;quot;normalized&amp;quot; to account for differences in cut site densities within those bins.&#xA;This normalization transforms the raw contact matrix into something resembling a stochastic transition matrix, where you can think of the elements of that matrix as &amp;quot;what is the probability that at any moment in time bin &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; is in contact with bin &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;So the probability you're looking for is the &lt;span class=&quot;math-container&quot;&gt;$(i, j)$&lt;/span&gt;th element of the normalized contact matrix, where bins &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; contain your regions of interest.&#xA;If your regions straddle the bin boundaries, you may want to average those bins.&lt;/p&gt;&#xA;&lt;p&gt;You get to pick the resolution of the contact matrix, but if you select a resolution that's too small, you run the risk of making your matrix too sparse and having no contact between those loci.&#xA;If you pick one that's too large, your matrix elements aren't really informative for your question at hand.&lt;/p&gt;&#xA;&lt;p&gt;Given your 500-5000 bp range and median of 1 kbp, it may make sense to pick a resolution between 1 kbp and 5 kbp and use that.&lt;/p&gt;&#xA;" OwnerUserId="240" LastEditorUserId="240" LastEditDate="2022-12-21T14:20:17.077" LastActivityDate="2022-12-21T14:20:17.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13219" PostTypeId="2" ParentId="13211" CreationDate="2020-05-07T23:11:44.763" Score="4" Body="&lt;p&gt;Not really an answer but an extended comment... and most likely something you don't like to hear&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess by technical replicates, it means taking the same biological sample and making 2 methylation libraries. If this is used as replicates in deseq2 or edger, the variance you are estimating is the technical variation that comes with preparing the library, bisulphite conversion etc.. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Often the biological variance will be much larger. That is if you take 2 independent samples of group A, their variation in methylation is much larger than the technical variation you see from handling the samples. If the technical replicates are treated as biological replicates and differential analysis is performed, it will be seriously misleading.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think both edgeR and DESeq2 doesn't estimate dispersion with 1 v 1 (which is right).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would suggest maybe the following:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Run a fisher.test at every gene (methylated / unmethylated), get the odds ratio and p-value, rank this, but explain that the p-value is only for ranking, and will underestimate the variation.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A bit more refined, look for other studies that might be close to yours, get an estimate of the dispersion in their study and use a negative binomial distribution to again rank genes that show the biggest difference&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="5077" LastEditorUserId="5077" LastEditDate="2020-05-08T08:02:46.343" LastActivityDate="2020-05-08T08:02:46.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13220" PostTypeId="1" CreationDate="2020-05-08T01:52:01.310" Score="1" ViewCount="164" Body="&lt;p&gt;I am looking at 59 mitochondrial CO1 DNA sequences of length 563bp, but it is a haplotype file and it has 9 haplotypes within.&lt;/p&gt;&#xA;&lt;p&gt;When I use .arp files in arlequin to run mismatch distribution it says that can’t converge beyond 2000 steps and also no graphs are produced. Can someone suggest what can I do to overcome this problem?&lt;/p&gt;&#xA;" OwnerUserId="8809" LastEditorUserId="73" LastEditDate="2022-05-30T04:17:54.850" LastActivityDate="2024-02-19T10:02:40.290" Title="Arlequin files not able to converge beyond 2000 steps for some .arp files" Tags="|statistics|sequence-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13221" PostTypeId="1" CreationDate="2020-05-08T12:02:33.013" Score="1" ViewCount="141" Body="&lt;p&gt;I have bins from metagenomes generated with &lt;a href=&quot;https://bitbucket.org/berkeleylab/metabat/src&quot; rel=&quot;nofollow noreferrer&quot;&gt;metabat2&lt;/a&gt; on &lt;code&gt;bitbucket/berkleylab/scr&lt;/code&gt; which are predicted as viral by &lt;a href=&quot;https://github.com/laboratoriobioinformatica/MARVEL&quot; rel=&quot;nofollow noreferrer&quot;&gt;MARVEL&lt;/a&gt;. &lt;code&gt;MARVEL&lt;/code&gt; suggests to merge contigs with overlapping ends with &lt;a href=&quot;http://www.phrap.org/phredphrapconsed.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PHRAP&lt;/a&gt; but the documentation on the tool doesn't seem great.&lt;/p&gt;&#xA;&lt;p&gt;As it appears to be an assembler would it be okay to use another assembler? For example one of my bins has 4 contigs ~1000-5000bp in length. 3 of these seem to overlap with each other enitirely.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I found a tool &lt;a href=&quot;http://github.com/LANL-Bioinformatics/MeGAMerge&quot; rel=&quot;nofollow noreferrer&quot;&gt;MeGAMerge&lt;/a&gt; on github which might work. Has anyone used this?&lt;/p&gt;&#xA;" OwnerUserId="7287" LastEditorUserId="3848" LastEditDate="2020-12-06T15:07:17.223" LastActivityDate="2020-12-06T15:07:17.223" Title="Alternatives to phrap for merging contigs with overlapping ends?" Tags="|covid-19|sars-cov-2|metagenome|microbial-genomics|virus|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13222" PostTypeId="1" CreationDate="2020-05-08T13:06:19.627" Score="0" ViewCount="203" Body="&lt;p&gt;I have always used &lt;code&gt;cellranger mkfastq&lt;/code&gt; to demultiplex 10x genomics runs manually, though recently the commands to do so have been incorporated into a script that should call the &lt;code&gt;cellranger&lt;/code&gt; whenever it detects that a run is finished (and can be demultiplexed).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The main difference when calling manually compared to being called automatically by the script is &lt;strong&gt;from where&lt;/strong&gt; the commands are being called.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So calling the commands manually &lt;strong&gt;from the directory where the samplesheet is and where the outputs shall be saved&lt;/strong&gt; with:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/home/ngs/installed/cellranger-3.1.0/cellranger mkfastq --id=demux_10x --run=/nexusb/Novaseq/200507_A00154_0492_AHL72WDRXX --samplesheet=/nexusb/Novaseq/200507_A00154_0492_AHL72WDRXX/20200507_NovaSeq_129_Chromium_Anne_Fred_Samplesheet.csv --qc &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;does the job.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but now cellranger is being called (automatically) from &lt;code&gt;/home/ngs&lt;/code&gt;. The error message that I am getting from cellranger is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;/home/ngs/installed/cellranger-3.1.0/cellranger mkfastq --id=/nexusb/Novaseq/200507_A00154_0492_AHL72WDRXX/demux_10x/ --run=/nexusb/Novaseq/200507_A00154_0492_AHL72WDRXX --samplesheet=/nexusb/Novaseq/200507_A00154_0492_AHL72WDRXX/20200507_NovaSeq_129_Chromium_Anne_Fred_Samplesheet.csv --qc&#xA;/home/ngs/installed/cellranger-3.1.0/cellranger-cs/3.1.0/bin&#xA;cellranger mkfastq (3.1.0)&#xA;Copyright (c) 2019 10x Genomics, Inc.  All rights reserved.&#xA;-------------------------------------------------------------------------------&#xA;&#xA;/home/ngs/installed/cellranger-3.1.0/cellranger-cs/3.1.0/tenkit/bin/_mkfastq: line 531: __/nexusb/Novaseq/200507_A00154_0492_AHL72WDRXX/demux_10x/.mro: No such file or directory&#xA;Martian Runtime - '3.1.0-v3.2.3'&#xA;&#xA;Invalid name: /nexusb/Novaseq/200507_A00154_0492_AHL72WDRXX/demux_10x/ (only numbers, letters, dash, and underscore allowed)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;According to the error message I cannot use slashes &lt;code&gt;/&lt;/code&gt; in the &lt;code&gt;--id&lt;/code&gt; flag.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anyone what I should pass to &lt;code&gt;--id&lt;/code&gt; in order to get the outputs saved as&#xA;&lt;code&gt;/nexusb/Novaseq/200507_A00154_0492_AHL72WDRXX/demux_10x/&lt;/code&gt;?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I already know that if I just pass &lt;code&gt;--id=demux_10x&lt;/code&gt; the outputs will be saved at &lt;code&gt;/home/ngs/demux_10x&lt;/code&gt;. (again, I am calling the script that calls &lt;code&gt;cellranger&lt;/code&gt; from &lt;code&gt;/home/ngs/&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Obviously I could change to the desired output directory i.e. &lt;code&gt;/nexusb/Novaseq/200507_A00154_0492_AHL72WDRXX/&lt;/code&gt; and do &lt;code&gt;--id=demux_10&lt;/code&gt;, though I am trying to avoid that for now.&lt;/p&gt;&#xA;" OwnerUserId="3452" LastActivityDate="2020-05-08T17:03:40.327" Title="cellranger mkfastq with full path to --id flag" Tags="|rna-seq|10x-genomics|cellranger|demultiplexing|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13223" PostTypeId="1" CreationDate="2020-05-08T14:06:28.940" Score="0" ViewCount="36" Body="&lt;p&gt;I have an enhancer list and I would like to see how it overlaps with histone lists from different cell types (different sizes) significantly. I would like to use a contingency table for testing but not sure which format: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Should I use 2xk format&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                         Enhancer overlap       Enhancer not overlap&#xA;-Histone Cell Type 1             n                       n  &#xA;-Histone Cell Type 2             n                       n  &#xA;-Histone Cell Type 3             n                       n  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or individual 2x2 formats&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                                  Enhancer overlap      Enhancer not overlap&#xA;-Histone Cell Type 1 present           n                         n  &#xA;-Histone Cell Type 1 absent            n                         n  &#xA;&#xA;                                 Enhancer overlap      Enhancer not overlap&#xA;-Histone Cell Type 2 present           n                         n  &#xA;-Histone Cell Type 2 absent            n                         n  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8813" LastEditorUserId="8813" LastEditDate="2020-05-08T15:55:31.163" LastActivityDate="2020-05-08T15:55:31.163" Title="2x2 contingency for genomic feature enrichment" Tags="|genomefeatures|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13224" PostTypeId="2" ParentId="13220" CreationDate="2020-05-08T14:10:07.000" Score="0" Body="&lt;p&gt;What you appear to have is a Bayesian MCMC for 2000 &quot;steps&quot; which isn't a lot and normally the term 'steps' is used for parsimony, so I'm a bit confused. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, if a Bayesian analysis does not reach convergence, i.e. duplicate runs do not achieve the same results then the data space has not been solved so you get no result. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;9 haplotypes does not sound much to me, it is possible there are too many sequences, it is also possible there is simply not enough variation (9 haplotypes ... how many alleles?). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suggestions&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Arlequin should provide a default convergence value which you can lower to force a result&lt;/li&gt;&#xA;&lt;li&gt;Decrease the number of sequences but maintain the number of haplotypes&lt;/li&gt;&#xA;&lt;li&gt;Try their example data set&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Do you get any associated file such as posterior probability, any output file with a minus sign in front of it?&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-05-08T14:10:07.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13225" PostTypeId="2" ParentId="13215" CreationDate="2020-05-08T14:29:02.880" Score="2" Body="&lt;p&gt;My comment above mentions the slow IO for the Python code (it was iterating at ~ 3600 reads/s, which, for a FASTQ with 500M reads would take ~1.6 d to complete), so I tried an alternative method that would cut down on the IO overhead. It's not super efficient, but I ended up using &lt;code&gt;zcat | awk&lt;/code&gt; with a &lt;code&gt;switch-case&lt;/code&gt; on the line number to stream the data.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;# first pass to generate _R1 FASTQ&#xA;zcat {input} | awk '{switch (NR % 4) {case 1: print &lt;span class=&quot;math-container&quot;&gt;$1; break; case 2: print substr($&lt;/span&gt;1, 1, 76); break; case 3: print &amp;quot;+&amp;quot;; break; case 0: print substr($1, 1, 76)} }' | gzip -nc &amp;gt; {output}_R1.fastq.gz&#xA;&#xA;# second pass to generate _R2 FASTQ&#xA;zcat {input} | awk '{switch (NR % 4) {case 1: print &lt;span class=&quot;math-container&quot;&gt;$1; break; case 2: print substr($&lt;/span&gt;1, 77, 76); break; case 3: print &amp;quot;+&amp;quot;; break; case 0: print substr($1, 77, 76)} }' | gzip -nc &amp;gt; {output}_R2.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I played around with trying to pipe each half of the combined read into its own file, but it was difficult to set that up and generalize it for the multiple FASTQs that I wanted to fix. I didn't feel like over-engineering something, and this was straightforward enough. It worked much quicker than I expected.&lt;/p&gt;&#xA;&lt;p&gt;All in all this took ~4 h to complete, which means an iteration speed of ~ 70 000 reads/s, or about 20x faster than the Biopython solution.&#xA;If I engineered a solution that would have only passed through the FASTQ once, it would've taken half the time, but in either case it was likely shorter than re-downloading the entire FASTQ.&lt;/p&gt;&#xA;&lt;p&gt;But I will probably make use of &lt;code&gt;fasterq-dump&lt;/code&gt; next time, and make sure I specify to split the read mates.&lt;/p&gt;&#xA;&lt;p&gt;I love how how extensive the default biopython fastq parser library is, but the slow IO for large sequencing files makes it unusable in a lot of cases, unfortunately. I wonder if there's a good way to use C bindings to make this faster, or whether you just need to use a different package/language.&lt;/p&gt;&#xA;" OwnerUserId="240" LastEditorUserId="73" LastEditDate="2021-07-05T04:29:54.200" LastActivityDate="2021-07-05T04:29:54.200" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13226" PostTypeId="2" ParentId="13222" CreationDate="2020-05-08T16:24:04.603" Score="0" Body="&lt;p&gt;I suspect the error message is a bit exaggerated.  Try removing the slash at the end of your --id&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-05-08T16:24:04.603" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13227" PostTypeId="2" ParentId="13198" CreationDate="2020-05-08T16:53:44.123" Score="0" Body="&lt;p&gt;Okay, I finally solved it. The reason was the &lt;code&gt;EGA_CLIENT_ID&lt;/code&gt; variable. In the &lt;code&gt;pyega3.py&lt;/code&gt; &lt;a href=&quot;https://github.com/EGA-archive/ega-download-client/blob/master/pyega3/pyega3.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;script&lt;/a&gt;, there is a definition for the &lt;code&gt;client_id&lt;/code&gt; for pyega3 v3.0.44 (&quot;f20cd2d3-682a-4568-a53e-4262ef54c8f4&quot;). After setting the &lt;code&gt;EGA_CLIENT_ID&lt;/code&gt; to this value login was successful. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;os.environ[&quot;EGA_CLIENT_ID&quot;] = &quot;f20cd2d3-682a-4568-a53e-4262ef54c8f4&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I think this is a bug and I filed it on Snakemake's git repository - the official documentation just requires definition of the &lt;code&gt;EGA_USERNAME&lt;/code&gt; and &lt;code&gt;EGA_PASSWORD&lt;/code&gt; variables, and while definition of &lt;code&gt;EGA_CLIENT_SECRET&lt;/code&gt; will probably need to be done by the user, the client ID should be pulled automatically from the pyega3 version installed locally. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of note, after successful login, the example download from the documentation failed with the following:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Missing input files for rule a:&#xA;ega/EGAD00001002142/COLO_829_EPleasance_TGENPipe.bam.bai&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, downloading files that I have access to worked, indicating that the paths to the files from the documentation are wrong or are access-protected. &lt;/p&gt;&#xA;" OwnerUserId="5046" LastActivityDate="2020-05-08T16:53:44.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13228" PostTypeId="2" ParentId="13222" CreationDate="2020-05-08T17:03:40.327" Score="1" Body="&lt;p&gt;You should be able to use the &lt;code&gt;--output-dir&lt;/code&gt; argument to specify a folder of your choice and use &lt;code&gt;--id=demux_10x&lt;/code&gt; as before.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-05-08T17:03:40.327" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13230" PostTypeId="1" CreationDate="2020-05-09T09:21:43.603" Score="0" ViewCount="1556" Body="&lt;p&gt;The example that I am trying to follow is &lt;a href=&quot;http://bioconductor.org/packages/devel/bioc/vignettes/PGA/inst/doc/PGA.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this PGA tutorial&lt;/a&gt;.&#xA;I want to use the information in the analysis of the mzML raw files from the proteomics data analysis.&#xA;When I try to load the fasta file, I get the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; msfile=openMSfile(&quot;/scratch/oknjav001/transcriptomics/proteogenomics/ms_rnaseqdata/msData/LTB/t16BCG.180731_C0_CH_T016BCG.mzML&quot;)&#xA;&amp;gt; msfile&#xA;Mass Spectrometry file handle.&#xA;Filename: 1807_C0_CH_T016BCG.mzML&#xA;Number of scans: 52332&#xA;&amp;gt; idfile &amp;lt;- runTandem(&#xA;+    spectra = msfile, fasta = dbfile, outdir = &quot;./&quot;, cpu = 6,&#xA;+    enzyme = &quot;[KR]|[X]&quot;, varmod = &quot;15.994915@M&quot;, itol = 0.05,&#xA;+    fixmod = &quot;57.021464@C&quot;, tol = 10, tolu = &quot;ppm&quot;,&#xA;+    itolu = &quot;Daltons&quot;, miss = 2, maxCharge = 8, ti = FALSE&#xA;+ )&#xA;Error in as.data.frame.default(x[[i]], optional = TRUE, stringsAsFactors = stringsAsFactors) :&#xA;  cannot coerce class '&quot;SeqFastaAA&quot;' to a data.frame&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I go about fixing this error?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The example that I am trying to follow this &lt;a href=&quot;http://bioconductor.org/packages/devel/bioc/vignettes/PGA/inst/doc/PGA.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PGA tutorial&lt;/a&gt;. I want to use this information in the analysis if the mzML raw files from the proteomics data analysis. When I try to load the fasta file I then get the above error.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;msfile &amp;lt;- system.file(&quot;extdata/input&quot;, &quot;pga.mgf&quot;,package=&quot;PGA&quot;)&#xA;idfile &amp;lt;- runTandem(spectra = msfile, fasta = dbfile, outdir = &quot;./&quot;, cpu = 6,enzyme = &quot;[KR]|[X]&quot;, varmod = &quot;15.994915@M&quot;,itol = 0.05,fixmod = &quot;57.021464@C&quot;, tol = 10, tolu = &quot;ppm&quot;,itolu = &quot;Daltons&quot;, miss = 2, maxCharge = 8, ti = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8818" LastEditorUserId="77" LastEditDate="2020-05-10T10:25:11.103" LastActivityDate="2020-05-10T10:25:11.103" Title="Error in as.data.frame.default(x[[i]], optional = TRUE, stringsAsFactors = stringsAsFactors) : cannot coerce class ‘&quot;SeqFastaAA&quot;’ to a data.frame" Tags="|r|bioconductor|proteomics|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13233" PostTypeId="1" AcceptedAnswerId="13235" CreationDate="2020-05-10T00:48:56.503" Score="2" ViewCount="522" Body="&lt;p&gt;I am manipulating some BAM files using &lt;code&gt;pysam&lt;/code&gt; package which seems to be very fast and handy. However, I ran into problem when I am trying to generate bigWig files from BAM files in Python. I used &lt;code&gt;bamCoverage&lt;/code&gt; from &lt;code&gt;deeptools&lt;/code&gt; to do this previously, but it is a standalone tool that cannot be used from within my Python scripts.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, I am wondering what Python package would allow me to read in BAM files and convert them to bigWig files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2021-02-28T20:03:42.263" Title="How to convert BAM file to bigWig in Python?" Tags="|python|sam|pysam|deeptools|bigwig|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13234" PostTypeId="1" AcceptedAnswerId="13236" CreationDate="2020-05-10T02:28:01.460" Score="0" ViewCount="1037" Body="&lt;p&gt;I have a data frame for two groups and 9 variables &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(aa)&#xA;  treatment variable     value&#xA;1       pre     Sig1 0.1173863&#xA;2       pre     Sig1 0.1888243&#xA;3       pre     Sig1 0.2191765&#xA;4       pre     Sig1 0.1277735&#xA;5       pre     Sig1 0.1711712&#xA;6       pre     Sig1 0.0000000&#xA;&amp;gt; class(aa)&#xA;[1] &quot;data.frame&quot;&#xA;&amp;gt; str(aa)&#xA;'data.frame':   1026 obs. of  3 variables:&#xA; &lt;span class=&quot;math-container&quot;&gt;$ treatment: Factor w/ 2 levels &quot;post&quot;,&quot;pre&quot;: 2 2 2 2 2 2 2 2 2 2 ...&#xA; $&lt;/span&gt; variable : Factor w/ 9 levels &quot;Sig1&quot;,&quot;Sig2&quot;,..: 1 1 1 1 1 1 1 1 1 1 ...&#xA; $ value    : num  0.117 0.189 0.219 0.128 0.171 ...&#xA;&amp;gt; tail(aa)&#xA;     treatment variable value&#xA;1021      post     Sig9     0&#xA;1022      post     Sig9     0&#xA;1023      post     Sig9     0&#xA;1024      post     Sig9     0&#xA;1025      post     Sig9     0&#xA;1026      post     Sig9     0&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I am using this code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(ggpubr)&#xA;&amp;gt; p &amp;lt;- ggboxplot(aa, x = &quot;variable&quot;, y = &quot;value&quot;,color = &quot;treatment&quot;)&#xA;&amp;gt; my_comparisons &amp;lt;- c(&quot;pre&quot;,&quot;post&quot;)&#xA;&amp;gt; p + stat_compare_means(comparisons = my_comparisons)+ stat_compare_means(label.y = 50)  &#xA;Warning message:&#xA;Computation failed in `stat_signif()`:&#xA;missing value where TRUE/FALSE needed &#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I am getting this weird plot&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/72a87.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/72a87.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While something like this is my goal&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kI85K.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kI85K.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I really don't know where I am doing wrong&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used ggplot where I got something like this but I need statistics that is why I have stuck on this &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XM5WB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XM5WB.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then used this code &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; DF &amp;lt;- aa %&amp;gt;% pivot_longer(.,cols = c(Sig1,Sig2,Sig3, Sig4,Sig5,Sig6,Sig7,Sig8,Sig9), names_to = &quot;var&quot;, values_to = &quot;val&quot;)&#xA;&amp;gt; View(DF)&#xA;&amp;gt; library(ggplot2)&#xA;&amp;gt; library(ggpubr)&#xA;&amp;gt; ggplot(DF, aes(x = treatment, y = val, fill = treatment))+&#xA;+     geom_boxplot()+&#xA;+     stat_compare_means(comparisons = c(&quot;pre&quot;,&quot;post&quot;)+&#xA;+     stat_compare_means()+&#xA;+     facet_grid(var~., scales = &quot;free&quot;)&#xA;+ &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But I got another weird plot&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pLvl1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pLvl1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But changing the scale I got this but gives one statistics&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mZMhI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mZMhI.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I need one stat for each variable between pre and post groups&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This solution gives to statistics&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; p &amp;lt;- ggboxplot(aa, x = &quot;variable&quot;, y = &quot;value&quot;,color = &quot;treatment&quot;,facet.by = &quot;variable&quot;)&#xA;&amp;gt; p + stat_compare_means(comparisons = my_comparisons)+ stat_compare_means(label.y =2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2C08u.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2C08u.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When I edited my code&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; p &amp;lt;- ggboxplot(aa, x = &quot;variable&quot;, y = &quot;value&quot;,color = &quot;treatment&quot;)&#xA;&amp;gt; p + stat_compare_means(aes(group = variable))&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;But again one stat&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hoxit.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hoxit.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-05-10T13:16:29.503" LastActivityDate="2020-05-10T18:41:27.870" Title="Getting weird plot with ggpubr package" Tags="|ggplot2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13235" PostTypeId="2" ParentId="13233" CreationDate="2020-05-10T06:50:10.850" Score="4" Body="&lt;p&gt;deepTools has a (somewhat poorly documented) API, since it's a python package too. The basic code framework is:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from deeptools import writeBedGraph&#xA;from deeptools.getScaleFactor import get_scale_factor&#xA;&#xA;wr = writeBedGraph.WriteBedGraph(options and input files)&#xA;func_args = {'scaleFactor': get_scale_factor(...options...)}&#xA;wr.run(writeBedGraph.scaleCoverage, func_args, output_file_name, ...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-05-10T06:50:10.850" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13236" PostTypeId="2" ParentId="13234" CreationDate="2020-05-10T07:30:40.367" Score="3" Body="&lt;p&gt;The problem is the scale used: For the plot you called &quot;weird&quot; (first from the top), the scale is 50 and for the &quot;ggplot only&quot; (third from the top) the scale is 1.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You should play with the &lt;code&gt;stat_compare_means(label.y = 50)&lt;/code&gt; bit, you can try setting the &lt;code&gt;label.y&lt;/code&gt; parameter to 1.5 or 2.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT BECAUSE OF THE ADDITIONAL REMARKS ON THE QUESTION:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Once again (and for sure the last time), I had to manually create example data as OP ignores repeated reminders on &lt;a href=&quot;https://stackoverflow.com/questions/5963269/how-to-make-a-great-r-reproducible-example&quot;&gt;how to make a reproducible example with R&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(data.table)&#xA;library(ggpubr)&#xA;&#xA;text &amp;lt;- &quot;&#xA;treatment variable     value&#xA;1       pre     Sig1 0.1173863&#xA;2       pre     Sig1 0.1888243&#xA;3       pre     Sig1 0.2191765&#xA;4       pre     Sig1 0.1277735&#xA;5       pre     Sig1 0.1711712&#xA;6       pre     Sig1 0.0000000&#xA;1       post     Sig1 0.1173863&#xA;2       post     Sig1 0.1888243&#xA;3       post     Sig1 0.2191765&#xA;4       post     Sig1 0.1277735&#xA;5       post     Sig1 0.1711712&#xA;6       post     Sig1 0.0000000&#xA;1021      pre     Sig9     0&#xA;1022      pre     Sig9     0&#xA;1023      pre     Sig9     0&#xA;1024      pre     Sig9     0&#xA;1025      pre     Sig9     0&#xA;1026      pre     Sig9     0&#xA;1021      post     Sig9     1&#xA;1022      post     Sig9     1&#xA;1023      post     Sig9     1&#xA;1024      post     Sig9     1&#xA;1025      post     Sig9     1&#xA;1026      post     Sig9     1&#xA;&quot;&#xA;my_data &amp;lt;-fread(text)&#xA;&#xA;&amp;gt; my_data&#xA;      V1 treatment variable     value&#xA; 1:    1       pre     Sig1 0.1173863&#xA; 2:    2       pre     Sig1 0.1888243&#xA; 3:    3       pre     Sig1 0.2191765&#xA; 4:    4       pre     Sig1 0.1277735&#xA; 5:    5       pre     Sig1 0.1711712&#xA; 6:    6       pre     Sig1 0.0000000&#xA; 7:    1      post     Sig1 0.1173863&#xA; 8:    2      post     Sig1 0.1888243&#xA; 9:    3      post     Sig1 0.2191765&#xA;10:    4      post     Sig1 0.1277735&#xA;11:    5      post     Sig1 0.1711712&#xA;12:    6      post     Sig1 0.0000000&#xA;13: 1021       pre     Sig9 0.0000000&#xA;14: 1022       pre     Sig9 0.0000000&#xA;15: 1023       pre     Sig9 0.0000000&#xA;16: 1024       pre     Sig9 0.0000000&#xA;17: 1025       pre     Sig9 0.0000000&#xA;18: 1026       pre     Sig9 0.0000000&#xA;19: 1021      post     Sig9 1.0000000&#xA;20: 1022      post     Sig9 1.0000000&#xA;21: 1023      post     Sig9 1.0000000&#xA;22: 1024      post     Sig9 1.0000000&#xA;23: 1025      post     Sig9 1.0000000&#xA;24: 1026      post     Sig9 1.0000000&#xA;      V1 treatment variable     value&#xA;&#xA;ggboxplot(my_data,&#xA;          x = &quot;treatment&quot;,&#xA;          y = &quot;value&quot;,&#xA;          color = &quot;treatment&quot;,&#xA;          facet.by = &quot;variable&quot;) + &#xA;  stat_compare_means(label = &quot;p.format&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this code generates:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3n2Xn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3n2Xn.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2020-05-10T18:41:27.870" LastActivityDate="2020-05-10T18:41:27.870" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="13237" PostTypeId="1" AcceptedAnswerId="13239" CreationDate="2020-05-10T14:40:45.103" Score="3" ViewCount="3480" Body="&lt;p&gt;Has anyone used bioconductor from Python? Is there any reason I would choose to use it from R instead of Python? It seems like there is a &lt;a href=&quot;https://pythonhosted.org/rpy2-bioconductor-extensions/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python extension&lt;/a&gt; for it. Also, is there any reason to use bioconductor over Biopython? &lt;/p&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="8601" LastEditDate="2020-05-10T15:34:50.287" LastActivityDate="2023-08-10T08:50:07.313" Title="Using bioconductor from Python" Tags="|r|python|bioconductor|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13238" PostTypeId="1" AcceptedAnswerId="13240" CreationDate="2020-05-10T19:12:23.763" Score="1" ViewCount="84" Body="&lt;p&gt;I've might have created a new algorithm for finding patterns in DNA. The technique has not been used before, and it managed to find a 33-mer with corresponding complement that exists both in E. Coli and in Salmonella. I think it's pretty neat but I don't know how to evaluate the results. In addition, I've developed this algorithm after a bioinformatics course where only the DNA sequences of E. Coli and Salmonella were provided, so I would like to test it on other sequences and have it evaluated by people who know what they are doing.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How would one get in contact with the right people for evaluating amateur bioinformatics result?&lt;/p&gt;&#xA;" OwnerUserId="8824" LastActivityDate="2020-05-10T19:58:45.053" Title="Evaluating amateur bioinformatics results" Tags="|sequence-analysis|algorithms|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="13239" PostTypeId="2" ParentId="13237" CreationDate="2020-05-10T19:15:47.220" Score="5" Body="&lt;p&gt;Bioconductor packages are R packages that can only be used directly from python via rpy2 or similar. If you've ever had reason to use rpy2 you'd probably realize quickly that while it works, it's no where near as easy as doing the same thing directly in R.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Biopython is a package for doing a number of useful things, but it can in no way come remotely close to performing even a percent of what Bioconductor packages do. That would be an impossible feat for a single python package.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-05-10T19:15:47.220" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13240" PostTypeId="2" ParentId="13238" CreationDate="2020-05-10T19:33:51.280" Score="1" Body="&lt;p&gt;You need to understand the biological significance of the 33-mer locus. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I'd blast your 33-mer on &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI&lt;/a&gt; and see what the biological significane is of the locus. If might be a transposon/ promoter region. &lt;/li&gt;&#xA;&lt;li&gt;Look at the translation of the protein and if its part of a protein the location within the protein structure.   &lt;/li&gt;&#xA;&lt;li&gt;Blast the entire locus including 5' and 3' regions of the 33-mer target&lt;/li&gt;&#xA;&lt;li&gt;Carefully assess the results for chance, e.g. conserved protein and the synonymous mutations just happened to drop out (Poison distribution).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Once you gain a greater understanding, it is likely to have been identified before and what you might contribute is a different method to obtain a known result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, you need the next level part of the course about translating a genetic correlation into a biological result/ mechanism. This isn't research grade work at present in my opinion.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-05-10T19:58:45.053" LastActivityDate="2020-05-10T19:58:45.053" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13241" PostTypeId="2" ParentId="4681" CreationDate="2020-05-10T21:08:09.610" Score="0" Body="&lt;p&gt;The motifbreakR package for R I believe does something like what you want to do;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.rdocumentation.org/packages/motifbreakR/versions/1.2.2/topics/motifbreakR&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rdocumentation.org/packages/motifbreakR/versions/1.2.2/topics/motifbreakR&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're looking for a general rule as to what kind of mutations/where in the motif mutations will significantly affect motif binding, I suggest generating a lot of sequences with simulated SNP's (single nucleotide polymorphisms) and seeing how these score against the particular motif. Then, you will have a large(ish) database of sequences with one base pair difference from the motif and their corresponding score. This will allow you to judge what kinds of mutations affect the score the most. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Furthermore, you could somehow make a distribution of scores versus mutations and generate an empirical p-value to determine statistical significance (for instance, such and such mutation caused a score that was in the lowest 5% of scores for all mutations). You would know better than me how to do this, since your background is in CS and my PhD subject is genetics.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make an assumption about the number of bases that need to be mutated, do this for one, two, three, etc. simulated SNP's.&lt;/p&gt;&#xA;" OwnerUserId="8825" LastEditorUserId="8825" LastEditDate="2020-05-10T22:46:43.803" LastActivityDate="2020-05-10T22:46:43.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13242" PostTypeId="1" AcceptedAnswerId="15965" CreationDate="2020-05-11T00:34:12.847" Score="0" ViewCount="638" Body="&lt;p&gt;I want to install &lt;code&gt;NNLM&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;install_github('linxihui/NNLM')&#xA;&#xA;lang: error: unsupported option '-fopenmp'&#xA;make: *** [RcppExports.o] Error 1&#xA;ERROR: compilation failed for package ‘NNLM’&#xA;* removing ‘/Library/Frameworks/R.framework/Versions/3.6/Resources/library/NNLM’&#xA;Error: Failed to install 'NNLM' from GitHub:&#xA;  (converted from warning) installation of package ‘/var/folders/wk/qwgr9yqs6ts0czx__y_w8gch0000gq/T//RtmpA8PudK/filec5bce1f1309/NNLM_0.4.4.tar.gz’ had non-zero exit status&#xA;&amp;gt; &#xA;&amp;gt; install.packages('NNLM')&#xA;Warning in install.packages :&#xA;  package ‘NNLM’ is not available (for R version 3.6.3)&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What should I do?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; session_info()&#xA;─ Session info ─────────────────────────────────────────────────────────────────&#xA; setting  value                       &#xA; version  R version 3.6.3 (2020-02-29)&#xA; os       macOS Mojave 10.14.6        &#xA; system   x86_64, darwin15.6.0        &#xA; ui       RStudio                     &#xA; language (EN)                        &#xA; collate  en_GB.UTF-8                 &#xA; ctype    en_GB.UTF-8                 &#xA; tz       Europe/London               &#xA; date     2020-05-11                  &#xA;&#xA;─ Packages ─────────────────────────────────────────────────────────────────────&#xA; package      * version date       lib source                               &#xA; abind          1.4-5   2016-07-21 [1] CRAN (R 3.6.0)                       &#xA; assertthat     0.2.1   2019-03-21 [1] CRAN (R 3.6.0)                       &#xA; backports      1.1.6   2020-04-05 [1] CRAN (R 3.6.3)                       &#xA; broom          0.5.6   2020-04-20 [1] CRAN (R 3.6.3)                       &#xA; callr          3.4.3   2020-03-28 [1] CRAN (R 3.6.2)                       &#xA; car            3.0-7   2020-03-11 [1] CRAN (R 3.6.0)                       &#xA; carData        3.0-3   2019-11-16 [1] CRAN (R 3.6.0)                       &#xA; cellranger     1.1.0   2016-07-27 [1] CRAN (R 3.6.0)                       &#xA; cli            2.0.2   2020-02-28 [1] CRAN (R 3.6.0)                       &#xA; colorspace     1.4-1   2019-03-18 [1] CRAN (R 3.6.0)                       &#xA; crayon         1.3.4   2017-09-16 [1] CRAN (R 3.6.0)                       &#xA; curl           4.3     2019-12-02 [1] CRAN (R 3.6.0)                       &#xA; data.table   * 1.12.8  2019-12-09 [1] CRAN (R 3.6.0)                       &#xA; desc           1.2.0   2018-05-01 [1] CRAN (R 3.6.0)                       &#xA; devtools     * 2.3.0   2020-04-10 [1] CRAN (R 3.6.3)                       &#xA; digest         0.6.25  2020-02-23 [1] CRAN (R 3.6.0)                       &#xA; dplyr          0.8.5   2020-03-07 [1] CRAN (R 3.6.0)                       &#xA; ellipsis       0.3.0   2019-09-20 [1] CRAN (R 3.6.0)                       &#xA; fansi          0.4.1   2020-01-08 [1] CRAN (R 3.6.0)                       &#xA; farver         2.0.3   2020-01-16 [1] CRAN (R 3.6.0)                       &#xA; forcats        0.5.0   2020-03-01 [1] CRAN (R 3.6.0)                       &#xA; foreign        0.8-76  2020-03-03 [1] CRAN (R 3.6.0)                       &#xA; fs             1.4.1   2020-04-04 [1] CRAN (R 3.6.3)                       &#xA; generics       0.0.2   2018-11-29 [1] CRAN (R 3.6.0)                       &#xA; ggplot2      * 3.3.0   2020-03-05 [1] CRAN (R 3.6.0)                       &#xA; ggpubr       * 0.3.0   2020-05-04 [1] CRAN (R 3.6.3)                       &#xA; ggsignif     * 0.6.0   2019-08-08 [1] CRAN (R 3.6.0)                       &#xA; glue           1.4.0   2020-04-03 [1] CRAN (R 3.6.3)                       &#xA; gtable         0.3.0   2019-03-25 [1] CRAN (R 3.6.0)                       &#xA; haven          2.2.0   2019-11-08 [1] CRAN (R 3.6.0)                       &#xA; hms            0.5.3   2020-01-08 [1] CRAN (R 3.6.0)                       &#xA; labeling       0.3     2014-08-23 [1] CRAN (R 3.6.0)                       &#xA; lattice        0.20-41 2020-04-02 [1] CRAN (R 3.6.3)                       &#xA; lifecycle      0.2.0   2020-03-06 [1] CRAN (R 3.6.0)                       &#xA; maftools     * 2.4.05  2020-05-10 [1] Github (PoisonAlien/maftools@6ddf526)&#xA; magrittr       1.5     2014-11-22 [1] CRAN (R 3.6.0)                       &#xA; Matrix         1.2-18  2019-11-27 [1] CRAN (R 3.6.3)                       &#xA; memoise        1.1.0   2017-04-21 [1] CRAN (R 3.6.0)                       &#xA; munsell        0.5.0   2018-06-12 [1] CRAN (R 3.6.0)                       &#xA; nlme           3.1-147 2020-04-13 [1] CRAN (R 3.6.2)                       &#xA; openxlsx       4.1.5   2020-05-06 [1] CRAN (R 3.6.2)                       &#xA; pillar         1.4.4   2020-05-05 [1] CRAN (R 3.6.3)                       &#xA; pkgbuild       1.0.8   2020-05-07 [1] CRAN (R 3.6.2)                       &#xA; pkgconfig      2.0.3   2019-09-22 [1] CRAN (R 3.6.0)                       &#xA; pkgload        1.0.2   2018-10-29 [1] CRAN (R 3.6.0)                       &#xA; plyr           1.8.6   2020-03-03 [1] CRAN (R 3.6.0)                       &#xA; prettyunits    1.1.1   2020-01-24 [1] CRAN (R 3.6.0)                       &#xA; processx       3.4.2   2020-02-09 [1] CRAN (R 3.6.0)                       &#xA; ps             1.3.3   2020-05-08 [1] CRAN (R 3.6.2)                       &#xA; purrr          0.3.4   2020-04-17 [1] CRAN (R 3.6.2)                       &#xA; R6             2.4.1   2019-11-12 [1] CRAN (R 3.6.0)                       &#xA; RColorBrewer   1.1-2   2014-12-07 [1] CRAN (R 3.6.0)                       &#xA; Rcpp           1.0.4.6 2020-04-09 [1] CRAN (R 3.6.3)                       &#xA; readxl         1.3.1   2019-03-13 [1] CRAN (R 3.6.0)                       &#xA; remotes      * 2.1.1   2020-02-15 [1] CRAN (R 3.6.0)                       &#xA; reshape      * 0.8.8   2018-10-23 [1] CRAN (R 3.6.0)                       &#xA; rio            0.5.16  2018-11-26 [1] CRAN (R 3.6.0)                       &#xA; rlang          0.4.6   2020-05-02 [1] CRAN (R 3.6.2)                       &#xA; rprojroot      1.3-2   2018-01-03 [1] CRAN (R 3.6.0)                       &#xA; rstatix        0.5.0   2020-04-28 [1] CRAN (R 3.6.2)                       &#xA; rstudioapi     0.11    2020-02-07 [1] CRAN (R 3.6.0)                       &#xA; scales         1.1.0   2019-11-18 [1] CRAN (R 3.6.0)                       &#xA; sessioninfo    1.1.1   2018-11-05 [1] CRAN (R 3.6.0)                       &#xA; stringi        1.4.6   2020-02-17 [1] CRAN (R 3.6.0)                       &#xA; survival       3.1-12  2020-04-10 [1] CRAN (R 3.6.3)                       &#xA; testthat       2.3.2   2020-03-02 [1] CRAN (R 3.6.0)                       &#xA; tibble         3.0.1   2020-04-20 [1] CRAN (R 3.6.3)                       &#xA; tidyr          1.0.3   2020-05-07 [1] CRAN (R 3.6.2)                       &#xA; tidyselect     1.0.0   2020-01-27 [1] CRAN (R 3.6.0)                       &#xA; usethis      * 1.6.1   2020-04-29 [1] CRAN (R 3.6.2)                       &#xA; vctrs          0.2.4   2020-03-10 [1] CRAN (R 3.6.0)                       &#xA; withr          2.2.0   2020-04-20 [1] CRAN (R 3.6.2)                       &#xA; zip            2.0.4   2019-09-01 [1] CRAN (R 3.6.0)                       &#xA;&#xA;[1] /Library/Frameworks/R.framework/Versions/3.6/Resources/library&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Update: I tried installing llvm, but don't know what put here to finish installing llvm-openmp pc-133-235:build fi1d18&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cmake ../llvm -DLLVM_ENABLE_PROJECTS=openmp -DCMAKE_C_COMPILER=&amp;lt;C compiler&amp;gt; -DCMAKE_CXX_COMPILER=&amp;lt;C++ compiler&amp;gt;&#xA;-bash: syntax error near unexpected token `newline' &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="73" LastEditDate="2021-05-23T13:24:47.963" LastActivityDate="2021-05-23T13:24:47.963" Title="Permanent error in installing R package" Tags="|r|software-installation|software-usage|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13243" PostTypeId="1" CreationDate="2020-05-11T06:07:31.537" Score="2" ViewCount="1163" Body="&lt;p&gt;I know that scaled data must be used for PCA for example as it is based on variance maximization.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However I'm wondering if it's the same case for UMAP ? If the data are single-cell RNA seq, after normalization can we do UMAP and Louvain clustering ? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried both and get similar results, however the Louvain clustering seems to be more adequate on normalized data than on scaled data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;By adequate I mean the clusters are the same but some are split into two, which makes sens looking at other results (like transcription factor analysis). &lt;/p&gt;&#xA;" OwnerUserId="8827" LastEditorUserId="104" LastEditDate="2021-12-21T21:55:32.260" LastActivityDate="2021-12-21T21:55:32.260" Title="umap and Louvain clustering on normalized data" Tags="|scrnaseq|single-cell|umap|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13244" PostTypeId="2" ParentId="13242" CreationDate="2020-05-11T06:13:04.437" Score="1" Body="&lt;p&gt;try &lt;code&gt;devtools::install_github('linxihui/NNLM')&lt;/code&gt;, if it doesn't work you need to install the gfortran binary on Mac. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it still doesn't work, try to upgrade to 4.0.0. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;When you have this kind of trouble, look at the issue page of the github page. You will almost always see that others have had the same issue. &lt;/p&gt;&#xA;" OwnerUserId="8827" LastActivityDate="2020-05-11T06:13:04.437" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13245" PostTypeId="1" AcceptedAnswerId="13248" CreationDate="2020-05-11T06:38:18.293" Score="2" ViewCount="709" Body="&lt;p&gt;Given a FASTQ file, I'd like to generate a new FASTQ file including only certain subsections of the original sequences specified by their positions. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, say I want to extract the nucleotides at position &lt;strong&gt;1, 3, 5-7&lt;/strong&gt; from each sequence:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@id1&#xA;AACCGGTCC&#xA;+&#xA;123456789&#xA;@id2&#xA;TACCGGCCC&#xA;+&#xA;123456789&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@id1&#xA;ACGGT&#xA;+&#xA;13567&#xA;@id2&#xA;TCGGC&#xA;+&#xA;13567&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Note, the phred-scores are constructed such that they correspond to the position for easier reading.&#xA;Note2, the phred-score subsections should be included accordingly (i.e. the same positions)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, a command should work this way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;subselect 1,3,5-7 input.fq &amp;gt; output.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Having looked through several poplar software, I couldn't find anything working out of the box. I was thinking to use either awk or even cutadapt (systematically trimming from the 5&lt;span class=&quot;math-container&quot;&gt;`&lt;/span&gt; and pasting) but maybe out of an box solution does already exist which evaded me. &lt;/p&gt;&#xA;" OwnerUserId="1461" LastActivityDate="2020-05-11T12:20:28.103" Title="Extracting specific sections from FASTQ determined by positions" Tags="|fastq|awk|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13246" PostTypeId="2" ParentId="13245" CreationDate="2020-05-11T08:21:19.313" Score="-1" Body="&lt;p&gt;This looks like a job for samtools fqidx (&lt;a href=&quot;http://www.htslib.org/doc/samtools-fqidx.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.htslib.org/doc/samtools-fqidx.html&lt;/a&gt;), assuming that your file is not enormous... read the warning in the link above.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;usage would be&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;samtools fqidx yourfile.fq &amp;lt;region&amp;gt;&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where region is specified using the fastq record identifier and coordinates.&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2020-05-11T08:21:19.313" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13248" PostTypeId="2" ParentId="13245" CreationDate="2020-05-11T11:28:13.973" Score="1" Body="&lt;p&gt;For the specific regions you mentioned, you could do this in &lt;code&gt;awk&lt;/code&gt;. It won't be fast for a large file, but it will work:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk 'NR%4==2 || NR%4==0{$0=substr($0,1,1)substr($0,3,1)substr($&lt;/span&gt;0,5,3)}1 ' file.fq &#xA;@id1&#xA;ACGGT&#xA;+&#xA;13567&#xA;@id2&#xA;TCGGC&#xA;+&#xA;13567&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;$0&lt;/code&gt; is the current line. The script above will replace the line with three substrings of itself, one starting at position &lt;code&gt;1&lt;/code&gt; with a length of one, one starting at position 3 with a length of one and the last starting at plsition 5 with a length of 3. The &lt;code&gt;NR%4==2 || NR%4==0&lt;/code&gt; ensures that this substitution only occurs on lines whose line number modulo 4 is either 2 or 0, so every 2nd and 4th line of the file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;IMPORTANT&lt;/strong&gt;: this assumes you &lt;em&gt;always&lt;/em&gt; have only one line of sequence in each section. The fastq format allows multiple lines of sequence (and quality scores). So your data might not conform to this. However, for regular short-read data, it should be fine. &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;And here's a slightly more sophisticated version in Perl, with some rudimentary error checking and which can take regions in the format you asked for (&lt;code&gt;1,3,5-7&lt;/code&gt;):&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env perl&#xA;&#xA;if (&lt;span class=&quot;math-container&quot;&gt;$#ARGV != 1) {&#xA;  die &quot;Need exactly 2 arguments.\n&quot;;&#xA;}&#xA;my $&lt;/span&gt;file = $ARGV[0];&#xA;&#xA;my @regions = split(/,/,&lt;span class=&quot;math-container&quot;&gt;$ARGV[1]);&#xA;my (@startPositions, @lengths);&#xA;for my $&lt;/span&gt;arg (0..&lt;span class=&quot;math-container&quot;&gt;$#regions) {&#xA;  if ($regions[$arg] =~ /^\d+$/) {&#xA;    push @startPositions, $regions[$arg];&#xA;  }&#xA;  elsif ($regions[$arg] =~ /^(\d+)-(\d+)$/) {&#xA;    for my $i ($1..$2) {&#xA;        push @startPositions, $&lt;/span&gt;i;&#xA;    }&#xA;  }&#xA;}&#xA;&#xA;open(my &lt;span class=&quot;math-container&quot;&gt;$inputFileHandle, '&amp;lt;', $&lt;/span&gt;file) or&#xA;    die &quot;Failed to open &lt;span class=&quot;math-container&quot;&gt;$file for reading: $&lt;/span&gt;!\n&quot;;&#xA;&#xA;while (&amp;lt;&lt;span class=&quot;math-container&quot;&gt;$inputFileHandle&amp;gt;) {&#xA;  if ($. % 4 == 2 || $. % 4 == 0) {&#xA;    my $line = &quot;&quot;;&#xA;    foreach my $start (@startPositions) {&#xA;      $line .= substr($_, $start-1, 1)&#xA;    }&#xA;    print &quot;$line\n&quot;&#xA;  }&#xA;  else {&#xA;    print&#xA;  }&#xA;  ## If we find a line that consists only of '+' but its line number&#xA;  ## modulo 4 isn't 3, that means that there was at least one multi-line&#xA;  ## sequence that we cannot handle correctly.&#xA;  if (/^\+$/ &amp;amp;&amp;amp; $. % 4 != 3) {&#xA;    die &quot;Line $&lt;/span&gt;. is just a '+', multi-line sequence found in file.\n&quot;&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Save that as &lt;code&gt;foo.pl&lt;/code&gt; and then run like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perl foo.pl file.fq 1,3,5-7&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2020-05-11T12:20:28.103" LastActivityDate="2020-05-11T12:20:28.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13249" PostTypeId="2" ParentId="11299" CreationDate="2020-05-11T11:42:01.077" Score="2" Body="&lt;p&gt;One potential problem here is that, as far as I can see, you're not taking the hierarchical structure of your scRNA-seq data into account. I.e. the fact that cells from the same patient / sample will be more similar than cells from different samples, even within the same group. This is also the problem addressed in the paper mentioned by @alan. Also, it usually makes more sense to do this kind of group vs. group comparison &lt;strong&gt;for each subpopulation (cluster/cell type) separately&lt;/strong&gt; (assuming you have multiple subpopulations in your data and did some kind of clustering).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way of doing this is by using a &lt;strong&gt;pseudobulk-level approach&lt;/strong&gt;, where you basically aggregate (by either averaging or summing) your data over each sample, for each cluster separately. So you end up with one counts matrix per subpopulation with one row for each gene and one column for each &lt;strong&gt;sample&lt;/strong&gt;. The sample columns are then the sums (or averages) of all cells belonging to that cluster-sample combination.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can then use bulk-RNAseq methods such as &lt;strong&gt;edgeR&lt;/strong&gt; or &lt;strong&gt;DESeq2&lt;/strong&gt; on the aggregated data to test differences between groups of samples for each cluster. The &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/muscat.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;muscat&lt;/a&gt; package implements all these steps and has quite extensive documentation. So you could check that and test it out on your data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The paper describing the method of &lt;em&gt;muscat&lt;/em&gt; can be found here: &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/713412v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/10.1101/713412v2&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8777" LastActivityDate="2020-05-11T11:42:01.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13250" PostTypeId="1" CreationDate="2020-05-11T13:32:26.120" Score="2" ViewCount="95" Body="&lt;p&gt;The attached figure (from deseq2) shows the sample to sample distances of a RNAseq experiment with 4 conditions (A,B,C,D) at 2 time points (0h,4h) with 2 biological replicates.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am a little bit puzzled that the biological replicates are not always the most similar pairs. In a perfect world biological replicates should be identical but I assumed that at least they should be very similar.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is this something I have to worry about, or is this more or less normal? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rPOxY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rPOxY.png&quot; alt=&quot;deseq2 sample2sample distance matrix&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8830" LastEditorUserId="3541" LastEditDate="2020-05-11T16:13:47.580" LastActivityDate="2020-05-13T14:53:22.023" Title="Should biological replicates be the most similar pairs in an RNAseq experiment?" Tags="|rna-seq|deseq2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13251" PostTypeId="2" ParentId="13250" CreationDate="2020-05-11T13:49:42.600" Score="1" Body="&lt;p&gt;In summary you basically have noisy data and to simply answer your question it depends on the genetic diversity of the samples you are dealing with. Note, from my recollection this was confusing to start with because the &lt;em&gt;&quot;replicates&quot;&lt;/em&gt; column was labeled as &lt;em&gt;&quot;treatment&quot;&lt;/em&gt;, but I could be wrong.  &lt;/p&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Revised answer, firstly there is zero stationarity - thus its a straight time-series.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; As a time series &lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The statistical noise between A and D is lower than the difference in phenotypic expression between each and all othe replicates in the analysis ABCD - FOR DAY 0 VERSUS DAY &#xA;4, so they cluster and are distinct. &lt;/li&gt;&#xA;&lt;li&gt;The statistical noise between B and C is greater than the phenotypic variation between their expression profiles for DAY 0 to DAY 4. I suspect they are genetically very similar. Thats the conclusion, you can spice it up with different analysis, but I'd leave it because might find a lack of statistical significance in your core observations (A and D).&lt;/li&gt;&#xA;&lt;li&gt;Within the time-points the data is very noisy and doesn't appear to show a distinct signal. Thats okay, this is essentially a time series.&lt;/li&gt;&#xA;&lt;li&gt;Sample D is distinct in all scenarios (which is good)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Result&lt;/strong&gt; To reiterate and expand a bit, the result you have is that sample A and D biological replicates are consisent within the analysis you are performing, i.e. day 0 verses day 4 (below). Thus they cluster together and this is proof of distinct phenotypes above the statistical noise (A with A and D with D). An intra-time point dendrogram would have a different result. The replicates samples B and C are not consistent and the interpretation is that the phenotypic expression profiles are so similar they are essentially identical within the boundaries of statistical noise. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Analysis&lt;/strong&gt; The dendrogram is combines day 0 to day 4 analysis with day 0 to day 0 analysis and day 4 to day 4 analysis. HOWEVER, the clustering algorithm will correct for this and separate based on the largest distance (day 0 versus day 4). The signal that you get in your dendrogram is the largest signal in the data set which is day 0 vs day 4, and this is how UPGMA clustering works (I probably have previous posts on the statistical algorithm of UPGMA). So actually that is fine.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Advice&lt;/strong&gt; With data this noisy I'd go no further in attempting to deploy critical probablity (a &lt;em&gt;p&lt;/em&gt; value)  because this data could be the result of a complex distribution. This would require a statistician. Personally &#xA; I'd upload the PCA as a separate post, but its just my opinion, because UPGMA versus eigenvalue based PCA are statistically quite different approaches. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-05-13T14:53:22.023" LastActivityDate="2020-05-13T14:53:22.023" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13252" PostTypeId="1" AcceptedAnswerId="13262" CreationDate="2020-05-11T14:48:09.737" Score="1" ViewCount="6357" Body="&lt;p&gt;I want to use a local &lt;code&gt;blastn&lt;/code&gt; command to BLAST a multifasta file ~700 sequences using following command: &lt;code&gt;blastn -db nt -query fasta_all.fasta -num_alignments 2 -out fasta_blasted.txt&lt;/code&gt; but I receive error :&lt;code&gt;BLAST Database error: Error: Not a valid version 4 database.&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I use a local &lt;code&gt;nt&lt;/code&gt; database which I downloaded today (11th of May 2020) by running &lt;code&gt;update_blastdb.pl --decompress nt [*]&lt;/code&gt;. I downloaded this but I did not do any post-processing and just the &quot;raw&quot; download files are in the folder (see image below which shows only a part of the files)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uGn4U.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uGn4U.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;code&gt;blast+&lt;/code&gt; I downloaded and installed is &lt;code&gt;ncbi-blast-2.10.0+-x64-linux.tar.gz&lt;/code&gt; from &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So, both database and &lt;code&gt;blastn&lt;/code&gt; version should be the latest. It probably has something to do with either the version of the database or the software, but as it both are the latest version, I find it strange that these would not be compatible. Any ideas what causes this error?&lt;/p&gt;&#xA;" OwnerUserId="7476" LastActivityDate="2022-06-17T06:58:20.227" Title="local BLAST error: BLAST Database error: Error: Not a valid version 4 database" Tags="|database|public-databases|blastn|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13253" PostTypeId="1" CreationDate="2020-05-11T16:38:26.967" Score="0" ViewCount="30" Body="&lt;p&gt;I would like to generate a list of gene markers that represent post-synaptic related genes. I will then use this list to search through single cell sequencing data, in order to assess whether certain cell types express post-synaptic related genes. I think there might be a way to get this list of gene markers from NCBI. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="8610" LastActivityDate="2020-05-11T16:38:26.967" Title="How do I generate a list of post-synaptic gene markers that will guide me in my search for these markers in intestinal stem cells?" Tags="|scrnaseq|gene|phylogenetics|gene-expression|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13254" PostTypeId="1" CreationDate="2020-05-11T20:59:32.313" Score="1" ViewCount="436" Body="&lt;p&gt;What are some good tools (in Python if possible) for creating gene co-expression networks from RNA seq data?&lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2020-05-12T07:10:29.800" Title="Tools for creating gene coexpression networks" Tags="|python|gene-expression|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13255" PostTypeId="1" CreationDate="2020-05-11T21:46:53.123" Score="3" ViewCount="920" Body="&lt;p&gt;I have a file contain multiple sequence, and I want to separate them by &amp;quot;gene:&amp;quot; into different file.&#xA;example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;example.fa&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;KQK21959&#xA;ATGGTGCTCGACCCCGTCGTCGGGGGCTCGGCCTCCCGCAAGGTCGAGCCCCTAACCCTA&#xA;&#xA;&amp;gt;KQK14546&#xA;ATGCCTCCGCCGGACGCGGCCACCTACGCTCCCACGCTGCCAATGGCGACGTACCTCCCA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How to get results like this? One sequence per file, and after &lt;code&gt;&amp;gt;&lt;/code&gt; is gene id, not cds id&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;KQK21959.fa&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;KQK21959&#xA;ATGGTGCTCGACCCCGTCGTCGGGGGCTCGGCCTCCCGCAAGGTCGAGCCCCTAACCCTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;KQK14546.fa&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;KQK14546&#xA;ATGCCTCCGCCGGACGCGGCCACCTACGCTCCCACGCTGCCAATGGCGACGTACCTCCCA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8692" LastEditorUserId="650" LastEditDate="2023-08-26T20:59:28.687" LastActivityDate="2023-09-07T02:48:32.470" Title="Separate multiple sequence into different file, one sequence per file" Tags="|fasta|ngs|gene|" AnswerCount="6" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13256" PostTypeId="2" ParentId="13254" CreationDate="2020-05-11T23:39:56.903" Score="1" Body="&lt;p&gt;After looking around, it seems like the &lt;a href=&quot;https://camoco.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Camoco library&lt;/a&gt; is suitable. &lt;/p&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="8601" LastEditDate="2020-05-12T00:43:32.157" LastActivityDate="2020-05-12T00:43:32.157" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13257" PostTypeId="2" ParentId="13255" CreationDate="2020-05-12T00:34:08.723" Score="2" Body="&lt;p&gt;That really depends on the tools you are using. If anything will do you can use the following python snippet.&lt;/p&gt;&#xA;&lt;p&gt;If you know that each sequence is on a single line and your file isnt gigantic use this&lt;/p&gt;&#xA;&lt;h1&gt;EDIT&lt;/h1&gt;&#xA;&lt;p&gt;Updated so the file isn't read to memory at once to address @terdon point. Although, the above solutions are probably much better.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;with open( &amp;quot;seqs.fasta&amp;quot; ) as f:&#xA;    out = &amp;quot;&amp;quot;&#xA;    &#xA;    for line in f:&#xA;        if not line.strip(): continue &#xA;        if line.startswith( &amp;quot;&amp;gt;&amp;quot; ):&#xA;            if not type( out ) == str: out.close()&#xA;            gene = line[1:].strip().split()[0]&#xA;            out = open( f&amp;quot;{gene}.fa&amp;quot;, &amp;quot;w&amp;quot; )&#xA;            out.write( line )&#xA;            continue &#xA;        out.write( line.strip() )&#xA;    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8795" LastEditorUserId="650" LastEditDate="2023-08-26T21:00:11.907" LastActivityDate="2023-08-26T21:00:11.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13258" PostTypeId="1" CreationDate="2020-05-12T04:30:53.270" Score="0" ViewCount="844" Body="&lt;p&gt;In order to perform an integration analysis of single cell RNA seq data, is it important to check the percentage expression of mitochondrial genes of cells as well as the feature counts to exclude dying or dead cells as well as multiplets from further analyses?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ask because the standard Seurat work flow to perform integration as outlined in &lt;a href=&quot;http://www.satijalab.org/seurat/v3.0/integration.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Integration and Label Transfer&lt;/a&gt; does not mention filtering steps prior to integration.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In this publication, &lt;a href=&quot;https://ard.bmj.com/content/78/10/1379&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eleanor Alenzi et al. 2018&lt;/a&gt; also did a canonical correlation analysis with Seurat but did not mention steps to filter out poor quality cells before doing the integrated analysis. This second &lt;a href=&quot;https://www.atsjournals.org/doi/full/10.1164/rccm.201712-2410OC&quot; rel=&quot;nofollow noreferrer&quot;&gt;publication&lt;/a&gt; does same. Both publications do not mention if their visualisations or differential gene expression analyses are done on integrated (discouraged) or raw uncorrected RNA data (recommended).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Following the styles of the publications linked above (many more do same), is it also efficient to perform integration without filtering out poor quality cells?&lt;/p&gt;&#xA;" OwnerUserId="3968" LastEditorUserId="3968" LastEditDate="2020-05-12T07:26:36.840" LastActivityDate="2020-05-12T07:35:56.227" Title="Is it important to filter out poor quality cells before performing an integration analysis on single cell RNA sequencing data?" Tags="|scrnaseq|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13259" PostTypeId="1" AcceptedAnswerId="13260" CreationDate="2020-05-12T05:40:19.043" Score="0" ViewCount="61" Body="&lt;p&gt;I am trying to understand expression of a certain protein across Pseudomonas species. I downloaded an SRA file from NCBI and converted it to a fastaq file. I am not able to understand how to interpret and analyse this file. I want to get the mRNA sequence and see which genes are being expressed and find the mRNA of interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How do I proceed with getting the mRNA sequences and then for mapping them to genome?&lt;/p&gt;&#xA;" OwnerUserId="8767" LastActivityDate="2020-05-12T06:58:16.857" Title="How to understand and analyse RNA-seq data (for a beginner)?" Tags="|rna-seq|rna|transcriptome|gene-expression|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13260" PostTypeId="2" ParentId="13259" CreationDate="2020-05-12T06:58:16.857" Score="1" Body="&lt;p&gt;SRA files are raw sequencing data that have been converted to the NCBI storage format. These can be converted to fastq files, e.g. with fastq-dump from the SRAtoolkit. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Beyond that I suggest you start with the &lt;a href=&quot;https://peerj.com/preprints/27283/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hitchhiker's GUide to RNA-seq&lt;/a&gt; and the &lt;a href=&quot;http://bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2 workflow&lt;/a&gt; to get a background. Be sure to extensively browse the web for vignettes, guides and tutorials. YOu will find helpful resources to get started.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-05-12T06:58:16.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13261" PostTypeId="2" ParentId="13254" CreationDate="2020-05-12T07:10:29.800" Score="1" Body="&lt;p&gt;it depends on the size of your data. If you have enough RAM available, then you may directly compute a correlation-based distance matrix using the &lt;em&gt;cor()&lt;/em&gt; function (with method=&quot;spearman&quot; or method=&quot;pearson&quot;). But you may also give a try to Mutual Information distance using the &lt;strong&gt;Parmigene&lt;/strong&gt; (&lt;a href=&quot;https://cran.r-project.org/web/packages/parmigene/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cran.r-project.org/web/packages/parmigene/index.html&lt;/a&gt;) R package or partial correlations with the &lt;strong&gt;Corpcor&lt;/strong&gt; (&lt;a href=&quot;https://cran.r-project.org/web/packages/corpcor/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cran.r-project.org/web/packages/corpcor/index.html&lt;/a&gt;) R package.&#xA;If you have a large expression matrix, remember to use the excellent &lt;strong&gt;data.table&lt;/strong&gt; package and its &lt;em&gt;fread()&lt;/em&gt; function.&lt;/p&gt;&#xA;" OwnerUserId="8749" LastActivityDate="2020-05-12T07:10:29.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13262" PostTypeId="2" ParentId="13252" CreationDate="2020-05-12T07:14:23.423" Score="2" Body="&lt;p&gt;first I recommend you to check your downloaded data with md5 to be sure there was no problem with that step. How many files did you download?&lt;/p&gt;&#xA;&lt;p&gt;Also I was curious if you had another BLAST+ version installed on your computer. Be sure that blastn is called is a version corresponding to the database you are using.&lt;/p&gt;&#xA;" OwnerUserId="8749" LastEditorUserId="57" LastEditDate="2021-03-25T11:08:35.863" LastActivityDate="2021-03-25T11:08:35.863" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13263" PostTypeId="2" ParentId="13258" CreationDate="2020-05-12T07:35:56.227" Score="4" Body="&lt;p&gt;It is absolutely necessary to remove low quality cells: In the case of &lt;code&gt;CCA&lt;/code&gt; (and this applies to other &quot;integration&quot; or &quot;data alignment&quot; methods as well), one would need to use &quot;anchors&quot;, basically same type/state/kind of cells from the samples to be &quot;integrated&quot; and are used to &quot;align&quot; the different samples. In a scenario where you have a set of cells whose distinguished feature is high mitochondrial content due to apoptosis irrespective of their origin (epithelial, T cell, ...) and such cells are picked up as so called anchors, the whole integration effort will be off. Even when not using integration methods, bad quality cells quite often form their own clusters (highly skewed biology) not contributing anything to analyses and are often removed at the downstream steps anyway.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And when it comes to published work, unfortunately it comes down to the backgrounds of the reviewers. I tend to think that there are way too many single cell studies than experts with required bioinformatics and statistics so that a thorough review can be conducted. And lack of a paragraph on QC does not mean no QC has been performed anyway.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-05-12T07:35:56.227" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13264" PostTypeId="2" ParentId="13255" CreationDate="2020-05-12T08:44:50.340" Score="3" Body="&lt;p&gt;A simple Biopython solution:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;for record in SeqIO.parse(&amp;quot;example.fa&amp;quot;, &amp;quot;fasta&amp;quot;):&#xA;    SeqIO.write(record, record.id + &amp;quot;.fa&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note, if you have multiple sequences with the same id, only the last sequence will be kept in the final output files&lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="650" LastEditDate="2023-08-26T20:59:40.537" LastActivityDate="2023-08-26T20:59:40.537" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13265" PostTypeId="1" CreationDate="2020-05-12T08:51:59.673" Score="0" ViewCount="104" Body="&lt;p&gt;I'm trying to figure out if I should be filtering out GWAS hits that have high standard error and I'm not quite sure what to do.  It seems like it might not matter, because the standard error is used to calculate the t-statistic, which is then used to calculate the p-value.  So in a way it's already built in.  But reporting SNPs that have very high standard error doesn't seem quite right.  What's the best way to handle this?&lt;/p&gt;&#xA;" OwnerUserId="59" LastActivityDate="2020-05-12T14:45:36.173" Title="Should you filter GWAS hits with high standard error?" Tags="|statistics|gwas|genetics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13266" PostTypeId="2" ParentId="13243" CreationDate="2020-05-12T09:03:56.097" Score="3" Body="&lt;blockquote&gt;&#xA;  &lt;p&gt;If the data are single-cell RNA seq, after normalization can we do&#xA;  UMAP and Louvain clustering ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Typically people run PCA, UMAP and Louvain clustering on the normalised and log-transformed expression counts (but do marker gene and differential expression analysis on the non-normalised values). However, this remains controversial. I recommend reading &lt;a href=&quot;https://www.embopress.org/doi/10.15252/msb.20188746&quot; rel=&quot;nofollow noreferrer&quot;&gt;&quot;Current best practices in single‐cell RNA‐seq analysis: a tutorial&quot;&lt;/a&gt;- it's a bit outdated, but still a good reference.&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;By adequate I mean the clusters are the same but some are split into&#xA;  two, which makes sens looking at other results (like transcription&#xA;  factor analysis).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;The number of (or size of) clusters should not be used to interpret how good the clustering methodology is, since it is generally dependant on the input parameters, e.g. a higher resolution parameter leads to more clusters. Instead, display the clusters on the UMAP and see if they are continuous 'visually' and check what genes are differentially expressed between clusters and see if they make sense 'biologically'.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2020-05-12T09:03:56.097" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13267" PostTypeId="2" ParentId="13243" CreationDate="2020-05-12T09:06:14.970" Score="2" Body="&lt;p&gt;There is &lt;a href=&quot;https://www.embopress.org/doi/10.15252/msb.20188746&quot; rel=&quot;nofollow noreferrer&quot;&gt;discussion&lt;/a&gt; if scaling the data (making the data range for all genes the same) is something you need for single cell data. The argument is that the expression differences between genes themselves are informative. Normalization on the other hand is always necessary.&lt;br&gt;&#xA;The big single cell pipelines like &lt;code&gt;Seurat&lt;/code&gt; or &lt;code&gt;Monocle&lt;/code&gt; use both normalization and scaling as standard.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would like some of the clusters you get split into subclusters you could either increase the resolution parameter that the clustering functions usually have. Or you can take the cells from a cluster, then subcluster them and add the subclusters to the original clustering results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For yor next question it would be great if you could provide the plots of the data because this way it is easier to understand what the problem is. &lt;/p&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2020-05-12T09:06:14.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13268" PostTypeId="2" ParentId="13211" CreationDate="2020-05-12T12:36:19.373" Score="1" Body="&lt;p&gt;You may run edgeR for methylation analysis without replicates (&lt;a href=&quot;https://f1000research.com/articles/6-2055&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://f1000research.com/articles/6-2055&lt;/a&gt;). But I also recommend you to have a look at the R DSS package (&lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/DSS.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://bioconductor.org/packages/release/bioc/html/DSS.html&lt;/a&gt;). It has a smoothing step which allows to modelize the biological variability by taking information from neighboring sites.&lt;/p&gt;&#xA;" OwnerUserId="8749" LastActivityDate="2020-05-12T12:36:19.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13269" PostTypeId="2" ParentId="13255" CreationDate="2020-05-12T12:49:55.047" Score="2" Body="&lt;p&gt;This can also be done using awk. This is harder to read, but probably faster than python:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;awk 'sub(/^&amp;gt;/,&amp;quot;&amp;quot;){&#xA;        FILE=&lt;span class=&quot;math-container&quot;&gt;$0&quot;.fa&quot;; &#xA;    print &quot;&amp;gt;&quot;$&lt;/span&gt;0 &amp;gt;FILE; &#xA;        next&#xA;    }{print &amp;gt; FILE}' file.fa. &#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will define a new output file &lt;code&gt;FILE&lt;/code&gt; at each header line, whose filename is the header minus the &lt;code&gt;&amp;gt;&lt;/code&gt; character, suffixed with &lt;code&gt;.fa&lt;/code&gt;. And all lines in the file are appended to the current output file.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: Changed to a faster version found on: &lt;a href=&quot;https://crashcourse.housegordon.org/split-fasta-files.html#using-awk-instead-of-csplitsed&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://crashcourse.housegordon.org/split-fasta-files.html#using-awk-instead-of-csplitsed&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="439" LastEditorUserId="650" LastEditDate="2023-08-26T20:59:58.800" LastActivityDate="2023-08-26T20:59:58.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13270" PostTypeId="2" ParentId="13160" CreationDate="2020-05-12T13:26:43.013" Score="-1" Body="&lt;p&gt;If you are working under a Linux environment, then you may easily use a piped grep search. For example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep deleterious file.vcf |grep damaging &amp;gt;filtered_file.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8749" LastActivityDate="2020-05-12T13:26:43.013" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13271" PostTypeId="2" ParentId="13265" CreationDate="2020-05-12T13:50:39.837" Score="1" Body="&lt;p&gt;I am curious about the relationship between your p-value (or effect sizes) and standard error. I would expect the significant signals to have smaller stand error compared to the non-significant, background signals. If this is the case, there is no need to do filter.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-05-12T13:50:39.837" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13272" PostTypeId="2" ParentId="13265" CreationDate="2020-05-12T14:38:39.467" Score="0" Body="&lt;p&gt;I think the effect size is more of an issue than the standard error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the standard error suggests that the effect direction could change sign, then it might be a good idea to filter it out. Otherwise, if the effect size remains large (or at least positive) even after accounting for a large error, then [to me] that would attach more weight to the likelihood that the association is valid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did a very quick analysis of the results of one UK Biobank study on Twitter, where I used the statistic of &quot;the least extreme value in a 95% confidence interval&quot; in a Manhattan plot instead of the more commonly-used p-value:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://twitter.com/gringene_bio/status/1207617723586371584&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://twitter.com/gringene_bio/status/1207617723586371584&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-05-12T14:45:36.173" LastActivityDate="2020-05-12T14:45:36.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13274" PostTypeId="2" ParentId="13255" CreationDate="2020-05-12T16:39:13.607" Score="3" Body="&lt;p&gt;There's an app for that! The &lt;code&gt;exonerate&lt;/code&gt; package (see &lt;a href=&quot;https://www.ebi.ac.uk/about/vertebrate-genomics/software/exonerate&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) includes a nice &lt;code&gt;fastaexplode&lt;/code&gt; command that does exactly what you're looking for and will be much, much faster than anything we give you here since it is written in C and not a scripting language. Just install &lt;code&gt;exonerate&lt;/code&gt; (you can do that with &lt;code&gt;sudo apt install exonerate&lt;/code&gt; on Debian-based systems) and then:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fastaexplode file.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will create these two files (&lt;code&gt;file.fa&lt;/code&gt; is the original input file from your question):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ ls&#xA;file.fa  KQK14546.fa  KQK21959.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="650" LastEditDate="2023-08-26T20:59:50.763" LastActivityDate="2023-08-26T20:59:50.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13275" PostTypeId="1" AcceptedAnswerId="13276" CreationDate="2020-05-12T19:01:59.237" Score="1" ViewCount="293" Body="&lt;p&gt;Is there a way to generate a WGCNA co-expression graph in R, and then to import that graph into Python such that I can represent it using a &lt;a href=&quot;https://networkx.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;networkX&lt;/a&gt; object? &lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2020-05-12T20:17:44.227" Title="Importing a WGCNA co-expression network into a networkX graph in Python" Tags="|r|python|networks|wgcna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13276" PostTypeId="2" ParentId="13275" CreationDate="2020-05-12T20:17:44.227" Score="2" Body="&lt;p&gt;You can use &lt;code&gt;exportNetworkToCytoscape&lt;/code&gt; or &lt;code&gt;exportNetworkToVisANT&lt;/code&gt; to generate and egde list file with the two nodes at the ends and weight for each edge. Hopefully there's a way to import that into networkX. &lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2020-05-12T20:17:44.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13277" PostTypeId="1" CreationDate="2020-05-13T02:16:24.547" Score="1" ViewCount="132" Body="&lt;p&gt;I have the CDS sequences for my genome. I want to align Illumina DNA seq reads from related fungal strains to these. This is to identify SNPs in just the coding sequences. I am not sure if I should use a DNA seq aligner like BWA and Bowtie, or to use a split aware aligner like STAR.&lt;/p&gt;&#xA;&lt;p&gt;I have the full genome. But I wanted the SNPs for each of the CDS sequences, so I can create consensus sequences for these CDS for the related strains.&lt;/p&gt;&#xA;&lt;p&gt;Can you provide examples or links to get alignment from CDS region or using VCFtools to get SNPs and INDELs from the CDS region? I have tried looking but haven't found anything useful. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="8844" LastEditorUserId="73" LastEditDate="2021-07-05T04:24:57.490" LastActivityDate="2021-07-05T04:27:10.650" Title="How to align Illumina DNA seq reads to CDS?" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13278" PostTypeId="1" CreationDate="2020-05-13T14:28:25.040" Score="1" ViewCount="37" Body="&lt;p&gt;I've recently discovered &lt;a href=&quot;https://orthovenn2.bioinfotoolkits.net/home&quot; rel=&quot;nofollow noreferrer&quot;&gt;OrthoVenn2&lt;/a&gt;, which I'm using to compare proteomes and extract likely clade-specific genes. I'm enjoying software, and one of the things it offers is a breakdown of protein clusters in terms of Gene Ontology (&lt;a href=&quot;https://orthovenn2.bioinfotoolkits.net/overlap/c7ae2470acac3cd894aae9c9d5ee1b88/jvennOverlap/ortho?bin=RnVzYXJpdW1fcHNldWRvZ3JhbWluZWFydW0&amp;amp;sp=RnVzYXJpdW1fb3h5c3BvcnVtLEZ1c2FyaXVtX3BzZXVkb2dyYW1pbmVhcnVt&quot; rel=&quot;nofollow noreferrer&quot;&gt;example from a mock dataset&lt;/a&gt;). However, I find the Gene Ontology terms a bit too general, and would appreciate more specificity for my data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I guess something more like the &lt;a href=&quot;https://orthovenn2.bioinfotoolkits.net/overlap/c7ae2470acac3cd894aae9c9d5ee1b88/jvennOverlap/ortho?bin=RnVzYXJpdW1fcHNldWRvZ3JhbWluZWFydW0&amp;amp;sp=RnVzYXJpdW1fb3h5c3BvcnVtLEZ1c2FyaXVtX3BzZXVkb2dyYW1pbmVhcnVt&quot; rel=&quot;nofollow noreferrer&quot;&gt;'Subsystems' figure on MG-Rast datasets&lt;/a&gt;? In an ideal case, I'd enjoy something interactive where I could explore different levels of annotation (e.g. if I click on 'transferase' in the pie-chart, it would be nice to see 'methyl-trasferase', 'formyl-trasferase' etc), but I understand that might not exist and I'd be happy with something that requires a bit of command line usage too. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm working with a relatively medium number of genes (50-400) at one time.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for your time!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastEditorUserId="298" LastEditDate="2020-05-13T15:37:42.807" LastActivityDate="2020-05-13T15:37:42.807" Title="Gene Ontology equivalent - higher specificity?" Tags="|software-recommendation|gene-ontology|functional-annotation|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13279" PostTypeId="1" AcceptedAnswerId="13281" CreationDate="2020-05-13T14:33:49.507" Score="2" ViewCount="51" Body="&lt;p&gt;I am performing a RQL analysis (&quot;&lt;a href=&quot;http://www.esapubs.org/archive/ecol/E095/002/suppl-1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.esapubs.org/archive/ecol/E095/002/suppl-1.pdf&lt;/a&gt;&quot;). &#xA;This analysis meassures the relationship between different traits of three different matrix showing if there is a positive or negative relationship. This RQL analysis make a fourth-corner method. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Well, this method have the option to  make a p-adjust method to solve multiple comparison problem. However, my variables are independent and I belive that I should not apply to my dataset. Am I right? Or always that two or more matrix are compared I should apply a multiple comparison adjustment?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="7430" LastActivityDate="2020-05-13T16:44:01.040" Title="When should I apply a multiple comparison adjustment?" Tags="|statistics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13281" PostTypeId="2" ParentId="13279" CreationDate="2020-05-13T16:36:05.350" Score="3" Body="&lt;p&gt;The p-value adjustments corrects the inflated number of significant results when performing many significance tests.&lt;br&gt;&#xA;If you have a test with 5 % false positive rate and but you run it several times the chances of getting a false positive results in any of these tests is far greater than 5 %. &lt;a href=&quot;https://www.statisticshowto.com/false-discovery-rate/&quot; rel=&quot;nofollow noreferrer&quot;&gt;For example&lt;/a&gt; repeating the test with 5 % false positive rate 20 times means that there is a 60 % chance of at least one false positive result.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see this in the tutorial you have linked to in your question, where the number of significant differences goes down from 26 without correction to 18 with correction.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Therefore, if you are performing multiple comparisons you should do a correction. The problem of inflated p-values will occur when you are performing many tests even if the tested variables are independent.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastEditorUserId="298" LastEditDate="2020-05-13T16:44:01.040" LastActivityDate="2020-05-13T16:44:01.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13282" PostTypeId="1" CreationDate="2020-05-13T22:38:28.017" Score="0" ViewCount="1193" Body="&lt;p&gt;I'm trying to use vcf-merge to combine 2 exome capture vcf files (~250K and ~330K in size) before trying it on all 96 samples. I'd appreciate any advice on the best way to do that! I've detailed what I've tried below. My issue seems to be with using tabix to convert the files to .tbi format.&lt;/p&gt;&#xA;&lt;p&gt;Step 1: BGZIP&lt;/p&gt;&#xA;&lt;p&gt;So far, I've zipped the files without issue:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bgzip sample1.vcf&#xA;bgzip sample2.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which produces:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sample1.vcf.gz&#xA;sample2.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Step 2: TABIX&lt;/p&gt;&#xA;&lt;p&gt;When I try to use this command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tabix -h -p vcf sample1.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;the stderr is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Region 536999277..536999278 cannot be stored in a tbi index. Try using&#xA;a csi index with min_shift = 14, n_lvls &amp;gt;= 6. tbx_index_build failed:&#xA;sample1.vcf.gz&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Using the -C option which works:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tabix -C -h -p vcf sample1.vcf.gz&#xA;tabix -C -h -p vcf sample2.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which produces:&#xA;sample1.vcf.gz.csi&#xA;sample2.vcf.gz.csi&lt;/p&gt;&#xA;&lt;p&gt;Step 3: VCF-MERGE&lt;/p&gt;&#xA;&lt;p&gt;When I use this command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;vcf-merge sample1.vcf.gz.csi sample2.vcf.gz.csi &amp;gt; out.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;the merge fails, and I get this stderr:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Broken VCF header, no column names? at /usr/share/perl5/Vcf.pm line&#xA;172, &amp;lt;&lt;strong&gt;ANONIO&lt;/strong&gt;&amp;gt; line 1. Vcf::throw(Vcf4_2=HASH(0x5645a760bc38),&#xA;&amp;quot;Broken VCF header, no column names?&amp;quot;) called at&#xA;/usr/share/perl5/Vcf.pm line 867&#xA;VcfReader::_read_column_names(Vcf4_2=HASH(0x5645a760bc38)) called at&#xA;/usr/share/perl5/Vcf.pm line 602&#xA;VcfReader::parse_header(Vcf4_2=HASH(0x5645a760bc38)) called at&#xA;/usr/bin/vcf-merge line 183 main::init_cols(HASH(0x5645a761f438),&#xA;Vcf4_2=HASH(0x5645a760b248)) called at /usr/bin/vcf-merge line 279&#xA;main::merge_vcf_files(HASH(0x5645a761f438)) called at&#xA;/usr/bin/vcf-merge line 12&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If I exclude the tabix files, the merge still fails and the stderr says:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The column names not tab-separated? Could not load .tbi index of&#xA;sample1.vcf.gz. The command exited with an error. Is the file tabix&#xA;indexed?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Any suggestions on how I can correctly merge these files?&lt;/p&gt;&#xA;" OwnerUserId="8850" LastEditorUserId="3694" LastEditDate="2022-02-05T12:55:58.507" LastActivityDate="2022-02-07T11:10:41.563" Title="VCF-merge fails due to tabix not producing .tbi files" Tags="|vcf|linux|vcftools|tabix|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13283" PostTypeId="2" ParentId="13282" CreationDate="2020-05-13T23:08:08.410" Score="2" Body="&lt;p&gt;You should go with the CSI index as it seems like the BAI indexes cannot handle chromosomes longer than 2^29.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Your problem is that you're trying to merge the index files using &lt;code&gt;vcf-merge&lt;/code&gt;. You should merge the VCF files, not the &lt;code&gt;.csi&lt;/code&gt; index files.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2020-05-13T23:08:08.410" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13284" PostTypeId="2" ParentId="11017" CreationDate="2020-05-14T12:34:17.913" Score="1" Body="&lt;p&gt;Try this:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(Seurat)&#xA;pbmc_small &amp;lt;- RunUMAP(pbmc_small, dims = 1:10, reduction = &quot;pca&quot;, n.components = 5)&#xA;&#xA;names(pbmc_small[[&quot;umap&quot;]])&#xA;#&amp;gt; [1] &quot;UMAP_1&quot; &quot;UMAP_2&quot; &quot;UMAP_3&quot; &quot;UMAP_4&quot; &quot;UMAP_5&quot;&#xA;&#xA;# plot&#xA;DimPlot(pbmc_small, reduction = &quot;umap&quot;, dims = c(4, 5))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/z3Vfd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/z3Vfd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This should work with Seurat version 3.1.5&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;packageVersion(&quot;Seurat&quot;)&#xA;#&amp;gt; [1] '3.1.5'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8777" LastActivityDate="2020-05-14T12:34:17.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13285" PostTypeId="2" ParentId="11431" CreationDate="2020-05-14T17:57:11.007" Score="2" Body="&lt;p&gt;RepeatMasker comes with a buildSummary.pl script.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can input the .out file from your RepeatMasker run and there is a flag to enter your species&lt;/p&gt;&#xA;&#xA;&lt;p&gt;e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;buildSummary.pl -species Vertebrates rmrun.out &amp;gt; mysummary.tbl&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In the mysummary.tbl, the summary will not assume your run to be &lt;em&gt;Homo&lt;/em&gt; (default).&lt;/p&gt;&#xA;" OwnerUserId="2893" LastActivityDate="2020-05-14T17:57:11.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13286" PostTypeId="1" CreationDate="2020-05-14T22:27:28.280" Score="0" ViewCount="91" Body="&lt;p&gt;I am struggling with using NCBI's standalone blast, particularly the blastn feature.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have a large number of nucleotide sequences that I want to use blastn to ID. The goal is to retrieve information from the blast program and eventually import this information to excel. I want to know the capabilities of, or the easiest way, to format the output. I am running a remote search to NCBI's nucleotide database and want to filter and organize the information retrieved as much as possible. The ultimate goal is automate a way to run a remote search to NCBI's nucleotide database, and to work with the results in excel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am extremely new to this and struggling to find the best path. From what I can gather, many prefer using external programs, scripts, parsers, interpreters etc to organize their results. I am okay with doing this as well, but would still like to format the output of my blastn search as much as possible beforehand. It is returning a lot of data to me that I don't need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I automate this in excel, and just use VBA/macros to further organize my data? It seems some people prefer to just import the output text file to a spreadsheet and work from there. I have also seen options like BioEdit, eutilities, APIs... I'm not sure which route I should invest my time into figuring out.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks so much for lending your time and expertise!&lt;/p&gt;&#xA;" OwnerUserId="8856" LastActivityDate="2020-05-15T06:56:00.797" Title="Standalone Blast+; Automating searches, Blastn formatting, interpretation" Tags="|blast|blastn|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13287" PostTypeId="2" ParentId="13243" CreationDate="2020-05-15T03:34:19.113" Score="0" Body="&lt;p&gt;If you working on known cell culture, I recommend using classification approach instead of clustering. One library that I used to publish my paper is SingleR. I checked the expression x cell type matrix by both approaches (Seurat Clustering and SingleR), the classification approach make much more sense.&lt;/p&gt;&#xA;" OwnerUserId="4134" LastActivityDate="2020-05-15T03:34:19.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13288" PostTypeId="2" ParentId="13286" CreationDate="2020-05-15T06:56:00.797" Score="2" Body="&lt;p&gt;You just have to change the output format using the &lt;code&gt;-outfmt&lt;/code&gt; parameter. To have an output in a tabular format just use &lt;code&gt;-outfmt 6&lt;/code&gt;. The resulting table have 12 columns containing the following informations &lt;em&gt;queryacc subjectacc pident length mismatch gapopen querystart queryend subjectstart subjectend evalue bitscore&lt;/em&gt;. You may restrict the number of hits per query using &lt;code&gt;-max_target_seqs 5&lt;/code&gt; for example.&lt;/p&gt;&#xA;" OwnerUserId="8749" LastActivityDate="2020-05-15T06:56:00.797" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13289" PostTypeId="1" AcceptedAnswerId="13725" CreationDate="2020-05-15T09:11:03.053" Score="1" ViewCount="228" Body="&lt;p&gt;I performed a blastn search of NGS data against ssRNA database download from Internet, with a expected value 10-4. The size of NGS data reads is of 125 bp.&lt;/p&gt;&#xA;&lt;p&gt;I have analyzed the blast results of the libraries. These libraries are from different habitats. The most difficult habitat, oakwood, produced a blast results with a maximum length of 51 bp. These results have a qcov of 100 (in most of the cases).&lt;/p&gt;&#xA;&lt;p&gt;I would like to know if these low length sizes comes from degradation?&lt;/p&gt;&#xA;&lt;p&gt;Should do  I filter these samples with a length = 125 bp with the aim to obtain results with better quality or with a qcov of 100 is enought to get reliable results?&lt;/p&gt;&#xA;&lt;p&gt;Thanks you in advance.&lt;/p&gt;&#xA;" OwnerUserId="7430" LastEditorUserId="7430" LastEditDate="2020-06-25T20:52:06.987" LastActivityDate="2020-07-05T12:31:34.910" Title="Blast results filters" Tags="|blast|ngs|blastn|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="13291" PostTypeId="1" AcceptedAnswerId="13292" CreationDate="2020-05-15T11:35:37.543" Score="0" ViewCount="62" Body="&lt;p&gt;I am confused about how to perform a functional analysis after a variant calling on a cohort of patients. I have anotated all vcfs and filter them in order to get only those deleterious variants.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was thinking on merging those vcf:&#xA; I tried to merge them with &lt;code&gt;bcftools merge&lt;/code&gt; and also using &lt;code&gt;VariantAnnotation&lt;/code&gt; and transforming each sample vcf into a dataframe containing the variant lines to then row bind all those dataframes together.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As an example, Ill show some fields of teh first row of a dataframe:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;CHROM   START REF ALT FILTER DP  GENEID  SIFT&#xA;  17     1000   T   A   PASS   62   TP53    0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, I would extract the genes affected by those variants and perform a gene set enrichment analysis, a pathway analysis similar to those performed after an RNA-seq analysis (with &lt;code&gt;enrichR&lt;/code&gt;, for example). But I am confused about the viability of this steps. For example, I would like that, during the pathways analysis, the program I use not just reads the list of mutated genes in my cohort, but also takes into account that, if a gene is mutated in more than one sample, it should wheight more in the analysis than a gene which appears in just one sample.&lt;/p&gt;&#xA;" OwnerUserId="8631" LastEditorUserId="8631" LastEditDate="2020-05-15T12:12:29.810" LastActivityDate="2020-05-15T14:35:45.763" Title="Best practice to perform a functional analysis on a set of VCFs from a cohort of patients" Tags="|variant-calling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13292" PostTypeId="2" ParentId="13291" CreationDate="2020-05-15T13:35:41.157" Score="1" Body="&lt;p&gt;As a simple approach you may want to try the &quot;fuzzy&quot; list implemented in EnrichR itself. There is some information about this in the &lt;a href=&quot;http://amp.pharm.mssm.edu/Enrichr/help#background&quot; rel=&quot;nofollow noreferrer&quot;&gt;Help Center&lt;/a&gt; under the &quot;What is a gene set&quot; heading.   &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This way you can code the number of data sets a gene appears as a value between 0 and 1. In the Help Center this is used to convert fold changes or p-values to a membership score but I don't see why this would not also work for the number of data sets a SNP is observed in.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Since it is relatively easy to implement this can give you an first impression of what to expect. It then may be combined with analyzing each data set independently and then comparing the overlap.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastEditorUserId="4885" LastEditDate="2020-05-15T14:35:45.763" LastActivityDate="2020-05-15T14:35:45.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13294" PostTypeId="1" CreationDate="2020-05-15T16:12:56.873" Score="1" ViewCount="142" Body="&lt;p&gt;X-posted from the Main Stack Overflow because folks here are possibly more familiar with PyMol(?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have been trying to assess the strength of an interface I want to mutate using Pymol. What I am interested in is a list of contacts withing a specified interface (which I have calculated using the InterfaceResidues.py script from the PyMol wiki script library). My aim is to produce a bunch of mutants using the mutagenesis wizard and assess how the HBond network changes.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Actions&gt;Find&gt;Polar Contacts does the trick in visualising them, but doesn't output a file with numbers which I can then crunch to assess which mutations weaken my interface most. &#xA;I have been trying the scripts list_contacts.py and list_hb.py from Prof. Robert Campbell of Queen's University at Kingston, but with limited success (nothing gets printed or written to the file).&#xA;The scripts can be found here &lt;a href=&quot;http://pldserver1.biochem.queensu.ca/~rlc/work/pymol/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://pldserver1.biochem.queensu.ca/~rlc/work/pymol/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am going to contact Prof. Campbell directly re: scripts but, in the meantime, and in case he's too busy to deal with a Pymol noob, does anyone have any ideas of what else I could try?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am using Pymol 2.3.1 and Python 3.7.1&lt;/p&gt;&#xA;" OwnerUserId="8863" LastActivityDate="2020-05-22T14:09:40.850" Title="Outputting crunchable list of HBonds from Pymol" Tags="|python|protein-structure|protein-protein-interaction|structural-biology|pymol|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13295" PostTypeId="1" CreationDate="2020-05-15T16:30:23.710" Score="0" ViewCount="1426" Body="&lt;p&gt;I am trying to follow a tutorial from Sanger institute (from May 2019) on analysis of single cell RNA Seq data. They use isSpike function to filter out ERCC (control) and MT (mitochondrial RNA) reads, but I am getting an error message that isSpike is a deprecated function. I am trying to use altExp instead as recommended by some people. I can't seem to figure this out. Can anyone provide some code to use instead of isSpike? I am new to bioinformatics. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried to swap out the isSpike function with the altExp function, but I am getting stuck on how to translate the grepl function.&lt;/p&gt;&#xA;" OwnerUserId="8610" LastEditorUserId="8610" LastEditDate="2020-05-15T17:51:07.187" LastActivityDate="2021-03-02T10:19:54.323" Title="isSpike function in SingleCellExperiment package is deprecated?" Tags="|r|scrnaseq|bioconductor|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13297" PostTypeId="1" AcceptedAnswerId="13299" CreationDate="2020-05-15T20:11:46.803" Score="0" ViewCount="1250" Body="&lt;p&gt;Suppose I have done two Chip-seq experiments. Now after the bias correction step such as MACS2, I get two set of genomic intervals (or peaks, as what is usually called) that corresponds to each of the experiment. I want to test the significance of the overlap of these two set genomic intervals that corresponds to each experiment.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The common test employed seemed to me is the Hypergeometic test. There's even a package specifically aim for this task: &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/vignettes/ChIPpeakAnno/inst/doc/ChIPpeakAnno.html&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;ChiPpeakAnno&quot;&gt;ChiPpeakAnno&lt;/a&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recall from the Hypergeometric distrbition -- the analogy is that we have a bag of marbles with two different colors, say one blue and another red. A sample from such distribution consists of marbles drawn from this bag, and a probability measure for this distribution is the number of blue marbles in the sample. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Could someone explain what is the correspondence between the hypergeometric test and the significance overlap between two different set of genomic intervals?&lt;/p&gt;&#xA;" OwnerUserId="4761" LastEditorUserId="4761" LastEditDate="2020-05-15T22:37:38.343" LastActivityDate="2020-05-16T00:31:48.073" Title="Hypergeometic test for the overlap of two set of genomic intervals (e.g. from Chip-seq data)" Tags="|statistics|chip-seq|peak-calling|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13298" PostTypeId="2" ParentId="13297" CreationDate="2020-05-15T22:57:29.573" Score="1" Body="&lt;p&gt;Just to make sure, I took a quick look at chippeakanno, and it uses a hypergeometric test for gene enrichment, not necessarily overlap of two chip-seq results.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you mentioned, you can imagine in your set of genes, you have those that overlap with your peaks, and those that do not. Hypergeometric distribution calculates the probability of getting a certain number of genes that overlap with your peaks (which is the same as sampling some balls and calculate the p-value of some balls with one color). The hypergeometric test calculates a p-value to see whether this number of overlapping genes is more than expected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I also read on Wikipedia that the P-value of hypergeometric test times 2 is exactly the P-value for Fisher's exact test. Fisher's exact test tests whether your overlap is more than (or less than) expectation. Hypergeometric tests just feel like a one-tailed version of Fisher's exact tests.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-05-15T22:57:29.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13299" PostTypeId="2" ParentId="13297" CreationDate="2020-05-16T00:31:48.073" Score="6" Body="&lt;p&gt;There is no such thing as a hypergeometric test, at least in statistical textbooks. It's a fisher test based on hypergeometric distribution.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If it is chip-seq for the same target, i.e biological replicates, significance of overlap is not quite meaningful. You get more information by for example checking the correlation of the coverage between your replicates &lt;a href=&quot;https://deeptools.readthedocs.io/en/develop/content/example_usage.html&quot; rel=&quot;noreferrer&quot;&gt;using deeptools&lt;/a&gt; or anything else.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are comparing between two different targets, for example, chip for two different histone marks, you can use the fisher test or chi-square this way. &lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Partition the genome in X bp bins, these are all your marbles&lt;/li&gt;&#xA;&lt;li&gt;Identity those that overlap with peaks from target 1 and those that don't&lt;/li&gt;&#xA;&lt;li&gt;Do 2. for target2 &lt;/li&gt;&#xA;&lt;li&gt;construct a contingency table using 2 and 3.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;So you should end up with something like this (purely made up): &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;                      no_overlap_target2 overlap_target2&#xA;  no_overlap_target1               9799             112&#xA;  overlap_target1                    88               1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And if there is overlap, the bottom diagonal with be higher.. less on the off-diagonal, and you can test it with a fisher.test. &lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-05-16T00:31:48.073" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13301" PostTypeId="1" CreationDate="2020-05-16T04:46:54.720" Score="0" ViewCount="23" Body="&lt;p&gt;E.g. bowhead whale genome has X contigs and Y scaffolds, but couldn't these easily be different numbers?&lt;/p&gt;&#xA;" OwnerUserId="8867" LastActivityDate="2020-05-16T12:13:13.873" Title="Is the number of contigs and scaffolds an artifact of genome assembly and rather arbitrary, or is it canonical for each species?" Tags="|genomics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13303" PostTypeId="1" CreationDate="2020-05-16T06:15:34.240" Score="1" ViewCount="29" Body="&lt;p&gt;I am from the field of Computer Science and I want to perform some experiments related to miRNA and mRNA prediction.I have been searching for datasets in this area and found one called mirWalk available at: &lt;a href=&quot;http://mirwalk.umm.uni-heidelberg.de/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mirwalk.umm.uni-heidelberg.de/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;when I enter, for example, the Entrez ID 10608 it appears to me the following table (I have only put the first record):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/R3V9e.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/R3V9e.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Woclj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Woclj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/m8ehk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m8ehk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have two questions:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) Can I use this dataset to try to make predictions about the interaction of this gene with, for example, miRNA hsa-let-7c-5p? For what I see this interaction appears in the table, but it seems it has not been validated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;b) What are the meaning of the different fields, there is no information about it in the website of mirWalk?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks for your help and consider that I am a newbie in this bioinformatics field.&lt;/p&gt;&#xA;" OwnerUserId="7436" LastActivityDate="2020-05-16T06:15:34.240" Title="miRNA dataset for prediction experiments" Tags="|gene|public-databases|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13304" PostTypeId="1" AcceptedAnswerId="13311" CreationDate="2020-05-16T08:43:44.137" Score="0" ViewCount="325" Body="&lt;p&gt;I have a problem in R. I have the following dataSet (the first three rows shown) (the 5th number is the methylation level in its region):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr1    15864   15866   FALSE   894 +&#xA;chr1    534241  534243  FALSE   921 -&#xA;chr1    710096  710098  FALSE   729 +&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and then I'm trying to get the sequences from every row in the dataSet (427.000 rows in total):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(BSgenome.Hsapiens.UCSC.hg19)&#xA;genome &amp;lt;- BSgenome.Hsapiens.UCSC.hg19&#xA;chr&amp;lt;-as.matrix(as.character(mydataSet[,1]))&#xA;start&amp;lt;-as.matrix(as.integer(as.character(mydataSet[,2]))-50)&#xA;end&amp;lt;-as.matrix(as.integer(as.character(mydataSet[,3]))+50)&#xA;Sequences_for_50_plus_bases&amp;lt;-getSeq(genome,chr,start=start,end=end)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'd like to find a new fuction where I can have this dataSet, and find for its rows (which represent sequences in the human genome) if they refer to a promoter, or an enhancer or any other DNA regions' information, because I'd like to compare the differences between the regions' methylation levels.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a way to do this?&lt;/p&gt;&#xA;" OwnerUserId="8600" LastEditorUserId="298" LastEditDate="2020-05-16T13:27:28.470" LastActivityDate="2020-05-17T06:45:22.653" Title="Get regions' information from DNA sequence data (bsgenome.hsapiens.ucsc.hg19)" Tags="|r|bioconductor|phylogenetics|methylation|hg19|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13305" PostTypeId="1" AcceptedAnswerId="13314" CreationDate="2020-05-16T10:07:43.860" Score="0" ViewCount="167" Body="&lt;p&gt;What's the difference between Barycenter Centrality and Closeness Centrality (Freeman)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Barycenter scores are calculated as 1 / (total distance from vertex v to all other vertices) in a strongly connected network. &lt;a href=&quot;https://www.centiserver.org/centrality/Barycenter_Centrality/&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Closeness Centrality (Freeman) are calculated  total distance from a node v to all the other nodes in a network. &lt;a href=&quot;https://www.centiserver.org/centrality/Closeness_Centrality/&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It looks like the same centrality .&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ps: &#xA;I want to analyze closeness for my protein–protein interaction network.&lt;/p&gt;&#xA;" OwnerUserId="8868" LastActivityDate="2020-05-17T22:01:55.440" Title="What's the difference between Barycenter Centrality and Closeness Centrality (Freeman)?" Tags="|networks|protein-protein-interaction|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13306" PostTypeId="2" ParentId="13301" CreationDate="2020-05-16T12:13:13.873" Score="1" Body="&lt;p&gt;This is purely arbitrary and an artifact of how well the assembly turned out. Better quality assemblies with have fewer and longer contigs.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-05-16T12:13:13.873" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13308" PostTypeId="1" AcceptedAnswerId="13309" CreationDate="2020-05-16T18:17:40.720" Score="0" ViewCount="208" Body="&lt;p&gt;I am following the &lt;a href=&quot;https://horvath.genetics.ucla.edu/html/CoexpressionNetwork/Rpackages/WGCNA/Tutorials/Consensus-NetworkConstruction-auto.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorials&lt;/a&gt; on the WGCNA website for creating a coexpression network. It is taking way too long. Is there a way to speed up execution? All I need is to create the co-expression graph. I will then export to Python and do my analysis there. &#xA;In particular, the step &quot;calculating module eigengenes&quot; is taking a lot of time. I do not really care about the modules. All I want is the graph with the genes and the weighted edges. &lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2020-05-16T18:40:51.517" Title="Speeding up network creation in WGCNA" Tags="|wgcna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13309" PostTypeId="2" ParentId="13308" CreationDate="2020-05-16T18:40:51.517" Score="0" Body="&lt;p&gt;You can use the &quot;manual network construction&quot; approach. Calculate TOM, then use &lt;code&gt;exportNetworkToCytoscape&lt;/code&gt; or &lt;code&gt;exportNetworkToVisANT&lt;/code&gt; to export the network (or simply save the TOM matrix as a table and load it in Python). &lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2020-05-16T18:40:51.517" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13310" PostTypeId="1" CreationDate="2020-05-17T00:33:10.067" Score="1" ViewCount="45" Body="&lt;p&gt;In Horvath's &lt;em&gt;Weighted network analysis&lt;/em&gt;, the author mentions that M^b,  where b is a real number and M is a matrix, denotes the element-wise power. This seems odd to me. The way I was taught, M^b would mean multiplying the matrix M, b times. Should element-wise power not be M^(ob) where where o denotes the Hadamard power? &lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2020-05-17T21:08:57.560" Title="Weighted Network Analysis Book by Steve Horvath, taking the power of a matrix" Tags="|wgcna|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13311" PostTypeId="2" ParentId="13304" CreationDate="2020-05-17T06:45:22.653" Score="1" Body="&lt;p&gt;You can get specific sequences efficiently using the following packages:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(GenomicFeatures)&#xA;library(Biostrings)&#xA;library(tidyverse)&#xA;library(data.table)&#xA;library(dtplyr)&#xA;library(BSgenome.Hsapiens.UCSC.hg19)&#xA;library(TxDb.Hsapiens.UCSC.hg19.knownGene)&#xA;&#xA;Methylation_data &amp;lt;- tribble(&#xA;  ~seqnames,~start,~end,~Logical,~&quot;Methylation Level&quot;,~strand,&#xA;  &quot;chr1&quot;,15864,15866,FALSE,894,&quot;+&quot;,&#xA;  &quot;chr1&quot;,534241,534243,FALSE,921,&quot;-&quot;,&#xA;  &quot;chr1&quot;,710096,710098,FALSE,729,&quot;+&quot;) # Arrange your dataset&#xA;&#xA;Methylation_GRanges &amp;lt;- GRanges(  # Construct a GRanges objects&#xA;  seqnames = Methylation_data %&amp;gt;% pull(seqnames), # First pull in the chromosome locations&#xA;  IRanges(start = (Methylation_data %&amp;gt;% pull(start))-50, # then the start position minus 50 nt&#xA;          end = (Methylation_data %&amp;gt;% pull(end))+50), # the end position plus 50 nt&#xA;  strand = Methylation_data %&amp;gt;% pull(strand) # the strand your sequence is located on&#xA;)&#xA;&amp;gt; getSeq(Hsapiens,Methylation_GRanges)&#xA;A DNAStringSet instance of length 3&#xA;width seq&#xA;[1]   103 GCTGCTGCTTCTCCAGCTTTCGCTCCTTCATGCTGCGCAGC...CTTGGCGGATGGACTCTAGCAGAGTGGCCAGCCACCGGAG&#xA;[2]   103 ACAGTTCCAATGTAATCAGAGAGAACATCACACACACACCA...ACCTGAGCAGCACTCTGCAAAGCTGTCAAGGCGGTGAAGC&#xA;[3]   103 GATAAAATAAAGCTTAGATTGGAAAAAATATTTAAGATTCT...GGAAGCTGAGTAATTGTATGTTCAAATACTTGCAAAACAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To write the sequences to a file, use &lt;code&gt;writeXStringSet()&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the second part of what you need, &lt;code&gt;GenomicFeatures&lt;/code&gt; has a number of functions that obtain the annotated sequences of the whole genome by its type:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;transcriptsBy(x, by=c(&quot;gene&quot;, &quot;exon&quot;, &quot;cds&quot;), ...)&#xA;exonsBy(x, by=c(&quot;tx&quot;, &quot;gene&quot;), ...)&#xA;cdsBy(x, by=c(&quot;tx&quot;, &quot;gene&quot;), ...)&#xA;intronsByTranscript(x, ...)&#xA;fiveUTRsByTranscript(x, ...)&#xA;threeUTRsByTranscript(x, ...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And this is how you use one of them to find whether your sequences overlap a particular type of DNA:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Find_overlap &amp;lt;- function(Ref_Data,&#xA;                         Overlap_Message){&#xA;&#xA;  Ref_Data &amp;lt;- as.data.table(Ref_Data) # Convert to data.table for efficiency&#xA;&#xA;  Methylation_data &amp;lt;- tribble(&#xA;    ~seqnames,~start,~end,~Logical,~&quot;Methylation Level&quot;,~strand,&#xA;    &quot;chr1&quot;,15864,15866,FALSE,894,&quot;+&quot;,&#xA;    &quot;chr1&quot;,534241,534243,FALSE,921,&quot;-&quot;,&#xA;    &quot;chr1&quot;,710096,710098,FALSE,729,&quot;+&quot;) %&amp;gt;%&#xA;    mutate(Overlap = &quot;-&quot;,&#xA;           start = start-50,&#xA;           end = end+50) %&amp;gt;%&#xA;    as.data.table()&#xA;&#xA;  Methylation_data[Ref_Data, &#xA;                   Overlap := Overlap_Message, &#xA;                   on = .(start &amp;gt;= start, &#xA;                          end &amp;lt;=  end)] # wherever the range overlaps, assign an identifier to it&#xA;}&#xA;&#xA;&#xA;Methylated_in_5UTR &amp;lt;- fiveUTRsByTranscript(TxDb.Hsapiens.UCSC.hg19.knownGene) %&amp;gt;% &#xA;  Find_overlap(., &quot;5' UTR&quot;) %&amp;gt;% &#xA;  print&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seqnames  start    end Logical Methylation Level strand Overlap&#xA;1:     chr1  15814  15916   FALSE               894      +       -&#xA;2:     chr1 534191 534293   FALSE               921      -       -&#xA;3:     chr1 710046 710148   FALSE               729      +  5' UTR&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8870" LastActivityDate="2020-05-17T06:45:22.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13312" PostTypeId="1" CreationDate="2020-05-17T16:07:07.573" Score="0" ViewCount="213" Body="&lt;p&gt;I am trying to process fastq files in order to build gene co-expression networks (by following &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pubmed/29525965&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt;). When I download any fastq file from ENA, and try to process it with the command:&#xA;&lt;code&gt;ls *.fastq.gz | xargs -n1 fastqc $1&lt;/code&gt;, I get:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Failed to process SRR494099.fastq.gz&#xA;java.io.EOFException&#xA;    at java.base/java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:269)&#xA;    at java.base/java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:259)&#xA;    at java.base/java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:165)&#xA;    at java.base/java.util.zip.GZIPInputStream.&amp;lt;init&amp;gt;(GZIPInputStream.java:80)&#xA;    at java.base/java.util.zip.GZIPInputStream.&amp;lt;init&amp;gt;(GZIPInputStream.java:92)&#xA;    at uk.ac.babraham.FastQC.Utilities.MultiMemberGZIPInputStream.&amp;lt;init&amp;gt;(MultiMemberGZIPInputStream.java:37)&#xA;    at uk.ac.babraham.FastQC.Sequence.FastQFile.&amp;lt;init&amp;gt;(FastQFile.java:80)&#xA;    at uk.ac.babraham.FastQC.Sequence.SequenceFactory.getSequenceFile(SequenceFactory.java:106)&#xA;    at uk.ac.babraham.FastQC.Sequence.SequenceFactory.getSequenceFile(SequenceFactory.java:62)&#xA;    at uk.ac.babraham.FastQC.Analysis.OfflineRunner.processFile(OfflineRunner.java:159)&#xA;    at uk.ac.babraham.FastQC.Analysis.OfflineRunner.&amp;lt;init&amp;gt;(OfflineRunner.java:121)&#xA;    at uk.ac.babraham.FastQC.FastQCApplication.main(FastQCApplication.java:316)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Similarly, when I just download the fastq files and try to decompress them using gunzip, I get, for example: &lt;code&gt;gzip: SRR494099.fastq.gz: unexpected end of file&lt;/code&gt;&#xA;Why is this happening?&lt;/p&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2020-05-17T16:07:07.573" Title="Unable to gunzip fastq files from ENA" Tags="|fastq|gene-expression|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="13313" PostTypeId="2" ParentId="13310" CreationDate="2020-05-17T21:08:57.560" Score="2" Body="&lt;p&gt;You CAN calculate M^b for b being float number, complex number or even another matrix. &#xA;As well as you can calculate exp(M), log(M), sin(M) whatever... That is very standard for math people. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Let me give two ways how you can think of it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1) M^b = exp(b log(M)), so you need to define what is exp and log for matrices.&#xA;That can be done using power series: exp(M) = 1 + M + M^2/2! + M^3/3! + ... &#xA;log(1+M) = M - M^2/2 + M^3/3 - ... &lt;/p&gt;&#xA;&#xA;&lt;p&gt;2) More general way to calculate any function of matrix f(M) , looks like that:&#xA;if your matrix is diagonal, then it is true that: f(diag(d_i)) = diag( f(d_i)  ) . &#xA;So the question how to reduce any matrix to diagonal one.&#xA;That &lt;a href=&quot;https://en.wikipedia.org/wiki/Jordan_normal_form&quot; rel=&quot;nofollow noreferrer&quot;&gt;can be done for almost all&lt;/a&gt; matrices by conjugation M = C D C^{-1}.&#xA;Then f(M) = C f(D) C^{-1}. &lt;/p&gt;&#xA;" OwnerUserId="7441" LastActivityDate="2020-05-17T21:08:57.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13314" PostTypeId="2" ParentId="13305" CreationDate="2020-05-17T22:01:55.440" Score="1" Body="&lt;p&gt;First, I'll observe that Freeman centrality as you define it is the reciprocal of the Barycenter centrality as you define. It looks like people often use the reciprocal instead of the original definition, so in that sense they do look the same.&lt;/p&gt;&#xA;&lt;p&gt;I found the following comparison &lt;a href=&quot;https://rdrr.io/cran/centiserve/man/barycenter.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (though it doesn't name it as Freeman):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;There are 2 types of distance centrality scores, Closeness Centrality&#xA;and Barycenter Centrality. Barycenter Centrality for vertex v defined&#xA;as:&lt;/p&gt;&#xA;&lt;p&gt;1 / (total distance from v to all other vertices)&lt;/p&gt;&#xA;&lt;p&gt;Closeness scores are calculated using the formula 1 / (average&#xA;distance from vertex v to all other vertices) and Barycenter scores&#xA;are calculated as 1 / (total distance from vertex v to all other&#xA;vertices).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So it appears that for this package it is a difference of the reciprocal of the total vs. reciprocal of the average.&lt;/p&gt;&#xA;&lt;p&gt;Notably, wikipedia differs here, in that it matches your source:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Freeman's closeness centrality, the total geodesic distance from a given vertex to all other vertices, is the best known example.&lt;a href=&quot;https://en.wikipedia.org/wiki/Centrality#cite_note-Borgatti2006-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;citation&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.analytictech.com/networks/centrali.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yet another source&lt;/a&gt; notes the existence of a &amp;quot;normalized&amp;quot; version of Freeman's closeness centrality, which normalizes centrality of a node by the degree of the network. It also comments on whether it is the reciprocal or not:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Closeness (Freeman '79)&#xA;The graph-theoretic distance of a given node to all other nodes. The sum of the rows/columns of the geodesic distance matrix of a graph.&lt;/p&gt;&#xA;&lt;p&gt;Simple closeness is an inverse measure of centrality: the larger the numbers, the more distant an actor is, and the less central. Should really be called &amp;quot;farness&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Normalized version divides the minimum &amp;quot;farness&amp;quot; possible (N-1) by &amp;quot;farness&amp;quot; to simultaneously make the range 0 to 1 and invert the measure so that larger values correspond to greater centrality (truly &amp;quot;closeness&amp;quot;)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So I guess that it might depend on what tools you are using. Most of the time, it might be the same. Other times, it might be normalized in different ways with either averages or for 0-1 ranges.&lt;/p&gt;&#xA;&lt;p&gt;Closeness centrality is probably the more standard term, so it is unlikely that anyone would criticize it. But it appears that there is sometimes some variation in what people mean by that term and whether they are using a normalized or a raw closeness centrality.&lt;/p&gt;&#xA;&lt;p&gt;In my opinion the normalization leading to a range from 0-1 is nice because it makes the measure at least somewhat generalizable across networks. But it's up to what you need to do.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2020-05-17T22:01:55.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13315" PostTypeId="2" ParentId="10825" CreationDate="2020-05-18T08:02:43.107" Score="1" Body="&lt;p&gt;As far as I'm concerned, bowtie 1 is still the method of choice since it rather mature and fast. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;And yes, pseudoaligner such as salmon are probably not the best choice For once, they need a reference &quot;transcriptome&quot;, which is usually available for RNA-Seq but not so much for small-RNAs. Also the way they handle multiple mapping locations is probably not ideal for small-RNAs that may have excessive number mapping location.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've also written a simple &lt;a href=&quot;https://github.com/seb-mueller/snakemake_sRNAseq&quot; rel=&quot;nofollow noreferrer&quot;&gt;snakemake-pipeline&lt;/a&gt; which simply performs some standard pre-processing and mapping against a reference genome along with QC and coverage tracks. It's still work in progress, but I'd be curious about any feedback.&lt;/p&gt;&#xA;" OwnerUserId="1461" LastActivityDate="2020-05-18T08:02:43.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13316" PostTypeId="1" CreationDate="2020-05-18T15:11:18.397" Score="2" ViewCount="294" Body="&lt;p&gt;Is NCBI's &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbSNP&lt;/a&gt; the most comprehensive database of SNP frequency?&lt;/p&gt;&#xA;&lt;p&gt;I'd like to get the largest possible database of allele frequencies per 'population' or geographic region. For example, on average, which country has the highest number of As at rsNNNN?&lt;/p&gt;&#xA;&lt;p&gt;So I'm looking for global coverage above all else.&lt;/p&gt;&#xA;" OwnerUserId="8875" LastEditorUserId="3848" LastEditDate="2020-11-11T17:21:30.960" LastActivityDate="2020-11-11T17:21:30.960" Title="Global SNP frequency database?" Tags="|database|snp|allele-frequency|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13317" PostTypeId="1" AcceptedAnswerId="13318" CreationDate="2020-05-18T18:32:49.203" Score="3" ViewCount="337" Body="&lt;p&gt;I am looking for a given mutation in &lt;a href=&quot;https://us.v-cdn.net/5019796/uploads/FileUpload/c4/d9900bc2df5ef6ed48c9709bb16de9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;IGV&lt;/a&gt;, which accepts coordinates in the form of&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr&amp;lt;X&amp;gt;:&amp;lt;Y&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Where X is the chromosome number and Y is the base location in that chromosome.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Eg. &lt;code&gt;chr6:26091179&lt;/code&gt; which has the following attributes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Gene: HFE|LOC108783645&#xA;Marker: rs1799945&#xA;Position: chr6:26091179&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Additionally, an interval can be written like&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;chr&amp;lt;X&amp;gt;:&amp;lt;Y&amp;gt;-&amp;lt;Z&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So basically from base located at Y to Z.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;My questions are:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;1)&lt;/strong&gt; How do I convert &lt;code&gt;HFE|LOC108783645&lt;/code&gt; to &lt;code&gt;chr6:26091179&lt;/code&gt; (and vice-versa)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;2)&lt;/strong&gt; How do I convert &lt;code&gt;rs1799945&lt;/code&gt; to &lt;code&gt;chr6:26091179&lt;/code&gt; (and vice-versa)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;3)&lt;/strong&gt; On a different example, how do I convert something like &lt;code&gt;codon 109 of exon 3 of beta globin gene (GTG-&amp;gt;TTG)&lt;/code&gt; to &lt;code&gt;chr&amp;lt;X&amp;gt;:&amp;lt;Y&amp;gt;&lt;/code&gt; (or &lt;code&gt;chr&amp;lt;X&amp;gt;:&amp;lt;Y&amp;gt;-&amp;lt;Z&amp;gt;&lt;/code&gt;)?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why are there so many ways to specify positions in genes? For me the most clear way is &lt;code&gt;chr&amp;lt;X&amp;gt;:&amp;lt;Y&amp;gt;&lt;/code&gt;, all the others leave me guessing...&lt;/p&gt;&#xA;" OwnerUserId="8577" LastActivityDate="2020-05-18T23:58:47.787" Title="How can I convert codon coordinates to genomic position?" Tags="|gene|genome|phylogenetics|igv|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13318" PostTypeId="2" ParentId="13317" CreationDate="2020-05-18T23:58:47.787" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;HFE|LOC108783645&lt;/code&gt; is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene/108783645&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/gene/108783645&lt;/a&gt;. You can use biomaRt to go from the &lt;code&gt;LOCxxxx&lt;/code&gt; ID to location. Searching by &lt;code&gt;HFE&lt;/code&gt; will give you a different location though, so you might have to account for such eventualities. It is a long and often manually intensive process.&lt;/li&gt;&#xA;&lt;li&gt;You should be able to use entrez-direct/eutils to go from rsID to position&lt;/li&gt;&#xA;&lt;li&gt;Unless there's some sort of natural language processing API, &lt;code&gt;codon 109 of exon 3 of beta globin gene&lt;/code&gt; cannot be parsed by existing APIs (Unless it's always &lt;code&gt;codon X of exon Y of Z&lt;/code&gt;, in which case you could use &lt;code&gt;grep&lt;/code&gt; too get the components. Even so, going from exon + codon + codon base change to exact base location will require a lot of wrangling.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="650" LastActivityDate="2020-05-18T23:58:47.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13319" PostTypeId="1" AcceptedAnswerId="13329" CreationDate="2020-05-19T08:18:59.650" Score="3" ViewCount="237" Body="&lt;p&gt;I am performing a quantitative trait association between the expression of one gene and ~400,000 methylation values. First, both variables are rank inverse normal transformed, adjusted for confounders (technical and for family structure) in a mixed-effects model and we use the residuals for the association. Then we do a linear regression between the residuals of expression and each methylation site, adding other four confounders in the model (age, BMI, etc.). The sample size is of ~500. For the association we are using MatrixEQTL.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When we check the p-values, they are clearly inflated. Is it something that I have to be concerned about? Did I probably miss something in the analysis? Thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/u6An4.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/u6An4.jpg&quot; alt=&quot;QQ-plot of association analysis&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8879" LastActivityDate="2020-05-20T16:27:13.883" Title="Inflated p-values in quantitative trait analysis" Tags="|statistics|gene-expression|methylation|linear-regression|qtls|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13320" PostTypeId="2" ParentId="13295" CreationDate="2020-05-19T09:52:42.857" Score="2" Body="&lt;p&gt;&lt;code&gt;isSpike&lt;/code&gt; was provided by &lt;code&gt;SingleCellExperiment&lt;/code&gt;. It is now deprecated. The notes in &lt;code&gt;SingleCellExperiment&lt;/code&gt; indicate that one should use &lt;code&gt;altExps&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am struggling with a similar problem. Can't compile a package (&lt;code&gt;powsimR&lt;/code&gt;) because a dependency uses &lt;code&gt;isSpike&lt;/code&gt; (not used in the code directly), so I have to hunt down where it is used by one of a very long list of dependencies.&lt;/p&gt;&#xA;" OwnerUserId="8880" LastEditorUserId="8880" LastEditDate="2020-05-19T10:01:10.007" LastActivityDate="2020-05-19T10:01:10.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13321" PostTypeId="1" CreationDate="2020-05-19T19:50:55.327" Score="3" ViewCount="1268" Body="&lt;p&gt;I was wondering if there was any R-package for receptor-ligand interaction similar to SingleCellSignal (&lt;a href=&quot;https://github.com/SCA-IRCM/SingleCellSignalR_v1/blob/master/SingleCellSignalR/vignettes/UsersGuide.Rmd&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/SCA-IRCM/SingleCellSignalR_v1/blob/master/SingleCellSignalR/vignettes/UsersGuide.Rmd&lt;/a&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;+++ if there are vignettes and/or the graphs can be customized.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;+++ if it can be linked to Seurat&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note: I have tried iTALK, but I keep getting an error that seems to be a common problem and have left a query. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="8882" LastActivityDate="2020-05-20T06:22:58.940" Title="Receptor-Ligand R-package for visualization" Tags="|r|scrnaseq|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13322" PostTypeId="2" ParentId="13295" CreationDate="2020-05-19T21:09:26.250" Score="1" Body="&lt;p&gt;You are on the latest version of SingleCellExperiment most likely. You should not go back to using the old version. One thing is to check out the &lt;a href=&quot;https://bioconductor.org/packages/devel/bioc/vignettes/SingleCellExperiment/inst/doc/intro.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;new tutorial&lt;/a&gt;. In any case, the purpose of altExp() is to provide another slot of ERCC reads and MT.. etc.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So in your example (super not recommended to watch video btw) :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;molecules = read.delim(&quot;https://raw.githubusercontent.com/jdblischak/singleCellSeq/master/data/molecules.txt&quot;,stringsAsFactors=FALSE,row.names=1)&#xA;&#xA;annotation = read.delim(&quot;https://raw.githubusercontent.com/jdblischak/singleCellSeq/master/data/annotation.txt&quot;,stringsAsFactors=FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We can pull out the spike in and mt genes:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(SingleCellExperiment)&#xA;library(EnsDb.Hsapiens.v86)&#xA;&#xA;Spikein_names = grep(&quot;^ERCC-&quot;,rownames(molecules),value=TRUE)&#xA;SpikeIn = molecules[Spikein_names,]&#xA;&#xA;g = genes(EnsDb.Hsapiens.v86)&#xA;MTgene_names = g[seqnames(g)==&quot;MT&quot;]$gene_id&#xA;MTgenes = molecules[rownames(molecules) %in% MTgene_names,]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Have a vector of genes to keep:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;keep = setdiff(rownames(molecules),c(SpikeIn,MTgenes))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then make the single cell object:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sce &amp;lt;- SingleCellExperiment(&#xA;assays=list(counts=as.matrix(molecules[keep,])),colData=annotation)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Use altExp to slot in the stuff:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;altExp(sce, &quot;spike-in&quot;) &amp;lt;- SummarizedExperiment(SpikeIn)&#xA;altExp(sce, &quot;MTgenes&quot;) &amp;lt;- SummarizedExperiment(MTgenes)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastEditorUserId="5077" LastEditDate="2020-05-19T21:15:50.240" LastActivityDate="2020-05-19T21:15:50.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13323" PostTypeId="1" CreationDate="2020-05-19T22:03:53.687" Score="0" ViewCount="48" Body="&lt;p&gt;I'm a beginner in this field (so pardon my rudimentary knowledge/explanation) trying to understand ways on how I could potentially identify genes of unknown function. I am trying to get as close as I can to the unknown gene's actual function.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;What I've currently done, is that I've used an annotation tool to annotate my genome sample for its genes. From it, I've acquired a list of potential genes of unknown function, and I am capable of collecting its sequence. From here, I've also used BLASTn and BLASTp to align my gene sequences to find similar gene sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From here, I'm absolutely clueless as to how I could proceed, since similarly sequenced genes are not always similarly functioned.&#xA;I would also like a list of other annotating tools, as well as other sequence alignment tools like BLAST, just so I could see the differences between tools.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be much appreciated.&#xA;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="8884" LastEditorUserId="8884" LastEditDate="2020-05-19T22:24:41.207" LastActivityDate="2020-05-19T22:24:41.207" Title="Determining a Gene's Unknown Function" Tags="|sequence-alignment|gene|sequence-analysis|sequence-annotation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13325" PostTypeId="1" AcceptedAnswerId="13326" CreationDate="2020-05-19T23:00:01.203" Score="1" ViewCount="383" Body="&lt;p&gt;Im trying to read a multifasta sequence file into phangorn I get the following error &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;tipseqfile &amp;lt;- (&quot;abhi_seq/clean_dup_rem.fasta&quot;)&#xA;tipseq &amp;lt;- read.phyDat(tipseqfile,format=&quot;fasta&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://drive.google.com/open?id=1kTXfnRVjpl3xjZFQUpGaJck9M4d5NSnu&quot; rel=&quot;nofollow noreferrer&quot;&gt;my file&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error in read.phyDat(tipseqfile, format = &quot;fasta&quot;) : &#xA;        sequences have different length&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Now is it necessary to have same length of each fasta sequence? or Im doing something wrong &lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="3848" LastEditDate="2023-07-12T22:26:56.497" LastActivityDate="2023-07-12T22:26:56.497" Title="phangorn fasta file read error" Tags="|r|phylogenetics|fasta|phylogeny|phangorn|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13326" PostTypeId="2" ParentId="13325" CreationDate="2020-05-19T23:52:48.930" Score="3" Body="&lt;p&gt;You have to perform an alignment first. Phangorn use phylogenetic estimation and this requires a precise alignment position to track homologous nucleotide mutation. Maximum likelihood theory will assess the probability of a given tree for each homologous nucleotide site, if they are not alignment it can't do it and will violate the assumptions of phylogeny. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the sequences are different lengths, which it means both nucleotide and indel ('-') sites, it means there hasn't been or there is an error with the alignment. I use &lt;code&gt;muscle&lt;/code&gt; these days having previously used &lt;code&gt;clustalo&lt;/code&gt;. Its really easy to use and easily configures on a linux server and its super quick. If you get stuck let me know.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I had a look at your alignment and can verify it hasn't been aligned.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-05-19T23:52:48.930" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13327" PostTypeId="1" AcceptedAnswerId="13337" CreationDate="2020-05-20T00:38:21.080" Score="1" ViewCount="2013" Body="&lt;p&gt;Nowadays, I am trying to calculate Pearson correlation values between two genes of my interest from single-cell RNA-data (features.tsv, barcodes.tsv, and matrix.mtx files) which are obtained from the 10xgenomics cellranger. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;FYI, The filtered_feature_bc_matrix includes an aggregated population of tumor-infiltrating immune cells that are derived from 6 different syngeneic mouse tumors (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE121861&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE121861&lt;/a&gt;). I downloaded fastq files and ran &quot;cellranger count&quot; and &quot;cellranger aggr&quot; commands to obtain combined data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My strategy includes the use of the &quot;feature scatter&quot; function deposited in the library(Seurat). Finally, I ended up getting results below. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0OLyF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0OLyF.png&quot; alt=&quot;FeatureScatter plot&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is that can we select a specific cluster(s) or identity on this plot? For example, I want to see correlation values in cluster 2 only.&#xA;The usage of Feature scatter is as follows. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;FeatureScatter(&#xA;  object,&#xA;  feature1,&#xA;  feature2,&#xA;  cells = NULL,&#xA;  group.by = NULL,&#xA;  cols = NULL,&#xA;  pt.size = 1,&#xA;  shape.by = NULL,&#xA;  span = NULL,&#xA;  smooth = FALSE,&#xA;  combine = TRUE,&#xA;  slot = &quot;data&quot;&#xA;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Do you know how to get the Pearson correlation of my two genes using Seurat? This is the best I can do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="8657" LastActivityDate="2020-05-20T14:26:02.863" Title="How to specifically select cluster on Featurescatter on Seurat?" Tags="|seurat|correlation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13329" PostTypeId="2" ParentId="13319" CreationDate="2020-05-20T05:44:40.123" Score="4" Body="&lt;p&gt;Permutation as suggested by @StupidWolf's comment is essential to understand what's going on. If permutation makes this pattern go away, then you have a problem with your model specification, there's something uncorrected.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If your data are weird, well, that's just how they are. But this argues to me that something else is going on confounding your associations. Cryptic population structure in the data would lead to this kind of pattern, e.g. &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/164939v1.full.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. That is the first thing that most reviewers would say, I'd guess.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some random thoughts:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What does the distribution of your methylation data look like after normalization? &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your observed distribution is basically never on the expected line, which argues strongly for confounding. &lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How are you adjusting for non-independence of methylation types? &lt;strong&gt;Family-wise correction is likely not specific enough. You may have to estimate an identity-by-state SNP similarity matrix for your population and fit that as part of your model,&lt;/strong&gt; or at least fit 10-20 principal components to account for population structure.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How are you representing your methylation status? If it's binary you might look at &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S000292971600063X&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2020-05-20T16:27:13.883" LastActivityDate="2020-05-20T16:27:13.883" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13330" PostTypeId="2" ParentId="13321" CreationDate="2020-05-20T06:22:58.940" Score="1" Body="&lt;p&gt;Two nice tools for receptor-ligand interaction analysis are:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.cellphonedb.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;CellPhoneDB&lt;/a&gt; by the Teichmann lab, although this is not an R package...&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/saeyslab/nichenetr&quot; rel=&quot;nofollow noreferrer&quot;&gt;NicheNet&lt;/a&gt; by the Saeys group&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;Note that these tools are mainly meant for analysis of receptor-ligand interactions, although I think NicheNet also has some visualization options. CellPhoneDB needs to be run from a shell, but the results can easily be imported into R for visualization. You might check the papers on these 2 methods for some ideas of visualization. Combining Seurat's visualization options and ggplot2 should get you already pretty far.&lt;/p&gt;&#xA;" OwnerUserId="8777" LastActivityDate="2020-05-20T06:22:58.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13331" PostTypeId="2" ParentId="3568" CreationDate="2020-05-20T09:22:48.993" Score="3" Body="&lt;p&gt;I know this is an old post, but if this plot is from &lt;em&gt;after&lt;/em&gt; trimming I would suggest a different explanation: some trimming tools remove poly-A sequences from reads. If that's the case then any read ending with &lt;code&gt;A&lt;/code&gt; will have that removed, this leads to a 0% &lt;code&gt;A&lt;/code&gt; base content in the final base position (and a dropping %&lt;code&gt;A&lt;/code&gt; in the final couple of bases).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This should have no effect on alignments or analysis.&lt;/p&gt;&#xA;" OwnerUserId="224" LastActivityDate="2020-05-20T09:22:48.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13332" PostTypeId="1" AcceptedAnswerId="13336" CreationDate="2020-05-20T09:39:06.473" Score="0" ViewCount="614" Body="&lt;p&gt;I have been following the last DESeq2 pipeline to perform an RNAseq analysis with a dataset with low rin samples in the experimental (or treated) and high rin on the control ones. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I read &lt;a href=&quot;https://ncbi.nlm.nih.gov/pmc/articles/PMC4071332&quot; rel=&quot;nofollow noreferrer&quot;&gt;a paper&lt;/a&gt; in which they perform RNAseq analysis with time-course RNA degradation and conclude that including RIN value as a covariate can mitigate some of the effects of low RIN in samples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is how I should construct the design in the DESeq2 object:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;~conditions+rin&#xA;~conditions*rin&#xA;~conditions:rin&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;none of them... :)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I cannot find proper resources where explain how to construct these models (I am new to the field...) and I recognise I crashed against a wall with these kinds of things. &lt;strong&gt;I would appreciate also some links to good resources to be able to understand which one is correct and why&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you in advance! &lt;/p&gt;&#xA;" OwnerUserId="8886" LastEditorUserId="57" LastEditDate="2020-05-25T14:09:42.133" LastActivityDate="2020-05-25T14:09:42.133" Title="How to incorportate RIN values as covariate in the design matrix?" Tags="|rna-seq|deseq2|edger|design|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13333" PostTypeId="1" CreationDate="2020-05-20T09:58:56.127" Score="1" ViewCount="25" Body="&lt;p&gt;I am a student and just trying to gain a base level understanding of PPI databases and how to interpret their data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, when inputing multiple genes into String, does it only give protein-protein interactions between those identified in my gene list? Or does it compare the input gene list to a background of your selected species? If it's the former, is it normal to only want protein-protein interactions between those in your input list? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there a resource/text where I can find more information about PPI databases? Any help in the right direction or where I can learn more would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="4242" LastActivityDate="2020-05-20T09:58:56.127" Title="How do protein-protein interaction databases work?" Tags="|database|public-databases|proteomics|protein-protein-interaction|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13336" PostTypeId="2" ParentId="13332" CreationDate="2020-05-20T12:09:48.090" Score="4" Body="&lt;p&gt;In the DESeq2 &lt;a href=&quot;http://bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#how-can-i-include-a-continuous-covariate-in-the-design-formula&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt; there is a section titled &quot;How can I include a continuous covariate in the design formula?&quot; that deals with your question.&lt;br&gt;&#xA;Basically the process is no different from using a discrete covariate.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;DESeq &amp;lt;- DESeqDataSetFromMatrix(countData = counts, colData = metadata, design = ~ RIN + condition)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In this example condition is the experimental groups and RIN is the continuous covariate for the RNA-integrity.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For this to work properly there has to be a linear relationship between the RNA integrity and the gene expression values. As stated in the manual: &quot;Continuous covariates might make sense in certain experiments, where a constant fold change might be expected for each unit of the covariate.&quot;&lt;br&gt;&#xA;Therefore, it may be better to code the RIN variable as a factor like &lt;code&gt;RIN_high&lt;/code&gt; or &lt;code&gt;RIN_low&lt;/code&gt;. This is more robust than using the absolute values of the variable itself.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, from your desription I get the impession that the RIN values are generally lower in the treatment group compared to the control group. If this is the case, using RIN as a covariate will by of limited use. DESeq2 will not be able to discriminate if a expression difference is due to a difference in RNA integrity or due to your treatment. Also, in this case coding RIN into high and low would result in an error because the RIN and treatment variable would define the same groups.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally, covariates work best when they are evenly distributed across the experimental groups.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2020-05-20T12:09:48.090" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13337" PostTypeId="2" ParentId="13327" CreationDate="2020-05-20T12:36:11.130" Score="3" Body="&lt;p&gt;You can use the &lt;code&gt;cells&lt;/code&gt; argument to only display the data for a specific subset of cells.  &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example to extract the values for a cluster 2:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# The cluster identity is stored in the meta.data&#xA;# In this example there is a clustering column called &quot;RNA_snn_res.0.7&quot; &#xA;# but any metadata column can be used&#xA;&#xA;clust2.cells &amp;lt;- rownames(Seurat.data@meta.data[Seurat.data$RNA_snn_res.0.7 == 2, ])&#xA;&#xA;FeatureScatter(Seurat.data, feature1 = &quot;gene_A&quot;, feature2 = &quot;gene_B&quot;, cells = clust2.cells)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4885" LastEditorUserId="4885" LastEditDate="2020-05-20T14:26:02.863" LastActivityDate="2020-05-20T14:26:02.863" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13338" PostTypeId="1" AcceptedAnswerId="13354" CreationDate="2020-05-20T13:15:16.960" Score="1" ViewCount="340" Body="&lt;p&gt;I have lot of SMILES IDs of the ligands. What I am interested in is the polarity and hydrophobicity score of each of the ligands.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In other words, does ligand have more polar groups and can it make more polar interactions or is it more hydrophobic?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Does anybody know the way/software to look at it?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Best&lt;/p&gt;&#xA;" OwnerUserId="8598" LastActivityDate="2020-05-21T17:37:15.730" Title="Calculating polarity and hydrophobicity of a ligand/small molecule" Tags="|database|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13339" PostTypeId="1" CreationDate="2020-05-20T14:15:05.607" Score="0" ViewCount="518" Body="&lt;p&gt;I am using a Genome Explorer tool to see all the mutations on my own DNA.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My particular interest is on listing the variants and get their names/ids.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here are a few screenshots:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XuJI6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XuJI6.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OL31u.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OL31u.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can see that they tell you the genomic position, the allele frequency of the mutation, the name of the disease, the Category, etc etc...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, the filters on this tool work very badly so I cannot filter all my mutations only...&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I was trying to find out an alternative tool to get this data in such a nice way this tool provides. I tried IGV but they just provide raw readings, I didnt find a way to get a list of variants the same way this Genome Explorer does.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The closest thing I've found was Geneious: &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZECl7.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZECl7.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;but it crashes on my computer and is too complex to use (I wasn't able to reach that screen above, from my BAM file). And it also eats a lot of RAM that I don't have.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Are there any alternatives? Even paying?&lt;/p&gt;&#xA;" OwnerUserId="8577" LastActivityDate="2020-06-20T00:04:55.460" Title="How can I extract all known mutations of my BAM (or SNP/INDL files)?" Tags="|snp|ngs|variant-calling|phylogenetics|indel|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13340" PostTypeId="1" AcceptedAnswerId="13351" CreationDate="2020-05-20T20:28:57.430" Score="0" ViewCount="1702" Body="&lt;p&gt;Nowadays, I am trying to calculate Pearson correlation values between two genes of my interest from single-cell RNA-data (features.tsv, barcodes.tsv, and matrix.mtx files) which are obtained from the 10xgenomics cellranger. I am interested in tumor-infiltrating Tregs. I downloaded fastq files from a published paper that describes tumor-infiltrating Tregs with IL10+IL35+ (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE126184&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE126184&lt;/a&gt;), and ran &quot;cellranger count&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Yesterday, I learned how to get a Pearson correlation value from a kind guy. I made a graph below.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bgVbh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bgVbh.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In general, Tregs are defined by CD4 and Foxp3 expression. As a trial, I want to find the correlation between two genes. The correlation value was 0.05, which might have been due to the presence of Tregs that have zero value of either CD4 or Foxp3. So, my question is that can we remove cells with 0 value of genes to get more high correlation value? My data looks like this.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;&amp;gt; Treg.data[c(&quot;Cd4&quot;, &quot;Foxp3&quot;), 1:30]&#xA;&#xA;------------------------------------- # results&#xA;&#xA;2 x 30 sparse Matrix of class &quot;dgCMatrix&quot;&#xA;   [[ suppressing 30 column names ‘AAACCTGAGAAACCAT-1’, ‘AAACCTGAGAAACCGC-1’, ‘AAACCTGAGAAACCTA-1’ ... ]]&#xA;&#xA;Cd4   . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .&#xA;&#xA;Foxp3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .&#xA;&#xA;&#xA;------------------------------------- # results&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Each dot represents 0 value. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Meanwhile, I performed to standard pre-processing workflow, as suggested in Seural Tutorial. For example, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; Treg[[&quot;percent.mt&quot;]] &amp;lt;- PercentageFeatureSet(Treg, pattern = &quot;^MT-&quot;)&#xA;&#xA;&amp;gt; Treg &amp;lt;- subset(Treg, subset = nFeature_RNA &amp;gt; 200 &amp;amp; nFeature_RNA &amp;lt; 2500 &amp;amp; percent.mt &amp;lt; 5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Can you help me? Thank you very much.&lt;/p&gt;&#xA;" OwnerUserId="8657" LastEditorUserId="48" LastEditDate="2020-05-21T09:38:05.117" LastActivityDate="2020-05-21T17:41:29.003" Title="How to remove zero value of gene on FeatureScatter plot using Seurat?" Tags="|r|scrnaseq|seurat|correlation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13341" PostTypeId="1" AcceptedAnswerId="15964" CreationDate="2020-05-20T21:29:55.377" Score="-3" ViewCount="113" Body="&lt;p&gt;I have a data frame one column is sample name like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Sample&#xA;&#xA;&#xA;TCGA-VR-A8EX-01A-11D-A36I-01&#xA;TCGA-L7-A56G-01A-21D-A25X-01&#xA;TCGA-VR-A8ER-01A-11D-A36I-01&#xA;TCGA-JY-A6FB-01A-11D-A33D-01&#xA;TCGA-2H-A9GO-01A-11D-A37B-01&#xA;TCGA-Z6-AAPN-01A-11D-A402-01&#xA;TCGA-L5-A4OP-01A-11D-A25X-01&#xA;TCGA-L5-A8NV-01A-11D-A37B-01&#xA;TCGA-R6-A8W5-01B-11D-A37B-01&#xA;TCGA-S8-A6BW-01A-11D-A31T-01&#xA;TCGA-R6-A6Y0-01B-11D-A33D-01&#xA;TCGA-R6-A8WC-01A-11D-A37B-01&#xA;TCGA-S8-A6BV-01A-21D-A31T-01&#xA;TCGA-L5-A4OJ-01A-11D-A25X-01&#xA;TCGA-LN-A7HV-01A-21D-A350-01&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to subset samples which have partial match with these name&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TCGA-IC-A6RE&#xA;TCGA-IG-A4QS&#xA;TCGA-JY-A6F8&#xA;TCGA-JY-A6FB&#xA;TCGA-L5-A43E&#xA;TCGA-L5-A4OG&#xA;TCGA-L5-A4OH&#xA;TCGA-L5-A4OJ&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have used&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data[grepl(c(&amp;quot;TCGA-IC-A6RE&amp;quot;, &amp;quot;TCGA-IG-A4QS&amp;quot;, &amp;quot;TCGA-JY-A6F8&amp;quot;, &amp;quot;TCGA-JY-A6FB&amp;quot;, &amp;quot;TCGA-L5-A43E&amp;quot;, &amp;quot;TCGA-L5-A4OG&amp;quot;, &amp;quot;TCGA-L5-A4OH&amp;quot;, &amp;quot;TCGA-L5-A4OJ&amp;quot;, &amp;quot;TCGA-L5-A4ON&amp;quot;, &amp;quot;TCGA-L5-A4OS&amp;quot;, &amp;quot;TCGA-L5-A4OW&amp;quot;, &amp;quot;TCGA-L5-A4OX&amp;quot;, &amp;quot;TCGA-L5-A88T&amp;quot;, &amp;quot;TCGA-Q9-A6FW&amp;quot;),data$Sample),]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But says that&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Warning message:&#xA;In grepl(c(&amp;quot;TCGA-IC-A6RE&amp;quot;, &amp;quot;TCGA-IG-A4QS&amp;quot;, &amp;quot;TCGA-JY-A6F8&amp;quot;, &amp;quot;TCGA-JY-A6FB&amp;quot;,  :&#xA;  argument 'pattern' has length &amp;gt; 1 and only the first element will be used&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Likely only the first entry from the vector being grepped&lt;/p&gt;&#xA;&lt;p&gt;Edit: As @StupidWolf suggested, I have tried:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; data[substr(data$Sample, 1, 12) %in% &#xA;    c(&amp;quot;TCGA-IC-A6RE&amp;quot;, &amp;quot;TCGA-IG-A4QS&amp;quot;, &amp;quot;TCGA-JY-A6F8&amp;quot;, &amp;quot;TCGA-JY-A6FB&amp;quot;,&#xA;      &amp;quot;TCGA-L5-A43E&amp;quot;, &amp;quot;TCGA-L5-A4OG&amp;quot;, &amp;quot;TCGA-L5-A4OH&amp;quot;, &amp;quot;TCGA-L5-A4OJ&amp;quot;,&#xA;      &amp;quot;TCGA-L5-A4ON&amp;quot;, &amp;quot;TCGA-L5-A4OS&amp;quot;, &amp;quot;TCGA-L5-A4OW&amp;quot;, &amp;quot;TCGA-L5-A4OX&amp;quot;,&#xA;      &amp;quot;TCGA-L5-A88T&amp;quot;, &amp;quot;TCGA-Q9-A6FW&amp;quot;),]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This works, thanks.&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="73" LastEditDate="2021-05-23T13:11:49.637" LastActivityDate="2021-05-23T13:17:53.187" Title="Selecting rows by partial match" Tags="|r|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13342" PostTypeId="1" AcceptedAnswerId="13348" CreationDate="2020-05-20T21:55:30.537" Score="0" ViewCount="1803" Body="&lt;p&gt;I have a multi fasta file similar to this (relatively new here so uncertain of best way to present this; I have gone for an output and the code i used to make it - belt and braces...):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# create a test dataset&#xA;with open('test_dat.txt', 'w') as test_dat:&#xA;    l1 = [&quot;&amp;gt;AAA a aa&quot;, &quot;&amp;gt;BBB b bb&quot;, &quot;&amp;gt;CCC c cc&quot;]&#xA;    l2 = [&quot;ATGATG&quot;, &quot;GACGAC&quot;, &quot;TTGTTG&quot;]&#xA;&#xA;    for i in range(len(l1)):&#xA;        test_dat.write(l1[i] + &quot;\n&quot;)&#xA;        test_dat.write(l2[i] + &quot;\n&quot;)&#xA;&#xA;# which outputs something like this&#xA;&amp;gt;AAA a aa&#xA;ATGATG&#xA;&amp;gt;BBB b bb&#xA;GACGAC&#xA;&amp;gt;CCC c cc&#xA;TTGTTG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to modify the fasta headers in my fasta file by adding some text to the beginning and, ideally, stripping out spaces (as i was advised that white space in the headings could cause problems for me down the line). My plan was to do this using BioPython and SeqIO.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I started like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# my attempt at altering the fasta headers using SeqIO&#xA;from Bio import SeqIO&#xA;&#xA;# open my test data and create an output file&#xA;with open('test_dat.txt', 'r') as inputs, open('test_dat_out.txt', 'w') as outputs:&#xA;&#xA;    # create dictionary of the fasta sequences using SeqIO&#xA;    my_dict = SeqIO.to_dict(SeqIO.parse(inputs, &quot;fasta&quot;))&#xA;&#xA;    # renaming my fasta headers - adding &quot;TEST~~~TEST~~~&quot; to front of header and remove white space&#xA;    for v in my_dict.values(): # using the values() to loop through...&#xA;        v.description = &quot;TEST~~~TEST~~~&quot; + v.description.replace(&quot; &quot;, &quot;_&quot;)&#xA;&#xA;    # write result to file&#xA;    SeqIO.write(my_dict.values(), outputs, 'fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;which gave me a result with some duplication in the fasta header:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;#result i want&#xA;&amp;gt;TEST~~~TEST~~~AAA_a_aa&#xA;ATGATG&#xA;&#xA;#result i get&#xA;&amp;gt;AAA TEST~~~TEST~~~AAA_a_aa&#xA;ATGATG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If i modify both the &lt;code&gt;.id&lt;/code&gt; and &lt;code&gt;.description&lt;/code&gt; in exactly the same way, as suggested &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/307/changing-the-record-id-in-a-fasta-file-using-biopython&quot;&gt;here&lt;/a&gt;, but &lt;strong&gt;without&lt;/strong&gt; removing white space:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# renaming my fasta headers - adding &quot;TEST~~~TEST~~~&quot; to front of header&#xA;    for v in my_dict.values(): # using the values() to loop through...&#xA;        v.id = &quot;TEST~~~TEST~~~&quot; + v.id&#xA;        v.description = &quot;TEST~~~TEST~~~&quot; + v.description&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get almost what i want, but the header still has spaces:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;TEST~~~TEST~~~AAA a aa&#xA;ATGATG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have tried to make the &lt;code&gt;.id&lt;/code&gt; and &lt;code&gt;.description&lt;/code&gt; match by adding a &quot;_&quot; to the end of &lt;code&gt;.id&lt;/code&gt;, or by adding &lt;code&gt;.replace(&quot; &quot;, &quot;_&quot;)&lt;/code&gt; to both &lt;code&gt;.id&lt;/code&gt; and &lt;code&gt;.description&lt;/code&gt; (despite there being no spaces in &lt;code&gt;.id&lt;/code&gt;), but I just cant seem to get exactly the output i want.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would really appreciate any pointers on this.&lt;/p&gt;&#xA;" OwnerUserId="6738" LastActivityDate="2020-05-21T12:31:57.140" Title="How can i modify FASTA headers in a multi fasta file using BioPython SeqIO" Tags="|fasta|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13343" PostTypeId="1" AcceptedAnswerId="13356" CreationDate="2020-05-20T22:14:41.630" Score="2" ViewCount="601" Body="&lt;p&gt;I am using &lt;a href=&quot;http://mordred.bioc.cam.ac.uk/~rapper/rampage.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rampage webserver&lt;/a&gt; to obtain a Ramachandran plot. However, the server does not reveal any plot just values (favored, allowed, outlier regions).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you recommend other similar web sites?&lt;/p&gt;&#xA;" OwnerUserId="8588" LastEditorUserId="48" LastEditDate="2020-05-21T09:23:29.150" LastActivityDate="2020-05-21T23:04:14.170" Title="Where can I visualize Ramachandran plot?" Tags="|3d-structure|validation|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13344" PostTypeId="2" ParentId="13339" CreationDate="2020-05-20T23:48:37.423" Score="1" Body="&lt;p&gt;If you have a BAM file, you could easily run a variant caller analysis software such as freebayes, though you would need a good amount of computing power like a cluster or a dedicated node with good memory. Freebayes outputs a vcf file that has all the variants listed. Then you can load it into the UCSC or Ensembl human genome browsers and look at what genes they are. &lt;/p&gt;&#xA;" OwnerUserId="8893" LastActivityDate="2020-05-20T23:48:37.423" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13345" PostTypeId="1" CreationDate="2020-05-21T00:43:12.570" Score="0" ViewCount="81" Body="&lt;p&gt;I have been working on clustering using RNAseq data. To compute distance, what kinds of normalization is optimal?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can we use normalization using relative log normalization? I understand this should be used for differential gene expression analysis, not for unsupervised learning in which we do not use information of disease condition.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you can show good papers, please share.&lt;/p&gt;&#xA;" OwnerUserId="8895" LastActivityDate="2020-05-21T13:53:36.113" Title="about the normalization of RNAseq data for calculating distance for unsupervised learning" Tags="|rna|clustering|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13347" PostTypeId="1" AcceptedAnswerId="13352" CreationDate="2020-05-21T12:25:40.883" Score="1" ViewCount="208" Body="&lt;p&gt;I am trying to represent a large (100-300+ sequences) phylogenetic tree in a way that gets my point across. I figured that reducing the number of branches (by collapsing clades which include highly similar sequences) would be a good idea. The software that I'm currently using (iTOL) technically has an option for that ('collapse node'), but I'm not exactly a fan of the particular graphical representation, so I'd like to have a look at other options. Thank you for your time!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastEditorUserId="3848" LastEditDate="2023-07-15T16:52:09.083" LastActivityDate="2023-07-15T16:52:09.083" Title="Phylogenetics software that can represent collapsed nodes as triangles" Tags="|r|python|phylogenetics|phylogeny|visualization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13348" PostTypeId="2" ParentId="13342" CreationDate="2020-05-21T12:31:57.140" Score="3" Body="&lt;p&gt;You're very close, but unfortunately for legacy reasons this aspect of the Biopython parser is not very intuitive. The key point is the &lt;code&gt;.id&lt;/code&gt; is anything up until the first white space character. One day I hope the &lt;code&gt;.description&lt;/code&gt; and &lt;code&gt;.id&lt;/code&gt; attributes are replaced by a single attribute like &lt;code&gt;.header&lt;/code&gt;. Anyway, here is a solution for your case:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;to_add = &quot;TEST~~~TEST~~~&quot;&#xA;&#xA;with open(&quot;out.fa&quot;, &quot;w&quot;) as outputs:&#xA;    for r in SeqIO.parse(&quot;in.fa&quot;, &quot;fasta&quot;):&#xA;        r.id = (to_add + r.description).replace(&quot; &quot;, &quot;_&quot;)&#xA;        r.description = r.id&#xA;        SeqIO.write(r, outputs, &quot;fasta&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output on your sample input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;TEST~~~TEST~~~AAA_a_aa&#xA;ATGATG&#xA;&amp;gt;TEST~~~TEST~~~BBB_b_bb&#xA;GACGAC&#xA;&amp;gt;TEST~~~TEST~~~CCC_c_cc&#xA;TTGTTG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastActivityDate="2020-05-21T12:31:57.140" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13349" PostTypeId="2" ParentId="13345" CreationDate="2020-05-21T13:53:36.113" Score="2" Body="&lt;p&gt;Typically some sort of variance stabilizing transform is used before clustering. Popular options are regularized log transformation or a vst transform, which are available in DESeq2. Note that these are NOT used for performing differential expression, just things like clustering and PCA. For differential expression one would use TMM, RLE or similar.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-05-21T13:53:36.113" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13350" PostTypeId="2" ParentId="13341" CreationDate="2020-05-21T14:20:23.907" Score="2" Body="&lt;p&gt;Not strictly bioinformatics, but a common problem:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;grep&lt;/code&gt; assumes a regular expression, not a vector of patterns.&#xA;Assuming your patterns are stored as a vector, so e.g.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;patt &amp;lt;- c(&quot;pattern1&quot;, &quot;pattern2&quot;, &quot;pattern3&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;you first have to write them as a regex:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;patt.regex &amp;lt;- paste(patt, collapse = &quot;|&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Output of &lt;code&gt;patt.regex&lt;/code&gt; is then &lt;code&gt;&quot;pattern1|pattern2|pattern3&quot;&lt;/code&gt; which is now ready for &lt;code&gt;grep&lt;/code&gt; or &lt;code&gt;grepl&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep(patt.regex, data$Sample)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;For exact matches it would be &lt;code&gt;paste(paste0(&quot;^&quot;, patt, &quot;$&quot;), collapse = &quot;|&quot;)&lt;/code&gt;&#xA;to get &lt;code&gt;&quot;^pattern1$|^pattern2$|^pattern3$&quot;&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-05-21T14:20:23.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13351" PostTypeId="2" ParentId="13340" CreationDate="2020-05-21T14:37:27.003" Score="1" Body="&lt;p&gt;It will not improve your correlation most likely, unless the zeros are not missing at random:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Seurat)&#xA;data &amp;lt;- pbmc_small&#xA;data[[&quot;percent.mt&quot;]] &amp;lt;- PercentageFeatureSet(data, pattern = &quot;^MT-&quot;)&#xA;data &amp;lt;- NormalizeData(data)&#xA;genes &amp;lt;-c(&quot;HLA-DRA&quot;,&quot;LYZ&quot;)&#xA;countdata = pbmc_small[[&quot;RNA&quot;]]@data&#xA;countdata &amp;lt;- t(as.matrix(countdata)[genes,])&#xA;plot(countdata)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ku1se.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ku1se.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now you need to subset the matrix, and remove the rows where at least one of the entries is 0, you can use rowSums(..==0)==0, meaning no zeros in the rows:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plot(countdata[rowSums(countdata==0)==0,])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Xsbyw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Xsbyw.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cor(countdata[rowSums(countdata==0)==0,])&#xA;          HLA-DRA       LYZ&#xA;HLA-DRA 1.0000000 0.2338371&#xA;LYZ     0.2338371 1.0000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastEditorUserId="5077" LastEditDate="2020-05-21T17:41:29.003" LastActivityDate="2020-05-21T17:41:29.003" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="13352" PostTypeId="2" ParentId="13347" CreationDate="2020-05-21T16:58:32.197" Score="2" Body="&lt;p&gt;For Mac Figtree &lt;a href=&quot;http://tree.bio.ed.ac.uk/software/figtree/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is highly recommended, it is also available for Windows and Linux &lt;a href=&quot;https://github.com/rambaut/figtree/releases&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The technique is called &quot;collapsing clades&quot; and is the only package I know that does it. I would recommend using it, because I would assume it is averaging the branch length of a given clade and the 'base' of the triangle is proportional to the number of taxa. Thus it recognised as being very reliable, albeit the precise algorhithm it uses ... I don't know :-O&lt;/p&gt;&#xA;&#xA;&lt;p&gt;With FigTree you get two different &quot;triangle options&quot; in the menu, which give different shaped triangles for collapsing the number of lineages within the select clade you chose. The big one, is the algorhithm appropriately described above. It has many options for producing publication quality diagrams, albeit I then import the output into a graphics program for further editing (there are shortcomings IMO in colouring and fonts. Then again for free is better than good.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you are outputing a tree within a script, ETE3 (Python) will do a &quot;collapsing of clade&quot; option, but it isn't pretty triangle stuff. ETE3 will produce a publication quality tree diagram from this and reduce to 'low resolution', which is very useful in large analyses of loads of trees. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, I don't think FigTree has any competition in the context you are seeking.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Moreover, you can configure your bin, or alias file so you can automatically open a treefile just typing, &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;figtree treefile.tre&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Also you can reduce you font size to next point 1 or 2 (okay you can do that in anything), if you are attempting optical analysis on loads of taxa. Its also really stable. I just find it really useful, like a &quot;jazz standard&quot; its a &quot;phylogeny standard&quot;.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-05-21T19:56:34.113" LastActivityDate="2020-05-21T19:56:34.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13354" PostTypeId="2" ParentId="13338" CreationDate="2020-05-21T17:37:15.730" Score="1" Body="&lt;p&gt;You can calculate their polar surface area or logo using rdkit or openbabel. Other molecular properties may be available as well. &lt;/p&gt;&#xA;" OwnerUserId="3861" LastActivityDate="2020-05-21T17:37:15.730" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13356" PostTypeId="2" ParentId="13343" CreationDate="2020-05-21T23:04:14.170" Score="4" Body="&lt;p&gt;&lt;a href=&quot;https://warwick.ac.uk/fac/sci/moac/people/students/peter_cock/python/ramachandran/other/&quot; rel=&quot;noreferrer&quot;&gt;Here&lt;/a&gt; is a page with several suggested methods. I agree that Rampage looks the best of the options, but it seems that it provides some &lt;a href=&quot;https://warwick.ac.uk/fac/sci/moac/people/students/peter_cock/r/ramachandran/&quot; rel=&quot;noreferrer&quot;&gt;suggested workflows for drawing it in R&lt;/a&gt; so that it is very customizable and you can probably get it to show any features you want if you're willing to customize a little.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Here is a &lt;a href=&quot;https://pypi.org/project/RamachanDraw/&quot; rel=&quot;noreferrer&quot;&gt;Python tool&lt;/a&gt; for drawing Ramachandran plots.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are additionally &lt;a href=&quot;http://services.mbi.ucla.edu/SAVES/Ramachandran/&quot; rel=&quot;noreferrer&quot;&gt;several&lt;/a&gt; &lt;a href=&quot;https://spdbv.vital-it.ch/rama.html&quot; rel=&quot;noreferrer&quot;&gt;other&lt;/a&gt; &lt;a href=&quot;https://swift.cmbi.umcn.nl/servers/html/ramaplot.html&quot; rel=&quot;noreferrer&quot;&gt;servers&lt;/a&gt;/tools.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-05-21T23:04:14.170" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13357" PostTypeId="1" CreationDate="2020-05-21T23:36:14.673" Score="1" ViewCount="106" Body="&lt;p&gt;It may better to move the &lt;a href=&quot;https://biology.stackexchange.com/questions/93654/available-protein-sequence-alignment-dataset-and-hmm-model&quot;&gt;question&lt;/a&gt; here.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am new to biology and I find my algorithm may be used in the Protein sequence alignment, since it is a henced HMM model. I find that people use HMM to generate noisy copies of the consensus sequence of different lengths. There is a figure show the process: &#xA;&lt;a href=&quot;https://i.stack.imgur.com/X1J65.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/X1J65.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am familiar with the machine learning or graphical model. To build the HMM, we need to set the model parameter, which we learn from the training data in ML. My question is whether the train&amp;amp;testing works for this question? If so where can I find data to training this HMM model?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems Professor Richard Durbin may release some dataset. But how can I find available datasets? I lost in the bio-vocabulary and fail to find datasets. Also I am wondering whether or not this topic is very minor in this community?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Update: &#xA;Since I may abuse the vocabulary, it may be better to post the original word I read.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/KPUHZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KPUHZ.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8899" LastActivityDate="2020-05-21T23:36:14.673" Title="Available Protein sequence alignment dataset and HMM model" Tags="|sequence-alignment|proteins|sequence-analysis|machine-learning|hidden-markov-models|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13358" PostTypeId="2" ParentId="13341" CreationDate="2020-05-22T07:56:01.127" Score="0" Body="&lt;p&gt;When the string follows certain format there is no need for partial match, see this example:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# example data&#xA;data &amp;lt;- read.table(text = &quot;&#xA;Sample&#xA;TCGA-VR-A8EX-01A-11D-A36I-01&#xA;TCGA-L7-A56G-01A-21D-A25X-01&#xA;TCGA-VR-A8ER-01A-11D-A36I-01&#xA;TCGA-JY-A6FB-01A-11D-A33D-01&#xA;&quot;, header = TRUE, stringsAsFactors = FALSE)&#xA;&#xA;x &amp;lt;- c(&quot;TCGA-VR-A8EX&quot;, &quot;TCGA-VR-A8ER&quot;)&#xA;&#xA;data[ substr(data$Sample, 1, 12) %in% x, &quot;Sample&quot;, drop = FALSE]&#xA;#                         Sample&#xA;# 1 TCGA-VR-A8EX-01A-11D-A36I-01&#xA;# 3 TCGA-VR-A8ER-01A-11D-A36I-01&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or when sample IDs lengths are not the same we can split on &quot;-&quot; then paste it back first 3 items, then subset as usual with &lt;code&gt;%in%&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;data[ sapply(strsplit(data$Sample, split = &quot;-&quot;), &#xA;             function(i) paste(i[1:3], collapse = &quot;-&quot;)&#xA;             ) %in% x, &quot;Sample&quot;, drop = FALSE]&#xA;#                         Sample&#xA;# 1 TCGA-VR-A8EX-01A-11D-A36I-01&#xA;# 3 TCGA-VR-A8ER-01A-11D-A36I-01&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="131" LastActivityDate="2020-05-22T07:56:01.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13360" PostTypeId="2" ParentId="12810" CreationDate="2020-05-22T08:59:04.577" Score="1" Body="&lt;p&gt;By default, &lt;a href=&quot;https://www.rdocumentation.org/packages/Seurat/versions/3.1.4/topics/DimPlot&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;DimPlot&lt;/code&gt;&lt;/a&gt; colors the cells by their &quot;identity class&quot;, stored in &lt;code&gt;Ident(scData)&lt;/code&gt; in your case. This indeed contains the clustering information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, you can you use the &lt;code&gt;group.by&lt;/code&gt; argument for &lt;code&gt;DimPlot&lt;/code&gt; to pass another column from &lt;code&gt;scData@meta.data&lt;/code&gt; to color cells by. In your case you would need a metadata column that stores the &lt;em&gt;ptx&lt;/em&gt;/&lt;em&gt;human&lt;/em&gt; information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://satijalab.org/seurat/v3.1/merge_vignette.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;merge&lt;/code&gt;&lt;/a&gt; function automatically stores the original &quot;project&quot; information of the individual Seurat objects that you combine under &lt;code&gt;orig.ident&lt;/code&gt; in the metadata. Since you set the project names to either &lt;code&gt;&quot;ptx&quot;&lt;/code&gt; or &lt;code&gt;&quot;human&quot;&lt;/code&gt;, the &lt;code&gt;orig.ident&lt;/code&gt; column should contain the information you need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Something like this should work:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;DimPlot(scData, reduction = &quot;umap&quot;, group.by = &quot;orig.ident&quot;, label = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8777" LastActivityDate="2020-05-22T08:59:04.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13361" PostTypeId="2" ParentId="13294" CreationDate="2020-05-22T14:09:40.850" Score="1" Body="&lt;p&gt;Thanks a lot to all who commented/replied, the author of the scripts provided a response. It was down to a syntax change in PyMol of which I wasn't aware that caused my selection to come up empty.&lt;/p&gt;&#xA;" OwnerUserId="8863" LastActivityDate="2020-05-22T14:09:40.850" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13363" PostTypeId="1" CreationDate="2020-05-22T18:08:16.843" Score="1" ViewCount="51" Body="&lt;p&gt;I'm using BLAT to align DNA sequences between similar species, and having trouble setting the parameters to return only significant matches along the 50bp query lengths.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using &lt;code&gt;-tilesize=11 -minMatch=3 -minScore=30 -maxGap=2&lt;/code&gt; my understanding is that it should return alignments with at least 3 non-overlapping tiles of 11 bases, so 33 bases minimum, with gaps of up to 2 bases between tiles, and bitscore at least 30. Instead the output is showing alignments as short as 4 bases and bitscore as low as 8. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried setting &lt;code&gt;-minMatch=5&lt;/code&gt; which should not find any 55bp matches in my 50bp queries, but even that gives a handful of results, with alignment length as low as 6. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can anyone enlighten me on the disconnect here? Thank you!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT: I think I see the problem: BLAT reports on a separate line for each fragment of the match. For example, one probe has a match from 1-41 and a match from 44-50, giving scores of 77 and 13. Now I will have a different set of questions!&lt;/p&gt;&#xA;" OwnerUserId="3505" LastEditorUserId="3505" LastEditDate="2020-05-25T19:38:17.757" LastActivityDate="2020-05-25T19:38:17.757" Title="setting BLAT alignment parameters to give long matches" Tags="|sequence-alignment|blat|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13365" PostTypeId="1" AcceptedAnswerId="13369" CreationDate="2020-05-23T02:51:19.510" Score="1" ViewCount="63" Body="&lt;p&gt;I manually selected all 4000 sequences on the 20 pages of the SARS-CoV-2 sequences on genbank &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/labs/virus/vssi/#/virus?SeqType_s=Nucleotide&amp;amp;VirusLineage_ss=Severe%20acute%20respiratory%20syndrome%20coronavirus%202,%20taxid:2697049&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;When I click on &lt;code&gt;Build phylogenetic tree&lt;/code&gt;, it only has 200 nodes.&lt;/p&gt;&#xA;&lt;p&gt;Is this expected? I was expecting to see 4000 nodes.&lt;/p&gt;&#xA;&lt;p&gt;Is it simply a displaying issue where only 200 sequences per page are revealed while browsing the list of sequences? Is there a way to get the tree for all 4000 sequences?&lt;/p&gt;&#xA;" OwnerUserId="8908" LastEditorUserId="3848" LastEditDate="2020-12-04T18:18:53.463" LastActivityDate="2020-12-04T18:18:53.463" Title="Why does the Genbank phylogenetic tree of 4000 sars-cov-2 sequences only display 200 nodes?" Tags="|phylogeny|covid-19|sars-cov-2|genbank|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13366" PostTypeId="1" CreationDate="2020-05-23T15:59:46.737" Score="0" ViewCount="58" Body="&lt;p&gt;I am interested in finding an ortholog for a particular gene in fungi. &lt;/p&gt;&#xA;" OwnerUserId="8909" LastEditorUserId="8909" LastEditDate="2020-06-05T15:11:26.570" LastActivityDate="2020-06-05T15:11:26.570" Title="Finding orthologs" Tags="|orthologues|" AnswerCount="0" CommentCount="2" ClosedDate="2020-05-25T19:39:27.137" ContentLicense="CC BY-SA 4.0" />
  <row Id="13367" PostTypeId="1" CreationDate="2020-05-23T21:44:50.130" Score="0" ViewCount="31" Body="&lt;p&gt;I have done a study on isoform switch between different tissue type. Now, many people asking me to study alternative splicing also. But, dont they will give me same results? If there is a significant splicing difference; that will definitely leads to isoform switch. Or my understanding is incorrect ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4951" LastActivityDate="2020-05-23T21:44:50.130" Title="Do I need to study alternative splicing and isoformswitch separately?" Tags="|differential-expression|rna-splicing|isoform|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13369" PostTypeId="2" ParentId="13365" CreationDate="2020-05-24T07:59:26.250" Score="2" Body="&lt;p&gt;Okay, I'll provide a formal answer, Genbank phylogeny is to get a gist of the diversity. It isn't a formal phylogenetic analysis. You can down the tree as a pdf, I don't think you can download the treefile (in fact I'm pretty certain you can't). Genbank will perform a reasonable distance method and from recollection it will &quot;collapse clades&quot;, so if you see little triangles on the tree it represents a large number of taxa represented as a triangle. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I was surprised to learn there are 4000 SARS-CoV-2 sequences. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;To make a tree you need to download the data, align (muscle) and send through a formal phylogeny program. If you think Genbank is okay, then try MegaX its very point and clicky. Generally, maximum likelihood phylogeny has returned to vogue for at least 1 decade now (previously Bayesian). However, the Beast package is a Bayesian analysis which is still going strong and the main tool now used in molecular dating. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;In summary, to perform a tree analysis beyond Genbank&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;use the download option (there are two variants - the entire genome or the cut locus based on the query sequence)&lt;/li&gt;&#xA;&lt;li&gt;align using muscle or clustalo (muscle is really quick(&lt;/li&gt;&#xA;&lt;li&gt;perform a tree analysis using MEGA X, phylml, raxml or Beast depending on what you objectives are&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-05-24T07:59:26.250" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13370" PostTypeId="1" CreationDate="2020-05-24T11:46:46.317" Score="0" ViewCount="1828" Body="&lt;p&gt;I am trying to follow a tutorial from Sanger institute (from May 2019) on analysis of single cell RNA Seq data. They use calculateQCMetric function to calculate the quality metrics, but I am getting an error message that calculateQCMetric is a deprecated (isSpike function is also deprecated). I want to calculate the quality metrics. Can anyone provide code for getting QCMetric step? I attached the previous defunct code below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;umi &amp;lt;- calculateQCMetrics(&#xA;   umi,&#xA;   feature_controls = list(&#xA;       ERCC = isSpike(umi, &quot;ERCC&quot;), &#xA;        MT = isSpike(umi, &quot;MT&quot;)&#xA;   )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And I use StupidWolf's code from this question &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/13295/isspike-function-in-singlecellexperiment-package-is-deprecated&quot;&gt;isSpike function in SingleCellExperiment package is deprecated?&lt;/a&gt; to instead isSpike function. I also attached below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Spikein_names = grep(&quot;^ERCC-&quot;,rownames(molecules),value=TRUE)&#xA;SpikeIn = molecules[Spikein_names,]&#xA;&#xA;g = genes(EnsDb.Hsapiens.v86)&#xA;MTgene_names = g[seqnames(g)==&quot;MT&quot;]$gene_id&#xA;MTgenes = molecules[rownames(molecules) %in% MTgene_names,]&#xA;Have a vector of genes to keep:&#xA;&#xA;keep = setdiff(rownames(molecules),c(SpikeIn,MTgenes))&#xA;Then make the single cell object:&#xA;&#xA;sce &amp;lt;- SingleCellExperiment(&#xA;assays=list(counts=as.matrix(molecules[keep,])),colData=annotation)&#xA;Use altExp to slot in the stuff:&#xA;&#xA;altExp(sce, &quot;spike-in&quot;) &amp;lt;- SummarizedExperiment(SpikeIn)&#xA;altExp(sce, &quot;MTgenes&quot;) &amp;lt;- SummarizedExperiment(MTgenes)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I try the below code, which did not work.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; perCellQCMetrics(sce,&#xA; subsets = list(Spikein_names,MTgene_names),&#xA; flatten = TRUE,&#xA; exprs_values = &quot;counts&quot;，&#xA; use_altexps = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance. &lt;/p&gt;&#xA;" OwnerUserId="8910" LastActivityDate="2020-12-21T20:05:25.440" Title="calculateQCMetrics defunct, how to calculate the quality metrics by perCellQCMetrics" Tags="|r|scrnaseq|bioconductor|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13371" PostTypeId="1" AcceptedAnswerId="13385" CreationDate="2020-05-24T18:47:29.017" Score="0" ViewCount="155" Body="&lt;p&gt;I thought that Needleman-Wunsch is the best approach to align sequences. However, I read that it is impossible to evaluate the statistical significance of the alignment if you do global alignment. So I would like to ask, what is the point of global alignment if we cannot evaluate statistical significance?&lt;/p&gt;&#xA;" OwnerUserId="8713" LastEditorUserId="77" LastEditDate="2020-05-25T16:53:27.490" LastActivityDate="2020-05-25T17:00:51.803" Title="Why use Needleman-Wunsch if there is no way to evaluate the statistical significance" Tags="|sequence-alignment|algorithms|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13372" PostTypeId="1" AcceptedAnswerId="13379" CreationDate="2020-05-24T20:00:29.613" Score="1" ViewCount="4499" Body="&lt;p&gt;I need to rename the following headers from a FASTA file. Something like:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;ENST00000488147.1::chr1:29570-30013(-)&#xA;&amp;gt;ENST00000405392.5::chr2:24919186-24920186(-)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;into different format like this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;chr1:29570-30013(-)_ENST00000488147.1&#xA;&amp;gt;chr2:24919186-24920186(-)_ENST00000405392.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can I rename the FASTA headers, perhaps using &lt;code&gt;sed&lt;/code&gt; or &lt;code&gt;awk&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="8912" LastEditorUserId="3541" LastEditDate="2020-05-26T12:26:32.013" LastActivityDate="2020-05-26T12:26:32.013" Title="Edit FASTA header using sed" Tags="|fasta|bash|linux|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13373" PostTypeId="2" ParentId="13372" CreationDate="2020-05-24T21:57:05.610" Score="8" Body="&lt;pre&gt;&lt;code&gt;awk 'BEGIN{FS=&quot;::&quot;}{if(&lt;span class=&quot;math-container&quot;&gt;$1~&quot;&amp;gt;&quot;){printf(&quot;&amp;gt;%s_%s\n&quot;,$2,substr($1,2))}else{print $&lt;/span&gt;0}}' input_file.fa &amp;gt; output.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;An explanation of the non-obvious bits:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;BEGIN{FS=&quot;::&quot;}&lt;/code&gt; split columns using :: as the designator&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$1~&quot;&amp;gt;&quot;&lt;/code&gt; If there's a &gt; in the line&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;substr($1,2)&lt;/code&gt; Trim the &lt;code&gt;&amp;gt;&lt;/code&gt; that was at the beginning of the line.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-05-24T21:57:05.610" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13374" PostTypeId="1" CreationDate="2020-05-25T01:09:47.870" Score="0" ViewCount="843" Body="&lt;p&gt;I run plotPCA for single cell RNA-seq data, while I get error message (I use R 4.0). I attached the code and error message here. Did anyone know the reason and how to deal with it. Thanks!&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(pcaMethods)&#xA;library(SC3)&#xA;library(scater)&#xA;library(SingleCellExperiment)&#xA;library(pheatmap)&#xA;library(mclust)&#xA;set.seed(1234567)&#xA;deng &amp;lt;- readRDS(&quot;deng-reads.rds&quot;)&#xA;plotPCA(deng, colour_by = &quot;cell_type2&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in value[[3L]](cond) : invalid subscript 'type' in 'reducedDimNames(&amp;lt;SingleCellExperiment&amp;gt;)'&#xA;'PCA' not in 'reducedDimNames(&amp;lt;SingleCellExperiment&amp;gt;)'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I also get error message when I run the code as below: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;colData(deng)$tSNE_kmeans &amp;lt;- as.character(kmeans(deng@reducedDims$TSNE, centers = 8)$clust)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;# Error in array(x, c(length(x), 1L), if (!is.null(names(x))) list(names(x), : 'data' must be of a vector type, was 'NULL'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8910" LastEditorUserId="8910" LastEditDate="2020-05-25T01:29:20.163" LastActivityDate="2020-05-25T06:37:19.573" Title="PCA plot did not work in single cell RNA-seq" Tags="|r|scrnaseq|bioconductor|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13376" PostTypeId="2" ParentId="13374" CreationDate="2020-05-25T06:37:19.573" Score="1" Body="&lt;p&gt;The error is pretty obvious. There is no PCA data in the respective slot of the SingleCellExperiment that &lt;code&gt;plotPCA&lt;/code&gt; could use, at least it is not called &lt;code&gt;PCA&lt;/code&gt; which by default is expected. Use &lt;code&gt;runPCA&lt;/code&gt; first, please read the manuals of &lt;code&gt;scater&lt;/code&gt; and &lt;code&gt;scran&lt;/code&gt; first, they have step-by-step code.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-05-25T06:37:19.573" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13377" PostTypeId="1" AcceptedAnswerId="13378" CreationDate="2020-05-25T07:44:42.823" Score="3" ViewCount="1174" Body="&lt;p&gt;So I want to compare the phylogeny created using two methods for example Maximum likelihood and maximum parsimony.Is there any way to compare the two phylogeny ?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did read about phangorn but not sure if its the right R library for comparative analysis.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion or helped would be highly appreciated &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My data &lt;a href=&quot;https://drive.google.com/open?id=1ErIpdS99Vu_tsWtvNQCKftzFmpJAUKuu&quot; rel=&quot;nofollow noreferrer&quot;&gt;file&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(phangorn)&#xA;library(phytools)&#xA;library(dendextend)&#xA;&#xA;&#xA;&#xA;data &amp;lt;- read.dna(&quot;abhi_seq/clean_dup_align_fast.fas&quot;, format = &quot;fasta&quot;)&#xA;data&#xA;dat &amp;lt;- as.phyDat(data)&#xA;&#xA;dm &amp;lt;- dist.ml(dat)&#xA;treeUPGMA &amp;lt;- upgma(dm)&#xA;treeNJ &amp;lt;- NJ(dm)&#xA;&#xA;layout(matrix(c(1,2), 2, 1), height=c(1,2))&#xA;par(mar = c(0,0,2,0)+ 0.1)&#xA;plot(treeUPGMA, main=&quot;UPGMA&quot;)&#xA;plot(treeNJ, &quot;phylogram&quot;, main=&quot;NJ&quot;)&#xA;&#xA;dev.off()&#xA;&#xA;parsimony(treeUPGMA, dat)&#xA;parsimony(treeNJ,dat)&#xA;&#xA;tr.mp &amp;lt;- optim.parsimony(treeNJ, dat)&#xA;&#xA;#tr.ml = optim.pml(treeNJ, dat)&#xA;&#xA;&#xA;&#xA;&#xA;fit &amp;lt;- pml(treeNJ, dat)&#xA;fit &amp;lt;- optim.pml(fit, rearrangement=&quot;NNI&quot;)&#xA;&#xA;&#xA;&#xA;&#xA;fit.ini &amp;lt;- pml(treeNJ, dat)&#xA;fit.ini&#xA;&#xA;&#xA;fit &amp;lt;- optim.pml(fit.ini, optNni=TRUE, optBf=TRUE, optQ=TRUE, optGamma=TRUE)&#xA;fit&#xA;&#xA;tr.ml &amp;lt;- root(fit$tree,1)&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;tr.mp.ultra&amp;lt;-force.ultrametric(tr.mp)&#xA;tr.ml.ultra&amp;lt;-force.ultrametric(tr.ml)&#xA;&#xA;is.ultrametric(tr.mp.ultra)&#xA;is.binary.tree(tr.mp.ultra)&#xA;is.rooted(tr.mp.ultra)&#xA;&#xA;&#xA;&#xA;dd.ml.ultra&amp;lt;-as.dendrogram(tr.mp.ultra)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;Error in ape::as.hclust.phylo(object) : the tree is not rooted&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;I ran into this error this error which says trees is not rooted &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Comparing phylogeny code updated ** &lt;em&gt;Working&lt;/em&gt; thanks to ***thomas duge de bernonville*&lt;/strong&gt; &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/8749/thomas-duge-de-bernonville&quot;&gt;thomas&lt;/a&gt; for putting codes together and fixing the errors &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(dendextend)&#xA;library(seqinr)&#xA;library(phytools)&#xA;library(phangorn)&#xA;&#xA;a&amp;lt;-read.alignment(&quot;abhi_seq/clean_dup_align_fast.fas&quot;, format=&quot;fasta&quot;)&#xA;a.phydat&amp;lt;-as.phyDat(a)&#xA;dist.a.phydat&amp;lt;-dist.dna(as.DNAbin(a.phydat))&#xA;upgma.a&amp;lt;-upgma(dist.a.phydat)&#xA;parsimony(upgma.a,a.phydat)&#xA;pars.a &amp;lt;- optim.parsimony(upgma.a, a.phydat)&#xA;pars.a&amp;lt;-acctran(pars.a, a.phydat)&#xA;pars.a.rooted&amp;lt;-root(pars.a, outgroup=&quot;AAA64460&quot;, resolve.root=T)&#xA;pars.a.rooted.dd&amp;lt;-as.dendrogram(force.ultrametric(pars.a.rooted))&#xA;&#xA;&#xA;mt &amp;lt;- modelTest(a.phydat, tree=upgma.a,multicore = TRUE,mc.cores=10)&#xA;&#xA;#ml.a = pml(upgma.a,a.phydat)&#xA;#fitJC &amp;lt;- optim.pml(ml.a, TRUE)&#xA;#ml.a.rooted&amp;lt;-root(midpoint(fitJC$tree), outgroup=&quot;AAA64460&quot;, resolve.root=T)&#xA;#ml.a.rooted.dd&amp;lt;-as.dendrogram(force.ultrametric(ml.a.rooted))&#xA;&#xA;#########################################################################&#xA;&#xA;######################################################################33&#xA;&#xA;#mt &amp;lt;- modelTest(dat, tree=tree, multicore=TRUE)&#xA;mt[order(mt&lt;span class=&quot;math-container&quot;&gt;$AICc),]# choose best model from the table according to AICc&#xA;bestmodel &amp;lt;- mt$&lt;/span&gt;Model[which.min(mt$AICc)]&#xA;env = attr(mt, &quot;env&quot;)&#xA;fitStart = eval(get(bestmodel, env), env)&#xA;fit = optim.pml(fitStart, rearrangement = &quot;stochastic&quot;,optGamma=TRUE, optInv=TRUE, model=&quot;GTR&quot;)#tree &#xA;bs=bootstrap.pml(fit, bs=25, optNni=TRUE, multicore=TRUE)&#xA;&#xA;ml.a.rooted&amp;lt;-root(midpoint(fit$tree), outgroup=&quot;AAA64460&quot;, resolve.root=T)&#xA;ml.a.rooted.dd&amp;lt;-as.dendrogram(force.ultrametric(ml.a.rooted))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="285" LastEditorUserId="3848" LastEditDate="2023-08-01T03:28:28.147" LastActivityDate="2023-08-01T03:28:28.147" Title="Comparing phylogeny in R" Tags="|r|phylogenetics|phylogeny|phangorn|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13378" PostTypeId="2" ParentId="13377" CreationDate="2020-05-25T10:01:46.293" Score="3" Body="&lt;p&gt;phangorn is a really powerful package for phylogenies. But to compare the trees, I think you may convert them into dendrograms and calculate a correlation measure, such as the Fowlkes-Mallows Index or a distance measure such as Baker’s Gamma Index. These can be easily computed using the &lt;code&gt;dendextend&lt;/code&gt; R package (&lt;a href=&quot;https://cran.r-project.org/web/packages/dendextend/vignettes/dendextend.html#correlation-measures&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cran.r-project.org/web/packages/dendextend/vignettes/dendextend.html#correlation-measures&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="8749" LastActivityDate="2020-05-25T10:01:46.293" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="13379" PostTypeId="2" ParentId="13372" CreationDate="2020-05-25T10:31:35.687" Score="6" Body="&lt;pre&gt;&lt;code&gt;awk -F &quot;::&quot; '{if(&lt;span class=&quot;math-container&quot;&gt;$1~&quot;&amp;gt;&quot;){gsub(&quot;&amp;gt;&quot;,&quot;&quot;);print &quot;&amp;gt;&quot;$2&quot;_&quot;$1} else {print $&lt;/span&gt;0}}' foo.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Basically the same as from Devon but using &lt;code&gt;-F&lt;/code&gt; to indicate the initial field delimiter and then using &lt;code&gt;gsub&lt;/code&gt; to remove the &lt;code&gt;&amp;gt;&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-05-25T10:31:35.687" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13380" PostTypeId="2" ParentId="13377" CreationDate="2020-05-25T10:33:18.020" Score="2" Body="&lt;p&gt;&lt;strong&gt;Bootstrapping&lt;/strong&gt; I agree correlation is one way in, but the classic way to do this is via bootstrapping, i.e. resampling the alignment with replacement and making a consensus phylogeny of of 100 to 1000 replicates. Values exceeding 75 to 80% define node robustness and you can calculate incongruence from there.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;code&gt;Phangorn&lt;/code&gt; certainly does bootstrapping, but the efficiency of the algorhithm for maximum likelihood (ML) I don't know (might take a very long time), just think how long one tree takes and times it by 100 (or even 1000).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you just performed a correlation measure there is a risk the differences are not robust, but if you've got loads of taxa you could bootstrap collapse any value less than the robustness threshold into a polytomy and then perform a correlation method. Its a bit complicated, but it would work.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Bootstrapping parsimony isn't hard nor computationally expensive, bootstrapping maximum likelihood sometimes requires specific efficient algorhithms notably &lt;a href=&quot;https://cme.h-its.org/exelixis/web/software/raxml/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;RAxML&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Non-R&lt;/strong&gt; The other way in is just dump the trees into &lt;a href=&quot;https://uni-tuebingen.de/fakultaeten/mathematisch-naturwissenschaftliche-fakultaet/fachbereiche/informatik/lehrstuehle/algorithms-in-bioinformatics/software/dendroscope/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;Dendroscope3&lt;/code&gt;&lt;/a&gt;, okay its not R but will work. Dendroscrope is hard for a non-tree person to read, however it will flag the differences for a specialist. Dendroscope will make a network tree for the topological differences, in this case between methods, i.e. non-bifurcating. You really then usually have to present the differences as a &quot;mirror tree&quot;, i.e. parsimony and ML side by side, because usually non-tree people wonder what a Dendroscope output means. Technically there is nothing wrong with describing incongruence via networks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Basically, the heart-beat is bootstrapping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Generally speaking parsimony will give the same basic output as ML, except when you have some taxa undergoing rapid evolution against their sister group. This does happen in pathogen evolution and parsimony falls over into a phenomina known as 'long-branch attraction'.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-05-25T10:38:44.093" LastActivityDate="2020-05-25T10:38:44.093" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13381" PostTypeId="1" AcceptedAnswerId="13384" CreationDate="2020-05-25T10:40:01.997" Score="0" ViewCount="116" Body="&lt;p&gt;I'm generating a density plot using the ggplot2 library:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(df_chr20, aes(x=start1)) + geom_density() + theme_bw() &#xA;ggplot(df_chr21, aes(x=start1)) + geom_density() + theme_bw() &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The data frame for &lt;em&gt;df_chr20&lt;/em&gt; has 1573 rows, and I wanted to plot the probability density of genomic regions as x = &lt;em&gt;start1&lt;/em&gt;. The same for &lt;em&gt;df_chr21&lt;/em&gt;, with 1271 rows. What I see in the plot is what I expect, but is there a reason why the y-axis probablilty densities are so low? The area under the curve is surely not equal to 1. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I expected the high localised high density in chr 21, and both the plots have similar scales in terms of y-axis values. But since they are so low, how can I interpret this plot?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I found some information &lt;a href=&quot;https://books.google.de/books?id=hBeJDwAAQBAJ&amp;amp;pg=PA7&amp;amp;lpg=PA7&amp;amp;dq=ggplot%20density%20plot%20values%20less%20than%201&amp;amp;source=bl&amp;amp;ots=F0CLApfVTs&amp;amp;sig=ACfU3U0J41Rxn-2kDJmSVRa7Aj3jO9HUpA&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ved=2ahUKEwiHi-H3zszpAhWO16YKHdIrAvoQ6AEwBnoECAoQAQ#v=onepage&amp;amp;q=ggplot%20density%20plot%20values%20less%20than%201&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, but my values are much lower than the examples.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be very kind!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QgUhG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QgUhG.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2094" LastEditorUserId="2094" LastEditDate="2020-05-25T13:43:26.763" LastActivityDate="2020-05-25T15:12:39.087" Title="Very low probability densities" Tags="|r|statistics|ggplot2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13382" PostTypeId="2" ParentId="13377" CreationDate="2020-05-25T11:01:46.407" Score="2" Body="&lt;p&gt;If you only have to compare two phylogenies (as opposed to more), I would suggest that an alternative to using R would be using iqtree . One of its parameters (-rf) can calculate the Robinson-Foulds metric between two trees.&#xA;(you can also use iqtree to calculate the phylogenies in the first place)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is command-line based, but can be easily installed with conda. Having anaconda installed in general makes bioinformatics work easier, I feel.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Alternatively, differences in trees can be visualized (without numbers) in Dendroscope, or Splitstree. They have different strategies for it. With Dendroscope it is easier to compare side-by-side and, if having precisely the same taxon names, to draw lines between taxa in different trees. Splitstree can accept an input of multiple trees (File&gt;Tools&gt;Load Multiple trees) and build a network (which can be a way to visualize the consensus of the trees).&lt;/p&gt;&#xA;" OwnerUserId="4401" LastEditorUserId="4401" LastEditDate="2020-05-25T13:23:06.547" LastActivityDate="2020-05-25T13:23:06.547" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13383" PostTypeId="2" ParentId="13372" CreationDate="2020-05-25T12:51:09.417" Score="3" Body="&lt;p&gt;&lt;code&gt;sed -E 's/&amp;gt;(.+)::(.+)/&amp;gt;\2_\1/' in.fa &amp;gt; out.fa&lt;/code&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to use sed. &lt;/p&gt;&#xA;" OwnerUserId="3360" LastEditorUserId="298" LastEditDate="2020-05-26T09:25:41.753" LastActivityDate="2020-05-26T09:25:41.753" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13384" PostTypeId="2" ParentId="13381" CreationDate="2020-05-25T15:12:39.087" Score="3" Body="&lt;p&gt;It is not like your conventional frequency which adds up to one. The density is low because the width of your bins is huge, and the number of observations you have is low. From &lt;a href=&quot;https://vita.had.co.nz/papers/density-estimation.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this by Wickham&lt;/a&gt;, the basic kernel is:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/82YEcm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/82YEcm.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where K is the kernel and h is the bandwidth (Scott, 1992b)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you don't specify it, by default h will revert to nrd0:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(rtracklayer)&#xA;gr_narrowPeak &amp;lt;- import(&quot;https://www.encodeproject.org/files/ENCFF072FBH/@@download/ENCFF072FBH.bed.gz&quot;,format = &quot;BED&quot;,extraCols = extraCols_narrowPeak)&#xA;gr_chr21 = gr_narrowPeak[seqnames(gr_narrowPeak)==&quot;chr21&quot;]&#xA;&#xA;da = as.data.frame(gr_chr21[rep(1:length(gr_chr21),gr_chr21&lt;span class=&quot;math-container&quot;&gt;$signalValue)])&#xA;dens = density(da$&lt;/span&gt;start)&#xA;dens$bw&#xA;[1] 1133889&#xA;&#xA;plot(dens)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jbclt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jbclt.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(da,aes(x=start))+geom_density()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hLzcr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hLzcr.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can decrease h such that you get something smaller (but at the cost of looking coarse). Important thing is it doesn't sum up to one.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you would like to explain it easier like a histogram, consider converting it into counts, like explained in this &lt;a href=&quot;https://stackoverflow.com/questions/27611438/density-curve-overlay-on-histogram-where-vertical-axis-is-frequency-aka-count&quot;&gt;post&lt;/a&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;binW = 50000&#xA;&#xA;ggplot(da,aes(x=start))+&#xA;geom_density(aes(y = ..density..*(nrow(da)*binW)),bw=50000) +&#xA;geom_histogram(binwidth=binW,fill=&quot;steelblue&quot;,alpha=0.5) + &#xA;theme_bw()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nq4vj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nq4vj.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(da,aes(x=start))+&#xA;geom_histogram(aes(y=..count../nrow(da)),binwidth=50000) + &#xA;geom_density(aes(y = ..density..*binW),bw=binW,fill=&quot;steelblue&quot;,alpha=0.5) +&#xA;theme_bw()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lwS6f.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lwS6f.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-05-25T15:12:39.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13385" PostTypeId="2" ParentId="13371" CreationDate="2020-05-25T17:00:51.803" Score="0" Body="&lt;p&gt;It's best to flip the question on its head and ask yourself instead, &quot;Do we care about the statistical significance of an individual alignment?&quot; The answer is &quot;yes&quot; if you're in a context like homology modeling, where you're aligning two protein sequences. The answer, however, is a very definite &quot;no&quot; in the context of NGS, where you just want the most likely alignment and a vague assessment of its likelihood of being wrong (for filtering purposes).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-05-25T17:00:51.803" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13388" PostTypeId="1" AcceptedAnswerId="13394" CreationDate="2020-05-25T23:39:57.397" Score="0" ViewCount="315" Body="&lt;p&gt;Is there a way to convert a hmmer output to a pandas dataframe? I am also unsure how to load a hmmer tblout table into python via the Bio module. I believe you can call a hmmer format with SeqIO.parse or SeqIO.search. How can you convert the table to a pandas dataframe?&lt;/p&gt;&#xA;" OwnerUserId="1833" LastActivityDate="2020-05-27T14:27:57.570" Title="Converting hmmer --tblout to pandas dataframe" Tags="|hmmer|pandas|dataframe|" AnswerCount="1" CommentCount="2" ClosedDate="2020-06-05T18:48:35.613" ContentLicense="CC BY-SA 4.0" />
  <row Id="13389" PostTypeId="2" ParentId="13372" CreationDate="2020-05-26T09:32:54.417" Score="2" Body="&lt;p&gt;I would use &lt;code&gt;perl&lt;/code&gt; here instead of &lt;code&gt;sed&lt;/code&gt; so you can use non-greedy patterns (e.g. &lt;code&gt;.*?&lt;/code&gt;) and so ensure that you always match the &lt;em&gt;first&lt;/em&gt; occurrence of &lt;code&gt;::&lt;/code&gt; if there are more than one on the line. Perl also has &lt;code&gt;-i&lt;/code&gt;, and in fact is where sed got the idea from, so you can edit the file in place just like you can with &lt;code&gt;sed&lt;/code&gt;. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using this example file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ cat file.fa&#xA;&amp;gt;ENST00000488147.1::chr1:29570-30013(-)&#xA;ACCTCTGAGAGCACCTCTGAGAGCACCTCTGAGAGCACCTCTGAGAGC&#xA;ATATATCGGCATATATCGGCATATATCGGCATATATCGGCATATATCG&#xA;&amp;gt;ENST00000405392.5::chr2:24919186-24920186(-)&#xA;ACTCTAGGATATCGCGATATCGCCTTAGGCTAGAATCGGCTTAGC&#xA;TCGCGATATCATCGCCTTAGGCTAGAATCGTCTAGGATATCGCG&#xA;CCTTAGGCTATCATCGCCTAGAATCGTCTAGGATATCGAATC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can do:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ perl -pe 's/&amp;gt;(.+?)::(.*)/&amp;gt;$&lt;/span&gt;2_$1/' file.fa &#xA;&amp;gt;chr1:29570-30013(-)_ENST00000488147.1&#xA;ACCTCTGAGAGCACCTCTGAGAGCACCTCTGAGAGCACCTCTGAGAGC&#xA;ATATATCGGCATATATCGGCATATATCGGCATATATCGGCATATATCG&#xA;&amp;gt;chr2:24919186-24920186(-)_ENST00000405392.5&#xA;ACTCTAGGATATCGCGATATCGCCTTAGGCTAGAATCGGCTTAGC&#xA;TCGCGATATCATCGCCTTAGGCTAGAATCGTCTAGGATATCGCG&#xA;CCTTAGGCTATCATCGCCTAGAATCGTCTAGGATATCGAATC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Or, to edit the file in place:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ perl -i -pe 's/&amp;gt;(.+?)::(.*)/&amp;gt;$&lt;/span&gt;2_&lt;span class=&quot;math-container&quot;&gt;$1/' file.fa &#xA;$&lt;/span&gt; cat file.fa&#xA;&amp;gt;chr1:29570-30013(-)_ENST00000488147.1&#xA;ACCTCTGAGAGCACCTCTGAGAGCACCTCTGAGAGCACCTCTGAGAGC&#xA;ATATATCGGCATATATCGGCATATATCGGCATATATCGGCATATATCG&#xA;&amp;gt;chr2:24919186-24920186(-)_ENST00000405392.5&#xA;ACTCTAGGATATCGCGATATCGCCTTAGGCTAGAATCGGCTTAGC&#xA;TCGCGATATCATCGCCTTAGGCTAGAATCGTCTAGGATATCGCG&#xA;CCTTAGGCTATCATCGCCTAGAATCGTCTAGGATATCGAATC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;h3&gt;Explanation&lt;/h3&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;perl -pe&lt;/code&gt;: &lt;strong&gt;p&lt;/strong&gt;rint each line of the input file after applying the script given by &lt;code&gt;-e&lt;/code&gt; to it. &lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;s/&amp;gt;(.+?)::(.*)/&amp;gt;$2_$1/'&lt;/code&gt;: the &lt;code&gt;s/old/new/&lt;/code&gt; is the substitution operator and will replace &lt;code&gt;old&lt;/code&gt; with &lt;code&gt;new&lt;/code&gt;. Here, we look for the shortest stretch of one or more characters (&lt;code&gt;.+?&lt;/code&gt;) from the &lt;code&gt;&amp;gt;&lt;/code&gt; until the first occurrence of &lt;code&gt;::&lt;/code&gt;, and capture it as &lt;code&gt;$1&lt;/code&gt; (because the pattern is in parentheses). Everything after the first &lt;code&gt;::&lt;/code&gt; is then captured as &lt;code&gt;$2&lt;/code&gt; (&lt;code&gt;(.*)&lt;/code&gt;). Then, we replace everything we matched with a &lt;code&gt;&amp;gt;&lt;/code&gt;, then the 2nd captured pattern, an underscore and the first captured pattern.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;hr&gt;&#xA;&#xA;&lt;p&gt;Just for fun, here's a golfed awk approach:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk -F'::' '$&lt;/span&gt;0=sub(/&amp;gt;/,&quot;&quot;)?&quot;&amp;gt;&quot;&lt;span class=&quot;math-container&quot;&gt;$2&quot;_&quot;$&lt;/span&gt;1:$0' file.fa&#xA;&amp;gt;chr1:29570-30013(-)_ENST00000488147.1&#xA;ACCTCTGAGAGCACCTCTGAGAGCACCTCTGAGAGCACCTCTGAGAGC&#xA;ATATATCGGCATATATCGGCATATATCGGCATATATCGGCATATATCG&#xA;&amp;gt;chr2:24919186-24920186(-)_ENST00000405392.5&#xA;ACTCTAGGATATCGCGATATCGCCTTAGGCTAGAATCGGCTTAGC&#xA;TCGCGATATCATCGCCTTAGGCTAGAATCGTCTAGGATATCGCG&#xA;CCTTAGGCTATCATCGCCTAGAATCGTCTAGGATATCGAATC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2020-05-26T09:40:26.040" LastActivityDate="2020-05-26T09:40:26.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13392" PostTypeId="2" ParentId="10628" CreationDate="2020-05-26T14:12:53.240" Score="0" Body="&lt;p&gt;DADA2 implements its own method-asignSpecies-for doing exactly this, which uses exact matches to species in a reference database and will be as accurate as any current method - for a single hyper variable region (which means not &lt;em&gt;hugely&lt;/em&gt; accurate). Basically there's not enough variability in any single hyper variable region to separate species across all genera.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Recent advances on species/strain identification, as far as I'm aware, have been concentrating on using the whole of the 16S region from 3rd generation long read technology.   &lt;/p&gt;&#xA;" OwnerUserId="8919" LastActivityDate="2020-05-26T14:12:53.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13394" PostTypeId="2" ParentId="13388" CreationDate="2020-05-26T21:28:25.397" Score="2" Body="&lt;p&gt;&lt;a href=&quot;http://eddylab.org/software/hmmer3/3.1b2/Userguide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The manual&lt;/a&gt; says fields in a tabular output file are space-delimited. It should be fairly straightforward to load a space-delimited file in pandas.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the manual:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;code&gt;--tblout &amp;lt;f&amp;gt;&lt;/code&gt; Save a simple tabular (space-delimited) file summarizing the per-target output, with one data line per homologous target model found.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="650" LastEditorUserId="650" LastEditDate="2020-05-27T14:27:57.570" LastActivityDate="2020-05-27T14:27:57.570" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13395" PostTypeId="1" CreationDate="2020-05-26T21:56:53.960" Score="1" ViewCount="953" Body="&lt;p&gt;I have previously performing Allelic scoring in Plink 1.9 using: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plink \ &#xA;--score sum sumstats.txt\&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, it seems this has changed in Plink 2.0. What is the equivalent function for Allelic scoring? &lt;/p&gt;&#xA;" OwnerUserId="8921" LastActivityDate="2022-09-14T22:58:09.957" Title="Plink 1.9 equivalent of --score sum in Plink 2.0" Tags="|plink|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13396" PostTypeId="2" ParentId="13395" CreationDate="2020-05-27T00:29:14.607" Score="3" Body="&lt;pre&gt;&lt;code&gt;plink2 --score cols=+scoresums&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;See &lt;a href=&quot;https://www.cog-genomics.org/plink/2.0/general_usage#colset&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cog-genomics.org/plink/2.0/general_usage#colset&lt;/a&gt; and &lt;a href=&quot;https://www.cog-genomics.org/plink/2.0/score&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cog-genomics.org/plink/2.0/score&lt;/a&gt; for details.&lt;/p&gt;&#xA;" OwnerUserId="472" LastEditorUserId="472" LastEditDate="2020-05-27T01:27:47.323" LastActivityDate="2020-05-27T01:27:47.323" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13397" PostTypeId="1" CreationDate="2020-05-27T06:58:14.170" Score="0" ViewCount="480" Body="&lt;p&gt;I am writing a snakemake that, up to now should&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;use raw sequencing reads to produce a draft assembly using &lt;a href=&quot;https://github.com/marbl/canu&quot; rel=&quot;nofollow noreferrer&quot;&gt;canu&lt;/a&gt; &lt;/li&gt;&#xA;&lt;li&gt;use &lt;a href=&quot;https://github.com/lh3/minimap2&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimap2&lt;/a&gt; to align the raw reads against the draft assembly, generated by canu.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The following two rules are described below:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rule canu:&#xA;    input:&#xA;        fqdir + &quot;{sample}.fastq&quot;&#xA;    output:&#xA;        directory(&quot;{sample}/canu&quot;)&#xA;    shell:&#xA;        &quot;/home/bhinckel/downloads/canu-1.9/Linux-amd64/bin/canu genomeSize=4.1m -p {wildcards.sample} -d {output} -nanopore-raw {input}&quot;&#xA;&#xA;&#xA;rule minimap:&#xA;    input:&#xA;        reads = fqdir + &quot;{sample}.fastq&quot;,&#xA;        draft_assembly = &quot;{sample}/canu/{sample}.contigs.fasta&quot;&#xA;    output:&#xA;        &quot;{sample}/aligned/{sample}.sam&quot;&#xA;    conda:&#xA;        &quot;envs/general.yaml&quot;&#xA;    shell:&#xA;        &quot;minimap2 -ax map-ont {input.draft_assembly} {input.reads} &amp;gt; {output}&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;However, if I do &lt;code&gt;snakemake -np&lt;/code&gt; I get &lt;code&gt;Missing input files for rule minimap&lt;/code&gt;. Which does make sense, after all I am not specifying the input for rule &lt;code&gt;minimap&lt;/code&gt; (&lt;code&gt;{sample}/canu/{sample}.contigs.fasta&lt;/code&gt;) anywhere i.e. snakemake cannot know that the file &lt;code&gt;{sample}/canu/{sample}.contigs.fasta&lt;/code&gt; will be generated by canu. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The thing is that &lt;code&gt;canu&lt;/code&gt; produces several files as output (amongst which the file &lt;code&gt;{sample}/canu/{sample}.contigs.fasta&lt;/code&gt; is one of them), that's why I specified its output as a &lt;code&gt;directory&lt;/code&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;One way of solving this I guess would be to specify &lt;a href=&quot;https://canu.readthedocs.io/en/latest/tutorial.html#outputs&quot; rel=&quot;nofollow noreferrer&quot;&gt;all output files&lt;/a&gt; produced by canu as an individual rule output (e.g. &lt;code&gt;{sample}/canu/{sample}.contigs.fasta&lt;/code&gt;, &lt;code&gt;{sample}/canu/{sample}.report&lt;/code&gt;, &lt;code&gt;{sample}/canu/{sample}.correctedReads.fasta.gz&lt;/code&gt;), though I am looking for a more robust way to do so (if that's possible at all).&lt;/p&gt;&#xA;" OwnerUserId="3452" LastActivityDate="2020-05-27T13:25:13.960" Title="snakemake - using output file not explicitly declared as input for subsequent rule" Tags="|ngs|nanopore|snakemake|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13399" PostTypeId="1" CreationDate="2020-05-27T12:02:48.157" Score="2" ViewCount="107" Body="&lt;p&gt;I have fasta file with alingned several sequences (from MUSCLE), when I open it (e.g. notepad++) they look like:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/SHsd1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SHsd1.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;And I want dot format of identities like &lt;a href=&quot;https://i.stack.imgur.com/suH4J.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/suH4J.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt; and then save it in txt file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I failed using MEGA, BioEdit, Jalview and CodonCode, how can I do this?&lt;/p&gt;&#xA;" OwnerUserId="2869" LastActivityDate="2021-10-26T22:27:29.600" Title="How to change sequence format in my alignment file?" Tags="|sequence-alignment|sequence-analysis|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13400" PostTypeId="1" CreationDate="2020-05-27T12:44:05.380" Score="1" ViewCount="88" Body="&lt;p&gt;I have a nucleotide sequence where I am interested in aligning reads distinctly to one or another portion of the sequence. However, I would like to use a visualization tool such as IGV to view these alignments in the context of the parent sequence. For example, I would like to split something along the lines of:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;seq1 parent_sequence&#xA;ACTGACTGACTGACTGACTGGTCAGTCAGTCAGTCAGTCAACACACACACACACACACAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Into:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;seq1_1:20 parent_sequence subsequence_1_to_20&#xA;ACTGACTGACTGACTGACTG&#xA;&amp;gt;seq1_21:40 parent_sequence subsequence_21_to_40&#xA;GTCAGTCAGTCAGTCAGTCA&#xA;&amp;gt;seq1_41:60 parent_sequence subsequence_41_to_60&#xA;ACACACACACACACACACAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then perform alignments using the split subsequences as a reference for alignment of reads, but visualize these read alignments on the parent sequence.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;While I'm sure it would be possible to write a script to manipulate all the alignments to all of the &quot;split&quot; subsequences back into the context of their original parent sequence, I am trying to avoid this if possible.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there a standard FASTA sequence description nomenclature for subsequences to accomplish what I'm describing that is recognized by programs like IGV?&lt;/li&gt;&#xA;&lt;li&gt;If there is not, I will likely still be splitting these sequences as described. Is there a suggested standard/common method for annotating/identifying these split sequence identifiers that maximizes readability?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The biological context for this question is analysis of RNA-Seq data collected after pulldown of a specific RNA-binding protein and digestion of overhanging read ends. The data should be enriched for a specific subsequence of interest, but will also contain the &quot;parent&quot; transcript. Alignments to both portions are of interest to me, but in different ways, and the alignments to the subsequence and the remainder of the parent sequence should be considered distinct. In terms of visualization it is helpful to view the alignments to all pieces in context of the &quot;parent&quot; transcript. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I realize that splitting the parent sequence into two subsequences for alignment will result in loss of reads straddling the two portions, but that is acceptable given the necessity of having the two sections be distinct for alignment purposes.&lt;/p&gt;&#xA;" OwnerUserId="7185" LastEditorUserId="7185" LastEditDate="2020-05-28T12:35:37.580" LastActivityDate="2020-05-28T12:35:37.580" Title="Is there a FASTA Sequence Description Nomenclature to Indicate a Split Sequence or Subsequence?" Tags="|fasta|identifiers|igv|nomenclature|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="13402" PostTypeId="2" ParentId="13397" CreationDate="2020-05-27T13:25:13.960" Score="1" Body="&lt;p&gt;Another way is to add one command after minimap: &lt;code&gt;touch done.txt&lt;/code&gt;. This creates an empty file after minimap is finished. This can then be used in output rule for snakemake.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But specifying all the output files may not be a bad idea if these outputs are used in subsequent rules. &lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-05-27T13:25:13.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13403" PostTypeId="1" AcceptedAnswerId="13409" CreationDate="2020-05-27T15:24:00.553" Score="1" ViewCount="401" Body="&lt;p&gt;I was looking to overlay annotation on my phylogeny made,which was solved here &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/13377/comparing-phylogeny-in-r&quot;&gt;question asked&lt;/a&gt; where i was looking to compare two different phylogeny which i could do.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now i want to label the tips into various groups such as if its human or canine etc.One of the possible solution i found is this &lt;a href=&quot;https://stackoverflow.com/questions/58554889/in-r-how-can-i-color-the-labels-from-my-phylogenetic-tree-using-bionj-from-ape&quot;&gt;one&lt;/a&gt; not sure how to implement it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;alignment  &lt;a href=&quot;https://drive.google.com/open?id=1ErIpdS99Vu_tsWtvNQCKftzFmpJAUKuu&quot; rel=&quot;nofollow noreferrer&quot;&gt;file&lt;/a&gt;&#xA;annotation &lt;a href=&quot;https://drive.google.com/file/d/1YQve01NEy1pyNeRNdvLAB9FVqHGNL37D/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;file&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(dendextend)&#xA;library(seqinr)&#xA;library(phytools)&#xA;library(phangorn)&#xA;&#xA;a&amp;lt;-read.alignment(&quot;clean_dup_align_fast.fas&quot;, format=&quot;fasta&quot;) &#xA;a.phydat&amp;lt;-as.phyDat(a)&#xA;dist.a.phydat&amp;lt;-dist.dna(as.DNAbin(a.phydat))&#xA;upgma.a&amp;lt;-upgma(dist.a.phydat)&#xA;parsimony(upgma.a,a.phydat)&#xA;pars.a &amp;lt;- optim.parsimony(upgma.a, a.phydat)&#xA;pars.a&amp;lt;-acctran(pars.a, a.phydat)&#xA;pars.a.rooted&amp;lt;-root(pars.a, outgroup=&quot;AAA64460&quot;, resolve.root=T)&#xA;pars.a.rooted.dd&amp;lt;-as.dendrogram(force.ultrametric(pars.a.rooted))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;i want to label according to the start column which is my source in my annotation file &#xA;Any suggestion or help would be really appreciated &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My working solution using &lt;a href=&quot;http://rforbiochemists.blogspot.com/2017/01/playing-with-phylogenetic-tree.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; , this works as I did the alignment of sequences using full header name which kind of help to look for the pattern and grep it.&#xA;The data &lt;a href=&quot;https://drive.google.com/file/d/1M7Mq0gKWvauqd-VKJCrNJ8KLMSXxx1Mu/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;file&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would still like to see the ggtree way instead of putting lots of label in the plot I would only like to see if I can annotate all the tips which have one organism source as one color rather putting all the labels &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pars.a&#xA;# &quot;Phylogenetic tree with 516 tips and 514 internal nodes&quot;&#xA;&#xA;# by using the class() function&#xA;class(pars.a)&#xA;# &quot;phylo&quot;&#xA;&#xA;# or by using the str() structure function&#xA;str(tree) &#xA;str(pars.a)&#xA;# &quot;List of 4&quot;&#xA;# this list includes &lt;span class=&quot;math-container&quot;&gt;$edge, $&lt;/span&gt;Nnode, &lt;span class=&quot;math-container&quot;&gt;$ tip.label and $&lt;/span&gt;edge.length&#xA;# the tree&lt;span class=&quot;math-container&quot;&gt;$tip.label includes family designation&#xA;#tree$&lt;/span&gt;tip.label  # 516 of these&#xA;pars.a$tip.label&#xA;# from the Science paper, we have seven kinase families:&#xA;# kinase categories... TK, TKL, STE, CK1, AGC, CAMK, CMGC&#xA;# with the following colours&#xA;# &quot;red&quot;, &quot;green&quot;, &quot;paleblue&quot;, &quot;orange&quot;, &quot;yellow&quot;, &quot;purple&quot;, &quot;pink&quot;, &quot;green&quot; &#xA;&#xA;&#xA;# by using the grep()function on the tree&lt;span class=&quot;math-container&quot;&gt;$tip.label part of the object&#xA;# we can find the tip labels that include &quot;TK/&quot; - i.e. tyrosine kinases&#xA;#grep(&quot;TK/&quot;, tree$&lt;/span&gt;tip.label)  # gives a list of numbers with &quot;TK/&quot; in tip label&#xA;#length(grep(&quot;TK/&quot;, tree$tip.label))&#xA;&#xA;grep(&quot;Human&quot;,pars.a&lt;span class=&quot;math-container&quot;&gt;$tip.label)&#xA;length(grep(&quot;Human&quot;,ml.a.rooted$&lt;/span&gt;tip.label)&#xA;)&#xA;# thus there are 94 tip labels with that are designated TK (not TKL tyrosine kinase like)&#xA;&#xA;# make a vector for each tip.label called tipcol with black on all of these...&#xA;#tipcol &amp;lt;- rep('black', length(tree&lt;span class=&quot;math-container&quot;&gt;$tip.label))&#xA;tipcol &amp;lt;- rep('black',length(pars.a$&lt;/span&gt;tip.label))&#xA;&#xA;# make a vector with our list of kinase categories&#xA;#kinaseCats &amp;lt;- c(&quot;TK/&quot;, &quot;TKL&quot;, &quot;STE&quot;, &quot;CK1&quot;, &quot;AGC&quot;, &quot;CAMK&quot;, &quot;CMGC&quot;, &quot;RGC&quot;)&#xA;kinaseCats &amp;lt;- c(&quot;Human&quot;,&quot;Rat&quot;,&quot;Murine&quot;,&quot;Bovine&quot;,&quot;Canine&quot;,&quot;Feline&quot;,&quot;Porcine&quot;,&quot;Gallus&quot;,&quot;Bat&quot;,&quot;Infectious&quot;,&quot;Transmissible&quot;)&#xA;&#xA;# make a vector of color we want:&#xA;colorsList &amp;lt;-c(&quot;red&quot;, &quot;darkolivegreen3&quot;, &quot;blue&quot;, &quot;orange&quot;, &quot;blueviolet&quot;, &quot;purple&quot;, &quot;pink&quot;, &quot;green&quot;,&quot;brown&quot;,&quot;navyblue&quot;,&quot;purple4&quot;)&#xA;&#xA;# replace colours where grep gives &quot;TK&quot; as red, etc in a loop&#xA;#for(i in 1:length(kinaseCats)){&#xA; # tipcol[grep(kinaseCats[i], tree$tip.label)] &amp;lt;- colorsList[i]&#xA;#}&#xA;&#xA;&#xA;for(i in 1:length(kinaseCats)){&#xA;  tipcol[grep(kinaseCats[i], pars.a$tip.label)] &amp;lt;- colorsList[i]&#xA;}&#xA;&#xA;&#xA;&#xA;# plot with edge length false to see nodes better&#xA;#plot(tree, &#xA; #    use.edge.length = FALSE,&#xA;  #   tip.color=tipcol, &#xA;   #  cex = 0.25)&#xA;&#xA;&#xA;plot(pars.a, &#xA;     use.edge.length = FALSE,&#xA;     tip.color=tipcol, &#xA;     cex = 0.5)&#xA;# slow to draw due to text - a bit annoying!&#xA;nodelabels(cex=0.4)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;figure &lt;a href=&quot;https://i.stack.imgur.com/IQCJJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IQCJJ.png&quot; alt=&quot;plot &quot;&gt;&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2020-05-27T19:22:40.513" LastActivityDate="2020-05-28T07:18:11.710" Title="Adding external annotation to the phylogeny tree" Tags="|r|phylogeny|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13404" PostTypeId="1" CreationDate="2020-05-27T18:53:17.717" Score="0" ViewCount="95" Body="&lt;p&gt;I used the Tximeta vignette to construct a summarised experiment from quant files (salmon, index prepared with NCBI transcriptome).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1)Tximeta return me a non-ranged se (which I am not sure about the real implications)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;because of this...&#xA;2) I used &lt;code&gt;makeLinkedTxome&lt;/code&gt; to link the transcriptome and the gff (local files used in the index creation). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;3)run &lt;code&gt;tximeta&lt;/code&gt; on the colData to create the summarised experiment &lt;code&gt;se&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;4) &lt;code&gt;seqinfo(se)&lt;/code&gt; returns me a NA populated table&lt;/p&gt;&#xA;&#xA;&lt;p&gt;5) I summarise to genes (&lt;code&gt;gse&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;6) I extracted the gene counts &lt;code&gt;counts &amp;lt;- assay(gse)&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions here are;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;a) Are these counts normalised? &#xA;b) If not, how can I normalise them? TMM for example? (CalcNormFactors will not work here right?)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is there any better way to obtain a normalised count matrix from tximeta? (maybe I don't need to link the transcriptome...)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help would be appreciated!&#xA;Thank you! and regards,&lt;/p&gt;&#xA;" OwnerUserId="8886" LastActivityDate="2020-05-27T18:53:17.717" Title="extracted counts matrix from Tximeta generalized experiment with no linked transcriptome are normalized?" Tags="|rna-seq|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13405" PostTypeId="1" AcceptedAnswerId="13406" CreationDate="2020-05-27T19:47:49.470" Score="0" ViewCount="1811" Body="&lt;p&gt;folks.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can you give me idea about an issue that I encountered?&#xA;I ran single-cell RNA sequencing data using Suerat in R, and when I tried to draw violin plot, there is an error as below. &lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;VlnPlot(tumors, &quot;Cd79a&quot;, &quot;Foxp3&quot;, &quot;CD14&quot;)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;Error: Insufficient values in manual scale. 24 needed but only 1 provided.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Run &lt;code&gt;rlang::last_error()&lt;/code&gt; to see where the error occurred.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I can draw the violin plot for Cd79a. The problem happened to only several genes. I haven't solved this problem yet. Do you know what should we do?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="8657" LastActivityDate="2020-05-27T20:51:44.943" Title="Error: Insufficient values in manual scale. 24 needed but only 1 provided?" Tags="|scrnaseq|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13406" PostTypeId="2" ParentId="13405" CreationDate="2020-05-27T20:51:44.943" Score="1" Body="&lt;p&gt;You must supply genes (or features) as a vector, not individually:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;VlnPlot(tumors, c(&quot;Cd79a&quot;, &quot;Foxp3&quot;, &quot;CD14&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;By the way it is probably &lt;code&gt;Cd14&lt;/code&gt;, not &lt;code&gt;CD14&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-05-27T20:51:44.943" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13407" PostTypeId="2" ParentId="13108" CreationDate="2020-05-27T22:18:57.470" Score="1" Body="&lt;p&gt;Venn diagrams can be helpful, but for peak calling they can be somewhat misleading.&#xA;This topic is addressed in the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/DiffBind/inst/doc/DiffBind.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DiffBind documentation&lt;/a&gt;, in the &quot;Comparison of occupancy and affinity based analyses&quot; section.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Suffice to say, asking &quot;which peaks are shared?&quot; is not the same as asking &quot;is there evidence for differential peaks, and where are they?&quot;.&#xA;For this reason, performing a differential binding analysis might be a better option.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you're just looking for Venn diagram plotting software, though, the &lt;a href=&quot;https://cran.r-project.org/web/packages/VennDiagram/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;VennDiagram&lt;/a&gt; (note, I have personally contributed to this tool in the past) and &lt;a href=&quot;https://cran.r-project.org/web/packages/UpSetR/vignettes/attribute.plots.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;UpSet&lt;/a&gt; R packages are useful tools.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2020-05-27T22:18:57.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13408" PostTypeId="2" ParentId="13377" CreationDate="2020-05-28T01:47:40.920" Score="4" Body="&lt;p&gt;Another 2 tree compare tools: &lt;a href=&quot;https://yulab-smu.github.io/treedata-book/chapter2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ggtree&lt;/a&gt; and &lt;a href=&quot;https://github.com/Don86/baltic3&quot; rel=&quot;nofollow noreferrer&quot;&gt;blatic 3&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8931" LastEditorUserId="48" LastEditDate="2020-06-02T09:49:55.660" LastActivityDate="2020-06-02T09:49:55.660" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13409" PostTypeId="2" ParentId="13403" CreationDate="2020-05-28T07:18:11.710" Score="1" Body="&lt;p&gt;there are nice features in R &lt;strong&gt;&lt;a href=&quot;https://yulab-smu.github.io/treedata-book/chapter5.html#visualizing-and-annotating-tree-using-grammar-of-graphics&quot; rel=&quot;nofollow noreferrer&quot;&gt;ggtree&lt;/a&gt;&lt;/strong&gt; package. You may use the &lt;em&gt;geom_cladelabel()&lt;/em&gt; function for your purposes. But I haven't find yet a way to automatize clade annotation. To do it, you have to be sure of the clade homogeneity before removing tiplabs. here are some commands to do clade label annotation:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;annot&amp;lt;-read.delim(&quot;annota.txt&quot;, header = T)&#xA;&#xA;ggtr&amp;lt;-as.treedata(pars.a)&#xA;g&amp;lt;-ggtree(ggtr)&#xA;#find the node number corresponding to the common ancestor of the top clade in you tree&#xA;MRCA(ggtr, &quot;AAL40400&quot;, &quot;AAF25519&quot;)&#xA;#here, check that all accessions belong to the same organism/group&#xA;subset(annot, label==&quot;AAL40400&quot; | label==&quot;AAF25519&quot;)&#xA;#in this case, sequences are related to bovine.&#xA;g&amp;lt;-g + geom_cladelabel(node=89, label=&quot;Bovine&quot;, align=F, fontsize=4)&#xA;plot(g)&#xA;&#xA;#another example&#xA;MRCA(ggtr, &quot;AAA46456&quot;, &quot;AAF97738&quot;)&#xA;subset(annot, label==&quot;AAA46456&quot; | label==&quot;AAF97738&quot;)&#xA;g&amp;lt;-g+ geom_cladelabel(node=73, label=&quot;Rodent&quot;, align=F, fontsize=4)&#xA;plot(g)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UbbRG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UbbRG.png&quot; alt=&quot;build with ggtree&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8749" LastActivityDate="2020-05-28T07:18:11.710" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="13412" PostTypeId="1" CreationDate="2020-05-28T14:09:32.580" Score="0" ViewCount="635" Body="&lt;p&gt;I am running an analysis of two phenotypes: mutant and control.&lt;/p&gt;&#xA;&lt;p&gt;I have a statistic used to rank the different genes, and the ranking is exactly the opposite between mutant and control. Moreover, there are no duplicates in the ranking (which excludes that genes are ranked alphabetically). I uploaded the test file &lt;a href=&quot;https://pastebin.com/raw/JKEJ20xF&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My expectation is that running GSEA using mutant as positive phenotype, should give me the same results (but in opposite direction) than running GSEA using control as a positive phenotype - since there are only two phenotypes, and the ranking statistic is computed mutant vs control.&lt;/p&gt;&#xA;&lt;p&gt;However, when I run Gene Set Enrichment Analysis using mutant as positive phenotype, the results I get are not the same with respect to running GSEA using mutant as negative phenotype.&lt;/p&gt;&#xA;&lt;p&gt;I initially thought that it might be related to the choice of the random seed, however, I get the same results even setting the same random seed in all runs.&lt;/p&gt;&#xA;&lt;p&gt;The following is an example of an enriched result of the positive phenotype:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ATA63.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ATA63.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1pBeX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1pBeX.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The same results with the negative phenotype:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZNitu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZNitu.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/dnGFo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dnGFo.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The ranking is exactly the same, however, the leading edge is different in the two cases. It's probably due to the fact that the ranking sum goes in one direction (increasing the enrichment score) in both phenotypes. This is probably the cause of different scores in the opposite phenotypes, however, which is the result I should use in this case?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2020-06-25T15:54:32.667" LastActivityDate="2022-12-13T13:05:20.807" Title="GSEA preranked different results opposing phenotypes" Tags="|gsea|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13414" PostTypeId="1" AcceptedAnswerId="13416" CreationDate="2020-05-28T20:33:08.853" Score="2" ViewCount="2211" Body="&lt;p&gt;I have many amino acids sequences in a fasta format that I did multiple sequence alignment. I was trying to plot something like binary a code as heatmap. If it had a change in the AA compared to the reference sequence it would be red, if it did not change would be yellow, for example.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I came across to &lt;code&gt;msaplot&lt;/code&gt; from &lt;code&gt;ggtree&lt;/code&gt; package. I also checked &lt;code&gt;ggmsa&lt;/code&gt; package for that. But so far, I did not get what I wanted. So basically I wanted to:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;change the multiple sequence alignment to a binary matrix (if the amino differs from the reference sequence, plot x, if not y)&lt;/p&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;plot as a heatmap with two colors based if the amino acid changed or not&lt;/p&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;here's some data example of what I doing, just instal &lt;code&gt;ggmsa&lt;/code&gt; and you can try this:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;protein_sequences &amp;lt;- system.file(&quot;extdata&quot;, &quot;sample.fasta&quot;, package = &quot;ggmsa&quot;)&#xA;ggmsa(protein_sequences, start = 265, end = 300, color = &quot;Chemistry_AA&quot;,font = NULL)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8938" LastEditorUserId="8938" LastEditDate="2020-05-28T20:54:55.760" LastActivityDate="2020-05-28T22:33:19.393" Title="How can I plot multiple sequence alignment in R as a heatmap" Tags="|r|sequence-alignment|heatmap|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13415" PostTypeId="1" AcceptedAnswerId="14047" CreationDate="2020-05-28T20:49:03.070" Score="2" ViewCount="1191" Body="&lt;p&gt;I have an alignment of multiple sequences in a FASTA file (output from MAFFT), for which I would like to simulate a phenotype using plink, but for that I need to have my alignment in a PED file or plink BED. There's plenty online about converting in the other direction, but the &lt;a href=&quot;https://www.biostars.org/p/284123/&quot; rel=&quot;nofollow noreferrer&quot;&gt;only mention I found of converting FASTA to BED&lt;/a&gt; is about aligning a raw FASTQ file for use in plink - my data is already aligned! How can I do this conversion?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/91/how-to-convert-fasta-to-bed&quot;&gt;How to convert FASTA to BED&lt;/a&gt; is about the UCSC BED format, not the plink binary PED.&lt;/p&gt;&#xA;" OwnerUserId="8937" LastActivityDate="2020-07-29T16:05:13.987" Title="Converting aligned fasta to plink ped/bed" Tags="|fasta|format-conversion|plink|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13416" PostTypeId="2" ParentId="13414" CreationDate="2020-05-28T22:33:19.393" Score="5" Body="&lt;p&gt;@StupidWolf answered me on stackoverflow and he's answer solved my question. I will share it here as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;@StupidWolf:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;We read in the alignment:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(Biostrings)&#xA;library(ggmsa)&#xA;protein_sequences &amp;lt;- system.file(&quot;extdata&quot;, &quot;sample.fasta&quot;, package = &quot;ggmsa&quot;)&#xA;aln = readAAMultipleAlignment(protein_sequences)&#xA;ggmsa(protein_sequences, start = 265, end = 300) &#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tU7Nd.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tU7Nd.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;Set the reference as the 1st sequence, some Rattus, you can also use the consensus with consensusString() :&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;aln = unmasked(aln)&#xA;names(aln)[1]&#xA;[1] &quot;PH4H_Rattus_norvegicus&quot;&#xA;&#xA;ref = aln[1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here we iterate through the sequence and make the binary for where the sequences are the same as the reference:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bm = sapply(1:length(aln),function(i){&#xA;as.numeric(as.matrix(aln[i])==as.matrix(ref))&#xA;})&#xA;&#xA;bm = t(bm)&#xA;rownames(bm) = names(aln)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The plot you see above has sequences reversed, so to visualize the same thing we reverse it, and also subset on 265 - 300:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(pheatmap)&#xA;pheatmap(bm[nrow(bm):1,265:300],cluster_rows=FALSE,cluster_cols=FALSE)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BA0vk.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BA0vk.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&#xA;The last row, is Rattus, the reference, and anything similar to that is read, as you can see in the alignment above, last 4 sequences are identical.&lt;/p&gt;&#xA;" OwnerUserId="8938" LastActivityDate="2020-05-28T22:33:19.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13417" PostTypeId="1" CreationDate="2020-05-29T00:14:17.417" Score="2" ViewCount="152" Body="&lt;p&gt;bioinformatics scientists.&#xA;I hope your work goes smoothly well and be safe.&lt;/p&gt;&#xA;&lt;p&gt;I'm stuck in trouble at the step of clustering.&lt;/p&gt;&#xA;&lt;p&gt;Briefly, I downloaded fastq files from a published study (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE121861&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE121861&lt;/a&gt;). There are 24 samples that come from 6 different mouse tumors.&#xA;Each tumor consists of two biological replicates where one biological replicate includes two technical replicates. So, I ran the command &amp;quot;cellranger count&amp;quot;, and generated 24 &amp;quot;filtered_feature_bc_matrix&amp;quot; files. For example, for the MC38, bio1-tech1, bio1-tech2, bio2-tech1, bio2-tech2 samples exist. FYI, These samples contain tumor cells as well as tumor-infiltrating immune cells.&lt;/p&gt;&#xA;&lt;p&gt;And then, I carried out the command &amp;quot;cellranger aggr&amp;quot; to combine all matrix files into one matrix file.&#xA;As suggested Seurat tutorial (&lt;a href=&quot;https://satijalab.org/seurat/v3.0/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/v3.0/pbmc3k_tutorial.html&lt;/a&gt;), Standard pre-processing workflow, linear dimensional reduction, determining dimensionality, clustering, and differential expression were done. Below are the commands I put.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;tumors &amp;lt;- NormalizeData(tumors)&lt;/p&gt;&#xA;&lt;p&gt;tumors &amp;lt;- FindVariableFeatures(tumors, selection.method = &amp;quot;vst&amp;quot;, nfeatures = 2000)&lt;/p&gt;&#xA;&lt;p&gt;all.genes &amp;lt;- rownames(tumors)&lt;/p&gt;&#xA;&lt;p&gt;tumors &amp;lt;- ScaleData(tumors, features = all.genes)&lt;/p&gt;&#xA;&lt;p&gt;tumors &amp;lt;- RunPCA(tumors, features = VariableFeatures(object = tumors))&lt;/p&gt;&#xA;&lt;p&gt;tumors &amp;lt;- JackStraw(tumors, num.replicate = 100)&lt;/p&gt;&#xA;&lt;p&gt;tumors &amp;lt;- ScoreJackStraw(tumors, dims = 1:20)&lt;/p&gt;&#xA;&lt;p&gt;ElbowPlot(tumors)&lt;/p&gt;&#xA;&lt;p&gt;tumors &amp;lt;- FindNeighbors(tumors, dims = 1:10) # I chose dims from 5, 6, 7, 8, 9, 10 or 20 as a cutoff.&lt;/p&gt;&#xA;&lt;p&gt;tumors &amp;lt;- FindClusters(tumors, resolution = 1.2)&lt;/p&gt;&#xA;&lt;p&gt;tumors &amp;lt;- RunUMAP(tumors, dims = 1:10)&lt;/p&gt;&#xA;&lt;p&gt;VlnPlot(tumors, features = c(&amp;quot;Cd79a&amp;quot;, &amp;quot;Foxp3&amp;quot;, &amp;quot;Cd3e&amp;quot;, &amp;quot;Cd4&amp;quot;, &amp;quot;Zap70&amp;quot;, &amp;quot;Cd8a&amp;quot;, &amp;quot;Cd14&amp;quot;, &amp;quot;Il2ra&amp;quot;))&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rhRyA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rhRyA.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Interestingly, as you can see here, Foxp3, Cd4, Cd8a are found to be overlapped. This phenomenon was the same when I drew vlnplot using different dimensionality from dims=1:5 to 1:10 or 1:20.&lt;/p&gt;&#xA;&lt;p&gt;Next, I performed this single-cell RNA sequencing analysis using each tumor; mc38, b16, sa1n, ll2, emt6, and ct26. Representatively, MC38, SA1N, EMT6, and CT26 are presented as below.&lt;/p&gt;&#xA;&lt;p&gt;For mc38, CD4, CD8, and Foxp3 are overlapped.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Uji2V.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Uji2V.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;For sa1n, CD4, CD8, and Foxp3 are separated.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/kd7cg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kd7cg.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;For emt6, CD4, CD8, and Foxp3 are overlapped.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/N11NV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/N11NV.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;For ct26, CD4, CD8, and Foxp3 are overlapped.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/TWm3P.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TWm3P.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, except for SA1N that only separates Foxp3 from CD8 T cells, I found that other tumor types can't separate Fopx3 from CD8 T cells. Do you think it is related to technical issues, such as values of dimensionality or PCA I put? Since I am not an expert about scRNA, I don't know the exact meaning of dims or PCA, though. Is there anyone who can tell me what's going on? And, do you know how to make the right clustering to get exact T cell subpopulations?&lt;/p&gt;&#xA;&lt;p&gt;I do appreciate your valuable time and thank you for reading this long story.&lt;/p&gt;&#xA;" OwnerUserId="8657" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2020-05-29T00:14:17.417" Title="Clustering issue about overlapping CD4, CD8, and Foxp3 genes in mouse tumor models" Tags="|scrnaseq|seurat|clustering|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="13418" PostTypeId="1" CreationDate="2020-05-29T04:16:13.150" Score="1" ViewCount="48" Body="&lt;p&gt;I'm currently learning how to use MEME-Suite (web tool) to find a specific known transcriptor binding site in the promoter region of 5 putative bacterial genes of interest, but the results obtained with just the five promtoer sequences have an E-value too high. What can I do to improve these results? Can I use as input some other reported sequences that have the motif I'm looking for without introducing any bias?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I used the option ZOOPS, with a background model of first order, and I limited the search to palindromes only, since I'm looking specifically for a 20 bp sequence with dyad symmetry. The input sequences are the promoter regions of each gene (200 pb upstream of the start codon).&lt;/p&gt;&#xA;" OwnerUserId="8939" LastActivityDate="2020-05-29T04:16:13.150" Title="MEME's motifs have an E-vaue too high" Tags="|gene-expression|motifs|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13419" PostTypeId="2" ParentId="9028" CreationDate="2020-05-29T09:50:06.543" Score="2" Body="&lt;p&gt;If need to get msa between every read and ref in the specific region , &#xA;you can try this &lt;a href=&quot;https://github.com/orangeSi/bam2msa.git&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/orangeSi/bam2msa.git&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bam2sam ref.fa input.bam --regions chr1:200-300&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8942" LastActivityDate="2020-05-29T09:50:06.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13420" PostTypeId="1" CreationDate="2020-05-29T13:59:50.373" Score="1" ViewCount="48" Body="&lt;p&gt;Is there any software that, given a DNA sequence as input, can predict what protein/ type of protein can bind to the DNA. The sequence of interest is a potential transcriptional regulator, I want to know for sure (or at least be reasonably certain) before I go to the wet lab to run any experiment.&lt;/p&gt;&#xA;" OwnerUserId="8767" LastActivityDate="2020-05-29T15:31:56.863" Title="Is there a software that can predict protein-DNA binding?" Tags="|proteins|software-recommendation|phylogenetics|transcription-factors|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13423" PostTypeId="1" CreationDate="2020-05-30T00:01:26.077" Score="1" ViewCount="33" Body="&lt;p&gt;I'm fairly new to plink software and wanted to get some additional practice after doing several tutorials. I obtained the data from this paper (I'm not using this paper's methods) to do some QC with subsequent analysis because they mostly use SNPstats.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Some background on what I did leading up to PCA:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My QC steps involved:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Filtering for low MAF&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Filtering for high missing values on the individual and SNP level&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Excluding outliers in heterozygosity (based on F statistic)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Excluding controls that deviated from Hardy-Weinberg&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finding related individuals and excluding members from the pairs that had the lower call rate&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After these steps, I had 657649 variants remaining, and no individuals had been filtered out from the original total (1401)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I then used a 1000 genomes dataset that I had already downloaded (unsure of which phase, but it contained 690 individuals) and merged it with my data (did similar QC steps on the 1000 genomes data, and also used the --flip flag to fix strand issues).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I did a rooted PCA analysis (with a LD threshold of 0.2, which included 112795 variants post-filter, but I also tried PCA without the filter with similar results) to look at population substructure and got this plot (in R):&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Where the pink &quot;OWN' labels in the plot are my data points. Does anyone have any idea why my data wouldn't be lining up with any of the ethnic groups from 1000 genomes? The data is said by the authors to be &quot;pre-filtered,&quot; which I'm assuming means fairly homogenous, so I wasn't expecting them to be in different ethnic groups, but they should be lining up with one of them, right?&lt;/p&gt;&#xA;" OwnerUserId="8946" LastActivityDate="2020-05-30T00:01:26.077" Title="GWAS Rooted PCA analysis problem" Tags="|gwas|plink|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13424" PostTypeId="1" CreationDate="2020-05-30T00:45:59.560" Score="1" ViewCount="191" Body="&lt;p&gt;When we use count data in RNA-seq analysis, we usually use normalization and sometimes vst, rlog transformation (DESeq2)or log2(CMP+4) transformation (edgeR) to perform K-means clustering. Can we use scaling after normalization and transformation?&#xA;Or Should we use scaling only when we just use normalization without transformation? &lt;/p&gt;&#xA;" OwnerUserId="8895" LastActivityDate="2020-05-30T00:45:59.560" Title="about the scaling after normalization and transformation in RNA-seq data" Tags="|r|rna|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13426" PostTypeId="1" AcceptedAnswerId="14474" CreationDate="2020-05-30T13:39:39.880" Score="4" ViewCount="163" Body="&lt;p&gt;In each iteration of the neighbour-joining method of phylogenetic trees construction, after joining the nearest neighbours the (additive) distance matrix &lt;span class=&quot;math-container&quot;&gt;$D_{m \times m}$&lt;/span&gt; is recomputed as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Q_{ij} = (m-2)d_{ij} - \Bigg( \sum_{k \neq i} d_{ik} \Bigg) - \Bigg( \sum_{k \neq j} d_{jk} \Bigg)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;As is stated in numerous sourced that I found, it prevents the joining of non-direct neighbours, but am looking for a more detailed explanation and motivation.&lt;/p&gt;&#xA;&lt;p&gt;Sources: &lt;a href=&quot;https://en.wikipedia.org/wiki/Neighbor_joining&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8608" LastEditorUserId="3848" LastEditDate="2023-08-03T15:43:23.500" LastActivityDate="2023-08-03T15:43:23.500" Title="Motivation behind the neighbor-joining distance matrix recomputation" Tags="|phylogenetics|sequence-alignment|phylogeny|sequence-analysis|matrix|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13427" PostTypeId="1" AcceptedAnswerId="13440" CreationDate="2020-05-30T14:20:44.923" Score="0" ViewCount="50" Body="&lt;p&gt;I have a file like this with 2 columns. I want to separate the columns based on plus and minus signs into separate files. How do I do this with awk or sed. Kindly help.    &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/my8CN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/my8CN.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7236" LastActivityDate="2020-06-01T12:21:53.193" Title="separate genes with expression values" Tags="|text-processing|awk|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13428" PostTypeId="2" ParentId="13427" CreationDate="2020-05-30T14:56:29.860" Score="1" Body="&lt;pre&gt;&lt;code&gt;awk '{if(&lt;span class=&quot;math-container&quot;&gt;$2&amp;gt;0) print $&lt;/span&gt;0}' expressionFile.txt &amp;gt;&amp;gt; positiveExpression.txt&#xA;awk '{if(&lt;span class=&quot;math-container&quot;&gt;$2&amp;lt;0) print $&lt;/span&gt;0}' expressionFile.txt &amp;gt;&amp;gt; negativeExpression.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Assuming that the file with gene expressions has the title: expressionFile.txt, this should do the trick.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Explanation of first line of code: if values in the 2nd column - &lt;code&gt;$2&lt;/code&gt; - are greater than 0, print all columns (&lt;code&gt;$0&lt;/code&gt;) and write to a file (&lt;code&gt;&amp;gt;&amp;gt;&lt;/code&gt;) titled, positiveExpression.txt. The use of &lt;code&gt;awk '{}'&lt;/code&gt; is just syntactical. &lt;/p&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="4001" LastEditDate="2020-05-30T15:01:45.293" LastActivityDate="2020-05-30T15:01:45.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13429" PostTypeId="1" CreationDate="2020-05-30T15:46:21.780" Score="1" ViewCount="201" Body="&lt;p&gt;I used Tximeta to import a summarisedExperiment from the salmon output (used with genocide transcriptome v34). I need to produce 4 matrix of counts:&#xA;- tx counts in TPM&#xA;- gene counts in TPM&#xA;- tx counts normalised&#xA;- gene counts normalised&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The first two I used &lt;code&gt;assay(se, &quot;counts&quot;)&lt;/code&gt; and &lt;code&gt;assay(gse, &quot;counts&quot;)&lt;/code&gt; respectively. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Mu problem is to apply the normalisation to them. I used deseq2 to estimate the size factors with a fake design because there are no groups yet in the dataset so I used ~gender to been able to generate a adds object to apply the &lt;code&gt;EstimateSizeFactor&lt;/code&gt;, but when I try to see them returns me 'NULL'&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dds_genes &amp;lt;-DESeqDataSet(gse, design = ~ gender)&#xA;&amp;gt; dss_genes &amp;lt;- DESeq2::estimateSizeFactors(dss_genes)&#xA;&#xA;&amp;gt; DESeq2::sizeFactors(dss_genes)&#xA;[1] NULL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get count values like this after using these commands (which I am not really sure they are really normalised;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;normalized_counts &amp;lt;- DESeq2::counts(dss_genes, normalized=T)&#xA;normalized_counts[1:10, 1:10]&#xA;&#xA;ENSG00000000003.15  654.92903  771.34051  500.1863  560.30530 1144.8295  938.40446 1173.208233  896.28964  388.1482   501.20545&#xA;ENSG00000000005.6     0.00000    0.00000    0.0000    0.00000    0.0000    0.00000    1.344747    0.00000    0.0000     0.00000&#xA;ENSG00000000419.12  416.59305  399.05434  675.0911  285.64627  551.8253  424.59423  461.448691  679.08099  365.8343   461.66276&#xA;ENSG00000000457.14  242.52226  244.91792  279.9109  414.24494  262.8790  354.50834  451.671501  427.08499  407.7070   329.50667&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I don't know how to proceed to just get a TMM normalised values from tximeta object. That would be my preferred option. I guess deseq2 can also provide me with normalised data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Maybe there are other ways? &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thank you for the help!!&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Regards&lt;/p&gt;&#xA;" OwnerUserId="8886" LastActivityDate="2020-05-30T15:46:21.780" Title="apply TMM on counts imported from salmon using Tximeta" Tags="|rna-seq|normalization|tmm|tpm|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13430" PostTypeId="2" ParentId="13426" CreationDate="2020-05-30T18:13:44.320" Score="2" Body="&lt;p&gt;NJ as with all clustering methods there isn't an explanation, It is a hierarchical clustering method but there are many algorithms to cluster a distance matrix. NJ is closely associated with the concept behind UPGMA (unweighted pair group mean algorithm)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The key difference with UPGMA is that after joining two taxa by NJ the reconstructed genetic distance between the node and one taxa versus the other taxa is not assumed to be equal, i.e. the rate of mutation is heterogeneous. All other clustering algorithms I know assume a &quot;global molecular clock&quot; if that makes any sense, the mutation is constant against time. NJ removes that assumption and thats why we like it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-05-30T18:13:44.320" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13433" PostTypeId="1" CreationDate="2020-05-30T19:47:28.037" Score="0" ViewCount="25" Body="&lt;p&gt;When we have a rescaled data with median (divided by median value)(e.g.,metabolome) and perform natural log transformation, do we need to use autoscaling again to perform PCA or to compute euclidian distance?&lt;/p&gt;&#xA;" OwnerUserId="8895" LastEditorUserId="8895" LastEditDate="2020-05-31T00:14:11.817" LastActivityDate="2020-05-31T00:14:11.817" Title="about rescaled data (median)" Tags="|r|pca|" AnswerCount="0" CommentCount="4" ClosedDate="2020-05-31T08:17:48.323" ContentLicense="CC BY-SA 4.0" />
  <row Id="13437" PostTypeId="1" AcceptedAnswerId="13482" CreationDate="2020-06-01T10:27:27.540" Score="-1" ViewCount="124" Body="&lt;p&gt;To begin with I have two nucleotide sequence to compare I want to see how they might have evolved in terms of sequence similarity and differences. One of the way i can think of is align the sequences. But i m not getting how do i associate it with mutation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Now question is how do i know about mutation. I have the respective amino acid data from which i know there been mutation which is making changes how the binding happens between two sequences.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So now i have to associate nucleotide mutation with amino acid sequence data.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any suggestion or help would be really appreciated. As always if there is a &lt;code&gt;R&lt;/code&gt; way to performing the analysis then it would be really good.&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2020-06-06T20:44:47.133" Title="Compare nucleotide mutation with amino acid" Tags="|r|sequence-alignment|sequence-analysis|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="13439" PostTypeId="1" CreationDate="2020-06-01T11:54:01.760" Score="1" ViewCount="87" Body="&lt;p&gt;Looking at the 1000 genomes project I see the following list of population groups and super-groups:&#xA;&lt;a href=&quot;https://www.internationalgenome.org/category/population/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.internationalgenome.org/category/population/&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, is there something like a standard list of global ethnicity? Like an ontology of population diversity or similar?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm trying to tie genome data to 'ancestry', and to do that I'd like to survey a set of people on their ancestral background (as far back as grandparents ideally).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How to capture diversity in terms of what people 'self report' in their backgrounds? e.g. middle / far east, north / south India, etc.&lt;/p&gt;&#xA;" OwnerUserId="8875" LastEditorUserId="8875" LastEditDate="2020-06-05T13:45:09.347" LastActivityDate="2020-07-05T14:05:40.497" Title="List of global ethnicities" Tags="|human-genome|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13440" PostTypeId="2" ParentId="13427" CreationDate="2020-06-01T12:21:53.193" Score="3" Body="&lt;p&gt;What do you want to do with NA values?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The below scripts ignore them (h3ab74's answer will print them to positiveExpression)&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '&lt;span class=&quot;math-container&quot;&gt;$2 ~ /^[0-9]+$&lt;/span&gt;/  {print &lt;span class=&quot;math-container&quot;&gt;$0}' expressionFile.txt &amp;gt; positiveExpression.txt&#xA;awk '$&lt;/span&gt;2 ~ /^-[0-9]+&lt;span class=&quot;math-container&quot;&gt;$/ {print $&lt;/span&gt;0}' expressionFile.txt &amp;gt; negativeExpression.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;These do a regex match to respectively, positive and negative numbers. If there's anything else in field 2 the line won't be printed. &lt;/p&gt;&#xA;" OwnerUserId="8919" LastActivityDate="2020-06-01T12:21:53.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13441" PostTypeId="1" CreationDate="2020-06-01T13:46:29.900" Score="0" ViewCount="402" Body="&lt;p&gt;I have 6 viral genome sequences of the same virus and 1 reference sequence in FASTA format.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How I can identify mutations and mutation sites in those genomes using FASTA sequences  but how I can do this for FASTA file?&lt;/li&gt;&#xA;&lt;li&gt;And how I can identify the mutation rate for one genome?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; If I've FASTQ file then I'll simply align the reads to the reference and by using variant calling tool I will get the mutate sites&lt;/p&gt;&#xA;" OwnerUserId="6504" LastEditorUserId="3848" LastEditDate="2020-12-01T10:24:13.437" LastActivityDate="2020-12-01T10:24:13.437" Title="How to calculate mutation rate and mutation sites in a genome using FASTA file?" Tags="|sequence-alignment|covid-19|mutations|sars-cov-2|virus|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13443" PostTypeId="1" AcceptedAnswerId="13444" CreationDate="2020-06-02T13:33:21.120" Score="1" ViewCount="1304" Body="&lt;p&gt;I am using fgsea in r to calculate and plot a bunch of GSEA graphs. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;My question is whether it is acceptable to use the Wald statistic from DeSeq2 to rank the gene list?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have seen in the GSEA application that the signal to noise ratio is used but is the Wald statistic sufficient?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If not then is there an easyish way to calculate the signal to noise ratio in R given a normalised count matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="8706" LastActivityDate="2020-06-05T21:31:42.020" Title="Ranking metric in GSEA" Tags="|r|deseq2|gsea|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13444" PostTypeId="2" ParentId="13443" CreationDate="2020-06-02T13:43:50.880" Score="2" Body="&lt;p&gt;I personally rank by signed nominal p-value. The fgsea author recommended on biostars.org to use either -log10(nominal p-value) or the F-statistics column (or whatever statistic the tool you use outputs) but not FDR/adjusted p-values as the latter produces a lot of ties for genes with low significances.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2020-06-03T15:40:54.457" LastActivityDate="2020-06-03T15:40:54.457" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13447" PostTypeId="1" CreationDate="2020-06-02T23:20:20.373" Score="1" ViewCount="22" Body="&lt;p&gt;Is there a method to download all the sequences for a particular geographic region (Panama) for 'DNA barcode' sequences (ITS2, rbcL or trnL). Hopefully, the specimen collection location will include this information.&lt;/p&gt;&#xA;&lt;p&gt;I want to use the Genbank sequences to create a DNA reference database of local plants for eDNA studies.&lt;/p&gt;&#xA;" OwnerUserId="8957" LastEditorUserId="3848" LastEditDate="2020-08-16T13:02:02.047" LastActivityDate="2020-08-16T13:02:02.047" Title="mining DNA barcodes from Genebank/BOLD per location" Tags="|phylogenetics|barcode|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13448" PostTypeId="2" ParentId="13441" CreationDate="2020-06-02T23:21:17.160" Score="1" Body="&lt;p&gt;You can still align a FASTA file with a tool like &lt;a href=&quot;https://github.com/lh3/bwa&quot; rel=&quot;nofollow noreferrer&quot;&gt;bwa mem&lt;/a&gt; ( if they are short reads ) or &lt;a href=&quot;https://github.com/lh3/minimap2&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimap2&lt;/a&gt; for long reads and run it through a variant caller like &lt;a href=&quot;https://github.com/ekg/freebayes&quot; rel=&quot;nofollow noreferrer&quot;&gt;freebayes&lt;/a&gt;. Alternatively, if you have sufficient reads to create an assembly of all of the genomes, or if they already are you can create a MSA of them using a tool like &lt;a href=&quot;https://github.com/ComparativeGenomicsToolkit/cactus&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cactus&lt;/a&gt; (as it is built for genomes) or MAFFT if the sequences are short enough and then use your favorite viewer to find variable sites &lt;/p&gt;&#xA;" OwnerUserId="8956" LastActivityDate="2020-06-02T23:21:17.160" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13450" PostTypeId="2" ParentId="9156" CreationDate="2020-06-03T07:33:53.610" Score="0" Body="&lt;p&gt;You may also use the excellent &lt;strong&gt;&lt;em&gt;pheatmap&lt;/em&gt;&lt;/strong&gt; package. It is highly customizable, so do not hesitate to look at its help page and examples found therein. The code may look like this:&#xA;&lt;code&gt;library(pheatmap)&#xA;data&amp;lt;-as.data.frame(data)&#xA;df&amp;lt;-data.frame(df, row.names=1)&#xA;pheatmap(data, annotation_col = df, cluster_cols = F, color=colorRampPalette(c(&quot;white&quot;, &quot;steelblue&quot;))(100))&#xA;&lt;/code&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Kk3bS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Kk3bS.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8749" LastActivityDate="2020-06-03T07:33:53.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13451" PostTypeId="1" AcceptedAnswerId="13452" CreationDate="2020-06-03T08:00:42.307" Score="1" ViewCount="38" Body="&lt;p&gt;When using branch-site model in &lt;a href=&quot;https://bitbucket.org/Davydov/godon/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Godon&lt;/a&gt;, model optimization results include the following parameters: &lt;code&gt;p01sum&lt;/code&gt; and &lt;code&gt;p0prop&lt;/code&gt;. What are those and how do they relate to &lt;code&gt;p0&lt;/code&gt; and &lt;code&gt;p1&lt;/code&gt; in the branch-site model?&lt;/p&gt;&#xA;" OwnerUserId="191" LastActivityDate="2020-06-03T08:25:54.353" Title="What are the p01sum and p0prop parameters in godon?" Tags="|codon|positive-selection|godon|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13452" PostTypeId="2" ParentId="13451" CreationDate="2020-06-03T08:00:42.307" Score="1" Body="&lt;p&gt;&lt;strong&gt;Disclaimer&lt;/strong&gt;: I'm the author of godon. The question above comes from the real communication with a software user.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In the godon implementation of the branch-site model I performed the following reparametrization:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p_{01sum} = p_0 + p_1$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p_{0prop} = \frac{p_0}{p_0 + p_1}$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This has the following advantages:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Both parameters have a well defined range &lt;span class=&quot;math-container&quot;&gt;$(0, 1)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;There is less dependency between the two. E.g., &lt;span class=&quot;math-container&quot;&gt;$p_2=1-p_0-p_1$&lt;/span&gt; (proportion of sites under positivie selection) does not depend on &lt;span class=&quot;math-container&quot;&gt;$p_{0prop}$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Both properties are very helpful for both likelihood maximization and MCMC.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;In case you are interested in &lt;span class=&quot;math-container&quot;&gt;$p_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$p_1$&lt;/span&gt; there is a straightforward way to go back to the original parameters:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p_0=p_{0prop}*p_{01sum}$&lt;/span&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p_1=p_{01sum}-p_0$&lt;/span&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;P.S. I also added this information to the &lt;a href=&quot;https://github.com/idavydov/godon-tutorial&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="191" LastEditorUserId="191" LastEditDate="2020-06-03T08:25:54.353" LastActivityDate="2020-06-03T08:25:54.353" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13453" PostTypeId="5" CreationDate="2020-06-03T08:03:52.387" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://bitbucket.org/Davydov/godon/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Godon&lt;/a&gt; is software for codon evolution models written in the Go language. It supports various classical models such as the branch-site, M0, M8. As well as an extension of those models which allow &lt;a href=&quot;https://dx.doi.org/10.1101/174839&quot; rel=&quot;nofollow noreferrer&quot;&gt;codon rate variation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="191" LastEditorUserId="191" LastEditDate="2020-06-03T13:11:38.153" LastActivityDate="2020-06-03T13:11:38.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13454" PostTypeId="4" CreationDate="2020-06-03T08:03:52.387" Score="0" Body="Use this tag to ask questions about the godon software. " OwnerUserId="191" LastEditorUserId="191" LastEditDate="2020-06-03T13:11:43.973" LastActivityDate="2020-06-03T13:11:43.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13455" PostTypeId="1" CreationDate="2020-06-03T09:56:43.210" Score="0" ViewCount="38" Body="&lt;p&gt;I am doing some metagenomic analysis of samples which can only contain microbes due to the experimental setup.&#xA;For this, it would be useful to be able to distinguish whether a specific organism is a microbe or not, as it allows:&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;to detect potentially false findings,&lt;/li&gt;&#xA;&lt;li&gt;to avoid an unnecessary inflation of the metagenomic database.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&#xA;&lt;p&gt;For animals and plants, this is straightforward, as they are never microbes.&#xA;However, for fungi (and some other clades) it gets complicated.&#xA;At the very least, I failed to find a simple list of microbial or non-microbial clades.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Ideally, I would like to have a function that maps a taxonomic ID to a boolean stating whether this organism is microbial. I acknowledge that this is utopic since &lt;em&gt;microbe&lt;/em&gt; is not that well defined, there are facultatively multicellular organisms, and for some cases it may simply not be known. I am therefore happy with any reasonable approximation such as unicellularity. In particular, given my application, a small chance of falsely identifying an organism as microbial is not an issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am currently working with the NCBI database accessed via the ETE 3 module for Python, but I am open to use other databases and tools.&lt;/p&gt;&#xA;" OwnerUserId="185" LastActivityDate="2020-06-04T05:09:20.263" Title="Determine if an organism is microbic or unicellular" Tags="|public-databases|phylogenetics|taxonomy|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13456" PostTypeId="1" CreationDate="2020-06-03T11:13:27.533" Score="1" ViewCount="2961" Body="&lt;p&gt;I have a fasta file (gene.fasta)with sequences with names patterns:  &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Hsc_gene_9627.t1&#xA;ATGGCACGCATTTTCCTCATTCTTTTATTGCTGCACAACATCTGCTGTGCAGCCGCTTCA&#xA;TTGCTCATTTTGAATGCCGTTACATTGGAGAAGGATGCTAATGATTATGCCGTTGGCGAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and I have the Ids in text file (seq.txt) which are not the exact as in the fasta file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;HSC_gene_996&#xA;HSC_gene_9734&#xA;and some of the names came as &#xA;HSC_gene_996|HSC_gene_9734 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How can extract the sequences?&#xA;I am not experienced in python so please use python for dummies language :)&#xA;Thank you in advance &lt;/p&gt;&#xA;" OwnerUserId="8959" LastEditorUserId="298" LastEditDate="2020-06-03T18:07:27.893" LastActivityDate="2021-01-18T01:00:30.077" Title="What is the full script to extract sequences from fasta file by using Ids in text file in python 3 and pycharm?" Tags="|python|fasta|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="13457" PostTypeId="1" CreationDate="2020-06-03T12:16:33.227" Score="1" ViewCount="65" Body="&lt;p&gt;I have a bed file which contains DNA sequences information as follow:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;**&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;track name=&quot;194&quot; description=&quot;194 methylation (sites)&quot; color=0,60,120 useScore=1&#xA;chr1    15864   15866   FALSE   894 +&#xA;chr1    534241  534243  FALSE   921 -&#xA;chr1    710096  710098  FALSE   729 +&#xA;chr1    714176  714178  FALSE   12  -&#xA;chr1    720864  720866  FALSE   988 -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;**&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I loaded the bed file in R and named the matrix &lt;strong&gt;DataSet&lt;/strong&gt;.&#xA;I used the follow code to get the sequences:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;mydataSet_Test1&amp;lt;-dataSet[,1:3]&#xA;&#xA;library(BSgenome.Hsapiens.UCSC.hg19)&#xA;genome &amp;lt;- BSgenome.Hsapiens.UCSC.hg19&#xA;chr&amp;lt;-as.matrix(as.character(mydataSet_Test1[,1]))&#xA;&#xA;#50&#xA;start&amp;lt;-as.matrix(as.integer(as.character(mydataSet_Test1[,2]))-50)&#xA;end&amp;lt;-as.matrix(as.integer(as.character(mydataSet_Test1[,3]))+50)&#xA;Seqs50_Test1&amp;lt;-getSeq(genome,chr,start=start,end=end)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I'm having DNA sequences and for each sequence, I have its Methylation Number (i.e. 800). I have split the data to hypomethylated (methylation &amp;lt; 200, class 0) and hypermethylated (methylation &gt; 800, class 1) and I want to build a model that can predict if a sequence (or a group of sequences) is hypo or hyper - methylated.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I now have a matrix of 358.367 data and 2 columns. The first column is the&#xA;DNA sequences and the second column is the class for each sequence (0 or 1).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I want to build a classification model in R, using SVM algorithm and 83 features (dinucleotides, trinucleotides, etc.).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Before calculating the features and building the model,&#xA;are there any steps I should follow for my DNA sequence data (each row of the matrix is a sequence from the human genome),&#xA;like filtering CpG islands or sequences, homolog reduction or overlapping sequences etc?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;As you understand I have a lack in DNA analysis.&lt;/p&gt;&#xA;" OwnerUserId="8600" LastEditorUserId="8600" LastEditDate="2020-06-04T11:19:27.693" LastActivityDate="2020-06-04T11:19:27.693" Title="What steps should I follow for DNA analysis, for a classification problem?" Tags="|r|sequence-analysis|phylogenetics|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13458" PostTypeId="1" CreationDate="2020-06-03T14:14:48.650" Score="0" ViewCount="572" Body="&lt;p&gt;Is it okay to use CPM normalization (with/without log transform) after using TMM normalization? Why do we &lt;a href=&quot;https://2-bitbio.com/2017/10/clustering-rnaseq-data-using-k-means.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;need both&lt;/a&gt;?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; library(edgeR)&#xA; library(SummarizedExperiment)&#xA; load(url(&amp;quot;http://duffel.rail.bio/recount/SRP049355/rse_gene.Rdata&amp;quot;))&#xA;  counts &amp;lt;- assays(rse_gene)$counts&#xA;  y &amp;lt;- as.matrix((counts))&#xA; y &amp;lt;- DGEList(counts = y, group=c(1,2,3,4,5,6,7,8,9,10))&#xA;  y &amp;lt;- calcNormFactors(y)&#xA; z &amp;lt;- cpm(y, normalized.lib.size=TRUE)&#xA;scaledata &amp;lt;- t(scale(t(z))) # Centers and scales data.&#xA;hc &amp;lt;- hclust(as.dist(1-cor(scaledata, method=&amp;quot;spearman&amp;quot;)), method=&amp;quot;complete&amp;quot;) # Clusters columns by Spearman correlation.&#xA; TreeC = as.dendrogram(hc, method=&amp;quot;average&amp;quot;)&#xA; plot(TreeC,&#xA; main = &amp;quot;Sample Clustering&amp;quot;,&#xA; ylab = &amp;quot;Height&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8895" LastEditorUserId="1157" LastEditDate="2020-07-08T11:23:53.567" LastActivityDate="2020-07-08T11:23:53.567" Title="about the cpm normalization after using normalization factor" Tags="|r|rna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13459" PostTypeId="2" ParentId="13458" CreationDate="2020-06-03T14:57:03.477" Score="2" Body="&lt;p&gt;In fact the &lt;code&gt;cpm&lt;/code&gt; function will use the size factors (TMM) that were calculated with &lt;code&gt;calcNormFactors&lt;/code&gt;. In the absence of the size factors &lt;code&gt;cpm&lt;/code&gt; would do a naive per-million scaling so only correct for differences in library size. The power of the size factors is to also correct for library composition. I recommend &lt;a href=&quot;https://www.youtube.com/watch?v=Wdt6jdi-NQo&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt; for details. In short: What you do is fine. I would suggest though that you use the &lt;code&gt;log = TRUE&lt;/code&gt; option for &lt;code&gt;cpm&lt;/code&gt; as one typically performs downstream applications (even when using the Z-score) on the log scale. Or alternatively &lt;code&gt;t(scale(t(log2(z+1))))&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2020-06-03T17:15:10.043" LastActivityDate="2020-06-03T17:15:10.043" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13461" PostTypeId="2" ParentId="13370" CreationDate="2020-06-03T17:05:08.840" Score="0" Body="&lt;p&gt;It would be helpful if you can provide the error message.&#xA;Guessing from your codes, it's possible that the alternative experiments weren't set correctly. My suggestion:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;altExp(sce, &quot;spike-in&quot;) &amp;lt;- SummarizedExperiment(assays=list(counts=as.matrix(SpikeIn)))&#xA;altExp(sce, &quot;MTgenes&quot;) &amp;lt;- SummarizedExperiment(assays=list(counts=as.matrix(MTgenes)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;If you're using altExp, the subsets argument in perCellQCMetrics is unnecessary in this case.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;perCellQCMetrics(sce,&#xA;exprs_values=&quot;counts&quot;,&#xA;use_altexps=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8962" LastEditorUserId="8962" LastEditDate="2020-06-03T17:17:06.060" LastActivityDate="2020-06-03T17:17:06.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13463" PostTypeId="2" ParentId="13456" CreationDate="2020-06-03T20:30:58.073" Score="1" Body="&lt;p&gt;For example, the fasta are like these:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gene.fasta:&#xA;&#xA;&amp;gt;Hsc_gene_9627.t1&#xA;ATGGCACGCATTTTCCTCATTCTTTTATTGCTGCACA&#xA;ACATCTGCTGTGCAGCCGCTTCA&#xA;&amp;gt;Hsc_gene_9627.t2&#xA;GGGGTTTTCCCC&#xA;&amp;gt;Hsc_gene_962.t1&#xA;AGTCGTCAGTCAGTAGTCGC&#xA;&#xA;seq.txt:&#xA;&#xA;HSC_gene_9627&#xA;HSC_gene_9734 &#xA;HSC_gene_996|HSC_gene_9734&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We use a few modules, read in the fasta:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;from Bio import SeqIO                                                                 &#xA;records = list(SeqIO.parse(&quot;gene.fasta&quot;,&quot;fasta&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Read in the list:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;seq = [i.rstrip().lower() for i in open('seq.txt').readlines()]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Here I use two functions, one to write fasta files, and the other to format the header for comparison:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;def fasta_out(rec):&#xA;    return(&quot;&amp;gt;&quot;+str(rec.description)+&quot;\n&quot;+str(rec.seq))&#xA;&#xA;def format(rec):&#xA;    return re.sub(&quot;[.][^ ]*$&quot;,&quot;&quot;,rec.description).lower()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Define the records to keep:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;keep = [fasta_out(i) for i in records if format(i) in seq]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Write them out:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;f=open('matched.fasta','w')&#xA;f.writelines(&quot;\n&quot;.join(keep))&#xA;f.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;We can check:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;open('matched.fasta').readlines()                                                          &#xA;Out[50]: &#xA;['&amp;gt;Hsc_gene_9627.t1\n',&#xA; 'ATGGCACGCATTTTCCTCATTCTTTTATTGCTGCACAACATCTGCTGTGCAGCCGCTTCA\n',&#xA; '&amp;gt;Hsc_gene_9627.t2\n',&#xA; 'GGGGTTTTCCCC']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Most likely there are some simple while, for loops that does not require modules..&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-06-03T20:30:58.073" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13464" PostTypeId="2" ParentId="13439" CreationDate="2020-06-03T21:30:59.003" Score="1" Body="&lt;p&gt;It's not clear exactly what your goal is. It might help to have an idea of how you are trying to use this data; and it's probably not a bioinformatics question, as ethnicities are social/political rather than biological constructs.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, there exist some &lt;a href=&quot;http://eprints.rclis.org/8831/1/ethnicgroups.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;controlled vocabularies&lt;/a&gt; used by LoC for ethnic groups; that looks a bit old so it might not be the best source.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am by no means an anthropologist but it does look like there are anthropological databases devoted to e.g. &lt;a href=&quot;https://ehrafworldcultures.yale.edu/ehrafe/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ethnography&lt;/a&gt; that may have complementary information.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Wikipedia has a &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_contemporary_ethnic_groups&quot; rel=&quot;nofollow noreferrer&quot;&gt;simple list&lt;/a&gt; of ethnicities that notes, instructively, &quot;There has been constant debate over the classification of ethnic groups.&quot;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I think that you are unlikely to find a single authoritative answer that is safe to use as a source of truth. I would normally suggest to try another stackexchange site, though it looks like there is none for sociology or anthropology. There is no anthropology tag available on this site, but you might try posting in e.g. history.stackexchange.com and adding an anthro tag.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-06-03T21:30:59.003" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13465" PostTypeId="2" ParentId="13443" CreationDate="2020-06-03T21:40:03.257" Score="3" Body="&lt;p&gt;If using DESeq2 with GSEA, I'd recommend ranking by shrunk log2FC values.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It'd also be worth considering ranking positive and negative associations separately, because the standard GSEA algorithm doesn't cross at the zero point when associations change from positive to negative.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;You shouldn't be using p-values to rank anything. A p-value gives information about the reliability of a result, but not its importance. See point #5 of this article about p-values:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://amstat.tandfonline.com/doi/full/10.1080/00031305.2016.1154108#_i31&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://amstat.tandfonline.com/doi/full/10.1080/00031305.2016.1154108#_i31&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-06-05T21:31:42.020" LastActivityDate="2020-06-05T21:31:42.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13466" PostTypeId="2" ParentId="13455" CreationDate="2020-06-04T04:57:06.027" Score="1" Body="&lt;p&gt;I would start by saying that your statement that no animals/plants are microbes is false. There are indeed microbial &lt;a href=&quot;https://microbiologyonline.org/index.php/about-microbiology/introducing-microbes/algae&quot; rel=&quot;nofollow noreferrer&quot;&gt;algae&lt;/a&gt; (plants) and &lt;a href=&quot;https://msystems.asm.org/content/4/3/e00109-19&quot; rel=&quot;nofollow noreferrer&quot;&gt;microbial animals&lt;/a&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So it is indeed difficult. (I'll briefly note that unless your protocol specifically gets rid of environmental DNA, you will probably have some contaminating non-microbial DNA no matter what.)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, we can use a different version of your proposition to make a simple (over-simple, most would argue) approximation: all prokaryotic organisms, e.g. bacteria and archaea, are microbial, and all eukaryotic organisms are not. This is not hard to do if you are already using NCBI taxonomy, as you can use e.g. &lt;a href=&quot;https://ftp.ncbi.nih.gov/pub/taxonomy/taxcat_readme.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;this mapping&lt;/a&gt;. It has the advantage that all bacteria and archaea are &lt;em&gt;definitely&lt;/em&gt; microbial. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Of course it's quite wrong, as you already know. In principle you could set up a complex decision tree sorting this out, though I imagine that's the thing you wish to avoid.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In terms of other genome databases, you might consider looking into &lt;a href=&quot;https://gold.jgi.doe.gov/organisms&quot; rel=&quot;nofollow noreferrer&quot;&gt;GOLD&lt;/a&gt;, which has extensive metadata in a slightly less arcane setup than NCBI, which might be helpful. But there is a high rate of missingness there, and also there is still not necessarily a great &quot;microbe&quot;/&quot;not microbe&quot; mapping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are also &lt;a href=&quot;https://jb.asm.org/content/197/15/2458#:~:text=The%20Global%20Catalog%20of%20Microorganisms,from%20different%20sources%20(78).&quot; rel=&quot;nofollow noreferrer&quot;&gt;existing databases&lt;/a&gt; of microorganisms, which may or may not satisfy your requirements; I have little familiarity with most of those so caveat emptor. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I believe (hope) that NCBI taxid is a good way to do lookups in all such databases, but it may not be true.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2020-06-04T05:09:20.263" LastActivityDate="2020-06-04T05:09:20.263" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13468" PostTypeId="1" AcceptedAnswerId="13470" CreationDate="2020-06-04T20:10:52.670" Score="1" ViewCount="421" Body="&lt;p&gt;I have two cancer subtypes data. Subtype A is 14 samples and Subtype B is 23 samples. I'm interested in identifying the functions of some LncRNAs in the Subtype A group. For this I'm using all protein-coding genes and interested LncRNAs expression data (normalised counts). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;From the WGCNA tutorial, I see that minimum number of samples needs to be 20. So, I guess using only Subtype A 14 samples for the analysis is not possible.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Can I use both Subtype A 14 samples and Subtype B 23 samples together for the analysis? In case if I use both, once I get the modules how do I know that the genes in modules are related to Subtype A or Subtype B? The pathways I get from the interested module is related to Subtype a or Subtype B? How this can be identified?&lt;/p&gt;&#xA;" OwnerUserId="1369" LastActivityDate="2020-06-04T20:42:16.483" Title="WGCNA co-expression network analysis with less than 20 samples" Tags="|r|rna-seq|bioconductor|gene-expression|wgcna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13469" PostTypeId="1" AcceptedAnswerId="13657" CreationDate="2020-06-04T20:39:59.033" Score="2" ViewCount="63" Body="&lt;p&gt;I would like to ask if the right Prosite pattern for this multiple alignment:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DnHv9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DnHv9.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it this: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A-T-[AT]-G-x-C-[AGC]-C-x(1,4)-A &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;or this: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;A-T-[AT]-G-x-C-[AGC]-C-x(1,3)-[ATGC]-A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Actually, I would like to ask if &lt;code&gt;x&lt;/code&gt; always implies a gap, and if it is proper to use it when any symbol but not gap can be in the sequence. Thank you!&lt;/p&gt;&#xA;" OwnerUserId="8713" LastEditorUserId="298" LastEditDate="2020-06-05T10:17:34.413" LastActivityDate="2020-06-28T00:03:35.327" Title="which is the right prosite pattern?" Tags="|sequence-alignment|prosite|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13470" PostTypeId="2" ParentId="13468" CreationDate="2020-06-04T20:42:16.483" Score="3" Body="&lt;p&gt;It's hard to say without knowing how different the subtypes are. If you do a common WGCNA, you may find modules related to the differences between A and B as well as to drivers of expression variation in A and B (common and separate). Since A has fewer samples, the variation will have to be stronger to be reflected by WGCNA modules. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;You can run the WGCNA, find modules that vary the most in A samples (these may or may not also vary in B samples), then see whether LncRNAs belong to these modules and have a high kME (module membership). &lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2020-06-04T20:42:16.483" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13471" PostTypeId="1" AcceptedAnswerId="13474" CreationDate="2020-06-04T23:42:23.063" Score="1" ViewCount="318" Body="&lt;p&gt;I would like to use RNA-seq data from the NIH GEO database. How can I tell if library generation for the RNA-seq experiments are stranded vs not stranded?&lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2020-06-05T22:16:34.723" Title="How to tell if library generation for RNA-seq experiments are stranded vs not stranded" Tags="|rna-seq|geo|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13472" PostTypeId="1" CreationDate="2020-06-04T23:54:03.337" Score="3" ViewCount="248" Body="&lt;p&gt;I am trying to investigate a membrane proteins influence on a lipid bilayer using molecular dynamics. I am new to MD and trying to run my first simulation. I have built the simulation input files using CHARMM-GUI and now I am using gromacs to perform the simulation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;CHARMM-GUI provides these commands to get gromacs to perform the initial minimization&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;gmx_d grompp -f step6.0_minimization.mdp -o step6.0_minimization.tpr -c step5_input.gro -r step5_input.gro -p topol.top&#xA;gmx_d mdrun -v -deffnm step6.0_minimization&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The first command runs fine. The second command starts the minimization, but at step 15 it freezes. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;&#xA;Step 15, time 0.015 (ps)  LINCS WARNING&#xA;relative constraint deviation after LINCS:&#xA;rms -nan, max 0.098209 (between atoms 16596 and 16597)&#xA;bonds that rotated more than 30 degrees:&#xA; atom 1 atom 2  angle  previous, current, constraint length&#xA;  15683  15684   39.2    0.1109   0.1121      0.1111&#xA;  15683  15685   38.4    0.1115   0.1107      0.1111&#xA;  16596  16597   72.1    0.1111   0.1220      0.1111&#xA;  16640  16642   35.3    0.1116   0.1120      0.1111&#xA;  16643  16644   51.9    0.1108   0.1136      0.1111&#xA;  16643  16645   47.2    0.1114   0.1079      0.1111&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In step 15, the rms is -nan, so I'm guessing that is the problem, and I am assuming that is coming from the difference between the &quot;current&quot; and &quot;previous&quot; is greater than constraint length like on this line here&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;16596  16597   72.1    0.1111   0.1220      0.1111&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The problem is I don't have any intuition on how to fix it? The pdb file is a homology model from phyre without any modification other than adding a single chain id to all the residues.&#xA;I'm using a 1 to 1 POPC and DOPC in both membrane layers with a CHARMM36m force field.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The previous steps also throw LINCS warnings about bonds exceeding 30 degrees, however they produce a real value for rms.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I've counted on tutorials to get this far and would appreciate some suggestions or good resources I can use to continue my simulation. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="8795" LastActivityDate="2021-06-25T11:11:45.950" Title="Gromacs molecular dynamic simulation throwing error during minimization" Tags="|molecular-dynamics|gromacs|membrane-protein|lipid-bilayer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13473" PostTypeId="1" CreationDate="2020-06-05T03:02:36.067" Score="1" ViewCount="18" Body="&lt;p&gt;Hello experts in residue coevolution analysis of proteins,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I am looking for a way to calculate residue coevolution (coupling scores) between all pairs of residues of a protein.&#xA;The issue is that rather than having a Multiple Sequence Alignment (MSA) which is used as an input by the relevant tools (e.g. evcouplings), I have position wise preferences for 20 amino acids for the protein. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;For the protein with length &lt;code&gt;L&lt;/code&gt;, I have a &lt;code&gt;20*L&lt;/code&gt; matrix of position-wise preferences for amino acids.&#xA;I would like to use this matrix as a direct input for coevolution analysis. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Would it be possible to calculate the residue coevolution direcly from a residue preference matrix?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any recommendation for a method of choice or a software tool would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="274" LastActivityDate="2020-06-05T03:02:36.067" Title="Residue coevoution from position wise residue preferences rather than multiple sequence alignments" Tags="|proteins|sequence-analysis|mutations|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13474" PostTypeId="2" ParentId="13471" CreationDate="2020-06-05T03:37:00.753" Score="3" Body="&lt;p&gt;If you already have a reference transcriptome assembly, then you may run the Salmon tool for quantification using your RNA-seq samples. It has an automated library type inference function (&lt;a href=&quot;https://salmon.readthedocs.io/en/latest/salmon.html#what-s-this-libtype&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://salmon.readthedocs.io/en/latest/salmon.html#what-s-this-libtype&lt;/a&gt;). It might help you to determine whether the libs are stranded or not.&lt;/p&gt;&#xA;" OwnerUserId="8749" LastActivityDate="2020-06-05T03:37:00.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13475" PostTypeId="1" CreationDate="2020-06-05T05:40:39.370" Score="3" ViewCount="60" Body="&lt;p&gt;Here we have a 3D model of a sequence that match with this sequence&#xA;VVNIQKEIDRLNEVAKNLNESLIDLQELGKYEQYIK&#xA;in Blast&lt;/p&gt;&#xA;&#xA;&lt;p&gt;the 3D structure is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Structure/icn3d/full.html?&amp;amp;mmdbid=38995&amp;amp;bu=1&amp;amp;showanno=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Why is there a chloride ion in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Structure/icn3d/full.html?&amp;amp;mmdbid=38995&amp;amp;bu=1&amp;amp;showanno=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; 3D model?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HMNJn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HMNJn.png&quot; alt=&quot;screenshot of the 3D structure&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4572" LastEditorUserId="3848" LastEditDate="2020-11-30T20:24:22.750" LastActivityDate="2020-11-30T20:24:22.750" Title="Why is there a chloride ion in this 3D model?" Tags="|protein-structure|covid-19|protein-protein-interaction|interactions|sars-cov-2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13476" PostTypeId="2" ParentId="13475" CreationDate="2020-06-05T12:05:49.070" Score="3" Body="&lt;p&gt;It is a crystallisation artefact. Namely, the protein are placed in a condition where they fall out of solution without aggregating a gloop (as happens in most well in xstal trial) and to pack as a crystal they need to be placed nice and orderly. It is coordinated by water so is not relevant. A lot of structures have these —DMSO and ions are the most commonly.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Two footnotes.&lt;/p&gt;&#xA;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1ZV7 is a crystal structure, &lt;em&gt;i.e.&lt;/em&gt; solved by X-ray diffraction. A model is generally used to described an &lt;em&gt;in silico&lt;/em&gt; model, such as those from Swissmodel, phyre, I-Tasser etc. These are generated &lt;em&gt;ab initio&lt;/em&gt; (from scratch) or threaded (mapped on top of) against crystal/EM/NMR structures. For Covid I-Tasser has a &lt;a href=&quot;https://zhanglab.ccmb.med.umich.edu/COVID-19/&quot; rel=&quot;nofollow noreferrer&quot;&gt;nice picture of its models&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The best source of info about structures is the PDB. &lt;a href=&quot;https://www.rcsb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RCSB PDB&lt;/a&gt; is more commonly used, while &lt;a href=&quot;https://www.ebi.ac.uk/pdbe/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDBe&lt;/a&gt; is more tricker to navigate (or used to be). Both have the same data, but are better for the extras than NCBI.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-06-05T12:05:49.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13477" PostTypeId="1" AcceptedAnswerId="19643" CreationDate="2020-06-05T17:08:21.673" Score="2" ViewCount="154" Body="&lt;p&gt;I am following the tutorial &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/29525965/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; to learn how to process raw RNA-seq data and get gene read counts. I've had no issues until I got to step 3.9, &amp;quot;Assignment of RNA-Seq Reads to Genes.&amp;quot; Here, the authors run the command&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fc0 &amp;lt;- featureCounts(sam.list, annot.ext = &#xA;&amp;quot;Ara-port11_GFF3_genes_transposons.201606.gtf&amp;quot;, allowMultiOverlap = T, isPairedEnd = F, &#xA;nthreads = 2, strandSpecific = 0, isGTFAnnotationFile = T)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I ran the same command except I set isGTFAnnotationFile = F because I couldn't find the gtf file, so I used the gff file &lt;code&gt;TAIR10_GFF3_genes_transposons.gff&lt;/code&gt; instead, downloaded from &lt;a href=&quot;https://www.arabidopsis.org/download/index-auto.jsp?dir=%2Fdownload_files%2FGenes%2FTAIR10_genome_release%2FTAIR10_gff3&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (they mentioned that using gtf only speeds up the process but that you can still do this using gff). I'm not sure if this is what is causing the issue, but the error I get is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error: Line 1 contains a format error. The expected annotation format is SAF.&#xA;Error in featureCounts(sam.list, annot.ext = &amp;quot;TAIR10_GFF3_genes_transposons.gff&amp;quot;,  : &#xA;  No counts were generated.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="73" LastEditDate="2022-09-02T11:15:19.757" LastActivityDate="2023-05-30T12:03:30.617" Title="Step-by-Step Construction of Gene Co-expression Networks from High-Throughput Arabidopsis RNA Sequencing Data" Tags="|r|rna-seq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13478" PostTypeId="1" AcceptedAnswerId="13486" CreationDate="2020-06-05T18:07:21.757" Score="1" ViewCount="91" Body="&lt;p&gt;If there is no significant difference between two groups of metagenomic samples according to &quot;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6459172/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multivariate differential abundance tests&lt;/a&gt;&quot; for example PERMANOVA and ANOSIM, does it makes sense to use &quot;Univariate differential abundance testing&quot; to see, for example, what species have significantly different abundances between groups?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The issue is that I found species that have significantly different abundances among samples of two groups using Wald test in DESeq2, but there is no significant difference between the two groups according to PERMANOVA and ANOSIM.&lt;/p&gt;&#xA;" OwnerUserId="6593" LastActivityDate="2020-06-07T17:57:36.663" Title="Is it conflicting: significant difference according to Wald test but not in PERMANOVA and ANOSIM to compare species abundances of metagenomic samples?" Tags="|differential-expression|deseq2|metagenome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13480" PostTypeId="2" ParentId="13471" CreationDate="2020-06-05T22:16:34.723" Score="2" Body="&lt;p&gt;Unfortunately, the stranded nature of cDNA sequencing is not typically included in the metadata attached to a project, which means that downloading a small subset of the data and checking for strand consistency (as suggested by &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/13474/73&quot;&gt;the other answer&lt;/a&gt;) is often the quickest way to work that out (when that information isn't included in the methods section of the associated paper). This knowledge is not particularly useful for casually browsing projects to find a stranded protocol that fits a particular study specification.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sometimes you get lucky, but not often. GEO RNASeq datasets &lt;em&gt;usually&lt;/em&gt; have links to the SRA accessions, and those accessions &lt;em&gt;usually&lt;/em&gt; include annotation about the library type. For example, consider this dataset on GEO:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE151535&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE151535&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It links to SRP265460 on SRA. Following the link for the first run for this project, I get to here:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra/SRX8436504[accn]&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/sra/SRX8436504[accn]&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This includes the following metadata about the run:&lt;/p&gt;&#xA;&#xA;&lt;blockquote&gt;&#xA;  &lt;p&gt;&lt;em&gt;Instrument:&lt;/em&gt; NextSeq 500&lt;br&gt;&#xA;  &lt;em&gt;Strategy:&lt;/em&gt; RNA-Seq&lt;br&gt;&#xA;  &lt;em&gt;Source:&lt;/em&gt; TRANSCRIPTOMIC&lt;br&gt;&#xA;  &lt;em&gt;Selection:&lt;/em&gt; cDNA&lt;br&gt;&#xA;  &lt;em&gt;Layout:&lt;/em&gt; SINGLE&lt;br&gt;&#xA;  &lt;em&gt;Construction protocol:&lt;/em&gt; Replicates of oropharyngeal explants at different stages of development (37/38, 39, 41) were immersed in RNALater. Samples were homogenized with a Dounce homogenizer in RLT buffer containing ß-mercaptoethanol (Qiagen). The homogenate was then processed using an RNAEasy kit (Qiagen, Hilden, Germany). Replicate samples for each stage were analyzed for RNA quantity and quality on a Nanodrop 8000 spectrophotometer. RNA libraries were prepared for sequencing using standard Illumina protocols&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&#xA;&lt;p&gt;But, as you can see, there's nothing about whether its stranded or not: Illumina has both stranded and unstranded &quot;standard&quot; protocols.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2020-06-05T22:16:34.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13481" PostTypeId="1" AcceptedAnswerId="13566" CreationDate="2020-06-06T18:23:17.947" Score="0" ViewCount="630" Body="&lt;p&gt;Maximum Likelihood analysis compares the tips or the species based on their sequence similarity while Parsimony analysis compares the characteristic features among the species.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So far I did comparison of both the tree using dendextend. Is there any other methods or library to make the comparison more quantitative.Due to my limited understanding I have this in mind. Can i even compare these two methods?  &lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2020-06-07T10:22:18.907" LastActivityDate="2020-07-23T08:11:43.827" Title="parsimony and maximum likelihood tree comparison in R" Tags="|r|phylogenetics|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="13482" PostTypeId="2" ParentId="13437" CreationDate="2020-06-06T20:38:18.450" Score="1" Body="&lt;p&gt;I  would output the aligned nucleotide sequences and input them into an alignment editor such as &lt;code&gt;Sequtron&lt;/code&gt; (Mac), &lt;code&gt;MEGAX&lt;/code&gt;(particularly PC) or &lt;code&gt;Jalview&lt;/code&gt; (Linux). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The alignment editors will allow you to flick between the triplet codon and the amino acid sequence. Therefore spotting which triplet codon mutations associate with each given amino acid mutation is easy.There is no &lt;code&gt;R&lt;/code&gt; equivalent that I am aware of.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;More generally I would do a non-synonymous / synonymous analysis which &lt;code&gt;fangorn&lt;/code&gt; will perform. This assesses the number of mutations in which cause an amino acid change against the number of mutations that cause a 'silent' mutation (no amino acid change). A synonymous phylogeny will represent all mutations which don't change the tree. A non-synonymous phylogeny is the same as an amino tree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There are character mapping approaches which will map an amino acid mutation onto a phylogeny as an alternative.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-06-06T20:44:47.133" LastActivityDate="2020-06-06T20:44:47.133" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13483" PostTypeId="1" AcceptedAnswerId="13485" CreationDate="2020-06-07T01:48:45.157" Score="1" ViewCount="741" Body="&lt;p&gt;I have raw gene read counts and would like to perform an analysis across multiple samples. I've found conflicting info online on how this should be done. One commonality however is that FPKM/RPKM should not be used. On the other hand, there seems (to me at least) to be conflict about whether TPM is suitable. For example, &lt;a href=&quot;https://www.youtube.com/watch?v=TTUrtCY2k-w&quot; rel=&quot;nofollow noreferrer&quot;&gt;this StatQuest video&lt;/a&gt; seems to suggest that TPM is suitable for analysis across samples. One person on &lt;a href=&quot;https://www.biostars.org/p/273537/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Biostar post&lt;/a&gt; (among other resources I've looked at) suggested that none of the units (FPKM, RPKM, TPM) are suitable for cross sample comparisons. &#xA;I am quite confused. Can someone clear up the confusion?&lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2020-06-07T11:47:19.037" Title="What unit (TPM, FPKM/RPKM, or other) to use when working across samples" Tags="|differential-expression|gene-expression|normalization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13485" PostTypeId="2" ParentId="13483" CreationDate="2020-06-07T11:47:19.037" Score="3" Body="&lt;p&gt;Neither of RPKM, FPKM or TPM is a good choice. All these methods are similar, they only perform the operations in a slightly different order. Eventually they correct for gene length and sequencing depth. Correction for sequencing depth is necessary but not sufficient in many cases. Correction for length is typically not done for a differential analysis as it reduces the counts of many genes and therefore throws away statistical power. I will not describe this in detail since the StatQuest video series you link includes videos about DESeq2 and edgeR normalization procedures which extensively cover the normalization procedures and why these approaches are superior to RPKM/FPKM/TPM. In short: You have to correct for differences in library composition to compare between samples. Please watch these videos and read the manuals of the established RNA-seq analysis packages such as but not exclusively limma, edgeR and DESeq2. &lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-06-07T11:47:19.037" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13486" PostTypeId="2" ParentId="13478" CreationDate="2020-06-07T17:57:36.663" Score="0" Body="&lt;p&gt;Yes, it does make sense to perform differential abundance testing after one doesn't find compositional differences with PERMANOVA / ANOSIM. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;It seems to me you are drawing an (implicit) analogy to ANOVA and post-hoc tests like the Tukey Honest Significant Difference test. Intuitively, the analogy makes sense, however, these situations are quite different. PERMANOVA / ANOSIM and DESeq2 are operating on different aspects of the data, with different transformations and under different assumptions.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;PERMANOVA and ANOSIM do not operate on the raw counts directly, rather, they test differences between groups by calculating some distance or dissimilarity between the samples. One can choose between several distance / dissimilarity measures, and the test result may be impacted by the chosen measure. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;DESeq2 (and other packages), on the other hand, test for species abundances under a parametric model, operating on the raw counts directly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus (sadly) one is not a natural, optional follow-up of the other, they are complementary and will tell different stories about the data.&lt;/p&gt;&#xA;" OwnerUserId="1642" LastActivityDate="2020-06-07T17:57:36.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13487" PostTypeId="2" ParentId="3477" CreationDate="2020-06-08T07:22:10.573" Score="4" Body="&lt;p&gt;In addition to the answer from @gringer there is a &lt;code&gt;bcftools&lt;/code&gt; &lt;a href=&quot;https://samtools.github.io/bcftools/howtos/plugins.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;plugin&lt;/a&gt; called &lt;code&gt;split&lt;/code&gt; that can do this, but gives you the added ability to output single-sample VCFs by specifying a filename for each sample.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools +split&#xA;&#xA;About: Split VCF by sample, creating single-sample VCFs.&#xA;&#xA;Usage: bcftools +split [Options]&#xA;Plugin options:&#xA;   -e, --exclude EXPR              exclude sites for which the expression is true (applied on the outputs)&#xA;   -i, --include EXPR              include only sites for which the expression is true (applied on the outputs)&#xA;   -k, --keep-tags LIST            list of tags to keep. By default all tags are preserved&#xA;   -o, --output DIR                write output to the directory DIR&#xA;   -O, --output-type b|u|z|v       b: compressed BCF, u: uncompressed BCF, z: compressed VCF, v: uncompressed VCF [v]&#xA;   -r, --regions REGION            restrict to comma-separated list of regions&#xA;   -R, --regions-file FILE         restrict to regions listed in a file&#xA;   -S, --samples-file FILE         list of samples to keep with second (optional) column for basename of the new file&#xA;   -t, --targets REGION            similar to -r but streams rather than index-jumps&#xA;   -T, --targets-file FILE         similar to -R but streams rather than index-jumps&#xA;Examples:&#xA;   # Split a VCF file&#xA;   bcftools +split input.bcf -Ob -o dir&#xA;&#xA;   # Exclude sites with missing or hom-ref genotypes&#xA;   bcftools +split input.bcf -Ob -o dir -i'GT=&quot;alt&quot;'&#xA;&#xA;   # Keep all INFO tags but only GT and PL in FORMAT&#xA;   bcftools +split input.bcf -Ob -o dir -k INFO,FMT/GT,PL&#xA;&#xA;   # Keep all FORMAT tags but drop all INFO tags&#xA;   bcftools +split input.bcf -Ob -o dir -k FMT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;So if you had the following samples file &lt;code&gt;samples.tsv&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sample1    sample1&#xA;sample2    sample2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;You can run it and get the following&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools +split -S samples.tsv -o outdir in.vcf&#xA;$ ls&#xA;in.vcf sample1.vcf sample2.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Without the second column, you would just get a single VCF with the two samples in it (as you would with &lt;code&gt;view&lt;/code&gt;)&lt;/p&gt;&#xA;" OwnerUserId="149" LastActivityDate="2020-06-08T07:22:10.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13489" PostTypeId="1" CreationDate="2020-06-08T15:03:10.450" Score="2" ViewCount="728" Body="&lt;p&gt;I have a FASTQ file:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@NS500455:80:HG7TNBGXB:1:11101:17723:1055 1:N:0:ATCACG&#xA;ACTTANGTGTATGTAAACTTCCGACTTCAACTGTATAGGGATCCNAGCTCCAATTCGCCCTATAGTGAGTCGTAT&#xA;+&#xA;/AAAA#EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE#EEEEEEEEEEEEEEEEEEEEEEEEEEEEEE&#xA;@NS500455:80:HG7TNBGXB:1:11101:8821:1057 1:N:0:ATCACG&#xA;ACTTANGTGATGTAAACTTCCGACTTCAACTGTATAATAAATATCTAGATCGGAAGAGCACACGTCCGAACTCCA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I want to remove the string  ACTTAAGTGTATGTAAACTTCCGACTTCAACTG from it while retaining the sequences starting from TA onwards.&#xA;I wrote &lt;code&gt;grep &quot;ACTTAAGTGTATGTAAACTTCCGACTTCAACTGTA&quot; SRR_1.fastq | sed &quot;s/ACTTAAGTGTATGTAAACTTCCGACTTCAACTGTA/TA/g&quot;&lt;/code&gt;, but it removes the headers of the FASTQ file as well.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The desired output would be &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;@NS500455:80:HG7TNBGXB:1:11101:17723:1055 1:N:0:ATCACG&#xA;TATAGGGATCCNAGCTCCAATTCGCCCTATAGTGAGTCGTAT&#xA;+&#xA;/AAAA#EEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEEE#EEEEEEEEEEEEEEEEEEEEEEEEEEEEEE&#xA;@NS500455:80:HG7TNBGXB:1:11101:8821:1057 1:N:0:ATCACG&#xA;TATAATAAATATCTAGATCGGAAGAGCACACGTCCGAACTCCA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;How to do this. I dont want to use cutadapt.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kindly help.&lt;/p&gt;&#xA;" OwnerUserId="7236" LastActivityDate="2020-06-11T17:31:57.770" Title="FASTQ file trimming" Tags="|fastq|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13490" PostTypeId="2" ParentId="13489" CreationDate="2020-06-08T15:10:12.843" Score="3" Body="&lt;p&gt;Skip the &lt;code&gt;grep&lt;/code&gt;, use just the &lt;code&gt;sed&lt;/code&gt;. Unless the header contains the long sequence as well (which I doubt it will), your command will work.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sed 's/^ACTTAAGTGTATGTAAACTTCCGACTTCAACTGTA/TA/' SRR_1.fastq &amp;gt; SRR_1.edited.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastActivityDate="2020-06-08T15:10:12.843" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13491" PostTypeId="2" ParentId="13489" CreationDate="2020-06-08T15:22:38.407" Score="2" Body="&lt;p&gt;If you need the quality vector for each read, then your code will create a shift because it does not cut the quality vector. You should rather use a trimming tool a systematically cut all 33 first characters. &lt;strong&gt;&lt;a href=&quot;https://github.com/OpenGene/fastp&quot; rel=&quot;nofollow noreferrer&quot;&gt;fastp&lt;/a&gt;&lt;/strong&gt; will allow you to do it easily with the &lt;code&gt;-f, --trim_front1&lt;/code&gt; argument.&lt;/p&gt;&#xA;" OwnerUserId="8749" LastActivityDate="2020-06-08T15:22:38.407" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13493" PostTypeId="1" CreationDate="2020-06-08T20:42:53.457" Score="2" ViewCount="23" Body="&lt;p&gt;Many scRNA-seq clustering/visualization methods are based on nearest neighbors. The maximum number of neighbors is limited by the neighborhood size. Is there a method to determine the minimum number of neighbors? Now that most datasets are well in the thousands of cells, this point is largely irrelevant, so maybe that is why not much guidance on this issue is available. However, if you something like 50 cells, it becomes far more critical.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have a deeply sequenced dataset, your neighbors should be more reliable (less noisy) than one with shallow coverage, so you should be able to use less. Is there a way to somehow quantify that?&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2020-06-08T20:42:53.457" Title="minimum number of neighbors for scRNA-seq analysis" Tags="|scrnaseq|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13494" PostTypeId="2" ParentId="10939" CreationDate="2020-06-08T22:38:04.400" Score="0" Body="&lt;p&gt;I think &quot;the best&quot; way to do this is going to depend on what your question is, unfortunately.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For example, let's say you're trying to find which genes vary by tumour stage.&#xA;Variance of a given gene might be high across all samples, in which case it has no relation to the stage and might be useless for you.&#xA;Alternatively, you may have a lowly-to-moderately expressed gene that does not vary much between samples of the same stage, but differs between stages, which is useful for you.&#xA;However, because it's not highly-expressed, its variance may be small compared to the variance of highly-expressed genes and skipped over.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But if you're trying to do sequencing batch correction to identify batch effects, looking at raw sequencing counts and doing some type of quantile normalization might be all you need.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;So I don't think there's a one-size-fits-all answer to your question, unless you're able to refine it a little bit&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2020-06-08T22:38:04.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13495" PostTypeId="1" AcceptedAnswerId="13516" CreationDate="2020-06-09T02:32:54.200" Score="0" ViewCount="84" Body="&lt;p&gt;When mapping reads to a reference genome, how is it possible to tell which part of the genome the read is referencing if the genome possibly has the same repeated sequences in different areas? What I mean by this is let's say for the sake of argument that you have a read &quot;AGCTAGCT&quot;. If the reference genome has two places where the sequence &quot;AGCTAGCT&quot; is present, how do we tell which one it's coming from? I'm assuming that the length of the read is chosen long enough such that this doesn't happen but perhaps I'm wrong?&lt;/p&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="8601" LastEditDate="2020-06-09T03:04:55.510" LastActivityDate="2020-06-11T14:38:41.833" Title="Clarification about how mapping to a reference genome works in RNA seq" Tags="|rna-seq|reference-genome|reads|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13496" PostTypeId="1" CreationDate="2020-06-09T03:51:39.107" Score="1" ViewCount="155" Body="&lt;p&gt;I am trying to use gffcompare to compare my assembled transcriptome to a reference gtf that contains information about small open reading frames (sORFs). The reference gtf was obtained by processing downloaded data from several sORF databases and is called &lt;code&gt;all_38.gtf&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I used the command: &lt;code&gt;gffcompare -r path/to/all_38.gtf -o /output_folder my_transcriptome.gtf&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;mytranscriptome.gtf&lt;/code&gt; is generated by assembling my Bam files with reference to ensembl hg38 v99 reference.&lt;/p&gt;&#xA;&lt;p&gt;Upon completion, the output of the error message log file reads:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;0 reference transcripts loaded.&#xA;  237788 query transfrags loaded.&#xA;  2714 duplicate query transfrags discarded.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The expected output files are generated except for the .refmap file which I need for downstream analysis. I assume this is because 0 reference transcripts were loaded into the program. Has anyone encountered similar issues? And is possibly something wrong with my custom reference gtf file?&lt;/p&gt;&#xA;&lt;p&gt;My reference gtf can be found &lt;a href=&quot;https://github.com/gpertea/gffcompare/files/4746893/all_38.gtf.zip&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The first few lines of my input gtf look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   # StringTie version 2.1.2&#xA;1       StringTie       transcript      11869   14409   1000    +       .       gene_id &amp;quot;MSTRG.1&amp;quot;; transcript_id &amp;quot;ENST00000456328&amp;quot;; gene_name &amp;quot;DDX11L1&amp;quot;; ref&#xA;_gene_id &amp;quot;ENSG00000223972&amp;quot;;&#xA;1       StringTie       exon    11869   12227   1000    +       .       gene_id &amp;quot;MSTRG.1&amp;quot;; transcript_id &amp;quot;ENST00000456328&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;DDX1&#xA;1L1&amp;quot;; ref_gene_id &amp;quot;ENSG00000223972&amp;quot;;&#xA;1       StringTie       exon    12613   12721   1000    +       .       gene_id &amp;quot;MSTRG.1&amp;quot;; transcript_id &amp;quot;ENST00000456328&amp;quot;; exon_number &amp;quot;2&amp;quot;; gene_name &amp;quot;DDX1&#xA;1L1&amp;quot;; ref_gene_id &amp;quot;ENSG00000223972&amp;quot;;&#xA;1       StringTie       exon    13221   14409   1000    +       .       gene_id &amp;quot;MSTRG.1&amp;quot;; transcript_id &amp;quot;ENST00000456328&amp;quot;; exon_number &amp;quot;3&amp;quot;; gene_name &amp;quot;DDX1&#xA;1L1&amp;quot;; ref_gene_id &amp;quot;ENSG00000223972&amp;quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3&gt;EDIT: Output when running with --debug option&lt;/h3&gt;&#xA;&lt;p&gt;Running with debug option set results in an error file with more information about duplicated transfrags but not about reference transcripts. I've included a few lines of the error file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Prefix for output files: norfs&#xA;Loading reference transcripts..&#xA;  0 reference transcripts loaded.&#xA;  237788 query transfrags loaded.&#xA;        Query transcript ENST00000655384 discarded (duplicate of ENST00000667728)&#xA;        Query transcript ENST00000415295 discarded (duplicate of ENST00000670824)&#xA;        Query transcript ENST00000263741 discarded (duplicate of ENST00000660930)&#xA;        Query transcript ENST00000473215 discarded (duplicate of ENST00000347370)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The --debug option also generates 3 additional files:&lt;code&gt;&amp;lt;outprefix&amp;gt;.Q_discarded.lst&lt;/code&gt;, &lt;code&gt;&amp;lt;outprefix&amp;gt;.missed_introns.gff&lt;/code&gt;, &lt;code&gt;&amp;lt;outprefix&amp;gt;.R_missed.lst&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, in my output folder, I only have the &lt;code&gt;missed_introns.gff&lt;/code&gt; and &lt;code&gt;R_missed.gff&lt;/code&gt; files. The &lt;code&gt;Q_discard.lst&lt;/code&gt; is missing.&lt;/p&gt;&#xA;&lt;p&gt;One thing that I'm not sure matters is that &lt;code&gt;my_transcriptome.gtf&lt;/code&gt; was generated by aligning and assembling with reference to an &lt;code&gt;Ensembl gtf&lt;/code&gt; file. The gtf that I am using as reference input in gffcompare is downloaded from &lt;a href=&quot;https://firebasestorage.googleapis.com/v0/b/phoenix-6686b.appspot.com/o/nORFsDB.1.1.gtf?alt=media&amp;amp;token=a3e41fe4-b1e5-4002-9384-b9d48bd5e25d&quot; rel=&quot;nofollow noreferrer&quot;&gt;norfs.org&lt;/a&gt;. It contains information about small Open Reading Frames (sORFs). Could the different gtf files used be causing this issue?&lt;/p&gt;&#xA;&lt;p&gt;If so, the solution would be to assemble my transcripts using the sORFs gtf as reference. However, I'm not sure if this is a good idea since sORFs have relatively short lengths (&amp;lt; 300bp) and this might screw up the assembly.&lt;/p&gt;&#xA;" OwnerUserId="6646" LastEditorUserId="-1" LastEditDate="2020-06-18T08:30:41.743" LastActivityDate="2020-06-14T03:02:12.607" Title="Gffcompare issue: 0 reference transcripts loaded" Tags="|rna-seq|sequence-annotation|assembly|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13497" PostTypeId="1" CreationDate="2020-06-09T07:36:04.513" Score="1" ViewCount="62" Body="&lt;p&gt;I have some questions regarding RNA seq analysis if you can suggest anything it will help me a lot.&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I am currently normalizing RNA seq data for comparing genes expression within and between samples. Which normalization method would you recommend for this type of analysis? FPKM, TPM, TMM? Also I want to make a heatmap to see genes expressed in different conditions. Do you think transforming normalized data (like log2, z-score) is a good idea for this?&lt;/li&gt;&#xA;&lt;li&gt;Also I want to build a co expression network, I am just wondering if normalization like FPKM, TPM, TMM  has any influence on building a coexpression network?&lt;/li&gt;&#xA;&lt;li&gt;Another thing I want to do is use Pearson correlation but I am confused if it will only find linear relationships among the normally distributed data.  But normalization methods do not assume that counts to be normally distributed. So, Is it a bad idea to find pairwise coexpression of genes using pearson correlation? If so, which method would you recommend is reliable for building coexpression networks?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;Please help me with this, I will be very grateful to you&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="8988" LastActivityDate="2020-06-09T07:36:04.513" Title="Regarding RNA seq data analysis and building coexpression network" Tags="|rna-seq|statistics|networks|normalization|correlation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13498" PostTypeId="1" CreationDate="2020-06-09T08:54:28.177" Score="1" ViewCount="333" Body="&lt;p&gt;I have a total of 35 tumor samples classified into 4 subtypes. Subtype A, B, C, and D. I have RNAseq data. I'm interested in identifying modules related to each subtype with co-expression network analysis. I have read the WGCNA tutorial and create a &lt;code&gt;trait.csv&lt;/code&gt; file like below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Showing here some rows of the &lt;code&gt;trait.csv&lt;/code&gt; file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/pP3NU.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;SamInfo &amp;lt;- read.csv(&quot;trait.csv&quot;)&#xA;&#xA;Samples = rownames(datExpr);&#xA;traitRows = match(Samples, SamInfo$Samples);&#xA;datTraits = SamInfo[traitRows, -1];&#xA;rownames(datTraits) = SamInfo[traitRows, 1];&#xA;&#xA;nGenes = ncol(datExpr);&#xA;#15648L&#xA;&#xA;nSamples = nrow(datExpr);&#xA;#35L&#xA;&#xA;#calculate eigengenes (1st principal component) of modules&#xA;MEs0 = moduleEigengenes(datExpr, dynamicColors)$eigengenes&#xA;MEs = orderMEs(MEs0)&#xA;moduleTraitCor = cor(MEs, datTraits, use = &quot;p&quot;);&#xA;moduleTraitPvalue = corPvalueStudent(moduleTraitCor, nSamples);&#xA;&#xA;#Will display correlations and their p-values&#xA;textMatrix = paste(signif(moduleTraitCor, 2), &quot;\n(&quot;, signif(moduleTraitPvalue, 1), &quot;)&quot;, &#xA;                   sep = &quot;&quot;);&#xA;dim(textMatrix) = dim(moduleTraitCor)&#xA;par(mar = c(6, 8.5, 3, 3));&#xA;&#xA;#Display the correlation values within a heatmap plot&#xA;labeledHeatmap(Matrix = moduleTraitCor, xLabels = names(datTraits), &#xA;               yLabels = names(MEs), ySymbols = names(MEs), colorLabels = FALSE, &#xA;               colors = blueWhiteRed(50), textMatrix = textMatrix, setStdMargins = FALSE, &#xA;               cex.text = 0.5, zlim = c(-1,1), main = paste(&quot;Module-trait relationships&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The Correlation between Subtypes and Modules Heatmap looks like below:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/F161r.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/F161r.jpg&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;For functions identification (Pathway Analysis), from the above output for each subtype, should I select only the modules that were significantly positively correlated with subtypes, or Can I also use the significantly negatively correlated modules?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Is it good to use the modules that were not significant for pathway analysis?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;In case if I can use positive and negative correlated modules, Can I combine those modules and use those genes for Pathway analysis?&lt;/p&gt;&#xA;" OwnerUserId="1377" LastEditorUserId="1377" LastEditDate="2020-06-12T09:20:06.417" LastActivityDate="2023-06-27T16:07:20.907" Title="Which module to select for Pathway analysis based on Module trait correlation and pvalue?" Tags="|r|rna-seq|pathway|wgcna|coexpression|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13499" PostTypeId="1" CreationDate="2020-06-09T14:02:55.130" Score="0" ViewCount="159" Body="&lt;pre&gt;&lt;code&gt;library(seqinr)&#xA;library(ape)&#xA;library(phangorn)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;alignment file &lt;a href=&quot;https://drive.google.com/file/d/10P6qF3nrXlnm-tUONK5bTAPqR4wapX-q/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;data file&lt;/a&gt;&#xA;    sylvia.aln &amp;lt;- read.alignment(&quot;abhi_seq/all_seq_no_gal.fas&quot;, format=&quot;fasta&quot;)&#xA;    sylvia.aln&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;kaks(sylvia.aln, verbose=TRUE)&#xA;&#xA;&#xA;Warning message:&#xA;In kaks(sylvia.aln) : sequence lengths are not a multiple of 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It ends with this warning and there are no result.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;When i ran one of the seqinr example data what I see there are no gaps in the sequence files. If i understand the results is due to the gap I suppose. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help or suggestion would be really appreciated.&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2020-06-09T15:04:10.987" Title="Calculating Synonymous and non-synonymous" Tags="|r|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13500" PostTypeId="1" CreationDate="2020-06-09T14:43:08.483" Score="0" ViewCount="79" Body="&lt;p&gt;I have the following dataframe that is comprised of enzyme substrates as the variable and then the unique number of times enzymes capiable of degrading this substrates appeared in my genomes and then,total count of times all of those enzymes appeard in my samples. For example, a 7 unique enzymes were found to be capable of degrading chitin, and they occured a total of 298 times across my genomes.&lt;/p&gt;&#xA;&lt;p&gt;I wish to see if any particular substrate has significantly larger occurence in both unique and total, than the rest of the substrates. Would outlier detection be appropriate for this or would a simple T-test do?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DdLVZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DdLVZ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7031" LastEditorUserId="7374" LastEditDate="2024-03-23T01:25:56.213" LastActivityDate="2024-03-23T01:25:56.213" Title="Is outlier detection the right choice for determining if a value lies significantly away from the mean in a population in gene frequency?" Tags="|statistics|outlier|chi-square|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13502" PostTypeId="1" AcceptedAnswerId="13509" CreationDate="2020-06-09T14:50:20.400" Score="0" ViewCount="235" Body="&lt;p&gt;I am looking at the videos at a &lt;a href=&quot;http://diytranscriptomics.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DIY Transcriptomics course&lt;/a&gt; and the speaker mentions that to run Kallisto for read alignment with paired end sequencing, one would enter the following:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3w2hf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3w2hf.png&quot; alt=&quot;enter image description here&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;where the sample files at the end refer to the two different end reads (forward and reverse read files). However, browsing the NCBI for paired end data, I noticed that only 1 Fastq file is provided per replicate for a paired end procedure (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Traces/study/?acc=PRJNA574273&amp;amp;o=acc_s%3Aa&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;'s an example). So I'm a bit confused about this. &lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2020-06-13T06:56:56.723" Title="Files for paired end RNA sequencing" Tags="|rna-seq|kallisto|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13503" PostTypeId="2" ParentId="13502" CreationDate="2020-06-09T15:03:29.427" Score="0" Body="&lt;p&gt;Actually after clicking on one of the &quot;Runs&quot; and going to &quot;data access&quot; I noticed two files are provided per run &lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2020-06-09T15:03:29.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13504" PostTypeId="2" ParentId="13499" CreationDate="2020-06-09T15:04:10.987" Score="1" Body="&lt;p&gt;Woaa that is complicated data. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;The reason it fails is easy, correcting it is harder. It fails because the open reading frame is lost during in silico translation, e.g. several gaps have 4 indels and the final nucleotides after the final indel are always 4. Thus it will invariably encounter a stop codon OR it will simply refuse to deal with an indel if it is incorporated into a triplet codon. There is no triplet codon model for an indel so the whole calculations stops.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Correcting data that complex ... the easiest way to do it is align you entire data set using amino acids, e.g. muscle. Have an identical nucleotide data set. MAKE SURE the names and positions are identical between your amino acid and nucleotide data, as a programmer this isn't hard to do however, everyone has their own scripts to do this. If you don't do this you next step will fall over. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use a amino acid to nucleotide alignment translator. I use &lt;code&gt;transalign&lt;/code&gt; (I think its EMBOSS). This will give you an inframe alignment. I noted many/all of you start codons are ATG which is a good sign. Check the nucleotide output and make sure there is no overhangs of the triplet codon at the terminal 3' end and check in &lt;code&gt;Jalview&lt;/code&gt; or whatever your alignment editor of choice is that the translation is okay. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will alignment work in the synonymous/non-synonymous analysis and all subsequent/complimentary analyses. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you have an intron in protein your you'll need to remove it (hopefully not).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Finally you need to be careful of '3rd codon saturation' in your data because that heavily affects your trees and its certainly a risk given the heterogeneity of the indel structure, but first I would get an inframe alignment. There are checks for saturation (I'm sure even R can do this).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-06-09T15:04:10.987" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13505" PostTypeId="2" ParentId="13500" CreationDate="2020-06-09T15:23:15.293" Score="0" Body="&lt;p&gt;I agree you have the right idea because that data doesn't look at all linear, so regression analysis looks really complicated. Your approach a straight comparison keeps it simple and is within the biological question you are asking, &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I would use a non-parametric such as Wilcoxon instead of a T-test. I don't fully know the null distribution you are comparing it against. Wilcoxan doesn't require an assumption of normality. T-tests invariably require a transformation to normality and that gets complicated, but a log or sqrt transformation might be okay. If Wilcoxan produces a no significant difference you report it and move onto a T-test.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is difficult to use outlier analysis with 9(?) observations, thats used if you have a way-out result and can afford to exclude it (&gt;3 standard deviations). Outlier analysis is to remove a single result that is messing up your analysis and doesn't affect the bulk of the data. You alot of observations to be considering outlier analysis.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-06-09T15:23:15.293" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13506" PostTypeId="1" AcceptedAnswerId="13513" CreationDate="2020-06-09T16:47:46.613" Score="0" ViewCount="217" Body="&lt;p&gt;I have a dataframe with two columns: &lt;code&gt;$1 = transcript ID&lt;/code&gt;, &lt;code&gt;$2 = expression counts (tpm)&lt;/code&gt;. This comes from a merging of several kallisto counts (if it matters). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I would like to build, in the most basic R way (no tidyverse), a dataframe with as column $1 the unique transcript IDs, and the expression countsas following columns (as many as needed). &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Sample input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ head test1&#xA;G01000001_1 0&#xA;G01000001_1 0.00855717&#xA;G01000001_1 0.0113467&#xA;G01000001_1 0.0121086&#xA;G01000001_1 0.0121108&#xA;G01000001_1 0.0148402&#xA;G01000001_1 0.0183057&#xA;G01000001_1 0.0268394&#xA;G01000001_1 0.0298547&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Sample output:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ head -3 test2&#xA;&#xA;G01000001_1 0  0.00855717 0.0113467 0.0121086 0.0121108 0.0148402 0.0183057 0.0268394 0.0298547 0.0444686 0.046795 0.0547494 0.0640871 0.0674897 0.0719834 0.082262 0.0867695 0.094905 0.10269 0.11488 0.121549 0.131143 0.141432 0.145758 0.167886 0.184162 0.192757 0.200921 0.249595 0.283296 0.430173 0.473344 0.518981 0.674573 0.701431 0.845816 2.77479 6.28394&#xA;G01000002_1 0  0.0277237 0.0294273 0.029438 0.0379258 0.0606153 0.0614727 0.069608 0.142541 0.197644 0.218152 0.22193 0.227401 0.230179 0.259295 0.279463 0.289982 0.292763 0.303268 0.305259 0.308236 0.319988 0.532091 0.632095 0.707766 0.746436 0.821887 0.822496 0.997334 1.06616 1.21752 1.48436 10.0862 2.01763 3.94584 5.02089 7.55767 8.3501&#xA;G01000003_1 0  0.00265959 0.00322424 0.00920435 0.454749 0.692183 0.709159 0.948286 1.11827 1.39854 1.39985 1.5781 1.86343 10.2554 10.7185 11.0315 14.342 15.2383 2.13068 2.41792 2.52833 2.77868 3.56981 3.84991 3.97521 4.10445 4.16344 4.26902 4.97202 47.8293 47.9114 5.21789 5.21926 5.35712 5.49942 5.7085 5.87654 50.7484 51.3721 56.7279 7.35062 7.54485 7.64363e-05 8.1367 8.34828&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This could be achieved in awk, but I would really like an R solution. &#xA;For the logic, here is the awk way:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sort test1 | awk '{if(a!=$1) {a=$1; printf &quot;\n%s%s&quot;,$0,FS} else {a=$1;$1=&quot;&quot;;printf $0 }} END {printf &quot;\n&quot; }' &amp;gt; test2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="294" LastActivityDate="2020-06-10T05:25:01.953" Title="merge rows (to columns) in R dataframe based on IDs" Tags="|r|text-processing|merge|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13508" PostTypeId="2" ParentId="13498" CreationDate="2020-06-09T17:08:55.100" Score="1" Body="&lt;p&gt;If you can run pathway analysis programmatically on all modules, yes, do it on all modules. Otherwise, select the modules with strongest associations with each subtype, both positive and negative. I would keep the modules separate because they may have very different biological functions.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2020-06-09T17:08:55.100" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13509" PostTypeId="2" ParentId="13502" CreationDate="2020-06-09T17:18:58.560" Score="0" Body="&lt;p&gt;Building off of the answer by @An Ignorant Wanderer:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Essentially there are indeed the paired end data files available, but it takes a little bit of extra navigation to reach it.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Using the specific example you provided of BioProject &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Traces/study/?acc=PRJNA574273&amp;amp;o=acc_s%3Aa&quot; rel=&quot;nofollow noreferrer&quot;&gt;PRJNA574273&lt;/a&gt;:&#xA;First select a specific sequencing &quot;Run,&quot; aka the sequencing of a specific sample / experimental condition. For example, click on the link to run: &lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra?run=SRR10189234&quot; rel=&quot;nofollow noreferrer&quot;&gt;SRR10189234&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;This brings you to the default tab: &quot;Metadata.&quot; As @An Ignorant Wanderer pointed out, then you can navigate to the &quot;Data Access&quot; tab.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Then look under the &quot;Original format&quot; subheading. Here you'll find links to two files, corresponding to the two orientations of the paired end sequence files.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Edit: Also please see the answer by @burger, the website linked in that answer is much more user friendly in my opinion.&lt;/p&gt;&#xA;" OwnerUserId="7185" LastEditorUserId="7185" LastEditDate="2020-06-12T13:11:03.340" LastActivityDate="2020-06-12T13:11:03.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13510" PostTypeId="1" CreationDate="2020-06-09T18:41:09.130" Score="0" ViewCount="540" Body="&lt;p&gt;I am trying to build an index on kallisto on mac using R studio and have saved my FASTA file to my desktop, however, when I run &lt;code&gt;kallisto index -i DI_mRNA.index DI_mRNA.fa.gz&lt;/code&gt;, there is an error saying &lt;code&gt;Error: FASTA file not found DI_mRNA.fa.gz&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="8994" LastEditorUserId="48" LastEditDate="2020-06-15T13:21:24.067" LastActivityDate="2020-06-15T13:21:24.067" Title="Error: FASTA file not found using building kallisto index" Tags="|kallisto|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13511" PostTypeId="2" ParentId="13502" CreationDate="2020-06-09T18:48:12.743" Score="1" Body="&lt;p&gt;The problem is that the SRA website can be a little confusing. View the same project on ENA where the organization is more human-readable: &lt;a href=&quot;https://www.ebi.ac.uk/ena/browser/view/PRJNA574273&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/ena/browser/view/PRJNA574273&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2020-06-09T18:48:12.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13513" PostTypeId="2" ParentId="13506" CreationDate="2020-06-09T21:26:21.887" Score="1" Body="&lt;p&gt;For a data.frame named &lt;code&gt;foo&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sink('some_output_file.txt')&#xA;uvals = unique(foo[,1])&#xA;for(uIDX in c(1:length(uvals))) {&#xA;    gname = uvals[uIDX]&#xA;    IDX = which(foo[,1] == gname)&#xA;    cat(sprintf(&quot;%s\t%s\n&quot;, gname, paste0(foo[IDX,2], collapse=&quot;\t&quot;)))&#xA;}&#xA;sink()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-06-09T21:26:21.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13514" PostTypeId="2" ParentId="13510" CreationDate="2020-06-09T21:30:32.853" Score="2" Body="&lt;p&gt;As mentioned in a comment, your working directory lacks the fasta file.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-06-09T21:30:32.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13515" PostTypeId="2" ParentId="13495" CreationDate="2020-06-09T21:40:44.150" Score="1" Body="&lt;p&gt;You can't always know exactly where every read originated, even if you use long reads to sequence full-length transcripts (odds are there's at least one transcript that's duplicated in the genome). This is also not exactly a problem, such reads can either be discarded or reported as multimapping.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-06-09T21:40:44.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13516" PostTypeId="2" ParentId="13495" CreationDate="2020-06-09T22:27:39.140" Score="1" Body="&lt;p&gt;As the other answers/comments have covered, some reads will inevitably align to multiple positions in the genome.&#xA;This can come from a variety of sources, like aligning to repetitive regions of the genome, short read fragments (that will likely match multiple regions in the genome, even if it, in theory, came from a single location), and low base call qualities (the aligner doesn't have lots of confidence aligning it it to one location because it could map to another, with some likely mismatched calls in the read).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This isn't inherently a problem, depending on the analysis you want to do after aligning.&#xA;In fact, having a single read that is compatible with multiple transcripts is one of the ideas behind the reference index used in &lt;a href=&quot;http://www.nature.com/nbt/journal/v34/n5/full/nbt.3519.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;kallisto&lt;/a&gt;, and the &quot;transcript compatibility counts&quot;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you want to minimize this effect, two easy ways are to create longer reads on your sequencer, or to use paired-end reads (in theory, both ends should align relatively close to each other, so if one of the mates maps to multiple places, you can borrow from the confidence of the alignment of its mate).&lt;/p&gt;&#xA;" OwnerUserId="240" LastEditorUserId="240" LastEditDate="2020-06-11T14:38:41.833" LastActivityDate="2020-06-11T14:38:41.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13517" PostTypeId="2" ParentId="13506" CreationDate="2020-06-10T05:25:01.953" Score="1" Body="&lt;p&gt;For that purpose, you may also use the &lt;strong&gt;reshape2&lt;/strong&gt; R package which allows you to switch wide to long format for data.frames. However, for that, you need an additional column with sample identifiers, e.g.:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;G01000001_1 Sample1 0&#xA;G01000001_1 Sample2 0.00855717&#xA;G01000001_1 Sample3 0.0113467&#xA;G01000002_1 Sample1 0.0121086&#xA;G01000002_1 Sample2 0.0121108&#xA;G01000002_1 Sample3 0.0148402&#xA;G01000003_1 Sample1 0.0183057&#xA;G01000003_1 Sample2 0.0268394&#xA;G01000003_1 Sample3 0.0298547&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Then, load and convert the data in R:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;library(reshape2)&#xA;a&amp;lt;-read.table(&quot;test&quot;)&#xA;dcast(a, V1~V2, value.var = &quot;V3&quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8749" LastActivityDate="2020-06-10T05:25:01.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13520" PostTypeId="1" CreationDate="2020-06-11T09:10:10.390" Score="1" ViewCount="6259" Body="&lt;p&gt;I am trying to remove a row of a dataframe in R however because the dataframe is so large when I use:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rawCounts &amp;lt;- rawCounts[rawCounts != geneID, ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I get the error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;ERROR in rawCounts[rawCounts != geneID, ]: (subscript) logical subscript too long&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there another way of removing a specific row in a large dataframe that is not so memory intensive?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have quite a bit of memory, memory.limit() returns:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; memory.limit()&#xA;[1] 16318 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8706" LastActivityDate="2020-06-11T10:53:57.777" Title="ERROR: (subscript) logical subscript too long" Tags="|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13521" PostTypeId="1" CreationDate="2020-06-11T09:37:38.203" Score="2" ViewCount="2114" Body="&lt;p&gt;I am suddenly running into an error when running the DESeqDataSetFromTximport&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;txi.rsem &amp;lt;- tximport(files, type = &quot;rsem&quot;, txIn = FALSE, txOut = FALSE)&#xA;dds &amp;lt;- DESeqDataSetFromTximport(txi = txi.rsem,&#xA;                                colData = SampleFile,&#xA;                                design = ~ Compound)`&#xA;&#xA;using counts and average transcript lengths from tximport&#xA;Error in DESeqDataSetFromTximport(txi = txi.rsem, colData = SampleFile,  : &#xA;  all(lengths &amp;gt; 0) is not TRUE &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;any suggestions what causes this error?&lt;/p&gt;&#xA;" OwnerUserId="9006" LastEditorUserId="77" LastEditDate="2020-06-11T11:58:28.967" LastActivityDate="2021-04-21T09:34:31.553" Title="DESeqDataSetFromTximport all(lengths &gt; 0) is not TRUE" Tags="|deseq2|tximport|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13522" PostTypeId="2" ParentId="13520" CreationDate="2020-06-11T09:42:58.360" Score="2" Body="&lt;p&gt;I am answering my own question here. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I realised I should have been using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;rawCounts &amp;lt;- rawCounts[rownames(rawCounts) != geneID, ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Problem solved.&lt;/p&gt;&#xA;" OwnerUserId="8706" LastEditorUserId="3541" LastEditDate="2020-06-11T10:53:57.777" LastActivityDate="2020-06-11T10:53:57.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13523" PostTypeId="1" CreationDate="2020-06-11T12:51:50.420" Score="1" ViewCount="34" Body="&lt;p&gt;I am attempting to create a reference library of DNA plant barcodes in the ITS2 barcode region for plants from a specific region (Panama). I downloaded all the sequences for plants that resulted in a search for “ITS2” “Panama” and “Viridiplantae” on NCBI GeneBank. For the purpose of assigning sequences of unknown plants (obtained from environmental DNA studies) based on similarity to the known sequences I want to clean up my data and present each sequence with its corresponding taxonomic ranks - Kingdom ; Phylum ; Class ; Order ; Family ; Genus; Species.&lt;/p&gt;&#xA;&lt;p&gt;My problem is that the taxonomic information for each sequence upload in NCBI is all over the place –e.g. some sequences have additional taxonomic ranks listed in their information and there is no clear definition of what corresponds to which taxonomic rank. Since there are a few hundred species I want to refrain from filling in the information manually and I hoped to solve this problem by mining the taxonomic ranks I need from a different database according to the species or genus name in the sequence information. However, I have yet to find a dataset that is comprehensive enough to include all the plants and is simple to access.&#xA;Do you have any suggestions on how to achieve my goal?&#xA;Thanks.&lt;/p&gt;&#xA;&lt;p&gt;According to a recent study, a mini DNA barcode region located on the ITS2 showed quite promising results for plant discrimination:&lt;/p&gt;&#xA;&lt;p&gt;Moorhouse-Gann et al.(2018) New universal ITS2 primers for high-resolution herbivory analyses using DNA metabarcoding in both tropical and temperate zone. [&lt;a href=&quot;https://www.nature.com/articles/s41598-018-26648-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;]&lt;/p&gt;&#xA;" OwnerUserId="8957" LastEditorUserId="73" LastEditDate="2022-09-02T11:05:39.790" LastActivityDate="2022-09-02T11:05:39.790" Title="Generating taxonomic hierarchy by species/genus name" Tags="|sequence-analysis|phylogenetics|taxonomy|barcode|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13524" PostTypeId="1" CreationDate="2020-06-11T14:46:55.117" Score="4" ViewCount="95" Body="&lt;p&gt;Are there any publicly available databases providing expression data for long non-coding RNAs (lncRNAs) across cell types of multicellular organisms? Alternatively, are there lesser known &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgGateway&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC&lt;/a&gt; tracks for this?&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Example 1:&lt;/strong&gt; I want to compare tissue-specific expression of the human lncRNA &lt;em&gt;Xist&lt;/em&gt; across HeLa, GM12878, and HEK293 cell types [or&#xA;more].&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Example 2:&lt;/strong&gt; I want to compare tissue-specific expression of the plant lncRNA &lt;em&gt;COLDAIR&lt;/em&gt; across cortex, phloem, and trichoblast cell&#xA;types [or more].&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9007" LastEditorUserId="9007" LastEditDate="2023-12-12T17:16:08.883" LastActivityDate="2023-12-12T17:16:08.883" Title="Where can I find lncRNA expression data for different cell types?" Tags="|public-databases|ucsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13525" PostTypeId="2" ParentId="13489" CreationDate="2020-06-11T17:31:57.770" Score="4" Body="&lt;p&gt;What application are you using that will be okay with a mismatching quality string?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Use trimmomatic or cutadapt, or bcl2fastq could probably do the job too.  Those will also have the benefit of dealing properly with one-off errors, which awk and sed won't.  &lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-06-11T17:31:57.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13528" PostTypeId="1" CreationDate="2020-06-12T08:59:53.500" Score="0" ViewCount="644" Body="&lt;p&gt;This is one of the &lt;a href=&quot;https://www.biostars.org/p/65618/#443249&quot; rel=&quot;nofollow noreferrer&quot;&gt;question&lt;/a&gt; which is very similar to my problem or almost same to my problem &lt;/p&gt;&#xA;&#xA;&lt;p&gt;I ran with dummy data nucleotide &lt;a href=&quot;https://drive.google.com/file/d/1XfmKO4GZPsSNonhcpwVK5iN5LDd-LS4L/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;alignment&lt;/a&gt; and the amino &lt;a href=&quot;https://drive.google.com/file/d/1gbBgaGNOrXgVjZzNxvNCizLq0n3LL5Lj/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;file&lt;/a&gt; it works perfectly fine,i can run the dn/ds from phangorn without any issue.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;But when i ran with my data nucleotide &lt;a href=&quot;https://drive.google.com/file/d/1I_8eCQlNG0r3vdVpkoGmKJEatmovgF9z/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;alignment&lt;/a&gt; and the amino &lt;a href=&quot;https://drive.google.com/file/d/1kJapQ9UyCPnIr8uIkLi8zrvlT5NGCj-d/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;file&lt;/a&gt; I do get this warning &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;In kaks(carps) : sequence lengths are not a multiple of 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and error.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The &lt;a href=&quot;https://drive.google.com/file/d/14s4YnTPoQPL0fkGxNEklUYrhPoyhxh4C/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Any help or suggestion would be highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2020-06-19T15:01:24.637" Title="Removing stop codons from alignment file for dn/ds calculation" Tags="|phylogenetics|perl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13529" PostTypeId="1" CreationDate="2020-06-12T14:19:01.093" Score="1" ViewCount="1890" Body="&lt;p&gt;I'm a student currently looking at antibody responses against a viral target protein of interest.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have my own, annotated PyMOL session of my protein and I also have .pdbs of crystallised antibody structures that bind to this protein and rougly know whereabouts it binds. I would like to know if it is possible to combine these two structures into a single session to observe or at least overlay the two structures together. Many thanks!&lt;/p&gt;&#xA;" OwnerUserId="9011" LastEditorUserId="1194" LastEditDate="2020-06-15T08:58:01.730" LastActivityDate="2020-06-15T13:41:12.093" Title="Combine structures in the same session - PyMOL" Tags="|structural-biology|pymol|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13530" PostTypeId="2" ParentId="11574" CreationDate="2020-06-12T17:54:36.220" Score="1" Body="&lt;p&gt;You can simply use grep for this purpose as shown below,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;grep -v /translation bio.txt | grep -B100000000 /db_xref= &amp;gt; output_file.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Just make sure that you keep the number with B bigger than the number of lines of your file.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;If you print the contents of the above file you get your desired output as given below,&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;:~$ cat output_file.txt &#xA;gene            complement(8972..9094)&#xA;                     /locus_tag=&quot;HAPS_0004&quot;&#xA;                     /db_xref=&quot;GeneID:7278619&quot;&#xA;     CDS             complement(8972..9094)&#xA;                     /locus_tag=&quot;HAPS_0004&quot;&#xA;                     /codon_start=1&#xA;                     /transl_table=11&#xA;                     /product=&quot;hypothetical protein&quot;&#xA;                     /protein_id=&quot;YP_002474657.1&quot;&#xA;                     /db_xref=&quot;GI:219870282&quot;&#xA;                     /db_xref=&quot;GeneID:7278619&quot;&#xA;     gene            9632..11416&#xA;                     /gene=&quot;frdA&quot;&#xA;                     /locus_tag=&quot;HAPS_0005&quot;&#xA;                     /db_xref=&quot;GeneID:7278620&quot;&#xA;     CDS             9632..11416&#xA;                     /gene=&quot;frdA&quot;&#xA;                     /locus_tag=&quot;HAPS_0005&quot;&#xA;                     /note=&quot;part of four member fumarate reductase enzyme&#xA;                     complex FrdABCD which catalyzes the reduction of fumarate&#xA;                     to succinate during anaerobic respiration; FrdAB are the&#xA;                     catalytic subcomplex consisting of a flavoprotein subunit&#xA;                     and an iron-sulfur subunit, respectively; FrdCD are the&#xA;                     membrane components which interact with quinone and are&#xA;                     involved in electron transfer; the catalytic subunits are&#xA;                     similar to succinate dehydrogenase SdhAB&quot;&#xA;                     /codon_start=1&#xA;                     /transl_table=11&#xA;                     /product=&quot;fumarate reductase flavoprotein subunit&quot;&#xA;                     /protein_id=&quot;YP_002474658.1&quot;&#xA;                     /db_xref=&quot;GI:219870283&quot;&#xA;                     /db_xref=&quot;GeneID:7278620&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1194" LastActivityDate="2020-06-12T17:54:36.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13531" PostTypeId="1" AcceptedAnswerId="16347" CreationDate="2020-06-12T19:19:32.093" Score="0" ViewCount="158" Body="&lt;p&gt;What is a good RNA seq normalization method that allows for across sample comparisons, &lt;strong&gt;and&lt;/strong&gt; allows between transcripts comparisons as well? I read that TMM for example allows across sample comparisons but not between transcripts. On the other hand, I read that TPM is not good enough for between sample comparisons &lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2021-07-09T13:03:48.607" Title="What is a good RNA seq normalization method that allows for across sample comparisons and between transcripts" Tags="|rna-seq|normalization|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13533" PostTypeId="1" AcceptedAnswerId="13534" CreationDate="2020-06-12T23:48:39.967" Score="0" ViewCount="35" Body="&lt;p&gt;Let's say we have two samples, sample 1 and sample 2, and we're running an RNA seq experiment. Consider genes A and B for both. For simplicity, let's say that when we run the RNA seq experiment, we get the following results:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;gene A sample 1 count: 2&lt;/p&gt;&#xA;&#xA;&lt;p&gt;gene B sample 1 count: 4&lt;/p&gt;&#xA;&#xA;&lt;p&gt;gene A sample 2 count: 2000&lt;/p&gt;&#xA;&#xA;&lt;p&gt;gene B sample 2 count: 4000&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Based on my understanding, RNA seq gives relative results. So I guess in this guess, how can you tell whether either one of the following are true:&lt;/p&gt;&#xA;&#xA;&lt;p&gt;1- You sequenced sample 2 a thousand fold more compared to sample 1&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2- Sample 2 really has a thousand fold more expression in both genes A and B &lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2020-06-13T16:13:28.230" Title="Understanding the &quot;relativeness&quot; in RNA seq experiments" Tags="|rna-seq|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13534" PostTypeId="2" ParentId="13533" CreationDate="2020-06-12T23:54:26.690" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;We cannot be sure about this. For all we know, they could have been sequenced to the same depth and there is a biological difference that drives the 1000x (ridiculously large) fold change.&lt;/li&gt;&#xA;&lt;li&gt;See above&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&#xA;&lt;p&gt;The only thing you can say for sure is that gene B is expressed twice as much as gene A in samples 1 and 2. That's what &quot;relative&quot; means - relative expression of genes within a sample.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2020-06-12T23:54:26.690" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13536" PostTypeId="2" ParentId="13533" CreationDate="2020-06-13T02:09:39.163" Score="0" Body="&lt;p&gt;The process of RNA-seq is a sampling process of all the RNA molecules. We cannot make reliable comparisons before we know how deep we sample. And since we know it is a sample from a population, the absolute value does not make sense.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-06-13T02:09:39.163" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13537" PostTypeId="2" ParentId="13502" CreationDate="2020-06-13T06:50:59.357" Score="0" Body="&lt;p&gt;Paired ends is a configuration in sequencing platforms. Illumina is the most popular at the moment and most single-cell sequencing is done using paired-end Illumina sequencing. It varies between single-cell technologies but most 3’ scRNA-Seq protocols use paired ends.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Kallisto is a tool for handling raw data. Many databases supported processed data in BAM format or UMI/read counts. You should not need to run kallisto unless you are involved in handling raw data from sequencing performed by your laboratory. Some databases such as SRA and EGA support raw data in FASTQ format, others do not for ethical reasons or the data is too large.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recommend downloading from SRA via their command line interface (this is documented on their website). It is possible to extract paired ends from an aligned BAM file using bedtools and samtools. I’ve done this with the Macosko et al (Cell 2015) DropSeq data for example. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;Note that in some cases, the barcode is stored in databases as the headers of the FASTQ, BAM, BED, or CTSS file rather than a separate read. Single cell transcriptomics is an emerging field so there isn’t a standard way to do this, it varies between labs and data centres. For single-cell specifically, some databases are being set up, for example: &lt;a href=&quot;https://data.humancellatlas.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://data.humancellatlas.org/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1855" LastEditorUserId="1855" LastEditDate="2020-06-13T06:56:56.723" LastActivityDate="2020-06-13T06:56:56.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13538" PostTypeId="1" CreationDate="2020-06-13T06:55:54.387" Score="1" ViewCount="88" Body="&lt;p&gt;I am reading the Wikipedia page on DNA profiling, specifically &lt;a href=&quot;https://en.wikipedia.org/wiki/DNA_profiling#STR_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;STR analysis&lt;/a&gt;. It says that DNA profiling relies on matching 20 or so loci. It also gives an &lt;a href=&quot;https://www.latimes.com/archives/la-xpm-2008-jul-20-me-dna20-story.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;account of finding false identification&lt;/a&gt;. But that account was published in 2008.&lt;/p&gt;&#xA;&lt;p&gt;What is current state of art on DNA profiling? How many loci are used? Do we still need a database for constructing the probabilistic distribution of matching?&lt;/p&gt;&#xA;&lt;p&gt;I am particularly interested in the move toward DNA sequencing, how does that affect the profiling and matching probability calculation.&lt;/p&gt;&#xA;&lt;p&gt;Suppose we sequence the whole available sample not restricted to particular loci.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;When we have a single clean sample, is it true the probability of matching bounded by the probability of monozygotic twins resulting in the mismatching probability very close to &lt;span class=&quot;math-container&quot;&gt;$1/2$&lt;/span&gt; of the monozygotic twin probability? The latter probability across all population is about &lt;span class=&quot;math-container&quot;&gt;$0.002$&lt;/span&gt;. It this upper bound reachable?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When there are admixture of several people/suspects' DNA in a sample and all the DNA (all 24 chromosomes) are intact.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When there are admixture of several people/suspects' DNA in a sample and some (the word covers the whole spectrum of degree/proportion) DNA's are damaged/missing.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In all the above cases, does the extraneous information such as gender, race/ethnicity affect the matching probability? I suppose the degree of effect increases in the order case 1 &amp;lt; case 2 &amp;lt; case 3.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="8725" LastEditorUserId="3694" LastEditDate="2022-09-02T12:25:55.943" LastActivityDate="2022-09-02T12:25:55.943" Title="How many markers or loci are used in forensic DNA profiling and the effect of the DNA sequencing?" Tags="|ngs|phylogenetics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13539" PostTypeId="5" CreationDate="2020-06-13T10:07:33.220" Score="0" Body="&lt;p&gt;Seurat is an R package designed for QC, analysis, and exploration of single-cell RNA-seq data. Seurat aims to enable users to identify and interpret sources of heterogeneity from single-cell transcriptomic measurements, and to integrate diverse types of single-cell data.&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2020-06-13T19:03:13.043" LastActivityDate="2020-06-13T19:03:13.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13540" PostTypeId="4" CreationDate="2020-06-13T10:07:33.220" Score="0" Body="Seurat is a package for analyzing single cell rna seq data." OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2020-06-13T19:04:18.320" LastActivityDate="2020-06-13T19:04:18.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13541" PostTypeId="5" CreationDate="2020-06-13T10:09:35.500" Score="0" Body="&lt;p&gt;kallisto is a program for quantifying abundances of transcripts from bulk and single-cell RNA-Seq data, or more generally of target sequences using high-throughput sequencing reads.&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2020-06-13T19:02:58.933" LastActivityDate="2020-06-13T19:02:58.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13542" PostTypeId="4" CreationDate="2020-06-13T10:09:35.500" Score="0" Body="Use this tag to refer to the questions related to kallisto" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2020-06-13T19:02:50.477" LastActivityDate="2020-06-13T19:02:50.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13543" PostTypeId="5" CreationDate="2020-06-13T10:17:22.380" Score="0" Body="&lt;p&gt;Snakemake is a workflow engine that provides a readable Python-based workflow definition language and a powerful execution environment that scales from single-core workstations to compute clusters without modifying the workflow. It is the first system to support the use of automatically inferred multiple named wildcards (or variables) in input and output filenames.&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2020-06-13T19:02:40.403" LastActivityDate="2020-06-13T19:02:40.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13544" PostTypeId="4" CreationDate="2020-06-13T10:17:22.380" Score="0" Body="Use this tag to refer to the questions related to snakemake" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2020-06-13T19:03:01.707" LastActivityDate="2020-06-13T19:03:01.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13545" PostTypeId="5" CreationDate="2020-06-13T10:20:04.467" Score="0" Body="&lt;p&gt;The National Center for Biotechnology Information advances science and health by providing access to biomedical and genomic information.&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2020-06-13T19:02:46.993" LastActivityDate="2020-06-13T19:02:46.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13546" PostTypeId="4" CreationDate="2020-06-13T10:20:04.467" Score="0" Body="Use this tag to refer to the questions related to NCBI" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2020-06-13T19:02:17.383" LastActivityDate="2020-06-13T19:02:17.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13547" PostTypeId="2" ParentId="13533" CreationDate="2020-06-13T16:02:11.397" Score="1" Body="&lt;p&gt;You obviously have to normalize for differences in depth followed by a proper statistical test to get fold changes and significances. Sequencing depth is a technical confounder that has to be removed, and that is what normalization does. You have been posting a lot of very basic RNA-seq questions, I suggest you read the RNA-seq Hitchhikers guide: &lt;a href=&quot;https://peerj.com/preprints/27283/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://peerj.com/preprints/27283/&lt;/a&gt; before posting new questions first.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2020-06-13T16:13:28.230" LastActivityDate="2020-06-13T16:13:28.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13548" PostTypeId="1" CreationDate="2020-06-13T16:13:11.390" Score="1" ViewCount="95" Body="&lt;p&gt;I am trying to use a free .py script from this site &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;  http://www.di-tector.cyame.eu/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;while unfortunately there's neither manual nor any documentation presented.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I have downloaded the .py script however I have absolutely no idea how it can be started. Can anyone give me an idea of how that can be done?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The corresponding article (&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/30012569/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pubmed.ncbi.nlm.nih.gov/30012569/&lt;/a&gt;) only displays parameters for executing it and there is no way to understand what steps must be performed firstly. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="6948" LastActivityDate="2020-06-13T20:26:24.890" Title="How to use the DI-tector software for defective genomes analysis?" Tags="|python|genome|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13549" PostTypeId="1" CreationDate="2020-06-13T19:23:36.007" Score="1" ViewCount="614" Body="&lt;p&gt;When we have three groups samples (A,B,C) with 25000 genes and the main interest is A vs B, should we limit samples only A and B for normalization to perform DEG analysis? Or better to include all samples (A,B, and C) for normalization and specify the comparison group within three groups?&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;res &amp;lt;- results(dds, contrast=c(&quot;tissu_type&quot;,&quot;A&quot;,&quot;B&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8895" LastActivityDate="2020-11-13T00:52:49.137" Title="about two group comparison of three group data in DESeq2 package" Tags="|r|rna-seq|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13550" PostTypeId="2" ParentId="13548" CreationDate="2020-06-13T20:26:24.890" Score="1" Body="&lt;p&gt;See the help:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;python DI-tector_06.py -h&#xA;usage: DI-tector_06.py [-h] [-g HOST_REF] [-s MIN_SEGMENT] [-m MIN_MAPQ]&#xA;                       [-n NB_READS] [-o OUTPUT_DIRECTORY] [-t TAG] [-d]&#xA;                       [-l INDEL_LENGTH] [-f] [-p POLARITY] [-q] [-k]&#xA;                       [-x NB_THREADS]&#xA;                       Virus_Ref Input_Data&#xA;&#xA;positional arguments:&#xA;  Virus_Ref             Virus genome reference sequence in FASTA format.&#xA;  Input_Data            File containing single reads in FASTQ format.&#xA;&#xA;optional arguments:&#xA;  -h, --help            show this help message and exit&#xA;  -g HOST_REF, --Host_Ref HOST_REF&#xA;                        Host genome reference sequence in FASTA format.&#xA;  -s MIN_SEGMENT, --Min_Segment MIN_SEGMENT&#xA;                        Minimum segment length. Default is 15.&#xA;  -m MIN_MAPQ, --Min_MAPQ MIN_MAPQ&#xA;                        Skip alignments with MAPQ smaller than INT. Default is&#xA;                        25.&#xA;  -n NB_READS, --Nb_Reads NB_READS&#xA;                        Show only DVGs with counts reads &amp;gt; or egal to INT.&#xA;                        Default is 2.))&#xA;  -o OUTPUT_DIRECTORY, --Output_Directory OUTPUT_DIRECTORY&#xA;                        Enter a directory name that all compiled output files&#xA;                        will be saved in.&#xA;  -t TAG, --Tag TAG     Enter a tag name that will be appended before each&#xA;                        output file. Default is 'DI-tector'.&#xA;  -d, --DVG_sequences   Generate multi-fasta file with DVG sequences. Default&#xA;                        is (OFF).&#xA;  -l INDEL_LENGTH, --InDel_Length INDEL_LENGTH&#xA;                        Skip alignments with size of InDels smaller or egal to&#xA;                        INT. Default size is 1.&#xA;  -f, --Fasta           Select '-f' if data is in FASTA format fasta. Default&#xA;                        is FASTQ.&#xA;  -p POLARITY, --Polarity POLARITY&#xA;                        [0] Positive strand genome / [1] Negative strand&#xA;                        genome. Default is 0.&#xA;  -q, --No_Quantification&#xA;                        Inactive percentage quantification. Quantification&#xA;                        need bedtools Default is (ON).&#xA;  -k, --Keep_files      Keep intermediaite files (i.e. alignment etc...).&#xA;                        Default is (OFF).&#xA;  -x NB_THREADS, --Nb_threads NB_THREADS&#xA;                        Number of threads. Default is 1.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-06-13T20:26:24.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13552" PostTypeId="2" ParentId="11574" CreationDate="2020-06-14T13:39:26.093" Score="1" Body="&lt;p&gt;You can do this very easily with &lt;code&gt;awk&lt;/code&gt;:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;$ awk '/^ {5}\w/{a=1} /\/translation/{a=0}a' file.gb &#xA;     gene            complement(8972..9094)&#xA;                     /locus_tag=&quot;HAPS_0004&quot;&#xA;                     /db_xref=&quot;GeneID:7278619&quot;&#xA;     CDS             complement(8972..9094)&#xA;                     /locus_tag=&quot;HAPS_0004&quot;&#xA;                     /codon_start=1&#xA;                     /transl_table=11&#xA;                     /product=&quot;hypothetical protein&quot;&#xA;                     /protein_id=&quot;YP_002474657.1&quot;&#xA;                     /db_xref=&quot;GI:219870282&quot;&#xA;                     /db_xref=&quot;GeneID:7278619&quot;&#xA;     gene            9632..11416&#xA;                     /gene=&quot;frdA&quot;&#xA;                     /locus_tag=&quot;HAPS_0005&quot;&#xA;                     /db_xref=&quot;GeneID:7278620&quot;&#xA;     CDS             9632..11416&#xA;                     /gene=&quot;frdA&quot;&#xA;                     /locus_tag=&quot;HAPS_0005&quot;&#xA;                     /note=&quot;part of four member fumarate reductase enzyme&#xA;                     complex FrdABCD which catalyzes the reduction of fumarate&#xA;                     to succinate during anaerobic respiration; FrdAB are the&#xA;                     catalytic subcomplex consisting of a flavoprotein subunit&#xA;                     and an iron-sulfur subunit, respectively; FrdCD are the&#xA;                     membrane components which interact with quinone and are&#xA;                     involved in electron transfer; the catalytic subunits are&#xA;                     similar to succinate dehydrogenase SdhAB&quot;&#xA;                     /codon_start=1&#xA;                     /transl_table=11&#xA;                     /product=&quot;fumarate reductase flavoprotein subunit&quot;&#xA;                     /protein_id=&quot;YP_002474658.1&quot;&#xA;                     /db_xref=&quot;GI:219870283&quot;&#xA;                     /db_xref=&quot;GeneID:7278620&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The idea here is to set &lt;code&gt;a&lt;/code&gt; to &lt;code&gt;1&lt;/code&gt; if this line starts with 5 spaces followed by a word character. Then, we set &lt;code&gt;a&lt;/code&gt; back to &lt;code&gt;0&lt;/code&gt; if this line matches &lt;code&gt;/translation&lt;/code&gt;. The default action for &lt;code&gt;awk&lt;/code&gt; when an expression evaluates to true (not 0) is to print, therefore the final &lt;code&gt;a&lt;/code&gt; will cause all lines read while &lt;code&gt;a&lt;/code&gt; is not 0 to be printed, effectively removing everything after each &lt;code&gt;/translation&lt;/code&gt; line.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;A more easily understandable version of the same code would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;awk '{ &#xA;        if(/^ {5}\w/){&#xA;            a=1&#xA;        }&#xA;        if(/\/translation/){&#xA;            a=0&#xA;        }&#xA;        if(a==1){&#xA;            print&#xA;        }&#xA;    }' file.gb &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2020-06-14T13:58:48.740" LastActivityDate="2020-06-14T13:58:48.740" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13553" PostTypeId="1" AcceptedAnswerId="13554" CreationDate="2020-06-14T17:55:48.180" Score="0" ViewCount="43" Body="&lt;p&gt;I'm working on rather standard whole-exome sequencing data and treat it the same as whole-genome sequencing data (aligning it to the full GRCh38 reference assembly and calling variants with no exome-related parameters).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I recently noticed that my variant calling results contain numerous variants mapped to a non-exomic region, incidentally, some of them were significantly associated with the phenotype I study.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;My questions are as follows:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Are the non-exome variants in my exome data invalid? or could it be that the exome kit captured more than it was designed to capture?&lt;/li&gt;&#xA;&lt;li&gt;Should I treat raw exome sequencing data any different than genome sequencing data, and if so, what parts of my pipeline should be modified to accommodate exome data?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9016" LastActivityDate="2022-12-30T11:39:06.643" Title="Non-exome variants called from whole-exome sequencing data" Tags="|sequence-alignment|variant-calling|exome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13554" PostTypeId="2" ParentId="13553" CreationDate="2020-06-14T19:26:14.053" Score="1" Body="&lt;p&gt;These are called off-targets due to unspecific binding of DNA to the capturing beads during library preparation. This is normal and expected since capturing is not a perfect process. It is correct to align against the whole genome but you should simply limit variant calling to exons (or basically the on-target regions) and ignore everything else.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-06-14T19:26:14.053" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13555" PostTypeId="1" CreationDate="2020-06-15T03:02:03.547" Score="2" ViewCount="42" Body="&lt;p&gt;I am having trouble grabbing specific pathway info using databases in R. I have RNAseq results and I want to remove immune related genes from the current list that I have.&lt;/p&gt;&#xA;&lt;p&gt;With a vector of gene names/symbols from RNAseq , I queried org.hs.eg.db to grab entrezIDs keys etc.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mapIds(org.Hs.eg.db,keys=mygenes,keytype =&amp;quot;ENTREZID&amp;quot;,column=&amp;quot;PATHID&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I thought about trying to use pathids from this database which I think are &lt;a href=&quot;https://www.genome.jp/kegg/&quot; rel=&quot;nofollow noreferrer&quot;&gt;KEGG&lt;/a&gt; pathway ids which I can subsequently filter out the appropriate  ones but my list goes from 19,000 to 4,000 before I drop immune related genes due to a lot of NAs. I tried reactomedb, but the pathnames don't really let me filter by immune related or not.&lt;/p&gt;&#xA;&lt;p&gt;Any advice is appreciated.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;mapIds Bioconductor &lt;a href=&quot;https://bioconductor.org/packages/devel/bioc/vignettes/AnnotationDbi/inst/doc/IntroToAnnotationPackages.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/devel/bioc/vignettes/AnnotationDbi/inst/doc/IntroToAnnotationPackages.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;KEGG &lt;a href=&quot;https://www.genome.jp/kegg/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.genome.jp/kegg/&lt;/a&gt;&#xA;&amp;quot;KEGG is a database resource for understanding high-level functions and utilities of the biological system, such as the cell, the organism and the ecosystem, from molecular-level information, especially large-scale molecular datasets generated by genome sequencing and other high-throughput experimental technologies.&#xA;See Release notes (April 1, 2022) for new and updated features.&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="9020" LastEditorUserId="3848" LastEditDate="2022-04-11T11:26:23.190" LastActivityDate="2022-04-11T11:26:23.190" Title="Grabbing all Immune related genes with databases in R" Tags="|rna-seq|bioconductor|database|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13556" PostTypeId="1" AcceptedAnswerId="13557" CreationDate="2020-06-15T06:56:02.903" Score="0" ViewCount="76" Body="&lt;p&gt;Im trying to find the out-group in the alignment my data files &lt;a href=&quot;https://drive.google.com/file/d/1N60Va-smHbsmCePOnkVL_jjAaxCagZWf/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;file a&lt;/a&gt; and  &lt;a href=&quot;https://drive.google.com/file/d/1_E0Ozgv_7Wtxr9AKPrvW606IWH99hIPb/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;file b&lt;/a&gt; both the case i'm getting NA when i run this     &lt;code&gt;pars.a$tip.label[pars.a$edge[which.max(pars.a$edge.length),2]]&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I tried NJ instead of upgma i do still get NA not sure what is the issue ,but at the same time when i used other alignment file &lt;a href=&quot;https://drive.google.com/file/d/10P6qF3nrXlnm-tUONK5bTAPqR4wapX-q/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;with 50 sequence &lt;/a&gt; works fine.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;a&amp;lt;-read.alignment(&quot;abhi_seq/final_abhi_tempa1.fas&quot;, format=&quot;fasta&quot;)&#xA;a.phydat&amp;lt;-as.phyDat(a)&#xA;dist.a.phydat&amp;lt;-dist.dna(as.DNAbin(a.phydat))&#xA;upgma.a&amp;lt;-upgma(dist.a.phydat)&#xA;parsimony(upgma.a,a.phydat)&#xA;pars.a &amp;lt;- optim.parsimony(upgma.a, a.phydat)&#xA;pars.a&amp;lt;-acctran(pars.a, a.phydat)&#xA;pars.a&lt;span class=&quot;math-container&quot;&gt;$tip.label[pars.a$&lt;/span&gt;edge[which.max(pars.a$edge.length),2]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="285" LastActivityDate="2020-06-15T08:22:06.550" Title="Finding out-group in unrooted tree" Tags="|r|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13557" PostTypeId="2" ParentId="13556" CreationDate="2020-06-15T08:22:06.550" Score="1" Body="&lt;p&gt;the fact is that the present upgma tree as a midpoint rooting here, corresponding to node 46.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;getRoot(upgma.a)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;so for this kind of tree, I recommend you to use the midpoint() rooting function.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;pars.a &amp;lt;- optim.parsimony(upgma.a, a.phydat)&#xA;pars.a&amp;lt;-acctran(pars.a, a.phydat)&#xA;plot(midpoint(pars.a))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8749" LastActivityDate="2020-06-15T08:22:06.550" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="13558" PostTypeId="2" ParentId="13549" CreationDate="2020-06-15T09:49:05.293" Score="1" Body="&lt;p&gt;As explained on the comments, having some other samples might distort your estimations, add an outlier or something alike.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The advantage of having all the three (or all) groups of samples before testing differences, because usually methods like limma, DESeq2 and others uses the information of variation of the genes across all samples to update their prior probabilities and the variation of the genes on the statistical model used to compare between groups. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;If the &lt;strong&gt;method&lt;/strong&gt; and your &lt;strong&gt;model&lt;/strong&gt; are robust use all the three groups (generally the methods used are robust, but sometimes the models are not). By robust I mean that it doesn't miss any know (and available) source of variation. Otherwise the comparison might be comparing only the samples not the effect of the condition on the samples. If you only compare the samples it would be equivalent to use a t.test to the limma methods (not completely sure on the DESeq2 case). Or explained differently, you wouldn't take advantage of the statistical improvements implemented on these methods&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Usually the reasons why these groups are sequenced on the same batch is because they are related (or to add some known ground truth), so you can release it with the data when communicating your results but you may omit that you used this third group on the methods section.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2020-06-15T09:49:05.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13560" PostTypeId="2" ParentId="13529" CreationDate="2020-06-15T13:41:12.093" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;Open the PyMOL session.&lt;/li&gt;&#xA;&lt;li&gt;Use the &lt;code&gt;load&lt;/code&gt; command to load your PDB file(s): &lt;code&gt;load file.pdb&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Use the &lt;code&gt;align&lt;/code&gt; or &lt;code&gt;super&lt;/code&gt; commands to overlay structures, e.g. &lt;code&gt;align prot1, prot2&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2020-06-15T13:41:12.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13561" PostTypeId="1" CreationDate="2020-06-15T18:19:33.860" Score="0" ViewCount="205" Body="&lt;p&gt;I have several sets of chip-seq data. I called the peaks using Macs2. I am pretty new to the field and I will appreciate any help. I wanted to annotate the peaks and see which peaks are shared between them and which ones are different. What is the best way to do this? which package should I use?&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Many thanks&lt;/p&gt;&#xA;" OwnerUserId="8543" LastActivityDate="2021-04-19T02:47:46.460" Title="differential analysis of chip-seq data" Tags="|sequence-annotation|sequence-analysis|chip-seq|peak-calling|macs2|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13562" PostTypeId="1" CreationDate="2020-06-15T21:07:05.310" Score="-1" ViewCount="603" Body="&lt;p&gt;I have a table containing different metadata information about a set of samples as an Array[Object] and I need to extract a couple of columns from this table and output those two columns into a file that will be read and used by the task. I wanted to know whether I can do this within a task rather than within a workflow, that is, without using the scatter construct which is not allowed inside a task.&lt;/p&gt;&#xA;&lt;p&gt;If I zip two arrays I obtain an Array[Pair[String,String]] variable.&lt;/p&gt;&#xA;&lt;p&gt;Now if I want to output to file an Array[Array[String]] I can use write_tsv(), if I want to output to file a Map[String,String] I can use write_map().&lt;/p&gt;&#xA;&lt;p&gt;But WDL does not let me coerce Array[Pair[String,String]] to either Array[Array[String]] nor Map[String,String] (though I can coerce Map[String,String] to Array[Pair[String,String]]), so neither option is available.&lt;/p&gt;&#xA;&lt;p&gt;Is there a solution to this that does not involve the scatter construct?&lt;/p&gt;&#xA;" OwnerUserId="9030" LastEditorUserId="73" LastEditDate="2021-06-07T02:29:44.307" LastActivityDate="2021-06-07T02:29:44.307" Title="output Array[Pair[X,Y]] to file inside a WDL task" Tags="|wdl|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13563" PostTypeId="1" CreationDate="2020-06-16T11:59:20.437" Score="0" ViewCount="559" Body="&lt;p&gt;Suppose I have two arrays Array[File] X and Array[String] Y both of length N, with Y being a sort of R-like factor that defines batch membership for X, for a total of M batches with M &amp;lt; N. A valid example would be:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Array[File] X = [&quot;file1&quot;, &quot;file2&quot;, &quot;file3&quot;, &quot;file4&quot;, &quot;file5&quot;]&#xA;Array[String] Y  =  [&quot;A&quot;,     &quot;A&quot;,     &quot;B&quot;,     &quot;A&quot;,     &quot;B&quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;In which case N=5 and M=2. I want to scatter M tasks so that each task takes only the files corresponding to the same batch. In the previous example, one task would take as input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Array[File] X =[&quot;file1&quot;, &quot;file2&quot;, &quot;file4&quot;]&#xA;String Y = &quot;A&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And the other task would take as input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Array[File] X =[&quot;file3&quot;, &quot;file5&quot;]&#xA;String Y = &quot;B&quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Is there an easy way to do this within a WDL workflow in the general case? I understand that there are less elegant ways to do this as its own task but, as this seems like it could be a general purpose scatter/gather problem, an easy solution might be of general interest to the community.&lt;/p&gt;&#xA;" OwnerUserId="9030" LastActivityDate="2020-06-17T16:49:03.970" Title="Scatter tasks around predefined batches inside a WDL workflow" Tags="|wdl|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13564" PostTypeId="1" CreationDate="2020-06-16T13:15:42.963" Score="0" ViewCount="51" Body="&lt;p&gt;I am looking for examples of (known) algorithms/data structures in biological systems. My current understanding is that these don't exist - or haven't been found - beyond a relatively low level of complexity, which would seem like strong evidence to me that evolution isn't able to come up with something as complicated (and multi-step) as, e.g., the Schönhage-Strassen algorithm.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I heard of Bloom filters for recognizing smells in fruit flies (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6304992/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6304992/&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="9034" LastActivityDate="2020-06-16T18:40:13.707" Title="Sophisticated algorithms in biological systems" Tags="|microbiology|systems-biology|" AnswerCount="1" CommentCount="1" ClosedDate="2020-06-16T20:49:16.637" ContentLicense="CC BY-SA 4.0" />
  <row Id="13565" PostTypeId="1" CreationDate="2020-06-16T14:35:27.687" Score="2" ViewCount="394" Body="&lt;p&gt;I am working with a RNA-seq data set in maize that has a relatively complex design. There are two levels of treatment A (nitrogen fertilizer level in the field, high or low), two levels of treatment B (nitrogen nutrients in &lt;em&gt;in vitro&lt;/em&gt; cultures, high and low) and two levels of treatment C (two time points of sampling), all with 3 reps. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; library(edgeR)&#xA;&amp;gt; load(&quot;KC_Raw.RData&quot;)&#xA;&amp;gt; y &amp;lt;- DGEList(counts = KCraw.data[,2:25])&#xA;&amp;gt; keep &amp;lt;- rowSums(cpm(y) &amp;gt; 10) &amp;gt;= 3&#xA;&amp;gt; targets &amp;lt;- data.frame(rownames=colnames(KCraw.data)[2:25] ,&#xA;+                       Time=rep(c(rep(&quot;2DIC&quot;,12),rep(&quot;5DIC&quot;,12))) ,&#xA;+                       FieldN=rep(c(rep(&quot;FH&quot;,6), rep(&quot;FL&quot;,6)),2) ,&#xA;+                       CultureN=rep(c(rep(&quot;CL&quot;,3),rep(&quot;CH&quot;,3)),4))&#xA;&amp;gt; Group &amp;lt;- factor(paste(targets$FieldN,targets$Time,targets$CultureN,sep=&quot;.&quot;))&#xA;&amp;gt; targets &amp;lt;- cbind(targets,Group=Group)&#xA;&amp;gt; targets&#xA;   rownames Time FieldN CultureN      Group&#xA;1   KC1_H2L 2DIC     FH       CL FH.2DIC.CL&#xA;2   KC2_H2L 2DIC     FH       CL FH.2DIC.CL&#xA;3   KC3_H2L 2DIC     FH       CL FH.2DIC.CL&#xA;4   KC4_H2H 2DIC     FH       CH FH.2DIC.CH&#xA;5   KC5_H2H 2DIC     FH       CH FH.2DIC.CH&#xA;6   KC6_H2H 2DIC     FH       CH FH.2DIC.CH&#xA;7   KC7_L2L 2DIC     FL       CL FL.2DIC.CL&#xA;8   KC8_L2L 2DIC     FL       CL FL.2DIC.CL&#xA;9   KC9_L2L 2DIC     FL       CL FL.2DIC.CL&#xA;10 KC10_L2H 2DIC     FL       CH FL.2DIC.CH&#xA;11 KC11_L2H 2DIC     FL       CH FL.2DIC.CH&#xA;12 KC12_L2H 2DIC     FL       CH FL.2DIC.CH&#xA;13 KC13_H5L 5DIC     FH       CL FH.5DIC.CL&#xA;14 KC14_H5L 5DIC     FH       CL FH.5DIC.CL&#xA;15 KC15_H5L 5DIC     FH       CL FH.5DIC.CL&#xA;16 KC16_H5H 5DIC     FH       CH FH.5DIC.CH&#xA;17 KC17_H5H 5DIC     FH       CH FH.5DIC.CH&#xA;18 KC18_H5H 5DIC     FH       CH FH.5DIC.CH&#xA;19 KC19_L5L 5DIC     FL       CL FL.5DIC.CL&#xA;20 KC20_L5L 5DIC     FL       CL FL.5DIC.CL&#xA;21 KC21_L5L 5DIC     FL       CL FL.5DIC.CL&#xA;22 KC22_L5H 5DIC     FL       CH FL.5DIC.CH&#xA;23 KC23_L5H 5DIC     FL       CH FL.5DIC.CH&#xA;24 KC24_L5H 5DIC     FL       CH FL.5DIC.CH&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have used edgeR in R to calculate differential expression for contrasts involving 3 reps at one treatment combination to 3 reps at another treatment combination, for example&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; y &amp;lt;- DGEList(counts = KCraw.data[keep,2:25], group = Group)&#xA;&amp;gt; y &amp;lt;- calcNormFactors(y)&#xA;&amp;gt; &#xA;&amp;gt; TMM &amp;lt;- KCraw.data[keep,2:25]&#xA;&amp;gt; for (i in 1:24) {&#xA;+   TMM[,i] &amp;lt;- TMM[,i] / (y&lt;span class=&quot;math-container&quot;&gt;$samples$&lt;/span&gt;lib.size[i] * y&lt;span class=&quot;math-container&quot;&gt;$samples$&lt;/span&gt;norm.factors[i]) * 1e6&#xA;+ }&#xA;&amp;gt; &#xA;&amp;gt; y &amp;lt;- DGEList(counts = TMM,group = Group)&#xA;&amp;gt; &#xA;&amp;gt; design &amp;lt;- model.matrix(~0+Group)&#xA;&amp;gt; colnames(design) &amp;lt;- levels(Group)&#xA;&amp;gt; y &amp;lt;- calcNormFactors(y,method = &quot;TMM&quot;)&#xA;&amp;gt; y &amp;lt;- estimateDisp(y,design)&#xA;&amp;gt; fitQL &amp;lt;- glmQLFit(y,design)&#xA;&amp;gt; fit &amp;lt;- glmFit(y,design)&#xA;&amp;gt; myKC.contrasts &amp;lt;- makeContrasts(&#xA;+   H2H.H2L = FH.2DIC.CH - FH.2DIC.CL,&#xA;+   L2H.L2L = FL.2DIC.CH - FL.2DIC.CL,&#xA;+   H2H.L2H = FH.2DIC.CH - FL.2DIC.CH,&#xA;+   H2L.L2L = FH.2DIC.CL - FL.2DIC.CL,&#xA;+   H5H.H5L = FH.5DIC.CH - FH.5DIC.CL,&#xA;+   L5H.L5L = FL.5DIC.CH - FL.5DIC.CL,&#xA;+   H5H.L5H = FH.5DIC.CH - FL.5DIC.CH,&#xA;+   H5L.L5L = FH.5DIC.CL - FL.5DIC.CL,&#xA;+   H2H.L2L = FH.2DIC.CH - FL.2DIC.CL,&#xA;+   H5H.L5L = FH.5DIC.CH - FL.5DIC.CL,&#xA;+   H5L.H2L = FH.5DIC.CL - FH.2DIC.CL, &#xA;+   H5H.H2H = FH.5DIC.CH - FH.2DIC.CH,&#xA;+   L5L.L2L = FL.5DIC.CL - FL.2DIC.CL,&#xA;+   L5H.L2H = FL.5DIC.CH - FL.2DIC.CH,&#xA;+   levels=design)&#xA;&amp;gt; design&#xA;   FH.2DIC.CH FH.2DIC.CL FH.5DIC.CH FH.5DIC.CL FL.2DIC.CH FL.2DIC.CL FL.5DIC.CH FL.5DIC.CL&#xA;1           0          1          0          0          0          0          0          0&#xA;2           0          1          0          0          0          0          0          0&#xA;3           0          1          0          0          0          0          0          0&#xA;4           1          0          0          0          0          0          0          0&#xA;5           1          0          0          0          0          0          0          0&#xA;6           1          0          0          0          0          0          0          0&#xA;7           0          0          0          0          0          1          0          0&#xA;8           0          0          0          0          0          1          0          0&#xA;9           0          0          0          0          0          1          0          0&#xA;10          0          0          0          0          1          0          0          0&#xA;11          0          0          0          0          1          0          0          0&#xA;12          0          0          0          0          1          0          0          0&#xA;13          0          0          0          1          0          0          0          0&#xA;14          0          0          0          1          0          0          0          0&#xA;15          0          0          0          1          0          0          0          0&#xA;16          0          0          1          0          0          0          0          0&#xA;17          0          0          1          0          0          0          0          0&#xA;18          0          0          1          0          0          0          0          0&#xA;19          0          0          0          0          0          0          0          1&#xA;20          0          0          0          0          0          0          0          1&#xA;21          0          0          0          0          0          0          0          1&#xA;22          0          0          0          0          0          0          1          0&#xA;23          0          0          0          0          0          0          1          0&#xA;24          0          0          0          0          0          0          1          0&#xA;attr(,&quot;assign&quot;)&#xA;[1] 1 1 1 1 1 1 1 1&#xA;attr(,&quot;contrasts&quot;)&#xA;attr(,&quot;contrasts&quot;)$Group&#xA;[1] &quot;contr.treatment&quot;&#xA;&#xA;&amp;gt; myKC.contrasts&#xA;            Contrasts&#xA;Levels       H2H.H2L L2H.L2L H2H.L2H H2L.L2L H5H.H5L L5H.L5L H5H.L5H H5L.L5L H2H.L2L H5H.L5L H5L.H2L H5H.H2H L5L.L2L&#xA;  FH.2DIC.CH       1       0       1       0       0       0       0       0       1       0       0      -1       0&#xA;  FH.2DIC.CL      -1       0       0       1       0       0       0       0       0       0      -1       0       0&#xA;  FH.5DIC.CH       0       0       0       0       1       0       1       0       0       1       0       1       0&#xA;  FH.5DIC.CL       0       0       0       0      -1       0       0       1       0       0       1       0       0&#xA;  FL.2DIC.CH       0       1      -1       0       0       0       0       0       0       0       0       0       0&#xA;  FL.2DIC.CL       0      -1       0      -1       0       0       0       0      -1       0       0       0      -1&#xA;  FL.5DIC.CH       0       0       0       0       0       1      -1       0       0       0       0       0       0&#xA;  FL.5DIC.CL       0       0       0       0       0      -1       0      -1       0      -1       0       0       1&#xA;            Contrasts&#xA;Levels       L5H.L2H&#xA;  FH.2DIC.CH       0&#xA;  FH.2DIC.CL       0&#xA;  FH.5DIC.CH       0&#xA;  FH.5DIC.CL       0&#xA;  FL.2DIC.CH      -1&#xA;  FL.2DIC.CL       0&#xA;  FL.5DIC.CH       1&#xA;  FL.5DIC.CL       0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;After analyzing these contrasts, I wanted to estimate some sort of simple effect, such as the culture media nitrogen level. To do this, I ran the following code.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; myKC.contrasts &amp;lt;- cbind(myKC.contrasts,&#xA;+                         Development = c(1,1,-1,-1,1,1,-1,-1),&#xA;+                         FieldN = c(1,1,1,1,-1,-1,-1,-1),&#xA;+                         CultureN = c(1,-1,1,-1,1,-1,1,-1)&#xA;+ )&#xA;&amp;gt; myKC.contrasts&#xA;           H2H.H2L L2H.L2L H2H.L2H H2L.L2L H5H.H5L L5H.L5L H5H.L5H H5L.L5L H2H.L2L H5H.L5L H5L.H2L H5H.H2H L5L.L2L&#xA;FH.2DIC.CH       1       0       1       0       0       0       0       0       1       0       0      -1       0&#xA;FH.2DIC.CL      -1       0       0       1       0       0       0       0       0       0      -1       0       0&#xA;FH.5DIC.CH       0       0       0       0       1       0       1       0       0       1       0       1       0&#xA;FH.5DIC.CL       0       0       0       0      -1       0       0       1       0       0       1       0       0&#xA;FL.2DIC.CH       0       1      -1       0       0       0       0       0       0       0       0       0       0&#xA;FL.2DIC.CL       0      -1       0      -1       0       0       0       0      -1       0       0       0      -1&#xA;FL.5DIC.CH       0       0       0       0       0       1      -1       0       0       0       0       0       0&#xA;FL.5DIC.CL       0       0       0       0       0      -1       0      -1       0      -1       0       0       1&#xA;           L5H.L2H Development FieldN CultureN&#xA;FH.2DIC.CH       0           1      1        1&#xA;FH.2DIC.CL       0           1      1       -1&#xA;FH.5DIC.CH       0          -1      1        1&#xA;FH.5DIC.CL       0          -1      1       -1&#xA;FL.2DIC.CH      -1           1     -1        1&#xA;FL.2DIC.CL       0           1     -1       -1&#xA;FL.5DIC.CH       1          -1     -1        1&#xA;FL.5DIC.CL       0          -1     -1       -1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;Once I rerun the analysis for the CultureN contrast and look at the result for a particular gene, I see that it's estimated log2FC is equal to the sum of every simple contrast. &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; lrt &amp;lt;- glmQLFTest(fitQL, contrast=myKC.contrasts[,&quot;CultureN&quot;])&#xA;&amp;gt; topTags(lrt,n=nrow(y$counts))[&quot;GRMZM2G445575&quot;,]&#xA;Coefficient:  1*FH.2DIC.CH -1*FH.2DIC.CL 1*FH.5DIC.CH -1*FH.5DIC.CL 1*FL.2DIC.CH -1*FL.2DIC.CL 1*FL.5DIC.CH -1*FL.5DIC.CL &#xA;                 logFC   logCPM        F       PValue          FDR&#xA;GRMZM2G445575 -6.63617 5.417106 151.5261 3.691525e-11 2.825777e-08&#xA;# FC is a data frame of the logFC of each constrast in columns for each gene in rows&#xA;&amp;gt; sum(FC[&quot;GRMZM2G445575&quot;,c(&quot;H2H.H2L&quot;,&quot;L2H.L2L&quot;,&quot;H5H.H5L&quot;,&quot;L5H.L5L&quot;)])&#xA;[1] -6.636197&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My first question is if this analysis is a valid way of summarizing the simple effects of each treatment.&#xA;I would like to be able to also include the effects of the H2H.L2L and H5H.L5L contrast in the FieldN and CultureN comparison, but I am not sure how to do this, or if this would be valid because each of these contrasts includes treatments that have different levels of two treatment factors. &lt;/p&gt;&#xA;" OwnerUserId="9036" LastActivityDate="2024-02-27T07:01:40.230" Title="Comparing multiple treatments to multiple other treatments in edgeR for simple effects in a complex experimental design" Tags="|rna-seq|differential-expression|edger|design|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13566" PostTypeId="2" ParentId="13481" CreationDate="2020-06-16T17:48:08.403" Score="1" Body="&lt;p&gt;I don't use R to build trees, I use established subprocessing stripts in Python and ETE3. So I've not run this code, but I do understand the critical parts of the code.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The hard bit is bootstrapping a maximum likelihood tree, first is the ML model&#xA;1.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;fitGTR &amp;lt;-optim.pml(fitGTR, model=&quot;GTR&quot;, optInv=TRUE, optGamma=TRUE, + rearrangement = &quot;NNI&quot;, control = pml.control(trace = 0))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The above model is perfect, general time reversible model with a gamma correction and an estimate of the invariant sites. This is the most important part of tree estimation. The NNI for tree rearrangements is a good one. So THIS is the model you want. You can emperical test this, but these days we don't bother.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The information given for bootstrapping,&lt;/p&gt;&#xA;&#xA;&lt;p&gt;2.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;BStree &amp;lt;- bootstrap.pml(x, bs = 500, trees = TRUE, multicore = FALSE,&#xA;  mc.cores = NULL, ...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The critical parameter is bs=500. By default it uses bs=100 which I don't think is enough. I would set multicore = TRUE to speed the calculation (it will speed it up alot) and I assume there must be a setting for the number of cores (?).&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I assume you can then stack all the parameters from 1/ into code 2. So in other words the model you used in part 1. (GTR PINVAR and Gammma) is same model used in the bootstrapping.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;This will produce a bootstrap file, probably of 500 bootstrapped ML trees, so you need to produce a consensus and represent the values on the optimal ML tree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Thus you need to overlay the bootstrap onto the tree that is output in 1. This says fitGTR and I suspect that will be a tree file with a likelihood, you might need to set a tree output and that would be 'tree' in the code below.&#xA;3.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;plotBS(tree, BStrees, type = &quot;myroot&quot;, bs.col = &quot;black&quot;,&#xA;  bs.adj = NULL, p = 50, frame = &quot;none&quot;, ...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;The type = myroot is your defined outgroup for the tree. This should be your nicely bootstrapped tree.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;There was also this code in the help manual and I'm not sure what this is doing. I suspect it is just producing bootstrapped datasets, so its not needed here.&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;bootstrap.phyDat(x, FUN, bs = 100, multicore = FALSE,&#xA;      mc.cores = NULL, jumble = TRUE, ...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;To do the same with parsimony then just replace pml for parsimony. Comparing 2 bootstrap trees is a node is robust when it scores &gt;80% &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-06-16T18:06:02.283" LastActivityDate="2020-06-16T18:06:02.283" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13567" PostTypeId="2" ParentId="13564" CreationDate="2020-06-16T18:40:13.707" Score="2" Body="&lt;p&gt;There are many examples of biological systems doing computation, according to my understanding:&lt;/p&gt;&#xA;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Regulatory networks encode decision trees of significant complexity: &lt;a href=&quot;https://en.wikipedia.org/wiki/Gene_regulatory_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Alan Turing did significant work on organismal morphogenesis showing how simple chemical patterns of inducers can lead to complex outcomes, &quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/The_Chemical_Basis_of_Morphogenesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;Turing patterns&lt;/a&gt;&quot;.&lt;/li&gt;&#xA;&lt;li&gt;All of organismal development, cell division, etc. are tightly regulated procedures of gene regulation and morphogenesis.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Neural_network&quot; rel=&quot;nofollow noreferrer&quot;&gt;inspiration for neural networks is brains&lt;/a&gt;. &lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-06-16T18:40:13.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13568" PostTypeId="2" ParentId="13563" CreationDate="2020-06-16T20:40:59.630" Score="1" Body="&lt;p&gt;Here's an alternative to using a custom task, albeit still rather unappetizing:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;workflow groupby {&#xA;  input {&#xA;    File? nil&#xA;  }&#xA;&#xA;  Array[File] X = [&quot;file1&quot;, &quot;file2&quot;, &quot;file3&quot;, &quot;file4&quot;, &quot;file5&quot;]&#xA;  Array[String] Y  =  [&quot;A&quot;,     &quot;A&quot;,     &quot;B&quot;,     &quot;A&quot;,     &quot;B&quot;]&#xA;&#xA;  scatter (y in [&quot;A&quot;, &quot;B&quot;]) {&#xA;    scatter (p in zip(X,Y)) {&#xA;      File? x_or_nil = if (p.right == y) then p.left else nil&#xA;    }&#xA;    Pair[String,Array[File]] group = (y, select_all(x_or_nil))&#xA;  }&#xA;&#xA;  output {&#xA;    Array[Pair[String,Array[File]]] ans = group&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;(It'll be slightly cleaner with the upcoming &lt;code&gt;None&lt;/code&gt; literal instead of dummy &lt;code&gt;nil&lt;/code&gt;)&lt;/p&gt;&#xA;&#xA;&lt;p&gt;FWIW, &lt;a href=&quot;https://github.com/openwdl/wdl/issues/312&quot; rel=&quot;nofollow noreferrer&quot;&gt;here's a lengthy discussion&lt;/a&gt; about the idea of allowing developers to define custom library functions which just run a small task that manipulates data structures (hinting that it needn't necessarily offload to a worker node, with the huge latency that typically entails).&lt;/p&gt;&#xA;" OwnerUserId="8701" LastActivityDate="2020-06-16T20:40:59.630" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13569" PostTypeId="2" ParentId="13562" CreationDate="2020-06-16T20:50:21.967" Score="0" Body="&lt;p&gt;&lt;code&gt;write_json()&lt;/code&gt; is supposed to provide the general way to serialize any data structure for processing/manipulation in the task command script (perhaps leveraging &lt;a href=&quot;https://stedolan.github.io/jq/&quot; rel=&quot;nofollow noreferrer&quot;&gt;jq&lt;/a&gt;). Unfortunately, last I heard, &lt;a href=&quot;https://github.com/broadinstitute/cromwell/labels/Read%2Fwrite%20JSON%20array%20bug&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cromwell's implementation doesn't work on some useful cases&lt;/a&gt; right now. (Possibly out of date; I haven't tried lately)&lt;/p&gt;&#xA;" OwnerUserId="8701" LastActivityDate="2020-06-16T20:50:21.967" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13570" PostTypeId="2" ParentId="13565" CreationDate="2020-06-16T21:24:03.403" Score="0" Body="&lt;p&gt;I think the problem is in the design. There is no room for error or variation from a common ground.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;I'm not sure of the solution, but I think you'll need to delete a column to give the design a degree of freedom, and probably add the intercept. You'll need to adjust the contrasts accordingly. &lt;/p&gt;&#xA;&#xA;&lt;p&gt;However, I would recommend to ask this at &lt;a href=&quot;https://support.bioconductor.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;support.bioconductor.org&lt;/a&gt;. There are more experts on linear modelling and contrasts there than me. (If you ask there make it easier to copy and paste your code)&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2020-06-16T21:24:03.403" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13571" PostTypeId="2" ParentId="13561" CreationDate="2020-06-16T21:30:14.707" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;For sharing, you can test the proportion of overlapping peaks using &lt;code&gt;bedtools intersect&lt;/code&gt; or the &lt;code&gt;find.overlap&lt;/code&gt; function in &lt;code&gt;GenomicRanges&lt;/code&gt; packages in R.&lt;/li&gt;&#xA;&lt;li&gt;What makes more sense is perhaps to merge all the peaks together using &lt;code&gt;bedtools merge&lt;/code&gt;, and then count the number of reads/fragments in each peak for each sample using &lt;code&gt;featureCounts&lt;/code&gt;. The output of this is a matrix, which is very similar to gene expression results. Then you can do some differential tests on this data.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-06-16T21:30:14.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13572" PostTypeId="1" AcceptedAnswerId="13573" CreationDate="2020-06-16T22:04:43.570" Score="2" ViewCount="472" Body="&lt;p&gt;I am trying to analyze my RNAseq reads for defective genomes and I use this program (&lt;a href=&quot;http://www.di-tector.cyame.eu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.di-tector.cyame.eu/&lt;/a&gt;) that is suggested by Beauclair et al (&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/30012569/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pubmed.ncbi.nlm.nih.gov/30012569/&lt;/a&gt;)  for that.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;After downloading a .py script and running it with the following command: &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sudo python3 DI-tector_06.py /mnt/e/nastya/SLX066-02/sequence.fasta /mnt/e/nastya/SLX066-02/B-dVMV-RIG-1/B-dVMV-RIG-1_ACAGTG_L008_R1_001.rc.fastq.gz &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;It says at the beginning that &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;=================================&#xA;Program: DI-tector&#xA;Version: 0.6 - Last modified 25/05/2018&#xA;=================================&#xA;Requirement: (must be in your $PATH)&#xA;-bwa&#xA;-samtools&#xA;&#xA;Optional: (must be in your $PATH)&#xA;-bedtools&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have tried to install bwa using:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;sudo apt-get update -y&#xA;sudo apt-get install -y bwa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;and samtools using &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;cd /usr/local/bin&#xA;sudo wget https://github.com/samtools/samtools/releases/download/1.9/samtools-1.9.tar.bz2&#xA;sudo tar -vxjf samtools-1.9.tar.bz2&#xA;cd samtools-1.9&#xA;sudo make&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I did not get exactly what does that mean: &lt;strong&gt;(must be in your $PATH)&lt;/strong&gt; but I have searched here and there and discovered that it might mean &lt;code&gt;/usr/local/bin&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;The program starts and, after some calculations it gives me the following error:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Input file:               /mnt/e/nastya/SLX066-02/B-dVMV-RIG-1/B-dVMV-RIG-1_ACAGTG_L008_R1_001.rc.fastq.gz&#xA;Host reference:           None&#xA;Virus reference:          /mnt/e/nastya/SLX066-02/sequence.fasta&#xA;Remove segment:         &amp;lt; 15 nt&#xA;Remove reads with MAPQ: &amp;lt; 25&#xA;Allow InDel. length:    &amp;gt; 1 nt&#xA;&#xA;=================================&#xA;Reading gziped input file...&#xA;=================================&#xA;Number of reads in input file: 18,035,544&#xA;=================================&#xA;&#xA;=================================&#xA;Step 1/5 : Alignment against Viral reference...Filtering...&#xA;=================================&#xA;[E::bwa_idx_load_from_disk] fail to locate the index files&#xA;Traceback (most recent call last):&#xA;  File &quot;DI-tector_06.py&quot;, line 183, in &amp;lt;module&amp;gt;&#xA;    Popen([&quot;samtools&quot;, &quot;view&quot;, &quot;-Sb&quot;, &quot;-F&quot;, &quot;4&quot;], stdin=out_file_onVirus, stdout=out_file_Virus).communicate()[0]&#xA;  File &quot;/usr/lib/python3.6/subprocess.py&quot;, line 709, in __init__&#xA;    restore_signals, start_new_session)&#xA;  File &quot;/usr/lib/python3.6/subprocess.py&quot;, line 1344, in _execute_child&#xA;    raise child_exception_type(errno_num, err_msg, err_filename)&#xA;PermissionError: [Errno 13] Permission denied: 'samtools'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;I have no idea how samtools must be accessed, I tried &lt;code&gt;sudo&lt;/code&gt; command with python3 but it did not work.&lt;/p&gt;&#xA;" OwnerUserId="6948" LastActivityDate="2020-06-18T19:45:11.370" Title="How to make work programs from the $PATH?" Tags="|python|samtools|linux|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13573" PostTypeId="2" ParentId="13572" CreationDate="2020-06-17T06:35:53.990" Score="3" Body="&lt;p&gt;&lt;code&gt;echo $PATH&lt;/code&gt; will show the directories in your search path. Either copy the samtools executable there or add its location to the $PATH variable. You would be well served with a unix command line course or tutorial.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-06-17T06:35:53.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13574" PostTypeId="1" CreationDate="2020-06-17T06:48:12.853" Score="0" ViewCount="1805" Body="&lt;p&gt;I am currently working on identifying effective antiviral compounds against the corona virus. I have downloaded the Covid-19 AntiViral Candidate Compounds Dataset from the www.cas.org webpage, the link to which I have provided below. Me and a few other doctors are using Docking software to predict the docking capacity of these anti-viral compounds. Docking software requires the anti-viral compounds to be in the .pdb file format only then is it recognized by Automated docking software. &lt;strong&gt;Is there any way I can convert the .sdf file containing the anti-viral compounds to .pdb file or is there a way of obtaining the .pdb files for list of anti-viral compounds&lt;/strong&gt;.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;em&gt;Thank you for your time and patience,&#xA;Anas Hussain.&lt;/em&gt; &lt;/p&gt;&#xA;&#xA;&lt;p&gt;&lt;strong&gt;Anti-Viral Candidate Compounds Dataset&lt;/strong&gt;&#xA;&lt;a href=&quot;https://www.cas.org/covid-19-antiviral-compounds-dataset&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cas.org/covid-19-antiviral-compounds-dataset&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9040" LastActivityDate="2020-06-22T16:24:48.280" Title=".SDF file to .PDB file" Tags="|pdb|docking|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13575" PostTypeId="1" CreationDate="2020-06-17T12:30:47.137" Score="1" ViewCount="323" Body="&lt;p&gt;I have a GRanges object with metadata that I want to plot. Many of the online tutorials which manipulate these GRanges for plotting show how to plot the main columns (chromosome,start/end,strand) but nothing to plot the metadata as a function of the position in each row.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;How precisely should I do this? Ideally, I would like to plot metadata from multiple GRanges objects on the same plot to compare them.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;EDIT (following comment):&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;GRanges object with 6 ranges and 140 metadata columns:&#xA;      seqnames        ranges strand |&#xA;         &amp;lt;Rle&amp;gt;     &amp;lt;IRanges&amp;gt;  &amp;lt;Rle&amp;gt; |&#xA;  [1]     chr1 100000-199999      * |&#xA;  [2]     chr1 400000-499999      * |&#xA;  [3]     chr1 600000-699999      * |&#xA;  [4]     chr1 700000-799999      * |&#xA;  [5]     chr1 800000-899999      * |&#xA;  [6]     chr1 900000-999999      * |&#xA;                  eigen   domain         arm&#xA;              &amp;lt;numeric&amp;gt; &amp;lt;factor&amp;gt; &amp;lt;character&amp;gt;&#xA;  [1] -1.62475825405065     open          1p&#xA;  [2] -1.62475825405065     open          1p&#xA;  [3] -1.62475825405065     open          1p&#xA;  [4] -1.62475825405065     open          1p&#xA;  [5] -1.63354915136132     open          1p&#xA;  [6]  -1.6322733358426     open          1p&#xA;      exon.fraction       gc     short      long&#xA;          &amp;lt;numeric&amp;gt; &amp;lt;matrix&amp;gt; &amp;lt;numeric&amp;gt; &amp;lt;numeric&amp;gt;&#xA;  [1]       0.02968  0.45815         0         0&#xA;  [2]       0.04305  0.40192         0         0&#xA;  [3]       0.00959  0.42091         0         0&#xA;  [4]       0.04593  0.43337         0         1&#xA;  [5]       0.02425  0.44822         2         1&#xA;  [6]       0.16977  0.62123        27        56&#xA;         nfrags             ratio   short.corrected&#xA;      &amp;lt;numeric&amp;gt;         &amp;lt;numeric&amp;gt;         &amp;lt;numeric&amp;gt;&#xA;  [1]         0               NaN              &amp;lt;NA&amp;gt;&#xA;  [2]         0               NaN              &amp;lt;NA&amp;gt;&#xA;  [3]         0               NaN              &amp;lt;NA&amp;gt;&#xA;  [4]         1                 0 -3.50822623240367&#xA;  [5]         3                 2  2.81796238367926&#xA;  [6]        83 0.482142857142857  39.8824095079959&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;My question regards plotting a column like &lt;code&gt;eigen&lt;/code&gt; as a function of the position (given by &lt;code&gt;seqnames&lt;/code&gt; and &lt;code&gt;ranges&lt;/code&gt;)&lt;/p&gt;&#xA;" OwnerUserId="2244" LastEditorUserId="2244" LastEditDate="2020-06-17T14:17:16.913" LastActivityDate="2020-06-18T14:56:08.977" Title="Plotting metadata from GenomicRanges object" Tags="|r|bioconductor|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13576" PostTypeId="2" ParentId="5475" CreationDate="2020-06-17T15:24:25.543" Score="0" Body="&lt;p&gt;Without jumping deep into the rabbit hole of orthologology, you can just use &lt;a href=&quot;http://www.ensembl.org/biomart/martview/&quot; rel=&quot;nofollow noreferrer&quot;&gt;biomart&lt;/a&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;An instruction is &lt;a href=&quot;http://www.ensembl.info/2009/01/21/how-to-get-all-the-orthologous-genes-between-two-species/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; &lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2020-06-17T15:24:25.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13577" PostTypeId="1" AcceptedAnswerId="13582" CreationDate="2020-06-17T16:01:11.610" Score="2" ViewCount="259" Body="&lt;p&gt;I am going to screen small molecules against a protein target. Do I need to remove all the solvent molecules before performing VLS? &lt;/p&gt;&#xA;" OwnerUserId="7150" LastActivityDate="2020-06-18T10:49:01.117" Title="Do I need to remove water molecules before protein-ligand docking?" Tags="|proteins|docking|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13578" PostTypeId="2" ParentId="13563" CreationDate="2020-06-17T16:49:03.970" Score="0" Body="&lt;p&gt;It turns out the development version has a function &lt;code&gt;collect_by_key&lt;/code&gt; that is particularly helpful for scattering around predefined batches:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Array[String] X  =  [&quot;A&quot;,     &quot;A&quot;,     &quot;B&quot;,     &quot;A&quot;,     &quot;B&quot;]&#xA;Array[File] Y = [&quot;file1&quot;, &quot;file2&quot;, &quot;file3&quot;, &quot;file4&quot;, &quot;file5&quot;]&#xA;scatter (pair in as_pairs(collect_by_key(zip(X,Y)))) {&#xA;  call test {input: X = pair.left, Y = pair.right}&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This will call task &lt;code&gt;t&lt;/code&gt; first with input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;String X = &quot;A&quot;&#xA;Array[File] Y =[&quot;file1&quot;, &quot;file2&quot;, &quot;file4&quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;And then with input:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;String X = &quot;B&quot;&#xA;Array[File] Y =[&quot;file3&quot;, &quot;file5&quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;This is a readable solution to this problem allowed by &lt;code&gt;collect_by_key&lt;/code&gt;. This function is not very well advertised (I had completely missed it) but kudos to who introduced it in the language. The only drawback is that &lt;code&gt;collect_by_key&lt;/code&gt; is not available in version 1.0.&lt;/p&gt;&#xA;" OwnerUserId="9030" LastActivityDate="2020-06-17T16:49:03.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13579" PostTypeId="2" ParentId="13572" CreationDate="2020-06-17T17:14:58.730" Score="1" Body="&lt;p&gt;You can run a piece of software by specifying exactly where it is, like &lt;code&gt;/home/usr/software/samtools_1.1.0/samtools&lt;/code&gt;&lt;/p&gt;&#xA;&#xA;&lt;p&gt;Or if that location is in your $PATH, you can run it with just &lt;code&gt;samtools&lt;/code&gt;.  The software you downloaded needs to run those programs, but there's no way for you to give it the specific path for them; it's going to try and run them with just &lt;code&gt;samtools&lt;/code&gt;.  So you need to make sure that will work, by putting their locations in your PATH.&lt;/p&gt;&#xA;&#xA;&lt;p&gt;It is not clear to me that running every command with sudo is wise.  If you don't understand what $PATH is, I'm pretty sure you should not be running everything with sudo.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-06-17T17:14:58.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13580" PostTypeId="2" ParentId="4522" CreationDate="2020-06-17T23:50:38.257" Score="3" Body="&lt;p&gt;The edgeR authors recommend that you use a relatively low logFC threshold for &lt;code&gt;glmTreat&lt;/code&gt; such as &lt;code&gt;lfc=log2(1.2)&lt;/code&gt;. A &lt;code&gt;lfc&lt;/code&gt; value as high as &lt;code&gt;lfc=log2(2)&lt;/code&gt; is seldom required and only would only be appropriate for datasets with really large amounts of DE.&#xA;The first threshold you used of &lt;code&gt;lfc=2&lt;/code&gt; is equivalent to &lt;code&gt;lfc=log2(4)&lt;/code&gt; and is too high for any dataset.&lt;/p&gt;&#xA;&lt;p&gt;There's a few subtleties here:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;First, there's the difference between logFC and FC as explained by Devon.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;code&gt;lfc&lt;/code&gt; in &lt;code&gt;glmTreat&lt;/code&gt; is not a fold-change cutoff! It is instead a testing threshold. All the significant genes will have actual logFCs well above the threshold.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The lfc threshold applies to the raw fold-changes (&lt;code&gt;unshrunk.logFC&lt;/code&gt; in the output), not to the shrunk versions (&lt;code&gt;logFC&lt;/code&gt; in the table).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The intent of &lt;code&gt;glmTreat&lt;/code&gt; is to give a better gene ranking than would be provided by FDR or fold-change, so please don't re-sort the results by fold-change -- that would tend to defeat the purpose of the method.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I understand the documentation for &lt;code&gt;glmTreat&lt;/code&gt; is a bit light, but you can get an idea of how to use it from our workflows.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://bioconductor.org/packages/release/workflows/vignettes/RnaSeqGeneEdgeRQL/inst/doc/edgeRQL.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/workflows/vignettes/RnaSeqGeneEdgeRQL/inst/doc/edgeRQL.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://bioconductor.org/packages/release/workflows/vignettes/RNAseq123/inst/doc/limmaWorkflow.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/workflows/vignettes/RNAseq123/inst/doc/limmaWorkflow.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I understand the simple fold-change cutoff of FC &amp;gt; 2 with FDR &amp;lt; 0.05 is often used in the literature, but the edgeR authors do not recommend it and hence we don't provide an automatic way to get it from the edgeR pipeline.&#xA;For RNA-seq, fold-change cutoffs tend to prioritize genes with low expression and low counts.&#xA;The &lt;code&gt;glmTreat&lt;/code&gt; approach is more subtle but we think gives better results, both in terms of biological relevance and statistical correctness.&lt;/p&gt;&#xA;" OwnerUserId="9042" LastEditorUserId="9042" LastEditDate="2020-06-21T22:43:57.867" LastActivityDate="2020-06-21T22:43:57.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13581" PostTypeId="1" AcceptedAnswerId="16014" CreationDate="2020-06-17T23:52:38.767" Score="0" ViewCount="168" Body="&lt;p&gt;I am trying to use the &lt;code&gt;getGeneLengthAndGCContent&lt;/code&gt; function from the &lt;code&gt;EDASeq&lt;/code&gt; library to retrieve gene lengths for c. elegans. I retrieved Ensemble gene ids from Biomart for c.elegans and I'm using those as input to getGeneLengthAndGCContent (I picked &quot;ensemble_gene_id&quot; from the filters of the worm annotation Mart). Here's an example of how I'm trying to use the function:&lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;getGeneLengthAndGCContent(&quot;WBGene00001042&quot;, org, mode=c(&quot;biomart&quot;, &quot;org.db&quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;&lt;p&gt;When I try to run this, I get &lt;/p&gt;&#xA;&#xA;&lt;pre&gt;&lt;code&gt;Error in getGeneLengthAndGCContent(&quot;WBGene00001042&quot;, org, mode = c(&quot;biomart&quot;,  : &#xA;  Only ENTREZ or ENSEMBL gene IDs are supported.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2021-05-30T19:02:46.603" Title="getGeneLengthAndGCContent in EDASeq complains about &quot;Ensemble gene ID&quot;" Tags="|ensembl|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13582" PostTypeId="2" ParentId="13577" CreationDate="2020-06-18T10:49:01.117" Score="3" Body="&lt;p&gt;Yes, if you are using implicit solvent you ought to remove all of them.&lt;/p&gt;&#xA;&lt;p&gt;Exception. Unless the crystallographic waters are trapped essential structural components of your active site and your ligand is not going to replace it, in which case keep it because its remove changes the energy of the active site —say you have a catalytic triad, with a water that is pinned between the acid (e.g. aspartate) and the base histidine, its removal would change the tautomerisation of latter and result in the nucleophile being protonated (and not nucleophilic). Having said that, if you don't have time to look at your structure (or cannot because of automation) and are okay with some artefacts from incorrect protonation strip them.&lt;/p&gt;&#xA;&lt;p&gt;If you are not using implicit solvent, then it depends on the tool used —most require them stripped and re-added. If you do keep them, make sure that the crystallographic water (&lt;code&gt;HOH&lt;/code&gt;) has the same residue code as the added water (&lt;code&gt;TP3&lt;/code&gt;, &lt;code&gt;WAT&lt;/code&gt; etc.).&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-06-18T10:49:01.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13583" PostTypeId="2" ParentId="13574" CreationDate="2020-06-18T11:26:12.290" Score="1" Body="&lt;p&gt;Use &lt;a href=&quot;http://openbabel.org/wiki/Main_Page&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenBabel&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Something like &lt;code&gt;babel -isdf file_in.sdf -opdb file_out.pdb&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2020-06-18T11:26:12.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13584" PostTypeId="2" ParentId="13572" CreationDate="2020-06-18T12:00:57.620" Score="2" Body="&lt;p&gt;you can add a directory to &lt;code&gt;$PATH&lt;/code&gt; by executing following command in the terminal:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;export PATH=&amp;quot;$HOME/Documents/Software/bbmap:$PATH&amp;quot;&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this example, all executable commands in the folder &lt;code&gt;bbmap&lt;/code&gt; can be executed from any other directory in the terminal.&lt;/p&gt;&#xA;&lt;p&gt;So you can install &lt;code&gt;samtools&lt;/code&gt; in an appropriate directory and then execute &lt;code&gt;export PATH=&amp;quot;$HOME/your/directory:$PATH&amp;quot;&lt;/code&gt;. Then check if it succeeded by running &lt;code&gt;echo $PATH&lt;/code&gt;. This latter command will return a list of directories which are assigned to &lt;code&gt;$PATH&lt;/code&gt; and your new directory (where &lt;code&gt;samtools&lt;/code&gt; is located) should be included.&lt;/p&gt;&#xA;" OwnerUserId="7476" LastActivityDate="2020-06-18T12:00:57.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13586" PostTypeId="1" CreationDate="2020-06-18T14:23:08.867" Score="0" ViewCount="200" Body="&lt;p&gt;I converted an .sdf file containing around 50,000 anti-viral agents to individual .pdb files using OpenBabel. However, when I tried opening the .pdb files with Autodock I kept getting the error below. What do you think is causing the error.&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your time and patience.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Error Message:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;“ Error pasing the following line in pdb:&#xA;HETATM 1 C UNL 1 52355.00975659.051 0.000 1.00 0.000 .&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/13I5n.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/13I5n.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9040" LastEditorUserId="9040" LastEditDate="2020-06-18T18:03:10.087" LastActivityDate="2020-06-19T17:24:26.857" Title="Error pasing the following line in pdb" Tags="|pdb|docking|errors|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13587" PostTypeId="2" ParentId="13575" CreationDate="2020-06-18T14:56:08.977" Score="1" Body="&lt;p&gt;You can access the data.frame just using &lt;code&gt;$&lt;/code&gt;. All of them can be accessed using &lt;code&gt;values()&lt;/code&gt; . Something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(GenomicRanges)&#xA;gr = GRanges(seqnames=1,IRanges(start=1:10,width=1))&#xA;values(gr) = data.frame(eigen=rnorm(10),ratio=runif(10))&#xA;&#xA;plot(start(gr),gr$eigen)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plot(start(gr),values(gr)$eigen)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-06-18T14:56:08.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13588" PostTypeId="2" ParentId="13572" CreationDate="2020-06-18T19:38:14.773" Score="1" Body="&lt;p&gt;The clean way to do this is to make your own &lt;code&gt;bin&lt;/code&gt; within your user directory&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cd ~&#xA;mkdir bin&#xA;vi .profile # or vi .profile_bash or for OS X using `csh` or `tcsh` vi .cshrc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For bash (Linux, or OS X) within vi type &amp;quot;i&amp;quot; then type,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;PATH=$PATH:~/bin&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;PATH=~/bin:$PATH&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... followed by &lt;code&gt;esc&lt;/code&gt; key and &amp;quot;wp&amp;quot; &lt;code&gt;enter&lt;/code&gt;&#xA;Then enter for &lt;code&gt;csh&lt;/code&gt; or &lt;code&gt;tcsh&lt;/code&gt; e.g. for OS X&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;set path = ($path /Users/username/bin /Users/username/anaconda3/bin)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Same instructions for using &lt;code&gt;vi&lt;/code&gt; as for &lt;code&gt;bash&lt;/code&gt;. Dump bedtools into any directory (not the desktop) and connect the programs into the bin using &lt;code&gt;ln&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cd ~/progs/bedtools&#xA;ln -si $PWD/prog ~/bin&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you are using &lt;code&gt;csh&lt;/code&gt; or &lt;code&gt;tcsh&lt;/code&gt; you need to type &lt;code&gt;rehash&lt;/code&gt; but &lt;code&gt;bash&lt;/code&gt; does this automatically.&lt;/p&gt;&#xA;&lt;p&gt;The programs will then be cleaning connect to your $PATH via &lt;code&gt;ln&lt;/code&gt; without massive memory usage and visible anywhere within your home space. Its good practice for any Linux / OS X (time gone by UNIX) program or script and whilst you don't need it for this operation as you add more stuff its better.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-06-18T19:45:11.370" LastActivityDate="2020-06-18T19:45:11.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13589" PostTypeId="1" CreationDate="2020-06-18T20:47:37.740" Score="1" ViewCount="223" Body="&lt;p&gt;I am trying to follow the Beauclair et al. paper (free version here &lt;a href=&quot;https://rnajournal.cshlp.org/content/24/10/1285.long&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rnajournal.cshlp.org/content/24/10/1285.long&lt;/a&gt;) for identifying defective genomes using their DI-tector program.&lt;/p&gt;&#xA;&lt;p&gt;According to figure 2 &lt;a href=&quot;https://i.stack.imgur.com/8e1SZ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8e1SZ.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Before using the program itself, it is highly recommended to filter my raw read using &lt;strong&gt;bwa&lt;/strong&gt; and then &lt;strong&gt;samtools&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately I have no idea where to start - I have browsed official bwa and samtools sites, I have installed them using the following commands:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sudo apt-get update -y&#xA;sudo apt-get install -y bwa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have indexed my reference genome with the following command (as I read it is required before the analysis)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bwa index sequence.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then, I have installed samtools&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cd ..&#xA;wget https://github.com/samtools/samtools/releases/download/1.9/samtools-1.9.tar.bz2&#xA;tar -vxjf samtools-1.9.tar.bz2&#xA;cd samtools-1.9&#xA;make&#xA;sudo apt install samtools&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I wonder what will be the further steps to get these &amp;quot;UNMATCHED READS at the end of step i&amp;quot;, since Di-tector program takes virus_reference genome (I have it) and these unmatched reads.&lt;/p&gt;&#xA;&lt;p&gt;First of all I want to align my .fastq files vs &lt;code&gt;GCF_000001405.39_GRCh38.p13_genomic.fna&lt;/code&gt; - what will be the output file? If I got it right - I need this output file to filter with samtools, am I correct?&#xA;If I am not mistaken, the output after samtools will be the .sam file that I will have to put into a DI-tector - correct?&lt;/p&gt;&#xA;" OwnerUserId="6948" LastActivityDate="2020-06-19T04:24:31.920" Title="How can I get unmatched reads for defective genomes analysis using bwa and samtools?" Tags="|samtools|bwa|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13590" PostTypeId="2" ParentId="13521" CreationDate="2020-06-18T21:12:16.903" Score="1" Body="&lt;p&gt;I found this same error as of today using this code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pad.gene.data &amp;lt;-&#xA;  tximport(files = file.list.for.gene.analysis,&#xA;           type = &amp;quot;rsem&amp;quot;,&#xA;           txIn = FALSE,&#xA;           txOut = FALSE)&#xA;&#xA;dds.gene.level &amp;lt;-&#xA;  DESeqDataSetFromTximport(txi = pad.gene.data,&#xA;                           colData = pad.characteristic.data,&#xA;                           design = ~ condition)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I had done this same differential gene expression analysis using transcript counts aggregated to the gene level, on a mac about 5 weeks ago, but as of this past week switched to a windows and re-installed R 3.6.3 and whatever the current version of tximport and DESeq2 is.&lt;/p&gt;&#xA;&lt;p&gt;I was able to replicate the results I had gotten on the mac 5 weeks ago removing all rows where all gene lengths are greater than 0:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pad.gene.data&lt;span class=&quot;math-container&quot;&gt;$abundance &amp;lt;-&#xA;  pad.gene.data$&lt;/span&gt;abundance[apply(pad.gene.data$length,&#xA;                                1,&#xA;                                function(row) all(row !=0 )),]&#xA;&#xA;pad.gene.data&lt;span class=&quot;math-container&quot;&gt;$counts &amp;lt;-&#xA;  pad.gene.data$&lt;/span&gt;counts[apply(pad.gene.data$length,&#xA;                             1,&#xA;                             function(row) all(row !=0 )),]&#xA;&#xA;pad.gene.data&lt;span class=&quot;math-container&quot;&gt;$length &amp;lt;-&#xA;  pad.gene.data$&lt;/span&gt;length[apply(pad.gene.data$length,&#xA;                             1,&#xA;                             function(row) all(row !=0 )),]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So first removing any gene where there is a length count of 0 from the abundance and counts matrix, then removing the genes from the length matrix so that they are all the same size. It was able to run that way.&lt;/p&gt;&#xA;&lt;p&gt;I am definitely skeptical of why I would need to do that as it doesn't seem that it was advertised, if this is a new version, that we need to preprocess the data beforehand. I hope someone who might know more of the code can comment on this.&lt;/p&gt;&#xA;" OwnerUserId="9045" LastActivityDate="2020-06-18T21:12:16.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13591" PostTypeId="2" ParentId="13589" CreationDate="2020-06-19T04:24:31.920" Score="1" Body="&lt;p&gt;If you want to keep track of the unmapped reads, I recommend you to use &lt;a href=&quot;http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bowtie2&lt;/a&gt; instead of bwa.&#xA;Start by building an index:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bowtie2-build genome.fa genome.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then map your reads, using arguments to save unmapped reads in a specific file like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bowtie2 -x genome.fa -1 reads_1.fq -2 reads_2.fq -S bowtie2_mapping.sam --un-conc-gz unmappedreads_%.fq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It will write the alignment as well as unmappedreads_1.fq.gz and unmappedreads_2.fq.gz that you may use in DI-tector.&lt;/p&gt;&#xA;" OwnerUserId="8749" LastActivityDate="2020-06-19T04:24:31.920" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="13592" PostTypeId="1" CreationDate="2020-06-19T05:43:32.307" Score="1" ViewCount="434" Body="&lt;p&gt;I have cancer genomic data (tumor/normal whole exome sequencing) from 50&#xA;patients that received the same type of treatment, half of whom responded. These come in the form of 50 .maf files, along with a supplemental file that, along with some other fields, has the Response (Responder vs Non-Responder) field. My question is how to aggregate all of this together so that I can perform some statistical tests on the data. I have a clever way (taken from &lt;a href=&quot;https://www.gerkelab.com/blog/2018/09/import-directory-csv-purrr-readr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog post&lt;/a&gt;) of reading in the 50 .maf files and combining them, but I wonder if this is an appropriate approach.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sample_info &amp;lt;- readr::read_tsv(file = &amp;quot;path/to/sample_info/sample-information.tsv&amp;quot;)&#xA;maf_files &amp;lt;- fs::dir_ls(&amp;quot;path/to/mafs/&amp;quot;)&#xA;patients_data &amp;lt;- maf_files %&amp;gt;%&#xA;  purrr::map_dfr(read_tsv, col_types = list(Chromosome = col_character()))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My thought then was to just &lt;code&gt;dplyr::left_join()&lt;/code&gt; the &lt;code&gt;sample_info&lt;/code&gt; with &lt;code&gt;patients_data&lt;/code&gt; like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;patients_data_final &amp;lt;- patients_data %&amp;gt;% dplyr::left_join(sample_info, by = c(&amp;quot;Tumor_Sample_Barcode&amp;quot;, &amp;quot;Matched_Norm_Sample_Barcode&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For clarity, here are the column names of both dataframes&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; colnames(patients_data)&#xA; [1] &amp;quot;Hugo_Symbol&amp;quot;                 &amp;quot;Chromosome&amp;quot;                  &amp;quot;Start_position&amp;quot;             &#xA; [4] &amp;quot;End_position&amp;quot;                &amp;quot;Variant_Classification&amp;quot;      &amp;quot;Variant_Type&amp;quot;               &#xA; [7] &amp;quot;Reference_Allele&amp;quot;            &amp;quot;Tumor_Seq_Allele1&amp;quot;           &amp;quot;Tumor_Seq_Allele2&amp;quot;          &#xA;[10] &amp;quot;Tumor_Sample_Barcode&amp;quot;        &amp;quot;Matched_Norm_Sample_Barcode&amp;quot; &amp;quot;Protein_Change&amp;quot;             &#xA;[13] &amp;quot;t_alt_count&amp;quot;                 &amp;quot;t_ref_count&amp;quot;                &#xA;&#xA;&amp;gt; colnames(sample_info)&#xA;[1] &amp;quot;Patient_ID&amp;quot;                     &amp;quot;Tumor_Sample_Barcode&amp;quot;           &amp;quot;Matched_Norm_Sample_Barcode&amp;quot;   &#xA;[4] &amp;quot;Response&amp;quot;                       &amp;quot;Silent_mutations_per_Mb&amp;quot;        &amp;quot;Nonsynonymous_mutations_per_Mb&amp;quot;&#xA;[7] &amp;quot;Mutations_per_Mb&amp;quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My task is to find genes that &amp;quot;are enriched in patients who either&#xA;responded or not.&amp;quot; So as a supplemental question, if anyone has suggestions on which statistical test to use (or how to go about deciding), I'd appreciate that as well.&lt;/p&gt;&#xA;&lt;p&gt;PS: I am aware of the &lt;a href=&quot;https://bioconductor.org/packages/devel/bioc/vignettes/maftools/inst/doc/maftools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;maftools&lt;/a&gt; package which probably has an easy solution to this, but unfortunately my machine is old (Late 2011 MacBook Pro) and unable to run it. (Old Mac --&amp;gt; Can't update OS --&amp;gt; Cant update version of R --&amp;gt; Can't install necessary packages to run maftools)&lt;/p&gt;&#xA;" OwnerUserId="1090" LastEditorUserId="1090" LastEditDate="2020-06-19T17:06:21.087" LastActivityDate="2020-06-19T17:06:21.087" Title="How to aggregate multiple MAF files together?" Tags="|r|genomics|cancer|data-preprocessing|maf|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13593" PostTypeId="1" CreationDate="2020-06-19T09:01:57.853" Score="0" ViewCount="54" Body="&lt;p&gt;I have a series of 15bp DNA sequences from single-cell barcodes and I would like to encode them into a shorter than 15 digits number so that each unique 15bp DNA sequence will create a unique numeric value. What's the best way of achieving this?&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2020-06-19T14:55:48.990" Title="Encoding a 15bp DNA sequence into a shorter than 15 digits number" Tags="|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13594" PostTypeId="2" ParentId="13593" CreationDate="2020-06-19T11:45:51.880" Score="3" Body="&lt;p&gt;What you are asking is called a &amp;quot;hash&amp;quot; (&lt;a href=&quot;https://en.wikipedia.org/wiki/Hash_function&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;In python, get your oligo as a string and use the standard library method &lt;code&gt;hash(oligo)&lt;/code&gt;, this is a number that gets used for a lot of operations. It is however always a 18+1 digit number in decimal, so a different solution is needed.&lt;/p&gt;&#xA;&lt;p&gt;There are four nucleobases, so binary conversion from scratch is easy and best option and each solution will be unique and small. Each base takes up 2 bits and you concatenate these and convert them to hex or decimal —the length should be 10 decimal digits. One issue is when you have a oligo of a different length which start with adenosine or not.&lt;/p&gt;&#xA;&lt;p&gt;One issue with this is the order. I am not sure where it comes from but the bases are nearly often expressed in the order ATGC. So...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mapping = {'A': '00', 'T': '01', 'G': '10', 'C': '11'}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It is satisfying how the first digit is the W/S, while the second is R/Y.&lt;/p&gt;&#xA;&lt;p&gt;Now simply covert and add them &lt;code&gt;compressed = functools.reduce(lambda acc,base : acc+mapping[base], oligo, '')&lt;/code&gt; and covert the string to decimal like so &lt;code&gt;int(compressed, 2)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-06-19T14:53:49.823" LastActivityDate="2020-06-19T14:53:49.823" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13595" PostTypeId="2" ParentId="13586" CreationDate="2020-06-19T11:53:47.087" Score="1" Body="&lt;p&gt;A PDB file has strict spacing rules as documented &lt;a href=&quot;http://www.wwpdb.org/documentation/file-format-content/format33/sect9.html#HETATM&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&#xA;The coordinates should be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;31 - 38       Real(8.3)      x             Orthogonal coordinates for X.&#xA;39 - 46       Real(8.3)      y             Orthogonal coordinates for Y.&#xA;47 - 54       Real(8.3)      z             Orthogonal coordinates for Z.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So trying this on your line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;'HETATM 1 C UNL 1 52355.00975659.051 0.000 1.00 0.000'[30:39]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get &lt;code&gt;9.051 0.&lt;/code&gt;, which is not a number.&lt;/p&gt;&#xA;&lt;p&gt;The whole line is formatted incorrectly however.&#xA;So as you say you are using Autodock why not pass a &lt;code&gt;mol2&lt;/code&gt; to &lt;code&gt;prepare_ligand4.py&lt;/code&gt; instead which is &lt;a href=&quot;http://autodock.scripps.edu/faqs-help/how-to/how-to-prepare-a-ligand-file-for-autodock4&quot; rel=&quot;nofollow noreferrer&quot;&gt;an acceptable format&lt;/a&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, you could convert your multi-entry sdf to pdb files via RDKit in Python:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;suppl = Chem.SDMolSupplier(sdfile)&#xA;for mol in suppl:&#xA;      # mol = Chem.AddHs(mol) # has hydrogens?&#xA;      # AllChem.EmbedMolecule(mol) # is 3D?&#xA;      AllChem.ComputeGasteigerCharges(m) # sdfs do not have partial charges&#xA;      Chem.MolToPDBFile(m, f&amp;quot;{mol.GetProp('_Name')}.pdb&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-06-19T17:24:26.857" LastActivityDate="2020-06-19T17:24:26.857" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="13596" PostTypeId="1" CreationDate="2020-06-19T12:59:42.190" Score="1" ViewCount="2877" Body="&lt;p&gt;I have recently found myself making multiple heatmaps for visualization of differential gene expression results between replicates in two experimental conditions.&lt;/p&gt;&#xA;&lt;p&gt;The default settings in the plotting software I use color:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Upregulated genes --&amp;gt; red&lt;/li&gt;&#xA;&lt;li&gt;Downregulated genes --&amp;gt; green&lt;/li&gt;&#xA;&lt;li&gt;Unchanged genes --&amp;gt; black&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;My intuition &lt;em&gt;screams&lt;/em&gt; at me that this color scheme is reversed, and that upregulated genes should be green and that downregulated genes should be red. I have asked several labmates about their intuition regarding this color scheme, and have found about a 50/50 split in those that intuitively agree with each different scheme.&lt;/p&gt;&#xA;&lt;p&gt;In my searching on this topic, I have found that &lt;a href=&quot;https://docs.patricbrc.org/user_guides/organisms_taxon/transcriptomics_gene_heatmap.html#:%7E:text=The%20default%20color%20scheme%20is,off%20setting%20are%20colored%20black.&quot; rel=&quot;nofollow noreferrer&quot;&gt;several programs&lt;/a&gt; plot heatmaps with the same default colors as the program I'm using (red=upregulated, green=downregulated). Additionally, I have been told in conversation that this is essentially an accepted standard.&lt;/p&gt;&#xA;&lt;p&gt;If this is indeed the convention, I will begin trying to rewire my intuition, but I want to confirm that it is before I do so. However I have not yet been able to find any &amp;quot;official&amp;quot; guidelines from agencies, publishers, authorities in the field, etc., specifying the use of red/green in heatmaps.&lt;/p&gt;&#xA;&lt;p&gt;I do know that there are many &lt;a href=&quot;https://blog.bioturing.com/2018/09/24/heatmap-color-scale/&quot; rel=&quot;nofollow noreferrer&quot;&gt;suggestions for alternative heatmap color schemes&lt;/a&gt;, but I'm interested in finding an official answer for the red/green scheme specifically.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt; Several comments have stated that there is no such standard as described above. Therefore I am modifying the following question somewhat to:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Are there &amp;quot;official&amp;quot; guidelines from agencies, publishers, authorities in the field, etc., specifying the use of red/green in heatmaps?&#xA;Or conversely are there any “official” guidelines stating that there is no such specification and/or that any such selection of colors is acceptable?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="7185" LastEditorUserId="7185" LastEditDate="2020-06-22T13:59:03.343" LastActivityDate="2020-07-05T16:27:00.753" Title="Is there an &quot;official&quot; standard usage of Red and Green on differential gene expression heatmaps?" Tags="|differential-expression|visualization|heatmap|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13598" PostTypeId="2" ParentId="13528" CreationDate="2020-06-19T15:01:24.637" Score="2" Body="&lt;p&gt;I would align your sequences with the software available here: &lt;a href=&quot;https://github.com/veg/hyphy-analyses/tree/master/codon-msa&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/veg/hyphy-analyses/tree/master/codon-msa&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9051" LastActivityDate="2020-06-19T15:01:24.637" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13599" PostTypeId="1" CreationDate="2020-06-19T18:55:09.003" Score="0" ViewCount="228" Body="&lt;p&gt;I have a bam file of, for instance, RNA-Seq, which contains patient-identifiable data in the form of Single Nucleotide Polymorphisms (SNPs) throughout.&lt;/p&gt;&#xA;&lt;p&gt;I would like a technique to take this aligned bam file and replace all the non-reference bases with the reference bases at the equivalent positions (hard-clipping any soft-clipped bases while doing so), in order to have an anonymised bam file without the patient-identifiable information.&lt;/p&gt;&#xA;&lt;p&gt;What is the best way to do this? I want the resulting anonymised bam file to have the reads in the same positions, for downstream tools that care about the alignment positions.&lt;/p&gt;&#xA;" OwnerUserId="2155" LastActivityDate="2020-06-19T21:53:05.560" Title="How to anonymise a bam file" Tags="|sam|ngs|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13600" PostTypeId="1" CreationDate="2020-06-19T21:14:16.943" Score="1" ViewCount="41" Body="&lt;p&gt;The steps I have done so far:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;used guppy to do base calling from fast5 to fastq. In each barcode file I have 1960 fastq files which I have merged.&lt;/li&gt;&#xA;&lt;li&gt;trimmed the fastq files using NanoFilt&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Now what I see from various resources is they do de-novo assembly using tools like canu followed by minimap.&lt;/p&gt;&#xA;&lt;p&gt;So is it needed to do de-novo assembly?&lt;/p&gt;&#xA;&lt;p&gt;Or I can simply use my fastq files and process them using tools like kraken for taxonomic classification?&lt;/p&gt;&#xA;&lt;p&gt;Any help or suggestion would be really appreciated&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="73" LastEditDate="2022-09-02T10:55:40.777" LastActivityDate="2022-09-02T10:55:40.777" Title="Nanopore microbiome data analysis 16s rRNA data" Tags="|nanopore|microbial-genomics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13601" PostTypeId="2" ParentId="13599" CreationDate="2020-06-19T21:53:05.560" Score="1" Body="&lt;p&gt;You can simply replace the sequence and quality scores with a single &lt;code&gt;*&lt;/code&gt; in each field. The CIGAR information will still be there, so it will be apparent where there were InDels and possibly mismatches (assuming your aligning uses &lt;code&gt;=&lt;/code&gt; and &lt;code&gt;X&lt;/code&gt; CIGAR operations), but since the actual sequence of these variants cannot be known this is quite anonymized.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-06-19T21:53:05.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13603" PostTypeId="1" AcceptedAnswerId="13614" CreationDate="2020-06-20T06:45:19.000" Score="0" ViewCount="356" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/444838/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt; and &lt;a href=&quot;https://stackoverflow.com/questions/62480055/trying-to-convert-ensembl-id-to-gene-name-in-r-biomart&quot;&gt;StackOverflow&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I've been trying to code (in R) a way to convert gene accession numbers to gene names (from RNAseq data). I've looked at all the related questions and tried to modify my code such, but for some reason it's still not working. Here is my code, where &lt;code&gt;charg&lt;/code&gt; is a character vector of the gene accession ID's of the data set &lt;code&gt;resdata&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;charg &amp;lt;- resdata$genes&#xA;head(charg)&#xA;&#xA;library(biomaRt)&#xA;&#xA;ensembl = useMart(&amp;quot;ensembl&amp;quot;,dataset=&amp;quot;hsapiens_gene_ensembl&amp;quot;)&#xA;&#xA;theBM &amp;lt;- getBM(attributes='ensembl_gene_id','hgnc_symbol', &#xA;      filters = 'external_gene_name', &#xA;      values = charg, &#xA;      mart = ensembl)&#xA;&#xA;resdata &amp;lt;- merge.data.frame(resdata, theBM, by.x=&amp;quot;genes&amp;quot;,by.y=&amp;quot;ensembl_gene_id&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here's some output (where I'm struggling):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(charg)&#xA;[1] &amp;quot;ENSG00000261150.2&amp;quot;  &amp;quot;ENSG00000164877.18&amp;quot; &amp;quot;ENSG00000120334.15&amp;quot;&#xA;[4] &amp;quot;ENSG00000100906.10&amp;quot; &amp;quot;ENSG00000182759.3&amp;quot;  &amp;quot;ENSG00000124145.6&amp;quot; &#xA;&#xA;&amp;gt; dim(theBM)&#xA;[1] 0 1&#xA;&#xA;&amp;gt; head(theBM)&#xA;[1] ensembl_gene_id&#xA;&amp;lt;0 rows&amp;gt; (or 0-length row.names)&#xA;&#xA;&amp;gt; dim(resdata)&#xA;[1] 20381    11&#xA;&amp;gt; resdata &amp;lt;- merge.data.frame(resdata, theBM, by.x=&amp;quot;genes&amp;quot;,by.y=&amp;quot;ensembl_gene_id&amp;quot;)&#xA;&amp;gt; dim(resdata) #after merge&#xA;[1]  0 11 #isn't correct -- just row names! where'd my genes go?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="9056" LastEditorUserId="73" LastEditDate="2021-06-15T10:42:55.203" LastActivityDate="2021-06-15T10:42:55.203" Title="Problem with merge data while trying to convert gene names in R" Tags="|r|rna-seq|gene|biomart|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13605" PostTypeId="1" AcceptedAnswerId="13607" CreationDate="2020-06-20T10:17:50.157" Score="1" ViewCount="303" Body="&lt;p&gt;PhyML is a tool used for maximum likelihood estimation analysis.&#xA;I was running a PhyML analysis to check the rate of nucleotide change in DNA sequences and in the PhyML output, there are rate parameters and an instantaneous rate matrix. Could you please clarify the difference between the two?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BIP3T.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BIP3T.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What I've understood so far is that the rate parameters indicate, for ex: that the A -C transition rate is 1.95 and so on. If so what does the instantaneous rate matrix tell us?&lt;/p&gt;&#xA;" OwnerUserId="8767" LastActivityDate="2020-06-21T02:27:04.953" Title="What does instantaneous rate matrix mean?" Tags="|phylogenetics|phylogeny|hidden-markov-models|nucleotide-models|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13606" PostTypeId="1" CreationDate="2020-06-20T12:22:08.063" Score="0" ViewCount="714" Body="&lt;p&gt;I have 2 .fa sequenes like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;some description of first...&#xA;ACTGACTACGTACGACTACGATCTGACTACACGTAGATAGACTAGTCACTACG&#xA;ACTGACTACGTACGACTGACTACATGACTGAC...&#xA;&amp;gt; some description of second...&#xA;ATCGATAGCTACGATCGATAGTAGATAG....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This file is 461mb (1 chromosome of homo sapiens &amp;amp; 1 chromosome of pan troglodytes) and I want to align it. I am using mafft.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mafft --globalpair --maxiterate 1000 Homo_sapiens_1_Pan_troglodytes_1.fa &amp;gt; ./mafft_results/Homo_sapiens_1_Pan_troglodytes_1_output.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And it returns me&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;generating a scoring matrix for nucleotide (dist=200) ... done&#xA;All-to-all alignment.&#xA;/usr/bin/mafft: line 2719:  6030 Killed                  &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$prefix/tbfast&quot; _ -u $&lt;/span&gt;unalignlevel &lt;span class=&quot;math-container&quot;&gt;$localparam -C $&lt;/span&gt;numthreads &lt;span class=&quot;math-container&quot;&gt;$seqtype $&lt;/span&gt;model -g &lt;span class=&quot;math-container&quot;&gt;$pgexp -f $&lt;/span&gt;pggop -Q &lt;span class=&quot;math-container&quot;&gt;$spfactor -h $&lt;/span&gt;pgaof -A &lt;span class=&quot;math-container&quot;&gt;$usenaivepairscore $&lt;/span&gt;focusarg _ -+ &lt;span class=&quot;math-container&quot;&gt;$iterate -W $&lt;/span&gt;minimumweight -V &amp;quot;-&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$gopdist -s $&lt;/span&gt;unalignlevel &lt;span class=&quot;math-container&quot;&gt;$legacygapopt $&lt;/span&gt;mergearg &lt;span class=&quot;math-container&quot;&gt;$outnum $&lt;/span&gt;addarg &lt;span class=&quot;math-container&quot;&gt;$add2ndhalfarg -C $&lt;/span&gt;numthreadstb &lt;span class=&quot;math-container&quot;&gt;$rnaopt $&lt;/span&gt;weightopt &lt;span class=&quot;math-container&quot;&gt;$treeinopt $&lt;/span&gt;treeoutopt &lt;span class=&quot;math-container&quot;&gt;$distoutopt $&lt;/span&gt;seqtype &lt;span class=&quot;math-container&quot;&gt;$model -f &quot;-&quot;$&lt;/span&gt;gop -Q &lt;span class=&quot;math-container&quot;&gt;$spfactor -h $&lt;/span&gt;aof &lt;span class=&quot;math-container&quot;&gt;$param_fft $&lt;/span&gt;localparam &lt;span class=&quot;math-container&quot;&gt;$algopt $&lt;/span&gt;treealg &lt;span class=&quot;math-container&quot;&gt;$scoreoutarg $&lt;/span&gt;focusarg &amp;lt; infile &amp;gt; /dev/null 2&amp;gt;&amp;gt; &amp;quot;$progressfile&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Why? How could I align it for further comparing?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 1&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;MAFFT working cool, when I tried to use it on test sequence file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;test1&#xA;ATCGATCGATGACTACGATCGACTACGTACG&#xA;ACTAGCTAGCTACGCATAGTAGATCGA&#xA;ATCGACTAGCTCAGATCGACTAGATCGATCGT&#xA;&amp;gt;test2&#xA;TTATCCACTGATCGACTCAGATCGACTCAGAC&#xA;ATCGATCAGCTACGATGACTACGTAC&#xA;ACTGACTAGCTAGACTGACTAGCTACGTCA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and after MAFFT &lt;code&gt;mafft test.fa &amp;gt; output.fa&lt;/code&gt; it returned&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;test1&#xA;------atcgatcgat-------------gactacgatcgactacgtacgactagctagc&#xA;tacgcatagtagatcgaatcgactagctcagatcgactagatcgatcgt&#xA;test2&#xA;ttatccactgatcgactcagatcgactcagacatcgatcagctac---------gatgac&#xA;tacgtac----------actgactagct-agactgactagctacgtca-&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So it works.&lt;/p&gt;&#xA;&lt;p&gt;I can use all 4 at the example. Now about my large file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;alex@alex:~/Desktop/test/BOTH/homo_sapiens_pan_troglodytes$ mafft --thread 4 --maxiterate 2 Homo_sapiens_1_Pan_troglodytes_1.fa &amp;gt; ./mafft_results/Homo_sapiens_1_Pan_troglodytes_1_output.fa&#xA;nthread = 4&#xA;nthreadpair = 4&#xA;nthreadtb = 4&#xA;ppenalty_ex = 0&#xA;reallocating...&#xA;done.&#xA;reallocating...&#xA;done.&#xA;reallocating...&#xA;    done.&#xA;reallocating...&#xA;done.&#xA;reallocating...&#xA;...&#xA;stacksize: 8192 kb&#xA;reallocating...&#xA;done.&#xA;reallocating...&#xA;done.&#xA;reallocating...&#xA;done.&#xA;...&#xA;generating a scoring matrix for nucleotide (dist=200) ... done&#xA;Gap Penalty = -1.53, +0.00, +0.00&#xA;&#xA;&#xA;&#xA;Making a distance matrix ..&#xA;    1 / 2 (thread    1)&#xA;done.&#xA;&#xA;Constructing a UPGMA tree (efffree=1) ... &#xA;    0 / 2&#xA;done.&#xA;&#xA;Progressive alignment 1/1... &#xA;&#xA;len1=230480994, len2=223796828, Switching to the memsave mode&#xA;STEP     1 / 1 (thread    0)m/usr/bin/mafft: line 2719:  3192 Killed                  &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$prefix/disttbfast&quot; -q $&lt;/span&gt;npickup -E &lt;span class=&quot;math-container&quot;&gt;$cycledisttbfast -V &quot;-&quot;$&lt;/span&gt;gopdist -s &lt;span class=&quot;math-container&quot;&gt;$unalignlevel $&lt;/span&gt;legacygapopt &lt;span class=&quot;math-container&quot;&gt;$mergearg -W $&lt;/span&gt;tuplesize &lt;span class=&quot;math-container&quot;&gt;$termgapopt $&lt;/span&gt;outnum &lt;span class=&quot;math-container&quot;&gt;$addarg $&lt;/span&gt;add2ndhalfarg -C &lt;span class=&quot;math-container&quot;&gt;$numthreads-$&lt;/span&gt;numthreadstb &lt;span class=&quot;math-container&quot;&gt;$memopt $&lt;/span&gt;weightopt &lt;span class=&quot;math-container&quot;&gt;$treeinopt $&lt;/span&gt;treeoutopt &lt;span class=&quot;math-container&quot;&gt;$distoutopt $&lt;/span&gt;seqtype &lt;span class=&quot;math-container&quot;&gt;$model -g $&lt;/span&gt;gexp -f &amp;quot;-&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$gop -Q $&lt;/span&gt;spfactor -h &lt;span class=&quot;math-container&quot;&gt;$aof $&lt;/span&gt;param_fft &lt;span class=&quot;math-container&quot;&gt;$algopt $&lt;/span&gt;treealg &lt;span class=&quot;math-container&quot;&gt;$scoreoutarg $&lt;/span&gt;anchoropt &amp;lt; infile &amp;gt; pre 2&amp;gt;&amp;gt; &amp;quot;$progressfile&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Doesnt matter I do not enter &lt;code&gt;--thread 4&lt;/code&gt; or not, &lt;code&gt;--maxiterate 2&lt;/code&gt; or &lt;code&gt;--maxiterate 1000&lt;/code&gt;, it still shows me same error. It takes around 2 minute. Ram usage when running at the first minute is 6656mb, after 1 min 11028mb and then it out of ram and error..&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;PS&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;discussion moved &lt;a href=&quot;https://stackoverflow.com/questions/62488623/out-of-ram-while-work-with-very-large-matrix&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9024" LastEditorUserId="-1" LastEditDate="2020-06-20T19:02:12.240" LastActivityDate="2020-06-20T19:02:12.240" Title="Can't align 2 sequences, (MAFFT killed)" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13607" PostTypeId="2" ParentId="13605" CreationDate="2020-06-20T13:04:39.353" Score="1" Body="&lt;p&gt;&lt;strong&gt;Reversible&lt;/strong&gt; At its simplist the GTR rate, is the General Time Reversible model and most importantly infers the matrix is &amp;quot;reversible&amp;quot;, thus as many A mutations will go to C as C mutations go to A and the number of mutations is relative to the G-T mutation rate. The G-T mutation rate is one of the least common mutations (transversion) so all other values are above 1. If you plotted this as a matrix the top right-hand matrix is identical to the bottom left hand matrix because its reversible, i.e. A &amp;lt;-&amp;gt; C . This is cool because you don't need to tell the algorithm which sequence is the root.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Directional&lt;/strong&gt; The &amp;quot;instantaneous&amp;quot; matrix is a directional matrix, thus A -&amp;gt; C is not the same as C -&amp;gt; A. In order to obtain an accurate directional matrix you need to assign a root, its like lets start from the deepest node in the tree (the one between the root/outgroup and all ingroup sequences) and now - if that is correct - we can remove the restrictions of reversibility and impose directionality on the mutation process. This is needed if you are performing molecular clock type calculations. However, the key is assigning the root/outgroup - if it assigns it e.g. by mid-point rooting that is not necessarily a good idea.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Advice.&lt;/strong&gt; My advice is if you are building a basic tree and haven't assigned a root/outgroup, obviously using GTR which is a good move, then you don't to worry about the intananeous matrix because it might not be meaningful. If you are running a molecular clock/dating/rate estimate type model and a very clear what the root/outgroup is and that is specified in the input, then the instananeous matrix becomes important.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The mutation rates of GTR and instananeous are not directly comparable because the GTR rate is with respect to G &amp;lt;-&amp;gt; T mutation, whilst instananeous is primarily concerned with the directionality of mutation and is a 12-parameter model, as opposed to GTR which 6-perameters, where one of those parameters is the reference.&lt;/p&gt;&#xA;&lt;p&gt;Thus G-T is reversible in GTR (the reference) and so e.g. A&amp;lt;-&amp;gt;C is reversible with respect to G&amp;lt;-&amp;gt;T.&#xA;In instananeous matrix G -&amp;gt; T and T -&amp;gt; G are two different rates as are A-&amp;gt; C and C -&amp;gt; A, so one approximation based on an assumption with respect to a second approximation based on the same assumption can't directly equate to a mutation rate which removes that assumption.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-06-21T02:27:04.953" LastActivityDate="2020-06-21T02:27:04.953" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13608" PostTypeId="2" ParentId="13603" CreationDate="2020-06-20T13:22:26.437" Score="1" Body="&lt;p&gt;Those aren't external_gene_name's, they're ensembl_gene_id_versions:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;theBM &amp;lt;- getBM(attributes='ensembl_gene_id','hgnc_symbol', &#xA;               filters = 'ensembl_gene_id_version', &#xA;               values = charg2, &#xA;               mart = ensembl)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that you'll get more hits if you strip the gene ID versions off:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;charg2 = sapply(strsplit(charg, '.', fixed=T), function(x) x[1])&#xA;theBM = getBM(attributes='ensembl_gene_id','hgnc_symbol', &#xA;              filters = 'ensembl_gene_id', &#xA;              values = charg2, &#xA;              mart = ensembl)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-06-20T13:22:26.437" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="13609" PostTypeId="2" ParentId="13606" CreationDate="2020-06-20T14:43:43.903" Score="1" Body="&lt;p&gt;&lt;code&gt;--thread 4&lt;/code&gt; is essential, &lt;code&gt;--maxiterate&lt;/code&gt; just try and get it working, not getting it to work optimally. Just make sure there are no other processes running on the machine when --thread 4 is operating, e.g. you are using another machine as your desktop&lt;/p&gt;&#xA;&lt;p&gt;The gist is as you spread the calculation across your processors (4) and simplify the calculation (lowering the iterations), you are getting further into the calculation, for example now you are getting into gap penalties.&lt;/p&gt;&#xA;&lt;p&gt;If you are agreed then I would try performing the alignment using just one chromosome of the human genome against the chimp genome and I suspect that would work. Then take it from there.&lt;/p&gt;&#xA;&lt;p&gt;The objective is to get the calculation working, for example I wouldn't trust a result with max. iterations of 2, but once you have it working you can then start refining the parameters towards optimal results. The i3 processor isn't perfect in context.&lt;/p&gt;&#xA;&lt;p&gt;Using &lt;code&gt;--thread 4&lt;/code&gt; is super important and you want to increase maxiterate towards the default if the machine supports it.&lt;/p&gt;&#xA;&lt;p&gt;I suspect the solution is to perform the calculation on an 8+-core machine and set &lt;code&gt;--thread 8&lt;/code&gt; (or higher depending on the number of cores on the machine), at least it will get further into the calculation. With 4-cores and an i3 (diagnostics of RAM usage would be nice) in any case it just doesn't seem to like it. The alternative is to perform this across the genome via sampling the chromosomes on your current i3.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Just to explain why this works. When you have a RAM intensive job and you spread it across cores the RAM overhead massively reduces for each core that is added. Don't ask me how this works in the hardware mecahanics of RAM distribution per core. The reason I know this is because of emperically parallelising Bayesian calculation using each chain on a different core.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-06-20T18:35:44.533" LastActivityDate="2020-06-20T18:35:44.533" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13612" PostTypeId="1" CreationDate="2020-06-20T19:35:00.487" Score="2" ViewCount="1819" Body="&lt;p&gt;The issue im facing is I'm assignig the colour to the annotation label but when it is rendered the label in my metadata does't match with the final output not sure what exactly is the problem.&lt;/p&gt;&#xA;&lt;p&gt;The code&lt;/p&gt;&#xA;&lt;p&gt;data file  &lt;a href=&quot;https://drive.google.com/file/d/1rjD0xCd3mK29KIFJH-BUZRBKJ7Oh9pcy/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;ka_ks&lt;/a&gt; and &lt;a href=&quot;https://drive.google.com/file/d/1Ofe8YjZ3Jns4oQ520bJu-lUGeFd0Op-f/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;metadata&lt;/a&gt;  &lt;a href=&quot;https://i.stack.imgur.com/Htgj5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Htgj5.png&quot; alt=&quot;figure&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As an example this sample Q6Q1S2 in my metadata is Alpha but in the figure it is labelled under Gamma&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion or help would be really appreciated&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    data &amp;lt;- read.csv(&amp;quot;ka_ks.txt&amp;quot;,header = TRUE,row.names = 1,sep = &amp;quot;,&amp;quot;)&#xA;head(data)&#xA;&#xA;&#xA;#metadata &amp;lt;- read.csv(&amp;quot;NUC_META_AMINO_heatmap_metadata_sans_nucleotide.txt&amp;quot;,sep = &amp;quot;,&amp;quot;)&#xA;metadata &amp;lt;- read.csv('vir_meta.txt',sep = &amp;quot;,&amp;quot;)&#xA;#head(data)&#xA;head(metadata)&#xA;#names(metadata)[3] = &amp;quot;Host&amp;quot;&#xA;&#xA;&#xA;#Set annotation&#xA;ann &amp;lt;- metadata[,c(2,3)]&#xA;colnames(ann) &amp;lt;- c(&amp;quot;Class&amp;quot;,&amp;quot;Host&amp;quot;)&#xA;#rownames(ann) = metadata$Sample&#xA;ann&#xA;#ann = metadata&#xA;colours &amp;lt;- list(&amp;quot;Host&amp;quot;=c(&amp;quot;Human SARS coronavirus 2&amp;quot;=&amp;quot;green4&amp;quot;,&#xA;                         &amp;quot;Human SARS coronavirus&amp;quot;=&amp;quot;darkgreen&amp;quot;,&#xA;                         &amp;quot;Bat coronavirus&amp;quot;=&amp;quot;wheat4&amp;quot;,&#xA;                         &amp;quot;Murine coronavirus&amp;quot;=&amp;quot;turquoise&amp;quot;,&amp;quot;MERS&amp;quot;=&amp;quot;orange&amp;quot;,&#xA;                         &amp;quot;Bovine coronavirus&amp;quot;=&amp;quot;coral&amp;quot;,&#xA;                         &amp;quot;Rat coronavirus&amp;quot;=&amp;quot;saddle brown&amp;quot;,&#xA;                         &amp;quot;Human coronavirus&amp;quot;=&amp;quot;dark green&amp;quot;,&#xA;                         &amp;quot;Porcine hemagglutinating encephalomyelitis virus&amp;quot;=&amp;quot;maroon&amp;quot;,&#xA;                         &amp;quot;Avian infectious bronchitis virus&amp;quot;=&amp;quot;dark slate gray&amp;quot;,&#xA;                         &amp;quot;Porcine respiratory coronavirus&amp;quot;=&amp;quot;slategrey&amp;quot;,&#xA;                         &amp;quot;Canine coronavirus&amp;quot;=&amp;quot;purple&amp;quot;,&#xA;                         &amp;quot;Porcine transmissible gastroenteritis coronavirus&amp;quot;=&amp;quot;pink&amp;quot;,&#xA;                         &amp;quot;Feline coronavirus&amp;quot;=&amp;quot;gold&amp;quot;,&#xA;                         &amp;quot;Porcine epidemic diarrhea virus&amp;quot;=&amp;quot;brown&amp;quot;),&#xA;                &amp;quot;Class&amp;quot;= c(&amp;quot;Alpha&amp;quot; = &amp;quot;blue&amp;quot; ,&amp;quot;Beta&amp;quot; =&amp;quot;black&amp;quot;,&amp;quot;Gamma&amp;quot; =&amp;quot;red&amp;quot;))&#xA;&#xA;&#xA;heat &amp;lt;- scale((data))&#xA;head(heat)&#xA;&#xA;colAnn &amp;lt;- HeatmapAnnotation(df=ann, which=&amp;quot;col&amp;quot;,col=colours, annotation_width=unit(c(1, 4), &amp;quot;cm&amp;quot;), gap=unit(1, &amp;quot;mm&amp;quot;))&#xA;myCol1 &amp;lt;- colorRampPalette(c(&amp;quot;navyblue&amp;quot;, &amp;quot;white&amp;quot;, &amp;quot;red&amp;quot;))(100)&#xA;myBreaks1 &amp;lt;- seq(-4, 4, length.out=100)&#xA;&#xA;&#xA;#Add column boxplot&#xA;boxplotCol &amp;lt;- HeatmapAnnotation(&#xA;  boxplot = anno_boxplot(data.matrix(heat),&#xA;                         border = TRUE,&#xA;                         gp = gpar(fill = '#CCCCCC'),&#xA;                         pch = '.',&#xA;                         size = unit(2, 'mm'),&#xA;                         axis = TRUE,&#xA;                         axis_side = 'left',&#xA;                         axis_gp = gpar(fontsize = 12)),&#xA;  annotation_width = unit(c(1, 5.0), 'cm'),&#xA;  which = 'col')&#xA;&#xA;&#xA;hmap1 &amp;lt;- Heatmap(heat,&#xA;                 name = 'ks_score',&#xA;                 #name=&amp;quot;Z-score&amp;quot;,&#xA;                 col=colorRamp2(myBreaks1, myCol1),&#xA;                 heatmap_legend_param=list(color_bar=&amp;quot;continuous&amp;quot;, &#xA;                                           legend_direction=&amp;quot;vertical&amp;quot;, legend_width=unit(5,&amp;quot;cm&amp;quot;),&#xA;                                           title_position=&amp;quot;lefttop&amp;quot;, title_gp=gpar(fontsize=10, fontface=&amp;quot;bold&amp;quot;)),&#xA;                 &#xA;                 #Split heatmap rows by gene family&#xA;                 #split=df$Family,&#xA;                 #split = NULL,&#xA;                 &#xA;                 #Row annotation configurations&#xA;                 cluster_rows=TRUE,&#xA;                 show_row_dend=TRUE,&#xA;                 row_dend_side = &amp;quot;left&amp;quot;,&#xA;                 #row_title=&amp;quot;Transcript&amp;quot;, #overridden by 'split' it seems&#xA;                 row_title_side=&amp;quot;left&amp;quot;,&#xA;                 row_title_gp=gpar(fontsize=30,fontface=&amp;quot;bold&amp;quot;),&#xA;                 row_names_gp = gpar(fontsize = 10),&#xA;                 show_row_names=TRUE,&#xA;                 row_names_side=&amp;quot;right&amp;quot;,&#xA;                 row_title_rot=0,&#xA;                 &#xA;                 column_dend_reorder = TRUE,&#xA;                 #row_dend_reorder = TRUE,&#xA;                 &#xA;                 #Column annotation configuratiions&#xA;                 cluster_columns=TRUE,&#xA;                 show_column_dend=FALSE,&#xA;                 column_title=&amp;quot;Samples&amp;quot;,&#xA;                 column_title_side=&amp;quot;top&amp;quot;,&#xA;                 column_title_gp=gpar(fontsize=20, fontface=&amp;quot;bold&amp;quot;),&#xA;                 column_title_rot=0,&#xA;                 column_names_gp = gpar(fontsize = 10, fontface=&amp;quot;bold&amp;quot;),&#xA;                 show_column_names=TRUE,&#xA;                 &#xA;                 #Dendrogram configurations: columns&#xA;                 #clustering_distance_columns=&amp;quot;euclidean&amp;quot;,&#xA;                 clustering_method_columns=&amp;quot;ward.D2&amp;quot;,&#xA;                 column_dend_height=unit(10,&amp;quot;mm&amp;quot;),&#xA;                 &#xA;                 #Dendrogram configurations: rows  &#xA;                 #clustering_distance_rows=&amp;quot;euclidean&amp;quot;,&#xA;                 clustering_method_rows=&amp;quot;ward.D2&amp;quot;,&#xA;                 row_dend_width=unit(15,&amp;quot;mm&amp;quot;),&#xA;                 &#xA;                 #top_annotation_height=unit(0.5,&amp;quot;cm&amp;quot;),&#xA;                 #top_annotation=ColAnn,&#xA;                 top_annotation=colAnn)&#xA;&#xA;#bottom_annotation_height=unit(2, &amp;quot;cm&amp;quot;),&#xA;#bottom_annotation=boxplotCol)&#xA;&#xA;&#xA;draw(hmap1 ,heatmap_legend_side ='right',annotation_legend_side = 'right')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="285" LastEditorUserId="3848" LastEditDate="2020-12-08T13:18:16.397" LastActivityDate="2020-12-08T13:18:16.397" Title="Annotation label not matching to the color assigned in complex heatmap" Tags="|r|covid-19|sars-cov-2|complexheatmap|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13613" PostTypeId="2" ParentId="13596" CreationDate="2020-06-20T21:35:49.350" Score="1" Body="&lt;p&gt;Although there is no standard as DeepTools' Devon Ryan said, there are still some habits. Usually red means upregulated, as you can see in the default color scheme in MATLAB's clustergram function. But you also want to make sure color-blinded people can see the difference. So another color scheme might be better, such as red=high while blue=low.&#xA;Anyway, if you can easily flip the color, using red to represent high is expected.&lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2020-06-20T21:35:49.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13614" PostTypeId="2" ParentId="13603" CreationDate="2020-06-21T09:01:37.907" Score="3" Body="&lt;p&gt;This is the code to get a look-up table to convert between Ensembl ID and HGNC:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ensembl = useMart(&amp;quot;ensembl&amp;quot;,dataset=&amp;quot;hsapiens_gene_ensembl&amp;quot;)&#xA;theBM &amp;lt;- getBM(attributes=c('ensembl_gene_id','hgnc_symbol'), &#xA;               filters = c('ensembl_gene_id'),&#xA;               values = gsub(&amp;quot;\\..*&amp;quot;, &amp;quot;&amp;quot;, charg),&#xA;               mart = ensembl)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What Devon was posting is correct but misses a &lt;code&gt;c()&lt;/code&gt; around the attributes values.&#xA;For further help please provide the content of &lt;code&gt;resdata&lt;/code&gt; which you should always do when posting a question, since we cannot read minds. &lt;code&gt;Does not work&lt;/code&gt; by the way is not a proper error description.&lt;/p&gt;&#xA;&lt;p&gt;Once you have the output do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;resdata&lt;span class=&quot;math-container&quot;&gt;$genes &amp;lt;- gsub(&quot;\\..*&quot;, &quot;&quot;, resdata$&lt;/span&gt;genes)&#xA;&#xA;merge(x = theBM,&#xA;      by.x = &amp;quot;ensembl_gene_id&amp;quot;,&#xA;      y = resdata,&#xA;      by.y = &amp;quot;genes&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that I had to go to that SE crosspost to get the content of &lt;code&gt;resdata&lt;/code&gt;, this is not how this goes. Please post all relevant data up front in the future otherwise your questions might get downvoted and closed. Please also avoid cross-posting. if you provide proper information you usually get a good answer in time.&lt;/p&gt;&#xA;&lt;p&gt;Edit: Just realized you also cross-posted this to Biostars even twice. Please stop this. I closed the Biostars posts and gave my two cents on this behaviour over there.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2020-06-21T13:01:09.167" LastActivityDate="2020-06-21T13:01:09.167" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13615" PostTypeId="1" AcceptedAnswerId="13616" CreationDate="2020-06-21T10:32:25.347" Score="2" ViewCount="1642" Body="&lt;p&gt;I have a single read (&lt;strong&gt;NOT paired&lt;/strong&gt;) that I need to pass through the workflow described in Beauclair et al. paper (free version here &lt;a href=&quot;https://rnajournal.cshlp.org/content/24/10/1285.long&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rnajournal.cshlp.org/content/24/10/1285.long&lt;/a&gt;) for identifying defective genomes using their DI-tector program.&lt;/p&gt;&#xA;&lt;p&gt;Here in Materials and methods the procedure is described as the following:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;The first step of the workflow consists of an alignment of the reads against the host genome (Fig. 2i).&#xA;This step aims at discarding reads that map to the host genome and may partially map to the viral genome after segmentation, and at reducing the working file size.&#xA;For example, MV and rMV-ΔV data sets were generated from total RNA samples of infected cells and contained mostly reads mapping the human genome (≈99%).&#xA;This step uses a combination of &lt;code&gt;bwa mem&lt;/code&gt; and &lt;code&gt;samtools&lt;/code&gt; view with the parameters &lt;code&gt;–bS –f4&lt;/code&gt;.&#xA;An additional step consists of an alignment of the reads against the viral genome of interest, in order to exclude reads perfectly mapping the viral genome.&#xA;Therefore, only unmapped reads are further analyzed.&#xA;Of note, clipped reads (i.e., CIGAR motif contain S or H) are also conserved. Some of these reads may map to viral genome recombination junctions that are present in DI genomes.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have already been suggested to use &lt;code&gt;bowtie2&lt;/code&gt; instead of &lt;code&gt;bwa&lt;/code&gt; but, firstly, the output is not clear for me and secondly, I would like to test the official protocol.&lt;/p&gt;&#xA;&lt;p&gt;Since the article suggests to use bwa and samtools on this very first step, that is what I have done so far:&lt;/p&gt;&#xA;&lt;ol start=&quot;0&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;(Optional) not sure if this is important but as suggested by someone I have transformed .fna --&amp;gt; .fa&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cp GCF_000001405.39_GRCh38.p13_genomic.fna GCF_000001405.39_GRCh38.p13_genomic.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Indexed human genome with bwa&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bwa index GCF_000001405.39_GRCh38.p13_genomic.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;aligned my only read with indexed genome from step 1&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bwa mem GCF_000001405.39_GRCh38.p13_genomic.fa /mnt/e/nastya/SLX066-02/B-dVMV-RIG-1/B-dVMV-RIG-1_ACAGTG_L008_R1_001.rc.fastq.gz &amp;gt; whole.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Transform &lt;code&gt;.sam&lt;/code&gt; --&amp;gt; &lt;code&gt;.bam&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -S -b whole.sam -o whole.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Separated unmapped reads (as it is recommended in Materials and Methods using &lt;code&gt;-f4&lt;/code&gt;)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -f4 whole.bam &amp;gt; sample.unmapped.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Converted unmapped reads into &lt;code&gt;.fastq&lt;/code&gt; format (since this is the format used by the software later)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools fastq sample.unmapped.sam &amp;gt; unmatched.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I wonder if these steps are correct?&lt;/p&gt;&#xA;" OwnerUserId="6948" LastEditorUserId="3848" LastEditDate="2020-06-21T18:58:12.080" LastActivityDate="2020-06-21T18:58:12.080" Title="How to extract unmatched reads using bwa and samtools?" Tags="|sequence-alignment|samtools|bwa|bowtie2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13616" PostTypeId="2" ParentId="13615" CreationDate="2020-06-21T11:34:09.703" Score="5" Body="&lt;p&gt;Yes, correct but overly many steps. There is no need to convert between bam and sam. &lt;code&gt;samtools&lt;/code&gt; can read from &lt;code&gt;stdin&lt;/code&gt; and handles both sam and bam and &lt;code&gt;samtools fastq&lt;/code&gt; can interpret flags, therefore one can shorten this to:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bwa mem (...options) | samtools view -o out.bam&#xA;samtools fastq -f 4 out.bam &amp;gt; unmatched.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-06-21T11:34:09.703" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13618" PostTypeId="2" ParentId="8767" CreationDate="2020-06-22T05:29:41.553" Score="0" Body="&lt;p&gt;Where are you running the software? In general and especially for servers and high performance clusters I would recommend to use a module system like &lt;a href=&quot;https://github.com/TACC/Lmod&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lmod&lt;/a&gt; in conjunction with &lt;a href=&quot;https://github.com/easybuilders/easybuild&quot; rel=&quot;nofollow noreferrer&quot;&gt;EasyBuild&lt;/a&gt;, which contains configs for &lt;a href=&quot;https://github.com/easybuilders/easybuild-easyconfigs/blob/master/easybuild/easyconfigs/b/bcl2fastq2&quot; rel=&quot;nofollow noreferrer&quot;&gt;bcl2fastq2&lt;/a&gt; that take care of the installation. A module system allows you to use different versions and to keep your environment clean among other benefits.&lt;/p&gt;&#xA;" OwnerUserId="858" LastActivityDate="2020-06-22T05:29:41.553" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13620" PostTypeId="2" ParentId="13574" CreationDate="2020-06-22T16:24:48.280" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Bizarrely, Obabel seems to have written a PDB that was incorrectly spaced for the OP as seen in &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/13586/error-pasing-the-following-line-in-pdb/13595&quot;&gt;Error pasing the following line in pdb&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If you can use python, &lt;code&gt;rdkit&lt;/code&gt; is a great package for cheminformatics —best installed via conda though.&lt;/p&gt;&#xA;&lt;p&gt;It allows you do all sorts of things, such as match substructures, embed conformers, edit compounds, compute advanced parameters and many forms of shape &amp;amp; colour finger-/foot-printings.&lt;/p&gt;&#xA;&lt;p&gt;Given a multi-compound sdf you can read each individual molecule and write them out as PDBs.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from rdkit import Chem&#xA;with i, mol in enumerate(Chem.SDMolSupplier('in.sdf')):&#xA;    Chem.MolToPDBFile(f'out_{i}.pdb')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We can be fancier with the output name&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from rdkit import Chem&#xA;import re&#xA;with mol in Chem.SDMolSupplier('in.sdf'):&#xA;    # The headerline of a mol file is retrieved oddly.&#xA;    name = mol.GetProp('_Name')&#xA;    # make filename safe&#xA;    slugify = lambda name: re.sub(r'[\W_.-]+', '-', name)&#xA;    Chem.MolToPDBFile(f'{slugify(name)}.pdb')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To specify what name (&lt;em&gt;e.g.&lt;/em&gt; &lt;code&gt;CA&lt;/code&gt;) each atom gets and what is the residue name, chain and residue position —information absent in the sdf, it gets a bit odd.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  names = [' CA ', ' CB ', ' CG ', ' CD '] # say&#xA;  for i, atom in enumerate(mol.GetAtoms()):&#xA;        info = Chem.AtomPDBResidueInfo(names[i], # or whatever.&#xA;                                       residueName='LIG')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For more see &lt;a href=&quot;http://blog.matteoferla.com/2020/03/atom-names-purely-in-rdkit.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog post&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-06-22T16:24:48.280" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13621" PostTypeId="1" CreationDate="2020-06-22T17:13:31.263" Score="0" ViewCount="51" Body="&lt;p&gt;I have been trying to convert a multi entry sdf file to pdb using openbabel. Everytime I check the Generate 3D coordinates box I end up with .ENT files instead of .PDB files. What am I doing wrong.&lt;/p&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="9040" LastActivityDate="2020-11-19T21:04:35.373" Title="Generating 3D coordinates error" Tags="|docking|3d-structure|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13622" PostTypeId="2" ParentId="13561" CreationDate="2020-06-22T19:41:30.067" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/13108/how-to-make-venn-diagram-for-the-macs-peak-calling-output-of-two-data-sets/13407#13407&quot;&gt;In a similar question&lt;/a&gt;, I recommended that directly comparing called peaks can be somewhat misleading. This topic is addressed in the documentation of the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/DiffBind.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;DiffBind&lt;/code&gt; R package on Bioconductor&lt;/a&gt;, in the &amp;quot;Comparison of occupancy and affinity based analyses&amp;quot; section.&lt;/p&gt;&#xA;&lt;p&gt;But if you're just looking for genome annotation software, the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/annotatr.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;annotatr&lt;/code&gt;&lt;/a&gt; package on Bioconductor is pretty useful&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2020-06-22T19:41:30.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13623" PostTypeId="2" ParentId="13621" CreationDate="2020-06-22T20:23:43.100" Score="2" Body="&lt;p&gt;An ent file is a pdb file. Just change the extension. &lt;a href=&quot;https://www.ebi.ac.uk/pdbe/node/1&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDBe&lt;/a&gt; for example provides their PDB files as such.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-06-22T20:23:43.100" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13624" PostTypeId="1" CreationDate="2020-06-22T21:58:44.790" Score="2" ViewCount="144" Body="&lt;p&gt;If I’m starting with Deseq2 normalized counts what are some preprocessing steps that I should apply to these data before estimating sample correlation using the cor function in R? For example, would it make sense to quantile normalize the data?&lt;/p&gt;&#xA;" OwnerUserId="9061" LastActivityDate="2020-06-23T12:50:38.593" Title="RNA-Seq data transformation prior to sample correlation analysis" Tags="|r|rna-seq|correlation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13625" PostTypeId="1" CreationDate="2020-06-22T22:41:37.507" Score="1" ViewCount="460" Body="&lt;p&gt;I have a script in R/RStudio which creates random datasets of binomial variables, feeds them through a neural network, and calculates their likelihood ratio statistic and deviance. I'd like the script to loop with the &lt;code&gt;seed&lt;/code&gt; incremented by 1, and collect the value from each run into a data frame, from which the values get averaged. For example, if the script returns &lt;code&gt;5, 10, and 15&lt;/code&gt; on the first, second, and third iterations of the loop, I'd like the script to create a data frame with those values and then calculate their mean.&lt;/p&gt;&#xA;&lt;p&gt;Here is my script, annotated:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(neuralnet)&#xA;&#xA;x=1021&#xA;&#xA;#change nrow to number of times code is looped&#xA;result &amp;lt;- data.frame(matrix(nrow = 1000, ncol = 2)) &#xA;colnames(result) &amp;lt;- c(&amp;quot;LogL&amp;quot;, &amp;quot;D&amp;quot;)&#xA;&#xA;repeat{&#xA;&#xA;set.seed(x)&#xA;&#xA;#2 inputs, max number of terms 3&#xA;x1=rbinom(2000,1,0.5)&#xA;x2=rbinom(2000,1,0.5)&#xA;y=rbinom(2000,1,0.5)&#xA;&#xA;data=data.frame(x1,x2,y)&#xA;&#xA;head(data)&#xA;&#xA;#hidden units = 2&#xA;nnet=neuralnet(y~x1+x2,data,hidden=2,err.fct = &amp;quot;ce&amp;quot;,linear.output = F,likelihood = T)&#xA;&#xA;predictions=compute(nnet,data[,1:2])&#xA;&#xA;predictions$net.result&#xA;&#xA;#likelihood ratio stat&#xA;argument=y*log(predictions&lt;span class=&quot;math-container&quot;&gt;$net.result)+(1-y)*log(1-predictions$&lt;/span&gt;net.result)&#xA;&#xA;head(argument)&#xA;&#xA;#sum of likelihood ratio stat&#xA;LogL=sum(argument)&#xA;&#xA;LogL&#xA;&#xA;D=-2*LogL&#xA;&#xA;D&#xA;&#xA;result[x-1022, 1] &amp;lt;- LogL&#xA;result[x-1022, 2] &amp;lt;- D&#xA;&#xA;result&#xA;&#xA;#results &amp;lt;- data.frame(LogL, D)&#xA;&#xA;x=x+1&#xA;if (x==2021) {&#xA;  break&#xA;}&#xA;&#xA;print(x)&#xA;&#xA;&#xA;}&#xA;&#xA;result&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My repeat loop works, but a thousand iterations, for example, takes multiple minutes. Is there a more efficient way to code such a loop?&lt;/p&gt;&#xA;" OwnerUserId="9056" LastEditorUserId="9056" LastEditDate="2020-06-22T23:48:16.797" LastActivityDate="2020-06-23T19:08:05.120" Title="How do I set a neural network to loop multiple times and average the resulting values?" Tags="|r|statistics|machine-learning|loop|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13626" PostTypeId="2" ParentId="13625" CreationDate="2020-06-23T01:09:06.760" Score="1" Body="&lt;p&gt;The concern with your approach is called 'leakage' because you are parameterising the same data set that you are training. This can easily lead to overtraining is in ANN (artificial neural networks) and is not cool. Overtraining is where the ANN very tightly wraps around the training data set that it performs poorly on any other data set. In other words it is overtightened.&lt;/p&gt;&#xA;&lt;p&gt;The way they get around it is to split the data set usually 60:20:20 into a training split, parameterisation (validation) split and test (testing) split. If the parameterisation split gets mixed with the training set - leakage happens and overtraining results. Normally the data is trained, and the trained model parameters are assessed/optimised in the parameter split. The trained data and optimised parameters are then tested in the test split.&lt;/p&gt;&#xA;&lt;p&gt;As long as you are aware of the approach you are using, that I guess that is ok. If you separate your training and parameterisation (validation) data that would be cool.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-06-23T19:08:05.120" LastActivityDate="2020-06-23T19:08:05.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13627" PostTypeId="1" AcceptedAnswerId="13630" CreationDate="2020-06-23T03:23:58.097" Score="4" ViewCount="406" Body="&lt;p&gt;We have a complicated experimental design that we would like to perform LRT analysis for. Our main goal is to discover significant genes for the &amp;quot;Injection:Social&amp;quot; interaction term across the entire dataset by removing it from the LRT reduced model, and as a bonus we are also interested in discovering significant genes for that interaction term for each respective brain region.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Sample  Injection   Social  Region  Individual  ind.n&#xA;HY06    L   ISO HY  S06 S1&#xA;NST6    L   ISO NS  S06 S1&#xA;TN06    L   ISO TN  S06 S1&#xA;HY08    L   ISO HY  S08 S2&#xA;NST8    L   ISO NS  S08 S2&#xA;TN08    L   ISO TN  S08 S2&#xA;HY30    L   KF  HY  S30 S1&#xA;NST30   L   KF  NS  S30 S1&#xA;TN30    L   KF  TN  S30 S1&#xA;HY32    L   KF  HY  S32 S2&#xA;NST32   L   KF  NS  S32 S2&#xA;TN32    L   KF  TN  S32 S2&#xA;HY64    L   KFC HY  S64 S1&#xA;NST64   L   KFC NS  S64 S1&#xA;TN64    L   KFC TN  S64 S1&#xA;HY65    L   KFC HY  S65 S2&#xA;NST65   L   KFC NS  S65 S2&#xA;TN65    L   KFC TN  S65 S2&#xA;HY19    L   NF  HY  S19 S1&#xA;NST19   L   NF  NS  S19 S1&#xA;TN19    L   NF  TN  S19 S1&#xA;HY24    L   NF  HY  S24 S2&#xA;NST24   L   NF  NS  S24 S2&#xA;TN24    L   NF  TN  S24 S2&#xA;HY05    S   ISO HY  S05 S1&#xA;NST5    S   ISO NS  S05 S1&#xA;TN05    S   ISO TN  S05 S1&#xA;HY12    S   ISO HY  S12 S2&#xA;NST12   S   ISO NS  S12 S2&#xA;TN12    S   ISO TN  S12 S2&#xA;HY31    S   KF  HY  S31 S1&#xA;NST31   S   KF  NS  S31 S1&#xA;TN31    S   KF  TN  S31 S1&#xA;HY34    S   KF  HY  S34 S2&#xA;NST34   S   KF  NS  S34 S2&#xA;TN34    S   KF  TN  S34 S2&#xA;HY62    S   KFC HY  S62 S1&#xA;NST62   S   KFC NS  S62 S1&#xA;TN62    S   KFC TN  S62 S1&#xA;HY63    S   KFC HY  S63 S2&#xA;NST63   S   KFC NS  S63 S2&#xA;TN63    S   KFC TN  S63 S2&#xA;HY04    S   NF  HY  S04 S1&#xA;NST4    S   NF  NS  S04 S1&#xA;TN04    S   NF  TN  S04 S1&#xA;HY20    S   NF  HY  S20 S2&#xA;NST20   S   NF  NS  S20 S2&#xA;TN20    S   NF  TN  S20 S2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My first attempt was building simple full (m1) and reduced (m2) models that gets directly at our question of interest but doesn't control for nested individuals.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;m1 &amp;lt;- model.matrix(~ Region + Social * Injection, colData_filt)&#xA;m2 &amp;lt;- model.matrix(~ Region + Social + Injection, colData_filt)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We want to control for individual/batch effects, which is nested within both &amp;quot;Injection&amp;quot; and &amp;quot;Social&amp;quot; but not region, as we have three brain regions per individual. I followed the example in the DESeq2 manual for creating a term (ind.n) distiguishing individuals nested within groups, but now I'm not sure how to create the full and reduced model given that I have one more level than the example.&lt;/p&gt;&#xA;&lt;p&gt;I've tried a really elaborate full model (m1) with the interaction term of interest (Injection:Social) removed for the reduced model (m2), but I'm not sure this is correct based on our design.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;m1 &amp;lt;- model.matrix(~ Injection + Injection:ind.n + Injection:Social + Injection:Region + Social + Social:ind.n + Social:Region + Region, colData_filt)&#xA;m2 &amp;lt;- model.matrix(~ Injection + Injection:ind.n + Injection:Region + Social + Social:ind.n + Social:Region + Region, colData_filt)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm assuming this is wrong, but even if this was by some miracle the correct formulation, would there be a way to extract genes that explain the &amp;quot;Injection:Social&amp;quot; interaction term for separate brain regions?&lt;/p&gt;&#xA;&lt;p&gt;As a work-around, I subsetted the data by region and ran three separate LRT analyses for each subset and compared the results. While this simplified the model to look like the first example above, I worry that we lose some power by ignoring the fact we have multiple brain region samples from single individuals across the dataset.&lt;/p&gt;&#xA;&lt;p&gt;Any guidance is much appreciated. Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="9069" LastEditorUserId="9069" LastEditDate="2020-06-23T17:15:18.773" LastActivityDate="2020-06-23T18:40:13.377" Title="How to design DESeq2 LRT model with individuals nested in 2 levels?" Tags="|deseq2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13628" PostTypeId="1" CreationDate="2020-06-23T11:40:12.243" Score="1" ViewCount="169" Body="&lt;p&gt;I would like to find the proportion of each cell-type in bulk RNA-Seq transcriptomics data. I am looking for some guidance on the following:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What are the state-of-the-art methods?&lt;/li&gt;&#xA;&lt;li&gt;What are their assumptions? (e.g. do they assume that cell-type expression signatures are available in advance?)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9072" LastActivityDate="2020-06-23T21:14:31.323" Title="What are the state-of-the-art cell-type RNA-Seq deconvolution methods?" Tags="|rna-seq|gene-expression|machine-learning|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13629" PostTypeId="2" ParentId="13624" CreationDate="2020-06-23T12:50:38.593" Score="4" Body="&lt;p&gt;You don't absolutely need to perform any further steps before calculating the correlation. However you would likely benefit from the following sort of workflow:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Transform your data with vst() or robust log (see the DESeq2 vignette) so genes are on a more or less similar scale.&lt;/li&gt;&#xA;&lt;li&gt;Select the top 300-500 most variable genes&lt;/li&gt;&#xA;&lt;li&gt;Use this subset with to compute sample-wise correlation.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This also what follows what you would find in the DESeq2 user guide, which I strongly encourage you to read.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-06-23T12:50:38.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13630" PostTypeId="2" ParentId="13627" CreationDate="2020-06-23T17:11:57.387" Score="4" Body="&lt;p&gt;From what I can gather, you want to account for the effect of individual, nested within region. That is, you want to see after accounting for these, is there a consistent effect for Injection:Social across all conditions.&lt;/p&gt;&#xA;&lt;p&gt;So you set up the model like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;m1 &amp;lt;- model.matrix(~ ind.n*Region + Injection + Social + Injection:Social,data=..)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The last term should be Injection:Region and you can just use the waldTest (default) in DESeq2 for this term.&lt;/p&gt;&#xA;&lt;p&gt;What does the terms do? &lt;code&gt;ind.n*Region&lt;/code&gt; is the equivalent of &lt;code&gt;ind.n + Region + ind.n:Region&lt;/code&gt; , and with this you effectively get an effect for every region in every individual.&lt;/p&gt;&#xA;&lt;p&gt;Why don't we need the &lt;code&gt;Injection:ind.n&lt;/code&gt; or &lt;code&gt;Social:ind.n&lt;/code&gt; or &lt;code&gt;Social:Region&lt;/code&gt;. These terms indicate the effect of Injection or Social can vary by individuals or regions. Most likely introducing too many parameters when you are interested in a common effect. Also you do not have the replicates or samples to distinguish this effect from region or other effects.&lt;/p&gt;&#xA;&lt;p&gt;Since you provided an example we can run DESeq2 and you can see how the results look like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mat = counts(makeExampleDESeqDataSet(n=1000,m=48))&#xA;dds = DESeqDataSetFromMatrix(mat,df,~ ind.n*Region + Injection + Social + Injection:Social)&#xA;&#xA;dds = DESeq(dds)&#xA;&#xA;resultsNames(dds)&#xA; [1] &amp;quot;Intercept&amp;quot;            &amp;quot;ind.n_S2_vs_S1&amp;quot;       &amp;quot;Region_NS_vs_HY&amp;quot;     &#xA; [4] &amp;quot;Region_TN_vs_HY&amp;quot;      &amp;quot;Injection_S_vs_L&amp;quot;     &amp;quot;Social_KF_vs_ISO&amp;quot;    &#xA; [7] &amp;quot;Social_KFC_vs_ISO&amp;quot;    &amp;quot;Social_NF_vs_ISO&amp;quot;     &amp;quot;ind.nS2.RegionNS&amp;quot;    &#xA;[10] &amp;quot;ind.nS2.RegionTN&amp;quot;     &amp;quot;InjectionS.SocialKF&amp;quot;  &amp;quot;InjectionS.SocialKFC&amp;quot;&#xA;[13] &amp;quot;InjectionS.SocialNF&amp;quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The terms you need are &amp;quot;InjectionS.SocialKF&amp;quot;,&amp;quot;InjectionS.SocialKFC&amp;quot;, &amp;quot;InjectionS.SocialNF&amp;quot;, and you can look at each of them:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head(results(dds,name=&amp;quot;InjectionS.SocialNF&amp;quot;))&#xA;log2 fold change (MLE): InjectionS.SocialNF &#xA;Wald test p-value: InjectionS.SocialNF &#xA;DataFrame with 6 rows and 6 columns&#xA;              baseMean     log2FoldChange             lfcSE               stat&#xA;             &amp;lt;numeric&amp;gt;          &amp;lt;numeric&amp;gt;         &amp;lt;numeric&amp;gt;          &amp;lt;numeric&amp;gt;&#xA;gene1  9.9811166787259   1.25304112986447 0.819806376919295   1.52845984752303&#xA;gene2 30.3449455820337 0.0329442893152027 0.705199688255367 0.0467162562092241&#xA;gene3 3.83223545055379    1.0281136369045  1.64095596190233  0.626533350543196&#xA;gene4  11.232305747171  0.595738624408923   0.8243883031544  0.722643227868976&#xA;gene5 6.70950627004097  0.756449993378065   1.0631622863378  0.711509430967263&#xA;gene6 26.1431134888287 -0.854784518963918 0.625714541243558  -1.36609342219393&#xA;                 pvalue              padj&#xA;              &amp;lt;numeric&amp;gt;         &amp;lt;numeric&amp;gt;&#xA;gene1 0.126398405431826 0.978671002658464&#xA;gene2 0.962739373909937 0.999897888026606&#xA;gene3 0.530965168963838 0.978671002658464&#xA;gene4 0.469899103018657 0.978671002658464&#xA;gene5 0.476768608734069 0.978671002658464&#xA;gene6 0.171909642630577 0.978671002658464&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As mentioned, you can do a LRT if you want to test all the Injection:Social term interactions terms at one go, that is, the null hypothesis is that all of them are zero:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dds = nbinomLRT(dds,reduced=~ ind.n*Region + Injection + Social)&#xA;results(dds)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Usually the individual terms are more intuitively than testing all of them are zero, but you might have a special need for this.&lt;/p&gt;&#xA;" OwnerUserId="5077" LastEditorUserId="5077" LastEditDate="2020-06-23T18:40:13.377" LastActivityDate="2020-06-23T18:40:13.377" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="13631" PostTypeId="2" ParentId="13628" CreationDate="2020-06-23T21:14:31.323" Score="1" Body="&lt;p&gt;I see a lot of people using xCell. There should also be papers systematically compare many different methods, which might interest you.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-06-23T21:14:31.323" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13632" PostTypeId="1" CreationDate="2020-06-24T00:24:59.570" Score="1" ViewCount="379" Body="&lt;p&gt;I'm working with primer analysis and design, and so far I've performing it in Excel but: a) I don't like Excel, b) I do it manually, c) I don't like Excel, d) it gets tedious when working with thousands of sequences and tens of primers. My analysis consists of establishing how many sequences have mismatches on each position of my primers, and also count how many mismatches each sequence has with the corresponding primers (see attached image). This last analysis I've been able to (partially) solve it by using ncbi-blast+ tool. However, &lt;strong&gt;I will find very helpful a program/script where you input your sequences and primers fasta files and retrieve the number of mismatches at every primer position. Does anyone know something that could help me achieve all this?&lt;/strong&gt; If it also has an output tab-separated file as the one attached that would be perfect for easy visualization of results. Suggestion/orientations are also grateful! I just want to know if there's something to help me out, otherwise I'll try writing a script to automatize my analysis. Thank you!&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oIc8h.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oIc8h.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9074" LastActivityDate="2020-06-24T07:01:50.107" Title="Do you know a program/script to count how many sequences have mismatches on each position of my primers?" Tags="|sequence-alignment|blast|sequence-analysis|primer|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13633" PostTypeId="2" ParentId="13632" CreationDate="2020-06-24T03:37:08.223" Score="1" Body="&lt;p&gt;I'm amazed at your tenacity in getting Excel to do what you want; that looks like many hours of work to count mismatches.&lt;/p&gt;&#xA;&lt;p&gt;The hammer I use at the moment for this sort of stuff is LAST, which will do &lt;em&gt;local&lt;/em&gt; alignments of sequences to references. If the number of mismatches are not too large (as would be expected for primer sequences), a local alignment will extend to the end of one of the sequences, and I expect that should work fine for your purposes.&lt;/p&gt;&#xA;&lt;p&gt;I'm going to demonstrate two options, one that can be done just using Excel and web services, and one that needs command-line access. Note that these are only two ways to fetch mismatch counts; you might want to do more than that (it's not clear from your question), and there are &lt;em&gt;many&lt;/em&gt; other ways. Also, this may not work for your particular circumstances, but I think it should be quick enough that it's worth a try:&lt;/p&gt;&#xA;&lt;h1&gt;Option 1 - Web-based LAST&lt;/h1&gt;&#xA;&lt;p&gt;In this, I use the &lt;a href=&quot;http://lastweb.cbrc.jp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Web version of LAST&lt;/a&gt; to generate alignments for the primers [as &amp;quot;reference&amp;quot;] mapped to the sequences [as &amp;quot;query&amp;quot;].&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;To help create the mismatch (or in this case more correctly, &amp;quot;edit distance&amp;quot;) statistic, in the &lt;em&gt;Advanced Parameters&lt;/em&gt; section, set the &lt;code&gt;a: gap existence cost&lt;/code&gt; to 1, and the &lt;code&gt;b: gap extension cost&lt;/code&gt; to 1.&lt;/li&gt;&#xA;&lt;li&gt;Click on &lt;code&gt;Align!&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Once the Alignment Results appear, click on the &lt;code&gt;Alignments in tabular format&lt;/code&gt; link down the bottom, and save the file to your computer as &lt;code&gt;last.tab.csv&lt;/code&gt; (the &lt;code&gt;.csv&lt;/code&gt; at the end is the most important bit, to make sure that Excel recognises it as something it can open).&lt;/li&gt;&#xA;&lt;li&gt;Open that file in Excel, set the delimiter to &lt;code&gt;&amp;lt;tab&amp;gt;&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;The first column should be the alignment score, which in this case is the &lt;em&gt;match&lt;/em&gt; score. This can be converted into a &lt;em&gt;mismatch&lt;/em&gt; score by subtracting it from the primer sequence length (column 6).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h1&gt;Option 2 - Local LAST Installation&lt;/h1&gt;&#xA;&lt;p&gt;&lt;strong&gt;Installation&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Install &lt;a href=&quot;http://last.cbrc.jp/doc/last.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;LAST&lt;/a&gt; and &lt;a href=&quot;http://www.htslib.org/download/&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Index Generation&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The first step is to generate a reference index. LAST doesn't care too much about what's query and what's reference, but because I want all the genome sequences to appear on each line, the genome sequences will be the &lt;em&gt;query&lt;/em&gt;, and the primer sequences will be the &lt;em&gt;target&lt;/em&gt;. I use Martin Frith's &lt;a href=&quot;https://github.com/mcfrith/last-rna/blob/master/last-long-reads.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;lastdb mantra&lt;/a&gt; to do this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ lastdb -uNEAR -R01 primers.fa primers.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Mapping&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The next step is to do the actual alignment with LAST. I set the match and mismatch to be both 1 (i.e. the default for the Web version), set the gap existence and extension to be 1, change the output format to &lt;code&gt;tab&lt;/code&gt;, and save the result to &lt;code&gt;last.tab.csv&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ lastal -r 1 -q 1 -a 1 -b 1 primers.fa sequences.fa -f tab &amp;gt; last.tab.csv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Aggregating data&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Finally, it's a matter of aggregating the count information. I use &lt;code&gt;grep -v&lt;/code&gt; to exclude lines that begin with a &lt;code&gt;#&lt;/code&gt;, use &lt;code&gt;awk&lt;/code&gt; to subtract field 1 from field 6, then &lt;code&gt;sort&lt;/code&gt; the results and use &lt;code&gt;uniq -c&lt;/code&gt; to count the mismatches:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ grep -v '^#' last.tab.csv | awk '{print ($6 - $&lt;/span&gt;1)}' | sort | uniq -c&#xA;2118 0&#xA;   5 1&#xA;   5 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-06-24T07:01:50.107" LastActivityDate="2020-06-24T07:01:50.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13634" PostTypeId="5" CreationDate="2020-06-24T09:08:39.217" Score="0" Body="&lt;p&gt;This topic is part of the &lt;code&gt;computational biochemistry&lt;/code&gt; field and has a large overlap with &lt;code&gt;cheminformatics&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Predicting &lt;em&gt;in silico&lt;/em&gt; where a ligand binds within a protein structure or how two proteins bind to each other.&#xA;Common operations include creating 3D conformers from SMILES, parameterising the ligand, docking itself and scoring.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-06-28T14:41:36.877" LastActivityDate="2020-06-28T14:41:36.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13635" PostTypeId="4" CreationDate="2020-06-24T09:08:39.217" Score="0" Body="Predicting in silico where a ligand binds within a protein structure or how two proteins bind to each other.&#xD;&#xA;Common operations include creating 3D conformers from SMILES, parameterising the ligand, docking itself and scoring." OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-06-28T14:41:40.597" LastActivityDate="2020-06-28T14:41:40.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13637" PostTypeId="1" CreationDate="2020-06-25T09:59:43.613" Score="1" ViewCount="65" Body="&lt;p&gt;I have a file like this :&#xA;&lt;a href=&quot;https://i.stack.imgur.com/448hu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/448hu.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and I want the output as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jNzse.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jNzse.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I tried using awk :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN { OFS=FS=&amp;quot;\t&amp;quot; } { sub(&amp;quot;\\..*&amp;quot;, &amp;quot;&amp;quot;, $1); print }'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and it prints the first id ENST.&#xA;But when I change it to:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN { OFS=FS=&amp;quot;\t&amp;quot; } { sub(&amp;quot;\\..*&amp;quot;, &amp;quot;&amp;quot;, $5); print }'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;it doesnt work.&lt;/p&gt;&#xA;&lt;p&gt;What is wrong that I am doing here. Kindly help.&lt;/p&gt;&#xA;" OwnerUserId="7236" LastEditorUserId="7236" LastEditDate="2020-06-26T13:37:19.110" LastActivityDate="2020-06-26T22:07:52.900" Title="awk field separtor" Tags="|awk|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13638" PostTypeId="2" ParentId="13637" CreationDate="2020-06-25T11:18:20.363" Score="0" Body="&lt;p&gt;You could just do this in libreoffice calc using text to columns (data menu)&lt;/p&gt;&#xA;&lt;p&gt;If you want to use awk (or maybe gawk) you could use capture groups; but rather that mess around with a fairly complex regex I'd probably do it in two stages like below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk -F&amp;quot;|&amp;quot; '{print &lt;span class=&quot;math-container&quot;&gt;$5,$&lt;/span&gt;0}' OFS=&amp;quot;\t&amp;quot; your_file|\&#xA;awk -F&amp;quot;\t&amp;quot; '{print &lt;span class=&quot;math-container&quot;&gt;$1,substr($&lt;/span&gt;0,index(&lt;span class=&quot;math-container&quot;&gt;$0,$&lt;/span&gt;3))}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The second awk prints everything except field 2&lt;/p&gt;&#xA;" OwnerUserId="8919" LastActivityDate="2020-06-25T11:18:20.363" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13639" PostTypeId="1" CreationDate="2020-06-25T13:29:45.153" Score="0" ViewCount="229" Body="&lt;p&gt;I am looking for the DNA methylation, specifically inside small RNAs; piRNA. I am mapping BS-seq reads on the genome with &lt;a href=&quot;https://www.bioinformatics.babraham.ac.uk/projects/bismark/Bismark_User_Guide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;bismark&lt;/a&gt;. From the user guide of bismark, it maps reads uniquely on the genome, which mean it reports first read with best score and discard rest of the alignment and if a read mapping to multiple location with same score it will discard all such reads.&lt;/p&gt;&#xA;&lt;p&gt;Typical command for obtaining uniquely mapped reads from bismark&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;~/Software/Bismark-0.22.3/bismark GenomePATH/Original_Genome/  --bowtie2 -p 10 -1 Methylation_reads/Sample1_R1.fq.gz -2 Methylation_reads/Sample1_R2.fq.gz --bam -o Sample1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Since, I am interested in piRNA only which may have multiple source in the genome (one piRNA coded from different location of genome), I wanted to take all mapped locations in consideration. Can you suggest what changes I have to make in bismark command. After reading bismark user guide I am confused because I could not able to find relevant parameter.&lt;/p&gt;&#xA;&lt;p&gt;Also, please let me know if this approach is correct, if not what may be best approach for this.&lt;/p&gt;&#xA;" OwnerUserId="4492" LastActivityDate="2023-02-28T15:26:54.330" Title="Question: Bismark methylation for non-uniquely mapping BS-seq reads" Tags="|ngs|read-mapping|methylation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13640" PostTypeId="2" ParentId="8767" CreationDate="2020-06-25T13:32:19.867" Score="4" Body="&lt;p&gt;Install it with conda.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c dranew bcl2fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9081" LastActivityDate="2020-06-25T13:32:19.867" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13641" PostTypeId="2" ParentId="13412" CreationDate="2020-06-25T16:00:27.100" Score="0" Body="&lt;p&gt;What do you mean by having the same results but &lt;strong&gt;opposite&lt;/strong&gt; direction?&lt;/p&gt;&#xA;&lt;p&gt;What GSEA does is that it goes through the gene list from the top to bottom, whenever it encounters a gene that belongs to a gene set, it adds a positive score to that gene set, and if not, it penalizes the score. That is what you see in the green line in the results.&lt;/p&gt;&#xA;&lt;p&gt;Now imagine that you reverse the order of your input gene list, it is obvious the results are going to be different if the program still goes through the list from top to bottom.&lt;/p&gt;&#xA;&lt;p&gt;I think as a general rule, whenever you change the order of input gene list, results are going to be different.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-06-25T16:00:27.100" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13642" PostTypeId="2" ParentId="4849" CreationDate="2020-06-25T18:40:18.930" Score="2" Body="&lt;p&gt;You probably have red blood cell contamination.  It's probably better to identify which cells they are.  They'll probably cluster away from the cells you are interested in.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-06-25T18:40:18.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13643" PostTypeId="1" CreationDate="2020-06-25T23:04:50.213" Score="1" ViewCount="451" Body="&lt;p&gt;I am somewhat familiar with the pymol userface, but not very good with commands. I'm working with a pentameric subunit protein and I want to color specific residues on the surface of the protein, not just the cartoon. I can use select resi #-#, and color those with the (sele) tab on the right, but I do not know how to color the surface this way. When you select the sequence it will only color the cartoon. Any suggestions?&lt;/p&gt;&#xA;&lt;p&gt;I'm using PyMOL 2.3.5 on Windows.&lt;/p&gt;&#xA;" OwnerUserId="9084" LastEditorUserId="73" LastEditDate="2022-09-02T10:50:45.470" LastActivityDate="2023-05-30T17:08:05.980" Title="How do I color selective residues on the surface of a protein in pymol?" Tags="|pymol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13644" PostTypeId="2" ParentId="13637" CreationDate="2020-06-26T01:37:30.643" Score="2" Body="&lt;p&gt;Assuming you want to only show the fifth id from the first field, the following Perl script should work:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -pe 's/^(.*?)\t//; @F=split(&amp;quot;\\|&amp;quot;, $1); print(@F[4].&amp;quot;\t&amp;quot;)' input.tsv &amp;gt; output.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Explanation of the Perl code:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Loop over the lines in the input file (this is enabled by the command-line option &lt;code&gt;-p&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Remove everything up to the first tab character from the input line, and save the removed part (excluding tab character) into the grouping variable &lt;code&gt;$1&lt;/code&gt; [identified by brackets].&lt;/li&gt;&#xA;&lt;li&gt;Split variable &lt;code&gt;$1&lt;/code&gt; at the pipe character, &lt;code&gt;|&lt;/code&gt;, saving the split components into the array &lt;code&gt;@F&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Print the fifth field [0,1,2,3,&lt;strong&gt;4&lt;/strong&gt;] from the array &lt;code&gt;@F&lt;/code&gt; [without a line break at the end]&lt;/li&gt;&#xA;&lt;li&gt;Print the remainder of the input line (this is also enabled by the command-line option &lt;code&gt;-p&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-06-26T22:07:52.900" LastActivityDate="2020-06-26T22:07:52.900" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13645" PostTypeId="1" CreationDate="2020-06-26T10:06:52.193" Score="1" ViewCount="65" Body="&lt;p&gt;I'm trying to print out all measured distances between a ligand and 2 residues (using PyMol), below is the script that I'm using. I only want to measure the distances between all the ligand's O &amp;amp; ND2 of N312, as well as all the ligand's N &amp;amp; OD2 of D113.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from pymol import stored&#xA;stored.a, stored.b, stored.c, stored.d = [],[],[],[]&#xA;cmd.iterate_state(1, &amp;quot;r55_01///UNL`1/N*&amp;quot;, &amp;quot;stored.a.append(ID)&amp;quot;)&#xA;cmd.iterate_state(1, &amp;quot;r55_01///UNL`1/O*&amp;quot;, &amp;quot;stored.b.append(ID)&amp;quot;)&#xA;cmd.iterate_state(1, &amp;quot;r55_01//R/ASP`113/OD2&amp;quot;, &amp;quot;stored.c.append(ID)&amp;quot;)&#xA;cmd.iterate_state(1, &amp;quot;/r55_01//R/ASN`312/ND2&amp;quot;, &amp;quot;stored.d.append(ID)&amp;quot;)&#xA;&#xA;f=open(&amp;quot;dst4.txt&amp;quot;, 'w')&#xA;&#xA;for a in stored.a:&#xA;  for b in stored.b:&#xA;    for c in stored.c:&#xA;      for d in stored.d:&#xA;        f.write( &amp;quot;distance from %s to %s is %s\n&amp;quot; % (a, c,&#xA;cmd.distance( &amp;quot;id %s&amp;quot; % a, &amp;quot;id %s&amp;quot; % c,cutoff=5,mode=2)))&#xA;        f.write( &amp;quot;distance from %s to %s is %s\n&amp;quot; % (b, d,&#xA;cmd.distance( &amp;quot;id %s&amp;quot; % b, &amp;quot;id %s&amp;quot; % d,cutoff=5,mode=2)))&#xA;&#xA;f.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output file currently looks like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;distance from 9 to 1334 is 4.589359760284424&#xA;distance from 8 to 4204 is 3.1509475708007812&#xA;distance from 9 to 1334 is 4.589359760284424&#xA;distance from 15 to 4204 is 0.0&#xA;distance from 9 to 1334 is 4.589359760284424&#xA;distance from 17 to 4204 is 0.0&#xA;distance from 9 to 1334 is 4.589359760284424&#xA;distance from 20 to 4204 is 0.0&#xA;distance from 23 to 1334 is 0.0&#xA;distance from 8 to 4204 is 3.1509475708007812&#xA;distance from 23 to 1334 is 0.0&#xA;distance from 15 to 4204 is 0.0&#xA;distance from 23 to 1334 is 0.0&#xA;distance from 17 to 4204 is 0.0&#xA;distance from 23 to 1334 is 0.0&#xA;distance from 20 to 4204 is 0.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want my output to look more like this. Thanks in advance!&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Lig_N D113 4.59&#xA;Lig_O N312 3.15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9083" LastActivityDate="2020-06-28T10:47:21.783" Title="Modify how a script prints the output?" Tags="|python|pymol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13647" PostTypeId="2" ParentId="10628" CreationDate="2020-06-26T13:07:49.573" Score="1" Body="&lt;p&gt;I have been using dada2 and phyloseq (phangorn for trees) combination to analyze a V4 region 16s rRNA data set. I find their approach and explanations very understandable and usable since I prefer R.&lt;/p&gt;&#xA;&lt;p&gt;I was under the impression that even full-length 16s rRNA gene sequences were inadequate to classify up to species level since &lt;em&gt;closely related species within a genus can have identical 16S rRNA gene sequences&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;But, turns out, intra-genomic variants of this gene within a species or even strain may allow their unambiguous identification according to this paper:&lt;/p&gt;&#xA;&lt;p&gt;Johnson, J.S., Spakowicz, D.J., Hong, B. et al. Evaluation of 16S rRNA gene sequencing for species and strain-level microbiome analysis. Nat Commun 10, 5029 (2019). &lt;a href=&quot;https://doi.org/10.1038/s41467-019-13036-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1038/s41467-019-13036-1&lt;/a&gt;) available at: &lt;a href=&quot;https://rdcu.be/b5cgB&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rdcu.be/b5cgB&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Another relevant article regarding using black box ML algorithms (random forests) versus a straightforward logistic regression is discussed in this one:&lt;/p&gt;&#xA;&lt;p&gt;Topçuoğlu BD, Lesniak NA, Ruffin MT 4th, Wiens J, Schloss PD. A Framework for Effective Application of Machine Learning to Microbiome-Based Classification Problems. mBio. 2020;11(3):e00434-20. Published 2020 Jun 9. doi:10.1128/mBio.00434-20&lt;/p&gt;&#xA;&lt;p&gt;Since the OP has probably already moved on long since the original query, I am writing this to document relevant current references for myself and anyone researching this topic on StackExchange Bioinformatics.&lt;/p&gt;&#xA;" OwnerUserId="9088" LastActivityDate="2020-06-26T13:07:49.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13648" PostTypeId="2" ParentId="13639" CreationDate="2020-06-26T13:17:05.920" Score="1" Body="&lt;p&gt;I'm not certain about your piRNA approach, maybe others will be able to help with that question.&#xA;As for multi-mapped reads, from &lt;a href=&quot;https://github.com/FelixKrueger/Bismark/tree/master/Docs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bismark's documentation&lt;/a&gt; you can find the &lt;code&gt;--ambig-bam&lt;/code&gt; flag:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For reads that have multiple alignments a random alignment is written out to a special file ending in .ambiguous.bam. The alignments are in Bowtie2 format and do not any contain Bismark specific entries such as the methylation call etc. These ambiguous BAM files are intended to be used as coverage estimators for variant callers.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It doesn't look like Bismark will return all of the multiple alignments, just a randomly selected one for a given read or read pair (this probably results from the way Bowtie2 handles multiple alignments within Bismark).&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2020-06-26T13:17:05.920" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13649" PostTypeId="2" ParentId="13637" CreationDate="2020-06-26T13:49:52.723" Score="1" Body="&lt;p&gt;You need the first column's 6th component when split by &lt;code&gt;|&lt;/code&gt; and then the remaining columns of the file where &lt;code&gt;\t&lt;/code&gt; is the delimiter. Use awk.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk -F &amp;quot;\t&amp;quot; -vOFS=&amp;quot;\t&amp;quot; 'NR==1 {print} NR&amp;gt;1{split(&lt;span class=&quot;math-container&quot;&gt;$1,arr,&quot;|&quot;); print arr[6],$&lt;/span&gt;2,&lt;span class=&quot;math-container&quot;&gt;$3,$&lt;/span&gt;4,$5}' your_file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastActivityDate="2020-06-26T13:49:52.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13650" PostTypeId="1" CreationDate="2020-06-26T14:27:25.630" Score="3" ViewCount="787" Body="&lt;p&gt;How would one determine the significance of a variable in a glm model?&lt;/p&gt;&#xA;&lt;p&gt;If I, for example, have a dataframe like the one seen below, how would I determine if the origin of the sample has a significant effect on the value? (this is the number of enzymes capable of degrading the substrate of that matter)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Substrate    variable value origin&#xA;cellulose       M09    8    free&#xA;mannan          M12    2    free&#xA;glycogen        M65    2    free&#xA;chitin          M87    4    free&#xA;cellulose       M90    2    isolate&#xA;manan           M78    1    isolate&#xA;glycogen        M21    4    isolate&#xA;chitin          M21    1    isolate&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So far I have tried:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;mcomp = glm.nb(value ~ origin, data = my_data)&#xA;&#xA;summary(mcomp)&#xA;Deviance Residuals: &#xA;    Min       1Q   Median       3Q      Max  &#xA;-0.9625  -0.9047  -0.9047   0.1212   3.5232  &#xA;&#xA;Coefficients:&#xA;              Estimate Std. Error z value Pr(&amp;gt;|z|)   &#xA;(Intercept)   -0.01657    0.06571  -0.252  0.80097   &#xA;originisolate -0.21911    0.08180  -2.679  0.00739 **&#xA;---&#xA;Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1&#xA;&#xA;(Dispersion parameter for Negative Binomial(0.3418) family taken to be 1)&#xA;&#xA;    Null deviance: 2053.5  on 2679  degrees of freedom&#xA;Residual deviance: 2046.3  on 2678  degrees of freedom&#xA;AIC: 6517.5&#xA;&#xA;Number of Fisher Scoring iterations: 1&#xA;&#xA;&#xA;              Theta:  0.3418 &#xA;          Std. Err.:  0.0186 &#xA;&#xA; 2 x log-likelihood:  -6511.4590 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So free becomes the intercept and then isolate if significantly different from that. Does this mean Origin has a significant effect on the value?&lt;/p&gt;&#xA;&lt;p&gt;Would the better approach be to do the following?:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;mcomp = glm.nb(value ~ origin + Substrate, data = comb_data) &#xA;summary(aov(mcomp))&#xA;              Df Sum Sq Mean Sq F value Pr(&amp;gt;F)    &#xA;origin         1     23   22.55   6.612 0.0102 *  &#xA;Substrate     44   1445   32.84   9.631 &amp;lt;2e-16 ***&#xA;Residuals   2634   8981    3.41                   &#xA;---&#xA;Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This shows me that origin and substrate have an effect on value if I understand correctly?&lt;/p&gt;&#xA;" OwnerUserId="7031" LastEditorUserId="650" LastEditDate="2024-03-19T15:11:55.943" LastActivityDate="2024-03-19T15:11:55.943" Title="Determining significance of a variable in a glm model" Tags="|modelling|linear-regression|statsmodels|log-likelihood|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13651" PostTypeId="1" CreationDate="2020-06-26T22:54:32.013" Score="0" ViewCount="122" Body="&lt;p&gt;I ran kraken2 on a 16s rRNA sample. I get two kinds of output, one like &lt;a href=&quot;https://drive.google.com/file/d/1uz_Ms9KBhJehIu9shzCFmbFoYy_albnY/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;&#xA;and the other one like &lt;a href=&quot;https://drive.google.com/file/d/19vpgWaelvGsnWb7oET5y4SysKb6n4_GV/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. So I have many files like this for each sample of my control and test group.&lt;/p&gt;&#xA;&lt;p&gt;How do I generate OTU table out of these outputs for phyloseq usage? Do I parse the taxa ID column of any of these and generate a frequency table of each taxa in sample to generate OTU?&lt;/p&gt;&#xA;&lt;p&gt;Any help or suggestion would be highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="73" LastEditDate="2022-09-02T10:49:17.157" LastActivityDate="2022-09-02T10:49:17.157" Title="kraken2 output to OTU table" Tags="|microbial-genomics|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13653" PostTypeId="1" CreationDate="2020-06-27T02:11:26.330" Score="0" ViewCount="222" Body="&lt;p&gt;I am pretty new to R. So I apologize for asking maybe a very basic question.&#xA;Let's say I have a fasta file with sequence below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;sequence_1&#xA;ACCTGC--A&#xA;&amp;gt;sequence_2&#xA;ACC-GCTTA&#xA;&amp;gt;sequence_3&#xA;ACCTGCTTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there a function or method to calculate entropy for each column in R?&#xA;I did some research and found a function that calculate entropy for Amino Acids but not for DNA nucleotides.&lt;/p&gt;&#xA;&lt;p&gt;I would like to have the output be stored in a vector or in a list.&lt;/p&gt;&#xA;&lt;p&gt;Thanks for the help in advance!&lt;/p&gt;&#xA;" OwnerUserId="9091" LastActivityDate="2020-06-27T02:11:26.330" Title="Calculate Entropy for DNA Multiple Sequence Alignment in R" Tags="|r|sequence-alignment|phylogenetics|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13654" PostTypeId="2" ParentId="13650" CreationDate="2020-06-27T12:06:33.487" Score="0" Body="&lt;p&gt;There is no better method, it's a matter of what you want to test or what is your question.&lt;/p&gt;&#xA;&lt;p&gt;Using the &lt;code&gt;anova()&lt;/code&gt; or &lt;code&gt;aov()&lt;/code&gt;, test the terms collectively. For example, in your example with Substrate, the null hypothesis is that the coefficients are all zero, meaning &lt;code&gt;cellulose =0, mannan =0 , ....&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;If the question is, &amp;quot;do the isolate samples have a higher value than origin samples?&amp;quot;, then you can use your first model, where &lt;code&gt;free&lt;/code&gt; is set as the reference and you test whether the effect of &lt;code&gt;isolate&lt;/code&gt; is non-zero. Likewise you can do this for substrate and set of them as your reference. You can also do other pairwise comparisons using this model.&lt;/p&gt;&#xA;&lt;p&gt;If the question is, &amp;quot;does origin have a significant effect on value, after controlling for substrate?&amp;quot;, then you can use your second model.&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-06-27T12:06:33.487" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13655" PostTypeId="2" ParentId="13650" CreationDate="2020-06-27T12:46:40.800" Score="1" Body="&lt;p&gt;Second viewing of the question from what I can see -0.22 as a coefficient of origin is a strong negative association, so yeah it has a major impact. Its not how I would have done it, but that looks to be the result.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;First viewing,&lt;/p&gt;&#xA;&lt;p&gt;I'm going to throw my hat in here. We don't know what 'origin' is about, anyway just throw everything, i.e. each substrate and the origin into the same regression calculation. Check for a low-residual and preferably do a Q-Q plot, transform your data it this doesn't look good.&lt;/p&gt;&#xA;&lt;p&gt;The key and the thing you are missing is your regression weights, without that I couldn't say very much. If the regression weight is near zero for 'origin' then it has zero impact. If the regression weight of 'origin' is positively greater than everything else ... I assume there are skewed distributions of 'substrates' between the 'origins'. If the regression weight of 'origin' is negative but still greater than all other regression weights then it is adversely affecting the 'value' you are seeking.&lt;/p&gt;&#xA;&lt;p&gt;I don't know the experiment, the biological system or really the 'substrate' assays, so I can't comment any further.&lt;/p&gt;&#xA;&lt;p&gt;The two issues I have are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Doing an ANOVA on the output of a regression analysis doesn't make much sense to me. It is not something I would do, nor something in ML or GLM I've encountered.&lt;/li&gt;&#xA;&lt;li&gt;Are you doing pairwise substrate/origin calculations? I presue not, but just in case this not how GLM works.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-06-29T00:35:18.887" LastActivityDate="2020-06-29T00:35:18.887" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13656" PostTypeId="1" AcceptedAnswerId="13711" CreationDate="2020-06-27T13:58:02.430" Score="1" ViewCount="796" Body="&lt;p&gt;I routinely use AlignIO in BioPython for manipulating alignments and in context here moving from fasta format to phylip format. Alot of phylogeny packages accept a relaxed (sequence ID) &amp;amp; sequential (sequence) phylip format. This format is user-friendly, helps double-check for alignment errors and can be the essential input format.&lt;/p&gt;&#xA;&lt;p&gt;A typical bit of code is,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;'''&#xA;Converts a fasta file to phylip format&#xA;'''&#xA;from Bio import AlignIO&#xA;&#xA;Class Convert (inheritClass):&#xA;   &#xA;    def fa2phy (self):&#xA;        fileOut, fileIn = self._inOut() # builds the path name and deletes any pre-exising outfile&#xA;        align = AlignIO.read(fileIn, &amp;quot;fasta&amp;quot;) # reads the alignment in fasta&#xA;        AlignIO.write(align, fileOut, &amp;quot;phylip-relaxed&amp;quot;) # writes in phylip&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output of &amp;quot;phylip-relaxed&amp;quot; is interleaved format&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;4 120&#xA;virusA_detailed_description ACGTGACG .... ATCCCCAA&#xA;virusB_detailed_description ACGTGACG .... ATCGGGAA&#xA;virusC_detailed_description ACGTGACG .... ATCAAAAA&#xA;virusD_detailed_description ACGTGACG .... ATTTTTAA&#xA;&#xA;ACGTGACG .... ATCCCCAA&#xA;ACGTGACG .... ATCGGGAA&#xA;ACGTGACG .... ATCAAAAA&#xA;ACGTGACG .... ATTTTTAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output sought is relaxed-sequential&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;4 120&#xA;virusA_detailed_description ACGTGACG .... ATCCCCAAACGTGACG .... ATCCCCAA&#xA;virusB_detailed_description ACGTGACG .... ATCGGGAAACGTGACG .... ATCGGGAA&#xA;virusC_detailed_description ACGTGACG .... ATCAAAAAACGTGACG .... ATCAAAAA&#xA;virusD_detailed_description ACGTGACG .... ATTTTTAAACGTGACG .... ATTTTTAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There is an option to allow a sequential format (above). HOWEVER, a &lt;code&gt;&amp;quot;phylip-sequential&amp;quot;&lt;/code&gt; option output is as follows (it truncates the sequence ID to 10 characters,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;4 120&#xA;virusA_det ACGTGACG .... ATCCCCAAACGTGACG .... ATCCCCAA&#xA;virusB_det ACGTGACG .... ATCGGGAAACGTGACG .... ATCGGGAA&#xA;virusC_det ACGTGACG .... ATCAAAAAACGTGACG .... ATCAAAAA&#xA;virusD_det ACGTGACG .... ATTTTTAAACGTGACG .... ATTTTTAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;No matter how I look through AlignIO options the above is the nearest format.  Could anyone point to a IO style solution?&#xA;I could write it, but that defeats the object (i.e. purpose not an instance of a method).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;em&gt;Note&lt;/em&gt; the &lt;code&gt;&amp;quot;....&amp;quot;&lt;/code&gt; in the outputs above are not a typical &amp;quot;sequence identity&amp;quot; symbols, it just means that total length of a line in an interleaved format is 60 nucleotides (it might be 80).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-06-28T17:27:09.887" LastActivityDate="2020-07-02T10:03:33.423" Title="Relaxed and sequential Phylip format conversion" Tags="|phylogenetics|python|format-conversion|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13657" PostTypeId="2" ParentId="13469" CreationDate="2020-06-27T21:20:49.040" Score="1" Body="&lt;p&gt;If nucleotides are acceptable in prosite format because I thought it was protein motifs, the most accurate would be,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A-T-[AT]-G-x-C-[AGC]-C-[GCA](0,1,3)-x-A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The key point is that,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;[GCA](0,1,3)&lt;/code&gt; of &lt;code&gt;x(0, 1, 3)&lt;/code&gt; means zero/ nucleotides (3 indels) or one/ G or C or A nucleotide, and 2 indels or three/ G or C or A nucleotides.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;x&lt;/code&gt; is any nucleotide i.e. A or G or T or C - it doesn't mean an indel. &lt;code&gt;x(0, 1)&lt;/code&gt; is the best representation of a single indel (you call them gaps), here meaning any nucleotide may or may not be present at a given site.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Example from Prosite documentation:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;code&gt;&amp;lt;A-x-[ST](2)-x(0,1)-V&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;This pattern, which must be in the N-terminal&#xA;of the sequence ('&amp;lt;'), is translated as:&lt;/p&gt;&#xA;&lt;p&gt;Ala-any-[Ser or Thr]-[Ser or Thr]-(any or none)-Val&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-06-28T00:03:35.327" LastActivityDate="2020-06-28T00:03:35.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13658" PostTypeId="2" ParentId="13645" CreationDate="2020-06-28T10:47:21.783" Score="1" Body="&lt;p&gt;Let's simplify...&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;cmd.distance(xx,xxx)&lt;/code&gt; accepts two selector strings and is fine with &lt;code&gt;and state 1&lt;/code&gt;. And best of all if you select multiple atoms it gives you the smallest distance!&lt;/p&gt;&#xA;&lt;p&gt;So all you need to do is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# measure&#xA;d_O = cmd.distance('UNL and element O', 'resi 312 and name ND2')&#xA;d_N = cmd.distance('UNL and element N', 'resi 113 and name OD2')&#xA;# write  &#xA;f=open(&amp;quot;dst4.txt&amp;quot;, 'w')&#xA;f.write(f'Lig_N D113 {d_N:.2f}\n')&#xA;f.write(f'Lig_O N312 {d_O:.2f}\n')&#xA;f.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you are using python 2 (if so do note pymol scripting is better in 3), then the second bit is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;f=open(&amp;quot;dst4.txt&amp;quot;, 'w')&#xA;f.write('Lig_N D113 %.2f\n' % (d_N,))&#xA;f.write(f'Lig_O N312 %.2f\n' % (d_O,))&#xA;f.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Unsolicited coding tips&lt;/h2&gt;&#xA;&lt;p&gt;A few unsolicited tips... Your original script is rather over-complicated and obfuscated and uncommented, so for better legibility why not do:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;cmd.iterate_state&lt;/code&gt; is always harder to follow than the &lt;code&gt;for atom in get_model('xxxx').atoms&lt;/code&gt; synthax&lt;/li&gt;&#xA;&lt;li&gt;unless you are stuck with python 2, use f-strings not the &lt;code&gt;%s&lt;/code&gt; sprintf style formatting&lt;/li&gt;&#xA;&lt;li&gt;the &lt;code&gt;/x/x/x/x&lt;/code&gt; selector synthax is never as readable as the longer&#xA;&lt;code&gt;resi&lt;/code&gt;/&lt;code&gt;resn&lt;/code&gt;/&lt;code&gt;name&lt;/code&gt; selection algebra.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Code golf is always tempting, but it can result in future-you getting stuck as happened here...&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-06-28T10:47:21.783" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13659" PostTypeId="1" AcceptedAnswerId="13660" CreationDate="2020-06-28T12:16:23.153" Score="0" ViewCount="46" Body="&lt;p&gt;I would like to work on a project that involves the prediction of protein-ligand binding kinetics. What might be the feature that is relevant for the prediction?&lt;/p&gt;&#xA;" OwnerUserId="7150" LastActivityDate="2020-06-29T08:10:29.220" Title="Is it possible to predict protein-ligand binding kinetics using machine learning?" Tags="|proteins|machine-learning|interactions|" AnswerCount="1" CommentCount="2" ClosedDate="2020-06-29T13:20:04.230" ContentLicense="CC BY-SA 4.0" />
  <row Id="13660" PostTypeId="2" ParentId="13659" CreationDate="2020-06-29T08:10:29.220" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;This is a comment to redirect to other answers, but go too long.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Docking is the &lt;em&gt;in silico&lt;/em&gt; prediction of where a ligand binds in a protein.&lt;/p&gt;&#xA;&lt;p&gt;There are several Q&amp;amp;As about docking here that are relevant, so please check out:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;machine learning, docking and the factors that affect it: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/12971/generate-ligands-candidates-based-on-protein-shape/12975#12975&quot;&gt;Generate ligands candidates based on protein shape&lt;/a&gt; —also worth a read the &lt;a href=&quot;https://academic.oup.com/bioinformatics/article-abstract/36/3/758/5554651&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;learning from the ligand&amp;quot; paper&lt;/a&gt; paper, which discusses why some ligand only parameters tell a lot about binding.&lt;/li&gt;&#xA;&lt;li&gt;correlation between docking scores and enzyme kinetics: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/7504/purpose-of-correlating-docking-score-with-catalytic-efficiencykcat-km/10593#10593&quot;&gt;Purpose of correlating docking score with catalytic efficiency(Kcat/km)&lt;/a&gt; —discusses a paper &lt;a href=&quot;https://www.nature.com/articles/ncomms12965&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/ncomms12965&lt;/a&gt; wherein a linear regression is done between docking scores and kcat/KM&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-06-29T08:10:29.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13661" PostTypeId="1" CreationDate="2020-06-29T09:40:57.173" Score="0" ViewCount="62" Body="&lt;p&gt;I have question regarding a project I have to do about a health student study. I cannot import the csv file via read.csv or read.csv2 despite having transformed all files to csv and being in the correct folder.&#xA;Here are the command lines:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;setwd (&amp;quot;C:/Users/fek/Desktop/Analyse i-Share&amp;quot;)&#xA;getwd()&#xA;TA&amp;lt;-read.csv2(file=Biobanque.csv, header=TRUE, sep=&amp;quot;,&amp;quot;, dec=&amp;quot;,&amp;quot;, fill=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And here are the results in the console window:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;setwd (&amp;quot;C:/Users/fek/Desktop/Analyse i-Share&amp;quot;)&#xA;&#xA;getwd()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;[1] &amp;quot;C:/Users/fek/Desktop/Analyse i-Share&amp;quot;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TA&amp;lt;-read.csv2(file=Biobanque.csv, header=TRUE, sep=&amp;quot;,&amp;quot;, dec=&amp;quot;,&amp;quot;, fill=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;[1] Error in read.table(file = file, header = header, sep = sep, quote = quote,  :&#xA;objet 'Biobanque.csv' introuvable&lt;/p&gt;&#xA;&lt;p&gt;To translate from french, it says it can't find the file!&lt;/p&gt;&#xA;" OwnerUserId="9100" LastEditorUserId="3694" LastEditDate="2020-07-01T09:13:45.427" LastActivityDate="2020-07-01T09:13:45.427" Title="R not being able to find the file despite being in correct directory" Tags="|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13662" PostTypeId="1" CreationDate="2020-06-29T09:58:57.417" Score="1" ViewCount="190" Body="&lt;p&gt;I have raw paired-end RNA-seq reads for two novel eukaryotic species. Some background: the reads represent a copepod (arthropod) species each. The mRNA for each read set was obtained by extracting mRNA from multiple individuals. I am not exactly sure what sequencer was used (some Illumina short read machine as far as I can tell; I am not directly involved in this project).&lt;/p&gt;&#xA;&lt;p&gt;I'd like to assemble these reads into transcriptomes &lt;em&gt;de novo&lt;/em&gt; as no reference genomes are available as far as I know; my hope is to assemble with &lt;code&gt;Trinity&lt;/code&gt; and then annotate with &lt;code&gt;Dammit&lt;/code&gt; using its default databases (and check for completeness w/ &lt;code&gt;BUSCO&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;However, it looks like something went wrong during the sequencing runs (or during sample preparation) because the sequence GC content plots from FastQC look like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aEL4k.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aEL4k.png&quot; alt=&quot;fastqc_gc_content_plot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here the curve with the tall peak is one sample, and the more flatter curve is the other. There are two closely related curves per sample because &lt;code&gt;FastQC&lt;/code&gt; apparently seems to have recognized each read (forward and reverse) as a separate sample.&lt;/p&gt;&#xA;&lt;p&gt;I first presumed that this is just rRNA contamination (I believe the mRNA was prepared via poly-A selection), and I filtered the reads against all of &lt;code&gt;sortmerna&lt;/code&gt;'s default databases (which I believe include sequences from &lt;code&gt;SILVA&lt;/code&gt; and &lt;code&gt;Rfam&lt;/code&gt; among others). Then I passed the &lt;code&gt;sortmerna&lt;/code&gt;'d read through &lt;code&gt;FastP&lt;/code&gt; to filter out adapters, paired read correction, and low complexity filtering (alongside whatever else &lt;code&gt;FastP&lt;/code&gt; does by default). I had ran &lt;code&gt;FastQC&lt;/code&gt; before &lt;code&gt;FastP&lt;/code&gt; to check if the rRNA filtering did anything and it hadn't, so I also asked &lt;code&gt;FastP&lt;/code&gt; to filter by length (&lt;code&gt;--length_required 151&lt;/code&gt;) since there were small amounts of reads with lengths shorter than this value which I thought could be the source of the additional peaks in the data.&lt;/p&gt;&#xA;&lt;p&gt;However, even after &lt;code&gt;sortmerna&lt;/code&gt; and &lt;code&gt;FastP&lt;/code&gt; together, nothing has really changed:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/De0qM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/De0qM.png&quot; alt=&quot;sortmerna_fastp_fastqc_gc_content_plot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;(There are only two traces/curves here instead of four because the paired reads were apparently recognized properly by FastQC now.)&lt;/p&gt;&#xA;&lt;p&gt;My plan was to run &lt;code&gt;Khmer&lt;/code&gt; to clean the reads a little bit more and then assemble them. I did do a quick assembly with &lt;code&gt;Trinity&lt;/code&gt; (with the raw reads) and the &lt;code&gt;BUSCO&lt;/code&gt; scores were pretty bad (only ~ 25% of the arthropod &lt;code&gt;BUSCO&lt;/code&gt; genes present in the assemblies). I was hoping I'd be able to get &lt;code&gt;Dammit&lt;/code&gt; to annotate whatever was left after the cleaning, despite the poor BUSCO prognosis.&lt;/p&gt;&#xA;&lt;p&gt;Now I'm not entirely sure what to do if I can't clean the reads to begin with, since preventing nonsensical contigs would be absolutely important for a decent annotation.&lt;/p&gt;&#xA;&lt;p&gt;So this leads me to my questions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Are these data even salvageable?&lt;/li&gt;&#xA;&lt;li&gt;Is there anything else I can do to clean the data?&lt;/li&gt;&#xA;&lt;li&gt;Is this even contamination to begin with that I'm seeing here? (I am presuming some kind of contamination because both samples share that peak at ca. 18% GC content, among other shared features.) Would mapping all the reads against each other and discarding the shared sequences help? If yes, is there a tool I can use for this?&lt;/li&gt;&#xA;&lt;li&gt;Any other thoughts in this regard?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I searched through the posts here and on Biostars as well as I could but I wasn't really able to find anything helpful in this regard (hence my own post).&lt;/p&gt;&#xA;&lt;p&gt;I'm very new to all of this, so any inputs, tips, advice, and/or guidance would be most appreciated.&lt;/p&gt;&#xA;" OwnerUserId="9070" LastActivityDate="2020-07-01T04:35:16.403" Title="Decontaminating RNA seqs for de novo transcriptome assembly and annotation of novel eukaryotes" Tags="|rna-seq|sequence-annotation|assembly|transcriptome|quality-control|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13663" PostTypeId="1" AcceptedAnswerId="13664" CreationDate="2020-06-29T11:25:15.883" Score="3" ViewCount="332" Body="&lt;p&gt;I've assembled RNAseq data into a transcriptoms using Trinity. There's a option to keep only the longest isoforms for each transcripts and it lead me to wonder how it deals with duplicated genes (--&amp;gt;transcripts). I haven't found an anwser in the original paper. Althought I reckon paralogs and isoforms are two distinct features I don't see a way to make a distinction when you only have trasncripts.&lt;/p&gt;&#xA;" OwnerUserId="6459" LastActivityDate="2020-06-30T02:03:19.617" Title="Difference between isoforms and paralogs in transcriptoms?" Tags="|rna-seq|transcriptome|isoform|paralogs|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13664" PostTypeId="2" ParentId="13663" CreationDate="2020-06-29T11:52:24.307" Score="1" Body="&lt;p&gt;It depends. If you are performing Illumina based RNAseq, which is likely, it is definitely tricky, You should be able to do this in an ideal world because,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;a paralogue will contain SNPs, i.e. there will be nucleotide variation between paralogues, but the mRNA can of an identical length&lt;/li&gt;&#xA;&lt;li&gt;isoform will also have identical sequence but the length of the mRNA will vary depending on what initiation codon it used&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I sort of agree however because,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;there will be 'grey-zones' where paralogues are different sizes and VCF/ SNP calling is not perfect and ...&lt;/li&gt;&#xA;&lt;li&gt;assembly of paralogues can be difficult, risking collapsing a paralogue into a single mRNA, but it is not impossible depending on the level of SNP variation&lt;/li&gt;&#xA;&lt;li&gt;assembly of isoforms would be really hard&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I personally wouldn't know how point 3 is solvable, but I guess it is if your isoform is relatively small, i.e. well within the read-length of Illumina output.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-06-29T11:52:24.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13665" PostTypeId="1" AcceptedAnswerId="13669" CreationDate="2020-06-29T12:49:09.297" Score="2" ViewCount="3620" Body="&lt;p&gt;I would like to do two things:&lt;/p&gt;&#xA;&lt;p&gt;(1) Obtain a list of all NCBI (Entrez) gene ID's.&lt;/p&gt;&#xA;&lt;p&gt;(2) Obtain the &amp;quot;Official Symbol&amp;quot;, &amp;quot;Official Full Name&amp;quot;, and &amp;quot;Also known as&amp;quot; fields from the NCBI gene pages for each of those genes.&lt;/p&gt;&#xA;&lt;p&gt;This information, since it is displayed on the NCBI website, must be located in some databases, but I have not had luck finding a tool that will allow me to extract this information.&lt;/p&gt;&#xA;" OwnerUserId="9101" LastActivityDate="2020-06-29T14:09:55.703" Title="How can I obtain a list of all NCBI gene ID's along with their full name, symbol, and also known as symbols?" Tags="|gene|phylogenetics|text-processing|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13666" PostTypeId="2" ParentId="13596" CreationDate="2020-06-29T12:59:22.327" Score="0" Body="&lt;p&gt;In the absence of any yet-identified official recommendations, I will update this answer with resources as I find them:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Example usage in tutorials:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ebi.ac.uk/training/online/course/functional-genomics-ii-common-technologies-and-data-analysis-methods/biological-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;EMBL Training: Red=Up, Black=NC, Green=Down&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Program Defaults:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://docs.patricbrc.org/user_guides/organisms_taxon/transcriptomics_gene_heatmap.html#:%7E:text=The%20default%20color%20scheme%20is,off%20setting%20are%20colored%20black.&quot; rel=&quot;nofollow noreferrer&quot;&gt;PATRIC Webiste: Red=Up, Black=NC, Green=Down&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Guides:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://blog.bioturing.com/2018/09/24/heatmap-color-scale/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bio Turing Blog: Dos and don’ts for a heatmap color scale&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://sebastianraschka.com/Articles/heatmaps_in_r.html#optional-choosing-custom-color-palettes-and-color-breaks&quot; rel=&quot;nofollow noreferrer&quot;&gt;sebastianraschka: A short tutorial for decent heat maps in R&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Tools (by year):&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;(-2020) Web, R, CLI: &lt;a href=&quot;https://software.broadinstitute.org/morpheus/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MORPHEUS: Versatile matrix visualization and analysis software&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;(2019) Web, R, CLI: &lt;a href=&quot;https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0204542&quot; rel=&quot;nofollow noreferrer&quot;&gt;NOJAH: NOt Just Another Heatmap for genome-wide cluster analysis&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;(2017) Web, Jupyter: &lt;a href=&quot;https://www.nature.com/articles/sdata2017151&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clustergrammer, a web-based heatmap visualization and analysis tool for high-dimensional biological data&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;(2017) Web, R: &lt;a href=&quot;https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0176334&quot; rel=&quot;nofollow noreferrer&quot;&gt;shinyheatmap: Ultra fast low memory heatmap web interface for big data genomics&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;(2016) R: &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/32/18/2847/1743594&quot; rel=&quot;nofollow noreferrer&quot;&gt;Complex heatmaps reveal patterns and correlations in multidimensional genomic data&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;(2016) Web: &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/27190236/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Heatmapper: Web-Enabled Heat Mapping for All&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;(&amp;lt;2016) GUI, Java, R: &lt;a href=&quot;https://software.broadinstitute.org/GENE-E/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GENE-E&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;(2015) R: &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-15-S10-P16&quot; rel=&quot;nofollow noreferrer&quot;&gt;Heatmap3: an improved heatmap package with more powerful and convenient features&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;(2011) GUI, CLI: &lt;a href=&quot;https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0019541&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gitools: Analysis and Visualisation of Genomic Data Using Interactive Heat-Maps&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7185" LastEditorUserId="7185" LastEditDate="2020-07-02T13:30:33.677" LastActivityDate="2020-07-02T13:30:33.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13667" PostTypeId="1" CreationDate="2020-06-29T13:20:49.260" Score="1" ViewCount="110" Body="&lt;p&gt;I'm trying to print out the measured distances for each .pdbqt file, and then write a new output file for each .pdbqt, using the original name of the .pdbqt file as the new .txt file name. Below is the script.&lt;/p&gt;&#xA;&lt;p&gt;I'm using python3 &amp;amp; I'm trying to do this via Linux CLI.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from pymol import stored&#xA;stored.a, stored.b, stored.c, stored.d = [],[],[],[]&#xA;cmd.iterate_state(1, &amp;quot;r55_01///UNL`1/N*&amp;quot;, &amp;quot;stored.a.append(ID)&amp;quot;)&#xA;cmd.iterate_state(1, &amp;quot;r55_01///UNL`1/O*&amp;quot;, &amp;quot;stored.b.append(ID)&amp;quot;)&#xA;cmd.iterate_state(1, &amp;quot;r55_01//R/ASP`113/OD2&amp;quot;, &amp;quot;stored.c.append(ID)&amp;quot;)&#xA;cmd.iterate_state(1, &amp;quot;r55_01//R/ASN`312/ND2&amp;quot;, &amp;quot;stored.d.append(ID)&amp;quot;)&#xA;&#xA;f=open(&amp;quot;tt.txt&amp;quot;, 'w')&#xA;&#xA;for a in stored.a:&#xA;  for c in stored.c:&#xA;      d1=cmd.distance(&amp;quot;id %s&amp;quot; % a, &amp;quot;id %s&amp;quot; % c,cutoff=5)&#xA;      f.write(&amp;quot;%s D113 %s\n&amp;quot; % (a,d1))&#xA;      print(d1)&#xA;&#xA;for b in stored.b:&#xA;  for d in stored.d:&#xA;      d2 = cmd.distance(&amp;quot;id %s&amp;quot; % b, &amp;quot;id %s&amp;quot; % d, cutoff=5)&#xA;      f.write(&amp;quot;%s N312 %s\n&amp;quot; % (b,d2))&#xA;      print(d2)&#xA;&#xA;f.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My folder looks something like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;.&#xA;├── dst6.txt&#xA;├── sp&#xA;│   ├── r55_01.pdbqt&#xA;│   ├── r55_02.pdbqt&#xA;│   ├── r55_03.pdbqt&#xA;│   ├── r55_04.pdbqt&#xA;│   ├── r55_05.pdbqt&#xA;│   ├── r55_06.pdbqt&#xA;│   ├── r55_07.pdbqt&#xA;│   ├── r55_08.pdbqt&#xA;│   ├── r55_09.pdbqt&#xA;│   ├── r55_10.pdbqt&#xA;│   ├── r56_01.pdbqt&#xA;│   ├── r56_02.pdbqt&#xA;│   ├── r56_03.pdbqt&#xA;│   ├── r56_04.pdbqt&#xA;│   ├── r56_05.pdbqt&#xA;│   ├── r56_06.pdbqt&#xA;│   ├── r56_07.pdbqt&#xA;│   ├── r56_08.pdbqt&#xA;│   ├── r56_09.pdbqt&#xA;│   └── r56_10.pdbqt&#xA;└── t6.py&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;THANKS IN ADVANCE!!&lt;/p&gt;&#xA;" OwnerUserId="9083" LastEditorUserId="9083" LastEditDate="2020-06-29T14:08:04.053" LastActivityDate="2020-06-30T19:41:47.913" Title="Multiple input &amp; output files?" Tags="|python|pymol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13668" PostTypeId="2" ParentId="13665" CreationDate="2020-06-29T13:25:34.927" Score="1" Body="&lt;p&gt;Go to Ensembl (&lt;a href=&quot;https://www.ensembl.org/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ensembl.org/index.html&lt;/a&gt;), click &lt;code&gt;BioMart&lt;/code&gt; on the menu at the top of the page. Select the dataset you want to use, then go to Attributes, select the feature of each gene you want to get. Then click &amp;quot;Results&amp;quot; button to make sure that the table looks like what you want. Then output the results to a file.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/y0A2l.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/y0A2l.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-06-29T13:25:34.927" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13669" PostTypeId="2" ParentId="13665" CreationDate="2020-06-29T14:09:55.703" Score="3" Body="&lt;p&gt;All of these data are in the &lt;code&gt;gene_info.gz&lt;/code&gt; file on the NCBI FTP site: &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/gene/DATA/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nlm.nih.gov/gene/DATA/&lt;/a&gt;&#xA;Specifically, you may want to look at the &lt;code&gt;gene_info.gz&lt;/code&gt; file. It has the following fields (shown for an example gene below):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;                              #tax_id [  1]: 9606&#xA;                               GeneID [  2]: 5768&#xA;                               Symbol [  3]: QSOX1&#xA;                             LocusTag [  4]: -&#xA;                             Synonyms [  5]: Q6|QSCN6&#xA;                              dbXrefs [  6]: MIM:603120|HGNC:HGNC:9756|Ensembl:ENSG00000116260&#xA;                           chromosome [  7]: 1&#xA;                         map_location [  8]: 1q25.2&#xA;                          description [  9]: quiescin sulfhydryl oxidase 1&#xA;                         type_of_gene [ 10]: protein-coding&#xA;   Symbol_from_nomenclature_authority [ 11]: QSOX1&#xA;Full_name_from_nomenclature_authority [ 12]: quiescin sulfhydryl oxidase 1&#xA;                  Nomenclature_status [ 13]: O&#xA;                   Other_designations [ 14]: sulfhydryl oxidase 1|quiescin Q6 sulfhydryl oxidase 1|testis tissue sperm-binding protein Li 62n|thiol oxidase 1&#xA;                    Modification_date [ 15]: 20200604&#xA;                         Feature_type [ 16]: -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you are starting  with a set of NCBI GeneIDs and want this sort of information for just those, you can use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez Direct&lt;/a&gt; for that. If you are starting with RefSeq transcript or protein accessions, be sure to check out the other files in the FTP path mentioned above.&lt;/p&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2020-06-29T14:09:55.703" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13670" PostTypeId="1" AcceptedAnswerId="15236" CreationDate="2020-06-29T14:35:16.353" Score="3" ViewCount="32" Body="&lt;p&gt;Where in an SBOL file could you specify the linkers that would be placed as prefixes / suffixes between parts for assembly? Lets say you have a final construct of your design and want to assemble the component parts - could you encode what linkers you used between each component, ie as an annotation? Would this be different between SBOL versions?&lt;/p&gt;&#xA;" OwnerUserId="9102" LastActivityDate="2021-02-18T04:04:10.487" Title="Can SBOL encode DNA Assembly linkers?" Tags="|assembly|sbol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13671" PostTypeId="1" CreationDate="2020-06-29T14:38:34.193" Score="0" ViewCount="136" Body="&lt;p&gt;I have downloaded a dataset from GEO and only the matrix (mx) files are available. How can I perform the analysis without the barcode and the gene features?&lt;/p&gt;&#xA;" OwnerUserId="9103" LastActivityDate="2020-06-29T14:38:34.193" Title="Seurat analysis only with matrix with no barcode and features" Tags="|seurat|single-cell|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13674" PostTypeId="1" AcceptedAnswerId="13677" CreationDate="2020-06-29T17:05:16.530" Score="1" ViewCount="79" Body="&lt;p&gt;I am going to screen a set of small molecules against a &lt;strong&gt;CYP450 2C9&lt;/strong&gt; protein target. The structure I am using exactly is &lt;a href=&quot;http://www.rcsb.org/structure/5A5I&quot; rel=&quot;nofollow noreferrer&quot;&gt;5A5I&lt;/a&gt;. Do I need to fill in missing side chains and loops before performing VLS? I assume while preparing the protein I should leave water molecules in the structure as there is a bond between &lt;span class=&quot;math-container&quot;&gt;$Fe^{2+}$&lt;/span&gt; and water that is involved in the binding of a ligand.&lt;/p&gt;&#xA;" OwnerUserId="7150" LastActivityDate="2020-06-29T19:10:29.407" Title="Do I need to fill in missing side chains and loops before protein-ligand docking for VLS?" Tags="|protein-structure|docking|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13676" PostTypeId="2" ParentId="13661" CreationDate="2020-06-29T17:54:36.117" Score="4" Body="&lt;p&gt;The problem is just what it says.  By omitting the quotes around &amp;quot;Biobanque.csv&amp;quot;, R thinks you are talking about an &lt;strong&gt;object whose name is Biobanque.csv&lt;/strong&gt;. And R is confused because of course you have not created an object by that name.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastEditorUserId="2388" LastEditDate="2020-06-29T20:28:28.813" LastActivityDate="2020-06-29T20:28:28.813" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13677" PostTypeId="2" ParentId="13674" CreationDate="2020-06-29T19:10:29.407" Score="2" Body="&lt;h1&gt;In general&lt;/h1&gt;&#xA;&lt;p&gt;Most if not all docking applications will tolerate a jump.&#xA;Furthermore, you really don't want to dock anything to the flexible regions even if you are using a program that does properly protein flexibility.&#xA;Furthermore, if your protein adopts a closed conformation upon native substrate binding (not really the case here), you should start with that for better docking results.&lt;/p&gt;&#xA;&lt;h1&gt;Cytochrome P450&lt;/h1&gt;&#xA;&lt;p&gt;So most Cytochrome P450s are strange though and in many those loops play a large part in their catalysis. I am on a paper under review (&lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2020.04.14.041590v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioarxiv&lt;/a&gt;) that deals exactly with this in a different cytochrome (P450-BM3, the biotech one). Namely even if the active site is fairly large for an active site in the core of an enzyme, the ligand (testosterone) did not really rotate around and instead the specificity of oxidation was controlled by how the ligand got in. So those crystallographically unresolved loops act as gatekeepers for what can bind. In terms of VLS, one aims for blocking off the cofactor, i.e. the heme. So you can do docking without worrying about them in a first instance, but there is a possibility that a lot of active drugs instead interact with the loops.&#xA;Certainly, dynamic undocking and other MD methods may be required a few iterations in, to see why some candidates failed and why others worked.&lt;/p&gt;&#xA;&lt;p&gt;An interesting unofficial further possible spin is the fact that solvent-exposed sequential residues Asp-Pro can hydrolyse under acidic conditions, in a mechanism similar to &lt;a href=&quot;https://en.wikipedia.org/wiki/Isoaspartate&quot; rel=&quot;nofollow noreferrer&quot;&gt;isoaspartate formation&lt;/a&gt;, and several cytochromes have these on the entrance loops and have weird kinetics and smeary SDS-PAGE gels.&lt;/p&gt;&#xA;&lt;p&gt;Lastly, you are totally correct in your observation. The water binding the heme in the crystal is very special. It is taking the place of oxygen. That could be modelled as is, changed to a pre-encounter O2, or event changing the heme to a heme bound to a radical oxygen —note that some forcefields (CHARMM, Rosetta Ref2015) do not like this weird valency iron.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-06-29T19:10:29.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13678" PostTypeId="1" AcceptedAnswerId="13680" CreationDate="2020-06-29T21:07:55.800" Score="0" ViewCount="99" Body="&lt;p&gt;I have a dataframe with putative alignments of thousands of probe sequences. I would like to pull out the adjacent nucleotide from the genome for each alignment. My dataframe includes &lt;code&gt;seq.name&lt;/code&gt; for the target chromosome and I've added an &lt;code&gt;adjacent_pos&lt;/code&gt; variable with the location on the target chromosome that I want. Using &lt;code&gt;read.fasta&lt;/code&gt; from the phylotools package gives me a genome in the format of a dataframe with &lt;code&gt;seq.name&lt;/code&gt; and &lt;code&gt;seq.text&lt;/code&gt; for each chromosome and unplaced scaffold.&lt;/p&gt;&#xA;&lt;p&gt;Working on a test data set with a single chromosome, I successfully used something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;adj &amp;lt;- cbind(alignments$adjacent_pos, alignments$adjacent_pos)&#xA;alignments$adjacent_base &amp;lt;- str_sub(genome$seq.text, adj)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now back to the whole genome, I can't wrap my head around how to direct &lt;code&gt;str_sub&lt;/code&gt; to the correct &lt;code&gt;seq.name&lt;/code&gt; for each &lt;code&gt;seq.position&lt;/code&gt;. I'm sure there's an elegant solution, but I haven't been able to find it!&lt;/p&gt;&#xA;" OwnerUserId="3505" LastActivityDate="2020-06-30T08:53:24.433" Title="how to extract a different substring for each row of a dataframe in R?" Tags="|r|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13679" PostTypeId="2" ParentId="13663" CreationDate="2020-06-30T02:03:19.617" Score="3" Body="&lt;p&gt;If a reference genome assembly exists, then reference-guided assembly with Trinity will keep paralogs, because it treats each genomic region as a different assembly set.&lt;/p&gt;&#xA;&lt;p&gt;If a reference genome doesn't exist, paralogs will probably result in the construction of non-representative &amp;quot;isoforms&amp;quot; containing variants present in multiple genes. Inchworm does a greedy assembly of reads, so if a chimeric transcript &lt;em&gt;can&lt;/em&gt; be constructed from the reads (which is likely with paralogous genes), then it might end up being fed to the subsequent steps. Whether that is actually emitted at the end by Butterfly will mostly depend on how the paths through the putative assemblies are traversed, which probably depends on the tolerance of Bowtie to errors.&lt;/p&gt;&#xA;&lt;p&gt;If paralogous transcripts are sufficiently similar that the differences are within the error tolerance for assembly, then I would expect that they'd be collapsed into a single transcript.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2020-06-30T02:03:19.617" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13680" PostTypeId="2" ParentId="13678" CreationDate="2020-06-30T08:53:24.433" Score="0" Body="&lt;p&gt;Say your file looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat test.fa&#xA;&amp;gt;chr01&#xA;ACCGGTT&#xA;&amp;gt;chr02&#xA;ATGCATGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Something like what you have done, to read in the fasta and extract the sequence. Key part is creating a GenomicRanges object:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(GenomicRanges)&#xA;library(BSgenome)&#xA;library(Biostrings)&#xA;&#xA;adj &amp;lt;- data.frame(chr=c(&amp;quot;chr01&amp;quot;,&amp;quot;chr02&amp;quot;),start=c(2,3),end=c(2,3))&#xA;&#xA;    chr start end&#xA;1 chr01     2   2&#xA;2 chr02     3   3&#xA;&#xA;adj = makeGRangesFromDataFrame(adj)&#xA;&#xA;fasta &amp;lt;- readDNAStringSet(&amp;quot;test.fa&amp;quot;)&#xA;SEQ = getSeq(fasta,makeGRangesFromDataFrame(adj))&#xA;&#xA;as.character(SEQ)&#xA;[1] &amp;quot;C&amp;quot; &amp;quot;G&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or if the fasta file is huge, you can use Rsamtools, indexing the fasta file first:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(Rsamtools)&#xA;fasta = FaFile(&amp;quot;test.fa&amp;quot;)&#xA;indexFa('test.fa')&#xA;&#xA;as.character(getSeq(FaFile(&amp;quot;test.fa&amp;quot;),adj))&#xA;chr01 chr02 &#xA;  &amp;quot;C&amp;quot;   &amp;quot;G&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-06-30T08:53:24.433" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13681" PostTypeId="1" AcceptedAnswerId="13683" CreationDate="2020-06-30T10:33:21.167" Score="2" ViewCount="79" Body="&lt;p&gt;I am removing illumina adapters of the NGS data with a loop. My NGS data is storage in /data/HTS_seq/.&#xA;I used this function:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; for infile in /data/HTS_seq/*_read1.fastq; do outfile=&lt;span class=&quot;math-container&quot;&gt;$infile\_trim.fastq cutadapt -a AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC -e 0.1 -O 3 -m 30 --max-n 2 -o $&lt;/span&gt;outfile $infile; done&#xA;&#xA; for infile in /data/HTS_seq/*_read2.fastq; do outfile=&lt;span class=&quot;math-container&quot;&gt;$infile\_trim.fastq cutadapt -a AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTA -e 0.1 -O 3 -m 30 --max-n 2 -o $&lt;/span&gt;outfile $infile; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and It gives to me this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  IOError: [Errno 20] Not a directory: '/data/HTS_seq/PV589_03664AAB_AGGACT_read1.fastq/_trim.fastq'&#xA;  Traceback (most recent call last):&#xA;  File &amp;quot;/usr/local/bin/cutadapt&amp;quot;, line 10, in &amp;lt;module&amp;gt;&#xA;cutadapt.main()&#xA;  File &amp;quot;/usr/local/lib/python2.7/dist-packages/cutadapt/scripts/cutadapt.py&amp;quot;, line 652, in main&#xA;options.discard_untrimmed)&#xA;  File &amp;quot;/usr/local/lib/python2.7/dist-packages/cutadapt/scripts/cutadapt.py&amp;quot;, line 327, in trimmed_and_untrimmed_files&#xA;trimmed = untrimmed = xopen(output_path, 'w')&#xA;  File &amp;quot;/usr/local/lib/python2.7/dist-packages/cutadapt/xopen.py&amp;quot;, line 178, in xopen&#xA;return open(filename, mode)&#xA;      IOError: [Errno 20] Not a directory: '/data/HTS_seq/PV589_03664AAB_AGGACT_read1.fastq/_trim.fastq'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which is my error?&#xA;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="7430" LastActivityDate="2020-06-30T15:48:14.667" Title="Error after trimming illumina adapters" Tags="|ngs|cutadapt|loop|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13682" PostTypeId="2" ParentId="13681" CreationDate="2020-06-30T12:18:56.133" Score="-1" Body="&lt;p&gt;There is a bug here&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/data/HTS_seq/*_read1.fastq &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will go to your root directory &lt;code&gt;/&lt;/code&gt; and there is definiately no folder called &lt;code&gt;data&lt;/code&gt; in the root. To prove it type &lt;code&gt;ls /&lt;/code&gt; and that is where the command is pointing, then compare this to &lt;code&gt;ls ./&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To solve it go to the directory &lt;code&gt;HTS_seq&lt;/code&gt; and type &lt;code&gt;pwd&lt;/code&gt; then copy and paste the directory structure. I suspect &lt;code&gt;./data/HTS_seq&lt;/code&gt; will solve it&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; for infile in ./data/HTS_seq; ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will look for the folder &lt;code&gt;data&lt;/code&gt; in your current working directory rather than the root directory.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Okay you've been assigned /data as root ... no biggie.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;PV589_03664AAB_AGGACT_read1.fastq/_trim.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Why is &lt;code&gt;/&lt;/code&gt; in the file name? What happens if you &lt;code&gt;ls /data/HTS_seq/PV589_03664AAB_AGGACT_read1.fastq/_trim.fastq&lt;/code&gt;?&#xA;and &lt;code&gt;ls /data/HTS_seq/&lt;/code&gt; Finally could you please &lt;code&gt;ls /data/HTS_seq/PV589_03664AAB_AGGACT_read1.fastq\/_trim.fastq&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;\/&lt;/code&gt; is important. What I think has happened is a &lt;code&gt;/&lt;/code&gt; is in the file name and that will cause problems because it will think that &lt;code&gt;_trim.fastq&lt;/code&gt; is a directory when its part of a file name, and would explain the error.&lt;/p&gt;&#xA;&lt;p&gt;Then do the same thing using &lt;code&gt;head&lt;/code&gt; instead of &lt;code&gt;ls&lt;/code&gt; see what happens.&lt;/p&gt;&#xA;&lt;p&gt;If the last &lt;code&gt;ls&lt;/code&gt; and &lt;code&gt;head&lt;/code&gt; works then rename the files by typing &lt;code&gt;rename&lt;/code&gt; in the command line. If &lt;code&gt;rename&lt;/code&gt; works it will be easy to remove &lt;code&gt;/&lt;/code&gt; from all your files using the power of the &lt;code&gt;rename&lt;/code&gt; function (its a Perl function BTW).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-06-30T14:16:39.270" LastActivityDate="2020-06-30T14:16:39.270" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13683" PostTypeId="2" ParentId="13681" CreationDate="2020-06-30T15:37:23.673" Score="1" Body="&lt;p&gt;First of all, you aren't actually setting the &lt;code&gt;$outfile&lt;/code&gt; variable. Your loop has this basic format:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for var in *; do newVar=&amp;quot;something&amp;quot; command $newVar; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ for var in 1 2 3 4; do newVar=&quot;new:$&lt;/span&gt;var&amp;quot; echo &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$var: $&lt;/span&gt;newVar&amp;quot;; done&#xA;1: &#xA;2: &#xA;3: &#xA;4: &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see, &lt;code&gt;newVar&lt;/code&gt; is never actually being set. That's because setting a variable is its own operation. So you need to do that, end it with a control character like &lt;code&gt;;&lt;/code&gt;, or &lt;code&gt;&amp;amp;&lt;/code&gt;, or a newline, and &lt;em&gt;then&lt;/em&gt; run your &lt;code&gt;cutadapt&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ for var in 1 2 3 4; do newVar=&quot;new:$&lt;/span&gt;var&amp;quot;; echo &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$var: $&lt;/span&gt;newVar&amp;quot;; done&#xA;1: new:1&#xA;2: new:2&#xA;3: new:3&#xA;4: new:4&#xA; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or, with your data:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for infile in /data/HTS_seq/*_read1.fastq; do &#xA;    outfile=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$infile&quot;_trim.fastq &#xA;    cutadapt -a AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC \ &#xA;      -e 0.1 -O 3 -m 30 --max-n 2 -o &quot;$&lt;/span&gt;outfile&amp;quot; &amp;quot;$infile&amp;quot;&#xA;done&#xA;&#xA;&#xA;for infile in /data/HTS_seq/*_read2.fastq; do &#xA;    outfile=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$infile&quot;_trim.fastq&#xA;    cutadapt -a AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTA \ &#xA;      -e 0.1 -O 3 -m 30 --max-n 2 -o $&lt;/span&gt;outfile $infile&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But that would give a weird file name, like &lt;code&gt;foo_S1_L001_read1.fastq_trim.fastq&lt;/code&gt;. You probably want names like &lt;code&gt;foo_S1_L001_read1_trim.fastq&lt;/code&gt; instead, so you would want:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for infile in /data/HTS_seq/*_read1.fastq; do &#xA;    outfile=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${infile%.fastq}&quot;_trim.fastq&#xA;    cutadapt -a AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC \ &#xA;      -e 0.1 -O 3 -m 30 --max-n 2 -o &quot;$&lt;/span&gt;outfile&amp;quot; &amp;quot;$infile&amp;quot;&#xA;done&#xA;&#xA;&#xA;for infile in /data/HTS_seq/*_read2.fastq; do &#xA;    outfile=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${infile%.fastq}&quot;_trim.fastq&#xA;    cutadapt -a AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTA \ &#xA;      -e 0.1 -O 3 -m 30 --max-n 2 -o $&lt;/span&gt;outfile $infile&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Now, the specific error you show makes me think that isn't the command you ran. Could you perhaps have run this one instead by mistake (note the &lt;code&gt;/&lt;/code&gt; instead of &lt;code&gt;\&lt;/code&gt; before the &lt;code&gt;_trim.fastq&lt;/code&gt;)?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for infile in /data/HTS_seq/*_read1.fastq; do&#xA; outfile=&lt;span class=&quot;math-container&quot;&gt;$infile/_trim.fastq cutadapt -a AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC \ &#xA;-e 0.1 -O 3 -m 30 --max-n 2 -o $&lt;/span&gt;outfile $infile&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Your error message is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  IOError: [Errno 20] Not a directory: '/data/HTS_seq/PV589_03664AAB_AGGACT_read1.fastq/_trim.fastq'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which means that &lt;code&gt;cutadapt&lt;/code&gt; tried to open &lt;code&gt;/data/HTS_seq/PV589_03664AAB_AGGACT_read1.fastq/&lt;/code&gt; as a directory and then look for a file named &lt;code&gt;_trim.fastq&lt;/code&gt; inside it. The only way I can think that this would happen is if you ran the command I gave above and not the one you show in your question.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2020-06-30T15:48:14.667" LastActivityDate="2020-06-30T15:48:14.667" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13684" PostTypeId="2" ParentId="13667" CreationDate="2020-06-30T19:41:47.913" Score="1" Body="&lt;p&gt;You made the hardest work, you were not far to achieve what you wanted ;)&lt;/p&gt;&#xA;&lt;p&gt;This script.py should work after you made it executable (in a terminal type: &lt;code&gt;chmod u+x script.py&lt;/code&gt;), just type ./script.py in your terminal in your working directory (so here you should be in your &amp;quot;sp&amp;quot; directory). Note that I got inspired by: &lt;a href=&quot;https://www.biostars.org/p/42474/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/42474/&lt;/a&gt; to know how to launch pymol within the script and also that I would personally not use pymol for doing this kind of analysis/data extraction from a pdb/pdbqt file but rather use a dedicated analysis package (there are plenty like MDanalysis). I personally use pymol (pymol script) for visualization purpose.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;&#xA;&#xA;import __main__&#xA;__main__.pymol_argv = ['pymol', '-qc']  # Quiet and no GUI&#xA;&#xA;from pymol import stored&#xA;import pymol, os&#xA;&#xA;pymol.finish_launching()&#xA;&#xA;#put what you've done into a function&#xA;def myfunc(file):&#xA;    spath = os.path.abspath(file)&#xA;    sname = spath.split('/')[-1].split('.')[0]&#xA;&#xA;    pymol.cmd.load(spath, sname)&#xA;&#xA;    stored.a, stored.b, stored.c, stored.d = [], [], [], []&#xA;    pymol.cmd.iterate_state(1, &amp;quot;%s///UNL`1/N*&amp;quot; % sname, &amp;quot;stored.a.append(ID)&amp;quot;)&#xA;    pymol.cmd.iterate_state(1, &amp;quot;%s///UNL`1/O*&amp;quot; % sname, &amp;quot;stored.b.append(ID)&amp;quot;)&#xA;    pymol.cmd.iterate_state(1, &amp;quot;%s//R/ASP`113/OD2&amp;quot; % sname, &amp;quot;stored.c.append(ID)&amp;quot;)&#xA;    pymol.cmd.iterate_state(1, &amp;quot;%s//R/ASN`312/ND2&amp;quot; % sname, &amp;quot;stored.d.append(ID)&amp;quot;)&#xA;&#xA;    f = open('%s.txt' % sname, 'w')&#xA;&#xA;    for a in stored.a:&#xA;        for c in stored.c:&#xA;            d1 = pymol.cmd.distance(&amp;quot;id %s&amp;quot; % a, &amp;quot;id %s&amp;quot; % c, cutoff=5)&#xA;            f.write(&amp;quot;%s D113 %s\n&amp;quot; % (a, d1))&#xA;            print(d1)&#xA;&#xA;    for b in stored.b:&#xA;        for d in stored.d:&#xA;            d2 = pymol.cmd.distance(&amp;quot;id %s&amp;quot; % b, &amp;quot;id %s&amp;quot; % d, cutoff=5)&#xA;            f.write(&amp;quot;%s N312 %s\n&amp;quot; % (b, d2))&#xA;            print(d2)&#xA;&#xA;    f.close()&#xA;&#xA;&#xA;mypdbqt = os.popen('ls *.pdbqt').read().split()&#xA;&#xA;for i in mypdbqt:&#xA;    myfunc(i)&#xA;&#xA;pymol.cmd.quit()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9109" LastActivityDate="2020-06-30T19:41:47.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13685" PostTypeId="1" AcceptedAnswerId="13707" CreationDate="2020-06-30T20:42:15.153" Score="1" ViewCount="547" Body="&lt;p&gt;I am using edgeR to perform differential expression (DE) analysis on a set of RNA-seq data samples (2 controls; 8 treatments). To correct for batch effects, I am using RUVSeq.&lt;/p&gt;&#xA;&lt;p&gt;I am able to get a list of DE genes without normalization:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;x &amp;lt;- as.factor(rep(c(&amp;quot;Ctl&amp;quot;,&amp;quot;Inf&amp;quot;),c(2,8)))&#xA;set &amp;lt;- newSeqExpressionSet(as.matrix(counttable),phenoData=data.frame(x,row.names=colnames(counttable)))&#xA;design &amp;lt;- model.matrix(~x, data=pData(set))&#xA;y &amp;lt;- DGEList(counts=counts(set), group=x)&#xA;y &amp;lt;- calcNormFactors(y, method=&amp;quot;upperquartile&amp;quot;)&#xA;y &amp;lt;- estimateGLMCommonDisp(y, design)&#xA;y &amp;lt;- estimateGLMTagwiseDisp(y, design)&#xA;fit &amp;lt;- glmFit(y, design)&#xA;lrt &amp;lt;- glmLRT(fit, coef=2)&#xA;top &amp;lt;- topTags(lrt, n=nrow(set))$table&#xA;write.table(top, paste(OUT, &amp;quot;DE_genelist.txt&amp;quot;, sep=&amp;quot;&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then immediately after creating the &amp;quot;top&amp;quot; object, I use RUVg to normalize:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# [...]&#xA;top &amp;lt;- topTags(lrt, n=nrow(set))$table&#xA;empirical &amp;lt;- rownames(set)[which(!(rownames(set) %in% rownames(top)[1:5000]))]&#xA;ruvg &amp;lt;- RUVg(set, empirical, k=1)&#xA;write.table(ruvg, paste(OUT, &amp;quot;DE_RUVg_genelist.txt&amp;quot;, sep=&amp;quot;&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And I get the error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in as.data.frame.default(x[[i]], optional = TRUE) : &#xA;  cannot coerce class ‘structure(&amp;quot;SeqExpressionSet&amp;quot;, package = &amp;quot;EDASeq&amp;quot;)’ to a data.frame&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am not sure how to print the list of normalized results like I can with the unnormalized data. Ideally, I would get a file with the same format as the edgeR output (as a .csv or .txt file):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;logFC&amp;quot; &amp;quot;logCPM&amp;quot; &amp;quot;LR&amp;quot; &amp;quot;PValue&amp;quot; &amp;quot;FDR&amp;quot;&#xA;&amp;quot;COBLL1&amp;quot; -2.150 4.427061248733 75.0739519350016 4.53408921348828e-18 9.51203608115384e-15&#xA;&amp;quot;UBE2D1&amp;quot; -2.178 3.577168782408 74.9346752854903 4.86549160161322e-18 9.51203608115384e-15&#xA;&amp;quot;NEK7&amp;quot; -2.404 4.020072739285 72.6539117671717 1.54500340443843e-17 2.71843349010941e-14&#xA;&amp;quot;SMC6&amp;quot; -2.300 5.674738981329 61.8130019860261 3.7767230643666e-15 3.4974443325016e-12&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;How can I get a list of genes as an output &lt;em&gt;after&lt;/em&gt; normalization with RUVSeq?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="9007" LastEditorUserId="9007" LastEditDate="2020-06-30T20:59:17.877" LastActivityDate="2020-07-01T22:36:05.630" Title="How to create a list of differentially expressed (DE) genes after normalization with RUVSeq?" Tags="|r|rna-seq|differential-expression|normalization|edger|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13686" PostTypeId="5" CreationDate="2020-06-30T23:30:38.313" Score="0" Body="&lt;p&gt;Supervised learning comprise both a variety of simple regression methods as well 'deep learning' appraoches such as Tensorflow. 'Deep learning' approaches are generally run under a GPU rather than a CPU and require specialist archetecture. 'Unsupervised learning' could be considered machine learning, but would be traditionally termed multi-variate statistics such as PCA.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-06-30T23:30:38.313" LastActivityDate="2020-06-30T23:30:38.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13687" PostTypeId="4" CreationDate="2020-06-30T23:30:38.313" Score="0" Body="A subset of articial intelligence methods which 'learn' through training on real-world data sets. The model is then tested on a 'test' data set and initially assessed through an accuracy measure. The approach is specifically termed 'supervised learning'." OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-06-30T23:30:38.313" LastActivityDate="2020-06-30T23:30:38.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13688" PostTypeId="5" CreationDate="2020-06-30T23:37:47.123" Score="0" Body="&lt;p&gt;Perl5 is currently 5.32 and is an actively maintained language usually coded as a functional programming due to the verbosity required to code in object orientated programming. Its reg-ex is particularly strong and versatile and is widely used as 'one-liners' in the bash command line.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-06-30T23:37:47.123" LastActivityDate="2020-06-30T23:37:47.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13689" PostTypeId="4" CreationDate="2020-06-30T23:37:47.123" Score="0" Body="Generally refers to Perl5 and is a much loved computer language of exceptional beauty and simplicity. It is the hallmark of exceptional character." OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-06-30T23:37:47.123" LastActivityDate="2020-06-30T23:37:47.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13690" PostTypeId="2" ParentId="11162" CreationDate="2020-07-01T00:51:16.427" Score="1" Body="&lt;p&gt;There is a &lt;a href=&quot;https://www.nature.com/articles/s41587-019-0071-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;recent publication&lt;/a&gt; that compares methods for trajectory inference. In &lt;a href=&quot;https://www.nature.com/articles/s41587-019-0071-9/figures/2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Figure 2&lt;/a&gt; in terms of scalability &lt;a href=&quot;https://doi.org/10.1101/208819&quot; rel=&quot;nofollow noreferrer&quot;&gt;PAGA&lt;/a&gt;, PAGA Tree and Component 1 get the highest rating. The authors &lt;a href=&quot;https://dynverse.org/users/3-user-guide/2-guidelines/&quot; rel=&quot;nofollow noreferrer&quot;&gt;offer a shiny app&lt;/a&gt; to help select the best method for your dataset.&lt;/p&gt;&#xA;" OwnerUserId="3575" LastActivityDate="2020-07-01T00:51:16.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13691" PostTypeId="1" CreationDate="2020-07-01T04:11:19.737" Score="0" ViewCount="566" Body="&lt;p&gt;How do I convert an Excel file(.xlsx) to .smi file format?&lt;/p&gt;&#xA;&lt;p&gt;The original file I have is an excel file containing over 27,000 ligands. I am going to using the Padel Descriptor software to calculate the descriptors for the ligands however, Padel Descriptor just supports .smi files. Screenshot of the excel file added below for more clarity.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/lz4em.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lz4em.jpg&quot; alt=&quot;Screenshot of the excel file added below for more clarity. &quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9040" LastEditorUserId="6322" LastEditDate="2020-07-01T08:58:31.820" LastActivityDate="2020-07-01T08:58:31.820" Title="How do I convert an Excel file (.xlsx) to an .smi file format?" Tags="|proteins|cheminformatics|computational-biochemistry|small-molecules|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13692" PostTypeId="2" ParentId="13662" CreationDate="2020-07-01T04:35:16.403" Score="1" Body="&lt;p&gt;according to your Fastqc files, your reads have a moderate quality. But I am sure there is something to do to improve them.&#xA;Try to run &lt;code&gt;fastp&lt;/code&gt; with the following options:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fastp -i _1.fq -I _2.fq --detect_adapter_for_pe -l 36 -5 -3 -q 20&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and run &lt;code&gt;FastQc&lt;/code&gt; again to check to efficacity of the trimming.&lt;/p&gt;&#xA;" OwnerUserId="8749" LastActivityDate="2020-07-01T04:35:16.403" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13693" PostTypeId="2" ParentId="13691" CreationDate="2020-07-01T08:55:07.470" Score="1" Body="&lt;p&gt;So a smi file is simply a text file formatted &lt;code&gt;SMILES&lt;/code&gt; tab &lt;code&gt;name&lt;/code&gt; newline.&lt;/p&gt;&#xA;&lt;p&gt;Therefore what you need to do is&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;save for safety&lt;/li&gt;&#xA;&lt;li&gt;make a new sheet&lt;/li&gt;&#xA;&lt;li&gt;copy the SMILES column and the name column&lt;/li&gt;&#xA;&lt;li&gt;save as tab separated file&lt;/li&gt;&#xA;&lt;li&gt;in Finder/Nautilus/File Explorer rename the file extension to &lt;code&gt;.smi&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-07-01T08:55:07.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13694" PostTypeId="5" CreationDate="2020-07-01T09:04:57.257" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Cheminformatics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cheminformatics&lt;/a&gt; is a field of chemistry, namely the intersection of computational methods and chemical theory, whose methods are extensively employed in docking, a subfield of computational biochemistry.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-07-01T14:02:55.193" LastActivityDate="2020-07-01T14:02:55.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13695" PostTypeId="4" CreationDate="2020-07-01T09:04:57.257" Score="0" Body="Cheminformatics is the computational analysis and enumeration of small molecules" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-07-01T14:02:59.720" LastActivityDate="2020-07-01T14:02:59.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13696" PostTypeId="5" CreationDate="2020-07-01T09:08:44.380" Score="0" Body="&lt;p&gt;A small molecule is a non-polymeric molecule.&#xA;Cofactors, such as ATP or PLP, or metabolites, such as glucose or alanine, are small molecules, whereas a protein or a DNA strand is not.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-07-01T14:02:43.200" LastActivityDate="2020-07-01T14:02:43.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13697" PostTypeId="4" CreationDate="2020-07-01T09:08:44.380" Score="0" Body="A small molecule is a non-polymeric molecule, such as a cofactor or metabolite" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-07-01T14:03:02.790" LastActivityDate="2020-07-01T14:03:02.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13698" PostTypeId="5" CreationDate="2020-07-01T09:16:31.470" Score="0" Body="&lt;p&gt;Rosetta, and its Python wrapper, Pyrosetta, are a computational biochemistry toolkit that enables a variety of force-field based calculations ranging from binding scores to missing loop addition and enzyme design.&lt;/p&gt;&#xA;&lt;p&gt;For more see &lt;a href=&quot;https://www.rosettacommons.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rosettacommons.org/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-07-01T14:02:38.103" LastActivityDate="2020-07-01T14:02:38.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13699" PostTypeId="4" CreationDate="2020-07-01T09:16:31.470" Score="0" Body="Rosetta and pyrosetta are a computational biochemistry toolkit that enables a variety of force-field based calculations ranging from binding scores to missing loop addition and enzyme design." OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-07-01T14:02:51.487" LastActivityDate="2020-07-01T14:02:51.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13700" PostTypeId="2" ParentId="13685" CreationDate="2020-07-01T14:45:43.413" Score="0" Body="&lt;p&gt;I have not used this package but from your code it seems that &lt;code&gt;ruvg&lt;/code&gt; is not a table. Instead, it is an R object, which means that you cannot use &lt;code&gt;write.table&lt;/code&gt;. I think the results you want is stored in the object. All R objects contain &amp;quot;slots&amp;quot; of data, which can be accessed by &lt;code&gt;@&lt;/code&gt;. If I were you, I would type &lt;code&gt;ruvg@&lt;/code&gt; and should be able to see which data slots are contained in the object.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-07-01T14:45:43.413" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13701" PostTypeId="1" CreationDate="2020-07-01T16:18:45.403" Score="4" ViewCount="431" Body="&lt;p&gt;I have a dataset of 50 breast cancer samples. These samples are classified into four subtypes Lum A, Lum B, Her2 and Basal. I have been working with lncRNAs and protein-coding genes. To identify the functions of lncRNAs, I have used WGCNA through which I get the modules with protein-coding genes and lncRNAs. The expression data is stored in a matrix &lt;code&gt;datExpr&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I have the trait information in &lt;code&gt;datTraits&lt;/code&gt; like below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/iwZ1U.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;I totally got 20 modules named with different colors stored in &lt;code&gt;dynamicColors&lt;/code&gt;. Like below I first calculated eigenvalues and then used it and trait information to check the association between Modules and trait data.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MEs0 = moduleEigengenes(datExpr, dynamicColors)$eigengenes&#xA;MEs = orderMEs(MEs0)&#xA;moduleTraitCor = cor(MEs, datTraits, use = &amp;quot;p&amp;quot;);&#xA;moduleTraitPvalue = corPvalueStudent(moduleTraitCor, nSamples);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What is the optimal method to associate modules and trait using binary data?&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is Pearson correlation a right way to check the association?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there any other way to check the association between modules and trait (which represented like above in binary)? Details of methology would be welcome.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Patients are not genetically related and I have no information on environmental risk factors. I was thinking about logistic regression, but before doing that, I would like to take suggestions.&lt;/p&gt;&#xA;" OwnerUserId="9114" LastEditorUserId="73" LastEditDate="2021-05-17T04:37:31.540" LastActivityDate="2022-02-11T06:05:45.500" Title="How to calculate module-trait relationship when trait data is in binary format?" Tags="|r|rna-seq|networks|wgcna|correlation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13702" PostTypeId="1" CreationDate="2020-07-01T17:21:33.237" Score="1" ViewCount="61" Body="&lt;p&gt;I have some RNA-seq samples from multiple glioblastoma tumours that I'm now trying to classify according to a specific gene signature (from Verhaak et al., 2010) using R. The gene signature is reported as a gene list with specific centroids for each of the 4 clusters (&lt;a href=&quot;https://api.gdc.cancer.gov/data/941f81a1-05d7-4f84-80ec-534b8dc1ebac&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://api.gdc.cancer.gov/data/941f81a1-05d7-4f84-80ec-534b8dc1ebac&lt;/a&gt;). I'm wondering how I can use this signature to classify my samples in R? Would it involve some sort of k-nearest neighbours method?&lt;/p&gt;&#xA;&lt;p&gt;Additionally, the signature was identified using microarray data, but I am classifying RNA-seq data. Is there any sort of adjustment I should make to the signature to account for this?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="9116" LastActivityDate="2020-08-01T00:02:10.240" Title="classifying samples by TCGA signature" Tags="|r|rna-seq|clustering|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13704" PostTypeId="1" CreationDate="2020-07-01T20:55:43.413" Score="1" ViewCount="65" Body="&lt;p&gt;I need to identify all loci in the short-read sequence at which the number of microsatellite repeats (i.e. number of copies of &amp;quot;AA,&amp;quot; &amp;quot;GTC,&amp;quot; etc.) differ from the reference genome, as well as the locations at which the number of repeats is the same in both the reference genome and the short-read sequence.&lt;/p&gt;&#xA;&lt;p&gt;I used the Burrows-Wheeler Aligner (BWA mem) to map a high-coverage short-read sequence (obtained from the NCBI's short-read sequence archive) to a reference genome. The output is in .sam format. I have also used a separate program to identify the loci in the reference genome at which microsatellites occur.&lt;/p&gt;&#xA;&lt;p&gt;I would like to identify all the loci in the short-read sequence at which the microsatellite length and loci differ from the reference genome. Does anyone know any tools or packages I could use to read a .sam/.bam file of a short-read sequence mapped to a reference genome and identify specific loci at which the short-read sequence differs from the reference genome? I am using RStudio and have access to my university's supercomputer clusters.&lt;/p&gt;&#xA;&lt;p&gt;For info on microsatellites, see here: &lt;a href=&quot;https://en.wikipedia.org/wiki/Microsatellite#:%7E:text=A%20microsatellite%20is%20a%20tract,locations%20within%20an%20organism%27s%20genome&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Microsatellite#:~:text=A%20microsatellite%20is%20a%20tract,locations%20within%20an%20organism's%20genome&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My end goal is to determine the loci at which microsatellite alleles tend to differ within a species, as well as the loci at which microsatellite alleles tend to be identical between all members of the same species.&lt;/p&gt;&#xA;" OwnerUserId="9118" LastEditorUserId="9118" LastEditDate="2020-07-07T17:25:19.820" LastActivityDate="2020-07-07T17:25:19.820" Title="BWA: Detecting Variation between Reference Genome and Short-Read Sequences" Tags="|genome|phylogenetics|variation|bwa|reference-genome|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13705" PostTypeId="1" AcceptedAnswerId="13713" CreationDate="2020-07-01T21:15:02.437" Score="0" ViewCount="116" Body="&lt;p&gt;So right now my observations in the dataset I've processed is stored as sample identifier columns and gene rows: an example of this would be for sample 1 and gene TRIM21 the observation is Missense Mutation;Frame Shift Deletion. I'm having issues separating the two alterations and instead of being able to show the gene in the sample having 2 different alterations, I end up having to generate a new colour classification (being that the gene in the specific sample exhibits specifically both a missense mutation and a frame shift deletion instead of being able to represent the observation with just colours and missense and frame shift deletion colours alone). Here is an a screenshot of what my matrix looks like: &lt;a href=&quot;https://i.stack.imgur.com/i6i1H.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/i6i1H.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As an example here is some test code generating the test plot:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(ComplexHeatmap)&#xA;&#xA;test_labels &amp;lt;- c(&amp;quot;Missense_Mutation&amp;quot;, &amp;quot;Splice_Site&amp;quot;, &amp;quot;Frame_Shift_Del&amp;quot;,&#xA;                 &amp;quot;Splice_Region&amp;quot;,&amp;quot;Missense_Mutation;Splice_Site&amp;quot;, &amp;quot;Splice_Site;Frame_Shift_Del&amp;quot;, &#xA;                 &amp;quot;Splice_Site;Missense_Mutation&amp;quot;, &amp;quot;Frame_Shift_Del;Splice_Site&amp;quot;)&#xA;                           &#xA;&#xA;plot &amp;lt;- Heatmap(test, row_names_side = &amp;quot;left&amp;quot;,&#xA;        row_names_gp = gpar(cex = 0.7), &#xA;        column_names_gp = gpar(cex = 0.7), &#xA;        heatmap_legend_param = list(title = &amp;quot;Alterations&amp;quot;, &#xA;                                    at = test_labels, labels = test_labels), show_heatmap_legend = TRUE)&#xA;&#xA;plot&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've attached a screenshot of my actual plot and as you will be able to see, this ends up creating multiple different classifications: &lt;a href=&quot;https://i.stack.imgur.com/Zr5QE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Zr5QE.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8904" LastEditorUserId="8904" LastEditDate="2020-07-01T22:46:50.007" LastActivityDate="2020-07-02T10:57:01.407" Title="Is there a way to customize what observations are plotted in ComplexHeatmap?" Tags="|bioconductor|mutations|heatmap|complexheatmap|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="13706" PostTypeId="2" ParentId="13702" CreationDate="2020-07-01T22:24:24.877" Score="1" Body="&lt;p&gt;This is not a very simple problem. I saved the table and &amp;quot;test.txt&amp;quot; and visualize it below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(pheatmap)&#xA;x = read.delim(&amp;quot;test.txt&amp;quot;)&#xA;&#xA;head(x)&#xA;  Gene.Symbol            Composite_chr_coords  Proneural      Neural&#xA;1      CDKN1B   [36.1:12:12761576-12766570:+] 0.03353670  0.03353670&#xA;2       EPB41    [36.1:1:29086215-29319545:+] 0.03882308  0.03882308&#xA;3        CLGN  [36.1:4:141529057-141568232:-] 2.12796056  0.41999847&#xA;4      PDE10A  [36.1:6:165664525-165995575:-] 1.66012916  0.25349464&#xA;5     RALGPS2  [36.1:1:176960923-177152465:+] 0.78971500 -0.01945154&#xA;6        TAF5 [36.1:10:105117714-105138812:+] 1.08354870  0.09461930&#xA;    Classical Mesenchymal&#xA;1  0.03353670 -0.84379400&#xA;2 -0.84821990  0.03882308&#xA;3  0.41999847  0.41999847&#xA;4  0.25349464  0.25349464&#xA;5 -0.01945154 -0.01945154&#xA;6  0.09461930  0.09461930&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WbklC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WbklC.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can see there are no so called markers genes and if you look at the scale of the data, it was normalized in microarray and it will be very hard for you to get your RNA-seq to fall into that range, and apply the medoids to identify your clusters or type.&lt;/p&gt;&#xA;&lt;p&gt;Since you did not provide any data, and it is not trivial to simulate data for this, I can suggest the approach in words here:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Subset your count table to these exact set of 840 genes.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Normalize the data and do a PCA or hierarchical clustering to see if you indeed observe 4 or 3 or 2 subtypes&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you indeed see 4 or 3 clusters this would be straight forward, take the centres of means of these clusters, and correlate with each of the sub types here. Chances are the subtype that is most correlated with one cluster, is the subtype you are interested in.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-07-01T22:24:24.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13707" PostTypeId="2" ParentId="13685" CreationDate="2020-07-01T22:36:05.630" Score="1" Body="&lt;p&gt;You do the normalization before running your edgeR. The purpose of &lt;code&gt;RUVg&lt;/code&gt; is to remove &amp;quot;Remove Unwanted Variation Using Control Genes&amp;quot;. In your code, you ran edgeR and then normalize the data using RUVg, which is only going to return you the normalized counts.&lt;/p&gt;&#xA;&lt;p&gt;Using the example dataset in &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/RUVSeq/inst/doc/RUVSeq.R&quot; rel=&quot;nofollow noreferrer&quot;&gt;vignette&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(RUVSeq)&#xA;library(zebrafishRNASeq)&#xA;data(zfGenes)&#xA;filter &amp;lt;- apply(zfGenes, 1, function(x) length(x[x&amp;gt;5])&amp;gt;=2)&#xA;filtered &amp;lt;- zfGenes[filter,]&#xA;genes &amp;lt;- rownames(filtered)[grep(&amp;quot;^ENS&amp;quot;, rownames(filtered))]&#xA;spikes &amp;lt;- rownames(filtered)[grep(&amp;quot;^ERCC&amp;quot;, rownames(filtered))]&#xA;&#xA;x &amp;lt;- as.factor(rep(c(&amp;quot;Ctl&amp;quot;, &amp;quot;Trt&amp;quot;), each=3))&#xA;set &amp;lt;- newSeqExpressionSet(as.matrix(filtered),&#xA;                           phenoData = data.frame(x, row.names=colnames(filtered)))&#xA;set &amp;lt;- betweenLaneNormalization(set, which=&amp;quot;upper&amp;quot;)&#xA;&#xA;set1 &amp;lt;- RUVg(set, spikes, k=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can look at it, it's an expression set with counts etc, not results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;set1&#xA;SeqExpressionSet (storageMode: lockedEnvironment)&#xA;assayData: 20865 features, 6 samples &#xA;  element names: counts, normalizedCounts, offset &#xA;protocolData: none&#xA;phenoData&#xA;  sampleNames: Ctl1 Ctl3 ... Trt13 (6 total)&#xA;  varLabels: x W_1&#xA;  varMetadata: labelDescription&#xA;featureData: none&#xA;experimentData: use 'experimentData(object)'&#xA;Annotation:  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You run edgeR now on the results of RUVg:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;design &amp;lt;- model.matrix(~x + W_1, data=pData(set1))&#xA;y &amp;lt;- DGEList(counts=counts(set1), group=x)&#xA;y &amp;lt;- calcNormFactors(y, method=&amp;quot;upperquartile&amp;quot;)&#xA;y &amp;lt;- estimateGLMCommonDisp(y, design)&#xA;y &amp;lt;- estimateGLMTagwiseDisp(y, design)&#xA;fit &amp;lt;- glmFit(y, design)&#xA;lrt &amp;lt;- glmLRT(fit, coef=2)&#xA;topTags(lrt)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-07-01T22:36:05.630" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13708" PostTypeId="1" AcceptedAnswerId="13710" CreationDate="2020-07-01T23:41:25.513" Score="2" ViewCount="809" Body="&lt;p&gt;I'm attempting to remove redundant sequences from a fasta file (from NCBI). When I execute this code, it returns the number of spots, not the number of sequences. (Number of spots: 408,293, Number of sequences: 40,830)&lt;/p&gt;&#xA;&lt;p&gt;Please let me know if you have run a code that can remove the duplicate ideas or have any ideas/suggestions.&lt;/p&gt;&#xA;&lt;p&gt;Thank you SO much.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;import time&#xA;&#xA;start = time.time() &#xA;&#xA;seen = []&#xA;records = []&#xA;&#xA;for record in SeqIO.parse(&amp;quot;b4r2.fasta&amp;quot;, &amp;quot;fasta&amp;quot;):  &#xA;    if str(record.seq) not in seen:&#xA;        seen.append(str(record.seq))&#xA;        records.append(record)&#xA;&#xA;&#xA;#writing to a fasta file&#xA;SeqIO.write(records, &amp;quot;no_dupes_b4r2.fasta&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;end = time.time()&#xA;&#xA;print(f&amp;quot;Run time is {(end- start)/60}&amp;quot;) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9120" LastEditorUserId="650" LastEditDate="2020-07-01T23:46:19.020" LastActivityDate="2020-07-02T07:35:05.583" Title="Remove Redundant Sequences from FASTA file in Python" Tags="|fasta|python|sequence|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13709" PostTypeId="2" ParentId="13704" CreationDate="2020-07-02T02:55:14.333" Score="3" Body="&lt;p&gt;I haven't done any whole-genome STR analysis from NGS data myself, but are aware of others that have used &lt;a href=&quot;http://lobstr.teamerlich.org/download.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;lobSTR&lt;/a&gt; for this. There's also a recent paper [&lt;a href=&quot;https://doi.org/10.1101/2020.02.03.933002&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;] that compares a few different STR analysis packages (i.e. RepeatSeq, LobSTR, HipSTR, GangSTR). Here's the concluding paragraph:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In conclusion, all these tools are built to genotype STRs but&#xA;have different strengths and weaknesses. Based on our analysis there is&#xA;no clear overall winner. RepeatSeq and HipSTR are the best when&#xA;considering genotyping error rate even with low coverage. On the other&#xA;hand, GangSTR has an advantage because it is the only tool among them&#xA;that can call alleles longer than the read length but shows higher error&#xA;rate unless looking at only the enclosed class of reads, which in turn&#xA;would lose the GangSTR's advantage of picking up long genotypes. In&#xA;addition, GangSTR is the newest tool and so comes with reference files for&#xA;different reference builds that are periodically updated according to&#xA;the tool's webpage. The correct choice of a tool and the&#xA;subsequent filtering depends on the aim of the analysis, and might be&#xA;influenced by available hardware resources and time limit for running&#xA;tools.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In other words, without more information about the specific problem you're trying to solve, it's not clear what the best tool to use is (this is a common issue with Bioinformatics problems).&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-07-02T04:28:44.753" LastActivityDate="2020-07-02T04:28:44.753" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13710" PostTypeId="2" ParentId="13708" CreationDate="2020-07-02T07:35:05.583" Score="5" Body="&lt;p&gt;Your code (as currently formatted) should generate the correct output, but it is not efficient. Make &lt;code&gt;seen&lt;/code&gt; a set to improve these O(n) &lt;code&gt;in&lt;/code&gt; calls to O(1) time complexity- this will speed it up a lot&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seen = set()&#xA;records = []&#xA;&#xA;for record in SeqIO.parse(&amp;quot;b4r2.fasta&amp;quot;, &amp;quot;fasta&amp;quot;):  &#xA;    if record.seq not in seen:&#xA;        seen.add(record.seq)&#xA;        records.append(record)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastActivityDate="2020-07-02T07:35:05.583" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13711" PostTypeId="2" ParentId="13656" CreationDate="2020-07-02T10:03:33.423" Score="1" Body="&lt;p&gt;I have nutted out a solution for you.&lt;/p&gt;&#xA;&lt;p&gt;If this is the content of input.fa:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;virusA_detailed_description&#xA;ACGTGACGATCCCCAAACGTGACGATCCCCAA&#xA;&amp;gt;virusB_detailed_description&#xA;ACGTGACGATCGGGAAACGTGACGATCGGGAA&#xA;&amp;gt;virusC_detailed_description&#xA;ACGTGACGATCAAAAAACGTGACGATCAAAAA&#xA;&amp;gt;virusD_detailed_description&#xA;ACGTGACGATTTTTAAACGTGACGATTTTTAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And this is the script:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;'''&#xA;Converts a fasta file to phylip format&#xA;'''&#xA;&#xA;from Bio import AlignIO&#xA;from Bio.AlignIO.PhylipIO import PhylipWriter&#xA;import sys&#xA;&#xA;infile = sys.argv[1]&#xA;outfile = sys.argv[2]&#xA;idwidth = int(sys.argv[3])&#xA;with open(infile, 'r') as input_handle:&#xA;    align = AlignIO.read(input_handle, &amp;quot;fasta&amp;quot;)&#xA;    with open(outfile, 'w') as output_handle:&#xA;        PhylipWriter(output_handle).write_alignment(align, id_width=idwidth)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then, save this script as &lt;code&gt;phylpwrtr.py&lt;/code&gt; and do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chmod +x phylpwrtr.py&#xA;./phylpwrtr.py input.fa output_longids.phy 30 # 30 is the width of the header, adjust the value to adjust header length&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The contents of &lt;code&gt;output_longids.phy&lt;/code&gt; looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; 4 32&#xA;virusA_detailed_description    ACGTGACGAT CCCCAAACGT GACGATCCCC AA&#xA;virusB_detailed_description    ACGTGACGAT CGGGAAACGT GACGATCGGG AA&#xA;virusC_detailed_description    ACGTGACGAT CAAAAAACGT GACGATCAAA AA&#xA;virusD_detailed_description    ACGTGACGAT TTTTAAACGT GACGATTTTT AA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3381" LastActivityDate="2020-07-02T10:03:33.423" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13713" PostTypeId="2" ParentId="13705" CreationDate="2020-07-02T10:57:01.407" Score="1" Body="&lt;p&gt;What is fed into &lt;code&gt;ComplexHeatmap&lt;/code&gt; is what it outputs! So if one of your genes has a &lt;code&gt;Missense_Mutation&lt;/code&gt; as well as a &lt;code&gt;Frame_Shift_Del&lt;/code&gt; and you would like to plot its value, of course you will have a category and color that is different than those of &lt;code&gt;Missense_Mutation&lt;/code&gt; or &lt;code&gt;Frame_Shift_Del&lt;/code&gt;, that is only natural.&lt;/p&gt;&#xA;&lt;p&gt;What you can do is to create additional variables like &lt;code&gt;your_gene_Missense_Mutation&lt;/code&gt; or &lt;code&gt;your_gene_Frame_Shift_Del&lt;/code&gt; each with 0 or 1 to indicate if a particular mutation type is present or not for a particular gene (you can think of it like one-hot-encoding). With a lot of different levels, the resulting heatmap could be difficult to interpret though.&lt;/p&gt;&#xA;&lt;p&gt;Another simpler alternative would be a using a &amp;quot;universal label&amp;quot; if your gene has multiple mutations. Basically if the cell contains a &lt;code&gt;;&lt;/code&gt;, its value will be changed to something like &lt;code&gt;mixed_mutation_status&lt;/code&gt; and instead of &lt;code&gt;multiple &amp;quot;mutation_type_1; mutation_type_2; ...&lt;/code&gt; like categories, you will just have one &lt;code&gt;mixed_mutation_status&lt;/code&gt; on top of &lt;code&gt;Missense_Mutation&lt;/code&gt;, &lt;code&gt;Frame_Shift_Del&lt;/code&gt;, ...&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-07-02T10:57:01.407" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13714" PostTypeId="5" CreationDate="2020-07-02T13:02:28.167" Score="0" Body="&lt;p&gt;Use this tag to refer to the questions related to gene expression&lt;/p&gt;&#xA;" OwnerUserId="1194" LastEditorUserId="1194" LastEditDate="2020-07-02T15:52:23.053" LastActivityDate="2020-07-02T15:52:23.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13715" PostTypeId="4" CreationDate="2020-07-02T13:02:28.167" Score="0" Body="" OwnerUserId="-1" LastEditorUserId="-1" LastEditDate="2020-07-02T13:02:28.167" LastActivityDate="2020-07-02T13:02:28.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13716" PostTypeId="1" AcceptedAnswerId="16248" CreationDate="2020-07-03T07:42:52.993" Score="0" ViewCount="70" Body="&lt;p&gt;I am learning to use Roary for preparing a pan genome for some lactobacillus strains.  In the 'gene_presence_absence.csv' output of Roary (which I view in excel), a 'source Gene ID' is given for each of the strains included in the analysis.  The &lt;a href=&quot;https://sanger-pathogens.github.io/Roary/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Roary web page&lt;/a&gt; states that this is:&lt;/p&gt;&#xA;&lt;ol start=&quot;15&quot;&gt;&#xA;&lt;li&gt;Presence and absence of genes in each sample, with the corresponding source Gene ID.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Is this 'source Gene ID' generated by Roary, or does it relate back directly to the .gff files I used as input?  If I take one of these 'source Gene ID' references and &lt;code&gt;Grep&lt;/code&gt; in the directory containing my .gff files I get no matches, so I am assuming it is generated by Roary...?&lt;/p&gt;&#xA;&lt;p&gt;I had hoped that this reference would link back directly to the gene within the annotation file, but that seems not to be the case?  Should/can I use this reference for anything?&lt;/p&gt;&#xA;" OwnerUserId="6738" LastEditorUserId="6738" LastEditDate="2021-06-28T19:38:50.370" LastActivityDate="2021-06-28T19:38:50.370" Title="What is the origin of the 'source Gene ID' references given in the 'gene_presence_absence.csv' output of Roary?" Tags="|genome|genomics|bacteria|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13717" PostTypeId="1" AcceptedAnswerId="13730" CreationDate="2020-07-03T11:29:38.870" Score="0" ViewCount="511" Body="&lt;p&gt;What is happening when you use &lt;code&gt;aov()&lt;/code&gt; on a glm model in R?&lt;/p&gt;&#xA;&lt;p&gt;Normal glm model &lt;code&gt;sumamry(m1)&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Call:&#xA;glm(formula = value ~ origin + variable, family = poisson(), &#xA;    data = comb_data)&#xA;&#xA;Deviance Residuals: &#xA;    Min       1Q   Median       3Q      Max  &#xA;-0.8227  -0.5547  -0.4385  -0.3038   2.5258  &#xA;&#xA;Coefficients: (1 not defined because of singularities)&#xA;                                                                  Estimate Std. Error z value Pr(&amp;gt;|z|)    &#xA;(Intercept)                                                     -2.853e+00  5.774e-01  -4.941 7.78e-07 ***&#xA;originisolate                                                    4.700e-01  7.071e-01   0.665  0.50625    &#xA;variableRB17                                                     2.877e-01  5.401e-01   0.533  0.59425    &#xA;variableM56                                                      1.299e+00  4.606e-01   2.821  0.00479 ** &#xA;variableRB68                                                    -6.269e-14  5.774e-01   0.000  1.00000    &#xA;variableM39b4                                                   -1.010e-13  5.774e-01   0.000  1.00000    &#xA;variableRB108                                                    1.542e-01  5.563e-01   0.277  0.78172    &#xA;variableM32                                                      4.055e-01  5.270e-01   0.769  0.44171    &#xA;variableRB33                                                    -6.931e-01  7.071e-01  -0.980  0.32696    &#xA;variableM41                                                      6.061e-01  5.075e-01   1.194  0.23236    &#xA;variableRB1                                                      6.931e-01  5.000e-01   1.386  0.16566    &#xA;variableM39w2                                                    1.542e-01  5.563e-01   0.277  0.78172    &#xA;variableRB20                                                    -1.792e+00  1.080e+00  -1.659  0.09714 .  &#xA;variableRB99                                                     1.542e-01  5.563e-01   0.277  0.78172    &#xA;variableRB110                                                    6.931e-01  5.000e-01   1.386  0.16566    &#xA;variableRB5                                                      1.542e-01  5.563e-01   0.277  0.78172    &#xA;variableRB29                                                    -1.823e-01  6.055e-01  -0.301  0.76334    &#xA;variableRB56                                                    -7.411e-14  5.774e-01   0.000  1.00000    &#xA;variableRB24                                                    -1.099e+00  8.165e-01  -1.346  0.17846    &#xA;variableStreptomyces_alboflavus_strain_MDJK44_RB5                1.204e+00  6.583e-01   1.829  0.06740 .  &#xA;variableStreptomyces_violaceoruber_RB1                           1.299e+00  6.513e-01   1.995  0.04607 *  &#xA;variableStreptomyces_T44_RB17                                    1.204e+00  6.583e-01   1.829  0.06740 .  &#xA;variableStreptomyces_violaceoruber_RB1_RB110                     1.299e+00  6.513e-01   1.995  0.04607 *  &#xA;variableActinomadura_verrucosospora_RB99                         5.108e-01  7.303e-01   0.699  0.48425    &#xA;variableStreptomyces_autolyticus_strain_CGMCC0516_M56            1.735e+00  6.262e-01   2.770  0.00561 ** &#xA;variableStreptomyces_atratus_RB108                               9.808e-01  6.770e-01   1.449  0.14740    &#xA;variableStreptomyces_sp_T44_RB17                                 1.204e+00  6.583e-01   1.829  0.06740 .  &#xA;variableStreptomyces_sp_QMT.28_M41                               8.473e-01  6.901e-01   1.228  0.21950    &#xA;variableAmycolatopsis_sp_AA4_M39                                 8.473e-01  6.901e-01   1.228  0.21950    &#xA;variableLuteimicrobium_xylanilyticum_RB22                       -1.099e+00  1.155e+00  -0.951  0.34138    &#xA;variableMycolicibacterium_parafortuitum_RB24                     1.239e-13  8.165e-01   0.000  1.00000    &#xA;variableNocardia_nova_SH22a_RB20_RB56                            2.877e-01  7.638e-01   0.377  0.70642    &#xA;variableActinomadura_atramentaria_RB29                           5.108e-01  7.303e-01   0.699  0.48425    &#xA;variableMycolicibacterium_madagascariense_strain_JCM_13574_RB33         NA         NA      NA       NA    &#xA;---&#xA;Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1&#xA;&#xA;(Dispersion parameter for poisson family taken to be 1)&#xA;&#xA;    Null deviance: 1024.95  on 1949  degrees of freedom&#xA;Residual deviance:  943.36  on 1917  degrees of freedom&#xA;AIC: 1507.4&#xA;&#xA;Number of Fisher Scoring iterations: 6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A &lt;code&gt;summary(aov(m1))&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;              Df Sum Sq Mean Sq F value   Pr(&amp;gt;F)    &#xA;origin         1   0.33  0.3285   3.050   0.0809 .  &#xA;variable      31  10.41  0.3357   3.117 2.06e-08 ***&#xA;Residuals   1917 206.47  0.1077 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7031" LastActivityDate="2021-12-24T01:34:41.273" Title="What does aov(glm model) actually do in R?" Tags="|r|statistics|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13718" PostTypeId="2" ParentId="13717" CreationDate="2020-07-03T14:34:14.690" Score="0" Body="&lt;p&gt;It's equivalent to doing &lt;code&gt;anova(m1, test=&amp;quot;F&amp;quot;)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;You probably want to do some sort of deviance test for a Poisson glm&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;anova(m1, test=&amp;quot;Chisq&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8919" LastActivityDate="2020-07-03T14:34:14.690" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13719" PostTypeId="2" ParentId="13596" CreationDate="2020-07-03T14:59:04.420" Score="2" Body="&lt;p&gt;As others already pointed out, there is no official heatmap color scheme. I am not sure there is an official color scheme for any other types of plots either.&lt;/p&gt;&#xA;&lt;p&gt;However, heatmaps seem to be far more standardized. Traditionally, heatmaps have been red-black-green as you originally stated. Most heatmaps from the microarray era and early RNA-seq are like that. You can have a heatmap for any type of data, but the majority is for gene expression. A lot of tools and tutorials were written back then, so they have that color scheme. From about 2008 to 2012, most people switched to the red-white-blue (or red-yellow-blue) color scheme. Switching green to blue avoids the color-blindness issue. Switching black to white is better for the ink if you print your papers and generally looks better on white background.&lt;/p&gt;&#xA;&lt;p&gt;I originally found red as high to be confusing because I was thinking in financial terms (if the market is red, it's down). Now I think of it as a water faucet (red is hot, blue is cold).&lt;/p&gt;&#xA;&lt;p&gt;Edit: In the last year or so, the &lt;a href=&quot;https://cran.r-project.org/web/packages/viridis/vignettes/intro-to-viridis.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;viridis&lt;/a&gt; palette started to become widely used. It's far from the standard, but could become more popular in the future.&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="35" LastEditDate="2020-07-05T16:27:00.753" LastActivityDate="2020-07-05T16:27:00.753" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13720" PostTypeId="1" CreationDate="2020-07-03T17:18:45.583" Score="0" ViewCount="154" Body="&lt;p&gt;So I have the following e-utils api-link:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=gene&amp;amp;db=snp&amp;amp;id=5726&amp;amp;retmode=json&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/elink.fcgi?dbfrom=gene&amp;amp;db=snp&amp;amp;id=5726&amp;amp;retmode=json&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;it returns all the SNP-ids for the gene with the id 5726 as an example. Additionally to the SNP-ids I also need the SNP position and the genomic distance. Also I would need the chromosome but I could query that from the gene as well rather than for each SNP. Do you know how I can add all these informations to the resulting json/xml?&lt;/p&gt;&#xA;&lt;p&gt;The first one in the resulting list is the SNP with the id 1490381980. Further information about it can be found on dbSNP: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/?term=1490381980%5BUID%5D&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/snp/?term=1490381980%5BUID%5D&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I really want to avoid creating a request for each single SNP and would prefer to query everything at once. Due to the unavoidable performance-issues. The Tas2R38 doesn't have that many SNPs but there are other genes with way more.&lt;/p&gt;&#xA;" OwnerUserId="9131" LastActivityDate="2020-09-29T20:56:42.570" Title="e-utils api: get chromosome position for each snp on a gene" Tags="|snp|phylogenetics|api|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13722" PostTypeId="1" AcceptedAnswerId="13723" CreationDate="2020-07-04T12:46:13.607" Score="5" ViewCount="10422" Body="&lt;p&gt;&lt;em&gt;(Full discolosure that this is my first time working with sequence data, and with the bash scripting.)&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I need to calculate the average coverage for any .bam file.&lt;/p&gt;&#xA;&lt;p&gt;After some searching I wrote the following script:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bsh prettyprint-override&quot;&gt;&lt;code&gt;# Script to calculate the average coverage of a genome sample&#xA;&#xA;# compute the total length of the sample&#xA;tot=&lt;span class=&quot;math-container&quot;&gt;$(samtools view -H $&lt;/span&gt;1 | grep -P '^@SQ' | cut -f 3 -d ':' | awk '{sum+=&lt;span class=&quot;math-container&quot;&gt;$1} END {print sum}')&#xA;echo $&lt;/span&gt;tot&#xA;&#xA;#compute the coverage at each point&#xA;samtools depth -ao cov_&lt;span class=&quot;math-container&quot;&gt;$1 $&lt;/span&gt;1 #your .bam file has depth calculated for each point&#xA;&#xA;#calculate the average coverage&#xA;sum=&lt;span class=&quot;math-container&quot;&gt;$(awk '{sum+=$3} END {print sum}' cov_$&lt;/span&gt;1)&#xA;echo $sum&#xA;echo&#xA;&#xA;avg=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;$&lt;/span&gt;sum/&lt;span class=&quot;math-container&quot;&gt;$tot&quot; | bc -l)&#xA;echo ''The average coverage of the sample $&lt;/span&gt;1 is $avg x.''&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The logic behind the calculation of tot is as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;samtools view -H &lt;/code&gt; gets the header of the bam file&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;grep -P&lt;/code&gt; gets all lines in this header with SQ&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;cut -f 3 -d ':'&lt;/code&gt; removes fields that are lists of more than 3 letters while -d uses delimiter ':' instead of tab and this leaves only the numbers i.e. lengths of the chromosomes&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;awk '{sum+=$1} END {tot=sum}'&lt;/code&gt; calculates the length/total bases in the chromosome&lt;/p&gt;&#xA;&lt;p&gt;However, my output seems incorrect. Any advice on where I might be making an error (in code or thought process), or a better method would be appreciated.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;One thing that worries me is that my tot variable is the total number of bases, so if the samtools depth function does not calculate coverage for each base then my sum and tot will not be in the same units and my calculated coverage will be wrong...&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="9134" LastEditorUserId="9134" LastEditDate="2020-07-04T13:09:17.303" LastActivityDate="2022-10-25T11:21:03.373" Title="Calculating average coverage for .bam files (sequence data)" Tags="|sequence-analysis|bash|coverage|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13723" PostTypeId="2" ParentId="13722" CreationDate="2020-07-04T13:26:23.603" Score="9" Body="&lt;p&gt;For a quick estimate you’re making it more complicated than necessary.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The theoretical average coverage is &lt;span class=&quot;math-container&quot;&gt;$\frac{n \cdot \hat l}{N}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of reads, &lt;span class=&quot;math-container&quot;&gt;$\hat l$&lt;/span&gt; is the average read length and &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the genome size.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;samtools idxstats&lt;/code&gt; gives you the chromosome lengths and number of mapped reads in one convenient list.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Putting this together, we get&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools idxstats data.bam \&#xA;| awk -vreadlen=120 '&#xA;    {&#xA;        len += $2&#xA;        nreads += $3&#xA;    }&#xA;    END {&#xA;        print nreads * readlen / len&#xA;    }&#xA;'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(In this case I’ve used 120 as the read length.)&lt;/p&gt;&#xA;&lt;p&gt;However, note that this over-estimates &lt;em&gt;empirical&lt;/em&gt; sequencing depth, mainly due to the fact that a given read may not be fully mapped; thus the the average read length is an upper bound on the average &lt;em&gt;mapped&lt;/em&gt; length. If you have an estimate for the average percentage of mapped bases per read, you can multiple that by the above coverage estimate for a more accurate number.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;If you need a more accurate estimate, your approach is fundamentally sound (and the result is probably correct!). For low-coverage data you can speed up your analysis by dropping the &lt;code&gt;-a&lt;/code&gt; argument to &lt;code&gt;samtools depth&lt;/code&gt;: you don’t need to record zero-coverage bases, they don’t contribute to the total. You also don’t need to store the coverage into an intermediate file, thus reducing IO cost:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sum=$(samtools depth &amp;quot;$1&amp;quot; | awk '{sum+=$3} END {print sum}')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Some more remarks on your code:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;code&gt;-P&lt;/code&gt; flag for &lt;code&gt;grep&lt;/code&gt; is not supported everywhere and thus limits portability, so you want to avoid using it if possible. It turns out that you don’t need it here: &lt;code&gt;grep '^@SQ'&lt;/code&gt; works just as well.&lt;/li&gt;&#xA;&lt;li&gt;While the philosophy of UNIX is based on solving problems by combining individual tools that each solve one part, it is sometimes convenient to merge individual steps:&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You can often combine &lt;code&gt;cut&lt;/code&gt; and &lt;code&gt;awk&lt;/code&gt;, in this case by setting the field separator:&#xA;&lt;pre&gt;&lt;code&gt;awk -vFS=: '{sum+=$3} END {print sum}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;You can often combine &lt;code&gt;grep&lt;/code&gt; and &lt;code&gt;awk&lt;/code&gt;:&#xA;&lt;pre&gt;&lt;code&gt;awk -vFS=: '/^@SQ/ {sum+=$3} END {print sum}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Always&lt;/em&gt; double-quote your shell variables, otherwise your script will break if the values contain unexpected spaces or other special characters. This is especially important for filenames, because spaces and other special characters &lt;em&gt;are valid characters in paths!&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;Your code is clean and readable. Well done.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2020-07-06T08:30:06.137" LastActivityDate="2020-07-06T08:30:06.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13724" PostTypeId="1" CreationDate="2020-07-04T20:18:24.997" Score="-2" ViewCount="731" Body="&lt;p&gt;I have around 300 paired-end files from 150 samples (1 forward and 1 reverse read for each sample). I want to merge (by sequence overlap) respective forward and reverse reads for each of the samples. The samples are named e.g. &lt;strong&gt;SRR2155174_1.fastq SRR2155174_2.fastq SRR2155319_1.fastq SRR2155319_2.fastq.&lt;/strong&gt; Can anyone please help me write a bash script to do this?&lt;/p&gt;&#xA;" OwnerUserId="9129" LastEditorUserId="77" LastEditDate="2020-07-07T07:08:44.543" LastActivityDate="2020-07-07T07:15:48.873" Title="Bash script for automatic overlap merging of respective paired-end reads from samples" Tags="|fastq|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13725" PostTypeId="2" ParentId="13289" CreationDate="2020-07-05T03:29:16.500" Score="1" Body="&lt;p&gt;I had a think about this one and am stumped. I thought there could be a connection with acute oak disease (AOD) or else chronic oak disease (COD), which are emergent infections threatening the population. However, they appear to be fungal and bacterial infections rather than viral.&lt;/p&gt;&#xA;&lt;p&gt;Single stranded RNA plant viruses are very common, more so than in mammals, but to achieve the results you are obtaining the trees would need to be really sick if the predominent RNA was viral. Short viral RNAs are produced regularly by viruses as defective RNA and its a major area of research, considering a mechanism of defective interference,&lt;/p&gt;&#xA;&lt;p&gt;In context it would be weird if this RNA swamped the tree/plant, you need an actively replicating virus for this stuff to appear. So I don't think its either viral or defective viral RNA.&lt;/p&gt;&#xA;&lt;p&gt;The antiviral response in mammals can involve RNase and this could explain your degradation. Essentially, mammals (probably vertebrates) kick-back hard when virus appears. However, I couldn't really find evidence of this in plants.&lt;/p&gt;&#xA;&lt;p&gt;I don't think the results you obtained have a clear biological explanation in pathogenesis. The best explanation is it was wet lab-technique, e.g. library construction resulted in overdigestion/fragmentation. The only thing I can suggest is template assembly using a normally distributed plant/oak RNA library.&lt;/p&gt;&#xA;&lt;p&gt;I was probably overthinking in forging a link with an emergent (tree) pathogen.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ayKYJ.jpg&quot; alt=&quot;enter image description here&quot; /&gt;!&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-05T12:31:34.910" LastActivityDate="2020-07-05T12:31:34.910" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13726" PostTypeId="5" CreationDate="2020-07-05T04:59:56.333" Score="0" Body="&lt;p&gt;Conda is automatically installed when either Anaconda 3 is installed or Miniconda. 'conda' is a notably preferred over 'pip', 'pip3' for Python package installation and management for at least numerous reasons, firstly dependencies tend to be installed, there is a nice GUI frontend which can be easily deployed within a given environment, a particularly important strategy in code development, and there is a conflict if both pip and conda are used.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-05T04:59:56.333" LastActivityDate="2020-07-05T04:59:56.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13727" PostTypeId="4" CreationDate="2020-07-05T04:59:56.333" Score="0" Body="Major method for Python package installation and management associated with Anaconda data science framework. " OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-05T04:59:56.333" LastActivityDate="2020-07-05T04:59:56.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13728" PostTypeId="5" CreationDate="2020-07-05T05:06:03.137" Score="0" Body="&lt;p&gt;Database focused on amino acids/ proteins curated to a high standard, generally it is a smaller database than many protein databases as a consequence of the manual/ high standard of curation. It can refer to a nomenclature system of motif delineation notably for proteins, but possibly may be applied to nucleic acids.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-05T05:06:03.137" LastActivityDate="2020-07-05T05:06:03.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13729" PostTypeId="4" CreationDate="2020-07-05T05:06:03.137" Score="0" Body="Database and protein motif nomenclature associated with Swiss-prot/Expasy" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-05T05:06:03.137" LastActivityDate="2020-07-05T05:06:03.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13730" PostTypeId="2" ParentId="13717" CreationDate="2020-07-05T10:37:58.103" Score="4" Body="&lt;p&gt;It's not appropriate to run &lt;code&gt;aov&lt;/code&gt; on a GLM model. &lt;code&gt;aov&lt;/code&gt; assumes normally distributed data and it runs on a GLM as if it was a least squares linear model. It uses the working responses and working weights from the GLM fit and performs an ordinary anova as if the responses were normally distributed.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, &lt;code&gt;anova&lt;/code&gt; is a generic function with a method for GLMs and it performs a sequential analysis of deviance.&lt;/p&gt;&#xA;&lt;p&gt;Purely statistical questions like this are best sent to CrossValidated, although they don't like questions that are posed entirely in terms of an R function. Questions purely about the functionality of functions in the core distribution of R are best sent to R-help, which is a mailing list rather than web forum (see &lt;a href=&quot;https://www.r-project.org/mail.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.r-project.org/mail.html&lt;/a&gt; and &lt;a href=&quot;https://stat.ethz.ch/mailman/listinfo/r-help&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://stat.ethz.ch/mailman/listinfo/r-help&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="9042" LastEditorUserId="9042" LastEditDate="2021-12-24T01:34:41.273" LastActivityDate="2021-12-24T01:34:41.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13731" PostTypeId="1" CreationDate="2020-07-05T15:07:25.583" Score="2" ViewCount="2830" Body="&lt;p&gt;I got a blast xml file after a DNA sequence search. The XML file is not very readable. What is the way to make it more understandable? This blast contains about twenty matches. &lt;a href=&quot;https://www.grosfichiers.com/Hm7h8kqcjW5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is the xml file&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9138" LastActivityDate="2020-07-06T02:04:14.933" Title="how to have a blast xml file in a readable and understandable way" Tags="|blast|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13732" PostTypeId="2" ParentId="13731" CreationDate="2020-07-05T17:47:55.637" Score="1" Body="&lt;p&gt;Is this what you are looking for: &lt;a href=&quot;https://codebeautify.org/xmlviewer&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://codebeautify.org/xmlviewer&lt;/a&gt; ? XML is a format designed to be readable firstly by machines, not humans.&lt;/p&gt;&#xA;&lt;p&gt;With xslt you can convert XML to various more readable formats see this blog for something that fits your XML: &lt;a href=&quot;http://plindenbaum.blogspot.com/2008/05/ncbi-blast-xslt-xhtml-svg.html?m=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://plindenbaum.blogspot.com/2008/05/ncbi-blast-xslt-xhtml-svg.html?m=1&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9131" LastActivityDate="2020-07-05T17:47:55.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13733" PostTypeId="2" ParentId="13731" CreationDate="2020-07-05T21:01:06.107" Score="2" Body="&lt;p&gt;The most human-readable blast output formats are 0-4, e.g. &lt;code&gt;-outfmt 0&lt;/code&gt; - which if I am not mistaken is the format used by web blast. However, these formats are a pain to automatically parse.&lt;/p&gt;&#xA;&lt;p&gt;As &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/13732/1642&quot;&gt;Cowboy_Patrick&lt;/a&gt; pointed out, XML is more of a machine-readable format. The tabular output format ( &lt;code&gt;-outfmt 6&lt;/code&gt;) is very commonly used, because it is easily parsed by unix command-line tools, and its structure also allows a somewhat easy human reading and understanding.&lt;/p&gt;&#xA;" OwnerUserId="1642" LastEditorUserId="1642" LastEditDate="2020-07-06T02:04:14.933" LastActivityDate="2020-07-06T02:04:14.933" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13734" PostTypeId="5" CreationDate="2020-07-05T21:51:40.513" Score="0" Body="&lt;p&gt;Many membrane proteins comprise at least one alpha helice domain enabling a protein to anchor into an animal cellular membrane. The 'anchor' can extensively 'spiral' in and out of the membrane, often supporting hydrophilic domains. The hydrophilic domains may exist within the cytoplasm side of the cell membrane or the external face, e.g. cellular receptors. Pathogens regularly deploy membrane proteins to anchor their B-cell antigenic domains into the pathogen lipid-bilayer as the method to attack the hosts cellular receptor.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-05T21:51:40.513" LastActivityDate="2020-07-05T21:51:40.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13735" PostTypeId="4" CreationDate="2020-07-05T21:51:40.513" Score="0" Body="A protein comprising at least one hybdrophobic domain which attaches/anchors within the lipid bilayer of an animal cell. A chanel protein is a special case of a membrane protein." OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-05T21:51:40.513" LastActivityDate="2020-07-05T21:51:40.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13736" PostTypeId="5" CreationDate="2020-07-05T21:58:27.937" Score="0" Body="&lt;p&gt;This describes a network for the intraspecific population structure or the mitochondrion, although has been used for bacterial genomes. The size of each 'node' within the network describes the allele frequency and the distance between nodes represents number of mutation differences. It is quite distinct and should not be confused with a phylogenetic tree.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-05T21:58:27.937" LastActivityDate="2020-07-05T21:58:27.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13737" PostTypeId="4" CreationDate="2020-07-05T21:58:27.937" Score="0" Body="A unrooted nodal network describing the intraspecific relationships using nucleotide mutations (alleles) and allele frequency, for eukaryotes this is constructed from mitochrondrial loci. " OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-05T21:58:27.937" LastActivityDate="2020-07-05T21:58:27.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13738" PostTypeId="1" CreationDate="2020-07-06T03:12:59.567" Score="0" ViewCount="1350" Body="&lt;p&gt;How can we get mapping from UniProt id to Entrez ID? I have UniProt id, but want to convert those ids to Entrez or Ensembl id.&lt;/p&gt;&#xA;" OwnerUserId="8895" LastEditorUserId="57" LastEditDate="2020-07-10T08:34:41.830" LastActivityDate="2020-07-10T08:34:41.830" Title="Mapping UniProt id to Entrez id" Tags="|r|identifiers|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13740" PostTypeId="1" AcceptedAnswerId="13753" CreationDate="2020-07-06T10:30:31.727" Score="2" ViewCount="735" Body="&lt;p&gt;I'm dealing with a very large and sparse dataset and the first issues I met occurred when I tried to use quickCluster that reported me this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;                'cannot allocate vector of size 156.6 Mb'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, given that I cannot wait to change the RAM of my computer and I can't afford to use a cluster, I want to rely on some other strategies like some package that would allow me to handle sparse matrices. I'm thinking about sparseM but given that I don't know well this package I'd like to know how to shrink the ram allocation for these kind of matrices. Any suggestion will be very appreciated!&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.gbmseq.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link to the dataset&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9142" LastEditorUserId="9142" LastEditDate="2020-07-06T10:43:17.413" LastActivityDate="2022-08-15T16:36:48.157" Title="How to reduce the occupied RAM when you are dealing with a very sparse matrix in a single-cell Experiment in R?" Tags="|rna-seq|scrnaseq|bioconductor|single-cell|scran|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13741" PostTypeId="2" ParentId="13740" CreationDate="2020-07-06T10:58:58.817" Score="1" Body="&lt;p&gt;Essentially you've hit a RAM bottleneck and the calculation will slow to zero, or in this instance refuse to go forward. The way to do this normally is to parallelize the calculation across the cores of your machine. This will likely remove the RAM bottlenack, dont ask me the computer archetectural reasons for why it works - but its works.&lt;/p&gt;&#xA;&lt;p&gt;However, my knowledge of R is minimal. I wouldn't know how to parallelize an R calculation. It is certainly doable in Perl and Python, but the calculation needs to be written to ensure parallelisation.&lt;/p&gt;&#xA;&lt;p&gt;The other way in is to reconfigure your calculation to remove sparse matrices OR find someone doing NGS where they've confgured their machine around heavy RAM.&lt;/p&gt;&#xA;&lt;p&gt;Looking at your calculation I don't quite understand why you need to use a specific package, it looks like unsupervised machine learning, PCA - tSNE that sort of thing and you don't need a given package to do that you just need to vectorise the inputs. If you worked out the statistical components of &lt;code&gt;Scran&lt;/code&gt; then there are a few extremely strong R statisticians/bioinformaticians on site who would have zero problem in replicating this within a few lines of code. Its not hard in Python's Sci-kit learn either. At a guess they perform PCA and resolve it via tSNE and this give nice clear clusters.&lt;/p&gt;&#xA;&lt;p&gt;GCP is free for 3 months so it will cost you zero in context to a single calculation.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-06T11:35:32.177" LastActivityDate="2020-07-06T11:35:32.177" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13742" PostTypeId="1" AcceptedAnswerId="13749" CreationDate="2020-07-06T11:00:49.553" Score="1" ViewCount="190" Body="&lt;p&gt;My workflow until now:&lt;/p&gt;&#xA;&lt;p&gt;Find fragments of a marker gene in unassembled metagenomes &amp;gt; download and assemble metagenomes &amp;gt; recover the gene neighborhood / gene set of interest&lt;/p&gt;&#xA;&lt;p&gt;Right now I have a rough estimate of how abundant these genes are by the 'depth' noted on the assembled contigs (I use MEGAHIT for assembly). I was wondering if there's a more thorough/proper way to do this. I would like to compare the abundance of specific genes between a) samples in the same study, and b) different studies. I imagine that the size of the individual metagenomes should be considered in both cases, but point b) might add additional difficulties such as different sequencing techniques. I would appreciate your insights.&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2020-07-06T18:12:32.373" Title="Comparing gene abundances between metagenomes" Tags="|gene|assembly|metagenome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13744" PostTypeId="2" ParentId="13740" CreationDate="2020-07-06T11:46:51.167" Score="3" Body="&lt;p&gt;Not a direct solution but some workarounds:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;As far as I know, &lt;code&gt;Seurat&lt;/code&gt; can work with sparse matrices.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The particular function of &lt;code&gt;scran&lt;/code&gt; that you are using eats up quite some memory. I believe it is needed for the &amp;quot;normalization&amp;quot; step (that is how I used it anyway). Whereas the scaling normalization performed by this function is superior to crude &amp;quot;log normalization&amp;quot;, you can give a try with the latter, which is far less computationally-intensive (does not do clustering). Seurat, once again, can help with this.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can downsample your data to the extent that it fits your into your RAM.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You can give a try with Python. More and more single cell packages are written in Python, to some extent because of the problem you have experienced. For example &lt;code&gt;Scanpy&lt;/code&gt; output is comparable to that of &lt;code&gt;Seurat&lt;/code&gt;, although I am not sure if you can use scaling normalization with &lt;code&gt;Scanpy&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2020-07-06T13:53:31.817" LastActivityDate="2020-07-06T13:53:31.817" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13745" PostTypeId="1" AcceptedAnswerId="13971" CreationDate="2020-07-06T11:52:29.307" Score="1" ViewCount="137" Body="&lt;p&gt;I have a set of 4 amino acids which i have aligned,but i want to compare then with respective  nucleotide sequence in terms of change codon level.&lt;/p&gt;&#xA;&lt;p&gt;I have the alignment file where i want to take only for the C-terminal domain region region only the CTD starts with amino acids &lt;code&gt;“NITNLC”&lt;/code&gt; till it ends with &lt;code&gt;“HAPATV”&lt;/code&gt; and rest of the sequence i dont want to take.&lt;/p&gt;&#xA;&lt;p&gt;In short i want to take the part sequence which starts from &amp;quot;NITNLC&amp;quot; to &amp;quot;HAPATV&amp;quot; then i would like to compare what are the changes in the nucleotide sequence at codon level.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;To start with I have used the seqinr library to generate nucleotide alignment from aligned protein sequence.&lt;/p&gt;&#xA;&lt;p&gt;reverse.align {seqinr}&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Then the next step that i want to compare each of my amino acid sequence to my reference sequence which in my case is &lt;code&gt;P0DTC2&lt;/code&gt;to rest of the sequence for example I want to compare &lt;code&gt;P0DTC2&lt;/code&gt; to &lt;code&gt;K9N5Q8&lt;/code&gt;to the above mentioned CTD region which starts from &lt;code&gt;NITNLC&lt;/code&gt; and ends with &lt;code&gt;HAPATV&lt;/code&gt;and find what are the codon level changes if there is any I would like to report both in amino acid level and codon level.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;First part I guess my approach is correct. Second part I'm not sure how to proceed i guess it's much more than simple parsing I guess!!.&lt;/p&gt;&#xA;&lt;p&gt;Any help or suggestion would be really appreciated ,and if there exist a R based solution that would be really welcome.&lt;/p&gt;&#xA;&lt;p&gt;Files&#xA;&lt;a href=&quot;https://drive.google.com/file/d/1J4-eE99I6MQgNw6m6sFE60Qrt_4Bg7KJ/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Amino acid file&lt;/a&gt; &lt;a href=&quot;https://drive.google.com/file/d/14cyaxEq84fYlHKbLjZmIhvGf60gakBKB/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;nucleotide file&lt;/a&gt; &lt;a href=&quot;https://drive.google.com/file/d/1FX-VAQixASrmiYZc-mIdHHOnwGtGZJ_j/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Aligned amino acid file&lt;/a&gt; &lt;a href=&quot;https://drive.google.com/file/d/1dyQHqYEj5TXt1CBGwieRG48odghBIYTq/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reverse alignment nucleotide file using seqinr&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2020-07-23T13:16:06.813" Title="Comparing aligned amino acids to codon" Tags="|r|sequence-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13746" PostTypeId="5" CreationDate="2020-07-06T13:08:48.440" Score="0" Body="&lt;p&gt;The lipid bilayer is central to animal membranes comprising phospho-lipids forming a dynamic structure into which transmembrane, membrane proteins and protein membrane channels are located. In viruses it is the singular component of the membrane/envelope and again not to be confused with either the membrane or envelope proteins. The viral membrane/envelope is formed from the host phospho-lipid membrane. Not all viruses contain an envelope.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-06T13:08:48.440" LastActivityDate="2020-07-06T13:08:48.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13747" PostTypeId="4" CreationDate="2020-07-06T13:08:48.440" Score="0" Body="A key component of the cellular membrane in animals comprising phosph-lipids. The central compoent of a viral membrane, but not to be confused with the membrane/envelope protein in viruses, which is a protein anchored into the lipid bilayer" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-06T13:08:48.440" LastActivityDate="2020-07-06T13:08:48.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13749" PostTypeId="2" ParentId="13742" CreationDate="2020-07-06T18:12:32.373" Score="2" Body="&lt;p&gt;I would avoid using assemblies to answer this question, as there's no guarantee that you will be able to assemble your genes of interest; you can however estimate their abundance even if they are relatively rare.&lt;/p&gt;&#xA;&lt;p&gt;How I understand your question as being one of estimating the abundance of either some specific genes (e.g. butyrate metabolism genes) or &lt;em&gt;all&lt;/em&gt; genes in a microbial community across multiple samples for comparative purposes. In other words, not 16S or marker gene analysis for the purposes of estimating organismal abundance, which is a rather different problem (though in that case I would still not use an assembly).&lt;/p&gt;&#xA;&lt;p&gt;A more standard workflow is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;align metagenomic reads against some existing database of genes annotated appropriately.&lt;/li&gt;&#xA;&lt;li&gt;estimate the number of reads aligning against some gene or ortholog in some way (using for example KEGG Orthology or similar).&lt;/li&gt;&#xA;&lt;li&gt;use counts from (2) as input to some statistical procedure, possibly summarizing across functional categories.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Some examples of how this has been done are &lt;a href=&quot;https://www.nature.com/articles/nature11450&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;http://borensteinlab.com/pub/keto_prast_nbm.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;http://borensteinlab.com/pub/funcvar_manor_mb.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I am sure that there are more recent/relevant references but I haven't been following the field closely in the last few years.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-07-06T18:12:32.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13750" PostTypeId="2" ParentId="13738" CreationDate="2020-07-06T19:18:37.293" Score="2" Body="&lt;p&gt;The Retrieve/ID mapping tool from Uniprot allows you to convert from Uniprot to entrez ID and vice-versa. It is available &lt;a href=&quot;https://www.uniprot.org/uploadlists/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Just select the appropriate values in the &lt;code&gt;2.Selection options&lt;/code&gt; panel.&lt;/p&gt;&#xA;" OwnerUserId="8749" LastActivityDate="2020-07-06T19:18:37.293" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13751" PostTypeId="5" CreationDate="2020-07-06T19:31:32.013" Score="0" Body="&lt;p&gt;Molecular clocks are concerned with identifying the rate of mutation per unit time and used in dating studies. The simplist molecular clock is to count all mutations in a phylogenetic tree and assume a single rate of mutation across all taxa and is known as a global clock. Global clocks rarely hold, due to mutation rate heterogeneity. Tip-dating is a popular mthod of dating in viruses, but eukaryotic molecular clocks require a calibration point within the tree. This is a date that conincides with a node in the phylogeny. Beast and PhyML are the two current packages widely associated with molecular clock dating.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-06T19:31:32.013" LastActivityDate="2020-07-06T19:31:32.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13752" PostTypeId="4" CreationDate="2020-07-06T19:31:32.013" Score="0" Body="Phylogenetic method to define a number of mutations (DNA or amino acid) per unit time. There are several methodological developments in this calculation. For RNA viruses this is defined as RNA or amino acid because the unit of inheritance is an RNA genome." OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-06T19:31:32.013" LastActivityDate="2020-07-06T19:31:32.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13753" PostTypeId="2" ParentId="13740" CreationDate="2020-07-06T21:08:22.150" Score="5" Body="&lt;p&gt;Ah, looks like I can't even procrastinate on StackExchange anymore without seeing work-related stuff. Oh well.&lt;/p&gt;&#xA;&lt;p&gt;Anyway, the other answers and comments are way off. &lt;strong&gt;scran&lt;/strong&gt; has supported sparse matrices for years, ever since we switched over to the &lt;code&gt;SingleCellExperiment&lt;/code&gt; class as our basic data structure. &lt;code&gt;quickCluster&lt;/code&gt; does no coercion to dense format unless you tell it so explicitly, e.g., with &lt;code&gt;use.ranks=TRUE&lt;/code&gt; (in which case you're asking for ranks, so there's little choice &lt;em&gt;but&lt;/em&gt; to collapse to a dense matrix).&lt;/p&gt;&#xA;&lt;p&gt;You don't provide an MWE or your session information, but this is how it rolls for me:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# Using the raw counts in the linked dataset. Despite being&#xA;# called a CSV, it's actually space delimited... typical.&#xA;library(scater)&#xA;mat &amp;lt;- readSparseCounts(&amp;quot;GBM_raw_gene_counts.csv&amp;quot;, sep=&amp;quot; &amp;quot;)&#xA;&#xA;# Making an SCE just for fun. Not strictly necessary for&#xA;# this example, but you'll find it useful later. &#xA;sce &amp;lt;- SingleCellExperiment(list(counts=mat))&#xA;&#xA;library(scran)&#xA;system.time(clust &amp;lt;- quickCluster(sce))&#xA;##   user  system elapsed&#xA;##  3.170   0.174   3.411&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is running on my laptop - 16 GB RAM but I'm definitely not using all of it. I only go full throttle when I'm working on some real data, e.g., the 300k HCA bone marrow dataset. Check out the &lt;a href=&quot;https://osca.bioconductor.org/&quot; rel=&quot;noreferrer&quot;&gt;book&lt;/a&gt; for more details.&lt;/p&gt;&#xA;&lt;p&gt;Session info below, I don't know quite enough SO-fu to know collapse it.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;R version 4.0.0 Patched (2020-04-27 r78316)&#xA;Platform: x86_64-apple-darwin17.7.0 (64-bit)&#xA;Running under: macOS High Sierra 10.13.6&#xA;&#xA;Matrix products: default&#xA;BLAS:   /Users/luna/Software/R/R-4-0-branch/lib/libRblas.dylib&#xA;LAPACK: /Users/luna/Software/R/R-4-0-branch/lib/libRlapack.dylib&#xA;&#xA;locale:&#xA;[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8&#xA;&#xA;attached base packages:&#xA;[1] parallel  stats4    stats     graphics  grDevices utils     datasets&#xA;[8] methods   base&#xA;&#xA;other attached packages:&#xA; [1] scran_1.16.0                scater_1.16.1&#xA; [3] ggplot2_3.3.2               SingleCellExperiment_1.10.1&#xA; [5] SummarizedExperiment_1.18.1 DelayedArray_0.14.0&#xA; [7] matrixStats_0.56.0          Biobase_2.48.0&#xA; [9] GenomicRanges_1.40.0        GenomeInfoDb_1.24.2&#xA;[11] IRanges_2.22.2              S4Vectors_0.26.1&#xA;[13] BiocGenerics_0.34.0&#xA;&#xA;loaded via a namespace (and not attached):&#xA; [1] beeswarm_0.2.3            statmod_1.4.34&#xA; [3] tidyselect_1.1.0          locfit_1.5-9.4&#xA; [5] purrr_0.3.4               BiocSingular_1.4.0&#xA; [7] lattice_0.20-41           colorspace_1.4-1&#xA; [9] vctrs_0.3.1               generics_0.0.2&#xA;[11] viridisLite_0.3.0         rlang_0.4.6&#xA;[13] pillar_1.4.4              glue_1.4.1&#xA;[15] withr_2.2.0               BiocParallel_1.22.0&#xA;[17] dqrng_0.2.1               GenomeInfoDbData_1.2.3&#xA;[19] lifecycle_0.2.0           zlibbioc_1.34.0&#xA;[21] munsell_0.5.0             gtable_0.3.0&#xA;[23] rsvd_1.0.3                vipor_0.4.5&#xA;[25] irlba_2.3.3               BiocNeighbors_1.6.0&#xA;[27] Rcpp_1.0.4.6              edgeR_3.30.3&#xA;[29] scales_1.1.1              limma_3.44.3&#xA;[31] XVector_0.28.0            gridExtra_2.3&#xA;[33] dplyr_1.0.0               grid_4.0.0&#xA;[35] tools_4.0.0               bitops_1.0-6&#xA;[37] magrittr_1.5              RCurl_1.98-1.2&#xA;[39] tibble_3.0.1              crayon_1.3.4&#xA;[41] pkgconfig_2.0.3           ellipsis_0.3.1&#xA;[43] Matrix_1.2-18             DelayedMatrixStats_1.10.0&#xA;[45] ggbeeswarm_0.6.0          viridis_0.5.1&#xA;[47] R6_2.4.1                  igraph_1.2.5&#xA;[49] compiler_4.0.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9146" LastActivityDate="2020-07-06T21:08:22.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13754" PostTypeId="5" CreationDate="2020-07-06T21:11:42.557" Score="0" Body="&lt;p&gt;HPC refers to any computer system specifically designed for number crunching. A typical system would compromise a head node and a series of 'minor' nodes (formally referred to as 'slave' nodes). The head node would distribute jobs to the minor nodes arranged in a cluster and connected via a queuing system. Modern HPC often comprise an advanced graphics processing unit (GPU) and used in deep learning calculations, but traditional CPU clusters are equally considered HPC.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-06T21:11:42.557" LastActivityDate="2020-07-06T21:11:42.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13755" PostTypeId="4" CreationDate="2020-07-06T21:11:42.557" Score="0" Body="Acronym for &quot;high performance computing&quot;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-06T21:11:42.557" LastActivityDate="2020-07-06T21:11:42.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13756" PostTypeId="5" CreationDate="2020-07-06T21:15:02.647" Score="0" Body="&lt;p&gt;Python module used for a limited range of R style calculations, such as regression based mathematics, but excludes graphing and machine learning learning calculations which are retained within the Matplotlib and sci-kit learn module repectively.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-06T21:15:02.647" LastActivityDate="2020-07-06T21:15:02.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13757" PostTypeId="4" CreationDate="2020-07-06T21:15:02.647" Score="0" Body="Refers to a Python module used for scientific computing, hence scipy" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-06T21:15:02.647" LastActivityDate="2020-07-06T21:15:02.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13758" PostTypeId="1" AcceptedAnswerId="13759" CreationDate="2020-07-06T21:41:41.053" Score="1" ViewCount="387" Body="&lt;p&gt;I currently am using this code to remove duplicate sequences from the fasta file. However, I would also like to write a new file with only the removed duplicates as well as a count for how many times they appear. Please let me know if you have any suggestions, I am hoping to keep it written in one py file, but I am not opposed to running a second file to do so. I attended to write the appended strings to a new file but it looked slightly off...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;import time&#xA;&#xA;start = time.time() &#xA;&#xA;seen = []&#xA;records = []&#xA;&#xA; for record in SeqIO.parse(&amp;quot;b4r2.fasta&amp;quot;, &amp;quot;fasta&amp;quot;):  &#xA;   if str(record.seq) not in seen:&#xA;       seen.append(str(record.seq))&#xA;   records.append(record)&#xA;&#xA;&#xA;#writing to a fasta file&#xA;SeqIO.write(records, &amp;quot;no_dupes_b4r2.fasta&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;end = time.time()&#xA;&#xA;print(f&amp;quot;Run time is {(end- start)/60}&amp;quot;) &#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9120" LastActivityDate="2020-07-07T22:37:00.690" Title="Python: How to write duplicate sequences removed from fasta file to new file" Tags="|fasta|python|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13759" PostTypeId="2" ParentId="13758" CreationDate="2020-07-06T21:56:41.063" Score="3" Body="&lt;p&gt;&lt;strong&gt;Edit: user @RamRS has a more robust answer for actually using this. Please use that. Leaving answer up as OP appears to be interested in implementing as a learning activity.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Please let us know why this is not doing what you intend.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I haven't checked it, but something like this would work:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;import time&#xA;from collections import defaultdict&#xA;&#xA;start = time.time() &#xA;&#xA;seq_counts = defaultdict(0)&#xA;records = []&#xA;&#xA; for record in SeqIO.parse(&amp;quot;b4r2.fasta&amp;quot;, &amp;quot;fasta&amp;quot;):  &#xA;   seq = str(record.seq)&#xA;   if seq in seq_counts:  # python is smart about dict keys&#xA;       records.append(record)&#xA;&#xA;   seq_counts[seq] += 1&#xA;&#xA;&#xA;#writing to a fasta file&#xA;SeqIO.write(records, &amp;quot;no_dupes_b4r2.fasta&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;end = time.time()&#xA;&#xA;# print the counts&#xA;for seq in seq_counts:&#xA;    print(seq_counts[seq], seq)&#xA;&#xA;print(f&amp;quot;Run time is {(end- start)/60}&amp;quot;) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It will be a bit slow for large fasta files and I am sure that some people will be very upset that I'm using a defaultdict for this.&lt;/p&gt;&#xA;&lt;p&gt;The code that you have would probably not work as you would want the &lt;code&gt;records.append()&lt;/code&gt; call to be only run in the if statement testing for non-membership.&lt;/p&gt;&#xA;&lt;p&gt;There is absolutely no reason to have to write more than one script for this.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2020-07-07T22:37:00.690" LastActivityDate="2020-07-07T22:37:00.690" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="13760" PostTypeId="5" CreationDate="2020-07-06T23:12:51.017" Score="0" Body="&lt;p&gt;Point mutation is a single mutation which may or may not cause a change in the amino acid. A point mutation is identified as a change at a given homologous position in alignment of DNA sequence. Phylogenetic trees are calculated of DNA (or RNA for RNA viruses) data are constructed by modelling of point mutations.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-06T23:12:51.017" LastActivityDate="2020-07-06T23:12:51.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13761" PostTypeId="4" CreationDate="2020-07-06T23:12:51.017" Score="0" Body="Single mutation in the DNA or RNA for RNA viruses, also known as single nucleotide polymorphism (SNPs)" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-06T23:12:51.017" LastActivityDate="2020-07-06T23:12:51.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13762" PostTypeId="1" CreationDate="2020-07-06T23:36:55.270" Score="0" ViewCount="46" Body="&lt;p&gt;I am trying to calculate the total exonic length (in bp) in order to see where the coding-noncoding ratio of roughly 1% to 99% comes from.&#xA;We know all chromosomes total about 3 billion base pairs. So Is it correct to assume that coding regions would total about 30 million base pairs? I am looking at the numbers in the sources below, but nothing there adds up...&lt;/p&gt;&#xA;&lt;p&gt;Take numbers from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genome/annotation_euk/Homo_sapiens/108/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI annothation&lt;/a&gt; for example; specifically&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UBhV4.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UBhV4.jpg&quot; alt=&quot;NCBI exon length&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;no matter how i count i can't get the expected ratio... What am  doing wrong?&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.encodeproject.org/report/?type=Gene&amp;amp;organism.scientific_name=Homo%20sapiens&amp;amp;searchTerm=gene&amp;amp;field=symbol&amp;amp;field=notes&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.encodeproject.org/report/?type=Gene&amp;amp;organism.scientific_name=Homo%20sapiens&amp;amp;searchTerm=gene&amp;amp;field=symbol&amp;amp;field=notes&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://archive.vn/20130414235101/http://useast.ensembl.org/Homo_sapiens/Location/Chromosome?r=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://archive.vn/20130414235101/http://useast.ensembl.org/Homo_sapiens/Location/Chromosome?r=1&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9145" LastActivityDate="2020-07-06T23:36:55.270" Title="Coding vs non-coding DNA length" Tags="|genome|phylogenetics|genetics|exon|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13763" PostTypeId="2" ParentId="13724" CreationDate="2020-07-07T07:15:48.873" Score="1" Body="&lt;p&gt;Your original question mentioned concatenating things, which is not what you want (that would result in all of the reads from the _2 files being after all of those in _1). What you want is to merge overlapping paired end reads. There are a variety of tools available for that, from &lt;a href=&quot;https://ccb.jhu.edu/software/FLASH/&quot; rel=&quot;nofollow noreferrer&quot;&gt;flash&lt;/a&gt; to &lt;a href=&quot;https://jgi.doe.gov/data-and-tools/bbtools/bb-tools-user-guide/bbmerge-guide/&quot; rel=&quot;nofollow noreferrer&quot;&gt;bbmerge&lt;/a&gt;. The general structure then is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for R1 in *_1.fastq ; do&#xA;    sampleName=${R1%%_1.fastq}&#xA;    some command with ${sampleName}_1.fastq and ${sampleName}_2.fastq producing ${sampleName}.fastq&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can figure out the &lt;code&gt;some command ...&lt;/code&gt; portion yourself.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-07-07T07:15:48.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13765" PostTypeId="5" CreationDate="2020-07-07T13:06:13.940" Score="0" Body="&lt;p&gt;For Python programming Jupyter notebook is a reasonable IDE. It is generally used in data science to code on a remote machine particularly using one of the major providers such as GCP (Google Cloud) or AWS (Amazon's version) for deep learning or else machine learning. GCP and AWS have the latest GPU machines which are required in deep learning/ machine learning based calculations.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T13:06:13.940" LastActivityDate="2020-07-07T13:06:13.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13766" PostTypeId="4" CreationDate="2020-07-07T13:06:13.940" Score="0" Body="Jupyter notebook is a widely used IDE in data science particularly used for Python, but can be used for any language." OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T13:06:13.940" LastActivityDate="2020-07-07T13:06:13.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13768" PostTypeId="5" CreationDate="2020-07-07T13:10:10.420" Score="0" Body="&lt;p&gt;The frequency of a given allele is essential in determining the intraspecific population structure. This is particularly true for species that actively recombine using Mendellian or alternative mechanisms and therefore applies to most (but not all) eukaryotes and many bacteria. However, it can also be used to understand the population structure of predominantly clonal organisms.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T13:10:10.420" LastActivityDate="2020-07-07T13:10:10.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13769" PostTypeId="4" CreationDate="2020-07-07T13:10:10.420" Score="0" Body="Allele frequency is the foundational basis of the vast majority of population genetic calculations" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T13:10:10.420" LastActivityDate="2020-07-07T13:10:10.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13770" PostTypeId="5" CreationDate="2020-07-07T13:58:40.213" Score="0" Body="&lt;p&gt;Comprising alpha to gamma Coronaviruses, the most notorious viruses being in order of emergence Severe Acute Respiratory Syndrome (SARS), Middle Eastern Respiratory Syndrome and SARS-CoV-2 also known as COVID-19. Genetically they are distinguished by their exceptionally large genomes for RNA viruses of around 30Kb, whereas most RNA viruses are around 10Kb&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T13:58:40.213" LastActivityDate="2020-07-07T13:58:40.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13771" PostTypeId="4" CreationDate="2020-07-07T13:58:40.213" Score="0" Body="Large family of single stranded, positive sense RNA viruses comprising SARS and SARS-CoV-2 (COVID-19)" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T13:58:40.213" LastActivityDate="2020-07-07T13:58:40.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13773" PostTypeId="5" CreationDate="2020-07-07T15:58:20.313" Score="0" Body="&lt;p&gt;Virulign is based on Libin et al (2019) found here &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/30295730/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pubmed.ncbi.nlm.nih.gov/30295730/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T15:58:20.313" LastActivityDate="2020-07-07T15:58:20.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13774" PostTypeId="4" CreationDate="2020-07-07T15:58:20.313" Score="0" Body="A codon based alignment editor for viruses" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T15:58:20.313" LastActivityDate="2020-07-07T15:58:20.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13775" PostTypeId="5" CreationDate="2020-07-07T16:00:30.697" Score="0" Body="&lt;p&gt;Windows based sequence alignment editor which will hand ABI Sanger sequencing output. Largely superseded by Sequtron (macOS), MEGA (any platform Windows included), or Jalview (Linux).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="1157" LastEditDate="2020-12-12T16:57:45.080" LastActivityDate="2020-12-12T16:57:45.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13776" PostTypeId="4" CreationDate="2020-07-07T16:00:30.697" Score="0" Body="Traditional sequence alignment and Sanger sequencing editor" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T16:00:30.697" LastActivityDate="2020-07-07T16:00:30.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13777" PostTypeId="5" CreationDate="2020-07-07T16:05:31.947" Score="0" Body="&lt;p&gt;Currently in version 3, very powerful package for all Python dependencies using 'conda' for maintence, installation, as well as ease of creation and maintence of a given environment. Highly recommended and uses standard installations such Spyder (IDE), Visual Studio and Jupyter Notebook. Largely incompatible with pip and pip3 installation.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T16:05:31.947" LastActivityDate="2020-07-07T16:05:31.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13778" PostTypeId="4" CreationDate="2020-07-07T16:05:31.947" Score="0" Body="Power all inclusive package used in data science for managing Python and its dependencies" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T16:05:31.947" LastActivityDate="2020-07-07T16:05:31.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13779" PostTypeId="5" CreationDate="2020-07-07T16:08:18.503" Score="0" Body="&lt;p&gt;Cloud platform for high performance computing (HPC) and also webhosting. Not as cheap for HCP as Google Cloud Platform (GCP).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T16:08:18.503" LastActivityDate="2020-07-07T16:08:18.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13780" PostTypeId="4" CreationDate="2020-07-07T16:08:18.503" Score="0" Body="Acronym for Amazon Webservices" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T16:08:18.503" LastActivityDate="2020-07-07T16:08:18.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13781" PostTypeId="5" CreationDate="2020-07-07T17:02:12.197" Score="0" Body="&lt;p&gt;The study of disease transmission within a population. Today this incorporates epizootiology and epidemiology under the singular term 'epidemiology', but historically this referred to zoonosis and clinical infections respectively.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T17:02:12.197" LastActivityDate="2020-07-07T17:02:12.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13782" PostTypeId="4" CreationDate="2020-07-07T17:02:12.197" Score="0" Body="A composition of two Greek words 1) demos approximating to 'people' or more specifically 'population' and 2) epi as in epidemic " OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T17:02:12.197" LastActivityDate="2020-07-07T17:02:12.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13783" PostTypeId="5" CreationDate="2020-07-07T17:05:20.477" Score="0" Body="&lt;p&gt;'efetch' is a major frontend controlling access to Genbank and extremely powerful commandline system for automating sequence retrieval and associated data. The commands are not necessarily userfriendly or intuitive, but alot on this site understand the relevant commands to achieve a specific goal.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T17:05:20.477" LastActivityDate="2020-07-07T17:05:20.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13784" PostTypeId="4" CreationDate="2020-07-07T17:05:20.477" Score="0" Body="NCBI's system for Genbank sequence retrieval and associated isolate data" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T17:05:20.477" LastActivityDate="2020-07-07T17:05:20.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13785" PostTypeId="1" CreationDate="2020-07-07T17:35:53.237" Score="1" ViewCount="130" Body="&lt;p&gt;I would like to use the python wrapper for packmol (MDAPackmol) for my research. In order to do so, I have tried to work with the example code and pdb files provided on &lt;a href=&quot;https://github.com/MDAnalysis/MDAPackmol&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt; so I can gain an understanding of the how to use the wrapper. However, when I run the code, I receive the following error message: &lt;code&gt;ValueError: invalid literal for int() with base 10: '' &lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here is the complete trace back.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Sa41Z.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Sa41Z.png&quot; alt=&quot;&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The error originates at &lt;code&gt;start.resname&lt;/code&gt; being a zero-length string. It seems to be a problem with the resname having a number and a letter and thus is a problem with &lt;code&gt;Universe&lt;/code&gt; (the object returned by &lt;code&gt;load_packmol_output&lt;/code&gt;). Thus, there is a problem with the residues. Any advice or insight would be greatly appreciated. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="9152" LastEditorUserId="9152" LastEditDate="2020-07-09T12:56:15.490" LastActivityDate="2020-07-09T12:56:15.490" Title="Question about Using MDAPackmol (packmol in python)" Tags="|python|computational-biochemistry|cheminformatics|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="13786" PostTypeId="5" CreationDate="2020-07-07T19:31:20.313" Score="0" Body="&lt;p&gt;Extremely powerful module enabling large data sets to be manipulated with ease. It contains database functions to facilitate this purpose, data pivoting and indexing. The closest comparison to the pandas 'dataframe' would be R's 'dataframe'.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T19:31:20.313" LastActivityDate="2020-07-07T19:31:20.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13787" PostTypeId="4" CreationDate="2020-07-07T19:31:20.313" Score="0" Body="Python module widely used in data science designed around database commands and calculations" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T19:31:20.313" LastActivityDate="2020-07-07T19:31:20.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13788" PostTypeId="5" CreationDate="2020-07-07T19:33:42.977" Score="0" Body="&lt;p&gt;The merge function is used in R and pandas to combine two dataframes around a single key common to both dataframes.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T19:33:42.977" LastActivityDate="2020-07-07T19:33:42.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13789" PostTypeId="4" CreationDate="2020-07-07T19:33:42.977" Score="0" Body="As a command 'merge' relates to merging two dataframes in either R or Python's pandas" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T19:33:42.977" LastActivityDate="2020-07-07T19:33:42.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13790" PostTypeId="5" CreationDate="2020-07-07T19:38:05.383" Score="0" Body="&lt;p&gt;An important term in genomics with the advent of whole genome sequencing and generally refers to the number of copies of a gene in a genome. In parasites this multi-copy genes are widespread and facilitate convergent evolution (evolution by recombination not associated with Mendelian inheritance). High gene copy number can lead to high levels of gene expression. Copy number transcripts are an alternative definition and can be subject to very difference control mechanisms.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T19:38:05.383" LastActivityDate="2020-07-07T19:38:05.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13791" PostTypeId="4" CreationDate="2020-07-07T19:38:05.383" Score="0" Body="The number of copies of a gene or transcript present in a genome or cell" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T19:38:05.383" LastActivityDate="2020-07-07T19:38:05.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13792" PostTypeId="5" CreationDate="2020-07-07T19:40:49.773" Score="0" Body="&lt;p&gt;In phylogenetic bioinformatics positive selecttion has a strict technical definition meaning that the number of mutations resulting in an amino acid mutation exceeds the number of mutations which are silent (synonymous). In general Darwinian evolution it refers to a mutation resulting in a fitness advantage to the organism.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T19:40:49.773" LastActivityDate="2020-07-07T19:40:49.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13793" PostTypeId="4" CreationDate="2020-07-07T19:40:49.773" Score="0" Body="Technical description in evoluionary biology where a mutation results in a fitness advantage" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T19:40:49.773" LastActivityDate="2020-07-07T19:40:49.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13794" PostTypeId="5" CreationDate="2020-07-07T19:47:01.180" Score="0" Body="&lt;p&gt;Vast biological kingdom comprising an enormous diversity of genome archectures which are either a DNA or RNA unit of replication that is either single or double stranded. Moreover, the genome can be segments (such as influenza virus) or single stranded (such as coronavirus). Furthermore, RNA viruses can be postivie or negatively stranded. RNA viruses are notorious for mutating very rapidly.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T19:47:01.180" LastActivityDate="2020-07-07T19:47:01.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13795" PostTypeId="4" CreationDate="2020-07-07T19:47:01.180" Score="0" Body="Biological kingdom of 'microparasite' that is unable to replicate without hijacking the hosts replication machinery" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T19:47:01.180" LastActivityDate="2020-07-07T19:47:01.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13796" PostTypeId="5" CreationDate="2020-07-07T19:52:36.777" Score="0" Body="&lt;p&gt;In phylogenetics this describes the frequency of A, G, T and C mutations as a 2 x 2 matrix which is either reversible A &amp;lt;-&amp;gt; C occurs at the same frequency as C &amp;lt;-&amp;gt; A or directional where the two frequencies are independent. Common models include Kimura's 2 parameter model combining an estimate of reversion mutation with the frequency of pyrimidine to pyrimidine (or purine to purine) mutations in comparison to pyrimidine to purine or purine to pyrimidine mutations, denoted as transitions and transversions respectively.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T19:52:36.777" LastActivityDate="2020-07-07T19:52:36.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13797" PostTypeId="4" CreationDate="2020-07-07T19:52:36.777" Score="0" Body="An important basis of modern phylogenetics describing the mathematical basis of mutation" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T19:52:36.777" LastActivityDate="2020-07-07T19:52:36.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13798" PostTypeId="5" CreationDate="2020-07-07T19:59:21.880" Score="0" Body="&lt;p&gt;Large and perhaps the oldest disapline in genetics stemming from Gregory Medel's work on inheritance which became the basis for the Hardy-Weinberg equiilibrium. The modern approach is to examine the frequency of alleles to define a &amp;quot;population&amp;quot;, which for Mendellian eukaryotes defines a group of freely reproducing individuals. Extremely useful series of approach to define genetic migration and genetic variation, such as Fst, Fis, and linkage disequilibrium. Mathematically, it is a very different approach to phylogenetics.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T19:59:21.880" LastActivityDate="2020-07-07T19:59:21.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13799" PostTypeId="4" CreationDate="2020-07-07T19:59:21.880" Score="0" Body="Disapline mostly concerned with allele frequencies to define the intraspecific (intra-species) population structure, but can be applied to phenotypic outcome, e.g. Medel's pea plants" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-07T19:59:21.880" LastActivityDate="2020-07-07T19:59:21.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13800" PostTypeId="1" CreationDate="2020-07-07T20:57:32.527" Score="0" ViewCount="305" Body="&lt;p&gt;I've downloaded neuronal RNA seq data from GEO. The files are in .counts.txt format. Could I convert them to bigwig? If so, how?&lt;/p&gt;&#xA;" OwnerUserId="9154" LastActivityDate="2020-07-08T02:21:10.283" Title="RNA seq .counts.txt to bigwig conversion" Tags="|rna-seq|rna|bigwig|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13802" PostTypeId="1" CreationDate="2020-07-07T21:26:18.350" Score="0" ViewCount="81" Body="&lt;p&gt;I am trying to use the &lt;a href=&quot;https://sra-explorer.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SRA explorer&lt;/a&gt; tool to download data from ENA. Whatever geo accession number I give it however, it will return no results. It only seems to return results for the suggestions it already has. I tried for example searching for &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE111797&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSE111797&lt;/a&gt; by typing it in the search tool but didn't get anything.&lt;/p&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="8601" LastEditDate="2020-07-08T15:00:28.597" LastActivityDate="2020-07-08T15:00:28.597" Title="Trying to retrieve sra data using sra explorer" Tags="|geo|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13803" PostTypeId="2" ParentId="13758" CreationDate="2020-07-07T22:06:04.610" Score="4" Body="&lt;p&gt;Please do not reinvent the wheel - &lt;code&gt;seqkit rmdup&lt;/code&gt; does what you want.&lt;/p&gt;&#xA;&lt;p&gt;From the website:&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h1&gt;rmdup&lt;/h1&gt;&#xA;&lt;h2&gt;Usage&lt;/h2&gt;&#xA;&lt;pre&gt;&lt;code&gt;remove duplicated sequences by id/name/sequence&#xA;&#xA;Usage:&#xA;  seqkit rmdup [flags]&#xA;&#xA;Flags:&#xA;  -n, --by-name                by full name instead of just id&#xA;  -s, --by-seq                 by seq&#xA;  -D, --dup-num-file string    file to save number and list of duplicated seqs&#xA;  -d, --dup-seqs-file string   file to save duplicated seqs&#xA;  -h, --help                   help for rmdup&#xA;  -i, --ignore-case            ignore case&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How does a &lt;code&gt;-D&lt;/code&gt; file look like, you ask:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ zcat hairpin.fa.gz \&#xA;    | seqkit rmdup -s -i -o clean.fa.gz -d duplicated.fa.gz -D duplicated.detail.txt&#xA;&#xA;$ cat duplicated.detail.txt   # here is not the entire list&#xA;3   hsa-mir-424, mml-mir-424, ppy-mir-424&#xA;3   hsa-mir-342, mml-mir-342, ppy-mir-342&#xA;2   ngi-mir-932, nlo-mir-932&#xA;2   ssc-mir-9784-1, ssc-mir-9784-2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastActivityDate="2020-07-07T22:06:04.610" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13804" PostTypeId="2" ParentId="13800" CreationDate="2020-07-08T02:21:10.283" Score="1" Body="&lt;p&gt;No, count-level data doesn't have the base-level resolution that is required in order to be usefully visualised on a genome browser. You need a BAM file (or raw FASTQ files that can be used to generate a BAM file) in order to do that.&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;No&amp;quot; isn't completely correct: it'd be &lt;em&gt;possible&lt;/em&gt; to take gene counts and information about gene annotation boundaries to shoehorn them into a BedGraph / BigWig file, but whatever comes out the other end of that isn't going to be a good representation of reality.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2020-07-08T02:21:10.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13806" PostTypeId="5" CreationDate="2020-07-08T07:55:57.227" Score="0" Body="&lt;p&gt;The technical definition is that for a given homologous position within an alignment, i.e. the inherited position from a parental copy, a base change occurs involving A, G, C or T. Mutations normally occur through errors in the copy proceedure involving the polymerase enzyme acting on the central unit of inheritance that which in the vast majority of cases is DNA, except in viruses where large numbers use RNA.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T07:55:57.227" LastActivityDate="2020-07-08T07:55:57.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13807" PostTypeId="4" CreationDate="2020-07-08T07:55:57.227" Score="0" Body="Nucleotide base changes in the genome from its parental copy which for eukaryotes or bacteria is DNA" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T07:55:57.227" LastActivityDate="2020-07-08T07:55:57.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13808" PostTypeId="5" CreationDate="2020-07-08T08:03:29.293" Score="0" Body="&lt;p&gt;A haplotype most accurately describes the genetic code of the mitochondrion or a bacterial genome. Mutational changes in the mitochondrion haplotype can only have occurred as a result of point mutation. In bacterial haplotypic change can involve recombination. Haplotypic data albeit often limited in its genetic variation is considered robust because there is no ambiguity in base calling, unless for bacteria there is a mixed infection.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T08:03:29.293" LastActivityDate="2020-07-08T08:03:29.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13809" PostTypeId="4" CreationDate="2020-07-08T08:03:29.293" Score="0" Body="Genetic code involving inherited DNA present as a single copy, which for eukaroyotes is the mitochondrion and in bacteria is the complete genome." OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T08:03:29.293" LastActivityDate="2020-07-08T08:03:29.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13810" PostTypeId="5" CreationDate="2020-07-08T08:07:04.420" Score="0" Body="&lt;p&gt;Data mining is assessing data for the essential structure and behaviour without imposing an a prior hypothesis. Cluster analysis, or PCA are good examples of data mining to identify groups of data that can be delineated and formally tested in subsequent analysis such as supervised learning.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T08:07:04.420" LastActivityDate="2020-07-08T08:07:04.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13811" PostTypeId="4" CreationDate="2020-07-08T08:07:04.420" Score="0" Body="Initial process of data analysis often used to define the hypothesis for subsequent analysis" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T08:07:04.420" LastActivityDate="2020-07-08T08:07:04.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13812" PostTypeId="5" CreationDate="2020-07-08T08:14:49.070" Score="0" Body="&lt;p&gt;Synonymous mutation, also known as 'silent mutation', is the most frequent mutation within a coding sequence, such as an exon, because it doesn't cause any major phenotypic change to the protein by altering its amino acid composition. Silent mutations occur because of degeneracy in the tiplet codon, i.e. there are only 20 amino acid but 64 possible permutations of the triplet codon, with only 3 permutations resulting in a stop codon. The antonym of a synonymous mutation is a non-synonymous mutation, i.e. a mutation in the nucleic acid which causes an amino acid mutation.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T08:14:49.070" LastActivityDate="2020-07-08T08:14:49.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13813" PostTypeId="4" CreationDate="2020-07-08T08:14:49.070" Score="0" Body="Mutation in a triplet codon that does not alter the amino acid also known as a 'silent mutation'" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T08:14:49.070" LastActivityDate="2020-07-08T08:14:49.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13814" PostTypeId="5" CreationDate="2020-07-08T08:20:38.863" Score="0" Body="&lt;p&gt;According to @gringers profile he is a freelance bioinformatician based in New Zealand. Very significant experience of next generation sequencing and associated informatics. Whilst @gringer is certainly knowledgeable it is not clear why a wiki tag has been created in their honour.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T08:20:38.863" LastActivityDate="2020-07-08T08:20:38.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13815" PostTypeId="4" CreationDate="2020-07-08T08:20:38.863" Score="0" Body="Senior member of the Bioinformatics site of Stackexchange, but unclear why they have been delineated a wiki tag" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T08:20:38.863" LastActivityDate="2020-07-08T08:20:38.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13816" PostTypeId="5" CreationDate="2020-07-08T08:25:56.447" Score="0" Body="&lt;p&gt;'Errors' in bioinformatics refers to wet-lab data robustness because generally 'errors' in code are referred to as 'bugs', although major faults in code could be termed 'errors'. 'Errors' can refer to misincorporation of the wrong nucleotide base when the DNA polymerase is copying a molecular of DNA and results in a mutation.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T08:25:56.447" LastActivityDate="2020-07-08T08:25:56.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13817" PostTypeId="4" CreationDate="2020-07-08T08:25:56.447" Score="0" Body="Within bioinformatics this generally refers to problems with wet-lab procedure, please see detailed explanation. In biological system, it refers to the process of mutation generation involving the DNA polymerase enzyme." OwnerUserId="3848" LastEditorUserId="9651" LastEditDate="2022-11-26T22:28:38.917" LastActivityDate="2022-11-26T22:28:38.917" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13818" PostTypeId="5" CreationDate="2020-07-08T08:30:47.467" Score="0" Body="&lt;p&gt;Oligonucleotide usually synthesised which initate replication of a much large locus through a DNA polymerase enzyme. If the DNA polymerase is Taq or a similar heat resistant enzyme the process of replication is known as PCR. A primer can be labeled with a tag such as a fluorecent label of e.g. biotin for subsequent reporting in for example qPCR or size fragment identification. The concept is central to next generation sequencing.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T08:30:47.467" LastActivityDate="2020-07-08T08:30:47.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13819" PostTypeId="4" CreationDate="2020-07-08T08:30:47.467" Score="0" Body="Short sequence of DNA which hybridises to a locus within a genome and is used to initate replication of a unit of DNA through the DNA polymerase enzyme. Essential for PCR" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T08:30:47.467" LastActivityDate="2020-07-08T08:30:47.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13820" PostTypeId="5" CreationDate="2020-07-08T08:39:12.513" Score="0" Body="&lt;p&gt;A dataframe could be considered to be a spreadsheet in R or Python pandas code, albeit it also comprises database functions. Dataframes comprise an index (rows) and data columns, known as 'columns' and can be readily merged rearranged and processed for a large variety of calculations. Most graphing and statistical packages within R or Python will accept or even require the data to be held within a dataframe.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T08:39:12.513" LastActivityDate="2020-07-08T08:39:12.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13821" PostTypeId="4" CreationDate="2020-07-08T08:39:12.513" Score="0" Body="Central technical term relating to the functional method of data used in R and Python's pandas calculations" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T08:39:12.513" LastActivityDate="2020-07-08T08:39:12.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13822" PostTypeId="5" CreationDate="2020-07-08T08:47:37.957" Score="0" Body="&lt;p&gt;Correlation is a formal statistical test describing the relationship between two variables and at simplist is defined as either Peasons correlation statistic (r2) or Spearman's rank sign correlation (r). When the value is zero, the relationship between the variables is random, when it is +1 the relationship is perfectly inter-related such that if one variable changes the other will change inaccordance with this. If the correlation is -1 the relationship is adversely affected. Correlations of 0.7 are considered reasonably robust for r2 and 0.8 for r&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T08:47:37.957" LastActivityDate="2020-07-08T08:47:37.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13823" PostTypeId="4" CreationDate="2020-07-08T08:47:37.957" Score="0" Body="Statistical term describing the association of two variables in accordance with a significance measure" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T08:47:37.957" LastActivityDate="2020-07-08T08:47:37.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13824" PostTypeId="1" CreationDate="2020-07-08T09:46:42.077" Score="0" ViewCount="145" Body="&lt;p&gt;If I go &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/BA000005.3&quot; rel=&quot;nofollow noreferrer&quot;&gt;to ncbi site&lt;/a&gt;, apply &lt;code&gt;customize view -&amp;gt; Customize -&amp;gt; All features -&amp;gt; Update view&lt;/code&gt; and then &lt;code&gt;ctrl + f : Alu&lt;/code&gt; I can get all alu repeaing regions coordinates from this chromosome.&lt;/p&gt;&#xA;&lt;p&gt;It looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; repeat_region   complement(37490..37800)&#xA;                 /note=&amp;quot;AluY&amp;quot;&#xA;                 /rpt_family=&amp;quot;SINE/Alu&amp;quot;&#xA;                 /rpt_type=DISPERSED&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How could I download all this coordinates and names of repeating regions from the internet to .txt file? Or some tool&lt;/p&gt;&#xA;" OwnerUserId="9159" LastActivityDate="2020-07-11T01:30:17.137" Title="How do I get repeating regions from fasta file or directly from internet" Tags="|fasta|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13826" PostTypeId="1" CreationDate="2020-07-08T15:13:08.317" Score="1" ViewCount="138" Body="&lt;p&gt;I have the PDB &lt;a href=&quot;https://www.rcsb.org/structure/6LU7&quot; rel=&quot;nofollow noreferrer&quot;&gt;6LU7&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I extracted the ligand using extract lig, org  and then the protein using extract prot, poly.  The ligand has three parts:  02J, PJE, and 010.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I want to select the 02J and 010 IN A SCRIPT and delete them.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I know how to do it by simply clicking, but want to incluse this in a script.&lt;/p&gt;&#xA;" OwnerUserId="9161" LastEditorUserId="3848" LastEditDate="2020-12-04T18:18:22.557" LastActivityDate="2020-12-04T18:18:22.557" Title="Selecting part of an extracted ligand" Tags="|covid-19|sars-cov-2|pymol|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13827" PostTypeId="2" ParentId="13826" CreationDate="2020-07-08T15:24:31.907" Score="1" Body="&lt;p&gt;I know that structure way too well —SARS-COV-2 MPro.&lt;/p&gt;&#xA;&lt;p&gt;In PyMOL command line&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; remove resn 02J or 010&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; cmd.remove('resn 02J or 010')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In python&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; import pymol2&#xA; with pymol2.PyMOL() as pymol:&#xA;      pymol.cmd.fetch('6LU7')&#xA;      pymol.cmd.remove('resn 02J or 010')&#xA;      pymol.cmd.save('6LU7.mod.pdb') &#xA;      #or pdbblock = pymol.cmd.get_pdbstr()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From the sounds of it you'd like to make the peptidomimetic a peptide. If you wanted to do the horrid thing of making PJE, the capped side chain glutamine ketamide, into glutamine, the command to rename atoms is a multitude of &lt;code&gt;cmd.alter('resn PJE and name &amp;lt;atomname&amp;gt;', 'name=&amp;quot;&amp;lt;newname&amp;gt;&amp;quot;)&lt;/code&gt; followed by a &lt;code&gt;cmd.alter('resn PJE', 'resn=&amp;quot;GLN&amp;quot;')&lt;/code&gt; and lastly &lt;code&gt;cmd.sort()&lt;/code&gt; to commit the changes. Not pretty, but is all in PyMOL.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-07-08T15:35:45.033" LastActivityDate="2020-07-08T15:35:45.033" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13828" PostTypeId="5" CreationDate="2020-07-08T17:37:22.920" Score="0" Body="&lt;p&gt;Formal process of defining data within a mathematical definition enabling significant insight and possibly predictive power into the data set. However, the model comprises a priori assumptions which the data must not violate. For example in phylogenetics the model depends on each mutation being independent of the occurance of any other mutation. Parametric statistics are a model assuming a normal distribution of the data. If the assumption is strongly violated the model is invalidated regardless of result.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T17:37:22.920" LastActivityDate="2020-07-08T17:37:22.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13829" PostTypeId="4" CreationDate="2020-07-08T17:37:22.920" Score="0" Body="Assignment of data into a defined mathematical description invariably associated with a priori assumptions" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T17:37:22.920" LastActivityDate="2020-07-08T17:37:22.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13830" PostTypeId="5" CreationDate="2020-07-08T17:42:10.303" Score="0" Body="&lt;p&gt;Motifs most notably relate to amino acids and regard a short region of physcially linked sequences that form a distinct pattern that is easily recognisable. In flaviviruses the fusion loop comprises a short universally recognisable motif associated with entry into the host cell.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T17:42:10.303" LastActivityDate="2020-07-08T17:42:10.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13831" PostTypeId="4" CreationDate="2020-07-08T17:42:10.303" Score="0" Body="Physically linked short region of DNA or most notably amino acids which characterise a 'pattern' identifyable by optical analysis." OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T17:42:10.303" LastActivityDate="2020-07-08T17:42:10.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13832" PostTypeId="5" CreationDate="2020-07-08T17:44:29.160" Score="0" Body="&lt;p&gt;Powerful IDE used in developing R code and universally regarded as the singular IDE for R. Rstudio is freeware and available for platforms.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T17:44:29.160" LastActivityDate="2020-07-08T17:44:29.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13833" PostTypeId="4" CreationDate="2020-07-08T17:44:29.160" Score="0" Body="Integrated Development Environment (IDE) for the R package" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T17:44:29.160" LastActivityDate="2020-07-08T17:44:29.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13834" PostTypeId="5" CreationDate="2020-07-08T17:52:53.047" Score="0" Body="&lt;p&gt;To understand an orthologue it is first better to consider a homologue. A homologue means each residue of DNA or amino acid has a shares direct ancestory with the same residue in a separate genome. Thus if the common ancestor mutated, each ancestor would carry the same mutation at exactly the same position. In other words we are confident this represents inheritance. An orthologue is a likely ancestor, but may not share direct ancestry for example if a gene has duplicated one genome may have inherited one copy a different genome inherited a different copy and therefore homology can not be assumed. In this case the sequences are considered orthologues.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T17:52:53.047" LastActivityDate="2020-07-08T17:52:53.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13835" PostTypeId="4" CreationDate="2020-07-08T17:52:53.047" Score="0" Body="Probable homologue for a given parental sequence, but may not have arisen by direct ancestry for example could have arisen indirectly for example through gene duplication" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T17:52:53.047" LastActivityDate="2020-07-08T17:52:53.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13836" PostTypeId="5" CreationDate="2020-07-08T17:58:26.630" Score="0" Body="&lt;p&gt;Subsets are a limited selection of the universal data set that is  specifically designated in advance. For example, using resampling to assess the error range a randomised subset might be selected, this being smaller than the universal data set but assesses whether the same pattern in the complete data set holds for a partial data set. Alternatively it is a way of teasing apart a data set to understand a given component, for example mosquito-borne flaviviruses are a subset of arboviruses.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T17:58:26.630" LastActivityDate="2020-07-08T17:58:26.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13837" PostTypeId="4" CreationDate="2020-07-08T17:58:26.630" Score="0" Body="Subset is a defined section of the universal data set" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T17:58:26.630" LastActivityDate="2020-07-08T17:58:26.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13838" PostTypeId="5" CreationDate="2020-07-08T18:04:52.077" Score="0" Body="&lt;p&gt;A p-value refers to the probability that an observation has not arisen by chance. An observtion &amp;lt;5% infers that 1 in 20 repetitions will result in the observation being true by chance. The 5% threshold was arbitarily designated by Ron Fisher who wanted to test a woman's claim that she could tell whether a cup of tea had milk poured in the cup first or the milk was added last. Fisher wanted to know how many cups of tea did she needed to accurately define before he could consider the observation was beyond chance. I suspect he choose 20 cups of tea and concluded the subject's claim was correct. The theory is known as critical probability and central to large areas of statistical theory.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T18:04:52.077" LastActivityDate="2020-07-08T18:04:52.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13839" PostTypeId="4" CreationDate="2020-07-08T18:04:52.077" Score="0" Body="Relates to the probaility of an observation being correct, usuaully considered significant at a threshold of 5%" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T18:04:52.077" LastActivityDate="2020-07-08T18:04:52.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13840" PostTypeId="5" CreationDate="2020-07-08T18:08:03.240" Score="0" Body="&lt;p&gt;Noise is a statistical term generally assigned by optical analysis of a scatter plot. If one is attempting to make a statistically valid assignment within noisy data then it is particularly difficult.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T18:08:03.240" LastActivityDate="2020-07-08T18:08:03.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13841" PostTypeId="4" CreationDate="2020-07-08T18:08:03.240" Score="0" Body="Noise in statistics refers to statistical variation, where noisy data approximate to a large standard deviation" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T18:08:03.240" LastActivityDate="2020-07-08T18:08:03.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13842" PostTypeId="5" CreationDate="2020-07-08T18:11:34.900" Score="0" Body="&lt;p&gt;Beast is a very popular phylogenetics package which reconstructs phylogenetic trees using DNA/RNA data. It uses a directional mutation matrix and is able to determine the rate of mutation between taxa as a consequence. This enables ease of molecular clock calculations but ensuring the Bayesian simulation has reached convergence is often difficult and overlooked within this calculation.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T18:11:34.900" LastActivityDate="2020-07-08T18:11:34.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13843" PostTypeId="4" CreationDate="2020-07-08T18:11:34.900" Score="0" Body="Phylogenetics package based on Bayesian statistics often used in molecular clock calculation" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-08T18:11:34.900" LastActivityDate="2020-07-08T18:11:34.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13844" PostTypeId="1" CreationDate="2020-07-08T20:16:11.110" Score="2" ViewCount="102" Body="&lt;p&gt;Gene set or pathway analysis in scRNA-seq data has its own challenges. Since the data is sparse and many genes will be missing for any given cell, some people simply add up all the counts across the gene set. The sum is treated as the expression value of the gene set. I still see this simple approach used in recent papers. Seurat AddModuleScore (and used by CellCycleScoring) is basically the same method, except it additionally subtracts the signal from a control gene set, but that is just shifting the score down.&lt;/p&gt;&#xA;&lt;p&gt;My concern is that the gene set score can be driven almost entirely by a single gene. If the max value of one gene in a gene set is 100x greater than for others, which certainly happens, then those other ones are not really contributing. What is even the point of using a whole gene set then? Yet, this approach is widely used, so it must be working well enough. What am I missing?&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2020-07-08T20:16:11.110" Title="gene set enrichment in single-cell data" Tags="|scrnaseq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13845" PostTypeId="1" AcceptedAnswerId="13847" CreationDate="2020-07-08T22:46:52.210" Score="2" ViewCount="188" Body="&lt;p&gt;I would like to programmatically retrieve the file &amp;quot;Accession List&amp;quot; from an &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Traces/study/?acc=PRJNA491191&amp;amp;o=acc_s%3Aa&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI page&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I'm running firefox, so I'm clicking the file and clicking &lt;code&gt;Ctrl&lt;/code&gt; + &lt;code&gt;Shift&lt;/code&gt; + &lt;code&gt;E&lt;/code&gt; to try and figure out what the file endpoint is. But I can't seem to figure it out (I haven't done this much, I just remember doing it once for a previous project).&lt;/p&gt;&#xA;&lt;p&gt;When I do this, I see two links popping up on my screen, as shown below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UY0oy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UY0oy.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I see that the first one has a field (I'm talking about the panel on the right) called &amp;quot;acc&amp;quot;, which varies from one data dump to the next, so I'm presuming this is the link I need to use to retrieve the file (though I'm not sure), and just change the &amp;quot;acc&amp;quot; field depending on the data I want. Assuming this is right, I'm not sure how to go from here.&#xA;I've included the text on the right below for clarity.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GET&#xA;    https://www.google-analytics.com/collect?v=1&amp;amp;_v=j83&amp;amp;a=1402353855&amp;amp;t=event&amp;amp;_s=4&amp;amp;dl=https://www.ncbi.nlm.nih.gov/Traces/study/?acc=PRJNA491191&amp;amp;o=acc_s%3Aa&amp;amp;ul=en-us&amp;amp;de=UTF-8&amp;amp;dt=Run Selector :: NCBI&amp;amp;sd=24-bit&amp;amp;sr=1280x720&amp;amp;vp=1232x601&amp;amp;je=0&amp;amp;ec=Link Click&amp;amp;ea=Metadata  &amp;amp;el=NONE&amp;amp;_u=SACAAEAB~&amp;amp;jid=&amp;amp;gjid=&amp;amp;cid=1350895533.1590937485&amp;amp;tid=UA-68648221-3&amp;amp;_gid=391407234.1594002343&amp;amp;cd1=run_selector&amp;amp;cd2=sra&amp;amp;cd3=&amp;amp;cd5=NO&amp;amp;cd6=run_selector:sra:&amp;amp;cd8=&amp;amp;cd9=&amp;amp;cd10=&amp;amp;cd11=&amp;amp;cd12=&amp;amp;cd13=&amp;amp;cd14=&amp;amp;cd15=&amp;amp;cd16=&amp;amp;cg1=run_selector&amp;amp;cg2=sra&amp;amp;cg3=&amp;amp;cg4=run_selector:sra:&amp;amp;z=508671370&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2020-07-09T00:43:39.897" Title="Programmatically retrieve Accession List from NCBI" Tags="|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13847" PostTypeId="2" ParentId="13845" CreationDate="2020-07-09T00:43:39.897" Score="2" Body="&lt;p&gt;Which accessions are you trying to get to? In the page you refer to, there is an option to download the list of accessions - one SRR run accession per line - by clicking the 'Accessions' button or the entire table by clicking the 'Metadata' button as highlighted in the screenshot below:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/TI7xh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TI7xh.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can get the same data from the command line using the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez Direct&lt;/a&gt;. The command for that would be something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db sra -query 'PRJNA491191[bioproject]' | efetch -format runinfo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to download the table in CSV format.&lt;/p&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2020-07-09T00:43:39.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13850" PostTypeId="1" CreationDate="2020-07-09T08:01:14.267" Score="0" ViewCount="79" Body="&lt;p&gt;I am attempting to use GSEA enrichr with 10x genomics differential_expression rankings. Reading what people seem to be doing with GSEA, there seems to be a pre-/post-singlecell gene expression versioning of the tools associated, so I want to make sure what I am doing makes sense.&lt;/p&gt;&#xA;&lt;p&gt;Explanation:&lt;/p&gt;&#xA;&lt;p&gt;For each gene in columns 1 and 2, there are 3 columns in each of the clusters that the 10X Genomics &lt;code&gt;cellranger count&lt;/code&gt; software produces, e.g.:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Cluster 3 Mean Counts   Cluster 3 Log2 fold change   Cluster 3 Adjusted p value&#xA;0.2392551160479023      -6.591931364165717           0.029177998608334674&#xA;0.1435530696287414      -5.608385347948343           0.0003833420249831818&#xA;1.107409394278862       -5.441829814465051           3.8059567638748296e-09&#xA;0.1948220230675776      -4.9155704967404             0.008739337761029243&#xA;0.20849374398460058     -4.65702722396748            1.0980978990257375e-11&#xA;0.36571853453036496     -4.6498340596466585          1.1896434946397147e-17&#xA;0.1811503021505546      -4.445327449243366           1.3416233207430882e-12&#xA;0.4409129995739914      -4.433552335619653           1.2400396349891802e-16&#xA;1.7226368355448967      -4.240367394494225           2.2072280062348652e-15&#xA;0.10937376733618391     -3.902215199856403           2.0315030904651526e-12&#xA;3.2948847410025404      -3.7580328996321235          0.028858857004902847&#xA;0.3588826740718535      -3.738237134795285           8.439629283346646e-13&#xA;[...]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In trying to give a tissue / cell type identity to each of these clusters, I've used the following method to feed a subset of genes to EnrichR:&lt;/p&gt;&#xA;&lt;p&gt;1- Sort the &lt;code&gt;Log Fold change&lt;/code&gt; column numerically so we get low negative values.&#xA;2- Filter out for the &lt;code&gt;Adjusted p-value&lt;/code&gt; to have a value smaller than 0.05.&#xA;3- Filter out for the &lt;code&gt;Means Counts&lt;/code&gt; value to be bigger than 0.01.&#xA;4- Take the top n genes from the list, and feed it to GSEA EnrichR to compare against Mouse_Gene_Atlas or ARCHS4_Tissues.&lt;/p&gt;&#xA;&lt;p&gt;Given that the fold change of the clusters is relative to each other, does it make more sense to rank the top n genes numerically or in inverse numerical value? E.g. selecting the first 100 genes in each cluster that have the most negative &lt;code&gt;Log2 fold change&lt;/code&gt; values or the most positive &lt;code&gt;Log2 fold change&lt;/code&gt; values or both (the top 100 genes with highest absolute &lt;code&gt;Log2 fold change&lt;/code&gt;)?&lt;/p&gt;&#xA;&lt;p&gt;Looking at previous answers, I get that one shouldn't rank by p-value: e.g. see @gringer reply&#xA;&lt;a href=&quot;https://bioinformatics.stackexchange.com/a/13465/180&quot;&gt;https://bioinformatics.stackexchange.com/a/13465/180&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2020-07-09T14:49:46.113" Title="GSEA enrichr with 10x genomics differential_expression ranks" Tags="|single-cell|differential-expression|gsea|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13851" PostTypeId="1" CreationDate="2020-07-09T08:21:56.547" Score="1" ViewCount="51" Body="&lt;p&gt;Assume that I have three different Affymetrix microarray experiments, 2x U133 (from 2015 and 2017) and 1xU133 Plus 2.0 (2019). Each array contains several different samples, some of them are common for 2 or 3 arrays. I'd like to combine these thee results to perform further MDS analysis using whole sets.&#xA;I'm aware about some batch effect that I'd like to avoid.&#xA;When I do MDS using separately normalized data from three experiments, I can see that expression pattern of sample present in two of 3 matrices is the same but the normalized signal values are raised by a certain value. It visibly affects further MDS analysis.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My questions are:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there some bioconductor package or strategy to do such analysis avoiding bias and batch effect?&lt;/li&gt;&#xA;&lt;li&gt;Can I just normalize all 3 experiments together using RMA? And then MDS analysis will be unbiased.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="2869" LastActivityDate="2020-07-14T12:16:50.933" Title="Combining several different affymetrix expression microarray experiments for further MDS analysis" Tags="|microarray|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13852" PostTypeId="2" ParentId="13850" CreationDate="2020-07-09T14:49:46.113" Score="1" Body="&lt;p&gt;You should be using the most positive fold changes, not the most negative. The positive ones are the ones that are most highly expressed in your cluster of interest. You probably want to use less than 100 genes. Depending on the experiment, you may not have that many good markers for a particular cluster.&lt;/p&gt;&#xA;&lt;p&gt;The other question you mentioned refers to DESeq2 specifically, so it is not applicable here.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2020-07-09T14:49:46.113" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13853" PostTypeId="1" AcceptedAnswerId="13867" CreationDate="2020-07-09T21:25:24.367" Score="1" ViewCount="396" Body="&lt;p&gt;I keep getting a &amp;quot;too many requests error&amp;quot; when querying the NCBI SRA database, even though I'm running less than 10 requests per second, and I have an API key, which supposedly should allow me to run 10 per second.&lt;/p&gt;&#xA;&lt;p&gt;Here is my (Python) code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import subprocess&#xA;import concurrent.futures&#xA;import time&#xA;&#xA;bioprojects = [&#xA;    &amp;quot;PRJNA644722&amp;quot;,&#xA;    &amp;quot;PRJNA644892&amp;quot;&#xA;]&#xA;&#xA;all_metadata_generators = []&#xA;&#xA;def fetch_metadata(bioproject):&#xA;    return subprocess.run(f&amp;quot;esearch -db sra -query '{bioproject}[bioproject]' | efetch -format runinfo&amp;quot;, shell=True, capture_output=True, text=True)&#xA;&#xA;with concurrent.futures.ThreadPoolExecutor() as executor:&#xA;    for i in range(0, len(bioprojects), 10):&#xA;        all_metadata_generators.append(executor.map(fetch_metadata, bioprojects[i:i+10]))&#xA;        time.sleep(1)&#xA;&#xA;&#xA;for metadata_generator in all_metadata_generators:&#xA;    for metadata in metadata_generator:&#xA;        print(metadata)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="8601" LastEditDate="2020-07-09T22:09:30.090" LastActivityDate="2020-07-11T23:43:22.900" Title="Too many requests error when querying NCBI" Tags="|python|phylogenetics|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13854" PostTypeId="2" ParentId="13853" CreationDate="2020-07-10T00:36:27.757" Score="2" Body="&lt;p&gt;The execution of your EDirect commands may be limited unless you are using an API Key, which you can create from your MyNCBI account. Your API Key can be used in one of several different ways.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;In an argument to an EDirect command:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db nuccore -query 'some query' -api_key 12345&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;In a UNIX environment:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can either run this just before you are running some edirect commands or add the following line to your &lt;code&gt;.bashrc&lt;/code&gt; file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;export NCBI_API_KEY=12345&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;In the environment, from a Perl script:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ENV{NCBI_API_KEY} = &amp;quot;12345&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2020-07-10T00:36:27.757" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13855" PostTypeId="1" CreationDate="2020-07-10T00:52:27.253" Score="0" ViewCount="229" Body="&lt;p&gt;I'd like to use protein structure data from PDBs visualized by Chimera to do two things.&#xA;I'd be happy to learn the appropriate tools and methods.&lt;/p&gt;&#xA;&lt;p&gt;・Mapping information about amino acids present on protein surfaces to sequences&lt;/p&gt;&#xA;&lt;p&gt;・Find the charges of exposed amino acids on the surface and apply color gradations to the amino acid molecules according to their values&lt;/p&gt;&#xA;&lt;p&gt;The second one in particular looks like it could be done with Chimera's &amp;quot;Coulomb surface coloring&amp;quot;. But I'm not sure what kind of value to enter underneath the color and how to find it, which is needed to add a gradient.&lt;/p&gt;&#xA;&lt;p&gt;Sorry for my bad English ...&lt;/p&gt;&#xA;" OwnerUserId="9174" LastActivityDate="2020-07-10T00:52:27.253" Title="Extract only the amino acid residues on the surface of protein structure data" Tags="|protein-structure|pdb|3d-structure|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13856" PostTypeId="1" CreationDate="2020-07-10T01:22:24.727" Score="1" ViewCount="49" Body="&lt;p&gt;I have data for 3 histone marks (2 for silencing and 1 for activation) each mark has three replicates.&lt;/p&gt;&#xA;&lt;p&gt;when I run the &lt;code&gt;diffBind&lt;/code&gt; package I have three contrast:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Silence1-Activation&#xA;Silence1-Silence2&#xA;Silence2-Activation&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I run dba.plotBox for any of contrasts I will have the same distribution of reads.like below:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/wchgv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wchgv.png&quot; alt=&quot;plotBox&quot; /&gt;&lt;/a&gt;&#xA;I even tried to start with 1000 peaks of each one that showed higher fold change but again got the same pattern for plotbox. How can I see which peaks belong to which histone marks?&lt;/p&gt;&#xA;" OwnerUserId="8543" LastEditorUserId="5077" LastEditDate="2020-07-10T12:41:00.973" LastActivityDate="2020-07-13T20:22:15.307" Title="DiffBind, diffferentially binding site" Tags="|sequence-alignment|bioconductor|sequence-analysis|phylogenetics|chip-seq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13857" PostTypeId="1" CreationDate="2020-07-10T01:51:16.133" Score="0" ViewCount="104" Body="&lt;p&gt;I would like to display a score or graph to visualize the degree of conservation of each residue from the amino acid alignment data.&#xA;If possible, I'd like to extract the parts of the game that scored above a certain level.&lt;/p&gt;&#xA;&lt;p&gt;Is there a tool or source code (python) to do that?&lt;/p&gt;&#xA;&lt;p&gt;Sorry for my bad English ...&lt;/p&gt;&#xA;" OwnerUserId="9174" LastActivityDate="2020-08-12T00:00:55.120" Title="Display a score or graph to visualize the degree of conservation of each residue from alignment data" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13859" PostTypeId="1" AcceptedAnswerId="13860" CreationDate="2020-07-10T11:54:37.280" Score="2" ViewCount="62" Body="&lt;p&gt;Having constructed two nested negbin glms in R, one has a dropped factor, We can do an &lt;code&gt;anova&lt;/code&gt; with &lt;code&gt;test = &amp;quot;Chisq&amp;quot;&lt;/code&gt; to see if there is a significant difference between the models. But what does the Pvalue as a result of this actually tell me?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;anova(mnegbin1, mnegbin2, test = &amp;quot;Chisq&amp;quot;)&#xA;Likelihood ratio tests of Negative Binomial Models&#xA;&#xA;Response: count&#xA;                        Model    theta Resid. df    2 x log-lik.   Test    df      LR stat. Pr(Chi)&#xA;1 origin + substrate + sample 18.46502      1232       -2459.886                                   &#xA;2          substrate + sample 18.46502      1232       -2459.886 1 vs 2     0 -1.500666e-11       1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7031" LastActivityDate="2020-07-10T13:16:11.217" Title="What is the pvalue from anova(glm1, glm2, test = “Chisq”) telling me in R when the glms have a negative binomial distribution?" Tags="|statistics|logistic-regression|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="13860" PostTypeId="2" ParentId="13859" CreationDate="2020-07-10T13:16:11.217" Score="2" Body="&lt;p&gt;When you fit a model, you need to check whether it's overly determined, meaning some terms cannot be determined. We can look at one of your samples, for example only for RB5 and Nocardia_nova_SH22a_RB20_RB56:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df = structure(list(X = c(226L, 227L, 228L, 229L, 230L, 231L, 232L, &#xA;233L, 234L, 235L, 236L, 237L, 238L, 239L, 240L, 241L, 242L, 243L, &#xA;244L, 245L, 246L, 247L, 248L, 249L, 250L, 251L, 252L, 253L, 254L, &#xA;255L, 256L, 257L, 258L, 259L, 260L, 261L, 262L, 263L, 264L, 265L, &#xA;266L, 267L, 268L, 269L, 270L, 1126L, 1127L, 1128L, 1129L, 1130L, &#xA;1131L, 1132L, 1133L, 1134L, 1135L, 1136L, 1137L, 1138L, 1139L, &#xA;1140L, 1141L, 1142L, 1143L, 1144L, 1145L, 1146L, 1147L, 1148L, &#xA;1149L, 1150L, 1151L, 1152L, 1153L, 1154L, 1155L, 1156L, 1157L, &#xA;1158L, 1159L, 1160L, 1161L, 1162L, 1163L, 1164L, 1165L, 1166L, &#xA;1167L, 1168L, 1169L, 1170L), sample = structure(c(1L, 1L, 1L, &#xA;1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, &#xA;1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, &#xA;1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, &#xA;2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, &#xA;2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, &#xA;2L, 2L, 2L, 2L, 2L, 2L, 2L), .Label = c(&amp;quot;Nocardia_nova_SH22a_RB20_RB56&amp;quot;, &#xA;&amp;quot;RB56&amp;quot;), class = &amp;quot;factor&amp;quot;), substrate = structure(c(10L, 37L, &#xA;29L, 11L, 24L, 9L, 39L, 30L, 42L, 14L, 21L, 40L, 38L, 45L, 28L, &#xA;5L, 22L, 3L, 41L, 16L, 32L, 27L, 18L, 26L, 20L, 34L, 7L, 12L, &#xA;23L, 17L, 44L, 35L, 4L, 19L, 25L, 13L, 1L, 33L, 31L, 43L, 36L, &#xA;6L, 2L, 8L, 15L, 10L, 37L, 29L, 11L, 24L, 9L, 39L, 30L, 42L, &#xA;14L, 21L, 40L, 38L, 45L, 28L, 5L, 22L, 3L, 41L, 16L, 32L, 27L, &#xA;18L, 26L, 20L, 34L, 7L, 12L, 23L, 17L, 44L, 35L, 4L, 19L, 25L, &#xA;13L, 1L, 33L, 31L, 43L, 36L, 6L, 2L, 8L, 15L), .Label = c(&amp;quot;acylsphingosine&amp;quot;, &#xA;&amp;quot;agarose&amp;quot;, &amp;quot;alcohol&amp;quot;, &amp;quot;alginate&amp;quot;, &amp;quot;alpha-glucan&amp;quot;, &amp;quot;amylaceous polysaccharides&amp;quot;, &#xA;&amp;quot;arabinogalactans&amp;quot;, &amp;quot;aribinose&amp;quot;, &amp;quot;beta-glucan&amp;quot;, &amp;quot;cellulose&amp;quot;, &#xA;&amp;quot;chitin&amp;quot;, &amp;quot;cutin&amp;quot;, &amp;quot;cyclomaltodextrin&amp;quot;, &amp;quot;D-galactose&amp;quot;, &amp;quot;D-glucan&amp;quot;, &#xA;&amp;quot;D-glucose&amp;quot;, &amp;quot;dermatan sulfate&amp;quot;, &amp;quot;dextran&amp;quot;, &amp;quot;fructans&amp;quot;, &amp;quot;fructose&amp;quot;, &#xA;&amp;quot;galactan&amp;quot;, &amp;quot;galactose&amp;quot;, &amp;quot;galacturonide&amp;quot;, &amp;quot;glucan&amp;quot;, &amp;quot;glucose&amp;quot;, &#xA;&amp;quot;glucuronoside&amp;quot;, &amp;quot;glycan&amp;quot;, &amp;quot;glycans&amp;quot;, &amp;quot;glycogen&amp;quot;, &amp;quot;hemicellulose&amp;quot;, &#xA;&amp;quot;hyaluronate&amp;quot;, &amp;quot;L-rhamnose&amp;quot;, &amp;quot;lacto-N-tetraose&amp;quot;, &amp;quot;levan&amp;quot;, &amp;quot;lignin&amp;quot;, &#xA;&amp;quot;maltose&amp;quot;, &amp;quot;mannan&amp;quot;, &amp;quot;mannose&amp;quot;, &amp;quot;pectin&amp;quot;, &amp;quot;peptidoglycan&amp;quot;, &amp;quot;pullulan&amp;quot;, &#xA;&amp;quot;trehalose&amp;quot;, &amp;quot;Vanillyl-alcohol&amp;quot;, &amp;quot;xanthan&amp;quot;, &amp;quot;xylose&amp;quot;), class = &amp;quot;factor&amp;quot;), &#xA;    count = c(2L, 0L, 2L, 2L, 0L, 1L, 2L, 2L, 3L, 0L, 0L, 1L, &#xA;    0L, 0L, 0L, 2L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;    0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;    0L, 0L, 0L, 3L, 1L, 3L, 3L, 0L, 3L, 3L, 5L, 2L, 0L, 0L, 4L, &#xA;    0L, 0L, 1L, 2L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;    1L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, &#xA;    0L, 0L, 0L), origin = structure(c(1L, 1L, 1L, 1L, 1L, 1L, &#xA;    1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, &#xA;    1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, &#xA;    1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, &#xA;    2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, &#xA;    2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, &#xA;    2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L), .Label = c(&amp;quot;free&amp;quot;, &amp;quot;isolate&amp;quot;&#xA;    ), class = &amp;quot;factor&amp;quot;)), row.names = c(226L, 227L, 228L, 229L, &#xA;230L, 231L, 232L, 233L, 234L, 235L, 236L, 237L, 238L, 239L, 240L, &#xA;241L, 242L, 243L, 244L, 245L, 246L, 247L, 248L, 249L, 250L, 251L, &#xA;252L, 253L, 254L, 255L, 256L, 257L, 258L, 259L, 260L, 261L, 262L, &#xA;263L, 264L, 265L, 266L, 267L, 268L, 269L, 270L, 1126L, 1127L, &#xA;1128L, 1129L, 1130L, 1131L, 1132L, 1133L, 1134L, 1135L, 1136L, &#xA;1137L, 1138L, 1139L, 1140L, 1141L, 1142L, 1143L, 1144L, 1145L, &#xA;1146L, 1147L, 1148L, 1149L, 1150L, 1151L, 1152L, 1153L, 1154L, &#xA;1155L, 1156L, 1157L, 1158L, 1159L, 1160L, 1161L, 1162L, 1163L, &#xA;1164L, 1165L, 1166L, 1167L, 1168L, 1169L, 1170L), class = &amp;quot;data.frame&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now we look at whether how it is distributed:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df&lt;span class=&quot;math-container&quot;&gt;$sample = droplevels(df$&lt;/span&gt;sample)&#xA;table(df&lt;span class=&quot;math-container&quot;&gt;$origin,df$&lt;/span&gt;sample)&#xA;         &#xA;          Nocardia_nova_SH22a_RB20_RB56 RB56&#xA;  free                               45    0&#xA;  isolate                             0   45&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can see, the samples are either free or isolate, and never both. So you cannot discern the effect of origin from samples. This means your model is over-determined and hence you see the error above.&lt;/p&gt;&#xA;&lt;p&gt;To know the effect of origin, you have to fit, i used a poisson for this example because if i use only the subset above, the counts are not overdispersed hence glm.nb throws some error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fit = glm(count~ origin+substrate,data=df,family=poisson)&#xA;summary(fit)&#xA;&#xA; head(coefficients(summary(fit)))&#xA;                           Estimate   Std. Error       z value   Pr(&amp;gt;|z|)&#xA;(Intercept)           -2.266968e+01 2.930556e+04 -7.735624e-04 0.99938279&#xA;originisolate          6.931472e-01 2.970443e-01  2.333481e+00 0.01962291&#xA;substrateagarose       2.282092e-06 4.144430e+04  5.506408e-11 1.00000000&#xA;substratealcohol       2.280854e-06 4.144430e+04  5.503420e-11 1.00000000&#xA;substratealginate      2.278865e-06 4.144430e+04  5.498621e-11 1.00000000&#xA;substratealpha-glucan  2.295736e+01 2.930556e+04  7.833790e-04 0.99937495&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can see that in this model, free is the reference, and isolate is compared to free in the &amp;quot;originisolate&amp;quot;. The coefficient 6.931472e-01 means isolate samples are higher than free samples, by exp(6.931472e-01) times.&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-07-10T13:16:11.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13861" PostTypeId="1" CreationDate="2020-07-10T18:05:10.260" Score="1" ViewCount="107" Body="&lt;p&gt;I am assembling a transcriptome obtained from the Internet. The transcriptome was extracted from a human cancer tissue that had been previously grafted into a mouse. I have detected that many assembled RNA sequences do not match precisely to their reference but have additional sequences at one of their tails containing bacterial DNA. It's not that there are reads pertaining to that bacterium (always the same one) but that the RNA sequences of the cell have incorporated fragments of bacterial sequences (the reads corroborate these insertions with high coverage), therefore I can't just filter out bacterial reads as there are not reads exclusively pertaining to bacteria but hybrids.&#xA;My question is how can it be possible? What possible mechanisms could have brought these bacterial fragments to the RNA of the cell. I have read in some papers that there is speculation that bacteria could be involved in the genesis of cancer cells. That would be an explanation if it wasn't because there are also mouse RNA for which the same circumstance applies: they also have bacterial sequences at the tails. I have also read in papers that there is speculation with the possibility that cancer cells are more susceptible to get DNA from bacteria transferred to their DNA. I am speculating that is the reason in this case (and somehow that weakness was transferred to contiguous mouse cells) but I would like to know if there is another possibility I am missing.&#xA;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="5190" LastEditorUserId="3848" LastEditDate="2020-12-03T12:47:32.463" LastActivityDate="2020-12-03T12:47:32.463" Title="Bacterial DNA at the tail of transcriptome reads. What does that mean?" Tags="|ngs|assembly|rna|cancer|bacteria|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13863" PostTypeId="2" ParentId="13824" CreationDate="2020-07-11T01:30:17.137" Score="1" Body="&lt;p&gt;Click on the 'GenBank' dropdown menu at the top left corner of &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/BA000005.3&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt; to see a &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/BA000005.3?report=graph&quot; rel=&quot;nofollow noreferrer&quot;&gt;graphical view&lt;/a&gt; of the sequence. You should see a 'Repeat region' track in the Sequence Viewer. Click on the 'Download' button in the top right corner of the Sequence Viewer, choose 'Download Track Data' to launch a dialog box where you can enter a range, select the 'Repeat region' track and download data in BED format. Unfortunately, the names for the features are not great.&lt;/p&gt;&#xA;&lt;p&gt;If you want the names to be the same ones in the flat file, you can download the feature table and parse it. To download the feature table, click on the 'Send To' link at the top right corner of &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/BA000005.3&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt;, choose 'Complete Record', 'File' and 'Feature Table' as 'Format'. After you download the feature table, you can parse it using &lt;code&gt;awk&lt;/code&gt; as shown below to create a 4-column table with the data.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat seq.ft \&#xA;  | awk 'BEGIN{FS=&amp;quot;\t&amp;quot;;OFS=&amp;quot;\t&amp;quot;}{&#xA;    if(NF==3) {print s,e,f,q ; s=$1; e=$2; f=$3; q=&amp;quot;&amp;quot;} &#xA;    else if(NF==2) {print s,e,f,q ; s=$1; e=$2; q=&amp;quot;&amp;quot;} &#xA;    else if(NF==5) {q=q&amp;quot;|&amp;quot;$4&amp;quot;=&amp;quot;$5} }' \&#xA;  &amp;gt; feat_table.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2020-07-11T01:30:17.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13865" PostTypeId="1" AcceptedAnswerId="13885" CreationDate="2020-07-11T20:55:57.410" Score="0" ViewCount="45" Body="&lt;p&gt;I want to categorize promoters based upon high, intermediate and low-CpG content (high-CpG-density promoters (HCPs), intermediate-CpG-density promoters(ICPs), and low-CpG-density promoters (LCPs)).&lt;/p&gt;&#xA;&lt;p&gt;So the data I have is for promoter which i have annotated 1000+/- around TSS and taken them as something less than 1kb is promoter region and beyond distal.&lt;/p&gt;&#xA;&lt;p&gt;In term of tool i have used chipseeker to annotate and do the above step.&lt;/p&gt;&#xA;&lt;p&gt;So now if i have to find the CpG density as i have mentioned above how do i get that?&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2020-07-14T10:11:33.260" Title="Chromatin accessibility level on promoters with different CpG ratio calculation" Tags="|atac-seq|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13866" PostTypeId="2" ParentId="13861" CreationDate="2020-07-11T21:59:50.050" Score="1" Body="&lt;p&gt;From time to time these situations are encountered, notably when PCR is being deployed in rare DNA samples.&lt;/p&gt;&#xA;&lt;p&gt;In this instance the wiki states,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Ralstonia solanacearum is an aerobic non-spore-forming,&#xA;Gram-negative, plant pathogenic bacterium.&amp;quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I'd leave it. The simple explanation is there has been a bad mix up. The best explanation is the people loading the Illumina machine, designating the adaptors etc ... In this scenario it has nothing to do with bacterial genome integration, plant bacteria don't touch mice and aerobic bacteria wouldn't grow in cancer cells.&lt;/p&gt;&#xA;&lt;p&gt;The second issue is what is a soil bacteria doing in a mouse cancer biological facility? For the staff this wouldn't be good news.&lt;/p&gt;&#xA;&lt;p&gt;The singular biological  possibility is its a Pseudomonas  and some species are serious opportunistic pathogens of immune-weakened humans, but there are a lot of Pseudomonas species and this one doesn't ring any bells, BUT its mice so who knows? One species in particular is associated with hospital outbreaks, but a biomedical unit that isn't quite the same thing.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The pathogen in humans is called Pseudomonas aeruginosa. If hospitals get this infection it is BAD NEWS, for example burns victims are really susceptible and without intervention it kills them. If you have evidence that the bacterial equivalent in mice is the bacterial pathogen you have found that is a substantial step. Biological Safety Facilities (BSFs) DEFINITELY get infected. They hate it when this happens and can be very picky for fear of a contamination, but it certainly happens. If you have evidence of a major contaminent pathogen in the long run it would be of HUGE help to BSFs globally. No-one really bothers with the disease pathology of mice - because their mice and spend their entire lives going 'squeek' - but when they are models for cancer their status changes beyond recognition. So I'd erase my initial thoughts and suggest, what you have found is publishable and the application is BSF contamination - don't say that though say this could be the mouse equivalent of P. aeruginosa. Its certainly a publication and goodluck.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-13T13:21:59.357" LastActivityDate="2020-07-13T13:21:59.357" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13867" PostTypeId="2" ParentId="13853" CreationDate="2020-07-11T23:43:22.900" Score="0" Body="&lt;p&gt;As @vkkodali pointed out, the problem turned out to be that you shouldn't use multithreading.&lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2020-07-11T23:43:22.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13868" PostTypeId="2" ParentId="13861" CreationDate="2020-07-12T03:03:07.410" Score="1" Body="&lt;p&gt;What exactly is the sequence?  Are you sure it's not an artifact of the library prep?&lt;/p&gt;&#xA;&lt;p&gt;5 seconds of googling turns up this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://support.illumina.com/content/dam/illumina-support/documents/documentation/chemistry_documentation/experiment-design/illumina-adapter-sequences-1000000002694-13.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.illumina.com/content/dam/illumina-support/documents/documentation/chemistry_documentation/experiment-design/illumina-adapter-sequences-1000000002694-13.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This sequence:&lt;/p&gt;&#xA;&lt;p&gt;ACCACCGAGATCTACA&lt;/p&gt;&#xA;&lt;p&gt;is on the adapters.&lt;/p&gt;&#xA;&lt;p&gt;So is this&lt;/p&gt;&#xA;&lt;p&gt;GACGGCATACGAGAT&lt;/p&gt;&#xA;&lt;p&gt;Bacterial contamination would not put bacterial sequence at the end of host transcripts!  But adapter contamination sure could be.&lt;/p&gt;&#xA;&lt;p&gt;Here's the index 2 adapter&lt;/p&gt;&#xA;&lt;p&gt;AATGATACGGCGAC&lt;strong&gt;CACCGAGATCTACAC[i5]TCGTCGGCAGCGTCAGATGTGTATAAGAGACAG&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here's your sequence&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;CACCACCGAGATCTACAC&lt;/strong&gt;CTAAGCCT&lt;strong&gt;TCGTCGGCAGCGTCAGATGTGTATAAGAGACAG&lt;/strong&gt;GTATCAACGCAGAGTACGGG&lt;/p&gt;&#xA;&lt;p&gt;Here's it's blast against nr&lt;/p&gt;&#xA;&lt;p&gt;AC&lt;strong&gt;CACCGAGATCTACACCTAAGCCTTCGTCGGCAGCGTCAGATGTGTATAAGAGACAG&lt;/strong&gt;GTATC&lt;/p&gt;&#xA;&lt;p&gt;It's all adapter.&lt;/p&gt;&#xA;&lt;p&gt;Whoever did the transcriptome assembly didn't trim for adapters first.  That's all.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastEditorUserId="2388" LastEditDate="2020-07-13T20:07:02.173" LastActivityDate="2020-07-13T20:07:02.173" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13869" PostTypeId="1" AcceptedAnswerId="13880" CreationDate="2020-07-12T03:45:35.017" Score="1" ViewCount="305" Body="&lt;p&gt;I have the following code to download data from the SRA using multithreading in Python. After running this a few times now (for testing purposes), I keep getting denied access to the data. Not sure how to fix this. In particular, &lt;code&gt;cat&lt;/code&gt;ing the output files gives:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;?xml version=&amp;quot;1.0&amp;quot; encoding=&amp;quot;UTF-8&amp;quot;?&amp;gt;&amp;lt;Error&amp;gt;&amp;lt;Code&amp;gt;AccessDenied&amp;lt;/Code&amp;gt;&amp;lt;Message&amp;gt;Access Denied&amp;lt;/Message&amp;gt;&amp;lt;RequestId&amp;gt;cb0f0e98-cafb-1dd7-9b7b-d8c49756ec52&amp;lt;/RequestId&amp;gt;&amp;lt;HostId&amp;gt;QeDGVwBXYp61J0B4_OUTn7UsEsiQEec0n18DAeR0kaE&amp;lt;/HostId&amp;gt;&amp;lt;/Error&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here is my code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import threading&#xA;import requests&#xA;import time &#xA;&#xA;start = time.perf_counter()&#xA;&#xA;class MyThread(threading.Thread):&#xA;    def __init__(self, url):&#xA;        threading.Thread.__init__(self)&#xA;        self.url = url&#xA;        self.result = None&#xA;        self.filename = url.split('/')[-1]&#xA;    def run(self):&#xA;        res = requests.get(self.url)&#xA;        with open(self.filename, 'wb') as f:&#xA;            f.write(res.content)&#xA;urls = [&#xA;        'https://sra-downloadb.be-md.ncbi.nlm.nih.gov/sos1/sra-pub-run-1/SRR000001/SRR000001.1',&#xA;        'https://sra-downloadb.be-md.ncbi.nlm.nih.gov/sos1/sra-pub-run-1/SRR000001/SRR000001.2',&#xA;        'https://sra-downloadb.be-md.ncbi.nlm.nih.gov/sos1/sra-pub-run-1/SRR000002/SRR000002.1',&#xA;        'https://sra-downloadb.be-md.ncbi.nlm.nih.gov/sos1/sra-pub-run-1/SRR000002/SRR000002.2']&#xA;&#xA;threads = [MyThread(url, ) for url in urls]&#xA;for thread in threads:&#xA;    thread.start()&#xA;for thread in threads:&#xA;    thread.join()&#xA;&#xA;finish = time.perf_counter()&#xA;&#xA;print(f'Finished in {round(finish-start, 2)} second(s)')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2020-07-13T15:07:51.370" Title="What is the best way to get a large number of RNA seq data from SRA in Python without being denied access" Tags="|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13871" PostTypeId="1" CreationDate="2020-07-12T18:17:11.670" Score="1" ViewCount="202" Body="&lt;p&gt;I am downloading a large number of files from the ENA in Python using Multithreading. As an example, to be more concrete, I tried downloading 12 files simultaneously using 12 threads. I noticed that 10 files download relatively quickly, while 2 lag behind. Even when the 10 files are done downloading, the speed at which the other 2 files download is very slow. If I then try downloading these 2 files again, the speed will be a lot better. So what exactly is happening and how can I fix this?&lt;/p&gt;&#xA;&lt;p&gt;PS: I tried decreasing the thread count to 8, so 8 files were downloading simultaneously, and I had 1 file that was lagging the rest again.&lt;/p&gt;&#xA;&lt;p&gt;PPS: I then tried decreasing to 6 threads, and again the same issue. One file will lag. I don't have this issue if I directly download from Firefox, and I get a significant speed up, but I'm trying to automate the process so downloading from Firefox isn't an option.&lt;/p&gt;&#xA;&lt;p&gt;For reference, here's the code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import queue&#xA;import threading&#xA;import time&#xA;&#xA;&#xA;start = time.perf_counter()&#xA;&#xA;    &#xA;class MyThread(threading.Thread):&#xA;    def __init__(self, name):&#xA;        threading.Thread.__init__(self)&#xA;        self.name = name&#xA;    def run(self):&#xA;        print('Starting thread %s.' % self.name)&#xA;        process_queue()&#xA;        print('Exiting thread %s.' % self.name)&#xA;        &#xA;def process_queue():&#xA;    while True:&#xA;        try:&#xA;            url = my_queue.get(block=False)&#xA;            filename = url.split('/')[-1]&#xA;            with closing(request.urlopen(url)) as r:&#xA;                with open(filename, 'wb') as f:&#xA;                    shutil.copyfileobj(r, f)&#xA;        except queue.Empty:&#xA;            return&#xA;        &#xA;# setting up variables&#xA;urls = [&#xA;        'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR824/000/SRR8240860/SRR8240860_1.fastq.gz',&#xA;        'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR824/000/SRR8240860/SRR8240860_2.fastq.gz',&#xA;        'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR824/001/SRR8240861/SRR8240861_1.fastq.gz',&#xA;        'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR824/001/SRR8240861/SRR8240861_2.fastq.gz',&#xA;        'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR824/002/SRR8240862/SRR8240862_1.fastq.gz',&#xA;        'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR824/002/SRR8240862/SRR8240862_2.fastq.gz',&#xA;        'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR824/003/SRR8240863/SRR8240863_1.fastq.gz',&#xA;        'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR824/003/SRR8240863/SRR8240863_2.fastq.gz',&#xA;        'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR824/004/SRR8240864/SRR8240864_1.fastq.gz',&#xA;        'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR824/004/SRR8240864/SRR8240864_2.fastq.gz',&#xA;        'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR824/005/SRR8240865/SRR8240865_1.fastq.gz',&#xA;        'ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR824/005/SRR8240865/SRR8240865_2.fastq.gz',&#xA;        ]&#xA;&#xA;# filling the queue&#xA;my_queue = queue.Queue()&#xA;&#xA;for url in urls:&#xA;    my_queue.put(url)&#xA;&#xA;# initializing and starting num_threads threads&#xA;num_threads = 8&#xA;threads = []&#xA;&#xA;for i in range(num_threads):&#xA;    thread = MyThread(i)&#xA;    threads.append(thread)&#xA;&#xA;for thread in threads:&#xA;    thread.start()&#xA;&#xA;for thread in threads:&#xA;    thread.join()&#xA;    &#xA;finish = time.perf_counter()&#xA;&#xA;print(f'Finished in {round(finish-start, 2)} second(s)')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="8601" LastEditDate="2020-07-12T18:35:07.977" LastActivityDate="2020-07-12T18:35:07.977" Title="Trying to download a large number of files from ENA programmatically" Tags="|python|data-download|multithreading|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13872" PostTypeId="1" CreationDate="2020-07-12T23:03:29.447" Score="2" ViewCount="582" Body="&lt;p&gt;How can I programmatically obtain ftp links to RNA seq fastq files in ENA? Here's an example of a link that I would be interested in obtaining:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR824/000/SRR8240860/SRR8240860_1.fastq.gz&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;In particular, is there some tool that, given the BioProject ID (here, PRJNA506829), would be able give me all ftp links for the runs in the project, or would I need to write a web scraper to do it?&lt;/p&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="73" LastEditDate="2022-07-05T01:39:48.637" LastActivityDate="2022-07-06T04:11:22.860" Title="How can I obtain FTP links to studies in ENA given a BioProject ID?" Tags="|rna-seq|public-databases|data-retrieval|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13873" PostTypeId="2" ParentId="13857" CreationDate="2020-07-12T23:58:20.723" Score="1" Body="&lt;p&gt;I would suggest a &lt;a href=&quot;https://weblogo.berkeley.edu/logo.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;logo plot&lt;/a&gt;, e.g.:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Udws0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Udws0.png&quot; alt=&quot;https://weblogo.berkeley.edu/examples.html&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would take @Devon Ryan's suggestion to create a multiple sequence alignment (MSA), and plug that into that logo plot tool (or others, there are many like it).&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-07-12T23:58:20.723" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13874" PostTypeId="2" ParentId="13872" CreationDate="2020-07-13T06:51:21.073" Score="2" Body="&lt;p&gt;Here are some command lines I used for that purpose in bash.&#xA;Simply prepare a text file containing each accession number (SRR/ERR) you want and create a for loop. Here I used &lt;a href=&quot;https://github.com/totosugito/prozilla-2.0.4&quot; rel=&quot;nofollow noreferrer&quot;&gt;prozilla&lt;/a&gt; to speed up downloads but you may use &lt;code&gt;wget&lt;/code&gt; either.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for index in $(cat list_of_accessions) ; do&#xA;&#xA;if [ &lt;span class=&quot;math-container&quot;&gt;${#index} -eq 9 ]; then&#xA;            proz -k=6 --no-curses ftp.sra.ebi.ac.uk/vol1/fastq/$&lt;/span&gt;{index:0:6}/&lt;span class=&quot;math-container&quot;&gt;${index}/$&lt;/span&gt;{index}_1.fastq.gz&#xA;            proz -k=6 --no-curses ftp.sra.ebi.ac.uk/vol1/fastq/&lt;span class=&quot;math-container&quot;&gt;${index:0:6}/$&lt;/span&gt;{index}/&lt;span class=&quot;math-container&quot;&gt;${index}_2.fastq.gz&#xA;            else&#xA;            if [ $&lt;/span&gt;{#index} -eq 10 ]; then&#xA;                    proz -k=6 --no-curses ftp.sra.ebi.ac.uk/vol1/fastq/&lt;span class=&quot;math-container&quot;&gt;${index:0:6}/00$&lt;/span&gt;{index:9:9}/&lt;span class=&quot;math-container&quot;&gt;${index}/$&lt;/span&gt;{index}_1.fastq.gz&#xA;                            proz -k=6 --no-curses ftp.sra.ebi.ac.uk/vol1/fastq/&lt;span class=&quot;math-container&quot;&gt;${index:0:6}/00$&lt;/span&gt;{index:9:9}/&lt;span class=&quot;math-container&quot;&gt;${index}/$&lt;/span&gt;{index}_2.fastq.gz&#xA;                    else&#xA;                    if [ &lt;span class=&quot;math-container&quot;&gt;${#index} -eq 11 ]; then&#xA;                     proz -k=6 --no-curses ftp.sra.ebi.ac.uk/vol1/fastq/$&lt;/span&gt;{index:0:6}/0&lt;span class=&quot;math-container&quot;&gt;${index:9:10}/$&lt;/span&gt;{index}/&lt;span class=&quot;math-container&quot;&gt;${index}_1.fastq.gz&#xA;                                        proz -k=6 --no-curses ftp.sra.ebi.ac.uk/vol1/fastq/$&lt;/span&gt;{index:0:6}/0&lt;span class=&quot;math-container&quot;&gt;${index:9:10}/$&lt;/span&gt;{index}/&lt;span class=&quot;math-container&quot;&gt;${index}_2.fastq.gz&#xA;                    else&#xA;                        proz -k=6 --no-curses ftp.sra.ebi.ac.uk/vol1/fastq/$&lt;/span&gt;{index:0:6}/&lt;span class=&quot;math-container&quot;&gt;${index:9:11}/$&lt;/span&gt;{index}/&lt;span class=&quot;math-container&quot;&gt;${index}_1.fastq.gz&#xA;                                                proz -k=6 --no-curses ftp.sra.ebi.ac.uk/vol1/fastq/$&lt;/span&gt;{index:0:6}/&lt;span class=&quot;math-container&quot;&gt;${index:9:11}/$&lt;/span&gt;{index}/${index}_2.fastq.gz&#xA;                        fi&#xA;                fi&#xA;            &#xA;            fi&#xA;done&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8749" LastActivityDate="2020-07-13T06:51:21.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13875" PostTypeId="1" CreationDate="2020-07-13T09:39:18.583" Score="0" ViewCount="104" Body="&lt;p&gt;I need to calculate Isoelectric point from a multifasta file, is there any python code or web tool that allows me to do that?&lt;/p&gt;&#xA;" OwnerUserId="9174" LastActivityDate="2020-07-13T11:14:08.837" Title="Calculating Isoelectric point from a multifasta file" Tags="|fasta|proteins|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13876" PostTypeId="2" ParentId="13875" CreationDate="2020-07-13T11:14:08.837" Score="4" Body="&lt;p&gt;You can use &lt;a href=&quot;https://www.ebi.ac.uk/Tools/seqstats/emboss_pepstats/&quot; rel=&quot;nofollow noreferrer&quot;&gt;EMBOSS Pepstats&lt;/a&gt; for this. It takes a multi-fasta as input and produces a table that contains various statistics related to the protein, including isoelectric point.&lt;/p&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2020-07-13T11:14:08.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13878" PostTypeId="2" ParentId="13872" CreationDate="2020-07-13T14:06:18.707" Score="1" Body="&lt;p&gt;They have an &lt;a href=&quot;https://ena-docs.readthedocs.io/en/latest/retrieval/programmatic-access/browser-api.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;API&lt;/a&gt; you can interact with.&lt;/p&gt;&#xA;&lt;p&gt;If you need to get files for only a few different projects:&#xA;Search for your accession ID in the browser (leading to &lt;a href=&quot;https://www.ebi.ac.uk/ena/browser/view/PRJNA506829&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/ena/browser/view/PRJNA506829&lt;/a&gt;). Filter the show selected columns to only fastq_ftp, click download tsv to get the list of ftp links.&lt;/p&gt;&#xA;" OwnerUserId="3818" LastEditorUserId="1157" LastEditDate="2022-07-04T07:48:17.030" LastActivityDate="2022-07-04T07:48:17.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13879" PostTypeId="1" AcceptedAnswerId="13882" CreationDate="2020-07-13T14:17:02.450" Score="2" ViewCount="128" Body="&lt;p&gt;I have bacterial chromosomes/plasmids that I have downloaded off refseq in a multi-fasta that I would like to fragment into non-overlapping fragments of between 1 kb and 15 kb.&lt;/p&gt;&#xA;&lt;p&gt;I have tried googling for related tools but to no avail. Does anyone know how I would achieve this? Something very similar is done in this paper:&#xA;&lt;a href=&quot;https://link.springer.com/article/10.1007/s40484-019-0187-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://link.springer.com/article/10.1007/s40484-019-0187-4&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="7287" LastActivityDate="2020-07-15T18:58:53.347" Title="How to fragment genomes into non-overlapping sequences of differing sizes?" Tags="|assembly|bacteria|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13880" PostTypeId="2" ParentId="13869" CreationDate="2020-07-13T15:07:51.370" Score="0" Body="&lt;p&gt;I once needed to download more than 1000 files from SRA, and I found a blog post that is very useful (&lt;a href=&quot;https://reneshbedre.github.io/blog/fqutil.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://reneshbedre.github.io/blog/fqutil.html&lt;/a&gt;). The idea is to use aspera to fetch SRA files. This is fast becasue of aspera. Then use fasterq-dump to convert SRA file to fastq. fasterq-dump is faster than fastq-dump partly because it does not compress the output file. I think compressing is the rate-limiting step here. After this, you can use pigz, which is a multi-threading version of gzip to compress the fastq files. Note that if you want to download files to a computing cluster, you can submit jobs for fasterq-dump and pigz. This makes things much faster. You cannot submit jobs for fetching SRA, because you need to connect to internet.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-07-13T15:07:51.370" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13881" PostTypeId="1" CreationDate="2020-07-13T15:08:51.747" Score="4" ViewCount="955" Body="&lt;p&gt;A collaborator has passed me over Kraken2 outputs *.report and *.kraken, from a metatranscriptomic sequencing experiment conducted on the minION.&lt;/p&gt;&#xA;&lt;p&gt;I would like to make a tree if the data using a standard phylogenetics package such as phyloseq, however, I am having trouble making an alignment via the OTU table..&lt;/p&gt;&#xA;&lt;p&gt;There has been a similar question on here already, but the author has resolved it and the solution is not available.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have any pointers?&lt;/p&gt;&#xA;" OwnerUserId="9191" LastEditorUserId="3848" LastEditDate="2022-02-23T09:38:57.120" LastActivityDate="2022-02-23T09:38:57.120" Title="Kraken2 &gt; OTU format &gt; Phyloseq" Tags="|phylogenetics|metagenome|minion|taxonomy|microbial-genomics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13882" PostTypeId="2" ParentId="13879" CreationDate="2020-07-13T19:33:37.747" Score="3" Body="&lt;p&gt;You could use &lt;code&gt;numpy.random.randint&lt;/code&gt; to sample integers from a uniform distribution, from 1000 to 15000 nt, extracting subsequences of that length from a linearized multi-FASTA file.&lt;/p&gt;&#xA;&lt;p&gt;Here's some code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;&#xA;import sys&#xA;import numpy as np&#xA;&#xA;low = int(sys.argv[1])&#xA;high = int(sys.argv[2])&#xA;&#xA;def process_record(header, sequence, low, high):&#xA;    next = 0&#xA;    fragment = 1&#xA;    while next &amp;lt; len(sequence):&#xA;        new_next = np.random.randint(low, high) + next  &#xA;        if new_next &amp;gt; len(sequence) - high:&#xA;            new_next = len(sequence)&#xA;        sys.stdout.write('{}:{}\n{}\n'.format(header, fragment, sequence[next:new_next]))&#xA;        next = new_next&#xA;        fragment += 1&#xA;&#xA;header = &amp;quot;&amp;quot;&#xA;sequence = &amp;quot;&amp;quot;&#xA;for line in sys.stdin:&#xA;    if line.startswith('&amp;gt;'):&#xA;        if len(sequence) &amp;gt; 0:&#xA;            process_record(header, sequence, low, high)&#xA;            sequence = &amp;quot;&amp;quot;&#xA;        header = line.rstrip()&#xA;    else:&#xA;        sequence += line.rstrip()&#xA;&#xA;if len(sequence) &amp;gt; 0:&#xA;    process_record(header, sequence, low, high)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To use it:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ ./13879.py 1000 15000 &amp;lt; in.fa &amp;gt; out.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Fragments will be disjoint (non-overlapping, contiguous) subsequences.&lt;/p&gt;&#xA;&lt;p&gt;I add a check to determine if the last fragment of a FASTA record would be shorter than &lt;code&gt;low&lt;/code&gt; (1kb) in length, and, if so, append that fragment to the final subsequence. It is possible for that last fragment to be up to 16 kb - 1 nt in length, given your parameters:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;if new_next &amp;gt; len(sequence) - high:&#xA;    new_next = len(sequence)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You could change this logic to discard the last fragment (i.e. &lt;code&gt;break&lt;/code&gt;), or simply allow fragments less than 1kb in size, depending on what you need.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here's a slight modification that should allow to you preserve fragment sizes within your bounds.&lt;/p&gt;&#xA;&lt;p&gt;A cartoon may help illustrate the problem. Let's set up a FASTA record, which has been broken up into &lt;code&gt;n&lt;/code&gt; fragments:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   1       2           n-2      n-1      n&#xA;|-----|---------|...|-------|---------|-----|&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We may get unlucky and sample a random integer for fragment &lt;code&gt;n-1&lt;/code&gt; (&lt;code&gt;frag(n-1)&lt;/code&gt;) which makes the length of &lt;code&gt;frag(n)&lt;/code&gt; less than &lt;code&gt;low&lt;/code&gt; (1 kb in your example).&lt;/p&gt;&#xA;&lt;p&gt;In my first suggested solution, &lt;code&gt;frag(n-1)&lt;/code&gt; and &lt;code&gt;frag(n)&lt;/code&gt; are appended to one another. This is why you get some fragments longer than &lt;code&gt;high&lt;/code&gt; (15 kb).&lt;/p&gt;&#xA;&lt;p&gt;Another option is to discard &lt;code&gt;frag(n)&lt;/code&gt;, which is of length less than &lt;code&gt;low&lt;/code&gt; (1 kb). Here's some code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;&#xA;import sys&#xA;import numpy as np&#xA;import timeit&#xA;&#xA;low = int(sys.argv[1])&#xA;high = int(sys.argv[2])&#xA;in_fn = sys.argv[3]&#xA;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;Globals&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;fragment_lengths = []&#xA;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;Process a record&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;def process_record(header, sequence, low, high):&#xA;    next = 0&#xA;    last = 0&#xA;    fragment = 1&#xA;    fragment_sequence = &amp;quot;&amp;quot;&#xA;    while last &amp;lt; len(sequence) - high:&#xA;        next = np.random.randint(low, high + 1) + last&#xA;        fragment_sequence += sequence[last:next]&#xA;        # Comment the sys.stdout.write and uncomment the fragment_lengths line, if measuring fragment length distributions &#xA;        sys.stdout.write('{}:{}-{}:{}\n{}\n'.format(header, last, next, fragment, sequence[last:next]))&#xA;        #fragment_lengths.append(next - last)                                                                                                                                                                                                                                                                                                                                                                                          &#xA;        last = next&#xA;        fragment += 1&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    Uncomment to compare input sequence with fragments&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    #sys.stderr.write('In &amp;gt; {}\nOut &amp;gt; {}\n'.format(sequence, fragment_sequence))&#xA;    &#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;Parse records&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;def parse_records(in_fn, low, high):&#xA;    header = &amp;quot;&amp;quot;&#xA;    sequence = &amp;quot;&amp;quot;&#xA;    with open(in_fn, 'r') as ifh:&#xA;        for line in ifh:&#xA;            if line.startswith('&amp;gt;'):&#xA;                if len(sequence) &amp;gt; 0:&#xA;                    process_record(header, sequence, low, high)&#xA;                    sequence = &amp;quot;&amp;quot;&#xA;                header = line.rstrip()&#xA;            else:&#xA;                sequence += line.rstrip()&#xA;        if len(sequence) &amp;gt; 0:&#xA;            process_record(header, sequence, low, high)&#xA;&#xA;def count_elements(seq):&#xA;    hist = {}&#xA;    for i in seq:&#xA;        hist[i] = hist.get(i, 0) + 1&#xA;    return hist&#xA;&#xA;def wrapper(func, *args, **kwargs):&#xA;    def wrapped():&#xA;        return func(*args, **kwargs)&#xA;    return wrapped&#xA;&#xA;def main(in_fn, low, high):&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    Comment below if testing the distribution of fragments&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    parse_records(in_fn, low, high)&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    Uncomment to test the distribution of fragments from &#xA;    running fns on same input 100k times&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    #wrapped = wrapper(parse_records, in_fn, low, high)&#xA;    #timeit.timeit(wrapped, number=100000)&#xA;    #print(count_elements(fragment_lengths))&#xA;&#xA;if __name__ == &amp;quot;__main__&amp;quot;:&#xA;    main(in_fn, low, high)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If &lt;code&gt;frag(n)&lt;/code&gt; is needed, and you need to preserve the uniform distribution (or whatever fragment length distribution you want to model), then &lt;a href=&quot;https://en.wikipedia.org/wiki/Rejection_sampling&quot; rel=&quot;nofollow noreferrer&quot;&gt;rejection sampling&lt;/a&gt; might be an option.&lt;/p&gt;&#xA;&lt;p&gt;With rejection sampling, you keep fragmenting your input sequence, until one of the distributions of lengths gets you a set of fragments that entirely cover the input sequence.&lt;/p&gt;&#xA;&lt;p&gt;If you need that, follow up and I'll take another look.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2020-07-15T18:58:53.347" LastActivityDate="2020-07-15T18:58:53.347" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13883" PostTypeId="2" ParentId="13856" CreationDate="2020-07-13T20:22:15.307" Score="1" Body="&lt;p&gt;I got the answer from Rory Stark. the samples should have a unique ID. (&lt;a href=&quot;https://support.bioconductor.org/p/132502/#132514&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.bioconductor.org/p/132502/#132514&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="8543" LastActivityDate="2020-07-13T20:22:15.307" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13884" PostTypeId="2" ParentId="13879" CreationDate="2020-07-14T04:54:58.570" Score="1" Body="&lt;p&gt;I've previously written &lt;a href=&quot;https://gitlab.com/gringer/bioinfscripts/-/blob/master/normalise_seqlengths.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;my own script&lt;/a&gt; to fragment DNA sequences (input as either fastq or fasta files) into identical-length fragments, with a given overlap. I used this because there was a particular assembler that demanded equal-length sequences, and I was working with nanopore reads, but it's since been useful for other purposes as well.&lt;/p&gt;&#xA;&lt;p&gt;Because it is most concerned about making lengths identical, there may be some forced overlap when setting the overlap to zero. The overlap will only be exactly zero when the sequence length is an exact multiple of the fragment size. Here's an example use:&lt;/p&gt;&#xA;&lt;h2&gt;zero overlap&lt;/h2&gt;&#xA;&lt;pre&gt;&lt;code&gt;normalise_seqlengths.pl -fraglength 25 -overlap 0 oneRead.fq&#xA;@A00187:180:HLGNGDSXX:3:1101:8486:1031#0 [150 bp, 6 fragments, 0 overlap] 1:N:0:CGCAAGAA+ACTCCTAA&#xA;ATTGCGCAATGACAGCCAGGGGAGT&#xA;+&#xA;FFFFFFFFFFFFFFFFF:,FFFFFF&#xA;@A00187:180:HLGNGDSXX:3:1101:8486:1031#1:25-50  1:N:0:CGCAAGAA+ACTCCTAA&#xA;GGGAGCTGTCTGGGGTAACCTGGCT&#xA;+&#xA;:FFFF:F:FFFFF::FFFFF,::FF&#xA;@A00187:180:HLGNGDSXX:3:1101:8486:1031#2:50-75  1:N:0:CGCAAGAA+ACTCCTAA&#xA;CTAGGACCAGGTGGAAAAAGCACCT&#xA;+&#xA;F:FFFFFFFFFFFFFFFFFFFFFFF&#xA;@A00187:180:HLGNGDSXX:3:1101:8486:1031#3:75-100  1:N:0:CGCAAGAA+ACTCCTAA&#xA;TGGCTGGCCCAGCCCTGTTCTTCTC&#xA;+&#xA;:FF,FFFFF,:,FFF::F:FFF,:F&#xA;@A00187:180:HLGNGDSXX:3:1101:8486:1031#4:100-125  1:N:0:CGCAAGAA+ACTCCTAA&#xA;CTGGGGAGGGAAGGGCAGGTCCCCT&#xA;+&#xA;F:FFFFFFFFF,FFFFF:FFF,F,,&#xA;@A00187:180:HLGNGDSXX:3:1101:8486:1031#5:125-150  1:N:0:CGCAAGAA+ACTCCTAA&#xA;AATGCCTGAACCACATGGTGGGCCA&#xA;+&#xA;FFF,F:FFFFFFF:F,,,FFF,F,,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;partial overlap&lt;/h2&gt;&#xA;&lt;pre&gt;&lt;code&gt;normalise_seqlengths.pl -fraglength 63 -overlap 0 oneRead.fq&#xA;@A00187:180:HLGNGDSXX:3:1101:8486:1031#0 [150 bp, 3 fragments, 20 overlap] 1:N:0:CGCAAGAA+ACTCCTAA&#xA;ATTGCGCAATGACAGCCAGGGGAGTGGGAGCTGTCTGGGGTAACCTGGCTCTAGGACCAGGTG&#xA;+&#xA;FFFFFFFFFFFFFFFFF:,FFFFFF:FFFF:F:FFFFF::FFFFF,::FFF:FFFFFFFFFFF&#xA;@A00187:180:HLGNGDSXX:3:1101:8486:1031#1:43-106  1:N:0:CGCAAGAA+ACTCCTAA&#xA;CCTGGCTCTAGGACCAGGTGGAAAAAGCACCTTGGCTGGCCCAGCCCTGTTCTTCTCCTGGGG&#xA;+&#xA;FF,::FFF:FFFFFFFFFFFFFFFFFFFFFFF:FF,FFFFF,:,FFF::F:FFF,:FF:FFFF&#xA;@A00187:180:HLGNGDSXX:3:1101:8486:1031#2:87-150  1:N:0:CGCAAGAA+ACTCCTAA&#xA;CCCTGTTCTTCTCCTGGGGAGGGAAGGGCAGGTCCCCTAATGCCTGAACCACATGGTGGGCCA&#xA;+&#xA;FFF::F:FFF,:FF:FFFFFFFFF,FFFFF:FFF,F,,FFF,F:FFFFFFF:F,,,FFF,F,,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2020-07-14T04:54:58.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13885" PostTypeId="2" ParentId="13865" CreationDate="2020-07-14T10:11:33.260" Score="2" Body="&lt;p&gt;Below I import the peaks call in narrowbed format from macs2 into a GRanges object. If you have a bed file, you can just use rtracklayer for it:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(rtracklayer)&#xA;library(BSgenome.Hsapiens.UCSC.hg38)&#xA;&#xA;peaks_narrowbed = &amp;quot;https://www.encodeproject.org/files/ENCFF846JAO/@@download/ENCFF846JAO.bed.gz&amp;quot;&#xA;&#xA;extraCols &amp;lt;- c(signalValue = &amp;quot;numeric&amp;quot;, pValue = &amp;quot;numeric&amp;quot;,&#xA;                          qValue = &amp;quot;numeric&amp;quot;, peak = &amp;quot;integer&amp;quot;)&#xA;gr &amp;lt;- import(peaks_narrowbed, format = &amp;quot;BED&amp;quot;,extraCols = extraCols)&#xA;&#xA;head(gr)&#xA;&#xA;GRanges object with 6 ranges and 6 metadata columns:&#xA;      seqnames        ranges strand |&#xA;         &amp;lt;Rle&amp;gt;     &amp;lt;IRanges&amp;gt;  &amp;lt;Rle&amp;gt; |&#xA;  [1]     chr1   10049-10326      * |&#xA;  [2]     chr1 180686-181040      * |&#xA;  [3]     chr1 186648-186903      * |&#xA;  [4]     chr1 191293-191604      * |&#xA;  [5]     chr1 267872-268146      * |&#xA;  [6]     chr1 586110-586324      * |&#xA;                                                   name     score signalValue&#xA;                                            &amp;lt;character&amp;gt; &amp;lt;numeric&amp;gt;   &amp;lt;numeric&amp;gt;&#xA;  [1] atacseq_170613.t8_rep1.dedup.masked.sorted_peak_1       135     7.10907&#xA;  [2] atacseq_170613.t8_rep1.dedup.masked.sorted_peak_2       309    11.65888&#xA;  [3] atacseq_170613.t8_rep1.dedup.masked.sorted_peak_3        81     5.40289&#xA;  [4] atacseq_170613.t8_rep1.dedup.masked.sorted_peak_4        56     4.54981&#xA;  [5] atacseq_170613.t8_rep1.dedup.masked.sorted_peak_5       145     7.39343&#xA;  [6] atacseq_170613.t8_rep1.dedup.masked.sorted_peak_6       116     6.54035&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then we extract the sequences that are within these regions, you can of course redefine the region by extending it from the peak:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(Biostrings)&#xA;SEQ = getSeq(Hsapiens,gr)&#xA;head(SEQ)&#xA;&#xA;A DNAStringSet instance of length 6&#xA;    width seq&#xA;[1]   278 TAACCCTAACCCTAACCCTAACCCTAACCCTAAC...CCCCAACCCCAACCCCAACCCCAACCCCAACCC&#xA;[2]   355 CTAACCCCTAACCCCTAACCCTAACCCTACCCTA...CTCAGCCGGCCCGCCCGCCCGGGTCTGACCTGA&#xA;[3]   256 GCAACCACCTGAGCGCGGGCATCCTGTGTGCAGA...CCCCTTCTTTCCATTGGTTTAATTAGGAACGGG&#xA;[4]   312 CAATCAGCAGGGACCGTGCACTCTCTTGGAGCCA...CCCTGCCCCTGTCTCTTCCGTGCAGGAGGAGCA&#xA;[5]   275 TGGATGTGTGCATTTTCCTGAGAGGAAAGCTTTC...TTGTTTAGTTTGCGTTGTGTTTCTCCAACTTTG&#xA;[6]   215 GAGGCTGAAGGAGACTGATGTGGTTTCTCCTCAG...CCAACTTTGTGCCTCATCAGGAAAAGCTTTGGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the CG, and you can decide how you want to bin your regions:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;freq = dinucleotideFrequency(SEQ,as.prob=TRUE)&#xA;hist(freq[,&amp;quot;CG&amp;quot;],br=100)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZTomm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZTomm.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-07-14T10:11:33.260" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="13887" PostTypeId="2" ParentId="5352" CreationDate="2020-07-14T12:28:23.420" Score="1" Body="&lt;p&gt;By now this will be of little interest to the original poster, but for future reference:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ samtools sort -n -T ~/[etc] -o sample${r}bowtiemapping_sorted.bam [etc]&#xA;$ samtools index sample${r}bowtiemapping_sorted.bam&#xA;[E::hts_idx_push] NO_COOR reads not in a single block at the end 1652 -1.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This error message indicates (admittedly somewhat obscurely) that the reads are not sorted by coordinate — in particular, there are mapped reads appearing after the unplaced unmapped reads (those with RNAME &lt;code&gt;*&lt;/code&gt;) that should be at the very end of the file.&lt;/p&gt;&#xA;&lt;p&gt;This is because &lt;code&gt;samtools sort -n&lt;/code&gt; has been used to sort the reads by name instead. Remove &lt;code&gt;-n&lt;/code&gt; to sort by position, which is what is needed to prepare a BAM file for indexing with &lt;code&gt;samtools index&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="134" LastActivityDate="2020-07-14T12:28:23.420" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13888" PostTypeId="1" CreationDate="2020-07-14T15:25:23.213" Score="1" ViewCount="18" Body="&lt;p&gt;A quick terminology question today. I see the terms &amp;quot;cross-ancestry&amp;quot; and &amp;quot;trans-ethnic&amp;quot; used seemingly interchangeably in literature. Is there any real difference between those two or are the 100% the same?&lt;/p&gt;&#xA;" OwnerUserId="9194" LastActivityDate="2020-07-14T15:25:23.213" Title="Difference between &quot;trans-ethnic&quot; and &quot;cross-ancestry&quot;" Tags="|human-genome|1000genomes|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13889" PostTypeId="1" CreationDate="2020-07-14T16:31:19.737" Score="1" ViewCount="108" Body="&lt;p&gt;I installed gzip package in Conda environment, then  to make sure the software was there, I used &amp;quot;ls&amp;quot; command, it was not in the list. Tried to install again, system says all the packages have already installed, Why? How do I check the list of software installed in the environment? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="9196" LastActivityDate="2020-07-14T17:41:29.590" Title="downloaded software in Conda environment does not show in ls command" Tags="|conda|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13890" PostTypeId="2" ParentId="10641" CreationDate="2020-07-14T16:37:00.587" Score="1" Body="&lt;p&gt;I don't see why you are trying to compare two totally different measurements.&lt;/p&gt;&#xA;&lt;p&gt;When FASTQC says a read is unique, it means that no other read shares its sequence.&lt;/p&gt;&#xA;&lt;p&gt;When STAR says a read is uniquely &lt;strong&gt;mapped&lt;/strong&gt;, that means it maps to one and only place.  That does not mean that there aren't a hundred more identical reads that map nicely to the exact same place.&lt;/p&gt;&#xA;&lt;p&gt;You could have a very high level of PCR duplication of reads that nicely align to one and only one place. Or you could have a whole bunch of reads that differ slightly from each other who all map to 5 locations each.&lt;/p&gt;&#xA;&lt;p&gt;FASTQC was designed for DNA sequencing, not RNASeq.  Some of its warnings make no sense in the context of RNASeq.  If I was doing RNASeq of a filtered set of pancreatic alpha cells,  I would expect a hell of a lot of reads for glucagon, and a lot of them would be natural duplicates, and that would be biologically normal, and not a problem.&lt;/p&gt;&#xA;&lt;p&gt;You could take the same batch of reads, and align them to one genome, and align them to a mix of two closely related genomes.  The second will have far more non-uniquely mapped reads, but the FASTQC stats will not have changed.&lt;/p&gt;&#xA;&lt;p&gt;That said, if you doing human or mouse RNASeq, 1% unique alignment means something is wrong.  Find out where all those reads are aligning, you might have rRNA contamination.  If you are doing an organism with a high level of duplication in its genome that might be normal.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-07-14T16:37:00.587" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13891" PostTypeId="2" ParentId="13889" CreationDate="2020-07-14T17:36:11.320" Score="1" Body="&lt;p&gt;Just type&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda list gzip&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;OR&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda list | grep gzip&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;OR if its installed in a given environment&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda activate myenv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or else just &lt;code&gt;activate myenv&lt;/code&gt; system depending followed by either of the first two commands.&lt;/p&gt;&#xA;&lt;p&gt;It is better to use &lt;code&gt;conda-navigate&lt;/code&gt; a very nice and under used GUI, and would work in this particular instance. The details are found in an earlier post I made on esentially the same question &lt;a href=&quot;https://stackoverflow.com/questions/57701571/what-is-the-right-way-to-update-anaconda-and-conda-base-environments/57936966#57936966&quot;&gt;here&lt;/a&gt; together with a pretty picture.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;ls&lt;/code&gt; will only work if you type &lt;code&gt;ls ~/Anaconda3/bin | grep gzip&lt;/code&gt; , but depends on the installation, e.g. you might have installed miniconda.&lt;/p&gt;&#xA;&lt;p&gt;Having said all that &lt;code&gt;conda&lt;/code&gt; is very cool, but whatever you do NEVER use &lt;code&gt;pip&lt;/code&gt; or &lt;code&gt;pip3&lt;/code&gt; from this moment onwards EVER, it messes up your conda and you'll need to &lt;code&gt;env&lt;/code&gt; your way out of it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-14T17:41:29.590" LastActivityDate="2020-07-14T17:41:29.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13892" PostTypeId="1" CreationDate="2020-07-14T19:52:39.230" Score="0" ViewCount="176" Body="&lt;p&gt;I am working with virus nanopore sequencing data. I have to remove host genome from the reads. I want to run the QC on the fastq file after host genome subtraction. I have used nanoplot. It is slow. I dont have summary.txt. Is there any nanopore read specific qc package which can directly take nanopore fastq files.&lt;/p&gt;&#xA;" OwnerUserId="9001" LastActivityDate="2020-07-14T19:52:39.230" Title="nanopore QC measures on fastq file" Tags="|fastq|nanopore|qc|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13893" PostTypeId="1" CreationDate="2020-07-14T22:27:00.573" Score="2" ViewCount="394" Body="&lt;p&gt;I found &lt;a href=&quot;https://pyopenms.readthedocs.io/en/latest/feature_detection.html?highlight=featurefinder&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; documentation on pyOpenMS mass spec library. I want to try different peak picking algorithms. Unfortunately, the docs are not very detailed and don't show how to use the other peak picking algorithms and the codebase itself is also quite cryptic for me.&lt;/p&gt;&#xA;&lt;p&gt;Any idea how to use, for example, the FeatureFindingMetabo algorithm insead of the centroided one?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from urllib.request import urlretrieve&#xA;# from urllib import urlretrieve  # use this code for Python 2.x&#xA;gh = &amp;quot;https://raw.githubusercontent.com/OpenMS/OpenMS/develop&amp;quot;&#xA;urlretrieve (gh +&amp;quot;/src/tests/topp/FeatureFinderCentroided_1_input.mzML&amp;quot;, &amp;quot;feature_test.mzML&amp;quot;)&#xA;&#xA;from pyopenms import *&#xA;&#xA;# Prepare data loading (save memory by only&#xA;# loading MS1 spectra into memory)&#xA;options = PeakFileOptions()&#xA;options.setMSLevels([1])&#xA;fh = MzMLFile()&#xA;fh.setOptions(options)&#xA;&#xA;# Load data&#xA;input_map = MSExperiment()&#xA;fh.load(&amp;quot;feature_test.mzML&amp;quot;, input_map)&#xA;input_map.updateRanges()&#xA;&#xA;ff = FeatureFinder()&#xA;ff.setLogType(LogType.CMD)&#xA;&#xA;# Run the feature finder&#xA;name = &amp;quot;centroided&amp;quot;&#xA;features = FeatureMap()&#xA;seeds = FeatureMap()&#xA;params = FeatureFinder().getParameters(name)&#xA;ff.run(name, input_map, features, params, seeds)&#xA;&#xA;features.setUniqueIds()&#xA;fh = FeatureXMLFile()&#xA;fh.store(&amp;quot;output.featureXML&amp;quot;, features)&#xA;print(&amp;quot;Found&amp;quot;, features.size(), &amp;quot;features&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="1157" LastEditDate="2021-02-01T12:12:12.390" LastActivityDate="2021-02-01T12:12:12.390" Title="How to use pyopenms.FeatureFindingMetabo() for peak picking in pyOpenMS?" Tags="|python|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13895" PostTypeId="1" AcceptedAnswerId="13908" CreationDate="2020-07-15T13:45:45.703" Score="0" ViewCount="62" Body="&lt;p&gt;I am using OrthoFinder to find orthogroups between species. I am using the latest version unpacked through tar as per their instructions.&lt;/p&gt;&#xA;&lt;p&gt;Orthofinder creates an output folder according to the current date, ie, Results_Dec25. However,&#xA;I want to integrate orthofinder in snakemake, so having a stable output directory is necessary.&#xA;The -o option allows specification of an output prefix, but the dated folder is then created within.&lt;/p&gt;&#xA;&lt;p&gt;Is it possible to force a given name to the output folder?&#xA;Or is there a way in snakemake to allow ambiguous output directory names?&lt;/p&gt;&#xA;&lt;p&gt;I know this would be possible via editing the source code of OrthoFinder, but this workflow needs to be reproducible without faff.&lt;/p&gt;&#xA;&lt;p&gt;Many thanks&lt;/p&gt;&#xA;&lt;p&gt;This question has already been asked here but remains unanswered.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/13527/is-there-an-option-in-orthofinder-to-choose-the-output-directory-name&quot;&gt;Is there an option in Orthofinder to choose the output directory name?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9203" LastActivityDate="2020-07-17T03:53:28.377" Title="Removing data from OrthoFinder output prefix" Tags="|snakemake|orthofinder|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13896" PostTypeId="2" ParentId="5235" CreationDate="2020-07-15T14:52:07.067" Score="1" Body="&lt;p&gt;If your string is 'GG&lt;em&gt;TAG&lt;/em&gt;CCAATT&lt;em&gt;ATG&lt;/em&gt;AACGAA&lt;em&gt;TAG&lt;/em&gt;GAC' then I would assume that you could separate this into coding and non-coding regions such as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GG*TAG*CCAATT*    (non-coding)&#xA;ATG*AACGAA*TAG    (coding region)&#xA;*GAC              (non-coding)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It starts with a start codon ATG and ends with an end codon TAG.&lt;/p&gt;&#xA;&lt;p&gt;Your first event position 12 does not make sense to me. What does * mean? It seems of that your sequence starts with GG which is not a triplet? Perhaps did you mean GGG?&lt;/p&gt;&#xA;&lt;p&gt;I'm the author of node based DNA visualisation &lt;code&gt;AminoSee&lt;/code&gt;. It has a bug in that it only renders half right: because DNA can be read backwards, and my program only does it &lt;em&gt;forwards&lt;/em&gt;. I actually came to BioInformatics today to try and get my head around reading backwards and how that might work.&lt;/p&gt;&#xA;&lt;p&gt;AminoSee only transcodes 3 base pairs at a time, and discards triplets that aren't, well, 3 base pairs longs. I'm self taught and so I guess unaware if it's possible to have any sequence that is not a multiple of 3. Perhaps it's possible. I assumed not. My program is more of an art piece that a science tool, but I plan to use it one day to perform a &amp;quot;visual diff&amp;quot; between two species (Humans and Chimp). See &lt;a href=&quot;http://aminosee.funk.nz&quot; rel=&quot;nofollow noreferrer&quot;&gt;AminoSee  DNA Visualisation&lt;/a&gt; over at aminosee.funk.nz&lt;/p&gt;&#xA;" OwnerUserId="9199" LastEditorUserId="9199" LastEditDate="2020-07-15T14:57:36.833" LastActivityDate="2020-07-15T14:57:36.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13897" PostTypeId="1" AcceptedAnswerId="13898" CreationDate="2020-07-15T16:41:39.043" Score="1" ViewCount="1729" Body="&lt;p&gt;I'm new to Python/Biopython programming. I'd like to install Biopython into my PyCharm IDE but I've tried all instructions on this, but it's not confirming it. I have the available biopython file already extracted. There's a SyntaxError for 'install' in:&#xA;pip install biopython&#xA;What's the proper way of installing and importing biopython? And how does one confirm the installation.&lt;/p&gt;&#xA;" OwnerUserId="9155" LastActivityDate="2020-07-15T19:14:59.837" Title="How to install/import Biopython into Python 3.8/ PyCharm IDE" Tags="|python|software-installation|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13898" PostTypeId="2" ParentId="13897" CreationDate="2020-07-15T19:09:46.577" Score="0" Body="&lt;p&gt;Install Anaconda3 the instructions are &lt;a href=&quot;https://www.google.co.uk/m?q=conda%20creating%20an%20environment&amp;amp;client=ms-opera-mobile&amp;amp;channel=new&amp;amp;espv=1#&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Then making sure you are connected to the internet type&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c anaconda biopython&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Once Anaconda is installed don't attempt to use &lt;code&gt;pip&lt;/code&gt;, if you do then you need to set an 'environment' in Anaconda and work from that environment.&lt;/p&gt;&#xA;&lt;p&gt;I suspect you used &lt;code&gt;pip&lt;/code&gt; or &lt;code&gt;pip3&lt;/code&gt; via&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pip install biopython&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-15T19:14:59.837" LastActivityDate="2020-07-15T19:14:59.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13901" PostTypeId="2" ParentId="11572" CreationDate="2020-07-16T03:40:39.120" Score="2" Body="&lt;p&gt;I have been running MaxQuant quite frequently on Linux. I would use conda to create a dedicated environment and install &lt;code&gt;mono&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# create the environment&#xA;conda create -n maxquant -c conda-forge mono&#xA;# activate the environment&#xA;conda activate maxquant&#xA;# run any maxquant version&#xA;mono /path/to/maxuant/MaxQuantCmd.exe mqpar.xml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This installs a conda environment with &lt;code&gt;mono&lt;/code&gt; which is a &lt;code&gt;.NET&lt;/code&gt; framework for Linux. I had problems in the past running certain MaxQuant versions that way.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;mqpar.xml&lt;/code&gt; that is required can be created with the MaxQuant GUI. And then the path for input files needs to be contained correctly. The path to the &lt;em&gt;fasta&lt;/em&gt; file and the &lt;em&gt;raw&lt;/em&gt; file. I would create a template file and then parse these filenames into the specific locations.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# mqpar.xml&#xA;...&#xA;&amp;lt;fastaFiles&amp;gt;&#xA;  &amp;lt;FastaFileInfo&amp;gt;&#xA;     &amp;lt;fastaFilePath&amp;gt;__FASTA__&amp;lt;/fastaFilePath&amp;gt;&#xA;     &amp;lt;identifierParseRule&amp;gt;&amp;gt;([^\s]*)&amp;lt;/identifierParseRule&amp;gt;&#xA;     &amp;lt;descriptionParseRule&amp;gt;&amp;gt;(.*)&amp;lt;/descriptionParseRule&amp;gt;&#xA;     &amp;lt;taxonomyParseRule&amp;gt;&amp;lt;/taxonomyParseRule&amp;gt;&#xA;     &amp;lt;variationParseRule&amp;gt;&amp;lt;/variationParseRule&amp;gt;&#xA;     &amp;lt;modificationParseRule&amp;gt;&amp;lt;/modificationParseRule&amp;gt;&#xA;     &amp;lt;taxonomyId&amp;gt;&amp;lt;/taxonomyId&amp;gt;&#xA;  &amp;lt;/FastaFileInfo&amp;gt;&#xA;&amp;lt;/fastaFiles&amp;gt;&#xA;...&#xA;&amp;lt;filePaths&amp;gt;&#xA;  &amp;lt;string&amp;gt;__RAW__&amp;lt;/string&amp;gt;&#xA;&amp;lt;/filePaths&amp;gt;&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The combination of &lt;code&gt;maxquant=1.6.10.43&lt;/code&gt; and &lt;code&gt;mono=5.14&lt;/code&gt; worked and a few other versions. Some versions between 1.6 and 2.x did not work at all.&lt;/p&gt;&#xA;&lt;p&gt;Specific versions of MaxQuant are also available directly via &lt;code&gt;conda&lt;/code&gt; and there is no need to download the zip file. Instead, you can simply install it with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c conda-forge -c bioconda maxquant mono=5.14&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Recently MaxQuant 2 has been added, and not all previous versions are available and run maxquant with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;maxquant mqpar.xml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2021-09-02T22:18:19.690" LastActivityDate="2021-09-02T22:18:19.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13902" PostTypeId="1" AcceptedAnswerId="13969" CreationDate="2020-07-16T10:16:38.493" Score="1" ViewCount="156" Body="&lt;p&gt;I have a &lt;em&gt;{maftools}&lt;/em&gt; plot:&lt;/p&gt;&#xA;&lt;p&gt;whatever I try to save that completely I see the down part of plot is trimmed like&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AGCY8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AGCY8.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have tried both save as pdf and save as image options which did not differ&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion please?&lt;/p&gt;&#xA;&lt;p&gt;Plot comes by this code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tcgaCompare(maf = laml,cohortName = &amp;quot;Cohort&amp;quot;,rm_zero = FALSE,decreasing = TRUE,logscale = TRUE,primarySite = TRUE,col = c(&amp;quot;grey30&amp;quot;, &amp;quot;red&amp;quot;),medianCol = &amp;quot;black&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-07-16T12:23:59.313" LastActivityDate="2020-07-23T00:44:58.913" Title="maftools tcgaCompare plot: x-axis labels getting clipped" Tags="|r|maftools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13904" PostTypeId="1" CreationDate="2020-07-16T20:46:00.070" Score="1" ViewCount="789" Body="&lt;p&gt;I wish to convert a kraken2 or metaphlan2 report into the R package to analyze using Phyloseq.&lt;/p&gt;&#xA;&lt;p&gt;I have performed obtained an importation and Phyloseq analysis, but the result was not correct.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(&amp;quot;phyloseq&amp;quot;)&#xA;library(&amp;quot;biomformat&amp;quot;) &#xA;otu_Rat11 &amp;lt;- read_biom(&amp;quot;Rat11.biom&amp;quot;) &#xA;&#xA;otumat_Rat11 = as(biom_data(otu_Rat11), &amp;quot;matrix&amp;quot;) &#xA;OTU_Rat11 &amp;lt;- otu_table(otumat_Rat11, taxa_are_rows=TRUE) &#xA;str(OTU_Rat11)&#xA;#Formal class 'otu_table' [package &amp;quot;phyloseq&amp;quot;] with 2 &#xA;#slots ..@ .Data : num [1:1018, 1] 942 433 3007 116765 15103 ... .. ..- &#xA;#attr(*, &amp;quot;dimnames&amp;quot;)=List of 2 .. .. ..:ℎ[1:1018]&amp;quot;186826&amp;quot;&amp;quot;33958&amp;quot;&amp;quot;1578&amp;quot;&#xA;#&amp;quot;33959&amp;quot;.........:chr[1:1018] &amp;quot;186826&amp;quot; &amp;quot;33958&amp;quot; &amp;quot;1578&amp;quot; &amp;quot;33959&amp;quot;......... :&#xA;# chr &amp;quot;sa1&amp;quot; ..@ taxa_are_rows: logi TRUE&#xA;&#xA;taxmat_Rat11 = as.matrix(observation_metadata(otu_Rat11), rownames.force=TRUE)&#xA;TAX_Rat11 = tax_table(taxmat_Rat11)&#xA;&#xA;physeq_Rat11 = phyloseq(OTU_Rat11, TAX_Rat11)&#xA;str(physeq_Rat11) &#xA;#Formal class 'phyloseq' [package &amp;quot;phyloseq&amp;quot;] with 5 slots ..@ otu_table:Formal &#xA;#class 'otu_table' [package &amp;quot;phyloseq&amp;quot;] with 2 slots .. .. ..@ .Data : num &#xA;#[1:1018, 1] 942 433 3007 116765 15103 ... .. .. .. ..- attr(*, &amp;quot;dimnames&amp;quot;)=List&#xA;# of 2 .. .. .. .. ..:ℎ[1:1018]&amp;quot;186826&amp;quot;&amp;quot;33958&amp;quot;&amp;quot;1578&amp;quot;&amp;quot;33959&amp;quot;.............: chr &#xA;#[1:7] &amp;quot;taxonomy1&amp;quot; &amp;quot;taxonomy2&amp;quot; &amp;quot;taxonomy3&amp;quot; &amp;quot;taxonomy4&amp;quot; ... ..@ sam_data : NULL ..&#xA;#@ phy_tree : NULL ..@ refseq : NULL –&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now I try to obtain @ sam_data&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sampledata = sample_data(otu_Rat11) &#xA;#Error in access(object, &amp;quot;sam_data&amp;quot;, errorIfNULL) : sam_data slot is empty.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What do with sample data and how to merge several phyloseq-format datasets from different metagenomes?&lt;/p&gt;&#xA;" OwnerUserId="9209" LastEditorUserId="9172" LastEditDate="2020-07-24T11:36:21.420" LastActivityDate="2021-12-21T15:59:01.160" Title="Kraken2 or metaphlan2 report to phyloseq" Tags="|r|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13906" PostTypeId="2" ParentId="13893" CreationDate="2020-07-16T22:12:15.470" Score="1" Body="&lt;p&gt;This is a draft answer, if the OP still need needs I will extend it later.&lt;/p&gt;&#xA;&lt;p&gt;Me too I found the documentation and examples of &lt;code&gt;pyopenms&lt;/code&gt; very scarce. At the same time  OpenMS is huge, sometimes you find key information not in various other sources instead of the official documentation. This is especially true if it comes to metabolomics.&lt;/p&gt;&#xA;&lt;p&gt;We built a Python OpenMS lipidomics LC MS/MS preprocessing pipeline, contained in this file:&#xA;&lt;a href=&quot;https://github.com/saezlab/lipyd/blob/master/src/lipyd/msproc.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/saezlab/lipyd/blob/master/src/lipyd/msproc.py&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The top class is the one called &lt;code&gt;MSPreprocess&lt;/code&gt;, it implements the workflow, the other classes implement its steps.&lt;/p&gt;&#xA;&lt;p&gt;Briefly, you start from profile mode mzML files, then&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Peak picking with PeakPickerHiRes results centroided data&lt;/li&gt;&#xA;&lt;li&gt;FeatureFindingMetabo assembles the traces in a metabolomics&#xA;compatible way&lt;/li&gt;&#xA;&lt;li&gt;A MapAlignment algorithm alignes the traces across&#xA;scans&lt;/li&gt;&#xA;&lt;li&gt;A FeatureGrouping algorithm alignes the features across multiple experiments&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Our software mentioned above (&lt;code&gt;lipyd&lt;/code&gt;) outputs data frames from the m/z, RT and intensity.&lt;/p&gt;&#xA;&lt;p&gt;About &lt;code&gt;OpenMS&lt;/code&gt; I can recommend to read the &lt;code&gt;C++&lt;/code&gt; code and docs. The Python classes and methods are automatically built minimal wrappers, everything works the same way as in &lt;code&gt;C++.&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="8977" LastActivityDate="2020-07-16T22:12:15.470" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13908" PostTypeId="2" ParentId="13895" CreationDate="2020-07-17T00:08:44.940" Score="2" Body="&lt;p&gt;The latest version solves the issue.&lt;/p&gt;&#xA;&lt;p&gt;Source : &lt;a href=&quot;https://github.com/davidemms/OrthoFinder/issues/403&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/davidemms/OrthoFinder/issues/403&lt;/a&gt;&lt;br /&gt;&#xA;Latest Version: &lt;a href=&quot;https://github.com/davidemms/OrthoFinder/releases/tag/2.4.0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/davidemms/OrthoFinder/releases/tag/2.4.0&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9212" LastEditorUserId="9212" LastEditDate="2020-07-17T03:53:28.377" LastActivityDate="2020-07-17T03:53:28.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13911" PostTypeId="1" CreationDate="2020-07-17T10:15:25.197" Score="2" ViewCount="146" Body="&lt;p&gt;Task: Normalize a single-cell RNA-Seq dataset to account for sequencing depth and gene-length.&lt;/p&gt;&#xA;&lt;p&gt;For UMI-count based protocols (like 10x) that don't suffer from gene-length biases, there are various methods developed to account for sequencing depth e.g. &lt;code&gt;sctransform&lt;/code&gt; from &lt;code&gt;Seurat&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For read-count based protocols (like Smart-Seq2), &lt;code&gt;SCnorm&lt;/code&gt; can for example be used to normalize for sequencing depth. However, read-count based protocols can suffer from gene-length biases that are not accounted for in &lt;code&gt;SCnorm&lt;/code&gt;. Furthermore, the method of &lt;code&gt;SCnorm&lt;/code&gt; expects raw counts, so I don't know if I can length-normalize the counts prior to using the method. Technically, &lt;code&gt;TPMs&lt;/code&gt; do account for both, but it is a global scale factor method, which has other drawbacks.&lt;/p&gt;&#xA;&lt;p&gt;Question: How can I normalize my read-count based datasets to account for gene-length biases and sequencing depth without using a global scale factor method?&lt;/p&gt;&#xA;" OwnerUserId="2047" LastActivityDate="2020-07-17T10:15:25.197" Title="scRNA-Seq: Account for sequencing depth and gene length?" Tags="|rna-seq|scrnaseq|single-cell|normalization|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13914" PostTypeId="1" CreationDate="2020-07-17T17:53:36.257" Score="1" ViewCount="4478" Body="&lt;p&gt;I want to download the following fastq files at the same time in Salmon:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; - SRR10611214&#xA; - SRR10611215&#xA; - SRR10611215&#xA; - SRR10611216&#xA; - SRR10611217&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there a way to do this using a bash for loop or fastq-dump? or prefetch&lt;/p&gt;&#xA;" OwnerUserId="9217" LastEditorUserId="240" LastEditDate="2020-07-22T06:29:14.437" LastActivityDate="2020-07-23T13:35:08.460" Title="Download multiple fastq files using fastq-dump" Tags="|rna-seq|fastq|data-download|salmon|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13916" PostTypeId="1" AcceptedAnswerId="15479" CreationDate="2020-07-17T22:23:23.243" Score="0" ViewCount="70" Body="&lt;p&gt;First of all I apologize without the question is very basic, I am taking my first steps in bioinformatics.&lt;/p&gt;&#xA;&lt;h2&gt;Data information&lt;/h2&gt;&#xA;&lt;p&gt;We are evaluating the correlation (using the Pearson, Kendall or Spearman method) between gene expression and miRNA expression using the &lt;a href=&quot;https://www.rdocumentation.org/packages/WGCNA/versions/1.69/topics/corAndPvalue&quot; rel=&quot;nofollow noreferrer&quot;&gt;corAndPvalue&lt;/a&gt; function of WCGNA.&lt;/p&gt;&#xA;&lt;p&gt;The resulting structure would be a DataFrame containing all combinations between each gene with each miRNAs, containing the following columns:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Gene     miRNA      Correlation P-value&#xA;Gen_1    miRNA_1    0,959       0.00311&#xA;Gen_1    miRNA_2    -0,039      0.1041&#xA;Gen_1    miRNA_3    -0,344      0.0021&#xA;Gen_2    miRNA_1    0,1333      0.00451&#xA;Gen_2    miRNA_2    0,877       0.07311&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Question&lt;/h2&gt;&#xA;&lt;p&gt;Considering the huge number of correlation tests we are going to evaluate, we need to adjust the p-values to avoid correlations due to chance. Bonferroni does not seem to be the best solution, so we would use Benjamini-Hochberg method (BH). The question is:&lt;/p&gt;&#xA;&lt;p&gt;The BH correction for the &lt;code&gt;Gen_1&lt;/code&gt; x &lt;code&gt;miRNA_1&lt;/code&gt; combination, should consider the p-values of all combinations that include Gen_1 (Option 1), or should consider all the p-values of all the genes x miRNA combinations (Option 2)?&lt;/p&gt;&#xA;&lt;p&gt;For example, let's assume an expression dataset of 20,000 genes and another of 15,000 miRNAs&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Option 1:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;To adjust &lt;code&gt;Gen_1&lt;/code&gt; x &lt;code&gt;miRNA_1&lt;/code&gt; we would use 15,000 p-values (&lt;code&gt;Gen_1&lt;/code&gt; x &lt;code&gt;miRNA_1&lt;/code&gt;, &lt;code&gt;Gen_1&lt;/code&gt; x &lt;code&gt;miRNA_2&lt;/code&gt;, ..., &lt;code&gt;Gen_1&lt;/code&gt; x &lt;code&gt;miRNA_15000&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Option 2:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;To adjust &lt;code&gt;Gen_1&lt;/code&gt; x &lt;code&gt;miRNA_1&lt;/code&gt; we would use 300,000,000 p-values (&lt;code&gt;Gen_1&lt;/code&gt; x &lt;code&gt;miRNA_1&lt;/code&gt;, &lt;code&gt;Gen_1&lt;/code&gt; x &lt;code&gt;miRNA_2&lt;/code&gt;, ..., &lt;code&gt;Gen_1&lt;/code&gt; x &lt;code&gt;miRNA_15000&lt;/code&gt;, &lt;code&gt;Gen_2&lt;/code&gt; x &lt;code&gt;miRNA_1&lt;/code&gt;, &lt;code&gt;Gen_2&lt;/code&gt; x &lt;code&gt;miRNA_2&lt;/code&gt;, ..., &lt;code&gt;Gen_2&lt;/code&gt; x &lt;code&gt;miRNA_15000&lt;/code&gt; and so on).&lt;/p&gt;&#xA;&lt;h2&gt;Suplementary question&lt;/h2&gt;&#xA;&lt;p&gt;Documentation of the method &lt;a href=&quot;https://www.statsmodels.org/stable/generated/statsmodels.stats.multitest.fdrcorrection.html?highlight=fdrcorrection#statsmodels.stats.multitest.fdrcorrection&quot; rel=&quot;nofollow noreferrer&quot;&gt;fdrcorrection&lt;/a&gt; from Python &lt;a href=&quot;https://www.statsmodels.org/stable/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Statsmodels library&lt;/a&gt; suggests that for negative correlations (that could be frequent in a mRNA x miRNA correlation analysis) Benjamini-Yekutieli would work better; is that right? Or Benjamini-Hochberg method would be appropiated for this case?&lt;/p&gt;&#xA;&lt;p&gt;Any kind of help would be much appreciated, thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="9220" LastActivityDate="2021-03-04T13:35:25.527" Title="P-value correction when evaluating correlation between gene and miRNA expression" Tags="|statistics|gene-expression|correlation|p-values|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13917" PostTypeId="1" CreationDate="2020-07-17T22:31:57.340" Score="0" ViewCount="257" Body="&lt;p&gt;The file system I'm running on doesn't allow the creation of hard-links.  Cromwell is trying to create hard-links to directories, and when that fails, it's copying them.  Which is leading to disk quota errors.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way that I can simply tell cromwell &amp;quot;try soft links first&amp;quot;?  Without creating a whole config file?&lt;/p&gt;&#xA;" OwnerUserId="9222" LastActivityDate="2020-07-17T22:31:57.340" Title="How do I get cromwell to create soft-links?" Tags="|wdl|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13919" PostTypeId="2" ParentId="13914" CreationDate="2020-07-18T02:18:07.913" Score="4" Body="&lt;p&gt;A sample code is given in the salmon documentation as follows. &lt;a href=&quot;https://combine-lab.github.io/salmon/getting_started/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Source&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;mkdir data&#xA;cd data&#xA;for i in `seq 25 40`; &#xA;do &#xA;  mkdir DRR0161${i}; &#xA;  cd DRR0161${i}; &#xA;  wget ftp://ftp.sra.ebi.ac.uk/vol1/fastq/DRR016/DRR0161${i}/DRR0161${i}_1.fastq.gz; &#xA;  wget ftp://ftp.sra.ebi.ac.uk/vol1/fastq/DRR016/DRR0161${i}/DRR0161${i}_2.fastq.gz; &#xA;  cd ..; &#xA;done&#xA;cd .. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This could be modified as follows.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;mkdir data&#xA;cd data&#xA;for i in `seq 14 17`; &#xA;do &#xA;  wget ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR106/0${i}/SRR106112${i}/SRR106112${i}_1.fastq.gz; &#xA;  wget ftp://ftp.sra.ebi.ac.uk/vol1/fastq/SRR106/0${i}/SRR106112${i}/SRR106112${i}_2.fastq.gz; &#xA;done&#xA;cd .. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can save the code as a shell script and run it from the linux terminal. For example &lt;code&gt;bash download_fastq.sh&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="9212" LastActivityDate="2020-07-18T02:18:07.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13922" PostTypeId="1" CreationDate="2020-07-18T17:41:14.303" Score="1" ViewCount="40" Body="&lt;p&gt;I'm writing a &lt;a href=&quot;https://github.com/revl/vcf_scanner&quot; rel=&quot;nofollow noreferrer&quot;&gt;VCF parser&lt;/a&gt;, so I have to consider and handle all corner cases regardless of how contrived they may seem. The specification is a bit unclear about the MISSING value ('.') in the ALTS column:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Options are base Strings made up of the bases A,C,G,T,N,*, (case insensitive) or a MISSING value ‘.’  (no variant) or an angle-bracketed ID String (“”) or a breakend replacement string.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I've seen examples with a single dot in the ALTS column:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;4       31789170        PTV021  G       .       77      PASS    .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The question is whether the following data lines are also valid:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1       12345   ID1     A       .,T,.   22.88   PASS    .&#xA;1       12346   ID2     G       C,.     22.88   PASS    .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In other words, does MISSING indicate that the entire ALTS field is missing, or does it mean that there's a missing allele?&lt;/p&gt;&#xA;&lt;p&gt;By extension, how do I represent the case when there's a single dot in the ALTS field (as in the first example)? Is it an empty list (because the whole field is MISSING) or is it a list containing a MISSING value? In other words, is it &lt;code&gt;[]&lt;/code&gt; or &lt;code&gt;[&amp;quot;.&amp;quot;]&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;At the moment, the parser does not validate the contents of the ALT field and does not even attempt to resolve any ALT IDs from the header. Interpretation of the ALT values is left entirely to the client code.&lt;/p&gt;&#xA;" OwnerUserId="9226" LastEditorUserId="3848" LastEditDate="2022-07-22T13:33:13.483" LastActivityDate="2022-07-22T13:33:13.483" Title="VCF spec: is it possible to have other alleles in addition to the MISSING value ('.') in ALTS?" Tags="|vcf|parsing|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13923" PostTypeId="1" AcceptedAnswerId="13929" CreationDate="2020-07-18T20:59:47.647" Score="0" ViewCount="45" Body="&lt;p&gt;I am using the following command to retrieve GEO Datasets entries for daf-2 c. elegans that have as their experiment type High Throughpout sequencing:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db gds -query &amp;quot;daf-2&amp;quot; | efilter -query &amp;quot;expression profiling by high throughput sequencing [DataSet Type] AND Caenorhabditis elegans [organism]&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, among the results that I got, I noticed that I got this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;23. Impaired Insulin-/IGF1-Signaling Extends Life Span by Promoting Mitochondrial L-Proline Catabolism to Induce a Transient ROS-Signal&#xA;(Submitter supplied) Transcriptome profiling of three models with impaired insulin/IGF1 signaling. 1. Deep sequencing of endogenous mRNA from Caenorhabditis elegans N2 var. Bristol (wildtype) and daf-2(e1370) mutant; 2. Deep sequencing  of endogenous mRNA from murine embryonic fibroblasts (MEF)  wildtype and irs1-/- knockout; 3. Deep sequencing of endogenous mRNA from murine embryoinic fibroblast (MEF) insr+/- -lox and insr+/- knockout  Jena Centre for Systems Biology of Ageing - JenAge (www.jenage.de)&#xA;Organism:   Mus musculus; Caenorhabditis elegans&#xA;Type:       Expression profiling by high throughput sequencing&#xA;Platforms: GPL11002 GPL13776 14 Samples&#xA;FTP download: GEO (CSV) ftp://ftp.ncbi.nlm.nih.gov/geo/series/GSE36nnn/GSE36041/&#xA;SRA Run Selector: https://www.ncbi.nlm.nih.gov/Traces/study/?acc=PRJNA151765&#xA;Series      Accession: GSE36041 ID: 200036041&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see, in the &lt;code&gt;Organism&lt;/code&gt; field, there are both &lt;code&gt;Mus musculus&lt;/code&gt; and &lt;code&gt;Caenorhabditis elegans&lt;/code&gt;. How can I restrict my search so that I only get entries that exclusively work on &lt;code&gt;Caenorhabditis elegans&lt;/code&gt;? One naive way to do this would be to include &lt;code&gt;NOT Mus Musculus&lt;/code&gt; in the query, but of course that would mean that I could get something else too.&lt;/p&gt;&#xA;&lt;p&gt;Another way I can think of solving this is just to write a script that would do the extra filtering using regexes, but I was wondering if there's a simpler solution using the e-utilities functionalities&lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2020-07-19T23:02:45.117" Title="How can I restrict the search space when querying GEO?" Tags="|geoquery|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13925" PostTypeId="2" ParentId="10980" CreationDate="2020-07-19T14:00:58.263" Score="0" Body="&lt;p&gt;You are probably using the wrong java version it should be 8&lt;/p&gt;&#xA;&lt;p&gt;try &amp;quot;java -version&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="9229" LastActivityDate="2020-07-19T14:00:58.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13926" PostTypeId="1" AcceptedAnswerId="13934" CreationDate="2020-07-19T18:37:55.153" Score="0" ViewCount="856" Body="&lt;p&gt;I'd like to add metadata to 6 individual Seurat objects so that after I merge the objects into one, I can later label or split by using these identifiers. For example, I'd like to append an age group and then interval across these 6 objects. For adding an interval, I've tried using the below:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;AddMetaData(control, metadata = 1hr, col.name = Interval)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However, this error will occur:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in .AddMetaData(object = object, metadata = metadata, col.name = col.name) : &#xA;  object '1hr' not found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Appreciate any help, thank you!&lt;/p&gt;&#xA;" OwnerUserId="9188" LastEditorUserId="3541" LastEditDate="2020-07-20T09:44:51.367" LastActivityDate="2020-07-20T09:46:39.743" Title="Adding treatment groups via metadata to Seurat object?" Tags="|scrnaseq|seurat|single-cell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13927" PostTypeId="1" AcceptedAnswerId="13928" CreationDate="2020-07-19T20:15:48.250" Score="2" ViewCount="115" Body="&lt;p&gt;Is it possible that the fusion with cell propriety of Human Herpes virus, I see &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/X14112.1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;X14112.1&lt;/a&gt; in GenBank is due to protein UL36 (tegument of virus) starting in linear position 2685 ???&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;H R R A R S L A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;there are similarities with SarsCov2 virus protein S in name MN908947.3 in GenBank position 681.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;P R R A R S V A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It is strange but the now (21-12-2020) the new Sars2 Great Britain strain is more near to that UL36 piece, it seems already:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;H R R A R S V A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Herpes virus protein UL36 and SarsCov2 virus protein S[1200 position], have  too&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;L Q E L G K&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;i find in internet they call it &amp;quot;Glutaminase kidney isoform, mitochondrial&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="7054" LastEditorUserId="7054" LastEditDate="2020-12-21T23:09:17.190" LastActivityDate="2020-12-21T23:09:17.190" Title="Human Herpes Virus, fusion motif in protein vs. SARS-CoV-2?" Tags="|proteins|covid-19|sars-cov-2|motifs|coronavirus|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13928" PostTypeId="2" ParentId="13927" CreationDate="2020-07-19T21:30:57.140" Score="2" Body="&lt;p&gt;I can answer that ... I'm afraid no. What you are asking is whether there is any similarity in their cellular receptor binding or &amp;quot;fusion loops&amp;quot; because the motifs look a bit similar.&lt;/p&gt;&#xA;&lt;p&gt;These are very different viruses and you need to appreciate just how different they are,&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Human herpes&lt;/em&gt; is a double stranded DNA virus resulting in a chronic infection. It has a vast genome of &amp;gt;150kb, which includes cytokine mimics and proteins that we have no idea what they do. The protein you are referring to is likely a surface antigen in a virus covered in different surface antigens. However, this protein is involved in blocking the beta inferon response is considered a protease - not fusion loop and appears to have some sort of complex replication function involving the capsid. Finally, there is also the question of which human herpes 1 to 8. Here you are referring to HHV1 but for example HHV3 is chickenpox (okay its an acute not chronic infection) and HHV5 is cytomegliovirus (congenitcal infection).&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;COVID-19&lt;/em&gt; is a simple virus of a 30Kb genome of a single stranded positive sense RNA, causing an acute infection. I agree the Spike protein is much bigger than most RNA viruses, but the receptor-binding is tightly defined with ACE-2. It forms a classic RNA icosahedral virus, singly covered in Spike protein. I have no doubt it will go through conformational change like other RNA viruses in receptor binding but the protein involved is singularly involved in cell entry - thats its job.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&#xA;There are large differences between HHV1 and COVID-19 for,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Viruses; DNA double stranded vs RNA single stranded&lt;/li&gt;&#xA;&lt;li&gt;Infections; chronic vs. acute&lt;/li&gt;&#xA;&lt;li&gt;Transmission; vertical/integration? direct contact vs. respiratory&lt;/li&gt;&#xA;&lt;li&gt;Ultimately very different proteins&lt;/li&gt;&#xA;&lt;li&gt;Finally very different functions&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;At best this would be heavily  scrutinised as a virological assumption.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Bioformatics&lt;/em&gt; If you really wanted to follow this up you could compare the structures around these motifs, because COVID-19 is close to SARS and assess this possibility ... but you would need to be careful about the conformational changes Spike goes through in receptor-binding.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Advice&lt;/strong&gt; before investing large amounts of time in a bioinformatics question, it is well-worth understanding the biological rationale for the question. In this instance that rationale appears to be lacking. If you found a comparison for example with gamma-coronaviruses, which was absent from many beta-coronaviruses that would be interesting. Here there is simply too much possibility of a chance similarity and given the size HHV1 that cannot be easily overlooked. However it is good that you are thinking and questioning, if you apply the same techniques, e.g. psi-blast, protein-blast into different questions ... that can work.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-19T21:44:37.750" LastActivityDate="2020-07-19T21:44:37.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13929" PostTypeId="2" ParentId="13923" CreationDate="2020-07-19T23:02:45.117" Score="0" Body="&lt;p&gt;You can use the following query to first search GEO to get a list of GEO Series of interest, then find all linked SRA runs and perform a second filter step to keep only the SRA runs that satisfy another set of criteria as shown below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch \&#xA;  -db gds \&#xA;  -query &amp;quot;daf-2[all fields] \&#xA;    AND expression profiling by high throughput sequencing [DataSet Type] \&#xA;    AND Caenorhabditis elegans [organism]&amp;quot; \&#xA;  | elink \&#xA;  -db gds -target sra \&#xA;  | esearch \&#xA;  -query &amp;quot;(#3) \&#xA;    AND Caenorhabditis elegans [organism]&amp;quot; \&#xA;  | efetch -format runinfo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here, I am combining two independent queries using the &lt;code&gt;(#3)&lt;/code&gt; term in the second query. More information about this is in the &amp;quot;Combining Independent Queries&amp;quot; subsection of &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/#_chapter6_Searching_and_Filtering_&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Searching and Filtering&amp;quot;&lt;/a&gt; section of the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez Direct documentation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2020-07-19T23:02:45.117" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13930" PostTypeId="2" ParentId="4657" CreationDate="2020-07-20T05:06:47.130" Score="2" Body="&lt;p&gt;You may combine all your transcriptomes into a single file and then apply a clustering method to group very similar transcripts into a single one. For that purpose, you may try &lt;a href=&quot;http://weizhongli-lab.org/cd-hit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CD-HIT-EST&lt;/a&gt; or &lt;a href=&quot;https://github.com/soedinglab/MMseqs2&quot; rel=&quot;nofollow noreferrer&quot;&gt;MMseqs2&lt;/a&gt;. For each identity threshold you are going to test, you may assess the final quality with BUSCO or by blasting against reference sequences.&lt;/p&gt;&#xA;" OwnerUserId="8749" LastActivityDate="2020-07-20T05:06:47.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13931" PostTypeId="2" ParentId="4657" CreationDate="2020-07-20T05:34:49.933" Score="2" Body="&lt;p&gt;A simple option would be to simply supply SPAdes with some contigs that you like with &lt;a href=&quot;http://cab.spbu.ru/files/release3.11.1/manual.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the &lt;code&gt;--trusted-contigs&lt;/code&gt; option&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There exist tools specifically addressing the problem of merging transcriptome assemblies. I am unsure whether this is notably different from genome assemblies sufficiently that they are better than the genome assembly mergers. Here is a partial list:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/cboursnell/transfuse&quot; rel=&quot;nofollow noreferrer&quot;&gt;transfuse&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-016-1406-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0091776&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7188282/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.sigenae.org/drap&quot; rel=&quot;nofollow noreferrer&quot;&gt;DRAP&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://academic.oup.com/gigascience/article/8/5/giz039/5488105&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; seems to have some more information on comparing tools, though it is not focused specifically on merging.&lt;/p&gt;&#xA;&lt;p&gt;See also &lt;a href=&quot;http://seqanswers.com/forums/showthread.php?t=38882&quot; rel=&quot;nofollow noreferrer&quot;&gt;the SeqAnswers&lt;/a&gt; on this topic, and &lt;a href=&quot;https://groups.google.com/forum/#!topic/trinityrnaseq-users/Rt9Wnrs3k0A&quot; rel=&quot;nofollow noreferrer&quot;&gt;this discussion too&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-07-20T05:34:49.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13932" PostTypeId="1" CreationDate="2020-07-20T08:01:51.917" Score="0" ViewCount="115" Body="&lt;h1&gt;identify where is the starting point for the synthesis of a protein in the DNA&lt;/h1&gt;&#xA;&lt;p&gt;For indentify a starting point in RNA or DNA my function search all ATG or M in the sequece and report the linear position of start, than the function search TAA and TAG for stop codon and report the end position.&lt;/p&gt;&#xA;&lt;p&gt;For the end codon what I have to search? only TAA and TAG or TGA and TGG too?&lt;/p&gt;&#xA;&lt;p&gt;How is possible some protein begin with &amp;quot;F&amp;quot; as in &amp;quot;FFRE...&amp;quot; and not &amp;quot;M...&amp;quot; ?&lt;/p&gt;&#xA;&lt;p&gt;Than it result more starting point solutions respect to protein found in GeoBank, what are the add restriction for limit that number?&lt;/p&gt;&#xA;&lt;p&gt;Are there easy to search sequences that make sure that is a start of the protein code? as seems to me &amp;quot;AGG AGG&amp;quot; but is not clear where they should be in the array and not seems general seen proteins.&lt;/p&gt;&#xA;&lt;p&gt;Can the synthesis of protein be affected by temperature? What other factors come into play?&lt;/p&gt;&#xA;&lt;p&gt;Passing my function to DNA of one virus, the first test, the function identifies&#xA;in the end of virus many little proteins of 2 3 4 5 10 protein letters. These little protein pieces are really sintetized? or there is a min lenght for sintetized proteins?&lt;/p&gt;&#xA;" OwnerUserId="7054" LastEditorUserId="3848" LastEditDate="2020-12-01T10:23:22.140" LastActivityDate="2020-12-01T10:23:22.140" Title="Identification of starting point for the synthesis of a protein" Tags="|proteins|covid-19|sars-cov-2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13934" PostTypeId="2" ParentId="13926" CreationDate="2020-07-20T09:46:39.743" Score="0" Body="&lt;p&gt;You should have fed &lt;code&gt;&amp;quot;1h&amp;quot;&lt;/code&gt; to your function call but you fed &lt;code&gt;1h&lt;/code&gt; instead. The latter is looking for an object with that name and as the error shows, there is no such object.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-07-20T09:46:39.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13935" PostTypeId="1" AcceptedAnswerId="13967" CreationDate="2020-07-20T10:04:40.897" Score="1" ViewCount="345" Body="&lt;p&gt;I am using Rcircos to make a chromosome ideogram plot for my gene list (n = 45). However, I am getting this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; Not all labels will be plotted.&#xA; Type RCircos.Get.Gene.Name.Plot.Parameters()&#xA; to see the number of labels for each chromosome. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am using the below-given script and what to know is there any way to fit all my genes in the chromosome ideogram plot&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; library(RCircos)&#xA; data(UCSC.HG19.Human.CytoBandIdeogram)&#xA; RCircos.Set.Core.Components(&#xA; cyto.info=UCSC.HG19.Human.CytoBandIdeogram, &#xA; chr.exclude = NULL, &#xA; tracks.inside = 10, tracks.outside = 0) &#xA; &#xA; #Making a plot with RCircos&#xA; out.file &amp;lt;- &amp;quot;RCircos_IF.pdf&amp;quot;;&#xA; pdf(file=out.file, height=8, width=8, compress=TRUE);&#xA; RCircos.Set.Plot.Area();&#xA; par(mai=c(0.25, 0.25, 0.25, 0.25));&#xA; plot.new();&#xA; plot.window(c(-2.5,2.5), c(-2.5, 2.5));&#xA;&#xA; # plot chromosome ideogram&#xA; RCircos.Set.Plot.Area();&#xA; RCircos.Chromosome.Ideogram.Plot() &#xA; gene.lable.data = read.table (&amp;quot;IF.txt&amp;quot;, sep=&amp;quot;\t&amp;quot;, header=T, row.names = 1) # IF.txt is my input file&#xA; RCircos.Gene.Connector.Plot(gene.lable.data, track.num = 1, side = &amp;quot;in&amp;quot;);&#xA; track.num &amp;lt;- 2;&#xA; RCircos.Gene.Name.Plot(gene.lable.data,name.col = 4, track.num)&#xA;&#xA; dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My input file looks like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    Chromosome  chromStart  chromEnd    Gene.name&#xA;    1   chr1    150363091   150476566   RPRD2&#xA;    2   chr1    150549369   150560937   ADAMTSL4&#xA;    3   chr1    91949371    92014426    BRDT&#xA;    4   chr1    31365625    31376850    FABP3&#xA;    5   chr1    150960583   150975004   CERS2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3766" LastActivityDate="2020-07-24T14:32:07.450" Title="How to fit all genes (labels) in chromosome ideogram plot made by RCircos package?" Tags="|r|circos|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13936" PostTypeId="1" CreationDate="2020-07-20T13:16:42.483" Score="0" ViewCount="4751" Body="&lt;p&gt;I'm working on a Seurat object and want to name the clusters according to 2 values alone (yes/no). So I want to add a new column to metadata and annotate the clusters (UMAP) with it.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head(B@meta.data)&#xA;        orig.ident nCount_RNA nFeature_RNA percent.mito RNA_snn_res.0.6 seurat_clusters&#xA;BC01_02       BC01   999789.2         9225 0.2385090196               6               6&#xA;BC01_03       BC01   999776.9         9568 0.2925109851               6               6&#xA;BC01_04       BC01   999595.1         7482 0.0009888404               6               6&#xA;BC01_05       BC01   999992.0         8301 0.3801925053               6               6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to add a new column imported from a csv file that contains the row.names same as the meta.data with the status column as yes and no.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head(status)&#xA;               status&#xA;BC01_02           yes&#xA;BC01_03           yes&#xA;BC01_04           yes&#xA;BC01_05           yes&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After adding the column, I want to name the clusters with yes or no, how is it possible?&lt;/p&gt;&#xA;" OwnerUserId="9234" LastActivityDate="2020-08-19T18:06:20.543" Title="How to add cluster name to metadata in Seurat?" Tags="|rna-seq|scrnaseq|seurat|sequence-annotation|clustering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13937" PostTypeId="2" ParentId="2503" CreationDate="2020-07-20T16:38:44.867" Score="0" Body="&lt;pre&gt;&lt;code&gt;BiocManager::install('biomaRt')&#xA;require(biomaRt)&#xA;ensembl &amp;lt;- useMart(&amp;quot;ENSEMBL_MART_SNP&amp;quot;, dataset = &amp;quot;hsapiens_snp&amp;quot;)&#xA;getBM(attributes=c(&amp;quot;refsnp_id&amp;quot;, &amp;quot;chr_name&amp;quot;, &amp;quot;chrom_start&amp;quot;, &amp;quot;chrom_end&amp;quot;,&#xA;                   &amp;quot;allele&amp;quot;, &amp;quot;mapweight&amp;quot;, &amp;quot;validated&amp;quot;, &amp;quot;allele_1&amp;quot;, &amp;quot;minor_allele&amp;quot;,&#xA;                   &amp;quot;minor_allele_freq&amp;quot;, &amp;quot;minor_allele_count&amp;quot;, &amp;quot;clinical_significance&amp;quot;,&#xA;                   &amp;quot;synonym_name&amp;quot;, &amp;quot;ensembl_gene_stable_id&amp;quot;),&#xA;      filters=&amp;quot;snp_filter&amp;quot;, values=c(&amp;quot;rs6025&amp;quot;,&amp;quot;rs431905509&amp;quot;),&#xA;      mart=ensembl, uniqueRows=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9235" LastActivityDate="2020-07-20T16:38:44.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13938" PostTypeId="2" ParentId="13936" CreationDate="2020-07-20T17:37:27.580" Score="1" Body="&lt;p&gt;To add the column &lt;code&gt;status&lt;/code&gt; can try the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;B@meta.data$status &amp;lt;- status[row.names(B@meta.data),]$status&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To rename the clusters you can try the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;B@meta.data$seurat_clusters &amp;lt;- B@meta.data$status&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, I think the plotting functions in Seurat do not use the cluster information from &lt;code&gt;B@meta.data&lt;/code&gt;, rather they use information from &lt;code&gt;B@active.idents&lt;/code&gt;.&#xA;So, I guess you have to do the following instead:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;B@active.idents &amp;lt;-  B@meta.data$status&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This should work, but, I will verify and update this answer later.&lt;/p&gt;&#xA;" OwnerUserId="3263" LastEditorUserId="3263" LastEditDate="2020-07-20T17:45:46.117" LastActivityDate="2020-07-20T17:45:46.117" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13939" PostTypeId="2" ParentId="10900" CreationDate="2020-07-20T18:30:19.637" Score="0" Body="&lt;p&gt;No, Maven still can read only mzXML files.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2020-07-20T18:30:19.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13940" PostTypeId="2" ParentId="9115" CreationDate="2020-07-20T19:12:13.900" Score="2" Body="&lt;p&gt;TMT stands for tandem-mass-tag. The idea is to tag different samples with one of the tags then pool all samples together and run them through the mass spectrometer together. Then the reporter groups are split off and quantified (ideally) for each peptide individually. There are different methods with different numbers of isobaric labels. Isobaric means all the labels have the same mass therefore they behave exactly the same in the mass spectrometer at first. E.g. TMT11 means you have 11 different tags with identical masses, so you can pool and process 11 samples at a time.&lt;/p&gt;&#xA;&lt;p&gt;The protein binding group attaches the label to the peptide. Then the tagged peptides from all samples are run through the chromatography and the spectrometer. At first, the MS instruments simply collects the tagged peptides with different m/z ratios at each point in time for a while. When enough peptides are collected reporter groups are cut off and the peptides then quantified. This cutting-off happens by increasing the internal energy of the collected peptides. Because the cleavage group has the weakest chemical bond it brakes first and releases the reporter group. Again, this step separates the reporter group from the rest. Finally, the cut-off reporter groups are quantified.&lt;/p&gt;&#xA;&lt;p&gt;The different reporter groups don't have the same mass, otherwise you would not be able to differentiate between them in the MS instrument. But you need same masses before quantification to ensure that the same peptides are all collected together. Therefore, you need to balance out the different masses of the reporter groups. That is the job of the normalization group. So, the normalization group ensures that overall the tags have the same mass and the peptides with different tags are not treated differently in the MS instrument before quantification. Then for quantification the reporter groups (with different masses between each other) are cut off and the MS instrument can therefore detect and quantify them individually. There is a time dependency of the whole process which happens in multiple phases. The whole process makes it possible to compare the amounts of peptides coming from the different samples.&lt;/p&gt;&#xA;&lt;p&gt;The chemistry in the chromatography column is quite complicated which makes it difficult to compare samples from different experiments. The presence of one peptide might enhance or suppress the signal of another peptide or impact the chromatographic retention time. Therefore, you should only compare within the same sample. More tags TMT11, TMT16 etc. allow you to run more samples in one run.&lt;/p&gt;&#xA;&lt;p&gt;You can then use one (or more) channels (mass-tags), to tag a standard sample and then compare everything to this standard that is assumed to be identical over multiple experiments. This way you can compare even more samples than available tags. e.g. more than 11.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2020-07-20T19:12:13.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13941" PostTypeId="1" AcceptedAnswerId="13950" CreationDate="2020-07-20T20:48:28.973" Score="2" ViewCount="36" Body="&lt;p&gt;I have 3 histone marks,I have used Macs2 for peak calling and diffBind to analyze the peaks. I was wondering if you know any way to calculate the peak numbers of one specific histone mark that occur for example within 2kb of peaks belong to another histone mark.&lt;br /&gt;&#xA;Much appreciated&lt;/p&gt;&#xA;" OwnerUserId="8543" LastActivityDate="2020-07-21T14:02:58.110" Title="How to calculate the number of peaks that are upstream/downstream of some other peaks" Tags="|sequence-analysis|chip-seq|bedtools|peak-calling|deeptools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13942" PostTypeId="1" AcceptedAnswerId="15459" CreationDate="2020-07-20T21:17:30.377" Score="3" ViewCount="1364" Body="&lt;p&gt;I would like to subset a GFF file (gene and nested features) from a gene list.&lt;/p&gt;&#xA;&lt;p&gt;The GFF file looks like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##gff-version 3&#xA;Scaffold_1      JGI     gene    22901   45904   .       +       .       ID=Genecv11000001m.g;Name=Genecv11000001m.g&#xA;Scaffold_1      JGI     mRNA    22901   45904   .       +       .       ID=PAC4GC:50510902;Name=Genecv11000001m;longest=1;Parent=Genecv11000001m.g&#xA;Scaffold_1      JGI     five_prime_UTR  22901   23284   .       +       .       ID=PAC4GC:50510902.five_prime_UTR.1;Parent=PAC4GC:50510902&#xA;Scaffold_1      JGI     CDS     23285   23423   .       +       0       ID=PAC4GC:50510902.CDS.1;Parent=PAC4GC:50510902&#xA;Scaffold_1      JGI     CDS     24031   24062   .       +       2       ID=PAC4GC:50510902.CDS.2;Parent=PAC4GC:50510902&#xA;Scaffold_1      JGI     CDS     24192   24254   .       +       0       ID=PAC4GC:50510902.CDS.3;Parent=PAC4GC:50510902&#xA;Scaffold_1      JGI     CDS     24509   24568   .       +       0       ID=PAC4GC:50510902.CDS.4;Parent=PAC4GC:50510902&#xA;Scaffold_1      JGI     CDS     37558   37603   .       +       0       ID=PAC4GC:50510902.CDS.5;Parent=PAC4GC:50510902&#xA;Scaffold_1      JGI     CDS     37775   37821   .       +       2       ID=PAC4GC:50510902.CDS.6;Parent=PAC4GC:50510902&#xA;Scaffold_1      JGI     CDS     37927   38228   .       +       0       ID=PAC4GC:50510902.CDS.7;Parent=PAC4GC:50510902&#xA;Scaffold_1      JGI     CDS     42345   42702   .       +       1       ID=PAC4GC:50510902.CDS.8;Parent=PAC4GC:50510902&#xA;Scaffold_1      JGI     CDS     42798   43343   .       +       0       ID=PAC4GC:50510902.CDS.9;Parent=PAC4GC:50510902&#xA;Scaffold_1      JGI     CDS     44798   45079   .       +       0       ID=PAC4GC:50510902.CDS.10;Parent=PAC4GC:50510902&#xA;Scaffold_1      JGI     three_prime_UTR 45080   45904   .       +       .       ID=PAC4GC:50510902.three_prime_UTR.1;Parent=PAC4GC:50510902&#xA;Scaffold_1      JGI     mRNA    22901   45904   .       +       .       ID=PAC4GC:50510903;Name=Genecv11000002m;longest=0;Parent=Genecv11000001m.g&#xA;Scaffold_1      JGI     five_prime_UTR  22901   23284   .       +       .       ID=PAC4GC:50510903.five_prime_UTR.1;Parent=PAC4GC:50510903&#xA;Scaffold_1      JGI     CDS     23285   23423   .       +       0       ID=PAC4GC:50510903.CDS.1;Parent=PAC4GC:50510903&#xA;Scaffold_1      JGI     CDS     24031   24062   .       +       2       ID=PAC4GC:50510903.CDS.2;Parent=PAC4GC:50510903&#xA;Scaffold_1      JGI     CDS     24198   24254   .       +       0       ID=PAC4GC:50510903.CDS.3;Parent=PAC4GC:50510903&#xA;Scaffold_1      JGI     CDS     24509   24568   .       +       0       ID=PAC4GC:50510903.CDS.4;Parent=PAC4GC:50510903&#xA;Scaffold_1      JGI     CDS     37558   37603   .       +       0       ID=PAC4GC:50510903.CDS.5;Parent=PAC4GC:50510903&#xA;Scaffold_1      JGI     CDS     37775   37821   .       +       2       ID=PAC4GC:50510903.CDS.6;Parent=PAC4GC:50510903&#xA;Scaffold_1      JGI     CDS     37927   38228   .       +       0       ID=PAC4GC:50510903.CDS.7;Parent=PAC4GC:50510903&#xA;Scaffold_1      JGI     CDS     42345   42702   .       +       1       ID=PAC4GC:50510903.CDS.8;Parent=PAC4GC:50510903&#xA;Scaffold_1      JGI     CDS     42798   43343   .       +       0       ID=PAC4GC:50510903.CDS.9;Parent=PAC4GC:50510903&#xA;Scaffold_1      JGI     CDS     44798   45079   .       +       0       ID=PAC4GC:50510903.CDS.10;Parent=PAC4GC:50510903&#xA;Scaffold_1      JGI     three_prime_UTR 45080   45904   .       +       .       ID=PAC4GC:50510903.three_prime_UTR.1;Parent=PAC4GC:50510903&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And a have the target genes in a list such as&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Genecv11033552m&#xA;Genecv11003131m&#xA;Genecv11036683m&#xA;Genecv11012576m&#xA;Genecv11003654m&#xA;Genecv11012587m&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I know that is possible to subset the gff using &lt;code&gt;grep -f gene_list.txt &amp;lt;gff_file&amp;gt;&lt;/code&gt;. However this extracts only the gene and mRNA features, missing the CDS and UTR entries, while I would like to subset the gene together with all its children features (mRNA, five_prime_UTR, CDS, three_prime_UTR).&lt;/p&gt;&#xA;&lt;p&gt;This happens because the ID in CDS and UTR features are the same of the mRNA and not as in the gene feature.&lt;/p&gt;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="6767" LastEditorUserId="6767" LastEditDate="2021-03-04T22:05:48.593" LastActivityDate="2021-03-04T22:05:48.593" Title="How to subset genes and its nested features from a GFF file using a gene list" Tags="|python|bash|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13943" PostTypeId="2" ParentId="13942" CreationDate="2020-07-20T22:52:46.793" Score="1" Body="&lt;p&gt;Given the nested structure of a GFF file, it may be easier to make a python script using a GFF file parser, like &lt;a href=&quot;https://pypi.org/project/gff3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gff3&lt;/a&gt;, &lt;a href=&quot;https://daler.github.io/gffutils/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gffutils&lt;/a&gt;, or &lt;a href=&quot;https://biopython.org/wiki/GFF_Parsing&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioPython&lt;/a&gt;.&#xA;Dealing with nested data in &lt;code&gt;bash&lt;/code&gt; or with some fancy &lt;code&gt;awk&lt;/code&gt; script is probably going to trickier to write than it's worth.&lt;/p&gt;&#xA;&lt;p&gt;You can start by ensuring your list of target genes is sorted in the same order as the GFF file, then iterate through the file. When you encounter a like with &lt;code&gt;mRNA&lt;/code&gt; in the 3rd column, store that ID value and extract all the subsequent lines with that same ID.&lt;/p&gt;&#xA;&lt;p&gt;If the line does not have that ID, you know you've collected all the genes, mRNAs, etc associated with that gene and can pop it from your queue.&lt;/p&gt;&#xA;&lt;p&gt;Repeat until you reach the end of the GFF file or your list of target genes is empty.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2020-07-20T22:52:46.793" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13944" PostTypeId="2" ParentId="10669" CreationDate="2020-07-20T22:58:27.170" Score="2" Body="&lt;p&gt;From the &lt;a href=&quot;https://broadinstitute.github.io/picard/picard-metric-definitions.html#DuplicationMetrics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Picard documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;DUPLICATION METRICS:&#xA;Metrics that are calculated during the process of marking duplicates within a stream of SAMRecords.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;UNMAPPED_READS    The total number of unmapped reads examined. (Primary, non-supplemental)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It won't alter the flags on these reads, but it will count them in the summary report it generates.&#xA;You should be able to test this yourself with a small set of mapped + unmapped reads&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2020-07-20T22:58:27.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13946" PostTypeId="1" AcceptedAnswerId="13947" CreationDate="2020-07-21T04:14:07.253" Score="1" ViewCount="230" Body="&lt;p&gt;I have been creating phylogenetic trees using RAxML (using Boostrap + ML) and I end up with a bestTree file, outputted in newick format. When I open up the tree in FigTree, I notice that it collapses many of the internal nodes with small values (1 x 10-6). I am working on visualizing the trees using ETE, which when I render the tree includes these small breakpoints. I was wondering if there is a standard for when to collapse nodes that are considered phylogenetically uninformative that I can apply. Thank you for any help.&lt;/p&gt;&#xA;" OwnerUserId="9238" LastActivityDate="2020-07-21T17:40:09.837" Title="Standard for when to collapse phylogenetically uninformative nodes" Tags="|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13947" PostTypeId="2" ParentId="13946" CreationDate="2020-07-21T08:26:23.357" Score="1" Body="&lt;p&gt;If 1E6 is genetic distance, this means one nucleotide difference per million base pairs, which is of course incredibly small and bootstrapping at this level doesn't really work. So what I suspect is happening is that the bootstrap is so low it is not present on the tree. You can assign a cut-off bootstrap but anything below 75% is not interpretable in any case. What might be happening is you are producing the bootstrap &amp;quot;cladogram&amp;quot; which is showing a default branch length, but when &amp;lt;1E6 genetic distance it is showing a trifurcation. This simply means the bootstrap of the consensus tree is so low it is collapsing the branch and this is a function of the very small genetic distance at this node.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Looking at your treefile I don't see any collapsed clades.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(Group_9:0.00000100000050002909,(Group_50:0.00507659947127924935, (.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The clade above has not been colapsed it is just a very small distance in a paraphyletic group. It looks fine to me, i.e. Group_9 is 1E6. It could look 'collapsed' simply because it is so small it is difficult to see. I hope thats ok?&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-21T17:40:09.837" LastActivityDate="2020-07-21T17:40:09.837" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="13948" PostTypeId="1" CreationDate="2020-07-21T09:17:47.730" Score="0" ViewCount="89" Body="&lt;p&gt;Say a database were to store human omics datasets. The human subjects are known and the sample size is rather small in size initially (n=500). The database contains genomics, transcriptomics, proteomics, gut metabolomics, and epigenomics. Since the sample size is rather small initially, it would be important that certain identifying features are mitigated so that individual subjects cannot be identified. This leads me to two questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What identifiable features could be present in these different types of raw data? (Such as race, sex, age, hair color, eye color, height, where someone has lived, and anything else I may not even be considering)? Which of these omics types would be the most and least dangerous for identifying individuals? How can the identifiable nature of these data be mitigated?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If environmental metagenomics is also collected by these human subjects, is it possible to identify the human subjects by contamination? (i.e. some of the reads from the metagenomics data inadvertently contain human reads?) How can the identifiable nature of these data be mitigated?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I think this subject may be a bit futuristic, but I am very uninformed. If there are any references that provide additional thinking about these topics, please kindly share. Thank you for sharing your thoughts.&lt;/p&gt;&#xA;" OwnerUserId="9240" LastActivityDate="2020-07-21T16:09:41.717" Title="How identifiable are human omics data and how to mitigate their identifying features?" Tags="|genomics|metagenome|proteomics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13949" PostTypeId="2" ParentId="13948" CreationDate="2020-07-21T10:43:04.783" Score="2" Body="&lt;p&gt;This is one of the major problems with genomic information in todays research. This was highlighted some years ago with the police using publicly available genomic data bases to idenitfy unkown murder suspects.&lt;/p&gt;&#xA;&lt;p&gt;The full extent of this issue was exemplified in a&#xA;&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/30309907/&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; by Yaniv Erlich from 2018 that should be a good starting point for you.&lt;br /&gt;&#xA;They claim that giving the current (2018) amount genetic information more than half the searches for a person of european descent will result in a thrid cousin or better match. And these odds will only inprove with time.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, the genetic information not only can give you a hint at inherited variables like the approximate height of a person but can even yield their name and address.&lt;/p&gt;&#xA;&lt;p&gt;About your second question, I would think that the amount of host (i.e human) material collected alongside the microbiome sample is rather small. This probably is not sufficient to obtain enough coverage for a profile. But I could be mistaken.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastEditorUserId="4885" LastEditDate="2020-07-21T14:12:30.033" LastActivityDate="2020-07-21T14:12:30.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13950" PostTypeId="2" ParentId="13941" CreationDate="2020-07-21T14:02:58.110" Score="1" Body="&lt;p&gt;You can use &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/window.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;windowBed&lt;/a&gt; from bedtools with &lt;code&gt;-c&lt;/code&gt; if you just want counts. Or use &lt;code&gt;-wa&lt;/code&gt; &lt;code&gt;-wb&lt;/code&gt; and then use &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/groupby.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;groupBy&lt;/a&gt; for any other operations.&lt;/p&gt;&#xA;" OwnerUserId="4017" LastActivityDate="2020-07-21T14:02:58.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13951" PostTypeId="2" ParentId="13948" CreationDate="2020-07-21T15:49:19.287" Score="1" Body="&lt;p&gt;PPK provides a great answer, but for question 2 I can provide a different perspective. For shotgun metagenome sequencing (without any enrichment/depletion protocols) it is common for &amp;gt;90% of reads to map to the human genome. There is variation across body sites, for example the gut microbiome is rich and so the % of human reads from a stool sample will be less compared to say a blood sample.&lt;/p&gt;&#xA;&lt;p&gt;Checkout &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-015-0759-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this study&lt;/a&gt;, where they mined human microbiome datasets for host (human) reads and were able to re-construct draft full human genomes with sometimes 20X coverage. See also part of their discussion:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We show here that it is possible to reconstruct complete host genomes&#xA;using metagenomic sequence data, which is potentially identifiable.&#xA;However, this was possible due to the unique study design of the HMP,&#xA;whereby multiple body sites from each individual were sequenced at a&#xA;high depth, allowing us to pool data across body sites and reach a 10x&#xA;mean coverage per host genome. Common metagenomic shotgun sequencing&#xA;studies, which usually include an order of magnitude less sequence&#xA;data, are unlikely to enable such an analysis. Moreover, the majority&#xA;of studies sequence stool samples, which include many fewer&#xA;host-derived reads. Nevertheless, we anticipate that future shotgun&#xA;metagenomics sequencing studies would consider these potential privacy&#xA;concerns.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="104" LastEditorUserId="104" LastEditDate="2020-07-21T16:09:41.717" LastActivityDate="2020-07-21T16:09:41.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13952" PostTypeId="2" ParentId="4687" CreationDate="2020-07-21T17:57:52.857" Score="0" Body="&lt;p&gt;You can now install MaxQuant simply with Conda:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c bioconda maxquant&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and to run MaxQuant type:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;maxquant mqpar.xml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioconda.github.io/recipes/maxquant/README.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reference&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2022-05-18T00:58:20.340" LastActivityDate="2022-05-18T00:58:20.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13953" PostTypeId="1" CreationDate="2020-07-21T18:52:43.677" Score="0" ViewCount="51" Body="&lt;p&gt;I want to extract metadata from experiment SRX1596422. I have sratoolkit and Entrex direct but not sure what line of code to use. Here's the link for the experiment: &lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/sra.cgi?view=run_browser&amp;amp;run=SRR390728&quot; rel=&quot;nofollow noreferrer&quot;&gt;experiment SRX1596422&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9245" LastActivityDate="2020-07-21T21:50:37.483" Title="How to extract metadata from NCBI's experiment?" Tags="|phylogenetics|sratoolkit|entrez|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13954" PostTypeId="1" AcceptedAnswerId="13956" CreationDate="2020-07-21T21:26:21.033" Score="0" ViewCount="52" Body="&lt;p&gt;My data contain several chip-seq results. I have the peaks called by MACS2.I wanted to only look at those peaks that their size is e.g 500bp to 1000bp. How can I separate those peaks efficiently?&lt;/p&gt;&#xA;&lt;p&gt;I know one way is to take the peaks.xls file sort them based on the length column and take out the peaks less than 1kb. But I was looking for a more efficient way to do that. Any suggestion?&lt;/p&gt;&#xA;&lt;p&gt;Many Thanks&lt;/p&gt;&#xA;" OwnerUserId="8543" LastEditorUserId="5077" LastEditDate="2020-07-22T09:28:35.330" LastActivityDate="2020-07-23T08:11:30.537" Title="Separating peaks of chip-seq with specific length" Tags="|sequence-analysis|differential-expression|chip-seq|peak-calling|macs2|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13955" PostTypeId="2" ParentId="13953" CreationDate="2020-07-21T21:50:37.483" Score="3" Body="&lt;p&gt;You'll want something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db sra -query SRX1596422 | efetch -format runinfo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will produce a CSV output to the screen with columns containing the meta information available in SRA:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Run,ReleaseDate,LoadDate,spots,bases,spots_with_mates,avgLength,size_MB,AssemblyName,download_path,Experiment,LibraryName,LibraryStrategy,LibrarySelection,LibrarySource,LibraryLayout,InsertSize,InsertDev,Platform,Model,SRAStudy,BioProject,Study_Pubmed_id,ProjectID,Sample,BioSample,SampleType,TaxID,ScientificName,SampleName,g1k_pop_code,source,g1k_analysis_group,Subject_ID,Sex,Disease,Tumor,Affection_Status,Analyte_Type,Histological_Type,Body_Site,CenterName,Submission,dbgap_study_accession,Consent,RunHash,ReadHash&#xA;SRR3184375,2016-02-24 00:00:00,2016-02-23 23:14:45,556018223,111203644600,556018223,200,57997,GCA_000001635.1,https://sra-downloadb.be-md.ncbi.nlm.nih.gov/sos2/sra-pub-run-7/SRR3184375/SRR3184375.1,SRX1596422,,WGS,size fractionation,GENOMIC,PAIRED,0,0,ILLUMINA,Illumina HiSeq 2000,SRP070623,PRJNA312530,,312530,SRS1306521,SAMN04492320,simple,10090,Mus musculus,MCNT_E1,,,,,female,,no,,,,,THE SCRIPPS RESEARCH INSTITUTE,SRA356784,,public,52A6F836C13F2D15755EDE3E52A18D78,240E7130CFE36405818DA5EE2A10E24C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The more relevant fields are: Subject_ID, Sex, Disease, Tumor, Affection_Status, Analyte_Type, Histological_Type, Body_Site. Note that not all of these are available for that particular sample.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-07-21T21:50:37.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13956" PostTypeId="2" ParentId="13954" CreationDate="2020-07-22T06:43:10.943" Score="1" Body="&lt;p&gt;You can do it by awk.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk -v OFS=&amp;quot;\t&amp;quot; '{ if ( (&lt;span class=&quot;math-container&quot;&gt;$3-$&lt;/span&gt;2) &amp;gt;= 500 &amp;amp;&amp;amp; (&lt;span class=&quot;math-container&quot;&gt;$3-$&lt;/span&gt;2) &amp;lt;= 1000) { print } }' in.bed &amp;gt; out.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Assuming there is no header.&lt;/p&gt;&#xA;" OwnerUserId="4017" LastActivityDate="2020-07-22T06:43:10.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13957" PostTypeId="2" ParentId="13954" CreationDate="2020-07-22T09:34:37.803" Score="1" Body="&lt;p&gt;You can use &lt;code&gt;rtracklayer&lt;/code&gt; from &lt;code&gt;R bioconductor&lt;/code&gt; to import it, filter and do much more stuff, i use the code from this &lt;a href=&quot;https://charlesjb.github.io/How_to_import_narrowPeak/&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog&lt;/a&gt; for importing the narrow bed file. Below I use an example file since you did not provide yours:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(rtracklayer)&#xA;fl = &amp;quot;zas1_default_peaks.narrowPeak&amp;quot;&#xA;&#xA;extraCols_narrowPeak &amp;lt;- c(signalValue = &amp;quot;numeric&amp;quot;, pValue = &amp;quot;numeric&amp;quot;,&#xA;                          qValue = &amp;quot;numeric&amp;quot;, peak = &amp;quot;integer&amp;quot;)&#xA;gr &amp;lt;- import(fl, format = &amp;quot;BED&amp;quot;,&#xA;                        extraCols = extraCols_narrowPeak)&#xA;&#xA;head(gr)&#xA;&#xA;GRanges object with 6 ranges and 6 metadata columns:&#xA;      seqnames        ranges strand |                name     score signalValue&#xA;         &amp;lt;Rle&amp;gt;     &amp;lt;IRanges&amp;gt;  &amp;lt;Rle&amp;gt; |         &amp;lt;character&amp;gt; &amp;lt;numeric&amp;gt;   &amp;lt;numeric&amp;gt;&#xA;  [1] AB325691     8982-9219      * | zas1_default_peak_1        18     1.31697&#xA;  [2] AB325691   16685-18050      * | zas1_default_peak_2        59     1.51283&#xA;  [3]        I 101971-102188      * | zas1_default_peak_3        17     1.23717&#xA;  [4]        I 102390-103538      * | zas1_default_peak_4        19     1.24967&#xA;  [5]        I 152286-152496      * | zas1_default_peak_5        23     1.37631&#xA;  [6]        I 175025-175577      * | zas1_default_peak_6        69     1.62063&#xA;         pValue    qValue      peak&#xA;      &amp;lt;numeric&amp;gt; &amp;lt;numeric&amp;gt; &amp;lt;integer&amp;gt;&#xA;  [1]   3.37358   1.89101       146&#xA;  [2]   8.12841   5.92232       419&#xA;  [3]   3.26012   1.79545        64&#xA;  [4]   3.47875   1.97673       708&#xA;  [5]   3.89921   2.32948       136&#xA;  [6]   9.34429    6.9608       374&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The narrowBed and peaks.xls contains the same information. If you really really need to read in the xls, you can do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(GenomicRanges)&#xA;peaks = read.table(&amp;quot;zas1_default_peaks.xls&amp;quot;,comment=&amp;quot;#&amp;quot;,header=TRUE)&#xA;gr = makeGRangesFromDataFrame(peaks,keep.extra.columns=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Once you have the genomic ranges object, filtering is the same:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; gr[width(gr)&amp;gt;500 &amp;amp; width(gr)&amp;lt;1000]&#xA;GRanges object with 208 ranges and 7 metadata columns:&#xA;        seqnames          ranges strand |    length abs_summit    pileup&#xA;           &amp;lt;Rle&amp;gt;       &amp;lt;IRanges&amp;gt;  &amp;lt;Rle&amp;gt; | &amp;lt;integer&amp;gt;  &amp;lt;integer&amp;gt; &amp;lt;numeric&amp;gt;&#xA;    [1]        I   175025-175577      * |       553     175399    186.99&#xA;    [2]        I   186807-187489      * |       683     187009    208.66&#xA;    [3]        I   339070-339700      * |       631     339258    222.91&#xA;    [4]        I   431557-432088      * |       532     431734    195.66&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastEditorUserId="5077" LastEditDate="2020-07-23T08:11:30.537" LastActivityDate="2020-07-23T08:11:30.537" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13958" PostTypeId="1" AcceptedAnswerId="13959" CreationDate="2020-07-22T11:13:24.880" Score="0" ViewCount="1300" Body="&lt;p&gt;I have a library I would like to split them based on their headers. Some are related to RNA and DNA.&lt;/p&gt;&#xA;&lt;p&gt;The header contains much information, but the most important the presence of DNA or RNA/LTR.., these partial words could be in between or at the beginning.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The point is the knowledge of how to extract sequences from a partial header occur in between the ID&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;My question is how to use grep or awk to grep the header that has one of these words along with sequences?&#xA;Note that The sequences are more than one line.&lt;/p&gt;&#xA;&lt;p&gt;Or perhaps grep the specific word and ignore what before and after?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Tigger16a#DNA/TcMar-Tigger DF0000028 TcMar-Tigger **DNA** transposon &#xA;&amp;gt;rnd-4_family-38#SINE/MIR ( Recon Family Size = 20, Final Multiple Alignmen &#xA;&amp;gt;rnd-6_family-31751#LTR/Gypsy ( Recon Family Size = 26, Final Multiple Alignment Size = 22 )&#xA;&amp;gt;RNA2558#LTR/ERVL &#xA;&amp;gt;NonDNA1#LINE/I-Jockey &#xA;&amp;gt;DNA5#DNA/TcMar-Tc1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have tried to create a list of the Required IDs using grep creating the list and extracting the sequences but for some reasons, the output has more sequences than those specified in the DNAID.txt list.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -A1000 -w -f DNAID.txt.fa MyLibrary &amp;gt; DNA_Sequence.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3332" LastEditorUserId="3332" LastEditDate="2020-07-22T12:43:08.580" LastActivityDate="2020-07-22T14:10:13.270" Title="Extract sequences from partial Header" Tags="|awk|filtering|grep|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13959" PostTypeId="2" ParentId="13958" CreationDate="2020-07-22T12:50:57.387" Score="2" Body="&lt;p&gt;You could use &lt;a href=&quot;http://biopython.org/DIST/docs/tutorial/Tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biopython&lt;/a&gt;. If its a fasta file, it will be complicated to write fasta output (especially multiline fasta) with &lt;code&gt;grep&lt;/code&gt; or &lt;code&gt;awk&lt;/code&gt;. Simple solution is to use biopython, so that you can even match for any complex patterns in the fasta header.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;rna_records  = []&#xA;dna_records = []&#xA;&#xA;for seq in SeqIO.parse(&amp;quot;in.fa&amp;quot;,&amp;quot;fasta&amp;quot;):&#xA;&#xA;    if &amp;quot;RNA&amp;quot; in seq.id:&#xA;        rna_records.append(seq)&#xA;&#xA;    elif &amp;quot;DNA&amp;quot; in seq.id:&#xA;        dna_records.append(seq)&#xA;&#xA;SeqIO.write(rna_records, &amp;quot;RNA_out.fa&amp;quot;,&amp;quot;fasta&amp;quot;)&#xA;SeqIO.write(dna_records, &amp;quot;DNA_out.fa&amp;quot;,&amp;quot;fasta&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;This code needs to be optimised if you are dealing with large sequences like human genome fasta as it holds everything in memory.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="4017" LastActivityDate="2020-07-22T12:50:57.387" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13960" PostTypeId="1" CreationDate="2020-07-22T13:24:53.937" Score="0" ViewCount="9255" Body="&lt;p&gt;I am pretty new to Seurat. I want to subset from my original seurat object (BC3) meta.data based on orig.ident. however, when i use subset(), it returns with Error.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ER_HER_P &amp;lt;- subset(BC3, idents = c(&amp;quot;BC03&amp;quot;))&#xA;Error: No cells found&#xA;&#xA;table(Idents(BC3))&#xA;BC01   BC02   BC03 BC03LN   BC04   BC05   BC06   BC07 BC07LN   BC08   BC09   BC10   BC11 &#xA;21     50     33     51     42     74     15     48     52     21     55     13     11 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There are 33 cells under the identity. I checked the active.ident to make sure the identity has not shifted to any other column, but still I am getting the error?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head(BC3@active.ident)&#xA;[1] BC01 BC01 BC01 BC01 BC01 BC01&#xA;Levels: BC01 BC02 BC03 BC03LN BC04 BC05 BC06 BC07 BC07LN BC08 BC09 BC10 BC11&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9234" LastActivityDate="2020-12-19T18:04:25.480" Title="Subsetting from seurat object based on orig.ident?" Tags="|r|scrnaseq|seurat|subset|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13961" PostTypeId="2" ParentId="13958" CreationDate="2020-07-22T14:10:13.270" Score="1" Body="&lt;p&gt;Here is a solution with &lt;code&gt;awk&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;p&gt;This is the example fasta file which I used:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Test DNA 1&#xA;ACGTAAGGATATAATAC&#xA;ACGTA&#xA;AGGATA&#xA;&amp;gt;Test 2 RNA &#xA;ACGGT&#xA;GGTA&#xA;CGGA&#xA;&amp;gt;DNA - TEST 3&#xA;ACG&#xA;GATA&#xA;AGGT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can create a file called &lt;code&gt;fasta_search.awk&lt;/code&gt; and insert following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/awk -f&#xA;BEGIN {&#xA;    hit=0&#xA;}&#xA;{&#xA;    if ($0 ~ /^&amp;gt;/) &#xA;    {   &#xA;        if ($0 ~ search)&#xA;        {&#xA;            hit=1;&#xA;            print $0; &#xA;        }&#xA;        else &#xA;        {&#xA;            hit=0; &#xA;        }&#xA;     }   &#xA;    else if(hit==1)&#xA;    {   &#xA;        print $0&#xA;    }   &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now you can run:&#xA;&lt;code&gt;àwk -v search=&amp;quot;DNA&amp;quot; -f fasta_search.awk my_fastas.fa&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;search&lt;/code&gt; is therefore a awk variable which could be any String you are looking for.&lt;/p&gt;&#xA;&lt;p&gt;As result I get this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Test DNA 1&#xA;ACGTAAGGATATAATAC&#xA;ACGTA&#xA;AGGATA&#xA;&amp;gt;DNA - TEST 3&#xA;ACG&#xA;GATA&#xA;AGGT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To get all your keywords, you could run:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in DNA RNA LTR; do awk -v search=&amp;quot;$i&amp;quot; -f fasta_search.awk my_fastas.fa &amp;gt; lib_$i.fasta; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This results in &lt;code&gt;head lib_*&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;==&amp;gt; lib_DNA.fasta &amp;lt;==&#xA;&amp;gt;Test DNA 1&#xA;ACGTAAGGATATAATAC&#xA;ACGTA&#xA;AGGATA&#xA;&amp;gt;DNA - TEST 3&#xA;ACG&#xA;GATA&#xA;AGGT&#xA;&#xA;==&amp;gt; lib_LTR.fasta &amp;lt;==&#xA;&#xA;==&amp;gt; lib_RNA.fasta &amp;lt;==&#xA;&amp;gt;Test 2 RNA&#xA;ACGGT&#xA;GGTA&#xA;CGGA&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2020-07-22T14:10:13.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13962" PostTypeId="2" ParentId="13960" CreationDate="2020-07-22T15:36:10.277" Score="0" Body="&lt;p&gt;A bit too long to type as a comment.&lt;/p&gt;&#xA;&lt;p&gt;To use subset on a Seurat object, (see &lt;code&gt;?subset.Seurat&lt;/code&gt;) , you have to provide:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Subset a Seurat object&#xA;&#xA;Description:&#xA;&#xA;     Subset a Seurat object&#xA;&#xA;Usage:&#xA;&#xA;     ## S3 method for class 'Seurat'&#xA;     x[i, j, ...]&#xA;     &#xA;     ## S3 method for class 'Seurat'&#xA;     subset(x, subset, cells = NULL, features = NULL, idents = NULL, ...)&#xA;     &#xA;Arguments:&#xA;&#xA;       x: Seurat object to be subsetted&#xA;&#xA;i, features: A vector of features to keep&#xA;&#xA;j, cells: A vector of cells to keep&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So you either use the matrix to subset:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(Seurat)&#xA;data(pbmc_small)&#xA;Idents(pbmc_small) = paste0(&amp;quot;BC&amp;quot;,Idents(pbmc_small))&#xA;table(Idents(pbmc_small))&#xA;&#xA;BC0 BC2 BC1 &#xA; 36  19  25&#xA;&#xA;test = pbmc_small[,Idents(pbmc_small)==&amp;quot;BC0&amp;quot;]&#xA;table(Idents(test))&#xA;&#xA;BC0 &#xA; 36 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or you provide the cells:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;subset(pbmc_small,cells=colnames(pbmc_small)[Idents(pbmc_small)==&amp;quot;BC0&amp;quot;])&#xA;An object of class Seurat &#xA;230 features across 36 samples within 1 assay &#xA;Active assay: RNA (230 features, 20 variable features)&#xA; 2 dimensional reductions calculated: pca, tsne&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What you have should work, but try calling the actual function (in case there are packages that clash):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Seurat:::subset.Seurat(pbmc_small,idents=&amp;quot;BC0&amp;quot;)&#xA;&#xA;An object of class Seurat &#xA;230 features across 36 samples within 1 assay &#xA;Active assay: RNA (230 features, 20 variable features)&#xA; 2 dimensional reductions calculated: pca, tsne&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-07-22T15:36:10.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13963" PostTypeId="2" ParentId="13960" CreationDate="2020-07-22T17:51:44.300" Score="1" Body="&lt;p&gt;If you are going to use idents like that, make sure that you have told the software what your default ident category is.&lt;/p&gt;&#xA;&lt;p&gt;This works for me, with the metadata column being called &amp;quot;group&amp;quot;, and &amp;quot;endo&amp;quot; being one possible group there.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Idents(combined.all) &amp;lt;- &amp;quot;group&amp;quot;&#xA;endo_subset &amp;lt;- subset(combined.all, idents = c(&amp;quot;endo&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I think this is basically what you did, but I think this looks a little nicer&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-07-22T17:51:44.300" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13964" PostTypeId="2" ParentId="13914" CreationDate="2020-07-22T21:59:10.343" Score="0" Body="&lt;p&gt;You can use &lt;code&gt;parallel&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;parallel -j 3 fastq-dump {} ::: SRR10611214 SRR10611215 SRR10611215 SRR10611216 SRR10611217&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The option &lt;code&gt;-j&lt;/code&gt; says how many jobs should maximal run parallel. So in this case maximal 3 identifier would be handled at the same time.&lt;/p&gt;&#xA;&lt;p&gt;How many jobs you can run parallel depends on your machine.&lt;/p&gt;&#xA;&lt;p&gt;You can also take a look at &lt;a href=&quot;https://github.com/rvalieris/parallel-fastq-dump&quot; rel=&quot;nofollow noreferrer&quot;&gt;parallel-fastq-dump&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2020-07-22T21:59:10.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13965" PostTypeId="1" CreationDate="2020-07-22T22:02:23.740" Score="0" ViewCount="383" Body="&lt;p&gt;I am new to the Seurat package and was looking at the tutorials on the Seurat website and am just curious to know if there is any way to perform a t.test for the expression of a specific gene between two groups within the Seurat package, even possibly showing these results with box plots? Or any way to extract this information from the Seurat object in order to do this using ggplot or anything of that sort? I don't have any specific examples as I've just been going through the Seurat tutorials trying to understand everything, but if anyone has any sort of advice on the topic I would greatly appreciate it!&lt;/p&gt;&#xA;" OwnerUserId="7036" LastActivityDate="2020-07-29T14:53:12.717" Title="Seurat DE t.test" Tags="|gene|proteins|genome|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13967" PostTypeId="2" ParentId="13935" CreationDate="2020-07-23T00:12:10.457" Score="2" Body="&lt;p&gt;To change the maximum number of genes in &lt;code&gt;RCircos&lt;/code&gt;, we could modified the &lt;code&gt;char.width&lt;/code&gt; according to this &lt;a href=&quot;https://www.biostars.org/p/80175/#285205&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;.&#xA;The actual value in a RCircos session could be modified with get and reset methods for plot parameters.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;params &amp;lt;- RCircos.Get.Plot.Parameters() #&lt;span class=&quot;math-container&quot;&gt;$char.width&#xA;params$&lt;/span&gt;char.width &amp;lt;- 100 #default 500&#xA;RCircos.Reset.Plot.Parameters(params)&#xA;&#xA;#the maxLabels are updated accordingly&#xA;RCircos.Get.Gene.Name.Plot.Parameters()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9172" LastActivityDate="2020-07-23T00:12:10.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13969" PostTypeId="2" ParentId="13902" CreationDate="2020-07-23T00:44:58.913" Score="2" Body="&lt;p&gt;One way is to increase the outer margins of plot.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;par(oma=c(7,0,0,0))&#xA;tcgaCompare(maf = laml, primarySite = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9172" LastActivityDate="2020-07-23T00:44:58.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13970" PostTypeId="1" CreationDate="2020-07-23T05:33:26.547" Score="2" ViewCount="93" Body="&lt;p&gt;In the &lt;a href=&quot;https://www.embopress.org/doi/full/10.1038/Msb.2011.75&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clustal Omega paper&lt;/a&gt;, it says the following about mBED (&lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the number of input sequences, I assume):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;we use a modified version of mBed (Blackshields et al , 2010), which has complexity of &lt;span class=&quot;math-container&quot;&gt;$O(N \log N)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But in the &lt;a href=&quot;https://link.springer.com/article/10.1186/1748-7188-5-21&quot; rel=&quot;nofollow noreferrer&quot;&gt;mBED paper&lt;/a&gt; it says the following (section 2):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A number of &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; sequences are initially sampled from the input dataset &lt;span class=&quot;math-container&quot;&gt;$X .$&lt;/span&gt; Following the LLR&#xA;algorithm, this value is set by default to &lt;span class=&quot;math-container&quot;&gt;$t=\left(\log _{2} N\right)^{2} .$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;After the seed sequences in &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; have been chosen, all sequences in the input data are associated with a &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt;-dimensional vector. This is done simply by computing the distances from all sequences to each of the &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; seeds.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Then my conclusion is that this would lead to a complexity of &lt;span class=&quot;math-container&quot;&gt;$O(N (\log N)^2)$&lt;/span&gt; because for each  sequence (in total &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; of them), distances to &lt;span class=&quot;math-container&quot;&gt;$t = (\log N)^2$&lt;/span&gt; sequences must be calculated. Where did I do wrong?&lt;/p&gt;&#xA;" OwnerUserId="9257" LastActivityDate="2020-07-25T08:22:30.147" Title="mBED has time complexity $O(n \lg n)$, claimed by Clustal Omega paper, why?" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13971" PostTypeId="2" ParentId="13745" CreationDate="2020-07-23T06:05:01.893" Score="2" Body="&lt;p&gt;To set the ball rolling, I tried to implement it manually with your files. Overall, there are three steps:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Locate &lt;code&gt;NITNLC&lt;/code&gt; (or &lt;code&gt;HAPATV&lt;/code&gt;) in protein &lt;code&gt;P0DTC2&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Locate mismatched amino acids between proteins &lt;code&gt;P0DTC2&lt;/code&gt; and &lt;code&gt;K9N5Q8&lt;/code&gt; within the range from step 1.&lt;/li&gt;&#xA;&lt;li&gt;Print the amino acid and the DNA codon of both proteins from step 2.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;It works, but only for the first 60 amino acids. I wonder if it's due to the amino sequences in &lt;code&gt;AMINOOO_seq_removed.fasta&lt;/code&gt; repeat itself every 60 acids. But why?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#For example, the first three lines of protein P0DTC2&#xA;&amp;gt;P0DTC2&#xA;MFVFLVLLPLVSSQCVNLTTRTQLPPAYTNSFTRGVYYPDKVFRSSVLHSTQDLFLPFFS MFVFLVLLPLVSSQCVNLTTRTQLPPAYTNSFTRGVYYPDKVFRSSVLHSTQDLFLPFFS&#xA;NVTWFHAIHVSGTNGTKRFDNPVLPFNDGVYFASTEKSNIIRGWIFGTTLDSKTQSLLIV NVTWFHAIHVSGTNGTKRFDNPVLPFNDGVYFASTEKSNIIRGWIFGTTLDSKTQSLLIV&#xA;NNATNVVIKVCEFQFCNDPFLGVYYHKNNKSWMESEFRVYSSANNCTFEYVSQPFLMDLE NNATNVVIKVCEFQFCNDPFLGVYYHKNNKSWMESEFRVYSSANNCTFEYVSQPFLMDLE&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Step 0: Read the files.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library('seqinr')&#xA;&#xA;#align file containing protein sequences&#xA;count_added &amp;lt;- read.alignment('count_added_.clustal_num', format='clustal')&#xA;names(count_added&lt;span class=&quot;math-container&quot;&gt;$seq) &amp;lt;- count_added$&lt;/span&gt;nam&#xA;&#xA;#DNA sequences&#xA;rev3.aln &amp;lt;- read.alignment('rev3.aln', format='fasta')&#xA;names(rev3.aln&lt;span class=&quot;math-container&quot;&gt;$seq) &amp;lt;- rev3.aln$&lt;/span&gt;nam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Step 1: Locate &lt;code&gt;NITNLC&lt;/code&gt; (or &lt;code&gt;HAPATV&lt;/code&gt;) in protein &lt;code&gt;P0DTC2&lt;/code&gt;. Again, there are two repeats of &lt;code&gt;NITNLC&lt;/code&gt; 60 acids apart (at 807 and 870).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;locate &amp;lt;- function(seq, find)&#xA;  {address &amp;lt;- gregexpr(paste(strsplit(find, '')[[1]], collapse='[^a-z]*'), seq)&#xA;  #substr(seq, address[[1]][1], address[[1]][1]+attr(address[[1]], 'match.length')[1]-1)&#xA;&#xA;  return(list(start=as.numeric(address[[1]]), &#xA;             end=as.numeric(address[[1]] + attr(address[[1]], 'match.length') - 1)))&#xA;  }&#xA;&#xA;locate(seq=count_added&lt;span class=&quot;math-container&quot;&gt;$seq[['P0DTC2']], find='nitnlc')&#xA;#start&#xA;#807 870&#xA;#end&#xA;#812 875&#xA;locate(seq=count_added$&lt;/span&gt;seq[['P0DTC2']], find='hapatv')&#xA;#start&#xA;#1212 1279&#xA;#end&#xA;#1217 1285&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Step 2: Locate mismatched amino acids between proteins &lt;code&gt;P0DTC2&lt;/code&gt; and &lt;code&gt;K9N5Q8&lt;/code&gt; within the range from step 1. Instead of using the range 812 - 1279, I chose range 1 - 20 for demonstration purposes.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;compare &amp;lt;- function(seq1, seq2, after=0, before=100000)&#xA;  {seq1_ = strsplit(seq1, '')[[1]]&#xA;  seq2_ = strsplit(seq2, '')[[1]]&#xA;  ind = which(seq1_ != seq2_ &amp;amp; grepl('[a-z]',seq1_) &amp;amp; grepl('[a-z]',seq2_))&#xA;  ind = ind[ind&amp;gt;after &amp;amp; ind&amp;lt;before]&#xA;  #seq1_[ind[1]]&#xA;  #seq2_[ind[1]]&#xA;&#xA;  return(ind)&#xA;  }&#xA;compare(seq1=count_added&lt;span class=&quot;math-container&quot;&gt;$seq[['P0DTC2']], seq2=count_added$&lt;/span&gt;seq[['K9N5Q8']], after=1, before=20)&#xA;# [1] 5  7  8  9 10 13 14&#xA;&#xA;#protein comparison&#xA;#K9N5Q8      MIHSVFLLMFLLTPTESYVD&#xA;#P0DTC2      ----MFVFLVLLPL------&#xA;#&amp;lt;mismatch&amp;gt;      5 7890  34&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Step 3: Print the amino acid and the DNA codon of both proteins from step 2. Note that &lt;code&gt;ind&lt;/code&gt; is based on the align file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print_amino_codon &amp;lt;- function(ind, seq, seq_gene)&#xA;  {locate_amino &amp;lt;- gregexpr('[a-z]', seq)[[1]]&#xA;  if (!ind %in% locate_amino) return(NA)&#xA;  ind2 = match(ind, locate_amino)&#xA;&#xA;  return(c(amino=substr(seq, ind, ind), codon=substr(seq_gene, ind2*3-2, ind2*3)))&#xA;  }&#xA; &#xA;codon(ind=5, seq=count_added&lt;span class=&quot;math-container&quot;&gt;$seq[['K9N5Q8']], seq_gene=rev3.aln$&lt;/span&gt;seq[['K9N5Q8']])&#xA;#amino codon &#xA;#  &amp;quot;v&amp;quot; &amp;quot;atg&amp;quot; &#xA;codon(ind=6, vseq=count_added&lt;span class=&quot;math-container&quot;&gt;$seq[['K9N5Q8']], seq_gene=rev3.aln$&lt;/span&gt;seq[['K9N5Q8']])&#xA;#amino codon &#xA;#  &amp;quot;f&amp;quot; &amp;quot;ttc&amp;quot; &#xA;codon(ind=7, seq=count_added&lt;span class=&quot;math-container&quot;&gt;$seq[['K9N5Q8']], seq_gene=rev3.aln$&lt;/span&gt;seq[['K9N5Q8']])&#xA;#amino codon &#xA;#  &amp;quot;l&amp;quot; &amp;quot;ttg&amp;quot;&#xA;&#xA;codon(ind=5, seq=count_added&lt;span class=&quot;math-container&quot;&gt;$seq[['P0DTC2']], seq_gene=rev3.aln$&lt;/span&gt;seq[['P0DTC2']])&#xA;#amino codon &#xA;#  &amp;quot;m&amp;quot; &amp;quot;atg&amp;quot; &#xA;codon(ind=6, seq=count_added&lt;span class=&quot;math-container&quot;&gt;$seq[['P0DTC2']], seq_gene=rev3.aln$&lt;/span&gt;seq[['P0DTC2']])&#xA;#amino codon &#xA;#  &amp;quot;f&amp;quot; &amp;quot;ttt&amp;quot; &#xA;codon(ind=7, seq=count_added&lt;span class=&quot;math-container&quot;&gt;$seq[['P0DTC2']], seq_gene=rev3.aln$&lt;/span&gt;seq[['P0DTC2']])&#xA;#amino codon &#xA;#  &amp;quot;v&amp;quot; &amp;quot;gtt&amp;quot; &#xA;&#xA;#protein comparison&#xA;#K9N5Q8      MIHSVFLLMFLLTPTESYVD&#xA;#P0DTC2      ----MFVFLVLLPL------&#xA;#&amp;lt;mismatch&amp;gt;      5 7890  34&#xA;#&amp;lt;print&amp;gt;         ^^^&#xA;&#xA;#K9N5Q8 gene codon&#xA;#gtg ttt cta ctg atg ttc ttg tta aca&#xA;#                ^^^ ^^^ ^^^&#xA;#P0DTC2 gene codon&#xA;#atg ttt gtt ttt ctt&#xA;#^^^ ^^^ ^^^ &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9172" LastEditorUserId="9172" LastEditDate="2020-07-23T13:16:06.813" LastActivityDate="2020-07-23T13:16:06.813" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13972" PostTypeId="2" ParentId="13481" CreationDate="2020-07-23T08:11:43.827" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Can i even compare these two methods?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;See if this paper helps:&#xA;&lt;a href=&quot;https://www.nature.com/articles/nature02917&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/nature02917&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9258" LastActivityDate="2020-07-23T08:11:43.827" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13973" PostTypeId="2" ParentId="997" CreationDate="2020-07-23T09:07:14.107" Score="3" Body="&lt;p&gt;Try MACSE v2 (&lt;a href=&quot;https://academic.oup.com/mbe/article/35/10/2582/5079334&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://academic.oup.com/mbe/article/35/10/2582/5079334&lt;/a&gt;) will align multiple protein-coding nucleotide sequences based on their amino acid translation while allowing for the occurrence of frameshifts&lt;/p&gt;&#xA;" OwnerUserId="9259" LastActivityDate="2020-07-23T09:07:14.107" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13974" PostTypeId="2" ParentId="6880" CreationDate="2020-07-23T13:12:40.710" Score="3" Body="&lt;p&gt;Unlike &lt;a href=&quot;https://en.wikipedia.org/wiki/Proportional_hazards_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia&lt;/a&gt;, the linear predictor (LP) in &lt;code&gt;coxph&lt;/code&gt; is made dependent from the baseline hazard function &lt;span class=&quot;math-container&quot;&gt;$\lambda$&lt;/span&gt;. For example, if a model has two predictors &lt;code&gt;x1&lt;/code&gt;, &lt;code&gt;x2&lt;/code&gt; stratified by &lt;code&gt;sex&lt;/code&gt;, the linear predictor would be like:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$LP = log (  \lambda _{sex}) + \beta _{1}x _{1}+\beta _{2}x _{2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where the &lt;span class=&quot;math-container&quot;&gt;$log(\lambda _{sex})$&lt;/span&gt; can be obtained by &lt;code&gt;predict()&lt;/code&gt; with &lt;code&gt;x1=x2=0&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;We could calculate the LP without strata information (sex) by hand using the &lt;span class=&quot;math-container&quot;&gt;$\beta _{1} \, \beta _{2}$&lt;/span&gt; provided by &lt;code&gt;coef(fit)&lt;/code&gt; and a customised &lt;span class=&quot;math-container&quot;&gt;$\lambda _{unkown \, sex}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Below is a simulation comparing the results from &lt;code&gt;predict()&lt;/code&gt; and by hand &lt;code&gt;coef(fit)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## Toy data&#xA;df &amp;lt;- list(time=c(4,3,1,1,2,2,3), &#xA;          status=c(1,1,1,0,1,1,0), &#xA;          x1=c(0,2,1,1,1,0,0), &#xA;          x2=1:7,&#xA;          sex=c(0,0,0,0,1,1,1)) &#xA;&#xA;## Fit a stratified model &#xA;fit &amp;lt;- coxph(Surv(time, status) ~ x1 + x2 + strata(sex), df) &#xA;coef(fit)&#xA;#         x1          x2 &#xA;# 0.79451881 -0.01917633&#xA;&#xA;## Baseline linear predictor&#xA;lambda_sex1 &amp;lt;- predict(fit, newdata=list(x1=0, x2=0, sex=0))&#xA;# -0.746578&#xA;lambda_sex2 &amp;lt;- predict(fit, newdata=list(x1=0, x2=0, sex=1))&#xA;# -0.1497816&#xA;&#xA;## by predict function&#xA;predict(fit, newdata=list(x1=1, x2=2, sex=0))&#xA;# 0.009588167&#xA;predict(fit, newdata=list(x1=1, x2=2, sex=1))&#xA;# 0.6063845&#xA;&#xA;## by hand&#xA;lambda_sex1 + coef(fit)[1]*1 + coef(fit)[2]*2&#xA;# 0.009588167&#xA;lambda_sex2 + coef(fit)[1]*1 + coef(fit)[2]*2&#xA;# 0.6063845&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Finally, to calculate &amp;quot;a&amp;quot; linear predictor for an unknown &lt;code&gt;sex&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# prevented by predict function&#xA;predict(fit, newdata=list(x1=1, x2=2, sex=2))&#xA;#Error in model.frame.default(data = list(x1 = 1, x2 = 2, sex = 2), formula = ~x1 +  : &#xA;# factor strata(sex) has new level sex=2&#xA;&#xA;# by hand, assuming lambda for sex 2 is 0&#xA;0 + coef(fit)[1]*1 + coef(fit)[2]*2&#xA;# 0.7561661&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9172" LastEditorUserId="9172" LastEditDate="2020-07-23T13:38:52.647" LastActivityDate="2020-07-23T13:38:52.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13975" PostTypeId="2" ParentId="13914" CreationDate="2020-07-23T13:35:08.460" Score="1" Body="&lt;p&gt;Make a list.txt file containing a single column of SRA numbers to download.&lt;/p&gt;&#xA;&lt;p&gt;then:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;for i in $(cat list.txt); do echo $i; date; fasterq-dump -S $i; done&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;It works well to use NCBI's web interface to find SRA samples of interest, download and open findings in Excel, then copy single column containing SRA numbers and paste into list.txt using document editor such as vim.&lt;/p&gt;&#xA;&lt;p&gt;After downloading including the &amp;quot;R&amp;quot; can be nice:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;for i in *_1.fastq; do mv $i ${i%_1.fastq}_R1.fastq; done&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;for i in *_2.fastq; do mv $i ${i%_2.fastq}_R1.fastq; done&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;and zip:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;pigz *fastq&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;If needed a conda option for downloading fasterq-dump:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;conda install -c bioconda sra-tools&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="6343" LastActivityDate="2020-07-23T13:35:08.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13976" PostTypeId="1" CreationDate="2020-07-23T14:44:04.213" Score="1" ViewCount="1982" Body="&lt;p&gt;I have RNAseq data from 4 samples with 3 biological replicates per sample. I am currently trying to do the differential expression analysis with DESeq2 but the biological replicates will not cluster together when I make the PCA plot or correlation heatmap. This is my first time with RNASeq analysis and so am not sure what the best route forward is? I would like to avoid repeating the experiment with new samples if possible!&lt;/p&gt;&#xA;&lt;p&gt;My pipeline prior to DESeq2 was the following:&lt;/p&gt;&#xA;&lt;p&gt;FastQC quality check -&amp;gt; Trimmomatic -&amp;gt; Kallisto&lt;/p&gt;&#xA;&lt;p&gt;I used tximport to convert kallisto files into suitable format for DESeq2&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/txw4G.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/txw4G.png&quot; alt=&quot;PCA plot with rlog transformed data&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9261" LastActivityDate="2022-09-12T18:04:30.427" Title="RNAseq biological replicates not clustering in PCA plots" Tags="|rna-seq|deseq2|clustering|pca|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13977" PostTypeId="1" CreationDate="2020-07-23T17:32:03.933" Score="0" ViewCount="52" Body="&lt;p&gt;I have a small scRNA-Seq dataset (n = 357, inhibitory neurons). This set of cells is split almost evenly between two conditions (Case and Control). I would like to test for differential expression with MAST [1]. However, with such a small number of cells and high heterogeneity of different inhibitory neuron cell types, I am able to get only a single gene as differentially expressed (False Discovery Rate 0.05, Log Fold Change 0.15).&lt;/p&gt;&#xA;&lt;p&gt;The best solution would be to enrich this subset of neurons in a new sample, however, are there other approaches that can be used to extract more information from this dataset? I'm not considering imputation at the moment since it seems to produce a large number of false positives [2].&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Finak, Greg, et al. &amp;quot;MAST: a flexible statistical framework for assessing transcriptional changes and characterizing heterogeneity in single-cell RNA sequencing data.&amp;quot; Genome biology 16.1 (2015): 1-13.&lt;/li&gt;&#xA;&lt;li&gt;Andrews, Tallulah S., and Martin Hemberg. &amp;quot;False signals induced by single-cell imputation.&amp;quot; F1000Research 7 (2018).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2020-07-23T17:32:03.933" Title="Alternatives to imputation for single-cell RNA-Seq analysis to increase statistical power of DE analysis" Tags="|scrnaseq|differential-expression|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="13978" PostTypeId="2" ParentId="13976" CreationDate="2020-07-23T18:37:57.283" Score="1" Body="&lt;p&gt;PC1 is 81% of the variance?&lt;/p&gt;&#xA;&lt;p&gt;This PCA plot confirms that different tissues are different.  You probably already knew that.  I'd make more PCA plots that are tissue specific.  That will be more informative than these.&lt;/p&gt;&#xA;&lt;p&gt;Personally, I'd also not do DESeq on all of these samples together, unless the goal of your experiment really is to learn the differences between these two tissues.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-07-23T18:37:57.283" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13981" PostTypeId="1" CreationDate="2020-07-24T02:30:27.630" Score="1" ViewCount="54" Body="&lt;p&gt;I have created a heat map using WGCNA using R. But the heat map command is not showing the plot and there is no error. I have a large number of modules and traits. I am not sure if that is the reason. But I tried making it in HPC R and it still is not showing any result plot. Thank you!&lt;/p&gt;&#xA;" OwnerUserId="8784" LastActivityDate="2020-07-27T20:55:19.243" Title="no result from heat map WGCNA" Tags="|r|hpc|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13982" PostTypeId="2" ParentId="13976" CreationDate="2020-07-24T03:02:05.267" Score="0" Body="&lt;p&gt;Clustering of replicates looks decent enough to me, so you should be abl to push ahead, but I agree the tissues are grouping, which could mask any differences based on sex or genotype.&lt;/p&gt;&#xA;&lt;p&gt;You might consider the EdgeR package for DE analysis here. It allows for flexibility when making complex comparisons while accounting for tissue/batch effects. I've had good luck with using it to compare across batch effects from complex experiments like this.&lt;/p&gt;&#xA;" OwnerUserId="8685" LastActivityDate="2020-07-24T03:02:05.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13984" PostTypeId="1" AcceptedAnswerId="14027" CreationDate="2020-07-24T10:38:47.743" Score="0" ViewCount="101" Body="&lt;p&gt;I'm trying to use this &lt;a href=&quot;https://ropensci.org/tutorials/europepmc_tutorial/&quot; rel=&quot;nofollow noreferrer&quot;&gt;europmc r libray&lt;/a&gt; where I have a list of pmids to look for. I tried with pubtator but its bit complicated.In Europmc i can all the annotated terms etc.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(&amp;quot;europepmc&amp;quot;)&#xA;&#xA;example list of PMIDS&#xA;30024784&#xA;30555165&#xA;30510081&#xA;31688884&#xA;31516032&#xA;28588019&#xA;29286103&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now what I'm doing is looking each ID using epmc_details function which is not the way i would do if i have to look for hundreds.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;epmc_details(ext_id = '30510081')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have to question how can I run the epmc_details in a loop or some other way where i can look for PMIDS one by one and save the result in a data frame.&lt;/p&gt;&#xA;&lt;p&gt;The epmc_details is returned as a list. The structure of the list is as such&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[1] &amp;quot;basic&amp;quot;           &amp;quot;author_details&amp;quot;  &amp;quot;journal_info&amp;quot;    &amp;quot;ftx&amp;quot;             &amp;quot;chemical&amp;quot;        &amp;quot;mesh_topic&amp;quot;      &amp;quot;mesh_qualifiers&amp;quot;&#xA;[8] &amp;quot;comments&amp;quot;        &amp;quot;grants&amp;quot;    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would only like to save basic,chemical,mesh_topic,mesh_qualifiers in one data frame.&lt;/p&gt;&#xA;&lt;p&gt;For example if my first id is this &lt;code&gt;30510081&lt;/code&gt; the dataframe should have first column as my ID which is basically &lt;code&gt;basic[1]&lt;/code&gt;and rest of the information appended to the next columns.&lt;/p&gt;&#xA;&lt;p&gt;such as&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; ID chemical mesh_qualifiers mesh_topic gene &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any suggestion or help would be really and highly appreciated&lt;/p&gt;&#xA;&lt;p&gt;I was looking at europmc site through the browser&lt;/p&gt;&#xA;&lt;p&gt;this was one of my &lt;a href=&quot;https://europepmc.org/article/MED/32700753&quot; rel=&quot;nofollow noreferrer&quot;&gt;query&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;when i highlight the keyterms i do see in the abstract itself all the keyterms are getting annotated but when i do the same query search through R I do see empty results as such why there is a difference?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$chemical&#xA;# A tibble: 0 x 0&#xA;&#xA;$mesh_topic&#xA;# A tibble: 0 x 0&#xA;&#xA;$mesh_qualifiers&#xA;# A tibble: 0 x 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I found better way of getting data from pubmed using the &lt;a href=&quot;https://docs.ropensci.org/tidypmc/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tidypmc&lt;/a&gt; library.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(tidypmc)&#xA;doc &amp;lt;- pmc_xml(&amp;quot;PMC6365492&amp;quot;)&#xA;doc&#xA;txt &amp;lt;- pmc_text(doc)&#xA;txt&#xA;&#xA;count(txt, &amp;quot;section&amp;quot;)&#xA;&#xA;cap1 &amp;lt;- pmc_caption(doc)&#xA;filter(cap1, sentence == 1)&#xA;&#xA;tab1 &amp;lt;- pmc_table(doc)&#xA;&#xA;sapply(tab1, nrow)&#xA;&#xA;tab1[[1]]&#xA;attributes(tab1[[2]])&#xA;&#xA;collapse_rows(tab1, na.string=&amp;quot;-&amp;quot;)&#xA;&#xA;library(tibble)&#xA;x &amp;lt;- xml_name(xml_find_all(doc, &amp;quot;//*&amp;quot;))&#xA;tibble(tag=x) %&amp;gt;% count(&amp;quot;tag&amp;quot;)&#xA;&#xA;library(tidytext)&#xA;x1 &amp;lt;- unnest_tokens(txt, word, text) %&amp;gt;%&#xA;  anti_join(stop_words) %&amp;gt;%&#xA;  filter(!word %in% 1:100)&#xA;#  Joining, by = &amp;quot;word&amp;quot;&#xA;#filter(x1, str_detect(section, &amp;quot;Case description&amp;quot;)) &#xA;&#xA;&#xA;filter(x1, str_detect(section, &amp;quot;Results&amp;quot;))&#xA;count(a$word)&#xA;&#xA;tbls &amp;lt;- pmc_table(doc)&#xA;map_int(tbls, nrow)&#xA;&#xA;tbls[[1]]&#xA;&#xA;collapse_rows(tbls, na.string=&amp;quot;-&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But if i understand it can use one PMC id at a time. Again keeping my original question how can i put this in a loop to query lets say i have 100 PMCID and get it result and store in a dataframe.&lt;/p&gt;&#xA;&lt;p&gt;After using tidypmc i found that i parse all the publication based on attributes or tags. Such as title,abstract, results etc etc.&lt;/p&gt;&#xA;&lt;p&gt;Lets say I'm interested in the table tags information where they have metadata of patients as well as others.So if a paper contain multiple tables I would like to store each of them in a data-frame under the respective publication.Since I have multiple IDs to search and do the save as above mentioned. How to put this through a loop or can it be done without using loop ?&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion or help would be really appreciated as always.&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2020-07-27T10:55:18.093" LastActivityDate="2020-07-27T15:23:35.993" Title="europmc and tidypmc R library for extracting or making metadata from publication" Tags="|r|public-databases|parsing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="13985" PostTypeId="2" ParentId="13935" CreationDate="2020-07-24T14:32:07.450" Score="0" Body="&lt;p&gt;Just want to add that apart from params$char.width, we can also use&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params&lt;span class=&quot;math-container&quot;&gt;$text.size&#xA;params$&lt;/span&gt;text.size &amp;lt;- 0.2 #default 0.4&#xA;params$base.per.unit &amp;lt; 50000 #imp #default value = 30000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3766" LastActivityDate="2020-07-24T14:32:07.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13986" PostTypeId="1" AcceptedAnswerId="14080" CreationDate="2020-07-24T14:48:49.043" Score="1" ViewCount="68" Body="&lt;p&gt;I have a dataset of genes I am trying to collect data on from public databases, to use as features in machine learning. I am trying to take some features from UCSC genome browser (e.g. number of CpG islands per gene, number of DNase clusters per gene, regulatory enrichment scores etc.) however I am not sure how to control for bias where a gene that is larger in length - and so will then have more CpG islands or higher regulatory enrichment scores simply due to gene length.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to correct for gene length when taking/condensing variant data to individual genes?&lt;/p&gt;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&lt;p&gt;For reference, my machine learning model aims to predict whether a gene is the most likely to be causal for a disease (out of all the genes given to the model). The model will score the genes as a regression classification between 0 to 1 (0 being least likely to cause disease and 1 being most likely to cause disease). I plan to later further investigate the genes with the highest scores.&lt;/p&gt;&#xA;&lt;p&gt;The model uses a variety of multi-omic features (e.g. GTEx gene expression the genes have for many tissues, GWAScatalog data, gene intolerance scores, protein-protein interaction data, drug interaction data, phenotypic scores etc.). However, I am missing epigenetic data to describe my genes so I've been looking to collect based on UCSC's variant data (CpG islands, histone modifications, DNase clusters) - however this leads to my gene length problem when I am trying to reliably take data from the variant level.&lt;/p&gt;&#xA;&lt;p&gt;I've been plotting my features and gene length, and seen that the UCSC epigenetic data does correlate with having a larger gene length if there is a higher count of regulatory sites (0.8 r2 for some), and so this is what I'm looking to correct.&lt;/p&gt;&#xA;" OwnerUserId="4242" LastEditorUserId="4242" LastEditDate="2020-08-04T13:08:13.987" LastActivityDate="2020-08-04T13:08:23.503" Title="How to compact variant data to their genes without bias?" Tags="|gene|statistics|public-databases|genomics|variants|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="13987" PostTypeId="1" CreationDate="2020-07-24T14:53:39.117" Score="0" ViewCount="977" Body="&lt;p&gt;I am trying to download DESeq2 in R 3.6.3. Is it possible?&#xA;This is printed when I am trying: Warning in install.packages : package ‘DESeq2’ is not available (for R version 3.6.3)&#xA;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="8713" LastActivityDate="2020-07-24T15:52:43.073" Title="can I download DESeq2 in R 3.6.3 in Linux MInt?" Tags="|r|rna-seq|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13988" PostTypeId="2" ParentId="13981" CreationDate="2020-07-24T15:16:38.957" Score="1" Body="&lt;p&gt;The plot is probably being drawn to a device that you cannot view on a cluster. If you're using the &lt;code&gt;plotNetworkHeatmap&lt;/code&gt; function, say, try something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;png(&amp;quot;plot.png&amp;quot;, ...) #specify height, width, etc&#xA;plotNetworkHeatmap(....) #call your function&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, &lt;code&gt;scp&lt;/code&gt; the &lt;code&gt;plot.png&lt;/code&gt; file over to your local machine to view the plot.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2020-07-24T15:16:38.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13989" PostTypeId="2" ParentId="13987" CreationDate="2020-07-24T15:40:56.900" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://bioconductor.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioconductor&lt;/a&gt; is a package management tool for R that allows you to install the appropriate package versions for your R version.&lt;/p&gt;&#xA;&lt;p&gt;For R 3.6.3 you'll need Bioconductor 3.10:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;if (!requireNamespace(&amp;quot;BiocManager&amp;quot;, quietly = TRUE))&#xA;    install.packages(&amp;quot;BiocManager&amp;quot;)&#xA;BiocManager::install(version = &amp;quot;3.10&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then you install  DESeq2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BiocManager::install(&amp;quot;DESeq2&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2020-07-24T15:40:56.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13990" PostTypeId="2" ParentId="13987" CreationDate="2020-07-24T15:52:43.073" Score="0" Body="&lt;p&gt;these errors are generated when I am doing what PPK suggested:&#xA;Bioconductor version 3.10 (BiocManager 1.30.10), R 3.6.3&#xA;(2020-02-29)&#xA;Installation path not writeable, unable to update packages: nlme,&#xA;spatial, survival&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;BiocManager::install(&amp;quot;DESeq2&amp;quot;)&#xA;Bioconductor version 3.10 (BiocManager 1.30.10), R 3.6.3&#xA;(2020-02-29)&#xA;Installing package(s) 'DESeq2'&#xA;Warning: dependency ‘XML’ is not available&#xA;also installing the dependencies ‘annotate’, ‘genefilter’, ‘geneplotter’&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h1&gt;trying URL 'https://bioconductor.org/packages/3.10/bioc/src/contrib/annotate_1.64.0.tar.gz'&#xA;Content type 'application/x-gzip' length 1873132 bytes (1.8 MB)&lt;/h1&gt;&#xA;&lt;p&gt;downloaded 1.8 MB&lt;/p&gt;&#xA;&lt;h1&gt;trying URL 'https://bioconductor.org/packages/3.10/bioc/src/contrib/genefilter_1.68.0.tar.gz'&#xA;Content type 'application/x-gzip' length 1421076 bytes (1.4 MB)&lt;/h1&gt;&#xA;&lt;p&gt;downloaded 1.4 MB&lt;/p&gt;&#xA;&lt;h1&gt;trying URL 'https://bioconductor.org/packages/3.10/bioc/src/contrib/geneplotter_1.64.0.tar.gz'&#xA;Content type 'application/x-gzip' length 1431450 bytes (1.4 MB)&lt;/h1&gt;&#xA;&lt;p&gt;downloaded 1.4 MB&lt;/p&gt;&#xA;&lt;h1&gt;trying URL 'https://bioconductor.org/packages/3.10/bioc/src/contrib/DESeq2_1.26.0.tar.gz'&#xA;Content type 'application/x-gzip' length 2060061 bytes (2.0 MB)&lt;/h1&gt;&#xA;&lt;p&gt;downloaded 2.0 MB&lt;/p&gt;&#xA;&lt;p&gt;ERROR: dependency ‘XML’ is not available for package ‘annotate’&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;removing ‘/home/nikolena/R/x86_64-pc-linux-gnu-library/3.6/annotate’&#xA;ERROR: dependency ‘annotate’ is not available for package ‘genefilter’&lt;/li&gt;&#xA;&lt;li&gt;removing ‘/home/nikolena/R/x86_64-pc-linux-gnu-library/3.6/genefilter’&#xA;ERROR: dependency ‘annotate’ is not available for package ‘geneplotter’&lt;/li&gt;&#xA;&lt;li&gt;removing ‘/home/nikolena/R/x86_64-pc-linux-gnu-library/3.6/geneplotter’&#xA;ERROR: dependencies ‘genefilter’, ‘geneplotter’ are not available for package ‘DESeq2’&lt;/li&gt;&#xA;&lt;li&gt;removing ‘/home/nikolena/R/x86_64-pc-linux-gnu-library/3.6/DESeq2’&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The downloaded source packages are in&#xA;‘/tmp/RtmphLXrZb/downloaded_packages’&#xA;Installation path not writeable, unable to update packages: nlme,&#xA;spatial, survival&#xA;Warning messages:&#xA;1: In install.packages(...) :&#xA;installation of package ‘annotate’ had non-zero exit status&#xA;2: In install.packages(...) :&#xA;installation of package ‘genefilter’ had non-zero exit status&#xA;3: In install.packages(...) :&#xA;installation of package ‘geneplotter’ had non-zero exit status&#xA;4: In install.packages(...) :&#xA;installation of package ‘DESeq2’ had non-zero exit status&lt;/p&gt;&#xA;&lt;p&gt;I have tried to instal XML and the other dependencies without success..&lt;/p&gt;&#xA;" OwnerUserId="8713" LastActivityDate="2020-07-24T15:52:43.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13993" PostTypeId="1" CreationDate="2020-07-24T16:58:19.067" Score="2" ViewCount="19062" Body="&lt;p&gt;Im trying to change the active.ident to another column in metadata but this error keeps popping up! I recently upgraded to R version 4.0.2 from 3.6.1 The older version was working but the new one isn't. I don't understand how to fix it?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;M@active.ident &amp;lt;- M@meta.data$status&#xA;Error in (function (cl, name, valueClass)  : &#xA;assignment of an object of class “character” is not valid for @‘active.ident’ in an object of class &#xA;“Seurat”; is(value, &amp;quot;factor&amp;quot;) is not TRUE&#xA;&#xA;head(M@meta.data) &#xA;        orig.ident nCount_RNA nFeature_RNA percent.mito   status  RNA_snn_res.0.6 seurat_clusters &#xA;BC03_03       BC03   998414.7         3356   0.07869098 nonTumor                0               0&#xA;BC03_09       BC03   999853.6         7616   0.12964336  primary                0               0&#xA;BC03_13       BC03   998206.5         8457   0.17672205  primary                0               0&#xA;BC03_17       BC03   999535.1         7494   0.11323008  primary                0               0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to run the UMAP with the clusters named according to the status and not seurat clusters.However, the code isnt working..&lt;/p&gt;&#xA;" OwnerUserId="9234" LastActivityDate="2020-07-24T17:42:29.613" Title="Changing active.ident in Seurat" Tags="|r|scrnaseq|seurat|sequence-annotation|clustering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13995" PostTypeId="2" ParentId="13993" CreationDate="2020-07-24T17:42:29.613" Score="2" Body="&lt;p&gt;&lt;code&gt;M &amp;lt;- SetIdent(M, value = &amp;quot;status&amp;quot;)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;or more explicitly&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;M &amp;lt;- SetIdent(M, value = M@meta.data$status)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can also use the &lt;code&gt;group.by&lt;/code&gt; argument of &lt;code&gt;UMAPPlot()&lt;/code&gt; or other plotting functions from &lt;code&gt;Seurat&lt;/code&gt; for that matter.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-07-24T17:42:29.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13996" PostTypeId="2" ParentId="13370" CreationDate="2020-07-24T18:33:03.087" Score="1" Body="&lt;p&gt;Ok I have figured it out and I am sharing this in case anyone can get help. I used&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sce &amp;lt;-addPerCellQC(sce)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which calculated the QC for all the altexps also and adds them to the colData.&lt;/p&gt;&#xA;" OwnerUserId="9047" LastActivityDate="2020-07-24T18:33:03.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13997" PostTypeId="1" AcceptedAnswerId="13999" CreationDate="2020-07-25T00:53:44.903" Score="4" ViewCount="8041" Body="&lt;p&gt;I am working with short-read whole-genome sequences from the NCBI's SRA. I have aligned and sorted all of my short-read sequences and am attempting to index each sequence into .bai format using samtools index, but am running into a couple of errors.&lt;/p&gt;&#xA;&lt;p&gt;I unpacked the original .sra files in the following manner:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fastq-dump --defline-seq '@&lt;span class=&quot;math-container&quot;&gt;$sn[_$&lt;/span&gt;rn]/$ri' --split-files SRR6509138.sra&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then aligned each fastq paired-end read, removed duplicate reads, and converted each to bam format like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bwa mem xiphophorus_birchmanni_10x_12Sep2018_yDAA6.fasta SRR6509136_1.fastq SRR6509136_2.fastq | samblaster -e -r| samtools view -Sb - &amp;gt; blasted_SRR6509136.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Some of my files did not come as paired-end reads. For those files, I used the --ignoreUnmated flag on samblaster.&lt;/p&gt;&#xA;&lt;p&gt;I then sorted each bam file like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools sort blasted_SRR6649368.bam -o sorted_SRR6649368.bam -n&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am attempting to obtain a bai file for each bam using the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools index sorted_SRR6649368.bam sorted_SRR6649368.bam.bai&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the error I run into for the unpaired reads:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[E::hts_idx_push] Chromosome blocks not continuous&#xA;[E::sam_index] Read 'HWI-ST387:164:D0CJWACXX:4:1101:1129:13264' with ref_name='ScdB1pO_646;HRSCAF=880', ref_length=33092979, flags=16, pos=23260108 cannot be indexed&#xA;samtools index: failed to create index for &amp;quot;sorted_SRR791885.bam&amp;quot;: No such file or directory&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the error I run into for the paired reads:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[E::hts_idx_push] Unsorted positions on sequence #79: 11159020 followed by 11158717&#xA;samtools index: failed to create index for &amp;quot;sorted_SRR6649368.bam&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can anyone help me figure out why this is happening?&lt;/p&gt;&#xA;" OwnerUserId="9118" LastActivityDate="2020-07-25T10:06:24.230" Title="Samtools Index: Chromosome Blocks not Continuous" Tags="|genome|samtools|bwa|indexing|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="13998" PostTypeId="2" ParentId="13970" CreationDate="2020-07-25T07:35:38.823" Score="1" Body="&lt;p&gt;Turns out that I was dumb: I misread.&lt;/p&gt;&#xA;&lt;p&gt;The Clustal Omega paper (in the supplementary materials) chose &lt;span class=&quot;math-container&quot;&gt;$t = \log N$&lt;/span&gt;, which I think originates from this claim from this part of the Linial et al. paper:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In random polynomial time &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; may be embedded in &lt;span class=&quot;math-container&quot;&gt;$l_{2}^{O(\log n)}$&lt;/span&gt; with distortion &lt;span class=&quot;math-container&quot;&gt;$\leq(1+\epsilon) \cdot c_{2}(X)$&lt;/span&gt; (for any &lt;span class=&quot;math-container&quot;&gt;$\epsilon&amp;gt;0)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Several questions still remain though:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The newest implementation of Clustal Omega (that I downloaded) has source code that instead chooses &lt;span class=&quot;math-container&quot;&gt;$t = (\log N)^2$&lt;/span&gt;, why changed?&lt;/li&gt;&#xA;&lt;li&gt;The original mBED paper chooses &lt;span class=&quot;math-container&quot;&gt;$t = (\log N)^2$&lt;/span&gt;, but it was not justified on how they chose this number instead of &lt;span class=&quot;math-container&quot;&gt;$\log N$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9257" LastEditorUserId="9257" LastEditDate="2020-07-25T08:22:30.147" LastActivityDate="2020-07-25T08:22:30.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="13999" PostTypeId="2" ParentId="13997" CreationDate="2020-07-25T10:06:24.230" Score="4" Body="&lt;pre&gt;&lt;code&gt;samtools sort blasted_SRR6649368.bam -o sorted_SRR6649368.bam -n&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These error messages indicate that the reads are not sorted by coordinate — in particular, that the reads mapped to &lt;em&gt;ScdB1pO_646;HRSCAF=880&lt;/em&gt; are not all together, and that the reads at positions 11159020 &amp;lt; 11158717 are not sorted by position on their chromosome.&lt;/p&gt;&#xA;&lt;p&gt;This is because &lt;code&gt;samtools sort -n&lt;/code&gt; has been used to sort the reads by name instead. Remove &lt;code&gt;-n&lt;/code&gt; to sort by position, which is what is needed to prepare a BAM file for indexing with &lt;code&gt;samtools index&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="134" LastActivityDate="2020-07-25T10:06:24.230" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14003" PostTypeId="1" CreationDate="2020-07-25T21:36:33.947" Score="1" ViewCount="91" Body="&lt;p&gt;I want to implement SEG in my MATLAB environment and I'm relying on Wootton and Federhen (&lt;a href=&quot;https://www.sciencedirect.com/science/article/abs/pii/009784859385006X&quot; rel=&quot;nofollow noreferrer&quot;&gt;1993&lt;/a&gt;)&#xA;Reading the article I cannot succeed to understand what kind of process I have to implement in order to build my script. Specifically what kind of statistics I should follow to compute the complexity state and the repetition vector and what kind of procedure I have to do to calculate these statistics over the amino acid sequence.&lt;/p&gt;&#xA;&lt;p&gt;Do you know some articles that explains in a better fashion what is needed, namely the mathematical process behind the results shown in the article? Code examples in other languages welcome.&#xA;Thank you in advance&lt;/p&gt;&#xA;" OwnerUserId="9142" LastEditorUserId="6322" LastEditDate="2020-07-29T12:38:26.183" LastActivityDate="2020-07-29T12:38:26.183" Title="How to implement SEG (Wootton and Federhen ,1993) algorithm?" Tags="|r|python|sequence-analysis|protein-structure|matlab|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14005" PostTypeId="2" ParentId="6873" CreationDate="2020-07-25T23:02:30.267" Score="0" Body="&lt;p&gt;genometools-genometools worked for me, while 2 other options (genometools and -c leylabmpi genometools) did not.&lt;/p&gt;&#xA;" OwnerUserId="9272" LastActivityDate="2020-07-25T23:02:30.267" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14006" PostTypeId="1" CreationDate="2020-07-26T02:18:00.683" Score="0" ViewCount="50" Body="&lt;p&gt;I am a computer scientist just starting out in bioinformatics topics, and would appreciate any guidance that can be given here:&lt;/p&gt;&#xA;&lt;p&gt;I have an &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NM_000551.4&quot; rel=&quot;nofollow noreferrer&quot;&gt;mRNA sequence&lt;/a&gt;- an isoform - whose length is about 4000 base pairs (bp) - the origin is provided in the link.&lt;/p&gt;&#xA;&lt;p&gt;I also have a &lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR10377489&quot; rel=&quot;nofollow noreferrer&quot;&gt;bam file/fasta file&lt;/a&gt; of the entire transcriptome of a kidney cell (see Data Access tab).&lt;/p&gt;&#xA;&lt;p&gt;My objective is to quantify mRNA expression levels of the mRNA sequence in the transcriptome.&lt;/p&gt;&#xA;&lt;p&gt;How does one get started? Should I use the original format (TenX bam, about 10 GB) or SRA archive data (fasta, about 3.5 GB)?  Why are the files so different in size?&lt;/p&gt;&#xA;&lt;p&gt;I have a MacBook with &lt;code&gt;samtools&lt;/code&gt; installed and 16GB of RAM. Is this enough to process the data and what basic commands should be used to quantify the 4000 bp sequence overlaps in the transcriptome?&lt;/p&gt;&#xA;" OwnerUserId="9271" LastEditorUserId="9271" LastEditDate="2020-07-26T15:09:14.633" LastActivityDate="2020-07-26T15:09:14.633" Title="Basic question about finding mRNA sequence in transcriptome" Tags="|sam|fasta|samtools|transcriptome|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14007" PostTypeId="2" ParentId="14006" CreationDate="2020-07-26T05:44:20.087" Score="2" Body="&lt;p&gt;Basically you want to know the VHL Gene expression in that 10x dataset. They have provided the barcode, matrix and feature files on GEO &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE131685&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSE131685&lt;/a&gt; as Supplementary file. You could use those files and get the gene expression matrix as described on &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/output/matrices&quot; rel=&quot;nofollow noreferrer&quot;&gt;10x website&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(Matrix)&#xA;&#xA;matrix_dir = &amp;quot;/opt/sample345/outs/filtered_feature_bc_matrix/&amp;quot;&#xA;barcode.path &amp;lt;- paste0(matrix_dir, &amp;quot;barcodes.tsv.gz&amp;quot;)&#xA;features.path &amp;lt;- paste0(matrix_dir, &amp;quot;features.tsv.gz&amp;quot;)&#xA;matrix.path &amp;lt;- paste0(matrix_dir, &amp;quot;matrix.mtx.gz&amp;quot;)&#xA;&#xA;mat &amp;lt;- readMM(file = matrix.path)&#xA;&#xA;feature.names = read.delim(features.path, &#xA;                           header = FALSE,&#xA;                           stringsAsFactors = FALSE)&#xA;&#xA;barcode.names = read.delim(barcode.path, &#xA;                           header = FALSE,&#xA;                           stringsAsFactors = FALSE)&#xA;&#xA;colnames(mat) = barcode.names&lt;span class=&quot;math-container&quot;&gt;$V1&#xA;rownames(mat) = feature.names$&lt;/span&gt;V1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This mat can be further loaded into &lt;a href=&quot;https://satijalab.org/seurat/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seurat&lt;/a&gt; for any downstream analysis. All this can be done on a laptop.&lt;/p&gt;&#xA;&lt;p&gt;If you want to re-quantify the gene expression for whatever the reason, you need to re-run the cellRanger pipeline using GTF of your interest (it needs some &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/tutorial_mr&quot; rel=&quot;nofollow noreferrer&quot;&gt;preparation&lt;/a&gt;), which may not work on your laptop. It requires large memory and long running times, ideally to submit as job on cluster.&lt;/p&gt;&#xA;" OwnerUserId="4017" LastActivityDate="2020-07-26T05:44:20.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14008" PostTypeId="2" ParentId="14006" CreationDate="2020-07-26T05:51:28.640" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Should I use the original format (TenX bam, about 10 GB) or SRA&#xA;archive data (fasta, about 3.5 GB)? Why are the files so different in&#xA;size?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I strongly recommend you hold your horses; how do you think you can analyze something if you don't even know what you are looking at?&lt;/p&gt;&#xA;&lt;p&gt;Since this is a &amp;quot;TenX&amp;quot; bam, you can probably do the job with samtools and grep.  Looking up 10XGenomics bam tags might be helpful.  You might need to look up SAM flags too.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;My objective is to quantify mRNA expression levels of the mRNA&#xA;sequence in the transcriptome.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You understand that this isn't really a thing that's possible?  With spike-ins, maybe, but you don't have that.  All you can do is work out the proportions of reads belonging to that transcript  (Or gene, really, unless you want to put in a great deal more work, beyond I believe what 10XGenomics does).  Which I guess you want to do by lumping all the cells together, and just pretending there is no QC to do?&lt;/p&gt;&#xA;" OwnerUserId="2388" LastEditorUserId="2388" LastEditDate="2020-07-26T06:03:23.530" LastActivityDate="2020-07-26T06:03:23.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14010" PostTypeId="1" CreationDate="2020-07-26T08:26:32.520" Score="4" ViewCount="800" Body="&lt;p&gt;I'm trying to determine what the difference and benefits of genome assembly and genome sequence alignments are when trying to identify structural variants or transposons in populations.&#xA;I've been scouring the internet but have only really come across the difference between short vs long reads and de novo assembly vs reference-based.&lt;/p&gt;&#xA;&lt;p&gt;My understanding is that to identify variations in structural variants within a population there seem to be 2 main comparative genomic methods, the first being what the 1KGP and SDGP did and sequence the whole genome, align the reads to the reference genome and end up with a BAM file.&lt;/p&gt;&#xA;&lt;p&gt;The second is to assemble personal genomes and then compare or align the assemblies to each other and the reference genome or using the Lastz/LiftOver/ChainNets Examples: 10.1016/j.gene.2005.09.031&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="9273" LastActivityDate="2020-07-26T14:39:52.850" Title="Difference between genome assembly and genome sequence alignment to a reference to find structural variants" Tags="|sequence-alignment|assembly|structural-variation|transposable-elements|1000genomes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14012" PostTypeId="1" AcceptedAnswerId="14013" CreationDate="2020-07-26T12:57:21.007" Score="1" ViewCount="1015" Body="&lt;p&gt;I have a fasta file containing the sequence of a gene across different species. In total there are around 900 samples and 12 species. (Each sequences is over multiple lines and longer than 100bp.)&lt;/p&gt;&#xA;&lt;p&gt;My fasta file looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Species-1-samplenameA&#xA;CTATCCTTAAACGCATATCTCGCACAGTAACTCCCCAATATGTGAGCATCTGATGTTGCCCGGGCCGAGTTAGTCTTGTGCTCACGGAACTTATTGTATG&#xA;&amp;gt;Species-2-samplenameB&#xA;AGTAGTGATTTGAAAGAGTTGTCAGTTAGCTCGTTCAGGTAATGGTTCCTCACACTACGTCAAAATAAGAGAGCGGTCGTGACATTATCCGTGATTTTCT&#xA;&amp;gt;Species-3-samplenameC&#xA;CACTACTATCAGTACTCACGACTCGATTCTGCCGCAGCCACGTATCGCCAGAAAGCCAGTCAGCATTAAGGAGTGCTCTGGGCAGGACAACTCGCATAGT&#xA;&amp;gt;Species-3-samplenameD&#xA;GAGAGTTACATGTTCGTTGGGCTCTTCCGACACGAACCTCAGTTGGCCTACATCCTACCTGAGGTCTGTGCCCCGGTGGTGAGAAGTGCGCATTTCGTTC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to split this file in one fasta file per species.&#xA;I think it’s possible to use the awk function for this but I’m stuck. Does anyone have a script/code that might help me?&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot.&lt;/p&gt;&#xA;" OwnerUserId="9275" LastEditorUserId="9275" LastEditDate="2020-07-26T14:26:47.830" LastActivityDate="2020-07-26T14:26:47.830" Title="Split fasta file based on groups in header information and output as separate files" Tags="|python|fasta|bash|awk|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14013" PostTypeId="2" ParentId="14012" CreationDate="2020-07-26T14:24:15.787" Score="3" Body="&lt;p&gt;A simple Biopython solution- iterate over the sequence records, identify the species, open the file handle using append mode to ensure no data is overwritten, and write the record:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;for record in SeqIO.parse(&amp;quot;myfile.fa&amp;quot;, &amp;quot;fasta&amp;quot;):&#xA;    species = record.id.split('_')[0]&#xA;    with open(f&amp;quot;{species}.fa&amp;quot;, &amp;quot;a&amp;quot;) as f:&#xA;        SeqIO.write(record, f, &amp;quot;fasta&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastActivityDate="2020-07-26T14:24:15.787" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14014" PostTypeId="2" ParentId="14010" CreationDate="2020-07-26T14:39:52.850" Score="4" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;the first being what the 1KGP and SDGP did and sequence the whole&#xA;genome, align the reads to the reference genome and end up with a BAM&#xA;file.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If you have a well defined reference genome (e.g. human, mouse etc.) and you are interested in &lt;em&gt;population level&lt;/em&gt; genetic variation, then this is the main approach. If you sequence a new human genome in the classical way (i.e. short reads ~30X coverage etc.), &lt;em&gt;de novo&lt;/em&gt; assembly is pointless generally and you can more rapidly map reads to the reference. The read mapping approach has the large advantage that when you get the to variant calling stage, you can use information about both the depth of sequencing and the base quality scores.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The second is to assemble personal genomes and then compare or align&#xA;the assemblies to each other and the reference genome or using the&#xA;Lastz/LiftOver/ChainNets&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is the traditional comparative genomics approach used for non-model organisms and for conducting evolutionary rate comparisons at the &lt;em&gt;cross-species level&lt;/em&gt;. You discard information on sequencing depth and quality scores, but you can compare many species' genomes at once genome-wide. You also worry less about sequencing errors etc. because you are not looking for relatively rare population level variation, but instead more common (depending on your species) variation between species.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2020-07-26T14:39:52.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14015" PostTypeId="1" AcceptedAnswerId="14030" CreationDate="2020-07-26T16:21:17.277" Score="3" ViewCount="810" Body="&lt;p&gt;I am new to working with the NCBI database so I am not sure how trivial this question is. I wanted to get DNA sequence of a specific version of the Xenopus tropicalis genome of a specific region (ie 1-100bp) without doing it manually each time (going on genome browser). I know there is a faster way and efficient way to do this through the command line but I found the NCBI tutorial hard to follow.&lt;/p&gt;&#xA;" OwnerUserId="9092" LastActivityDate="2020-07-27T20:08:33.387" Title="How to retrieve the DNA sequence from a particular species and region through command line?" Tags="|phylogenetics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14016" PostTypeId="2" ParentId="14015" CreationDate="2020-07-26T17:27:28.560" Score="-2" Body="&lt;p&gt;I'd download the fasta, and use samtools, or maybe bedtools.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-07-26T17:27:28.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14019" PostTypeId="1" CreationDate="2020-07-27T08:30:36.550" Score="1" ViewCount="484" Body="&lt;p&gt;I am working with profiled metagenomic taxonomy abundance data. I want to generate beta diversity (bray-curtis) boxplot from a phyloseq object where two groups (control and test) will be shown. Something like this figure:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pgauk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pgauk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;How can I make it? Can anyone share any code or tutorial?&lt;/p&gt;&#xA;&lt;p&gt;Thanks,&#xA;dpc&lt;/p&gt;&#xA;" OwnerUserId="7427" LastActivityDate="2021-06-15T15:20:12.897" Title="How to generate Beta diversity boxplot from phyloseq object?" Tags="|statistics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14020" PostTypeId="1" AcceptedAnswerId="14026" CreationDate="2020-07-27T10:01:38.243" Score="0" ViewCount="1786" Body="&lt;p&gt;I can't seem to find HGDP data in VCF format.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know where I can download it from?&lt;/p&gt;&#xA;&lt;p&gt;Perhaps there a Snakemake pipeline for calling VCFs from this data?&lt;/p&gt;&#xA;" OwnerUserId="8875" LastActivityDate="2022-03-15T10:18:17.410" Title="Download or create VCF from Human Genome Diversity Project (HGDP)" Tags="|vcf|variant-calling|data-download|human-genome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14021" PostTypeId="2" ParentId="11011" CreationDate="2020-07-27T10:03:45.767" Score="0" Body="&lt;p&gt;You can download HGDP data in FASTQ format here: &lt;a href=&quot;https://www.internationalgenome.org/data-portal/data-collection/hgdp&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.internationalgenome.org/data-portal/data-collection/hgdp&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8875" LastActivityDate="2020-07-27T10:03:45.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14022" PostTypeId="1" AcceptedAnswerId="14023" CreationDate="2020-07-27T11:18:26.013" Score="1" ViewCount="1574" Body="&lt;p&gt;How can I convert a VCF file into a genotype table (SNP matrix)?&lt;/p&gt;&#xA;&lt;p&gt;I have this format:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.0&#xA;##fileDate=20200722&#xA;##source=test&#xA;##reference=test&#xA;##FILTER=&amp;lt;ID=PASS,Description=&amp;quot;All filters passed&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&amp;quot;Genotype&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=DP,Number=1,Type=Integer,Description=&amp;quot;Read Depth&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=DV,Number=1,Type=Integer,Description=&amp;quot;Number of high-quality non-reference bases&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GQ,Number=1,Type=Integer,Description=&amp;quot;Genotype Quality&amp;quot;&amp;gt;&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  HOR_997 HOR_1020    HOR_1251    HOR_2970    HOR_12367   HOR_473 HOR_8148    HOR_17169   HOR_21258   HOR_4606    HOR_15922   HOR_20226   HOR_9724    HOR_10742   HOR_1071    HOR_10702   HOR_18382   HOR_8819    HOR_9558    HOR_7428    HOR_3023    HOR_18300   HOR_13924   HOR_1080    HOR_4030    HOR_6962    HOR_3460    HOR_8117    HOR_337 HOR_16112   HOR_1626    HOR_8659    HOR_16569   HOR_11083   HOR_18087   HOR_4273    HOR_2237    HOR_8440    HOR_7056    HOR_869 HOR_13987   HOR_12184   HOR_13800   HOR_9808    HOR_10192   HOR_4264    HOR_12830   HOR_10764   HOR_12171   HOR_10784   HOR_610 HOR_11875   HOR_2637    HOR_4136    HOR_282 HOR_10725   HOR_17649   HOR_1384    HOR_2598    HOR_15841   HOR_14342   HOR_18917   HOR_3125    HOR_11662   HOR_6220    HOR_18201   HOR_12412   HOR_14876   HOR_19952   HOR_16358   HOR_6892    HOR_3155    HOR_19184   HOR_9927    HOR_14787   HOR_8370    HOR_4258    HOR_14411   HOR_21548   HOR_12070   HOR_7251    HOR_8904    HOR_4653    HOR_2749    HOR_2957    HOR_2028    HOR_1816    HOR_7236    HOR_495 HOR_2403    HOR_15860   HOR_14439   HOR_11409   HOR_9043    HOR_5876    HOR_13836   HOR_10886   HOR_13938   HOR_20269   HOR_11448   HOR_13979   HOR_14557   HOR_12311   HOR_10004   HOR_15908   HOR_1531    HOR_13821   HOR_18681   HOR_6936    HOR_1990    HOR_10990   HOR_3187    HOR_3728    HOR_15857   HOR_20909   HOR_1581    HOR_13462   HOR_14914   HOR_16149   HOR_12168   HOR_6952    HOR_1048    HOR_18824   HOR_1361    HOR_16071   HOR_3233    HOR_21599   HOR_15871   HOR_11101   HOR_13724   HOR_1780    HOR_8711    HOR_3369    HOR_3909    HOR_21461   HOR_3093    HOR_11013   HOR_18795   HOR_15254   HOR_5486    HOR_17591   HOR_19568   HOR_7129    HOR_6934    HOR_8760    HOR_18924   HOR_21687   HOR_15895   HOR_15898   HOR_15905   HOR_21641   HOR_3686    HOR_7657    HOR_18385   HOR_8710    HOR_17077   HOR_2293    HOR_3081    HOR_7172    HOR_4201    HOR_2383    HOR_19379   HOR_7124    HOR_3912    HOR_15401   HOR_18381   HOR_4034    HOR_16078   HOR_12072   HOR_13716   HOR_4403    HOR_4829    HOR_11431   HOR_4138    HOR_2290    HOR_15788   HOR_7552    HOR_10775   HOR_3153    HOR_14689   HOR_11126   HOR_2180    HOR_2369    HOR_1044    HOR_21428   HOR_18220   HOR_3359    HOR_15778   HOR_15840   HOR_18364   HOR_9250    HOR_11922   HOR_3926    HOR_7474    HOR_12830   HOR_5020    HOR_11450   HOR_9492    HOR_5373    HOR_18913   HOR_12791&#xA;1H  42240   .   C   A   999 .   .   GT:DP:DV:GQ 0/0:15:0:100    0/1:9:3:100 0/0:3:0:100 0/0:20:0:100    0/0:4:0:100 1/1:3:3:100 ./.:.:.:.   0/0:24:0:100    0/0:9:0:100 0/0:12:0:100    0/0:14:0:100    ./.:.:.:.   0/0:9:0:100 ./.:.:.:.   0/0:29:0:100    0/0:10:0:100    0/0:5:0:100 0/0:14:0:100    0/0:3:0:100 0/0:18:0:100    0/0:18:0:100    0/0:7:0:100 0/0:9:0:100 0/0:9:0:100 1/1:5:5:100 0/0:13:0:100    ./.:.:.:.   0/0:22:0:100    ./.:.:.:.   ./.:.:.:.   0/0:4:0:100 0/0:18:0:100    ./.:.:.:.   0/0:7:0:100 0/0:11:0:100    0/0:32:0:100    0/0:13:0:100    ./.:.:.:.   0/0:15:0:100    0/0:12:0:100    0/0:8:0:100 ./.:.:.:.   0/0:20:0:100    0/0:11:0:100    0/0:6:0:100 0/0:14:0:100    0/0:20:0:100    0/0:5:0:100 ./.:.:.:.   0/0:3:0:100 0/0:5:0:100 0/0:16:0:100    1/1:3:3:100 0/0:32:0:100    0/0:15:0:100    0/0:17:0:100    0/0:16:0:100    0/0:4:0:100 0/0:17:0:100    0/0:2:0:100 0/0:4:0:100 0/0:7:0:100 0/0:20:0:100    ./.:.:.:.   0/0:20:0:100    0/0:8:0:100 0/0:20:0:100    0/0:3:0:100 0/0:21:0:100    0/0:5:0:100 ./.:.:.:.   0/0:12:0:100    0/0:16:0:100    0/0:87:0:100    0/0:21:0:100    0/0:5:0:100 0/0:20:0:100    1/1:6:6:100 0/0:6:0:100 0/0:10:0:100    ./.:.:.:.   0/0:6:0:100 0/0:4:0:100 0/0:8:0:100 0/0:19:0:100    0/0:17:0:100    0/0:4:0:100 0/0:3:0:100 1/1:7:7:100 0/0:18:0:100    0/0:12:0:100    0/0:11:0:100    0/0:13:0:100    ./.:.:.:.   0/0:18:0:100    0/0:18:0:100    0/0:2:0:100 0/0:12:0:100    0/0:10:0:100    0/0:3:0:100 ./.:.:.:.   0/0:12:0:100    0/0:2:0:100 0/0:8:0:100 0/0:15:0:100    0/0:7:0:100 0/0:22:0:100    0/0:12:0:100    0/0:14:0:100    0/0:8:0:100 0/0:10:0:100    0/0:22:0:100    0/0:10:0:100    0/0:11:0:100    0/0:3:0:100 0/0:6:0:100 0/0:6:0:100 0/0:8:0:100 0/0:18:0:100    ./.:.:.:.   0/0:13:0:100    0/0:20:0:100    0/0:17:0:100    0/0:8:0:100 0/0:4:0:100 0/0:11:0:100    0/0:12:0:100    0/0:8:0:100 0/0:12:0:100    ./.:.:.:.   0/0:6:0:100 ./.:.:.:.   0/0:10:0:100    0/0:6:0:100 0/0:10:0:100    0/0:15:0:100    0/0:15:0:100    0/0:17:0:100    0/0:4:0:100 0/0:9:0:100 ./.:.:.:.   0/0:14:0:100    0/0:13:0:100    0/0:9:0:100 0/0:3:0:100 0/0:12:0:100    0/0:12:0:100    ./.:.:.:.   ./.:.:.:.   0/1:5:2:100 0/0:21:0:100    0/0:9:0:100 0/0:14:0:100    0/0:18:0:100    0/0:11:0:100    1/1:3:3:100 0/0:19:0:100    0/0:25:0:100    ./.:.:.:.   ./.:.:.:.   0/0:19:0:100    0/0:12:0:100    0/0:7:0:100 0/0:10:0:100    0/0:13:0:100    0/0:11:0:100    0/0:13:0:100    0/0:13:0:100    0/0:6:0:100 ./.:.:.:.   0/0:4:0:100 0/0:10:0:100    0/0:9:0:100 0/0:14:2:100    1/1:2:2:100 0/0:16:0:100    ./.:.:.:.   0/0:27:0:100    0/0:18:0:100    0/0:6:0:100 0/0:10:0:100    0/0:14:0:100    0/0:7:0:100 1/1:3:3:100 0/0:15:0:100    0/0:4:0:100 0/0:15:0:100    ./.:.:.:.   ./.:.:.:.   0/0:8:0:100 0/0:16:0:100    0/0:4:0:100 ./.:.:.:.   0/0:12:0:100    0/0:12:0:100    0/0:10:0:100    0/0:7:0:100 0/0:29:0:100    0/0:7:0:100 0/0:8:0:100 0/0:32:0:100&#xA;1H  42247   .   C   A   999 .   .   GT:DP:DV:GQ 0/0:15:0:100    0/0:11:0:100    1/1:2:2:100 0/0:19:0:100    0/0:5:0:100 ./.:.:.:.   ./.:.:.:.   0/0:23:0:100    0/0:12:0:100    1/1:13:13:100   0/0:14:0:100    ./.:.:.:.   0/0:9:0:100 ./.:.:.:.   1/1:28:27:100   0/0:12:0:100    1/1:6:6:100 0/0:15:0:100    1/1:3:3:100 0/0:17:0:100    0/0:19:0:100    0/0:7:0:100 0/0:9:0:100 0/0:10:0:100    0/0:4:0:100 1/1:12:12:100   ./.:.:.:.   0/0:22:0:100    ./.:.:.:.   ./.:.:.:.   0/0:6:0:100 0/0:17:0:100    ./.:.:.:.   0/0:8:0:100 0/0:11:0:100    0/0:33:0:100    0/0:14:0:100    ./.:.:.:.   0/0:15:0:100    0/0:11:0:100    0/0:8:0:100 ./.:.:.:.   0/0:21:0:100    0/0:11:0:100    0/0:6:0:100 1/1:15:15:100   0/0:20:0:100    0/0:5:0:100 ./.:.:.:.   0/0:3:0:100 0/0:6:0:100 0/0:17:0:100    0/0:4:0:100 0/0:32:0:100    1/1:15:15:100   0/0:16:0:100    0/0:18:0:100    0/0:5:0:100 0/0:17:0:100    0/0:2:0:100 0/0:5:0:100 0/0:7:0:100 0/0:20:0:100    ./.:.:.:.   0/0:21:0:100    0/0:8:0:100 0/0:20:0:100    0/0:3:0:100 1/1:21:21:100   0/0:6:0:100 0/0:2:0:100 0/0:11:0:100    0/0:15:0:100    0/0:88:0:100    0/0:21:0:100    1/1:5:5:100 0/0:22:0:100    0/0:8:0:100 0/0:6:0:100 0/0:10:0:100    ./.:.:.:.   0/0:5:0:100 1/1:4:4:100 0/0:8:0:100 0/0:19:0:100    0/0:17:0:100    1/1:4:4:100 1/1:3:3:100 0/0:5:0:100 0/0:18:0:100    0/0:11:0:100    0/0:11:0:100    0/0:15:0:100    ./.:.:.:.   0/0:18:0:100    0/0:18:0:100    0/0:2:0:100 0/0:15:0:100    0/0:11:0:100    1/1:3:3:100 ./.:.:.:.   0/0:14:0:100    ./.:.:.:.   1/1:7:7:100 0/0:16:0:100    0/0:7:0:100 0/0:23:0:100    0/0:14:0:100    0/0:14:0:100    0/0:8:0:100 0/0:10:0:100    0/0:24:0:100    0/0:10:0:100    0/0:11:0:100    0/0:3:0:100 0/0:5:0:100 0/0:7:0:100 1/1:8:8:100 0/0:17:0:100    ./.:.:.:.   0/0:14:0:100    0/0:20:0:100    0/0:17:0:100    1/1:8:8:100 0/0:4:0:100 0/0:11:0:100    0/0:13:0:100    0/0:8:0:100 0/0:15:0:100    ./.:.:.:.   0/0:6:0:100 ./.:.:.:.   0/0:11:0:100    0/0:7:0:100 0/0:10:0:100    0/0:17:0:100    0/0:18:0:100    0/0:16:0:100    0/0:4:0:100 0/0:9:0:100 ./.:.:.:.   0/0:14:0:100    0/0:13:1:100    0/0:9:0:100 0/0:3:0:100 0/0:12:0:100    0/0:12:0:100    ./.:.:.:.   0/0:3:0:100 0/0:4:0:100 0/0:21:0:100    0/0:9:0:100 0/0:16:0:100    0/0:16:0:100    0/0:12:0:100    0/0:4:0:100 0/0:20:0:100    0/0:24:0:100    ./.:.:.:.   ./.:.:.:.   0/0:18:0:100    0/0:12:0:100    0/0:7:0:100 0/0:10:0:100    0/0:12:0:100    0/0:12:0:100    0/0:14:0:100    0/0:13:0:100    1/1:6:6:100 ./.:.:.:.   0/0:5:0:100 0/0:13:0:100    0/0:8:0:100 0/0:13:0:100    0/0:4:0:100 0/0:15:0:100    ./.:.:.:.   0/0:29:0:100    0/0:19:0:100    0/0:6:0:100 0/0:9:0:100 0/0:15:0:100    1/1:7:7:100 0/0:4:0:100 1/1:14:14:100   1/1:4:4:100 0/0:16:0:100    ./.:.:.:.   ./.:.:.:.   0/0:8:0:100 0/0:15:0:100    1/1:4:4:100 ./.:.:.:.   1/1:13:13:100   0/0:12:0:100    0/0:10:0:100    1/1:6:6:100 0/0:30:0:100    0/0:8:0:100 0/0:7:0:100 0/0:31:0:100&#xA;1H  42264   .   G   T   999 .   .   GT:DP:DV:GQ 0/0:23:0:100    0/0:17:0:100    0/0:3:0:100 0/0:21:0:100    0/0:6:0:100 0/0:8:0:100 ./.:.:.:.   0/0:29:0:100    0/0:20:0:100    0/0:9:0:100 0/0:20:0:100    ./.:.:.:.   0/0:11:0:100    ./.:.:.:.   0/0:26:0:100    0/0:13:0:100    0/0:8:0:100 0/0:19:0:100    0/0:3:0:100 0/0:19:0:100    0/0:18:0:100    0/0:7:0:100 0/0:9:0:100 0/0:18:0:100    0/0:9:0:100 0/0:13:0:100    ./.:.:.:.   0/0:22:0:100    0/0:6:0:100 ./.:.:.:.   0/0:7:0:100 0/0:22:0:100    ./.:.:.:.   0/0:10:0:100    0/0:14:0:100    0/0:39:0:100    0/0:16:0:100    ./.:.:.:.   0/0:21:0:100    0/0:19:0:100    0/0:8:0:100 ./.:.:.:.   0/0:25:0:100    0/0:13:0:100    0/0:6:0:100 0/0:19:0:100    0/0:19:0:100    0/0:4:0:100 ./.:.:.:.   0/0:2:0:100 0/0:11:0:100    0/0:28:0:100    0/0:11:0:100    0/0:37:0:100    0/0:12:0:100    0/0:25:0:100    0/0:19:0:100    0/0:9:0:100 0/0:17:0:100    ./.:.:.:.   0/0:6:0:100 0/0:9:0:100 0/0:28:0:100    0/0:3:0:100 0/0:24:0:100    0/0:11:0:100    0/0:23:0:100    0/0:4:0:100 0/0:16:0:100    0/0:7:0:100 0/0:7:0:100 0/0:19:0:100    0/0:17:0:100    0/0:105:0:100   0/0:23:0:100    0/0:6:0:100 0/0:28:0:100    0/0:18:0:100    0/0:9:0:100 0/0:15:0:100    ./.:.:.:.   0/0:7:0:100 0/0:6:0:100 0/0:11:0:100    0/0:23:0:100    0/0:19:0:100    0/0:7:0:100 0/0:7:0:100 0/0:32:0:100    0/0:20:0:100    0/0:13:0:100    0/0:14:0:100    0/0:18:0:100    0/0:11:0:100    0/0:23:0:100    0/0:19:0:100    0/0:2:0:100 0/0:20:0:100    0/0:15:0:100    0/0:4:0:100 ./.:.:.:.   0/0:17:0:100    0/0:2:0:100 0/0:10:0:100    0/0:17:0:100    0/0:8:0:100 0/0:23:0:100    0/0:20:0:100    0/0:17:0:100    0/0:8:0:100 0/0:14:0:100    0/0:30:0:100    0/0:12:0:100    0/0:15:0:100    0/0:3:0:100 0/0:6:0:100 0/0:9:0:100 0/0:13:0:100    0/0:18:0:100    ./.:.:.:.   0/0:18:0:100    0/0:21:0:100    0/0:23:0:100    0/0:9:0:100 0/0:6:0:100 0/0:14:0:100    0/0:14:0:100    0/0:9:0:100 0/0:18:0:100    ./.:.:.:.   0/0:6:0:100 ./.:.:.:.   0/0:18:0:100    0/0:8:0:100 0/0:15:0:100    0/0:17:0:100    0/0:23:0:100    0/0:26:0:100    0/0:10:0:100    0/0:12:0:100    ./.:.:.:.   0/0:17:0:100    0/0:15:0:100    0/0:11:0:100    0/0:4:0:100 0/0:16:0:100    0/0:15:0:100    ./.:.:.:.   0/0:10:0:100    0/0:7:0:100 0/0:22:0:100    0/0:10:0:100    0/0:21:0:100    0/0:24:0:100    0/0:13:0:100    0/0:19:0:100    0/0:24:0:100    0/0:32:0:100    ./.:.:.:.   0/0:6:0:100 0/0:23:0:100    0/0:17:0:100    0/0:15:0:100    0/0:15:0:100    0/0:15:0:100    0/0:14:0:100    0/0:16:0:100    0/0:17:0:100    0/0:7:0:100 ./.:.:.:.   0/0:8:0:100 0/0:14:0:100    0/0:12:0:100    0/0:30:0:100    0/0:19:0:100    0/0:24:0:100    ./.:.:.:.   0/0:36:0:100    0/0:26:0:100    0/0:7:0:100 0/0:10:0:100    0/0:21:0:100    0/0:8:0:100 0/0:13:0:100    0/0:17:0:100    0/0:4:0:100 0/0:18:0:100    0/0:2:0:100 ./.:.:.:.   0/0:9:0:100 0/0:18:0:100    0/0:5:0:100 ./.:.:.:.   0/0:9:0:100 0/0:14:0:100    0/0:15:0:100    0/0:11:0:100    0/0:32:0:100    0/0:9:0:100 0/0:12:0:100    0/0:34:0:100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And I need a SNP matrix in this format:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; , pos1, pos2, pos3&#xA;ID1   0,    1,    0&#xA;ID2   1,    1,    0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9278" LastActivityDate="2020-07-27T12:17:09.873" Title="Convert VCF to genotype table" Tags="|python|vcf|snp|gwas|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14023" PostTypeId="2" ParentId="14022" CreationDate="2020-07-27T11:46:20.487" Score="1" Body="&lt;p&gt;If you are familiar with plink and python, you can use &lt;a href=&quot;https://pandas-plink.readthedocs.io/en/latest/api/pandas_plink.read_plink.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;pandas_plink&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas_plink&#xA;snp_info,sample_info,genotypes  = pandas_plink.read_plink('genotypes/chr.1')&#xA;genotype_mat = genotypes.compute()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;genotype_mat&lt;/code&gt; is a matrix of genotypes (in 0,1,2) and &lt;code&gt;sample_info&lt;/code&gt; has the sample names and &lt;code&gt;snp_info&lt;/code&gt; has the SNP positions, ref, alt etc. You need to convert vcf to plink format first. Please make sure ref and alt alleles are extracted as expected.&lt;/p&gt;&#xA;" OwnerUserId="4017" LastEditorUserId="4017" LastEditDate="2020-07-27T12:17:09.873" LastActivityDate="2020-07-27T12:17:09.873" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14024" PostTypeId="1" CreationDate="2020-07-27T12:55:05.367" Score="3" ViewCount="190" Body="&lt;p&gt;I want to add the frequencies of the genotypes to a baseline of raw DNA data I've got from the DNA testing companies 23andMe, Ancestry and MyHeritage (tested at all). All in build 37 human genome.&lt;/p&gt;&#xA;&lt;p&gt;Where can I find that information? Checked HapMap project to no avail, can't align with the rsId or positions used in the text files.&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance for your help&lt;/p&gt;&#xA;" OwnerUserId="6743" LastActivityDate="2020-08-22T11:38:44.677" Title="Where can I find the genotype frequencies for the SNP's that are tested at 23andMe, Ancestry and FTDNA/MyHeritage?" Tags="|snp|public-databases|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14025" PostTypeId="2" ParentId="14024" CreationDate="2020-07-27T13:43:55.887" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://gnomad.broadinstitute.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;gnomad&lt;/a&gt; is probably the most comprehensive allele frequency database that exists at the moment.&lt;/p&gt;&#xA;&lt;p&gt;Depending on how many variants you need to query, you can either manually input them into the search bar, or if you have a lot more, then you can also directly download the vcfs under the downloads tab.&lt;/p&gt;&#xA;&lt;p&gt;The default for V2 gnomad set is on the gr37 build.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2020-07-27T13:43:55.887" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14026" PostTypeId="2" ParentId="14020" CreationDate="2020-07-27T13:53:33.440" Score="3" Body="&lt;p&gt;I presume you are looking for the recently sequenced WGS data rather than the genotype data? If so, the different files can be found here: &lt;a href=&quot;ftp://ngs.sanger.ac.uk/production/hgdp/hgdp_wgs.20190516/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ngs.sanger.ac.uk/production/hgdp/hgdp_wgs.20190516/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Each chromsome has a seperate vcf file with a corresponding .tbi index file.&lt;/p&gt;&#xA;&lt;p&gt;The gvcfs can be found here: &lt;a href=&quot;ftp://ngs.sanger.ac.uk/production/hgdp/hgdp_wgs.20190516/gVCFs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ngs.sanger.ac.uk/production/hgdp/hgdp_wgs.20190516/gVCFs/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;These files are more compressed and smaller in size than the regular vcfs.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2020-07-27T13:53:33.440" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14027" PostTypeId="2" ParentId="13984" CreationDate="2020-07-27T15:23:35.993" Score="3" Body="&lt;p&gt;The idea for this code is to first convert PIDs to PMCIDs, then run the &lt;code&gt;tidypmc&lt;/code&gt; in a loop over the PMCIDs. The only problem is that &lt;code&gt;tidypmc&lt;/code&gt; failed to retrieve tables from most of the IDs in your example list.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse)&#xA;library(tidypmc)&#xA;library(httr)&#xA;library(jsonlite)&#xA;example_pids &amp;lt;- c(30024784, 30555165, 30510081, 31688884, 31516032, 28588019, 29286103) %&amp;gt;% as.character()&#xA;&#xA;#-- Convert to PMC ids&#xA;convertPIDtoPMCID &amp;lt;- function(pids) {&#xA;    #-------- Make API request&#xA;    pids4query &amp;lt;- paste(pids, collapse = &amp;quot;%0D%0A&amp;quot;)&#xA;    idconv_req &amp;lt;- paste0(&amp;quot;https://www.ncbi.nlm.nih.gov/pmc/utils/idconv/v1.0/?ids=&amp;quot;, pids4query, &amp;quot;&amp;amp;idtype=pmid&amp;amp;format=json&amp;amp;versions=no&amp;amp;showaiid=no&amp;amp;tool=&amp;amp;email=&amp;amp;.submit=Submit&amp;quot;)&#xA;    pids_json &amp;lt;- GET(idconv_req) %&amp;gt;% content(&amp;quot;text&amp;quot;) %&amp;gt;% fromJSON()&#xA;    #-------- Get info from JSON&#xA;    pid2pmc &amp;lt;- pids_json&lt;span class=&quot;math-container&quot;&gt;$records %&amp;gt;% &#xA;        select(pmcid, pmid) %&amp;gt;% &#xA;        as.data.frame()&#xA;    rownames(pid2pmc) &amp;lt;- pid2pmc$&lt;/span&gt;pmid&#xA;    pmcids &amp;lt;- pmcids &amp;lt;- pid2pmc[pids, &amp;quot;pmcid&amp;quot;]&#xA;    return(pmcids)&#xA;}&#xA;example_pmcids &amp;lt;- convertPIDtoPMCID(example_pids)&#xA;&#xA;#-- Try to get data with tidypmc&#xA;pub_tables &amp;lt;- lapply(example_pmcids, function(pmc_id) {&#xA;    message(&amp;quot;-- Trying &amp;quot;, pmc_id, &amp;quot;...&amp;quot;)&#xA;    doc &amp;lt;- tryCatch(pmc_xml(pmc_id), &#xA;                    error = function(e) {&#xA;                        message(&amp;quot;------ Failed to recover PMCID&amp;quot;)&#xA;                        return(NULL)&#xA;                        })&#xA;    if(!is.null(doc)) { &#xA;        #-- If succeed, try to get table&#xA;        tables &amp;lt;- pmc_table(doc)&#xA;        if(!is.null(tables)) {&#xA;            #-- If succeed, try to get table name&#xA;            table_caps &amp;lt;- pmc_caption(doc) %&amp;gt;%&#xA;                filter(tag == &amp;quot;table&amp;quot;)&#xA;            names(tables) &amp;lt;- paste(table_caps&lt;span class=&quot;math-container&quot;&gt;$label, table_caps$&lt;/span&gt;text, sep = &amp;quot; - &amp;quot;)&#xA;        }&#xA;        return(tables) &#xA;    } else {&#xA;        #-- If fail, return NA&#xA;        return(NA)&#xA;    }&#xA;})&#xA;names(pub_tables) &amp;lt;- example_pids&#xA;&#xA;#-- Inspect results&#xA;pub_tables&lt;span class=&quot;math-container&quot;&gt;$`30555165`$&lt;/span&gt;`Table 1 - Patient demographic and baseline characteristics`&#xA;pub_tables&lt;span class=&quot;math-container&quot;&gt;$`29286103`$&lt;/span&gt;`Table I - Sample summary.`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Tables will require quite a bit of tidying after this. Good luck!&lt;/p&gt;&#xA;" OwnerUserId="8943" LastActivityDate="2020-07-27T15:23:35.993" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="14028" PostTypeId="1" CreationDate="2020-07-27T17:33:37.110" Score="3" ViewCount="273" Body="&lt;p&gt;I'm a computer scientist teaching algorithms development in the Fall. One of the algorithms we teach is called Edit Distance, and our folklore is that it is used to compare RNA sequences (is this actually true in practice?).&lt;/p&gt;&#xA;&lt;p&gt;I would like to have students implement the edit distance algorithm and run it on actual SARS-COV-2 sequences, so I'm trying to understand exactly what I get from the GenBank database. I downloaded this one: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/1798174254&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/1798174254&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am looking at the genomic.fna file. So this is apparently the FASTA file format, and the lines that begin with &amp;gt;MN988669.1 ... are comments. I see comments like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;MN988669.1 Severe acute respiratory syndrome coronavirus 2 isolate 2019-nCoV WHU02, complete genome&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Followed by an RNA string. Is this the start of a new sequence for a different coronavirus specimen? So I could have students extract each of these and run edit distance and then produce a dendrogram or something? How do I find more information on where the samples came from? Is this the right file to be using, or should I use the gbff file? And are the PDB files interesting to me at all (I actually do know what PDB files are)?&lt;/p&gt;&#xA;&lt;p&gt;Also, are there any recommended data sets where we could do something like track mutations in the virus (and see, e.g., that the NYC outbreak originated from Europe and not China)? Are there other useful algorithms / data that might be interesting for students to study in this vein? Specifically interesting to me would be graph-search algorithms, minimum spanning trees, and network flow. Also any NP-complete algorithms that we could run backtracking on. Obviously taking the theoretical study of algorithms to something as currently topical as the coronavirus has pedagogical value.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&lt;p&gt;Based on comments below here is what is taking shape.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Have students implement vanilla EditDistance (which there seems to be some disagreement about which algorithms are named what, so let's say insertions and deletions only, which I would call Longest Common Subsequence LCS). Then a variant that also does alignment (i.e. full Levenshtein distance computation, which I would call EditDistance, but Wikipedia calls the Needleman-Wunsch algorithm with a gap penalty of 1), then maybe Needleman-Wunsch with different gap penalties (if someone tells me what would make sense biologically).&lt;/li&gt;&#xA;&lt;li&gt;Have students implement basic heirarchical clustering / phylogenetic tree generation algorithms &lt;em&gt;a la&lt;/em&gt; &lt;a href=&quot;https://www.ncss.com/wp-content/themes/ncss/pdf/Procedures/NCSS/Hierarchical_Clustering-Dendrograms.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncss.com/wp-content/themes/ncss/pdf/Procedures/NCSS/Hierarchical_Clustering-Dendrograms.pdf&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Have students run their sequence alignment variants and different clustering algorithms on SARS-COV-2 sequences and report on how the choices of parameters in 1 and 2 change the result and therefore potentially the analysis.&lt;/li&gt;&#xA;&lt;li&gt;Ask some open ended written response questions on what this might mean for society, whether this introduces ethical considerations for an algorithms designer or are they doing just the math, etc.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My learning objectives (as they are now shaping up) are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Students will understand that just because their algorithm comes with a proof of correctness, it doesn't mean it's the correct algorithm for the job.&lt;/li&gt;&#xA;&lt;li&gt;Students will understand that different models / parameters to a model result in different outcomes, and thus even computational problems are not purely computational.&lt;/li&gt;&#xA;&lt;li&gt;Doing theoretical computer science / mathematics is not devoid of ethical considerations.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I would very much appreciate thoughts on the above.&lt;/p&gt;&#xA;" OwnerUserId="9281" LastEditorUserId="3848" LastEditDate="2023-09-08T23:02:12.997" LastActivityDate="2023-09-08T23:02:12.997" Title="Understanding the fasta file format and using edit distance for alignment" Tags="|phylogenetics|sequence-alignment|phylogeny|fasta|covid-19|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="14030" PostTypeId="2" ParentId="14015" CreationDate="2020-07-27T20:08:33.387" Score="4" Body="&lt;p&gt;&lt;strong&gt;Samtools&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Samtools is a toolkit that includes a binary called &lt;code&gt;samtoools&lt;/code&gt;, which can be used to query a FASTA file — to extract a subsequence based on some coordinate range you specify.&lt;/p&gt;&#xA;&lt;p&gt;If you do not have &lt;code&gt;samtools&lt;/code&gt; installed, you need to add it to your computer. For OS X, you could use &lt;a href=&quot;https://brew.sh/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Homebrew&lt;/a&gt;, for example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ brew install samtools&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For Ubuntu, you could use &lt;code&gt;apt-get&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ sudo apt-get install samtools&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Reference assembly&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The next step is to download a reference assembly for your genome and genomic build-of-interest.&lt;/p&gt;&#xA;&lt;p&gt;There is a &lt;a href=&quot;https://www.xenbase.org/entry/&quot; rel=&quot;nofollow noreferrer&quot;&gt;research group&lt;/a&gt; dedicated to &lt;a href=&quot;https://www.xenbase.org/other/static-xenbase/ftpDatafiles.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;making &lt;em&gt;Xenopus&lt;/em&gt; datasets available&lt;/a&gt;. If you were interested in v10 of the &lt;code&gt;Xentr&lt;/code&gt; genome, for instance, you could do the following or similar to download it:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ curl --output &amp;quot;XENTR_10.0_genome.fasta.gz&amp;quot; &amp;quot;http://ftp.xenbase.org/pub/Genomics/JGI/Xentr10.0/XENTR_10.0_genome.fasta.gz&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You could also dig through the NCBI site to get a similar link. It's the same idea — just change the web address, accordingly.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Indexing&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Indexing makes it possible to extract an arbitrary subsequence of interest from this FASTA file. You need to index, but you only need to do it once:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ samtools faidx XENTR_10.0_genome.fasta.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This creates two new files: &lt;code&gt;XENTR_10.0_genome.fasta.gz.fai&lt;/code&gt; and &lt;code&gt;XENTR_10.0_genome.fasta.gz.gzi&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Note&lt;/em&gt;: You need to keep these two files in the same directory as the original, compressed FASTA file &lt;code&gt;XENTR_10.0_genome.fasta.gz&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Querying&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now you are ready to query, or extract a subsequence of interest. If you wanted the first hundred bases of the first chromosome, for instance:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ samtools faidx XENTR_10.0_genome.fasta.gz Chr1:1-100&#xA;&amp;gt;Chr1:1-100&#xA;cctcagcgtcttcttctttctgctcccactgcattagcatagggagagagggccacagag&#xA;gctaggtagcattgccggctagcaatcttcagcgtcctct&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The search term used here is &lt;code&gt;Chr1:1-100&lt;/code&gt;. Replace &lt;code&gt;Chr1&lt;/code&gt; with the chromosome of interest and &lt;code&gt;1&lt;/code&gt; and &lt;code&gt;100&lt;/code&gt; with the start and stop coordinates, respectively, of interest. Coordinates used with &lt;code&gt;samtools&lt;/code&gt; queries use a &lt;a href=&quot;https://www.biostars.org/p/84686/&quot; rel=&quot;nofollow noreferrer&quot;&gt;one-based indexing scheme&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Note&lt;/em&gt;: This uses the chromosome naming scheme in the &lt;code&gt;XENTR_10.0_genome.fasta.gz&lt;/code&gt; file (&lt;code&gt;Chr1&lt;/code&gt;, etc.). Other research groups (NCBI) might use different chromosome names. If you get your FASTA from somewhere else, investigate the FASTA file to determine the naming scheme being used.&lt;/p&gt;&#xA;&lt;p&gt;If you want to redirect output to a file, use the output redirect operator (&lt;code&gt;&amp;gt;&lt;/code&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ samtools faidx XENTR_10.0_genome.fasta.gz Chr1:1-100 &amp;gt; mySubsequence.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="776" LastActivityDate="2020-07-27T20:08:33.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14031" PostTypeId="2" ParentId="3253" CreationDate="2020-07-27T20:10:17.930" Score="0" Body="&lt;p&gt;This is an old post, but we have recently published a web-based tool called &lt;a href=&quot;https://aekiz.shinyapps.io/CIPR/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CIPR&lt;/a&gt; (cluster identity predictor) that enables users to upload their data along with a custom reference file for annotating unknown clusters. You can probably obtain relevant reference datasets from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/browse/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GEO&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You can read the manuscript in &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-020-3538-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;BMC Bioinformatics&lt;/a&gt; for more information about how the algorithm works.&lt;/p&gt;&#xA;" OwnerUserId="9039" LastActivityDate="2020-07-27T20:10:17.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14032" PostTypeId="2" ParentId="4367" CreationDate="2020-07-27T20:18:54.310" Score="0" Body="&lt;p&gt;We have recently published a web-based tool called &lt;a href=&quot;https://aekiz.shinyapps.io/CIPR/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CIPR&lt;/a&gt; (cluster identity predictor) that helps with cluster annotations. You can select one of 7 reference datasets (some contains neurons) or upload a custom reference gene expression file for predicting cluster identity. You can read the &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-020-3538-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;manuscript&lt;/a&gt; for more information about how the algorithm works, but in a nutshell, it compares differentially expressed genes or global expression patterns within your data to known reference datasets and creates quick visual outputs. I find it helpful for interactive analyses where I try different clustering parameters and such.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1795-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;This article&lt;/a&gt; is a good resource that compares different cluster calling algorithms.&lt;/p&gt;&#xA;" OwnerUserId="9039" LastActivityDate="2020-07-27T20:18:54.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14033" PostTypeId="2" ParentId="8929" CreationDate="2020-07-27T20:25:05.410" Score="1" Body="&lt;p&gt;This may not be a viable answer for all experimental contexts, but you can compare expression profiles of individual cells against datasets that contain known reference cell types (migratory and not). We have recently published a web-based tool called &lt;a href=&quot;https://aekiz.shinyapps.io/CIPR/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CIPR&lt;/a&gt; (cluster identity predictor) that helps with cluster annotations in scRNAseq experiments. There are various reference datasets available, but you can also provide a custom reference by preparing data from databases like &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/browse/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GEO&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You can read the &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-020-3538-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;manuscript&lt;/a&gt; for more information about how the algorithm works, but in a nutshell, it compares differentially expressed genes or global expression patterns within your data to known reference datasets and creates quick visual outputs. If your reference contains both migratory and non-migratory cell types, maybe CIPR can help differentiate them. For other identity prediction tools, check out &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1795-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; is a good resource that compares different cluster calling algorithms.&lt;/p&gt;&#xA;" OwnerUserId="9039" LastActivityDate="2020-07-27T20:25:05.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14034" PostTypeId="2" ParentId="14028" CreationDate="2020-07-27T20:49:27.890" Score="3" Body="&lt;p&gt;Your understanding of FASTA format is about right. The type of basic problem you're eluding to we term &amp;quot;sequence alignment&amp;quot;- edit distance might be okay for teaching but in practise we use other algorithms, e.g. you might be interested in the Needleman–Wunsch or Smith–Waterman algorithms. Richard Durbin et al. wrote a great book that covers these any much more &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/0521629713&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.amazon.com/Biological-Sequence-Analysis-Probabilistic-Proteins/dp/0521629713&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Tracking mutations etc. requires more than just alignment though, see &amp;quot;phylogenetics&amp;quot; (i.e. building genetic trees) and &amp;quot;variant calling&amp;quot;. Also checkout what the nextstrain team is doing &lt;a href=&quot;https://nextstrain.org/ncov/global&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nextstrain.org/ncov/global&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In general, looking for practical applications for your algorithms is great, but be very careful before drawing any real world conclusions about the coronavirus outbreak from such analyses&lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="104" LastEditDate="2020-07-27T21:33:13.493" LastActivityDate="2020-07-27T21:33:13.493" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14035" PostTypeId="2" ParentId="13981" CreationDate="2020-07-27T20:55:19.243" Score="1" Body="&lt;p&gt;If you've copied over from some of the WGNCA tutorials you might have additionally copied the sizeGrWindow command - it's caused some of my plots to disappear.&lt;/p&gt;&#xA;&lt;p&gt;ie:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sizeGrWindow(9,6)  # delete this line&#xA;plot(...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9282" LastActivityDate="2020-07-27T20:55:19.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14036" PostTypeId="1" CreationDate="2020-07-27T21:31:49.233" Score="2" ViewCount="157" Body="&lt;p&gt;I'm hoping to quantitatively show differences in &lt;em&gt;total&lt;/em&gt; RNA expression for a gene in a cluster of interest between different experimental groups. My exported &lt;em&gt;average&lt;/em&gt; RNA values for each experimental group in my macrophage cluster are shown below:&lt;/p&gt;&#xA;&lt;p&gt;00 d Control: 10.43&lt;/p&gt;&#xA;&lt;p&gt;00 d PLX5622: 14.48&lt;/p&gt;&#xA;&lt;p&gt;07 d Control: 16.08&lt;/p&gt;&#xA;&lt;p&gt;07 d PLX5622: 17.12&lt;/p&gt;&#xA;&lt;p&gt;28 d Control: 15.90&lt;/p&gt;&#xA;&lt;p&gt;28 d PLX6722: 19.40&lt;/p&gt;&#xA;&lt;p&gt;However, the violin plot for Cd68 in the macrophage cluster looks like this&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;VlnPlot(combine.combined, features = (&amp;quot;Cd68&amp;quot;), pt.size = 1, idents = &amp;quot;Macrophages&amp;quot;, group.by = &amp;quot;orig.ident&amp;quot;, split.by = NULL, assay = &amp;quot;RNA&amp;quot;):&lt;a href=&quot;https://i.stack.imgur.com/vBnzn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vBnzn.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The violin plot suggests that the Control groups actually have more total CD68 RNA in them than in the PLX5622 groups, which isn't what the average RNA values show. Any code you can share showing me how to get total RNA per group would be an amazing help!&lt;/p&gt;&#xA;&lt;p&gt;Thanks so much!&lt;/p&gt;&#xA;" OwnerDisplayName="user9283" LastActivityDate="2020-08-28T00:01:15.117" Title="How can I take cell number into account to find total RNA expression?" Tags="|seurat|single-cell|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14037" PostTypeId="2" ParentId="14036" CreationDate="2020-07-28T00:49:08.297" Score="-1" Body="&lt;p&gt;To me, it looks like the widest parts of the plots are higher in the treated.  I don't see why you are so sure that the violin plots are hugely different from the calculated averages.&lt;/p&gt;&#xA;&lt;p&gt;Either way, the differences are really small.  Maybe not statistically significant.&lt;/p&gt;&#xA;&lt;p&gt;BTW, in the future, you might want to disguise your treatment and gene of interest when on a public forum.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastEditorUserId="2388" LastEditDate="2020-07-28T23:59:20.567" LastActivityDate="2020-07-28T23:59:20.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14038" PostTypeId="2" ParentId="14036" CreationDate="2020-07-28T11:16:33.753" Score="1" Body="&lt;p&gt;(I currently cant comment as I used up all my rep for a bounty.)&lt;/p&gt;&#xA;&lt;p&gt;Could you explain how you calculate the average? In you plot the averages are between an expression level of 2 and 3, while you report values over 10. Given the data you show, there is no significant difference and I would guess p-values are above 0.8!&lt;/p&gt;&#xA;&lt;p&gt;And you mention that the violin plots suggest the control groups have higher expression.. but how do you see that? For me they are identical with minimal trends that the PLX5622 groups are higher.&lt;/p&gt;&#xA;&lt;p&gt;And to address your question regarding showing the total: No, you would want to show the mean or median, as the total is pointless, given deviating sample sizes.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: Overall, I have a feeling you uploaded the wrong picture!&lt;/p&gt;&#xA;&lt;p&gt;EDIT2: I think I got your point now: The difference in mean is caused by an increased number of outliers in your controls at the value 0. Despite those, the means are identical and this is what can be seen through the violin plots. You should rather try to diminish the effect of these outliers.&lt;/p&gt;&#xA;" OwnerUserId="6463" LastEditorUserId="6463" LastEditDate="2020-07-28T11:30:54.050" LastActivityDate="2020-07-28T11:30:54.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14039" PostTypeId="1" CreationDate="2020-07-28T13:53:31.203" Score="3" ViewCount="689" Body="&lt;p&gt;Snakemake is not really designed to store temporary files to local disk, but in the documentation, they suggest a &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/project_info/faq.html#how-can-i-make-use-of-node-local-storage-when-running-cluster-jobs&quot; rel=&quot;nofollow noreferrer&quot;&gt;workaround using &amp;quot;shadow rules&amp;quot;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The problem I have is, that our front node does not have &lt;code&gt;/scratch/..&lt;/code&gt; directory and therefore snakemake complains right away&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;snakemake my_result.tsv -p --jobs 10 --shadow-prefix /scratch/$USER --cluster &amp;quot;qsub &amp;quot;&#xA;   ...&#xA;   PermissionError: [Errno 13] Permission denied: '/scratch'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Every single computational node does have the directory, only the frontend does not. Is there any easy workaround?&lt;/p&gt;&#xA;&lt;p&gt;--- edit ---&lt;/p&gt;&#xA;&lt;p&gt;I will try to clarify where I think is the problem.&lt;/p&gt;&#xA;&lt;p&gt;The head node does not have anything mounted to &lt;code&gt;/scratch&lt;/code&gt;, I think that's why I am getting that error. I sort of expected that snakemake will attempt do create the shadow environments for individual executed jobs, because if it does, everything would be fine. All the computing nodes do have their local disks mounted to &lt;code&gt;/scratch&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Of course, I could use a different shadow directory, but all the directories that are mounted both at the front-end and computing nodes are NOT local disks of individual nodes, which would cause more network traveling of the temporary data.&lt;/p&gt;&#xA;&lt;p&gt;--- edit 2 ---&lt;/p&gt;&#xA;&lt;p&gt;Thanks @Chris_Rands for finding the relevant lines. I opened an &lt;a href=&quot;https://github.com/snakemake/snakemake/issues/522&quot; rel=&quot;nofollow noreferrer&quot;&gt;Issue on Snakemake GitHub&lt;/a&gt;. Once I will know more, I will get back with the answer here.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2020-07-29T10:59:36.567" LastActivityDate="2023-12-11T16:06:45.860" Title="using snakemake shadow rules to store temp files on local nodes" Tags="|snakemake|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="14041" PostTypeId="2" ParentId="14039" CreationDate="2020-07-28T18:19:01.767" Score="0" Body="&lt;p&gt;What is happening is your Admin is not allowing you access to /scratch ... which is undertandable. Essentially you are attempting to access the headnode &lt;code&gt;/scratch&lt;/code&gt;. If every user on the cluster had access, it would be a bit chaotic.&lt;/p&gt;&#xA;&lt;p&gt;I am surprised by the error, not that I would use Snakemake I'd code it, because you would expect a data pipeline tool as established as Snakemake to have a solution to working on a cluster. What you are describing is a standard cluster set-up between headnode and 'sub-node' (the technical term is slavenode, but I hate that term). The headnode is mirrored across the cluster, hence the error. Once you submit via &lt;code&gt;qsub&lt;/code&gt; the mirror will 'display' the headnode layout on the subnode.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You could ask your admin for permission to &lt;code&gt;/scratch&lt;/code&gt;, this is root privilege. It is a bit unlikely they will say 'yes' but its worth a try.&lt;/li&gt;&#xA;&lt;li&gt;Otherwise what about a virtual environment? When Perl suffers this problem I simply use 'perlbrew', anaconda has very nice virtual environments that are easy to setup&lt;/li&gt;&#xA;&lt;li&gt;@Chris_Rand points out piping to a different output.&lt;/li&gt;&#xA;&lt;li&gt;I bet there is a solution in '.profile' and thats how I solved comparable problems in the past.&lt;/li&gt;&#xA;&lt;li&gt;Of course &lt;code&gt;Snakemake&lt;/code&gt; do a Q&amp;amp;A interaction on Overflow SE (but please come back here!) ... they must know the work around.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Frankly I am astonished because the whole point of Snakemake, is the ease of constructing a data pipeline and data pipelines are processed on clusters, thus this error seems to up-end it, but who am I?&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-07-28T18:24:12.970" LastActivityDate="2020-07-28T18:24:12.970" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14042" PostTypeId="1" CreationDate="2020-07-28T19:04:10.440" Score="2" ViewCount="1405" Body="&lt;p&gt;For the differentially expressed genes analysis, is it possible to check for DEGs based on the levels already identified in the object? For example, my dataset contains cells from 11 subjects, which are marked as so (orig.ident). I want to analyze the differentially expressed markers between them. Will it be accurate to do so? Or should I follow the clustering analysis in Seurat? I want to see the difference in genes between the cells according to the subjects but I'm afraid that may not be the right way to do so.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  head(BC@meta.data)&#xA;         orig.ident nCount_RNA nFeature_RNA percent.mito&#xA; BC01_02       BC01   998944.0         9189 0.2387108038&#xA; BC01_03       BC01   999696.4         9548 0.2925345423&#xA; BC01_04       BC01   999057.5         7440 0.0009893724&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is clustering analysis absolutely necessary to see the DEGs between them? or can I directly move to FindMarkers based on the levels (orig.ident)?&lt;/p&gt;&#xA;" OwnerUserId="9234" LastActivityDate="2022-09-24T01:05:15.533" Title="Differentially expressed genes analysis in Seurat" Tags="|r|scrnaseq|seurat|differential-expression|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14043" PostTypeId="2" ParentId="14042" CreationDate="2020-07-28T21:53:49.453" Score="0" Body="&lt;p&gt;I'm not quite sure what you are asking, but you can use anything in the metadata as a grouping with FindMarkers.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;DefaultAssay(combined.all) &amp;lt;- &amp;quot;RNA&amp;quot;&#xA;Idents(combined.all) &amp;lt;- &amp;quot;orig.ident&amp;quot;&#xA;&#xA;&#xA;for (sample in unique(Idents(combined.all))) {&#xA;  clustermarkers &amp;lt;- FindMarkers(combined.all, ident.1 = sample)&#xA;  print(paste(&amp;quot;Cluster&amp;quot;, sample))&#xA;  print(head(clustermarkers, n = 10))&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-07-28T21:53:49.453" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14044" PostTypeId="2" ParentId="14019" CreationDate="2020-07-29T07:10:42.130" Score="1" Body="&lt;p&gt;You could convert the phyloseq object to a dataframe and just plot it using base R or ggplot2. I'm not exactly sure what the bray curtis df will be called but you can access it using the @ notation then convert with data.frame e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bray_for_plot &amp;lt;- data.frame(phylo_obj@bray)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="220" LastActivityDate="2020-07-29T07:10:42.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14047" PostTypeId="2" ParentId="13415" CreationDate="2020-07-29T12:04:09.680" Score="2" Body="&lt;p&gt;An alignment can be the result of two slightly different analyses:&lt;/p&gt;&#xA;&lt;p&gt;There is &lt;strong&gt;multiple sequence alignment&lt;/strong&gt; (which is what you get from MAFFT) where sequences are aligned so that similar regions are on top of each other. This may require introducing indels (insertions / deletions) if a particular region is absent in some of the sequences.&lt;br /&gt;&#xA;Then there is &lt;strong&gt;alignment to a reference sequence&lt;/strong&gt;. This is usually how you get variant data and most of the tools deal with this use case. Therefore, in the link you provided the solution includes aligning the data to a reference sequence. Usually the refernce sequence is much larger (e.g. the human genome) and after alignment the result is a file (.bam) that tells you where the query sequences match in the reference.&lt;/p&gt;&#xA;&lt;p&gt;I did a quick search for ways of converting a multiple sequence alignment (MSA) to VCF. There is a tool called &lt;code&gt;msa2vcf&lt;/code&gt; in the &lt;a href=&quot;http://lindenb.github.io/jvarkit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jvarkit&lt;/a&gt; collection of utilities that can do this. The example is for the CLUSTAW format but FASTA is accepted as well.&lt;br /&gt;&#xA;You shoud check that indels were treated correctly because these are most likely to cause trouble.&#xA;Then you can simply &lt;a href=&quot;https://stackoverflow.com/questions/39671015/how-to-convert-vcf-file-to-ped-file-using-plink&quot;&gt;convert the VCF to PLINK format&lt;/a&gt;  using PLINK.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastEditorUserId="4885" LastEditDate="2020-07-29T16:05:13.987" LastActivityDate="2020-07-29T16:05:13.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14049" PostTypeId="1" CreationDate="2020-07-29T13:37:27.077" Score="1" ViewCount="40" Body="&lt;p&gt;I apologise for very basic question but I am very new to biology and have very specific knowledge of this field, I am currently working on Bioinformatics in my machine learning project which is why I need to be sure about obtained results before I proceed. I am not aware of all the tools used in genomics/bioinformatics. I would like to know that what tools can be used if I want to trace and observe the strand orientations in the ancestral inversion mutations? I would like to know the key concepts as well, using which I can automate the process in my program and display the orientation.&lt;/p&gt;&#xA;" OwnerUserId="9173" OwnerDisplayName="Pragati Bhingare" LastActivityDate="2020-07-29T13:58:57.267" Title="&quot;How the clustal omega can be reverse engineered, to trace ancestral inversion mutations via the guide tree?&quot;" Tags="|phylogenetics|genomics|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14050" PostTypeId="1" CreationDate="2020-07-29T14:26:58.437" Score="1" ViewCount="15" Body="&lt;p&gt;I am attempting to create a markerInfoFile for use in a program called popSTR (GitHub Documentation: &lt;a href=&quot;https://github.com/DecodeGenetics/popSTR&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/DecodeGenetics/popSTR&lt;/a&gt;). The marker info file should contain information about microsatellites in a reference genome. Microsatellites are stretches of DNA where a short motif (i.e. &amp;quot;AGG&amp;quot;) is repeated multiple times (i.e. three repeats of &amp;quot;AGG&amp;quot; is &amp;quot;AGGAGGAGG&amp;quot;). Here is an example line from the popSTR author's marker info file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr10 10589 10601 GCCC 3.2 NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTAACCCTCACCCTTCTAACTGGACTCTGACCCTGATTGTTGAGGGCTGCAAAGAGGAAGAATTTTATTTACCGTCGCTGTGGCCCCGAGTTGTCCCAAAGCGAGGTAATGCCCGCAAGGTCTGTGCTGATCAGGACGCAGCTCTGCCTTCGGGGTGCCCCTGGACT GGTCTGTGCTGAGGAGAACGCTGCTCCGCCTCCGCGGTACTCCGGACATATGTGCAGAGAAGAACGCAGCTGCGCCCTCGCCATGCTCTGCGAGTCTCTGCTGATGAGAACACAGCTTCACTTTCGCAAAGGCGCAGCGCCGGCGCAGGCGCGGAGGGGCGCGCAGCGCCGGCGCAGGCGCGGAGGGGCGCGCCCGAACCCGAACCCTAATGCCGTCATAAGAGCCCTAGGGAGACCTTAGGGAACAAGCATTAAACTGACACTCGATTCTGTAGCCGGCTCTGCCAAGAGACATGGCGTTGCGGTGATATGAGGGCAGGGGTCATGGAAGAAAGCCTTCTGGTTTTAGACCCACAGGAAGATCTGTGACGCGCTCTTGGGTAGAGCACACGTTGCTGGGCGTGCGCTTGAAAAGAGCCTAAGAAGAGGGGGCGTCTGGAAGGAACCGCAACGCCAAGGGAGGGTGTCCAGCCTTCCCGCTTCAACACCTGGACACATTCTGGAAAGTTTCCTAAGAAAGCCAGAAAAATAATTTAAAAAAAAATCCAGAGGCCAGACGGGCTAATGGGGCTTTACTGCGACTATCTGGCTTAATCCTCCAAACAACCTTGCCATACCAGCCCATCAGTCCTCTGAGACAGGTGAAGAACCTGAGGTCGCAGGAGGACACCCAGAAGGTCCAGAGAGAGCCTCCTAGGCCCCCCACCTCCCCCCGTGGCAGCTCCAACCCCAGCTTTTTCACTAGTAAGGCAGTCGGGCCCCTGGGCCACGCCCACTCCCCCAAGCGGGGAAGGAGCTTCGCGCTGCCGCTTGGCTGGGGACTGGGCACCGCCCTCCCGCGGCTCCTGAGCCGGCTGCCACCAGGGGGCGCGCCAGCGGTGTCCGGGAGCCTAGCGGCGCGTGTGCAGCGGCCAGTGCACCTGCTCTGGCCCTCGCCGCGGTCTCTGCCAGGACCCCGACGCCCAGCCTGACCCTGCCATTCAGCGGGGCTGCGGCTCCA GCCCGCCCGCCCG 4 4 1.00 0.0149855 0.949846 0 0.75 0.25 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Though it may be difficult to tell from the above example, the desired output is a 17 column space-delimited file formatted like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  chrom startCoordinate endCoordinate repeatMotif numOfRepeatsInRef 1000refBasesBeforeStart 1000refBasesAfterEnd repeatSeqFromRef minFlankLeft minFlankRight repeatPurity defaultSlippage defaultStutter fractionAinMotif fractionCinMotif fractionGinMotif fractionTinMotif&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Since I am working with non-human genomes, I cannot simply use the author's sample marker info files. I could not find explicit instructions for the creation of this markerInfoFile in popSTR's full journal article (&lt;a href=&quot;https://academic.oup.com/bioinformatics/article/33/24/4041/2525679&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://academic.oup.com/bioinformatics/article/33/24/4041/2525679&lt;/a&gt;), so I emailed the authors. they instructed me to use Tandem Repeat Finder (in the Linux command prompt) with the following options:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./trf409.linux64 reference_genome.fasta 2 7 7 80 10 22 1000 -d -h -ngs &amp;gt; reference_genome_TRFmap&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here is the documentation for TRF:&#xA;&lt;a href=&quot;https://blaxter-lab-documentation.readthedocs.io/en/latest/trf.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blaxter-lab-documentation.readthedocs.io/en/latest/trf.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I applied TRF to my reference genome, but the output was in the following format:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Chrom&#xA;    startCoordinate endCoordinate PeriodSize    numOfRepeatsInRef   SizeofConsensusPattern  PercentofMatchesBetweenAdjacentCopies   PercentofIndelsBetweenAdjacentCopies OverallAlignmentScore  fractionAinMotif fractionCinMotif fractionGinMotif fractionTinMotif Entropy repeatMotif repeatSeqFromRef    1000refBasesBefore  1000refBasesAfter&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here is an example of the first two lines of my current output file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@ScyDAA6_1;HRSCAF=23&#xA;108 120 5 2.6 5 100 0 26 0 0 15 84 0.62 TTTGT TTTGTTTTGTTTT CAGTAAAGTCTTTCTTTCCTCTAACATAGAAAGTACTACTAGATTAGTGC TCTGTGTATGCTCTCTATTCTCAACCTCCAGATGCCCGTTCACACTGAGC&#xA;414 424 5 2.2 5 100 0 22 63 0 18 18 1.31 ATGAA ATGAAATGAAA TGAACTGGAATGTACATGATTGAATTTAAATTACTTCTTTAAAAAATTCC AGTATTGTGAATTGGTGCTAAATAAATAAACTGAATGAAAAATAACTCAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It looks like my current TRF output is sorted with the chromosome names at the top, and the microsatellites found at each chromosome following underneath the name. I think I can include the appropriate chromosome names into each row with no problem, and I can easily rearrange the data into the appropriate columns. My main issue is with the information itself. As you can see, my current output includes data I don't need: period size, size of consensus pattern, percent of matches between adjacent copies, percent of indels between adjacent copies, overall alignment score, and entropy are not required for my desired output (unless some of these terms are synonymous with information included in the desired output and I am simply not aware). Furthermore, my output needs to include the minimum flanking on the left and right, repeat purity, default slippage, and default stutter. Which TRF options do I need to use for this information to be included in my output?&lt;/p&gt;&#xA;" OwnerUserId="9118" LastActivityDate="2020-07-29T14:26:58.437" Title="How do I include repeat purity, default slippage, default stutter, and minimum flanking (left and right) in Tandem Repeat Finder's output?" Tags="|genome|reference-genome|mutations|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14051" PostTypeId="2" ParentId="13965" CreationDate="2020-07-29T14:53:12.717" Score="1" Body="&lt;p&gt;Most of your questions are part of the Vignettes on the Seurat home page.&lt;br /&gt;&#xA;In the &lt;a href=&quot;https://satijalab.org/seurat/v3.2/de_vignette.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;differential expression&lt;/a&gt; Vignette you can see that you can run Student's t-tests in the &lt;code&gt;FindMarkers&lt;/code&gt; wrapper function.&lt;br /&gt;&#xA;However, simple t-tests may not be the best option for single cell data. There are &lt;a href=&quot;https://www.nature.com/articles/nmeth.4612&quot; rel=&quot;nofollow noreferrer&quot;&gt;many algorithms&lt;/a&gt; for DE testing with some being directly accessible through &lt;code&gt;FindMarkers&lt;/code&gt;. The alogrithm implemented in &lt;code&gt;MAST&lt;/code&gt; is &lt;a href=&quot;https://www.embopress.org/doi/10.15252/msb.20188746&quot; rel=&quot;nofollow noreferrer&quot;&gt;recommended&lt;/a&gt; frequently.&lt;/p&gt;&#xA;&lt;p&gt;Regarding the box plots you should have a look at the violin plot function of Seurat:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;VlnPlot(SeuratData, c(&amp;quot;gene_of_interest&amp;quot;), group.by = &amp;quot;groups_compared_in_DE_test&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2020-07-29T14:53:12.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14053" PostTypeId="1" AcceptedAnswerId="14057" CreationDate="2020-07-29T19:42:36.890" Score="0" ViewCount="1612" Body="&lt;p&gt;On my merged seurat object of 6 samples, when I use the split.by function in tandem with the Dimplot/UMAP plot, all six samples are displayed in series along a commonly labeled 'UMAP_1' x-axis in an order of (these are arbitrary for simplicity, here) Z, Y, X, C, B, A. Instead, I would like to specify these plots to output as a 2 row x 3 column arrangement ordered (1st Row) A, B, C and (2nd Row) X, Y, Z.&lt;/p&gt;&#xA;&lt;p&gt;I'm assuming I could use something like: group_levels &amp;lt;- c(&amp;quot;A&amp;quot;, &amp;quot;B&amp;quot;, &amp;quot;C&amp;quot;, &amp;quot;X&amp;quot;, &amp;quot;Y&amp;quot;, &amp;quot;Z&amp;quot;) to define the order I'd like, but I'm lost on how to get this to arrange the UMAP plots into a 2x3 grid I'd like.&lt;/p&gt;&#xA;&lt;p&gt;Appreciate any help!&lt;/p&gt;&#xA;" OwnerUserId="9188" LastActivityDate="2020-07-30T09:02:01.597" Title="Help with setting DimPlot UMAP output into a 2x3 grid in Seurat" Tags="|seurat|single-cell|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14054" PostTypeId="1" CreationDate="2020-07-29T21:40:51.237" Score="1" ViewCount="1519" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/K14Nx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/K14Nx.png&quot; alt=&quot;FeaturePlot of the first four features&quot; /&gt;&lt;/a&gt;I am using FindMarkers() between 2 groups of cells, my results are listed but i'm having hard time in choosing the right markers. Do I choose according to both the p-values or just one of them? If one of them is good enough, which one should I prefer? I am interested in the marker-genes that are differentiating the groups, so what are the parameters i should look for?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;             p_val avg_logFC pct.1 pct.2  p_val_adj&#xA;UBD   3.080608e-06 1.7755753     1 0.000 0.07683959&#xA;CFB   1.262305e-05 1.3989233     1 0.067 0.31485675&#xA;RGS13 1.548593e-05 0.9009480     1 0.200 0.38626551&#xA;PYDC1 1.896309e-05 0.9622537     1 0.133 0.47299636&#xA;BIRC3 4.099414e-05 1.0129472     1 1.000 1.00000000&#xA;ICAM1 4.379895e-05 0.8219610     1 0.200 1.00000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;as you can see, p-value seems significant, however the adjusted p-value is not. and when i performed the test i got this warning &lt;code&gt;In wilcox.test.default(x = c(BC03LN_05 = 0.249819542916203,  ... : cannot compute exact p-value with ties&lt;/code&gt;&#xA;I am completely new to this field, and more importantly to mathematics. Please help me understand in an easy way.&lt;/p&gt;&#xA;" OwnerUserId="9234" LastEditorUserId="9234" LastEditDate="2020-07-30T14:01:33.100" LastActivityDate="2020-07-30T18:28:45.500" Title="Seurat FindMarkers() output interpretation" Tags="|r|scrnaseq|seurat|p-values|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14056" PostTypeId="2" ParentId="14054" CreationDate="2020-07-30T08:02:59.637" Score="1" Body="&lt;p&gt;The p-values are not very very significant, so the adj. p-value. You need to plot the gene counts and see why it is the case. It could be because they are captured/expressed only in very very few cells. &lt;code&gt;VlnPlot&lt;/code&gt; or &lt;code&gt;FeaturePlot&lt;/code&gt; functions should help. You haven't shown the TSNE/UMAP plots of the two clusters, so its hard to comment more.&lt;/p&gt;&#xA;" OwnerUserId="4017" LastEditorUserId="3541" LastEditDate="2020-07-30T08:54:39.053" LastActivityDate="2020-07-30T08:54:39.053" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14057" PostTypeId="2" ParentId="14053" CreationDate="2020-07-30T09:02:01.597" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;how to get this to arrange the UMAP plots into a 2x3 grid&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You can use the &lt;code&gt;ncol&lt;/code&gt; parameter, which will set the number of columns in the &amp;quot;combined plot&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;And for future reference, if you get the help page of the function you are using with &lt;code&gt;?DimPlot&lt;/code&gt;, you will see the &lt;code&gt;...&lt;/code&gt;. This means that you can feed additional arguments to other functions that &lt;code&gt;DimPlot()&lt;/code&gt; uses. Apparently, &lt;code&gt; CombinePlots()&lt;/code&gt; is used under the hood and has an &lt;code&gt;ncol&lt;/code&gt; parameter.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-07-30T09:02:01.597" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14059" PostTypeId="2" ParentId="13932" CreationDate="2020-07-30T10:59:58.933" Score="1" Body="&lt;p&gt;You seem to be interested in identifying potenital coding regions in a DNA sequence. These regions are called &lt;a href=&quot;https://en.wikipedia.org/wiki/Open_reading_frame&quot; rel=&quot;nofollow noreferrer&quot;&gt;open reading frames&lt;/a&gt; (ORF) because they are open to translation.&lt;br /&gt;&#xA;As detailed in the linked Wikipedia article, an ORF usually starts with an ATG and ends with one of the stop codons (TAA, TGA or TAG in DNA space). However, there are many ORFs that are not translated at all, particularly short ones.&lt;/p&gt;&#xA;&lt;p&gt;Since finding ORFs is an old and established task in bioinformatics there are many algorithms availabe to do this. I would suggest using such an existing tool because they often have additional features and bugfixes. There is a list in the link above.&lt;/p&gt;&#xA;&lt;p&gt;Estimating the stability and likelihood of a particular amino acid sequence is much more complicated. There are many &lt;a href=&quot;https://www.expasy.org/proteomics/protein_structure&quot; rel=&quot;nofollow noreferrer&quot;&gt;tools&lt;/a&gt; for protein structure prediction that may be worth having a look at.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2020-07-30T10:59:58.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14061" PostTypeId="1" CreationDate="2020-07-30T20:58:38.573" Score="0" ViewCount="717" Body="&lt;p&gt;I guess the title is self-explanatory. I'd like to find the reference transcriptome files for a few human viruses. My RNA-seq samples are from human tumor cells that were infected with viruses. The main part of my analysis is quantifying human gene expression from these samples. However, I also would like to find what proportion of my raw data is originated from virus RNA.&lt;/p&gt;&#xA;&lt;p&gt;So, I'd like to learn how to find the reference transcriptome data for human viruses. Moreover, I appreciate it if you could give me some insight into your workflow for approaching this problem. Finally, if there is any specific command line, R, or python tool for this specific task, I'd like to know about it.&lt;/p&gt;&#xA;" OwnerUserId="9299" LastEditorUserId="3848" LastEditDate="2020-12-01T10:23:07.043" LastActivityDate="2020-12-01T10:23:07.043" Title="How to get transcriptome FASTA file for viruses for Kallisto pseudo-alignment?" Tags="|rna-seq|covid-19|sars-cov-2|kallisto|virus|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14062" PostTypeId="2" ParentId="14061" CreationDate="2020-07-31T01:52:29.697" Score="1" Body="&lt;p&gt;You might be interested in the &lt;a href=&quot;https://github.com/broadinstitute/catch#downloading-viral-sequence-data&quot; rel=&quot;nofollow noreferrer&quot;&gt;viral sequence data&lt;/a&gt; from CATCH. There are various different viral subsets available.&lt;/p&gt;&#xA;&lt;p&gt;You could also try the NCBI virus website, filtering on human hosts. There's a &amp;quot;Download&amp;quot; link on this page that allows you to download the full sequence set:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/labs/virus/vssi/#/virus?VirusLineage_ss=Viruses,%20taxid:10239&amp;amp;HostLineage_ss=humans,%20taxid:9605&amp;amp;SeqType_s=Nucleotide&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/labs/virus/vssi/#/virus?VirusLineage_ss=Viruses,%20taxid:10239&amp;amp;HostLineage_ss=humans,%20taxid:9605&amp;amp;SeqType_s=Nucleotide&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You've asked a very specific question, so I'm giving you a very specific answer. It seems that what you &lt;em&gt;actually&lt;/em&gt; want to do is different from the self-explanatory question.&lt;/p&gt;&#xA;&lt;p&gt;For the purpose of creating an index for Kallisto or Salmon, it should be okay to use the full dataset. Those programs are designed to handle eukaryotic transcriptomes as input, which typically include many gene copies and isoforms, so transcript / genome duplication shouldn't be much of an issue.&lt;/p&gt;&#xA;&lt;p&gt;Without knowing more about your particular application, it's difficult to provide additional advice. From what you've written, i.e. &amp;quot;what proportion of my raw data is originated from virus RNA&amp;quot;, a hit to &lt;em&gt;any&lt;/em&gt; viral sequence would be sufficient. For example, if you want any more specific information about viral proportion / species, then you should be using a metagenomic tool like Kraken2, rather than Kallisto / Salmon.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-08-02T01:01:15.190" LastActivityDate="2020-08-02T01:01:15.190" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14064" PostTypeId="1" AcceptedAnswerId="14071" CreationDate="2020-07-31T12:55:58.670" Score="0" ViewCount="69" Body="&lt;p&gt;I am trying to find a way to read and transcribe a list of DNA sequences (list of lists) only when the for loop finds a start codon (triplet of the list items) and until it finds a stop codon, over and over, but the if function will only transcribe the first codon and won't go through the whole sequence.&#xA;I'm trying to do this with a mix of normal python and biopython functions cause I think it's simpler.&#xA;&lt;br&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is an example:&lt;/p&gt;&#xA;&lt;p&gt;Start codon: ATG&lt;/p&gt;&#xA;&lt;p&gt;Stop codon: TAG&lt;/p&gt;&#xA;&lt;p&gt;gene_1= 'ACGGACTATTC'&lt;/p&gt;&#xA;&lt;p&gt;gene_2= 'GGCC&lt;strong&gt;ATG&lt;/strong&gt;AGTAACGCA&lt;strong&gt;TAG&lt;/strong&gt;GGCCC&lt;/p&gt;&#xA;&lt;p&gt;gene_3=GGGCCC&lt;strong&gt;ATG&lt;/strong&gt;ACGTAC&lt;strong&gt;TAG&lt;/strong&gt;GGGCCC&lt;strong&gt;ATG&lt;/strong&gt;CATTCA&lt;strong&gt;TAG&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, the first gene does not contain any transcript, whereas the second contains 1 and the third contains 2. I'm trying to get rid of everything outside these reading frames and append these transcripts into a list that should look something like this.&lt;/p&gt;&#xA;&lt;p&gt;mRNAs = ['AGUAACGCA', 'ACGUAC', 'CAUUCA']&lt;/p&gt;&#xA;&lt;p&gt;This is what I have so far:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;genelist=[]&#xA;&#xA;gene_1= 'A','C','G','G','A','C','T','A','T','T','C'&#xA;gene_2= 'G','G','C','C','A','T','G','A','G','T','A','A','C','G','C','A','T','A','G','G','G','C','C','C'&#xA;gene_3='G','G','G','C','C','C','A','T','G','A','C','G','T','A','C','T','A','G','G','G','G','C','C','C','A','T','G','C','A','T','T','C','A','T','A','G'&#xA;&#xA;genelist.append(gene_1)&#xA;genelist.append(gene_2)&#xA;genelist.append(gene_3)&#xA;&#xA;def transcription(ORF):&#xA;    mRNA= ''&#xA;    for i in range(0, len(ORF), 3):&#xA;        codon= ORF[i:i+3]&#xA;        if codon != 'ATG':&#xA;            next(codon)&#xA;            if codon == 'ATG':&#xA;                mRNA=codon.transcribe()&#xA;                if codon == 'TAG':&#xA;                    break&#xA;    return(mRNA)&#xA;&#xA;mRNAs=[]&#xA;for gene in genelist:&#xA;    for codon in gene:&#xA;        mRNA= transcription(codon)&#xA;        mRNAs.append(mRNA)&#xA;print(mRNAs)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But it is not really giving anything back, I wonder if the code it's too redundant and I really don't need to define a function here, do you know any better way to do this?&#xA;Thaaanks!!&lt;/p&gt;&#xA;" OwnerUserId="9303" LastEditorUserId="73" LastEditDate="2020-08-02T01:09:44.610" LastActivityDate="2020-08-02T01:13:56.207" Title="Transcribing DNA exons and append transcripts into a list" Tags="|python|phylogenetics|homework|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14065" PostTypeId="1" AcceptedAnswerId="14111" CreationDate="2020-07-31T14:24:35.813" Score="1" ViewCount="520" Body="&lt;p&gt;Hi im looking for the aligned bases in the reads for a given reference position. im using the following script from the pysam documentataion.&#xA;I adjusted it to find the specified position. in this case 24793.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pysam&#xA;&#xA;samfile = pysam.AlignmentFile(&amp;quot;generated_alignment_sorted.bam&amp;quot;, &amp;quot;rb&amp;quot; )&#xA;for pileupcolumn in samfile.pileup(&amp;quot;chr05_modified.copy0&amp;quot;, 10, 52000000):&#xA;    if pileupcolumn.pos == 24793:&#xA;        print (&amp;quot;\ncoverage at base %s = %s&amp;quot; %&#xA;               (pileupcolumn.pos, pileupcolumn.n))&#xA;        for pileupread in pileupcolumn.pileups:&#xA;                print ('\tbase in read %s = %s' %&#xA;                      (pileupread.alignment.query_name.split(';')[0],&#xA;                       pileupread.alignment.query_sequence[pileupread.query_position]))&#xA;&#xA;samfile.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This outputs:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;coverage at base 24793 = 6&#xA;        base in read m419941/6207/CCS Read=419941 = C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For all the position I tried it doesnt print out the same number of bases as it says that the coverage is.&#xA;shouldnt this print out 6 times: base in read m419941/6207/CCS Read=419941 = C for different reads?&lt;/p&gt;&#xA;&lt;p&gt;Why is this?&lt;/p&gt;&#xA;" OwnerUserId="9304" LastActivityDate="2020-08-09T02:33:37.840" Title="Find all the bases for given reference position" Tags="|sam|pysam|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14066" PostTypeId="1" AcceptedAnswerId="14076" CreationDate="2020-07-31T18:04:08.540" Score="1" ViewCount="168" Body="&lt;p&gt;Is there a way to give weight arguments to each of the residues when using protein_scale in the ProteinAnalysis module of Bio.SeqUtils.ProtParam? I thought this would be done with the window and edge arguments, but from the documentation and the question &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/7396/how-to-allow-half-windows-in-biopython&quot;&gt;How to allow &amp;quot;half windows&amp;quot; in biopython?&lt;/a&gt; I understand that the weights are symmetric and calculated automatically given a window size.&lt;/p&gt;&#xA;&lt;p&gt;I'd like to understand what the idea behind this was, just to check that I understand this correctly. For example, if one calculates eg. hydrophobicity of a protein, why should it be a value per residue that depends on a particular window?&lt;/p&gt;&#xA;&lt;p&gt;I was looking for a functionality like where the value given for each residue in the ParamData dictionary would be multiplied by a specified weight, like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;```Y = ProteinAnalysis('MSTAEFAQLL')&#xA;Y.protein_scale(Bio.SeqUtils.ProtParamData.kd,10)&#xA;Y.protein_scale(Bio.SeqUtils.ProtParamData.kd,10,edge[0,0.2,0.3,0.1,0.,0.4,0.5,0.8,0.8,1.])```&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Of course I can code this myself, but was wondering if that functionality already existed.&lt;/p&gt;&#xA;" OwnerUserId="9305" LastActivityDate="2020-08-02T11:39:50.360" Title="Modifying protein_scale in ProteinAnalysis in SeqUtil in Biopython?" Tags="|proteins|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14068" PostTypeId="2" ParentId="14061" CreationDate="2020-08-01T09:10:59.460" Score="1" Body="&lt;p&gt;I found a possible answer to my own question by reading this &lt;a href=&quot;https://www.biostars.org/p/386742/&quot; rel=&quot;nofollow noreferrer&quot;&gt;biostar&lt;/a&gt; page and also finding an NCBI page for downloading the reference genome file for viruses.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;You should go to this NCBI &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genomes/GenomesGroup.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;page&lt;/a&gt; and search for your target virus.&lt;/li&gt;&#xA;&lt;li&gt;Then click on the virus name and you will be redirected to a page like &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?lvl=0&amp;amp;id=11277&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; which gives you an Entrez records table. You should click on the direct link in the genome row in the table.&lt;/li&gt;&#xA;&lt;li&gt;Click on &lt;code&gt;Genome Assembly and Annotation report &lt;/code&gt; on the new page. You will be redirected to a page like &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genome/browse/#!/viruses/4858/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; with some information on reference files submitted to NCBI. The last column in this table is for links to the NCBI FTP site. There are two options to access RefSeq or GenBank FTPs. You can choose either of them. I chose RefSeq. Then, you will be redirected to a page like &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/002/815/995/GCF_002815995.1_ASM281599v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; with different fasta files for that specific virus.&lt;/li&gt;&#xA;&lt;li&gt;the file with the name &lt;code&gt;GCF_002815995.1_ASM281599v1_cds_from_genomic.fna.gz&lt;/code&gt; is the transcriptome file annotated with CDS info which can be used for Kallisto alignment.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: If you couldn't find your desired virus genome in the NCBI page mentioned above, you can proceed with the method for generating your desired transcriptome file from genome and annotation file using &lt;code&gt;gffread&lt;/code&gt; package. The details on this method are provided in the answer to the Biostar question mentioned above.&lt;/p&gt;&#xA;" OwnerUserId="9299" LastActivityDate="2020-08-01T09:10:59.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14069" PostTypeId="2" ParentId="14028" CreationDate="2020-08-01T11:41:21.840" Score="2" Body="&lt;p&gt;The correct method is laborious and it is better to give the students a ready made tree at &lt;a href=&quot;https://www.gisaid.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;GISAID&lt;/a&gt; to investigate the dispersion of COVID-19 into Europe.&lt;/p&gt;&#xA;&lt;p&gt;However, a quick approach to getting to the alignment and drawing a tree is easy and will readily complement your established teaching method. What this will give you is a very different phylogeny from &lt;code&gt;edit distances&lt;/code&gt; and you would explain the matrix differences between the approaches. I suspect NCBI use Jukes Cantor distances.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Go to blast.ncbi.nlm.nih.gov&lt;/li&gt;&#xA;&lt;li&gt;Select nucleotides&lt;/li&gt;&#xA;&lt;li&gt;Paste &amp;quot;MN988669&amp;quot; and hit return&lt;/li&gt;&#xA;&lt;li&gt;This will produce 100 hits&lt;/li&gt;&#xA;&lt;li&gt;On the menu page select the &amp;quot;draw tree option&amp;quot;&lt;/li&gt;&#xA;&lt;li&gt;This will produce a reasonable nucleotide tree for 100 COVID-19 sequences&lt;/li&gt;&#xA;&lt;li&gt;Select &amp;quot;neighbor-joining&amp;quot; rather than &amp;quot;minimum evolution&amp;quot; (this is hierarchical clustering geared to towards mutation rate heterogeneity between taxa)&lt;/li&gt;&#xA;&lt;li&gt;There are various point and click options including &amp;quot;examine alignment&amp;quot;&lt;/li&gt;&#xA;&lt;li&gt;You can then select &amp;quot;minimum evolution&amp;quot; and see the changes in the tree (and there are changes) - TEACHING OBJECTIVE 1 &amp;amp; 2&lt;/li&gt;&#xA;&lt;li&gt;If you want to examine COVID-19 from the European perspective then select a European isolate, e.g. French isolates and input this into the blast. However, European isolates are basically Wuhan original.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I have provided an example below, which I &amp;quot;rerooted&amp;quot; and represents the closest 100 to you sequence using collapsed clade format (your students can undo this, to examine the contents of a &amp;quot;collapsed clade&amp;quot;). The tree shows the dispersion of strains from the Wuhan seafood market.&lt;/p&gt;&#xA;&lt;p&gt;There's lots of flexibility and the students can do everything inside ½ hour easily and this would complement your approach. The advantage of my approach to teaching phylogeny and regardless of what  you do and how you do it Blast is central to rapidly obtaining alignment data for students and investigators alike. We use different blast options but blasting through is prerequisite to understand the diversity and clean some information on the population structure.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RtUed.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RtUed.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Also, are there any recommended data sets where we could do something like track mutations in the virus (and see, e.g., that the NYC outbreak originated from Europe and not China)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes there are, today this would be Mesquite available &lt;a href=&quot;https://www.mesquiteproject.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. In my opinion this is a bit advanced. You can track amino acid mutations, which in my view is easier.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-08-01T12:49:00.007" LastActivityDate="2020-08-01T12:49:00.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14071" PostTypeId="2" ParentId="14064" CreationDate="2020-08-01T17:36:56.963" Score="0" Body="&lt;p&gt;I'm afraid none of your code makes much sense. You have written a function that expects an array, and then you are passing it a single letter&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;for gene in genelist:&#xA;    for codon in gene:&#xA;        mRNA= transcription(codon)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then, the function also doesn't make sense. What does &lt;code&gt;next(codon)&lt;/code&gt; mean? How would your script know what a codon is or how to move to the next one? And your &lt;code&gt;if codon == 'ATG':&lt;/code&gt; is within the &lt;code&gt;if codon != 'ATG':&lt;/code&gt; block, so it will never be executed by definition.&lt;/p&gt;&#xA;&lt;p&gt;I think what you are looking for is something like this, using the same basic approach as your script:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/bin/python3&#xA;from Bio.Seq import Seq&#xA;from Bio.Seq import transcribe &#xA;&#xA;genelist=[]&#xA;&#xA;gene_1= 'A','C','G','G','A','C','T','A','T','T','C'&#xA;gene_2= 'G','G','C','C','A','T','G','A','G','T','A','A','C','G','C','A','T','A','G','G','G','C','C','C'&#xA;gene_3='G','G','G','C','C','C','A','T','G','A','C','G','T','A','C','T','A','G','G','G','G','C','C','C','A','T','G','C','A','T','T','C','A','T','A','G'&#xA;&#xA;genelist.append(gene_1)&#xA;genelist.append(gene_2)&#xA;genelist.append(gene_3)&#xA;&#xA;def transcription(ORF):&#xA;    mRNA= ''&#xA;    foundStart = False&#xA;    foundEnd = False&#xA;    for i in range(0, len(ORF), 3):&#xA;        codon= &amp;quot;&amp;quot;.join(ORF[i:i+3])&#xA;        if codon == 'ATG' and not foundStart:&#xA;            foundStart = True&#xA;        if foundStart and not foundEnd:&#xA;            cc=transcribe(codon)&#xA;            mRNA = mRNA + transcribe(codon)&#xA;        if codon == 'TAG':&#xA;            foundEnd = True&#xA;       &#xA;    return(mRNA)&#xA;&#xA;mRNAs=[]&#xA;for gene in genelist:&#xA;    mRNA = transcription(gene)&#xA;    mRNAs.append(mRNA)&#xA;print(mRNAs)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which returns:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ foo.py &#xA;['', '', 'AUGACGUACUAG']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Of course, it only returns one mRNA since you only have one gene with an ORF in the 1st frame and your script was only looking at the forward strand and the first frame. In reality, ORFs can be in any of the 6 possible frames. You are also assuming that all ORFs will end with a TAG which is perhaps the case in the species you are working in? I don't remember if there is any genetic code that has TAG as the &lt;em&gt;only&lt;/em&gt; stop.&lt;/p&gt;&#xA;&lt;p&gt;If you do actually need to find ORFs in any frame, then I hope the script above should be enough to get you started.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2020-08-01T17:36:56.963" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14073" PostTypeId="2" ParentId="14064" CreationDate="2020-08-02T01:13:56.207" Score="0" Body="&lt;p&gt;Just in case this &lt;em&gt;isn't&lt;/em&gt; a homework problem, there are already plenty of programs available to do this. One program I use is &lt;code&gt;getorf&lt;/code&gt; from the EMBOSS toolkit:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://emboss.bioinformatics.nl/cgi-bin/emboss/getorf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://emboss.bioinformatics.nl/cgi-bin/emboss/getorf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2020-08-02T01:13:56.207" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14074" PostTypeId="1" CreationDate="2020-08-02T03:13:59.903" Score="1" ViewCount="40" Body="&lt;p&gt;Has anyone worked with the IMGT HLA database/dataset before? &lt;a href=&quot;https://github.com/ANHIG/IMGTHLA&quot; rel=&quot;nofollow noreferrer&quot;&gt;IMGT-HLA git repo&lt;/a&gt; they have some convenient text files (eg. &lt;a href=&quot;https://raw.githubusercontent.com/ANHIG/IMGTHLA/Latest/alignments/A_gen.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;link to gene A .txt file&lt;/a&gt;) with the genomic data for the different alleles of each gene; &lt;strong&gt;what is the name of this type of alignment format&lt;/strong&gt;? sample:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# file: A_gen.txt&#xA;# date: 2020-07-13&#xA;# version: IPD-IMGT/HLA 3.41.0&#xA;# origin: http://hla.alleles.org/wmda/A_gen.txt&#xA;# repository: https://raw.githubusercontent.com/ANHIG/IMGTHLA/Latest/alignments/A_gen.txt&#xA;# author: WHO, Steven G. E. Marsh (steven.marsh@ucl.ac.uk)&#xA;                   &#xA; gDNA              -300&#xA;                   |&#xA; A*01:01:01:01     CAGGAGCAGA GGGGTCAGGG CGAAGTCCCA GGGCCCCAGG CGTGGCTCTC AGGGTCTCAG GCCCCGAAGG CGGTGTATGG ATTGGGGAGT CCCAGCCTTG &#xA; A*01:01:01:02N    ********** ********** ********** ********** ********** ********** ********** *********- ---------- ---------- &#xA; A*01:01:01:03     ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- ---------- &#xA; A*01:01:01:04     ********** ********** ********** ********** ********** ********** ********** ********** ********** ********** &#xA;...&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There is the &lt;code&gt;.msf&lt;/code&gt; format of the same dataset at, &lt;a href=&quot;https://raw.githubusercontent.com/ANHIG/IMGTHLA/Latest/msf/A_gen.msf&quot; rel=&quot;nofollow noreferrer&quot;&gt;HLA gene A in .msf format, git repo&lt;/a&gt;, and there are 2 types of characters in the data [A,C,G,T,.], where the '.' denotes the gap (correct?). For this above, unknown to me, format there are 3 extra symbols, [A,C,G,T,.,&lt;span class=&quot;math-container&quot;&gt;$*$&lt;/span&gt;,|,_]. I assume that the symbol '|' (pipe character) is for the exon, then what do the symbol characters &lt;span class=&quot;math-container&quot;&gt;$'*'$&lt;/span&gt;, '&lt;span class=&quot;math-container&quot;&gt;$\_$&lt;/span&gt;', and '.' denote in this format? I assume that the '.' still denotes the gap, so what is the &lt;span class=&quot;math-container&quot;&gt;$'*'$&lt;/span&gt; and '&lt;span class=&quot;math-container&quot;&gt;$\_$&lt;/span&gt;'? Does &lt;span class=&quot;math-container&quot;&gt;$'*'$&lt;/span&gt; reference missing data in terms of the reference sequence? If so the .msf format lacks this information? and would it be reasonable to assume that '&lt;span class=&quot;math-container&quot;&gt;$\_$&lt;/span&gt;' is that the data at that base pair is equal to the reference sequence as sequence 1?&lt;/p&gt;&#xA;&lt;p&gt;If my assumption is correct and this new dataset accounts for 'missing genomic data' in the alignment and the msf does not, then the .msf is penalizing with a gap data that is actually missing?&lt;/p&gt;&#xA;&lt;p&gt;(the FASTA data for this gene, &lt;a href=&quot;https://raw.githubusercontent.com/ANHIG/IMGTHLA/Latest/fasta/A_gen.fasta&quot; rel=&quot;nofollow noreferrer&quot;&gt;link to the fasta file&lt;/a&gt;, shows that the reads for allele are of a different length)&lt;/p&gt;&#xA;" OwnerUserId="9311" LastEditorUserId="9311" LastEditDate="2020-08-02T03:25:10.687" LastActivityDate="2020-08-02T03:25:10.687" Title="What are the symbols of $*$ and '$\_$' in an unknown alignment format for HLA data from the IMGT dataset?" Tags="|sequence-alignment|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14075" PostTypeId="1" CreationDate="2020-08-02T07:56:49.133" Score="0" ViewCount="48" Body="&lt;p&gt;I have a dataset of RNA-seq samples for testing different drugs on the presence of another drug. One of my samples is the normal cells with no drugs (control negative) and another is the cells with just the base drug (control positive). All of the other samples are cells with the base drug and another testing drug. So, my question is how to do design a proper Design Matrix for comparing these samples. I can separately compare with control negative and control positive. But, my intention is to somehow neutralize the effect of one of the controls in all the samples and compare to the other one. Is that possible? How should I design the matrix?&lt;/p&gt;&#xA;" OwnerUserId="9299" LastActivityDate="2020-08-02T07:56:49.133" Title="a proper Design Matrix for several drug treatments with both control negative and control positive" Tags="|rna-seq|bioconductor|differential-expression|deseq2|edger|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="14076" PostTypeId="2" ParentId="14066" CreationDate="2020-08-02T11:39:50.360" Score="3" Body="&lt;p&gt;There is not a way to control the edge weights as written, but making a modified version is straightforward. Namely make a class that inherits the class &lt;code&gt;Bio.SeqUtils.ProtParam.ProteinAnalysis&lt;/code&gt; and overwrites the method relevant methods.&lt;/p&gt;&#xA;&lt;p&gt;Having a gander at the files one sees the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def protein_scale(self, param_dict, window, edge_list: list):&#xA;    &amp;quot;&amp;quot;&amp;quot;[...]&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    # generate the weights&#xA;    #   _weight_list returns only one tail. If the list should be&#xA;    #   [0.4,0.7,1.0,0.7,0.4] what you actually get from _weights_list&#xA;    #   is [0.4,0.7]. The correct calculation is done in the loop.&#xA;    weights = self._weight_list(window, edge)&#xA;    scores = []&#xA;    ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So the weights are generated by the 'private' method &lt;code&gt;._weight_list&lt;/code&gt;. Underscore means it is for internal use and for production code it is bad to assume that the name of the private method many not change, but Biopython has not changed in years so you'd be safe. And this is for a quick hack anyway...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;class JankyProteinAnalysis(Bio.SeqUtils.ProtParam.ProteinAnalysis):&#xA;    overriding_weights = None&#xA;    &#xA;    def _weight_list(self, window, edge):&#xA;          if self.overriding_weights is not None: # dodgy override!&#xA;               print(&amp;quot;Weight method hack done!&amp;quot;)&#xA;               return overriding_weights&#xA;          else: # behave normally&#xA;               return super()._weight_list(window, edge)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So you'd do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Y = JankyProteinAnalysis('MSTAEFAQLL')&#xA;Y.overriding_weights = [0,0.2,0.3,0.1,0.,0.4,0.5,0.8,0.8,1.]&#xA;Y.protein_scale(Bio.SeqUtils.ProtParamData.kd,10,float('nan'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that the &lt;code&gt;edge&lt;/code&gt; argument is no longer doing anything.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-08-02T11:39:50.360" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14077" PostTypeId="1" CreationDate="2020-08-03T09:23:16.843" Score="1" ViewCount="215" Body="&lt;p&gt;&lt;em&gt;[this question has also been posted on &lt;a href=&quot;https://www.biostars.org/p/452978/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;; some additional clarification from there has been copied into this question]&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I've been asked to analyse a set of samples in which their control sample has been done in duplicate (biological replicate) but the other samples have just one sample each.&lt;/p&gt;&#xA;&lt;p&gt;The analysis purpose is just hypothesis generation for further experiment design. So, is it possible to have duplicate samples for one or two of the groups and use one sample for other groups? Could we estimate a constant dispersion from these replicates and apply it to all of the samples for differential gene expression analysis.&lt;/p&gt;&#xA;&lt;p&gt;I'm using &lt;code&gt;EdgeR/Limma&lt;/code&gt; for my differential gene expression analysis. When there are no replicates the &lt;code&gt;eBayes&lt;/code&gt; function gives me an error. However, if I add a replicate to just one of the samples, it won't give any errors. Does this mean that it uses the calculated dispersion from the duplicate condition for all of the other one sample conditions? Or is it a bug that it doesn't give an &lt;code&gt;error&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Should I consider this method of using duplicates for some samples instead of just using no replicates in such large experiments that are being done for hypothesis generation?&lt;/p&gt;&#xA;&lt;p&gt;I have read the EdgeR &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/vignettes/edgeR/inst/doc/edgeRUsersGuide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;recommendations&lt;/a&gt; (page 23-24) on what to do without (enough) replicates, but the explanation is not complete. Like it doesn't say how to find these housekeeping genes. About its first case of using a common fixed dispersion. I guess what I'm asking here is I'd like a more reliable measure of this common fixed dispersion. I say instead of using a theoretical dispersion why not try to estimate dispersion from the few duplicate samples and then use it for all the samples. I'm asking if someone has already done this thing and I'd like to know about their experience and how reloable could thing method be.&lt;/p&gt;&#xA;" OwnerUserId="9299" LastEditorUserId="73" LastEditDate="2020-08-05T10:37:05.163" LastActivityDate="2020-08-05T10:37:05.163" Title="Differential Gene Expression with Replicates for some of the samples" Tags="|rna-seq|differential-expression|deseq2|edger|limma|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14078" PostTypeId="1" CreationDate="2020-08-03T10:11:21.107" Score="1" ViewCount="88" Body="&lt;p&gt;I encountered a problem while performing analysis using the PANTHER website. I run the statistical overrepresentation test for a genelist of 99 genes (GO-Slim BP, Binomial, Bonferroni) in February 2020. My result was Bonferroni count = 1 and a list of several enriched GO terms. Next we run the same analysis with the same settings in July 2020 (two days ago) and the result is dramatically different, the Bonferroni count is 1770 and no enriched terms are found.&lt;/p&gt;&#xA;&lt;p&gt;I am confused - is the result Bonferroni count = 1 possible? Why these results are so different?&lt;/p&gt;&#xA;&lt;p&gt;This is how the correction is described on &lt;a href=&quot;http://pantherdb.org/tips/tips_bonferroni.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;PANTHER website&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The expression data analysis statistics now include a Bonferroni correction for multiple testing. The Bonferroni correction is important because we are performing many statistical tests (one for each pathway, or each ontology term) at the same time. This correction multiplies the single-test P-value by the number of independent tests to obtain an expected error rate.&lt;/p&gt;&#xA;&lt;p&gt;For pathways, we now correct the reported P-values by multiplying by the number of associated pathways with two or more genes. Some proteins participate in multiple pathways, so the tests are not completely independent of each other and the Bonferroni correction is conservative. For ontology terms, the simple Bonferroni correction becomes extremely conservative because parent (more general) and child (more specific) terms are not independent at all: any gene or protein associated with a child term is also associated with the parent (and grandparent, etc.) terms as well.&lt;/p&gt;&#xA;&lt;p&gt;To estimate the number of independent tests for an ontology, we count the number of classes with at least two genes in the reference list that are annotated directly to that class (i.e. not indirectly via an annotation to a more specific subclass).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9316" LastEditorUserId="48" LastEditDate="2020-09-03T15:18:26.497" LastActivityDate="2024-01-17T00:01:10.157" Title="Bonferroni count = 1 in GO-Slim overrepresentation test (PANTHER) is it possible?" Tags="|statistics|gene-ontology|go-enrichment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14079" PostTypeId="2" ParentId="14078" CreationDate="2020-08-03T10:40:57.417" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;GO-Slim BP, Binomial, Bonferroni&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This seems easy you because I suspect have run too many tests at 5%.&lt;/p&gt;&#xA;&lt;p&gt;If you run 10 tests at 5% rejection threshold, the Bonferroni correction threshold is 0.5% - you divide the number of tests by the rejection threshold.&lt;/p&gt;&#xA;&lt;p&gt;One possibility is the rejection threshold is given in number of genems: in one instance you've run a pairwise test with several thousand genes so the rejection threshold has been set accordingly. The other test you ran I suspect was a multi-variate test, thus the shift in rejection threshold measured in number of genes.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The altnative is that Bonf = 1 is in fact &lt;strong&gt;P&lt;/strong&gt; = 1, in other words there is no significant difference at all.  If &lt;strong&gt;P&lt;/strong&gt; = 1 then this could be no genes show deviation hence 1770, because 1770/1770 =1&lt;/p&gt;&#xA;&lt;p&gt;Looking at it again the above seems a better explanation.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-08-03T17:22:44.130" LastActivityDate="2020-08-03T17:22:44.130" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14080" PostTypeId="2" ParentId="13986" CreationDate="2020-08-04T13:08:23.503" Score="2" Body="&lt;p&gt;Its very easy, just let the ML sort this out for you and that is its advantage, You're thinking of GLM style calculation where you pre-screen the data with bivariate plots, where there needs to be nice Q-Q plots and low residual.&lt;/p&gt;&#xA;&lt;p&gt;For ML simply include the gene length as one of your parameters along with CpG etc ... and the ML regression analysis SVC, lasso, ridge, random forest will figure the relationship out between gene length and CpG. You do zero, the ML does everything, hence from a statistical point of view purists object because you don't know the relationshiop the ML has deduced between the variables, but you will get regression weights for non-DNN stuff, which will give you some idea of the impact of length.&lt;/p&gt;&#xA;&lt;p&gt;There is the issue of transformations and that can be complicated, but I'd try untransformed data first. The only disadvantage of this approach is the user will have to input the gene size when they want to check out your training algorithm.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-08-04T13:08:23.503" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="14081" PostTypeId="1" CreationDate="2020-08-04T20:40:18.890" Score="1" ViewCount="250" Body="&lt;p&gt;I have attached a histogram plot of the number of genes per cell in a single cell RNA seq data set of lung endothelial cells. I do not find a bimodal or multimodal distribution of the number of genes per cell within any particular condition. Doublets/multiplets should have an aberrantly high number of gene expression and if many, can lead to a bi or multimodal histogram plot of the number of genes detected per cell.  Can I conclude that I have an insignificant presence of doublets or multiplets in my data set from this plot and hence continue with downstream analyses?&lt;/p&gt;&#xA;&lt;p&gt;I am reluctant to use automatic doublet-finding algorithms because they have a high possibility to remove cells in a transitional differentiating phase. I also think that the default threshold of 2500 genes per cell given by Seurat might sometimes lead to the loss of good cells depending on the cell type.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/olCSB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/olCSB.png&quot; alt=&quot;nGene per cell&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Apart from trajectory analyses, are there any other analyses on single cell RNA seq data that are sensitve to doublets/multiplets?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance for your help.&lt;/p&gt;&#xA;" OwnerUserId="3968" LastActivityDate="2021-05-29T14:01:15.090" Title="What is the best way to address the question of doublets and multiplets in a single cell RNA seq data set?" Tags="|rna-seq|scrnaseq|seurat|single-cell|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14082" PostTypeId="1" AcceptedAnswerId="16085" CreationDate="2020-08-04T22:56:07.480" Score="1" ViewCount="48" Body="&lt;p&gt;I have RPK values (RNA seq) and I'm wondering what is a good rule of thumb for what is considered to be noise versus what is considered to be signal? I.e what should I choose as a threshold value for what is considered noise versus not noise? (just as a rough guideline)&lt;/p&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="8601" LastEditDate="2020-08-04T23:13:13.083" LastActivityDate="2021-07-08T14:06:35.650" Title="What is a good rule of thumb for the threshold of noise versus signal for RPK in RNA seq?" Tags="|rna-seq|normalization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14083" PostTypeId="1" AcceptedAnswerId="14234" CreationDate="2020-08-05T01:56:31.837" Score="1" ViewCount="464" Body="&lt;p&gt;On short reads, Kallisto/Salmon is a standard workflow for measuring RNA transcript counts in TPM. However, when I tried to Google a similar workflow for long reads, it's not clear there is a replacement.&lt;/p&gt;&#xA;" OwnerUserId="174" LastActivityDate="2021-03-22T19:11:48.430" Title="Recommended workflow for RNA long read Kallisto-like TPM estimation?" Tags="|rna-seq|nanopore|long-reads|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14084" PostTypeId="1" AcceptedAnswerId="14088" CreationDate="2020-08-05T03:06:12.693" Score="1" ViewCount="115" Body="&lt;p&gt;For Bisulfite Sequencing data, we can calculate &lt;strong&gt;methylation level&lt;/strong&gt; for a specific locus,&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\mbox{methylation level} = \frac{\mbox{# methelyated reads}}{\mbox{# total reads}}&#xA;$$&lt;/span&gt;&#xA;Here I think, for this locus, it can only have one status, methelyated or not. Or to say, &lt;strong&gt;methylation level&lt;/strong&gt; here can only be 0 or 1. Why does it have other values in (0,1)?&lt;/p&gt;&#xA;" OwnerUserId="9327" LastActivityDate="2020-08-05T10:12:40.117" Title="How to understand methylation level?" Tags="|sequence-alignment|sequence-analysis|methylation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14086" PostTypeId="1" AcceptedAnswerId="14486" CreationDate="2020-08-05T07:28:03.967" Score="1" ViewCount="89" Body="&lt;p&gt;I have scRNA-seq data held in a &lt;code&gt;SEURAT&lt;/code&gt; object, and cell-types were identified with &lt;code&gt;SingleR&lt;/code&gt; (essentially a vector of strings - Monocytes, Granulocytes etc.). I have two conditions - treated and untreated. I want to know if the difference in the relative frequency (%) of a given cell type between the conditions is significant. For this, I thought to perform bootstrapping, to obtain frequencies for many different 'simulated' datasets - and then perform a statistical test. However, I cannot understand how best to do this. Should I subsample the scRNA-seq data, with replacement and calculate the frequencies each time? Can somebody provide a step by step workflow on how to do this?&lt;/p&gt;&#xA;" OwnerUserId="9329" LastActivityDate="2020-10-05T17:24:05.533" Title="Bootstrapping cell type identifications (scRNA-seq)" Tags="|scrnaseq|statistics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14087" PostTypeId="1" AcceptedAnswerId="14089" CreationDate="2020-08-05T10:06:41.227" Score="0" ViewCount="231" Body="&lt;p&gt;I am trying to download multiple fastq files from the SRA NCBI database which, conveniently enough, have their IDs in a range. Building a simple loop doesn't seem to do the trick, as bash appears to interpret my variable as an additional command? Help would be appreciated. This is what I have for now:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in {256..345};do&#xA;    sra_code = &amp;quot;DRR163&amp;quot;&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$i&quot;&#xA;fasterq-dump &quot;$&lt;/span&gt;sra_code&amp;quot; -p -v --outdir ./output/ -e 8 &#xA;&#xA;done&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2020-08-10T16:08:14.737" Title="Using variables with fasterq-dump?" Tags="|bash|sratoolkit|fastq-dump|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14088" PostTypeId="2" ParentId="14084" CreationDate="2020-08-05T10:12:40.117" Score="6" Body="&lt;p&gt;You are right that a single molecule in a single position is either methylated or not methylated. However:&lt;/p&gt;&#xA;&lt;p&gt;First, assuming your organism of interest is diploid (or of higher ploidy) one of the chromosomes could be methylated, the other not. That would give you a level of 0.5 and can be found in imprinted regions (where the paternally inherited chromosome is methylated and maternal not, or vice versa).&lt;/p&gt;&#xA;&lt;p&gt;Second, except if you use a pure cell population of a single cell type it is likely that there will be cellular heterogeneity in your sample, e.g. from different types of blood cells or neurons vs glial cells in the brain. Some of these may have methylated CpGs in this locus (depending on cell-type-specific gene activity) while others can be unmethylated. Depending on the relative proportions of these cell types the methylation level can be anything between 0 and 1 for a mixture of cell types.&lt;/p&gt;&#xA;&lt;p&gt;Thirdly, sometimes CpGs get grouped together. For example, 5 nearby CpGs are combined to give a single methylation level for this locus, and then the level can be 0; 0.2; 0.4; 0.6; 0.8 and 1.&lt;/p&gt;&#xA;&lt;p&gt;Lastly, there could be technical artifacts or technical reasons why a position was not called as methylated (insufficient quality for a single read?), leading to deviation from 0 or 1.&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2020-08-05T10:12:40.117" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14089" PostTypeId="2" ParentId="14087" CreationDate="2020-08-05T10:25:33.053" Score="5" Body="&lt;p&gt;You have an obvious error in your shell script that should have given you an error message:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sra_code = &amp;quot;DRR163&amp;quot;&amp;quot;$i&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Should be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sra_code=&amp;quot;DRR163&amp;quot;&amp;quot;$i&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You cannot have spaces around the &lt;code&gt;=&lt;/code&gt; sign in a variable assignment. The line &lt;code&gt;sra_code = &amp;quot;DRR163&amp;quot;&amp;quot;$i&amp;quot;&lt;/code&gt; means &amp;quot;run the command &lt;code&gt;sra_code&lt;/code&gt; with the arguments &lt;code&gt;= &amp;quot;DRR163&amp;quot;&amp;quot;$i&amp;quot;&lt;/code&gt;. Since there is no such command, you see an error.&lt;/p&gt;&#xA;&lt;p&gt;Here's a working version of your command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in {256..345}; do &#xA;    sra_code=&amp;quot;DRR163&lt;span class=&quot;math-container&quot;&gt;$i&quot;&#xA;    fasterq-dump &quot;$&lt;/span&gt;sra_code&amp;quot; -p -v --outdir ./output/ -e 8&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2020-08-05T13:51:11.723" LastActivityDate="2020-08-05T13:51:11.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14090" PostTypeId="1" CreationDate="2020-08-05T10:33:38.407" Score="1" ViewCount="117" Body="&lt;p&gt;I am trying to draw venn diagrams with a GenomicVis package (R).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(GenomicVis)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It is loaded with a warning:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Warning messages:&#xA;1: replacing previous import ‘IRanges::shift’ by ‘data.table::shift’ when loading ‘GenomicVis’ &#xA;2: replacing previous import ‘S4Vectors::second’ by ‘data.table::second’ when loading ‘GenomicVis’ &#xA;3: replacing previous import ‘S4Vectors::first’ by ‘data.table::first’ when loading ‘GenomicVis’ &#xA;&#xA;f2 &amp;lt;- system.file(&amp;quot;extdata&amp;quot;, &amp;quot;S038_10_wes_S43.vcf&amp;quot;, package =&amp;quot;GenomicVis&amp;quot;)&#xA;f1 &amp;lt;- system.file(&amp;quot;extdata&amp;quot;, &amp;quot;S038_1_wes_S25.vcf.gz&amp;quot;,package = &amp;quot;GenomicVis&amp;quot;)&#xA;vcf.files &amp;lt;- c(f1, f2)&#xA;sample.names &amp;lt;- c('Sample 1','Sample 10') &#xA;v &amp;lt;- vcf.venn(vcf.files, 'GRCh38', sample.names)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But then this Error comes up, no clue how to solve it:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in rowData(read.vcf(f, genome, ...)) : &#xA; could not find function &amp;quot;rowData&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also think that the problem is reading my vcf files as calling f1 or f2 results in [1] &amp;quot;&amp;quot;, however when I open them with cat I can see vcf table as it is. The first line is ##fileformat=VCFv4.2. I tried using fullpath or running R from this folder and still the file was not loaded as system file is used only for loading example files from a package.&lt;/p&gt;&#xA;&lt;p&gt;Loading file with a readVcf('S038_10_wes_043.vcf', 'hg38') worked well.&lt;/p&gt;&#xA;" OwnerUserId="9330" LastEditorUserId="73" LastEditDate="2021-05-07T08:42:30.407" LastActivityDate="2021-05-07T08:42:30.407" Title="Error running GenomicVis vcf.venn(vcf.files, 'GRCh38', sample.names)" Tags="|r|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14091" PostTypeId="1" CreationDate="2020-08-05T12:12:48.533" Score="1" ViewCount="213" Body="&lt;p&gt;I want to investigate, how much sample size i needed to obtain 80% power for GWAS/EWAS studies. Phynotype trait is discrete (not case/control) for human disease.&lt;/p&gt;&#xA;&lt;p&gt;I wonder, does anyone has came across this? Really appreciate any suggestion or any tool/software which can answer this?&lt;/p&gt;&#xA;" OwnerUserId="2365" LastEditorUserId="2365" LastEditDate="2020-08-05T14:29:35.740" LastActivityDate="2020-08-05T14:29:35.740" Title="Power calculation for GWAS/EWAS" Tags="|bioconductor|gwas|human-genome|phylogenetics|molecular-genetics|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="14092" PostTypeId="2" ParentId="14091" CreationDate="2020-08-05T12:29:43.440" Score="1" Body="&lt;p&gt;Interesting question I would recommend the effective population size &lt;span class=&quot;math-container&quot;&gt;$N_{e}$&lt;/span&gt;,  and error bars around this. Ne can be calculated via &lt;span class=&quot;math-container&quot;&gt;$F_{st}$&lt;/span&gt; and is perhaps the most important parameter to determine the genetic population size.&lt;/p&gt;&#xA;&lt;p&gt;Under the Fisher definition &lt;span class=&quot;math-container&quot;&gt;$N_{e}$&lt;/span&gt; is the number of breeding individuals in the population.&lt;/p&gt;&#xA;&lt;p&gt;The central issue with the approach you are suggesting is that population size does not equate to genetic population size. Thus the population could be huge, but if it is genetically homogeneous it will appear very small. In contrast the number of individuals could be very small, but the genetic variation between between them very large so &lt;span class=&quot;math-container&quot;&gt;$N_{e}$&lt;/span&gt; would be huge.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-08-05T12:29:43.440" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14093" PostTypeId="1" CreationDate="2020-08-05T20:42:45.840" Score="1" ViewCount="159" Body="&lt;p&gt;I would like to ask why in RNA seq analysis (alignment step) we use sometimes reference genome instead of reference transcriptome? thank you!&lt;/p&gt;&#xA;" OwnerUserId="8713" LastActivityDate="2020-08-05T21:55:32.733" Title="why in RNA seq don't we only use reference transcriptome?" Tags="|rna-seq|sequence-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14094" PostTypeId="2" ParentId="14093" CreationDate="2020-08-05T20:50:19.523" Score="7" Body="&lt;p&gt;Pseudoaligmnent to a reference transcriptome with something like Kallisto is perfectly valid, assuming you are in an organism whose reference transcriptome is very well documented.&lt;/p&gt;&#xA;&lt;p&gt;But if you want to find novel transcripts or novel splicing junctions, you need to align to genome.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastEditorUserId="2388" LastEditDate="2020-08-05T21:55:32.733" LastActivityDate="2020-08-05T21:55:32.733" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="14095" PostTypeId="1" CreationDate="2020-08-06T17:13:47.563" Score="0" ViewCount="69" Body="&lt;p&gt;I would like to extract all information about each &lt;code&gt;SAMPLE&lt;/code&gt; after running the following query (run the query and add a &lt;code&gt;| grep SAMPLE&lt;/code&gt; for clarification on what I mean by &lt;code&gt;SAMPLE&lt;/code&gt;):&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;esearch -db sra -query PRJNA514750 | efetch -format xml&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I tried the following:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;esearch -db sra -query PRJNA514750 | efetch -format xml | xtract -pattern EXPERIMENT -element SAMPLE&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;but this returns nothing (PS: &lt;code&gt;SAMPLE&lt;/code&gt;s are within an &lt;code&gt;EXPERIMENT&lt;/code&gt; tag). I read in the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK25497/&quot; rel=&quot;nofollow noreferrer&quot;&gt;e-utilities guide&lt;/a&gt; that &lt;code&gt;-pattern&lt;/code&gt; will divide the data into rows, and &lt;code&gt;-element&lt;/code&gt; into columns, so I'm presuming that this didn't work because &lt;code&gt;SAMPLE&lt;/code&gt; has multiple tags within it. So I then tried:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db sra -query PRJNA514750 | efetch -format xml | xtract -pattern SAMPLE -element random_SAMPLE_tag&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where &lt;code&gt;random_SAMPLE_tag&lt;/code&gt; is any tag within &lt;code&gt;SAMPLE&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here's a concrete example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db sra -query PRJNA514750 | efetch -format xml | xtract -pattern SAMPLE -element TITLE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This works, but I want to get all the information about each &lt;code&gt;SAMPLE&lt;/code&gt;, and I do not know beforehand what the tags within it are (I manually got &lt;code&gt;TITLE&lt;/code&gt; in this case), and since I want to get this info for a quite a few studies, I can't manually check this.&lt;/p&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="8601" LastEditDate="2020-08-07T02:37:05.690" LastActivityDate="2020-09-16T20:45:54.253" Title="Extracting all information about a sample when using xtract from e-utilities" Tags="|phylogenetics|e-utilities|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14098" PostTypeId="1" AcceptedAnswerId="14112" CreationDate="2020-08-07T00:05:01.623" Score="5" ViewCount="151" Body="&lt;p&gt;I am reading Section 5.2, Kinship and Inbreeding Coefficients, of &lt;a href=&quot;https://books.google.com/books?id=CBzjBwAAQBAJ&amp;amp;pg=PA81&amp;amp;lpg=PA81&amp;amp;dq=if%20they%20are%20both%20physical%20copies%20of%20the%20same%20ancestral%20gene.%20Two%20genes%20are%20identical%20by%20state%20if%20they%20represent%20the%20same%20allele.%20Identity%20by%20descent%20implies%20identity%20by%20state,%20but%20not%20conversely.%20The%20simplest%20measure%20of%20relationship%20between%20two%20relatives&amp;amp;source=bl&amp;amp;ots=730RYtaQkZ&amp;amp;sig=ACfU3U1XOc2uutgH-akYsZINUPwng_AJOQ&amp;amp;hl=en&amp;amp;sa=X&amp;amp;ved=2ahUKEwihr9qGkv7qAhVCnKwKHQNMCuAQ6AEwBHoECAkQAQ#v=onepage&amp;amp;q=if%20they%20are%20both%20physical%20copies%20of%20the%20same%20ancestral%20gene.%20Two%20genes%20are%20identical%20by%20state%20if%20they%20represent%20the%20same%20allele.%20Identity%20by%20descent%20implies%20identity%20by%20state%2C%20but%20not%20conversely.%20The%20simplest%20measure%20of%20relationship%20between%20two%20relatives&amp;amp;f=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kenneth Lange, Mathematical and Statistical Methods for Genetic Analysis&lt;/a&gt;. There the kinship coefficient &lt;span class=&quot;math-container&quot;&gt;$\Phi_{i,j}$&lt;/span&gt; is defined for two relatives &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; as the probability that a gene selected randomly from &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and a gene selected randomly from the same autosomal locus of &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; are identical by descent.&lt;/p&gt;&#xA;&lt;p&gt;I would think &lt;span class=&quot;math-container&quot;&gt;$\Phi_{i,j}$&lt;/span&gt; should depend on the particular population frequency of the gene or allele at that locus. However, the book does not seem to indicate that or at least does not stress that at all. Is there a dependence or not? The same question applies to the inbreeding coefficient.&lt;/p&gt;&#xA;&lt;p&gt;More importantly, I doubt this definition is mathematically rigorous.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Here is an example to make what puzzles me clearer. It must be that the probability I am computing is not what the kinship coefficient. But this is the definition means to me. Some expert please elucidate the correct meaning of the definition.&lt;/p&gt;&#xA;&lt;p&gt;Consider the simplest pedigree tree of one family of two parents with brothers &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;. Allele &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is detected at a particular locus. Let &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; be the other allele of the same gene. We ask for the probability of the brother &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; having allele &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; conditioned on &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; having allele &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In the following, to clutter symbols, we abuse the symbols by use &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; to denote the population frequency or the unconditional probability of finding allele &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; at the locus for the whole population as &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. The same goes for allele &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$a,b\in[0,1]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$a+b=1$&lt;/span&gt;. We solve this problem by listing as follows all possible parental gene configurations, the configuration probability (considering the symmetry of paternal and maternal loci), as well as the probability finding allele &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; for either &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; at the locus.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{align}&#xA;\text{parent config}~~~~ &amp;amp; \text{Pr(config)} &amp;amp; \text{Pr($a$|config)} \\&#xA;aa|aa~~~~~~~~~  &amp;amp;  ~~a^4   &amp;amp; 1 \\&#xA;aa|ab~~~~~~~~~  &amp;amp;  4a^3b   &amp;amp; 1 \\&#xA;aa|bb~~~~~~~~~  &amp;amp;  2a^2b^2 &amp;amp; 1 \\&#xA;ab|ab~~~~~~~~~  &amp;amp;  4a^2b^2 &amp;amp; \frac34 \\&#xA;ab|bb~~~~~~~~~  &amp;amp;  4ab^3   &amp;amp; \frac12 \\&#xA;bb|bb~~~~~~~~~  &amp;amp;  ~~~~b^4 &amp;amp; 0 \\&#xA;\end{align}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The joint conditional probability of both &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; have allele &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; at the locus is Pr(&lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;|config)&lt;span class=&quot;math-container&quot;&gt;$^2$&lt;/span&gt;. So the required conditional probability is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$P(r):=\text{Pr}(B\text{ has } a|A\text{ has }a)=\frac{\sum_\text{config} \text{Pr(config)Pr(a|config)}^2}{\sum_\text{config} \text{Pr(config)Pr(a|config)}} = \frac{r^3+4r^2+2r+4r(\frac34)^2+\frac4{2^2}}{r^3+4r^2+2r+4r\frac34+\frac42}, \quad r:=\frac ab\in [0,\infty).$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$P(r)$&lt;/span&gt; depends on &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; which depends on the population frequency of the alleles as I have claimed.&lt;/p&gt;&#xA;&lt;p&gt;Consider &lt;span class=&quot;math-container&quot;&gt;$[r^3+4r^2,2r,3r, 2]$&lt;/span&gt; as a positive entry vector. &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; is a convex combination of &lt;span class=&quot;math-container&quot;&gt;$v:=\big[1,1,1,\frac34,\frac12\big]$&lt;/span&gt;. So &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; is between the minimum and maximum of the entries of &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;, and&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\frac12\le P\le 1.$$&lt;/span&gt;&#xA;We see the extrema are achieved at&#xA;&lt;span class=&quot;math-container&quot;&gt;$$P(r)=\begin{cases}&#xA;\frac12, \quad r=0 \\[0.7ex]&#xA;1,\quad r\to\infty&#xA;\end{cases}$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="8725" LastEditorUserId="8725" LastEditDate="2020-08-14T15:42:15.107" LastActivityDate="2020-09-27T20:45:05.503" Title="Does the kinship and inbreeding coefficients depend on population frequency of an allele?" Tags="|phylogenetics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14099" PostTypeId="2" ParentId="14098" CreationDate="2020-08-07T13:29:10.880" Score="0" Body="&lt;p&gt;Kinship selection has a long history in evolutionary genetics, it is where the whole &amp;quot;selfish gene&amp;quot; hypothesis was formed (Maynard Smith, Bill Hamilton, and George Price ). I would be amazed if it was not rigorous. The problem the inuitive understanding of how the co-efficient works in real data appears lacking. So for example &lt;span class=&quot;math-container&quot;&gt;$F_{st}$&lt;/span&gt; simulation studies demonstrate that strong values occur above 0.2.&lt;/p&gt;&#xA;&lt;p&gt;In terms of population size impact on the calculation, this is complex because it is a consequence of &lt;span class=&quot;math-container&quot;&gt;$N_{e}$&lt;/span&gt;, studies have been done on this - and in general extremely small populations don't adhere and tend to exhibit 'Muller's racket&amp;quot; mechanisms. The general opinion, e.g. in HW equilibrium, is that deviation from expected behaviour can occur due to a variety of factors not represented in the original equation, but given a reasonable  &lt;span class=&quot;math-container&quot;&gt;$N_{e}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$F_{st}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$F_{is}$&lt;/span&gt; the essential.&lt;/p&gt;&#xA;&lt;p&gt;In every day terms the parameters can be explored by sensitivity analysis, so e.g. drive the genetic variation to extremes and look at the change in the coefficient, reduce the genetic variation and assess again. Look for past simulation studies etc ...&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-08-07T13:29:10.880" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14100" PostTypeId="1" CreationDate="2020-08-07T15:13:28.540" Score="0" ViewCount="76" Body="&lt;p&gt;I am studying about RNA seq. I have found that in 3'prime rna seq (quantseq, tagseq) we should use a reference  genome  with    good    annotation  (plus known UTRs). My question is why?&#xA;Why could not we use reference transcriptome?&#xA;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="8713" LastActivityDate="2020-08-07T22:32:35.117" Title="why in 3' quantseq should we use reference genome and not transcriptome?" Tags="|rna-seq|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14101" PostTypeId="1" AcceptedAnswerId="14459" CreationDate="2020-08-07T18:51:38.480" Score="6" ViewCount="10562" Body="&lt;p&gt;I'm new to Biopython and I'd like to extract the sequence of residues from a pdb file.&lt;/p&gt;&#xA;&lt;p&gt;My two questions are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What is the simplest way to do this? (Esp. when there is more than one sequence)&#xA;and&lt;/li&gt;&#xA;&lt;li&gt;Should I be worried about &amp;quot;PDBConstructionWarning: WARNING: Chain B is discontinuous&amp;quot;?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;So far, I've obtained the residue sequence via:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;p = PDBParser()&#xA;structure = p.get_structure(&amp;quot;1ppi&amp;quot;, &amp;quot;1ppi.pdb&amp;quot;)&#xA;ppb=PPBuilder()&#xA;for pp in ppb.build_peptides(structure):&#xA;     print(pp.get_sequence())&#xA;&#xA;seq = pp.get_sequence().__str__()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This seems to work well for this molecule. However, is there an easier way, especially when there is more than one sequence?&lt;/p&gt;&#xA;&lt;p&gt;For example, I've read that one can also do&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;res_list = Bio.PDB.Selection.unfold_entities(structure, 'R')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but res_list is not a sequence of residues in str, and I don't know how to convert the output from res_list to a str sequence.&lt;/p&gt;&#xA;&lt;p&gt;In addition (or perhaps because I'm going through the PPBuilder), I've recently gotten a lot of warnings of the type:&#xA;/usr/local/lib/python3.8/site-packages/Bio/PDB/StructureBuilder.py:89: PDBConstructionWarning: WARNING: Chain A is discontinuous at line..&#xA;(For example, with &lt;code&gt;structure = p.get_structure(&amp;quot;5owu&amp;quot;, &amp;quot;5owu.pdb&amp;quot;)&lt;/code&gt;)&lt;/p&gt;&#xA;&lt;p&gt;I've seen a discussion about how to silence such warnings, but should I be worried about this? I've also noticed that when I get these warnings, pp builder seems to give me more sequences then are there.&lt;/p&gt;&#xA;" OwnerUserId="9305" LastActivityDate="2021-04-08T13:57:06.427" Title="Extract residue sequence from pdb file (in biopython, but open to recommendations)" Tags="|python|pdb|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14102" PostTypeId="2" ParentId="14100" CreationDate="2020-08-07T19:09:45.393" Score="0" Body="&lt;p&gt;I'm not sure that's true:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/s41598-019-55434-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41598-019-55434-x&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;QuantSeq. 3′ Sequencing combined with Salmon provides a fast, reliable approach for high throughput RNA expression analysis&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-08-07T19:09:45.393" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14103" PostTypeId="2" ParentId="14100" CreationDate="2020-08-07T22:32:35.117" Score="2" Body="&lt;p&gt;It would be great if you can elaborate on where you &amp;quot;found&amp;quot; this information.&lt;/p&gt;&#xA;&lt;p&gt;In the transcriptome annotation you have a defined number of transcript per genes, but there might be others that are present in your sample and not in the annotation. For example you have treated the cells with some extreme condition that changes the 3' processing machinery.&lt;/p&gt;&#xA;&lt;p&gt;With 3' quantseq, you are left with the read fragment that is at the end of your transcript. If the ends of these transcripts are not part of the sequences in the annotated transcriptome, these will be unaligned.&lt;/p&gt;&#xA;&lt;p&gt;These instances might be rare if your transcriptome annotation is good, but let's say if you find that a gene of interest which should be expressed is not, and you suspect it might be due to a different isoform used, you can always check it with the genome alignment (see where reads might be falling in the gene). With a transcriptome alignment, this will not be possible.&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-08-07T22:32:35.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14104" PostTypeId="1" AcceptedAnswerId="14109" CreationDate="2020-08-08T09:38:00.020" Score="-1" ViewCount="43" Body="&lt;p&gt;I need to use the cut command on a human insulin sequence row wise but i don't know the command.I already ran byte wise command but the gave me column wise results. I cant use the -f option since i need to use a delimiter with it and the seq does not have any delimiters.&#xA;please answer asap&lt;/p&gt;&#xA;" OwnerUserId="9337" LastActivityDate="2020-08-08T20:32:16.893" Title="DNA seq cutting using Linux" Tags="|linux|" AnswerCount="1" CommentCount="3" ClosedDate="2020-08-16T19:58:24.680" ContentLicense="CC BY-SA 4.0" />
  <row Id="14105" PostTypeId="2" ParentId="14101" CreationDate="2020-08-08T10:16:57.923" Score="7" Body="&lt;p&gt;The discontinuity is normal: it is a missing stretch due to lack of density. In PyMOL you see them as dotted lines.&lt;/p&gt;&#xA;&lt;p&gt;I am sorry to say, the easiest/safest way to get the sequences is not using the PDB files...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;data = requests.get(f'https://www.ebi.ac.uk/pdbe/api/pdb/entry/molecules/{code}').json()[code.lower()]&#xA;print(data[0]['sequence'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;One gets back a lot of information: &lt;a href=&quot;https://www.ebi.ac.uk/pdbe/api/pdb/entry/molecules/1UBQ&quot; rel=&quot;noreferrer&quot;&gt;example&lt;/a&gt;. This will tell you when you have multiple copies of the same peptide. The only information you do not get is what the PDB numbering is relative to the canonical Uniprot for that chain, which has to be taken from SIFTS.&lt;/p&gt;&#xA;&lt;p&gt;A side note... Biopython.PDB is one of the best BioPython packages, but I personally I much prefer PyMOL as a python module (&lt;code&gt;pymol2&lt;/code&gt; module) to Biopython.PDB as it is functionally more complete and straightforward even if everything operates in a stage. You might want to check it out.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with pymol2.PyMOL() as pymol:&#xA;    pymol.cmd.fetch('1UBQ', 'prot')&#xA;    print(pymol.cmd.get_fastastr('prot'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-08-08T10:16:57.923" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14106" PostTypeId="1" CreationDate="2020-08-08T10:21:35.287" Score="2" ViewCount="48" Body="&lt;p&gt;I need someone to explain to me from the nuts and bolts how algorithms/ maths is used to work out RNA folding. Explain it to me like I am an alien or child.&lt;/p&gt;&#xA;&lt;p&gt;I am looking at this paper - &lt;a href=&quot;https://eprint.ncl.ac.uk/240069&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://eprint.ncl.ac.uk/240069&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;All the books on maths, and information theory &amp;amp; mathematical methods in bioinformatics are either too basic or fail to describe adequately what the research paper is doing in a way I can grasp.&lt;/p&gt;&#xA;&lt;p&gt;I want to understand how ViennaRNA &amp;amp; NUPACK SPECIFICALLY uses maths to describe and work out RNA folding.&lt;/p&gt;&#xA;&lt;p&gt;What do the math symbols mean, what are they representing, and why? Why do you write it like that?&lt;/p&gt;&#xA;&lt;p&gt;In the paper I mention above, it describes how it calculates single stranded folding as per below?&lt;/p&gt;&#xA;&lt;p&gt;a) What does all this mean?&#xA;x ∈ {0, 1}&#xA;|x |&#xA;= 1 if base i is specified in the&#xA;design of x to be bound and 0 otherwise. Further, let p&#xA;x ∈ [0, 1]|x |&lt;/p&gt;&#xA;&lt;p&gt;^ I know conceptually it is referring to vectors and components, I know that the idea is to represent RNA folding as a graph structure in order to describe mathematically, but I do not know or understand exactly how or what it is describing in terms of the reality of an RNA folding. Two vertices connected by an edge represent a base-pair etc etc, that I get, but I cant seem to understand any further than that.&lt;/p&gt;&#xA;&lt;p&gt;Here is the description from the paper:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Single stranded folding. Firstly, we evaluate the ability of a single&#xA;DNA strand to fold into its specified target structure (as shown in&#xA;Figure 1 top). Using the secondary structure predictor of the&#xA;ViennaRNA 2.0 so‰ware suite [13], we calculate the partition function&#xA;of all secondary structures that x might fold into.1 For any strand x,&#xA;let |x | denote its length and d x ∈ {0, 1} |x | a vector whose&#xA;component d x i&#xA;= 1 if base i is specified in the design of x to be bound and 0 otherwise. Further, let p x ∈ [0, 1]|x | denote a vector whose i-th&#xA;component p x i denotes the Boltzmann probability (obtained from the&#xA;partition function) that base i of strand x is paired with another&#xA;base. The single-stranded folding score Ssf is then defined as the&#xA;normalized Euclidean distance || · || between d x and p x as Ssf(x) =&#xA;1 − 1 |x | ||d x − p x ||. (1) Note that 0 ≤ Ssf ≤ 1 and Ssf(x) = 1 if&#xA;x folds unambiguously into its target structure.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9338" LastEditorUserId="3848" LastEditDate="2022-03-22T17:12:38.027" LastActivityDate="2022-03-22T17:12:38.027" Title="Explaining the algorithm of RNA folding: what each symbol &amp; value represents?" Tags="|rna|algorithms|rna-structure|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14107" PostTypeId="1" AcceptedAnswerId="14110" CreationDate="2020-08-08T14:16:11.973" Score="0" ViewCount="47" Body="&lt;p&gt;I want to install &lt;a href=&quot;http://wasabiapp.org/software/prank/prank_installation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PRANK&lt;/a&gt; on the Windows 10 Linux subsystem (Ubuntu 20.04 LTS), I have followed the installation instructions to no avail.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;wget http://wasabiapp.org/download/prank/prank.source.140603.tgz&#xA;tar xvzf prank.source.140603.tgz&#xA;cd prank-msa/src&#xA;make&#xA;./prank&#xA;&#xA;[ cp prank ~/bin ]&#xA;[ sudo cp prank /usr/bin ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I followed those instructions but got stuck with &lt;code&gt;cd prank-msa/src&lt;/code&gt; I don't know why they want me to enter a directory that doesn't exist. I also followed this video on how to install linux files but it wasn't helpful. Any suggestions would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="9340" LastActivityDate="2020-08-08T23:12:00.963" Title="Installation of PRANK MSA in WLS Ubuntu 20.04 LTS" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14108" PostTypeId="1" CreationDate="2020-08-08T14:49:50.287" Score="1" ViewCount="135" Body="&lt;p&gt;recently I came across a situation where RNASeq sample quality was expresses trough DESeq sizeFactor. So authors reported quality of their samples with respect to some publicly available datasets as good/bad if the scaling factor of their samples was equal/higher/lower than 1. What is more, they computed it using FPKM's from Cufflinks and executed geometric mean calculation over genes (as the one done in DESeq after log transforming the raw counts). Underlying logic being that with FPKM the samples were normalized &amp;quot;within&amp;quot; and with the additional DESeq, normalization &amp;quot;between&amp;quot;, thus ready ready for any downstream analysis (differential expression, AS, etc.)&lt;/p&gt;&#xA;&lt;p&gt;I haven't seen something like that before nor the use of DESeq scaling factor for estimating the quality of a sample (all samples are from the same species/tissue/condition/etc. )&lt;/p&gt;&#xA;&lt;p&gt;Can anyone comment on why is this ok/not ok because I cannot wrap my head around the logic (then again I am just trying to replicate the study, no value contribution on my end.)&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="9341" LastActivityDate="2020-08-10T09:07:08.447" Title="Within and between sample count normalization" Tags="|rna-seq|deseq2|fpkm|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14109" PostTypeId="2" ParentId="14104" CreationDate="2020-08-08T20:32:16.893" Score="0" Body="&lt;p&gt;Use either generic *NIX &lt;code&gt;cut&lt;/code&gt; (you do not need a delimiter if you use &lt;code&gt;-c&lt;/code&gt; option) or &lt;code&gt;head&lt;/code&gt; tools. Even better, use any of a number of bioinformatics tools for sequence manipulation, such as EMBOSS &lt;code&gt;extractseq&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EXAMPLES:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Write into file output.txt characters 1-20 from each line of file input.txt:&#xA;cut -c1-20 input.txt &amp;gt; output.txt&#xA;&#xA;# Write the first 5 lines of input.txt:&#xA;head -n5 input.txt &amp;gt; output.txt&#xA;&#xA;# It is easier if you have your sequence in fasta format.&#xA;# Write characters 1..20 and 100-110 from input.fasta to output.fasta:&#xA;extractseq input.fasta output.fasta -regions &amp;quot;1-20 100-110&amp;quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;One of the easy ways to install EMBOSS suite is using &lt;code&gt;conda&lt;/code&gt; (from bioconda channel).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;SEE ALSO:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;EMBOSS &lt;code&gt;extractseq&lt;/code&gt; help: &lt;a href=&quot;https://www.bioinformatics.nl/cgi-bin/emboss/help/extractseq&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioinformatics.nl/cgi-bin/emboss/help/extractseq&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6251" LastActivityDate="2020-08-08T20:32:16.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14110" PostTypeId="2" ParentId="14107" CreationDate="2020-08-08T23:12:00.963" Score="0" Body="&lt;p&gt;Their instructions are outdated and don't match what exists on their server. The problem you're running into is not at the &lt;code&gt;cd&lt;/code&gt; step but even before that - at your first step even. Your &lt;code&gt;wget&lt;/code&gt; fails because that file does not exist. Go to &lt;a href=&quot;http://wasabiapp.org/download/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://wasabiapp.org/download/&lt;/a&gt; and see for yourself - there is no &lt;code&gt;/prank/&lt;/code&gt; directory in there.&lt;/p&gt;&#xA;&lt;p&gt;I tried using &lt;code&gt;curl&lt;/code&gt;, and it wrote the 404 page as a &lt;code&gt;.tar.gz&lt;/code&gt; file, so if you did that, the tar step would error out.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2020-08-08T23:12:00.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14111" PostTypeId="2" ParentId="14065" CreationDate="2020-08-09T02:33:37.840" Score="1" Body="&lt;p&gt;&lt;code&gt;pileupcolumn.n&lt;/code&gt; shows the total number of the reads that cover this basepair (in this case 24793). However, &lt;code&gt;pileupcolumn.pileups&lt;/code&gt; iterates over the reads that cover this base pair AND this base pair has a minimum quality in those reads. So it has an additional filter for the reads to be considered. You can modify the quality threshold by setting &lt;code&gt;min_base_quality&lt;/code&gt; parameter when you call pileup.&lt;/p&gt;&#xA;&lt;p&gt;To have a similar filter applied to both coverage and reads, you can use &lt;code&gt;piliecolumn.get_num_aligned()&lt;/code&gt; to get the number of segments and &lt;code&gt;pileupcolumn.pileups&lt;/code&gt; to get the segments themselves.&lt;/p&gt;&#xA;" OwnerUserId="1392" LastActivityDate="2020-08-09T02:33:37.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14112" PostTypeId="2" ParentId="14098" CreationDate="2020-08-09T06:29:43.533" Score="2" Body="&lt;p&gt;The definition as set by Kenneth Lange's book is indeed quite vague and thus unrigorous. It is a probability not conditioned on observing any particular allele at a locus thus not on the population frequency of that allele, but conditioned on the following where we focus on one particular locus. It is rigorously defined as follows&lt;/p&gt;&#xA;&lt;p&gt;A graph of lineage contains nodes, which we call persons, of a particular locus. Each node, or person, contains exactly two genes. We draw one and only one directed edge (parental relationship) from a gene of a parent person to the gene of a child person if the latter is inherited from the former.  This is a &lt;a href=&quot;https://en.wikipedia.org/wiki/Directed_acyclic_graph&quot; rel=&quot;nofollow noreferrer&quot;&gt;directed acyclic graph&lt;/a&gt; (DAG). Each gene can be connected by no more than &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; edge directed to (as opposed to away from) it. The two genes in one person cannot be connected by edges. Two persons are said to be connected if there is an edge pointing from a gene of one node to a gene of the other node. Two persons cannot be connected by more than one edge. For a pair of connected persons and their directed edge between them, the edge is equally probable to connect to one gene as the other gene of a person, i.e. the probability is &lt;span class=&quot;math-container&quot;&gt;$\frac12$&lt;/span&gt;. A pair of genes (persons) are said to be connected if there exists an undirected (simple) path from one gene (person) to the other. Two thus connected genes are called identical by descent.&lt;/p&gt;&#xA;&lt;p&gt;In this setting, the DAG with respect to the persons is given and deterministic whilst any directed edge in the DAG assigned to two persons are random with independent probability &lt;span class=&quot;math-container&quot;&gt;$\frac12$&lt;/span&gt; with respect to the genes within each person.&lt;/p&gt;&#xA;&lt;p&gt;For a pair of persons on a lineage DAG that is connected, the probability of one gene equally probably chosen amongst the two genes of one person being connected, or identical by descent, to one gene equally probably chosen amongst the two genes of the other person is the coefficient of kinship. It can be proven that the probability is &lt;span class=&quot;math-container&quot;&gt;$\displaystyle\sum_{p\in P} \frac1{2^{n(p)}}$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; is the set of all distinct (simple) paths connecting one person (&lt;strong&gt;not&lt;/strong&gt; gene) to to the other and &lt;span class=&quot;math-container&quot;&gt;$n(p)$&lt;/span&gt; is the number of persons on person connecting path &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As an additional assumption, the genes that are connected have to be the same and thus assume the same allele.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;With regard to the conditional probability example in the question, the coefficient of kinship is not adequate to be used to compute the desired probability. Part of the reason is the question inquires about the existence of an allele rather than picking one gene with 1/2 of the probability. We need some more fundamental probability to compute that. We will demonstrate that as follows.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;We will compute the probability of the two siblings having both allele &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; as follows.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;For a pair of full siblings, they can have exactly &lt;span class=&quot;math-container&quot;&gt;$0, 1, 2$&lt;/span&gt; identical by descent genes. Let the respective probabilities be &lt;span class=&quot;math-container&quot;&gt;$r_0, r_1, r_2$&lt;/span&gt;. Notice we are not picking genes of a person at random but talking about the existence of particular types of genes in a person. For two full siblings, some thoughts lead to &lt;span class=&quot;math-container&quot;&gt;$r_0=\frac14, r_1=\frac12, r_2=\frac14$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;By symmetry and without the loss of generality, we fix the connecting genes in the two siblings of the respective edges. We only need to consider the connections of the edges relative to the genes of the parents.&lt;/p&gt;&#xA;&lt;p&gt;(1) Conditioned on there is no identical by descent genes, the sibling genes can independently assume any allele. The probability of having the configuration of &lt;span class=&quot;math-container&quot;&gt;$ab|ab$&lt;/span&gt; (not considering gene ordering) is &lt;span class=&quot;math-container&quot;&gt;$2^2a^2b^2$&lt;/span&gt; considering the distinct ordering of the alleles, that of &lt;span class=&quot;math-container&quot;&gt;$aa|ab$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$\binom41 a^3b$&lt;/span&gt;, and that of &lt;span class=&quot;math-container&quot;&gt;$aa|aa$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$a^4$&lt;/span&gt;. The total probability is thus&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Pr_0:=\frac14(4a^2b^2+4a^3b+a^4).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;(2) When there is exactly &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt; pair of identical by descent genes, the two identical by descent genes can only assume the same allele, while the other genes can assume alleles independently. When the identical by descent genes assume allele &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; the probability of the two siblings both having allele &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. When the two identical by descent genes assume allele &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; the desired probability is then &lt;span class=&quot;math-container&quot;&gt;$ba^2$&lt;/span&gt;. The total probability is then&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Pr_1:=\frac12(a+ba^2).$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;(3) When there are exactly &lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt; pairs of identical by descent genes, each pair can only assume the same allele but independently of each other. The only way that the two siblings not to both have allele &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is for the two identical by descent pairs to assume allele &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;. The total desired probability for the two siblings to both have allele &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is then&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Pr_2:=\frac14(1-b^2)$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Now we compute the probability of any one of the siblings having allele &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;. Each gene can of course assume alleles independently. So the probability is simply&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Pr_3:=1-b^2.$$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Therefor the desired probability of both siblings having allele &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; given that one of them has allele &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Pr(a):=\frac{Pr_0+Pr_1+Pr_2}{Pr_3}=\frac{\frac14a(a^3-6a^2+5a+4)}{a(2-a)}$$&lt;/span&gt;&#xA;since &lt;span class=&quot;math-container&quot;&gt;$b=1-a$&lt;/span&gt;. Some tedious algebra which can be carried out by, say, Mathematica shows this is identical to the expression in the question in terms of &lt;span class=&quot;math-container&quot;&gt;$r=\frac{a}{1-a}$&lt;/span&gt;.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$Pr'(a)=\frac12\Big(2-\frac1{(2-a)^2}-a\Big).$$&lt;/span&gt;&#xA;It is easy to see &lt;span class=&quot;math-container&quot;&gt;$2Pr'(a)-2$&lt;/span&gt; strictly decreases for &lt;span class=&quot;math-container&quot;&gt;$a\in[0,2)$&lt;/span&gt; as the last two terms decreases in that interval. &lt;span class=&quot;math-container&quot;&gt;$Pr'(1)=0$&lt;/span&gt; and thus &lt;span class=&quot;math-container&quot;&gt;$Pr'(a)&amp;gt;0,\ \forall a\in[0,1)$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$Pr(a)$&lt;/span&gt; thus strictly increases for &lt;span class=&quot;math-container&quot;&gt;$a\in[0,1]$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\frac12=Pr(0)\le Pr(a)\le Pr(1)=1$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="8725" LastEditorUserId="8725" LastEditDate="2020-09-27T20:45:05.503" LastActivityDate="2020-09-27T20:45:05.503" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14113" PostTypeId="1" AcceptedAnswerId="21541" CreationDate="2020-08-09T09:14:09.193" Score="2" ViewCount="227" Body="&lt;p&gt;To generate aligned protein sequence truncate to the same length, I use blastp in biopython for MSA. I have multiple queries and one subject sequence. However, the alignment never returns the same length.&lt;/p&gt;&#xA;&lt;p&gt;How can I align all sequences altogether, thus resulting in a set of sequences of the same length?&lt;/p&gt;&#xA;" OwnerUserId="9344" LastActivityDate="2023-09-12T23:24:24.687" Title="Blastp MSA to the same length" Tags="|python|blastp|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14114" PostTypeId="1" CreationDate="2020-08-09T12:08:48.700" Score="0" ViewCount="201" Body="&lt;p&gt;I am trying to use AutoDock Vina to do docking however I am getting this error. I would really appreciate any help as I have not been able to resolve this error despite trying numerous times.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/c5yR4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/c5yR4.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4uwpM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4uwpM.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9346" LastEditorUserId="9346" LastEditDate="2020-08-11T15:04:27.787" LastActivityDate="2020-08-11T15:04:27.787" Title="Issues with AutoDock Vina" Tags="|protein-structure|bash|docking|computational-biochemistry|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="14115" PostTypeId="1" CreationDate="2020-08-09T17:32:02.410" Score="1" ViewCount="589" Body="&lt;p&gt;I am trying to run the ADMIXTURE software for the first time to analyse the structure of an in-house dataset of samples comparing to 1000 genomes project ancestral populations. I am trying to determine ancestry just from the X chromosome of those individuals.&lt;/p&gt;&#xA;&lt;p&gt;To do that, I have downloaded the X chromosome file from 1000 genomes (ALL.chrX.phase3_shapeit2_mvncall_integrated_v1b.20130502.genotypes.vcf.gz). I have removed duplicated with bcftools and converted to plink binary with the following command in plink 1.9:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plink --bcf ALL.chrX.phase3_shapeit2_mvncall_integrated_v1b.20130502.genotypes.bcf --keep-allele-order --vcf-idspace-to _ --const-fid --allow-extra-chr 0 --split-x b37 no-fail --make-bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Next, I modified the .bim files to change the polymorphisms ID so them where the same as their position, as a way to standarize their names. Then I filtered related individuals, also with plink and found the intersection of .bim files from 1000 genomes and my in-house dataset (a bed file with microarray genotyped variants only from X non PAR region), that was already filtered to contain only variants genotyped against the plus strand (without indels).&lt;/p&gt;&#xA;&lt;p&gt;Then I merged both datasets with the following plink command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plink --bfile filename1 --bmerge filename2.bed filename2.bim filename2.fam --make-bed --out filemerged&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That command generated a file with all the SNPS that would have more than 3 alleles in the merged file so I removed all them from both datasets and merged them again. On that merged file I pruned SNPs by linkage disequilibrium:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plink --bfile cromXmerged2 --indep-pairwise 50 10 0.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;, assign sample sex on the .fam file and filtered individual with missingnes higher than 10%:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plink --file mergedfile --mind 0.1 --make-bed --out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also set all heterozigous calls as missing with --set-hh-missing plink command.&lt;/p&gt;&#xA;&lt;p&gt;With the final input I runned a supervised ADMIXTURE as following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./admixture finalinput.bed 3 --haploid=&amp;quot;male:23&amp;quot; --supervised -j4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Before running of course I created the finalinput.pop file in open office calc, by creating a list with the assigned population for each sample, with my in-house dataset left blanc. I just used african, european and east asian 1000 genomes samples because my own dataset has brazilian admixed samples. The final dataset has approximately 1500 SNPs and about 1600 individuals. My computer set up is CPU: i3-2370M with two cores, 4 threads@2.4 GHz, RAM 8 GB@1333 MHz and an SSD 120 GB Sata 3.0. OS Ubuntu 20.04 64 Bits.&lt;/p&gt;&#xA;&lt;p&gt;The thing is that ADMIXTURE have been running for three days without showing any error or producing result yet. I would like to know if it is normal to this analysis to take so long or if it could be any problem with my dataset.&#xA;I would appreciate any suggestion or advice!!!&#xA;Thanks in advance,&#xA;Iriel&lt;/p&gt;&#xA;" OwnerUserId="9347" LastEditorUserId="3694" LastEditDate="2020-08-10T08:58:21.050" LastActivityDate="2020-08-10T19:03:38.293" Title="Time for running ADMIXTURE analysis" Tags="|microarray|bed|phylogenetics|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="14116" PostTypeId="1" CreationDate="2020-08-09T20:59:24.187" Score="0" ViewCount="60" Body="&lt;p&gt;I'm working on understanding the evolution of a gene in a phylogenetic tree. I know the rates of evolution of each organism (from the tree).&lt;/p&gt;&#xA;&lt;p&gt;I am taking a random DNA sequence with my gene, evolving it along the tree with the said rates and comparing this to the actual gene in these various organisms.&#xA;I am 'evolving' the gene if randomly at a probability determined by the rate at each step in the tree.&lt;/p&gt;&#xA;&lt;p&gt;The gene from my simulation is different from what is actually in these organisms. How do I modify my model, so that my simulation gives the same results?&lt;/p&gt;&#xA;" OwnerUserId="8767" LastActivityDate="2020-08-16T21:34:26.843" Title="How to modify DNA evolution model to fit actual data?" Tags="|phylogenetics|simulated-data|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14117" PostTypeId="2" ParentId="14108" CreationDate="2020-08-10T09:07:08.447" Score="1" Body="&lt;p&gt;The approach you are describing seems very strange.&lt;br /&gt;&#xA;Crucially, the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#why-un-normalized-counts&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vignette&lt;/a&gt; for DESeq2 states that the model only works correctly with unnormalized counts as input:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;It is important to provide count matrices as input for DESeq2’s&#xA;statistical model (Love, Huber, and Anders 2014) to hold, as only the&#xA;count values allow assessing the measurement precision correctly. The&#xA;DESeq2 model internally corrects for library size, so transformed or&#xA;normalized values such as counts scaled by library size should not be&#xA;used as input.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Normalized counts like FPKM are not supported and should be converted to counts using the appropriate import funcion.&lt;/p&gt;&#xA;&lt;p&gt;I don't really understand the use of the normalization factors for quality control because the number of reads in a sample does not correlate well with the quality. Maybe you could link to the paper in case there is something hidden in the experimental design that would explain this.&lt;/p&gt;&#xA;&lt;p&gt;If you are interested in quality control a useful metric in my experience is the 3' sequencing bias that can be calculated using, for example, &lt;a href=&quot;https://github.com/getzlab/rnaseqc&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNA-SeQC&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2020-08-10T09:07:08.447" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14118" PostTypeId="2" ParentId="14116" CreationDate="2020-08-10T10:58:04.283" Score="1" Body="&lt;p&gt;Mutation is not a random process because there are many mutational biases, e.g. the &lt;a href=&quot;https://en.wikipedia.org/wiki/Transition_(genetics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;transition vs. transversion ratio&lt;/a&gt;. Ideally use a proper evolutionary model. Maybe try a tool that incorporates substitution and indel models. I'm not up-to-date on the best packages, but e.g.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;INDELible &lt;a href=&quot;http://abacus.gene.ucl.ac.uk/software/indelible/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://abacus.gene.ucl.ac.uk/software/indelible/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Evolver &lt;a href=&quot;https://www.drive5.com/evolver/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.drive5.com/evolver/&lt;/a&gt; (not to be confused with the PAML package of the same name)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="104" LastActivityDate="2020-08-10T10:58:04.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14119" PostTypeId="1" CreationDate="2020-08-10T13:27:34.650" Score="1" ViewCount="105" Body="&lt;p&gt;I would like to gap-fill (or correct) a mammalian-sized assembly for which we have BACs that have been sequenced with PacBio CLR data. I have seen there are pbm22/gcpp tools available for correcting assemblies, but looking at how they've been used, they seem to correct existing assemblies built with FALCON or similar.&lt;/p&gt;&#xA;&lt;p&gt;Can pbmm2 and gcpp be used with a reference constructed without pacbio data? E.g. given a mammalian-sized reference and a .bam file of PacBio reads for a BAC, what's the best way to gap-fill the current reference? Thx&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2020-08-16T23:21:20.277" Title="Gap-fill assembly with PacBio reads" Tags="|assembly|reference-genome|pacbio|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14120" PostTypeId="1" CreationDate="2020-08-10T15:45:13.910" Score="1" ViewCount="30" Body="&lt;p&gt;Is it currently common in metabolomics to combine two quantification methods, namely NMR (nuclear magnetic resonance) and MS (mass spectrometry, either liquid or gas)? If so, what would be profits from such an approach? Is it a promising direction or people are going to stay with a simple approach and they will be sticking to either of these two?&lt;/p&gt;&#xA;" OwnerUserId="9354" LastActivityDate="2020-08-10T15:45:13.910" Title="Hybrid approach in metabolomics (NMR and MS)" Tags="|mass-spectrometry|" AnswerCount="0" CommentCount="4" ClosedDate="2022-02-08T17:07:05.277" ContentLicense="CC BY-SA 4.0" />
  <row Id="14121" PostTypeId="2" ParentId="14087" CreationDate="2020-08-10T16:02:54.917" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;which, conveniently enough, have their IDs in a range&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I think it's a bit risky to rely on that - for this particular case they are not going to change but you may need to download more samples in the future.&lt;/p&gt;&#xA;&lt;p&gt;I would suggest avoiding &lt;code&gt;sra&lt;/code&gt; and &lt;code&gt;fastq(er)-dump&lt;/code&gt; altogether and use &lt;a href=&quot;https://www.ebi.ac.uk/ena/browser/home&quot; rel=&quot;nofollow noreferrer&quot;&gt;ena&lt;/a&gt; instead.&lt;/p&gt;&#xA;&lt;p&gt;In your case, it seems you want the runs in project PRJDB7736. From here &lt;a href=&quot;https://www.ebi.ac.uk/ena/browser/view/PRJDB7736&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/ena/browser/view/PRJDB7736&lt;/a&gt; you can get a tabular file (see link in &lt;em&gt;Download report ... TSV&lt;/em&gt;) with the sample characteristics. Column &lt;code&gt;fastq_ftp&lt;/code&gt; gives you the ftp link to the fastq files that you can pass to &lt;code&gt;wget&lt;/code&gt; or &lt;code&gt;curl&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You can also get the same report programmatically with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;curl -s -X GET &amp;quot;https://www.ebi.ac.uk/ena/portal/api/filereport?accession=PRJDB7736&amp;amp;fields=run_accession,sample_description,library_name,library_layout,fastq_ftp&amp;amp;result=read_run&amp;quot; &amp;gt; ena.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You should edit the &lt;code&gt;fields=...&lt;/code&gt; list to suite your needs. The fields available are those from:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;curl -X GET &amp;quot;https://www.ebi.ac.uk/ena/portal/api/returnFields?dataPortal=ena&amp;amp;result=read_run&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I don't know what you think... I find this route much more useful and transparent than sra's.&lt;/p&gt;&#xA;" OwnerUserId="339" LastEditorUserId="339" LastEditDate="2020-08-10T16:08:14.737" LastActivityDate="2020-08-10T16:08:14.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14124" PostTypeId="2" ParentId="14115" CreationDate="2020-08-10T19:03:38.293" Score="2" Body="&lt;p&gt;Apparently there is a bug in ADMIXTURE that prevents for converging when using at the same time the haploid mode and multithreading (flags --haploid and -j respectively). The problem is solved by avoiding using one of them.&lt;/p&gt;&#xA;" OwnerUserId="9347" LastActivityDate="2020-08-10T19:03:38.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14125" PostTypeId="1" AcceptedAnswerId="15963" CreationDate="2020-08-10T22:32:12.777" Score="2" ViewCount="1759" Body="&lt;p&gt;I have &lt;code&gt;RNA-seq&lt;/code&gt; for two groups of patients: &lt;code&gt;Responders&lt;/code&gt; to chemotherapy (n=9) versus &lt;code&gt;non-responders&lt;/code&gt; to chemotherapy (n=24)&lt;/p&gt;&#xA;&lt;p&gt;I have done &lt;code&gt;DESeq2&lt;/code&gt; and obtain &lt;code&gt;fold change of responders versus non-responders&lt;/code&gt; in which &lt;code&gt;positive fold change&lt;/code&gt; means &lt;code&gt;down-regulation in responders&lt;/code&gt; (if I am not wrong)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(df)&#xA;    gene       FC&#xA;1 286499 7.116071&#xA;2 260436 6.691189&#xA;3   2267 5.751322&#xA;4   5968 5.522079&#xA;5   4103 5.328008&#xA;6 442709 5.036087&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have done &lt;code&gt;GSEA&lt;/code&gt; in &lt;code&gt;pre-ranked&lt;/code&gt; way&lt;/p&gt;&#xA;&lt;p&gt;got this plot for &lt;code&gt;KRAS pathway&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mwPGW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mwPGW.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I interpret this as; &lt;code&gt;KRAS pathway&lt;/code&gt; in being &lt;code&gt;down regulated&lt;/code&gt; in &lt;code&gt;responders&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Am I right?&lt;/p&gt;&#xA;&lt;p&gt;Please correct me&lt;/p&gt;&#xA;&lt;p&gt;When I separated genes for up and down regulated&lt;/p&gt;&#xA;&lt;p&gt;For up regulated genes I did not get any significant GSEA but for down regulated genes in responders I got this&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Rq30k.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Rq30k.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;SO does this mean that &lt;code&gt;KRAS pathway&lt;/code&gt; is being &lt;code&gt;up regulated&lt;/code&gt; in &lt;code&gt;responders&lt;/code&gt; because I have used &lt;code&gt;down regulated genes in responders&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-08-11T16:13:13.483" LastActivityDate="2021-05-24T13:07:59.537" Title="Interpreting this plot from GSEA" Tags="|r|rna-seq|pathway|gsea|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14126" PostTypeId="2" ParentId="14083" CreationDate="2020-08-11T01:45:42.727" Score="0" Body="&lt;p&gt;While the selective-alignment algorithm that is part of salmon is designed for short reads, salmon also supports quantification via alignments in a SAM / BAM file (aligned by other tools).  You have to be somewhat careful about setting the proper command line flags, but nonetheless, the quantification should work fine.  You can, for example, find a pipeline that uses minimap2 for alignment followed by salmon for quantification of long reads &lt;a href=&quot;https://github.com/nanoporetech/pipeline-transcriptome-de&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="491" LastActivityDate="2020-08-11T01:45:42.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14127" PostTypeId="1" CreationDate="2020-08-11T03:20:57.503" Score="1" ViewCount="25" Body="&lt;p&gt;I have made a UMAP of malignant cells and the result is split into 3 clusters, as seen below.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LxREx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LxREx.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For the sake of example, let's say I want to find the differentially expressed genes in the uppermost cluster (the brown, red, and gray one). Does this methodology sound correct:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Load in the EBSeq library. Use the EBTest function - for the Data argument, use the expression matrix for malignant cells. For the Sizes argument, run MedianNorm on said expression matrix. For the condition argument, create a vector denoting whether the malignant cells in the expression matrix are in the uppermost cluster or not (two conditions), for the maxround argument use a number that fits within that works based on convergence.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Are there any glaring flaws with this method? Am I using condition incorrectly? Should I be using all of the malignant cells as my input data when I am trying to find differentially expressed genes in the uppermost cluster?&lt;/p&gt;&#xA;" OwnerUserId="9274" LastActivityDate="2020-08-11T03:20:57.503" Title="Questions about EBSeq (in R)" Tags="|r|bioconductor|differential-expression|ebseq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14128" PostTypeId="2" ParentId="14125" CreationDate="2020-08-11T05:34:53.607" Score="1" Body="&lt;p&gt;In Deseq2, it depends on contrast eg. if your &lt;code&gt;contrast &amp;lt;- c(&amp;quot;Condition&amp;quot;, firstC, SecondC)&lt;/code&gt; is then &lt;code&gt;-ve&lt;/code&gt; is downregulated in &lt;code&gt;FirstC&lt;/code&gt; and same for &lt;code&gt;+ve&lt;/code&gt;. It is always better to take look at count data for the confirmation (responders versus non-responders samples).&lt;/p&gt;&#xA;&lt;p&gt;[for more detail you can refer here][1]&lt;/p&gt;&#xA;&lt;p&gt;[1]&lt;a href=&quot;https://github.com/amarinderthind/RNA-seq-tutorial-for-gene-differential-expression-analysis/blob/master/RNA-Seq-DGE.R&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/amarinderthind/RNA-seq-tutorial-for-gene-differential-expression-analysis/blob/master/RNA-Seq-DGE.R&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8924" LastEditorUserId="8924" LastEditDate="2021-05-24T13:07:59.537" LastActivityDate="2021-05-24T13:07:59.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14129" PostTypeId="1" AcceptedAnswerId="14159" CreationDate="2020-08-11T11:44:35.140" Score="1" ViewCount="44" Body="&lt;p&gt;I have several organisms and I want to calculate for each one of these their codon composition implementing a pairwise distance matrix between these codon vectors to build a phylogenetic tree. Therefore, the matrix  will have on the principal diagonal  0 values (since each organism will have a distance of zero from itself) and the other matrix members will represent the euclidean distance among one organism and another one. Thus we will have a square matrix.  To assess to robustness of the various phylogenetic relationships I would bootstrap this matrix building the tree on these data.&#xA;Is this a good strategy?&lt;/p&gt;&#xA;" OwnerUserId="9142" LastEditorUserId="3848" LastEditDate="2023-07-29T21:18:12.537" LastActivityDate="2023-07-29T21:18:12.537" Title="How to assess if this strategy is good to have some phylogenetic information?" Tags="|phylogenetics|phylogeny|matlab|matrix|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14130" PostTypeId="1" CreationDate="2020-08-11T15:18:04.303" Score="0" ViewCount="415" Body="&lt;p&gt;I've got the reference genome with Python like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sequences_by_chr = {}&#xA;&#xA;with gzip.open(&amp;quot;data/Zea_mays.B73_RefGen_v4.dna.toplevel.fa.gz&amp;quot;, &amp;quot;rt&amp;quot;) as f:&#xA;    for seq in SeqIO.parse(f, &amp;quot;fasta&amp;quot;):&#xA;        sequences_by_chr[seq.id] = seq.seq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've also parsed a GFF3 annotation and would like to get the gene sequence based on said annotation.&#xA;Suppose the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chromosome = 1&#xA;strand = &amp;quot;-&amp;quot;&#xA;start = 45286331&#xA;end = 45290076&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Since it's on the &amp;quot;minus&amp;quot; strand, should I do the following:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;sequences_by_chr[chromosome].complement()[start:end]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;or should I use the &lt;code&gt;reverse_complement()&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;I'm completely confused by this, would appreciate any hint!&lt;/p&gt;&#xA;" OwnerUserId="9361" LastActivityDate="2021-01-10T20:07:10.843" Title="Get gene sequence based on the annotation" Tags="|fasta|python|reference-genome|gff3|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14131" PostTypeId="2" ParentId="14130" CreationDate="2020-08-11T15:46:26.017" Score="2" Body="&lt;p&gt;You can parse the GFF3 to generate a bed-like file with gene ranges and use that with &lt;code&gt;bedtools getfasta&lt;/code&gt; to obtain a multi-FASTA file with gene sequences. An example case is shown below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## get fasta and gff3 files&#xA;wget ftp://ftp.ensembl.org//pub/current_gff3/saccharomyces_cerevisiae/Saccharomyces_cerevisiae.R64-1-1.100.gff3.gz&#xA;wget ftp://ftp.ensembl.org:/pub/current_fasta/saccharomyces_cerevisiae/dna/Saccharomyces_cerevisiae.R64-1-1.dna.toplevel.fa.gz&#xA;&#xA;## unzip fasta&#xA;gunzip Saccharomyces_cerevisiae.R64-1-1.dna.toplevel.fa.gz&#xA;&#xA;## generate genes.bed &#xA;zgrep -v '^#' Saccharomyces_cerevisiae.R64-1-1.100.gff3.gz | awk 'BEGIN{FS=&amp;quot;\t&amp;quot;;OFS=&amp;quot;\t&amp;quot;}(&lt;span class=&quot;math-container&quot;&gt;$3==&quot;gene&quot;){print $1,$4-1,$5,&quot;name&quot;,1000,$&lt;/span&gt;7}' &amp;gt; genes.bed&#xA;&#xA;## create gene fasta&#xA;bedtools getfasta -fi Saccharomyces_cerevisiae.R64-1-1.dna.toplevel.fa -bed genes.bed -s -fo genes.fa&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2020-08-11T15:46:26.017" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14132" PostTypeId="1" CreationDate="2020-08-11T17:57:36.700" Score="0" ViewCount="148" Body="&lt;p&gt;I am looking at how to build a &lt;code&gt;phylogenetic tree&lt;/code&gt; in &lt;code&gt;R&lt;/code&gt; from &lt;strong&gt;aligned&lt;/strong&gt; genetic information (dna sequences). There are multiple tutorials on how to go about this task eg &lt;a href=&quot;https://yulab-smu.github.io/treedata-book/chapter2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial1&lt;/a&gt;, &lt;a href=&quot;https://cran.r-project.org/web/packages/ape/ape.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial2 (&amp;quot;ape&amp;quot; docs)&lt;/a&gt;, &lt;a href=&quot;http://www.phytools.org/Cordoba2017/ex/2/Intro-to-phylogenies.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial3&lt;/a&gt;, and there are many more. There appears to be quite a few steps with dependencies from multiple packages: eg. &amp;quot;ape&amp;quot;, &amp;quot;phangorn&amp;quot;, &amp;quot;phytools&amp;quot;, &amp;quot;geiger&amp;quot;. I would like to learn a minimal working example (mwe) for constructing a phylogenetic tree from a basic data-structure holding dna-strings.&lt;/p&gt;&#xA;&lt;p&gt;Eg. I have a vector of nucleotide strings, and a separate vector for the 'names' of each string in the vector, how can I construct a phylogenetic tree from this information? (not a fasta file)&lt;/p&gt;&#xA;" OwnerUserId="9311" LastEditorUserId="9311" LastEditDate="2020-08-23T02:49:11.150" LastActivityDate="2020-08-23T02:49:11.150" Title="Minimal working example for phylogenetic tree construction in R using an alignment of DNA strings and a vector of names?" Tags="|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14133" PostTypeId="1" AcceptedAnswerId="14142" CreationDate="2020-08-11T18:30:33.047" Score="0" ViewCount="99" Body="&lt;p&gt;I am following an example from ‘Analysis of Phylogenetics Second Edition and Evolution with R’ from Emmanuel Paradis.&lt;/p&gt;&#xA;&lt;p&gt;He is doing:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7uTVB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7uTVB.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am doing pretty much the same:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;x=c('garras','garras','garras', 'tejido','tejido','tejido')&#xA;y &amp;lt;- phyDat(matrix(x), &amp;quot;USER&amp;quot;, levels = unique(x))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but I get:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error in phyDat.default(data, levels = levels, return.index =&#xA;return.index, : data object must contain taxa names&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Screenshot:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bIRKC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bIRKC.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What am I doing wrong? how come the example from the book works...&lt;/p&gt;&#xA;" OwnerUserId="6829" LastActivityDate="2020-08-12T19:49:29.510" Title="Unable to use phangorn::phyDat" Tags="|r|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14134" PostTypeId="1" AcceptedAnswerId="15962" CreationDate="2020-08-11T18:48:16.430" Score="0" ViewCount="294" Body="&lt;p&gt;I have a heat map but top and bottom annotation don't have legend&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Top_ha = HeatmapAnnotation(Group = anno_simple(top&lt;span class=&quot;math-container&quot;&gt;$Group, &#xA;                                               col =structure(hue_pal()(length(unique(top$&lt;/span&gt;Group))), &#xA;                                                              names = unique(top&lt;span class=&quot;math-container&quot;&gt;$Group)),&#xA;                                           border = TRUE, &#xA;                                           height = unit(5, &quot;mm&quot;)),&#xA;                       &quot;Total Mutation Burden&quot;=anno_barplot(clin$&lt;/span&gt;TMB, height = unit(15, &amp;quot;mm&amp;quot;), width=unit(30, &amp;quot;mm&amp;quot;),gp = gpar(fill = structure(hue_pal()(length(unique(clin&lt;span class=&quot;math-container&quot;&gt;$TMB))), &#xA;                                                                                                                                               names = unique(clin$&lt;/span&gt;TMB)))))&#xA;&#xA;&#xA;&#xA;Bottom_ha = HeatmapAnnotation(Response = anno_simple(clin&lt;span class=&quot;math-container&quot;&gt;$Response, &#xA;                                                 border = T, &#xA;                                                 col = structure(c(&quot;purple4&quot;, &quot;grey&quot;), names = c(&quot;Responders&quot;, &quot;NonResponders&quot;)),&#xA;                                                 height = unit(5, &quot;mm&quot;),show_),&#xA;                          &quot;TP53 Status&quot; = anno_simple(clin$&lt;/span&gt;TP53_status, &#xA;                                                          border = TRUE, &#xA;                                                          col = structure(c(&amp;quot;firebrick&amp;quot;, &amp;quot;grey&amp;quot;), names = c(&amp;quot;Mutant&amp;quot;,&amp;quot;Wild type&amp;quot;)),&#xA;                                                          height = unit(5, &amp;quot;mm&amp;quot;)))&#xA;&#xA;&#xA;Right_Anno &amp;lt;- HeatmapAnnotation(Signature = anno_text(right$Signature), which = &amp;quot;row&amp;quot;)&#xA;&#xA;Heatmap(mat,&#xA;        bottom_annotation = Bottom_ha,&#xA;        top_annotation = Top_ha,&#xA;        name = &amp;quot;Exposure&amp;quot;,&#xA;        right_annotation = Right_Anno,&#xA;        show_row_dend = FALSE,&#xA;        show_parent_dend_line = FALSE,&#xA;        column_split = c(top$Group),&#xA;        column_title = c(&amp;quot;Mutagenic&amp;quot;, &amp;quot;DDR Impaired&amp;quot;, &amp;quot;C&amp;gt;A/T Dominant&amp;quot;),&#xA;        col = colorRamp2(c( 0, 0.5), c(&amp;quot;white&amp;quot;, &amp;quot;red&amp;quot;)),&#xA;        show_column_names = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you are seeing Response, group and total mutation burden don't have any legend and annotation&lt;/p&gt;&#xA;&lt;p&gt;Please can you help me to put legend for annotation?&lt;/p&gt;&#xA;&lt;p&gt;I have tried &lt;code&gt;annotation_lable&lt;/code&gt; but says &lt;code&gt;Error: either provide &lt;/code&gt;levels&lt;code&gt;or provide named&lt;/code&gt;colors&lt;code&gt;.&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Also &lt;code&gt;show_annotation_name = TRUE,show_legend = TRUE&lt;/code&gt; did not work&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-08-13T19:33:50.033" LastActivityDate="2021-05-23T12:42:34.537" Title="Adding legend to Botton and top annotation in complexheatmap" Tags="|r|heatmap|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14135" PostTypeId="1" AcceptedAnswerId="14326" CreationDate="2020-08-11T19:43:07.550" Score="2" ViewCount="56" Body="&lt;p&gt;I'm updating a galaxy tool wrapper for Entrez's eutils suite and I'm trying to create a form with valid link selections (among other things) based on the &amp;quot;from&amp;quot; &amp;amp; &amp;quot;to&amp;quot; databases to reduce the size of the select list.&lt;/p&gt;&#xA;&lt;p&gt;Einfo doesn't return these links for the gene or nuccore databases. I'd found &lt;a href=&quot;https://eutils.ncbi.nlm.nih.gov/entrez/query/static/entrezlinks.html#gene&quot; rel=&quot;nofollow noreferrer&quot;&gt;this resource&lt;/a&gt; which seemed to show all the possible links, but it it doesn't show these links either. Yet, in working on &lt;code&gt;elink&lt;/code&gt;'s &lt;code&gt;acheck&lt;/code&gt; feature, using some gene IDs in a test run, I saw 2 links that were not listed in either resource:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;gene_nucleotide&lt;/li&gt;&#xA;&lt;li&gt;gene_nucleotide_pos&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I manually checked that I can use those as link names in valid elink queries. It did return valid results (i.e. no error).&lt;/p&gt;&#xA;&lt;p&gt;So I ran elink on the &lt;code&gt;gene&lt;/code&gt; database using a few random UIDs to see if I get back a comprehensive list of links (albeit possibly empty results for those links), but it also was missing these 2 links.  I'm concerned that I'm not getting all of the possible links for my form interface and that it could produce valid results and that users may want to link via these links, given prior knowledge that they exist.&lt;/p&gt;&#xA;&lt;p&gt;How do I get a comprehensive list of database links from Entrez and why doesn't einfo return all possible links?&lt;/p&gt;&#xA;" OwnerUserId="5063" LastEditorUserId="5063" LastEditDate="2020-09-08T16:15:57.197" LastActivityDate="2020-09-08T16:55:15.400" Title="Why doesn't an (Entrez eutils) einfo request for &quot;gene&quot; return the link gene_nucleotide or gene_nucleotide_pos links?" Tags="|entrez|e-utilities|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14136" PostTypeId="1" CreationDate="2020-08-11T21:56:08.157" Score="1" ViewCount="282" Body="&lt;p&gt;I am in google colab and I have combined and set up a data frame list of sequences that goes like this&lt;/p&gt;&#xA;&lt;p&gt;Location ID Sequence&lt;/p&gt;&#xA;&lt;p&gt;1.1     ........ A ........ AAGAGATA&lt;/p&gt;&#xA;&lt;p&gt;1.2     ........ A...........   CGGATGAT&lt;/p&gt;&#xA;&lt;p&gt;1.3     ........ B................   ACGTGATC&lt;/p&gt;&#xA;&lt;p&gt;1.1     ........ B ........ AAGAGATA&lt;/p&gt;&#xA;&lt;p&gt;1.2     ........ B...........   CGGATGAT&lt;/p&gt;&#xA;&lt;p&gt;1.3     ........ C................   ACGTGATC&lt;/p&gt;&#xA;&lt;p&gt;So I need to make a alternative to Venn diagram because there are about 20+ ID types within a couple hundred rows of list just like this. And some of the sequences repeat or are unique to its id so I need something to be like&lt;/p&gt;&#xA;&lt;p&gt;CGGATGAT Shows up 4 times in B,C,A, and D&#xA;ACGTGATC shows up 1 time in B&#xA;AGTGTCTC shows up 5 times in E,A,B,C, and D.&lt;/p&gt;&#xA;&lt;p&gt;And then have some kind of graphical visualation to show all of these common sequences and where they are located and which are unique. Using python. This might be asking for too much but I'm not sure. I would use something like vennr to make a bigger complex version of &lt;a href=&quot;https://raw.githubusercontent.com/montilab/vennr/master/README_files/figure-gfm/unnamed-chunk-7-1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; but I don't really know R.&lt;/p&gt;&#xA;" OwnerUserId="9365" LastActivityDate="2020-08-12T23:21:35.157" Title="Need a alternative or more complex version of venn diagram in python for matching dna sequences" Tags="|python|ngs|phylogenetics|visualization|pcr|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14137" PostTypeId="2" ParentId="14136" CreationDate="2020-08-11T23:38:06.313" Score="5" Body="&lt;p&gt;To get around limitations in using Venn diagrams for set overlaps, when there are more than three sets, &lt;a href=&quot;https://www.biostars.org/p/77362/#77377&quot; rel=&quot;nofollow noreferrer&quot;&gt;back around 2013 or so&lt;/a&gt; I created something I called an &lt;a href=&quot;https://gist.github.com/jguenther/a13310c0403ac2c3663b&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eulergrid&lt;/a&gt; plot (example at the bottom of the page), which an &lt;a href=&quot;https://cran.r-project.org/web/packages/UpSetR/vignettes/basic.usage.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;UpsetR&lt;/a&gt; plot appears to recapitulate, today.&lt;/p&gt;&#xA;&lt;p&gt;The Eulergrid code I wrote was in a mix of Perl and R; the UpsetR plot code uses R. There appears to be a &lt;a href=&quot;https://pypi.org/project/UpSetPlot/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python port&lt;/a&gt; of UpsetR that might work, as well.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2020-08-12T23:21:35.157" LastActivityDate="2020-08-12T23:21:35.157" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14138" PostTypeId="1" CreationDate="2020-08-12T13:21:31.117" Score="0" ViewCount="130" Body="&lt;p&gt;I'm using &lt;code&gt;structure&lt;/code&gt; for the first time and I fail to execute a &amp;quot;job&amp;quot;. For some reason, &lt;code&gt;structure&lt;/code&gt; interprets the &lt;code&gt;pop ID&lt;/code&gt; column as locus.&lt;/p&gt;&#xA;&lt;p&gt;I have following &lt;code&gt;.txt&lt;/code&gt; file&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4dIW6.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4dIW6.jpg&quot; alt=&quot;head of the .txt file&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The first column are &lt;code&gt;sample labels&lt;/code&gt;, the second column are &lt;code&gt;population IDs&lt;/code&gt; and then there are 26 columns containing the locus information (i.e. 13 markers). Thus, in total 28 columns. It is a diploïd organism so each locus consists of 2 consecutive columns (as described in &lt;code&gt;structure&lt;/code&gt; manual). In total there are 64 rows (so 64 samples)&lt;/p&gt;&#xA;&lt;p&gt;I load this &lt;code&gt;.txt&lt;/code&gt; file into &lt;code&gt;structure&lt;/code&gt;, specifying following parameters:&lt;/p&gt;&#xA;&lt;p&gt;-&lt;code&gt;number of individuals&lt;/code&gt; 64&lt;/p&gt;&#xA;&lt;p&gt;-&lt;code&gt;Ploidy of data&lt;/code&gt; 2&lt;/p&gt;&#xA;&lt;p&gt;-&lt;code&gt;number of loci&lt;/code&gt; 13&lt;/p&gt;&#xA;&lt;p&gt;-&lt;code&gt;missing data value&lt;/code&gt; -9&lt;/p&gt;&#xA;&lt;p&gt;-&lt;code&gt;special format: data file stores data for individuals in single line&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;-&lt;code&gt;individual ID for each individual&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;-&lt;code&gt;putative population origin for each individual&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The loaded file looks like:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fX27p.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fX27p.jpg&quot; alt=&quot;head of structure file&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This looks good. My samples have a separate column, &lt;code&gt;pop ID&lt;/code&gt; is a separate column for possible populations an then for each &lt;code&gt;locus&lt;/code&gt; there are 2 columns. Not all loci are visible on screenshot, but there are 13.&lt;/p&gt;&#xA;&lt;p&gt;I want to calculate &lt;code&gt;Delta K&lt;/code&gt; and therefore I want to run a job. However when I run it, I get following error:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XESWZ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XESWZ.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I get this error for each row/individual. It appears that &lt;code&gt;structure&lt;/code&gt; interprets the &lt;code&gt;pop ID&lt;/code&gt; values (&amp;quot;Noord&amp;quot;,&amp;quot;Zuid&amp;quot;, &amp;quot;onbekend&amp;quot;) as 14th locus, but I have never specified a 14th locus.  I fail to understand why this happens.&lt;/p&gt;&#xA;&lt;p&gt;When I run the program without &lt;code&gt;pop ID&lt;/code&gt; column, it works.&lt;/p&gt;&#xA;&lt;p&gt;Could anyone please advise me on how to solve this error?&lt;/p&gt;&#xA;" OwnerUserId="7476" LastActivityDate="2020-08-13T00:05:25.450" Title="STRUCTURE interprets pop ID as locus" Tags="|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14139" PostTypeId="1" AcceptedAnswerId="14141" CreationDate="2020-08-12T14:39:37.137" Score="2" ViewCount="65" Body="&lt;p&gt;I have a couple of hundred bacterial sequences of 2-30 genes of interest each, recovered from metagenomics. None of them encode rRNA. Normally I'd just BLAST the one gene I already know to be reliable phylogenetic marker, where present, but the scale of my current dataset makes it a bit daunting.&lt;/p&gt;&#xA;&lt;p&gt;Is there any software for taxonomic assignment that you could recommend? It's fine if it requires command line use. There are so many options listed online that I didn't even know where to start... Please and thank you.&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2020-08-14T09:57:18.637" Title="Software for taxonomic assignment?" Tags="|blast|metagenome|taxonomy|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14140" PostTypeId="1" CreationDate="2020-08-12T19:06:25.070" Score="1" ViewCount="975" Body="&lt;p&gt;We have a docking target protein with multiple chains (A,B and C). However, I just need chain A. How can I delete the 2 other chains from the docking target protein.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yOrFq.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yOrFq.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XmPcv.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XmPcv.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9346" LastEditorUserId="48" LastEditDate="2020-08-14T09:57:03.277" LastActivityDate="2020-08-17T10:46:15.357" Title="Delete chains from a protein" Tags="|protein-structure|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14141" PostTypeId="2" ParentId="14139" CreationDate="2020-08-12T19:18:47.967" Score="4" Body="&lt;p&gt;You could take a look at &lt;a href=&quot;https://ccb.jhu.edu/software/kraken2&quot; rel=&quot;nofollow noreferrer&quot;&gt;kraken2&lt;/a&gt; It should fit your purpose.&lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2020-08-12T19:18:47.967" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14142" PostTypeId="2" ParentId="14133" CreationDate="2020-08-12T19:49:29.510" Score="1" Body="&lt;p&gt;I cannot access the book, it might be an older version or there are some steps in between. The screen shot doesn't help.&lt;/p&gt;&#xA;&lt;p&gt;If you look at the &lt;a href=&quot;https://cran.r-project.org/web/packages/phangorn/vignettes/phangorn-specials.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;vignette&lt;/a&gt; and the error, what you need to provide is a matrix of characters with defined row names, for example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;set.seed(111)&#xA;x = sample(letters,100,replace=TRUE)&#xA;phyDat(matrix(x),&amp;quot;USER&amp;quot;,levels=unique(x))&#xA;&#xA;Error in phyDat.default(data, levels = levels, return.index = return.index,  : &#xA;  data object must contain taxa names&#xA;&#xA;mat = matrix(x,10,10,dimnames=list(paste0(&amp;quot;t&amp;quot;,1:10),NULL))&#xA;phyDat(mat,&amp;quot;USER&amp;quot;,levels=unique(x))&#xA;10 sequences with 10 character and 10 different site patterns.&#xA;The states are n t s o y e c h z q m j d o h n z y x p q g u o a i h y c g t d f y t d f e u e f d i j r z x w u m a j p j g d t q y v o x p m s o z x z a y j z r u s a y k l f g l f r i l r f m t t e f x v f g q j &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-08-12T19:49:29.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14143" PostTypeId="1" CreationDate="2020-08-12T20:00:19.870" Score="2" ViewCount="540" Body="&lt;p&gt;I got tasked with a problem, that i thought would be quite simple to solve, but turned out to be quite tricky. Our lab is running targeted mutagenesis experiments in yeast using crispr base editors. What we have done is set up an experiment where the base editor selectively mutates a region inside of a gene of interest in a culture of yeast cells. We then extract the DNA of the whole culture and perform sequencing of our gene of interest (the size of which is around 1kb).&#xA;What i would like to do is plot the mutation frequency of say, C -&amp;gt; G edits, across the length of the entire gene (around 300bp), with the hope that i see a spike in mutations at the site where the base editor binds. Or at least, higher mutation rates at this site compared to the background mutation rate.&lt;/p&gt;&#xA;&lt;p&gt;These edits are quite rare (we think), and since it's in a culture of multiple cells, they don't occur at the exact same position on the gene.&lt;/p&gt;&#xA;&lt;p&gt;I initially thought i could do this by quality filtering the reads, aligning them to the gene sequence in order to produce a .bam file, and then piping this file through variant calling algorithms such as vcftools. However, i noticed that i get very few variants out of this, even when using extremely relaxed settings (e.g. p-value &amp;lt; 1 for the variant call).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Im therefore wondering if anyone knows how to extract from a .bam file ALL the mutations across all reads that do not match the reference sequence?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm aware that this will probably lead to quite a huge .vcf file, but since the reference sequence is only 300bp, it should still be manageable? It could also be the the experiment does not work, but i would at least expect some mutations in the sequencing reads compared to the reference gene sequence.&lt;/p&gt;&#xA;&lt;p&gt;Any help would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="9371" LastActivityDate="2020-10-06T10:39:24.597" Title="Produce .vcf file of ALL mutations in .bam file" Tags="|sam|vcf|variation|variants|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14145" PostTypeId="2" ParentId="14143" CreationDate="2020-08-12T20:18:18.163" Score="2" Body="&lt;p&gt;Ensure that the variant caller stringency is relaxed all the way. For example in &lt;a href=&quot;https://github.com/ekg/freebayes&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;freebayes&lt;/code&gt;&lt;/a&gt; use &lt;code&gt;--min-alternate-count 1 --min-alternate-fraction 0&lt;/code&gt; .&lt;/p&gt;&#xA;" OwnerUserId="6251" LastActivityDate="2020-08-12T20:18:18.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14146" PostTypeId="2" ParentId="14138" CreationDate="2020-08-12T23:57:57.593" Score="1" Body="&lt;p&gt;Hmmmm .... you don't actually need pop ID to work for Structure, providing their are no logical errors (you get a wrong answer). The bit that must work is the &lt;em&gt;a priori&lt;/em&gt; number of populations in the data set. You are then going to manually (or via a script) feed a range of &lt;em&gt;a priori&lt;/em&gt; values into Structure, run each and plot the likelihood. Where the likelihood plateaus thats the number of populations contained within your data. This is the key output of Structure.&lt;/p&gt;&#xA;&lt;p&gt;Personally, I would perform a Structure run on the test data and remove the &lt;code&gt;pop ID&lt;/code&gt; column and check the answers (likelihood -lnL) are identical. Thereafter use an approach such as &lt;span class=&quot;math-container&quot;&gt;$F_{ST}$&lt;/span&gt; to interrogate the exact members of each population delineation.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I think your test will work .... The only thing I can think of is the header comprises a space, This format just needs checking against the test data.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-08-13T00:05:25.450" LastActivityDate="2020-08-13T00:05:25.450" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14147" PostTypeId="1" CreationDate="2020-08-13T09:30:16.723" Score="3" ViewCount="79" Body="&lt;p&gt;I have an assembly of an algae and want to find if it has anything analogous to endogenous retroviruses' subsequences in its DNA.&lt;/p&gt;&#xA;&lt;p&gt;My guess is I should manually blastn all subsequences until I find (or not something) that could have come from a virus.&lt;/p&gt;&#xA;&lt;p&gt;Is there a better way to do this?&lt;/p&gt;&#xA;" OwnerUserId="9373" LastActivityDate="2020-09-11T00:11:29.120" Title="How to find possible viral subsequences in a Eukaryote genome?" Tags="|blast|blastn|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14148" PostTypeId="2" ParentId="14140" CreationDate="2020-08-13T10:00:05.577" Score="2" Body="&lt;p&gt;If the tag &lt;code&gt;r&lt;/code&gt; is incorrect, the easiest way is to download and install &lt;code&gt;PyMOL&lt;/code&gt;: &lt;a href=&quot;https://pymol.org/2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pymol.org/2/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Drap and drop your pdb file and type &lt;code&gt;remove not chain A&lt;/code&gt;.&#xA;Then save molecule via the menu as pdb or type &lt;code&gt;save some_file_name.pdb&lt;/code&gt; (will be in your home root folder).&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-08-13T10:00:05.577" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14150" PostTypeId="1" CreationDate="2020-08-13T10:23:35.857" Score="1" ViewCount="1054" Body="&lt;p&gt;I am trying to dock ligands with proteins to estimate the docking potential however, I am facing this error while docking. Any help will be highly appreciated.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Jm0mr.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Jm0mr.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9346" LastActivityDate="2020-08-13T10:23:35.857" Title="Parse Error while using AutoDock Vina" Tags="|database|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14151" PostTypeId="2" ParentId="14130" CreationDate="2020-08-13T10:27:32.137" Score="0" Body="&lt;p&gt;DNA sequence is always provided on the plus strand, if the gff tells you it is on the minus strand, you need to reverse complement it, because the start of the gene in your example will be 45290076 and the end will now be 45286331.&lt;/p&gt;&#xA;&lt;p&gt;I can give you an example below, this is a potential coding sequence from yeast, no introns and on the minus strand:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gunzip -c test.fa.gz  &#xA;&amp;gt;1&#xA;TCATCCCCTTATATAAGATGGGAAACGAAGGTAAAAGATTAACGATAGCAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If we read it in like you did, and if you apply the complement, this is not a coding sequence.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chromosome=&amp;quot;1&amp;quot;&#xA;start = 1&#xA;end =51&#xA;sequences_by_chr[chromosome].complement()[(start-1):end]&#xA;&#xA;Seq('AGTAGGGGAATATATTCTACCCTTTGCTTCCATTTTCTAATTGCTATCGTA', SingleLetterAlphabet())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Getting the sequence followed by reverse complement gives you a coding sequence, starting with ATG ending with TGA:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sequences_by_chr[&amp;quot;1&amp;quot;][(start-1):end].reverse_complement()&#xA;&#xA;Seq('ATGCTATCGTTAATCTTTTACCTTCGTTTCCCATCTTATATAAGGGGATGA', SingleLetterAlphabet())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-08-13T10:27:32.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14152" PostTypeId="1" CreationDate="2020-08-13T10:54:22.300" Score="0" ViewCount="124" Body="&lt;p&gt;I was able to execute the algorithm correctly. But I'm still trying to get a hold of the result I'm achieving.&lt;/p&gt;&#xA;&lt;p&gt;This is the algorithm I'm trying to understand -&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://rosalind.info/problems/ba2d/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://rosalind.info/problems/ba2d/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If anyone can, can anybody give a description on why we are using the first DNA's k-mer, as the initial Motif to get a profile, which is further is used on a different DNA string to get its most probable pattern.&lt;/p&gt;&#xA;&lt;p&gt;How this helps, How they are dependent?&lt;/p&gt;&#xA;&lt;p&gt;And why a lower score between those Motifs, help us to identify the BestMotifs?&lt;/p&gt;&#xA;&lt;p&gt;I have gone through the below question, but the perspective is different, it seems to be the same course. -&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2390/greedy-motif-search-using-probability-matrices&quot;&gt;Greedy Motif Search Using Probability Matrices&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Any clear resources also will be helpful&lt;/p&gt;&#xA;" OwnerUserId="9370" LastEditorUserId="9370" LastEditDate="2020-08-13T11:12:03.983" LastActivityDate="2020-08-13T11:12:03.983" Title="Greedy Motif Search [ Reasoning ]" Tags="|phylogenetics|motifs|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14153" PostTypeId="2" ParentId="14130" CreationDate="2020-08-13T18:18:09.340" Score="0" Body="&lt;p&gt;You should do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;if sequences_by_chrom[strand] == &amp;quot;-&amp;quot;:    &#xA;    rev_comp( sequences_by_chr[chromosome][start:end] )&#xA;else:&#xA;    sequences_by_chr[chromosome][start:end]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def rev_comp(sequence):&#xA;    d = {'A':'T', 'T':'A','C':'G','G':'C'}&#xA;    return ''.join([d[i] for i in sequence][::-1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2970" LastActivityDate="2020-08-13T18:18:09.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14154" PostTypeId="2" ParentId="14143" CreationDate="2020-08-13T20:06:32.377" Score="2" Body="&lt;p&gt;You could perform an mpileup with samtools. I think this would be the most sensitive approach. Here's the man page: &lt;a href=&quot;http://www.htslib.org/doc/samtools-mpileup.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.htslib.org/doc/samtools-mpileup.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You might want to disable BAQ correction. If you specify a reference sequence, then any letter in the fifth column would represent a non reference base. Reference bases are coded as &lt;code&gt;.&lt;/code&gt; or &lt;code&gt;,&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The reason why vcftools doesn't work is probably because it assumes a diploid individual, which is not what you have. You have lots of samples mixed together. Freebayes allows relaxation of the diploid assumption. Here is the relevant example from the documentation &lt;a href=&quot;https://github.com/ekg/freebayes&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ekg/freebayes&lt;/a&gt; :&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Generate frequency-based calls for all variants passing input thresholds. You'd do this in the case that you didn't know the number of samples in the pool.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;freebayes -f ref.fa -F 0.01 -C 1 --pooled-continuous aln.bam &amp;gt;var.vcf&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You could try a cancer/normal variant caller like mutect, but I don't think you'll have much luck.&lt;/p&gt;&#xA;&lt;p&gt;The most straightforward approach is to use samtools mpileup and to process the output yourself. You could also use bcftools mpileup to get a VCF, and then extract the allele count from the INFO field. If you are a Python lover, then you could also use pysam to both perform the pileup and process the results.&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="492" LastEditDate="2020-10-06T10:39:24.597" LastActivityDate="2020-10-06T10:39:24.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14155" PostTypeId="1" AcceptedAnswerId="14175" CreationDate="2020-08-14T05:23:57.277" Score="0" ViewCount="455" Body="&lt;p&gt;How do I parse the json file from ontology output.&#xA;My file &lt;a href=&quot;https://drive.google.com/file/d/16_eL_2ec7anSnJ8i38KJPYtF2x7ydKX6/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;json file&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;jdata &amp;lt;- read_json(&amp;quot;analysis.json&amp;quot;, simplifyVector = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to convert it into a &lt;code&gt;data.frame&lt;/code&gt; format. I can able to take the GO term and respective p value etc. But I would like to know the genes that are getting part of the enriched list which is under the upload_1 for example under DNA Replication there are 33 terms but 8 are from my input. So my objective is to get the 8 gene so on and so forth.&lt;/p&gt;&#xA;&lt;p&gt;So I would like to see a &lt;code&gt;data.frame&lt;/code&gt; where I want to have GO term in the first column gene mapped into the term would be second i.e upload_[#] third one fold enrichment and last one p value.&lt;/p&gt;&#xA;&lt;p&gt;This is how i would like to see this one is the text output in the column &lt;code&gt;upload_1 (450)&lt;/code&gt; instead of that number i would like to see then gene names which are present in the json file.&lt;/p&gt;&#xA;&lt;p&gt;Some progress here is my code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;l &amp;lt;- fromJSON(&amp;quot;analysis.json&amp;quot;)&#xA;&#xA;m &amp;lt;- lapply(&#xA;  l[[1]]&lt;span class=&quot;math-container&quot;&gt;$result, &#xA;  function(x) c(x$&lt;/span&gt;input_list&lt;span class=&quot;math-container&quot;&gt;$mapped_id_list['mapped_id'], x$&lt;/span&gt;term['id'],x&lt;span class=&quot;math-container&quot;&gt;$input_list['pValue'],x$&lt;/span&gt;term['label'])&#xA;)&#xA;m &amp;lt;- do.call(rbind, m)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mapped_id     id           pValue       label                       &#xA;[1,] Character,8   &amp;quot;GO:0006270&amp;quot; 0.007540801  &amp;quot;DNA replication initiation&amp;quot;&#xA;[2,] Character,388 &amp;quot;GO:0009987&amp;quot; 3.232843e-10 &amp;quot;cellular process&amp;quot;          &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Right now I can parse out first two row.&lt;/p&gt;&#xA;&lt;p&gt;Things I would like to append to the code is&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I would like to do for all the terms&lt;/li&gt;&#xA;&lt;li&gt;The first column where i am trying to get all the genes now it is coming as character while number suggests gene present. That I wont need probably since i need only gene names which is being shown as character now.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;dput(head(b))&#xA;    structure(list(`GO biological process complete` = c(&amp;quot;DNA replication initiation (GO:0006270)&amp;quot;, &#xA;    &amp;quot;regulation of blood coagulation (GO:0030193)&amp;quot;, &amp;quot;regulation of hemostasis (GO:1900046)&amp;quot;, &#xA;    &amp;quot;regulation of coagulation (GO:0050818)&amp;quot;, &amp;quot;platelet degranulation (GO:0002576)&amp;quot;, &#xA;    &amp;quot;cell cycle G1/S phase transition (GO:0044843)&amp;quot;), `upload_1 (450)` = c(8, &#xA;    13, 13, 13, 16, 15), `upload_1 (fold Enrichment)` = c(11.23, &#xA;    7.09, 7, 6.62, 5.75, 5.35), `upload_1 (P-value)` = c(0.00754, &#xA;    0.000627, 0.000716, 0.00135, 0.000347, 0.00228)), row.names = c(NA, &#xA;    -6L), class = c(&amp;quot;tbl_df&amp;quot;, &amp;quot;tbl&amp;quot;, &amp;quot;data.frame&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any suggestion or help would be really appreciated.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6rKke.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6rKke.png&quot; alt=&quot;pathway&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="48" LastEditDate="2020-08-14T09:54:37.863" LastActivityDate="2020-08-18T13:17:54.660" Title="Parsing the json file from gene ontology result into dataframe" Tags="|r|gene-ontology|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14156" PostTypeId="2" ParentId="14139" CreationDate="2020-08-14T09:57:18.637" Score="3" Body="&lt;p&gt;If you like blast, maybe you can try &lt;a href=&quot;https://uni-tuebingen.de/fakultaeten/mathematisch-naturwissenschaftliche-fakultaet/fachbereiche/informatik/lehrstuehle/algorithms-in-bioinformatics/software/diamond/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DIAMOND&lt;/a&gt; - the authors claim it's much faster than BLAST.&lt;/p&gt;&#xA;" OwnerUserId="2245" LastActivityDate="2020-08-14T09:57:18.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14157" PostTypeId="1" AcceptedAnswerId="14162" CreationDate="2020-08-15T12:54:44.643" Score="3" ViewCount="1785" Body="&lt;p&gt;I'm working on a dataset in which the first replicate of each group is one batch and the second replicate is in a second batch. After checking the &lt;code&gt;PCA&lt;/code&gt; plot and seeing the batch effect in PC1, I used &lt;code&gt;removeBatchEffect&lt;/code&gt; function from limma to subtract the batch effect from my count data. Then, using PCA gives me a plot that doesn't seem to have any apparent batch effect left! However, it is recommended not to use batch effect correction for differential gene analysis but use the batch variable along with the group variable in constructing the &lt;code&gt;model.matrix&lt;/code&gt;. So, I did that, ran &lt;code&gt;limma/voom&lt;/code&gt; on the normalized counts, and extracted differentially expressed genes. However, when I'm trying to make a heatmap from the DEGs, I still see that the samples from different batches are clustered separately, instead of seeing the clustering of replicates of the same sample. So, my question is whether I should use &lt;code&gt;removeBatchEffect&lt;/code&gt; on the count data from DEGs and then use the transformed dataset for heatmaps or there is another way to fix this?&lt;/p&gt;&#xA;" OwnerUserId="9299" LastActivityDate="2020-08-16T16:58:19.663" Title="Removing Batch Effect in Heatmaps after Differential Gene Expression Analysis" Tags="|rna-seq|edger|limma|heatmap|batch-effects|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="14158" PostTypeId="2" ParentId="14147" CreationDate="2020-08-15T17:44:15.247" Score="2" Body="&lt;p&gt;Your guess sounds reasonable to me but you should consider using &lt;a href=&quot;https://github.com/DerrickWood/kraken2/wiki/Manual&quot; rel=&quot;nofollow noreferrer&quot;&gt;KRAKEN2&lt;/a&gt;. You can create your custom database (you would have done it with makeblastdb anyway) and kraken will check k-mers against the database to find homologous sequences.&lt;/p&gt;&#xA;" OwnerUserId="2970" LastActivityDate="2020-08-15T17:44:15.247" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14159" PostTypeId="2" ParentId="14129" CreationDate="2020-08-15T21:37:16.163" Score="2" Body="&lt;p&gt;The method you are proposing is called a distance method, because you are constructing a distance matrix and then using a clustering algorithm to make a tree. The things to watch out for are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Multiple substitutions, traditionally this is take care of using a Jukes-Cantor correction. You could argue this is less of a concern because this is a 64 substition matrix model rather than just 4 or 20 substition matrix for nucleotides or amino acids.&lt;/li&gt;&#xA;&lt;li&gt;Use the neighbour-joining clustering algorithm. Do not use a standard hierarchical clustering algorithm such as UPGMA. I'm sure R must have the nj in something like R's ape, pretty sure scipy will have something like that&lt;/li&gt;&#xA;&lt;li&gt;Code for bootstrap resampling. That is a method for measuring robustness&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The codon model for phylogenetic trees has been implemented in MrBayes, I suspect it will be in phyloBayes. This method will be Bayesian and you should run this is parallel with your own solution.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Concerns&lt;/em&gt;&#xA;Codon models are very prone to mutation saturation (which is complicated) so you must only use it for closely related sequences. If e.g. 3rd codon distances start exceeding 70% codon models go kaput, because it will select a different codon, which is a completely separate part of the matrix - completely by chance. So whilst mutation saturation in a nucleotide model (4 substitution matrix) would invalidate the tree in theory, they would be sufficient 'good' signal for key phylogenetic signals to be identified, this wouldn't happen for a codon model.&lt;/p&gt;&#xA;&lt;p&gt;I recognise the 'concerns' might sound a bit confusing at present, but just to reiterate, codon models need sequence data that is really closely related. For this data they excel and give excellent bootstrap support.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-08-15T21:37:16.163" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14160" PostTypeId="2" ParentId="14132" CreationDate="2020-08-16T01:04:17.417" Score="1" Body="&lt;p&gt;A key difference from many examples provided is that they assume that the data provided is in the form of sequences rather than an already aligned sequence alignment.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(&amp;quot;Biostrings&amp;quot;)&#xA;library(&amp;quot;seqinr&amp;quot;)&#xA;library(&amp;quot;ape&amp;quot;)&#xA;library(phylogram)&#xA;library(&amp;quot;dendextend&amp;quot;)&#xA;&#xA;&#xA;#read in the data from a format with the aligned sequences (eg msf)&#xA;msf.res &amp;lt;- read.alignment(file = &amp;quot;geneticAlignment.msf&amp;quot;, format = &amp;quot;msf&amp;quot;)&#xA;#dist.alignment and not dna.dist commonly used in tutorials&#xA;msf.res.dist.alignment = dist.alignment(msf.res, matrix = &amp;quot;identity&amp;quot;)&#xA;#perform neighbor joining upon the distance set&#xA;msf.res.dist.alignment.nj = nj(msf.res.dist.alignment)&#xA;#then plot the joined sequences&#xA;plot(msf.res.dist.alignment.nj, main = &amp;quot;Neighbor Joining&amp;quot;)&#xA; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9311" LastActivityDate="2020-08-16T01:04:17.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14161" PostTypeId="1" CreationDate="2020-08-16T16:22:12.427" Score="0" ViewCount="136" Body="&lt;p&gt;I have sequencing data of a few samples of a DNA virus.&lt;/p&gt;&#xA;&lt;p&gt;I'd like to learn &lt;em&gt;de novo assembly&lt;/em&gt; of 'short read' data, to construct a scaffold and then count the abundance of each strain in the data.&lt;/p&gt;&#xA;&lt;p&gt;I heard about &lt;code&gt;SPAdes&lt;/code&gt; as a good choice for these kinds of very short genomes. and also &lt;code&gt;BBmap&lt;/code&gt; for statistics related to contigs.&lt;/p&gt;&#xA;&lt;p&gt;I new to &lt;em&gt;de novo&lt;/em&gt; assembly could you also recommend introductory material? For example a full workflow post/paper on genome assembly.&lt;/p&gt;&#xA;&lt;p&gt;I have some experience with RNA-seq and small RNA-seq data.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: I am seeking an overview and test data prior reading Spades and BBmap manuals. Thus I need to learn RNA-seq analysis rather than a &lt;code&gt;aligner&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;cross-posted in &lt;a href=&quot;https://www.biostars.org/p/455717/&quot; rel=&quot;nofollow noreferrer&quot;&gt;biostars&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9299" LastEditorUserId="3848" LastEditDate="2022-03-05T14:50:56.630" LastActivityDate="2023-03-30T17:02:20.290" Title="Resources to learn genome assembly workflow for small genomes (like viruses)" Tags="|genome|assembly|covid-19|scaffold|virus|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14162" PostTypeId="2" ParentId="14157" CreationDate="2020-08-16T16:58:19.663" Score="5" Body="&lt;p&gt;It is indeed true that for the DE analysis one should include batch into the formula to avoid changing the original counts. Still, for everything else such as plotting heatmaps use of &lt;code&gt;removeBatchEffects&lt;/code&gt; is perfectly fine and (at least to me) a standard and well-accepted procedure. It essentially does not matter what you use to correct for the batch effect for the counts you use downstream. Results will probably be similar. &lt;code&gt;Combat-Seq&lt;/code&gt; from the sva package is a recent adaptation of ComBat specifically for RNA-seq which (from what I understand) better deals with the integer-count nature of the data. This operates on raw counts and avoids the infamous negative values that happen at times with both &lt;code&gt;limma&lt;/code&gt; and &lt;code&gt;Combat&lt;/code&gt;. After applying &lt;code&gt;ComBat-Seq&lt;/code&gt; on your raw counts you can normalize them as usual with edgeR (or any tool you like) and then make the heatmaps. See &lt;a href=&quot;https://github.com/zhangyuqing/ComBat-seq&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/zhangyuqing/ComBat-seq&lt;/a&gt;. The DE results still should come from the normal DE pipeline with batch as covariate as discussed above.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-08-16T16:58:19.663" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14163" PostTypeId="2" ParentId="14116" CreationDate="2020-08-16T21:34:26.843" Score="1" Body="&lt;p&gt;I decided that this was too involved for a comment, so I'm making it an answer.&lt;/p&gt;&#xA;&lt;p&gt;It is still quite difficult to answer the question given the absence of methods explanation; but I think that there's some misunderstanding of phylogenetic modeling going on here, and I can at least address that. I'm going to try to break it down into its components, starting with the basic statistical idea of simulation and then going back to phylogenies.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Statistical reasoning&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let's say that I flip 10 coins (2 outcomes: head (H) and tails (T)), and I observe the sequence &lt;code&gt;HTHTTTHHTT&lt;/code&gt;. I then fit a maximum likelihood model, and find that the best model for the data has&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Pr(H) = 0.4$&lt;/span&gt;,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Pr(T) = 0.6$&lt;/span&gt;,&lt;/p&gt;&#xA;&lt;p&gt;and that these two outcomes are mutually exhaustive:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$Pr(H) + Pr(T) = 1.0$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I can then use a binomial model to re-simulate data based on these parameters. I will observe various sequences, for example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;HTTTTTTHHT&lt;/code&gt;&#xA;&lt;code&gt;TTTTHHTHHH&lt;/code&gt;&#xA;&lt;code&gt;HTHHTHHTHH&lt;/code&gt;...&lt;/p&gt;&#xA;&lt;p&gt;However, based purely on the parameters of the model, I don't expect every new simulation to recapitulate the exact same &lt;code&gt;HTHTTTHHTT&lt;/code&gt; sequence, because I have reduced its information to some rate parameters for the simulation.&lt;/p&gt;&#xA;&lt;p&gt;Each of those new sequences will yield model parameters (&lt;span class=&quot;math-container&quot;&gt;$Pr(H), Pr(T)$&lt;/span&gt;) that are similar to the training data, but they will not necessarily be particularly similar to the training data in sequence.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Phylogenetic modeling&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Phylogenetic modeling works in the same way, such that you reduce the observations at the tips of the tree into some model with some parameters. Most of the time you are estimating e.g. a rate matrix of parameters for transition between states for a set of residues (in simplest case for DNA 4 nucleotide states and for proteins, 20 amino acid states usually). If you are working with gene sequences you are probably using the 4 nucleotide states or some derivation thereof.&lt;/p&gt;&#xA;&lt;p&gt;Simulations usually work by starting with some fixed ancestral state at the root (which can take various states), and then evolving forward from there along each branch of the tree using the rate matrix in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Continuous-time_Markov_chain&quot; rel=&quot;nofollow noreferrer&quot;&gt;continuous-time Markov chain&lt;/a&gt; to &amp;quot;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3509945/&quot; rel=&quot;nofollow noreferrer&quot;&gt;replay the tape of life&lt;/a&gt;&amp;quot;, by assuming that the model you've fit somewhat reflects truth.&lt;/p&gt;&#xA;&lt;p&gt;While opinions differ somewhat on what to expect from &amp;quot;replaying the tape of life&amp;quot; (see that linked review for more), basically no one expects to get the same results as the input data, or even 90% similar to the input data. That would indicate that evolution is highly deterministic, which just isn't reliably true. Of course there are situations where such determinism happens to be the case, but they are relatively rare, and I don't think that sequence evolution is considered to be very deterministic at all.&lt;/p&gt;&#xA;&lt;p&gt;It's a little different in that the tree defines a correlation structure of the tips, which constrains the simulation results to be somewhat more similar to the training data than we expect from coin-flipping. But there's still no reason to expect particularly similar results on the tips.&lt;/p&gt;&#xA;&lt;p&gt;But you can also model discrete characters on a tree, which can be even simpler. For example, I used the &lt;a href=&quot;https://rdrr.io/cran/ape/&quot; rel=&quot;nofollow noreferrer&quot;&gt;R APE package&lt;/a&gt; to simulate the following character states (A or B for each tip) on a simulated tree of 10 tips with the following commands:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tr = rtree(10)&#xA;c_tr = rTraitDisc(tr)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sfQVk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sfQVk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I can then build a model for this character and this tree using the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rates = ace(c_tr, tr, type=&amp;quot;discrete&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and use it to simulate new sets of states:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# A (index 1) is estimated to be ancestral state with 99% confidence&#xA;new_sim = rTraitDisc(tr, rate=rates$rates, states=c(&amp;quot;A&amp;quot;, &amp;quot;B&amp;quot;), root.value=1)&#xA;plot(tr)&#xA;tiplabels(new_sim, adj = c(1,0))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ei9Rk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ei9Rk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Look! Not a single observation of state B! Let's try it again:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;new_sim2 = rTraitDisc(tr, rate=rates$rates, states=c(&amp;quot;A&amp;quot;, &amp;quot;B&amp;quot;), root.value=1)&#xA;plot(tr)&#xA;tiplabels(new_sim2, adj = c(1,0))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CYM3y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CYM3y.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Ok, this is more similar, but there's still another state B on the tips.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Evolution is not very deterministic. The correlation structure of the tree makes the outcomes less random than independent trials as in coin-flipping, but it is not at all expected that simulation results from phylogenetic models will match the training observations well. The best that can be said is that each simulated dataset can hopefully be &lt;em&gt;reduced to the same model parameters&lt;/em&gt; as the input data, but that's a very different statement from yielding the exact same outcomes.&lt;/p&gt;&#xA;&lt;p&gt;If what you are looking for from simulation is to yield the exact same sequence states at the tips, then you are unlikely to get that from phylogenetic simulation. You might occasionally get pretty close, but that is not at all the expectation.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-08-16T21:34:26.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14164" PostTypeId="2" ParentId="14119" CreationDate="2020-08-16T23:21:20.277" Score="1" Body="&lt;p&gt;My understanding is that &lt;a href=&quot;https://sourceforge.net/p/pb-jelly/wiki/Home/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PBJelly&lt;/a&gt; is the tool intended to solve this problem.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://eagle.fish.washington.edu/trilobite/notebooks/sam/onsnetwork.org/kubu4/tag/pbjelly/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;This person&lt;/a&gt; seems to have used it starting with Illumina scaffolds. I have not personally used it but have heard that it works pretty well.&lt;/p&gt;&#xA;&lt;p&gt;Edit: note that the PacBio ecosystem is not super easy to use from scratch. However, they have made pretty good efforts to facilitate setup using conda, as for instance documented &lt;a href=&quot;https://github.com/PacificBiosciences/pbbioconda&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. &lt;a href=&quot;https://github.com/cgjosephlee/PBJelly&quot; rel=&quot;nofollow noreferrer&quot;&gt;This person&lt;/a&gt; shows a conda recipe for pbjelly.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-08-16T23:21:20.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14165" PostTypeId="2" ParentId="14140" CreationDate="2020-08-17T10:46:15.357" Score="1" Body="&lt;p&gt;You can also do this with &lt;a href=&quot;http://www.bonvinlab.org/pdb-tools/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pdb-tools&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pdb_selchain -A prot.pdb &amp;gt; prot_A.pdb&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2020-08-17T10:46:15.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14166" PostTypeId="1" AcceptedAnswerId="14187" CreationDate="2020-08-17T14:54:14.867" Score="1" ViewCount="175" Body="&lt;p&gt;I want to download RNA-seq datasets of 4 or 5 different types of cancer from the TCGA to investigate what is happening with my gene of interest. The problem is that I'm a physician and I don't know people used to deal with Big Data to help me.&lt;/p&gt;&#xA;&lt;p&gt;I'm using a Bioconductor program called TCGAbiolinks and I'm able to download some datasets using the following script:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;if (!requireNamespace(&amp;quot;BiocManager&amp;quot;, quietly = TRUE))&#xA;  install.packages(&amp;quot;BiocManager&amp;quot;)&#xA;&#xA;BiocManager::install(&amp;quot;TCGAbiolinks&amp;quot;)&#xA;&#xA;browseVignettes(&amp;quot;TCGAbiolinks&amp;quot;)&#xA;&#xA;library(TCGAbiolinks)&#xA;library(dplyr)&#xA;library(DT)&#xA;&#xA;query &amp;lt;- GDCquery(project = &amp;quot;TCGA-BRCA&amp;quot;, &#xA;                  data.category = &amp;quot;Gene expression&amp;quot;,&#xA;                  file.type  = &amp;quot;normalized_results&amp;quot;,&#xA;                  data.type = &amp;quot;Gene expression quantification&amp;quot;,&#xA;                  experimental.strategy = &amp;quot;RNA-Seq&amp;quot;, &#xA;                  barcode = c(&amp;quot;TCGA-AN-A046&amp;quot;, &amp;quot;TCGA-AC-A23H&amp;quot;),&#xA;                  legacy = TRUE)&#xA;&#xA;GDCdownload(query, method = &amp;quot;api&amp;quot;, files.per.chunk = 10)&#xA;&#xA;# GDCprepare transforms data in R format&#xA;&#xA;# The default GDCprepare output is a SummarizedExperiment (SE) object,&#xA;# but you can also set to output the data frame.&#xA;&#xA;data &amp;lt;- GDCprepare(query, summarizedExperiment = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eJLGY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eJLGY.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;The problems is that: 1) the gene list is not appearing in the first column; 2) I don't know how many datasets I'll be able to download and analyse without a server.. I'm working with my Macbook..&lt;/p&gt;&#xA;&lt;p&gt;Can someone familiar with the TCGA website or with the TCGAbiolinks help me with this project? I've been trying to progress in the last months, but the TCGA is a complex website and I'm feeling lost. I use R, I don't know SQL or other programming languages..&lt;/p&gt;&#xA;" OwnerUserId="6509" LastEditorUserId="6509" LastEditDate="2020-08-17T16:01:32.940" LastActivityDate="2020-08-19T17:42:00.250" Title="TCGA: gene IDs not appearing and other concerns" Tags="|r|rna-seq|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="14168" PostTypeId="1" CreationDate="2020-08-18T01:52:50.860" Score="1" ViewCount="109" Body="&lt;p&gt;I'm bit of a newbie with ONT analyses and am having some trouble with the Master Of Pores installation instructions (&lt;a href=&quot;https://biocorecrg.github.io/master_of_pores/install.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biocorecrg.github.io/master_of_pores/install.html&lt;/a&gt;). I've managed to install nextflow, cloned the repository and have downloaded albacore and guppy. But when I run the pipeline below, i get an error saying&#xA;nextflow: command not found. Any help would be most appreciated.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;curl https://get.nextflow.io | bash&#xA;mv nextflow ~/master_of_pores/NanoPreProcess/bin&#xA;nextflow run nanopreprocess.nf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9387" LastEditorUserId="9387" LastEditDate="2020-08-18T03:34:16.677" LastActivityDate="2021-05-16T16:02:00.263" Title="Master of Pores Installation Mac OS Mojave 10.14" Tags="|nanopore|software-installation|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14169" PostTypeId="1" CreationDate="2020-08-18T05:10:41.553" Score="0" ViewCount="118" Body="&lt;p&gt;I am doing my research on nanobodies. Currently, I am trying to do a comparison study of the flexibility of CDR loops using B-factor values. I would like to know if there is any statistical analysis tool for this or if anyone has done anything similar.&lt;/p&gt;&#xA;" OwnerUserId="9383" LastEditorUserId="9383" LastEditDate="2020-08-18T21:11:44.830" LastActivityDate="2020-08-18T21:56:24.383" Title="B-factor analysis in nanobodies" Tags="|proteins|statistics|protein-structure|computational-biochemistry|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14171" PostTypeId="1" CreationDate="2020-08-18T09:33:41.273" Score="1" ViewCount="195" Body="&lt;pre&gt;&lt;code&gt;0.980634         -                    572         sp|P06672|PDC_ZYMMO                         -                     568                               1.2e-263             874.5          0.0            1        1      6.6e-267  1.4e-263   874.2      0.0       5         562    15     570    12     572    0.98   Uncharacterized                         protein                   OS=Lomentospora            prolificans        OX=41688             GN=jhhlp_005678  PE=3             SV=1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've asked a similar question before. I have a better understanding of python now and would like to revisit this. Can someone critique my thinking?&lt;/p&gt;&#xA;&lt;p&gt;I have an HMM output file that contains about 20 fields. There are two sets of IDs, e.g., P006672 and jhhlp_005678&lt;/p&gt;&#xA;&lt;p&gt;In a separate file are FASTA-formatted sequences linked to these IDs.&lt;/p&gt;&#xA;&lt;p&gt;Proposed solution:&lt;/p&gt;&#xA;&lt;p&gt;1&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   a) read in fields containing IDs, lets called them fields 8 and 6; &#xA;&#xA;&#xA;   b) store field values in an array or dictionary (whatever the proper data structure is for &#xA;  this). Let's called this Fields_8_and_6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;a) read proteome file containing FASTA-formatted sequences&lt;/p&gt;&#xA;&lt;p&gt;b) Grep and store as ID:sequence key:value pairs; let's call this ID_to_sequence&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a) Compare Fields_8_and_6 to ID_to_sequence&lt;/p&gt;&#xA;&lt;p&gt;b) Write ID1:sequence1 ID2:sequence2 as pairs into one file&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;send to ClustalO&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7459" LastActivityDate="2024-04-02T13:27:26.603" Title="How can I match sequences with IDs using python dictionaries?" Tags="|bioconductor|python|proteomics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14172" PostTypeId="2" ParentId="14171" CreationDate="2020-08-18T09:45:18.050" Score="3" Body="&lt;p&gt;&lt;a href=&quot;http://biopython.org/DIST/docs/tutorial/Tutorial.html#sec63&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biopython can index a fastafile&lt;/a&gt; and you can access the sequences by their ID in a dict-like manner.&lt;/p&gt;&#xA;&lt;p&gt;All that is left for you to do is extract the IDs from your file. (&lt;a href=&quot;https://biopython.org/DIST/docs/api/Bio.SearchIO.HmmerIO-module.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;more Biopython&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="3818" LastActivityDate="2020-08-18T09:45:18.050" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14173" PostTypeId="1" AcceptedAnswerId="14185" CreationDate="2020-08-18T10:29:29.023" Score="0" ViewCount="38" Body="&lt;p&gt;Apologies if this is the wrong location for this question, and please feel free to point me in the right direction with appropriate Anglo-Saxon embellishments...&lt;/p&gt;&#xA;&lt;p&gt;I'm after some ball-park numbers for the following questions (plus any hints how to calculate them for different params - e.g. less generations, larger starting pop.):&lt;/p&gt;&#xA;&lt;p&gt;Given a starting population of 10,000, and given 2,500 generations, with an end population of 5,000,000,000&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What are the chances of being a direct descendant of 1 individual of the original 10,000?&lt;/li&gt;&#xA;&lt;li&gt;If you had a DNA sample of that individual, would you be able to tell if you were a direct descendant?&lt;/li&gt;&#xA;&lt;li&gt;Would siblings of the original individual confuse things with regard to Q2?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9389" LastActivityDate="2020-08-19T15:14:15.797" Title="Tracing and quantifying inheritance over many generations" Tags="|phylogenetics|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14174" PostTypeId="2" ParentId="14169" CreationDate="2020-08-18T11:56:23.410" Score="1" Body="&lt;p&gt;So B-factors, for &amp;quot;blur factors&amp;quot;, are how much movement an atom has in the crystal structure. They are not reliable. B-factor putty in PyMOL is the most you should push them.&lt;/p&gt;&#xA;&lt;h2&gt;B-factors bad&lt;/h2&gt;&#xA;&lt;p&gt;This is partly a result of flexibility, but it is highly influenced by crystal packing, which is not natural. Many enzymes, for example, are seen with an active site entrance way that is closed up in a crystal structure whereas in an MD run there is a gate loop that flops around.&lt;/p&gt;&#xA;&lt;p&gt;Additionally, a B-factor is a single value that masks lots of forms of movement. Partial occupancy after all is at the discretion of the experimenter, so you might even have a residue in two or three low energy states that result nevertheless in B-factor disorder.&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, incorrectly fitted residues will have a higher B-factor.&lt;/p&gt;&#xA;&lt;p&gt;Also, the meaning of B-factor are mighty cryptic, and depend on the refinement —the units are Å^2 as opposed to say Å, like a standard deviation.&lt;/p&gt;&#xA;&lt;h2&gt;MD RMSF good&lt;/h2&gt;&#xA;&lt;p&gt;The best way to determine the movement of a residue is to do an MD run. &lt;a href=&quot;http://www.gromacs.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gromacs&lt;/a&gt; has very nice tutorial and is the most accessible.&#xA;The metric in that case is the RMSF, which is the RMSD relative to the average. Gromacs has a &lt;a href=&quot;http://manual.gromacs.org/documentation/5.1/onlinehelp/gmx-rmsf.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;utility for it called rmsf&lt;/a&gt;. Alternatively RMSD relative to your original structure is fine —but less of a metric of movement.&#xA;MD is complained about for three reasons, none are valid here:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Tricky to set up: this is a straightforward case&lt;/li&gt;&#xA;&lt;li&gt;Lengthy: You would not be after large scale conformational changes so a limited amount of steps would suffice —possibly even on a laptop on a single core for a couple of hours (cluster is always better though!). Running at different temperatures and switching the intermediate steps (replica exchange) is often done to speed things up, but would just confuse the results.&lt;/li&gt;&#xA;&lt;li&gt;Small molecules: A nanobody has no cofactors or small molecule ligands so you'd avoid the topology parameterisation issues. If Eukaryotic made, your nanobodies may be glycosylated — play it safe and ignore.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;B-factor extraction&lt;/h2&gt;&#xA;&lt;p&gt;If you want to extract the B-factors, you can use pymol as a python module —&lt;a href=&quot;https://anaconda.org/schrodinger/pymol&quot; rel=&quot;nofollow noreferrer&quot;&gt;installed with conda&lt;/a&gt; and not the binary/exe installer. And use python as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pdb_filename = &amp;quot;&amp;quot;&#xA;selector = &amp;quot;name CA&amp;quot;&#xA;import pymol2&#xA;with pymol2.PyMOL() as pymol:&#xA;     pymol.cmd.load(pdb_filename)&#xA;     bfactors = {atom.resi: atom.b for atom in pymol.cmd.get_model(selector).atom}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From there do whatever you fancy with the b-factors. For alternative selector strings see &lt;a href=&quot;https://pymolwiki.org/index.php/Selection_Algebra&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pymol selection algebra&lt;/a&gt;. One operation is to divide the values by 8 pi^2 and take the square root of that, which is the amplitude of oscillation (U) of the atom around its equilibrium position —ie. that stated in the coordinates.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-08-18T21:56:24.383" LastActivityDate="2020-08-18T21:56:24.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14175" PostTypeId="2" ParentId="14155" CreationDate="2020-08-18T13:17:54.660" Score="2" Body="&lt;p&gt;I don't know how to do that in R but I think this is what you want:&lt;/p&gt;&#xA;&lt;p&gt;I used &lt;a href=&quot;http://json2table.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://json2table.com&lt;/a&gt; for view the data structure of the JSON file, then I used &lt;code&gt;json_normalize&lt;/code&gt; to normalize the JSON data into a flat table using the key &lt;code&gt;[result]&lt;/code&gt; to a list of DataFrames for each index in the JSON file. Then, I concatenated the list of DataFrames in a single one with &lt;code&gt;df.concat()&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import json&#xA;import pandas as pd&#xA;&#xA;with open('analysis.json') as f:&#xA;    d = json.load(f)&#xA;&#xA;dfs = []&#xA;for x, y in enumerate(d):&#xA;    data = pd.json_normalize(d[x]['result'])&#xA;    dfs.append(data)&#xA;&#xA;df = pd.concat(dfs, ignore_index=True)&#xA;&#xA;df.to_csv('analysis.csv', index=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2020-08-18T13:17:54.660" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14176" PostTypeId="2" ParentId="278" CreationDate="2020-08-18T14:18:02.363" Score="0" Body="&lt;p&gt;You can download public single-cell RNA-seq data or bulk RNA-seq for purified primary cells for reference and then use CIBERSORT or MuSiC to deconvolve.&#xA;One thing to notice: to make bulk RNA-seq and single-cell reference compatible, TPM from RNA-seq can be compared with CPM in 3' (10X,Drop-seq etc.) single-cell data and TPM in full-length single-cell data (SMARTseq etc.)&lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2020-08-18T14:18:02.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14177" PostTypeId="2" ParentId="278" CreationDate="2020-08-18T16:31:11.030" Score="0" Body="&lt;p&gt;Actually a month ago, I was searching for the same question and I found a few packages that can be used. First of all the functionality of them can be cell proportion estimation, cell specific gene expression estimation, or both of these functions. Secondly, their primary input, as expected, is bulk RNA-seq data and some also want a single cell RNA-seq data as secondary input.&#xA;The most popular one is the child of famous cibersort, CIBERSORTX. I went through their website and read the tutorials. They have several example datasets that can be used for learning their platform. However, the two biggest flaws were that they haven't shared their source code which means we can't reproduce the result on our own systems and we have to upload all of our data to Stanford servers (security concerns!). The second problem is that there is no API access to their server and everything should be done online which means very low compatibility with other tools.&lt;/p&gt;&#xA;&lt;p&gt;The idea, using deconvolution and deep learning to infer subpopulation of cells from bulk RNA-seq, is great. So, I searched to see if others have also done something similar and maybe even better. I found three other tools:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;MuSiC:   &lt;a href=&quot;https://www.nature.com/articles/s41467-018-08023-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41467-018-08023-x&lt;/a&gt;&lt;br /&gt;&#xA;Published in nature communication and a few months older than cibersortx. I haven't studied their algorithm yet and so I can't say if their method is better than cibersortx or not. However, the part that makes the difference is that all of their works, including their source code, is open-source. So, I can access their code and we don't need to upload our unpublished data into some not safe online server (unlike cibersortx). The second good thing is that it is implemented in R language which I use for most of my RNA-seq data analysis, so it is easily compatible with other tools.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Scaden : &lt;a href=&quot;https://advances.sciencemag.org/content/6/30/eaba2619.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://advances.sciencemag.org/content/6/30/eaba2619.full&lt;/a&gt;&#xA;This one is also open-source and accessible easily. It is written in Python and runs in the shell. So, it is somewhat compatible with other tools (not as compatible as MuSic). We again can process our data locally and don't need to upload our data to any servers (which is good). They claim that their method has a better correlation with the real scRNA-seq data than both cibersortx and MuSic (not always!!). Running their tools seems easy based on their tutorials (I haven't run it on my system yet!). However, their tutorials seem to be a little confusing and it seems like it is still in pre-release version and they have bugs!!&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;CDSeq:&#xA;&lt;a href=&quot;https://github.com/kkang7/CDSeq_R_Package&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/kkang7/CDSeq_R_Package&lt;/a&gt;&#xA;Finally, I found the CDSeq package which doesn't need any scRNA-seq input to find single cell proportions and cell-specific gene expression in bulk data. It just gets your buld counts as input and info about very few parameters and does the whole thing automatically, and suprisingly good, I checked! If you could find gene expression pattern of your specific cells and also use it as an input, it is optional, it will give you even a better estimation.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9299" LastActivityDate="2020-08-18T16:31:11.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14178" PostTypeId="1" CreationDate="2020-08-19T03:13:46.610" Score="1" ViewCount="2203" Body="&lt;p&gt;It seems that we have two calculations of log fold change:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Actual log2(FC) = log2(mean(Group1)/mean(Group2))&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Limma's &amp;quot;Log(FC)&amp;quot; = mean(log2(Group1)) - mean(log2(Group2))&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;For volcano plots, can we use the second one?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/100460/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/100460/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9392" LastEditorUserId="3694" LastEditDate="2022-01-13T22:40:53.997" LastActivityDate="2022-01-13T22:40:53.997" Title="About the log2 fold change" Tags="|r|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14179" PostTypeId="1" CreationDate="2020-08-19T06:52:23.020" Score="0" ViewCount="1325" Body="&lt;p&gt;I am doing an RNA seq analysis with mouse samples. I had my reads in several lanes, so I concatenated those fastq reads. Then I've mapped my reads with Hisat2 against the new version of mouse  &amp;quot;GCA_000001635.9_GRCm39_genomic.fna&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Apparently everything works well up to here. Now I'm using samtools 1.9 to convert sam to bam, to sort those bam and create the index, just like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -bS &lt;span class=&quot;math-container&quot;&gt;${file} &amp;gt; $&lt;/span&gt;{file}.bam&#xA;&#xA;samtools sort -n &lt;span class=&quot;math-container&quot;&gt;${file} -o $&lt;/span&gt;{file}_sorted.bam&#xA;&#xA;samtools index  ${file}_sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But when indexing I get this error:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;[bam_sort_core] merging from 11 files and 1 in-memory blocks...&#xA;[E::hts_idx_push] NO_COOR reads not in a single block at the end 0 -1&#xA;samtools index: failed to create index for&#xA;&amp;quot;CD4_1A_concat.bam_sorted.bam&amp;quot;: No such file or directory&lt;/p&gt;&#xA;&lt;p&gt;[bam_sort_core] merging from 11 files and 1 in-memory blocks...&#xA;[E::hts_idx_push] NO_COOR reads not in a single block at the end 23 -1&#xA;samtools index: failed to create index for&#xA;&amp;quot;CD4_1B_concat.bam_sorted.bam&amp;quot;: No such file or directory&#xA;[bam_sort_core] merging from 11 files and 1 in-memory blocks...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And more...&lt;/p&gt;&#xA;&lt;p&gt;Any help would be appreciated!&lt;/p&gt;&#xA;" OwnerUserId="9393" LastEditorUserId="3694" LastEditDate="2020-08-19T17:22:28.197" LastActivityDate="2020-08-20T07:06:22.450" Title="NO_COOR reads not in a single block at the end 0 -1" Tags="|rna-seq|samtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14180" PostTypeId="2" ParentId="14179" CreationDate="2020-08-19T07:29:20.887" Score="2" Body="&lt;p&gt;This is essentially a duplicate of &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/13997/samtools-index-chromosome-blocks-not-continuous&quot;&gt;Samtools Index: Chromosome Blocks not Continuous&lt;/a&gt; but with different error messages, and a duplicate of &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/5352/error-given-while-trying-to-index-a-bam-file-with-samtools-index-no-coor&quot;&gt;Error given while trying to index a BAM file with Samtools Index - NO COOR?&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools sort -n ${file} -o ${file}_sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These error messages also indicate that the reads are not sorted by coordinate — in particular, that the unplaced unmapped reads with RNAME &lt;code&gt;*&lt;/code&gt; are not all together at the end of the file.&lt;/p&gt;&#xA;&lt;p&gt;This is because &lt;code&gt;samtools sort -n&lt;/code&gt; has been used to sort the reads by name instead. Remove &lt;code&gt;-n&lt;/code&gt; to sort by position, which is what is needed to prepare a BAM file for indexing with &lt;code&gt;samtools index&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="134" LastEditorUserId="134" LastEditDate="2020-08-20T07:06:22.450" LastActivityDate="2020-08-20T07:06:22.450" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14181" PostTypeId="1" AcceptedAnswerId="14199" CreationDate="2020-08-19T08:10:06.870" Score="1" ViewCount="117" Body="&lt;p&gt;I am performing a GWAS analysis and try to understand the influence of the minor allele count filter.Setting the filter to 1 % gave me this plot and I am confused about the same -log10 pvalues around ~ 3.8 which shows a strange pattern (many dots in one line with same value).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/w9x4I.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/w9x4I.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If I increase the filter to 6 % the dots in a line disappear but I loose also some significant SNPs:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4tMdD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4tMdD.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Sample are about 77000 SNP positions (barley), unimputed from GWAS (all NaNs removed). GWAS was performed with 200 phenotype samples.&lt;/p&gt;&#xA;&lt;p&gt;How to interpret the dots in  a line?&lt;/p&gt;&#xA;" OwnerUserId="9278" LastEditorUserId="9278" LastEditDate="2020-08-21T07:04:03.690" LastActivityDate="2020-08-21T07:21:06.680" Title="GWAS MAC filter Interpretation" Tags="|python|gwas|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14182" PostTypeId="2" ParentId="14168" CreationDate="2020-08-19T10:48:30.693" Score="3" Body="&lt;p&gt;you should not move nextflow within &lt;code&gt;~/master_of_pores/NanoPreProcess/bin&lt;/code&gt; .&#xA;Just add nextflow in your path.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;export PATH=$PATH:where_is_your_nf_executable&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Feel free to open an issue on github if you need more help.&lt;/p&gt;&#xA;" OwnerUserId="9395" LastEditorUserId="57" LastEditDate="2020-08-19T15:59:28.483" LastActivityDate="2020-08-19T15:59:28.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14184" PostTypeId="1" AcceptedAnswerId="14233" CreationDate="2020-08-19T13:53:05.200" Score="3" ViewCount="965" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/google/deepvariant/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepVariant&lt;/a&gt; is a pipeline to call genetic variants from DNA sequencing data.&lt;/p&gt;&#xA;&lt;p&gt;A major step, before feeding the CNN, is to translate these DNA sequences into images. It's unclear why and how Google &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/092890v6&quot; rel=&quot;nofollow noreferrer&quot;&gt;constructs the RGB images&lt;/a&gt; from the DNA data. Obviously, DNA is a string over an alphabet with the characters: {A, T, C, G}.&lt;/p&gt;&#xA;&lt;p&gt;It is even hard to understand how the &lt;a href=&quot;http://ai.googleblog.com/2017/12/deepvariant-highly-accurate-genomes.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;mapping&lt;/a&gt; works based on the source code of their &lt;a href=&quot;https://github.com/google/deepvariant/blob/r0.10/deepvariant/pileup_image_test.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;unit tests&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In their figure from the paper: A is Red, C is Green, G is blue, and T is Yellow (G+R), but this is still unclear how they construct the &lt;code&gt;3xNxN&lt;/code&gt; image.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/d9NPSm.pnge&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/d9NPSm.pnge&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;EDIT from &lt;a href=&quot;https://google.github.io/deepvariant/posts/2020-02-20-looking-through-deepvariants-eyes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;google's blog&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In this article we will show the six channels in a row, but in&#xA;DeepVariant they are encoded as six layers in the third dimension,&#xA;giving each tensor a shape of (100, 221, 6) corresponding to (height,&#xA;width, channels). The variant in question is always in the center of&#xA;each pileup image, here marked with a small line at the top.&lt;/p&gt;&#xA;&lt;p&gt;Channels are shown in greyscale below in the following order:&lt;/p&gt;&#xA;&lt;p&gt;Read base: different intensities represent A, C, G, and T.&lt;/p&gt;&#xA;&lt;p&gt;Base quality: set by the sequencing machine. White is higher quality.&lt;/p&gt;&#xA;&lt;p&gt;Mapping quality: set by the aligner. White is higher quality.&lt;/p&gt;&#xA;&lt;p&gt;Strand of alignment: Black is forward; white is reverse.&lt;/p&gt;&#xA;&lt;p&gt;Read supports variant: White means the read supports the given&#xA;alternate allele, grey means it does not.&lt;/p&gt;&#xA;&lt;p&gt;Base differs from ref: White means the base is different from the&#xA;reference, dark grey means the base matches the reference.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MoGCs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MoGCs.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2021-01-02T11:27:10.823" LastActivityDate="2021-01-02T11:27:10.823" Title="How does DeepVariant construct RGB images from DNA sequences?" Tags="|python|variant-calling|machine-learning|deep-learning|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14185" PostTypeId="2" ParentId="14173" CreationDate="2020-08-19T14:15:53.340" Score="0" Body="&lt;p&gt;Since it seems my comment helped, I've been suggested to post this as an answer...&lt;/p&gt;&#xA;&lt;p&gt;I'd reccomend everyone interested in these sort of population genetics to have a look at 'The Ancestors Tale' by Richard Dawkins. In particular, chapter 0, which deals with human/hominid genetics. That should help answer questions. The relevant part of the book appears to be available online for free in GoogleBooks (&lt;a href=&quot;https://books.google.co.uk/books/about/The_Ancestor_s_Tale.html?id=s86_CQAAQBAJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://books.google.co.uk/books/about/The_Ancestor_s_Tale.html?id=s86_CQAAQBAJ&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Below I'll note down the answers from the comments section, not because I wrote them but I figured it might help anyone who stumbled over this question afterwards:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The chances of being a direct descendant of 1 individual of the original 10,000 is 100%, assuming they have any descendants. - Tom Melly&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If one had a DNA sample of that individual, one could use uniparental markers (either Y chromosome for men, or mtDNA for women) to infer whether they are a direct descendant. - 4galaxy7&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;(My guess is that it doesn't have to be uniparental markers. Other genes might work just as well. I'm thinking of Neanderthal/Denisovan work. But I might be wrong.)&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2020-08-19T14:15:53.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14186" PostTypeId="2" ParentId="14173" CreationDate="2020-08-19T14:36:20.577" Score="0" Body="&lt;h2&gt;Question 1&lt;/h2&gt;&#xA;&lt;p&gt;If I understand the question, then you are asking 'Given a starting population of 10 000 individuals and 2 500 generations, what is the probability that one of the 5 000 000 000 individuals descendend individuals is directly descended from one of the founders?'.&lt;/p&gt;&#xA;&lt;p&gt;If we are assuming no migration in or out of the system, then the answer must be 1, since there are no ways to be descended from someone outside the system. Please let me know if I have misunderstood the question.&lt;/p&gt;&#xA;&lt;h2&gt;Question 2&lt;/h2&gt;&#xA;&lt;p&gt;For this, it's only possible to use uni-parental markers. This is because inferring exact genealogical relationships using autosomal markers beyond something like 3rd cousins is extremely difficult, &lt;a href=&quot;https://gcbias.org/2013/10/20/how-much-of-your-genome-do-you-inherit-from-a-particular-grandparent/&quot; rel=&quot;nofollow noreferrer&quot;&gt;owing to the stochastic way in which recombination breaks up segments&lt;/a&gt;. Beyond a few generations, only the uniparentally inherited mtDNA and non-recombining Y-chromosome can be informative about relatedness.&lt;/p&gt;&#xA;&lt;p&gt;Uniparental markers don't recombine, so mutations aside, you expect to inherit an exact copy of your mothers mtDNA marker. If you know the number of generations which are likely to separate you and the putative ancestor, you can estimate the some kind of likelihood of being directly descended from the ancestor given the number of pairwise differences between the individual and the ancestor.&lt;/p&gt;&#xA;&lt;p&gt;This is what they did when they wanted to &lt;a href=&quot;https://www.nature.com/articles/ncomms6631&quot; rel=&quot;nofollow noreferrer&quot;&gt;identify the remains of Richard III.&lt;/a&gt; It's a good paper and easy to read / open access, so I'd recommend checking it out.&lt;/p&gt;&#xA;&lt;h2&gt;Question 3&lt;/h2&gt;&#xA;&lt;p&gt;Simply, no. The inference of being a direct descendent from one of the ancestors would be independent from any of the other potential descendants in the sample.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="3694" LastEditDate="2020-08-19T15:14:15.797" LastActivityDate="2020-08-19T15:14:15.797" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14187" PostTypeId="2" ParentId="14166" CreationDate="2020-08-19T17:33:23.260" Score="3" Body="&lt;p&gt;I had to play around a little to understand the object, but here's what I got:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;R version 4.0.2 (2020-06-22) -- &amp;quot;Taking Off Again&amp;quot;&#xA;Copyright (C) 2020 The R Foundation for Statistical Computing&#xA;Platform: x86_64-apple-darwin17.0 (64-bit)&#xA;&#xA;R is free software and comes with ABSOLUTELY NO WARRANTY.&#xA;You are welcome to redistribute it under certain conditions.&#xA;Type 'license()' or 'licence()' for distribution details.&#xA;&#xA;  Natural language support but running in an English locale&#xA;&#xA;R is a collaborative project with many contributors.&#xA;Type 'contributors()' for more information and&#xA;'citation()' on how to cite R or R packages in publications.&#xA;&#xA;Type 'demo()' for some demos, 'help()' for on-line help, or&#xA;'help.start()' for an HTML browser interface to help.&#xA;Type 'q()' to quit R.&#xA;&#xA;&amp;gt; library(TCGAbiolinks)&#xA;&amp;gt; query &amp;lt;- GDCquery(project = &amp;quot;TCGA-BRCA&amp;quot;,&#xA;+                   data.category = &amp;quot;Gene expression&amp;quot;,&#xA;+                   file.type  = &amp;quot;normalized_results&amp;quot;,&#xA;+                   data.type = &amp;quot;Gene expression quantification&amp;quot;,&#xA;+                   experimental.strategy = &amp;quot;RNA-Seq&amp;quot;,&#xA;+                   barcode = c(&amp;quot;TCGA-AN-A046&amp;quot;, &amp;quot;TCGA-AC-A23H&amp;quot;),&#xA;+                   legacy = TRUE)&#xA;--------------------------------------&#xA;o GDCquery: Searching in GDC database&#xA;--------------------------------------&#xA;Genome of reference: hg19&#xA;--------------------------------------------&#xA;oo Accessing GDC. This might take a while...&#xA;--------------------------------------------&#xA;ooo Project: TCGA-BRCA&#xA;--------------------&#xA;oo Filtering results&#xA;--------------------&#xA;ooo By experimental.strategy&#xA;ooo By data.type&#xA;ooo By file.type&#xA;ooo By barcode&#xA;----------------&#xA;oo Checking data&#xA;----------------&#xA;ooo Check if there are duplicated cases&#xA;ooo Check if there results for the query&#xA;-------------------&#xA;o Preparing output&#xA;-------------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; GDCdownload(query, method = &amp;quot;api&amp;quot;, files.per.chunk = 10)&#xA;Downloading data for project TCGA-BRCA&#xA;GDCdownload will download 3 files. A total of 1.306357 MB&#xA;Downloading chunk 1 of 1 (3 files, size = 1.306357 MB) as Wed_Aug_19_12_20_11_2020_0.tar.gz&#xA;&#xA;Downloading: 580 kB&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; data &amp;lt;- GDCprepare(query, summarizedExperiment = TRUE)&#xA;------------------&#xA;oo Reading 3 files&#xA;------------------&#xA;|====================================================|100%                      Completed after 0 s&#xA;------------------&#xA;oo Merging 3 files&#xA;------------------&#xA;Accessing grch37.ensembl.org to get gene information&#xA;Downloading genome information (try:0) Using: Human genes (GRCh37.p13)&#xA;Starting to add information to samples&#xA; =&amp;gt; Add clinical information to samples&#xA; =&amp;gt; Adding TCGA molecular information from marker papers&#xA; =&amp;gt; Information will have prefix 'paper_'&#xA;brca subtype information from:doi.org/10.1016/j.ccell.2018.03.014&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(data)&#xA;class: RangedSummarizedExperiment&#xA;dim: 6 3&#xA;metadata(1): data_release&#xA;assays(1): normalized_count&#xA;rownames(6): A1BG A2M ... RP11-986E7.7 AADAC&#xA;rowData names(3): gene_id entrezgene ensembl_gene_id&#xA;colnames(3): TCGA-AN-A046-01A-21R-A034-07 TCGA-AC-A23H-11A-12R-A157-07&#xA;  TCGA-AC-A23H-01A-11R-A157-07&#xA;colData names(83): barcode patient ... paper_PARADIGM Clusters&#xA;  paper_Pan-Gyn Clusters&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I see from this representation that I can use &lt;code&gt;metadata(data)&lt;/code&gt;, &lt;code&gt;assays(data)&lt;/code&gt;, etc to get more information.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(SummarizedExperiment::assays(data))&#xA;List of length 1&#xA;names(1): normalized_count&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alright, so this is a named list. I know how to deal with lists: I can use either the index (&lt;code&gt;[[1]]&lt;/code&gt;) or the name (&lt;code&gt;$normalized_count&lt;/code&gt;)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(SummarizedExperiment::assays(data)$normalized_count)&#xA;             TCGA-AN-A046-01A-21R-A034-07 TCGA-AC-A23H-11A-12R-A157-07 TCGA-AC-A23H-01A-11R-A157-07&#xA;A1BG                             121.9879                      38.5795                      13.6874&#xA;A2M                             9703.8622                   37640.8373                    4767.0490&#xA;NAT1                            8488.0727                     114.8582                     270.4753&#xA;NAT2                               6.0583                       4.2938                       2.3804&#xA;RP11-986E7.7                   33016.6641                   11308.3460                    3894.9639&#xA;AADAC                              1.1359                     219.6976                       1.1902&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I can now get more information on the samples using &lt;code&gt;colData()&lt;/code&gt;. I'm truncating the output below because there are 83 columns.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; SummarizedExperiment::colData(data)&#xA;DataFrame with 3 rows and 83 columns&#xA;                                                  barcode      patient           sample shortLetterCode          definition sample_submitter_id         sample_type       state&#xA;                                              &amp;lt;character&amp;gt;  &amp;lt;character&amp;gt;      &amp;lt;character&amp;gt;     &amp;lt;character&amp;gt;         &amp;lt;character&amp;gt;         &amp;lt;character&amp;gt;         &amp;lt;character&amp;gt; &amp;lt;character&amp;gt;&#xA;TCGA-AN-A046-01A-21R-A034-07 TCGA-AN-A046-01A-21R-A034-07 TCGA-AN-A046 TCGA-AN-A046-01A              TP Primary solid Tumor    TCGA-AN-A046-01A       Primary Tumor    released&#xA;TCGA-AC-A23H-11A-12R-A157-07 TCGA-AC-A23H-11A-12R-A157-07 TCGA-AC-A23H TCGA-AC-A23H-11A              NT Solid Tissue Normal    TCGA-AC-A23H-11A Solid Tissue Normal    released&#xA;TCGA-AC-A23H-01A-11R-A157-07 TCGA-AC-A23H-01A-11R-A157-07 TCGA-AC-A23H TCGA-AC-A23H-01A              TP Primary solid Tumor    TCGA-AC-A23H-01A       Primary Tumor    released&#xA;                                                        sample_id   is_ffpe sample_type_id  tissue_type submitter_id days_to_collection oct_embedded                pathology_report_uuid&#xA;                                                      &amp;lt;character&amp;gt; &amp;lt;logical&amp;gt;    &amp;lt;character&amp;gt;  &amp;lt;character&amp;gt;  &amp;lt;character&amp;gt;          &amp;lt;integer&amp;gt;  &amp;lt;character&amp;gt;                          &amp;lt;character&amp;gt;&#xA;TCGA-AN-A046-01A-21R-A034-07 57aff09f-0e97-4e65-a227-3dc7a8516367     FALSE             01 Not Reported TCGA-AN-A046                 27         true 1304FB17-A20A-4EBC-9CC8-1554808AC1F6&#xA;TCGA-AC-A23H-11A-12R-A157-07 7df59ca8-7e51-4581-9d7f-8bba0395ce17     FALSE             11 Not Reported TCGA-AC-A23H                478        false                                   NA&#xA;TCGA-AC-A23H-01A-11R-A157-07 d7e3b628-d5fd-4e79-9c4a-6409330fb8a7     FALSE             01 Not Reported TCGA-AC-A23H                478        false A7C7D409-D086-4A9B-8C8F-E7E231D5891D&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I can get more information on the genes using &lt;code&gt;rowData()&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; SummarizedExperiment::rowData(data)&#xA;DataFrame with 19947 rows and 3 columns&#xA;                  gene_id entrezgene ensembl_gene_id&#xA;              &amp;lt;character&amp;gt;  &amp;lt;integer&amp;gt;     &amp;lt;character&amp;gt;&#xA;A1BG                 A1BG          1 ENSG00000121410&#xA;A2M                   A2M          2 ENSG00000175899&#xA;NAT1                 NAT1          9 ENSG00000171428&#xA;NAT2                 NAT2         10 ENSG00000156006&#xA;RP11-986E7.7 RP11-986E7.7         12 ENSG00000273259&#xA;...                   ...        ...             ...&#xA;RASAL2-AS1     RASAL2-AS1  100302401 ENSG00000224687&#xA;LINC00882       LINC00882  100302640 ENSG00000242759&#xA;FTX                   FTX  100302692 ENSG00000230590&#xA;TICAM2             TICAM2  100302736 ENSG00000243414&#xA;SLC25A5-AS1   SLC25A5-AS1  100303728 ENSG00000224281&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You want the assay data as a data.matrix, I guess. That'd be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; data_as_matrix &amp;lt;- data.matrix(SummarizedExperiment::assays(data)$normalized_count)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My session Info:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; sessionInfo()&#xA;R version 4.0.2 (2020-06-22)&#xA;Platform: x86_64-apple-darwin17.0 (64-bit)&#xA;Running under: macOS Catalina 10.15.5&#xA;&#xA;Matrix products: default&#xA;BLAS:   /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRblas.dylib&#xA;LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib&#xA;&#xA;locale:&#xA;[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8&#xA;&#xA;attached base packages:&#xA;[1] stats     graphics  grDevices utils     datasets  methods   base&#xA;&#xA;other attached packages:&#xA;[1] TCGAbiolinks_2.16.1&#xA;&#xA;loaded via a namespace (and not attached):&#xA;  [1] bitops_1.0-6                matrixStats_0.56.0          bit64_0.9-7.1               doParallel_1.0.15           RColorBrewer_1.1-2          progress_1.2.2&#xA;  [7] httr_1.4.2                  GenomeInfoDb_1.24.2         backports_1.1.8             tools_4.0.2                 R6_2.4.1                    DBI_1.1.0&#xA; [13] BiocGenerics_0.34.0         colorspace_1.4-1            tidyselect_1.1.0            gridExtra_2.3               prettyunits_1.1.1           bit_1.1-15.2&#xA; [19] curl_4.3                    compiler_4.0.2              rvest_0.3.6                 Biobase_2.48.0              xml2_1.3.2                  DelayedArray_0.14.0&#xA; [25] rtracklayer_1.48.0          scales_1.1.1                survMisc_0.5.5              readr_1.3.1                 genefilter_1.70.0           askpass_1.1&#xA; [31] rappdirs_0.3.1              stringr_1.4.0               digest_0.6.25               Rsamtools_2.4.0             foreign_0.8-80              R.utils_2.9.2&#xA; [37] rio_0.5.16                  XVector_0.28.0              pkgconfig_2.0.3             dbplyr_1.4.4                readxl_1.3.1                rlang_0.4.7&#xA; [43] ggthemes_4.2.0              RSQLite_2.2.0               generics_0.0.2              zoo_1.8-8                   jsonlite_1.7.0              BiocParallel_1.22.0&#xA; [49] zip_2.0.4                   car_3.0-8                   dplyr_1.0.0                 R.oo_1.23.0                 RCurl_1.98-1.2              magrittr_1.5&#xA; [55] GenomeInfoDbData_1.2.3      Matrix_1.2-18               Rcpp_1.0.5                  munsell_0.5.0               S4Vectors_0.26.1            abind_1.4-5&#xA; [61] lifecycle_0.2.0             R.methodsS3_1.8.0           stringi_1.4.6               carData_3.0-4               SummarizedExperiment_1.18.1 zlibbioc_1.34.0&#xA; [67] plyr_1.8.6                  BiocFileCache_1.12.0        grid_4.0.2                  blob_1.2.1                  parallel_4.0.2              ggrepel_0.8.2&#xA; [73] forcats_0.5.0               crayon_1.3.4                survminer_0.4.8             lattice_0.20-41             haven_2.3.1                 Biostrings_2.56.0&#xA; [79] splines_4.0.2               GenomicFeatures_1.40.0      annotate_1.66.0             hms_0.5.3                   knitr_1.29                  pillar_1.4.6&#xA; [85] ggpubr_0.4.0                GenomicRanges_1.40.0        ggsignif_0.6.0              codetools_0.2-16            biomaRt_2.44.1              stats4_4.0.2&#xA; [91] XML_3.99-0.5                glue_1.4.1                  downloader_0.4              data.table_1.13.0           vctrs_0.3.2                 selectr_0.4-2&#xA; [97] foreach_1.5.0               cellranger_1.1.0            gtable_0.3.0                openssl_1.4.2               purrr_0.3.4                 tidyr_1.1.0&#xA;[103] km.ci_0.5-2                 assertthat_0.2.1            ggplot2_3.3.2               openxlsx_4.1.5              xfun_0.16                   xtable_1.8-4&#xA;[109] broom_0.7.0                 rstatix_0.6.0               survival_3.1-12             tibble_3.0.3                iterators_1.0.12            KMsurv_0.1-5&#xA;[115] GenomicAlignments_1.24.0    AnnotationDbi_1.50.1        memoise_1.1.0               IRanges_2.22.2              ellipsis_0.3.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastEditorUserId="650" LastEditDate="2020-08-19T17:42:00.250" LastActivityDate="2020-08-19T17:42:00.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14189" PostTypeId="1" AcceptedAnswerId="14190" CreationDate="2020-08-20T11:08:40.250" Score="-2" ViewCount="62" Body="&lt;p&gt;I have two groups of patients&lt;/p&gt;&#xA;&lt;p&gt;I have a list of driver genes&lt;/p&gt;&#xA;&lt;p&gt;Clearly each group have mutation in a subset of these gene some of which are common between two groups and some of which are unique to one group&lt;/p&gt;&#xA;&lt;p&gt;I want a union count matrix of two groups&lt;/p&gt;&#xA;&lt;p&gt;Let's TP53 only has been mutated in one group, I want TP53 in union matrix putting 0 where absent in one group&lt;/p&gt;&#xA;&lt;p&gt;This is how my data look&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dput(d1)&#xA;structure(c(1L, 1L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 1L, 2L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, &#xA;0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;1L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, &#xA;0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;1L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, &#xA;1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, &#xA;1L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 1L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 2L, 0L, 0L, 0L, 1L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 1L, 0L, 0L, 0L, 1L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;1L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, &#xA;0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 1L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;1L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L), .Dim = c(31L, 26L), .Dimnames = list(&#xA;    c(&amp;quot;TP53&amp;quot;, &amp;quot;TSHZ3&amp;quot;, &amp;quot;ARID2&amp;quot;, &amp;quot;CDKN2A&amp;quot;, &amp;quot;NOTCH1&amp;quot;, &amp;quot;APC&amp;quot;, &amp;quot;ARID1A&amp;quot;, &#xA;    &amp;quot;EPHA3&amp;quot;, &amp;quot;RNF43&amp;quot;, &amp;quot;SCN3A&amp;quot;, &amp;quot;SMAD4&amp;quot;, &amp;quot;ACVR2A&amp;quot;, &amp;quot;ARID1B&amp;quot;, &amp;quot;BRAF&amp;quot;, &#xA;    &amp;quot;CCDC102B&amp;quot;, &amp;quot;CCNE1&amp;quot;, &amp;quot;CDH1&amp;quot;, &amp;quot;CHD4&amp;quot;, &amp;quot;EGFR&amp;quot;, &amp;quot;EPHA2&amp;quot;, &amp;quot;ERBB2&amp;quot;, &#xA;    &amp;quot;FBXW7&amp;quot;, &amp;quot;KDM6A&amp;quot;, &amp;quot;LIN7A&amp;quot;, &amp;quot;MAP2K7&amp;quot;, &amp;quot;MET&amp;quot;, &amp;quot;MSH3&amp;quot;, &amp;quot;PIK3CA&amp;quot;, &#xA;    &amp;quot;PTEN&amp;quot;, &amp;quot;RASA1&amp;quot;, &amp;quot;SLIT2&amp;quot;), c(&amp;quot;LP6008336-DNA_G01&amp;quot;, &amp;quot;LP6008334-DNA_B02&amp;quot;, &#xA;    &amp;quot;LP6008334-DNA_A04&amp;quot;, &amp;quot;s24&amp;quot;, &amp;quot;s15&amp;quot;, &amp;quot;LP6008334-DNA_C02&amp;quot;, &amp;quot;LP6008460-DNA_D01&amp;quot;, &#xA;    &amp;quot;LP6005500-DNA_D03&amp;quot;, &amp;quot;LP6008337-DNA_A07&amp;quot;, &amp;quot;LP6007600&amp;quot;, &amp;quot;LP6008337-DNA_H06&amp;quot;, &#xA;    &amp;quot;LP6005334-DNA_H01&amp;quot;, &amp;quot;LP6008460-DNA_A04&amp;quot;, &amp;quot;LP6008334-DNA_A03&amp;quot;, &#xA;    &amp;quot;LP6008334-DNA_D02&amp;quot;, &amp;quot;LP6008336-DNA_H01&amp;quot;, &amp;quot;s30&amp;quot;, &amp;quot;s86&amp;quot;, &amp;quot;s59&amp;quot;, &#xA;    &amp;quot;LP6008336-DNA_F02&amp;quot;, &amp;quot;s67&amp;quot;, &amp;quot;LP6008460-DNA_F02&amp;quot;, &amp;quot;s94&amp;quot;, &amp;quot;s18&amp;quot;, &#xA;    &amp;quot;LP6008460-DNA_G03&amp;quot;, &amp;quot;s80&amp;quot;)))&#xA;&amp;gt; dput(d2)&#xA;structure(c(1L, 1L, 1L, 1L, 0L, 0L, 0L, 1L, 0L, 1L, 0L, 1L, 0L, &#xA;2L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 1L, 1L, 1L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 1L, 1L, 1L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 1L, 1L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;1L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 1L, 2L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 1L, 0L, 2L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 1L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 1L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 1L, &#xA;1L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, &#xA;1L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, &#xA;0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 3L, &#xA;0L, 0L, 0L, 0L, 1L, 1L, 1L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;1L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, &#xA;1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L, &#xA;0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;1L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, &#xA;0L, 0L, 1L, 2L, 2L, 0L, 1L, 0L, 0L, 1L, 1L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 0L, 1L, 0L, 0L, &#xA;0L, 0L, 0L, 0L, 0L, 0L), .Dim = c(33L, 35L), .Dimnames = list(&#xA;    c(&amp;quot;TP53&amp;quot;, &amp;quot;NAV3&amp;quot;, &amp;quot;CDKN2A&amp;quot;, &amp;quot;SMAD4&amp;quot;, &amp;quot;KCNQ3&amp;quot;, &amp;quot;ARID1A&amp;quot;, &amp;quot;ARID2&amp;quot;, &#xA;    &amp;quot;LRRK2&amp;quot;, &amp;quot;EPHA3&amp;quot;, &amp;quot;KRAS&amp;quot;, &amp;quot;LIN7A&amp;quot;, &amp;quot;PIK3CA&amp;quot;, &amp;quot;PTEN&amp;quot;, &amp;quot;RNF43&amp;quot;, &#xA;    &amp;quot;NOTCH1&amp;quot;, &amp;quot;PBRM1&amp;quot;, &amp;quot;SCN3A&amp;quot;, &amp;quot;SLIT2&amp;quot;, &amp;quot;TSHZ3&amp;quot;, &amp;quot;APC&amp;quot;, &amp;quot;ARID1B&amp;quot;, &#xA;    &amp;quot;CCDC102B&amp;quot;, &amp;quot;CTNNB1&amp;quot;, &amp;quot;EGFR&amp;quot;, &amp;quot;ERBB2&amp;quot;, &amp;quot;GPATCH8&amp;quot;, &amp;quot;MSH3&amp;quot;, &#xA;    &amp;quot;NIPBL&amp;quot;, &amp;quot;POLQ&amp;quot;, &amp;quot;SIN3A&amp;quot;, &amp;quot;SMARCA4&amp;quot;, &amp;quot;STK11&amp;quot;, &amp;quot;ZFHX3&amp;quot;), c(&amp;quot;LP6005500-DNA_H03&amp;quot;, &#xA;    &amp;quot;LP6008031-DNA_B02&amp;quot;, &amp;quot;LP6008269-DNA_E10&amp;quot;, &amp;quot;LP6005334-DNA_B02&amp;quot;, &#xA;    &amp;quot;LP6008269-DNA_H05&amp;quot;, &amp;quot;LP6008031-DNA_D02&amp;quot;, &amp;quot;LP6005500-DNA_G01&amp;quot;, &#xA;    &amp;quot;LP6005334-DNA_H03&amp;quot;, &amp;quot;LP2000329-DNA_A01&amp;quot;, &amp;quot;LP6005690-DNA_F02&amp;quot;, &#xA;    &amp;quot;LP6008337-DNA_B07&amp;quot;, &amp;quot;LP6008031-DNA_B03&amp;quot;, &amp;quot;LP6005500-DNA_C03&amp;quot;, &#xA;    &amp;quot;LP6005690-DNA_A02&amp;quot;, &amp;quot;LP6005690-DNA_E01&amp;quot;, &amp;quot;LP6008334-DNA_B01&amp;quot;, &#xA;    &amp;quot;LP6008051-DNA_B01&amp;quot;, &amp;quot;LP6005409-DNA_B02&amp;quot;, &amp;quot;LP6005409-DNA_F01&amp;quot;, &#xA;    &amp;quot;LP6005935-DNA_D02&amp;quot;, &amp;quot;LP6005935-DNA_F01&amp;quot;, &amp;quot;LP6008221-DNA_C03&amp;quot;, &#xA;    &amp;quot;LP6008280-DNA_D02&amp;quot;, &amp;quot;LP6008337-DNA_F06&amp;quot;, &amp;quot;LP6008202-DNA_F02&amp;quot;, &#xA;    &amp;quot;LP6008280-DNA_B02&amp;quot;, &amp;quot;LP6005334-DNA_C02&amp;quot;, &amp;quot;LP6008031-DNA_E01&amp;quot;, &#xA;    &amp;quot;SM-4AX87&amp;quot;, &amp;quot;LP6005409-DNA_E02&amp;quot;, &amp;quot;LP6008031-DNA_D01&amp;quot;, &amp;quot;LP6008202-DNA_A03&amp;quot;, &#xA;    &amp;quot;LP6008051-DNA_A01&amp;quot;, &amp;quot;LP2000325-DNA_A01&amp;quot;, &amp;quot;LP6005409-DNA_H01&amp;quot;&#xA;    )))&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have tried&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plyr::rbind.fill.matrix(d1,d2)&#xA;&#xA;rbind(d1,d2,all=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But the outputs seem crab while  need a trustful data for machine learning&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2020-08-20T11:40:30.900" Title="Making a union data from different list of genes" Tags="|r|merge|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14190" PostTypeId="2" ParentId="14189" CreationDate="2020-08-20T11:40:30.900" Score="1" Body="&lt;p&gt;Maybe something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df = merge(&#xA;data.frame(gene=rownames(d1),d1),&#xA;data.frame(gene=rownames(d2),d2),all=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you want to replace NA with 0:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df[is.na(df)] = 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-08-20T11:40:30.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14192" PostTypeId="1" CreationDate="2020-08-20T19:40:08.777" Score="0" ViewCount="85" Body="&lt;p&gt;I have a set of &lt;code&gt;RNA-seq&lt;/code&gt; samples from targeting different proteins in a complex with &lt;code&gt;siRNAs&lt;/code&gt;. However, the &lt;code&gt;PCA&lt;/code&gt; plot shows that &lt;strong&gt;the distance between the two replicates of each treatment is more than the distance between the treatments&lt;/strong&gt;. So, there is &lt;strong&gt;no clear border between treatments&lt;/strong&gt; to cluster them separately.&lt;/p&gt;&#xA;&lt;p&gt;If I go through my usual RNA-seq data analysis pipeline using &lt;code&gt;EdgeR&lt;/code&gt; normalization and &lt;code&gt;Limma/Voom&lt;/code&gt; DEG analysis, I get a lot of DEGs. However, when I try to visualize them using a heatmap, the replicates do not cluster together, as expected.&lt;/p&gt;&#xA;&lt;p&gt;What should I do in this situation? I mean should I trust the DEGs, or try to still extract co-expressed gene modules from this data?&lt;/p&gt;&#xA;&lt;p&gt;my code for normalization:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;myDGEList &amp;lt;- DGEList(Txi_gene$counts)&#xA;log2.cpm &amp;lt;- cpm(myDGEList, log=TRUE)&#xA;&#xA;cpm &amp;lt;- cpm(myDGEList)&#xA;&#xA;# prepare your grouping matrix&#xA;group &amp;lt;- factor(targets$group)&#xA;design &amp;lt;- model.matrix(~group)&#xA;&#xA;# filtering&#xA;keepers &amp;lt;- filterByExpr(cpm, design)&#xA;myDGEList.filtered &amp;lt;- myDGEList[keepers,]&#xA;log2.cpm.filtered &amp;lt;- cpm(myDGEList.filtered, log=TRUE)&#xA;&#xA;myDGEList.filtered.norm &amp;lt;- calcNormFactors(myDGEList.filtered, method = &amp;quot;TMM&amp;quot;)&#xA;log2.cpm.filtered.norm &amp;lt;- cpm(myDGEList.filtered.norm, log=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;my code for PCA plot:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pca.res &amp;lt;- prcomp(t(log2.cpm.filtered.norm), scale.=F, retx=T)&#xA;pc.var &amp;lt;- pca.res&lt;span class=&quot;math-container&quot;&gt;$sdev^2&#xA;pc.per &amp;lt;- round(pc.var/sum(pc.var)*100, 1) &#xA;pca.res.df &amp;lt;- as_tibble(pca.res$&lt;/span&gt;x)&#xA;&#xA;# PC1 vs PC2&#xA;pca.plot &amp;lt;- ggplot(pca.res.df) +&#xA;  aes(x=PC1, y=PC2, label=sampleLabels, color = group, shape = batch) +&#xA;  geom_point(size=4) +&#xA;  xlab(paste0(&amp;quot;PC1 (&amp;quot;,pc.per[1],&amp;quot;%&amp;quot;,&amp;quot;)&amp;quot;)) + &#xA;  ylab(paste0(&amp;quot;PC2 (&amp;quot;,pc.per[2],&amp;quot;%&amp;quot;,&amp;quot;)&amp;quot;)) +&#xA;  labs(title=&amp;quot;PCA plot&amp;quot;,&#xA;       caption=paste0(&amp;quot;produced on &amp;quot;, Sys.time())) +&#xA;  coord_fixed() +&#xA;  theme_bw()&#xA;&#xA;ggplotly(pca.plot)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;the PCA plot figure. The legend is cropped out for privacy reasons. Colors represent replicates. Shapes represent batch effect which was has been fixed using &lt;code&gt;combat_seq&lt;/code&gt; on raw-data.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/QxbWh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QxbWh.png&quot; alt=&quot;PCA plot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9299" LastEditorUserId="9299" LastEditDate="2020-08-21T08:18:41.010" LastActivityDate="2020-08-21T16:22:31.293" Title="the variation between treatments is less than the variation between replicates in RNA-seq data" Tags="|rna-seq|deseq2|edger|pca|limma|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14193" PostTypeId="2" ParentId="10459" CreationDate="2020-08-20T20:10:12.247" Score="2" Body="&lt;p&gt;rsIDs change with the dbSNP builds, not with the reference genome builds. rsIDs do not depend on a reference genome. They point to a specified locus regardless of the differences in genomic assemblies. rsIDs numbersare assigned by dbSNP database maintainers. The numbers may be deleted from database (e.g. when a submitter withdraw data) and may change on merges (e.g. when multiple rsIDs were assigned at the same genomic location).&lt;/p&gt;&#xA;&lt;p&gt;Specifying the location by rsID is stable, and can be only affected by an eventual merge or deletion, if any, with a new dbSNP database build. It cannot be affected by a reference genome build.&lt;/p&gt;&#xA;&lt;p&gt;However, a base position (offset) of a locus within a chromosome may change between the builds of the reference genome. So, when specifying the location by the chromosome and base position, the reference genome build should be also given, to locate an appropriate rsID.&lt;/p&gt;&#xA;&lt;p&gt;You can download dbSNP database files (in VCF format) of the database build that was actual at a time when a specific build of a reference genome was also actual, but these files are huge. A possible place to download is &lt;a href=&quot;https://ftp.ncbi.nih.gov/snp/pre_build152/organisms/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nih.gov/snp/pre_build152/organisms/&lt;/a&gt; or &lt;a href=&quot;https://ftp.ncbi.nih.gov/snp/archive/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nih.gov/snp/archive/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, these files are arranged by the dbSNP database build (e.g. 152, 153, 154), not by the reference genome build (e.g. 37 or 38).&lt;/p&gt;&#xA;&lt;p&gt;If your list is relatively small, you can fetch the data rs numbers via dbSNP API by chromosome and location. The API is available at &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/docs/eutils_help/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/snp/docs/eutils_help/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For instance, do “advanced search” of dbSNP at &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/advanced&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/snp/advanced&lt;/a&gt;&#xA;When it comes to search by the base position, you can only specify positions for GRCh37 (called “Base Position Previous”) and GRCh38 (“Base Position”). This URL will help you construct the request string that you will further use for the dbSNP API requests.&lt;/p&gt;&#xA;&lt;p&gt;You may also need to add “NOT MergedRS[All Fields]” to exclude rsIDs that have already been merged. I wish I knew what to type instead of “All Fields” here to refine the search, but the documentation is very sparse.&lt;/p&gt;&#xA;&lt;p&gt;In your examples, for &lt;code&gt;13_28025615_G_C_b37&lt;/code&gt; specify Chromosome=13 and Base Position Previous=28025615 in the “Advanced Search”.&lt;/p&gt;&#xA;&lt;p&gt;As about the API, for &lt;code&gt;13_28025615_G_C_b37&lt;/code&gt; query &lt;a href=&quot;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=snp&amp;amp;term=13%5BChromosome%5D+AND+28025615%5BBase+Position+Previous%5D+NOT+MergedRS%5BAll+Fields%5D&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=snp&amp;amp;term=13[Chromosome]+AND+28025615[Base+Position+Previous]+NOT+MergedRS[All+Fields]&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and for &lt;code&gt;chr22_50473581_G_A_b38&lt;/code&gt; query &lt;a href=&quot;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=snp&amp;amp;term=22%5BChromosome%5D+AND+50473581%5BBase+Position%5D+NOT+MergedRS%5BAll+Fields%5D&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=snp&amp;amp;term=22[Chromosome]+AND+50473581[Base+Position]+NOT+MergedRS[All+Fields]&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The first rsID that you will get will be the main current rsID, the others (if any) will be the merged ones.&lt;/p&gt;&#xA;&lt;p&gt;Here is a small perl example on how to retrieve data from the dbSNP. The fetch_GRCh37 function returns the rsID for a given chromosome and base position. For GRCh38, remove the “+Previous” from the URL. You can call the fetch_GRCh37 in a script multiple times. It’s better that you get rsIDs for your both GRCh37 and GRCh38 data to make sure that you refer to the same rsIDs that are actual after eventual merges.&lt;/p&gt;&#xA;&lt;p&gt;Please make sure to read and comply to the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK25497/#chapter2.Usage_Guidelines_and_Requiremen&quot; rel=&quot;nofollow noreferrer&quot;&gt;requirements&lt;/a&gt; before using the API.&lt;/p&gt;&#xA;&lt;p&gt;A sample Perl script snippet follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;use XML::LibXML;&#xA;use LWP::UserAgent;&#xA;&#xA;sub fetch_GRCh37 {&#xA;        my &lt;span class=&quot;math-container&quot;&gt;$chromosome = $&lt;/span&gt;_[0];&#xA;        my &lt;span class=&quot;math-container&quot;&gt;$basepos = $&lt;/span&gt;_[1];&#xA;        my &lt;span class=&quot;math-container&quot;&gt;$url = 'https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=snp&amp;amp;term='.$&lt;/span&gt;chromosome.'[Chromosome]+AND+'.&lt;span class=&quot;math-container&quot;&gt;$basepos.'[Base+Position+Previous]+NOT+MergedRS[All+Fields]';&#xA;        my $&lt;/span&gt;UserAgent = new LWP::UserAgent ();&#xA;        my &lt;span class=&quot;math-container&quot;&gt;$GET = $&lt;/span&gt;UserAgent-&amp;gt;get(&lt;span class=&quot;math-container&quot;&gt;$url)-&amp;gt;content;&#xA;        my $&lt;/span&gt;dom = XML::LibXML-&amp;gt;load_xml(string =&amp;gt; &lt;span class=&quot;math-container&quot;&gt;$GET);&#xA;        my $&lt;/span&gt;numberofnodes = int(&lt;span class=&quot;math-container&quot;&gt;$dom-&amp;gt;findnodes('eSearchResult/Count')-&amp;gt;to_literal());&#xA;        if ($&lt;/span&gt;numberofnodes &amp;gt; 0) {foreach my &lt;span class=&quot;math-container&quot;&gt;$title ($&lt;/span&gt;dom-&amp;gt;findnodes('eSearchResult/IdList/Id')){return(&amp;quot;rs&amp;quot;.$title-&amp;gt;to_literal());}}&#xA;}&#xA;&#xA;print fetch_GRCh37(13, 28025615);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9402" LastEditorUserId="9402" LastEditDate="2020-08-22T17:44:50.183" LastActivityDate="2020-08-22T17:44:50.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14195" PostTypeId="1" CreationDate="2020-08-20T20:20:43.140" Score="0" ViewCount="130" Body="&lt;p&gt;I have an &lt;code&gt;RNA-seq&lt;/code&gt; dataset from &lt;code&gt;siRNAs&lt;/code&gt; targeting different components of a molecular pathway. the problem is that the &lt;strong&gt;siRNA target gene is less down-regulated from several other genes&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;I'd like to know which genes are down-regulated due to the off-target effect of the siRNA and which ones are down-regulated because of the down-regulation of my target gene.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, how can I find the &lt;strong&gt;off-targets&lt;/strong&gt; of my siRNA considering having the RNA-seq data and also the siRNA sequence?&lt;/p&gt;&#xA;&lt;p&gt;If there is no package in R, Python, or shell. Could you please introduce me to an online tool for finding potential off-targets using my siRNA sequence. That way I can search for off-targets and then exclude them from my DGE list manually.&lt;/p&gt;&#xA;" OwnerUserId="9299" LastEditorUserId="9299" LastEditDate="2020-08-21T08:23:14.317" LastActivityDate="2020-08-21T08:23:14.317" Title="How to find siRNA off-targets from RNA-seq data?" Tags="|rna-seq|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14196" PostTypeId="2" ParentId="3114" CreationDate="2020-08-20T20:26:13.350" Score="0" Body="&lt;p&gt;These are only the base positions that have changed between GRCh37 and GRCh38, not the IDs. For example if a study (e.g. found via pubmed) refers to a particular rsID, the genome build does not matter.&lt;/p&gt;&#xA;&lt;p&gt;rsIDs do not depend on a reference genome. They point to a specified locus regardless of the differences in genomic assemblies. As a result, you cannot convert rsIDs of one genome build to rsIDs of another build. And there is no such mapping tool available, even if they pretend to be. rsIDs are assigned by dbSNP maintainers, and are stable. The numbers may only change on merge, where multiple numbers were assigned at the same genomic location, and the higher number gets merged into a lower number by the dbSNP maintainers, unless there is a very well-published, notable rsID mentioned in the press, in this case that merged number is considered the mentioned one, not the smaller one.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/docs/RefSNP_about/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefSNP number is the stable accession regardless of the differences in genomic assemblies.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;For more information go to the “RefSNP Number Stability” section at &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK21088/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/books/NBK21088/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The rsIDs may also be deleted in some cases see the following links on the explanation:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK44467/#Build.i_found_rs123456_in_the_current_bu&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/books/NBK44467/#Build.i_found_rs123456_in_the_current_bu&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK44468/#Build.the__rscurrent_field_in_rsmergearc&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/books/NBK44468/#Build.the__rscurrent_field_in_rsmergearc&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Thus, specifying the location by rsID is stable, and can be only affected by an eventual merge or delete from the dbSNP database, if any, not by a new genome build. You can only “update” the rsIDs from time to time to make sure they all are the main rsID, not those larger numbers already merged to a smaller one. It should be noted that the base positions (offsets) of the same genomic location may change between the builds of the reference genome. And they did surely change between GRCh37 and GRCh38.&lt;/p&gt;&#xA;&lt;p&gt;If your list is relatively small, you can get most recent rsIDs for your chromosome and offset via dbSNP API. It is available at &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/docs/eutils_help/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/snp/docs/eutils_help/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The query for the API is the same as for the “advanced search” of dbSNP at &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/advanced&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/snp/advanced&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you specify data in a “Base Position” line, it will look for GRCh38 base position. If you need to specify the base position for a GRCh37, use “Base Position Previous” line. You may also need to add &lt;code&gt;NOT MergedRS[All Fields]&lt;/code&gt; to exclude rsIDs that have already been merged.&lt;/p&gt;&#xA;&lt;p&gt;For example, to get the rsID for a chromosome 1 GRCh37 base position 249222527, the API query URL is the following &lt;a href=&quot;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=snp&amp;amp;term=1%5BChromosome%5D+AND+249222527%5BBase+Position+Previous%5D+NOT+MergedRS%5BAll+Fields%5D&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esearch.fcgi?db=snp&amp;amp;term=1[Chromosome]+AND+249222527[Base+Position+Previous]+NOT+MergedRS[All+Fields]&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can do queries from a script if the number of requests is relatively small. You may find a perl script, for example, that does the job.&lt;/p&gt;&#xA;&lt;p&gt;Even though the rsID numbers can be merged or deleted, and the changes are reflected in a new dbSNP database build, given the fact that the changes in numbers do not depend on the genome build, is better to always rely to the rsID number from the latest dbSNP database build.&lt;/p&gt;&#xA;&lt;p&gt;The changes in rsID numbers are changed due to merges or deletions between dbSNP builds relatively infrequently, given the total number of the IDs in the database.&lt;/p&gt;&#xA;" OwnerUserId="9402" LastEditorUserId="9402" LastEditDate="2020-08-22T17:27:17.867" LastActivityDate="2020-08-22T17:27:17.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14197" PostTypeId="2" ParentId="14178" CreationDate="2020-08-20T21:38:11.743" Score="0" Body="&lt;p&gt;You can use what limma reports for everything, including volcano plots. In the post you link to I describe in a comment why this is done in every package that I know.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-08-20T21:38:11.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14198" PostTypeId="1" AcceptedAnswerId="14241" CreationDate="2020-08-20T23:38:04.243" Score="6" ViewCount="447" Body="&lt;p&gt;I'm not sponsored or anything, just interested in their challenge to decipher their DNA code.&lt;/p&gt;&#xA;&lt;p&gt;They encoded their first episode of &amp;quot;Biohackers&amp;quot; video/binary file to DNA code and said if we could decode it we can watch it (without Netflix).&#xA;Here's their page: &lt;a href=&quot;https://biohackersnetflix.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biohackersnetflix.com&lt;/a&gt; with description and download for the DNA sequence file.&#xA;(Don't know if it's just in German or you can translate it. If questions regarding this page, ask me.)&lt;/p&gt;&#xA;&lt;p&gt;The file is ~550MB small and contains 3.882.771 lines (not in fasta format). Every line has a length of 147 characters including primers at both ends (Illumina?).&#xA;Here are the first 5 lines:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ACACGACGCTCTTCCGATCTCTCCCAGGGACAAAGGTTCTGCATTTGCAGCAAGACTCCTGTAGTGCTGCAGATTCTCTGGTTGGATAGTACGGCGTACATTTCTGTATTGTAGCACCATGGGGTAGATCGGAAGAGCACACGTCT&#xA;ACACGACGCTCTTCCGATCTTAAGGCTTCGTAACAGATATTCTATATCGTCACATTGGTCTGAAGGAAGTCGCCTATAATCGCTCCTCTGTTTTTTAAAACTGCTATGGACCCGCTGTTCGGTGGAGATCGGAAGAGCACACGTCT&#xA;ACACGACGCTCTTCCGATCTCATGGTATAAGTGTTAAGGGTAATAACCACCTACCCCCCTCATTGCTCGTTTTTCCTGGAACCTTAACATTCGCAATAGCTAGCTGTTTCCTAGTAGAACCAAGGAGATCGGAAGAGCACACGTCT&#xA;ACACGACGCTCTTCCGATCTAGGATGTAGTCACAGGTCATTGTCATTAACTCAACCGAGGACATAACACTAAGTCCCACTAGGCCTGGATTCTCTAACGCGGTCTCTCTATTGGGGGAAGGGGTGAGATCGGAAGAGCACACGTCT&#xA;ACACGACGCTCTTCCGATCTTCTGGTAAGGCGGGTTGATATCAGTCACCTCCCTTTGAGCTAAAATACGATGGCGATTTAGTGTGAAACTAATAATGCTTGTCATACCAGCAGTACCGGATCGGGAGATCGGAAGAGCACACGTCT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I trimmed all the primers and tried to decode {A, C, G, T} considering every permutation {00, 01, 10, 11} as the obvious(?) decryption method (4! = 24 possible decodings) using python.&lt;/p&gt;&#xA;&lt;p&gt;Then I hoped to get 1 of these 24 files loaded into VLC media player or something to be played, but it didn't work and every file seemed to be broken in the same way. I think I'm missing something here.&lt;/p&gt;&#xA;&lt;p&gt;Can I assume that a text file containing only 0's and 1's should be playable in VLC if the DNA code is correctly decrypted?&lt;/p&gt;&#xA;&lt;p&gt;(If I am wrong here, please tell or move me.)&lt;/p&gt;&#xA;&lt;p&gt;//Edit: I converted all 24 files to ASCII to see if there's some kind of &amp;quot;video-like header&amp;quot;. (All videos have some sort of description in their first lines if opened in text editor?) But there's just gibberish.&lt;/p&gt;&#xA;&lt;p&gt;//Edit: I saw that every 84th sequence position has a &amp;quot;T&amp;quot;, which is kind of weird.&#xA;So I tried to run my script again with these T's removed, but still no solution.&lt;/p&gt;&#xA;&lt;p&gt;//Edit: I searched for &amp;quot;AVI&amp;quot;, &amp;quot;264&amp;quot;, &amp;quot;codec&amp;quot; and some other strings in every video file I created and hexdumped. Nothing found.&#xA;For clarification: I translated the DNA into every 24 binary and then into their ASCII representation following the 19 upvotes answer: &lt;a href=&quot;https://stackoverflow.com/questions/7290943/write-a-string-of-1s-and-0s-to-a-binary-file&quot;&gt;https://stackoverflow.com/questions/7290943/write-a-string-of-1s-and-0s-to-a-binary-file&lt;/a&gt;. The 104 bases / 208 Bits (removed repetetive &amp;quot;T&amp;quot; and primers) are actually a multiple of 8 (respectively 26 Bytes) so I could be on the right way (even if not 32 Bytes?).&#xA;De novo Assembly didn't work and I found no obvious ORF &amp;quot;genes&amp;quot; representing some kind of URL to the video or something which was a neat idea considering the video file would be only ~150MB. (See comments.)&lt;/p&gt;&#xA;" OwnerUserId="9403" LastEditorUserId="9403" LastEditDate="2020-08-25T21:22:32.967" LastActivityDate="2020-08-26T20:40:27.850" Title="Biohackers Netflix - DNA to binary and video" Tags="|python|phylogenetics|encode|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="14199" PostTypeId="2" ParentId="14181" CreationDate="2020-08-21T07:21:06.680" Score="1" Body="&lt;p&gt;This looks like genotyping failed for some samples. It's unclear from the op whether the genotyping data was controlled for quality, but I would recommend performing sample QC.&lt;/p&gt;&#xA;&lt;p&gt;On a side note: in human gwas with imputed snps, I would expect to see clusters of significant snps as a sign that the significant snps are not an artifact. In the op there are some single significant snps. This might be a sign that those significant snps are artifacts. There are also some snps that form clusters and therefore look more promising.&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2020-08-21T07:21:06.680" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14200" PostTypeId="1" CreationDate="2020-08-21T11:08:19.040" Score="0" ViewCount="30" Body="&lt;p&gt;From a RNA seq experiment we look for the counts of transcripts of tRNA but we are also interested of other types of RNA we can find. Because of the protocol used for the library prep we can't analyse miRNA but we can analyse ncRNA.&#xA;I never analyzed transcripts for ncRNA and I have some questions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The name of the transcripts are like ENST00000611544.1, ENST00000387069.1, since they are non coding, are they linked to a gene? Should I rename the name of each ncRNA with a gene name or working with these long names is ok?&lt;/li&gt;&#xA;&lt;li&gt;When analyzing transcripts of RNA I did enrichment analysis to see what pathways are more active. But when analyzing transcripts for ncRNA is possible to do something similar to enrichment analysis? How can analyse the functions of the ncRNA that we get from the study?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Thank you very much in advance&lt;/p&gt;&#xA;" OwnerUserId="9010" LastActivityDate="2020-08-21T17:28:04.423" Title="Analyzing transcripts for non-coding RNA" Tags="|rna-seq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14202" PostTypeId="1" CreationDate="2020-08-21T12:43:08.633" Score="0" ViewCount="487" Body="&lt;p&gt;I did molecular dynamic simulations with DESMOND.&lt;/p&gt;&#xA;&lt;p&gt;Now, I want to obtain the interaction free energy using MM/PB(GB)SA.&lt;/p&gt;&#xA;&lt;p&gt;The free tools I found to perform such calculations are &lt;a href=&quot;https://ambermd.org/AmberTools.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;AmberTools&lt;/a&gt; and &lt;a href=&quot;http://rashmikumari.github.io/g_mmpbsa/&quot; rel=&quot;nofollow noreferrer&quot;&gt;g_mmpbsa&lt;/a&gt;.&#xA;AmberTools work only on Amber-type files, whereas g_mmbbsa works with GROMACS-type. This imply that I have to convert the DESMOND trajectories to AMBER or GROMACS formats.&lt;/p&gt;&#xA;&lt;p&gt;I did that using VMD. In case of AMBER, I got the files &lt;em&gt;prmtop&lt;/em&gt; and &lt;em&gt;mdcrd&lt;/em&gt; whereas for GROMACS I got &lt;em&gt;gro&lt;/em&gt; and &lt;em&gt;top&lt;/em&gt; files. I also tried to use &lt;a href=&quot;https://github.com/shirtsgroup/InterMol&quot; rel=&quot;nofollow noreferrer&quot;&gt;InterMol&lt;/a&gt;, but I am getting an error on conversions.&lt;/p&gt;&#xA;&lt;p&gt;For both tools, I need to prepare the input files. In case of AmberTools, there is a Python script (ante-MMPBSA) that extract the solvated complex, the complex, the ligand and the receptor from the trajectory files. This scrip is falling to do that from the converted trajectories. In case of g_mmpbsa, it also need the index.ndx file.&lt;/p&gt;&#xA;&lt;p&gt;My question: &lt;strong&gt;Is there a way to accomplish this task?&lt;/strong&gt; Any suggestion is welcome!&lt;/p&gt;&#xA;" OwnerUserId="9085" LastActivityDate="2020-08-21T12:43:08.633" Title="MM/PB(GB)SA calculations for DESMOND trajectories" Tags="|software-recommendation|gromacs|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14204" PostTypeId="2" ParentId="14192" CreationDate="2020-08-21T16:22:31.293" Score="1" Body="&lt;p&gt;If you got DEGs with statistically significant p-values after multiple testing following a valid pipeline, &lt;strong&gt;I'd believe that&lt;/strong&gt;, even if looking at the first few PCs doesn't look promising.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-08-21T16:22:31.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14205" PostTypeId="2" ParentId="14200" CreationDate="2020-08-21T17:28:04.423" Score="2" Body="&lt;p&gt;ncRNAs are by itself not special. Sure they might be more unstable and less abundant than other transcripts but for the analysis it is more or less the same. You should take your transcript level counts and aggregate them to the gene level. The tximport package comes to mind. Every transcript (given it is annotated) is linked to a gene. ENST00000611544 is connected to the U2 gene but these snRNAs are also short and I doubt you see them in normal RNA-seq.&lt;/p&gt;&#xA;&lt;p&gt;In the end you can process data as any RNA-seq. Quantify your data against a transcriptome, e.g. with &lt;code&gt;salmon&lt;/code&gt;, then aggregate transcript level abundance estimates (so basically counts) to the gene level and see whether your genes of interest have counts you can work with or whether you mostly have zeros. A transcriptome can be obtaine e.g. from GENCODE. The available annotation files will tell you which genes these transcripts originate from. Check for the GTF files.&lt;/p&gt;&#xA;&lt;p&gt;Towards enrichment analysis, well it all comes down to the annotations you have. If the ncRNAs are annotated in any pathway then you can perform standard analysis with tools such as goseq or gprofiler2. If they are not annotated, well then there is no information available.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-08-21T17:28:04.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14206" PostTypeId="1" CreationDate="2020-08-21T18:14:57.567" Score="1" ViewCount="320" Body="&lt;p&gt;Hi I have a dataset with chromosome and position.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;VARIANT_ID   chromosome position&#xA;17_26797415_147499  17  26797415&#xA;17_26797556_147500  17  26797556    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I assign rs id using R? This data have more than 500 SNPs.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Update: &lt;a href=&quot;https://www.ensembl.org/info/data/biomart/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioMart&lt;/a&gt; was used as suggested by @RamRS&lt;/li&gt;&#xA;&lt;li&gt;'NA NOT FOUND' was a resulting error.&lt;/li&gt;&#xA;&lt;li&gt;@RamRS recommended inputing &lt;code&gt;na.omit&lt;/code&gt; in the input vector&lt;/li&gt;&#xA;&lt;li&gt;This appear to have resolved the situation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9392" LastEditorUserId="3848" LastEditDate="2022-07-12T20:26:30.963" LastActivityDate="2022-07-12T20:26:30.963" Title="About getting rs id from chromosome and position" Tags="|r|variants|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14208" PostTypeId="2" ParentId="3881" CreationDate="2020-08-22T10:38:53.220" Score="1" Body="&lt;p&gt;For different variation type at the same location, each variation type is being assigned its own rsID by the dbSNP maintainers.&lt;/p&gt;&#xA;&lt;p&gt;In your example, one chromosome in the pair has a single-nucleotide variation (T allele), while the other chromosome has an insertion variation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK21088/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quote:&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;dbSNP only merges rs numbers that have an identical set of mappings to a genome and the same allele type (e.g. both must be the same variation type and share one allele in common).  We therefore would not merge a SNP and an indel (insertion/deletion) into a single rs number (different variation classes) since they represent two different types of mutational &amp;quot;events&amp;quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In your case, rs6010164 is of the Single Nucleotide Variation (SNV) variation type, while rs267607236 is of the Small deletions or insertions (DELINS) variation type.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/docs/RefSNP_about/#variation-type&quot; rel=&quot;nofollow noreferrer&quot;&gt;Quote:&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Despite its name, RefSNP is assigned to all variation types listed below with precise locations for both common and rare variations, including mutations. Most are typically small variations (&amp;lt;= 50bp).&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Single nucleotide variation (SNV)&lt;/li&gt;&#xA;&lt;li&gt;Short multi-nucleotide changes (MNV)&lt;/li&gt;&#xA;&lt;li&gt;Small deletions or insertions&lt;/li&gt;&#xA;&lt;li&gt;Small STR repeats&lt;/li&gt;&#xA;&lt;li&gt;retrotransposable element insertions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9402" LastActivityDate="2020-08-22T10:38:53.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14209" PostTypeId="2" ParentId="14024" CreationDate="2020-08-22T11:38:44.677" Score="2" Body="&lt;p&gt;The largest number of samples is at the dbSNP database. To view frequencies of a single rsID in a web browser, for example, rs11950646, use the following URL:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/rs11950646#frequency_tab&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/snp/rs11950646#frequency_tab&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The dbSNP has more samples to calculate the frequency than gnomad, at least for this rsID.&lt;/p&gt;&#xA;&lt;p&gt;If you need to automatically process multiple rsIDs, as you said, for the whole genotype file that you got from the companies like 23andMe, you can download a VCF with allele frequency data from &lt;a href=&quot;https://ftp.ncbi.nih.gov/snp/population_frequency/latest_release/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nih.gov/snp/population_frequency/latest_release/&lt;/a&gt; and write a script to automatically process the data and make the output in a format you want. If you only need frequency of some rsIDs within a single chromosome, you can download the whole database for that chromosome from &lt;a href=&quot;https://ftp.ncbi.nih.gov/snp/latest_release/JSON/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nih.gov/snp/latest_release/JSON/&lt;/a&gt;&#xA;and will find the data in the JSON files, each file for each chromosome (plus the mtDNA). Look for the &amp;quot;frequency&amp;quot; field.&lt;/p&gt;&#xA;&lt;p&gt;If you are not engaged to write scripts, and just wish to interactively browse various rsIDs in context of a frequency, you can upload your genotype file to selfdecode.com - it is a convenient, albeit a paid service.&lt;/p&gt;&#xA;" OwnerUserId="9402" LastActivityDate="2020-08-22T11:38:44.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14210" PostTypeId="1" AcceptedAnswerId="14211" CreationDate="2020-08-21T17:56:48.507" Score="3" ViewCount="785" Body="&lt;p&gt;The residue numbering in PDB (in my understanding) coincides with UniProtKB align. However, this might not be consistent with that in PDB fasta file. To complicate it further, UniProtKB align can be a few mutations away from the PDB sequence. Disordered/unidentified residues in structures also create gaps in numbering.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way/suggestion to create a matched residue numbering between sequence and structure? For example&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;12345678 (residue numbering)&#xA;ATPWQMSQ (sequence)&#xA;---W-MSQ (PDB residues) &amp;quot;-&amp;quot; for missing residue in structure&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9344" OwnerDisplayName="Simon" LastActivityDate="2020-08-22T14:01:37.303" Title="PDB residue numbering" Tags="|proteins|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14211" PostTypeId="2" ParentId="14210" CreationDate="2020-08-22T13:43:35.660" Score="2" Body="&lt;p&gt;Unfortunately, PDB numbering can be one of different schemes. The main three:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Uniprot numbering (canonical isoform)&lt;/li&gt;&#xA;&lt;li&gt;Construct numbering with or without tag removal&lt;/li&gt;&#xA;&lt;li&gt;Start from 1 for what has density&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, the are many many corner cases. In Proteopedia there is &lt;a href=&quot;https://proteopedia.org/wiki/index.php/Unusual_sequence_numbering&quot; rel=&quot;nofollow noreferrer&quot;&gt;a lovely article about weird PDB numbering&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The best way to find the details about what part of the sequence is covered is to use the &lt;a href=&quot;https://swissmodel.expasy.org/docs/smr_openapi&quot; rel=&quot;nofollow noreferrer&quot;&gt;Swissmodel API&lt;/a&gt;. It's actually better than either the PDBe and RCSB PDB APIs. Swissmodel is a homology modelling service that maintains close threaded models (&amp;gt;30% ID), but also has PDB data. Here is a Python3 method I use, (&lt;code&gt;provider='pdb'&lt;/code&gt;) is what is required here).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def get_data(uniprot: str, provider='swissmodel') -&amp;gt; List[dict]:&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        Gets data from Swissmodel. Convert each entry with convert_SM_entry&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;    assert provider in ('swissmodel', 'pdb'), 'Provider has to be swissmodel or pdb'&#xA;        url = f'https://swissmodel.expasy.org/repository/uniprot/{uniprot.strip()}.json?provider={provider.strip()}'&#xA;        data = requests.get(url).json()&#xA;        return data['result']['structures']  # [0] # from to coordinates description&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The returned dictionary contains a start and end of the uniprot &amp;lt;--&amp;gt; PDB. It does not however, say where the missing density residues are. But in &lt;em&gt;deposited&lt;/em&gt; structures the PDB numbering skips where these residues ought to be... So might not be an important problem.&lt;/p&gt;&#xA;&lt;p&gt;Another database of interest is &lt;a href=&quot;https://www.ebi.ac.uk/pdbe/docs/sifts/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SIFTS&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To make a multiple sequence alignment of a non-deposited PDB, you need to extract the sequence and do the alignment here is a snippet using &lt;a href=&quot;https://anaconda.org/schrodinger/pymol&quot; rel=&quot;nofollow noreferrer&quot;&gt;pymol module (from conda)&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pymol2&#xA;&#xA;pdbfile = '' #your pdb file name..&#xA;selector = 'chain A' # or whatever chain you want&#xA;with pymol2.PyMOL() as pymol:&#xA;    pymol.cmd.load(pdbfile)&#xA;    fasta = pymol.cmd.get_fastastr(selector)&#xA;&#xA;seq = ''.join(fasta.replace('?','').split('\n')[1:]) # remove header and non AA residues&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Biopython has a dozen sequence alignment tools. But the original one still works fine&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import pairwise2&#xA;from Bio.Seq import Seq&#xA;&#xA;alignment = pairwise2.align.globalxx(''.join(seq).replace('?',''), &amp;quot;ACQQG&amp;quot;)[0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;alignments&lt;/code&gt; is a tuple of the MSA sequences.&lt;/p&gt;&#xA;&lt;p&gt;If you want to do the latter without coding. Extract the sequence from the PDB using PyMOL the normal application and load the pdb and then &lt;code&gt;print cmd.get_fastastr('chain A')&lt;/code&gt; in the command line part of the GUI. Copy this. Go to &lt;a href=&quot;https://www.ebi.ac.uk/Tools/msa/muscle/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Muscle&lt;/a&gt; or other onliner aligner and align!&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-08-22T14:01:37.303" LastActivityDate="2020-08-22T14:01:37.303" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14212" PostTypeId="1" CreationDate="2020-08-22T16:30:57.760" Score="0" ViewCount="168" Body="&lt;p&gt;I use clustal() from library ape every now and then, but now it doesnt work...&lt;/p&gt;&#xA;&lt;p&gt;code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sylvia.clus = clustal(sylvia.seq)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; sh: /opt/X11/bin/clustalw2: Bad CPU type in executable cannot open&#xA;&amp;gt; file&#xA;&amp;gt; '/var/folders/46/vt_whvms133190kjmlx5gf4m0000gn/T//Rtmpa82i72/clustald5d11dadcdf.fas':&#xA;&amp;gt; No such file or directoryError in file(con, &amp;quot;rb&amp;quot;) : cannot open the&#xA;&amp;gt; connection&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there any workaround? (I try reinstalling ape library, but I got the same error)&lt;/p&gt;&#xA;&lt;p&gt;Screenshot from Rstudio:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/XiB88.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XiB88.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6829" LastEditorUserId="6829" LastEditDate="2020-08-23T11:39:54.963" LastActivityDate="2020-08-23T11:39:54.963" Title="workaround for error: Bad CPU type in executable c" Tags="|r|sequence-alignment|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14213" PostTypeId="1" CreationDate="2020-08-22T18:10:47.287" Score="1" ViewCount="102" Body="&lt;p&gt;I have a tab-separated file (the columns are rsID CHROM POS GENOTYPE) generated by Diganomics with Illumina GSAMD (v2.2) from male DNA, and there are a few hundred loci within X or Y chromosome that contain two alleles, even heterozygous, e.g.&lt;/p&gt;&#xA;&lt;p&gt;[cut]&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rs28463388  X   191998  TT&#xA;rs28669107  X   195014  GC&#xA;rs6655397   X   201935  GG&#xA;rs7890186   X   202458  TG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;[skipped]&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rs1016793029    Y   8619592 GA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;[skipped]&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rs527288580 Y   8771750 GA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;[skipped]&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rs917158084 Y   14871973    TG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;[…etc]&lt;/p&gt;&#xA;&lt;p&gt;There are 1270 such double alleles (7 in Y chromosome and 1163 in the X chromosome) from a total of 734162 number of lines, or 0.17%. What is remarkable is that these 1163 doubles within the X chromosome tend to be clustered, i.e. come consequently one after another mostly, but the 7 in the Y were sparse single lines.&lt;/p&gt;&#xA;" OwnerUserId="9402" LastEditorUserId="9402" LastEditDate="2020-08-24T23:08:22.700" LastActivityDate="2020-08-24T23:08:22.700" Title="Male DNA genotyped with Illumina GSAMD gives two alleles for certain loci within sex chromosomes" Tags="|genotyping|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14214" PostTypeId="2" ParentId="13976" CreationDate="2020-08-23T14:34:54.283" Score="1" Body="&lt;p&gt;You used Kallisto for alignment. I think Kallisto reports TPM values, are you using this value? DEseq2 uses count data, so I am not sure whether these two methods are compatible.&lt;/p&gt;&#xA;&lt;p&gt;Also, I agree with previous answers that your PCA actually looks OK. One possible way to improve is to choose top variable genes. For example, you can try top 3,000, 5,000, 7,000 genes and so on. The idea is that for the genes that do not show much variation between samples, including them in PCA may just introduce noise. You can also try to color samples in your PCA by some other variables, like batch, sequencing depth and so on to trouble-shooting.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-08-23T14:34:54.283" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14215" PostTypeId="2" ParentId="14184" CreationDate="2020-08-23T15:48:37.587" Score="0" Body="&lt;p&gt;Based on the &lt;a href=&quot;https://www.biorxiv.org/content/biorxiv/suppl/2016/12/19/092890.DC2/092890-1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;supplementary material&lt;/a&gt; as mentioned in a comment by Devon Ryan:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The second phase of DeepVariant encodes the reference and read support&#xA;for each candidate variant into an RGB image. The pseudo-code for this&#xA;component is shown below; it contains all of the key operations to&#xA;build the image, leaving out for clarity error handling, code to deal&#xA;with edge cases like when variants occur close to the start or end of&#xA;the chromosome, and the implementation of non-essential and/or obvious&#xA;functions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here is the main function&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;WIDTH = 221&#xA;HEIGHT = 100;&#xA;def create_pileup_images(candidate_variants):&#xA; for candidate in candidate_variants:&#xA; for biallelic_variant in split_into_biallelics(candidate):&#xA; start = biallelic_variant.start - (WIDTH-1) / 2&#xA; end = WIDTH - span_start&#xA; ref_bases = reference.get_bases(start, end)&#xA; image = Image(WIDTH, HEIGHT)&#xA; row_i = fill_reference_pixels(ref, image)&#xA; for read in reads.get_overlapping(start, end):&#xA; if row_i &amp;lt; HEIGHT and is_usable_read(read):&#xA; add_read(image, read, row_i)&#xA; row_i += 1&#xA; yield image&#xA;&#xA;def fill_reference_pixels(ref, image):&#xA; for row in range(5):&#xA; for col in range(WIDTH):&#xA; alpha = 0.4&#xA; ref_base = ref[col]&#xA; red = get_base_color(ref_base)&#xA; green = get_quality_color(60) # The reference is high quality&#xA; blue = get_strand_color(True) # The reference is on the positive strand&#xA; image[row, col] = make_pixel(red, green, blue, alpha)&#xA; return 5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="734" LastActivityDate="2020-08-23T15:48:37.587" CommentCount="0" CommunityOwnedDate="2020-08-23T15:48:37.587" ContentLicense="CC BY-SA 4.0" />
  <row Id="14216" PostTypeId="1" CreationDate="2020-08-23T19:19:32.337" Score="1" ViewCount="60" Body="&lt;p&gt;I am new to bioinformatics, I am reading ‘Analysis of Phylogenetics Second Edition and Evolution with R’ from Emmanuel Paradis.&lt;/p&gt;&#xA;&lt;p&gt;I can create phylogeny from DNA sequences, by first calculating the distances and then using (ie nj() function).&lt;/p&gt;&#xA;&lt;p&gt;My question is: if instead of DNA I have trait data (ie height and weight of species), how can I estimate a phylogeny? (is the same process valid? somehow calculate distances and pass that to nj()?)&lt;/p&gt;&#xA;" OwnerUserId="6829" LastActivityDate="2020-08-25T17:26:27.277" Title="how to estimate phylogeny without DNA sequences?" Tags="|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14217" PostTypeId="1" CreationDate="2020-08-23T20:35:41.090" Score="2" ViewCount="1673" Body="&lt;p&gt;&lt;strong&gt;Problem:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am trying to convert some codes written in R to Python and part of that conversion process is find classes equivalent to the GRanges and IRanges from the GenomicRanges R package in Python. &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/GenomicRanges/inst/doc/GenomicRangesIntroduction.html#granges-genomic-ranges&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/vignettes/GenomicRanges/inst/doc/GenomicRangesIntroduction.html#granges-genomic-ranges&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I couldn't find any equivalent library written in Python that does the same operations as the aforementioned R classes. There is however pyranges library in Python but it is not as flexible as GRanges and doesn't have IRanges implemented in it. &lt;a href=&quot;https://github.com/biocore-ntnu/pyranges&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/biocore-ntnu/pyranges&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For example in R I could add the following in the GRanges class:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gr = GRanges(&#xA;seq = Rle(DF&lt;span class=&quot;math-container&quot;&gt;$chr),&#xA;ranges = IRanges(DF$&lt;/span&gt;Start,DF&lt;span class=&quot;math-container&quot;&gt;$End),&#xA;ref = DF$&lt;/span&gt;ref,&#xA;alt = DF&lt;span class=&quot;math-container&quot;&gt;$alt,&#xA;sampleID=DF$&lt;/span&gt;sampleID,&#xA;seqlengths = chromSize)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, pyranges don't allow you to add arguments other than (chr, start, end, strand). However, in my case, I need to have the option to add sample_id, seqlengths and other arguments.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance.&#xA;Best&lt;/p&gt;&#xA;" OwnerUserId="9407" LastActivityDate="2021-02-28T19:55:44.923" Title="Python equivalent to R GRanges and IRanges" Tags="|r|python|bioconductor|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14218" PostTypeId="2" ParentId="14217" CreationDate="2020-08-24T09:11:55.160" Score="3" Body="&lt;p&gt;I am not sure if your question has anything to do with IRanges. If I get it correct, the limitation you have with pyranges now is adding extra columns to the meta data, like adding values to &lt;code&gt;@values&lt;/code&gt; slot in &lt;code&gt;granges&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You can do it using &lt;code&gt;insert&lt;/code&gt; or &lt;code&gt;setattr&lt;/code&gt; :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pyranges&#xA;import pandas as pd&#xA;import numpy as np&#xA;gr = pyranges.random(10)&#xA;&#xA;setattr(gr,'alt',np.random.choice(['A','T','G','C'],10))&#xA;&#xA;gr.insert(pd.DataFrame({'ref':np.random.choice(['A','T','G','C'],10)}))&#xA;&#xA;+--------------+-----------+-----------+--------------+------------+------------+&#xA;| Chromosome   | Start     | End       | Strand       | alt        | ref        |&#xA;| (category)   | (int32)   | (int32)   | (category)   | (object)   | (object)   |&#xA;|--------------+-----------+-----------+--------------+------------+------------|&#xA;| chr1         | 27431195  | 27431295  | -            | C          | G          |&#xA;| chr2         | 197045893 | 197045993 | -            | A          | G          |&#xA;| chr4         | 86316012  | 86316112  | -            | C          | A          |&#xA;| chr8         | 4560598   | 4560698   | -            | G          | G          |&#xA;| ...          | ...       | ...       | ...          | ...        | ...        |&#xA;| chr11        | 130395526 | 130395626 | -            | A          | A          |&#xA;| chr12        | 28557156  | 28557256  | +            | T          | G          |&#xA;| chr13        | 55519337  | 55519437  | -            | T          | C          |&#xA;| chr14        | 1807028   | 1807128   | +            | G          | A          |&#xA;+--------------+-----------+-----------+--------------+------------+------------+&#xA;Stranded PyRanges object has 10 rows and 6 columns from 10 chromosomes.&#xA;For printing, the PyRanges was sorted on Chromosome and Strand.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or directly adding them like this (thanks to @ChrisRands for pointing it out):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gr.alt = np.random.choice(['A','T','G','C'],10)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastEditorUserId="5077" LastEditDate="2020-08-24T09:58:48.420" LastActivityDate="2020-08-24T09:58:48.420" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="14221" PostTypeId="1" CreationDate="2020-08-24T22:52:31.427" Score="0" ViewCount="2423" Body="&lt;p&gt;I am working with data from &lt;a href=&quot;ftp://ngs.sanger.ac.uk/production/hgdp/hgdp_wgs.20190516/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the HGDP public files&lt;/a&gt;. These are massive VCFs from whole genome sequencing. To begin with, I converted these to BEDs using plink (1.9). The next problem is that these files use rsid names instead of chrpos names. I would like to change to chrpos. I am wondering whether one can simply edit the BIMs or whether I have to do use plink's &lt;code&gt;--update-name&lt;/code&gt; call. The problem with the latter is that it doesn't support duplicates. The HGDP files are full of duplicate variant names because many of the variants don't have rsid's and are just named &amp;quot;.&amp;quot;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$&amp;gt; plink --bfile hgdp_chr21 --update-name updateids --make-bed --out hgdp_chr21_chrpos&#xA;PLINK v1.90b6.17 64-bit (28 Apr 2020)          www.cog-genomics.org/plink/1.9/&#xA;(C) 2005-2020 Shaun Purcell, Christopher Chang   GNU General Public License v3&#xA;Logging to hgdp_chr21_chrpos.log.&#xA;Options in effect:&#xA;  --bfile hgdp_chr21&#xA;  --make-bed&#xA;  --out hgdp_chr21_chrpos&#xA;  --update-name updateids&#xA;&#xA;64259 MB RAM detected; reserving 32129 MB for main workspace.&#xA;1086522 variants loaded from .bim file.&#xA;929 people (0 males, 0 females, 929 ambiguous) loaded from .fam.&#xA;Ambiguous sex IDs written to hgdp_chr21_chrpos.nosex .&#xA;Error: Duplicate ID '.'.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thus, one gets an error when trying &lt;code&gt;--update-name&lt;/code&gt; and I am wondering if I can simply edit the BIM files to be chrpos instead of rsid. I can't find an answer to this anywhere. It seems that if the BED files don't contain a reference to the variant IDs, but only to the row in the BIM file, then this approach should be possible.&lt;/p&gt;&#xA;&lt;p&gt;Here's the current BIM file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;user@desktop:/media/luks8tb1/data/genomics/HGDP$ head hgdp_hg38.bim&#xA;1   rs1331781659    0   10153   G   A&#xA;1   rs1179689498    0   10163   C   T&#xA;1   rs201694901 0   10180   C   T&#xA;1   rs199706086 0   10250   C   A&#xA;1   rs111200574 0   10257   C   A&#xA;1   rs145427775 0   10291   T   C&#xA;1   rs868413313 0   10297   T   C&#xA;1   rs112750067 0   10327   C   T&#xA;1   rs1035171912    0   10330   A   C&#xA;1   rs868804735 0   10333   T   C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As can be seen, very few lines have the &amp;quot;.&amp;quot; variant name. In fact, in the first 100 lines there are 0.&lt;/p&gt;&#xA;" OwnerUserId="9417" LastEditorUserId="9417" LastEditDate="2020-08-26T18:11:33.057" LastActivityDate="2023-01-11T14:32:35.557" Title="plink: --update-name vs. editing the BIM" Tags="|bed|plink|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14223" PostTypeId="2" ParentId="14221" CreationDate="2020-08-25T07:22:53.223" Score="2" Body="&lt;p&gt;I would reccomend using &lt;code&gt;bcftools&lt;/code&gt; on the original vcf files before you convert them to plink, to fill in missing IDs using the command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools annotate --set-id +'%CHROM\_%POS\_%REF\_%FIRST_ALT' file.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This means you won't have any duplicate ID names and plink should be happy about that. It will preserve the positional information in the .bim file as well.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2020-08-25T07:22:53.223" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14224" PostTypeId="1" AcceptedAnswerId="14237" CreationDate="2020-08-25T08:17:37.123" Score="1" ViewCount="116" Body="&lt;p&gt;I am working on four data sets of yeast genes and one data set of mouse gene. I need the list of human orthologs of those genes. There are a lot of databases available online which is in use for this purpose&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://questfororthologs.org/orthology_databases&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://questfororthologs.org/orthology_databases&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I don't know which one to choose and why. Can anyone tell me on what basis should I choose a database over any other database? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="7168" LastActivityDate="2020-08-26T16:41:27.953" Title="How to know which ortholog database is the best for my work?" Tags="|orthologues|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14225" PostTypeId="1" AcceptedAnswerId="14230" CreationDate="2020-08-25T12:17:44.977" Score="0" ViewCount="105" Body="&lt;p&gt;I have a list of HGNC gene symbols, I am looking to get the gene length of each gene. Although I also describe these genes with lots of UCSC datasets as features, so I am wondering if there is a dataset in UCSC I can use to also get gene length from?&lt;/p&gt;&#xA;&lt;p&gt;I've been looking in the data that is downloadable from UCSC table browser (I've been aiming to find start and ends for each gene to subtract to get gene length) but there are a lot of files and I'm not sure which dataset to take from which will also match to my HGNC gene symbols.&lt;/p&gt;&#xA;" OwnerUserId="4242" LastEditorUserId="4242" LastEditDate="2020-08-25T16:42:15.863" LastActivityDate="2020-08-25T20:13:17.977" Title="How to annotate gene length to a list of gene symbols using UCSC data?" Tags="|gene|genomics|ucsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14226" PostTypeId="2" ParentId="14221" CreationDate="2020-08-25T14:39:17.133" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/379361/&quot; rel=&quot;nofollow noreferrer&quot;&gt;I found this 2019 question on biostars&lt;/a&gt; in which user zx8754 mentions that plink2 has a command for this purpose &lt;code&gt;--set-all-var-ids&lt;/code&gt;, &lt;a href=&quot;https://www.cog-genomics.org/plink/2.0/data#set_all_var_ids&quot; rel=&quot;nofollow noreferrer&quot;&gt;from the plink2 docs&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Whole-exome and whole-genome sequencing results frequently contain&#xA;variants which have not been assigned standard IDs. If you don't want&#xA;to throw out all of that data, you'll usually want to assign them&#xA;chromosome-and-position-based IDs.&lt;/p&gt;&#xA;&lt;p&gt;--set-missing-var-ids (which just replaces missing IDs) and --set-all-var-ids (which overwrites everything) provide one way to do this. The parameter taken by these flags is a special template string,&#xA;with a '@' where the chromosome code should go, and a '#' where the&#xA;base-pair position belongs. (Exactly one @ and one # must be present.)&#xA;For example, given a .pvar file starting with&lt;/p&gt;&#xA;&lt;p&gt;#CHROM  POS     ID  REF  ALT chr1    10583   .   G    A chr1    886817  .   T    C chr1    886817  .   C    CATTTT&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;--set-missing-var-ids @:#[b37]&amp;quot; would name the first variant&#xA;'chr1:10583[b37]', the second variant 'chr1:886817[b37]'... and the&#xA;third variant also gets the name 'chr1:886817[b37]'.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Thus, in my case, I used:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;user@desktop:/media/luks8tb1/data/genomics/HGDP$ plink2 --bfile hgdp_hg38 --set-all-var-ids @:# --make-bed --out hgdp_hg38_chrpos&#xA;PLINK v2.00a3LM AVX2 Intel (27 Jul 2020)       www.cog-genomics.org/plink/2.0/&#xA;(C) 2005-2020 Shaun Purcell, Christopher Chang   GNU General Public License v3&#xA;Logging to hgdp_hg38_chrpos.log.&#xA;Options in effect:&#xA;  --bfile hgdp_hg38&#xA;  --make-bed&#xA;  --out hgdp_hg38_chrpos&#xA;  --set-all-var-ids @:#&#xA;&#xA;Start time: Tue Aug 25 19:16:10 2020&#xA;64259 MiB RAM detected; reserving 32129 MiB for main workspace.&#xA;Using up to 8 compute threads.&#xA;929 samples (0 females, 0 males, 929 ambiguous; 929 founders) loaded from&#xA;hgdp_hg38.fam.&#xA;56475980 variants loaded from hgdp_hg38.bim.&#xA;Note: No phenotype data present.&#xA;Writing hgdp_hg38_chrpos.fam ... done.&#xA;Writing hgdp_hg38_chrpos.bim ... done.&#xA;Writing hgdp_hg38_chrpos.bed ... done.&#xA;End time: Tue Aug 25 19:17:41 2020&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results, BIM:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;emil@test-desktop:/media/luks8tb1/data/genomics/HGDP$ head hgdp_hg38_chrpos.bim&#xA;1   1:10153 0   10153   G   A&#xA;1   1:10163 0   10163   C   T&#xA;1   1:10180 0   10180   C   T&#xA;1   1:10250 0   10250   C   A&#xA;1   1:10257 0   10257   C   A&#xA;1   1:10291 0   10291   T   C&#xA;1   1:10297 0   10297   T   C&#xA;1   1:10327 0   10327   C   T&#xA;1   1:10330 0   10330   A   C&#xA;1   1:10333 0   10333   T   C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9417" LastEditorUserId="9417" LastEditDate="2020-08-26T18:13:28.527" LastActivityDate="2020-08-26T18:13:28.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14227" PostTypeId="2" ParentId="14216" CreationDate="2020-08-25T17:26:27.277" Score="2" Body="&lt;p&gt;Sequence data is basically always better and at this point in history it is de rigueur for any serious study, but originally phylogenies were all based on (ideally discrete) phenotypic characters, so there's no reason why it can't work in principle. As @Pallie mentions in a comment, the clustering methods in e.g. NJ or UPGMA will work on any set of distance measures, and they were in fact &lt;a href=&quot;https://en.wikipedia.org/wiki/Phenetics&quot; rel=&quot;nofollow noreferrer&quot;&gt;originally developed on phenotypic characters rather than sequence data&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I'd recommend taking a look at chapter 24 of Joe Felsenstein's book on phylogenies, which is digitized &lt;a href=&quot;http://zmmu.msu.ru/files/%D0%91%D0%B8%D0%B1%D0%BB%D0%B8%D0%BE%D1%82%D0%B5%D0%BA%D0%B0%20%D0%9F%D0%B0%D0%B2%D0%BB%D0%B8%D0%BD%D0%BE%D0%B2%D0%B0/felsenstein-2004_inferr_phylog.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (yes the link looks sketchy, sorry). There is a section on p. 425 &amp;quot;Inferring phylogenies and correlations&amp;quot;, which discusses how you can in principle use quantitative characters to infer phylogenies. The theory and math for why this is supposed to work is earlier in the chapter.&lt;/p&gt;&#xA;&lt;p&gt;In many cases people prefer to code the characters as discrete because it makes it easier to work with. But there are also methods that work with the raw quantitative characters.&lt;/p&gt;&#xA;&lt;p&gt;In R, you should be able to do something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# some simulated data, species are rows and characters are columns&#xA;&amp;gt; data&#xA;      char1 char2 char3&#xA;spec1     0     3     6&#xA;spec2     1     4     7&#xA;spec3     2     5     8&#xA;&#xA;&amp;gt; library(ape)&#xA;&amp;gt; tr = nj(dist(data))&#xA;&amp;gt; plot(tr)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UJSGy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UJSGy.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-08-25T17:26:27.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14228" PostTypeId="1" AcceptedAnswerId="14247" CreationDate="2020-08-25T18:11:11.320" Score="2" ViewCount="1388" Body="&lt;p&gt;I am having strange issues lifting this dataset to hg37 from hg38. I am using the HGDP WGS data from &lt;a href=&quot;ftp://ngs.sanger.ac.uk/production/hgdp/hgdp_wgs.20190516/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ngs.sanger.ac.uk/production/hgdp/hgdp_wgs.20190516/&lt;/a&gt;. The data set split by chromosome, so I first converted them to BED then merged them to a single file using plink1.9. However, when trying to lift to hg37/19 using either &lt;a href=&quot;http://hgdownload.cse.ucsc.edu/admin/exe/linux.x86_64/&quot; rel=&quot;nofollow noreferrer&quot;&gt;liftOver&lt;/a&gt;  or &lt;a href=&quot;https://crossmap.readthedocs.io/en/latest/#&quot; rel=&quot;nofollow noreferrer&quot;&gt;CrossMap v0.5.1&lt;/a&gt; I get strange errors. I tried both the chain from &lt;a href=&quot;ftp://ftp.ensembl.org/pub/assembly_mapping/homo_sapiens/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ensemble.org&lt;/a&gt; and &lt;a href=&quot;http://hgdownload.cse.ucsc.edu/goldenPath/hg38/liftOver/&quot; rel=&quot;nofollow noreferrer&quot;&gt;from UCSC&lt;/a&gt;, I tried them both compressed and uncompressed. I am starting to think I need to try the conversion on the VCF files but these are so large I would prefer not to duplicate them unnecessarily.&lt;/p&gt;&#xA;&lt;h1&gt;Data&lt;/h1&gt;&#xA;&lt;p&gt;The data looks fine. If we look at the BIM:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;user@desktop:/media/luks8tb1/data/genomics/HGDP&lt;span class=&quot;math-container&quot;&gt;$ head hgdp_hg38_chrpos.bim&#xA;1   1:10153 0   10153   G   A&#xA;1   1:10163 0   10163   C   T&#xA;1   1:10180 0   10180   C   T&#xA;1   1:10250 0   10250   C   A&#xA;1   1:10257 0   10257   C   A&#xA;1   1:10291 0   10291   T   C&#xA;1   1:10297 0   10297   T   C&#xA;1   1:10327 0   10327   C   T&#xA;1   1:10330 0   10330   A   C&#xA;1   1:10333 0   10333   T   C&#xA;user@desktop:/media/luks8tb1/data/genomics/HGDP$&lt;/span&gt; tail hgdp_hg38_chrpos.bim&#xA;Y   Y:56887228  0   56887228    C   T&#xA;Y   Y:56887285  0   56887285    C   A&#xA;Y   Y:56887316  0   56887316    A   G&#xA;Y   Y:56887463  0   56887463    A   C&#xA;Y   Y:56887491  0   56887491    A   C&#xA;Y   Y:56887503  0   56887503    T   G&#xA;Y   Y:56887583  0   56887583    G   A&#xA;Y   Y:56887631  0   56887631    T   C&#xA;Y   Y:56887837  0   56887837    A   G&#xA;Y   Y:56887844  0   56887844    C   T&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The data are in chrpos variant names here but I had the same result using the original rsid.&lt;/p&gt;&#xA;&lt;h1&gt;Chains&lt;/h1&gt;&#xA;&lt;p&gt;Chains look fine, and I am able to read them into R without problems:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;user@desktop:/media/luks8tb1/data/genomics/HGDP$ head GRCh38_to_GRCh37.chain&#xA;chain 1 1 248956422 + 10000 297968 1 249250621 + 10000 267719 2&#xA;167417 80249 50000&#xA;40302&#xA;&#xA;chain 1 1 248956422 + 347968 501617 1 249250621 - 248779253 248932902 3&#xA;153649&#xA;&#xA;chain 1 1 248956422 + 585988 12949384 1 249250621 + 521368 13009210 4&#xA;1044707 1 0&#xA;3709 3 0&#xA;user@desktop:/media/luks8tb1/data/genomics/HGDP$ tail GRCh38_to_GRCh37.chain&#xA;&#xA;chain 1 chrY 57227415 + 56821509 57217415 Y 59373566 + 58967656 59363566 11183&#xA;85168 0 1&#xA;78580 0 1&#xA;63496 0 1&#xA;3158 1 0&#xA;30382 0 1&#xA;54147 0 1&#xA;80974&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h1&gt;CrossMap&lt;/h1&gt;&#xA;&lt;pre&gt;&lt;code&gt;user@desktop:/media/luks8tb1/data/genomics/HGDP$ CrossMap.py bed GRCh38_to_GRCh37.chain hgdp_hg38_chrpos.bed hgdp_hg37_chrpos.bed&#xA;@ 2020-08-25 20:04:26: Read the chain file:  GRCh38_to_GRCh37.chain&#xA;Traceback (most recent call last):&#xA;  File &amp;quot;/home/user/.local/bin/CrossMap.py&amp;quot;, line 166, in &amp;lt;module&amp;gt;&#xA;    crossmap_bed_file(mapTree, in_file, out_file)&#xA;  File &amp;quot;/home/user/.local/lib/python3.6/site-packages/cmmodule/mapbed.py&amp;quot;, line 32, in crossmap_bed_file&#xA;    for line in ireader.reader(inbed):&#xA;  File &amp;quot;/home/user/.local/lib/python3.6/site-packages/cmmodule/ireader.py&amp;quot;, line 27, in reader&#xA;    yield l.decode('utf8').strip().replace(&amp;quot;\r&amp;quot;, &amp;quot;&amp;quot;)&#xA;UnicodeDecodeError: 'utf-8' codec can't decode byte 0xdd in position 20: invalid continuation byte&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h1&gt;liftOver&lt;/h1&gt;&#xA;&lt;pre&gt;&lt;code&gt;user@desktop:/media/luks8tb1/data/genomics/HGDP$ liftOver hgdp_hg38.bed hg38ToHg19.over.chain hgdp_hg37.bed hgdp_hg37_errors.bed&#xA;Reading liftover chains&#xA;Mapping coordinates&#xA;Data format error: expecting at least 3 fields in BED file (hgdp_hg38.bed)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h1&gt;Trying the PED file with liftOver&lt;/h1&gt;&#xA;&lt;p&gt;I converted BED file to PED with plink1.9:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plink --bfile hgdp_hg38 --recode --out hgdp_hg38&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;user@desktop:/media/luks8tb1/data/genomics/HGDP$ liftOver hgdp_hg38.ped hg38ToHg19.over.chain hgdp_hg37.ped hgdp_hg37_errors.ped&#xA;Reading liftover chains&#xA;Mapping coordinates&#xA;invalid unsigned integer: &amp;quot;HGDP00001&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.google.com/search?q=%22liftover%22%20%22invalid%20unsigned%20integer%22&quot; rel=&quot;nofollow noreferrer&quot;&gt;There's a few question on Google about this, but nothing that looks useful.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;CrossMap does not support PED.&lt;/p&gt;&#xA;" OwnerUserId="9417" LastEditorUserId="9417" LastEditDate="2020-08-26T19:35:29.073" LastActivityDate="2021-07-25T03:26:36.933" Title="Lifting from hg38 to hg37" Tags="|plink|liftover|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14230" PostTypeId="2" ParentId="14225" CreationDate="2020-08-25T20:13:17.977" Score="2" Body="&lt;p&gt;You could do a few command-line operations to answer this question. This assumes the use of &lt;code&gt;hg38&lt;/code&gt; assembly.&lt;/p&gt;&#xA;&lt;p&gt;First, get a list of genes from GENCODE:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ wget -qO- ftp://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_28/gencode.v28.annotation.gff3.gz \&#xA;    | gunzip --stdout - \&#xA;    | awk '$&lt;/span&gt;3 == &amp;quot;gene&amp;quot;' - \&#xA;    | convert2bed -i gff - \&#xA;    &amp;gt; genes.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then use &lt;code&gt;grep&lt;/code&gt; to filter those genes with your list of HGNC symbols:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -wFf hgnc_symbols.txt genes.bed &amp;gt; filtered_genes.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can run this through &lt;code&gt;awk&lt;/code&gt; to get lengths:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk -vFS=&quot;\t&quot; '{ print $3-$&lt;/span&gt;2 }' filtered_genes.bed &amp;gt; filtered_gene_lengths.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="776" LastActivityDate="2020-08-25T20:13:17.977" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14231" PostTypeId="2" ParentId="11235" CreationDate="2020-08-25T20:35:40.233" Score="2" Body="&lt;p&gt;The author writes in the quote that &lt;span class=&quot;math-container&quot;&gt;$O$&lt;/span&gt; is to be taken for the whole set of positions &lt;span class=&quot;math-container&quot;&gt;$O_{i}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In the extracts that you provide there is no reason to believe that it is only for a sequence of length 1. The author writes &amp;quot;the observed sequence &lt;span class=&quot;math-container&quot;&gt;$O = O_1, O_2...$&lt;/span&gt;&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Thus as you say &lt;span class=&quot;math-container&quot;&gt;$Pr(sequence) = Pr(O) = Pr(O_1) * Pr(O_2)...$&lt;/span&gt; according to my understanding of the quotes.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-08-25T20:35:40.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14232" PostTypeId="2" ParentId="14228" CreationDate="2020-08-25T21:45:10.913" Score="0" Body="&lt;p&gt;Not sure about crossMap but liftOver requires the BED format where column 1 is the chromosome, column 2 the 0-based start and column 3 the end coordinate. Additional columns are optional and can be used to store the other columns you have. Simply convert BIM to BED and run liftOver.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-08-25T21:45:10.913" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14233" PostTypeId="2" ParentId="14184" CreationDate="2020-08-26T00:03:53.703" Score="1" Body="&lt;p&gt;Actually the paper has made it clear how they did it. You just have to read the supplementary materials closer.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In their figure from the paper: A is Red, C is Green, G is blue, and T&#xA;is Yellow (G+R), but this is still unclear how they the 3xNxN image.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In RGB, each dimension is an NxN image. Since you have three dimensions, so it's 3xNxN. The red dimension was used to encode the nucleotide bases. The green dimension was used to encode quality scores. Finally, the blue dimension was used to encode the strand information.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Obviously, DNA is a string over an alphabet with the characters: {A,&#xA;T, C, G}.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is easy. Lot's of ways. You could do a one-hot encoding or what DeepVariant used:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def get_base_color(base):&#xA;    base_to_color = {'A': 250, 'G': 180, 'T': 100, 'C': 30}&#xA;    return base_to_color.get(base, 0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="174" LastActivityDate="2020-08-26T00:03:53.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14234" PostTypeId="2" ParentId="14083" CreationDate="2020-08-26T00:11:13.973" Score="-1" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/BrooksLabUCSC/flair&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/BrooksLabUCSC/flair&lt;/a&gt; was excellent. Highly recommended. One could also ask for TPM.&lt;/p&gt;&#xA;" OwnerUserId="174" LastEditorUserId="174" LastEditDate="2020-08-26T04:59:09.013" LastActivityDate="2020-08-26T04:59:09.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14237" PostTypeId="2" ParentId="14224" CreationDate="2020-08-26T14:29:45.273" Score="0" Body="&lt;p&gt;&lt;strong&gt;To recover orthologous sequence for model species&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;You should be able to find your orthologs using &lt;a href=&quot;https://m.ensembl.org/info/data/biomart/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioMart&lt;/a&gt; and your gene IDs.&lt;/p&gt;&#xA;&lt;p&gt;There are numerous way to do it, directly on the ensembl website or through API. You can start from these links (quite old though) :&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.ensembl.info/2009/01/21/how-to-get-all-the-orthologous-genes-between-two-species/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ensembl.info/2009/01/21/how-to-get-all-the-orthologous-genes-between-two-species/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.youtube.com/watch?v=EwfOQsr2O-M&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=EwfOQsr2O-M&lt;/a&gt; #Get orthologs via BioMart&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.biostars.org/p/12245/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/12245/&lt;/a&gt; #Finding Human/Yeast Orthologs Using Embl?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Which database should I use&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is broad and depends on the nature of your work. In your case, working with &amp;quot;major model species&amp;quot;, &lt;strong&gt;DIOPT&lt;/strong&gt; (the first line of the table in Quest For Orthology) has the very feature you need on its front page for going from mouse to human. More eukaryotic/viral oriented database will be used for taxonomic identification, annotation inference, etc.&lt;/p&gt;&#xA;&lt;p&gt;I hope it helps.&lt;/p&gt;&#xA;" OwnerUserId="6273" LastEditorUserId="6273" LastEditDate="2020-08-26T16:41:27.953" LastActivityDate="2020-08-26T16:41:27.953" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14238" PostTypeId="1" AcceptedAnswerId="14239" CreationDate="2020-08-26T17:46:44.517" Score="2" ViewCount="67" Body="&lt;p&gt;&lt;strong&gt;Disclaimer&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;I'm a physicist and I'm fairly new to Bioinformatics therefore somethings below may not make sense.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;My purpose&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;I would like to simulate genetic evolution of bacterial populations, implementing interactions among different bacterial strains on a graph.&lt;/p&gt;&#xA;&lt;p&gt;My starting point is to generate a graph with a certain topology.&lt;/p&gt;&#xA;&lt;p&gt;Each vertex &lt;span class=&quot;math-container&quot;&gt;$V_i$&lt;/span&gt; should contain a population of &lt;span class=&quot;math-container&quot;&gt;$N_i$&lt;/span&gt; individuals for which I choose a certain allelic profile from an &lt;a href=&quot;https://pubmlst.org/databases/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MLST database&lt;/a&gt;. Population numerosity and allelic profile is assigned by random choice.&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\\\\$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;At starting time each vertex is populated with a &amp;quot;pure&amp;quot; strain of &lt;span class=&quot;math-container&quot;&gt;$N_i$&lt;/span&gt; clones, identified with a randomly choosen &lt;a href=&quot;https://pubmlst.org/bigsdb?db=pubmlst_blicheniformis_seqdef&amp;amp;page=downloadProfiles&amp;amp;scheme_id=14&quot; rel=&quot;nofollow noreferrer&quot;&gt;allelic profile&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Dynamics assumptions&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I will assume that:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Topology does not change with time&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Vertex dynamic flows as a &lt;em&gt;Moran process&lt;/em&gt;:&#xA;At time &lt;span class=&quot;math-container&quot;&gt;$t$&lt;/span&gt; , one individual is chosen randomly to reproduce and one&#xA;individual is chosen to die. The same individual can be chosen to reproduce and&#xA;then die. Thus, an individual has either zero, one, or two descendants. Zero and&#xA;two with equal probability &lt;span class=&quot;math-container&quot;&gt;$p_0 = p_2 = (N_i -1 )/N_i^2$&lt;/span&gt; , and one with probability &lt;span class=&quot;math-container&quot;&gt;$p_1 = 1 − 2 p_2$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Migration from one Vertex to one another is possible for nearest neighborhood with some fixed probability depending on choosen topology. I will assume that when migration occurs &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; new individuals enters into &lt;span class=&quot;math-container&quot;&gt;$V_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$O$&lt;/span&gt; individuals exits from &lt;span class=&quot;math-container&quot;&gt;$V_i$&lt;/span&gt;, with &lt;span class=&quot;math-container&quot;&gt;$I$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$O$&lt;/span&gt; depending on origin vertex numerosity. Migrants carry allelic frequencies according to population they came from. When migration occurs a new population is established: allelic profile frequencies changes according to that.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Migration and &lt;em&gt;Moran process&lt;/em&gt; evolution does not happen simultaneously; This allows me to guarantee that each population performs at least one evolution.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;(1) Is there something fundamental that I'm missing?&lt;/p&gt;&#xA;&lt;p&gt;(2) Moran model ensures that the population size remains constant (apart from migration effects obviously). Is this appropriate in order to simulate dynamic in stationary growing phase or should I introduce some demographic effects?&lt;/p&gt;&#xA;&lt;p&gt;(3) Which could be a proper &amp;quot;timing&amp;quot; for migration and evolution steps?&lt;/p&gt;&#xA;" OwnerUserId="9414" LastEditorUserId="3848" LastEditDate="2023-07-19T21:34:34.083" LastActivityDate="2023-07-19T21:34:34.083" Title="Populations genetics and dynamics of bacteria on a Graph" Tags="|phylogenetics|phylogeny|networks|bacteria|allele-frequency|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14239" PostTypeId="2" ParentId="14238" CreationDate="2020-08-26T19:13:04.240" Score="3" Body="&lt;p&gt;I assume you are simulating a null distribution. Are you investigation recombination??&lt;/p&gt;&#xA;&lt;p&gt;My main advice is to use population genetic terminology rather than geomometry to describe your simulation (e.g. vertex is a migration event between allopatric populations). Migration is investigated between populations investigated via &lt;span class=&quot;math-container&quot;&gt;$F_{ST}$&lt;/span&gt;. You appear to be assuming clonality, and population movement appear to follow a 'stepping stone model' (I might have misunderstand the question).&lt;/p&gt;&#xA;&lt;p&gt;Mutation rate ... I'll have a guess at &lt;span class=&quot;math-container&quot;&gt;$10^5$&lt;/span&gt; per site per year. Migration rate depends on the bacterial species being modeled, eg. MRSA spread in a very similiar fashion to that described in your model (along a moterway), in contrast soil bacteria don't adhere to allopatry at all (when I looked at them).&lt;/p&gt;&#xA;&lt;p&gt;Fixing one parameter (population size) whilst investigation a second (migration) would be normal to assess the parameter space initially. Bacterial populations do undergo selective sweeps and using an appropriate growth model I would look at a standard population growth model (e.g. the one Bob May investigated in chaotic interactions within a differential equation). There are several population growth models and fixing migration whilst assessing the model would seem sensible. The parameter space in the combined model of migration, space and population size becomes complex.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Okay I get your model now. Its fine, the thing I'd be cautious about is the population density of the bacterial host because this would skew migration across the nodes of your graph. A quick literature review will demonstrate the importance,  double paeroto log normal distribution and its cousin (the name escapes me .... ) are sort of key words that might flag up. Bacteria are not my active thing so I'm rusty.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-08-27T21:14:38.377" LastActivityDate="2020-08-27T21:14:38.377" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14240" PostTypeId="1" AcceptedAnswerId="14260" CreationDate="2020-08-26T19:51:49.587" Score="4" ViewCount="312" Body="&lt;p&gt;I have an indexed BAM file containing long-read sequencing data and I'd like to split the reads contained within into those with a known deletion and those without the deletion (I have the deletion coordinates available to me) when mapped against the hg38 genome. Ideally I'd like to end up with two BAM files (one with reads containing the deletion and one without) for downstream analysis. I've searched a bit online and haven't really found any tools that do exactly that, but this seems like a common enough task that there has to be something.&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance for any help/guidance!&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In addition to @gringer's great answer, I decided to write a small python script myself that solves my issue using pysam, the fruits of which can be found &lt;a href=&quot;https://github.com/brookshenry3/bam_splitter&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. If anyone has any feedback or suggestions on it I'll happily take them, as I'm still very new to this. It's undergone very little testing, so user be warned.&lt;/p&gt;&#xA;" OwnerUserId="9424" LastEditorUserId="9424" LastEditDate="2020-09-02T20:18:18.037" LastActivityDate="2020-09-02T20:18:18.037" Title="Sort reads in BAM file based on presence of specific deletion?" Tags="|sam|long-reads|indel|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14241" PostTypeId="2" ParentId="14198" CreationDate="2020-08-26T20:40:27.850" Score="6" Body="&lt;p&gt;I was quite curious when I saw that as well. I've spent more time than I'd care to admit trying the sort of things you have. I've gotten it decoded now, but I can't really claim any sort of victory, as the problem was pre-solved.&lt;/p&gt;&#xA;&lt;p&gt;After struggling through some of the same experiments you did, I decided to take a closer look at their explainer video here: &lt;a href=&quot;https://youtu.be/DMYgjOHgHxc&quot; rel=&quot;noreferrer&quot;&gt;https://youtu.be/DMYgjOHgHxc&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;First I tried to decipher the blackboard sketch around 1:30, but it was quite vague. A closer investigation into the speaker, however, yielded some luck. Googling Dr. Reinhard Heckel brought me to his website here: &lt;a href=&quot;http://www.reinhardheckel.com/&quot; rel=&quot;noreferrer&quot;&gt;http://www.reinhardheckel.com/&lt;/a&gt; which shows his &lt;a href=&quot;https://sci-hub.tw/10.1038/s41596-019-0244-5&quot; rel=&quot;noreferrer&quot;&gt;most recent publication&lt;/a&gt; – a paper on encoding digital data in DNA.&lt;/p&gt;&#xA;&lt;p&gt;The encoding is relatively complex (something nearly impossible to stumble upon accidentally, I'd reckon) but fascinating. The fragments are indexed and have two, layered error-correcting codes. Perhaps making our job harder (but serving the practical purpose of minimizing homopolymers and unwanted annealing) is the fact that the data is XORed with a pseudo-random noise that shuffles the data. The paper obviously has all of the details.&lt;/p&gt;&#xA;&lt;p&gt;From my skim, however, I stumbled across this Github repo linked in the paper: &lt;a href=&quot;https://github.com/reinhardh/dna_rs_coding&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/reinhardh/dna_rs_coding&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As it happens, the README was just updated in this last week, describing how to decode the episode from the file provided. If you have Docker, you just need to copy-paste some commands.&lt;/p&gt;&#xA;&lt;p&gt;The product is indeed the full and final episode – coming in at a dainty 63.1MB. How did they get 40 minutes of video that compressed? Well, in short, 720x360 at 24 fps. It looks a little awful and is in German only, but is certainly a cool little easter egg. If nothing else, I have a cool paper to read.&lt;/p&gt;&#xA;" OwnerUserId="9425" LastActivityDate="2020-08-26T20:40:27.850" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14242" PostTypeId="2" ParentId="4024" CreationDate="2020-08-26T21:45:26.303" Score="0" Body="&lt;p&gt;Apart from supporting only Windows-style and Linux-style line endings and a restricted character set as mentioned above, the sample sheet &lt;a href=&quot;https://www.illumina.com/content/dam/illumina-marketing/documents/products/technotes/sequencing-sheet-format-specifications-technical-note-970-2017-004.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;has to be in the correct encoding which is UTF-8&lt;/a&gt; (or its subset US-ASCII).&#xA;Many command lines support these &lt;a href=&quot;https://stackoverflow.com/questions/805418/how-to-find-encoding-of-a-file-via-script-on-linux&quot;&gt;lookup&lt;/a&gt; and &lt;a href=&quot;https://stackoverflow.com/questions/64860/best-way-to-convert-text-files-between-character-sets&quot;&gt;conversion&lt;/a&gt; commands:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;# lookup&#xA;file -i my_sample_sheet.csv&#xA;&#xA;# conversion&#xA;iconv -f ISO-8859-15 -t UTF-8 in.csv &amp;gt; out.csv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3575" LastActivityDate="2020-08-26T21:45:26.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14243" PostTypeId="2" ParentId="14240" CreationDate="2020-08-27T00:43:10.707" Score="2" Body="&lt;p&gt;The CIGAR string should tell you if a read has a deletion in it.  So filter the bam files based on that.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-08-27T00:43:10.707" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14244" PostTypeId="1" CreationDate="2020-08-27T03:55:06.450" Score="1" ViewCount="38" Body="&lt;p&gt;I ran HiSat2, MarkDuplicate, removed reads with the lower quality score than 40 and finally only kept properly paired reads.&lt;/p&gt;&#xA;&lt;p&gt;After the BAM filtering steps, I used the Scallop results with TransDecoder. Next, I used bamCoverage to create RNA-Seq profile. I noticed that not always the first isoform is the best one compared to the RNA-Seq profile as could be seen below a few examples:&lt;/p&gt;&#xA;&lt;p&gt;1.&#xA;&lt;img src=&quot;https://user-images.githubusercontent.com/588297/90951101-e1a4ec80-e49a-11ea-8d9d-beea34a26d5e.png&quot; alt=&quot;Screen Shot 2020-08-22 at 4 35 26 PM&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;2.&#xA;&lt;img src=&quot;https://user-images.githubusercontent.com/588297/90951106-e9fd2780-e49a-11ea-85ce-6082a39bd110.png&quot; alt=&quot;Screen Shot 2020-08-22 at 4 38 45 PM&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;3.&#xA;&lt;img src=&quot;https://user-images.githubusercontent.com/588297/90951109-f41f2600-e49a-11ea-86c4-b61c63a2466d.png&quot; alt=&quot;Screen Shot 2020-08-22 at 4 51 37 PM&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;4.&#xA;&lt;img src=&quot;https://user-images.githubusercontent.com/588297/90951113-000ae800-e49b-11ea-85a4-077c77a9d2e3.png&quot; alt=&quot;Screen Shot 2020-08-22 at 4 58 42 PM&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;5.&#xA;&lt;img src=&quot;https://user-images.githubusercontent.com/588297/90951118-07ca8c80-e49b-11ea-80ac-421c3f85ca98.png&quot; alt=&quot;Screen Shot 2020-08-22 at 5 04 34 PM&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;By any chance, is there a way to keep only the best isoform which is in red box?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance,&lt;/p&gt;&#xA;" OwnerUserId="2477" LastActivityDate="2020-08-27T03:55:06.450" Title="BAM file filteing to remain best isoform" Tags="|sequence-alignment|sam|sequence-annotation|rna-splicing|isoform|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14247" PostTypeId="2" ParentId="14228" CreationDate="2020-08-27T13:39:59.777" Score="2" Body="&lt;p&gt;BED format can refer to the &amp;quot;.txt&amp;quot; file format that stores chromosomes and positions of genomic features as well as some other things like name. &lt;code&gt;bed&lt;/code&gt; format can also refer to the plink format, which belongs to bed, bim, fam trio. The plink bed format stores all the genetic variants, but the program needs to refer to bim file for their positions.&lt;/p&gt;&#xA;&lt;p&gt;Genome liftover obviously only needs the coordination of each SNPs, and it does not need individual genotypes at all. In fact, the bed format in PLINK does not provide any useful information. Knowing this means that providing the plink bed file is wrong. And this is pretty clear from the error message.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, since the three plink file always needs to be used together, most programs refer to them as &lt;strong&gt;plink file format&lt;/strong&gt;. So whenever people say BED format, it is the txt file format.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-08-27T13:39:59.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14249" PostTypeId="1" AcceptedAnswerId="14284" CreationDate="2020-08-27T16:11:04.383" Score="1" ViewCount="89" Body="&lt;p&gt;I have a gtex variant file- the head of which looks as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;phenotype_id                                    variant_id     &#xA;chr1:15947:16607:clu_36198:ENSG00000227232.5    chr1_13550_G_A_b38  ...&#xA;chr1:15947:16607:clu_36198:ENSG00000227232.5    chr1_14671_G_C_b38  ...&#xA;chr1:15947:16607:clu_36198:ENSG00000227232.5    chr1_14677_G_A_b38  ...&#xA;chr1:15947:16607:clu_36198:ENSG00000227232.5    chr1_16841_G_T_b38  ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Effectively I would like to be able to lookup the variants inside a particular gene (the file is sorted) and put them in a temporary file:&#xA;E.g. If the variants in gene &amp;quot;ENSG00000148481- MINDY3&amp;quot; are on line number 87528225 to 87536766- effectively what I would like is the equivalent to &lt;code&gt;zcat file.gz | sed -n '87528225,87536766p' &amp;gt; MINDY3.txt&lt;/code&gt;. However &lt;code&gt;zgrep ENSG00000148481 file.gz&lt;/code&gt; is just as fast as the above...&lt;/p&gt;&#xA;&lt;p&gt;Hence I thought tabix would be the right tool for this-&lt;/p&gt;&#xA;&lt;p&gt;I would like to tabix index it to make lookups faster. It is gzip compressed and I will firstly have to do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zcat gtex.txt.gz | bgzip &amp;gt; gtex.txt.bgz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However I am not quite sure how to proceed from there given that the data is not tab-delimited.&lt;/p&gt;&#xA;&lt;p&gt;As a trial I tried the first 1000 lines:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zcat gtex.txt.gz | head -n 1000 | bgzip &amp;gt; gtex_1000.txt.bgz&#xA;&#xA;./tabix -p bed gtex_1000.gz #index as a bed file&#xA;[get_intv] the following line cannot be parsed and skipped: chr1:15947:16607:clu_36198:ENSG00000227232.5        chr1_13550_G_A_b38     ......&#xA;&#xA;./tabix -p vcf gtex_1000.gz #index as a vcf file&#xA;Indexing as a bed file results in a warning while indexing as a vcf gives no warning yet either way when I try to retrieve a sequence:&#xA;&#xA;./tabix test.gz chr1:15000:17000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It returns nothing.&lt;/p&gt;&#xA;&lt;p&gt;I am starting to think that I will just have to write a script that splits on the ':' and writes the data to a new file.... and then index that file- which will take a huge amount of time&#xA;... Does anyone know of a trick to index the files with unconventional delimiting?&lt;/p&gt;&#xA;" OwnerUserId="1908" LastEditorUserId="1908" LastEditDate="2020-08-31T09:50:57.923" LastActivityDate="2020-09-02T14:39:46.240" Title="tabix indexing a GTEx variant file- ':' delimited" Tags="|parsing|tabix|gtex|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14251" PostTypeId="1" CreationDate="2020-08-28T01:38:49.737" Score="0" ViewCount="596" Body="&lt;p&gt;I am running into an error when I tried to run the following lines of code, the &lt;code&gt;error&lt;/code&gt; at line 3:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1 from rpy2.robjects.packages import importr&#xA;2 base = importr('base')&#xA;3 base.source(&amp;quot;http://www.bioconductor.org/biocLite.R&amp;quot;)&#xA;4 biocinstaller = importr(&amp;quot;BiocInstaller&amp;quot;)&#xA;5 biocinstaller.biocLite(&amp;quot;seqLogo&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;rpy2.rinterface_lib.embedded.RRuntimeError: Error: With R version 3.5 or greater, install Bioconductor packages using BiocManager; see https://bioconductor.org/install&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;How can I use &lt;code&gt;BiocManager&lt;/code&gt; installer instead of &lt;code&gt;biocLite &lt;/code&gt; and reflect that on my code? I couldn't find any documentation or source that talk about this error from within Python.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance&lt;/p&gt;&#xA;" OwnerUserId="9407" LastActivityDate="2020-08-28T12:59:43.053" Title="Error When Using biocLite as an installer in rpy2 python library" Tags="|r|python|rpy2|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14253" PostTypeId="2" ParentId="14251" CreationDate="2020-08-28T11:23:51.950" Score="2" Body="&lt;p&gt;You're using an outdated version of R. You can use the R 4.0 from &lt;code&gt;conda-forge&lt;/code&gt; channel, just create an environment with &lt;code&gt;r-base&lt;/code&gt; and then install &lt;code&gt;rpy2&lt;/code&gt;, the latest version compatible with R 4.0 is available through &lt;code&gt;pip&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda create -n rpy2 -c conda-forge r-base&#xA;conda activate rpy2&#xA;pip install rpy2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now you can use rpy2 with R 4.0 in your python code.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import rpy2.robjects as robjects&#xA;robjects.r('install.packages(&amp;quot;BiocManager&amp;quot;, '&#xA;           'repos=&amp;quot;http://cran.r-project.org&amp;quot;)')&#xA;robjects.r('BiocManager::install(&amp;quot;seqLogo&amp;quot;)')&#xA;robjects.r('library(seqLogo)')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Remember deactivate your env with &lt;code&gt;conda deactivate&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2020-08-28T11:23:51.950" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14256" PostTypeId="1" AcceptedAnswerId="14343" CreationDate="2020-08-29T06:11:00.937" Score="1" ViewCount="134" Body="&lt;p&gt;Is there a limit of query results on RCSB API?&lt;/p&gt;&#xA;&lt;p&gt;Interestingly, no matter how to adjust my query criteria, only 10 assemblies are returned. Here is an example query json: (truncated from a longer query file)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;  &amp;quot;query&amp;quot;: {&#xA;    &amp;quot;type&amp;quot;: &amp;quot;group&amp;quot;,&#xA;    &amp;quot;logical_operator&amp;quot;: &amp;quot;and&amp;quot;,&#xA;    &amp;quot;nodes&amp;quot;: [&#xA;      {&#xA;        &amp;quot;type&amp;quot;: &amp;quot;terminal&amp;quot;,&#xA;        &amp;quot;service&amp;quot;: &amp;quot;text&amp;quot;,&#xA;        &amp;quot;parameters&amp;quot;: {&#xA;          &amp;quot;operator&amp;quot;: &amp;quot;greater&amp;quot;,&#xA;          &amp;quot;value&amp;quot;: 1,&#xA;          &amp;quot;attribute&amp;quot;: &amp;quot;rcsb_assembly_info.polymer_entity_instance_count&amp;quot;&#xA;        }&#xA;      }&#xA;    ]&#xA;  },&#xA;  &amp;quot;return_type&amp;quot;: &amp;quot;assembly&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Example python code requesting query:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;url_query = 'https://search.rcsb.org/rcsbsearch/v1/query?json='&#xA;query_str = open(query_json, 'r').read()&#xA;query_str = query_str.replace('\n', '').replace(' ', '')&#xA;url = url_query + query_str&#xA;&#xA;r = requests.get(url)&#xA;content = r.content.decode('utf-8')&#xA;result = json.loads(content)&#xA;print(len(result['result_set']) # always equal to 10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9344" LastActivityDate="2020-09-11T14:50:43.193" Title="RCSB API: limited returned result" Tags="|assembly|api|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14258" PostTypeId="1" CreationDate="2020-08-29T09:12:20.020" Score="2" ViewCount="52" Body="&lt;p&gt;I have a microarray dataset with Agilent platform, all of the samples are cancerous, I need normal or control samples to compare with cancerous samples, can I combine control samples from another platform (for example Illumina) to Agilent platform? and then remove the batch effect (by &lt;code&gt;ComBat&lt;/code&gt; function in the sva package).&lt;/p&gt;&#xA;&lt;p&gt;I would be grateful if you could guide me to do it.&lt;/p&gt;&#xA;" OwnerUserId="9441" LastActivityDate="2020-09-01T10:38:29.963" Title="Microarray Meta-Analysis and Cross-Platform" Tags="|r|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14259" PostTypeId="2" ParentId="14258" CreationDate="2020-08-29T11:10:08.413" Score="4" Body="&lt;p&gt;In short the answer is no because you have to standardise between the arrays. Thus both arrays would need to run exactly the same control sample and you would then perform statistical standardisation (its a formal term), which is a varience measure where you look at the variation around the 'mean' for all samples in the array and here the 'mean' is the standard sample. If that can be done for both arrays at that point they can be combined.&lt;/p&gt;&#xA;&lt;p&gt;The problem is intensity per run varies and you have to control against this&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;In this scenario you  can certainly combine samples by assessing whether the control samples between runs approximates to identity. What you could use is a uniform distribution to assess this, variance must MUCH smaller than the mean for the STANDARD sample(s). I don't really do uniform distribution statistics, so I'd need to research this, because 'uniform' means identity to standard deviations against the uniform distribution is a bit meaningless. However again the uniform distribution is denoted by a very variance in comparison to the mean. The statistic of dividing mean by standard deviation/variance from memory is called the Index of Association (I could be wrong). If this value is at an extreme you're good to go &amp;lt;&amp;lt;&amp;lt;1, if it is remotely close to &lt;code&gt;1&lt;/code&gt; you're in trouble, if it is &amp;gt;1 you need to use standardisation.&lt;/p&gt;&#xA;&lt;p&gt;Presumably you have several standards to ensure a mean is meaningful. A T-test and non-parametric equivalents is not stringent enough. If there is a difference assess a correction factor and rerun the uniform distribution check. The same correction will be applied to all samples.&lt;/p&gt;&#xA;&lt;p&gt;The undisputed method to combine is standardisation, but accept in this scenario the differences between runs is potentially very small, so here what you are attempting is essentially to combine raw data or at least keep one of the runs as pure raw data.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-09-01T10:38:29.963" LastActivityDate="2020-09-01T10:38:29.963" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14260" PostTypeId="2" ParentId="14240" CreationDate="2020-08-29T20:58:37.440" Score="1" Body="&lt;p&gt;I was asked to write some code sometime near the end of last year to do this, so that viral sequences could be split based on variants at a particular location. The code I wrote doesn't actually do the splitting - it creates SAM read groups based on those variants - but that splitting can be done after processing with this script using &lt;code&gt;samtools split&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As with most of my scripts, it hasn't had much testing / use:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://gitlab.com/gringer/bioinfscripts/-/blob/master/samVarSplitter.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gitlab.com/gringer/bioinfscripts/-/blob/master/samVarSplitter.pl&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;usage:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h mapped_reads.bam | ./samVarSplitter.pl [-ref &amp;lt;ref&amp;gt;] [-pos &amp;lt;int&amp;gt;] [options]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h mapped_reads.bam | ./samVarSplitter.pl -ref Wuhan-Hu-1 -pos 11083&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If the input file contains a lot of reads that don't include the variant location, I'd recommend pre-filtering reads to only include those that cover that location, because expanding the CIGAR string and storing a record of the positions can take some time.&lt;/p&gt;&#xA;&lt;p&gt;Note: currently the position argument only supports a single location. I could extend this to multiple positions (i.e. a position range) if that were a desired feature.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-09-02T04:05:31.463" LastActivityDate="2020-09-02T04:05:31.463" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14261" PostTypeId="1" CreationDate="2020-08-30T11:19:20.850" Score="1" ViewCount="762" Body="&lt;p&gt;I got the following error:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;attempting to use to_dict to index a multiple sequence FASTA file with duplicate keys.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Are there methods for working around duplicate keys in biopython? I was thinking to simply remove the duplicate entries, but I would need to delete the entire entry associated with the keys.&lt;/p&gt;&#xA;" OwnerUserId="7459" LastEditorUserId="48" LastEditDate="2020-09-01T12:32:35.110" LastActivityDate="2020-09-01T12:32:35.110" Title="Dealing with duplicate keys" Tags="|python|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14262" PostTypeId="2" ParentId="14261" CreationDate="2020-08-30T20:57:07.067" Score="2" Body="&lt;p&gt;Yes, Biopython deliberately does not allow duplicate keys with &lt;code&gt;.to_dict()&lt;/code&gt;, e.g.&lt;/p&gt;&#xA;&lt;p&gt;Input file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat test.fa &#xA;&amp;gt;1&#xA;ATG&#xA;&amp;gt;2&#xA;AAA&#xA;&amp;gt;1&#xA;CG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;print(SeqIO.to_dict(SeqIO.parse(&amp;quot;test.fa&amp;quot;, &amp;quot;fasta&amp;quot;)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Traceback:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  File &amp;quot;script.py&amp;quot;, line 3, in &amp;lt;module&amp;gt;&#xA;    print(SeqIO.to_dict(SeqIO.parse(&amp;quot;test.fa&amp;quot;, &amp;quot;fasta&amp;quot;)))&#xA;  File &amp;quot;/path/lib/python3.7/site-packages/Bio/SeqIO/__init__.py&amp;quot;, line 789, in to_dict&#xA;    raise ValueError(&amp;quot;Duplicate key '%s'&amp;quot; % key)&#xA;ValueError: Duplicate key '1'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, you can easily write your own function, e.g.:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;def to_dict_remove_dups(sequences):&#xA;    return {record.id: record for record in sequences}&#xA;&#xA;print(to_dict_remove_dups(SeqIO.parse(&amp;quot;test.fa&amp;quot;, &amp;quot;fasta&amp;quot;)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{'1': SeqRecord(seq=Seq('CG', SingleLetterAlphabet()), id='1', name='1', description='1', dbxrefs=[]), '2': SeqRecord(seq=Seq('AAA', SingleLetterAlphabet()), id='2', name='2', description='2', dbxrefs=[])}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alternatively, you Biopython's &lt;code&gt;.to_dict()&lt;/code&gt; accepts a &lt;code&gt;key_function&lt;/code&gt; argument, e.g. if you wish to make all the headers unique&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;from itertools import count&#xA;&#xA;c = count()&#xA;&#xA;print(SeqIO.to_dict(SeqIO.parse(&amp;quot;test.fa&amp;quot;, &amp;quot;fasta&amp;quot;),&#xA;                    key_function = lambda x: f&amp;quot;{x.id}_seqNum{next(c)}&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{'1_seqNum0': SeqRecord(seq=Seq('ATG', SingleLetterAlphabet()), id='1', name='1', description='1', dbxrefs=[]), '2_seqNum1': SeqRecord(seq=Seq('AAA', SingleLetterAlphabet()), id='2', name='2', description='2', dbxrefs=[]), '1_seqNum2': SeqRecord(seq=Seq('CG', SingleLetterAlphabet()), id='1', name='1', description='1', dbxrefs=[])}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastActivityDate="2020-08-30T20:57:07.067" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14263" PostTypeId="1" AcceptedAnswerId="14264" CreationDate="2020-08-30T23:47:50.697" Score="2" ViewCount="139" Body="&lt;p&gt;In a genomic study of patients infected with SARS-CoV-2, the authors  detected cross-replicating associations with rs11385942 at locus 3p21.31 , the association signal spanned the genes SLC6A20, LZTFL1, CCR9, FYCO1, CXCR6 and XCR1&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;I would like to know what's a cross-replication association.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Source pdf is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7315890/pdf/NEJMoa2020283.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4572" LastEditorUserId="3848" LastEditDate="2020-12-03T08:33:46.723" LastActivityDate="2020-12-03T08:33:46.723" Title="COVID-19 GWAS: what is a cross-replicating association?" Tags="|genomics|gene-expression|covid-19|sars-cov-2|coronavirus|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14264" PostTypeId="2" ParentId="14263" CreationDate="2020-08-31T02:07:55.530" Score="2" Body="&lt;p&gt;Replication in GWAS is an analytical term, approximating to 'robustness'. It is not biological phenomenon of a genome replicating. What the authors are saying is these genes associate with an allelic variant(s) which correlate with the disease phenotypic of COVID-19.&lt;/p&gt;&#xA;&lt;p&gt;Three of those genes are very interesting indeed notably CCR9, CXCR6 and XCR1, because these are chemokine receptors. The later for example is associated with regulating between the innate and adaptive (CD8+, i.e. cytotoxic T-cell) immune responses (described in the image below). The innate/adaptive interface is an extremely trendy area in immunology.&lt;/p&gt;&#xA;&lt;p&gt;The authors are implying the immunological signalling associated with one or more of these chemokines is fundamental to combating COVID-19 infection. 'Replication' is simply the GWA geek for 'its real'.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BEENg.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BEENg.jpg&quot; alt=&quot;XCR1&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-08-31T02:13:54.300" LastActivityDate="2020-08-31T02:13:54.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14265" PostTypeId="2" ParentId="14249" CreationDate="2020-08-31T06:03:53.403" Score="0" Body="&lt;p&gt;Tabix files need a name and a genomic coordinate at least. Tabix can be (ab)used by creating a fake genomic coordinate and then it just indexes names but not sure I would recommend this usage. What is your actual expected application of this?&lt;/p&gt;&#xA;" OwnerUserId="60" LastActivityDate="2020-08-31T06:03:53.403" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14266" PostTypeId="2" ParentId="4163" CreationDate="2020-08-31T12:29:15.133" Score="1" Body="&lt;p&gt;This is almost right, but it's missing a little context. In ChIP-seq, it's typical to shift the positive and negative reads because otherwise the peaks end up offset from each other. The amount of shift is determined by trying different shifts and measuring how well the read densities correlate between positive and negative strands. In the question, the first sense talks about a peak on the plot of read density vs genomic position. The second has different x and y axes because it's talking about the cross-correlation between strands. X is the offset between positive and negative strands, and y is the correlation observed at that offset. Look at figures 4D,E in this ENCODE paper for visuals.&lt;/p&gt;&#xA;&lt;p&gt;Landt, S. G., Marinov, G. K., Kundaje, A., Kheradpour, P., Pauli, F., Batzoglou, S., ... &amp;amp; Chen, Y. (2012). ChIP-seq guidelines and practices of the ENCODE and modENCODE consortia. Genome research, 22(9), 1813-1831.&#xA;&lt;a href=&quot;https://genome.cshlp.org/content/22/9/1813.full.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genome.cshlp.org/content/22/9/1813.full.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2ioYL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2ioYL.png&quot; alt=&quot;x-axis: genomic position; y-axis: read density&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/2mM3P.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2mM3P.png&quot; alt=&quot;x-axis: genomic position; y-axis: read density&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1708" LastActivityDate="2020-08-31T12:29:15.133" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14267" PostTypeId="1" CreationDate="2020-08-31T13:31:45.307" Score="1" ViewCount="2199" Body="&lt;p&gt;I'm trying to download data from the TCGA for gene expression analyses in R, but I'm in doubt if I should use FPKM, FPKM-UQ or counts? When the dataset is in counts, I suppose it's raw data, isn't it? So what's the best unit to compare multiple datasets?&lt;/p&gt;&#xA;&lt;p&gt;I'm planning to use limma or Dseq2 for GE analysis and found that with Dseq2 I need to use count(non-normalized???) data... is that correct? so what's the best package and working strategy?&lt;/p&gt;&#xA;&lt;p&gt;thank you very much, Fabiano&lt;/p&gt;&#xA;" OwnerUserId="6509" LastEditorUserId="6509" LastEditDate="2020-08-31T15:24:00.577" LastActivityDate="2020-09-03T17:06:19.947" Title="FPKM, FPKM-UQ, TPM or counts: How do I know which kind of unit should I use?" Tags="|r|gene-expression|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14268" PostTypeId="2" ParentId="14267" CreationDate="2020-08-31T15:07:01.587" Score="3" Body="&lt;p&gt;Type of data you need depends on the downstream applications and since you would like to carry out DEA with &lt;code&gt;DESeq2&lt;/code&gt;, you would need raw counts (non-normalized).&lt;/p&gt;&#xA;&lt;p&gt;There are many ways to import/download TCGA data, one such tool, the &lt;code&gt;TCGAbiolinks&lt;/code&gt; package gives a nice interface for not only downloading the read count (or pre-processed) data but also associated clinical data.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-08-31T15:07:01.587" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="14269" PostTypeId="1" CreationDate="2020-08-31T15:11:18.523" Score="2" ViewCount="119" Body="&lt;p&gt;The Rosetta software can use a variety of different energy functions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ref2015&lt;/li&gt;&#xA;&lt;li&gt;GenPot&lt;/li&gt;&#xA;&lt;li&gt;Beta-Nov16&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;How are these energy functions developed and validated? Is it common to do a single protein design with several of these energy functions?&lt;/p&gt;&#xA;" OwnerUserId="6966" LastEditorUserId="4622" LastEditDate="2020-09-04T18:27:25.223" LastActivityDate="2020-12-17T16:12:57.070" Title="How are different forcefields developed/evaluated?" Tags="|protein-structure|computational-biochemistry|rosetta|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14270" PostTypeId="2" ParentId="13893" CreationDate="2020-08-31T21:29:53.087" Score="0" Body="&lt;p&gt;My minimal procedure based on the code that @deeenes shared for a single file. Comments to improve this solution are highly appreciated:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;files = list_of_filenames&#xA;&#xA;feature_map = oms.FeatureMap()&#xA;mass_traces = []&#xA;mass_traces_split = []&#xA;mass_traces_filtered  = []&#xA;&#xA;exp = oms.MSExperiment()&#xA;&#xA;options = oms.PeakFileOptions()&#xA;options.setMSLevels([1])&#xA;&#xA;fh = oms.MzXMLFile()&#xA;fh.setOptions(options)&#xA;&#xA;# Peak map&#xA;peak_map = oms.PeakMap()&#xA;&#xA;file = files[0]&#xA;&#xA;print('# Filename:', basename(file))&#xA;fh.load(file, exp)&#xA;&#xA;print('# Spectra:', len( exp.getSpectra() ))&#xA;print('# Chromatograms:', len( exp.getChromatograms() ) )&#xA;&#xA;for chrom in exp.getChromatograms():&#xA;    peak_map.addChrom(chrom)&#xA;    &#xA;for spec in exp.getSpectra():&#xA;    peak_map.addSpectrum(spec)&#xA;&#xA;mass_trace_detect = oms.MassTraceDetection()&#xA;mass_trace_detect.run(peak_map, mass_traces, 100000)&#xA;&#xA;print('# Mass traces:', len(mass_traces) )&#xA;&#xA;elution_peak_detection = oms.ElutionPeakDetection()&#xA;elution_peak_detection.detectPeaks(mass_traces, mass_traces_split)&#xA;print('# Mass traces split:', len(mass_traces_split) )&#xA;&#xA;&#xA;&#xA;feature_finding_metabo = oms.FeatureFindingMetabo()&#xA;feature_finding_metabo.run(&#xA;            mass_traces_split,&#xA;            feature_map,&#xA;            mass_traces_filtered)&#xA;&#xA;print('# Mass traces filtered:', len(mass_traces_filtered) )&#xA;print('# Features:', feature_map.size() )&#xA;&#xA;oms.FeatureXMLFile().store('FeatureFindingMetabo.featureXML', feature_map)&#xA;&#xA;&amp;gt;&amp;gt;&amp;gt;&#xA;# Filename: CA_1.mzXML&#xA;# Spectra: 989&#xA;# Chromatograms: 0&#xA;# Mass traces: 18476&#xA;# Mass traces split: 25194&#xA;# Mass traces filtered: 0&#xA;# Features: 23549&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2020-08-31T22:47:16.880" LastActivityDate="2020-08-31T22:47:16.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14271" PostTypeId="2" ParentId="14269" CreationDate="2020-09-01T09:35:31.377" Score="1" Body="&lt;p&gt;Some force-fields rely solely on theoretical physical principles, while others are tweaked specifically for the goal of accurately depicting certain structures like sheets and helices or movement/diffusion aspects at different theoretical temperatures of a set of well-studied proteins. (Top-Down vs. Bottom-Up)&lt;/p&gt;&#xA;&lt;p&gt;I haven't done MD myself but I am in an MD group and people seem to normally stick to a single force-field for a given MD project but it happens that they do very short simulation runs with force-fields just as a test.&#xA;They definitely don't plan on doing the full simulation on different force-fields unless the comparison of those is their main goal.&lt;/p&gt;&#xA;" OwnerUserId="6463" LastActivityDate="2020-09-01T09:35:31.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14272" PostTypeId="1" AcceptedAnswerId="14279" CreationDate="2020-09-01T09:42:00.750" Score="2" ViewCount="3027" Body="&lt;p&gt;Or Kbps/Gbps. It feels like it should be conceptually very simple, but I can't seem to figure out the right combination of keywords to find it via my search engine. Help would be appreciated!&lt;/p&gt;&#xA;&lt;p&gt;I have BBMAP, SRAtoolkit and MEGAHIT already installed, and also use bash. I'd be very happy if this can be answered with software that I already have, but if not that's perfectly fine.&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2023-10-11T05:37:31.383" Title="How to measure the total size of a fastq file in base pairs?" Tags="|fastq|bash|" AnswerCount="6" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14273" PostTypeId="2" ParentId="14272" CreationDate="2020-09-01T10:08:50.737" Score="1" Body="&lt;p&gt;The number of bases in a fastq file can be counted in bash with awk and wc&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'NR % 4 == 0' ORS=&amp;quot;&amp;quot; fastqfile|wc -m&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The awk code prints every fourth line (which is actually the quality scores, but that doesn't matter here). wc -m returns the total number of characters.&lt;/p&gt;&#xA;&lt;p&gt;Edited to add ORS=&amp;quot;&amp;quot; to prevent count of newlines (the double quotes are not strictly required)&lt;/p&gt;&#xA;" OwnerUserId="8919" LastEditorUserId="8919" LastEditDate="2020-09-01T12:35:09.083" LastActivityDate="2020-09-01T12:35:09.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14274" PostTypeId="2" ParentId="14269" CreationDate="2020-09-01T10:28:56.863" Score="1" Body="&lt;p&gt;The Rosetta ref2015 is well described in two papers, which I wish I had read much earlier, see &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5717763/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this JChIM paper&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;It's a statistical forcefield, where the terms are weighted to best match the empirical total energy in kcal/mol and the paper does a swell job at explaining what they do.&lt;/p&gt;&#xA;&lt;p&gt;For several Rosetta experiments, you switch between &lt;em&gt;full-atom&lt;/em&gt; and &lt;em&gt;centroid&lt;/em&gt; (=coarse grain). These have different weights, as you can see from the files in the database folder or via pyrosetta, but majorly they have different &amp;quot;params&amp;quot; (topologies) for the residues as the centroid lacks many atoms as it's a simplification. In some instances, you may switch between &lt;em&gt;dihedral&lt;/em&gt; (normal) to &lt;em&gt;cartesian&lt;/em&gt; (explained in the FastRelax paper), wherein the former used internal coordinates (e.g. atom A is 1.5 Å away from B and A,B,C is a 107ª angle etc —cf. a params file &lt;code&gt;ICOOR&lt;/code&gt; block), while the later is cartesian coordinates ( A is at [x, y, z] —cf. PDB file).&lt;/p&gt;&#xA;&lt;p&gt;The GenPot uses its own &lt;code&gt;AtomTypes&lt;/code&gt;, which means it uses its own params for each residue —like centroid/fa. The topology of a residue (ResidueType) is based on its conformation (ICOOR), bonding and atom definition, which is atom name, gasteiger-marsili charge and AtomType (an amalgamation of element, sp-hybridization, size, H-bond donor/acceptor etc.) These AtomTypes are better for ligands as many groups behave different from peptide atoms —say, a nitrile nitrogen is definitely different from a deprotonated aromatic nitrogen.&lt;/p&gt;&#xA;&lt;p&gt;Some application, such as hydration, ligand docking etc. have specifically tweaked scorefunctions for use by their movers. FastRelax is a nice example as the cycle weighting changes are spat out by the tracer at vertiginous speed, where the protocol is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;repeat 5&#xA;coord_cst_weight 1.0&#xA;scale:fa_rep 0.040&#xA;repack&#xA;scale:fa_rep 0.051&#xA;min 0.01&#xA;coord_cst_weight 0.5&#xA;scale:fa_rep 0.265&#xA;repack&#xA;scale:fa_rep 0.280&#xA;min 0.01&#xA;coord_cst_weight 0.0&#xA;scale:fa_rep 0.559&#xA;repack&#xA;scale:fa_rep 0.581&#xA;min 0.01&#xA;coord_cst_weight 0.0&#xA;scale:fa_rep 1&#xA;repack&#xA;min 0.00001&#xA;accept_to_best&#xA;endrepeat&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see, the repulsion term of the Lenard-Jones is going from 0.04 up. These values were chosen not for thermodynamic reasons, but worked best for the application (energy minimisation of the structure).&lt;/p&gt;&#xA;&lt;p&gt;However, this switching is all internal. Scores are always reported with the same &lt;code&gt;scorefunction&lt;/code&gt; and comparing between different ones is not advised even for the minority of &lt;code&gt;scorefunction&lt;/code&gt;s calibrated to be in kcal/mol.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6966" LastEditDate="2020-12-17T16:12:57.070" LastActivityDate="2020-12-17T16:12:57.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14277" PostTypeId="2" ParentId="14272" CreationDate="2020-09-01T11:54:13.557" Score="0" Body="&lt;p&gt;With bbmap:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;reformat.sh &amp;lt;/path/to/fastq&amp;gt;&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3818" LastActivityDate="2020-09-01T11:54:13.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14278" PostTypeId="2" ParentId="14272" CreationDate="2020-09-01T13:11:46.020" Score="1" Body="&lt;p&gt;&lt;strong&gt;Using Perl:&lt;/strong&gt;&lt;br /&gt;&#xA;Print the sequence lines (line number 2, 6, 10, etc). Remove the newlines with &lt;code&gt;chomp&lt;/code&gt;. Count the bytes (here, bases) using &lt;code&gt;wc -c&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -ne 'if ( $. % 4 == 2 ) { chomp; print; }' | wc -c&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6251" LastActivityDate="2020-09-01T13:11:46.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14279" PostTypeId="2" ParentId="14272" CreationDate="2020-09-01T13:17:57.003" Score="5" Body="&lt;p&gt;I've been using this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;cat file.fastq | paste - - - - | cut -f 2 | tr -d '\n' | wc -c &lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Explanation :&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;paste - - - -&lt;/code&gt; : print four consecutive lines in one row (tab delimited), to merge the info for each read&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;cut -f2&lt;/code&gt; : print only the second column, to access the sequence after the paste&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;wc -c&lt;/code&gt; : count the characters&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;tr -d '\n'&lt;/code&gt;: to remove from count the eventual newline characters&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;(a tip for your googling: try search for &amp;quot;counting number of bases in fastq file&amp;quot;)&lt;/p&gt;&#xA;" OwnerUserId="294" LastActivityDate="2020-09-01T13:17:57.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14280" PostTypeId="1" CreationDate="2020-09-01T13:55:41.943" Score="0" ViewCount="95" Body="&lt;p&gt;I'm analyzing Sn-RNA seq datasets which include different brain regions and spinal cord in patients and normal controls. I would have different comparison within and between individuals. I've got a question which I could not find the answer. First, I know that in the clustering step FindConservedMarkers is the best choice when you are comparing controls &amp;amp; patients. I was wondering does that work the same when I want to compare two different regions like the brain and spinal cord? if not what do you recommend for this comparison? as there might be different cell population in two different regions and those regions are not the same&lt;/p&gt;&#xA;" OwnerUserId="9456" LastActivityDate="2020-09-01T13:55:41.943" Title="which marker identifications method is recommended when single-nuclei RNA seq datasets are integrated across regions" Tags="|single-cell|clustering|10x-genomics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14281" PostTypeId="2" ParentId="14267" CreationDate="2020-09-01T16:19:30.750" Score="1" Body="&lt;p&gt;You should use &amp;quot;counts&amp;quot; from the list of gene expression metrics that you provided rather than either FPKM or FPKM-UQ. FPKM is a normalized gene expression metric and an acronym for fragments per kilobase per million mapped reads. FPKM normalizes for both sequencing depth and genome size. Those specific packages want non-normalized counts as the expression metric. This means the number of reads mapped to a particular gene or feature. This is a slightly ambiguous metric but it can be the number of single-end reads mapped to a loci, the amount of pair end reads counted towards a loci, or the amount of super-reads (the theoretical sequence that spans the pair of paired-end reads which includes the presumed sequence between the first and second strand of a set of pair end reads) counted toward a loci. These values should be integers based on the assumptions of RNA-Seq and microarray technology which measures RNA transcripts present in a sample and precludes transcripts from spanning multiple genes. It does not matter which of these metrics you use as long as you are consistent. Those Bioconductor packages provide several different normalization methods but rely on read counts as their input.&lt;/p&gt;&#xA;" OwnerUserId="4507" LastActivityDate="2020-09-01T16:19:30.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14282" PostTypeId="2" ParentId="7029" CreationDate="2020-09-01T18:53:49.183" Score="1" Body="&lt;p&gt;There is some description of homocompressed polymers (-p kmer) in this paper: &lt;a href=&quot;https://arxiv.org/pdf/1708.01492.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1708.01492.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6660" LastActivityDate="2020-09-01T18:53:49.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14283" PostTypeId="1" AcceptedAnswerId="14288" CreationDate="2020-09-02T12:37:44.690" Score="2" ViewCount="311" Body="&lt;p&gt;I was able to compute the significance of the overlap between 2 gene sets using the cdf function of &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.hypergeom.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy hypergeometric distribution&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I wish to be able to perform the same calculation for more than 2 gene sets; should I use the multivariate hypergeometric distribution cdf function for that?&lt;/p&gt;&#xA;&lt;p&gt;Are there any websites that provides the same calculations over gene sets so I can validate my results?&lt;/p&gt;&#xA;" OwnerUserId="9460" LastActivityDate="2020-09-03T14:54:56.783" Title="Compute the significance of the overlap between 2 or more gene sets" Tags="|statistics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14284" PostTypeId="2" ParentId="14249" CreationDate="2020-09-02T13:23:41.480" Score="0" Body="&lt;p&gt;I proceeded as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import gzip&#xA;import sys&#xA;&#xA;qtls=['sqtls','eqtls']&#xA;tissues=['Adipose_Subcutaneous', 'Adipose_Visceral_Omentum', 'Adrenal_Gland', 'Artery_Aorta', 'Artery_Coronary', 'Artery_Tibial', 'Brain_Amygdala', 'Brain_Anterior_cingulate_cortex_BA24', 'Brain_Caudate_basal_ganglia', 'Brain_Cerebellar_Hemisphere', 'Brain_Cerebellum', 'Brain_Cortex', 'Brain_Frontal_Cortex_BA9', 'Brain_Hippocampus', 'Brain_Hypothalamus', 'Brain_Nucleus_accumbens_basal_ganglia', 'Brain_Putamen_basal_ganglia', 'Brain_Spinal_cord_cervical_c-1', 'Brain_Substantia_nigra', 'Breast_Mammary_Tissue', 'Cells_Cultured_fibroblasts', 'Cells_EBV-transformed_lymphocytes', 'Colon_Sigmoid', 'Colon_Transverse', 'Esophagus_Gastroesophageal_Junction', 'Esophagus_Mucosa', 'Esophagus_Muscularis', 'Heart_Atrial_Appendage', 'Heart_Left_Ventricle', 'Kidney_Cortex', 'Liver', 'Lung', 'Minor_Salivary_Gland', 'Muscle_Skeletal', 'Nerve_Tibial', 'Ovary', 'Pancreas', 'Pituitary', 'Prostate', 'Skin_Not_Sun_Exposed_Suprapubic', 'Skin_Sun_Exposed_Lower_leg', 'Small_Intestine_Terminal_Ileum', 'Spleen', 'Stomach', 'Testis', 'Thyroid', 'Uterus', 'Vagina', 'Whole_Blood']&#xA;&#xA;gene_old=''&#xA;&#xA;&#xA;def index(tissue,qtl,gene_old):&#xA;        counter_old,counter=1,1&#xA;        if qtl=='sqtls':&#xA;                path='/path/to/file/'+tissue+'.v8.sqtl_allpairs.txt.gz'&#xA;        else:&#xA;                path='/path/to/file/'+tissue+'.allpairs.txt.gz'&#xA;        f = gzip.open(path, 'r')&#xA;        next(f)&#xA;        for line in f:&#xA;                try:&#xA;                        line=line.decode()&#xA;                        line=line[:line.find('\t')]&#xA;                        gene=line[line.rfind('ENS'):].split('.')[0]&#xA;                        #print(gene,gene_old,counter_old,counter)&#xA;                        with open(qtl+'_'+tissue+'.idx','a') as indexed:          &#xA;                                 indexed.write(str(gene_old)+'\t'+str(counter_old)+'\t'+str(counter)+'\n')&#xA;                        gene_old=gene&#xA;                        counter +=1&#xA;                        counter_old=counter&#xA;                except IndexError:&#xA;                        print('tissue,qtl,gene_old,counter')&#xA;                        counter+=1&#xA;        f.close()&#xA;&#xA;&#xA;for qtl in qtls:&#xA;        for tissue in tissues:&#xA;               qtl=row.qtls&#xA;               tissue=row.tissues&#xA;               print(qtl, tissue)&#xA;               index(tissue,qtl,gene_old)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above script writes an index for each gene which should look as follows (took 2 days to finish all files):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ensid    row_start    row_end&#xA;......&#xA;ENSG00000230337     1526391 1534329&#xA;ENSG00000171819 1534330 1542321&#xA;ENSG00000198793    1542322 1550178&#xA;ENSG00000120942  1550179 1558010&#xA;.....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And to get a gene of interest you will have to look up the coordinates for an ensemblid/gene and for example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zcat Brain_Nucleus_accumbens_basal_ganglia.v8.sqtl_allpairs.txt.gz | sed -n '1550179,1558010p;1558011q' &amp;gt; ENSG00000120942.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1908" LastEditorUserId="1908" LastEditDate="2020-09-02T14:39:46.240" LastActivityDate="2020-09-02T14:39:46.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14286" PostTypeId="1" CreationDate="2020-09-03T09:06:31.303" Score="2" ViewCount="339" Body="&lt;p&gt;I'm trying to create a snp distance matrix from &lt;code&gt;M tuberculosis&lt;/code&gt; isolates to try and infer transmission networks. My plan was to use &lt;a href=&quot;https://github.com/tseemann/snippy&quot; rel=&quot;nofollow noreferrer&quot;&gt;snippy&lt;/a&gt; to make a core genome alignment and then &lt;a href=&quot;https://github.com/tseemann/snp-dists&quot; rel=&quot;nofollow noreferrer&quot;&gt;snp-dists&lt;/a&gt; to create the matrix but have hit a snag with creating a core genome alignment. When I try it with all 972 of my samples, snippy halts when trying to run &lt;a href=&quot;https://github.com/sanger-pathogens/snp-sites&quot; rel=&quot;nofollow noreferrer&quot;&gt;snp-sites&lt;/a&gt;, saying &lt;code&gt;Warning: No SNPs were detected so there is nothing to output.&lt;/code&gt; The issue is that with all the samples included, the core genome size is 0.&lt;/p&gt;&#xA;&lt;p&gt;When looking at the alignment statistics provided by snippy, it looks like some isolates have very low numbers of aligned base pairs (??due to low coverage or contamination). I have progressively filtered out samples with low numbers of aligned base pairs and can then get snippy/snp-sites to work. As I increase my filter stringency, the resulting core genome is still short. Eg Filtering out those with aligned base pairs of &amp;lt;90% of the reference had a core genome of only 5747bp (from a reference of 4.4 million bp), while excluding 88 (9%) of my samples.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;snippy&lt;/code&gt;'s author recommends using it's output file &lt;code&gt;core.txt&lt;/code&gt; to figure out which samples are the &amp;quot;bad&amp;quot; outliers. That file provides 1) the length of the reference, 2) the number of aligned base pairs, 3) the number of unaligned base pairs, 3) the number of variant sites, 4) the number of heterogenous sites, 5) the number of masked sites, and 6) the number of low coverage sites.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#&amp;gt;   ID                        LENGTH ALIGNED UNALIGNED VARIANT   HET MASKED LOWCOV&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;                      &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 R15795_CATCAAGT_S34_L006 4411532 4192432      6818     755   428 209178   2676&#xA;#&amp;gt; 2 R15842_GTCTGTCA_S49_L006 4411532 4187344     10304     784   441 209178   4265&#xA;#&amp;gt; 3 R15876_CGCTGATC_S36_L006 4411532 4176662     18992    1292   660 209178   6040&#xA;#&amp;gt; 4 R15951_ATTGGCTC_S7_L002  4411532 4170649     14732    1281   980 209178  15993&#xA;#&amp;gt; 5 R16019_TGGAACAA_S78_L001 4411532 4190733      8132     712   715 209178   2774&#xA;#&amp;gt; 6 R16046_GACTAGTA_S6_L002  4411532 4186069      4309    1257  1084 209178  10892&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My question is: what heuristic would you use to filter out samples prior to creating a core genome?&lt;/p&gt;&#xA;&lt;p&gt;And secondarily, what would a reasonable core genome size be for Mtb?&lt;/p&gt;&#xA;" OwnerUserId="1152" LastEditorUserId="1152" LastEditDate="2020-09-03T09:12:41.843" LastActivityDate="2020-09-03T18:40:28.860" Title="Core genome too short" Tags="|snp|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14288" PostTypeId="2" ParentId="14283" CreationDate="2020-09-03T14:54:56.783" Score="3" Body="&lt;p&gt;This is a shot at it, first an example dataset:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt  &#xA;import numpy as np&#xA;import functools&#xA;from matplotlib_venn import venn3&#xA;    &#xA;# define universe&#xA;uni = [&amp;quot;gene&amp;quot;+str(i) for i in range(1000)]&#xA;# some overlap&#xA;gs1 = uni[250:300] + uni[900:950]&#xA;gs2 = uni[:300]&#xA;gs3 = uni[250:500]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The ever amazing venn diagram:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;venn3([set(gs1),set(gs2),set(gs3)],set_labels=[&amp;quot;gs1&amp;quot;,&amp;quot;gs2&amp;quot;,&amp;quot;gs3&amp;quot;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZV7Yl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZV7Yl.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then a function to draw a set with length equivalent of each set, randomly from the universe and find length of intersection (all 3):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def sim_intersect(uni,set_lengths):&#xA;    randomsets = [np.random.choice(uni,n) for n in set_lengths]&#xA;    return len(functools.reduce(np.intersect1d,randomsets))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We run this 1000 times:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;permuted_values = [sim_intersect(uni,[len(gs1),len(gs2),len(gs3)]) for i in range(1000)]&#xA;&#xA;plt.hist(permuted_values,bins=range(50))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/plpKC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/plpKC.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The probability of observing the starting result, using (B+1)/(M+1) as estimator, see this &lt;a href=&quot;https://stats.stackexchange.com/questions/109207/p-values-equal-to-0-in-permutation-test&quot;&gt;post&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(sum(np.array(permuted_values)&amp;gt;obs_n)+1)/(1000+1)&#xA;0.000999000999000999&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-09-03T14:54:56.783" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14289" PostTypeId="2" ParentId="4341" CreationDate="2020-09-03T15:34:44.757" Score="0" Body="&lt;p&gt;Check our recent &lt;a href=&quot;https://academic.oup.com/narcancer/article/2/2/zcaa011/5862620&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; where we, not only characterized pathway activities in scRNAseq experiments, but also took advantage of mechanistic models in order to make some in silico prediction of drug effects.&lt;/p&gt;&#xA;" OwnerUserId="5139" LastActivityDate="2020-09-03T15:34:44.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14290" PostTypeId="1" CreationDate="2020-09-03T16:01:38.460" Score="2" ViewCount="206" Body="&lt;p&gt;Assume we have a &lt;code&gt;query.fa&lt;/code&gt; file that contains sequences and we run:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;blat -stepSize=5 -repMatch=2253 -minScore=20 -minIdentity=0 -out=pslx /genomes/mm10.fa.qz query.fa output.pslx&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;the output &lt;code&gt;output.pslx&lt;/code&gt; file looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;match   mis-    rep.    N's     Q gap   Q gap   T gap   T gap   strand  Q               Q       Q       Q       T               T       T       T       block       blockSizes      qStarts  tStarts&#xA;        match   match           count   bases   count   bases           name            size    start   end     name            size    start   end     count&#xA;---------------------------------------------------------------------------------------------------------------------------------------------------------------&#xA;20      0       0       0       0       0       0       0       +       seq     20      0       20      chr9    124595110       44046930        44046950   20,      0,      44046930,       aaaagtatcagtgtgtatag,   aaaagtatcagtgtgtatag,&#xA;20      0       0       0       0       0       0       0       +       seq     20      0       20      chr9    124595110       44046930        44046950   20,      0,      44046930,       aaaagtatcagtgtgtatag,   aaaagtatcagtgtgtatag,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What would be a reasonable way to get the genomic contexts (5bp upsteam and 5bp downstream) for each aligned sequence.&lt;/p&gt;&#xA;&lt;p&gt;For example, assume that &lt;code&gt;blat&lt;/code&gt; found that the seq: &lt;code&gt;AAATTGGGGAAAA&lt;/code&gt; aligns to &lt;code&gt;chr2:100-113&lt;/code&gt;, so the question is how to get &lt;code&gt;chr2:95-118&lt;/code&gt; easily without reinventing the wheel.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I couldn't make it work with bedtools, because my genome's index file is corrupted, but this should work for others who have successfully used &lt;code&gt;bwa&lt;/code&gt; or &lt;code&gt;samtools&lt;/code&gt; to index their reference genome:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blat -stepSize=5 -repMatch=2253 -minScore=20 -minIdentity=0 -out=pslx /genomes/mm10.fa.qz query.fa output.pslx&#xA;awk 'NR&amp;gt;5 {print &lt;span class=&quot;math-container&quot;&gt;$14 &quot;\t&quot; $&lt;/span&gt;16-10&amp;quot;\t&amp;quot; $17+10}' output.pslx &amp;gt; regions.bed&#xA;bedtools getfasta -fi /genomes/mm10.fa.gz -bed regions.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2020-09-06T12:55:33.050" LastActivityDate="2020-09-23T23:11:17.050" Title="How to get the genomic sequences from a blat result?" Tags="|python|bedtools|data-preprocessing|blat|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14294" PostTypeId="2" ParentId="14286" CreationDate="2020-09-03T18:40:28.860" Score="2" Body="&lt;p&gt;Targeting Illumina reads with &amp;gt;70X depth-of-coverage is a good place to start.  Once a robust database is established lower coverage isolates can be used effectively too.&#xA;More specific to individual SNPs... filtering on VCF QUAL value, Map Quality and/or AC value have shown to be useful.&lt;/p&gt;&#xA;&lt;p&gt;Core genome will likely be around 1,000 SNPs, but it will depend on the lineage the reference belongs to and the aligned sample lineage.  If both are from the same lineage you're likely to be less than 700 SNPs.  When from different lineages SNP counts will obviously be higher in addition to the alignment error seen.&lt;/p&gt;&#xA;&lt;p&gt;Pertaining to filtering individual SNPs...&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/USDA-VS/vSNP&quot; rel=&quot;nofollow noreferrer&quot;&gt;vSNP&lt;/a&gt; is a tool developed for SNP analysis.  It can be applied to different specie types but was originally developed for use with TB complex isolates.  Code detail is &lt;a href=&quot;https://github.com/USDA-VS/vSNP/blob/master/docs/detailed_usage.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;vSNP is a 2-step process.  First, generating VCF files.  Second, using those VCF files to output SNP tables and trees.  The goal of the script is to achieve a high resolution genotyping analysis that can be validated with minimal effort.  Some steps are included to help achieve this such as reference selection, automatic grouping and position filtering.&lt;/p&gt;&#xA;&lt;p&gt;Pseudocode:&lt;/p&gt;&#xA;&lt;p&gt;For set of VCF files created from the same reference...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def find_positions()&#xA;    AC=2&#xA;    QUAL &amp;gt; 300 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For each found position...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def get_snps()&#xA;    AC=2 (solid call)&#xA;    QUAL &amp;gt; 50&#xA;        THEN --&amp;gt; SNP&#xA;    AC=1 (mix call)&#xA;    QUAL &amp;gt; 150&#xA;        THEN --&amp;gt; AMBIGIOUS&#xA;    AC=2&#xA;    QUAL &amp;gt;= 50&#xA;       THEN --&amp;gt; REF&#xA;    QUAL &amp;lt; 50&#xA;       THEN --&amp;gt; N&#xA;    ALT == &amp;quot;None&amp;quot;&#xA;       THEN --&amp;gt; -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;First find all possible positions that could be informative in all samples/VCF files: &lt;code&gt;find_positions()&lt;/code&gt;, then define each position to be included in the alignment: &lt;code&gt;get_snps()&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Then it can be convenient to filter consistently poor alignment regions such as PPE/PGRS.&lt;/p&gt;&#xA;" OwnerUserId="6343" LastActivityDate="2020-09-03T18:40:28.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14295" PostTypeId="2" ParentId="11477" CreationDate="2020-09-03T21:54:05.597" Score="1" Body="&lt;p&gt;You need to provide sorted bam files, ideally with an index. If you run &lt;code&gt;ChIPQCsample&lt;/code&gt;, it calls &lt;code&gt;ChIPQC:::sampleQC&lt;/code&gt; which iterates through the chromosomes or contigs, using the index, and the absence of the index will crash it.&lt;/p&gt;&#xA;&lt;p&gt;We can use an example, first in linux:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;wget ftp://hgdownload.cse.ucsc.edu/goldenPath/hg19/encodeDCC/wgEncodeHaibTfbs/wgEncodeHaibTfbsA549RxlchV0416101Etoh02AlnRep1.bam -O Etoh02AlnRep1.bam&#xA;&#xA;samtools index ./Etoh02AlnRep1.bam&#xA;# s&#xA;samtools view -uh Etoh02AlnRep1.bam chr1 &amp;gt; sorted.bam&#xA;samtools index sorted.bam&#xA;# quick way to unsort&#xA;samtools sort -n -o unsorted.bam sorted.bam&#xA;# it fails to index&#xA;samtools index unsorted.bam &#xA;[E::hts_idx_push] Unsorted positions on sequence #1: 246814554 followed by 42922635&#xA;samtools index: failed to create index for &amp;quot;unsorted.bam&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then in R:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(ChIPQC)&#xA;ChIPQCsample(&amp;quot;unsorted.bam&amp;quot;)&#xA;Bam file has 25 contigs&#xA;Error in value[[3L]](cond) : &#xA;  failed to open BamFile: failed to load BAM index&#xA;  file: unsorted.bam&#xA;&#xA;ChIPQCsample(&amp;quot;sorted.bam&amp;quot;,annotation=NULL)&#xA;Bam file has 25 contigs&#xA;Calculating coverage histogram for chr1&#xA;&#xA;Calculating SSD for chr1&#xA;&#xA;Calculating unique positions per strand for chr1&#xA;&#xA;Calculating shift for chr1&#xA;&#xA; 300 / 300&#xA;Counting reads in features for chr1&#xA;&#xA;....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-09-03T21:54:05.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14296" PostTypeId="1" CreationDate="2020-09-03T13:53:58.613" Score="1" ViewCount="2832" Body="&lt;p&gt;I have created a DESeq object (dds) and now I want to access the results and apply the lfc shrinkage on them.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;res &amp;lt;- results(dds, alpha=0.05)&#xA;resultsNames(dds) &#xA;[1] &amp;quot;Intercept&amp;quot;                                                  &amp;quot;Cell_type_Interstitial_macrophages_vs_Alveolar_macrophages&amp;quot;&#xA;[3] &amp;quot;Cell_type_T_cells_vs_Alveolar_macrophages&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here we see all the coefficients I got from the results function.&#xA;Now I try to use the lfcShrink() function on the second coefficient:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;res_im_vs_am &amp;lt;- lfcShrink(dds, coef=2, res=res, type=&amp;quot;apeglm&amp;quot;)&#xA;Error in lfcShrink(dds, coef = 2, res = res, type = &amp;quot;apeglm&amp;quot;) : &#xA;  'coef' should specify same coefficient as in results 'res'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;it doesn't work, so I try with the name directly copy-pasted from the above code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;res_im_vs_am &amp;lt;- lfcShrink(dds, coef = &amp;quot;Cell_type_Interstitial_macrophages_vs_Alveolar_macrophages&amp;quot;, res=res, type=&amp;quot;apeglm&amp;quot;)&#xA;Error in lfcShrink(dds, coef = &amp;quot;Cell_type_Interstitial_macrophages_vs_Alveolar_macrophages&amp;quot;,  : &#xA;  'coef' should specify same coefficient as in results 'res'    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Same error.&#xA;According to the &lt;code&gt;?lfcShrink&lt;/code&gt; for coef: the name or number of the coefficient (LFC) to shrink, consult resultsNames(dds) after running DESeq(dds).&lt;/p&gt;&#xA;&lt;p&gt;So I do not understand why it is not working. Anyone can help?&lt;/p&gt;&#xA;" OwnerDisplayName="Paula" LastActivityDate="2020-09-03T22:04:44.433" Title="DESeq2 - Error in lfcShrink: 'coef' should specify same coefficient as in results 'res'" Tags="|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14297" PostTypeId="2" ParentId="14296" CreationDate="2020-09-03T14:15:08.533" Score="1" Body="&lt;p&gt;We use an example dataset:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(DESeq2)&#xA;mat = matrix(rnbinom(9000,mu=100,size=1),ncol=9)&#xA;rownames(mat) = paste0(&amp;quot;gene&amp;quot;,1:1000)&#xA;grps = rep(c(&amp;quot;A&amp;quot;,&amp;quot;B&amp;quot;,&amp;quot;C&amp;quot;),each=3)&#xA;&#xA;dds = DESeqDataSetFromMatrix(mat,data.frame(grps),~grps)&#xA;dds = DESeq(dds)&#xA;&#xA;resultsNames(dds) &#xA;[1] &amp;quot;Intercept&amp;quot;   &amp;quot;grps_B_vs_A&amp;quot; &amp;quot;grps_C_vs_A&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When you call &lt;code&gt;results&lt;/code&gt;, by default it takes the last coefficient, in this example is C vs A:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;res &amp;lt;- results(dds, alpha=0.05)&#xA;&#xA;log2 fold change (MLE): grps C vs A &#xA;Wald test p-value: grps C vs A &#xA;DataFrame with 6 rows and 6 columns&#xA;              baseMean     log2FoldChange             lfcSE               stat&#xA;             &amp;lt;numeric&amp;gt;          &amp;lt;numeric&amp;gt;         &amp;lt;numeric&amp;gt;          &amp;lt;numeric&amp;gt;&#xA;gene1 74.3974631643997 -0.439258650876538  1.22842645044656 -0.357578307367818&#xA;gene2 99.4576039995999    1.2903547180366  1.12500005531808   1.14698191518912&#xA;gene3 87.1593850381565  -3.11763876685787  1.19848628038255  -2.60131368868297&#xA;gene4   51.47821785473  -2.16622171024749 0.953571440023498  -2.27169315200349&#xA;gene5 62.4630151830988   1.05879388186169  1.31999477669387  0.802119751195993&#xA;gene6 111.967714581788  -1.23916751032406  1.17612862134365  -1.05359863524824&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you want to look at the second comparison:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;res &amp;lt;- results(dds, alpha=0.05,name=&amp;quot;grps_B_vs_A&amp;quot;)&#xA;res_lfc &amp;lt;- lfcShrink(dds, coef=2, res=res, type=&amp;quot;apeglm&amp;quot;)&#xA;&#xA;log2 fold change (MAP): grps B vs A &#xA;Wald test p-value: grps B vs A &#xA;DataFrame with 6 rows and 5 columns&#xA;              baseMean      log2FoldChange             lfcSE            pvalue&#xA;             &amp;lt;numeric&amp;gt;           &amp;lt;numeric&amp;gt;         &amp;lt;numeric&amp;gt;         &amp;lt;numeric&amp;gt;&#xA;gene1 74.3974631643997 0.00143892076074213 0.275057978370776 0.987946388824352&#xA;gene2 99.4576039995999  -0.031292774512063 0.276439771853046 0.632995722892249&#xA;gene3 87.1593850381565 0.00806769808245902 0.274672487627629                NA&#xA;gene4   51.47821785473 -0.0394804572502463 0.274244572145665 0.609999235800452&#xA;gene5 62.4630151830988   0.065875708916621 0.288411208595865 0.234448669110134&#xA;gene6 111.967714581788 -0.0871847461207727 0.295733383995144 0.153081913500716&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or simply:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;res_lfc &amp;lt;- lfcShrink(dds, coef=2, type=&amp;quot;apeglm&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" OwnerDisplayName="StupidWolf" LastActivityDate="2020-09-03T14:30:57.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14298" PostTypeId="1" CreationDate="2020-09-03T22:20:53.543" Score="2" ViewCount="46" Body="&lt;p&gt;My data look like this&#xA;&lt;a href=&quot;https://i.stack.imgur.com/nEDDO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nEDDO.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;They are compound similarity estimations(the whole file is around 10GB). What I am trying to achieve is to compare the similarity distributions of each compound using the Kolmogorov-Smirnoff test so as to use the data for AIC calculation.&lt;/p&gt;&#xA;&lt;p&gt;This is my code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;from scipy.stats import ks_2samp&#xA;import re&#xA;&#xA;with open('ecfp4_file.csv', 'r') as f, open('Metrics.tsv', 'a') as f_out:&#xA;    f_out.write('compound_1' + '\t' + 'compound_2' + '\t' + 'Similarity' + '\t' + 'KS Distance' + '\n')&#xA;    df = pd.read_csv(f, delimiter = ',', lineterminator = '\n', header = 0)&#xA;&#xA;    for m in range(10, 99):&#xA;        dc1 = []&#xA;        dc2 = []&#xA;        m1 = m/100&#xA;        m2 = m/100+0.009&#xA;        df2 = df.loc[(df['ECFP4'] &amp;gt;= m1) &amp;amp; (df['ECFP4'] &amp;lt; m2)]          #Work on different thresholds each time&#xA;    &#xA;        for i in range(0, df2.shape[0]):&#xA;            if len(df2.index[df2['id1'] == df2.iloc[i, 0]].tolist()) &amp;gt;= 5:&#xA;                dc1.append(df2.iloc[i, 0])&#xA;                dc2.append(df2.iloc[i, 1])&#xA;        for i in range(0, len(dc1)):&#xA;            dc1_l = []&#xA;            dc2_l = []&#xA;            df3 = df.loc[df['id1'] == dc1[i]]&#xA;            dc1_l = df3.iloc[:, 2].tolist()&#xA;            df4 = df.loc[df['id1'] == dc2[i]]&#xA;            dc2_l = df4.iloc[:, 2].tolist()&#xA;            x1 = re.findall(r&amp;quot;statistic=(.*)\,.*$&amp;quot;, str(ks_2samp(dc1_l, dc2_l)))&#xA;            f_out.write(str(dc1[i]) + '\t' + str(dc2[i]) + '\t' + str(m1) + '\t' + str(x1[0]) + '\n')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which so far has taken 10 hours to run and it has not yet passed the 0.1 threshold, so it is highly unpractical. Does anyone know of a better way to do this? Or can help me refine my code for faster and better results?&lt;/p&gt;&#xA;&lt;p&gt;I also tried to refine it and run it in R, but I get a vector memory exhausted (limit reached) error when I run it&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;install.packages(&amp;quot;ggplot2&amp;quot;)&#xA;library(ggplot2)&#xA;PS22_ECFP4 &amp;lt;- read.csv(&amp;quot;./PS22_ECFP4.csv&amp;quot;)&#xA;&#xA;df &amp;lt;- data.frame(matrix(ncol = 4, nrow = 0))&#xA;cols &amp;lt;- c(&amp;quot;id1&amp;quot;, &amp;quot;id2&amp;quot;, &amp;quot;Similarity&amp;quot;, &amp;quot;KS Distance&amp;quot;)&#xA;colnames(df) &amp;lt;- cols&#xA;&#xA;for (i in PS22_ECFP4) {&#xA;  sim_df &amp;lt;- PS22_ECFP4[PS22_ECFP4&lt;span class=&quot;math-container&quot;&gt;$id1 == i,]&#xA;  dist1 &amp;lt;- sim_df[ , 3]&#xA;  for (k in 1:nrow(sim_df)) {&#xA;    sim &amp;lt;- sim_df[k , 3]&#xA;    sim_r &amp;lt;- floor(sim * 100) / 100&#xA;    comp2 &amp;lt;- sim_df[k, 2]&#xA;    comp2_df &amp;lt;- PS22_ECFP4[PS22_ECFP4$id1==comp2,]&#xA;    dist2 &amp;lt;- comp2_df[ , 3]&#xA;    if (length(dist1) &amp;gt; 1 &amp;amp; length(dist2) &amp;gt; 1) {&#xA;      ks &amp;lt;- ks.test(dist1, dist2)&#xA;      df[nrow(df) + 1,] = c(i, comp2, sim_r, ks$&lt;/span&gt;statistic)&#xA;    }&#xA;  }&#xA;}&#xA;&#xA;write.csv(df, &amp;quot;./Metrics.csv&amp;quot;, row.names = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9466" LastActivityDate="2020-09-04T10:03:26.810" Title="How to refine this distribution comparison script" Tags="|r|python|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14299" PostTypeId="2" ParentId="14290" CreationDate="2020-09-03T22:28:34.873" Score="0" Body="&lt;p&gt;I couldn't get it to work with varied: &lt;code&gt;pcl2bed&lt;/code&gt;, &lt;code&gt;samtools&lt;/code&gt;, &lt;code&gt;bedtools&lt;/code&gt; commands. I think this biopython should work:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import gzip&#xA;from Bio import SeqIO&#xA;n=5&#xA;&#xA;exact_matches = {}&#xA;blat_qresult = SearchIO.read('blat_output.pslx', 'blat-psl', pslx=True)&#xA;for hit in blat_qresult:&#xA;    for f in hit.fragments:&#xA;        if f.hit_span == f.query_span:&#xA;            chrom = hit.id&#xA;            if chrom not in exact_matches:&#xA;                exact_matches[chrom] = []&#xA;            exact_matches[chrom].append(f.hit_range)&#xA;&#xA;&#xA;with gzip.open('/genomes/mm10.fa.gz', &amp;quot;rt&amp;quot;) as handle:&#xA;    for record in SeqIO.parse(handle, &amp;quot;fasta&amp;quot;):&#xA;        if record.id in exact_matches:&#xA;            for subseq in exact_matches[record.id]:&#xA;                print(record.seq[subseq[0]-n:subseq[1]+n])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="734" LastActivityDate="2020-09-03T22:28:34.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14300" PostTypeId="2" ParentId="14272" CreationDate="2020-09-04T07:09:20.300" Score="1" Body="&lt;p&gt;Using a fastq parser in Python (for instance the one provided by &lt;code&gt;mappy&lt;/code&gt;), it is rather straightforward to compute the sum of the lengths of the sequences present in a fastq file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ python3 -m pip install mappy  # Only if mappy is not installed yet&#xA;$&lt;/span&gt; python3 -c 'from mappy import fastx_read; print(sum(len(seq) for (_, seq, *_) in fastx_read(&amp;quot;SRR077487_2.filt.fastq.gz&amp;quot;)))'&#xA;2386161200&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;python3 -c '&amp;lt;insert some python code here&amp;gt;'&lt;/code&gt; uses the &lt;code&gt;-c&lt;/code&gt; option of &lt;code&gt;python3&lt;/code&gt; that tells Python to execute the provided code (instead of starting an interactive interpreter, or running code present in a file).&#xA;Here the code has two steps (separated by a &lt;code&gt;;&lt;/code&gt;, in a standard script, we could use a new line instead): first making the &lt;code&gt;fastx_read&lt;/code&gt; function available (because it is part of an optional module), second displaying the total length of the sequences.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;fastx_read(&amp;quot;path_to_a_file&amp;quot;)&lt;/code&gt; generates (name, sequence, quality) triplets when a fastq file is provided, or (name, sequence) pairs when a fasta file is provided.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;(_, seq, *_)&lt;/code&gt; syntax is a form of &amp;quot;tuple unpacking&amp;quot; where we store the second element (the sequence) in a &lt;code&gt;seq&lt;/code&gt; variable.&#xA;The &lt;code&gt;_&lt;/code&gt; in is just a way to ignore the first element in the triplet/pair. The &lt;code&gt;*_&lt;/code&gt; is a way to ignore whatever is after the second element in the triplet/pair.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;len(seq) for (_, seq, *_) in fastx_read(&amp;quot;SRR077487_2.filt.fastq.gz&amp;quot;)&lt;/code&gt; is an expression representing the lengths (obtained with the predefined &lt;code&gt;len&lt;/code&gt; function) of the sequences present in the file &lt;code&gt;&amp;quot;SRR077487_2.filt.fastq.gz&amp;quot;&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So the code uses the predefined &lt;code&gt;sum&lt;/code&gt; function to compute the sum of values that are the lengths of the sequences extracted from the fastq file, and this is displayed using the &lt;code&gt;print&lt;/code&gt; function.&lt;/p&gt;&#xA;&lt;p&gt;This should also work with either fasta or fastq format, and with or without gz compression.&lt;/p&gt;&#xA;&lt;p&gt;I tested more parsers on a slightly different problem here: &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/380/292&quot;&gt;https://bioinformatics.stackexchange.com/a/380/292&lt;/a&gt; and you can find inspiration with the other approaches proposed in the other answers there.&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2020-09-04T07:32:54.577" LastActivityDate="2020-09-04T07:32:54.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14301" PostTypeId="2" ParentId="14298" CreationDate="2020-09-04T08:16:16.970" Score="1" Body="&lt;p&gt;I am not sure how useful this answer is. But may be you could try this.&lt;/p&gt;&#xA;&lt;p&gt;For your python script:&lt;/p&gt;&#xA;&lt;p&gt;You use &lt;code&gt;f_out.write&lt;/code&gt; within a loop. I think usually this I/O operation is resource intensive. I guess the speed would improve significantly if you store the string in a variable and write the output only once outside the for loop like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;out_str = ''&#xA;for i in range(0, len(dc1)):&#xA;            dc1_l = []&#xA;            dc2_l = []&#xA;            df3 = df.loc[df['id1'] == dc1[i]]&#xA;            dc1_l = df3.iloc[:, 2].tolist()&#xA;            df4 = df.loc[df['id1'] == dc2[i]]&#xA;            dc2_l = df4.iloc[:, 2].tolist()&#xA;            x1 = re.findall(r&amp;quot;statistic=(.*)\,.*$&amp;quot;, str(ks_2samp(dc1_l, dc2_l)))&#xA;            out_str +=  (str(dc1[i]) + '\t' + str(dc2[i]) + '\t' + str(m1) + '\t' + str(x1[0]) + '\n')&#xA;&#xA;f_out.write(out_str)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Regarding the memory error in your R script, are you using 32bit build of R?&lt;/p&gt;&#xA;&lt;p&gt;Please see this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://web.mit.edu/%7Er/current/arch/i386_linux26/lib/R/library/base/html/Memory-limits.html#&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://web.mit.edu/~r/current/arch/i386_linux26/lib/R/library/base/html/Memory-limits.html#&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;They say &amp;quot;Under most 64-bit versions of Windows the limit for a 32-bit build of R is 4Gb: for the oldest ones it is 2Gb. The limit for a 64-bit build of R (imposed by the OS) is 8Tb.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion:&lt;/strong&gt; Please try using your python script, but use the write operation outside the loop.&lt;/p&gt;&#xA;" OwnerUserId="3263" LastActivityDate="2020-09-04T08:16:16.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14302" PostTypeId="2" ParentId="14298" CreationDate="2020-09-04T09:26:20.270" Score="0" Body="&lt;p&gt;If I was doing something like this in R (10G file), I'd use data.table&lt;/p&gt;&#xA;&lt;p&gt;I can't quite work out what you're R code is doing (does it do what you want it to?)&lt;/p&gt;&#xA;&lt;p&gt;I think the below might do what you want&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(data.table)&#xA;dat &amp;lt;- fread(&amp;quot;./PS22_ECFP4.csv&amp;quot;)&#xA;dat[,Similarity:=floor(ECFP4*100)/100]&#xA;dat[,'KS Distance':=lapply(1:nrow(dat),function(i)&#xA;  ks.test(dat[id1==dat&lt;span class=&quot;math-container&quot;&gt;$id1[i],ECFP4],dat[id2==dat$&lt;/span&gt;id2[i],ECFP4])$statistic  &#xA;)]&#xA;dat[,ECFP4:=NULL]&#xA;fwrite(dat,&amp;quot;./Metrics.csv&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Edit:&#xA;The script loops through the data, and for each row creates two subsets of the data matching id1 and id2. Then does the ks.test on the similarity columns of the two subsets.&lt;/p&gt;&#xA;" OwnerUserId="8919" LastEditorUserId="8919" LastEditDate="2020-09-04T10:03:26.810" LastActivityDate="2020-09-04T10:03:26.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14303" PostTypeId="1" CreationDate="2020-09-04T09:36:22.977" Score="7" ViewCount="1328" Body="&lt;p&gt;I've been playing with snakemake, which is really powerful, however, I'd like to create a 'unit test' for the whole pipeline. e.g. a way to call the pipeline that tests that all the rules work together, all the steps complete, etc.&lt;/p&gt;&#xA;&lt;p&gt;I assumed I could find some guides about how to do this, but I don't find any clear information about how to tests pipelines.&lt;/p&gt;&#xA;&lt;p&gt;Any advice or suggestions would be welcome.&lt;/p&gt;&#xA;" OwnerUserId="8875" LastActivityDate="2020-09-10T13:14:04.080" Title="How do I write tests for a snakemake pipeline?" Tags="|snakemake|reproducibility|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14304" PostTypeId="2" ParentId="14303" CreationDate="2020-09-04T10:37:57.517" Score="5" Body="&lt;p&gt;You can run CI on a snakemake workflow when provoding a minimal example dataset. Maybe an example from the snakemake-workflows repo will be helpful: &lt;a href=&quot;https://github.com/snakemake-workflows/dna-seq-varlociraptor&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/snakemake-workflows/dna-seq-varlociraptor&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9469" LastActivityDate="2020-09-04T10:37:57.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14305" PostTypeId="1" AcceptedAnswerId="14344" CreationDate="2020-09-04T10:52:49.273" Score="0" ViewCount="150" Body="&lt;p&gt;I did a QTL analysis with a panel of 7M SNPs, and want to analyze the enrichment of the significant qtl-SNPs in different genic regions (promoters, gene bodies, TFBS, etc.).&lt;/p&gt;&#xA;&lt;p&gt;A straightforward way to do it would be with an odds ratio (OR), using as 'background' the full 7M SNPs, but I prefer to compare with a background with a similar MAF distribution to that of the qtl-SNPs. For that, I'm thinking in two alternatives:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Take a single &lt;strong&gt;random sample of all SNPs&lt;/strong&gt; with size 10&lt;em&gt;n&lt;/em&gt; (&lt;em&gt;n&lt;/em&gt; is the number of qtl-SNPs), keeping the same MAF distribution of the qtl-SNPs results, as well as the same proportion of qtl-SNPs in the sample compared to the full 7M panel. Then, test enrichments using ONLY the random sample (with OR and Fisher's exact test).&lt;/li&gt;&#xA;&lt;li&gt;Do a &lt;strong&gt;bootstrapping&lt;/strong&gt;, taking multiple random samples (excluding qtl-SNPs), each of size &lt;em&gt;n&lt;/em&gt;, and keeping as well the same MAF distribution. For each iteration, get the OR between the full qtl-SNPs and the random sample. Then, the mean of the OR will be the point estimate and the quantiles the confidence intervals. For this approach, the problem would be to compute a &lt;em&gt;p&lt;/em&gt;-value.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Which approach seems to fit better? If the second one, how to quantify the significance? Can you suggest a better approach?&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="8879" LastActivityDate="2020-10-11T15:08:15.780" Title="Odds ratio and enrichment of SNPs in gene regions?" Tags="|statistics|snp|gwas|p-values|qtls|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14306" PostTypeId="2" ParentId="8758" CreationDate="2020-09-04T11:01:39.533" Score="1" Body="&lt;p&gt;JASPAR is only based on TF motifs, which are very short sequences and not enough to precisely tell you the true TF-motif relationships, especially when multiple TFs have complex logic gates regulating overlapped targets.&lt;/p&gt;&#xA;&lt;p&gt;An alternative way is to find some transcriptomic data upon perturbation of each TF. The drawbacks are that the redundancy of TFs will mask a lot of targets, and that the up/down regulated genes may not be direct targets but targets of targets (of targets of targets...).&lt;/p&gt;&#xA;&lt;p&gt;Another way is to find ChIP-seq data for TFs you care about, and hopefully you can find the right biological context. But still ChIP signal isn't exactly binding pattern, which in turn isn't exactly indicating target relationship.&lt;/p&gt;&#xA;&lt;p&gt;Overall, it's a challenging task to know what are the true targets of a TF.&lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2020-09-04T11:01:39.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14307" PostTypeId="2" ParentId="6841" CreationDate="2020-09-04T11:03:59.413" Score="0" Body="&lt;p&gt;There are two places:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.encodeproject.org/matrix/?type=Experiment&amp;amp;status=released&amp;amp;perturbed=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;encodeproject.org&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://screen.encodeproject.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;screen.encodeproject.org&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Details can be found in the &lt;a href=&quot;https://www.nature.com/immersive/d42859-020-00027-2/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENCODE3 papers&lt;/a&gt; published in July 2020&lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2020-09-04T11:03:59.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14308" PostTypeId="2" ParentId="5267" CreationDate="2020-09-04T11:07:14.913" Score="0" Body="&lt;p&gt;The codes are usually available at &lt;a href=&quot;https://github.com/ENCODE-DCC&quot; rel=&quot;nofollow noreferrer&quot;&gt;DCC's GitHub&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You can always email them if you have more questions.&lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2020-09-04T11:07:14.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14309" PostTypeId="2" ParentId="4736" CreationDate="2020-09-04T11:09:00.637" Score="1" Body="&lt;p&gt;You can use &lt;a href=&quot;http://screen.encodeproject.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;screen.encodeproject.org&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It collected a lot of data from multiple cell lines or tissues. And you can export tracks to UCSC browser or other browsers.&lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2020-09-04T11:09:00.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14312" PostTypeId="2" ParentId="2637" CreationDate="2020-09-04T15:42:38.657" Score="0" Body="&lt;p&gt;The current version of Seurat and Scanpy can both handle Drop-seq data. If you want to include more low-reads cells you can loosen the filtering thresholds in either of these packages. These two packages really can handle both low-read-high-cell data and high-read-low-cell data.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/s41586-020-2536-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;This ENCODE paper&lt;/a&gt; used Seurat to process both C1 Fluidigm data (&amp;gt;4000 genes per cell) and 10X data (&amp;gt;1000 genes per cell only).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S1934590920301569&quot; rel=&quot;nofollow noreferrer&quot;&gt;This human limb paper&lt;/a&gt; used Seurat to process Drop-seq data, although they didn't annotate the cell types well. I used Seurat to reprocess their data and it works much better than what the paper presents.&lt;/p&gt;&#xA;&lt;p&gt;I guess the key is parameter tuning.&lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2020-09-04T15:42:38.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14315" PostTypeId="2" ParentId="7019" CreationDate="2020-09-06T01:54:32.970" Score="2" Body="&lt;p&gt;(if I understand your situation correctly) &lt;code&gt;https://www.rdocumentation.org/packages/seqinr/versions/3.6-1/topics/read.alignment&lt;/code&gt; shows how to use the function &lt;strong&gt;read.alignment&lt;/strong&gt; which can take fasta msf etc. The docs provide the example' &lt;code&gt;read.alignment(file = system.file(&amp;quot;sequences/LTPs128_SSU_aligned_First_Two.fasta&amp;quot;,  package = &amp;quot;seqinr&amp;quot;), format = &amp;quot;fasta&amp;quot;, whole.header = TRUE)&lt;/code&gt; but you can use this code below (assumes those files are aligned) to go from reading the tree to getting the distances, producing the neighbor joining phylogenetic tree, and then plotting the tree.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(&amp;quot;Biostrings&amp;quot;)&#xA;library(&amp;quot;seqinr&amp;quot;)&#xA;library(&amp;quot;ape&amp;quot;)&#xA;library(phylogram)&#xA;library(&amp;quot;dendextend&amp;quot;)&#xA;&#xA;fasta.res &amp;lt;- read.alignment(file = &amp;quot;geneticAlignment.msf&amp;quot;, format = &amp;quot;fasta&amp;quot;)&#xA;fasta.res.dist.alignment = dist.alignment(msf.res, matrix = &amp;quot;identity&amp;quot;)&#xA;fasta.res.dist.alignment.nj = nj(fasta.res.dist.alignment)&#xA;plot(fasta.res.dist.alignment.nj, main = &amp;quot;from fasta files&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9311" LastActivityDate="2020-09-06T01:54:32.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14316" PostTypeId="1" AcceptedAnswerId="14320" CreationDate="2020-09-06T16:55:07.680" Score="1" ViewCount="203" Body="&lt;p&gt;I am a computer engineer and nowadays trying to grasp some concepts of Bioinformatics particularly, reference genomes and genomic variants. My aim is to find the effect of sequence features on variant types that occur in bacteria genomes. I chose Escherichia coli as the bacteria for my study since there seem to be more publication and data about it and its genomic sequences.&lt;/p&gt;&#xA;&lt;p&gt;When I made a research about variants, I found out that we need to have a reference genome for the comparison with other alleles in order to find variants. And, I even read that we can use genomic sequence of one bacteria as the reference but it won't represent the cohort in an accurate manner. I got the genomic sequence data for several strains of Escherichia coli bacteria that been sequenced before. However, most of these sequences are not complete ones. I filtered out the complete DNA genomes. When I checked the reports on these genomes, I learned that these are complete genomes but not reference genomes. Only the ones which have &lt;code&gt;RefSeq category: Reference Genome&lt;/code&gt; in their report are reference genomes for Escherichia coli.&lt;/p&gt;&#xA;&lt;p&gt;Learning that most of the data I got for different strains of Escherichia coli were not reference genomes, I tried to seek to find reference genomes and I got this website of NCBI &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genome/browse#!/prokaryotes/refseq_category:reference&quot; rel=&quot;nofollow noreferrer&quot;&gt;ref sequences for prokaryotes&lt;/a&gt;. This website shows only two reference sequences which match with what I have find in all sequencing data I got before. So, I want to ask two questions&lt;/p&gt;&#xA;&lt;p&gt;&lt;b&gt;First:&lt;/b&gt; How can I get a reference sequence for other strains of E. coli?&lt;/p&gt;&#xA;&lt;p&gt;&lt;b&gt;Second:&lt;/b&gt; If I cannot get a reference sequence for other strains, can I use complete genome data as the reference one? When I check the complete genome data, I cannot understand their order because there are other data for same strains that show loci of sequences but in the complete genome, I cannot see any order of the loci. Even, I cannot see data of some loci in the complete genome which blows my mind more.&lt;/p&gt;&#xA;&lt;p&gt;I know it may not be appropriate to ask two questions in a single post, however, these two questions seem to support each other. I will appreciate any of your help.&lt;/p&gt;&#xA;" OwnerUserId="9477" LastActivityDate="2021-01-02T22:40:58.827" Title="How can I get or create a reference genome for Bacteria?" Tags="|genome|assembly|variant-calling|reference-genome|bacteria|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14320" PostTypeId="2" ParentId="14316" CreationDate="2020-09-07T09:58:41.753" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;How can I get a reference sequence for other strains of E. coli?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You can use any sequence you want as reference for a variation analysis. It just depends on the question you want to answer.&lt;/p&gt;&#xA;&lt;p&gt;So to keep it simple, your Question is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What are the variant region from a fresh sequenced &lt;em&gt;E. coli&lt;/em&gt; K12 sample&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;p&gt;To know the differences from your known strain &lt;em&gt;E. coli&lt;/em&gt; K12. A reason for this could be that you work with the strain K12 through out several years. During your studies you changed the conditions for the bacteria, like adding antibiotics or change the growth media. So some of the &lt;em&gt;E. coli&lt;/em&gt; would adapt to the antibiotic and become resistant to it or they adapt better to the new media. This happens because some nucleotide changed. With each new experiment you used the best adapted colonies from the previouse one.&lt;/p&gt;&#xA;&lt;p&gt;So you sequence your bacteria and compare it with the &lt;em&gt;E. coli&lt;/em&gt; K12 strain. In this case this is your reference. If you use another strain, like SE15 or SE11, as reference, your result would not give you any answer for your question.&lt;/p&gt;&#xA;&lt;p&gt;If your used data and publications gives you the information which strain were used, then use the given strain as reference.&lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2020-09-07T09:58:41.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14321" PostTypeId="1" CreationDate="2020-09-08T02:23:12.717" Score="1" ViewCount="134" Body="&lt;p&gt;I am trying to collect gene expression data for the point of fitting gene regulatory networks. My background is primarily in computer science and so I am finding the biological literature a bit difficult to penetrate.&lt;/p&gt;&#xA;&lt;p&gt;Specifically, I want to fit two regulatory networks for several cancers in TCGA from tumor and normal samples. I have queries using the &lt;code&gt;R&lt;/code&gt; package &lt;code&gt;TCGABiolinks&lt;/code&gt; and was able to follow the tutorial &lt;a href=&quot;https://costalab.ukaachen.de/open_data/Bioinformatics_Analysis_in_R_2019/BIAR_D3/handout.html#2_tcga_data&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; up to the normalization part.&lt;/p&gt;&#xA;&lt;p&gt;So I have three questions;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there a standard way to normalize gene expression counts? If so, what is it, and is it available as a library in &lt;code&gt;R&lt;/code&gt;, &lt;code&gt;python&lt;/code&gt; or some other language?&lt;/li&gt;&#xA;&lt;li&gt;If not, can I just center and scale (z-transform) the samples? I am &lt;strong&gt;not&lt;/strong&gt; interested in differential expression, only quantifying &lt;strong&gt;expression&lt;/strong&gt; from counts.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9487" LastActivityDate="2020-11-16T20:01:25.190" Title="Standard Way to Preprocess Gene Expression?" Tags="|r|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14322" PostTypeId="1" CreationDate="2020-09-08T02:39:17.550" Score="2" ViewCount="71" Body="&lt;p&gt;Differentially methylated site is technically measured by each probe. However, CpG island within gene promoter is comprised of numerous probes. Do we have a consensus of how to determine the methylation level of CpG island using methylation level of probes within corresponding CpG island?&lt;/p&gt;&#xA;" OwnerUserId="4905" LastActivityDate="2020-10-08T17:00:40.073" Title="Defining methylation level of CpG island" Tags="|methylation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14323" PostTypeId="1" CreationDate="2020-09-08T07:57:36.700" Score="1" ViewCount="55" Body="&lt;p&gt;I am trying to identify somatic mutations on healthy subjects (specifically in WES samples). As of now, I have found a couple of papers who have done similar studies such as &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1919-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; where somatic mutations in healthy subjects are identified (using RNA-SEQ data). But in my case, I want to find somatic mutations from normal WES samples. For somatic mutations, we need both tumors and matched normal samples to identify somatic mutations but how to identify somatic mutations from normal samples only. Please suggest. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="2141" LastActivityDate="2020-09-08T07:57:36.700" Title="Somatic mutations for normal WES samples" Tags="|mutations|exome|somatic|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="14324" PostTypeId="2" ParentId="14322" CreationDate="2020-09-08T10:11:58.657" Score="1" Body="&lt;p&gt;In my experience, the most common approach to find DMR is to use the &lt;code&gt;DSS&lt;/code&gt; R package to call and merge DML into regions.&lt;/p&gt;&#xA;&lt;p&gt;DMC/DML (Differential Methylation Cytosine/Loci) is often calculated by comparing the counts of methylated and unmethylated cytosine at one position between samples.&lt;/p&gt;&#xA;&lt;p&gt;Steps are to first call DMLs with the &lt;code&gt;callDML()&lt;/code&gt; function. Then call DMRs from these first results with &lt;code&gt;callDMR()&lt;/code&gt;. Finally annotate the DMLs or DMRs using a &lt;code&gt;GFF&lt;/code&gt; file. From there you can look for DMR falling into CpG island. Some of the parameters that can be tuned are the &lt;strong&gt;minimum region length&lt;/strong&gt;, &lt;strong&gt;minimum number of differential CpG sites&lt;/strong&gt;, &lt;strong&gt;percentage of CpG sites required to be significant&lt;/strong&gt;. However I have never heard of what would be the standard values for these parameters. It may not be clearly established yet.&lt;/p&gt;&#xA;&lt;p&gt;So because the process of calling DMLs and DMRs need robust statistical testing, it is suggested to first call the differential methylated states and then annotate it.&lt;/p&gt;&#xA;" OwnerUserId="6273" LastEditorUserId="6273" LastEditDate="2020-09-08T16:22:22.213" LastActivityDate="2020-09-08T16:22:22.213" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14326" PostTypeId="2" ParentId="14135" CreationDate="2020-09-08T16:55:15.400" Score="3" Body="&lt;p&gt;I've been corresponding with NLM about this issue and I finally took the time to try out their suggestion (which personally I found hard to see between the lines and which is not a discrete solution, but rather a very time-consuming manual process containing false positives because they say that to get a formal and comprehensive response to my query:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Collecting the linkname will be a difficult task, that will take time and coordinate/check with relevant groups maintaining individual databases. Your patience will be greatly apprecired. [sic]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It took me a bit to decipher their suggestion on how to manually find all links, so I will share what I've learned.  There is (currently) no codified or formal means to obtain all possible link names and I infer that many links exist simply because they are utilized in the internal workings of their website.  I could be wrong about that, but suffice it to say that there are many undocumented links and they change constantly.&lt;/p&gt;&#xA;&lt;p&gt;You can get a list of filter items using each individual database's advanced search web interface, &amp;quot;most of which should be the linkname from that source database to other target database&amp;quot;.  (So this should result in all possible links among a series of false positives.)&lt;/p&gt;&#xA;&lt;p&gt;Here's how you do that.  Let's take the gene database as an example:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Go to the advanced search web page for that database, e.g. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene/advanced&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/gene/advanced&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;In the &amp;quot;Builder&amp;quot; section, select &amp;quot;Filter&amp;quot;&lt;/li&gt;&#xA;&lt;li&gt;Click &amp;quot;Show Index List&amp;quot; to the right of the search term field next to the select list where you selected &amp;quot;Filter&amp;quot; (you will see a multi-select list appear below the search field)&lt;/li&gt;&#xA;&lt;li&gt;Click &amp;quot;Next 200&amp;quot; until you find a selection in that new select list matching the database name followed by &amp;quot;all&amp;quot; and a number in parentheses, e.g. &lt;code&gt;gene all (29994947)&lt;/code&gt; and click to select that item &lt;a href=&quot;https://i.stack.imgur.com/eW5Vq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eW5Vq.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Click &amp;quot;Refresh index&amp;quot; at the lower right corner next to the multi-select list &lt;a href=&quot;https://i.stack.imgur.com/WZMmd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WZMmd.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The multi-select list will be repopulated and most of the items will be linknames (though you have to replace spaces with underscores in order for those linknames to work with the elink utility).&lt;/p&gt;&#xA;&lt;p&gt;Doing this for the gene database and then scrolling through just links to nuccore (and its alias &amp;quot;nucleotide&amp;quot; - ignoring est and gss), you will find:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;gene nuccore&lt;/li&gt;&#xA;&lt;li&gt;gene nuccore clust&lt;/li&gt;&#xA;&lt;li&gt;gene nuccore mgc&lt;/li&gt;&#xA;&lt;li&gt;gene nuccore pos&lt;/li&gt;&#xA;&lt;li&gt;gene nuccore refseqgene&lt;/li&gt;&#xA;&lt;li&gt;gene nuccore refseqrna&lt;/li&gt;&#xA;&lt;li&gt;gene nucleotide&lt;/li&gt;&#xA;&lt;li&gt;gene nucleotide clust&lt;/li&gt;&#xA;&lt;li&gt;gene nucleotide mgc&lt;/li&gt;&#xA;&lt;li&gt;gene nucleotide mgc url&lt;/li&gt;&#xA;&lt;li&gt;gene nucleotide pos&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If you look only at the linknames in the documentation or via einfo (by supplying the &lt;code&gt;gene&lt;/code&gt; database and looking at similar links to nuccore), you only get these links:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;gene_nuccore&lt;/li&gt;&#xA;&lt;li&gt;gene_nuccore_mgc&lt;/li&gt;&#xA;&lt;li&gt;gene_nuccore_refseqgene&lt;/li&gt;&#xA;&lt;li&gt;gene_nuccore_refseqrna&lt;/li&gt;&#xA;&lt;li&gt;gene_nucleotide_mgc_url&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So I believe that given this information and the ever-changing nature of these links, I believe I will allow the user the option to enter a linkname manually, if the set in the select list I generate via einfo does not contain the link they need.&lt;/p&gt;&#xA;" OwnerUserId="5063" LastActivityDate="2020-09-08T16:55:15.400" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14327" PostTypeId="1" CreationDate="2020-09-08T23:30:12.157" Score="0" ViewCount="679" Body="&lt;p&gt;I found &lt;a href=&quot;https://github.com/openvax/gtfparse/issues/19&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; GTF parser for Python. However, it does not work with Scallop's GTF file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr01_pilon_pilon       scallop transcript      168145  169166  1000    .       .       gene_id &amp;quot;gene.1.0&amp;quot;; transcript_id &amp;quot;gene.1.0.0&amp;quot;; RPKM &amp;quot;9.8016&amp;quot;; cov &amp;quot;27.2554&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    168145  169166  1000    .       .       gene_id &amp;quot;gene.1.0&amp;quot;; transcript_id &amp;quot;gene.1.0.0&amp;quot;; exon &amp;quot;1&amp;quot;;&#xA;chr01_pilon_pilon       scallop transcript      410672  444190  1000    +       .       gene_id &amp;quot;gene.3.0&amp;quot;; transcript_id &amp;quot;gene.3.0.1&amp;quot;; RPKM &amp;quot;0.5936&amp;quot;; cov &amp;quot;1.6506&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    410672  411205  1000    +       .       gene_id &amp;quot;gene.3.0&amp;quot;; transcript_id &amp;quot;gene.3.0.1&amp;quot;; exon &amp;quot;1&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    411326  411482  1000    +       .       gene_id &amp;quot;gene.3.0&amp;quot;; transcript_id &amp;quot;gene.3.0.1&amp;quot;; exon &amp;quot;2&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    444085  444190  1000    +       .       gene_id &amp;quot;gene.3.0&amp;quot;; transcript_id &amp;quot;gene.3.0.1&amp;quot;; exon &amp;quot;3&amp;quot;;&#xA;chr01_pilon_pilon       scallop transcript      678431  681781  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.1&amp;quot;; RPKM &amp;quot;0.4733&amp;quot;; cov &amp;quot;1.3162&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    678431  678822  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.1&amp;quot;; exon &amp;quot;1&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    678917  679068  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.1&amp;quot;; exon &amp;quot;2&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    680616  681781  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.1&amp;quot;; exon &amp;quot;3&amp;quot;;&#xA;chr01_pilon_pilon       scallop transcript      678431  685851  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.2&amp;quot;; RPKM &amp;quot;7.9200&amp;quot;; cov &amp;quot;22.0230&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    678431  678822  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.2&amp;quot;; exon &amp;quot;1&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    678917  679068  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.2&amp;quot;; exon &amp;quot;2&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    680616  681627  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.2&amp;quot;; exon &amp;quot;3&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    681874  681965  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.2&amp;quot;; exon &amp;quot;4&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    683200  683274  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.2&amp;quot;; exon &amp;quot;5&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    683744  683805  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.2&amp;quot;; exon &amp;quot;6&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    684640  684709  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.2&amp;quot;; exon &amp;quot;7&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    684842  685851  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.2&amp;quot;; exon &amp;quot;8&amp;quot;;&#xA;chr01_pilon_pilon       scallop transcript      681834  685851  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.3&amp;quot;; RPKM &amp;quot;1.1177&amp;quot;; cov &amp;quot;3.1080&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    681834  681965  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.3&amp;quot;; exon &amp;quot;1&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    683200  683274  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.3&amp;quot;; exon &amp;quot;2&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    683744  683805  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.3&amp;quot;; exon &amp;quot;3&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    684640  684709  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.3&amp;quot;; exon &amp;quot;4&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    684842  685851  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.3&amp;quot;; exon &amp;quot;5&amp;quot;;&#xA;chr01_pilon_pilon       scallop transcript      683032  685851  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.0&amp;quot;; RPKM &amp;quot;0.6001&amp;quot;; cov &amp;quot;1.6686&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    683032  683274  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.0&amp;quot;; exon &amp;quot;1&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    683744  683805  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.0&amp;quot;; exon &amp;quot;2&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    684640  684709  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.0&amp;quot;; exon &amp;quot;3&amp;quot;;&#xA;chr01_pilon_pilon       scallop exon    684842  685851  1000    +       .       gene_id &amp;quot;gene.6.0&amp;quot;; transcript_id &amp;quot;gene.6.0.0&amp;quot;; exon &amp;quot;4&amp;quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, the below code caused the following errors:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &amp;quot;/anaconda/envs/bioinf-scripts/lib/python3.7/site-packages/gtfparse/read_gtf.py&amp;quot;, line 103, in parse_gtf&#xA;    for df in chunk_iterator:&#xA;  File &amp;quot;/anaconda/envs/bioinf-scripts/lib/python3.7/site-packages/pandas/io/parsers.py&amp;quot;, line 1128, in __next__&#xA;    return self.get_chunk()&#xA;  File &amp;quot;/anaconda/envs/bioinf-scripts/lib/python3.7/site-packages/pandas/io/parsers.py&amp;quot;, line 1188, in get_chunk&#xA;    return self.read(nrows=size)&#xA;  File &amp;quot;/anaconda/envs/bioinf-scripts/lib/python3.7/site-packages/pandas/io/parsers.py&amp;quot;, line 1154, in read&#xA;    ret = self._engine.read(nrows)&#xA;  File &amp;quot;/anaconda/envs/bioinf-scripts/lib/python3.7/site-packages/pandas/io/parsers.py&amp;quot;, line 2059, in read&#xA;    data = self._reader.read(nrows)&#xA;  File &amp;quot;pandas/_libs/parsers.pyx&amp;quot;, line 881, in pandas._libs.parsers.TextReader.read&#xA;  File &amp;quot;pandas/_libs/parsers.pyx&amp;quot;, line 908, in pandas._libs.parsers.TextReader._read_low_memory&#xA;  File &amp;quot;pandas/_libs/parsers.pyx&amp;quot;, line 973, in pandas._libs.parsers.TextReader._read_rows&#xA;  File &amp;quot;pandas/_libs/parsers.pyx&amp;quot;, line 1105, in pandas._libs.parsers.TextReader._convert_column_data&#xA;  File &amp;quot;pandas/_libs/parsers.pyx&amp;quot;, line 1136, in pandas._libs.parsers.TextReader._convert_tokens&#xA;  File &amp;quot;pandas/_libs/parsers.pyx&amp;quot;, line 1232, in pandas._libs.parsers.TextReader._convert_with_dtype&#xA;ValueError: Integer column has NA values in column 3&#xA;&#xA;During handling of the above exception, another exception occurred:&#xA;&#xA;Traceback (most recent call last):&#xA;  File &amp;quot;/Users/lorencm/projects/bioinf-scripts/reduceIsoforms.py&amp;quot;, line 24, in &amp;lt;module&amp;gt;&#xA;    df = read_gtf(&amp;quot;data/hybrid.gtf&amp;quot;)&#xA;  File &amp;quot;/anaconda/envs/bioinf-scripts/lib/python3.7/site-packages/gtfparse/read_gtf.py&amp;quot;, line 211, in read_gtf&#xA;    restrict_attribute_columns=usecols)&#xA;  File &amp;quot;/anaconda/envs/bioinf-scripts/lib/python3.7/site-packages/gtfparse/read_gtf.py&amp;quot;, line 154, in parse_gtf_and_expand_attributes&#xA;    features=features)&#xA;  File &amp;quot;/anaconda/envs/bioinf-scripts/lib/python3.7/site-packages/gtfparse/read_gtf.py&amp;quot;, line 122, in parse_gtf&#xA;    raise ParsingError(str(e))&#xA;gtfparse.parsing_error.ParsingError: Integer column has NA values in column 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df = read_gtf(&amp;quot;data/hybrid.gtf&amp;quot;)&#xA;df_genes = df[df[&amp;quot;feature&amp;quot;] == &amp;quot;gene&amp;quot;]&#xA;print(df_genes)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thank you in advance,&lt;/p&gt;&#xA;" OwnerUserId="2477" LastActivityDate="2020-09-08T23:30:12.157" Title="GTF parser for Python" Tags="|python|gtf|parsing|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14328" PostTypeId="1" CreationDate="2020-09-09T05:18:53.067" Score="0" ViewCount="128" Body="&lt;p&gt;I am trying to use &lt;code&gt;optitype&lt;/code&gt; on an HPC without docker support or sudo privileges. A search brought me &lt;a href=&quot;https://github.com/researchapps/optitype&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/researchapps/optitype&lt;/a&gt; and &lt;a href=&quot;https://github.com/nf-core/hlatyping&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nf-core/hlatyping&lt;/a&gt;. I tried bootstrapping the singularity image as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;singularity build optitype.simg docker://fred2/optitype&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, this does not seem to run as expected. Could someone with experience bootstrapping docker files onto singularity suggest how I could change this? How do the commands change when doing a bootstrap? I would appreciate your help!&lt;/p&gt;&#xA;" OwnerUserId="9499" LastEditorUserId="57" LastEditDate="2020-09-09T16:51:35.920" LastActivityDate="2023-02-02T10:04:55.623" Title="Optitype for Singularity" Tags="|docker|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14329" PostTypeId="1" CreationDate="2020-09-09T06:29:15.383" Score="0" ViewCount="283" Body="&lt;p&gt;I am trying to run a &lt;code&gt;blatn&lt;/code&gt; command from both my laptop and within a google colab notebook. I am not sure why, but this command runs properly once and then, on the second execution, it gets stuck for hours without a reply.&lt;/p&gt;&#xA;&lt;p&gt;Is it possible that NCBI blocks my IP? I've tried also to run &lt;code&gt;web_blast.pl&lt;/code&gt; and this didn't work either.&lt;/p&gt;&#xA;&lt;p&gt;biopython:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.Blast import NCBIWWW&#xA;&#xA;fasta = '&amp;gt;seq0\n'+'AACCACCTGACGGGCTCTGT'&#xA;blast_res = NCBIWWW.qblast('blastn', 'nt', fasta)&#xA;print(blast_res.read())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;web_blast.pl&lt;/code&gt; command:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;web_blast.pl blastn nt input.fa &amp;gt; blast.out&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note, if this is due to throttling by NCBI, I am not sure how do they expect people to query their web service if they block you after one call.&lt;/p&gt;&#xA;" OwnerUserId="734" LastActivityDate="2020-09-09T15:02:17.897" Title="Does NCBI's blast API block my IP?" Tags="|blast|phylogenetics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14332" PostTypeId="2" ParentId="14329" CreationDate="2020-09-09T15:02:17.897" Score="1" Body="&lt;p&gt;You should check the &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&amp;amp;PAGE_TYPE=BlastDocs&amp;amp;DOC_TYPE=DeveloperInfo&quot; rel=&quot;nofollow noreferrer&quot;&gt;User Guide&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The NCBI BLAST servers are a shared resource. We give priority to&#xA;interactive users. In order to ensure availability of the service to&#xA;the entire community, we may limit searches for some high volume&#xA;users. Interactive users of the NCBI webpages through a web browser&#xA;should not encounter problems. We will move searches of users who&#xA;submit more than 100 searches in a 24 hour period to a slower queue,&#xA;or, in extreme cases, will block the requests. To avoid problems, API&#xA;users should comply with the following guidelines:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Do not contact the server more often than once every 10 seconds.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do not poll for any single RID more often than once a minute.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use the URL parameter email and tool, so that the NCBI can contact you&#xA;if there is a problem.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Run scripts weekends or between 9 pm and 5 am Eastern time on weekdays&#xA;if more than 50 searches will be submitted.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2020-09-09T15:02:17.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14334" PostTypeId="1" CreationDate="2020-09-10T04:05:26.433" Score="0" ViewCount="262" Body="&lt;p&gt;How can I import a biom file?&lt;/p&gt;&#xA;&lt;p&gt;When I run the following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(phyloseq)&#xA;import_biom(&#xA;    &amp;quot;/Users/name/Dropbox/*****.biom&amp;quot;, &#xA;    parseFunction=parse_taxonomy_greengenes, &#xA;    parallel=F)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get the following error message:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;biom either as JSON (BIOM-v1) or HDF5 (BIOM-v2). Check file path, file&#xA;name, file itself, then try agai&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="8895" LastEditorUserId="3694" LastEditDate="2020-09-11T21:24:31.927" LastActivityDate="2020-09-11T21:24:31.927" Title="Help importing a biom file into R" Tags="|r|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14335" PostTypeId="1" CreationDate="2020-09-10T12:28:45.480" Score="0" ViewCount="40" Body="&lt;p&gt;I am relatively new to SNP analysis. Is there a database to find annotation about the Mendelian inheritance of SNPs?&lt;/p&gt;&#xA;&lt;p&gt;I have a small list of nsSNPs (non-synonymous SNPs) and I need to find how they are they passed to from parents to offspring.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="9508" LastEditorUserId="3694" LastEditDate="2020-09-11T09:36:05.560" LastActivityDate="2020-09-11T09:36:05.560" Title="Inheritance annotation for nsSNPs" Tags="|sequence-annotation|snp|ngs|variants|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14336" PostTypeId="2" ParentId="14303" CreationDate="2020-09-10T13:14:04.080" Score="2" Body="&lt;p&gt;This is what I do when I want to check whether a program runs from start to finish and produces the expected output on test data. &lt;a href=&quot;https://github.com/dariober/fxstat&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; I used it to test a C program but the idea can be applied also to a snakemake pipeline as in your case.&lt;/p&gt;&#xA;&lt;p&gt;The idea is pretty simple really: use python's &lt;code&gt;subprocess&lt;/code&gt; module to run the pipeline then check the results using the &lt;code&gt;unittest&lt;/code&gt; module (or your favourite unit test package).&lt;/p&gt;&#xA;&lt;p&gt;For example, save this in &lt;code&gt;test.py&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;import unittest&#xA;import os&#xA;import subprocess as sp&#xA;&#xA;class Tester(unittest.TestCase):&#xA;&#xA;    def testPipelineMakesStuff(self):&#xA;        # Run the pipeline&#xA;        p= sp.Popen('snakemake [some parameters]', shell=True, stdout= sp.PIPE, stderr= sp.PIPE)&#xA;        &#xA;        # Optionally, get stdout and stderr&#xA;        stdout, stderr= p.communicate()&#xA;&#xA;        # Check exits code and other expected output            &#xA;        self.assertEqual(0, p.returncode)&#xA;        self.assertTrue(os.path.isfile('some.pipeline.output.txt'))&#xA;&#xA;        ...Maybe read some.pipeline.output.txt and check it looks ok&#xA;&#xA;    def testPipelineReturnsWithError(self):&#xA;        p= sp.Popen('snakemake [some parameters]', shell=True, stdout= sp.PIPE, stderr= sp.PIPE)&#xA;        stdout, stderr= p.communicate()&#xA;        self.assertTrue(p.returncode != 0)&#xA;        self.assertTrue('Something went wrong' in stderr.decode())&#xA;    &#xA;    def ...more test cases...&#xA;&#xA;if __name__ == '__main__':&#xA;    unittest.main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Run this script as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; python test.py &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and check all tests pass ok. Of course, you can embed &lt;code&gt;python test.py&lt;/code&gt; in travis or other continuous integration system as I've done &lt;a href=&quot;https://github.com/dariober/fxstat/blob/master/.travis.yml&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="339" LastActivityDate="2020-09-10T13:14:04.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14337" PostTypeId="2" ParentId="14335" CreationDate="2020-09-10T13:36:19.920" Score="2" Body="&lt;p&gt;If you only have a small number of genes, &lt;a href=&quot;https://omim.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;OMIM&lt;/a&gt; is a very easy way to do this.&lt;/p&gt;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MPB4W.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MPB4W.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you have too many to do manually, I suspect there is an easy way to query lists, although I'm not sure exactly the way myself. Perhaps someone can add a new answer if so.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2020-09-10T13:36:19.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14338" PostTypeId="1" AcceptedAnswerId="14339" CreationDate="2020-09-10T19:37:24.870" Score="1" ViewCount="3912" Body="&lt;p&gt;I have a total RNAseq dataset that I aligned using STAR producing BAM files (sorted by coordinates). I am now trying to get counts for the lncRNA sequences using htseq-count, with the command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;htseq-count -m intersection-nonempty -f bam -i gene_id -r pos -s reverse ./OVCA429_Aligned.sortedByCoord.out.bam ./GRCh38_latest_genomic.gtf &amp;gt; ./OVCA429.counts.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[E::idx_find_and_load] Could not retrieve index file for './OVCA429_587680_1_1.rnaseq.fastqAligned.sortedByCoord.out.bam'&#xA;100000 GFF lines processed.&#xA;200000 GFF lines processed.&#xA;300000 GFF lines processed.&#xA;400000 GFF lines processed.&#xA;...(truncated output here)...&#xA;3775001 GFF lines processed.&#xA;[E::idx_find_and_load] Could not retrieve index file for './OVCA429_587680_1_1.rnaseq.fastqAligned.sortedByCoord.out.bam'&#xA;100000 alignment records processed.&#xA;200000 alignment records processed.&#xA;300000 alignment records processed.&#xA;...(truncated output here)...&#xA;&#xA;Output text file:&#xA;__no_feature    20368875&#xA;__ambiguous 0&#xA;__too_low_aQual 0&#xA;__not_aligned   0&#xA;__alignment_not_unique  2634439&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've read through the htseq-count literature but it never asks to include an index file, therefore i'm not sure how to deal with this problem. Any help/ suggestions would be welcome! Thank you, in advance.&lt;/p&gt;&#xA;" OwnerUserId="9509" LastEditorUserId="57" LastEditDate="2020-09-10T20:06:51.593" LastActivityDate="2022-01-12T20:49:30.430" Title="Error using htseq-count: Could not retrieve index file" Tags="|rna-seq|sam|errors|featurecounts|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14339" PostTypeId="2" ParentId="14338" CreationDate="2020-09-10T20:09:42.787" Score="0" Body="&lt;p&gt;The problem seems to be missing index files of the bam files (therefore the error message that the index file is missing). You should be able to generate those with &lt;code&gt;samtools&lt;/code&gt;, e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools index ./OVCA429_587680_1_1.rnaseq.fastqAligned.sortedByCoord.out.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="57" LastActivityDate="2020-09-10T20:09:42.787" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14340" PostTypeId="2" ParentId="14147" CreationDate="2020-09-11T00:11:29.120" Score="-1" Body="&lt;p&gt;Searching directly for a reverse transcriptase may find recently incorporated retroviruses.&lt;/p&gt;&#xA;" OwnerUserId="9510" LastActivityDate="2020-09-11T00:11:29.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14341" PostTypeId="1" AcceptedAnswerId="14355" CreationDate="2020-09-11T07:18:00.283" Score="0" ViewCount="105" Body="&lt;p&gt;I'm trying to run the WDL tasks from the &lt;a href=&quot;https://github.com/broadinstitute/gtex-pipeline&quot; rel=&quot;nofollow noreferrer&quot;&gt;GTEx RNA-Seq pipeline&lt;/a&gt; with Cromwell using AWS Batch as a backend.&lt;/p&gt;&#xA;&lt;p&gt;I store the STAR alignment index on an Elastic Block Store since my Cromwell server instance lacks sufficient disk space. The EBS Block is mounted to my Cromwell server at &lt;code&gt;/data&lt;/code&gt;. The STAR index is located at &lt;code&gt;/data/genomic-references/rnaseq/star_index_oh50&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Try as I might, I cannot get Cromwell to run the STAR task correctly.&lt;/p&gt;&#xA;&lt;p&gt;My JSON inputs file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; {&#xA;   &amp;quot;rnaseq_pipeline_fastq_workflow.prefix&amp;quot;: &amp;quot;s3://mybucket/test/CL100148068&amp;quot;,&#xA;   &amp;quot;rnaseq_pipeline_fastq_workflow.star.fastq1&amp;quot;: &amp;quot;s3://anotherbucket/genotypedata/sample1.fq.gz&amp;quot;,&#xA;   &amp;quot;rnaseq_pipeline_fastq_workflow.star.star_index&amp;quot;: &amp;quot;/data/genomic-references/rnaseq/star_index_oh50/&amp;quot;,&#xA;   &amp;quot;rnaseq_pipeline_fastq_workflow.star.memory&amp;quot;: &amp;quot;32&amp;quot;,&#xA;   &amp;quot;rnaseq_pipeline_fastq_workflow.star.num_threads&amp;quot;: &amp;quot;4&amp;quot;,&#xA;   &amp;quot;rnaseq_pipeline_fastq_workflow.star.disk_space&amp;quot;: &amp;quot;200&amp;quot;,&#xA;   &amp;quot;rnaseq_pipeline_fastq_workflow.star.num_preempt&amp;quot;: &amp;quot;0&amp;quot; &#xA; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Error message:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tar: /data/genomic-references/rnaseq/star_index_oh50/: Cannot open: No such file or directory&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that &lt;code&gt;/data/genomic-references/rnaseq/star_index_oh50/&lt;/code&gt; does exist and is readable.&lt;/p&gt;&#xA;&lt;p&gt;The relevant command in the GTEx container is &lt;a href=&quot;https://github.com/broadinstitute/gtex-pipeline/blob/b53b734a9b096caed237952b34cbce88b38485bd/rnaseq/star.wdl#L69&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tar -xvvf ${star_index} -C star_index --strip-components=1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If the files are stored locally, then why does &lt;code&gt;tar&lt;/code&gt; fail to wrap the directory?&lt;/p&gt;&#xA;" OwnerUserId="3355" LastActivityDate="2020-09-14T02:41:15.123" Title="Pointing Cromwell to local EBS mounted storage on AWS Batch" Tags="|rna-seq|genomics|wdl|gtex|aws|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14342" PostTypeId="1" AcceptedAnswerId="14416" CreationDate="2020-09-11T09:04:50.863" Score="2" ViewCount="884" Body="&lt;p&gt;I am using the data from a proteomics study were the data was log2 transformed and then a median normalization was applied. The data was normalized by groups of conditions (normal, mutant), not for all the arrays at the same time, so the median for the normal group is slightly higher than for the mutant group.&#xA;I am wondering if this is correct or should I normalized the data en general, I mean taking into account all the arrays and normalizing all at the same time so the 2 groups have the same median.&lt;/p&gt;&#xA;&lt;p&gt;Regards&lt;/p&gt;&#xA;" OwnerUserId="9010" LastActivityDate="2020-09-23T17:41:42.577" Title="median normalization for proteomics" Tags="|normalization|proteomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14343" PostTypeId="2" ParentId="14256" CreationDate="2020-09-11T14:50:43.193" Score="2" Body="&lt;p&gt;Staff from rcsb kindly reminds me the &lt;a href=&quot;https://search.rcsb.org/index.html#pagination&quot; rel=&quot;nofollow noreferrer&quot;&gt;solution&lt;/a&gt; - add the following lines in the json query.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;request_options&amp;quot;: {&#xA;&amp;quot;return_all_hits&amp;quot;: true&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9344" LastActivityDate="2020-09-11T14:50:43.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14344" PostTypeId="2" ParentId="14305" CreationDate="2020-09-11T15:05:47.680" Score="3" Body="&lt;p&gt;The first approach only address the question of how likely are you to end up with the observed over-representation given the MAF distribution.&lt;/p&gt;&#xA;&lt;p&gt;My suggestion is to use the second approach, but I am not sure if you would call it bootstrap. Bootstrap in general means sampling with replacement to estimate the uncertainty of a parameter, in this case, OR. So even if you resample your full dataset, you get an estimate of how variable your enrichment is, not how likely you are to observe it by chance. so just name it permutation or sampling.&lt;/p&gt;&#xA;&lt;p&gt;This is what I think might work&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Calculate your true OR, qtl snps versus all others for say TFBS&lt;/li&gt;&#xA;&lt;li&gt;Divide your snps according to MAF into 10 bins / stratas&lt;/li&gt;&#xA;&lt;li&gt;Within each bin, we permute the label whether the snp is a qtl&lt;/li&gt;&#xA;&lt;li&gt;Calculate OR of selection / others for TFBS&lt;/li&gt;&#xA;&lt;li&gt;Repeat 1-3 N times to collect the permutated OR&lt;/li&gt;&#xA;&lt;li&gt;p value would be number of (permutated OR &amp;gt; observed OR+1)/(N+1)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Below is an example implementation in R, first we have 1e5 signficant snps and they are 5 times as likely to be in TFBS:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;snp_is_qtl = rep(c(1,0),c(0.1e6,6.9e6))&#xA;snp_is_TFBS = c(sample(0:1,0.1e6,replace=TRUE,prob=c(0.95,0.05)),&#xA;sample(0:1,6.9e6,replace=TRUE,prob=c(0.99,0.01)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We have the MAF for all snps, with those in qtl being different from others and put them in bins.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;snp_MAF = c(runif(0.1e6,0.4,0.8),runif(6.9e6,0.1,0.9))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We sort the snps according to MAF to make it easier:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;o = order(snp_MAF)&#xA;snp_is_qtl = snp_is_qtl[o]&#xA;snp_is_TFBS = snp_is_TFBS[o]&#xA;snp_MAF = snp_MAF[o]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And we create the bin:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bin_MAF = cut(snp_MAF,10)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Lastly a function to calculate OR:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;calOR = function(snps,TBFS){&#xA;tab = table(snps,TBFS)&#xA;log(tab[1,1])+log(tab[2,2]) -log(tab[1,2]) -log(tab[2,1])&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So our observed OR (in log) is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;obs_OR = calOR(snp_is_qtl,snp_is_TFBS)&#xA;obs_OR&#xA;[1] 1.656859&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now we collect the OR under permutation:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;N = 100&#xA;permutated_OR = sapply(1:100,function(i){&#xA;print(i)&#xA;perm_qtl = unlist(tapply(snp_is_qtl,bin_MAF,sample))&#xA;calOR(perm_qtl,snp_is_TFBS)&#xA;})&#xA;&#xA;hist(permutated_OR)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sK6zH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sK6zH.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Your observed OR way exceeds the permuted, even after controlling for MAF. Note the computation is a bit heavy because of the large number of snps. So it makes sense to calculate all the ORs (i.e TFBS, promoters..) with each permutation&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-09-11T15:05:47.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14346" PostTypeId="1" CreationDate="2020-09-12T01:24:04.633" Score="0" ViewCount="456" Body="&lt;p&gt;In my intro to bioinformatics course, we mentioned that TopHat2 and HISAT2 will both try to align as many reads as possible to the reference genome (TopHat2 has been superseded by HISAT2). For the reads that were not mapped, these probably cross exon boundaries. To solve this issue, both TopHat2 and HISAT2 chop up the unmapped reads into around 25bp fragments. Then:&#xA;&amp;quot;TopHat2 tries to align many fragments from each read using all possible exons as the search space. HISAT2 focuses on one fragment from each read, and once that fragment has been anchored it only searches locally.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;My question is about that last part &amp;quot;once that fragment has been anchored, [HISAT2] only searches locally.&amp;quot; Isn't it possible that a given fragment will map to many different regions of the genome? So how is it possible that it will be &amp;quot;anchored&amp;quot;? Are we going to anchor it multiple times and see which anchor makes most sense or what exactly?&lt;/p&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="1157" LastEditDate="2021-04-12T09:15:51.903" LastActivityDate="2021-04-12T09:15:51.903" Title="TopHat2 versus HISAT2 inner workings" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14347" PostTypeId="1" CreationDate="2020-09-12T10:18:19.523" Score="1" ViewCount="320" Body="&lt;p&gt;I have just started to learn bioinformatics and pangenomics. So if this question seems to you pretty basic then I apologize in advance.&lt;/p&gt;&#xA;&lt;p&gt;As we know, NCBI for the genome database, there are two kinds of sequences available RefSeq and Genbank. I have read the differences in both RefSeq and Genbank. But I was just curious which one would be preferred for pangenome studies? For example, phylogeny built by RefSeq sequences would be different from GenBank sequences? As I read, RefSeq is already curated, annotated and contaminants are removed. Will it affect the phylogeny that we will build after using any pangenome pipeline?&lt;/p&gt;&#xA;&lt;p&gt;I am particularly interested in non-synonymous mutations (frameshifts or stop codons inside coding sequences). Do RefSeq curators also remove this kind of unusual errors or mutations? If yes then it would be difficult to study such mutations in RefSeq sequences.&lt;/p&gt;&#xA;&lt;p&gt;Somebody may say a combination of both Genbank and RefSeq would be good, after removing the duplicate one. Which one should I remove RefSeq or Genbank?.&lt;/p&gt;&#xA;&lt;p&gt;Regards&lt;/p&gt;&#xA;" OwnerUserId="3626" LastActivityDate="2020-09-12T10:18:19.523" Title="GCF VS GCA or combination of both for pangenomic studies" Tags="|phylogeny|snp|genbank|refseq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14348" PostTypeId="1" AcceptedAnswerId="14404" CreationDate="2020-09-12T15:15:30.333" Score="3" ViewCount="3745" Body="&lt;p&gt;One of the advantages of paired end sequencing over single end is that it doubles the amount of data. Another &lt;strong&gt;supposed&lt;/strong&gt; advantage is that it leads to more accurate reads because if say Read 1 (see picture below) maps to two different regions of the genome, Read 2 can be used to help determine which one of the two regions makes more sense. However, I don't understand how this second advantage pertains only to paired end sequencing. With single-end sequencing, couldn't you just use the read that comes right after read 1 to essentially get a longer net read?&#xA;&lt;a href=&quot;https://i.stack.imgur.com/xqJYt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xqJYt.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2020-09-21T19:27:49.040" Title="Advantages of paired-end sequencing compared to single end" Tags="|rna-seq|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="14349" PostTypeId="1" CreationDate="2020-09-12T15:20:25.580" Score="1" ViewCount="146" Body="&lt;p&gt;I am trying to read a .sam file into a single string in C++. My end goal is to read this string into a vector, where tabs indicate separations between elements of the vector. After making a vector containing all of the data from the sam file, I will separate the relevant data types (chromosome name, locus, etc.) into their own vectors based on position in the master vector.&lt;/p&gt;&#xA;&lt;p&gt;The sam file was pre-fetched from the SRA database, split into paired-end reads, and aligned to the reference genome using the following Linux commands:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;prefetch --output-directory ./ SRR&#xA;fastq-dump --defline-seq '@&lt;span class=&quot;math-container&quot;&gt;$sn[_$&lt;/span&gt;rn]/$ri' --split-files SRR.sra&#xA;bwa mem -M reference_genome.fasta SRR_1.fastq SRR_2.fastq &amp;gt; SRR.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To make the file easier to read in C++, I altered the .sam file in Linux by removing the header, replacing newline characters with tabs, and replacing gaps between &amp;quot;columns&amp;quot; with true tabs rather than spaces. Since the number of spaces between each column was variable, I accomplished this last part by replacing successively smaller numbers of spaces (12, 11, 10, etc) with tabs. Here is the Linux command line code for the three steps listed above:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -v '^@' SRR.sam &amp;gt; headerless_SRR.sam&#xA;&#xA;# Condense sam files into a single line&#xA;sed 's/\n/\t/g' headerless_SRR.sam &amp;gt; condensed_SRR.sam&#xA;&#xA;# Replace all spaces in sam files with tabs. Since some columns have multiple spaces, the spaces must be replaced in progressively smaller chunks&#xA;sed 's/            /\t/g' condensed_SRR791885.sam &amp;gt; tch_SRR791885.sam     # &amp;quot;tcd&amp;quot; is tabbed, condensed, headerless&#xA;sed 's/           /\t/g' tch_SRR.sam &amp;gt; tch2_SRR791885.sam     # Repeat this line with progressively smaller numbers of spaces until all spaces are replaced with single tabs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To open the resulting file and read it into a string, I created the following C++ script, compiled it using the g++ command in Linux, and ran the resulting .out file using &amp;quot;./&amp;quot;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;fstream&amp;gt;&#xA;#include &amp;lt;sstream&amp;gt;&#xA;#include &amp;lt;vector&amp;gt;&#xA;#include &amp;lt;string&amp;gt;&#xA;&#xA;int main(){&#xA;    // Creates an input file stream opbject and fills it with data from the sam file&#xA;    std::ifstream sam_file;&#xA;    sam_file.open(&amp;quot;Sample_Sam.txt&amp;quot;, std::ifstream::in);&#xA;    // Reads the first character in the sam file and stores it as a &amp;quot;character&amp;quot; object&#xA;    char sam_character = sam_file.get();&#xA;    // Creates a string into which characters can be pushed&#xA;    std::string sam_string;&#xA;    // While characters are still detected in the sam file, add the characters to a string and move to the next character in the file&#xA;    while(sam_file.good()){&#xA;        sam_string.push_back(sam_character);&#xA;        sam_character = sam_file.get();&#xA;    }&#xA;    sam_file.close();&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I run the C++ script on a dummy sam file containing random blocks of data, it runs fine and I can even output the resulting string to &amp;quot;std::cout&amp;quot; or another file. However, when I run it on a &amp;quot;tch_SRR&amp;quot; sam file, the &amp;quot;./&amp;quot; file hangs without creating an output or anything. Does anyone know how I could fix this?&lt;/p&gt;&#xA;" OwnerUserId="9118" LastActivityDate="2020-09-12T15:20:25.580" Title="How do I Read a Sam File into a String Using ifstream in C++?" Tags="|sam|c++|grep|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14351" PostTypeId="1" AcceptedAnswerId="14354" CreationDate="2020-09-13T04:27:10.437" Score="0" ViewCount="223" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/461091/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioStars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am trying to recreate &lt;a href=&quot;https://www.nature.com/articles/s41586-019-1467-x/figures/6&quot; rel=&quot;nofollow noreferrer&quot;&gt;this heatmap&lt;/a&gt;. How would I compare the four variables: Ly49+/- and MOG/MOGSP to get a single heatmap? Thank you for helping me through this.&lt;/p&gt;&#xA;&lt;p&gt;I need to compare both Ly49+ versus Ly49− and MOG versus MOG plus SP. Column data in at the bottom. The function I used for DESeq2 is,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dds = DESeqDataSetFromMatrix(countData=countData,&#xA;&#xA;colData=colData,&#xA;&#xA;design= ~ cell_treatment)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am not sure if the design is right. For the results, how should I write the contrasts? I am very confused and would really appreciate any help!&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;DataFrame with 11 rows and 2 columns&#xA;&#xA;cell treatment&#xA;&#xA;&amp;lt;character&amp;gt; &amp;lt;character&amp;gt;&#xA;&#xA;MOGSP_2 SPtetramer+CD8+TCell MOGSP&#xA;&#xA;MOGSP_3 SPtetramer+CD8+TCell MOGSP&#xA;&#xA;MOG_3 SPtetramer+CD8+TCell MOG&#xA;&#xA;MOG_4 SPtetramer+CD8+TCell MOG&#xA;&#xA;MOG_5 SPtetramer+CD8+TCell MOG&#xA;&#xA;Ly49N_1 Ly49-CD8+TCell MOGSP&#xA;&#xA;Ly49P_1 Ly49+CD8+TCell MOGSP&#xA;&#xA;Ly49N_2 Ly49-CD8+TCell MOGSP&#xA;&#xA;Ly49P_2 Ly49+CD8+TCell MOGSP&#xA;&#xA;Ly49N_3 Ly49-CD8+TCell MOGSP&#xA;&#xA;Ly49P_3 Ly49+CD8+TCell MOGSP&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thank you!!&lt;/p&gt;&#xA;" OwnerUserId="9515" LastEditorUserId="73" LastEditDate="2020-09-14T21:30:10.257" LastActivityDate="2020-09-14T21:30:10.257" Title="Setting Contrast for DESeq2 results" Tags="|rna-seq|deseq2|design|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14352" PostTypeId="1" CreationDate="2020-09-13T06:24:17.600" Score="0" ViewCount="62" Body="&lt;p&gt;If there is a need to perform structural alignment of different parts of distant homologs, which program one should use?&lt;/p&gt;&#xA;&lt;p&gt;Since distant homologs often have significant structural changes, meaning the assumption of a similar physico-chemical environment for aligned residues would be violated.&lt;/p&gt;&#xA;&lt;p&gt;Any program/tools or suggested reference will be a great help.&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="3656" LastActivityDate="2020-09-19T23:23:57.570" Title="What programs account for structural alignment of different parts of distant homologs which have significant structural differences?" Tags="|sequence-alignment|proteins|protein-structure|3d-structure|structural-biology|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14353" PostTypeId="1" AcceptedAnswerId="14357" CreationDate="2020-09-13T14:01:42.993" Score="0" ViewCount="1593" Body="&lt;p&gt;When correcting my data for a batch effect using removeBatchEffect, some of the gene expression values become negative.&lt;/p&gt;&#xA;&lt;p&gt;When searching for differentially expressed genes, I do not use the data above, but rather model the batch using deseq2 (design=~Batch + Condition).&lt;/p&gt;&#xA;&lt;p&gt;However, I started worrying. When DESeq2 introduces a batch into the model, does it allow for negative values &amp;quot;behind the scenes&amp;quot;?&lt;/p&gt;&#xA;&lt;p&gt;If it does, I do not understand how that can make sense, in the context of RNA-Seq data.&lt;/p&gt;&#xA;" OwnerUserId="6317" LastActivityDate="2020-09-14T07:18:04.920" Title="Details of DESeq2 modeling a batch effect" Tags="|deseq2|limma|batch-effects|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14354" PostTypeId="2" ParentId="14351" CreationDate="2020-09-13T18:34:53.213" Score="3" Body="&lt;p&gt;Your table is not very clear. If I understood it correctly, there are three groups of t-cells, SPtetramer+, Ly49- and Ly49+ and within it there is a MOGSP vs MOG comparison. I would set colData like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;                   name             Ly treatment&#xA;1  SPtetramer+CD8+TCell SPtetramerplus     MOGSP&#xA;2  SPtetramer+CD8+TCell SPtetramerplus     MOGSP&#xA;3  SPtetramer+CD8+TCell SPtetramerplus       MOG&#xA;4  SPtetramer+CD8+TCell SPtetramerplus       MOG&#xA;5  SPtetramer+CD8+TCell SPtetramerplus       MOG&#xA;6        Ly49-CD8+TCell      Ly49minus     MOGSP&#xA;7        Ly49+CD8+TCell       Ly49plus     MOGSP&#xA;8        Ly49-CD8+TCell      Ly49minus     MOGSP&#xA;9        Ly49+CD8+TCell       Ly49plus     MOGSP&#xA;10       Ly49-CD8+TCell      Ly49minus     MOGSP&#xA;11       Ly49+CD8+TCell       Ly49plus     MOGSP&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I generate an example dataset:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;counts = matrix(rnbinom(11000,mu=100,size=1),ncol=11)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And in the design you specify these two factors, meaning gene expression can be affected by these two conditions:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dds = DESeqDataSetFromMatrix(counts,colData=colData,~Ly +treatment)&#xA;dds = DESeq(dds)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can compare them like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head(results(dds,c(&amp;quot;Ly&amp;quot;,&amp;quot;Ly49plus&amp;quot;,&amp;quot;Ly49minus&amp;quot;)))&#xA;log2 fold change (MLE): Ly Ly49plus vs Ly49minus &#xA;Wald test p-value: Ly Ly49plus vs Ly49minus &#xA;DataFrame with 6 rows and 6 columns&#xA;          baseMean     log2FoldChange            lfcSE               stat&#xA;         &amp;lt;numeric&amp;gt;          &amp;lt;numeric&amp;gt;        &amp;lt;numeric&amp;gt;          &amp;lt;numeric&amp;gt;&#xA;1 91.4458114779915  0.712987165797688 1.28291188260395  0.555756927241586&#xA;2 81.7945234843954 -0.418266530066077 1.14378290088875 -0.365686993345568&#xA;3 94.4351681904941 -0.775592496878288 1.32965355129901 -0.583304196886903&#xA;4 107.484335203298  -1.36694885973712 1.38595911743023 -0.986283680770933&#xA;5 61.2205696982841 -0.760219864564472 1.16779888688769 -0.650985262188887&#xA;6 95.5908168437137  0.497427543096335 1.12768919469662   0.44110340458672&#xA;             pvalue              padj&#xA;          &amp;lt;numeric&amp;gt;         &amp;lt;numeric&amp;gt;&#xA;1 0.578377034355333 0.977537706056193&#xA;2    0.714598652706 0.992321035598301&#xA;3 0.559688537757548 0.977537706056193&#xA;4 0.323993925580938 0.954772990635094&#xA;5 0.515056000552832 0.977537706056193&#xA;6 0.659138138867294 0.992321035598301&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To compare groups within another factor:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;results(dds,c(&amp;quot;treatment&amp;quot;,&amp;quot;MOGSP&amp;quot;,&amp;quot;MOG&amp;quot;))&#xA;log2 fold change (MLE): treatment MOGSP vs MOG &#xA;Wald test p-value: treatment MOGSP vs MOG &#xA;DataFrame with 6 rows and 6 columns&#xA;          baseMean     log2FoldChange            lfcSE               stat&#xA;         &amp;lt;numeric&amp;gt;          &amp;lt;numeric&amp;gt;        &amp;lt;numeric&amp;gt;          &amp;lt;numeric&amp;gt;&#xA;1 91.4458114779915 -0.875905107403943 1.43007567177694 -0.612488642867122&#xA;2 81.7945234843954  0.595588650280833 1.28109080878515  0.464907441530728&#xA;3 94.4351681904941 0.0255578400583986 1.48592401497315 0.0171999643325371&#xA;4 107.484335203298  -1.46652057431159 1.55285829334496 -0.944400774105796&#xA;5 61.2205696982841  -1.27442596494316 1.30489822606005  -0.97664778715433&#xA;6 95.5908168437137  -2.35402796630456 1.26608624796739  -1.85929510733078&#xA;              pvalue              padj&#xA;           &amp;lt;numeric&amp;gt;         &amp;lt;numeric&amp;gt;&#xA;1  0.540214509839105 0.944881849859735&#xA;2  0.641997741235223 0.973651054534338&#xA;3  0.986277090644156 0.994944323499595&#xA;4  0.344964886056295  0.87421063696937&#xA;5  0.328743552427222                NA&#xA;6 0.0629853201285643 0.525889480109579&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The colData i used:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(name = structure(c(3L, 3L, 3L, 3L, 3L, 1L, 2L, &#xA;1L, 2L, 1L, 2L), .Label = c(&amp;quot;Ly49-CD8+TCell&amp;quot;, &amp;quot;Ly49+CD8+TCell&amp;quot;, &#xA;&amp;quot;SPtetramer+CD8+TCell&amp;quot;), class = &amp;quot;factor&amp;quot;), Ly = c(&amp;quot;SPtetramerplus&amp;quot;, &#xA;&amp;quot;SPtetramerplus&amp;quot;, &amp;quot;SPtetramerplus&amp;quot;, &amp;quot;SPtetramerplus&amp;quot;, &amp;quot;SPtetramerplus&amp;quot;, &#xA;&amp;quot;Ly49minus&amp;quot;, &amp;quot;Ly49plus&amp;quot;, &amp;quot;Ly49minus&amp;quot;, &amp;quot;Ly49plus&amp;quot;, &amp;quot;Ly49minus&amp;quot;, &#xA;&amp;quot;Ly49plus&amp;quot;), treatment = structure(c(2L, 2L, 1L, 1L, 1L, 2L, &#xA;2L, 2L, 2L, 2L, 2L), .Label = c(&amp;quot;MOG&amp;quot;, &amp;quot;MOGSP&amp;quot;), class = &amp;quot;factor&amp;quot;)), class = &amp;quot;data.frame&amp;quot;, row.names = c(NA, &#xA;-11L))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-09-13T18:34:53.213" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14355" PostTypeId="2" ParentId="14341" CreationDate="2020-09-14T02:41:15.123" Score="0" Body="&lt;p&gt;Somebody helped me with this one through other channels, but I wanted to make it publicly available:&lt;/p&gt;&#xA;&lt;p&gt;Make a tarball of the STAR index. The option &lt;code&gt;--strip-components=1&lt;/code&gt; strips the first level of the path from the untarred items; thus, the expected tarball was made with something like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tar -cvf ${star_index} the_star_index_directory/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then point Cromwell/WDL to the tarball, e.g. on S3. The GTEx container will unwrap the tarball on the fly during execution.&lt;/p&gt;&#xA;" OwnerUserId="3355" LastActivityDate="2020-09-14T02:41:15.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14356" PostTypeId="1" AcceptedAnswerId="14360" CreationDate="2020-09-14T06:06:38.050" Score="8" ViewCount="6675" Body="&lt;p&gt;As &lt;a href=&quot;https://stackoverflow.com/q/63851080/680068&quot;&gt;my question in SO&lt;/a&gt; was closed and asked to be posted in this forum, I am posting it here.&lt;/p&gt;&#xA;&lt;p&gt;I am not from the bioinformatics domain. However, for the sake of analysis, I am trying to pick up certain basics related to the GT field in the VCF file.&lt;/p&gt;&#xA;&lt;p&gt;I know we have a field called &lt;code&gt;Alleles&lt;/code&gt;. May I know under what circumstances GT takes a certain value and how they are called? Can you confirm my understanding?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Ref Alt    GT   Name &#xA;A    A     0/0  Homozygous&#xA;A    G     0/1  Heterozygous (does 1/0 also mean the same?) What's this 0 and 1 actually?&#xA;A   [C,CA] ??    ??&#xA;??   ??    1/1  HOM_ALT? How and why? &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can the experts here help me to fill the &lt;code&gt;question marks&lt;/code&gt; and also help me understand with the Ref and Alt combinations when a genotype can take a value of 0/0 or 1/0 or 1/1 or 1/2 etc. and what are the names for those values? Like when it is called HOM_ALT etc.?&lt;/p&gt;&#xA;&lt;p&gt;Any simple explanation for beginner like me (with no background in bioinformatics/biology) can be helpful.&lt;/p&gt;&#xA;" OwnerUserId="6380" LastEditorUserId="-1" LastEditDate="2023-06-19T13:01:09.910" LastActivityDate="2023-06-19T13:01:09.910" Title="How is the GT field in a VCF file defined?" Tags="|vcf|file-formats|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14357" PostTypeId="2" ParentId="14353" CreationDate="2020-09-14T07:18:04.920" Score="1" Body="&lt;p&gt;DESeq2 uses the batch information (and everything else in the design) to produce offsets for its GLM. For a background on that please check how linear models work, e.g. using the StatQuest series of statistics videos over at YouTube.&#xA;It still operates on the raw counts. The same goes for the normalization factors.&#xA;&lt;code&gt;removeBatchEffect&lt;/code&gt; fits a linear model to the data including the batch information and then subtracts the batch component from the counts (that is basically the baseline difference).&#xA;If you are interested in preserving the integer nature of counts and preserving zeros as smallest values after explicit batch correction you may want to check &lt;code&gt;ComBat-seq()&lt;/code&gt; from the sva package. It operates on raw counts and returns batch-corrected raw counts which you could then normalize and calculate CPMs from (if you need batch corrected CPMs with no negative values). I find it useful and prefer it over &lt;code&gt;removeBatchEffect&lt;/code&gt; as it avoids the unfortunate negative counts which sometimes messes up plotting scripts that expect zero as the smallest possible value.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-09-14T07:18:04.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14359" PostTypeId="2" ParentId="14356" CreationDate="2020-09-14T10:23:10.190" Score="7" Body="&lt;p&gt;0 and 1 is just a way of coding the reference (0) and alternate (1) allele. These could be A/G, C/T etc. It's just a simplified way of expressing the different alleles.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;0/1&lt;/code&gt; and &lt;code&gt;1/0&lt;/code&gt; functionally mean the same thing (that the individual is a heterozygote) - since the genotype is unphased, the alleles aren't ordered. The &lt;code&gt;/&lt;/code&gt; symbol tells you the genotype is unphased. However, in practice, a heterozygous genotype is always written as &lt;code&gt;0/1&lt;/code&gt; as a matter of convention. &lt;code&gt;0/0&lt;/code&gt; is referred to as homozygous reference, and &lt;code&gt;1/1&lt;/code&gt; as homozygous alternative.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;0|1&lt;/code&gt; and &lt;code&gt;1|0&lt;/code&gt; &lt;strong&gt;do&lt;/strong&gt; mean something different, since the pipe symbol &lt;code&gt;|&lt;/code&gt; tell us the order of the alleles matters.&lt;/p&gt;&#xA;&lt;p&gt;Occasionally you will have multi allelic positions, where you have more than one alternate allele, and therefore the field will look like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Ref   Alt&#xA;A   [C,GT]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The extra alt allele is given the number 2, so in this case if an individual had the genotype CT then their genotype code would be 0/2.&lt;/p&gt;&#xA;&lt;p&gt;Full detail is given in the official &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.2.pdf&quot; rel=&quot;noreferrer&quot;&gt;VCF specification page.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="3694" LastEditDate="2020-09-14T11:35:11.667" LastActivityDate="2020-09-14T11:35:11.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14360" PostTypeId="2" ParentId="14356" CreationDate="2020-09-14T12:13:07.950" Score="7" Body="&lt;p&gt;You can get most of the info from this &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3137218&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;. See Fig. 1 and the surrounding text. Quoting from there, &amp;quot;GT, genotype, encodes alleles as numbers: 0 for the reference allele, 1 for the first allele listed in ALT column, 2 for the second allele listed in ALT and so on.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;In your case, the reference allele, here a single nucleotide, is A. When the alternate allele is also A, the genotype GT is reference, or 0. There are 2 copies of each allele in the human genome in non-sex chromosomes chr1-chr22, hence 0/0, or homozygous reference or HOM_REF.&lt;/p&gt;&#xA;&lt;p&gt;When ALT=G, and the GT column is 0/1, this means that you have 1 reference allele (0), and 1 alternate allele (1). This means that you have A on one copy of this locus, and G on another. The convention is write GT field in ascending order, so 0/1 rather than 1/0. This is called heterozygous, or HET.&lt;/p&gt;&#xA;&lt;p&gt;When ALT=C,CA, the GT is probably 1/2, because there are 2 alternate alleles, and I assume we continue with the same chromosome present in 2 copies. This means there are no reference alleles here at all, only alternate alleles. It is a heterozygous genotype composed of two different ALT alleles, or HET_ALT. Note that it is not enclosed in square brackets in the vcf file format: &lt;code&gt;A &amp;lt;tab&amp;gt; C,CA &amp;lt;tab&amp;gt; 1/2 ...&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Finally, these are some examples of HOM_ALT:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A    C     1/1&#xA;A    G     1/1&#xA;A    CA    1/1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This means that the same ALT allele (either C, or G, or CA) is present in 2 copies. There is no reference allele present. This is called homozygous alternate genotype.&lt;/p&gt;&#xA;&lt;p&gt;In general, the name &lt;em&gt;homo&lt;/em&gt; means the same, and &lt;em&gt;hetero&lt;/em&gt; means different, in the context of genotypes.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;REFERENCES:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Danecek P, Auton A, Abecasis G, et al. The variant call format and VCFtools. Bioinformatics. 2011;27(15):2156-2158. doi:10.1093/bioinformatics/btr330 : &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3137218/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3137218/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;SEE ALSO:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035531692-VCF-Variant-Call-Format&quot; rel=&quot;nofollow noreferrer&quot;&gt;VCF - Variant Call Format&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.biostars.org/p/86321/&quot; rel=&quot;nofollow noreferrer&quot;&gt;What Does Genotype (&amp;quot;0/0&amp;quot;, &amp;quot;0/1&amp;quot; Or &amp;quot;1/1&amp;quot;) In *.Vcf File Represent?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.biostars.org/p/377242/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Difference between 0/0 and ./. for genotype in VCF&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.biostars.org/p/187068/&quot; rel=&quot;nofollow noreferrer&quot;&gt;1/2 in VCF genotype field?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6251" LastEditorUserId="6251" LastEditDate="2023-04-04T16:08:47.913" LastActivityDate="2023-04-04T16:08:47.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14361" PostTypeId="1" AcceptedAnswerId="14380" CreationDate="2020-09-14T13:31:52.477" Score="0" ViewCount="739" Body="&lt;p&gt;I am setting up a HiChIPseq pipeline, but installing software necessary for the HiC side of things has been unbecoming.&lt;/p&gt;&#xA;&lt;p&gt;Both my PI and I have tried to install Hi-C Pro onto our Mac Pro that we use as our compute server (though in a couple of weeks we hope to migrate to an EPYC 64 Core computer server of our own!).&lt;/p&gt;&#xA;&lt;p&gt;There are three approaches that I have taken to install Hi-C Pro.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/nservant/HiC-Pro&quot; rel=&quot;nofollow noreferrer&quot;&gt;From Source&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;To install from source, you need to supply the paths to its dependencies. Believing we did a good job, we run it and find problems accessing bx-python.&lt;/p&gt;&#xA;&lt;p&gt;No matter how many times we update pip and try to install bx-python and specify that we want it installed for python2.&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;From singularity&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This is the one I tried. From what I understand this creates a VM with the necessary dependencies installed, and allows you to use the program without building. However, it appears to abort when conda tries to install numpy.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;+ conda install -y -c anaconda numpy=1.16.3&#xA;Collecting package metadata (current_repodata.json): done&#xA;Solving environment: failed with initial frozen solve. Retrying with flexible solve.&#xA;Collecting package metadata (repodata.json): / Killed&#xA;ABORT: Aborting with RETVAL=255&#xA;Cleaning up...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have seen the failed solving environment bug many times, but have no idea how to solve it. I have tried updating conda, but that doesn't solve it.&lt;/p&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/dovetail-genomics/dovetail_tools&quot; rel=&quot;nofollow noreferrer&quot;&gt;Install Someone's Premade Conda Environment&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Running conda.sh give me this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Solving environment: failed&#xA;&#xA;ResolvePackageNotFound:&#xA;&amp;lt;Lots of packages...&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Sorry if this is sloppy.&lt;/p&gt;&#xA;" OwnerUserId="9521" LastEditorUserId="9521" LastEditDate="2020-09-16T19:58:42.337" LastActivityDate="2022-03-31T04:05:46.530" Title="Installing HiC-Pro issues" Tags="|chip-seq|software-installation|hi-c|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14362" PostTypeId="1" CreationDate="2020-09-14T15:01:38.300" Score="0" ViewCount="81" Body="&lt;p&gt;Using Biojava 5, I am trying to convert a sequence of Amino Acids to a Nucleotide sequence. Any idea how to do this?&lt;/p&gt;&#xA;" OwnerUserId="4660" LastActivityDate="2020-09-16T05:31:03.153" Title="BioJava - Protein to DNA" Tags="|phylogenetics|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14364" PostTypeId="2" ParentId="14361" CreationDate="2020-09-14T18:40:08.137" Score="1" Body="&lt;p&gt;I could install HiC-Pro using conda when creating a separate environment (which I always do for side-projects), and then pull the conda build of HiC-Pro:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda create --name HiCPro python=2.7&#xA;conda activate HiCPro&#xA;conda install -c davebx hicpro&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Not sure what &lt;code&gt;conda.sh&lt;/code&gt; is that you describe, sometimes tricky to get random code from GitHub running. Give it a go with a standard conda install as I described.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-09-14T18:40:08.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14367" PostTypeId="1" AcceptedAnswerId="14382" CreationDate="2020-09-15T10:18:40.247" Score="0" ViewCount="878" Body="&lt;p&gt;&lt;a href=&quot;https://drive.google.com/file/d/1sUjrKaPgqJRixmGoWrKfNo1WYOA_qloj/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;file&lt;/a&gt; input file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;a &amp;lt;- read.csv(&amp;quot;Studies_studies.tsv&amp;quot;,sep = &amp;quot;\t&amp;quot;)&#xA;&#xA;#https://stackoverflow.com/questions/43658982/cross-comparison-of-columns-of-the-same-data-frame&#xA;bb &amp;lt;- tcrossprod(table(melt(as.matrix(a))[-1])) * !diag(37)&#xA;&#xA;dd&amp;lt;- setNames(cbind.data.frame(t(combn(names(a), 2)),&#xA;                               combn(names(a), 2, function(x) length(intersect(a[, x[1]],a[, x[2]])))),&#xA;              c(&amp;quot;col1&amp;quot;, &amp;quot;col2&amp;quot;, &amp;quot;count&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So far my code.&lt;/p&gt;&#xA;&lt;p&gt;I can get the count between each pairwise intersection happening between the columns but Im not able to get the elements that is common or intersected between two columns.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;For example&lt;/strong&gt;&lt;br /&gt;&#xA;This &lt;code&gt;C0026998 C1879321  1310&lt;/code&gt; I do find that there are 1310 elements common between those to ID but I'm not able to find the name of the ID that are column.&lt;/p&gt;&#xA;&lt;p&gt;So that way if I extrapolate my problem what i would like to see is the pairwise intersection among all the elements and i would like to store the output dataframe.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion or help would be really appreciated.&lt;/p&gt;&#xA;&lt;p&gt;input file dput&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(C0026998 = c(32621177L, 32586363L, 32504186L, &#xA;32391628L, 32333156L, 32171069L), C1879321 = c(27276561L, 29286103L, &#xA;29988143L, 30076173L, 30024784L, 30892988L), C0023470 = c(27276561L, &#xA;29988143L, 30076173L, 30024784L, 26837842L, 29286103L), C0280449 = c(30024784L, &#xA;30076173L, 30892988L, 31688884L, 30651561L, 31516032L), C0522631 = c(26055299L, &#xA;12801839L, 10516750L, 28825596L, 12652469L, 16203817L), C1292775 = c(28883080L, &#xA;10400416L, 22581002L, 1503929L, 9499663L, NA), C0023472 = c(26837842L, &#xA;29091516L, 29498925L, 20525995L, 27217448L, 20525993L), C1292773 = c(30135184L, &#xA;24440648L, 26273061L, 21508125L, NA, NA), C4545381 = c(30093401L, &#xA;29933073L, 29437791L, 31227358L, 30541745L, 28555084L), C4707228 = c(26492932L, &#xA;21791467L, 28179273L, 24077845L, 30287390L, 22362038L), C2939461 = c(28588020L, &#xA;29988143L, 30076173L, 30510081L, 30024784L, 30963592L), C1336735 = c(28645776L, &#xA;28844816L, 31400961L, 30545576L, 29241450L, 28588020L), C2827362 = c(29119847L, &#xA;29486661L, 29567772L, 31775485L, 30670826L, NA), C3839741 = c(28494506L, &#xA;26492932L, 30923103L, 25715404L, 28556489L, 25006131L), C3839868 = c(28473620L, &#xA;25512507L, 31015209L, 29933069L, 29321554L, 21791467L), C4520840 = c(28246192L, &#xA;23935018L, 21750082L, 10741700L, 9407724L, NA), C4528668 = c(28645776L, &#xA;28588020L, 30510081L, 30409776L, 30093401L, 28844816L), C2825139 = c(29241450L, &#xA;30541745L, 31152020L, 24663049L, NA, NA), C1332153 = c(24440648L, &#xA;26273061L, NA, NA, NA, NA), C4721505 = c(18623376L, 21618422L, &#xA;21685468L, 25196856L, 16985182L, 18353632L), C2826177 = c(27276561L, &#xA;29286103L, 25512507L, 29079128L, 28473620L, 26789727L), C4324477 = c(29286103L, &#xA;22375971L, 30153096L, 30368038L, 25006131L, 30054307L), C2826176 = c(27276561L, &#xA;29988143L, 27959731L, 29286103L, 25512507L, 28588019L), C2827356 = c(27716285L, &#xA;29177434L, 30952642L, 30418178L, 27126994L, 29656442L), C3711382 = c(30510081L, &#xA;26492932L, 29991558L, 27927766L, 28490572L, 29860938L), C4704767 = c(30555165L, &#xA;30055822L, 25540937L, 30135184L, 30545923L, 30361063L), C0027013 = c(22375971L, &#xA;28962635L, 22375970L, 29515114L, 30185431L, 23216616L), C0153886 = c(30510081L, &#xA;28644114L, 28588020L, 30555165L, 30024784L, 29304833L), C0279094 = c(28644114L, &#xA;28588020L, 30510081L, 30555165L, 30076173L, 30024784L), C0279623 = c(27276561L, &#xA;30555165L, 28588020L, 30076173L, 30024784L, 30510081L), C1142169 = c(28588020L, &#xA;30510081L, 25512507L, 25540937L, 30409776L, 30668890L), C1275661 = c(29660836L, &#xA;30963592L, 22579233L, 21880637L, 26772158L, 27121471L), C1292776 = c(30055822L, &#xA;30115541L, 10025899L, 31152020L, 26577691L, 23129738L), C1320638 = c(27959731L, &#xA;18489989L, 25425683L, 28179273L, 30510079L, 30655375L), C1737261 = c(29988143L, &#xA;28645776L, 29933073L, 30545923L, 30361063L, 28336527L), C2826178 = c(25512507L, &#xA;29286103L, 28473620L, 25611491L, 21242187L, 19059939L), C4744720 = c(30555165L, &#xA;28644114L, 29597002L, 30541745L, 30118897L, 29660836L), relapse = c(32786187L, &#xA;32601798L, 32564196L, 32391628L, 32333156L, 32062741L)), row.names = c(NA, &#xA;6L), class = &amp;quot;data.frame&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output what i have&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dput(head(dd))&#xA;structure(list(col1 = structure(c(3L, 3L, 3L, 3L, 3L, 3L), .Label = c(&amp;quot;C0023470&amp;quot;, &#xA;&amp;quot;C0023472&amp;quot;, &amp;quot;C0026998&amp;quot;, &amp;quot;C0027013&amp;quot;, &amp;quot;C0153886&amp;quot;, &amp;quot;C0279094&amp;quot;, &amp;quot;C0279623&amp;quot;, &#xA;&amp;quot;C0280449&amp;quot;, &amp;quot;C0522631&amp;quot;, &amp;quot;C1142169&amp;quot;, &amp;quot;C1275661&amp;quot;, &amp;quot;C1292773&amp;quot;, &amp;quot;C1292775&amp;quot;, &#xA;&amp;quot;C1292776&amp;quot;, &amp;quot;C1320638&amp;quot;, &amp;quot;C1332153&amp;quot;, &amp;quot;C1336735&amp;quot;, &amp;quot;C1737261&amp;quot;, &amp;quot;C1879321&amp;quot;, &#xA;&amp;quot;C2825139&amp;quot;, &amp;quot;C2826176&amp;quot;, &amp;quot;C2826177&amp;quot;, &amp;quot;C2826178&amp;quot;, &amp;quot;C2827356&amp;quot;, &amp;quot;C2827362&amp;quot;, &#xA;&amp;quot;C2939461&amp;quot;, &amp;quot;C3711382&amp;quot;, &amp;quot;C3839741&amp;quot;, &amp;quot;C3839868&amp;quot;, &amp;quot;C4324477&amp;quot;, &amp;quot;C4520840&amp;quot;, &#xA;&amp;quot;C4528668&amp;quot;, &amp;quot;C4545381&amp;quot;, &amp;quot;C4704767&amp;quot;, &amp;quot;C4707228&amp;quot;, &amp;quot;C4721505&amp;quot;, &amp;quot;C4744720&amp;quot;&#xA;), class = &amp;quot;factor&amp;quot;), col2 = structure(c(18L, 1L, 7L, 8L, 12L, &#xA;2L), .Label = c(&amp;quot;C0023470&amp;quot;, &amp;quot;C0023472&amp;quot;, &amp;quot;C0027013&amp;quot;, &amp;quot;C0153886&amp;quot;, &#xA;&amp;quot;C0279094&amp;quot;, &amp;quot;C0279623&amp;quot;, &amp;quot;C0280449&amp;quot;, &amp;quot;C0522631&amp;quot;, &amp;quot;C1142169&amp;quot;, &amp;quot;C1275661&amp;quot;, &#xA;&amp;quot;C1292773&amp;quot;, &amp;quot;C1292775&amp;quot;, &amp;quot;C1292776&amp;quot;, &amp;quot;C1320638&amp;quot;, &amp;quot;C1332153&amp;quot;, &amp;quot;C1336735&amp;quot;, &#xA;&amp;quot;C1737261&amp;quot;, &amp;quot;C1879321&amp;quot;, &amp;quot;C2825139&amp;quot;, &amp;quot;C2826176&amp;quot;, &amp;quot;C2826177&amp;quot;, &amp;quot;C2826178&amp;quot;, &#xA;&amp;quot;C2827356&amp;quot;, &amp;quot;C2827362&amp;quot;, &amp;quot;C2939461&amp;quot;, &amp;quot;C3711382&amp;quot;, &amp;quot;C3839741&amp;quot;, &amp;quot;C3839868&amp;quot;, &#xA;&amp;quot;C4324477&amp;quot;, &amp;quot;C4520840&amp;quot;, &amp;quot;C4528668&amp;quot;, &amp;quot;C4545381&amp;quot;, &amp;quot;C4704767&amp;quot;, &amp;quot;C4707228&amp;quot;, &#xA;&amp;quot;C4721505&amp;quot;, &amp;quot;C4744720&amp;quot;, &amp;quot;relapse&amp;quot;), class = &amp;quot;factor&amp;quot;), count = structure(c(1310L, &#xA;1310L, 164L, 9L, 6L, 23L), .Dim = 6L)), row.names = c(NA, 6L), class = &amp;quot;data.frame&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;from the above output i do know that  &lt;code&gt;C0026998 C1879321  1310&lt;/code&gt; between C0026998 and C1879321 there are 1310 elements are common. Now the next thing i would like to find out is what are the elements that are common between those two IDs same goes for rest of the column which are to be compared pairwise.&lt;/p&gt;&#xA;&lt;p&gt;The output I'm expecting is this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(`intersect(a$C0026998, a$C1879321)` = c(32621177L, &#xA;32586363L, 32504186L, 32391628L, 32333156L, 32171069L)), row.names = c(NA, &#xA;6L), class = &amp;quot;data.frame&amp;quot;)&#xA;&#xA;&#xA;structure(list(`intersect(a$C0026998, a$C0280449)` = c(32171069L, &#xA;31841594L, 31770437L, 31753773L, 31688884L, 31516032L)), row.names = c(NA, &#xA;6L), class = &amp;quot;data.frame&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So in each column contain one Mesh term where there PMIDS which i want to find if these studies are overlapping or not.So in the above example i have given intersection between first four columns so i can see the IDs which are common. So i want to do the same with all the columns and save the results.&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2020-09-15T20:46:21.660" LastActivityDate="2020-09-17T12:29:00.160" Title="Find intersection among multiple columns and find the intersected elements" Tags="|r|" AnswerCount="1" CommentCount="8" ClosedDate="2020-09-24T11:27:08.963" ContentLicense="CC BY-SA 4.0" />
  <row Id="14368" PostTypeId="1" CreationDate="2020-09-15T13:08:59.913" Score="-2" ViewCount="57" Body="&lt;p&gt;I have a fasta file and the sequences in them are arranged like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;chr8:106028475-106028517()&#xA;GTaccaggcctagggttgcactcctgtagtctcagcattggg&#xA;&amp;gt;chr8:106040430-106040472()&#xA;TAACCGGCTCTCCGCTGCCCTCATCTGTCTAGAATTCAGGTA&#xA;&amp;gt;chr9:100245605-100245648()&#xA;TACCACCTCCCTCACTCTTTCCCAAATTTATGAGTTAGGCTAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to extract only those which begin with TA. so my final output should look like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;chr8:106040430-106040472()&#xA;TAACCGGCTCTCCGCTGCCCTCATCTGTCTAGAATTCAGGTA&#xA;&amp;gt;chr9:100245605-100245648()&#xA;TACCACCTCCCTCACTCTTTCCCAAATTTATGAGTTAGGCTAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Kindly help.&lt;/p&gt;&#xA;" OwnerUserId="7236" LastActivityDate="2020-09-16T17:02:27.253" Title="get sequences begining with TA" Tags="|fasta|sequence|" AnswerCount="2" CommentCount="1" ClosedDate="2020-09-20T11:02:17.923" ContentLicense="CC BY-SA 4.0" />
  <row Id="14370" PostTypeId="1" CreationDate="2020-09-15T21:16:58.340" Score="2" ViewCount="25" Body="&lt;p&gt;I am wondering if there's a database to look up the annotation history of an arbitrary gene/ORF, with links to the primary literature for each step of the annotation. I have found that neither NCBI Gene nor NCBI Protein makes this easy.&lt;/p&gt;&#xA;&lt;p&gt;As a case in point, I was looking at the &amp;quot;TMEM79&amp;quot; gene, which was recently reported as an inhibitor of Wnt signaling: &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/32924931/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pubmed.ncbi.nlm.nih.gov/32924931/&lt;/a&gt;. Searching Pubmed for &amp;quot;TMEM79&amp;quot; gave references establishing that this gene has the synonym &amp;quot;Mattin&amp;quot; and has roles in skin allergy and secretion from the Golgi. However, none of the papers I found reference a paper for the origin of the name &amp;quot;TMEM79&amp;quot; itself. Looking at both the gene and the protein in NCBI only gives these same &amp;quot;late&amp;quot; papers, that were evidently published after the annotation &amp;quot;TMEM79&amp;quot; had already been applied to this ORF for some time, and all acted as if this name had simply fallen from the sky without quoting a reference.&lt;/p&gt;&#xA;&lt;p&gt;Presumably the ORF was first detected by some automated gene scanning program, then subsequently someone (possibly the same group or possibly a completely different one) ran it through a transmembrane prediction program that detected that it was likely an integral membrane protein. However, strangely nobody provides a citation for either of these steps, which I would certainly want to do if I were writing a paper on this gene (as this protein certainly seems never to have been cloned and biochemically characterized in even the most rudimentary manner, computational inferences at the sequence level are the only means to contextualize a role in any biological process).&lt;/p&gt;&#xA;&lt;p&gt;So I'm wondering, is there a standard database that makes this easy, i.e. given an ORF and an identifier, it returns the literature citation where that identifier was first applied to that ORF? In this case that would clearly point to the transmembrane prediction step because nobody would name an ORF &amp;quot;TMEMxx&amp;quot; unless they thought it was a transmembrane protein. And that would hopefully cite the first identification of the ORF itself, if those were not one and the same paper.&lt;/p&gt;&#xA;" OwnerUserId="9534" LastEditorUserId="9534" LastEditDate="2020-09-15T21:26:48.277" LastActivityDate="2020-09-15T21:26:48.277" Title="Finding original papers for gene annotations" Tags="|database|sequence-annotation|literature-search|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14372" PostTypeId="2" ParentId="14362" CreationDate="2020-09-16T05:31:03.153" Score="1" Body="&lt;p&gt;Note that unlike the reverse (nucleotide sequence to protein sequence), which is uniquely defined given a reading frame, the mapping from protein sequence to nucleotide sequence is not unique (it is not a function in the mathematical sense). The method you use to select which of the many (exponential in input sequence length) possible sequences to output depends critically on what the question is.&lt;/p&gt;&#xA;&lt;p&gt;If the goal is to design a synthetic gene that will produce a desired protein, then you need to optimize it for expression in the organism into which the gene will be inserted, subject to the constraint of the encoded amino acid sequence.&lt;/p&gt;&#xA;&lt;p&gt;If the goal is to search for nucleotide sequences in one or more genomes that could give rise to a certain polypeptide, there is a much better way than to do reverse translation at all. In particular, most putative protein-coding open reading frames in most sequenced genomes have already been run through forward translation code and the expected amino acid sequence encoded in them can be found in databases. So, this problem is reduced to a substring search in these databases.&lt;/p&gt;&#xA;" OwnerUserId="9534" LastActivityDate="2020-09-16T05:31:03.153" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14373" PostTypeId="1" AcceptedAnswerId="14375" CreationDate="2020-09-16T07:19:47.040" Score="2" ViewCount="133" Body="&lt;p&gt;Bioconda packages have associated containers images. E.g. &lt;code&gt;bioconductor-csaw&lt;/code&gt; has the image &lt;code&gt;quay.io/biocontainers/bioconductor-csaw&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I would like to customize the image, in particular, I would like to add rstudio.&lt;/p&gt;&#xA;&lt;p&gt;It is possible to get the Dockerfile associated to the image of a bioconda package?&lt;/p&gt;&#xA;&lt;p&gt;How do you suggest to proceed to customize the image?&lt;/p&gt;&#xA;" OwnerUserId="3772" LastActivityDate="2020-10-16T09:05:15.670" Title="Get Dockerfile of a bioconda package" Tags="|rstudio|bioconda|docker|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14374" PostTypeId="2" ParentId="14368" CreationDate="2020-09-16T08:23:57.833" Score="0" Body="&lt;p&gt;Assuming your sequences are always on a single line&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -B 1 &amp;quot;^TA&amp;quot; seqences.fasta | grep -v &amp;quot;\-\-&amp;quot; &amp;gt; filtered_seqences.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;should do the job. Argument &lt;code&gt;-B&lt;/code&gt; specifies how many lines before the pattern match should be printed. But there grep separates individual entries with &lt;code&gt;--&lt;/code&gt; that's why I used a second grep to get rid of those.&lt;/p&gt;&#xA;&lt;p&gt;Ha, apparently there is an undocumented &lt;code&gt;--no-group-separator&lt;/code&gt; flag, so this should do as well&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -B 1 &amp;quot;^TA&amp;quot; seqences.fasta --no-group-separator &amp;gt; filtered_seqences.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2020-09-16T17:02:27.253" LastActivityDate="2020-09-16T17:02:27.253" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14375" PostTypeId="2" ParentId="14373" CreationDate="2020-09-16T08:59:44.873" Score="1" Body="&lt;p&gt;The Dockerfile is generated by galaxy-utils, so I'm not entirely sure what it looks like. However you should be able to generate such a &amp;quot;mulled&amp;quot; container as documented &lt;a href=&quot;https://github.com/BioContainers/mulled&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. In short, something of the form:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mulled-build build 'bioconductor-csaw=someversion--somebuildnumber' 'someotherpackage=someversion--somebuildnumber' ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That probably requires setting the CONDA_IMAGE environment variable. Bioconda currently sets that to &lt;code&gt;quay.io/dpryan79/mulled_container:latest&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-09-16T08:59:44.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14376" PostTypeId="2" ParentId="14346" CreationDate="2020-09-16T15:08:13.243" Score="2" Body="&lt;p&gt;I don't believe this statement is true:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;To solve this issue, both TopHat2 and HISAT2 chop up the unmapped reads into around 25bp fragments.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;TopHat2 did not support spliced alignment directly.  This is why it followed this general approach of chopping the read up into small parts, followed by &amp;quot;contiguous&amp;quot; alignment of the sub-reads.  Hisat2 / Hisat-genotype support spliced alignment directly insofar as they don't pre-partition the reads into small parts, but allow dynamic extensions of alignments across splice junctions (in a way similar to how STAR works).  This is generally a fundamentally better approach and is one reason, among many, why TopHat/TopHat2 have been put into maintenance-only mode and have been officially superseded by Hisat2/genotype.&lt;/p&gt;&#xA;" OwnerUserId="9368" LastActivityDate="2020-09-16T15:08:13.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14377" PostTypeId="2" ParentId="14368" CreationDate="2020-09-16T16:13:42.460" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/fhcrc/seqmagick&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqmagick&lt;/a&gt; has a handy &lt;code&gt;--seq-pattern-include&lt;/code&gt; feature that works similarly to what Kamil did with a regular grep.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat sequences.fasta &#xA;&amp;gt;chr8:106028475-106028517()&#xA;GTaccaggcctagggttgcactcctgtagtctcagcattggg&#xA;&amp;gt;chr8:106040430-106040472()&#xA;TAACCGGCTCTCCGCTGCCCTCATCTGTCTAGAATTCAGGTA&#xA;&amp;gt;chr9:100245605-100245648()&#xA;TACCACCTCCCTCACTCTTTCCCAAATTTATGAGTTAGGCTAT&#xA;&#xA;$ seqmagick convert --seq-pattern-include &amp;quot;^TA&amp;quot; sequences.fasta -&#xA;&amp;gt;chr8:106040430-106040472()&#xA;TAACCGGCTCTCCGCTGCCCTCATCTGTCTAGAATTCAGGTA&#xA;&amp;gt;chr9:100245605-100245648()&#xA;TACCACCTCCCTCACTCTTTCCCAAATTTATGAGTTAGGCTAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It's a great little tool for this sort of thing built around Biopython, though it's been falling behind in compatibility with the latest versions as you can see from some of the open GitHub issues.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2020-09-16T16:13:42.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14378" PostTypeId="2" ParentId="14095" CreationDate="2020-09-16T20:45:54.253" Score="1" Body="&lt;p&gt;You can give &lt;a href=&quot;https://github.com/saketkc/pysradb&quot; rel=&quot;nofollow noreferrer&quot;&gt;pysradb&lt;/a&gt; a shot.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;study_accession experiment_accession experiment_title                                                   experiment_desc              &#xA;                                      organism_taxid  organism_name           library_strategy library_source  library_selection library_layout sample_accession sample_title instrument           total_spots total_size  run_accession run_total_spots run_total_bases&#xA;SRP178555       SRX5243234             GSM3560726: 97-gem-Day3-OEA115; Caenorhabditis elegans; RNA-Seq    GSM3560726: 97-gem-Day3-OEA115; Caenorhabditis elegans; RNA-Seq  6239            Caenorhabditis elegans  RNA-Seq          TRANSCRIPTOMIC  cDNA              SINGLE         SRS4245909                    Illumina HiSeq 2500  19052916     446793895  SRR8435699    19052916         971698716    &#xA;SRP178555       SRX5243233             GSM3560725: 97-gem-Day3-OEA087; Caenorhabditis elegans; RNA-Seq    GSM3560725: 97-gem-Day3-OEA087; Caenorhabditis elegans; RNA-Seq  6239            Caenorhabditis elegans  RNA-Seq          TRANSCRIPTOMIC  cDNA              SINGLE         SRS4245908                    Illumina HiSeq 2500  18156224     428457264  SRR8435698    18156224         925967424 &#xA;.. [truncated]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="161" LastActivityDate="2020-09-16T20:45:54.253" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14379" PostTypeId="2" ParentId="13872" CreationDate="2020-09-16T20:48:34.050" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/saketkc/pysradb&quot; rel=&quot;nofollow noreferrer&quot;&gt;pysradb&lt;/a&gt; can fetch ENA/SRA fastq/bam links (if available):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$ pysradb metadata SRR8240860 --detailed&#xA;run_accession study_accession experiment_accession experiment_title                                               experiment_desc                                                organism_taxid  organism_name           library_strategy library_source  library_selection library_layout sample_accession sample_title instrument   total_spots total_size  run_total_spots run_total_bases run_alias      sra_url_alt1                                    sra_url_alt2                                    sra_url                                                                                  experiment_alias source_name strain/genotype developmental stage ena_fastq_http ena_fastq_http_1                                                                 ena_fastq_http_2                                                                 ena_fastq_ftp ena_fastq_ftp_1                                                                     ena_fastq_ftp_2                                                                    &#xA;SRR8240860    SRP170618       SRX5059122           GSM3487689: fer-15 Day1 rep1; Caenorhabditis elegans; RNA-Seq  GSM3487689: fer-15 Day1 rep1; Caenorhabditis elegans; RNA-Seq  6239            Caenorhabditis elegans  RNA-Seq          TRANSCRIPTOMIC  cDNA              PAIRED         SRS4075216       N/A          HiSeq X Ten  40494415    4755049874  40494415        12148324500     GSM3487689_r1  gs://sra-pub-src-3/SRR8240860/RRA0719_R2.fq.gz  s3://sra-pub-src-3/SRR8240860/RRA0719_R2.fq.gz  https://sra-downloadb.st-va.ncbi.nlm.nih.gov/sos2/sra-pub-run-3/SRR8240860/SRR8240860.1  GSM3487689       whole worm  fer-15(b26ts)   Adult day 1         N/A            http://ftp.sra.ebi.ac.uk/vol1/fastq/SRR824/000/SRR8240860/SRR8240860_1.fastq.gz  http://ftp.sra.ebi.ac.uk/vol1/fastq/SRR824/000/SRR8240860/SRR8240860_2.fastq.gz  N/A           era-fasp@fasp.sra.ebi.ac.uk:vol1/fastq/SRR824/000/SRR8240860/SRR8240860_1.fastq.gz  era-fasp@fasp.sra.ebi.ac.uk:vol1/fastq/SRR824/000/SRR8240860/SRR8240860_2.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="161" LastActivityDate="2020-09-16T20:48:34.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14380" PostTypeId="2" ParentId="14361" CreationDate="2020-09-16T21:06:33.923" Score="1" Body="&lt;p&gt;If you're reading this, I can only assume that's because you had trouble installing HiC-Pro too. I was finally able to install from source and run the software, so here is how I did it, giving detail to the error that I was hit with and the clues I used to solve the puzzles and get through installing this program:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;From Source To install from source, you need to supply the paths to&#xA;its dependencies. Believing we did a good job, we run it and find&#xA;problems accessing bx-python.&lt;/p&gt;&#xA;&lt;p&gt;No matter how many times we update pip and try to install bx-python&#xA;and specify that we want it installed for p&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h1&gt;Setup&lt;/h1&gt;&#xA;&lt;p&gt;I downloaded from source once again using &lt;code&gt;git clone&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The I &lt;code&gt;cat&lt;/code&gt;ed the config file to find the programs I needed to find the path of, which I did using &lt;code&gt;which&lt;/code&gt;. I ensured that I was using python2.&lt;/p&gt;&#xA;&lt;p&gt;I also used &lt;code&gt;pip2&lt;/code&gt; to install pysam, numpy, and scipy. I needed to use sudo to install pysam, though. I also installed something called bx-python.&lt;/p&gt;&#xA;&lt;h1&gt;Make Configure&lt;/h1&gt;&#xA;&lt;p&gt;Here is the error I get when I run &lt;code&gt;make configure&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Checking Python libraries ...&#xA;Traceback (most recent call last):&#xA;File &amp;quot;../scripts/install/check_pythonlib.py&amp;quot;, line 24, in &amp;lt;module&amp;gt;&#xA;      import bx.intervals&#xA;  ModuleNotFoundError: No module named 'bx'&#xA;&#xA;  Traceback (most recent call last):&#xA;    File &amp;quot;../scripts/install/check_pythonlib.py&amp;quot;, line 28, in &amp;lt;module&amp;gt;&#xA;      raise ImportError('Error - bx-python cannot be imported')  &#xA;ImportError: Error - bx-python cannot be imported&#xA;Can not proceed without the required Python libraries, please install them and re-run&#xA;make[1]: *** [configure] Error 1&#xA;make: *** [configure] Error 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Looking at this error for the fifth time, I realize that the first part, and not the last part, is the important one. It's yelling at me that I don't have the bx module installed. It also gives me the path to the file that threw the exception, so I read it and saw that this was what was being run:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## bx&#xA;try:&#xA; import bx.intervals&#xA; if vcmp(bx.__version__, '0.5.0') == -1:&#xA;   raise ValueError('bx-python '+bx.__version__+' detected. Version &amp;gt;= 0.5.0 required')&#xA; except ImportError:&#xA;   raise ImportError('Error - bx-python cannot be imported')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I notice that there's nothing fancy, the thing just needs to be imported, and its actual name is bx. Wonderful, I pip2 it. But... I hit another error in pip:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pyrsistent requires Python '&amp;gt;=3.5' but the running Python is 2.7.16&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/aws/aws-elastic-beanstalk-cli/issues/48&quot; rel=&quot;nofollow noreferrer&quot;&gt;Apparently&lt;/a&gt;, pyrsistent dropped support for python2. Wonderful. I get around it by:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sudo pip2 install awsebcli pyrsistent==0.16.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In the middle of this installation, Im reading the output and see:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;bx-python 0.8.9 has requirement six&amp;gt;=1.13.0, but you'll have six 1.11.0 which is incompatible.&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;So:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;sudo pip2 install six==1.13.0&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;At this point, I, without any more problems, &lt;code&gt;sudo pip2 install bx&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I run &lt;code&gt;make configure&lt;/code&gt; again, and I run into...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Checking Python libraries ...&#xA;Traceback (most recent call last):&#xA;File &amp;quot;../scripts/install/check_pythonlib.py&amp;quot;, line 24, in &amp;lt;module&amp;gt;&#xA;      import bx.intervals&#xA;  ModuleNotFoundError: No module named 'bx'&#xA;&#xA;  Traceback (most recent call last):&#xA;    File &amp;quot;../scripts/install/check_pythonlib.py&amp;quot;, line 28, in &amp;lt;module&amp;gt;&#xA;      raise ImportError('Error - bx-python cannot be imported')  &#xA;ImportError: Error - bx-python cannot be imported&#xA;Can not proceed without the required Python libraries, please install them and re-run&#xA;make[1]: *** [configure] Error 1&#xA;make: *** [configure] Error 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;huh? Didn't I fix this?? Something seems fishy. For whatever reason, my mind then wanders to the Makefile to see if anything there pops out, cause... what else would be calling that python file? In the main Makefile I see:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;make -f ./scripts/install/Makefile CONFIG_SYS=$(CONFIG_SYS)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So I head on over to ./scripts/install/Makefile and see:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./scripts/install/install_dependencies.sh -c &lt;span class=&quot;math-container&quot;&gt;$(CONFIG_SYS) -o  $&lt;/span&gt;(realpath &lt;span class=&quot;math-container&quot;&gt;$(PREFIX))/HiC-Proo    _$&lt;/span&gt;(VNUM) -q&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Nothing like a goose chase. So I head on over to install_dependencies.sh, where I see:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# python&#xA;which python &amp;gt; /dev/null;&#xA;if [ &lt;span class=&quot;math-container&quot;&gt;$? != &quot;0&quot; ]; then&#xA;     echo -e &quot;$&lt;/span&gt;RED&amp;quot;&amp;quot;Can not proceed without Python, please install and re-run&amp;quot;&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$NORMAL&quot;&#xA;     exit 1;&#xA;else&#xA;     pver=`python --version 2&amp;gt;&amp;amp;1 | cut -d&quot; &quot; -f2` #&amp;lt;- THIS LINE&#xA;     vercomp $&lt;/span&gt;pver &amp;quot;2.7.0&amp;quot;&#xA;     if [[ &lt;span class=&quot;math-container&quot;&gt;$? == 2 ]]; then&#xA;         echo -e &quot;$&lt;/span&gt;RED&amp;quot;&amp;quot;Python v2.7.0 or higher is needed [&lt;span class=&quot;math-container&quot;&gt;$pver detected].&quot;&quot;$&lt;/span&gt;NORMAL&amp;quot;&#xA;         exit 1;&#xA;     fi&#xA;fi&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I check that lines output, AND IT'S PYTHON3!! For some reason, it didn't give me that error message, it just kept going from that line.&lt;/p&gt;&#xA;&lt;p&gt;So, I changed all instances of &lt;code&gt;python&lt;/code&gt; with &lt;code&gt;python2&lt;/code&gt; in that file.&lt;/p&gt;&#xA;&lt;p&gt;I reran &lt;code&gt;make configure&lt;/code&gt; and got &lt;code&gt;The required Python libraries appear to be already installed.&lt;/code&gt; Bliss.&lt;/p&gt;&#xA;&lt;h1&gt;Make Install&lt;/h1&gt;&#xA;&lt;p&gt;Next step was &lt;code&gt;make install&lt;/code&gt;, and guess what? A roadblock:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;File &amp;quot;setup.py&amp;quot;, line 2, in &amp;lt;module&amp;gt;&#xA;from numpy.distutils.core import setup&#xA;  ModuleNotFoundError: No module named 'numpy'&#xA;make: *** [iced] Error 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Hmm... well, it's probably the same thing. python3 being called where python2 should be. So I double checked the main Makefile and changed the setup.py call to use python2.&lt;/p&gt;&#xA;&lt;p&gt;I run &lt;code&gt;make install&lt;/code&gt; and it runs for awhile, chugging along using the g++ compiler. Then I hit an odd error:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;cp: /Users/bosslab/Downloads/HiC-Pro-master/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4/HiC-Pro_2.11.4: name too long (not copied)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm guessing that the Makefile has a recursive path somewhere. I check the appropriate section and find:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;cp -Ri $(MK_PATH) $(INSTALL_PATH)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I simply replace that line with the actual paths, with the install_path being adjacent instead of inside the mk_path.&lt;/p&gt;&#xA;&lt;p&gt;Then I get hit with:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;HiC-Pro installed in /Users/bosslab/Downloads/HiC-Pro-master/HiC-Pro_2.11.4 !&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;So running &lt;code&gt;$install_dir/HiC-Pro --help&lt;/code&gt; returns the help. YAY!&lt;/p&gt;&#xA;&lt;h1&gt;Leftovers&lt;/h1&gt;&#xA;&lt;p&gt;I installed it, but how about when I actually run it?&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Error: The 'ice' command is not in your path. Please check where the 'iced' python package has been installed and update your PATH !&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Ok:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ sudo pip2 install iced&#xA;Requirement already satisfied: iced in /Users/bosslab/Library/Python/2.7/lib/python/site-packages (0.4.2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;...so, I set up a test to see whether it actually is installed:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ python2&#xA;Python 2.7.16 (default, Nov  9 2019, 05:55:08) &#xA;[GCC 4.2.1 Compatible Apple LLVM 11.0.0 (clang-1100.0.32.4) (-macos10.15-objc-s on darwin&#xA;Type &amp;quot;help&amp;quot;, &amp;quot;copyright&amp;quot;, &amp;quot;credits&amp;quot; or &amp;quot;license&amp;quot; for more information.&#xA;&amp;gt;&amp;gt;&amp;gt; iimport iced&#xA;&amp;gt;&amp;gt;&amp;gt; quit()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It's already installed. But then I had an epiphany:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ python3&#xA;Python 3.8.5 (default, Jul 21 2020, 10:48:26) &#xA;[Clang 11.0.3 (clang-1103.0.32.62)] on darwin&#xA;Type &amp;quot;help&amp;quot;, &amp;quot;copyright&amp;quot;, &amp;quot;credits&amp;quot; or &amp;quot;license&amp;quot; for more information.&#xA;&amp;gt;&amp;gt;&amp;gt; ipomport iced&#xA;Traceback (most recent call last):&#xA;File &amp;quot;&amp;lt;stdin&amp;gt;&amp;quot;, line 1, in &amp;lt;module&amp;gt;&#xA;ModuleNotFoundError: No module named 'iced'&#xA;&amp;gt;&amp;gt;&amp;gt; quit()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Aha! Somewhere (i don't know where though) python3 is being called. So I just installed that module for python3 with &lt;code&gt;sudo pip3 install iced&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Then I had some trouble with pointing to the right bowtie index (just give the directory where the index exists) and then I hit my next error dealing with HiC-Pro itself:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Pairing of R1 and R2 tags ...&#xA;Logs: logs/dixon_2M/mergeSAM.log&#xA;make: *** [bowtie_pairing] Error 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Checking the log file named in the error message:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/usr/local/bin/python /Users/bosslab/Downloads/HiC-Pro-master/HiC-Pro_2.11.4/scripts/mergeSAM. py -q 0 -t -v -f bowtie_results/bwt2/dixon_2M/SRR400264_00_R1_hg19.bwt2merged.bam -r bowtie_re sults/bwt2/dixon_2M/SRR400264_00_R2_hg19.bwt2merged.bam -o bowtie_results/bwt2/dixon_2M/SRR400 264_00_hg19.bwt2pairs.bam&#xA;   File &amp;quot;/Users/bosslab/Downloads/HiC-Pro-master/HiC-Pro_2.11.4/scripts/mergeSAM.py&amp;quot;, line 26&#xA;   print &amp;quot;Usage : python mergeSAM.py&amp;quot;&#xA;    ^&#xA;SyntaxError: Missing parentheses in call to 'print'. Did you mean print(&amp;quot;Usage : python mergeS AM.py&amp;quot;)?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Missing parentheses? Oh my god, it's using python3 for some reason again. I grep the installation directory for a file that calls merge_sam.py, and find config-system.txt. I go in there and just need to change the python path from &lt;code&gt;/usr/bin/local/&lt;/code&gt; where python3 is installed to &lt;code&gt;/usr/bin/&lt;/code&gt; where python2 is installed.&lt;/p&gt;&#xA;&lt;p&gt;Lastly, I get an error during the ICE normalization step that numpy is not installed. Using my big fat brain, I isntall numpy to my python3 and call it a day. Now it all works as intended.&lt;/p&gt;&#xA;" OwnerUserId="9521" LastActivityDate="2020-09-16T21:06:33.923" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14382" PostTypeId="2" ParentId="14367" CreationDate="2020-09-17T02:06:40.783" Score="2" Body="&lt;p&gt;A tidyverse answer.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(&amp;quot;tidyverse&amp;quot;)&#xA;&#xA;results &amp;lt;- df %&amp;gt;%&#xA;  {gtools::combinations(ncol(.), 2, colnames(.))} %&amp;gt;%&#xA;  as_tibble(.name_repair=&amp;quot;unique&amp;quot;) %&amp;gt;%&#xA;  rename(col1=1, col2=2) %&amp;gt;%&#xA;  rowwise %&amp;gt;%&#xA;  mutate(intersection=list(intersect({{df}}[[col1]], {{df}}[[col2]]))) %&amp;gt;%&#xA;  unnest_wider(col=intersection, names_repair=&amp;quot;unique&amp;quot;) %&amp;gt;%&#xA;  pivot_longer(starts_with(&amp;quot;...&amp;quot;), names_to=&amp;quot;col&amp;quot;, values_to=&amp;quot;val&amp;quot;) %&amp;gt;%&#xA;  pivot_wider(names_from=c(col1, col2), names_sep=&amp;quot;_&amp;quot;, values_from=val) %&amp;gt;%&#xA;  select(!col)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The first few results.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; results[1:5, 1:5]&#xA;# A tibble: 5 x 5&#xA;  C0023470_C00234~ C0023470_C00269~ C0023470_C00270~ C0023470_C01538~&#xA;             &amp;lt;int&amp;gt;            &amp;lt;int&amp;gt;            &amp;lt;int&amp;gt;            &amp;lt;int&amp;gt;&#xA;1         26837842               NA               NA         30024784&#xA;2               NA               NA               NA               NA&#xA;3               NA               NA               NA               NA&#xA;4               NA               NA               NA               NA&#xA;5               NA               NA               NA               NA&#xA;# ... with 1 more variable: C0023470_C0279094 &amp;lt;int&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9471" LastEditorUserId="9471" LastEditDate="2020-09-17T12:29:00.160" LastActivityDate="2020-09-17T12:29:00.160" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="14384" PostTypeId="1" AcceptedAnswerId="14389" CreationDate="2020-09-17T07:21:15.930" Score="0" ViewCount="1138" Body="&lt;p&gt;The &lt;a href=&quot;http://www.pantherdb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PANTHER&lt;/a&gt; website offers a tool to obtain the GO-based overrepresentation of a gene list (the analyzed list) versus a reference gene list. After entering both these lists, the tool asks to choose between the Fisher's Exact or Binomial tests.&lt;/p&gt;&#xA;&lt;p&gt;I guess that the convenience for one or another test would depend on the number of genes to analyze. If that is the case, which would be the ideal or advisable rank of gene numbers in both analyzed and reference lists (or their ratio) for each type of test?&lt;/p&gt;&#xA;&lt;p&gt;(In particular I am working with a list of around 150 versus a reference list of around 2000 genes).&lt;/p&gt;&#xA;" OwnerUserId="4543" LastActivityDate="2020-09-18T00:04:35.063" Title="Choosing Fisher's Exact or Binomial test for overrepresentation in PANTHER" Tags="|gene|statistics|go|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14385" PostTypeId="1" AcceptedAnswerId="14415" CreationDate="2020-09-17T16:15:18.153" Score="3" ViewCount="354" Body="&lt;p&gt;I currently use the 'UniProt website REST API' (&lt;a href=&quot;https://www.uniprot.org/uniref/UniRef100_P31946.xml&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt;) to collect the UniParc IDs of all the members of the given cluster. Fetching each entry individually is very slow though, so I wanted to ask if there is a way that I can do this request in batches.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What I have already tried:&lt;/strong&gt;&#xA;There are pages mentioning a &lt;a href=&quot;https://www.uniprot.org/help/api_batch_retrieval&quot; rel=&quot;nofollow noreferrer&quot;&gt;batch retrieval of entries&lt;/a&gt;. As I understood it, all the examples only adress the ID mapping service. Here I have tried adjusting the given example to the following&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import urllib.parse&#xA;import urllib.request&#xA;&#xA;url = 'https://www.uniprot.org/uploadlists/'&#xA;&#xA;params = {&#xA;'from': 'NF100',&#xA;'to': 'UPARC',&#xA;'format': 'tab',&#xA;'query': 'UniRef100_Q8RKD8'&#xA;}&#xA;&#xA;data = urllib.parse.urlencode(params)&#xA;data = data.encode('utf-8')&#xA;req = urllib.request.Request(url, data)&#xA;with urllib.request.urlopen(req) as f:&#xA;   response = f.read()&#xA;print(response.decode('utf-8'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which resulted in the output&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;From    To&#xA;UniRef100_Q8RKD8        null&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also checked the interface of this service on the website and there I can't seem to map NF100 to UPARC, so I'm assuming that is why I'm not getting a result via the API.&lt;/p&gt;&#xA;&lt;p&gt;I also know that the UniRef Id contains either a UniParc ID or the UniProtKB accession, so I could use the UniParc ID directly or fetch it for the UniProtKB via the &lt;a href=&quot;https://www.ebi.ac.uk/proteins/api/uniparc/accession/P31946.json&quot; rel=&quot;nofollow noreferrer&quot;&gt;proteins api with the uniparc endpoint&lt;/a&gt;, but from what I can see I only get the UniParc ID/entry of one of the members.&lt;/p&gt;&#xA;" OwnerUserId="4441" LastActivityDate="2022-10-18T07:29:41.423" Title="How to get the uniparc ids for all members of uniref cluster in batch request?" Tags="|python|uniprot|embl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14386" PostTypeId="1" CreationDate="2020-09-17T19:35:24.053" Score="0" ViewCount="165" Body="&lt;p&gt;I am trying to donwload the microbial genome using the repophlan_get_microbes.py (&lt;a href=&quot;https://github.com/SegataLab/repophlan&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/SegataLab/repophlan&lt;/a&gt;) but now it is running more than 10 days and still not finished on slurm HPC.&lt;/p&gt;&#xA;&lt;p&gt;Could anyone please suggest what is the issue with the script?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;#SBATCH --job-name=download.GENOME    # Job name&#xA;#SBATCH --ntasks=60&#xA;#SBATCH --nodes=3                  # Number of nodes           &#xA;#SBATCH --time=2000:00:00              # Time limit hrs:min:sec&#xA;#SBATCH --partition=k2-himem&#xA;#SBATCH --mem=800G&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;module add apps/python3/3.7.4/gcc-4.8.5&#xA;module add libs/biopython/1.73/gcc-4.8.5+python-2.7.8+numpy-1.16.0&#xA;&#xA;&#xA;python repophlan_get_microbes.py --taxonomy taxonomy_reduced.txt --out_dir microbes --nproc 20 --out_summary repophlan_microbes.txt | tee repophlan_microbes.log&#xA;&#xA;$ head repophlan_microbes.log&#xA;2020-09-17 20:44:59,964 ./repophlan_get_microbes.py INFO     Reading the taxonomy from taxonomy_reduced.txt...&#xA;2020-09-17 20:45:04,921 ./repophlan_get_microbes.py INFO     Done.&#xA;2020-09-17 20:45:10,716 ./repophlan_get_microbes.py WARNING  GCF_000001215.4 [Drosophila melanogaster ] excluded from download because of uninteresting phyla!&#xA;2020-09-17 20:45:10,716 ./repophlan_get_microbes.py WARNING  GCF_000001405.39 [Homo sapiens ] excluded from download because of uninteresting phyla!&#xA;2020-09-17 20:45:10,716 ./repophlan_get_microbes.py WARNING  GCF_000001635.26 [Mus musculus ] excluded from download because of uninteresting phyla!&#xA;2020-09-17 20:45:10,716 ./repophlan_get_microbes.py WARNING  GCF_000001735.4 [Arabidopsis thaliana ecotype=Columbia] excluded from download because of uninteresting phyla!&#xA;2020-09-17 20:45:10,716 ./repophlan_get_microbes.py WARNING  GCF_000001895.5 [Rattus norvegicus strain=mixed] excluded from download because of uninteresting phyla!&#xA;2020-09-17 20:45:10,716 ./repophlan_get_microbes.py WARNING  GCF_000001905.1 [Loxodonta africana ] excluded from download because of uninteresting phyla!&#xA;2020-09-17 20:45:10,716 ./repophlan_get_microbes.py WARNING  GCF_000002035.6 [Danio rerio ] excluded from download because of uninteresting phyla!&#xA;2020-09-17 20:45:10,716 ./repophlan_get_microbes.py WARNING  GCF_000002075.1 [Aplysia californica ] excluded from download because of uninteresting phyla!&#xA;&#xA;&#xA;$ tail repophlan_microbes.log&#xA;&#xA;&#xA;2020-09-21 20:42:26,871 repophlan_get_microbes.py INFO     Parsing of ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/001/635/075/GCA_001635075.1_ASM163507v1/GCA_001635075.1_ASM163507v1_genomic.gbff.gz&#xA;2020-09-21 20:42:27,198 repophlan_get_microbes.py INFO     Parsing of ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/010/319/175/GCA_010319175.1_PDT000567389.1/GCA_010319175.1_PDT000567389.1_genomic.gbff.gz&#xA;2020-09-21 20:42:27,482 repophlan_get_microbes.py INFO     Parsing of ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/003/518/025/GCA_003518025.1_ASM351802v1/GCA_003518025.1_ASM351802v1_protein.faa.gz&#xA;2020-09-21 20:42:27,608 repophlan_get_microbes.py INFO     Parsing of ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/001/788/535/GCA_001788535.1_ASM178853v1/GCA_001788535.1_ASM178853v1_protein.faa.gz&#xA;2020-09-21 20:42:27,628 repophlan_get_microbes.py WARNING  No remote file found (some ffn and faa are know to be missing remotely). Aborting the download of ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/010/997/445/GCA_010997445.1_ASM1099744v1/GCA_010997445.1_ASM1099744v1_protein.faa.gz. &amp;lt;urlopen error ftp error: [Errno ftp error] 550 GCA_010997445.1_ASM1099744v1_protein.faa.gz: No such file or directory&amp;gt;&#xA;2020-09-21 20:42:27,628 repophlan_get_microbes.py ERROR    Error in downloading of ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/010/997/445/GCA_010997445.1_ASM1099744v1/GCA_010997445.1_ASM1099744v1_protein.faa.gz &amp;lt;urlopen error ftp error: [Errno ftp error] 550 GCA_010997445.1_ASM1099744v1_protein.faa.gz: No such file or directory&amp;gt;&#xA;2020-09-21 20:42:27,628 repophlan_get_microbes.py ERROR    Error in downloading of ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/010/997/445/GCA_010997445.1_ASM1099744v1/GCA_010997445.1_ASM1099744v1_protein.faa.gz: empty fna file!&#xA;2020-09-21 20:42:28,239 repophlan_get_microbes.py WARNING  No remote file found (some ffn and faa are know to be missing remotely). Aborting the download of ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/902/778/895/GCA_902778895.1_Rumen_uncultured_genome_RUG12414/GCA_902778895.1_Rumen_uncultured_genome_RUG12414_protein.faa.gz. &amp;lt;urlopen error ftp error: [Errno ftp error] 550 GCA_902778895.1_Rumen_uncultured_genome_RUG12414_protein.faa.gz: No such file or directory&amp;gt;&#xA;2020-09-21 20:42:28,239 repophlan_get_microbes.py ERROR    Error in downloading of ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/902/778/895/GCA_902778895.1_Rumen_uncultured_genome_RUG12414/GCA_902778895.1_Rumen_uncultured_genome_RUG12414_protein.faa.gz &amp;lt;urlopen error ftp error: [Errno ftp error] 550 GCA_902778895.1_Rumen_uncultured_genome_RUG12414_protein.faa.gz: No such file or directory&amp;gt;&#xA;2020-09-21 20:42:28,239 repophlan_get_microbes.py ERROR    Error in downloading of ftp://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/902/778/895/GCA_902778895.1_Rumen_uncultured_genome_RUG12414/GCA_902778895.1_Rumen_uncultured_genome_RUG12414_protein.faa.gz: empty fna file!&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Many thanks&lt;/p&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="1008" LastEditDate="2020-09-21T19:51:14.170" LastActivityDate="2020-09-21T20:25:41.180" Title="repophlan script to download bacterial genome" Tags="|python|bash|microbial-genomics|hpc|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14388" PostTypeId="1" AcceptedAnswerId="14390" CreationDate="2020-09-17T23:28:45.830" Score="2" ViewCount="75" Body="&lt;p&gt;My operating system is Mac OS X.&lt;/p&gt;&#xA;&lt;p&gt;I would like to compute DNA distances between different sequences. For this, the dnadist package from the Phylip software would be good but hard to automate. &lt;a href=&quot;http://emboss.sourceforge.net/apps/release/6.0/embassy/phylipnew/fdnadist.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;EMBOSS&lt;/a&gt; seems to have an fdnadist command for this. I have installed emboss version 6.6.0 from the bioconda package but if I type &lt;code&gt;fdnadist&lt;/code&gt;, the command is not found. How can I use &lt;code&gt;dnadist&lt;/code&gt; or &lt;code&gt;fdnadist&lt;/code&gt; on the command line?&lt;/p&gt;&#xA;" OwnerUserId="1075" LastActivityDate="2023-08-16T13:06:22.213" Title="How can I run the fdnadist command from the EMBOSS package?" Tags="|sequence-analysis|phylogenetics|software-installation|emboss|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14389" PostTypeId="2" ParentId="14384" CreationDate="2020-09-18T00:04:35.063" Score="2" Body="&lt;p&gt;For a guide on using the tests in PANTHER, see &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6519457/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6519457/&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In answer to this specific question, the defaults are the best choice (Fisher's Exact Test, with FDR correction). Fisher's Exact test gives more accurate statistics for relatively small counts (e.g. when only a few genes in your list are associated with a particular GO term). The FDR is a better correction to use for these types of analysis in general, especially given the large overlap between classes.&lt;/p&gt;&#xA;&lt;p&gt;The other options are there mostly to support replication of older analyses, e.g. to replicate a published analysis that used the PANTHER over-representation tool prior to 2018, when the binomial with Bonferroni correction was the default.&lt;/p&gt;&#xA;" OwnerUserId="9544" LastActivityDate="2020-09-18T00:04:35.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14390" PostTypeId="2" ParentId="14388" CreationDate="2020-09-18T00:12:43.933" Score="2" Body="&lt;p&gt;To find &lt;code&gt;emboss&lt;/code&gt; packages type &lt;code&gt;wossname&lt;/code&gt;, then hit enter twice.  You will see all the programs.  There is one called &lt;code&gt;distmat&lt;/code&gt;.  Then do &lt;code&gt;tfm distmat&lt;/code&gt;, to get &lt;code&gt;the fkn manual&lt;/code&gt; for &lt;code&gt;distmat&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3381" LastEditorUserId="1075" LastEditDate="2020-09-18T13:04:37.950" LastActivityDate="2020-09-18T13:04:37.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14391" PostTypeId="2" ParentId="14352" CreationDate="2020-09-18T00:27:19.487" Score="0" Body="&lt;p&gt;You can use &lt;code&gt;progressiveMauve&lt;/code&gt; (&lt;a href=&quot;http://darlinglab.org/mauve/user-guide/progressivemauve.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://darlinglab.org/mauve/user-guide/progressivemauve.html&lt;/a&gt;), or &lt;code&gt;bubbz&lt;/code&gt; (&lt;a href=&quot;https://www.cell.com/iscience/fulltext/S2589-0042%2820%2930409-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cell.com/iscience/fulltext/S2589-0042%2820%2930409-0&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="3381" LastActivityDate="2020-09-18T00:27:19.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14392" PostTypeId="1" CreationDate="2020-09-18T09:56:36.477" Score="3" ViewCount="93" Body="&lt;p&gt;The genome sequence for the GenBank virus JX869059.2 (a human betacoronavirus) contains, among others, two proteins:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;N protein (range 28566..29807) and&lt;/li&gt;&#xA;&lt;li&gt;Orf8b (range 28762..29100).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The interval for Orf8b is contained in the interval for N.&lt;/p&gt;&#xA;&lt;p&gt;How are these proteins translated? Is the orf8 protein created with post-translational processing after N is translated, or does the ribosome translate the two proteins separately?&lt;/p&gt;&#xA;" OwnerUserId="7054" LastEditorUserId="3848" LastEditDate="2020-12-03T16:35:34.913" LastActivityDate="2020-12-03T16:44:50.960" Title="Virus genome: protein inside another protein" Tags="|ngs|covid-19|sars-cov-2|coronavirus|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14393" PostTypeId="1" AcceptedAnswerId="14410" CreationDate="2020-09-18T12:50:35.900" Score="1" ViewCount="284" Body="&lt;p&gt;How to predict the half-maximal inhibitory concentration (IC50) of my ligands.&lt;/p&gt;&#xA;&lt;p&gt;I'm currently using Schrodinger suite software and I don't know how to predict the IC50?&lt;/p&gt;&#xA;" OwnerUserId="4794" LastEditorUserId="57" LastEditDate="2020-09-24T11:29:57.810" LastActivityDate="2020-09-29T16:09:22.483" Title="predict the IC50 after using Schrodinger suite Glide tool for docking" Tags="|docking|" AnswerCount="1" CommentCount="1" ClosedDate="2020-10-02T17:42:21.397" ContentLicense="CC BY-SA 4.0" />
  <row Id="14395" PostTypeId="2" ParentId="14386" CreationDate="2020-09-18T18:59:08.163" Score="1" Body="&lt;p&gt;&lt;strong&gt;UPDATE 2: new log posted&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;take-home message: I think it is working, it's just that a few errors happen sometimes when downloading things from big databases.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It looks like there are a few warnings or errors associated with getting &lt;em&gt;specific&lt;/em&gt; files. I think this is actually ok. For example for one of the genomes, I logged into the ftp manually to check if the file was there:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ ftp ftp://ftp.ncbi.nlm.nih.gov&#xA;Trying 130.14.250.13...&#xA;Connected to ftp.wip.ncbi.nlm.nih.gov.&#xA;220-&#xA; This warning banner provides privacy and security notices consistent with &#xA; applicable federal laws, directives, and other federal guidance for accessing &#xA; this Government system, which includes all devices/storage media attached to &#xA; this system. This system is provided for Government-authorized use only. &#xA; Unauthorized or improper use of this system is prohibited and may result in &#xA; disciplinary action and/or civil and criminal penalties. At any time, and for &#xA; any lawful Government purpose, the government may monitor, record, and audit &#xA; your system usage and/or intercept, search and seize any communication or data &#xA; transiting or stored on this system. Therefore, you have no reasonable &#xA; expectation of privacy. Any communication or data transiting or stored on this &#xA; system may be disclosed or used for any lawful Government purpose.&#xA;220 FTP Server ready.&#xA;331 Anonymous login ok, send your complete email address as your password&#xA;230 Anonymous access granted, restrictions apply&#xA;Remote system type is UNIX.&#xA;Using binary mode to transfer files.&#xA;200 Type set to I&#xA;ftp&amp;gt; get genomes/all/GCA/902/778/895/GCA_902778895.1_Rumen_uncultured_genome_RUG12414/GCA_902778895.1_Rumen_uncultured_genome_RUG12414_protein.faa.gz&#xA;local: genomes/all/GCA/902/778/895/GCA_902778895.1_Rumen_uncultured_genome_RUG12414/GCA_902778895.1_Rumen_uncultured_genome_RUG12414_protein.faa.gz remote: genomes/all/GCA/902/778/895/GCA_902778895.1_Rumen_uncultured_genome_RUG12414/GCA_902778895.1_Rumen_uncultured_genome_RUG12414_protein.faa.gz&#xA;ftp: Can't access `genomes/all/GCA/902/778/895/GCA_902778895.1_Rumen_uncultured_genome_RUG12414/GCA_902778895.1_Rumen_uncultured_genome_RUG12414_protein.faa.gz': No such file or directory&#xA;&#xA;ftp&amp;gt; ls genomes/all/GCA/902/778/895/GCA_902778895.1_Rumen_uncultured_genome_RUG12414/&#xA;&#xA;229 Entering Extended Passive Mode (|||50025|)&#xA;150 Opening ASCII mode data connection for file list&#xA;-r--r--r--   1 ftp      anonymous     2725 Feb 23  2020 GCA_902778895.1_Rumen_uncultured_genome_RUG12414_assembly_report.txt&#xA;-r--r--r--   1 ftp      anonymous     3456 Feb 23  2020 GCA_902778895.1_Rumen_uncultured_genome_RUG12414_assembly_stats.txt&#xA;dr-xr-xr-x   3 ftp      anonymous     4096 Feb 23  2020 GCA_902778895.1_Rumen_uncultured_genome_RUG12414_assembly_structure&#xA;-r--r--r--   1 ftp      anonymous      149 Apr 21 19:29 GCA_902778895.1_Rumen_uncultured_genome_RUG12414_feature_count.txt.gz&#xA;-r--r--r--   1 ftp      anonymous   949241 Feb 23  2020 GCA_902778895.1_Rumen_uncultured_genome_RUG12414_genomic.fna.gz&#xA;-r--r--r--   1 ftp      anonymous  1353865 Feb 23  2020 GCA_902778895.1_Rumen_uncultured_genome_RUG12414_genomic.gbff.gz&#xA;-r--r--r--   1 ftp      anonymous      150 Feb 23  2020 GCA_902778895.1_Rumen_uncultured_genome_RUG12414_genomic_gaps.txt.gz&#xA;-r--r--r--   1 ftp      anonymous      840 Feb 23  2020 GCA_902778895.1_Rumen_uncultured_genome_RUG12414_wgsmaster.gbff.gz&#xA;-r--r--r--   1 ftp      anonymous    44529 Sep  2 20:26 README.txt&#xA;-r--r--r--   1 ftp      anonymous      410 Feb 23  2020 annotation_hashes.txt&#xA;-r--r--r--   1 ftp      anonymous       14 Sep 21 07:11 assembly_status.txt&#xA;-r--r--r--   1 ftp      anonymous     1399 May 20 23:00 md5checksums.txt&#xA;226 Transfer complete&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So in this particular case, that genome &lt;em&gt;exists&lt;/em&gt; but is just missing a &lt;code&gt;faa.gz&lt;/code&gt; file. So &lt;strong&gt;the script appears to be working as expected&lt;/strong&gt;, it's just that some of the NCBI submissions don't have everything they are supposed to have.&lt;/p&gt;&#xA;&lt;p&gt;(As a tangent, that Rumen genome came from a paper I'm an author on I'm pretty sure!! So I guess you could blame me for the missing file if you wanted, though I was in no way involved with annotations or submission...)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE: now the log is posted&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It does not look like a connectivity issue based on the new information provided in the first and last lines of the log file. There it looks like there are a lot of processes finishing quickly towards the beginning and the end of the process (e.g. 48 hours apart).&lt;/p&gt;&#xA;&lt;p&gt;Based on this new information, I am wondering if you have simply managed to get a taxonomy file that is much larger (e.g. &amp;gt;10X larger to explain the time difference) than anticipated when the README for the script was written, and thus it simply takes a long time.&lt;/p&gt;&#xA;&lt;p&gt;It would still probably be best to go through the log file rather carefully and see if there is a big hang in the process, errors, warnings, etc.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Here is what they say in the repo:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;*** Download the microbial (bacteria+archaea) genomes (fna,ffn,faa,frn) with &#xA;    corresponding taxonomy&#xA;$ ./repophlan_get_microbes.py --taxonomy taxonomy_reduced.txt --out_dir /&#xA;  microbes --nproc 20 --out_summary repophlan_microbes.txt /&#xA;  | tee repophlan_microbes.log&#xA;[ ~6 hours ]. Downloads and save all available sequences for microbes. It uses&#xA;multiple parallel connections to speed up the retrieval: common exceptions &#xA;including issues with the NCBI ftp which temporarily rejects connections for&#xA;ftp load problems are handled gracefully with a 'delay and retry' policy. &#xA;IMPORTANT: specifying more than 20 processors to run in parallel causes serious&#xA;problems in terms of exceeding the allowed number of connections by NCBI causing&#xA;long delays.&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Since your script is taking much much longer than 6 hours, it probably didn't work.&lt;/p&gt;&#xA;&lt;p&gt;Some thoughts based only on the info in the repo:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&amp;quot;IMPORTANT: specifying more than 20 processors to run in parallel causes serious problems in terms of exceeding the allowed number of connections by NCBI causing long delays.&amp;quot; &lt;strong&gt;this sounds like your problem. I think you are asking for 60 nodes (&amp;quot;tasks&amp;quot;? &amp;quot;nodes&amp;quot; is 3 which should be ok). Drop tasks to 20 or less and see what happens, maybe.&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&amp;quot;...temporarily rejects connections for ftp load problems are handled gracefully with a 'delay and retry' policy.&amp;quot; &lt;strong&gt;It is possible that there is a bad request in there that it keeps trying over and over again unsuccessfully, it could keep going forever if such a problem exists and you don't stop it.&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;As pointed out in the comments, you should check the log and post that here.&lt;/p&gt;&#xA;&lt;p&gt;As also pointed out in the comments, you might consider posting an issue at the repo as they are more familiar than we are.&lt;/p&gt;&#xA;&lt;p&gt;Other things to check:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;how much disk space do your nodes have access to? is it enough for the download?&lt;/li&gt;&#xA;&lt;li&gt;are you satisfying all dependencies of the script? In other words do you need to load more modules?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2020-09-21T20:25:41.180" LastActivityDate="2020-09-21T20:25:41.180" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14398" PostTypeId="1" AcceptedAnswerId="14402" CreationDate="2020-09-19T15:03:58.617" Score="1" ViewCount="253" Body="&lt;p&gt;I did a bunch of GWAS analysis (linear model without covariates) with applying different quality controls. How to choose the optimal settings when filtering for minor allele frequency (maf), Hardy-Weinberg equilibrium (HW) and missing call rates (mcr)? Are there any rules? Here are some examples:&lt;/p&gt;&#xA;&lt;p&gt;Mahnhatten and QQ plot with settings maf=0.02, HW=1e-10, mcr = 0.2:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YEgXO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YEgXO.png&quot; alt=&quot;maf=0.02, HW=1e-10, missing call = 0.2&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lLKB7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lLKB7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Manhattan and QQ plot with settings maf=0.01, HW=0.0001, mcr = 0.01:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Co5r7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Co5r7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/shWXk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/shWXk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Mahnhatten and QQ plot with settings maf=0.02, HW=1e-10, mcr = 0.05:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/eqvnc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eqvnc.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eqvnc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eqvnc.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lOrHz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lOrHz.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9278" LastActivityDate="2020-09-21T13:59:19.157" Title="Interpreting GWAS results with different settings" Tags="|python|gwas|plink|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14400" PostTypeId="2" ParentId="14398" CreationDate="2020-09-19T20:01:04.687" Score="2" Body="&lt;p&gt;Based on the QQ plots, filter out variants with p &amp;gt; 0.01, then display the beta statistic (or whatever other statistic you're using to demonstrate association). If you want to incorporate the p-value in the displayed results, show the minimum test statistic from a 95% confidence interval. Here's an attempt I made at doing that with some GWAS summary statistics relating to household income (something that wouldn't be expected to have a heritable component):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://twitter.com/gringene_bio/status/1207617723586371584&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://twitter.com/gringene_bio/status/1207617723586371584&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;After looking at the test statistic, you may be able to make better decisions about thresholding with the other values. One thing you could try is to incorporate those values into a scatter plot that shows more than two dimensions. The other statistics could be represented as size, colour, or Z location on a 3D scatter plot.&lt;/p&gt;&#xA;&lt;p&gt;P-values should not be overemphasised, used for ranking, or displayed on their own. See the American Statistical Association's report on the p-value for more information:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://amstat.tandfonline.com/doi/full/10.1080/00031305.2016.1154108#_i30&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://amstat.tandfonline.com/doi/full/10.1080/00031305.2016.1154108#_i30&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As far as I'm aware, the p-values as calculated by GWAS software represent uncertainty in the test statistic (i.e. t, or beta), and &lt;em&gt;not&lt;/em&gt; the probability of association. If you want to calculate that association probability, carry out bootstrap sub-sampling of the individuals and then determine the proportion of sub-samples in which the tested variant appears in, say, the highest 5% of variants, ranked by the test statistic. Instead of filtering on HWE, MAF and MCR, you could try filtering on the bootstrap proportion instead, which should incorporate all statistics that introduce uncertainty into the dataset.&lt;/p&gt;&#xA;&lt;p&gt;More details and figures can be found on this poster:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://f1000research.com/posters/5-2190&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://f1000research.com/posters/5-2190&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you're going to be generating a PCA on these data, please consider labelling the cases and controls (or low/high for quantitative traits) on the PCA to verify that the groups look similar in terms of overall population structure. Hopefully you won't find any blobs in the PCA that have only one of the two groups in them, but if you do, they should be dealt with first. Any differences in population structure (even very small, subtle differences) will dominate the GWAS results because structure-associated signal will cover a huge region of the genome in comparison to effects from a few (or a few hundred) SNPs. Here are some papers I've found insightful regarding this issue:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1006/tpbi.2001.1543&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1006/tpbi.2001.1543&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://doi.org/10.1038/s41467-018-05257-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1038/s41467-018-05257-7&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-09-21T11:08:19.177" LastActivityDate="2020-09-21T11:08:19.177" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14401" PostTypeId="2" ParentId="14352" CreationDate="2020-09-19T23:23:57.570" Score="1" Body="&lt;p&gt;TM-align has some options to do this, see for example &lt;a href=&quot;https://zhanglab.ccmb.med.umich.edu/TM-align/help.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://zhanglab.ccmb.med.umich.edu/TM-align/help.html&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;TM-align is similar to TM-score but the alignment is purely structural.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2020-09-19T23:23:57.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14402" PostTypeId="2" ParentId="14398" CreationDate="2020-09-20T18:07:55.287" Score="0" Body="&lt;p&gt;The genomes under analysis are under predominant linkage disequilibrium, i.e. linkage. Chromosome 1 shows strong (or stronger - cut-off depending) HW signal as do one or two telomeres - telomeres are known to undergo recombination, so this may not be unsurprising. Nothing wrong with the HW analysis for the core conclusions, its the stuff between the red and blue lines thats difficult (Q-Q plots).&lt;/p&gt;&#xA;&lt;p&gt;The Q-Q plots are a bit more complex and I would go with the lower graph you presented together with its HW-analysis. There is clearly part of the data you are not accounting for, @gringer says the Q-Q plot conform to the beta distribution .... they should be bang linear. Lets look in more detail,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;0 to 2 ... predominant disequilibrium that looks fine, nice and linear&lt;/li&gt;&#xA;&lt;li&gt;5.5+ .... HW signal that looks ok, nice and linear&lt;/li&gt;&#xA;&lt;li&gt;2 to 5.5 ... complicated&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Given you have a cut-off (red line) at around 5.5 then I'd sort of try and talk your way out of this one. You can do a full investigation, such as transformation to linearise the entire Q-Q plot. Personally I've never done this within HW analysis so I dunno, I've done it for GLM. I'd just say they between the blue and red line represents a complex HW signal which we can't be interpreted but either side of that boundary we're good to go. In other words keep it simple.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-09-21T13:59:19.157" LastActivityDate="2020-09-21T13:59:19.157" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="14403" PostTypeId="2" ParentId="14348" CreationDate="2020-09-21T00:16:32.260" Score="1" Body="&lt;p&gt;In paired-end sequencing, you have an expected fragment length. So if Read 1 maps to two different places, whereas Read 2 only maps to one place. Then you can determine which one is more likely for Read 1 to map to. It cannot be too far away from Read 2 given the expected fragment length.&lt;/p&gt;&#xA;&lt;p&gt;It is not necessarily true that paired-end sequencing gets twice the amount of data. The total amount of data can be determined when running sequencing. If single-end has 20M reads, I can just stop sequencing at 20M as well for paired-end.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-09-21T00:16:32.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14404" PostTypeId="2" ParentId="14348" CreationDate="2020-09-21T19:27:49.040" Score="2" Body="&lt;p&gt;Like you mentioned, having read pairs can help with alignment.&#xA;Your FASTQ for a single-end sequencing run will look something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@SEQ_ID_0001&#xA;AGCTAGCGCGGTTGGCTTAGCGACT&#xA;+&#xA;!''*((((***+))%%%++)(%%%%&#xA;@SEQ_ID_0002&#xA;AGGTGGTTGTAGGGAAAAAAGTCTC&#xA;+&#xA;!''*((((***+))%%%++)(%%%%&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There is not necessarily a relationship between &lt;code&gt;@SEQ_ID_0001&lt;/code&gt; and &lt;code&gt;@SEQ_ID_0002&lt;/code&gt;.&#xA;They could come from neigbouring regions in the genome or they could come from different species if that's what your experiment involves.&#xA;So aligning &lt;code&gt;@SEQ_ID_0001&lt;/code&gt; doesn't exactly give you information about where &lt;code&gt;SEQ_ID_0002&lt;/code&gt; goes.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;@SEQ_ID_0001&lt;/code&gt; could equally map to multiple locations in the genome with the same probability.&#xA;How do we decide where it &amp;quot;should&amp;quot; go?&#xA;We can randomly distribute multi-mapped reads or do some other fancy stuff, but that usually relies on facts about the data as a whole, and not something specific to the original DNA molecule that sequencing read came from.&lt;/p&gt;&#xA;&lt;p&gt;If we have paired-end sequencing, then like your diagram shows, you know that read 1 (&lt;code&gt;R1&lt;/code&gt;) and read 2 (&lt;code&gt;R2&lt;/code&gt;; these have the same &lt;code&gt;@SEQ_ID&lt;/code&gt;) will relate to each other in some way.&#xA;&lt;code&gt;@SEQ_ID_0001 R1&lt;/code&gt; may be multi-mapped, but if &lt;code&gt;@SEQ_ID_0001 R2&lt;/code&gt; isn't, then we know that &lt;code&gt;R1&lt;/code&gt; should probably be relatively close to &lt;code&gt;R2&lt;/code&gt;.&#xA;Because of how you prepare your DNA samples before sequencing, you'll have some information about the insert length distribution.&#xA;You can use this information with sequence aligners to better identify where &lt;code&gt;R1&lt;/code&gt; should be placed.&lt;/p&gt;&#xA;&lt;p&gt;You can also do this if both &lt;code&gt;R1&lt;/code&gt; and &lt;code&gt;R2&lt;/code&gt; are multi-mapped; certain pairwise alignments will be more likely than others, even if the marginal alignment probabilities for each read are the same.&lt;/p&gt;&#xA;&lt;p&gt;Finally, there are other unexpected benefits that you can get from paired-end sequencing.&#xA;All of the above applies to just the mapping step, but applications like structural variant detection and chromosome conformation capture really benefit from paired-end sequencing.&lt;/p&gt;&#xA;&lt;p&gt;If you have single-end reads for structural variant detection, then you will only detect breakpoints if the read overlaps a breakpoint.&#xA;But if you have paired-end, you not only have twice the amount of DNA sequenced to look for breakpoints, you can also estimate if there is a breakpoint between the two reads (in the inner distance from your diagram) if the two reads align really far apart from each other, or on separate chromosomes.&#xA;The idea is similar for chromosome conformation capture and the ligation junction sites.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2020-09-21T19:27:49.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14405" PostTypeId="1" CreationDate="2020-09-22T02:14:54.237" Score="1" ViewCount="154" Body="&lt;p&gt;I am trying to split to a pdbqt file containing 178 compounds into individual pdbqt files. I have been using the Autodock Vina_split.exe file to split the multiple entry file however I keep running into the following error. What can I do to prevent the error.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xuWYB.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xuWYB.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9346" LastActivityDate="2020-09-22T02:14:54.237" Title="Error while splitting PDBQT file" Tags="|proteins|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14406" PostTypeId="1" AcceptedAnswerId="14408" CreationDate="2020-09-22T14:11:43.613" Score="1" ViewCount="45" Body="&lt;p&gt;I'm trying to figure out how I would plot a tree with number of segregating sites on display on the branches&lt;/p&gt;&#xA;&lt;p&gt;I'm using acctran from phangorn to plot a parsimony tree (using phydat object from fasta files)&lt;/p&gt;&#xA;&lt;p&gt;I have been trying to use seg.sites from the ape package to somehow merge these objects together but there has to be some easy way to plot a tree with the information that it was made from&lt;/p&gt;&#xA;" OwnerUserId="5182" LastEditorUserId="3848" LastEditDate="2020-09-22T19:30:58.073" LastActivityDate="2020-09-23T18:14:12.397" Title="Add segregating sites to branches of a tree" Tags="|r|fasta|phylogenetics|linux|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14408" PostTypeId="2" ParentId="14406" CreationDate="2020-09-22T19:22:13.520" Score="1" Body="&lt;p&gt;The &lt;em&gt;segregating sites&lt;/em&gt; in parsimony are formally called &lt;em&gt;steps&lt;/em&gt;. Any number of programs will do this but I don't really use R, particularly for phylogeny, You can of course simply look at the raw treefile, all the data is there (described in the last paragraph). If look at the options for &lt;em&gt;genetic distance&lt;/em&gt;, in parsimony this will be an integer and again the technical name is &lt;em&gt;steps&lt;/em&gt; and is the &lt;em&gt;segreating sites&lt;/em&gt; you want.&lt;/p&gt;&#xA;&lt;p&gt;Python's &lt;code&gt;ETE3&lt;/code&gt; package will work (but you'd need to pipe it through Biopython), as well as specific standalone packages &lt;code&gt;Figtree&lt;/code&gt;, the steps or 'genetic distance' on the branches on the branches option is in the check box. The heavy weight program for tracking step associated phylogeny using parsimony is &lt;code&gt;Mesquite&lt;/code&gt; and its famous grandparent is called &lt;code&gt;MacClade&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I suspect its a nomenclature thing, the options probably talk about &lt;em&gt;genetic distance&lt;/em&gt; because few people do parsimony phylogeny now. Thats the option you want  and you will know because it will be &amp;gt;&amp;gt;1, whilst distance and model-based methods will be between 0 to 1. From a tree viewing it doesn't matter if the genetic distance is 0 to 1 or 1 and beyond because its all proportional. Its just an option thing you are missing. Thus if there is an option, e.g. to include &lt;em&gt;bootstrap support&lt;/em&gt; on the tree, there'll be an option for &lt;em&gt;genetic distance&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Just make sure you don't confuse the number of steps per branch with a consensus statistic, because there're completely different but might give similar looking outputs. Again its easily checked against the raw treefile because the &amp;quot;steps&amp;quot; will be immediately adajacent to the taxa name for the &amp;quot;twigs&amp;quot; (terminal branches) in the tree.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-09-23T18:14:12.397" LastActivityDate="2020-09-23T18:14:12.397" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14409" PostTypeId="2" ParentId="14392" CreationDate="2020-09-22T22:03:06.440" Score="3" Body="&lt;p&gt;Note that the protein sequence of ORF8b is not contained within the protein sequence of protein N. That means that they are translated from different frames (though they are in the same orientation). Indeed, if we just subtract the start coordinates we do not find a multiple of 3.&lt;/p&gt;&#xA;&lt;p&gt;Thus, they cannot be results of the same translational event. This means that the smaller protein could result from translational slippage (as suggested by Matteo Ferla), &amp;quot;leaky scanning&amp;quot; of the translation machines for start codons, or it could simply result from a different ribosome assembly site, recruited via the viral mechanism of an &lt;a href=&quot;https://en.wikipedia.org/wiki/Internal_ribosome_entry_site&quot; rel=&quot;nofollow noreferrer&quot;&gt;internal ribosome entry site&lt;/a&gt;, which allows a ribosome to be recruited in a mRNA cap-independent fashion (though this may be less likely).&lt;/p&gt;&#xA;&lt;p&gt;For some mechanistic details of how one such ORF is translated, see &lt;a href=&quot;https://jvi.asm.org/content/78/17/9325&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It looks like such things are sometimes called &amp;quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Subgenomic_mRNA&quot; rel=&quot;nofollow noreferrer&quot;&gt;subgenomic mRNAs&lt;/a&gt;&amp;quot;. For a paper on such nested genes in coronavirus, including I believe Orf8b, see &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7125658/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-09-22T22:03:06.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14410" PostTypeId="2" ParentId="14393" CreationDate="2020-09-23T02:20:03.463" Score="2" Body="&lt;p&gt;I don't think it is possible just from the docking scores. Docking scores are designed to give you an estimate of the binding affinity (entalphy), but they do not take entropic contributions into account very well. The best you can do are molecular dynamics simulations to calculate the free energy of binding. Which can be computationally quite costly e.g. with GROMACS or CHARMM.&lt;/p&gt;&#xA;&lt;p&gt;You can estimate the IC50 with molecular dynamics simulations as well, but that is not straightforward. E.g. with Umbrella simulations. An example for a potassium channel and a very simple ligand can be found &lt;a href=&quot;https://www.mpibpc.mpg.de/278397/Wacker-Soeren_2008_diploma_thesis_Goe.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Another approach would be to build a predictive model. If you have a library of compounds at hand you could use the chemical structures and e.g. the docking scores to estimate the IC50 based on statistics or machine learning. But biology is very messy in general, though if you get a representative dataset with very similar compounds to the one that you have in your dataset then you might be successful. You have to be sure that the binding site and pose are correct though.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2020-09-29T16:09:22.483" LastActivityDate="2020-09-29T16:09:22.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14411" PostTypeId="1" CreationDate="2020-09-23T12:04:11.607" Score="0" ViewCount="146" Body="&lt;p&gt;I have merged together 2 different .bam files in order to simulate sample contamination. So the reads can come from one of two samples, as shown by the read group info:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@RG ID:0    PL:ILLUMINA SM:LP4100018-DNA_C11_Proband    PU:HGY3WDSXX:1:none&#xA;@RG ID:1    PL:ILLUMINA SM:LP4100018-DNA_C11_Proband    PU:HGY3WDSXX:2:none&#xA;@RG ID:2    PL:ILLUMINA SM:LP4100018-DNA_C11_Proband    PU:HGY3WDSXX:3:none&#xA;@RG ID:3    PL:ILLUMINA SM:LP4100018-DNA_C11_Proband    PU:HGY3WDSXX:4:none&#xA;@RG ID:0-11EFC00B   PL:ILLUMINA SM:LP4100018-DNA_E11_Proband    PU:HGY3WDSXX:1:none&#xA;@RG ID:1-B8A1099    PL:ILLUMINA SM:LP4100018-DNA_E11_Proband    PU:HGY3WDSXX:2:none&#xA;@RG ID:2-330086F    PL:ILLUMINA SM:LP4100018-DNA_E11_Proband    PU:HGY3WDSXX:3:none&#xA;@RG ID:3-7681F092   PL:ILLUMINA SM:LP4100018-DNA_E11_Proband    PU:HGY3WDSXX:4:none&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'd like to check that the correct proportion of read groups originate from each sample.&lt;/p&gt;&#xA;&lt;p&gt;Currently I am using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view example.bam | rev | cut -f 1 | rev &amp;gt; output.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;, but this is not very elegant and only works because the RG field is last in the .bam.&lt;/p&gt;&#xA;&lt;p&gt;Is there a quick way to tabulate the number of reads groups with different IDs? E.g. produce an output like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ID:0 1000&#xA;ID:1 2000&#xA;ID:2 3000&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A solution in samtools would be ideal, along the lines of the output produced in &lt;code&gt;samtools stats&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="3694" LastEditDate="2020-09-24T10:16:22.023" LastActivityDate="2020-09-24T10:16:22.023" Title="summarising read group information from a .bam file" Tags="|sam|samtools|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14412" PostTypeId="1" AcceptedAnswerId="14413" CreationDate="2020-09-23T12:16:04.873" Score="1" ViewCount="98" Body="&lt;p&gt;The structure file for fastSTRUCTURE seem to be a table with rows and columns but the cells are all numbers, while my genotypes are simply bases. How could I know what I should insert there?&lt;/p&gt;&#xA;" OwnerUserId="6573" LastEditorUserId="3694" LastEditDate="2020-09-23T20:48:42.360" LastActivityDate="2020-09-23T20:48:42.360" Title="What should be the numbers in the fastSTRUCTURE file?" Tags="|file-formats|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14413" PostTypeId="2" ParentId="14412" CreationDate="2020-09-23T12:52:07.487" Score="4" Body="&lt;p&gt;Numbers, usually in the form of 0/1/2, is a common way many formats represent bases.&lt;/p&gt;&#xA;&lt;p&gt;For example, if a SNPs has bases which are A/T, then these are encoded as 0/1, with the reference allele/base taking the 0 and the alternate allele/base taking 1.&lt;/p&gt;&#xA;&lt;p&gt;However, if you read &lt;a href=&quot;https://rajanil.github.io/fastStructure/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the documentation&lt;/a&gt;, it says:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Specifically, rows in the data file correspond to samples, with two&#xA;rows per sample (note that only diploids are handled by this&#xA;software), and columns correspond to SNPs. The first 6 columns of the&#xA;file will be ignored; these typically would include IDs, metadata,&#xA;etc. This software only handles bi-allelic loci. &lt;strong&gt;The two alleles at&#xA;each locus can be encoded as desired&lt;/strong&gt;; however, missing data should be&#xA;encoded as `-9'.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So you should be able to use your base letters here and it will work OK as long as the columns and rows are formatted correctly.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2020-09-23T12:52:07.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14414" PostTypeId="1" AcceptedAnswerId="14417" CreationDate="2020-09-23T16:32:26.490" Score="2" ViewCount="113" Body="&lt;p&gt;Sorry if this is a stupid question I am new to analysing targeted hybrid capture data.&lt;/p&gt;&#xA;&lt;p&gt;I am analysing my HTS data from a targeted hybrid capture enrichment method and subsequent sequencing on a NextSeq.&lt;/p&gt;&#xA;&lt;p&gt;Looking at my bam files, I see there are reads that are overlapping into my non target regions.&lt;/p&gt;&#xA;&lt;p&gt;For example - my .bed file has two target regions in chr17:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vF7BB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vF7BB.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However I still have reads overlapping between 7578739-7578918 (following example is position 7578850):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BMWSZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BMWSZ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Quite a lot of these reads don't hit any part of the targeted regions in my bed file. Are these mistakes? How have they been pulled down for sequencing when they aren't in the target regions of the baits?&lt;/p&gt;&#xA;" OwnerUserId="9570" LastActivityDate="2020-09-23T18:49:11.243" Title="Targeted hybrid capture bam files showing reads outside target regions in my bed file" Tags="|sam|hts|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14415" PostTypeId="2" ParentId="14385" CreationDate="2020-09-23T17:15:45.170" Score="2" Body="&lt;p&gt;&lt;strong&gt;Update 17.10.22&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It seems that the api changed and the previous solution doesn't work anymore. I updated my solution by using the &lt;a href=&quot;https://www.uniprot.org/help/id_mapping&quot; rel=&quot;nofollow noreferrer&quot;&gt;Uniprot id mapping service&lt;/a&gt;. All the following instructions are copied from the documentation page but added here because link answers are discouraged. The documentation page also includes an implementation in python. The basic example is as follows (using commandline tools)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ curl &#xA;    --request POST 'https://rest.uniprot.org/idmapping/run' &#xA;    --form 'ids=&amp;quot;P99999,P31946&amp;quot;' &#xA;    --form 'from=&amp;quot;UniProtKB_AC-ID&amp;quot;' &#xA;    --form 'to=&amp;quot;UniRef100&amp;quot;'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This returns a job id as a response&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{&amp;quot;jobId&amp;quot;:&amp;quot;27a020f6334184c4eb382111fbcad0e848f40300&amp;quot;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You then have to poll the api for the status of the job&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ curl -i 'https://rest.uniprot.org/idmapping/status/{jobId}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which returns you a status of FINISHED and the url where you can fetch the result in the HTTP Header. Note that since the job might take some time to finish you need to poll this url until you get the status FINISHED.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;HTTP/1.1 303&#xA;Server: nginx/1.17.7&#xA;Location: https://rest.uniprot.org/idmapping/uniref/results/27a020f6334184c4eb382111fbcad0e848f40300&#xA;Content-Type: application/json&#xA;Access-Control-Allow-Origin: *&#xA;...&#xA;{&amp;quot;jobStatus&amp;quot;:&amp;quot;FINISHED&amp;quot;}  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Access the result url with curl&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ curl -s &amp;quot;https://rest.uniprot.org/idmapping/uniref/results/27a020f6334184c4eb382111fbcad0e848f40300&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which gives you a json dictionary&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{ &amp;quot;results&amp;quot;: [...], &amp;quot;failedIds&amp;quot;: []}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Each entry in the result list is a dictionary similar to the following example&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;    'from': 'P99999'\n\t&#xA;    , 'to': {&#xA;        'id': 'UniRef100_P99999'&#xA;        , 'memberCount': 17&#xA;        , 'representativeMember': {&#xA;            'uniparcId': 'UPI000013EAA0'&#xA;            , ...&#xA;        }&#xA;        , 'memberIdTypes': [&#xA;            'UniProtKB Unreviewed (TrEMBL)'&#xA;            , 'UniProtKB Reviewed (Swiss-Prot)'&#xA;        ]&#xA;        , 'members': [&#xA;            'P99999,0'&#xA;            , 'Q6WUX8,0'&#xA;            , 'P99998,0'&#xA;            , 'Q5RFH4,0'&#xA;            , 'G4XXL9,1'&#xA;            , 'A0A2J8VDX6,1'&#xA;            , 'K7CAI8,1'&#xA;            , 'A0A663D840,1'&#xA;            , 'G4XXM0,1'&#xA;            , 'G4XXM2,1'&#xA;            , 'G4XXM4,1'&#xA;            , 'G1RY04,1'&#xA;            , 'G4XXM3,1'&#xA;            , 'G4XXM6,1'&#xA;            , 'C9JFR7,1'&#xA;            , 'A0A2J8VE06,1'&#xA;            , 'A0A2J8KZC0,1'&#xA;        ]&#xA;        , ...&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The uniparc id of the cluster is given in the representative Member.&lt;/p&gt;&#xA;&lt;p&gt;If you want the group members directly they are given as a list of strings that contain the accession and the memberIdType index separated with a comma. You would use the index to look up the type of the accession in the memberIdTypes array.&lt;/p&gt;&#xA;&lt;p&gt;The result also contains information such as the sequence of the representative Member, or the number of organisms in the cluster.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; From what I can tell in the UniRef100 Clusters there will only be one Uniparc Id which you will find with the representative Member. If you want to map to UniRef90 or UniRef50 there will be other Uniparc ids in the member list, however the representative Uniparc Id will still only be in the representative Member, so you it's probably safest to parse both.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Old solution&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I found a way to batch the mapping from uniref cluster ids to the ids of all members. The url&#xA;&lt;code&gt;https://www.uniprot.org/uniref/?query=UniRef100_A0A003&amp;amp;format=tab&amp;amp;columns=id,members&lt;/code&gt;&#xA;can be used to get the link from one uniref Cluster to it's member ids. Multiple uniref cluster can be fetched at once by connecting the ids with a &lt;code&gt;+OR+&lt;/code&gt; like so &lt;code&gt;https://www.uniprot.org/uniref/?query=UniRef100_P99999+OR+UniRef100_P31946&amp;amp;format=tab&amp;amp;columns=id,members&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Because the uniref cluster ids are not stable I ended up using the following syntax, where instead of querying the UniRef cluster by it's id, I query by one of it's members, namely the one that was used to create the current uniref cluster id: &lt;code&gt;https://www.uniprot.org/uniref/?query=*&amp;amp;format=tab&amp;amp;limit=10&amp;amp;columns=id,members&amp;amp;fil=identity:1.0+AND+member:(P99999+OR+P31946)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the notes section of the &lt;a href=&quot;https://www.uniprot.org/help/uploadlists&quot; rel=&quot;nofollow noreferrer&quot;&gt;'Retrieve / ID mapping'&lt;/a&gt; documentation of the uniprot they talk about fetching chunks up to 20.000 identifiers at a time. I am not sure if I use the service they are talking about though, so I'm keeping my requests at 100 identifiers per request and that seems to work.&lt;/p&gt;&#xA;&lt;p&gt;One thing I noticed was that fetching results via the &lt;a href=&quot;https://www.uniprot.org/uniref/UniRef100_P31946.xml&quot; rel=&quot;nofollow noreferrer&quot;&gt;uniref endpoint&lt;/a&gt; is faster for small sets of identifiers (&amp;lt;20) since the fetch time of the batch request is similar for smaller and larger (&amp;lt;100) sets. So batching only makes sense for more than 20 identifiers at a time. (Disclaimer: I have only done very basic tests on this)&lt;/p&gt;&#xA;&lt;p&gt;Once I have all the members of a cluster I can use the proteins/uniparc endpoint of the &lt;a href=&quot;https://www.ebi.ac.uk/proteins/api/doc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;proteins api&lt;/a&gt; to fetch the uniparc data. For uniparc accessions (starting with the string 'UPI') I use the &lt;code&gt;GET /uniparc/upi/{upi}&lt;/code&gt; endpoint. For the other accession, which should all be Uniprot accessions, I use the &lt;code&gt;GET /uniparc/accession/{accession}&lt;/code&gt; endpoint.&lt;/p&gt;&#xA;&lt;p&gt;This doesn't answer the question of a direct mapping between the uniref ids and uniparc member ids, but it is a workable solution.&lt;/p&gt;&#xA;" OwnerUserId="4441" LastEditorUserId="4441" LastEditDate="2022-10-18T07:29:41.423" LastActivityDate="2022-10-18T07:29:41.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14416" PostTypeId="2" ParentId="14342" CreationDate="2020-09-23T17:41:42.577" Score="2" Body="&lt;p&gt;The kind of normalization depends on what you what to explore. So, there is no absolute answer to this. Different normalizations highlight different aspects of your dataset. There is a nice paper for a higher throughput quantitative study that I would recommend reading:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://linkinghub.elsevier.com/retrieve/pii/S0092867419313856&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nusinow, David P., John Szpyt, Mahmoud Ghandi, Christopher M. Rose, E. Robert McDonald 3rd, Marian Kalocsay, Judit Jané-Valbuena, et al. 2020. “Quantitative Proteomics of the Cancer Cell Line Encyclopedia.” Cell 180 (2): 387–402.e16.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What you want to aim for is a normalization that removes batch effects and that clusters together your quality control samples that you know are very similar to each other.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2020-09-23T17:41:42.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14417" PostTypeId="2" ParentId="14414" CreationDate="2020-09-23T18:49:11.243" Score="3" Body="&lt;p&gt;It strongly depends on exactly what your protocol is. Note that there are a good number of tools for QCing capture data (&lt;a href=&quot;https://academic.oup.com/bioinformatics/article/30/12/1767/2748161&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/TEQC/inst/doc/TEQC.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Off-target reads are a standard observation in capture experiments; recall that it is target &lt;strong&gt;enrichment&lt;/strong&gt;, not target &lt;strong&gt;perfect purification&lt;/strong&gt;, as commenter swbarnes2 suggests.&lt;/p&gt;&#xA;&lt;p&gt;The real question is whether the proportion of those off-target reads is too high for your application, or otherwise suggests problems. Those depend on details of the technique that you will likely have to figure out yourself.&lt;/p&gt;&#xA;&lt;p&gt;Here are some tables from the first of those QC papers (supplement), showing on-target proportions for various capture samples (sorry, it would have been a bit of effort to get text of the tables rather than images):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ILmxD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ILmxD.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sSqkS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sSqkS.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;These data suggest that on-target reads are 60%-80% of mapped reads. Note that they also lose a lot of reads in mapping, so we cannot exclude the possibility of mis-mapped garbage reads contributing to off-target mapping.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-09-23T18:49:11.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14418" PostTypeId="2" ParentId="14290" CreationDate="2020-09-23T20:24:21.760" Score="1" Body="&lt;p&gt;Via BEDOPS &lt;code&gt;convert2bed&lt;/code&gt; (&lt;code&gt;psl2bed&lt;/code&gt;) and &lt;code&gt;bedops&lt;/code&gt; operations:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ psl2bed &amp;lt; hits.psl | bedops --range 5 --everything - &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The file &lt;code&gt;answer.bed&lt;/code&gt; will contain target intervals from the PSL (BLAT) input, padded up- and downstream by five bases.&lt;/p&gt;&#xA;&lt;p&gt;This BED file can be run through &lt;code&gt;samtools faidx&lt;/code&gt; or &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/5374/776&quot;&gt;similar&lt;/a&gt; to get sequence data.&lt;/p&gt;&#xA;&lt;p&gt;References:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/file-management/conversion/psl2bed.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedops.readthedocs.io/en/latest/content/reference/file-management/conversion/psl2bed.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/set-operations/bedops.html#range-range&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedops.readthedocs.io/en/latest/content/reference/set-operations/bedops.html#range-range&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2020-09-23T23:11:17.050" LastActivityDate="2020-09-23T23:11:17.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14419" PostTypeId="1" CreationDate="2020-09-24T05:31:19.003" Score="1" ViewCount="429" Body="&lt;p&gt;Now jupyter installed on the server and I am using below code to plot the rarefaction plot but I am still getting some error. could you please suggest how I can get rid of it?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    #Run it in jupyter lab or notebook&#xA;&#xA;&#xA;import seaborn as sns&#xA;import matplotlib.pyplot as plt&#xA;import pandas as pd&#xA;import numpy as np&#xA;import os&#xA;&#xA;#%matplotlib inline &#xA;&#xA;qza = 'alpha-rarefaction-20000.qzv' # path to rarefaction file from qiime2&#xA;os.system(&amp;quot;unzip qza&amp;quot;)&#xA;&#xA;digest = a[1].split('/')[0].replace('  inflating: ','')&#xA;inf = digest + '/data/observed_otus.csv'&#xA;otus = pd.read_csv(inf,index_col=0, sep=',')&#xA;os.system('rm -r $digest')&#xA;&#xA;cols = [col for col in otus.columns if 'iter' not in col]&#xA;mean,data = otus[cols],pd.DataFrame(columns=cols)&#xA;depths = [col.split('_')[0] for col in otus.columns if 'depth' in col]&#xA;otus = otus.drop(cols,axis=1)&#xA;otus.columns = depths&#xA;for depth in depths:&#xA;    mean['ASV'] = otus[depth].mean(axis=1)&#xA;    mean['depth']= depth.split('-')[-1]&#xA;    data = pd.concat([data,mean])&#xA;# here provide colors for each item that will be plotted&#xA;pal={'Irrigated':'brown','Rainfed':'red'} &#xA;fig,ax = plt.subplots(figsize=(2.5,2),dpi=600,tight_layout=True)&#xA;sns.set(style='ticks',rc={&amp;quot;lines.linewidth&amp;quot;:0.7,&amp;quot;axes.linewidth&amp;quot;:0.5})&#xA;# use your column name to plot here&#xA;sns.lineplot(x='depth',y='ASV',data=data,palette=pal,hue='Season',sort=False,err_style='bars',\&#xA;             dashes=True,style='Season',ci=67)&#xA;ax.set_xlabel('Sequencing depth',fontsize=8)&#xA;ax.set_ylabel('Observed ASVs',fontsize=8)&#xA;ax.tick_params(axis='x', labelrotation=90)&#xA;ax.tick_params(axis='both',which='major',length=2,pad=0.5,labelsize=6)&#xA;handles, labels = ax.get_legend_handles_labels()&#xA;ax.legend(handles=handles[1:],labels=labels[1:],fontsize=5,frameon=False,numpoints=4,borderaxespad=0,handletextpad=0.2,loc=2,)&#xA;plt.savefig('Infected_observed_ASVs.png', bbox_inches='tight')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Error;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; $ python3 myscript.rarefaction.py&#xA;Traceback (most recent call last):&#xA;  File &amp;quot;myscript.rarefaction.py&amp;quot;, line 4, in &amp;lt;module&amp;gt;&#xA;    import seaborn as sns&#xA;  File &amp;quot;/opt/apps/seaborn/0.11.0/python3-3.7.4/lib/python3.7/site-packages/seaborn/__init__.py&amp;quot;, line 2, in &amp;lt;module&amp;gt;&#xA;    from .rcmod import *  # noqa: F401,F403&#xA;  File &amp;quot;/opt/apps/seaborn/0.11.0/python3-3.7.4/lib/python3.7/site-packages/seaborn/rcmod.py&amp;quot;, line 7, in &amp;lt;module&amp;gt;&#xA;    from . import palettes&#xA;  File &amp;quot;/opt/apps/seaborn/0.11.0/python3-3.7.4/lib/python3.7/site-packages/seaborn/palettes.py&amp;quot;, line 9, in &amp;lt;module&amp;gt;&#xA;    from .utils import desaturate, get_color_cycle&#xA;  File &amp;quot;/opt/apps/seaborn/0.11.0/python3-3.7.4/lib/python3.7/site-packages/seaborn/utils.py&amp;quot;, line 11, in &amp;lt;module&amp;gt;&#xA;    import pandas as pd&#xA;  File &amp;quot;/opt/apps/seaborn/0.11.0/python3-3.7.4/lib/python3.7/site-packages/pandas/__init__.py&amp;quot;, line 11, in &amp;lt;module&amp;gt;&#xA;    __import__(dependency)&#xA;  File &amp;quot;&amp;lt;frozen importlib._bootstrap&amp;gt;&amp;quot;, line 983, in _find_and_load&#xA;  File &amp;quot;&amp;lt;frozen importlib._bootstrap&amp;gt;&amp;quot;, line 967, in _find_and_load_unlocked&#xA;  File &amp;quot;&amp;lt;frozen importlib._bootstrap&amp;gt;&amp;quot;, line 668, in _load_unlocked&#xA;  File &amp;quot;&amp;lt;frozen importlib._bootstrap&amp;gt;&amp;quot;, line 638, in _load_backward_compatible&#xA;  File &amp;quot;/opt/gridware/depots/54e7fb3c/el7/pkg/libs/matplotlib_python34/1.5.1/gcc-4.8.5+python3-3.4.3+numpy_python34-1.11.3/python/lib/python3.4/site-packages/pytz-2019.1-py3.4.egg/pytz/__init__.py&amp;quot;, line 1101, in &amp;lt;module&amp;gt;&#xA;  File &amp;quot;/opt/gridware/depots/54e7fb3c/el7/pkg/libs/matplotlib_python34/1.5.1/gcc-4.8.5+python3-3.4.3+numpy_python34-1.11.3/python/lib/python3.4/site-packages/pytz-2019.1-py3.4.egg/pytz/lazy.py&amp;quot;, line 101, in _lazy&#xA;  File &amp;quot;/opt/gridware/depots/54e7fb3c/el7/pkg/libs/matplotlib_python34/1.5.1/gcc-4.8.5+python3-3.4.3+numpy_python34-1.11.3/python/lib/python3.4/site-packages/pytz-2019.1-py3.4.egg/pytz/__init__.py&amp;quot;, line 1098, in &amp;lt;genexpr&amp;gt;&#xA;  File &amp;quot;/opt/gridware/depots/54e7fb3c/el7/pkg/libs/matplotlib_python34/1.5.1/gcc-4.8.5+python3-3.4.3+numpy_python34-1.11.3/python/lib/python3.4/site-packages/pytz-2019.1-py3.4.egg/pytz/__init__.py&amp;quot;, line 114, in resource_exists&#xA;  File &amp;quot;/opt/gridware/depots/54e7fb3c/el7/pkg/libs/matplotlib_python34/1.5.1/gcc-4.8.5+python3-3.4.3+numpy_python34-1.11.3/python/lib/python3.4/site-packages/pytz-2019.1-py3.4.egg/pytz/__init__.py&amp;quot;, line 102, in open_resource&#xA;  File &amp;quot;&amp;lt;frozen importlib._bootstrap&amp;gt;&amp;quot;, line 983, in _find_and_load&#xA;  File &amp;quot;&amp;lt;frozen importlib._bootstrap&amp;gt;&amp;quot;, line 967, in _find_and_load_unlocked&#xA;  File &amp;quot;&amp;lt;frozen importlib._bootstrap&amp;gt;&amp;quot;, line 668, in _load_unlocked&#xA;  File &amp;quot;&amp;lt;frozen importlib._bootstrap&amp;gt;&amp;quot;, line 638, in _load_backward_compatible&#xA;  File &amp;quot;/opt/gridware/depots/54e7fb3c/el7/pkg/apps/setuptools/24.0.1/python-2.7.8/python/lib/python2.7/site-packages/setuptools-24.0.1-py2.7.egg/pkg_resources/__init__.py&amp;quot;, line 2927, in &amp;lt;module&amp;gt;&#xA;  File &amp;quot;/opt/gridware/depots/54e7fb3c/el7/pkg/apps/setuptools/24.0.1/python-2.7.8/python/lib/python2.7/site-packages/setuptools-24.0.1-py2.7.egg/pkg_resources/__init__.py&amp;quot;, line 2913, in _call_aside&#xA;  File &amp;quot;/opt/gridware/depots/54e7fb3c/el7/pkg/apps/setuptools/24.0.1/python-2.7.8/python/lib/python2.7/site-packages/setuptools-24.0.1-py2.7.egg/pkg_resources/__init__.py&amp;quot;, line 2952, in _initialize_master_working_set&#xA;  File &amp;quot;/opt/gridware/depots/54e7fb3c/el7/pkg/apps/setuptools/24.0.1/python-2.7.8/python/lib/python2.7/site-packages/setuptools-24.0.1-py2.7.egg/pkg_resources/__init__.py&amp;quot;, line 956, in subscribe&#xA;  File &amp;quot;/opt/gridware/depots/54e7fb3c/el7/pkg/apps/setuptools/24.0.1/python-2.7.8/python/lib/python2.7/site-packages/setuptools-24.0.1-py2.7.egg/pkg_resources/__init__.py&amp;quot;, line 2952, in &amp;lt;lambda&amp;gt;&#xA;  File &amp;quot;/opt/gridware/depots/54e7fb3c/el7/pkg/apps/setuptools/24.0.1/python-2.7.8/python/lib/python2.7/site-packages/setuptools-24.0.1-py2.7.egg/pkg_resources/__init__.py&amp;quot;, line 2515, in activate&#xA;  File &amp;quot;/opt/gridware/depots/54e7fb3c/el7/pkg/apps/setuptools/24.0.1/python-2.7.8/python/lib/python2.7/site-packages/setuptools-24.0.1-py2.7.egg/pkg_resources/__init__.py&amp;quot;, line 2097, in declare_namespace&#xA;  File &amp;quot;/opt/gridware/depots/54e7fb3c/el7/pkg/apps/setuptools/24.0.1/python-2.7.8/python/lib/python2.7/site-packages/setuptools-24.0.1-py2.7.egg/pkg_resources/__init__.py&amp;quot;, line 2047, in _handle_ns&#xA;  File &amp;quot;/opt/gridware/depots/54e7fb3c/el7/pkg/apps/setuptools/24.0.1/python-2.7.8/python/lib/python2.7/site-packages/setuptools-24.0.1-py2.7.egg/pkg_resources/__init__.py&amp;quot;, line 2066, in _rebuild_mod_path&#xA;AttributeError: '_NamespacePath' object has no attribute 'sort'&#xA;&#xA;         &#xA;     &#xA;    &#xA;    metadata file ;&#xA;    #SampleID       Villages        Region  Treatment       Batch   Season&#xA;    PN0086D.1.S1    Kholabari       Mymensingh      T1      batch1  Irrigated&#xA;    PN0086D.2.S2    Bogajan Mymensingh      T1      batch1  Irrigated&#xA;    PN0086D.3.S3    Bonkua  Mymensingh      T1      batch1  Irrigated&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="1008" LastEditDate="2020-09-25T15:13:37.723" LastActivityDate="2020-09-26T12:13:51.617" Title="How to run Jupyter script on Slurm HPC" Tags="|python|bash|hpc|jupyter|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14420" PostTypeId="2" ParentId="14419" CreationDate="2020-09-24T12:47:05.750" Score="2" Body="&lt;p&gt;This is a bash script (#!/bin/bash) and you try to run python in it.&#xA;You should save your python script (e.g. in &amp;quot;myscript.py&amp;quot;) and call it within your bash script.&#xA;Maybe first try to see if you are able to run it in your terminal and then run it on your server with SLURM.&lt;/p&gt;&#xA;&lt;p&gt;EDIT1: Also I don't think you need 3 nodes to run such script and this will probably not be even possible here. I think you need some specific modules to parallelize a python script with MPI (see &lt;a href=&quot;https://stackoverflow.com/questions/25792357/how-to-distribute-a-python-script-across-multiple-nodes&quot;&gt;https://stackoverflow.com/questions/25792357/how-to-distribute-a-python-script-across-multiple-nodes&lt;/a&gt;), but again I think this would be an overkill. Try first with &lt;code&gt;#SBATCH -c 1&lt;/code&gt; instead.&lt;/p&gt;&#xA;&lt;p&gt;Edit2: Finally it might be possible to run jupyter-notebook on a node allocated by SLRUM instead of putting this into a python script by using &lt;code&gt;jupyter-notebook --no-browser --port=${port} --ip=${node}&lt;/code&gt; command in your bash slurm script. Maybe you'll find this doc interesting: &lt;a href=&quot;https://docs.ycrc.yale.edu/clusters-at-yale/guides/jupyter/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.ycrc.yale.edu/clusters-at-yale/guides/jupyter/&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="9109" LastEditorUserId="9109" LastEditDate="2020-09-24T14:45:43.693" LastActivityDate="2020-09-24T14:45:43.693" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="14421" PostTypeId="2" ParentId="3931" CreationDate="2020-09-24T15:27:05.260" Score="1" Body="&lt;p&gt;This is what &lt;strong&gt;&lt;code&gt;samtools faidx&lt;/code&gt;&lt;/strong&gt; is intended for.&lt;/p&gt;&#xA;&lt;p&gt;It needs to be called twice.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;First, to create a FASTA index (&lt;code&gt;*.fai&lt;/code&gt; file):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;samtools faidx input.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Secondly, to filter the FASTA file with the help of the index:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;samtools faidx -o output.fasta input.fasta ids…&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;code&gt;ids…&lt;/code&gt; are the IDs to retain, as individual arguments separated by space. If you’ve got a file &lt;code&gt;blocklist.txt&lt;/code&gt; with IDs you want to &lt;em&gt;discard&lt;/em&gt; (one per line), you first need to invert this, after having created the index (using Bash syntax):&lt;sup&gt;1&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;remove_ids=($(awk '{print $1}' input.fasta.fai | grep -v -f blocklist.txt))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;… and then we can invoke the command as&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools faidx -o output.fasta input.fasta &amp;quot;${remove_ids[@]}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What’s nice about &lt;code&gt;samtools faidx&lt;/code&gt; is that it also works with &lt;a href=&quot;http://www.htslib.org/doc/bgzip.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BGZF-compressed FASTA&lt;/a&gt; data — i.e. &lt;code&gt;fa.gz&lt;/code&gt; files (but &lt;em&gt;only&lt;/em&gt; if they were compressed using &lt;code&gt;bgzip&lt;/code&gt;, which ships with Samtools, and the index must be generated on the compressed file). However, it will always write FASTA. If you want gzipped FASTA output, you need to manually stream the result into &lt;code&gt;bgzip&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;samtools faidx input.fasta.gz &amp;quot;${remove_ids[@]}&amp;quot; | bgzip -c &amp;gt;output.fasta.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;sup&gt;1&lt;/sup&gt; &lt;sub&gt;&lt;strong&gt;Careful&lt;/strong&gt;, this syntax is &lt;em&gt;generally unsafe&lt;/em&gt; for reading a command output into a variable; we can use it here only because we assume that we want to generate a &lt;em&gt;space&lt;/em&gt;-separated list of identifiers; the generally safe way is &lt;a href=&quot;https://unix.stackexchange.com/a/485262/3651&quot;&gt;a lot more convoluted&lt;/a&gt;.&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="29" LastActivityDate="2020-09-24T15:27:05.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14423" PostTypeId="1" AcceptedAnswerId="14433" CreationDate="2020-09-25T02:45:38.817" Score="2" ViewCount="444" Body="&lt;p&gt;I'm trying to Blast for exact matches to the sequence: 'ATTGNNNNGCAAACCA' in the human transcriptome using NCBI Blast on its 'refseq_rna' database. However, when I do a basic query I get &amp;quot;No significant similarity found.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Note the N's in the middle of the sequence, if I get rid of those N's, yielding the sequence 'AGCGGATTGCAAAGCAAACCA', I get a match in the 3' UTR of the human MeCP2 gene (which is correct). I don't understand why adding the N's here make it not work.&lt;/p&gt;&#xA;&lt;p&gt;I looked at the help section and I believe this may be due to shorter sequences being less &amp;quot;statistically significant&amp;quot;. Statistical significance of output is completely irrelevant to me, so I set 'Expect Value' to 100000000000000. However, I still get the same result.&lt;/p&gt;&#xA;&lt;p&gt;I also tried to modify the HTML of the page to allow me to submit a word size of 4 to the form. This went through, but I still got &amp;quot;No significant similarity found&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Could someone help me out with this? I feel like searching for a short sequence shouldn't be this hard.&lt;/p&gt;&#xA;" OwnerUserId="9581" LastActivityDate="2020-09-26T21:00:54.343" Title="NCBI blast for exact match of a short sequence" Tags="|blast|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14425" PostTypeId="1" AcceptedAnswerId="14426" CreationDate="2020-09-25T03:05:18.260" Score="0" ViewCount="150" Body="&lt;p&gt;I am working with an RNA-seq dataset generated from a complex experimental design. Our main question is focused on the interaction between two variables, one (INJ) with two conditions (L,S) and the other (SOC) with four conditions (A,B,C,D). We also have a third variable for tissue type (TIS) from 3 tissues (H,N,T), although we are not interested in this variable for this particular question.&lt;/p&gt;&#xA;&lt;p&gt;We were able extract DE genes underlying the interaction of interest in DESeq2 using an LRT test. We also ran WGCNA to get gene clusters and looked at trait correlations. Using these WGNCA clusters and VST normalized count data, I would like to run GSEA or a similar analysis to find WGCNCA clusters that differ based on the interaction between the two main variables.&lt;/p&gt;&#xA;&lt;p&gt;What I've tried so far was generating a pseudo-variable that is a combination of INJ and SOC with 8 separate conditions for running in GSEA, but this doesn't quite get at the interaction between variables since we really want to see if the effect of INJ differs between SOC conditions.&lt;/p&gt;&#xA;&lt;p&gt;One thing I was thinking was running a GSEA-like analysis on log2 ratios between INJ states for each SOC condition, but these would have to be summary statistics for groups of samples and I would lose the variance of expression within groups (maybe that doesn't matter though for exploratory analysis?). This would also reduce my &amp;quot;sample&amp;quot; count to 1 per condition, which isn't ideal for this type of analysis and I would probably have to look at differences between gene sets instead of phenotypes for more robust stats.&lt;/p&gt;&#xA;&lt;p&gt;Any thoughts on ways to do this? I don't need to use GSEA if there is another analysis I could run on gene sets that allows for more complex experimental models. Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="9069" LastActivityDate="2020-09-25T05:42:17.077" Title="Running GSEA or comparable analysis with multiple variable interaction?" Tags="|rna-seq|wgcna|gsea|interactions|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14426" PostTypeId="2" ParentId="14425" CreationDate="2020-09-25T05:42:17.077" Score="2" Body="&lt;p&gt;I'm not sure I understand &amp;quot;I would like to run GSEA or a similar analysis to find WGCNCA clusters that differ based on the interaction between the two main variables&amp;quot;. I would run an association analysis (regression) for module eigengenes and the appropriate interaction term. The (most) significant modules are your candidates. This step is simply regression (in the simplest case a linear model should be sufficient), no GSEA needed. You can then subject the identified modules to GSEA or similar to get an idea of their biological function.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2020-09-25T05:42:17.077" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14430" PostTypeId="2" ParentId="14419" CreationDate="2020-09-25T14:36:15.357" Score="0" Body="&lt;p&gt;I'll make a contribution and keep editing it rather than use 'comments', the error is about&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;At the moment this isn't about Jupyter, its about getting Python working on your server. Is that a good assessment? With Jupyter notebook you write a little bit of code, check it, write a bit more. That's how it is designed to operate. The other way in is to use Python in your shebang statement otherwise it will default to &lt;code&gt;sh&lt;/code&gt; or &lt;code&gt;bash&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The code you supplied doesn't explicitly use &lt;code&gt;numpy&lt;/code&gt;. Pandas does use numpy and &lt;code&gt;np&lt;/code&gt; is the standard shorthand, &lt;code&gt;Multiarray&lt;/code&gt; is probably something pandas would use. Did you get this code working on your local machine?&lt;/p&gt;&#xA;&lt;p&gt;I am wondering whether the numpy module on your server is up to date? Did you creat a specific Python environment, use &lt;code&gt;pip&lt;/code&gt; or &lt;code&gt;conda&lt;/code&gt;? Checking the Python version on your server and whether &lt;code&gt;numpy&lt;/code&gt; needs updating is perhaps where I might start.&lt;/p&gt;&#xA;&lt;p&gt;It is not pretty getting a HPC to run with your stuff and can take real work.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I read your new post, I would simply reinstall Python from &lt;code&gt;Anaconda3&lt;/code&gt; and run the script for now from shell pointing the shebang into Anaconda3 (I can't remember if you need to do this, I can check if needed). This a more professional way of using a HPC than using the resident Python. In good old days of Perl, when everything was nice and simple, &lt;code&gt;perlbrew&lt;/code&gt; was essentially the Perl equivalent and no-one would use the system Perl.&lt;/p&gt;&#xA;&lt;p&gt;The reason why I'm saying  this is I get this feeling it will be an endless cycle of import errors. Anaconda3 will give you control of your Python system without admin privelidge.&lt;/p&gt;&#xA;&lt;p&gt;Just to explain if you are not calling &lt;code&gt;np&lt;/code&gt; explicitly (and looking at your code I don't think you are) then explicitly stating &lt;code&gt;import numpy&lt;/code&gt; isn't needed. The fact you wrote it and it then jumped onto a different error is a bit worrying. I would just use your own version of Python. In Jupyter I would guess, but don't know they use their own Python ... but from a data science point of view Jupyter is the front end to AWS and GCS because those guys want some control of what you are doing, not running riot. In DNN jupyter is big. If you want an absolute diagnosis then Stackoverflow might help, but I think you'll just another bug and another etc... One thing is that &lt;code&gt;pandas&lt;/code&gt; changes alot and the version you are using might be ahead of the curve, thats my guess&lt;/p&gt;&#xA;&lt;p&gt;Final point is if you are just plotting the &lt;code&gt;ggplot2&lt;/code&gt; is really good, but that is &lt;code&gt;R&lt;/code&gt;. &lt;code&gt;seaborne&lt;/code&gt; is really good, &lt;code&gt;matplotlib&lt;/code&gt; but hmmmm, but nothing is as clear or explicit as &lt;code&gt;ggplot2&lt;/code&gt;. Python is good at lots of things, pandas is fabulous, but plotting isn't on par with &lt;code&gt;ggplot2&lt;/code&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;What is 100% clear is that you are loading the HPC version of Python and this is out of date (3.4) should be 3.7 (or even 3.8?). The thing I don't understand is why is it loads Python 2 modules?? This is a complete mystery to me, I thought Python 2 and Python 3 were incompatible, this is suggesting backwards compatibility or else the reason it is falling over. I have no idea, but I wouldn't use the resident Python.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-09-26T12:13:51.617" LastActivityDate="2020-09-26T12:13:51.617" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14431" PostTypeId="1" CreationDate="2020-09-25T14:37:29.283" Score="6" ViewCount="333" Body="&lt;p&gt;I have a list of PDB IDs with realtive chains IDs that have to be extracted, and then run on &lt;code&gt;dssp&lt;/code&gt;.&#xA;For the single chain extraction I tried several methods, such as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pymol2&#xA;for entry in list:&#xA;    with pymol2.PyMOL() as pymol:&#xA;        pymol.cmd.fetch(entry)&#xA;        pymol.cmd.save(entry + '.cif')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Where, for example, if I want to extract chain &lt;code&gt;AC&lt;/code&gt; for &lt;code&gt;4YBB&lt;/code&gt;, entry = &lt;code&gt;4YBBAC&lt;/code&gt;.&#xA;This works well in saving the cif files (pymol correctly visualizes them), but makes them not suitable for mkdssp. Indeed, if i run&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mkdssp 4YBBAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;DSSP could not be created due to an error:&#xA;Is this an mmCIF file?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried also using Biopython:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.PDB import MMCIFParser&#xA;from Bio.PDB.mmcifio import MMCIFIO&#xA;parser = MMCIFParser()&#xA;io = MMCIFIO()&#xA;structure = parser.get_structure(&amp;quot;4YBB&amp;quot;, &amp;quot;4YBB.cif&amp;quot;)&#xA;model = structure[0]&#xA;chain = model['AC']&#xA;io.set_structure(chain)&#xA;io.save(&amp;quot;4YBBAC.cif&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But again, when running mkdssp, I get:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;DSSP could not be created due to an error:&#xA;bad lexical cast: source type value could not be interpreted as target&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;On a closer inspection, both methods drop the columns &lt;code&gt;_atom_site.pdbx_formal_charge  _atom_site.auth_comp_id _atom_site.auth_atom_id&lt;/code&gt; from the file, while retaining all the others.&#xA;This is the first line of the coordinates from the biopython/pymol file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATOM   1    N N   . GLY A ? 1   ? -125.713 46.378 -19.108 1.0 77.15  2   AC 1 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;While this is the original file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATOM   34684  N  N     . GLY C   3  1    ? -125.713 46.378   -19.108  1.00 77.15  ?  2    GLY AC N     1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I think that renaming the AC chain to a single-character code would in principle resolve the issue, but it's more of a workaround.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;mkdssp&lt;/code&gt; does support multicharacter chain ids. I've proved it first it by cutting and pasting the chain into the file and then with an ad-hoc parser. I think the issue is with how biopython and pymol extract the chains.&lt;/p&gt;&#xA;&lt;p&gt;Is this a bug? Am I using the tools incorrectly? Can my purpose be achieved with other (possibly hassle free) methods?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;MORE ON THIS TOPIC ON Biopython GitHub :&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/biopython/biopython/issues/3439&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/biopython/biopython/issues/3439&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9572" LastEditorUserId="10049" LastEditDate="2023-02-01T21:52:04.697" LastActivityDate="2023-02-01T21:52:04.697" Title="Using dssp after chain extraction" Tags="|python|pymol|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14432" PostTypeId="1" CreationDate="2020-09-25T15:15:09.070" Score="1" ViewCount="31" Body="&lt;p&gt;&lt;em&gt;[Note: this question has also been posted on &lt;a href=&quot;https://forum.knime.com/t/chemical-names/27644&quot; rel=&quot;nofollow noreferrer&quot;&gt;KNIME&lt;/a&gt;]&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Good Morning,&lt;/p&gt;&#xA;&lt;p&gt;I am currently researching treatment options for the Corona Virus from the CAS COVID-19 Antiviral Candidate Compounds Dataset available on the &lt;a href=&quot;http://www.CAS.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;www.CAS.org&lt;/a&gt;. One issue we have run into is that the Chemical Names of the Antiviral Compounds in the Dataset are truncated as depicted in the example below. We will be needing the entire Chemical Names for Downstream Analysis. Is there anyway we can obtain the complete/ untruncated Chemical Names of the Antiviral Compounds?&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Example of an Antiviral Agent with a truncated Chemical Name&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Acetamide 2-amino-N-(1-tricyclo[3.3.1.137]dec-1-ylethyl)-&#xA;  Mrv1921 09212018412D          &#xA;&#xA; 17 19  0  0  0  0            999 V2000&#xA;    0.0000    0.0000    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    0.0000    0.0000    0.0000 N   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    0.0000    0.0000    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    0.0000    0.0000    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    0.0000    0.0000    0.0000 N   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    0.0000    0.0000    0.0000 O   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    0.0000    0.0000    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    0.8517   -1.6585    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    0.1163   -2.0322    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    0.5657   -1.2419    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    1.3012   -1.6156    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    2.0368   -1.2419    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    1.5873   -2.0322    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    2.0368   -0.4945    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    1.3012   -0.1208    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    0.8517   -0.9111    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    0.5657   -0.4945    0.0000 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;  1  2  1  0  0  0  0&#xA;  2  3  1  0  0  0  0&#xA;  3  4  1  0  0  0  0&#xA;  4  5  1  0  0  0  0&#xA;  3  6  2  0  0  0  0&#xA;  1  7  1  0  0  0  0&#xA;  1  8  1  0  0  0  0&#xA;  8  9  1  0  0  0  0&#xA;  9 10  1  0  0  0  0&#xA; 10 11  1  0  0  0  0&#xA; 11 12  1  0  0  0  0&#xA; 12 13  1  0  0  0  0&#xA;  8 13  1  0  0  0  0&#xA; 15 17  1  0  0  0  0&#xA; 14 15  1  0  0  0  0&#xA; 15 16  1  0  0  0  0&#xA;  8 16  1  0  0  0  0&#xA; 12 14  1  0  0  0  0&#xA; 10 17  1  0  0  0  0&#xA;M  END&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TsVjZ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TsVjZ.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="9346" LastEditorUserId="73" LastEditDate="2020-09-30T11:34:21.313" LastActivityDate="2020-09-30T11:34:21.313" Title="Truncated Chemical Names" Tags="|statistics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14433" PostTypeId="2" ParentId="14423" CreationDate="2020-09-26T10:15:53.720" Score="2" Body="&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt; while procrastinating, I worked up the below throwaway code into a more generally applicable (but still bare-bones) &lt;a href=&quot;https://github.com/maximilianpress/teaching_code/blob/master/exact_match.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;script&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;ORIGINAL:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I feel like you might want to use something other than BLAST to do this.&lt;/p&gt;&#xA;&lt;p&gt;BLAST is not really designed to solve this problem, I think it's just easy to use. Also, your N-containing sequence (16nt) is shorter than the version without Ns (21 nt), which can't be correct if you care about &amp;quot;exact&amp;quot; matches.&lt;/p&gt;&#xA;&lt;p&gt;As a side note, the idea of exact matches is a bit tricky with Ns involved. The whole point of ambiguity codes is that things won't be exact.&lt;/p&gt;&#xA;&lt;p&gt;And all of this is on top of the short-string problem- most alignment programs won't even return results that don't meet some minimum alignment score, which is exacerbated a bit by the Ns. For how BLAST handles ambiguity codes, you can see section 7.2 &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/blast/BLAST_guide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Why not use regular expressions instead to search your sequence against the transcriptome?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# python&#xA;import re&#xA;from Bio import SeqIO&#xA;&#xA;fwd = r'ATTG[ACGT]{4}GCAAACCA'&#xA;rev = r'TGGTTTGC[ACGT]{4}CAAT'  # reverse complement&#xA;&#xA;# reference file&#xA;seqfile = SeqIO.parse(&amp;quot;your_reference_file.fa&amp;quot;, &amp;quot;fasta&amp;quot;)  # a transcriptome reference fasta&#xA;&#xA;for seq in seqfile:&#xA;    # maybe use re.match() to get coords but easier to show it with findall&#xA;    a = re.findall(fwd, str(seq.seq))&#xA;    b = re.findall(rev, str(seq.seq))&#xA;    if len(a) &amp;gt; 0:&#xA;        print(seq.id, a)&#xA;    if len(b) &amp;gt; 0:&#xA;        print(seq.id, b)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I put this in a file and run it against a test case with &lt;code&gt;ATTGAAAAGCAAACCA&lt;/code&gt;, it works:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ python exact_match.py &#xA;the_contig ['ATTGAAAAGCAAACCA']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alternately, you could just blast the 16 non-N versions of that sequence and combine them, as you've shown.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2020-09-26T21:00:54.343" LastActivityDate="2020-09-26T21:00:54.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14434" PostTypeId="1" CreationDate="2020-09-26T16:29:27.037" Score="0" ViewCount="272" Body="&lt;p&gt;I am trying to install bowtie2 aligner from sources without the root access.&#xA;Bowtie2 needs tbb package to be installed and it is recommended to install oneTBB.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# install one TBB&#xA;mkdir -p SHOME/soft&#xA;cd $HOME/soft&#xA;git clone https://github.com/oneapi-src/oneTBB.git&#xA;cd oneTBB&#xA;make&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then I add C_INCLUDE_PATH to .bashrc:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;echo &amp;quot;export C_INCLUDE_PATH=$HOME/soft/oneTBB/include&amp;quot; &amp;gt;&amp;gt; ~./bashrc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and continue to install bowtie2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# install bowtie2&#xA;mkdir -p $HOME/soft/bowtie/source&#xA;cd $HOME/soft/bowtie/source&#xA;wget https://sourceforge.net/projects/bowtie-bio/files/bowtie2/2.4.1/bowtie2-2.4.1-source.zip&#xA;unzip bowtie2-2.4.1-source.zip -d ..&#xA;cd ../bowtie2-2.4.1&#xA;make&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;TBB library is recognized but in the end I get this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/usr/bin/ld: cannot find -ltbb&#xA;/usr/bin/ld: cannot find -ltbbmalloc_proxy&#xA;collect2: error: ld returned 1 exit status&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;ltbb and ltbballoc_proxy are not in TBB but libtbb and libtbballoc_proxy instead.&#xA;I also tryed to add variables to .bashrc:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TBB_LIB_PATH=$HOME/soft/oneTBB&#xA;TBB_INCLUDE_PATH=$HOME/soft/oneTBB/include&#xA;TBB_RELEASE_DIR=$HOME/soft/oneTBB/build/home/suvar/apps/oneTBB/build/linux_intel64_gcc_cc9.3.0_libc2.31_kernel5.4.0_release&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but seems it does not help.&lt;/p&gt;&#xA;&lt;p&gt;I need help to complete the installation of bowtie2 on our server.&lt;/p&gt;&#xA;" OwnerUserId="943" LastActivityDate="2021-07-09T21:04:55.420" Title="install bowtie2 from sources cannot find -ltbb" Tags="|software-installation|bowtie2|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14435" PostTypeId="1" AcceptedAnswerId="14436" CreationDate="2020-09-27T10:11:12.360" Score="1" ViewCount="402" Body="&lt;p&gt;Below are the statistics for RNA-seq mapped and unmapped paired-end reads to rice genome using reformat.sh from bbtools on bam files. It gives 77% mapped and 5% unmapped, what about the remaining 18% reads? How I can get information for all reads?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Command;&#xA;reformat.sh in=Leaf_T1_F_R10_S1_L001.bam out=Leaf_T1_F_R10_S1_L001.mapped.bam, mappedonly&#xA;&#xA;&#xA;Unspecified format for output Leaf_T1_F_R10_S1_L001.mapped.bam,; defaulting to fastq.&#xA;Found sambamba.&#xA;Input is being processed as unpaired&#xA;Input:                   67471075 reads           9312399845 bases&#xA;Output:                 52020538 reads (77.10%) 7208830835 bases (77.41%)&#xA;&#xA;Time:                         171.212 seconds.&#xA;Reads Processed:      67471k 394.08k reads/sec&#xA;Bases Processed:       9312m 54.39m bases/sec&#xA;&#xA;For unmapped read;&#xA;Command;&#xA;reformat.sh in=Leaf_T1_F_R10_S1_L001.bam out=Leaf_T1_F_R10_S1_L001.unmapped.bam, unmappedonly&#xA;&#xA;Unspecified format for output Leaf_T1_F_R10_S1_L001.unmapped.bam,; defaulting to fastq.&#xA;Found sambamba.&#xA;Input is being processed as unpaired&#xA;Input:                   67471075 reads           9312399845 bases&#xA;Output:                 3435326 reads (5.09%) 465456267 bases (5.00%)&#xA;&#xA;Time:                         142.978 seconds.&#xA;Reads Processed:      67471k 471.90k reads/sec&#xA;Bases Processed:       9312m 65.13m bases/sec&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With sambamba; Can I say 94.91% reads are mapped and 5.09% are unmapped?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;67471075 + 0 in total (QC-passed reads + QC-failed reads)&#xA;12015211 + 0 secondary&#xA;0 + 0 supplementary&#xA;0 + 0 duplicates&#xA;64035749 + 0 mapped (94.91%:N/A)&#xA;55455864 + 0 paired in sequencing&#xA;27727932 + 0 read1&#xA;27727932 + 0 read2&#xA;51006046 + 0 properly paired (91.98%:N/A)&#xA;51419222 + 0 with itself and mate mapped&#xA;601316 + 0 singletons (1.08%:N/A)&#xA;263816 + 0 with mate mapped to a different chr&#xA;237111 + 0 with mate mapped to a different chr (mapQ&amp;gt;=5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="6251" LastEditDate="2020-09-27T19:49:18.613" LastActivityDate="2020-09-29T12:53:30.487" Title="Mapping statistics from bam file using bbtools and sambamba" Tags="|rna-seq|samtools|bash|java|sambamba|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14436" PostTypeId="2" ParentId="14435" CreationDate="2020-09-27T13:55:02.873" Score="1" Body="&lt;p&gt;Try &lt;code&gt;samtools flagstat&lt;/code&gt; or &lt;code&gt;sambamba flagstat&lt;/code&gt; to get similar type of information about your bam file. It may be easier to interpret. Both of these tools can be installed using &lt;code&gt;conda&lt;/code&gt; if needed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Yes, the results of &lt;code&gt;sambamba flagstat&lt;/code&gt; you showed in your edited post indicate exactly what you said: &lt;code&gt;94.91%&lt;/code&gt; mapped and &lt;code&gt;100% - 94.91% = 5.09%&lt;/code&gt; unmapped:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;64035749 + 0 mapped (94.91%:N/A)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6251" LastEditorUserId="6251" LastEditDate="2020-09-27T18:25:49.280" LastActivityDate="2020-09-27T18:25:49.280" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14437" PostTypeId="1" CreationDate="2020-09-27T16:36:54.083" Score="3" ViewCount="120" Body="&lt;p&gt;Hi there I have this code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from functools import partial&#xA;import multiprocessing&#xA;&#xA;def kmer_count(sequence, alphabet, k):&#xA;    &amp;quot;&amp;quot;&amp;quot;Returns a dictionary with kmers and it counts.&amp;quot;&amp;quot;&amp;quot;&#xA;    seq = sequence.upper()&#xA;    seq_len = len(seq)&#xA;    kmers = [seq[i:i+k] for i in range(0, seq_len - k + 1)]&#xA;    filterd_kmers = [kmer for kmer in kmers if &#xA;                     all(base in set(alphabet) for base in kmer)]&#xA;    return Counter(filterd_kmers)&#xA;&#xA;pool = multiprocessing.Pool(4)&#xA;&#xA;&#xA;args = [''.join(alphabet.iupac_dna), 4] # in the real code it came from argparse&#xA;f = partial(kmer_count,seq)&#xA;pool.starmap(f, *args)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Result:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;---------------------------------------------------------------------------&#xA;RemoteTraceback                           Traceback (most recent call last)&#xA;RemoteTraceback: &#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;Traceback (most recent call last):&#xA;  File &amp;quot;/opt/anaconda3/lib/python3.7/multiprocessing/pool.py&amp;quot;, line 121, in worker&#xA;    result = (True, func(*args, **kwds))&#xA;  File &amp;quot;/opt/anaconda3/lib/python3.7/multiprocessing/pool.py&amp;quot;, line 47, in starmapstar&#xA;    return list(itertools.starmap(args[0], args[1]))&#xA;TypeError: kmer_count() missing 1 required positional argument: 'k'&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;The above exception was the direct cause of the following exception:&#xA;&#xA;TypeError                                 Traceback (most recent call last)&#xA;&amp;lt;ipython-input-79-0f5ba8381a51&amp;gt; in &amp;lt;module&amp;gt;&#xA;      1 f = partial(kmer_count,seq)&#xA;----&amp;gt; 2 pool.starmap(f, *args)&#xA;&#xA;/opt/anaconda3/lib/python3.7/multiprocessing/pool.py in starmap(self, func, iterable, chunksize)&#xA;    274         `func` and (a, b) becomes func(a, b).&#xA;    275         '''&#xA;--&amp;gt; 276         return self._map_async(func, iterable, starmapstar, chunksize).get()&#xA;    277 &#xA;    278     def starmap_async(self, func, iterable, chunksize=None, callback=None,&#xA;&#xA;/opt/anaconda3/lib/python3.7/multiprocessing/pool.py in get(self, timeout)&#xA;    655             return self._value&#xA;    656         else:&#xA;--&amp;gt; 657             raise self._value&#xA;    658 &#xA;    659     def _set(self, i, obj):&#xA;&#xA;TypeError: kmer_count() missing 1 required positional argument: 'k'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If I run like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;f(*args)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I got this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Counter({'GTTC': 9780,&#xA;         'TTCG': 11291,&#xA;         'TCGC': 14803,&#xA;         'CGCC': 14428,&#xA;         'GCCA': 19254,&#xA;         'CCAG': 11512,&#xA;         'CAGA': 11641,&#xA;         'AGAG': 10387,&#xA;         'GAGC': 13164,&#xA;         'AGCG': 16025,&#xA;         'GCGG': 13067,&#xA;         'CGGT': 12329,&#xA;         'GGTT': 14217,&#xA;         'GTTT': 18733,&#xA;         'TTTT': 25488,&#xA;         'TTTG': 22006,&#xA;         'TTGA': 20004,&#xA;         'TGAC': 11351,&#xA;         'GACT': 7262,&#xA;         'ACTA': 5663,&#xA;         'CTAG': 4190,&#xA;         'TAGC': 8543,&#xA;         'AGCT': 12898,&#xA;         'GCTT': 19114,&#xA;         'CTTG': 16557,&#xA;         'TGAA': 17703,&#xA;         'GAAC': 9334,&#xA;         'AACA': 14050,&#xA;         'ACAC': 8812,&#xA;         'CACA': 11050,&#xA;         'ACAT': 9705,&#xA;         'CATC': 17357,&#xA;         'ATCC': 11157,&#xA;         'TCCC': 5754,&#xA;         'CCCG': 5926,&#xA;         'CCGT': 8601,&#xA;         'CGTC': 7401,&#xA;         'GTCC': 3811,&#xA;         'TCCT': 5890,&#xA;         'CCTG': 7985,&#xA;         'CTGC': 15324,&#xA;         'TGCG': 14856,&#xA;         'TTTA': 15607,&#xA;         'TTAG': 8366,&#xA;         'TAGG': 5393,&#xA;         'AGGC': 9580,&#xA;         'GGCA': 13939,&#xA;         'GCAC': 11686,&#xA;         'CACC': 14496,&#xA;         'ACCA': 16950,&#xA;         'CCAC': 13028,&#xA;         'CACT': 13194,&#xA;         'ACTT': 12585,&#xA;         'TTGC': 18877,&#xA;         'TGCA': 13770,&#xA;         'GCAT': 13307,&#xA;         'CATT': 14102,&#xA;         'ATTA': 9776,&#xA;         'TTAA': 12585,&#xA;         'TAAG': 9048,&#xA;         'AAGC': 18834,&#xA;         'AGCC': 12734,&#xA;         'CCAT': 13799,&#xA;         'CATA': 8188,&#xA;         'ATAA': 12278,&#xA;         'GCGT': 12546,&#xA;         'CGTT': 12599,&#xA;         'GTTG': 15136,&#xA;         'TTGG': 18903,&#xA;         'TGGC': 19391,&#xA;         'GGCC': 7685,&#xA;         'GCCC': 7404,&#xA;         'CCCA': 9752,&#xA;         'CCAA': 18755,&#xA;         'CAAC': 14852,&#xA;         'AACT': 11886,&#xA;         'ACTC': 10028,&#xA;         'CTCA': 13518,&#xA;         'TCAC': 15385,&#xA;         'ACCG': 12027,&#xA;         'CCGC': 13105,&#xA;         'CGCA': 14853,&#xA;         'ATAG': 7066,&#xA;         'AGGG': 5273,&#xA;         'GGGG': 4927,&#xA;         'GGGT': 7972,&#xA;         'GAAA': 16511,&#xA;         'AAAC': 18215,&#xA;         'ACTG': 12639,&#xA;         'TGCT': 16663,&#xA;         'CTTT': 18228,&#xA;         'TTTC': 17497,&#xA;         'TCGG': 10116,&#xA;         'TGAT': 20593,&#xA;         'GATG': 17301,&#xA;         'ATGA': 14218,&#xA;         'GATC': 14479,&#xA;         'ATCG': 16604,&#xA;         'CGCG': 13405,&#xA;         'CGTA': 8086,&#xA;         'GTAG': 6829,&#xA;         'TAGA': 6656,&#xA;         'AGAT': 11833,&#xA;         'ATCA': 20344,&#xA;         'TCAG': 13868,&#xA;         'ATCT': 11892,&#xA;         'TCTT': 14706,&#xA;         'CTTC': 13897,&#xA;         'TTCC': 9522,&#xA;         'TCCG': 7099,&#xA;         'CCGG': 5935,&#xA;         'CGGG': 5786,&#xA;         'GGGA': 5615,&#xA;         'GGAT': 11054,&#xA;         'GATA': 12085,&#xA;         'ATAT': 9205,&#xA;         'TATC': 11941,&#xA;         'TCGT': 8802,&#xA;         'GTAT': 8661,&#xA;         'TATT': 12620,&#xA;         'ATTT': 18149,&#xA;         'TGAG': 13150,&#xA;         'GAGA': 8658,&#xA;         'GAGG': 5644,&#xA;         'GCCT': 9673,&#xA;         'CCTT': 11377,&#xA;         'CTTA': 9100,&#xA;         'TTAC': 11418,&#xA;         'TACG': 8250,&#xA;         'ACGC': 12774,&#xA;         'CACG': 10171,&#xA;         'ACGA': 8951,&#xA;         'CGAT': 16617,&#xA;         'ATTG': 17372,&#xA;         'GGCG': 14257,&#xA;         'TTGT': 13287,&#xA;         'TGTA': 7808,&#xA;         'ACGG': 8529,&#xA;         'CGGA': 7109,&#xA;         'ATGT': 9975,&#xA;         'TGTC': 8286,&#xA;         'GTCG': 9088,&#xA;         'TCGA': 11986,&#xA;         'GATT': 16232,&#xA;         'GACC': 7608,&#xA;         'CATG': 11552,&#xA;         'ATGG': 13703,&#xA;         'TGGG': 9728,&#xA;         'CGAC': 9020,&#xA;         'GACG': 7394,&#xA;         'ACGT': 8085,&#xA;         'GTAA': 11360,&#xA;         'TAAT': 9895,&#xA;         'AATA': 12342,&#xA;         'GGCT': 12713,&#xA;         'AAAT': 17834,&#xA;         'AATC': 16220,&#xA;         'TTCA': 18526,&#xA;         'TCAA': 20304,&#xA;         'CAAA': 21752,&#xA;         'AATG': 14200,&#xA;         'TGGT': 16952,&#xA;         'GTTA': 10179,&#xA;         'GCGA': 14624,&#xA;         'AGGT': 9338,&#xA;         'GGTA': 10326,&#xA;         'CGGC': 12423,&#xA;         'TAAC': 9956,&#xA;         'AACG': 12524,&#xA;         'GCGC': 18306,&#xA;         'CTGG': 11344,&#xA;         'TGGA': 10805,&#xA;         'GCTG': 17455,&#xA;         'CTGA': 13532,&#xA;         'CGAG': 8649,&#xA;         'GAGT': 10014,&#xA;         'AGTA': 8668,&#xA;         'GTAC': 8038,&#xA;         'TACC': 10217,&#xA;         'ACCT': 9204,&#xA;         'ATAC': 8527,&#xA;         'TACA': 7453,&#xA;         'ACAA': 12591,&#xA;         'AAAA': 24650,&#xA;         'AAAG': 17848,&#xA;         'AAGG': 11240,&#xA;         'AGTT': 12431,&#xA;         'CAAT': 17456,&#xA;         'GTCT': 6235,&#xA;         'AGGA': 5937,&#xA;         'GGAG': 5294,&#xA;         'AGAA': 12516,&#xA;         'AAGA': 13980,&#xA;         'CGTG': 9948,&#xA;         'GTGC': 11762,&#xA;         'TCAT': 14388,&#xA;         'AGTC': 7451,&#xA;         'GTCA': 11557,&#xA;         'CAGT': 13149,&#xA;         'GGTC': 7553,&#xA;         'GTGA': 15043,&#xA;         'TAAA': 15634,&#xA;         'GGGC': 7200,&#xA;         'GGTG': 14495,&#xA;         'TTCT': 13230,&#xA;         'TCTA': 6621,&#xA;         'CTAC': 6510,&#xA;         'TACT': 8573,&#xA;         'CTAT': 6856,&#xA;         'TATG': 7993,&#xA;         'ATTC': 11395,&#xA;         'GAAG': 13449,&#xA;         'AAGT': 12578,&#xA;         'GCTA': 8346,&#xA;         'CTAA': 8310,&#xA;         'TAGT': 5859,&#xA;         'GACA': 8060,&#xA;         'TTAT': 12293,&#xA;         'TCCA': 11138,&#xA;         'CAGC': 17389,&#xA;         'GCTC': 13661,&#xA;         'CTCG': 8724,&#xA;         'CCCC': 5036,&#xA;         'TGCC': 13824,&#xA;         'GCCG': 12340,&#xA;         'GGAA': 9364,&#xA;         'CTGT': 9651,&#xA;         'TGTT': 14580,&#xA;         'ATGC': 13150,&#xA;         'AGTG': 13050,&#xA;         'GTGT': 9052,&#xA;         'TCTG': 11772,&#xA;         'AATT': 13738,&#xA;         'CGAA': 11026,&#xA;         'GTGG': 12927,&#xA;         'TCTC': 8866,&#xA;         'ACAG': 9505,&#xA;         'CCTC': 5686,&#xA;         'CTCC': 5391,&#xA;         'CCTA': 5236,&#xA;         'CGCT': 16302,&#xA;         'AACC': 13897,&#xA;         'ACCC': 8037,&#xA;         'GCAA': 18697,&#xA;         'CAAG': 16287,&#xA;         'AGCA': 16273,&#xA;         'GCAG': 15145,&#xA;         'CCGA': 9751,&#xA;         'CAGG': 7851,&#xA;         'TATA': 4461,&#xA;         'GGAC': 3754,&#xA;         'CTCT': 10608,&#xA;         'TGTG': 11291,&#xA;         'AGAC': 6199,&#xA;         'CCCT': 5517,&#xA;         'GAAT': 11315}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What I am doing wrong with the multiprocessing step. I want to speed up my analysis, because I have a lot of genomes in a directory to process.&#xA;I just need to figure out how can I implement the pool, because probably all other functions a similar.&lt;/p&gt;&#xA;&lt;p&gt;Thanks for your time and attention&lt;/p&gt;&#xA;" OwnerUserId="4222" LastActivityDate="2020-09-28T11:08:46.157" Title="Truble to run a multiprocessing kmer count script" Tags="|python|genome|k-mer|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14438" PostTypeId="1" AcceptedAnswerId="14445" CreationDate="2020-09-27T22:41:38.550" Score="0" ViewCount="225" Body="&lt;p&gt;I have a figure in a paper where I show the log2(fold-change) (obtained with DESeq2) between two groups (based on genotype) for genes within a specific hallmark gene set, and a reviewer is asking whether the differences are statistically significant.&lt;/p&gt;&#xA;&lt;p&gt;I have not been able to find any methods for comparing expression of groups of genes like this, and i'm sure there is a method or a reason why it isn't done that I'm not aware of.&lt;/p&gt;&#xA;&lt;p&gt;To be extra clear, what i have is two sets of log2FC values for a set of genes (one for one genotype (drug-treated vs. untreated) and the same for a second genotype), and I'm looking for a statistical comparison of these two groups of Log2FC measurements.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="39" LastEditorUserId="73" LastEditDate="2020-09-30T11:28:43.840" LastActivityDate="2020-09-30T11:28:43.840" Title="Is there an established method for comparing expression of groups of genes (gene sets)?" Tags="|differential-expression|deseq2|gsea|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14439" PostTypeId="2" ParentId="14437" CreationDate="2020-09-28T00:12:47.420" Score="2" Body="&lt;p&gt;This code help me to get the answer I was looking for.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;r = pool.apply_async(f, a)&#xA;print(r.get())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4222" LastEditorUserId="3694" LastEditDate="2020-09-28T11:08:46.157" LastActivityDate="2020-09-28T11:08:46.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14441" PostTypeId="1" AcceptedAnswerId="14444" CreationDate="2020-09-28T10:38:13.080" Score="0" ViewCount="160" Body="&lt;p&gt;I'll start by saying I'm a total stranger to this field, so I'd love to be corrected if I misunderstood something along the way.&lt;/p&gt;&#xA;&lt;p&gt;I'm writing a python code to identify all the reading frames in a sequence of nucleotides. To my understanding, a reading frame is defined by a start codon and a stop codon, which made total sense to me until I learned you can have nested/overlapping reading frames.&lt;/p&gt;&#xA;&lt;p&gt;My question is:&lt;/p&gt;&#xA;&lt;p&gt;Assuming the start codon &amp;quot;AUG&amp;quot;, and the stop codon &amp;quot;UAA&amp;quot;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;AUG&lt;/strong&gt; AAA GGG &lt;strong&gt;AUG&lt;/strong&gt; CCC UUU &lt;strong&gt;UAA&lt;/strong&gt; AGA CUC &lt;strong&gt;UAA&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;How do you identify the reading frames? Is each stop codon referring to a single start codon? If so, how can you tell to which one?&lt;/p&gt;&#xA;&lt;p&gt;I tried looking this up and could only come up with articles explaining the effects of overlapping reading frames, rather than how are they interpreted. I would greatly appreciate even a nudge in the right direction.&lt;/p&gt;&#xA;&lt;p&gt;Please let me know if I was not clear enough or if I got something wrong.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&lt;p&gt;EDIT: After reading the answers here and researching further I learned that ORF identification on DNA sequences isn't as simple as finding a single start and stop codon, and is complicated beyond the scope of my question. I'll probably default to using online tools to do this from now on.&lt;/p&gt;&#xA;" OwnerUserId="9598" LastEditorUserId="9598" LastEditDate="2020-09-29T19:04:27.617" LastActivityDate="2020-09-29T19:04:27.617" Title="Identifying nested/overlapping reading frames in nucleotide sequences" Tags="|rna-seq|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14442" PostTypeId="2" ParentId="14441" CreationDate="2020-09-28T20:20:21.513" Score="0" Body="&lt;p&gt;An RNA sequence has six possible reading frames depending on the offset (0, 1, 2) and the direction (forward, reverse complement). All frames can potentially be used for  protein translation, but only one frame per protein (i.e. there is no shifting going on when translation is happening, excluding shifts introduced by splicing).&lt;/p&gt;&#xA;&lt;p&gt;If you need to find all the possible protein sequences within an RNA sequence, there are plenty of programs already available (e.g. &lt;a href=&quot;http://emboss.bioinformatics.nl/cgi-bin/emboss/getorf&quot; rel=&quot;nofollow noreferrer&quot;&gt;getorf&lt;/a&gt; and &lt;a href=&quot;http://emboss.bioinformatics.nl/cgi-bin/emboss/transeq&quot; rel=&quot;nofollow noreferrer&quot;&gt;transseq&lt;/a&gt; from the EMBOSS tool kit).&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-09-28T20:25:22.120" LastActivityDate="2020-09-28T20:25:22.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14443" PostTypeId="1" AcceptedAnswerId="14447" CreationDate="2020-09-28T22:16:17.887" Score="1" ViewCount="594" Body="&lt;p&gt;I am trying to run seurat on ~5000 sinngle cells. I am expecting minimum 15 cell types to be present in the data. I tried to runn it with multiple conditions;I can see there is 27 clusters; I believe many of them might be sub clusters. Is there a way to know which clusters are actually a sub-clusters.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QjGtS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QjGtS.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As suggested by the members I have used clustree to check the cluster stability also changes the clustering parameter.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/L18gk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/L18gk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4951" LastEditorUserId="4951" LastEditDate="2020-09-30T23:11:40.477" LastActivityDate="2020-09-30T23:11:40.477" Title="Single cell RNAseq cell cluster (true cluster or sub cluster)" Tags="|seurat|single-cell|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14444" PostTypeId="2" ParentId="14441" CreationDate="2020-09-28T23:09:28.887" Score="0" Body="&lt;p&gt;There are 6 reading frames.  Three forward, three in the reverse direction.&lt;/p&gt;&#xA;&lt;p&gt;An Open Reading Frame, an ORF, is a sequence which begins with a start codon, and ends with a stop &lt;strong&gt;in the same frame&lt;/strong&gt;.  You find them by looking for sequences that fit that pattern.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is each stop codon referring to a single start codon? If so, how can&#xA;you tell to which one?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You could have two different transcripts for the same gene with different start codons but the same stop codon.  But in general, the protein translation apparatus will not accidently read through a stop codon.&lt;/p&gt;&#xA;&lt;p&gt;And yeah, I hope you are doing this for fun, because obviously people have already made publicly available tools to find ORFs.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-09-28T23:09:28.887" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14445" PostTypeId="2" ParentId="14438" CreationDate="2020-09-29T08:10:41.613" Score="2" Body="&lt;p&gt;You could apply the &lt;a href=&quot;https://bioconductor.org/packages/GSVA&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSVA&lt;/a&gt; transformation to later compare them. You would need to give a step back and start from the expression values not log2FC, but I think it would be better than using the log2FC directly.&lt;/p&gt;&#xA;&lt;p&gt;If you need to use them you could compare the gene set enrichment value of these two sets on the same comparison. I would use &lt;a href=&quot;https://bioconductor.org/packages/fgsea&quot; rel=&quot;nofollow noreferrer&quot;&gt;fgsea&lt;/a&gt; with the log2FC and the two sets of genes. If the normalized enrichment value (NES) is different then you could explain that these sets expression are indeed different.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2020-09-29T08:10:41.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14446" PostTypeId="2" ParentId="14443" CreationDate="2020-09-29T09:33:35.560" Score="1" Body="&lt;p&gt;You should browse the web for methods to assess cluster stability. The idea is to permutate clusters by randomly adding and subtracting cells of a given cluster and see whether it collapses or not. I would bet money that many of your clusters are not stable, e.g. 17 and 2. Try to change clustering parameters to decrease resolution in order to get larger clusters. I am not a Seurat user but typically there are resolution and nearest-neighbor parameters, check the documentation. See for example section 10.6 in &lt;a href=&quot;https://osca.bioconductor.org/clustering.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://osca.bioconductor.org/clustering.html&lt;/a&gt; as well as &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2020.05.26.116640v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/10.1101/2020.05.26.116640v1&lt;/a&gt; for inspiration towards cluster stability assessment.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-09-29T09:33:35.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14447" PostTypeId="2" ParentId="14443" CreationDate="2020-09-29T10:22:03.017" Score="2" Body="&lt;p&gt;I wouldn't be surprised if the 15 cell types that you are expecting were mostly characterized via &amp;quot;protein-based&amp;quot; methods; antibody staining, flow cytometry using fluorophore coupled antibodies, ... The data that scRNA-seq is RNA based and should not be expected to provide the same information as protein based assays. Moreover, the scRNA-seq data, especially those from microfluidic systems like 10x, are sparse and might not be enough to resolve closely related cell types/states such as those of T cells.&lt;/p&gt;&#xA;&lt;p&gt;On top of the aforementioned, evaluating clusters or clustering stability is a difficult task and in my experience, quite some cluster evaluation metrics cannot be easily applied to single cell data simply because of the shear size of the data. Having asked your question and looked for answers (and I am still doing that), I am doing the following:&lt;/p&gt;&#xA;&lt;p&gt;i) Try to use transcriptomic markers as much as possible. Even then a lack of a marker would not mean much in terms of scRNA-seq, it might very well not be detected simply due to sensitivity&lt;/p&gt;&#xA;&lt;p&gt;ii) Use &lt;a href=&quot;https://cran.r-project.org/web/packages/clustree/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;clustree&lt;/a&gt; do select the most plausible number of clusters (resolution parameter in Seurat). The package is compatible with Seurat and some other scRNA-seq packages.&lt;/p&gt;&#xA;&lt;p&gt;iii) Use silhouette width as a metric for clustering. This is computationally expensive, however, should be fine for your 5000 cells. For larger cell numbers, I use another package that approximates this (will add a link but first I have go through my scripts for the name).&lt;/p&gt;&#xA;&lt;p&gt;iv) Check if batch effects are accounting for what you call &amp;quot;sub-clusters&amp;quot;. For example clusters 0, 14, 24 and 25 above might correspond to different samples processed in different days.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2020-09-29T10:31:49.230" LastActivityDate="2020-09-29T10:31:49.230" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14448" PostTypeId="2" ParentId="14435" CreationDate="2020-09-29T12:53:30.487" Score="0" Body="&lt;p&gt;More of a comment than an answer... In my opinion the terminology is a bit confusing since the term &lt;em&gt;reads&lt;/em&gt; actually refers to a more generic &lt;em&gt;records&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Look at this test file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view test.sam | cut -f 1-4&#xA;HSQ9103:404:C6F0VANXX:1:2208:4363:50381 0   chr7    5529094&#xA;HSQ9103:404:C6F0VANXX:1:2208:4363:50381 0   chr7    5529194&#xA;HSQ9103:404:C6F0VANXX:1:2208:4363:50381 0   chr7    5529194&#xA;HSQ9103:404:C6F0VANXX:1:2208:4363:50381 0   chr7    5529194&#xA;HSQ9103:403:C6F0HANXX:8:2215:4792:20065 4   chr7    5529871&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would say that here there are 5 records (of which 4 are alignments) but only 2 &lt;strong&gt;reads&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;samtools flagstat&lt;/code&gt; reports 80% mapping rate but maybe I would say this is 50%.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools flagstat test.sam &#xA;5 + 0 in total (QC-passed reads + QC-failed reads)&#xA;0 + 0 secondary&#xA;0 + 0 supplementary&#xA;0 + 0 duplicates&#xA;4 + 0 mapped (80.00% : N/A)&#xA;0 + 0 paired in sequencing&#xA;0 + 0 read1&#xA;0 + 0 read2&#xA;0 + 0 properly paired (N/A : N/A)&#xA;0 + 0 with itself and mate mapped&#xA;0 + 0 singletons (N/A : N/A)&#xA;0 + 0 with mate mapped to a different chr&#xA;0 + 0 with mate mapped to a different chr (mapQ&amp;gt;=5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="339" LastActivityDate="2020-09-29T12:53:30.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14449" PostTypeId="2" ParentId="4976" CreationDate="2020-09-29T20:44:38.243" Score="2" Body="&lt;p&gt;As an alternative, you can use the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets/docs/command-line-start/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Datasets CLI&lt;/a&gt;, in a two step process to generate a dehydrated dataset for later &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets/docs/rehydrate/&quot; rel=&quot;nofollow noreferrer&quot;&gt;hydration&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Download metadata for all Cyanobacteria &#xA;# And pull out accession for 'Complete' genomes&#xA;./datasets assembly-descriptors taxon Cyanobacteria --limit ALL \&#xA;     | jq -r '.assemblies[].assembly | &amp;quot;\(.assembly_accession) \(.assembly_level)&amp;quot;' \&#xA;     | grep Complete \&#xA;     | cut -f 1 -d ' ' &amp;gt; accs.txt&#xA;&#xA;# Download a dehydrated dataset for the provided set of accessions&#xA;./datasets download assembly -i accs.txt -f 1117_complete.zip \&#xA;     --dehydrated \&#xA;     --exclude-protein \&#xA;     --exclude-rna \&#xA;     --exclude-seq \&#xA;     --exclude-gff3 \&#xA;     --include-gbff &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You may be interested in some of the other files that are excluded.&lt;/p&gt;&#xA;&lt;p&gt;Once you have the dehydrated dataset, you can rehydrate it when you are ready to analyze the data with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;unzip 1117_complete.zip -d 1117_complete&#xA;./datasets rehydrate -f 1117_complete&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2020-09-29T20:44:38.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14450" PostTypeId="2" ParentId="13720" CreationDate="2020-09-29T20:56:42.570" Score="2" Body="&lt;p&gt;If your dataset is limited to human SNPs, and you know the list of SNPs, I would advise taking a look at the dedicated &lt;a href=&quot;https://api.ncbi.nlm.nih.gov/variation/v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Variation Services&lt;/a&gt;.  This would be one request for each SNP, but if you fire off 1-2 per second, I think you'll be ok.&lt;/p&gt;&#xA;&lt;p&gt;For example, for rs1490381980, you could try something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;curl -X GET 'https://api.ncbi.nlm.nih.gov/variation/v0/refsnp/1490381980' \&#xA;   | jq '.primary_snapshot_data.placements_with_allele[] | &amp;quot;\(.seq_id) \(.alleles[0].allele.spdi.position)&amp;quot;'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It would be better to query for the SNP and parse the JSON in python, but &lt;code&gt;jq&lt;/code&gt; helps provide an idea of what is possible.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, you could download the JSON or VCF data from the &lt;a href=&quot;https://ftp.ncbi.nih.gov/snp/latest_release/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI dbSNP FTP site&lt;/a&gt;, and process out the information you need.&lt;/p&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2020-09-29T20:56:42.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14452" PostTypeId="1" CreationDate="2020-09-30T10:14:03.980" Score="2" ViewCount="60" Body="&lt;p&gt;When (as a mistake) I did &lt;strong&gt;not&lt;/strong&gt; remove low counts at all (beside those that equal zero for all samples), I got the following ma plot (using Glimma) :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/s5far.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/s5far.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;On  the right you see the individual counts. The gene that is displayed &lt;strong&gt;is&lt;/strong&gt; signficant. However, the reality seems that the gene is not expressed both in treatment and in control. The batch I have corrected for made it a significantly DE however.&lt;/p&gt;&#xA;&lt;p&gt;The batch correction algorithm of DESeq2 doesn't care whether the counts of the gene are almost all zeroes, and does not take this as a special case.&#xA;In reality, it seems to me that it &lt;strong&gt;is&lt;/strong&gt; a special case - if it is questionable whether the gene is expressed in the first place, it makes no sense to apply batch correction to zero counts.&lt;/p&gt;&#xA;&lt;p&gt;The solution to this seems to be to make sure to remove low counts when applying batch correction. Usually it's only a recommendation; but when applying batch, it seems a necessity.&lt;/p&gt;&#xA;&lt;p&gt;Would you agree?&lt;/p&gt;&#xA;&lt;p&gt;I even wonder if one should remove counts that are low per-condition. Suppose that a gene is not expressed in treatment (and only in treatment), does it really make sense to apply batch correction to this gene?&lt;/p&gt;&#xA;" OwnerUserId="6317" LastEditorUserId="6317" LastEditDate="2020-09-30T10:31:19.590" LastActivityDate="2020-10-30T20:04:00.413" Title="Should one be especially conscious of removing low counts, when applying batch correction?" Tags="|batch-effects|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14453" PostTypeId="1" AcceptedAnswerId="14472" CreationDate="2020-09-30T12:06:43.207" Score="2" ViewCount="137" Body="&lt;p&gt;I am doing a pathway enrichment analysis of a sets of proteins.&#xA;Specific cell types (cardiospheres) were used to perform the experiment and for the quantification of proteins was used mass spectrometry.&#xA;I am using gProfiler to perform pathway analysis, should I use custom background of all of those proteins identified from the experiment? or should I used without custom background?&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;&lt;p&gt;UPDATE: Also, I am doing an enrichment analysis of the interactome of the set of proteins of interest. In that case should I used a custom background for the enrichment analysis or not?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!!&lt;/p&gt;&#xA;" OwnerUserId="9010" LastEditorUserId="9010" LastEditDate="2020-10-22T09:13:33.390" LastActivityDate="2020-10-22T09:13:33.390" Title="Pathway enrichment analysis with or without custom background" Tags="|pathway|go-enrichment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14454" PostTypeId="1" CreationDate="2020-09-30T12:56:47.153" Score="1" ViewCount="17" Body="&lt;p&gt;I am looking to find a way to map a list of biomarkers of interest, say genes or bacterial functions, to existing annotation databases. Usually this is done in reverse, as such that based on metagenomics profiling you can see which pathways or genes are present. However, I want to be able to determine which genes or pathways to extract beforehand i.e. as I think it will be quite difficult to condense pathways or genes of interest from annotation data.&lt;/p&gt;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&lt;p&gt;Biomarker gene for better sleep - maps to annotation database at ...&#xA;Biomarker gene for glucose metabolism - maps to annotation database at..&#xA;.. which in turn allows to directly quantify those biomarkers.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="9613" LastActivityDate="2020-09-30T12:56:47.153" Title="Finding the relevant annotations for biomarkers" Tags="|sequence-annotation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14455" PostTypeId="1" CreationDate="2020-09-30T14:37:33.490" Score="2" ViewCount="324" Body="&lt;p&gt;I have set of annotated term of drugs extracted from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/research/pubtator/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PubTator&lt;/a&gt; those drugs are annotated in form of MeSH terms. I would like to map them into drugbank ID. I do search the the MeSH term i find the reference to the drug and link to the drugbank. But not any database as such. How do i map the MeSH term to the drugbank ID.?&lt;/p&gt;&#xA;&lt;p&gt;My small dataset.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Terms         MESH&#xA;1        ara-C MESH:D003561&#xA;2 mitoxantrone MESH:D008942&#xA;3    etoposide MESH:D005047&#xA;4  fludarabine MESH:C024352&#xA;5   gemtuzumab MESH:C406061&#xA;6   ozogamicin            -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As an example of one of the drug I would give this drug &amp;quot;gemtuzumab&amp;quot;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;DB00056 Gemtuzumab ozogamicin   Myeloid cell surface antigen CD33   target&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So i want to map the above drugs into drugbank ID and add the information such as the class which is here target or it can be a transporter or enzyme etc.&lt;/p&gt;&#xA;&lt;p&gt;The issue is there is no common ID between the drug MEsH and the drug-bank ID.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion or help would be highly appreciated if anyone can cite me some resources.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;em&gt;PubTrator description from the website&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;PubTator Central (PTC) is a Web-based system providing automatic annotations of biomedical concepts such as genes and mutations in PubMed abstracts and PMC full-text articles.&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="3848" LastEditDate="2022-03-18T13:29:59.267" LastActivityDate="2022-03-18T13:29:59.267" Title="Convert MeSH term to drugbank ID" Tags="|public-databases|drugs|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14456" PostTypeId="1" CreationDate="2020-09-30T19:02:44.687" Score="0" ViewCount="34" Body="&lt;p&gt;I am trying to understand library normalization in DESeq2. I would like to ask the following: I know that some samples have been run 15 cycles and some others 20, can I give this information to DESeq2, would it be useful? I mean, I know that DESEQ2 uses an algorithm to normalize library size, but since I have this piece of information, I wonder if it would be useful to provide it somehow to DESEQ2. If it is not, would it make sense to incorporate this information in the design matrix, something like: design(~cycle+condition)? Thank you a lot!&lt;/p&gt;&#xA;" OwnerUserId="8713" LastEditorUserId="8713" LastEditDate="2020-09-30T19:53:46.357" LastActivityDate="2020-09-30T19:56:17.207" Title="if I know the number of sequencing circles can I give this information to DESeq2?" Tags="|rna-seq|normalization|library|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="14457" PostTypeId="2" ParentId="14452" CreationDate="2020-09-30T19:34:05.417" Score="1" Body="&lt;p&gt;I don't think this has anything to do with batch correction.&lt;/p&gt;&#xA;&lt;p&gt;The averages of that gene are distinctly different between groups, so it's correct that the gene is statistically significant.  But with one sample driving all the difference, it's likely to be an artifact, or otherwise not fruitful to follow up on.&lt;/p&gt;&#xA;&lt;p&gt;As geek_y suggested, a filter like &amp;quot;omit genes unless 2 samples have &amp;gt; 10 counts&amp;quot; would drop that gene.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-09-30T19:34:05.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14458" PostTypeId="2" ParentId="14456" CreationDate="2020-09-30T19:56:17.207" Score="2" Body="&lt;p&gt;You can include that info in the design, though you'd have to decide whether to include it as a number of just a factor.  I've never heard of anyone including PCR cycles.  I've heard of people including RIN scores, though I've never done it.&lt;/p&gt;&#xA;&lt;p&gt;If the PCA looks alright, I wouldn't worry about.  I've got tons of examples of the lab people saying &amp;quot;I did a few more cycles on these few samples&amp;quot; and then you can't tell any difference in the PCA.  I think needing to do more cycles likely has more to do with variance in the prep than a real biological difference.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-09-30T19:56:17.207" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14459" PostTypeId="2" ParentId="14101" CreationDate="2020-09-30T22:24:07.020" Score="4" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;There are other ways to do it (many and some might be easier), but if you want to do it with BioPython PDB module you can iterate the residues like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; # You can use a dict to convert three letter code to one letter code&#xA; d3to1 = {'CYS': 'C', 'ASP': 'D', 'SER': 'S', 'GLN': 'Q', 'LYS': 'K',&#xA; 'ILE': 'I', 'PRO': 'P', 'THR': 'T', 'PHE': 'F', 'ASN': 'N', &#xA; 'GLY': 'G', 'HIS': 'H', 'LEU': 'L', 'ARG': 'R', 'TRP': 'W', &#xA; 'ALA': 'A', 'VAL':'V', 'GLU': 'E', 'TYR': 'Y', 'MET': 'M'}&#xA;&#xA;&#xA; # Just an example input pdb&#xA; record = '1pa2.pdb'&#xA;&#xA; # run parser&#xA; parser = PDBParser(QUIET=True)&#xA; structure = parser.get_structure('struct', record)    &#xA;&#xA; # iterate each model, chain, and residue&#xA; # printing out the sequence for each chain&#xA;&#xA; for model in structure:&#xA;     for chain in model:&#xA;         seq = []&#xA;         for residue in chain:&#xA;             seq.append(d3to1[residue.resname])&#xA;         print('&amp;gt;some_header\n',''.join(seq))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Not really, is very common, you can turn it off like this (taken from &lt;a href=&quot;https://lists.open-bio.org/pipermail/biopython/2014-July/015371.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://lists.open-bio.org/pipermail/biopython/2014-July/015371.html&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; import warnings&#xA; from Bio import PDBConstructionWarning&#xA; #your code&#xA; with warnings.catch_warnings():&#xA; warnings.simplefilter('ignore', PDBConstructionWarning)&#xA; #your code which might trigger the warning&#xA; #rest of your code here&#xA;&#xA; However, because this is so common, you can just use the&#xA; QUIET=True option on the PDBParser object:&#xA;&#xA; from Bio.PDB.PDBParser import PDBParser&#xA; struct = PDBParser(QUIET=True).get_structure('tmp', pdbf)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Try help(PDBParser) for more, or see:&#xA;&lt;a href=&quot;http://biopython.org/DIST/docs/api/Bio.PDB.PDBParser%27.PDBParser-class.html#__init_&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://biopython.org/DIST/docs/api/Bio.PDB.PDBParser%27.PDBParser-class.html#__init_&lt;/a&gt;_&lt;/p&gt;&#xA;" OwnerUserId="9615" LastActivityDate="2020-09-30T22:24:07.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14462" PostTypeId="1" AcceptedAnswerId="14463" CreationDate="2020-10-01T10:07:54.050" Score="0" ViewCount="29" Body="&lt;p&gt;I have several groups of 16S rRNA sequences associated with taxonomic groups (let's call them A, B and C). The sequences within each group have a common ancestor and are on average more closely related to sequences within the same group. The groups are also related, but less.&lt;/p&gt;&#xA;&lt;p&gt;Ideally I'd like to have a table like this (where percentages represent the average similarity):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;           Group A     Group B     Group C&#xA;Group A      99%                  &#xA;Group B      97%         98%              &#xA;Group C      94%         95%         99%&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there software that can calculate these percentages for me? I'm thinking that I could set up a many-against-many BLAST search and then try to use pandas in python in order to parse the resulting table...But if something like this already exists, I'd rather not reinvent the wheel. Thank you!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2020-10-01T11:53:32.927" Title="Comparing groups of 16S rRNA sequences - how to?" Tags="|phylogenetics|taxonomy|sequence-homology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14463" PostTypeId="2" ParentId="14462" CreationDate="2020-10-01T11:53:32.927" Score="1" Body="&lt;p&gt;If you want to make an all vs all with blast this can be helpful. You can create a blast database with your sequences and then search against itself.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;makeblastdb -in 16S_sequences.fasta -dbtype nucl -out my_16S_sequences_db&#xA;&#xA;blastn -db my_16S_sequences_db -query 16S_sequences.fasta -outfmt 6 \&#xA;       -out allvsall_16S.tsv -num_threads &amp;quot;$(nproc)&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you want a percent identity matrix for a pairwise alignment you can try with this approach: &lt;a href=&quot;https://www.biostars.org/p/220154/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/220154/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2020-10-01T11:53:32.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14464" PostTypeId="2" ParentId="7458" CreationDate="2020-10-01T14:30:59.010" Score="7" Body="&lt;p&gt;Reverse complement FASTA/Q:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqtk seq -r in.fa &amp;gt; out.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/lh3/seqtk&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/lh3/seqtk&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9619" LastActivityDate="2020-10-01T14:30:59.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14465" PostTypeId="1" CreationDate="2020-10-01T15:33:13.493" Score="1" ViewCount="124" Body="&lt;p&gt;I've been seeing using sequence alignment to help with prediction of protein folding, but am unable to understand why. Google returns other research papers for these search terms, hence I'm hoping the stackoverflow community can help thank you.&lt;/p&gt;&#xA;" OwnerUserId="9620" LastActivityDate="2020-10-02T14:17:52.913" Title="How does sequence alignment help with protein folding prediction?" Tags="|sequence-alignment|protein-structure|" AnswerCount="1" CommentCount="1" ClosedDate="2020-10-05T17:33:57.673" ContentLicense="CC BY-SA 4.0" />
  <row Id="14466" PostTypeId="2" ParentId="13194" CreationDate="2020-10-01T15:48:15.690" Score="1" Body="&lt;p&gt;For this particular query, try &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/?term=host%3DBemisia%20tabaci%20B%5BWORD%5D&quot; rel=&quot;nofollow noreferrer&quot;&gt;host=Bemisia tabaci B[WORD]&lt;/a&gt;.  However, you may also be interested in related queries like &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/?term=host%3DBemisia%20tabaci%20B%20biotype%5BWORD%5D&quot; rel=&quot;nofollow noreferrer&quot;&gt;host=Bemisia tabaci B biotype[WORD]&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;But, I am not sure a general solution exists, as I do not believe all qualifiers are indexed.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Also, you can see how your query is interpreted on the right side under &amp;quot;Search details&amp;quot;.  For the original attempt (&lt;code&gt;[host]&amp;quot;Bemisia tabaci B&amp;quot;&lt;/code&gt;), it is interpreted as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;host[All Fields] AND &amp;quot;Bemisia tabaci B&amp;quot;[All Fields]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;while the goal is to produce&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;host=Bemisia tabaci B biotype[WORD]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9608" LastActivityDate="2020-10-01T15:48:15.690" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14467" PostTypeId="1" CreationDate="2020-10-01T17:51:13.143" Score="1" ViewCount="525" Body="&lt;p&gt;Using the BLASTn webtool with,&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Query Seq:&lt;/strong&gt; CAATTGCGAAGCCACATTATT&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Subject Seq:&lt;/strong&gt; CP031332.1&lt;/p&gt;&#xA;&lt;p&gt;BLASTn returns a single hit to CP031332.1 with query coverage at 100%.&lt;/p&gt;&#xA;&lt;p&gt;However if one looks at the alignment, the query coverage is clearly 17/21 ≈ 81%. Could someone explain to me the reason for this discrepancy?&lt;/p&gt;&#xA;&lt;p&gt;Edit,&lt;/p&gt;&#xA;&lt;p&gt;Web version: &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PROGRAM=blastn&amp;amp;PAGE_TYPE=BlastSearch&amp;amp;LINK_LOC=blasthome&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PROGRAM=blastn&amp;amp;PAGE_TYPE=BlastSearch&amp;amp;LINK_LOC=blasthome&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Output: &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Get&amp;amp;RID=RCK3F55Y114&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Get&amp;amp;RID=RCK3F55Y114&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1456" LastEditorUserId="1456" LastEditDate="2020-10-01T21:38:06.990" LastActivityDate="2020-10-04T16:59:03.890" Title="BLASTn query coverage discrepancy" Tags="|blast|blastn|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14468" PostTypeId="1" CreationDate="2020-10-01T21:12:51.357" Score="0" ViewCount="356" Body="&lt;p&gt;Recently, I have downloaded a publicly available dataset, which are 350 tumor samples. I see the following information from the published paper.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rz7LG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rz7LG.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;They used Ribo Zero Gold and rRNA was depleted. Strand specific data. After aligning the data I did some alignment quality check with Qualimap RNA-Seq QC tool. I visualised the bam files in IGV. Alignment is good. For all samples 90% alignment rate was seen. I observed that in all samples Higher percentage of mapped reads were originating in Intronic regions. Followed by Exonic and intergenic regions.&lt;/p&gt;&#xA;&lt;p&gt;I have seen a post here &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/4246/reads-mapped-to-exonic-intronic-and-intergenic-regions&quot;&gt;Reads mapped to exonic, intronic and intergenic regions&lt;/a&gt; where they say high intronic reads could be because of contamination. I googled about higher reads in intronic regions and found some papers &lt;a href=&quot;https://www.nature.com/articles/s41598-018-23226-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evaluation of two main RNA-seq approaches for gene quantification in clinical RNA sequencing: polyA+ selection versus rRNA depletion&lt;/a&gt; and some other links &lt;a href=&quot;http://blog.sitoolsbiotech.com/2019/08/ribo-depletion-rna-seq-ribosomal-rna-depletion-method-works-best/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RIBO-DEPLETION IN RNA-SEQ – WHICH RIBOSOMAL RNA DEPLETION METHOD WORKS BEST?&lt;/a&gt; in which they said Greater intronic reads were with rRNA depletion protocol.&lt;/p&gt;&#xA;&lt;p&gt;And even in this &lt;a href=&quot;https://hbctraining.github.io/Intro-to-rnaseq-hpc-salmon/lessons/03_QC_STAR_and_Qualimap_run.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNA-SEQ tutorial&lt;/a&gt;, it is mentioned that - A higher intronic mapping rate is expected for rRNA removal compared to polyA selection.&lt;/p&gt;&#xA;&lt;p&gt;So, my question:&lt;/p&gt;&#xA;&lt;p&gt;I am working with lncRNAs. So, I'm using the samples prepared with rRNA depletion protocol. Is this higher intronic rate is common in rRNA depleted dataset or do I have to check anything else to proceed further with these samples?&lt;/p&gt;&#xA;" OwnerUserId="9622" LastActivityDate="2020-10-02T06:39:56.223" Title="Does rRNA depletion protocol give higher number of mapped reads in Intronic regions?" Tags="|rna-seq|reads|rna-alignment|ribosomal|strandedness|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14469" PostTypeId="2" ParentId="14468" CreationDate="2020-10-01T23:03:44.390" Score="0" Body="&lt;p&gt;I can't possibly see how intron contamination is linked to removal of rRNA depletion.&lt;/p&gt;&#xA;&lt;p&gt;The only reason it would appear to have increased the number of contaminants is because post-rRNA removal the proporation of intron contaminants has increased against the total remaining RNA content. However, the actual total number of contaminants remains exactly the same pre- and post-rRNA removal. By the same token the RNA of interest lncRNAs will also have proportionally increased, so you get a better depth of its predominance and diversity.&lt;/p&gt;&#xA;&lt;p&gt;Thats just life and perhaps just filter this bioinformatically.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-10-01T23:03:44.390" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14470" PostTypeId="2" ParentId="14468" CreationDate="2020-10-02T06:39:56.223" Score="1" Body="&lt;p&gt;It's not so much that you have &amp;quot;intronic contamination&amp;quot; or &amp;quot;genomic contamination&amp;quot;, rather you're not selecting explicitly for full-length mature transcripts with rRNA depletion. That is the most common cause for higher intronic read rates. There's nothing you can do about this post-hoc, just continue along.&lt;/p&gt;&#xA;&lt;p&gt;BTW, many lncRNA's are polyadenylated, so you'll keep them with poly-A selection.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-10-02T06:39:56.223" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="14471" PostTypeId="2" ParentId="14465" CreationDate="2020-10-02T11:34:02.883" Score="4" Body="&lt;p&gt;Protein sequences from the same family form a sequence alignment. Some positions in the alignment will be conserved, i.e. the same in all sequences. These are often associated with function, e.g. a catalytic residue at position 50.&lt;/p&gt;&#xA;&lt;p&gt;However there is a more subtle signal in sequence alignments called &lt;em&gt;covariation&lt;/em&gt;. Two positions, say residue 30 and residue 60, may appear to change together across the alignment. We infer that they are close in 3D space in the structure:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XQ4az.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XQ4az.png&quot; alt=&quot;covariation&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Initially statistical techniques (DCA, PSICOV etc.) were used to separate direct from indirect (chained) couplings and extract predicted contacts from sequence alignments. Now deep neural networks have improved performance and even allowed the prediction of residue-residue distances and orientations.&lt;/p&gt;&#xA;&lt;p&gt;These constraints on the structure allow model building, for example using the CNS software. A good set of constraints effectively guarantees the correct fold. The depth and quality of the sequence alignment is the major factor in the accuracy of the predicted constraints.&lt;/p&gt;&#xA;&lt;p&gt;This approach to protein structure prediction is the current state of the art. Use of covariation from sequence alignments is the only approach that has reliably given accurate models for larger proteins without templates. See &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/31942072/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; and &lt;a href=&quot;https://www.pnas.org/content/117/3/1496.abstract&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; for the best current methods, and tune into the &lt;a href=&quot;http://predictioncenter.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;CASP&lt;/a&gt; conference this December to see if anyone can improve on them.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastEditorUserId="4622" LastEditDate="2020-10-02T14:17:52.913" LastActivityDate="2020-10-02T14:17:52.913" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14472" PostTypeId="2" ParentId="14453" CreationDate="2020-10-02T15:40:57.910" Score="1" Body="&lt;p&gt;I would defninitely go with a custom background which should be all the proteins that were actually measured in your experiment since you should only take into account what was measured.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2020-10-04T21:13:00.513" LastActivityDate="2020-10-04T21:13:00.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14473" PostTypeId="1" AcceptedAnswerId="14476" CreationDate="2020-10-02T18:42:13.693" Score="2" ViewCount="139" Body="&lt;p&gt;I have following example data from a large file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cont100 1128 1125&#xA;cont1005 3642 3642&#xA;cont1006 103 19&#xA;cont1037 3146 3146&#xA;cont104 895 890&#xA;cont1056 934 934&#xA;cont1059 1750 1750&#xA;cont1072 2577 2577&#xA;cont1078 43 42&#xA;cont1081 727 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I need to get only the rows with similar values in column 2 and column 3. The output should be like this.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cont1005 3642 3642&#xA;cont1037 3146 3146&#xA;cont1056 934 934&#xA;cont1059 1750 1750&#xA;cont1072 2577 2577&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Also I need to get the inverse of that. That means I need to get the rows with values that are not similar in the columns. The output should be like this.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cont100 1128 1125&#xA;cont1006 103 19&#xA;cont104 895 890&#xA;cont1078 43 42&#xA;cont1081 727 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="856" LastEditorUserId="3694" LastEditDate="2020-10-02T22:06:20.343" LastActivityDate="2020-10-03T15:41:57.663" Title="How to get rows with similar values in two different columns using command line?" Tags="|bash|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14474" PostTypeId="2" ParentId="13426" CreationDate="2020-10-02T18:46:55.387" Score="2" Body="&lt;p&gt;Let's look at the components of the equation one by one first since it is easier to describe each part on its own. We can do this because they are added together in computing &lt;span class=&quot;math-container&quot;&gt;$Q_{ij}$&lt;/span&gt;. Component1, &lt;span class=&quot;math-container&quot;&gt;$(m-2)d_{ij}$&lt;/span&gt; is saying that we want a quantity which would be the total penalization, if every distance for &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; sequences was &lt;span class=&quot;math-container&quot;&gt;$d_{ij}$&lt;/span&gt; excluding the 2 distances between themselves (it's a direct proportionality so that the &lt;span class=&quot;math-container&quot;&gt;$Q_{ij} \propto (m-2)d_{ij}$&lt;/span&gt;). Component2,&#xA;&lt;span class=&quot;math-container&quot;&gt;$\left(\sum_{k\neq i}d_{ik}\right)$&lt;/span&gt;, says that we look at the sum of the distances sequence &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; has against all other sequences excluding itself&#xA;(it's an inverse proportionality so that the &lt;span class=&quot;math-container&quot;&gt;$Q_{ij} \propto \left(\sum_{k\neq i}d_{ik}\right)^{-1}$&lt;/span&gt;), and &lt;span class=&quot;math-container&quot;&gt;$\left(\sum_{k\neq j}d_{jk}\right)$&lt;/span&gt; as component3 is analogous.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;non-direct neighbours&lt;/em&gt; means that on the phylogenetic tree, 2 extant sequences (sequences in the alignment) have another sequence set that can link them together. If two sequences are definitely related to each other then on the phylogenetic tree they will produce a bifurcation and the smallest distance between themselves as being &lt;em&gt;directly&lt;/em&gt; related, &lt;span class=&quot;math-container&quot;&gt;$(m-2)d_{ij} &amp;lt; \left(\sum_{k\neq i}d_{ik} + \sum_{k\neq j}d_{jk}\right)$&lt;/span&gt;, eg. the &lt;span class=&quot;math-container&quot;&gt;$d_{ij}=1$&lt;/span&gt; and the rest are all greater than 1 gives us a low number. If &lt;span class=&quot;math-container&quot;&gt;$f_{max_{i,j}}(d_{ij})$&lt;/span&gt; is used we'd be using the maximum distance for component1 so the overall measure would be positive and as large as possible.&lt;/p&gt;&#xA;&lt;p&gt;In summary it provides a measure for placing the sequences together with the closest distance. You may say that this is 'overkill' as we can just search the &lt;span class=&quot;math-container&quot;&gt;$m^2/2-m$&lt;/span&gt; set of distances and choose the minimum. In the goal for producing the tree, things get more interesting. Let's say look at 3 sequences &lt;span class=&quot;math-container&quot;&gt;$i,j,k$&lt;/span&gt; of a larger set &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;, and see that sequences together have the same distance between them &lt;span class=&quot;math-container&quot;&gt;$d_{ij}=d_{ik}$&lt;/span&gt; in respect to &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;. &lt;span class=&quot;math-container&quot;&gt;$d_{ij}=d_{j*}$&lt;/span&gt; would mean that &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; is equally distance to all other sequences as it is to &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and let's say &lt;span class=&quot;math-container&quot;&gt;$d_{kz}=max \land d_{ky,y\neq z}=d_{ij}$&lt;/span&gt; (the sequence &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is equidistance to all sequences except to a particular sequence &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; and can happen in some region in the alignment). Which do we join with &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;? According to the formula we choose &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; over &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; because &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is more distant from other sequence(s) as it would provide a more negative number (lower values cause of the 2nd and 3rd components although component1 is the same in both cases). Effectively we want the minimum distance to &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; and simultaneously the maximal distance from other sequences (penalize sequences which could be closely related to the other sequences helps produce an overall better tree).&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Anders Gorm Pedersen (of DTU)&lt;/em&gt; in his &lt;a href=&quot;https://www.youtube.com/watch?v=ro2MFmVZypQ&amp;amp;list=PLXwjzs_mabFrlRF7uALEomfGGckG0sG5y&amp;amp;index=17&quot; rel=&quot;nofollow noreferrer&quot;&gt;coursera videos (also on YT)&lt;/a&gt; provides some numerical examples to see how the numbers are produced on an example tree.&lt;/p&gt;&#xA;" OwnerUserId="9311" LastActivityDate="2020-10-02T18:46:55.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14476" PostTypeId="2" ParentId="14473" CreationDate="2020-10-02T19:49:48.030" Score="8" Body="&lt;p&gt;I think it is easy to do with awk, e.g. see &lt;a href=&quot;https://stackoverflow.com/questions/42092693/awk-concise-way-to-check-if-2-fields-match-the-same-string&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For your example, with your data in &lt;code&gt;file&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;%  awk '&lt;span class=&quot;math-container&quot;&gt;$2 != $&lt;/span&gt;3 {print $0}' file&#xA;# outputs&#xA;cont100 1128 1125&#xA;cont1006 103 19&#xA;cont104 895 890&#xA;cont1078 43 42&#xA;cont1081 727 2&#xA;&#xA;%  awk '&lt;span class=&quot;math-container&quot;&gt;$2 == $&lt;/span&gt;3 {print $0}' file&#xA;# outputs:&#xA;cont1005 3642 3642&#xA;cont1037 3146 3146&#xA;cont1056 934 934&#xA;cont1059 1750 1750&#xA;cont1072 2577 2577&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that where you say &amp;quot;similar&amp;quot; I am interpreting that to mean &amp;quot;exactly the same&amp;quot;, because that is the case in your example.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-10-02T19:49:48.030" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14477" PostTypeId="2" ParentId="14467" CreationDate="2020-10-03T11:58:16.397" Score="0" Body="&lt;p&gt;Your blast result is not composed of a single alignment of all of the length of your query vs the subject but from a series of alignments: in each a subsequence of your query is used and these alignments are sorted by &amp;quot;E value&amp;quot;, &amp;quot;score&amp;quot;, &amp;quot;percent identity&amp;quot;, ...&lt;/p&gt;&#xA;&lt;p&gt;Now in your case, you are talking about the first hit, in which a subsequence of your query (positions 1-17) match perfectly to a portion of the subject. Query coverage is not calculated from this first hit though, it takes all of the alignments into account and again in your case all 21 bases were used in one or more alignments. I believe it would be fair to filter out all but the first hit given the E-values and re-calculate the query coverage as 17/21.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-10-03T11:58:16.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14478" PostTypeId="2" ParentId="14473" CreationDate="2020-10-03T15:41:57.663" Score="2" Body="&lt;p&gt;The Perl solution is similar to the awk solution from &lt;em&gt;MaximilianPress&lt;/em&gt;:&lt;/p&gt;&#xA;&lt;p&gt;Print lines with identical values in columns 2 and 3:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -lane 'print if &lt;span class=&quot;math-container&quot;&gt;$F[1] == $&lt;/span&gt;F[2];' in.txt&#xA;&#xA;# Prints:&#xA;cont1005 3642 3642&#xA;cont1037 3146 3146&#xA;cont1056 934 934&#xA;cont1059 1750 1750&#xA;cont1072 2577 2577&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Print lines with different values in columns 2 and 3:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -lane 'print if &lt;span class=&quot;math-container&quot;&gt;$F[1] != $&lt;/span&gt;F[2];' in.txt&#xA;&#xA;# Prints:&#xA;cont100 1128 1125&#xA;cont1006 103 19&#xA;cont104 895 890&#xA;cont1078 43 42&#xA;cont1081 727 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The Perl one-liner uses these command line flags:&lt;br /&gt;&#xA;&lt;code&gt;-e&lt;/code&gt; : Tells Perl to look for code in-line, instead of in a file.&lt;br /&gt;&#xA;&lt;code&gt;-n&lt;/code&gt; : Loop over the input one line at a time, assigning it to &lt;code&gt;$_&lt;/code&gt; by default.&lt;br /&gt;&#xA;&lt;code&gt;-l&lt;/code&gt; : Strip the input line separator (&lt;code&gt;&amp;quot;\n&amp;quot;&lt;/code&gt; on *NIX by default) before executing the code in-line, and append it when printing.&lt;br /&gt;&#xA;&lt;code&gt;-a&lt;/code&gt; : Split &lt;code&gt;$_&lt;/code&gt; into array &lt;code&gt;@F&lt;/code&gt; on whitespace or on the regex specified in &lt;code&gt;-F&lt;/code&gt; option. Arrays in Perl are 0-indexed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;SEE ALSO:&lt;/strong&gt;&lt;br /&gt;&#xA;&lt;a href=&quot;https://perldoc.perl.org/perlrun.html#Command-Switches&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;perldoc perlrun&lt;/code&gt;: how to execute the Perl interpreter: command line switches&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6251" LastActivityDate="2020-10-03T15:41:57.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14479" PostTypeId="2" ParentId="14467" CreationDate="2020-10-04T16:59:03.890" Score="1" Body="&lt;p&gt;To expand on the excellent answer by &lt;em&gt;haci&lt;/em&gt;:&lt;br /&gt;&#xA;BLAST reports (a) the summary of alignments and (b) individual alignments, see below. Note that your link to the results expired (as NCBI BLAST results links tend to do). It is considered best practice to post the text version of the relevant portion of the results in the question itself.&lt;/p&gt;&#xA;&lt;p&gt;The summary provides an overview of all the query sequence &lt;strong&gt;individual hits&lt;/strong&gt; in the subject sequence. Because the individual hits (alignments, or ranges) &lt;strong&gt;together&lt;/strong&gt; cover 100% of the query and each of these has the max identity = 100%, that is what it reports in the summary. It is not very intuitive for many users.&lt;/p&gt;&#xA;&lt;p&gt;You should always look at the individual hits in detail. The above 100% coverage in the summary consists of ranges 1 and 14, shown below. The top hit (range 1) covers only 17 out of 21 nt with 100% identity, and this is the most relevant information from BLAST. The other hit (range 14) covers 10 out of 21 nt with 100% identity. This is the cause of the discrepancy.&lt;/p&gt;&#xA;&lt;p&gt;These 2 alignments are not close together in the subject sequence. Depending on your goal, choose one or more of the individual alignments and consider alignment length and % identity of those hits individually, rather than summarized together as in the BLAST report summary. For most applications, the top hit (range 1) would be the most relevant alignment.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Program: BLASTN &#xA;Subject:Neisseria meningitidis strain M22814 chromosome, complete genome ID: CP031332.1(nucleic acid) Length: 2190201&#xA;Query #1: seq1 Query ID: lcl|Query_56011 Length: 21&#xA;&#xA;Sequences producing significant alignments:&#xA;                                                                  Max    Total Query   E   Per.                  &#xA;Description                                                       Score  Score cover Value Ident  Accession        &#xA;Neisseria meningitidis strain M22814 chromosome, complete genome  34.2   57729 100%  0.001 100.00 CP031332.1       &#xA;&#xA;&#xA;Alignments:&#xA;&amp;gt;Neisseria meningitidis strain M22814 chromosome, complete genome&#xA;Sequence ID: CP031332.1 Length: 2190201 &#xA;Range 1: 2074883 to 2074899&#xA;&#xA;Score:34.2 bits(17), Expect:0.001, &#xA;Identities:17/17(100%),  Gaps:0/17(0%), Strand: Plus/Plus&#xA;&#xA;Query  1        CAATTGCGAAGCCACAT  17&#xA;                |||||||||||||||||&#xA;Sbjct  2074883  CAATTGCGAAGCCACAT  2074899&#xA;&#xA;...&#xA;&#xA;Range 14: 354292 to 354301&#xA;&#xA;Score:20.3 bits(10), Expect:15, &#xA;Identities:10/10(100%),  Gaps:0/10(0%), Strand: Plus/Minus&#xA;&#xA;Query  12      CCACATTATT  21&#xA;               ||||||||||&#xA;Sbjct  354301  CCACATTATT  354292&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6251" LastActivityDate="2020-10-04T16:59:03.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14481" PostTypeId="1" CreationDate="2020-10-05T13:52:32.983" Score="0" ViewCount="115" Body="&lt;p&gt;I would like to create a simple plot of a given human gene containing the main features, e.g. exons, introns, ORFs, transcripts etc. I understand that &lt;code&gt;gviz&lt;/code&gt;  is a popular choice for this kind of thing, but most tutorials are massive overkill for what I want to do (I don't need fancy features such as importing BAM files and plotting coverage).&lt;/p&gt;&#xA;&lt;p&gt;I was wondering whether someone can point me towards some sample code for creating a basic gene plot. It probably would be good to start by creating a dataframe containing all the features of my gene - is there a handy package that can do that?&lt;/p&gt;&#xA;" OwnerUserId="9025" LastEditorUserId="9025" LastEditDate="2020-10-05T13:59:25.167" LastActivityDate="2020-10-05T16:16:48.637" Title="Easy way to create gene plot in gviz?" Tags="|gene|visualization|gviz|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14482" PostTypeId="2" ParentId="14481" CreationDate="2020-10-05T16:16:29.517" Score="1" Body="&lt;p&gt;If Python is an option, you could check &lt;a href=&quot;http://biopython.org/DIST/docs/tutorial/Tutorial.html#sec336&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioPython and GenomDiagram&lt;/a&gt;. This should get the job done.&lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2020-10-05T16:16:29.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14483" PostTypeId="2" ParentId="14481" CreationDate="2020-10-05T16:16:48.637" Score="0" Body="&lt;p&gt;I assume you are trying to make something like a &amp;quot;gene arrow map&amp;quot; and &lt;code&gt;[gggenes][1]&lt;/code&gt; can help you (with some little tweaks) in case you are familiar with &lt;code&gt;ggplot2&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-10-05T16:16:48.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14484" PostTypeId="1" AcceptedAnswerId="14485" CreationDate="2020-10-05T16:47:43.277" Score="0" ViewCount="41" Body="&lt;p&gt;I keep getting the error while using E-utilities on Slurm HPC, could you suggest how I can resolve it?&lt;/p&gt;&#xA;&lt;p&gt;SCRIPT:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;#SBATCH --job-name=tax_job_test      # Job name&#xA;#SBATCH --mail-type=END,FAIL         # Mail events (NONE, BEGIN, END, FAIL, ALL)&#xA;#SBATCH --mail-user=na@gmail.com    # Where to send mail&#xA;#SBATCH --ntasks=8                  # Number of MPI ranks&#xA;#SBATCH --nodes=1                    # Number of nodes&#xA;#SBATCH --time=23:05:00              # Time limit hrs:min:sec&#xA;#SBATCH --partition=k2-medpri&#xA;&#xA;module load e-utilities/03.02.19&#xA;module load apps/perl/5.20.2/gcc-4.8.5&#xA;&#xA;&#xA;for i in `cat taxonomy.id.txt` ; do efetch -db nuccore -format gbc -id ${i} |xtract  -pattern INSDSeq -element INSDSeq_accession-version INSDSeq_taxonomy INSDSeq_organism &amp;gt;&amp;gt;taxo.2020.txt; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;input &lt;code&gt;taxonomy.id.txt&lt;/code&gt; is like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;AKN43077.1&#xA;AKN41814.1&#xA;PNE10086.1&#xA;AKN42672.1&#xA;AKN41654.2&#xA;AKN43077.1&#xA;PNE10086.1&#xA;AKN43189.1&#xA;TMK08501.1&#xA;TMK08501.1&#xA;AKN42976.1&#xA;AKN43189.1&#xA;AIZ47938.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Error:&lt;/p&gt;&#xA;&lt;p&gt;The script keep giving below error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;501 Protocol scheme 'https' is not supported (LWP::Protocol::https not installed)&#xA;No do_post output returned from 'https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=nuccore&amp;amp;id=AKN43077.1&#xA;&amp;amp;rettype=gbc&amp;amp;retmode=xml&amp;amp;edirect_os=linux&amp;amp;edirect=11.7&amp;amp;tool=edirect&amp;amp;email=3052771@node126.pri.kelvin2.alces.network'&#xA;&#xA;Result of do_post http request is&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;$VAR1 = bless( {&#xA;                 '_rc' =&amp;gt; 501,&#xA;                 '_request' =&amp;gt; bless( {&#xA;                                        '_method' =&amp;gt; 'POST',&#xA;                                        '_uri' =&amp;gt; bless( do{\(my $&lt;/span&gt;o = 'https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi')}, 'URI::https' ),&#xA;                                        '_content' =&amp;gt; 'db=nuccore&amp;amp;id=AKN43077.1  &#xA;&amp;amp;rettype=gbc&amp;amp;retmode=xml&amp;amp;edirect_os=linux&amp;amp;edirect=11.7&amp;amp;tool=edirect&amp;amp;email=3052771@node126.pri.kelvin2.alces.network',&#xA;                                        '_headers' =&amp;gt; bless( {&#xA;                                                               'content-type' =&amp;gt; 'application/x-www-form-urlencoded',&#xA;                                                               'user-agent' =&amp;gt; 'libwww-perl/6.39'&#xA;                                                             }, 'HTTP::Headers' )&#xA;                                      }, 'HTTP::Request' ),&#xA;                 '_content' =&amp;gt; 'LWP will support https URLs if the LWP::Protocol::https module&#xA;is installed.&#xA;',&#xA;                 '_headers' =&amp;gt; bless( {&#xA;                                        'client-warning' =&amp;gt; 'Internal response',&#xA;                                        '::std_case' =&amp;gt; {&#xA;                                                          'client-warning' =&amp;gt; 'Client-Warning',&#xA;                                                          'client-date' =&amp;gt; 'Client-Date'&#xA;                                                        },&#xA;                                        'client-date' =&amp;gt; 'Mon, 05 Oct 2020 14:27:37 GMT',&#xA;                                        'content-type' =&amp;gt; 'text/plain'&#xA;                                      }, 'HTTP::Headers' ),&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Many thanks&lt;/p&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="650" LastEditDate="2020-10-05T17:55:56.127" LastActivityDate="2020-10-05T17:55:56.127" Title="Error while using E-utilities on slurm HPC" Tags="|phylogenetics|hpc|e-utilities|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14485" PostTypeId="2" ParentId="14484" CreationDate="2020-10-05T16:59:02.777" Score="2" Body="&lt;p&gt;&lt;code&gt;501 Protocol scheme 'https' is not supported (LWP::Protocol::https not installed)&lt;/code&gt; indicates that a specific perl module is missing.&lt;/p&gt;&#xA;&lt;p&gt;If indeed your edirect installation is from March 2019 (as I interpret from &lt;code&gt;e-utilities/03.02.19&lt;/code&gt;), you should update your edirect installation to the latest version. If you update, I expect the necessary perl module will be installed and the error will go away.&lt;/p&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2020-10-05T16:59:02.777" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14486" PostTypeId="2" ParentId="14086" CreationDate="2020-10-05T17:24:05.533" Score="0" Body="&lt;p&gt;I would use permutation:&lt;/p&gt;&#xA;&lt;p&gt;i) Get the difference that you are interested in, i.e. number/proportion of monocytes in one group vs the other.&lt;/p&gt;&#xA;&lt;p&gt;ii) Get the relevant columns (condition, cell label, ...) from the &lt;code&gt;@meta.data&lt;/code&gt; slot of your seurat objects and &lt;code&gt;rbind()&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;iii) Shuffle the cell labels randomly and split the data frame into two (per condition).&lt;/p&gt;&#xA;&lt;p&gt;iv) Compare the difference between the cell counts/proportions of interest from the two permuted data frames and save this value.&lt;/p&gt;&#xA;&lt;p&gt;iv) Repeat iii and iv for 100s of times and calculate how many times have you observed a difference at least as extreme as your real observation (i). Once divided by the number of trials, this would give you a p-value stating how likely it is that you get the observed value.&lt;/p&gt;&#xA;&lt;p&gt;I have to add that batch effects are severe in scRNA-seq and hence a linear model where you might be able to account for batches could be more helpful.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-10-05T17:24:05.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14487" PostTypeId="1" AcceptedAnswerId="14489" CreationDate="2020-10-06T03:38:26.817" Score="1" ViewCount="225" Body="&lt;p&gt;I am a Biochemist that is unfamiliar with bioinformatic tools and new to academia as a whole.&lt;/p&gt;&#xA;&lt;p&gt;I am currently using ILLUMINA PE data, which I trimmed (Trimmomatic), corrected (Rcorrector) and assembled (SPAdes). I am now interested in using the genetic sequences from my contigs to identify the original organism (I know it is a bacterium).&lt;/p&gt;&#xA;&lt;p&gt;I have tried using BLASTn (with default parameters) on one of my contigs. I let the query run for 30 minutes before stopping, nothing came of it.&lt;/p&gt;&#xA;&lt;p&gt;What would be the best way to go about doing this?&lt;/p&gt;&#xA;" OwnerUserId="9637" LastEditorUserId="9637" LastEditDate="2020-11-30T17:39:34.193" LastActivityDate="2020-11-30T17:39:34.193" Title="How to identify unknow bacterium species from whole genome genetic sequence" Tags="|fasta|blast|genomics|blastn|genetics|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14488" PostTypeId="1" CreationDate="2020-10-06T05:57:00.077" Score="0" ViewCount="1341" Body="&lt;p&gt;I am analyzing a rather small scRNAseq dataset (39797 features across 3522 samples) and I have observed that dots in the violin plot are plotted in one line next to each other rather than in a cloud of dots (see plot below):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xMXtu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xMXtu.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am just wonndering what the reason might be for this?&lt;/p&gt;&#xA;&lt;p&gt;I have followed the Integration and Label Transfer tutorial using SCTransform (&lt;a href=&quot;https://satijalab.org/seurat/v3.1/integration.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/v3.1/integration.html&lt;/a&gt;). After the integration process I switched back to the SCT assay, which should contain the normalized counts. For plotting I used the following command:&#xA;VlnPlot(..., assay = &amp;quot;SCT&amp;quot;, slot = &amp;quot;data&amp;quot;), again specifying to use the normalized counts.&lt;/p&gt;&#xA;&lt;p&gt;Can someone explain me why this is the case?&lt;/p&gt;&#xA;&lt;p&gt;Thank you already in advance!&lt;/p&gt;&#xA;" OwnerUserId="9638" LastActivityDate="2021-03-05T10:00:30.030" Title="Seurat Violin Plot: Why do dots align in one row?" Tags="|scrnaseq|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14489" PostTypeId="2" ParentId="14487" CreationDate="2020-10-06T08:19:35.333" Score="1" Body="&lt;p&gt;You can use &lt;a href=&quot;https://cge.cbs.dtu.dk/services/KmerFinder/&quot; rel=&quot;nofollow noreferrer&quot;&gt;KmerFinder&lt;/a&gt; and Kraken2 for taxonomy assignment&lt;/p&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2020-10-06T08:19:35.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14490" PostTypeId="2" ParentId="14488" CreationDate="2020-10-06T08:23:09.200" Score="1" Body="&lt;p&gt;That is because SCTransform (mostly) outputs discrete values, you can check the @slot of your Seurat object.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-10-06T08:23:09.200" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14491" PostTypeId="1" CreationDate="2020-10-06T10:21:51.807" Score="0" ViewCount="818" Body="&lt;p&gt;I am new to Python. I am trying to parse a fasta file containing 10,000 sequences to look for motifs (microsatellites in particular). I tried using Seq Utils to parse my sequences for a particular motif, that is, (TG)20. I suspect because I used the string function (str), it treated the entire fasta file as a one single fasta sequence. I then tried using BruteForce on the motif (TG)17 and found that an error appeared which said that &amp;quot;...the object of type 'FastaIterator' has no len()...&amp;quot;. The script however worked when only a single sequence was in the fasta file.&lt;/p&gt;&#xA;&lt;p&gt;In essence, I need some help figuring out how to:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Prepare an instances object(is this correct?) with microsatellite&#xA;motifs (for starters, I intend to parse the sequences for dinucleotide repeats (TG)10,(AC)10,(AT)10&#xA;and (AG)10;&lt;/li&gt;&#xA;&lt;li&gt;Parse my 10,000 long sequence file using this object(?); and&lt;/li&gt;&#xA;&lt;li&gt;Print an output specifying which of my sequences contain microsatellite motifs.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Appreciate any help I can get on this.&lt;/p&gt;&#xA;&lt;p&gt;#Seq Utils to find Motifs&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; pattern = Seq(&amp;quot;TGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTG&amp;quot;) &#xA;&amp;gt;&amp;gt;&amp;gt; sequence = Seq(a)&#xA;&amp;gt;&amp;gt;&amp;gt; results = SeqUtils.nt_search(str(a),pattern)&#xA;&amp;gt;&amp;gt;&amp;gt; print(results)&#xA;['TGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTG', 38, 40, 42, 44, 46, 48, 50, 52, 54,&#xA; 56, 58, 68682, 68684, 68686, 370986, 370988, 370990, 370992, 370994, 370996, 43&#xA;8620, 438622, 438624, 438626, 438628, 438630, 438632, 438634, 438636, 438638, 43&#xA;8640, 556703, 556705, 556707, 784973, 784.... # it's too long so I am just pasting the first few lines&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;#BrutForce to find Motifs in a fasta file containing 10,000 sequences&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; def BruteForce(s, t):&#xA;...     occurrences = []&#xA;...     for i in range(len(s)-len(t)+1):&#xA;...         match = True&#xA;...         for j in range(len(t)):&#xA;...             if s[i+j] != t[j]:&#xA;...                 match = False&#xA;...                 break&#xA;...         if match:&#xA;...             occurrences.append(i)&#xA;...             &#xA;...     print(occurrences)&#xA;...     &#xA;... &#xA;&#xA;&amp;gt;&amp;gt;&amp;gt; t = &amp;quot;TGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTG&amp;quot;&#xA;&amp;gt;&amp;gt;&amp;gt; len(t)&#xA;34&#xA;&amp;gt;&amp;gt;&amp;gt; s = SeqIO.parse(&amp;quot;group23.fas&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;&#xA;&amp;gt;&amp;gt;&amp;gt; BruteForce(s,t)&#xA;Traceback (most recent call last):&#xA;  File &amp;quot;&amp;lt;console&amp;gt;&amp;quot;, line 1, in &amp;lt;module&amp;gt;&#xA;  File &amp;quot;&amp;lt;console&amp;gt;&amp;quot;, line 3, in BruteForce&#xA;TypeError: object of type 'FastaIterator' has no len()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;#BrutForce to find Motifs in a fasta file containing a single sequence&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; s = SeqIO.read(&amp;quot;group23v.fas&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;&amp;gt;&amp;gt;&amp;gt; BruteForce(s,t)&#xA;[0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20, 22, 24, 26, 28, 30, 32, 34, 36, 38, 40, 42, 44, 46, 48, 50, 52, 54, 56, 58, 60, 62, 64, 66, 68, 70, 79, 98, 117, 175, 193, 198, 217, 221, 277, 279, 282, 289, 294, 304, 311, 335, 381, 437, 464, 468, 510, 514, 522, 593, 602, 622, 634, 647, 658, 678, 704, 727, 750, 773, 804, 889, 895, 908, 911, 922, 937, 969, 975, 978, 1002]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9641" LastActivityDate="2021-03-18T03:01:25.177" Title="Finding motifs: fasta file with 10,000 sequences" Tags="|python|sequence-analysis|motifs|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14492" PostTypeId="1" AcceptedAnswerId="14511" CreationDate="2020-10-06T11:05:45.807" Score="0" ViewCount="799" Body="&lt;p&gt;I wondered how would I be able to obtain variant based on chrom and location range.&lt;/p&gt;&#xA;&lt;p&gt;Optimally, this function would be like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import vcf&#xA;vcf_reader = vcf.Reader(filename=some_file_name)&#xA;vcf_read = vcf.variants_between(CHROM, START, END)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does anyone how to do so?&lt;/p&gt;&#xA;" OwnerUserId="9639" LastActivityDate="2020-10-08T20:31:01.133" Title="How do I obtain a vcf read from a pyvcf.Reader?" Tags="|vcf|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14493" PostTypeId="1" CreationDate="2020-10-06T14:37:44.103" Score="1" ViewCount="59" Body="&lt;p&gt;Is there a way to figure out what percentage of entries in PDB have SS bonds?&#xA;In addition, what other types of bonds in PDB may be worth looking into (not H bonds)?&lt;/p&gt;&#xA;" OwnerUserId="4449" LastActivityDate="2020-10-06T15:17:34.467" Title="How many entries in PDB have SS bonds?" Tags="|proteins|database|protein-structure|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14494" PostTypeId="2" ParentId="14491" CreationDate="2020-10-06T14:49:55.327" Score="1" Body="&lt;p&gt;&lt;code&gt;Bio.SeqIO.parse()&lt;/code&gt; returns a &lt;code&gt;SeqRecord iterator&lt;/code&gt;. To access the sequence of this sequence record, you need to use the &lt;code&gt;.seq&lt;/code&gt; attribute so you should update your &lt;code&gt;s&lt;/code&gt; with &lt;code&gt;s.seq&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-10-06T14:49:55.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14495" PostTypeId="1" CreationDate="2020-10-06T15:10:58.283" Score="1" ViewCount="39" Body="&lt;p&gt;I want to write a script that filters a pileup file from a site lists file.&lt;br /&gt;&#xA;As an input I get a reference genome, pileup and site lists files.&lt;/p&gt;&#xA;&lt;p&gt;Example of an output for this script:&lt;br /&gt;&#xA;Pileup File :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;        NC_000001.10    11456   A   0   *   *&#xA;        NC_000001.10    11457   C   0   *   *&#xA;        NC_000001.10    11458   T   0   *   *&#xA;        NC_000002.11    11460   G   1   ,   E&#xA;        NC_000002.11    13914   G   1   ,   E&#xA;        NC_000003.11    10503990    C   1   ,   E&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Reference File (After using grep by '&amp;gt;'):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;        &amp;gt;NC_000001.10 Homo sapiens chromosome 1, GRCh37.p13 Primary Assembly&#xA;        &amp;gt;NT_113878.1 Homo sapiens chromosome 1 unlocalized genomic contig, GRCh37.p13 Primary Assembly&#xA;        &amp;gt;NT_167207.1 Homo sapiens chromosome 1 unlocalized genomic contig, GRCh37.p13 Primary Assembly&#xA;        &amp;gt;NC_000002.11 Homo sapiens chromosome 2, GRCh37.p13 Primary Assembly&#xA;        &amp;gt;NC_000003.11 Homo sapiens chromosome 3, GRCh37.p13 Primary Assembly&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Lists File:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;        chr1    11456   C1orf186    -   intronic    intronic    no  no  N   N   N&#xA;        chr1    11457   C1orf186    -   intronic    intronic    no  no  N   N   N&#xA;        chr2    13914   intergenic  -   intergenic  intergenic  no  no  N   N   N&#xA;        chr7    30504355    NOD1    -   intronic    intronic    no  no  N   N   N&#xA;        chr3    10503990    SSX2IP  -   Syn Gln-&amp;gt;Gln    no  no  N   N   N&#xA;        chr1    13131320    MEF2A   +   intronic    intronic    no  no  N   N   N&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output File:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;        NC_000001.10    11456   A   0   *   *&#xA;        NC_000001.10    11457   C   0   *   *&#xA;        NC_000002.11    13914   G   1   ,   E&#xA;        NC_000003.11    10503990    C   1   ,   E&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So for every refseq in the pileup file I would convert it to the matching chromose in which is found in the reference file, and then overlap it with my lists file so I would get the lines with the same chromose and position.&lt;/p&gt;&#xA;&lt;p&gt;The problem is that while there are many refseq coordinates that belong to chr1 for example, alignment to each of them will give a position, with respect to the specific coordinate. But on the list there is a full, single, chr1.&lt;/p&gt;&#xA;&lt;p&gt;I need to be able to compare between:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a pileup file that was created based on a refseq reference genome&lt;/li&gt;&#xA;&lt;li&gt;a file with editing sites that was downloaded from RADAR (we may give the file)&#xA;Maybe there is a more appropriate reference genome file or that there is a way to convert the refseq positions according to the list&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;EDIT: If I were to convert my pileup file into a bed file (by repeating the position twice) could I use somehow &lt;a href=&quot;https://daler.github.io/pybedtools/autodocs/pybedtools.bedtool.BedTool.intersect.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools&lt;/a&gt; to solve my problem?&lt;/p&gt;&#xA;" OwnerUserId="9460" LastEditorUserId="9460" LastEditDate="2020-10-26T12:49:11.073" LastActivityDate="2020-10-26T12:49:11.073" Title="Filtering pileup from site lists" Tags="|sequence-alignment|reference-genome|filtering|refseq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14496" PostTypeId="2" ParentId="14493" CreationDate="2020-10-06T15:17:34.467" Score="3" Body="&lt;p&gt;The easiest way is to check annotations in mmCIF or PDB files.&lt;/p&gt;&#xA;&lt;p&gt;Looking at my page with &lt;a href=&quot;https://project-gemmi.github.io/pdb-stats/tags.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;statistics of mmCIF tags&lt;/a&gt;, last updated in March:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;_struct_conn_type.id        59137×metalc 42706×covale 36195×disulf 10725×hydrog&#xA;_struct_conn.conn_type_id   5350402×hydrog 1733277×metalc 559092×covale 226472×disulf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which means 226472 disulfide bonds in 36195 PDB entries at the time when the PDB had 161470 entries.&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2020-10-06T15:17:34.467" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14499" PostTypeId="1" AcceptedAnswerId="14503" CreationDate="2020-10-07T08:12:00.560" Score="1" ViewCount="185" Body="&lt;p&gt;I have count data from a RNAseq experiment (2 samples are from normal cells and 3 samples are cells with a disease), and the data is already standardized by trimmed mean of M values (TMM).&#xA;I want to do some plots: biplot of Principal Component Analysis (PCA) and a cluster dendrogram to see if the samples normal/disease are well separated (there is a clear difference between them).&#xA;Since the data is already standardized (TMM) should I scale and center the data prior to perform PCA and cluster dendrogram??&lt;/p&gt;&#xA;&lt;p&gt;thank you!&lt;/p&gt;&#xA;" OwnerUserId="9010" LastEditorUserId="5077" LastEditDate="2020-10-07T12:57:16.043" LastActivityDate="2020-10-07T13:43:36.320" Title="data visualization RNAseq : scaling data for PCA and cluster dendogram" Tags="|rna-seq|clustering|pca|tmm|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14500" PostTypeId="1" CreationDate="2020-10-07T08:29:19.440" Score="0" ViewCount="210" Body="&lt;p&gt;I have a set of different phenotypes which I want to use for a GWAS analysis (general linear model). I have a couple of questions and uncertainty about the phenotype data input.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I have control and samples and usually we normalize all sample to the control for other calculations.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&lt;p&gt;Mean control - 60%&lt;/p&gt;&#xA;&lt;p&gt;Mean sample1 - 20 %&lt;/p&gt;&#xA;&lt;p&gt;Mean sample1 normalized to control - 20*100/60 = 33,3 %&lt;/p&gt;&#xA;&lt;p&gt;Is this normalization step correct for a GWAS analysis?&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Should the phenotype data scaled into 0...1 range? I have different scales for different phenotypes but the results should be compared and compatible.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Some of my phenotypes are not normality distributed. What is the best way to handle such data? I have read that the data should be normally distributed for GWAS analysis.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Here are the histograms of two phenotypes:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yhCcS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yhCcS.png&quot; alt=&quot;phenotype 1&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/S5jNS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/S5jNS.png&quot; alt=&quot;phenotype 2&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9278" LastActivityDate="2020-10-07T08:29:19.440" Title="GWAS phenotype data format and preprocessing" Tags="|python|normalization|gwas|plink|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14501" PostTypeId="1" CreationDate="2020-10-07T10:17:20.483" Score="3" ViewCount="294" Body="&lt;p&gt;I am trying to do Wilcox test to detect the significant difference in alpha diversity but it is showing an error? physeqN2 is a phyloseq object and Season is a metadata column.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;results = estimate_richness(physeqN2, measures = 'Chao1')&#xA;&#xA;d = sample_data(physeqN2)&#xA;&#xA;# calculate Wilcox Test&#xA;Wet.root = results[d[,'Season'] == 'Wet',]&#xA;Dry.root = results[d[,'Season'] == 'Dry',]&#xA;pmv &amp;lt;-pairwise.wilcox.test(Wet.root, Dry.root)&#xA;&#xA;capture.output(pmv, file = &amp;quot;chao1.16s.wet_vs_Dry.root.txt&amp;quot;)&#xA;&#xA;Error;&#xA;Error in wilcox.test.default(xi, xj, paired = paired, ...) :&#xA;  'x' must be numeric&#xA;Calls: pairwise.wilcox.test ... compare.levels -&amp;gt; wilcox.test -&amp;gt; wilcox.test.def                                                                                        ault&#xA;Execution halted&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I hvae tried to do it in another way but here also getting error?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;alpha.diversity &amp;lt;- estimate_richness(physeqN2, measures=c(&amp;quot;Chao1&amp;quot;))&#xA;&#xA;&#xA;## Wilcox on observed richness&#xA;data &amp;lt;- cbind(sample_data(physeqN2), alpha.diversity)&#xA;pmv &amp;lt;-pairwise.wilcox.test(Chao1 ~ Season, data)&#xA;&#xA;capture.output(pmv, file = &amp;quot;chao1.16s.wet_vs_Dry.root.txt&amp;quot;)&#xA;&#xA;Error:&#xA;&#xA;Error in if (length(ans) == 0L || as.character(ans[[1L]])[1L] == &amp;quot;~&amp;quot;) { :&#xA;  missing value where TRUE/FALSE needed&#xA;Calls: pairwise.wilcox.test ... sapply -&amp;gt; lapply -&amp;gt; FUN -&amp;gt; compare.levels -&amp;gt; [ -&amp;gt; [.formula&#xA;Execution halted&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Many thanks&lt;/p&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="1008" LastEditDate="2020-10-07T10:44:44.230" LastActivityDate="2024-02-19T23:03:30.463" Title="alpha diversity wilcox.test" Tags="|r|bioconductor|statistics|microbiology|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14502" PostTypeId="2" ParentId="14501" CreationDate="2020-10-07T11:48:30.267" Score="0" Body="&lt;p&gt;I think this is a bug in the input data.&lt;/p&gt;&#xA;&lt;p&gt;At a wild guess what I think is happening is the input data is not regarded as paired, hence the error &lt;code&gt;paired = paired&lt;/code&gt;. The other weird error is &lt;code&gt;x must be numeric&lt;/code&gt;. Firstly,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;how are you scoring missing data, its not via &lt;code&gt;-&lt;/code&gt; by any chance? Missing data must be via &lt;code&gt;nan&lt;/code&gt; type syntax.&lt;/li&gt;&#xA;&lt;li&gt;are you sure there are no heading bugs? Like a header is getting dragged into the numeric data?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Wilcoxan is a non-parametric T test, so the wet root and dry root must match exactly the same sample. Thus it is not a general wet root versus generic dry root, but wet and dry from the same specimen. If this is the data you have the program doesn't think it is matching.&lt;/p&gt;&#xA;&lt;p&gt;If it is truely paired, again I think the source of error is you have some missing value which is leaving the pairings unequal lengths or else a header bug.&lt;/p&gt;&#xA;&lt;p&gt;If the data genuinely isn't paired, then I would look at the Mann-Whitney U test, it would be easier in this scenario.&lt;/p&gt;&#xA;&lt;p&gt;I don't use R syntax (just pandas dataframes) so can't say whether the syntax in the first example is valid, but it is pretty clear regarding your input. In Python you'd generally just give it the entire dataframe and tell it to go figure.&lt;/p&gt;&#xA;&lt;p&gt;The reality is that loads of programs do Wilcoxan, you don't need to use R if its a one-off test. Obviously if this a part of a pipeline (automation) thats a bit different.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-10-07T16:35:37.420" LastActivityDate="2020-10-07T16:35:37.420" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14503" PostTypeId="2" ParentId="14499" CreationDate="2020-10-07T13:43:36.320" Score="1" Body="&lt;p&gt;Scaling (or centering) makes the genes comparable: Putting the expression levels of genes in the same scale (i.e. between 0 and 1) sustains that all of your genes contribute equally to the PCA or distance calculations. On the other hand, without this step, such calculations would be dominated by the highly/lowly expressed genes. I get &amp;quot;better&amp;quot; results with scaling.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-10-07T13:43:36.320" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14505" PostTypeId="1" CreationDate="2020-10-07T22:42:49.617" Score="1" ViewCount="29" Body="&lt;p&gt;I am storing a sequence aligment file on a webserver and now would like to visualize the results. I used the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/projects/msaviewer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Multiple Sequence Alignment Viewer &lt;/a&gt;. The issue is that there are several manual steps involved, that could be streamlined.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;open the file dialog&lt;/li&gt;&#xA;&lt;li&gt;search the add file or type in the urls&lt;/li&gt;&#xA;&lt;li&gt;click on open&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;etc.&lt;/p&gt;&#xA;&lt;p&gt;I wonder if there is a way to call the viewer (or any other alignment viewer) with a single HTTP request, that contains the URL of the alignment file?&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3848" LastEditDate="2020-10-08T10:05:54.897" LastActivityDate="2020-10-08T10:05:54.897" Title="Call NCBI Multiple Sequence Alignment Viewer with target URL?" Tags="|multiple-sequence-alignment|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14506" PostTypeId="1" AcceptedAnswerId="14507" CreationDate="2020-10-08T11:36:12.543" Score="0" ViewCount="103" Body="&lt;p&gt;I am stuck on how to do correlation for two independent data sets with common row and column names.&#xA;A and B are datasets that contain as many rows as genes and as many columns as samples.&#xA;The rows in A and B represent a common set of genes but measured in two different tissues.&#xA;The columns represent measurements in the same 5 samples in both A and B.&#xA;I want to do a correlation between the set of genes in A and B. This is to see if the same genes in both tissues are correlated or not.&#xA;Since the matrix would be big in my actual data, I only want to retain a correlation coefficient higher than 0.5.&lt;/p&gt;&#xA;&lt;p&gt;Here I simulate the data set.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;set.seed(1)&#xA;A &amp;lt;- data.frame(rnorm(100), &#xA;                rnorm(100),&#xA;                rnorm(100), &#xA;                rnorm(100),&#xA;                rnorm(100))&#xA;row.names(A) &amp;lt;- paste0(&amp;quot;G_&amp;quot;, 1:100)&#xA;colnames(A) &amp;lt;- paste0(&amp;quot;M_&amp;quot;, 1:5)&#xA;&#xA;set.seed(42)  &#xA;B &amp;lt;- data.frame(rnorm(100), &#xA;                rnorm(100),&#xA;                rnorm(100), &#xA;                rnorm(100),&#xA;                rnorm(100))&#xA;row.names(B) &amp;lt;- paste0(&amp;quot;G_&amp;quot;, 1:100)&#xA;colnames(B) &amp;lt;- paste0(&amp;quot;I_&amp;quot;, 1:5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="9160" LastEditorUserId="9160" LastEditDate="2020-10-08T12:37:40.913" LastActivityDate="2020-10-08T12:37:40.913" Title="gene-gene correlation from two different Tissues" Tags="|r|gene|gene-expression|correlation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14507" PostTypeId="2" ParentId="14506" CreationDate="2020-10-08T12:01:43.547" Score="1" Body="&lt;p&gt;You can use &lt;code&gt;mapply()&lt;/code&gt;. &lt;code&gt;ta&lt;/code&gt; and &lt;code&gt;tb&lt;/code&gt; being transposed data frames of your &lt;code&gt;A&lt;/code&gt; and &lt;code&gt;B&lt;/code&gt; data frames respectively:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; mapply(cor, ta, tb)[mapply(cor, ta, tb) &amp;gt; 0.5]&#xA;&#xA;      G_3       G_5       G_9      G_10      G_11      G_15      G_20      G_23      G_25      G_26      G_33      G_40      G_43      G_48      G_57      G_60 &#xA;0.5346591 0.8066507 0.8379777 0.6752681 0.7221359 0.5285787 0.7333045 0.5627962 0.6533379 0.7256878 0.5996492 0.6486557 0.5108215 0.7386332 0.6596823 0.6919915 &#xA;     G_63      G_72      G_76      G_80      G_81      G_90      G_97      G_98      G_99 &#xA;0.5589583 0.8391917 0.7608801 0.8003665 0.6364557 0.5030968 0.7298439 0.5693024 0.5709411 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-10-08T12:01:43.547" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14508" PostTypeId="1" CreationDate="2020-10-08T14:45:52.773" Score="1" ViewCount="128" Body="&lt;p&gt;I am performing amino acid analysis but require assistance in the underlying amino acid substitution model, i.e. the predetermined rate/frequency of amino acid mutations which are used for example in constructing phylogenetics trees using maximum likelihood or Bayesian methods.&lt;/p&gt;&#xA;&lt;p&gt;There are numerous matrices of probabilities for every combination of amino acids (something like &lt;a href=&quot;https://en.wikipedia.org/wiki/Substitution_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;BLOSUM&lt;/em&gt; or &lt;em&gt;PAM&lt;/em&gt;&lt;/a&gt; matrices and I am wondering within the large number of models which the most appropriate. I am aware there is no 'universal' matrix.&lt;/p&gt;&#xA;&lt;p&gt;I want to use the matrix to check the probability of amino acid substitution in sequence on a pairwise basis.&lt;/p&gt;&#xA;&lt;p&gt;For example, I'm not sure whether the BLOSUM or PAM matrix will be useful to me.&lt;/p&gt;&#xA;&lt;p&gt;[&lt;strong&gt;EXTRA&lt;/strong&gt;] If all diagonal values in such a matrix would be equal to &lt;code&gt;1&lt;/code&gt; that would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="9651" LastEditorUserId="3848" LastEditDate="2022-12-20T17:20:49.970" LastActivityDate="2022-12-20T17:26:40.757" Title="amino acids substitution probability" Tags="|phylogenetics|proteins|phylogeny|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14509" PostTypeId="1" AcceptedAnswerId="14537" CreationDate="2020-10-08T18:13:36.077" Score="0" ViewCount="63" Body="&lt;p&gt;I have 5 samples (from a RNA seq experiment) from patients but I have no control, so since we can't look for the differentially expressed genes we want to look for the genes that have similar expression in all the sample.&#xA;We want to do that to see if some genes that can be involved are among those expressed in all the samples.&#xA;I have never done that, I do not know how to do it and I cant find any info about it, so any ideas are welcome.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="9010" LastEditorUserId="9010" LastEditDate="2020-10-12T14:56:10.137" LastActivityDate="2020-10-12T16:30:56.613" Title="Looking for genes that are expressed the same in all the samples" Tags="|gene-expression|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="14510" PostTypeId="2" ParentId="14491" CreationDate="2020-10-08T20:03:10.050" Score="1" Body="&lt;p&gt;To expand on haci's answer (and since you mentioned being new to Python) you can loop over the iterator object and use the &lt;code&gt;seq&lt;/code&gt; attribute like this:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;pattern = Seq(&amp;quot;TGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTG&amp;quot;)&#xA;for record in SeqIO.parse(&amp;quot;group23.fas&amp;quot;, &amp;quot;fasta&amp;quot;):&#xA;    results = SeqUtils.nt_search(str(record.seq), pattern)&#xA;    print(results)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(More info on iterators &lt;a href=&quot;https://wiki.python.org/moin/Iterator&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.)  That will work the same whatever the number of sequences in group23.fas.&lt;/p&gt;&#xA;&lt;p&gt;I'd be cautious about looking for microsatellites with fixed strings.  If the sequences you're looking for are short and have imperfections among the repeats, you might miss them, depending on how many perfect repeats your search pattern has.  For example with these stub example sequences:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;seq1&#xA;TGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTG&#xA;&amp;gt;seq2&#xA;ACGATGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTG&#xA;&amp;gt;seq3&#xA;TGTGTGTGTGTGTGTGTGTGTCTGTGTGTGTGTGTGTGTG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The nt_search output gives you:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;['TGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTG', 0]&#xA;['TGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTG', 4]&#xA;['TGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTGTG']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That is, one match at position 0 of the first sequence, one at position 4 of the second, and no matches for the third (there's a C I put in the middle).  This might not be relevant for you if you're looking microsatellites much longer than your search patterns, but for shorter ones it might be a problem.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2020-10-08T20:03:10.050" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14511" PostTypeId="2" ParentId="14492" CreationDate="2020-10-08T20:31:01.133" Score="1" Body="&lt;p&gt;I think &lt;code&gt;vcf_reader.fetch&lt;/code&gt; is what you want.  You have to index the file first though.&lt;/p&gt;&#xA;&lt;p&gt;If starting from regular text VCF you can run:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bgzip example.vcf&#xA;tabix example.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then the Python code is almost the same as yours:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import vcf&#xA;vcf_reader = vcf.Reader(filename=&amp;quot;example.vcf.gz&amp;quot;)&#xA;vcf_read = vcf_reader.fetch(&amp;quot;NC_023663.1&amp;quot;, 7803980, 7804000)&#xA;for record in vcf_read:&#xA;    print(record)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For this example VCF you get:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Record(CHROM=NC_023663.1, POS=7803981, REF=G, ALT=[&amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7803982, REF=T, ALT=[&amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7803983, REF=A, ALT=[&amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7803984, REF=A, ALT=[&amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7803985, REF=G, ALT=[&amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7803986, REF=G, ALT=[&amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7803987, REF=G, ALT=[&amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7803988, REF=C, ALT=[&amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7803989, REF=A, ALT=[&amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7803990, REF=T, ALT=[&amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7803991, REF=A, ALT=[G, &amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7803992, REF=G, ALT=[&amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7803993, REF=A, ALT=[&amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7803994, REF=C, ALT=[&amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7803995, REF=T, ALT=[&amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7803996, REF=A, ALT=[&amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7803997, REF=A, ALT=[&amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7803998, REF=C, ALT=[&amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7803999, REF=A, ALT=[&amp;lt;*&amp;gt;])&#xA;Record(CHROM=NC_023663.1, POS=7804000, REF=T, ALT=[&amp;lt;*&amp;gt;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;API reference:&#xA;&lt;a href=&quot;https://pyvcf.readthedocs.io/en/latest/API.html#vcf.Reader.fetch&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pyvcf.readthedocs.io/en/latest/API.html#vcf.Reader.fetch&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note the &amp;quot;zero-based, half-open coordinate system&amp;quot; it describes (which is why 7803980 in the second Python argument becomes POS=7803981 from the VCF but then it stops at POS=7804000).&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2020-10-08T20:31:01.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14512" PostTypeId="2" ParentId="14508" CreationDate="2020-10-08T21:10:11.953" Score="3" Body="&lt;p&gt;The LG model is here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;0.425093 &#xA;0.276818 0.751878 &#xA;0.395144 0.123954 5.076149 &#xA;2.489084 0.534551 0.528768 0.062556 &#xA;0.969894 2.807908 1.695752 0.523386 0.084808 &#xA;1.038545 0.363970 0.541712 5.243870 0.003499 4.128591 &#xA;2.066040 0.390192 1.437645 0.844926 0.569265 0.267959 0.348847 &#xA;0.358858 2.426601 4.509238 0.927114 0.640543 4.813505 0.423881 0.311484 &#xA;0.149830 0.126991 0.191503 0.010690 0.320627 0.072854 0.044265 0.008705 0.108882 &#xA;0.395337 0.301848 0.068427 0.015076 0.594007 0.582457 0.069673 0.044261 0.366317 4.145067 &#xA;0.536518 6.326067 2.145078 0.282959 0.013266 3.234294 1.807177 0.296636 0.697264 0.159069 0.137500 &#xA;1.124035 0.484133 0.371004 0.025548 0.893680 1.672569 0.173735 0.139538 0.442472 4.273607 6.312358 0.656604 &#xA;0.253701 0.052722 0.089525 0.017416 1.105251 0.035855 0.018811 0.089586 0.682139 1.112727 2.592692 0.023918 1.798853 &#xA;1.177651 0.332533 0.161787 0.394456 0.075382 0.624294 0.419409 0.196961 0.508851 0.078281 0.249060 0.390322 0.099849 0.094464 &#xA;4.727182 0.858151 4.008358 1.240275 2.784478 1.223828 0.611973 1.739990 0.990012 0.064105 0.182287 0.748683 0.346960 0.361819 1.338132 &#xA;2.139501 0.578987 2.000679 0.425860 1.143480 1.080136 0.604545 0.129836 0.584262 1.033739 0.302936 1.136863 2.020366 0.165001 0.571468 6.472279 &#xA;0.180717 0.593607 0.045376 0.029890 0.670128 0.236199 0.077852 0.268491 0.597054 0.111660 0.619632 0.049906 0.696175 2.457121 0.095131 0.248862 0.140825 &#xA;0.218959 0.314440 0.612025 0.135107 1.165532 0.257336 0.120037 0.054679 5.306834 0.232523 0.299648 0.131932 0.481306 7.803902 0.089613 0.400547 0.245841 3.151815 &#xA;2.547870 0.170887 0.083688 0.037967 1.959291 0.210332 0.245034 0.076701 0.119013 10.649107 1.702745 0.185202 1.898718 0.654683 0.296501 0.098369 2.188158 0.189510 0.249313 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Where the amino acid frequencies are:&lt;/p&gt;&#xA;&lt;p&gt;0.079066 0.055941 0.041977 0.053052 0.012937 0.040767 0.071586 0.057337 0.022355 0.062157 0.099081 0.064600 0.022951 0.042302 0.044040 0.061197 0.053287 0.012066 0.034155 0.069147&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Amino acid order&lt;/strong&gt; (matrix and frequencies)&lt;/p&gt;&#xA;&lt;p&gt;A R N D C Q E G H I L K M F P S T W Y V&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Reference&lt;/strong&gt;&#xA;Le S.Q., Gascuel O. Molecular Biology and Evolution. 2008 25(7):1307-20.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-10-08T21:17:33.430" LastActivityDate="2020-10-08T21:17:33.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14513" PostTypeId="1" CreationDate="2020-10-09T07:06:16.727" Score="1" ViewCount="38" Body="&lt;p&gt;I am trying to replicate the results for the creator of this repo:  &lt;a href=&quot;https://github.com/aldro61/kover2_paper&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/aldro61/kover2_paper&lt;/a&gt; for genotype to phenotype machine learning interpretable (decision tree) prediction.&lt;/p&gt;&#xA;&lt;p&gt;Specifically in the data section when I try to create the dataset as explained here:&#xA;&lt;a href=&quot;https://github.com/aldro61/kover2_paper/tree/master/data&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/aldro61/kover2_paper/tree/master/data&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I make a directory called genomes_dir in the data directory. Then I do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;python download_genomes.py 'mycobacterium tuberculosis/kanamycin/metadata.tsv' genomes_dir&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;at the end of the download it tells me that it saved file &lt;code&gt;genome_paths.tsv&lt;/code&gt; the data directory, not genomes_dir. So I run command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;kover dataset create from-tsv --genomic-data genome_paths.tsv ---phenotype-description &amp;quot;Kanamycin resistance&amp;quot; --phenotype-metadata 'mycobacterium tuberculosis/kanamycin/metadata.tsv  --output data_example.kover --progress&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and it just hangs with a prompt:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(kover) nobu@gold3forever:~/Desktop/BioInformatics/ResearchAssistant/kover2_paper/data$ kover dataset create from-tsv --genomic-data genome_paths.tsv ---phenotype-description &amp;quot;Kanamycin resistance&amp;quot; --phenotype-metadata 'mycobacterium tuberculosis/kanamycin/metadata.tsv  --output data_example.kover --progress&#xA;&amp;gt; ^C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;it does nothing after hours and hours so I control-C'd to end the process.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have any idea what is happening?&lt;/p&gt;&#xA;&lt;p&gt;I also had to install this to get the datasets from PATRIC:  &lt;a href=&quot;https://github.com/aldro61/patric_tools&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/aldro61/patric_tools&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9650" LastActivityDate="2020-11-08T16:02:45.313" Title="Problems replicating Kover paper results" Tags="|machine-learning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14514" PostTypeId="1" CreationDate="2020-10-09T14:03:11.217" Score="0" ViewCount="30" Body="&lt;p&gt;I am studying about RNA seq, and especially about 3'Quantseq(tagseq, 3primeseq). I wonder if there is a cutoff for the reads length. By this I mean that given that 3'quantseq targets the end of the genes and 3'UTR, then for example if reads are 75bp, then could be the case that only 3'UTR is mapped and not coding region? I am interested in Human Sapiens. Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="9656" LastEditorUserId="9656" LastEditDate="2020-10-09T14:15:46.770" LastActivityDate="2020-10-09T14:51:43.597" Title="smallest length of reads in 3'quantseq?" Tags="|rna-seq|rna|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14515" PostTypeId="2" ParentId="14514" CreationDate="2020-10-09T14:51:43.597" Score="1" Body="&lt;p&gt;Simple answer: Yes, that can very well happen depending on the average fragment size, the read length and the length of the UTR.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-10-09T14:51:43.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14516" PostTypeId="1" AcceptedAnswerId="14928" CreationDate="2020-10-09T15:02:39.973" Score="0" ViewCount="190" Body="&lt;p&gt;According to &lt;a href=&quot;https://www.encodeproject.org/chip-seq/histone/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENCODE recommendations&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For broad-peak histone experiments, each replicate should have 45 million usable fragments&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;According to my understanding, a usable fragment means a uniquely mapped read / pair of reads.&lt;/p&gt;&#xA;&lt;p&gt;However&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;H3K9me3 is an exception as it is enriched in repetitive regions of the genome. ... Tissues and primary cells should have 45 million total mapped reads per replicate.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I fail to grasp Encode's language.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does &amp;quot;total mapped reads&amp;quot; mean uniquely mapped reads?&lt;/li&gt;&#xA;&lt;li&gt;If so, why is 45 million &amp;quot;an exception&amp;quot; ? It is the same as the other broad peaks.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Any ideas where else can I look for an explanation?&lt;/p&gt;&#xA;" OwnerUserId="6317" LastActivityDate="2020-11-25T06:10:17.450" Title="Recommended coverage depth for ChIP Seq (H3K9me3 )" Tags="|chip-seq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14517" PostTypeId="2" ParentId="14513" CreationDate="2020-10-09T15:32:03.987" Score="1" Body="&lt;p&gt;The single quote at &lt;code&gt;'mycobacterium&lt;/code&gt; is not matched so the command prompt is waiting indefinitely for the command to be completed. Maybe you want &lt;code&gt;'mycobacterium tuberculosis/kanamycin/metadata.tsv'&lt;/code&gt; (note closing quote)&lt;/p&gt;&#xA;" OwnerUserId="339" LastActivityDate="2020-10-09T15:32:03.987" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14518" PostTypeId="1" CreationDate="2020-10-09T16:34:12.527" Score="2" ViewCount="784" Body="&lt;p&gt;As I understand it, bowtie2 can easily be used to split reads into one of two groups:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;reads for which both of a pair align well to a reference (using e.g. &lt;code&gt;--al-conc-gz&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;reads for which one or both of a pair do not align a reference (using e.g &lt;code&gt;--un-conc-gz&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;But I really want to split this second group into reads for which neither of a pair align to the reference.&lt;/p&gt;&#xA;&lt;p&gt;bowtie2's report give a nice breakdown, but it's not clear to me how to extract specific members of these groups:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1000000 reads; of these:&#xA;  1000000 (100.00%) were paired; of these:&#xA;    958118 (95.81%) aligned concordantly 0 times&#xA;    1329 (0.13%) aligned concordantly exactly 1 time&#xA;    40553 (4.06%) aligned concordantly &amp;gt;1 times&#xA;    ----&#xA;    958118 pairs aligned concordantly 0 times; of these:&#xA;      179 (0.02%) aligned discordantly 1 time&#xA;    ----&#xA;    957939 pairs aligned 0 times concordantly or discordantly; of these:&#xA;      1915878 mates make up the pairs; of these:&#xA;        711518 (37.14%) aligned 0 times&#xA;        32134 (1.68%) aligned exactly 1 time&#xA;        1172226 (61.18%) aligned &amp;gt;1 times&#xA;64.42% overall alignment rate&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In this case, I really want those that didn't align at all (i.e. the 35.58% not included in the &lt;code&gt;64.42% overall alignment rate&lt;/code&gt; which is the same as the &lt;code&gt;37.14%&lt;/code&gt; of the pairs that did not align concordantly).&lt;/p&gt;&#xA;&lt;p&gt;Since in my case the reference is rRNA and many of those sequences are shorter than my reads, I'm not surprised that so there are so few concordant reads. I'm really interested in reads that aren't concordant and aren't discordant discordant either.&lt;/p&gt;&#xA;&lt;p&gt;So, if I'm sticking to bowtie2, it looks like my only option is to take the resulting SAM/BAM file and filter what I want based on flags and then convert that back into FASTQ files.&lt;/p&gt;&#xA;&lt;p&gt;Is there another option that I'm missing? It seems that the remaining output options relate to unpaired reads or SAM files.&lt;/p&gt;&#xA;&lt;p&gt;I'm looking into other aligners now...&lt;/p&gt;&#xA;&lt;p&gt;My goal is to go from FASTQ files to FASTQ files without extra steps, but I can go through a SAM/BAM file stage if I need to and even build a solution around that if needed.&lt;/p&gt;&#xA;" OwnerUserId="2878" LastEditorUserId="2878" LastEditDate="2020-10-09T18:46:44.077" LastActivityDate="2020-11-08T19:24:01.217" Title="Filtering out reads from a reference (e.g. rRNA) using bowtie2" Tags="|sequence-alignment|bowtie2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14519" PostTypeId="2" ParentId="14518" CreationDate="2020-10-09T17:42:58.597" Score="0" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Bam flags will indicate which reads have both pairs not mapping.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Nothing mapped.  You need to figure out why, and you can look at just about anything out figure that out.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-10-09T17:42:58.597" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14520" PostTypeId="1" AcceptedAnswerId="14606" CreationDate="2020-10-09T17:43:02.307" Score="2" ViewCount="330" Body="&lt;p&gt;I am trying to assemble the run SRR12196449 with &lt;code&gt;SPAdes&lt;/code&gt;. The description of their project is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This project expected to standardize a method for amplification and&#xA;sequencing of FIV genome in a simple way, allowing a broader analysis&#xA;to increase the knowledge on the biology and evolution of the virus&#xA;and virus-host interaction.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is from School of Veterinary Medicine; University of Sao Paulo.&lt;/p&gt;&#xA;&lt;p&gt;I have tried in three ways:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Using a fasta file downloaded from NCBI (using Filtered Download method)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;SPAdes-3.14.0/spades.py    --only-assembler -s sra_data.fa -o raw-fiv1&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Using a fastq file also downloaded from NCBI using Filtered Download method.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;SPAdes-3.14.0/spades.py   --12 sra_data.fastq.gz -o raw-fiv1-b&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Downloading the original file and dumping to a fastq using sratools.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;SPAdes-3.14.0/spades.py    --12 SRR12196449.fastq -o raw-fiv1-c&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the third case I get a very bad assembly, likely because it needs trimming. In the two former cases, I get an assembly with a max contig length of around 4k-5k bps. But when I use &lt;code&gt;quast&lt;/code&gt; to evaluate the assembly I obtain a largest alignment of ~500bps. I use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nucleotide/MN630242.1?report=genbank&amp;amp;log$=nuclalign&amp;amp;blast_rank=1&amp;amp;RID=S1190BA401R&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; as reference.&lt;br /&gt;&#xA;Furthermore, if make a blast search it finds that the 4k-5k contig matches other FIV sequences at ~90% . In between these other sequences, there is at least one (MF370550.1) submitted by the University of Sao Paulo (likely another run from another sample in the same project).&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, if I assemble the run using another software I obtain a contig that is an almost perfect match to my reference sequence (~99%).&lt;/p&gt;&#xA;&lt;p&gt;I have a few questions.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Am I using &lt;code&gt;SPAdes&lt;/code&gt; correctly? Is there a better way to use it?&lt;/li&gt;&#xA;&lt;li&gt;Is it possible that the sequences that blast finds matching with the &lt;code&gt;SPAdes&lt;/code&gt; contigs are indeed sequences that someone got using &lt;code&gt;SPAdes&lt;/code&gt; and that's why these match (not because these are correct)?&lt;/li&gt;&#xA;&lt;li&gt;What other option could explain these circumstances that I could be missing?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;EDIT:&#xA;Some clarifications regarding some comments from @MaximilianPress. I can confirm that I used the same reference genome in both cases. I used that command to get the quast results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;quast-5.0.2/quast.py -r raw-fiv1/sequence.fasta SPAdes-3.14.0/raw-fiv1/contigs.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The other assembler is an overlap-layout-consensus algorithm that I implemented. The methodology is similar: I use exactly the same input fasta file.&lt;/p&gt;&#xA;&lt;p&gt;I have also tried using the &lt;code&gt;SPAdes&lt;/code&gt; --plasmid flag and that doesn't improve much the result. Max contig length is now 3642 bps and largest alignment 737 bps, covering only 26% of the reference.&#xA;BTW, I made also tests with other virus sequences. I am just interested to know if I am using &lt;code&gt;SPAdes&lt;/code&gt; correctly so that I can compare the performance of my algorithm Here there is a link to &lt;a href=&quot;https://www.researchgate.net/publication/344433108_Raw_data_and_results_of_the_tests&quot; rel=&quot;nofollow noreferrer&quot;&gt;additional cases with other genomes&lt;/a&gt;. There you can find all the data and results used (raw files, assembly results and quast results)&lt;/p&gt;&#xA;&lt;p&gt;Another clarification: I have also got perfect assemblies using &lt;code&gt;SPAdes&lt;/code&gt; for other runs corresponding to other viruses... I am following exactly the same procedure in this case.&lt;/p&gt;&#xA;" OwnerUserId="5190" LastEditorUserId="3848" LastEditDate="2022-03-10T14:35:17.290" LastActivityDate="2022-03-10T14:35:17.290" Title="Genome assembly of SRR12196449 with SPAdes" Tags="|assembly|covid-19|genome-sequencing|software-usage|virus|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="14521" PostTypeId="1" CreationDate="2020-10-09T20:16:27.513" Score="1" ViewCount="280" Body="&lt;p&gt;I am using library(biomaRt) in R to annotate SNP rsid. It takes too much time for only 1000 SNPs....... I would like to know how I can run this code faster.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; coords&amp;lt;-data&#xA; coords&lt;span class=&quot;math-container&quot;&gt;$chr_name&amp;lt;-coords$&lt;/span&gt;chromosome&#xA; coords&lt;span class=&quot;math-container&quot;&gt;$start&amp;lt;-coords$&lt;/span&gt;position&#xA; coords&lt;span class=&quot;math-container&quot;&gt;$end&amp;lt;-coords$&lt;/span&gt;position&#xA; coords&amp;lt;-coords%&amp;gt;%dplyr::select(chr_name,start,end)&#xA; mart=useMart(biomart=&amp;quot;ENSEMBL_MART_SNP&amp;quot;, host=&amp;quot;grch37.ensembl.org&amp;quot;,     &#xA; path=&amp;quot;/biomart/martservice&amp;quot;, dataset=&amp;quot;hsapiens_snp&amp;quot;)&#xA; results=data.frame(matrix(ncol=4,nrow=0))&#xA; for (i in seq (1,nrow(coords)))&#xA;{&#xA;ens=getBM(attributes = c('refsnp_id','allele','chrom_start','chrom_strand'),     &#xA;filters = c('chr_name','start','end'), values = as.list(coords[i,]), mart = mart)&#xA;results=rbind(results,ens)}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am using the following code, but still not working......&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;coords&lt;span class=&quot;math-container&quot;&gt;$newposition&amp;lt;-paste0(coords$&lt;/span&gt;chromosome,&amp;quot;:&amp;quot;,coords&lt;span class=&quot;math-container&quot;&gt;$position)&#xA;coords$&lt;/span&gt;chromosomal_region&amp;lt;-paste0(coords&lt;span class=&quot;math-container&quot;&gt;$newposition,&quot;:&quot;,coords$&lt;/span&gt;position)&#xA;coords&amp;lt;-coords$chromosomal_region&#xA;res&amp;lt;-getBM(attributes = c('refsnp_id','allele','chrom_start','chrom_strand'), &#xA;   filters = 'chromosomal_region', &#xA;   values = coords, &#xA;   mart = mart)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8895" LastEditorUserId="73" LastEditDate="2021-06-07T02:14:23.473" LastActivityDate="2023-01-13T07:51:25.483" Title="Too slow issue of BioMart" Tags="|r|biomart|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14523" PostTypeId="2" ParentId="14521" CreationDate="2020-10-09T21:08:18.503" Score="1" Body="&lt;p&gt;Don't run getBM 1000 times.  It can handle whole vectors, and get you all your results at once.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-10-09T21:08:18.503" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14524" PostTypeId="1" CreationDate="2020-10-10T02:35:55.110" Score="0" ViewCount="1623" Body="&lt;pre&gt;&lt;code&gt;library(gplots)&#xA;&#xA;data=structure(c(-0.1152, 0.1548, 0.0037, 0, 0, 0, 0, 0, 0, 0.1369, &#xA;0.1914, 0, 0.0499, 0, 0, 0, 0, 0.1259, 0, 0.1854, -0.4412, 0, &#xA;-0.4425, 0, 0, -0.5141, 0, -0.3463, 0.1529, -0.1176, 0.4153, &#xA;-0.4412, 0.2169, 0.1894, 0.3288, 0.5549, -0.2549, 0.1915, -0.0759, &#xA;0.1255, 0.2269, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.4412, -0.1923, &#xA;0.1502, 0, 0, 0, -0.1978, 0.0828, 0, 0.2214, 0.192, 0, -0.2651, &#xA;0.192, -0.1412, -0.287, 0, -0.2299, 0, 0.1095, 0, 0.0166, 0, &#xA;0.1617, -0.16, 0, 0, 0.1255, 0, 0.2298, 0.153, 0.1627, 0, 0.293, &#xA;0.2476, -0.1565, 0.0899, 0, 0.2217, 0.2274, 0.1764, 0.1538, 0, &#xA;0.1747, 0.1857, -0.2309, 0.0827, -0.0609, 0.0754, -0.2653, 0.1461, &#xA;0.1611, -0.1201, 0, 0, 0.114, 0.237, 0.1738, 0, 0.2269, 0, 0.0068, &#xA;0, 0.1288, -0.1727, -0.1187, -0.1502, -0.1502, 0, -0.4104, -0.1305, &#xA;-0.1126, -0.2642, -0.1264, -0.2403, 0.1522, 0.0211, 0.1175, -0.0549, &#xA;-0.2571, -0.1305, -0.2143, -0.2153, -0.0566, -0.2068, 0.0406, &#xA;-0.087, 0.079, -0.0238), .Dim = c(10L, 14L), .Dimnames = list(&#xA;    c(&amp;quot;AI&amp;quot;, &amp;quot;AUDC&amp;quot;, &amp;quot;AUDT&amp;quot;, &amp;quot;AUT&amp;quot;, &amp;quot;BD&amp;quot;, &amp;quot;CB&amp;quot;, &amp;quot;CD&amp;quot;, &amp;quot;DPW&amp;quot;, &amp;quot;MDD&amp;quot;, &#xA;    &amp;quot;SZ&amp;quot;), c(&amp;quot;BI&amp;quot;, &amp;quot;BB&amp;quot;, &amp;quot;CD&amp;quot;, &amp;quot;CGC&amp;quot;, &amp;quot;CTF&amp;quot;, &amp;quot;HDT&amp;quot;, &amp;quot;MTS&amp;quot;, &amp;quot;NOI&amp;quot;, &#xA;    &amp;quot;OFI&amp;quot;, &amp;quot;SRV&amp;quot;, &amp;quot;SAF&amp;quot;, &amp;quot;FFI&amp;quot;, &amp;quot;SUM&amp;quot;, &amp;quot;WIN&amp;quot;)))&#xA;&#xA;data &amp;lt;- read.csv(&amp;quot;C:/Users/Lenovo/Desktop/Correlation matrix VITD _Disorder_R/Z1.txt&amp;quot;, header=TRUE, row.names=1,sep=&amp;quot;&amp;quot;)&#xA;data = as.matrix(data[![enter image description here][1]][1])&#xA;heatmap.2(data,&#xA;          trace =&amp;quot;none&amp;quot;, &#xA;          density = &amp;quot;none&amp;quot;, &#xA;          col = redblue(100),&#xA;         key=TRUE, &#xA;         keysize=1,&#xA;          cexRow =1, &#xA;          cexCol =1,&#xA;          margins = c(5,5),&#xA;          labCol = colnames(data),&#xA;          scale=&amp;quot;row&amp;quot;,&#xA;          hclust=function(x) hclust(x,method = &amp;quot;complete&amp;quot;))&#xA;&#xA;&#xA;  [1]: https://i.stack.imgur.com/jlwf4.png&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9659" LastEditorUserId="9659" LastEditDate="2020-10-11T18:27:27.030" LastActivityDate="2020-10-11T18:27:27.030" Title="I want to display color key on my heatmap" Tags="|heatmap|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14525" PostTypeId="2" ParentId="14524" CreationDate="2020-10-10T13:13:46.363" Score="1" Body="&lt;p&gt;I'm not sure exactly what was going on with your data, but I read it in as a raw data.frame using &lt;code&gt;as.matrix(read.table(., row.names=1, header=T))&lt;/code&gt;. The issue was that you were reading it in without a header, so the col.names were being read in as part of the matrix. But apart from that, the rest of your command was fine:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(gplots)&#xA;&#xA;data = structure(c(-0.1152, 0.1548, 0.0037, 0, 0, 0, 0, 0, 0, 0.1369, &#xA;0.1914, 0, 0.0499, 0, 0, 0, 0, 0.1259, 0, 0.1854, -0.4412, 0, &#xA;-0.4425, 0, 0, -0.5141, 0, -0.3463, 0.1529, -0.1176, 0.4153, &#xA;-0.4412, 0.2169, 0.1894, 0.3288, 0.5549, -0.2549, 0.1915, -0.0759, &#xA;0.1255, 0.2269, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -0.4412, -0.1923, &#xA;0.1502, 0, 0, 0, -0.1978, 0.0828, 0, 0.2214, 0.192, 0, -0.2651, &#xA;0.192, -0.1412, -0.287, 0, -0.2299, 0, 0.1095, 0, 0.0166, 0, &#xA;0.1617, -0.16, 0, 0, 0.1255, 0, 0.2298, 0.153, 0.1627, 0, 0.293, &#xA;0.2476, -0.1565, 0.0899, 0, 0.2217, 0.2274, 0.1764, 0.1538, 0, &#xA;0.1747, 0.1857, -0.2309, 0.0827, -0.0609, 0.0754, -0.2653, 0.1461, &#xA;0.1611, -0.1201, 0, 0, 0.114, 0.237, 0.1738, 0, 0.2269, 0, 0.0068, &#xA;0, 0.1288, -0.1727, -0.1187, -0.1502, -0.1502, 0, -0.4104, -0.1305, &#xA;-0.1126, -0.2642, -0.1264, -0.2403, 0.1522, 0.0211, 0.1175, -0.0549, &#xA;-0.2571, -0.1305, -0.2143, -0.2153, -0.0566, -0.2068, 0.0406, &#xA;-0.087, 0.079, -0.0238), .Dim = c(10L, 14L), .Dimnames = list(&#xA;    c(&amp;quot;AI&amp;quot;, &amp;quot;AUDC&amp;quot;, &amp;quot;AUDT&amp;quot;, &amp;quot;AUT&amp;quot;, &amp;quot;BD&amp;quot;, &amp;quot;CB&amp;quot;, &amp;quot;CD&amp;quot;, &amp;quot;DPW&amp;quot;, &amp;quot;MDD&amp;quot;, &#xA;    &amp;quot;SZ&amp;quot;), c(&amp;quot;BI&amp;quot;, &amp;quot;BB&amp;quot;, &amp;quot;CD&amp;quot;, &amp;quot;CGC&amp;quot;, &amp;quot;CTF&amp;quot;, &amp;quot;HDT&amp;quot;, &amp;quot;MTS&amp;quot;, &amp;quot;NOI&amp;quot;, &#xA;    &amp;quot;OFI&amp;quot;, &amp;quot;SRV&amp;quot;, &amp;quot;SAF&amp;quot;, &amp;quot;FFI&amp;quot;, &amp;quot;SUM&amp;quot;, &amp;quot;WIN&amp;quot;)))&#xA;&#xA;&#xA;heatmap.2(data,&#xA;          trace =&amp;quot;none&amp;quot;, &#xA;          density = &amp;quot;none&amp;quot;, &#xA;          col = redblue(100),&#xA;         key=TRUE, &#xA;         keysize=1,&#xA;          cexRow =1, &#xA;          cexCol =1,&#xA;          margins = c(5,5),&#xA;          labCol = colnames(data),&#xA;          scale=&amp;quot;row&amp;quot;,&#xA;          hclust=function(x) hclust(x,method = &amp;quot;complete&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/a7voK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/a7voK.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2020-10-10T13:13:46.363" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="14526" PostTypeId="1" CreationDate="2020-10-10T18:13:25.050" Score="1" ViewCount="79" Body="&lt;p&gt;I am now struggling to do random forest analysis, I will be thankful if you could help with code for random forest analysis.&lt;/p&gt;&#xA;&lt;p&gt;I got samples from the root, soil, and leaf from two regions (bau &amp;amp; mau) and these samples belong to two seasons (Wet and Dry).&lt;/p&gt;&#xA;&lt;p&gt;Now I am interested to do random forest analysis at genera or family level to identify the taxa which contribute the differences like in root samples based on region as well as season.&lt;/p&gt;&#xA;&lt;p&gt;Here is my code, but I am getting the error.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    library(randomForest)&#xA;    library(knitr)&#xA;    &#xA;    &#xA;    &#xA;    &#xA;    &#xA;    #### RANDOM FOREST ANALYSIS #####&#xA;     #### Prepare data ####&#xA;     #Load OTU table&#xA;     OTU_table=t(read.table(&amp;quot;asv.table.txt&amp;quot;, row.names=1,sep=&amp;quot;\t&amp;quot;, header=T, blank.lines.skip=F, check.names=F))&#xA;     table(apply(OTU_table,1,sum)) #verify rarefaction&#xA;    &#xA;     #Load metadata&#xA;     Meta=read.table(&amp;quot;metadata.txt&amp;quot;, header=T, row.names=1, stringsAsFactors=F, na.strings=&amp;quot;NA&amp;quot;,check.names=FALSE)&#xA;     Meta$sampleid=rownames(Meta)&#xA;    &#xA;     #Load taxonomy&#xA;     taxo=read.table(&amp;quot;taxonomy.txt&amp;quot;, row.names=1, sep=&amp;quot;\t&amp;quot;, header=F ,stringsAsFactors=F,quote=&amp;quot;&amp;quot;)&#xA;     rownames(taxo)=paste(&amp;quot;a.&amp;quot;,row.names(taxo),sep=&amp;quot;&amp;quot;)&#xA;    &#xA;    &#xA;     #### Run models ####&#xA;     #1. Root only&#xA;     # 1.1. both region&#xA;     # 1.2. bau only&#xA;     # 1.3. mau only&#xA;     #2. Soil only&#xA;    # 2.1. both region&#xA;     # 2.2. bau only&#xA;     # 2.3. mau only&#xA;     #Params RF&#xA;     NTREE=1000 # Number of Trees&#xA;     NbVar=1000 # Number of variables tested at each split&#xA;    &#xA;    &#xA;     &#xA;    &#xA;     &#xA;     #### Root ONLY 1-3 ####&#xA;&#xA; # 1. BOTH Region&#xA;&#xA; #Subset of data&#xA; RootSamples=as.character(Meta[Meta$Compartment==&amp;quot;Root&amp;quot;,&amp;quot;sampleid&amp;quot;])&#xA; Root_OTU_table=OTU_table[RootSamples,]&#xA;&#xA;&#xA;&#xA;&#xA; #Model with microbiome based on Season, region&#xA;&#xA;whole_root_pred=data.frame(Season=Meta[RootSamples,&amp;quot;Season&amp;quot;],Region=Meta[RootSamples,&amp;quot;Region&amp;quot;],a=Root_OTU_table)&#xA; &#xA;head(whole_root_pred)&#xA;     Season Region a.d2ec9f3b77975c0f457e4b7413b217ff&#xA;     a.3147790f0d5a78316fb9dd64f53b9473 a.97aecc1f35cc1f50db507ad71dd22367&#xA;     a.bfad6370d28182cc6304844e9bec7fb6 a.5fa2a987221a1d9ca416148570c18086 &#xA;&#xA;&#xA;    **RF_model_Root_all=randomForest(y=?,sampsize=c(143,143),strata=?,x=whole_Root_pred,importance = T,proximity = T,ntree =&#xA;    NTREE,mtry = NbVar)**&#xA;&#xA;print(RF_model_Root_all)&#xA;     #plot summary using the 5% most important OTUs ERROR ON LAST LINE&#xA;     imp=data.frame(importance(RF_model_Root_all))&#xA;     imp&lt;span class=&quot;math-container&quot;&gt;$genus=as.character(taxo[rownames(imp),&quot;Genus&quot;])&#xA;     Best=imp[imp$&lt;/span&gt;MeanDecreaseAccuracy&amp;gt;quantile(x = imp$MeanDecreaseAccuracy,.95),]&#xA;     bymedian &amp;lt;- with(Best, reorder(genus, -MeanDecreaseAccuracy, median))&#xA;    &#xA;     pdf(width = 20,height = 10,file=paste(pathforplots,&amp;quot;Variable_Importance_Root_BothRegion_raref.pdf&amp;quot;,sep=&amp;quot;&amp;quot;))&#xA;     par(mar=c(15,5,1,1))&#xA;     boxplot(Best&lt;span class=&quot;math-container&quot;&gt;$MeanDecreaseAccuracy ~ bymedian, data = Best,&#xA;     xlab = &quot;&quot;, ylab = &quot;Variable Importance&quot;,&#xA;     main = paste(&quot;Root in Both Countries; Error Rate=&quot;,round(RF_model_Feces_all$&lt;/span&gt;err.rate[NTREE,&amp;quot;OOB&amp;quot;],3),sep=&amp;quot;&amp;quot;), varwidth = TRUE,&#xA;     col = &amp;quot;lightgray&amp;quot;,las=2)&#xA;    &#xA;    &#xA;     dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Many thanks&lt;/p&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="1008" LastEditDate="2020-10-15T13:32:33.883" LastActivityDate="2020-10-15T13:32:33.883" Title="error in random forest analysis" Tags="|r|bioconductor|statistics|microbial-genomics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14528" PostTypeId="1" CreationDate="2020-10-11T04:40:21.173" Score="1" ViewCount="82" Body="&lt;p&gt;I am enrolled in an online undergrad CS program in Canada and also Georgia Tech OMSCS where I finished computational photography and AI for Robotics.  I also have some deep learning programming knowledge.&lt;br /&gt;&#xA;For my capstone project in the undergrad, I was building a nutrition app but I lost interest after finding so many other apps and repos with similar ideas.  So I though since I'm taking an intro bioinformatics course this final term before I graduate, I could do a bioinformatics programming project.&lt;br /&gt;&#xA;Does anyone have any good ideas?  I'm free labor if I can get approval from my course supervisor for your proposal.  Time length would be approximately 10 weeks.&lt;br /&gt;&#xA;My languages in order of familiarity: Python, C++, C#, Java, C, Javascript, HTML, CSS.&lt;/p&gt;&#xA;" OwnerUserId="9650" LastActivityDate="2020-10-11T04:40:21.173" Title="project ideas for a final year CS undergrad/CS masters student" Tags="|python|" AnswerCount="0" CommentCount="1" ClosedDate="2020-10-11T15:57:57.340" ContentLicense="CC BY-SA 4.0" />
  <row Id="14530" PostTypeId="2" ParentId="14526" CreationDate="2020-10-11T17:45:53.293" Score="2" Body="&lt;p&gt;This question will be a little hard to answer without more information.&lt;/p&gt;&#xA;&lt;p&gt;For example, we will need to see your dataset (&lt;code&gt;whole_root_pred&lt;/code&gt;), to decide why &lt;code&gt;Stunting_Root&lt;/code&gt; is NULL.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You might need to initialize &lt;code&gt;Stunting_Root&lt;/code&gt; as a variable. It is currently not clear if it is e.g. a column of your dataframe, or just uninitialized. Uninitialized variables are NULL, which would explain your problem. randomForest might not know to look for &lt;code&gt;strata&lt;/code&gt; inside your dataframe, for example. Is it in your dataframe?&lt;/li&gt;&#xA;&lt;li&gt;Also, I might be missing something, but why are you passing &lt;code&gt;?&lt;/code&gt; as a response? I'm not an expert but I believe that is an illegal character in R (&lt;a href=&quot;https://www.quora.com/How-can-we-treat-question-mark-in-R-dataset&quot; rel=&quot;nofollow noreferrer&quot;&gt;I'm pretty sure&lt;/a&gt;?).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-10-11T17:45:53.293" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14531" PostTypeId="1" CreationDate="2020-10-12T00:04:23.057" Score="1" ViewCount="47" Body="&lt;p&gt;I am using &lt;code&gt;godon&lt;/code&gt; to find genes under positive selection with the &lt;strong&gt;branch-site test&lt;/strong&gt;. However, after running for a little while I get the error &lt;em&gt;&amp;quot;panic: division of zero by zero or infinity by infinity&amp;quot;&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Below is the full output with the error report. I can supply the phylogenetic tree and fasta file I am using if that would be helpful. Thank you.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;iter=33 lnL=-73104.072&lt;br /&gt;&#xA;iter=66 lnL=-72956.855&lt;br /&gt;&#xA;iter=99 lnL=-72912.956&lt;br /&gt;&#xA;iter=131 lnL=-72902.900&lt;br /&gt;&#xA;iter=162 lnL=-72900.147&lt;br /&gt;&#xA;iter=194 lnL=-72898.562&lt;br /&gt;&#xA;iter=226 lnL=-72897.970&lt;br /&gt;&#xA;iter=258 lnL=-72897.651&lt;br /&gt;&#xA;iter=289 lnL=-72897.435&lt;br /&gt;&#xA;iter=321 lnL=-72897.284&lt;br /&gt;&#xA;iter=353 lnL=-72897.199&lt;br /&gt;&#xA;iter=385 lnL=-72897.153&lt;br /&gt;&#xA;iter=416 lnL=-72897.108&lt;br /&gt;&#xA;iter=448 lnL=-72897.076&lt;br /&gt;&#xA;iter=17 lnL=-72890.551&lt;br /&gt;&#xA;panic: division of zero by zero or infinity by infinity&#xA;goroutine 1 [running]:&#xA;math/big.(*Float).Quo(0xc4273a8f40, 0xc4273a8f68, 0xc4273a8f18, 0xc42e88c118)&#xA;/usr/local/go/src/math/big/float.go:1639 +0x15c&#xA;bitbucket.org/Davydov/godon/cmodel.(*BranchSite).BEBPosterior(0xc422164120, 0xc4259bb900, 0x91, 0x91)&#xA;/go/src/bitbucket.org/Davydov/godon/cmodel/bs.go:416 +0x6e9&#xA;bitbucket.org/Davydov/godon/cmodel.(*BranchSite).Final(0xc422164120, 0x101)&#xA;/go/src/bitbucket.org/Davydov/godon/cmodel/bs.go:455 +0x112&#xA;main.computeFinal(0x1f263a0, 0xc422164120, 0xc4201522a0, 0x101, 0x6a80, 0x0)&#xA;/go/src/bitbucket.org/Davydov/godon/godon/optimize.go:150 +0x93&#xA;main.performSingleTest(0xc420162320, 0xc428266000, 0x9399, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)&#xA;/go/src/bitbucket.org/Davydov/godon/godon/hyptest.go:335 +0x1b52&#xA;main.hypTest(0x1ee03a7, 0x1ee03a7, 0x4, 0xc4201aa501, 0x1, 0x1)&#xA;/go/src/bitbucket.org/Davydov/godon/godon/hyptest.go:88 +0x71a&#xA;main.main()&#xA;/go/src/bitbucket.org/Davydov/godon/godon/godon.go:316 +0xefa&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9665" LastEditorUserId="3848" LastEditDate="2020-10-12T01:50:14.653" LastActivityDate="2020-10-12T01:54:58.687" Title="panic: division of zero by zero or infinity by infinity error when running GODON positive selection analysis" Tags="|phylogenetics|phylogeny|positive-selection|codon|godon|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14532" PostTypeId="2" ParentId="14531" CreationDate="2020-10-12T01:48:19.120" Score="1" Body="&lt;p&gt;Ah wait you mentioned a branch sites test, right that makes sense. You've got two or more identical sequences in the analysis. Ensure there are no identical sequences in your alignments and ... PRESTO! It will work :-)&lt;/p&gt;&#xA;&lt;p&gt;Below are my original thoughts before I re-read your question. I've kept the notes because it helps explain the theory a bit. There is no need to supply the tree, it should be resolved.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The tree likelihood looks to be decreasing nicely, so it can see both the tree and the codons in the alignment.&lt;/p&gt;&#xA;&lt;p&gt;At a guess the error relates to two or more indentical nucleotide sequences in your alignment. Thus zero non-synonymous sites (amino acid change) and zero synonymous (silent) sites. This error would only occur if you were calculating selection for each 'node' in the tree (internal branch).  Alternatively, possibly if the number of synonymous sites was zero, regardless of the non-synonymous mutations this error could appear, but this doesn't seem to fit the error output.&lt;/p&gt;&#xA;&lt;p&gt;I don't know &amp;quot;Godon&amp;quot;, but recommend &lt;code&gt;PAML&lt;/code&gt; for this analysis. Admittedly it is complicated to deploy, but it is a very robust algorithm and hugely respected.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-10-12T01:54:58.687" LastActivityDate="2020-10-12T01:54:58.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14533" PostTypeId="1" AcceptedAnswerId="14535" CreationDate="2020-10-12T03:17:55.133" Score="1" ViewCount="171" Body="&lt;p&gt;I'm trying to run bwa on HiC data. This is the command I'm giving:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bwa mem -5SP -T0 -t4 -R '@RG\tID:\tSM:\tLB:\tPL:ILLUMINA\tPU:none' REFERENCE.fa PAIR1.fq PAIR2.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and here's the error message:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mem: illegal option -- 5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Supposedly, the 5SP option is for HiC data (&lt;a href=&quot;https://phasegenomics.github.io/2019/09/19/hic-alignment-and-qc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;cite&lt;/a&gt;), but my BWA isn't recognizing it. Anybody have any ideas? My initial thought is to update BWA to the newest version, but I don't want to mess with our replicatibility since I am new to the lab and don't want to be updating software when I don't need to. The server is a Mac Pro and bwa version is 0.7.12-r1039. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="9521" LastActivityDate="2020-10-12T14:47:32.080" Title="BWA doesn't recognize HiC options" Tags="|bwa|hi-c|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14534" PostTypeId="2" ParentId="14533" CreationDate="2020-10-12T05:17:58.757" Score="3" Body="&lt;p&gt;Your link with those command line options is 1 year old.  Looks like that version of bwa is 6 years old.  You probably should update.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-10-12T05:17:58.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14535" PostTypeId="2" ParentId="14533" CreationDate="2020-10-12T14:47:32.080" Score="3" Body="&lt;p&gt;This option has been added in 0.7.16a, see &lt;a href=&quot;https://github.com/lh3/bwa/releases&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/lh3/bwa/releases&lt;/a&gt;, so consider updating.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-10-12T14:47:32.080" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14537" PostTypeId="2" ParentId="14509" CreationDate="2020-10-12T16:30:56.613" Score="1" Body="&lt;p&gt;I cannot think of a way without performing differential expression analysis. Since you don't have controls, what you could do is assigning some of the samples as controls.&lt;/p&gt;&#xA;&lt;p&gt;Let's say you have samples A, B, C, D, E and F. You can perform differential expression analysis A, B, C (i.e. treatment) versus D, E (control). If you perform this for all possible combinations (that would be 5 choose 2 which is equal to 5 choose 3 if you decide to go for 3 vs 2 splits) and filter for genes which do not show up in any DE list you end up with the genes that are expressed similarly in al of these samples.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-10-12T16:30:56.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14540" PostTypeId="1" CreationDate="2020-10-12T18:02:19.137" Score="0" ViewCount="68" Body="&lt;p&gt;I am running into this issue lately when I am trying to install the Cobratoolbox on matlab along with having installed the prerequisite gurobi solver and adding it to the MATLAB path. I have run the gurobi_setup.m file as well. However I am unable to uninstall the cobratoolbox, as everytime i run initCobraToolbox.m it fails as it not able to recognize the solver. Is there anything I am missing in the installation? I have attached a screenshot for reference&lt;a href=&quot;https://i.stack.imgur.com/km20R.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/km20R.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9671" LastActivityDate="2020-10-12T18:02:19.137" Title="Cobratoolbox unable to identify gurobi solver when passing initCobraToolbox" Tags="|matlab|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14541" PostTypeId="2" ParentId="14434" CreationDate="2020-10-12T19:50:38.977" Score="1" Body="&lt;p&gt;You need to give the path to the source files for the compiler &lt;em&gt;and&lt;/em&gt; the full path to the compiled libraries (all parent directories included) for the linker.  You also need to use either CPATH or CPLUS_INCLUDE_PATH instead of C_INCLUDE_PATH to point to the oneTBB source, since bowtie2 is C++ and C_INCLUDE_PATH will only have an effect for C (see &lt;a href=&quot;http://gcc.gnu.org/onlinedocs/gcc/Environment-Variables.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Environment Variables Affecting GCC&lt;/a&gt; in the GCC documentation for more info.)&lt;/p&gt;&#xA;&lt;p&gt;For example like this, inside bowtie2-2.4.1, without making any lasting changes to the shell's setup in this case:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ ONETBB=$&lt;/span&gt;HOME/soft/oneTBB&#xA;&lt;span class=&quot;math-container&quot;&gt;$ export LIBRARY_PATH=$&lt;/span&gt;ONETBB/build/linux_intel64_gcc_cc5.4.0_libc2.23_kernel4.4.0_release&#xA;&lt;span class=&quot;math-container&quot;&gt;$ export LD_LIBRARY_PATH=$&lt;/span&gt;LIBRARY_PATH&#xA;&lt;span class=&quot;math-container&quot;&gt;$ export CPATH=$&lt;/span&gt;ONETBB/include&#xA;$ make&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After it's built you'll still need the library path to oneTBB defined (via LD_LIBRARY_PATH) so bowtie2 can find it at run-time.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;But if you don't have a specific reason to compile from source I'd recommend avoiding it.&lt;/strong&gt;  You could go with what h.mon suggested and use an existing binary, or use a package manger to install it like zorbax suggested.  &lt;a href=&quot;https://conda.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conda&lt;/a&gt; has &lt;a href=&quot;https://anaconda.org/bioconda/bowtie2&quot; rel=&quot;nofollow noreferrer&quot;&gt;a package for bowtie2 in the bioconda channel&lt;/a&gt;, up through and beyond version 2.4.1, and you can use that as a package manager without root access.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2020-10-12T19:50:38.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14542" PostTypeId="2" ParentId="3086" CreationDate="2020-10-12T21:29:34.070" Score="1" Body="&lt;p&gt;all. I know this question was asked 2 years ago, but I wrote a script in R, with a little bash, to parse the tsv file in the old version of prokka. In the new version of prokka this problem is fixed.&lt;/p&gt;&#xA;&lt;p&gt;First use bash to get the rows where ftype is CDS.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep &amp;quot;CDS&amp;quot; all_prokka_annotations.tsv &amp;gt; all_prokka_annotations_CDS.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Next we can parse the TSV with the CDS lines in R.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;prokka_annotations = readLines(&amp;quot;all_prokka_annotations_CDS.tsv&amp;quot;)&#xA;prokka_annotations_split = strsplit(prokka_annotations,split=&amp;quot;\t&amp;quot;)&#xA;# first get the protein product name. &#xA;prokka_annotations_product = sapply(prokka_annotations_split, function(x) x[length(x)])&#xA;# now the I have all protein product names remove them&#xA;prokka_annotations_split = sapply(prokka_annotations_split, function(x) x[-length(x)])&#xA;#next get locus tag and ftype&#xA;locus_tag_cds = sapply(prokka_annotations_split, function(x) x[c(1,2)])&#xA;locus_tag_cds = t(locus_tag_cds)&#xA;# now remove locus tag and ftype&#xA;prokka_annotations_split = sapply(prokka_annotations_split, function(x) x[-c(1,2)])&#xA;# next get EC number&#xA;ec_number = lapply(prokka_annotations_split, function(x) x[grep(&amp;quot;.&amp;quot;,x,fixed=T)])&#xA;ec_number = as.character(ec_number)&#xA;ec_number[ec_number==&amp;quot;character(0)&amp;quot;] &amp;lt;- NA&#xA;## now get the gene names if available. to do this remove the EC numbers. whats left are gene names&#xA;prokka_gene_names = lapply(prokka_annotations_split, function(x) x[!grepl(&amp;quot;.&amp;quot;,x,fixed=T)])&#xA;prokka_gene_names = as.character(prokka_gene_names)&#xA;prokka_gene_names[prokka_gene_names==&amp;quot;character(0)&amp;quot;] &amp;lt;- NA&#xA;prokka_parsed_tsv_correct = cbind(locus_tag_cds,product=prokka_annotations_product,gene=prokka_gene_names,EC=ec_number)&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9665" LastActivityDate="2020-10-12T21:29:34.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14543" PostTypeId="2" ParentId="14516" CreationDate="2020-10-13T01:26:35.997" Score="1" Body="&lt;p&gt;In the initial description for &amp;quot;usable fragments&amp;quot;, ENCODE has the following (emphasis mine):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Usable fragment – A fragment is defined as the sequencing output &lt;strong&gt;corresponding to one location in the genome&lt;/strong&gt;. If single-ended sequencing is performed, one read is considered a fragment. If paired-ended sequencing is performed, one pair of reads is considered a fragment. &lt;strong&gt;Fragments are considered usable if they pass the various filters in the ChIP-seq uniform processing pipelines&lt;/strong&gt;. Used to evaluate ChIP-seq data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Aligning reads to repetitive regions is difficult because they don't give unique alignments.&#xA;Reads from these regions will often have poor alignment scores and may not pass alignment QC checks.&#xA;Reads may also be counted as multi-mapped, and often methods call for uniquely mapped reads, which again, may not be true for a large number of reads from H3K9me3 ChIP-seq.&lt;/p&gt;&#xA;&lt;p&gt;So for this particular histone mark, ENCODE loosens the definition for what you should count towards a minimum sequencing depth threshold.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2020-10-13T01:26:35.997" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14544" PostTypeId="1" AcceptedAnswerId="15016" CreationDate="2020-10-13T13:09:14.370" Score="1" ViewCount="84" Body="&lt;p&gt;I’m very new to python, and having some difficulty getting hang of some more complicated things&lt;/p&gt;&#xA;&lt;p&gt;I have multiple files which look like so:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;hCoV-19/Singapore/4/2020|EPI_ISL_410535|2020-02-03&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;hCoV-19/USA/WA13-UW9/2020|EPI_ISL_413601|2020-03-02&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;hCoV-19/USA/WA-UW142/2020|EPI_ISL_416680|2020-03-11&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Please be aware that the lines above are meant to be one file&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want to extract the EPI_ISL_000000 for an easy comparison among files.&lt;/p&gt;&#xA;&lt;p&gt;Could someone please advise on:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;A programme to extract this data into new files (There’s many lines in each file - 1000+)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A programme to then give a % comparison between two or more files - comparing all lines in one file against all lines in a second+ file&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9672" LastEditorUserId="3848" LastEditDate="2020-12-01T10:23:38.243" LastActivityDate="2020-12-06T20:19:30.873" Title="How to get % similarity between strains and mutation files" Tags="|phylogenetics|genomics|covid-19|sars-cov-2|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14545" PostTypeId="1" CreationDate="2020-10-13T13:30:34.793" Score="3" ViewCount="67" Body="&lt;p&gt;Sometimes dbSNP reports very different allele frequencies for different large-scale genome projects e.g. between 1000 Genomes and GnomAD&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/rs11822440#frequency_tab&quot; rel=&quot;nofollow noreferrer&quot;&gt;rs11822440&lt;/a&gt; 1000Genomes A=0.4629    C=0.5371 GnomAD A=0.99997   T=0.00003&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/rs28531832#frequency_tab&quot; rel=&quot;nofollow noreferrer&quot;&gt;rs28531832&lt;/a&gt; 1000Genomes T=0.4379    C=0.5621 GnomAD T=0.99993   A=0.00007&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/rs78931618#frequency_tab&quot; rel=&quot;nofollow noreferrer&quot;&gt;rs78931618&lt;/a&gt; 1000Genomes A=0.7334    C=0.2666 GnomAD A=0.9999    T=0.0001&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/rs75716913#frequency_tab&quot; rel=&quot;nofollow noreferrer&quot;&gt;rs75716913&lt;/a&gt; 1000Genomes T=0.9006    C=0.0994 GnomAD T=0.99149   A=0.00851&lt;/p&gt;&#xA;&lt;p&gt;Is this an issue with one or other of these projects data processing? Is something else going on here biologically or technically that might cause this?&lt;/p&gt;&#xA;" OwnerUserId="9125" LastActivityDate="2023-10-29T09:04:45.523" Title="dbnSNP frequency anomalies" Tags="|snp|1000genomes|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14546" PostTypeId="1" CreationDate="2020-10-13T14:03:50.473" Score="1" ViewCount="131" Body="&lt;p&gt;I have a paired-end data. It is from small part of human genome not a WGS or WES. I use BWA for alignment and do variant calling. Since I had some false positives I wanted to do trimming.&lt;/p&gt;&#xA;&lt;p&gt;I used trimmomatic with command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;trimmomatic PE -phred33 -threads 2 read1.fastq.gz read2.fastq.gz read1_trimmed.fastq read2_trimmed.fastq LEADING:3 TRAILING:3 SLIDINGWINDOW:4:15`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I do not know why but I have this output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Input Read Pairs: 1834513 &#xA;Both Surviving: 1822034 (99.32%) &#xA;Forward Only Surviving: 5740 (0.31%)&#xA;Reverse Only Surviving: 417 (0.02%)&#xA;Dropped: 6322 (0.34%)&#xA;TrimmomaticPE: Completed successfully&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I do not have any adapters since the are removed in &lt;code&gt;bcl2fastq&lt;/code&gt; step. In FASTQ quality control I have seen that quality of reverse and forward reads are similar in quality but trimmomatic trimmed many reads in reverse file. I cannot do alignment with these files, BWA gives a warning that says:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[bseq_read] the 2nd file has fewer sequences. &#xA;[process] read 155012 sequences (16693324 bp)... &#xA;[bseq_read] the 2nd file has fewer sequences.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there anyone who faced a problem like this before?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance&lt;/p&gt;&#xA;" OwnerUserId="9508" LastEditorUserId="4282" LastEditDate="2020-10-13T15:55:19.477" LastActivityDate="2020-11-12T16:03:40.293" Title="Trimmomatic trims most of the reverse strand" Tags="|ngs|quality-control|trimming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14548" PostTypeId="2" ParentId="14546" CreationDate="2020-10-13T14:45:10.610" Score="1" Body="&lt;p&gt;Maybe you need to remove the unpaired reads:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;trimmomatic PE -phred33 -threads 2 \&#xA;    read1.fastq.gz read2.fastq.gz \&#xA;    read1_trimmed.fastq.gz read1_unique.fastq.gz \&#xA;    read2_trimmed.fastq.gz read2_unique.fastq.gz \&#xA;    LEADING:3 TRAILING:3 SLIDINGWINDOW:4:15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then you can try with bwa.&lt;/p&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2020-10-13T14:45:10.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14549" PostTypeId="1" CreationDate="2020-10-13T15:20:58.187" Score="2" ViewCount="100" Body="&lt;p&gt;First of all let me apologize if this is not the place for this question. While it can be quite broad, I will try to make it as specific as I can.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Context&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am testing the correlation between Whole Genome Bisulfite Sequencing and Oxford Nanopore methylation calls for one individual (mammal, blood tissue) with replicates at all CpG sites. In the (very) best cases I find a close to 0.9 correlation. Yet, for the part that does not correlate I would like to know which technology is closer to the &amp;quot;biological truth&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My strategy&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;My plan would be to first find CpG sites/short CpG regions showing differential methylation states between WGBS and ONT. Then I would validate the methylation states of these sites using a third party technology such as pyromark.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What I am asking to the SE community&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;First, has anyone done this before ? I would love to hear about your experience&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you see some critical aspects I should be watching out for ?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Last if you have comments about the strategy I propose here, you are welcome&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6273" LastEditorUserId="6273" LastEditDate="2020-10-13T20:16:43.783" LastActivityDate="2020-10-15T20:59:57.303" Title="WGBS vs ONT, which one should I trust?" Tags="|methylation|epigenetics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14550" PostTypeId="2" ParentId="14549" CreationDate="2020-10-13T19:27:44.313" Score="2" Body="&lt;p&gt;Methylation can be cell-specific, which makes it difficult to evaluate accuracy on a bulk-cell level (even within the same tissue). How can you tell that the differences you're seeing are due to platform differences, or due to biological variation?&lt;/p&gt;&#xA;&lt;p&gt;I find that adding more haystacks doesn't help much in working out the truth of a dataset. If you want to investigate biological truth, it would be better to create a biological system with a known methylation (or demethylation) pattern and test that.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-10-15T20:59:57.303" LastActivityDate="2020-10-15T20:59:57.303" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14551" PostTypeId="2" ParentId="14549" CreationDate="2020-10-14T02:16:42.007" Score="1" Body="&lt;p&gt;I would assess directionality and accuracy of prediction by 1) WGBS predicting ONT and then 2) ONT predicting WGBS.&lt;/p&gt;&#xA;&lt;p&gt;Firstly, I would use deep learning (or machine learning) and train WGBS against ONT, parameterise and then test. Then conversely train ONT against WGBS, parameterise and then test. The approach with highest accuracy of prediction (using the 'accuracy' index) would be the approach assumed to be &amp;quot;closer to the biological truth&amp;quot;. If both calculations produced comparable accuracies I would conclude what @gringer has stated that natural variation / natural heterogeneity is the predominant signal in the signal. This conclusion depends on the ability of deep learning to map the biological process.&lt;/p&gt;&#xA;&lt;p&gt;It is an approach highly applicable to a deep learning estimate regardless and may not replace a truely controlled test, this will depend on what controls you have run, but could provide a valuable and easily obtainable insight in its absence.&lt;/p&gt;&#xA;&lt;p&gt;If you did run a control sample, then you are truely in, because this would provide the training set and the WBGS and ONT provide the test data.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;You asked me how it works&#xA;It doesn't replace good controls, but it is 'trendy workaround'.&lt;/p&gt;&#xA;&lt;p&gt;Two steps&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In deep learning (artificial neural networks, ANN, in this case you would use RNN such as LSTM) or machine learning (ML) e.g. random forest - random forests are MUCH easier to do, you supply a series of known values, thus CpG sites and their WGBS scores and use ANN (RNN approach) or ML (e.g. random forest). Both methods will look for pattens between the observed DNA data and the CpG occurence (training set). Usually this is 60% of your data set is used in training. A separate 20% of the data set is often used for parameterisation and a final 20% of the data set is used for testing. Thus how well can a ANN or ML predict a CpG site based on WGBS data. Likewise you would do the same for ONT. There is a caveat here which I discuss below.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The idea is if you give it a bit of DNA it will assign whether it is CpG (see caveat below).&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;However, instead of testing WGBS to WGBS you also test it WGBS to ONT and ONT to WGBS and the method with the highest 'accuracy' (positive hits, against true hits) is 'better method'. This is because you already know which is ONT positive and ask a WGBS trained algorithm whether it thinks its positive. The relationship between observed CpG and 'true (experimental) CpG is your accuracy score.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The caveat is that you need to supply the training set with sites that are known to be negative under WGBS or ONT. You need to think the best strategy for doing this. If some of those negatives were positve in the other method that would affect the outcome alot.&lt;/p&gt;&#xA;&lt;p&gt;There is an issue about vectorisation this is often done using k-mers, and is a headache. Basically you have to make a bit of DNA into numbers and that in my opinion is the difficult part. In your case I ain't so sure its so difficult. If you can get that bit correct and biologically meaningful you are in business. My stuff is phylogeny based and I understand the relationship between biology (mutation) and numbers. In your case that needs thought, however someone will have thought about it and solved it.&lt;/p&gt;&#xA;&lt;p&gt;As a personal statement I would do this via random forests, because ANN is difficult.&lt;/p&gt;&#xA;&lt;p&gt;One such vectorisation method is &lt;a href=&quot;https://www.nature.com/articles/s41598-017-12493-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; by Li et al (2017), however as they have used it, the method has flaws due to equal weighting of mutation frequencies. However, these flaws do not apply to your work - it would actually be a good method, they singly apply to tree building.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The final thing is that the more experimental data you can feed it the better. You don't have to complicated about it, just create a new column in the input section, e.g. along side teh vectorised DNA and the algorithm will figure out whether it helps. You can't feed it enough data, because it is doesn't help it wouldn't use it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-10-15T18:51:26.510" LastActivityDate="2020-10-15T18:51:26.510" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14552" PostTypeId="2" ParentId="14545" CreationDate="2020-10-14T02:33:29.103" Score="1" Body="&lt;p&gt;I'll stick my neck out and simply state that allopatry alone would account for the pattern you are observing. Thus populations can exihibit variation in allelic profiles, particularly where Mendellian inheritance is operating (i.e. higher eukaryotes), singly due to geographic isolation.&lt;/p&gt;&#xA;&lt;p&gt;This represents the Meyers view of population genetics and would account for variation within neutral theory, e.g. silent mutations. Obviously I'm guessing the populations were selected by geographic foci in generating the allelic frequencies.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-10-14T02:33:29.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14556" PostTypeId="2" ParentId="3380" CreationDate="2020-10-14T18:17:40.730" Score="7" Body="&lt;p&gt;&lt;strong&gt;UPDATE 2021/06/28:&lt;/strong&gt; since version 1.12, samtools now accepts option &lt;code&gt;-N&lt;/code&gt;, which takes a file containing read names of interest.&lt;/p&gt;&#xA;&lt;p&gt;Using samtools 1.12 or greater:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -N qnames_list.txt -o filtered_output.bam input.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;OLD ANSWER:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;When it comes to filter by a list, this is my favourite (much faster than grep):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# given a bam file &amp;quot;aln.bam&amp;quot; and a list of read names &amp;quot;reads.txt&amp;quot;:&#xA;samtools view -h aln.bam \&#xA;| awk 'FNR==NR {reads[$1];next} /^@/||($1 in reads)' reads.txt - \&#xA;| samtools view -b - &amp;gt; filtered_aln.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Basically, the awk command first parses the reads.txt file, creates an array of QNAMEs, then parses the alignments and print if the QNAME is present in the array (or if the line belongs to the header).&lt;/p&gt;&#xA;" OwnerUserId="2355" LastEditorUserId="2355" LastEditDate="2021-10-22T15:06:13.123" LastActivityDate="2021-10-22T15:06:13.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14557" PostTypeId="1" AcceptedAnswerId="15015" CreationDate="2020-10-14T20:36:46.090" Score="1" ViewCount="41" Body="&lt;p&gt;I'm very new to python and I'm getting along nicely (I'd like to believe); however, I must be missing something here. I'm looking to read each and every file and compare each line against one another.&lt;/p&gt;&#xA;&lt;p&gt;The aim of this is to see if any mutations are shared in any of the strains - to determine the genotype.&lt;/p&gt;&#xA;&lt;p&gt;The inner-workings of the files are single spaced headers, as those you would find on GISAID.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;os.chdir(&amp;quot;C:)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with open('A D614G.txt', 'r') as file2:&#xA;with open('A E780Q.txt', 'r') as file3:&#xA;with open('A G476S.txt', 'r') as file4:&#xA;with open('A L18F.txt', 'r') as file5:&#xA;with open('A N439K.txt', 'r') as file6:&#xA;with open('A S477N.txt', 'r') as file7:&#xA;with open('A T478I.txt', 'r') as file8:&#xA;with open('A V483A.txt', 'r') as file9:&#xA;                               &#xA;files = [file1, file2, file3,file4, file5, file6, file7, file8, file9]&#xA;&#xA;it = itertools.permutations((files), len(files))&#xA;&#xA;for x in it:&#xA;    print(x)&#xA;&#xA;os.chdir(&amp;quot;C:)&#xA;with open('tester.txt', 'wt') as file_out:&#xA;    for x in it: &#xA;        file_out.write()```&#xA;&#xA;1 I recieve the following error on a very long repeated stretch of text&#xA;&#xA;```&amp;lt;_io.TextIOWrapper name='A G476S.txt' mode='r' encoding='cp1252'&amp;gt;```&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9672" LastEditorUserId="3848" LastEditDate="2020-12-03T16:31:22.100" LastActivityDate="2020-12-06T20:11:42.263" Title="How to find Genotype, from list of mutations:lineage list?" Tags="|genomics|covid-19|sars-cov-2|python|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14559" PostTypeId="1" AcceptedAnswerId="14565" CreationDate="2020-10-15T10:55:21.440" Score="1" ViewCount="365" Body="&lt;p&gt;I have 10 samples from a RNAseq experiment (5 control, 5 disease), I have performed a cluster analysis for the samples and for the genes (4000 genes aprox) to see how they cluster (to see which samples are similar and which genes have similar expressions).&#xA;I was wondering if in terms of statistics it makes any sense to perform a PCA of the samples instead of the genes?&lt;/p&gt;&#xA;&lt;p&gt;thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="9010" LastEditorUserId="9010" LastEditDate="2020-10-15T11:44:21.057" LastActivityDate="2020-10-15T21:09:35.730" Title="Performing PCA for the samples and for the genes" Tags="|pca|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14561" PostTypeId="1" AcceptedAnswerId="14566" CreationDate="2020-10-15T14:10:24.250" Score="1" ViewCount="1268" Body="&lt;p&gt;can you please give me an advice how to convert .cif files into .pdb preferably using python?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance!&#xA;Best,&#xA;Balint Biro&lt;/p&gt;&#xA;" OwnerUserId="9689" LastActivityDate="2020-10-15T21:26:11.710" Title=".cif .pdb conversion with python" Tags="|python|pdb|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14562" PostTypeId="2" ParentId="14559" CreationDate="2020-10-15T16:07:12.050" Score="1" Body="&lt;p&gt;If you use, say, R's prcomp, you can use &lt;code&gt;my_pca_object$x&lt;/code&gt; to get PC coordinates for the samples, and &lt;code&gt;my_pca_object$rotation&lt;/code&gt; to get how much each gene contributes to each PCA.  (You might need to transpose your transformed count file to get it to work right)&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-10-15T16:07:12.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14563" PostTypeId="2" ParentId="14559" CreationDate="2020-10-15T18:10:18.677" Score="1" Body="&lt;p&gt;It is quite possible to make a PCA plot with the samples. Follow this tutorial : &lt;a href=&quot;https://hbctraining.github.io/DGE_workshop/lessons/03_DGE_QC_analysis.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hbctraining.github.io/DGE_workshop/lessons/03_DGE_QC_analysis.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6694" LastActivityDate="2020-10-15T18:10:18.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14565" PostTypeId="2" ParentId="14559" CreationDate="2020-10-15T21:09:35.730" Score="2" Body="&lt;p&gt;First of all, I am not a statistician but have been performing RNA-seq analysis for a while, here is my take:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I think performing PCA on the samples makes sense in terms of mathematics. What I mean is you can compute eigenvectors and subsequently principal components either on your genes or on your samples.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I don't think performing PCA on the samples makes sense in terms of statistics in RNA-seq context (and probably in any other context): Your genes (features) describe your samples and not the other way around. So the computed PCs should describe/explain your samples and not the other way around. You would not perform PCA on your samples (individual cells in scRNA-seq) even in the case of scRNA-seq, where number of samples sometimes exceeds the number of genes, there PCA is performed on genes and the resulting few PCs are used for downstream analysis, usually clustering and visualization, both used to infer about samples (cells).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-10-15T21:09:35.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14566" PostTypeId="2" ParentId="14561" CreationDate="2020-10-15T21:26:11.710" Score="2" Body="&lt;p&gt;As @Michael said PyMOL can do this. PyMOL is not only an app, but also a python package —installed &lt;a href=&quot;https://anaconda.org/schrodinger/pymol&quot; rel=&quot;nofollow noreferrer&quot;&gt;via conda&lt;/a&gt; not via the regular download (you can have both).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pymol2&#xA;with pymol2.PyMOL() as pymol:&#xA;     pymol.cmd.load(infile,'myprotein')&#xA;     pymol.cmd.save(infile.replace('.cif', '.pdb'), selection='myprotein')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This runs the parallelisable instance, so you can have that context manager block running on how many threads or processes you fancy.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-10-15T21:26:11.710" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14567" PostTypeId="1" CreationDate="2020-10-16T08:39:47.153" Score="0" ViewCount="53" Body="&lt;p&gt;I am studying about tissue cross-talk using gene expression data. My first goal is to identify the genes responsible for communication signaling(cross talk).  Let say the tissues are fat and muscle.&#xA;One thing that came to my mind was doing correlation analysis and visualizing using a heatmap.&#xA;Here I would like to get your help with the following points:&#xA;How to extract the one where both tissues showing a possible correlation from the four possible quadrants of the correlation matrix?&#xA;How best I can visualize that quadrant using heatmap or any other plotting methods.&#xA;Here I simulated the data as follows. Rows are genes and columns as samples.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;set.seed(1)&#xA;A &amp;lt;- data.frame(rnorm(100), &#xA;                rnorm(100),&#xA;                rnorm(100), &#xA;                rnorm(100),&#xA;                rnorm(100))&#xA;row.names(A) &amp;lt;- paste0(&amp;quot;G_&amp;quot;, 1:100)&#xA;colnames(A) &amp;lt;- paste0(&amp;quot;S_&amp;quot;, 1:5)&#xA;&#xA;set.seed(42)  &#xA;B &amp;lt;- data.frame(rnorm(100), &#xA;                rnorm(100),&#xA;                rnorm(100), &#xA;                rnorm(100),&#xA;                rnorm(100))&#xA;row.names(B) &amp;lt;- paste0(&amp;quot;G_&amp;quot;, 1:100)&#xA;colnames(B) &amp;lt;- paste0(&amp;quot;S_&amp;quot;, 1:5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Tank you!&lt;/p&gt;&#xA;" OwnerUserId="9160" LastActivityDate="2020-10-16T08:39:47.153" Title="tissue cross talk using gene expression data" Tags="|r|gene|correlation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14568" PostTypeId="1" AcceptedAnswerId="14569" CreationDate="2020-10-16T14:35:24.733" Score="0" ViewCount="725" Body="&lt;p&gt;I did a Pearson co-expression analysis for generating networks for my tissue-specific (chondrocytes) RNA-seq data. I used R package poweRlaw to check for the power-law distribution. We got the following values:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; Xmin = 14&#xA; goodness of fit = 0.035&#xA; α = 4.76&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xoJtW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xoJtW.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;My question is what does alpha (α) of more than 4 indicates?&lt;/li&gt;&#xA;&lt;li&gt;Is it is okay to have such values?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="3766" LastEditorUserId="3848" LastEditDate="2020-10-16T19:19:45.450" LastActivityDate="2020-10-16T19:34:31.937" Title="Power law distribution alpha values" Tags="|rna-seq|statistics|correlation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14569" PostTypeId="2" ParentId="14568" CreationDate="2020-10-16T14:52:59.177" Score="3" Body="&lt;p&gt;It means the distribution is heavily skewed and has a 'long-tail', thus the variance is much greater than the mean. In other words most values are around zero, with a small number of values having extreme values.&lt;/p&gt;&#xA;&lt;p&gt;[&lt;img src=&quot;https://i.stack.imgur.com/Puv45.png&quot; alt=&quot;ePower laws distribution 2&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;You need to check the goodness of fit because that is saying the deviation from a power-laws distribution is just significant. So it ain't quite power-laws, but the general gist of a 'long-tailed distribution' is correct.&lt;/p&gt;&#xA;&lt;p&gt;Its a good result and a nice analysis, but you are not quite there at present (needs to be &amp;gt;0.05). Check the negative binomial distribution.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I've seen you data, it looks very good and the power series is not a bad explanation of the data. There are two possibilities for the goodness of fit being 0.035&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The goodness of fit is low because of 'outliers' at the beginning of the distribution&lt;/li&gt;&#xA;&lt;li&gt;An alternative distribution, such as a negative binomial distribution fits better&lt;/li&gt;&#xA;&lt;li&gt;Check it is 0.035 not 0.35 because that changes everything.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The best way to do this is to plot the power series expected distribution onto the graph and look where the fit 'fails'.&#xA;If it is not fitting at the beginning of the distribution thats is about outliers and easily solvable. If it is not really capturing the shape then its probably a slightly different distribution.&lt;/p&gt;&#xA;&lt;p&gt;If its point 1 then just remove the very first point in the graph and try again and check the p-value. Or take the mean of the clump of three points next to the first point and try again (or just remove those three point). Don't do any more than that and the technical word is &amp;quot;outlier&amp;quot; analysis. In your write-up you must explain the issue around any 'outliers' and that is perfectly acceptable. If it is a power-series distrupted by an outlier(s) there is formal analysis to demonstrate an 'outlier' using outlier analysis if any one ever asks you.&lt;/p&gt;&#xA;&lt;p&gt;If its point 2. then use e.g. negative bionomial or the gamma distribution (alpha &amp;lt;1) and check the goodness of fit.&lt;/p&gt;&#xA;&lt;p&gt;You want a goodness of fit &amp;gt; 0.05 and preferably &amp;gt;0.1 to be sure its the right distribution. There are formal statistics to do this, but that is the main one.&lt;/p&gt;&#xA;&lt;p&gt;My guess is there are outliers at the beginning because your sample size is not large enough to identify the variation correctly, hence the fit is low. In 'long-tails' its easy to have a 'low sample size' because of the way they work. Thus in this case mapping a distribution is a perfect way to compensate for the sampling.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-10-16T19:34:31.937" LastActivityDate="2020-10-16T19:34:31.937" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14571" PostTypeId="1" CreationDate="2020-10-16T15:21:06.407" Score="0" ViewCount="83" Body="&lt;p&gt;I am trying to replicate the results of &lt;a href=&quot;https://www.nature.com/articles/onc2014136&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper roughly guided by &lt;a href=&quot;https://www.bioconductor.org/packages/devel/workflows/vignettes/maEndToEnd/inst/doc/MA-Workflow.html#&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; pipeline.&#xA;Basically, we are trying to detect a differential expression between mesenchymal and epithelial cells under cisplatin treatment vs control.&#xA;However, I am facing a problem when trying to calculate the p-value for the Mann-Whitney U-test. This is the part of the paper I am referring to:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Robust Multichip Average (RMA) normalization was performed at the&#xA;transcript level on the results from the Affymetrix Human Gene ST1.0&#xA;arrays using Affymetrix Power Tool 1.14.4.1 for all 46 sham- or&#xA;cisplatin-treated ovarian cancer cell lines. The normalized data were&#xA;subsequently standardized using ComBat71 to remove the batch effect.&#xA;In this experiment, the cisplatin treatment assay was performed in&#xA;triplicate on 20 cell lines, while single assays (without replicate)&#xA;were performed on the remaining 26 cell lines. Taking advantage of the&#xA;triplicate data, potentially fragile probes with strong variations (an&#xA;s.d. of &amp;gt;0.2) within the triplicates were removed, decreasing the&#xA;probe number from 33 297 to 21 329. To perform a fair comparison, the&#xA;triplicate data were then log-averaged into one value so that one&#xA;result for each cell line could be used in the following analyses. To&#xA;detect differential responses to cisplatin between epithelial- and&#xA;mesenchymal-like cell lines, the transcriptomic responses to cisplatin&#xA;were computed by subtracting the gene expression value of control&#xA;(cisplatin untreated) cells from that of cisplatin-treated cells.&#xA;Mann–Whitney U-test (P&amp;lt;0.01 as a cut-off value) was subsequently used&#xA;to detect the differential transcriptomic responses between the&#xA;expression changes by cisplatin treatment in epithelial-like cell&#xA;lines with those in mesenchymal-like cell lines (Supplementary Table&#xA;3).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;My p-values for a given gene are astronomically different to that of the paper...&#xA;Here is my code:&lt;/p&gt;&#xA;&lt;p&gt;expr_f is the expression values matrix from the Expression set (log-transformed).&#xA;SDRF_f is the pData from the same expression set.&#xA;I am feeding into the wilcox.test function a vector of expression from epithelial cells (epi_values) and mesencymal cells (mes_values) for the same probe (8143663).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; genelist=rownames(expr_f)&#xA;epi_samples = rownames(subset(SDRF_f, SDRF_f[,&amp;quot;Classification&amp;quot;]==&amp;quot;Epithelial-like&amp;quot;))&#xA;mes_samples = rownames(subset(SDRF_f, SDRF_f[,&amp;quot;Classification&amp;quot;]==&amp;quot;Mesenchymal-like&amp;quot;))&#xA;&#xA;epi_values = expr_f[&amp;quot;8143663&amp;quot;, epi_samples]&#xA;mes_values = expr_f[&amp;quot;8143663&amp;quot;, mes_samples]&#xA;&#xA;&#xA;wilcox.test(mes_values, epi_values, exact=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In the paper, their p-value for this probe is 0.000372259258165627 while mine is 0.8785. The difference makes no sense at all to me.&#xA;I don't know if I am not quite getting the concept of the u-test, if I am using the function wrong, or if it's something else. Any input is appreciated&lt;/p&gt;&#xA;" OwnerUserId="7382" LastActivityDate="2020-10-18T07:43:36.243" Title="Why is Mann-Whitney U-Test over an ExpressionSet yielding different results than original paper" Tags="|r|microarray|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14572" PostTypeId="1" AcceptedAnswerId="14577" CreationDate="2020-10-16T23:30:01.180" Score="2" ViewCount="145" Body="&lt;p&gt;We are strongly interested in assembly a good transcriptome of reference for a non-model organism and build a local database. We have sequenced the same individual with Illumina (150 millions of pair-end reads) and PacBios IsoSeq v3 (2 SMRT cells, one for shorter transcripts, shorter than 5kb and other for longer transcripts, up to 5kb).&lt;/p&gt;&#xA;&lt;p&gt;To process long-reads, I have followed the PacBio IsoSeq pipeline proposed in their Github repo (&lt;a href=&quot;https://github.com/PacificBiosciences/IsoSeq&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/PacificBiosciences/IsoSeq&lt;/a&gt;). The final result was removing 70% of the long-reads. Is that normal?&lt;/p&gt;&#xA;&lt;p&gt;Using this data, I have assembled the transcriptome using only short reads and another combining long- and short-reads. In the end, I have not found any difference... Approx. The same N50, the same number of transcripts assembled, the rate of misassemblies... Does anyone know if PacBio data does not worth for transcriptome de novo assembly?&lt;/p&gt;&#xA;" OwnerUserId="9701" LastActivityDate="2020-10-18T11:14:53.123" Title="PacBio long-reads impact in transcriptome de novo assembly?" Tags="|rna-seq|assembly|transcriptome|long-reads|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14573" PostTypeId="1" AcceptedAnswerId="14605" CreationDate="2020-10-17T01:32:02.497" Score="1" ViewCount="65" Body="&lt;p&gt;I am using PRSice to compute the PRS over a train set and want to use the coefficient used on the train set to apply it on another set which I will call the test set.&lt;/p&gt;&#xA;&lt;p&gt;Once I compute the PRS I get a file called analysis.prsice&lt;/p&gt;&#xA;&lt;p&gt;This is the few first lines&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;pre&gt;&lt;code&gt;Pheno Set Threshold R2 P Coefficient Standard.Error Num_SNP&#xA;&#xA;- Base 5e-08 0.00550157 0.0659631 -21.0513 11.4492 43&#xA;&#xA;- Base 5.005e-05 0.00396637 0.113759 -61.8545 39.1108 184&#xA;&#xA;- Base 0.00010005 0.00236122 0.220463 -68.9527 56.2742 286&#xA;&#xA;- Base 0.00015005 0.00113035 0.394503 -59.4058 69.7677 382&#xA;&#xA;- Base 0.00020005 0.00072253 0.495129 -55.2605 81.0064 471&#xA;&#xA;- Base 0.00025005 0.000334179 0.642121 -41.5674 89.4432 538&#xA;&#xA;- Base 0.00030005 7.5306e-05 0.825111 -21.391 96.8021 601&#xA;&#xA;- Base 0.00035005 3.95773e-06 0.959527 5.25804 103.612 668&#xA;&#xA;- Base 0.00040005 0.000175506 0.735018 37.723 111.455 743&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I thought that I could apply the PRS on another dataset by multiplying the coefficient in the result by the allelic dosage of the corresponding SNPs. However, I am not sure how to know which are the corresponding SNPs of the coefficients.&lt;/p&gt;&#xA;&lt;p&gt;Also, if there was a command that does that and it would be easier, could you please tell me how to do it?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="9702" LastActivityDate="2020-10-21T15:49:36.963" Title="Apply trained PRS on another dataset" Tags="|snp|gwas|phylogenetics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14575" PostTypeId="1" AcceptedAnswerId="14576" CreationDate="2020-10-17T12:57:03.080" Score="1" ViewCount="954" Body="&lt;p&gt;I writing here because I have some questions for you.&lt;/p&gt;&#xA;&lt;p&gt;I wondered what the essential differences were between &lt;code&gt;paired-end&lt;/code&gt;, &lt;code&gt;mate-pair&lt;/code&gt; and &lt;code&gt;long read&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;for me &lt;code&gt;mate-pair&lt;/code&gt; and classic &lt;code&gt;paired-end&lt;/code&gt; are both &lt;code&gt;paired-end reads&lt;/code&gt;, with the difference that :&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;For classical &lt;code&gt;paired-end&lt;/code&gt;: the insert size on classic paired-end is&#xA;smaller (about 500bp)&lt;/li&gt;&#xA;&lt;li&gt;while the insert size of &lt;code&gt;mate-pair&lt;/code&gt; is much longer (several Kb)&#xA;which allows to join the contiguous between them especially is it?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In revenge for the &lt;code&gt;long-reads&lt;/code&gt;, I imagine that they are simply reads that are synthesized with a large read size but that do not allow like the maite-pair to make junctions between contigs?&lt;/p&gt;&#xA;" OwnerUserId="9104" LastActivityDate="2020-10-17T13:14:41.990" Title="Difference between paired-end, mate-pair and long read" Tags="|assembly|read-mapping|long-reads|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14576" PostTypeId="2" ParentId="14575" CreationDate="2020-10-17T13:14:41.990" Score="4" Body="&lt;p&gt;They are all very different in separate regards, but they all refer to different wet-lab and sequencing protocols/technologies.&lt;/p&gt;&#xA;&lt;p&gt;First, PE (paired end) reads are typically short (50-300) reads, most often Illumina HiSeq, MiSeq or NovaSeq protocols. Both pairs originate from a single fragment which is sequenced from either end:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;------&amp;gt; PE1&#xA;-------------------------------- Fragment&#xA;                         &amp;lt;------ PE2&#xA;       ^^^^^^^^^^^^^^^^^^        Insert&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In contrast mate pairs arise from a fragment that is circularized before sequencing. Now imagine the fragment is circularized:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;                     -----&amp;gt; MP1&#xA;....--------------------------------.......&#xA;          MP2 &amp;lt;-----&#xA;                    ^Circularized (joined) here&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Long reads typically refer to PacBio or Oxford nanopore data, with very long reads lengths (up to several hundred kb).&lt;/p&gt;&#xA;" OwnerUserId="9693" LastActivityDate="2020-10-17T13:14:41.990" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14577" PostTypeId="2" ParentId="14572" CreationDate="2020-10-17T13:32:16.580" Score="5" Body="&lt;p&gt;A few comments:&lt;/p&gt;&#xA;&lt;p&gt;Never use N50 as a metric &lt;em&gt;especially&lt;/em&gt; for transcriptomes. It has some semblance of relevance for genome assembly, but all that is void for a transcriptome with inherently dynamic lengths.&lt;/p&gt;&#xA;&lt;p&gt;At the end of your IsoSeq pipeline, you should have (ideally) full-length transcripts. Have you considered that you don't miss much in your IsoSeq data? The biggest gain using IsoSeq is that it should have fewer non-existent isoforms assembled than a De-Bruin graph approach.&lt;/p&gt;&#xA;&lt;p&gt;Let's say you have the following locus in your genome:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;5'                                                                 3'&#xA;------------=======-----------=======----------=======-------------&#xA;Upstream      E1     Intron     E2    Intron     E3      Downstream&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There are theoretically 7 isoforms:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;E1&lt;/li&gt;&#xA;&lt;li&gt;E2&lt;/li&gt;&#xA;&lt;li&gt;E3&lt;/li&gt;&#xA;&lt;li&gt;E1,E2&lt;/li&gt;&#xA;&lt;li&gt;E2,E3&lt;/li&gt;&#xA;&lt;li&gt;E1,E3&lt;/li&gt;&#xA;&lt;li&gt;E1,E2,E3&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;A short read assembler has to look at evidence that connects exons to see which isoforms are represented. Let's say you have the following PE reads:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;5'                                                                 3'&#xA;------------=======-----------=======----------=======-------------&#xA;               --&amp;gt;..............&amp;lt;--- Read 1&#xA;                                 --&amp;gt;...............&amp;lt;-- Read 2&#xA;                 --___________--&amp;gt;..&amp;lt;-__________--- Read 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Read 1 and 2 both have a mate in either exon, so that's good evidence that they come from a single transcript. You can also have reads like Read 3, which spans two exon-exon junctions, again good evidence. There are some problems, however, if things get more complicated: if the cumulative length of your exons is greater than the fragment length of your short reads, or you have many exons in a gene, you will never get a &lt;em&gt;single&lt;/em&gt; read pair that confirms the existence of some transcripts. The same goes for the opposite. If you find a read pair on a single exon, that is not evidence that this single exon is a transcript in and of itself.&lt;/p&gt;&#xA;&lt;p&gt;Here's where short read assemblers take different approaches. Some make educated guesses about the existence of some transcripts, e.g. by utilizing expression counts in an EM or dynamic programming approach, some just enumerate possibilities.&lt;/p&gt;&#xA;&lt;p&gt;And that's where IsoSeq comes in. A single, long read from a single transcript helps &lt;em&gt;a lot&lt;/em&gt; to add evidence and make sure your transcripts actually represent the transcripts that exist in your organism. So, often, your IsoSeq should be &lt;em&gt;fewer&lt;/em&gt; transcripts than your short read assembly.&lt;/p&gt;&#xA;&lt;p&gt;In essence, what you should do, is take the IsoSeq and curate those as a set of high-confidence transcripts, i.e. those that you have observed from a single read source. That's not to say that some transcripts that exist only in the short read assembly aren't real, just that you have less definitive evidence for them.&lt;/p&gt;&#xA;&lt;p&gt;If you are mostly interested in the gene level, that doesn't make a huge difference, but once you need to look into transcript level usage/expression etc it helps a lot to be able to eliminate low confidence transcripts.&lt;/p&gt;&#xA;&lt;p&gt;Use tools better suited for transcriptome assembly evaluation:&#xA;Detonate, TransRate and (Busco) and compare your assemblies using those.&lt;/p&gt;&#xA;" OwnerUserId="9693" LastEditorUserId="9693" LastEditDate="2020-10-18T11:14:53.123" LastActivityDate="2020-10-18T11:14:53.123" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14579" PostTypeId="2" ParentId="14321" CreationDate="2020-10-17T19:41:07.370" Score="2" Body="&lt;p&gt;First off - &lt;strong&gt;do not use FPKM for across-sample normalization&lt;/strong&gt;. This is such a common issue that I have a prepared text for it:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Why not to use FPKM&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;They have a poor sensitivity when looking at low expressed genes [1, 2]. Short transcripts/genes may have falsely over-estimated expression values [5].&#xA;They may not normalise adequately across samples [3]. The reason for this is: FPKM are normalised across libraries using a single variable, namely the “total number of reads”. This library size factor estimation can easily be biased by highly expressed, highly variable genes, unlike the method used by the other approaches, which model the library size factor based on summary statistics extracted from all genes. In other words as stated in [4]: “The median ratio method is usually quite robust. The biggest problem for library size correction is not actually the genes with low counts, but the genes with very high counts, which have high variance and disproportionately influence estimators like the total sum. This is why the total sum is a notoriously bad estimator for library size.” Or the same by Lior’s Pachter himself [6].&#xA;They have a high False Positive Rate in differential expression analysis [1, 2]. A lot of the genes/transcripts deemed to be differentially expressed are not, mostly due to the effects of point 1), i.e. high artefactual variability in the expression and point 2) i.e. inadequate library size estimation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;On the other hand, approaches such as DESeq, DESeq2, vst or voom have adequate between-sample normalisation, have a controlled FDR and can reliably determine differential expression even for lowly expressed genes if there is enough replication or if an adequate log2 fold change cutoff is selected [7]. If you really insist on using cufflinks, consider performing upper quartile normalisation, which corrects for the point 1) issue.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;References&#xA;[1] Dillies M.A., Briefings in Bioinf. 2012&#xA;[2] Soneson &amp;amp; Delorenzi, BMC Bioinf. 2013&#xA;[3] Lior Pachter’s lecture at CSHL (Cold Spring Harbour Laboratory) from minutes 32 on: &lt;a href=&quot;https://www.youtube.com/watch?v=5NiFibnbE8o&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=5NiFibnbE8o&lt;/a&gt;&#xA;[4] From Answer: &amp;quot;Over-correction&amp;quot; in the size-factors of the DESeq2 package:&#xA;[5] &lt;a href=&quot;http://www.biomedcentral.com/1471-2105/14/370&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.biomedcentral.com/1471-2105/14/370&lt;/a&gt;&#xA;[6] &lt;a href=&quot;https://liorpachter.wordpress.com/2014/04/30/estimating-number-of-transcripts-from-rna-seq-measurements-and-why-i-believe-in-paywall/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://liorpachter.wordpress.com/2014/04/30/estimating-number-of-transcripts-from-rna-seq-measurements-and-why-i-believe-in-paywall/&lt;/a&gt;&#xA;[7] &lt;a href=&quot;http://rnajournal.cshlp.org/content/22/6/839&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://rnajournal.cshlp.org/content/22/6/839&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What I recommend, is that you get the raw count data from TCGA, create a count matrix using the DESeq2 package in R, use the function &lt;code&gt;varianceStabilizingTransformation()&lt;/code&gt; from DESeq2 to further process your counts and model the networks from there. Your pipeline already mentions &lt;code&gt;voom&lt;/code&gt;, which is another good alternative, as is &lt;code&gt;limma&lt;/code&gt;. All of these are available on Bioconductor.&lt;/p&gt;&#xA;" OwnerUserId="9693" LastActivityDate="2020-10-17T19:41:07.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14580" PostTypeId="2" ParentId="14571" CreationDate="2020-10-17T19:46:38.447" Score="0" Body="&lt;p&gt;The most likely explanation here is paired versus unpaired.&lt;/p&gt;&#xA;&lt;p&gt;Mann-Whitney is normally associated with unpaired data, but the authors may have performed a paired test. You performed an unpaired test and thus there is a massive discrepancy in results. The authors should have stated the effect of pairing - it that is what they did - but they are under no strict obligations to do so. You however have performed a Wilcoxan rank sign test, albeit the results shouldn't be so different.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-10-17T19:46:38.447" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14581" PostTypeId="1" CreationDate="2020-10-18T00:36:29.723" Score="0" ViewCount="39" Body="&lt;p&gt;I am looking for a command to count the total number of unique proteins in a file. For instance I wrote a command to BLAST XYZ proteins as query against the DLY proteins as a database to determine the number of genes or proteins that are shared between XYZ and DLY:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastp -query ~/user/XYZ.faa -db DLY -evalue 1e-10 -outfmt '6 qseqid' | sort | uniq | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I used the -evalue option to provide me with alignments with an e-value that is 10^-10 or lower.&lt;/p&gt;&#xA;&lt;p&gt;How can I represent my command so that that it can provide me with a proportion of the total proteins in each species (i.e. XYZ and DLY).&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="9708" LastEditorUserId="9708" LastEditDate="2020-10-18T01:48:30.490" LastActivityDate="2020-10-18T01:48:30.490" Title="Please help me with my command!" Tags="|sequence-alignment|fasta|proteins|database|blast|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14582" PostTypeId="1" AcceptedAnswerId="14585" CreationDate="2020-10-18T03:17:54.497" Score="0" ViewCount="522" Body="&lt;p&gt;I'm trying to plot a heat map for a &amp;quot;GSE133399&amp;quot;, I retrieved the GSE using GEOquery package and then assigned to an object and then tried to plot a heatmap, but I was not successful, I used the following code in RStudio:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;install.packages(&amp;quot;BiocManager&amp;quot;)&#xA;BiocManager::install(&amp;quot;GEOquery&amp;quot;)&#xA;library(GEOquery)&#xA;## change my_id to be the dataset that you want.&#xA;my_id &amp;lt;- &amp;quot;GSE133399&amp;quot;&#xA;gse &amp;lt;- getGEO(my_id)&#xA;length(gse)&#xA;gse &amp;lt;- gse[[1]]&#xA;gse&#xA;pData(gse) ## print the sample information&#xA;fData(gse) ## print the gene annotation&#xA;exprs(gse) ## print the expression data&#xA;&#xA;## exprs get the expression levels as a data frame and get the distribution&#xA;summary(exprs(gse))&#xA;exprs(gse) &amp;lt;- log2(exprs(gse))&#xA;boxplot(exprs(gse),outline=FALSE)&#xA;library(dplyr)&#xA;sampleInfo &amp;lt;- pData(gse)&#xA;sampleInfo&#xA;## source_name_ch1 and characteristics_ch1.1 seem to contain factors we might need for the analysis. Let's pick just those columns&#xA;&#xA;sampleInfo &amp;lt;- select(sampleInfo, source_name_ch1,characteristics_ch1.1)&#xA;&#xA;## Optionally, rename to more convenient column names&#xA;sampleInfo &amp;lt;- rename(sampleInfo,group = source_name_ch1, patient=characteristics_ch1.1)&#xA;sampleInfo&#xA;install.packages(&amp;quot;pheatmap&amp;quot;)&#xA;library(pheatmap)&#xA;## argument use=&amp;quot;c&amp;quot; stops an error if there are any missing data points&#xA;&#xA;corMatrix &amp;lt;- cor(exprs(gse),use=&amp;quot;c&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in cor(exprs(gse), use = &amp;quot;c&amp;quot;) : no complete element pairs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also  tried this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;corMatrix &amp;lt;- cor(exprs(gse),use=&amp;quot;c&amp;quot;, use=&amp;quot;complete.obs&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in cor(exprs(gse), use = &amp;quot;c&amp;quot;, use = &amp;quot;complete.obs&amp;quot;) : &#xA;  formal argument &amp;quot;use&amp;quot; matched by multiple actual arguments&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to achieve this plot:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Dk2eV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Dk2eV.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9709" LastActivityDate="2020-10-18T10:18:52.933" Title="Error in cor(exprs(gse), use = &quot;c&quot;) : no complete element pairs" Tags="|r|rna-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14584" PostTypeId="2" ParentId="14571" CreationDate="2020-10-18T07:43:36.243" Score="1" Body="&lt;p&gt;I think you may need to check your pipeline in this regard. I ran the analysis and replicated as closely as I could their M&amp;amp;M (I didn't filter the &amp;quot;fragile probes&amp;quot; since you anyway care only about a specific probe). In any case, I get something relatively close to their result (P=0.004059), which is still one order of magnitude off, but without more detailed M&amp;amp;M, this is probably as close as I can get to their analysis. Here's what I did (if you prefer to read the notebook html, that is &lt;a href=&quot;https://bschiffthaler.s3-eu-west-1.amazonaws.com/Public/sebio_14571.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(Biobase)&#xA;library(GEOquery)&#xA;library(sva) # ComBat&#xA;library(oligo)&#xA;library(googlesheets4)&#xA;library(stringr)&#xA;library(dplyr)&#xA;&#xA;#' Get the GEO dataset mostly for the phenoData annotation&#xA;gset &amp;lt;- getGEO(&amp;quot;GSE47856&amp;quot;, GSEMatrix =TRUE, getGPL=FALSE)&#xA;adf &amp;lt;- Biobase::phenoData(gset[[1]])&#xA;sampleNames(adf)&#xA;&#xA;#' Read RAW cel files from the GEO data supplementary rar archive&#xA;#' https://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE47856&amp;amp;format=file&#xA;#' In my case extracted to ~/Downloads/cel&#xA;#' Ensure that we read them in the same order as the phenoData object&#xA;celfiles &amp;lt;- sapply(sampleNames(adf), function(f) {&#xA;  pat &amp;lt;- paste0(f, &amp;quot;.*.CEL.gz$&amp;quot;)&#xA;  dir(&amp;quot;~/Downloads/cel&amp;quot;, pattern = pat, full.names = TRUE)&#xA;})&#xA;raw_data &amp;lt;- oligo::read.celfiles(celfiles,&#xA;                                 phenoData = adf, &#xA;                                 experimentData = experimentData(gset[[1]]))&#xA;&#xA;#' Get the raw expression and RMA on transcript clusters&#xA;rma_expr &amp;lt;- oligo::rma(raw_data, target = &amp;quot;core&amp;quot;)&#xA;rma_expr_assay &amp;lt;- assayData(rma_expr)$exprs&#xA;&#xA;#' Use ComBat for batch effect removal&#xA;batch_data &amp;lt;- phenoData(rma_expr)[[&amp;quot;batch:ch1&amp;quot;]]&#xA;cb_expr &amp;lt;- sva::ComBat(rma_expr_assay, batch_data)&#xA;&#xA;#' Get supplementary table 1 from the paper (which I copied on this Google Sheet)&#xA;supp_data &amp;lt;- read_sheet(&amp;quot;https://docs.google.com/spreadsheets/d/142__1O35zvlttejvz62XQX33uLRJhqIDvb604smVSRo&amp;quot;)&#xA;&#xA;#' Match the 'cell line' phenotype from the phenoData to the sample info of&#xA;#' the supplementary table and extract the cell classification. Also remove&#xA;#' all white space and convert everything to lower case for easier matching.&#xA;#' Some cell lines also have a/b attached, so I'm dropping that to match them&#xA;#' up. Mayb not be correct to do that here and treat all a/b the same (?)&#xA;cell_line &amp;lt;- str_replace(&#xA;  str_replace(&#xA;  str_remove(&#xA;    str_to_lower(adf[[&amp;quot;cell line:ch1&amp;quot;]]),&#xA;    &amp;quot; +&amp;quot;),&#xA;  &amp;quot;^tay$&amp;quot;, &amp;quot;taya&amp;quot;),&#xA;  &amp;quot;dov13[ab]&amp;quot;, &amp;quot;dov13&amp;quot;&#xA;)&#xA;&#xA;sample_info &amp;lt;- str_remove(str_to_lower(supp_data&lt;span class=&quot;math-container&quot;&gt;$Sample), &quot; +&quot;)&#xA;classification &amp;lt;- supp_data$&lt;/span&gt;Classification[match(cell_line, sample_info)]&#xA;&#xA;#' Get the treament from the phenoData object&#xA;treatment &amp;lt;- adf[[&amp;quot;treatment:ch1&amp;quot;]]&#xA;&#xA;#' We'll need the geometric mean later (R has no built-in)&#xA;gm_mean = function(x, na.rm=TRUE){&#xA;  exp(sum(log(x[x &amp;gt; 0]), na.rm=na.rm) / length(x))&#xA;}&#xA;&#xA;#' Set up a tibble (==data frame) with all that info for your probe&#xA;tibble(Expression = cb_expr[&amp;quot;8143663&amp;quot;, ], &#xA;       Line = cell_line, &#xA;       Treatment = treatment, &#xA;       Classification = classification) %&amp;gt;%&#xA;  # Summarise replicated data with the geometric mean&#xA;  group_by(Line, Treatment, Classification) %&amp;gt;%&#xA;  summarise(Emean = gm_mean(Expression)) %&amp;gt;% &#xA;  # Make sure data is in order&#xA;  arrange(Line, Treatment, Classification)%&amp;gt;%&#xA;  # Calculate the difference between the cell types&#xA;  group_by(Line, Classification) %&amp;gt;%&#xA;  summarise(EmeanDiff = diff(Emean)) -&amp;gt; xdata&#xA;&#xA;#' Test the difference&#xA;xdata &amp;lt;- split(xdata&lt;span class=&quot;math-container&quot;&gt;$EmeanDiff, xdata$&lt;/span&gt;Classification)&#xA;wilcox.test(x = xdata&lt;span class=&quot;math-container&quot;&gt;$`Epithelial-like`, y = xdata$&lt;/span&gt;`Mesenchymal-like`,&#xA;            exact = TRUE)&#xA;&#xA;#' Session info&#xA;sessionInfo()&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9693" LastActivityDate="2020-10-18T07:43:36.243" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14585" PostTypeId="2" ParentId="14582" CreationDate="2020-10-18T09:10:07.177" Score="2" Body="&lt;p&gt;The GSE133399 dataset contains both ATAC-seq and RNA-seq data, none of which can be queried with GEOquery which works only for microarray data.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;exprs(gse) ## print the expression data&lt;/code&gt; returns an empty object as you might have noticed, therefore the errors in the lines that follow this command.&lt;/p&gt;&#xA;&lt;p&gt;On the bottom of &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE133399&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE133399&lt;/a&gt; at the &lt;code&gt;Supplementary file&lt;/code&gt; section you can download FPKM-transformed count data and use these. I would be careful though as FPKM is not ideal for inter-sample comparison. I personally always (if feasable) download the raw data and then process myself, but for a guess on what is going on the FPKMs probably will do.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2020-10-18T10:18:52.933" LastActivityDate="2020-10-18T10:18:52.933" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14587" PostTypeId="1" CreationDate="2020-10-18T11:43:18.550" Score="0" ViewCount="205" Body="&lt;p&gt;I have Fasta file contain multiple sequences, some of those sequences contain N's or n's, I'm trying to discard all the sequences and related header that contain Ns or ns in the file.&lt;/p&gt;&#xA;&lt;p&gt;I have tried multiple python code but return an error.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import sys&#xA;from Bio import SeqIO&#xA;handle = open(sys.argv[1], &amp;quot;rU&amp;quot;)&#xA;filtered = [record for record in SeqIO.parse(handle, &amp;quot;fasta&amp;quot;) if record.seq.count('N') == 0]&#xA;output_handle = open(&amp;quot;N_removed.fasta&amp;quot;, &amp;quot;w&amp;quot;)&#xA;SeqIO.write(filtered, output_handle, &amp;quot;fasta&amp;quot;)&#xA;output_handle.close()&#xA;handle.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;NsScript.py:3: DeprecationWarning: 'U' mode is deprecated&#xA;  handle = open(sys.argv[1], &amp;quot;rU&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import sys&#xA;from Bio import SeqIO&#xA;for record in SeqIO.parse(sys.argv[1], &amp;quot;fasta&amp;quot;):&#xA;    if record.seq.count('N') == 0:&#xA;        print(record.format(&amp;quot;fasta&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/RemoveNs.py&amp;quot;, line 6&#xA;    &#xA;                                        ^&#xA;SyntaxError: unexpected EOF while parsing&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3332" LastEditorUserId="3332" LastEditDate="2020-10-20T09:14:56.803" LastActivityDate="2020-10-20T20:44:36.117" Title="Discard sequences contain Ns" Tags="|filtering|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14588" PostTypeId="2" ParentId="14491" CreationDate="2020-10-19T01:55:51.063" Score="1" Body="&lt;p&gt;Here's a solution that I came up with after help from both haci and Jesse. It involves using the following modules: os, glob, SeqIO, Seq, SeqUtils.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import os, glob&#xA;&#xA;os.chdir(&amp;quot;C:\python38\Lib\site-packages\Bio\SeqIO&amp;quot;)&#xA;&#xA;from Bio import SeqIO&#xA;&#xA;from Bio.Seq import Seq&#xA;&#xA;from Bio import SeqUtils&#xA;&#xA;patttern = Seq(&amp;quot;ACACACACACACACACACAC&amp;quot;)&#xA;&#xA;folder_path = 'C:\python38\Lib\site-packages\Bio\SeqIO' #specified the folder where my sequences were stored&#xA;&#xA;fasta_paths = glob.glob(os.path.join(folder_path, '*.fasta')) #glob module finds all the pathnames matching a specified pattern i.e. '*.fasta'&#xA;&#xA;with open(&amp;quot;motif4.txt&amp;quot;, &amp;quot;w&amp;quot;) as f:&#xA;... for fasta_path in fasta_paths:&#xA;...     print(fasta_paths)&#xA;...     for seq_record in SeqIO.parse(fasta_path, &amp;quot;fasta&amp;quot;):&#xA;...         results = SeqUtils.nt_search(str(seq_record.seq),pattern)&#xA;...         f.write(str(results)) #replace with seq_record.seq to save the list of node names of all sequences that were parsed&#xA;...         f.write(&amp;quot;\n&amp;quot;) #ensures results of the parse through all the sequences in the fasta_path is written to a new line&#xA;... f.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The nt_search output gave the following output (similar to what Jesse showed):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1 ['ACACACACACACACACACAC']&#xA;2 ['ACACACACACACACACACAC']&#xA;3 ['ACACACACACACACACACAC']&#xA;4 ['ACACACACACACACACACAC']&#xA;5 ['ACACACACACACACACACAC']&#xA;6 ['ACACACACACACACACACAC', 16636, 16638, 16640, 16642, 16644, 16646, 16648, 16650, 16652, 16654, 16656, 16658, 16660, 16662, 16664]&#xA;7 ['ACACACACACACACACACAC']&#xA;8 ['ACACACACACACACACACAC']&#xA;9 ['ACACACACACACACACACAC']&#xA;10 ['ACACACACACACACACACAC']&#xA;11 ['ACACACACACACACACACAC', 9174]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It would miss sequences that did not match (AC)10 or those with imperfect repeat motifs. However, in cases where the motif was &amp;gt; (AC)10, nt_search listed down all locations where the motif was found sequentially. For example, in line 6, all locations were listed and showed the motif was in fact 36 nucleotides long in total (pos #16,636 to #16,673).&lt;/p&gt;&#xA;&lt;p&gt;You can then import the txt. files into a databasing app and analyse the results.&lt;/p&gt;&#xA;" OwnerUserId="9641" LastActivityDate="2020-10-19T01:55:51.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14589" PostTypeId="1" CreationDate="2020-10-19T08:32:37.727" Score="1" ViewCount="846" Body="&lt;p&gt;I have a bam file from paired-end sequencing, in which each R1 has the &lt;code&gt;RX&lt;/code&gt; tag assigned (via &lt;code&gt;UMItools group&lt;/code&gt;) denoting a Unique Molecular Identifier (UMI).&lt;/p&gt;&#xA;&lt;p&gt;I want to call molecular consensus reads using &lt;code&gt;fgbio&lt;/code&gt; but &lt;code&gt;fgbio GroupReadsByUmi&lt;/code&gt; gives me an error, because it expects both R1 and R2 to have the &lt;code&gt;RX&lt;/code&gt; tag set.&lt;/p&gt;&#xA;&lt;p&gt;I would like a tool that will go through my bam file and assign each R2 the same &lt;code&gt;RX&lt;/code&gt; tag as the R1.&lt;/p&gt;&#xA;" OwnerUserId="2155" LastActivityDate="2020-10-19T20:30:32.413" Title="Add tags to the read pairs in a bam file" Tags="|sam|umi|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14590" PostTypeId="2" ParentId="14589" CreationDate="2020-10-19T20:16:46.993" Score="1" Body="&lt;p&gt;The best thing would be to figure out how to do this in UMItools. I don't know that tool at all, but it seems that &lt;a href=&quot;https://github.com/CGATOxford/UMI-tools/issues/347&quot; rel=&quot;nofollow noreferrer&quot;&gt;it is supposed to be able to deal with / tag both reads in paired data&lt;/a&gt;. e.g. did you use &lt;code&gt;--paired&lt;/code&gt; flag, etc.&lt;/p&gt;&#xA;&lt;p&gt;I would suggest investigating that route first before you look at the next part of my answer.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Per Devon Ryan's comment suggestion I would suggest a workflow similar to this:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;sort bam file by query name (&lt;code&gt;samtools sort -n&lt;/code&gt;) to get the R1s immediately following the R2s&lt;/li&gt;&#xA;&lt;li&gt;Write a short script/command that&#xA;&lt;ol&gt;&#xA;&lt;li&gt;reads through the bam file and for each line&lt;/li&gt;&#xA;&lt;li&gt;looks up the tag in the R1, prints R1 line&lt;/li&gt;&#xA;&lt;li&gt;reads the R2, substitutes the RX tag in, prints R2 line&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;pipe that through &lt;code&gt;samtools view&lt;/code&gt; and possibly &lt;code&gt;samtools sort&lt;/code&gt; back to a bam file of the same type that you started with.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;My pysam is a little rusty but something like this might work&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pysam&#xA;&#xA;bam = pysam.AlignmentFile(&amp;quot;bam&amp;quot;, &amp;quot;rb&amp;quot;)&#xA;# print header&#xA;print(bam.header)&#xA;&#xA;tag = None&#xA;for read in bam:&#xA;    r2 = False&#xA;    if tag:&#xA;        read.assign_umi_tag() # assign to read- not sure where/how&#xA;        r2 = True&#xA;    else:&#xA;        tag = read.lookup_umi_tag()  # i don't know where this tag is in bam format&#xA;    print(read)&#xA;    if r2:&#xA;        tag = None&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I haven't tested that in any way, caveat emptor.&lt;/p&gt;&#xA;&lt;p&gt;I am not personally familiar with UMItools and how it assigns tags so can't suggest anything more specific, sorry.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2020-10-19T20:30:32.413" LastActivityDate="2020-10-19T20:30:32.413" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14591" PostTypeId="1" CreationDate="2020-10-20T02:34:29.353" Score="1" ViewCount="32" Body="&lt;p&gt;I'm doing a project for DNA archival storage systems&lt;/p&gt;&#xA;&lt;p&gt;Is there an open-source dataset used as ground truth for clustering?&lt;/p&gt;&#xA;&lt;p&gt;E.g&lt;/p&gt;&#xA;&lt;p&gt;A file containing the ground truth strand sequences, which have been synthesized:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Dataset 1&#xA;st1: ATGCGTGC&#xA;st2: GCTGATTC&#xA;st3: AATGCGTT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And another file containing the sequenced strands, where for each original strand we read multiple 'erroneous' strands:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Dataset 2&#xA;st1 cluster:&#xA;ATTCTTGC&#xA;TTGCGTGC&#xA;ATGCGTGT&#xA;&#xA;st2 cluster:&#xA;GCTGATTC&#xA;GATGAATC&#xA;GCAGATTA&#xA;&#xA;st3 cluster:&#xA;AATGCGTT&#xA;ATTGCTTT&#xA;AATATGTT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Where the clusters can then be used for a consensus alignment algorithm.&lt;/p&gt;&#xA;&lt;p&gt;What I'm aiming to do is to use an algorithm to learn a mapping from dataset1 to dataset2, so that for an arbitrary strand (stx) I can generate a corresponding 'erroneous' cluster (stx cluster)&lt;/p&gt;&#xA;&lt;p&gt;The two datasets should come from a real synthesis and sequencing pipeline.&lt;/p&gt;&#xA;" OwnerUserId="9717" LastActivityDate="2020-10-20T02:34:29.353" Title="Fastq dataset after clustering" Tags="|sequence-alignment|ngs|public-databases|public-dataset|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14593" PostTypeId="2" ParentId="14587" CreationDate="2020-10-20T09:48:17.173" Score="1" Body="&lt;p&gt;With Visual Studio Code you can delete them using a regular expression. Replace &lt;code&gt;^&amp;gt;.*\n[^&amp;gt;]*N[^&amp;gt;]*&lt;/code&gt; with an empty string&#xA;You can also do that in a similar way using the sed Linux command. You can use regular expressions in Python too.&lt;/p&gt;&#xA;" OwnerUserId="5190" LastEditorUserId="5190" LastEditDate="2020-10-20T13:17:07.133" LastActivityDate="2020-10-20T13:17:07.133" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14594" PostTypeId="2" ParentId="14587" CreationDate="2020-10-20T13:02:37.253" Score="2" Body="&lt;p&gt;The 1st code snippet is not an error, it's a deprecation warning. You can remove that warning by dropping the &lt;code&gt;&amp;quot;U&amp;quot;&lt;/code&gt; and just specifying &lt;code&gt;&amp;quot;r&amp;quot;&lt;/code&gt; as the mode. (The &lt;code&gt;&amp;quot;U&amp;quot;&lt;/code&gt; is for universal newlines, which is now the default behaviour since python 3, and should be specified by a separate &lt;code&gt;newline&lt;/code&gt; arg anyway).&lt;/p&gt;&#xA;&lt;p&gt;The 2nd code snippet is a trivial error because you're missing a closing parantheses &lt;code&gt;)&lt;/code&gt; on the final line.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2020-10-20T13:02:37.253" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14596" PostTypeId="1" CreationDate="2020-10-20T15:29:17.017" Score="1" ViewCount="88" Body="&lt;p&gt;I have a gbk files for a viral genome that I have used biopython to add features of the &lt;code&gt;3'&lt;/code&gt; and &lt;code&gt;5'&lt;/code&gt; &lt;code&gt;UTRs&lt;/code&gt;. However when looking in a genome viewer (IGV) I don't see these features. I am not unsure if I encoded the UTR features into the gbk file correctly.&lt;/p&gt;&#xA;&lt;p&gt;They look like this for example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;5'UTR           39..104&#xA;3'UTR           961..2956&#xA;5'UTR           1098..1126&#xA;3'UTR           2781..2956&#xA;5'UTR           3138..3193&#xA;3'UTR           6594..7013&#xA;5'UTR           15611..15770&#xA;3'UTR           16996..17054&#xA;5'UTR           complement(17856..17960)&#xA;3'UTR           complement(17164..17243)&#xA;5'UTR           complement(19557..21099)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Whole gbk file &lt;a href=&quot;http://pastebin.com/8jm2LEk2&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I encoded them in the following way:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   seqID = some-gene_name&#xA;   Start = SeqFeature.ExactPosition(some_number)&#xA;   End = SeqFeature.ExactPosition(some_number)&#xA;   seqStrand = 1 #or -1&#xA;   FeatureLoc = SeqFeature.FeatureLocation(Start, End)&#xA;   Feature = SeqFeature.SeqFeature(FeatureLoc, type = &amp;quot;5'UTR&amp;quot;, strand = seqStrand, id =seqID)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have also tried converting the gbk into a gff file and the line concerning the UTR's look like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GQ994935.1  feature 5'UTR   39  104 .   +   .   &#xA;GQ994935.1  feature 3'UTR   961 2956    .   +   .   &#xA;GQ994935.1  feature 5'UTR   1098    1126    .   +   .   &#xA;GQ994935.1  feature 3'UTR   2781    2956    .   +   .   &#xA;GQ994935.1  feature 5'UTR   3138    3193    .   +   .   &#xA;GQ994935.1  feature 3'UTR   6594    7013    .   +   .   &#xA;GQ994935.1  feature 5'UTR   15611   15770   .   +   .   &#xA;GQ994935.1  feature 3'UTR   16996   17054   .   +   .   &#xA;GQ994935.1  feature 5'UTR   17856   17960   .   -   .   &#xA;GQ994935.1  feature 3'UTR   17164   17243   .   -   .   &#xA;GQ994935.1  feature 5'UTR   19557   21099   .   -   .   &#xA;GQ994935.1  feature 3'UTR   18576   18590   .   -   .   &#xA;GQ994935.1  feature 5'UTR   21051   21099   .   -   .   &#xA;GQ994935.1  feature 3'UTR   18576   20039   .   -   .   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is what I see in IGV:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/0M2Sv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0M2Sv.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;THis is just the regular CDS features with no UTRs&lt;/p&gt;&#xA;&lt;p&gt;Am I doing this incorrectly?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; from comments&lt;/p&gt;&#xA;&lt;p&gt;The gff3 file has been inspected (presented above), however it remains uncertain how the UTR regions should be represented. The intention is for the UTR regions represented as a features the same as a CDS.&lt;/p&gt;&#xA;&lt;p&gt;The information copied and pasted is an excerpt from the gbk file, with supporting information from GFF files as requested (below). The formatted described is the Biopython output.&lt;/p&gt;&#xA;&lt;p&gt;Further information is here: &lt;a href=&quot;http://pastebin.com/8jm2LEk2&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://pastebin.com/8jm2LEk2&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7031" LastEditorUserId="3848" LastEditDate="2022-03-15T01:23:16.080" LastActivityDate="2022-03-15T01:23:16.080" Title="UTR features in gbk files" Tags="|python|sequence-annotation|covid-19|genbank|virus|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="14597" PostTypeId="2" ParentId="7195" CreationDate="2020-10-20T20:14:45.613" Score="1" Body="&lt;p&gt;I work a lot with large networks and I agree with what most other posters have said, in that it is hardly useful to explore a network like this visually and Cytoscape - while it is one of best GUIs in general - is pretty bad with large graphs. Here are some alternatives you could try &lt;em&gt;if you want to go down that road&lt;/em&gt;:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://gephi.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gephi&lt;/a&gt; also probably won't handle that number of nodes/edges well, but it may not crash. Development on it has stagnated though.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://graphia.app/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Graphia&lt;/a&gt; I haven't tried this, but it is supposed to be better at handling large graphs. This project is in its infancy, so don't expect too much&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If you care about the analysis of the graph itself, I'd go with &lt;a href=&quot;https://networkit.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NetworKit&lt;/a&gt; or &lt;a href=&quot;https://snap.stanford.edu&quot; rel=&quot;nofollow noreferrer&quot;&gt;SNAP&lt;/a&gt; which are much better suited to large graphs.&lt;/p&gt;&#xA;&lt;p&gt;If you want to visualize the networks, something that tends to perform well here is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Graph embeddings with &lt;a href=&quot;https://github.com/xgfs/verse&quot; rel=&quot;nofollow noreferrer&quot;&gt;VERSE&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Dimensionality reduction of the embeddings with &lt;a href=&quot;https://umap-learn.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;UMAP&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Plot with &lt;a href=&quot;https://datashader.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Datashader&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This can handle absolutely massive graphs if you have the compute - which you seem to have. Here's one I am just working on (albeit small compared to yours with 30K nodes and 800K edges) - from that exact pipeline. Took ~10 minutes read to render on my desktop:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/fITVg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fITVg.png&quot; alt=&quot;graph example&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Finally, as others have mentioned, graph partitioning might be the way to go. I really like &lt;a href=&quot;https://www.mapequation.org/code.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;InfoMap&lt;/a&gt;. It partitions the graph into a hierarchical structure that you can explore top down.&lt;/p&gt;&#xA;" OwnerUserId="9693" LastActivityDate="2020-10-20T20:14:45.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14599" PostTypeId="2" ParentId="14587" CreationDate="2020-10-20T20:44:36.117" Score="2" Body="&lt;p&gt;If you're not married to Python, here's a one liner in &lt;a href=&quot;https://github.com/lh3/bioawk&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioawk&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bioawk -c fastx '{ if($seq !~ /[Nn]/) {print &amp;quot;&amp;gt;&amp;quot;$name; print $seq}}' old_file.fa &amp;gt; new_file.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9693" LastActivityDate="2020-10-20T20:44:36.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14600" PostTypeId="1" CreationDate="2020-10-21T09:04:44.930" Score="1" ViewCount="989" Body="&lt;p&gt;Can I eject the flow cell MinION from USB port when doing basecalling? Because its taking long and its not possible to wait for its finished, will it affects basecalling?&lt;/p&gt;&#xA;" OwnerUserId="9724" LastActivityDate="2023-01-20T13:13:02.990" Title="Can I eject the flow cell MinION from USB port when doing basecalling?" Tags="|fastq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14601" PostTypeId="1" CreationDate="2020-10-21T09:12:21.037" Score="2" ViewCount="1004" Body="&lt;p&gt;I would like to integrate the phylogenetic tree into a website. Does anyone know where I can download the complete tree? Best would be a JSON, which looks something like this:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-json prettyprint-override&quot;&gt;&lt;code&gt;[{&#xA;  &amp;quot;level&amp;quot;: &amp;quot;domain&amp;quot;,&#xA;  &amp;quot;name&amp;quot;: &amp;quot;eukaryota&amp;quot;,&#xA;  &amp;quot;commonNames&amp;quot;: {&#xA;    &amp;quot;en&amp;quot;: [&amp;quot;eukaryotes&amp;quot;],&#xA;    &amp;quot;de&amp;quot;: [&amp;quot;Eukaryoten&amp;quot;, &amp;quot;Eukaryonten&amp;quot;]&#xA;  },&#xA;  &amp;quot;children&amp;quot;: [&#xA;    {&#xA;      &amp;quot;level&amp;quot;: &amp;quot;kingdom&amp;quot;,&#xA;      &amp;quot;name&amp;quot;: &amp;quot;plantae&amp;quot;,&#xA;      &amp;quot;commonNames&amp;quot;: {&#xA;        &amp;quot;en&amp;quot;: [&amp;quot;plants&amp;quot;],&#xA;        &amp;quot;de&amp;quot;: [&amp;quot;Pflanzen&amp;quot;]&#xA;      },&#xA;      &amp;quot;children&amp;quot;: [...]&#xA;    },&#xA;    {&#xA;      &amp;quot;level&amp;quot;: &amp;quot;kingdom&amp;quot;,&#xA;      &amp;quot;name&amp;quot;: &amp;quot;animalia&amp;quot;,&#xA;      &amp;quot;commonNames&amp;quot;: {&#xA;        &amp;quot;en&amp;quot;: [&amp;quot;animals&amp;quot;],&#xA;        &amp;quot;de&amp;quot;: [&amp;quot;Tiere&amp;quot;]&#xA;      },&#xA;      &amp;quot;children&amp;quot;: [...]&#xA;    },&#xA;    ...&#xA;  ]&#xA;},&#xA;{&#xA;  &amp;quot;level&amp;quot;: &amp;quot;domain&amp;quot;,&#xA;  &amp;quot;name&amp;quot;: &amp;quot;bacteria&amp;quot;,&#xA;  &amp;quot;commonNames&amp;quot;: {&#xA;    &amp;quot;en&amp;quot;: [&amp;quot;bacteria&amp;quot;],&#xA;    &amp;quot;de&amp;quot;: [&amp;quot;Bakterien&amp;quot;]&#xA;  },&#xA;  &amp;quot;children&amp;quot;: [...]&#xA;}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Other formats and structures are ok, too. The important thing is that the level of the nodes can be retrieved. In the end I only want to have the main levels Domain, Kingdom, Phylum, Class, Order, Family, Genus and Species in the tree. (&lt;a href=&quot;https://en.wikipedia.org/wiki/Taxonomy_(biology)#/media/File:Biological_classification_L_Pengo_vflip.svg&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Taxonomy_(biology)#/media/File:Biological_classification_L_Pengo_vflip.svg&lt;/a&gt;) So if there are more nodes from other levels like Clade, Tribe or Subfamily, it must be possible to identify and remove them.&#xA;Thanks a lot!&lt;/p&gt;&#xA;" OwnerUserId="9725" LastActivityDate="2020-10-21T11:59:01.017" Title="Download complete Phylogenetic Tree as JSON" Tags="|phylogenetics|phylogeny|taxonomy|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14602" PostTypeId="2" ParentId="14601" CreationDate="2020-10-21T09:28:13.947" Score="4" Body="&lt;p&gt;NCBI maintains an SQL dump of their &lt;a href=&quot;https://ncbiinsights.ncbi.nlm.nih.gov/2018/02/22/new-taxonomy-files-available-with-lineage-type-and-host-information/&quot; rel=&quot;nofollow noreferrer&quot;&gt;taxonomic database&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can see what that contains &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/pub/taxonomy/new_taxdump/taxdump_readme.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For smaller requests (depending on what you need), you might also consider the ENSEMBL REST API - although I don't know of a way to retrieve the complete tree (shoot them an email): &lt;a href=&quot;https://rest.ensembl.org/taxonomy/classification/53399?content-type=application/json&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rest.ensembl.org/taxonomy/classification/53399?content-type=application/json&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9693" LastActivityDate="2020-10-21T09:28:13.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14603" PostTypeId="2" ParentId="14601" CreationDate="2020-10-21T11:59:01.017" Score="1" Body="&lt;p&gt;It would be worth looking at the &lt;a href=&quot;http://tolweb.org/tree/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tree of Life&lt;/a&gt; web project, it isn't downloadable but its a reference source that can be improved on and updated in your website.&lt;/p&gt;&#xA;&lt;p&gt;ToL contains trees linking everything from bacteria, archaea through to the complete diversity of eukaryotes, but its a bit old with 2008 from my taxonomic knowledge being the last update for some of the entries, e.g. Plasmodium spp..&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-10-21T11:59:01.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14604" PostTypeId="2" ParentId="14600" CreationDate="2020-10-21T15:17:07.173" Score="1" Body="&lt;p&gt;You can, you will lose the transfer from the MinION. It would probably be better to just stop the run. The computer will continue to basecall untill it is done. All the raw reads are stored locally on the machine so it should be fine.&lt;/p&gt;&#xA;" OwnerUserId="9726" LastActivityDate="2020-10-21T15:17:07.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14605" PostTypeId="2" ParentId="14573" CreationDate="2020-10-21T15:49:36.963" Score="0" Body="&lt;p&gt;The coefficient is the third column from the right, you can tell because some are positive and other negative. When the value is negative the association is a negative correlation. When it is positive (last two) that is more like the result you are seeking. The further away from zero the stronger the correlation (positive or negative)&lt;/p&gt;&#xA;&lt;p&gt;These are very large values for the coefficients. The SNP is the last column on the right. I would disgard the top two results because the probabilities are too borderline, e.g. if you used Bonferoni's correction.&lt;/p&gt;&#xA;&lt;p&gt;There isn't enough information on this calculation nor the data set to make any further insights other than you have 7 SNPs showing PRS&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-10-21T15:49:36.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14606" PostTypeId="2" ParentId="14520" CreationDate="2020-10-21T18:40:07.383" Score="4" Body="&lt;p&gt;&lt;strong&gt;Update 2:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I looked into this a little more, with the various data sources.&lt;/p&gt;&#xA;&lt;p&gt;This is related in part to the answer submitted by OP juanjo75es, in addition to discussion on chat. I don't entirely understand the logic, but the general thrust seems to be that SPAdes makes weird assemblies somehow.&lt;/p&gt;&#xA;&lt;p&gt;Some notes that I made:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;REFERENCE ASSEMBLIES&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;FIV sequence U11820.1 was deposited in 1996, before SPAdes existed. Unclear assembly method.&lt;/li&gt;&#xA;&lt;li&gt;FIV sequence MN630242 was deposited in 2020, but was assembled with CLC workbench.&lt;/li&gt;&#xA;&lt;li&gt;These two sequences align together ok with minimap2. Low exact identity, it's true. This is the PAF output:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;MN630242.1      8977    194     8964    +       U11820.1        9462    575     9326    1130    8778    60      tp:A:P  cm:i:116        s1:i:1125&#xA;       s2:i:55 dv:f:0.1761&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;These two assemblies are syntenic, and both have similar gene calls using prokka. GFFs:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;MN630242.1      Prodigal:2.6    CDS     253     1605    .       +       0       ID=AMGANPBD_00001;inference=ab initio prediction:Prodigal:2.6;locus_tag=AMGANPBD_00001;product=hypothetical protein&#xA;MN630242.1      Prodigal:2.6    CDS     1656    4868    .       +       0       ID=AMGANPBD_00002;eC_number=3.6.1.23;Name=dut;gene=dut;inference=ab initio prediction:Prodigal:2.6,similar to AA sequence:UniProtKB:Q2YRG4;locus_tag=AMGANPBD_00002;product=Deoxyuridine 5'-triphosphate nucleotidohydrolase&#xA;MN630242.1      Prodigal:2.6    CDS     4861    5616    .       +       0       ID=AMGANPBD_00003;inference=ab initio prediction:Prodigal:2.6;locus_tag=AMGANPBD_00003;product=hypothetical protein&#xA;MN630242.1      Prodigal:2.6    CDS     5891    8461    .       +       0       ID=AMGANPBD_00004;inference=ab initio prediction:Prodigal:2.6;locus_tag=AMGANPBD_00004;product=hypothetical protein&#xA;MN630242.1      Prodigal:2.6    CDS     8626    8790    .       +       0       ID=AMGANPBD_00005;inference=ab initio prediction:Prodigal:2.6;locus_tag=AMGANPBD_00005;product=hypothetical protein&#xA;&#xA;&#xA;U11820.1        Prodigal:2.6    CDS     634     1983    .       +       0       ID=EANIPDKN_00001;inference=ab initio prediction:Prodigal:2.6;locus_tag=EANIPDKN_00001;product=hypothetical protein&#xA;U11820.1        Prodigal:2.6    CDS     1995    5246    .       +       0       ID=EANIPDKN_00002;eC_number=3.6.1.23;Name=dut;gene=dut;inference=ab initio prediction:Prodigal:2.6,similar to AA sequence:UniProtKB:Q2YRG4;locus_tag=EANIPDKN_00002;product=Deoxyuridine 5'-triphosphate nucleotidohydrolase&#xA;U11820.1        Prodigal:2.6    CDS     5239    5994    .       +       0       ID=EANIPDKN_00003;inference=ab initio prediction:Prodigal:2.6;locus_tag=EANIPDKN_00003;product=hypothetical protein&#xA;U11820.1        Prodigal:2.6    CDS     6269    8830    .       +       0       ID=EANIPDKN_00004;inference=ab initio prediction:Prodigal:2.6;locus_tag=EANIPDKN_00004;product=hypothetical protein&#xA;U11820.1        Prodigal:2.6    CDS     8904    9152    .       +       0       ID=EANIPDKN_00005;inference=ab initio prediction:Prodigal:2.6;locus_tag=EANIPDKN_00005;product=hypothetical protein&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I can share FAA files of the proteins if needed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;READ ASSEMBLY&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I also assembled the indicated reads using SPAdes. For reference it is a &lt;strong&gt;~9Kbp&lt;/strong&gt; virus, but this is a &lt;strong&gt;4.3Mbp&lt;/strong&gt; assembly. There is a lot of non-virus sequence in there. The second-largest contig is a shuttle vector. the third-largest contig is cat (host). Many more are cat, so I think it's pretty cat-oriented. The original authors used CLC workbench, so I guess that just worked a lot better in this instance, even in the presence of all the contamination. Unclear why, it appears that CLC works similarly to OP's assembly tool.&lt;/p&gt;&#xA;&lt;p&gt;I aligned these to MN630242.1 with minimap2. If I understand OP, they are unhappy about the mapping of these contigs to this genome reference. The identities are mostly high (though there is indeed some oddness with the largest contig, which only finds very low coverage- at the same time that there are overlapping contigs with very high ID?), and they cover the entire reference genome:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MN630242.1  8977    2155    3199    -   NODE_14_length_1054_cov_3786.620280 1054    2   1046    1021    1044    60  tp:A:P  cm:i:189    s1:i:1021   s2:i:378    dv:f:0.0031&#xA;MN630242.1  8977    7893    8626    -   NODE_25_length_743_cov_3814.258117  743 8   739 713 733 60  tp:A:P  cm:i:129    s1:i:713    s2:i:318    dv:f:0.0045&#xA;MN630242.1  8977    72  4904    -   NODE_1_length_4942_cov_25.814123    4942    34  4863    628 4832    41  tp:A:P  cm:i:7s1:i:628  s2:i:512    dv:f:0.1703&#xA;MN630242.1  8977    3376    3897    +   NODE_39_length_526_cov_5254.155388  526 3   524 512 521 0   tp:A:S  cm:i:8s1:i:512  dv:f:0.0023&#xA;MN630242.1  8977    6270    6673    +   NODE_82_length_409_cov_1.744681 409 3   406 357 403 0   tp:A:P  cm:i:53 s1:i:357    s2:i:351    dv:f:0.0185&#xA;MN630242.1  8977    6852    7223    -   NODE_87_length_381_cov_23.334646    381 6   376 355 371 2   tp:A:P  cm:i:5s1:i:355  s2:i:351    dv:f:0.0079&#xA;MN630242.1  8977    6187    6558    -   NODE_88_length_380_cov_2338.675889  380 7   378 351 371 0   tp:A:S  cm:i:5s1:i:351  dv:f:0.0085&#xA;MN630242.1  8977    6852    7308    +   NODE_56_length_471_cov_0.933140 471 15  471 351 456 0   tp:A:S  cm:i:43 s1:i:351    dv:f:0.0397&#xA;MN630242.1  8977    7332    7707    +   NODE_86_length_389_cov_3506.202290  389 5   380 334 375 17  tp:A:P  cm:i:5s1:i:334  s2:i:309    dv:f:0.0111&#xA;MN630242.1  8977    5181    5568    -   NODE_60_length_464_cov_1.005935 464 50  437 314 387 17  tp:A:P  cm:i:41 s1:i:314    s2:i:282    dv:f:0.0357&#xA;MN630242.1  8977    7277    7707    +   NODE_76_length_438_cov_0.919614 438 5   435 309 430 0   tp:A:S  cm:i:40 s1:i:309    dv:f:0.0437&#xA;MN630242.1  8977    6953    7285    +   NODE_94_length_345_cov_2.903670 345 8   340 301 332 0   tp:A:S  cm:i:50 s1:i:301    dv:f:0.0110&#xA;MN630242.1  8977    5744    6056    -   NODE_113_length_316_cov_2.169312    316 4   316 289 312 35  tp:A:P  cm:i:4s1:i:289  s2:i:244    dv:f:0.0096&#xA;MN630242.1  8977    5615    5927    +   NODE_108_length_322_cov_1103.400000 322 6   318 283 312 58  tp:A:P  cm:i:4s1:i:283  s2:i:209    dv:f:0.0143&#xA;MN630242.1  8977    6543    6813    -   NODE_147_length_279_cov_2592.519737 279 8   278 261 270 21  tp:A:P  cm:i:4s1:i:261  s2:i:236    dv:f:0.0032&#xA;MN630242.1  8977    8544    8872    +   NODE_100_length_333_cov_2259.189320 333 1   329 252 328 0   tp:A:P  cm:i:3s1:i:252  s2:i:245    dv:f:0.0318&#xA;MN630242.1  8977    7601    7853    -   NODE_200_length_255_cov_25.125000   255 0   252 250 252 12  tp:A:P  cm:i:3s1:i:250  s2:i:236    dv:f:0.0017&#xA;MN630242.1  8977    6648    6968    -   NODE_106_length_324_cov_2.538071    324 4   324 248 320 0   tp:A:P  cm:i:3s1:i:248  s2:i:248    dv:f:0.0303&#xA;MN630242.1  8977    5940    6193    -   NODE_162_length_266_cov_1809.287770 266 6   259 246 253 5   tp:A:P  cm:i:4s1:i:246  s2:i:240    dv:f:0.0033&#xA;MN630242.1  8977    5416    5664    -   NODE_264_length_255_cov_9.531250    255 5   253 243 248 0   tp:A:P  cm:i:4s1:i:243  s2:i:242    dv:f:0.0033&#xA;MN630242.1  8977    7221    7466    -   NODE_247_length_255_cov_12.539062   255 1   246 241 245 0   tp:A:P  cm:i:4s1:i:241  s2:i:241    dv:f:0.0032&#xA;MN630242.1  8977    4830    5071    -   NODE_1121_length_245_cov_11.711864  245 4   245 239 241 0   tp:A:P  cm:i:4s1:i:239  s2:i:235    dv:f:0.0042&#xA;MN630242.1  8977    7769    8013    +   NODE_677_length_251_cov_3.951613    251 6   249 238 244 7   tp:A:P  cm:i:4s1:i:238  s2:i:230    dv:f:0.0075&#xA;MN630242.1  8977    4901    5148    +   NODE_270_length_255_cov_8.726562    255 8   255 238 247 0   tp:A:P  cm:i:4s1:i:238  s2:i:234    dv:f:0.0073&#xA;MN630242.1  8977    8705    8948    -   NODE_382_length_254_cov_5.763780    254 7   250 234 243 15  tp:A:P  cm:i:4s1:i:234  s2:i:217    dv:f:0.0062&#xA;MN630242.1  8977    6065    6307    +   NODE_410_length_254_cov_3.488189    254 1   243 227 242 10  tp:A:P  cm:i:3s1:i:227  s2:i:210    dv:f:0.0122&#xA;MN630242.1  8977    5062    5239    -   NODE_7003_length_180_cov_9.207547   180 3   180 172 177 17  tp:A:P  cm:i:2s1:i:172  s2:i:157    dv:f:0.0066&#xA;MN630242.1  8977    11  136 +   NODE_21321_length_128_cov_1195.000000   128 2   127 125 125 3   tp:A:P  cm:i:2s1:i:125  s2:i:123    dv:f:0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Only a (relatively) few contigs align well (figure), as might be expected with heavy contamination, and of those some are overlapping:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oGO01.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oGO01.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;To look into the odd behavior of the biggest contig, I ran prokka on it too and found the expected genes for its position, especially the largest gene in the virus, &lt;code&gt;dut&lt;/code&gt; (Deoxyuridine 5'-triphosphate nucleotidohydrolase). I then took the protein sequences of the 3 assemblies and aligned them with clustalo. They're all quite similar:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CLUSTAL O(1.2.4) multiple sequence alignment&#xA;&#xA;&#xA;NODE1_KJHFFCBH_00001         ------MEKRPEIQIFVNGHPIKFLLDTGADITILNRKEFIIGNSIENGKQNMIGVGGGK&#xA;MN630242_AMGANPBD_00002      -------------MIFVNGYPIKFLLDTGADITILNRRDFQVKNSIENGRQNMIGVGGGK&#xA;U11820.1_EANIPDKN_00002      MGTTTTLERRLEIQIFVNGHPIKFLLDTGADITILNRKDFQIGNSIENGKQNMIGVGGGK&#xA;                                           *****:*****************::* : ******:**********&#xA;&#xA;NODE1_KJHFFCBH_00001         RGTNYINVHLEIRDENYKTQCIFGNVCVLEDNSLIQPLLGRDNMIKFNIRLVMAQISEKI&#xA;MN630242_AMGANPBD_00002      RGTNYTNVHLEIRDENYKTQCIFGNVCVLEDNSLIQPLLGRDNMIKFNIRLVMAQISDKI&#xA;U11820.1_EANIPDKN_00002      RGTNYINVHLEIRDENYRMQCIFGNVCVLEDNSLIQPLLGRDNMIKFNIRLVMAQISEKI&#xA;                             ***** ***********: **************************************:**&#xA;&#xA;NODE1_KJHFFCBH_00001         PIVKVRMKDPTQGPQVKQWPLSNEKIEALTEIVERLEQEGKVKRADPNNPWNTPVFAIKK&#xA;MN630242_AMGANPBD_00002      PIVKVKMKDPNKGPQIKQWPLSNEKIEALTEIVERLEKEGKVKRADPNNPWNTPVFAIKK&#xA;U11820.1_EANIPDKN_00002      PIVKVRMRDPIQGPQVKQWPLSNEKIEALTDIVERLESEGKVKRADPNNPWNTPVFAIKK&#xA;                             *****:*:** :***:**************:******.**********************&#xA;&#xA;NODE1_KJHFFCBH_00001         KSGKWRMLIDFRVLNKLTDKGAEVQLGLPHPAGLQWKKQVTVLDIGDAYFTIPLDPDYAP&#xA;MN630242_AMGANPBD_00002      KSGKWRMLIDFRELNKLTEKGAEVQLGLPHPAGLQMKKQVTVLDIGDAYFTIPLDPDYAP&#xA;U11820.1_EANIPDKN_00002      KSGKWRMLIDFRVLNKLTDKGAEVQLGLPHPAGLQMKKQVTVLDIGDAYFTIPLDPDYAP&#xA;                             ************ *****:**************** ************************&#xA;&#xA;NODE1_KJHFFCBH_00001         YTAFTLPRKNNAGPGKRYVWCSLPQGWVLSPLIYQSTLDNILQPYIKQNPELDIYQYMDD&#xA;MN630242_AMGANPBD_00002      YTAFTLPRKNNAGPGRRYIWCSLPQGWILSPLIYQSTLDNIIQPFIRQNPQLDIYQYMDD&#xA;U11820.1_EANIPDKN_00002      YTAFTLPRKNNAGPGRRYVWCSLPQGWVLSPLIYQSTLDNILQPFIKQNSELDIYQYMDD&#xA;                             ***************:**:********:*************:**:*:** :*********&#xA;&#xA;NODE1_KJHFFCBH_00001         IYIGSNLSRQEHKQKVEELRKLLLWWGFETPEDKLQEEPPYKWMGYELHPLTWSIQQKQL&#xA;MN630242_AMGANPBD_00002      IYIGSNLNKKEHKEKVGELRKLLLWWGFETPEDKLQEEPPYKWMGYELHPLTWTIQQKQL&#xA;U11820.1_EANIPDKN_00002      IYIGSNLSKKEHKQKVEELRKLLLWWGFETPEDKLQEEPPYKWMGYELHPLTWSIQQKQL&#xA;                             *******.::***:** ************************************:******&#xA;&#xA;NODE1_KJHFFCBH_00001         EIPERPTLNDLQKLAGKINWASQTIPKLSIKALTHMMRGDQKLDSIREWTEEAKKEVQKA&#xA;MN630242_AMGANPBD_00002      DIPEQPTLNELQKLAGKINWASQAIPDLSIKALTNMMRGNQNLNSIREWTKEARLEVQKA&#xA;U11820.1_EANIPDKN_00002      EIPERPTLNELQKLAGKINWASQTIPDLSIKELTNMMRGDQKLDSIREWTVEAKREVQKA&#xA;                             :***:****:*************:**.**** **:****:*:*:****** **: *****&#xA;&#xA;NODE1_KJHFFCBH_00001         KEAIEKQAQLYYYDPNRELYAKISLVGPHQLCYQVYHKNPEQILWYGKMNRQKKRAENTC&#xA;MN630242_AMGANPBD_00002      KKAIEGQVQLGYYDPSKELYAKLSLVGPHQISYQVYQRNPERILWYGKMSRQKKKAENTC&#xA;U11820.1_EANIPDKN_00002      KEAIEKQAQLNYYDPNRGLYAKLSLVGPHQICYQVYQKNPEHILWYGKMNRQKKKAENTC&#xA;                             *:*** *.** ****.: ****:*******:.****::***:*******.****:*****&#xA;&#xA;NODE1_KJHFFCBH_00001         DIALRACYKIREESIVRIGKEPVYEIPASREAWESNIIRSPYLKASPPEVEFIHAALNIK&#xA;MN630242_AMGANPBD_00002      DIALRACYKIREESIIRIGKEPKYEIPTSREAWESNLINSPYLKAPPPEVEYIHAALNIK&#xA;U11820.1_EANIPDKN_00002      DIALRACYKIREESIIRIGKEPMYEIPASREAWESNLIRSPYLKAPPPEVEFIHAALSIK&#xA;                             ***************:****** ****:********:*.****** *****:*****.**&#xA;&#xA;NODE1_KJHFFCBH_00001         RALSMVQEVPILGAETWYIDGGRRQGKAARAAYWTNTGRWQVMEIEGSNQKAEVQALLMA&#xA;MN630242_AMGANPBD_00002      RALSMIKDAPILGAETWYIDGGRKLGKAAKAAYWTDTGKWQVMELEGSNQKAEIQALLLA&#xA;U11820.1_EANIPDKN_00002      RALSMIQDAPITGAETWYIDGSRKQGKAARAAYWTDTGKWQIMEIEGSNQKAEVQALLLA&#xA;                             *****:::.** *********.*: ****:*****:**:**:**:********:****:*&#xA;&#xA;NODE1_KJHFFCBH_00001         LKEGPEEMNIITDSQYILNIMNQQPDLMEGIWQEVLEEMEKKIAIFIDWVPGHKGIPGNE&#xA;MN630242_AMGANPBD_00002      LKAGPEEMNIITDSQYVINIILQQPDMMEGIWQEVLEELEKKTAIFIDWVPGHKGIPGNE&#xA;U11820.1_EANIPDKN_00002      LKAGSEEMNIITDSQYILNIINQQPDLMEGLWQEVLEEMEKKIAIFIDWVPGHKGIPGNE&#xA;                             ** * ***********::**: ****:***:*******:*** *****************&#xA;&#xA;NODE1_KJHFFCBH_00001         EVDKLCQTMMIIEGDGILNKRPEDAGYDLLAAQEIHILPGEVRIVPTRTRIMLPKGYWGL&#xA;MN630242_AMGANPBD_00002      EVDKLCQTMMIIEGDGILDKRSEDAGYDLLAAKEMHLLPGEVKVIPTGVKIMLPKGYWGL&#xA;U11820.1_EANIPDKN_00002      EVDKLCQTMMIIEGEGILDKRSEDAGYDLLAAQETHFLPGEVRIVPTKTRIMLPKGHWGL&#xA;                             **************:***:** **********:* *:*****:::** .:******:***&#xA;&#xA;NODE1_KJHFFCBH_00001         IMGKSSIGNKGMDVLGGVIDEGYRGEIGVIMINLSKKSTTILEKQKVAQLIILPCKHESL&#xA;MN630242_AMGANPBD_00002      IIGKSSIGSKGLDVLGGVIDEGYRGEIGVIMINLSRKSITLLEQQKIAQLIILPCKHEVL&#xA;U11820.1_EANIPDKN_00002      IMGKSSIGSKGMDVLGGVIDEGYRGELGVIMINLTKKSITILEKQKVAQLIILPCRHESL&#xA;                             *:******.**:**************:*******::** *:**:**:********:** *&#xA;&#xA;NODE1_KJHFFCBH_00001         EQGEIIMNSERGEKGFGSTGVFSSWVDRIEEAELNHEKFHSDPQYLRTEFNIPRIVAEEI&#xA;MN630242_AMGANPBD_00002      EQGKVIMDSERGDKGYGSTGVFSSWVDRIEEAEINHEKFHSDPQYLRTEFNLPKMVAEEI&#xA;U11820.1_EANIPDKN_00002      QQGEIQMDSERGEKGFGSTGVFSSWVDRIEEAELNHEKFHSDPQYLRTEFNLPRIVAEEI&#xA;                             :**:: *:****:**:*****************:*****************:*::*****&#xA;&#xA;NODE1_KJHFFCBH_00001         KRKCPLCRIRGEQVEGKLKIGPGIWQMDCTHFNGKIIIVAIHVESGLLWAQIIPQETADC&#xA;MN630242_AMGANPBD_00002      RRKCPVCRIRGEQVGGQLKIGPGIWQMDCTHFDGKIILVAIHVESGYIWAQIISQETADC&#xA;U11820.1_EANIPDKN_00002      KRKCPLCRIRGEQVGGQLKIGPGIWQMDCTHFNGKIIIVAVHVESGFLWAQIIPQETAEC&#xA;                             :****:******** *:***************:****:**:***** :***** ****:*&#xA;&#xA;NODE1_KJHFFCBH_00001         TLKAIMQLVSTHNVTEIQTDNGPNFKNQKIEGLLSYMGIKHKLGIPGNPQSQALVENANS&#xA;MN630242_AMGANPBD_00002      TVKAVLQLLSAHNVTELQTDNGPNFKNQKMEGALNYMGVKHKFGIPGNPQSQALVENVNQ&#xA;U11820.1_EANIPDKN_00002      TVKALLQLICAHNVTELQTDNGPNFKNQKMEGLLNYMGIKHKLGIPGNPQSQALVENANN&#xA;                             *:**::**:.:*****:************:** *.***:***:**************.*.&#xA;&#xA;NODE1_KJHFFCBH_00001         TLKVWIQKFLPETTSLDNALALALHCLNFKQRGRLGKMAPYELYTQQESLRIQDYFSQLP&#xA;MN630242_AMGANPBD_00002      TLKAWIQKFLPETTSLENALALAVHCLNFKQRGRIGGMAPYELLAQQESLRIQEFFSKIP&#xA;U11820.1_EANIPDKN_00002      TLKAWIQKFLPETTSLDNALALALHCLNFKQRGRLGKMAPYELYIQQESLRIQDYFSQIP&#xA;                             ***.************:******:**********:* ******  ********::**::*&#xA;&#xA;NODE1_KJHFFCBH_00001         QKLMMQWVYYKDQKDKKWKGPMRVEYWGQGSVLLKDEEKGYFLVPRRHIRRVPEPCTLPE&#xA;MN630242_AMGANPBD_00002      QKLQAQWIYYKDQKDKKWKGPMRVEYWGQGSVLLKDEEKGYFLVPRRHIKRVPEPCALPE&#xA;U11820.1_EANIPDKN_00002      QKLMMQWVYYKDQKDKKWKGPMRVEYWGQGSVLLKDEEKGYFLVPRRHIRRVPEPCTLPE&#xA;                             ***  **:*****************************************:******:***&#xA;&#xA;NODE1_KJHFFCBH_00001         GDE&#xA;MN630242_AMGANPBD_00002      GDE&#xA;U11820.1_EANIPDKN_00002      GDE&#xA;                             ***&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Overall, the level of homology is quite high at the protein level. There's maybe a little evidence that this contig's protein is closer to U11820.1 than to MN630242 (though there are also several positions where U11820.1 is the outgroup). So I'm not sure why pieces of this contig are so hard to align at the DNA level without further investigation, but honestly overall of these are looking like very similar viral sequences, as might be expected.&lt;/p&gt;&#xA;&lt;p&gt;I also aligned the big contig to both U11820.1 and MN630242 at the DNA level in 3-way &lt;code&gt;clustalo&lt;/code&gt; alignment it is ok. Not sure what the difference is there. There is no obvious reason to prefer one reference over the other, according to my eye. When I have &lt;code&gt;clustalo&lt;/code&gt; output a clustering solution the big contig is essentially equidistant between the two references, so I don't buy that U11820.1 is &amp;quot;better&amp;quot; as a reference:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(&#xA;MN630242.1:0.116151&#xA;,&#xA;(&#xA;NODE1_rc:0.0504856&#xA;,&#xA;U11820.1:0.0504856&#xA;):0.0656651&#xA;)&#xA;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As I noted in the comments, it's unfortunate that QUAST and SPAdes aren't working very well. I don't know what the deal is there. But it seems like overall the assembly is quite close to both references. I don't have any intuition for why SPAdes is doing what it's doing, or why alignment at the DNA level is acting weird (lots of artificial gaps?). But I don't think it's a pathology of SPAdes particularly, it seems like it's just something weird that happened with this dataset interacting with DNA aligners. Possibly something went strangely with read deposition?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;See OP's self-answer as well (and also discussion in the comments).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Based on the (extremely informative!) follow-up updates that you have provided, I think that we can tentatively answer your questions:&lt;/p&gt;&#xA;&lt;p&gt;The tl;dr is that you can check your reads to make sure they aren't weird and you can check your output sequence to see if it's weird.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Am I using SPAdes correctly? Is there a better way to use it?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I think that you are using SPAdes correctly, based on your experience getting ~finished viral genomes in the past with the same workflow. (THis is also my experience using SPAdes on viral genomes with Illumina PE data, is that it &amp;quot;just works&amp;quot;.)&lt;/li&gt;&#xA;&lt;li&gt;It is possible that you are using &lt;em&gt;some other tools&lt;/em&gt; in your workflow in a suboptimal way. For example, maybe your trimming is not quite correct (it does seem based on your answer that you are already trimming). The question in my mind is then whether there is some non-intuitive thing in the reads that should still be trimmed.&lt;/li&gt;&#xA;&lt;li&gt;I would suggest running &lt;a href=&quot;https://i.stack.imgur.com/bitFc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;FASTQC&lt;/a&gt; on your reads for this run and also your reads for the other runs that have worked, to see if something weird is going on. FASTQC directly checks for adapter sequence, weird GC composition, quality scores etc. That will give you a lot of information.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Is it possible that the sequences that blast finds matching with the SPAdes contigs are indeed sequences that someone got using SPAdes and that's why these match (not because these are correct)?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;This is formally possible, of course. Our main way of knowing that you're getting the right genome is... does it look like other genomes that have been generated in the same way?&lt;/li&gt;&#xA;&lt;li&gt;If you are really worried about this, I would suggest directly inspecting the sequence to see what the matches are. If you like, you can then blast the matches to see if they&lt;/li&gt;&#xA;&lt;li&gt;Another approach would be to annotate the genome and make sure that it &amp;quot;looks like&amp;quot; a phage genome. E.g. it has the expected genes from FIV. &lt;a href=&quot;https://i.stack.imgur.com/oGO01.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;Prokka&lt;/a&gt; is very easy to run and works ok on phage.&lt;/li&gt;&#xA;&lt;li&gt;A slightly self-serving suggestion is to use &lt;a href=&quot;https://github.com/maximilianpress/phageterm&quot; rel=&quot;nofollow noreferrer&quot;&gt;PhageTerm&lt;/a&gt; to check your reads against your final genomes to ensure that they behave like phage- e.g. they have termini in the correct places, etc. (it does this with test coverage.) I am a contributor to PhageTerm, so I happen to know it a bit. Likely there are other options.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;What other option could explain these circumstances that I could be missing?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;As I've suggested, I think that that leaves the reads. How sure are you that they are comparable to your other runs? They look like they should be sufficient from my glance at SRA, but maybe someone bumped the MiSeq while it was running. I've given suggestions above for checking with &lt;a href=&quot;https://i.stack.imgur.com/bitFc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;FASTQC&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2020-10-25T19:55:08.857" LastActivityDate="2020-10-25T19:55:08.857" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14607" PostTypeId="2" ParentId="14600" CreationDate="2020-10-21T21:59:57.820" Score="0" Body="&lt;p&gt;Yes, it should be fine, especially if the run is finished. MinKNOW should still continue basecalling even after the MinION is disconnected. Unfortunately I can't find good posts in the ONT community about that because search is currently broken. &lt;a href=&quot;https://community.nanoporetech.com/posts/removing-flowcell-from-min#comment_16947&quot; rel=&quot;nofollow noreferrer&quot;&gt;This post&lt;/a&gt; might be useful (found by searching my email records), but it's two years old:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Hi, thank you for your post. You can certainly remove the flow cell and wash it without affecting the catch-up mode. Please see the downloads page here for the most up-to-date installer version.&lt;br /&gt;&#xA;- Monolina Binny, ONT&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If you're likely to be doing more long runs in the future (&amp;gt;6h), I'd recommend getting a cheap video card with CUDA capability (at least v10) to save time in the future.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2020-10-21T21:59:57.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14608" PostTypeId="1" CreationDate="2020-10-22T04:10:39.027" Score="4" ViewCount="146" Body="&lt;p&gt;I am looking for tools to extract features from short DNA sequences. For example, entropy, complexity, GC-content, etc.&lt;/p&gt;&#xA;&lt;p&gt;I have found the &lt;a href=&quot;https://github.com/mrzResearchArena/PyFeat/blob/master/Codes/generateFeatures.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;generateFeatures.py&lt;/a&gt; script from the &lt;a href=&quot;https://github.com/mrzResearchArena/PyFeat/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyFeat&lt;/a&gt; repo, but is there love a more widely used source code or a standard way to extract features from sequences biopython or similar?&lt;/p&gt;&#xA;&lt;p&gt;Also, I think that since there are &lt;span class=&quot;math-container&quot;&gt;$2N$&lt;/span&gt; encoded bits in a sequence of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; nucleotides, we have &lt;strong&gt;at most&lt;/strong&gt; &lt;span class=&quot;math-container&quot;&gt;$2N$&lt;/span&gt; independent features that could be extracted.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;In addition, I am curios if there are any &lt;a href=&quot;https://huggingface.co/transformers/&quot; rel=&quot;nofollow noreferrer&quot;&gt;transformers&lt;/a&gt; models for DNA sequences.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Edit&lt;/strong&gt;&lt;/em&gt;:&#xA;In addition, you can use &lt;a href=&quot;https://github.com/crispr-il/DeepHF/blob/d05444e6fa209e734dabc8defc853585afdffe5b/DeepHF/scripts/feature_util.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepHF's feature utils&lt;/a&gt;, which can be accessed and consumed as seen &lt;a href=&quot;https://github.com/crispr-il/DeepHF/blob/d05444e6fa209e734dabc8defc853585afdffe5b/DeepHF/scripts/preprocess.py#L133&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2021-08-01T12:16:32.727" LastActivityDate="2021-08-01T12:16:32.727" Title="Are there computational tools to extract features of DNA sequences?" Tags="|python|phylogenetics|features|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14609" PostTypeId="1" AcceptedAnswerId="14615" CreationDate="2020-10-22T04:49:50.227" Score="0" ViewCount="70" Body="&lt;p&gt;I want to compare two AUCs using a bootstraping method from pROC package (roc.test).&lt;/p&gt;&#xA;&lt;p&gt;I have done a classification using Logistic Regression and Serum miRNAs as prediction factors. The Controls are in both cases the same, the test group is different.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1. Cancer (high risk) vs. Control = AUC 1&#xA;2. Cancer (low risk) vs. Control = AUC 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If I want to compare this two AUCs - should I use a paired or unpaired test?&lt;/p&gt;&#xA;&lt;p&gt;I would tend to using a paired test, because of exact the same Control group (Control groups are correlated).&lt;/p&gt;&#xA;" OwnerUserId="9729" LastEditorUserId="9729" LastEditDate="2020-10-22T14:28:24.777" LastActivityDate="2020-10-23T14:32:17.923" Title="Comparing AUCs: Discrimination of same Control from different Test Group - paired or unpaired? miRNA" Tags="|statistics|cancer|design|logistic-regression|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14610" PostTypeId="1" CreationDate="2020-10-22T06:59:56.900" Score="0" ViewCount="434" Body="&lt;p&gt;I have this file:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DWAGf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DWAGf.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;and I get this error&lt;/p&gt;&#xA;&lt;p&gt;I googled and found this solution but it still doesn't work and I get this error&#xA;&lt;a href=&quot;https://i.stack.imgur.com/qzLc9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qzLc9.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;Error File 'GSM2797523_FOXA2_IDR0.02_narrowPeak_try.bed' - line 1 of custom input: variable needs to start with letter '&amp;quot;Peak name=&amp;quot;&amp;quot;Somite narrowPeak&amp;quot;&amp;quot;&amp;quot; '&lt;/p&gt;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;&lt;p&gt;Many thanks&lt;/p&gt;&#xA;&lt;p&gt;Found a way!!!&#xA;opened it with text editor and not exel as before and added the line:&#xA;track type=narrowPeak name=&amp;quot;Somite narrowPeak&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="9732" LastEditorUserId="9732" LastEditDate="2020-10-26T11:45:52.127" LastActivityDate="2020-10-26T11:45:52.127" Title="How to visualize called narrowPeak files in UCSC Genome browser?" Tags="|chip-seq|bed|ucsc|" AnswerCount="1" CommentCount="2" ClosedDate="2020-10-24T13:53:24.787" ContentLicense="CC BY-SA 4.0" />
  <row Id="14611" PostTypeId="1" AcceptedAnswerId="14612" CreationDate="2020-10-22T07:33:54.210" Score="1" ViewCount="704" Body="&lt;p&gt;I'm trying to use the code specified in &lt;a href=&quot;https://biology.stackexchange.com/questions/59493/how-to-convert-bwa-mem-output-to-bam-format-without-saving-sam-file?fbclid=IwAR1oClXdwhtsqvQqkmmc8a1af-dTaALbtkQ2kKW3cGnIrLZCaa_6HxjvrfY&quot;&gt;this link&lt;/a&gt; to create a .bam file without the need for a .sam file.&lt;/p&gt;&#xA;&lt;p&gt;Here is the code I'm using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bwa mem -R &amp;quot;@RG\tID:&lt;span class=&quot;math-container&quot;&gt;$SAMPLE\tPL:illumina\tSM:$&lt;/span&gt;SAMPLE&amp;quot; \&#xA;    &lt;span class=&quot;math-container&quot;&gt;$REF_PATH/hg38.fasta \&#xA;    mapping/R1.fastq.gz \&#xA;    mapping/R2.fastq.gz | \&#xA;    samtools sort - | \&#xA;    tee mapping/$&lt;/span&gt;SAMPLE\.bam | \&#xA;    samtools index - mapping/$SAMPLE\.bam.bai \&#xA;    2&amp;gt;&amp;gt; log&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and I get the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[M::bwa_idx_load_from_disk] read 0 ALT contigs&#xA;`[E::main_mem] fail to open file ` '.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The ref path and file locations work when creating and converting a .sam to a .bam file.&#xA;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="9669" LastEditorUserId="57" LastEditDate="2020-10-22T10:30:54.773" LastActivityDate="2020-10-22T23:17:17.553" Title="Trying to create a .bam file without the need for a .sam file" Tags="|samtools|bash|bwa|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14612" PostTypeId="2" ParentId="14611" CreationDate="2020-10-22T08:54:09.107" Score="5" Body="&lt;p&gt;This is an error from &lt;code&gt;bwa&lt;/code&gt; saying that it can't find your input files:&lt;/p&gt;&#xA;&lt;p&gt;Here's a checklist:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Check that you have no whitespace after your &lt;code&gt;\&lt;/code&gt; newline breaks. I.e. &lt;code&gt;\&lt;/code&gt; is the &lt;em&gt;very last character&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;Check that you have no whitespace in any path names&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Quote your variables&lt;/em&gt;: &lt;code&gt;$REF_PATH&lt;/code&gt; -&amp;gt; &lt;code&gt;&amp;quot;$REF_PATH&amp;quot;&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Use &lt;code&gt;set -x&lt;/code&gt; at the top of your bash script to debug.&lt;/li&gt;&#xA;&lt;li&gt;In general, read &lt;a href=&quot;https://kvz.io/bash-best-practices.html&quot; rel=&quot;noreferrer&quot;&gt;this&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9693" LastActivityDate="2020-10-22T08:54:09.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14615" PostTypeId="2" ParentId="14609" CreationDate="2020-10-22T12:25:28.477" Score="0" Body="&lt;p&gt;Generally its paired. The bootstrapping personally I find concerning. I presume you are using it to augment your data set.&lt;/p&gt;&#xA;&lt;p&gt;Bootstrapping at best tends to be a controversial statistical operation and could result in the non-significance you are observing. Anyway if its not significant between high and low - thats your answer.&lt;/p&gt;&#xA;&lt;p&gt;You could of course perform a single ML classifying low, and high cancer risk against control. That would provide a method to understand the associations of miRNA.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Bootstrapping issue, I had a think about this. It depends on how the bootstrapping is done. In a paired sample this (I assume) samples with replacement at each paired position for both ROCs. If this is not the case - the test is junk. However, if this is the case, its actually a good test, because the paired structure of the data is always preserved no matter how many bootstrap samples are performed. So like if a 100 were done and the pairwise statistic (which we don't know what it is - but something like KS test) and its &amp;gt;0.05 for all 100 samples - I don't think there's any doubt thats a robust result.&lt;/p&gt;&#xA;&lt;p&gt;Again if the two ROCs are bootstrapped independently, in my opinion the test is junk.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Reason&lt;/strong&gt; The concern of bootstrapping is it causes weird distributions which were not present in the original data. This is only okay if there's lots of experience about interpreting the result. For example, in phylogenetics there's decades of experience interpreting bootstrapping and a consensus about what it means. This often isn't the case for most other areas of statistics.&lt;/p&gt;&#xA;&lt;p&gt;I note this particular issue is really ramping up the rep score ;0)&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-10-23T14:32:17.923" LastActivityDate="2020-10-23T14:32:17.923" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14616" PostTypeId="2" ParentId="14520" CreationDate="2020-10-22T16:56:57.513" Score="2" Body="&lt;p&gt;After many considerations, I am going to accept the response from Maximilian Press. I see now that some viruses have high variability (HIV even 50% of the sequence). Therefore MN630242.1. and U11820.1 are apparently two strains. There are things I still don't understand but these are beyond the initial goal of my question. Concretely:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Why SPAdes returns one strain and rnaSPAdes the other one.&lt;/li&gt;&#xA;&lt;li&gt;Why one strain matches 99,9% with MN630242.1 (and at least another assembly) while the other one has the (now) expected variability of this kind of virus.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I also want to point that apparently, Quast is not effective for viruses with such high variation.&lt;/p&gt;&#xA;&lt;p&gt;I'm not going to delete this answer given that it responds to part of my question that is not responded in Maximilian's answer.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, directly answering my questions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Am I using SPAdes correctly? Is there a better way to use it?&lt;/strong&gt;&#xA;I am using SPAdes mostly in the correct way, similarly to how an average user would. But rnaSPAdes seems to be more appropriate for RNA viruses and it indeed works far better in this case.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Is it possible that the sequences that Blast finds matching with the SPAdes contigs are indeed sequences that someone got using SPAdes and that's why these match?&lt;/strong&gt;&#xA;That could be possible in some cases but that doesn't mean these sequences are wrong as I initially considered. For whatever reason SPAdes seems to miss one strain (if that's what really is happening)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;What other option could explain these circumstances that I could be missing?&lt;/strong&gt;&#xA;See Maximilian Press answer.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5190" LastEditorUserId="5190" LastEditDate="2020-10-27T01:32:00.910" LastActivityDate="2020-10-27T01:32:00.910" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14617" PostTypeId="2" ParentId="14611" CreationDate="2020-10-22T23:17:17.553" Score="0" Body="&lt;p&gt;Why make this so complicated with the tee command?&lt;/p&gt;&#xA;&lt;p&gt;I don't think you even can index an unsorted bam, so you have to wait until sorting is done before you index.  Indexing is really short compared to mapping and sorting, you aren't saving much by trying to do both at once.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-10-22T23:17:17.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14618" PostTypeId="2" ParentId="14610" CreationDate="2020-10-23T06:54:44.520" Score="2" Body="&lt;p&gt;Headers should not contain tabs:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;track type=narrowPeak name=&amp;quot;Somite narrowPeak&amp;quot; description=&amp;quot;Somite narrowPeak&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Ensure that you have NO tabs on that line.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-10-23T06:54:44.520" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14619" PostTypeId="1" CreationDate="2020-10-23T15:13:49.143" Score="4" ViewCount="76" Body="&lt;p&gt;I'm interested in sequencing a human sputum sample using an ONT MinION without performing any type of whole genome DNA amplification or targeted PCR.&lt;/p&gt;&#xA;&lt;p&gt;Has anyone found a good reference (or anecdotal evidence) for a range of human to pathogen DNA ratios typically found in sputum samples?&lt;/p&gt;&#xA;&lt;p&gt;Does this depend on whether the patient is infected by a virus or bacteria, and the species that is infecting the patient?&lt;/p&gt;&#xA;&lt;p&gt;I am seeking a ballpark estimate. Can I expect to find 10x as much human DNA as pathogen DNA, or 100x-1000x?&lt;/p&gt;&#xA;" OwnerUserId="6332" LastEditorUserId="73" LastEditDate="2022-07-31T21:38:30.537" LastActivityDate="2024-01-23T00:05:45.647" Title="What is the typical host-to-bug DNA ratio found in nanopore sequencing without amplification?" Tags="|nanopore|covid-19|bacteria|virus|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14620" PostTypeId="1" AcceptedAnswerId="14629" CreationDate="2020-10-23T15:30:20.710" Score="1" ViewCount="80" Body="&lt;pre&gt;&lt;code&gt;mutations = ['A222V', 'D614G', 'E484Q', 'E780Q', 'G476S', 'L18F', 'N439K',&#xA;             'S477', 'S477N', 'T478I', 'V483A']&#xA;&#xA;combinations = []&#xA;for M in range(1, len(mutations)+1):&#xA;    for subset in itertools.combinations(mutations, M):&#xA;        combinations.append(subset)&#xA;        &#xA;        combinations = ['_'.join(sorted(x)) for x in combinations]&#xA;        combinations = [x.split('_') for x in list(set(combinations))]&#xA;&#xA;root = &amp;quot;C:&amp;quot;&#xA;&#xA;os.chdir(root)&#xA;&#xA;lineages = os.listdir('Results')&#xA;&#xA;combination_labels = []&#xA;combination_counts = []&#xA;&#xA;for lineage in lineages:&#xA;    &#xA;    df = pd.read_csv('Results/' + lineage).dropna()&#xA;    &#xA;    for combination in combinations:&#xA;    &#xA;            combination_df = df[list(df)]&#xA;        &#xA;            for mutation in combination:&#xA;                combination_df = combination_df[combination_df[mutation] == 1]&#xA;    &#xA;        #print(combination_df.shape[0])&#xA;        &#xA;            combination_labels.append('_'.join(combination))&#xA;            combination_counts.append(combination_df.shape[0])&#xA;        &#xA;            out_df = pd.DataFrame({'combination':combination_labels,&#xA;                               'count':combination_counts})&#xA;            out_df['percentage'] = (out_df['count'] / df.shape[0]) * 100&#xA;            out_df = out_df.sort_values('percentage', ascending = False)   &#xA;        &#xA;            out_df.to_csv('Results_2/' + lineage.replace(&amp;quot;.csv&amp;quot;, &amp;quot;&amp;quot;) + '_3.csv',&#xA;                     header = True, &#xA;                     index = False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The input CSV&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;lineage,A222V,D614G,E484Q,E780Q,G476S,L18F,N439K,S477,S477N,T478I,V483A&#xA;417941,0,1,0,0,0,0,0,0,0,0,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output CSV&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;combination,count,percentage&#xA;D614G,87355,90.7084929856806&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My above code is used to count all occurences of combinations of spike protein mutations&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;to determine the genotypes.&#xA;However, looking at the values processed into the .csv file, i can see ' D614G' is being counted even when&#xA;it is in combination with other combinations.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;My question is how do i ensure once something has been counted (a row in the imported .csv file)&#xA;it is not counted for a second time?&lt;/p&gt;&#xA;&lt;p&gt;OR possibly how can I edit this code to prevent counting of singular mutations even when presented within a combination?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="9672" LastEditorUserId="3848" LastEditDate="2020-12-03T16:33:56.057" LastActivityDate="2020-12-03T16:33:56.057" Title="&gt;My counter is counting genotypic combination occurences more than once, how do I ensure it counts one combination and doesnt count it again?" Tags="|python|genomics|covid-19|sars-cov-2|mutations|" AnswerCount="1" CommentCount="5" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14621" PostTypeId="2" ParentId="13093" CreationDate="2020-10-23T15:50:18.447" Score="1" Body="&lt;p&gt;UniProt also provides a BLAST service (&lt;a href=&quot;https://www.uniprot.org/blast&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/blast&lt;/a&gt;), or you could use the Expasy BLAST too, at &lt;a href=&quot;https://web.expasy.org/blast&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://web.expasy.org/blast&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you have performed your similarity search at NCBI or elsewhere, you can map the identifiers from that result to UniProt by using the UniProt IDmapping service at &lt;a href=&quot;https://www.uniprot.org/uploadlists&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uploadlists&lt;/a&gt;. This can also be used programmatically, cf &lt;a href=&quot;https://www.uniprot.org/help/api_idmapping&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/help/api_idmapping&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1459" LastActivityDate="2020-10-23T15:50:18.447" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14622" PostTypeId="2" ParentId="13082" CreationDate="2020-10-23T15:52:04.287" Score="0" Body="&lt;p&gt;UniProt provides a BLAST service (&lt;a href=&quot;https://www.uniprot.org/blast&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/blast&lt;/a&gt;) which you can run against several UniProt databases, or you could use the Expasy BLAST too, at &lt;a href=&quot;https://web.expasy.org/blast&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://web.expasy.org/blast&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1459" LastActivityDate="2020-10-23T15:52:04.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14625" PostTypeId="1" CreationDate="2020-10-23T19:01:31.553" Score="1" ViewCount="223" Body="&lt;p&gt;I'm trying to programmatically replace a set of amino acid residues on an MSA with a &amp;quot;Z&amp;quot; from a list of unaligned positions. Any ideas on how I could do this?&lt;/p&gt;&#xA;&lt;p&gt;Input: a list of unaligned positions for each sequence + an MSA&lt;br /&gt;&#xA;Output: an MSA with those unaligned positions on the aligned MSA replaced with a Z for each sequence.&lt;/p&gt;&#xA;&lt;p&gt;I have a multiple sequence alignment i'm working with and a list of sites on that alignment that are glycosylated. I was trying to run a script that would take the input of the positions as a list of characters along with the MSA to replace the Nth character with a &amp;quot;Z&amp;quot; to easily visualize the sites. So essentially if I feed it the following sequence&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;M----KFLAFLCLLGFANAQ-------------D-GKCG----TLSNKSPS--------------------K&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and positions&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;2, 5, 10, 11, 14&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;it should spit back&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;M----ZFLZFLCLZZFAZAQ-------------D-GKCG----TLSNKSPS--------------------K&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9740" LastEditorUserId="3848" LastEditDate="2022-07-27T19:28:27.707" LastActivityDate="2022-07-28T02:16:22.137" Title="Python/Biopython - Replace amino acid residue on MSA with &quot;z&quot; from a list of unaligned positions" Tags="|fasta|python|proteins|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14626" PostTypeId="2" ParentId="14619" CreationDate="2020-10-24T10:32:57.610" Score="1" Body="&lt;p&gt;My recollection of a conversation I had with Justin O'Grady is that it's about 99% host DNA for unfiltered samples. There are a few ways to reduce that portion prior to sequencing. He has written a paper that mentions a saponin-based host depletion method that is quite effective:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1038/s41587-019-0156-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1038/s41587-019-0156-5&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-07-31T21:39:00.057" LastActivityDate="2022-07-31T21:39:00.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14627" PostTypeId="1" CreationDate="2020-10-24T11:34:59.073" Score="1" ViewCount="46" Body="&lt;p&gt;Copy number variation (CNV) is a phenomenon in which sections of the genome are repeated and the number of repeats in the genome varies between individuals.Copy number variation is a type of structural variation: specifically, it is a type of duplication or deletion event that affects a considerable number of base pairs. I have a set of protein coding genes and I would like to verify them to be the product of CNV. My first idea was to overlap their domains, that are well separated by homopeptide regions but before to do this I would like to know whether more suitable indexes exist to infer this characteristic. Can anyone help?&lt;/p&gt;&#xA;" OwnerUserId="9142" LastActivityDate="2020-10-24T11:34:59.073" Title="How to see if a gene is the product of copy number variation?" Tags="|protein-structure|genetics|copy-number|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14629" PostTypeId="2" ParentId="14620" CreationDate="2020-10-24T14:42:32.840" Score="3" Body="&lt;p&gt;Alright, so there are a number of problematic patterns in your code - as far as I understand what you are trying to do. Next time, try to post a reproducible example that people can use and more people will be willing to help.&lt;/p&gt;&#xA;&lt;h3&gt;L18-21:&lt;/h3&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;combination_labels = []&#xA;combination_counts = []&#xA;&#xA;for lineage in lineages:&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Declaring these two lists &lt;em&gt;before&lt;/em&gt; the loop, then appending all your combination labels inside the loop means that these two lists will contain all labels and counts for all runs (all your lineages). If I am understanding your intentions correctly this is probably not something you want. The simple fix here is to just move them into the for-loop.&lt;/p&gt;&#xA;&lt;h3&gt;L29-30:&lt;/h3&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;for mutation in combination:&#xA;    combination_df = combination_df[combination_df[mutation] == 1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here you are iteratively checking if the value for each mutation is set to '1', but you are also &lt;em&gt;overwriting&lt;/em&gt; your &lt;code&gt;combination_df&lt;/code&gt; variable at each iteration of the loop. After this loop, &lt;code&gt;combination_df&lt;/code&gt; will be whatever mutation last was set to '1' in your current combination. I will get back to solving this in my solution below.&lt;/p&gt;&#xA;&lt;h3&gt;L37-44:&lt;/h3&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;combination_labels.append('_'.join(combination))&#xA;combination_counts.append(combination_df.shape[0])&#xA;&#xA;out_df = pd.DataFrame({'combination':combination_labels,&#xA;                   'count':combination_counts})&#xA;out_df['percentage'] = (out_df['count'] / df.shape[0]) * 100&#xA;out_df = out_df.sort_values('percentage', ascending = False)   &#xA;&#xA;out_df.to_csv('Results_2/' + lineage.replace(&amp;quot;.csv&amp;quot;, &amp;quot;&amp;quot;) + '_3.csv',&#xA;         header = True, &#xA;         index = False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Since this part is again scoped inside the for-loop, each iteration (each combination) will overwrite the same &lt;code&gt;'Results_2/' + lineage.replace(&amp;quot;.csv&amp;quot;, &amp;quot;&amp;quot;) + '_3.csv'&lt;/code&gt; file. This you also want to move out of the &lt;code&gt;for combination in ...&lt;/code&gt; loop.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;A big concern is also performance. You have 2000-something combinations and 100,000 lines in your file. As you loop over combinations, you also process the DataFrame each time, when this is a proble you can solve in one iteration. It's not much data so it's still fine, but these habits are still good to develop early.&lt;/p&gt;&#xA;&lt;p&gt;Now here's an idea for a rewrite that should work better. I don't have a csv from you to test with, so there might (proabably are) still things to fix, but hopefully it'll give you a starting point:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;mutations = ['A222V', 'D614G', 'E484Q', 'E780Q', 'G476S', 'L18F', 'N439K',&#xA;             'S477', 'S477N', 'T478I', 'V483A']&#xA;&#xA;combinations = []&#xA;for M in range(1, len(mutations)+1):&#xA;    for subset in itertools.combinations(mutations, M):&#xA;        combinations.append(subset)  &#xA;        &#xA;combinations = ['_'.join(sorted(x)) for x in combinations]&#xA;&#xA;root = &amp;quot;C:&amp;quot;&#xA;&#xA;os.chdir(root)&#xA;&#xA;lineages = os.listdir('Results')&#xA;&#xA;for lineage in lineages:&#xA;&#xA;    out_df = pd.DataFrame({&amp;quot;Count&amp;quot;: 0}, index=combinations)&#xA;    &#xA;    df = pd.read_csv('Results/' + lineage, index_col=&amp;quot;lineage&amp;quot;).dropna()&#xA;    &#xA;    for index, row in df.iterrows():&#xA;        combination = '_'.join(sorted(df.columns[row==1]))&#xA;        out_df.loc[combination, &amp;quot;Count&amp;quot;] += 1&#xA;&#xA;    out_df['Percentage'] = (out_df['Count'] / df.shape[0]) * 100&#xA;    out_df = out_df.sort_values('Percentage', ascending = False)   &#xA;&#xA;    out_df.to_csv('Results_2/' + lineage.replace(&amp;quot;.csv&amp;quot;, &amp;quot;&amp;quot;) + '_3.csv', header = True, &#xA;        index = True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9693" LastEditorUserId="9693" LastEditDate="2020-10-24T14:54:55.073" LastActivityDate="2020-10-24T14:54:55.073" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14630" PostTypeId="2" ParentId="14608" CreationDate="2020-10-25T12:02:50.460" Score="2" Body="&lt;p&gt;If you want to do this with biopython, the &lt;a href=&quot;https://biopython.org/docs/1.75/api/Bio.SeqUtils.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SeqUtils&lt;/a&gt; package could be a solution.&lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2020-10-25T12:02:50.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14631" PostTypeId="1" AcceptedAnswerId="14645" CreationDate="2020-10-25T15:40:34.413" Score="0" ViewCount="840" Body="&lt;p&gt;I have a tricky data set with cells that will have a higher percent of mitochondria genes than &amp;quot;typical&amp;quot; data sets. I would like to look at the mito percentage in each cluster without any filtering (&lt;em&gt;e.g.&lt;/em&gt;, &lt;code&gt;subset(seurat_object, subset = nFeature_RNA &amp;gt; 200 &amp;amp; nFeature_RNA &amp;lt; 3000 &amp;amp; percent.mito &amp;lt; 10)&lt;/code&gt;)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FeaturePlot(seurat_object, features=c('percent.mito'), cols = c(&amp;quot;gold&amp;quot;, &amp;quot;firebrick4&amp;quot;), pt.size=0.5, label=TRUE, repel=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Done!&lt;/p&gt;&#xA;&lt;p&gt;I would also like to calculate the average percent.mito for each cluster and place the value on the FeaturePlot/TSNE plots. I have no idea how to manipulate a Seurat object to calculate these values but I can probably figure out how to place those values on the graph -- probably with &lt;code&gt;geom_text&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My Hail Mary approach is below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;p3 &amp;lt;- FeaturePlot(seurat_object, features=c('percent.mito'), cols = c(&amp;quot;gold&amp;quot;, &amp;quot;firebrick4&amp;quot;), pt.size=0.5, label=TRUE, repel=TRUE)&#xA;p3 &amp;lt;- lapply(X = p3, FUN = function(x) mean(p3))&#xA;CombinePlots(plots = p3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My attempts yield the error below; I don't know if my approach is close to correct and therefore, I have spent little time working through the error message.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;In mean.default(p3) : argument is not numeric or logical: returning NA&#xA;Execution halted&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;p3 &amp;lt;- FeaturePlot(seurat_object, features=c('percent.mito'), cols = c(&amp;quot;gold&amp;quot;, &amp;quot;firebrick4&amp;quot;), pt.size=0.5, label=TRUE, repel=TRUE)&#xA;p3 &amp;lt;- lapply(X = p3, FUN = function(x) mean(seurat_object$meta.data$seurat_clusters))&#xA;CombinePlots(plots = p3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in `[[.Seurat`(x, i, drop = TRUE) :&#xA;  Cannot find 'meta.data' in this Seurat object&#xA;Calls: lapply ... lapply -&amp;gt; FUN -&amp;gt; mean -&amp;gt; $ -&amp;gt; $.Seurat -&amp;gt; [[ -&amp;gt; [[.Seurat&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;p3 &amp;lt;- FeaturePlot(seurat_object, features=c('percent.mito'), cols = c(&amp;quot;gold&amp;quot;, &amp;quot;firebrick4&amp;quot;), pt.size=0.5, label=TRUE, repel=TRUE)&#xA;p3 &amp;lt;- lapply(X = seurat_object, FUN = function(x) mean(seurat_object$meta.data$seurat_clusters))&#xA;CombinePlots(plots = p3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in as.list.default(X) : &#xA;  no method for coercing this S4 class to a vector&#xA;Calls: lapply -&amp;gt; lapply -&amp;gt; as.list -&amp;gt; as.list.default```&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If anyone can provide some guidance -- you will be inducted into my personal hall of fame. :)&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your time.&lt;/p&gt;&#xA;&lt;p&gt;P.S. I am using Seurat 3.1&lt;/p&gt;&#xA;&lt;p&gt;Update:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;md = surate_object@meta.data &#xA;md_mean &amp;lt;- ddply(md, .(seurat_clusters), summarize, mean_mito_per=mean(percent.mito))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This was not the approach I was hoping for but I have the values -- which when expanded to the length of the original df, I can insert it into a free slot in the seurat_object.&lt;/p&gt;&#xA;&lt;p&gt;If anyone has a better approach I would love to hear it.&lt;/p&gt;&#xA;&lt;p&gt;UPDATE:&#xA;Using @PPK's approach&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# First the cluster annotation and the tsne embeddings are merged&#xA;label.df &amp;lt;- cbind(as.data.frame(seurat_object@meta.data), as.data.frame(seurat_object@reductions$tsne@cell.embeddings)) &#xA;&#xA;# using dplyr across to calculate the mean mitochondrial percentage and &#xA;# the median tsne values per cluster&#xA;label.df &amp;lt;- label.df %&amp;gt;% dplyr::group_by(seurat_clusters) %&amp;gt;% dplyr::summarise(dplyr::across(percent.mito, ~ mean(.), .names = &amp;quot;mean_{.col}&amp;quot;), dplyr::across(contains(&amp;quot;tSNE&amp;quot;), ~ median(.)))&#xA;&#xA;# running FeaturePlot and adding the label layer&#xA;FeaturePlot(seurat_object, reduction = &amp;quot;tsne&amp;quot;, features = &amp;quot;percent.mito&amp;quot;) + &#xA;  geom_text(data = label.df, aes(x = tSNE_1, y = tSNE_2, label = mean_percent.mito))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;* object '.col' is a function.&#xA;ℹ Input `..1` is `dplyr::across(percent.mito, ~mean(.), .names = &amp;quot;mean_{.col}&amp;quot;)`.&#xA;ℹ The error occured in group 1: seurat_clusters = &amp;quot;0&amp;quot;.&#xA;Backtrace:&#xA;     █&#xA;  1. └─global::initial_clustering(BM0H, &amp;quot;BM0H&amp;quot;)&#xA;  2.   └─`%&amp;gt;%`(...)&#xA;  3.     ├─base::withVisible(eval(quote(`_fseq`(`_lhs`)), env, env))&#xA;  4.     └─base::eval(quote(`_fseq`(`_lhs`)), env, env)&#xA;  5.       └─base::eval(quote(`_fseq`(`_lhs`)), env, env)&#xA;  6.         └─`_fseq`(`_lhs`)&#xA;  7.           └─magrittr::freduce(value, `_function_list`)&#xA;  8.             ├─base::withVisible(function_list[[k]](value))&#xA;  9.             └─function_list[[k]](value)&#xA; 10.               ├─dplyr::summarise(...)&#xA; 11.               └&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2934" LastEditorUserId="2934" LastEditDate="2020-10-27T21:32:14.867" LastActivityDate="2020-10-27T21:32:14.867" Title="Calculating average mito.percentage for each cluster (seurat)" Tags="|seurat|single-cell|clustering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14632" PostTypeId="1" AcceptedAnswerId="14633" CreationDate="2020-10-25T19:27:25.173" Score="0" ViewCount="137" Body="&lt;p&gt;I have this GSE dataset ( GSE104279 ) (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE104279&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE104279&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;I want to make a table with set IDs and ftp urls to use it as a table in galaxy.org&lt;/p&gt;&#xA;&lt;p&gt;I know that we can use ENA to get specific arrangement:&#xA;&lt;a href=&quot;https://www.ebi.ac.uk/ena/browser/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/ena/browser/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I tried to get:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SampleID    Group   URL&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;so I used :&#xA;&lt;a href=&quot;https://www.ebi.ac.uk/ena/browser/view/PRJNA412223&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/ena/browser/view/PRJNA412223&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But nothing is showing.&lt;/p&gt;&#xA;&lt;p&gt;Is there away to get these urls in the arrangement above?&lt;/p&gt;&#xA;" OwnerUserId="9709" LastActivityDate="2020-10-26T00:48:49.073" Title="Get list of urls of GSM data set of a GSE set" Tags="|sequence-alignment|sam|bioconductor|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14633" PostTypeId="2" ParentId="14632" CreationDate="2020-10-25T21:09:09.073" Score="1" Body="&lt;p&gt;You can use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez Direct&lt;/a&gt; for this as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db gds -query 'GSE104279' \&#xA;  | esummary \&#xA;  | xtract -pattern DocumentSummary \&#xA;    -if 'entryType' -equals 'GSM' \&#xA;    -def 'NA' -element Accession title summary FTPLink&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will return a table with data similar to this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GSM2580330      18_Z13_2_d5_Zika        cortical organoids_Zika_5d      ftp://ftp.ncbi.nlm.nih.gov/geo/samples/GSM2580nnn/GSM2580330/&#xA;GSM2580329      17_Z13_2_d5_Control     cortical organoids_mock_5d      ftp://ftp.ncbi.nlm.nih.gov/geo/samples/GSM2580nnn/GSM2580329/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To download sequence reads, you should follow links to SRA. Using Entrez Direct you can do this as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db gds -query 'GSE104279' \&#xA;  | elink -target sra \&#xA;  | efetch -format runinfo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will return a comma-delimited table containing SRA identifiers and an FTP path to the SRA data. These FTP paths won't lead you to FASTQ files though. You can pass the SRA run identifiers of the format SRR### (or ERR### or DRR###) to &lt;code&gt;fastq-dump&lt;/code&gt; or &lt;code&gt;fasterq-dump&lt;/code&gt; tools from the &lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/sra.cgi?view=software&quot; rel=&quot;nofollow noreferrer&quot;&gt;SRAToolkit&lt;/a&gt; to download data in FASTQ format.&lt;/p&gt;&#xA;" OwnerUserId="3577" LastEditorUserId="3577" LastEditDate="2020-10-26T00:48:49.073" LastActivityDate="2020-10-26T00:48:49.073" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14634" PostTypeId="1" AcceptedAnswerId="14658" CreationDate="2020-10-25T22:37:33.593" Score="1" ViewCount="721" Body="&lt;p&gt;I have run RepeatModelor and I don't see the classified output file. I got this error.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Missing /home/mudith/.conda/envs/braker/share/RepeatMasker/Libraries/RepeatMasker.lib.nsq!&#xA;Please rerun the configure program in the RepeatModeler directory before running this script.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I think I have to run configure file. I found it. But it doesn't run when I just try&#xA;&lt;code&gt;perl ./configure&lt;/code&gt;&#xA;Can someone please explain me how to run this? I'm new to this.&#xA;Also I found this in github. But I don't understand.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cd &lt;span class=&quot;math-container&quot;&gt;$(dirname $&lt;/span&gt;(which RepeatMasker))/../share/RepeatMasker&#xA;# ./configure downloads required databases&#xA;echo -e &amp;quot;\n2\n&lt;span class=&quot;math-container&quot;&gt;$(dirname $&lt;/span&gt;(which rmblastn))\n\n5\n&amp;quot; &amp;gt; tmp &amp;amp;&amp;amp; ./configure &amp;lt; tmp&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I don't understand the third line.&lt;/p&gt;&#xA;" OwnerUserId="856" LastActivityDate="2020-10-27T16:54:09.440" Title="How to fix RepeatMasker.lib.nsq missing error when running RepeatModelor?" Tags="|genome|sequence-annotation|repeatmasker|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14635" PostTypeId="1" AcceptedAnswerId="14641" CreationDate="2020-10-25T22:52:52.930" Score="1" ViewCount="1597" Body="&lt;p&gt;I have this GSE dataset ( GSE104279 ) (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE104279&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE104279&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;I want to run cutadapt, but how would I find the adapter sequence so I can run cutadapt?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6RMNp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6RMNp.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9709" LastEditorUserId="9709" LastEditDate="2020-10-26T17:54:18.390" LastActivityDate="2020-10-26T17:54:18.390" Title="How to find adapter sequence" Tags="|rna-seq|sequence-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14636" PostTypeId="1" CreationDate="2020-10-26T01:55:43.030" Score="1" ViewCount="146" Body="&lt;p&gt;I was wondering how I can create images showing the bonds and the residue codes involved in the bound Protein Ligand complex obtained after Molecular Docking as depicted in the image below.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/64U3G.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/64U3G.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9346" LastActivityDate="2020-10-26T20:16:19.093" Title="How do I create Protein Ligand images" Tags="|python|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14637" PostTypeId="1" CreationDate="2020-10-26T03:42:44.030" Score="0" ViewCount="88" Body="&lt;p&gt;I have a fasta file assembly and combining it with the raw reads we produced a .bam file which I converted to .sam .&lt;/p&gt;&#xA;&lt;p&gt;The .sam information lines look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A00321:42:HLLVYDSXX:2:2302:6153:3505    99      NODE_1_length_3415511_cov_137.721502    16      60      128M    =       607     742     CGATTAGTCCGGCCAAATCGCCGTCGAGCGCAATGAACATAACGGTCTTGCCCTCAGCGCGCAGCGCATCGGCCTTGGCGTCGATTGTGGAGTGCTCGACGCCCATGATGTCCATCATAGCACCATTG        FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF        RX:Z:TTGAGGGTATAGTAGT   QX:Z:FFFFFFFFFFFFFFFF   TR:Z:GACACCG    TQ:Z:FFFFFFF    BC:Z:AGTTGCAG   QT:Z:FFFFFFFF   XS:i:-10        AS:i:0  XM:Z:0  AM:Z:0  XT:i:1  RG:Z:over_1kb:LibraryNotSpecified:1:unknown_fc:0        OM:i:60&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Separated by mandatory fields it would be something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;QNAME: A00321:42:HLLVYDSXX:1:1644:2248:3881&#xA;FLAG: 99&#xA;RNAME: NODE_1_length_3415511_cov_137.721502&#xA;POS: 1&#xA;MAPQ: 60&#xA;CIGAR: 1S127M&#xA;RNEXT: =&#xA;PNEXT: 536&#xA;TLEN: 386&#xA;SEQ:  ATCGGGTCTGACACCGCGATTAGTCCGGCCAAATCGCCGTCGAGCGCAATGAACATAACGGTCTTGCCCTCAGCGCGCAGCGCATCGGCCTTGGCGTCGATTGTGGAGTGCTCGACGCCCATGATGTC&#xA;QUAL: FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm actually interested in the meta data. I want to know how the RX: and BC: fields are distributed across the scaffolds in the original assembly.&lt;/p&gt;&#xA;&lt;p&gt;I imagined the .sam file already contains the information about the assembly used to produce it. If I'm wrong, I'm sorry and please correct me, I'm just assuming.&lt;/p&gt;&#xA;&lt;p&gt;What I want to do is, for each read in the .sam file, I find out its position in the assembled scaffold, and I record, Read_ID,Scaffold_ID,Read_Position_Inside_Scaffold,RX,BC&lt;/p&gt;&#xA;&lt;p&gt;Then I want to use that database to analyse the distribution of RX and BC inside each scaffold.&lt;/p&gt;&#xA;&lt;p&gt;That's what I want.&lt;/p&gt;&#xA;&lt;p&gt;Ultimately what I'm trying to do is evaluate the quality of my assemblies based on the Barcode distribution.&lt;/p&gt;&#xA;&lt;p&gt;I'm good at programming and parsing, I'm just having trouble figuring out, where, inside the .sam file, can I find the scaffold and scaffold position of each read.&lt;/p&gt;&#xA;" OwnerUserId="9373" LastActivityDate="2020-10-26T10:19:18.677" Title="How to identify to each scaffold a read belongs to, inside a .sam file?" Tags="|sam|fasta|samtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14639" PostTypeId="2" ParentId="14637" CreationDate="2020-10-26T10:19:18.677" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What I want to do is, for each read in the .sam file, I find out its position in the assembled scaffold, and I record, Read_ID,Scaffold_ID,Read_Position_Inside_Scaffold,RX,BC&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I'm just having trouble figuring out, where, inside the .sam file, can I find the scaffold and scaffold position of each read.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You already listed all those:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ReadID -&amp;gt; &lt;code&gt;QNAME&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;ScaffoldID -&amp;gt; &lt;code&gt;RNAME&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Position -&amp;gt; &lt;code&gt;Pos&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Assembly evaluation is a wide topic, but other software you can use includes &lt;a href=&quot;https://busco.ezlab.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BUSCO&lt;/a&gt;, &lt;a href=&quot;http://bioinf.spbau.ru/quast&quot; rel=&quot;nofollow noreferrer&quot;&gt;QUAST&lt;/a&gt; or something like &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/30107434/&quot; rel=&quot;nofollow noreferrer&quot;&gt;LTRi&lt;/a&gt;. Here's a more &lt;a href=&quot;https://scilifelab.github.io/courses/assembly/2017-11-15/exercises/Exercise_Assembly_Validation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;in-depth&lt;/a&gt; guide by the SciLife lab in Sweden. Usually, when aligning reads back to an assembly, you care about the mapped insert size distribution compared to the theoretical insert size distribution, which tells you about collapsing or exapnding regions (&lt;em&gt;i.e.&lt;/em&gt; structural misassemblies). This is something that tools like &lt;a href=&quot;https://github.com/vezzi/FRC_align&quot; rel=&quot;nofollow noreferrer&quot;&gt;FRC_Align&lt;/a&gt; do.&lt;/p&gt;&#xA;" OwnerUserId="9693" LastActivityDate="2020-10-26T10:19:18.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14640" PostTypeId="1" CreationDate="2020-10-26T10:21:06.423" Score="2" ViewCount="69" Body="&lt;p&gt;I am trying to calculate polygenic risk scores (PRS) scores for a new dataset. This dataset does not have all the variants that the PRS score needs. The PRS score I am interested in has 40 variants, but in my dataset only 20 of those variants are present. What is the correct way of assessing the decreased predictive power of the PRS score due to the missing variants?&lt;/p&gt;&#xA;&lt;p&gt;I assume it is not as easy as summing the effective weights of those variants I have and comparing to the full sum (all variants needed for the PRS).&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot and sorry for newbie question!&lt;/p&gt;&#xA;" OwnerUserId="9748" LastEditorUserId="3694" LastEditDate="2023-06-15T11:41:20.663" LastActivityDate="2023-06-15T11:41:20.663" Title="Loss of predictive power of polygenic risk score when dataset contains missing variants" Tags="|snp|gwas|polygenic-risk-score|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14641" PostTypeId="2" ParentId="14635" CreationDate="2020-10-26T10:25:55.093" Score="4" Body="&lt;p&gt;The definitive answer is the sequencing institute/lab. They know what protocol/chemistry etc. was used.&lt;/p&gt;&#xA;&lt;p&gt;If you don't have access to that a number of tools check for known adapter sequences. Run e.g. &lt;a href=&quot;https://www.bioinformatics.babraham.ac.uk/projects/fastqc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FASTQC&lt;/a&gt;, which will tell you the proportions of adapter sequence found. Tools like &lt;a href=&quot;https://github.com/FelixKrueger/TrimGalore&quot; rel=&quot;nofollow noreferrer&quot;&gt;TrimGalore&lt;/a&gt; can also autodetect the most common adapters.&lt;/p&gt;&#xA;&lt;p&gt;UPDATE:&#xA;I checked 2M reads from one run of your dataset and it seems to me that it was uploaded already preprocessed:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MY5qw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MY5qw.png&quot; alt=&quot;per-sequence-quality&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/C6keF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/C6keF.png&quot; alt=&quot;length-distribution&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/eLvPC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eLvPC.png&quot; alt=&quot;adapter-content&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Check all your samples, but I don't see a need to do any preprocessing if all of them look like this.&lt;/p&gt;&#xA;" OwnerUserId="9693" LastEditorUserId="9693" LastEditDate="2020-10-26T11:12:17.440" LastActivityDate="2020-10-26T11:12:17.440" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14642" PostTypeId="1" CreationDate="2020-10-26T11:16:59.717" Score="0" ViewCount="95" Body="&lt;p&gt;I want to create custom tracks from these&lt;a href=&quot;https://i.stack.imgur.com/NqJjt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NqJjt.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt; files&lt;/p&gt;&#xA;&lt;p&gt;I can add the line : track type=narrowPeak name=“narrowPeak” manually by opening it with text editor:&#xA;track type=narrowPeak name=“narrowPeak”&#xA;&lt;a href=&quot;https://i.stack.imgur.com/kl8KN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kl8KN.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;but I wonder if there is no simple way to do it with code- to open all files that end with bed and add them this line.&#xA;Didnt find this option in bed tools, If it is possible to add the name extracting it from the file name it would be amazing.&lt;/p&gt;&#xA;&lt;p&gt;Many thanks&lt;/p&gt;&#xA;" OwnerUserId="9732" LastActivityDate="2020-10-26T12:32:50.613" Title="How can I add for several bed files the header : track type=narrowPeak name=“narrowPeak” preferably in python ,can handle with R" Tags="|python|chip-seq|bed|ucsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14643" PostTypeId="2" ParentId="14642" CreationDate="2020-10-26T11:31:03.177" Score="1" Body="&lt;p&gt;I don't think you need to go to R or Python to do that. In bash&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;set -euxo pipefail&#xA;for f in *.bed&#xA;do&#xA;  bn=&amp;quot;$(basename ${f})&amp;quot;&#xA;  on=&amp;quot;${bn%%.bed}_with_header.bed&amp;quot;&#xA;  echo 'track type=narrowPeak name=&amp;quot;narrowPeak&amp;quot;' &amp;gt; &amp;quot;${on}&amp;quot;&#xA;  cat &amp;quot;${f}&amp;quot; &amp;gt;&amp;gt; &amp;quot;${on}&amp;quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;EDIT: I just read you want to set the name based on the filename.&lt;/p&gt;&#xA;&lt;p&gt;EDIT2: And also to add an arbitrary directory&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;set -euxo pipefail&#xA;&#xA;INDIR=&amp;quot;/path/to/bedfiles&amp;quot;&#xA;&#xA;for f in &lt;span class=&quot;math-container&quot;&gt;$(find &quot;$&lt;/span&gt;{INDIR}&amp;quot; -name &amp;quot;*.bed&amp;quot;)&#xA;do&#xA;  bn=&lt;span class=&quot;math-container&quot;&gt;$(basename &quot;$&lt;/span&gt;{f}&amp;quot;)&#xA;  on=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${bn%%.bed}_with_header.bed&quot;&#xA;  dn=$&lt;/span&gt;(dirname &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${f}&quot;)&#xA;  echo &quot;track type=narrowPeak name=\&quot;$&lt;/span&gt;{bn}\&amp;quot;&amp;quot; &amp;gt; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${dn}&quot;/&quot;$&lt;/span&gt;{on}&amp;quot;&#xA;  cat &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${f}&quot; &amp;gt;&amp;gt; &quot;$&lt;/span&gt;{dn}&amp;quot;/&amp;quot;${on}&amp;quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9693" LastEditorUserId="9693" LastEditDate="2020-10-26T12:32:50.613" LastActivityDate="2020-10-26T12:32:50.613" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14645" PostTypeId="2" ParentId="14631" CreationDate="2020-10-26T15:27:51.103" Score="2" Body="&lt;p&gt;You can extract the necessary values and add them directly the plot as a second layer using &lt;code&gt;plot + geom_text()&lt;/code&gt;. This is very similar to the inner workings of the &lt;code&gt;DimPlot&lt;/code&gt; function with &lt;code&gt;label = TRUE&lt;/code&gt; but allows you to use anything as label.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# I use dplyr v1.0.2 for all data frame manipulations &#xA;# First the cluster annotation and the tsne embeddings are merged&#xA;label.df &amp;lt;- cbind(as.data.frame(seurat_object@meta.data), as.data.frame(seurat_object@reductions$tsne@cell.embeddings)) &#xA;&#xA;# using dplyr across to calculate the mean mitochondrial percentage and &#xA;# the median tsne values per cluster&#xA;label.df &amp;lt;- label.df %&amp;gt;% dplyr::group_by(seurat_clusters) %&amp;gt;% dplyr::summarise(dplyr::across(percent.mito, ~ mean(.), .names = &amp;quot;mean_{.col}&amp;quot;), dplyr::across(contains(&amp;quot;tSNE&amp;quot;), ~ median(.)))&#xA;&#xA;# running FeaturePlot and adding the label layer&#xA;FeaturePlot(seurat_object, reduction = &amp;quot;tsne&amp;quot;, features = &amp;quot;percent.mito&amp;quot;) + &#xA;  geom_text(data = label.df, aes(x = tSNE_1, y = tSNE_2, label = mean_percent.mito))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The labels are positioned at the median position of the cells of a cluster. Typing &lt;code&gt;LabelClusters&lt;/code&gt; (without brackets) in the R console will show you the labeling function used by Seurat that contains some additional ways of positioning the labels.&lt;br /&gt;&#xA;If you want to avoid overlapping labels you can use &lt;code&gt;geom_text_repel&lt;/code&gt; from the &lt;code&gt;ggrepel&lt;/code&gt; package.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastEditorUserId="4885" LastEditDate="2020-10-27T16:48:17.967" LastActivityDate="2020-10-27T16:48:17.967" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="14646" PostTypeId="1" AcceptedAnswerId="14648" CreationDate="2020-10-26T16:15:48.810" Score="1" ViewCount="178" Body="&lt;p&gt;I am analyzing nucleotide base-pair patterns in RNA and DNA, and had a thought about RNA and DNA (Let me first state though, I am not a biologist; I am an algorithmatician, employing a sort of recursive sequential neural network).&lt;/p&gt;&#xA;&lt;p&gt;In any event, I was wondering if we are misinterpreting the way that we look at codons in RNA.  Are we?  Are we missing something when we interpret them the way that we do?  A picture is worth a thousand words, so I made a powerpoint slide of my question and screen captured it.  It highlights the specific question I am asking, which is: are codons layered?&lt;/p&gt;&#xA;&lt;p&gt;Why is this important to me?  When I look at patterns, I know for a fact that unless it is the end of the sequence (a stop codon) that if I have a U-nucleotide, then the next two nucleotides cannot be AA, AG, nor GA.  Additionally, I know that unless there is a U-nucleotide, the sequence will definitely not end in two or less nucleotides.  This is what got me to wondering if there indeed is layered information in the combination of nucleotides.  Please see the picture.&lt;/p&gt;&#xA;&lt;p&gt;So if we have (as in the picture) an RNA sequence:&#xA;AUGGUCAGUCCAUAA&lt;/p&gt;&#xA;&lt;p&gt;We might interpret that as 5 codons {Methionine, Valine, Serine, Proline, STOP}&lt;/p&gt;&#xA;&lt;p&gt;But what if there are actually 13 codons: {Methionine, Tryptophan, Glycine, Valine, Serine, Glutamine, Serine, Valine, Serine, Proline, Histidine, Isoleucine, STOP}&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wpXID.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wpXID.jpg&quot; alt=&quot;What we might be misintepreting about RNA and DNA codons&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9754" LastEditorUserId="9754" LastEditDate="2020-10-26T16:32:41.257" LastActivityDate="2020-10-27T02:25:36.890" Title="Are codons in RNA layered? Are we misinterpreting RNA codons?" Tags="|sequence-alignment|sequence-analysis|rna|computational-biochemistry|codon|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14647" PostTypeId="1" CreationDate="2020-10-26T16:24:25.977" Score="1" ViewCount="53" Body="&lt;p&gt;I am very new so bare with me. I have a file called data (.csv that I turned into a data table) with 2 columns (geneid, normalized counts) and about 9000 rows. I have a second file called Xgenes of genes that I am interested in pulling out of the first table (1 column- geneid, about 50 rows).&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to make 2 new files- I want one with all the geneid/normalized counts minus the ones in the second file (so minus about 50 genes). Then I want to make another file that only has the geneid/normalized counts of the 50 genes in the second file.&lt;/p&gt;&#xA;&lt;p&gt;Currently I am doing this manually which does indeed work but if there is a way to do it through R it would make my future analysis much easier.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="9755" LastActivityDate="2020-10-27T10:29:03.810" Title="Splitting data table up based on separate file of gene names" Tags="|r|rna-seq|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14648" PostTypeId="2" ParentId="14646" CreationDate="2020-10-26T16:41:48.117" Score="3" Body="&lt;p&gt;This relates to the concept of the &lt;em&gt;&lt;strong&gt;reading frame&lt;/strong&gt;&lt;/em&gt;, which is the point at which you start dividing 3-letter sequences into codons. As you point out, there are multiple ways to divide a string of amino acids into triplets - in fact, there are six unique ways, found by shifting the start letter by one three times, and doing the same but reading in reverse.&lt;/p&gt;&#xA;&lt;p&gt;Typically, there is one commonly used frame which can be transcribed and turned into protein, called the open reading frame. But it is not unheard of for a single piece of DNA to encode multiple, overlapping genes, depending on where you start reading it. So, the sequence you show cannot encode 13 amino acids in a single string, but it can encode three different strings, one of which has 5 amino acids and the other two which have 4 amino acids each.&lt;/p&gt;&#xA;&lt;p&gt;See &lt;a href=&quot;https://en.wikipedia.org/wiki/Reading_frame#Multiple_reading_frames&quot; rel=&quot;nofollow noreferrer&quot;&gt;Multiple Reading Frames&lt;/a&gt; at Wikipedia, which shows the following example of a single piece of human DNA which encodes two different genes, MT-ATP6 and MT-ATP8.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UKklY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UKklY.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1930" LastActivityDate="2020-10-26T16:41:48.117" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14649" PostTypeId="1" CreationDate="2020-10-26T17:03:27.443" Score="3" ViewCount="1396" Body="&lt;p&gt;Is there an accepted boundary of the &lt;em&gt;extended&lt;/em&gt; major histocompatibility complex (xMHC) in human? Specifically, I am interested in the boundary coordinates for Genome Reference Consortium Human Build 38 (GRCh38 / hg38).&lt;/p&gt;&#xA;&lt;p&gt;I'd like to use these coordinates to query MyGene.info to retrieve a list of genes in the xMHC. For example, King et al 2019 &lt;a href=&quot;https://doi.org/10.1371/journal.pgen.1008489.s002&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;S2 Data. Updating Pipeline Data: Supplementary Methods And Results&quot;&gt;used&lt;/a&gt; the following definition:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;we excluded drugs with nonhuman or xMHC gene targets. We considered xMHC to include HIST1H2AA and KIFC1 and all genes between them (Chromosome 6 25.7 Mb-33.4 Mb).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But I was curious whether there was an authoritative definition of the xMHC region in humans that would specify down the the base pair rather than just megabase pair.&lt;/p&gt;&#xA;" OwnerUserId="9750" LastActivityDate="2020-11-02T14:30:31.163" Title="What are the coordinates of the extended major histocompatibility complex (xMHC) in GRCh38?" Tags="|gene|genomics|human-genome|human|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14650" PostTypeId="1" CreationDate="2020-10-26T17:11:27.893" Score="2" ViewCount="67" Body="&lt;p&gt;The text mining literature has an emphasis on identifying and normalizing gene names, mutations, pathways, concepts, and so on. I haven't been able to find much, however, on methods for extracting nucleotide and peptide sequences from documents. I did find this tidbit from &lt;a href=&quot;https://doi.org/10.1186/gb-2008-9-2-r31&quot; rel=&quot;nofollow noreferrer&quot;&gt;Aerts &lt;em&gt;et al.&lt;/em&gt; (2008)&lt;/a&gt;, emphasis mine:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Text was split into words and words greater than 10 characters in length with greater than 40% of characters from the capitalized DNA alphabet [ACGT] were extracted using regular expressions to isolate putative DNA sequences.&#xA;All putative DNA sequences extracted from each paper were concatenated in the order they appeared in the text into a single fasta sequence and labeled with the corresponding PMID.&#xA;Concatenation of sequences was performed to merge sequences split by line breaks in the text conversion, and &lt;strong&gt;because we reasoned that inappropriate joins would be reconciled at the genome level by local alignment procedures&lt;/strong&gt;.&#xA;Extracted, concatenated sequences were used as queries to BLAST RepeatMasked versions of genome sequences downloaded from the UCSC genome database...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I'm struck by how simple the author's method for extracting DNA sequences is. But I guess with the limited complexity of the DNA alphabet, that makes sense. This approach wouldn't work with the much more complex protein alphabet.&lt;/p&gt;&#xA;&lt;p&gt;Are simple regular expressions the text mining community's state of the art for finding DNA/RNA sequences in documents? What methods are used to identify protein sequences?&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2020-10-26T19:35:17.060" Title="Mining documents for nucleotide and peptide sequences" Tags="|proteins|phylogenetics|text-mining|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14652" PostTypeId="1" CreationDate="2020-10-26T17:39:41.977" Score="1" ViewCount="60" Body="&lt;p&gt;I have these files that I want to make table of (&lt;a href=&quot;https://github.com/learnseq/RNAseqfiles01.git&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/learnseq/RNAseqfiles01.git&lt;/a&gt;), the table that I want (since all the files have the same ID column),, I want merge the ID column with the count column from each file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ID    File 1 count column     File 2 count column     File 3 count column   File 4 count column&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I cleaned up the files from extra lines using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -vE '^{' GSM2580324_counts.txt | grep -vE '^\\' | grep -v '^ *$' | sed '/^{/d'  | tr '\\' ' ' | sed '/}/d' | grep -vE '^__' &amp;gt; GSM2580324_counts04.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But when I try to make a table in R using this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;%R files = list.files(path=&amp;quot;.&amp;quot;, pattern=&amp;quot;GSM&amp;quot;)&#xA;%R file1 = read.table(&amp;quot;GSM2580319_counts01.txt&amp;quot; , col.names=c(&amp;quot;gene_id&amp;quot;,&amp;quot;GSM2580319_counts01.txt&amp;quot; ))&#xA;%R file2 = read.table(&amp;quot;GSM2580320_counts02.txt&amp;quot; , col.names=c(&amp;quot;gene_id&amp;quot;,&amp;quot;GSM2580320_counts02.txt&amp;quot; ))&#xA;%R file3 = read.table(&amp;quot;GSM2580323_counts03.txt&amp;quot;, col.names=c(&amp;quot;gene_id&amp;quot;,&amp;quot;GSM2580323_counts03.txt&amp;quot; ))&#xA;%R file4 = read.table(&amp;quot;GSM2580324_counts04.txt&amp;quot;, col.names=c(&amp;quot;gene_id&amp;quot;,&amp;quot;GSM2580324_counts04.txt&amp;quot;))&#xA;%R out.file = merge (file1, file2, by=c(&amp;quot;gene_id&amp;quot;))&#xA;%R out.file2 &amp;lt;- merge(out.file, file3, by=c(\&amp;quot;gene_id\&amp;quot;))&#xA;%R out.file3 &amp;lt;- merge(out.file2, file4, by=c(\&amp;quot;gene_id\&amp;quot;))&#xA;%R outtable01 &amp;lt;- write.table(out.file, file = &amp;quot;out.file3&amp;quot;,sep=&amp;quot;\t&amp;quot;, row.names = FALSE)&#xA;%R outtable01                        &#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;RParsingError                             Traceback (most recent call last)&#xA;&amp;lt;ipython-input-179-2c471cf65113&amp;gt; in &amp;lt;module&amp;gt;&#xA;      7 get_ipython().run_line_magic('R', 'file4 = read.table(&amp;quot;GSM2580324_counts04.txt&amp;quot;, col.names=c(&amp;quot;gene_id&amp;quot;,&amp;quot;GSM2580324_counts04.txt&amp;quot;))')&#xA;      8 get_ipython().run_line_magic('R', 'out.file = merge (file1, file2, by=c(&amp;quot;gene_id&amp;quot;))')&#xA;----&amp;gt; 9 get_ipython().run_line_magic('R', 'out.file2 &amp;lt;- merge(out.file, file3, by=c(\\&amp;quot;gene_id\\&amp;quot;))')&#xA;     10 get_ipython().run_line_magic('R', 'out.file3 &amp;lt;- merge(out.file2, file4, by=c(\\&amp;quot;gene_id\\&amp;quot;))')&#xA;     11 get_ipython().run_line_magic('R', 'outtable01 &amp;lt;- write.table(out.file, file = &amp;quot;out.file3&amp;quot;,sep=&amp;quot;\\t&amp;quot;, row.names = FALSE)')&#xA;&#xA;/opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/IPython/core/interactiveshell.py in run_line_magic(self, magic_name, line, _stack_depth)&#xA;   2315                 kwargs['local_ns'] = sys._getframe(stack_depth).f_locals&#xA;   2316             with self.builtin_trap:&#xA;-&amp;gt; 2317                 result = fn(*args, **kwargs)&#xA;   2318             return result&#xA;   2319 &#xA;&#xA;&amp;lt;decorator-gen-160&amp;gt; in R(self, line, cell, local_ns)&#xA;&#xA;/opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/IPython/core/magic.py in &amp;lt;lambda&amp;gt;(f, *a, **k)&#xA;    185     # but it's overkill for just that one bit of state.&#xA;    186     def magic_deco(arg):&#xA;--&amp;gt; 187         call = lambda f, *a, **k: f(*a, **k)&#xA;    188 &#xA;    189         if callable(arg):&#xA;&#xA;/opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/rpy2/ipython/rmagic.py in R(self, line, cell, local_ns)&#xA;    729             if line_mode:&#xA;    730                 for line in code.split(';'):&#xA;--&amp;gt; 731                     text_result, result, visible = self.eval(line)&#xA;    732                     text_output += text_result&#xA;    733                 if text_result:&#xA;&#xA;/opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/rpy2/ipython/rmagic.py in eval(self, code)&#xA;    262             try:&#xA;    263                 # Need the newline in case the last line in code is a comment.&#xA;--&amp;gt; 264                 value, visible = ro.r(&amp;quot;withVisible({%s\n})&amp;quot; % code)&#xA;    265             except (ri.embedded.RRuntimeError, ValueError) as exception:&#xA;    266                 # Otherwise next return seems to have copy of error.&#xA;&#xA;/opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/rpy2/robjects/__init__.py in __call__(self, string)&#xA;    385 &#xA;    386     def __call__(self, string):&#xA;--&amp;gt; 387         p = rinterface.parse(string)&#xA;    388         res = self.eval(p)&#xA;    389         return conversion.rpy2py(res)&#xA;&#xA;/opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/rpy2/rinterface_lib/conversion.py in _(*args, **kwargs)&#xA;     38 def _cdata_res_to_rinterface(function):&#xA;     39     def _(*args, **kwargs):&#xA;---&amp;gt; 40         cdata = function(*args, **kwargs)&#xA;     41         # TODO: test cdata is of the expected CType&#xA;     42         return _cdata_to_rinterface(cdata)&#xA;&#xA;/opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/rpy2/rinterface.py in parse(text, num)&#xA;     44         raise TypeError('text must be a string.')&#xA;     45     robj = StrSexpVector([text])&#xA;---&amp;gt; 46     return _rinterface._parse(robj.__sexp__._cdata, num)&#xA;     47 &#xA;     48 &#xA;&#xA;/opt/applications/python/3.6.3/gnu/lib/python3.6/site-packages/rpy2/rinterface_lib/_rinterface_capi.py in _parse(cdata, num)&#xA;    556         if status[0] != openrlib.rlib.PARSE_OK:&#xA;    557             raise RParsingError('Parsing status not OK',&#xA;--&amp;gt; 558                                 PARSING_STATUS(status[0]))&#xA;    559     return res&#xA;&#xA;RParsingError: Parsing status not OK&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What should I do?&lt;/p&gt;&#xA;" OwnerUserId="9709" LastEditorUserId="9709" LastEditDate="2020-10-26T18:19:31.843" LastActivityDate="2021-07-23T21:52:56.403" Title="Table error when combining counts columns" Tags="|r|rna-seq|bash|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14653" PostTypeId="2" ParentId="14636" CreationDate="2020-10-26T17:54:39.143" Score="3" Body="&lt;p&gt;Pymol, in either its commercial or open source edition, will be sufficient to make figures such as your examples. Pymol recognises &amp;quot;polar contacts&amp;quot; for the intermolecular interactions and has labels that can be used for residue codes.&#xA;It is also scriptable and is widely used.&lt;/p&gt;&#xA;" OwnerUserId="9757" LastActivityDate="2020-10-26T17:54:39.143" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14654" PostTypeId="2" ParentId="14652" CreationDate="2020-10-26T18:32:35.930" Score="1" Body="&lt;p&gt;Regarding your &lt;code&gt;grep&lt;/code&gt;, why not just grep for what you want, &lt;em&gt;i.e.&lt;/em&gt; &lt;code&gt;^ENSG.*&lt;/code&gt;? (These last few lines that htseq-count outputs - which is the software that produces your files - are really important though. You should absolutely look into them to see if something strange is going on with one or more samples.)&lt;/p&gt;&#xA;&lt;p&gt;Your error comes from the escaped quotes &lt;code&gt;\&amp;quot;&lt;/code&gt; you use here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;out.file2 &amp;lt;- merge(out.file, file3, by=c(\&amp;quot;gene_id\&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Before, you used:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;out.file = merge (file1, file2, by=c(&amp;quot;gene_id&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which is the correct way to quote here. On a sidenote, you don't need &lt;code&gt;c()&lt;/code&gt; if it's only a vector with one element. You can just write &lt;code&gt;by = &amp;quot;gene_id&amp;quot;&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here's a different way you could read the files:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;files &amp;lt;- list.files(path=&amp;quot;.&amp;quot;, pattern=&amp;quot;GSM&amp;quot;)&#xA;count_files &amp;lt;- lapply(files, read.table, col.names = c(&amp;quot;GeneID&amp;quot;, &amp;quot;Count&amp;quot;))&#xA;count_matrix &amp;lt;- do.call(cbind, lapply(count_files, &amp;quot;[[&amp;quot;, &amp;quot;Count&amp;quot;))&#xA;dimnames(count_matrix) &amp;lt;- list(count_files[[1]][&amp;quot;GeneID&amp;quot;], files)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9693" LastEditorUserId="73" LastEditDate="2021-07-23T21:52:56.403" LastActivityDate="2021-07-23T21:52:56.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14655" PostTypeId="2" ParentId="14636" CreationDate="2020-10-26T18:55:46.110" Score="3" Body="&lt;p&gt;Yes, PyMol is one of the best programs for rendering IMO. Usually, you would create folder with your PDB file and a file that contains all the commands to create your image like shown &lt;a href=&quot;https://pymolwiki.org/index.php/Gallery&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. You can create more than one file. Because PyMol is based on Python you can use all python syntax here. E.g. you can create multiple images and with for loops you can create series of images that you can combine to a movie later on.&lt;/p&gt;&#xA;&lt;p&gt;Other programs to check out would be VMD. I use &lt;a href=&quot;https://www.ks.uiuc.edu/Research/vmd/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VMD&lt;/a&gt; for quick videos and images. The PyMol images I like better most of the time.&lt;/p&gt;&#xA;&lt;p&gt;Or newer software like &lt;a href=&quot;https://documentation.samson-connect.net/whats-new-in-samson-2020-the-open-molecular-modeling-platform/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Samson modeling platform&lt;/a&gt;. I am not sure about the licensing conditions. Have not tried it yet.&lt;/p&gt;&#xA;&lt;p&gt;Biopython does not support the generation of 3D molecular images. It is more a library for sequence data.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2020-10-26T19:00:59.317" LastActivityDate="2020-10-26T19:00:59.317" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14656" PostTypeId="2" ParentId="14650" CreationDate="2020-10-26T19:35:17.060" Score="2" Body="&lt;p&gt;I am not sure there is a state of the art for that.&lt;/p&gt;&#xA;&lt;p&gt;Some possible strategies I can think of:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Applying a dictionary to identify language words and discard these.&lt;/li&gt;&#xA;&lt;li&gt;Using some context information like identifying some words in the same sentence than the sequence.&lt;/li&gt;&#xA;&lt;li&gt;I guess the format in papers for sequences will not be the same as for text. Maybe you can find a pattern.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I would use an incremental strategy. Use the simplest algorithm then see what it recovers. If there is garbage, find a strategy to remove some garbage with a common pattern. Repeat until you are happy with the result.&lt;/p&gt;&#xA;&lt;p&gt;Disclaimer: I have some experience in Natural Language Processing.&lt;/p&gt;&#xA;" OwnerUserId="5190" LastActivityDate="2020-10-26T19:35:17.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14657" PostTypeId="2" ParentId="14636" CreationDate="2020-10-26T20:16:19.093" Score="2" Body="&lt;p&gt;The other contributors have given great suggestions. I thought I ought to mention &lt;a href=&quot;https://michelanglo.sgc.ox.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Michelaɴɢʟo&lt;/a&gt;, which is an online PDB file/PDB code/PyMOL-PSE/gene-name visualiser, where one can create an interactive page, add a description with special protein links (&lt;em&gt;prolinks&lt;/em&gt;) that change the interactive protein view and then share the URL securely.&lt;/p&gt;&#xA;&lt;p&gt;It has a modal to make the selection easier, but uses the NGL selection language and to show a ligand, one would add a residue type prolink selecting &lt;code&gt;ligand&lt;/code&gt;. If the neighbourhood-showing radius setting is greater than 1.0 Å (default is 3.0 Å) one will likely see hydrogen bonds with the neighbours.&lt;/p&gt;&#xA;&lt;p&gt;One can try it without registering (free), but cannot keep the page for longer than a month or edit the description.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Disclaimer&lt;/strong&gt;: I am the author.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-10-26T20:16:19.093" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14658" PostTypeId="2" ParentId="14634" CreationDate="2020-10-26T20:56:47.840" Score="2" Body="&lt;p&gt;&lt;strong&gt;What that code snippet is doing:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;That snippet you found on github (where is that from exactly?) looks like it tries to run RepeatMasker's configure script, feeding it some options that are needed by RepeatModeler.  The cryptic third line generates and dumps a block of text to a file named tmp:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;2&#xA;path/to/bin/rmblastn&#xA;&#xA;5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And then passes that in to RepeatMasker's own configure script as though it were entered on the keyboard.  It's the equivalent of hitting enter to accept a default, entering &amp;quot;2&amp;quot; to select the RMBlast option, typing in the path to rmblastn, hitting enter to set RMBlast as the default, and then typing &amp;quot;5&amp;quot; to save and quit.&lt;/p&gt;&#xA;&lt;p&gt;That all looks fine, but I think this might be a problem with your (and mine, see below) RepeatMasker install rather than RepeatModeler itself.  These tools are new to me but I'm interesting in seeing this work too, so I gave a shot at troubleshooting it here.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The way I was able to get it working (note I assume you're working with conda packages here) was:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;pushd $CONDA_PREFIX/share/RepeatMasker&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;./util/buildRMLibFromEMBL.pl Libraries/RepeatMaskerLib.embl &amp;gt; Libraries/RepeatMasker.lib&lt;/code&gt; to &amp;quot;automatically create the old RepeatMasker.lib FASTA style database&amp;quot; as the script's comments put it.  &lt;strong&gt;NOTE: you might want to first delete or move the old empty file rather than alter it because of how conda shares data between environments.&lt;/strong&gt;  If you update it in place (which &lt;code&gt;&amp;gt;&lt;/code&gt; will still do), you'll alter the file for all conda environments using the package, and conda may complaint about unexpected file sizes when checking packages.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;perl configure&lt;/code&gt; to do RepeatMasker's setup including some makeblastdb commands using RepeatMasker.lib.  (Note this is RepeatMasker's configure script and not RepeatModeler's.) You can type in the options manually or you could probably use that code snippet you found.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;popd&lt;/code&gt; and run RepeatModeler/RepeatClassifier and hopefully it finds the BLAST library files now&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;All the gritty detail&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;I see a conda path in your logs.  Are you using the bioconda packages named repeatmodeler and repeatmasker?  It looks to me like the repeatmasker package in particular includes an empty RepeatMasker.lib file and that's the cause of the trouble.  I don't see any way that RepeatMasker.lib is set up automatically via the configure script, and the BLAST database files like RepeatMasker.lib.nsq that RepeatModeler expects are built from that (empty) file.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;Please rerun the configure program in the RepeatModeler directory&lt;/code&gt; message is a red herring, I think, in this case because the &lt;code&gt;$CONDA_PREFIX/share/RepeatMasker/Libraries/RepeatMasker.lib&lt;/code&gt; file &lt;em&gt;does&lt;/em&gt; exist, just not the other files for a database it assumes should exist.  If I instead run &lt;code&gt;configure&lt;/code&gt; for RepeatMasker &lt;em&gt;and&lt;/em&gt; update its script to not hide error messages, I see this coming from makeblastdb:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Building a new DB, current time: 10/26/2020 16:10:07&#xA;New DB name:   /home/jesse/miniconda3/envs/example-repeatmodeler/share/RepeatMasker/Libraries/RepeatMasker.lib&#xA;New DB title:  /home/jesse/miniconda3/envs/example-repeatmodeler/share/RepeatMasker/Libraries/RepeatMasker.lib&#xA;Sequence type: Nucleotide&#xA;Keep MBits: T&#xA;Maximum file size: 1000000000B&#xA;BLAST options error: File /home/jesse/miniconda3/envs/example-repeatmodeler/share/RepeatMasker/Libraries/RepeatMasker.lib is empty&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And it's true, that's an empty file provided by the conda package (the metadata in &lt;code&gt;$CONDA_PREFIX/conda-meta/repeatmasker-4.1.0-pl526_0.json&lt;/code&gt; shows &lt;code&gt;size_in_bytes: 0&lt;/code&gt; for that one so that's apparently &amp;quot;correct&amp;quot; for the package.)&lt;/p&gt;&#xA;&lt;p&gt;I saw a reference to that filename in a buildRMLibFromEMBL.pl script in the RepeatMasker share directory, and running that produces a FASTA file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./util/buildRMLibFromEMBL.pl Libraries/RepeatMaskerLib.embl &amp;gt; Libraries/RepeatMasker.lib&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then &lt;code&gt;./configure&lt;/code&gt; sees a non-empty .lib file and builds the database, and &lt;code&gt;RepeatClassifier&lt;/code&gt; finally sees the resulting database files.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;But why is this needed?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Shouldn't RepeatMasker come with a non-empty FASTA file if it's needed by its own configure script?  I can see &lt;a href=&quot;https://github.com/bioconda/bioconda-recipes/blob/master/recipes/repeatmasker/build.sh&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioconda's build.sh&lt;/a&gt; doesn't do anything in particular to handle that file, but, maybe it should?  This is probably where somebody who knows these tools better can chime in.)&lt;/p&gt;&#xA;" OwnerUserId="4980" LastEditorUserId="4980" LastEditDate="2020-10-27T16:54:09.440" LastActivityDate="2020-10-27T16:54:09.440" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14659" PostTypeId="2" ParentId="14646" CreationDate="2020-10-26T21:48:05.927" Score="1" Body="&lt;p&gt;To add complexity to this discussion, there are examples in some mitochondrial genomes where the stop point of a gene is based on the polyA-attached transcript, rather than the genomic sequence. I worked on a paper about &lt;em&gt;Nippostrongylus brasiliensis&lt;/em&gt; mtDNA, and needed polyA completion to explain the transcript expression I observed:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.12688/f1000research.10545.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.12688/f1000research.10545.1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There are plenty of other examples of gene transcription happening in a situation other than one-exon-per-fragment: transcription that's on &lt;a href=&quot;https://dx.doi.org/10.1101%2Fgr.1590904&quot; rel=&quot;nofollow noreferrer&quot;&gt;both sides of the DNA&lt;/a&gt;, exon sharing for different genes present on the same strand (they're usually &lt;a href=&quot;https://doi.org/10.1006/geno.1999.6112&quot; rel=&quot;nofollow noreferrer&quot;&gt;in the same frame&lt;/a&gt;, but there's the MT-ATP6/MT-ATP8 example that @Nuclear_Hoagie pointed out), &lt;a href=&quot;https://doi.org/10.1371/journal.pgen.1007764&quot; rel=&quot;nofollow noreferrer&quot;&gt;micropeptide&lt;/a&gt; expression from smaller chunks of genes. Some also have some degree of &lt;a href=&quot;https://doi.org/10.7554/eLife.01179.001&quot; rel=&quot;nofollow noreferrer&quot;&gt;stop-codon read-through&lt;/a&gt;, so the stop codons are not a strict stop in all cases. Some viruses are also &lt;a href=&quot;https://doi.org/10.1016/0042-6822(86)90011-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;particularly good&lt;/a&gt; at exploiting transcription / translation complexity.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2020-10-27T02:25:36.890" LastActivityDate="2020-10-27T02:25:36.890" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14660" PostTypeId="1" CreationDate="2020-10-26T22:53:38.813" Score="0" ViewCount="28" Body="&lt;p&gt;I'm trying to learn graphical analysis in a biological setting.&lt;/p&gt;&#xA;&lt;p&gt;The online course suggested using the Go Programming language, so I'm new to it and help to understand why &lt;strong&gt;the number of edges&lt;/strong&gt; that I read from the following code is different than from the site I'm downloading it from.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import (&#xA;    &amp;quot;fmt&amp;quot;&#xA;    &amp;quot;log&amp;quot;&#xA;&#xA;    &amp;quot;github.com/kortschak/graphprac&amp;quot;&#xA;)&#xA;&#xA;infile := &amp;quot;YeastL.dot&amp;quot;&#xA;g, err := graphprac.NewGraph(infile)&#xA;if err != nil {&#xA;    log.Fatalf(&amp;quot;failed to read graph: %v&amp;quot;, err)&#xA;}&#xA;{&#xA;    fmt.Printf(&amp;quot;G has %d nodes and %d edges.\n&amp;quot;, g.Nodes().Len(), g.Edges().Len())&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output I'm getting is: G has 2361 nodes and 6646 edges.&lt;/p&gt;&#xA;&lt;p&gt;But from the website &lt;a href=&quot;http://vlado.fmf.uni-lj.si/pub/networks/data/bio/Yeast/Yeast.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://vlado.fmf.uni-lj.si/pub/networks/data/bio/Yeast/Yeast.htm&lt;/a&gt; it's suppose to read 7182 edges.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="9759" LastActivityDate="2020-10-27T17:10:07.327" Title="Number of edges in Graphical Analysis using Jupyter with Go Programming language" Tags="|go|jupyter|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14661" PostTypeId="1" CreationDate="2020-10-27T02:17:19.380" Score="1" ViewCount="167" Body="&lt;p&gt;Both DESeq2 and edgeR have an FPKM/RPKM function that by default uses normalized library sizes (&amp;quot;robust&amp;quot; option in DESeq2). FPKMs have their own issues, but I thought the main benefit was to have comparable units across independent experiments. If you are adding an experiment-specific normalization factor, then why even use FPKMs?&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2020-10-27T07:23:12.487" Title="Why use &quot;robust&quot; FPKMs?" Tags="|rna-seq|deseq2|edger|fpkm|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14662" PostTypeId="2" ParentId="14661" CreationDate="2020-10-27T07:23:12.487" Score="5" Body="&lt;p&gt;FPKM are inherently experiment specific and can not be used to compare across samples. Let's consider the following two sequencing runs. Let &lt;span class=&quot;math-container&quot;&gt;$E1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$E2$&lt;/span&gt; be the true, underlying expression in two samples of genes 1-6. Let &lt;span class=&quot;math-container&quot;&gt;$S1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$S2$&lt;/span&gt; be the observed expression in our sequencing.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{matrix}&#xA;Gene &amp;amp; E1 &amp;amp; S1 &amp;amp; E2 &amp;amp; S2 \\&#xA;  G1 &amp;amp; 100 &amp;amp; 10 &amp;amp; 100 &amp;amp; 20 \\&#xA;  G2 &amp;amp; 100 &amp;amp; 10 &amp;amp; 100 &amp;amp; 20 \\&#xA;  G3 &amp;amp; 100 &amp;amp; 10 &amp;amp; 100 &amp;amp; 20 \\&#xA;  G4 &amp;amp; 100 &amp;amp; 10 &amp;amp; 0 &amp;amp; 0 \\&#xA;  G5 &amp;amp; 100 &amp;amp; 10 &amp;amp; 0 &amp;amp; 0 \\&#xA;  G6 &amp;amp; 100 &amp;amp; 10 &amp;amp; 0 &amp;amp; 0 \\&#xA;\end{matrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Our totals are 60 counts for &lt;span class=&quot;math-container&quot;&gt;$S1$&lt;/span&gt; and 60 for &lt;span class=&quot;math-container&quot;&gt;$S2$&lt;/span&gt;. We sequenced both libraries to the same depth, but as there are fewer genes expressed in &lt;span class=&quot;math-container&quot;&gt;$E2$&lt;/span&gt;, we just naturally capture the genes that are expressed more often. For simplicity's sake, let's assume the genes all have the same length of 1, so we ignore the &amp;quot;K&amp;quot; in FPKM. Let's also forget the scaling by a million, since that's just to make nice numbers:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{matrix}&#xA;Gene &amp;amp; E1 &amp;amp; S1 &amp;amp; E2 &amp;amp; S2 \\&#xA;  G1 &amp;amp; 100 &amp;amp; 0.167 &amp;amp; 100 &amp;amp; 0.333 \\&#xA;  G2 &amp;amp; 100 &amp;amp; 0.167 &amp;amp; 100 &amp;amp; 0.333 \\&#xA;  G3 &amp;amp; 100 &amp;amp; 0.167 &amp;amp; 100 &amp;amp; 0.333 \\&#xA;  G4 &amp;amp; 100 &amp;amp; 0.167 &amp;amp; 0 &amp;amp; 0 \\&#xA;  G5 &amp;amp; 100 &amp;amp; 0.167 &amp;amp; 0 &amp;amp; 0 \\&#xA;  G6 &amp;amp; 100 &amp;amp; 0.167 &amp;amp; 0 &amp;amp; 0 \\&#xA;\end{matrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now let's calculate size factors. In DESeq2, we create a pseudo-sample which is the geometric mean of all counts of each gene, so &lt;span class=&quot;math-container&quot;&gt;$\sqrt{10 \cdot 20} = 14.14$&lt;/span&gt; for genes 1-3 and genes 3-6 are ignored as they have a member with 0-valued counts. The size factor for each library is the median ratio for each library compared to that pseudo-sample:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;med([\frac{10}{14.14}, \frac{10}{14.14}, \frac{10}{14.14}) = 0.71&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;med([\frac{20}{14.14}, \frac{20}{14.14}, \frac{20}{14.14}]) = 1.41&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So where do we end up if we divide by the size factors?&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\begin{matrix}&#xA;Gene &amp;amp; E1 &amp;amp; S1 &amp;amp; E2 &amp;amp; S2 \\&#xA;  G1 &amp;amp; 100 &amp;amp; 141 &amp;amp; 100 &amp;amp; 142 \\&#xA;  G2 &amp;amp; 100 &amp;amp; 141 &amp;amp; 100 &amp;amp; 142 \\&#xA;  G3 &amp;amp; 100 &amp;amp; 141 &amp;amp; 100 &amp;amp; 142 \\&#xA;  G4 &amp;amp; 100 &amp;amp; 141 &amp;amp; 0 &amp;amp; 0 \\&#xA;  G5 &amp;amp; 100 &amp;amp; 141 &amp;amp; 0 &amp;amp; 0 \\&#xA;  G6 &amp;amp; 100 &amp;amp; 141 &amp;amp; 0 &amp;amp; 0 \\&#xA;\end{matrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So in this simple example, size factors produce a better estimate than FPKM or TPM to compare values &lt;em&gt;across experiments&lt;/em&gt;, hence the &amp;quot;robustness&amp;quot;. Please note that this is still not great for any statistical testing, and in general should be used with caution anywhere.&lt;/p&gt;&#xA;&lt;p&gt;Finally, I'll leave some more reading material concerning FPKMs and cross-sample comparisons here:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;[1] Dillies M.A., Briefings in Bioinf. 2012&lt;/li&gt;&#xA;&lt;li&gt;[2] Soneson &amp;amp; Delorenzi, BMC Bioinf. 2013&lt;/li&gt;&#xA;&lt;li&gt;[3] Lior Pachter’s lecture at CSHL (Cold Spring Harbour Laboratory) from minutes 32 on: &lt;a href=&quot;https://www.youtube.com/watch?v=5NiFibnbE8o&amp;amp;t=32m&quot; rel=&quot;noreferrer&quot;&gt;https://www.youtube.com/watch?v=5NiFibnbE8o&amp;amp;t=32m&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;[4] From Answer: &amp;quot;Over-correction&amp;quot; in the size-factors of the DESeq2 package: &lt;a href=&quot;https://support.bioconductor.org/p/80733/#80736&quot; rel=&quot;noreferrer&quot;&gt;https://support.bioconductor.org/p/80733/#80736&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;[5] &lt;a href=&quot;http://www.biomedcentral.com/1471-2105/14/370&quot; rel=&quot;noreferrer&quot;&gt;http://www.biomedcentral.com/1471-2105/14/370&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;[6] &lt;a href=&quot;https://liorpachter.wordpress.com/2014/04/30/estimating-number-of-transcripts-from-rna-seq-measurements-and-why-i-believe-in-paywall/&quot; rel=&quot;noreferrer&quot;&gt;https://liorpachter.wordpress.com/2014/04/30/estimating-number-of-transcripts-from-rna-seq-measurements-and-why-i-believe-in-paywall/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;[7] &lt;a href=&quot;http://rnajournal.cshlp.org/content/22/6/839&quot; rel=&quot;noreferrer&quot;&gt;http://rnajournal.cshlp.org/content/22/6/839&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9693" LastActivityDate="2020-10-27T07:23:12.487" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14663" PostTypeId="2" ParentId="14647" CreationDate="2020-10-27T10:29:03.810" Score="3" Body="&lt;p&gt;There are plenty of efficient and smarter solutions, but if you have so small tables, you can just use the very basics of R to do what you need:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# you need to adjust the following two lines to fit to your file formating and file names&#xA;genes_filt_table &amp;lt;- read.table(..., col.names = c('geneid'), stringsAsFactors = F)&#xA;norm_counts &amp;lt;- read.table(..., col.names = c('geneid', 'norm_counts'), sep = ',', stringsAsFactors = F)&#xA;&#xA;genes_to_filt &amp;lt;- norm_counts&lt;span class=&quot;math-container&quot;&gt;$geneid %in% genes_filt_table$&lt;/span&gt;geneid&#xA;filt_genes &amp;lt;- norm_counts[genes_to_filt, ]&#xA;non_filt_genes &amp;lt;- norm_counts[!genes_to_filt, ]&#xA;&#xA;write.table(filt_genes, 'filtered_genes.csv', quote = F, row.names = F, sep = ',')&#xA;write.table(non_filt_genes, 'non-filtered_genes.csv', quote = F, row.names = F, sep = ',')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This being said, there are like milions of more efficient solutions out there. I provided one that is using as basic R functions as possible, as it seemed that you would like to see a simple R solution.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2020-10-27T10:29:03.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14664" PostTypeId="2" ParentId="14660" CreationDate="2020-10-27T17:10:07.327" Score="2" Body="&lt;p&gt;the &lt;a href=&quot;https://github.com/kortschak/graphprac/blob/master/YeastL.dot&quot; rel=&quot;nofollow noreferrer&quot;&gt;dot file in the Git repo&lt;/a&gt; has 6646 edges, the &lt;a href=&quot;http://vlado.fmf.uni-lj.si/pub/networks/data/bio/Yeast/yeast.zip&quot; rel=&quot;nofollow noreferrer&quot;&gt;pajek file in your link&lt;/a&gt; has 7182.&lt;/p&gt;&#xA;&lt;p&gt;Chances are the latter has been updated since.&lt;/p&gt;&#xA;" OwnerUserId="9693" LastActivityDate="2020-10-27T17:10:07.327" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14666" PostTypeId="1" CreationDate="2020-10-27T21:17:25.187" Score="1" ViewCount="75" Body="&lt;p&gt;I have access to EPIC DNA methylation profiles from approximately 40 samples. I want to use these .idat files in order to predict copy number alterations within specific promoter regions of interest. Can you recommend any R packages that could perform this task?&lt;/p&gt;&#xA;&lt;p&gt;Through research I have stumbled upon epicopy (&lt;a href=&quot;https://github.com/sean-cho/Epicopy/blob/master/vignettes/Epicopy.Rmd&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/sean-cho/Epicopy/blob/master/vignettes/Epicopy.Rmd&lt;/a&gt;), but I'm uncertain whether this is the best package for this problem.&lt;/p&gt;&#xA;&lt;p&gt;Cheers,&#xA;GB&lt;/p&gt;&#xA;" OwnerUserId="9116" LastActivityDate="2020-10-27T21:17:25.187" Title="How to use EPIC DNAm files (.idat) to infer copy number alteration" Tags="|r|methylation|copy-number|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14667" PostTypeId="1" AcceptedAnswerId="14668" CreationDate="2020-10-27T23:57:54.333" Score="0" ViewCount="171" Body="&lt;p&gt;Does single cell data typically have the following meta-data: the &amp;quot;coordinates&amp;quot; (e.g. on a tissue, adjacent tissues) saying where each cell in the sample was located relative to other cells? If not, is it possible to reconstruct this with other meta-data on the cells?&lt;/p&gt;&#xA;&lt;p&gt;With the ultimate goal of working hands on with such location-tagged data, I am hoping for suggestions on the correct terms to search for, and even references to previous studies and/or easy to use public datasets.&lt;/p&gt;&#xA;&lt;p&gt;For visual reference of the idea I have in mind, consider this picture:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZxBUJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZxBUJ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I expect expression of any given gene in the skin cells in Condition 1 to be very different according to the location of the cell. For condition 2, I wouldn't expect the expression of a gene in the skin cells to be overly different, if at all. I want to see if I can formalize this idea using data mining techniques that were created for other purposes, but first I need the proper data.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;edit&lt;/strong&gt;: Here is an example of the format I eventually hope to work with.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Fake table 1&lt;/strong&gt;: Each row represents measurements taken for a unique cell, while the last three columns have coordinates for its location.&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{array}{r|lllllllll}&#xA;  \hline&#xA; cell &amp;amp; phenotype1 &amp;amp; phenotype2 &amp;amp; \dots &amp;amp; gene1.expr &amp;amp; gene2.expr &amp;amp; \dots &amp;amp; x.loc.coord &amp;amp; y.loc.coord &amp;amp; z.loc.coord \\ &#xA;  \hline&#xA;1 &amp;amp; 0.69 &amp;amp; 1.34 &amp;amp; \dots &amp;amp; 1.91 &amp;amp; 0.21 &amp;amp; \dots &amp;amp; 1.12 &amp;amp; 0.05 &amp;amp; 1.09 \\    2 &amp;amp; 0.34 &amp;amp; 0.92 &amp;amp; \dots &amp;amp; 1.74 &amp;amp; 2.03 &amp;amp; \dots &amp;amp; 0.57 &amp;amp; 0.46 &amp;amp; 0.24 \\ &#xA;   \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots &amp;amp; \vdots \\ &#xA;  n &amp;amp; 1.97 &amp;amp; 1.3 &amp;amp; 0.96 &amp;amp; 0.19 &amp;amp; 0.66 &amp;amp; \dots &amp;amp; 0.25 &amp;amp; 0.02 &amp;amp; 1.27 \\ &#xA;   \hline&#xA;\end{array}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, if the (x,y,z) coordinates of &lt;strong&gt;Fake Table 1&lt;/strong&gt; are unavailable or not reconstructable, are there datasets which help us construct an adjacency list of cell pairs according to their location, such as &lt;strong&gt;Fake Table 2&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Fake Table 2:&lt;/strong&gt; An adjacency table with pairs of cells which were next to each other when the measurement was taken.&#xA;&lt;span class=&quot;math-container&quot;&gt;\begin{array}{r|cc}&#xA;  \hline&#xA;pair &amp;amp; cell1 &amp;amp; cell2 \\ &#xA;  \hline&#xA;1 &amp;amp; 1 &amp;amp; 2 \\ &#xA;  2 &amp;amp; 1 &amp;amp; 3 \\ &#xA;  3 &amp;amp; 1 &amp;amp; 4 \\ &#xA;  4 &amp;amp; 2 &amp;amp; 3 \\ &#xA;  5 &amp;amp; 2 &amp;amp; 4 \\ &#xA;  \vdots &amp;amp; \vdots &amp;amp; \vdots \\ &#xA;  m-1 &amp;amp; n &amp;amp; n-2 \\ &#xA;  m &amp;amp; n &amp;amp; n-1 \\ &#xA;   \hline&#xA;\end{array}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I come from a statistics background with a basic understanding of this type of data from talks I've attended, but not hands on experience. I mostly just want to explore this type of data to inform my future work. I am open to any type of scRNA-seq data with cell location &amp;quot;coordinates,&amp;quot; if that's the correct thing to ask.&lt;/p&gt;&#xA;" OwnerUserId="9765" LastEditorUserId="9765" LastEditDate="2020-10-28T23:43:34.903" LastActivityDate="2022-04-16T14:39:35.190" Title="Where can I find Single Cell Data with Location &quot;Coordinates&quot;?" Tags="|rna-seq|single-cell|public-databases|differential-expression|gene-expression|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14668" PostTypeId="2" ParentId="14667" CreationDate="2020-10-28T02:14:18.140" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Does single cell data typically have the following meta-data: the&#xA;&amp;quot;coordinates&amp;quot; (e.g. on a tissue, adjacent tissues) saying where each&#xA;cell in the sample was located relative to other cells?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No. Typical scRNA-seq is just capturing random cells in a tube with no additional information. The technology you are looking for is spatial transcriptomics where you are measuring RNA levels at a particular location, but that is still not on the single-cell level.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If not, is it possible to reconstruct this with other meta-data on the&#xA;cells?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There have been efforts to reconstruct spatial relationship. For example: &lt;a href=&quot;https://doi.org/10.1038/nbt.3192&quot; rel=&quot;nofollow noreferrer&quot;&gt;Satija et al. 2015&lt;/a&gt;. There should be more recent approaches that I am not aware of.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2020-10-28T02:14:18.140" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14669" PostTypeId="1" CreationDate="2020-10-28T04:05:13.627" Score="1" ViewCount="402" Body="&lt;p&gt;I have been calculating the binding Affinity between a Ligand and a Protein using both AutoDock Vina and AutoDock tools but the binding affinity estimates obtained from the 2 molecular docking softwares are very dissimilar. For instance, in the example below the binding affinity estimated by AutoDock Vina is -12.6 but for the same Ligand and Protein the Binding Affinity according to AutoDock Tools is +158.0. Why would there be such a massive difference.&#xA;Thank you&lt;a href=&quot;https://i.stack.imgur.com/9H2lk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9H2lk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mMKkg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mMKkg.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9346" LastEditorUserId="9346" LastEditDate="2020-10-28T18:53:09.817" LastActivityDate="2020-10-28T18:53:09.817" Title="Discrepany between AutoDock Vina and AutoDock Tools" Tags="|statistics|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14670" PostTypeId="1" CreationDate="2020-10-28T08:44:32.440" Score="1" ViewCount="88" Body="&lt;p&gt;I have data from a lipidomic experiment comprising: 2 conditions and 5 samples in each condition; and for each lipid the value is a peak area.&#xA;I am doing a lipidomics analysis using the web app &lt;a href=&quot;http://www.lipidontology.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;lipid ontology&lt;/a&gt;, and they recommend to normalize the data prior to doing the analysis,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;There are many ways to normalize data, so LION/web cannot check&#xA;whether data is properly normalized. If you don't know how to&#xA;normalize the data, we recommend normalization by expressing all lipid&#xA;species as fraction of the sum. A popular website that supports&#xA;several normalization algorithms is MetaboAnalyst.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't know how to normalize my data, I cant find any paper related to lipidomics normalization to know to best way to do that, and also I can't find normalization function in the MetaboAnalyst.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="9010" LastEditorUserId="3848" LastEditDate="2020-10-29T21:56:35.120" LastActivityDate="2020-10-29T21:56:35.120" Title="Lipidomics normalization (peak area)" Tags="|lipid-bilayer|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14671" PostTypeId="1" CreationDate="2020-10-28T15:14:26.460" Score="2" ViewCount="37" Body="&lt;p&gt;This is the data I have now: 30 simple sequence repeat (SSR) markers for 80 cultivars of cucumber. 10 of the 80 cultivars belong to one cultivar (let's say A).&lt;/p&gt;&#xA;&lt;p&gt;My goal is to classify an unknown cultivar into A or &amp;quot;not A&amp;quot; using tge 30 SSR markers.&lt;/p&gt;&#xA;&lt;p&gt;I'm think of considering it as a classification problem (A vs non-A) and use machine learning method to build a model using the SSR markers as features. But the problem is that A cultivars don't have enough number of samples.&lt;/p&gt;&#xA;&lt;p&gt;Do you have any suggestions which statistical method(s) I can try to solve this problem? Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="886" LastEditorUserId="3848" LastEditDate="2023-04-15T15:58:16.270" LastActivityDate="2023-04-15T15:58:16.270" Title="Any suggestions for cultivar identification using SSR (simple sequence repeat) markers" Tags="|python|phylogenetics|phylogeny|machine-learning|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14672" PostTypeId="2" ParentId="14671" CreationDate="2020-10-28T16:21:35.617" Score="2" Body="&lt;p&gt;Use Fst (migration statistic) such as the Fstat program to generate a distance matrix and solve it via clustering, such as UPGMA or neighbor-joining. This analysis is particularly useful for diploids. I presume you have heterozygotes in your analysis. You can switch Fis (inbreeding) if you have excessive homozygosity.&lt;/p&gt;&#xA;&lt;p&gt;The other way in is a haplotype network. When I get a minute I'll post the link to the programs. Machine learning isn't needed in this instance, although you could do a PCA style analysis, i.e. unsupervised learning. Unsupervised learning doesn't need a large sample size.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The other solution is a stepwise mutation phylogeny, but this is for microsatellites (one to four repeats). This is called Delta Mu squared and describes the method of repeat unit mutation. It is a distance method.&lt;/p&gt;&#xA;&lt;p&gt;Again when I have a moment I will supply weblinks.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-10-29T00:22:43.677" LastActivityDate="2020-10-29T00:22:43.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14673" PostTypeId="1" CreationDate="2020-10-28T17:13:26.827" Score="1" ViewCount="338" Body="&lt;p&gt;I'm having trouble getting WDL's &lt;code&gt;read_json&lt;/code&gt; to work as expected when trying to create a &lt;code&gt;Map[String, Map[String, String]]&lt;/code&gt; data structure. If I explicitly declare the data structure it works, as shown below (the &lt;code&gt;gm&lt;/code&gt; variable):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow TestWorkflow {&#xA;&#xA;    String genome_choice&#xA;    String output_name = &amp;quot;xyz.txt&amp;quot;&#xA;    Map[String, Map[String,String]] gm = {&#xA;        &amp;quot;A&amp;quot;: {&amp;quot;gtf&amp;quot;:&amp;quot;A.gtf&amp;quot;, &amp;quot;idx&amp;quot;:&amp;quot;A.idx&amp;quot;}, &#xA;        &amp;quot;B&amp;quot;: {&amp;quot;gtf&amp;quot;:&amp;quot;B.gtf&amp;quot;, &amp;quot;idx&amp;quot;:&amp;quot;B.idx&amp;quot;} &#xA;    }&#xA;&#xA;    call something {&#xA;        input:&#xA;            genome_file_map = gm[genome_choice],&#xA;            outname = output_name&#xA;    }&#xA;&#xA;    output {&#xA;        File fout = something.fout&#xA;    }&#xA;}&#xA;&#xA;task something {&#xA;&#xA;    &#xA;    String outname&#xA;    Map[String, String] genome_file_map&#xA;&#xA;    String idx = genome_file_map[&amp;quot;idx&amp;quot;]&#xA;    String gtf = genome_file_map[&amp;quot;gtf&amp;quot;]&#xA;&#xA;    command {&#xA;        echo &lt;span class=&quot;math-container&quot;&gt;${idx} &amp;gt;&amp;gt; $&lt;/span&gt;{outname}&#xA;        echo &lt;span class=&quot;math-container&quot;&gt;${gtf} &amp;gt;&amp;gt; $&lt;/span&gt;{outname}&#xA;    }&#xA;    output {&#xA;        File fout = &amp;quot;${outname}&amp;quot;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;However&lt;/em&gt;, instead of explicitly writing &lt;code&gt;gm&lt;/code&gt;, I would like to read that from a file. If I then create a JSON-format file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;  &amp;quot;A&amp;quot;: {&amp;quot;gtf&amp;quot;:&amp;quot;A.gtf&amp;quot;, &amp;quot;idx&amp;quot;:&amp;quot;A.idx&amp;quot;},&#xA;  &amp;quot;B&amp;quot;: {&amp;quot;gtf&amp;quot;:&amp;quot;B.gtf&amp;quot;, &amp;quot;idx&amp;quot;:&amp;quot;B.idx&amp;quot;}&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and submit that as an input file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow TestWorkflow {&#xA;    ...&#xA;    File resource_map_jsonfile&#xA;    ...&#xA;    Map[String, Map[String,String]] gm = read_json(resource_map_jsonfile)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;it fails to parse, giving me:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ERROR: gm is declared as a Map[String, Map[String, String]] but the expression evaluates to a Object:&#xA;&#xA;    Map[String, Map[String,String]] gm = read_json(resource_map_jsonfile)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If I change the type declaration to &lt;code&gt;Object&lt;/code&gt;, then all the bracket-style indexing fails. So that's out.&lt;/p&gt;&#xA;&lt;p&gt;Curiously, if I instead declare &lt;code&gt;gm&lt;/code&gt; as a &lt;code&gt;Map[String,String]&lt;/code&gt; it then fails giving:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java.lang.IllegalArgumentException: No coercion defined from wom value(s) '{&amp;quot;gtf&amp;quot;: &amp;quot;B.gtf&amp;quot;, &amp;quot;idx&amp;quot;: &amp;quot;B.idx&amp;quot;}' of type 'Map[String, String]' to 'String'.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;so clearly it is parsing the JSON file correctly, as it identifies that nested &lt;code&gt;Map[String, String]&lt;/code&gt;. I can probably find ways around this (using other &lt;code&gt;read_*&lt;/code&gt; functions), but that's less than ideal.&lt;/p&gt;&#xA;" OwnerUserId="9769" LastActivityDate="2020-10-28T17:13:26.827" Title="Reading nested Map data structures in WDL" Tags="|wdl|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14674" PostTypeId="2" ParentId="14640" CreationDate="2020-10-28T18:59:35.143" Score="0" Body="&lt;p&gt;Which software package around you using, PRSice? Generally missing values comprising half the total observations are extremely difficult to calculate using regression analysis (or many statistical analyses for that matter.&lt;/p&gt;&#xA;&lt;p&gt;One way forward is to include (an) example(s) of all 40 variants into your data set and perform the analysis with and without the added data, then compare this against a data set where all 40 variants are observed and your samples are excluded. You might find the missing values have low probability of PRS and this would then explain your results.&lt;/p&gt;&#xA;&lt;p&gt;Missing analysis in regression is a specialist area of statistics and is not for the faint hearted, so a work-around is the best way forward.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-10-28T18:59:35.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14675" PostTypeId="2" ParentId="14671" CreationDate="2020-10-28T20:42:36.127" Score="2" Body="&lt;p&gt;Not to discount @Michael's answer (unsupervised learning is very handy for descriptive analysis of this kind of problem), but classification based on microsatellite data should not be too difficult. For papers with examples, you can see e.g. &lt;a href=&quot;https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0143465&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; and &lt;a href=&quot;https://link.springer.com/article/10.1007/s10722-015-0307-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. They seem to apply various algorithms, you might be able to learn something from those papers.&lt;/p&gt;&#xA;&lt;p&gt;More generally, I bet that you could get pretty far with just (multinomial) &lt;a href=&quot;https://en.wikipedia.org/wiki/Multinomial_logistic_regression&quot; rel=&quot;nofollow noreferrer&quot;&gt;logistic regression&lt;/a&gt; for your outcome variables.&lt;/p&gt;&#xA;&lt;p&gt;Of course, this depends to a certain extent on how you code the data and what the input data look like. Microsatellite allele coding and evolutionary/error models are a field in themselves.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-10-28T20:42:36.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14676" PostTypeId="2" ParentId="14667" CreationDate="2020-10-28T20:55:12.547" Score="1" Body="&lt;p&gt;Is any of this what you want?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://support.10xgenomics.com/spatial-gene-expression/datasets&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.10xgenomics.com/spatial-gene-expression/datasets&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-10-28T20:55:12.547" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14677" PostTypeId="2" ParentId="14667" CreationDate="2020-10-29T00:55:41.587" Score="0" Body="&lt;p&gt;A good solution is to use Human papilloma virus (HPV) PCR (see below) or particularly RNA expression of protein E5, E6 and E7 of the viral genome in skin cells. High expression of E5, E6 or E7 would be diagnostic of skin cancer because of over deregulation of latency of the virus. This is because the viral control regions (L1/L2? - I forget) get disrupted mostly by genome integration leading to E5-E7 disrupting the human cell cycle control.&lt;/p&gt;&#xA;&lt;p&gt;Over expression would be diagnositic because alot (most? all?) skin cancer is a result of HPV. HPV types 16 and 18 are the key, but most work done in this area focuses on cervical cancer because cervical HPV infection has a high mortality rate. I suspect skin cancer has a comparatively low mortality rate (but is of considerable imporant of course).&lt;/p&gt;&#xA;&lt;p&gt;Alot of work has been done on the tandem arrays of HPV genomic insertion into the human geneome disrupting the regulatory genes leading to over production of E5-E7 and again is considered important in malignancy. However, its the protein expression of (E5), E6 and E7 that is critical. It is not clear whether human genome integration resulting in a viral tandem arrays that disrupt viral proteins L1/L2 will automatically trigger cancer, or whether it is merely a high risk factor (i.e. tandem array human genome insertions could still remain benign).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-10-29T00:55:41.587" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14678" PostTypeId="1" AcceptedAnswerId="14679" CreationDate="2020-10-29T10:03:10.260" Score="1" ViewCount="488" Body="&lt;p&gt;I have around 10,000 FASTA files of Influenza A virus.&lt;/p&gt;&#xA;&lt;p&gt;These files contains sequences of each of the 8 segments of the viral genome and I want to separate each genome segment into a different folder and subdivide each directory into different files based on their geographic region of isolation.&lt;/p&gt;&#xA;&lt;p&gt;In each FASTA file for each segment,the first line has the segment number, for example.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;KM368312.1 Influenza A virus (A/swine/Shandong/01/2009(H1N1)) segment 3 polymerase PA (PA) and PA-X protein (PA-X) genes, complete cds&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;To be clear I want&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;file1.txt has segment 1&lt;/li&gt;&#xA;&lt;li&gt;file2.txt has segment 2&#xA;etc ...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I want to ensure all segment 1 sequences are grouped into one folder, and each file is grouped according to its geographic origin. The geographic groupings are mirrored for all 8 segements and each placed into their own directory.&lt;/p&gt;&#xA;" OwnerUserId="9772" LastEditorUserId="3848" LastEditDate="2022-03-11T17:19:18.870" LastActivityDate="2022-03-11T17:19:18.870" Title="How to sort multiple FASTA files based on their content?" Tags="|python|covid-19|virus|" AnswerCount="2" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14679" PostTypeId="2" ParentId="14678" CreationDate="2020-10-29T10:19:46.490" Score="4" Body="&lt;p&gt;The general idea is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;iterate over all files&#xA;&lt;ol&gt;&#xA;&lt;li&gt;read the first line of each file, extract the &lt;code&gt;segment&lt;/code&gt; bit&lt;/li&gt;&#xA;&lt;li&gt;move the file based on that&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In Python, this can be done in many ways. Here’s one way:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import os&#xA;import re&#xA;&#xA;&#xA;def find_segment(id):&#xA;    return re.search(r'segment (\d)', id).group(1)&#xA;&#xA;&#xA;for segment in range(8):&#xA;    os.mkdir(f'segment-{segment + 1}')&#xA;&#xA;for entry in os.scandir():&#xA;    if not entry.is_file():&#xA;        continue&#xA;&#xA;    with open(entry.path, 'r') as file:&#xA;        segment = find_segment(next(file))&#xA;    os.rename(entry.path, f'segment-{segment}/{entry.path}')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note the absence of error handling — this code assumes that every operation succeeds; empty files, or files without “&lt;code&gt;segment X&lt;/code&gt;” in their FASTA ID, will trigger an error, as will the usual IO failures (missing permissions, etc.).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;But most people would probably use a shell scripting language instead of Python to solve this. For example, here’s something equivalent in Bash (untested):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;mkdir segment-{1..8}&#xA;&#xA;for file in *.fa *.fasta; do&#xA;    segment=&lt;span class=&quot;math-container&quot;&gt;$(sed 's/.*segment \([[:digit:]]\).*/\1/; q' &quot;$&lt;/span&gt;file&amp;quot;)&#xA;    if [[ -n &lt;span class=&quot;math-container&quot;&gt;$segment ]]; then&#xA;    mv &quot;$&lt;/span&gt;file&amp;quot; &amp;quot;segment-$segment&amp;quot;&#xA;    done&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29" LastEditorUserId="29" LastEditDate="2020-12-17T11:05:16.237" LastActivityDate="2020-12-17T11:05:16.237" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14680" PostTypeId="2" ParentId="14670" CreationDate="2020-10-29T11:11:39.693" Score="1" Body="&lt;p&gt;Basically and briefly,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Normalisation = transforming a data set to conform to a normal distribution. Converting to a frequency is NEVER a normalised transformation&lt;/li&gt;&#xA;&lt;li&gt;Standardisation = ensuring each point of the data set is comparable to the others, either directly or relatively depending on the calculation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The information you have received is incorrect, moreover you only convert to a frequency for a final assessment of the data and in my experience NEVER as an intermediate step. Normally data point between 0 and 1 are bad news and often require a transformation to get them away from fractions.&lt;/p&gt;&#xA;&lt;p&gt;I don't understand the behaviour of the variance involved in metabolism and particularly lipids, however you should refer this issue to &amp;quot;cross-validation&amp;quot; Stackexchange. I did have some understanding of enzyme kinetics but in a different area.&lt;/p&gt;&#xA;&lt;p&gt;Cross-Validation Stackexchance statisticians will understand less of the biology than I do (which is a bit worrying of course), so you will need to give them quite detailed information on the data set. They will almost certain support my concern of the advice you are given and provide a better method of standardisation. One method is to take the difference in mean and divide it by the standard deviation, you can then normalise the standardised data. They will want to know the goals of your investigation to understand who to process the data further.&lt;/p&gt;&#xA;&lt;p&gt;What I can say is variation between runs is often an issue (in your apparatus perhaps not, I dunno) and standardising against a common control is one approach to overcome this, e.g. the control same common to all runs would be the mean about which all other data is centred.&lt;/p&gt;&#xA;&lt;p&gt;What I suspect is that lipids are less popular than other fields, so standard protocol may not be observed.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-10-29T11:17:21.267" LastActivityDate="2020-10-29T11:17:21.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14681" PostTypeId="1" CreationDate="2020-10-29T13:40:29.163" Score="0" ViewCount="337" Body="&lt;p&gt;Im coding in c++ and reading in different reference genomes to examine regions across the chromosomes a few hundred basepairs at a time.&lt;/p&gt;&#xA;&lt;p&gt;To do it in c++ i use the library htslib and the command faidx_fetch_seq() which is similar to read specific regions using samtools faidx.&lt;/p&gt;&#xA;&lt;p&gt;The problem is when im reading in the reference genomes, there are of course multiple regions consisting only of NNNNN (and so on). Since i only need the actual DNA sequence i wish to read in the entire file but removing the regions with NNNN.&lt;/p&gt;&#xA;&lt;p&gt;I have tried reading in the regions in c++ and then just creating an if-statement to check the actual bases consist of N. This works but is incredibly slow.&lt;/p&gt;&#xA;&lt;p&gt;So my question is: do anyone know how to only read in the actual DNA sequences, removing the NNN regions? either with samtools or with the c++ funciton faidx_fetch_seq().&lt;/p&gt;&#xA;" OwnerUserId="8630" LastActivityDate="2020-10-29T23:27:32.803" Title="Reading fasta sequence regions with faidx" Tags="|samtools|phylogenetics|c++|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14682" PostTypeId="1" AcceptedAnswerId="14706" CreationDate="2020-10-29T17:50:26.723" Score="1" ViewCount="52" Body="&lt;p&gt;I'm trying to reuse some code I have previously used to add extra parameters.&#xA;In the initial code, it was used to give a 1/0 counter if a specific mutation was present (first code below)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;import os&#xA;import pandas as pd&#xA;import logging&#xA;import sys&#xA;import csv&#xA;&#xA;root = &amp;quot;C:&amp;quot;&#xA;&#xA;os.chdir(root)&#xA;&#xA;lineages = os.listdir('Lineages')&#xA;mutations = os.listdir('Mutations')&#xA;&#xA;for lineage in lineages:&#xA;    &#xA;    df = pd.read_csv('Lineages/' + lineage).dropna()&#xA;    df.columns = [&amp;quot;lineage&amp;quot;]&#xA;    df['lineage'] = [x.split('|')[0].split('_')[-1]&#xA;                     for x in df['lineage']]&#xA;               &#xA;&#xA;    for mutation in mutations:&#xA;        &#xA;        mutation_df = pd.read_csv('Mutations/' + mutation).dropna()&#xA;        mutation_header = mutation.replace(&amp;quot;.txt&amp;quot;, &amp;quot;&amp;quot;)&#xA;        mutation_df.columns = [mutation_header]&#xA;        mutation_df[mutation_header] = [x.split('_')[-1].split('|')[0] &#xA;                          for x in mutation_df[mutation_header]]&#xA;        &#xA;        mutation_set = set(mutation_df[mutation_header])&#xA;        &#xA;        df[mutation_header] = [1 if x in mutation_set else 0&#xA;                               for x in df['lineage']]&#xA;        &#xA;    df.to_csv('Results9/' + lineage.replace(&amp;quot;.txt&amp;quot;, &amp;quot;&amp;quot;) + '.csv',&#xA;                 header = True, &#xA;                 index = False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, I now wish to add extra parameters such as date of sample and location of sample.&#xA;The first code produces this type of counter system:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;lineage,A222V,D614G,E484Q,E780Q,G476S,L18F,N439K,S477,S477N,T478I,V483A&#xA;417941,0,1,0,0,0,0,0,0,0,0,0&#xA;417942,0,1,0,0,0,0,0,0,0,0,0&#xA;417944,0,1,0,0,0,0,0,0,0,0,0&#xA;417946,0,1,0,0,0,0,0,0,0,0,0&#xA;417947,0,1,0,0,0,0,0,0,0,0,0&#xA;417948,0,1,0,0,0,0,0,0,0,0,0&#xA;417950,0,0,0,0,0,0,0,0,0,0,0&#xA;417955,0,1,0,0,0,0,0,0,0,0,0&#xA;431011,0,1,0,0,0,0,0,0,0,0,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This code is fine and working, but if I try and add the second parameter (date/location) with the following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;import os&#xA;import pandas as pd&#xA;import logging&#xA;import sys&#xA;import csv&#xA;&#xA;root = &amp;quot;C:/Users/jones/Downloads/SARSCOV2/&amp;quot;&#xA;&#xA;os.chdir(root)&#xA;&#xA;lineages = os.listdir('Lineages')&#xA;mutations = os.listdir('Mutations')&#xA;dates = os.listdir('Dates')&#xA;locations = os.listdir('Locations')&#xA;&#xA;for date in dates:&#xA;            &#xA;    date_df = pd.read_csv('Dates/' + date).dropna()&#xA;    date_df.columns = [&amp;quot;Date&amp;quot;]&#xA;    date_df['Date'] = [x.split('|')[-1].split('/')[-1] &#xA;                       for x in date_df['Date']]&#xA;            &#xA;    for location in locations:&#xA;                &#xA;        location_df = pd.read_csv('Locations/' + location).dropna()&#xA;        location_df.columns = [&amp;quot;Location&amp;quot;]&#xA;        location_df['Location'] = [x.split('/')[1].split('_')[0]&#xA;                                   for x in location_df['Location']]&#xA;            &#xA;        for lineage in lineages:&#xA;    &#xA;            df = pd.read_csv('Lineages/' + lineage).dropna()&#xA;            df.columns = [&amp;quot;lineage&amp;quot;]&#xA;            df['lineage'] = [x.split('_')[-1].split('|')[0]&#xA;                             for x in df['lineage']]&#xA;               &#xA;&#xA;            for mutation in mutations:&#xA;        &#xA;                mutation_df = pd.read_csv('Mutations/' + mutation).dropna()&#xA;                mutation_header = mutation.replace(&amp;quot;.txt&amp;quot;, &amp;quot;&amp;quot;)&#xA;                mutation_df.columns = [mutation_header]&#xA;                mutation_df[mutation_header] = [x.split('_')[-1].split('|')[0] &#xA;                                                for x in mutation_df[mutation_header]]&#xA;        &#xA;                mutation_set = set(mutation_df[mutation_header])&#xA;        &#xA;                df[mutation_header] = [1 if x in mutation_set else 0&#xA;                                       for x in df['lineage']]&#xA;   &#xA;  &#xA;            df.to_csv('Results_3/' + lineage.replace(&amp;quot;.txt&amp;quot;, &amp;quot;&amp;quot;) + '.csv',&#xA;                 header = True, &#xA;                 index = False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I retrieve this data, where the counting is maintaibned, but the new parameters (DATE/LOCATION) are not added&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;lineage,A222V,D614G,E484Q,E780Q,G476S,L18F,N439K,S477,S477N,T478I,V483A&#xA;417941,0,1,0,0,0,0,0,0,0,0,0&#xA;417942,0,1,0,0,0,0,0,0,0,0,0&#xA;417944,0,1,0,0,0,0,0,0,0,0,0&#xA;417946,0,1,0,0,0,0,0,0,0,0,0&#xA;417947,0,1,0,0,0,0,0,0,0,0,0&#xA;417948,0,1,0,0,0,0,0,0,0,0,0&#xA;417950,0,0,0,0,0,0,0,0,0,0,0&#xA;417955,0,1,0,0,0,0,0,0,0,0,0&#xA;431011,0,1,0,0,0,0,0,0,0,0,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In short, my end goal is to:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Add Lineage ID|Location of Sample|DD/MM/YYY|Mutations| -- as the headers&lt;/li&gt;&#xA;&lt;li&gt;still maintain the mutation counter&lt;/li&gt;&#xA;&lt;li&gt;Use the mutation counter as a guide to determine where and when the genotypes (mutation combinations) arose.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The files where I am retrieving the parameters looks like this also:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;hCoV-19/Singapore/4/2020|EPI_ISL_410535|2020-02-03&#xA;&#xA;hCoV-19/USA/WA13-UW9/2020|EPI_ISL_413601|2020-03-02&#xA;&#xA;hCoV-19/USA/WA-UW142/2020|EPI_ISL_416680|2020-03-11&#xA;&#xA;hCoV-19/USA/WA-UW143/2020|EPI_ISL_416681|2020-03-11&#xA;&#xA;hCoV-19/USA/WA-UW144/2020|EPI_ISL_416682|2020-03-09&#xA;&#xA;hCoV-19/USA/WA-UW145/2020|EPI_ISL_416683|2020-03-15&#xA;&#xA;hCoV-19/USA/WA-UW146/2020|EPI_ISL_416684|2020-03-14&#xA;&#xA;hCoV-19/USA/WA-UW147/2020|EPI_ISL_416685|2020-03-15&#xA;&#xA;hCoV-19/USA/WA-UW148/2020|EPI_ISL_416686|2020-03-14&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A very laborious, and poorly explained question; I apologise in advance!&lt;/p&gt;&#xA;" OwnerUserId="9672" LastEditorUserId="3848" LastEditDate="2020-12-03T17:07:16.873" LastActivityDate="2020-12-03T17:07:16.873" Title="Adding multiple dataframes to compare against genotypes - DATE/LOCATION - geotemporal incidences of SARS2 genotypes" Tags="|covid-19|sars-cov-2|python|pandas|dataframe|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14684" PostTypeId="2" ParentId="14681" CreationDate="2020-10-29T23:27:32.803" Score="1" Body="&lt;p&gt;If you can't read in the entire sequence for a given reference genome:&lt;/p&gt;&#xA;&lt;p&gt;(1) Use a &lt;a href=&quot;https://www.danielecook.com/generate-a-bedfile-of-masked-ranges-a-fasta-file/&quot; rel=&quot;nofollow noreferrer&quot;&gt;script&lt;/a&gt; to generate a BED file of masked regions of low complexity (i.e. where you find &lt;code&gt;N&lt;/code&gt;s in your FASTA file). (2) Use &lt;code&gt;bedops --complement&lt;/code&gt; with this mask file and the output of UCSC &lt;code&gt;fetchChromSizes&lt;/code&gt; to generate a BED file containing regions that are &lt;em&gt;unmasked&lt;/em&gt;. (3) Repeat for other reference genomes, as needed.&lt;/p&gt;&#xA;&lt;p&gt;You can now loop over the unmasked regions in this file, making calls to &lt;code&gt;samtools faidx&lt;/code&gt; or &lt;code&gt;faidx_fetch_seq()&lt;/code&gt;. These calls will return unmasked sequence.&lt;/p&gt;&#xA;&lt;p&gt;For performance or convenience, you could read chunks of sequence into a &lt;code&gt;std::deque&lt;/code&gt; or &lt;code&gt;std::vector&lt;/code&gt; buffer and then read through that however many hundreds of bases, refreshing the buffer, as needed.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2020-10-29T23:27:32.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14685" PostTypeId="1" AcceptedAnswerId="14686" CreationDate="2020-10-29T07:49:03.027" Score="0" ViewCount="135" Body="&lt;p&gt;So I have around 10,000 FASTA files of Influenza A virus. These files contains sequences of each of the 8 segments of the viral genome and I want to separate these files into different locations based on the content of these FASTA files.&#xA;In each FASTA file, first line has the segment number.&#xA;Eg.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;KM368312.1 Influenza A virus (A/swine/Shandong/01/2009(H1N1)) segment 3 polymerase PA (PA) and PA-X protein (PA-X) genes, complete cds&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Okay imagine you have 10,000 text files in one folder and there are 8 types of these text files based on the content of them. Eg. file1.txt has segment 1 written in it, file2.txt has segment 2 written in it upto segment 8. So I want to group the files containing segment 1 together in one folder, segment 2 in other folder together and so on.&lt;/p&gt;&#xA;" OwnerUserId="9772" OwnerDisplayName="Neeleshwar Pandey" LastEditorUserId="3848" LastEditDate="2022-03-08T13:42:26.427" LastActivityDate="2022-03-08T13:42:26.427" Title="How to sort multiple FASTA files based on their content?" Tags="|python|covid-19|virus|" AnswerCount="2" CommentCount="11" ClosedDate="2020-10-30T11:34:53.863" ContentLicense="CC BY-SA 4.0" />
  <row Id="14686" PostTypeId="2" ParentId="14685" CreationDate="2020-10-29T09:59:25.047" Score="2" Body="&lt;p&gt;That's quite easy with &lt;code&gt;cmd&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@echo off&#xA;for /l %%i in (1,1,8) do (&#xA;  for /f &amp;quot;delims=&amp;quot; %%f in ('findstr  /mc:&amp;quot; segment %%i &amp;quot; *.txt') do (&#xA;    ECHO move &amp;quot;%%f&amp;quot; &amp;quot;segment %%i\&amp;quot;&#xA;  )&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;findstr /mc:&amp;quot;string&amp;quot; *.txt&lt;/code&gt; searches each matching file for the string and outputs the filename, if the string was found. The &lt;code&gt;for /f&lt;/code&gt; loop around catches those filenames, so you are able to move them. The &lt;code&gt;for /l&lt;/code&gt; loop just iterates from 1 to 8.&lt;/p&gt;&#xA;&lt;p&gt;See &lt;code&gt;for /?&lt;/code&gt; and &lt;code&gt;findstr /?&lt;/code&gt; for more information about those commands.&lt;/p&gt;&#xA;&lt;p&gt;You probably want to adapt the filemask &lt;code&gt;*.txt&lt;/code&gt; to your needs.&lt;/p&gt;&#xA;&lt;p&gt;Note: I &amp;quot;disarmed&amp;quot; the &lt;code&gt;move&lt;/code&gt; command for troubleshooting. Remove the &lt;code&gt;ECHO&lt;/code&gt; when you are sure it does what you want.&lt;/p&gt;&#xA;" OwnerDisplayName="Stephan" LastActivityDate="2020-10-29T09:59:25.047" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14687" PostTypeId="2" ParentId="14685" CreationDate="2020-10-29T10:08:58.387" Score="1" Body="&lt;p&gt;The general idea is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;iterate over all files&#xA;&lt;ol&gt;&#xA;&lt;li&gt;read the first line of each file, extract the &lt;code&gt;segment&lt;/code&gt; bit&lt;/li&gt;&#xA;&lt;li&gt;move the file based on that&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In Python, this can be done in many ways. Here’s one way:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import os&#xA;import re&#xA;&#xA;&#xA;def find_segment(id):&#xA;    return re.search(r'segment (\d)', id).group(1)&#xA;&#xA;&#xA;for segment in range(8):&#xA;    os.mkdir(f'segment-{segment + 1}')&#xA;&#xA;for entry in os.scandir():&#xA;    if not entry.is_file():&#xA;        continue&#xA;&#xA;    with open(entry.path, 'r') as file:&#xA;        segment = find_segment(next(file))&#xA;    os.rename(entry.path, f'segment-{segment}/{entry.path}')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note the absence of error handling — this code assumes that every operation succeeds; empty files, or files without “&lt;code&gt;segment X&lt;/code&gt;” in their FASTA ID, will trigger an error, as will usual IO failures (missing permissions, etc.).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;But many people wouldn’t use Python to solve this but a shell scripting language. For example, here’s something equivalent in Bash (untested):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;mkdir segment-{1..8}&#xA;&#xA;for file in *.fa *.fasta; do&#xA;    segment=$(sed 's/.*segment \([[:digit:]]\).*/\1/; q' &amp;quot;$file&amp;quot;)&#xA;    if [[ &amp;quot;$segment&amp;quot; != '' ]]; then&#xA;        mv &amp;quot;$file&amp;quot; &amp;quot;segment-$segment&amp;quot;&#xA;    done&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="29" OwnerDisplayName="Konrad Rudolph" LastActivityDate="2020-10-29T12:08:45.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14688" PostTypeId="1" AcceptedAnswerId="14689" CreationDate="2020-10-30T04:22:45.757" Score="1" ViewCount="45" Body="&lt;p&gt;I am trying to download sequence alignments for families against the UniProtKB sequence database as shown in the example below using R code. Can someone kindly suggest me how to?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;https://pfam.xfam.org/family/PF02171/alignment/uniprot/format?format=pfam&amp;amp;alnType=uniprot&amp;amp;order=t&amp;amp;case=l&amp;amp;gaps=default&amp;amp;download=1 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9780" LastEditorUserId="9780" LastEditDate="2020-10-30T06:09:27.930" LastActivityDate="2020-10-30T06:09:27.930" Title="How to download sequence alignments against UniProtKB sequence database for a particular Pfam Id using R?" Tags="|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14689" PostTypeId="2" ParentId="14688" CreationDate="2020-10-30T05:40:17.413" Score="2" Body="&lt;p&gt;The following code served my purpose.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(readr)&#xA;pfam_url &amp;lt;-&#xA;  'https://pfam.xfam.org/family/PF02171/alignment/uniprot/format?format=pfam&amp;amp;alnType=uniprot&amp;amp;order=t&amp;amp;case=l&amp;amp;gaps=default&amp;amp;download=1'&#xA;seqs &amp;lt;-&#xA;  read_table(&#xA;    pfam_url,&#xA;    col_names = FALSE,&#xA;    col_types = cols(X1 = col_character(), X2 = col_character())&#xA;  )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9780" LastEditorUserId="9780" LastEditDate="2020-10-30T06:07:00.517" LastActivityDate="2020-10-30T06:07:00.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14690" PostTypeId="1" AcceptedAnswerId="14693" CreationDate="2020-10-30T11:04:04.980" Score="2" ViewCount="289" Body="&lt;p&gt;My data frame compares the RNA-seq reads from many genes in different tissues.&#xA;The reads look as follows.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/48HxQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/48HxQ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I tried using log to make it better but still looks pretty skewed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Fdrqy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Fdrqy.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Are there any better transformations/functions for this in R?&lt;/p&gt;&#xA;&lt;p&gt;This is part of an assignment I have, I have used in the past edge software and agree it will do a better job but in this work I'm not meant to use any of the pipelines.&lt;/p&gt;&#xA;" OwnerUserId="9669" LastEditorUserId="73" LastEditDate="2020-11-01T11:51:58.567" LastActivityDate="2020-11-01T12:33:57.263" Title="I have really skewed RNA-seq data, what's the best way to normalise it? Preferably in R!" Tags="|r|rna-seq|homework|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14691" PostTypeId="1" AcceptedAnswerId="14692" CreationDate="2020-10-30T12:16:36.480" Score="0" ViewCount="441" Body="&lt;p&gt;I am trying to do GO analysis. However, when I run DAVID, I am getting this error:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;You are either not sure which identifier type your list contains, or less than 80% of your list has mapped to your chosen identifier type. Please use the Gene Conversion Tool to determine the identifier type.&#xA;In addition, when I am trying to add a column with gene names next to esemble ids in R in the file with differential expression genes, many genes say NA under gene name column.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I have used STAR and Salmon for the alignment,and DESEQ2 for differential expression analysis.&lt;br /&gt;&#xA;I have used premade indexes for both tools.&lt;br /&gt;&#xA;Thank you in advance!&lt;/p&gt;&#xA;&lt;p&gt;Some of the ENSEMBL codes I used:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;ENSG00000249200 ENSG00000239920 ENSG00000242295 ENSG00000235763 ENSG00000226933 ENSG00000283632 ENSG00000235838 ENSG00000232429 ENSG00000281076 ENSG00000271042 ENSG00000229119 ENSG00000253384 ENSG00000285645 ENSG00000240627 ENSG00000213590 ENSG00000270739 ENSG00000240201 ENSG00000203531 ENSG00000271626 ENSG00000250316 ENSG00000218713 ENSG00000231548 ENSG00000231049 ENSG00000286185 ENSG00000214535 ENSG00000212961 ENSG00000219747 ENSG00000259100 ENSG00000231767 ENSG00000243175 ENSG00000236804 ENSG00000240634 ENSG00000270773 ENSG00000230267&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9656" LastEditorUserId="48" LastEditDate="2020-11-03T12:21:52.703" LastActivityDate="2020-11-03T12:21:52.703" Title="less than 80% of your list has mapped to your chosen identifier type" Tags="|rna-seq|go-enrichment|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14692" PostTypeId="2" ParentId="14691" CreationDate="2020-10-30T12:41:07.907" Score="2" Body="&lt;p&gt;DAVID last received an update in May 2016 (&lt;a href=&quot;https://david.ncifcrf.gov/content.jsp?file=update.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://david.ncifcrf.gov/content.jsp?file=update.html&lt;/a&gt;), so any additions to the Gene Ontology database since then won't be present. I checked the examples you posted in &lt;a href=&quot;http://www.ensembl.org/biomart/martview/8b2b5b98dc138d40cfcb7843544f6922?VIRTUALSCHEMANAME=default&amp;amp;ATTRIBUTES=hsapiens_gene_ensembl.default.feature_page.ensembl_gene_id%7Chsapiens_gene_ensembl.default.feature_page.go_id&amp;amp;FILTERS=hsapiens_gene_ensembl.default.filters.ensembl_gene_id.%22ENSG00000249200ENSG00000239920,ENSG00000242295,ENSG00000235763,ENSG00000226933,ENSG00000283632,ENSG00000235838,ENSG00000232429,ENSG00000281076,ENSG00000271042,ENSG00000229119,ENSG00000253384,ENSG00000285645,ENSG00000240627,ENSG00000213590,ENSG00000270739,ENSG00000240201,ENSG00000203531,ENSG00000271626,ENSG00000250316,ENSG00000218713,ENSG00000231548,ENSG00000231049,ENSG00000286185,ENSG00000214535,ENSG00000212961,ENSG00000219747,ENSG00000259100,ENSG00000231767,ENSG00000243175,ENSG00000236804,ENSG00000240634,ENSG00000270773,ENSG00000230267%22&amp;amp;VISIBLEPANEL=resultspanel&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biomart&lt;/a&gt; and found that only 2 have GO annotations, so you're likely not missing an immense number of genes. Having said that, you'd be best served to use a more frequently updated tool than DAVID. Around 2015 there was an outcry because it hadn't been updated in 6 years and the database was so out of date that people were getting questionable results. Have a look at one of the many Bioconductor packages for convenient free solutions that are better maintained.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-10-30T12:41:07.907" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14693" PostTypeId="2" ParentId="14690" CreationDate="2020-10-30T16:13:03.560" Score="6" Body="&lt;p&gt;Welcome to RNA-seq. This is not at all unusual, as RNA-seq raw data are not normally distributed. People typically approximate it via the Negative Binomial distribution. I suggest you check existing packages as Devon suggests in his comment for normalization and differential expression (or anything else). Typical choices are DESeq2, edgeR or limma, but there are others as well.&#xA;This &lt;a href=&quot;https://peerj.com/preprints/27283/&quot; rel=&quot;nofollow noreferrer&quot;&gt;guide&lt;/a&gt; might help you get started. Please also use google to explore existing threads both here at SE and over at &lt;a href=&quot;http://biostars.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;biostars.org&lt;/a&gt;.&#xA;Most basic questions have already been discussed there and are informative to get started.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2020-11-01T12:33:57.263" LastActivityDate="2020-11-01T12:33:57.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14694" PostTypeId="2" ParentId="14678" CreationDate="2020-10-30T20:00:09.343" Score="1" Body="&lt;p&gt;One &amp;quot;out there&amp;quot; approach you could take is to create a fasta file containing your regions of interest (as separate sequences), and then map your fasta files to that reference fasta file, generating BAM/SAM output. After that, position-sort the BAM file, and you've now got a content-sorted list of sequences.&lt;/p&gt;&#xA;&lt;p&gt;The process to do this would be something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 -a regions.fasta input1.fasta input2.fasta ... | samtools sort &amp;gt; mapped.bam&#xA;samtools fastq mapped.bam &amp;gt; sorted.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2020-10-30T20:00:09.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14695" PostTypeId="2" ParentId="14690" CreationDate="2020-10-30T20:04:22.463" Score="2" Body="&lt;p&gt;I like using DESeq2. There's a great document written by the developers about how to process gene count tables and look at differential expression:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The document also mentions other tools that have similar approaches:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Other Bioconductor packages with similar aims are &lt;a href=&quot;http://bioconductor.org/packages/edgeR&quot; rel=&quot;nofollow noreferrer&quot;&gt;edgeR&lt;/a&gt;, &lt;a href=&quot;http://bioconductor.org/packages/limma&quot; rel=&quot;nofollow noreferrer&quot;&gt;limma&lt;/a&gt;, &lt;a href=&quot;http://bioconductor.org/packages/DSS&quot; rel=&quot;nofollow noreferrer&quot;&gt;DSS&lt;/a&gt;, &lt;a href=&quot;http://bioconductor.org/packages/EBSeq&quot; rel=&quot;nofollow noreferrer&quot;&gt;EBSeq&lt;/a&gt;, and &lt;a href=&quot;http://bioconductor.org/packages/baySeq&quot; rel=&quot;nofollow noreferrer&quot;&gt;baySeq&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastActivityDate="2020-10-30T20:04:22.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14696" PostTypeId="1" CreationDate="2020-10-30T21:45:49.980" Score="3" ViewCount="33" Body="&lt;p&gt;I am in process of submiting my FASTQ files to the SRA database. I am a little confused about how to fill the biosample attributes and SRA metadata.&lt;/p&gt;&#xA;&lt;p&gt;I would really appreciate if you anyone could share example templates or help with the given database fields.&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="4845" LastEditorUserId="3848" LastEditDate="2020-11-06T13:06:04.510" LastActivityDate="2020-11-06T13:06:04.510" Title="Template files for bioattributes and meta-data" Tags="|ngs|fastq|genbank|sequence|" AnswerCount="0" CommentCount="1" ClosedDate="2021-05-10T06:05:22.380" ContentLicense="CC BY-SA 4.0" />
  <row Id="14697" PostTypeId="1" AcceptedAnswerId="14702" CreationDate="2020-10-31T03:09:36.223" Score="1" ViewCount="2609" Body="&lt;p&gt;I'm new to bioinformatics and &lt;a href=&quot;http://www.gromacs.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gromacs&lt;/a&gt; I tried to run the code and got this error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Program:     gmx mdrun, version 2020&#xA;Source file: src/gromacs/options/options.cpp (line 179)&#xA;Function:    void gmx::internal::OptionSectionImpl::finish()&#xA;&#xA;Error in user input:&#xA;Invalid input values&#xA;In option s&#xA;Required option was not provided, and the default file 'topol' does not&#xA;exist or is not accessible.&#xA;The following extensions were tried to complete the file name:&#xA;  .tpr&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I already have the file &lt;code&gt;topol&lt;/code&gt;. I searched and found that I need to run &lt;code&gt;mdrun-h&lt;/code&gt; to ensure that the command work but when I check&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mdrun -h &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I got&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mdrun: command not found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7248" LastEditorUserId="9109" LastEditDate="2020-11-02T20:36:46.083" LastActivityDate="2020-11-02T20:36:46.083" Title="gmx mdrun error: &quot;Error in user input: Invalid input values In option s&quot;" Tags="|gromacs|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14698" PostTypeId="1" CreationDate="2020-10-31T06:19:31.163" Score="1" ViewCount="77" Body="&lt;p&gt;I seem to have landed on the &lt;a href=&quot;https://www.mitomap.org/MITOMAP&quot; rel=&quot;nofollow noreferrer&quot;&gt;mitomap.org&lt;/a&gt; site, but I don't know what to make of it or what do with it / how to get the genomes onto my computer. It sounds like the genomes are stored in GenBank, but that mitomap simply lists which of the GenBank genomes are human mitochondrial haplogroup genomes.&lt;/p&gt;&#xA;&lt;p&gt;First of all, I can't find a simple list of links or list of names of the haplogroups the mitomap has... Can you show me where I can get or download that? They say:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This brings our total number of FL sequences to 51,673, and the number of CR sequences to 74,660. Our SNVs now total 19,227.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't know what FL or CR (Control Region sequences?) means, but I am expecting to see lists of genomes or something. What am I to do with this information? Where do I find this data? Is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/?term=(015400%5BSLEN%5D%3A016700%5BSLEN%5D)+AND+Homo%5BOrganism%5D+AND+mitochondrion%5BFILT%5D+NOT+(unverified+OR+Homo+sp.+Altai+OR+Denisova+hominin+OR+neanderthalensis+OR+heidelbergensis+OR+consensus)&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; it? If I click on one of the results, I see &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/MT742594.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. It has path &lt;code&gt;nuccore/MT742594.1&lt;/code&gt;, can I find this in some NCBI FTP server somewhere? I don't see anything of that folder structure in the &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/genbank/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GenBank FTP server&lt;/a&gt;...&lt;/p&gt;&#xA;&lt;p&gt;Basically my question is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Where is the complete list of human mitochondrial haplogroup genomes?&lt;/li&gt;&#xA;&lt;li&gt;Where can I download all of those human mitochondrial haplogroup genomes so I can get them onto my computer?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9787" LastActivityDate="2022-03-25T18:03:50.420" Title="How do I download the mitochondrial haplogroup datasets for human genetics online?" Tags="|genome|data-download|public-dataset|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14699" PostTypeId="1" AcceptedAnswerId="14725" CreationDate="2020-10-31T12:01:17.107" Score="1" ViewCount="537" Body="&lt;p&gt;I have a list of about 4000 PDB IDs and would like to get the actual names of the proteins (e.g. lactate dehydrogenase, cytochrome c). I tried the batch header section at the &lt;a href=&quot;https://www.rcsb.org/downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;Protein Databank Download page&lt;/a&gt; but it refused to accept my PDB IDs in formats (xxxx or xxxx.pdb, individually or space-separated) that worked in an interactive search for the protein structure.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="1255" LastActivityDate="2020-11-07T19:46:22.703" Title="Get protein names corresponding to PDB ID" Tags="|protein-structure|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14700" PostTypeId="1" CreationDate="2020-10-31T13:22:21.850" Score="0" ViewCount="1888" Body="&lt;p&gt;I am using DESEQ2 for RNAseq analysis but i do not understand why I get this error.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dds &amp;lt;- DESeqDataSetFromMatrix(countData = data, &#xA;+                               colData = labeldata, &#xA;+                               design =~ subtype, tidy = TRUE)&#xA;&#xA;**Error** in `.rowNamesDF&amp;lt;-`(x, value = value) : &#xA;  duplicate 'row.names' are not allowed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I really do not know what is the issue. Could you please help?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="9788" LastEditorUserId="650" LastEditDate="2020-10-31T17:40:28.463" LastActivityDate="2020-11-01T18:21:06.297" Title="DESEQ2:Error in rownames, what is the problem?" Tags="|r|rna-seq|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14701" PostTypeId="2" ParentId="14698" CreationDate="2020-10-31T14:15:54.900" Score="1" Body="&lt;p&gt;Hmmm, I am not experienced enough in biology or bioinformatics to truly understand the question, but I am guessing you want to download the FASTA file or genebank file containing the sequence of the accession number you listed above.&#xA;There are 2 ways to do this -&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Using your browser, if you want to download a small sample of files. On the right hand side of your screen you can see the &amp;quot;send to&amp;quot; button on this link, click on it and choose the what's appropriate for you and then click on create file.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;code&gt;https://www.ncbi.nlm.nih.gov/nuccore/MT742594.1&lt;/code&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;If you familiar with Biopython and python programming you can download multiple files at once in whatever format you want, there is an easy script to do this and has been taken from Biopython's documentation on biopython.org&lt;/p&gt;&#xA;&lt;p&gt;import os&lt;/p&gt;&#xA;&lt;p&gt;from Bio import SeqIO&lt;/p&gt;&#xA;&lt;p&gt;from Bio import Entrez&lt;/p&gt;&#xA;&lt;p&gt;Entrez.email = &amp;quot;A.N.Other@example.com&amp;quot;  # Always tell NCBI who you are&lt;/p&gt;&#xA;&lt;p&gt;filename = &amp;quot;MT742594.1.gb&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;if not os.path.isfile(filename):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; # Downloading...&#xA;&#xA; net_handle = Entrez.efetch(&#xA;&#xA;     db=&amp;quot;nucleotide&amp;quot;, id=&amp;quot;MG762674&amp;quot;, rettype=&amp;quot;gb&amp;quot;, retmode=&amp;quot;text&amp;quot;&#xA;&#xA; )&#xA; out_handle = open(filename, &amp;quot;w&amp;quot;)&#xA;&#xA; out_handle.write(net_handle.read())&#xA;&#xA; out_handle.close()&#xA;&#xA; net_handle.close()&#xA;&#xA; print(&amp;quot;Saved&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;print(&amp;quot;Parsing...&amp;quot;)&lt;/p&gt;&#xA;&lt;p&gt;record = SeqIO.read(filename, &amp;quot;gb&amp;quot;)&lt;/p&gt;&#xA;&lt;p&gt;print(record)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;You can make a list of the accession numbers of the sequences you want to download and pass the list in the id, if you are curious as to how the code works visit biopython.org .&lt;/p&gt;&#xA;" OwnerUserId="9772" LastActivityDate="2020-10-31T14:15:54.900" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14702" PostTypeId="2" ParentId="14697" CreationDate="2020-10-31T14:49:17.837" Score="2" Body="&lt;p&gt;If the command line says &amp;quot;command not found&amp;quot;, it means that the binary is not executable or not found in any of the paths mentioned in the &lt;code&gt;$PATH&lt;/code&gt; environmental variable. You can check for example &lt;a href=&quot;https://www.cyberciti.biz/faq/linux-unix-command-not-found-error-and-how-to-get-rid-of-it/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial&lt;/a&gt; for details on how it works and how to resolve problems related to this variable.&lt;/p&gt;&#xA;&lt;p&gt;Which probably means, that you need to go a step back, to &lt;a href=&quot;http://manual.gromacs.org/documentation/current/install-guide/index.html#building-just-the-mdrun-binary&quot; rel=&quot;nofollow noreferrer&quot;&gt;the installation instructions of gromacs&lt;/a&gt; and try to install the &lt;code&gt;mdrun&lt;/code&gt; binary. However, if the &lt;code&gt;mdrun&lt;/code&gt; was not installed successfully, it's very likely that more stuff of the package will be missing, so it might be worth reinstalling it all.&lt;/p&gt;&#xA;&lt;p&gt;P.S. It's not obvious to me if getting &lt;code&gt;mdrun&lt;/code&gt; it will resolve the first posted problem (I have never worked with gromacs). If you would share the link to the research you have done, that would help others to help you.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2020-10-31T14:49:17.837" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14703" PostTypeId="2" ParentId="14699" CreationDate="2020-10-31T16:11:32.100" Score="2" Body="&lt;p&gt;Assuming you can use &lt;code&gt;R&lt;/code&gt;, have you tried with &lt;code&gt;biomaRt&lt;/code&gt;? For example, using &lt;code&gt;2bhl&lt;/code&gt; (my PhD lover :D)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(biomaRt)&#xA;ensembl &amp;lt;- useMart(&amp;quot;ensembl&amp;quot;,dataset=&amp;quot;hsapiens_gene_ensembl&amp;quot;)&#xA;&#xA;# get list of all available info&#xA;filters &amp;lt;- listFilters(ensembl)&#xA;attributes &amp;lt;- listAttributes(ensembl)&#xA;&#xA;getBM(attributes=c('hgnc_symbol','ensembl_gene_id','entrezgene_id',&#xA;                'protein_id','description',&amp;quot;superfamily&amp;quot;), &#xA;      filters = 'pdb', &#xA;      values = &amp;quot;2bhl&amp;quot;, &#xA;      mart = ensembl)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Returns&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;hgnc_symbol ensembl_gene_id entrezgene_id protein_id description&#xA;1         G6PD ENSG00000160211          2539   ADO22353 glucose-6-phosphate dehydrogenase [Source:HGNC Symbol;Acc:HGNC:4057]&#xA;2         G6PD ENSG00000160211          2539   CAA27309 glucose-6-phosphate dehydrogenase [Source:HGNC Symbol;Acc:HGNC:4057]&#xA;3         G6PD ENSG00000160211          2539   AAA63175 glucose-6-phosphate dehydrogenase [Source:HGNC Symbol;Acc:HGNC:4057]&#xA;4         G6PD ENSG00000160211          2539   AAA52500 glucose-6-phosphate dehydrogenase [Source:HGNC Symbol;Acc:HGNC:4057]&#xA;5         G6PD ENSG00000160211          2539   AAA52501 glucose-6-phosphate dehydrogenase [Source:HGNC Symbol;Acc:HGNC:4057]&#xA;6         G6PD ENSG00000160211          2539   CAA39089 glucose-6-phosphate dehydrogenase [Source:HGNC Symbol;Acc:HGNC:4057]&#xA;7 ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am sure if you look at all the available filters and attributes, you can pinpoint the ID you are looking for.&lt;/p&gt;&#xA;" OwnerUserId="3631" LastActivityDate="2020-10-31T16:11:32.100" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14704" PostTypeId="2" ParentId="14690" CreationDate="2020-10-31T16:46:30.297" Score="1" Body="&lt;p&gt;Another approach, that does not replace the methods already mentions, but could be useful if you don't have raw counts to start with, or if you want to screen hundreds of genes, and you want to focus on the most interesting.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;either &lt;code&gt;log transform&lt;/code&gt; or &lt;code&gt;zscale&lt;/code&gt; your expression values&lt;/li&gt;&#xA;&lt;li&gt;do a boxplot to see how your gene is expressed in the different tissues&lt;/li&gt;&#xA;&lt;li&gt;run a non-parametric test to compare the distributions. I usually use a &lt;code&gt;kruskal.test&lt;/code&gt; with multiple comparisons and a Mann-Whitney U test (&lt;code&gt;wilcox.test&lt;/code&gt;) if I only have two groups.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This should give you an idea if the expression of a gene is different between tissue types.&lt;/p&gt;&#xA;" OwnerUserId="3631" LastActivityDate="2020-10-31T16:46:30.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14705" PostTypeId="2" ParentId="14700" CreationDate="2020-10-31T19:10:05.693" Score="3" Body="&lt;p&gt;The hint is here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;duplicate 'row.names' are not allowed&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;rownames(data)&lt;/code&gt; should return a vector of only unique values, since it doesn't make sense to have the same gene/transcript in two different rows of your count matrix. How did you generate it?&lt;/p&gt;&#xA;&lt;p&gt;What does &lt;code&gt;which(duplicated(rownames(data)))&lt;/code&gt; return? Maybe that'll give you a hint on what went wrong. In general, if you use &lt;code&gt;salmon&lt;/code&gt; or &lt;code&gt;kallisto&lt;/code&gt; for counting, you should use the excellent &lt;code&gt;tximport&lt;/code&gt; package to get the counts into R, then create a DESeq dataset with &lt;code&gt;DESeqDataSetFromTximport()&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;EDITS:&lt;/p&gt;&#xA;&lt;p&gt;The replies to my answer prompted me to re-read your post and I saw that you had set &lt;code&gt;tidy=TRUE&lt;/code&gt;, which means the duplicates are actually in your first column of your &lt;code&gt;data&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9693" LastEditorUserId="9693" LastEditDate="2020-11-01T18:21:06.297" LastActivityDate="2020-11-01T18:21:06.297" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14706" PostTypeId="2" ParentId="14682" CreationDate="2020-10-31T19:56:09.070" Score="2" Body="&lt;p&gt;Just to clarify, when I asked you to share some of your files, I meant to share them in a way that I can reproduce your issue from start to finish. You read from at least four different files in a quadruple for-loop and I'm pretty sure your goal could be achieved much more straightforwardly than your current implementation.&lt;/p&gt;&#xA;&lt;p&gt;Without a reproducible example, I can offer a best-guess on what's happening: you create &lt;code&gt;date_df&lt;/code&gt; and &lt;code&gt;location_df&lt;/code&gt;, but after you instantiate them, you never do anything with the objects. Your final write happens from &lt;code&gt;df&lt;/code&gt;, so you'll need to add those columns, e.g.:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;df[&amp;quot;Date&amp;quot;] = date_df[&amp;quot;Date&amp;quot;][0]&#xA;df[&amp;quot;Location&amp;quot;] = location_df[&amp;quot;Location&amp;quot;][0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above is just an example, you'll need to make sure the &lt;em&gt;correct&lt;/em&gt; value gets assigned to your &lt;code&gt;df&lt;/code&gt; object instead of simply the first one.&lt;/p&gt;&#xA;" OwnerUserId="9693" LastActivityDate="2020-10-31T19:56:09.070" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14707" PostTypeId="1" CreationDate="2020-10-31T23:25:58.193" Score="1" ViewCount="60" Body="&lt;p&gt;I am completely new to meta-analysis and am a bit lost on how to proceed.&lt;/p&gt;&#xA;&lt;p&gt;I have two biomarker datasets; each one is using a different microarray platform. I have the protein names, the concentration in each subject (one dataset has 15 subjects while another has 22) as well as fold change, p-value, and AUC values.&lt;/p&gt;&#xA;&lt;p&gt;Is there a simple way to do a meta-analysis with this information? For example, is there any way to join the datasets and calculate a combined fold change and AUC? I was thinking of using sumlog() from the metap package in R to do a combined p-value, but I am lost on how to combine the other two values, or if this would even be appropriate. I've attached a screenshot of one of the datasets&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3JfSP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3JfSP.png&quot; alt=&quot;dataset preview&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Any help or guidance would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="7036" LastEditorUserId="48" LastEditDate="2020-11-03T12:17:28.160" LastActivityDate="2020-11-03T12:17:28.160" Title="Meta-analysis in R" Tags="|r|microarray|meta-analysis|" AnswerCount="0" CommentCount="2" ClosedDate="2020-11-06T12:03:12.260" ContentLicense="CC BY-SA 4.0" />
  <row Id="14709" PostTypeId="2" ParentId="14690" CreationDate="2020-11-01T08:50:26.107" Score="1" Body="&lt;p&gt;Thanks for all your answers.&#xA;Although the pipeline packages would have been useful I was trying to normalise the data before analysing it later and so I used the bestnormalize package which looked like it did a pretty good job.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/G1BB2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/G1BB2.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aBWCx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aBWCx.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9669" LastActivityDate="2020-11-01T08:50:26.107" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14710" PostTypeId="1" AcceptedAnswerId="14711" CreationDate="2020-11-01T08:53:40.947" Score="1" ViewCount="546" Body="&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;from Bio import Align&#xA;&#xA;ref_seq_1 = SeqIO.read('C:/Users/King_Arthur/Desktop/ref_seq/segment 1/ref_seq_8.fasta','fasta')&#xA;&#xA;seq1 = SeqIO.read('C:/Users/King_Arthur/Desktop/file/segment 1/Myfile_1 (1).fasta','fasta')&#xA;&#xA;aligner = Align.PairwiseAligner()&#xA;aligner.mode = 'global'&#xA;aligner.match_score = 1&#xA;aligner.mismatch_score = -2&#xA;aligner.gap_score = -2&#xA;&#xA;alignments = aligner.score(ref_seq_1.seq , seq1.seq)&#xA;&#xA;print(alignments)&#xA;for alignment in sorted(alignments):&#xA;    print(alignment)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So this is my code and as you can see in the last section i am trying to iterate over my alignment but I am getting this error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TypeError: 'float' object is not iterable&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have tried various things like using &lt;code&gt;str()&lt;/code&gt; but it gives some strange values and I also tried to read the source code by using the &lt;code&gt;inspect&lt;/code&gt; module but I can't figure out the problem.&lt;/p&gt;&#xA;&lt;p&gt;Any help would be really appreciated.&lt;/p&gt;&#xA;&lt;p&gt;My final objective is to find out how many matches, mismatches and gaps are present in the final alignment using biopython.&lt;/p&gt;&#xA;&lt;p&gt;if there is any other better way to do it in python please feel free to suggest.&lt;/p&gt;&#xA;" OwnerUserId="9772" LastActivityDate="2020-11-01T09:58:20.127" Title="Need help with Pairwise Alignment module in iterating over the alignment" Tags="|sequence-alignment|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14711" PostTypeId="2" ParentId="14710" CreationDate="2020-11-01T09:58:20.127" Score="2" Body="&lt;p&gt;&lt;code&gt;aligner.score(ref_seq_1.seq , seq1.seq)&lt;/code&gt; doesn't produce an alignment. It just gives you the score of the alignment as float value.&lt;/p&gt;&#xA;&lt;p&gt;Instead you need to run:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;alignments = aligner.align(ref_seq_1.seq , seq1.seq)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now &lt;code&gt;alignments&lt;/code&gt; is an &lt;code&gt;Bio.Align.PairwiseAlignments&lt;/code&gt; you can iterate over &lt;code&gt;alignments&lt;/code&gt; or get the score with &lt;code&gt;alignments.score&lt;/code&gt; which gives you the same as aligner.score(ref_seq_1.seq , seq1.seq)`&lt;/p&gt;&#xA;&lt;p&gt;If you need more information, you could check the &lt;a href=&quot;https://biopython.org/docs/1.75/api/Bio.Align.html?highlight=pairwisealigner#Bio.Align.PairwiseAligner&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2020-11-01T09:58:20.127" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14712" PostTypeId="1" CreationDate="2020-11-01T22:05:55.280" Score="2" ViewCount="369" Body="&lt;p&gt;I was going over different sequencing library protocols and noticed that the adapter sequence can vary between protocols. In some adapter sequences the index sits between the primer binding site and the insert. In this case during sequencing the index will be sequenced and then the insert that comes right after it. In other cases, the adapter sequence is built so that the index is outside the primer binding site, i.e. index - primer binding site - insert (see two leftmost parts of the attached figure). Here the insert is sequenced using the Read1 primer and afterwards the index is sequenced in a separate reaction using the 17 index primer.&#xA;&lt;img src=&quot;https://supportassets.illumina.com/content/dam/illumina-support/images/bulletins/PEcell1.png&quot; alt=&quot;figure 1&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this case are the index reads just added on to the ends of the insert reads in the FastQ files? Is there a way to tell from the FastQ files where the index was positioned within the adapter?&lt;/p&gt;&#xA;" OwnerUserId="9794" LastEditorUserId="77" LastEditDate="2020-11-03T21:26:41.920" LastActivityDate="2020-12-04T17:06:45.693" Title="Are sequencing reads written differently when inserts and indexes are sequenced seperately?" Tags="|rna-seq|ngs|adapter|library|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14713" PostTypeId="1" CreationDate="2020-11-02T01:15:39.693" Score="1" ViewCount="520" Body="&lt;p&gt;i'm using this command&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gmx mdrun -deffnm nvt0 -nb gpu&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but got&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Nonbonded interactions on the GPU were requested with -nb gpu, but the GROMACS&#xA;binary has been built without GPU support. Either run without selecting GPU&#xA;options or recompile GROMACS with GPU support enabled&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've Nvidia rtx2060 with cuda 11.1 on ubnutu 18.4.5&lt;/p&gt;&#xA;" OwnerUserId="7248" LastActivityDate="2020-11-02T11:00:06.630" Title="Nonbonded interactions on the GPU" Tags="|python|gromacs|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14714" PostTypeId="2" ParentId="14713" CreationDate="2020-11-02T09:46:57.173" Score="4" Body="&lt;p&gt;As the error tells you: &lt;code&gt;the GROMACS binary has been built without GPU support&lt;/code&gt;.&#xA;Did you install gromacs with &lt;code&gt;apt-get&lt;/code&gt;? Usually these kind of programs are shipped with a &lt;code&gt;basic&lt;/code&gt; built, with a lot of features disabled.&#xA;If you did built gromacs from source, then cuda was not found during configuration.&#xA;I would try building gromacs from source and:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;be sure cuda is in your path and check that &lt;code&gt;nvcc&lt;/code&gt; and the drivers have a matching version&lt;/li&gt;&#xA;&lt;li&gt;run cmake and check the output, to see if the video card is recognised&lt;/li&gt;&#xA;&lt;li&gt;run cmake with &lt;code&gt;-DGMX_GPU=ON&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Note that it seems that gromacs doesn't support multiple gpu accelerations( e.g. it is either &lt;code&gt;openCL&lt;/code&gt; or &lt;code&gt;cuda&lt;/code&gt;, you can't have both)&lt;/p&gt;&#xA;" OwnerUserId="3631" LastEditorUserId="3631" LastEditDate="2020-11-02T11:00:06.630" LastActivityDate="2020-11-02T11:00:06.630" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14715" PostTypeId="1" AcceptedAnswerId="14727" CreationDate="2020-11-02T09:47:59.453" Score="2" ViewCount="52" Body="&lt;p&gt;(Cross-post with Biostars: &lt;a href=&quot;https://www.biostars.org/p/470788/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/470788/&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;We have some genotype data that we are putting through quality control in PLINK 1.9. As part of this QC, we have limited the data to subjects of self-reported and genomic European ancestries, and have run basic QC steps, removing call rates below 95% for individuals and variants, and removing variants excessively out of Hardy-Weinberg equilibrium (&amp;lt; 10^-10).&lt;/p&gt;&#xA;&lt;p&gt;Pruning for LD and limiting to the autosomes, we have calculated estimated IBD between our participants and the estimated the average pairwise IBD for each participant. We have found that an unusual number of our participants (233/13049) show a high mean pairwise IBD with others (&amp;gt;3SD from the cohort mean, &amp;gt; 0.028 in this instance, maximum 0.075). Examining examples of the individuals in question shows that this is driven by low-level relatedness (~0.03-0.1) with many other individuals, rather than a small subset of relatives in the data. In addition, the relatedness in each case stems entirely from IBD1 (one shared variant) and not at all from IBD2 (two shared variants). Individual average IBD correlates negatively (-0.69) with the genome-wide inbreeding coefficient.&lt;/p&gt;&#xA;&lt;p&gt;I have one idea about what could be causing this (partial cross-contamination of the samples), but I am interested to know if others have other ideas what would be the cause?&lt;/p&gt;&#xA;" OwnerUserId="4092" LastActivityDate="2020-11-05T06:11:48.927" Title="Excess average estimated identical-by-descent in genotype data" Tags="|gwas|quality-control|genotyping|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14716" PostTypeId="1" CreationDate="2020-11-02T10:19:19.317" Score="1" ViewCount="49" Body="&lt;p&gt;In a parent directory, I have several sub-directories, each of them contain one .vcf.gz file on which analysis should be performed. I want to run following commands on all .vcf.gz files in all subdirectories. What is the best way of doing it?? I am the beginner with shell scripting.&lt;/p&gt;&#xA;&lt;p&gt;I will add short explanations. The input is .vcf file from every subdirectory. at the end, GenePy score is calculated.&lt;/p&gt;&#xA;&lt;p&gt;To improve score consistency, filtering step:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;vcftools --gzvcf GENOTYPED_ALL.vcf.gz --minGQ 20 --recode --out GENO_FILT_GQ20 # Set GQ&amp;lt;20 as missing ( retain high confidence calls)&#xA;vcftools --vcf GENO_FILT_GQ20.vcf --max-missing 0.7 --out filtered # Remove SNVs with missing rate &amp;gt;30%&#xA;vcftools --vcf filtered.recode.vcf --min-alleles 2 --max-alleles 2 # Keep only Biallelic SNVs &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Make annovar-ready file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./annovar/convert2annovar.pl \&#xA;    -format vcf4 FINAL_GQ20M30_BIALL.recode.vcf.gz \&#xA;    -outfile ALL_genepy.input \&#xA;    -allsample \&#xA;    -withfreq \&#xA;    -include 2&amp;gt;annovar.log&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Annotate against refGene,gnomad_exome,cadd13,eigen,revel,gwava,dann using Annovar&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./annovar/table_annovar.pl \&#xA;        ALL_genepy.input \&#xA;        ./annovar/humandb/ \&#xA;        -buildver hg19 \&#xA;        -out ALL_genepy \&#xA;        -remove \&#xA;        -protocol refGene,gnomad_exome,cadd13,eigen,revel,gwava,dann \&#xA;        -operation g,f,f,f,f,f,f \&#xA;        --thread 40 \&#xA;        --maxgenethread 40 \&#xA;        -nastring . &amp;gt;&amp;gt;annovar.log&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Combine Genotypes and annotations&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cut -f 18- ALL_genepy.input &amp;gt; a1&#xA;zgrep '^#CHR' FINAL_GQ20M30_BIALL.recode.vcf.gz | cut -f 10- &amp;gt; b1&#xA;cat b1 a1 &amp;gt; ALL_temp&#xA;paste ALL_genepy.hg19_multianno.txt ALL_temp &amp;gt; ALL_genepy.meta&#xA;rm a1 b1 ALL_temp&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Prepare output folders/dependecies, making new folders in a current directory to store raw GenePy score files&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mkdir CADD13_RawScore Eigen GWAVA_region_score GWAVA_tss_score dann REVEL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Take the header from the ALL_genepy.meta file and stores it in a newly created header file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep &amp;quot;^Chr&amp;quot; ALL_genepy.meta&amp;gt; header&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Compute GenePy scores&lt;/p&gt;&#xA;&lt;p&gt;Once the ALL_genepy.meta file is created, GenePy_1.2.sh can be run by simply iterating through the list of deisred genes. The make_scores_mat_5.py file is in the same directory of GenePy_1.2.sh (every subdirectory)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chmod +x GENEPY_1.2.sh&#xA;while read gene; do&#xA;  sh GENEPY_1.2.sh $gene&#xA;done &amp;lt; gene.list&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is code that I have run for .vcf files when I had only one .vcf file. Now, I have over 200 subdirectories, in each there is one .vcf file for analysis.&lt;/p&gt;&#xA;&lt;p&gt;How to wrap up this code to be executable for all subdirectories at once?&lt;/p&gt;&#xA;&lt;p&gt;I tried with the folowing code. But &lt;strong&gt;the result file is not saved in the corresponding subdirectory&lt;/strong&gt;. It is saved in main directory, and every time it got overwritten with next one.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    find . -type f -name '*.vcf'      \&#xA;   -exec vcftools --gzvcf {} --minGQ 20 --recode --out GENO_FILT_GQ20 ';'   \&#xA;   -exec vcftools --vcf GENO_FILT_GQ20.recode.vcf --max-missing 0.7 --out filtered  ';'\&#xA;   -exec vcftools --vcf GENO_FILT_GQ20.recode.vcf --min-alleles 2 --max-alleles 2  ';'\&#xA;   -exec /home/bioinformatics/Dokumente/pipeline_test/annovar/convert2annovar.pl \&#xA;    -format vcf4 GENO_FILT_GQ20.recode.vcf \&#xA;    -outfile ALL_genepy.input \&#xA;    -allsample \&#xA;    -withfreq \&#xA;    -include 2&amp;gt;annovar.log ';'\&#xA;   -exec /home/bioinformatics/Dokumente/pipeline_test/annovar/table_annovar.pl \&#xA;        ALL_genepy.input \&#xA;        /home/bioinformatics/Dokumente/pipeline_test/annovar/humandb/ \&#xA;        -buildver hg19 \&#xA;        -out ALL_genepy \&#xA;        -remove \&#xA;        -protocol refGene,gnomad_exome,cadd13,eigen,revel,gwava,dann \&#xA;        -operation g,f,f,f,f,f,f \&#xA;        --thread 40 \&#xA;        --maxgenethread 40 \&#xA;        -nastring . &amp;gt;&amp;gt;annovar.log';'\&#xA;    -exec cut -f 18- ALL_genepy.input &amp;gt; a1';'\&#xA;    -exec zgrep '^#CHR' GENO_FILT_GQ20.recode.vcf | cut -f 10- &amp;gt; b1';'\&#xA;    -exec cat b1 a1 &amp;gt; ALL_temp';'\&#xA;    -exec paste ALL_genepy.hg19_multianno.txt ALL_temp &amp;gt; ALL_genepy.meta';'\&#xA;    -exec rm a1 b1 ALL_temp #remove';'\&#xA;    -exec mkdir CADD13_RawScore Eigen GWAVA_region_score GWAVA_tss_score dann REVEL ';'\&#xA;    -exec grep &amp;quot;^Chr&amp;quot; ALL_genepy.meta&amp;gt; header ';'\&#xA;    -exec chmod +x GENEPY_1.2.sh ';'\&#xA;    -exec while read gene; do&#xA;  sh GENEPY_1.2.sh $gene&#xA;done &amp;lt; gene.list ';'\&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thank you for all suggestion, I appreciate it.&lt;/p&gt;&#xA;" OwnerUserId="9797" LastEditorUserId="9797" LastEditDate="2020-11-02T11:11:59.570" LastActivityDate="2020-11-02T11:11:59.570" Title="How to calculate GenePy score for vcf files in all subdirectories?" Tags="|bash|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14717" PostTypeId="1" CreationDate="2020-11-02T10:45:55.447" Score="1" ViewCount="177" Body="&lt;p&gt;I have applied two de novo genome assembly tools to data from the run SRR12707453, corresponding to a phage (I downloaded the data and have no relation with the authors of the study).&lt;/p&gt;&#xA;&lt;p&gt;Using rnaSPAdes I get many long contigs (&amp;gt;50.000 bp). When analyzing the contigs using Quast against a reference genome (MN813694.1) most long contigs are discarded and the seventh-longest one is the only one selected. If I blast the other contigs I find correspondence with other phages, which makes me conclude that many phages were present in the same probe.&lt;/p&gt;&#xA;&lt;p&gt;This seventh contig has 47686 bp length, while the reference has 47713 bp length. But they are not equivalently arranged. Apparently, the genome was &lt;a href=&quot;https://viralzone.expasy.org/3968?outline=all_by_species&quot; rel=&quot;nofollow noreferrer&quot;&gt;circularized&lt;/a&gt; and then it was broken around the position 32400 in the reference. Also, the sequence does not match between positions 45000 and 45951 (in the reference), and a little misassembly seems to occur around position 46300.&lt;/p&gt;&#xA;&lt;p&gt;See here some screen captures for the Icarus visualization obtained with Quast:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/F56BQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/F56BQ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/N4bFY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/N4bFY.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In contrast, using Quast with the contigs obtained from the other assembly tool, I get some differences. There is also a contig with almost the same size as the reference genome (47573 bp) that has exactly the same structure than the contig obtained with rnaSPAdes: circularized, broken around position 32400, does not match between position 45000 and 45951, and small misassemblie aronund position 46300. But there are also some contigs in which there is no break around position 32400.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/P2SSl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/P2SSl.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;May I interpret that these contigs indicate that the phage genome is sometimes circularized and sometimes broken at position ~32400 and that's why some contigs don't find a sequence break at that position?&lt;/p&gt;&#xA;&lt;p&gt;Am I interpreting anything wrong? Any other conclusion can be obtained from the data?&lt;/p&gt;&#xA;&lt;p&gt;You can (temporaly) &lt;a href=&quot;https://www.contignant.com/data/quast_results.zip&quot; rel=&quot;nofollow noreferrer&quot;&gt;download here the Quast results&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5190" LastActivityDate="2021-11-28T03:06:50.620" Title="Interpreting contig alignments to a reference genome" Tags="|sequence-analysis|assembly|reference-genome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14718" PostTypeId="2" ParentId="14697" CreationDate="2020-11-02T10:46:36.677" Score="1" Body="&lt;p&gt;I think there was a misunderstanding here. Two errors are in user5520049's post.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A .tpr is not present in the working&#xA;directory so gromacs complains, if your .tpr is &lt;code&gt;topol&lt;/code&gt; then you have to give this file&#xA;to &lt;strong&gt;gmx mdrun&lt;/strong&gt;:  &lt;code&gt;gmx mdrun -s topol&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;mdrun is not a command, you should have called instead: &lt;code&gt;gmx mdrun -h&lt;/code&gt;. That's why you had command not found and not because gmx is not in your $PATH. You couldn't have run &lt;code&gt;gmx mdrun&lt;/code&gt; in the first place otherwise.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Side tip: run &lt;code&gt;gmx mdrun -h -hidden&lt;/code&gt; to see hidden options.&lt;/p&gt;&#xA;" OwnerUserId="9109" LastEditorUserId="9109" LastEditDate="2020-11-02T11:07:02.127" LastActivityDate="2020-11-02T11:07:02.127" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14719" PostTypeId="2" ParentId="14649" CreationDate="2020-11-02T14:01:08.067" Score="4" Body="&lt;p&gt;Here is one definition of the classical MHC region: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/grc/human/regions/MHC?asm=GRCh38.p13&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/grc/human/regions/MHC?asm=GRCh38.p13&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;which defines it as the 5mb region chr6:28510120-33480577 in GRCh38 coordinates.&lt;/p&gt;&#xA;&lt;p&gt;The &amp;quot;extended MHC&amp;quot; defined here [1] is 7.2 mb and encompasses an additional &amp;quot;extended Class I&amp;quot; region upstream of the region given above. The xMHC is defined in [1] as the regions between &lt;em&gt;HIST1H2AA&lt;/em&gt; and &lt;em&gt;RPL12P1&lt;/em&gt; genes, which would give you chr6:25726063-33400644 in Grch38. This is very similar to your HIST1H2AA - KIFC1 definition (chr6:25726063-33410226). Of course if you're trying to reproduce a result, probably better to use the original paper's definition.&lt;/p&gt;&#xA;&lt;p&gt;[1] &lt;a href=&quot;https://www.nature.com/articles/nrg1489&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/nrg1489&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9799" LastEditorUserId="9799" LastEditDate="2020-11-02T14:30:31.163" LastActivityDate="2020-11-02T14:30:31.163" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14720" PostTypeId="2" ParentId="14717" CreationDate="2020-11-02T23:25:07.397" Score="1" Body="&lt;p&gt;some additional information to your request.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/genomes/Viruses/FamilyPhylogeneticTree/Graph/SRR12707453.MN813694.1.gfa&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nlm.nih.gov/genomes/Viruses/FamilyPhylogeneticTree/Graph/SRR12707453.MN813694.1.gfa&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;it contain assembled by reference graph&#xA;using ncbi tool saute(skesa) which will be public later.&lt;/p&gt;&#xA;&lt;p&gt;install bandage app and load this graph&#xA;&lt;a href=&quot;https://rrwick.github.io/Bandage/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rrwick.github.io/Bandage/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;picture looks intriguing,I would say , run contain many errors located in one position.&#xA;Reason? I don’t really know.&lt;/p&gt;&#xA;" OwnerUserId="6690" LastActivityDate="2020-11-02T23:25:07.397" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14721" PostTypeId="1" AcceptedAnswerId="14723" CreationDate="2020-11-03T10:25:37.673" Score="1" ViewCount="395" Body="&lt;p&gt;I am a college student and just started bioinformatics. I am trying to write a script (not for school) that finds all potential NGG and TTTN protospacer adjacent motif sequences in a genome string. I wrote the following script:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TargetFile = open('file.fasta')&#xA;readTF = TargetFile.read()&#xA;useTF = &amp;quot;&amp;quot;.join(readTF[66:].split())&#xA;&#xA;&#xA;#no_of_basepairs = len(useTF)&#xA;&#xA;&#xA;sites = {&amp;quot;ngg_strand&amp;quot; : [],&#xA;         &amp;quot;ngg_antistrand&amp;quot; : [],     #location counted from reverse direction&#xA;         &amp;quot;tttn_strand&amp;quot; : [],&#xA;         &amp;quot;tttn_antistrand&amp;quot; : []}    #location counted from reverse direction&#xA;&#xA;&#xA;for i in range(0, len(useTF)-2):&#xA;    gg_mer = useTF[i:i + 2]&#xA;    ttt_mer = useTF[i:i + 3]&#xA;&#xA;    if gg_mer == &amp;quot;GG&amp;quot;:&#xA;        sites[&amp;quot;ngg_strand&amp;quot;].append(i)&#xA;    elif gg_mer == &amp;quot;CC&amp;quot;:&#xA;        #look for reverse complement, record location on antistrand from reverse direction&#xA;        sites[&amp;quot;ngg_antistrand&amp;quot;].append(len(useTF)-i-2)&#xA;    elif ttt_mer == &amp;quot;TTT&amp;quot;:&#xA;        sites[&amp;quot;tttn_strand&amp;quot;].append(i+1)&#xA;    elif ttt_mer == &amp;quot;AAA&amp;quot;:&#xA;        # look for reverse complement, record location on antistrand from reverse direction&#xA;        sites[&amp;quot;tttn_antistrand&amp;quot;].append(len(useTF)-i-2)&#xA;&#xA;&#xA;print(len(sites[&amp;quot;ngg_strand&amp;quot;]))&#xA;print(useTF.count(&amp;quot;GG&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;At the bottom I used the .count() method to compare results. I noticed that the method returns a smaller number than my code. I believe this is due to the .count() method not counting overlap, whereas my code does. I.e. in the string &amp;quot;NNNGGGGNNN&amp;quot; my code would count 3 occurences of the &amp;quot;GG&amp;quot; substring, whereas the .count() method returns 2 occurences.&lt;/p&gt;&#xA;&lt;p&gt;My Questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If I want to find all possible PAM sites, which way of counting is correct? I believe my way would be correct, since as far as I know the RNA molecule doesn't advance in &amp;quot;triplet-steps&amp;quot;. However, I am very much new to this and might be missing something.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the reverse complement is found, is the location usually recorded from the same direction as on the strand, or in the reverse direction (as I implemented in the code)?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Like I said, I am new to this so if you see any other glaring problems I would appreciate any and all help. :-)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="9805" LastEditorUserId="3848" LastEditDate="2020-11-11T12:05:28.833" LastActivityDate="2020-11-11T12:05:28.833" Title="Find CRISPR PAM-sites with python" Tags="|crispr|python|" AnswerCount="1" CommentCount="6" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14722" PostTypeId="2" ParentId="14712" CreationDate="2020-11-03T16:54:14.070" Score="0" Body="&lt;p&gt;This is definitely a bioinformatics question! Understanding how sequencing libraries are created in the wet-lab and how they are sequenced is fundamental to bioinformatics.&lt;/p&gt;&#xA;&lt;p&gt;To answer the question, the index sequences are not included in the &lt;code&gt;fastq&lt;/code&gt; file output from Illumina, or other machines. Illumina sequencers just use the index sequence to demultiplex the complex mixture of molecules in the sequencing pool. If a small &amp;quot;dot&amp;quot; on the glass sequencing slide has some sequence, but the indices don't match what the technician specified for a library, then that read is not included in the fastq output of that library since it could be contamination of some sort.&lt;/p&gt;&#xA;&lt;p&gt;The sequences in the fasta file theoretically all should be the insert from the library that you prepared, but sometimes the molecule being sequenced is short, and there is read-through of the adapter sequence. This is the only circumstance under which the i7 or i5 index sequences might be in the normal &lt;code&gt;fastq&lt;/code&gt; output sequence line. This is not ideal, and this is why people perform adapter trimming.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2020-11-03T16:54:14.070" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14723" PostTypeId="2" ParentId="14721" CreationDate="2020-11-04T13:31:21.460" Score="1" Body="&lt;p&gt;The easiest solution is &lt;code&gt;finditer&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;use re&#xA;&#xA;# Compile you search term where [A|T|G|C] means A or G or C or T &#xA;p = re.compile(&amp;quot;[A|T|G|C]GG&amp;quot;)&#xA;&#xA;iterator = p.finditer(useTF)&#xA;for match in iterator:&#xA;    print(match.span())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;** Example Output**&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(0, 2)&#xA;(22, 24)&#xA;(29, 31)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Easy does it. Its worth looking at the documentation for &lt;code&gt;re&lt;/code&gt; here, &lt;a href=&quot;https://docs.python.org/3/howto/regex.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.python.org/3/howto/regex.html&lt;/a&gt; for alternative approaches using &lt;code&gt;match&lt;/code&gt; or &lt;code&gt;search&lt;/code&gt; below. &lt;code&gt;finditer&lt;/code&gt; is definitely the solution in context.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I support your coding approach because it provides lots of flexibility. For example, you might want to target GG or TTT around a given locus and you can easily modify the code to accomodate this. Someone elses code would be less flexible.&lt;/p&gt;&#xA;&lt;p&gt;In summary, I kinda like your substrings and do like the count method. I'm not a fan of your &lt;code&gt;==&lt;/code&gt; in this instance and your C-style matching should be substituted for reg-ex which in Python is implemented via &lt;code&gt;re&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The basic issues are,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Biopython would be used via AlignIO or SeqIO. This will allow any given format to be analysed. Yours approach is okay but prone to bugs (below). [probably a minor issue]&lt;/li&gt;&#xA;&lt;li&gt;Your sequence is in some sort of weird format, like extended phylip format and it is not fasta (that wouldn't work). It needs verifying because removing the first 65 characters might remove bits of sequence [probably a minor issue]&lt;/li&gt;&#xA;&lt;li&gt;Chopping of -2 from the length ... I dunno I'd check out -1 [minor]&lt;/li&gt;&#xA;&lt;li&gt;C-style matching. I don't like this. I would definitely use &lt;code&gt;re&lt;/code&gt;. One solution uses &lt;code&gt;match&lt;/code&gt; or &lt;code&gt;search&lt;/code&gt;, but the solution above uses &lt;code&gt;finditer&lt;/code&gt;, Reg-ex will allow to search for NGG and TTTN, which you are not doing&lt;/li&gt;&#xA;&lt;li&gt;I think you are looping a locus of 3 and 4 bp and I'm not sure how the C-style &lt;code&gt;==&lt;/code&gt; would match this because thats 2 and 3 bp respectively. Just thoughts. You will have 'edge effects' at the beginning and end of sequences, e.g. TTT 3' will match but that isn't TTTN.&lt;/li&gt;&#xA;&lt;li&gt;I like your &lt;code&gt;count&lt;/code&gt; method, but I think &lt;code&gt;finditer&lt;/code&gt; is better because it gives you location information&lt;/li&gt;&#xA;&lt;li&gt;The dictionary is sort of fine. Personally, I would dump the dictionary into a &lt;code&gt;pandas&lt;/code&gt; dataframe This way the results are output as e.g. CSV, spreadsheet flat-files for your colleagues to view.&lt;/li&gt;&#xA;&lt;li&gt;the &lt;code&gt;if&lt;/code&gt; and &lt;code&gt;elif&lt;/code&gt; would be placed in a subroutine, this is good practice to convert to an object orientated approach.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-11-06T16:37:46.250" LastActivityDate="2020-11-06T16:37:46.250" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14724" PostTypeId="2" ParentId="14712" CreationDate="2020-11-04T16:16:24.573" Score="1" Body="&lt;p&gt;The order of sequencing in a dual index flow cell is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Read 1&lt;/li&gt;&#xA;&lt;li&gt;Index 1 (I7)&lt;/li&gt;&#xA;&lt;li&gt;Index 2 (I5)&lt;/li&gt;&#xA;&lt;li&gt;Read 2&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Between each step there is a switch of the primer used (Read1-primer -&amp;gt; Index1-primer (I7) -&amp;gt; Index2-primer (I5) -&amp;gt; Read2-primer)&lt;br /&gt;&#xA;There are differences in how the I5 is dealt with between dual- and single-read flow cells as explained in the &lt;a href=&quot;https://emea.support.illumina.com/content/dam/illumina-support/documents/documentation/system_documentation/miseq/indexed-sequencing-overview-guide-15057455-07.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Illumina Indexed Sequencing Overview&lt;/a&gt;. In this document there is a lot more detailed explanation of the sequencing workflow that may be helpful to you.&lt;/p&gt;&#xA;&lt;p&gt;But as explained by the other answers, none of this will matter in most cases. Even if you decide to output the Index sequences they will be exported as separate FASTQ files.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2020-11-04T16:16:24.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14725" PostTypeId="2" ParentId="14699" CreationDate="2020-11-04T16:41:34.437" Score="5" Body="&lt;p&gt;You can use one of the &lt;a href=&quot;https://www.ebi.ac.uk/proteins/api/doc/#!/proteins/getByCrossReference&quot; rel=&quot;nofollow noreferrer&quot;&gt;UniProt Protein APIs&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As you said you have your pdb entries in a text file line by line you can, like this &lt;code&gt;example.txt&lt;/code&gt; containing:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1brr&#xA;4lzm&#xA;2dyi&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Using the commandline, you can use a little script like this to download the name, if it is available for the given pdb entry.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;while read line;&#xA;do &#xA; curl -X GET --header 'Accept:application/json' &amp;quot;https://www.ebi.ac.uk/proteins/api/proteins/pdb:$line&amp;quot; | &#xA; jq -r '.[].protein.recommendedName.fullName.value'  |&#xA; sed &amp;quot;s/^/$line\t/&amp;quot; &amp;gt;&amp;gt; pdb_names.txt;&#xA;done &amp;lt; example.txt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You need to have &lt;code&gt;curl&lt;/code&gt;, &lt;code&gt;sed&lt;/code&gt; and &lt;code&gt;jq&lt;/code&gt; installed on your system.&lt;/p&gt;&#xA;&lt;p&gt;This gives you following output in &lt;code&gt;pdb_names.txt&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1brr    Bacteriorhodopsin&#xA;4lzm    Endolysin&#xA;2dyi    Ribosome maturation factor RimM&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Update&lt;/p&gt;&#xA;&lt;p&gt;if you want to speed it up, you can run it with &lt;code&gt;parallel&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;parallel -j 4 'curl -X GET --header &amp;quot;Accept:application/json&amp;quot; &amp;quot;https://www.ebi.ac.uk/proteins/api/proteins/pdb:{}&amp;quot; |   jq -r &amp;quot;.[]. .protein.recommendedName.fullName.value&amp;quot;  |  sed &amp;quot;s/^/{}\t/&amp;quot; &amp;gt;&amp;gt; pdb_names_parallel.txt' :::: example.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With the &lt;code&gt;-j&lt;/code&gt; option you call how many jobs should run in parallel. The limit of the UniProt API is 200 request per second per user.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Update 7. Nov 2020&lt;/p&gt;&#xA;&lt;p&gt;To get another info beside the protein name, you need to know how the &lt;code&gt;JSON&lt;/code&gt; response from UniProt looks like.&lt;/p&gt;&#xA;&lt;p&gt;To get also the scientific name, you can run following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;parallel -j 4 'curl -X GET --header &amp;quot;Accept:application/json&amp;quot; &amp;quot;https://www.ebi.ac.uk/proteins/api/proteins/pdb:{}&amp;quot; | jq -r &amp;quot;.[] | .protein.recommendedName.fullName.value + \&amp;quot; - \&amp;quot; + .organism.names[0].value&amp;quot;  |  sed &amp;quot;s/^/{}\t/&amp;quot; &amp;gt;&amp;gt; pdb_names_parallel.txt' :::: example.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As result you get this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1brr    Bacteriorhodopsin - Halobacterium salinarum (strain ATCC 700922 / JCM 11081 / NRC-1)&#xA;4lzm    Endolysin - Enterobacteria phage T4&#xA;2dyi    Ribosome maturation factor RimM - Thermus thermophilus (strain HB8 / ATCC 27634 / DSM 579)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3360" LastEditorUserId="3360" LastEditDate="2020-11-07T19:46:22.703" LastActivityDate="2020-11-07T19:46:22.703" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14726" PostTypeId="1" CreationDate="2020-11-04T17:28:51.313" Score="1" ViewCount="324" Body="&lt;p&gt;I have a VEP annotated VCF file, and I wish to use bcftools to filter it. bcftools work very well on standard VCF fields, but I'm not able to make it work on VEP fields.&lt;/p&gt;&#xA;&lt;p&gt;For example, the following command would report &amp;quot;not defined in the vcf header&amp;quot; (it's not a standard VCF field but part of the VEP fields in the VCF file). &lt;code&gt;split-vep&lt;/code&gt; is a VEP plugin offered by bcftools.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools +split-vep MyFile.vcf -i 'gnomAD_exomes_AF&amp;lt;0.01'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="174" LastEditorUserId="174" LastEditDate="2020-11-04T18:09:06.080" LastActivityDate="2020-11-04T18:09:06.080" Title="How to filter allele frequency VEP with bcftools?" Tags="|bcftools|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14727" PostTypeId="2" ParentId="14715" CreationDate="2020-11-05T02:44:23.057" Score="1" Body="&lt;p&gt;A clear negative association within an inbreeding coefficient suggests convergent evolution driven by shared selective pressure. There is an outside chance of purifying selection, or else balancing selection, but selection is the key word. The phenotype of the locus needs investigating, i.e. outside nearly neutral theory what is constitutes positive selection?&lt;/p&gt;&#xA;&lt;p&gt;Allopatry caused by genetic isolation and genetic drift is your null hypothesis. However Darwinian positive selection is the alternative hypothesis.&lt;/p&gt;&#xA;&lt;p&gt;I would be wary against suggesting contamination unless you've a clear suspicion this is occuring. I do accept that in human genetics its often difficult to avoid, particularly in ancient DNA type work. In this case you would need to try and control for it. The argument I would put against contamination is that 'genetic relatedness' is localised within the genome, whereas contamination would be a genome-wide phenomena.&lt;/p&gt;&#xA;&lt;p&gt;Convergence is common within for example HLA genotypes (T-cell) loci between global populations.&lt;/p&gt;&#xA;&lt;p&gt;You could perform formal tests for postive selection either within pop gen (the locus will associate with significant homozygosity) or phylogeny (point mutation).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-11-05T06:11:48.927" LastActivityDate="2020-11-05T06:11:48.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14728" PostTypeId="1" CreationDate="2020-11-05T11:26:31.557" Score="0" ViewCount="50" Body="&lt;p&gt;We have some RNA from knock-in mice, there are two different sequences we're looking for. We have aligned to the mouse genome using STAR but the sequence isn't there which isn't too surprising What is the best way to check for the presence of the sequence? Should I just turn them into a sequence and use them as the reference genome for STAR to align to? Is there a way I can easily incorporate it into the existing genome so I could get read counts for everyhting including those reads? Should I just BLAST using something like magicblast?&lt;/p&gt;&#xA;" OwnerUserId="1843" LastActivityDate="2020-11-05T11:26:31.557" Title="Best way to align to find inserted sequence" Tags="|rna-seq|sequence-alignment|rna-alignment|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14730" PostTypeId="1" CreationDate="2020-11-05T17:56:26.940" Score="3" ViewCount="197" Body="&lt;h3&gt;Some background&lt;/h3&gt;&#xA;&lt;p&gt;Similar to the OP of &lt;a href=&quot;https://www.biostars.org/p/377840/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/377840/&lt;/a&gt;, I would like to programmatically BLAST a sequence to a local database of all WGS assemblies.&lt;br/&gt;&#xA;Since this isn't feasible for the average biology lab server (correct me if I am wrong), I plan to use &lt;a href=&quot;https://github.com/kblin/ncbi-acc-download&quot; rel=&quot;nofollow noreferrer&quot;&gt;ncbi-acc-download&lt;/a&gt; to download all WGS assemblies of the species of interest (not a popular species like E. coli, so it should be feasible). Then, I will create a BLAST database for the downloaded assemblies and BLAST the sequence to it.&lt;/p&gt;&#xA;&lt;h3&gt;My question&lt;/h3&gt;&#xA;&lt;p&gt;How can I find all WGS assemblies accessions of a species?&lt;br/&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;br/&gt;&#xA;&lt;p&gt;My current plan is to search the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Assembly database&lt;/a&gt; using &lt;a href=&quot;https://biopython.org/docs/latest/api/Bio.Entrez.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez&lt;/a&gt; and a search term such as &lt;code&gt;&amp;quot;wgs&amp;quot;[Properties] AND txid1337[orgn:exp]&lt;/code&gt;.&lt;br/&gt;&#xA;&lt;strong&gt;EDIT: IIUC, this approach might miss some WGS assemblies.&lt;/strong&gt; See &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/14749/4868&quot;&gt;my answer&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I am worried (and thus ask for your help) this isn't the right approach because there seem to be at least 3 other places in which assemblies can be found:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The assembly_summary.txt files that are described in &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/genomes/README_assembly_summary.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nlm.nih.gov/genomes/README_assembly_summary.txt&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Traces/wgs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/Traces/wgs/&lt;/a&gt;. &lt;strong&gt;This is the one I am most worried about.&lt;/strong&gt; Both because it seems to have much more entries, and also because &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genbank/wgs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/genbank/wgs/&lt;/a&gt; says: &amp;quot;The public WGS projects are at the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Traces/wgs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;list of WGS projects&lt;/a&gt; .&amp;quot;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;ftp://ftp.ncbi.nih.gov/blast/WGS_TOOLS/&quot; rel=&quot;nofollow noreferrer&quot;&gt;taxid2wgs.pl&lt;/a&gt;. (As mentioned in &lt;a href=&quot;https://www.biostars.org/p/377840/#471535&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/377840/#471535&lt;/a&gt;, this one seems to miss some assemblies.)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4868" LastEditorUserId="4868" LastEditDate="2020-11-08T19:01:14.893" LastActivityDate="2020-11-08T19:01:14.893" Title="How to find all WGS assemblies accessions of a species" Tags="|assembly|phylogenetics|wgs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14731" PostTypeId="1" AcceptedAnswerId="14732" CreationDate="2020-11-05T20:40:42.047" Score="1" ViewCount="152" Body="&lt;p&gt;I'm currently working on testing some matching algorithms for strings. I would like to do some tests on raw genomic data as I expect different results from random strings given the lower entropy. How can I get a whole human genome sequence? (just the raw basis)&lt;/p&gt;&#xA;&lt;p&gt;I have tried websites like &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/&lt;/a&gt; it seems very confusing and full of unnecessary information given that I just want to have the raw basis without annotations.&lt;/p&gt;&#xA;" OwnerUserId="9815" LastEditorUserId="6251" LastEditDate="2020-11-06T12:02:04.990" LastActivityDate="2020-11-06T12:02:04.990" Title="Get raw genome sequence" Tags="|fasta|genome|phylogenetics|human-genome|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14732" PostTypeId="2" ParentId="14731" CreationDate="2020-11-05T20:47:16.283" Score="3" Body="&lt;p&gt;You want &lt;a href=&quot;https://en.wikipedia.org/wiki/FASTA_format&quot; rel=&quot;nofollow noreferrer&quot;&gt;FASTA&lt;/a&gt; files from &lt;a href=&quot;ftp://ftp.ensembl.org/pub/release-101/fasta/homo_sapiens/dna/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you want the entirety of the genome, get the dna-toplevel file. Files with rm and sm have repeat sequences masked, the former by replacing the repeats with Ns (hard mask) the latter by designating repeats as lower case letters (soft masked).&lt;/p&gt;&#xA;&lt;p&gt;See the &lt;a href=&quot;ftp://ftp.ensembl.org/pub/release-101/fasta/homo_sapiens/dna/README&quot; rel=&quot;nofollow noreferrer&quot;&gt;README&lt;/a&gt; for more info.&lt;/p&gt;&#xA;" OwnerUserId="9693" LastActivityDate="2020-11-05T20:47:16.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14734" PostTypeId="1" AcceptedAnswerId="14737" CreationDate="2020-11-06T03:40:59.900" Score="0" ViewCount="254" Body="&lt;p&gt;I have an RNAseq gene expression file (Count data) as follows, I need to conduct a Differential gene expression analysis between Patients, for that, I need to have this file as a Matrix of Rows as Genes and columns as counts (samples), how best can I get this as matrix file?&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot for your help, Stay safe&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pOUdw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pOUdw.png&quot; alt=&quot;My data looks like the following&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9817" LastActivityDate="2020-11-06T08:56:02.087" Title="Gene expression Table to Expression Matrix converstion" Tags="|rna-seq|gene-expression|format-conversion|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14737" PostTypeId="2" ParentId="14734" CreationDate="2020-11-06T08:56:02.087" Score="0" Body="&lt;p&gt;Your question stated differently is as follows:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I have data in long format and need to change it to wide format. For that, I want to keep the &lt;code&gt;gene_raw&lt;/code&gt;, use &lt;code&gt;patient_id&lt;/code&gt; for the column names and &lt;code&gt;gene_code&lt;/code&gt; for the row names. How can I do this easily?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Assuming you know how to read that into R as a data.frame that I'll call &lt;code&gt;d&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyr)&#xA;library(dplyr)&#xA;reformatted = d %&amp;gt;% select(patient_id, gene_code, gene_raw) %&amp;gt;% spread(patient_id, gene_raw) %&amp;gt;% column_to_rownames(gene_code)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or something along those lines.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-11-06T08:56:02.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14738" PostTypeId="1" AcceptedAnswerId="14743" CreationDate="2020-11-06T09:45:02.513" Score="6" ViewCount="244" Body="&lt;p&gt;Both file formats are advertised as more memory/parsing efficent than old PDB format and its successor mmCIF format, but I can't understand the context (I'm a developer, but pretty new to the field of bioinformatics).&lt;/p&gt;&#xA;&lt;p&gt;Is BinaryCIF a later iteration of MMTF concepts/ideas (so it's a new competing format) or is the purpose of the two formats different ?&lt;/p&gt;&#xA;&lt;p&gt;Refs :&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;BinaryCIF: &lt;a href=&quot;https://github.com/dsehnal/BinaryCIF&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/dsehnal/BinaryCIF&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;MMTF: &lt;a href=&quot;http://mmtf.rcsb.org/index.html&quot; rel=&quot;noreferrer&quot;&gt;http://mmtf.rcsb.org/index.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9818" LastActivityDate="2021-09-29T04:27:00.717" Title="BinaryCIF vs MMTF formats, which one to choose?" Tags="|file-formats|pdb|3d-structure|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14739" PostTypeId="1" CreationDate="2020-11-07T08:33:37.260" Score="1" ViewCount="1004" Body="&lt;p&gt;Hello I am a Pymol user&lt;/p&gt;&#xA;&lt;p&gt;I want to color all alpha carbons (ca) and beta carbons (cb) from a certain ressidue in different color. How can I do this?&lt;/p&gt;&#xA;&lt;p&gt;I selected the spheres with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;show spheres, C/1-9/ca &#xA;show spheres, C/1-9/cb&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and if I use&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;set sphere_color, theColor&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I color all the spheres what I don't want.&lt;/p&gt;&#xA;&lt;p&gt;according to: &lt;code&gt;color    gold,    resi    200-­‐221&lt;/code&gt;&lt;br /&gt;&#xA;I tried&#xA;&lt;code&gt;gold,    spheres, C/1-9/ca&lt;/code&gt;&#xA;but this didn't work&lt;/p&gt;&#xA;" OwnerUserId="9824" LastEditorUserId="9824" LastEditDate="2020-11-07T08:41:38.880" LastActivityDate="2021-08-23T10:51:33.643" Title="Pymol: color spheres with different colors" Tags="|pymol|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14740" PostTypeId="1" CreationDate="2020-11-07T16:32:19.693" Score="2" ViewCount="142" Body="&lt;p&gt;If you sequence a positive-sense  RNA virus genome using NGS via Nanopore, Illumina or even the old Sanger method, is it possible to tell if the virus has 5' (five prime) cap at the 5' end of the RNA genome?&lt;/p&gt;&#xA;" OwnerUserId="1684" LastEditorUserId="3848" LastEditDate="2020-11-28T15:16:24.920" LastActivityDate="2023-09-15T09:01:20.097" Title="mRNA cap identification by sequencing?" Tags="|sequence-alignment|covid-19|sars-cov-2|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14741" PostTypeId="1" CreationDate="2020-11-07T16:48:32.997" Score="1" ViewCount="62" Body="&lt;p&gt;If you look at the Coronavirus genome below, it contains poly-A region at the end. I know that the viral RNA contains a poly-A tail. But, the genome in GenBank is stored in cDNA format. That means the end should have poly-T instead of A. Can anybody help me to understand this, please ?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;MN988713.1 Wuhan seafood market pneumonia virus isolate 2019-nCoV/USA-IL1/2020, complete genome&#xA;ATTAAAGGTTTATACCTTCCCAGGTAACAAACCAACCAACTTTCGATCTCTTGTAGATCTGTTCTCTAAA&#xA;--------------------------------------------------------------------&#xA;GATCGAGTGTACAGTGAACAATGCTAGGGAGAGCTGCCTATATGGAAGAGCCCTAATGTGTAAAATTAAT&#xA;TTTAGTAGTGCTATCCCCATGTGATTTTAATAGCTTCTTAGGAGAATGACAAAAAAAAAAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1684" LastEditorUserId="3848" LastEditDate="2020-12-04T15:36:19.050" LastActivityDate="2020-12-04T15:36:19.050" Title="Why does Coronavirus genome contain poly A region at the 3' end in GenBank?" Tags="|genome|covid-19|sars-cov-2|coronavirus|" AnswerCount="2" CommentCount="0" ClosedDate="2020-11-10T18:28:57.720" ContentLicense="CC BY-SA 4.0" />
  <row Id="14742" PostTypeId="2" ParentId="14739" CreationDate="2020-11-07T18:06:17.653" Score="1" Body="&lt;h2&gt;Spheres&lt;/h2&gt;&#xA;&lt;p&gt;So how PyMOL stores how it shows atoms is that each atom has a colour and a represention, which is a binary number telling what to show. Therefore the colour for &lt;code&gt;spheres&lt;/code&gt; is the same as the colour for &lt;code&gt;sticks&lt;/code&gt;. The only way round this is to copy the object (&lt;code&gt;create copycat, original&lt;/code&gt; and you set copycat differently)&lt;/p&gt;&#xA;&lt;p&gt;To set the atom you do&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;select iliketheschaps, resi 200‐221 and name CA+CB&#xA;color tv_green, iliketheschaps&#xA;show spheres, iliketheschaps&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The object &lt;code&gt;iliketheschaps&lt;/code&gt; is a selection. Do make sure the range is a hyphen-minus (character next to 0) as opposed to an en-dash (–) which Word subs and on a mac you get with &lt;kbd&gt;cmd&lt;/kbd&gt;+&lt;kbd&gt;-&lt;/kbd&gt;&lt;/p&gt;&#xA;&lt;h2&gt;non-PyMOL colours&lt;/h2&gt;&#xA;&lt;p&gt;Now you asked for &amp;quot;gold&amp;quot;. This is not a colour in the menu —famous for the colour &lt;code&gt;limon&lt;/code&gt;. Gold is a CSS colour name, but it won't work.&#xA;You will have to colour by hex code. Say you want &amp;quot;gold&amp;quot; you need to google the colour name and hex. #FFD700.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;color 0xFFD700, iliketheschaps&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;The Fashion Police have been alerted&lt;/h2&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This paragraph is in jest.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However, as you may know bioinformaticians are very fashion savvy for colours. Mustard was the in-colour from last year, so you really ought to use mustard (&lt;code&gt;0xffdb58&lt;/code&gt;). This year is all about nectarine (&lt;code&gt;0xffa168&lt;/code&gt;) (which is not salmon —this ain't 2015!). Coral (&lt;code&gt;0xff7f50&lt;/code&gt;) and teal (&lt;code&gt;0x008080&lt;/code&gt;) were from previous years and are still are going strong!&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-11-07T18:06:17.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14743" PostTypeId="2" ParentId="14738" CreationDate="2020-11-07T19:41:34.710" Score="6" Body="&lt;p&gt;So binaryCIF is newer and is heavily inspired by MMTF.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/nglviewer/ngl&quot; rel=&quot;nofollow noreferrer&quot;&gt;NGL&lt;/a&gt;, the most &amp;quot;loved&amp;quot; JS library (IMO) to show protein fetches proteins as MMTF, while Mol* uses bCIF —Mol* has the official repo of the bCIF format. Alex Rose, developed the former at the RCSB PDB, and the latter in collaboration with the PDBe. He also worked with Antony Bradley, the main author on the MMTF format, who left the RCSB PDB, some 4+ years ago. So there is definitely no competition there.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The MMTF download of the RCSB works well: &lt;a href=&quot;https://mmtf.rcsb.org/download.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://mmtf.rcsb.org/download.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;The MMTF format did not catch on —okay, mmCIF is only slowly replacing PDB, which is still the only accepted format for analyses—, hence the need for a bCIF format&lt;/li&gt;&#xA;&lt;li&gt;The libraries to do the conversion aren't without glitches&lt;/li&gt;&#xA;&lt;li&gt;In the distant future bCIF will be the sole binary format&lt;/li&gt;&#xA;&lt;li&gt;mmCIF &amp;lt;--&amp;gt; bCIF conversion is easier&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, I really ought to stress that mmCIF is a better choice to bCIF or mmTF. And even then many tools still hate mmCIFs and use PDB format.&#xA;If memory footprint is not really a problem, then stick with mmCIF. If you are running a webserver, you may be sending data as gzipped making this a moot point.&lt;/p&gt;&#xA;&lt;p&gt;Last year for my site, &lt;a href=&quot;https://michelanglo.sgc.ox.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;michelanglo&lt;/a&gt;, I was faced with a dilemma: Mol* has dotted lines for gaps and does not crash the browser with a virus or  someone wanting to show hyperball representation on all residues. However, it was not documented (and then glitchy) so I firmly stuck with NGL, which is excellent. For file formats, I stick with PDB and a metadata dictionary as I know the format well and is despite having the jankiest syntax and no real metadata support (REMARK... ha!), does not break isopetides, disulfides or for no apparent reason.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="73" LastEditDate="2021-09-29T04:27:00.717" LastActivityDate="2021-09-29T04:27:00.717" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14744" PostTypeId="1" CreationDate="2020-11-07T20:05:20.443" Score="1" ViewCount="12" Body="&lt;p&gt;Is there any database that give information about the drug class. Right now I have all the MeSH id of the drugs from different publications. So right now I'm looking one by one drug looking at the drug description,usage as well as I'm looking at this &lt;a href=&quot;https://meshb.nlm.nih.gov/record/ui?name=Mitoxantrone&quot; rel=&quot;nofollow noreferrer&quot;&gt;MeSh tree structure&lt;/a&gt;. There are many drug which overlaps mostly due to the core structure . For example Doxorubicin and Aclarubicin both have the same origin which is Anthracyclines. I would like to group them together based on this first.&lt;/p&gt;&#xA;&lt;p&gt;So Is there any database where I can make query in batches to get my result.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;drug description&lt;/li&gt;&#xA;&lt;li&gt;usage&lt;/li&gt;&#xA;&lt;li&gt;MeSH tree structure or similar information which gives output about the drugs hierarchical classification.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="285" LastActivityDate="2020-11-07T20:05:20.443" Title="Drug classification database information" Tags="|public-databases|data-retrieval|drugs|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14745" PostTypeId="1" CreationDate="2020-11-07T22:58:08.140" Score="1" ViewCount="23" Body="&lt;p&gt;I have gene expression datasets A and B that contain as many rows as genes and as many columns as samples. The rows in A and B represent a common set of genes measured in different tissues of the same 5 patients. My objective is to do a correlation between the set of genes in A and B to infer whether genes in both tissues are co-expressed based on the given condition or not?&lt;/p&gt;&#xA;&lt;p&gt;Here are sample and patients IDs information with their Expression profile&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Patient_ID &amp;lt;- as.character(quote(c(p_1, p_2,p_3, p_4, p_5)))[-1] &#xA;SampleA &amp;lt;- as.character(quote(c(A_1, A_2, A_3, A_4, A_5)))[-1]&#xA;SampleB &amp;lt;- as.character(quote(c(B_1, B_2, B_3, B_4, B_5)))[-1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Expression profile of the two tissues&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;set.seed(1)&#xA;A &amp;lt;- matrix(rnorm(250), nrow = 50)&#xA;set.seed(2)&#xA;B &amp;lt;- matrix(rnorm(250), nrow = 50)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have tried two approaches&lt;/p&gt;&#xA;&lt;p&gt;first approach&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse)&#xA;rownames(A) &amp;lt;- paste0(&amp;quot;g_&amp;quot;,1:50) ## genes &#xA;colnames(A) &amp;lt;- Patient_ID # patient IDs &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;prefix &amp;quot;b.&amp;quot; is added to every gene in Tissue B to identify that they are from tissue B.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rownames(B) &amp;lt;- paste0(&amp;quot;b.g_&amp;quot;,1:50) ## genes &#xA;colnames(B) &amp;lt;- Patient_ID  ## patient IDs &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Since both have the same column IDs, I used rbind function to join the two datasets.&#xA;Then I did correlation as a transposed matrix and extracted the top right bock of the matrix where I can get correlation values between Tissue A and Tissue B&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bind &amp;lt;- Hmisc::rcorr(t(rbind(A,B)))&#xA;n = nrow(A)&#xA;final &amp;lt;- bind$r[1:n, (n+1):(2*n)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Second approach&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rownames(A) &amp;lt;- paste0(&amp;quot;g_&amp;quot;,1:50) ## genes &#xA;colnames(A) &amp;lt;- SampleA ## Sample IDs in A&#xA;&#xA;rownames(B) &amp;lt;- paste0(&amp;quot;g_&amp;quot;,1:50) ## genes in B tissue  &#xA;colnames(B) &amp;lt;- SampleB  ## Sample IDs in B &#xA;bind.col &amp;lt;- Hmisc::rcorr(t(cbind(A,B)))&#xA;bind.col &amp;lt;- bind.col$r&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Therefore, I would like to ask which approach is correct to determine if genes of Tissue A and Tissue B are co-expressed based on the given condition?&#xA;If both are incorrect I wish to get feedback on how to proceed?&#xA;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="9160" LastActivityDate="2020-11-07T22:58:08.140" Title="co-expression analysis for two different tissues of same sample" Tags="|r|gene-expression|correlation|coexpression|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14747" PostTypeId="1" AcceptedAnswerId="14763" CreationDate="2020-11-08T11:35:22.167" Score="1" ViewCount="1039" Body="&lt;p&gt;Seq =&#xA;AAAAAAAAAAATTTTTTTTTTTGAATTCCCCCCCCCCCGGGGGGGGGGG&lt;/p&gt;&#xA;&lt;p&gt;Restriction site = GAATTC&lt;/p&gt;&#xA;&lt;p&gt;I want to cut the sequence at GA/ATTC&lt;/p&gt;&#xA;&lt;p&gt;I tried split method in python but It does not cut at GA/ATTC.&#xA;Please help&lt;/p&gt;&#xA;" OwnerUserId="9397" LastEditorUserId="3848" LastEditDate="2020-11-11T11:45:36.203" LastActivityDate="2020-11-11T11:45:36.203" Title="I want to write python program to cut a DNA sequence at an EcoRI restriction site and print the two fragments after cutting" Tags="|python|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14748" PostTypeId="2" ParentId="3583" CreationDate="2020-11-08T13:54:27.530" Score="0" Body="&lt;pre&gt;&lt;code&gt;def revcomplement(s):&#xA;    s = s.upper()&#xA;    c = { &amp;quot;A&amp;quot;:&amp;quot;T&amp;quot;,&amp;quot;C&amp;quot;:&amp;quot;G&amp;quot;,&amp;quot;T&amp;quot;:&amp;quot;A&amp;quot;,&amp;quot;G&amp;quot;:&amp;quot;C&amp;quot;}&#xA;    co = str()&#xA;    for i in s:&#xA;        co = c[i] + co&#xA;    return co&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9397" LastEditorUserId="3541" LastEditDate="2020-11-08T15:01:38.253" LastActivityDate="2020-11-08T15:01:38.253" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14749" PostTypeId="2" ParentId="14730" CreationDate="2020-11-08T18:59:34.680" Score="2" Body="&lt;p&gt;There seem to be WGS assemblies that can't be found in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Assembly database&lt;/a&gt;, e.g.: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/1779902990&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/1779902990&lt;/a&gt;. I guess that such assemblies also cannot be found in The assembly_summary.txt files that are described in &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/genomes/README_assembly_summary.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nlm.nih.gov/genomes/README_assembly_summary.txt&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My current best guess is that each WGS assembly has a &amp;quot;WGS master record&amp;quot; in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Nuccore database&lt;/a&gt;. To find all WGS assemblies of a taxon whose uid in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/taxonomy&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Taxonomy database&lt;/a&gt; is &lt;code&gt;1337&lt;/code&gt;, search &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Nuccore database&lt;/a&gt; using &lt;a href=&quot;https://biopython.org/docs/latest/api/Bio.Entrez.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez&lt;/a&gt; and the search term &lt;code&gt;&amp;quot;wgs master&amp;quot;[Properties] AND txid1337[orgn:exp]&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4868" LastActivityDate="2020-11-08T18:59:34.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14750" PostTypeId="2" ParentId="14518" CreationDate="2020-11-08T19:24:01.217" Score="2" Body="&lt;p&gt;If you want to remove rRNA, use tools that are made for this. We usually go for &lt;a href=&quot;https://github.com/biocore/sortmerna/wiki/2.-User-manual-(todo)#Filtering_pairedend_reads&quot; rel=&quot;nofollow noreferrer&quot;&gt;SortMeRNA&lt;/a&gt;, which can do exactly what you describe your goal is, &lt;em&gt;i.e.&lt;/em&gt; go from FASTQ to FASTQ, splitting your reads into rRNA and non-rRNA.&lt;/p&gt;&#xA;&lt;p&gt;In our experience, it is very accurate, if a bit slow at times.&lt;/p&gt;&#xA;" OwnerUserId="9693" LastActivityDate="2020-11-08T19:24:01.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14751" PostTypeId="1" CreationDate="2020-11-08T23:10:38.573" Score="2" ViewCount="164" Body="&lt;p&gt;I am new in this field and I am having some problems regarding a new project.&lt;/p&gt;&#xA;&lt;p&gt;I built a graph using Drugbank Data connected to SIDER Adverse Reactions.&#xA;I used Organ- level Terms to classify the ADRs in 25 different groups.&#xA;I aim to use the algorithm Metapath2vec in order to cluster drugs based on the group of ADRs they lead to.&#xA;I am currently using the python package for Machine Learning on graphs Stellargraph and the related implementation of Metapath2vec.&lt;/p&gt;&#xA;&lt;p&gt;I chose Clustering as downstream task with the resulting node embeddings, e.g. DBSCAN but the results are not promising.&#xA;I would like to have some clusters of drugs related to different adverse drug reaction.&#xA;Since I am new in this field everything I am trying is based on scientific literature but I don't know if this is the right approach for my objective.&lt;/p&gt;&#xA;&lt;p&gt;This is the code related to the metapath2vec algorithm:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;walk_length = 100  # maximum length of a random walk to use throughout this notebook&#xA;&#xA;# specify the metapath schemas as a list of lists of node types.&#xA;metapaths = [&#xA;    [&amp;quot;drug&amp;quot;, &amp;quot;adr&amp;quot;, &amp;quot;drug&amp;quot;],&#xA;    [&amp;quot;drug&amp;quot;, &amp;quot;adr&amp;quot;, &amp;quot;drug&amp;quot;, &amp;quot;drug&amp;quot;],&#xA;    [&amp;quot;drug&amp;quot;, &amp;quot;drug&amp;quot;],&#xA;    [&amp;quot;drug&amp;quot;, &amp;quot;adr&amp;quot;, &amp;quot;group_adr&amp;quot;, &amp;quot;adr&amp;quot;, &amp;quot;drug&amp;quot;],&#xA;    ]&#xA;&#xA;# Create the random walker&#xA;rw = UniformRandomMetaPathWalk(graph)&#xA;&#xA;walks = rw.run(&#xA;    nodes=list(graph.nodes()),  # root nodes&#xA;    length=walk_length,  # maximum length of a random walk&#xA;    n=1,  # number of random walks per root node&#xA;    metapaths=metapaths,  # the metapaths&#xA;)&#xA;&#xA;&#xA;from gensim.models import Word2Vec&#xA;&#xA;model = Word2Vec(walks, size=128, window=5, min_count=0, sg=1, workers=2, iter=1)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Should I change approach?&#xA;what other algorithm for representation learning can I use in order to reach my goal?&#xA;what can I improve in the presented approach to have better node embeddings?&lt;/p&gt;&#xA;" OwnerUserId="9830" LastEditorUserId="3848" LastEditDate="2020-11-11T11:31:23.030" LastActivityDate="2020-11-11T11:31:23.030" Title="Metapath2vec on Drug-ADR Heterogeneous Graph" Tags="|networks|machine-learning|deep-learning|pytorch|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14752" PostTypeId="1" AcceptedAnswerId="14757" CreationDate="2020-11-09T02:05:15.893" Score="0" ViewCount="2859" Body="&lt;p&gt;I used FindMarkers to find the DEGs between two clusters in my dataset using Seurat. I am trying to visualize the outcome using a heatmap but I failed to write the command in R.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dput(cluster4_3.markers)&#xA;              p_val  avg_logFC pct.1 pct.2    p_val_adj&#xA;Vim    1.803527e-44 -1.1689785 0.996 0.999 3.458985e-40&#xA;Lgals7 1.479278e-38  1.7615232 0.993 0.552 2.837108e-34&#xA;Anxa2  1.797016e-37 -1.3971764 0.629 0.991 3.446497e-33&#xA;Tmsb10 6.936830e-31  0.7937881 1.000 1.000 1.330415e-26&#xA;Ptma   7.518439e-27  0.5622808 1.000 1.000 1.441961e-22&#xA;Jund   2.044576e-25 -0.9881766 0.993 0.999 3.921293e-21&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I tried:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;DoHeatmap(cluster4_3.markers, features = NULL) + &#xA;scale_fill_gradientn(colors = rev(RColorBrewer::brewer.pal(n = 7, name = &amp;quot;RdYlBu&amp;quot;)))&#xA;Error in UseMethod(generic = &amp;quot;DefaultAssay&amp;quot;, object = object) : &#xA;  no applicable method for 'DefaultAssay' applied to an object of class &amp;quot;data.frame&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It didn't work. I was trying to replicate this command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;top10 &amp;lt;- pbmc.markers %&amp;gt;% group_by(cluster) %&amp;gt;% top_n(n = 10, wt = avg_logFC)&#xA;DoHeatmap(pbmc, features = top10$gene) + NoLegend() + &#xA;  scale_fill_gradientn(colors = rev(RColorBrewer::brewer.pal(n = 7, name = &amp;quot;RdYlBu&amp;quot;)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried to extract some info from the csv file but I couldn't determine which parameter is important (p_val avg_logFC   pct.1   pct.2   p_val_adj). Also, I didn't know which parameter to use to use the group_by function. That's one of my attempts:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;clus4vs3 &amp;lt;- cluster4_3.markers$p_val %&amp;gt;% &#xA;    group_by(pct.1) %&amp;gt;% &#xA;    top_n(n = 10, wt = avg_logFC)&#xA;Error in UseMethod(&amp;quot;group_by_&amp;quot;) :&#xA;  no applicable method for 'group_by_' applied to an object of class &amp;quot;c('double', 'numeric')&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now that I understand 'group_by' only accepts a categorical variable, how can I write a command that displays the top20 differentially expressed genes between cluster 4 and 3 in a heatmap ?&lt;/p&gt;&#xA;" OwnerUserId="9831" LastEditorUserId="9831" LastEditDate="2020-11-10T01:42:37.010" LastActivityDate="2020-11-10T01:42:37.010" Title="Visualizing FindMarkers result in Seurat using Heatmap" Tags="|seurat|heatmap|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14753" PostTypeId="2" ParentId="14741" CreationDate="2020-11-09T03:41:15.640" Score="0" Body="&lt;p&gt;I figured out that it has to do with second strand of cDNA. Usually genbank sequences are equivalent to 5’ to 3’ strand of cDNA. This strand is generated using second strand synthesis enzyme.&lt;/p&gt;&#xA;" OwnerUserId="1684" LastActivityDate="2020-11-09T03:41:15.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14754" PostTypeId="2" ParentId="14741" CreationDate="2020-11-09T08:27:43.463" Score="0" Body="&lt;p&gt;No its definitely a poly-A tail for both the &lt;em&gt;in situ&lt;/em&gt; virus and the cDNA sequence. It couldn't be a poly-T &lt;em&gt;in situ&lt;/em&gt; because,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the Ts are uracils (Us) because its an RNA virus. Thus if you substituted all the 'T's for 'U's you'd recover the &lt;em&gt;in situ&lt;/em&gt; genome&lt;/li&gt;&#xA;&lt;li&gt;specifically its a positive single-stranded RNA virus and the cDNA isn't a reverse complement but the positive strand.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;If you look at the poly-A together with the 5' capping then you get the idea its to assist translocation and transcription to the endoplasmic reticulum. The genome pulls a few tricks which make it a bit more complicated than that, but thats the gist.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-11-09T08:27:43.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14755" PostTypeId="1" AcceptedAnswerId="14756" CreationDate="2020-11-09T08:48:39.327" Score="0" ViewCount="94" Body="&lt;p&gt;I would like to ask if there is any tool or R script that translates Ensembl IDs to gene names in a gprofiler GO output. An example of this file is the following one:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/1h2UP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1h2UP.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8713" LastActivityDate="2020-11-09T11:27:33.687" Title="gprofiler output ENSG to gene name" Tags="|gene-ontology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14756" PostTypeId="2" ParentId="14755" CreationDate="2020-11-09T09:34:06.813" Score="1" Body="&lt;p&gt;This is usually done using &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/AnnotationDbi.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;AnnotationDbi&lt;/a&gt; in combination with the annotation database for human &lt;a href=&quot;https://bioconductor.org/packages/release/data/annotation/html/org.Hs.eg.db.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;org.Hs.eg.db&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(org.Hs.eg.db)&#xA;genes &amp;lt;- c(&amp;quot;ENSG00000102055&amp;quot;, &amp;quot;ENSG00000232605&amp;quot;, &amp;quot;ENSG00000136828&amp;quot;,&#xA;           &amp;quot;ENSG00000159197&amp;quot;, &amp;quot;ENSG00000201394&amp;quot;, &amp;quot;ENSG00000278580&amp;quot;,&#xA;           &amp;quot;ENSG00000213440&amp;quot;, &amp;quot;ENSG00000166816&amp;quot;, &amp;quot;ENSG00000275895&amp;quot;,&#xA;           &amp;quot;ENSG00000105723&amp;quot;)&#xA;select(org.Hs.eg.db, genes, c(&amp;quot;ENSEMBL&amp;quot;, &amp;quot;ALIAS&amp;quot;), &amp;quot;ENSEMBL&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This returns:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;           ENSEMBL     ALIAS&#xA;1  ENSG00000102055       I-4&#xA;2  ENSG00000102055  PPP1R2P9&#xA;3  ENSG00000102055   PPP1R2C&#xA;4  ENSG00000232605      &amp;lt;NA&amp;gt;&#xA;5  ENSG00000136828   RALGEF2&#xA;6  ENSG00000136828  RALGPS1A&#xA;7  ENSG00000136828   RALGPS1&#xA;8  ENSG00000159197     ATFB4&#xA;9  ENSG00000159197      LQT5&#xA;10 ENSG00000159197      LQT6&#xA;11 ENSG00000159197     MIRP1&#xA;12 ENSG00000159197     KCNE2&#xA;13 ENSG00000201394   RN5S140&#xA;14 ENSG00000201394 RNA5SP140&#xA;15 ENSG00000278580      &amp;lt;NA&amp;gt;&#xA;16 ENSG00000213440      &amp;lt;NA&amp;gt;&#xA;17 ENSG00000166816     DLACD&#xA;18 ENSG00000166816       DLD&#xA;19 ENSG00000166816      LDHD&#xA;20 ENSG00000275895   U2AF1L5&#xA;21 ENSG00000105723     GSK3A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you want to apply a 1:1 mapping of gene IDs to common names, you can use the &amp;quot;SYMBOL&amp;quot; column. In a &lt;code&gt;data.frame&lt;/code&gt;, you can do something like this:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;ensembl_to_symbol &amp;lt;- function(e) {&#xA;  hits &amp;lt;- select(org.Hs.eg.db, e, &amp;quot;SYMBOL&amp;quot;, &amp;quot;ENSEMBL&amp;quot;)$SYMBOL&#xA;  # Gene IDs without symbols should just stay as IDs&#xA;  no_hit &amp;lt;- is.na(hits)&#xA;  hits[no_hit] &amp;lt;- e[no_hit]&#xA;  hits&#xA;}&#xA;&#xA;recode_symbols &amp;lt;- function(s) {&#xA;  e &amp;lt;- ensembl_to_symbol(unlist(strsplit(s, split = &amp;quot;,&amp;quot;)))&#xA;  paste(e, collapse = &amp;quot;,&amp;quot;)&#xA;}&#xA;&#xA;df &amp;lt;- data.frame(&#xA;  genes = c(paste(genes[1:5], collapse = &amp;quot;,&amp;quot;),&#xA;            paste(genes[6:10], collapse = &amp;quot;,&amp;quot;)&#xA;  )&#xA;)&#xA;&#xA;df&lt;span class=&quot;math-container&quot;&gt;$genes &amp;lt;- sapply(df$&lt;/span&gt;genes, recode_symbols)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This isn't the most vectorized (fast) solution, but it's probably easier to understand this way.&lt;/p&gt;&#xA;" OwnerUserId="9693" LastEditorUserId="9693" LastEditDate="2020-11-09T11:27:33.687" LastActivityDate="2020-11-09T11:27:33.687" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14757" PostTypeId="2" ParentId="14752" CreationDate="2020-11-09T12:23:43.847" Score="0" Body="&lt;p&gt;In your &lt;code&gt;DoHeatmap()&lt;/code&gt; call, you do not provide features so the function does not know which genes/features to use for the heatmap.&lt;/p&gt;&#xA;&lt;p&gt;In your last function call, you are trying to group based on a continuous variable &lt;code&gt;pct.1&lt;/code&gt; whereas &lt;code&gt;group_by&lt;/code&gt; expects a categorical variable.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-11-09T12:23:43.847" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="14758" PostTypeId="1" CreationDate="2020-11-09T14:51:11.527" Score="2" ViewCount="38" Body="&lt;p&gt;For example, I have a paper that says: &amp;quot;the atomic RMS distribution about the mean coordinate positions is 0.41 +/- 0.04 A for the backbone atoms and 0.82 +/- 0.04 A for all atoms&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;And I have no idea what any of this means, I found several papers with this kind of information but none of them explain why this is relevant in order to understand the structure of a protein.&lt;/p&gt;&#xA;" OwnerUserId="9832" LastEditorUserId="650" LastEditDate="2020-11-10T18:30:00.683" LastActivityDate="2021-12-05T23:07:00.867" Title="How can atomic RMS differences / atomic RMS distributions help us understand the structure of a protein?" Tags="|protein-structure|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14759" PostTypeId="1" CreationDate="2020-11-09T15:13:32.860" Score="1" ViewCount="1274" Body="&lt;p&gt;Question: Why is there a minus in the &lt;code&gt;2ˆ(–delta delta CT)&lt;/code&gt; formula?&lt;/p&gt;&#xA;&lt;p&gt;My line of thoughs:&lt;/p&gt;&#xA;&lt;p&gt;Consider &lt;code&gt;Ct&lt;/code&gt; values of some pPCR experiment (some gene of interest X and a reference gene REF is measured under some experiment condition and a control condition).&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The delta CT (&lt;code&gt;dCt&lt;/code&gt;) would correspond to normalized &lt;s&gt;expression&lt;/s&gt; Ct of X against REF of both conditions. (dCt = X-REF)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The delta delta CT (&lt;code&gt;ddCt&lt;/code&gt;) would then correspond the the difference of the dCt of the experiment condition minus the dCt of the control condition.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;=&amp;gt; If ddCT &amp;gt; 0 then the gene of interest is &lt;s&gt;upregulated&lt;/s&gt; downregulated (Ct is antiproportional to gene expression). (the difference of the dCt in the experiment condition is larger than in the control condition)&lt;/p&gt;&#xA;&lt;p&gt;But now: the ratio &lt;code&gt;R = 2ˆ(–delta delta CT)&lt;/code&gt; would then correspond to a value smaller than 1 (because of the minus).&lt;/p&gt;&#xA;&lt;p&gt;I read sentences like&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;We see a 10% decrease of expression (R=0.9)...&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;s&gt;But actually the raw data clearly shows a upregulation&lt;/s&gt;... This is confusing since only the ratio R decreases by 10% but the this would correspond to an increase of raw expression...&lt;/p&gt;&#xA;&lt;p&gt;&lt;s&gt;I could interprete R as the ratio of the control against the gene of interest. But why not the other way around?&lt;/s&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit: Ct is antiproportional to gene expression =&amp;gt; dCt does not directly reflect the gene expression.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Sources:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-020-03696-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-020-03696-y&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;We can predict that a plant with a hormone concentration of&#xA;8.85 pg/mL would have an expression of Gene G that is 27% (̂ =0.73) lower than that of plants with average hormone concentration&amp;quot; (just&#xA;below equation 8)&lt;/p&gt;&#xA;&lt;p&gt;But clearly in Fig. 3a you can see that the expression at 8.85 is&#xA;obviously higher than with average hormone concentration (x0 = 9.85)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2766" LastEditorUserId="2766" LastEditDate="2020-11-26T14:32:41.377" LastActivityDate="2020-11-26T14:32:41.377" Title="Why is there a minus in the 2ˆ(–delta delta CT) method (qPCR)" Tags="|normalization|qpcr|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14760" PostTypeId="1" CreationDate="2020-11-09T15:26:33.000" Score="0" ViewCount="40" Body="&lt;p&gt;Lets say I performed a single cell RNA sequencing analysis on mice blood samples. Eventually I am able to cluster cells together and get a set of genes of interest for each cell type since I want to study the gene expression of specific cells after a treatment.&lt;/p&gt;&#xA;&lt;p&gt;At this point I want to perform an enrichment analysis to get a list of those GO biological processes and/or molecular functions that might be affected by the novel genes that are up or down regulated after the treatment. I can use several tools such as ClusterProfiler, Panther, EnrichR but, and this is the point, should I use gene sets that are compiled for mouse only or can I enrich my genes using gene sets compiled from other organisms too such as, for instance, human gene sets? Is it ok to mix genes and gene sets from different organisms when performing an enrichment?&lt;/p&gt;&#xA;" OwnerUserId="3119" LastActivityDate="2020-11-09T15:26:33.000" Title="is it ok to use gene sets from an organism to enrich genes from a different organism?" Tags="|go-enrichment|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14761" PostTypeId="1" CreationDate="2020-11-10T01:26:19.700" Score="1" ViewCount="811" Body="&lt;p&gt;I was curious to give &lt;a href=&quot;https://github.com/jerryji1993/DNABERT&quot; rel=&quot;nofollow noreferrer&quot;&gt;DNA BERT&lt;/a&gt; a try. This is a &lt;a href=&quot;https://en.wikipedia.org/wiki/BERT_(language_model)&quot; rel=&quot;nofollow noreferrer&quot;&gt;BERT&lt;/a&gt; (Bidirectional Encoder Representations from Transformers) model that was trained on short (&lt;code&gt;k=3,4,5, or 6&lt;/code&gt;) k-mers of DNA. Overall, It is exciting work since it bridges NLP approaches and genetics data.&lt;/p&gt;&#xA;&lt;p&gt;I span up a pytroch3.6 machine on SageMaker with a GPU and tried to assess the prediction of its pre-trained DNABERT model:&lt;/p&gt;&#xA;&lt;p&gt;Preparing the data:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;!git clone https://github.com/jerryji1993/DNABERT&#xA;%cd DNABERT&#xA;!python -m pip install --editable .&#xA;%cd examples&#xA;!python -m pip install -r requirements.txt&#xA;!curl -L https://northwestern.app.box.com/shared/static/g8m974tr86h0pvnpymxq84f1yxlhnvbi --output a6-new-12w-0.zip&#xA;!unzip a6-new-12w-0.zip&#xA;!cp ./sample_data/ft/prom-core/6/dev.tsv /home/ec2-user/SageMaker/DNABERT/examples/6-new-12w-0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Running &lt;a href=&quot;https://github.com/jerryji1993/DNABERT#4-prediction&quot; rel=&quot;nofollow noreferrer&quot;&gt;prediction&lt;/a&gt; based on the pre-trained data:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;!python run_finetune.py \&#xA;   --model_type=dna \&#xA;   --tokenizer_name=dna6 \&#xA;   --model_name_or_path=/home/ec2-user/SageMaker/DNABERT/examples/6-new-12w-0 \&#xA;   --task_name=dnaprom \&#xA;   --do_predict \&#xA;   --data_dir=/home/ec2-user/SageMaker/DNABERT/examples/sample_data/ft/prom-core/6  \&#xA;   --max_seq_length=75 \&#xA;   --per_gpu_pred_batch_size=128   \&#xA;   --output_dir=/home/ec2-user/SageMaker/DNABERT/examples/6-new-12w-0/out \&#xA;   --predict_dir=/home/ec2-user/SageMaker/DNABERT/examples/result \&#xA;   --n_process=4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;11/09/2020 04:35:57 - WARNING - __main__ -   Process rank: -1, device: cuda, n_gpu: 1, distributed training: False, 16-bits training: False&#xA;11/09/2020 04:35:57 - INFO - transformers.configuration_utils -   loading configuration file /home/ec2-user/SageMaker/DNABERT/examples/6-new-12w-0/config.json&#xA;11/09/2020 04:35:57 - INFO - transformers.configuration_utils -   Model config BertConfig {&#xA;  &amp;quot;architectures&amp;quot;: [&#xA;    &amp;quot;BertForMaskedLM&amp;quot;&#xA;  ],&#xA;  &amp;quot;attention_probs_dropout_prob&amp;quot;: 0.1,&#xA;  &amp;quot;bos_token_id&amp;quot;: 0,&#xA;  &amp;quot;do_sample&amp;quot;: false,&#xA;  &amp;quot;eos_token_ids&amp;quot;: 0,&#xA;  &amp;quot;finetuning_task&amp;quot;: &amp;quot;dnaprom&amp;quot;,&#xA;  &amp;quot;hidden_act&amp;quot;: &amp;quot;gelu&amp;quot;,&#xA;  &amp;quot;hidden_dropout_prob&amp;quot;: 0.1,&#xA;  &amp;quot;hidden_size&amp;quot;: 768,&#xA;  &amp;quot;id2label&amp;quot;: {&#xA;    &amp;quot;0&amp;quot;: &amp;quot;LABEL_0&amp;quot;,&#xA;    &amp;quot;1&amp;quot;: &amp;quot;LABEL_1&amp;quot;&#xA;  },&#xA;  &amp;quot;initializer_range&amp;quot;: 0.02,&#xA;  &amp;quot;intermediate_size&amp;quot;: 3072,&#xA;  &amp;quot;is_decoder&amp;quot;: false,&#xA;  &amp;quot;label2id&amp;quot;: {&#xA;    &amp;quot;LABEL_0&amp;quot;: 0,&#xA;    &amp;quot;LABEL_1&amp;quot;: 1&#xA;  },&#xA;  &amp;quot;layer_norm_eps&amp;quot;: 1e-12,&#xA;  &amp;quot;length_penalty&amp;quot;: 1.0,&#xA;  &amp;quot;max_length&amp;quot;: 20,&#xA;  &amp;quot;max_position_embeddings&amp;quot;: 512,&#xA;  &amp;quot;model_type&amp;quot;: &amp;quot;bert&amp;quot;,&#xA;  &amp;quot;num_attention_heads&amp;quot;: 12,&#xA;  &amp;quot;num_beams&amp;quot;: 1,&#xA;  &amp;quot;num_hidden_layers&amp;quot;: 12,&#xA;  &amp;quot;num_labels&amp;quot;: 2,&#xA;  &amp;quot;num_return_sequences&amp;quot;: 1,&#xA;  &amp;quot;num_rnn_layer&amp;quot;: 1,&#xA;  &amp;quot;output_attentions&amp;quot;: false,&#xA;  &amp;quot;output_hidden_states&amp;quot;: false,&#xA;  &amp;quot;output_past&amp;quot;: true,&#xA;  &amp;quot;pad_token_id&amp;quot;: 0,&#xA;  &amp;quot;pruned_heads&amp;quot;: {},&#xA;  &amp;quot;repetition_penalty&amp;quot;: 1.0,&#xA;  &amp;quot;rnn&amp;quot;: &amp;quot;lstm&amp;quot;,&#xA;  &amp;quot;rnn_dropout&amp;quot;: 0.0,&#xA;  &amp;quot;rnn_hidden&amp;quot;: 768,&#xA;  &amp;quot;split&amp;quot;: 10,&#xA;  &amp;quot;temperature&amp;quot;: 1.0,&#xA;  &amp;quot;top_k&amp;quot;: 50,&#xA;  &amp;quot;top_p&amp;quot;: 1.0,&#xA;  &amp;quot;torchscript&amp;quot;: false,&#xA;  &amp;quot;type_vocab_size&amp;quot;: 2,&#xA;  &amp;quot;use_bfloat16&amp;quot;: false,&#xA;  &amp;quot;vocab_size&amp;quot;: 4101&#xA;}&#xA;&#xA;============================================================&#xA;&amp;lt;class 'transformers.tokenization_dna.DNATokenizer'&amp;gt;&#xA;11/09/2020 04:35:57 - INFO - filelock -   Lock 140334164036072 acquired on /home/ec2-user/.cache/torch/transformers/ea1474aad40c1c8ed4e1cb7c11345ddda6df27a857fb29e1d4c901d9b900d32d.26f8bd5a32e49c2a8271a46950754a4a767726709b7741c68723bc1db840a87e.lock&#xA;11/09/2020 04:35:57 - INFO - transformers.file_utils -   https://raw.githubusercontent.com/jerryji1993/DNABERT/master/src/transformers/dnabert-config/bert-config-6/vocab.txt not found in cache or force_download set to True, downloading to /home/ec2-user/.cache/torch/transformers/tmpa9hkn2q1&#xA;Downloading: 28.7kB [00:00, 12.5MB/s]                                           &#xA;11/09/2020 04:35:57 - INFO - transformers.file_utils -   storing https://raw.githubusercontent.com/jerryji1993/DNABERT/master/src/transformers/dnabert-config/bert-config-6/vocab.txt in cache at /home/ec2-user/.cache/torch/transformers/ea1474aad40c1c8ed4e1cb7c11345ddda6df27a857fb29e1d4c901d9b900d32d.26f8bd5a32e49c2a8271a46950754a4a767726709b7741c68723bc1db840a87e&#xA;11/09/2020 04:35:57 - INFO - transformers.file_utils -   creating metadata file for /home/ec2-user/.cache/torch/transformers/ea1474aad40c1c8ed4e1cb7c11345ddda6df27a857fb29e1d4c901d9b900d32d.26f8bd5a32e49c2a8271a46950754a4a767726709b7741c68723bc1db840a87e&#xA;11/09/2020 04:35:57 - INFO - filelock -   Lock 140334164036072 released on /home/ec2-user/.cache/torch/transformers/ea1474aad40c1c8ed4e1cb7c11345ddda6df27a857fb29e1d4c901d9b900d32d.26f8bd5a32e49c2a8271a46950754a4a767726709b7741c68723bc1db840a87e.lock&#xA;11/09/2020 04:35:57 - INFO - transformers.tokenization_utils -   loading file https://raw.githubusercontent.com/jerryji1993/DNABERT/master/src/transformers/dnabert-config/bert-config-6/vocab.txt from cache at /home/ec2-user/.cache/torch/transformers/ea1474aad40c1c8ed4e1cb7c11345ddda6df27a857fb29e1d4c901d9b900d32d.26f8bd5a32e49c2a8271a46950754a4a767726709b7741c68723bc1db840a87e&#xA;11/09/2020 04:35:57 - INFO - transformers.modeling_utils -   loading weights file /home/ec2-user/SageMaker/DNABERT/examples/6-new-12w-0/pytorch_model.bin&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, the &lt;code&gt;result&lt;/code&gt; directory is empty and I am not sure what did I miss?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;!ls -lh /home/ec2-user/SageMaker/DNABERT/examples/result&#xA;total 0&#xA;&#xA;&#xA;!ls -l  /home/ec2-user/SageMaker/DNABERT/examples/6-new-12w-0/out&#xA;total 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&#xA;I have tried also to run this on another ec2 machine and apparently it gets a segmentation fault, so my conclusion is that it's unstable. I will train a RoBERTAa model or similar by myself:&lt;/p&gt;&#xA;&lt;p&gt;SageMaker doesn't propagate the segmentation fault to the notebook which is another weird thing about it.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;11/12/2020 14:01:18 - INFO - filelock -   Lock 139866003860056 acquired on /root/.cache/torch/transformers/ea1474aad40c1c8ed4e1cb7c11345ddda6df27a857fb29e1d4c901d9b900d32d.26f8bd5a32e49c2a8271a46950754a4a767726709b7741c68723bc1db840a87e.lock&#xA;11/12/2020 14:01:18 - INFO - transformers.file_utils -   https://raw.githubusercontent.com/jerryji1993/DNABERT/master/src/transformers/dnabert-config/bert-config-6/vocab.txt not found in cache or force_download set to True, downloading to /root/.cache/torch/transformers/tmpwpxfkxlm&#xA;Downloading: 28.7kB [00:00, 20.2MB/s]&#xA;11/12/2020 14:01:18 - INFO - transformers.file_utils -   storing https://raw.githubusercontent.com/jerryji1993/DNABERT/master/src/transformers/dnabert-config/bert-config-6/vocab.txt in cache at /root/.cache/torch/transformers/ea1474aad40c1c8ed4e1cb7c11345ddda6df27a857fb29e1d4c901d9b900d32d.26f8bd5a32e49c2a8271a46950754a4a767726709b7741c68723bc1db840a87e&#xA;11/12/2020 14:01:18 - INFO - transformers.file_utils -   creating metadata file for /root/.cache/torch/transformers/ea1474aad40c1c8ed4e1cb7c11345ddda6df27a857fb29e1d4c901d9b900d32d.26f8bd5a32e49c2a8271a46950754a4a767726709b7741c68723bc1db840a87e&#xA;11/12/2020 14:01:18 - INFO - filelock -   Lock 139866003860056 released on /root/.cache/torch/transformers/ea1474aad40c1c8ed4e1cb7c11345ddda6df27a857fb29e1d4c901d9b900d32d.26f8bd5a32e49c2a8271a46950754a4a767726709b7741c68723bc1db840a87e.lock&#xA;11/12/2020 14:01:18 - INFO - transformers.tokenization_utils -   loading file https://raw.githubusercontent.com/jerryji1993/DNABERT/master/src/transformers/dnabert-config/bert-config-6/vocab.txt from cache at /root/.cache/torch/transformers/ea1474aad40c1c8ed4e1cb7c11345ddda6df27a857fb29e1d4c901d9b900d32d.26f8bd5a32e49c2a8271a46950754a4a767726709b7741c68723bc1db840a87e&#xA;11/12/2020 14:01:18 - INFO - transformers.modeling_utils -   loading weights file /home/ec2-user/DNABERT/examples/6-new-12w-0/pytorch_model.bin&#xA;Segmentation fault&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2020-11-12T14:12:21.577" LastActivityDate="2020-11-12T14:12:21.577" Title="How to predict with the pre-trained DNABERT model?" Tags="|deep-learning|pytorch|python|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14762" PostTypeId="2" ParentId="14761" CreationDate="2020-11-10T01:46:32.797" Score="1" Body="&lt;p&gt;I don't why this RNN k-mer Pytorch model is not producing an output. I do know that,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; &amp;quot;rnn_dropout&amp;quot;: 0.0,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... is bad news for any ANN (artificial neural network) let alone LSTM RNN, which is a pretty complex model.&lt;/p&gt;&#xA;&lt;p&gt;Firstly, you have to assign a meaningful drop out rate and 0.5 is okay.&lt;/p&gt;&#xA;&lt;p&gt;If dropout rate = 0 as an output then the model hasn't worked. A dropout layer is essential to present overfitting. Its a foundational part of deep learning. No-one emperically knows quite why it works, there's a few approximate good guesses, but a dropout is essential.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The other thing to note is what does &amp;quot;n_process&amp;quot; mean? If you are parallelising a GPU across 4-cores, okay its got a lot of layers but even so thats alot of cores. It ain't a CPU and I'd get it working on a single GPU core first.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-11-10T01:51:57.023" LastActivityDate="2020-11-10T01:51:57.023" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14763" PostTypeId="2" ParentId="14747" CreationDate="2020-11-10T04:25:06.340" Score="3" Body="&lt;pre&gt;&lt;code&gt;import Bio&#xA;from Bio.Seq import Seq&#xA;my_seq = Seq(&amp;quot;aaaaaaaaaaatttttttttttgaattccccccccccggggggggggg&amp;quot;)&#xA;from Bio.Restriction import *&#xA;print(EcoRI.search(my_seq))&#xA;print(EcoRI.catalyse(my_seq))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Actully the EcoRI cuts the DNA after G in restriction site GAATTC.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9397" LastActivityDate="2020-11-10T04:25:06.340" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14765" PostTypeId="1" CreationDate="2020-11-10T13:46:34.500" Score="3" ViewCount="254" Body="&lt;p&gt;I had a question for the community.&lt;/p&gt;&#xA;&lt;p&gt;I have a genome of a new species that has been sequenced via 150pb Illumina paired-end.&lt;/p&gt;&#xA;&lt;p&gt;To verify the quality of the assembly I used the &lt;code&gt;BUSCO pipeline&lt;/code&gt; which allows me to search for 99% of the BUSCO genes (gene not duplicated) present in a taxon in the genome of my species.&lt;/p&gt;&#xA;&lt;p&gt;Globally I only miss &lt;code&gt;18%&lt;/code&gt; of the BUSCO genes, which means that the assembly is not too bad.&lt;/p&gt;&#xA;&lt;p&gt;Now when I plot the &lt;strong&gt;G+C%&lt;/strong&gt; and the &lt;strong&gt;coverage&lt;/strong&gt; of all the scaffolds containing BUSCO genes (so a priori non-contaminating) I should find a fairly homogeneous cloud.&lt;/p&gt;&#xA;&lt;p&gt;Instead I end up with two different coverage profiles (noted red and blue below):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ibb.co/6X34kfq&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I then thought that the low coverage sequencing profile around 4x might be a eukaryotic contaminant like a acarians, however when I blast the BUSCOs present on these low coverage scaffolds I get hits that are very close to the genome of my species...&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have any idea what these are?&lt;/p&gt;&#xA;&lt;p&gt;here is an output of the &lt;code&gt;contigsTable.csv&lt;/code&gt; where the two first row are scaffold with low coverage in the figure, and two last are scaffold with high coverage in the figure&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Dm85F.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Dm85F.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;FRC output&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Z9pTV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Z9pTV.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Coverage distribution&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jDU6S.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jDU6S.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is an Icarus summary of the assembly :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Assembly&#xA;# contigs (&amp;gt;= 0 bp) 1038631&#xA;# contigs (&amp;gt;= 1000 bp) 120132&#xA;# contigs (&amp;gt;= 5000 bp) 22245&#xA;# contigs (&amp;gt;= 10000 bp) 9986&#xA;# contigs (&amp;gt;= 25000 bp) 1322&#xA;# contigs (&amp;gt;= 50000 bp) 97&#xA;Total length (&amp;gt;= 0 bp) 655530043&#xA;Total length (&amp;gt;= 1000 bp) 451497861&#xA;Total length (&amp;gt;= 5000 bp) 260894112&#xA;Total length (&amp;gt;= 10000 bp) 174183054&#xA;Total length (&amp;gt;= 25000 bp) 45006012&#xA;Total length (&amp;gt;= 50000 bp) 5956145&#xA;# contigs 214542&#xA;Largest contig 120620&#xA;Total length 519138286&#xA;GC (%) 32.94&#xA;N50 5062&#xA;N75 1624&#xA;L50 21982&#xA;L75 71030&#xA;# N's per 100 kbp 22.28&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here is the BUSCO summary as well :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;2544 Complete BUSCOs (C)&#xA;2312 Complete and single-copy BUSCOs (S)&#xA;232 Complete and duplicated BUSCOs (D)&#xA;1060 Fragmented BUSCOs (F)&#xA;811 Missing BUSCOs (M)&#xA;4415 Total BUSCO groups searched&#xA;C:57.7%[S:52.4%,D:5.3%],F:24.0%,M:18.3%,n:4415&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;EDIT&lt;/p&gt;&#xA;&lt;p&gt;I ran the fastq sequences with another aligner (MEGAHIT) to see if it was not because of an assembly issue and I got the same profils coverage see here :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6cq6m.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6cq6m.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9104" LastEditorUserId="9104" LastEditDate="2020-11-14T14:07:25.050" LastActivityDate="2020-11-15T18:39:00.427" Title="Contamination on genome assembly" Tags="|assembly|read-mapping|coverage|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14766" PostTypeId="2" ParentId="14758" CreationDate="2020-11-10T15:46:35.880" Score="2" Body="&lt;p&gt;The &amp;quot;RMS distribution about the mean coordinate positions&amp;quot; could be restated as &amp;quot;certainty of the structure&amp;quot;. In other words, the main backbone that defines the large-scale shape in the structure is expected to be accurate to within about 0.41 angstrom, while the atoms more on the periphery (or otherwise not forming the backbone of the structure) are less accurately solved. How important this is will depend on what you want to do with the structure. If you want to get an idea for what it looks like then the accuracy is less important, if you want to do docking then the more accurate the atoms around any sort of pocket/interaction surface are the better.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-11-10T15:46:35.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14767" PostTypeId="1" AcceptedAnswerId="14848" CreationDate="2020-11-10T18:08:52.030" Score="-1" ViewCount="157" Body="&lt;p&gt;I have a relative abundance table in .tsv format where samples are in column and rows contain the features (pathways). Something like this reproducible example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Pathway   ERR2601 ERR26014    ERR260145_Abu   ERR260  ERR2601 ERR2601 ERR2607 ERR2600 ERR26&#xA;1CMET2-PWY: N10-formyl-tetrahydrofolate biosynthesis    0.00826631  0.00946472  0.00962571  0.00411153  0.00811879  0.00844421  0.00915602  0.00789428  0.0091798&#xA;3-HYDROXYPHENYLACETATE-DEGRADATION-PWY: 4-hydroxyphenylacetate degradation  0   0   0   0.000695608 0   0   0   0   0&#xA;7ALPHADEHYDROX-PWY: cholate degradation (bacteria, anaerobic)   0   0   0   0   0   0   0   0   0&#xA;AEROBACTINSYN-PWY: aerobactin biosynthesis  0   0   0   0   0   0   0   0   0&#xA;ALLANTOINDEG-PWY: superpathway of allantoin degradation in yeast    0.0000331927    0   0.0000335389    0   0   0   0   0.0000189167    0.0000908934&#xA;ANAEROFRUCAT-PWY: homolactic fermentation   0.00682776  0.00681683  0.00675732  0.00495716  0.00691969  0.0053589   0.00514634  0.00779249  0.00426403&#xA;ANAGLYCOLYSIS-PWY: glycolysis III (from glucose)    0.00866693  0.0100437   0.00829714  0.00524566  0.00960262  0.0075093   0.00716347  0.00916205  0.00866289&#xA;ARG+POLYAMINE-SYN: superpathway of arginine and polyamine biosynthesis  0.00106395  0.000544088 0.000774645 0.00370237  0.00102368  0.00133848  0.00148691  0.00114062  0.00188739&#xA;ARGDEG-PWY: superpathway of L-arginine, putrescine, and 4-aminobutanoate degradation    0   0   0   0.00164625  0.0000278133    0   0   0   0&#xA;ARGININE-SYN4-PWY: L-ornithine de novo biosynthesis 0.00166327  0.00383993  0.000105528 0.00230283  0.00404469  0.00115975  0.000460711 0.00101816  0.00367102&#xA;ARGORNPROST-PWY: arginine, ornithine and proline interconversion    0.000132331 0.000244069 0.0000796126    0.000479006 0.000237704 0.000290783 0   0.00066458  0.0000581229&#xA;ARGSYN-PWY: L-arginine biosynthesis I (via L-ornithine) 0.0124159   0.00955758  0.0118644   0.00533077  0.00774959  0.0098352   0.0127232   0.00996349  0.0108839&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, I want to keep features (i.e. pathways) which are present with abundance &amp;gt;0.0001 and present in at least 10% of samples. What bash command can I use in order to do that?&lt;/p&gt;&#xA;" OwnerUserId="9840" LastEditorUserId="650" LastEditDate="2020-11-10T20:51:09.457" LastActivityDate="2020-11-22T19:22:54.103" Title="How to remove low abundance and less prevalent data from my dataset?" Tags="|bash|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14768" PostTypeId="2" ParentId="14751" CreationDate="2020-11-10T18:33:43.057" Score="1" Body="&lt;p&gt;Clearly the additional references do put Metapath2vec into good use. Its a many to many ML relationship. Most catagorization/classification particularly in deep learning is many  to one. So thats the good thing about the technique you  are using, you've 25 catagories, which is quite large for machine learning(ML) and large for deep learning. The technique appears to be half way between supervised and unsupervised learning. The random walk establishes a zone (circle) of catagorisation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;NLP Background&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The situation where it would work in NLP (natural language processing) is for example a spell checker, where you have one correct word and loads of possible typos, some of which you not observed in training. So what will happen is the random walk will pick up the local vacinity around the 'correct spelling' by the vectorization (converting a word into numbers) of the incorrect spelling will land it close to 'true spelling' in a 2D vectorization plot. The training following the random walk says  'yep you belong with me'.&lt;/p&gt;&#xA;&lt;p&gt;The papers given talk about being able to identify the context of a word within a sentence which is a very NLP thing, one vectorisation being 'bags of words'. So here the sentence represents one of many categories of information, i.e. each is the central node within the graph, the vectorisation of the sentence places of meaning closer to one 'hub' than another and again the local vacinity is identified by the random walk. In training this 'random walk vacinity' is identified, so it will algorithmically recreate the 'random walk locality' so that sentences that have not been trained can have their meaning identified.&lt;/p&gt;&#xA;&lt;p&gt;The central theory of deep learning, which not quite true of other ML, is any algorithm can be recreated through training. So like it will sort of recreate a 'random walk zone'&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Difference with drug discovery&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The papers you have map drugs to genes, but it means you have to have a &amp;quot;true drug to gene&amp;quot; relationship. The problem I see here is what is 'true' for a drug-protein interaction? With a spell-checker there is one 'true spelling', with a sentence there is pretty much one 'type of meaning', like a movie was great, average, terrible based on a ratings score and stuff like gender classification. If its not 'true' the random walk will get it wrong. So its a big assumption. If you have a circle and pick a point on the edge of the circle then try and draw a circle around the original circle, you can't unless the circle is so large it risks including stuff that is not part of the original circle. The problem the papers is how do they know what is the 'centre' of the circle? Its an a priori assumption and you need really good biological rationale.&lt;/p&gt;&#xA;&lt;p&gt;With your stuff I agree its easier because adverse drug reaction, some drugs could be fatal or toxic (traditional cancer drugs or parasite drugs), so you'd pick the worst case adverse reaction. That then centres the definition around which everything else is relative. If however your 'true catagory' turned out to be an outlier its not going to work, because the random walk only works if its starts bang in the centre of the 'catagory'. Random walks form spheres, in case you are not aware. I can see this working as an advanced 'unsupervised learning' technique.&lt;/p&gt;&#xA;&lt;p&gt;One very real advantage is if a drug doesn't have an adverse reaction it MIGHT (I'm guessing) not give a result, in which case you are good to go - that a good drug. If that is what happened that would be amazing because its not how ML, or particularly DL works. Traditional ML/DL will force a classification even if it doesn't fit. There are methods to accept some sort of &amp;quot;half-classification&amp;quot; but I don't know how to do it. This is one of the reaons you need lots of things to train.&lt;/p&gt;&#xA;&lt;p&gt;I am assuming that vectorisation of your drug is fine because there is a huge history of this in drug-discovery going right back to the origins of the field.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;So yeah I get the idea, I see the flaws and the advantages. The final issue is that most ML requires pretty large data sets and DL require huge data sets and when you've got 25 catagories. For DL approaches the data set size might need to be huge unless you can do 'transfer learning' (backing your training onto a pre-trained model). If your method resulted in a zero classification i.e. &amp;quot;doesn't fit anything I've got&amp;quot; - that would be cool and that is the key advantage I can see in the 'random walk' approach which is not available using traditional methods.&lt;/p&gt;&#xA;&lt;p&gt;As long as you are aware, you do not have to use this algorithm to get a classification. This final point what appears missing in the papers is using a 'traditional' method by comparison. The ML before deep learning, stuff like random forests, routinely takes loads of catagories. If you are using a non-deep learning ML you would traditional use several different approaches to justify your result.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;My answer is a bit rambly but I think you'll get the gist. Peronally I would complement with a standard ML, but all methods have their flaws. Deep learning (DL) has a big history in drug discovery (alot of work was done via Tensorflow), but I'm not sure you've data set size for this.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-11-11T08:49:13.787" LastActivityDate="2020-11-11T08:49:13.787" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14771" PostTypeId="2" ParentId="14767" CreationDate="2020-11-10T22:07:09.523" Score="2" Body="&lt;p&gt;Assuming you have this stored in a matrix called &lt;code&gt;m&lt;/code&gt;, where the pathway labels are row names:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;IDX = apply(m, 1, function(x) sum(x &amp;gt; 0.0001)/length(x) &amp;gt;= 0.1)&#xA;m_subset = m[which(IDX),]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-11-10T22:07:09.523" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14772" PostTypeId="1" AcceptedAnswerId="14773" CreationDate="2020-11-10T22:22:17.403" Score="1" ViewCount="642" Body="&lt;p&gt;I am trying to follow the tutorial from from the Biopython website &lt;a href=&quot;http://biopython.org/DIST/docs/tutorial/Tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;  and I am right at the beginning (2.4.1), where I am trying a &lt;em&gt;Simple FASTA parsing example&lt;/em&gt; which is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;for seq_record in SeqIO.parse(&amp;quot;ls_orchid.fasta&amp;quot;, &amp;quot;fasta&amp;quot;):&#xA;    print(seq_record.id)&#xA;    print(repr(seq_record.seq))&#xA;    print(len(seq_record))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For an unknown reason, I get the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;---------------------------------------------------------------------------&#xA;FileNotFoundError                         Traceback (most recent call last)&#xA;&amp;lt;ipython-input-19-c64128d7dcfd&amp;gt; in &amp;lt;module&amp;gt;&#xA;      1 from Bio import SeqIO&#xA;----&amp;gt; 2 for seq_record in SeqIO.parse(&amp;quot;ls_orchid.fasta&amp;quot;, &amp;quot;fasta&amp;quot;):&#xA;      3     print(seq_record.id)&#xA;      4     print(repr(seq_record.seq))&#xA;      5     print(len(seq_record))&#xA;&#xA;~\Anaconda3\lib\site-packages\Bio\SeqIO\__init__.py in parse(handle, format, alphabet)&#xA;    605     iterator_generator = _FormatToIterator.get(format)&#xA;    606     if iterator_generator:&#xA;--&amp;gt; 607         return iterator_generator(handle)&#xA;    608     if format in AlignIO._FormatToIterator:&#xA;    609         # Use Bio.AlignIO to read in the alignments&#xA;&#xA;~\Anaconda3\lib\site-packages\Bio\SeqIO\FastaIO.py in __init__(self, source, alphabet, title2ids)&#xA;    181             raise ValueError(&amp;quot;The alphabet argument is no longer supported&amp;quot;)&#xA;    182         self.title2ids = title2ids&#xA;--&amp;gt; 183         super().__init__(source, mode=&amp;quot;t&amp;quot;, fmt=&amp;quot;Fasta&amp;quot;)&#xA;    184 &#xA;    185     def parse(self, handle):&#xA;&#xA;~\Anaconda3\lib\site-packages\Bio\SeqIO\Interfaces.py in __init__(self, source, alphabet, mode, fmt)&#xA;     45             raise ValueError(&amp;quot;The alphabet argument is no longer supported&amp;quot;)&#xA;     46         try:&#xA;---&amp;gt; 47             self.stream = open(source, &amp;quot;r&amp;quot; + mode)&#xA;     48             self.should_close_stream = True&#xA;     49         except TypeError:  # not a path, assume we received a stream&#xA;&#xA;FileNotFoundError: [Errno 2] No such file or directory: 'ls_orchid.fasta'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am using a Jupyter Notebook.&lt;/p&gt;&#xA;" OwnerUserId="6948" LastEditorUserId="3848" LastEditDate="2023-07-01T01:38:36.870" LastActivityDate="2023-07-01T01:38:36.870" Title="Why SeqIO.parse method isn't working?" Tags="|python|phylogenetics|phylogeny|seqio|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14773" PostTypeId="2" ParentId="14772" CreationDate="2020-11-11T00:08:22.020" Score="2" Body="&lt;p&gt;Personally I think its late and we're all tired.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;FileNotFoundError: [Errno 2] No such file or directory: 'ls_orchid.fasta'&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The traceback is pointing to line 2, the SeqIO as you point out.&lt;/p&gt;&#xA;&lt;p&gt;In my opinion, the script should be in the same directory as the fasta file. You know how to identify the path in bash (&lt;code&gt;pwd&lt;/code&gt; or &lt;code&gt;echo $PWD&lt;/code&gt;), although you must be using Windows (Anaconda has \ rather than /). Alternatively the file names has a typo. You know the&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for seq_record in SeqIO.parse(&amp;quot;~\path\ls_orchid.fasta&amp;quot;, &amp;quot;fasta&amp;quot;): # where path is the dir(s) leading to ls_orchid.fasta, but obviously use / if its Linux.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would assume you can alternatively dump the .ipynb file in the location where the fasta file is.&lt;/p&gt;&#xA;&lt;p&gt;As personal choice I prefer a formal IDE such as 'Visual Studio Code' or &amp;quot;PyCharm&amp;quot;. In my personal opinion, I am aware some will disagree, Jupyter notebook good is for running code on a remote commercial server for data science work. However, the traceback - which you clearly have activated - looks good. I can see that you're Anaconda3 installation looks good, you must be using Windows (which I've never worked with) and you're running Jupiter from Anaconda3 ... so cool.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-11-11T00:18:05.663" LastActivityDate="2020-11-11T00:18:05.663" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14775" PostTypeId="2" ParentId="14765" CreationDate="2020-11-11T08:14:21.997" Score="5" Body="&lt;p&gt;This could be organism-specific. We don't have a lot of info so far, so I would check a few more things:&lt;/p&gt;&#xA;&lt;p&gt;Run something like &lt;a href=&quot;https://github.com/vezzi/FRC_align&quot; rel=&quot;nofollow noreferrer&quot;&gt;FRC_align&lt;/a&gt;. Check if there's a clear signal between regions flagged as suspicious by it and your coverage graph.&lt;/p&gt;&#xA;&lt;p&gt;Is it a eukaryote? Plant? Check where mitchondria and chloroplasts are on the plot. They will have different GC/coverage signals than the rest of your assembly, which is fine, expected even.&lt;/p&gt;&#xA;&lt;p&gt;Worried about contamination? Run &lt;a href=&quot;https://ccb.jhu.edu/software/kraken2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;kraken2&lt;/a&gt; on your raw data and see what you get there.&lt;/p&gt;&#xA;&lt;p&gt;Run &lt;a href=&quot;http://www.repeatmasker.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RepeatMasker&lt;/a&gt; and check if repetive regions contribute to the effect you see.&lt;/p&gt;&#xA;&lt;p&gt;Finally, unless your specimen is low heterozygosity, haploid, or inbred, you probably have some effect from ploidy. There are numerous ways to check that. A self-self dot plot with &lt;a href=&quot;http://mummer.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MUMmer&lt;/a&gt; and looking coverage histograms are a good start. The idea is that you should see two peaks (let's assume diploid). One for the split haplotigs, and one for collapsed.&lt;/p&gt;&#xA;&lt;p&gt;I checked the GC distribution of 45 genomes in your order. It's kind of all over the place, but there are some bimodal ones:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/APLhU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/APLhU.png&quot; alt=&quot;GC distributions&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What does GC vs Length look like? Here's the one from the 45 genomes mentioned above:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Yme7I.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Yme7I.png&quot; alt=&quot;GC vs Length&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9693" LastEditorUserId="9693" LastEditDate="2020-11-11T18:47:37.073" LastActivityDate="2020-11-11T18:47:37.073" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14776" PostTypeId="1" AcceptedAnswerId="14778" CreationDate="2020-11-11T13:39:40.810" Score="1" ViewCount="884" Body="&lt;p&gt;I'm attempting to demux a nextseq run which I've successfully done several times in the past on other runs. I uploaded the run from my PC to the server (Ubuntu 18.04.4 LTS) without issue. Then I ran:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;```bcl2fastq --runfolder-dir /data/201109_NB551126_0128_AH5G5KBGXF --sample-sheet ./SampleSheet.csv  --no-lane-splitting --barcode-mismatches 0```&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From which I got the error:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;'/data/201109_NB551126_0128_AH5G5KBGXF/RunInfo.xml' file for&#xA;reading```&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However, the RunInfo.xml file is present and readable:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ ls -l&#xA;total 96&#xA;drwxrwxrwx 10 user user 90112 Nov 11 08:08 201109_NB551126_0128_AH5G5KBGXF&#xA;-rwxrwxr-x  1 user user 1929 Nov 11 07:37 SampleSheet.csv&#xA;$ &#xA;$ ls -l 201109_NB551126_0128_AH5G5KBGXF/RunInfo.xml&#xA;-rwxrwxr-x 1 user user 28510 Nov 11 08:08 201109_NB551126_0128_AH5G5KBGXF/RunInfo.xml&#xA;$ &#xA;$ head 201109_NB551126_0128_AH5G5KBGXF/RunInfo.xml&#xA;&amp;lt;?xml version=&amp;quot;1.0&amp;quot;?&amp;gt;&#xA;&amp;lt;RunInfo xmlns:xsd=&amp;quot;http://www.w3.org/2001/XMLSchema&amp;quot; xmlns:xsi=&amp;quot;http://www.w3.org/2001/XMLSchema-instance&amp;quot; Version=&amp;quot;4&amp;quot;&amp;gt;&#xA;  &amp;lt;Run Id=&amp;quot;201109_NB551126_0128_AH5G5KBGXF&amp;quot; Number=&amp;quot;128&amp;quot;&amp;gt;&#xA;    &amp;lt;Flowcell&amp;gt;H5G5KBGXF&amp;lt;/Flowcell&amp;gt;&#xA;    &amp;lt;Instrument&amp;gt;NB551126&amp;lt;/Instrument&amp;gt;&#xA;    &amp;lt;Date&amp;gt;201109&amp;lt;/Date&amp;gt;&#xA;    &amp;lt;Reads&amp;gt;&#xA;      &amp;lt;Read Number=&amp;quot;1&amp;quot; NumCycles=&amp;quot;75&amp;quot; IsIndexedRead=&amp;quot;N&amp;quot; /&amp;gt;&#xA;      &amp;lt;Read Number=&amp;quot;2&amp;quot; NumCycles=&amp;quot;6&amp;quot; IsIndexedRead=&amp;quot;Y&amp;quot; /&amp;gt;&#xA;    &amp;lt;/Reads&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Has anyone had this issue or know of a fix? Thanks!&lt;/p&gt;&#xA;&lt;p&gt;Update: contents of run folder&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;total 2452&#xA;-rwxrwxr-x 1 stewart stewart   977 Nov 11 07:59 '2020-11-09__14_05_58_94F12309 GetBestZ Camera 1 Focus Red top surface.snr.csv'&#xA;....&#xA;-rwxrwxr-x 1 stewart stewart   244 Nov 11 07:59 '2020-11-09__14_08_54_FE05AB9F SafeParabola called by CalculateSignalStd.snr.csv'&#xA;-rwxrwxr-x 1 stewart stewart   253 Nov 11 07:59 '2020-11-09__14_08_54_FE0620BC SafeParabola called by FindPeakUsingParabolaFit.snr.csv'&#xA;....&#xA;drwxrwxrwx 2 stewart stewart  4096 Nov 11 07:59  Config&#xA;drwxrwxrwx 3 stewart stewart  4096 Nov 11 07:59  Data&#xA;drwxrwxrwx 3 stewart stewart  4096 Nov 11 08:07  Images&#xA;drwxrwxrwx 2 stewart stewart 12288 Nov 11 08:07  InstrumentAnalyticsLogs&#xA;drwxrwxrwx 2 stewart stewart  4096 Nov 11 08:08  InterOp&#xA;drwxrwxrwx 2 stewart stewart  4096 Nov 11 08:08  Logs&#xA;drwxrwxrwx 2 stewart stewart  4096 Nov 11 08:08  Recipe&#xA;-rwxrwxr-x 1 stewart stewart    48 Nov 11 08:08  RTAComplete.txt&#xA;-rwxrwxr-x 1 stewart stewart  6200 Nov 11 08:08  RTAConfiguration.xml&#xA;drwxrwxrwx 2 stewart stewart  4096 Nov 11 08:08  RTALogs&#xA;-rwxrwxr-x 1 stewart stewart    36 Nov 11 08:08  RTARead1Complete.txt&#xA;-rwxrwxr-x 1 stewart stewart    36 Nov 11 08:08  RTARead2Complete.txt&#xA;-rwxrwxr-x 1 stewart stewart   925 Nov 11 08:08  RunCompletionStatus.xml&#xA;-rwxrwxr-x 1 stewart stewart 28510 Nov 11 08:08  RunInfo.xml&#xA;-rwxrwxr-x 1 stewart stewart 26802 Nov 11 08:08  RunParameters.xml&#xA;-rwxrwxr-x 1 stewart stewart  1929 Nov 11 08:08  SampleSheet.csv```&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2391" LastEditorUserId="3848" LastEditDate="2020-11-11T14:14:13.970" LastActivityDate="2020-11-11T14:14:13.970" Title="Error processing run file with bcl2fastq - Unable to open '.../RunInfo.xml' file for reading" Tags="|illumina|demultiplexing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14777" PostTypeId="1" CreationDate="2020-11-11T13:45:44.100" Score="0" ViewCount="37" Body="&lt;p&gt;I would like to ask if someone uses two different aligners to produce count matrices and then run the same alogrithm for DEG analysis, would it make sense to find the intersection of the DEGs in order to conclude to the &amp;quot;real&amp;quot; DEGS?&lt;/p&gt;&#xA;" OwnerUserId="8713" LastActivityDate="2020-11-12T07:35:17.437" Title="two aligners combined results" Tags="|rna-seq|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14778" PostTypeId="2" ParentId="14776" CreationDate="2020-11-11T14:07:59.013" Score="0" Body="&lt;p&gt;I repro the error.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ls /ml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;ls: cannot access '/ml': No such file or directory&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If run&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcl2fastq --runfolder-dir ./data/201109_NB551126_0128_AH5G5KBGXF --sample-sheet ./SampleSheet.csv  --no-lane-splitting --barcode-mismatches 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I suspect it will work.&lt;/p&gt;&#xA;&lt;p&gt;At present you are attempting to call a directory &lt;code&gt;/data&lt;/code&gt;, which doesn't exit. If you type &lt;code&gt;ls /etc&lt;/code&gt; that will work. The difference between &lt;code&gt;./data&lt;/code&gt; and &lt;code&gt;/data&lt;/code&gt; in a file structure is alot, but on a command line it appears minimal.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Wait ... if that doesn't work .. because I suspect the calling directory is &lt;code&gt;data&lt;/code&gt; then the following will,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcl2fastq --runfolder-dir ./201109_NB551126_0128_AH5G5KBGXF --sample-sheet ./SampleSheet.csv  --no-lane-splitting --barcode-mismatches 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;OR&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcl2fastq --runfolder-dir ./data/201109_NB551126_0128_AH5G5KBGXF --sample-sheet ./data/SampleSheet.csv  --no-lane-splitting --barcode-mismatches 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It depends on the path where you are making the call. You could sort this out using &lt;code&gt;$PWD&lt;/code&gt; or just &lt;code&gt;echo $PWD&lt;/code&gt; then copy and paste the path name.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-11-11T14:07:59.013" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14780" PostTypeId="1" CreationDate="2020-11-11T22:37:08.747" Score="0" ViewCount="137" Body="&lt;p&gt;I installed cmapPy but I don't know how to use it to read gctx files. I read the documentation. If anyone knows how to do it with cmapPy or any faster method your help would be greatly appreciated. I don't know a lot about this stuff so sorry for the nonspecificity of the question.&lt;/p&gt;&#xA;" OwnerUserId="9850" LastActivityDate="2020-11-11T22:37:08.747" Title="Opening gctx files" Tags="|python|gene-expression|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14781" PostTypeId="1" AcceptedAnswerId="14787" CreationDate="2020-11-12T02:29:02.943" Score="0" ViewCount="776" Body="&lt;p&gt;I am having trouble with some underlying questions about my project.&lt;/p&gt;&#xA;&lt;p&gt;I have ran a Trinity tool to create a trinity.fasta file.&lt;/p&gt;&#xA;&lt;p&gt;To determine some underlying questions, I used the utility asm_stats to gather some basic statistics.&lt;/p&gt;&#xA;&lt;p&gt;I am just having some issues of calculating data.&lt;/p&gt;&#xA;&lt;p&gt;DATA:&lt;/p&gt;&#xA;&lt;p&gt;Main genome scaffold N/L50: 7464/1.032 KB&lt;br /&gt;&#xA;Main genome contig N/L50: 7464/1.032 KB&lt;br /&gt;&#xA;Main genome scaffold N/L90: 14314/106&lt;br /&gt;&#xA;Main genome contig N/L90: 14314/106&lt;/p&gt;&#xA;&lt;p&gt;Number of contigs (scaffold length):&lt;/p&gt;&#xA;&lt;p&gt;25,534 (ALL)&lt;br /&gt;&#xA;25,534 (100)&lt;br /&gt;&#xA;21,609 (250)&lt;br /&gt;&#xA;10,818 (500)&lt;br /&gt;&#xA;907 (1 KB)&lt;br /&gt;&#xA;38 (2.5KB)&lt;br /&gt;&#xA;5 (5 KB)&lt;br /&gt;&#xA;2 (10KB)&lt;/p&gt;&#xA;&lt;p&gt;With the data attached above, would my N50 and N90 length in &lt;strong&gt;BASE PAIRS&lt;/strong&gt; so excluding the kb, be 7464 and 14314, respectively?&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Would the number of contigs produced by this assembly&lt;/em&gt; be 7, since there are 7 different contigs produced or would it be 25,534?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="9708" LastEditorUserId="73" LastEditDate="2020-11-13T06:59:45.520" LastActivityDate="2020-11-13T06:59:45.520" Title="How to find the number of contigs produced? N50 length in base pairs? N90 length in base pairs?" Tags="|rna-seq|fasta|fastq|file-formats|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14783" PostTypeId="2" ParentId="14777" CreationDate="2020-11-12T07:35:17.437" Score="1" Body="&lt;p&gt;This sort of procedure is done in tool papers to benchmark results compared to different aligners/mappers. See the salmon or kallisto papers for examples. Using the intersect of DE results from different mappers will end up yielding somewhat conservative results, not unlike using different differential expression packages and taking the intersect of their outputs. In practice you could just take a more conservative p-value cut-off and save the hassle of using multiple tools, the results would end up being similar.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-11-12T07:35:17.437" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14784" PostTypeId="1" AcceptedAnswerId="14826" CreationDate="2020-11-12T16:44:54.600" Score="0" ViewCount="43" Body="&lt;p&gt;I got a FASTA file from NCBI RefSeq.&#xA;These proteins are annotated with IDs like this : protein_id=XP_019970915.1.&#xA;What type of ID is this? I would like to have them converted in TrEMBL IDs in order to do, later, a GO term enrichment but I have to know where I am starting from.&#xA;Thank you in advance&lt;/p&gt;&#xA;" OwnerUserId="9142" LastActivityDate="2020-11-15T22:59:26.480" Title="What type of Protein ID is this?" Tags="|go-enrichment|identifiers|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14785" PostTypeId="2" ParentId="3026" CreationDate="2020-11-12T17:15:43.207" Score="3" Body="&lt;h2&gt;CATH&lt;/h2&gt;&#xA;&lt;p&gt;The CATH database classifies protein by fold: &lt;a href=&quot;https://www.cathdb.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cathdb.info/&lt;/a&gt;&#xA;So the value from that is probably the most useful for you.&lt;/p&gt;&#xA;&lt;h2&gt;Crystallographic conditions&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;pH&lt;/li&gt;&#xA;&lt;li&gt;Salinity&lt;/li&gt;&#xA;&lt;li&gt;Dissolved Sugars&lt;/li&gt;&#xA;&lt;li&gt;Reducing Agent Concentration&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Your crystallising conditions do not mean much. They are a solution that is close to precipitating your protein but slowly enough for it to crystallise. A protein can crystallise under multiple different conditions and a crystal may be chosen simply because it is a lower index well in the Hampton screening conditions...&#xA;These mean very very little. And unless you are planning on doing ML to determine the best screening condition these should be avoided.&lt;/p&gt;&#xA;&lt;h2&gt;Data from the PDB&lt;/h2&gt;&#xA;&lt;p&gt;You have probably noticed that the only stats PDB/PDBe give are crystallographic, like space group, unit cell dimensions, resolution, completeness in shells, Rfree, Rmerge etc. So nothing there is of much use... except for bound ligand. Often this is the cofactor, but often it's a cofactor analog or a product (as opposed to the substrate). But that information can come from Uniprot etc.&lt;/p&gt;&#xA;&lt;h2&gt;pH&lt;/h2&gt;&#xA;&lt;p&gt;A solution has a pH, a protein has a pI. At a pH equal to the pI it will be neutral —and less insoluble. ProtParam can calculate the pI from a linear sequence. However, surface charge totally shows different spots. APBS can map the solution to the Poisson-Boltzmann equation to your protein. There is a PyMOL wrapper, a Pyrosetta wrapper and more. But you still would need to some stats and 3D operations to convert it into a small vector of values —rototranslations and projects are not for the faint of heart or in a rush.&lt;/p&gt;&#xA;&lt;h2&gt;Melting temperature&lt;/h2&gt;&#xA;&lt;p&gt;BRENDA has lots of enzymatic data on enzymes including melting temperature or kinetics at different temperatures (which follow the Eyring equation (TST), until MMRT kicks in) which could be useful.&lt;/p&gt;&#xA;&lt;p&gt;For a given protein you can calculate the &lt;em&gt;relative&lt;/em&gt; folding ∆∆G (eg. with Rosetta Score), which is a proxy to melting temperature in a buffer it is happy in.&lt;/p&gt;&#xA;&lt;p&gt;Despite paining me to say... some fanciful Poisson-Boltzmann map derived vector and ∆∆G are probably all you could get from analysing structures. Whereas the rest are from CATH classification and other databases that are not structure based.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-11-12T17:15:43.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14786" PostTypeId="1" CreationDate="2020-11-12T19:25:48.207" Score="1" ViewCount="1180" Body="&lt;p&gt;I did run kraken2 and I get two kinds of output&#xA;How do I generate OTU table from these outputs for phyloseq usage?&lt;/p&gt;&#xA;" OwnerUserId="9854" LastActivityDate="2022-12-02T23:07:26.003" Title="kraken2 to OTU table creation" Tags="|genome|microbial-genomics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14787" PostTypeId="2" ParentId="14781" CreationDate="2020-11-12T20:09:30.733" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;With the data attached above, would my N50 and N90 length in BASE PAIRS so excluding the kb, be 7464 and 14314, respectively?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Looks like this is inverse so your N50 is 1032bp and your N90 is 106bp. N90 cannot be longer than N50 by definition. You L50 and L90 are 7464 and 14314&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Would the number of contigs produced by this assembly be 7, since there are 7 different contigs produced or would it be 25,534?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;25534&lt;/p&gt;&#xA;&lt;p&gt;Now for the real answer. &lt;strong&gt;Do not calculate or report N50 on transcript assemblies. It is more than useless.&lt;/strong&gt; Transcriptomes are - by nature - dynamic in length. Longer is not better in this regard. Are your short transcripts lower quality assemblies, probably not, but N50 treats them as such. Please read the &lt;a href=&quot;https://github.com/trinityrnaseq/trinityrnaseq/wiki/Transcriptome-Assembly-Quality-Assessment&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trinity wiki&lt;/a&gt; for other, better ways to QC your transcript assembly.&lt;/p&gt;&#xA;" OwnerUserId="9693" LastActivityDate="2020-11-12T20:09:30.733" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="14788" PostTypeId="2" ParentId="14786" CreationDate="2020-11-12T20:41:04.873" Score="1" Body="&lt;p&gt;You need to run &lt;a href=&quot;https://github.com/jenniferlu717/Bracken&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bracken&lt;/a&gt; to the Kraken2 report output to estimate abundance.&lt;/p&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2020-11-12T20:41:04.873" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14789" PostTypeId="2" ParentId="14765" CreationDate="2020-11-12T20:58:35.833" Score="0" Body="&lt;p&gt;That is indeed puzzling. My best guess would be contamination, even if they carry some of the BUSCOs. I run all de-novo genomes through &lt;a href=&quot;https://github.com/DRL/blobtools&quot; rel=&quot;nofollow noreferrer&quot;&gt;blobtools&lt;/a&gt;, which creates a similar plot to the one you made manually, but also adds taxonomical annotations. It can look like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wDDMW.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wDDMW.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Also, what is the proportion of the genome that is low-coverage? Does it have a significant span? I almost always see a bunch of tiny contigs with low coverage in de novo assemblies, but they usually represent a very small fraction of the genome.&lt;/p&gt;&#xA;&lt;p&gt;-- edit --&lt;/p&gt;&#xA;&lt;p&gt;One more thought, are you sure you don't have a tetraploid species? The two coverage peaks kind of correspond to 8 and 32x right? One is 1/4 of the other. We developed some tools to investigate this &lt;a href=&quot;https://www.nature.com/articles/s41467-020-14998-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;kind of problems in data&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;-- edit 2 --&lt;/p&gt;&#xA;&lt;p&gt;Just a few more comments. Coverage differences can have various reasons, but all the sequencing biases would cause a skew of the distribution, hence the apparent bimodality of your coverage (more apparent from the two 2d plots) suggests two distinct sources. In general, could be due to various ploidy levels or contamination either by bacteria, endosymbionts or even other species getting into DNA isolation. Now that polyploidy is ruled out, it boils down to contamination, and to figure out what contamination it is, I recommend the tool I liked above.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2020-11-15T18:39:00.427" LastActivityDate="2020-11-15T18:39:00.427" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="14790" PostTypeId="1" CreationDate="2020-11-12T23:00:21.080" Score="0" ViewCount="185" Body="&lt;p&gt;Here is the input file for nab:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;molecule m;&#xA;m = arna( &amp;quot;gcuucuucuucuucgc&amp;quot; );&#xA;putpdb( &amp;quot;lr16.pdb&amp;quot;, m, &amp;quot;-wwpdb -nocid -tr&amp;quot;);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am trying to run nab to create the pdb file with this command :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$AMBERHOME/bin/nab lr16.nab\./a.out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But it comes out saying :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gcc: error: lr16.nab./a.out: No such file or directory&#xA;cc failed!&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have loaded the appropriate modules , please could someone help&lt;/p&gt;&#xA;" OwnerDisplayName="user9855" LastEditorUserId="650" LastEditDate="2022-08-15T15:06:06.140" LastActivityDate="2022-08-15T15:06:06.140" Title="I am trying to run nab to create the pdb file but it isn`t working (AMBER)" Tags="|rna-seq|pdb|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14791" PostTypeId="2" ParentId="14790" CreationDate="2020-11-13T00:27:52.070" Score="1" Body="&lt;p&gt;I have no idea about &lt;code&gt;nab&lt;/code&gt; ... however&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;“$AMBERHOME/bin/nab lr14.nab./a.out”&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Looking at your error &lt;code&gt;lr14.nab./a.out:&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;It suggests you didn't put a space character between &lt;code&gt;lr16.nab&lt;/code&gt; and &lt;code&gt;a.out&lt;/code&gt;. If you run&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$AMBERHOME/bin/nab lr14.nab ./a.out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I suspect it will work because at present it is looking for a file named lr14.nab./a.out which could be interpreted as a directory of &lt;code&gt;lr14.nab.&lt;/code&gt; and a input file of &lt;code&gt;a.out&lt;/code&gt; or the error suggests it is reading &lt;code&gt;/&lt;/code&gt; as part of the file name. &lt;code&gt;a.out&lt;/code&gt; of course doesn't exist its your specified output file name&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;output of &lt;code&gt;ls -l&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-rw-r--r-- 1 phys 89 Oct 6 12:58 lr14.nab – Peter 5 hours ago&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-11-13T09:17:45.717" LastActivityDate="2020-11-13T09:17:45.717" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="14792" PostTypeId="2" ParentId="13549" CreationDate="2020-11-13T00:52:49.137" Score="0" Body="&lt;p&gt;Plot out the PCA of your samples.  If, say, PC1 splits the C's away from the others, and PC1 is more than, say, 50% of the variance, or if the C samples are a lot more spread out than the A and B samples, keep it apart.  Otherwise, do them all together.&lt;/p&gt;&#xA;&lt;p&gt;If the C's aren't too different from A and B, the math will benefit from including them.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-11-13T00:52:49.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14793" PostTypeId="2" ParentId="10678" CreationDate="2020-11-13T05:24:37.073" Score="2" Body="&lt;p&gt;I came across the same problem of Nextera transposase contamination in my shotgun metagenome sequence. I specified the library in trimmomatic and Nextera transposase adapters were successfully removed. Please see the code below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar /opt/software/Trimmomatic/0.39-Java-1.8/trimmomatic-0.39.jar PE -phred33 1004_R1.fastq.gz 1004_R2.fastq.gz ../QC_data/1004_R1.fastq.gz ../QC_data/1004.qcup_R1.fastq.gz ../QC_data/1004_R2.fastq.gz ../QC_data/1004.qcup_R2.fastq.gz ILLUMINACLIP:/opt/software/Trimmomatic/0.39-Java-1.8/adapters/NexteraPE-PE.fa:2:30:10 LEADING:3 TRAILING:3 SLIDINGWINDOW:4:15 MINLEN:36&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output showed completed successfully with the following message:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TrimmomaticPE: Started with arguments:&#xA; -phred33 1004_R1.fastq.gz 1004_R2.fastq.gz ../QC_data/1004_R1.fastq.gz ../QC_data/1004.qcup_R1.fastq.gz ../QC_data/1004_R2.fastq.gz ../QC_data/1004.qcup_R2.fastq.gz ILLUMINACLIP:/opt/software/Trimmomatic/0.39-Java-1.8/adapters/NexteraPE-PE.fa:2:30:10 LEADING:3 TRAILING:3 SLIDINGWINDOW:4:15 MINLEN:36&#xA;Using PrefixPair: 'AGATGTGTATAAGAGACAG' and 'AGATGTGTATAAGAGACAG'&#xA;Using Long Clipping Sequence: 'GTCTCGTGGGCTCGGAGATGTGTATAAGAGACAG'&#xA;Using Long Clipping Sequence: 'TCGTCGGCAGCGTCAGATGTGTATAAGAGACAG'&#xA;Using Long Clipping Sequence: 'CTGTCTCTTATACACATCTCCGAGCCCACGAGAC'&#xA;Using Long Clipping Sequence: 'CTGTCTCTTATACACATCTGACGCTGCCGACGA'&#xA;ILLUMINACLIP: Using 1 prefix pairs, 4 forward/reverse sequences, 0 forward only sequences, 0 reverse only sequences&#xA;Input Read Pairs: 18205918 Both Surviving: 10467310 (57.49%) Forward Only Surviving: 7470543 (41.03%) Reverse Only Surviving: 48503 (0.27%) Dropped: 219562 (1.21%)&#xA;TrimmomaticPE: Completed successfully&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the QC report showed the Nextera transposase was successfully removed.&#xA;[![QC report][2]][2]&#xA;&lt;a href=&quot;https://i.stack.imgur.com/a2bqH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/a2bqH.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9856" LastEditorUserId="9856" LastEditDate="2020-11-13T15:35:29.927" LastActivityDate="2020-11-13T15:35:29.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14794" PostTypeId="2" ParentId="8858" CreationDate="2020-11-13T09:29:40.040" Score="0" Body="&lt;p&gt;I think it is quite simple you need to install &lt;code&gt;perlbrew&lt;/code&gt; &lt;a href=&quot;https://perlbrew.pl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You then run the software under perlbrew it should work.&lt;/p&gt;&#xA;&lt;p&gt;At present you are running the system perl5 and that is causing all sorts issues, for example the age of the installation and the fact system operations and your software are using the same version of perl.&lt;/p&gt;&#xA;&lt;p&gt;All the instructions are in the website link.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-11-13T09:29:40.040" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14795" PostTypeId="1" CreationDate="2020-11-13T10:18:58.330" Score="2" ViewCount="153" Body="&lt;p&gt;For processing genomics workflows I make use of widdle and cromwell with an AWS back-end. I have followed all examples on &lt;a href=&quot;https://docs.opendata.aws/genomics-workflows/orchestration/cromwell/cromwell-overview/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the AWS cromwell tutorial&lt;/a&gt;. In my AWS batch console I can see there are 2 job queues (priority-cromwell and default-cromwell) and both queues can submit jobs to 2 compute environments (ondemand-cromwell and spot-cromwell). I have however no idea how I can control which jobs are put in which queue and are executed in which compute environment. I'd like to make optimal use of the spot-instances, how can I do that? Do I have to add something in the runtime block of my widdle task definitions, should I include something in the cromwell configuration file, or are for example only scattered jobs executed on spot instances?&lt;/p&gt;&#xA;" OwnerUserId="9857" LastActivityDate="2021-09-27T18:07:15.680" Title="How does cromwell and AWS batch decide which compute environment (on-demand or spot) to use?" Tags="|workflow-management|aws|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14796" PostTypeId="2" ParentId="5273" CreationDate="2020-11-13T11:31:47.040" Score="1" Body="&lt;p&gt;Maynard's Smith's homoplasy is what you want and it is available in the recombination detection package (RDP) described &lt;a href=&quot;https://academic.oup.com/ve/article/1/1/vev003/2568683&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Its really a Windows package rather than R.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-11-13T11:31:47.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14797" PostTypeId="1" AcceptedAnswerId="14798" CreationDate="2020-11-13T13:25:01.037" Score="1" ViewCount="448" Body="&lt;p&gt;I am trying to extract a specific sequence from a multifasta file, from each sequence in the aligned file.&lt;/p&gt;&#xA;&lt;p&gt;The sequences look like this, and there are 32 sequences within the multiFASTA:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;NC_045512.2_Severe_acute_respiratory_syndrome_coronavirus_2_isolate_Wuhan-Hu-1_complete_genome &#xA;ATT AAA GGT TTA TAC CTT CCC AGG TAA CAA ACC AAC CAA CTT TCG ATC TCT TGT AGA TCT GTT CTC TAA ACG AAC TTT &#xA;AAA ATC TGT GTG GCT GTC ACT CGG CTG CAT GCT TAG TGC ACT CAC GCA GTA TAA TTA ATA ACT AAT TAC TGT CGT TGA &#xA;CAG GAC ACG AGT AAC TCG TCT ATC TTC TGC AGG CTG CTT ACG GTT TCG TCC GTG TTG CAG CCG ATC ATC AGC ACA TCT &#xA;AGG TTT CGT CCG GGT GTG ACC GAA AGG TAA GAT GGA GAG CCT TGT CCC TGG TTT CAA CGA GAA AAC ACA CGT CCA ACT &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Using the following code I can pull the sequence, using coordinates, from a single FASTA file, but how would one iterate this to pull the section (by coordinates) from each sequence within the multiFASTA?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;from Bio.SeqIO.FastaIO import FastaWriter&#xA;import os&#xA;&#xA;os.chdir(&amp;quot;C:/Users/&amp;quot;)&#xA;&#xA;record = SeqIO.read(&amp;quot;Whole genome.fas&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;&#xA;with open(&amp;quot;Spike protein.fasta&amp;quot;, &amp;quot;wt&amp;quot;) as out:&#xA;    writer = FastaWriter(out, wrap = [])&#xA;    writer.write_header()&#xA;    writer.write_record(record[21563:25384])&#xA;    writer.write_footer()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="9672" LastEditorUserId="3848" LastEditDate="2020-12-03T12:11:43.200" LastActivityDate="2020-12-03T12:11:43.200" Title="Extract sequences from MultiFASTA aligned file, by coordinates" Tags="|fasta|python|covid-19|sars-cov-2|seqio|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14798" PostTypeId="2" ParentId="14797" CreationDate="2020-11-13T14:30:40.240" Score="1" Body="&lt;p&gt;I developed an answer as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;from Bio.SeqIO.FastaIO import FastaWriter&#xA;import os&#xA;&#xA;os.chdir(&amp;quot;C:/Users/&amp;quot;)&#xA;&#xA;with open(&amp;quot;Spike protein.fasta&amp;quot;, &amp;quot;wt&amp;quot;) as out:&#xA;    for seq_record in SeqIO.parse(&amp;quot;Whole genome.fas&amp;quot;, &amp;quot;fasta&amp;quot;):&#xA;        out.write(str(seq_record.id) + &amp;quot;\n&amp;quot;)&#xA;        out.write(str(seq_record.seq[21563:25384]) + &amp;quot;\n&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9672" LastActivityDate="2020-11-13T14:30:40.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14799" PostTypeId="1" CreationDate="2020-11-13T16:20:30.607" Score="2" ViewCount="129" Body="&lt;p&gt;I'm trying to read gctx files with &lt;a href=&quot;https://pypi.org/project/cmapPy/&quot; rel=&quot;nofollow noreferrer&quot;&gt;cmapPy&lt;/a&gt;. This is my code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from cmapPy.pandasGEXpress.parse_gct import parse&#xA;from cmapPy.pandasGEXpress.write_gct import write&#xA;parse(&amp;quot;C:/Users/User/Desktop/gctx/CAS9_BASELINE_LEVEL3_BING__n10721x10174.gctx&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(base) C:\Users\User\Desktop\hello&amp;gt;D:/anaconda/python.exe c:/Users/User/Desktop/gctx/readgcxtx.py&#xA;Traceback (most recent call last):&#xA;  File &amp;quot;c:/Users/User/Desktop/gctx/readgcxtx.py&amp;quot;, line 3, in &amp;lt;module&amp;gt;&#xA;    parse(&amp;quot;C:/Users/User/Desktop/gctx/CAS9_BASELINE_LEVEL3_BING__n10721x10174.gctx&amp;quot;)&#xA;  File &amp;quot;D:\anaconda\lib\site-packages\cmapPy\pandasGEXpress\parse_gct.py&amp;quot;, line 136, in parse&#xA;    num_row_metadata, num_col_metadata) = read_version_and_dims(file_path)&#xA;  File &amp;quot;D:\anaconda\lib\site-packages\cmapPy\pandasGEXpress\parse_gct.py&amp;quot;, line 168, in  read_version_and_dims&#xA;    version = f.readline().strip().lstrip(&amp;quot;#&amp;quot;)&#xA;  File &amp;quot;D:\anaconda\lib\encodings\cp1252.py&amp;quot;, line 23, in decode&#xA;    return codecs.charmap_decode(input,self.errors,decoding_table)[0]&#xA;UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 2440: character maps to &amp;lt;undefined&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried changing the path with double slashes or r&amp;quot;...&amp;quot; but it doesn't work. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="9850" LastEditorUserId="3848" LastEditDate="2020-12-17T15:53:31.020" LastActivityDate="2023-04-19T01:47:13.393" Title="cmapPy exception" Tags="|python|anaconda|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14800" PostTypeId="1" CreationDate="2020-11-13T17:23:17.790" Score="0" ViewCount="238" Body="&lt;p&gt;I Have multiple BAM files that are referenced to UCSC genome assembly GRCh37/hg19 that are read in different time frames. Now, I am planning a different studies that require assembling all the data against one assembly. Will it be possible to convert all the BAM files that I have to an upgraded version or convert the assembly to a different new assembly and how?&#xA;I am new in this and not a bioinformatician so, if you have any questions regarding my question to answer you can ask, any type of help is highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="9858" LastActivityDate="2020-11-13T22:12:31.050" Title="Is it possible to convert BAM file from one genome assembly to the other?" Tags="|sam|genome|assembly|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14801" PostTypeId="1" AcceptedAnswerId="14811" CreationDate="2020-11-13T17:28:27.687" Score="2" ViewCount="89" Body="&lt;p&gt;Is there any biological application/case where someone would be interested in the estimated total cost of the alignment between a set of sequences (genes or amino acids) without the aligned sequence itself?&lt;/p&gt;&#xA;&lt;p&gt;Edit: to clarify by cost I mean the cost function such as the &lt;a href=&quot;https://en.m.wikipedia.org/wiki/Edit_distance&quot; rel=&quot;nofollow noreferrer&quot;&gt;edit distance&lt;/a&gt;. I have some ideas for heuristics on the multiple sequences alignment problem (which is intractable) but these would not retrieve the aligned sequence, therefore, I was wondering whether such an algorithm would be useful.&lt;/p&gt;&#xA;" OwnerUserId="9815" LastEditorUserId="73" LastEditDate="2023-07-09T20:05:52.070" LastActivityDate="2023-07-09T20:05:52.070" Title="Multiple Alignment cost application" Tags="|phylogenetics|phylogeny|gene|sequence-analysis|multiple-sequence-alignment|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14802" PostTypeId="2" ParentId="14797" CreationDate="2020-11-13T17:31:15.053" Score="2" Body="&lt;p&gt;You can use &lt;code&gt;SeqIO&lt;/code&gt; for the writing in addition to reading the sequences. This has the advantage that it will always write the correct fasta header, which is not always just the &lt;code&gt;.id&lt;/code&gt; if there is whitespace, e.g. (untested)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;with open(&amp;quot;Spike protein.fasta&amp;quot;, &amp;quot;w&amp;quot;) as out:&#xA;    for seq_record in SeqIO.parse(&amp;quot;Whole genome.fas&amp;quot;, &amp;quot;fasta&amp;quot;):&#xA;        seq_record.seq = seq_record.seq[21563:25384]&#xA;        SeqIO.write(seq_record, out, &amp;quot;fasta&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;BTW don't put spaces in your filenames, it can lead to issues&lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="104" LastEditDate="2020-11-14T17:45:56.350" LastActivityDate="2020-11-14T17:45:56.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14803" PostTypeId="1" CreationDate="2020-11-13T18:48:55.783" Score="3" ViewCount="1290" Body="&lt;p&gt;I'm performing correlation analysis using the corrr package in R, but I'm unable to extract the p values. Does anyone know how to get the p values in corrr package in R?&lt;/p&gt;&#xA;" OwnerUserId="9860" LastActivityDate="2021-08-18T14:26:11.633" Title="Extracting p-values using corrr package in R" Tags="|r|correlation|p-values|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14804" PostTypeId="1" AcceptedAnswerId="14805" CreationDate="2020-11-13T20:17:33.977" Score="0" ViewCount="741" Body="&lt;p&gt;We are working with a lot of samples, more than 50,000 people! We are using WGCNA to get modules and it seems like there are less number of module genes the more people we use. We are following the tutorial guidelines. We set the min modulesize = 30 and deepSplit = 4 (max sensitivity) However, we are getting very small modules with numbers almost 30 for all. We want to know if using this many samples creates an artifact and the variation between the samples may reduce the correlation seen between the genes? If that is the case, do you suggest any way we can fix that? Also do you think z-scaling expression data improves the module detection?&lt;/p&gt;&#xA;&lt;p&gt;Thank you so much.&lt;/p&gt;&#xA;" OwnerUserId="9861" LastActivityDate="2020-11-13T22:10:19.383" Title="Does module size decrease as the number of samples increase in WGCNA?" Tags="|wgcna|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14805" PostTypeId="2" ParentId="14804" CreationDate="2020-11-13T22:10:19.383" Score="1" Body="&lt;p&gt;The biggest difference a large number of samples makes is that you can usually decrease the soft thresholding power. If you use power 6 (or 12 for a signed network), try decreasing it to 3 or even 2 (6 or 4 for signed network). Apart from that, it could also be that the co-expression modules in your data are simply smaller than what people usually find, nothing apriori wrong with that.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2020-11-13T22:10:19.383" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14806" PostTypeId="2" ParentId="14800" CreationDate="2020-11-13T22:12:31.050" Score="1" Body="&lt;p&gt;It is possible, with tools like &lt;a href=&quot;https://crossmap.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CrossMap&lt;/a&gt;, but my recommendation is to re-align. Assemblies can change a lot between versions and your reads will undoubdedly align differently. Re-aligning is cheap, it will take a few days and you're good. Re-doing your analysis because you realize the coordinate translation messed something up will cost you much more time and energy.&lt;/p&gt;&#xA;" OwnerUserId="9693" LastActivityDate="2020-11-13T22:12:31.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14807" PostTypeId="2" ParentId="14803" CreationDate="2020-11-13T23:33:55.393" Score="5" Body="&lt;p&gt;I would suggest looking at some of the &lt;a href=&quot;https://cran.r-project.org/web/packages/corrr/index.html&quot; rel=&quot;noreferrer&quot;&gt;documentation&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As far as I can tell, the intention of the package is to visualize and organize correlation coefficient estimates. It does not seem that the author of the package has any interest in p-value estimation.&lt;/p&gt;&#xA;&lt;p&gt;If you are interested in computing p-values, I would suggest using a different tool, such as the builtin &lt;code&gt;cor.test()&lt;/code&gt; function.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-11-13T23:33:55.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14808" PostTypeId="1" AcceptedAnswerId="14812" CreationDate="2020-11-14T01:16:40.953" Score="0" ViewCount="27" Body="&lt;p&gt;I've downloaded some ChIP-seq data from &lt;a href=&quot;http://www.chip-atlas.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;www.chip-atlas.org&lt;/a&gt; and I want to use this to conduct some enrichment tests with genomic regions from my experiment. The .bed file generated by that website is set up so that each row represents a genomic region from a specific ChIP-seq experiment. I have the data set up as a dataframe in R.&lt;/p&gt;&#xA;&lt;p&gt;Each row has a single column with all of the metadata, including such important information as the target transcription factor, model system, and tissue type.  These are delimited by semi-colons, but the number of pieces of information varies between rows so it's impossible to simply split by the delimiter.&lt;/p&gt;&#xA;&lt;p&gt;The metadata is structured something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;name=CTCF; strain=B6; age=adult; tissue=brain&#xA;name=Mecp2; age=embryo; tissue=5t3 cells; passages=5-10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So within each row, the order and number of metadata terms varies.&#xA;I want to reshape this column into something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;name; strain; age; tissue; passages&#xA;CTCF; B6; adult; brain; NA&#xA;Mecp2; NA; embryo; 5t3 cells; 5-10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I extract and reshape this data in R? It seems like it should be possible, but the normal approaches to splitting columns by delimiters are failing me since the rows all have different numbers of items and order.&lt;/p&gt;&#xA;&lt;p&gt;Here's a slice of the real data which is much more messy than my little example.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;test_df &amp;lt;- data.frame(pos=c(3002983, 3002881, 3010946, 3021775), metadata=c(&amp;quot;ID=SRX661585;Name=Ctcf%20(@%20Brain);Title=GSM1446329:%20CTCF%20brain%20ChIPSeq%3B%20Mus%20musculus%20x%20Mus%20spretus%3B%20ChIP-Seq;Cell%20group=Neural;&amp;lt;br&amp;gt;source_name=Mouse%20adult%20hybrid%20brain;tissue=Adult%20whole%20brain;strain=BL6/spretus;chip%20antibody=CTCF,%20MillIpore%2007-729;&amp;quot;, &amp;quot;ID=SRX2957352;Name=Smc1a%20(@%20AtT-20);Title=GSM2684855:%20SMC1%20ChIPseq%20in%20AtT-20%3B%20Mus%20musculus%3B%20ChIP-Seq;Cell%20group=Neural;&amp;lt;br&amp;gt;source_name=AtT-20%20cells;cell%20line=AtT-20%20cells;cell%20type=pituitary%20corticotroph%20cell;passages=10-20;chip%20antibody=Bethyl%20A300-055A;&amp;quot;, &amp;quot;ID=SRX5287318;Name=Mecp2%20(@%20Olfactory%20Nerve);Title=GSM3577864:%20GSM1827604%20MeCP2%20ChIP%20WT%20rep1%3B%20Mus%20musculus%3B%20ChIP-Seq;Cell%20group=Neural;&amp;lt;br&amp;gt;source_name=Olfactory%20epithelium;strain/background=C57BL/6J;genotype/variation=Wild-type,%20MeCP2(%2B/y);age=8%20weeks;cell%20type=Olfactory%20sensory%20neuron;chip%20antibody=MeCP2%20antibody%20(Diagenode,%20pAb-052-050);&amp;quot;, &amp;quot;ID=SRX5287318;Name=Mecp2%20(@%20Olfactory%20Nerve);Title=GSM3577864:%20GSM1827604%20MeCP2%20ChIP%20WT%20rep1%3B%20Mus%20musculus%3B%20ChIP-Seq;Cell%20group=Neural;&amp;lt;br&amp;gt;source_name=Olfactory%20epithelium;strain/background=C57BL/6J;genotype/variation=Wild-type,%20MeCP2(%2B/y);age=8%20weeks;cell%20type=Olfactory%20sensory%20neuron;chip%20antibody=MeCP2%20antibody%20(Diagenode,%20pAb-052-050);&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thank you for your help!&lt;/p&gt;&#xA;" OwnerUserId="9863" LastActivityDate="2020-11-14T11:12:48.183" Title="Extracting concatenated metadata from a column in R" Tags="|r|chip-seq|data-preprocessing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14809" PostTypeId="2" ParentId="5144" CreationDate="2020-11-14T09:10:12.963" Score="1" Body="&lt;p&gt;40% missing data is huge. Missing data analysis is complicated on the underlying distribution. If the data set is periodic then missing data periodicity is needed. Non-periodic data can be solved using regression, I think &lt;code&gt;SciPy&lt;/code&gt; has an automated method for this, but the volume of missing data makes this approach complex. If you can bring the missing data down to 20% by compartmenatlising the data set that would be the starting point.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-11-14T09:10:12.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14810" PostTypeId="2" ParentId="4308" CreationDate="2020-11-14T09:14:39.327" Score="0" Body="&lt;p&gt;Sure, you would use the negative binomial distribution when the majority of data was around 0. In other words the data is clustered with the standard deviation far greater than the mean. Most general linear modelling (GLM) will incorporate this approach.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-11-14T09:14:39.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14811" PostTypeId="2" ParentId="14801" CreationDate="2020-11-14T10:44:00.493" Score="1" Body="&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In response to comments, it seems that the question is more along the lines of:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;&amp;quot;what are some applications where computing summary statistics associated with a specific MSA result could be useful, even if we can't return the sequences?&amp;quot;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have 2 responses to this:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;in keeping with my original answer, I would say that it would still be useful to obtain some measure of identity, similarity, or pairwise distances between sequences (or any summary statistic about the relationship of the sequences). There are lots of applications for this level of data. I don't know whether this is something that can also be extracted from the computation, I'm a little unclear on what exactly the proposed computation is doing if it's getting a cost but not sequences.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Michael says below (and I agree) that an alignment cost is likely to strongly depend on particularities of the underlying alignment model as well as the data, and things get nasty when you try to account for gaps or weirdness. Nonetheless, it's possible that a cost could be used as a sort of guide heuristic for a separate follow-up alignment procedure, to indicate a portion of parameter space which is likely to yield good results. I don't know what this looks like, but it could be simple as resource provisioning for the computation (e.g. CPUs/RAM/disk). Alternately, maybe the cost result can be a little richer and actually seed some parameters in the next computation.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Again, without more detail I'm not sure what this looks like (and I'm not an alignment expert so my suggestions wouldn't be very good anyways).&lt;/p&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;The simplest application might be to come up with a quick &amp;quot;check&amp;quot; of the sequences to ask whether they are alignable in the first place. Like, as a QC step (did you forget to reverse complement?) maybe. (this assumes that the alignment in question is pretty onerous and you don't want to have to try several times.)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In summary, a lot depends on how you get that cost and what else you &lt;em&gt;could&lt;/em&gt; get in addition to the cost. I wonder if rather than focusing on cost you can output some other richer summary of the alignment (such as e.g. sequence similarities, or gapping stats, etc.).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;ORIGINAL ANSWER&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The short answer is &amp;quot;yes&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Quite frequently we are interested in measuring &amp;quot;identity&amp;quot; or &amp;quot;similarity&amp;quot; of two sequence strings. As one totally arbitrary example, &lt;a href=&quot;https://sites.google.com/site/wiki4metagenomics/pdf/definition/operational-taxonomic-unit-otu&quot; rel=&quot;nofollow noreferrer&quot;&gt;in metagenomics, quite frequently two rDNA amplicons that show 97% identity are considered to be in the same species, 95% same genus, 98% same strain, etc.&lt;/a&gt; So these measures are frequently useful as heuristics.&lt;/p&gt;&#xA;&lt;p&gt;Identity itself is just a length-normalized edit distance across a sequence, after all.&lt;/p&gt;&#xA;&lt;p&gt;But more generally, such measures of identity or similarity can themselves be understood as cost measures, in the sense that e.g. a &lt;a href=&quot;https://courses.cs.washington.edu/courses/cse421/15wi/slides/06dp-align-katie.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;dynamic programming alignment algorithm&lt;/a&gt; is trying to find an optimal set of matches between two sequences, which necessarily will maximize identity. Generally what it is actually maximizing is similarity (according to some cost matrix such as e.g. BLOSUM or Jukes-Cantor or whatever).&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2020-11-15T19:25:11.227" LastActivityDate="2020-11-15T19:25:11.227" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14812" PostTypeId="2" ParentId="14808" CreationDate="2020-11-14T11:12:48.183" Score="1" Body="&lt;p&gt;In general you want to:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Clean your text&lt;/li&gt;&#xA;&lt;li&gt;Split the features by semicolons&lt;/li&gt;&#xA;&lt;li&gt;Split keys and values by '='&lt;/li&gt;&#xA;&lt;li&gt;Create a temporary data frame with keys and value&lt;/li&gt;&#xA;&lt;li&gt;Reshape&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(tidyverse)&#xA;library(urltools)&#xA;library(textclean)&#xA;test_df &amp;lt;- data.frame(pos=c(3002983, 3002881, 3010946, 3021775), metadata=c(&amp;quot;ID=SRX661585;Name=Ctcf%20(@%20Brain);Title=GSM1446329:%20CTCF%20brain%20ChIPSeq%3B%20Mus%20musculus%20x%20Mus%20spretus%3B%20ChIP-Seq;Cell%20group=Neural;&amp;lt;br&amp;gt;source_name=Mouse%20adult%20hybrid%20brain;tissue=Adult%20whole%20brain;strain=BL6/spretus;chip%20antibody=CTCF,%20MillIpore%2007-729;&amp;quot;, &amp;quot;ID=SRX2957352;Name=Smc1a%20(@%20AtT-20);Title=GSM2684855:%20SMC1%20ChIPseq%20in%20AtT-20%3B%20Mus%20musculus%3B%20ChIP-Seq;Cell%20group=Neural;&amp;lt;br&amp;gt;source_name=AtT-20%20cells;cell%20line=AtT-20%20cells;cell%20type=pituitary%20corticotroph%20cell;passages=10-20;chip%20antibody=Bethyl%20A300-055A;&amp;quot;, &amp;quot;ID=SRX5287318;Name=Mecp2%20(@%20Olfactory%20Nerve);Title=GSM3577864:%20GSM1827604%20MeCP2%20ChIP%20WT%20rep1%3B%20Mus%20musculus%3B%20ChIP-Seq;Cell%20group=Neural;&amp;lt;br&amp;gt;source_name=Olfactory%20epithelium;strain/background=C57BL/6J;genotype/variation=Wild-type,%20MeCP2(%2B/y);age=8%20weeks;cell%20type=Olfactory%20sensory%20neuron;chip%20antibody=MeCP2%20antibody%20(Diagenode,%20pAb-052-050);&amp;quot;, &amp;quot;ID=SRX5287318;Name=Mecp2%20(@%20Olfactory%20Nerve);Title=GSM3577864:%20GSM1827604%20MeCP2%20ChIP%20WT%20rep1%3B%20Mus%20musculus%3B%20ChIP-Seq;Cell%20group=Neural;&amp;lt;br&amp;gt;source_name=Olfactory%20epithelium;strain/background=C57BL/6J;genotype/variation=Wild-type,%20MeCP2(%2B/y);age=8%20weeks;cell%20type=Olfactory%20sensory%20neuron;chip%20antibody=MeCP2%20antibody%20(Diagenode,%20pAb-052-050);&amp;quot;))&#xA;&#xA;tbl &amp;lt;- apply(test_df, 1, function(row){&#xA;  # Split by smicolons and clean html tags&#xA;  n &amp;lt;- replace_html(unlist(strsplit(row[2], &amp;quot;;&amp;quot;)))&#xA;  # Decode urlencoded characters, mostly space&#xA;  n &amp;lt;- url_decode(n)&#xA;  # Split into keys and values&#xA;  s &amp;lt;- do.call(rbind, str_split(n, &amp;quot;=&amp;quot;))&#xA;  # Get keys and values into vectors and clean up leading and trailing whitespace&#xA;  v &amp;lt;- s[, 2]&#xA;  v &amp;lt;- str_replace_all(v, &amp;quot;(^ +)|( +&lt;span class=&quot;math-container&quot;&gt;$)&quot;, &quot;&quot;)&#xA;  k &amp;lt;- s[, 1]&#xA;  k &amp;lt;- str_replace_all(k, &quot;(^ +)|( +$&lt;/span&gt;)&amp;quot;, &amp;quot;&amp;quot;)&#xA;  # Create final data frame with original columns (pos), and also columns from&#xA;  # keys and values&#xA;  tibble(pos = row[1], key = k, value = v)&#xA;})&#xA;&#xA;# Concatenate list into tibble&#xA;(tbl &amp;lt;- do.call(rbind, tbl))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This tibble looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# A tibble: 37 x 3&#xA;   pos     key           value                                                               &#xA;   &amp;lt;chr&amp;gt;   &amp;lt;chr&amp;gt;         &amp;lt;chr&amp;gt;                                                               &#xA; 1 3002983 ID            SRX661585                                                           &#xA; 2 3002983 Name          Ctcf (@ Brain)                                                      &#xA; 3 3002983 Title         GSM1446329: CTCF brain ChIPSeq; Mus musculus x Mus spretus; ChIP-Seq&#xA; 4 3002983 Cell group    Neural                                                              &#xA; 5 3002983 source_name   Mouse adult hybrid brain                                            &#xA; 6 3002983 tissue        Adult whole brain                                                   &#xA; 7 3002983 strain        BL6/spretus                                                         &#xA; 8 3002983 chip antibody CTCF, MillIpore 07-729                                              &#xA; 9 3002881 ID            SRX2957352                                                          &#xA;10 3002881 Name          Smc1a (@ AtT-20)                                                    &#xA;# … with 27 more rows&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Using this, you can:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# Pivot wider&#xA;tbl %&amp;gt;% &#xA;  pivot_wider(names_from = key, values_from = value)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which nets you:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# A tibble: 4 x 15&#xA;  pos   ID    Name  Title `Cell group` source_name tissue strain `chip antibody` `cell line` `cell type`&#xA;  &amp;lt;chr&amp;gt; &amp;lt;chr&amp;gt; &amp;lt;chr&amp;gt; &amp;lt;chr&amp;gt; &amp;lt;chr&amp;gt;        &amp;lt;chr&amp;gt;       &amp;lt;chr&amp;gt;  &amp;lt;chr&amp;gt;  &amp;lt;chr&amp;gt;           &amp;lt;chr&amp;gt;       &amp;lt;chr&amp;gt;      &#xA;1 3002… SRX6… Ctcf… GSM1… Neural       Mouse adul… Adult… BL6/s… CTCF, MillIpor… NA          NA         &#xA;2 3002… SRX2… Smc1… GSM2… Neural       AtT-20 cel… NA     NA     Bethyl A300-05… AtT-20 cel… pituitary …&#xA;3 3010… SRX5… Mecp… GSM3… Neural       Olfactory … NA     NA     MeCP2 antibody… NA          Olfactory …&#xA;4 3021… SRX5… Mecp… GSM3… Neural       Olfactory … NA     NA     MeCP2 antibody… NA          Olfactory …&#xA;# … with 4 more variables: passages &amp;lt;chr&amp;gt;, `strain/background` &amp;lt;chr&amp;gt;, `genotype/variation` &amp;lt;chr&amp;gt;, age &amp;lt;chr&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9693" LastActivityDate="2020-11-14T11:12:48.183" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14813" PostTypeId="2" ParentId="14801" CreationDate="2020-11-14T14:41:02.280" Score="2" Body="&lt;p&gt;I'm not a fan personally, but it is context specific. The problem is that it is heading towards optimisation, particularly algorithmic optimisation. If the alignment is the final calculation - that is cool, but the situations are select. It depends how far you want to take it.&lt;/p&gt;&#xA;&lt;p&gt;The rationale of my skepticism is simple - if an alignment was decided on a 'maximization' criteria, be it likelihood or some other metric - the subsequent calculations based on the alignment, such as phylogenetics Jukes Cantor, Kimura 2 and 3-parameter etc ... we enter circular mathematics. The critiera that defined the alignment is the criteria we are using to perform unrelated post hoc analysis. In truth there is a degree of circular mathematics in any algorithm used to produce an alignment, but we overcome this by optical verification precisely avoid this.&lt;/p&gt;&#xA;&lt;p&gt;In this sense the alignment is simply to establish homologous amino acid/nucleotide positions for downstream analysis. Thus optical optimisation, whilst cautioned against in every other area of bioinformatics, is considered useful in alignments because of the risk that a logical error in the alignment is amplified in downstream analysis, potentially leading to an erroneous conclusion.&lt;/p&gt;&#xA;&lt;p&gt;Algorithms are a problem where indels are concerned and aligning across an indel. It is very easy to make errors here and 'optical analysis' would simple introduce a blockwise deletion to avoid the uncertainty, i.e. if you are not sure just chop it out - better to lose data than risk artefacts on an equivocal judgement call.&lt;/p&gt;&#xA;&lt;p&gt;I'm not saying don't try, I am saying don't look to overstate the applicability of the metric.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-11-15T11:12:30.107" LastActivityDate="2020-11-15T11:12:30.107" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14815" PostTypeId="1" CreationDate="2020-11-14T19:25:32.213" Score="2" ViewCount="86" Body="&lt;p&gt;I am starting a project using Python to track, archive, assign and manage bioinformatics bugs. This software will be able to be integrated into project management software or stand alone.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How do you currently track bugs/assign bioinformatics bugs using freely available bioinformatics software/websites?&lt;/li&gt;&#xA;&lt;li&gt;What are features that you like/dislike? What are features that you would like to see in a bioinformatics bug tracker?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I am a Biology student with basic/core Python skills.&lt;/p&gt;&#xA;" OwnerUserId="6454" LastEditorUserId="57" LastEditDate="2021-05-27T10:31:42.257" LastActivityDate="2021-05-27T10:31:48.330" Title="How do you currently track bugs in python using freely available software/websites?" Tags="|python|bugs|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14817" PostTypeId="2" ParentId="14815" CreationDate="2020-11-14T21:20:22.680" Score="2" Body="&lt;p&gt;Use IPython and activate the trace in the Python debugger as follows&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;%pdb 1 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;%pdb on&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to turn this off &lt;code&gt;%pdb 0&lt;/code&gt; or &lt;code&gt;%pdf off&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I do recommend using a formal IDE such as PyCharm or Visual Studio Code. The professional version of PyCharm is highly rated and within that use &lt;strong&gt;breakpoints&lt;/strong&gt; . In Python3.7 there is supposed to be &lt;code&gt;breakpoint()&lt;/code&gt; so you may not need an IDE, i.e. you set the breakpoint in your code using &lt;code&gt;breakpoint()&lt;/code&gt;. In older versions of Python you can import this function via, &lt;code&gt;import pdb; pdb.set_trace()&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Finally check out &lt;code&gt;PDB++&lt;/code&gt; which is found &lt;a href=&quot;https://pypi.org/project/pdbpp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Its important to note that experienced programmers often debug by simply inserting &lt;code&gt;print()&lt;/code&gt; throughout their code and don't bother with an IDE&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-11-14T21:27:56.877" LastActivityDate="2020-11-14T21:27:56.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14818" PostTypeId="1" CreationDate="2020-11-14T21:35:34.980" Score="0" ViewCount="1805" Body="&lt;p&gt;I am working on a project using a fasta file. I am writing my command in nano within command-line and executing using python, also within my command-line.&lt;/p&gt;&#xA;&lt;p&gt;I would like my command to provide me with a tab delimited file with three columns: first column should contain my sequence name, second column should provide me with my sequence length, and the third column should show the sequence itself.&lt;/p&gt;&#xA;&lt;p&gt;I have written the following command so far within nano:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;import sys&#xA;for hello_fasta in SeqIO.parse(sys.argv[1], &amp;quot;fasta&amp;quot;):&#xA;&#xA;  list = hello_fasta.split(&amp;quot;\t&amp;quot;)&#xA;&#xA;  print hello_fasta.description&#xA;  print (len(hello_fasta.seq))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For example, I would like my command to provide me with the desired output and with the following order: Gene name ; Gene length ; Gene seq&lt;/p&gt;&#xA;&lt;p&gt;H0192X 26 FORUWOHRPPTRWFAWWEAKJNFWEJ&lt;/p&gt;&#xA;" OwnerUserId="9869" LastEditorUserId="9869" LastEditDate="2020-11-14T21:52:33.367" LastActivityDate="2021-02-07T21:21:09.287" Title="creating a tab delimited file" Tags="|python|assembly|file-formats|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14819" PostTypeId="1" AcceptedAnswerId="14822" CreationDate="2020-11-15T03:36:20.380" Score="1" ViewCount="70" Body="&lt;p&gt;I am working on a fasta file and am writing my command in nano within command-line and executing using python, also within a command line.&lt;/p&gt;&#xA;&lt;p&gt;My objective is to get my command to provide me with a tab delimited file with three columns: first column should contain my &lt;strong&gt;sequence name&lt;/strong&gt;, second column should provide me with my &lt;strong&gt;sequence length&lt;/strong&gt;, and the third column should show the &lt;strong&gt;sequence itself&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I have written the following command so far within nano:&lt;/p&gt;&#xA;&lt;p&gt;For example, I would like my command to provide me with the desired output and with the following order: Gene name ; Gene length ; Gene seq&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A06842X 45 GORHQRIHQEERHEUWOHRPPTRWFAWWEAKJNFWEJQEFQEPRT&#xA;&#xA;Jb4329PC 21 LDHKQAJTYOWEPLKFREEQW&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9708" LastEditorUserId="9708" LastEditDate="2020-11-15T21:12:09.877" LastActivityDate="2020-11-15T21:12:09.877" Title="Is there a simple command for outputting a tab delimited columns?" Tags="|python|fasta|assembly|file-formats|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14821" PostTypeId="1" CreationDate="2020-11-15T14:18:09.033" Score="1" ViewCount="161" Body="&lt;p&gt;I was wondering if there was a Python program that detects convergent and divergent evolution on a tree. I am also curious if it is just better to look at a phylogenetic tree and conclude what is convergent or divergent evolution than what a program tells you.&lt;/p&gt;&#xA;" OwnerUserId="9872" LastActivityDate="2022-08-09T04:04:05.463" Title="Is there a Python program that is able to show convergent and divergent evolution on a phylogenetic tree?" Tags="|python|phylogenetics|gene|database|genetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14822" PostTypeId="2" ParentId="14819" CreationDate="2020-11-15T16:22:03.680" Score="1" Body="&lt;p&gt;I have no idea what do you want to do with fasta files reformatted like this, but I think something like this should do the job (untested):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO &#xA;import sys &#xA;&#xA;for olig_fasta in SeqIO.parse(sys.argv[1], &amp;quot;fasta&amp;quot;):&#xA;&#xA;  name = olig_fasta.name&#xA;  seq = str(olig_fasta.seq)&#xA;  seq_len = str(len(olig_fasta.seq))&#xA;&#xA;  print(&amp;quot;\t&amp;quot;.join([name, seq_len, seq]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2020-11-15T18:16:27.293" LastActivityDate="2020-11-15T18:16:27.293" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14825" PostTypeId="1" AcceptedAnswerId="14829" CreationDate="2020-11-15T22:46:59.017" Score="2" ViewCount="313" Body="&lt;p&gt;I would like to compare mutational signatures&lt;sup&gt;1&lt;/sup&gt; in patients from different studies, however some studies are based on exome seq (i.e. ~20,000 coding variants) and some are from whole genome seq (i.e. ~22,000 coding variants) - is there a way to 'downsample' WGS data to better reflect WES data and effectively 'ignore' the coordinates of those ~2000 coding variants in the VCF files?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;sup&gt;1&lt;/sup&gt;Alexandrov, L.B., Kim, J., Haradhvala, N.J., Huang, M.N.,&#xA;Ng, A.W.T., Wu, Y., Boot, A., Covington, K.R., Gordenin, D.A.,&#xA;Bergstrom, E.N. and Islam, S.A., 2020. The repertoire of mutational&#xA;signatures in human cancer. Nature, 578(7793), pp.94-101.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9098" LastEditorUserId="3694" LastEditDate="2020-11-16T11:24:00.027" LastActivityDate="2020-11-16T11:24:00.027" Title="How can I subset WGS data to the level of WES variants?" Tags="|genome|ngs|vcf|software-recommendation|exome|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14826" PostTypeId="2" ParentId="14784" CreationDate="2020-11-15T22:59:26.480" Score="2" Body="&lt;p&gt;I'm not sure I understand the question, but XP_019970915.1 is an 'NCBI Reference Sequence' ID for 'erythrocyte binding antigen-181' (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/protein/XP_019970915.1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/protein/XP_019970915.1/&lt;/a&gt;). If you look that protein up in UniProt you can find other identifiers/info and GO annotations etc (&lt;a href=&quot;https://www.uniprot.org/uniprot/A0A151LVZ9&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uniprot/A0A151LVZ9&lt;/a&gt;). Does that answer your question?&lt;/p&gt;&#xA;" OwnerUserId="9098" LastActivityDate="2020-11-15T22:59:26.480" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14827" PostTypeId="2" ParentId="14799" CreationDate="2020-11-15T23:58:56.307" Score="1" Body="&lt;p&gt;The error I see is,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position&#xA;2440: character maps to &lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This suggests there is an error in the input format that the program doesn't like.&lt;/p&gt;&#xA;&lt;p&gt;The thing that bothers me is why you have a mix of / and \ for directory paths. Windows uses \ and Linux and OS X use / . I don't Windows so I really can't comment.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;It looks like you've used a character at position 2440 that the code doesn't like, this is often the case with ambiguous characters such as degenerate nucleotides or ambiguity codes for example &amp;quot;?&amp;quot; or &amp;quot;N&amp;quot;. It seems to suggest its neither or these but something like ...&lt;/li&gt;&#xA;&lt;li&gt;I'd be cautious of is the carriage returns because Windows is of course very different to Linux in this regard. \r (I think) versus \n&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;code&gt;undefined&lt;/code&gt; mapping suggests is quite a specific error and therefore resolvable.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-11-15T23:58:56.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14828" PostTypeId="1" CreationDate="2020-11-16T07:12:18.850" Score="2" ViewCount="918" Body="&lt;p&gt;I want to play some differentially expressed genes for gliomas cancer. I collected cancer tissue profiles from the TCGA database and normal tissue profiles from the GTEx database. I straightforward use ComBat from R package &amp;quot;sva&amp;quot;, and the result shows little differentially expressed genes (DEGs) were found. How can I remove batch effects correctly?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4665" LastActivityDate="2022-04-14T01:01:35.163" Title="How to remove batch effect from TCGA and GTEx data" Tags="|combat|gtex|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14829" PostTypeId="2" ParentId="14825" CreationDate="2020-11-16T09:53:35.747" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://samtools.github.io/bcftools/bcftools.html#view&quot; rel=&quot;nofollow noreferrer&quot;&gt;bcftools&lt;/a&gt; would be my choice, I'm sure bedtools could do the trick, too.&#xA;Something along this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view --regions-file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or  --targets, --regions-file might require a tabix index.&lt;/p&gt;&#xA;" OwnerUserId="3252" LastActivityDate="2020-11-16T09:53:35.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14830" PostTypeId="2" ParentId="14828" CreationDate="2020-11-16T11:06:49.443" Score="0" Body="&lt;p&gt;This has been asked and answered here and Biostars. My take home message from many similar posts is &amp;quot;not to bother with correcting batch effects when performing differential expression analysis but to include 'batch' as a covariate&amp;quot;. The latter accounts for the batch effects.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: This does not really answer OP's question, as @swbarnes2 and @ATpoint pointed out, they have a completely confounded design and the linear regression model would not accept such a model matrix.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2020-11-17T08:50:07.797" LastActivityDate="2020-11-17T08:50:07.797" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14831" PostTypeId="2" ParentId="14825" CreationDate="2020-11-16T11:18:46.983" Score="2" Body="&lt;p&gt;I'll expand slightly on the previous answer. First print off the positions for the exome file and then use bcftools view to filter the variants from the whole genome file. You could also index the whole_genome.vcf file to make the filtering faster.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools query \&#xA;    -f'%CHROM\t%POS\n' \&#xA;    exome.vcf &amp;gt; exome_variants.txt&#xA;&#xA;bcftools view \&#xA;    -T exome_variants.txt \&#xA;    whole_genome.vcf &amp;gt; whole_genome.exome_positions.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2020-11-16T11:18:46.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14833" PostTypeId="2" ParentId="14818" CreationDate="2020-11-16T14:31:45.463" Score="1" Body="&lt;p&gt;You can use a list and &lt;code&gt;insert()&lt;/code&gt; to add an element in a specific order, then expand the list with &lt;code&gt;*&lt;/code&gt;. Or you can use &lt;code&gt;join()&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;import sys&#xA;&#xA;for hello_fasta in SeqIO.parse(sys.argv[1], &amp;quot;fasta&amp;quot;):&#xA;  sequences = []&#xA;  sequences.insert(0, hello_fasta.description)&#xA;  sequences.insert(1, len(hello_fasta.seq))&#xA;  sequences.insert(2, hello_fasta.seq)&#xA;  # option 1&#xA;  print(*sequences, sep='\t')&#xA;  # option 2&#xA;  print('\t'.join(map(str, sequences)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2020-11-16T14:31:45.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14834" PostTypeId="1" CreationDate="2020-11-16T17:52:40.360" Score="2" ViewCount="490" Body="&lt;p&gt;I have differential binding sites object obtain from diffBind (dba.report). I am using the ChIP Seeker package to annotate the peaks but keep getting the following error: Error in (function (classes, fdef, mtable) : unable to find an inherited method for function ‘NSBS’ for signature ‘&amp;quot;SortedByQueryHits&amp;quot;’&lt;/p&gt;&#xA;&lt;p&gt;I used diffloop package to add chr to the chromosome column but it did not work. Result in the same error.&lt;/p&gt;&#xA;&lt;p&gt;Please let me know if you have any idea how to fix it. Highly appreciated&#xA;Here is my peakset&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BD&#xA;&#xA;  seqnames              ranges strand |      Conc Conc_condition Conc_Control      Fold      p-value         FDR&#xA;               &amp;lt;Rle&amp;gt;           &amp;lt;IRanges&amp;gt;  &amp;lt;Rle&amp;gt; | &amp;lt;numeric&amp;gt;     &amp;lt;numeric&amp;gt;    &amp;lt;numeric&amp;gt; &amp;lt;numeric&amp;gt;    &amp;lt;numeric&amp;gt;   &amp;lt;numeric&amp;gt;&#xA;    [1] NC_000003.12   75668856-75670016      * |  12.27197    12.8243533      0.76175  12.06260 1.43013e-102 1.41852e-97&#xA;    [2] NC_000024.10   11320736-11323578      * |  10.58988    11.1419124      0.76175  10.38016 1.15927e-100 5.74930e-96&#xA;    [3] NC_000004.12         37431-38634      * |   6.42233     0.0596816      8.06241  -8.00273  6.67673e-98 2.20750e-93&#xA;    [4] NC_000017.11   21678164-21686752      * |   9.26507     9.8163416      0.76175   9.05459  3.54723e-95 8.79607e-91&#xA;    [5] NC_000022.11   12692637-12694102      * |   9.64945    10.2010195      0.76175   9.43927  2.35838e-87 4.67845e-83&#xA;    ...          ...                 ...    ... .       ...           ...          ...       ...          ...         ...&#xA;  [226] NC_000004.12 131722954-131744030      * |   11.8156      12.36301     5.337173   7.02584  2.54289e-17 4.78151e-16&#xA;  [227] NC_000015.10   68134159-68136364      * |    8.2514       8.80099     0.969455   7.83153  2.28289e-16 3.73532e-15&#xA;  [228] NC_000005.10   34189082-34194027      * |   10.4671      11.01445     3.990668   7.02378  1.46745e-15 2.12829e-14&#xA;  [229] NC_000018.10       108003-108613      * |   11.2783       5.46414    12.912836  -7.44869  3.68643e-12 3.26794e-11&#xA;  [230] NC_000004.12   49709306-49710113      * |   12.0662       6.40350    13.698765  -7.29526  1.23922e-08 6.37694e-0&#xA;&#xA;peakAnno &amp;lt;- annotatePeak(BD, tssRegion=c(-3000, 3000), TxDb=txdb, annoDb=&amp;quot;org.Hs.eg.db&amp;quot;)&#xA;&amp;gt; peakAnno &amp;lt;- annotatePeak(BD, tssRegion=c(-3000, 3000), TxDb=txdb, annoDb=&amp;quot;org.Hs.eg.db&amp;quot;)&#xA;&amp;gt;&amp;gt; preparing features information...         2020-11-12 01:08:37 PM &#xA;&amp;gt;&amp;gt; identifying nearest features...       2020-11-12 01:08:37 PM &#xA;&amp;gt;&amp;gt; calculating distance from peak to TSS...  2020-11-12 01:08:38 PM &#xA;&amp;gt;&amp;gt; assigning genomic annotation...       2020-11-12 01:08:38 PM &#xA;Error in (function (classes, fdef, mtable)  : &#xA;  unable to find an inherited method for function ‘NSBS’ for signature ‘&amp;quot;SortedByQueryHits&amp;quot;’```&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8543" LastActivityDate="2021-05-15T23:01:00.113" Title="annotation using ChIPseeker package error" Tags="|differential-expression|chip-seq|sequence-annotation|peak-calling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14836" PostTypeId="2" ParentId="14821" CreationDate="2020-11-17T00:58:30.510" Score="1" Body="&lt;p&gt;Well in the absence of a definition of 'divergent evolution' I would simply use a definition of allopatry. If the divergence is congruent with geographic distance or geographic foci this provides some proof of divergent evolution.&lt;/p&gt;&#xA;&lt;p&gt;In this context you are looking for something like the biogeographic BAyesian analysis such as &lt;a href=&quot;https://academic.oup.com/sysbio/article/62/6/789/1708738&quot; rel=&quot;nofollow noreferrer&quot;&gt;BayArea&lt;/a&gt; or geographic add-ons to &lt;code&gt;Beast&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;There isn't a singular Python library that will handle this but there is some functionality with Biopython &lt;a href=&quot;https://biopython.org/wiki/Phylo&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Phylo&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-11-17T00:58:30.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14837" PostTypeId="1" CreationDate="2020-11-17T07:39:20.317" Score="2" ViewCount="627" Body="&lt;p&gt;I have the following files:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;V300066187_L4_B5RDBATtnuRAAAAA-408_2.fq.gz&#xA;V300066187_L4_B5RDBATtnuRAAAAA-408_1.fq.gz&#xA;V300068047_L2_B5RDBATtnuRAAAAA-408_2.fq.gz&#xA;V300068047_L2_B5RDBATtnuRAAAAA-408_1.fq.gz&#xA;V300068047_L2_B5RDBATtnuRAAAAA-407_2.fq.gz&#xA;V300068047_L2_B5RDBATtnuRAAAAA-407_1.fq.gz&#xA;V300068047_L2_B5RDBATtnuRAAAAA-405_1.fq.gz&#xA;V300068047_L2_B5RDBATtnuRAAAAA-406_1.fq.gz&#xA;V300068047_L2_B5RDBATtnuRAAAAA-405_2.fq.gz&#xA;V300066187_L4_B5RDBATtnuRAAAAA-407_2.fq.gz&#xA;V300066187_L4_B5RDBATtnuRAAAAA-407_1.fq.gz&#xA;V300066187_L4_B5RDBATtnuRAAAAA-406_2.fq.gz&#xA;V300066187_L4_B5RDBATtnuRAAAAA-406_1.fq.gz&#xA;V300066187_L4_B5RDBATtnuRAAAAA-405_2.fq.gz&#xA;V300066187_L4_B5RDBATtnuRAAAAA-405_1.fq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;the ending defines whether it is forward _1 or reverse _2.&lt;/p&gt;&#xA;&lt;p&gt;I want to merge all the &lt;code&gt;_1.fq.gz&lt;/code&gt; files into like &lt;code&gt;Merge1_.fq.gz&lt;/code&gt; and the same for &lt;code&gt;_2.fq.gz&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I tried doing &lt;code&gt;cat *_(number).fq.gz &amp;gt; merged(number).fq.gz&lt;/code&gt; and when I used the merges for assembly with MaSurca, it didn't detect any reads and failed. I used one pair and it worked well, but the genome size was smaller than expected, real size = 2 GB, estimated = 1.8GB.&lt;/p&gt;&#xA;&lt;p&gt;Would appreciate how to solve this issue and use all my short reads for assembly - I need to input one file for all forward reads and one file for all reverse reads.&lt;/p&gt;&#xA;" OwnerUserId="9882" LastEditorUserId="3848" LastEditDate="2020-11-17T12:25:47.790" LastActivityDate="2020-11-17T18:54:29.767" Title="Merging several fq.gz files or R1 and R2 classes into a single one" Tags="|genome|fastq|bash|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14840" PostTypeId="1" AcceptedAnswerId="14851" CreationDate="2020-11-17T13:20:33.550" Score="1" ViewCount="72" Body="&lt;p&gt;I received a ddRAD Seq data from my supervisor without barcodes and restriction enzymes. I asked him for both and he said I don't need it since the data has been cleaned by the company. Now, I am kinda confused and don't know where to start.&lt;/p&gt;&#xA;" OwnerUserId="9884" LastActivityDate="2020-11-18T03:28:32.153" Title="RAD Seq Data Analysis without barcode" Tags="|snp|phylogenetics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14841" PostTypeId="1" AcceptedAnswerId="14853" CreationDate="2020-11-17T15:29:27.783" Score="1" ViewCount="77" Body="&lt;p&gt;I was wondering if there is a tool/script/program that randomly generates GWAS data. The purpose of such a tool would be to use it for educational purposes. So you generate some random .ped, .map and other filetypes. Various distributions like the MAF of all the SNPs, X-Chromosome classificaiton error rates and so on could be set before generating the data. General stuff one can then filter out with plink or similar software to increase the quality of the data.&lt;/p&gt;&#xA;&lt;p&gt;Then a student would analyze these random files in plink and I already know what the outcome should be, therefore I can see right away if he or she did it right.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know if such a program already exists? Or is there a somewhat easy way to program this myself?&lt;/p&gt;&#xA;&lt;p&gt;Maybe it works if I use existing gene-data from NCBI where I use the given MAF to generate n variations the alleles based on that? Just some random SNPs from dbSNP?&lt;/p&gt;&#xA;" OwnerUserId="9131" LastEditorUserId="73" LastEditDate="2021-05-09T07:05:20.437" LastActivityDate="2021-05-09T07:05:20.437" Title="Random GWAS data generator" Tags="|snp|gwas|plink|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14844" PostTypeId="1" CreationDate="2020-11-17T18:14:20.113" Score="1" ViewCount="55" Body="&lt;p&gt;I’ve built two phylogeny at minimum evolution in MEGA-X and extracted these as newick files.&lt;/p&gt;&#xA;&lt;p&gt;My next step is to compare the two phylogeny to look at if the mutations within SARS2 are more prevalent within the spike, or as a global event across the whole genome. In order to do this I have chosen to use VisualTreeCMP. However, when it comes to choosing how to compare these I am very confused by all the possible options.&lt;/p&gt;&#xA;&lt;p&gt;Could someone please give me some guidance on the most suitable parameters to give a good image, and comparison between the two phylogeny.&lt;/p&gt;&#xA;&lt;p&gt;please see the newick format below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;((((((((((((((hCoV-19/Scotland/CVR140/2020|EPI_ISL_425684|2020-03-16:0.00026192,hCoV-19/England/CAMB-73211/2020|EPI_ISL_440287|2020-03-21:0.00026198):0.00000084,hCoV-19/Wuhan/HBCDC-HB-06/2020|EPI_ISL_412982|2020-02-07:0.00000000):0.00000034,hCoV-19/Gambia/NPHL-287/2020|EPI_ISL_561300|2020:0.00000000):0.00000372,hCoV-19/England/NORT-286167/2020|EPI_ISL_573313|2020-07-21:0.00025782):0.00002652,(hCoV-19/South_Africa/KRISP-0543/2020|EPI_ISL_498084|2020-07-12:0.00020599,(hCoV-19/England/ALDP-49EC3C/2020|EPI_ISL_559586|2020-05-20:0.00027741,hCoV-19/Belgium/HCM-0324151/2020|EPI_ISL_420408|2020-03-24:0.00024635):0.00005585):0.00021443):0.00002118,(hCoV-19/Colombia/MAG-INS-87968/2020|EPI_ISL_498152|2020-03-25:0.00015372,((hCoV-19/Australia/VIC8930/2020|EPI_ISL_565286|2020-07-29:0.00000000,hCoV-19/Australia/VIC8929/2020|EPI_ISL_565285|2020-07-28:0.00000000):0.00047159,(hCoV-19/Australia/VIC5819/2020|EPI_ISL_518613|2020-07-20:0.00024688,(hCoV-19/Australia/VIC4715/2020|EPI_ISL_519103|2020:0.00000000,hCoV-19/Australia/VIC2165/2020|EPI_ISL_480772|2020-03-19:0.00000000):0.00001495):0.00004719):0.00036022):0.00009570):0.00004876,((((hCoV-19/Wales/PHWC-16BD74/2020|EPI_ISL_573970|2020-09-15:0.00058675,hCoV-19/Spain/PV-IBV-001568/2020|EPI_ISL_467121|2020-03-13:0.00000000):0.00015901,hCoV-19/England/CAMB-7AAD6/2020|EPI_ISL_433837|2020-04-07:0.00009990):0.00003774,hCoV-19/Greece/HPI-6465/2020|EPI_ISL_451311|2020-03-29:0.00009055):0.00010675,((hCoV-19/USA/CA-SR0097/2020|EPI_ISL_445096|2020-03-13:0.00026193,hCoV-19/Iceland/13/2020|EPI_ISL_417765|2020-02-27:0.00026193):0.00020549,(hCoV-19/England/LIVE-A4CDD/2020|EPI_ISL_500083|2020-03-12:0.00007450,(hCoV-19/Wales/PHWC-16C441/2020|EPI_ISL_574056|2020-09-19:0.00022387,hCoV-19/England/SHEF-10B2F49/2020|EPI_ISL_537266|2020-05-21:0.00003797):0.00018736):0.00008143):0.00010981):0.00001973):0.00006712,(hCoV-19/Italy/LOM-UniMI-L134/2020|EPI_ISL_542130|2020-03-17:0.00064152,hCoV-19/Hong_Kong/HKU-200823-001/2020|EPI_ISL_516798|2020-03-26:0.00015299):0.00016120):0.00001351,(hCoV-19/USA/TX-HMH-1807/2020|EPI_ISL_542855|2020-04-20:0.00010829,hCoV-19/USA/CA-ALSR-1462/2020|EPI_ISL_494618|2020-03-20:0.00015349):0.00017136):0.00008773,(hCoV-19/USA/MA-MGH-00617/2020|EPI_ISL_460393|2020-03-28:0.00022036,(hCoV-19/USA/WA-S28/2020|EPI_ISL_417081|2020-03-02:0.00000726,hCoV-19/Singapore/746/2020|EPI_ISL_500560|2020-07-08:0.00025453):0.00022704):0.00003603):0.00001521,(hCoV-19/Spain/MD-IBV-006433/2020|EPI_ISL_510145|2020-04-03:0.00030013,hCoV-19/Austria/CeMM0009/2020|EPI_ISL_419662|2020-03-14:0.00000000):0.00025838):0.00001169,hCoV-19/Wuhan/IME-WH01/2020|EPI_ISL_529213|2019-12-30:0.00000000):0.00000016,hCoV-19/Wuhan/IPBCAMS-WH-01/2019|EPI_ISL_402123|2019-12-24:0.00000000):0.00000045,hCoV-19/USA/WA-S106/2020|EPI_ISL_417159|2020-02-29:0.00026265,NC_045512.2_Severe_acute_respiratory_syndrome_coronavirus_2_isolate_Wuhan-Hu-1_complete_genome:0.00000000);&#xA;&#xA;(((((((((((hCoV-19/England/ALDP-49EC3C/2020|EPI_ISL_559586|2020-05-20:0.00013635,hCoV-19/Belgium/HCM-0324151/2020|EPI_ISL_420408|2020-03-24:0.00006529):0.00000773,hCoV-19/South_Africa/KRISP-0543/2020|EPI_ISL_498084|2020-07-12:0.00026099):0.00013025,(hCoV-19/England/LIVE-A4CDD/2020|EPI_ISL_500083|2020-03-12:0.00016259,(hCoV-19/Gambia/NPHL-287/2020|EPI_ISL_561300|2020:0.00016807,hCoV-19/England/CAMB-73211/2020|EPI_ISL_440287|2020-03-21:0.00013706):0.00000534):0.00000008):0.00000278,(hCoV-19/England/NORT-286167/2020|EPI_ISL_573313|2020-07-21:0.00036517,((hCoV-19/Australia/VIC8930/2020|EPI_ISL_565286|2020-07-29:0.00000000,hCoV-19/Australia/VIC8929/2020|EPI_ISL_565285|2020-07-28:0.00000012):0.00009405,(hCoV-19/Australia/VIC5819/2020|EPI_ISL_518613|2020-07-20:0.00006516,(hCoV-19/Australia/VIC4715/2020|EPI_ISL_519103|2020:0.00003344,hCoV-19/Australia/VIC2165/2020|EPI_ISL_480772|2020-03-19:0.00003367):0.00000193):0.00000601):0.00017105):0.00002318):0.00000716,hCoV-19/Greece/HPI-6465/2020|EPI_ISL_451311|2020-03-29:0.00004599):0.00008629,(hCoV-19/England/SHEF-10B2F49/2020|EPI_ISL_537266|2020-05-21:0.00015654,(hCoV-19/Wales/PHWC-16C441/2020|EPI_ISL_574056|2020-09-19:0.00019758,hCoV-19/Wales/PHWC-16BD74/2020|EPI_ISL_573970|2020-09-15:0.00023814):0.00021545):0.00003087):0.00000571,((hCoV-19/USA/TX-HMH-1807/2020|EPI_ISL_542855|2020-04-20:0.00023928,hCoV-19/Colombia/MAG-INS-87968/2020|EPI_ISL_498152|2020-03-25:0.00013017):0.00003266,(hCoV-19/Scotland/CVR140/2020|EPI_ISL_425684|2020-03-16:0.00003644,hCoV-19/Italy/LOM-UniMI-L134/2020|EPI_ISL_542130|2020-03-17:0.00023391):0.00000153):0.00000121):0.00012657,(hCoV-19/Singapore/746/2020|EPI_ISL_500560|2020-07-08:0.00034258,(hCoV-19/Iceland/13/2020|EPI_ISL_417765|2020-02-27:0.00012787,(hCoV-19/England/CAMB-7AAD6/2020|EPI_ISL_433837|2020-04-07:0.00012752,(hCoV-19/USA/CA-ALSR-1462/2020|EPI_ISL_494618|2020-03-20:0.00010172,hCoV-19/Hong_Kong/HKU-200823-001/2020|EPI_ISL_516798|2020-03-26:0.00020057):0.00000680):0.00004213):0.00002475):0.00002767):0.00000390,(hCoV-19/Wuhan/HBCDC-HB-06/2020|EPI_ISL_412982|2020-02-07:0.00007999,(hCoV-19/Spain/PV-IBV-001568/2020|EPI_ISL_467121|2020-03-13:0.00019054,((hCoV-19/Wuhan/IME-WH01/2020|EPI_ISL_529213|2019-12-30:0.00003592,hCoV-19/Spain/MD-IBV-006433/2020|EPI_ISL_510145|2020-04-03:0.00020087):0.00000317,(hCoV-19/USA/CA-SR0097/2020|EPI_ISL_445096|2020-03-13:0.00011733,(hCoV-19/USA/MA-MGH-00617/2020|EPI_ISL_460393|2020-03-28:0.00013271,(hCoV-19/USA/WA-S28/2020|EPI_ISL_417081|2020-03-02:0.00003240,hCoV-19/USA/WA-S106/2020|EPI_ISL_417159|2020-02-29:0.00006806):0.00001270):0.00000336):0.00008627):0.00001038):0.00001150):0.00004587):0.00000335,hCoV-19/Austria/CeMM0009/2020|EPI_ISL_419662|2020-03-14:0.00013033):0.00000452,hCoV-19/Wuhan/IPBCAMS-WH-01/2019|EPI_ISL_402123|2019-12-24:0.00010072,NC_045512.2_Severe_acute_respiratory_syndrome_coronavirus_2_isolate_Wuhan-Hu-1_complete_genome:0.00000000);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And a link to the tree viewer:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://eti.pg.edu.pl/treecmp/WEB&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://eti.pg.edu.pl/treecmp/WEB&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I am just requesting the basic parameters, on how one would get the most out of the programme... or failing that another programme with less in depth parameterization.&lt;/p&gt;&#xA;" OwnerUserId="9672" LastEditorUserId="3848" LastEditDate="2020-12-03T12:42:23.583" LastActivityDate="2020-12-03T12:42:23.583" Title="Comparison of phylogeny of SARS2 whole genome vs spike genome" Tags="|phylogenetics|phylogeny|covid-19|sars-cov-2|homework|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="14845" PostTypeId="2" ParentId="14837" CreationDate="2020-11-17T18:54:29.767" Score="2" Body="&lt;p&gt;What you did should work.  You don't need to decompress and recompress.&lt;/p&gt;&#xA;&lt;p&gt;I'd check the files.  Were they trimmed?  Can you get the top, say 1000 lines of the merged files to work?  (You will need to decompress to count lines accurately)&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-11-17T18:54:29.767" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14846" PostTypeId="2" ParentId="14840" CreationDate="2020-11-17T20:30:41.230" Score="0" Body="&lt;p&gt;The fundamental problem with RAD is that if you hit paralogue - you're done.&lt;/p&gt;&#xA;&lt;p&gt;Identifying a paraologue isn't necessarily easy depends on the species. Essentially if you're doing pop gen you need homologues and RAD has too many situations where it can fall over without you knowing.&lt;/p&gt;&#xA;&lt;p&gt;What happens if you've a paralogue is that you start confusing concerted evolution (pop gen of a genome) with population structure between individuals. I do agree that without barcodes identifying individuals - thats a bit tricky ;-) , however RAD an approach loaded with pitfalls.&lt;/p&gt;&#xA;&lt;p&gt;Pop gen means being able to identify populations and on face value that will not be trivial without barcodes (I suspect they are still there), but RAD is far from an ideal tool and can give weird and wonderful metrics which have little to do with the population structure.&lt;/p&gt;&#xA;&lt;p&gt;Its not the only problem with RAD for the purpose you are using it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-11-17T21:26:09.273" LastActivityDate="2020-11-17T21:26:09.273" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14848" PostTypeId="2" ParentId="14767" CreationDate="2020-11-17T23:58:58.727" Score="1" Body="&lt;p&gt;Here is a bash/awk solution:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk -F'\t' 'NR==1{print}; NR&amp;gt;1{N=0;for(i=2; i&amp;lt;(NF+1); i++){if($i &amp;gt;= 0.0001) N++}; if(N&amp;gt;0.1*NF) print}' input.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We make sure the input field separator is tab, then we loop on the number of fields (NF) and add to the tally the &lt;code&gt;N&lt;/code&gt; of fields above threshold. At the end we simply &lt;code&gt;print&lt;/code&gt; the entire line if &lt;code&gt;N&lt;/code&gt; is high enough.&lt;/p&gt;&#xA;&lt;p&gt;EDITED: for the header, we print the first line (&lt;code&gt;NR==1&lt;/code&gt;) as is, then apply the expression when Number of Records &amp;gt; 1.&lt;/p&gt;&#xA;" OwnerUserId="9497" LastEditorUserId="9497" LastEditDate="2020-11-22T19:22:54.103" LastActivityDate="2020-11-22T19:22:54.103" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14849" PostTypeId="1" CreationDate="2020-11-18T00:04:01.943" Score="3" ViewCount="562" Body="&lt;p&gt;I am currently investigating how to predict protein structure and contact map predictions. There, I see things like top L/5 and top L/2 scores as a way to evaluate the contact map. What formulas do you use to evaluate these? Also, what is a meaningful evaluation method?&lt;/p&gt;&#xA;" OwnerUserId="9890" LastActivityDate="2021-04-22T12:10:03.267" Title="How do you calculate the top L/5 score?" Tags="|proteins|protein-structure|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14851" PostTypeId="2" ParentId="14840" CreationDate="2020-11-18T03:28:32.153" Score="2" Body="&lt;p&gt;Wow Biobash, that's kinda harsh. It is cool that sometimes you unexpectedly learned something new. What Michael mentioned is pretty interested, no one has ever mentioned that or I have never heard that.&lt;/p&gt;&#xA;&lt;p&gt;So, my approach would be that you now are doing a Poolseq study, since you can't identify individuals. The other way to try to catch the paralogues is to map your reads to a sister species. My question is what are you trying to estimate? Genetic differentation? Genetic diversity? Pop Gen structure?&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6116966/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6116966/&lt;/a&gt;&#xA;&lt;a href=&quot;https://onlinelibrary.wiley.com/doi/full/10.1002/ece3.5240&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://onlinelibrary.wiley.com/doi/full/10.1002/ece3.5240&lt;/a&gt;&#xA;&lt;a href=&quot;https://www.g3journal.org/content/9/12/4159&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.g3journal.org/content/9/12/4159&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;One semi-robust approach to assess usefulness is thinking about your system and defining what you expect to find, let's say in terms of genetic differentiation or genetic structure, and estimating that from your data. If that hypothesis is confirmed, your dataset might be useful. You can test the same by first estimating the loci by mapping your reads to a sister reference genome, to try to detect paralogues. Since you don't have barcodes, then each &amp;quot;locus&amp;quot; is independent, and you lose power, but there is nothing else you can do.&lt;/p&gt;&#xA;&lt;p&gt;Good luck;&lt;/p&gt;&#xA;" OwnerUserId="9882" LastActivityDate="2020-11-18T03:28:32.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14852" PostTypeId="1" CreationDate="2020-11-18T09:11:11.903" Score="0" ViewCount="307" Body="&lt;p&gt;I have three backbone and i want to concatenate 70 sequence into these backbone.&#xA;such like:&#xA;fasta file 1:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Backbone1: ACTGCGTATGTCAGCATATAGACCGCCT&#xA;Backbone2: CGTAGCTAGCTAGCTCGTGTCGTGATGCTACTC&#xA;Backbone3: GTCATGCTACGTGCTACGTGCTCGTCGCGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;fasta file 2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sequence_1:CGCTAGTGTAGCTGCTACTCCG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output sequence:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Backbone1+sequence_1: ACTGCGTATGTCAGCATATAGACCGCCTCGCTAGTGTAGCTGCTACTCCG&#xA;Backbone2+sequence_1: CGTAGCTAGCTAGCTCGTGTCGTGATGCTACTCCGCTAGTGTAGCTGCTACTCCG&#xA;Backbone3+sequence_1: GTCATGCTACGTGCTACGTGCTCGTCGCGCCGCTAGTGTAGCTGCTACTCCGs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9893" LastEditorUserId="298" LastEditDate="2020-11-20T10:49:42.470" LastActivityDate="2020-11-20T11:18:28.540" Title="How to concatenate two fasta sequences by py" Tags="|python|sequence-analysis|phylogenetics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14853" PostTypeId="2" ParentId="14841" CreationDate="2020-11-18T10:48:11.853" Score="1" Body="&lt;p&gt;Agreed with the above I would use an existing dataset from NCBIs dbSNP &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Random doesn't work well because HW signal will be meaningless as soon as the Jukes-Cantor distance is exceeded (which it will) and plink needs homologues to generate any signal either in 'reasonable' panmixia or clonality. The result will be panmictic beyond even the most panmictic organism ever known. I don't mind having a simulation of 100% clonalty, because 100% clonality does occur, either through inbreeding or binary division, but this simulation would generate the complete opposite result.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-11-18T10:48:11.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14854" PostTypeId="1" CreationDate="2020-11-18T11:38:39.460" Score="2" ViewCount="68" Body="&lt;p&gt;I am reading a book about RNAseq analysis and it says&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;To calculate the probability that a read will map to a specific gene, we can assume an average gene size of 4000 nt (100 M nt divided by 25,000 genes). At 30 M reads equivalent to 30× coverage, at single read 100 nt (or paired-end read 50 nt) length, we can expect a single read to map to the average expressed and length gene 4000 nt× 30 coverage/100 nt 1200 times. Thus, if the gene is expressed at a level of 1/1200 compared to the average gene, then we have a 50:50 probability to have a read map to it.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I understand that a single read would map to the average expressed 1200 times but I don't understand how the 50:50 probability is calculated. Please could you explain how this probability is determined.&lt;/p&gt;&#xA;" OwnerUserId="6787" LastActivityDate="2020-11-19T14:06:02.783" Title="If a gene is expressed at a level of 1/1200 compared to the average gene, how is probability 50:50 that we have a read mapped to it?" Tags="|rna-seq|reads|coverage|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14855" PostTypeId="1" AcceptedAnswerId="14861" CreationDate="2020-11-18T13:37:48.280" Score="2" ViewCount="236" Body="&lt;p&gt;Hi and sorry for the long post in advance,&lt;/p&gt;&#xA;&lt;p&gt;I'm doing a survival analysis of lung cancer patients using Python's lifelines package.&#xA;According to the documentation, the function plot_partial_effects_on_outcome() plots the effect of a covariate on the observer's survival. I do understand that the CoxPH model assumes that the log-hazard of an individual is modelled by a  linear function of their covariates, however, in some cases the effect of these covariates seems to perfect when plotted. I'll illustrate these with three images below describing the issues I have in my data analysis.&lt;/p&gt;&#xA;&lt;p&gt;Example 1:&#xA;This one I'm providing as it looks realistic to me, there is a clear difference in the effect of the covariate when positive or negative.&#xA;&lt;img src=&quot;https://user-images.githubusercontent.com/25772555/99532844-da5b8500-299c-11eb-9b02-b0a81bfa6887.png&quot; alt=&quot;image&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Example 2:&#xA;This one just looks gimmicky, my concern is firstly there seems to be a perfect relation between the pathologic stage of cancer and the survival, it's logical that higher stages should impact the survival negatively, however, I don't understand why the graph is so symmetric as the survivability of cancer shouldn't be proportional to the patient cancer staging.&#xA;&lt;img src=&quot;https://user-images.githubusercontent.com/25772555/99533165-5ce44480-299d-11eb-90fd-02255a88b9a6.png&quot; alt=&quot;image&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Example 3:&#xA;Same problems as for example 2, this time with a binary not categorical variable (so I'm less reluctant on disbelieving it), however, the curves do seem to symmetric, almost like it's the same curve but with a standard deviation. I also highlighted a region of interest which I don't understand why both curves stabilize (maybe lack of patients in those regions?)&#xA;&lt;img src=&quot;https://user-images.githubusercontent.com/25772555/99533751-4ee2f380-299e-11eb-9770-513204b9e11c.png&quot; alt=&quot;image&quot; /&gt;&lt;/p&gt;&#xA;" OwnerUserId="9894" LastActivityDate="2020-11-19T12:46:39.400" Title="Survival analysis using CoxPH - Effect of covariates" Tags="|python|visualization|cancer|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14858" PostTypeId="1" AcceptedAnswerId="14866" CreationDate="2020-11-18T23:33:44.583" Score="1" ViewCount="81" Body="&lt;p&gt;I need to find dissimilar proteins. Looking through the PDB I found the weekly BLASTClust results of proteins that are 30% &lt;em&gt;similar&lt;/em&gt;. However, I do not know if protein A in cluster 1 is guaranteed to be less than 30% similar to protein B in cluster 2.&lt;/p&gt;&#xA;&lt;p&gt;The best and only documentation I can find for BLASTClust is here &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Web/Newsltr/Spring04/blastlab.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;(link)&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;It states:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The program begins with pairwise matches and places a sequence in a cluster if the sequence matches at least one sequence already in the cluster.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;That's good, but consider a set of three proteins (A, B, and C). A is 40% similar to B and 40% similar to C. B is 20% similar to C. Say that the algorithm places B in cluster 1 and then C in cluster 2 as they are dissimilar. What happens when it tries to place A? Because A is similar to both B and C will they be joined into one cluster (this would be the best result for me) or is A just placed into one cluster even though it is similar to a protein in another cluster?&lt;/p&gt;&#xA;" OwnerUserId="9897" OwnerDisplayName="lazer-guided-lazerbeam" LastActivityDate="2020-11-21T21:41:57.070" Title="Does BLASTClust guarantee that proteins in different clusters are dissimilar?" Tags="|blast|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14860" PostTypeId="1" AcceptedAnswerId="14864" CreationDate="2020-11-19T08:17:26.823" Score="1" ViewCount="42" Body="&lt;p&gt;I am posting this message because I need clarity about the analysis I want to perform.&#xA;In my analysis, I have a homologous gene in &lt;strong&gt;13 species&lt;/strong&gt;, and I would like to evaluate the selection pressure of this gene (its dN/dS) in &lt;strong&gt;3 species&lt;/strong&gt; of the tree and test if this value is significantly different from the neutral model (dN/dS = 1).&lt;/p&gt;&#xA;&lt;p&gt;So here is the protocol I am considering, and I would like you to tell me if I am right or not:&lt;/p&gt;&#xA;&lt;p&gt;To calculate this selection pressure, the ratio of non-synonymous to synonymous substitution rates (dN/dS) must be calculated.&lt;/p&gt;&#xA;&lt;p&gt;For this I use CODEML implemented in ete3.&lt;/p&gt;&#xA;&lt;p&gt;So I need :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A phylogenetic tree of the gene (performed with the AA alignment of the gene via Iqtree)&#xA;&#xA;A codon alignment (performed with the DNA alignment of the gene with pal2nal)&#xA;&#xA;Mark branches in the tree. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here is the exemple where I want to get the &lt;strong&gt;dN/dS&lt;/strong&gt; value of &lt;strong&gt;SP5,6&lt;/strong&gt; and &lt;strong&gt;7&lt;/strong&gt;.:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;         /-SP1&#xA;        /-|&#xA;       |   \-SP2&#xA;       |&#xA;     /-|      /-SP3&#xA;    |  |   /-|&#xA;    |  |  |   \-SP4&#xA;    |   \-|&#xA;    |     |   /-SP5&#xA;  /-|      \- |  /-SP6&#xA; |  |         \-|   &#xA; |  |           |&#xA; |  |            \-SP7&#xA; |  |&#xA; |  |   /-SP8&#xA;-|   \-|&#xA; |      \-SP9&#xA; |&#xA; |     /-SP10&#xA; |  /-|&#xA; | |   \-SP11&#xA; |-|     &#xA;   |   /-SP12&#xA;    \-|&#xA;       \-SP13&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And I want to calculate the &lt;strong&gt;dN/dS&lt;/strong&gt; of &lt;code&gt;SP5,SP6&lt;/code&gt; and &lt;code&gt;SP7&lt;/code&gt; in order to confirme the hypotesis that the gene is under strong purifying selection on this 3 species.&lt;/p&gt;&#xA;&lt;p&gt;I mark the SP5, SP6 and SP7, run a &lt;code&gt;free-branch&lt;/code&gt; model and look at the Codeml output (the dN/dS value should be the same for the 3 marked species right?) Then this value is kind of the mean dN/dS value between three leafs?&lt;/p&gt;&#xA;&lt;p&gt;Suppose I get an omega = &lt;strong&gt;0.15&lt;/strong&gt;&#xA;If I want to test that this value is significantly different from 1, I need to mark the same sheets and run the neutral model, then compare the two models (&lt;code&gt;neutral&lt;/code&gt; vs. &lt;code&gt;free&lt;/code&gt;) using chi2, and if the p-value is &lt;code&gt;&amp;lt;0.5&lt;/code&gt;, then the dN/dS value (&lt;strong&gt;0.15&lt;/strong&gt;) is significantly different from 1.&#xA;I'm right  ?&lt;/p&gt;&#xA;" OwnerUserId="9104" LastEditorUserId="9104" LastEditDate="2020-11-19T09:44:08.213" LastActivityDate="2020-11-19T17:41:42.153" Title="Measure the purifying selection for certain taxa along a phylogeny" Tags="|sequence-alignment|phylogeny|positive-selection|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="14861" PostTypeId="2" ParentId="14855" CreationDate="2020-11-19T12:46:39.400" Score="3" Body="&lt;p&gt;Example 2 and 3 seem symmetric because they are forced to by the model. The Cox proportional hazard relies on the proportional hazards assumption. This basically means that the ratio of the hazards are constant over time. If you were to plot the hazard ratio, it would look like a flat line over all &lt;code&gt;Event_at&lt;/code&gt;. Another way of saying this is that the hazard ratios are constant over time.&lt;/p&gt;&#xA;&lt;p&gt;How that shows up in a survival plot is a little different because of the relationship between hazards and survival. The survival curves end up following the pattern exemplified in Example 2.&lt;/p&gt;&#xA;&lt;p&gt;As for example 3, I would guess that there are few individuals with event times past 3500. This is why you have those large jumps.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Proportional Hazards&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;So what does the proportional hazards assumption mean? Below are some plots of hazard functions to help your intuition on what the proportional hazards assumption means. In all of the following scenarios, the hazard ratio is 2 for blue-to-red. However, the proportional hazards assumption is only true in A, B, and C.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UxBe6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UxBe6.png&quot; alt=&quot;Plot of hazards&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since the &lt;em&gt;proportion&lt;/em&gt; is remaining constant, it can look a little hard to see. To make it easier on the eyes, you can log-transform the hazard functions. Below is a plot that does that.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eB7yG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eB7yG.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you can see, the hazards are a constant distance from each other for A, B, and C. Basically, the proportional hazards assumption is saying that your scenario looks something like one of those plots.&lt;/p&gt;&#xA;&lt;p&gt;In scenario D, the hazard ratio varies over time. At the beginning it is basically HR=1. At the end it is basically HR=4. With the proportional hazards assumption, Cox regression will essentially take the 'average' of the hazards over time. In this case it would give you an estimate of HR=2. Depending on the scenario, this 'smoothing' may be more or less acceptable. But this is basically what the proportional hazards assumption does in the background.&lt;/p&gt;&#xA;&lt;p&gt;The proportional hazards assumption is actually more problematic in the cases where the two hazard functions cross each other. Below is an example of that.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CMWZO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CMWZO.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this scenario, blue has an increased hazard at the start (relative to red) then a decreased at the end. A Cox proportional hazards model would give you HR=1 in this case, because the average over all time points is no relationship.&lt;/p&gt;&#xA;&lt;p&gt;So how do you tell if you violate the proportional hazards assumption? You may want to use a stratified Kaplan-Meier by your variables to get an idea of what the hazards look like without stipulating the proportional hazards assumption. You can use the Kaplan-Meier estimator to get a non-parametric estimate of the discrete hazard (i.e. it does not make the proportional hazards assumption). Then you can log-transform those hazards to get a plot like the second. There are also statistical tests and other diagnostics (Schoenfeld residuals) to check this assumption.&lt;/p&gt;&#xA;&lt;p&gt;The next question is do those violations matter? This somewhat depends on the question you are trying to answer and how substantial the violations are. In cases where the two log-hazard curves don't cross (like in D), it may be acceptable to smooth over the variations. However, it settings where the curves cross in the final figure, it is probably best to not ignore this problem.&lt;/p&gt;&#xA;&lt;p&gt;The original write-up of this answer was &lt;a href=&quot;https://github.com/CamDavidsonPilon/lifelines/issues/1168&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9896" LastActivityDate="2020-11-19T12:46:39.400" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14863" PostTypeId="2" ParentId="14854" CreationDate="2020-11-19T14:06:02.783" Score="3" Body="&lt;p&gt;I actually disagree with that.. I guess I write this down as a discussion.&lt;/p&gt;&#xA;&lt;p&gt;The expected value for an average gene is 1200. For this gene at 1/1200 expression, you expect 1 read.&lt;/p&gt;&#xA;&lt;p&gt;However, because of sampling, sometimes you get 0, 1 or 2. If we assume the sampling follows a poisson distribution, we can calculate the probability of getting 0, 1 , 2 ... reads:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plot(dpois(0:10,lambda=1),ylab=&amp;quot;Probability&amp;quot;,xlab=&amp;quot;No of reads&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/R3q8V.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/R3q8V.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The probabilities are like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data.frame(n=0:5,p=dpois(0:5,lambda=1))&#xA;&#xA;  n           p&#xA;1 0 0.367879441&#xA;2 1 0.367879441&#xA;3 2 0.183939721&#xA;4 3 0.061313240&#xA;5 4 0.015328310&#xA;6 5 0.003065662&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The probability of getting exactly 0 or 1 read should be ~0.37. The probability of getting at least 1 read would be 1 - P(n=0) = 1- 0.367879441 = 0.6321206&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-11-19T14:06:02.783" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14864" PostTypeId="2" ParentId="14860" CreationDate="2020-11-19T17:34:55.923" Score="1" Body="&lt;p&gt;The following is not the clearest explanation of dN/dS humankind has seen, but I hope you get the gist and core idea of likelihood is absolutely correct. I am wondering if you are thinking of a different test to the way I would do this calculation. This ain't a text book clarity answer.&lt;/p&gt;&#xA;&lt;p&gt;Anyway what you are proposing is a likelihood ratio test (LRT) for  selection based on the parameterisation of omega. Your idea isn't quite correct ... its not dN/dS value you are looking at to assess significance, its the likelihood. This point is critical and is the most important difference in our thinking on this test.&lt;/p&gt;&#xA;&lt;p&gt;LRT works because its a log likelihood and subtracting two log likelihood is the same as dividing them (I forget the theory - but you subtract the likelihood to work out the p-value via Chi squared). There's some theory about why the Chi-squared distribution works but I can't remember it.&lt;/p&gt;&#xA;&lt;p&gt;You need to work out the degrees of freedom which is the parameter differences between 'free branch model' and the neutral model and this value is LOW. If the neutral model is the &amp;quot;global model&amp;quot;, then its 2 degrees of freedom - IN YOUR TREE. This is because the SP you are looking at form a single clade (monophyly). From memory you stick a #1 on the branch leading into the clade, ie. defining the node and thats your 'model' on the treefile. Here you have two dN/dS one for the whole tree excluding your clade and one for your taxa. This gives you a likelihood.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; you could stick a #1 on each branch within the clade, i.e. the treefile. That means something different - like every branch was subject to ... in this case purifying selection ... not just the common ancestor.&lt;/p&gt;&#xA;&lt;p&gt;You now work out the likelihood of the 'global' model... i.e. remove the #1 and make sure it gives you only 1 dN/dS value.&lt;/p&gt;&#xA;&lt;p&gt;Subtract the likelihoods and multiple by two - if its LRT (super important).&lt;/p&gt;&#xA;&lt;p&gt;You now need to work out the degrees of freedom. If its the same model but one has an additional node with a different dN/dS ... the df = 2. If you've dumped a #1 on every branch in the clade ... I still think df = 2 - you need to check.&lt;/p&gt;&#xA;&lt;p&gt;If the difference in likelihood results in &lt;em&gt;p&lt;/em&gt; &amp;lt; 0.05 you are correct the difference is significant ... whatever that difference might be.&lt;/p&gt;&#xA;&lt;p&gt;If the neutral model is some different calculation of omega within the population genetics process (see &lt;a href=&quot;https://journals.plos.org/plosgenetics/article?id=10.1371/journal.pgen.1000304&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) then it will be the parameter differences between your model and the 'neutral' model.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;To reiterate&lt;/strong&gt;&#xA;I like simulations so much I type in reiterations ... So again the LRT usual will usually subtract one likelihood from another and THEM multiply the probability by two ... and compare the result on the Chi squared table for the difference in degrees of freedom. And again what you are looking at is not the value of dN/dS but the difference in LIKELIHOOD. That is very important. You need to be careful with the degrees of freedom on an LRT because they can introduce an additional subtraction constant, test dependings (its ages since I've done one so I can't remember).&lt;/p&gt;&#xA;&lt;p&gt;If the test is significant and one dN/dS is x for the clade and y for the rest of the tree, then that differences is significant and if y is &amp;gt;1 (bit unlikely) and x &amp;lt; 1 then sure thats purifying selection. Usually clades evolve via positive selection rather than purifying selection.&lt;/p&gt;&#xA;&lt;p&gt;From memory that is my understanding ... but if you have a neutral model of omega and a purifying model of omega ... yeah sort of, but usually thats for a 'global' model, i.e. no differences within the tree. In your case your interested in a specific clade. The idea is still the same its measured through likelihood.&lt;/p&gt;&#xA;&lt;p&gt;Its so long since I've done this, but I do caution not to confuse two different tests.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I do understand Yang et al did an MCMC for dN/dS ... what I'm thinking of here is maximum likelihood. You'd need to watch for that carefully and thats what you are referring to when you say Chi-squared, because that is LRT 100%. LRT is maximum likelihood.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-11-19T17:41:42.153" LastActivityDate="2020-11-19T17:41:42.153" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14866" PostTypeId="2" ParentId="14858" CreationDate="2020-11-19T19:09:14.583" Score="2" Body="&lt;p&gt;Based on the documentation (that I linked to in comment), it seems that the clustering algorithm used by blastclust is &lt;a href=&quot;https://en.wikipedia.org/wiki/Single-linkage_clustering&quot; rel=&quot;nofollow noreferrer&quot;&gt;single-linkage hierarchical clustering&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;SO what you can expect is that A will be placed with the protein to which it is most similar. And then the last protein will be added as an outgroup to that cluster due to its similarity to A (assuming that there are no other proteins).&lt;/p&gt;&#xA;&lt;p&gt;So if your goal is &amp;quot;I want clusters that are guaranteed to contain dissimilar proteins&amp;quot;, then you might not be able to naively take blastclust output and interpret it as such. You could either postprocess it to find the set of clusters in which no protein is more than 30% similar to a protein in any other cluster (which may just be one big cluster!), or find another clustering algorithm that is more suited to your task.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It does look like blastclust uses a minimum &lt;em&gt;similarity&lt;/em&gt; (not identity) threshold (from blastclust docs). So they use an implementation of single-linkage clustering that stops before merging everything into a single cluster:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;If the coverage is above a certain threshold&#xA; AND&#xA;the score density is above a certain threshold,&#xA;&#xA;these two sequences are considered to be neighbored.&#xA;&#xA;Thus determined neighbor relationships is considered symmetric and provides&#xA;the base for clustering by a single-linkage method (which puts a sequence&#xA;to a cluster if the sequence is a neighbor to at least one sequence in the&#xA;cluster).&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I don't think that this is a general property of the generic clustering algorithm though (see the &amp;quot;Naive algorithm&amp;quot; section).&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2020-11-21T21:41:57.070" LastActivityDate="2020-11-21T21:41:57.070" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14869" PostTypeId="2" ParentId="14828" CreationDate="2020-11-19T22:38:30.883" Score="1" Body="&lt;p&gt;There is a paper that did what you are asking: &lt;a href=&quot;https://www.nature.com/articles/sdata201861&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unifying cancer and normal RNA sequencing data from different sources&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2020-11-19T22:38:30.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14871" PostTypeId="1" AcceptedAnswerId="14872" CreationDate="2020-11-20T07:02:53.533" Score="0" ViewCount="280" Body="&lt;p&gt;&lt;strong&gt;Control sample chip read stat&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;56234445 + 0 in total (QC-passed reads + QC-failed reads)&#xA;0 + 0 secondary&#xA;0 + 0 supplementary&#xA;0 + 0 duplicates&#xA;55173337 + 0 mapped (98.11% : N/A)&#xA;56234445 + 0 paired in sequencing&#xA;28078024 + 0 read1&#xA;28156421 + 0 read2&#xA;53808040 + 0 properly paired (95.69% : N/A)&#xA;54626508 + 0 with itself and mate mapped&#xA;546829 + 0 singletons (0.97% : N/A)&#xA;162718 + 0 with mate mapped to a different chr&#xA;112602 + 0 with mate mapped to a different chr (mapQ&amp;gt;=5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;IgG sample chip seq read stat&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;32745255 + 0 in total (QC-passed reads + QC-failed reads)&#xA;0 + 0 secondary&#xA;0 + 0 supplementary&#xA;0 + 0 duplicates&#xA;31546173 + 0 mapped (96.34% : N/A)&#xA;32745255 + 0 paired in sequencing&#xA;16362391 + 0 read1&#xA;16382864 + 0 read2&#xA;30934194 + 0 properly paired (94.47% : N/A)&#xA;31269484 + 0 with itself and mate mapped&#xA;276689 + 0 singletons (0.84% : N/A)&#xA;79786 + 0 with mate mapped to a different chr&#xA;49658 + 0 with mate mapped to a different chr (mapQ&amp;gt;=5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;knockout sample Rep 1 stat&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;28044731 + 0 in total (QC-passed reads + QC-failed reads)&#xA;0 + 0 secondary&#xA;0 + 0 supplementary&#xA;0 + 0 duplicates&#xA;27459222 + 0 mapped (97.91% : N/A)&#xA;28044731 + 0 paired in sequencing&#xA;14014483 + 0 read1&#xA;14030248 + 0 read2&#xA;26685254 + 0 properly paired (95.15% : N/A)&#xA;27162846 + 0 with itself and mate mapped&#xA;296376 + 0 singletons (1.06% : N/A)&#xA;116684 + 0 with mate mapped to a different chr&#xA;88965 + 0 with mate mapped to a different chr (mapQ&amp;gt;=5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Knockout sample rep 2 stat&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;32410094 + 0 in total (QC-passed reads + QC-failed reads)&#xA;0 + 0 secondary&#xA;0 + 0 supplementary&#xA;0 + 0 duplicates&#xA;31774629 + 0 mapped (98.04% : N/A)&#xA;32410094 + 0 paired in sequencing&#xA;16196814 + 0 read1&#xA;16213280 + 0 read2&#xA;30802808 + 0 properly paired (95.04% : N/A)&#xA;31561218 + 0 with itself and mate mapped&#xA;213411 + 0 singletons (0.66% : N/A)&#xA;89688 + 0 with mate mapped to a different chr&#xA;63838 + 0 with mate mapped to a different chr (mapQ&amp;gt;=5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Do i need to down-sample with respect to the IgG for control sample which has lot more reads at the same time Knockout sample 1 has less read. So is it good to go for further analysis or i need to down sample them ?&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion is really appreciated&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2020-11-20T09:28:26.933" Title="Chip-seq down-sampling problem" Tags="|chip-seq|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14872" PostTypeId="2" ParentId="14871" CreationDate="2020-11-20T09:28:26.933" Score="1" Body="&lt;p&gt;No, downsampling in general has no use as it reduces power. Peak callers such as &lt;code&gt;macs2&lt;/code&gt; will internally scale data to account for sequencing depth differences.&lt;/p&gt;&#xA;&lt;p&gt;The IgG itself is only useful for peak calling as it somewhat corrects for the unspecificity of the antibody to precipitate any kind of protein rather than the protein of interest. I hear people say, and personally agree with this by experience with my own experiments, that of course IgG pulls down notably fewer DNA that the target antibody, therefore the library requires notably PCR to produce enough material for sequencing. And this then makes it questionable how representative IgG is as a control. I personally prefer chromatin input to account for the intrinsic sequence/GC bias of the underlying genomic region in the sequencing/mapping. To my knowledge there are currently few established tools to really take these controls into account for downstream analysis such as differential testing via interaction terms because the library composition is simply so different from the IPs making normalization difficult.&lt;/p&gt;&#xA;&lt;p&gt;Your focus imho should be rather to properly normalize your samples to correct for differences is signal/noise ratio, see for example &lt;a href=&quot;https://www.biostars.org/p/413626/#414440&quot; rel=&quot;nofollow noreferrer&quot;&gt;my post over at biostars&lt;/a&gt; on how one could do that beyond naive per-million scaling.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-11-20T09:28:26.933" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14874" PostTypeId="2" ParentId="14852" CreationDate="2020-11-20T10:58:13.010" Score="1" Body="&lt;p&gt;First of all, those are not fasta sequences. Fasta is a specific format, and requires a &lt;code&gt;&amp;gt;&lt;/code&gt; followed by a sequence identifier on one line and the one or more lines of sequence.&lt;/p&gt;&#xA;&lt;p&gt;Assuming you really have two files in the format you show, you can do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk -F: '{ gsub(/ /,&quot;&quot;); if(NR==FNR){bbone[$2]; next} for(b in bbone){print b$&lt;/span&gt;2}}' file1 file2 &#xA;GTCATGCTACGTGCTACGTGCTCGTCGCGCCGCTAGTGTAGCTGCTACTCCG&#xA;ACTGCGTATGTCAGCATATAGACCGCCTCGCTAGTGTAGCTGCTACTCCG&#xA;CGTAGCTAGCTAGCTCGTGTCGTGATGCTACTCCGCTAGTGTAGCTGCTACTCCG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;gsub(/ /,&amp;quot;&amp;quot;);&lt;/code&gt; just removes the extra space you have in &lt;code&gt;file1&lt;/code&gt; after each &lt;code&gt;:&lt;/code&gt;. If you also want to have the sequence names as you show, use:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk -F: '{ gsub(/ /,&quot;&quot;); if(NR==FNR){bbone[$2]=&quot;Backbone&quot;NR; next} for(b in bbone){print bbone[b]&quot;+&quot;$1&quot;:&quot;, b$&lt;/span&gt;2}}' file1 file2 &#xA;Backbone3+sequence_1: GTCATGCTACGTGCTACGTGCTCGTCGCGCCGCTAGTGTAGCTGCTACTCCG&#xA;Backbone1+sequence_1: ACTGCGTATGTCAGCATATAGACCGCCTCGCTAGTGTAGCTGCTACTCCG&#xA;Backbone2+sequence_1: CGTAGCTAGCTAGCTCGTGTCGTGATGCTACTCCGCTAGTGTAGCTGCTACTCCG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you really need to do this in python for some reason, please edit your question and show us your python script so we have something to work on and expand. I really don't want to write a whole python program for something so trivial.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2020-11-20T11:18:28.540" LastActivityDate="2020-11-20T11:18:28.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14877" PostTypeId="1" CreationDate="2020-11-21T05:54:05.513" Score="1" ViewCount="731" Body="&lt;p&gt;I am using count_overlap() for counting the kmers from Biopython .&lt;/p&gt;&#xA;&lt;p&gt;Does it take the reverse complement of kmer into account?&lt;/p&gt;&#xA;&lt;p&gt;I need to count reverse complement too&lt;/p&gt;&#xA;" OwnerUserId="9912" LastEditorUserId="3848" LastEditDate="2020-12-06T15:08:24.383" LastActivityDate="2020-12-06T21:32:35.623" Title="How to count the kmer occurrence in FASTA file considering overlapping and reverse complement?" Tags="|python|covid-19|sars-cov-2|k-mer|seqio|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14878" PostTypeId="2" ParentId="14877" CreationDate="2020-11-21T10:08:44.393" Score="1" Body="&lt;p&gt;There's a lot detail missing here, but at a guess could you try,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;from Bio.Seq import Seq&#xA;for record in SeqIO.parse(&amp;quot;ls_orchid.gbk&amp;quot;, &amp;quot;genbank&amp;quot;):&#xA;    reversec = record.seq.reverse_complement())&#xA;    revcount = reversec.count_overlap('AT')&#xA;    print (revcount + count_overlap('AT'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Essentially &lt;code&gt;count_overlap&lt;/code&gt; only counts the forward strand if this is the Biopython method. Not debugged the above.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-11-21T10:19:32.927" LastActivityDate="2020-11-21T10:19:32.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14879" PostTypeId="2" ParentId="11548" CreationDate="2020-11-21T18:15:55.703" Score="1" Body="&lt;p&gt;I am having trouble finding anything that directly addresses the issue of failed collaborations leading to scooping, but here are some examples of similar behavior:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;it is sort of implicit in the discussion &lt;a href=&quot;https://drugmonkey.scientopia.org/2016/04/07/the-girl-who-cried-scoop/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;This question on &lt;a href=&quot;https://academia.stackexchange.com/questions/88219/how-do-i-address-my-research-publication-being-scooped&quot;&gt;academia SE&lt;/a&gt; is related but not exactly right.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There are some related stories &lt;a href=&quot;https://www.immpressmagazine.com/avoiding-the-scoop/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3967854/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is an article about plagiarism among collaborators&lt;/a&gt;, which is quite similar.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I feel like a lot of &lt;a href=&quot;https://advances.sciencemag.org/content/4/5/eaar2133&quot; rel=&quot;nofollow noreferrer&quot;&gt;the discourse around preprinting etc.&lt;/a&gt; is similar but again not quite there.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;As for punishment, that sounds hard. Scooping is sort of built into science; fighting over precedence is a big deal and has been for a long time. My solution would be to get rid of current academic publication and have everyone's open digital lab notebooks streaming results to publication feeds in real time; that way there's no ambiguity. Preprints and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5836261/&quot; rel=&quot;nofollow noreferrer&quot;&gt;micropublications&lt;/a&gt; are sort of a poor man's version of this.&lt;/p&gt;&#xA;&lt;p&gt;This is not a very complete answer but there may be enough there to help.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-11-21T18:15:55.703" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14882" PostTypeId="1" CreationDate="2020-11-22T01:17:29.283" Score="2" ViewCount="4013" Body="&lt;p&gt;As described by the &lt;a href=&quot;https://github.com/lh3/miniasm/blob/master/PAF.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;developer&lt;/a&gt; of &lt;a href=&quot;https://github.com/lh3/minimap2/blob/master/cookbook.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimap2&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;PAF is a text format describing the approximate mapping positions between two set of sequences.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;He also suggest to use either &lt;a href=&quot;https://genomeribbon.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ribbon&lt;/a&gt; to visualize PAF files, or to use &lt;code&gt;minidot&lt;/code&gt; from &lt;code&gt;miniasm&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;minimap2 -c -x??? asm1.fa asm2.fa &amp;gt; out.paf&#xA;minidot out.paf &amp;gt; out.eps &amp;amp;&amp;amp; epstopdf out.eps&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What does &lt;code&gt;-x???&lt;/code&gt; stand for? And how do you visualize PAF files? Could you think of a way to use &lt;a href=&quot;https://software.broadinstitute.org/software/igv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IGV&lt;/a&gt; to visualize PAF files?&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="1157" LastEditDate="2022-07-03T13:25:40.130" LastActivityDate="2022-07-03T13:25:40.130" Title="How do you visualize minimap2's PAF output format?" Tags="|minimap2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14884" PostTypeId="2" ParentId="14882" CreationDate="2020-11-22T01:42:50.470" Score="5" Body="&lt;p&gt;As described in the &lt;a href=&quot;https://lh3.github.io/minimap2/minimap2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimap2 manual&lt;/a&gt;, &lt;code&gt;-x STR&lt;/code&gt; is used to specify a preset option like &lt;code&gt;-x map-pb&lt;/code&gt; for &lt;code&gt;PacBio/Oxford Nanopore read to reference mapping (-Hk19)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In general, I use the &lt;code&gt;-a&lt;/code&gt; flag to output alignments in sam format that can then be  sorted and converted to bam format, and viewed in any genome browser. The support for PAF format when it comes to visualization appears to be limited. A few options that I have found, but not tested are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/thackl/minidot&quot; rel=&quot;nofollow noreferrer&quot;&gt;minidot&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/dwinter/pafr&quot; rel=&quot;nofollow noreferrer&quot;&gt;pafr&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/mbhall88/pafpy&quot; rel=&quot;nofollow noreferrer&quot;&gt;pafpy&lt;/a&gt; -- this one may not have the plotting capabilities though&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3577" LastEditorUserId="3577" LastEditDate="2020-11-22T19:11:36.577" LastActivityDate="2020-11-22T19:11:36.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14886" PostTypeId="1" CreationDate="2020-11-22T09:17:12.520" Score="1" ViewCount="129" Body="&lt;p&gt;I am currently working on a network pharmacology project where I am supposed to find out the possible diseases our drug might have an effect on. I am not a bioinformatics student, so my knowledge in the subject matter is very limited. I have all the targets and I am supposed to get the pathways associated with the targets. I used &lt;code&gt;STRING&lt;/code&gt; for the purpose.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I got both &lt;code&gt;KEGG&lt;/code&gt; functional annotation and enrichment for my list of targets.&lt;/li&gt;&#xA;&lt;li&gt;I have to build a two-mode network&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; I am not very clear about&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the difference between enrichment and functional annotation and&lt;/li&gt;&#xA;&lt;li&gt;which data should be used for the next step.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9915" LastEditorUserId="3848" LastEditDate="2020-11-22T09:37:44.303" LastActivityDate="2020-12-23T05:35:40.000" Title="Functional Annotation vs Functional Enrichment--which one should I use for Network Analysis?" Tags="|proteins|networks|go-enrichment|functional-annotation|kegg|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14887" PostTypeId="1" CreationDate="2020-11-22T09:40:10.677" Score="2" ViewCount="88" Body="&lt;p&gt;I’m now not sure what I need to alter in my PDB to get it to work in leap. I know breaking the bonds turns the formerly connected residues into terminal residues.&#xA;It keeps saying that 3 of my atoms no longer have a type , how can I fix this ?&lt;/p&gt;&#xA;" OwnerDisplayName="user9855" LastActivityDate="2020-11-22T10:58:47.313" Title="I have used TER to break the long bonds of a chain in my PDB" Tags="|rna|pdb|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14890" PostTypeId="2" ParentId="14887" CreationDate="2020-11-22T10:52:22.933" Score="2" Body="&lt;p&gt;Normally you have three options in a PDB:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Add a &lt;code&gt;TER&lt;/code&gt; record&lt;/li&gt;&#xA;&lt;li&gt;Add the &lt;code&gt; OXT&lt;/code&gt; atom, if your chain was cleaved or cleaved spontaneously (&lt;em&gt;i.e.&lt;/em&gt; Asp-Pro sequence), the C-terminal would sill have a terminal oxygen. i.e. making it a terminal residue. If you have protons, you'll need to add also &lt;code&gt;3H &lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Assign a different chain&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If you are using forcefields as is your case, the second is the only choice. The &lt;code&gt;C &lt;/code&gt; atom and the &lt;code&gt;N &lt;/code&gt; atoms have a type that does not match because they have the wrong number of bonds —hence the type error.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-11-22T10:58:47.313" LastActivityDate="2020-11-22T10:58:47.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14891" PostTypeId="1" CreationDate="2020-11-22T11:08:22.583" Score="0" ViewCount="521" Body="&lt;p&gt;I have a bam file with Unique Molecular Identifiers (UMIs) for each read present on the RX tag ( such as &lt;code&gt;RX:Z:TGAGAAGGG&lt;/code&gt;), as expected by picard and fgbio tools.&lt;/p&gt;&#xA;&lt;p&gt;However, several other tools (such as UMI-tools) expect the UMIs to be present at the end of the read name, such as (&lt;code&gt;NB501943:73:HK333BGXC:3:12402:20313:16437_TGAGAAGGG&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know of a tool to do this?&#xA;I have an &lt;code&gt;awk&lt;/code&gt; solution which I'll self-answer with, but I don't know whether that is robust.&lt;/p&gt;&#xA;" OwnerUserId="2155" LastEditorUserId="3694" LastEditDate="2020-11-23T12:07:13.433" LastActivityDate="2021-04-22T15:40:08.733" Title="Moving UMI from tag into read name" Tags="|sam|umi|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14892" PostTypeId="2" ParentId="14891" CreationDate="2020-11-22T11:08:22.583" Score="1" Body="&lt;p&gt;My current solution is (&lt;code&gt;awk&lt;/code&gt; is not a tool I'm very familiar with, so this may not be good &lt;code&gt;awk&lt;/code&gt; code, there is presumably a way to get it to print the header lines unmodified.):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -H temp.sam &amp;gt; temp2.sam&#xA;samtools view temp.sam | &#xA;awk '{OFS = &amp;quot;\t&amp;quot;} {for(i=1;i&amp;lt;=NF;i++)  if (&lt;span class=&quot;math-container&quot;&gt;$i ~ /^RX:Z:/) {$1=$1&quot;_&quot;$i; gsub(&quot;RX:Z:&quot;,&quot;&quot;,$&lt;/span&gt;1); print}}' &amp;gt;&amp;gt; temp2.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This seems to work when I test it, but I don't know if I'm missing anything.&lt;/p&gt;&#xA;" OwnerUserId="2155" LastEditorUserId="2155" LastEditDate="2020-11-23T11:35:37.933" LastActivityDate="2020-11-23T11:35:37.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14893" PostTypeId="1" AcceptedAnswerId="14897" CreationDate="2020-11-22T11:55:36.223" Score="2" ViewCount="390" Body="&lt;p&gt;Hello to the entire Stackoverflow community!&#xA;I'm writing to you because I'm currently building a phylogeny with ggtree and I have in parallel a table like this one:&#xA;&lt;strong&gt;df&lt;/strong&gt; :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Node_number Number &#xA;50   31&#xA;1    15&#xA;13   25&#xA;35   12&#xA;40   4&#xA;5    2 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and a tree such as :&#xA;&lt;a href=&quot;https://i.stack.imgur.com/UhkFw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UhkFw.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and the idea is that I woule like to incorporate the df$Number information in each Node in the tree and get somethhing like I put in the picture (expected part)&lt;/p&gt;&#xA;&lt;p&gt;tree data :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(edge = structure(c(31L, 32L, 33L, 34L, 34L, 35L, &#xA;35L, 33L, 32L, 36L, 37L, 37L, 36L, 38L, 39L, 40L, 40L, 39L, 41L, &#xA;42L, 42L, 43L, 43L, 41L, 44L, 44L, 45L, 45L, 46L, 46L, 47L, 48L, &#xA;48L, 47L, 38L, 49L, 50L, 51L, 51L, 50L, 52L, 53L, 53L, 54L, 54L, &#xA;52L, 49L, 55L, 55L, 31L, 56L, 57L, 57L, 58L, 58L, 56L, 59L, 59L, &#xA;32L, 33L, 34L, 1L, 35L, 2L, 3L, 4L, 36L, 37L, 5L, 6L, 38L, 39L, &#xA;40L, 7L, 8L, 41L, 42L, 9L, 43L, 10L, 11L, 44L, 12L, 45L, 13L, &#xA;46L, 14L, 47L, 48L, 15L, 16L, 17L, 49L, 50L, 51L, 18L, 19L, 52L, &#xA;53L, 20L, 54L, 21L, 22L, 23L, 55L, 24L, 25L, 56L, 57L, 26L, 58L, &#xA;27L, 28L, 59L, 29L, 30L), .Dim = c(58L, 2L)), tip.label = c(&amp;quot;t5&amp;quot;, &#xA;&amp;quot;t9&amp;quot;, &amp;quot;t25&amp;quot;, &amp;quot;t14&amp;quot;, &amp;quot;t30&amp;quot;, &amp;quot;t26&amp;quot;, &amp;quot;t2&amp;quot;, &amp;quot;t10&amp;quot;, &amp;quot;t12&amp;quot;, &amp;quot;t15&amp;quot;, &#xA;&amp;quot;t1&amp;quot;, &amp;quot;t3&amp;quot;, &amp;quot;t6&amp;quot;, &amp;quot;t23&amp;quot;, &amp;quot;t17&amp;quot;, &amp;quot;t18&amp;quot;, &amp;quot;t4&amp;quot;, &amp;quot;t22&amp;quot;, &amp;quot;t27&amp;quot;, &amp;quot;t16&amp;quot;, &#xA;&amp;quot;t29&amp;quot;, &amp;quot;t7&amp;quot;, &amp;quot;t21&amp;quot;, &amp;quot;t20&amp;quot;, &amp;quot;t13&amp;quot;, &amp;quot;t11&amp;quot;, &amp;quot;t19&amp;quot;, &amp;quot;t24&amp;quot;, &amp;quot;t8&amp;quot;, &#xA;&amp;quot;t28&amp;quot;), edge.length = c(0.912875924259424, 0.293603372760117, &#xA;0.459065726259723, 0.332394674187526, 0.65087046707049, 0.258016780717298, &#xA;0.478545248275623, 0.766310670645908, 0.0842469143681228, 0.875321330036968, &#xA;0.339072937844321, 0.839440350187942, 0.34668348915875, 0.333774930797517, &#xA;0.476351245073602, 0.892198335845023, 0.864339470630512, 0.389989543473348, &#xA;0.777320698834956, 0.960617997217923, 0.434659484773874, 0.712514678714797, &#xA;0.399994368897751, 0.325352151878178, 0.757087148027495, 0.202692255144939, &#xA;0.711121222469956, 0.121691921027377, 0.245488513959572, 0.14330437942408, &#xA;0.239629415096715, 0.0589343772735447, 0.642288258532062, 0.876269212691113, &#xA;0.778914677444845, 0.79730882588774, 0.455274453619495, 0.410084082046524, &#xA;0.810870242770761, 0.604933290276676, 0.654723928077146, 0.353197271935642, &#xA;0.270260145887733, 0.99268406117335, 0.633493264438584, 0.213208135217428, &#xA;0.129372348077595, 0.478118034312502, 0.924074469832703, 0.59876096714288, &#xA;0.976170694921166, 0.731792511884123, 0.356726912083104, 0.431473690550774, &#xA;0.148211560677737, 0.0130775754805654, 0.715566066093743, 0.103184235747904&#xA;), Nnode = 29L), class = &amp;quot;phylo&amp;quot;, order = &amp;quot;cladewise&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;does someone have an idea ?&lt;/p&gt;&#xA;" OwnerUserId="9104" LastActivityDate="2020-11-22T23:01:42.813" Title="Plot information stored in dataframe within a tree (ggtree)" Tags="|r|phylogenetics|phylogeny|ggplot2|ggtree|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14895" PostTypeId="1" CreationDate="2020-11-22T21:31:15.100" Score="1" ViewCount="86" Body="&lt;p&gt;I am looking for publicly available data for a genomics deep learning project. My goal is to compare different architectures to predict biological insights from DNA sequences.&lt;/p&gt;&#xA;&lt;p&gt;I have heard about &lt;a href=&quot;https://janggu.readthedocs.io/en/latest/tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Janggu&lt;/a&gt; before, but I am looking for a few datasets which are like the &lt;a href=&quot;https://en.m.wikipedia.org/wiki/MNIST_database&quot; rel=&quot;nofollow noreferrer&quot;&gt;MNIST&lt;/a&gt; equivalents in genomics. &lt;a href=&quot;https://www.kaggle.com/tags/human-genetics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kaggle&lt;/a&gt; doesn't seem to have many such datasets, and I would be happy if you share your insights about potential datasets that one could use.&lt;/p&gt;&#xA;&lt;h2&gt;For example, it could be datasets that map 100-Mers to their ChIP-seq values or datasets that annotate a DNA sequence with the probability that a transcription factor would bind to it.&lt;/h2&gt;&#xA;&lt;p&gt;Choong and Lee lists a few &lt;a href=&quot;https://www.biorxiv.org/content/biorxiv/early/2017/10/25/186965.full.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;datasets&lt;/a&gt; on their IEEE paper from 2017:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Hosg5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Hosg5.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="3848" LastEditDate="2023-04-28T20:14:15.667" LastActivityDate="2023-04-28T20:14:15.667" Title="What datasets are available out there for prediction based on DNA sequences?" Tags="|phylogenetics|phylogeny|deep-learning|tensorflow|pytorch|" AnswerCount="1" CommentCount="5" ClosedDate="2020-11-24T13:07:23.267" ContentLicense="CC BY-SA 4.0" />
  <row Id="14896" PostTypeId="2" ParentId="14895" CreationDate="2020-11-22T22:02:21.540" Score="2" Body="&lt;p&gt;Good question and the Janggu link is cool. Thanks!&lt;/p&gt;&#xA;&lt;p&gt;Kaggle is/was heavily image recognition. There isn't an MNIST type data set in genomics, in case anyone is wondering this is a classic natural language processing (NLP) type analysis to identify handwritten single digit numbers.&lt;/p&gt;&#xA;&lt;p&gt;There has been several FDA precision challenges, one being described here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://precision.fda.gov/challenges/3/view/results&quot; rel=&quot;nofollow noreferrer&quot;&gt;FDA precision challenges&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This could provide something towards a 'standard'.&lt;/p&gt;&#xA;&lt;p&gt;A key issue remains the accurate vectorisation of sequence and whether you think that is/was satisfactory in my personal opinion.&lt;/p&gt;&#xA;&lt;p&gt;There is an old review article &lt;a href=&quot;https://arxiv.org/abs/1802.00810&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-11-22T22:02:21.540" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14897" PostTypeId="2" ParentId="14893" CreationDate="2020-11-22T22:56:07.653" Score="4" Body="&lt;p&gt;You can accomplish this using the ggtree package available on &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/ggtree.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioconductor&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;First you will need to combine your tree with the data.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse)&#xA;library(ggtree)&#xA;&#xA;ftree &amp;lt;- tree$edge %&amp;gt;%&#xA;   as_tibble() %&amp;gt;%&#xA;   mutate(Node_number = 1:n()) %&amp;gt;% # finds edge numbering&#xA;   right_join(data, by = &amp;quot;Node_number&amp;quot;) %&amp;gt;% # find internal node associated with your node numbering &#xA;   left_join(fortify(tree, ladderize = FALSE), ., by = c(&amp;quot;node&amp;quot; = &amp;quot;V2&amp;quot;)) # merge tree and data&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that this will produce a warning, but it does work.&lt;/p&gt;&#xA;&lt;p&gt;Then plot the tree using ggtree and its &lt;code&gt;geom_nodepoint&lt;/code&gt; layer.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ggtree(ftree, ladderize = FALSE) + # plots tree&#xA;   geom_tiplab() + # adds tip labels&#xA;   geom_nodepoint(aes(subset = !is.na(Number), size = Number, x = branch)) + # adds your data as points&#xA;   scale_size_continuous(range = c(4, 12)) # scales point sizes&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Nncrw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Nncrw.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can adjust the size of the points with the &lt;code&gt;scale_size_continuous&lt;/code&gt; function.&lt;/p&gt;&#xA;&lt;p&gt;Data&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data &amp;lt;- read_csv('Node_number,Number &#xA;50,31&#xA;1,15&#xA;13,25&#xA;35,12&#xA;40,4&#xA;5,2')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and &lt;code&gt;tree&lt;/code&gt; is the same as you provided.&lt;/p&gt;&#xA;&lt;p&gt;Ggtree also has a decent manual available at &lt;a href=&quot;https://yulab-smu.top/treedata-book/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://yulab-smu.top/treedata-book/&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1028" LastEditorUserId="1028" LastEditDate="2020-11-22T23:01:42.813" LastActivityDate="2020-11-22T23:01:42.813" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14898" PostTypeId="2" ParentId="14877" CreationDate="2020-11-23T01:00:22.850" Score="0" Body="&lt;p&gt;I would NOT recommend using biopython for a task like this. First, it's slow, but also it's really not necessary. There are several other existing tools optimised for exactly this purpose.&lt;/p&gt;&#xA;&lt;p&gt;My favourite is &lt;a href=&quot;https://github.com/refresh-bio/KMC&quot; rel=&quot;nofollow noreferrer&quot;&gt;KMC&lt;/a&gt;. It's very fast, and by default, it reports canonical kmers (Parameter &lt;code&gt;-b&lt;/code&gt; would turn this off). So, assuming you have KMC installed, this is how you get all the kmers from a range of occurances (1 - 10000).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# just making a directory for temprary files&#xA;mkdir tmp&#xA;&#xA;# building a kmer database with k = 21, &#xA;# 16 cores and &#xA;# up to 64GB of memory. &#xA;# Counting occurances between 1 to 10000, &#xA;# `-fa` specifies that the input is fasta, you can also input other formats&#xA;# the database name will be kmer_counts&#xA;# &amp;amp; saving temporary data to tmp directory; &#xA;kmc -k21 -t16 -m64 -ci1 -cs10000 -fa my_seq.fasta kmer_counts tmp # run kmc&#xA;&#xA;# extract kmers from the database in a text file&#xA;kmc_dump -ci1 -cx10000 kmer_counts kmer_k21.dump&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="57" LastActivityDate="2020-11-23T01:00:22.850" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14899" PostTypeId="1" CreationDate="2020-11-23T11:25:41.430" Score="1" ViewCount="66" Body="&lt;p&gt;I'am comparing bats coronavirus with Sars2, i.e. COVID-19. I have one question...&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;one Bat Sars coronaviurs identify in GenBank as DQ071615.1 has the ORF13(28122..28415) and ORF14(28572..28784) proteins&lt;/li&gt;&#xA;&lt;li&gt;Sars2 identify in GenBank as MN908947_3, has range 28284..28577&#xA;not labeled as ORF13 and the same 28734..28955 is not labeled as ORF14?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The ORF13(28122..28415) of Bat coronavirus and  PSEUDO_ORF13(28284..28577) of sars2 share 72% homology&#xA;and ORF14(28572..28784) of Bat coronavirus and  PSEUDO_ORF14(28734..28955) of sars2 share 75% homology&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;This seems one publication for ORF14 and ORF13 in Bat Viruses and relations in Sars2 &lt;a href=&quot;https://www.hindawi.com/journals/bmri/2020/7234961/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The author say that ORF14 in Sars2 is all contained in N protein...&#xA;And ORF13 is know in that above link as ORF9B&#xA;and in that written is said ORF9B is implied in immunitary innate suppression...&lt;/p&gt;&#xA;" OwnerUserId="7054" LastEditorUserId="3848" LastEditDate="2020-12-03T11:39:09.293" LastActivityDate="2020-12-03T11:39:09.293" Title="Why ORF13 and ORF14 of Bat Sars coronavirus Rp3 have no corrispondence in Sars2?" Tags="|covid-19|sars-cov-2|coronavirus|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14900" PostTypeId="2" ParentId="14849" CreationDate="2020-11-23T11:33:33.167" Score="2" Body="&lt;p&gt;A contact map predictor will usually output the probability of a contact for every residue pair in a protein, often ommitting very close pairs (e.g. less than 6 residue separation).&lt;/p&gt;&#xA;&lt;p&gt;To calculate the L/n score, rank the predicted contacts descending by probability and look at the first L/n contacts. The fraction of these that are actually contacts in the native structure is the L/n contact precision score.&lt;/p&gt;&#xA;&lt;p&gt;Often, you would filter the list to look at only contacts of a certain sequence separation, e.g. long range contacts between residues of &amp;gt;23 separation (definitions vary between papers). If you have a good predictor this score is high for L/10 and falls as you go to L/1, which is expected as proteins don't usually have that many long range contacts to find.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2020-11-23T11:33:33.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14901" PostTypeId="1" AcceptedAnswerId="14907" CreationDate="2020-11-23T12:50:37.587" Score="2" ViewCount="367" Body="&lt;p&gt;I made a virtual screening of a large database (~5M) and want to filter ligand structures for further more accurate screening (from Schrodinger HTVS to SP resolution). My next step is to define a cutoff docking score, but it seems arbitrary to choose one. Are there any established methods for filtering the ligands out? &lt;a href=&quot;https://i.stack.imgur.com/SredH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SredH.png&quot; alt=&quot;distribution of Schrodinger gscore values for 4 different conformations of a protein structure&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7150" LastActivityDate="2020-11-24T09:52:48.580" Title="How to choose docking score cutoff?" Tags="|protein-structure|docking|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14902" PostTypeId="2" ParentId="11345" CreationDate="2020-11-23T16:01:58.590" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://edinburgh-genome-foundry.github.io/DnaChisel/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DNA Chisel&lt;/a&gt; (written in Python) can &lt;a href=&quot;https://edinburgh-genome-foundry.github.io/DnaChisel/ref/biotools.html?highlight=reverse_translate#dnachisel.biotools.reverse_translate&quot; rel=&quot;nofollow noreferrer&quot;&gt;reverse translate&lt;/a&gt; a protein sequence:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import dnachisel&#xA;from dnachisel.biotools import reverse_translate&#xA;&#xA;record = dnachisel.load_record(&amp;quot;seq.fa&amp;quot;)&#xA;reverse_translate(str(record.seq))&#xA;# GGTCATATTTTAAAAATGTTTCCT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="208" LastActivityDate="2020-11-23T16:01:58.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14903" PostTypeId="1" AcceptedAnswerId="14904" CreationDate="2020-11-23T16:41:21.660" Score="1" ViewCount="503" Body="&lt;p&gt;How can I convert a sequence provided in a Microsoft doc file into a fasta or genbank format?&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2022-06-27T13:33:30.960" Title="Convert sequence in MS doc to fasta or genbank" Tags="|fasta|format-conversion|genbank|egf|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14904" PostTypeId="2" ParentId="14903" CreationDate="2020-11-23T16:41:21.660" Score="2" Body="&lt;p&gt;The &lt;a href=&quot;https://github.com/Edinburgh-Genome-Foundry/crazydoc&quot; rel=&quot;nofollow noreferrer&quot;&gt;CrazyDoc&lt;/a&gt; Python package can convert Genbank/Fasta/Snapgene/MSDoc sequences into Biopython records, which can be saved as Genbank or Fasta.&lt;/p&gt;&#xA;&lt;p&gt;A web interface / demo is provided at EGF CUBA (Collection&#xA;of Useful Biological Apps): &lt;a href=&quot;https://cuba.genomefoundry.org/convert_sequence_files&quot; rel=&quot;nofollow noreferrer&quot;&gt;Convert Sequence Files&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Disclaimer: I'm the current maintainer of CrazyDoc.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2020-11-23T16:41:21.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14905" PostTypeId="5" CreationDate="2020-11-23T16:46:45.957" Score="0" Body="&lt;p&gt;EGF Codons (&lt;a href=&quot;https://edinburgh-genome-foundry.github.io&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://edinburgh-genome-foundry.github.io&lt;/a&gt;) is a collection of packages and web applications for DNA design and assembly, lab automation, and more.&lt;/p&gt;&#xA;" OwnerUserId="208" LastEditorUserId="208" LastEditDate="2020-11-23T17:49:36.133" LastActivityDate="2020-11-23T17:49:36.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14906" PostTypeId="4" CreationDate="2020-11-23T16:46:45.957" Score="0" Body="for questions related to EGF Codons, the software suite of the Edinburgh Genome Foundry" OwnerUserId="208" LastEditorUserId="208" LastEditDate="2020-11-23T17:49:40.550" LastActivityDate="2020-11-23T17:49:40.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14907" PostTypeId="2" ParentId="14901" CreationDate="2020-11-23T16:53:05.380" Score="3" Body="&lt;p&gt;It is utterly arbitrary and depends on your resources.&lt;/p&gt;&#xA;&lt;p&gt;We can all agree your cutoff would be a lot lower during the ordering step than at followup &lt;em&gt;in silico&lt;/em&gt; analyses, but even then if you were ordering the top &lt;em&gt;N&lt;/em&gt; at \$100-250 each, should you spend \$1,000 or \$10,000?&lt;/p&gt;&#xA;&lt;p&gt;In the &lt;em&gt;in silico&lt;/em&gt; step you have real time and CPU-time to factor in making this call even blurrier.&lt;/p&gt;&#xA;&lt;p&gt;But there are some things to put those scores into context:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.schrodinger.com/kb/1015&quot; rel=&quot;nofollow noreferrer&quot;&gt;The units are kcal/mol&lt;/a&gt;, i.e. these are predicted ∆∆G of binding. &lt;a href=&quot;https://michelanglo.sgc.ox.ac.uk/docs/venus_energetics&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is an explanation of what does a kcal/mol mean for docking&lt;/a&gt; — 1 kcal/mol is a hydrogen bond or the average collision energy of water at 37°C.&lt;/li&gt;&#xA;&lt;li&gt;If you have a crystal structure with a known ligand, re-docking and cross-docking that would give you a benchmark of what you should expect ——keeping my mind that natural ligand are horrid and don't obey Lipinski's rule. The natural ligand will wander and to assess how good the docking went RMSD is used, but for this I mean looking at its GlideScore as if it were any other compound.&lt;/li&gt;&#xA;&lt;li&gt;Rescore with a different tool —Gold etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;Ligand efficiency&lt;/h2&gt;&#xA;&lt;p&gt;An important thing to remember is that ∆∆G is size dependent —a huge peptidomimetic will score more than a smaller molecule that hits the hotspots perfectly. Therefore, ligand efficiency (LE), ∆∆G over heavy atoms (&lt;a href=&quot;https://jcheminf.biomedcentral.com/articles/10.1186/s13321-019-0330-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;reviewed in detail here&lt;/a&gt;) is a good metric to prevent huge molecules dominating spuriously. If your library has a variety of different sizes, it is best to sort by LE than ∆∆G alone. In fact, if you get a few small hits you can merge/link them or expand on them, whereas if you get nothing because you gambled on larger hits you have to start from square one. I would argue that this is true for repurposing screens and the low success of repurposing VLS despite the enthusiasm is due to the neglect of this.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-11-24T09:52:48.580" LastActivityDate="2020-11-24T09:52:48.580" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14908" PostTypeId="1" AcceptedAnswerId="14911" CreationDate="2020-11-23T18:48:24.870" Score="0" ViewCount="146" Body="&lt;p&gt;I am working on a project and am having issues with the following code that I have written in &lt;code&gt;nano&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;import sys&#xA;import re &#xA;&#xA;&#xA;    fasta_file = (sys.argv[1])&#xA;    for myfile in SeqIO.parse(fasta_file, &amp;quot;fasta&amp;quot;):&#xA;      if len(myfile) &amp;gt; 250:&#xA;       gene_id = myfile.id&#xA;       mylist = re.match('H149xcV\_\w+\_\w+\_\w+',gene_id)&#xA;       print (&amp;quot;&amp;gt;&amp;quot;+list.group(1)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the error I receive when I execute my command on &lt;code&gt;command-line&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;File &amp;quot;mpo.py&amp;quot;, line 7, in &amp;lt;module&amp;gt;&#xA;    gene_id = myfile.id&#xA;NameError: name 'myfile' is not defined&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have a fasta file with the format&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt;H149xcV_Fge342_r3_h2_d1 len=210 path=[0:0-206]&#xA;ACTATACATGAGGAGAACATAGAACAAAAATGGGACCATAGATATATAACAATAGAAGATATAGAGAACACAATAGACAACTTATTAGGAAAGAGGTGTGTCGTCATGGAGCTGATGTTCGAGGATACTTTGCATGGTCATTCTTGGATAATTTTGAGTGGGCTATGGGATACACCAAGAGGTTTGGCATTGTTTATGTTGATTATAAGAACGGGC&#xA;&#xA;&amp;gt;H149xcV_ytR1oP_r3_h2_d1 len=306 path=[0:0-207]&#xA;ATTAGAGTCTGAGAGAGTCTTGATTTGTCGTCGTCGAGAAATATAGGAGATCTGATTAGAGGAGAGAGCGGCCTAGGCGATGCGCGATATAGCGCTATATAGGCCTAGAGGAGAGTCTCTCTCTTTTAGAAGAGATAATATATATATATATATGGCTCTCCGGCGGGGCCGCGCGAGAGCTCGATCGATCGATATTAGCTGTACGATGCTAGCTAGCTTATATTCGATCGATTATAGCTTAGATCTCTCTCTAAAGGTCGATATCGCTTATGCGCGCGTATATCG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I wish to reformat my file so that it only provides me with the unique gene id's for genes with a length greater than 250 bp.&lt;/p&gt;&#xA;&lt;p&gt;My desired output is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;H149xcV_Fge342_r3_h2&#xA;&amp;gt;H149xcV_ytR1oP_r3_h2&#xA;&amp;gt;H149xcV_DPN78333_r3_h2&#xA;&amp;gt;H149xcV_AgV472_r3_h2&#xA;&amp;gt;H149xcV_DNP733_r3_h2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9868" LastEditorUserId="3848" LastEditDate="2023-04-07T15:18:05.733" LastActivityDate="2023-04-07T15:18:05.733" Title="How to fix NameError?" Tags="|python|phylogenetics|fasta|phylogeny|seqio|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14910" PostTypeId="2" ParentId="14908" CreationDate="2020-11-23T19:08:16.690" Score="3" Body="&lt;p&gt;You wrote,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mylist = re.match(H149xcV\_\w+\_\w+\_\w+)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;try...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;myregex = re.compile(r'H149xcV\_\w+\_\w+\_\w+')&#xA;mylist = myregex.match(myfile.id)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mylist = re.match(r'H149xcV\_\w+\_\w+\_\w+',myfile.id)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Watch your indentation it should be 4 spaces. Note you are losing the last '_\w+', but don't think it matters.&lt;/p&gt;&#xA;&lt;p&gt;I think it is better to use,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mylist, _, _ = re.split(r' ', myfile.id)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The problem with your code is &lt;code&gt;list&lt;/code&gt; will just return &lt;code&gt;true&lt;/code&gt; ... and that is not what you want. If you are using &lt;code&gt;split&lt;/code&gt; carefully watch the number of ' ' in your code you can also use &lt;code&gt;r'\s'&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can also set maxsplit as a parameter, which might be useful.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;code&gt;myfile&lt;/code&gt; not defined.&lt;/p&gt;&#xA;&lt;p&gt;Please try,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fasta_file = (sys.argv[0])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Generally your argv command isn't cool and not just [0] versus [1]. I forget the argv module everyone uses .. there's two of them, they are a bit fiddle but are very cool of options. You way in is okay, okayish, sort of okay ... Anyway you appear to be requesting the second input into the commandline, wherea you are probably placing the fasta file in the first position.&lt;/p&gt;&#xA;&lt;p&gt;Also&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print (fasta_file)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Check something happens that you picked up the file.&lt;/p&gt;&#xA;&lt;p&gt;The other way in is,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mypath = '/User/username/location/fasta.fa'&#xA;for myfile in SeqIO.parse(mypath, &amp;quot;fasta&amp;quot;):&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where &lt;code&gt;mypath&lt;/code&gt; is a OS X style, you're probably using Linux so its... /home/username/pathtofile/fastafile.fa (something like that) or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for myfile in SeqIO.parse('/User/username/location/fasta.fa', &amp;quot;fasta&amp;quot;):&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I forgot .... don't ever use &lt;code&gt;list&lt;/code&gt; as a variable, by &lt;code&gt;mylist&lt;/code&gt; is fine. Again I've not debugged my code.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-11-23T20:46:42.260" LastActivityDate="2020-11-23T20:46:42.260" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14911" PostTypeId="2" ParentId="14908" CreationDate="2020-11-23T20:28:07.413" Score="1" Body="&lt;p&gt;You might consider changing the code to make the whitespace a little more consistent (as it was, it failed when i tried to run it).&lt;/p&gt;&#xA;&lt;p&gt;Also, why are you using regular expressions? You have already extracted the &lt;code&gt;gene_id&lt;/code&gt; using Biopython. The following works for me:&lt;/p&gt;&#xA;&lt;p&gt;(updated to handle the underscore delimiting)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;import sys&#xA;import re &#xA;&#xA;fasta_file = (sys.argv[1])&#xA;for myfile in SeqIO.parse(fasta_file, &amp;quot;fasta&amp;quot;):&#xA;    if len(myfile.seq) &amp;gt; 250:  # note .seq here!&#xA;        gene_id = myfile.id&#xA;        # unclear why using regex&#xA;        #match_list = re.match('H149xcV\_\w+\_\w+\_\w+',gene_id)&#xA;        #print (&amp;quot;&amp;gt;&amp;quot;+match_list.group(0)) &#xA;        print(&amp;quot;_&amp;quot;.join(gene_id.split(&amp;quot;_&amp;quot;)[0:-1]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2020-11-23T22:50:26.163" LastActivityDate="2020-11-23T22:50:26.163" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="14912" PostTypeId="2" ParentId="14899" CreationDate="2020-11-23T21:35:31.473" Score="1" Body="&lt;p&gt;The virus Rp3 stands for Rhi pea isolated in China in 2004.&lt;/p&gt;&#xA;&lt;p&gt;The trees is &lt;a href=&quot;https://jvi.asm.org/content/84/21/11336&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and groups with the Bulgarian sarbecoviruses and is closely related to the African sarbecoviruses. Thus although it is a Chinese sarbecovirus is not with the &amp;quot;Chinease originating&amp;quot; clade as is SARS and Covid-19.&lt;/p&gt;&#xA;&lt;p&gt;Thats why the more mysterious ORFs ORF13 and 14 are missing, because over notable genetic distances these genes along with ORF10 are present/absent. This is a common feature of the betacoronaviruses.&lt;/p&gt;&#xA;&lt;p&gt;In summmary this virus is notably distance from the major human pathogenic SARS/covid-19 viruses. Thus the absence of these auxilary genes isn't really surprising.&lt;/p&gt;&#xA;&lt;p&gt;Its the very last taxa on the SARS-related viruses clade (monophyly) and is in a different group to SARS, which is the Beta-1 clade on this most excellent paper.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To answer the comment. They are outgroups, basically ancestral to COVID-19 and SARS-1 clade. So they lost these genes whilst the ingroup including COVID-109 and SARS-1 retained ORF13, 14 ... but I think lost ORF10 from memory&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/L2tCm.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/L2tCm.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-11-23T22:39:37.700" LastActivityDate="2020-11-23T22:39:37.700" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14913" PostTypeId="1" AcceptedAnswerId="14914" CreationDate="2020-11-23T21:57:35.947" Score="0" ViewCount="59" Body="&lt;p&gt;I am working on a project using the following command within &lt;code&gt;nano&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;import sys&#xA;import re &#xA;&#xA;&#xA;    fasta_file = (sys.argv[1])&#xA;    for myfile in SeqIO.parse(fasta_file, &amp;quot;fasta&amp;quot;):&#xA;      if len(myfile) &amp;gt; 250:&#xA;       gene_id = myfile.id&#xA;       mylist = re.match(r&amp;quot;H149xcV_\w+_\w+_\w+&amp;quot;, gene_id)&#xA;       print (&amp;quot;&amp;gt;&amp;quot;+mylist.group(0))   # edited from list to mylist&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and its providing with the following outout:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;H149xcV_Fge342_r3_h2_d1&#xA;&amp;gt;H149xcV_bTr423_r3_h2_d1&#xA;&amp;gt;H149xcV_kN893_r3_h2_d1&#xA;&amp;gt;H149xcV_DNp021_r3_h2_d1&#xA;&amp;gt;H149xcV_JEP3324_r3_h2_d1&#xA;&amp;gt;H149xcV_SRt424234_r3_h2_d1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I change my command so that it provides me with the desired format and only the &lt;strong&gt;UNIQUE&lt;/strong&gt; gene id's:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;H149xcV_Fge342_r3_h2&#xA;&amp;gt;H149xcV_bTr423_r3_h2&#xA;&amp;gt;H149xcV_kN893_r3_h2&#xA;&amp;gt;H149xcV_DNp021_r3_h2&#xA;&amp;gt;H149xcV_JEP3324_r3_h2&#xA;&amp;gt;H149xcV_SRt424234_r3_h2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9868" LastEditorUserId="3848" LastEditDate="2023-05-19T22:57:58.947" LastActivityDate="2023-05-19T22:57:58.947" Title="How to obtain desired output?" Tags="|python|phylogenetics|fasta|phylogeny|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14914" PostTypeId="2" ParentId="14913" CreationDate="2020-11-23T22:18:54.220" Score="2" Body="&lt;p&gt;its easy replace &lt;code&gt;match&lt;/code&gt; with &lt;code&gt;sub&lt;/code&gt; but please stop using &lt;code&gt;list&lt;/code&gt; as a variable ... mylist is fine.&lt;/p&gt;&#xA;&lt;p&gt;This might work&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   mylist = re.sub(r'H149xcV_\w+_\w+_\w+', gene_id)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;otherwise&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;myregex = re.compile('_\w+\s+.*') &#xA;fastaid = myregex.sub('', myfile)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;OR from @MaximilianPress&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;myregex2 = re.compile('_\w+\n') # or myregex2 = re.compile('_\w+$')&#xA;fastaid2 = myregex.sub('\n', myfile) # or fasaid2 = myregex.sub('', myfile)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above will work ... As with all my code I never proof it ..&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-11-23T22:57:57.713" LastActivityDate="2020-11-23T22:57:57.713" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14915" PostTypeId="1" CreationDate="2020-11-23T23:24:54.000" Score="1" ViewCount="393" Body="&lt;p&gt;I am using the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;import sys&#xA;import re &#xA;&#xA;&#xA;fasta_file = (sys.argv[1])&#xA;for myfile in SeqIO.parse(fasta_file, &amp;quot;fasta&amp;quot;):&#xA;if len(myfile.seq) &amp;gt; 250:&#xA;    gene_id = myfile.id&#xA;    mylist = re.match(r&amp;quot;H149xcV_[^\W_]+_[^\W_]+_[^\W_])_[^\W_]+&amp;quot;, gene_id)&#xA;    print (&amp;quot;&amp;gt;&amp;quot;+mylist.group(1)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And this is providing me with duplicates of the same gene:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;H149xcV_Fge342_r3_h2&#xA;&amp;gt;H149xcV_bTr423_r3_h2&#xA;&amp;gt;H149xcV_kN893_r3_h2&#xA;&amp;gt;H149xcV_DNp021_r3_h2&#xA;&amp;gt;H149xcV_JEP3324_r3_h2&#xA;&amp;gt;H149xcV_JEP3324_r3_h2&#xA;&amp;gt;H149xcV_JEP3324_r3_h2&#xA;&amp;gt;H149xcV_SRt424234_r3_h2&#xA;&amp;gt;H149xcV_Fge342_r3_h2&#xA;&amp;gt;H149xcV_Fge342_r3_h2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I reformat my command so that I only receive unique gene id's:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;H149xcV_Fge342_r3_h2&#xA;&amp;gt;H149xcV_bTr423_r3_h2&#xA;&amp;gt;H149xcV_kN893_r3_h2&#xA;&amp;gt;H149xcV_DNp021_r3_h2&#xA;&amp;gt;H149xcV_JEP3324_r3_h2&#xA;&amp;gt;H149xcV_SRt424234_r3_h2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9868" LastEditorUserId="3848" LastEditDate="2022-02-24T04:06:19.723" LastActivityDate="2022-02-24T12:09:38.957" Title="How to remove duplicates from a fasta file using python" Tags="|fasta|python|" AnswerCount="6" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14916" PostTypeId="1" CreationDate="2020-11-24T10:43:20.407" Score="1" ViewCount="135" Body="&lt;p&gt;I would like to print the alternative allele from a vcf file which i load in with htslib.&#xA;The description of some functions:&#xA;&lt;a href=&quot;https://github.com/samtools/htslib/blob/develop/htslib/vcf.h&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/samtools/htslib/blob/develop/htslib/vcf.h&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have used bcf1_t objects by creating a header and reading the information.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;htsFile *test_vcf = NULL;&#xA;&#xA;  // creates header&#xA;  bcf_hdr_t *test_header = NULL;&#xA;  &#xA;  // initialize and allocate bcf1_t object&#xA;  bcf1_t *test_record = bcf_init();&#xA;&#xA;  test_vcf = vcf_open(&amp;quot;Testfile.vcf&amp;quot;, &amp;quot;r&amp;quot;);&#xA;&#xA;  // returning a bcf_hdr_t struct &#xA;  test_header = bcf_hdr_read(test_vcf);&#xA;  &#xA;  while(bcf_read(test_vcf, test_header, test_record) == 0){&#xA;    std::cout &amp;lt;&amp;lt; &amp;quot;chr&amp;quot; &amp;lt;&amp;lt; bcf_hdr_id2name(test_header, test_record-&amp;gt;rid) &amp;lt;&amp;lt; &amp;quot;:&amp;quot; &amp;lt;&amp;lt;test_record-&amp;gt;pos+1 &amp;lt;&amp;lt; std::endl;&#xA;  }&#xA;  bcf_hdr_destroy(test_header);&#xA;  bcf_destroy(test_record); &#xA;  bcf_close(test_vcf);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But im not able to extract the reference or alternative allele information using this structure bcf1_t.&lt;/p&gt;&#xA;&lt;p&gt;As far as i can read its perhaps possible with the structure and function:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcf_dec_t&#xA;bcf_unpack()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But i am not able to get it to work since all the other functions seems limited to the bcf1_t struct.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone help with extracting the REF and ALT sequence?&#xA;Thanks for your time&lt;/p&gt;&#xA;" OwnerUserId="8630" LastActivityDate="2020-11-24T11:40:29.370" Title="How to print alternative allele from VCF with htslib c++" Tags="|vcf|c++|htslib|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14917" PostTypeId="2" ParentId="14916" CreationDate="2020-11-24T11:40:29.370" Score="1" Body="&lt;p&gt;see the code in &lt;a href=&quot;https://github.com/samtools/htslib/blob/develop/vcf.c#L3104&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/samtools/htslib/blob/develop/vcf.c#L3104&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; bcf_unpack((bcf1_t*)v, BCF_UN_ALL);// if you need genotypes too, might use another constant to avoid decoding all the structure&#xA; (...)&#xA; kputc('\t', s); // ALT&#xA; if (v-&amp;gt;n_allele &amp;gt; 1) {&#xA;        for (i = 1; i &amp;lt; v-&amp;gt;n_allele; ++i) {&#xA;            if (i &amp;gt; 1) kputc(',', s);&#xA;            kputs(v-&amp;gt;d.allele[i], s);&#xA;        }&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastActivityDate="2020-11-24T11:40:29.370" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14918" PostTypeId="1" CreationDate="2020-11-24T13:39:52.127" Score="1" ViewCount="951" Body="&lt;p&gt;I'm pretty new to Unix and bioinformatics and having a hard time accomplishing the following. I have one FASTA file with sequences and headers, and one OTU ID mapping file (.txt) with OTU IDs and their sequences. The sequence header in the .fasta file partially matches the sequence listed in the OTU ID file. What I want to do is replace the current headers in the FASTA file with just the OTU IDs from the .txt file. The files look like something like this:&lt;/p&gt;&#xA;&lt;p&gt;OTU_ID.txt&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;denovo123 U1019.1010.2020_20384&#xA;denovo453 U1472.2992.2020_20443&#xA;denovo763 U1928.1898.2020_20343&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;.FASTA&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;U1019.1010.2020_20384 0000-AHDJ-38347_223&#xA;GAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA&#xA;&amp;gt;U1928.1898.2020_20343 0000-AHDJ-24247_242&#xA;GAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and my desired output is that the .fasta file would look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;denovo123&#xA;GAAATACCGAGAGAGAAATACCGAGAGAGAAAGAAATACCGAGAGAGAAATACCGAGAAATACCGAGAGAGAAATACCGAGAGA&#xA;&amp;gt;denovo763&#xA;GAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is it possible to do this in the command line?&lt;/p&gt;&#xA;" OwnerUserId="9931" LastActivityDate="2020-11-24T17:43:49.493" Title="How to replace sequence identifiers in a fasta with OTU IDs from another file?" Tags="|fasta|shell|identifiers|grep|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14919" PostTypeId="1" AcceptedAnswerId="14936" CreationDate="2020-11-24T13:45:01.580" Score="1" ViewCount="800" Body="&lt;p&gt;I use cealign in PyMol for structure alignment. Instead of visualization, I want to return the alignment object to my python script for further analysis. Is there a function to return the object without I/O to a clustalw file?&lt;/p&gt;&#xA;&lt;p&gt;edit:&lt;/p&gt;&#xA;&lt;p&gt;By alignment object, I meant the sequence alignment in cealign.&lt;/p&gt;&#xA;" OwnerUserId="9344" LastEditorUserId="9344" LastEditDate="2020-11-25T15:15:39.497" LastActivityDate="2020-11-25T18:18:05.813" Title="Assessing PyMol sequence alignment object" Tags="|sequence-alignment|pymol|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14920" PostTypeId="1" CreationDate="2020-11-24T15:39:23.810" Score="1" ViewCount="120" Body="&lt;p&gt;I've been handed RNA-seq data with a lot of associated covariates.&#xA;This data has been put through the DESeq2 package and as a result normalised the data.&#xA;One of the transcripts of interest still has non-normal distribution for two genotypes. This is because many of the normalised counts are now clustered around a value of 4.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BBdRf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BBdRf.png&quot; alt=&quot;QQplot of transcript expression against a genotype&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is making it tricky to perform statistical tests because in order to perform parametric tests (in this case I want to perform a ANCOVA or multiple linear regression to include the covariates) the data must be normally distributed.&lt;/p&gt;&#xA;&lt;p&gt;I also don't want to use a non-parametric test because I can't include all the covariates.&lt;/p&gt;&#xA;&lt;p&gt;Should I just perform parametric tests ignoring this violation or do I have to just perform non-parametric on all my data for a fair comparrison?&#xA;Or is there another alternative option I am missing?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="9669" LastEditorUserId="9669" LastEditDate="2020-11-25T06:42:32.237" LastActivityDate="2020-11-26T08:42:23.670" Title="How would we test for significant changes in expression when many of the normalised counts are clustered together?" Tags="|r|rna-seq|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14922" PostTypeId="2" ParentId="14918" CreationDate="2020-11-24T16:49:20.323" Score="1" Body="&lt;p&gt;You can do it with bash quiet easy. First create a backup of your fasta file with&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cp otu.fasta otu_backup.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then you can run:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;while IFS=&amp;quot; &amp;quot; read -r otu fastaId rest; &#xA;do&#xA;sed -E -i &amp;quot;s/&amp;gt;$fastaId .+/&amp;gt;$otu/&amp;quot; otu.fasta; done &amp;lt; ids.txt;&#xA;done &amp;lt; OTU_ID.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What does the code?&lt;/p&gt;&#xA;&lt;p&gt;With &lt;code&gt;IFS=&amp;quot; &amp;quot;&lt;/code&gt; you declare a whitespace as field seperator and by reading each line of your &lt;code&gt;OTU_ID.txt&lt;/code&gt;, you have the two variables &lt;code&gt;otu&lt;/code&gt; and &lt;code&gt;fastaId&lt;/code&gt; which hold the otu name and the id of the Fasta Header.&lt;/p&gt;&#xA;&lt;p&gt;Within the &lt;code&gt;while&lt;/code&gt; loop, sed is looking each loop for the corresponding id within a RegEx and substitute it with the corresponding otu name. By using the &lt;code&gt;-i&lt;/code&gt; option, you overwrite the existing file. This is why you need to create a backup.&lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2020-11-24T16:49:20.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14923" PostTypeId="2" ParentId="14918" CreationDate="2020-11-24T17:18:30.877" Score="1" Body="&lt;p&gt;You can put you fasta file in a single line per sequence:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -pe 'if(/\&amp;gt;/){s/\n/\t/; s/&amp;gt;//; s/\n//}' file.fasta &amp;gt; file.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This line will create the next output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;U1019.1010.2020_20384 0000-AHDJ-38347_223   GAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA&#xA;U1928.1898.2020_20343 0000-AHDJ-24247_242   GAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then with awk you can match columns separated with tab:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk -F' ' 'FNR==NR{a[&lt;span class=&quot;math-container&quot;&gt;$2]=$&lt;/span&gt;1;next} (&lt;span class=&quot;math-container&quot;&gt;$1 in a) {print &quot;&amp;gt;&quot;a[$1]&quot;\n&quot;$&lt;/span&gt;3}' otu_id.txt file.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You will get this output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;denovo123&#xA;GAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA&#xA;&amp;gt;denovo763&#xA;AAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGAGAAATACCGAGAGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can add &lt;strong&gt;&lt;code&gt;$1&lt;/code&gt;&lt;/strong&gt; to check if you are getting the right id.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk -F' ' 'FNR==NR{a[&lt;span class=&quot;math-container&quot;&gt;$2]=$&lt;/span&gt;1;next} (&lt;span class=&quot;math-container&quot;&gt;$1 in a) {print &quot;&amp;gt;&quot;a[$1], $1, &quot;\n&quot;$&lt;/span&gt;3}' otu_id.txt file.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If everything is alright you can redirect to a new file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk -F' ' 'FNR==NR{a[&lt;span class=&quot;math-container&quot;&gt;$2]=$&lt;/span&gt;1;next} (&lt;span class=&quot;math-container&quot;&gt;$1 in a) {print &quot;&amp;gt;&quot;a[$1]&quot;\n&quot;$&lt;/span&gt;3}' otu_id.txt file.tsv &amp;gt; file_ids.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How it works, well &lt;code&gt;awk&lt;/code&gt; implicitly loops through each file, one line at a time.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;FNR==NR{a[$2]=$1;next}&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;NR&lt;/code&gt; is the number of lines that awk has read so far and &lt;code&gt;FNR&lt;/code&gt; is the number of lines that &lt;code&gt;awk&lt;/code&gt; has read from the current file. Thus, if &lt;code&gt;FNR==NR&lt;/code&gt;, we are still reading the first-named file: otu_id.txt. For every line in file.tsv, we assign &lt;code&gt;a[$2]=$1&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here, &lt;code&gt;a&lt;/code&gt; is an associative array, and &lt;code&gt;a[$2]=$1&lt;/code&gt; means saving file.tsv first column, denoted &lt;code&gt;$1&lt;/code&gt;, as &lt;code&gt;a&lt;/code&gt; value in array a using file.tsv second column, &lt;code&gt;$2&lt;/code&gt;, as the key.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;next&lt;/code&gt; tells &lt;code&gt;awk&lt;/code&gt; to skip the rest of the commands and start over with the next line.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;($1 in a) {print &amp;quot;&amp;gt;&amp;quot;a[$1]&amp;quot;\n&amp;quot;$3}&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;This means that we are reading the second file: otu_id.tsv. If match the first field of the line in file.tsv, as determined by the contents of array &lt;code&gt;a&lt;/code&gt;, it will print out a line with the values of fields &lt;code&gt;a[$1]&lt;/code&gt; and &lt;code&gt;$3&lt;/code&gt; separated with a new line character.&lt;/p&gt;&#xA;" OwnerUserId="4282" LastEditorUserId="4282" LastEditDate="2020-11-24T17:43:49.493" LastActivityDate="2020-11-24T17:43:49.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14926" PostTypeId="1" CreationDate="2020-11-24T22:16:45.617" Score="1" ViewCount="114" Body="&lt;p&gt;I am looking at the publicly available gnomAD v3 VCF file.&#xA;The header describes the AN field as: &amp;quot;Total number of alleles in samples.&amp;quot;&#xA;This suggests to me that it should be a constant integer for each site.&lt;/p&gt;&#xA;&lt;p&gt;However, as the example query below shows (unfortunately they removed the link to download this file, and you need to use cloud services to download the current files -&amp;gt; &lt;a href=&quot;https://gnomad.broadinstitute.org/downloads#v3-variants&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gnomad.broadinstitute.org/downloads#v3-variants&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;bcftools view -r chr1:15274 -H gnomad.genomes.r3.0.sites.vcf.bgz | \&#xA;awk -F[&amp;quot;\t;&amp;quot;] '{print $1,$2,$4,$5,$68}' -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This outputs (chr, pos, ref, alt, AN_afr):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1 15274 A G AN_afr=19388&#xA;chr1 15274 A T AN_afr=17866&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;, i.e. the AN_afr (AN for African ancestry) is variable for the same site for two different SNVs. I've found other examples where there are three different SNVs, each with its own AN_afr. This seems to be the rule rather than the exception for multi-allelic sites, regardless of the FILTER. Can someone explain why this is?&lt;/p&gt;&#xA;&lt;p&gt;I understand how it can vary between sites according to whether or not genotypes are called. What I don't understand is why it can vary at the same site? If it is indeed the case that it can vary, then what is the rule for picking a subset of samples for computing the allele frequency for one particular SNV?&lt;/p&gt;&#xA;" OwnerUserId="9936" LastEditorUserId="9936" LastEditDate="2020-11-30T13:07:08.567" LastActivityDate="2020-11-30T13:07:08.567" Title="How does Allele Number get determined in VCF files?" Tags="|vcf|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14927" PostTypeId="1" AcceptedAnswerId="14932" CreationDate="2020-11-25T00:40:55.297" Score="0" ViewCount="721" Body="&lt;p&gt;For proteins that have multiple chains (e.g., &lt;a href=&quot;https://www.rcsb.org/structure/1EMS&quot; rel=&quot;nofollow noreferrer&quot;&gt;1EMS&lt;/a&gt;), is there an easy way to specify which chain I want to use for blastp?&lt;/p&gt;&#xA;&lt;p&gt;I cannot imagine that I am the first person to have this problem, but so far my only solution has been to write my own script that gets the chain I want from the .pdb file and convert that to a .fasta file.&lt;/p&gt;&#xA;" OwnerUserId="9897" LastActivityDate="2020-11-25T16:11:02.553" Title="FASTA and PDB: How to specify chain?" Tags="|fasta|pdb|blastp|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14928" PostTypeId="2" ParentId="14516" CreationDate="2020-11-25T06:10:17.450" Score="1" Body="&lt;p&gt;The ENCODE Data Coordination Center, that developed and maintains the ENCODE portal, have implemented scripts validating adherence to the standards the ENCODE Consortium agreed upon when evaluating ChIP-seq analysis results. For any ChIP-seq experiment, except for H3K9me3, the adherence to the read depth standard is assessed looking on the number of &amp;quot;usable fragemnts&amp;quot; (&lt;a href=&quot;https://www.encodeproject.org/data-standards/terms/#read-depth&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.encodeproject.org/data-standards/terms/#read-depth&lt;/a&gt;) (see answer by James Hawley). For H3K9me3 the read depth is assessed not by looking on usable fragments, but by looking on the total number of mapped reads (if it is a PE sequencing run, the number of mapped read pairs). It is an exception, because the number of mapped reads, especially for H3K9me3 is substantially higher than the number of usable fragments (where the duplicated and low mapping quality reads are filtered out). The decision to apply different read depth standard in case of H3K9me3 came up due to the logic explained here (&lt;a href=&quot;https://www.encodeproject.org/chip-seq/histone/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.encodeproject.org/chip-seq/histone/&lt;/a&gt;), briefly: because the H3K9me3 mark is enriched in repetitive regions of the genome, the number of usable fragments (post-filtering) is not expected to be as high as other marks (45M), despite high-quality libraries and sufficient sequencing depth. Instead of marking these experiments as failing read depth standard, the script checks the total number of mapped reads (or read pairs for PE sequencing runs) and makes sure it is above the read depth threshold (45M).&lt;/p&gt;&#xA;" OwnerUserId="9938" LastActivityDate="2020-11-25T06:10:17.450" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14929" PostTypeId="2" ParentId="14920" CreationDate="2020-11-25T08:20:14.583" Score="0" Body="&lt;p&gt;Your assessment is perfectly correct. The easiest approach is to perform a transformation to normalise your data and then you are good to go. DeSeq2 has a standard transformation (I think it is called deseq2) and you can check your results via a Q-Q plot and you are looking for an approximately linear relationship.&lt;/p&gt;&#xA;&lt;p&gt;There are a multitude of transformations, but the transformation proceedure has been heavily investigated and standardised in RNAseq&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;If normalisation by deseq2 has failed (Q-Q) you simply examine an alternative transformation. However, do not transformation a transformation. If you cannot normalise the data then you are stuck with non-parametric stats in my personal opinion.&lt;/p&gt;&#xA;&lt;p&gt;However, mixed models can work if you trap the underlying distribution and in a regression will result in a very low residual.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-11-26T08:42:23.670" LastActivityDate="2020-11-26T08:42:23.670" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14930" PostTypeId="2" ParentId="14927" CreationDate="2020-11-25T10:07:26.167" Score="3" Body="&lt;p&gt;The NCBI BLAST can be used with the PDB DB (which NCBI has). The PDB codes are stored as 4 letter codes underscore chain, &lt;em&gt;e.g.&lt;/em&gt; &lt;code&gt;1GFL_B&lt;/code&gt;. The catch is segment identifiers —but generally they are the same peptide so shouldn't be an issue.&lt;/p&gt;&#xA;&lt;p&gt;You can search specifically the PDB DB in NCBI (not the RCSB PDB) by setting the database to &lt;code&gt;PDB&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2020-11-25T16:10:33.723" LastActivityDate="2020-11-25T16:10:33.723" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14931" PostTypeId="2" ParentId="14920" CreationDate="2020-11-25T10:27:50.783" Score="1" Body="&lt;p&gt;TLDR- just do the analysis in DESeq2 (it can handle multiple covariates)&lt;/p&gt;&#xA;&lt;p&gt;If I've interpreted your question correctly there seem to be a few confusions here.&lt;/p&gt;&#xA;&lt;p&gt;First off, for a multiple linear regression aka ANCOVA, one of the assumptions are the &lt;em&gt;residuals&lt;/em&gt; are normally distributed - however this is almost never an issue in practice. The distribution of the data is not so important, you make the assumption that the data has come from a population which is continuous and normally distributed that's why you've chosen to do an ANCOVA. But, RNAseq produces count data - not continuous and bounded at zero. You need to be looking at using a Poisson GLM, or negative binomial GLM as done by DESeq2 (lookup overdispersion)&lt;/p&gt;&#xA;&lt;p&gt;The normalisation done by DESeq is to account for different library sizes,it doesn't &lt;em&gt;transform&lt;/em&gt; the data to make it conform to a normal distribution. You'd need to do some form of data transformation to do that, but don't it's unnecessary.&lt;/p&gt;&#xA;&lt;p&gt;I'd suggest reading the DESeq vignettes and following their standard pipeline.&lt;/p&gt;&#xA;&lt;p&gt;Hope that is of some help.&lt;/p&gt;&#xA;" OwnerUserId="8919" LastActivityDate="2020-11-25T10:27:50.783" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14932" PostTypeId="2" ParentId="14927" CreationDate="2020-11-25T16:11:02.553" Score="0" Body="&lt;p&gt;WEB VERSION OF BLASTP&lt;/p&gt;&#xA;&lt;p&gt;When you run a blastp query using a fasta file with multiple chains you can choose which chain's results to examine using a drop-down box that appears &lt;strong&gt;after&lt;/strong&gt; the search has run.&lt;/p&gt;&#xA;&lt;p&gt;TERMINAL BLASTP&lt;/p&gt;&#xA;&lt;p&gt;The results for each chain will be listed separately.&#xA;For ease of searching you can specify the output to be a csv of your desired format (See blastp -help and look up the -outfmt option for more information).&lt;/p&gt;&#xA;&lt;p&gt;Note that for some proteins, such as the 1EMS you chose for an example, the multiple chains are identical. When you get the fasta file for that specific protein it will only have one entry. However if you really need a fasta file formated so each chain has its own sequence you can get the &lt;a href=&quot;https://www.rcsb.org/downloads/fasta&quot; rel=&quot;nofollow noreferrer&quot;&gt;fasta file for all PDB entries&lt;/a&gt; and extract the specific protein_chain you want.&lt;/p&gt;&#xA;" OwnerUserId="9897" LastActivityDate="2020-11-25T16:11:02.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14933" PostTypeId="1" CreationDate="2020-11-25T16:38:42.953" Score="0" ViewCount="290" Body="&lt;p&gt;A publication that we have been working on for over 6 months on COVID-19 has been rejected by 2 journals. The objective of our research was to identify effective compounds against the COVID-19 virus by using Molecular Docking studies performed with AutoDock Vina. Which Journal should we submit our manuscript to? Following are the Rejection emails I received from them. What do you think is the reason for the rejection and what should I do to resolve the issue to get my manuscript successfully published.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Rejection Notification from International Journal of Molecular Sciences&lt;/strong&gt;&lt;/em&gt;&#xA;&amp;quot;We are writing to inform you that we will not be able to process your&#xA;submission further. Submissions sent for peer-review are selected on the&#xA;basis of discipline, novelty and general significance, in addition to the&#xA;usual criteria for publication in scholarly journals. Therefore, our decision&#xA;is not necessarily a reflection of the quality of your work.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Rejection Notification from International Journal of Biological Macromolecules&lt;/strong&gt;&lt;/em&gt;&#xA;&amp;quot;Thank you for submitting your manuscript to International Journal of Biological Macromolecules.&#xA;I have reviewed your manuscript and I must regretfully inform you that your work presents information that is specific to a selective audience in its nature and interest and is more appropriate for a more specialized journal. The focus of your present work, a computational study, is unfortunately outside the scope of the journal, therefore, unsuitable for publication in IJBIOMAC.&#xA;For IJBIOMAC publication, the computational analysis should be used only as a supporting tool to non-computational analysis.&#xA;We appreciate you submitting your manuscript to International Journal of Biological Macromolecules and thank you for giving us the opportunity to consider your work.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your time.&lt;/p&gt;&#xA;" OwnerUserId="9346" LastEditorUserId="3848" LastEditDate="2020-11-25T21:41:55.110" LastActivityDate="2020-11-25T21:41:55.110" Title="Rejected publication" Tags="|covid-19|publishing|" AnswerCount="1" CommentCount="1" ClosedDate="2020-11-25T22:10:00.500" ContentLicense="CC BY-SA 4.0" />
  <row Id="14934" PostTypeId="1" AcceptedAnswerId="14970" CreationDate="2020-11-25T16:51:34.210" Score="0" ViewCount="27" Body="&lt;p&gt;I'm working with a set of homologous genes (let's call it gene A) from several bacterial species.&lt;/p&gt;&#xA;&lt;p&gt;I know (from previously published research) that in gene B (a close paralogue of gene A), there is a non-coding, antisense RNA. This antisense RNA regulates gene B by forming a duplex with the gene B mRNA.&lt;/p&gt;&#xA;&lt;p&gt;I have noticed that, sometimes, an antisense RNA would be predicted for gene A in the same location in the IMG/MER database. This is not predicted for all of the available gene A sequences, only some of them. IMG/MER uses INFERNAL to predict RNAs, I think.&lt;/p&gt;&#xA;&lt;p&gt;My question is - is there any way to assess how real this is without wet lab data? As it is, it could be that there's no actual antisense RNA in gene A, and this is a spurious prediction due to homology. Or it could be that the antisense RNA is present in all of the gene A sequences, and the RNA prediction software simply missed some of them. Is there a computational way of testing which of these options (if any) is more likely? Thank you for your time!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2020-11-30T11:21:09.763" Title="Is is possible to predict ncRNAs from sequence and homology alone?" Tags="|rna|software-recommendation|non-coding|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14935" PostTypeId="2" ParentId="14933" CreationDate="2020-11-25T18:05:55.897" Score="2" Body="&lt;p&gt;I cannot comment on your specific manuscript since I obviously cannot read it but in general the question is whether a study does add an interesting novelty.&lt;/p&gt;&#xA;&lt;p&gt;The problem with pure in silico studies is that there is essentially no confirmation towards the in vivo relevance. Drug companies simulate and also test many thousands of compounds in a high-throughput fashion on a daily basis so it is often questionable to what extend individual pure in silico studies without in vivo confirmation do actually add to the bigger picture.&lt;/p&gt;&#xA;&lt;p&gt;That &lt;em&gt;might&lt;/em&gt; be the reason your work was rejected but as said this is a rather general comment and I can neither judge content nor quality of your work so please do not feel offended and read it with a grain of salt.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-11-25T18:05:55.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14936" PostTypeId="2" ParentId="14919" CreationDate="2020-11-25T18:18:05.813" Score="0" Body="&lt;p&gt;In the application, unfortunately the only option is doing&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;cealign ,, object =&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;Menu &amp;gt; Export Alignment...&lt;/code&gt; on your cealign &lt;code&gt;&lt;/code&gt; object&lt;/li&gt;&#xA;&lt;li&gt;and converting the Clustal format using one of the millions of tools online&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;get_fastastr&lt;/h2&gt;&#xA;&lt;p&gt;As you know &lt;code&gt;cealign&lt;/code&gt; adds gaps, but &lt;code&gt;cmd.get_fastastr('')&lt;/code&gt; strips them. There is no option to keep the gaps —annoyingly especially with missing loop info!&#xA;The help has this to say:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; get_fastastr(selection='all', state=-1, quiet=1, key='', _self=&amp;lt;module 'pymol.cmd' from '/Users/matteo/miniconda3/lib/python3.7/site-packages/pymol/cmd.py'&amp;gt;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Correction and dissection&lt;/h2&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;So I had a quick check to see if there is an alternative route to get the sequence string with gaps. These are my observations using the Python3 module, sorry if they aren't polished up.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the comments I said something incorrect. &lt;code&gt;cmd.cealign&lt;/code&gt; returns a dictionary with the 4x4 rototranslational matrix —no need for an object. Using Python3 and two TIM barrels as an example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pymol2&#xA;&#xA;with pymol2.PyMOL() as pymol:&#xA;    pymol.cmd.fetch('1TIM', 'tim')&#xA;    pymol.cmd.remove('not chain A')&#xA;    pymol.cmd.fetch('5AHE', 'hisA')&#xA;    x = pymol.cmd.cealign('tim', 'hisA')&#xA;    print(x)&#xA;# {'alignment_length': 184, 'RMSD': 3.8800425577361755, 'rotation_matrix': [...]}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;While the object does not contain any useful info from PyMol &lt;em&gt;directly&lt;/em&gt;*:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    ...&#xA;    pymol.cmd.cealign('tim', 'hisA', object ='rototrans')&#xA;    pymol.cmd.get_session('rototrans')['names'][0][5]&#xA;    # useful info is generally in ['names'][0][5]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(*) The code is actually wrapped C++ code, so the object &lt;code&gt;rototrans&lt;/code&gt; is in the C++ layer and can only be access by session or possibly by some functions in &lt;code&gt;cmd.exporting&lt;/code&gt;. In fact, it can be exported to an aln string (ClustalW).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    aln = pymol.cmd.exporting.get_alnstr('rototrans')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In the PyMOL application, doing the &lt;code&gt;cealign&lt;/code&gt; with an object (&lt;code&gt;cmd.cealign('tim', 'hisA', object ='rototrans')&lt;/code&gt;, allows the  Menu&amp;gt;Export Alignment...&amp;gt;'rototrans' which is saved as a ClustalW file, as you mention. Namely:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CLUSTAL&#xA;&#xA;tim          APRKFFVGGN----WK-------MNGKRKSLGELIHTLDGAKLSADTEVVCGAPSI------Y&#xA;hisA         MIIPALDLIDGTVVRLHQGDYARQRDYGN---DPLPRLQDYAAQGAGVLHLVDLTGAKDPAKR&#xA;                           ..       ......   .....*..................      .&#xA;&#xA;tim          LDFARQK---LDAKIGVAAQNCYKVPKGAFTGEISPAMIKDI-GAAWVILGHSERRHVFGESD&#xA;hisA         QIPLIKTLVAGVNVPVQVGGGVRTEE--------DVAALLKAGVARVVIGS---------TAV&#xA;             .......   ................        ..*..... .*..**..         ...&#xA;&#xA;tim          ELIGQKVAHALAEG---LGVIACIGEKLDEREAGIT-----------EKVVFQETKAIADNVK&#xA;hisA         KSPDV-VKGWFERFGAQALVLALDV---RIDEHGTKQVAVSGWQENSGVSLEQLVETYLPVGL&#xA;             ..... *.......   ..*.*...   ...*.*..           .....*..........&#xA;&#xA;tim          DWSKVVLAYEPVWAIGTGKTATP------QQAQEVHEKLRGWLKTHVSDAVAVQSRIIYGGSV&#xA;hisA         ----KHVLCTDI----------SRDGTLAGSNVSLYEEVCARYP---------QIAFQSSGGI&#xA;                 ........          .      .......*.......         *......*..&#xA;&#xA;tim          TGGNCKELASQHDVDGFLVGGASLKPEFVDIINAKH------------               &#xA;hisA         GDIDDIAALRGTGVRGVIVGRALLEGKFTVKEAIQ?WQNVKGHHHHHH               &#xA;             .............*.*..**.*.*...*..                                 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, there is no method in &lt;code&gt;cmd.exporting&lt;/code&gt; that gives out just the sequence with gaps.&lt;/p&gt;&#xA;&lt;p&gt;Whereas the &lt;code&gt;cmd.exporting&lt;/code&gt; has the following... none applicable.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; print(dir(pymol.cmd.exporting))&#xA;['DEFAULT_ERROR', 'DEFAULT_SUCCESS', 'QuietException', 'SDF', 'SDFRec', 'Shortcut', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '_cmd', '_feedback', '_get_dump_str', '_get_mtl_obj', '_raising', '_resn_to_aa', '_session_convert_legacy', '_unit2px', 'absolute_import', 'assign_atom_types', 'cPickle', 'cache', 'cache_action_dict', 'cache_action_sc', 'cmd', 'copy', 'copy_image', 'export_coords', 'fb_mask', 'fb_module', 'get_alnstr', 'get_bytes', 'get_ccp4str', 'get_cifstr', 'get_fastastr', 'get_maestr', 'get_mol2str', 'get_pdbstr', 'get_pqrstr', 'get_psestr', 'get_sdfstr', 'get_session', 'get_str', 'get_xyzstr', 'io', 'is_dict', 'is_error', 'is_list', 'is_ok', 'is_tuple', 'loadable', 'lock', 'multifilenamegen', 'multifilesave', 'multisave', 'os', 'png', 'print_function', 'pymol', 're', 'save', 'savefunctions', 'selector', 'sys', 'thread', 'traceback', 'unlock']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2020-11-25T18:18:05.813" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14937" PostTypeId="1" CreationDate="2020-11-25T19:07:01.693" Score="2" ViewCount="266" Body="&lt;p&gt;There are some questions about lifting between reference builds, e.g. &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/14228/lifting-from-hg38-to-hg37&quot;&gt;this one&lt;/a&gt;. But there doesn't appear to be a question about lifting a GWAS results file to a new reference build (except off-site). GWAS results files are not VCFs or plink files, so they aren't covered by the usual lifting tools (such as &lt;a href=&quot;http://crossmap.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CrossMap&lt;/a&gt; or &lt;a href=&quot;https://genome.sph.umich.edu/wiki/LiftOver&quot; rel=&quot;nofollow noreferrer&quot;&gt;LiftOver&lt;/a&gt;). How do I lift a GWAS results file to hg38?&lt;/p&gt;&#xA;&lt;p&gt;For instance, the &lt;a href=&quot;https://www.thessgac.org/data&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lee et al 2018 GWAS on education&lt;/a&gt; has this file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;user@computer:/home/user/Lee_2018$ head GWAS_EA.to10K.txt&#xA;MarkerName  CHR POS A1  A2  EAF Beta    SE  Pval&#xA;rs9859556   3   49455986    G   T   0.6905  -0.02901    0.00151 4.61e-82&#xA;rs7623659   3   49414791    T   C   0.3095  0.02899 0.00151 6.05e-82&#xA;rs11917431  3   49644012    C   T   0.6973  -0.0292 0.00152 7.40e-82&#xA;rs1873625   3   49666964    C   A   0.6973  -0.02916    0.00152 8.10e-82&#xA;rs11921590  3   49644193    T   C   0.6973  -0.02916    0.00152 1.17e-81&#xA;rs2352974   3   49890613    T   C   0.5 -0.02569    0.00141 6.35e-74&#xA;rs7029718   9   23358495    G   A   0.5646  -0.0244 0.00143 7.82e-65&#xA;rs7868984   9   23357826    C   T   0.4388  0.02437 0.00143 8.21e-65&#xA;rs11793831  9   23362311    T   G   0.4354  0.02418 0.00143 9.34e-64&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9417" LastActivityDate="2024-03-15T16:13:53.337" Title="How do I lift GWAS results to hg38?" Tags="|gwas|liftover|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14938" PostTypeId="2" ParentId="14937" CreationDate="2020-11-25T19:07:01.693" Score="2" Body="&lt;p&gt;It turned out to be easier than expected. You figure out which dbSNP version they used. dbSNP is the classification system for snps based on the rsid format. Pretty much every GWAS publishes their results with the rsids for the variants, as done above as well (MarkerName). So you download one of the dbSNP files &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/projects/SNP/snp_summary.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;from the official site&lt;/a&gt;. In this case, I am using version 142 since this was used by the 1000 genomes dataset.&lt;/p&gt;&#xA;&lt;p&gt;After this, it is a matter of joining the tables. This can be done easily in R (you need about 15GB free memory), we load the dbSNP database into memory. Since we don't need all the columns, we skip some of them to save memory:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#library&#xA;library(tidyverse)&#xA;&#xA;#cols&#xA;vcf_cols = cols_only(&#xA;  `#CHROM` = col_character(),&#xA;  `POS` = col_number(),&#xA;  `ID` = col_character(),&#xA;  `REF` = col_character(),&#xA;  `ALT` = col_character()&#xA;  )&#xA;&#xA;#this is from the 1kg&#xA;#ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/technical/reference/GRCh38_reference_genome/other_mapping_resources/&#xA;dbsnp142 = readr::read_tsv(&#xA;  &amp;quot;ALL_20141222.dbSNP142_human_GRCh38.snps.vcf&amp;quot;),&#xA;  comment = &amp;quot;##&amp;quot;,&#xA;  col_types = vcf_cols&#xA;  )&#xA;&#xA;GWAS_EA = read_tsv(GWAS_EA.to10K.txt,&#xA;                   col_types = cols(chrpos37 = col_character(),&#xA;                                    chrpos38 = col_character()))&#xA;&#xA;#make chrpos&#xA;dbsnp142 %&amp;lt;&amp;gt;% mutate(&#xA;  chrpos38 = str_c((CHROM %&amp;gt;% str_replace(&amp;quot;chr&amp;quot;, &amp;quot;&amp;quot;)), &amp;quot;:&amp;quot;, POS)&#xA;)&#xA;&#xA;#count overlap&#xA;(GWAS_EA&lt;span class=&quot;math-container&quot;&gt;$MarkerName %in% dbsnp142$&lt;/span&gt;ID) %&amp;gt;% table2()&#xA;&#xA;#mutate and merge&#xA;GWAS_EA = GWAS_EA %&amp;gt;%&#xA;  mutate(&#xA;    chrpos37 = str_c(CHR, &amp;quot;:&amp;quot;, POS)&#xA;  ) %&amp;gt;%&#xA;  left_join(dbsnp142 %&amp;gt;% select(ID, chrpos38), by = c(&amp;quot;MarkerName&amp;quot; = &amp;quot;ID&amp;quot;))&#xA;&#xA;#update file on disk&#xA;GWAS_EA %&amp;gt;% write_tsv(&amp;quot;GWAS_EA.to10K.txt&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The two new columns, chrpos37 and chrpos38, have the chr:pos variant names in both versions.&lt;/p&gt;&#xA;" OwnerUserId="9417" LastActivityDate="2020-11-25T19:07:01.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14939" PostTypeId="1" CreationDate="2020-11-25T19:54:35.783" Score="1" ViewCount="26" Body="&lt;p&gt;I am interested in detecting allelic imbalance for a credible set of SNPs. Currently, I am looking for some packages, hopefully, for Python, that can assist with this. I am aware of this R package, &lt;a href=&quot;https://bioconductor.org/packages/3.13/bioc/html/AllelicImbalance.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/3.13/bioc/html/AllelicImbalance.html&lt;/a&gt;, but I am hoping someone knows of a similar Python library.&lt;/p&gt;&#xA;&lt;p&gt;Cheers&lt;/p&gt;&#xA;" OwnerUserId="9614" LastActivityDate="2020-11-25T19:54:35.783" Title="Detecting Allelic Imbalance" Tags="|python|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14940" PostTypeId="1" CreationDate="2020-11-26T05:00:07.863" Score="0" ViewCount="31" Body="&lt;p&gt;Suppose I generate a DNA sequence of the following pattern.&lt;/p&gt;&#xA;&lt;p&gt;AAGTC&lt;/p&gt;&#xA;&lt;p&gt;And after being passed through a channel with insertion, deletion and substitution errors obtain the following sequence&lt;/p&gt;&#xA;&lt;p&gt;AAAGGC&lt;/p&gt;&#xA;&lt;p&gt;Where there was an insertion error in the first position, second position and deletion in the third position and no error in the final position.&lt;/p&gt;&#xA;&lt;p&gt;I currently know the error pattern but is there a way to automatically identify the error patterns and annotate them?&lt;/p&gt;&#xA;" OwnerUserId="9947" LastActivityDate="2020-11-26T05:00:07.863" Title="DNA sequence error annotation" Tags="|sequence-alignment|ngs|phylogenetics|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14942" PostTypeId="1" CreationDate="2020-11-26T10:10:09.973" Score="1" ViewCount="23" Body="&lt;p&gt;I have obtained ( interspersed repeats ) libraries in the Palm plant genome. Still, unfortunately, the candidates obtained seems false positive a lot of tandem repeats found, due to denovo identifications tools I assume, then employing TRF tandem repeat finder to mask any tandem repeat, I have succeeded to mask them.&lt;/p&gt;&#xA;&lt;p&gt;But my question here, how can I discard sequence with a belonging header, if the Candidates masked more than 40% of the full length for each of the identified candidates?&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="3332" LastEditorUserId="3332" LastEditDate="2020-11-26T12:58:45.727" LastActivityDate="2020-11-26T12:58:45.727" Title="Discard Candidates if N &gt;= 40%" Tags="|filtering|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14943" PostTypeId="1" CreationDate="2020-11-26T14:23:28.883" Score="1" ViewCount="2052" Body="&lt;p&gt;My problem is (should be) pretty simple.&lt;/p&gt;&#xA;&lt;p&gt;I have a series of 10 sequences which I wish to convert into a Newick format file - what would be good software to do this? I've tried using Jalview and am getting nowhere.&lt;/p&gt;&#xA;&lt;p&gt;Any help appreciated.&lt;/p&gt;&#xA;" OwnerUserId="9952" LastEditorUserId="3848" LastEditDate="2023-01-07T15:09:56.220" LastActivityDate="2023-01-07T15:15:49.063" Title="Convert a fasta file to Newick format" Tags="|phylogenetics|fasta|phylogeny|format-conversion|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14944" PostTypeId="2" ParentId="14943" CreationDate="2020-11-26T14:36:13.043" Score="3" Body="&lt;p&gt;The best would be Biopython using AlignIO, given they are all in a single file otherise SeqIO. The alignment approach,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import AlignIO&#xA;align = AlignIO.read(&amp;quot;/home/path/myseq.fa&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;print(align)&#xA;with open(&amp;quot;/home/path/example.nex&amp;quot;, &amp;quot;w&amp;quot;) as handle:&#xA;    SeqIO.write(align, handle, &amp;quot;nexus&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(not debugged)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.megasoftware.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MEGAX&lt;/a&gt; or &lt;a href=&quot;http://www.clustal.org/clustal2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ClustalX&lt;/a&gt; which are nice GUI's that will also work. If you've not got Biopython installed then ClustalX is probably the easiest.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-11-26T15:01:44.783" LastActivityDate="2020-11-26T15:01:44.783" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14945" PostTypeId="1" CreationDate="2020-11-26T18:22:13.523" Score="0" ViewCount="31" Body="&lt;p&gt;An example of a result. So far I have been unable to find the answer on my own.&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;&amp;gt;CP000000.0:c2005000-2006000 Escherichia coli strain XYZ chromosome&lt;/p&gt;&#xA;&lt;p&gt;ATG...&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="9954" LastActivityDate="2020-11-26T18:22:13.523" Title="What is the meaning of letter 'c' in tblastn results before genomic sequence position?" Tags="|blast|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14946" PostTypeId="2" ParentId="8975" CreationDate="2020-11-26T18:26:48.493" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/Edinburgh-Genome-Foundry/geneblocks&quot; rel=&quot;nofollow noreferrer&quot;&gt;GeneBlocks&lt;/a&gt; is a Python library for comparing DNA sequences. It can be used to find common blocks (identical regions) in a group of DNA sequences. Using a modified version of your example (I duplicated a small region):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import geneblocks&#xA;&#xA;sequence = geneblocks.load_record('sequence.gb', name='seq')&#xA;sequence2 = geneblocks.load_record('sequence.gb', name='seq2')&#xA;blocks = geneblocks.CommonBlocks.from_sequences([sequence, sequence2])&#xA;&#xA;blocks.plot_common_blocks()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WtTtT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WtTtT.png&quot; alt=&quot;GeneBlocks plot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note that it wasn't designed to plot only one sequence, so as a workaround the sequence is given twice, with different names. As an alternative, you can use one sequence and plot with &lt;a href=&quot;https://github.com/Edinburgh-Genome-Foundry/DnaFeaturesViewer&quot; rel=&quot;nofollow noreferrer&quot;&gt;DNA Features Viewer&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import dna_features_viewer&#xA;&#xA;blocks = geneblocks.CommonBlocks.from_sequences([sequence])&#xA;seq_to_plot = blocks.sequences_with_annotated_blocks()[&amp;quot;seq&amp;quot;]&#xA;graphic_record = dna_features_viewer.BiopythonTranslator().translate_record(seq_to_plot)&#xA;ax, _ = graphic_record.plot(figure_width=10, strand_in_label_threshold=7)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CnSv3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CnSv3.png&quot; alt=&quot;DNA Features Viewer plot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;See the &lt;a href=&quot;https://edinburgh-genome-foundry.github.io/Geneblocks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; for more details. A web interface is also provided at EGF CUBA: &lt;a href=&quot;https://cuba.genomefoundry.org/find-common-blocks&quot; rel=&quot;nofollow noreferrer&quot;&gt;find common blocks&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Disclaimer: I'm the current maintainer of GeneBlocks&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="208" LastEditorUserId="208" LastEditDate="2020-11-26T18:45:15.327" LastActivityDate="2020-11-26T18:45:15.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14947" PostTypeId="1" CreationDate="2020-11-26T20:52:30.740" Score="1" ViewCount="20" Body="&lt;p&gt;I have a bunch of sequences created with &lt;code&gt;Roary&lt;/code&gt; which I want to preprocess. When I read the sequences (genomics data) I want to translate them to protein sequences using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;from Bio.Seq import Seq&#xA;from Bio import SeqIO&#xA;&#xA;def read_fasta_file(fn):&#xA;    with open(fn,'r') as file:&#xA;        fasta_sequences = SeqIO.parse(file, 'fasta')&#xA;        sequences = pd.DataFrame([{&#xA;            'filename': basename(fn),&#xA;            'id': rec.id,&#xA;            'seq': str(rec.seq.translate())} for rec in fasta_sequences])&#xA;    return sequences&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The problem is that &lt;code&gt;.translate()&lt;/code&gt; fails with:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;TranslationError: Codon 'T--' is invalid&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Is there a build-in function to ignore the dashes?&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2020-11-27T00:45:19.653" LastActivityDate="2020-11-27T00:45:19.653" Title="What is the best way to drop alignment artifacts in sequence objects in biopython?" Tags="|python|seqio|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14948" PostTypeId="2" ParentId="14947" CreationDate="2020-11-26T20:56:14.323" Score="2" Body="&lt;p&gt;I think the &lt;code&gt;.ungap()&lt;/code&gt; is what I was looking for.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def read_fasta_file(fn):&#xA;    with open(fn,'r') as file:&#xA;        fasta_sequences = SeqIO.parse(file, 'fasta')&#xA;        sequences = pd.DataFrame([{&#xA;            'filename': basename(fn),&#xA;            'id': rec.id,&#xA;            'seq': str(rec.seq.ungap().translate())} for rec in fasta_sequences])&#xA;    return sequences&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2020-11-26T20:56:14.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14949" PostTypeId="1" CreationDate="2020-11-27T00:53:33.227" Score="2" ViewCount="20" Body="&lt;p&gt;I have a large dataset of assemblies of bacterial &lt;em&gt;S. aureus&lt;/em&gt; genomes (~1000, same species). I am looking at different tools to estimate the presence of drug-resistance factors such as &lt;em&gt;blaZ&lt;/em&gt; or &lt;em&gt;ampC&lt;/em&gt;. Though, using three different approaches, I get three different answers. Some of the tools I use are &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/29177089/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ariba&lt;/a&gt; or &lt;a href=&quot;http://sanger-pathogens.github.io/Roary/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Roary&lt;/a&gt; after gene annotation with Prokka.&lt;/p&gt;&#xA;&lt;p&gt;I wonder what are the most reliable tools used in the field to recognize drug-resistance and virulence factors?&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2020-11-27T00:53:33.227" Title="When to use Ariba vs Roary or other genomics tools to call drug-resistance variants?" Tags="|genomics|microbial-genomics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14950" PostTypeId="2" ParentId="14943" CreationDate="2020-11-27T03:23:05.337" Score="1" Body="&lt;p&gt;@M__'s &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/14944/9952&quot;&gt;answer&lt;/a&gt; led me to &lt;a href=&quot;http://www.clustal.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;clustal.org&lt;/a&gt; and some searching led me to this page (&lt;a href=&quot;http://wiki.christophchamp.com/index.php?title=ClustalW&quot; rel=&quot;nofollow noreferrer&quot;&gt;Christoph's&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;I tried compiling Clustalx (GUI - problem with qmake) which didn't work, but Clustalw (compiled to exe clustalw2) - the command line tool - worked a treat.&lt;/p&gt;&#xA;&lt;p&gt;So, I had my &lt;code&gt;clustalw2&lt;/code&gt; executable and my 25 &lt;em&gt;Drosophila&lt;/em&gt; alcohol dehydrogenase protein sequences and I ran the following commands (from Christoph's page):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./clustalw2 -align -type=protein -infile=droso.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This produces a droso.aln file which looks like this (only a few lines shown):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;D_paulistorum               ETRVAEKLLEHPTQTTLACAQNFVKAIELNKNGAIWKLDLGTLEPIEWTK&#xA;D_insularis                 ETRVAEKLLEHPTQTTLACAQNFVKAIELNKNGAIWKLDLGTLEPIEWTK&#xA;D_equinoxialis              ETRVAEKLLEHPTQTTLACAQNFVKAIELNKNGAIWKLDLGTLEPIEWTK&#xA;D_willistoni                ETRVAEKLLEHPTQTTLACAQNFVKAIELNKNGAIWKLDLGTLEPIEWTK&#xA;                            *. *:**** ** *.:  *.:*********:*****:** .***.* *::&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then on Christoph's page, there is a section:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Generate a Newick tree file&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So, I followed the instructions and ran:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./clustalw2 -tree -infile=droso.aln&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and obtained a file droso.ph which looks like a Newick format file to me (some lines):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(&#xA;(&#xA;(&#xA;D_madeirensis:0.00000,&#xA;D_subobscura:0.00787)&#xA;:0.00257,&#xA;D_guanche:0.00530)&#xA;:0.00336,&#xA;(&#xA;(&#xA;D_pseudoobscura:0.00000,&#xA;D_pseudoobscura_pseudo:0.00000)&#xA;:0.00262,&#xA;D_persimilis:0.00131)&#xA;:0.01239)&#xA;:0.01132,&#xA;D_ambigua:0.01821)&#xA;:0.00839,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, it looks like I have a path to doing what I wanted in the first place - getting from .fasta to Newick format.&lt;/p&gt;&#xA;&lt;p&gt;Then, using the programs &lt;a href=&quot;https://www.jalview.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;jalview&lt;/a&gt; and &lt;a href=&quot;http://tree.bio.ed.ac.uk/software/figtree/&quot; rel=&quot;nofollow noreferrer&quot;&gt;figtree&lt;/a&gt;, I was able to draw trees and the like!&lt;/p&gt;&#xA;" OwnerUserId="9952" LastEditorUserId="3848" LastEditDate="2023-01-07T15:15:49.063" LastActivityDate="2023-01-07T15:15:49.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14951" PostTypeId="1" AcceptedAnswerId="14956" CreationDate="2020-11-27T04:10:17.470" Score="1" ViewCount="45" Body="&lt;p&gt;I am working on a project and used the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;vsearch --derep_fulllength filtered_merged.fa -sizeout -relabel Uniq -output dereplicated_filtered_merged.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and got the following output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;87373926 nt in 203453 seqs, min 310, max 480, avg 352&#xA;Sorting 100%&#xA;10981 unique sequences, avg cluster 2.0, median 1, max 1287&#xA;Writing output file 100% &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output had provided me with the data that 10981 unique sequences have been identified. But I cant seem to identify how many reads of the most common sequence were present in the input data.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestions will be kindly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="9955" LastEditorUserId="3848" LastEditDate="2020-11-27T21:23:00.503" LastActivityDate="2020-11-27T21:23:00.503" Title="Metagenomics: Identifying most common sequences" Tags="|fasta|clustering|metagenome|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14952" PostTypeId="1" AcceptedAnswerId="14955" CreationDate="2020-11-27T09:32:15.203" Score="0" ViewCount="486" Body="&lt;p&gt;After I got dds from my count matrix by DESeq(), I use results() method to get Padj and log2 fold changes for volcano plotting,&#xA;then I found some gene's Padj values are infinite and the P-value is very close to 0;&#xA;this makes those dots on my volcano plotting located very top to Y-aix and also confused me:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;is this normal or not, such a p-value (very close to 0) and -logPadj is infinite;&lt;/li&gt;&#xA;&lt;li&gt;how should I treat those data: are they very different expressed between samples and control or just abnormal data which should be taken off?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Here follows the table:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Gju8a.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Gju8a.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;table 1 is dds results, -log10(padj) show inf;&#xA;table 2 is count matrix from featureCount&#xA;table 3 is size factor of samples&lt;/p&gt;&#xA;&lt;p&gt;Please help me !&lt;/p&gt;&#xA;" OwnerUserId="9916" LastActivityDate="2020-11-27T15:09:37.247" Title="Question of Padj value (very high, infinite) got from dds" Tags="|rna-seq|deseq2|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="14953" PostTypeId="2" ParentId="14943" CreationDate="2020-11-27T09:43:53.153" Score="1" Body="&lt;p&gt;I think you can do this in Jalview.&#xA;In your alignment window, open the &lt;strong&gt;Calculate&lt;/strong&gt; menu and select &lt;strong&gt;Calculate Tree or PCA...&lt;/strong&gt;.&#xA;This will open a &lt;strong&gt;Choose Calculation&lt;/strong&gt; window which you can use to select what kind of tree to build.&#xA;When you click on the &lt;strong&gt;Calculate&lt;/strong&gt; button you will get another window with the tree in it.&#xA;In that window you can open the &lt;strong&gt;File&lt;/strong&gt; menu and then select &lt;strong&gt;Save as...&lt;/strong&gt; → &lt;strong&gt;Newick Format&lt;/strong&gt;.  Hopefully this will do what you need.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/AgaRT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AgaRT.png&quot; alt=&quot;Combined 2-stage screenshot. Calculate-&amp;gt;Calculate Tree or PCA...-&amp;gt;Calculate-&amp;gt;Save as...-&amp;gt;Newick Format&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9957" LastActivityDate="2020-11-27T09:43:53.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14954" PostTypeId="2" ParentId="13023" CreationDate="2020-11-27T14:54:18.717" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://edinburgh-genome-foundry.github.io/DnaChisel/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DNA Chisel&lt;/a&gt; can introduce synonymous mutations into your sequence.&#xA;Annotate the coding sequence with the &lt;a href=&quot;https://edinburgh-genome-foundry.github.io/DnaChisel/genbank/genbank_api.html#indicating-coding-sequences&quot; rel=&quot;nofollow noreferrer&quot;&gt;@cds constraint&lt;/a&gt; and the &lt;a href=&quot;https://edinburgh-genome-foundry.github.io/DnaChisel/genbank/genbank_api.html#forcing-changes&quot; rel=&quot;nofollow noreferrer&quot;&gt;~change objective&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Additionally, you may also optimize the codon usage to match your species with &lt;a href=&quot;https://edinburgh-genome-foundry.github.io/DnaChisel/genbank/genbank_api.html#codon-optimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;~match_codon_usage&lt;/a&gt; or one of the other options.&lt;/p&gt;&#xA;&lt;p&gt;A web interface is also provided at EGF CUBA (Collection of Useful Biological Apps): &lt;a href=&quot;https://cuba.genomefoundry.org/sculpt_a_sequence&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sculpt a Sequence&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Disclaimer: I'm the current maintainer of DNA Chisel&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2020-11-27T14:54:18.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14955" PostTypeId="2" ParentId="14952" CreationDate="2020-11-27T15:09:37.247" Score="0" Body="&lt;ol&gt;&#xA;&lt;li&gt;The fold changes for those are huge (on the order of 32x for the largest), so it's unsurprising for the adjusted p-values to be absurdly small.&lt;/li&gt;&#xA;&lt;li&gt;There's no reason to treat them differently.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-11-27T15:09:37.247" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14956" PostTypeId="2" ParentId="14951" CreationDate="2020-11-27T20:53:21.960" Score="1" Body="&lt;p&gt;According to the &lt;a href=&quot;https://github.com/torognes/vsearch/releases/download/v2.15.1/vsearch_manual.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VSEARCH docs&lt;/a&gt;, since you have specified &lt;code&gt;--sizeout&lt;/code&gt; your abundances have been written into the FASTA headers:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;--sizeout&lt;/p&gt;&#xA;&lt;p&gt;Take into account the abundance annotations present in the input fasta file (search for the pattern ’[&amp;gt;;]size=integer[;]’ in sequence headers). That option is active by default when rereplicating.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Add abundance annotations to the output fasta file (add the pattern ’;size=integer;’ to sequence headers).&lt;/strong&gt; If --sizein is specified, each unique sequence receives a new abun- dance value corresponding to its total abundance (sum of the abundances of its occur- rences). If --sizein is not specified, input abundances are set to 1, and each unique sequence receives a new abundance value corresponding to its number of occurrences in the input file.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-11-27T20:53:21.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14957" PostTypeId="2" ParentId="14882" CreationDate="2020-11-27T21:39:40.327" Score="4" Body="&lt;p&gt;Depending on what level of information you need, it is not difficult to just homebrew a plot in your plotting package of choice, as PAF has relevant coordinate information. Unless you are interested in some more complex features of a plotter such as IGV, I think that this is a good flexible solution.&lt;/p&gt;&#xA;&lt;p&gt;For example, this R code makes a dot plot of the synteny between the reference and the query:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(ggplot2)&#xA;paf = read.table(&amp;quot;out.paf&amp;quot;, sep=&amp;quot;\t&amp;quot;)&#xA;ggplot(data=paf, aes(x=V3, xend=V4, y=V8, yend=V9, color=factor(V1))) + geom_segment() + labs(x=&amp;quot;Reference coordinate&amp;quot;, y=&amp;quot;Query coordinate&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The plot looks like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2nDg1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2nDg1.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is effectively a rather minimalist version of what &lt;a href=&quot;https://github.com/dwinter/pafr&quot; rel=&quot;nofollow noreferrer&quot;&gt;pafr&lt;/a&gt; (as suggested by vkkodali) or minidot is doing.&lt;/p&gt;&#xA;&lt;p&gt;(h/t Zev K.)&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-11-27T21:39:40.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14958" PostTypeId="1" AcceptedAnswerId="14959" CreationDate="2020-11-28T13:06:26.677" Score="2" ViewCount="121" Body="&lt;p&gt;How can I create an &lt;a href=&quot;https://en.wikipedia.org/wiki/Identicon&quot; rel=&quot;nofollow noreferrer&quot;&gt;identicon&lt;/a&gt; (a visual representation) of a DNA sequence?&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2020-11-28T13:06:26.677" Title="Create identicon of a DNA sequence" Tags="|visualization|egf|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14959" PostTypeId="2" ParentId="14958" CreationDate="2020-11-28T13:06:26.677" Score="5" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/Edinburgh-Genome-Foundry/sequenticon&quot; rel=&quot;noreferrer&quot;&gt;Sequenticon&lt;/a&gt; is a Python library for generating identicons for DNA sequences. For instance, the sequence &lt;code&gt;ATGGTGCA&lt;/code&gt; gets converted into the following icon:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4dW8is.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4dW8is.png&quot; alt=&quot;ATGGTGCA&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;A web interface is also provided at EGF CUBA (Collection of Useful Biological Apps): &lt;a href=&quot;https://cuba.genomefoundry.org/render-sequenticons&quot; rel=&quot;noreferrer&quot;&gt;Render Sequenticons&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Disclaimer: I'm the current maintainer of Sequenticon&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2020-11-28T13:06:26.677" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14960" PostTypeId="2" ParentId="14740" CreationDate="2020-11-28T15:27:48.997" Score="0" Body="&lt;p&gt;In short the answer is no, however the 'car-jumping method' described &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC92527/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; it is possible, but it is about the enzymic MMLV reverse transcription in response to the capping structure. I assume that an alternative reverse transcriptase doesn't do 'car jumping' therefore any sequencing method will pick this up, but the difference between MMLV verse the heat-tolerant style reverse transcriptases. I would assume the secondary structure associated with viral capping prevents a low temperature reverse transcriptase, i.e. MMLV from correctly reproducing the vRNA into cDNA.&lt;/p&gt;&#xA;&lt;p&gt;It is important to note that a cap, isn't just a virus thing, its mRNA eukaryotes per se. The cap is a 7-methylguanosine (m7G) linked to the 5′ end of the transcript by a 5′-5′ triphosphate bridge [m7G(5′)ppp(5′)N- or m7G(5′)ppp(5′)Nm in which Nm is a 2′-O methylated nucleoside].&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-11-28T15:36:13.817" LastActivityDate="2020-11-28T15:36:13.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14962" PostTypeId="1" AcceptedAnswerId="14963" CreationDate="2020-11-28T19:44:22.673" Score="1" ViewCount="236" Body="&lt;p&gt;I have these Ct data from qPCR for a series of samples A-H, I want to quantify the RNA copies of my samples using R:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Sample_ID   Replicate1_Ct Replicate2_Ct&#xA;A           22.92         24.21           &#xA;B           24.29         22.16           &#xA;C           24.29         22.16           &#xA;D           27.91         23.76           &#xA;E           21.44         22.81           &#xA;F           16.50         18.31           &#xA;G           17.79         16.56                 &#xA;H           17.34         19.09 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In addition, I have measured a standard (STD) curve of copy number of the RNA in question, that I want to use as a reference:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;STD_Ct      STD_copies      Log10(STD_copies)&#xA;33          4.00E+03        3&#xA;28          4.00E+04        4&#xA;24          4.00E+05        5&#xA;20          4.00E+06        6&#xA;15          4.00E+07        7&#xA;12          4.00E+08        8&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It seems&lt;/p&gt;&#xA;&lt;p&gt;This the original data form:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/amPJv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/amPJv.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9709" LastEditorUserId="9709" LastEditDate="2020-11-29T18:28:24.807" LastActivityDate="2020-11-29T18:28:24.807" Title="How to calculate RNA copies in qPCR using R?" Tags="|r|statistics|qpcr|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14963" PostTypeId="2" ParentId="14962" CreationDate="2020-11-28T22:17:44.523" Score="1" Body="&lt;p&gt;Now that we've straightened out the data, it seems like the simplest solution is to just linearly interpolate the copy number from your standard curve.&lt;/p&gt;&#xA;&lt;p&gt;When I read your standard curve (&lt;code&gt;std&lt;/code&gt;) and your sample data (&lt;code&gt;ct&lt;/code&gt;) into R, I see that the log10(Ct) fits the copy number fairly well, whereas the linear is a bit harder:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2dhUM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2dhUM.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So we can fit a model to predict copy number based on Ct on the STD data:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; fit = lm(std&lt;span class=&quot;math-container&quot;&gt;$Log10.STD_copies. ~ std$&lt;/span&gt;STD_Ct)&#xA;&amp;gt; fit&#xA;&#xA;Call:&#xA;lm(formula = std&lt;span class=&quot;math-container&quot;&gt;$Log10.STD_copies. ~ std$&lt;/span&gt;STD_Ct)&#xA;&#xA;Coefficients:&#xA;(Intercept)   std$STD_Ct  &#xA;    10.6847      -0.2357  &#xA;&#xA;# we plug those numbers into the linear equation,&#xA;# taking the mean of the two replicates for a best estimate.&#xA;&#xA;&amp;gt; rowMeans(ct[,2:3]) * -.2357 + 10.6847&#xA;[1] 5.130429 5.210567 5.210567 4.595390 5.469837 6.582341&#xA;[7] 6.636553 6.391424&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So those are log10(copy number) estimates, of a sort. You can get much more elaborate, of course.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-11-28T22:17:44.523" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="14964" PostTypeId="1" AcceptedAnswerId="14965" CreationDate="2020-11-29T03:21:57.780" Score="-3" ViewCount="460" Body="&lt;p&gt;I have a blast file produced. I executed a &lt;code&gt;blast(x)&lt;/code&gt; command outputting both &lt;code&gt;&amp;quot;qeseqid&amp;quot;&lt;/code&gt; and &lt;code&gt;&amp;quot;sseqid&amp;quot;&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;QRv313_NP342_d0_h2_l9    YN13213&#xA;QRv313_NP9080_d0_h1_l1   YN5345&#xA;QRv313_NP123_d0_h1_l7    YN756&#xA;QRv313_NP123_d0_h1_l113  YN9768&#xA;QRv313_NP654_d0_h2_l6    YN432&#xA;QRv313_NP8_d0_h1_l1      YN3242&#xA;QRv313_NP756_d0_h1_l2    YN85686&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have written a command in &lt;code&gt;nano&lt;/code&gt; within &lt;code&gt;command-line&lt;/code&gt; to obtain the following desired output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;NP342    YN13213&#xA;NP9080   YN5345&#xA;NP123    YN756&#xA;NP123    YN9768&#xA;NP654    YN432&#xA;NP8_d0   YN3242&#xA;NP756    YN85686&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have written a &lt;code&gt;nano&lt;/code&gt; script to provide me a tab delimited column of my query and subject id. I am just having trouble moving forward from here. I am unsure as to how I would modify my script to provide me with my desired output.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import sys&#xA;file_object = open(sys.argv[1])&#xA;&#xA;for my_data in file_object:&#xA;&#xA;  list =  my_data.split(&amp;quot;\t&amp;quot;)&#xA;&#xA;  print (list [0], list [1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there a way t alter my command so I can receive the desired output?&lt;/p&gt;&#xA;&lt;p&gt;Any suggestions would be kindly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="9955" LastEditorUserId="9955" LastEditDate="2020-11-29T08:31:39.580" LastActivityDate="2020-11-29T20:32:22.317" Title="Creating a tab delimited column" Tags="|python|file-formats|" AnswerCount="1" CommentCount="0" ClosedDate="2020-12-01T23:54:31.063" ContentLicense="CC BY-SA 4.0" />
  <row Id="14965" PostTypeId="2" ParentId="14964" CreationDate="2020-11-29T10:47:46.420" Score="3" Body="&lt;p&gt;as far as I understand your question, you want to reformat your output file that consists of two columns:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;QRv313_NP342_d0_h2_l9    YN13213&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to extract the second &amp;quot;field&amp;quot; from your first column, where fields are delimited by &lt;code&gt;_&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;NP342    YN13213&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In general, your code is close. You could just split again by &lt;code&gt;_&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import sys&#xA;file_object = open(sys.argv[1])&#xA;&#xA;for my_data in file_object:&#xA;  list =  my_data.strip().split(&amp;quot;\t&amp;quot;)&#xA;  print (list[0].split(&amp;quot;_&amp;quot;)[1], list[1], sep=&amp;quot;\t&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now when I teach students, I usually tell them to always expect the unexpected, so you might also want to add some minimal error checking, so here's a version that checks some&#xA;minimal assumptions:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;&#xA;from sys import argv&#xA;from os import path&#xA;&#xA;usage = f&amp;quot;&amp;quot;&amp;quot;&#xA;usage: {argv[0]} &amp;lt;blast_output&amp;gt;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;&#xA;def main(in_file):&#xA;    with open(argv[1], &amp;quot;r&amp;quot;) as in_handle:&#xA;        for lno, line in enumerate(in_handle):&#xA;            line = line.strip()&#xA;            cols = line.split(&amp;quot;\t&amp;quot;)&#xA;            if len(cols) != 2:&#xA;                raise ValueError(&#xA;                    f&amp;quot;Error in line {lno}: Expected two columns,&amp;quot;&#xA;                    &amp;quot; found {len(cols)}. Offending line: {cols}&amp;quot;&#xA;                )&#xA;            query = cols[0].split(&amp;quot;_&amp;quot;)&#xA;            if len(query) != 5:&#xA;                raise ValueError(&#xA;                    f&amp;quot;Error in line {lno}: Expected five fields&amp;quot;&#xA;                    &amp;quot;in query string, found {len(query)}. Offending line {query}&amp;quot;&#xA;                )&#xA;            print(query[1], cols[1], sep=&amp;quot;\t&amp;quot;)&#xA;&#xA;&#xA;if __name__ == &amp;quot;__main__&amp;quot;:&#xA;    if len(argv) != 2:&#xA;        print(usage)&#xA;        raise ValueError(&amp;quot;Illegal number of arguments&amp;quot;)&#xA;    if not path.exists(argv[1]):&#xA;        raise FileNotFoundError(f&amp;quot;File {argv[1]} could not be found. Check spelling&amp;quot;)&#xA;&#xA;    main(argv[1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And this is what it produces:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;python3 ex.py in.txt&#xA;NP342   YN13213&#xA;NP9080  YN5345&#xA;NP123   YN756&#xA;NP123   YN9768&#xA;NP654   YN432&#xA;NP8     YN3242&#xA;NP756   YN85686&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9693" LastEditorUserId="9693" LastEditDate="2020-11-29T20:32:22.317" LastActivityDate="2020-11-29T20:32:22.317" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14966" PostTypeId="1" AcceptedAnswerId="14969" CreationDate="2020-11-29T17:28:43.130" Score="1" ViewCount="744" Body="&lt;p&gt;I am working with a set of RNASeq dataset. I have about 4000 observations (genes) on 20 samples and plotting a PCA I found the clustering doesn't vary much when I use different number of genes, but the % variance of PC1 and PC2 does. I have ~1300 DEG. But how do you select a sensible cutoff for the number of genes/observations to explain PCA variance? Would it be correct to select the number of genes that are DEG which account for &amp;gt;50% of the variance between PC1 and 2?&lt;/p&gt;&#xA;&lt;p&gt;Few examples:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;All genes: PC1(34%), PC2(14%)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Top1000: PC1(45%), PC2(17%)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Top500: PC1(49%), PC2(19%)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Top50: PC1(55%), PC2(24%)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Top 5: PC1(75%), PC2(24%)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9967" LastActivityDate="2020-11-30T08:34:10.510" Title="What is a sensible number of gene/observations to explain PCA variance?" Tags="|rna-seq|deseq2|pca|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="14969" PostTypeId="2" ParentId="14966" CreationDate="2020-11-30T08:34:10.510" Score="3" Body="&lt;p&gt;The general goal of PCA in RNA-seq can be stated as, &amp;quot;I'd like a low-dimension representation of my data to allow easy assessment of the gross structure of my samples, specifically for assessing missing batch effects, samples swaps, etc.&amp;quot; Ideally, this low-dimension representation can fit into a plot or two. In other words, we'd like the first 2 (maybe 3) PCs to explain a reasonable amount of variation. A few things to note from this alone:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The number of differentially expressed genes were not mentioned. They have no relevance here.&lt;/li&gt;&#xA;&lt;li&gt;There was no mention made of the treatment groups clustering together. Seeing this is &lt;strong&gt;NOT&lt;/strong&gt; a goal of PCA.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;As you have noticed, the more input rows (genes, assuming you haven't already transposed it) the less variation explained by the first 2 principal components (PCs). On the flip side, the more rows you input the better PCA is describing the structure of your entire dataset.&lt;/p&gt;&#xA;&lt;p&gt;Question: Do I really care about the gross structure of the entire dataset?&lt;/p&gt;&#xA;&lt;p&gt;Answer: No, I just want to see problems that I might need to account for.&lt;/p&gt;&#xA;&lt;p&gt;So you really don't have to use all of the data in the PCA, just the bits that will allow you to see any issues. In general that'll be the few hundred most-variable genes (after some reasonable transformation so you're not just looking at the most highly expressed genes!). &lt;strong&gt;The exact number shouldn't really matter very much, something in the range of 300-500 most variable genes should suffice.&lt;/strong&gt; A common practice is to just randomly pick a nice round number somewhere in or around this range. This is not a number that needs optimization unless perhaps you have a very large number of samples and therefore PC1 and PC2 only account for a small (say 10-20%) percentage of the variation, in which case you're better off looking at either a number of difference principal components or using a different dimension reduction technique, such as UMAP.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-11-30T08:34:10.510" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14970" PostTypeId="2" ParentId="14934" CreationDate="2020-11-30T11:21:09.763" Score="1" Body="&lt;p&gt;Is there any conserved secondary structure in this RNA? One could align all known instances of this RNA and see if there is any significant covariation according to &lt;a href=&quot;http://eddylab.org/R-scape&quot; rel=&quot;nofollow noreferrer&quot;&gt;R-scape&lt;/a&gt;. That could serve as a computational signal that the RNA is real (although wet lab experiments would still be needed).&lt;/p&gt;&#xA;&lt;p&gt;Another question is whether this RNA is already in &lt;a href=&quot;https://rfam.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rfam&lt;/a&gt;? If so, then the job of making an alignment of known instances is easier.&lt;/p&gt;&#xA;" OwnerUserId="1045" LastActivityDate="2020-11-30T11:21:09.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14972" PostTypeId="2" ParentId="11412" CreationDate="2020-11-30T17:21:27.120" Score="0" Body="&lt;p&gt;If you need to engineer a trait, then it has not evolved, so it probably does not confer a selectional advantage to the organism.&lt;/p&gt;&#xA;&lt;p&gt;So a fingerprint of an engineered organism is that the genetic difference between it and its relatives does not have a selectional explanation.&lt;/p&gt;&#xA;" OwnerUserId="9972" LastActivityDate="2020-11-30T17:21:27.120" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14973" PostTypeId="1" AcceptedAnswerId="14980" CreationDate="2020-12-01T00:10:15.717" Score="1" ViewCount="22" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gqqRM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gqqRM.png&quot; alt=&quot;denovoParent&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In that example, 129+216+327 does not equal the parent's mass of 492.&#xA;My understanding is that during MS, the protein is broken down into peptide masses of varying length of approx. 5 to 20 amino acids. They also lose ions such as water or ammonia like in the example., the experimental spectra I thought would show the masses of these &amp;quot;children&amp;quot; that can be summed up to the parent (if you also add all the ammonia and water ions that are broken off). But that is not the case. Where am I wrong in my understanding?&lt;/p&gt;&#xA;" OwnerUserId="9650" LastActivityDate="2020-12-01T19:11:13.143" Title="Why don't the &quot;child&quot; peptide masses sum to the parent in this De novo sequencing example?" Tags="|mass-spectrometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14974" PostTypeId="1" CreationDate="2020-12-01T05:31:11.220" Score="2" ViewCount="674" Body="&lt;p&gt;I made a wdl (genomics pipeline language by the Broad Institute) task for fastqc that outputs the zip files, of which there will be as many as reads that the sample I'm working on at that moment has. We create the output using glob:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  output {&#xA;    Array[File] fastqcArray = glob(&amp;quot;*.zip&amp;quot;)&#xA;  } &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to give the folder a name that is representative of what it contains, for example &lt;code&gt;fastqc&lt;/code&gt;, rather than glob-xxxxxxx, so that it's clear what it is in my task's output folder. Is this possible, and if so, how can I achieve it?&lt;/p&gt;&#xA;" OwnerUserId="9973" LastEditorUserId="9973" LastEditDate="2020-12-02T22:41:18.543" LastActivityDate="2022-04-28T02:02:50.290" Title="How do I rename output folder created using WDL's glob function, with cromwell execution engine?" Tags="|wdl|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14975" PostTypeId="1" CreationDate="2020-12-01T07:25:51.817" Score="0" ViewCount="170" Body="&lt;p&gt;Figure from this &lt;a href=&quot;https://www.embopress.org/doi/pdf/10.15252/embj.201591279&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; which shows a profile plot in figure4 first figure.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/sASxO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sASxO.png&quot; alt=&quot;figure&quot; /&gt;&lt;/a&gt;.This i guess is made using deeptools perhaps. Meanwhile&lt;/p&gt;&#xA;&lt;p&gt;When i tried to do similar plot &lt;a href=&quot;https://i.stack.imgur.com/C4awO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/C4awO.png&quot; alt=&quot;figure 2&quot; /&gt;&lt;/a&gt; the scale doesn't start from 0. So my question how did they set the scale from 0 or the data is as such in other words their data point starts from 0. In my case it starts above 0. So is there a way to scale or normalize to make it start from 0 as i see in the embo paper or its just the characteristics of the data itself generated.&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2020-12-01T07:25:51.817" Title="Setting scale of profile plot from zero for chip seq profile plot" Tags="|deeptools|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="14976" PostTypeId="1" AcceptedAnswerId="14977" CreationDate="2020-12-01T13:42:29.240" Score="0" ViewCount="31" Body="&lt;p&gt;I have expression data for 100 samples and clinical data for 50 among them. I want to carryout k-means clustering using expression data and then do a survival analysis (Kaplan-Meier analysis) using the labels obtained from k-means.&lt;/p&gt;&#xA;&lt;p&gt;Can I used the expression data of all 100 samples to carryout clustering or should I use only 50 samples for clustering?&lt;/p&gt;&#xA;" OwnerUserId="2247" LastActivityDate="2020-12-01T14:57:52.570" Title="Picking number of samples for clustering" Tags="|clustering|" AnswerCount="1" CommentCount="1" ClosedDate="2020-12-01T21:36:05.600" ContentLicense="CC BY-SA 4.0" />
  <row Id="14977" PostTypeId="2" ParentId="14976" CreationDate="2020-12-01T14:57:52.570" Score="1" Body="&lt;p&gt;In case there is no systematic difference between the samples with or without the clinical data (i.e. one set is microarray and the other is RNA-seq), I would use all of the samples. And if the cluster labels obtained with &amp;quot;clustering 50 samples&amp;quot; and those obtained with &amp;quot;clustering 100 samples&amp;quot; does not differ too much, then doing either of two would not have much effect anyway. If you would like to compare cluster labels in that way and assuming you would want to test a number of &lt;code&gt;k&lt;/code&gt;s, it would be better to compare each combination of &lt;code&gt;k&lt;/code&gt;s and number of samples.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2020-12-01T14:57:52.570" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14978" PostTypeId="1" AcceptedAnswerId="14979" CreationDate="2020-12-01T17:01:51.283" Score="1" ViewCount="713" Body="&lt;p&gt;De novo genome assembly for non-model organisms face the issue of bacterial contamination. For assembled contigs with mostly bacterial-like sequences (based on BLAST search), the entire contig can be dropped off. But for a long contig with small fragments of bacterial-like sequences, one way to solve it is to break the contigs into smaller pieces and drop the bacterial-like pieces. It would work better if the raw reads could be removed BEFORE assembly. This was commonly done for Illumina reads, but for PacBio long reads (non-HiFi reads) with a higher error rate, is it possible to filter contaminated bacterial sequences based on sequence similarity, like BLAST, Kraken? Of course, the potentially contaminated region should be long enough to drop the entire sequence, for PacBio long reads. Maybe 30%? Any suggestion of the processing details?&lt;/p&gt;&#xA;" OwnerUserId="9975" LastActivityDate="2021-01-29T23:34:42.847" Title="Is it possible to filter contaminated reads for raw PacBio sequences (not HiFi reads) before assembly?" Tags="|genome|assembly|long-reads|pacbio|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14979" PostTypeId="2" ParentId="14978" CreationDate="2020-12-01T19:01:34.320" Score="1" Body="&lt;p&gt;I don't think that in principle the issue is any different for PacBio and Illumina; it's just that the process might be noisier with higher error rates. Note that &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5553090/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PacBio errors aren't randomly distributed substitutions&lt;/a&gt; and probably many (most?) short-ish k-mers (&amp;lt;30nt) will likely be intact. I would suggest doing k-mer filtering of your reads as suggested &lt;a href=&quot;https://www.biostars.org/p/12988/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; or &lt;a href=&quot;https://www.researchgate.net/post/What_methods_programs_are_recommended_to_clean-up_a_genome_from_contaminating_reads&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; or &lt;a href=&quot;https://www.biostars.org/p/130/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Alternately, use your favorite alignment-based technique, if you have a good idea of what your contaminant is. In principle PB reads will be &lt;em&gt;better&lt;/em&gt; filtered by this method than Illumina as local alignment will be able to figure out the errors quite well with longer queries.&lt;/p&gt;&#xA;&lt;p&gt;If you go the route of trying to do alignment to some set of contaminant references, the obvious answer would be to use &lt;a href=&quot;https://github.com/lh3/minimap2&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimap2&lt;/a&gt; as your aligner as this is exactly the task that it is designed for. You can then use &lt;a href=&quot;http://www.htslib.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools&lt;/a&gt; to filter out anything that maps well.&lt;/p&gt;&#xA;&lt;p&gt;Here is a related question/answer that also addresses the contaminant issue and suggests using &lt;a href=&quot;https://github.com/blaxterlab/blobology&quot; rel=&quot;nofollow noreferrer&quot;&gt;Blobology&lt;/a&gt;, which I'm not familiar with but may be helpful (granted it also seems to be Illumina-centric). Genome-size estimation using k-mers on PB reads appears to be pretty accurate too, so I think that it's likely ok to work with raw PB k-mers.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2020-12-01T22:00:36.590" LastActivityDate="2020-12-01T22:00:36.590" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14980" PostTypeId="2" ParentId="14973" CreationDate="2020-12-01T19:11:13.143" Score="1" Body="&lt;p&gt;In a peptide tandem mass spectrometry experiment, each fragment (child) ion is typically the result of breaking a single bond in the peptide backbone yield b and y ions. Because each ion fragment ion is the result of one fragmentation event, we don't expect the fragment ion masses to sum to the precursor mass.&lt;/p&gt;&#xA;&lt;p&gt;If your peptide was fragmented well, you should get a ladder of b and y ions where the differences in masses between ions in the b and y ion series correspond to the masses of amino acids (+/- any modifications and neutral losses, such as the NH3 and H2O mentioned above).&lt;/p&gt;&#xA;&lt;p&gt;A great resource for understanding these concepts is this tutorial:&lt;br /&gt;&#xA;&lt;a href=&quot;https://www.ionsource.com/tutorial/DeNovo/introduction.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ionsource.com/tutorial/DeNovo/introduction.htm&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9197" LastActivityDate="2020-12-01T19:11:13.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14981" PostTypeId="1" AcceptedAnswerId="14982" CreationDate="2020-12-01T20:43:33.157" Score="1" ViewCount="230" Body="&lt;p&gt;I have a data file named &amp;quot;geneexp.csv&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;the data contains information about gene expression of three different cell types (CD4 and CD8, CD19) I want to classify cells by performing the nearest shrunken centroid classification of training data in which the threshold is chosen by cross-validation. I split the data (70% train and 30% test).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data = read.csv(&amp;quot;geneexp.csv&amp;quot;)&#xA;&#xA;splitData &amp;lt;- function(data, trainRate) {&#xA;  n &amp;lt;- dim(data)[1]&#xA;  idxs &amp;lt;- sample(1:n, floor(trainRate*n))&#xA;  train &amp;lt;- data[idxs,]&#xA;  test &amp;lt;- data[-idxs,]&#xA;  return (list(train = train, test = test))&#xA;}&#xA;&#xA;split &amp;lt;- splitData(data, .7)&#xA;train &amp;lt;- split&lt;span class=&quot;math-container&quot;&gt;$train&#xA;test  &amp;lt;- split$&lt;/span&gt;test&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;then with the use of &lt;code&gt;pamr&lt;/code&gt; package I tried to buid the following model and plot :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;y &amp;lt;- train[[ncol(train)]]&#xA;x &amp;lt;- t(train[,-ncol(train)]) &#xA;mydata &amp;lt;- list(&#xA;  x = x, &#xA;  y = as.factor(as.factor(y)),&#xA;  geneid = as.character(1:nrow(x)),&#xA;  genenames = rownames(x)&#xA;)&#xA;&#xA;# Training and cross-validating threshold&#xA;model &amp;lt;- pamr.train(mydata)&#xA;cvmodel &amp;lt;- pamr.cv(model, mydata)&#xA;pamr.plotcv(cvmodel)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but I can't make it work. I get the following error:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error in &lt;code&gt;contrasts&amp;lt;-&lt;/code&gt;(&lt;code&gt;*tmp*&lt;/code&gt;, value = contr.funs[1 + isOF[nn]]) :&lt;br /&gt;&#xA;contrasts can be applied only to factors with 2 or more levels&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I have already transfer y to the factors. Can you help me? How can I fix it?&lt;/p&gt;&#xA;" OwnerUserId="9978" LastEditorUserId="9978" LastEditDate="2020-12-01T23:07:21.683" LastActivityDate="2020-12-01T23:33:51.343" Title="how to train a gene dataset with a nearest shrunken centroid classifier?" Tags="|r|gene|genome|machine-learning|data-preprocessing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14982" PostTypeId="2" ParentId="14981" CreationDate="2020-12-01T23:33:51.343" Score="0" Body="&lt;p&gt;It's a bit different from usual R, but if you check the help:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;data: The input data. A list with components: x- an expression genes&#xA;in the rows, samples in the columns)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So in your case, you need to transpose the matrix:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mydata &amp;lt;- list(&#xA;  x = t(x), &#xA;  y = as.factor(y),&#xA;  geneid = as.character(1:nrow(x)),&#xA;  genenames = rownames(x)&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As I don't have your amazing data, I can only use iris below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pamr.train(list(x=t(iris[,1:4]),y=iris[,5]))&#xA;&#xA;pamr.train(data = list(x = t(iris[, 1:4]), y = iris[, 5]))&#xA;   threshold nonzero errors&#xA;1   0.000    4       6     &#xA;2   0.841    4       7     &#xA;3   1.682    4       10    &#xA;4   2.523    4       11    &#xA;5   3.364    4       13    &#xA;6   4.205    4       18    &#xA;7   5.046    3       22    &#xA;8   5.887    3       23    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-12-01T23:33:51.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14983" PostTypeId="1" CreationDate="2020-12-02T05:01:02.890" Score="1" ViewCount="189" Body="&lt;p&gt;I want to perform pairwise alignment and get the resulting CIGAR string (Concise Idiosyncratic Gapped Alignment Report). A fuller description being &lt;a href=&quot;https://www.drive5.com/usearch/manual/cigar.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The channel can emit insertions, deletions and substitution errors.&lt;/p&gt;&#xA;&lt;p&gt;For example, if the input was &lt;em&gt;AAGCT&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then the output could be &lt;em&gt;AAAAGGC&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where there were the following errors in the following positions.&lt;/p&gt;&#xA;&lt;p&gt;1, Insertion of A at first position&#xA;2, Insertion of A at second position&#xA;3, Insertion of G at the third position&#xA;4, Deletion of T at the last position&lt;/p&gt;&#xA;&lt;p&gt;I want to know how I could obtain the CIGAR string given two sequences. In addition, could I use viterbi algorithm for this?&lt;/p&gt;&#xA;" OwnerUserId="9982" LastEditorUserId="3848" LastEditDate="2022-07-17T17:07:31.877" LastActivityDate="2023-09-10T17:07:50.890" Title="Pairwise alignment using CIGAR String" Tags="|sequence-alignment|phylogenetics|errors|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14984" PostTypeId="1" AcceptedAnswerId="14985" CreationDate="2020-12-02T09:51:12.160" Score="2" ViewCount="406" Body="&lt;pre&gt;&lt;code&gt;library(ChIPseeker)&#xA;library(TxDb.Hsapiens.UCSC.hg38.knownGene)&#xA;txdb &amp;lt;- TxDb.Hsapiens.UCSC.hg38.knownGene&#xA;library(clusterProfiler)&#xA;library(org.Hs.eg.db)&#xA;&#xA;peaks &amp;lt;- readPeakFile(&amp;quot;MACS_folder/pol2_chip/Pol_2pausing/pol2_diff_peak.bed&amp;quot;)&#xA;&#xA;print(peaks)&#xA;&#xA;&#xA;#peakAnno &amp;lt;- annotatePeak(&amp;quot;merge/blast.bed&amp;quot;, tssRegion=c(-3000, 3000),&#xA;#                        TxDb=txdb, annoDb=&amp;quot;org.Hs.eg.db&amp;quot;)&#xA;&#xA;peakAnno &amp;lt;- annotatePeak(peaks, tssRegion=c(-1000, 1000),&#xA;                         TxDb=txdb, annoDb=&amp;quot;org.Hs.eg.db&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So this is basically what im doing to annotate the peaks.&lt;/p&gt;&#xA;&lt;p&gt;Now the issue comes with annotation for example I see TLR4 annotated to this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;118082896   118083202   307 *   shp1_R1_peak_679    Exon (ENST00000413759.1/ENST00000413759.1, exon 1 of 1) 9   117708197   117708979   783 1   7099    ENST00000490685.1   374699  ENSG00000136869 TLR4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now The peak is 679 as you can see from the IGV plot as well as annotated file. But when i see in IGV browser TLR4 is far away from the peak where i get the annotation.&lt;/p&gt;&#xA;&lt;p&gt;So now is it an issue with the code im using or due to the inherent nature of the data.&lt;/p&gt;&#xA;&lt;p&gt;Any help or suggestion would be really appreciated.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fnIzD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fnIzD.png&quot; alt=&quot;TLR4&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2020-12-02T10:28:21.107" Title="chipseeker annotation issue" Tags="|ngs|chip-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14985" PostTypeId="2" ParentId="14984" CreationDate="2020-12-02T10:18:10.210" Score="2" Body="&lt;p&gt;I had a similar case just recently.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;tssRegion&lt;/code&gt; argument ChIPseeker is used to define what is considered a promoter, which is frequently the question at hand. However, ChIPseeker will not perform any kind of filtering on the annotated peak file.&lt;/p&gt;&#xA;&lt;p&gt;In the results there is a column &lt;code&gt;annotation&lt;/code&gt; that contains information what kind of genomic region the peak is in. If it is within your &lt;code&gt;tssRegion&lt;/code&gt; window it will be labeled as a promoter.&lt;br /&gt;&#xA;In your case the annotation is &lt;code&gt;1st Exon = Exon (ENST00000413759.1/ENST00000413759.1, exon 1 of 1) 9&lt;/code&gt;. This tells you that the peak is not part of the promoter region you defined and located within the first exon of the indicated transcript. The peak is really far away from TLR4 (374699 bp) but I think all other sequences nearby are pseudogenes and this is why TLR4 is selected as the closest gene.&lt;/p&gt;&#xA;&lt;p&gt;You can get an overview over the annotations using &lt;code&gt;plotAnnoPie(peakAnno)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;tl;dr: you need to filter the results of the ChIPseeker annotation to get the genomic regions you want.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastEditorUserId="4885" LastEditDate="2020-12-02T10:28:21.107" LastActivityDate="2020-12-02T10:28:21.107" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="14988" PostTypeId="1" AcceptedAnswerId="15014" CreationDate="2020-12-02T20:39:22.717" Score="0" ViewCount="45" Body="&lt;p&gt;I am frequently using a ballgown package for my rnaseq analysis, but recently I have had a new task to have my reads mapped on two different genomes to understand the level of alignment between the two, therefore I wonder what is the best tool to do this? I have single-end reads.&#xA;What I need is to have a percentage of uniquely mapped fragments, mapped to multiple loci and, say, unmapped. Can &lt;code&gt;bowtie&lt;/code&gt; do the job? Is there a standard explanation procedure explained elsewhere? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="6948" LastActivityDate="2020-12-06T20:04:50.557" Title="What is the best way to map/align my reads on a given genome?" Tags="|rna-seq|sequence-alignment|rna-alignment|bowtie2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14990" PostTypeId="1" CreationDate="2020-12-03T05:28:12.870" Score="1" ViewCount="416" Body="&lt;p&gt;My third year comp sci friend got me this far, I hope you guys can help me with the last inch! For the record, I have practically no coding experience. I am using the 3.9.4 alpha version of MUMmer.&lt;/p&gt;&#xA;&lt;p&gt;Here is the manual I have been following: &lt;a href=&quot;http://mummer.sourceforge.net/examples/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mummer.sourceforge.net/examples/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am at part 2.2.2 Running the Mummer plot, and I type in the final line to generate the plot.&lt;/p&gt;&#xA;&lt;p&gt;I type:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;mummerplot -x &amp;quot;[0,8666124]&amp;quot; -y &amp;quot;[0,9025688]&amp;quot; -postscript -p mummer mummer.mums&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;And the error is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Can't use 'defined(%hash)' (Maybe you should omit the defined ()?) at /usr/bin/mummerplot line 884&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Line at 884:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;if ( defined (%&lt;span class=&quot;math-container&quot;&gt;$rref) ) {                                                                                                            #-- skip reference sequence or set atts from hash&#xA;        if ( !exists ($rref-&amp;gt;{$idR}) ) { next; }&#xA;        else { ($refoff, $reflen, $refdir) = @{$rref-&amp;gt;{$&lt;/span&gt;idR}}; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Its perl&lt;/p&gt;&#xA;" OwnerUserId="9988" LastEditorUserId="9988" LastEditDate="2020-12-03T14:00:04.873" LastActivityDate="2022-02-07T11:29:10.723" Title="MUMmer plot error. Line 884. What to do?" Tags="|errors|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14991" PostTypeId="1" AcceptedAnswerId="14996" CreationDate="2020-12-03T09:39:44.590" Score="3" ViewCount="523" Body="&lt;p&gt;I have the following two rules:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule extract:&#xA;    input: &amp;quot;data/{dir}/{file}.gz&amp;quot;&#xA;    output: &amp;quot;data/{dir}/{file,^.*(?&amp;lt;!\.gz)$}&amp;quot;&#xA;    conda: &amp;quot;config/conda-extract-env.yml&amp;quot;&#xA;    shell: &amp;quot;cd 'data/{wildcards.dir}'; gunzip -k {wildcards.file}.gz&amp;quot;&#xA;&#xA;rule extract_dot:&#xA;    input: &amp;quot;data/{dir}/wtdbg2.3.dot.gz&amp;quot;&#xA;    output: &amp;quot;data/{dir}/wtdbg2.3.dot&amp;quot;&#xA;    conda: &amp;quot;config/conda-extract-env.yml&amp;quot;&#xA;    shell: &amp;quot;cd 'data/{wildcards.dir}'; gunzip -k wtdbg2.3.dot.gz&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The second one is actually a hack because the first does not work on files named &lt;code&gt;wtdbg2.3.dot&lt;/code&gt;.&#xA;I have also tried giving the constraint using the &lt;code&gt;wildcard_constraints&lt;/code&gt; member of the rule, and prefixed the string with an &lt;code&gt;r&lt;/code&gt; or not.&lt;/p&gt;&#xA;&lt;p&gt;Whatever I do, as soon as I uncomment the &lt;code&gt;extract_dot&lt;/code&gt; rule, snakemake complains about the missing input file &lt;code&gt;data/C.elegans/wtdbg2.3.dot&lt;/code&gt; in another rule. Note that there is a rule with output &lt;code&gt;data/C.elegans/wtdbg2.3.dot.gz&lt;/code&gt;, and when changing this rule to output &lt;code&gt;data/C.elegans/wtdbg2.3.dot&lt;/code&gt; instead, everything works as well.&lt;/p&gt;&#xA;&lt;p&gt;So I conclude that the problem is in the regex. I have tested the regex on &lt;a href=&quot;https://regex101.com/r/lwxU3e/1&quot; rel=&quot;nofollow noreferrer&quot;&gt;regex101.com&lt;/a&gt;, and it accepts &lt;code&gt;wtdbg2.3.dot&lt;/code&gt;. But snakemake seems not to. Does anyone know if there is anything special about the snakemake regexes? Should they not be normal python regexes?&lt;/p&gt;&#xA;" OwnerUserId="9255" LastActivityDate="2020-12-03T16:27:35.300" Title="Snakemake wildcard constraints reject too much" Tags="|snakemake|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14992" PostTypeId="1" AcceptedAnswerId="14993" CreationDate="2020-12-03T10:05:02.827" Score="2" ViewCount="461" Body="&lt;p&gt;I am trying to create a shiny board for my scRNA seq data. I have saved my seurat objects adn I use following code as below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(shiny)&#xA;library(shinythemes)&#xA;library(Seurat)&#xA;library(uwot)&#xA;&#xA;&#xA;set_1 &amp;lt;- readRDS(&amp;quot;/temp2/data/mydataB10.rds&amp;quot;)&#xA;set_2 &amp;lt;- readRDS(&amp;quot;/temp2/data/mydataB5.rds&amp;quot;)&#xA;&#xA;ui &amp;lt;- fluidPage(&#xA;    &#xA;    # Application title&#xA;    titlePanel(&amp;quot;scRNAseq data browser&amp;quot;),&#xA;    &#xA;    fluidRow(&#xA;        column(4,&#xA;               &#xA;               sidebarPanel(&#xA;                   radioButtons(&amp;quot;dataset&amp;quot;, label = h3(&amp;quot;Dataset&amp;quot;),&#xA;                                choices = list(&amp;quot;sample1&amp;quot; = &amp;quot;set_1&amp;quot;, &amp;quot;sample2&amp;quot; = &amp;quot;set_2&amp;quot;),&#xA;                                selected = &amp;quot;set_1&amp;quot;),&#xA;                   helpText(&amp;quot;Enter gene names in CAPS.  Gene names must be exact.&amp;quot;),&#xA;                   textInput(&amp;quot;gene2&amp;quot;, label = &amp;quot;Gene Name&amp;quot;, value = &amp;quot;GAPDH&amp;quot;)&#xA;                   &#xA;               )&#xA;        ),&#xA;        column(8,&#xA;               column(6, &#xA;                      mainPanel(&#xA;                          plotOutput(&amp;quot;dimPlot1&amp;quot;, width = &amp;quot;150%&amp;quot;),&#xA;                          plotOutput(&amp;quot;genePlot2&amp;quot;, width = &amp;quot;150%&amp;quot;)&#xA;                      )&#xA;               )&#xA;        )&#xA;    )&#xA;)&#xA;&#xA;&#xA;server &amp;lt;- function(input, output, session) {&#xA;    &#xA;    datasetInput &amp;lt;- reactive({&#xA;        df &amp;lt;- (input$dataset)&#xA;            })&#xA;    &#xA;    output$dimPlot1 &amp;lt;- renderPlot({&#xA;        DimPlot(datasetInput(), reduction = &amp;quot;umap&amp;quot;)&#xA;    })&#xA;    &#xA;    output&lt;span class=&quot;math-container&quot;&gt;$genePlot2 &amp;lt;- renderPlot({&#xA;        FeaturePlot(datasetInput(), features = (input$&lt;/span&gt;gene2), reduction = &amp;quot;umap&amp;quot;)&#xA;    })&#xA;    &#xA;}&#xA;&#xA;shinyApp(ui, server)  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That error that I get on the section where image needs to be displayed is &amp;quot;Subscript out of bound&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Following dataset can be downloaded and used as an example &lt;code&gt;curl -Lo pbmc_10k_v3.rds https://www.dropbox.com/s/3f3p5nxrn5b3y4y/pbmc_10k_v3.rds?dl=1&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can anyone help in fixing this shiny script so that it works.&lt;/p&gt;&#xA;&lt;p&gt;Many thanks.&lt;/p&gt;&#xA;" OwnerUserId="5062" LastEditorUserId="5062" LastEditDate="2020-12-03T10:55:07.163" LastActivityDate="2020-12-03T10:55:07.163" Title="shiny code with seurat object" Tags="|r|scrnaseq|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14993" PostTypeId="2" ParentId="14992" CreationDate="2020-12-03T10:37:09.583" Score="3" Body="&lt;p&gt;Not easy to reproduce your error but here is my guess. This is a string, you can check it:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;input&lt;span class=&quot;math-container&quot;&gt;$dataset&#xA;class(input$&lt;/span&gt;dataset)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is a simple solution. If you want to call out the dataset depending on input, use a list:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;set_1 &amp;lt;- readRDS(&amp;quot;/temp2/data/mydataB10.rds&amp;quot;)&#xA;set_2 &amp;lt;- readRDS(&amp;quot;/temp2/data/mydataB5.rds&amp;quot;)&#xA;data_list = list(set_1=set_1,set_2=set_2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then in the server :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;datasetInput &amp;lt;- reactive({&#xA;        df &amp;lt;- data_list[[input$dataset]]&#xA;            })&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-12-03T10:37:09.583" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="14994" PostTypeId="1" CreationDate="2020-12-03T11:39:54.530" Score="0" ViewCount="544" Body="&lt;p&gt;i try to implement LIMMA for paired samples in order to compare gene expression before/after treatment. But...i'm not confident in my results since every single gene expression seems to be significantly associeted with the treatment!&lt;/p&gt;&#xA;&lt;p&gt;Here is my methodology:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;mat&amp;lt;-t(df)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;where df is a dataframe with:&#xA;-1000 columns for different gene expression&#xA;-+ 1 column 'id' (10 different id, repeated 2 times)&#xA;-+ 1 column 'ttt' (0=before, 1=after treatment)&#xA;-20 rows of observations (10 patients observed twice)&lt;/p&gt;&#xA;&lt;p&gt;So once I get my matrix:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;subject_id&amp;lt;-df$id&lt;/code&gt; A vector with 10 different id repeated 2 times&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;ttt&amp;lt;-as.factor(df$ttt)&lt;/code&gt; A vector with the treatment status (before/after)&lt;/p&gt;&#xA;&lt;p&gt;Here is my design matrix:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;design&amp;lt;-model.matrix(~ 0+ttt)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;colnames(design) &amp;lt;- levels(ttt)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;As asked in limma() description i specify the correlations:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;corfit &amp;lt;- duplicateCorrelation(mat,design,block=subject_id)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Fiting of the model:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;fit&amp;lt;-lmFit(mat,design,block =subject_id,correlation = corfit$consensus)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;fit2&amp;lt;-eBayes(fit)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Getting adjusted pvalues for every gene comparison before/after treatment:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;topTable(fit2, adjust.method='BH',number=1000)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;But every single gene expression seems to be relevant, so i'm very doubtful about my methodology&#xA;Any idea?&#xA;Many thanks!&lt;/p&gt;&#xA;" OwnerUserId="9989" LastActivityDate="2020-12-03T11:39:54.530" Title="Troubles implementing LIMMA for paired samples (before/after treatment) comparaisons" Tags="|gene-expression|limma|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="14995" PostTypeId="1" CreationDate="2020-12-03T11:56:44.500" Score="0" ViewCount="50" Body="&lt;p&gt;I would like to find a few genes that are similar in length to a gene of interest, and by that I mean the total length of a canonical transcript including introns. This is proving to be surprisingly hard using web based lookups.&lt;/p&gt;&#xA;&lt;p&gt;The gene of interest is 14.5Kb long and I wanted a table of genes and their canonical lengths to compare and contrast to in order to pick a few to have a look at in some vcfs I have.&lt;/p&gt;&#xA;&lt;p&gt;I have a low bioinformatics ability but happy to give things a go!&lt;/p&gt;&#xA;&lt;p&gt;All the best&lt;/p&gt;&#xA;&lt;p&gt;I have tried:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;first download a file of all gene files from ensembl which looks like this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/GJrPI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GJrPI.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Then in R:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;x &amp;lt;- read.delim(&amp;quot;all_genes_bed.txt&amp;quot;)&#xA;x&lt;span class=&quot;math-container&quot;&gt;$length &amp;lt;- (x$&lt;/span&gt;Gene.end..bp.-x&lt;span class=&quot;math-container&quot;&gt;$Gene.start..bp.)&#xA;x &amp;lt;- subset(x, x$&lt;/span&gt;length &amp;gt;14000 &amp;amp; x$length &amp;lt;15000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then I just had a look in the RStudio browser.&lt;/p&gt;&#xA;&lt;p&gt;That seemed to work!&lt;/p&gt;&#xA;" OwnerUserId="9990" LastEditorUserId="9990" LastEditDate="2020-12-03T15:37:12.100" LastActivityDate="2020-12-03T15:37:12.100" Title="Finding a gene of similar length" Tags="|gene|ensembl|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="14996" PostTypeId="2" ParentId="14991" CreationDate="2020-12-03T16:02:25.517" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Does anyone know if there is anything special about the snakemake regexes? Should they not be normal python regexes?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No, nothing special.  The &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#wildcards&quot; rel=&quot;nofollow noreferrer&quot;&gt;Snakemake documentation&lt;/a&gt; just links to the &lt;a href=&quot;https://docs.python.org/3/library/re.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Python re module documentation&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I think this is something strange in how the anchors &lt;code&gt;^&lt;/code&gt; and &lt;code&gt;$&lt;/code&gt; behave when matching wildcards, and not anything else in your particular regular expression.  Check out the pattern-matching with this dummy Snakefile:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule all:&#xA;    input: [&amp;quot;data/subdir/example&amp;quot;, &amp;quot;data/example&amp;quot;, &amp;quot;example&amp;quot;]&#xA;&#xA;rule anchored_example_of_subdir:&#xA;    &amp;quot;&amp;quot;&amp;quot;Can't add either ^ or $ or this will fail&amp;quot;&amp;quot;&amp;quot;&#xA;    output: &amp;quot;data/{subdir,subdir}/example&amp;quot;&#xA;&#xA;rule anchored_example_in_subdir:&#xA;    &amp;quot;&amp;quot;&amp;quot;Can't add ^ or this wil fail&amp;quot;&amp;quot;&amp;quot;&#xA;    output: &amp;quot;data/{file,example$}&amp;quot;&#xA;&#xA;rule anchored_example:&#xA;    &amp;quot;&amp;quot;&amp;quot;This works with both ^ and &lt;span class=&quot;math-container&quot;&gt;$&quot;&quot;&quot;&#xA;output: &quot;{file,^example$&lt;/span&gt;}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(There are no inputs or actions so it's just a matter of what filenames match what rules, and the regular expressions do nothing but match exact strings.)  This &amp;quot;works&amp;quot; as is (&lt;code&gt;snakemake -nrp&lt;/code&gt; shows what you expect) but if you add in anchors as described in the docstrings above the upper two rules no longer match the expected filenames.  It's as if (and maybe is?) under the hood Snakemake is trying to apply the expression to the entire path.&lt;/p&gt;&#xA;&lt;p&gt;Can you just drop your &lt;code&gt;^&lt;/code&gt; and &lt;code&gt;$&lt;/code&gt;?  The snakemake documentation implies that it's using the equivalent of &lt;code&gt;re.match&lt;/code&gt; rather than &lt;code&gt;re.search&lt;/code&gt; already, with wording like &amp;quot;Here we could restrict the wildcard &lt;code&gt;dataset&lt;/code&gt; to consist of digits only using &lt;code&gt;\d+&lt;/code&gt; as the corresponding regular expression&amp;quot; so I don't think it's necessary anyway.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Edit: I think I found the issue &lt;a href=&quot;https://github.com/snakemake/snakemake/blob/b5a89cc8dd359c2eb726acf55385a1ab89fa871c/snakemake/io.py#L859&quot; rel=&quot;nofollow noreferrer&quot;&gt;in the snakemake source&lt;/a&gt;.  It munges your constraints together into one larger regular expression for the entire file path, so &lt;code&gt;^&lt;/code&gt; and &lt;code&gt;$&lt;/code&gt; get embedded &lt;em&gt;inside&lt;/em&gt; the resulting expression.  This looks like something that should at least be clarified in the documentation.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastEditorUserId="4980" LastEditDate="2020-12-03T16:27:35.300" LastActivityDate="2020-12-03T16:27:35.300" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="14997" PostTypeId="2" ParentId="14392" CreationDate="2020-12-03T16:44:50.960" Score="2" Body="&lt;p&gt;Essentially @MaximilianPress hit the nail on the head. sgRNA is a major feature of SARS-CoV-2.&lt;/p&gt;&#xA;&lt;p&gt;This is where the 5' UTR is spliced at numerous locations in the genome, such that although it is a single stranded positive sense RNA genome it behaves like a segmented viral genome because it is translated as a series of sub-genomic RNA molecules. So influenza is a classic example of a segmented genome. SARS-CoV-2 is more slightly complicated because it is using frameshifts to achieve to achieve the different sgRNAs.&lt;/p&gt;&#xA;&lt;p&gt;However it isn't a particularly strong example of frameshifts because virus such as HBV (hep B) are pass masters, where two different proteins are encoded within a single locus.&lt;/p&gt;&#xA;&lt;p&gt;My personal view is that slippage isn't operating here, unlike say HIV, but that it is a function of the sgRNA construction, thus N protein occurs on one sgeRNA and ORF8b occurs on a second sgRNA and sg signal is within the N protein.&lt;/p&gt;&#xA;&lt;p&gt;If SARS-CoV-2 is your thing then stuff like this is a big player in interpreting bioinformatic output.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-12-03T16:44:50.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="14999" PostTypeId="1" AcceptedAnswerId="15005" CreationDate="2020-12-03T20:03:37.050" Score="0" ViewCount="609" Body="&lt;p&gt;I'm wanting to create a merged object in Seurat using 2 10x Visium 'slices'. However, when I create an object, Seurat assigns an identity &amp;quot;SeuratProject&amp;quot; to the objects (by default I'm guessing; see attached plot). This will be problematic when I attempt to merge the objects as there will be unintended duplicates of the objects.&lt;/p&gt;&#xA;&lt;p&gt;Here's what I'm attempting to use to create the 10x Spatial Object:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;brain1 &amp;lt;- Load10X_Spatial(data.dir = &amp;quot;~/xxx&amp;quot;, filename = &amp;quot;filtered_feature_bc_matrix.h5&amp;quot;, assay = &amp;quot;Spatial&amp;quot;, slice = &amp;quot;slice1&amp;quot;, filter.matrix = TRUE, to.upper = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'd need the ability to assign unique ident to each of these (and any subsequent) slices/data.&lt;/p&gt;&#xA;&lt;p&gt;Thank you!!&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wAfo2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wAfo2.png&quot; alt=&quot;Plot output showing the &amp;quot;SeuratProject&amp;quot; Identity&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9188" LastEditorUserId="57" LastEditDate="2021-08-26T22:32:44.133" LastActivityDate="2021-08-26T22:32:44.133" Title="How do I change the identity of a sample from spatial 'Visium' data preprocessing in Seurat v3?" Tags="|seurat|single-cell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15000" PostTypeId="2" ParentId="14974" CreationDate="2020-12-03T23:40:44.707" Score="2" Body="&lt;p&gt;Ok, thanks to &lt;a href=&quot;https://github.com/rhpvorderman&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ruben Vorderman&lt;/a&gt; on the &lt;a href=&quot;https://github.com/openwdl/wdl/issues/427&quot; rel=&quot;nofollow noreferrer&quot;&gt;openwdl github page&lt;/a&gt;, I found a solution that does not involve renaming the glob folder (which I believe is not possible?). The only way is to write out the names of the files and pass them explicitly to the Array[File] output. This is how I did it (simplified), in case it helps anyone else in the future:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;task fastqc_task {&#xA;  File forwardReads&#xA;  File reverseReads&#xA;  String fwdName = sub(basename(forwardReads),&amp;quot;.f.*q.*&lt;span class=&quot;math-container&quot;&gt;$&quot;,&quot;&quot;)&#xA;  String revName = sub(basename(reverseReads),&quot;.f.*q.*$&lt;/span&gt;&amp;quot;,&amp;quot;&amp;quot;)&#xA;  command {&#xA;    mkdir -p ./qc/fastqc_zip&#xA;    fastqc &lt;span class=&quot;math-container&quot;&gt;${forwardReads} $&lt;/span&gt;{reverseReads} --outdir ./qc/fastqc_zip&#xA;  }&#xA;  output {&#xA;    Array[File] fastqcArray = [&amp;quot;./qc/fastqc_zip/&lt;span class=&quot;math-container&quot;&gt;${fwdName}_fastqc.zip&quot;, &quot;./qc/fastqc_zip/$&lt;/span&gt;{revName}_fastqc.zip&amp;quot;]&#xA;  } &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9973" LastActivityDate="2020-12-03T23:40:44.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15001" PostTypeId="1" CreationDate="2020-12-04T09:20:20.210" Score="1" ViewCount="55" Body="&lt;p&gt;I have a protein refseq (eg, NP_000029). How can I get the corresponding gene ID (ag, APC) from NCBI using an R package?&lt;/p&gt;&#xA;" OwnerUserId="9995" LastActivityDate="2020-12-04T11:07:42.153" Title="Mapping protein refseq to Gene ID" Tags="|gene|genome|phylogenetics|proteomics|refseq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15002" PostTypeId="2" ParentId="15001" CreationDate="2020-12-04T11:07:42.153" Score="2" Body="&lt;p&gt;You can do this with the &lt;a href=&quot;https://github.com/gschofl/reutils&quot; rel=&quot;nofollow noreferrer&quot;&gt;reutils&lt;/a&gt; package, which provides an API to NCBI's &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK25500/&quot; rel=&quot;nofollow noreferrer&quot;&gt;E-utilities&lt;/a&gt;. Here's an example for your specific question:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;install.packages(&amp;quot;reutils&amp;quot;)&#xA;library(reutils)&#xA;&#xA;# Get universal identifier&#xA;uid &amp;lt;- esearch(&amp;quot;NP_000029&amp;quot;, db = &amp;quot;gene&amp;quot;)&#xA;# Fetch summary&#xA;sm &amp;lt;- esummary(uid, db = &amp;quot;gene&amp;quot;)&#xA;# Extract specific tag&#xA;sm$xmlValue(&amp;quot;//Name&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9693" LastActivityDate="2020-12-04T11:07:42.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15003" PostTypeId="1" CreationDate="2020-12-04T14:14:31.940" Score="1" ViewCount="49" Body="&lt;p&gt;I am starting out in metagenomics,&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Aim&lt;/strong&gt; I wish to understand the whole analysis from the raw reads to interpretation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; My current pipeline is used for:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;analyzing human gut microbiota cleans reads,&lt;/li&gt;&#xA;&lt;li&gt;filters out human genome, and&lt;/li&gt;&#xA;&lt;li&gt;maps the reads to known microbial species using &lt;code&gt;IGC&lt;/code&gt; and &lt;code&gt;mOTUs&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; I am looking for basic tools to visualize the output, in order to get some biological and statistical insights (Linux and python are preferable environments).&lt;/p&gt;&#xA;&lt;p&gt;For example, an obvious step would be to display the &lt;code&gt;mOTUs&lt;/code&gt; analysis results on the phylogenetic tree, using the full phylogeny provided with this package. Whilst I could implement it myself, I was wondering if anyone else had previous experience?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;br /&gt;&#xA;One package that I found useful is &lt;a href=&quot;https://github.com/eprifti/momr&quot; rel=&quot;nofollow noreferrer&quot;&gt;MOMR&lt;/a&gt; (through in R).&lt;/p&gt;&#xA;" OwnerUserId="1813" LastEditorUserId="1813" LastEditDate="2020-12-20T12:39:40.600" LastActivityDate="2020-12-20T12:39:40.600" Title="Analyzing and visualizing metagenomic data" Tags="|phylogeny|visualization|metagenome|microbial-genomics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15005" PostTypeId="2" ParentId="14999" CreationDate="2020-12-04T21:31:07.393" Score="1" Body="&lt;p&gt;You can re-assign &lt;code&gt;orig.ident&lt;/code&gt; after creating the object. For example:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;brain1 &amp;lt;-Load10X_Spatial(data.dir = &amp;quot;~/xxx&amp;quot;, filename = &amp;quot;filtered_feature_bc_matrix.h5&amp;quot;, assay = &amp;quot;Spatial&amp;quot;, slice = &amp;quot;slice1&amp;quot;, filter.matrix = TRUE, to.upper = FALSE)&#xA;brain1$orig.ident &amp;lt;- &amp;quot;Visium&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3415" LastActivityDate="2020-12-04T21:31:07.393" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15006" PostTypeId="1" AcceptedAnswerId="15013" CreationDate="2020-12-05T07:02:16.940" Score="1" ViewCount="151" Body="&lt;p&gt;Is it really required to do GC normalization ATAC seq data ?&lt;/p&gt;&#xA;&lt;p&gt;One of the paper where they have ATAC seq data did the normaisation of GC bias after peak calling. When i looked for the library and the reason why GC bias i found this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Could it be true that genes with higher GC content are higher expressed? It has been suggested that genes that are either extremely high or extremely low expressed are under some form of selection leading to “extreme” GC content. What CQN does, is making the effect of GC content comparable across samples, and we show in [1] that this leads to improved inference. It also flattens the effect of GC content on gene expression, but we believe this is better than having the effect of GC content depend on the sample.&#xA;https://www.bioconductor.org/packages/release/bioc/vignettes/cqn/inst/doc/cqn.pdf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now that scenario is for RNA seq would it be logical to do the same for ATAC seq?&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2020-12-06T07:56:09.673" Title="GC normalization ATAC seq data" Tags="|atac-seq|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15007" PostTypeId="1" CreationDate="2020-12-05T13:55:07.670" Score="-4" ViewCount="72" Body="&lt;p&gt;I have a dataframe with 27 samples and for each of them I have different number of values&lt;/p&gt;&#xA;&lt;p&gt;I want to sum up the values for each sample, leaving each sample with a single value&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dput(d[1:10,c(1,8)])&#xA;structure(list(SampleID = c(&amp;quot;LP6008460-DNA_A04&amp;quot;, &amp;quot;LP6008460-DNA_A04&amp;quot;, &#xA;&amp;quot;s30&amp;quot;, &amp;quot;s30&amp;quot;, &amp;quot;s67&amp;quot;, &amp;quot;s15&amp;quot;, &amp;quot;s67&amp;quot;, &amp;quot;s67&amp;quot;, &amp;quot;s67&amp;quot;, &amp;quot;s30&amp;quot;), X.1 = c(8.17407e-05, &#xA;1.40067e-06, 3.86017e-05, 9.50247e-05, 7.08333e-07, 3.19133e-05, &#xA;3.1602e-05, 5.00513e-05, 3.29613e-05, 1.003e-06)), row.names = c(NA, &#xA;10L), class = &amp;quot;data.frame&amp;quot;)&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I started to do that manually but that was a pain and inaccurate&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2020-12-05T20:07:36.193" LastActivityDate="2020-12-05T20:07:36.193" Title="Summing up one column entries for each sample" Tags="|r|" AnswerCount="1" CommentCount="1" ClosedDate="2020-12-07T21:58:02.753" ContentLicense="CC BY-SA 4.0" />
  <row Id="15008" PostTypeId="2" ParentId="15007" CreationDate="2020-12-05T14:37:05.723" Score="2" Body="&lt;p&gt;It's worth learning how to use &lt;code&gt;dplyr&lt;/code&gt; for this sort of thing:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dat %&amp;gt;% &#xA;    dplyr::group_by(SampleID) %&amp;gt;% &#xA;    dplyr::summarise(sample_mean = mean(X.1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;# A tibble: 4 x 2&#xA;  SampleID          sample_mean&#xA;  &amp;lt;chr&amp;gt;                   &amp;lt;dbl&amp;gt;&#xA;1 LP6008460-DNA_A04   0.0000416&#xA;2 s15                 0.0000319&#xA;3 s30                 0.0000449&#xA;4 s67                 0.0000288&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2020-12-05T14:37:05.723" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15013" PostTypeId="2" ParentId="15006" CreationDate="2020-12-06T07:56:09.673" Score="2" Body="&lt;p&gt;The only reason to normalize for GC content in RNA-seq is if it differs notably between samples/groups. If that's not the case and you aren't trying to compare genes withing samples then you have no reason to try to account for GC content. The same goes for ATAC-seq, though there the danger with trying to normalize for GC content is that you then mask changes in global accessibility.&lt;/p&gt;&#xA;&lt;p&gt;In short, normalize for things that should cause problems with your analysis and nothing else.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-12-06T07:56:09.673" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15014" PostTypeId="2" ParentId="14988" CreationDate="2020-12-06T20:04:50.557" Score="0" Body="&lt;p&gt;This is a pretty straightforward analysis, assuming that you want to measure something like &amp;quot;how many reads align to genome 1, and how many reads align to genome 2?&amp;quot;.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;a href=&quot;http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml#getting-started-with-bowtie-2-lambda-phage-example&quot; rel=&quot;nofollow noreferrer&quot;&gt;bowtie2 manual has a section&lt;/a&gt; on this.&lt;/li&gt;&#xA;&lt;li&gt;There are &lt;a href=&quot;https://wikis.utexas.edu/display/bioiteam/Mapping+with+bowtie2+Tutorial&quot; rel=&quot;nofollow noreferrer&quot;&gt;other tutorials for bowtie2&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://bio-bwa.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;bwa&lt;/a&gt; is also &lt;a href=&quot;http://evomics.org/wp-content/uploads/2019/01/genomics_tutorial_2019.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;well-suited to this task&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You should be able to then use &lt;a href=&quot;http://www.htslib.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools&lt;/a&gt; to compute &lt;a href=&quot;http://www.htslib.org/doc/samtools-stats.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;statistics&lt;/a&gt; on your alignments.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2020-12-06T20:04:50.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15015" PostTypeId="2" ParentId="14557" CreationDate="2020-12-06T20:11:42.263" Score="0" Body="&lt;pre&gt;&lt;code&gt;import re&#xA;import os&#xA;import pandas as pd&#xA;import logging&#xA;import sys&#xA;import csv&#xA;import itertools&#xA;&#xA;mutations = ['A222V', 'D614G', 'E484Q', 'E780Q', 'G476S', 'L18F', 'N439K',&#xA;             'S477', 'S477N', 'T478I', 'V483A']&#xA;&#xA;combinations = []&#xA;for M in range(1, len(mutations)+1):&#xA;    for subset in itertools.combinations(mutations, M):&#xA;        combinations.append(subset)&#xA;        &#xA;        combinations = ['_'.join(sorted(x)) for x in combinations]&#xA;        combinations = [x.split('_') for x in list(set(combinations))]&#xA;&#xA;root = &amp;quot;C:&amp;quot;&#xA;&#xA;os.chdir(root)&#xA;&#xA;lineages = os.listdir('Results')&#xA;&#xA;combination_labels = []&#xA;combination_counts = []&#xA;&#xA;for lineage in lineages:&#xA;    &#xA;    df = pd.read_csv('Results/' + lineage).dropna()&#xA;    &#xA;    for combination in combinations:&#xA;    &#xA;            combination_df = df[list(df)]&#xA;        &#xA;            for mutation in combination:&#xA;                combination_df = combination_df[combination_df[mutation] == 1]&#xA;    &#xA;        #print(combination_df.shape[0])&#xA;        &#xA;            combination_labels.append('_'.join(combination))&#xA;            combination_counts.append(combination_df.shape[0])&#xA;        &#xA;            out_df = pd.DataFrame({'combination':combination_labels,&#xA;                               'count':combination_counts})&#xA;            out_df['percentage'] = (out_df['count'] / df.shape[0]) * 100&#xA;            out_df = out_df.sort_values('percentage', ascending = False)   &#xA;        &#xA;            out_df.to_csv('Results_2/' + lineage.replace(&amp;quot;.csv&amp;quot;, &amp;quot;&amp;quot;) + '_3.csv',&#xA;                     header = True, &#xA;                     index = False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9672" LastActivityDate="2020-12-06T20:11:42.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15016" PostTypeId="2" ParentId="14544" CreationDate="2020-12-06T20:19:30.873" Score="0" Body="&lt;pre&gt;&lt;code&gt;   left_lineagelist = [x.split('_')[-1].split('|')[0] &#xA;                          for x in left_lineagelist]&#xA;        right_lineagelist = set([x.split('_')[-1].split('|')[0] &#xA;                          for x in right_lineagelist])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Allows for extraction of 6 digit EPI, provided the file has had sequences removed prior; as such:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for line in lines:&#xA;    if line[0] == '&amp;gt;':&#xA;        print(line[1:])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9672" LastActivityDate="2020-12-06T20:19:30.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15018" PostTypeId="2" ParentId="14877" CreationDate="2020-12-06T21:32:35.623" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/alexpreynolds/kmer-counter&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's a C++ kmer counter&lt;/a&gt; I wrote for my own purposes. It can count canonical and reverse complement kmers from FASTA files up to length 31. Run &lt;code&gt;kmer-counter --help&lt;/code&gt; or read the README for count options.&lt;/p&gt;&#xA;&lt;p&gt;For longer kmers, you'd want to look at an approximation tool like &lt;a href=&quot;https://github.com/gmarcais/Jellyfish&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jellyfish&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2020-12-06T21:32:35.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15020" PostTypeId="1" CreationDate="2020-12-07T04:07:38.133" Score="1" ViewCount="145" Body="&lt;p&gt;For most of the time, I rely on gene ids to combine different datasets. However, in some instances, I have to combine datasets based on gene names. Then, if I don't know the source of gene names in the dataset, I get to this issue of choosing a source of gene names, be it Ensembl, HGNC etc for human genes. I wonder if this is a common issue and if there is an reliable method out there to deal with this issue.&lt;/p&gt;&#xA;&lt;p&gt;To demonstrate the mismatch between different sources, I compared gene names for all human genes. I obtained them from 4 different sources as listed below, using BioMart (&lt;code&gt;pybiomart&lt;/code&gt;) :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;+-----------------+-----------------------+--------------------------------------------+&#xA;|     source      |     attribute_name    |                display_name                |&#xA;+-----------------+-----------------------+--------------------------------------------+&#xA;| HGNC            |  hgnc_symbol          |  HGNC symbol                               |&#xA;| NCBI            |  entrezgene_accession |  NCBI gene (formerly Entrezgene) accession |&#xA;| Uniprot         |  uniprot_gn_symbol    |  UniProtKB Gene Name symbol                |&#xA;| Ensembl (maybe) |  external_gene_name   |  Gene name                                 |&#xA;+-----------------+-----------------------+--------------------------------------------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Upon this comparison, I found several things that are clearly apparent.&lt;/p&gt;&#xA;&lt;h3&gt;1. Genes names of protein coding genes are best matched across different sources.&lt;/h3&gt;&#xA;&lt;p&gt;I saw that protein coding genes have the best matching (left, measured in terms of Jaccard index) across different sources, with majority of genes having a single unique names (shown on right).&lt;br /&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/WIXxB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WIXxB.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;br /&gt;&#xA;However, there isn't a good enough matching in the case of not protein coding genes. Here, HGNC and Ensembl have the best match. (I don't expect Uniprot gene names to match because they are of course only for protein coding genes.) Remarkably most of the genes have 2 unique ids (shown on right).&lt;br /&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/wgPHn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wgPHn.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h3&gt;2. Gene names from some databases match with each other.&lt;/h3&gt;&#xA;&lt;p&gt;Comparison of all genes shows that some pairs of the sources do not have a good match e.g. Ensembl and Uniprot, with many genes having 2 unique gene names(!).&lt;br /&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/z6xNp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/z6xNp.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I saw similar pattern for genes on chromosomes (autosomes,X,Y) and on the scaffolds.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/4V7J5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4V7J5.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;br /&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/StHB8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/StHB8.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h3&gt;3. Mitochondrial gene names do not match at all(!).&lt;/h3&gt;&#xA;&lt;p&gt;Mitochondrial genes clearly have different names in different databases. None of the genes have a single unique gene names (!).&lt;br /&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/VNGa2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VNGa2.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;How to deal with such a mismatch between different sources?&lt;br /&gt;&#xA;Should I prefer one particular source or is there a way to make use of the synonymous gene names from different sources?&lt;/p&gt;&#xA;" OwnerUserId="274" LastActivityDate="2020-12-07T04:07:38.133" Title="How to deal with the mismatches between gene names obtained from different sources?" Tags="|database|phylogenetics|ensembl|biomart|uniprot|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15023" PostTypeId="1" CreationDate="2020-12-08T09:13:31.487" Score="1" ViewCount="92" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/477807/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am trying to run htseq-count for carrying out rna-seq analysis for solanum tuberosum and i used the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;htseq-count --format bam --order pos -s no -a 10 -t exon -i gene_name --idattr gene_id SO_8612_L1.bam GCF_000226075.1_SolTub_3.0_genomic.gff &amp;gt; L1_htseq_count.tsv &#xA;and im getting an error message saying: &#xA;[E::idx_find_and_load] Could not retrieve index file for 'SO_8612_L1.bam' Error processing GFF file (line 12 of file GCF_000226075.1_SolTub_3.0_genomic.gff): Feature exon-XM_015312074.1-1 does not contain a 'gene_id' attribute [Exception type: ValueError, raised in features.py:326]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How to resolve this? and also can gff be used instead of gtf file. what is the standard gtf file used for Solanum tuberosum?&lt;/p&gt;&#xA;" OwnerUserId="10032" LastEditorUserId="73" LastEditDate="2020-12-10T00:40:43.647" LastActivityDate="2020-12-10T00:40:43.647" Title="Query on htseq count" Tags="|rna-seq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15024" PostTypeId="1" CreationDate="2020-12-08T17:33:40.000" Score="1" ViewCount="133" Body="&lt;p&gt;I would like to look at the non-coding regions of some genes. I have around 1000 full genome assemblies and I was able to extract the nucleotide sequences for certain proteins with &lt;code&gt;Roary&lt;/code&gt;. However, we are also interested in comparing non-coding regions. Is there a tool that can do that or a tool that tells me what region in the full assembly is relevant? The assembly and extracted genes are present as &lt;code&gt;fasta&lt;/code&gt; and &lt;code&gt;fa.ali&lt;/code&gt; (alignment) files. The genomes are from bacteria.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2020-12-08T17:33:40.000" Title="How to extract certain genes including non-coding regions?" Tags="|genome|genomics|microbial-genomics|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="15025" PostTypeId="1" AcceptedAnswerId="15055" CreationDate="2020-12-08T20:51:17.467" Score="2" ViewCount="41" Body="&lt;p&gt;I am using Cromwell to test a &lt;a href=&quot;https://www.commonwl.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CWL&lt;/a&gt; (common workflow language) implementation of the GATK somatic CNV (copy number variation) pipeline. I am currently getting the following error (full log at the end):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Cannot find a tool or workflow with ID file:///var/folders/cm/0j97xqy577z2dlfht90hv_6h0000gq/T/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.temp.8067760554642643818/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.cwl in file /var/folders/cm/0j97xqy577z2dlfht90hv_6h0000gq/T/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.temp.8067760554642643818/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.cwl&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The ID's are the same except for the first starts with &amp;quot;file://&amp;quot;. I am not very familiar with Cromwell so I'm not sure how to proceed in resolving this error. Any help would be appreciated.&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;&lt;p&gt;Full log:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;[2020-12-03 15:20:24,82] [info] Running with database db.url = jdbc:hsqldb:mem:28b7f7a3-d261-42ee-b941-e329cdcf46ba;shutdown=false;hsqldb.tx=mvcc&#xA;[2020-12-03 15:20:29,97] [info] Running migration RenameWorkflowOptionsInMetadata with a read batch size of 100000 and a write batch size of 100000&#xA;[2020-12-03 15:20:29,98] [info] [RenameWorkflowOptionsInMetadata] 100%&#xA;[2020-12-03 15:20:30,03] [info] Running with database db.url = jdbc:hsqldb:mem:ae8f4f13-d319-4439-8681-b684e13ca6ac;shutdown=false;hsqldb.tx=mvcc&#xA;[2020-12-03 15:20:30,23] [info] Slf4jLogger started&#xA;[2020-12-03 15:20:30,25] [info] Pre Processing Workflow...&#xA;[2020-12-03 15:20:30,27] [info] Pre Processing Inputs...&#xA;[2020-12-03 15:20:30,46] [info] Workflow heartbeat configuration:&#xA;{&#xA;&amp;quot;cromwellId&amp;quot; : &amp;quot;cromid-d8084bc&amp;quot;,&#xA;&amp;quot;heartbeatInterval&amp;quot; : &amp;quot;2 minutes&amp;quot;,&#xA;&amp;quot;ttl&amp;quot; : &amp;quot;10 minutes&amp;quot;,&#xA;&amp;quot;writeBatchSize&amp;quot; : 10000,&#xA;&amp;quot;writeThreshold&amp;quot; : 10000&#xA;}&#xA;[2020-12-03 15:20:30,48] [info] Metadata summary refreshing every 2 seconds.&#xA;[2020-12-03 15:20:30,50] [info] KvWriteActor configured to flush with batch size 200 and process rate 5 seconds.&#xA;[2020-12-03 15:20:30,50] [info] WriteMetadataActor configured to flush with batch size 200 and process rate 5 seconds.&#xA;[2020-12-03 15:20:30,50] [info] CallCacheWriteActor configured to flush with batch size 100 and process rate 3 seconds.&#xA;[2020-12-03 15:20:30,78] [info] JobExecutionTokenDispenser - Distribution rate: 50 per 1 seconds.&#xA;[2020-12-03 15:20:30,80] [info] SingleWorkflowRunnerActor: Submitting workflow&#xA;[2020-12-03 15:20:30,82] [info] cwl (Unspecified version) workflow 837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a submitted&#xA;[2020-12-03 15:20:30,82] [info] SingleWorkflowRunnerActor: Workflow submitted [38;5;2m837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a[0m&#xA;[2020-12-03 15:20:30,82] [info] 1 new workflows fetched&#xA;[2020-12-03 15:20:30,82] [info] WorkflowManagerActor Starting workflow [38;5;2m837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a[0m&#xA;[2020-12-03 15:20:30,83] [[38;5;220mwarn[0m] SingleWorkflowRunnerActor: received unexpected message: Done in state RunningSwraData&#xA;[2020-12-03 15:20:30,84] [[38;5;220mwarn[0m] Couldn't find a suitable DSN, defaulting to a Noop one.&#xA;[2020-12-03 15:20:30,84] [info] Using noop to send events.&#xA;[2020-12-03 15:20:30,85] [info] WorkflowManagerActor Successfully started WorkflowActor-837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a&#xA;[2020-12-03 15:20:30,85] [info] Retrieved 1 workflows from the WorkflowStoreActor&#xA;[2020-12-03 15:20:30,85] [info] WorkflowStoreHeartbeatWriteActor configured to flush with batch size 10000 and process rate 2 minutes.&#xA;[2020-12-03 15:20:30,86] [info] MaterializeWorkflowDescriptorActor [[38;5;2m837d7e7d[0m]: Parsing workflow as CWL v1.0&#xA;[2020-12-03 15:20:30,91] [info] Pre-Processing /var/folders/cm/0j97xqy577z2dlfht90hv_6h0000gq/T/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.temp.8067760554642643818/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.cwl&#xA;[2020-12-03 15:20:32,24] [[38;5;1merror[0m] WorkflowManagerActor Workflow 837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a failed (during MaterializingWorkflowDescriptorState): cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor&lt;span class=&quot;math-container&quot;&gt;$$anon$1: Workflow input processing failed:&#xA;Cannot find a tool or workflow with ID file:///var/folders/cm/0j97xqy577z2dlfht90hv_6h0000gq/T/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.temp.8067760554642643818/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.cwl in file /var/folders/cm/0j97xqy577z2dlfht90hv_6h0000gq/T/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.temp.8067760554642643818/837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a.cwl&#xA;at cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor.cromwell$engine$workflow$lifecycle$materialization$MaterializeWorkflowDescriptorActor$$&lt;/span&gt;workflowInitializationFailed(MaterializeWorkflowDescriptorActor.scala:200)&#xA;at cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor&lt;span class=&quot;math-container&quot;&gt;$$anonfun$2.applyOrElse(MaterializeWorkflowDescriptorActor.scala:170)&#xA;at cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor$$&lt;/span&gt;anonfun&lt;span class=&quot;math-container&quot;&gt;$2.applyOrElse(MaterializeWorkflowDescriptorActor.scala:165)&#xA;at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:34)&#xA;at akka.actor.FSM.processEvent(FSM.scala:665)&#xA;at akka.actor.FSM.processEvent$&lt;/span&gt;(FSM.scala:662)&#xA;at cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor.akka&lt;span class=&quot;math-container&quot;&gt;$actor$&lt;/span&gt;LoggingFSM&lt;span class=&quot;math-container&quot;&gt;$$super$processEvent(MaterializeWorkflowDescriptorActor.scala:123)&#xA;at akka.actor.LoggingFSM.processEvent(FSM.scala:801)&#xA;at akka.actor.LoggingFSM.processEvent$(FSM.scala:783)&#xA;at cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor.processEvent(MaterializeWorkflowDescriptorActor.scala:123)&#xA;at akka.actor.FSM.akka$actor$FSM$$&lt;/span&gt;processMsg(FSM.scala:659)&#xA;at akka.actor.FSM$$anonfun$receive$1.applyOrElse(FSM.scala:653)&#xA;at akka.actor.Actor.aroundReceive(Actor.scala:514)&#xA;at akka.actor.Actor.aroundReceive$(Actor.scala:512)&#xA;at cromwell.engine.workflow.lifecycle.materialization.MaterializeWorkflowDescriptorActor.aroundReceive(MaterializeWorkflowDescriptorActor.scala:123)&#xA;at akka.actor.ActorCell.receiveMessage(ActorCell.scala:527)&#xA;at akka.actor.ActorCell.invoke(ActorCell.scala:496)&#xA;at akka.dispatch.Mailbox.processMailbox(Mailbox.scala:257)&#xA;at akka.dispatch.Mailbox.run(Mailbox.scala:224)&#xA;at akka.dispatch.Mailbox.exec(Mailbox.scala:234)&#xA;at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)&#xA;at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)&#xA;at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)&#xA;at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)&#xA;&#xA;[2020-12-03 15:20:32,24] [info] WorkflowManagerActor WorkflowActor-837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a is in a terminal state: WorkflowFailedState&#xA;[2020-12-03 15:20:37,26] [info] SingleWorkflowRunnerActor workflow finished with status 'Failed'.&#xA;[2020-12-03 15:20:40,51] [info] Workflow polling stopped&#xA;[2020-12-03 15:20:40,52] [info] Shutting down WorkflowStoreActor - Timeout = 5 seconds&#xA;[2020-12-03 15:20:40,52] [info] Shutting down WorkflowLogCopyRouter - Timeout = 5 seconds&#xA;[2020-12-03 15:20:40,52] [info] Shutting down JobExecutionTokenDispenser - Timeout = 5 seconds&#xA;[2020-12-03 15:20:40,52] [info] Aborting all running workflows.&#xA;[2020-12-03 15:20:40,52] [info] JobExecutionTokenDispenser stopped&#xA;[2020-12-03 15:20:40,52] [info] WorkflowStoreActor stopped&#xA;[2020-12-03 15:20:40,53] [info] WorkflowLogCopyRouter stopped&#xA;[2020-12-03 15:20:40,53] [info] Shutting down WorkflowManagerActor - Timeout = 3600 seconds&#xA;[2020-12-03 15:20:40,53] [info] WorkflowManagerActor All workflows finished&#xA;[2020-12-03 15:20:40,53] [info] WorkflowManagerActor stopped&#xA;[2020-12-03 15:20:40,53] [info] Connection pools shut down&#xA;[2020-12-03 15:20:40,53] [info] Shutting down SubWorkflowStoreActor - Timeout = 1800 seconds&#xA;[2020-12-03 15:20:40,53] [info] Shutting down JobStoreActor - Timeout = 1800 seconds&#xA;[2020-12-03 15:20:40,53] [info] Shutting down CallCacheWriteActor - Timeout = 1800 seconds&#xA;[2020-12-03 15:20:40,53] [info] SubWorkflowStoreActor stopped&#xA;[2020-12-03 15:20:40,53] [info] Shutting down ServiceRegistryActor - Timeout = 1800 seconds&#xA;[2020-12-03 15:20:40,53] [info] Shutting down DockerHashActor - Timeout = 1800 seconds&#xA;[2020-12-03 15:20:40,53] [info] JobStoreActor stopped&#xA;[2020-12-03 15:20:40,53] [info] Shutting down IoProxy - Timeout = 1800 seconds&#xA;[2020-12-03 15:20:40,53] [info] CallCacheWriteActor Shutting down: 0 queued messages to process&#xA;[2020-12-03 15:20:40,53] [info] WriteMetadataActor Shutting down: 0 queued messages to process&#xA;[2020-12-03 15:20:40,53] [info] KvWriteActor Shutting down: 0 queued messages to process&#xA;[2020-12-03 15:20:40,53] [info] CallCacheWriteActor stopped&#xA;[2020-12-03 15:20:40,53] [info] DockerHashActor stopped&#xA;[2020-12-03 15:20:40,53] [info] IoProxy stopped&#xA;[2020-12-03 15:20:40,53] [info] ServiceRegistryActor stopped&#xA;[2020-12-03 15:20:40,54] [info] Database closed&#xA;[2020-12-03 15:20:40,54] [info] Stream materializer shut down&#xA;Workflow 837d7e7d-ec48-4cfc-8f9e-4736c8bc5a8a transitioned to state Failed&#xA;[2020-12-03 15:20:40,57] [info] Automatic shutdown of the async connection&#xA;[2020-12-03 15:20:40,57] [info] Gracefully shutdown sentry threads.&#xA;[2020-12-03 15:20:40,57] [info] Shutdown finished.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10036" LastEditorUserId="3848" LastEditDate="2020-12-12T12:04:08.037" LastActivityDate="2021-01-13T22:00:55.117" Title="Cromwell: can't find file ID" Tags="|gatk|cnv|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15026" PostTypeId="1" CreationDate="2020-12-08T21:42:01.890" Score="1" ViewCount="83" Body="&lt;p&gt;I am trying to simulate metagenome sequencing. So I will start with a file with a lot of concatenated genomes. From there, I would like to randomly extract 10,000 sequences of length 200bps.&lt;/p&gt;&#xA;&lt;p&gt;I got help subtracting a random 200 sequence from a fasta file like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;JQ086372.1 Enterobacteria phage HK446, complete genome &#xA;ATGGCAGATAAACGAATCCGTTCCGACAGTTCGGCGGCAGCGGTTCAGGCCATGAAAAATGCAGCTGTGGCTTCTCCCGCTTCAGTCTGCCAATATGGATGTGAAACTCGTCGGAAAAAAAGTGGTTTATCGCTATCAACGCAACAGCGAATACGCCGACTTTTCGCAGAAAGAGATTTTTCACCTTAAAGGCTTCGGATTCACCGGGCTTGTAGGCCTGTCAATTTTTTCCACTTATTTAAATGTTTGTAAATTAAGTAAATAACGTTTTTAATCTAAAAATTCCACAATTTCCACAATAACTACTACTACTAAATTTTTTTAATTATTTTATTTATTTATATTAGAAAAGAGAACAATATAAGTTATTGAATTTAATTTACGAAAAAAGTATATTGTGTATGTGTTAAAAACTTCCAAATTCCGATGGAAATTATTTGTAATCAAAATGAGTTAAACAATGCTATACAACTAGTAAGTAAAGCAGTTGCTTCACGACCAACTCATCCAATTCTTGCAAATATACTTTTAACAGCTGACCAAGGAACTAATAAAATTAGTTTGACAGGATTTGATCTTAATTTAGGAATTCAAACTTCTTTTGATGGAACTGTTAAAAATAGCGGTGCTATTACGATACCTTCAAAACTTTTATCTGAAATAGTAAACAAATTAC...(this still goes for a lot more A or C or G or T)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;using this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tail -n +2 file.fasta | tr -d '\n' &amp;gt; newfile&#xA;n=$(stat -c &amp;quot;%s&amp;quot; newfile)&#xA;r=$(shuf -i1-&amp;quot;$((n-200+1))&amp;quot; -n1)&#xA;newfile tail -c+&amp;quot;$r&amp;quot; | head -c200&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And as an output I got: &lt;code&gt;GTTAAGGGCGCCGTTCTGCTTGTCTTTGCTGACATGTTTGAACACCGGACGGCACAAAGCGAAGTGCAGCTTTATGAGAATGCAGCCGCCGAACGCATGATGTTCATTCATCGCAACTGGCGCGGTAAATCTGAACCTGAGGAGGGCTCCTGATGGAACCTGGACGATTCAGGCACAGGGTAAAAATTCTCACCTTCACG&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;But now I want to extract instead of 1 fragment of 200 characters; 10,0000 random fragments of 200 characters using the strategy above. For this, I am looking to exclude all those lines that begin with &amp;quot;&amp;gt;&amp;quot; (p.e. &amp;gt;JQ086372.1 Enterobacteria phage HK446, complete genome).The input file looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;JQ086372.1 Enterobacteria phage HK446, complete genome &#xA;ATGGCAGATAAACGAATCCGTTCCGACAGTTCGGCGGCAGCGGTTCAGGCCATGAAAAATGCAGCTGTGGCTTCTCCCGCTTCAGTCTGCCAATATGGATGTGAAACTCGTCGGAAAAAAAGTGGTTTATCGCTATCAACGCAACAGCGAATACGCCGACTTTTCGCAGAAAGAGATTTTTCACCTTAAAGGCTTCGGATTCACCGGGCTTGTAGGCCTGTCAATTTTTTCCACTTATTTAAATGTTTGTAAATTAAGTAAATAACGTTTTTAATCTAAAAATTCCACAATTTCCACAATAACTACTACTACTAAATTTTTTTAATTATTTTATTTATTTATATTAGAAAAGAGAACAATATAAGTTATTGAATTTAATTTACGAAAAAAGTATATTGTGTATGTGTTAAAAACTTCCAAATTCCGATGGAAATTATTTGTAATCAAAATGAGTTAAACAATGCTATACAACTAGTAAGTAAAGCAGTTGCTTCACGACCAACTCATCCAATTCTTGCAAATATACTTTTAACAGCTGACCAAGGAACTAATAAAATTAGTTTGACAGGATTTGATCTTAATTTAGGAATTCAAACTTCTTTTGATGGAACTGTTAAAAATAGCGGTGCTATTACGATACCTTCAAAACTTTTATCTGAAATAGTAAACAAATTAC...(this still goes for a lot more A or C or G or T)&#xA;&#xA;&amp;gt;NC_010391.1 Salmonella phage Fels-1, complete genome &#xA;TGATTGCATAACCACTTAACATCTTGTTTTATCTAAATAAAATTAAGCATGTTATCTTTTTGGGGCACTCCTGGGGCAGTAGATGCCAGTTGTTGATTCAGTATATCTACTTGTGACTGGTTATTATCCCGATTTTTTTAGTTTTAAGGTGTTGACATAGCCATCCATGCTCCATATACTGTATAGACCATCTGAGCGTTTGAGTGCCCCATTTGTGAAGCAATAAAGTTCGGGTTCGCGCCAGCGGCAAGCGCCCAGCATGCACCGATTTTTTTAGTTTTAAGGTGTTGACATTAGGTATGTCGGGACTGGTATGCTTTCCTGTGTCGCAGCCCGGCGCGTCTCAATGCAGATTCCCATATCCTGTTCAGGGAAGGTGCAGCATAATGTGCTTTGGATTTACCTGAGCGATCGGTAATTTGCGGATTGAAGACAAAGGTAATTTTTTCCACTTATTTAAATGTTTGTAAATTAAGTAAATAACGTTTTTAATCTAAAAATTCCACAATTTCCACAATAACTACTACTACTAAATTTTTTTAATTATTTTATTTATTTATATTAGAAAAGAGAACAATATAAGTTATTGAATTTAATTTACGAAAAAAGTATATTGTGTATGTGTTAAAAACTTCCAAATTCCGATGGAAATTATTTGTAATCAAAATGAGTTAAACAATGCTATACAACTAGTAAGTAAAGCAGTTGCTTCACGACCAACTCATCCAATTCTTGCAAATATACTTTTAACAGCTGACCAAGGAACTAATAAAATTAGTTTGACAGGATTTGATCTTAATTTAGGAATTCAAACTTCTTTTGATGGAACTGTTAAAAATAGCGGTGCTATTACGATACCTTCAAAACTTTTATCTGAAATAGTAAACAAATTAC...(this still goes for a lot more A or C or G or T)&#xA;&#xA;&amp;gt;NC_007577.1 Prochlorococcus marinus str. MIT 9312, complete genome &#xA;AATTTTTTCCACTTATTTAAATGTTTGTAAATTAAGTAAATAACGTTTTTAATCTAAAAATTCCACAATTTCCACAATAACTACTACTACTAAATTTTTTTAATTATTTTATTTATTTATATTAGAAAAGAGAACAATATAAGTTATTGAATTTAATTTACGAAAAAAGTATATTGTGTATGTGTTAAAAACTTCCAAATTCCGATGGAAATTATTTGTAATCAAAATGAGTTAAACAATGCTATACAACTAGTAAGTAAAGCAGTTGCTTCACGACCAACTCATCCAATTCTTGCAAATATACTTTTAACAGCTGACCAAGGAACTAATAAAATTAGTTTGACAGGATTTGATCTTAATTTAGGAATTCAAACTTCTTTTGATGGAACTGTTAAAAATAGCGGTGCTATTACGATACCTTCAAAACTTTTATCTGAAATAGTAAACAAATTACAATTTTTTCCACTTATTTAAATGTTTGTAAATTAAGTAAATAACGTTTTTAATCTAAAAATTCCACAATTTCCACAATAACTACTACTACTAAATTTTTTTAATTATTTTATTTATTTATATTAGAAAAGAGAACAATATAAGTTATTGAATTTAATTTACGAAAAAAGTATATTGTGTATGTGTTAAAAACTTCCAAATTCCGATGGAAATTATTTGTAATCAAAATGAGTTAAACAATGCTATACAACTAGTAAGTAAAGCAGTTGCTTCACGACCAACTCATCCAATTCTTGCAAATATACTTTTAACAGCTGACCAAGGAACTAATAAAATTAGTTTGACAGGATTTGATCTTAATTTAGGAATTCAAACTTCTTTTGATGGAACTGTTAAAAATAGCGGTGCTATTACGATACCTTCAAAACTTTTATCTGAAATAGTAAACAAATTAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So for my new output, I will end up with a file with 10,000 subsets of 200 characters that would look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1&#xA;ATGGCAGATAAACGAATCCGTTCCGACAGTTCGGCGGCAGCGGTTCAGGCCATGAAAAATGCAGCTGTGGCTTCTCCCGCTTCAGTCTGCCAATATGGATGTGAAACTCGTCGGAAAAAAAGTGGTTTATCGCTATCAACGCAACAGCGAATACGCCGACTTTTTCAGTCTGCCAATATGCGTTCCGACAGCAGATAAAC&#xA;&#xA;&amp;gt;2&#xA;TGATTGCATAACCACTTAACATCTTGTTTTATCTAAATAAAATTAAGCATGTTATCTTTTTGGGGCACTCCTGGGGCAGTAGATGCCAGTTGTTGATTCAGTATATCTACTTGTGACTGGTTATTATCCCGATTTTTTTAGTTTTAAGGTGTTGACATAGCCATCCATGCTCCATATACTGTATAGACCATCTGCAGATA&#xA;&#xA;&amp;gt;3&#xA;AATTTTTTCCACTTATTTAAATGTTTGTAAATTAAGTAAATAACGTTTTTAATCTAAAAATTCCACAATTTCCACAATAACTACTACTACTAAATTTTTTTAATTATTTTATTTATTTATATTAGAAAAGAGAACAATATAAGTTATTGAATTTAATTTACGAAAAAAGTATATTGTGTATGTGTTAAAAACTTCCAAAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Makes sense? Any help?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="10037" LastEditorUserId="73" LastEditDate="2021-06-07T01:35:17.763" LastActivityDate="2022-03-04T03:06:15.580" Title="Metagenome simulation from a concatenated FASTA file" Tags="|fasta|ngs|file-formats|shell|subset|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15028" PostTypeId="2" ParentId="5245" CreationDate="2020-12-09T02:23:26.690" Score="0" Body="&lt;p&gt;&lt;a href=&quot;http://tcm.zju.edu.cn/celltalkdb/search.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;CellTalkDB&lt;/a&gt; is another option for a searchable ligand-receptor database. According to the webpage it is curated from other databases and the literature. Details are available in the &lt;a href=&quot;https://academic.oup.com/bib/advance-article-abstract/doi/10.1093/bib/bbaa269/5955941&quot; rel=&quot;nofollow noreferrer&quot;&gt;publication&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3575" LastActivityDate="2020-12-09T02:23:26.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15031" PostTypeId="1" CreationDate="2020-12-09T15:06:51.310" Score="1" ViewCount="64" Body="&lt;p&gt;I am reading &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/30096299/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Saunders et al. (2018)&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In the methods' subsection &amp;quot;ICA based analysis and clustering&amp;quot; (&lt;a href=&quot;https://en.wikipedia.org/wiki/Independent_component_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;independent component analysis&lt;/a&gt;), a set of independent components from ICA is labelled as &amp;quot;Artifact&amp;quot; with the following explanation:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Artifact (n = 315), in which either: (a) cell scores showed obvious replicate-specific biases, or (b) the top loading genes largely intersected with those observed to vary strongly with digest time (Figures 3 and S3), or contained many heat-shock proteins;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Aside from a possible typo (Figure S3 instead of S2) I couldn't find a list of genes that vary with digest time. The list of genes they found in few technical components is shown in Fig. S2 (D-H). The legend of the figure indicates:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Example of digest-related ICs from FC co-analysis of under- and over-digested cells. Left, cell loadings color-coded by digest condition. Right, gene-loading plots for the 10 top and bottom loading genes. IC 15 (cluster 2, Syt6+ deep-layer glutamatergic) and IC 68 (cluster 1, Nptxr+ upper-layer glutamatergic) show contributions from similar genes. Genes related to ATP synthesis and Calmodulins (eg., Cox6a1, Ndufa4, Calm2) load on the under-digested cells, while another set of genes including the nuclear-enriched Meg3 and Malat1 load highly onto the over-digested condition.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However, a full list of genes associated with digestion is not provided. I searched for other articles reporting such genes but I was unable to find any. Is there such a resource or well-known characterization of genes associated with technical signal in scRNA-Seq?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2020-12-12T18:30:27.540" LastActivityDate="2020-12-12T18:30:27.540" Title="Genes associated with digestion in single cell RNASeq" Tags="|scrnaseq|transcriptome|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15032" PostTypeId="2" ParentId="15026" CreationDate="2020-12-10T05:22:32.597" Score="1" Body="&lt;p&gt;Given that you want to simulate short-read sequencing from reference genomes, there are read simulator programs that already exist. For example, see &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/81/73&quot;&gt;this answer&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2020-12-10T05:22:32.597" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15034" PostTypeId="1" AcceptedAnswerId="15040" CreationDate="2020-12-10T12:06:03.277" Score="1" ViewCount="102" Body="&lt;p&gt;I've got some images of cell organelles and I really want to avoid labeling them by hand. The images all look something like the image below. Is there already an existing model specifially for cells? I tried it with existing models like &lt;code&gt;mask_rcnn_coco.h5&lt;/code&gt; or &lt;code&gt;deeplabv3_xception65_ade20k.h5&lt;/code&gt; but as they aren't built for cells the results aren't good.&lt;/p&gt;&#xA;&lt;p&gt;My code is pretty basic right now:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from pixellib.instance import instance_segmentation&#xA;&#xA;from PIL import Image&#xA;&#xA;    def segmentImage():&#xA;        segment_image = instance_segmentation()&#xA;    &#xA;        segment_image.load_model(&amp;quot;mask_rcnn_coco.h5&amp;quot;)&#xA;        segment_image.segmentImage(&amp;quot;cell.jpg&amp;quot;, output_image_name=&amp;quot;cell_seg.jpg&amp;quot;, show_bboxes=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Cell example image:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zr3LZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zr3LZ.png&quot; alt=&quot;cell organelle example image&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Example segmentation result:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jkXGw.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jkXGw.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9131" LastActivityDate="2020-12-11T09:41:49.763" Title="Segment cell organelles with pixellib" Tags="|python|single-cell|tensorflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15035" PostTypeId="1" AcceptedAnswerId="15043" CreationDate="2020-12-10T16:13:49.190" Score="3" ViewCount="366" Body="&lt;p&gt;I wish to understand the term &lt;code&gt;suppressed&lt;/code&gt; in folder nomenclature on the NCBI ftp genomes website?&lt;/p&gt;&#xA;&lt;p&gt;Example: &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/genomes/refseq/bacteria/Oscillatoriales_cyanobacterium_C42_A2020_001/all_assembly_versions/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/genomes/refseq/bacteria/Oscillatoriales_cyanobacterium_C42_A2020_001/all_assembly_versions/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ncbi.nlm.nih.gov/genomes/refseq/bacteria/Oscillatoriales_cyanobacterium_C42_A2020_001/all_assembly_versions/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Normally, I'd see folders with names like &lt;code&gt;representative&lt;/code&gt; or &lt;code&gt;latest assembly&lt;/code&gt;, which are pretty straightforward. But what does &lt;code&gt;suppressed&lt;/code&gt; refer to in this case?&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your time!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastEditorUserId="3848" LastEditDate="2020-12-11T11:13:32.760" LastActivityDate="2020-12-11T13:37:20.740" Title="What does the 'suppressed' folder mean on the NCBI ftp genomes website?" Tags="|phylogenetics|webservice|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15037" PostTypeId="1" CreationDate="2020-12-10T21:24:37.253" Score="1" ViewCount="88" Body="&lt;p&gt;I have a Plink genetic map with 4 columns: Chromosome, rs-snp, Map(cM), Position(bp) and approximately 8,000 lines find the recombination rate  [Rate(cM/Mb)] from a HapMapII genetic map of 4 columns: Chromosome, Position(bp), Rate(cM/Mb), Map(cM) ~ 47,000 lines ... some chromosomes will be 6 times larger than this; not all Plink entries will be found in HapMapII, but 99% will.&lt;/p&gt;&#xA;&lt;p&gt;I need to parse PLINK output and spreadsheet VLookUP is too slow.&lt;/p&gt;&#xA;&lt;p&gt;Basically I have to find one missing field matching two given fields in a much larger file. I have tried variations of grep on Linux and FindStr in windows but they have not worked.&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;&lt;p&gt;Here is the Plink map&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Chromosome, rs-snp, Map(cM), Position(bp)&#xA;19  rs2312724   0.00187 266034&#xA;19  rs1020382   0.002681    267039&#xA;19  rs12459906  0.010664    276245&#xA;19  rs10409452  0.068867    288246&#xA;19  rs3813154   0.092406    292082&#xA;19  rs895344    0.099705    292901&#xA;... 8300 more rows&#xA;&#xA;Here is the HapMapII map&#xA;Chromosome  Position(bp)    Rate(cM/Mb) Map(cM)&#xA;chr19   253938  0.154179    0.000000&#xA;chr19   256859  0.154528    0.000450&#xA;chr19   259772  0.154530    0.000900&#xA;chr19   260970  0.154697    0.001086&#xA;chr19   261033  0.154888    0.001095&#xA;chr19   266034  0.347963    0.001870  &amp;lt;-&#xA;chr19   266544  1.256622    0.002047&#xA;chr19   266888  1.333840    0.002480&#xA;chr19   267039  1.542843    0.002681&#xA;chr19   267213  2.171166    0.002950&#xA;chr19   267614  1.108918    0.003820&#xA;chr19   269380  0.711702    0.005779&#xA;chr19   276245  1.791471    0.010664&#xA;chr19   277717  2.004476    0.013301&#xA;chr19   277776  3.133699    0.013420&#xA;chr19   279645  4.135431    0.019277&#xA;chr19   280299  4.235347    0.021981&#xA;chr19   281360  6.164471    0.026475&#xA;chr19   287129  6.113884    0.062038&#xA;chr19   288246  6.135026    0.068867&#xA;chr19   288320  6.136509    0.069321&#xA;chr19   292082  8.817796    0.092406&#xA;chr19   292583  9.059132    0.096824&#xA;chr19   292901  9.062522    0.099705&#xA;.... over 46,000 more lines&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I need the Rate(cM/Mb) field imported into a version of the Plink map.&#xA;Position(bp) 266034 ; and Map(cM) 0.00187 exist in both files; but Rate(cM/Mb) 0.347963 does not. I need to import Rate(cM/Mb) from the larger file into a new version of the smaller file. The columns can be resorted. I don't need the chromosome column. I will add a final column with mutation rate as well.&lt;/p&gt;&#xA;" OwnerUserId="10054" LastEditorUserId="3848" LastEditDate="2022-08-16T17:58:00.073" LastActivityDate="2022-08-16T17:58:00.073" Title="Parsing PLINK recombination genetic map" Tags="|snp|genomics|plink|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15038" PostTypeId="1" CreationDate="2020-12-11T08:51:24.113" Score="1" ViewCount="62" Body="&lt;p&gt;I am seeking the definition of the genetic distance matrices used in genomic prediction.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What is the difference between a Rodger distance and kinship matrix?&lt;/li&gt;&#xA;&lt;li&gt;what people mean when they talk about genomic relationship matrix as in this paper:&#xA;&lt;a href=&quot;https://www.g3journal.org/content/9/5/1545#ref-30&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.g3journal.org/content/9/5/1545#ref-30&lt;/a&gt; , viz.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;then we obtained the Cholesky decomposition of the genomic&#xA;relationship matrix&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="9278" LastEditorUserId="3848" LastEditDate="2020-12-11T11:09:24.293" LastActivityDate="2020-12-11T11:09:24.293" Title="Genomic relationship matrix explanation" Tags="|ngs|genomics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15039" PostTypeId="1" CreationDate="2020-12-11T09:28:33.743" Score="1" ViewCount="78" Body="&lt;p&gt;I am seeking advice regarding DeepVariant analysis.&lt;/p&gt;&#xA;&lt;p&gt;To avoid false positives I'm using several variant callers and then the resulting common set will be considered as TP variants.&lt;/p&gt;&#xA;&lt;p&gt;One of the callers is &lt;a href=&quot;https://github.com/google/deepvariant&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepVariant&lt;/a&gt; (I'm using it by the first time). I used &lt;code&gt;DeepVariant-inception_v3-1.0.0+data-wes_standard&lt;/code&gt; deep learning model (checkpoint) as a argument in &lt;code&gt;call_variants&lt;/code&gt; function. However I don't have typical WES data (whole exome sequencing), but custom/target NGS pointed to plenty potential regulatory sequences i.e. promoters, TFBS and also pseudogenes and other non-coding stuff.&lt;/p&gt;&#xA;&lt;p&gt;I have doubts whether my approach is correct, hence many variants may be discarded.&lt;/p&gt;&#xA;&lt;p&gt;Should I use WGS (whole genome sequencing) model instead of WES model regarding training set preparation?&lt;/p&gt;&#xA;&lt;p&gt;Or maybe this tool isn't a good choice regarding such custom experiment?&lt;/p&gt;&#xA;&lt;p&gt;Best Regards, Adam&lt;/p&gt;&#xA;" OwnerUserId="2869" LastEditorUserId="3848" LastEditDate="2020-12-11T11:05:48.830" LastActivityDate="2020-12-11T11:05:48.830" Title="Calling variants with DeepVariant on targeted NGS sequencing (custom library)" Tags="|ngs|variant-calling|deep-learning|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15040" PostTypeId="2" ParentId="15034" CreationDate="2020-12-11T09:41:49.763" Score="2" Body="&lt;p&gt;I am the creator of PixelLib. PixelLib supports instance segmentation of 80 classes of objects using pre-trained coco model. Cell organelles is not part of the classes supported.&#xA;The only solution is to train your own custom model. PixelLib supports custom training of a model using your own labelled dataset. Read this &lt;a href=&quot;https://towardsdatascience.com/custom-instance-segmentation-training-with-7-lines-of-code-ff340851e99b&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; I published on how to train a custom model with PixelLib.&lt;/p&gt;&#xA;" OwnerUserId="10058" LastActivityDate="2020-12-11T09:41:49.763" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15041" PostTypeId="1" CreationDate="2020-12-11T11:35:29.990" Score="3" ViewCount="156" Body="&lt;p&gt;After doing peak call I'm annotating the peaks using chipseeker tool, which I want to take further downstream analysis. Now unlike RNA seq where i have single gene and their respective expression. But here the peaks for a single gene multiple peaks. As an example have filtered promoter peaks.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqnames  start    end width strand     V4 annotation geneChr geneStart geneEnd geneLength geneStrand geneId&#xA;1     chr1 826797 828101  1305      *  Peak1   Promoter       1    826832  852225      25394          1 643837&#xA;2     chr1 869647 870324   678      *  Peak4   Promoter       1    868240  870201       1962          2 284593&#xA;3     chr1 876462 877140   679      *  Peak5   Promoter       1    868071  876903       8833          2 284593&#xA;4     chr1 877219 878105   887      *  Peak6   Promoter       1    874529  877234       2706          2 284593&#xA;5     chr1 923459 924318   860      * Peak14   Promoter       1    923928  939291      15364          1 148398&#xA;6     chr1 924379 926062  1684      * Peak15   Promoter       1    925150  935793      10644          1 148398&#xA;       transcriptId distanceToTSS         ENSEMBL    SYMBOL                                    GENENAME&#xA;1 ENST00000623808.3             0 ENSG00000228794 LINC01128 long intergenic non-protein coding RNA 1128&#xA;2 ENST00000432963.1             0 ENSG00000230368    FAM41C family with sequence similarity 41 member C&#xA;3 ENST00000446136.1             0 ENSG00000230368    FAM41C family with sequence similarity 41 member C&#xA;4 ENST00000635557.1             0 ENSG00000230368    FAM41C family with sequence similarity 41 member C&#xA;5 ENST00000420190.6             0 ENSG00000187634    SAMD11    sterile alpha motif domain containing 11&#xA;6 ENST00000437963.5             0 ENSG00000187634    SAMD11    sterile alpha motif domain containing 11&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now if we see first 6 rows we have two genes with three different peaks. When i load and see in the igv browser I do see the there are subtle differences in the accessibility signals.&lt;/p&gt;&#xA;&lt;p&gt;How to address this issue ,in other words which peak to consider or take all the peak and consider their accessibility as one.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion or help would be really appreciated.&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2020-12-11T23:32:17.377" Title="ATAC seq peak annotation : Which peak to choose for downstream analysis" Tags="|r|atac-seq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15042" PostTypeId="2" ParentId="15041" CreationDate="2020-12-11T12:32:48.917" Score="2" Body="&lt;p&gt;I'll consider all peaks around promoter of particular gene as one peak. Moreover I'll interpret this as an open chromatin region = active gene. Why? Why there's no one wide peak which represents open chromatin in promoter of particular gene? Why is divided into several narrower peaks? There might be some transcription factors or other proteins binded to DNA, hence you see peak (open, free DNA) - break (valley, TF binded) - peak(again free DNA where Tn5 cut DNA and incorporate adapters) in IGV.&#xA;In my study I compared ATAC-seq and H3k27Ac. In such case usually you may notice that peaks from chip and atac never literally overlap. Because ATAC represents open chromatin (DNA without any binded proteins) and chip histone modification (DNA is actually wound on histone proteins), hence we have something like &lt;a href=&quot;https://www.researchgate.net/publication/318540233/figure/fig2/AS:613860998410247@1523367275394/Mapping-of-ATAC-Seq-Reads-at-Known-Endothelial-Enhancers-A-C-Visualization-of-ATAC-seq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; figure C, first we see high ATAC peak - open chromatin, then H3k27ac peak - lysine acetylated = gene activated. In other words ATAC-seq won't show you peaks around histones, binded polymerase, or transcription factors.&lt;/p&gt;&#xA;" OwnerUserId="2869" LastActivityDate="2020-12-11T12:32:48.917" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15043" PostTypeId="2" ParentId="15035" CreationDate="2020-12-11T13:37:20.740" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/refseq/about/prokaryotes/reannotation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; you have the explanation:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Suppressed genomes: Several hundred RefSeq bacterial genomes that did&#xA;not pass assembly or annotation quality validation have been&#xA;suppressed(RefSeq genome quality criteria). An FTP report file listing&#xA;the most recent suppressions will be provided soon in the RefSeq FTP&#xA;site.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2869" LastActivityDate="2020-12-11T13:37:20.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15044" PostTypeId="2" ParentId="4965" CreationDate="2020-12-11T17:02:57.150" Score="2" Body="&lt;p&gt;The &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/4966/9750&quot;&gt;answer by terdon&lt;/a&gt; is probably good enough, but some commenters have noted it will allow something like &lt;code&gt;ENSABCDEFGHIJKL12345678901&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;It's helpful to reference the official documentation of &lt;a href=&quot;https://uswest.ensembl.org/info/genome/stable_ids/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl Stable Ids&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Stable IDs are created in the form ENS[species prefix][feature type prefix][a unique eleven digit number]. For example a mouse gene might be ENSMUSG###########. This means that we can immediately tell from a stable ID what kind of feature they refer to and what species they are in. The prefixes are listed on our &lt;a href=&quot;https://uswest.ensembl.org/info/genome/stable_ids/prefixes.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;prefix page&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The two places I look first for regexes are identifiers.org and wikidata. If these are wrong, best to report that upstream. Here are their ensembl regexes:&lt;/p&gt;&#xA;&lt;p&gt;The regex used by the &lt;a href=&quot;https://registry.identifiers.org/registry/ensembl&quot; rel=&quot;nofollow noreferrer&quot;&gt;identifiers.org registry&lt;/a&gt; (for any ensembl ID) is quite long:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;^((ENS[FPTG]\\d{11}(\\.\\d+)?)|(FB\\w{2}\\d{7})|(Y[A-Z]{2}\\d{3}[a-zA-Z](\\-[A-Z])?)|([A-Z_a-z0-9]+(\\.)?(t)?(\\d+)?([a-z])?))$&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Meanwhile Wikidata has this regex for an &lt;a href=&quot;https://www.wikidata.org/wiki/Property:P594&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl gene ID&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(ENS(|MUS|RNO)G\d{11})|(Y[A-P][LR]\d{3}[CW](-[A-G])?)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9750" LastActivityDate="2020-12-11T17:02:57.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15045" PostTypeId="1" AcceptedAnswerId="15047" CreationDate="2020-12-11T18:03:25.397" Score="1" ViewCount="67" Body="&lt;p&gt;&lt;strong&gt;Align the HIV-1 Gag protein with the Gag protein of Visna virus&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; Visna virus is a lentivirus causing encephalitis in sheep&lt;/p&gt;&#xA;&lt;p&gt;The problem is align the follow two protein Gag of viruses HIV-1 (GenBank KU749412.1) and Gag Visna virus (GenBank L06906.1) that are:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;MGARASILRGEKLDTWERIRLRPGGKKHYMLKHLVWASRELDRFALNPGLLETLEGCKQIIKQLQPALQTGTEELRSLFNTVATLYCVHAGIPVKDTKEALDKIEEEQNKCQQKAKQAEAAAAGKISLNYPIVQNLQGQMVHQAISPRTLNAWVKVIEEKAFSPEVIPMFTALSEGATPQDLNTMLNTVGGHQAAMQMLKDTINEEAAEWDRLHPVHAGPVAPGQMREPRGSDIAGTTSTLAEQIAWITGNPPVPVGEIYKRWIILGLNKIVRMYSPXSILDIRQGPKEPFRDYVDRFFKTLRAEQATQDVKNWMTDTLLVQNANPDCKTILRALGPGATLEEMMTACQGVGGPGHKARVLAEAMSQTTNAIMMQRSNFKGPKRTIKCFNCGKEGHLARNCRAPRKKGCWKCGKEGHQMKDCNERQANFLGKIWPSHKGRPGNFLQSRPEPTAPPAESFRFEETTPAPKQEQKDKEPLTSLKSLFGSDPLSQ&amp;quot;   &#xA;&amp;quot;MAKQGSKEKKGYPELKEVIKATCKIRVGPGKETLTEGNCLWALKTIDFIFEDLKTEPWTITKMYTVWDRLKGLTPEETSKREFASLQATLACIMCSQMGMKPETVQAAKGIISMKEGLHENKEAKGEKVEQLYPNLEKHREVYPIVNLQAGGRSWKAVESVVFQQLQTVAMQHGLVSEDFERQLAYYATTWTSKDILEVLAMMPGNRAQKELIQGKLNEEAERWVRQNPPGPNVLTVDQIMGVGQTNQQASQANMDQARQICLQWVITALRSVRHMSHRPGNPMLVKQKNTESYEDFIARLLEAIDAEPVTDPIKTYLKVTLSYTNASTDCQKQMDRTLGTRVQQATVEEKMQACRDVGSEGFKMQLLAQALRPQGKAGQKGVNQKCYNCGKPGHLARQCRQGIICHHCGKRGHMQKDCRQKKQQGNNRRGPRVVPSAPPML&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I propose that the better alignment method is the one that maximize the function&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;f(string1, string2)=(number of letter equal between strign1 and string2)/(common length of strings). &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With that definition I found f(s1,s2)=0.25760 (and 0.2949 between POL protein) for the two strings above.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What score might others find?&lt;/li&gt;&#xA;&lt;li&gt;Is that definition one of biological significative?&lt;/li&gt;&#xA;&lt;li&gt;Is there is a better one, what definition for align proteins?&lt;/li&gt;&#xA;&lt;li&gt;Are Visna and HIV viruses relatives?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="7054" LastEditorUserId="3848" LastEditDate="2023-08-06T00:01:47.513" LastActivityDate="2023-08-06T00:01:47.513" Title="Align the HIV-1 Gag protein with the Gag protein of Visna virus" Tags="|phylogenetics|sequence-alignment|phylogeny|proteins|virus|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15046" PostTypeId="2" ParentId="15041" CreationDate="2020-12-11T23:32:17.377" Score="2" Body="&lt;p&gt;You can see from your table that the different peaks correspond to different promoters for different transcripts, for example, FAM41C has ENST00000432963.1, ENST00000446136.1, ENST00000635557.1. And the fact that they are all within the TSS (see distanceToTSS) suggest these peaks are actually falling onto actually TSS.&lt;/p&gt;&#xA;&lt;p&gt;The underlying issue is are these TSS or do will we mis classify the peaks that are inside genes or other non-coding elements as TSS peaks. Nothing to do with TF, histones etc.&lt;/p&gt;&#xA;&lt;p&gt;I cannot see the number of reads in those peaks, but if one of them is really strong, and the rest almost non-existent it would make sense.&lt;/p&gt;&#xA;&lt;p&gt;How can we improve this, I think it depends on the aim of your analysis. Since you are looking at TSS, I guess it makes sense to be able to link it to something that is actually expressed. You can do:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;To use an updated annotation (i see that some of the transcripts come from old hg37 annotation)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;you have prior knowledge which transcripts might be expressed from RNA seq data, I would use that to reduce the number of transcripts&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2020-12-11T23:32:17.377" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15047" PostTypeId="2" ParentId="15045" CreationDate="2020-12-12T05:30:01.210" Score="2" Body="&lt;p&gt;Both are lentiviruses and &lt;code&gt;env&lt;/code&gt; is a fast evolving gene. What you are looking at here is the location of indels and this will vary between alignment algorithms.&lt;/p&gt;&#xA;&lt;p&gt;There will always be an element of error in attempting to align over this genetic distance. It depends what you end goal is, if it is to maximise homology than okay, however optical enhancement of alignments are common if your downstream analysis is phylogenetics. If it is protein structure homology style modelling, where you have the protein structure of HIV env experimentally determined mulitple times you want to use it to help estimate that of the cat lentivirus then optimising a scoring metric is okay.&lt;/p&gt;&#xA;&lt;p&gt;What you have to understand is there is no 'right' method, in phylogenetics this doesn't matter as much because indels are excluded in the subsequent calculation, so it is less sensative to indel positioning errors and optical enhancement is used. You can also simply exclude regions that appear 'garbled'. For protein homology modeling this is a big problem, because the indels will make drastic changes to the structure prediction. So the only approach here in my opinion is to produce a separate structural prediction for every variation in indel alignment you have obtained. In the latter context getting the alignment right is everything and I suspect thats the output here, because its a classic surface antigen to understand, which will readily crystalise.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2020-12-12T11:59:18.573" LastActivityDate="2020-12-12T11:59:18.573" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15049" PostTypeId="1" CreationDate="2020-12-12T15:46:01.157" Score="1" ViewCount="487" Body="&lt;p&gt;I am trying to run Prokka but the following error message appears:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Could not run command: makeblastdb -hash_index -dbtype prot -in /cluster/software/prokka/1.13.7-gompi-2019a/db/kingdom/Archaea/sprot -logfile /dev/null&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;When I removed the &lt;code&gt;--setupdb&lt;/code&gt;, the following error message appeared:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The sequence databases have not been indexed. Please run 'prokka --setupdb' first&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;after that, I changed my script to&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;prokka --setupdb --kingdom Bacteria -o &lt;span class=&quot;math-container&quot;&gt;$OUTPUTFILE.txt o- $&lt;/span&gt;OUTPUTFILE.tsv o- &lt;span class=&quot;math-container&quot;&gt;$OUTPUTFILE.tbl o- $&lt;/span&gt;OUTPUTFILE.faa o- &lt;span class=&quot;math-container&quot;&gt;$OUTPUTFILE.gff /cluster/work/users/ahmedbar/191129_M01132.Project_Petersen-DNAlibs2-2019-11-06/prokka_A_C/$&lt;/span&gt;SAMPLE_ID_1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and still not working and the following error message appeared&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Could not run command: makeblastdb -hash_index -dbtype prot -in /cluster/software/prokka/1.13.4-foss-2018b/db/kingdom/Archaea/sprot -logfile /dev/null&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="10015" LastEditorUserId="3848" LastEditDate="2020-12-12T17:05:53.263" LastActivityDate="2020-12-12T17:05:53.263" Title="Prokka error message" Tags="|sequence-alignment|sequence-annotation|software-usage|orf|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15050" PostTypeId="1" AcceptedAnswerId="15053" CreationDate="2020-12-13T11:12:41.210" Score="-1" ViewCount="490" Body="&lt;p&gt;Hi I am trying to learnt python3 and Biopython,&lt;/p&gt;&#xA;&lt;p&gt;I am trying to check imported fasta file before processing so far using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sequences = SeqIO.parse(&amp;quot;fake_fasta_file&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;&#xA;if not any(sequa):&#xA;        print('\n  SOMETHING IS WRONG WITH YOUR INPUT FILE \n')&#xA;        exit()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;works but as soon as I enter a '&amp;gt;' in my fake_fasta_file , as an example&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;nihvivicòvo-l&#xA;,mnjobùJN&#xA;, Békibpç&#xA;, mnkn&#xA;&#xA;nbigp&#xA;ù&#xA;mlkn&#xA;&#xA;mlnù&#xA;&amp;gt;  ftfyfiuuogpigèi&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;any(sequences) returns True even if:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for seq_record in sequences:&#xA;       print(seq_record)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;doesnt print anything&lt;/p&gt;&#xA;&lt;p&gt;Any better way to check for real fasta file in Biopython ?&lt;/p&gt;&#xA;&lt;p&gt;Or maybe it my fault using any() on the iterator instead of creating&lt;/p&gt;&#xA;&lt;p&gt;a list of seq_record and checking on that ?&lt;/p&gt;&#xA;" OwnerUserId="10049" LastActivityDate="2020-12-13T17:58:42.347" Title="Biopython SeqIO check input file" Tags="|fasta|python|seqio|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15053" PostTypeId="2" ParentId="15050" CreationDate="2020-12-13T15:46:25.593" Score="1" Body="&lt;p&gt;&lt;code&gt;any()&lt;/code&gt; loops over the iterator until the first truthy item is found or the end is reached. The iterator is not reset to the start after this. Using a list will change that behaviour since a list is not exhausted like a iterator. So, remove your &lt;code&gt;any()&lt;/code&gt; call and you will see a sequence record printed. Your &lt;code&gt;'fake_fasta_file.fa'&lt;/code&gt; is actually recognised by Biopython as valid fasta:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; SeqIO.read('fake_fasta_file.fa', 'fasta')&#xA;SeqRecord(seq=Seq(''), id='ftfyfiuuogpigèi', name='ftfyfiuuogpigèi', description='  ftfyfiuuogpigèi', dbxrefs=[])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Biopython ignores lines before the first line beginning with &lt;code&gt;'&amp;gt;'&lt;/code&gt; (as possible comment lines) and then starts collecting records, in this case an empty sequence record is still a record. This behaviour is deliberate from Biopython, although you could argue if it's best or not, it's unlikely to change for historical reasons.&lt;/p&gt;&#xA;&lt;p&gt;What is &lt;em&gt;real&lt;/em&gt; fasta? (philosophical pause) If you want to validate a particular definition of FASTA format then you can write your own parser depending on the exact specification and rules you want.&lt;/p&gt;&#xA;&lt;p&gt;Edit, e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;records = SeqIO.parse(&amp;quot;myfile.fa&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;first = next(records, None)&#xA;if first is None:&#xA;    print(&amp;quot;No fasta records&amp;quot;)&#xA;elif first.seq == &amp;quot;&amp;quot;:&#xA;    print(&amp;quot;Empty sequence&amp;quot;)&#xA;elif all(c not in &amp;quot;ATCGN&amp;quot; for c in first.seq.upper()):&#xA;    print(&amp;quot;Non ATCGN characters&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;etc.&lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="104" LastEditDate="2020-12-13T17:58:42.347" LastActivityDate="2020-12-13T17:58:42.347" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15054" PostTypeId="2" ParentId="10845" CreationDate="2020-12-14T19:17:14.747" Score="0" Body="&lt;p&gt;tximport is recommended by the DESeq author, that can account for gene lengths.&lt;/p&gt;&#xA;&lt;p&gt;Of course, you don't really want to account for gene length, so much as transcript length.  And since you might have different transcripts in there, you will have to use something like RSEM or kallisto which will intelligently figure out the right transcript length.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-12-14T19:17:14.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15055" PostTypeId="2" ParentId="15025" CreationDate="2020-12-14T21:53:46.137" Score="1" Body="&lt;p&gt;I updated to the newest version of Cromwell and the error went away!&lt;/p&gt;&#xA;" OwnerUserId="10036" LastActivityDate="2020-12-14T21:53:46.137" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15056" PostTypeId="2" ParentId="5144" CreationDate="2020-12-15T00:47:48.457" Score="1" Body="&lt;p&gt;This looks like a TMT proteomics experiment. What tools/pipeline did you use to arrive at your quantitative values?&lt;/p&gt;&#xA;&lt;p&gt;There are a variety of tools that are useful for analyzing this type of data and supply appropriate imputation methods. If you are comfortable programming in R, one I would suggest is the MSstatsTMT R package: &lt;a href=&quot;http://msstats.org/msstatstmt/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://msstats.org/msstatstmt/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;That being said, any ways you can improve the upstream analysis to decrease the amount of missing values will be extremely beneficial.&lt;/p&gt;&#xA;" OwnerUserId="9197" LastActivityDate="2020-12-15T00:47:48.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15058" PostTypeId="1" CreationDate="2020-12-15T10:09:21.777" Score="1" ViewCount="18" Body="&lt;p&gt;I have a hundred insect genomes and I'm looking for &lt;code&gt;repeated regions&lt;/code&gt; along these &lt;code&gt;assemblies&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;First of all, I thought of using &lt;code&gt;RepeatModeler&lt;/code&gt;, only the most familiar among you know that it is not at all optimized software that requires a lot of resources and calculation time, which is therefore not feasible in my case.&lt;/p&gt;&#xA;&lt;p&gt;One possible solution &lt;code&gt;Repbase&lt;/code&gt; is not free.&lt;/p&gt;&#xA;&lt;p&gt;I thought for example to use a &lt;code&gt;blastn&lt;/code&gt; approach by using in query the assemblages and in database of nucleotide databases containing &lt;strong&gt;ETs&lt;/strong&gt;, which would be an approach maybe less sensitive but much faster! Only I can't find a database in ET representative of &lt;code&gt;eukaryotes&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have two questions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there an alternative algorithm that is fast enough to process a large number of insect genomes?&lt;/li&gt;&#xA;&lt;li&gt;Is there a database of representative ETs to enable a &lt;code&gt;blastn&lt;/code&gt; style solution?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Thanks a lot.&lt;/p&gt;&#xA;" OwnerUserId="2328" LastEditorUserId="3848" LastEditDate="2020-12-15T11:46:30.353" LastActivityDate="2020-12-15T11:46:30.353" Title="Repeat analysis on eukaryotic assemblies" Tags="|assembly|repeat-elements|repeat|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15059" PostTypeId="1" CreationDate="2020-12-15T10:42:18.607" Score="1" ViewCount="548" Body="&lt;p&gt;I want to select multiple residues from different PDB files loaded on the same pymol session.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;select resi LIG&#xA;get_area sele&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How do I loop the above command to enable the command below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;select ligands,resn LIG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6445" LastEditorUserId="3848" LastEditDate="2020-12-15T12:06:03.180" LastActivityDate="2023-06-30T17:44:52.640" Title="Loop Pymol command get_area" Tags="|python|pymol|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15060" PostTypeId="1" CreationDate="2020-12-15T11:45:01.970" Score="0" ViewCount="37" Body="&lt;p&gt;Im using python and wish to get properties from each one letter amino acid?&lt;/p&gt;&#xA;&lt;p&gt;I mean&#xA;M(Met) =&amp;gt; Density:1.340, MW:149.21...&lt;/p&gt;&#xA;&lt;p&gt;The RDkit methods atom.GetSymbol(), atom.GetDegree() permit this and I am looking for an equivalent solution in Python.&lt;/p&gt;&#xA;&lt;p&gt;Are there any suitable package, for example in Biopython?&lt;/p&gt;&#xA;" OwnerUserId="10077" LastEditorUserId="3848" LastEditDate="2020-12-15T12:10:08.823" LastActivityDate="2020-12-15T12:10:08.823" Title="How can I get properties from each one letter amino acid?" Tags="|python|proteins|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15061" PostTypeId="1" AcceptedAnswerId="19548" CreationDate="2020-12-15T14:16:10.337" Score="2" ViewCount="417" Body="&lt;p&gt;I am using Biopython Phylo and &lt;a href=&quot;https://www.rdkit.org/docs/source/rdkit.ML.Cluster.Butina.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;RDkit Cluster&lt;/a&gt; to obtain a UPGMA tree from a distance matrix of 5k x 5k entries. How can I extract the taxa names within a given clade(s)?&lt;/p&gt;&#xA;&lt;p&gt;For example, Bio.Phylo can return plain-text representation of a tree, so I think I can parse it by the number of tabs:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Phylogeny()&#xA;     Clade()&#xA;         Clade(branch_length=0.06)&#xA;             Clade(branch_length=0.102, name='A')&#xA;             Clade(branch_length=0.23, name='B')&#xA;         Clade(branch_length=0.4, name='C') &#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or, ASCII format:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;          _____________ A&#xA;  _______|&#xA;_|       |_______________________________ B&#xA; |&#xA; |_______________________________________________________ C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which, I think is more difficult to parse but possible.&#xA;Either way, those are not established methods for this kind of task, so I would like a hint.&lt;/p&gt;&#xA;" OwnerUserId="7150" LastEditorUserId="3848" LastEditDate="2022-08-11T02:21:51.747" LastActivityDate="2022-08-16T12:26:48.513" Title="How to extract clusters from a phylogenetic tree?" Tags="|phylogenetics|python|phylogeny|clustering|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15062" PostTypeId="1" CreationDate="2020-12-15T19:00:49.630" Score="2" ViewCount="66" Body="&lt;p&gt;I have an &lt;code&gt;awk&lt;/code&gt; command that identifies 30 pb from two multifasta files.&lt;/p&gt;&#xA;&lt;p&gt;When I used two input files:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;E.g. 100 sequences each, I get the same result with the &lt;code&gt;awk&lt;/code&gt; command and &lt;code&gt;blastn&lt;/code&gt; command.&lt;/li&gt;&#xA;&lt;li&gt;E.g. with 1,000 sequences each I get the same result with both commands.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; However, when I increase the number of sequences to 10,000 I get different outputs, any idea why this could happend?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Command-#sequences-matches&#xA;awk      -100-       1&#xA;blastn   -100-       1&#xA;awk     -1000-      73&#xA;blastn  -1000-      73&#xA;awk     -10000-     8384&#xA;blastn  -10000-     8280&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;awk command:&#xA;&#xA;BEGIN { wid = 30 }&#xA;&#xA;sub(/^&amp;gt;/,&amp;quot;&amp;quot;) { hdr=$1; next }&#xA;&#xA;NR == FNR { a[hdr]=$0; next }&#xA;&#xA;{&#xA;&#xA;    for ( hdrA in a ) {&#xA;&#xA;        strA  = a[hdrA]&#xA;&#xA;        lgthA = length(strA)&#xA;&#xA;        for ( idxA=1; idxA&amp;lt;=(lgthA - wid); idxA++ ) {&#xA;&#xA;            substrA = substr(strA,idxA,wid)&#xA;&#xA;            if ( index($0, substrA) ) {&#xA;&#xA;                printf &amp;quot;[%s,%s]\n&amp;quot;, hdrA, hdr&#xA;&#xA;                break&#xA;            }&#xA;        }&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;awk -f tst.awk file_with_100seqs file_with_100seqsv2&lt;/code&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;blastn command:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;code&gt;blastn -query file_with_100seqs -subject file_with_100seqsv2 -out blastn_output_seqs1vs2_100.txt -word_size 30 -perc_identity 100 -outfmt &amp;quot;10 qseqid sseqid&amp;quot;&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The input files look like this:&lt;/p&gt;&#xA;&lt;p&gt;#File1&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1  &#xA;TGATTGCATAACCACTTAACATCTTGTTTTATCTAAATAAAATTAAGCATGTTATCTTTTTGGGGCACTCCTGGGGCAGTAGATGCCAGTTGTTGATTCAGTATATCTACTTGTGACTGGTTATTATCCCGATTTTTTTAGTTTTAAGGTGTTGACATAGCCATCCATGCTCCATATACTGTATAGACCATCTGAGCGTT  &#xA;&amp;gt;2  &#xA;TGGGAAAACAGCATTCAGCGGTGGCTTATTCCTGCTAAGGATGTTGGCCGCATTCATGCTGAGCACAACCTCGACGGCCTGCTGAGGGGCGATTCGGCATCCCGCGCTGCCTTTATGAAGGCAATGGGAGAGGCAGGGCTACGCACCATCAACGAGATGCGACGAACGGACAACCTCCCGCCATTGCCGGGTGGCGATGT  &#xA;&amp;gt;3  &#xA;GAAATGGGAACCGCGAACATGCCTGCACATCCGTTTGTGCGACCCGCTTACGATACTCGCGAGGAAGAGGCCGCCAGCGTCGCCATTGCCAGGATGAATCAGGCTATTGATGAGGTATTGAGCAAGTGAATGAAGATAATATCTACGCCTTGCTTTCTCCCCTGGCAGAAGGACGGGTATATCCCTATGTTGCGCCATTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;#File2&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1  &#xA;TGATTGCATAACCACTTAACATCTTGTTTTATCTAAATAAAATTAAGCATGTTATCTTTTTGGGGCACTCCTGGGGCAGTAGATGCCAGTTGTTGATTCAGTATATCTACTTGTGACTGGTTATTATCCCGATTTTTTTAGTTTTAAGGTGTTGACATAGCCATCCATGCGGGAAGGTGCAGCATAATGTGCTTTGGATT  &#xA;&amp;gt;2  &#xA;TGAGTGCCCCATTTGTGAAGCAATAAAGTTCGGGTTCGCGCCAGCGGCAAGCGCCCAGCATGCACCGATTTTTTTAGTTTTAAGGTGTTGACATTAGGTATGTCGGGACTGGTATGCTTTCCTGTGTCGCAGCCCGGCGCGTCTCAATGCAGATTCCCATATCCTGTTCATCCATATACTGTATAGACCATCTGAGCGTT  &#xA;&amp;gt;3  &#xA;TACCTGAGCGATCGGTAATTTGCGGATTGAAGACAAAGGTGCAGGAATGAGTTTTTGTACGACCGTATTCGCGCAGCTTTACTTCAATTTTGTGCTGTTTGCTCAGCTTCGTGAAAGAGGCCTGACTTTTTAAAGCATCAATTGCTGGCTGCACAAGATGTATCACCCTGTCGGTTCCTGCCTGGGTTTTCGGCAGGGTG  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10037" LastEditorUserId="3848" LastEditDate="2020-12-20T10:50:05.227" LastActivityDate="2020-12-20T10:50:05.227" Title="Why do I obtain different output results with blast vs awk commands" Tags="|fasta|shell|awk|blastn|multiple-sequence-alignment|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15063" PostTypeId="2" ParentId="15059" CreationDate="2020-12-15T19:13:55.397" Score="3" Body="&lt;p&gt;from &lt;a href=&quot;https://www.blopig.com/blog/2019/08/how-to-iterate-in-pymol/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.blopig.com/blog/2019/08/how-to-iterate-in-pymol/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Example 1: Print the names of all the selected objects&lt;/p&gt;&#xA;&lt;p&gt;You’ve selected several objects in PyMOL, and you want to see their names. By default, a new selection object called “(sele)” is created. We can create a Python variable called seleobjs, say, and then loop over them:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seleobjs = cmd.get_object_list('(sele)')&#xA;    for obj in seleobjs: print obj&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Example 2: Disable all the selected objects&lt;/p&gt;&#xA;&lt;p&gt;If you still have your selection, “(sele)”, you can disable them (effectively turn them off on the right-hand sidebar):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seleobjs = cmd.get_object_list('(sele)')&#xA;    for obj in seleobjs: cmd.disable(obj)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Example 3: Delete all the selected objects&lt;/p&gt;&#xA;&lt;p&gt;You’re done with the objects in your selection, “(sele)”: you might be able to delete them using “delete ”, but if you can’t write a wildcard that covers all the names of the objects, you can iterate over a selection and delete them:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seleobjs = cmd.get_object_list('(sele)')&#xA;    for obj in seleobjs: cmd.delete(obj)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;from &lt;a href=&quot;https://pymolwiki.org/index.php/Get_Area&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pymolwiki.org/index.php/Get_Area&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;get_area calculates the surface area in square Angstroms of the selection given. Note that the accessibility is assessed in the context of the object(s) that the selection is part of. So, to get the surface areas of e.g. a component of a complex, you should make a new object containing a copy of just that component and calculate its area.&lt;/p&gt;&#xA;&lt;p&gt;please let us know if it worked&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2020-12-16T09:23:50.497" LastActivityDate="2020-12-16T09:23:50.497" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="15064" PostTypeId="1" CreationDate="2020-12-15T21:26:02.727" Score="3" ViewCount="139" Body="&lt;p&gt;What is the origin of HIV1?&lt;/p&gt;&#xA;&lt;p&gt;This is the alignment of a small part of the POL protein of the HIV1 virus GenBank KU749412.1, with the POL protein of the Visna virus GenBank L06906.1&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;IKKKDSTKWRKLVDFRELNKRTQDFWEVQLGIPHPAGLKKKKSVTVLDVGDAYFSVPLDESFRKYTAFTI--PSTNNETPGIRYQYNVLPQGWKGSPAIFQSSMTKILEPFRAQNPEIVIYQYMDDLYVGSDLEIGQHRAKIEELREHLLRWGFTTPDKKHQKEPPF--------LWMGYELHPDKWTVQPIQLPE--KDSWTVNDIQKLV&#xA;|||| | ||| | ||||||| | |  | ||| ||| ||  || || || |||||  || |  | || |    |  ||  |  ||   ||||||| |||  |  | |||       | |    |||| | ||||  |      || |                                | |||| ||  |   |||      | |  ||||                 &#xA;IKKK-SGKWRMLIDFRELNKQTEDLAEAQLGLPHPGGLQRKKHVTILDIGDAYFTIPLYEPYRQYTCFTMLSP--NNLGPCVRYYWKVLPQGWKLSPAVYQFTMQKILRGWIEEHPMIQFGIYMDDIYIGSDL--GL-----EEHRGIVNELASYIAQYGFMLPEDKRQEGYPAKWL-GFELHPEKWKFQKHTLPEITEGPITLNKLQKLV&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Are these matches,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;By chance or convergence,&lt;/li&gt;&#xA;&lt;li&gt;HIV1 and Visna virus share a recent common ancestor or,&lt;/li&gt;&#xA;&lt;li&gt;the common ancestor was closer Visna virus from where HIV descended?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Could it be that HIV1 really comes from some kind of experiment involving Visna virus, as I read on the Internet?&lt;/p&gt;&#xA;&lt;p&gt;The disease caused by the Visna virus is (100% deadly see &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/16425963/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pubmed.ncbi.nlm.nih.gov/16425963/&lt;/a&gt; ). Could there be parallels to sars2?&lt;/p&gt;&#xA;&lt;p&gt;But look at it: the Visna virus is transmitted by air the first symptoms are like pneumonia (or a brain infection) and otherwise are chronic progressing slowly like SARS-CoV-2 infection resulting in long Covid.&lt;/p&gt;&#xA;&lt;p&gt;Symptoms are also felt after many days that the virus has come into action, this seems to me not to happen for coronaviruses or for flu viruses but for HIV and Visna virus.&lt;/p&gt;&#xA;&lt;p&gt;Related links:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/8289690/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pubmed.ncbi.nlm.nih.gov/8289690/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/16425963/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pubmed.ncbi.nlm.nih.gov/16425963/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;(preprint sars2 as retrovirus) &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2020.12.12.422516v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/10.1101/2020.12.12.422516v1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.gao.gov/assets/100/91319.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.gao.gov/assets/100/91319.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="7054" LastEditorUserId="3694" LastEditDate="2022-03-01T18:09:39.433" LastActivityDate="2022-03-01T18:09:39.433" Title="What is the origin of HIV1?" Tags="|phylogenetics|proteins|phylogeny|covid-19|virus|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15066" PostTypeId="1" CreationDate="2020-12-16T02:05:53.013" Score="1" ViewCount="1158" Body="&lt;p&gt;Ideally, I'd like to do something like the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;f = pysam.AlignmentFile(filename, index_filename=index_filename)&#xA;read = list(f.fetch('Chromosome', 1, 10000))[0]&#xA;read.get_variations()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And have that output something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[&#xA;  (1, 'A', 'C'), # An A -&amp;gt; C mismatch at position 1&#xA;  (15, 'I', 'CGGG'), # An insertion of the string 'CGGG' starting at position 15 of the read&#xA;  (32, 'D', 4), # A deletion of 4 base pairs starting at position 32&#xA;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I know that it's possible to figure out all of those values by parsing the CIGAR string and MD tags but I'm wondering if there's a more intuitive interface.&lt;/p&gt;&#xA;" OwnerUserId="9" LastActivityDate="2020-12-16T07:49:43.570" Title="Is there a straightforward way to get mismatches/indels in a BAM file using pysam?" Tags="|sam|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15067" PostTypeId="2" ParentId="15066" CreationDate="2020-12-16T07:49:43.570" Score="1" Body="&lt;p&gt;You're looking for &lt;code&gt;pileup&lt;/code&gt;, which is the htslib (and thus samtools/bcftools) method for finding variants. See &lt;a href=&quot;https://pysam.readthedocs.io/en/latest/api.html#introduction&quot; rel=&quot;nofollow noreferrer&quot;&gt;this section&lt;/a&gt; of the pysam documentation. You may want to peruse about &lt;a href=&quot;https://pysam.readthedocs.io/en/latest/api.html#pysam.PileupColumn&quot; rel=&quot;nofollow noreferrer&quot;&gt;the output&lt;/a&gt; as well. The gist is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pysam&#xA;samfile = pysam.AlignmentFile(&amp;quot;ex1.bam&amp;quot;, &amp;quot;rb&amp;quot; )&#xA;for pileupcolumn in samfile.pileup(&amp;quot;chr1&amp;quot;, 100, 120):&#xA;    for pileupread in pileupcolumn.pileups:&#xA;        if not pileupread.is_del and not pileupread.is_refskip:&#xA;            # query position is None if is_del or is_refskip is set.&#xA;            ...handle mismatches...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-12-16T07:49:43.570" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15068" PostTypeId="2" ParentId="14913" CreationDate="2020-12-16T09:37:39.423" Score="0" Body="&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;import sys&#xA;import re &#xA;&#xA;unique = []&#xA;fasta_file = (sys.argv[1])&#xA;for myfile in SeqIO.parse(fasta_file, &amp;quot;fasta&amp;quot;):&#xA;    if len(myfile) &amp;gt; 250:&#xA;        gene_id = myfile.id&#xA;        if gene.id not in unique:&#xA;            unique.append(gene.id)&#xA;            mylist = re.match(r&amp;quot;H149xcV_\w+_\w+_\w+&amp;quot;, gene_id)&#xA;            print (&amp;quot;&amp;gt;&amp;quot;+mylist.group(0)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;let me know if it works I am learning too&lt;/p&gt;&#xA;" OwnerUserId="10049" LastActivityDate="2020-12-16T09:37:39.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15069" PostTypeId="2" ParentId="14101" CreationDate="2020-12-16T10:01:56.187" Score="0" Body="&lt;p&gt;you can try PDBx Python Parser too:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://mmcif.wwpdb.org/docs/sw-examples/python/html/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mmcif.wwpdb.org/docs/sw-examples/python/html/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;it has an example showing exactly what you need&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://mmcif.wwpdb.org/docs/sw-examples/python/html/fasta.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mmcif.wwpdb.org/docs/sw-examples/python/html/fasta.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10049" LastActivityDate="2020-12-16T10:01:56.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15070" PostTypeId="1" CreationDate="2020-12-16T10:06:16.020" Score="1" ViewCount="146" Body="&lt;p&gt;I have a number of plink files from different arrays (mixed Illumina and Affymetrix) that need to be merged. However, these use different allele codings, so trying to merge without fixing this causes an error (3+ alleles). One can fix this by using (on each file) &lt;code&gt;--recode 12&lt;/code&gt; to a PED file, and then converting back to BED (&lt;code&gt;--make-bed&lt;/code&gt;) and then merging (&lt;code&gt;--merge-list&lt;/code&gt;). Doing this, I was able to merge the files without explicit errors. However, this method is suboptimal, I think, because a minor allele in dataset 1 might not be the same one as in dataset 2, since this depends on the people in the dataset.&lt;/p&gt;&#xA;&lt;p&gt;For instance, rs746969. Illumina 1MDuo:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;10  rs746969    1.00698e+06 100008620   1   2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Affymetrix 6.0:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;10  rs746969    0   100008620   A   G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alternatively, I could keep the &lt;code&gt;12-encoded&lt;/code&gt; merged data, and attempt to convert back to ACGT. Which is the best approach?&lt;/p&gt;&#xA;" OwnerUserId="9417" LastActivityDate="2020-12-16T10:06:16.020" Title="Merging plink files with divergent allele codings" Tags="|plink|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15071" PostTypeId="1" CreationDate="2020-12-16T10:10:48.540" Score="1" ViewCount="24" Body="&lt;p&gt;I have an .mzTab file, with PRT and PSM values.  Does anyone know of any specific functions that I could use to calculate protein abundance values from this data?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;More details&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/HUPO-PSI/mzTab/blob/master/specification_document-releases/1_0-Proteomics-Release/20minute_guide_mzTab.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This is a summary of what the mzTab file is.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The data I have has two main sections within the file - a PRT section and a PSM section.&lt;/p&gt;&#xA;&lt;p&gt;The PRT section has protein information.  It has the following column headers:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; [1] &amp;quot;accession&amp;quot;                        &amp;quot;description&amp;quot;                     &#xA; [3] &amp;quot;taxid&amp;quot;                            &amp;quot;species&amp;quot;                         &#xA; [5] &amp;quot;database&amp;quot;                         &amp;quot;database_version&amp;quot;                &#xA; [7] &amp;quot;search_engine&amp;quot;                    &amp;quot;best_search_engine_score[1]&amp;quot;     &#xA; [9] &amp;quot;search_engine_score[1]_ms_run[1]&amp;quot; &amp;quot;num_psms_ms_run[1]&amp;quot;  &#xA;[11] &amp;quot;modifications&amp;quot;                   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Where &lt;code&gt;num_psms_ms_run&lt;/code&gt; is the count of the total significant PSMs that can be mapped to the reported protein.&lt;/p&gt;&#xA;&lt;p&gt;The PSM section has peptide spectrum match information.  It has the following column headers:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; [1] &amp;quot;sequence&amp;quot;                              &#xA; [2] &amp;quot;PSM_ID&amp;quot;                                &#xA; [3] &amp;quot;accession&amp;quot;                             &#xA; [4] &amp;quot;unique&amp;quot;                                &#xA; [5] &amp;quot;database&amp;quot;                              &#xA; [6] &amp;quot;database_version&amp;quot;                      &#xA; [7] &amp;quot;search_engine&amp;quot;                         &#xA; [8] &amp;quot;search_engine_score[1]&amp;quot;                &#xA; [9] &amp;quot;search_engine_score[2]&amp;quot;                &#xA;[10] &amp;quot;search_engine_score[3]&amp;quot;                &#xA;[11] &amp;quot;modifications&amp;quot;                         &#xA;[12] &amp;quot;retention_time&amp;quot;                        &#xA;[13] &amp;quot;charge&amp;quot;                                &#xA;[14] &amp;quot;exp_mass_to_charge&amp;quot;                    &#xA;[15] &amp;quot;calc_mass_to_charge&amp;quot;                   &#xA;[16] &amp;quot;spectra_ref&amp;quot;                           &#xA;[17] &amp;quot;pre&amp;quot;                                   &#xA;[18] &amp;quot;post&amp;quot;                                  &#xA;[19] &amp;quot;start&amp;quot;                                 &#xA;[20] &amp;quot;end&amp;quot;                                   &#xA;[21] &amp;quot;opt_global_mzidentml_original_ID&amp;quot;      &#xA;[22] &amp;quot;opt_global_cv_MS:1002217_decoy_peptide&amp;quot;&#xA;[23] &amp;quot;opt_global_cv_PRIDE:0000091_rank&amp;quot;     &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;More information about the column definitions &lt;a href=&quot;http://psidev.info/sites/default/files/2017-07/v_1_0_0_mzTab_format_specification.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;can be found here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="10083" LastEditorUserId="10083" LastEditDate="2020-12-16T10:33:23.230" LastActivityDate="2020-12-16T10:33:23.230" Title="How to calculate protein abundance from an .mzTab file?" Tags="|proteomics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15072" PostTypeId="2" ParentId="15064" CreationDate="2020-12-16T10:22:24.077" Score="6" Body="&lt;p&gt;They are both lentiviruses and share a distant common ancestor.&lt;/p&gt;&#xA;&lt;p&gt;HIV-1 and HIV-2 are descendents from simian immunodeficiency virus (SIV). The following tree shows the relationships very clearly, from Wertheim and Worobey (2009) Dating the Age of the SIV Lineages That Gave Rise to HIV-1 and HIV-2 PLoS Comp Biol. &lt;a href=&quot;https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1000377&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;. The evolution of HIV-1 is heavily mixed with chimanpzee (cpz) SIV, and there are regions of highly anomalous homology juxtaposed with regions of notably lower homology.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nS3t7.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nS3t7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2020-12-16T10:22:24.077" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="15073" PostTypeId="1" AcceptedAnswerId="15107" CreationDate="2020-12-16T12:56:37.760" Score="1" ViewCount="86" Body="&lt;p&gt;I have developed a software for de novo genome assembly. Its performance varies gradually according to how much data you employ. At initial stages it often produces contigs that look like that when aligned to a reference genome:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4iNvj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4iNvj.png&quot; alt=&quot;s-aligner assembly of SRR8357422&quot; /&gt;&lt;/a&gt;&#xA;&lt;strong&gt;Initial-stage s-aligner assembly of SRR8357422&lt;/strong&gt;&#xA;Genome fraction:98,8% -&#xA;Largest Alignment: 61.544 -&#xA;NGA50: 59.409 -&#xA;LGA50: 2&lt;/p&gt;&#xA;&lt;p&gt;If I add more data, the contigs end up being larger, but that obviously takes more time. My question is, how does this result compare to, for example, one like that?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RRAfB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RRAfB.png&quot; alt=&quot;SPAdes assembly of SRR8357422&quot; /&gt;&lt;/a&gt;&#xA;&lt;strong&gt;SPAdes assembly of SRR8357422&lt;/strong&gt;&#xA;Genome fraction:93,46% -&#xA;Largest Alignment: 128.391 -&#xA;NGA50: 128.391 -&#xA;LGA50: 1&lt;/p&gt;&#xA;&lt;p&gt;Note how you can construct a contig covering almost the full reference genome assembling green contigs in the first case. Therefore, a blast search of genes will likely let you find more genes given that it covers more genome fraction.&lt;/p&gt;&#xA;&lt;p&gt;Apart from determining what option is more convenient, I would also like to know how much more convenient is to have both results. Or in which cases it's more convenient.&lt;/p&gt;&#xA;&lt;p&gt;Last (but not least!), is there a way to generate automatically larger contigs from these overlapping contigs in assembly 1? I have tried using miniasm with no success. Even minimap2 seems not to find overlaps well enough. I am developing my own algorithm also for that but would like to know if there is another way.&lt;/p&gt;&#xA;&lt;p&gt;[EDITED on 17/12 to add another example with real data for both assemblies]&lt;/p&gt;&#xA;" OwnerUserId="5190" LastEditorUserId="5190" LastEditDate="2020-12-17T21:11:44.413" LastActivityDate="2020-12-23T18:53:12.557" Title="Which one is a more convenient assembly?" Tags="|assembly|reference-genome|genome-sequencing|scaffold|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="15074" PostTypeId="1" CreationDate="2020-12-16T13:29:50.367" Score="1" ViewCount="201" Body="&lt;p&gt;I am independently working on data retrieved SRA database, paired-end data as separate inputs.&#xA;The proceedure I followed is,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;After running &lt;strong&gt;FastQC&lt;/strong&gt; using Galaxy, the majority of the modules have failed.&lt;/li&gt;&#xA;&lt;li&gt;I tried &lt;strong&gt;Trimmomatic&lt;/strong&gt; with default settings (Avg quality =20, number of  bases to avg across = 4), it resulted in R1 and R2 paired and unpaired (4 total outputs).&lt;/li&gt;&#xA;&lt;li&gt;I ran &lt;strong&gt;FastQC&lt;/strong&gt; again, which rendered some changes (per base sequence quality passed) but resulted in other modules (Sequence length distribution and per tile quality scores) getting a warning sign or failing.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My question is,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is  there specific setting other than default that provides better results? What output should i use from Trimmomatic for fastqc again?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9234" LastEditorUserId="3848" LastEditDate="2020-12-17T15:50:16.337" LastActivityDate="2020-12-17T15:50:16.337" Title="FastQC and Trimmomatic in Galaxy?" Tags="|trimming|fastqc|galaxy|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15075" PostTypeId="1" AcceptedAnswerId="15108" CreationDate="2020-12-16T15:17:14.477" Score="2" ViewCount="794" Body="&lt;p&gt;I have a pipeline that has an input list of samples. I would like to split those samples by chromosomes, process each chromosome apart, and then merge them back by sample name.&lt;/p&gt;&#xA;&lt;p&gt;Here's an example:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-java prettyprint-override&quot;&gt;&lt;code&gt;samplesList = Channel.from([&amp;quot;sample1&amp;quot;, &amp;quot;sample2&amp;quot;])&#xA;&#xA;// A process that splits the samples by chromosome as an example&#xA;// In my actual case, I have a VCF that is split by `snpSift Split` command&#xA;// and generates files like [sample]_[chr].vcf&#xA;// In the example below, I only gave sample1&#xA;// But take into consideration that I have multiple samples&#xA;&#xA;process bar {&#xA;   echo true&#xA;&#xA;  output:&#xA;    file(&amp;quot;*.tsv&amp;quot;) into bar_output&#xA;&#xA;  script:&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  echo 1 &amp;gt; sample1_chr1.tsv&#xA;  echo 2 &amp;gt; sample1_chr2.tsv&#xA;  echo 3 &amp;gt; sample1_chr3.tsv&#xA;  echo 4 &amp;gt; sample1_chr4.tsv&#xA;  echo 5 &amp;gt; sample1_chr5.tsv&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;// Process each chromosome apart for each sample&#xA;// This means the output of the process foo&#xA;// is only ONE chromosome per sample&#xA;process foo {&#xA;  input:&#xA;    each x from bar_output  // to treat each file separately from the channel&#xA;  output:&#xA;    file(&amp;quot;*.tsv&amp;quot;) into xasxa&#xA;  script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    echo \\\$(cat &lt;span class=&quot;math-container&quot;&gt;$x)\\\$(cat $&lt;/span&gt;x) &amp;gt; sample1_chr\\\$(cat $x)_2.tsv&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;// How should I write the next process to group back them together by sample name?&#xA;// I do not wish to do a collect() on channel foo because it will force &#xA;// all the processes of foo to finish first.&#xA;// In this example, foo would emit 5 different channels of chromosome for sample1&#xA;// This process should merge sample1 only when all chromosomes are processed&#xA;// independently of other samples. &#xA;process mergeChromosomesBySamples {&#xA;   input:&#xA;     ???&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As the example above, the process &lt;code&gt;mergeChromosomesBySamples&lt;/code&gt; should return something similar/close to&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-java prettyprint-override&quot;&gt;&lt;code&gt;[&amp;quot;sample1&amp;quot;, [&amp;quot;/path/to/workdir/sample1_chr1.tsv&amp;quot;,&#xA;             &amp;quot;/path/to/workdir/sample1_chr2.tsv&amp;quot;,&#xA;             &amp;quot;/path/to/workdir/sample1_chr3.tsv&amp;quot;,&#xA;             &amp;quot;/path/to/workdir/sample1_chr4.tsv&amp;quot;,&#xA;             &amp;quot;/path/to/workdir/sample1_chr5.tsv&amp;quot;]&#xA;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;EDIT1: more explication for the process &lt;code&gt;mergeChromosomesBySamples&lt;/code&gt; is given in the comments and as follows ...&lt;/p&gt;&#xA;&lt;p&gt;The channel &lt;code&gt;xasxa&lt;/code&gt; processes each chromosome apart, this means that the output of the channel &lt;code&gt;xasxa&lt;/code&gt; contains only one chromosome of the sample like &lt;code&gt;&amp;quot;/path/to/workdir/sample1_chr1.tsv&amp;quot;&lt;/code&gt; only. My goal is then to merge the processes out &lt;code&gt;xasxa&lt;/code&gt; for each sample together.&lt;/p&gt;&#xA;&lt;p&gt;I do not wish to use &lt;code&gt;collect()&lt;/code&gt; in this case because it would have to force all the processes of foo to finish before. I want them to work independently for each sample.&lt;/p&gt;&#xA;&lt;p&gt;Where each sample has its own VCF generated and I process them independently.&lt;/p&gt;&#xA;&lt;p&gt;EDIT2: Here's a graph of what I am trying to achieve:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qDygl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qDygl.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If I use &lt;code&gt;.collect()&lt;/code&gt;, then all the processing on the chromosomes of all the samples has to finish first before being able to merge them, which is something I do not want.&lt;/p&gt;&#xA;&lt;p&gt;I appreciate your help.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="4413" LastEditDate="2023-07-12T08:03:24.493" LastActivityDate="2023-07-12T08:14:32.040" Title="How to merge back the processes by sample after splitting by chromosome in Nextflow?" Tags="|nextflow|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15078" PostTypeId="2" ParentId="15075" CreationDate="2020-12-16T16:54:20.570" Score="5" Body="&lt;p&gt;Use the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#groupby&quot; rel=&quot;noreferrer&quot;&gt;groupBy operator&lt;/a&gt;. If the content of channel &lt;code&gt;xasxa&lt;/code&gt; is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;[&amp;quot;/path/to/workdir/sample1_chr1.tsv&amp;quot;, &amp;quot;/path/to/workdir/sample1_chr2.tsv&amp;quot;, &amp;quot;/path/to/workdir/sample2_chr1.tsv&amp;quot;, &amp;quot;/path/to/workdir/sample2_chr2.tsv&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;You could probably do:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;xasxa.groupBy{ it -&amp;gt; it.fileName.toString().split(&amp;quot;_&amp;quot;)[0] }&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The result would be a channel with:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;[sample1:[/path/to/workdir/sample1_chr1.tsv, /path/to/workdir/sample1_chr2.tsv, sample2:[/path/to/workdir/sample2_chr1.tsv, /path/to/workdir/sample2_chr1.tsv]]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;This breaks if you have an underscore in your samplename before &lt;code&gt;_chr*&lt;/code&gt;. More robust would be to initialise all channels with a sample value, so that each channel is in the form of [samplename, chromosome]. Then all you need is a &lt;code&gt;groupTuple&lt;/code&gt; on your channel.&lt;/p&gt;&#xA;" OwnerUserId="3818" LastActivityDate="2020-12-16T16:54:20.570" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15079" PostTypeId="2" ParentId="14834" CreationDate="2020-12-16T21:28:45.520" Score="2" Body="&lt;p&gt;that was solved by changing seq level:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;newnames &amp;lt;- paste0(c(&amp;quot;1&amp;quot;,&amp;quot;2&amp;quot;,&amp;quot;3&amp;quot;,&amp;quot;4&amp;quot;, &amp;quot;5&amp;quot;,&amp;quot;6&amp;quot;,&amp;quot;7&amp;quot;, &amp;quot;8&amp;quot;,&amp;quot;9&amp;quot;,&amp;quot;10&amp;quot;,&amp;quot;11&amp;quot;,&amp;quot;12&amp;quot;,&amp;quot;13&amp;quot;,&amp;quot;14&amp;quot;,&amp;quot;15&amp;quot;,&amp;quot;16&amp;quot;,&amp;quot;17&amp;quot;,&amp;quot;18&amp;quot;,&amp;quot;19&amp;quot;,&amp;quot;20&amp;quot;,&amp;quot;21&amp;quot;,&amp;quot;22&amp;quot;,&amp;quot;X&amp;quot;,&amp;quot;Y&amp;quot;,&amp;quot;MT&amp;quot;))&#xA;names(newnames) &amp;lt;- paste0(c(&amp;quot;NC_000001.11&amp;quot;,&amp;quot;NC_000002.12&amp;quot;,&amp;quot;NC_000003.12&amp;quot;,&amp;quot;NC_000004.12&amp;quot;, &amp;quot;NC_000005.10&amp;quot;,&amp;quot;NC_000006.12&amp;quot;,&amp;quot;NC_000007.14&amp;quot;, &amp;quot;NC_000008.11&amp;quot;,&amp;quot;NC_000009.12&amp;quot;,&amp;quot;NC_000010.11&amp;quot;,&amp;quot;NC_000011.10&amp;quot;,&amp;quot;NC_000012.12&amp;quot;,&amp;quot;NC_000013.11&amp;quot;,&amp;quot;NC_000014.9&amp;quot;,&amp;quot;NC_000015.10&amp;quot;,&amp;quot;NC_000016.10&amp;quot;,&amp;quot;NC_000017.11&amp;quot;,&amp;quot;NC_000018.10&amp;quot;,&amp;quot;NC_000019.10&amp;quot;,&amp;quot;NC_000020.11&amp;quot;,&amp;quot;NC_000021.9&amp;quot;,&amp;quot;NC_000022.11&amp;quot;,&amp;quot;NC_000023.11&amp;quot;,&amp;quot;NC_000024.10&amp;quot;,&amp;quot;NC_012920.1&amp;quot;))&#xA;&#xA;peaks_with_new_seq_level &amp;lt;- renameSeqlevels(peaks_with_old_seq_level,newnames)&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8543" LastActivityDate="2020-12-16T21:28:45.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15080" PostTypeId="1" CreationDate="2020-12-17T09:53:06.523" Score="-1" ViewCount="513" Body="&lt;p&gt;I have a list of Protein ID such as Q9S9U6, A2XLL2, Q06402.&#xA;What name is this ID? UniProtKB ID?&#xA;And how can I get PDB ID from this list?&#xA;Is its PDB ID only one?&lt;/p&gt;&#xA;&lt;p&gt;I'm using Python.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/4959/get-a-list-of-pdb-ids-from-a-list-of-sequence-protein-number&quot;&gt;Get a list of PDB ids from a list of sequence protein number?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;That method of this link gives me some IDs but not only one and usually empty.&lt;/p&gt;&#xA;" OwnerUserId="10090" LastActivityDate="2020-12-22T19:03:19.033" Title="How can I convert Protein ID	to PDB ID?" Tags="|python|proteins|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15083" PostTypeId="2" ParentId="15080" CreationDate="2020-12-18T06:32:46.683" Score="1" Body="&lt;p&gt;Get their sequences and then search the pdb (by his API) for    the obtained sequences&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://search.rcsb.org/#search-services&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://search.rcsb.org/#search-services&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&lt;p&gt;sequence&#xA;This service employs the MMseqs2 software and performs fast sequence matching searches (BLAST-like) based on a user-provided FASTA sequence (with E-value or % Identity cutoffs). Following search targets are available:&#xA;pdb_protein_sequence: all current protein sequences in PDB...&lt;/p&gt;&#xA;" OwnerUserId="10049" LastActivityDate="2020-12-18T06:32:46.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15084" PostTypeId="1" AcceptedAnswerId="15085" CreationDate="2020-12-18T10:34:25.537" Score="3" ViewCount="186" Body="&lt;p&gt;Saw this answer &lt;a href=&quot;https://www.biostars.org/p/472591/&quot; rel=&quot;nofollow noreferrer&quot;&gt;biostar&lt;/a&gt;. Tried to make my metadata as such but still the&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;Error in checkFullRank(modelMatrix) : &amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is my coldata&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dput(coldata)&#xA;structure(list(Group = structure(c(2L, 2L, 2L, 2L, 2L, 2L, 2L, &#xA;2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, &#xA;3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 4L, 4L, 4L, 4L, 4L, 4L, 4L, 1L, &#xA;1L, 1L, 1L, 1L, 1L, 1L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 5L, 6L, 6L, &#xA;6L, 6L, 6L, 6L), .Label = c(&amp;quot;HSC&amp;quot;, &amp;quot;Blast&amp;quot;, &amp;quot;CMP&amp;quot;, &amp;quot;GMP&amp;quot;, &amp;quot;LSC&amp;quot;, &#xA;&amp;quot;Mono&amp;quot;), class = &amp;quot;factor&amp;quot;), Mutation = structure(c(10L, 4L, 3L, &#xA;5L, 2L, 3L, 9L, 11L, 1L, 1L, 7L, 7L, 10L, 7L, 8L, 2L, 10L, 10L, &#xA;10L, 10L, 2L, 2L, 7L, 6L, 6L, 6L, 6L, 6L, 6L, 6L, 6L, 6L, 6L, &#xA;6L, 6L, 6L, 6L, 6L, 6L, 6L, 6L, 6L, 6L, 6L, 6L, 2L, 2L, 5L, 2L, &#xA;9L, 1L, 1L, 8L, 6L, 6L, 6L, 6L, 6L, 6L), .Label = c(&amp;quot;DNMT3A-R882H&amp;quot;, &#xA;&amp;quot;FLT3-ITD&amp;quot;, &amp;quot;IDH2-R140Q&amp;quot;, &amp;quot;KRAS-G13D&amp;quot;, &amp;quot;MAU2-Q98H&amp;quot;, &amp;quot;No&amp;quot;, &amp;quot;NPM1-L287ins(TCTG)&amp;quot;, &#xA;&amp;quot;NRAS-G13D&amp;quot;, &amp;quot;STAG2-R614&amp;quot;, &amp;quot;TET2-E1357&amp;quot;, &amp;quot;TET2-R550&amp;quot;), class = &amp;quot;factor&amp;quot;), &#xA;    Satus = structure(c(2L, 5L, 4L, 6L, 3L, 4L, 9L, 10L, 1L, &#xA;    1L, 7L, 7L, 2L, 7L, 8L, 3L, 2L, 2L, 2L, 2L, 3L, 3L, 7L, 16L, &#xA;    16L, 16L, 16L, 16L, 16L, 16L, 16L, 16L, 16L, 16L, 16L, 16L, &#xA;    16L, 16L, 17L, 17L, 17L, 17L, 17L, 17L, 17L, 12L, 12L, 13L, &#xA;    12L, 9L, 11L, 11L, 14L, 15L, 15L, 15L, 15L, 15L, 15L), .Label = c(&amp;quot;Blast-DNMT3A-R882H&amp;quot;, &#xA;    &amp;quot;Blast-E1357&amp;quot;, &amp;quot;Blast-FLT-ITD&amp;quot;, &amp;quot;Blast-IDH2-R140Q&amp;quot;, &amp;quot;Blast-KRAS-G13D&amp;quot;, &#xA;    &amp;quot;Blast-MAU2-Q98H&amp;quot;, &amp;quot;Blast-NPM1-L287ins(TCTG)&amp;quot;, &amp;quot;Blast-NRAS-G13D&amp;quot;, &#xA;    &amp;quot;Blast-STAG2-R614&amp;quot;, &amp;quot;Blast-TET2-R550&amp;quot;, &amp;quot;LSC-FLT-DNMT3A-R882H&amp;quot;, &#xA;    &amp;quot;LSC-FLT-ITD&amp;quot;, &amp;quot;LSC-MAU2-Q98H&amp;quot;, &amp;quot;LSC-NRAS-G13D&amp;quot;, &amp;quot;Mature-Normal&amp;quot;, &#xA;    &amp;quot;Progenitor-Normal&amp;quot;, &amp;quot;Stem-Normal&amp;quot;), class = &amp;quot;factor&amp;quot;)), row.names = c(&amp;quot;Blast1&amp;quot;, &#xA;&amp;quot;Blast10&amp;quot;, &amp;quot;Blast11&amp;quot;, &amp;quot;Blast12&amp;quot;, &amp;quot;Blast13&amp;quot;, &amp;quot;Blast14&amp;quot;, &amp;quot;Blast15&amp;quot;, &#xA;&amp;quot;Blast16&amp;quot;, &amp;quot;Blast17&amp;quot;, &amp;quot;Blast18&amp;quot;, &amp;quot;Blast19&amp;quot;, &amp;quot;Blast20&amp;quot;, &amp;quot;Blast2&amp;quot;, &#xA;&amp;quot;Blast21&amp;quot;, &amp;quot;Blast22&amp;quot;, &amp;quot;Blast23&amp;quot;, &amp;quot;Blast3&amp;quot;, &amp;quot;Blast4&amp;quot;, &amp;quot;Blast5&amp;quot;, &#xA;&amp;quot;Blast6&amp;quot;, &amp;quot;Blast7&amp;quot;, &amp;quot;Blast8&amp;quot;, &amp;quot;Blast9&amp;quot;, &amp;quot;CMP1&amp;quot;, &amp;quot;CMP2&amp;quot;, &amp;quot;CMP3&amp;quot;, &#xA;&amp;quot;CMP4&amp;quot;, &amp;quot;CMP5&amp;quot;, &amp;quot;CMP6&amp;quot;, &amp;quot;CMP7&amp;quot;, &amp;quot;CMP8&amp;quot;, &amp;quot;GMP1&amp;quot;, &amp;quot;GMP2&amp;quot;, &amp;quot;GMP3&amp;quot;, &#xA;&amp;quot;GMP4&amp;quot;, &amp;quot;GMP5&amp;quot;, &amp;quot;GMP6&amp;quot;, &amp;quot;GMP7&amp;quot;, &amp;quot;HSC1&amp;quot;, &amp;quot;HSC2&amp;quot;, &amp;quot;HSC3&amp;quot;, &amp;quot;HSC4&amp;quot;, &#xA;&amp;quot;HSC5&amp;quot;, &amp;quot;HSC6&amp;quot;, &amp;quot;HSC7&amp;quot;, &amp;quot;LSC1&amp;quot;, &amp;quot;LSC2&amp;quot;, &amp;quot;LSC3&amp;quot;, &amp;quot;LSC4&amp;quot;, &amp;quot;LSC5&amp;quot;, &#xA;&amp;quot;LSC6&amp;quot;, &amp;quot;LSC7&amp;quot;, &amp;quot;LSC8&amp;quot;, &amp;quot;Mono1&amp;quot;, &amp;quot;Mono2&amp;quot;, &amp;quot;Mono3&amp;quot;, &amp;quot;Mono4&amp;quot;, &amp;quot;Mono5&amp;quot;, &#xA;&amp;quot;Mono6&amp;quot;), class = &amp;quot;data.frame&amp;quot;)&#xA;&#xA;dds &amp;lt;- DESeqDataSetFromMatrix(countData=df2, colData=coldata, design= ~ Group + Mutation)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the design I'm trying to incorporate. What i get is my design is nested in one of the column by looking at numerous example.&lt;/p&gt;&#xA;&lt;p&gt;What i would like to do is I'm trying to run a LRT test on this dataset which works fine if i give &amp;quot;&lt;code&gt;Group&lt;/code&gt;&amp;quot; as my design and then i run this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dds_lrt &amp;lt;- DESeq(dds, test=&amp;quot;LRT&amp;quot;, reduced = ~ 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which gives me diffrenitaly expressed across multiple groups. Now there are two question&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is it possible to include interaction such as in my case &lt;code&gt;Group+Mutation&lt;/code&gt; and do the lrt test?&lt;/li&gt;&#xA;&lt;li&gt;Or the above is conceptually wrong to do ?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="285" LastActivityDate="2020-12-19T00:37:42.063" Title="the model matrix is not full rank : This is a classic question which a biologist face without clear understanding of the model design" Tags="|r|deseq2|design|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="15085" PostTypeId="2" ParentId="15084" CreationDate="2020-12-18T11:24:47.697" Score="3" Body="&lt;pre&gt;&lt;code&gt;&amp;gt; table(doldat&lt;span class=&quot;math-container&quot;&gt;$Group, coldat$&lt;/span&gt;Mutation)&#xA;       &#xA;        DNMT3A-R882H FLT3-ITD IDH2-R140Q KRAS-G13D MAU2-Q98H No NPM1-L287ins(TCTG) NRAS-G13D STAG2-R614 TET2-E1357 TET2-R550&#xA;  HSC              0        0          0         0         0  7                  0         0          0          0         0&#xA;  Blast            2        4          2         1         1  0                  4         1          1          6         1&#xA;  CMP              0        0          0         0         0  8                  0         0          0          0         0&#xA;  GMP              0        0          0         0         0  7                  0         0          0          0         0&#xA;  LSC              2        3          0         0         1  0                  0         1          1          0         0&#xA;  Mono             0        0          0         0         0  6                  0         0          0          0         0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Look at this table, you see by eye how some mutations are nested with Group. If you want to include it then you will need to subset the experiment into multiple separate runs, depending on the question you want to answer.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-12-18T11:24:47.697" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15087" PostTypeId="1" CreationDate="2020-12-18T15:15:42.660" Score="1" ViewCount="100" Body="&lt;p&gt;I have 25 individuals' whole-genome sequence (collected from 5 different locations) and after doing a variant calling analysis, I found 2734 SNP+INDELS marker (SNPs + INDELS = 2734 ). Then I plotted the PCA plot (pc1 and pc2) using variant calling data, i.e. raw SNPs.&lt;/p&gt;&#xA;&lt;p&gt;Now I am trying to interpret the population structure.&lt;/p&gt;&#xA;&lt;p&gt;Here I can see that these individuals from different locations are genetically different and there is no difference between individuals from the same geographic foci.&lt;/p&gt;&#xA;&lt;p&gt;Are there any other things that I can report?&lt;/p&gt;&#xA;&lt;p&gt;EDIT: Each circle(except green) represents a cluster of 4 individuals (that overlap with each other) and the green dot represent one individual&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PCHzx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PCHzx.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then I calculate nucleotide diversity (π) values (across the whole genome) of each cluster observed in PCA plot: What is best way to show that information?&lt;/p&gt;&#xA;&lt;p&gt;The mean pi value of each cluster:&#xA;1 5.61 E-05&#xA;2 4.76E-0.5&#xA;3 0.00045&#xA;4 2.56e-05&#xA;5 4.69e-0.5&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there any test that should I perform to see if there are any significant differences, such as panmixia, migration or neutrality?&lt;/li&gt;&#xA;&lt;li&gt;Can someone explain to me how to interpret and report the finding from PCA and π?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="10096" LastEditorUserId="3848" LastEditDate="2022-04-29T23:46:13.293" LastActivityDate="2022-04-29T23:46:13.293" Title="PCA analysis and nucleotide diversity pi" Tags="|phylogenetics|pca|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15088" PostTypeId="1" AcceptedAnswerId="15100" CreationDate="2020-12-18T16:11:25.533" Score="0" ViewCount="853" Body="&lt;p&gt;Sorry if this question is obvious - I'm not familiar with proteomics so I'm not sure of the standard units.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2805705/&quot; rel=&quot;nofollow noreferrer&quot;&gt;In this paper&lt;/a&gt;, it talks about using the length of a protein to calculate the NSAF value.  However, it doesn't specify the units.  Would it be peptides, amino acids or nucleic acids? Or something else?&lt;/p&gt;&#xA;" OwnerUserId="10083" LastActivityDate="2020-12-20T14:20:57.067" Title="What unit is the length of a protein measured in?" Tags="|proteins|proteomics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15089" PostTypeId="1" CreationDate="2020-12-18T16:18:29.347" Score="1" ViewCount="51" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xprJn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xprJn.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;There is a scRNA dataset on Glioblastoma from &lt;a href=&quot;https://cf.10xgenomics.com/samples/cell-exp/4.0.0/Targeted_SC3v3_Human_Glioblastoma_Neuroscience/Targeted_SC3v3_Human_Glioblastoma_Neuroscience_web_summary.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;10xgenomics&lt;/a&gt;.&#xA;Unfortunately, there seems to be no research paper related to that data.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 0 :&lt;/strong&gt; May be there is paper  ? How to find it ? I am not familiar with 10xgenomics - may be there is some standard way to find papers related to their datasets ? May be their data also placed in GEO database, then what is GSE-id ?&lt;/p&gt;&#xA;&lt;p&gt;Any way, we can see several clusters in the data (see figure above or switch to tab 'analysis' at the link above).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 1:&lt;/strong&gt; What can be biological interpretation of these clusters ? I would expect cancer cells to be more homogeneous, but here we see several well-separated clusters, probably something like that has been already discussed in the literature ?&lt;/p&gt;&#xA;&lt;p&gt;PS&lt;/p&gt;&#xA;&lt;p&gt;That dataset has been placed on kaggle &lt;a href=&quot;https://www.kaggle.com/reminho/human-glioblastoma-dataset&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kaggle.com/reminho/human-glioblastoma-dataset&lt;/a&gt;&#xA;by enthusiast&#xA;and one can find several notebooks in R and Python to play with it. Clusters are also present there.&lt;/p&gt;&#xA;&lt;p&gt;In general there are many datasets/notebooks/people related to bioinformatics on kaggle, it is quite convenient platform for everything related to data science - data sharing, data processing on their servers for free (like colab), commenting on the works of others and so on.&#xA;It seems it would be useful to create bioinformatics &amp;quot;community&amp;quot; on kaggle (there are already 12 other &amp;quot;communities&amp;quot;) if you find that idea - useful please share your comments/make upvote/show yourself at:&#xA;&lt;a href=&quot;https://www.kaggle.com/general/203136&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.kaggle.com/general/203136&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7441" LastActivityDate="2020-12-18T16:18:29.347" Title="Clusters in scRNA of Glioblastoma" Tags="|scrnaseq|clustering|cancer|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15090" PostTypeId="1" CreationDate="2020-12-18T20:21:28.707" Score="1" ViewCount="418" Body="&lt;p&gt;Relate to finding soft power, I ran &amp;quot;pickSoftThreshold&amp;quot; function and the result of that(sft) is as follow:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;      &amp;quot;Power&amp;quot;   &amp;quot;SFT.R.sq&amp;quot;  &amp;quot;slope&amp;quot;   &amp;quot;truncated.R.sq&amp;quot;  &amp;quot;mean.k.&amp;quot;   &amp;quot;median.k.&amp;quot;     &amp;quot;max.k.&amp;quot;&#xA;&amp;quot;1&amp;quot;     1         0.30       0.92         0.89           2056.1      1903.5          4558.9&#xA;&amp;quot;2&amp;quot;     2         0.50      -1.52         0.91            548.1       445.5          1973.9&#xA;&amp;quot;3&amp;quot;     3         0.78      -2.05         0.96            188.3       128.3          1015.9&#xA;&amp;quot;4&amp;quot;     4         0.81      -2.21         0.97            77.21       42.3            594.9&#xA;&amp;quot;5&amp;quot;     5         0.82      -2.21         0.98             36.1       16.2            373.2&#xA;&amp;quot;6&amp;quot;     6         0.80      -2.18         0.96             18.8       6.9             245.8&#xA;&amp;quot;7&amp;quot;     7         0.78      -2.07         0.94             10.6       3.1             167.8&#xA;&amp;quot;8&amp;quot;     8         0.93      -1.75         0.99             6.4        1.5             121.7&#xA;&amp;quot;9&amp;quot;     9         0.94      -1.74         0.99             4.2        0.79            103.0&#xA;&amp;quot;10&amp;quot;    10        0.95      -1.69         0.99            2.87        0.42            88.2&#xA;&amp;quot;11&amp;quot;    12        0.97      -1.60         0.98            1.52        0.13            66.4&#xA;&amp;quot;12&amp;quot;    13        0.96      -1.56         0.98            1.17        0.07            58.7&#xA;&amp;quot;13&amp;quot;    14        0.96      -1.52         0.98            0.92        0.04            52.3&#xA;&amp;quot;14&amp;quot;    15        0.96      -1.48         0.98            0.75        0.02            46.9&#xA;&amp;quot;15&amp;quot;    16        0.95      -1.45         0.98            0.62        0.01            42.2&#xA;&amp;quot;16&amp;quot;    17        0.95      -1.41         0.98            0.53        0.01            38.1&#xA;&amp;quot;17&amp;quot;    18        0.95      -1.36         0.99            0.45        0.01            34.6&#xA;&amp;quot;18&amp;quot;    19        0.94      -1.33         0.99            0.40        0.00            31.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My network type is 'Signed Hybrid' and cor type is 'bicor' as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sft = pickSoftThreshold(datExpr, powerVector = powers, networkType = &amp;quot;signed hybrid&amp;quot;, corFnc=&amp;quot;bicor&amp;quot; ,corOptions = list(maxPOutliers =0.1), verbose = 5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;based on the sft result that I presented in this post and also mean connectivity &lt;a href=&quot;https://www.dropbox.com/s/kxmwt1n98ph37l2/Rplot_Scale%20independer_and_Mean_connectivity.pdf?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;plot&lt;/a&gt;, I selected power = 7 to construct a co-expression network for 5000 most connected genes and 58 colerectal cancer patients. Now, I have 5 modules as below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;green     brown    grey   turquoise    yellow &#xA; 550       345      3400       610        95 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you see, 3400 genes are in the &amp;quot;grey&amp;quot; module. My question is that are these genes just in the &amp;quot;grey&amp;quot; module is reasonable? if it's not reasonable, how can I modify my constructing process? do the selected power as 7 is correct?&lt;/p&gt;&#xA;&lt;p&gt;I really appreciated it if anybody share his/her comment with me.&lt;/p&gt;&#xA;" OwnerUserId="4915" LastEditorUserId="4915" LastEditDate="2020-12-18T21:10:12.607" LastActivityDate="2020-12-18T21:10:12.607" Title="How does decrease the size of &quot;grey&quot; module in WGCNA framework?" Tags="|wgcna|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15091" PostTypeId="1" CreationDate="2020-12-18T20:47:45.703" Score="0" ViewCount="45" Body="&lt;p&gt;I used the &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035531192-RNAseq-short-variant-discovery-SNPs-Indels-&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATK pipeline&lt;/a&gt; to call SNPs on males and females using RNA-Seq data. But the males have a higher read count (~43-46M reads) than the females (~40-42M reads). This causes SNP counts to be higher for males and lower for females. This is strange because I am interested in X-chromosome and hence the question &amp;quot;How to control for the number of reads when calling SNPs using RNA-Seq?&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="9212" LastActivityDate="2020-12-18T20:47:45.703" Title="How to control/normalize for number of reads when calling SNPs using RNA-Seq?" Tags="|rna-seq|snp|read-mapping|reads|gatk|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15092" PostTypeId="1" CreationDate="2020-12-18T23:01:59.453" Score="1" ViewCount="230" Body="&lt;p&gt;What is the biology behind SNP based PCA analysis to study population structure?  I am reading some articles where they used PCA analysis to compare isolates of drosophila that is collected from a different location. AND by PCA they found that SNP is different between all populations. What does that mean? (what it tell you about population structure/genetics?)&lt;/p&gt;&#xA;" OwnerUserId="10096" LastActivityDate="2020-12-18T23:25:00.890" Title="Biology behind PCA analysis based on SNP" Tags="|snp|pca|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15093" PostTypeId="2" ParentId="15092" CreationDate="2020-12-18T23:25:00.890" Score="1" Body="&lt;p&gt;First of all, PCA is a technique for dimension reduction. Basically, the goal is to compare tens of thousands of SNPs in Drosophila. Now if you only have 2 SNPs, you can plot them on a 2D scatter plot. If you have 3 SNPs, you may try a 3D plot. But now imagine you have 30,000 SNPs, but you CANNOT plot a 30000-dimensional plot. To visualize this high dimensional data, what we can do is to perform dimensional reduction like PCA. PCA tries to find a set of orthogonal coordinations that explains most of the variation in the data (if there no variation, there is no information contained in the data, which essentially means there is no data). The idea is that PC1 carries most variation can be explained, and PC2 carries the second most... For lower PCs like PC50 or PC60, they probably only carry noise in the data. Therefore, the higher PCs (PC1, PC2 and so on) effectively summarizes the useful information in the data. So you can visualize the &amp;quot;structure&amp;quot; of the data in a 2D PCA plot.&lt;/p&gt;&#xA;&lt;p&gt;By looking at the distance between points on a PCA plot, you can tell how similar the two data points are. But if you see two populations that are perfectly separated on PCA plot, it does not mean that the 2 population differ completely at every SNP, because PCA is a summarization of all SNP included.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2020-12-18T23:25:00.890" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15097" PostTypeId="1" CreationDate="2020-12-20T07:02:23.893" Score="1" ViewCount="788" Body="&lt;p&gt;I am plotting a barplot using &lt;code&gt;ggplot&lt;/code&gt; + &lt;code&gt;geom_bar()&lt;/code&gt; function in R.&lt;/p&gt;&#xA;&lt;p&gt;I have some the recoreds which share the same x-axis identifiers.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;geom_bar()&lt;/code&gt; did merge all values that shared the same x-axis identifiers.&lt;/p&gt;&#xA;&lt;p&gt;Like the following example.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(&amp;quot;ggplot2&amp;quot;)&#xA;&#xA;df &amp;lt;- data.frame(x = c(&amp;quot;a&amp;quot;,&amp;quot;a&amp;quot;,&amp;quot;b&amp;quot;,&amp;quot;c&amp;quot;,&amp;quot;d&amp;quot;,&amp;quot;e&amp;quot;),&#xA;                 y = c(5,5,5,5,5,5),&#xA;                 stringsAsFactors = FALSE)&#xA;&#xA;ggplot(df, aes(x, y)) + geom_bar(stat = &amp;quot;identity&amp;quot;)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The erroneous graph is displayed below.&lt;/p&gt;&#xA;&lt;p&gt;The highest bar means the sum of multiple y-values shared same x-axis names.&lt;/p&gt;&#xA;&lt;p&gt;Anyone else could help me to solve this problem? Thanks.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LnPrH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LnPrH.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8757" LastEditorUserId="8757" LastEditDate="2020-12-20T10:54:36.897" LastActivityDate="2020-12-21T14:47:06.927" Title="ggplot2 geom_bar() , do not merge y-values shared same x-ids" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="3" ClosedDate="2021-01-18T15:25:03.053" ContentLicense="CC BY-SA 4.0" />
  <row Id="15098" PostTypeId="1" AcceptedAnswerId="15151" CreationDate="2020-12-20T11:17:31.080" Score="0" ViewCount="964" Body="&lt;p&gt;Trying to learn a little bit of Python and Biopython I ended up figuring out what a Rest API is when the &lt;a href=&quot;https://data.rcsb.org/#data-api&quot; rel=&quot;nofollow noreferrer&quot;&gt;RCSB PDB repository&lt;/a&gt; released its new Rest API.&lt;/p&gt;&#xA;&lt;p&gt;I tried to retrieve info from the PDB repository using python3 requests,&lt;/p&gt;&#xA;&lt;p&gt;then I managed to use &lt;a href=&quot;https://openapi-generator.tech/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenAPI-generator&lt;/a&gt; to generate what they call a 'Python3 Client'.&lt;/p&gt;&#xA;&lt;p&gt;Now I am at loss I have this ApiClient class:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print dir(): &#xA;&#xA; ['NATIVE_TYPES_MAPPING', 'PRIMITIVE_TYPES', '_ApiClient__call_api',&#xA;  '_ApiClient__deserialize', '_ApiClient__deserialize_date',&#xA;  '_ApiClient__deserialize_datetime', '_ApiClient__deserialize_file',&#xA;  '_ApiClient__deserialize_model', '_ApiClient__deserialize_object',&#xA;  '_ApiClient__deserialize_primitive', '__class__', '__delattr__',&#xA;  '__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__',&#xA;  '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__',&#xA;  '__init__', '__init_subclass__', '__le__', '__lt__', '__module__',&#xA;  '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__',&#xA;  '__setattr__', '__sizeof__', '__str__', '__subclasshook__',&#xA;  '__weakref__', '_pool', 'call_api', 'client_side_validation',&#xA;  'close', 'configuration', 'cookie', 'default_headers',&#xA;  'deserialize', 'files_parameters', 'parameters_to_tuples',&#xA;  'pool', 'pool_threads', 'request', 'rest_client',&#xA;  'sanitize_for_serialization', 'select_header_accept',&#xA;  'select_header_content_type', 'set_default_header',&#xA;  'update_params_for_auth', 'user_agent']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and don't know how to use it. Googling for examples or searching for info on the OpenAPI-generator website doesn't seem to work for me, I find plenty of tutorials and examples on how to generate the client but none about how to use it.&lt;/p&gt;&#xA;&lt;p&gt;The generated client class should help to better interact with the server as I understood (I could be wrong here). I am asking for helps and hints here because the OpenAPI webpage under Community points to stackoverflow under [openapi-generator] tag. Any tutorial, easy documentation that could help me understand the output of the generator?&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="650" LastEditDate="2021-01-04T16:39:35.823" LastActivityDate="2021-01-04T18:31:10.880" Title="OpenAPI-Generator Python Client Instructions Examples" Tags="|python|public-databases|pdb|api|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15099" PostTypeId="1" CreationDate="2020-12-20T13:39:14.113" Score="0" ViewCount="237" Body="&lt;p&gt;I did some de novo genome assemblies with Illumina PE data using SPAdes, whereas most of them consisting of a large number of contigs(&amp;gt;1000). I have several questions below.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Do we need to filter short contigs?&lt;/li&gt;&#xA;&lt;li&gt;How to determine the appropriate length threshold？&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="8757" LastActivityDate="2020-12-20T13:39:14.113" Title="How to filter a genome assembly consistsing of a large number of contigs?" Tags="|genome|assembly|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15100" PostTypeId="2" ParentId="15088" CreationDate="2020-12-20T14:20:57.067" Score="1" Body="&lt;p&gt;As mentioned by others, it's the number of amino acids. Alternative measurements could be in Angstroms or similar, but that would require at least an approximate structure.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-12-20T14:20:57.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15101" PostTypeId="1" CreationDate="2020-12-20T19:09:25.257" Score="0" ViewCount="1001" Body="&lt;p&gt;I have  a data frame that contain lot of columns and lines, like this for example&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Column1   Column2 &#xA;5            T1&#xA;1            T2&#xA;1            T3&#xA;1            T1&#xA;2            T1&#xA;1            T1&#xA;2            T1&#xA;1            T1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to extract just lines that have the value 1 in column 1 and Value T1 in Column 2&lt;/p&gt;&#xA;&lt;p&gt;So the result I want is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Column1   Column2 &#xA;1            T1&#xA;1            T1&#xA;1            T1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can you tell me please how I can do that&lt;/p&gt;&#xA;" OwnerUserId="10039" LastEditorUserId="3694" LastEditDate="2022-11-27T09:14:03.803" LastActivityDate="2022-11-27T09:14:03.803" Title="How to select specific lines and specific columns" Tags="|r|statistics|database|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15102" PostTypeId="2" ParentId="15101" CreationDate="2020-12-20T20:18:32.877" Score="2" Body="&lt;p&gt;You can simply use subset function in R and use the code given below,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    new_dataframe &amp;lt;- &#xA;    subset(name_of_your_dataframe,Column1 == 1 &amp;amp;&amp;amp; &#xA;    Column2 ==  &amp;quot;T1&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Please keep in mind that R is case sensitive and you should be careful while using the variable names.&lt;/p&gt;&#xA;" OwnerUserId="1194" LastActivityDate="2020-12-20T20:18:32.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15103" PostTypeId="2" ParentId="15097" CreationDate="2020-12-21T14:47:06.927" Score="1" Body="&lt;p&gt;As you seem to have surmised, the graph is not erroneous, but correct. You have a total of 10 for &lt;code&gt;a&lt;/code&gt; and that's what it's showing. If you want it to ignore duplicate X-axis values then you need to tell ggplot which datapoints it should actually be using. &lt;code&gt;barplot()&lt;/code&gt; is assuming that your duplicate &lt;code&gt;a&lt;/code&gt; values are intentional, ggplot assumes they should be grouped. These are both reasonable assumptions and it's your responsibility to determine beforehand which is correct.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-12-21T14:47:06.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15105" PostTypeId="1" CreationDate="2020-12-22T13:46:38.923" Score="1" ViewCount="30" Body="&lt;p&gt;I am relatively new to programming and only have a basic knowledge base for R. I am keen on learning how to manipulate DNA methylation arrays for analysis related to Glioblastoma Multiforme. More specficially I will want to try and find patterns and trends associated with differences in survival linked to expression of mutations and how significant the relationship is with progression free and overall survival.&lt;/p&gt;&#xA;&lt;p&gt;Any help would be MASSIVELY appreciated. Thank you!&lt;/p&gt;&#xA;" OwnerUserId="10135" LastActivityDate="2020-12-22T13:46:38.923" Title="What is the best way to start for analysing DNA methylation TGCA datasets using R? I want to learn but I'm a novice" Tags="|r|genetics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15106" PostTypeId="2" ParentId="15080" CreationDate="2020-12-22T19:03:19.033" Score="0" Body="&lt;p&gt;I think this should help you : &lt;a href=&quot;https://biit.cs.ut.ee/gprofiler/convert&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biit.cs.ut.ee/gprofiler/convert&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6694" LastActivityDate="2020-12-22T19:03:19.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15107" PostTypeId="2" ParentId="15073" CreationDate="2020-12-22T22:58:11.073" Score="0" Body="&lt;p&gt;I asked that on Reddit. I didn't get an answer to all my questions but I got an answer for how to assemble the overlapping contigs.&lt;/p&gt;&#xA;&lt;p&gt;If I use Flye I can assemble these and get larger contigs, better NG50, and higher genome fraction.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UaqXM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UaqXM.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;strong&gt;s-aligner + Flye assembly of SRR8357422&lt;/strong&gt; Genome fraction:95,95% - Largest Alignment: 134.103 - NGA50: 134.103 - LGA50: 1&lt;/p&gt;&#xA;&lt;p&gt;Given that the data in the first assembly apart from being more useful itself in my opinion, is objectively more suitable to generate a better result in all senses with a simple additional step that takes a few seconds, my only possible conclusion is that it is overall a better result and therefore this sort of result will often be preferable.&lt;/p&gt;&#xA;&lt;p&gt;Of course, there are other aspects to consider, like error rate, that are not considered here as it wasn't part of the question (mainly because I think Quast is not adequate for calculating it). But BTW, mismatches per 100kbp are also lower.&lt;/p&gt;&#xA;&lt;p&gt;[EDIT] I want to add that this is not granted that in all cases Flye will generate a clearly better assembly.&lt;/p&gt;&#xA;" OwnerUserId="5190" LastEditorUserId="5190" LastEditDate="2020-12-23T18:53:12.557" LastActivityDate="2020-12-23T18:53:12.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15108" PostTypeId="2" ParentId="15075" CreationDate="2020-12-23T02:57:46.017" Score="2" Body="&lt;p&gt;I was able to find a solution that answers my problem finally.&lt;/p&gt;&#xA;&lt;p&gt;So, here are two different methods below, one which does exactly what I want while the other has to wait for all the previous processes to finish:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;First method&lt;/strong&gt;: waiting for previous process to finish using &lt;code&gt;.groupTuple()&lt;/code&gt; like what @Pallie mentionned. This method is simpler than the second method. The problem is that it has to wait for the previous processes to finish. I want my samples to be treated in parallel. If I missed something with the logic of the method, please feel free to edit.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-java prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env nextflow&#xA;&#xA;// my samples&#xA;samplesList = Channel.from([&amp;quot;sample1&amp;quot;, &amp;quot;sample2&amp;quot;])&#xA;&#xA;// the process that is supposed to split the sample by chromosomes&#xA;process foo {&#xA;      echo true&#xA;      cpus 1&#xA;      memory '1 MB'&#xA;      executor 'local'&#xA;&#xA;      input:&#xA;         val(x) from samplesList&#xA;      output:&#xA;         file(&amp;quot;*.vcf&amp;quot;) into foo_out&#xA;         val(x) into output_sample&#xA;      script:&#xA;      &amp;quot;&amp;quot;&amp;quot;&#xA;         echo 1 &amp;gt; &lt;span class=&quot;math-container&quot;&gt;${x}_chr1.vcf&#xA;         echo 2 &amp;gt; $&lt;/span&gt;{x}_chr2.vcf&#xA;      &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;//Combine the samples by chromosome and filter out only the samples that match the files output&#xA;foo_out.flatten().combine(output_sample).filter{ it[0].toString().contains(it[1].toString()) }.set{maybework}&#xA;&#xA;// the process that is supposed to treat each chromosome apart&#xA;process bar {&#xA;      echo true&#xA;      cpus 1&#xA;      memory '1 MB'&#xA;      executor 'local'&#xA;&#xA;      input:&#xA;         set file(y), val(x) from maybework&#xA;      output:&#xA;         set val(x), file(&amp;quot;*chr*vcf&amp;quot;) into bar_out&#xA;      script:&#xA;      &amp;quot;&amp;quot;&amp;quot;&#xA;         echo &lt;span class=&quot;math-container&quot;&gt;${x}&#xA;         filename=\\\$(basename -- $&lt;/span&gt;{y})&#xA;         chromosome=\\\$(echo &amp;quot;\\\${filename%.*}&amp;quot; | cut -f2 -d'_')&#xA;         cat &lt;span class=&quot;math-container&quot;&gt;${y} &amp;gt; $&lt;/span&gt;{x}_&amp;quot;\\\$chromosome&amp;quot;_&amp;quot;\\\$chromosome&amp;quot;.vcf&#xA;         if [ ${x} == &amp;quot;sample2&amp;quot; ]; then sleep 20; fi&#xA;      &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;// the process that merges back the chromosomes by sample&#xA;process mergefoobar {&#xA;      echo true&#xA;      cpus 1&#xA;      memory '1 MB'&#xA;      executor 'local'&#xA;&#xA;      input:&#xA;      set val(sample), file(x) from bar_out.groupTuple()&#xA;&#xA;      script:&#xA;      &amp;quot;&amp;quot;&amp;quot;&#xA;         echo &lt;span class=&quot;math-container&quot;&gt;$sample&#xA;         echo $&lt;/span&gt;{x}&#xA;      &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;will output:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bjnxO.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bjnxO.jpg&quot; alt=&quot;FirstMethod&quot; /&gt;&lt;/a&gt;&#xA;As you can see above, the process &lt;code&gt;mergefoobar&lt;/code&gt; is still waiting for all the processes of &lt;code&gt;bar&lt;/code&gt; to finish.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Second method&lt;/strong&gt;: does not wait for the previous processes to finish, uses &lt;code&gt;.join()&lt;/code&gt; for each chromosome... Yes, the downside is that you would have to write the &lt;code&gt;join()&lt;/code&gt; for all the chromosomes. Additionally, if you use &lt;code&gt;snpSift Split&lt;/code&gt;, one sample may generate 3 files out of 5 (because only 3 chromosomes had variants listed in them), you would need to create the other empty VCF files because the output excepts them.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-java prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env nextflow&#xA;&#xA;// my samples&#xA;samplesList = Channel.from([&amp;quot;sample1&amp;quot;, &amp;quot;sample2&amp;quot;])&#xA;&#xA;// the process that is supposed to split the sample by chromosomes&#xA;process foo {&#xA;      echo true&#xA;      cpus 1&#xA;      memory '1 MB'&#xA;      executor 'local'&#xA;&#xA;      input:&#xA;         val(x) from samplesList&#xA;      output:&#xA;         file(&amp;quot;*.vcf&amp;quot;) into foo_out&#xA;         val(x) into output_sample&#xA;      script:&#xA;      &amp;quot;&amp;quot;&amp;quot;&#xA;         echo 1 &amp;gt; &lt;span class=&quot;math-container&quot;&gt;${x}_chr1.vcf&#xA;     echo 2 &amp;gt; $&lt;/span&gt;{x}_chr2.vcf&#xA;         echo X &amp;gt; &lt;span class=&quot;math-container&quot;&gt;${x}_chrX.vcf&#xA;     echo Y &amp;gt; $&lt;/span&gt;{x}_chrY.vcf&#xA;      &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;//Combine the samples by chromosome and filter out only the samples that match the files output&#xA;foo_out.flatten().combine(output_sample).filter{ it[0].toString().contains(it[1].toString()) }.set{maybework}&#xA;&#xA;// the process that is supposed to treat each chromosome apart&#xA;process bar {&#xA;      echo true&#xA;      cpus 1&#xA;      memory '1 MB'&#xA;      executor 'local'&#xA;&#xA;      input:&#xA;         set file(y), val(x) from maybework&#xA;      output:&#xA;         set val(x), file(&amp;quot;*chr1*vcf&amp;quot;) optional true into bar_out1&#xA;         set val(x), file(&amp;quot;*chr2*vcf&amp;quot;) optional true into bar_out2&#xA;         set val(x), file(&amp;quot;*chrX*vcf&amp;quot;) optional true into bar_outX&#xA;         set val(x), file(&amp;quot;*chrY*vcf&amp;quot;) optional true into bar_outY&#xA;      script:&#xA;      &amp;quot;&amp;quot;&amp;quot;&#xA;         echo &lt;span class=&quot;math-container&quot;&gt;${x}&#xA;     filename=\\\$(basename -- $&lt;/span&gt;{y})&#xA;         chromosome=\\\$(echo &amp;quot;\\\${filename%.*}&amp;quot; | cut -f2 -d'_')&#xA;         cat &lt;span class=&quot;math-container&quot;&gt;${y} &amp;gt; $&lt;/span&gt;{x}_&amp;quot;\\\$chromosome&amp;quot;_&amp;quot;\\\$chromosome&amp;quot;.vcf&#xA;         if [ ${x} == &amp;quot;sample2&amp;quot; ]; then sleep 20; fi&#xA;      &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;// the process that merges back the chromosomes by sample&#xA;process mergefoobar {&#xA;      echo true&#xA;      cpus 1&#xA;      memory '1 MB'&#xA;      executor 'local'&#xA;&#xA;      input:&#xA;      set val(sample), file(x), file(y), file(i), file(t) from bar_out1.join(bar_out2).join(bar_outX).join(bar_outY)&#xA;&#xA;      script:&#xA;      &amp;quot;&amp;quot;&amp;quot;&#xA;         echo &lt;span class=&quot;math-container&quot;&gt;$sample&#xA;     echo $&lt;/span&gt;{x}&#xA;         echo &lt;span class=&quot;math-container&quot;&gt;${y}&#xA;     echo $&lt;/span&gt;{i}&#xA;         echo ${t}&#xA;      &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/z9R66.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/z9R66.jpg&quot; alt=&quot;SecondMethod&quot; /&gt;&lt;/a&gt;&#xA;Above, you see that the process mergefoobar has launched even if the previous bar has not finished yet.&lt;/p&gt;&#xA;&lt;p&gt;I hope this also helps anyone in the future who might have this issue.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="4413" LastEditDate="2023-07-12T08:14:32.040" LastActivityDate="2023-07-12T08:14:32.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15109" PostTypeId="1" CreationDate="2020-12-23T04:41:01.230" Score="1" ViewCount="75" Body="&lt;p&gt;I am trying to do RNA seq analysis and my goal is to filter gene counts less than 5 using custom made python scripts. The code chunk goes as follows&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/python&#xA;#Load relevant modules&#xA;import sys&#xA;    &#xA;#Read in the HTseq gene count output files for each of the 8 samples&#xA;inputlist = [sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4], sys.argv[5], sys.argv[6], sys.argv[7], sys.argv[8]]  &#xA;##Determine the genes with sufficient coverage&#xA;#Set up a variable to hold genes with sufficient coverage in any of the 8 samples&#xA;genelist = []&#xA;&#xA;#For each HTseq file&#xA;for i in range(len(inputlist)):&#xA;    #Open it&#xA;    inputfile = open(inputlist[i], 'r')&#xA;    #For each line&#xA;    for line in inputfile:&#xA;        #Determine the number of reads/transcripts for this gene&#xA;        splitline = line.strip().split('\t')&#xA;        #Does it pass the threshold?&#xA;        if int(splitline[1]) &amp;gt;= 5: &#xA;            #If so, add it to list&#xA;            if not splitline[0] in genelist:&#xA;                genelist.append(splitline[0])&#xA;                    &#xA;    #Close the file&#xA;    inputfile.close()&#xA;    &#xA;#Write out the list of sufficiently expressed genes&#xA;outputlist = open(sys.argv[9], 'w') &#xA;    &#xA;for gene in genelist:&#xA;    outputlist.write(gene + '\n')&#xA;        &#xA;outputlist.close()&#xA;##Filter each of the HTseq files for only sufficiently expressed genes across our samples&#xA;#For each HTseq file&#xA;    &#xA;for i in range(len(inputlist)):&#xA;    #Open it&#xA;    inputfile = open(inputlist[i], 'r')&#xA;    &#xA;#Open an output 'filtered' file&#xA;outputfile = open(sys.argv[i+10], 'w')&#xA;    &#xA;#For each line&#xA;for line in inputfile:&#xA;    #Determine the gene&#xA;    splitline = line.strip().split('\t')&#xA;        &#xA;    #Is it in our list?&#xA;    if splitline[0] in genelist:&#xA;        #If so, write out the original line&#xA;        outputfile.write(line)&#xA;            &#xA;#Close the files&#xA;inputfile.close()&#xA;outputfile.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;python filtercounts.py 'wth1count.txt' 'wth2count.txt' 'wth3count.txt' 'wth4count.txt' '99h1count.txt' '99h2count.txt' '99h3count.txt' '99h4count.txt' 'genelist.txt' 'wth1filtered.txt' 'wth2filtered.txt' 'wth3filtered.txt' 'wth4filtered.txt'  '99h1filtered.txt' '99h2filtered.txt' '99h3filtered.txt' '99h4filtered.txt'&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The program works fine but at the end of the code execution I am not getting the last two output files '99h3filtered.txt' '99h4filtered.txt'.&lt;/p&gt;&#xA;" OwnerUserId="9659" LastEditorUserId="77" LastEditDate="2020-12-23T16:24:23.763" LastActivityDate="2022-05-17T21:02:05.217" Title="How to count RNA sequencereads using custom made python scripts?" Tags="|genomics|rna|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15110" PostTypeId="2" ParentId="14886" CreationDate="2020-12-23T05:35:40.000" Score="1" Body="&lt;p&gt;The answer to your main doubt about &amp;quot;annotation vs. enrichment&amp;quot; is that you need both, but you already have one annotation.&lt;/p&gt;&#xA;&lt;p&gt;You can only perform a functional enrichment analysis after you have a functional annotation. In your question, you state that you have KEGG annotation for your targets. That is it, knowing which target &amp;quot;belongs&amp;quot; to each KEGG module is an example of functional annotation of your targets. Several tools can annotate genes, proteins, etc. Knowing which protein is related to each drug or disease is also a functional annotation.&lt;/p&gt;&#xA;&lt;p&gt;Now, knowing which targets are significantly important can be achieved by functional enrichment analysis.&lt;/p&gt;&#xA;&lt;p&gt;For the network, both information can be used together. For example, you could link all elements to their functional annotation and use the available enrichment score as a quantitative parameter of importance for calculating correlations.&lt;/p&gt;&#xA;" OwnerUserId="10137" LastActivityDate="2020-12-23T05:35:40.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15111" PostTypeId="1" CreationDate="2020-12-23T09:37:48.517" Score="1" ViewCount="161" Body="&lt;p&gt;&lt;strong&gt;Genome information：&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;~50M nt&lt;/li&gt;&#xA;&lt;li&gt;2300+ contigs&lt;/li&gt;&#xA;&lt;li&gt;No pre-trained parameters in &lt;a href=&quot;http://bioinf.uni-greifswald.de/augustus/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Augustus&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;There are several well annotated RefSeq genomes of this genus.&lt;/li&gt;&#xA;&lt;li&gt;There are several RNA-seq data of this genus, but not in this species.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;My current strategy:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;EST evidence:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I selected many RNA-seq data that include as many species as&#xA;possible, and de novo assembled these data using &lt;code&gt;Trinity&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Only the longest transcripts were saved.&lt;/li&gt;&#xA;&lt;li&gt;All transcript files were catenated into a &lt;code&gt;Total_transcript.fasta&lt;/code&gt; file, and &lt;code&gt;cd-hit-est&lt;/code&gt; was used to reduce data size with options &lt;code&gt;-c 0.8 -n 5 -s 0.8&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Protein evidence&lt;/p&gt;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;All fungal proteins from SwissProt was download and reduced using &lt;code&gt;cd-hit&lt;/code&gt; with options &lt;code&gt;-c 0.8 -n 5 -s 0.8&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Proteinsets from RefSeq genomes of this genus were download, catenated, reduced using same methods.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I planed to annotate genome using &lt;code&gt;Maker2&lt;/code&gt; with above Evidence.&lt;/p&gt;&#xA;&lt;p&gt;I have a doubt about my strategy: Maker2 calls Augustus to ab-initio annotation, but no pre-trained model. How should I do to solve this?&lt;/p&gt;&#xA;&lt;p&gt;Anyone else could share your valuable suggestions? Thanks anyway.&lt;/p&gt;&#xA;" OwnerUserId="8757" LastActivityDate="2020-12-23T17:52:08.557" Title="How to annotate optimally a fungal genome without RNA-seq evidence?" Tags="|sequence-annotation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15112" PostTypeId="1" CreationDate="2020-12-23T14:33:08.027" Score="0" ViewCount="79" Body="&lt;p&gt;I've recently downloaded the FastqCLeaner to process paired-end fastq files locally using R (4.0.3). I'm having trouble with the shiny app. It is not uploading my second fastq file, and is repeated asking for file 1. Can someone help me? is there another package i can use for trimming and alignment in R? I've tried GREP2 but had massive installation issues, i wasted 2 days trying to install in R.&lt;/p&gt;&#xA;" OwnerUserId="9234" LastActivityDate="2020-12-23T14:33:08.027" Title="FastqCleaner in R, not recognizing paired-end fastq file?" Tags="|r|trimming|fastqc|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15113" PostTypeId="2" ParentId="15109" CreationDate="2020-12-23T17:46:23.567" Score="1" Body="&lt;p&gt;What software are you using for the next step?  DESeq in R will do this filtering for you in one line.&lt;/p&gt;&#xA;&lt;p&gt;I think a more typical way to do this would be to make your entire gene count matrix, and then remove rows where the max value for that row is &amp;lt; 5.  Your way, you have to go through each file twice; once to know which genes to keep, and then again to actually remove lines.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2020-12-23T17:46:23.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15114" PostTypeId="2" ParentId="15111" CreationDate="2020-12-23T17:52:08.557" Score="1" Body="&lt;p&gt;The Maker documentation does include &lt;a href=&quot;http://gmod.org/wiki/MAKER_Tutorial#Re-annotation_Using_MAKER_Derived_GFF3&quot; rel=&quot;nofollow noreferrer&quot;&gt;some instructions for training &lt;em&gt;ab initio&lt;/em&gt; gene predictors&lt;/a&gt;, but it assumes an abundant EST database is available. (Assumptions about what kinds of sequences will be available for a draft genome assembly have changed drastically in the last decade.)&lt;/p&gt;&#xA;&lt;p&gt;It may be worth exploring an iterative approach in any case. You can do a first-pass annotation with Maker using the evidence you discussed, along with &lt;em&gt;ab initio&lt;/em&gt; predictions from Augustus (and I'd recommend SNAP as well). Each gene model will be scored by Maker with the &lt;em&gt;annotation edit distance (AED)&lt;/em&gt;, representing how well the gene model agrees with the evidence (0.0 is best). From this first pass, if you can identify a few hundred reliable gene models with good AED scores (or maybe even as many as one or two thousand), that should be plenty of training data. You could use these gene models to train a new model for Augustus, using Maker's instructions or Augustus' own instruction. (In my experience, it was quite a bit of work in either case.)&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2020-12-23T17:52:08.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15115" PostTypeId="1" AcceptedAnswerId="15117" CreationDate="2020-12-24T13:10:46.953" Score="1" ViewCount="366" Body="&lt;p&gt;I have a data frame where the row names are the species names, and each column can either be an Amino-Acid character or &amp;quot;-&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I wish to write it in a &lt;a href=&quot;https://en.wikipedia.org/wiki/FASTA_format&quot; rel=&quot;nofollow noreferrer&quot;&gt;fasta format&lt;/a&gt;. Simple example of the format is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;UniRef90_A0A1V4QIJ3&#xA;------------------------------------------------------------&#xA;--------------------------MDISTFSP-----PVAG-----------VE----&#xA;------ITAF-------------V------------------------------------&#xA;LN--------G-------------------------------------------------&#xA;----EKIPA-----------------------------------------IVLSKD----&#xA;---LNPFL------------------Q-ELLDQEPAC-----------------------&#xA;------------EHDIGTCYG------------------------DA-------------&#xA;--------------------------------------------------PC-----IS-&#xA;-----------------------GPDAWNHSGFQC----Q---------VC---------&#xA;------------GLVKSWNL-------------------VGDFVVYRQE-----------&#xA;&amp;gt;UniRef90_A0A0L9V5F8&#xA;------------------------------------------------------------&#xA;RFHTLFRNEYGHLRVLQRFDQRSKQIQNLENYRLVEFKSKPNTLLLPHHADADFLLVVLN&#xA;GRALLTLVNP-------------D---------------------------------G--&#xA;RDSYILEQGHA-------------------------------------------------&#xA;----QKIPAGTIFFLVNPNDNENLRI------------------------IKIATPINNP&#xA;HRFQDFFL-------------SSTEAQQSYLQG--FS-----------------------&#xA;----------------KNVLEASFDSEFKEINRVLFGEEGQQQQGEESQQEGVIVELERE&#xA;QIRELIKHA-----------------------------------------KSSSRRSLS-&#xA;-----------------------SQ-----DEPFNLRNRKP--------IY--SN-----&#xA;------------KF-GRWYEITPEKNP------------Q--------------------&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I can do it using the following for loop:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for (n in row.names(df)){&#xA;    ss = do.call(paste, c(as.list(mu_df[1, ]), sep = &amp;quot;&amp;quot;))&#xA;    write.fasta(ss,&#xA;                  names=n,&#xA;                  as.string=TRUE,&#xA;                  open=&amp;quot;a&amp;quot;,&#xA;                  new_file_path)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there a faster way of writing it? I have to write millions of those files and I wish to optimize it.&lt;/p&gt;&#xA;&lt;p&gt;In order to create the data frame:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(V1 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V2 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V3 = c(&amp;quot;-&amp;quot;, &#xA;&amp;quot;-&amp;quot;), V4 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V5 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V6 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V7 = c(&amp;quot;-&amp;quot;, &#xA;&amp;quot;-&amp;quot;), V8 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V9 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V10 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), &#xA;    V11 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V12 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V13 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), &#xA;    V14 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V15 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V16 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), &#xA;    V17 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V18 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V19 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), &#xA;    V20 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V21 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V22 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), &#xA;    V23 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V24 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V25 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), &#xA;    V26 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V27 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V28 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), &#xA;    V29 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V30 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V31 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), &#xA;    V32 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V33 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V34 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), &#xA;    V35 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V36 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V37 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), &#xA;    V38 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V39 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V40 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), &#xA;    V41 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V42 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V43 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), &#xA;    V44 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V45 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V46 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), &#xA;    V47 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V48 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V49 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), &#xA;    V50 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V51 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V52 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), &#xA;    V53 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V54 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V55 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), &#xA;    V56 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V57 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V58 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), &#xA;    V59 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V60 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V61 = c(&amp;quot;-&amp;quot;, &amp;quot;R&amp;quot;), &#xA;    V62 = c(&amp;quot;-&amp;quot;, &amp;quot;F&amp;quot;), V63 = c(&amp;quot;-&amp;quot;, &amp;quot;H&amp;quot;), V64 = c(&amp;quot;-&amp;quot;, &amp;quot;T&amp;quot;), &#xA;    V65 = c(&amp;quot;-&amp;quot;, &amp;quot;L&amp;quot;), V66 = c(&amp;quot;-&amp;quot;, &amp;quot;F&amp;quot;), V67 = c(&amp;quot;-&amp;quot;, &amp;quot;R&amp;quot;), &#xA;    V68 = c(&amp;quot;-&amp;quot;, &amp;quot;N&amp;quot;), V69 = c(&amp;quot;-&amp;quot;, &amp;quot;E&amp;quot;), V70 = c(&amp;quot;-&amp;quot;, &amp;quot;Y&amp;quot;), &#xA;    V71 = c(&amp;quot;-&amp;quot;, &amp;quot;G&amp;quot;), V72 = c(&amp;quot;-&amp;quot;, &amp;quot;H&amp;quot;), V73 = c(&amp;quot;-&amp;quot;, &amp;quot;L&amp;quot;), &#xA;    V74 = c(&amp;quot;-&amp;quot;, &amp;quot;R&amp;quot;), V75 = c(&amp;quot;-&amp;quot;, &amp;quot;V&amp;quot;), V76 = c(&amp;quot;-&amp;quot;, &amp;quot;L&amp;quot;), &#xA;    V77 = c(&amp;quot;-&amp;quot;, &amp;quot;Q&amp;quot;), V78 = c(&amp;quot;-&amp;quot;, &amp;quot;R&amp;quot;), V79 = c(&amp;quot;-&amp;quot;, &amp;quot;F&amp;quot;), &#xA;    V80 = c(&amp;quot;-&amp;quot;, &amp;quot;D&amp;quot;), V81 = c(&amp;quot;-&amp;quot;, &amp;quot;Q&amp;quot;), V82 = c(&amp;quot;-&amp;quot;, &amp;quot;R&amp;quot;), &#xA;    V83 = c(&amp;quot;-&amp;quot;, &amp;quot;S&amp;quot;), V84 = c(&amp;quot;-&amp;quot;, &amp;quot;K&amp;quot;), V85 = c(&amp;quot;-&amp;quot;, &amp;quot;Q&amp;quot;), &#xA;    V86 = c(&amp;quot;-&amp;quot;, &amp;quot;I&amp;quot;), V87 = c(&amp;quot;M&amp;quot;, &amp;quot;Q&amp;quot;), V88 = c(&amp;quot;D&amp;quot;, &amp;quot;N&amp;quot;), &#xA;    V89 = c(&amp;quot;I&amp;quot;, &amp;quot;L&amp;quot;), V90 = c(&amp;quot;S&amp;quot;, &amp;quot;E&amp;quot;), V91 = c(&amp;quot;T&amp;quot;, &amp;quot;N&amp;quot;), &#xA;    V92 = c(&amp;quot;F&amp;quot;, &amp;quot;Y&amp;quot;), V93 = c(&amp;quot;S&amp;quot;, &amp;quot;R&amp;quot;), V94 = c(&amp;quot;P&amp;quot;, &amp;quot;L&amp;quot;), &#xA;    V95 = c(&amp;quot;-&amp;quot;, &amp;quot;V&amp;quot;), V96 = c(&amp;quot;-&amp;quot;, &amp;quot;E&amp;quot;), V97 = c(&amp;quot;-&amp;quot;, &amp;quot;F&amp;quot;), &#xA;    V98 = c(&amp;quot;-&amp;quot;, &amp;quot;K&amp;quot;), V99 = c(&amp;quot;-&amp;quot;, &amp;quot;S&amp;quot;), V100 = c(&amp;quot;P&amp;quot;, &amp;quot;K&amp;quot;), &#xA;    V101 = c(&amp;quot;V&amp;quot;, &amp;quot;P&amp;quot;), V102 = c(&amp;quot;A&amp;quot;, &amp;quot;N&amp;quot;), V103 = c(&amp;quot;G&amp;quot;, &amp;quot;T&amp;quot;&#xA;    ), V104 = c(&amp;quot;-&amp;quot;, &amp;quot;L&amp;quot;), V105 = c(&amp;quot;-&amp;quot;, &amp;quot;L&amp;quot;), V106 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;L&amp;quot;), V107 = c(&amp;quot;-&amp;quot;, &amp;quot;P&amp;quot;), V108 = c(&amp;quot;-&amp;quot;, &amp;quot;H&amp;quot;), V109 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;H&amp;quot;), V110 = c(&amp;quot;-&amp;quot;, &amp;quot;A&amp;quot;), V111 = c(&amp;quot;-&amp;quot;, &amp;quot;D&amp;quot;), V112 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;A&amp;quot;), V113 = c(&amp;quot;-&amp;quot;, &amp;quot;D&amp;quot;), V114 = c(&amp;quot;-&amp;quot;, &amp;quot;F&amp;quot;), V115 = c(&amp;quot;V&amp;quot;, &#xA;    &amp;quot;L&amp;quot;), V116 = c(&amp;quot;E&amp;quot;, &amp;quot;L&amp;quot;), V117 = c(&amp;quot;-&amp;quot;, &amp;quot;V&amp;quot;), V118 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;V&amp;quot;), V119 = c(&amp;quot;-&amp;quot;, &amp;quot;L&amp;quot;), V120 = c(&amp;quot;-&amp;quot;, &amp;quot;N&amp;quot;), V121 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;G&amp;quot;), V122 = c(&amp;quot;-&amp;quot;, &amp;quot;R&amp;quot;), V123 = c(&amp;quot;-&amp;quot;, &amp;quot;A&amp;quot;), V124 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;L&amp;quot;), V125 = c(&amp;quot;-&amp;quot;, &amp;quot;L&amp;quot;), V126 = c(&amp;quot;-&amp;quot;, &amp;quot;T&amp;quot;), V127 = c(&amp;quot;I&amp;quot;, &#xA;    &amp;quot;L&amp;quot;), V128 = c(&amp;quot;T&amp;quot;, &amp;quot;V&amp;quot;), V129 = c(&amp;quot;A&amp;quot;, &amp;quot;N&amp;quot;), V130 = c(&amp;quot;F&amp;quot;, &#xA;    &amp;quot;P&amp;quot;), V131 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V132 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V133 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V134 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V135 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V136 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V137 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V138 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V139 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V140 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V141 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V142 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V143 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V144 = c(&amp;quot;V&amp;quot;, &amp;quot;D&amp;quot;), V145 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V146 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V147 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V148 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V149 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V150 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V151 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V152 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V153 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V154 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V155 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V156 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V157 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V158 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V159 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V160 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V161 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V162 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V163 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V164 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V165 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V166 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V167 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V168 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V169 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V170 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V171 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V172 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V173 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V174 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V175 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V176 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V177 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V178 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;G&amp;quot;), V179 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V180 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V181 = c(&amp;quot;L&amp;quot;, &#xA;    &amp;quot;R&amp;quot;), V182 = c(&amp;quot;N&amp;quot;, &amp;quot;D&amp;quot;), V183 = c(&amp;quot;-&amp;quot;, &amp;quot;S&amp;quot;), V184 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;Y&amp;quot;), V185 = c(&amp;quot;-&amp;quot;, &amp;quot;I&amp;quot;), V186 = c(&amp;quot;-&amp;quot;, &amp;quot;L&amp;quot;), V187 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;E&amp;quot;), V188 = c(&amp;quot;-&amp;quot;, &amp;quot;Q&amp;quot;), V189 = c(&amp;quot;-&amp;quot;, &amp;quot;G&amp;quot;), V190 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;H&amp;quot;), V191 = c(&amp;quot;G&amp;quot;, &amp;quot;A&amp;quot;), V192 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V193 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V194 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V195 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V196 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V197 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V198 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V199 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V200 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V201 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V202 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V203 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V204 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V205 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V206 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V207 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V208 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V209 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V210 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V211 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V212 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V213 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V214 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V215 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V216 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V217 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V218 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V219 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V220 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V221 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V222 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V223 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V224 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V225 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V226 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V227 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V228 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V229 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V230 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V231 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V232 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V233 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V234 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V235 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V236 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V237 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V238 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V239 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V240 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V241 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V242 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V243 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V244 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V245 = c(&amp;quot;E&amp;quot;, &amp;quot;Q&amp;quot;), V246 = c(&amp;quot;K&amp;quot;, &amp;quot;K&amp;quot;), V247 = c(&amp;quot;I&amp;quot;, &#xA;    &amp;quot;I&amp;quot;), V248 = c(&amp;quot;P&amp;quot;, &amp;quot;P&amp;quot;), V249 = c(&amp;quot;A&amp;quot;, &amp;quot;A&amp;quot;), V250 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;G&amp;quot;), V251 = c(&amp;quot;-&amp;quot;, &amp;quot;T&amp;quot;), V252 = c(&amp;quot;-&amp;quot;, &amp;quot;I&amp;quot;), V253 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;F&amp;quot;), V254 = c(&amp;quot;-&amp;quot;, &amp;quot;F&amp;quot;), V255 = c(&amp;quot;-&amp;quot;, &amp;quot;L&amp;quot;), V256 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;V&amp;quot;), V257 = c(&amp;quot;-&amp;quot;, &amp;quot;N&amp;quot;), V258 = c(&amp;quot;-&amp;quot;, &amp;quot;P&amp;quot;), V259 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;N&amp;quot;), V260 = c(&amp;quot;-&amp;quot;, &amp;quot;D&amp;quot;), V261 = c(&amp;quot;-&amp;quot;, &amp;quot;N&amp;quot;), V262 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;E&amp;quot;), V263 = c(&amp;quot;-&amp;quot;, &amp;quot;N&amp;quot;), V264 = c(&amp;quot;-&amp;quot;, &amp;quot;L&amp;quot;), V265 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;R&amp;quot;), V266 = c(&amp;quot;-&amp;quot;, &amp;quot;I&amp;quot;), V267 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V268 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V269 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V270 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V271 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V272 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V273 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V274 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V275 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V276 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V277 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V278 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V279 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V280 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V281 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V282 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V283 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V284 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V285 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V286 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V287 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V288 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V289 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V290 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V291 = c(&amp;quot;I&amp;quot;, &amp;quot;I&amp;quot;), V292 = c(&amp;quot;V&amp;quot;, &#xA;    &amp;quot;K&amp;quot;), V293 = c(&amp;quot;L&amp;quot;, &amp;quot;I&amp;quot;), V294 = c(&amp;quot;S&amp;quot;, &amp;quot;A&amp;quot;), V295 = c(&amp;quot;K&amp;quot;, &#xA;    &amp;quot;T&amp;quot;), V296 = c(&amp;quot;D&amp;quot;, &amp;quot;P&amp;quot;), V297 = c(&amp;quot;-&amp;quot;, &amp;quot;I&amp;quot;), V298 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;N&amp;quot;), V299 = c(&amp;quot;-&amp;quot;, &amp;quot;N&amp;quot;), V300 = c(&amp;quot;-&amp;quot;, &amp;quot;P&amp;quot;), V301 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;H&amp;quot;), V302 = c(&amp;quot;-&amp;quot;, &amp;quot;R&amp;quot;), V303 = c(&amp;quot;-&amp;quot;, &amp;quot;F&amp;quot;), V304 = c(&amp;quot;L&amp;quot;, &#xA;    &amp;quot;Q&amp;quot;), V305 = c(&amp;quot;N&amp;quot;, &amp;quot;D&amp;quot;), V306 = c(&amp;quot;P&amp;quot;, &amp;quot;F&amp;quot;), V307 = c(&amp;quot;F&amp;quot;, &#xA;    &amp;quot;F&amp;quot;), V308 = c(&amp;quot;L&amp;quot;, &amp;quot;L&amp;quot;), V309 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V310 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V311 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V312 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V313 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V314 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V315 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V316 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V317 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V318 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V319 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V320 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V321 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V322 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;S&amp;quot;), V323 = c(&amp;quot;-&amp;quot;, &amp;quot;S&amp;quot;), V324 = c(&amp;quot;-&amp;quot;, &amp;quot;T&amp;quot;), V325 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;E&amp;quot;), V326 = c(&amp;quot;-&amp;quot;, &amp;quot;A&amp;quot;), V327 = c(&amp;quot;Q&amp;quot;, &amp;quot;Q&amp;quot;), V328 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;Q&amp;quot;), V329 = c(&amp;quot;E&amp;quot;, &amp;quot;S&amp;quot;), V330 = c(&amp;quot;L&amp;quot;, &amp;quot;Y&amp;quot;), V331 = c(&amp;quot;L&amp;quot;, &#xA;    &amp;quot;L&amp;quot;), V332 = c(&amp;quot;D&amp;quot;, &amp;quot;Q&amp;quot;), V333 = c(&amp;quot;Q&amp;quot;, &amp;quot;G&amp;quot;), V334 = c(&amp;quot;E&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V335 = c(&amp;quot;P&amp;quot;, &amp;quot;-&amp;quot;), V336 = c(&amp;quot;A&amp;quot;, &amp;quot;F&amp;quot;), V337 = c(&amp;quot;C&amp;quot;, &#xA;    &amp;quot;S&amp;quot;), V338 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V339 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V340 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V341 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V342 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V343 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V344 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V345 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V346 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V347 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V348 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V349 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V350 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V351 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V352 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V353 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V354 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V355 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V356 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V357 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V358 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V359 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V360 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V361 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V362 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V363 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V364 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V365 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V366 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V367 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V368 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V369 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V370 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V371 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V372 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V373 = c(&amp;quot;E&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V374 = c(&amp;quot;H&amp;quot;, &amp;quot;-&amp;quot;), V375 = c(&amp;quot;D&amp;quot;, &amp;quot;-&amp;quot;), V376 = c(&amp;quot;I&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V377 = c(&amp;quot;G&amp;quot;, &amp;quot;K&amp;quot;), V378 = c(&amp;quot;T&amp;quot;, &amp;quot;N&amp;quot;), V379 = c(&amp;quot;C&amp;quot;, &#xA;    &amp;quot;V&amp;quot;), V380 = c(&amp;quot;Y&amp;quot;, &amp;quot;L&amp;quot;), V381 = c(&amp;quot;G&amp;quot;, &amp;quot;E&amp;quot;), V382 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;A&amp;quot;), V383 = c(&amp;quot;-&amp;quot;, &amp;quot;S&amp;quot;), V384 = c(&amp;quot;-&amp;quot;, &amp;quot;F&amp;quot;), V385 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;D&amp;quot;), V386 = c(&amp;quot;-&amp;quot;, &amp;quot;S&amp;quot;), V387 = c(&amp;quot;-&amp;quot;, &amp;quot;E&amp;quot;), V388 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;F&amp;quot;), V389 = c(&amp;quot;-&amp;quot;, &amp;quot;K&amp;quot;), V390 = c(&amp;quot;-&amp;quot;, &amp;quot;E&amp;quot;), V391 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;I&amp;quot;), V392 = c(&amp;quot;-&amp;quot;, &amp;quot;N&amp;quot;), V393 = c(&amp;quot;-&amp;quot;, &amp;quot;R&amp;quot;), V394 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;V&amp;quot;), V395 = c(&amp;quot;-&amp;quot;, &amp;quot;L&amp;quot;), V396 = c(&amp;quot;-&amp;quot;, &amp;quot;F&amp;quot;), V397 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;G&amp;quot;), V398 = c(&amp;quot;-&amp;quot;, &amp;quot;E&amp;quot;), V399 = c(&amp;quot;-&amp;quot;, &amp;quot;E&amp;quot;), V400 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;G&amp;quot;), V401 = c(&amp;quot;-&amp;quot;, &amp;quot;Q&amp;quot;), V402 = c(&amp;quot;-&amp;quot;, &amp;quot;Q&amp;quot;), V403 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;Q&amp;quot;), V404 = c(&amp;quot;-&amp;quot;, &amp;quot;Q&amp;quot;), V405 = c(&amp;quot;-&amp;quot;, &amp;quot;G&amp;quot;), V406 = c(&amp;quot;D&amp;quot;, &#xA;    &amp;quot;E&amp;quot;), V407 = c(&amp;quot;A&amp;quot;, &amp;quot;E&amp;quot;), V408 = c(&amp;quot;-&amp;quot;, &amp;quot;S&amp;quot;), V409 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;Q&amp;quot;), V410 = c(&amp;quot;-&amp;quot;, &amp;quot;Q&amp;quot;), V411 = c(&amp;quot;-&amp;quot;, &amp;quot;E&amp;quot;), V412 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;G&amp;quot;), V413 = c(&amp;quot;-&amp;quot;, &amp;quot;V&amp;quot;), V414 = c(&amp;quot;-&amp;quot;, &amp;quot;I&amp;quot;), V415 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;V&amp;quot;), V416 = c(&amp;quot;-&amp;quot;, &amp;quot;E&amp;quot;), V417 = c(&amp;quot;-&amp;quot;, &amp;quot;L&amp;quot;), V418 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;E&amp;quot;), V419 = c(&amp;quot;-&amp;quot;, &amp;quot;R&amp;quot;), V420 = c(&amp;quot;-&amp;quot;, &amp;quot;E&amp;quot;), V421 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;Q&amp;quot;), V422 = c(&amp;quot;-&amp;quot;, &amp;quot;I&amp;quot;), V423 = c(&amp;quot;-&amp;quot;, &amp;quot;R&amp;quot;), V424 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;E&amp;quot;), V425 = c(&amp;quot;-&amp;quot;, &amp;quot;L&amp;quot;), V426 = c(&amp;quot;-&amp;quot;, &amp;quot;I&amp;quot;), V427 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;K&amp;quot;), V428 = c(&amp;quot;-&amp;quot;, &amp;quot;H&amp;quot;), V429 = c(&amp;quot;-&amp;quot;, &amp;quot;A&amp;quot;), V430 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V431 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V432 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V433 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V434 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V435 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V436 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V437 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V438 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V439 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V440 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V441 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V442 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V443 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V444 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V445 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V446 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V447 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V448 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V449 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V450 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V451 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V452 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V453 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V454 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V455 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V456 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V457 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V458 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V459 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V460 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V461 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V462 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V463 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V464 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V465 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V466 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V467 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V468 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V469 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V470 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V471 = c(&amp;quot;P&amp;quot;, &amp;quot;K&amp;quot;), V472 = c(&amp;quot;C&amp;quot;, &#xA;    &amp;quot;S&amp;quot;), V473 = c(&amp;quot;-&amp;quot;, &amp;quot;S&amp;quot;), V474 = c(&amp;quot;-&amp;quot;, &amp;quot;S&amp;quot;), V475 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;R&amp;quot;), V476 = c(&amp;quot;-&amp;quot;, &amp;quot;R&amp;quot;), V477 = c(&amp;quot;-&amp;quot;, &amp;quot;S&amp;quot;), V478 = c(&amp;quot;I&amp;quot;, &#xA;    &amp;quot;L&amp;quot;), V479 = c(&amp;quot;S&amp;quot;, &amp;quot;S&amp;quot;), V480 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V481 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V482 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V483 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V484 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V485 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V486 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V487 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V488 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V489 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V490 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V491 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V492 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V493 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V494 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V495 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V496 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V497 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V498 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V499 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V500 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V501 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V502 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V503 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V504 = c(&amp;quot;G&amp;quot;, &amp;quot;S&amp;quot;), V505 = c(&amp;quot;P&amp;quot;, &#xA;    &amp;quot;Q&amp;quot;), V506 = c(&amp;quot;D&amp;quot;, &amp;quot;-&amp;quot;), V507 = c(&amp;quot;A&amp;quot;, &amp;quot;-&amp;quot;), V508 = c(&amp;quot;W&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V509 = c(&amp;quot;N&amp;quot;, &amp;quot;-&amp;quot;), V510 = c(&amp;quot;H&amp;quot;, &amp;quot;-&amp;quot;), V511 = c(&amp;quot;S&amp;quot;, &#xA;    &amp;quot;D&amp;quot;), V512 = c(&amp;quot;G&amp;quot;, &amp;quot;E&amp;quot;), V513 = c(&amp;quot;F&amp;quot;, &amp;quot;P&amp;quot;), V514 = c(&amp;quot;Q&amp;quot;, &#xA;    &amp;quot;F&amp;quot;), V515 = c(&amp;quot;C&amp;quot;, &amp;quot;N&amp;quot;), V516 = c(&amp;quot;-&amp;quot;, &amp;quot;L&amp;quot;), V517 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;R&amp;quot;), V518 = c(&amp;quot;-&amp;quot;, &amp;quot;N&amp;quot;), V519 = c(&amp;quot;-&amp;quot;, &amp;quot;R&amp;quot;), V520 = c(&amp;quot;Q&amp;quot;, &#xA;    &amp;quot;K&amp;quot;), V521 = c(&amp;quot;-&amp;quot;, &amp;quot;P&amp;quot;), V522 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V523 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V524 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V525 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V526 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V527 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V528 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V529 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V530 = c(&amp;quot;V&amp;quot;, &amp;quot;I&amp;quot;), V531 = c(&amp;quot;C&amp;quot;, &amp;quot;Y&amp;quot;), V532 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V533 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V534 = c(&amp;quot;-&amp;quot;, &amp;quot;S&amp;quot;), V535 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;N&amp;quot;), V536 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V537 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V538 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V539 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V540 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V541 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V542 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V543 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V544 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V545 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V546 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V547 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V548 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V549 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V550 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V551 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V552 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V553 = c(&amp;quot;G&amp;quot;, &#xA;    &amp;quot;K&amp;quot;), V554 = c(&amp;quot;L&amp;quot;, &amp;quot;F&amp;quot;), V555 = c(&amp;quot;V&amp;quot;, &amp;quot;-&amp;quot;), V556 = c(&amp;quot;K&amp;quot;, &#xA;    &amp;quot;G&amp;quot;), V557 = c(&amp;quot;S&amp;quot;, &amp;quot;R&amp;quot;), V558 = c(&amp;quot;W&amp;quot;, &amp;quot;W&amp;quot;), V559 = c(&amp;quot;N&amp;quot;, &#xA;    &amp;quot;Y&amp;quot;), V560 = c(&amp;quot;L&amp;quot;, &amp;quot;E&amp;quot;), V561 = c(&amp;quot;-&amp;quot;, &amp;quot;I&amp;quot;), V562 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;T&amp;quot;), V563 = c(&amp;quot;-&amp;quot;, &amp;quot;P&amp;quot;), V564 = c(&amp;quot;-&amp;quot;, &amp;quot;E&amp;quot;), V565 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;K&amp;quot;), V566 = c(&amp;quot;-&amp;quot;, &amp;quot;N&amp;quot;), V567 = c(&amp;quot;-&amp;quot;, &amp;quot;P&amp;quot;), V568 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V569 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V570 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V571 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V572 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V573 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V574 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V575 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V576 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V577 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V578 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V579 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V580 = c(&amp;quot;V&amp;quot;, &#xA;    &amp;quot;Q&amp;quot;), V581 = c(&amp;quot;G&amp;quot;, &amp;quot;-&amp;quot;), V582 = c(&amp;quot;D&amp;quot;, &amp;quot;-&amp;quot;), V583 = c(&amp;quot;F&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V584 = c(&amp;quot;V&amp;quot;, &amp;quot;-&amp;quot;), V585 = c(&amp;quot;V&amp;quot;, &amp;quot;-&amp;quot;), V586 = c(&amp;quot;Y&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V587 = c(&amp;quot;R&amp;quot;, &amp;quot;-&amp;quot;), V588 = c(&amp;quot;Q&amp;quot;, &amp;quot;-&amp;quot;), V589 = c(&amp;quot;E&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V590 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V591 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V592 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V593 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V594 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V595 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V596 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V597 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V598 = c(&amp;quot;-&amp;quot;, &#xA;    &amp;quot;-&amp;quot;), V599 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;), V600 = c(&amp;quot;-&amp;quot;, &amp;quot;-&amp;quot;)), class = &amp;quot;data.frame&amp;quot;, row.names = c(&amp;quot;UniRef90_A0A1V4QIJ3&amp;quot;, &#xA;&amp;quot;UniRef90_A0A0L9V5F8&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5003" LastActivityDate="2020-12-27T00:03:54.753" Title="How to write multi-sequence alignment data frame as a fasta format fast" Tags="|r|fasta|dataframe|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15117" PostTypeId="2" ParentId="15115" CreationDate="2020-12-24T19:01:40.597" Score="3" Body="&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It seems that part of the desired behavior is to write the sequences in the alignment &lt;em&gt;separately&lt;/em&gt;. I am not sure that I understand why it is necessary to write them separately. Generally it is inadvisable to write many small files rather than one big file, based purely on issues of performance; you are likely to hit the I/O bound on performance very quickly, and that will be much slower.&lt;/p&gt;&#xA;&lt;p&gt;One solution that &lt;a href=&quot;https://stackoverflow.com/questions/55305833/extracting-subset-alignment-based-in-sequence-name-using-seqinr&quot;&gt;has been suggested previously for this kind of problem&lt;/a&gt; is to write the whole alignment to a single file, and then post-process that filed in a second step to extract the sequences of interest. This could be as trivial as a &lt;code&gt;samtools faidx&lt;/code&gt; call from the shell, or you could work with it in seqinr as suggested in the previous answer.&lt;/p&gt;&#xA;&lt;p&gt;Of course, it seems like it would be simpler for most purposes to just have the big alignment file and use e.g. samtools indexing / faidx / biopython / whatever to extract the sequences of interest. But it seems that that is specifically not desired in this case.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Original answer&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would strongly suggest using the &lt;a href=&quot;https://cran.r-project.org/web/packages/seqinr/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqinr&lt;/a&gt; package, which is optimized for solving exactly these problems. See e.g. &lt;a href=&quot;https://www.biostars.org/p/270009/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this similar question&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You will be able to read in an alignment (or whatever), cast it to as.matrix.alignment, do whatever your analysis is, then cast it back to as.alignment, which can then be passed to write.fasta to write the files.&lt;/p&gt;&#xA;&lt;p&gt;Whenever possible it is preferable to work with such libraries, which handle these utility functions well, fast, and easy.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2020-12-27T00:03:54.753" LastActivityDate="2020-12-27T00:03:54.753" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="15119" PostTypeId="1" CreationDate="2020-12-26T10:18:31.280" Score="0" ViewCount="94" Body="&lt;p&gt;I am working on a virtual project for WGS combined with RNA seq for annotation. The RNA will be sequenced using PacBio Isoseq (Sequel II, HiFi-reads). With some research, I found that StringTie2 is useful for this. In their &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1910-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;release paper&lt;/a&gt; , they sometimes mention previous alignment or mapping with TopHat or minihat, but I don't find a manual where they say the same. Now I don't know what is supposed to be the input of StringTie2: raw RNA reads, or a mapped BAM-file?&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="10147" LastActivityDate="2020-12-26T10:18:31.280" Title="Required input files for StringTie2" Tags="|transcriptome|pacbio|stringtie|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15120" PostTypeId="1" CreationDate="2020-12-26T12:29:23.940" Score="6" ViewCount="129" Body="&lt;h3&gt;The Question&lt;/h3&gt;&#xA;&lt;p&gt;I want to find high coverage SRA entries, e.g., above 100x.&lt;br/&gt;&lt;/p&gt;&#xA;&lt;p&gt;I guess the best way is to use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra&quot; rel=&quot;noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/sra&lt;/a&gt; with an appropriate search term. I don't mind if the search results contain some &amp;quot;false-positives&amp;quot;. (I.e., if some of the found entries have coverage above 80x, while I aimed for 100x, that's fine.)&#xA;I want to avoid using Entrez.esummary on a very large number of SRA entries (e.g., through Biopython), as it would be significantly slower (if I understand correctly).&lt;/p&gt;&#xA;&lt;p&gt;How can I do that?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;My Attempt&lt;/h3&gt;&#xA;&lt;p&gt;I tried using the &lt;code&gt;Mbases&lt;/code&gt; field.&lt;br/&gt;Assuming the reference genome is at most 100kbp (e.g., in case I am interested in viruses), I thought a good approximation would be to search for SRA entries with more than 100*100k bases, i.e., at least 10M bases.&#xA;According to &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK3837/&quot; rel=&quot;noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/books/NBK3837/&lt;/a&gt;, I thought I could use the search term &lt;code&gt;10:9999999[Mbases]&lt;/code&gt;, but disappointingly, this didn't work. Just to make sure my syntax is right, I verified that &lt;code&gt;100:101[ReadLength]&lt;/code&gt; works. So it seems that &lt;code&gt;Mbases&lt;/code&gt; doesn't support the range syntax.&lt;/p&gt;&#xA;" OwnerUserId="4868" LastActivityDate="2021-02-26T13:41:27.953" Title="How to search for high coverage SRA entries" Tags="|ngs|database|public-databases|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15121" PostTypeId="1" CreationDate="2020-12-26T17:54:44.883" Score="3" ViewCount="74" Body="&lt;p&gt;Intel and the Broad Institute have developed &lt;a href=&quot;https://github.com/Intel-HLS/BIGstack&quot; rel=&quot;nofollow noreferrer&quot;&gt;BIGstack&lt;/a&gt;, described in this Broad-Intel Genomics stack &lt;a href=&quot;https://www.intel.com/content/dam/www/public/us/en/documents/white-papers/accelerate-genomics-research-with-the-broad-intel-genomics-stack-paper.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;whitepaper&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Eventually I would love to perform a variant calling on BIGstack, but it isn't clear how to run a “simple” example of BIGstack on Xeon with Altera FPGA.&lt;/p&gt;&#xA;&lt;p&gt;Since I have my own old Altera DE2 FPGA board and I would like to run the simplest possible toy example which is possible to run on my old Altera DE2 board. Is it possible to use my Altera DE2 to run something simple on top of BIGstack?&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="1157" LastEditDate="2023-05-09T12:23:28.820" LastActivityDate="2023-05-09T12:23:28.820" Title="Is there a baby Hello World example using BIGstack?" Tags="|untagged|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15122" PostTypeId="1" CreationDate="2020-12-27T19:39:10.803" Score="1" ViewCount="291" Body="&lt;ol&gt;&#xA;&lt;li&gt;Hi there I am still a novice at this. I'm trying to run this perl script called FullSSR &lt;a href=&quot;https://sourceforge.net/projects/fullssr&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sourceforge.net/projects/fullssr&lt;/a&gt;, by following the instructions in their 'README.TXT' &lt;a href=&quot;https://sourceforge.net/projects/fullssr/files/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sourceforge.net/projects/fullssr/files/&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;I did everything as instructed by the app developer(s) which included among others, replacing the Primer3.pm file provided in the *.tar.gz file into /usr/share/perl5/Bio/Tools/Run/; opening the config.txt and specifying the path of primer3_core (file) and primer3_config/ (folder); placing the sequence file I wanted to analyse into the FullSSR folder and executing the perl script using the command line: &lt;code&gt;perl 'FullSSR\(ver-1.1\).p; -i sequence.fasta -m 5&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;However, it was to no avail. Instead, I got the message &amp;quot;Can't locate object method &amp;quot;new&amp;quot; via package Bio::Tools::Primer3&amp;quot; (perhaps you forgot to load &amp;quot;Bio::Tools::Primer3?) at /usr/share/perl5/Bio/Tools/Run/Primer3.pm line 497.&lt;/li&gt;&#xA;&lt;li&gt;I am going to try to install/load Bio::Tools::Primer3 with the cpan module (cpanm) and will see if that does it.&lt;/li&gt;&#xA;&lt;li&gt;Perhaps someone out there could suggest a helpful (easy to understand) manual on CPAN?!&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9641" LastEditorUserId="9641" LastEditDate="2020-12-27T19:45:16.040" LastActivityDate="2020-12-27T19:46:03.503" Title="Can't locate object method &quot;new&quot; via package Bio::Tools::Primer3" Tags="|perl|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15123" PostTypeId="2" ParentId="15122" CreationDate="2020-12-27T19:46:03.503" Score="1" Body="&lt;p&gt;I managed to figure out the answer to my question. I installed Bio::Tools::Primer3 by typing the following into terminal: '&lt;code&gt;sudo cpanm Bio::Tools::Primer3 --force&lt;/code&gt;' to install it. It only works when I used 'sudo' and 'force'&lt;/p&gt;&#xA;" OwnerUserId="9641" LastActivityDate="2020-12-27T19:46:03.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15125" PostTypeId="1" AcceptedAnswerId="15161" CreationDate="2020-12-28T05:32:43.083" Score="0" ViewCount="225" Body="&lt;p&gt;I am trying to implement Nvidia-parabrick fq2bam pipeline to process my WES dataset. The reason for opting this pipeline is the huge number of samples (1004) which take many months to process for somatic mutation identification. THe GPU accelerated pipeline will be more helpful to get somatic mutations in a quick manner. After pulling parabrick docker image from &lt;a href=&quot;https://ngc.nvidia.com/catalog/containers/hpc:parabricks&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, the following command works fine and return the options :&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;docker run -it nvcr.io/hpc/parabricks:v2.5.0 -h&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But whenever I give the following command to run fq2bam pipeline, I am getting an error of &lt;code&gt;No such file or directory&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;docker run -it nvcr.io/hpc/parabricks:v2.5.0 fq2bam --ref ucsc.hg19.fasta --in-fq sample_1.fastq.gz sample_2.fastq.gz --knownSites 1000G_phase1.indels.hg19.sites.vcf.gz --out-bam sample_dups_gpu.bam out-recal-file sample_recal_gpu.txt --tmp-dir /tmp&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The error is as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;File &amp;quot;/parabricks/run_pipeline.py&amp;quot;, line 7, in &amp;lt;module&amp;gt;&#xA;sys.exit(PB.pb_main())&#xA;File &amp;quot;PB.pyx&amp;quot;, line 846, in PB.pb_main&#xA;File &amp;quot;/parabricks/pbargs.py&amp;quot;, line 1287, in getArgs&#xA;return PBRun(sys.argv)&#xA;File &amp;quot;/parabricks/pbargs.py&amp;quot;, line 392, in __init__&#xA;self.runArgs = getattr(self, args.command)(argList)&#xA;File &amp;quot;/parabricks/pbargs.py&amp;quot;, line 447, in fq2bam&#xA;self.addToParser(bsmb_parser_sysgroup, sysOptionGenerator().allOptions)&#xA;File &amp;quot;/parabricks/pbargs.py&amp;quot;, line 50, in __init__&#xA;PBOption(category=&amp;quot;sysOption&amp;quot;, name=&amp;quot;--num-gpus&amp;quot;, default=GetNumGPUs(), typeName=int, helpStr=&amp;quot;Number of GPUs to use for a run&amp;quot;)&#xA;File &amp;quot;/parabricks/pbutils.py&amp;quot;, line 48, in GetNumGPUs&#xA;output = subprocess.check_output([&amp;quot;nvidia-smi&amp;quot;, &amp;quot;-L&amp;quot;], universal_newlines=True)&#xA;File &amp;quot;/usr/lib/python2.7/subprocess.py&amp;quot;, line 567, in check_output&#xA;process = Popen(stdout=PIPE, *popenargs, **kwargs)&#xA;File &amp;quot;/usr/lib/python2.7/subprocess.py&amp;quot;, line 711, in __init__&#xA;errread, errwrite)&#xA;File &amp;quot;/usr/lib/python2.7/subprocess.py&amp;quot;, line 1343, in _execute_child&#xA;raise child_exception&#xA;OSError: [Errno 2] No such file or directory&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As I am using the docker image as it is,it is not clear to me which file is missing. Kindly suggest. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="2141" LastEditorUserId="3694" LastEditDate="2020-12-30T05:47:11.557" LastActivityDate="2021-01-06T13:40:03.117" Title="Nvidia Parabrick fq2bam pipeline error - No such file or directory" Tags="|sam|genomics|fastq|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15126" PostTypeId="1" CreationDate="2020-12-28T05:49:23.007" Score="1" ViewCount="162" Body="&lt;p&gt;Im using bamCoverage on a file and ive set the bin size to 1 but also normalizing using RPKM for defined regions using a bedgraph file.  The output from bamCoverage however is in binned segments each with a (RPKM) score.  Am I correct in interpreting then that&lt;/p&gt;&#xA;&lt;p&gt;1 - the output bedgraph tells me the (total) RPKM score for that particular bin?&lt;/p&gt;&#xA;&lt;p&gt;2 - to get the total RPKM score for the region defined in the bedgraph I need to sum all the RPKM scores given for that particular region?&lt;/p&gt;&#xA;" OwnerUserId="10155" LastActivityDate="2020-12-28T16:27:27.863" Title="Clarification regarding (deepTools) bamCoverage output" Tags="|rna-seq|deeptools|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15128" PostTypeId="2" ParentId="14915" CreationDate="2020-12-28T11:46:17.773" Score="1" Body="&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;import sys&#xA;import re &#xA;&#xA;unique = []&#xA;fasta_file = (sys.argv[1])&#xA;for myfile in SeqIO.parse(fasta_file, &amp;quot;fasta&amp;quot;):&#xA;    if len(myfile) &amp;gt; 250:&#xA;        gene_id = myfile.id&#xA;        if gene.id not in unique:&#xA;            unique.append(gene.id)&#xA;            mylist = re.match(r&amp;quot;H149xcV_[^\W_]+_[^\W_]+_[^\W_])_[^\W_]+&amp;quot;, gene_id)&#xA;            print (&amp;quot;&amp;gt;&amp;quot;+mylist.group(1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="3694" LastEditDate="2022-02-23T21:36:32.557" LastActivityDate="2022-02-23T21:36:32.557" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15129" PostTypeId="2" ParentId="15126" CreationDate="2020-12-28T16:27:27.863" Score="3" Body="&lt;ol&gt;&#xA;&lt;li&gt;Yes&lt;/li&gt;&#xA;&lt;li&gt;The total RPKM for the region in the bedgraph file is the value already present in the bedgraph file. If you want to estimate the RPKM for a larger region composed of multiple bins then you can average the per-bin RPKM values. Note that adjacent bins with the same value are merged by deepTools, so you will need to account for this.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Note that RPKM is NOT a robust normalization and that I should remove it from deepTools so people stop using it.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-12-28T16:27:27.863" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15130" PostTypeId="1" CreationDate="2020-12-28T18:07:26.903" Score="0" ViewCount="43" Body="&lt;p&gt;is there a dataset online containing a HiC contact matrix and file with TAD boundary locations, which were generated, either by hand or automated, with high accuracy? Although I already used programs such as SpectralTAD or GMAP to generate TADs myself, a dataset which was computed with higher accuracy would be nice to check my project against.&lt;/p&gt;&#xA;&lt;p&gt;Preferred attributes would be:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;human genome&lt;/li&gt;&#xA;&lt;li&gt;whole genome&lt;/li&gt;&#xA;&lt;li&gt;10 kb resolution or higher&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Thank you very much in advance!&lt;/p&gt;&#xA;&lt;p&gt;edit: One dataset I found is &lt;a href=&quot;http://chromosome.sdsc.edu/mouse/hi-c/download.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://chromosome.sdsc.edu/mouse/hi-c/download.html&lt;/a&gt;, which includes multiple 10 kb datasets with TADs called.&lt;/p&gt;&#xA;" OwnerUserId="10158" LastEditorUserId="10158" LastEditDate="2020-12-30T16:01:33.283" LastActivityDate="2021-05-29T23:05:30.660" Title="Can anyone point me towards a HiC Dataset with domains of high confidence?" Tags="|hi-c|chromosome-capture|domains|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15132" PostTypeId="1" AcceptedAnswerId="15136" CreationDate="2020-12-29T11:01:55.150" Score="-1" ViewCount="64" Body="&lt;p&gt;I m following this &lt;a href=&quot;https://rockefelleruniversity.github.io/RU_ATAC_Workshop.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; for my ATAC seq analysis. Here in of the section they show profile of open chromatin region plots.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;**Plotting insert sizes with Greenleaf open, mono- and di-nucleosome profiles&#xA;This looks very similar to the image from the Greenleaf paper.&#xA;&#xA;We can now annotate our nucleosome free (&amp;lt; 100bp), mono-nucleosome (180bp-247bp) and di-nucleosome (315-437) as in the Greenleaf study.**&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now Since i have done the differential accessibility analysis. So from that information how do i find out region which are &amp;quot;nucleosome free (&amp;lt; 100bp), mono-nucleosome (180bp-247bp) and di-nucleosome (315-437) &amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;My objective&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I would like to see this &amp;quot;nucleosome free (&amp;lt; 100bp), mono-nucleosome (180bp-247bp) and di-nucleosome (315-437) &amp;quot; from my differential accessible region which was done from readcount and then I ran deseq2.&lt;/li&gt;&#xA;&lt;li&gt;My goal is not to plot the way they have done rather find those regions mentioned above and see it between different conditions.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Now based on the tutorial of that section where they do for each bam file.&lt;/p&gt;&#xA;&lt;p&gt;Do i need to run the analysis on each bam file or each condition or Can I find out the above from my deseq2 analysis which has regions and those regions I have annotated it .&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2020-12-30T07:33:37.917" Title="Open chromatin region : nucleosome , mono-nucleosome , di-nucleosome" Tags="|atac-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15134" PostTypeId="1" AcceptedAnswerId="15137" CreationDate="2020-12-30T05:55:32.933" Score="-1" ViewCount="310" Body="&lt;p&gt;The transcription factor track in UCSC is named as Txn factor. Why Txn == Transcription?&lt;/p&gt;&#xA;" OwnerUserId="9327" LastEditorUserId="57" LastEditDate="2021-01-12T14:56:14.420" LastActivityDate="2021-01-12T14:56:14.420" Title="What's the meaning of Txn factor?" Tags="|ucsc|transcription-factors|transcriptome-regulation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15135" PostTypeId="1" CreationDate="2020-12-30T06:38:32.320" Score="1" ViewCount="41" Body="&lt;p&gt;I have project where I am required to build a visual analytics tool into an existing Java project to visually compare and analyse protein sequences from SwissProt, GenBank, EMBL etc. (downloaded from the databases in plain text and locally converted to XML)&lt;/p&gt;&#xA;&lt;p&gt;Assuming I have the XML files of the protein sequences already, I am unsure regarding the following:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The best ways to visualize protein sequence differences.&#xA;(e.g. what types of graphs/ diagrams? Any sample images or links I could reference from?)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What information should be derivable from these visualizations and how they help researchers.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Any advice on the above?&lt;/p&gt;&#xA;&lt;p&gt;Note: I have no background in bioinformatics.&lt;/p&gt;&#xA;&lt;p&gt;Any help would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="10169" LastActivityDate="2020-12-30T06:38:32.320" Title="Java Protein Comparison Visualisation" Tags="|proteins|protein-structure|visualization|java|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15136" PostTypeId="2" ParentId="15132" CreationDate="2020-12-30T07:33:37.917" Score="1" Body="&lt;p&gt;It's impossible to find nucleosome free and such regions without a BAM file. Your only options are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Filter the BAM files to contain only the size-range of interest and rerun the counting and statistics.&lt;/li&gt;&#xA;&lt;li&gt;Find or write a counting program that can track fragment sizes.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;BTW, those size ranges could use some adjustments, you're throwing away all fragments between 100 and 180 bases using their ranges.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-12-30T07:33:37.917" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15137" PostTypeId="2" ParentId="15134" CreationDate="2020-12-30T07:39:04.580" Score="0" Body="&lt;p&gt;Any other abbreviation is likely to work for &amp;quot;translation&amp;quot; as well and the x makes it sound a bit like transcription (txn would be pronounced something like &amp;quot;tixon&amp;quot;).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2020-12-30T07:39:04.580" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15139" PostTypeId="2" ParentId="14081" CreationDate="2020-12-30T13:50:40.023" Score="2" Body="&lt;p&gt;Indeed there's no satisfactory doublet/multiplet identification methods as of 2020.&lt;/p&gt;&#xA;&lt;p&gt;The most reliable way to remove doublets is still manual removal of doublet subclusters guided by marker genes.&lt;/p&gt;&#xA;&lt;p&gt;Not only trajectory analysis can be impacted by doublets, but clustering and embedding can also be affected for obvious reasons.&lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2020-12-30T13:50:40.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15140" PostTypeId="2" ParentId="15130" CreationDate="2020-12-30T20:12:38.880" Score="1" Body="&lt;p&gt;I guess your best chance is one of the in-situ HiC papers from the Lieberman-Aiden lab. There are not too many HiC datasets with 10kb-or-better resolution out there for human. I would check this one:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://cell.com/fulltext/S0092-8674(14)01497-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;A 3D Map of the Human Genome at Kilobase Resolution Reveals Principles of Chromatin Looping&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2020-12-30T20:12:38.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15141" PostTypeId="1" AcceptedAnswerId="15303" CreationDate="2020-12-31T05:18:03.107" Score="0" ViewCount="134" Body="&lt;p&gt;&lt;strong&gt;AIM:&lt;/strong&gt; Download &amp;quot;Unique Identifier List&amp;quot; for the following query from &lt;a href=&quot;https://www-ncbi-nlm-nih-gov.ezproxy.u-pec.fr/gds&quot; rel=&quot;nofollow noreferrer&quot;&gt;GEO DataSets&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Query:&lt;/strong&gt; &lt;em&gt;(&amp;quot;Expression profiling by high throughput sequencing&amp;quot;[DataSet Type] AND (&amp;quot;Homo sapiens&amp;quot;[Organism] OR &amp;quot;Mus musculus&amp;quot;[Organism] OR &amp;quot;rattus norvegicus&amp;quot;[Organism])) AND (&amp;quot;2020/01/01&amp;quot;[PDAT] : &amp;quot;3000&amp;quot;[PDAT])&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;which means, all RNASeq studies deposited on GEO in the year 2020 for humans, mice or rat.&lt;/p&gt;&#xA;&lt;p&gt;Problem: I need the GSE ID list for ~9k datasets, but while trying to download the list of ids, it loads to a blank page and nothing happens. Also, clicking on &amp;quot;Next Page&amp;quot; gives error.&#xA;I have been trying for the last 3-4 days but it doesn't work.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Steps to generate file:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&amp;quot;Send To&amp;quot; -&amp;gt; &amp;quot;File&amp;quot; -&amp;gt; &amp;quot;Format&amp;quot; (Unique Identifier List) -&amp;gt; &amp;quot;Sort By&amp;quot; (Default Order) -&amp;gt; &amp;quot;Create File&amp;quot;&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="4121" LastActivityDate="2021-01-31T01:56:28.673" Title="Getting Unique Identifier List for GEO Datasets NCBI" Tags="|phylogenetics|geo|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15142" PostTypeId="2" ParentId="15141" CreationDate="2020-12-31T22:30:56.663" Score="0" Body="&lt;p&gt;I think I would just try to do this with GEOquery.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/GEOquery.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/html/GEOquery.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10175" LastActivityDate="2020-12-31T22:30:56.663" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15143" PostTypeId="1" CreationDate="2021-01-01T09:26:28.250" Score="1" ViewCount="101" Body="&lt;p&gt;I've been assigned a task to fetch sequence records from &amp;quot;&lt;strong&gt;pubmed&lt;/strong&gt;&amp;quot; database:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;fetch 30 record of type &amp;quot;fasta&amp;quot; from (pubmed database, with term hemoglobin AND alpha) as a sequence records&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It doesn't make any sense to me!&lt;/p&gt;&#xA;&lt;p&gt;I know the PubMed database is for publications and citations, but how can I retrieve those records from it? is that even possible?&lt;/p&gt;&#xA;&lt;p&gt;I tried to search for the IDs with a given term in it and then retrieve the records from another database (like &amp;quot;&lt;strong&gt;nuccore&lt;/strong&gt;&amp;quot;), but I don't know if this is right.&lt;/p&gt;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="10177" LastEditorUserId="73" LastEditDate="2021-12-02T00:37:20.813" LastActivityDate="2023-08-24T06:08:20.937" Title="BioPython - Retrieve sequence records from pubmed database" Tags="|python|homework|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15144" PostTypeId="1" AcceptedAnswerId="15165" CreationDate="2021-01-01T12:32:02.710" Score="2" ViewCount="438" Body="&lt;p&gt;I think there is a problem with &lt;code&gt;TxDb.Hsapiens.UCSC.hg19.knownGene&lt;/code&gt; and  &lt;code&gt;EnsDb.Hsapiens.v75&lt;/code&gt;.  I'm struggling to install all of these packages as they pause at  &lt;code&gt;byte-compile and prepare package for lazy loading &lt;/code&gt;. Can anyone try to re-install them in their local machine because am not the only one who is having the problem, please? They used to work fine until a week ago!&lt;/p&gt;&#xA;&lt;p&gt;As usual, I install the packages by:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BiocManager::install(c(&amp;quot;Homo.sapiens&amp;quot;, &amp;quot;EnsDb.Hsapiens.v75&amp;quot;, &amp;quot;TxDb.Hsapiens.UCSC.hg19.knownGene&amp;quot;))&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also tried installing them individually:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BiocManager::install(&amp;quot;EnsDb.Hsapiens.v75&amp;quot;)&#xA;BiocManager::install(&amp;quot;TxDb.Hsapiens.UCSC.hg19.knownGene&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It pause at:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;* installing *source* package ‘TxDb.Hsapiens.UCSC.hg19.knownGene’ ...&#xA;** using staged installation&#xA;** R&#xA;** inst&#xA;** byte-compile and prepare package for lazy loading&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;sessionInfo( )&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;R version 4.0.3 (2020-10-10)&#xA;Platform: x86_64-apple-darwin17.0 (64-bit)&#xA;Running under: macOS Big Sur 10.16&#xA;&#xA;Matrix products: default&#xA;BLAS:   /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRblas.dylib&#xA;LAPACK: /Library/Frameworks/R.framework/Versions/4.0/Resources/lib/libRlapack.dylib&#xA;&#xA;locale:&#xA;[1] en_GB.UTF-8/en_GB.UTF-8/en_GB.UTF-8/C/en_GB.UTF-8/en_GB.UTF-8&#xA;&#xA;attached base packages:&#xA;[1] stats     graphics  grDevices utils     datasets  methods   base     &#xA;&#xA;loaded via a namespace (and not attached):&#xA;[1] BiocManager_1.30.10 compiler_4.0.3      tools_4.0.3    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've tried with and without RStudio.&#xA;I'd be grateful if someone can try to replicate the error, please?&lt;/p&gt;&#xA;" OwnerUserId="10179" LastEditorUserId="10179" LastEditDate="2021-01-03T17:52:19.157" LastActivityDate="2021-01-07T22:12:14.380" Title="`TxDb.Hsapiens.UCSC.hg19.knownGene` and `EnsDb.Hsapiens.v75` can't be installed" Tags="|bioconductor|software-installation|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="15145" PostTypeId="2" ParentId="14316" CreationDate="2021-01-02T22:40:58.827" Score="0" Body="&lt;p&gt;in genomics, organisms have a single reference genome which is updated( or not) over a period of time. E.coli has different strains and this means there might be reference genomes for each of the strain. If there is no reference genome available the its likely that in constructing the sequence of those strains they might have used any of those two as their reference. This information you can find by reading the paper that published those strains. Therefore I will advice you pick any of those two available reference genome as your reference and work with and make sure you indicate which one you used in your report or document.&lt;/p&gt;&#xA;" OwnerUserId="10184" LastActivityDate="2021-01-02T22:40:58.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15146" PostTypeId="2" ParentId="11536" CreationDate="2021-01-02T22:48:36.153" Score="1" Body="&lt;p&gt;Before you compare your genome assembly fasta, I would recommend you first do a reference guided scaffolding using the tool &lt;a href=&quot;https://github.com/malonge/RagTag&quot; rel=&quot;nofollow noreferrer&quot;&gt;RagTag&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Once you are done you can compare the newly generated scaffolds (with other genomes or not) using the software BRIG. I made a video of how to use this software &lt;a href=&quot;https://youtu.be/pobQgE4z-5Q&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In addition you can also do a genome annotation on your scaffold using prokka. I made a video for that as well &lt;a href=&quot;https://youtu.be/MqDhhuIXjgg&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Good luck&lt;/p&gt;&#xA;" OwnerUserId="10184" LastActivityDate="2021-01-02T22:48:36.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15147" PostTypeId="1" CreationDate="2021-01-03T14:36:28.483" Score="1" ViewCount="294" Body="&lt;p&gt;I want to create motif of different length kmers.&#xA;Following code works for kmers of same length but not different length.What to do ?&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio import motifs&#xA;&#xA;sequences = SeqIO.parse(&amp;quot;myfile.fasta&amp;quot;,&amp;quot;fasta&amp;quot;)&#xA;    instances = []&#xA;    for data in sequences:&#xA;      instance = data.seq&#xA;      instances.append(instance)&#xA;    &#xA;motif = motifs.create(instances)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9912" LastEditorUserId="1310" LastEditDate="2021-01-03T19:16:09.467" LastActivityDate="2021-01-04T21:39:59.880" Title="biopython-motifs , How can I create motif from sequences of different length?" Tags="|python|motifs|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15148" PostTypeId="2" ParentId="174" CreationDate="2021-01-04T12:43:00.533" Score="0" Body="&lt;p&gt;There is a new tool called &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2020.12.14.422624v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hapo-G&lt;/a&gt; for diploid-aware polishing. Unfortunately, it seems it's designed for short reads only. It seems to be a more practical choice with respect to inferring the phase information on individual reads.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2021-01-04T12:43:00.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15150" PostTypeId="2" ParentId="15147" CreationDate="2021-01-04T18:05:16.053" Score="1" Body="&lt;p&gt;All sequences represented in a &lt;a href=&quot;https://biopython.org/docs/1.75/api/Bio.motifs.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biopython motif object&lt;/a&gt; must be the same length. For example, a motif has a &lt;code&gt;__len__&lt;/code&gt; property, which has to consist of a single value.&lt;/p&gt;&#xA;&lt;p&gt;There are &lt;a href=&quot;https://www.nature.com/articles/s41598-019-38746-w&quot; rel=&quot;nofollow noreferrer&quot;&gt;other methods&lt;/a&gt; that &lt;em&gt;do&lt;/em&gt; support variable-length motifs, I suggest that you look into them, but unfortunately biopython seems to be limited to fixed-length motifs.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-01-04T18:05:16.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15151" PostTypeId="2" ParentId="15098" CreationDate="2021-01-04T18:31:10.880" Score="0" Body="&lt;p&gt;OK fine, found useful information on READme.md generated by OpenAPI-generator itself&lt;/p&gt;&#xA;&lt;p&gt;and tried &lt;code&gt;pdoc&lt;/code&gt; to get html document of my OpenAPI-generator generated client Module&lt;/p&gt;&#xA;&lt;p&gt;Moreover on stackoverflow &lt;a href=&quot;https://stackoverflow.com/questions/59727169/how-to-generate-api-documentation-using-openapi-generator&quot;&gt;How to generate API Documentation using openapi-generator?&lt;/a&gt; you can find how OpenApi Generator can do generation of html documentation from your yaml or json definition of your API.&lt;/p&gt;&#xA;&lt;p&gt;For openAPI json api specification I found &lt;a href=&quot;https://mrin9.github.io/RapiPdf/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RapiPDF&lt;/a&gt; too : Web Component to generate PDF from OpenAPI Spec (json yml file).&lt;/p&gt;&#xA;" OwnerUserId="10049" LastActivityDate="2021-01-04T18:31:10.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15153" PostTypeId="2" ParentId="15147" CreationDate="2021-01-04T21:39:59.880" Score="3" Body="&lt;p&gt;Motifs are not designed to be variable length in general since PSSMs/PVMs don't support insertions/deletions (indels). Instead, I suggest profile HMMs, such as those offered by HMMER3, which do support indels. You can parse HMMER3 output with Biopython via &lt;code&gt;Bio.SearchIO.HmmerIO&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2021-01-04T21:39:59.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15154" PostTypeId="2" ParentId="637" CreationDate="2021-01-05T15:49:51.597" Score="3" Body="&lt;h2&gt;Part 3&lt;/h2&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I thought I'd add a &lt;strong&gt;part 3&lt;/strong&gt; . Having found via VEP a truncation (missense or frameshift) or a missense, how do you find what are the consequences at the protein level?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The effect a mutation has on a protein can be diverse and how much imbalance in function can the cell's equilibrium absorb depends on the component. As a result you get recessive, dominant and embryonically lethal mutations.&#xA;In this figure are shown some of the effects, divided into core and surface mutations:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/W8Faw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W8Faw.png&quot; alt=&quot;effects&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Core variants&lt;/h2&gt;&#xA;&lt;p&gt;Core mutations are the most black-and-white to analyse, but require a protein structure available. A severe truncation and a highly deleterious missense will result in a protein that will likely be degraded. The difference in folding Gibbs free energy (∆∆G, where &lt;span class=&quot;math-container&quot;&gt;$\Delta \Delta G = \Delta G_{\rm{mutant}} – \Delta G_{\rm{WT}}$&lt;/span&gt;)) can be calculated for a missense, but how large this must be to result in a certain percentage being degraded depends on factors that cannot be calculated analytically. Alternatively, the Gibbs free energy of ligand binding is affected resulting in loss of catalysis.&lt;/p&gt;&#xA;&lt;p&gt;To calculate the ∆∆G effect (stability), there are many tools:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Online &lt;a href=&quot;http://marid.bioc.cam.ac.uk/sdm2&quot; rel=&quot;nofollow noreferrer&quot;&gt;SDM&lt;/a&gt; uses ML to estimate the ∆∆G quickly (do note that positive kcal/mol normally is a loss of stability, but here the sign is inverted so positive is a gain in stability&lt;/li&gt;&#xA;&lt;li&gt;Rosetta toolkit. A highly flexible panel of movers  (samplers) can calculate ∆∆G&lt;/li&gt;&#xA;&lt;li&gt;FoldX&lt;/li&gt;&#xA;&lt;li&gt;MD runs&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.sbg.bio.ic.ac.uk/%7Emissense3d/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Missense3D&lt;/a&gt; assesses a variant based on several structural criteria&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In the case of ligand binding modelling as above but using the ligand bound structure and calculating the binding energy as the difference between bound and unbound (far far away). For altered catalysis, modelling as above but using the transition state and assessing the difference in the transition hill can be done. However, an active site variant is very likely to affect either ligand binding, transition state binding (catalysis) or product binding that it's surely deleterious.&lt;/p&gt;&#xA;&lt;p&gt;This is all (mostly) straightforward. If the domain is a regulatory domain however, there is loss of inhibition, which results in a gain of function (dominant variant).&lt;/p&gt;&#xA;&lt;h2&gt;Surface variants&lt;/h2&gt;&#xA;&lt;p&gt;Then stuff gets messier when you get to surface variants and these can cause dominant effects. However, if you are close to a phosphorylation, ubiquitination or other PTM site, as identified in &lt;a href=&quot;https://www.phosphosite.org/homeAction.action&quot; rel=&quot;nofollow noreferrer&quot;&gt;phosphosite plus&lt;/a&gt;, you likely have a case of deregulation. For linear motifs, &lt;a href=&quot;http://elm.eu.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ELM&lt;/a&gt; is useful, but requires a lot of literature scanning to confirm any find. Alternative a loss of an interface may break the protein function, however, what binds where can only be ascertained by available structures, cross-linking mass spec or in some cases well defined Western experiments, so this data is rare.&#xA;For this type of variant there are two pipelines:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://miscast.broadinstitute.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Miscast&lt;/a&gt; collates information on the protein&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://venus.sgc.ox.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VENUS (beta)&lt;/a&gt; calculates ∆∆G and finds any relevant close-by PTM or gnomAD variants [disclaimer: this is mine].&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;Footnote&lt;/h2&gt;&#xA;&lt;p&gt;As the variants can have a variety of effects, none of these &lt;em&gt;score&lt;/em&gt; (unlike the genetics parts), but instead try to &lt;em&gt;inform&lt;/em&gt; as to what may be going on.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-01-05T15:49:51.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15155" PostTypeId="1" CreationDate="2021-01-06T01:57:13.993" Score="2" ViewCount="400" Body="&lt;p&gt;I'm assuming we don't do pattern matching in DNA or RNA for the fun of it. So I'd like to know what are the applications of pattern matching or where does it fit in in larger applications?&lt;/p&gt;&#xA;&lt;p&gt;I'm a computer scientist and when studying certain topics in CS we see a lot of references to applying algorithms to areas of biology or genetics one of which is applying text pattern matching algorithms to DNA to find sub sequences. Whenever I see those kinds of examples I think &amp;quot;Oh that's pretty cool&amp;quot; but then I don't see how it's applied in the real world for benefit and I can't find anything to suggest where these algorithms are applied in these fields.&lt;/p&gt;&#xA;" OwnerUserId="10240" LastActivityDate="2021-04-26T20:49:50.620" Title="What are the applications of DNA or RNA pattern matching?" Tags="|rna-seq|phylogenetics|" AnswerCount="4" CommentCount="0" ClosedDate="2021-04-28T03:30:06.743" ContentLicense="CC BY-SA 4.0" />
  <row Id="15156" PostTypeId="2" ParentId="15155" CreationDate="2021-01-06T03:37:55.087" Score="2" Body="&lt;p&gt;One useful application of pattern matching is for &lt;a href=&quot;https://www.nature.com/articles/nbt0406-423&quot; rel=&quot;nofollow noreferrer&quot;&gt;DNA sequence motifs&lt;/a&gt;. As an example of a direct application, there is a class of proteins called &lt;a href=&quot;https://en.wikipedia.org/wiki/Transcription_factor&quot; rel=&quot;nofollow noreferrer&quot;&gt;transcription factors&lt;/a&gt; (TFs), which bind to DNA and regulate the expression of genes. There are many TFs. Each TF likes to bind to its particular conserved sequence of DNA, with some wiggle room for some parts of the sequence.&lt;/p&gt;&#xA;&lt;p&gt;Algorithms for finding and scoring these patterns (and mismatches in patterns) is important for identifying regions where regulation occurs, which proteins are involved, and for answering biological questions related to gene regulation and basic biological functionality — and malfunction, in case of disease.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2021-01-06T03:37:55.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15157" PostTypeId="1" AcceptedAnswerId="15159" CreationDate="2021-01-06T10:37:01.660" Score="0" ViewCount="193" Body="&lt;p&gt;My tutorial &lt;a href=&quot;https://rockefelleruniversity.github.io/RU_ATAC_Workshop.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(GenomicAlignments)&#xA;&#xA;atacReads &amp;lt;- readGAlignmentPairs(&amp;quot;MACS2_peak_call/ALL_ATAC_data/HSC1.bam&amp;quot;, param = ScanBamParam(mapqFilter = 1, &#xA;            flag = scanBamFlag(isPaired = TRUE, isProperPair = TRUE), what = c(&amp;quot;qname&amp;quot;, &#xA;        &amp;quot;mapq&amp;quot;, &amp;quot;isize&amp;quot;), which = GRanges(&amp;quot;chr20&amp;quot;, IRanges(1, 63025520))))&#xA;# length(atacReads)&#xA;atacReads&#xA;&#xA;&#xA;atacReads_read1 &amp;lt;- GenomicAlignments::first(atacReads)&#xA;insertSizes &amp;lt;- abs(elementMetadata(atacReads_read1)$isize)&#xA;head(insertSizes)&#xA;&#xA;&#xA;library(magrittr)&#xA;library(dplyr)&#xA;library(ggplot2)&#xA;fragLenPlot &amp;lt;- table(insertSizes) %&amp;gt;% data.frame %&amp;gt;% dplyr::rename(InsertSize = insertSizes, &#xA;    Count = Freq) %&amp;gt;% mutate(InsertSize = as.numeric(as.vector(InsertSize)), &#xA; Count = as.numeric(as.vector(Count))) %&amp;gt;% ggplot(aes(x = InsertSize, y = Count)) + &#xA;  geom_line()&#xA;&#xA;fragLenPlot + theme_bw()&#xA;&#xA;fragLenPlot + scale_y_continuous(trans = &amp;quot;log2&amp;quot;) + theme_bw()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How to i specify the Grange to span the total chromosome. Now I use only &amp;quot;chr20&amp;quot; as shown in the example.&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2021-01-06T12:28:15.470" Title="ATAC seq : Plotting insert sizes" Tags="|atac-seq|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15159" PostTypeId="2" ParentId="15157" CreationDate="2021-01-06T12:28:15.470" Score="1" Body="&lt;p&gt;You can exclude the &lt;code&gt;which&lt;/code&gt; statement and get the entirety of all of the chromosomes. Note however that R is a very bad platform to use for this, since you end up reading everything into memory (I hope the BAM file isn't huge).&lt;/p&gt;&#xA;&lt;p&gt;Note that you're seriously reinventing the wheel. There are a number of packages that will directly produce the plot you want, such as &lt;code&gt;bamPEFragmentSize&lt;/code&gt; from deepTools.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-01-06T12:28:15.470" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15161" PostTypeId="2" ParentId="15125" CreationDate="2021-01-06T13:40:03.117" Score="1" Body="&lt;p&gt;My query was answered by NVIDIA Parabrick developer team. The installation instruction are available at &lt;a href=&quot;https://forums.developer.nvidia.com/uploads/short-url/t5uXsZgdMXx0G2wDcYamFl9sO2x.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://forums.developer.nvidia.com/uploads/short-url/t5uXsZgdMXx0G2wDcYamFl9sO2x.pdf&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2141" LastActivityDate="2021-01-06T13:40:03.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15162" PostTypeId="1" AcceptedAnswerId="15167" CreationDate="2021-01-06T15:48:51.343" Score="3" ViewCount="89" Body="&lt;p&gt;This page is claimed to contain a gold standard benchmark for viral genome assembly.&#xA;&lt;a href=&quot;https://github.com/cbg-ethz/5-virus-mix&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/cbg-ethz/5-virus-mix&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The claim is here:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5411778/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5411778/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Baaijens &lt;em&gt;et al&lt;/em&gt;. De novo assembly of viral quasispecies using overlap graphs&#xA;Genome Research 2017 May; 27: 835–848.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&amp;quot;&lt;em&gt;This data set was recently presented as a gold standard benchmark (Di Giallonardo et al. 2014)&lt;/em&gt;&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;But at (Di Giallonardo et al. 2014) I can only find that mention:&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;We previously estimated the frequencies of the virus strains in the 5-virus-mix by amplifying the protease gene using single-genome amplification (SGA), the current gold standard for studying diversity of virus populations&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;That doesn't seem to mean precisely that the dataset is a gold standard benchmark for genome assembly but, as I understand, that SGA is a gold standard for something else.&lt;/p&gt;&#xA;&lt;p&gt;My question is: is it? And who decided it is? Is still valid? Is there an official list of results somewhere for this benchmark set?&lt;/p&gt;&#xA;&lt;p&gt;I tried to contact the owner of the page but still no answer.&lt;/p&gt;&#xA;" OwnerUserId="5190" LastEditorUserId="3848" LastEditDate="2022-02-28T14:36:33.477" LastActivityDate="2022-02-28T15:09:13.157" Title="Gold standard benchmark" Tags="|genome|assembly|covid-19|benchmarking|virus|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15163" PostTypeId="1" AcceptedAnswerId="15175" CreationDate="2021-01-06T21:18:17.227" Score="1" ViewCount="168" Body="&lt;p&gt;Currently, I am implementing a descriptor of protein structure and would like to calculate properties per atoms like charges, hydrophobicity, hydrogen bond donors / acceptors, hydrophobicity, aromaticity etc.&lt;/p&gt;&#xA;&lt;p&gt;I tried using PDB2PQR and succeeded with protein structures but it crashed while calculating nucleic acid bound proteins and choosing other force fields did not help.&#xA;Also, as far as I know, Rosetta tool can calculate hydrophobicity however, it assigns the property per residue but not per atom (correct me if I am wrong).&lt;/p&gt;&#xA;&lt;p&gt;Any other suitable tool suggestions would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="7150" LastEditorUserId="7150" LastEditDate="2021-01-07T22:04:42.907" LastActivityDate="2021-02-09T11:11:43.587" Title="How to calculate various properties of a protein structure per atom using PDB2PQR and Rosetta tools (or any other tools)?" Tags="|pdb|mutations|rosetta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15164" PostTypeId="1" AcceptedAnswerId="15170" CreationDate="2021-01-07T02:05:59.827" Score="1" ViewCount="618" Body="&lt;p&gt;I'm updating my WDL from draft 2 (whichs works fine) to version 1. I'm stuck with a Py snippet&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;task getReadGroup {&#xA;&#xA;    input {&#xA;        String inputFile&#xA;    }&#xA;&#xA;    command {echo input = ${inputFile}}&#xA;&#xA;    # command &amp;lt;&amp;lt;&amp;lt;&#xA;    # python &amp;lt;&amp;lt;CODE&#xA;    # print (&amp;quot;${inputFile}&amp;quot;.strip().split('/'))&#xA;    # CODE&#xA;    # &amp;gt;&amp;gt;&amp;gt;&#xA;&#xA;    output {&#xA;    Array[String] o = read_lines(stdout())&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This command gives the expected result:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;input = /mnt/lustre/working/genomeinfo/data/20191015_LongRead/20191015_LongRead_1/data/LXBAB201271/LXBAB201271/LXBAB201271/20190417_0612_1-A7-D7_PAD44174_42dcde1a/fast5/test/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But the commented out python code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print (&amp;quot;${inputFile}&amp;quot;.strip().split('/'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Gives an empty list - &amp;quot;${inputFile}&amp;quot; is empty in the code command for some reason!&lt;/p&gt;&#xA;&lt;p&gt;Any ideas? I've tried using a File instead of String to make it just like the example in the docs (below) other than that I see no diff:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;task heredoc {&#xA;  input {&#xA;    File in&#xA;  }&#xA;&#xA;  command&amp;lt;&amp;lt;&amp;lt;&#xA;  python &amp;lt;&amp;lt;CODE&#xA;    with open(&amp;quot;${in}&amp;quot;) as fp:&#xA;      for line in fp:&#xA;        if not line.startswith('#'):&#xA;          print(line.strip())&#xA;  CODE&#xA;  &amp;gt;&amp;gt;&amp;gt;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-01-07T20:42:43.473" Title="What changed wwith Python in WDL change from draft 2 to version 1?" Tags="|python|wdl|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="15165" PostTypeId="2" ParentId="15144" CreationDate="2021-01-07T04:11:25.000" Score="1" Body="&lt;p&gt;I am able to successfully install these packages using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BiocManager::install(c(&amp;quot;Homo.sapiens&amp;quot;, &amp;quot;EnsDb.Hsapiens.v75&amp;quot;, &amp;quot;TxDb.Hsapiens.UCSC.hg19.knownGene&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This completes without any errors / warnings (R v4.0.3 / macOS 10.16), so I don't believe the packages themselves/Bioconductor are 'broken', but I suspect you are having compiler issues due to the Big Sur update.&lt;/p&gt;&#xA;&lt;p&gt;These are the steps I used to solve the problem:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Reinstall xcode command line tools (even if it says &amp;quot;up to date&amp;quot;)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;sudo rm -rf /Library/Developer/CommandLineTools&#xA;sudo xcode-select --install&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Install gcc &amp;amp; llvm via Homebrew (&lt;a href=&quot;https://docs.brew.sh/Installation&quot; rel=&quot;nofollow noreferrer&quot;&gt;instructions for installing Homebrew&lt;/a&gt;) or, if you already have gcc/llvm installed via Homebrew, skip to the next step&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;# This can take several hours&#xA;brew install gcc&#xA;brew install llvm&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Once you have gcc &amp;amp; llvm installed via Homebrew:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;brew cleanup&#xA;brew update&#xA;brew upgrade&#xA;brew reinstall gcc&#xA;brew reinstall llvm&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;Link some headers into /usr/local/include&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;sudo ln -s /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/* /usr/local/include/&#xA;&#xA;# You can safely ignore warnings like this:&#xA;#ln: /usr/local/include//tcl.h: File exists&#xA;#ln: /usr/local/include//tclDecls.h: File exists&#xA;#ln: /usr/local/include//tclPlatDecls.h: File exists&#xA;#ln: /usr/local/include//tclTomMath.h: File exists&#xA;#ln: /usr/local/include//tclTomMathDecls.h: File exists&#xA;#ln: /usr/local/include//tk.h: File exists&#xA;#ln: /usr/local/include//tkDecls.h: File exists&#xA;#ln: /usr/local/include//tkPlatDecls.h: File exists&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;5&quot;&gt;&#xA;&lt;li&gt;Create a new &lt;code&gt;~/.R/Makevars&lt;/code&gt; file (if you already have a &lt;code&gt;~/.R/Makevars&lt;/code&gt; file, save it in a different directory (away from &lt;code&gt;~/.R/&lt;/code&gt;)) and include only these lines in the file:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;FLIBS=-L/usr/local/gfortran/lib/gcc/x86_64-apple-darwin19/10.2.0 -L/usr/local/gfortran/lib -lgfortran -lquadmath -lm&#xA;CXX1X=/usr/local/gfortran/bin/g++&#xA;CXX98=/usr/local/gfortran/bin/g++&#xA;CXX11=/usr/local/gfortran/bin/g++&#xA;CXX14=/usr/local/gfortran/bin/g++&#xA;CXX17=/usr/local/gfortran/bin/g++&#xA;&#xA;LLVM_LOC = /usr/local/opt/llvm&#xA;CC=/usr/local/gfortran/bin/gcc -fopenmp&#xA;CXX=/usr/local/gfortran/bin/g++ -fopenmp&#xA;CFLAGS=-g -O3 -Wall -pedantic -std=gnu99 -mtune=native -pipe&#xA;CXXFLAGS=-g -O3 -Wall -pedantic -std=c++11 -mtune=native -pipe&#xA;LDFLAGS=-L/usr/local/opt/gettext/lib -L&lt;span class=&quot;math-container&quot;&gt;$(LLVM_LOC)/lib -Wl,-rpath,$&lt;/span&gt;(LLVM_LOC)/lib&#xA;CPPFLAGS=-I/usr/local/opt/gettext/include -I$(LLVM_LOC)/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;6&quot;&gt;&#xA;&lt;li&gt;Compile a package from source in R/Rstudio&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;# To check whether openmp is enabled, compile data.table:&#xA;install.packages(&amp;quot;data.table&amp;quot;, type = &amp;quot;source&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;7&quot;&gt;&#xA;&lt;li&gt;If your package fails to compile, a couple of SO users have had to install a fresh gfortran (re: &lt;a href=&quot;https://stackoverflow.com/a/65334247/12957340&quot;&gt;https://stackoverflow.com/a/65334247/12957340&lt;/a&gt;), which you can download from &lt;a href=&quot;https://github.com/fxcoudert/gfortran-for-macOS/releases/tag/10.2-bigsur-intel&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/fxcoudert/gfortran-for-macOS/releases/tag/10.2-bigsur-intel&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Once you can install packages from source without error/issue, update Bioconductor&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BiocManager::install(version = &amp;quot;3.12&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then, finally, you should be able to install your packages using&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BiocManager::install(c(&amp;quot;Homo.sapiens&amp;quot;, &amp;quot;EnsDb.Hsapiens.v75&amp;quot;, &amp;quot;TxDb.Hsapiens.UCSC.hg19.knownGene&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9098" LastEditorUserId="9098" LastEditDate="2021-01-07T22:12:14.380" LastActivityDate="2021-01-07T22:12:14.380" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15167" PostTypeId="2" ParentId="15162" CreationDate="2021-01-07T12:14:03.937" Score="1" Body="&lt;p&gt;Anyone can declare anything to be a &amp;quot;gold-standard&amp;quot;, the term is meaningless. All if actually means is, &amp;quot;we think this is the test that everything else should be benchmarked against&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-01-07T12:14:03.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15168" PostTypeId="1" AcceptedAnswerId="15171" CreationDate="2021-01-07T12:34:06.170" Score="2" ViewCount="2442" Body="&lt;p&gt;I am trying to generate a consensus sequence from a BAM file that was generated by mapping reads to a reference FASTA containing multiple sequences.&lt;/p&gt;&#xA;&lt;p&gt;Usually, I generate consensus sequences from BAM files using samtools and bcftools:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools mpileup -vf reference.fasta filename.sorted.bam | bcftools call -m -O z - &amp;gt; filename.vcf.gz&#xA;&#xA;bcftools index filename.vcf.gz&#xA;&#xA;bcftools consensus -f reference.fasta filename.vcf.gz &amp;gt; filename.consensus.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, this doesn't work when the reference FASTA contains more than 1 sequence.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to generate a consensus sequence from a multi-reference BAM file?&lt;/p&gt;&#xA;&lt;p&gt;It would also be awesome if the method filled in gaps in coverage with Ns or Xs in the generated consensus sequence but this isn't essential.&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your help!&lt;/p&gt;&#xA;" OwnerUserId="10249" LastActivityDate="2021-01-07T23:46:01.440" Title="How to generate a consensus sequence from a multi-reference BAM file?" Tags="|sam|samtools|ngs|mpileup|bcftools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15169" PostTypeId="2" ParentId="4963" CreationDate="2021-01-07T17:08:39.090" Score="0" Body="&lt;p&gt;Try using ridgeplots instead of violin plots.  Ridge plots will show you that bump at zero&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2021-01-07T17:08:39.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15170" PostTypeId="2" ParentId="15164" CreationDate="2021-01-07T20:42:43.473" Score="2" Body="&lt;p&gt;Complements of Phil Shapiro;&lt;/p&gt;&#xA;&lt;p&gt;Use ~{} instead of ${}.&lt;/p&gt;&#xA;&lt;p&gt;So this example from the current docs is wrong:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;task heredoc {&#xA;  input {&#xA;    File in&#xA;  }&#xA;&#xA;  command&amp;lt;&amp;lt;&amp;lt;&#xA;  python &amp;lt;&amp;lt;CODE&#xA;    with open(&amp;quot;${in}&amp;quot;) as fp:&#xA;      for line in fp:&#xA;        if not line.startswith('#'):&#xA;          print(line.strip())&#xA;  CODE&#xA;  &amp;gt;&amp;gt;&amp;gt;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It should be :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;task heredoc {&#xA;  input {&#xA;    File in&#xA;  }&#xA;&#xA;  command&amp;lt;&amp;lt;&amp;lt;&#xA;  python &amp;lt;&amp;lt;CODE&#xA;    with open(&amp;quot;~{in}&amp;quot;) as fp:&#xA;      for line in fp:&#xA;        if not line.startswith('#'):&#xA;          print(line.strip())&#xA;  CODE&#xA;  &amp;gt;&amp;gt;&amp;gt;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-01-07T20:42:43.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15171" PostTypeId="2" ParentId="15168" CreationDate="2021-01-07T23:46:01.440" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/367626/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This post on biostars&lt;/a&gt; has several suggestions.&lt;/p&gt;&#xA;&lt;p&gt;THe one that seems most straightforward, starting where you are, is to use &lt;a href=&quot;http://vcftools.sourceforge.net/perl_module.html#vcf-consensus&quot; rel=&quot;nofollow noreferrer&quot;&gt;vcftools&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat ref.fa | vcf-consensus file.vcf.gz &amp;gt; out.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Based on that I think that you should also be able to solve this problem using &lt;a href=&quot;https://github.com/tobiasrausch/alfred&quot; rel=&quot;nofollow noreferrer&quot;&gt;alfred&lt;/a&gt; and &lt;a href=&quot;https://software.broadinstitute.org/gatk/documentation/tooldocs/3.8-0/org_broadinstitute_gatk_tools_walkers_fasta_FastaAlternateReferenceMaker.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATK&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If all of these suffer the same problem of needing a single sequence (which I very much doubt that they do), you could split the bam / ref files into multiple files, run whatever consensus workflow, and then combine them again. But that doesn't seem like it should be necessary.&lt;/p&gt;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://www.biostars.org/p/210219/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt; and &lt;a href=&quot;https://www.biostars.org/p/106888/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-01-07T23:46:01.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15172" PostTypeId="2" ParentId="13162" CreationDate="2021-01-08T11:30:22.840" Score="1" Body="&lt;p&gt;In my personal experience, MUSCLE is the easiest program to use in conjunction with Biopython. Biopython features a &lt;a href=&quot;https://biopython.org/docs/1.75/api/Bio.Align.Applications.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;command line wrapper&lt;/a&gt; for this program, which makes it very easy to use. Make sure to &lt;a href=&quot;http://www.drive5.com/muscle/downloads.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;download the appropriate MUSCLE program from drive5&lt;/a&gt; and save it somewhere. E.g., if you are using Jupyter in Linux:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;!wget https://www.drive5.com/muscle/downloads3.8.31/muscle3.8.31_i86linux64.tar.gz&#xA;!tar -xzvf muscle3.8.31_i86linux64.tar.gz&#xA;!cp muscle3.8.31_i86linux64 /usr/local/bin&#xA;!chmod 755 /usr/local/bin/muscle3.8.31_i86linux64&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then, you can run MUSCLE like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.Align.Applications import MuscleCommandline&#xA;def runMUSCLE(infile, outfile):&#xA;    muscle_exe = r&amp;quot;/usr/local/bin/muscle3.8.31_i86linux64&amp;quot; #Here is where we installed MUSCLE&#xA;    muscle_cline = MuscleCommandline(muscle_exe,&#xA;                                     input=infile,&#xA;                                     out=outfile,&#xA;                                     clwstrict=True #Output in clustal format (more visually pleasing), otherwise the output is in FASTA. Whichever you need.&#xA;                                    )&#xA;    muscle_cline()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You don't need to specify that your sequence is amino acidic, however keep in mind that the input file must be in FASTA format.&lt;/p&gt;&#xA;" OwnerUserId="10254" LastActivityDate="2021-01-08T11:30:22.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15173" PostTypeId="1" CreationDate="2021-01-08T14:03:53.283" Score="0" ViewCount="56" Body="&lt;p&gt;Brain data, particualrly human brain data have very heterogeneus cell types, so it's important to normalise by cell type/add proportoins of different cell types to the formula when performing differential expression. We have some bulk RNA-seq brain data from disease samples and wildtypes. I have used the package BrainInABlender to get a matrix of the proportions of each cell type in each sample&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;Cell type&lt;/th&gt;&#xA;&lt;th&gt;Sample_1&lt;/th&gt;&#xA;&lt;th&gt;Sample 2&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Neuron&lt;/td&gt;&#xA;&lt;td&gt;0.5&lt;/td&gt;&#xA;&lt;td&gt;0.7&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Microglia&lt;/td&gt;&#xA;&lt;td&gt;0.5&lt;/td&gt;&#xA;&lt;td&gt;0.3&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;The rows are the types of cells, the columns are the samples. I wanted to know what's the best way to add this as a confounding variable in Enrichment Browser (or if that's not possible in DESeq2).&lt;/p&gt;&#xA;&lt;p&gt;Edit: Current way I've tried to correct for this is in DESeq2. I have transposed the matrix of cell type proportoins then set DESeq's formula:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;~ Condition + Condition:Neuron:Microglia:etc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This has drastically reduced the number of genes found to be differentially expressed but I don't know if it's the right way to do it.&lt;/p&gt;&#xA;&lt;p&gt;Edit 2: When I tried removing the interaction terms, replacing them with +&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;~ Condition + Neuron + Microglia etc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As suggested &lt;a href=&quot;https://support.bioconductor.org/p/76265/&quot; rel=&quot;nofollow noreferrer&quot;&gt;on the Bioconductor support section&lt;/a&gt; the number of genes went up by an order of magnitude from just using condition which is the opposite of what should have happened. This persisted whether condition was at the start or end of my formula.&lt;/p&gt;&#xA;" OwnerUserId="1843" LastEditorUserId="1843" LastEditDate="2021-01-11T15:56:23.107" LastActivityDate="2021-01-11T15:56:23.107" Title="Adding matrix as a confounder in EnrichmentBrowser" Tags="|r|differential-expression|deseq2|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="15174" PostTypeId="1" CreationDate="2021-01-08T16:38:02.250" Score="1" ViewCount="213" Body="&lt;p&gt;I am trying to update the gene symbols provided by the &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/downloads/latest&quot; rel=&quot;nofollow noreferrer&quot;&gt;10x human precompiled reference (2020-A)&lt;/a&gt;, using the mapping Ensembl ID to gene symbol provided in release 102.&lt;/p&gt;&#xA;&lt;p&gt;However, I found out that 56 of the Ensembl IDs (which I presumed to be stable) are not present in the latest release. See the attached image.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jQIXV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jQIXV.png&quot; alt=&quot;table of missing ensembl ids&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;See, for example, gene OCLM (ENSG00000262180) which seems to have a retired identifier:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FAsjs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FAsjs.png&quot; alt=&quot;OCLM&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to automatically filter or correct these identifiers, or should it be done manually? What is the most correct approach to take in this case (e.g., ignore them)?&lt;/p&gt;&#xA;&lt;h3&gt;Update&lt;/h3&gt;&#xA;&lt;p&gt;All but one are associated with deprecated identifiers, e.g., &lt;a href=&quot;https://uswest.ensembl.org/Multi/Search/Results?q=ENSG00000262180&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENSG00000262180&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The only one with 3 currently mapped identifiers is &lt;a href=&quot;https://uswest.ensembl.org/Homo_sapiens/Gene/Idhistory?g=ENSG00000183791&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENSG00000183791&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Is it correct at this point to filter them out because associated with deprecated identifiers? What about the one with currently mapped identifiers?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2021-01-08T17:26:19.113" LastActivityDate="2021-01-08T17:26:19.113" Title="10x Single-cell RNA-Seq - Updating gene symbol with latest ensembl release - Ensembl IDs missing" Tags="|scrnaseq|gene|ensembl|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15175" PostTypeId="2" ParentId="15163" CreationDate="2021-01-08T21:44:35.443" Score="2" Body="&lt;h2&gt;Per Atom&lt;/h2&gt;&#xA;&lt;p&gt;You are correct. Rosetta scorefunction does not store any per atom data. The scoring operates at the per residue level. Whereas each atom has its coordinates and properties in full atom mode, in Pyrosetta it is clear that an atom is a just part of a residue and every operation is applied at the residue level. It's a team effort: the functional group of the side chain works as one and in most cases two atoms have the same role (e.g. &lt;code&gt;OE1&lt;/code&gt; and &lt;code&gt;OE2&lt;/code&gt; in glutamate behave identically).&lt;/p&gt;&#xA;&lt;p&gt;So, the real question is whether you really care about the atom level in a classical mechanical setting?&lt;/p&gt;&#xA;&lt;h2&gt;Formal charges&lt;/h2&gt;&#xA;&lt;p&gt;A wee parenthesis, we need to make a difference between formal charge and Marsilli-Gasteiger partial charge —one is an integer and the other is a float. Both require a stated protonation state in most applications —say &lt;code&gt;aspartate&lt;/code&gt; vs. &lt;code&gt;aspartic acid&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;PDB2PQR&lt;/code&gt; formats to a PQR file for the ligand docking application &lt;code&gt;autodock&lt;/code&gt;. It does not assign how a residue is protonated.&#xA;In the case of Rosetta, the partial charge depends on the 4 column of the topology (params file) of the amino acid, i.e. it is fixed for all amino acids —Rosetta does not implement a Drude particle to make something polarisable. I very strongly suspect this is the same in PDB2PQR.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, something like the Advance Poisson-Boltzmann Solver may give more interesting data in terms of partial charges, but also requires the uses to specify the protonation state.&lt;/p&gt;&#xA;&lt;p&gt;A small parenthesis is histidine, which can adopt four forms. Two tautomers at neutral pH: single proton on either the epsilon (&lt;code&gt;HIE&lt;/code&gt; in Amber) or delta nitrogen (&lt;code&gt;HID&lt;/code&gt;), while it can be deprotonated (anion) or diprotonated (&lt;code&gt;HID&lt;/code&gt;, cation). Rosetta switches between these two automatically.&lt;/p&gt;&#xA;&lt;h2&gt;Properties&lt;/h2&gt;&#xA;&lt;p&gt;You list a few properties of interest, which all seem to be based on the residue a priori of structure.&lt;/p&gt;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;hydrophobicity, unless you have a polarisable model (Drude particle) this is structure independent. RSA, relative surface area, is a must better indication (per residue), which then can get classified as core or surface based on this. Per atom would require just a wee tinker.&lt;/li&gt;&#xA;&lt;li&gt;aromaticity, this is also structure independent, but π-π stacking or π-cation interactions are noted at the residue level.&lt;/li&gt;&#xA;&lt;li&gt;hydrogen bond acceptor/donor. Whether an atom is a donor or acceptor is actually defined in the atom type used in the topology file. Whether that atom fulfils that bonding is a different matter. If you really really wanted to get per atom values of this, in Rosetta you could exclude the backbone atoms you could mutate to alanine without minimising and see if it's maintained. So differentiating between backbone and sidechain is fine, but per atom is a different matter.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;A structure-dependent property is how flexible the residue is, as measured as an RMS. This can be calculated per atom. This is akin to B-factors, which is how big is the &amp;quot;blur&amp;quot; of the atom. However, if you get the &lt;code&gt;NZ&lt;/code&gt; atom on a lysine that is switching between two or more positions you get a partial occupancy. If this atom is basically hand-waving along an axis in an arc you get an anisotropic B-factor. In other words it does not vibrate in all 3 cartesian axes equally. So a B-factor is a terrible approximation. An RSMD of an ensemble is even worse.&#xA;In Rosetta, you can a quick vibration using Backrub, which is very quick at telling you how much motion does an atom experience.&lt;/p&gt;&#xA;&lt;h2&gt;Here be dragons&lt;/h2&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Warning: As mentioned, it takes two atoms to resulting in a force, hence why per atom scores are a bad idea.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However if for some &lt;em&gt;exceptional&lt;/em&gt; reason this is required, say&#xA;figuring out which group on a ligand is causing a clash, in pyrosetta it is possible:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;import pyrosetta&#xA;pyrosetta.init()&#xA;pose = pyrosetta.pose_from_sequence(&amp;quot;ELVISISALIVE&amp;quot;)&#xA;target_resi = 9&#xA;residue = pose.residue(target_resi)&#xA;scores = []&#xA;for i in range(1, residue.natoms() + 1):&#xA;    iname = residue.atom_name(i)&#xA;    for r in range(1, pose.total_residue() + 1):&#xA;        other = pose.residue(r)&#xA;        for o in range(1, other.natoms() + 1):&#xA;            oname = other.atom_name(o)&#xA;            score = pyrosetta.toolbox.atom_pair_energy.etable_atom_pair_energies(residue, &#xA;                                                                                 i,&#xA;                                                                                 other,&#xA;                                                                                 o, &#xA;                                                                                 scorefxn)&#xA;        scores.append(dict(zip(['residue_index_A', 'atom_name_A', 'residue_index_A', 'atom_name_B', 'lj_atr', 'lj_rep', 'fa_solv', 'fa_elec'],&#xA;                         [target_resi, residue.atom_name(i), r, other.atom_name(o), *score])))&#xA;&#xA;df = pd.DataFrame(scores)&#xA;df.pivot_table(index='atom_name_A',&#xA;               aggfunc='sum')[['lj_atr','lj_rep','fa_elec','fa_solv']]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2021-02-09T11:11:43.587" LastActivityDate="2021-02-09T11:11:43.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15176" PostTypeId="1" AcceptedAnswerId="15185" CreationDate="2021-01-09T12:15:45.847" Score="0" ViewCount="116" Body="&lt;p&gt;I am one ignorant...&lt;/p&gt;&#xA;&lt;p&gt;Why GAG protein HIV-1 (virus) is in Staphylococcus sp. SS21 and in Klebsiella quasipneumoniae (Bacteria)?&#xA;These are the links in GenBank:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Staphylococcus sp. SS21&#xA;LOCUS       WP_193573829             503 aa            linear   BCT 29-OCT-2020&#xA;DEFINITION  hypothetical protein [Staphylococcus sp. SS21].&#xA;&#xA;HIV-1&#xA;LOCUS       KU749412                9188 bp    RNA     linear   VRL 08-DEC-2016&#xA;DEFINITION  HIV-1 isolate DEMC14PK009 from Pakistan, partial genome.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;protein align:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mgarasllsggqldrwerirlrpggkkkyrlkhivwasrelerfavnpgllessegcrkileqlqpslqtgseelrslyntvavlycvqqridvkdtkealekieeeqnkskkkaqqtaadkgnnspvsqnfpivqnlqgqmvhqpisprtlnawvkvveekafspevipmfsalsegatpqdlntmlntvgghqaamqmlketineeaaewdrlhpvhagpvapgqmreprgsdiagttstlqeqigwmtnnpaipvgeiykrwiilglnkivrmysptsildirqgpkepfrdyvdrfyktlraeqasqdvknwmtetllvqnanpdcktilkalgpaatleemmtacqgvggpshkarilaeamsqvtqptnilmqrgnfrtprktvkcfncgkeghiakncraprkkgcwkcgkeghqmkdcserqanflgkiwpshkgrpgnflqnrpeptappappeesfrfgegtttpaqkqepidkdlyplaslkslfgndpssq&#xA;|||||| | |  || |||||||||||| | ||| ||||||| ||| ||||||  |||  |  |||| |||| |||||| |||| ||||   | |||||||| ||||||||   || |  |        | | ||||||||||||| |||||||||||| ||||||||||||| ||||||||||||||||||||||||||||| |||||||||||||||||||||||||||||||||||||||| ||| | | ||  ||||||||||||||||||||||| |||||||||||||||||||| |||||||| |||||||| ||||||||||||||| |||| |||||||||||||||| |||| ||||||| |      ||| ||     | |||||||||| | ||||||||||||||||||||||| ||||||||||||||||||||||| |||||||||   ||||| | || || |||  ||   || ||||||| || ||         &#xA;MGARASILRGEKLDTWERIRLRPGGKKHYMLKHLVWASRELDRFALNPGLLETLEGCKQIIKQLQPALQTGTEELRSLFNTVATLYCVHAGIPVKDTKEALDKIEEEQNKCQQKAKQ--AEAAAAGKISLNYPIVQNLQGQMVHQAISPRTLNAWVKVIEEKAFSPEVIPMFTALSEGATPQDLNTMLNTVGGHQAAMQMLKDTINEEAAEWDRLHPVHAGPVAPGQMREPRGSDIAGTTSTLAEQIAWITGNPPVPVGEIYKRWIILGLNKIVRMYSPXSILDIRQGPKEPFRDYVDRFFKTLRAEQATQDVKNWMTDTLLVQNANPDCKTILRALGPGATLEEMMTACQGVGGPGHKARVLAEAMSQTT--NAIMMQRSNFKGPKRTIKCFNCGKEGHLARNCRAPRKKGCWKCGKEGHQMKDCNERQANFLGKIWPSHKGRPGNFLQSRPEPTAPPA---ESFRF-EETT-PAPKQEQKDK--EPLTSLKSLFGSDPLSQ&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The links in GenBank of Klebsiella quasipneumoniae :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;LOCUS       WP_199660587             458 aa            linear   BCT 06-JAN-2021&#xA;DEFINITION  hypothetical protein, partial [Klebsiella quasipneumoniae].&#xA;ACCESSION   WP_199660587&#xA;VERSION     WP_199660587.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;protein align:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-----------------------grkkyk-lkhlvwasrelerfalnpgllesaegcqqimeqlqsalgtgseelkslfntvatlwcvhqrieikdtkealdkveevkskqkaqqaqrtqqaaaatgsssqnypivqnaqgqmvhqsmsprtlnawvkvveekafspevipmftalsegatpqdlnmmlnivgghqaamqmlkdtineeaaewdrthpvhagpvapgqireprgsdiagttstlqeqigwmtnnppipvgeiykrwivlglnkivrmyspvsildirqgpkepfrdyvdrffktlraeqatqevknwmtesllvqnanpdcktilrglgpgatleemmtacqgvggpghkarvlaeamsqaqqsnimmqrgnfrg-qrtikcfncgkeghiarnckaprkrgcwkcgkeghqmkdcnerqanflgriwpshkgrpgnfpqsrpeptappaesf-geevtpsprqe-----pvd--kel-----ypp&#xA;                       | ||   ||||||||||| ||||||||||  ||| ||  ||| || || ||| ||||||||| |||  |  ||||||||  ||   |   | |   |  ||| |  | ||||||| |||||||  ||||||||||| |||||||||||||||||||||||||| ||| |||||||||||||||||||||||| |||||||||||| ||||||||||||||| ||| | | ||| |||||||||| |||||||||||| ||||||||||||||||||||||||||||||||||||||| ||||||||||||||| |||||||||||||||||||||||||||||||||     ||||| || |  ||||||||||||| |||| |||| |||||||||||||||||||||||| |||||||||||| ||||||||||||||  || || | ||     |    | |   &#xA;MGARASILRGEKLDTWERIRLRPGGKKHYMLKHLVWASRELDRFALNPGLLETLEGCKQIIKQLQPALQTGTEELRSLFNTVATLYCVHAGIPVKDTKEALDKIEEEQNK-CQQKAK--QAEAAAAGKISLNYPIVQNLQGQMVHQAISPRTLNAWVKVIEEKAFSPEVIPMFTALSEGATPQDLNTMLNTVGGHQAAMQMLKDTINEEAAEWDRLHPVHAGPVAPGQMREPRGSDIAGTTSTLAEQIAWITGNPPVPVGEIYKRWIILGLNKIVRMYSPXSILDIRQGPKEPFRDYVDRFFKTLRAEQATQDVKNWMTDTLLVQNANPDCKTILRALGPGATLEEMMTACQGVGGPGHKARVLAEAMSQ-TTNAIMMQRSNFKGPKRTIKCFNCGKEGHLARNCRAPRKKGCWKCGKEGHQMKDCNERQANFLGKIWPSHKGRPGNFLQSRPEPTAPPAESFRFEETTPAPKQEQKDKEPLTSLKSLFGSDPLSQ&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The links in GenBank of hypothetical protein HMV63_24500 [Escherichia coli] and protein align with HIV-1 GAG:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;LOCUS       HAJ5901352               246 aa            linear   BCT 11-MAY-2020&#xA;DEFINITION  TPA: hypothetical protein HMV63_24500 [Escherichia coli].&#xA;ACCESSION   HAJ5901352&#xA;VERSION     HAJ5901352.1&#xA;&#xA;-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------mtsnppipvgdiykrwiilglnkivrmyspvsildirqgpkepfrdyvdrffktlraeqatqdvknwmtetllvqnanpdcktilralgpgasleemmtacqgvggpshkarvlaeamsqannanimmqksnfkgprrnvkcfncgkeghiarncraprkrgcwkcgkeghqmkdcterqanflgkiwpshkgrpgnflqsrpeptappaesfkfeettptprqeprdrepltslkslfgsdplsq&#xA;                                                                                                                                                                                                                                                        | ||| ||| ||||||||||||||||||| |||||||||||||||||||||||||||||||||||||| |||||||||||||||||||||| |||||||||||||| ||||||||||||  || |||| |||||| |  |||||||||||||||||||| ||||||||||||||| |||||||||||||||||||||||||||||||||||| |||||| | ||| | |||||||||||||||||      &#xA;MGARASILRGEKLDTWERIRLRPGGKKHYMLKHLVWASRELDRFALNPGLLETLEGCKQIIKQLQPALQTGTEELRSLFNTVATLYCVHAGIPVKDTKEALDKIEEEQNKCQQKAKQAEAAAAGKISLNYPIVQNLQGQMVHQAISPRTLNAWVKVIEEKAFSPEVIPMFTALSEGATPQDLNTMLNTVGGHQAAMQMLKDTINEEAAEWDRLHPVHAGPVAPGQMREPRGSDIAGTTSTLAEQIAWITGNPPVPVGEIYKRWIILGLNKIVRMYSPXSILDIRQGPKEPFRDYVDRFFKTLRAEQATQDVKNWMTDTLLVQNANPDCKTILRALGPGATLEEMMTACQGVGGPGHKARVLAEAMSQTTNA-IMMQRSNFKGPKRTIKCFNCGKEGHLARNCRAPRKKGCWKCGKEGHQMKDCNERQANFLGKIWPSHKGRPGNFLQSRPEPTAPPAESFRFEETTPAPKQEQKDKEPLTSLKSLFGSDPLSQ&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I remember Blast find result in &lt;strong&gt;plant&lt;/strong&gt; bacteria or virus too&lt;/p&gt;&#xA;" OwnerUserId="7054" LastEditorUserId="7054" LastEditDate="2021-01-09T14:32:11.127" LastActivityDate="2021-01-11T11:44:16.753" Title="Why GAG protein HIV-1 (virus) is in Staphylococcus sp. SS21 and in Klebsiella quasipneumoniae (Bacteria)?" Tags="|sequence-alignment|proteins|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15177" PostTypeId="1" AcceptedAnswerId="17822" CreationDate="2021-01-09T13:26:57.707" Score="3" ViewCount="130" Body="&lt;p&gt;I have got a quite large VCF file (about 20k samples with 160k SNPs after filtering for quality etc.) and I would like to get a phylogeny for it. However, the whole dataset's too large for my computational resources (unless I go for fasttree which would be my last resort). Therefore, I need to subset the SNPs and there's plenty of possible ways to do this (dump all rare SNPs with MAF below a threshold, only use SNPs in coding regions, only use missense SNPs; just to name a few).&lt;/p&gt;&#xA;&lt;p&gt;Is there a consensus (or at least some comparative work or recent review; I found nothing) on what method works best and gives the tree that's closest to using the full dataset?&lt;/p&gt;&#xA;" OwnerUserId="7189" LastEditorUserId="7189" LastEditDate="2021-01-09T15:05:24.377" LastActivityDate="2021-10-10T13:23:14.887" Title="Consensus on how to select SNPs for creating a phylogeny when the full dataset is too large" Tags="|phylogenetics|vcf|phylogeny|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15178" PostTypeId="2" ParentId="15176" CreationDate="2021-01-09T14:37:31.733" Score="2" Body="&lt;p&gt;The interesting numbers of an alignment are coverage (91%) and identity (82.68%) of API70200.1 (the protein encoded in KU749412) against WP_199660587.&lt;/p&gt;&#xA;&lt;p&gt;There are two cases:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;technical error: samples were contaminated, which happens not infrequently.&lt;/li&gt;&#xA;&lt;li&gt;biology did something cool: horizontal gene transfer&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The former is likely to be present when you have a single sequence where this gene is present and it is not from a complete genome.&lt;/p&gt;&#xA;&lt;p&gt;The latter, unless it's an ancient transfer, would likely result in flanking regions and genes.&lt;/p&gt;&#xA;&lt;p&gt;So blasting the gene to the Klebsiella quasipneumoniae proteome, shows there are three hits (two poor). There is no strain name in list, whereas there are three complete genomes of three strains. So the sequence WP_199660587 is a one off and not from a complete genome. Therefore, the clues strongly point towards it being a technical error.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gLyBE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gLyBE.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-01-09T14:37:31.733" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15179" PostTypeId="1" CreationDate="2021-01-09T16:25:04.417" Score="0" ViewCount="86" Body="&lt;p&gt;I m not sure if I'm going the right way or not but when the gene and region and both upregulated in RNA seq and ATAC seq data I should be seeing  a positive correlation but it seems other wise.&lt;/p&gt;&#xA;&lt;p&gt;Am i doing it in a wrong way or this can occur?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DjMut.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DjMut.png&quot; alt=&quot;plot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Adding the scatter with points&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rHEfI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rHEfI.png&quot; alt=&quot;scatter &quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Updated Plot&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;But the issue is with the R^2 value it is very low&#xA;&lt;a href=&quot;https://i.stack.imgur.com/YTmXX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YTmXX.png&quot; alt=&quot;updated plot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2021-01-11T07:27:29.017" LastActivityDate="2021-01-11T07:27:29.017" Title="ATAC and RNA seq Up regulated region gene and scatter" Tags="|rna-seq|atac-seq|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="15181" PostTypeId="2" ParentId="9015" CreationDate="2021-01-09T21:21:33.833" Score="3" Body="&lt;p&gt;As mentioned by OP, another option is to use &lt;a href=&quot;https://daler.github.io/pybedtools/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pybedtools&lt;/a&gt;, which in my opinion is pretty convenient for people already familiar with BedTools.&lt;/p&gt;&#xA;&lt;p&gt;Let's even say df1's format is slightly different than df2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;dict1 = {0: ['chr1', 'chr1', 'chr1', 'chr1', 'chr2'], 1: [1, 100, 150, 900, 1], &#xA;         2: [100, 200, 500, 950, 100], &#xA;         3: ['feature1', 'feature2', 'feature3', 'feature4', 'feature4'], &#xA;         4: [0, 0, 0, 0, 0], 5: ['+', '+', '-', '+', '+'], &#xA;         6: [&amp;quot;remember&amp;quot;, &amp;quot;the&amp;quot;, &amp;quot;5th&amp;quot;, &amp;quot;of&amp;quot;, &amp;quot;november&amp;quot;]}&#xA;&#xA;df1 = pd.DataFrame(dict1)&#xA;&#xA;print(df1)&#xA;&#xA;      0    1    2         3  4  5         6&#xA;0  chr1    1  100  feature1  0  +  remember&#xA;1  chr1  100  200  feature2  0  +       the&#xA;2  chr1  150  500  feature3  0  -       5th&#xA;3  chr1  900  950  feature4  0  +        of&#xA;4  chr2    1  100  feature4  0  +  November&#xA;&#xA;dict2 = {0: ['chr1', 'chr1'], 1: [155, 800], 2: [200, 901], &#xA;         3: ['feature5', 'feature6'], 4: [0, 0], 5: ['-', '+']}&#xA;&#xA;df2 = pd.DataFrame(dict2)&#xA;&#xA;print(df2)&#xA;&#xA;      0    1    2         3  4  5&#xA;0  chr1  155  200  feature5  0  -&#xA;1  chr1  800  901  feature6  0  +&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Using pybedtools we get:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from pybedtools import BedTool&#xA;&#xA;bed1 = BedTool.from_dataframe(df1)&#xA;bed2 = BedTool.from_dataframe(df2)&#xA;&#xA;intersected_bed = bed1.intersect(bed2)&#xA;&#xA;print(intersected_bed)&#xA;&#xA;chr1    155 200 feature2    0   +   the&#xA;chr1    155 200 feature3    0   -   5th&#xA;chr1    900 901 feature4    0   +   of&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We can then keep using the intersected BedTool object or convert it back into a DataFrame:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# create a pandas.DataFrame, passing args and kwargs to pandas.read_table&#xA;intersected_df = intersected_bed.to_dataframe(header=None, &#xA;                                              names=[&amp;quot;chrom&amp;quot;, &amp;quot;start&amp;quot;, &amp;quot;end&amp;quot;, &amp;quot;name&amp;quot;, &#xA;                                                     &amp;quot;score&amp;quot;, &amp;quot;strand&amp;quot;, &amp;quot;whatever&amp;quot;])&#xA;&#xA;print(intersected_df)&#xA;&#xA;  chrom  start  end      name  score strand whatever&#xA;0  chr1    155  200  feature2      0      +      the&#xA;1  chr1    155  200  feature3      0      -      5th&#xA;2  chr1    900  901  feature4      0      +       of&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2570" LastActivityDate="2021-01-09T21:21:33.833" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15182" PostTypeId="2" ParentId="15177" CreationDate="2021-01-09T21:43:06.877" Score="1" Body="&lt;p&gt;The subset that gives you the tree closest to the one you obtain with the full dataset does not necessarily is the tree nearest to reality... It could be that you get a more realistic result using only partial data for multiple reasons. Your best chance is in my opinion, as it usually is, making lots of tests and contrasting hypotheses.&lt;/p&gt;&#xA;" OwnerUserId="5190" LastEditorUserId="5190" LastEditDate="2021-01-09T21:51:27.587" LastActivityDate="2021-01-09T21:51:27.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15183" PostTypeId="1" CreationDate="2021-01-10T10:52:43.400" Score="2" ViewCount="398" Body="&lt;p&gt;I would like to create a list (probably .vcf format would be good) of all known pathogenic missense mutations in a human gene of interest and then add other variants that could lead to the same pathogenic amino acid substitutions. I intend to search targeted sequencing data of my gene of interest for variants that would cause these particular substitions.&lt;/p&gt;&#xA;&lt;p&gt;I'm very new to bioinformatics so I'm not so familiar with the appropriate databases and tools to use, but this is a rough idea of how it could be done:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;extract all mutations listed in Ensembl within my gene of interest that are labelled with &lt;code&gt;HGMD_MUTATION&lt;/code&gt;  (is there an option to download a vcf file with all HGMD mutations within my gene? I've only managed to do so for individual mutations.)&lt;/li&gt;&#xA;&lt;li&gt;annotate list with reference and mutant amino acids for every mutation in the list&lt;/li&gt;&#xA;&lt;li&gt;add additional mutations (which may not be present in HGMD) which could lead to the same substitution. E.g. I'm interested in the substitution Proline (reference codon: CCG) to Leucine (TTR or CTN) at AA position 102, so I would like to list all hypothetical mutations that would create a leucine codon in this position, not only the one given in Ensembl (which is CCG -&amp;gt; CTG).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Could you point me towards the best tools to use for these purposes? Unfortunately I haven't got a HGMD subscription. Many thanks!&lt;/p&gt;&#xA;" OwnerUserId="9025" LastActivityDate="2021-10-11T09:31:16.143" Title="How do I create a VCF file of all known pathogenic mutations in a gene of interest?" Tags="|vcf|genomics|ensembl|mutations|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15184" PostTypeId="1" CreationDate="2021-01-10T21:08:47.217" Score="0" ViewCount="42" Body="&lt;p&gt;I am converting scores into frequencies (from nanopolish results) using &lt;a href=&quot;https://nanopolish.readthedocs.io/en/latest/quickstart_call_methylation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nanopore scripts&lt;/a&gt;.&#xA;The script filters out lots of rows on the premise that reads where &lt;code&gt;LLR &amp;lt; T * N&lt;/code&gt; are ambiguous.&lt;/p&gt;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;LLR = log-likelihood ratio (log-likelihood methylated / log-likelihood unmethylated)  &#xA;T   = arbitrary threshold. Above that value the C is methylated.  &#xA;N   = number of Cs (CpG) in the fragment analyzed.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What? Why? Where?&lt;/p&gt;&#xA;" OwnerUserId="2970" LastEditorUserId="2108" LastEditDate="2021-01-11T19:07:23.883" LastActivityDate="2021-01-11T19:44:10.537" Title="ambiguous calls in nanopolish" Tags="|nanopore|methylation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15185" PostTypeId="2" ParentId="15176" CreationDate="2021-01-11T08:48:22.877" Score="-3" Body="&lt;p&gt;for me could be one lab escape...&lt;/p&gt;&#xA;&lt;p&gt;it seems the page is for me this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  https://science.sciencemag.org/content/266/5193/1981&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;there is a pay wall so I can not read that...&#xA;The title is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Crystal structure of the catalytic domain &#xA;of HIV-1 integrase: &#xA;similarity to other polynucleotidyl transferases&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;i know nothing of argument, but it seems someone unknow for unknow reason infect one bacteria of Klebsiella pneumonia with the virus HIV-1. Someone know what happen if one common bacteria infect with the HIV-1 is diffused? what happen if there is something activate virus replication in the bacteria when the bacteria is in some human?&lt;/p&gt;&#xA;&lt;p&gt;the genbank file would be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;LOCUS WP_142766724 415 aa linear BCT 07-JAN-2020&#xA;DEFINITION DDE-type integrase/transposase/recombinase, partial&#xA; [Klebsiella pneumoniae].&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The general speech&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;We, You don't have the right (or the intelligence) to change &#xA;one iota of any DNA or RNA. Genetic material, if we still &#xA;want to live a few years, must always be read-only, &#xA;never re-written from us because consequences are unknows.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7054" LastEditorUserId="7054" LastEditDate="2021-01-11T11:44:16.753" LastActivityDate="2021-01-11T11:44:16.753" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15186" PostTypeId="1" CreationDate="2021-01-11T15:49:12.743" Score="1" ViewCount="30" Body="&lt;p&gt;I'm currently working on TCGA miRNA dataset. After constructing a reads matrix, I'm trying to find the isoform sequence. In my data, I have the genomic location (isoform_coords). I found that entries with different accession ID (MIMAT) were mapped to the same location.&#xA;I looked at the source files to make sure there were no errors during merging.&#xA;The data frame looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;miRNA_ID        isoform_coords                      miRNA_region&#xA;hsa-mir-143     hg38:chr5:149428962-149428983:+     mature,MIMAT0000435&#xA;hsa-mir-143     hg38:chr5:149428962-149428983:+     mature,MIMAT0004599&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I searched miRBase using the MIMAT ID and found that MIMAT0000435 and MIMAT0004599 are on different arms (5p and 3p) of hsa-mir-143:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;hsa-miR-143-3p MIMAT0000435&#xA;hsa-miR-143-5p MIMAT0004599&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This seems strange since there should be some distance between the 5p and 3p sequences.&#xA;Is there an explanation for this?&lt;/p&gt;&#xA;&lt;p&gt;I need to get the sequence for each entry. I can do this using the Ensembl API. Is there a way I could perhaps do some mapping to get the 5p and 3p arms (so far I just did a manual search but I think there should be a more credible way to do this)? I would like to later analyse the expression patterns of the isoforms in cancer tissue so it would be useful to have isoform IDs as specific as possible.&lt;/p&gt;&#xA;" OwnerUserId="10267" LastActivityDate="2021-01-11T15:49:12.743" Title="TCGA dataset: different accession IDs mapped to same location?" Tags="|rna-seq|sequence-alignment|data-preprocessing|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15187" PostTypeId="2" ParentId="15184" CreationDate="2021-01-11T19:44:10.537" Score="1" Body="&lt;p&gt;Assuming that it is the postprocessing script &lt;code&gt;calculate_methylation_frequency.py&lt;/code&gt; that you are using, user Wouter de Coster's comment is correct that the methylation event is filtered out at the frequency estimation stage if it has a low LLR. It will have  a low LLR if it is considered not confident enough to report. This means not only &lt;code&gt;LLR &amp;gt; T * N&lt;/code&gt;, but also the statistical threshold &lt;code&gt;LLR &amp;gt; 0&lt;/code&gt; (see below).&lt;/p&gt;&#xA;&lt;p&gt;Specifically, here is the code in the &lt;a href=&quot;https://github.com/jts/nanopolish/blob/master/scripts/calculate_methylation_frequency.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;script&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;def update_call_stats(key, num_called_cpg_sites, is_methylated, sequence):&#xA;    if key not in sites:&#xA;        sites[key] = SiteStats(num_called_cpg_sites, sequence)&#xA;&#xA;    sites[key].num_reads += 1&#xA;    sites[key].called_sites += num_called_cpg_sites&#xA;    if is_methylated &amp;gt; 0:&#xA;        sites[key].called_sites_methylated += num_called_cpg_sites&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That is the function that decides whether to add a site to output, which depends on &lt;code&gt;is_methylated&lt;/code&gt;, which is set here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    # [skip several lines of code]&#xA;        num_sites = int(record['num_motifs'])&#xA;        llr = float(record['log_lik_ratio'])&#xA;&#xA;        # Skip ambiguous call&#xA;        if abs(llr) &amp;lt; args.call_threshold * num_sites:&#xA;            # [MP NOTE: SKIP if LLR &amp;lt; T * N]&#xA;            continue&#xA;        sequence = record['sequence']&#xA;&#xA;        # [MP NOTE: SKIP IF LLR &amp;lt;= 0]&#xA;        is_methylated = llr &amp;gt; 0&#xA;&#xA;        # [skip several lines of code]&#xA;&#xA;        update_call_stats(key, num_sites, is_methylated, sequence) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It's a little messy to read because the &lt;code&gt;sites&lt;/code&gt; dictionary is a global outside of &lt;code&gt;update_call_stats()&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So overall, it seems like &lt;strong&gt;if you want sites included that are below this threshold, you will have to either:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;use an earlier stage of processing (output of &lt;code&gt;nanopolish call-methylation&lt;/code&gt; I believe according to the docs) or&lt;/li&gt;&#xA;&lt;li&gt;alter the script to change the heuristic thresholds.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-01-11T19:44:10.537" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15189" PostTypeId="1" CreationDate="2021-01-12T09:12:23.060" Score="1" ViewCount="155" Body="&lt;p&gt;I'm currently trying to learn RNA-seq data analysis and differential expression in R. I've been using the &lt;a href=&quot;https://combine-australia.github.io/RNAseq-R/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Combine lab tutorial&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Everything was running well until the tutorial asked to use the &lt;code&gt;propmapped&lt;/code&gt; function (on &lt;a href=&quot;http://combine-australia.github.io/RNAseq-R/07-rnaseq-day2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt;) on my BAM files and then view the promapped proportion for each sample. In this tutorial we should be getting promapped proportion ranging from 0.05 to 0.150, however in my case I have no reads that align to my mapped genome (promapped = 0 in my case, for each sample). I don't know why as I've been using the data they provided for this tutorial.&lt;/p&gt;&#xA;&lt;p&gt;I was wondering if someone could follow the first few steps of this tutorial until the propmapped step and tell me if they eventually get the right proportion of reads or not after the propmapped function.&#xA;This could tell me if the tutorial is the culprit or not.&lt;/p&gt;&#xA;" OwnerUserId="10270" LastEditorUserId="57" LastEditDate="2021-01-13T12:39:32.017" LastActivityDate="2021-06-12T14:01:00.120" Title="RNA_Seq Analysis in R, propmapped function issue" Tags="|r|rna-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15190" PostTypeId="1" AcceptedAnswerId="15196" CreationDate="2021-01-12T12:55:41.530" Score="0" ViewCount="50" Body="&lt;p&gt;In this &lt;a href=&quot;https://www.ahajournals.org/doi/10.1161/CIRCHEARTFAILURE.120.006926&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; they classify the promoters into two categories, highlighted in italics below.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Two Types of Promoters&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Mammalian promoters can initiate transcription across broad or narrow&#xA;genomic regions, and these promoter shapes, broad or narrow (sharp),&#xA;correlate with distinct transcriptional regulatory mechanisms.33 We&#xA;evaluated cardiac promoters predicted from CAGE clusters for these 2&#xA;major types of promoters by calculating the interquartile range of&#xA;promoter CAGE clusters by determining the base pair distance between&#xA;10% and 90% of a promoter’s total signal. &lt;em&gt;We observed the expected 2&#xA;distinct populations, defined as sharp (interquartile range &amp;lt;10 bp)&#xA;and broad promoters (interquartile range ≥10 bp; Figure 2A). Broad&#xA;promoters were those associated many different cellular functions,&#xA;including housekeeping functions. In contrast, genes with sharp&#xA;(narrow) promoters were those encoding proteins critical for&#xA;tissue-specific functions seen by the presence myofilament, muscle&#xA;tissue, and muscle contractions genes (Figure 2B). Thus,&#xA;tissue-specific genes important for LV specification and function were&#xA;more likely to have sharp promoters.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Now My question is can I do this only based on ATAC seq data. If yes how do i get that information of narrow and broad promoters.&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="235" LastEditDate="2021-01-13T00:21:48.793" LastActivityDate="2021-01-13T00:29:12.837" Title="Getting two class of promoters : ATAC seq data" Tags="|atac-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15191" PostTypeId="1" AcceptedAnswerId="15213" CreationDate="2021-01-12T13:22:35.510" Score="0" ViewCount="249" Body="&lt;p&gt;&lt;strong&gt;Background:&lt;/strong&gt;  I am developing a pipeline for metagenomic studies of human gut microbiote. In particular, I am mapping the reads data originating from shotgun whole genome sequencing onto a gene catalogue (similar to IGC) and counting reads mapped on each feature/gene. Till now I have been using &lt;a href=&quot;https://ngless.embl.de/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;NGLess&lt;/a&gt;, which is a rather convenient tool. However, we have some questions about how exactly it counts reads and it might be not the best option for scaling up our pipeline.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&lt;br /&gt;&#xA;I am looking for alternative tools for gene count. A brief search has shown that &lt;code&gt;htseq-count&lt;/code&gt; and &lt;code&gt;featureCount&lt;/code&gt; are two frequently used tools. However they mainly appear in RNAseq context, and I am not certain whether they could be used in my case. I would also appreciate comments on the possibility of using &lt;code&gt;samtools mpileup&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1813" LastActivityDate="2021-01-14T19:22:35.820" Title="Reads count in metagenomics" Tags="|read-mapping|metagenome|featurecounts|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15192" PostTypeId="2" ParentId="15183" CreationDate="2021-01-12T13:31:39.163" Score="1" Body="&lt;p&gt;I'm not sure how to generate the additional mutations, but I would say that HGMD is not the way to find all the pathogenic variants. I would probably filter &lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Gene/Variation_Gene/Table?db=core;g=ENSG00000108055;r=10:110567695-110606048&quot; rel=&quot;nofollow noreferrer&quot;&gt;this table&lt;/a&gt; by either Clinical significance or Evidence-&amp;gt;Phenotype association.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2021-01-12T13:31:39.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15194" PostTypeId="1" CreationDate="2021-01-12T17:38:56.663" Score="2" ViewCount="43" Body="&lt;p&gt;Let's say 50 samples were sequenced at 25x. After a while, the group realizes that the experiment is under-powered and adds 100 samples more, but at a coverage of 5x. I'm talking about WGS.&lt;/p&gt;&#xA;&lt;p&gt;The scientist runs the samples following GATK's best practices until he/she is done running haplotypeCaller on them.&lt;/p&gt;&#xA;&lt;p&gt;Then the question is how to proceed:&lt;/p&gt;&#xA;&lt;p&gt;a) combine all 150 gVCFs and do joint calling. Then do site filtering followed by genotype filtering.&lt;/p&gt;&#xA;&lt;p&gt;b) Do joint calling with the high coverage samples and in parallele do joint calling with the low coverage samples. Then do site filtering, merge both VCFs and filter by genotype.&lt;/p&gt;&#xA;&lt;p&gt;c) combine all 150 gVCFs and do joint calling. Split VCF into two according to coverage and do site filtering. Merge both VCFs and filter by genotype.&lt;/p&gt;&#xA;&lt;p&gt;Option &amp;quot;a&amp;quot; sticks to GATK's recommendations, but it ignores the high difference in coverage between sample sets. The opposite is the case for option &amp;quot;b&amp;quot;. And option &amp;quot;c&amp;quot; is like a hybrid approach of &amp;quot;a&amp;quot; and &amp;quot;b&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I guess what I want to ask if it is an issue to do joint-calling and then site-filtering when groups of samples have large differences in coverage.&lt;/p&gt;&#xA;" OwnerUserId="10271" LastActivityDate="2021-01-12T17:38:56.663" Title="Joint variant calling of two sample sets with different coverages (~5x vs ~25x)" Tags="|variant-calling|gatk|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15195" PostTypeId="1" CreationDate="2021-01-12T18:29:04.337" Score="1" ViewCount="52" Body="&lt;p&gt;I'm learning to use R in data analysis. I'm getting fluent in baseR and the tidyverse, but thus far only dealt with medium throughput plate-based experiments.&lt;/p&gt;&#xA;&lt;p&gt;I am currently trying to learn how to work with a data set of normalized RNAseq results. I have a matrix consisting of tens of thousands of rows corresponding to genes. The first two columns identify 'gene_ID' and 'gene_name'. The remaining 96 columns are different samples (different treatments, timepoints, replicates, ...).&lt;/p&gt;&#xA;&lt;p&gt;I would like to analyze this dataset to identify differentially expressed gene based on the different conditions, perform some cluster analysis, run GESA on these clusters, and generate heatmaps at various stages of the process.&lt;/p&gt;&#xA;&lt;p&gt;Can anybody recommend the best libraries to look into in order to achieve these goals? I started looking at several tutorials, but much of them deal with the earlier steps of RNAseq data analysis and don't really focus on these later steps once the data is already processed and cleaned up.&lt;/p&gt;&#xA;" OwnerUserId="10272" LastActivityDate="2021-01-12T18:29:04.337" Title="workflow for processed RNAseq data analysis in R" Tags="|r|rna-seq|bioconductor|" AnswerCount="0" CommentCount="4" ClosedDate="2021-02-01T12:13:03.610" ContentLicense="CC BY-SA 4.0" />
  <row Id="15196" PostTypeId="2" ParentId="15190" CreationDate="2021-01-13T00:29:12.837" Score="1" Body="&lt;p&gt;Its highly unlikely that you will be able to categorize promoters on this basis using only ATAC data. This is a consequence of people using the terms &amp;quot;promoters&amp;quot; and &amp;quot;transcription start sites&amp;quot; interchangeably, when they are actaully different things. CAGE data measures transcription start sites which is the specific location where transcription starts. CAGE does not identify promoters, which are the complete region around the start of a gene that contains, at a minimum, all the sequences necessary to assemble the RNA polymerase II pre-initiation complex. Some definitions of promoter are even broader, including all regulatory sequence proximal the the gene start.&lt;/p&gt;&#xA;&lt;p&gt;ATAC identifies regions of open chromatin, and I would expect the open chromatin regions around the starts of genes to be more like the broadest definition of promoter, and therefore to be significantly larger than the transcription starts sites they contain, whether they be broad or narrow TSSs.&lt;/p&gt;&#xA;&lt;p&gt;You could of course look at a distribution of the peak sizes over TSSs and see if you found a bimodal distribution. If you did you could correlate the genes with broad/narrow TSS peaks with their known functions so see if it fitted the idea that narrows TSSs&lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2021-01-13T00:29:12.837" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15197" PostTypeId="1" AcceptedAnswerId="15250" CreationDate="2021-01-13T03:09:05.390" Score="1" ViewCount="769" Body="&lt;p&gt;I have a cromwell task that, among other variables, takes these three variables:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   File Ref             # Reference genome fasta&#xA;   File RefFai          # Reference Genome index&#xA;   File RefDict         # Reference Dictionary&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;While all three files make it into the task's &amp;quot;inputs&amp;quot; directory, they end up in different sub-directories of inputs:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ME $ find inputs&#xA;inputs&#xA;&#xA;inputs/-181926642&#xA;inputs/-181926642/Sample.vcf.bgz.tbi&#xA;inputs/-181926642/Sample.vcf.bgz&#xA;inputs/-1548908437&#xA;inputs/-1548908437/GCA_000001405.15_GRCh38_no_alt_short_headers_nonACTG_to_N.dict&#xA;inputs/44588251&#xA;inputs/44588251/GCA_000001405.15_GRCh38_no_alt_short_headers_nonACTG_to_N.fa&#xA;inputs/44588251/GCA_000001405.15_GRCh38_no_alt_short_headers_nonACTG_to_N.fa.fai&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which means that when I call GATK and pass it the path to the .fa file, it gets rather cranky that the .dict file isn't in the same directory.&lt;/p&gt;&#xA;&lt;p&gt;How do I force cromwell to put all three files in the same directory?&lt;/p&gt;&#xA;" OwnerUserId="9222" LastActivityDate="2023-08-24T15:58:17.290" Title="How do I force Cromwell / WDL to put input files in the same directory?" Tags="|wdl|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15198" PostTypeId="1" AcceptedAnswerId="15242" CreationDate="2021-01-13T10:14:09.993" Score="0" ViewCount="58" Body="&lt;p&gt;I have a microarray gene expression dataset consisting of placenta samples. I want to check whether these placenta samples are mixed with maternal decidua tissue. I have marker genes for decidua samples SCARA5, PIGF, SLPI. How to check whether these genes are expressed in placenta samples? In other words, how to check if a gene is up or down regulated without control group?&lt;/p&gt;&#xA;" OwnerUserId="4971" LastActivityDate="2021-01-21T01:15:16.717" Title="How to check if a given gene is expressed in a group of microarray samples if I do not have control group to compare with?" Tags="|gene-expression|transcriptome|microarray|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="15199" PostTypeId="2" ParentId="8835" CreationDate="2021-01-13T10:21:38.430" Score="0" Body="&lt;p&gt;Another way to reduce dimensionality of your dataset is via principal component analysis or PCA. It is implemented as &lt;code&gt;pcaMethods&lt;/code&gt; R package, for instance.&lt;/p&gt;&#xA;" OwnerUserId="4971" LastActivityDate="2021-01-13T10:21:38.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15200" PostTypeId="1" CreationDate="2021-01-13T12:10:57.977" Score="1" ViewCount="67" Body="&lt;p&gt;when I plotting the volcano plot, I got some dots very top of the Y-axis because their padj nearly reached 0（I checked the data, I think they are reliable）, so I followed the advice to add a very small value to padj(machine$double.xmin), but still got those dots formed a line. I think I heard these lines have a name like Texas arm(?) or Tech arm(?) something before, but I can't remember. Can someone tell me what's the name?&#xA;Please help me. here is the plot :&#xA;&lt;a href=&quot;https://i.stack.imgur.com/x1r6d.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/x1r6d.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9916" LastActivityDate="2021-01-13T12:10:57.977" Title="How to call this : some dots in volcano plot aligned a line after add a very small value to padj" Tags="|differential-expression|ggplot2|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15201" PostTypeId="2" ParentId="15189" CreationDate="2021-01-13T13:00:14.980" Score="1" Body="&lt;p&gt;There must be a problem in the BAM files you are using. Here are steps I would do to figure where exactly is the problem.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Check the data you have downloaded. The tutorial is using 12 fastq.gz files (SRR1552444 - SRR1552455). You can check on sra how big they are supposed to be (For example &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra/?term=SRR1552444&quot; rel=&quot;nofollow noreferrer&quot;&gt;SRR1552444 is supposed to be an ~1.7Gb file&lt;/a&gt;) and if your downloaded files are about the same. Check if the files are fine (In the terminal you can type &lt;code&gt;zcat data/SRR1552444.fastq.gz | head&lt;/code&gt; to check the first few reads). If the files are 0 sized or somehow corrupted, you need to redownload them.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Check if your mapping reference looks alright (size wise, look at the first few lines) and if the reference is indexed (there is a &lt;code&gt;.fai&lt;/code&gt; in the directory).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Check the size of the BAM files, they also need to be in the range of a few hundred Mb to a few Gb. I expect these will be corrupted for sure, otherwise, you would get your numbers. You might also want to check if they are indexed (although this should not be a problem, I suspect they will get indexed automatically). If there is nothing wrong the bamfiles, you might want to check another tool to summarise a bamfile (like &lt;code&gt;samtools flagstat&lt;/code&gt;) to check if at the end the problem really is in &lt;code&gt;propmapped&lt;/code&gt;. If that's the case.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This should at least help you to zoom to the core of your problem if not resolve it.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2021-01-13T13:00:14.980" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15202" PostTypeId="1" CreationDate="2021-01-13T13:03:44.240" Score="0" ViewCount="56" Body="&lt;p&gt;The authors of &lt;a href=&quot;https://www.nature.com/articles/s41586-020-2424-4/&quot; rel=&quot;nofollow noreferrer&quot;&gt;an article&lt;/a&gt; are comparing different cell type and the organ type and they show the ontology enrichment of the RNA-seq:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Cell-type-specific and organ-specific marker genes were identified&#xA;using a two-step procedure. First, we performed pairwise differential&#xA;expression analysis for each of the three type of structural cells,&#xA;comparing them across organs. Second, based on the resulting pairwise&#xA;comparisons, we identified for each organ those genes that were&#xA;upregulated in each cell type compared to at least five other organs.&#xA;Pairwise comparisons between organs were performed using the limma&#xA;package in R, separately for each of the three structural cell types.&#xA;Significantly differential genes were selected based on statistical&#xA;significance (adjusted P &amp;lt; 0.05), average expression (log2(CPM) &amp;gt; −1)&#xA;and sequencing coverage (median number of reads greater than 10 in the&#xA;group with stronger signal). On the basis of these pairwise&#xA;comparisons, we counted the total number of times each gene was&#xA;upregulated in a specific organ compared to all other organs. Genes&#xA;that were upregulated in comparison to five or more other organs were&#xA;selected as marker genes of the corresponding organ.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Now how is this part done ? &lt;strong&gt;&amp;quot;Genes that were upregulated in comparison to five or more other organs were selected as marker genes of the corresponding organ.*&amp;quot;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The figure for this results is this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xTcVD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xTcVD.png&quot; alt=&quot;fig 2c&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="48" LastEditDate="2021-01-20T16:31:07.837" LastActivityDate="2021-01-20T16:31:07.837" Title="Multi Omics comparison RNA seq : Condition specific gene signature" Tags="|rna-seq|figure-explanation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15203" PostTypeId="1" CreationDate="2021-01-13T16:46:20.113" Score="1" ViewCount="973" Body="&lt;p&gt;I have paired end reads and want to extract only those which have this sequence within them:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TGTATGTAAACTTCCGACTTCAACTGTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried using &lt;code&gt;grep -B1 -A2 &amp;quot;TGTATGTAAACTTCCGACTTCAACTGTA&amp;quot; file_1.fq | grep -v &amp;quot;^--$&amp;quot;&lt;/code&gt;&#xA;and the same from the mate pair. After this I tried aligning them , but they dont align because the reads headers are different.&#xA;Can someone help in extracting the mate pair of one read from the other file (2.fq), so that they can be aligned ?&#xA;regards.&lt;/p&gt;&#xA;" OwnerUserId="7236" LastActivityDate="2021-07-25T07:02:27.770" Title="FASTQ file reads extraction paired ends" Tags="|grep|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15205" PostTypeId="1" CreationDate="2021-01-14T01:19:39.067" Score="1" ViewCount="194" Body="&lt;p&gt;I'm trying to modify this &lt;a href=&quot;https://www.biostars.org/p/253774/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/253774/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;To get reads with deletions &amp;gt; 20bp&lt;/p&gt;&#xA;&lt;p&gt;I think this gives reads with exactly 20bp dels:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h in.bam chr1:12340-200000 \ | awk '&lt;span class=&quot;math-container&quot;&gt;$1 ~ &quot;^@&quot; || $&lt;/span&gt;6 ! &amp;quot;20D&amp;quot;' \ | samtools view -b - &amp;gt; out.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;is it possibe to do &amp;gt;?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="7153" LastEditorUserId="77" LastEditDate="2021-01-14T10:55:05.927" LastActivityDate="2021-01-14T21:20:59.850" Title="How to extract reads with INDELs &gt; a given size?" Tags="|sam|samtools|reads|awk|indel|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15206" PostTypeId="1" AcceptedAnswerId="15207" CreationDate="2021-01-14T08:23:32.743" Score="1" ViewCount="1385" Body="&lt;p&gt;&lt;code&gt;samtools idxstats&lt;/code&gt; produces a four column output  (&lt;a href=&quot;http://www.htslib.org/doc/samtools-idxstats.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;see here&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;The output is TAB-delimited with each line consisting of reference sequence name, sequence length, &#xA;# mapped read-segments and # unmapped read-segments. It is written to stdout. Note this may count reads &#xA;multiple times if they are mapped more than once or in multiple fragments. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What is the meaning of &lt;em&gt;unmapped reads&lt;/em&gt; here? Naively, I would take it to mean all the reads that didn't map to a specific feature... however in a sample with millions of reads most of them should be counted as such, which is not the case in my calculations, where the number of unmapped reads is of the same order as the number of mapped reads.&lt;/p&gt;&#xA;&lt;p&gt;I will very much appreciate explanations from people experience with &lt;code&gt;samtools&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1813" LastActivityDate="2021-01-14T10:54:18.877" Title="What is &quot;unmapped read segments&quot; in the output of samtools idxstats?" Tags="|samtools|read-mapping|reads|featurecounts|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15207" PostTypeId="2" ParentId="15206" CreationDate="2021-01-14T10:54:18.877" Score="2" Body="&lt;p&gt;An unmapped read segment is any entry with bit 4 of the flag field set (i.e., it's unmapped). Assuming the reference to which you aligned reasonably matches the sample from which the reads were generated, less than ~10% of to total reads should be unmapped.&lt;/p&gt;&#xA;&lt;p&gt;Please note that BAM files have no idea what a &amp;quot;feature&amp;quot; is. They only contain information about reads and the chromosomes/contigs to which they align.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-01-14T10:54:18.877" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15208" PostTypeId="2" ParentId="15205" CreationDate="2021-01-14T10:58:11.970" Score="2" Body="&lt;p&gt;There's no simple way to do that with awk, since you would need to parse the CIGAR string and iterate over the operations. In python using pysam you would iterate over the reads and filter as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;keep = False&#xA;for OP, LEN in b.cigartuples:&#xA;    if OP == 2 and LEN &amp;gt; 20:&#xA;        keep = True&#xA;        break&#xA;&#xA;if keep:&#xA;    ...do stuff...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2021-01-14T21:20:59.850" LastActivityDate="2021-01-14T21:20:59.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15209" PostTypeId="2" ParentId="15203" CreationDate="2021-01-14T11:02:08.837" Score="3" Body="&lt;p&gt;Your question can be rewritten as, &amp;quot;how do I get the union of reads in two files?&amp;quot; Since you have already subset the mates, the steps are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Process each file, storing the list of reads names in each.&lt;/li&gt;&#xA;&lt;li&gt;Take the union of these.&lt;/li&gt;&#xA;&lt;li&gt;Iterate through the original/non-subset files and write only the reads with names in the union to the output.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The files will be both subset correctly and synced properly such that bowtie2 or any other aligner can correctly process them.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="77" LastEditDate="2021-01-14T13:41:16.293" LastActivityDate="2021-01-14T13:41:16.293" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="15211" PostTypeId="2" ParentId="15202" CreationDate="2021-01-14T11:45:02.037" Score="1" Body="&lt;p&gt;From their methods section:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;On the basis of these lists of marker genes and receptor–ligand pairs, we inferred potential cell–cell interactions for all pairs of one structural cell type and one haematopoietic immune cell type, quantifying the enrichment for known receptor–ligand pairs among all pairs of marker genes between the structural cell type and the haematopoietic cell type (Fig. 2a). First, we counted all pairs of marker genes for each pair of structural and immune cell types. Second, we calculated the fraction of these gene pairs that were annotated as receptor–ligand pairs. Third, we tested whether this fraction was greater than the fraction of annotated receptor–ligand pairs across all pairs of genes. Fisher’s exact test was used to obtain P values and odds ratios, as implemented by the function ‘fisher.test’ in R. P values were adjusted for multiple testing using the Benjamini–Hochberg method, as implemented the function ‘p.adjust’ in R. Finally, significantly enriched pairs (adjusted P &amp;lt; 0.05) of structural and immune cell types were connected by edges to generate a graph of cell–cell interactions.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In addition to testing for significant enrichment (as described in the previous paragraph), we calculated aggregated receptor and ligand gene signatures (Fig. 2c) based on a manually curated list of immune-related receptors and ligands (Supplementary Table 4). These gene signatures were derived by normalizing expression values for each gene and averaging across all genes in a given set of biologically related receptors and ligands.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The actual values plotted then seem to be from EnrichR, since they mention that elsewhere in the methods.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-01-14T11:45:02.037" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15213" PostTypeId="2" ParentId="15191" CreationDate="2021-01-14T19:22:35.820" Score="2" Body="&lt;p&gt;I think that there is a good argument for using technologies adapted from RNA-seq for metagenomic abundance quantification. The tool that I am most familiar with (that I use for this purpose) is &lt;a href=&quot;https://github.com/pachterlab/metakallisto&quot; rel=&quot;nofollow noreferrer&quot;&gt;kallisto&lt;/a&gt;, which uses pseudoalignment instead and is therefore less compute intensive.&lt;/p&gt;&#xA;&lt;p&gt;For some background on the conceptual linkages between metagenomic abundance and transcriptomic abundance you can see the &lt;a href=&quot;https://arxiv.org/abs/1510.07371&quot; rel=&quot;nofollow noreferrer&quot;&gt;kallisto metagenome abundance paper&lt;/a&gt; or the &lt;a href=&quot;https://liorpachter.wordpress.com/2015/10/27/straining-metagenomics/&quot; rel=&quot;nofollow noreferrer&quot;&gt;explanatory blog post&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I think that it is preferable to use a tool that is specifically adapted for metagenomics, as there are always some numerical issues that might have been accounted for by the authors. I think that for this purpose it is maybe best not to naively use tools like &lt;code&gt;mpileup&lt;/code&gt;, though I have no specific objection to it. However, if you look around you can see various papers using e.g. &lt;a href=&quot;https://www.nature.com/articles/s41598-019-50194-0#Sec4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cufflinks and Cuffdiff for effectively the same purpose&lt;/a&gt;, and &lt;a href=&quot;http://cole-trapnell-lab.github.io/cufflinks/manual/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cufflinks&lt;/a&gt; is definitely intended for RNA-seq.&lt;/p&gt;&#xA;&lt;p&gt;Depending on your specific application, you might want to use a different tool like &lt;a href=&quot;https://www.nature.com/articles/s41467-019-08844-4#Sec7&quot; rel=&quot;nofollow noreferrer&quot;&gt;mOTUS2&lt;/a&gt;, if you are interested in estimating abundance of specific genomes or lineages in a metagenome mixture. This method is probably more accurate as it is based on averaging across the whole genome, but it is not very useful if what you are interested in is more e.g. functional profiling of the metagenome.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-01-14T19:22:35.820" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15214" PostTypeId="1" CreationDate="2021-01-15T02:24:31.090" Score="1" ViewCount="121" Body="&lt;p&gt;I am getting this error with MaxQuant 1.6.14 running with mono (5.14.0.177=he1b5a44_1001).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;System.IO.FileNotFoundException: Could not find file &amp;quot;/home/ubuntu/run-dir/combined/combinedRunInfo&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And I not sure why I am getting this error. The process has read and write permissions to that location.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2021-06-14T04:07:40.597" Title="Getting System.IO.FileNotFoundException with MaxQuant running on Linux with mono" Tags="|maxquant|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15215" PostTypeId="2" ParentId="15214" CreationDate="2021-01-15T02:25:31.430" Score="1" Body="&lt;p&gt;...seems the mqpar file was not compatible with the maxquant version.&lt;/p&gt;&#xA;&lt;p&gt;Make sure the &lt;code&gt;mqpar.xml&lt;/code&gt; file shows the same version number as the MaxQuant binary.&#xA;&lt;code&gt;MaxQuant&lt;/code&gt; is not really made for the command line. So, the errors shown are sometimes quite misleading.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2021-01-15T02:25:31.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15216" PostTypeId="2" ParentId="4308" CreationDate="2021-01-15T02:58:41.197" Score="2" Body="&lt;p&gt;Yes, that can be done.&lt;/p&gt;&#xA;&lt;p&gt;A common approach for these types of analysis is to carry out a transformation on the data first in order for it to have a normal distribution (the most general approach called an &lt;a href=&quot;https://en.wikipedia.org/wiki/Inverse_transform_sampling&quot; rel=&quot;nofollow noreferrer&quot;&gt;inverse normal transformation&lt;/a&gt;, among other names), then run the test on the transformed data.&lt;/p&gt;&#xA;&lt;p&gt;As an example, for data that are positive and zero-skewed, a log transformation may be appropriate.&lt;/p&gt;&#xA;&lt;p&gt;The transformation approach does require variation in the data values, though. For a trait that is exactly zero for most values, it might be better to treat it as a binary/categorical trait for the purpose of association testing, rather than a quantitative trait. Most association testing packages should allow associations for binary traits to be tested.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-02-17T23:57:13.547" LastActivityDate="2021-02-17T23:57:13.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15217" PostTypeId="2" ParentId="15203" CreationDate="2021-01-15T03:08:16.860" Score="1" Body="&lt;p&gt;It's a little bit unclear what you're asking, but if you want to repair paired-end files that have become disordered (or the linking has been lost), then the repair function of bbtools should work. This will pair up any coincident read IDs, and create a separate file for any reads that have no pairs in the other file:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://jgi.doe.gov/data-and-tools/bbtools/bb-tools-user-guide/repair-guide/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://jgi.doe.gov/data-and-tools/bbtools/bb-tools-user-guide/repair-guide/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;repair.sh in1=broken1.fq in2=broken2 out1=fixed1.fq out2=fixed2.fq outs=singletons.fq repair&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If grep is used on one file, and repair used on that file in combination with the original file from the matching read, then the relevant read pairs should appear in the output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;repair.sh in1=filtered_1.fq in2=file_2 out1=fixed_1.fq out2=fixed_2.fq outs=unpaired_2.fq repair&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-01-17T01:37:24.090" LastActivityDate="2021-01-17T01:37:24.090" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15218" PostTypeId="2" ParentId="10845" CreationDate="2021-01-15T03:18:26.627" Score="1" Body="&lt;p&gt;I've found that normalisation of variance-stabilised counts from short-read sequencing based on the length of the longest transcript isoform seems to work well. Here's a bit more detail on that from the methods of &lt;a href=&quot;https://doi.org/10.1084/jem.20160470&quot; rel=&quot;nofollow noreferrer&quot;&gt;our paper&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A variance-stabilizing transformation (VST) was applied to count data using the DESeq2 function varianceStabilizingTransformation, producing gene expression values that are adjusted for variance across sample conditions. To compare different genes to each other, this matrix was further transformed by dividing by transcript length in kbp to generate VSTPk values.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;[FWIW, single-molecule long reads don't need any length-normalisation carried out on them]&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-01-15T03:18:26.627" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15219" PostTypeId="1" AcceptedAnswerId="15547" CreationDate="2021-01-15T11:03:43.310" Score="1" ViewCount="1013" Body="&lt;p&gt;Could you please help me to understand what is the difference between Permutational multivariate analysis of variance (PERMANOVA,adonis) and  Variance partitioning&#xA;(permutest).&lt;/p&gt;&#xA;&lt;p&gt;I understood that permutest is used to confirm the PERMANOVA result that differences in PERMANOVA is not due to dispersion.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;###adonis_calculation&#xA;BC &amp;lt;- phyloseq::distance(phyloseq_prop, &amp;quot;bray&amp;quot;)&#xA;pmv =adonis(BC ~Season*Region, as(sample_data(physeqN3), &amp;quot;data.frame&amp;quot;),permutations = 5000)&#xA;pmv&#xA;&#xA;Call:&#xA;adonis(formula = BC ~ Season * Region, data = as(sample_data(physeqN3),      &amp;quot;data.frame&amp;quot;), permutations = 5000) &#xA;&#xA;Permutation: free&#xA;Number of permutations: 5000&#xA;&#xA;Terms added sequenti         &#xA;Total         99   19.8578                 1.00000 ally (first to last)&#xA;&#xA;              Df SumsOfSqs MeanSqs F.Model      R2 Pr(&amp;gt;F)    &#xA;Season         1    1.8586 1.85862  12.036 0.09360  2e-04 ***&#xA;Region         1    2.4218 2.42183  15.682 0.12196  2e-04 ***&#xA;Season:Region  1    0.7522 0.75222   4.871 0.03788  2e-04 ***&#xA;Residuals     96   14.8251 0.15443         0.74656  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Many thanks&lt;/p&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="1008" LastEditDate="2021-01-20T19:04:59.777" LastActivityDate="2021-03-15T09:37:51.027" Title="Confirmation about PERMANOVA and Variance partitioning (permutest)" Tags="|r|statistics|phyloseq|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15220" PostTypeId="1" CreationDate="2021-01-15T13:08:55.417" Score="1" ViewCount="278" Body="&lt;p&gt;I periodically run into problems with Read Groups. I am in a desperate need for a tool that would allow me to fix them.&lt;/p&gt;&#xA;&lt;p&gt;This time, when I was mapping reads, I formatted all the information in the Read Groups right, but the name of the read groups (which I, unfortunately, chose to be RUN_LANE, not realizing that some of the samples were on the same lane in the same run). And once I tried to do a joint genotyping, freebayes got mad at me.&lt;/p&gt;&#xA;&lt;p&gt;So, is there an easy way to rename read groups?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://broadinstitute.github.io/picard/command-line-overview.html#AddOrReplaceReadGroups&quot; rel=&quot;nofollow noreferrer&quot;&gt;Piccard tools' AddOrReplaceReadGroups&lt;/a&gt; won't do the job as some of the samples were multiplexed on different lanes/runs and this information should be preserved.&lt;/p&gt;&#xA;&lt;p&gt;Basically, I need all bam files with read groups like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@RG     ID:HJ5FNDSXY_L2 SM:WW5-6        PL:ILLUMINA     LB:WW5-6&#xA;@RG     ID:HJ5JVDSXY_L2 SM:WW5-6        PL:ILLUMINA     LB:WW5-6&#xA;@RG     ID:HJHGLDSXY_L4 SM:WW5-6        PL:ILLUMINA     LB:WW5-6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to change to something like (so the ID string will be unique for every combination of sample, run and library)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@RG     ID:WW5-6_HJ5FNDSXY_L2 SM:WW5-6        PL:ILLUMINA     LB:WW5-6&#xA;@RG     ID:WW5-6_HJ5JVDSXY_L2 SM:WW5-6        PL:ILLUMINA     LB:WW5-6&#xA;@RG     ID:WW5-6_HJHGLDSXY_L4 SM:WW5-6        PL:ILLUMINA     LB:WW5-6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There is a similar question asking about &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/5182/57&quot;&gt;adjustments of read groups using pysam&lt;/a&gt;. That is actually quite close to what I need, but it's rather hacky solution I would rather avoid if I can.&lt;/p&gt;&#xA;&lt;p&gt;I also vaguely remember seeing a tool for fixing read groups problems while ago on Twitter, but I can not find it now and I don't know if I just made it up or not.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2021-01-15T13:08:55.417" Title="How to rename Read Groups" Tags="|sam|read-mapping|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15224" PostTypeId="1" AcceptedAnswerId="15225" CreationDate="2021-01-17T11:13:11.620" Score="1" ViewCount="56" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Qdrm0.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Qdrm0.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;How to iterate/repeat a record n times in a fastq file using bioawk? I wrote a python code using biopython, but it is very very slow. So, I am wondering if I can get some help by using bioawk. Thank you so much!&lt;/p&gt;&#xA;&lt;p&gt;for example, in the file.fq file, I'd like to repeat read_id1 for 100times, read_id2 for 100times, ..., and write them to a new file.&lt;/p&gt;&#xA;" OwnerUserId="10294" LastEditorUserId="10294" LastEditDate="2021-01-17T13:39:02.550" LastActivityDate="2021-01-20T08:07:29.723" Title="how to repeat records in fastq n times efficiently?" Tags="|fastq|shell|perl|bioawk|repeat|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15225" PostTypeId="2" ParentId="15224" CreationDate="2021-01-17T12:30:12.000" Score="0" Body="&lt;p&gt;Not sure why you would want to do this, but if the aim is to repeat &lt;strong&gt;all&lt;/strong&gt; records in the fastq/fasta files &lt;code&gt;n&lt;/code&gt; times, the most efficient way I can think of is just to concatenate the orginal file &lt;code&gt;n&lt;/code&gt; times, similar to &lt;a href=&quot;https://superuser.com/questions/338616/linux-command-to-concatenate-a-file-to-itself-n-times&quot;&gt;this thread&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;i.e adapting the accepted answer would look something like this (I have not tested this):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in {1..n};do cat file.fq &amp;gt;&amp;gt; file_ntimes.fq ; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update based on comments:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;To replicate each fastq entry back to back, maybe this hacky unix command can be adjusted (say for n=4):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;n=10&#xA;paste - - - - &amp;lt;file.fq | awk '{for(i=0;i&amp;lt;$n;i++)print}' | tr '\t' '\n' &amp;gt; file_10times.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The idea is to &lt;code&gt;paste&lt;/code&gt; blocks of 4 lines (i.e. 1 fastq entry) into one line and replicate this line n times. &lt;code&gt;tr&lt;/code&gt; is reversing the paste action (since the separation by paste is a tab character). This won't work if fastq file for some reason has tabs in it which I don't expect but didn't test either.&#xA;Note, for fasta file this would be &lt;code&gt;paste - -&lt;/code&gt; rather than &lt;code&gt;paste - - - -&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I haven't tested this, but if it works for you I could polish it up if needed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Based on Xioafeis comment this is solution seems to work as well:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bioawk -cfastx '{for(i=1; i&amp;lt;=10; i++) {print &amp;quot;@&amp;quot;$name&amp;quot;\n&amp;quot;$seq&amp;quot;\n+\n&amp;quot;$qual}}' file.fq &amp;gt; file_10times.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1461" LastEditorUserId="1461" LastEditDate="2021-01-20T08:07:29.723" LastActivityDate="2021-01-20T08:07:29.723" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="15226" PostTypeId="1" AcceptedAnswerId="15227" CreationDate="2021-01-17T20:20:38.880" Score="1" ViewCount="132" Body="&lt;p&gt;I have a file containing multiple fast sequneces. For a specific consensus pattern as input, I extracted all the matching patterns from target fasta sequences with&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -o -E &amp;quot;CC[GT]AAA[GC][AC]TT[GC]&amp;quot; input.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, the above command will retrieve just the matching sequences from fasta sequences and &lt;strong&gt;I  also wanted to get the corresponding fasta header of each match&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For example, if &lt;strong&gt;input.fasta&lt;/strong&gt;  file is something like this,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Gene 1&#xA;TGATGAAAAATGATAGAT&#xA;ATTGGGGGAAAAAAAAAT&#xA;&#xA;&amp;gt;Gene 2&#xA;TTTCCTAAAGATTGT&#xA;AAATTTAAAAATGTTTTT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(Gene 2 has matching subsequence CCTAAAGATTG)&lt;/p&gt;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CCTAAAGATTG   Gene2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I prefer a solution with grep. But other possible solutions also helpful.&lt;/p&gt;&#xA;" OwnerUserId="10244" LastEditorUserId="10244" LastEditDate="2021-01-17T20:27:56.587" LastActivityDate="2021-01-17T21:40:42.950" Title="How to get matching pattern along with ID in a single command in grep?" Tags="|fasta|motifs|grep|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15227" PostTypeId="2" ParentId="15226" CreationDate="2021-01-17T21:40:42.950" Score="1" Body="&lt;p&gt;Use this Perl one-liner:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -lne '$id = $1 if /^&amp;gt;(.+)/; ($m) = /(CC[GT]AAA[GC][AC]TT[GC])/; print join &amp;quot;\t&amp;quot;, $id, $m if $m;' input.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The Perl one-liner uses these command line flags:&lt;br /&gt;&#xA;&lt;code&gt;-e&lt;/code&gt; : Tells Perl to look for code in-line, instead of in a file.&lt;br /&gt;&#xA;&lt;code&gt;-n&lt;/code&gt; : Loop over the input one line at a time, assigning it to &lt;code&gt;$_&lt;/code&gt; by default.&lt;br /&gt;&#xA;&lt;code&gt;-l&lt;/code&gt; : Strip the input line separator (&lt;code&gt;&amp;quot;\n&amp;quot;&lt;/code&gt; on *NIX by default) before executing the code in-line, and append it when printing.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;SEE ALSO:&lt;/strong&gt;&lt;br /&gt;&#xA;&lt;a href=&quot;https://perldoc.perl.org/perlrun.html#Command-Switches&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;perldoc perlrun&lt;/code&gt;: how to execute the Perl interpreter: command line switches&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6251" LastActivityDate="2021-01-17T21:40:42.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15228" PostTypeId="1" CreationDate="2021-01-18T07:18:47.973" Score="0" ViewCount="79" Body="&lt;p&gt;How it is define in terms if accessibility. For example in expression data when I compare A vs B either A or B as reference we define that genes are getting differentially expressed to A or B with our choice of cotrol.&lt;/p&gt;&#xA;&lt;p&gt;Similarly in ATAC seq data when I'm comparing two stages how do i define the stage specific peaks? Such as set of peaks are only for A and other for B?&lt;/p&gt;&#xA;&lt;p&gt;So far what i have done is I get a consensus peaks then from those peaks I get read-count and from those read count I do run deseq2 on those accessible data.&lt;/p&gt;&#xA;&lt;p&gt;Can I do the same as I do for RNA seq data ,and compare peaks of A vs B using deseq2 and say if a peak is differentially accessible then it can be considered as that condition specific peaks.&lt;/p&gt;&#xA;&lt;p&gt;Or&lt;/p&gt;&#xA;&lt;p&gt;There is other way of finding out condition or stage specific peaks ?&#xA;&amp;quot;Identification of tissue-specific chromatin accessible regions&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We used a strategy described previously based on the Shannon entropy&#xA;to compute a tissue specificity index for each peak4,36,37.&#xA;Specifically, for each peak, we defined its relative accessibility in&#xA;a tissue type i as Ri = Ei/ΣE, where Ei is the RPM value for the peak&#xA;in the tissue i, ΣE is the sum of RPM values in all tissues, and N is&#xA;the total number of tissues. The entropy score for each peak across&#xA;tissues can be defined as H = −1 * sum(Ri * log2Ri) (1 &amp;lt; i &amp;lt; N), where&#xA;the value of H ranges between 0 to log2(N). An entropy score close to&#xA;zero indicates the accessibility of this peak is highly&#xA;tissue-specific, while an entropy score close to log2(N) indicates&#xA;that this peak is ubiquitously accessible38. Based on the distribution&#xA;of entropy scores, peaks with score less than 3.5 were selected as&#xA;tissue-restricted peaks.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The above is one of the way they saying tissue specific peaks from this &lt;a href=&quot;https://www.nature.com/articles/s41597-019-0071-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion would be really appreciated&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2021-01-18T10:00:33.020" LastActivityDate="2021-01-18T10:00:33.020" Title="Peak Specific to condition in ATAC seq data" Tags="|atac-seq|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15231" PostTypeId="1" CreationDate="2021-01-18T17:18:38.060" Score="0" ViewCount="18" Body="&lt;p&gt;I'm sorry if this is a stupid question - I have not analysed two channel array data before.&lt;/p&gt;&#xA;&lt;p&gt;I have a GenePix data file, which does not seem to indicate which channel (red or green) is used as reference channel.&lt;/p&gt;&#xA;&lt;p&gt;If it possible to tell this from the data itself?&lt;/p&gt;&#xA;&lt;p&gt;Where raw is my data object, and G is a dataframe of the green channel data and R is the data frame of the red channel data, the ranges are as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#range(raw&lt;span class=&quot;math-container&quot;&gt;$G)&#xA;[1]    45 21939&#xA;&amp;gt; &#xA;&amp;gt; range(raw$&lt;/span&gt;R)&#xA;[1]  39 332&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10303" LastActivityDate="2021-01-21T01:20:47.073" Title="How to tell which channel is the reference channel in two channel (red green) array data?" Tags="|microarray|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15232" PostTypeId="1" CreationDate="2021-01-18T17:26:49.960" Score="1" ViewCount="155" Body="&lt;p&gt;I have a draft, Illumina-based, genome and bunch of reseq samples. I generated bam files using &lt;code&gt;bowtie2&lt;/code&gt; and marked duplicates using &lt;code&gt;picard&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now, when I tried to call SNPs using freebayes, I got cryptic &lt;code&gt;Segmentation fault (core dumped)&lt;/code&gt; even when I tried to run it in verbose debugging mode (&lt;code&gt;-dd&lt;/code&gt;). Do you have an idea where might be a problem?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;freebayes -f data/Afus1/genome.fa -b reseq1.sorted.rmdup.bam reseq2.sorted.rmdup.bam -dd                                                                                                                    &#xA;  loading fasta reference data/Afus1/genome.fa                                                                                              &#xA;                                                                         Opening 2 BAM format alignment input files                         &#xA;                                                                                                                                            &#xA;     done                                                                                                                                   &#xA;Number of ref seqs: 821466                                                                                                                  &#xA;Number of target regions: 0                                                                                                                 &#xA;no sample list file given, reading sample names from bam file                                                                               &#xA;found 2 samples in BAM file                                                                                                                 &#xA;Segmentation fault (core dumped)   &#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="57" LastActivityDate="2021-01-18T17:26:49.960" Title="freebayes segmentation fault" Tags="|snp|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15233" PostTypeId="2" ParentId="15232" CreationDate="2021-01-18T17:26:49.960" Score="2" Body="&lt;p&gt;Oh you silly sausage!&lt;/p&gt;&#xA;&lt;p&gt;This obviously means that you are running out of stack memory. &lt;a href=&quot;https://github.com/freebayes/freebayes/issues/344#issuecomment-273276797&quot; rel=&quot;nofollow noreferrer&quot;&gt;Apparently&lt;/a&gt;...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;freebayes is creating a single string containing the entire vcf header, which gets quite big when the reference has a lot of contigs with long names, causing stack space depletion.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Try increasing the maximum stack size (in this example to 128M) and running your command again.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ulimit -S -s 131072 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="57" LastActivityDate="2021-01-18T17:26:49.960" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15236" PostTypeId="2" ParentId="13670" CreationDate="2021-01-19T03:11:11.583" Score="2" Body="&lt;p&gt;Every version of SBOL allows you to specify prefixes/suffixes as annotations on a sequence.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In SBOL 3, the construct would be a &lt;code&gt;Component&lt;/code&gt; of type &lt;code&gt;DNA&lt;/code&gt; (&lt;a href=&quot;https://identifiers.org/SBO:0000251&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://identifiers.org/SBO:0000251&lt;/a&gt;), and the linker a &lt;code&gt;SequenceFeature&lt;/code&gt; on that &lt;code&gt;Component&lt;/code&gt;. The &lt;code&gt;role&lt;/code&gt; of the sequence feature might depend on the particular assembly you are describing, and whether you're talking before or after assembly (in those cases where the process changes the sequence), but a good one in many cases would be Nucleotide Binding Site (&lt;a href=&quot;https://identifiers.org/SO:0001655&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://identifiers.org/SO:0001655&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;In SBOL 2, the representation is the same, except the construct would be a &lt;code&gt;ComponentDefinition&lt;/code&gt; and the linker a &lt;code&gt;SequenceAnnotation&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;In SBOL 1, the representation is still the same, except the construct would be a &lt;code&gt;DnaComponent&lt;/code&gt; and the linker an SBOL 1 &lt;code&gt;SequenceAnnotation&lt;/code&gt; (which has different fields than in SBOL 2).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If the assembly method does not change the sequence, then all three can represent this pretty much equally well.  If you want to represent a sequence that is modified by assembly or the plasmid carriers from which parts are extracted, then SBOL 3 is the best because it cleanly represents deletions during assembly via the &lt;code&gt;sourceLocation&lt;/code&gt; field.&lt;/p&gt;&#xA;" OwnerUserId="4869" LastActivityDate="2021-01-19T03:11:11.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15237" PostTypeId="1" AcceptedAnswerId="15241" CreationDate="2021-01-19T11:51:01.870" Score="0" ViewCount="666" Body="&lt;p&gt;I used the below code to do CAP analysis to link microbial data with elements, but not sure how to do statistical analysis to show which element variation is responsible most for the difference in microbial composition. I can only able to see two axis in figure so how I can get complete information about the variation based on these mineral element and individually like which mineral responsible most for microbial diversity?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    # CAP ordinate&#xA;    cap_ord &amp;lt;- ordinate(&#xA;        physeq = phyloseq_prop, &#xA;        method = &amp;quot;CAP&amp;quot;,&#xA;        distance = bray_not_na,&#xA;        formula = ~ ~ P + Mn + Cu +Zn + Rb + Mg )&#xA;    &#xA;        &#xA;        # CAP plot&#xA;    cap_plot &amp;lt;- plot_ordination(&#xA;      physeq =phyloseq_prop , &#xA;      ordination = cap_ord, &#xA;        color = &amp;quot;SampleType&amp;quot;, &#xA;        axes = c(1,2)&#xA;    ) + &#xA;        aes(shape = Region) + &#xA;        geom_point(aes(colour = SampleType), alpha = 0.4, size = 4) + &#xA;        geom_point(colour = &amp;quot;grey90&amp;quot;, size = 1.5) + &#xA;        scale_color_manual(values = c(&amp;quot;#CBD588&amp;quot;, &amp;quot;#5F7FC7&amp;quot;, &amp;quot;orange&amp;quot;,&amp;quot;#DA5724&amp;quot;, &amp;quot;#508578&amp;quot;, &amp;quot;#CD9BCD&amp;quot;,&#xA;       &amp;quot;#AD6F3B&amp;quot;, &amp;quot;#673770&amp;quot;,&amp;quot;#D14285&amp;quot;,&amp;quot;#652926&amp;quot;, &amp;quot;#C84248&amp;quot;, &#xA;      &amp;quot;#8569D5&amp;quot;)&#xA;        )&#xA;        &#xA;        # Now add the environmental variables as arrows&#xA;    arrowmat &amp;lt;- vegan::scores(cap_ord, display = &amp;quot;bp&amp;quot;)&#xA;    &#xA;    # Add labels, make a data.frame&#xA;    arrowdf &amp;lt;- data.frame(labels = rownames(arrowmat), arrowmat)&#xA;    &#xA;    # Define the arrow aesthetic mapping&#xA;    arrow_map &amp;lt;- aes(xend = CAP1, &#xA;        yend = CAP2, &#xA;        x = 0, &#xA;        y = 0, &#xA;        shape = NULL, &#xA;        color = NULL, &#xA;        label = labels)&#xA;    &#xA;    label_map &amp;lt;- aes(x = 1.3 * CAP1, &#xA;        y = 1.3 * CAP2, &#xA;        shape = NULL, &#xA;        color = NULL, &#xA;        label = labels)&#xA;    &#xA;    arrowhead = arrow(length = unit(0.02, &amp;quot;npc&amp;quot;))&#xA;    &#xA;    pdf(&amp;quot;cap2.pdf&amp;quot;, height=8, width=8)&#xA;    # Make a new graphic&#xA;    cap_plot + &#xA;      geom_segment(&#xA;        mapping = arrow_map, &#xA;        size = .5, &#xA;        data = arrowdf, &#xA;        color = &amp;quot;gray&amp;quot;, &#xA;        arrow = arrowhead&#xA;      ) + &#xA;      geom_text(&#xA;        mapping = label_map, &#xA;        size = 4,  &#xA;        data = arrowdf, &#xA;        show.legend = FALSE&#xA;      )&#xA;      &#xA;      dev.off();&#xA;      &#xA;        &#xA;     cap_ord&#xA;    &#xA;     Call: capscale(formula = distance ~  P + Mn + Cu +Zn + Rb + Mg, data = data)&#xA;&#xA;                                  Inertia Proportion Rank&#xA;Total          2.069e+01  1.000e+00&#xA;Constrained    5.240e+00  2.533e-01   10&#xA;Unconstrained  1.545e+01  7.468e-01   97&#xA;Imaginary     -1.090e-03 -5.268e-05    1&#xA;Inertia is squared Bray distance&#xA;&#xA;Eigenvalues for constrained axes:&#xA;  CAP1   CAP2   CAP3   CAP4   CAP5   CAP6   CAP7   CAP8   CAP9  CAP10&#xA;2.5733 0.8789 0.5647 0.3797 0.3037 0.1873 0.1243 0.1030 0.0800 0.0453&#xA;&#xA;Eigenvalues for unconstrained axes:&#xA; MDS1  MDS2  MDS3  MDS4  MDS5  MDS6  MDS7  MDS8&#xA;4.790 1.343 1.138 0.916 0.784 0.649 0.518 0.501&#xA;(Showing 8 of 97 unconstrained eigenvalues)&#xA;  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Many thanks&lt;/p&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="1008" LastEditDate="2021-01-20T21:48:08.927" LastActivityDate="2021-01-20T21:48:08.927" Title="Regarding canonical analysis of principal coordinates (CAP) for linking microbial data with elements" Tags="|r|statistics|microbial-genomics|phyloseq|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="15238" PostTypeId="1" AcceptedAnswerId="15498" CreationDate="2021-01-19T14:22:12.273" Score="2" ViewCount="69" Body="&lt;p&gt;Say I found an interesting protein in a genbank file, e.g.:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; gene            273792..275552&#xA;                 /locus_tag=&amp;quot;ECs_0242&amp;quot;&#xA; CDS             273792..275552&#xA;                 /locus_tag=&amp;quot;ECs_0242&amp;quot;&#xA;                 /codon_start=1&#xA;                 /transl_table=11&#xA;                 /product=&amp;quot;type IV secretion protein Rhs&amp;quot;&#xA;                 /protein_id=&amp;quot;NP_308269.1&amp;quot;&#xA;                 /translation=&amp;quot;[...]&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NC_002695.2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/NC_002695.2&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;I would like to programmatically get a classification of this protein.&lt;br/&gt;&#xA;The (not new) motivation is that my pipeline finds many proteins, and I want to identify enriched or depleted protein classes (e.g., enrichment for DNA-binding proteins).&lt;/p&gt;&#xA;&lt;p&gt;By googling for &amp;quot;type IV secretion protein Rhs&amp;quot;, I found a &lt;a href=&quot;https://www.uniprot.org/uniprot/A0A0L1CB30&quot; rel=&quot;nofollow noreferrer&quot;&gt;UniProt entry&lt;/a&gt; which has a link to &lt;a href=&quot;https://www.ebi.ac.uk/QuickGO/annotations?geneProductId=A0A0L1CB30&quot; rel=&quot;nofollow noreferrer&quot;&gt;a QuickGO page&lt;/a&gt;, which includes three categories: &amp;quot;transposase activity&amp;quot;, &amp;quot;transposition, DNA-mediated&amp;quot; and &amp;quot;DNA binding&amp;quot;. From this page I can also get an ancestor chart for each of these categories, e.g.:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/O3Att.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/O3Att.png&quot; alt=&quot;DNA binding chart&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Optimally, the code would work quickly (preferably locally), and for a description from a CDS feature (e.g.: &amp;quot;type IV secretion protein Rhs&amp;quot;) return the GO categories it belongs to. It would be even better to also have the ancestry of each category in a computer-readable format.&lt;/p&gt;&#xA;&lt;p&gt;What is the best way to do that?&lt;br/&gt;&#xA;Is there a specific &lt;a href=&quot;https://www.uniprot.org/downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;UniProt subset&lt;/a&gt; that fits my need (finding GO categories given a protein description)?&lt;br/&gt;&#xA;Is there a specific &lt;a href=&quot;http://geneontology.org/docs/download-ontology/#subsets&quot; rel=&quot;nofollow noreferrer&quot;&gt;GO subset&lt;/a&gt; that fits my need (extracting ancestor GO categories as explained &lt;a href=&quot;https://github.com/owlcollab/owltools/wiki/Finding-Common-Ancestors&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;)?&lt;br/&gt;&lt;/p&gt;&#xA;&lt;p&gt;(I figured that GO categories fit my need of classifying proteins, but if a better option exists, please let me know)&lt;/p&gt;&#xA;" OwnerUserId="4868" LastEditorUserId="4868" LastEditDate="2021-01-19T16:18:55.147" LastActivityDate="2021-03-08T08:54:10.150" Title="How to programmatically classify a protein according to its genbank feature" Tags="|sequence-annotation|uniprot|genbank|gene-ontology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15239" PostTypeId="2" ParentId="10977" CreationDate="2021-01-19T17:34:33.033" Score="1" Body="&lt;p&gt;You can view all the available datasets at this URL &lt;a href=&quot;https://github.com/vincentarelbundock/Rdatasets/blob/master/datasets.csv&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/vincentarelbundock/Rdatasets/blob/master/datasets.csv&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The available ones in statsmodels are those with the tag &lt;code&gt;datasets&lt;/code&gt; which looks like line 372 - 455.&lt;/p&gt;&#xA;&lt;p&gt;I didn't test them all, but everyone I tried from a small random sample seemed to work fine.&lt;/p&gt;&#xA;" OwnerUserId="10312" LastActivityDate="2021-01-19T17:34:33.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15240" PostTypeId="1" CreationDate="2021-01-20T14:17:38.987" Score="0" ViewCount="82" Body="&lt;p&gt;I am doing a differential gene expression analysis to find out genes that are differentially expressed in one cell type from the lung in two conditions, control and idipathic pulmonary fibrosis (IPF). In total I have 152538 cells. Can anyone help me provide a hint to understand why only NAs get returned for the pvalues and adjusted pvalues?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; table(seurat$group)&#xA;&#xA;Control     IPF &#xA;  64652   87886 &#xA;&#xA;&amp;gt; degs &amp;lt;- FindMarkers(seurat, &#xA;                            ident.1 = &amp;quot;IPF&amp;quot;,&#xA;                            ident.2 = &amp;quot;Control&amp;quot;)&#xA;&amp;gt; head(degs) &#xA;         p_val  avg_logFC pct.1 pct.2 p_val_adj&#xA;SPP1        NA  1.2721607 0.390 0.125        NA&#xA;FN1         NA  1.1768771 0.741 0.409        NA&#xA;CRIP1       NA  1.0484934 0.938 0.832        NA&#xA;CCL18       NA  0.9830185 0.734 0.469        NA&#xA;S100A10     NA  0.5799861 0.965 0.902        NA&#xA;VIM         NA  0.5666287 0.992 0.964        NA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3968" LastActivityDate="2021-01-20T14:17:38.987" Title="Why could I be having only NAs for p_val and p_val_adj in Seurat DGE analysis?" Tags="|scrnaseq|seurat|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15241" PostTypeId="2" ParentId="15237" CreationDate="2021-01-20T20:05:49.670" Score="1" Body="&lt;p&gt;I don't think that you can say that the 25% can be said to relate to the chemical variable specifically; rather, that 25% is the variance that is explained by all constrained axes. While these axes are drawn in such a way that they maximize variance associated with your independent variables, I do not think that they are directly interpretable as being 100% due to the elements.&lt;/p&gt;&#xA;&lt;p&gt;The number of constrained axes is a parameter chosen as an input to the &lt;a href=&quot;https://esajournals.onlinelibrary.wiley.com/doi/abs/10.1890/0012-9658%282003%29084%5B0511%3ACAOPCA%5D2.0.CO%3B2&quot; rel=&quot;nofollow noreferrer&quot;&gt;CAPscale method&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;An extremely important point is to determine how many PCO axes should be retained; i.e., what should be the choice for &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;? One might be concerned that the method could suffer from a poor choice for &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;. If &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is too small, then there may be some ecologically important information in the data that will not be included in the canonical analysis. On the other hand, if &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is too large relative to &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;, then a misleading canonical plot could result. For example, we cannot set &lt;span class=&quot;math-container&quot;&gt;$m = N - 1$&lt;/span&gt;, for then the squared canonical correlations would all be equal to 1.0, because we have &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; points!&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So the value of the variance there will vary depending on how many constrained axes you use. If I understand right, you could obtain 100% distance variance explained by choosing as many constrained axes as you have observations.&lt;/p&gt;&#xA;&lt;p&gt;I think that it is worth looking into how others have solved this problem. For example, one has to first decide whether or not this 25% is an unexpectedly large number, i.e. compute  p-values (I am not crazy about p-values in most contexts but here they could serve as a helpful heuristic), as suggested by the CAPscale paper:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;As a further option, one may test the hypothesis of either (1) no significant differences in multivariate location among groups (CDA), or (2) no significant relationship with quantitative environmental or other variables (CCorA). This is done by using the trace statistic (sum of canonical eigenvalues (sum of squared canonical correlations; see the Appendix) and obtaining a P value by permutation (e.g., Anderson 2001b).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This seems to be &lt;a href=&quot;https://rdrr.io/rforge/vegan/man/anova.cca.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;directly implemented as the &lt;code&gt;anova.cca()&lt;/code&gt; function&lt;/a&gt; in the vegan package (I believe that they have decided that CAP is a flavor of CCA for that package). I understand the motivation to get % variance as it's intuitive, but in complex statistical analysis of distances it is a little bit less clear what it really means. Some kind of comparison to a null hypothesis is probably necessary to say whether it is meaningful.&lt;/p&gt;&#xA;&lt;p&gt;For how others have approached similar questions you can see &lt;a href=&quot;https://www.researchgate.net/post/How-to-analyse-the-effect-of-a-factor-on-species-distribution-and-a-species-functional-trait&quot; rel=&quot;nofollow noreferrer&quot;&gt;this discussion&lt;/a&gt;. For other implementations that may be more user-friendly, you might consider the vegan package, which is a richer toolbox for ordination in ecology (even if it is less focused on metagenomics). &lt;a href=&quot;https://rdrr.io/rforge/vegan/man/capscale.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is the CAPscale function&lt;/a&gt; in vegan. The phyloseq authors have correctly decided to build large pieces of phyloseq out of vegan, so &lt;a href=&quot;https://jacobrprice.github.io/2017/08/26/phyloseq-to-vegan-and-back.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;I think that you can pass phyloseq back into vegan functions&lt;/a&gt; and e.g. use those analyses directly on your &lt;code&gt;cap_ord&lt;/code&gt; object. But see that link for more info.&lt;/p&gt;&#xA;&lt;p&gt;For a little more easy-to-understand background on statistics of distance methods (particularly in microbiomes), you might see &lt;a href=&quot;https://si.biostat.washington.edu/sites/default/files/modules/Lecture6_0.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-01-20T20:05:49.670" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15242" PostTypeId="2" ParentId="15198" CreationDate="2021-01-21T01:09:41.153" Score="3" Body="&lt;p&gt;You ask about which &amp;quot;genes are expressed&amp;quot; and then you mention &amp;quot;if a gene is up or down regulated&amp;quot;. These are different, and given your application I think what you actually want to know if these marker genes are expressed. This is not a question of up/down-regulation relative to a control, so you do not need control samples for this.&lt;/p&gt;&#xA;&lt;p&gt;It seems most of your datasets are Affymetrix, and there are R/Bioconductor packages that will give you the probability a gene is expressed. One such I found is frma (&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/frma.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/html/frma.html&lt;/a&gt;), which returns an object &amp;quot;weight&amp;quot; that is &amp;quot;a vector of weights which roughly correspond to the probability of expression for each gene.&amp;quot; I haven't worked with microarrays in a while, but I recall many other R/Bioconductor packages offering this functionality.&lt;/p&gt;&#xA;&lt;p&gt;A detection p-value is calculated for Illumina array probes in Illumina's native BeadStudio software.&lt;/p&gt;&#xA;" OwnerUserId="7220" LastEditorUserId="7220" LastEditDate="2021-01-21T01:15:16.717" LastActivityDate="2021-01-21T01:15:16.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15243" PostTypeId="2" ParentId="15231" CreationDate="2021-01-21T01:20:47.073" Score="2" Body="&lt;p&gt;No, you can't get this from the data itself. If there is a control/reference sample on one channel, you need someone to annotate which channel it's on.&lt;/p&gt;&#xA;" OwnerUserId="7220" LastActivityDate="2021-01-21T01:20:47.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15245" PostTypeId="2" ParentId="933" CreationDate="2021-01-21T11:17:45.127" Score="2" Body="&lt;p&gt;I found Jesse's code quite usefull ! For those who try to reproduce it, you should now add the reference genome ID, such as :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;&#xA;import requests&#xA;# import pprint&#xA;# prettyprint = pprint.PrettyPrinter(indent=2).pprint&#xA;&#xA;def fetch(jsondata, url=&amp;quot;https://gnomad.broadinstitute.org/api&amp;quot;):&#xA;    # The server gives a generic error message if the content type isn't&#xA;    # explicitly set&#xA;    headers = {&amp;quot;Content-Type&amp;quot;: &amp;quot;application/json&amp;quot;}&#xA;    response = requests.post(url, json=jsondata, headers=headers)&#xA;    json = response.json()&#xA;    if &amp;quot;errors&amp;quot; in json:&#xA;        raise Exception(str(json[&amp;quot;errors&amp;quot;]))&#xA;    return json&#xA;&#xA;def get_variant_list(gene_id, dataset=&amp;quot;gnomad_r2_1&amp;quot;):&#xA;    # Note that this is GraphQL, not JSON.&#xA;    fmt_graphql = &amp;quot;&amp;quot;&amp;quot; &#xA;    {&#xA;        gene(gene_id: &amp;quot;%s&amp;quot;, reference_genome: GRCh38) {&#xA;          variants(dataset: %s) {&#xA;            consequence&#xA;            pos&#xA;            rsid&#xA;            variant_id: variantId&#xA;          }&#xA;        }&#xA;      }&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    # This part will be JSON encoded, but with the GraphQL part left as a&#xA;    # glob of text.&#xA;    req_variantlist = {&#xA;        &amp;quot;query&amp;quot;: fmt_graphql % (gene_id, dataset),&#xA;        &amp;quot;variables&amp;quot;: {&amp;quot;withFriends&amp;quot;: False}&#xA;        }&#xA;    response = fetch(req_variantlist)&#xA;    return response[&amp;quot;data&amp;quot;][&amp;quot;gene&amp;quot;][&amp;quot;variants&amp;quot;]&#xA;&#xA;prettyprint(get_variant_list(&amp;quot;ENSG00000010610&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10325" LastActivityDate="2021-01-21T11:17:45.127" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15246" PostTypeId="1" AcceptedAnswerId="15247" CreationDate="2021-01-21T12:39:23.863" Score="1" ViewCount="296" Body="&lt;p&gt;I want to retrieve all overlaps between two sets of genomic intervals (GRanges) using Bioconductor. I am interested in the widths of the resulting overlapping ranges.&lt;/p&gt;&#xA;&lt;p&gt;In cases with discontinuous ranges, &lt;code&gt;intersect()&lt;/code&gt; works, but I am interested in the case where both sets contain contiguous ranges (no gaps).&lt;/p&gt;&#xA;&lt;p&gt;I am using the findOverlaps function as in the example below:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;gr1 &amp;lt;- GRanges(seqnames=&amp;quot;chr2&amp;quot;, ranges=IRanges(c(1, 11, 21, 31), c(10, 20, 30, 40)))&#xA;gr2 &amp;lt;- GRanges(seqnames=&amp;quot;chr2&amp;quot;, ranges=IRanges(c(1, 5, 15), c(4, 14, 30)))&#xA;hits &amp;lt;- findOverlaps(gr1, gr2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This code yields the following intersections in a &amp;quot;Hits&amp;quot; object:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Hits object with 5 hits and 0 metadata columns:&#xA;      queryHits subjectHits&#xA;      &amp;lt;integer&amp;gt;   &amp;lt;integer&amp;gt;&#xA;  [1]         1           1&#xA;  [2]         1           2&#xA;  [3]         2           2&#xA;  [4]         2           3&#xA;  [5]         3           3&#xA;  -------&#xA;  queryLength: 4 / subjectLength: 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I could not find a way to extract the intersections widths from the resulting object.&lt;/p&gt;&#xA;&lt;p&gt;It this the wrong approach ?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Note: Using &lt;code&gt;intersect(gr1, gr2)&lt;/code&gt; in this case yields a single contiguous overlap range spanning the whole chromosome, but I want each overlap to be returned independently, just like findOverlaps does.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="439" LastEditorUserId="439" LastEditDate="2021-01-21T14:36:08.317" LastActivityDate="2021-01-23T12:10:49.933" Title="Retrieve overlap widths with Bioconductor" Tags="|r|bioconductor|genomicranges|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15247" PostTypeId="2" ParentId="15246" CreationDate="2021-01-21T13:34:56.740" Score="1" Body="&lt;p&gt;Whilst &lt;code&gt;intersect&lt;/code&gt; is sort of doing what you want, it merges adjacent overlaps together which is not what you want I suppose. There doesn't seem to be dedicated &lt;code&gt;Grange&lt;/code&gt; function to do this out of the box, but you could go through all overlapping pairs resulting from &lt;code&gt;findOverlap&lt;/code&gt; and calculate it's widths using &lt;a href=&quot;https://www.rdocumentation.org/packages/IRanges/versions/2.0.1/topics/setops-methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;pintersect&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;punion, pintersect, psetdiff and pgap are generic functions that compute the element-wise (aka &amp;quot;parallel&amp;quot;) union, intersection, (asymmetric!) difference and gap between each element in x and its corresponding element in y.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;pintersect&amp;quot;(x, y, resolve.empty=c(&amp;quot;none&amp;quot;, &amp;quot;max.start&amp;quot;, &amp;quot;start.x&amp;quot;), ...) # x, y   IRanges objects.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;hits &amp;lt;- findOverlaps(gr1, gr2)&#xA;&#xA;width(pintersect(gr1[queryHits(hits)],&#xA;                 gr2[subjectHits(hits)]))&#xA;[1]  2  6  4  6 10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1461" LastEditorUserId="1461" LastEditDate="2021-01-23T12:10:49.933" LastActivityDate="2021-01-23T12:10:49.933" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15248" PostTypeId="1" CreationDate="2021-01-21T18:45:41.930" Score="9" ViewCount="753" Body="&lt;p&gt;Are there best practices to load different bioinformatics file formats such as VCF, BED, GFF, and SAM to SQL databases? I am wondering how people out there do that efficiently.&lt;/p&gt;&#xA;&lt;p&gt;All of these three formats are tab-separated files, so basically the following should work. I feel weird about it since most people I know don't use MySQL to work with these files.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;LOAD DATA LOCAL INFILE 'bed.bed'&#xA;INTO TABLE bed-file&#xA;FIELDS TERMINATED BY '\t'&#xA;LINES TERMINATED BY '\n'&#xA;IGNORE 1 ROWS (list of the columns)&#xA;SET creation_date  = STR_TO_DATE(@creation_date, '%m/%d/%y');&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2022-11-13T05:23:46.940" LastActivityDate="2022-11-13T05:23:46.940" Title="How to import a large amount of .bed, .gff, .vcf, .paf, .sam files into an SQL database?" Tags="|vcf|database|bed|sam|gff|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15250" PostTypeId="2" ParentId="15197" CreationDate="2021-01-21T22:37:30.273" Score="0" Body="&lt;p&gt;My hacktastic fix for this is to use basename to get the name of each file, then do&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;refName=`basename &amp;quot;${Ref}&amp;quot;`&#xA;ln -s &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${Ref}&quot; &quot;$&lt;/span&gt;refName&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;for each file.  This puts aliases to each file in the executions dir, and then I pass &amp;quot;refName&amp;quot; to GATK rather than &amp;quot;${Ref}&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Not a good solution, but a solution&lt;/p&gt;&#xA;" OwnerUserId="9222" LastEditorUserId="9222" LastEditDate="2023-08-24T15:58:17.290" LastActivityDate="2023-08-24T15:58:17.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15251" PostTypeId="1" AcceptedAnswerId="15252" CreationDate="2021-01-22T00:02:07.417" Score="0" ViewCount="30" Body="&lt;p&gt;For a a few SNPs from given gene, I know there happen to be several haplotypes that most of the population has. But from a person's data on those SNPs, we have two alleles at each SNP, so it's not trivial to match the SNP data to infer the person's haplotypes. When I search this I find tools that seem to be for haplotype reconstruction and SNP imputation, such as SHAPEIT, but this is not what I'm looking for.&lt;/p&gt;&#xA;&lt;p&gt;Might someone know of a tool for this inference of known haplotypes from SNP data? It would be helpful even if someone could give me a name for this problem, as I'm not sure of the right term to search.&lt;/p&gt;&#xA;" OwnerUserId="7220" LastActivityDate="2021-01-22T00:58:55.027" Title="How do I infer someone's haplotype of a gene from data on several of the gene's SNPs?" Tags="|snp|haplotypes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15252" PostTypeId="2" ParentId="15251" CreationDate="2021-01-22T00:58:55.027" Score="1" Body="&lt;p&gt;It depends on the scale of the data (i.e. number of individuals, number of SNPs). Generally this can be done by matching up individual SNP genotypes to haplotype tag SNPs. The specific combination of genotyped SNPs is used to determine the underlying haplotype, based on tag SNP frequencies for each haplotype.&lt;/p&gt;&#xA;&lt;p&gt;More details of the process of identifying tag SNPs can be found here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.broadinstitute.org/haploview/tagger&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.broadinstitute.org/haploview/tagger&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-01-22T00:58:55.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15253" PostTypeId="1" CreationDate="2021-01-22T10:41:55.587" Score="0" ViewCount="26" Body="&lt;p&gt;I have 5 strains of P.falciparum. Each FASTA file has all its annotated CDSs. After a first pre-processing phase, where I eliminated the strangest sequences (perhaps the longest or shorter ones, which did not start with methionine or which did not have a length multiple of three) I wanted to align the CDS of these strains to evaluate how much every single gene was prone to acquire structural polymorphisms (insertions, SNPs deletions, etc.). A first step I thought could be BLAST to find the orthologs and associate the identifiers of the genes. The problem is that BLASTn has a limit on the number of sequences it can use. How can I go about getting multiple alignments? Also, is there any useful statistic that you can apply that you know of in the literature that can provide a mathematical criterion for obtaining a statistic about it?&#xA;thanks a lot&lt;/p&gt;&#xA;" OwnerUserId="9142" LastActivityDate="2021-01-22T10:41:55.587" Title="How to assess how prone genes may be to acquire structural polymorphisms?" Tags="|sequence-alignment|snp|blastn|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="15254" PostTypeId="1" AcceptedAnswerId="15260" CreationDate="2021-01-22T10:53:31.997" Score="2" ViewCount="83" Body="&lt;p&gt;Can some explain why when I run a REST API query on UniProt for one ID ('Q9UJL9'), the results for three IDs are returned?&lt;/p&gt;&#xA;&lt;p&gt;The query I'm using is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.uniprot.org/uniprot/?query=Q9UJL9&amp;amp;sort=score&amp;amp;columns=id,go(molecular%20function)&amp;amp;format=tab&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uniprot/?query=Q9UJL9&amp;amp;sort=score&amp;amp;columns=id,go(molecular%20function)&amp;amp;format=tab&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;(I didn't hyperlink it since you need to see the query to see where I'm going wrong).&lt;/p&gt;&#xA;&lt;p&gt;When I run the query above, the output is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Entry   Gene ontology (molecular function)&#xA;Q9UJL9  DNA-binding transcription factor activity, RNA polymerase II-specific [GO:0000981]; DNA-binding transcription repressor activity, RNA polymerase II-specific [GO:0001227]; metal ion binding [GO:0046872]; RNA polymerase II transcription regulatory region sequence-specific DNA binding [GO:0000977]&#xA;Q8WWY3  identical protein binding [GO:0042802]; ribonucleoprotein complex binding [GO:0043021]; RNA binding [GO:0003723]; snRNP binding [GO:0070990]; U4atac snRNA binding [GO:0030622]; U4 snRNA binding [GO:0030621]&#xA;Q6A162  structural molecule activity [GO:0005198]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I checked the uniprot page of the query I want (Q9UJL9), the other items are interactors (along with many others) but not synonyms/linked (it seems, e.g my query is a zinc finger, one of the linked records, Q6A162, is a keratin protein).&lt;/p&gt;&#xA;" OwnerUserId="10328" LastActivityDate="2021-01-23T15:06:36.280" Title="Why is the UniProt REST API returning multiple results, when I am only providing one ID to be checked?" Tags="|uniprot|api|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15256" PostTypeId="1" AcceptedAnswerId="15277" CreationDate="2021-01-22T16:16:17.030" Score="1" ViewCount="115" Body="&lt;p&gt;When analyzing Quast results it seems that it doesn't calculate mismatches and indels in a useful way if the &amp;quot;Duplication ratio&amp;quot; is over 1.&#xA;For example, that's what I get for an assembly with two contigs covering almost the full genome&#xA;&lt;a href=&quot;https://i.stack.imgur.com/iGGbL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iGGbL.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And that's what I get if I remove one of the contigs:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/L91VH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/L91VH.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;See how the mismatches and indels per 100kbp are almost double in the first case but that does not correspond to the reality.&lt;/p&gt;&#xA;&lt;p&gt;Any better way to get these metrics? Would it be safe to divide error rates by the duplication ratio?&lt;/p&gt;&#xA;" OwnerUserId="5190" LastActivityDate="2021-01-28T00:48:50.603" Title="Quast duplication ratio and mismatches percent" Tags="|ngs|assembly|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15257" PostTypeId="1" CreationDate="2021-01-22T18:26:46.507" Score="0" ViewCount="304" Body="&lt;p&gt;I am trying to calculate the substitution probabilities for amino acids (i.e. the probability of one amino acid given another). In theory, you should be able to derive them from a log-likelihood matrix (also called log-odds), such as Blosum62.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know how to calculate probabilities from log-likelihoods or have a good modern source for a substitution probability matrix?&lt;/p&gt;&#xA;" OwnerUserId="10330" OwnerDisplayName="chubs" LastEditorUserId="10330" LastEditDate="2021-01-23T21:23:51.857" LastActivityDate="2021-01-23T21:23:51.857" Title="How does one convert a log-likelihood substitution matrix into a probability matrix?" Tags="|sequence-alignment|phylogenetics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15258" PostTypeId="2" ParentId="15257" CreationDate="2021-01-22T20:01:27.773" Score="1" Body="&lt;p&gt;&lt;strong&gt;UPDATE2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;@chubs wants to compare different clades. This is possible within recent releases of ConSurf:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xyu0W.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xyu0W.png&quot; alt=&quot;consurf table 1&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is additionally quite possible to use ConSurf's &lt;a href=&quot;https://consurf.tau.ac.il/overview.php#_Toc311131301&quot; rel=&quot;nofollow noreferrer&quot;&gt;default outputs&lt;/a&gt; to get per-position conservation scores that can be used for post-processing in any fashion, as well as prefab scripts that can be used with PyMol to make custom visualization:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For all cases, ConSurf creates the following outputs:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The sequence and MSA colored by ConSurf conservation scores.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A text file that summarizes for each position the normalized score calculated, the assigned grade (1-through-9), the reliability estimation (for the Bayesian method) and the amino acids/nucleotides observed in the respective MSA column.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The sequences selected for the MSA and the MSA constructed (unless those files were uploaded by the user).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A file with the frequency of each amino acid/nucleotide observed in each column of the MSA.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The evolutionary tree, which was calculated by the server or uploaded by the user, together with the MSA are shown using the WASABI platform (Veidenberg et al., 2015).  Moreover, using WASABI, a user can select a subtree containing a fraction of the homologous sequences and conduct a follow-up ConSurf analysis with these selected sequences. To refine a ConSurf analysis to a selected subtree, a user can choose any internal node on the phylogenetic tree and open a WASABI menu by using a right mouse click. Selecting the option 'run ConSurf on subtree' will issue a new window with the new ConSurf run for the selected subtree sequences (see example in Figure 3).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It seems that what @chubs really wants to do is map protein conservation onto a structure. There are various tools for doing this (&lt;a href=&quot;https://academic.oup.com/bioinformatics/article/35/2/332/5040318&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, &lt;a href=&quot;https://academic.oup.com/nar/article/44/W1/W344/2499373&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). ConSurf is the older and more established tool, using evolutionary conservation to rule out trivial apparent associations due to phylogenetic non-independence.&lt;/p&gt;&#xA;&lt;p&gt;here is the  somewhat related 3DPatch workflow, which uses profile HMM information as a measure of conservation:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fDWGY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fDWGY.png&quot; alt=&quot;3Dpatch workflow&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is the output that they get:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eAUSL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eAUSL.png&quot; alt=&quot;3dpatch figure&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Likewise ConSurf gets similar looking outputs:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EyfY6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EyfY6.png&quot; alt=&quot;consurf figure&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;ORIGINAL ANSWER&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I think that more information is needed here to answer this question properly (what is your data? what is your ultimate goal? what are the things you've already tried?), but we can say a few things right away.&lt;/p&gt;&#xA;&lt;p&gt;I would strongly suggest looking into &lt;a href=&quot;https://en.wikipedia.org/wiki/Substitution_model#The_mathematics_of_substitution_models&quot; rel=&quot;nofollow noreferrer&quot;&gt;substitution models&lt;/a&gt;, which are literally substitution probability matrices. Note that these are probabilities &lt;em&gt;per unit time&lt;/em&gt;, which is to say that they are calibrated to e.g. evolutionary rates along a specific phylogenetic tree. I would therefore strongly suggest incorporating a phylogenetic model of some kind.&lt;/p&gt;&#xA;&lt;p&gt;If you have e.g. a multiple sequence alignment, you can estimate a phylogenetic tree by one of the many tree estimation methods, which will give you a substitution probability matrix &amp;quot;for free&amp;quot; as this estimation is taken care of in the process of tree estimation. These will again be substitution probabilities per unit time; you will honestly not be able to get rid of the time-dependence here, as that's how evolution works.&lt;/p&gt;&#xA;&lt;p&gt;I am a little behind the times, but some tools you might look into for tree estimation are &lt;a href=&quot;http://www.atgc-montpellier.fr/phyml/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PhyML&lt;/a&gt; and &lt;a href=&quot;https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0009490&quot; rel=&quot;nofollow noreferrer&quot;&gt;FastTree&lt;/a&gt;. If you only have sequences and haven't aligned them yet, you can use a tool like MUSCLE, ClustalO, or MAFFT.&lt;/p&gt;&#xA;&lt;p&gt;BLOSUM and related models are derived from specific slices of protein sequence identity (evolutionary history) calibrated to certain scopes, it's unlikely that they are appropriate to your case, unless your data fits inside those rather narrow slices.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2021-01-22T23:06:46.453" LastActivityDate="2021-01-22T23:06:46.453" CommentCount="15" ContentLicense="CC BY-SA 4.0" />
  <row Id="15259" PostTypeId="1" AcceptedAnswerId="15261" CreationDate="2021-01-23T00:29:04.657" Score="5" ViewCount="2339" Body="&lt;p&gt;I was wondering if there is more elegant way of installing and loading multiple packages in Bioconductor similar to &lt;code&gt;pacman&lt;/code&gt; with CRAN packages.&lt;/p&gt;&#xA;&lt;p&gt;I tried:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;# install and load  the package  manager&#xA; if (!requireNamespace(&amp;quot;BiocManager&amp;quot;, quietly = TRUE))&#xA;    install.packages(&amp;quot;BiocManager&amp;quot;)&#xA;&#xA;# install and load bioconductor packages &#xA;bio_pkgs = c(&amp;quot;BiocManager&amp;quot;, &amp;quot;ggbio&amp;quot;, &amp;quot;biomaRt&amp;quot;, &amp;quot;EnsDb.Hsapiens.v75&amp;quot;, &amp;quot;cBioPortalData&amp;quot;, &amp;quot;GenomicRanges&amp;quot;, &amp;quot;GenomicFeatures&amp;quot;, &amp;quot;TxDb.Hsapiens.UCSC.hg19.knownGene&amp;quot;, &amp;quot;Homo.sapiens&amp;quot;)&#xA;&#xA; if (!requireNamespace(bio_pkgs, quietly = TRUE))&#xA;    BiocManager::install(bio_pkgs)&#xA;&#xA;library(easypackages)&#xA;libraries(&amp;quot;ggbio&amp;quot;, &amp;quot;biomaRt&amp;quot;, &amp;quot;EnsDb.Hsapiens.v75&amp;quot;, &amp;quot;renvcBioPortalData&amp;quot;, &amp;quot;GenomicRanges&amp;quot;, &amp;quot;GenomicFeatures&amp;quot;, &amp;quot;TxDb.Hsapiens.UCSC.hg19.knownGene&amp;quot;, &amp;quot;Homo.sapiens&amp;quot;)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10179" LastEditorUserId="10179" LastEditDate="2021-01-23T00:44:35.453" LastActivityDate="2021-12-14T12:59:06.223" Title="installing multiple bioconductor packages at once" Tags="|bioconductor|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15260" PostTypeId="2" ParentId="15254" CreationDate="2021-01-23T15:06:36.280" Score="3" Body="&lt;p&gt;it is because you do a text search and the &amp;quot;word&amp;quot; Q9UJL9 appears in multiple entries. If you want an id search, specifically say so and for that the query syntax is &lt;code&gt;id:Q9UJL9&lt;/code&gt;. You will want to follow a redirect, and there are some rare cases where that might still lead to more than one entry if the id was made secondary and attached to multiple entries. e.g. P00001.&lt;/p&gt;&#xA;" OwnerUserId="2499" LastActivityDate="2021-01-23T15:06:36.280" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15261" PostTypeId="2" ParentId="15259" CreationDate="2021-01-23T16:11:47.317" Score="7" Body="&lt;p&gt;You can do something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bio_pkgs &amp;lt;- c(&amp;quot;ggbio&amp;quot;, &amp;quot;biomaRt&amp;quot;, &amp;quot;EnsDb.Hsapiens.v75&amp;quot;, &amp;quot;renvcBioPortalData&amp;quot;, &#xA;          &amp;quot;GenomicRanges&amp;quot;, &amp;quot;GenomicFeatures&amp;quot;, &amp;quot;TxDb.Hsapiens.UCSC.hg19.knownGene&amp;quot;, &amp;quot;Homo.sapiens&amp;quot;)&#xA;&#xA;# install:&#xA;BiocManager::install(bio_pkgs)&#xA;&#xA;# load all at once&#xA;invisible(lapply(pkgs, function(x) library(x, character.only=TRUE)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Be aware though that if you version-control your packages with something like &lt;code&gt;renv&lt;/code&gt; then it is encouraged to explicitely load every package as suggested &lt;a href=&quot;https://github.com/rstudio/renv/issues/720&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; so &lt;code&gt;renv&lt;/code&gt; can properly recognize the packages to snapshot.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2021-12-14T12:59:06.223" LastActivityDate="2021-12-14T12:59:06.223" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15263" PostTypeId="1" CreationDate="2021-01-25T05:44:39.783" Score="4" ViewCount="321" Body="&lt;p&gt;I have a fastq file of 400,000 reads (so speed is important). In the sequences there are barcodes integrated that should be present twice. Given a barcode, I want to find the sequences that have the barcode present twice with &amp;lt;= 2 mismatches. So, with a barcode 'ATTCGACCGATAGG', I would like to retrieve all of the following sequences-&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;TATCTTGTGGAAAGGACGAAACACCGAACACAAAGCATAGATGCGTTTAAGAGCTATGCTGGAAAACAGCATAGCAAGTTTAAATAAGGCTAGTCCGTTATCAACTTGAAAAAGTGGCACCGAGTCGGTGCTTTTTTT&lt;strong&gt;ATTCGACCGATAGG&lt;/strong&gt;GGTGGCAGGGGAGGCCGAGGAGGAAGAAGGGGAGGTGGCAG&lt;strong&gt;ATTCGACCGATAGG&lt;/strong&gt;TGGCGTAACTAGATCTTGAGACAAA&#xA;TATCTTGTGGAAAGGACGAAACACCGGTCCGAGCAGAAGAAGAAGTTTAAGAGCTATGCTGGAAACAGCATAGCAAGTTTAAATAAGGCTAGTCCGTTATCAACTTGAAAAAGTGGCACCGAGTCGGTGCTTTTTTT&lt;strong&gt;ATTCGACCGATAGG&lt;/strong&gt;GGTGGCAGGGGAGGCCGAGGAGGAAGAAGGGGAGGTGGCAG&lt;strong&gt;ATTCGACCGATAGG&lt;/strong&gt;TGGCGTAACTAGATCTTGAGACAAA&#xA;TATCTTGTGGAAAGGACGAAACACCGAGTCCGAGCAGAAGAAGAAGTTTAAGAGCTATGCTGGAAACAGCATAGCAAGTTTAAATAAGGCTAGTCCGTTATCAACTTGAAAAAGTGGCACCGAGTCGGTGCTTTTTTT&lt;strong&gt;ATTCGACCGATAGG&lt;/strong&gt;GGTGGCAGGGGAGGCCGAGGAGGAAGAAGGGGAGGTGGCAG&lt;strong&gt;ATTCGACCGATAGG&lt;/strong&gt;TGGCGTAACTAGATCTTGAGACAAA&#xA;TATCTTGTGGAAAGGACGAAACACCGAGTCCGAGCAGAAGAAGAAGTTTAAGAGCTATGCTGGAAACAGCATAGCAAGTTTAAATAAGGCTAGTCCGTTATCAACTTGAAAAAGTGGCACCGAGTCGGTGCTTTTTTT&lt;strong&gt;ATTCGACGATAGG&lt;/strong&gt;GGTGGCAGGGGAGGCCGAGGAGGAAGAAGGGGAGGTGGCAG&lt;strong&gt;ATTCGACCGATAGG&lt;/strong&gt;TGGCGTAACTAGATCTTGAGACAAA&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Note that the first barcode in the fourth sequence is short of one character. I have tried with biopython and regex but it's just too slow given I have 5k barcodes. I am wondering if there is a fast solution available in python or in something like grep, awk or anything else. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="1911" LastActivityDate="2021-01-27T07:19:08.207" Title="Find pattern that is present twice and allow &lt;=2 mismatches on each" Tags="|sequence-alignment|fastq|long-reads|bioawk|barcode|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15265" PostTypeId="1" CreationDate="2021-01-25T21:32:07.913" Score="2" ViewCount="571" Body="&lt;p&gt;From my microarray data I have selected three subsets of probes. I would like to display these subsets as a Euler diagram to visualize the overlaps, but am struggling with inputting the combinations into Eulerr. All three subsets need to be nested within the full set of probes; one subset is fully contained within another, and the third overlaps with the other two. Is there a package into which I could feed the vectors of probe names and let it determine the overlaps for me? Many thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="3505" LastActivityDate="2021-01-26T19:45:59.967" Title="how to easily make complex Euler/Venn diagrams in R?" Tags="|r|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15266" PostTypeId="2" ParentId="15265" CreationDate="2021-01-25T22:02:01.930" Score="2" Body="&lt;p&gt;Since you are in R, consider using the UpsetR library.  It doesn't make Venn diagrams but it helps to visualize overlaps between any number of groups.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://gehlenborglab.org/research/projects/upsetr/#:%7E:text=UpSetR%20is%20an%20R%20package,based%20on%20groupings%20and%20queries&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://gehlenborglab.org/research/projects/upsetr/#:~:text=UpSetR%20is%20an%20R%20package,based%20on%20groupings%20and%20queries&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2021-01-25T22:02:01.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15267" PostTypeId="1" CreationDate="2021-01-26T10:19:37.040" Score="0" ViewCount="43" Body="&lt;p&gt;How do you verify if a RNAseq dataset is unstranded, stranded or revesely stranded from a fastq file?&lt;/p&gt;&#xA;" OwnerUserId="10360" OwnerDisplayName="indigoblueraspberry" LastEditorUserId="29" LastEditDate="2021-01-26T11:17:20.010" LastActivityDate="2021-01-26T11:17:20.010" Title="How do you determine the strandedness of a RNAseq dataset?" Tags="|rna-seq|fastq|" AnswerCount="0" CommentCount="2" ClosedDate="2021-01-27T12:35:36.283" ContentLicense="CC BY-SA 4.0" />
  <row Id="15268" PostTypeId="2" ParentId="15265" CreationDate="2021-01-26T19:11:19.097" Score="0" Body="&lt;p&gt;You might consider different ways of using the &lt;code&gt;venneuler&lt;/code&gt; package. I'm not super thrilled with its usability (or with the accuracy of the overlaps), but you can get it to make custom Venn diagrams.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://stat.ethz.ch/pipermail/r-help/2010-May/239639.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is a thread with some suggestions along these lines.&lt;/p&gt;&#xA;&lt;p&gt;Below is some sample R code that I slightly modified from an old project that I used to make a 3-category Venn diagram.&lt;/p&gt;&#xA;&lt;p&gt;I didn't run this to make sure that it works because of an irritating Java dependency issue on my OSX laptop that I don't feel like fixing right now, but as long as you have a JDK on your machine and can install the packages it &lt;em&gt;should&lt;/em&gt; work.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# note irritating rJava dependency of venneuler&#xA;library(plotrix)&#xA;library(venneuler)&#xA;&#xA;#A-B sharing: 200414&#xA;#B-C sharing: 17561&#xA;#A-C sharing: 16764&#xA;#All share 14201&#xA;&#xA;# get coordinates for circles- replace as needed&#xA;# this function is supposed to plot stuff, but doesn't actually&#xA;ven = venneuler(c(C=48147-17561-16764+14201, &#xA;                  B=347181-17561-200414+14201, &#xA;                  A=325971-200414-16764+14201, &#xA;                  &amp;quot;C&amp;amp;B&amp;quot;=17561, &amp;quot;B&amp;amp;A&amp;quot;=200414, &amp;quot;C&amp;amp;A&amp;quot;=16764, &amp;quot;A&amp;amp;B&amp;amp;C&amp;quot;=14201))&#xA;&#xA;cats = c('A','B','C')&#xA;&#xA;# note- this following only shows up in the .pdf printer, not in the X11 utility plotter&#xA;pdf('venn.pdf')&#xA;plot(c(-.5,1.5), c(-.5,1.5), type='n', xaxt='n', yaxt='n', bty='n', xlab='', ylab='')&#xA;draw.circle(ven&lt;span class=&quot;math-container&quot;&gt;$center[1,1], ven$&lt;/span&gt;center[1,2], ven&lt;span class=&quot;math-container&quot;&gt;$diameters[1],&#xA;        col = hsv(1,1,1,.5))&#xA;draw.circle(ven$&lt;/span&gt;center[2,1], ven&lt;span class=&quot;math-container&quot;&gt;$center[2,2], ven$&lt;/span&gt;diameters[2], &#xA;            col = hsv(.5,1,1,.5))&#xA;draw.circle(ven&lt;span class=&quot;math-container&quot;&gt;$center[3,1], ven$&lt;/span&gt;center[3,2], ven$diameters[3],&#xA;            col = hsv(.1,1,1,.5))&#xA;&#xA;text(ven&lt;span class=&quot;math-container&quot;&gt;$center[1,1]+.065, ven$&lt;/span&gt;center[1,2], labels=cats[3], cex=1.3)&#xA;text(ven&lt;span class=&quot;math-container&quot;&gt;$center[2,1], ven$&lt;/span&gt;center[2,2]-.2, labels=cats[2], cex=1.3)&#xA;text(ven&lt;span class=&quot;math-container&quot;&gt;$center[3,1], ven$&lt;/span&gt;center[3,2]+.2, labels=cats[1], cex=1.3)&#xA;&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2021-01-26T19:45:59.967" LastActivityDate="2021-01-26T19:45:59.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15269" PostTypeId="1" AcceptedAnswerId="15270" CreationDate="2021-01-26T19:51:58.263" Score="1" ViewCount="1662" Body="&lt;p&gt;I have 10 &lt;code&gt;.sam&lt;/code&gt; files after my &lt;code&gt;bowtie2&lt;/code&gt; alignment on ten single-pair sequences. I would like to build a graph based on that output data, however I don't see any other solution but just writing the alignment percentage manually after each iteration.&lt;/p&gt;&#xA;&lt;p&gt;Is there a tool that can help me to build a &lt;code&gt;.csv&lt;/code&gt; file so I can manipulate the output data more easily?&lt;/p&gt;&#xA;" OwnerUserId="6948" LastEditorUserId="6251" LastEditDate="2021-01-29T10:29:06.780" LastActivityDate="2021-01-29T10:29:06.780" Title="How can I extract information from .sam files?" Tags="|samtools|sam|bowtie2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15270" PostTypeId="2" ParentId="15269" CreationDate="2021-01-26T20:04:50.437" Score="2" Body="&lt;p&gt;Use &lt;a href=&quot;http://www.htslib.org/doc/samtools-flagstat.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;samtools flagstat&lt;/code&gt;&lt;/a&gt; with option &lt;code&gt;-O tsv&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Using &lt;strong&gt;-O tsv&lt;/strong&gt; selects a tab-separated values format that can easily be imported into spreadsheet software. In this format the first column contains the values for QC-passed reads, the second column has the values for QC-failed reads and the third contains the category names.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;SEE ALSO:&lt;/strong&gt;&lt;br /&gt;&#xA;&lt;a href=&quot;https://www.biostars.org/p/12475/&quot; rel=&quot;nofollow noreferrer&quot;&gt;What Does Samtools Flagstat Results Mean?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;INSTALLATION:&lt;/strong&gt;&lt;br /&gt;&#xA;Some old versions of &lt;code&gt;samtools&lt;/code&gt; do not have option &lt;code&gt;-O tsv&lt;/code&gt;. To install a recent versions of &lt;code&gt;samtools&lt;/code&gt;, use either:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://anaconda.org/bioconda/samtools&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;conda&lt;/code&gt;&lt;/a&gt; (macOS, Linux) (but see also: &lt;a href=&quot;https://www.biostars.org/p/455593/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tutorial: installing samtools with conda - overcoming two common errors&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda create --name samtools samtools&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or &lt;a href=&quot;https://formulae.brew.sh/formula/samtools&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;brew&lt;/code&gt;&lt;/a&gt; (macOS):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;brew install samtools&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6251" LastEditorUserId="6251" LastEditDate="2021-01-26T20:53:14.153" LastActivityDate="2021-01-26T20:53:14.153" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="15272" PostTypeId="2" ParentId="15263" CreationDate="2021-01-27T07:19:08.207" Score="1" Body="&lt;p&gt;This isn't perfect but you can build on it to deal with any issues (ie overalpping hits). Also if you need it faster just chunk it and run in parallel threads.&lt;/p&gt;&#xA;&lt;p&gt;I just copy pasted your examples seqs into files to read (called them seqs and code).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#need blast installed &#xA;import subprocess&#xA;import pandas as pd&#xA;&#xA;path = '/Users/ur/dir/'&#xA;#make a blast db&#xA;with open (path + 'seqs.txt', 'r') as fin:&#xA;    with open (path + 'seqs.fa', 'w') as fout:&#xA;        for i, line in enumerate(fin):&#xA;            fout.write('&amp;gt;'+str(i)+'\n')&#xA;            fout.write(line)&#xA;            &#xA;subprocess.run([&#xA;    'makeblastdb',&#xA;    '-in', path + 'seqs.fa',&#xA;    '-dbtype', 'nucl'&#xA;])&#xA;&#xA;#use blast&#xA;subprocess.run([&#xA;    'blastn',&#xA;    '-task', 'blastn-short',&#xA;    '-query', path + 'code.txt',&#xA;    '-db', path + 'seqs.fa',&#xA;    '-out', path + 'out.txt',&#xA;    '-outfmt', '6'&#xA;])&#xA;&#xA;#grab seqs with 2 hits (ie ~2x len(query)&#xA;df = pd.read_csv(&#xA;    path + 'out.txt',&#xA;    sep='\t',&#xA;    header=None,&#xA;    names=['query', 'db_name', 'percent','len'],&#xA;    usecols = [0,1,2,3]&#xA;)&#xA;df = df.groupby('db_name').agg('sum')&#xA;print(*list(df.query('len &amp;gt; 27').index))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I this case they are all hits. If you look at the blast output you'll see one hit was fragmented (blast isn't perfect with small seqs). But total len works to resolve this.&lt;/p&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-01-27T07:19:08.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15273" PostTypeId="1" CreationDate="2021-01-27T07:26:50.170" Score="3" ViewCount="128" Body="&lt;p&gt;I bought a &amp;quot;Whole Genome Sequence&amp;quot; kit, which gave me the following VCF files (this service did not include any analysis, just the raw data):&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;cnv.vcf&lt;/code&gt;&#xA;&lt;code&gt;indel.vcf&lt;/code&gt;&#xA;&lt;code&gt;snp.vcf&lt;/code&gt;&#xA;&lt;code&gt;sv.vcf&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;My understanding is that these files show the differences of my DNA relative to a reference human genome (hg19 in my case)&lt;/p&gt;&#xA;&lt;p&gt;What tools and resources can I use to find out which known alleles/variants of genes I have?&lt;/p&gt;&#xA;&lt;h2&gt;An example: Blood type&lt;/h2&gt;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://en.wikipedia.org/wiki/ABO_(gene)#Common_alleles&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;, there are 6 common alleles of the &lt;a href=&quot;http://uswest.ensembl.org/Homo_sapiens/Gene/Splice?db=core;g=ENSG00000123095;r=12:26120030-26125037&quot; rel=&quot;nofollow noreferrer&quot;&gt;ABO gene&lt;/a&gt;, and which alleles I have determine my blood type.&lt;/p&gt;&#xA;&lt;p&gt;Looking at my VCF results, I have these SNPs and INDELs within that gene:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;removed&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Given this, how would I figure out my blood type?&lt;/p&gt;&#xA;&lt;p&gt;I understand that my data may not be 100% accurate, and that many phenotypes are the result of more than one gene, environmental factors, or are not well studied at all, but this should work for some simple things ie blood type, &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/17185601/&quot; rel=&quot;nofollow noreferrer&quot;&gt;COMT&lt;/a&gt;, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4096202/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BHLHE41&lt;/a&gt;, &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/16522833/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CYP1A2&lt;/a&gt;, etc&lt;/p&gt;&#xA;" OwnerUserId="10358" LastEditorUserId="3694" LastEditDate="2023-09-12T15:50:30.730" LastActivityDate="2023-09-12T15:50:30.730" Title="What tools can I use to look up my alleles, genotypes, or phenotypes from my sequenced DNA (WGS)?" Tags="|genome|vcf|public-databases|wgs|human-genome|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="15275" PostTypeId="1" CreationDate="2021-01-27T19:49:11.323" Score="0" ViewCount="439" Body="&lt;p&gt;I am trying to write a code for importing sra files as fastq files in Rstudio (for obtaining and aligning RNA-seq reads from public repositories). I keep getting &amp;quot;Error in if (is.na(sra_acc$run[i])) { : argument is of length zero&amp;quot; right after I run this code. Any suggestions on what might be going wrong? My code is as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(SRAdb)&#xA;sqlfile &amp;lt;-'SRAmetadb.sqlite'&#xA;if(!file.exists('SRAmetadb.sqlite')) sqlfile &amp;lt;&amp;lt;- getSRAdbFile()&#xA;sra_con &amp;lt;- dbConnect(SQLite(),sqlfile)&#xA;sraInf &amp;lt;- getSRAinfo(&amp;quot;SRP045534&amp;quot;,sra_con, sraType=&amp;quot;sra&amp;quot;)&#xA;&#xA;Error in if (is.na(sra_acc$run[i])) { : argument is of length zero&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10364" LastEditorUserId="5077" LastEditDate="2021-01-27T20:33:16.693" LastActivityDate="2022-04-10T21:26:37.237" Title="Error in if (is.na(sra_acc$run[i])) { : argument is of length zero" Tags="|r|sratoolkit|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15276" PostTypeId="1" CreationDate="2021-01-27T19:50:41.547" Score="0" ViewCount="39" Body="&lt;p&gt;Can anyone recommend a protocol for building the 3D structure of a small molecule &lt;em&gt;de novo&lt;/em&gt;, given only a 2D rendering? So far my idea has been to build the molecule using the fragment builder in PyMol, then make conformers using Frog2. But I have no idea how realistic the starting structure is from PyMol and how exhaustively the conformers are sampled in Frog2.&lt;/p&gt;&#xA;&lt;p&gt;The downstream application of this would be to dock the ligand into a protein binding pocket. If anyone has a more rigorous protocol for making sure all the conformational space is fully sampled, it would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="388" LastEditorUserId="388" LastEditDate="2021-01-28T17:26:35.770" LastActivityDate="2021-01-28T17:26:35.770" Title="Building small molecules de novo" Tags="|docking|computational-biochemistry|small-molecules|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15277" PostTypeId="2" ParentId="15256" CreationDate="2021-01-28T00:48:50.603" Score="1" Body="&lt;p&gt;I took a quick look at the &lt;a href=&quot;https://github.com/ablab/quast/blob/master/quast_libs/contigs_analyzer.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;quast code&lt;/a&gt; that I believe is responsible here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def analyze_coverage(ref_aligns, reference_chromosomes, ns_by_chromosomes, used_snps_fpath):&#xA;    indels_info = IndelsInfo()&#xA;    genome_mapping = {}&#xA;    for chr_name, chr_len in reference_chromosomes.items():&#xA;        genome_mapping[chr_name] = [0] * (chr_len + 1)&#xA;    with open(used_snps_fpath, 'w') as used_snps_f:&#xA;        for chr_name, aligns in ref_aligns.items():&#xA;            for align in aligns:&#xA;                ref_pos, ctg_pos = align.s1, align.s2&#xA;                strand_direction = 1 if align.s2 &amp;lt; align.e2 else -1&#xA;                for op in parse_cs_tag(align.cigar):&#xA;                    if op.startswith(':'):&#xA;                        n_bases = int(op[1:])&#xA;                    else:&#xA;                        n_bases = len(op) - 1&#xA;                    if op.startswith('*'):&#xA;                        ref_nucl, ctg_nucl = op[1].upper(), op[2].upper()&#xA;                        if ctg_nucl != 'N' and ref_nucl != 'N':&#xA;                            # MISMATCHES INCREMENTED HERE&#xA;                            indels_info.mismatches += 1&#xA;                            if qconfig.show_snps:&#xA;                                used_snps_f.write('%s\t%s\t%d\t%s\t%s\t%d\n' % (chr_name, align.contig, ref_pos, ref_nucl, ctg_nucl, ctg_pos))&#xA;                        ref_pos += 1&#xA;                        ctg_pos += 1 * strand_direction&#xA;                    elif op.startswith('+'):&#xA;                        # INDELS INCREMENTED HERE&#xA;                        indels_info.indels_list.append(n_bases)&#xA;                        indels_info.insertions += n_bases&#xA;                        if qconfig.show_snps and n_bases &amp;lt; qconfig.MAX_INDEL_LENGTH:&#xA;                            ref_nucl, ctg_nucl = '.', op[1:].upper()&#xA;                            used_snps_f.write('%s\t%s\t%d\t%s\t%s\t%d\n' % (chr_name, align.contig, ref_pos, ref_nucl, ctg_nucl, ctg_pos))&#xA;                        ctg_pos += n_bases * strand_direction&#xA;                    elif op.startswith('-'):&#xA;                        indels_info.indels_list.append(n_bases)&#xA;                        indels_info.deletions += n_bases&#xA;                        if qconfig.show_snps and n_bases &amp;lt; qconfig.MAX_INDEL_LENGTH:&#xA;                            ref_nucl, ctg_nucl = op[1:].upper(), '.'&#xA;                            used_snps_f.write('%s\t%s\t%d\t%s\t%s\t%d\n' % (chr_name, align.contig, ref_pos, ref_nucl, ctg_nucl, ctg_pos))&#xA;                        ref_pos += n_bases&#xA;                    else:&#xA;                        ref_pos += n_bases&#xA;                        ctg_pos += n_bases * strand_direction&#xA;                if align.s1 &amp;lt; align.e1:&#xA;                    for pos in range(align.s1, align.e1 + 1):&#xA;                        genome_mapping[align.ref][pos] = 1&#xA;                else:&#xA;                    for pos in range(align.s1, len(genome_mapping[align.ref])):&#xA;                        genome_mapping[align.ref][pos] = 1&#xA;                    for pos in range(1, align.e1 + 1):&#xA;                        genome_mapping[align.ref][pos] = 1&#xA;            for i in ns_by_chromosomes[align.ref]:&#xA;                genome_mapping[align.ref][i] = 0&#xA;&#xA;    covered_ref_bases = sum([sum(genome_mapping[chrom]) for chrom in genome_mapping])&#xA;    return covered_ref_bases, indels_info&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This indicates to me that it simply iterates over contigs (or rather, called SNPs from contig alignments to a reference) and counts up the number of mismatches/indels that occur in total across all aligned contigs.&lt;/p&gt;&#xA;&lt;p&gt;This suggests to me that if haplotigs are present, the areas that they cover will be double-counted (or n-counted where there are n haplotigs, assuming that all haplotigs covering a reference position have roughly equal divergence).&lt;/p&gt;&#xA;&lt;p&gt;From this point of view, I think that it is heuristically reasonable to divide by genome fraction. However, if you want something really rigorous you might consider digging into the quast &lt;code&gt;used_snps&lt;/code&gt; file that this code iterates over to decide which SNPs/indels/whatever you think are worth keeping.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-01-28T00:48:50.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15278" PostTypeId="2" ParentId="9021" CreationDate="2021-01-28T01:46:19.450" Score="1" Body="&lt;p&gt;So, after a year I came out with the solution - as M__ kindly suggested, using Clustal Omega.&lt;/p&gt;&#xA;&lt;p&gt;First, align usign muscle:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;muscle -in fasta -quiet -out alignment&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then, generate percent distance matrix using clustal omega:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;clustalo --infile alignment --distmat-out output --percent-id --full&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="10365" LastActivityDate="2021-01-28T01:46:19.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15279" PostTypeId="2" ParentId="14915" CreationDate="2021-01-28T03:33:54.757" Score="0" Body="&lt;p&gt;Your question isn't well formatted, for example list is used as a variable name where it should be reserved as a keyword...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print (&amp;quot;&amp;gt;&amp;quot;+list.group(1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, the answer you're after is going to be something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print (&amp;quot;&amp;gt;&amp;quot;+set(name_of_list.group(1)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Anytime you want to get the unique elements of a list, just wrap it in set()&lt;/p&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-01-28T03:33:54.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15280" PostTypeId="2" ParentId="5280" CreationDate="2021-01-28T03:47:39.187" Score="1" Body="&lt;p&gt;You just need to check the flag for suplementary mapping. If there is suplementary mapping then use the coordinates; if they are all the same the seq is the same and you just have one event, if different reads have different suplementary mapping locations then there's multiple events. The problem is more complex than this though so just use an existing tool.&lt;/p&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-01-28T03:47:39.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15281" PostTypeId="1" CreationDate="2021-01-28T06:53:12.807" Score="0" ViewCount="517" Body="&lt;p&gt;I have a &lt;a href=&quot;https://drive.google.com/file/d/18ImTLxSIv2o85TdwNcJOoQD64ibOmJUK/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;genbank file&lt;/a&gt; that I have been trying to run in HMMER using the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;hmmscan --domtblout NC_000962.3.final.gbk.out -E 0.000001 prot_profile.hmm NC_000962.3.final.gbk&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, since the file is too big for HMMER, I get the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Fatal exception (source file p7_pipeline.c, line 697): &#xA;Target sequence length &amp;gt; 100K, over comparison pipeline limit. &#xA;(Did you mean to use nhmmer/nhmmscan?) &#xA;Aborted (core dumped)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I need to split this file into smaller files for the HMMER run. The problem is, I would need the &lt;em&gt;ORIGIN&lt;/em&gt; field (containing the nucleotide sequence) at the bottom of the file split into relevant chunks into the subfile as well. This is where I am getting stuck.&lt;/p&gt;&#xA;&lt;p&gt;I have already looked at &lt;em&gt;GenBank Parser&lt;/em&gt; and &lt;em&gt;seqretsplit&lt;/em&gt; to no avail.&lt;/p&gt;&#xA;&lt;p&gt;I would greatly appreciate some help with this. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="10366" LastActivityDate="2021-01-28T23:23:58.257" Title="Splitting a GenBank file into smaller files" Tags="|genbank|hmmer|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15282" PostTypeId="2" ParentId="14915" CreationDate="2021-01-28T08:57:04.337" Score="1" Body="&lt;p&gt;Perhaps itertools might be an option? From the itertools page / moreitertools project&#xA;&lt;a href=&quot;https://docs.python.org/3/library/itertools.html#itertools-recipes&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.python.org/3/library/itertools.html#itertools-recipes&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;def unique_everseen(iterable, key=None):&#xA;    &amp;quot;List unique elements, preserving order. Remember all elements ever seen.&amp;quot;&#xA;    # unique_everseen('AAAABBBCCDAABBB') --&amp;gt; A B C D&#xA;    # unique_everseen('ABBCcAD', str.lower) --&amp;gt; A B C D&#xA;    seen = set()&#xA;    seen_add = seen.add&#xA;    if key is None:&#xA;        for element in filterfalse(seen.__contains__, iterable):&#xA;            seen_add(element)&#xA;            yield element&#xA;    else:&#xA;        for element in iterable:&#xA;            k = key(element)&#xA;            if k not in seen:&#xA;                seen_add(k)&#xA;                yield element&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10367" LastActivityDate="2021-01-28T08:57:04.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15283" PostTypeId="2" ParentId="15276" CreationDate="2021-01-28T09:59:59.630" Score="1" Body="&lt;p&gt;Most universities have a ChemDraw site lincence, so you may want to check that for drawing a non–3D-embedded molecule. Alternatively there are many other programs. PyMOL is not really a good choice. Also there are online drawing tools everywhere, where you can download a SMILES (see below).&lt;/p&gt;&#xA;&lt;p&gt;Chemical formula is the wrong way to go, unless you are &amp;quot;enumerating&amp;quot;.&#xA;The chemical formula of glucose is &lt;code&gt;C6H12O6&lt;/code&gt;. This is the same for all hexoses. What you want is a SMILES. This string gives the connectivity of the atoms. For D-glucose it's &lt;code&gt;OC[C@H]1OC(O)[C@H](O)[C@@H](O)[C@@H]1O&lt;/code&gt;, the &lt;code&gt;@&lt;/code&gt; symbols are chiral centres.&#xA;In Cheminformatics making a list of all possible compound with n-carbons is a big deal, this is called enumeration and you likely don't have to reinvent wheel. Furthermore, there are datasets that are filtered by the Lipinski rule.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://gdb.unibe.ch/downloads/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GDB datasets&lt;/a&gt; (if you don't care if you cannot buy it)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://zinc.docking.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zinc-DB&lt;/a&gt;  (if you care if you can buy it)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://enamine.net/library-synthesis/real-compounds/real-database&quot; rel=&quot;nofollow noreferrer&quot;&gt;Enamine Real&lt;/a&gt; (if you care if you can buy it reasonably)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-01-28T09:59:59.630" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15284" PostTypeId="1" CreationDate="2021-01-28T11:16:07.890" Score="0" ViewCount="363" Body="&lt;p&gt;I have a dataset that looks something like this&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;&lt;/th&gt;&#xA;&lt;th&gt;Treatment1&lt;/th&gt;&#xA;&lt;th&gt;Treatment 2&lt;/th&gt;&#xA;&lt;th&gt;Control&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Sample1&lt;/td&gt;&#xA;&lt;td&gt;3.23.&lt;/td&gt;&#xA;&lt;td&gt;0.87.&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Sample2&lt;/td&gt;&#xA;&lt;td&gt;1.71.&lt;/td&gt;&#xA;&lt;td&gt;1.&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Sample3&lt;/td&gt;&#xA;&lt;td&gt;2.88.&lt;/td&gt;&#xA;&lt;td&gt;5.65&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Sample4&lt;/td&gt;&#xA;&lt;td&gt;0.77.&lt;/td&gt;&#xA;&lt;td&gt;1.34&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;The numbers describe a fold change with respect to the control (where the control is always set to 1).&lt;/p&gt;&#xA;&lt;p&gt;Does it make sense to compute values such as averages, sums, and differences (e.g. the average fold change for treatment1)? And yes, wary that doing things on log fold-change is not a thing, but how about &lt;em&gt;just&lt;/em&gt; fold change?&lt;/p&gt;&#xA;&lt;p&gt;By extension, does it also make sense to compute something like Euclidean distances (e.g. fold change vector of Sample1 vs. Sample2) for clustering fold change data? I think clustering on counts/TPM makes more sense, but wasn't sure about doing it on fold changes.&lt;/p&gt;&#xA;&lt;p&gt;Something tells me that it's not possible because the fold changes are relative values as opposed to measured numbers. I've opted for looking at correlations (e.g. Spearman) for now though; are there alternative metrics/strategies to consider?&lt;/p&gt;&#xA;&lt;p&gt;If there are some good papers talking about clustering and manipulating fold change data I'd be equally grateful, thanks!&lt;/p&gt;&#xA;" OwnerUserId="10368" LastActivityDate="2021-01-29T07:50:05.777" Title="Clustering RNAseq fold-change data" Tags="|rna-seq|differential-expression|clustering|fold-change|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15285" PostTypeId="2" ParentId="14915" CreationDate="2021-01-28T11:45:42.850" Score="0" Body="&lt;p&gt;I use the following code. It doesn't only remove repeated sequences but similar sequences (with a quite simplistic approach).&#xA;I use it just sporadically and I guess it can be largely depurated. Remove the debug prints if these annoy you.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import sys&#xA;import requests &#xA;import shlex, subprocess&#xA;import random&#xA;from Bio import SeqIO&#xA;import editdistance&#xA;&#xA;if len(sys.argv)&amp;lt;3:&#xA;    print(&amp;quot;Uso: &amp;quot;+sys.argv[0]+&amp;quot; input_fasta output_fasta&amp;quot;)&#xA;    sys.exit(0)&#xA;&#xA;savefile = sys.argv[2]&#xA;content_file = sys.argv[1]&#xA;&#xA;class Sequence:&#xA;    id=&amp;quot;&amp;quot;&#xA;    sequence=&amp;quot;&amp;quot;&#xA;&#xA;def are_similar(seq1, seq2):&#xA;    l1 = len(seq1.sequence)&#xA;    l2 = len(seq2.sequence)&#xA;    if abs(l1-l2) &amp;lt; l1/40:&#xA;        d = editdistance.eval(seq1.sequence[0:250], seq2.sequence[0:250])&#xA;        #d = editdistance.eval(seq1.sequence, seq2.sequence)&#xA;        print(seq1.id+&amp;quot; &amp;quot;+seq2.id+&amp;quot; -&amp;gt; &amp;quot;+str(d))&#xA;        #if d &amp;lt; l1/20:&#xA;        if d &amp;lt; 50:&#xA;            return True&#xA;    return False&#xA;&#xA;def find_repeats(sequences, seq, repeated):&#xA;    b=False&#xA;    for seq2 in sequences:&#xA;        if(b and not seq2.id in repeated):&#xA;            if are_similar(seq, seq2):&#xA;                repeated.add(seq2.id)        &#xA;        if seq2.id==seq.id:&#xA;            b=True&#xA;&#xA;sequences=[]&#xA;&#xA;res=&amp;quot;&amp;quot;&#xA;&#xA;for record in SeqIO.parse(content_file, &amp;quot;fasta&amp;quot;):&#xA;    seq=Sequence()&#xA;    seq.id=record.id&#xA;    seq.sequence=record.seq&#xA;    sequences.append(seq)&#xA;&#xA;repeated_sequences=set()&#xA;for seq in sequences:&#xA;    find_repeats(sequences, seq, repeated_sequences)&#xA;&#xA;print(repeated_sequences)&#xA;&#xA;def mySortFunc(e):&#xA;    return len(e.sequence)&#xA;&#xA;sequences.sort(key=mySortFunc)&#xA;&#xA;for seq in sequences:&#xA;    b= seq.id in repeated_sequences&#xA;    if not b:&#xA;        res+=&amp;quot;&amp;gt;&amp;quot;+seq.id+&amp;quot;\n&amp;quot;&#xA;        res+=str(seq.sequence)+&amp;quot;\n&amp;quot;&#xA;&#xA;output_file = open(savefile, &amp;quot;w&amp;quot;)&#xA;&#xA;output_file.write(res)&#xA;&#xA;output_file.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;EDIT: I see that's not exactly what OP is asking ...&lt;/p&gt;&#xA;" OwnerUserId="5190" LastEditorUserId="5190" LastEditDate="2021-01-29T18:24:15.830" LastActivityDate="2021-01-29T18:24:15.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15286" PostTypeId="2" ParentId="15281" CreationDate="2021-01-28T12:05:13.113" Score="1" Body="&lt;p&gt;You can use NCBI &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;EntrezDirect&lt;/a&gt; to download data in GenBank format for a specific region of the sequence as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;efetch -db nuccore -id NC_000962.3 -format gb -seq_start 1 -seq_stop 99000 &amp;gt; seq.gb&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2021-01-28T12:05:13.113" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15288" PostTypeId="2" ParentId="14915" CreationDate="2021-01-28T19:29:37.060" Score="1" Body="&lt;p&gt;I would use a &lt;code&gt;dict&lt;/code&gt; so that the lookup time is constant. If you are creating an array with all the unique gene IDs, then checking if a gene is in that array, your runtime is O(N&lt;sup&gt;2&lt;/sup&gt;). When you create an entry in the &lt;code&gt;dict&lt;/code&gt;, the time to access is constant, and if the entry is already present, it will just overwrite it. Something like this should do it for you:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;    &#xA;genes = {}&#xA;fasta_file = sys.argv[1]&#xA;for entry in SeqIO.parse( fasta_file, &amp;quot;fasta&amp;quot; ):&#xA;    genes[ entry.id ] = entry.seq&#xA;    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then you can write a new, unique fasta file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with open('unique_genes.fasta', 'w') as out:&#xA;    for key,value in genes.items():&#xA;        out.write( '&amp;gt;%s\n%s\n'%(key,value) )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="388" LastActivityDate="2021-01-28T19:29:37.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15289" PostTypeId="2" ParentId="15281" CreationDate="2021-01-28T22:50:40.050" Score="1" Body="&lt;p&gt;I don't have time to give you a polished fully functional version but this will get you started. The next step will be a bit harder as you will need to capture the start of the first CDS in the chunk and end of the last CDS instead of just hardcoding as I have here.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fout = open('/path/first3.gbk','w')&#xA;for record in SeqIO.parse('/path/NC_000962.3.final.gbk','genbank'):&#xA;    for i, feature in enumerate(record.features):&#xA;        new_record = record&#xA;        new_record.seq = new_record.seq[0:4437]&#xA;        new_record.features = new_record.features[:3]&#xA;        if i &amp;gt; 2:&#xA;            SeqIO.write(new_record, fout, 'genbank')&#xA;            fout.close()&#xA;            break&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7153" LastEditorUserId="7153" LastEditDate="2021-01-28T23:23:58.257" LastActivityDate="2021-01-28T23:23:58.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15290" PostTypeId="1" CreationDate="2021-01-28T10:33:19.303" Score="2" ViewCount="820" Body="&lt;p&gt;I have annotated files from annvar and I changed them into MAF by using&#xA;&lt;code&gt;annovarToMaf&lt;/code&gt; from &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/maftools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;‘maftools’&lt;/a&gt;. Then I used &lt;code&gt;read.maf&lt;/code&gt; to read them into R and then I used &lt;code&gt;merge_mafs&lt;/code&gt; to integrate them together inside the last file.&lt;/p&gt;&#xA;&lt;p&gt;I had several data frames but it separated the synonymous mutations (&lt;code&gt;maf.silent&lt;/code&gt;) from the non-synonymous mutations, and I want to create an oncoplot using the &lt;code&gt;oncoplot&lt;/code&gt; function for both synonymous mutations from the non-synonymous mutations.&lt;/p&gt;&#xA;&lt;p&gt;But once I use this command I don't know how to integrate both data frames inside of the MAF file.&lt;/p&gt;&#xA;&lt;p&gt;How can I create the oncoplot using &lt;code&gt;oncoplot&lt;/code&gt; for both synonymous mutations and non-synonymous mutations?&lt;/p&gt;&#xA;" OwnerDisplayName="Noorhan Kamal" LastActivityDate="2023-03-28T17:46:10.973" Title="How do I include silent mutations in an oncoplot using maftools?" Tags="|r|bioconductor|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15291" PostTypeId="2" ParentId="15284" CreationDate="2021-01-29T07:50:05.777" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Does it make sense to compute values such as averages, sums, and differences (e.g. the average fold change for treatment1)? And yes, wary that doing things on log fold-change is not a thing, but how about just fold change?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is problematic, if you want to compare treatments then compute the actual fold change and its significance. You can't really base anything off of the fold-change (log or otherwise) alone.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;does it also make sense to compute something like Euclidean distances for clustering fold change data&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It's not unheard of to do this. The problem is you lose all information about how reliable those fold-change estimates are. This is something maintained when you cluster sample-level information (e.g., TPM or robust log transformed data). Not that using variance-stabilized data for clustering will work better, since it tamps down on how much highly-expressed genes influence your clustering (assuming you don't want that to occur).&lt;/p&gt;&#xA;&lt;p&gt;Have a look at the DESeq2 vignette for some nice examples of what good practices are.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-01-29T07:50:05.777" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15292" PostTypeId="1" AcceptedAnswerId="17999" CreationDate="2021-01-29T10:13:06.180" Score="4" ViewCount="5762" Body="&lt;p&gt;I'm working in bioinformatics, but my computational skills far outstrip my knowledge of biology or genomics. So forgive the noobish question.&lt;/p&gt;&#xA;&lt;p&gt;According to the &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VCF specification&lt;/a&gt;, the &lt;code&gt;INFO&lt;/code&gt; column for each coordinate &lt;code&gt;[CHROM, POS]&lt;/code&gt; can contain values for&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;AC :&lt;/strong&gt; allele count in genotypes, for each ALT allele, in the same order as listed&lt;br /&gt;&#xA;&lt;strong&gt;AF :&lt;/strong&gt; allele frequency for each ALT allele in the same order as listed: use this when estimated from primary data, not called genotypes&lt;br /&gt;&#xA;&lt;strong&gt;AN :&lt;/strong&gt; total number of alleles in called genotypes&lt;/p&gt;&#xA;&lt;p&gt;For example, I'm working with the the &lt;a href=&quot;http://db.systemsbiology.net/kaviar/Kaviar.downloads.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;VCF file you can download from Kaviar&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I think I understand how to calculate those numbers from raw sequences from a pileup. If for a given coordinate &lt;code&gt;[CHROM, POS]&lt;/code&gt;, &lt;span class=&quot;math-container&quot;&gt;$N_0$&lt;/span&gt; is the number of hits for the reference allele, &lt;span class=&quot;math-container&quot;&gt;$\textrm{AC}_i \equiv N_i$&lt;/span&gt; is the number of hits for the &lt;span class=&quot;math-container&quot;&gt;$i^{th}$&lt;/span&gt; variant (maybe only one), and AN is the total number of alleles counted: &lt;span class=&quot;math-container&quot;&gt;$\textrm{AN} \equiv \sum_0^n N_i$&lt;/span&gt;, then the allele frequency for the &lt;span class=&quot;math-container&quot;&gt;$i^{th}$&lt;/span&gt; variant is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\textrm{AF}_i = \frac{\textrm{AC}_i}{\textrm{AN}} = \frac{N_i}{\sum_0^n N_i}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So the phrase &amp;quot;total number of alleles in the dataset&amp;quot; would mean &amp;quot;total number of hits to coordinate &lt;code&gt;[CHROM, POS]&lt;/code&gt; for &lt;em&gt;any&lt;/em&gt; allele. And by &amp;quot;hit&amp;quot; I mean an aligned sequence fragment that includes that coordinate.&lt;/p&gt;&#xA;&lt;p&gt;But I may not be taking into account the caveat for AF: &amp;quot;use this when estimated from primary data, not called genotypes.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Am I correctly understanding AF, AC, and AN in VCF files?&lt;/p&gt;&#xA;" OwnerUserId="4758" LastEditorUserId="4758" LastEditDate="2021-02-01T01:04:21.397" LastActivityDate="2023-04-10T08:03:55.123" Title="Allele Count and Allele Frequency in VCF files" Tags="|vcf|allele-frequency|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15293" PostTypeId="1" AcceptedAnswerId="15308" CreationDate="2021-01-29T13:05:53.440" Score="2" ViewCount="61" Body="&lt;p&gt;Is it possible to fit branch models &lt;a href=&quot;https://link.springer.com/article/10.1007/PL00006320&quot; rel=&quot;nofollow noreferrer&quot;&gt;(Yang and Nielsen, 1998)&lt;/a&gt; in &lt;a href=&quot;https://bitbucket.org/Davydov/godon/src&quot; rel=&quot;nofollow noreferrer&quot;&gt;godon&lt;/a&gt;? That is, the model which averages over all sites, as opposed to the branch site models?&lt;/p&gt;&#xA;" OwnerUserId="2841" LastEditorUserId="2841" LastEditDate="2021-02-02T11:50:13.317" LastActivityDate="2021-02-02T12:50:37.607" Title="Fitting branch models with godon" Tags="|phylogenetics|codon|godon|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15295" PostTypeId="1" CreationDate="2021-01-29T17:42:08.627" Score="2" ViewCount="234" Body="&lt;p&gt;I know how to generally pull down information for a UniProt entry using the REST API, for example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.uniprot.org/uniprot/?query=id:Q9UJL9&amp;amp;sort=score&amp;amp;columns=id,go(molecular%20function)&amp;amp;format=tab&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uniprot/?query=id:Q9UJL9&amp;amp;sort=score&amp;amp;columns=id,go(molecular%20function)&amp;amp;format=tab&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm looking at &lt;a href=&quot;https://www.uniprot.org/uniprot/Q9UBU3&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; entry as an example. I'm able to pull down most of the attributes I want with a variation of the API query above, however, I'm stuck on one part.&lt;/p&gt;&#xA;&lt;p&gt;For all human proteins, I want to pull down the peptide sequences that come from that protein.&lt;/p&gt;&#xA;&lt;p&gt;In the example above, it would be the sequences of the peptides associated with this section:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HcDzq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HcDzq.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can this be done with the API or do I have to come with a scraping method for all human proteins where I (1) get the protein sequence of each entry, (2) get this table, (3) calculate where the peptides start and end from the sequence in step 1?&#xA;I feel like this alternative will be much slower, so I want to check I'm not missing anything obvious.&lt;/p&gt;&#xA;" OwnerUserId="10328" LastActivityDate="2021-03-08T08:36:09.570" Title="How to extract all peptide sequences for a UniProt protein entry with an API?" Tags="|uniprot|api|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15296" PostTypeId="2" ParentId="14978" CreationDate="2021-01-29T23:24:13.793" Score="0" Body="&lt;p&gt;You can use the kraken database. It will classify your reads based on what organism it matches with. Then can write a custom script to extract the reads you are interested in for downstream analysis.&lt;/p&gt;&#xA;" OwnerUserId="10184" LastEditorUserId="10184" LastEditDate="2021-01-29T23:34:42.847" LastActivityDate="2021-01-29T23:34:42.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15298" PostTypeId="2" ParentId="11489" CreationDate="2021-01-29T23:32:03.450" Score="0" Body="&lt;p&gt;You can perform genome annotation using tools like prokka or PGAP. Using some python scripts  you can extract the predicted 16S and blast to confirm.&lt;/p&gt;&#xA;" OwnerUserId="10184" LastEditorUserId="10184" LastEditDate="2022-08-11T09:55:37.173" LastActivityDate="2022-08-11T09:55:37.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15302" PostTypeId="1" AcceptedAnswerId="15493" CreationDate="2021-01-30T18:02:00.687" Score="1" ViewCount="177" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GQgx9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GQgx9.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;This is a revised version of my original question, edited to include the portions already answered by @Maximilian Press.&lt;/p&gt;&#xA;&lt;h1&gt;Goal:&lt;/h1&gt;&#xA;&lt;h2&gt;Find SNP and indel alleles which are &lt;em&gt;fixed between&lt;/em&gt; but &lt;em&gt;lack polymorphism within&lt;/em&gt; two closely related species (I'm looking for the BLUE region of the Venn diagram)&lt;/h2&gt;&#xA;&lt;h1&gt;Desired Outputs:&lt;/h1&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;All SNPs/Indels of Species A which are &lt;em&gt;exclusive to&lt;/em&gt; and &lt;em&gt;lack variation within&lt;/em&gt; Species A&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;All SNPs/Indels of Species B which are &lt;em&gt;exclusive to&lt;/em&gt; and &lt;em&gt;lack variation within&lt;/em&gt; Species B&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h1&gt;Scratch Materials:&lt;/h1&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Species A reference genome&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Species B reference genome&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Illumina short reads for Species A&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;Illumina short reads for Species B&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;My current plan is to...&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Map Species B to Species A:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 -ax asm5 referenceA.fasta referenceB.fasta &amp;gt; BtoA.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Find all SNP/Indel alleles of Species B relative to Species A (Represents the &lt;em&gt;entire&lt;/em&gt; circle in the above Venn diagram):&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools mpileup -f referenceA.fasta -s BtoA.sam | bcftools call -o B_compared_to_A.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Find all SNPs/Indel of Species B reads relative to Species B reference (Represents the &lt;em&gt;red&lt;/em&gt; region of the Venn diagram):&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools mpileup -f referenceB.fasta -b bam_listB | bcftools call -o B_compared_to_B.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;Perform a liftover on &amp;quot;B_compared_to_B.vcf&amp;quot; to correct for coordinate differences between Species A and Species B references (I haven't completely figured out how to do this - I want to make sure I have the general pipeline nailed down before I get into the weeds)&lt;/li&gt;&#xA;&lt;li&gt;Write a program which extracts all loci found in &amp;quot;B_compared_to_A.vcf&amp;quot; but NOT in &amp;quot;B_compared_to_B_liftedOver.vcf&amp;quot; (This should yield the BLUE region of the Venn Diagram&lt;/li&gt;&#xA;&lt;li&gt;Repeat reciprocally to get alleles exclusive to Species A&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Does the logic for this new pipeline check out?&lt;/p&gt;&#xA;" OwnerUserId="9118" LastEditorUserId="9118" LastEditDate="2021-03-01T23:06:05.167" LastActivityDate="2021-03-07T03:07:14.550" Title="Finding SNP/Indel Markers of Interspecific Variation" Tags="|vcf|genomics|variant-calling|software-recommendation|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="15303" PostTypeId="2" ParentId="15141" CreationDate="2021-01-31T01:56:28.673" Score="1" Body="&lt;p&gt;You can use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez Direct&lt;/a&gt; for this. The following returns Unique Identifiers which are just bare integers.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ geo_query='&amp;quot;Expression profiling by high throughput sequencing&amp;quot;[DataSet Type] AND (&amp;quot;Homo sapiens&amp;quot;[Organism] OR &amp;quot;Mus musculus&amp;quot;[Organism] OR &amp;quot;rattus norvegicus&amp;quot;[Organism]) AND (&amp;quot;2020/01/01&amp;quot;[PDAT] : &amp;quot;3000&amp;quot;[PDAT])'&#xA;$ esearch -db gds -query &amp;quot;$geo_query&amp;quot; | efetch -format uid &amp;gt; gds_results.txt &#xA;$ wc -l gds_results.txt &#xA;9981 gds_results.txt&#xA;$ head -n2 gds_results.txt &#xA;200134092&#xA;200120931&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Instead, if you are looking for a way to get the GSE accessions, you can use the built-in &lt;code&gt;xtract&lt;/code&gt; command to parse the XML returned by &lt;code&gt;esummary&lt;/code&gt; as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ esearch -db gds -query &amp;quot;$geo_query&amp;quot; | esummary | xtract -pattern DocumentSummary -first Accession &amp;gt; gse_accs.txt &amp;gt; gse_accs.txt&#xA;$ wc -l gse_accs.txt &#xA;9981 gse_accs.txt&#xA;$ head -n2 gse_accs.txt &#xA;GSE165829&#xA;GSE165824&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2021-01-31T01:56:28.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15304" PostTypeId="2" ParentId="15273" CreationDate="2021-01-31T20:34:28.040" Score="1" Body="&lt;p&gt;If this is your own DNA and you are trying to analyse yourself you should talk to a genetic counselor. I personally think it's cool, but I deeply understand the potential consequences.&lt;/p&gt;&#xA;&lt;p&gt;What you want is a variant annotation tool. There are many, but I normally write my own. Which is simply a case of pulling down a database and using pandas to match variants with annotation.&lt;/p&gt;&#xA;&lt;p&gt;There are many databases. The biggest popular database is gnomad and the biggest disease database is clinvar. Others include omim, snpdb and 1k genomes. I'm not sure about blood sorry but I'm sure you can find one.&lt;/p&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-01-31T20:34:28.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15306" PostTypeId="1" CreationDate="2021-02-01T11:49:24.080" Score="0" ViewCount="44" Body="&lt;p&gt;I'm trying to extract the host field from the features table using the &lt;a href=&quot;https://github.com/gschofl/biofiles&quot; rel=&quot;nofollow noreferrer&quot;&gt;biofiles&lt;/a&gt; package. From what I understand, the function filter (from the same package) could be of help but I can't extract the information of interest. This is the simplified series of steps that I'm following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(biofiles)&#xA;x_search &amp;lt;- reutils::esearch(db = &amp;quot;nuccore&amp;quot;, #retmax = 9999,&#xA;             term = &amp;quot;((bacteria[ORGN]) AND host[ALL]) AND Loripes[ALL]&amp;quot;)&#xA;&#xA;x_fetch &amp;lt;- reutils::efetch(x_search[1:3], &amp;quot;nuccore&amp;quot;,rettype = &amp;quot;gbwithparts&amp;quot;, &#xA;            retmode = &amp;quot;text&amp;quot;)&#xA;&#xA;x_rec &amp;lt;- biofiles::gbRecord(x_fetch)&#xA;&#xA;x_ft &amp;lt;- biofiles::getFeatures(x_rec)&#xA;&#xA;x_ft[1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The last output should show the list of features for that entry, including the source with the qualifier &amp;quot;host&amp;quot;, which I would like to retrieve for all the ids.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: I'm open to solutions outside the biofiles package.&lt;/p&gt;&#xA;" OwnerUserId="2065" LastEditorUserId="2065" LastEditDate="2021-02-01T14:41:00.177" LastActivityDate="2021-02-01T14:41:00.177" Title="How can I retreive specific qualifiers from genbank files in R?" Tags="|r|genbank|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15307" PostTypeId="1" CreationDate="2021-02-02T02:22:09.660" Score="0" ViewCount="1529" Body="&lt;p&gt;The &lt;code&gt;-e&lt;/code&gt; and &lt;code&gt;-i&lt;/code&gt; options of the &lt;code&gt;bcftools filter&lt;/code&gt; command appear, by default, to only allow for including or excluding &lt;em&gt;sites&lt;/em&gt;.  E.g., &lt;code&gt;-e 'FMT/DP &amp;lt; 10'&lt;/code&gt; removes sites where any sample has &lt;code&gt;DP &amp;lt; 10&lt;/code&gt;, and &lt;code&gt;-e 'MEAN(FMT/DP) &amp;lt; 10'&lt;/code&gt; removes sites where average depth across samples is &amp;lt; 10.&lt;/p&gt;&#xA;&lt;p&gt;I would like to perform effectively similar filtering commands, but in a way that includes or excludes samples, instead of sites.  E.g., exclude all &lt;em&gt;samples&lt;/em&gt; that have &lt;code&gt;DP &amp;lt; 10&lt;/code&gt; at any &lt;em&gt;site&lt;/em&gt;, or exclude all samples with an average depth across all sites &amp;lt; 10, or exclude all samples that have a genotype quality of &amp;lt; 20 at 10% of genotyped sites.&lt;/p&gt;&#xA;&lt;p&gt;Such commands can very easily be written to apply to sites, but there does not seem to be a straightforward way of applying such filtering expressions to samples in &lt;code&gt;bcftools&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;One option mentioned in the manual is to use the double &lt;code&gt;||&lt;/code&gt; operator, as described in the &lt;a href=&quot;http://samtools.github.io/bcftools/bcftools.html#expressions&quot; rel=&quot;nofollow noreferrer&quot;&gt;man page&lt;/a&gt;.  As described in my &lt;a href=&quot;https://github.com/samtools/bcftools/issues/1384#issuecomment-770082770&quot; rel=&quot;nofollow noreferrer&quot;&gt;comment&lt;/a&gt; on a Github issue, however, this does not seem to work as a filtering operation for including or excluding entire columns in a VCF file, but rather only for selecting more or less individual records (e.g., as illustrated in the example on &lt;a href=&quot;http://samtools.github.io/bcftools/howtos/filtering.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tutorial page&lt;/a&gt;).  In addition, it would seem limited in functionality (e.g., functions such as &lt;code&gt;MEAN(FMT/DP) &amp;lt; 10&lt;/code&gt; would still be evaluated across samples, not sites, when combined with a &lt;code&gt;||&lt;/code&gt; operator).&lt;/p&gt;&#xA;&lt;p&gt;The only other method mentioned in the manual is to use the &lt;code&gt;SMPL&lt;/code&gt; or &lt;code&gt;s&lt;/code&gt; prefix to functions which evaluate &lt;code&gt;FORMAT&lt;/code&gt; fields in expressions passed to &lt;code&gt;filter&lt;/code&gt;.  This also does not appear to work as I would expect.  I'm not sure I understand what it &lt;em&gt;is&lt;/em&gt; doing, but it definitely &lt;em&gt;isn't&lt;/em&gt; removing columns from the VCF file.  Instead it seems to be using as a metric calculated across sites on a per sample basis as a filtering criteria applied to sites (but I could just be confused):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;nSites_unfiltered=&lt;span class=&quot;math-container&quot;&gt;$(bcftools view -H unfiltered.vcf.gz | wc -l) &#xA;nSamples_unfiltered=$&lt;/span&gt;(bcftools query unfiltered.bcf.gz -l | wc -l)&#xA;&#xA;nSites_DPfilter=&lt;span class=&quot;math-container&quot;&gt;$(bcftools filter unfiltered.bcf.gz -e 'sMEAN(FORMAT/DP)&amp;lt;5' | bcftools view -H | wc -l)&#xA;nSamples_DPfilter=$&lt;/span&gt;(bcftools filter unfiltered.bcf.gz -e 'sMEAN(FORMAT/DP)&amp;lt;5' | bcftools query -l | wc -l)&#xA;&#xA;echo &amp;quot;scale=5; &lt;span class=&quot;math-container&quot;&gt;$nSites_DPfilter / $&lt;/span&gt;nSites_unfiltered&amp;quot; | bc #0.17349 - sites are removed, even though the filter is being applied to samples&#xA;echo &amp;quot;scale=5; &lt;span class=&quot;math-container&quot;&gt;$nSamples_DPfilter / $&lt;/span&gt;nSamples_unfiltered&amp;quot; | bc #1.00000 - no samples removed, despite, again, the filter being applied to samples&#xA;&#xA;nSites_normalDPFilter=&lt;span class=&quot;math-container&quot;&gt;$(bcftools filter unfiltered.bcf.gz -e 'MEAN(FMT/DP) &amp;lt; 5' | bcftools view -H | wc -l)&#xA;echo &quot;scale=5; $&lt;/span&gt;nSites_normalDPFilter / $nSites_unfiltered&amp;quot; | bc #0.83121 - fewer sites removed than the sMEAN filter&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;.bcf.gz&lt;/code&gt; passed to these commands is here: &lt;a href=&quot;https://www.dropbox.com/s/fwj1snabsb763sn/unfiltered.bcf.gz?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/fwj1snabsb763sn/unfiltered.bcf.gz?dl=0&lt;/a&gt;&#xA;Its &lt;code&gt;.csi&lt;/code&gt; index file is here: &lt;a href=&quot;https://www.dropbox.com/s/jzdq6y91nkencmy/unfiltered.bcf.gz.csi?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/jzdq6y91nkencmy/unfiltered.bcf.gz.csi?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In any event, the only option I have been able to come up with that actually excludes or includes entire sample IDs is to extract summary statistics using &lt;code&gt;bcftools stats&lt;/code&gt;, export the desired statistics from the &lt;code&gt;PSC, Per-sample counts&lt;/code&gt; table, and perform my own exploratory analysis outside of &lt;code&gt;bcftools&lt;/code&gt;.  Then I can use &lt;code&gt;bcftools view -S&lt;/code&gt; to explicitly remove samples by their ID, which I have identified manually as failing some thresholding criteria.  This also is obviously cumbersome, and only works for &lt;code&gt;FMT&lt;/code&gt; fields that are included in &lt;code&gt;stats&lt;/code&gt; (e.g., not &lt;code&gt;GQ&lt;/code&gt; or &lt;code&gt;PL&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;I am not sure if I am missing some basic functionality of &lt;code&gt;bcftools&lt;/code&gt;, or if this is more of a feature request.  Any information would be appreciated.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="10398" LastEditorUserId="10398" LastEditDate="2021-02-02T06:41:26.343" LastActivityDate="2021-02-02T06:41:26.343" Title="Filtering samples by FORMAT attributes using bcftools" Tags="|vcf|snp|filtering|bcftools|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15308" PostTypeId="2" ParentId="15293" CreationDate="2021-02-02T12:50:37.607" Score="2" Body="&lt;p&gt;Branch-models are currently not implemented in godon. In the PAML manual Ziheng Yang writes:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This model is very parameter-rich and its use is discouraged.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This was one of the reasons I did not implement it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Disclaimer: I am the author of godon.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="191" LastActivityDate="2021-02-02T12:50:37.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15309" PostTypeId="2" ParentId="15155" CreationDate="2021-02-02T15:25:26.603" Score="1" Body="&lt;p&gt;The applications can be categorized based on whether you study the sequence for analysis, or for DNA synthesis and assembly.&lt;/p&gt;&#xA;&lt;p&gt;In addition to TF patterns mentioned in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/15156/208&quot;&gt;this answer&lt;/a&gt;, other patterns for &lt;a href=&quot;https://en.wikipedia.org/wiki/DNA-binding_protein#Binding_to_specific_DNA_sequences&quot; rel=&quot;nofollow noreferrer&quot;&gt;DNA binding proteins&lt;/a&gt;, enchancers, promoters, and the &lt;a href=&quot;https://www.promega.co.uk/resources/pubhub/enotes/what-is-a-kozak-consensus-sequence/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kozak sequence&lt;/a&gt; (&lt;code&gt;GCCGCC[A|G]CCATGG..&lt;/code&gt;) may also be useful for discovery of ORFs (genes) / regulation.&lt;/p&gt;&#xA;&lt;p&gt;Certain methylase enzymes recognize a DNA pattern. For example EcoKI methylates adenine in &lt;code&gt;AACNNNNNNGTGC&lt;/code&gt; (2nd A) and its reverse complement, &lt;code&gt;GCACNNNNNNGTT&lt;/code&gt; (3rd A). Another such enzyme is EcoBI. This is useful for finding methylation sites, which may interfere with DNA restriction or expression.&lt;/p&gt;&#xA;&lt;p&gt;Some &lt;a href=&quot;https://en.wikipedia.org/wiki/Restriction_enzyme#Examples&quot; rel=&quot;nofollow noreferrer&quot;&gt;restriction enzymes&lt;/a&gt; also recognize a DNA pattern: HinFI recognizes&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;5'GANTC&#xA;3'CTNAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Other such enzymes are EcoP15I, EcoRII. Finding these sites are very important for designing a DNA cloning or manipulation experiment.&lt;/p&gt;&#xA;&lt;p&gt;For DNA synthesis, it is useful to avoid homopolymers (&lt;code&gt;N+&lt;/code&gt;) and repeated k-mers and hairpins (a sequence segment which has a reverse complement &amp;quot;nearby&amp;quot;) as these can cause problems during synthesis.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Direct_repeat&quot; rel=&quot;nofollow noreferrer&quot;&gt;Direct&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Tandem_repeat&quot; rel=&quot;nofollow noreferrer&quot;&gt;tandem&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Inverted_repeat&quot; rel=&quot;nofollow noreferrer&quot;&gt;inverted&lt;/a&gt; repeats can cause problems during cloning.&lt;/p&gt;&#xA;&lt;p&gt;Finally, a perhaps trivial example is matching a polypeptide sequence, using DNA, as the &lt;a href=&quot;https://en.wikipedia.org/wiki/DNA_and_RNA_codon_tables&quot; rel=&quot;nofollow noreferrer&quot;&gt;genetic code is redundant&lt;/a&gt;. An example would be searching for &lt;a href=&quot;https://en.wikipedia.org/wiki/Target_peptide#Examples_of_target_peptides&quot; rel=&quot;nofollow noreferrer&quot;&gt;target peptides in a protein&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2021-02-02T15:25:26.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15311" PostTypeId="1" CreationDate="2021-02-02T23:32:17.650" Score="0" ViewCount="181" Body="&lt;p&gt;I am trying to run this kallisto (latest) quant and all I get are abundance and json files.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;kallisto quant --plaintext -t 4  -o rf-stranded.nob.1K1_L7.0 -i mm9_ref.idx  --rf-stranded 1K1_L7.0 2K1_L7.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;the results files in stderr is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[quant] fragment length distribution will be estimated from the data&#xA;[index] k-mer length: 15&#xA;[index] number of targets: 23&#xA;[index] number of k-mers: 363,704,628&#xA;[index] number of equivalence classes: 1,945,936&#xA;[quant] running in paired-end mode&#xA;[quant] will process pair 1: 1K1_L1.0&#xA;                             2K1_L1.0&#xA;[quant] finding pseudoalignments for the reads ... done&#xA;[quant] processed 22,897,424 reads, 4,476,549 reads pseudoaligned&#xA;[quant] estimated average fragment length: 24.2427&#xA;[   em] quantifying the abundances ... done&#xA;[   em] the Expectation-Maximization algorithm ran for 52 rounds&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I get an abundance file with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;target_id       length  eff_length      est_counts      tpm&#xA;chr1    197195432       1.89745e+08     253015  21.3249&#xA;chr10   129993255       1.31621e+08     210320  25.5544&#xA;chr11   121843856       1.38776e+08     263606  30.3773&#xA;chr12   121257530       1.23195e+08     143568  18.6371&#xA;chr13   120284312       1.22484e+08     132969  17.3613&#xA;chr14   125194864       1.20926e+08     112711  14.9058&#xA;chr15   103494974       1.07866e+08     109335  16.2102&#xA;chr16   98319150        9.70075e+07     109045  17.9767&#xA;chr17   95272651        1.06375e+08     169166  25.4322&#xA;chr18   90772031        9.23542e+07     121990  21.1241&#xA;chr19   61342430        7.23377e+07     157142  34.7407&#xA;chr2    181748087       1.81791e+08     244870  21.5414&#xA;chr3    159599783       1.45978e+08     209257  22.9247&#xA;chr4    155630120       1.61978e+08     226893  22.4014&#xA;chr5    152537259       1.65672e+08     194951  18.8186&#xA;chr6    149517037       1.43243e+08     246652  27.5373&#xA;chr7    152524553       1.82795e+08     197527  17.2812&#xA;chr8    131738871       1.50556e+08     203954  21.6643&#xA;chr9    124076172       1.30446e+08     222639  27.295&#xA;chrX    166650296       1.40278e+08     176259  20.0942&#xA;chrY    15902555        5.76797e+07     123.462 0.0342311&#xA;mtDNA   16299   11804.8 737285  998819&#xA;rDNA    249368  720854  33272.5 738.159&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;run_info.json:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;        &amp;quot;n_targets&amp;quot;: 23,&#xA;        &amp;quot;n_bootstraps&amp;quot;: 100,&#xA;        &amp;quot;n_processed&amp;quot;: 22897424,&#xA;        &amp;quot;n_pseudoaligned&amp;quot;: 4476549,&#xA;        &amp;quot;n_unique&amp;quot;: 3641646,&#xA;        &amp;quot;p_pseudoaligned&amp;quot;: 19.6,&#xA;        &amp;quot;p_unique&amp;quot;: 15.9,&#xA;        &amp;quot;kallisto_version&amp;quot;: &amp;quot;0.46.1&amp;quot;,&#xA;        &amp;quot;index_version&amp;quot;: 10,&#xA;        &amp;quot;start_time&amp;quot;: &amp;quot;&amp;quot;,&#xA;        &amp;quot;call&amp;quot;: &amp;quot;kallisto quant --plaintext -t 4 -b 100 -o rf-stranded.1K1_L1.0 -i mm9_ref.idx --bias --rf-stranded --gtf UCSC_genes.gtf --pseudobam --genomebam 1K1_L1.0.gz 2K1_L1.0.gz&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;When I ran it with boot strapping (-b), -gtf and --pseudobam --genomebam it gave me a bam  file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;44879990 + 0 in total (QC-passed reads + QC-failed reads)&#xA;0 + 0 secondary&#xA;0 + 0 supplementary&#xA;0 + 0 duplicates&#xA;8078785 + 0 mapped (18.00% : N/A)&#xA;44879990 + 0 paired in sequencing&#xA;22439995 + 0 read1&#xA;22439995 + 0 read2&#xA;8077160 + 0 properly paired (18.00% : N/A)&#xA;8077160 + 0 with itself and mate mapped&#xA;1625 + 0 singletons (0.00% : N/A)&#xA;0 + 0 with mate mapped to a different chr&#xA;0 + 0 with mate mapped to a different chr (mapQ&amp;gt;=5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm not sure what is going wrong, can someone help?&lt;/p&gt;&#xA;&lt;p&gt;NOTE I use a 15nt kmer, and this about 1/2 of what is suggested (31 I think), is that a problem?&lt;/p&gt;&#xA;" OwnerUserId="10409" LastEditorUserId="2108" LastEditDate="2021-02-03T00:13:34.380" LastActivityDate="2021-02-03T09:34:32.177" Title="Kallisto psudoaligner quant not giving me expected output" Tags="|rna-seq|kallisto|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15312" PostTypeId="1" CreationDate="2021-02-03T02:56:27.170" Score="0" ViewCount="77" Body="&lt;p&gt;I’m trying to create a pipeline for genome assembly. How best can I “redirect/pipe” from existing fasta files (or files in general) to other steps of the pipeline?&lt;/p&gt;&#xA;&lt;p&gt;I was thinking of going from the SRA Download step (prefetch and fasterq-dump) to fastqc, then to trimmomatic or bbduk to remove the adapters, one more fastqc check and then to SPAdes for the assembly.&lt;/p&gt;&#xA;&lt;p&gt;In my (amateur) mind it would be cool to be like:&lt;/p&gt;&#xA;&lt;p&gt;“SRRXXXXXX” | prefetch | fasterq-dump | fastqc | bbduk.sh | fastqc (second) | spades.py&lt;/p&gt;&#xA;&lt;p&gt;Any and all help is greatly appreciate! I know this stuff is pretty basic but I am so new to everything it’s painfully obvious to even me.&lt;/p&gt;&#xA;" OwnerUserId="10415" LastActivityDate="2021-02-24T08:55:08.740" Title="Genome QC + Assembly Pipeline semantics" Tags="|fasta|genome|assembly|sratoolkit|fastqc|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15313" PostTypeId="2" ParentId="15312" CreationDate="2021-02-03T07:26:44.897" Score="0" Body="&lt;p&gt;I understand the desire to pipe everything like that but I don't think it will work, even for such a small workflow. And as your pipe grows it'll make it harder to troubleshoot/debug.&lt;/p&gt;&#xA;&lt;p&gt;These days everyone uses workflow languages to write pipelines. It's a bit more work to get set up but worth it in the long run. I use WDL. Snakemake is very popular. BASH will of course work for small pipes but not with | for every step.&lt;/p&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-02-03T07:26:44.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15314" PostTypeId="2" ParentId="15311" CreationDate="2021-02-03T09:34:32.177" Score="3" Body="&lt;p&gt;As you surmised, kallisto requires a transcriptome fasta file, not a genome fasta file.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-02-03T09:34:32.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15315" PostTypeId="2" ParentId="15155" CreationDate="2021-02-03T10:51:00.920" Score="2" Body="&lt;p&gt;These algorithms are used quite a lot, and I will touch only a few aspects:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Genetic sequence alignment.&lt;/strong&gt; Different (but still closely related) organisms have very similar genetic sequences, which differ only at some positions due to &lt;em&gt;point mutations&lt;/em&gt;, which can be nucleotide substitutions (another letter in a sequence) or insertion/deletion of a few nucleotides. Many biological questions such as studying new coronavirus strains, predicting genetic diseases, and identifying cancerous cells are based on such sequence comparisons. This requires aligning many sequences, using the appropriate algorithms. This may be not hard for two sequences, but for hundreds or thousands of them it may require serious computational effort.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Sequencing reads mapping&lt;/strong&gt; Much of the current progress in genetics is due to the advent of the &lt;em&gt;next generation sequencing&lt;/em&gt;, also called &lt;em&gt;massively parallel sequencing&lt;/em&gt;, as opposed to Sanger sequencing used in the first human genome project (&lt;a href=&quot;https://discovernanotech.com/complete-genome-in-a-few-hours-the-application-of-nanotechnology-in-dna-sequencing/&quot; rel=&quot;nofollow noreferrer&quot;&gt;see here&lt;/a&gt;, for example). The basis of this technology is splitting the genetic sequence (a few billion nucleotides in a human) into small fragments of a few hundred nucleotides long and sequencing them simultaneously. One then faces the problem of putting them back together, which can be achieved either via matching them to a known sequence for a similar organism (&lt;em&gt;sequence mapping&lt;/em&gt;) or by putting the reads together on the basis of their similarity (&lt;em&gt;sequence assembly&lt;/em&gt;). There exist multiple sophisticated bioinformatic tools for achieving this (BWA, bowtie2, TopHat, Megahit to name just a few, so that you can google them).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Blasting&lt;/strong&gt; Another application, closely related to my first point, is quickly identifying the sequence as belonging to a certain gene of a certain organism by comparing it to a large database. BLAST is one of the well-known collections of tools for doing this.&lt;/li&gt;&#xA;&lt;li&gt;Just to mention (only a bit) more exotic applications: the algorithms associated with parsing context free grammars are used for &lt;strong&gt;RNA structure prediction&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.amazon.fr/Biological-Sequence-Analysis-Probabilistic-Proteins/dp/0521629713&quot; rel=&quot;nofollow noreferrer&quot;&gt;Durbin's book&lt;/a&gt; is a good casual reading (for somebody with a computer science background) to get an overview without knowing too much biology.&lt;/p&gt;&#xA;" OwnerUserId="1813" LastEditorUserId="6628" LastEditDate="2021-04-26T20:49:50.620" LastActivityDate="2021-04-26T20:49:50.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15316" PostTypeId="1" CreationDate="2021-02-03T17:10:51.967" Score="-2" ViewCount="159" Body="&lt;p&gt;TTGAAACACTGGATGAATGAAAAGCCCTGCTTTGCAACCCCTCAGC this is the DNA code Sequence. I have tried converting each into the amino acid and ended up with this LKHWMNEKPCFATPQX but I was told that this isn't right.&lt;/p&gt;&#xA;" OwnerUserId="10421" LastActivityDate="2021-02-03T21:10:33.460" Title="Does anyone know how I can convert DNA code into FASTA for this TTGAAACACTGGATGAATGAAAAGCCCTGCTTTGCAACCCCTCAGC" Tags="|sequence-alignment|fasta|sequence-analysis|phylogenetics|" AnswerCount="1" CommentCount="2" ClosedDate="2021-02-05T14:05:02.163" ContentLicense="CC BY-SA 4.0" />
  <row Id="15317" PostTypeId="1" CreationDate="2021-02-03T20:35:03.717" Score="0" ViewCount="149" Body="&lt;p&gt;My goal is to generate a diploid consensus genome. By &amp;quot;diploid consensus&amp;quot;, I mean that I want to merge all supporting reads for a genomic region into a single haplome such that the resulting sam file contains TWO sets of reads, one from each haplome.&lt;/p&gt;&#xA;&lt;p&gt;I have Illumina short-read files from the NCBI's SRA. These short-reads were obtained from a diploid species.&lt;/p&gt;&#xA;&lt;p&gt;I am worried that, by simply using Picard's MarkDuplicates with the &amp;quot;REMOVE_DUPLICATES&amp;quot; option set to &amp;quot;TRUE&amp;quot;, I will retain only the homolog with the highest quality and output a haploid version of my genome.&#xA;Here is my current workflow:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;## Map Illumina reads to the reference genome. Include read IDs&#xA;bwa mem -M -R '@RG\tID:BioSample.LibraryName\tSM:BioSample\tLB:LibraryName\tPU:SingleFlowcell.SingleLane\tPL:ILLUMINA\t' reference.fasta SRR[number]_1.fastq SRR[number]_2.fastq &amp;gt; SRR[number].sam  &#xA;&#xA;## Sort Illumina reads by coordinate&#xA;java -jar &lt;span class=&quot;math-container&quot;&gt;$EBROOTPICARD/picard.jar SortSam I=SRR[number].sam O=sorted_SRR[number].sam CREATE_INDEX=true SORT_ORDER=coordinate TMP_DIR=$&lt;/span&gt;TMPDIR&#xA;&#xA;## Remove duplicate reads from sam file&#xA;## *Note: How do I remove sampling error reads?&#xA;java -jar &lt;span class=&quot;math-container&quot;&gt;$EBROOTPICARD/picard.jar MarkDuplicates REMOVE_DUPLICATES=true REMOVE_SEQUENCING_DUPLICATES=true I= 'sorted_SRR[number].sam' O= nodup_SRR[number].sam METRICS_FILE= metrics_SRR[number].txt TMP_DIR=$&lt;/span&gt;TMPDIR&#xA;&#xA;## Convert duplicate-free sam file to bam file&#xA;samtools view -S -b nodup_SRR[number].sam &amp;gt; nodup_SRR[number].bam &#xA;&#xA;## Re-align reads around indels to reduce slippage.&#xA;gatk LeftAlignIndels -R reference.fasta -I nodup_SRR[number].bam -O aligned_SRR[number].bam &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does anyone know of a program which will whittle my sam files down to ONLY two homologs per region?&lt;/p&gt;&#xA;" OwnerUserId="9118" LastEditorUserId="9118" LastEditDate="2021-02-05T03:14:54.437" LastActivityDate="2021-02-08T08:14:11.463" Title="Diploid Consensus Sequences" Tags="|sam|markduplicate|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="15318" PostTypeId="2" ParentId="15316" CreationDate="2021-02-03T21:10:33.460" Score="1" Body="&lt;p&gt;This is your nuc seq as a fasta:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;sequence_1&#xA;TTGAAACACTGGATGAATGAAAAGCCCTGCTTTGCAACCCCTCAGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To translate it use an online tool (ie &lt;a href=&quot;https://web.expasy.org/translate/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://web.expasy.org/translate/&lt;/a&gt;) or do it in biopython:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;for record in SeqIO.parse(&amp;quot;/path/seq1.fa&amp;quot;, &amp;quot;fasta&amp;quot;):&#xA;    print ('nuc', record, sep='\n')&#xA;    print ('prot', record.translate(), sep='\n')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which gives:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;nuc&#xA;ID: sequence_1&#xA;Name: sequence_1&#xA;Description: sequence_1&#xA;Number of features: 0&#xA;Seq('TTGAAACACTGGATGAATGAAAAGCCCTGCTTTGCAACCCCTCAGC', SingleLetterAlphabet())&#xA;prot&#xA;ID: &amp;lt;unknown id&amp;gt;&#xA;Name: &amp;lt;unknown name&amp;gt;&#xA;Description: &amp;lt;unknown description&amp;gt;&#xA;Number of features: 0&#xA;Seq('LKHWMNEKPCFATPQ', ExtendedIUPACProtein())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note, this defaults to the human codon table. Other codon tables are available with the parameter 'table'.&lt;/p&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-02-03T21:10:33.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15319" PostTypeId="2" ParentId="15155" CreationDate="2021-02-03T21:25:32.190" Score="1" Body="&lt;p&gt;To extend on the existing answers I would say that pattern matching algorithms are at the core of bioinformatics, certainly any NGS realted bioinformatics. The Smith-Waterman algorithm and its variants are probably the most used &lt;a href=&quot;https://en.wikipedia.org/wiki/Smith%E2%80%93Waterman_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Smith%E2%80%93Waterman_algorithm&lt;/a&gt; .&lt;/p&gt;&#xA;&lt;p&gt;Reference genome assembly (the human genome project for example) is based on kmer based sequence matching. Variant calling is based on mapping sequence to this reference, so thats pretty much all clinical bioinformatics. And a million other tools - one random example is dectecting horizontal gene transfer of antibiotic resistence genes.&lt;/p&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-02-03T21:25:32.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15321" PostTypeId="1" AcceptedAnswerId="15323" CreationDate="2021-02-04T05:58:11.680" Score="1" ViewCount="223" Body="&lt;p&gt;I have following two lists.&#xA;List1&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;m64041_200717_231916/100664277/0_42364 id=2440&#xA;m64041_200717_231916/100008029/0_42112 id=244006&#xA;m64041_200717_231916/100665802/0_46854 id=2440067&#xA;m64041_200717_231916/100008044/0_60161 id=24400&#xA;m64041_200717_231916/100928073/0_11804 id=2440152&#xA;m64041_200717_231916/10094492/0_44157 id=2440199&#xA;m64041_200717_231916/100008373/4102_50601 id=2440234&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;List2&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;m64041_200717_231916/100664277/0_42364&#xA;m64041_200717_231916/100665802/0_46854&#xA;m64041_200717_231916/100928073/0_11804&#xA;m64041_200717_231916/10094492/0_44157&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These are sequence read ids from a genome file. List1 contains all the ids from the List2 but with some id numbers (id=2440). So the sequence ids are only similar to some extent. I want to compare these two files and I need to filter out List2 sequence ids from List one (with id numbers) as below.&#xA;Output.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;m64041_200717_231916/100664277/0_42364 id=2440&#xA;m64041_200717_231916/100665802/0_46854 id=2440067&#xA;m64041_200717_231916/100928073/0_11804 id=2440152&#xA;m64041_200717_231916/10094492/0_44157 id=2440199&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I will be grateful if someone can help me with this issue.&lt;/p&gt;&#xA;" OwnerUserId="856" LastEditorUserId="3694" LastEditDate="2021-02-04T06:13:12.713" LastActivityDate="2021-02-25T04:52:30.033" Title="How to filter out partially similar strings from two lists and make one list?" Tags="|bash|filtering|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15322" PostTypeId="1" AcceptedAnswerId="15357" CreationDate="2021-02-04T06:11:23.613" Score="2" ViewCount="198" Body="&lt;p&gt;I'm looking for books on applications of machine learning and statistical data mining in bioinformatics with example codes provided in R and/or python.&lt;/p&gt;&#xA;&lt;p&gt;I came across the following two books: the first being a quite good one, provides the code in MATLAB, which I'm not very familiar with. The second one too is a good book with examples in R.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Feature Selection and Ensemble Methods for Bioinformatics by Oleg Okun&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Data Science and Predictive Analytics: Biomedical and Health Applications using R by Ivo Dinov&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Can people suggest other books based on their first hand experiences?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="6649" LastActivityDate="2021-02-11T04:35:22.147" Title="Books on application of Machine Learning in Bioinformatics" Tags="|machine-learning|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15323" PostTypeId="2" ParentId="15321" CreationDate="2021-02-04T07:31:24.250" Score="3" Body="&lt;p&gt;You really need to learn some python :)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with open('/path/list2.txt', 'r') as fin:&#xA;    to_keep = set([])&#xA;    for line in fin:&#xA;        to_keep.add(line.strip())&#xA;        &#xA;with open('/path/list1.txt', 'r') as fin:&#xA;    with open('/path/list3.txt', 'w') as fout:&#xA;        for line in fin:&#xA;            if line.strip().split(' ')[0] in to_keep:&#xA;                fout.write(line)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-02-04T07:31:24.250" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15325" PostTypeId="2" ParentId="15322" CreationDate="2021-02-04T08:52:55.853" Score="0" Body="&lt;p&gt;One could read &lt;a href=&quot;https://rads.stackoverflow.com/amzn/click/com/026202506X&quot; rel=&quot;nofollow noreferrer&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Bioinformatics: The machine learning approach&lt;/em&gt;&lt;/a&gt; as an overview of the established applications of machine learning to sequence alignment, RNA structure prediction, etc...&lt;/p&gt;&#xA;&lt;p&gt;...Yet, this book is 20 years old, and there have been many important developments in bioinformatics and machine learning in this period, which have not yet been covered in any books. In other words, reading books on bioinformatics and/or machine learning is good for getting firm basic background in either of them, but any such books are inevitably out of date at the time of their publication.&lt;/p&gt;&#xA;&lt;p&gt;I would like however to stress the importance of the basic background - one regularly sees announcements about &lt;em&gt;breakthroughs&lt;/em&gt; achieved by reinventing techniques that were already described in the book that I mentioned.&lt;/p&gt;&#xA;" OwnerUserId="1813" LastActivityDate="2021-02-04T08:52:55.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15326" PostTypeId="1" CreationDate="2021-02-04T17:54:28.760" Score="3" ViewCount="650" Body="&lt;p&gt;I have a fasta file that contains sequence reads and sequence id file that needed to be removed from the fasta file. I have done this earlier, but since id contains a space my piece of code is not working and I don't know how to change it since I'm still learning. fasta file looks like this.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;m64041_200717_231916/74/12723_24868 id=30&#xA;CCGCACCTCCTCAATCTGCAGCAGTTGAGGCCACTACCCTTCTGCTCAATGGTTCCTGCAGACTTTATCATAGTCACTCACACTTGTCCA&#xA;&amp;gt;m64041_200717_231916/77/1941_50622 id=3115&#xA;TGAGCCGCACCTCCTCAATCTGCAGCAGTTGAGGCCACTACCCTTCTGCTCAATGGTTCCTGCAGACTTTATCATAGTCACTCACACTTGTCCATGAG&#xA;&amp;gt;m64041_200717_231916/105/20691_65844 id=488&#xA;AGGCCACTACCCTTCTGCTCAATGGTTCCTGCAGACTTTATCATAGTCACTCACACTTGTCCAAGACTTTAT&#xA;&amp;gt;m64041_200717_231916/108/17414_67048 id=4956&#xA;TGAGGCCACTACCCTTCTGCTCAATGGTTCCTGCAGACTTTATCATAGTCACTCACACTTGTCCATGAGGCAGACTTTATCATAGTCACTCACACTTG&#xA;&amp;gt;m64041_200717_231916/162/0_6615 id=857&#xA;CAGACTTTATCATAGTCACTCACACTTGTCCATGAGGCAGACTTTATCATAGTCACTCACACTTG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;IDList file is like this.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;m64041_200717_231916/74/12723_24868 id=30&#xA;m64041_200717_231916/108/17414_67048 id=4956&#xA;m64041_200717_231916/105/20691_65844 id=488&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I need to remove these sequences from that fasta file. I have a code I used earlier. But it is not working since id contains a space (I think). Previous code is like this.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# use as follows&#xA;# ./filter_seq.bash &amp;lt;fastafile&amp;gt; &amp;lt;list_of_ids_you_want_to_remove&amp;gt;&#xA;# ./filter_seq.bash all.fasta bacterial_IDlist&#xA;&#xA;grep &amp;quot;&amp;gt;&amp;quot; &lt;span class=&quot;math-container&quot;&gt;$1 | sed 's/&amp;gt;//g' | sort | uniq &amp;gt; all_del&#xA;sort $&lt;/span&gt;2 | uniq &amp;gt; query_del&#xA;comm -23 all_del query_del | sort | uniq &amp;gt; filtered_del&#xA;perl -ne 'if(/^&amp;gt;(\S+)/){&lt;span class=&quot;math-container&quot;&gt;$c=$&lt;/span&gt;i{&lt;span class=&quot;math-container&quot;&gt;$1}}$&lt;/span&gt;c?print:chomp;&lt;span class=&quot;math-container&quot;&gt;$i{$_}=1 if @ARGV' filtered_del $&lt;/span&gt;1 &amp;gt; filtered.fasta&#xA;rm *_del&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would be grateful if someone can help me.&lt;/p&gt;&#xA;" OwnerUserId="856" LastEditorUserId="3694" LastEditDate="2021-02-04T21:01:04.410" LastActivityDate="2021-02-05T13:55:38.527" Title="How to remove sequences from a fasta file using a sequence ID list which contains a space within the id?" Tags="|bash|linux|filtering|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15328" PostTypeId="2" ParentId="15326" CreationDate="2021-02-04T20:55:08.470" Score="1" Body="&lt;p&gt;Install (pip) biopython if u don't have it.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;with open('/path/ids.txt', 'r') as fin:&#xA;    to_keep = set([])&#xA;    for line in fin:&#xA;        to_keep.add(line.strip().split(' ')[0])&#xA;with open('/path/output.fa', 'w') as fout:&#xA;    for record in SeqIO.parse('/path/sequences.fa','fasta'):&#xA;        if record.id in to_keep:&#xA;            SeqIO.write(record, fout, 'fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-02-04T20:55:08.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15329" PostTypeId="1" AcceptedAnswerId="15330" CreationDate="2021-02-04T23:54:45.157" Score="1" ViewCount="2519" Body="&lt;p&gt;What is the best python tool to convert a SDF file to a structured data frame (pandas.DataFrame). Similar to Molconvert from ChemAxon? SDF can have different datafields for each compound which I would like to have in a dataframe. The 2D or 3D structures can be disregarded.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2021-02-05T00:23:17.880" LastActivityDate="2021-02-05T00:23:17.880" Title="How to convert a SDF molecular structure file to a pandas dataframe with Python?" Tags="|python|rdkit|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15330" PostTypeId="2" ParentId="15329" CreationDate="2021-02-05T00:22:57.377" Score="2" Body="&lt;p&gt;It is easily done with RDKit:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from rdkit.Chem import PandasTools&#xA;&#xA;fn = '/mypath/dataset.sdf'&#xA;&#xA;df = PandasTools.LoadSDF(fn, embedProps=True, molColName=None, smilesName='smiles')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2021-02-05T00:22:57.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15331" PostTypeId="1" CreationDate="2021-02-05T02:39:49.107" Score="1" ViewCount="70" Body="&lt;p&gt;I tried running this command &lt;code&gt;python blastp.py CmydPro tlr TLR.txt&lt;/code&gt; to extract the query sequence for the target species I'm looking at. However, whenever I try running it always comes up as a blast p error and says there's no directory. I downloaded blastp.py as a file made from my lab's GitHub but I have no idea how to troubleshoot the error since it's not a required directory. I made sure all my directories were correct and that all my files were intact. Any help with this would be greatly appreciated.&lt;/p&gt;&#xA;&lt;p&gt;This is the error I keep getting:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;Traceback (most recent call last):&#xA;File &amp;quot;blastp.py&amp;quot;, line 53, in&#xA;blastp_scout = subprocess.Popen(['blastp', '-db', organism_db, '-query', 'temp.fa', '-out', 'blast', '-num_descriptions', '5', '-num_alignments', '0'])&#xA;File &amp;quot;/home/dylan_w190/miniconda3/lib/python3.8/subprocess.py&amp;quot;, line 854, in __init__&#xA;self._execute_child(args, executable, preexec_fn, close_fds,&#xA;File &amp;quot;/home/dylan_w190/miniconda3/lib/python3.8/subprocess.py&amp;quot;, line 1702, in _execute_child&#xA;raise child_exception_type(errno_num, err_msg, err_filename)&#xA;FileNotFoundError: [Errno 2] No such file or directory: 'blastp'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These are all the files I have stored in my directory:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;'$RECYCLE.BIN' 'Genome Files' TLR.txt&#xA;'ASM 28112' 'Genome Files (1)' Tham&#xA;'ASM 28964' 'Genome Files (2)' Vim&#xA;'ASM 3597' 'Genome Files (3)' Xenopus&#xA;'ASM 45574' 'Genome Files (4)' blast-2.11.0+&#xA;AnoCar Git blastp.py&#xA;Chrysemys GopAga get-pip.py&#xA;Cliv GopAga2.0-Genome human&#xA;CmydPro Halia ls&#xA;CmydPro_results.txt LICENCE mouse&#xA;Config.Msi Lib ncbi-blast-2.6.0+&#xA;CroPor Miniconda3-latest-Linux-x86_64.sh ncbi-blast-2.6.0+-win64.exe&#xA;'FASTAs (2)' OpenSSH-Win64 pip&#xA;Format Ornitho pvi&#xA;GCF_015237465.1_rCheMyd1.pri_genomic.fna PelSin pyzo&#xA;GFFs Python setup.py&#xA;'GFFs (2)' PythonX temp.fa&#xA;GRCh38 README.txt tlr_CmydPro.fa&#xA;GRCm38 Rnor tlr_ref.fasta&#xA;Gallus 'Rnor - Copy' vs_buildtools__576366542.1609221376.exe&#xA;GavGan Scripts vs_community__576366542.1609221376.exe&#xA;Gekko 'System Volume Information'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There is a windows version. It seems that I already downloaded blast but I need to do this now:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Command 'blastp' not found, but can be installed with: sudo apt install ncbi-blast+&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Do you have any idea how to might download this?&lt;/p&gt;&#xA;&lt;p&gt;I already downloaded blast as a tar file and the 2.11.0+ version. I'm just not sure how to download just ncbi-blast+ on its own.&lt;/p&gt;&#xA;" OwnerUserId="10432" LastEditorUserId="73" LastEditDate="2021-07-06T01:43:47.370" LastActivityDate="2022-04-02T06:09:16.027" Title="blast p-error with making directory" Tags="|file-formats|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15332" PostTypeId="2" ParentId="15331" CreationDate="2021-02-05T05:03:54.737" Score="1" Body="&lt;p&gt;You don't have blastp in your path. Blast is either not installed on your system or it's just not been added to path right.&lt;/p&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-02-05T05:03:54.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15333" PostTypeId="2" ParentId="4521" CreationDate="2021-02-05T10:32:26.533" Score="1" Body="&lt;p&gt;ProTherm2 contains thermodynamic properties of proteins and mutants. &lt;a href=&quot;https://www.iitm.ac.in/bioinfo/ProTherm2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.iitm.ac.in/bioinfo/ProTherm2/&lt;/a&gt; .&lt;/p&gt;&#xA;&lt;p&gt;NAR Database issue advertises it as containing melting temperatures.&lt;/p&gt;&#xA;&lt;p&gt;For the reference - found it with &lt;a href=&quot;https://biokeanos.com/search?q=protein+%2Bmelting&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biokeanos.com/search?q=protein+%2Bmelting&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4628" LastActivityDate="2021-02-05T10:32:26.533" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15334" PostTypeId="2" ParentId="4335" CreationDate="2021-02-05T11:49:16.253" Score="1" Body="&lt;p&gt;It seems like the ggbio object already contains the corresponding ggplot object.&#xA;You should be able to extract the ggplot from the &lt;code&gt;ggbio_obj@ggplot&lt;/code&gt; slot and extract your gtables.&lt;/p&gt;&#xA;&lt;p&gt;Small example:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(ggbio)&#xA;library(ggplot2)&#xA;library(patchwork)&#xA;&#xA;fl.bam &amp;lt;- system.file(&amp;quot;extdata&amp;quot;, &amp;quot;wg-brca1.sorted.bam&amp;quot;, package = &amp;quot;biovizBase&amp;quot;)&#xA;wh &amp;lt;- as(c(&amp;quot;chr17:41239394-41319151:+&amp;quot;), &amp;quot;GRanges&amp;quot;)&#xA;p_cov &amp;lt;- autoplot(fl.bam, which = wh)&#xA;#&amp;gt; reading in as Bamfile&#xA;#&amp;gt; Parsing raw coverage...&#xA;#&amp;gt; Read GAlignments from BamFile...&#xA;#&amp;gt; extracting information...&#xA;&#xA;&#xA;df &amp;lt;-  data.frame(x= sample(1:100,size = 100,replace = TRUE))&#xA;p_hist &amp;lt;- ggplot(df) + &#xA;    geom_histogram(aes(x))&#xA;&#xA;&#xA;p_hist / p_cov&#xA;#&amp;gt; Error: Can't add `e2` to a ggplot object.&#xA;#&amp;gt; Backtrace:&#xA;#&amp;gt;     █&#xA;#&amp;gt;  1. └─patchwork:::`/.ggplot`(p_hist, p_cov)&#xA;#&amp;gt;  2.   └─GGally:::`+.gg`(e1, e2)&#xA;#&amp;gt;  3.     └─e1 %+% e2&#xA;#&amp;gt;  4.       └─ggplot2:::add_ggplot(e1, e2, e2name)&#xA;#&amp;gt;  5.         ├─ggplot2::ggplot_add(object, p, objectname)&#xA;#&amp;gt;  6.         └─ggplot2:::ggplot_add.default(object, p, objectname)&#xA;&#xA;p_hist / p_cov@ggplot&#xA;#&amp;gt; `stat_bin()` using `bins = 30`. Pick better value with `binwidth`.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/tOPJq.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;sup&gt;Created on 2021-02-05 by the &lt;a href=&quot;https://reprex.tidyverse.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;reprex package&lt;/a&gt; (v1.0.0)&lt;/sup&gt;&lt;/p&gt;&#xA;Session info&#xA;&lt;p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;sessioninfo::session_info()&#xA;#&amp;gt; ─ Session info ───────────────────────────────────────────────────────────────&#xA;#&amp;gt;  setting  value                       &#xA;#&amp;gt;  version  R version 3.6.0 (2019-04-26)&#xA;#&amp;gt;  os       Red Hat Enterprise Linux    &#xA;#&amp;gt;  system   x86_64, linux-gnu           &#xA;#&amp;gt;  ui       X11                         &#xA;#&amp;gt;  language (EN)                        &#xA;#&amp;gt;  collate  en_US.UTF-8                 &#xA;#&amp;gt;  ctype    en_US.UTF-8                 &#xA;#&amp;gt;  tz       Europe/Berlin               &#xA;#&amp;gt;  date     2021-02-05                  &#xA;#&amp;gt; &#xA;#&amp;gt; ─ Packages ───────────────────────────────────────────────────────────────────&#xA;#&amp;gt;  package              * version  date       lib source        &#xA;#&amp;gt;  acepack                1.4.1    2016-10-29 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  AnnotationDbi          1.48.0   2019-10-29 [1] Bioconductor  &#xA;#&amp;gt;  AnnotationFilter       1.10.0   2019-10-29 [1] Bioconductor  &#xA;#&amp;gt;  askpass                1.1      2019-01-13 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  assertthat             0.2.1    2019-03-21 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  backports              1.1.8    2020-06-17 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  base64enc              0.1-3    2015-07-28 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  Biobase                2.46.0   2019-10-29 [1] Bioconductor  &#xA;#&amp;gt;  BiocFileCache          1.10.2   2019-11-08 [1] Bioconductor  &#xA;#&amp;gt;  BiocGenerics         * 0.32.0   2019-10-29 [1] Bioconductor  &#xA;#&amp;gt;  BiocManager            1.30.10  2019-11-16 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  BiocParallel           1.20.1   2019-12-21 [1] Bioconductor  &#xA;#&amp;gt;  biomaRt                2.42.1   2020-03-26 [1] Bioconductor  &#xA;#&amp;gt;  Biostrings             2.54.0   2019-10-29 [1] Bioconductor  &#xA;#&amp;gt;  biovizBase             1.34.1   2019-12-04 [1] Bioconductor  &#xA;#&amp;gt;  bit                    1.1-15.2 2020-02-10 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  bit64                  0.9-7    2017-05-08 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  bitops                 1.0-6    2013-08-17 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  blob                   1.2.1    2020-01-20 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  BSgenome               1.54.0   2019-10-29 [1] Bioconductor  &#xA;#&amp;gt;  checkmate              2.0.0    2020-02-06 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  cli                    2.0.2    2020-02-28 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  cluster                2.0.8    2019-04-05 [2] CRAN (R 3.6.0)&#xA;#&amp;gt;  colorspace             1.4-1    2019-03-18 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  crayon                 1.3.4    2017-09-16 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  curl                   4.3      2019-12-02 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  data.table             1.12.8   2019-12-09 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  DBI                    1.1.0    2019-12-15 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  dbplyr                 1.4.2    2019-06-17 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  DelayedArray           0.12.2   2020-01-06 [1] Bioconductor  &#xA;#&amp;gt;  dichromat              2.0-0    2013-01-24 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  digest                 0.6.25   2020-02-23 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  dplyr                  0.8.5    2020-03-07 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  ellipsis               0.3.1    2020-05-15 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  ensembldb              2.10.2   2019-11-20 [1] Bioconductor  &#xA;#&amp;gt;  evaluate               0.14     2019-05-28 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  fansi                  0.4.1    2020-01-08 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  farver                 2.0.3    2020-01-16 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  foreign                0.8-71   2018-07-20 [2] CRAN (R 3.6.0)&#xA;#&amp;gt;  Formula                1.2-3    2018-05-03 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  fs                     1.4.0    2020-03-31 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  GenomeInfoDb           1.22.1   2020-03-27 [1] Bioconductor  &#xA;#&amp;gt;  GenomeInfoDbData       1.2.2    2020-04-01 [1] Bioconductor  &#xA;#&amp;gt;  GenomicAlignments      1.22.1   2019-11-12 [1] Bioconductor  &#xA;#&amp;gt;  GenomicFeatures        1.38.2   2020-02-15 [1] Bioconductor  &#xA;#&amp;gt;  GenomicRanges          1.38.0   2019-10-29 [1] Bioconductor  &#xA;#&amp;gt;  GGally                 1.5.0    2020-03-25 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  ggbio                * 1.34.0   2019-10-29 [1] Bioconductor  &#xA;#&amp;gt;  ggplot2              * 3.3.2    2020-06-19 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  glue                   1.4.1    2020-05-13 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  graph                  1.64.0   2019-10-29 [1] Bioconductor  &#xA;#&amp;gt;  gridExtra              2.3      2017-09-09 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  gtable                 0.3.0    2019-03-25 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  highr                  0.8      2019-03-20 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  Hmisc                  4.4-0    2020-03-23 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  hms                    0.5.3    2020-01-08 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  htmlTable              1.13.3   2019-12-04 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  htmltools              0.5.0    2020-06-16 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  htmlwidgets            1.5.1    2019-10-08 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  httr                   1.4.1    2019-08-05 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  IRanges                2.20.2   2020-01-13 [1] Bioconductor  &#xA;#&amp;gt;  jpeg                   0.1-8.1  2019-10-24 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  knitr                  1.29     2020-06-23 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  labeling               0.3      2014-08-23 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  lattice                0.20-38  2018-11-04 [2] CRAN (R 3.6.0)&#xA;#&amp;gt;  latticeExtra           0.6-29   2019-12-19 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  lazyeval               0.2.2    2019-03-15 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  lifecycle              0.2.0    2020-03-06 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  magrittr               1.5      2014-11-22 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  Matrix                 1.2-17   2019-03-22 [2] CRAN (R 3.6.0)&#xA;#&amp;gt;  matrixStats            0.56.0   2020-03-13 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  memoise                1.1.0    2017-04-21 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  mime                   0.9      2020-02-04 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  munsell                0.5.0    2018-06-12 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  nnet                   7.3-12   2016-02-02 [2] CRAN (R 3.6.0)&#xA;#&amp;gt;  openssl                1.4.1    2019-07-18 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  OrganismDbi            1.28.0   2019-10-29 [1] Bioconductor  &#xA;#&amp;gt;  patchwork            * 1.0.0    2019-12-01 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  pillar                 1.4.3    2019-12-20 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  pkgconfig              2.0.3    2019-09-22 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  plyr                   1.8.6    2020-03-03 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  png                    0.1-7    2013-12-03 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  prettyunits            1.1.1    2020-01-24 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  progress               1.2.2    2019-05-16 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  ProtGenerics           1.18.0   2019-10-29 [1] Bioconductor  &#xA;#&amp;gt;  purrr                  0.3.4    2020-04-17 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  R6                     2.4.1    2019-11-12 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  rappdirs               0.3.1    2016-03-28 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  RBGL                   1.62.1   2019-10-30 [1] Bioconductor  &#xA;#&amp;gt;  RColorBrewer           1.1-2    2014-12-07 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  Rcpp                   1.0.4.6  2020-04-09 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  RCurl                  1.98-1.1 2020-01-19 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  reprex                 1.0.0    2021-01-27 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  reshape                0.8.8    2018-10-23 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  reshape2               1.4.4    2020-04-09 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  rlang                  0.4.6    2020-05-02 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  rmarkdown              2.1      2020-01-20 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  rpart                  4.1-15   2019-04-12 [2] CRAN (R 3.6.0)&#xA;#&amp;gt;  Rsamtools              2.2.3    2020-02-23 [1] Bioconductor  &#xA;#&amp;gt;  RSQLite                2.2.0    2020-01-07 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  rstudioapi             0.11     2020-02-07 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  rtracklayer            1.46.0   2019-10-29 [1] Bioconductor  &#xA;#&amp;gt;  S4Vectors              0.24.3   2020-01-18 [1] Bioconductor  &#xA;#&amp;gt;  scales                 1.1.1    2020-05-11 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  sessioninfo            1.1.1    2018-11-05 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  stringi                1.4.6    2020-02-17 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  stringr                1.4.0    2019-02-10 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  styler                 1.3.2    2020-02-23 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  SummarizedExperiment   1.16.1   2019-12-19 [1] Bioconductor  &#xA;#&amp;gt;  survival               3.1-11   2020-03-07 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  tibble                 3.0.0    2020-03-30 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  tidyselect             1.0.0    2020-01-27 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  VariantAnnotation      1.32.0   2019-10-29 [1] Bioconductor  &#xA;#&amp;gt;  vctrs                  0.2.4    2020-03-10 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  withr                  2.4.1    2021-01-26 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  xfun                   0.15     2020-06-21 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  XML                    3.99-0.3 2020-01-20 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  xml2                   1.3.0    2020-04-01 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  XVector                0.26.0   2019-10-29 [1] Bioconductor  &#xA;#&amp;gt;  yaml                   2.2.1    2020-02-01 [1] CRAN (R 3.6.0)&#xA;#&amp;gt;  zlibbioc               1.32.0   2019-10-29 [1] Bioconductor  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/p&gt;&#xA;&#xA;" OwnerUserId="10434" LastActivityDate="2021-02-05T11:49:16.253" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15335" PostTypeId="2" ParentId="3958" CreationDate="2021-02-05T13:11:45.763" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/wilkox/gggenes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gggenes&lt;/a&gt; is a ggplot2 based solution and looks very appealing. Plus, it has a nice introduction with &lt;a href=&quot;https://wilkox.org/gggenes/articles/introduction-to-gggenes.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;examples&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(ggplot2)&#xA;library(gggenes)&#xA;ggplot(example_genes, aes(xmin = start, xmax = end, y = molecule, fill = gene)) +&#xA;  geom_gene_arrow() +&#xA;  facet_wrap(~ molecule, scales = &amp;quot;free&amp;quot;, ncol = 1) +&#xA;  scale_fill_brewer(palette = &amp;quot;Set3&amp;quot;) +&#xA;  theme_genes()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://wilkox.org/gggenes/articles/introduction-to-gggenes_files/figure-html/theme_genes-1.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;" OwnerUserId="10434" LastActivityDate="2021-02-05T13:11:45.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15336" PostTypeId="2" ParentId="15326" CreationDate="2021-02-05T13:55:38.527" Score="0" Body="&lt;p&gt;I tend to do this using the &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2651/298&quot;&gt;FastaToTbl and TblToFasta scripts I have posted before&lt;/a&gt;. &lt;code&gt;FastaToTbl&lt;/code&gt; will convert fastq sequences to single-line, tab-separated recors which makes them far easier to parse. Using these two scripts, you can do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ FastaToTbl file.fa | grep -vwf idList | TblToFasta &#xA;&amp;gt;m64041_200717_231916/77/1941_50622 id=3115 &#xA;TGAGCCGCACCTCCTCAATCTGCAGCAGTTGAGGCCACTACCCTTCTGCTCAATGGTTCC&#xA;TGCAGACTTTATCATAGTCACTCACACTTGTCCATGAG&#xA;&amp;gt;m64041_200717_231916/162/0_6615 id=857 &#xA;CAGACTTTATCATAGTCACTCACACTTGTCCATGAGGCAGACTTTATCATAGTCACTCAC&#xA;ACTTG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;-v&lt;/code&gt; option will tell &lt;code&gt;grep&lt;/code&gt; to return non-matching lines, and the &lt;code&gt;-f&lt;/code&gt; option tells it to read the patterns to search for from a file. In this case, we give it the file with the IDs to skip (&lt;code&gt;idList&lt;/code&gt;). The &lt;code&gt;-w&lt;/code&gt; ensures we only find &amp;quot;whole word&amp;quot; matches, so that something like &lt;code&gt;m64041_200717_231916/77/1941_50622 id=31151111&lt;/code&gt; will not count as a match for &lt;code&gt;m64041_200717_231916/77/1941_50622 id=31151111&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Another approach would be to just use &lt;code&gt;awk&lt;/code&gt; directly:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ awk '(NR==FNR){s[&amp;quot;&amp;gt;&amp;quot;$0]++} (/^&amp;gt;/){ s[$0] ? a=0 : a=1}a' idList file.fa &#xA;&amp;gt;m64041_200717_231916/77/1941_50622 id=3115&#xA;TGAGCCGCACCTCCTCAATCTGCAGCAGTTGAGGCCACTACCCTTCTGCTCAATGGTTCCTGCAGACTTTATCATAGTCACTCACACTTGTCCATGAG&#xA;&amp;gt;m64041_200717_231916/162/0_6615 id=857&#xA;CAGACTTTATCATAGTCACTCACACTTGTCCATGAGGCAGACTTTATCATAGTCACTCACACTTG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here is the same &lt;code&gt;awk&lt;/code&gt; script slightly expanded to make it easier to understand:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; awk '{&#xA;        if (NR==FNR){idsToSkip[&amp;quot;&amp;gt;&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$0]++} &#xA;        else if (/^&amp;gt;/){ &#xA;            if($&lt;/span&gt;0 in idsToSkip){&#xA;                wantThisId = 0&#xA;            }&#xA;            else{&#xA;                wantThisId = 1&#xA;            }&#xA;        }&#xA;        if(wantThisId == 1){&#xA;            print&#xA;        }&#xA;      }' idList file.fa &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2021-02-05T13:55:38.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15337" PostTypeId="1" CreationDate="2021-02-05T16:49:59.000" Score="0" ViewCount="27" Body="&lt;p&gt;I'm trying to check the quality of my paired end read sequencing data. I am following this pipeline (&lt;a href=&quot;https://f1000research.com/articles/4-1062#ref-21&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://f1000research.com/articles/4-1062#ref-21&lt;/a&gt;) which uses QuasR in the first step.&lt;/p&gt;&#xA;&lt;p&gt;My list of paired files is stored in &lt;code&gt;sampleFile&lt;/code&gt;. Currently this file cannot be read into the function. Not sure why. I'm using R studio on a windows operating system.&lt;/p&gt;&#xA;&lt;p&gt;Also open to using other alignment or QC pathways if I can align to multiple different genomes.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;auxFile &amp;lt;- &amp;quot;Ref_Genomes/auxiliaries.txt&amp;quot;&#xA;sampleFile &amp;lt;- &amp;quot;./HCV_rna_paired.txt&amp;quot;&#xA;genomeName &amp;lt;- &amp;quot;BSgenome.Hsapiens.UCSC.hg38&amp;quot;&#xA;td &amp;lt;- tempdir()&#xA;    &#xA;proj1&amp;lt;- qAlign(sampleFile, genome=genomeName, auxiliaryFile=auxFile)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error: cannot open ./HCV_rna_paired.txt&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="10436" LastEditorUserId="298" LastEditDate="2021-02-05T17:23:10.520" LastActivityDate="2021-02-05T17:23:10.520" Title="Trouble aligning next generation sequencing data to reference genomes using QuasR package in Bioconductor. Cannot import .txt" Tags="|rna-seq|sequence-alignment|bioconductor|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15338" PostTypeId="1" CreationDate="2021-02-03T17:54:16.117" Score="0" ViewCount="86" Body="&lt;p&gt;I'd like to retreive all the genes belonging to a GO term.&lt;/p&gt;&#xA;&lt;p&gt;I tried (in R):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(org.Hs.eg.db)&#xA;&#xA;xx &amp;lt;- as.list(org.Hs.egGO2ALLEGS)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;xx&lt;/code&gt; is a list of GO terms, and for each GO term there's a list of the corresponding genes with an &amp;quot;entrezID&amp;quot; format.&lt;/p&gt;&#xA;&lt;p&gt;The problem is that when I check this output on other databases (GeneCards, etc...) I can't find the same results and correspondence between the genes and the GO term that I found with &lt;code&gt;org.Hs.eg.db&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Do you know a better way? Am I missing something here?&lt;/p&gt;&#xA;" OwnerDisplayName="Francois Piumi" LastActivityDate="2021-02-06T09:34:02.520" Title="How to retreive all the genes belonging to one GO term" Tags="|r|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15339" PostTypeId="2" ParentId="15321" CreationDate="2021-02-06T22:04:12.167" Score="2" Body="&lt;p&gt;There is no need for a highlevel language in such a simple case, an &lt;code&gt;awk&lt;/code&gt; oneliner does just fine:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk -F &quot; &quot; 'FNR==NR {a[$1]; next} $&lt;/span&gt;1 in a' List2 List1&#xA;m64041_200717_231916/100664277/0_42364 id=2440&#xA;m64041_200717_231916/100665802/0_46854 id=2440067&#xA;m64041_200717_231916/100928073/0_11804 id=2440152&#xA;m64041_200717_231916/10094492/0_44157 id=2440199&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-02-06T22:04:12.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15340" PostTypeId="2" ParentId="14818" CreationDate="2021-02-07T21:21:09.287" Score="0" Body="&lt;p&gt;Here's a solution using &lt;code&gt;pandas&lt;/code&gt; if you want to save the tsv:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio import SeqIO&#xA;import pandas as pd&#xA;from io import StringIO&#xA;&#xA;example = &amp;quot;&amp;quot;&amp;quot;&#xA;&amp;gt;seq0&#xA;FQTWEEFSRAAEKLYLADPMKVRVVLKYRHVDGNLCIKVTDDLVCLVYRTDQAQDVKKIEKF&#xA;&amp;gt;seq1&#xA;KYRTWEEFTRAAEKLYQADPMKVRVVLKYRHCDGNLCIKVTDDVVCLLYRTDQAQDVKKIEKFHSQLMRLMELKVTDNKECLKFKTDQAQEAKKMEKLNNIFFTLM&#xA;&amp;gt;seq2&#xA;EEYQTWEEFARAAEKLYLTDPMKVRVVLKYRHCDGNLCMKVTDDAVCLQYKTDQAQDVKKVEKLHGK&#xA;&amp;gt;seq3&#xA;MYQVWEEFSRAVEKLYLTDPMKVRVVLKYRHCDGNLCIKVTDNSVCLQYKTDQAQDVK&#xA;&amp;gt;seq4&#xA;EEFSRAVEKLYLTDPMKVRVVLKYRHCDGNLCIKVTDNSVVSYEMRLFGVQKDNFALEHSLL&#xA;&amp;gt;seq5&#xA;SWEEFAKAAEVLYLEDPMKCRMCTKYRHVDHKLVVKLTDNHTVLKYVTDMAQDVKKIEKLTTLLMR&#xA;&amp;gt;seq6&#xA;FTNWEEFAKAAERLHSANPEKCRFVTKYNHTKGELVLKLTDDVVCLQYSTNQLQDVKKLEKLSSTLLRSI&#xA;&amp;gt;seq7&#xA;SWEEFVERSVQLFRGDPNATRYVMKYRHCEGKLVLKVTDDRECLKFKTDQAQDAKKMEKLNNIFF&#xA;&amp;gt;seq8&#xA;SWDEFVDRSVQLFRADPESTRYVMKYRHCDGKLVLKVTDNKECLKFKTDQAQEAKKMEKLNNIFFTLM&#xA;&amp;gt;seq9&#xA;KNWEDFEIAAENMYMANPQNCRYTMKYVHSKGHILLKMSDNVKCVQYRAENMPDLKK&#xA;&amp;gt;seq10&#xA;FDSWDEFVSKSVELFRNHPDTTRYVVKYRHCEGKLVLKVTDNHECLKFKTDQAQDAKKMEK&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;# This example just happens to be a string, just load your&#xA;# fasta file using the method you're already using&#xA;example_records = SeqIO.parse( StringIO(example), 'fasta')&#xA;&#xA;# Dictionary to hold the data you eventually want in the tsv&#xA;data = {&amp;quot;Gene name&amp;quot; : list(),&#xA;        &amp;quot;Gene length&amp;quot; : list(),&#xA;        &amp;quot;Gene seq&amp;quot; : list()}&#xA;&#xA;# Append the necessary into the data dictionary&#xA;for record in example_records:&#xA;    data['Gene name'].append(record.description)&#xA;    data['Gene length'].append(len(record.seq))&#xA;    data['Gene seq'].append(str(record.seq))&#xA;&#xA;# Convert your data into a pandas DataFrame and save as a tsv&#xA;gene_df = pd.DataFrame(data)&#xA;gene_df.to_csv(&amp;quot;gene_info.tsv&amp;quot;, sep = '\t', index = False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This results in a tsv that looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ head gene_info.tsv&#xA;Gene name       Gene length     Gene seq&#xA;seq0    62      FQTWEEFSRAAEKLYLADPMKVRVVLKYRHVDGNLCIKVTDDLVCLVYRTDQAQDVKKIEKF&#xA;seq1    106     KYRTWEEFTRAAEKLYQADPMKVRVVLKYRHCDGNLCIKVTDDVVCLLYRTDQAQDVKKIEKFHSQLMRLMELKVTDNKECLKFKTDQAQEAKKMEKLNNIFFTLM&#xA;seq2    67      EEYQTWEEFARAAEKLYLTDPMKVRVVLKYRHCDGNLCMKVTDDAVCLQYKTDQAQDVKKVEKLHGK&#xA;seq3    58      MYQVWEEFSRAVEKLYLTDPMKVRVVLKYRHCDGNLCIKVTDNSVCLQYKTDQAQDVK&#xA;seq4    62      EEFSRAVEKLYLTDPMKVRVVLKYRHCDGNLCIKVTDNSVVSYEMRLFGVQKDNFALEHSLL&#xA;seq5    66      SWEEFAKAAEVLYLEDPMKCRMCTKYRHVDHKLVVKLTDNHTVLKYVTDMAQDVKKIEKLTTLLMR&#xA;seq6    70      FTNWEEFAKAAERLHSANPEKCRFVTKYNHTKGELVLKLTDDVVCLQYSTNQLQDVKKLEKLSSTLLRSI&#xA;seq7    65      SWEEFVERSVQLFRGDPNATRYVMKYRHCEGKLVLKVTDDRECLKFKTDQAQDAKKMEKLNNIFF&#xA;seq8    68      SWDEFVDRSVQLFRADPESTRYVMKYRHCDGKLVLKVTDNKECLKFKTDQAQEAKKMEKLNNIFFTLM&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Hopefully this helps!&lt;/p&gt;&#xA;" OwnerUserId="1150" LastActivityDate="2021-02-07T21:21:09.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15341" PostTypeId="1" CreationDate="2021-02-08T01:19:08.297" Score="1" ViewCount="78" Body="&lt;p&gt;Is there a good definition of pseudotime? Some tools are clearly labeled as pseudotime and produce values along a trajectory, but there are more complex approaches that involve branching and disconnected populations. There are 70 of them in the &lt;a href=&quot;https://dynverse.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dynverse&lt;/a&gt;, so there are many options.&lt;/p&gt;&#xA;&lt;p&gt;UMAP will also order cells, but that is generally not considered pseudotime. You can even overlay Monocle or PAGA results onto a UMAP so the cells are still ordered the same way. SPRING is specifically designed to capture continuous trajectories, but is also not considered pseudotime. Why not?&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2021-02-08T11:17:44.783" Title="Is there a difference between any scRNA-seq visualization and pseudotime?" Tags="|scrnaseq|visualization|umap|monocle|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15342" PostTypeId="2" ParentId="15317" CreationDate="2021-02-08T08:14:11.463" Score="1" Body="&lt;p&gt;How to separate reads, one for each reference haplotype.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Map all the reads against the reference genome&lt;/li&gt;&#xA;&lt;li&gt;Use GATK to call SNPs&lt;/li&gt;&#xA;&lt;li&gt;Use GATK tool [FastaAlternateReferenceMaker][1] to build the alternative reference&lt;/li&gt;&#xA;&lt;li&gt;You now have the two &amp;quot;haploid references&amp;quot; (quotes needed because you have to take into account that they will not necessarily reflect the exact haplotypic structure, it could be a mix of the two haplotypes)&lt;/li&gt;&#xA;&lt;li&gt;Align the reads against the two references, and attribute each read to the reference to which they map better. My colleagues developed for a different purpose (detection of Allelic Imbalance) a pipeline that includes the step for determining to which of two references a read belongs. The tool is called [BayesASE][2]&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In general you may study all the work that has been done in the field of allelic imbalance and allele specific expression, where this problem has been encountered and solved.&#xA;[1]: &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360037594571-FastaAlternateReferenceMaker&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360037594571-FastaAlternateReferenceMaker&lt;/a&gt;&#xA;[2]: &lt;a href=&quot;https://github.com/McIntyre-Lab/BayesASE&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/McIntyre-Lab/BayesASE&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1140" LastActivityDate="2021-02-08T08:14:11.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15343" PostTypeId="2" ParentId="15275" CreationDate="2021-02-08T08:46:17.553" Score="2" Body="&lt;p&gt;After checking with &lt;code&gt;file.exists()&lt;/code&gt;, I would suggest you try the same command with another SRA run. It may happen that some runs are not accessible for reasons I do not know. I always have problems in accessing sra formatted files and use a workaround to download fastq.&lt;/p&gt;&#xA;&lt;p&gt;Here is my not official workaround:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;runlist&amp;lt;-sraConvert(&amp;quot;SRP045534&amp;quot;,sra_con=sra_con)&#xA;runlist[1:2,]&#xA;&#xA;study submission    sample experiment        run&#xA;1 SRP045534  SRA179382 SRS685546  SRX681992 SRR1552451&#xA;2 SRP045534  SRA179382 SRS685544  SRX681990 SRR1552449&#xA;&#xA;for (dloadrun in runlist$run) &#xA;{&#xA;    getSRAfile(dloadrun,sra_con=sra_con,fileType=&amp;quot;fastq&amp;quot;)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This should iteratively download all the runs associated to the project you are studying.&#xA;It is possible that you can do the same with sra type file, but I have never been able to use them with sraDB, and never investigated the issue.&lt;/p&gt;&#xA;" OwnerUserId="1140" LastEditorUserId="1140" LastEditDate="2021-02-08T13:23:20.647" LastActivityDate="2021-02-08T13:23:20.647" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15345" PostTypeId="2" ParentId="15341" CreationDate="2021-02-08T11:17:44.783" Score="1" Body="&lt;p&gt;Pseudotime is the order of cells along a trajectory which in the simplest case is a minimum-spanning tree based on a (low-dimensional) manifold such as PCA or UMAP. Without a defined trajectory you have no pseudotime, and therefore UMAP is not pseudotime, but in fact can be used to define the trajectory.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-02-08T11:17:44.783" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15346" PostTypeId="1" CreationDate="2021-02-08T12:06:05.833" Score="1" ViewCount="499" Body="&lt;p&gt;I'm looking for an easy way to retrieve all the genes in a list that are associated with a certain GO term, preferably using R/Bioconductor packages. I'm not interested in under/overrepresentation or enrichment.&lt;/p&gt;&#xA;&lt;p&gt;For instance, I want a list of all genes known to be located in 'presynaptic endosome' (GO:009883).&lt;/p&gt;&#xA;&lt;p&gt;I tried the method referred to in an older post (&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/5398/retrieve-all-genes-for-a-specific-gene-ontology&quot;&gt;Retrieve all genes for a specific gene ontology&lt;/a&gt; and &lt;a href=&quot;https://www.biostars.org/p/52101/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/52101/&lt;/a&gt;); following is my code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ensembl = useMart(&amp;quot;ensembl&amp;quot;,dataset=&amp;quot;hsapiens_gene_ensembl&amp;quot;) #retrieve human.ensembl data via biomaRt&#xA;&#xA;go_1 = c(&amp;quot;GO:0098830&amp;quot;, &amp;quot;GO:0098954&amp;quot;, &amp;quot;GO:0099007&amp;quot;, &amp;quot;GO:0099067&amp;quot;, &amp;quot;GO:0099037&amp;quot;, &amp;quot;GO:0098955&amp;quot;, &amp;quot;GO:0099592&amp;quot;, &amp;quot;GO:0099532&amp;quot;) #GO IDs of all terms associated with &amp;quot;presynaptic endosome&amp;quot; (inclusive of child terms) &#xA;&#xA;pre.gene.data &amp;lt;- getBM(attributes=c('hgnc_symbol', 'ensembl_gene_id', 'go_id', 'go_linkage_type'),&#xA;                    filters = 'go', values = list(go_1), mart = ensembl)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but my code doesn't filter the genes based on the GO terms. It gives me the following output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GO:0006886 AP3D1 ENSG00000065000 intracellular protein transport IEA&#xA;GO:0016192 AP3D1 ENSG00000065000 vesicle-mediated transport IEA&#xA;GO:0030117 AP3D1 ENSG00000065000 membrane coat IEA&#xA;&#xA;... and 49 other entries for the gene &amp;quot;AP3D1&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm unable to understand what might be the problem. I replaced the filter &amp;quot;go_id&amp;quot; to &amp;quot;go&amp;quot; because of the updated filters used:listFilters(ensembl).&lt;/p&gt;&#xA;&lt;p&gt;Please help!&lt;/p&gt;&#xA;" OwnerUserId="10449" LastEditorUserId="3541" LastEditDate="2021-02-08T13:33:05.857" LastActivityDate="2022-07-03T16:04:15.433" Title="Retrieving all genes for a gene ontology term" Tags="|bioconductor|gene|biomart|gene-ontology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15348" PostTypeId="2" ParentId="15346" CreationDate="2021-02-08T14:10:15.667" Score="1" Body="&lt;p&gt;It is filtering by your GO terms, it is only giving you genes associated with those GO terms. However the GO terms in the output are not linked to the GO terms in your input, so it is giving you all the GO terms linked to the genes which are associated with the GO terms you filtered by.&lt;/p&gt;&#xA;&lt;p&gt;Essentially BioMart is not very clever. It has a table of all the GO terms linked to the genes, it filters the genes to only include those linked to the GO terms you've given it. Then it takes that list of genes and queries that same table completely separately, getting the GO terms for each gene.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2021-02-08T14:10:15.667" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15350" PostTypeId="1" CreationDate="2021-02-09T08:09:30.117" Score="2" ViewCount="156" Body="&lt;p&gt;I would like to compare GATK with graph-based variant callers. I have seen the &lt;a href=&quot;https://www.nature.com/articles/s41588-018-0316-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fast and accurate genomic analyses using genome graphs&lt;/a&gt; paper from &lt;a href=&quot;https://www.sevenbridges.com/pan-genome-graphs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SevenBridges&lt;/a&gt; and the &lt;a href=&quot;https://github.com/Illumina/paragraph&quot; rel=&quot;nofollow noreferrer&quot;&gt;paragraph&lt;/a&gt; tool by Illumina, though it does not look like folks use their platform or it is not open-sourced (in the case of SevenBridges's GRAF).&lt;/p&gt;&#xA;&lt;p&gt;The idea is to see if &lt;a href=&quot;https://www.biomedcentral.com/collections/graphgenomes&quot; rel=&quot;nofollow noreferrer&quot;&gt;graph-based approaches&lt;/a&gt; detect better both SNPs and structural variants compared with existing methods. Additionally, I believe that if there exists some novel graph-based aligner/assembler one could use the metadata downstream to identify variants.&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2021-03-31T22:32:32.083" LastActivityDate="2021-03-31T22:32:32.083" Title="Are there open-sourced graph-based variant callers?" Tags="|variant-calling|reference-genome|graphs|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15351" PostTypeId="1" CreationDate="2021-02-09T14:58:09.343" Score="1" ViewCount="358" Body="&lt;p&gt;I am seeking clarification on how to read the  &lt;code&gt;E&lt;/code&gt; value annotation on NCBI's BLAST results.&lt;/p&gt;&#xA;&lt;p&gt;What I understand so far:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;E -values are the expected value of the number of random querys of the same length as the input that would result in a even or better S-score on the same sized database. Since E-values are estimates, not probabilities , they can be lower than 0.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However, if NCBI BLAST retuns an e-value of say: &amp;quot;4e -19&amp;quot; do they mean:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$ {4 \cdot 10^{-19}} $&lt;/span&gt; (this would not result in negative values )&lt;/li&gt;&#xA;&lt;li&gt;or do they mean 4 E-19 as in : &amp;quot;&lt;em&gt;four queries score better in S with a -19 difference&lt;/em&gt;&amp;quot;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9670" LastEditorUserId="3848" LastEditDate="2022-07-30T12:30:29.490" LastActivityDate="2022-07-30T12:30:29.490" Title="How to read E-value annotation on NCBI BLAST?" Tags="|sequence-alignment|statistics|blast|phylogenetics|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="15352" PostTypeId="1" AcceptedAnswerId="15353" CreationDate="2021-02-09T15:45:06.090" Score="0" ViewCount="114" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/489848/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have a doubt, are supplementary alignment usually considered when the variant allele frequency is calculated? Thanks a lot.&lt;/p&gt;&#xA;&lt;p&gt;In my case I have some Illumina short data edited with Crisp-Cas9 and I want to calculate for a specific position all the frequency of the 4 nucleotides. I would like to understand if the supplementary reads aligned in that position must be taken into consideration.&lt;/p&gt;&#xA;&lt;p&gt;I've used pysamstats to calculate and it consider them, but my doubt is more if it right to consider them, if it is normally done or not. We are doing it on human and we expect only a change of a single nucleotide variant in the region if the editing works.&lt;/p&gt;&#xA;&lt;p&gt;I'm interested in what is done generally, if there is a &amp;quot;rule&amp;quot; to consider or not the supplementary alignment when the AF is calculated.&#xA;In my case I have some Illumina short data edited with Crisp-Cas9 and I want to calculate for a specific position all the frequency of the 4 nucleotides. I was trying to understand if the supplementary reads aligned in that position must be taken into consideration.&lt;/p&gt;&#xA;&lt;p&gt;GATK HaplotypeCaller and Mutect2 take them into consideration? I'm not looking for a specific variant caller, but if I want to take one variant caller as &amp;quot;reference&amp;quot;, I would probably look at them.&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot.&lt;/p&gt;&#xA;" OwnerUserId="8933" LastEditorUserId="73" LastEditDate="2021-05-09T06:59:31.593" LastActivityDate="2021-05-09T06:59:31.593" Title="Supplementary aligments in VAF" Tags="|sequence-alignment|snp|ngs|variant-calling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15353" PostTypeId="2" ParentId="15352" CreationDate="2021-02-09T21:20:16.173" Score="1" Body="&lt;p&gt;It depends on the tool and the use case but generally they should be.&lt;/p&gt;&#xA;&lt;p&gt;It's not as important with short reads as only a small portion of reads have supplementary mapping (unless you're looking at breast cancer post chromothripsis, for example). But imagine the case where you're working with long reads (as I do). I have samples with a mean read length of 60kb. The human reference is structurally different to any given sample meaning that the chance of any given long read having supplementary mapping is high. If supplementary mapping didn't count towards allele counts then many would be artificially low. This is my use case, others are different.&lt;/p&gt;&#xA;&lt;p&gt;I would count them unless you have some reason to suspect that they are are artefact of the editing and or the mapping. Sorry to never give a straight answer but that's bioinformatics! From what you described I would guess 99% are real but there's normally some noise and if that's of concern your code should include some QC. I.e, consider if multiple reads support the sup mapping and their quality.&lt;/p&gt;&#xA;" OwnerUserId="7153" LastEditorUserId="73" LastEditDate="2021-05-09T06:58:13.357" LastActivityDate="2021-05-09T06:58:13.357" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15356" PostTypeId="2" ParentId="15350" CreationDate="2021-02-10T12:32:02.733" Score="4" Body="&lt;p&gt;Yes, there are. There are some suggestions in the comments (VG, WhatsHap, GraphAligner, Minigraph).&lt;/p&gt;&#xA;&lt;p&gt;However, to be clear, the current default variant calling algorithm used in the GATK (the haplotype caller) is a graph-based algorithm. It constructs a De Bruijn graph from mapped reads for local assembly around variants. Other tools that also follow this strategy include &lt;a href=&quot;https://www.well.ox.ac.uk/research/research-groups/lunter-group/lunter-group/platypus-a-haplotype-based-variant-caller-for-next-generation-sequence-data&quot; rel=&quot;nofollow noreferrer&quot;&gt;Platypus&lt;/a&gt; and &lt;a href=&quot;https://github.com/freebayes/freebayes&quot; rel=&quot;nofollow noreferrer&quot;&gt;freebayes&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You might also consider multi-color genome assemblers. For example, &lt;a href=&quot;http://cortexassembler.sourceforge.net/index_cortex_var.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cortex&lt;/a&gt; and &lt;a href=&quot;https://github.com/mcveanlab/Corticall&quot; rel=&quot;nofollow noreferrer&quot;&gt;Corticall&lt;/a&gt; use colored De Bruijn graphs to aggregate reads from multiple genomes and call variants.&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2021-02-10T12:32:02.733" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15357" PostTypeId="2" ParentId="15322" CreationDate="2021-02-10T15:10:28.403" Score="-2" Body="&lt;p&gt;There are plenty of books when we talk about bioinformatics but only a few of them are available for application of machine learning in bioinformatics. From these selected books I would suggest that you go ahead with &lt;a href=&quot;https://www.springer.com/gp/book/9789811524448&quot; rel=&quot;nofollow noreferrer&quot;&gt;Statistical Modelling and Machine Learning Principles for Bioinformatics Techniques, Tools, and Applications&lt;/a&gt;. Its a recent book published in 2020 and covers most of the important topics in bioinformatics along with their ML applications. Apart from that you can also refer &lt;a href=&quot;https://www.wiley.com/en-in/Machine+Learning+in+Bioinformatics-p-9780470116623&quot; rel=&quot;nofollow noreferrer&quot;&gt;Machine Learning in Bioinformatics&lt;/a&gt; it’s quite old but still it covers most of the concepts with in-depth explanation. Along with that you can refer to this amazing book &lt;a href=&quot;https://compgenomr.github.io/book/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computational Genomics with R&lt;/a&gt; available free online. This will cover both theory and code for different ML algorithms (both supervised and unsupervised) with their application in bioinformatics. It also covers important topics like RNA Seq Analysis, ChIP Seq Analysis in depth with code explanation of different real world examples which gives you head start in this beautiful subject.&lt;/p&gt;&#xA;" OwnerUserId="10470" LastEditorUserId="10470" LastEditDate="2021-02-11T04:35:22.147" LastActivityDate="2021-02-11T04:35:22.147" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15358" PostTypeId="1" AcceptedAnswerId="15359" CreationDate="2021-02-10T21:11:33.667" Score="0" ViewCount="321" Body="&lt;p&gt;I am trying to run my alignment script that works locally, using &lt;code&gt;SBATCH&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Official manual for &lt;code&gt;bowtie2&lt;/code&gt; says I can use &lt;code&gt;samtools&lt;/code&gt; for that. However when I run the following script I have an error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;line 22: bowtie2: command not found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Below is the script:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;#SBATCH --time=05:00:00&#xA;#SBATCH --account=def-myaccount&#xA;#SBATCH --mem=200000M&#xA;#SBATCH --mail-user=mymail@gmail.com&#xA;#SBATCH --mail-type=All&#xA;&#xA;&#xA;module load nixpkgs/16.09  intel/2018.3&#xA;module load samtools/1.9&#xA;&#xA;&#xA;&#xA;for FILE in *R1_001.fastq.gz;&#xA;&#xA;do&#xA;        BASENAME=${file%%_R1_001.fastq.gz*}&#xA;&#xA;        NUM=`echo $FILE | sed 's/_S.*$//'`&#xA;&#xA;        bowtie2 -x cov_nastya -U &lt;span class=&quot;math-container&quot;&gt;$FILE -S $&lt;/span&gt;{NUM}exit.sam&#xA;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6948" LastActivityDate="2021-02-10T21:35:10.410" Title="How to perform bowtie2 analysis with slurm?" Tags="|samtools|bowtie2|slurm|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15359" PostTypeId="2" ParentId="15358" CreationDate="2021-02-10T21:35:10.410" Score="3" Body="&lt;p&gt;It seems you do not have bowtie installed or loaded. Use &lt;code&gt;module load bowtie2&lt;/code&gt; together when you load &lt;code&gt;samtools&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2021-02-10T21:35:10.410" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15360" PostTypeId="1" CreationDate="2021-02-11T13:02:00.387" Score="2" ViewCount="697" Body="&lt;p&gt;I had my DNA tested by Myheritage and they sent me a csv file with RSID, Chromosome, position and result (which base) with about 700,000 rows.&#xA;I understand most analyses of DNA use VCF files but is there anything i can do with this csv file i.e. check for genetic health-related genes?&#xA;I am not a bioinformaticist but I am a scientist and I can use R and python.  Ive heard of the Gnomad database but not sure if i can match things to my csv file.&lt;/p&gt;&#xA;" OwnerUserId="10478" LastActivityDate="2021-03-13T22:03:10.307" Title="How can I use my Myheritage DNA results file for further analysis?" Tags="|vcf|phylogenetics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15361" PostTypeId="1" CreationDate="2021-02-11T14:45:15.947" Score="1" ViewCount="516" Body="&lt;p&gt;I was wondering if it is possible to download all biological assemblies from the PDB at once (so not the standard asymmetrical files). For example, I want a file containing the coordinates of all atoms in a trimer, instead of only the coordinates of a monomer.&lt;/p&gt;&#xA;&lt;p&gt;I saw that you can batch download with a shell script, but I'm not sure if you can also do that for biological assemblies.&lt;/p&gt;&#xA;" OwnerUserId="10479" LastActivityDate="2021-07-30T23:17:19.863" Title="PDB: Download all biological assemblies" Tags="|protein-structure|pdb|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15362" PostTypeId="1" AcceptedAnswerId="15365" CreationDate="2021-02-11T16:09:11.757" Score="2" ViewCount="123" Body="&lt;p&gt;As an end-user of my own data, specifically raw DNA sequence (WGS, whole genome sequencing). How or where do one obtain such DNA data so that a biology hobbyist can perform bioinformatics on this?&lt;/p&gt;&#xA;&lt;p&gt;What keywords is best describe to the search engines when finding these online WGS websites).&lt;/p&gt;&#xA;&lt;p&gt;It would be nice to have a USB stick mailed to me after such DNA HomeKit.  What file format is most commonly offered? (VCF?)&lt;/p&gt;&#xA;&lt;p&gt;Am I looking too far ahead into the future?&lt;/p&gt;&#xA;" OwnerUserId="10480" LastEditorUserId="3360" LastEditDate="2021-02-17T21:00:13.653" LastActivityDate="2021-02-17T21:00:13.653" Title="Obtaining Whole Genetic Sequence" Tags="|vcf|file-formats|wgs|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15363" PostTypeId="2" ParentId="15362" CreationDate="2021-02-11T18:45:53.407" Score="3" Body="&lt;p&gt;&amp;quot;DNA data&amp;quot; comes in several forms dependent on the technology used to produce them.&lt;/p&gt;&#xA;&lt;p&gt;Companies like 23andMe are using &lt;a href=&quot;https://en.wikipedia.org/wiki/SNP_array&quot; rel=&quot;nofollow noreferrer&quot;&gt;SNP chips&lt;/a&gt; and those are available for an only rather a limited number of species. To be completely honest, I don't even know what would be the raw data they would provide, but I would suspect it could be a vcf file.&lt;/p&gt;&#xA;&lt;p&gt;If the chosen technology would be &lt;a href=&quot;https://en.wikipedia.org/wiki/DNA_sequencing&quot; rel=&quot;nofollow noreferrer&quot;&gt;DNA sequencing&lt;/a&gt;. DNA sequencing is provided by a few dozen of sequencing centres around the world, some of them associated with universities. DNA sequencing generates files that contain individual sequencing reads with assigned quality scores (&amp;quot;how much the sequencer trusts each base&amp;quot;), the typical formats are &lt;code&gt;fastq&lt;/code&gt; (Illumina) or &lt;code&gt;.bam&lt;/code&gt; (long read technologies). Usually, you would also need to find a suitable reference in one of the public databases, for instance, the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/grc/human&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI portal&lt;/a&gt; for human reference. Then map your reads and do variant calling. Of course, then it depends on what would you be interested in.&lt;/p&gt;&#xA;&lt;p&gt;If you don't care for your own DNA and you are more interested in life around you, you can check &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ncbi&lt;/a&gt; for all sorts of genomic data. If you would be interested to understand how the raw sequencing data work, &lt;a href=&quot;https://www.ebi.ac.uk/ena/browser/home&quot; rel=&quot;nofollow noreferrer&quot;&gt;ebi&lt;/a&gt; is a good place to look. Also, in every single paper generating/using sequencing data you there should be mention where in public databases you can find them.&lt;/p&gt;&#xA;&lt;p&gt;Finally, let me warn you. Some of the genomic analyses are quite computationally heavy (not all though). If you don't have a specific question in mind, it would be wise to start with bacterial sequencing.&lt;/p&gt;&#xA;&lt;p&gt;Hope you will have fun with DNA data!&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2021-02-11T18:45:53.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15364" PostTypeId="1" AcceptedAnswerId="15367" CreationDate="2021-02-11T20:30:43.463" Score="0" ViewCount="218" Body="&lt;p&gt;I am trying to run the following script on slurm to extract information from .sam files. While everything is perfect on my local machine with the samtools version 1.11, it does not work on slurm. I have this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools: command not found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That means slurm cannot understand the module if I am not mistaken. What would be a solution here?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;&#xA;#SBATCH --time=05:00:00&#xA;#SBATCH --account=def-myaccount&#xA;#SBATCH --mem=200000M&#xA;#SBATCH --mail-user=mymail@gmail.com&#xA;#SBATCH --mail-type=All&#xA;&#xA;&#xA;module load nixpkgs/16.09  intel/2018.3&#xA;# module load samtools/1.11 # this is what I tried first&#xA;module spider samtools/1.11 # this does not help either...&#xA;&#xA;&#xA;for FILE in *.sam;&#xA;&#xA;do&#xA;&#xA;&#xA;        NUM=`echo $FILE | sed -r 's/.sam//' | sed -r 's/nastya//'`&#xA;&#xA;#         echo &amp;quot;$FILE&amp;quot; ;&#xA;&#xA;&#xA;        samtools flagstat -O tsv &lt;span class=&quot;math-container&quot;&gt;$FILE &amp;gt; file$&lt;/span&gt;{NUM}.txt&#xA;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6948" LastActivityDate="2021-02-11T22:11:45.157" Title="How can I run the latest version of samtools on Slurm?" Tags="|samtools|slurm|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15365" PostTypeId="2" ParentId="15362" CreationDate="2021-02-11T20:49:09.857" Score="2" Body="&lt;p&gt;Are you looking too far ahead into the future? No. This is certainly possible now. The gold standard being Illumina or BGI short read WGS. Long read sequencing can capture some extra data but is very noisy.&lt;/p&gt;&#xA;&lt;p&gt;How or where do one obtain such DNA data? There are now many companies. You want one the uses NGS with reads at least 150 bp and garantees at least a mean of 30x read depth. In my opinion you also want one that gives you ownership of your data. Ideally they will give you; fastqs (raw reads), a BAM (mapped fastqs) and a VCF (variant file). I good company will ask you if you want to donate your data to science. Be warned these are not defaults. Many (most?) companies won't give you your raw data citing medico legal risk and then they will on sell it to a pharma companies. I think these guys are OK (as they are in Europe with good privacy laws) &lt;a href=&quot;https://www.dantelabs.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dantelabs.com/&lt;/a&gt; but plz do your own research.&lt;/p&gt;&#xA;&lt;p&gt;Last, these data could include info pertaining to your health and your family's health. Always good to speak to a genetic counsellor to understand these things.&lt;/p&gt;&#xA;" OwnerUserId="7153" LastEditorUserId="7153" LastEditDate="2021-02-11T20:58:53.477" LastActivityDate="2021-02-11T20:58:53.477" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15366" PostTypeId="2" ParentId="15360" CreationDate="2021-02-11T21:57:23.887" Score="1" Body="&lt;p&gt;A csv file with RSID, chromosome, position and result is enough for what you want to do and these are the core columns of a VCF (which is just a TSV with some headers describing how it was made).&lt;/p&gt;&#xA;&lt;p&gt;Given you have ~700k rows I suspect that yes there will be genetic health-related SNVs (single nucleotide variant). Disclaimer; these data could include info pertaining to your health and your family's health, I strongly advise that you speak to a genetic counsellor to understand these things.&lt;/p&gt;&#xA;&lt;p&gt;Gnomad is a population database so generally speaking (plenty of exceptions of course) if there is a SNV in there more than a few times its probably not causing disease. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/clinvar/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/clinvar/&lt;/a&gt; and &lt;a href=&quot;https://omim.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://omim.org/&lt;/a&gt; are example of disease databases.&lt;/p&gt;&#xA;&lt;p&gt;Here's a pandas example with clinvar (&lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/pub/clinvar/vcf_GRCh38/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nlm.nih.gov/pub/clinvar/vcf_GRCh38/&lt;/a&gt; you'll need to match your version).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;df = pd.read_csv(&#xA;    'clinvar.vcf.gz',&#xA;    sep='\t',&#xA;    comment = '#',&#xA;    header=None, &#xA;    names = ['CHROM','POS','ID','REF','ALT','QUAL','FILTER','INFO']&#xA;)&#xA;df.head()&#xA;&#xA;CHROM   POS ID  REF ALT QUAL    FILTER  INFO&#xA;0   1   930188  846933  G   A   .   .   ALLELEID=824438;CLNDISDB=MedGen:CN517202;CLNDN...&#xA;1   1   930203  972363  C   T   .   .   ALLELEID=959431;CLNDISDB=MedGen:CN517202;CLNDN...&#xA;2   1   930248  789256  G   A   .   .   AF_ESP=0.00347;AF_EXAC=0.00622;AF_TGP=0.00280;...&#xA;3   1   930275  969662  T   G   .   .   ALLELEID=959432;CLNDISDB=MedGen:CN517202;CLNDN...&#xA;4   1   930336  843786  G   A   .   .   ALLELEID=824439;CLNDISDB=MedGen:CN517202;CLNDN...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then lets say one of your 700k SNVs is at chrom 1, position 930336 and you have an A&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print(*df.query('(CHROM == 1) &amp;amp; (POS == 930336) &amp;amp; (ALT == &amp;quot;A&amp;quot;)')['INFO'].str.split(';'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Gives a list from which you could pull out CLNSIG which here is Uncertain_significance&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;['ALLELEID=824439', 'CLNDISDB=MedGen:CN517202', 'CLNDN=not_provided', 'CLNHGVS=NC_000001.11:g.930336G&amp;gt;A', 'CLNREVSTAT=criteria_provided,_single_submitter', 'CLNSIG=Uncertain_significance', 'CLNVC=single_nucleotide_variant', 'CLNVCSO=SO:0001483', 'GENEINFO=SAMD11:148398', 'MC=SO:0001583|missense_variant', 'ORIGIN=1']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So you could easily parse your 700k SNVs and get their respective clinical info this way. Please think about this carefully before you do it though, there are many implications including life insurance and mental health! I'm of the opinion that this sort of data is useful but MANY people disagree. Further, this advice is general in nature only, I accept no liabilty for what you or anyone else chooses to do with this publicly available data.&lt;/p&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-02-11T21:57:23.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15367" PostTypeId="2" ParentId="15364" CreationDate="2021-02-11T22:11:45.157" Score="2" Body="&lt;p&gt;I would start with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;module avail&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then look to see (a) if there is a samtools module avilable and (b) exactly what it's called.&lt;/p&gt;&#xA;&lt;p&gt;If it's called samtools/1.11 then:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;module load samtools/1.11&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Should work, if it doesn't speak to your sys admin.&lt;/p&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-02-11T22:11:45.157" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15368" PostTypeId="1" CreationDate="2021-02-11T23:34:15.617" Score="0" ViewCount="125" Body="&lt;p&gt;I am using below code to plot multiple box plot but I am still struggling to understand how I can add whisker to box plot and how to change sample order like in this figure SampleType?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(reshape2)&#xA;library(ggplot2)&#xA;list.files()&#xA;morphData &amp;lt;- read.table(&amp;quot;Box.plot.chemical.txt&amp;quot;, sep=&amp;quot;\t&amp;quot;, header=TRUE)&#xA;morphData&#xA;df.m &amp;lt;- melt(morphData, id.var = &amp;quot;SampleType&amp;quot;)&#xA;&#xA;&#xA;pdf(&amp;quot;box.plot.pdf&amp;quot;, width = 18, height = 12)&#xA;&#xA;&#xA;p &amp;lt;- ggplot(data = df.m, aes(x=&amp;quot;&amp;quot;, y=value)) +&#xA;&#xA;theme_bw() +&#xA;theme(&#xA;      panel.grid.major=element_blank(),&#xA;      panel.grid.minor=element_blank(),&#xA;      plot.title = element_text(vjust = -8.5,hjust = 0.1),&#xA;      line=element_line(size=1),&#xA;     &#xA;      axis.text.x = element_text(size = 10, face =&amp;quot;bold&amp;quot;, hjust = 0.5, colour = &amp;quot;black&amp;quot;),&#xA;      axis.text.y = element_text(size = 12, face =&amp;quot;bold&amp;quot;, hjust = 0.5, colour = &amp;quot;black&amp;quot;),&#xA;      axis.ticks.x = element_blank()) +&#xA;       &#xA;&#xA;theme(strip.background=element_rect(fill=&amp;quot;white&amp;quot;)) +&#xA;theme(strip.text=element_text(color=&amp;quot;black&amp;quot;,face=&amp;quot;bold&amp;quot;, size =14)) + &#xA;geom_boxplot(aes(fill=SampleType),outlier.colour = NA) &#xA;p + facet_wrap(.~variable, scales=&amp;quot;free&amp;quot;) &#xA;dev.off();&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;update:&lt;/p&gt;&#xA;&lt;p&gt;I was able to remove the variable name from the bottom by using above updated code but still getting tick mark on x-axis, and also how I can make these label bold?&lt;/p&gt;&#xA;&lt;p&gt;update:&lt;/p&gt;&#xA;&lt;p&gt;It will be a great help if you could also guide me how I can add Tukey's posthoc significance letter in above box plot to make this plot more meaningful.  I was able to make bold facet wrap text, now please help me with posthoc test letter. I have updated the code again.&lt;/p&gt;&#xA;&lt;p&gt;update:&lt;/p&gt;&#xA;&lt;p&gt;Can you please help to change the order of samples; like I want to change the order of SampleType in this box plot and also how we can remove outlier and will it be possible to add whisker so it will look better. I was able to remove outlier by using outlier.colour = NA, Now It just whisker, but I am not able to understand How I can add whisker to box plot? I have updated the code and figure above.&lt;/p&gt;&#xA;&lt;p&gt;update:&lt;/p&gt;&#xA;&lt;p&gt;sorry, I wanted to add errorbar and with this command, it is not adding error bar on all box: &lt;code&gt;stat_boxplot(geom =&amp;quot;errorbar&amp;quot;)&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="73" LastEditDate="2021-06-07T02:29:01.383" LastActivityDate="2021-06-07T02:29:01.383" Title="Multiple box plot" Tags="|r|statistics|ggplot2|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15369" PostTypeId="2" ParentId="15368" CreationDate="2021-02-12T03:20:14.970" Score="1" Body="&lt;p&gt;what do you mean by &lt;code&gt;How I can put the variable name on the top of each box plot like Rice_P at top and at the bottom as well&lt;/code&gt;? Do you want to keep the label at both top and bottom or remove one of them? From my opinion it's not necessary to use facet in this case, since you only have one variable on x-axis in each facet now. You can just put &lt;code&gt;variable&lt;/code&gt; as x-axis, and color your box as it is now. Boxes with the same &lt;code&gt;variable&lt;/code&gt; are going to be grouped together.&lt;/p&gt;&#xA;&lt;p&gt;You can use &lt;code&gt;axis.ticks.x&lt;/code&gt; in &lt;code&gt;theme&lt;/code&gt; to remove the ticks on x-axis similar to &lt;code&gt;axis.text&lt;/code&gt;. To make label bold maybe you can try &lt;code&gt;fontface=&amp;quot;bold&amp;quot;&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Adding testing letter may be harder, you could check out ggpubr or ggprism package.&lt;/p&gt;&#xA;&lt;p&gt;you can use &lt;code&gt;as.factor()&lt;/code&gt; to change sample name to factors and plot them in order. Or you can use &lt;code&gt;scale_x_discrete(limit = &amp;lt;your sample order&amp;gt;)&lt;/code&gt; to force an order. I don't see outlier in plot. Outliers are usually black dots in box plot. And the box plot you show already has whiskers, so I 'm not sure what you want to add.&lt;/p&gt;&#xA;&lt;p&gt;I'm not sure whether it's a good idea to plot error bars in boxplot because that is not the convention. But probably you can try using &lt;code&gt;geom_errorbar&lt;/code&gt; in addition to &lt;code&gt;geom_boxplot&lt;/code&gt; as another layer but it requires more customization.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastEditorUserId="73" LastEditDate="2021-06-07T02:28:49.567" LastActivityDate="2021-06-07T02:28:49.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15370" PostTypeId="1" AcceptedAnswerId="15423" CreationDate="2021-02-12T04:33:07.777" Score="2" ViewCount="75" Body="&lt;p&gt;can anyone help me improve this pandas code?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;df = pd.DataFrame(&#xA;    [&#xA;        [&#xA;            'chr1', 222&#xA;        ],&#xA;        [&#xA;            'chr1', 233&#xA;        ],&#xA;        [&#xA;            'chr1', 2123&#xA;        ],&#xA;        [&#xA;            'chr2', 244&#xA;        ]&#xA;    ], columns = ['chrom', 'pos']&#xA;)&#xA;&#xA;df2 = pd.DataFrame(&#xA;    [&#xA;        [&#xA;            'chr1', 221, 223&#xA;        ],&#xA;        [&#xA;            'chr1', 230, 240&#xA;        ],&#xA;    ], columns = ['chrom', 'start', 'end']&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Gives me 2 dfs with genomic coordinates. The first one is an exact position:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    chrom   pos&#xA;0   chr1    222&#xA;1   chr1    233&#xA;2   chr1    2123&#xA;3   chr2    244&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and the second is a range:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    chrom   start   end&#xA;0   chr1    221 223&#xA;1   chr1    230 240&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I need to find the count of exact coordinates that are in one of the ranges (in the same chrom)&lt;/p&gt;&#xA;&lt;p&gt;This works but is slow:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;c=0&#xA;for chrom, data in df.groupby('chrom'):&#xA;    tmp = df2.query(f'chrom == &amp;quot;{chrom}&amp;quot;')&#xA;    for p in data.pos:&#xA;        for s, e in zip(tmp.start, tmp.end):&#xA;            if s &amp;lt; p &amp;lt; e:&#xA;                c+=1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then c = 2&lt;/p&gt;&#xA;&lt;p&gt;I think I can use agg to do this without iteration (and hopefully faster) but I can't get it working. Can anyone show me how?&lt;/p&gt;&#xA;&lt;p&gt;PS I am also asking this on stackoverflow.&lt;/p&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-02-24T20:43:36.393" Title="How can I use pandas agg here to avoid iteration?" Tags="|python|pandas|chromosomes|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15372" PostTypeId="1" AcceptedAnswerId="15383" CreationDate="2021-02-12T16:22:12.930" Score="1" ViewCount="56" Body="&lt;p&gt;I have a files which has columns representing genes,sample numbers and sample names (55). The genes are displayed by occurrence in the samples (highest to lowest-55 and 2).I wish to calculate the genes which may co-occur in samples.&#xA;I tried doing this with &lt;code&gt;awk 'n=x[$1,$2,$3,$4,$5,$6]{print n&amp;quot;\n&amp;quot;$0;} {x[$1,$2,$3,$4,$5,$6]=$0;}'&lt;/code&gt; adding from 1 to 55 columns, but it doesn't work.&lt;/p&gt;&#xA;&lt;p&gt;An example here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Nfib    55  A1.S1   A2.S2   A3.S3   A4.S4   A6.S6   A7.S7   A8.S8   A9.S9   A9.S9*  A10.S10 A11.S11 A12.S12 IZ_BH_2.1   IZ_BH_2.2   IZ_BH_2.3   IZ_BH_2.4   IZ_BH_2.5   IZ_BH_2.6   IZ_BH_2.7   IZ_BH_2.8   IZ_BH_2.9   IZ_BH_2.10  IZ_BH_2.11  IZ_BH_2.12  IZ_BH_3.1   IZ_BH_3.2   IZ_BH_3.3   IZ_BH_3.4   IZ_BH_3.5   IZ_BH_3.6   IZ_BH_3.7   IZ_BH_3.8   IZ_BH_3.9   IZ_BH_3.10  IZ_BH_3.11  IZ_BH_3.11* IZ_BH_3.12  IZ_BH_4.1   IZ_BH_4.2   IZ_BH_4.4   IZ_BH_4.5   IZ_BH_4.6   IZ_BH_4.7   IZ_BH_4.8   IZ_BH_4.9   IZ_BH_4.10  IZ_BH_4.11  IZ_BH_5.1   IZ_BH_5.2   IZ_BH_5.3   IZ_BH_5.4   IZ_BH_5.5   IZ_BH_5.6   IZ_BH_5.7   IZ_BH_5.8&#xA;&#xA;&#xA;Lyn 41  A1.S1   A4.S4   A7.S7   A8.S8   A9.S9   A9.S9*  A10.S10 A11.S11 IZ_BH_2.1   IZ_BH_2.2   IZ_BH_2.3   IZ_BH_2.4   IZ_BH_2.5   IZ_BH_2.6   IZ_BH_2.7   IZ_BH_2.9   IZ_BH_2.11  IZ_BH_2.12  IZ_BH_3.2   IZ_BH_3.3   IZ_BH_3.4   IZ_BH_3.5   IZ_BH_3.6   IZ_BH_3.8   IZ_BH_3.9   IZ_BH_3.10  IZ_BH_3.11  IZ_BH_4.1   IZ_BH_4.2   IZ_BH_4.3   IZ_BH_4.4   IZ_BH_4.7   IZ_BH_4.8   IZ_BH_4.11  IZ_BH_5.1   IZ_BH_5.3   IZ_BH_5.4   IZ_BH_5.5   IZ_BH_5.6   IZ_BH_5.7   IZ_BH_5.8&#xA;&#xA;&#xA;Cdk5rap2    52  A1.S1   A2.S2   A3.S3   A6.S6   A7.S7   A8.S8   A9.S9   A9.S9*  A11.S11 A12.S12 IZ_BH_2.1   IZ_BH_2.2   IZ_BH_2.3   IZ_BH_2.4   IZ_BH_2.5   IZ_BH_2.6   IZ_BH_2.7   IZ_BH_2.8   IZ_BH_2.9   IZ_BH_2.10  IZ_BH_2.11  IZ_BH_2.12  IZ_BH_3.1   IZ_BH_3.2   IZ_BH_3.3   IZ_BH_3.4   IZ_BH_3.5   IZ_BH_3.6   IZ_BH_3.7   IZ_BH_3.8   IZ_BH_3.9   IZ_BH_3.10  IZ_BH_3.11  IZ_BH_3.11* IZ_BH_3.12  IZ_BH_4.1   IZ_BH_4.2   IZ_BH_4.3   IZ_BH_4.4   IZ_BH_4.6   IZ_BH_4.8   IZ_BH_4.9   IZ_BH_4.10  IZ_BH_4.11  IZ_BH_5.1   IZ_BH_5.2   IZ_BH_5.3   IZ_BH_5.4   IZ_BH_5.5   IZ_BH_5.6   IZ_BH_5.7   IZ_BH_5.8&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The first name is of the gene, followed by number of samples and sample names.&lt;/p&gt;&#xA;&lt;p&gt;for these 3 genes these are the commonly occuring samples:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A1.S1, A7.S7, A8.S8, A9.S9*, A9.S9, A11.S11, IZ_BH_2.1, IZ_BH_2.2, IZ_BH_2.3,IZ_BH_2.4, IZ_BH_2.5,IZ_BH_2.6,IZ_BH_2.7, IZ_BH_2.9, IZ_BH_2.11,IZ_BH_2.12, IZ_BH_3.2,IZ_BH_3.3,IZ_BH_3.4, IZ_BH_3.5, IZ_BH_3.6,IZ_BH_3.8,&#xA;IZ_BH_3.9,IZ_BH_3.10,IZ_BH_3.11,IZ_BH_4.1,IZ_BH_4.2,IZ_BH_4.4,IZ_BH_4.8,IZ_BH_4.11,IZ_BH_5.1,IZ_BH_5.3,IZ_BH_5.4,IZ_BH_5.5,IZ_BH_5.6,IZ_BH_5.7,IZ_BH_5.8&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Kindly help.&lt;/p&gt;&#xA;" OwnerUserId="7236" LastEditorUserId="57" LastEditDate="2021-02-15T15:40:16.460" LastActivityDate="2021-02-16T16:22:06.593" Title="co-occuring genes in my list" Tags="|awk|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15373" PostTypeId="1" CreationDate="2021-02-12T18:02:59.847" Score="4" ViewCount="35" Body="&lt;p&gt;The repeat families predicted by &lt;a href=&quot;https://www.pnas.org/content/117/17/9451&quot; rel=&quot;nofollow noreferrer&quot;&gt;RepeatModeler&lt;/a&gt; contain known transposable elements (TEs) and unknown ones.&lt;/p&gt;&#xA;&lt;p&gt;How do we know whether some of these may actually:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;within a functional domain of a gene or&lt;/li&gt;&#xA;&lt;li&gt;represent a functional domain?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Could it be predicted because repeats may simply due to this domain appeared in multiple gene loci?&#xA;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="9975" LastEditorUserId="3848" LastEditDate="2022-02-21T23:28:28.933" LastActivityDate="2022-02-21T23:28:28.933" Title="Do RepeatModeler results contain functional domains?" Tags="|genome|sequence-annotation|repeat-elements|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15374" PostTypeId="1" CreationDate="2021-02-12T23:27:00.600" Score="0" ViewCount="57" Body="&lt;p&gt;I'd like to ask you for help with an university assignment. First let me post the entire question:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Design primers for bisulfite sequencing of the promoter of the ACE2&#xA;gene in microbat  Myotis lucifugus (based on the annotation of the&#xA;Broad Institute Myoluc2.0/myoLuc2). The gene is localized on the&#xA;contig GL429816. As a transcription start, use the start of the ACE2&#xA;transcript variant XM_023753670.1; once you find ACE2 in the UCSC&#xA;genome browser .You will find it in NCBI RefSeq track&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't want anyone to solve it for me but I do have a problem with one aspect. I am not clear on how to determine what is a valid promoter for this gene in microbat. I do know it is the upstream sequence however I am not clear on how to tell the number of upstream bases that are the actual promoter. I did find some sources for the actual promoter sequence determined by much smarter people than myself however those are for ACE in humans so I am afraid that it is not applicable in my case.&lt;/p&gt;&#xA;&lt;p&gt;I'd like someone to point me in the right direction where I could find my answer would be enough.&lt;/p&gt;&#xA;" OwnerUserId="10491" LastEditorUserId="3694" LastEditDate="2022-03-11T17:12:58.493" LastActivityDate="2022-04-10T18:01:10.103" Title="How to determine a correct promoter sequence? (Bisulfite sequencing)" Tags="|gene|ngs|methylation|homework|epigenetics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15375" PostTypeId="1" CreationDate="2021-02-13T00:43:19.973" Score="0" ViewCount="33" Body="&lt;p&gt;I've recently been pulling TLR genes for different species and keep running across an error called file.fa whenever I start pulling them. Whenever I pull the first gene it's fine and makes the file.fa but when I pull the second gene (TLR4) it comes up with a file.fa error and deleted the previous file.fa&#xA;made from when I pulled the first gene. Any feedback would be greatly appreciated.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;['NP_001273862', 'XP_006138208', 'XP_006138164', 'XP_014427015', 'XP_014429739']&#xA;Traceback (most recent call last):&#xA;  File &amp;quot;blastp.py&amp;quot;, line 104, in &amp;lt;module&amp;gt;&#xA;    SeqIO.write(record, &amp;quot;file.fa&amp;quot;, 'fasta')&#xA;  File &amp;quot;/home/dylan_w190/miniconda3/lib/python3.8/site-packages/Bio/SeqIO/__init__.py&amp;quot;, line 522, in write&#xA;    with as_handle(handle, &amp;quot;w&amp;quot;) as fp:&#xA;  File &amp;quot;/home/dylan_w190/miniconda3/lib/python3.8/contextlib.py&amp;quot;, line 113, in __enter__&#xA;    return next(self.gen)&#xA;  File &amp;quot;/home/dylan_w190/miniconda3/lib/python3.8/site-packages/Bio/File.py&amp;quot;, line 72, in as_handle&#xA;    with open(handleish, mode, **kwargs) as fp:&#xA;FileNotFoundError: [Errno 2] No such file or directory: 'file.fa'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10432" LastEditorDisplayName="user3051" LastEditDate="2021-02-13T09:19:11.563" LastActivityDate="2021-02-13T09:19:11.563" Title="Not Able to pull genes due to file.fa error" Tags="|file-formats|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15376" PostTypeId="1" CreationDate="2021-02-13T02:37:37.487" Score="1" ViewCount="81" Body="&lt;p&gt;I had a quick question on virtual libraries. I'm hoping to perform a virtual screen using a library that contains very small, soluble compounds - think glycerol and smaller (&amp;lt;100MW). As far as I can tell, the majority of virtual libraries contain mostly larger, drug-like compounds. Is anyone aware of any libraries that might contain ideally 10,000+ or so of these tiny soluble compounds?&lt;/p&gt;&#xA;&lt;p&gt;If not, any idea what the process might be (and how difficult it would be) to construct a custom virtual library enriched for these compounds?&lt;/p&gt;&#xA;&lt;p&gt;Thank you!!&lt;/p&gt;&#xA;" OwnerUserId="10492" LastEditorUserId="6322" LastEditDate="2021-02-14T13:22:33.223" LastActivityDate="2022-11-10T11:26:18.520" Title="Do you know any virtual screening libraries with small, soluble compounds?" Tags="|docking|computational-biochemistry|cheminformatics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15377" PostTypeId="2" ParentId="15374" CreationDate="2021-02-14T13:41:18.727" Score="1" Body="&lt;p&gt;I think there is no fixed size. Ideally you would have something like histone ChIP-seq or ATAC-seq data to approximate the relevant upstream interval. Since this is not always possible most people go with -500bp. Others might have different suggestions though. Maybe matching known motifs of transcription factors that regulate that promoter to the upstream region can help approximating a better window. But as Sanger sequencing usually gives about 1kb of sequence I'd just take a 500bp to 1kb window and sequence that, fwd and rev primers for sequencing to have double confirmation.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-02-14T13:41:18.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15378" PostTypeId="2" ParentId="15376" CreationDate="2021-02-14T14:00:07.977" Score="1" Body="&lt;h2&gt;Lipinski and logP&lt;/h2&gt;&#xA;&lt;p&gt;Most datasets will be filtered by &lt;a href=&quot;https://en.wikipedia.org/wiki/Lipinski%27s_rule_of_five&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Lipinski rule&lt;/a&gt;, plus in the some cases, such as &lt;a href=&quot;https://enamine.net/hit-finding/compound-collections/real-database&quot; rel=&quot;nofollow noreferrer&quot;&gt;Enamine Real&lt;/a&gt; whether they can be bought.&#xA;One criterion in the Lipinski rule filter is the logP, or octanol-water partition coefficient. This is super important, because a compound that cannot cross the membrane is likely of limited uses —as a consequence &lt;a href=&quot;https://en.wikipedia.org/wiki/Prodrug&quot; rel=&quot;nofollow noreferrer&quot;&gt;prodrugs&lt;/a&gt; are a commonly taken approach.&lt;/p&gt;&#xA;&lt;p&gt;However, albeit obvious, not all datasets in &lt;a href=&quot;https://enamine.net/hit-finding/compound-collections/real-database&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zinc&lt;/a&gt; are filtered by the Lipinski rule, they just need more browsing...&lt;/p&gt;&#xA;&lt;h2&gt;Natural?&lt;/h2&gt;&#xA;&lt;p&gt;However, you &lt;em&gt;may&lt;/em&gt; have a valid reason to eschew logP.&#xA;Nature tries its best to have intermediate metabolites that have hydrophilic handles attaches, eg. CoA, phosphates and sugars. Therefore, if you want a natural metabolite this is a completely different question. Many of these are not very purchasable, but a PDB-derived dataset of natural ligands will likely be purchasable or easily extractable. Alternatively Zinc has several natural catalogues...&lt;/p&gt;&#xA;&lt;h2&gt;Too small&lt;/h2&gt;&#xA;&lt;p&gt;Glycerol is a 3 carbon molecule and has a MW of 90 Da. Many of the old drugs are small, but not that small. For example aspirin is 180 Da, which is twice the size of glycerol. Lipinski rule says not to go over 500 Da and 250-400 Da is a lovely range. Say you have a perfectly bound glycerol, with a perfect ligand efficiency of -1.5 kcal/mol/atom, that would mean it would bind at –9 kcal/mol in the best case scenario, i.e. it is too small to bind anything with nanomolar affinity and will actually bind a lot of things. Glycerol after all is commonly cocrystallised in protein...&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://gdb.unibe.ch/downloads/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GDB datasets&lt;/a&gt; have enumerated compounds of a given size, but don't go that low.&lt;/p&gt;&#xA;&lt;h2&gt;Docking&lt;/h2&gt;&#xA;&lt;p&gt;There will be two further complications that will hinder the accuracy.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;protonated and deprotonated atoms&lt;/li&gt;&#xA;&lt;li&gt;polarisable groups&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The former requires correct conformers, the latter requires a force field that has a &lt;a href=&quot;http://docs.openmm.org/latest/api-python/generated/simtk.openmm.openmm.DrudeForce.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Drude particle&lt;/a&gt;, e.g. in openMM.&lt;/p&gt;&#xA;&lt;h2&gt;Caveat against sorting by binding score&lt;/h2&gt;&#xA;&lt;p&gt;Also, it should be said that if you sort by binding score (kcal/mol or kJ/mol) you will get bigger compounds (hence the whole point of ligand efficiency) and compounds with high logP. The former is because they will have more contact points to make, the latter is because given that ∆∆G is ∆G_bound - ∆G_unbound, an unbound high-logP compound is highly unhappy in &lt;em&gt;aqueous&lt;/em&gt; solution.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-02-14T14:00:07.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15380" PostTypeId="2" ParentId="4308" CreationDate="2021-02-15T16:38:13.927" Score="1" Body="&lt;p&gt;as mentioned in previous answers, transformations are frequently used. One commonly used method is quantile normal transformation. Basically you calculate the quantile from the original data and match it to a standard normal distribution. This transforms your data to a standard normal distribution. Even for traits that are naturally normal distributed like height, it is still a good idea to do this transformation as it has been shown to improve power as well.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2021-02-15T16:38:13.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15381" PostTypeId="1" CreationDate="2021-02-15T18:05:40.600" Score="0" ViewCount="39" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/491078/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have a challenge in analyzing GSE34941( It has 4 platforms) according to the expression of a specific gene in R. First, which is the best way to group my patients according to the expression of a gene to high- and low-expression? Second, which is the best way to remove the batch effect of multi-platform analysis here?&lt;/p&gt;&#xA;&lt;p&gt;I really appreciate any help you can provide.&lt;/p&gt;&#xA;" OwnerUserId="10502" LastEditorUserId="73" LastEditDate="2021-06-17T10:47:50.833" LastActivityDate="2021-06-17T10:47:50.833" Title="multi--platform GEO analysis based in expression of single gene" Tags="|r|microarray|geo|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15382" PostTypeId="1" AcceptedAnswerId="15384" CreationDate="2021-02-16T11:42:09.307" Score="0" ViewCount="76" Body="&lt;p&gt;I want to check the differential expression of a specific class of transcripts (say, long non-coding RNAs) using DESeq2. Now, I know that the normalization step takes into account the total number of reads per sample. If I only input a subset of all the transcripts, the sequencing depth will change.&lt;/p&gt;&#xA;&lt;p&gt;So my question is: can I simply input the subset of interest? Or should I input all transcripts, and subset the results later?&lt;/p&gt;&#xA;" OwnerUserId="6477" LastActivityDate="2021-02-16T18:58:17.633" Title="Differential expression analysis for a subset of transcripts" Tags="|deseq2|normalization|subset|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15383" PostTypeId="2" ParentId="15372" CreationDate="2021-02-16T16:16:30.590" Score="1" Body="&lt;p&gt;I think what makes this especially hard is the way your data is formatted.&#xA;The problem would be way easier if you had a matrix of presence / absence instead. Each row would be a gene, each column a sample, and the values would be 0 (absent) or 1 (present).&lt;/p&gt;&#xA;&lt;p&gt;Awk becomes a bit inconvenient for this task, so I'll use python instead. To get such a matrix from your file, you could do the following:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np&#xA;import pandas as pd&#xA;&#xA;# Get total number of genes / samples&#xA;all_samples = set()&#xA;all_genes = set()&#xA;with open('genes.txt', 'r') as genes:&#xA;  for line in genes:&#xA;    line = line.split('\t')&#xA;    all_genes.add(line[0])&#xA;    all_samples.update(line[2:])&#xA;all_samples = list(all_samples)&#xA;all_genes = list(all_genes)&#xA;&#xA;# Initialize presence matrix with correct dimensions&#xA;pres_mat = np.zeros((len(all_genes), len(all_samples)), dtype=int)&#xA;&#xA;# Map each sample/gene to a row/col&#xA;row_map = {gene: i for i, gene in enumerate(all_genes)}&#xA;col_map = {sample: i for i, sample in enumerate(all_samples)}&#xA;&#xA;# Fill matrix incrementally&#xA;with open('genes.txt', 'r') as genes:&#xA;  for line in genes:&#xA;    line = line.split('\t')&#xA;    curr_gene = line[0]&#xA;    row = row_map[curr_gene]&#xA;    for sample in line[2:]:&#xA;      col = col_map[sample]&#xA;      pres_mat[row, col] = 1&#xA;&#xA;# Add names and write to file&#xA;df = pd.DataFrame(pres_mat, columns=all_samples)&#xA;df.insert(0, 'gene', all_genes)&#xA;df.to_csv('gene_presence.tsv, sep='\t')&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You should get something that looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;       gene  IZ_BH_5.5  A8.S8  IZ_BH_2.1  A9.S9*  IZ_BH_5.3  A9.S9  ...  IZ_BH_3.11  IZ_BH_4.7  IZ_BH_4.9  IZ_BH_2.8  IZ_BH_3.5  IZ_BH_2.6  IZ_BH_4.3&#xA;0       Lyn          1      1          1       1          1      1  ...           1          1          0          0          1          1          1&#xA;1      Nfib          1      1          1       1          1      1  ...           1          1          1          1          1          1          0&#xA;2  Cdk5rap2          1      1          1       1          1      1  ...           1          0          1          1          1          1          1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With this format, it is now easy to compute the most frequent samples. E.g. &lt;code&gt;df.sum(axis=0)&lt;/code&gt; to get the number of occurence of each sample for these 3 genes. It would also allow you to do different things, such as computing the pairwise similarity between samples.&lt;/p&gt;&#xA;" OwnerUserId="439" LastEditorUserId="439" LastEditDate="2021-02-16T16:22:06.593" LastActivityDate="2021-02-16T16:22:06.593" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15384" PostTypeId="2" ParentId="15382" CreationDate="2021-02-16T18:58:17.633" Score="3" Body="&lt;p&gt;It's safer to import everything.  You'll want all the data for normalization and dispersion estimates.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2021-02-16T18:58:17.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15385" PostTypeId="1" CreationDate="2021-02-16T19:40:45.173" Score="1" ViewCount="927" Body="&lt;p&gt;I have a big VCF file that I need to convert to, preferably, bed/bim/fam files that are readable by plink. Currently using plink 2. I am aware that this version of plink can be used to convert VCFs into binary peds using&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;plink --vcf file.vcf.gz --make-bed --out out&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Or something similar. I also know that in this case, plink will automatically fill the phenotype and sex columns in the bed and fam files with zeros. I know there is a &lt;code&gt;--pheno&lt;/code&gt; flag that allows specifying a txt file for the phenotypes and a &lt;code&gt;--make-pheno&lt;/code&gt; flag that can be used for the same effect.&lt;/p&gt;&#xA;&lt;p&gt;How exactly does this work? How can I encode the phenotype and sex information correctly? Can I just use --make-bed and then manually make a phenotype/sex file to use down the line with the --pheno flag? Or would I do something like&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;plink --vcf file --pheno phenotype_file --make_bed --out out_file&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;directly? In this case, will there not be a problem with the fact that the generated .bed in the first step does not have this information? What about with sex?&lt;/p&gt;&#xA;&lt;p&gt;Another option I can think of is just edit the generated .fam to include the correct information and use it downstream, but that information will still not be in the .bed file.&lt;/p&gt;&#xA;&lt;p&gt;I have no family relationships, so the FID fields would just be 0s (I guess I can add the --const-fid flag for that effect when I'm converting, right?).&lt;/p&gt;&#xA;" OwnerUserId="10506" LastActivityDate="2021-07-18T01:02:10.277" Title="Making PLINK compatible files from VCF file without phenotype information" Tags="|vcf|gwas|plink|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15386" PostTypeId="1" CreationDate="2021-02-17T07:37:46.283" Score="0" ViewCount="991" Body="&lt;p&gt;I am trying to extract all chimeric and multi-map reads from either SAM/BAM file. Is there any simple command to do that? Can I use &lt;code&gt;htslib&lt;/code&gt; for parsing sam/bam files and extracting based on any flags. Any help on how to start will be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2594" LastActivityDate="2021-02-17T10:45:38.887" Title="extract chimeric and multimap reads from bam file" Tags="|sam|htslib|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15387" PostTypeId="2" ParentId="15386" CreationDate="2021-02-17T10:45:38.887" Score="1" Body="&lt;p&gt;This is how I do it.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -@5 -f 0x800 -hb /path/sample.bam &amp;gt; sup.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then if you want it as a fasta&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -F 0x1 -hb sup.bam | samtools fasta -F 0x1 - &amp;gt; sup.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-02-17T10:45:38.887" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15388" PostTypeId="1" AcceptedAnswerId="15392" CreationDate="2021-02-17T13:16:39.243" Score="1" ViewCount="35" Body="&lt;br&gt;&#xA;I'm a master's student working on genomic prediction of complex traits using deep learning.&#xA;i'm looking for a dataset of human genotypes and phenotypes that has been used for a GWAS. &#xA;The only thing i can get from the gwas i found is the summary statistics not the raw data. &lt;br&gt;&#xA;Does anyone know if it's possible to get such data (for free ) for a research project? &#xA;&lt;br&gt;&#xA;Thanks.&#xA;" OwnerUserId="10510" LastActivityDate="2021-02-17T19:58:09.810" Title="how to get access to genotypes and phenotypes used for a GWAS" Tags="|genomics|gwas|deep-learning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15389" PostTypeId="1" CreationDate="2021-02-17T14:41:35.657" Score="1" ViewCount="92" Body="&lt;p&gt;For a study I need some kind of &amp;quot;decoy&amp;quot; GWAS-data. For this I created a panel with some genes, the same ones as in the real data. From NCBI I fetch some of their SNPs and create my own artificial .ped and .map file. Furthermore I save for each snp if it lies on the X-Chromosome.&lt;/p&gt;&#xA;&lt;p&gt;I then go on and randomly create female individuals by giving them heterozygous alleles on their X-Chromosome genes. Males only get homozygous ones.&lt;/p&gt;&#xA;&lt;p&gt;But when I then run pLink with the &lt;code&gt;--check-sex&lt;/code&gt; argument I get a sexcheck file where there's only zeroes in the &lt;code&gt;SNPSEX&lt;/code&gt; column. So do you know how &lt;code&gt;--check-sex&lt;/code&gt; works that I can then create decoy-individuals that reliably (as much as possible) get detected as a given gender?&lt;/p&gt;&#xA;" OwnerUserId="9131" LastActivityDate="2021-02-17T14:41:35.657" Title="Reliably create female and male individuals in a -ped file for plink" Tags="|python|plink|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15390" PostTypeId="1" CreationDate="2021-02-17T16:23:03.310" Score="0" ViewCount="24" Body="&lt;p&gt;I have to reproduce a result for school reasons (bioinformatics exam) and I was assigned a certain article. In particular I have to reproduce what some authors did in: doi: 10.1371 / journal.pone.0217889. From what I understand, they make functional enrichments in respect of three variants of protein disorder. They consider vectors whose components are proteins (belonging to a certain type of disorder flavour) in a functional class divided by the total number of proteins in that functional variant. The problem is that if I go for an enrichment in GO terms for a certain set of proteins, the same GO terms do not necessarily come out for another set of proteins. So my question is: in relation to the paragraph 'Clustering of the variants of disorder through functional profiles' how do I construct the vectors that they then use in clustering?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;&lt;p&gt;The full quote to the article is&#xA;Deiana A, Forcelloni S, Porrello A, Giansanti A. Intrinsically disordered proteins and structured proteins with intrinsically disordered regions have different functional roles in the cell. PLoS One. 2019; 14 (8): e0217889. Published 2019 Aug 19. doi: 10.1371 / journal.pone.0217889&lt;/p&gt;&#xA;" OwnerUserId="9142" LastActivityDate="2021-02-17T16:23:03.310" Title="How to Interpret Author's Methods in this article?" Tags="|go-enrichment|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15391" PostTypeId="1" CreationDate="2021-02-17T17:39:50.540" Score="1" ViewCount="730" Body="&lt;p&gt;A colleague is analysing RNA-seq data - the study design is 2 treatments, 3 replicates, 3 tissues. In their PCA plot the samples clustered neatly by tissue. Except for two samples - two tissue samples originating from the same animal clustered with the wrong tissue.&lt;/p&gt;&#xA;&lt;p&gt;I suggested that the two samples had likely been switched accidently somewhere along the way. They think its explained simply by biological variation which I'm not convinced of. Are there any technical explanations for why samples could cluster on a PCA in such a way?&lt;/p&gt;&#xA;" OwnerUserId="10511" LastActivityDate="2022-12-02T00:07:35.883" Title="Explanation for RNA-seq samples not clustering in PCA as expected" Tags="|rna-seq|pca|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15392" PostTypeId="2" ParentId="15388" CreationDate="2021-02-17T19:21:06.097" Score="0" Body="&lt;p&gt;How about:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;dbGaP - &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gap/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/gap/&lt;/a&gt; - in case you are searching for supporting datasets I leave its entry in &lt;a href=&quot;http://biokeanos.com/source/Database%20of%20Genotypes%20and%20Phenotypes&quot; rel=&quot;nofollow noreferrer&quot;&gt;BiŌkeanós&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.ddbj.nig.ac.jp/jga/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ddbj.nig.ac.jp/jga/&lt;/a&gt; - the access might be a bit more complicated.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.internationalgenome.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.internationalgenome.org&lt;/a&gt;, not particularly large, in their FAQ they state that the individuals sequenced have not self-reported any diseases - perhaps you might want to add those with to your negative set.  Again, its entry in &lt;a href=&quot;https://biokeanos.com/source/The%20International%20Genome%20Sample%20Resource&quot; rel=&quot;nofollow noreferrer&quot;&gt;BiŌkeanós&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If you are looking for more comprehensive information, you could try biobanks - e.g. &lt;a href=&quot;https://www.ukbiobank.ac.uk&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ukbiobank.ac.uk&lt;/a&gt;. These are usually open (or reasonably priced) for research but the approval process can take time.&lt;/p&gt;&#xA;&lt;p&gt;A side note - some of GWAS catalogues will have a link to the study where the association originated from, &lt;a href=&quot;https://www.ebi.ac.uk/gwas/studies/GCST005312&quot; rel=&quot;nofollow noreferrer&quot;&gt;e.g. here&lt;/a&gt;. You can sometimes find the datasource there.&lt;/p&gt;&#xA;" OwnerUserId="4628" LastEditorUserId="4628" LastEditDate="2021-02-17T19:58:09.810" LastActivityDate="2021-02-17T19:58:09.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15393" PostTypeId="1" CreationDate="2021-02-17T19:26:25.723" Score="2" ViewCount="3370" Body="&lt;p&gt;First of all, sorry if this question has been posted previously (I could not found a solution accessing the previous Q&amp;amp;A).&lt;/p&gt;&#xA;&lt;p&gt;I have a fasta file as follow:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;LNIV02000036.519060.520603 Pseudomonas aeruginosa&#xA;GAACTGAAGAGTTTGATCATGGCTCAGATTGAACGCTGGCGGCAGGCCTAA...&#xA;&#xA;&amp;gt;LDLH01000045.361.1876 Pseudomonas aeruginosa&#xA;AGAGTTTGATCATGGCTCAGATTGAACGCTGGCGGCAGGCCTAACACATG...&#xA;&#xA;&amp;gt;LNMQ01000087.5825.7394 Enterococcus faecium&#xA;TTTTTATGAGAGTTTGATCCTGGCTCAGGACGAACGCTGGCGGCGTGC...&#xA;&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to filter sequences or extract entries for a specific species, let's say Pseudomonas (I have others in mind...). I tried a few things but I think the main problem is my skills associated with bioinformatics. I would like to create a file content Pseudomonas entries (~9k entries in the current database). Any suggestion or tip?&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your time.&lt;/p&gt;&#xA;&lt;p&gt;Best,&lt;br /&gt;&#xA;Fabricio&lt;/p&gt;&#xA;" OwnerUserId="10513" LastEditorUserId="650" LastEditDate="2021-02-17T20:58:29.363" LastActivityDate="2021-02-22T16:34:06.940" Title="Filtering Sequences (entries) by headers ID from a Fasta file database" Tags="|fasta|database|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15394" PostTypeId="2" ParentId="15393" CreationDate="2021-02-17T19:35:43.563" Score="2" Body="&lt;p&gt;Using Biopython:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;for record in SeqIO.parse(&amp;quot;input.fa&amp;quot;, &amp;quot;fasta&amp;quot;):&#xA;    if record.description.split()[-2:] == [&amp;quot;Pseudomonas&amp;quot;, &amp;quot;aeruginosa&amp;quot;]:&#xA;        print(record.format(&amp;quot;fasta&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastActivityDate="2021-02-17T19:35:43.563" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15395" PostTypeId="2" ParentId="15312" CreationDate="2021-02-17T21:52:05.920" Score="0" Body="&lt;p&gt;Directly piping will definitely not work, as Liam McIntyre suggests. However, if you don't feel up to learning a workflow language just to do this, it is perfectly possible to just put the various commands in a bash driver script (or whatever you're comfortable with).&lt;/p&gt;&#xA;&lt;p&gt;Each of those tools will take parameters to define output filenames, or have systematic ways of writing FASTA files and defining inputs. You will have to look at the command line interface of each of those tools to figure it out (for example, sratools &lt;a href=&quot;https://github.com/ncbi/sra-tools/wiki/08.-prefetch-and-fasterq-dump&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, bbduk.sh &lt;a href=&quot;https://jgi.doe.gov/data-and-tools/bbtools/bb-tools-user-guide/bbduk-guide/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and spades &lt;a href=&quot;http://cab.spbu.ru/files/release3.12.0/manual.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). an example would be as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/bash&#xA;SRA=$1&#xA;&#xA;# get reads&#xA;prefetch &lt;span class=&quot;math-container&quot;&gt;$SRA  # writes to a known path locally&#xA;fasterq-dump $&lt;/span&gt;SRA --split-3  # looks for path_to_sra_file and outputs it. assumes you are splitting.&#xA;&#xA;# trim reads&#xA;bbduk.sh in1=&lt;span class=&quot;math-container&quot;&gt;$SRA''_1.fastq in2=$&lt;/span&gt;SRA''_2.fastq out1=&lt;span class=&quot;math-container&quot;&gt;$SRA''_clean_1.fq out2=$&lt;/span&gt;SRA''_clean_2.fq&#xA;&#xA;# re-QC the files- this step does not feed into the other steps!!&#xA;fastqc &lt;span class=&quot;math-container&quot;&gt;$SRA''*.fastq $&lt;/span&gt;SRA''*.fq  # runs fastqc on all fastq files&#xA;&#xA;# run assembly&#xA;spades.py -o my_assembly --pe1 &lt;span class=&quot;math-container&quot;&gt;$SRA''_clean_1.fq --pe2 $&lt;/span&gt;SRA''_clean_2.fq&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and so on.&lt;/p&gt;&#xA;&lt;p&gt;I suppose that from a formal perspective it's better to use a workflow engine as Liam McIntyre says, but with such a straightforward workflow I'd say start simple and get more elaborate only if you need to; you'd spend 10x as long just figuring out the workflow language. This is ok if it's a goal to learn it or if you already know it, but if you just want some results putting it into a script is fine.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-02-17T21:52:05.920" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15396" PostTypeId="2" ParentId="15391" CreationDate="2021-02-18T00:07:54.070" Score="1" Body="&lt;p&gt;It's hard to say what is going on without looking at the plots. But it is a reasonable guess that samples are mislabelled and I think it is worth checking the experimental log carefully. It is also possible that the two samples are of low quality. People often use PCA to check and remove outliers from their RNA-seq libraries. So you may also check the RNA-seq library quality for those two samples.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2021-02-18T00:07:54.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15397" PostTypeId="2" ParentId="15385" CreationDate="2021-02-18T00:29:05.007" Score="2" Body="&lt;p&gt;You can use &lt;code&gt;--pheno&lt;/code&gt; to specify a phenotype file you want to use, the 1st and 2nd columns being FID and IID. These are just used on the fly and cannot be written to the BED files as I understand. &lt;code&gt;--make-pheno&lt;/code&gt; is used to write binary phenotypes, which is unsuitable if your phenotype is actually continuous.&#xA;But I still do not quite understand what you want to do here.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2021-02-18T00:29:05.007" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15398" PostTypeId="2" ParentId="15393" CreationDate="2021-02-18T02:01:19.337" Score="4" Body="&lt;p&gt;You can do this using &lt;a href=&quot;https://bioinf.shenwei.me/seqkit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;seqkit&lt;/code&gt;&lt;/a&gt; as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqkit grep -r -n -p '.*Pseudomonas.*' temp.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To explain a little, &lt;code&gt;seqkit grep&lt;/code&gt; will allow you to search FASTA/Q files by sequence name or sequence itself. In this instance:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;-r&lt;/code&gt; tells that the pattern is a regular expression&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;-n&lt;/code&gt; to match by full name instead of just id&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;-p&lt;/code&gt; to specify the regular expression pattern to search&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;There are a bunch of other tools that come with &lt;code&gt;seqkit&lt;/code&gt; that can prove indispensable for basic sequence analyses.&lt;/p&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2021-02-18T02:01:19.337" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="15399" PostTypeId="2" ParentId="15381" CreationDate="2021-02-18T07:21:30.483" Score="-1" Body="&lt;p&gt;Maybe can try the ComBat to remove the batch effect&lt;/p&gt;&#xA;" OwnerUserId="10514" LastActivityDate="2021-02-18T07:21:30.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15400" PostTypeId="1" CreationDate="2021-02-18T08:41:41.147" Score="1" ViewCount="105" Body="&lt;p&gt;I'm trying to do some calculus in parallel with BAM files. For that, I need to split the input bam file in chunks, process them in parallel and join them at the end. But I'm facing a read duplication issue on the split points.&lt;/p&gt;&#xA;&lt;p&gt;I'm trying to split the input files by equally sized intervals (computed by GATK SplitIntervals), but that means the one of the files will contain all reads that covers the last base of the interval and the next file will contain all the reads that covers the following position. This means that we are putting in both files all the reads that crosses the boundary.&lt;/p&gt;&#xA;&lt;p&gt;When we have to merge the results back into one file (via merge, sort or GatherBamFiles), all this repeated reads go into the result file, givin us a bam file with some extra reads that happen to have the same ID as others (because, in fact, they are the same).&lt;/p&gt;&#xA;&lt;p&gt;Which would be the way to go to accomplish this task? How could I gather the files without duplicates? Or how could I split the files without putting the same read in more than one file?&lt;/p&gt;&#xA;&lt;p&gt;For reference, these are the commands I'm trying to check that I'm duplicating reads. I extract a one base interval from a file, the next base interval from the same file and finally counting the number of reads in the files:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ samtools view -c test.bam 1:18454-18454  # Reads covering position 18454&#xA;251&#xA;$&lt;/span&gt; samtools view -c test.bam 1:18455-18455  # Reads covering position 18455&#xA;250&#xA;$ samtools view -c test.bam 1:18454-18455  # Reads covering 18454-18455 range&#xA;251&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;$ samtools view -b -o test.1_18454_18454.bam test.bam 1:18454-18454&#xA;$&lt;/span&gt; samtools view -b -o test.1_18455_18455.bam test.bam 1:18455-18455&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;$ samtools view -c test.1_18454_18454.bam  # Reads extracted that covers position 18454&#xA;251&#xA;$&lt;/span&gt; samtools view -c test.1_18455_18455.bam  # Reads extracted that covers position 18455&#xA;250&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;$ samtools merge test.1_18454_18455.merged.bam test.1_18454_18454.bam test.1_18455_18455.bam&#xA;$&lt;/span&gt; samtools view -c test.1_18454_18455.merged.bam  # Reads merged that covers 18454-18455 range&#xA;501&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3037" LastActivityDate="2021-02-18T08:41:41.147" Title="Proper scatter/gather of bam files" Tags="|sam|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15403" PostTypeId="1" AcceptedAnswerId="15407" CreationDate="2021-02-19T06:33:09.983" Score="0" ViewCount="268" Body="&lt;p&gt;We had same set of sample ran on nanopore platform twice.&lt;/p&gt;&#xA;&lt;p&gt;The issue on hand is.&lt;/p&gt;&#xA;&lt;p&gt;First run was done it was nearly done but the power went off.&#xA;Second run was done completely with enough sequencing depth.&lt;/p&gt;&#xA;&lt;p&gt;Now the question I ran the analysis with the second set which was complete that set of result I have. But so far I have't ran the first part.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question1&lt;/strong&gt; : if I run the first part do i get major differences? in the result&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question2&lt;/strong&gt; : would it be advisable to merge both run since they are now can be called as twi different batches plus would be bring drastic changes in the output ? And in terms of tool for merging I googled which shows poretools.&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2021-02-19T18:56:32.430" Title="Nanopore data clarification regarding merging of samples" Tags="|nanopore|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15405" PostTypeId="1" CreationDate="2021-02-19T10:57:42.070" Score="2" ViewCount="196" Body="&lt;p&gt;If 2 protein sequence X of length m and Y of length n and if there are several highest scoring global alignments, then I want to get one alignment that has largest number of columns in which a letter from X is aligned to an identical letter from Y. I need an algorithm to find alignment with maximum number of identical matches. Can anyone help me?&lt;/p&gt;&#xA;" OwnerUserId="10523" LastActivityDate="2021-06-13T14:00:15.030" Title="Global alignment between two sequence X and Y with maximum number of identical matches" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15406" PostTypeId="1" AcceptedAnswerId="18066" CreationDate="2021-02-19T16:55:33.483" Score="0" ViewCount="665" Body="&lt;p&gt;Is there a functionality within STRINGdb package or in another environment to reverse map the STRING identifiers to Gene names?&lt;/p&gt;&#xA;&lt;p&gt;For example, the entire STRING network can be downloaded using the commands:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(STRINGdb)&#xA;string_db &amp;lt;- STRINGdb&lt;span class=&quot;math-container&quot;&gt;$new(version=&quot;11&quot;, species=9606,&#xA;                          score_threshold=0, input_directory=&quot;&quot; )&#xA;full.graph &amp;lt;- string_db$&lt;/span&gt;load_all()&#xA;netFull&#xA;# IGRAPH c07d6e9 UN-- 19354 11759454 -- &#xA;# + attr: name (v/c), combined_score (e/n)&#xA;# + edges from c07d6e9 (vertex names):&#xA;#  [1] 9606.ENSP00000000233--9606.ENSP00000272298 9606.ENSP00000000233--9606.ENSP00000253401&#xA;#  [3] 9606.ENSP00000000233--9606.ENSP00000401445 9606.ENSP00000000233--9606.ENSP00000418915&#xA;#  [5] 9606.ENSP00000000233--9606.ENSP00000327801 9606.ENSP00000000233--9606.ENSP00000466298&#xA;#  [7] 9606.ENSP00000000233--9606.ENSP00000232564 9606.ENSP00000000233--9606.ENSP00000393379&#xA;#  [9] 9606.ENSP00000000233--9606.ENSP00000371253 9606.ENSP00000000233--9606.ENSP00000373713&#xA;# [11] 9606.ENSP00000000233--9606.ENSP00000248929 9606.ENSP00000000233--9606.ENSP00000356737&#xA;# [13] 9606.ENSP00000000233--9606.ENSP00000338766 9606.ENSP00000000233--9606.ENSP00000386135&#xA;# [15] 9606.ENSP00000000233--9606.ENSP00000319192 9606.ENSP00000000233--9606.ENSP00000432568&#xA;# + ... omitted several edges&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;V(netFull)$name&lt;/code&gt; will provide all the STRING identifiers within this network &lt;code&gt;netFull&lt;/code&gt;. How to get the corresponding gene names?&lt;/p&gt;&#xA;&lt;p&gt;For example, &lt;code&gt;string_db$map()&lt;/code&gt; maps the gene identifiers of the input dataframe to STRING identifiers. It returns the input dataframe with the &amp;quot;STRING_id&amp;quot; as an additional column. Now, I would like to explore in the reverse direction going from STRING identifiers to gene names.&lt;/p&gt;&#xA;" OwnerUserId="23" LastActivityDate="2021-11-20T06:31:38.230" Title="STRINGdb: reverse map STRING identifiers to gene names?" Tags="|r|bioconductor|networks|stringdb|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15407" PostTypeId="2" ParentId="15403" CreationDate="2021-02-19T18:56:32.430" Score="1" Body="&lt;p&gt;It is entirely unclear which type of analysis you aim to do. You probably should be more specific when asking questions.&lt;/p&gt;&#xA;&lt;p&gt;That said, given that the run was going perfectly fine until a power issue stopped it, all data produced should be fine and you can indeed combine those runs.&lt;/p&gt;&#xA;&lt;p&gt;You probably shouldn't bother with poretools. If you are using fastq files to start your analysis you can just cat the basecalled files together. Also after alignment, you can merge the bam files. If your analysis requires the fast5 files then you can probably just put these in the same folder, but you should tell us which analysis you are doing.&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2021-02-19T18:56:32.430" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15408" PostTypeId="1" CreationDate="2021-02-20T02:35:05.563" Score="0" ViewCount="221" Body="&lt;p&gt;My data is proportional data, but variables are correlated with each other. So I am wondering what kinds of distance or dissimilarity we should use to take correlation into account.&lt;/p&gt;&#xA;&lt;p&gt;If we would like to use data with proportion (e.g.,total sum scaling or compositional data), what kinds of distance or dissimilarity can we use?&#xA;Bray-Curtis is one of them, but does this make sense to use arcisin or logit transformation and use Pearson correlation?&lt;/p&gt;&#xA;&lt;p&gt;In continuous variables, we can use Pearson and Spearman distance. In our case, what kinds of distance can we use? Pearson distance cannot be applied to data with 0-1 scale. Logit transformation -&amp;gt; Pearson?&lt;/p&gt;&#xA;" OwnerUserId="8895" LastEditorUserId="73" LastEditDate="2021-05-09T06:28:51.103" LastActivityDate="2021-05-09T06:28:51.103" Title="About optimal distance (or dissimilarity) measure in proportion data" Tags="|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15410" PostTypeId="2" ParentId="15406" CreationDate="2021-02-21T17:10:16.560" Score="2" Body="&lt;p&gt;This answer does not rely on STRINGdb package.&lt;/p&gt;&#xA;&lt;p&gt;STRING provides supplementary identifier mapping files here: &lt;a href=&quot;https://string-db.org/mapping_files/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://string-db.org/mapping_files/&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;It looks like you are interested in human STRING name one - &lt;a href=&quot;https://string-db.org/mapping_files/STRING_display_names/human.name_2_string.tsv.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://string-db.org/mapping_files/STRING_display_names/human.name_2_string.tsv.gz&lt;/a&gt;. You could read it as a separate lookup.&lt;/p&gt;&#xA;" OwnerUserId="4628" LastActivityDate="2021-02-21T17:10:16.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15412" PostTypeId="1" CreationDate="2021-02-22T14:17:17.740" Score="1" ViewCount="1524" Body="&lt;p&gt;I am trying to update the &lt;code&gt;guppy_basecaller&lt;/code&gt; (current version installed is 3.2.10+aabd4ec)  on a ONT Gridion device. I am initially trying to update it via the nanopore PPA, as described &lt;a href=&quot;https://community.nanoporetech.com/protocols/Guppy-protocol/v/gpb_2003_v1_revu_14dec2018/linux-guppy&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (requires login access to the nanopore community)&lt;/p&gt;&#xA;&lt;p&gt;After I issued &lt;code&gt;sudo apt update&lt;/code&gt;, I got a GPG error, saying that the public key was not found in the system.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://europe.oxfordnanoportal.com/apt xenial-stable InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY AD9D09AF2EBBA5A8&#xA;W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://mirror.oxfordnanoportal.com/apt xenial-stable-gridion InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY AD9D09AF2EBBA5A8&#xA;W: Failed to fetch https://mirror.oxfordnanoportal.com/apt/dists/xenial-stable-gridion/InRelease  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY AD9D09AF2EBBA5A8&#xA;W: Failed to fetch http://mirror.oxfordnanoportal.com/apt/dists/xenial-stable/InRelease  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY AD9D09AF2EBBA5A8&#xA;W: Some index files failed to download. They have been ignored, or old ones used instead&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I came across a possible solution shown &lt;a href=&quot;https://itsfoss.com/solve-gpg-error-signatures-verified-ubuntu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, so I tried to add the key to the system with the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys AD9D09AF2EBBA5A8&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The terminal was busy for too long and, after some time I got an timeout error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Executing: /tmp/tmp.EWnOg4cPJk/gpg.1.sh --keyserver&#xA;keyserver.ubuntu.com&#xA;--recv-keys&#xA;AD9D09AF2EBBA5A8&#xA;gpg: requesting key 2EBBA5A8 from hkp server keyserver.ubuntu.com&#xA;gpg: keyserver timed out&#xA;gpg: keyserver receive failed: keyserver error&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then came across &lt;a href=&quot;https://unix.stackexchange.com/questions/75892/keyserver-timed-out-when-trying-to-add-a-gpg-public-key&quot;&gt;this post&lt;/a&gt; where one potential solution was to change the port, so I issued:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys AD9D09AF2EBBA5A8&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But then I got:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Executing: /tmp/tmp.sfqFuJpVb2/gpg.1.sh --keyserver&#xA;hkp://keyserver.ubuntu.com:80&#xA;--recv-keys&#xA;AD9D09AF2EBBA5A8&#xA;gpg: requesting key 2EBBA5A8 from hkp server keyserver.ubuntu.com&#xA;gpgkeys: key AD9D09AF2EBBA5A8 not found on keyserver&#xA;gpg: no valid OpenPGP data found.&#xA;gpg: Total number processed: 0&#xA;gpg: keyserver communications error: key not found&#xA;gpg: keyserver communications error: bad public key&#xA;gpg: keyserver receive failed: bad public key&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Has anyone managed to update the &lt;code&gt;guppy_basecaller&lt;/code&gt; via PPA, as described in the nanopore community page?&lt;/p&gt;&#xA;&lt;p&gt;An alternative would be to install via the &lt;code&gt;.tar.gz&lt;/code&gt; file, though I am first trying to upgrade the current version from the PPA to avoid conflicts with the current installation.&lt;/p&gt;&#xA;" OwnerUserId="3452" LastActivityDate="2021-02-25T23:04:16.457" Title="Upgrading guppy basecaller on Oxford Nanopore Gridion via PPA" Tags="|nanopore|long-reads|base-calling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15413" PostTypeId="1" CreationDate="2021-02-22T15:51:29.790" Score="0" ViewCount="195" Body="&lt;p&gt;I have &lt;code&gt;.vcf&lt;/code&gt; file with partly decoded genome provided by some service. How can I convert this file into other popular formats which may be used in other services? I've tried &lt;a href=&quot;https://github.com/chrchang/plink-ng&quot; rel=&quot;nofollow noreferrer&quot;&gt;plink2&lt;/a&gt; but unfortunately, rather I just can't use it correctly.&lt;/p&gt;&#xA;&lt;p&gt;Also, where could I get a most completed list of a free DNA services?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Here's part of my DNA raw data with file headers:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.2&#xA;##source=Genotek&#xA;##reference=hg19&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&amp;quot;Genotype&amp;quot;&amp;gt;&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  vh6913&#xA;chr1    752721  rs3131972   A   G   .   .   .   GT  1/1&#xA;chr1    759036  rs114525117 G   A   .   .   .   GT  0/0&#xA;chr1    801536  rs79373928  T   G   .   .   .   GT  0/0&#xA;chr1    834830  rs116452738 G   A   .   .   .   GT  0/0&#xA;chr1    835092  rs72631887  T   G   .   .   .   GT  0/0&#xA;chr1    838555  rs4970383   C   A   .   .   .   GT  0/1&#xA;chr1    838665  rs28678693  T   C   .   .   .   GT  0/0&#xA;chr1    840753  rs4970382   T   C   .   .   .   GT  1/1&#xA;chr1    846808  rs4475691   C   T   .   .   .   GT  0/1&#xA;chr1    851390  rs72631889  G   T   .   .   .   GT  0/0&#xA;chr1    854250  rs7537756   A   G   .   .   .   GT  0/1&#xA;chr1    861808  rs13302982  A   G   .   .   .   GT  1/1&#xA;chr1    863130  rs376747791 A   G   .   .   .   GT  0/0&#xA;chr1    866893  rs2880024   T   C   .   .   .   GT  0/0&#xA;chr1    868404  rs13302914  C   T   .   .   .   GT  1/1&#xA;chr1    872952  rs76723341  C   T   .   .   .   GT  0/0&#xA;chr1    878331  rs148327885 C   T   .   .   .   GT  0/1&#xA;chr1    879911  rs143853699 G   A   .   .   .   GT  0/0&#xA;chr1    881627  rs2272757   G   A   .   .   .   GT  0/0&#xA;chr1    884767  rs67274836  G   A   .   .   .   GT  0/0&#xA;chr1    888659  rs3748597   T   C   .   .   .   GT  1/1&#xA;chr1    889238  rs3828049   G   A   .   .   .   GT  0/0&#xA;chr1    891277  rs77608078  C   T   .   .   .   GT  0/0&#xA;... and more than 636 thousand rows&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user10558" LastEditorDisplayName="user10558" LastEditDate="2021-03-01T13:42:46.313" LastActivityDate="2021-03-01T13:42:46.313" Title="Convert DNA raw data between a different formats" Tags="|gene|vcf|phylogenetics|format-conversion|" AnswerCount="0" CommentCount="13" ClosedDate="2021-02-28T14:22:29.993" ContentLicense="CC BY-SA 4.0" />
  <row Id="15414" PostTypeId="2" ParentId="15393" CreationDate="2021-02-22T16:34:06.940" Score="0" Body="&lt;p&gt;&amp;quot;describing the entire issue&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I have a database (silva 16S formatted by DADA2 - fasta format) that contains ~300k entries for bacteria (16S rRNA gene). e.g.,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;AB001441.1.1538 Pseudomonas syringae pv. broussonetiae&#xA;AACTGAAGAGTTTGATCATGGCTCAGATTGAACGCTGGCGGCAGG...&#xA;KR054096.1.1224 Marinococcus halophilus&#xA;GAAGGTAAAGGCTTCCCCAAGGGCGACGATGGGTAGCCGACTTGA...&#xA;AB681405.1.1478 Paenibacillus barengoltzii&#xA;GACGAACGCTGGCGGCGTGCCTAATACATGCAAGTCGAGCGGAGT...&#xA;...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I would like to retrieve sequences for some genus, e.g., Pseudomonas. After running the seqkit command (mentioned by you above) I ended up with ~9 of Pseudomonas entries in a fasta file (example below).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;AB001441.1.1538 Pseudomonas syringae pv. broussonetiae&#xA;AACTGAAGAGTTTGATCATGGCTCAGATTGAACGCTGGCGGCAGGCCTAACA...&#xA;AB001442.1.1538 Pseudomonas amygdali pv. eriobotryae&#xA;AACTGAAGAGTTTGATCATGGCTCAGATTGAACGCTGGCGGCAGGCCTAACA...&#xA;...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This pool of 9k sequences of Pseudomonas has several species of Pseudomonas (e.g., P. putida, P. syringe Et Cetera). Since some species of Pseudomonas have multiple sequences in the database (e.g., P. putida has 1.8k entries) I would like to know how many different species of Pseudomonas are there (in the 9k Pseudomonas entries), by names (i.e., in the filtered Psedumonas fasta file). At the end of the day, I would like to say that in this database I have 9k entries of Pseudomonas which represents 'X' number of Pseudomonas species.&lt;/p&gt;&#xA;&lt;p&gt;What I'm thinking... Retrieve the full ID (AB001442.1.1538 Pseudomonas amygdali pv. eriobotryaeall) or if possible the portion of the ID (headers) that contains the species names (e.g. Psudomonas amygdali pv. eriobotryaeall) and with this, I can list the species names and count them. Does this explanation help to understand what I'm trying to do?&lt;/p&gt;&#xA;&lt;p&gt;Let me know if you need more info.&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your time.&#xA;Fabricio&lt;/p&gt;&#xA;" OwnerUserId="10513" LastActivityDate="2021-02-22T16:34:06.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15415" PostTypeId="1" CreationDate="2021-02-22T17:48:33.337" Score="1" ViewCount="148" Body="&lt;p&gt;I read about the median string problem as an introduction to the multiple sequence alignment, however none of the MSA algorithms used seems to be using the idea of finding the median string.&lt;/p&gt;&#xA;&lt;p&gt;To my understanding from the consensus (median) string, it would be possible to retrieve the actual alignment by simply doing the pairwise alignment of the consensus string with each of the strings.&lt;/p&gt;&#xA;&lt;p&gt;Why is this approach not used? Are there any algorithms that use it?&lt;/p&gt;&#xA;&lt;p&gt;And related to this, what kind of tasks in bioinformatics/biology are median/consensus string algorithms used for?&lt;/p&gt;&#xA;" OwnerUserId="9815" LastEditorUserId="9815" LastEditDate="2021-02-25T18:37:24.740" LastActivityDate="2021-04-21T19:29:19.160" Title="Median string problem &amp; multiple sequence alignment" Tags="|sequence-alignment|multiple-sequence-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15416" PostTypeId="1" AcceptedAnswerId="15417" CreationDate="2021-02-23T02:54:32.627" Score="1" ViewCount="43" Body="&lt;p&gt;I am currently working on gene clustering based on co-expression pattern in mouse brain. The problem is I do not have some solid way to test my result. Are there any suggestions for databases indicating groups of genes that co-express (in mouse brain), or even better indicating some similarity metrics between each pair of genes based on expression pattern?&lt;/p&gt;&#xA;&lt;p&gt;Also, I do want some databases for differentially-expressed genes ;)&lt;/p&gt;&#xA;&lt;p&gt;PS: By &amp;quot;differentially-expressed&amp;quot; I am referring to a pair of genes whose expression patterns in a single individual differ.&lt;/p&gt;&#xA;" OwnerUserId="10560" LastActivityDate="2021-02-23T09:50:06.163" Title="Are there any databases for gene co-expression or expression pattern clustering?" Tags="|database|public-databases|differential-expression|gene-expression|coexpression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15417" PostTypeId="2" ParentId="15416" CreationDate="2021-02-23T09:50:06.163" Score="1" Body="&lt;p&gt;How about:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.informatics.jax.org/expression.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.informatics.jax.org/expression.shtml&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.emouseatlas.org/emage/home.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.emouseatlas.org/emage/home.php&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Other potential databases of interest:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://biokeanos.com/search?q=co-expression+%2Bmouse&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biokeanos.com/search?q=co-expression+%2Bmouse&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4628" LastActivityDate="2021-02-23T09:50:06.163" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15419" PostTypeId="1" CreationDate="2021-02-23T13:25:40.060" Score="0" ViewCount="236" Body="&lt;p&gt;I have a vcf file created with GATK with spanning deletion &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035531912-Spanning-or-overlapping-deletions-allele-&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360035531912-Spanning-or-overlapping-deletions-allele-&lt;/a&gt; and I now need to compare this VCF with its gold standard using hap.py (&lt;a href=&quot;https://github.com/Illumina/hap.py/blob/master/doc/happy.md#getting-started&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Illumina/hap.py/blob/master/doc/happy.md#getting-started&lt;/a&gt;) but the spanning allele gives me the following error:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;[W] too many AD fields at chr10:133017183 max_ad = 2 retrieved: 3&lt;/code&gt; and I've read that is due to * in the ALT column.&lt;/p&gt;&#xA;&lt;p&gt;So, is there a way to convert the spanning alleles into simple vcf files?&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot,&#xA;Denise&lt;/p&gt;&#xA;" OwnerUserId="8933" LastEditorUserId="298" LastEditDate="2021-02-27T18:38:13.610" LastActivityDate="2021-02-27T18:38:13.610" Title="Spanning deletion conversion" Tags="|vcf|indel|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15420" PostTypeId="2" ParentId="15312" CreationDate="2021-02-24T08:55:08.740" Score="1" Body="&lt;p&gt;Maybe take a look at workflow management systems, like&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;snakemake&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://nextflow.io/example1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nextflow&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://toil.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Toil&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://cromwell.readthedocs.io/en/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cromwell&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Admitted, each of those adds their additional learning curve. However, especially snakemake is not so much more complicated than a standard bash workflow. Those systems do help a lot organize and orchestrate your workflows&lt;/p&gt;&#xA;" OwnerUserId="3252" LastActivityDate="2021-02-24T08:55:08.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15421" PostTypeId="1" AcceptedAnswerId="15453" CreationDate="2021-02-24T16:20:09.067" Score="3" ViewCount="282" Body="&lt;p&gt;I have a pyranges object that I am looking to iterate over. For each range, I would like to &amp;quot;resize&amp;quot; it to its centre (so &lt;code&gt;start+end/2&lt;/code&gt;) and save that into a separate pyranges object. There will be additional processing associated based on what ranges already exist nearby, so I can't just use a &lt;code&gt;resize&lt;/code&gt; function, I have to iterate over the object.&lt;/p&gt;&#xA;&lt;p&gt;First, how would I go about manually creating a new pyranges object, then inserting one by one new ranges into it?&lt;/p&gt;&#xA;&lt;p&gt;Second, when I try to iterate over my current pyranges object (using this line &lt;code&gt;for myspot in pr.itergrs(hotspots, strand=False) :&lt;/code&gt;), it gives me this error: &lt;code&gt;AttributeError: 'tuple' object has no attribute 'stranded'&lt;/code&gt;. This error occurs whether or not I have strands in my pyranges, and whether or not I specify &lt;code&gt;strand=False&lt;/code&gt;. Why does this occur?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The data I am working with: a pyranges covering hotspots in the human genome (all chr1 for the moment)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;+--------------+-----------+-----------+-----------+&#xA;| Chromosome   | Start     | End       | Score     |&#xA;| (category)   | (int32)   | (int32)   | (int64)   |&#xA;|--------------+-----------+-----------+-----------|&#xA;| chr1         | 154421820 | 154421821 | 26413     |&#xA;| chr1         | 30749469  | 30749470  | 14833     |&#xA;| chr1         | 55259503  | 55259504  | 9900      |&#xA;| chr1         | 14985097  | 14985098  | 7735      |&#xA;| ...          | ...       | ...       | ...       |&#xA;| chr1         | 247975751 | 247976497 | 1         |&#xA;| chr1         | 247977599 | 247981391 | 1         |&#xA;| chr1         | 248018012 | 248019454 | 1         |&#xA;| chr1         | 248177298 | 248178490 | 1         |&#xA;+--------------+-----------+-----------+-----------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The hotspots are sorted by their score, and I'd like to go through and for each one, resize it to its centre, keeping the score, but if there is already a hotspot within a certain range (say +/-5kb), to discard it instead, so that I end up with clear chunks between hotspots. This is the &amp;quot;pseudocode&amp;quot; I was planning to do to implement this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;add centre of hotspot 1 to a new pyranges object&#xA;create ranges with score = -1 for 5kb flanking each side&#xA;for myhotspot in hotspots:&#xA;  resize to centre&#xA;  if not overlap with new pyranges object&#xA;    add centre&#xA;    create flanking regions&#xA;discard ranges with score&amp;lt;0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This may not be the most elegant solution, but I'll never have more than ~2000 ranges to process so it should work fine. Is this implementable with pyranges?&lt;/p&gt;&#xA;" OwnerUserId="6316" LastEditorUserId="6316" LastEditDate="2021-02-25T14:36:50.937" LastActivityDate="2021-03-01T19:40:42.117" Title="Create new range based from existing range (python)" Tags="|python|pyranges|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15422" PostTypeId="2" ParentId="15421" CreationDate="2021-02-24T17:41:08.803" Score="4" Body="&lt;p&gt;Answer to main question:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pyranges as pr&#xA;assert int(pr.__version__.split(&amp;quot;.&amp;quot;)[2]) &amp;gt;= 93, &amp;quot;pip install pyranges==0.0.93&amp;quot;&#xA;import numpy as np&#xA;np.random.seed(42 * 10)&#xA;&#xA;# create large df to test on&#xA;gr = pr.random(int(1e5), length=1, chromsizes={&amp;quot;chr1&amp;quot;: 249250621})&#xA;gr.Score = np.random.randint(250, size=len(gr))&#xA;&#xA;result = gr.max_disjoint(slack=5000)&#xA;&#xA;# small example that is easy to validate&#xA;gr2 = pr.from_string(&amp;quot;&amp;quot;&amp;quot;Chromosome Start End Score&#xA;chr1 1 2 5&#xA;chr1 4999 5000 10&#xA;chr1 10001 10002 1&#xA;chr1 15000 15001 2&#xA;chr1 20100 20101 5&#xA;&amp;quot;&amp;quot;&amp;quot;)&#xA;&#xA;gr2.max_disjoint(slack=5000)&#xA;&#xA;# +--------------+-----------+-----------+-----------+&#xA;# | Chromosome   |     Start |       End |     Score |&#xA;# | (category)   |   (int32) |   (int32) |   (int64) |&#xA;# |--------------+-----------+-----------+-----------|&#xA;# | chr1         |         1 |         2 |         5 |&#xA;# | chr1         |     10001 |     10002 |         1 |&#xA;# | chr1         |     20100 |     20101 |         5 |&#xA;# +--------------+-----------+-----------+-----------+&#xA;# Unstranded PyRanges object has 3 rows and 4 columns from 1 chromosomes.&#xA;# For printing, the PyRanges was sorted on Chromosome.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Could you please post an example of what data you have and what you want to end up with?&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Inserting row after row is going to be slow, whether it is pandas or pyranges. It is best to collect all the data and then create a new dataframe or PyRanges.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;pr.itergrs&lt;/code&gt; is for iterating over multiple pyranges at the same time. It takes a list of pyranges while you seem to have a collection of tuples. The function iterates over every chromosome/strand pair in the list and for it extracts the data from every pyranges. See the code example at the end.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;To iterate over a pyranges you can do &lt;code&gt;for k, df in gr: # ...&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Anyways, to do what you want:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pyranges as pr&#xA;import numpy as np&#xA;&#xA;f1 = pr.data.f1()&#xA;f1.Start = ((f1.Start + f1.End)/2).astype(np.int32) # int is 64 bit&#xA;f1.End = f1.Start + 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;I understand I have not done the addititonal processing you speak of, but if you can post an example I'm sure I'd be able to help you :)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Code example &lt;code&gt;pr.itergrs&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pyranges as pr&#xA;f1 = pr.data.f1()&#xA;f2 = pr.data.f2()&#xA;for k, dfs in pr.itergrs([f1, f2], keys=True):&#xA;    print(k)&#xA;     for df in dfs:&#xA;         print(&amp;quot;df:&amp;quot;)&#xA;         print(df)&#xA;&#xA;('chr1', '+')&#xA;df:&#xA;  Chromosome  Start  End       Name  Score Strand&#xA;0       chr1      3    6  interval1      0      +&#xA;2       chr1      8    9  interval3      0      +&#xA;df:&#xA;  Chromosome  Start  End Name  Score Strand&#xA;0       chr1      1    2    a      0      +&#xA;('chr1', '-')&#xA;df:&#xA;  Chromosome  Start  End       Name  Score Strand&#xA;1       chr1      5    7  interval2      0      -&#xA;df:&#xA;  Chromosome  Start  End Name  Score Strand&#xA;1       chr1      6    7    b      0      -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2688" LastEditorUserId="2688" LastEditDate="2021-02-25T20:52:27.593" LastActivityDate="2021-02-25T20:52:27.593" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="15423" PostTypeId="2" ParentId="15370" CreationDate="2021-02-24T20:43:36.393" Score="3" Body="&lt;pre&gt;&lt;code&gt;# pip install pyranges or conda install -c bioconda pyranges&#xA;&#xA;import pyranges as pr&#xA;&#xA;g1 = pr.from_string(&amp;quot;&amp;quot;&amp;quot;Chromosome Start End&#xA;chr1   222 223&#xA;chr1   223 224&#xA;chr1   233 234&#xA;chr1   235 236&#xA;chr1   2237 238&#xA;chr1  2123 2124&#xA;chr2   244 245&amp;quot;&amp;quot;&amp;quot;)&#xA;&#xA;g2 = pr.from_string(&amp;quot;&amp;quot;&amp;quot;Chromosome Start End&#xA;chr1    221  223&#xA;chr1    230  240&#xA;chr2    0 1000&amp;quot;&amp;quot;&amp;quot;)&#xA;&#xA;r = g2.count_overlaps(g1)&#xA;r&#xA;# +--------------+-----------+-----------+------------------+&#xA;# | Chromosome   |     Start |       End |   NumberOverlaps |&#xA;# | (category)   |   (int32) |   (int32) |          (int64) |&#xA;# |--------------+-----------+-----------+------------------|&#xA;# | chr1         |       221 |       223 |                1 |&#xA;# | chr1         |       230 |       240 |                2 |&#xA;# | chr2         |         0 |      1000 |                1 |&#xA;# +--------------+-----------+-----------+------------------+&#xA;# Unstranded PyRanges object has 3 rows and 4 columns from 2 chromosomes.&#xA;# For printing, the PyRanges was sorted on Chromosome.&#xA;&#xA;r.df&#xA;# 0       chr1    221   223               1&#xA;# 1       chr1    230   240               2&#xA;# 2       chr2      0  1000               1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2688" LastActivityDate="2021-02-24T20:43:36.393" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15424" PostTypeId="1" CreationDate="2021-02-24T21:08:58.020" Score="0" ViewCount="29" Body="&lt;p&gt;For chip-seq data, I had analyzed the samples until motif discovery using both cluster profiler and Homer for each histone mark. I was wondering what is the efficient method to determine if each GO term or identified motifs has a biological meaning or not or if there is any way to filter out the genes/motifs for only specific cell types such as PBMCs, how to prioritize genes? I know random matches might even surpass the biologically relevant matches but right now I am not sure how to narrow down the motifs/genes? Your help would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="8543" LastActivityDate="2021-02-24T21:08:58.020" Title="How to identify specific motif for chip-seq data" Tags="|chip-seq|gene-ontology|motifs|homer|clusterprofiler|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15425" PostTypeId="2" ParentId="15321" CreationDate="2021-02-25T04:52:30.033" Score="0" Body="&lt;p&gt;This can be done in python easily. If you have different text files then you can load them into a python list and follow from step 2&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Step 1&#xA;list1 = [&amp;quot;m64041_200717_231916/100664277/0_42364 id=2440&amp;quot;,&#xA;&amp;quot;m64041_200717_231916/100008029/0_42112 id=244006&amp;quot;,&#xA;&amp;quot;m64041_200717_231916/100665802/0_46854 id=2440067&amp;quot;,&#xA;&amp;quot;m64041_200717_231916/100008044/0_60161 id=24400&amp;quot;,&#xA;&amp;quot;m64041_200717_231916/100928073/0_11804 id=2440152&amp;quot;,&#xA;&amp;quot;m64041_200717_231916/10094492/0_44157 id=2440199&amp;quot;,&#xA;&amp;quot;m64041_200717_231916/100008373/4102_50601 id=2440234&amp;quot;]&#xA;&#xA;list2 = [&amp;quot;m64041_200717_231916/100664277/0_42364&amp;quot;,&#xA;&amp;quot;m64041_200717_231916/100665802/0_46854&amp;quot;,&#xA;&amp;quot;m64041_200717_231916/100928073/0_11804&amp;quot;,&#xA;&amp;quot;m64041_200717_231916/10094492/0_44157&amp;quot;]&#xA;&#xA;# Step 2&#xA;split_list1 = [ele.split(&amp;quot; &amp;quot;)[0] for ele in list1 ]&#xA;mapdict = {split_list1[i]:list1[i] for i in range(len(list1))}&#xA;common = [value for value in split_list1 if value in list2] &#xA;required = [mapdict[x] for x in common]&#xA;print(required)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10470" LastActivityDate="2021-02-25T04:52:30.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15429" PostTypeId="2" ParentId="15412" CreationDate="2021-02-25T11:49:10.370" Score="1" Body="&lt;p&gt;This is an ONT problem, not an ubuntu problem. Keys can expire over time, and need to be updated.&lt;/p&gt;&#xA;&lt;p&gt;The keys for the Guppy basecaller are not present in the ubuntu repository; you need to get them from ONT. This is indicated by the &lt;a href=&quot;https://community.nanoporetech.com/protocols/Guppy-protocol/v/gpb_2003_v1_revu_14dec2018/linux-guppy&quot; rel=&quot;nofollow noreferrer&quot;&gt;installation instructions&lt;/a&gt; for Guppy (which you've already linked to):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sudo apt-get update&#xA;sudo apt-get install wget lsb-release&#xA;export PLATFORM=&lt;span class=&quot;math-container&quot;&gt;$(lsb_release -cs)&#xA;###&#xA;wget -O- https://mirror.oxfordnanoportal.com/apt/ont-repo.pub | sudo apt-key add -&#xA;###&#xA;echo &quot;deb http://mirror.oxfordnanoportal.com/apt $&lt;/span&gt;{PLATFORM}-stable non-free&amp;quot; | sudo tee /etc/apt/sources.list.d/nanoporetech.sources.list&#xA;sudo apt-get update&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-02-25T23:04:16.457" LastActivityDate="2021-02-25T23:04:16.457" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15430" PostTypeId="1" CreationDate="2021-02-25T11:52:36.600" Score="1" ViewCount="249" Body="&lt;p&gt;If you have worked on cluster with submitting jobs, please help me in resolving the following error.&lt;/p&gt;&#xA;&lt;p&gt;I am running rsem pipeline to generate star index. In principal STAR requires more than 30G of RAM for human genome.&lt;/p&gt;&#xA;&lt;p&gt;I have following script&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;#SBATCH --mail-user=&amp;lt;your_email_address&amp;gt;&#xA;#SBATCH --mail-type=end,fail&#xA;#SBATCH --job-name=&amp;quot;job01&amp;quot;&#xA;#SBATCH --mem-per-cpu=10G&#xA;&#xA;rsem-prepare-reference --gtf ~/Genomes/Homo_sapiens.GRCh38.100.chr.gtf -p 5 --star --star-path ~/Tools/STAR/bin/Linux_x86_64/ ~/Genomes/Homo_sapiens.GRCh38.dna.toplevel.fa ~/Genomes/GRCh38_100&#xA; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The process fails as soon as it starts to generate genome files using star. Below is the error complaining about the memory.&lt;/p&gt;&#xA;&lt;p&gt;I have put 5 threads for star usage and 10G for each cpu.&lt;/p&gt;&#xA;&lt;p&gt;here is the error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rsem-preref /storage/homefs/as21u994/Genomes/GRCh38_100.transcripts.fa 1 /storage/homefs/as21u994/Genomes/GRCh38_100&#xA;Refs.makeRefs finished!&#xA;Refs.saveRefs finished!&#xA;/storage/homefs/as21u994/Genomes/GRCh38_100.idx.fa is generated!&#xA;/storage/homefs/as21u994/Genomes/GRCh38_100.n2g.idx.fa is generated!&#xA;&#xA;/storage/homefs/as21u994/Tools/STAR/bin/Linux_x86_64//STAR  --runThreadN 8  --runMode genomeGenerate  --genomeDir /storage/homefs/as21u994/Genomes  --genomeFastaFiles /storage/homefs/as21u994/Genomes/Homo_sapiens.GRCh38.dna.toplevel.fa  --sjdbGTFfile /storage/homefs/as21u994/Genomes/Homo_sapiens.GRCh38.100.chr.gtf  --sjdbOverhang 100  --outFileNamePrefix /storage/homefs/as21u994/Genomes/GRCh38_100&#xA;Feb 25 12:10:58 ..... started STAR run&#xA;Feb 25 12:10:58 ... starting to generate Genome files&#xA;&#xA;EXITING because of FATAL PARAMETER ERROR: limitGenomeGenerateRAM=31000000000is too small for your genome&#xA;SOLUTION: please specify --limitGenomeGenerateRAM not less than 168632691637 and make that much RAM available&#xA;&#xA;Feb 25 12:18:11 ...... FATAL ERROR, exiting&#xA;&amp;quot;/storage/homefs/as21u994/Tools/STAR/bin/Linux_x86_64//STAR  --runThreadN 8  --runMode genomeGenerate  --genomeDir /storage/homefs/as21u994/Genomes  --genomeFastaFiles /storage/homefs/as21u994/Genomes/Homo_sapiens.GRCh38.dna.toplevel.fa  --sjdbGTFfile /storage/homefs/as21u994/Genomes/Homo_sapiens.GRCh38.100.chr.gtf  --sjdbOverhang 100  --outFileNamePrefix /storage/homefs/as21u994/Genomes/GRCh38_100&amp;quot; failed! Plase check if you provide correct parameters/options for the pipeline!&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Could anyone of you tell me how to fix this.&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="5062" LastEditorUserId="5062" LastEditDate="2021-02-25T14:22:11.710" LastActivityDate="2021-02-25T14:22:11.710" Title="SLURM script for running RSEM star fails" Tags="|star|rsem|slurm|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15431" PostTypeId="1" CreationDate="2021-02-25T17:08:15.483" Score="2" ViewCount="393" Body="&lt;p&gt;I am using Snakemake from inside a Singularity container. Because of that, the directory that Snakemake wants to log to cannot be created, and I get an &lt;code&gt;OSError [Errno 30] Read-only file system: '.snakemake&lt;/code&gt;, more specifically the traceback&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &amp;quot;/usr/local/bin/snakemake&amp;quot;, line 8, in &amp;lt;module&amp;gt;&#xA;    sys.exit(main())&#xA;  File &amp;quot;/usr/local/lib/python3.8/dist-packages/snakemake/__init__.py&amp;quot;, line 2562, in main&#xA;    success = snakemake(&#xA;  File &amp;quot;/usr/local/lib/python3.8/dist-packages/snakemake/__init__.py&amp;quot;, line 525, in snakemake&#xA;    logger.setup_logfile()&#xA;  File &amp;quot;/usr/local/lib/python3.8/dist-packages/snakemake/logging.py&amp;quot;, line 137, in setup_logfile&#xA;    os.makedirs(os.path.join(&amp;quot;.snakemake&amp;quot;, &amp;quot;log&amp;quot;), exist_ok=True)&#xA;  File &amp;quot;/usr/lib/python3.8/os.py&amp;quot;, line 213, in makedirs&#xA;    makedirs(head, exist_ok=exist_ok)&#xA;  File &amp;quot;/usr/lib/python3.8/os.py&amp;quot;, line 223, in makedirs&#xA;    mkdir(name, mode)&#xA;OSError: [Errno 30] Read-only file system: '.snakemake'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The Snakefile I am using is inside a Singularity-container (there's no way around that, I am on a hpc and don't have root, also using a &lt;code&gt;--sandbox&lt;/code&gt;ed container doesn't work), and I call it like this (I've also played around with binding other directories without success):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;singularity run --bind $(realpath .) --bind ~/container.sif ~/container.sif test_simple --cores 4 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The test_simple rule is a really simple one that only unzips a gzip, for testing - the problem however lies generally in Snakemake wanting to create the log at &lt;code&gt;&amp;lt;snakemake_base&amp;gt;/.snakemake/log&lt;/code&gt; (see &lt;a href=&quot;https://github.com/snakemake/snakemake/blob/bd5a9239b2684b57ab0143bb709e77afa1493cdc/snakemake/logging.py#L308&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/snakemake/snakemake/blob/bd5a9239b2684b57ab0143bb709e77afa1493cdc/snakemake/logging.py#L308&lt;/a&gt;), which in my case is inside the container.&lt;/p&gt;&#xA;&lt;p&gt;It doesn't help to use an overlay &lt;code&gt;--overlay overlay.img&lt;/code&gt; for some reason, and I've also tried using my own (empty) log-handle (&lt;code&gt;--log-handler-script ~/loghandler.py&lt;/code&gt;) without any success.&lt;/p&gt;&#xA;&lt;p&gt;Any ideas on how to circumvent this or disable the logging or write it to somewhere else?&lt;/p&gt;&#xA;" OwnerUserId="10588" LastEditorUserId="3061" LastEditDate="2021-03-31T15:44:19.813" LastActivityDate="2021-03-31T15:44:19.813" Title="Snakemake fails because it cannot create logs inside a read-only Singularity Container" Tags="|python|snakemake|hpc|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15432" PostTypeId="1" AcceptedAnswerId="15433" CreationDate="2021-02-25T18:45:35.000" Score="0" ViewCount="34" Body="&lt;p&gt;As a way to exemplify how the hierarchical clustering of particular DNA sequences can be used to derive phylogenetic trees, I'm looking for a (even very small) dataset of the same genes (or other biological sequences) from different organisms (for example animals where the evolutionary tree is known). Do you have any pointers?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="9815" LastActivityDate="2021-02-25T19:53:05.400" Title="Dataset showing evolution of gene" Tags="|gene|phylogeny|phylogenetics|clustering|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15433" PostTypeId="2" ParentId="15432" CreationDate="2021-02-25T19:53:05.400" Score="0" Body="&lt;p&gt;If you're building a tree of different species, it's probably better to use protein sequences. A common resource is &lt;a href=&quot;https://www.uniprot.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Uniprot&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you still want DNA sequences you can turn to &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Nucleotide&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Some examples that give decent trees: p53, TOP1, HBA.&lt;/p&gt;&#xA;&lt;p&gt;A common way to construct a phylogenetic tree is to start with a &lt;a href=&quot;https://en.wikipedia.org/wiki/Multiple_sequence_alignment&quot; rel=&quot;nofollow noreferrer&quot;&gt;multiple sequence alignment (MSA)&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="10591" LastActivityDate="2021-02-25T19:53:05.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15434" PostTypeId="1" AcceptedAnswerId="15443" CreationDate="2021-02-25T20:33:39.410" Score="1" ViewCount="157" Body="&lt;p&gt;This paper &lt;a href=&quot;https://www.nature.com/articles/s41467-018-03856-y/figures/2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fig2b&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;b Changes of chromatin accessibility in AMD (n = 14) relative to&#xA;normal (n = 11) in all retina samples. Each dot represents one&#xA;ATAC-Seq peak. Blue line in the left panel indicates average fold&#xA;changes of peaks with the same ATAC-Seq intensity. &lt;strong&gt;The percentage of&#xA;reduced peaks is shown under the density curve in the right panel.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How is the density calculated or what is being shown there? average ATAC seq signal or Fold change?&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2021-02-28T14:44:10.033" Title="ATAC seq density calculation" Tags="|atac-seq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15435" PostTypeId="2" ParentId="15434" CreationDate="2021-02-26T03:01:30.727" Score="0" Body="&lt;p&gt;Just count the number of reads overlapping an ATAC-seq peak and divide it by the length of the peak gives you the read density. Each dot is a fold change, but to calculate fold change, average density from all the samples are taken, that is what the &amp;quot;average&amp;quot; is referring to.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2021-02-26T03:01:30.727" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15436" PostTypeId="2" ParentId="15120" CreationDate="2021-02-26T13:41:27.953" Score="1" Body="&lt;p&gt;If processing a huge table with lots of &amp;quot;false positives&amp;quot; is OK, then you could enter your search term, from top right choose &amp;quot;Send to&amp;quot; &amp;gt; &amp;quot;File&amp;quot; Format: &amp;quot;RunInfo&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="10591" LastActivityDate="2021-02-26T13:41:27.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15439" PostTypeId="1" AcceptedAnswerId="15440" CreationDate="2021-02-27T00:05:56.997" Score="1" ViewCount="116" Body="&lt;p&gt;I have two full genome assemblies for C. Elegans samples collected from two different geographical areas that I found on WormBase. These are in fasta format. I want to go gene-by-gene and compare the nucleotide sequences corresponding to the gene between the two samples. My goal with this is to count how many single-base differences there are in each gene. However, I'm not sure how to match the sequences in the assembly to known reference sequences (by the way is there a good resource to download all of these at once?) for C. Elegans genes.&lt;/p&gt;&#xA;&lt;p&gt;My thought was to run BLAST between the reference sequence and each genome and find the match with the best score, if any. Is this a reasonable approach, or is there some better way to do this?&lt;/p&gt;&#xA;" OwnerUserId="10598" LastEditorUserId="10598" LastEditDate="2021-02-27T00:16:38.803" LastActivityDate="2021-03-02T09:19:00.487" Title="How to identify genes from a genome assembly of C. Elegans?" Tags="|sequence-alignment|gene|sequence-annotation|blast|public-databases|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15440" PostTypeId="2" ParentId="15439" CreationDate="2021-02-27T18:28:12.490" Score="2" Body="&lt;p&gt;You don't want BLAST here, or at least not regular BLASTn. A better method would be to use tBLASTn to map the known &lt;em&gt;C. Elegans&lt;/em&gt; proteins to the translated genome. However, I would recommend using a slightly more sophisticated approach:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Collect fasta sequences of all known &lt;em&gt;C. Elegans&lt;/em&gt; proteins (amino acid sequences, not nucleotide). This should be easy enough to do on &lt;a href=&quot;https://wormbase.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;WormBase&lt;/a&gt;. For example, the protein sequences for &lt;em&gt;C. elegans&lt;/em&gt; strain VC2010, assembly &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/bioproject/PRJEB28388&quot; rel=&quot;nofollow noreferrer&quot;&gt;PRJEB28388&lt;/a&gt;, can be found here: &lt;a href=&quot;ftp://ftp.wormbase.org/pub/wormbase/releases/current-production-release/species/c_elegans/PRJEB28388/c_elegans.PRJEB28388.WS279.protein.fa.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.wormbase.org/pub/wormbase/releases/current-production-release/species/c_elegans/PRJEB28388/c_elegans.PRJEB28388.WS279.protein.fa.gz&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use a tool that can model introns and exons to map a protein into a genome. I haven't worked in this field in more than 10 years now, so it is very likely that there are other programs around today, but back in the day, I would do this using either &lt;a href=&quot;https://www.ebi.ac.uk/about/vertebrate-genomics/software/exonerate-user-guide&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;exonerate&lt;/code&gt;&lt;/a&gt; or &lt;a href=&quot;https://www.ebi.ac.uk/Tools/psa/genewise/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;genewise&lt;/code&gt;&lt;/a&gt;. With &lt;code&gt;exonerate&lt;/code&gt;, the command would be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;exonerate -m protein2genome -n 1 -t assembly.fasta -q allproteins.pep &amp;gt; out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;That will give you a list of best-hits for each of your input proteins and should accurately map the protein to the genome. Once you have that, you can extract the matches and start comparing.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2021-03-02T09:19:00.487" LastActivityDate="2021-03-02T09:19:00.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15441" PostTypeId="1" AcceptedAnswerId="15442" CreationDate="2021-02-28T00:53:25.170" Score="0" ViewCount="84" Body="&lt;p&gt;I'm trying to align the Acinetobacter baumanii genome to a genome. I've already done the alignment, and I want to use bedtools to see which genes are being expressed exactly. When I try running the following command,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bedtools coverage -a outCSEQSAligned.out.bam -b GCF_000746645.1_ASM74664v1_genomic.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;unfortunately, it doesn't tell me which genes are being expressed at each place but instead tells me that the sequence aligns to the single Acinetobacter chromosome which isn't too helpful:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;NZ_CP009257.1   282134  282254  GWNJ-1012:340:GW200702637th:1:1105:15519:10019/1        255     +       282134  282254  0,0,0   1       120,    0,4       120     120     1.0000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm assuming that 282134-282254 are the positions on the chromosome that correspond to that alignment. I suppose I could extract these positions and figure out which genes they correspond to, but I'm wondering if there's an easier way to use bedtools to figure out which genes are being expressed. For reference, outCSEQSAligned.out.bam is my bamfile which I aligned the Acinetobacter genome to, and GCF_000746645.1_ASM74664v1_genomic.bed is a bedfile corresponding to the Acinetobacter genome.&lt;/p&gt;&#xA;&lt;p&gt;Thanks for the help!&lt;/p&gt;&#xA;" OwnerUserId="10604" LastActivityDate="2021-02-28T01:51:52.457" Title="How can I get bedtools to tell me which genes are being expressed?" Tags="|sequence-alignment|gene-expression|bedtools|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15442" PostTypeId="2" ParentId="15441" CreationDate="2021-02-28T01:45:31.637" Score="0" Body="&lt;p&gt;&lt;code&gt;Bedtools&lt;/code&gt; is not the best tool for this. You can use &lt;code&gt;featureCounts&lt;/code&gt;, which gives you the number of read that overlap with features in a genome. In this case, your features are genes.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;bedtools coverage&lt;/code&gt; is used to calculate the overlap between each read in BAM and each line in BED. It is not very straightforward to quantify gene expression. Plus bedtools does not account for exon in each gene and summarize them to gene level, like &lt;code&gt;featureCounts&lt;/code&gt; does.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastEditorUserId="4766" LastEditDate="2021-02-28T01:51:52.457" LastActivityDate="2021-02-28T01:51:52.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15443" PostTypeId="2" ParentId="15434" CreationDate="2021-02-28T14:44:10.033" Score="1" Body="&lt;p&gt;They are not referring to density of reads. It is the density of the fold change, the y variable in the plot to left of Fig2b. It tells you the distribution of the foldchange and 92% of the fold change is &amp;lt;0.&lt;/p&gt;&#xA;&lt;p&gt;See &lt;a href=&quot;https://www.statmethods.net/graphs/density.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page for more details on density and histograms&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Suppose your data is something like this (purely making this up):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;set.seed(555)&#xA;fc = rnorm(1000,-0.5,2)&#xA;averATAC = -0.5*(fc - 0.5)^2 + 0.2*fc + 3  + rnorm(100,25,10)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You fit a density to it and you get the plot:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;par(mfrow=c(1,2))&#xA;plot(averATAC,fc)&#xA;plot(dens&lt;span class=&quot;math-container&quot;&gt;$y,dens$&lt;/span&gt;x,xlab = &amp;quot;density&amp;quot;,ylab=&amp;quot;fc&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VYK9s.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VYK9s.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2021-02-28T14:44:10.033" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15444" PostTypeId="2" ParentId="14217" CreationDate="2021-02-28T19:55:44.923" Score="0" Body="&lt;p&gt;In pyranges you would not need a call like the one you show. You can just convert a dataframe with the columns you mention above to a pyranges like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pyranges as pr&#xA;gr = pr.PyRanges(your_df)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This avoids the code duplication shown in your example. The column names in the df are preserved in the PyRanges.&lt;/p&gt;&#xA;" OwnerUserId="2688" LastActivityDate="2021-02-28T19:55:44.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15445" PostTypeId="2" ParentId="13233" CreationDate="2021-02-28T20:03:42.263" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/biocore-ntnu/pyranges&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyranges&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pyranges as pr&#xA;gr = pr.read_bam(&amp;quot;your_file.bam&amp;quot;)&#xA;gr.to_bigwig(&amp;quot;out.bw&amp;quot;, chromosome_sizes=pr.data.chromsizes()) # for hg19&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2688" LastActivityDate="2021-02-28T20:03:42.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15446" PostTypeId="2" ParentId="5652" CreationDate="2021-02-28T23:39:02.237" Score="1" Body="&lt;p&gt;This is a bit late. However, I am hoping someone else will find this useful since I had the same problem.&lt;/p&gt;&#xA;&lt;p&gt;The data matrix is your infile.nex file (the nex file you supplied for the analysis)&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Bracket out the mcmc command in your infile.nex&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Prevents the file from being run again.&lt;/li&gt;&#xA;&lt;li&gt;May not be present.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;In Mr. Bayes: Execute infile.nex&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Loads you infile as the data matrix&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;In Mr. Bayes: sumt Filename=infile.nex nruns=2&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you have nruns set in you Mr. Bayes block you can skip this.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="10608" LastActivityDate="2021-02-28T23:39:02.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15447" PostTypeId="1" CreationDate="2021-03-01T08:40:57.890" Score="2" ViewCount="203" Body="&lt;p&gt;I have fasta files with multiple sequences. They are reads that mapped to the genome outside of  probe-targeted regions.&#xA;From a quick perusal, they appear to be repetitive and have low complexity. Is there a way I can quantify their complexity and use that as an explanation as to why those regions were mapped to?&lt;/p&gt;&#xA;" OwnerUserId="10553" LastEditorUserId="10553" LastEditDate="2021-03-01T09:12:21.190" LastActivityDate="2021-03-03T21:22:44.453" Title="Quantify low complexity regions in DNA sequences" Tags="|ngs|read-mapping|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15448" PostTypeId="2" ParentId="15447" CreationDate="2021-03-01T16:43:30.820" Score="1" Body="&lt;p&gt;&amp;quot;Complexity&amp;quot; is also known as entropy. A quick and dirty way is to use a file compression utility - zip will get good compression for a repetitive sequence.&lt;/p&gt;&#xA;&lt;p&gt;You can also look at the k-mer distribution for some suitable k: in a repetitive sequence it will be biased. Here is a tool that give a local plot: &lt;a href=&quot;https://rdrr.io/github/vsbuffalo/qrqc/man/kmerEntropyPlot.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rdrr.io/github/vsbuffalo/qrqc/man/kmerEntropyPlot.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9972" LastActivityDate="2021-03-01T16:43:30.820" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15449" PostTypeId="1" CreationDate="2021-03-01T17:28:32.767" Score="1" ViewCount="287" Body="&lt;p&gt;Are there any good options for calling variants or even just a pileup from R? R is not ideal for this, but I'd like to integrate with other functions.&lt;/p&gt;&#xA;&lt;p&gt;I found &lt;a href=&quot;https://www.rdocumentation.org/packages/deepSNV/versions/1.18.3/topics/bam2R&quot; rel=&quot;nofollow noreferrer&quot;&gt;deepSNV::bam2R&lt;/a&gt; which roughly does what I am looking for. Are there any alternatives?&lt;/p&gt;&#xA;" OwnerUserId="35" LastActivityDate="2021-03-31T20:03:35.317" Title="genotyping or variant calling in R" Tags="|r|sam|variant-calling|genotyping|variants|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15450" PostTypeId="2" ParentId="15302" CreationDate="2021-03-01T18:08:33.317" Score="2" Body="&lt;h2&gt;Problem statement&lt;/h2&gt;&#xA;&lt;p&gt;You want to find a set of SNPs in populations of two species, for each of which you have a distinct reference genome. You then want to know which of those SNPs are &lt;em&gt;shared&lt;/em&gt; between the two genomes (there is variation in each species), and which ones are &lt;em&gt;private&lt;/em&gt; to only one of the species (in the other genome there is no variant at that location).&lt;/p&gt;&#xA;&lt;p&gt;To accomplish this task, you need to first find SNP variation in each genome, and then figure out which SNPs are the same between the two genomes. In other words, line up two VCF files for different genomes and figure out which SNPs are the same between them. In comparing genome assemblies, this is sometimes called a &lt;em&gt;liftover&lt;/em&gt; operation.&lt;/p&gt;&#xA;&lt;h2&gt;Genomes can have different coordinate systems.&lt;/h2&gt;&#xA;&lt;p&gt;Let's first think about what exactly you are looking at with 2 different reference genomes. For example, here are outputs of &lt;code&gt;samtools faidx&lt;/code&gt; for 2 different S. cerevisiae strains, S288C and Kyokai-7:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# S288C&#xA;ref|NC_001133|  230203  95  60  61&#xA;CHR2.19980913   813142  234173  60  61&#xA;CHR3.19980521   315339  1060907 60  61&#xA;CHR4.19990210   1531929 1381540 60  61&#xA;CHR5.19970727   576870  2939039 60  61&#xA;CHR6.19970727   270148  3525562 60  61&#xA;CHR7.19970703   1090936 3800252 60  61&#xA;CHR8.19970727   562638  4909411 60  61&#xA;ref|NC_001141|  439885  5481523 60  61&#xA;CHR10.19970727  745440  5928778 60  61&#xA;CHR11.19970727  666448  6686681 60  61&#xA;CHR12.19970730  1078172 7364277 60  61&#xA;ref|NC_001145|  924430  8460517 60  61&#xA;ref|NC_001146|  784328  9400452 60  61&#xA;CHR15.19970811  1091283 10197892    60  61&#xA;ref|NC_001148|  948061  11307461    60  61&#xA;ref|NC_001224|  85779   12271444    60  61&#xA;&#xA;# Kyokai first 10 lines&#xA;gi|347729946|dbj|BABQ01000001.1|    5491    137 80  81&#xA;gi|347729945|dbj|BABQ01000002.1|    4704    5834    80  81&#xA;gi|347729944|dbj|BABQ01000003.1|    203046  10734   80  81&#xA;gi|347729943|dbj|BABQ01000004.1|    1036    216456  80  81&#xA;gi|347729942|dbj|BABQ01000005.1|    126714  217642  80  81&#xA;gi|347729941|dbj|BABQ01000006.1|    121900  346077  80  81&#xA;gi|347729940|dbj|BABQ01000007.1|    20385   469638  80  81&#xA;gi|347729939|dbj|BABQ01000008.1|    243458  490415  80  81&#xA;gi|347729938|dbj|BABQ01000009.1|    189876  737054  80  81&#xA;gi|347729937|dbj|BABQ01000010.1|    81973   929441  80  81&#xA;# ...&#xA;# goes on for many thousands of lines&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What you will notice is that there are significant differences between the 2 assemblies. The first is almost all in chromosomes, the second is largely &lt;em&gt;mapped&lt;/em&gt; to various chromosomes but exists in thousands of tiny contigs. There is no way to just map reads to them, call variants, and assume that you (or bcftools) can figure out what is going on.&lt;/p&gt;&#xA;&lt;p&gt;This is possibly an extreme example, but even in the case of two genomes assembled to chromosome pseudomolecules, there is basically no chance that those two chromosomes are on the same coordinate system or have similar metadata. That means again that bcftools probably won't be able to figure it out, because a SNP at position 112929 in genome A will be homologous to position 110201 in genome B.&lt;/p&gt;&#xA;&lt;h2&gt;Liftover: a way to solve the mapping problem between 2 genomes.&lt;/h2&gt;&#xA;&lt;p&gt;You can create a mapping between the two genomes. This is 100% a solved problem in bioinformatics, and it is more or less the same as doing &lt;a href=&quot;https://genome.sph.umich.edu/wiki/LiftOver&quot; rel=&quot;nofollow noreferrer&quot;&gt;liftovers&lt;/a&gt; between different assembly versions of the same genome. I think that you can more or less just &lt;a href=&quot;http://genomewiki.ucsc.edu/index.php/LiftOver_Howto#:%7E:text=A%20liftOver%20file%20is%20a,of%20a%20genome%20to%20another.&quot; rel=&quot;nofollow noreferrer&quot;&gt;use the UCSC LiftOver tool to convert SNP coordinates&lt;/a&gt;, unless I am missing something. That how-to guide that I linked is going to give you much better instructions than I can give you, &lt;a href=&quot;http://genomewiki.ucsc.edu/index.php/Minimal_Steps_For_LiftOver&quot; rel=&quot;nofollow noreferrer&quot;&gt;here is a more minimal version&lt;/a&gt; that might be simpler.&lt;/p&gt;&#xA;&lt;p&gt;The only extra suggestion that I have is that the guide uses BLAT to align the two assemblies, and minimap2 is a &lt;em&gt;much&lt;/em&gt; faster tool. You may need to fight the input formatting a little bit, but if you run into runtime issues you might make that change. This is why I suggested it in the comment, though I wasn't aware that the liftover docs were so complete.&lt;/p&gt;&#xA;&lt;h2&gt;Short pipeline:&lt;/h2&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Align reads (individuals of species A to reference A, individuals of species B to reference B).&lt;/li&gt;&#xA;&lt;li&gt;call variants--&amp;gt;get VCF files using preferred workflow for each of A and B independently.&lt;/li&gt;&#xA;&lt;li&gt;compute liftover mapping (chain file, usually) from reference A to reference B.&lt;/li&gt;&#xA;&lt;li&gt;use computed liftover mapping to actually lift VCF over from one genome to another.&lt;/li&gt;&#xA;&lt;li&gt;align A and B references, call SNPs between them (possibly can be done from (3) output). Ensure that you are using as reference the liftover target genome so that the VCF is informative.&lt;/li&gt;&#xA;&lt;li&gt;For each SNP private to each species (output of 5), query the BAMs to ensure that no variation exists.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2021-03-01T22:42:21.120" LastActivityDate="2021-03-01T22:42:21.120" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15451" PostTypeId="2" ParentId="15447" CreationDate="2021-03-01T18:37:55.913" Score="1" Body="&lt;p&gt;Use Picard &lt;a href=&quot;http://broadinstitute.github.io/picard/command-line-overview.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;EstimateLibraryComplexity&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;From the docs:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Estimates the numbers of unique molecules in a sequencing library.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar picard.jar EstimateLibraryComplexity \&#xA;     I=input.bam \&#xA;     O=est_lib_complex_metrics.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6251" LastActivityDate="2021-03-01T18:37:55.913" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15452" PostTypeId="2" ParentId="15449" CreationDate="2021-03-01T19:25:29.497" Score="1" Body="&lt;p&gt;Samtools/bcftools/tabix is available in R via &lt;a href=&quot;https://kasperdanielhansen.github.io/genbioconductor/html/Rsamtools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rsamtools&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/omicsCore/SEQprocess/blob/master/Reference_Manual.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SEQprocess&lt;/a&gt; also provides wrappers for different tools such as GATK and bowtie.&lt;/p&gt;&#xA;&lt;p&gt;More utility packages are noted &lt;a href=&quot;https://docs.rc.fas.harvard.edu/kb/ngs-analysis-with-r/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-03-01T19:25:29.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15453" PostTypeId="2" ParentId="15421" CreationDate="2021-03-01T19:40:42.117" Score="1" Body="&lt;p&gt;Quick and dirty (not to mention atrociously bad &lt;code&gt;O(n)&lt;/code&gt;) solution:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pyranges as pr&#xA;import numpy as np&#xA;np.random.seed(42 * 10)&#xA;&#xA;# create large df to test on&#xA;gr = pr.random(int(1e5), length=10000, chromsizes={&amp;quot;chr1&amp;quot;: 249250621})&#xA;gr.Score = np.random.randint(250, size=len(gr))&#xA;&#xA;def remove_worst_scores_until_no_overlap(gr):&#xA;    df = gr.df&#xA;    old_length = -1&#xA;    new_length = len(df)&#xA;    while old_length != new_length:&#xA;        df = df.drop([&amp;quot;Cluster&amp;quot;, &amp;quot;Count&amp;quot;], axis=1, errors=&amp;quot;ignore&amp;quot;)&#xA;        df = pr.PyRanges(df).cluster(count=True).df.sort_values(&amp;quot;Score&amp;quot;, ascending=False)&#xA;        df = df[df.duplicated('Cluster', keep='last') | (df.Count == 1)]&#xA;        old_length = new_length&#xA;        new_length = len(df)&#xA;&#xA;    return pr.PyRanges(df.drop(&amp;quot;Cluster Count&amp;quot;.split(), axis=1)).sort()&#xA;&#xA;hotspots_no_overlaps = remove_worst_scores_until_no_overlap(gr)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2688" LastActivityDate="2021-03-01T19:40:42.117" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15454" PostTypeId="1" AcceptedAnswerId="15456" CreationDate="2021-03-02T07:10:00.740" Score="5" ViewCount="385" Body="&lt;p&gt;I'm very new to python but i've been using it to extract the sequence of a gene from a genbank file. The issue is is that sometimes i'll get the output i want (prints the sequence to a file) and sometimes it will return a key error. This depends on which accession i'm using. Does anyone know why it sometimes might give a key error? I thought it might be something to do with the genbank records themselves, but they look pretty similar and the gene is there (in the gene feature qualifier). EG works with HG738867.1 but not AP019703.1. Here's my code -&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez, SeqIO&#xA;&#xA;gi_genome = 'accession'&#xA;name = 'acrA'&#xA;Entrez.email = 'email'&#xA;handle = Entrez.efetch(db=&amp;quot;nucleotide&amp;quot;, id=gi_genome, rettype=&amp;quot;gbwithparts&amp;quot;, retmode=&amp;quot;text&amp;quot;)&#xA;record = SeqIO.read(handle, &amp;quot;gb&amp;quot;)&#xA;handle.close()&#xA;element = 0&#xA;for feature in record.features:&#xA;    if feature.type == 'CDS' and name in feature.qualifiers[&amp;quot;gene&amp;quot;]:&#xA;        report = 'record.features[%s]' % str(element)&#xA;        gene_sequence = feature.extract(record.seq)&#xA;        with open('output.fasta', 'a') as f:&#xA;            print('&amp;gt;' + gi_genome + ' ' + name, file=f)&#xA;            print(gene_sequence, file=f)&#xA;        break&#xA;    else:&#xA;        element = element + 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here's the traceback -&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &amp;quot;/home/ubuntu/Documents/Git_Branches/Project_planning/Learning/In_progress/utils/data.py&amp;quot;, line 11, in &amp;lt;module&amp;gt;&#xA;    if feature.type == 'CDS' and name in feature.qualifiers[&amp;quot;gene&amp;quot;]:&#xA;KeyError: 'gene'&#xA;&#xA;Process finished with exit code 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="10648" LastActivityDate="2021-03-02T08:58:06.323" Title="KeyError when getting features from a genbank file with biopython with some accessions but not others" Tags="|python|genbank|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15455" PostTypeId="1" AcceptedAnswerId="18838" CreationDate="2021-03-02T07:25:11.330" Score="2" ViewCount="177" Body="&lt;p&gt;I need help to add information in my phylogenetic tree on each branch.&lt;/p&gt;&#xA;&lt;p&gt;I have two pieces of information at my disposal:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A phylogenetic tree in newick format :&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;(mammal:0.14,turtle:0.02,(rayfinfish:0.25,(frog:0.01,salamander:0.01)50:0.12)95:0.09);&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;A matrix with the edge number and information in the Information column:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;Edge    Inf&#xA;7        0&#xA;6        0&#xA;5        2&#xA;4        1&#xA;3        2&#xA;2        2&#xA;1        2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Does someone have an idea in order to add these informations by creating a &lt;code&gt;NHX format&lt;/code&gt; in python (with &lt;strong&gt;ete3&lt;/strong&gt; for instance) or with an &lt;strong&gt;R package&lt;/strong&gt; ?&lt;/p&gt;&#xA;&lt;p&gt;Here is the tree :&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Xdpch.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Xdpch.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9104" LastEditorUserId="3848" LastEditDate="2022-03-28T17:01:32.243" LastActivityDate="2022-03-28T17:01:32.243" Title="Create a NHX phylogeny format with r or python" Tags="|r|python|phylogenetics|phylogeny|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15456" PostTypeId="2" ParentId="15454" CreationDate="2021-03-02T08:58:06.323" Score="3" Body="&lt;p&gt;Good question.&lt;/p&gt;&#xA;&lt;p&gt;feature.qualifiers is a dict, if the dict doesn't have that key it will throw a KeyError. The way your code works it expects feature.qualifiers to have 'gene' as a key every time feature.type == 'CDS'. However, it is possible to have feature.type == 'CDS' without a gene entry and in the case of AP019703.1 this comes up before you reach the gene acrA.&lt;/p&gt;&#xA;&lt;p&gt;As simeple fix is to simply use get instead:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;feature.qualifiers.get(&amp;quot;gene&amp;quot;,[])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now returns a empty list if there is not key &amp;quot;gene&amp;quot;, which also satisfies the 'name in' part of your if statement.&lt;/p&gt;&#xA;&lt;p&gt;If you get stuck on stuff like this use more print statements, break up your if statements and look at the whole file (example below) to understand the structure.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gi_genome = 'AP019703.1'&#xA;name = 'acrA'&#xA;Entrez.email = 'email'&#xA;handle = Entrez.efetch(db=&amp;quot;nucleotide&amp;quot;, id=gi_genome, rettype=&amp;quot;gbwithparts&amp;quot;, retmode=&amp;quot;text&amp;quot;)&#xA;record = SeqIO.read(handle, &amp;quot;gb&amp;quot;)&#xA;handle.close()&#xA;with open('AP019703.gb' ,'w') as fout:&#xA;    SeqIO.write(record, fout, 'genbank')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-03-02T08:58:06.323" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15457" PostTypeId="2" ParentId="13295" CreationDate="2021-03-02T10:19:54.323" Score="0" Body="&lt;p&gt;If you already have everything in a SingleCellExperiment called &lt;code&gt;sce&lt;/code&gt;, and the old code called for something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# old syntax&#xA;isSpike(sce, &amp;quot;ERCC&amp;quot;) &amp;lt;- grepl(&amp;quot;^ERCC&amp;quot;, rownames(sce))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can do the following instead:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# new syntax&#xA;is.spike &amp;lt;- grepl(&amp;quot;^ERCC&amp;quot;, rownames(sce))&#xA;sce &amp;lt;- splitAltExps(sce, ifelse(is.spike, &amp;quot;ERCC&amp;quot;, &amp;quot;gene&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will remove your ERCC counts from the regular counts assay and put them in an 'alternative experiment', calling &lt;code&gt;sce&lt;/code&gt; will print &lt;code&gt;altExpNames(1): ERCC&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="10650" LastActivityDate="2021-03-02T10:19:54.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15458" PostTypeId="2" ParentId="15455" CreationDate="2021-03-02T11:57:17.793" Score="0" Body="&lt;p&gt;The treedata book has a &lt;a href=&quot;https://yulab-smu.top/treedata-book/chapter5.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;section&lt;/a&gt; on tree annotations that should work for you.&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2021-03-02T11:57:17.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15459" PostTypeId="2" ParentId="13942" CreationDate="2021-03-02T20:27:52.727" Score="2" Body="&lt;p&gt;I think this simple script that uses &lt;a href=&quot;https://github.com/biocore-ntnu/pyranges&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyranges&lt;/a&gt; solves it. Half of the below is just setup to make a minimal reproducible example.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# pip install pyranges&#xA;# or&#xA;# conda install -c bioconda pyranges&#xA;&#xA;import pandas as pd&#xA;from io import StringIO &#xA;import pyranges as pr&#xA;&#xA;#########&#xA;# setup #&#xA;#########&#xA;&#xA;gff_name = &amp;quot;gtfo.gtf&amp;quot;&#xA;contents = StringIO(&amp;quot;&amp;quot;&amp;quot;Scaffold_1   JGI gene    22901   45904   .   +   .   ID=Genecv11000001m.g;Name=Genecv11000001m.g&#xA;Scaffold_1  JGI mRNA    22901   45904   .   +   .   ID=PAC4GC:50510902;Name=Genecv11000001m;longest=1;Parent=Genecv11000001m.g&#xA;Scaffold_1  JGI five_prime_UTR  22901   23284   .   +   .   ID=PAC4GC:50510902.five_prime_UTR.1;Parent=PAC4GC:50510902&#xA;Scaffold_1  JGI CDS 23285   23423   .   +   0   ID=PAC4GC:50510902.CDS.1;Parent=PAC4GC:50510902&#xA;Scaffold_1  JGI CDS 24031   24062   .   +   2   ID=PAC4GC:50510902.CDS.2;Parent=PAC4GC:50510902&#xA;Scaffold_1  JGI gene    22901   45904   .   +   .   ID=Gene2.g;Name=Gene2.g&#xA;Scaffold_1  JGI mRNA    22901   45904   .   +   .   ID=PAC4GC:WHATEVZ;Name=Gene2;longest=1;Parent=Gene2.g&#xA;Scaffold_1  JGI five_prime_UTR  22901   23284   .   +   .   ID=PAC4GC:WHATEVZ.five_prime_UTR.1;Parent=PAC4GC:WHATEVZ&#xA;Scaffold_1  JGI CDS 23285   23423   .   +   0   ID=PAC4GC:WHATEVZ.CDS.1;Parent=PAC4GC:WHATEVZ&amp;quot;&amp;quot;&amp;quot;)&#xA;&#xA;pd.read_table(contents, sep=&amp;quot;\s+&amp;quot;, header=None).to_csv(gff_name, sep=&amp;quot;\t&amp;quot;, index=False, header=False)&#xA;&#xA;genes_to_keep = set([&amp;quot;Genecv11000001m.g&amp;quot;])&#xA;&#xA;############&#xA;# solution #&#xA;############&#xA;&#xA;df = pr.read_gff3(gff_name, as_df=True)&#xA;&#xA;def all_descendants(df, genes_to_keep):&#xA;    old_len, new_len = -1, len(genes_to_keep)&#xA;    while old_len != new_len:&#xA;        genes_to_keep.update(df[df.Parent.isin(genes_to_keep)].ID.drop_duplicates())&#xA;        old_len, new_len = new_len, len(genes_to_keep)&#xA;    return df[df.ID.isin(genes_to_keep)]&#xA;&#xA;desired_gff = all_descendants(df, genes_to_keep)&#xA;print(desired_gff)&#xA;#    Chromosome Source         Feature  Start    End Score Strand Frame                                ID               Name longest             Parent&#xA;# 0  Scaffold_1    JGI            gene  22900  45904     .      +     .                 Genecv11000001m.g  Genecv11000001m.g     NaN                NaN&#xA;# 1  Scaffold_1    JGI            mRNA  22900  45904     .      +     .                   PAC4GC:50510902    Genecv11000001m       1  Genecv11000001m.g&#xA;# 2  Scaffold_1    JGI  five_prime_UTR  22900  23284     .      +     .  PAC4GC:50510902.five_prime_UTR.1                NaN     NaN    PAC4GC:50510902&#xA;# 3  Scaffold_1    JGI             CDS  23284  23423     .      +     0             PAC4GC:50510902.CDS.1                NaN     NaN    PAC4GC:50510902&#xA;# 4  Scaffold_1    JGI             CDS  24030  24062     .      +     2             PAC4GC:50510902.CDS.2                NaN     NaN    PAC4GC:50510902&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2688" LastEditorUserId="2688" LastEditDate="2021-03-02T21:00:30.827" LastActivityDate="2021-03-02T21:00:30.827" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15460" PostTypeId="1" AcceptedAnswerId="15467" CreationDate="2021-03-03T02:38:29.187" Score="1" ViewCount="1283" Body="&lt;p&gt;just a quick question about fast5 pass fail folders. I have been working under the assumption that the fast5 pass fail folders I was given as raw data from our ONT vendor came from them doing base calling with guppy 3.something. Is that right? (The person that worked for the vendor has since moved on so I can't ask them - so my question is really does guppy3 divide fast5s into pass/fail?)&lt;/p&gt;&#xA;&lt;p&gt;I further assumed that given I am now doing my own basecalling with guppy4.4 that I should use all the fast5s (both pass and fail) as some of the fails might become pass with the improved NN. Again, am I right? If not could someone please explain?&lt;/p&gt;&#xA;&lt;p&gt;I'm going to also put this question on the ONT community site, FYI&lt;/p&gt;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;&lt;p&gt;Liam&lt;/p&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-03-04T10:22:16.453" Title="Oxford nanopore promethion; does guppy3 separate fast5 into pass and fail? If not what does?" Tags="|nanopore|long-reads|qc|fast5|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15461" PostTypeId="1" CreationDate="2021-03-03T03:57:45.143" Score="1" ViewCount="436" Body="&lt;p&gt;I have a phylogenetic tree in nexus format that was generated with RAxML. In the tree file, there are node support values. I want to plot this tree in R with the support values showing. However, when I import the tree file in R with the read.nexus function in ape, the node values do not appear in $node.labels. Can someone recommend another option in R?&lt;/p&gt;&#xA;" OwnerUserId="10652" LastActivityDate="2021-04-03T06:07:45.827" Title="How to import a phylogenetic tree (in nexus format) with node support values in R" Tags="|r|phylogeny|ggtree|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15462" PostTypeId="1" AcceptedAnswerId="15464" CreationDate="2021-03-03T11:19:40.653" Score="1" ViewCount="81" Body="&lt;p&gt;Here is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6181532/&quot; rel=&quot;nofollow noreferrer&quot;&gt;microarray analysis&lt;/a&gt; where author uses GSE50632 series on an  Agilent microarray GPL17660 platform with 4 eligible samples which I guess are GSM1224991, GSM1224992, GSM1224993 and GSM1224994 among which two are healthy and two are patient samples.&lt;/p&gt;&#xA;&lt;p&gt;However I found the difference in the calculations-&lt;/p&gt;&#xA;&lt;p&gt;Through &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/geo2r/?acc=GSE50632&amp;amp;platform=GPL16294&quot; rel=&quot;nofollow noreferrer&quot;&gt;GEO2R analysis&lt;/a&gt; table it is-&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ID  adj.P.Val   P.Value t   B   logFC   miRNA_ID_LIST   SPOT_ID&#xA;148000  1.86e-02    5.82e-04    3.47    -0.402958   3.91    hsa-miR-3195    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However in the paper the calculated values are far different specially logFC value.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IyapB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IyapB.png&quot; alt=&quot;Paper Cited Table&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What could be the reason for this?&lt;/p&gt;&#xA;" OwnerUserId="10653" LastActivityDate="2023-07-21T17:08:16.037" Title="Difference in Calculation through GEO2R and difference in Article" Tags="|geo|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15463" PostTypeId="1" CreationDate="2021-03-03T11:42:37.203" Score="1" ViewCount="310" Body="&lt;p&gt;In a school project we got the task of finding the species from two FASTA files. We got some hints of what to look for, like it is a unicellular eukaryote and that we can use blast. We don't know if the two files are the same species.&lt;/p&gt;&#xA;&lt;p&gt;So I wonder if there is a standard way of finding species from FASTA files using the terminal?&lt;/p&gt;&#xA;&lt;p&gt;I have a background in informatics, so I'm not well known with bioinformatics tools.&lt;/p&gt;&#xA;" OwnerUserId="10651" LastEditorUserId="73" LastEditDate="2021-03-04T10:29:00.237" LastActivityDate="2021-06-07T05:00:15.063" Title="Find species from FASTA files" Tags="|fasta|blast|taxonomy|homework|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15464" PostTypeId="2" ParentId="15462" CreationDate="2021-03-03T14:22:55.027" Score="1" Body="&lt;p&gt;Here are two example of calculations of logFC validated in GEO2R analysis which depends if values are already log2 transformed. During GEOQuery analysis there is calculation to transform where expression set is checked for value distribution and log2 transformation is applied-&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ex &amp;lt;- exprs(gset)&#xA;qx &amp;lt;- as.numeric(quantile(ex, c(0., 0.25, 0.5, 0.75, 0.99, 1.0), na.rm=T))&#xA;LogC &amp;lt;- (qx[5] &amp;gt; 100) ||&#xA;          (qx[6]-qx[1] &amp;gt; 50 &amp;amp;&amp;amp; qx[2] &amp;gt; 0)&#xA;if (LogC) { ex[which(ex &amp;lt;= 0)] &amp;lt;- NaN&#xA;  exprs(gset) &amp;lt;- log2(ex) }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Example 1:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;=&amp;gt; Sample Values are already log2 Transformed&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;n(TumorSamples) = 45&#xA;n(ControlSamples) = 46&#xA;&#xA;a = average(TumorSamples) = 9.199053111&#xA;b = average(ControlSamples) = 4.988139348&#xA;&#xA;logFC = |a - b| = 4.210914&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Example 2:&lt;/strong&gt;&#xA;=&amp;gt; Sample Values are not log2 transformed&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;n(TumorSamples) = 2&#xA;n(ControlSamples) = 2&#xA;&#xA;a = average(log2(TumorSample1) + log2(TumorSample2) ... ) = (log2(84) + log2(21.8)) / 2 = 5.419287&#xA;&#xA;b = average(log2(ControlSample1) + log2(ControlSample2) ... ) = (log2(624) + log2(665)) / 2 = 9.331306&#xA;&#xA;logFC = |a - b| = 3.91202&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10653" LastEditorUserId="10653" LastEditDate="2021-03-03T14:33:23.793" LastActivityDate="2021-03-03T14:33:23.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15465" PostTypeId="2" ParentId="15447" CreationDate="2021-03-03T21:22:44.453" Score="0" Body="&lt;p&gt;How long are your reads, and how many?  Some colleagues of mine wrote &lt;a href=&quot;https://github.com/eclarke/komplexity&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/eclarke/komplexity&lt;/a&gt; to quickly score complexity in large numbers of short reads.  The score is derived from the number of unique k-mers normalized by the sequence length, so it's simple and fast, though the score inherently becomes less informative for longer and longer sequences (so, best for short reads).  We've used it for filtering repetitive or other low-complexity sequences in metagenomics projects.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2021-03-03T21:22:44.453" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15467" PostTypeId="2" ParentId="15460" CreationDate="2021-03-04T02:40:45.663" Score="1" Body="&lt;p&gt;Only if they've enabled the &lt;code&gt;--qscore_filtering&lt;/code&gt; flag when using Guppy; this flag isn't enabled by default when running Guppy from the command line.&lt;/p&gt;&#xA;&lt;p&gt;It's a good idea to use all fast5 reads for recalling, because accuracy improves over time, so some that previously failed may slide into the pass folder.&lt;/p&gt;&#xA;&lt;p&gt;FYI, I'd be much happier if people asked their nanopore sequencing questions here. It's a publicly-accessible website which is indexed by search engines, so makes it easier to find answers, and easier to see the answers when found (i.e. no login needed).&lt;/p&gt;&#xA;&lt;p&gt;Note that when recalling, you set the input and output directories (I think via the &lt;code&gt;-i&lt;/code&gt; and &lt;code&gt;-s&lt;/code&gt; flags). These should be &lt;em&gt;different&lt;/em&gt; directories, and ideally different output directories for each recalling attempt, to avoid duplicate file entries. The original fast5 files shouldn't be modified by recalling as long as the input and output directories are different.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-03-04T10:22:16.453" LastActivityDate="2021-03-04T10:22:16.453" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15468" PostTypeId="2" ParentId="15461" CreationDate="2021-03-04T05:41:40.677" Score="1" Body="&lt;p&gt;I recommend you this tutorial about ggtree. You will find the solution.&#xA;[https://guangchuangyu.github.io/ggtree-book/short-introduction-to-r.html][ggtree]&lt;/p&gt;&#xA;" OwnerUserId="8757" LastActivityDate="2021-03-04T05:41:40.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15470" PostTypeId="1" CreationDate="2021-03-04T09:06:27.207" Score="0" ViewCount="74" Body="&lt;p&gt;I was trying to calculate t-statistics using Python's &lt;code&gt;scipy&lt;/code&gt; and &lt;code&gt;numpy as np&lt;/code&gt;-&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;scipy.stats.ttest_ind_from_stats(&#xA;    NP.mean(x[['GSM1224991', 'GSM1224992']]),&#xA;    NP.std(x[['GSM1224991', 'GSM1224992']]),&#xA;    len(['GSM1224991', 'GSM1224992']),&#xA;    NP.mean(x[['GSM1224993', 'GSM1224994']]),&#xA;    NP.std(x[['GSM1224993', 'GSM1224994']]),&#xA;    len(['GSM1224993', 'GSM1224994']),&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For example in the dataset given below, &lt;code&gt;GSM1224991 &amp;amp; GSM1224992&lt;/code&gt; are in first group and &lt;code&gt;GSM1224993 &amp;amp; GSM1224994&lt;/code&gt; form second group.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ID_REF,GSM1224991,GSM1224992,GSM1224993,GSM1224994&#xA;148000,6.436150368369428,6.499787040655854,4.430816798843313,3.0819099697950434&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here is the calculation given in GEO2R analysis for the same reference but while I calculate t test it is &lt;code&gt;0.029630766122806237&lt;/code&gt; instead of &lt;code&gt;-0.33768&lt;/code&gt; something as shown below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ID  adj.P.Val   P.Value t   B   logFC&#xA;148000  1   0.001426    -9.44   -0.33768    -2.513&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How to calculate t-stat here? Or considering single row is wrong approach?&lt;/p&gt;&#xA;" OwnerUserId="10653" LastActivityDate="2021-03-04T09:31:18.530" Title="How is the t-statistic value calculated in GEO2R or Bioconductor?" Tags="|statistics|limma|geoquery|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15471" PostTypeId="2" ParentId="15470" CreationDate="2021-03-04T09:31:18.530" Score="1" Body="&lt;p&gt;That is not trivial. GEO2R uses the Bayesian linear model-based framework &lt;code&gt;limma&lt;/code&gt; &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/limma.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/html/limma.html&lt;/a&gt; for the analysis. I doubt you can easily and meaningfully implement the relevant code in Python without a major effort and notable background knowledge. The user guide &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/limma/inst/doc/usersguide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/vignettes/limma/inst/doc/usersguide.pdf&lt;/a&gt; has background information plus links to the relevant papers. Don't try to analyze things by hand, use expert software. If it is not significant, well, then it is not significant. A 2vs2 comparison is strongly underpowered, therefore lack of DEGs are not unexpected, especially for microarrays.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-03-04T09:31:18.530" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15472" PostTypeId="1" CreationDate="2021-03-04T10:30:40.223" Score="0" ViewCount="1727" Body="&lt;p&gt;I have a list of Genes with their alleles (RS IDs) associated with a specific condition. I have to find their genotyping data and the beta values from published studies. I looked through several studies and none of them had anything mentioned about the said terms?&lt;/p&gt;&#xA;&lt;p&gt;Apparently its for calculating the polygenic score of the listed SNPs. Does that mean the genotyping data is the allelic possibilities and beta score as effect size? Effect size is the population size (n)?&lt;/p&gt;&#xA;" OwnerUserId="9234" LastEditorUserId="73" LastEditDate="2021-03-05T00:55:02.987" LastActivityDate="2021-04-04T01:06:25.580" Title="What does genotyping data and beta values for SNPs mean for GWAS study?" Tags="|gwas|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15473" PostTypeId="2" ParentId="15463" CreationDate="2021-03-04T10:31:14.903" Score="2" Body="&lt;p&gt;Are you not allowed to use the &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PAGE=MegaBlast&amp;amp;PROGRAM=blastn&amp;amp;PAGE_TYPE=BlastSearch&quot; rel=&quot;nofollow noreferrer&quot;&gt;web BLAST tool&lt;/a&gt;? That's what I use if I need to quickly find out the likely origin of a DNA sequence.&lt;/p&gt;&#xA;&lt;p&gt;There are command-line ways to do the same thing, but I only use them if I need to search hundreds or thousands of sequences at once.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-03-04T10:31:14.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15474" PostTypeId="2" ParentId="15472" CreationDate="2021-03-04T10:49:21.500" Score="1" Body="&lt;p&gt;The beta statistic is a common output field produced by programs that carry out genome-wide associations. There's a good explanation &lt;a href=&quot;https://www.biostars.org/p/122437/#122440&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I interpret a beta statistic as the degree to which a change in genotype from the reference allele alters the phenotype of interest. The association (and beta statistic) can be negative or positive. It should always be tied to a particular variant, usually called the &amp;quot;risk allele&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;The beta statistic is related to the number of individuals found in each group with a particular genotype. In that sense it could be argued to be related to the population size (at least, assuming pre-grouping of populations by genotype), but that's not a typical interpretation.&lt;/p&gt;&#xA;&lt;p&gt;The GWAS catalogue is a good place to start off looking for data associated with GWAS:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ebi.ac.uk/gwas/home&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/gwas/home&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, I think that's for summary statistics. If someone has asked you for raw genotype data, that is trickier to get, and tends to depend on the condition. Genotype data is personally-identifying information, and there are lots of issues associated with a blanket public release of such information, which is why it's so hard to find.&lt;/p&gt;&#xA;&lt;p&gt;Without more information on the particular phenotype, I can't give much guidance, but useful places to look for most traits are &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gap/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbGaP&lt;/a&gt; and &lt;a href=&quot;https://www.ukbiobank.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;UK Biobank&lt;/a&gt;. Both of those data sources need forms filled out and approved (to demonstrate an ethically-responsible research aim) in order to access genotype-level data.&lt;/p&gt;&#xA;&lt;p&gt;There are a few public datasets around that include data uploaded by individuals (who have explicitly consented to their genotype data being public), but it tends to be trait-specific.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-03-05T00:58:35.323" LastActivityDate="2021-03-05T00:58:35.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15475" PostTypeId="2" ParentId="5660" CreationDate="2021-03-04T10:58:10.500" Score="1" Body="&lt;p&gt;Collecting all the question comments as a community answer (feel free to edit this to make it more readable):&lt;/p&gt;&#xA;&lt;p&gt;There tend to be far fewer counts in single cell experiments. By reducing the scaling factor you are avoiding having massive expression values for your samples.&lt;/p&gt;&#xA;&lt;p&gt;It isn't uncommon to scale single cell experiments by the median cell library size. That way small values typically won't be too small and large values won't be too large. People also choose 10,000 because that's roughly around the library size for a single cell.&lt;/p&gt;&#xA;&lt;p&gt;I haven't seen an explanation yet. Here is an example paper from last week (reputable journal and lab): doi.org/10.1016/j.celrep.2018.11.056 . &amp;quot;... We also excluded outlier cells from the MPPs group which were likely caused by cell contamination. To correct for batch issues, the two experimental batches were centered to have the same mean log(TP100K+1) (hereafter referred to as log(TPM), where TPM stands for ‘transcripts per million’) per gene. PCA was done in R with scaled...&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-03-04T10:58:10.500" CommentCount="0" CommunityOwnedDate="2021-03-04T10:58:10.500" ContentLicense="CC BY-SA 4.0" />
  <row Id="15476" PostTypeId="2" ParentId="7197" CreationDate="2021-03-04T11:01:50.713" Score="2" Body="&lt;p&gt;&lt;em&gt;[Most likely answer based on comments attached to the question]&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Hi Courtney, We've looked into the data and identified the source of the different mapping rates. Specifically, the cause is discordant reads; in this case reads that are dovetailing with respect to their positions on the reference. If you run Bowtie2 with &amp;quot;RNA-seq&amp;quot; flags (i.e. disallowing discordant mappings), then you get essentially the same mapping rate as salmon here. Is allowing dovetailing reads something that should be very important in your analysis? If so, we can consider what would be necessary to add back support, but generally, they provide evidence of tenuous quality.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-03-04T11:01:50.713" CommentCount="0" CommunityOwnedDate="2021-03-04T11:01:50.713" ContentLicense="CC BY-SA 4.0" />
  <row Id="15479" PostTypeId="2" ParentId="13916" CreationDate="2021-03-04T13:35:25.527" Score="0" Body="&lt;p&gt;I made the same &lt;a href=&quot;https://stats.stackexchange.com/q/478037/208550&quot;&gt;question in CrossValidated&lt;/a&gt; forum and got an excellent answer!&lt;/p&gt;&#xA;&lt;p&gt;The important part:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;You need to correct for all of the comparisons you are doing. So if&#xA;that's 300,000,000 comparisons you need to correct for that many&#xA;multiple comparisons.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;For more information check the answer in the link above&lt;/p&gt;&#xA;" OwnerUserId="9220" LastActivityDate="2021-03-04T13:35:25.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15480" PostTypeId="2" ParentId="3838" CreationDate="2021-03-04T15:05:14.110" Score="0" Body="&lt;pre&gt;&lt;code&gt;# pip install pyranges&#xA;# pip install bamread&#xA;&#xA;# wget http://hgdownload-test.cse.ucsc.edu/goldenPath/hg19/encodeDCC/wgEncodeRikenCage/releaseLatest/wgEncodeRikenCageGm12878CellPapAlnRep1.bam # (half a gig)    &#xA;# samtools index wgEncodeRikenCageGm12878CellPapAlnRep1.bam&#xA;&#xA;f = &amp;quot;wgEncodeRikenCageGm12878CellPapAlnRep1.bam&amp;quot;&#xA;&#xA;import pyranges as pr&#xA;df = pr.read_bam(f, sparse=False, as_df=True)&#xA;&#xA;df[[&amp;quot;Chromosome&amp;quot;, &amp;quot;Start&amp;quot;, &amp;quot;End&amp;quot;, &amp;quot;Strand&amp;quot;, &amp;quot;Cigar&amp;quot;]].drop_duplicates(&amp;quot;Cigar&amp;quot;)&#xA;#          Chromosome  Start    End Strand             Cigar&#xA;# 0              chr1  10579  10606      -               27M&#xA;# 36             chr1  29334  29360      +           8M1I18M&#xA;# 37             chr1  29334  29360      +           7M1I19M&#xA;# 38             chr1  29334  29360      +           5M1I21M&#xA;# 44             chr1  29339  29365      +           4M1I22M&#xA;# ...             ...    ...    ...    ...               ...&#xA;# 19655298       chrM  14550  14575      +       16M5I3M3D3M&#xA;# 19660213       chrM  14747  14775      -       15M5D5M4I3M&#xA;# 19663987       chrM  15374  15399      -   2M1I2M2D4M3I15M&#xA;# 19676864       chrM  16234  16271      +  15M1I4M12D5M1I1M&#xA;# 19677003       chrM  16264  16291      +   14M1D3M2D1M3I6M&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2688" LastActivityDate="2021-03-04T15:05:14.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15481" PostTypeId="1" CreationDate="2021-03-04T19:31:48.483" Score="1" ViewCount="1792" Body="&lt;p&gt;I’m using STAR to align fastq files from SMART-seq2. I have raw data folder containing sub-folders with samples names the sub-folders each contain fastq file. How can I make a bash command in order for STAR to go in each of sub-folders and make alignment for the fastq file and retrieve the bam files in one folder?&lt;/p&gt;&#xA;&lt;p&gt;Folder structure:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/plates/plate_1/rawdata/A1/A1_R1.fastq&#xA;/plates/plate_1/rawdata/B1/B1_R1.fastq&#xA;...&#xA;/plates/plate_1/rawdata/O24/O24_R1.fastq&#xA;/plates/plate_1/rawdata/P24/P24_R1.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It’s 384 sample folders A1, A2, B3, F5,,, so on&lt;/p&gt;&#xA;" OwnerUserId="10595" LastEditorUserId="92" LastEditDate="2021-12-09T20:58:43.327" LastActivityDate="2022-01-08T21:00:30.050" Title="STAR aligner multiple fastq files" Tags="|sequence-alignment|genome|bash|star|loop|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15482" PostTypeId="1" CreationDate="2021-03-04T21:06:48.437" Score="0" ViewCount="48" Body="&lt;p&gt;I am making some R markdown documents for my stats class.  I've noticed I get an error in some functions when I have a code block with {r}  instead of just r.  Like so...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;``` r&#xA;this code will be fine&#xA;``` &#xA;&#xA;``` {r}&#xA;this code will have an error&#xA;```&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When should I be using the brackets for codeblocks in .Rmd and when should I leave them out?&lt;/p&gt;&#xA;" OwnerUserId="2482" LastActivityDate="2022-10-05T06:05:10.597" Title="Brackets in .Rmd code blocks" Tags="|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15483" PostTypeId="2" ParentId="15482" CreationDate="2021-03-05T02:35:11.433" Score="2" Body="&lt;p&gt;Does it work without a space between the &lt;code&gt;```&lt;/code&gt; and the &lt;code&gt;{r}&lt;/code&gt;? It may be that the Rmd processor is getting confused about the spaces, but ignoring the bare &lt;code&gt;r&lt;/code&gt;, and assuming the code is R by default.&lt;/p&gt;&#xA;&lt;p&gt;The examples here don't have spaces:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://rmarkdown.rstudio.com/lesson-3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rmarkdown.rstudio.com/lesson-3.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-03-05T02:35:11.433" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15484" PostTypeId="1" CreationDate="2021-03-05T13:54:01.303" Score="2" ViewCount="100" Body="&lt;p&gt;I am trying to calculate beta diversity* using the UniFrac distance on my data, each sample having OTUs values.&lt;/p&gt;&#xA;&lt;p&gt;To calculate it, I saw it required building a tree with percentages that represented the OTUs.&#xA;However, I was unable to figure what percentage fit each OTU to build the tree from the instructions I found on the internet.&lt;/p&gt;&#xA;&lt;p&gt;For example, I saw in this link: &lt;a href=&quot;http://readiab.org/book/0.1.1/3/1#4.1.2&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://readiab.org/book/0.1.1/3/1#4.1.2&lt;/a&gt;,  that they build this tree:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WcpvO.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WcpvO.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Could you please explain to me how to build the tree?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This is the diversity between local sites and enables the total in a landscape to be calculated (gamma). The tree would therefore appear (presumably) to represent the diversity between sites.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="10679" LastEditorUserId="3848" LastEditDate="2022-02-21T01:42:34.967" LastActivityDate="2024-03-12T07:06:10.710" Title="Build tree to calculate beta diversity" Tags="|phylogeny|phylogenetics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15485" PostTypeId="1" CreationDate="2021-03-05T15:09:57.603" Score="0" ViewCount="89" Body="&lt;p&gt;I am new to the bioinformatics field. I have positive and negative protein sequences for acetylation PTM. Now, I want to train a classifier, say SVM. What will be the next step? How can I convert these sequences into usable features? Any information or links would help.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;P31327_55|1|testing&#xA;KAQTAHIVLEDGTKMKGYSFGHPSSVA&#xA;&amp;gt;P31327_57|1|testing&#xA;QTAHIVLEDGTKMKGYSFGHPSSVAGE&#xA;&amp;gt;P31327_119|1|testing&#xA;APDTTALDELGLSKYLESNGIKVSGLL&#xA;&amp;gt;P31327_157|1|testing&#xA;LATKSLGQWLQEEKVPAIYGVDTRMLT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10680" LastActivityDate="2021-03-16T09:32:33.963" Title="Extract features from fasta sequences and train the classifier" Tags="|fasta|proteins|machine-learning|features|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15486" PostTypeId="1" AcceptedAnswerId="15491" CreationDate="2021-03-05T23:25:29.790" Score="1" ViewCount="159" Body="&lt;p&gt;&lt;em&gt;Note: this question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/494967/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I started getting a strange error when trying to load GEOquerry with &lt;code&gt;library(GEOquerry)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error: package or namespace load failed for ‘GEOquery’ in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]): namespace ‘rlang’ 0.4.9 is already loaded, but &amp;gt;= 0.4.10 is required&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I run &lt;code&gt;&amp;gt; devtools::install_github('r-lib/rlang')&lt;/code&gt; I get ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]) : &#xA;  namespace ‘rlang’ 0.4.9 is already loaded, but &amp;gt;= 0.4.10 is required&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Has anyone dealt with this before?  Restarting my R session has no effect.&lt;/p&gt;&#xA;" OwnerUserId="2482" LastEditorUserId="73" LastEditDate="2021-06-17T10:43:33.127" LastActivityDate="2021-06-17T10:43:33.127" Title="Calling GEOquerry error while loading the library" Tags="|r|bioconductor|geoquery|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15487" PostTypeId="1" AcceptedAnswerId="15495" CreationDate="2021-03-05T23:31:02.067" Score="0" ViewCount="83" Body="&lt;p&gt;I have a matrix file (expression.txt) in which the first column is a gene_id and from the second column the sample id's start. This matrix has 20,000 columns with sample id's corresponding to different tissue. I want to extract the samples of one tissue. Example:  SM-5GZZ7 SM-5GIEN SM-5EGGH are samples from brain tissue. So I want to separate the list of samples [(file=brain.txt;n=5000] corresponding to brain tissue from this matrix (with bash or R)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Name    SM-5GZZ7    SM-5EGHI    SM-5EGHJ    SM-5N9CS    SM-5GIEN    SM-5EGHH    SM-5EGGH&#xA;ENSG00000223972.5   0   0   0   0   0   0   0&#xA;ENSG00000227232.5   187 109 143 251 113 139 199&#xA;ENSG00000278267.1   0   0   1   0   0   0   0&#xA;ENSG00000243485.5   1   0   0   1   0   0   0&#xA;ENSG00000237613.2   0   0   0   0   0   0   0&#xA;ENSG00000268020.3   0   1   0   0   0   1   1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The file brain.txt looks like (Number of lines in this files = 5000)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SM-5GZZ7&#xA;SM-5GIEN &#xA;SM-5EGGH&#xA;SM-5GZXC&#xA;SM-5GZXO&#xA;SM-5GZY1&#xA;SM-5EGHK&#xA;SM-5EGHL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3766" LastEditorUserId="3766" LastEditDate="2021-03-06T08:59:28.677" LastActivityDate="2021-03-07T18:35:01.487" Title="Print specific columns in a matrix on the basis of sample id's in the header" Tags="|gene-expression|bash|data-retrieval|data-preprocessing|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15488" PostTypeId="1" CreationDate="2021-03-06T00:10:41.427" Score="1" ViewCount="45" Body="&lt;p&gt;Conversion of a multi-entry sdf to a multi-entry pdb file with open babel results in the loss of the compound names in the resulting PDB file. Why do you think this might occur and how can I resolve this issue.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QHDDF.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QHDDF.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/i7dHF.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/i7dHF.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9346" LastEditorUserId="9346" LastEditDate="2021-03-09T01:02:06.437" LastActivityDate="2021-03-16T19:30:33.797" Title="Error during File conversion" Tags="|python|database|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15490" PostTypeId="2" ParentId="15487" CreationDate="2021-03-06T02:28:06.723" Score="1" Body="&lt;p&gt;There's a million different ways to do simple taks like this. Pick one and learn it:) Python is my go to as it can do pretty much everything you'd ever want. I saved your example as csv with no spaces called expression.txt.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;df = pd.read_csv('expression.txt')&#xA;df[['SM-5GZZ7','SM-5GIEN','SM-5EGGH']]&#xA;&#xA;SM-5GZZ7    SM-5GIEN    SM-5EGGH&#xA;0   0   0.0 0.0&#xA;1   187 109 143 251 113 139 199 NaN NaN&#xA;2   0   0.0 0.0&#xA;3   1   0.0 0.0&#xA;4   0   0.0 0.0&#xA;5   0   0.0 1.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-03-06T02:28:06.723" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15491" PostTypeId="2" ParentId="15486" CreationDate="2021-03-06T15:27:18.180" Score="1" Body="&lt;p&gt;rlang 0.4.10 is the current version at CRAN: &lt;a href=&quot;https://cran.r-project.org/web/packages/rlang/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cran.r-project.org/web/packages/rlang/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Try to install it via &lt;code&gt;install.packages&lt;/code&gt;, not via GitHub directly.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-03-06T15:27:18.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15493" PostTypeId="2" ParentId="15302" CreationDate="2021-03-07T03:07:14.550" Score="1" Body="&lt;p&gt;Ok, with invaluable aid from @Maximillian Press, I've figured out how to find alleles which are variable between two species but fixed within one species.&lt;/p&gt;&#xA;&lt;p&gt;Here is how you find SNP/Indel loci which are distinct between species A (hereafter sA) and species B (hereafter sB) but are fixed WITHIN sA:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Maximillian Press's Contribution: Map sA reference to sB reference using minimap2, then convert to bam. &#xA;&#xA;minimap2 -a sB_ref.fasta sA_ref.fasta &amp;gt; AtoB.sam &#xA;samtools view -bh AtoB.sam &amp;gt; AtoB.bam &#xA;&#xA;&#xA;# Sort bam by coordinates, then use bcftools to generate a pileup and call all alleles of sA relative to sB&#xA;&#xA;java -jar &lt;span class=&quot;math-container&quot;&gt;$EBROOTPICARD/picard.jar SortSam I=AtoB.bam O=sorted_AtoB.bam CREATE_INDEX=true SORT_ORDER=coordinate TMP_DIR=$&lt;/span&gt;TMPDIR    &#xA;bcftools mpileup -O b -o AtoB.bcf -f sB_ref.fasta sorted_AtoB.bam&#xA;bcftools call -m -v -o AtoB.vcf AtoB.bcf &#xA;&#xA;&#xA; # Call all alleles of sA relative to sA. &#xA;# This represents the SNP/Indel loci which are variable WITHIN sA and thus ought to be discarded from the final output.&#xA;# &amp;quot;bam_listA.txt&amp;quot; contains names of all the coordinate-sorted bam files for species A&#xA;bcftools mpileup -O b -o AtoA.bcf -f sA_ref.fasta -b bam_listA.txt&#xA;bcftools call -m -v -o AtoA.vcf AtoA.bcf &#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then use the &amp;quot;sed&amp;quot; function to replace chromosome names with their analogs (happy to give more details if anyone else needs to do this) and also removed the VCF headers prior to making a C++ program which extracts all the rows found in AtoB.vcf but not in AtoA.vcf. My final output was thus all SNP/Indel loci which vary between sA and sB but NOT within sA.&#xA;Happy to share the C++ program if anyone else needs to do this.&lt;/p&gt;&#xA;" OwnerUserId="9118" LastActivityDate="2021-03-07T03:07:14.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15494" PostTypeId="1" CreationDate="2021-03-07T17:43:31.877" Score="2" ViewCount="586" Body="&lt;p&gt;I need help with my Biopython code, I am kind of new to Python and Biopython so please don't be harsh on me.&lt;/p&gt;&#xA;&lt;p&gt;Everything started from here: &lt;a href=&quot;https://stackoverflow.com/questions/66462611/remove-duplicated-sequences-in-fasta-with-python/66463550#66463550&quot;&gt;Remove duplicated sequences in FASTA with Python&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I answered copying code from here: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/13708/remove-redundant-sequences-from-fasta-file-in-python&quot;&gt;Remove Redundant Sequences from FASTA file in Python&lt;/a&gt; but I am still bugged about the OP's first question: &lt;a href=&quot;https://stackoverflow.com/questions/66462611/remove-duplicated-sequences-in-fasta-with-python/66463550#66463550&quot;&gt;Remove duplicated sequences in FASTA with Python&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So I decided to try to minimize the code footprint on RAM. I ended up copying from Biopython documentation &lt;a href=&quot;https://biopython.org/wiki/Split_large_file&quot; rel=&quot;nofollow noreferrer&quot;&gt;Split large file&lt;/a&gt; and producing:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;# -*- coding: utf-8 -*-&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;Created on Thu Nov  11 17:40:03 2020&#xA;&#xA;@author: Pietro&#xA;&#xA;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;from memory_profiler import memory_usage&#xA;&#xA;import memory_profiler &#xA;&#xA;&#xA;def batch_iterator(iterator, batch_size):&#xA;    &amp;quot;&amp;quot;&amp;quot; from https://biopython.org/wiki/Split_large_file&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    entry = True  # Make sure we loop once&#xA;    while entry:&#xA;        batch = []&#xA;        while len(batch) &amp;lt; batch_size:&#xA;            try:&#xA;                entry = iterator.__next__()&#xA;            except StopIteration:&#xA;                entry = None&#xA;            if entry is None:&#xA;                # End of file&#xA;                break&#xA;            batch.append(entry)&#xA;        if batch:&#xA;            yield batch&#xA;&#xA;&#xA;# @profile&#xA;def f():&#xA;    &#xA;    from Bio import SeqIO &#xA;    &#xA;    import time&#xA;    &#xA;    import sys&#xA;   &#xA;    start = time.time() &#xA;&#xA;    data_input_iter = SeqIO.parse(&amp;quot;Newtextfile-03&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;    print('data_input_iter : ',sys.getsizeof(data_input_iter),'\n')&#xA;    &#xA;&#xA;    c=0&#xA;    for i , batch in enumerate(batch_iterator(data_input_iter , 1)):&#xA;        record_iter = batch &#xA;&#xA;        val = True&#xA;        c += 1&#xA;        if c == 1:&#xA;            with open(&amp;quot;removed-double_Newtextfile-03&amp;quot;, 'w+') as output_file:&#xA;                    SeqIO.write(record_iter, output_file, &amp;quot;fasta&amp;quot;)&#xA;                    # print('copy   : ',(record_iter))&#xA;                    output_file.flush()&#xA;            output_file.close()&#xA;        &#xA;        else:&#xA;            &#xA;            SeqIO.write(record_iter, &amp;quot;temp_file&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;            &#xA;            record_iter = SeqIO.read(&amp;quot;temp_file&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;            &#xA;            data_output_iter = SeqIO.parse(&amp;quot;removed-double_Newtextfile-03&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;            &#xA;            d = 0&#xA;            &#xA;            try:&#xA;                for i , batch in enumerate(batch_iterator(data_output_iter  , 1)):&#xA;                        SeqIO.write(batch, &amp;quot;temp_file2&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;                    &#xA;                        record_output = SeqIO.read(&amp;quot;temp_file2&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;                    &#xA;                        d += 1&#xA;                        if record_iter.seq != record_output.seq  :&#xA;                            # print('pass   : ',record_iter.name)&#xA;                            pass&#xA;                        else:&#xA;                            val = False&#xA;                            # print('false')&#xA;                            break&#xA;&#xA;                &#xA;            except:&#xA;                print('ERROR !!!!!!!!!!!')&#xA;                quit()&#xA;&#xA;            if val:&#xA;                        # print('copy   : ',record_iter.name)&#xA;                        with open(&amp;quot;removed-double_Newtextfile-03&amp;quot;, 'a+') as output_file:&#xA;                            output_file.write(str(record_iter.format(&amp;quot;fasta&amp;quot;)))&#xA;                        output_file.close()&#xA;            del data_output_iter&#xA;    &#xA;    end = time.time()&#xA;    &#xA;    print(f&amp;quot;Run time is {(end- start)}&amp;quot;) &#xA;&#xA;&#xA;mem_usage = memory_usage(f)&#xA;# print('Memory usage (in chunks of .1 seconds): %s' % mem_usage)&#xA;print('Maximum memory usage: %s' % max(mem_usage))&#xA;&#xA;&#xA;# f()&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but I have to major problems :&lt;/p&gt;&#xA;&lt;p&gt;FIRST: I wasn't able to find a way to pass a SeqRecord object from a list to a file without saving it to a temp file first. That is without:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i , batch in enumerate(batch_iterator(data_input_iter , 1)):&#xA;        record_iter = batch &#xA;......&#xA;&#xA;SeqIO.write(record_iter, &amp;quot;temp_file&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;            &#xA;record_iter = SeqIO.read(&amp;quot;temp_file&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;the line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; if record_iter.seq != record_output.seq  :&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;will throw:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;AttributeError: 'list' object has no attribute 'format'&#xA;&#xA;AttributeError: 'list' object has no attribute 'seq'&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;you get first error and the if remove format from:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;output_file.write(str(record_iter.format(&amp;quot;fasta&amp;quot;)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and #quit() you get the second error&#xA;How should I cope with this ? I mean is there a way to bypass the save and read lines ?&lt;/p&gt;&#xA;&lt;p&gt;SECOND : and more important. I am sure my code doesn't use a set and a list .... and that's why is extremely slow compared to the copied &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/13708/remove-redundant-sequences-from-fasta-file-in-python&quot;&gt;Remove Redundant Sequences from FASTA file in Python&lt;/a&gt; posted below too. But how does it use even more RAM that the fastest code ? I tried to minimize the memory footprint but wasn't able !! Am I using a bad way to try to calculate the amount of RAM used ? And does Biopythom SeqIO.parse object already is the optimal choice for , lets say &amp;quot;2/4 Gb size&amp;quot; fasta files on a laptop with just 4Gb of RAM (remember the original OP post  &lt;a href=&quot;https://stackoverflow.com/questions/66462611/remove-duplicated-sequences-in-fasta-with-python/66463550#66463550&quot;&gt;Remove duplicated sequences in FASTA with Python&lt;/a&gt;). Maybe my test file is not big enough ? Does anybody have a 2GB fasta file with a dozen of duplicates in it ?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/66462611/remove-duplicated-sequences-in-fasta-with-python/66463550#66463550&quot;&gt;Remove duplicated sequences in FASTA with Python&lt;/a&gt; memory usage:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;size of parser :  48&#xA;memory seen :  32984&#xA;memory records :  2888&#xA;Run time is 0.0008047382036844889&#xA;Maximum memory usage: 55.359375&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;my code memory usage and speed:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data_input_iter :  48 &#xA;&#xA;Run time is 37.781240463256836&#xA;Maximum memory usage: 56.40234375&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My fasta test set is: &lt;a href=&quot;https://easyupload.io/z2yss1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://easyupload.io/z2yss1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Biopython method to remove duplicated from  &lt;a href=&quot;https://stackoverflow.com/questions/66462611/remove-duplicated-sequences-in-fasta-with-python/66463550#66463550&quot;&gt;Remove duplicated sequences in FASTA with Python&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;# -*- coding: utf-8 -*-&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;copied from https://bioinformatics.stackexchange.com/questions/13708/remove-redundant-sequences-from-fasta-file-in-python&#xA;&#xA;uses set() instead of list for seen&#xA;&#xA;should work faster for big files&#xA;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;from memory_profiler import memory_usage&#xA;&#xA;def f():&#xA;    &#xA;    import sys&#xA;&#xA;    from Bio import SeqIO&#xA;    import time&#xA;    &#xA;    start = time.time() &#xA;    &#xA;    seen = set()&#xA;    records = []&#xA;    &#xA;    &#xA;    parser = SeqIO.parse(&amp;quot;Newtextfile-03&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;    for record in parser:  &#xA;        # print('running')&#xA;        if str(record.seq) not in seen:&#xA;            seen.add(str(record.seq))&#xA;            records.append(record)&#xA;            # print('copying')&#xA;    &#xA;    &#xA;    print('size of parser : ',sys.getsizeof(parser) )&#xA;    print('memory seen : ',sys.getsizeof(seen))&#xA;    print('memory records : ',sys.getsizeof(records))&#xA;    &#xA;    #writing to a fasta file&#xA;    SeqIO.write(records, &amp;quot;removed-double_Newtextfile-03-from-002&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;    end = time.time()&#xA;    &#xA;    print(f&amp;quot;Run time is {(end- start)/60}&amp;quot;) &#xA;&#xA;mem_usage = memory_usage(f)&#xA;# print('Memory usage (in chunks of .1 seconds): %s' % mem_usage)&#xA;print('Maximum memory usage: %s' % max(mem_usage))&#xA;&#xA;# f()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2023-01-24T11:40:16.287" LastActivityDate="2023-01-24T11:40:16.287" Title="Remove Redundant Sequences from FASTA file with Biopython reducing memory footprint" Tags="|python|fasta|seqio|biopython|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15495" PostTypeId="2" ParentId="15487" CreationDate="2021-03-07T18:35:01.487" Score="1" Body="&lt;p&gt;Example in R. If you have a file of column names, &lt;code&gt;names.txt&lt;/code&gt;, such as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mpg &#xA;cyl  &#xA;disp&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and a data.frame like &lt;code&gt;mtcars&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; mtcars&#xA;                   mpg cyl  disp  hp drat    wt  qsec vs am gear carb&#xA;Mazda RX4         21.0   6 160.0 110 3.90 2.620 16.46  0  1    4    4&#xA;Mazda RX4 Wag     21.0   6 160.0 110 3.90 2.875 17.02  0  1    4    4&#xA;Datsun 710        22.8   4 108.0  93 3.85 2.320 18.61  1  1    4    1&#xA;Hornet 4 Drive    21.4   6 258.0 110 3.08 3.215 19.44  1  0    3    1&#xA;Hornet Sportabout 18.7   8 360.0 175 3.15 3.440 17.02  0  0    3    2&#xA;Valiant           18.1   6 225.0 105 2.76 3.460 20.22  1  0    3    1&#xA;Duster 360        14.3   8 360.0 245 3.21 3.570 15.84  0  0    3    4&#xA;Merc 240D         24.4   4 146.7  62 3.69 3.190 20.00  1  0    4    2&#xA;Merc 230          22.8   4 140.8  95 3.92 3.150 22.90  1  0    4    2&#xA; [ reached 'max' / getOption(&amp;quot;max.print&amp;quot;) -- omitted 23 rows ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then you can select the columns by:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(dplyr)&#xA;list_names = read.table(&amp;quot;names.txt&amp;quot;, header=F) %&amp;gt;% dplyr::select(V1)&#xA;&amp;gt; mtcars %&amp;gt;% dplyr::select(list_names$V1)&#xA;                     mpg cyl  disp&#xA;Mazda RX4           21.0   6 160.0&#xA;Mazda RX4 Wag       21.0   6 160.0&#xA;Datsun 710          22.8   4 108.0&#xA;Hornet 4 Drive      21.4   6 258.0&#xA;Hornet Sportabout   18.7   8 360.0&#xA;Valiant             18.1   6 225.0&#xA;Duster 360          14.3   8 360.0&#xA;Merc 240D           24.4   4 146.7&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2021-03-07T18:35:01.487" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15496" PostTypeId="2" ParentId="15494" CreationDate="2021-03-07T18:43:28.810" Score="2" Body="&lt;p&gt;I wrote the answer &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/13710/104&quot;&gt;here&lt;/a&gt; and while I'm not clear on all aspects of this question, I can offer two suggestions to reduce the memory footprint considerably while maintaining high speed.&lt;/p&gt;&#xA;&lt;p&gt;First, rather than appending the records to a list you could use a generator that generates elements on-the-fly rather than storing them in memory, e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seen = set()&#xA;&#xA;def yield_seqs():&#xA;    for record in SeqIO.parse(&amp;quot;b4r2.fasta&amp;quot;, &amp;quot;fasta&amp;quot;):  &#xA;        if record.seq not in seen:&#xA;            seen.add(record.seq)&#xA;            yield record&#xA;&#xA;SeqIO.write(yield_seqs(), &amp;quot;output.fasta&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Second you could save even more memory, &lt;strong&gt;but with the risk of false positives&lt;/strong&gt;, by using a &lt;a href=&quot;https://en.wikipedia.org/wiki/Bloom_filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;bloom filter&lt;/a&gt;, e.g. using &lt;a href=&quot;https://github.com/hiway/python-bloom-filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;python-bloom-filter&lt;/a&gt;, just replace the set with &lt;code&gt;seen = BloomFilter(max_elements=10000, error_rate=0.001)&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2021-03-07T18:43:28.810" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15497" PostTypeId="2" ParentId="15295" CreationDate="2021-03-08T08:36:09.570" Score="1" Body="&lt;p&gt;If downloading the complete database is not an option, you may able to use the instructions from this help page (based on the gff format for any given query):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.uniprot.org/help/download_subsequences&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/help/download_subsequences&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1459" LastActivityDate="2021-03-08T08:36:09.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15498" PostTypeId="2" ParentId="15238" CreationDate="2021-03-08T08:54:10.150" Score="2" Body="&lt;p&gt;Using the UniProt website API (&lt;a href=&quot;https://www.uniprot.org/help/api&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/help/api&lt;/a&gt;) this could be done with a query like&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.uniprot.org/uniprot/?query=name%3A%22type%20IV%20secretion%20protein%20Rhs%22&amp;amp;columns=id%2Centry%20name%2Creviewed%2Cprotein%20names%2Cgenes%2Cgo(biological%20process)%2Cgo(molecular%20function)%2Cgo(cellular%20component)&amp;amp;sort=score&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uniprot/?query=name%3A%22type%20IV%20secretion%20protein%20Rhs%22&amp;amp;columns=id%2Centry%20name%2Creviewed%2Cprotein%20names%2Cgenes%2Cgo(biological%20process)%2Cgo(molecular%20function)%2Cgo(cellular%20component)&amp;amp;sort=score&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;or, in tab-separated format:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.uniprot.org/uniprot/?query=name:%22type%20IV%20secretion%20protein%20Rhs%22&amp;amp;format=tab&amp;amp;limit=10&amp;amp;columns=id,entry%20name,reviewed,protein%20names,genes,go(biological%20process),go(molecular%20function),go(cellular%20component)&amp;amp;sort=score&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uniprot/?query=name:%22type%20IV%20secretion%20protein%20Rhs%22&amp;amp;format=tab&amp;amp;limit=10&amp;amp;columns=id,entry%20name,reviewed,protein%20names,genes,go(biological%20process),go(molecular%20function),go(cellular%20component)&amp;amp;sort=score&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;(Remove the &amp;quot;&amp;amp;limit=10&amp;quot; part from the URL to obtain the full set instead of the first 10 entries.)&lt;/p&gt;&#xA;&lt;p&gt;You can add colums as described here:&#xA;&lt;a href=&quot;https://www.uniprot.org/help/customize&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/help/customize&lt;/a&gt;,&#xA;&lt;a href=&quot;https://www.youtube.com/watch?v=ado1r8IDm3U&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=ado1r8IDm3U&lt;/a&gt; (video, at about 1min25)&#xA;or&#xA;&lt;a href=&quot;https://insideuniprot.blogspot.com/2015/03&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://insideuniprot.blogspot.com/2015/03&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note that while UniProt tries to improve protein name annotation, the unreviewed section of UniProtKB (i.e. UniProtKB/TrEMBL) is often far from perfect and its accuracy relies on the information provided by the submitter of the nucleotide entry, unless an automatic annotation rule is available to assign a more suitable name (see &lt;a href=&quot;https://www.uniprot.org/help/protein_names&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/help/protein_names&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Please don't hesitate to contact the UniProt helpdesk if you have any additional questions.&lt;/p&gt;&#xA;" OwnerUserId="1459" LastActivityDate="2021-03-08T08:54:10.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15499" PostTypeId="1" CreationDate="2021-03-08T21:41:06.647" Score="1" ViewCount="88" Body="&lt;p&gt;I have 20 samples out of which 14 are 100 bp in length and 6 are 150 bp. Is there a way to normalize the read length for cross-sample differential expression comparison? What would be the best way to approach this?&lt;/p&gt;&#xA;&lt;p&gt;Note: The samples are NOT from reference/model species.&lt;/p&gt;&#xA;" OwnerUserId="10697" LastActivityDate="2021-03-09T03:11:52.820" Title="Bulk RNA-Seq Read Length Normalization across different samples" Tags="|rna-seq|deseq2|normalization|edger|salmon|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15500" PostTypeId="2" ParentId="15499" CreationDate="2021-03-08T21:53:11.747" Score="1" Body="&lt;p&gt;The simplest thing to do is to trim the 150 bp fasts so that they are 100 long.  I don't think there is an easy way to correct for the fact that the 150 bp long reads will have a higher unambiguous rate of alignment and gene assignment than the 100 bp long reads.&lt;/p&gt;&#xA;&lt;p&gt;If you have a mix of all the experimental conditions across all the lengths, you can include length as an element in the design, so that DESeq2 may be able to model the difference between the two batches.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2021-03-08T21:53:11.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15501" PostTypeId="2" ParentId="12866" CreationDate="2021-03-09T02:03:12.823" Score="0" Body="&lt;p&gt;Have a look here and the entire tutorial:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/hbctraining/scRNA-seq/blob/master/lessons/03_SC_quality_control-setup.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/hbctraining/scRNA-seq/blob/master/lessons/03_SC_quality_control-setup.md&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10699" LastActivityDate="2021-03-09T02:03:12.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15502" PostTypeId="2" ParentId="15499" CreationDate="2021-03-09T03:11:52.820" Score="0" Body="&lt;p&gt;I've done this by doing a RPkM-like transformation which I call VSTPk. Transformations that adjust for transcript length will correct a little bit for the noise associated with mapping short reads to transcripts of varying lengths:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/a/15218/73&quot;&gt;https://bioinformatics.stackexchange.com/a/15218/73&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Another, probably better, alternative is to use Salmon to map reads to transcripts, which applies its own mapping-based correction to read counts. Salmon produces a statistic that relates to the mappable length of a transcript, and normalises counts based on how many molecules the model predicts came from each transcript isoform.&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately all these normalisation techniques require good gene models, which are difficult to obtain for non-model organisms.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-03-09T03:11:52.820" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15503" PostTypeId="1" CreationDate="2021-03-09T04:37:24.590" Score="0" ViewCount="121" Body="&lt;p&gt;I am using Mrbayes v 3.2.7 MPI version to infer phylogeny, and I set the ngeneration to 10,000,000, but it often stop at certain generation (usually far smaller than given generations). Even if I restart the analysis by invoking the &lt;code&gt;append=yes&lt;/code&gt; , but it stops again.&lt;/p&gt;&#xA;&lt;p&gt;Anyone have this problem? I would appreciate you for sharing your useful experience.&lt;/p&gt;&#xA;&lt;p&gt;Best regards,&lt;/p&gt;&#xA;&lt;p&gt;Yanpeng&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Version info&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;Version:   3.2.7&#xA;Features:  SSE AVX FMA Beagle readline&#xA;Host type: x86_64-conda-linux-gnu (CPU: x86_64)&#xA;Compiler:  gnu 7.5.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Command line:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;nohup mpirun -np 8 mb-mpi concatenate_phylipID.nex &amp;lt; /dev/null &amp;gt; /dev/null &amp;amp;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Mrbayes block in nexus file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;begin mrbayes;&#xA;&#xA;    outgroup t1;&#xA;&#xA;    charset Subset1 = 1-373 1953-2032;&#xA;    charset Subset2 = 374-531;&#xA;    charset Subset3 = 532-677 1583-1618;&#xA;    charset Subset4 = 678-1442;&#xA;    charset Subset5 = 1443-1582 1619-1725;&#xA;    charset Subset6 = 1726-1802 1803-1952;&#xA;    charset Subset7 = 2033-2661;&#xA;&#xA;    partition PartitionFinder = 7:Subset1, Subset2, Subset3, Subset4, Subset5, Subset6, Subset7;&#xA;    set partition=PartitionFinder;&#xA;&#xA;    lset applyto=(1) nst=6 rates=invgamma;&#xA;    lset applyto=(2) nst=6 rates=invgamma;&#xA;    prset applyto=(2) statefreqpr=fixed(equal);&#xA;    lset applyto=(3) nst=6 rates=invgamma;&#xA;    lset applyto=(4) nst=6 rates=invgamma;&#xA;    lset applyto=(5) nst=6 rates=invgamma;&#xA;    lset applyto=(6) nst=6 rates=gamma;&#xA;    lset applyto=(7) nst=6 rates=invgamma;&#xA;&#xA;    prset applyto=(all) ratepr=variable;&#xA;    unlink statefreq=(all) revmat=(all) shape=(all) pinvar=(all) tratio=(all);&#xA;    mcmc nchains = 4 ngen = 10000000 samplefreq = 10000 printfreq = 10000 diagnfreq = 10000;&#xA;    sump;&#xA;    sumt;&#xA;    quit;&#xA;end;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8757" LastActivityDate="2021-03-09T04:37:24.590" Title="Mrbayes stop at certain generation without error" Tags="|phylogenetics|phylogeny|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15504" PostTypeId="1" CreationDate="2021-03-09T12:09:52.430" Score="1" ViewCount="718" Body="&lt;p&gt;I have a text file with a list of name of genes (alias names) like this (&lt;code&gt;**&lt;/code&gt; added for emphasis):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Solyc00g019980&#xA;**Solyc01g005020**&#xA;.. &#xA;.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And i have a gff file like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SL4.0ch01   maker_ITAG  gene    46201   59605   .   -   .   ID=gene:Solyc01g005020.3;Alias=**Solyc01g005020**;Name=Solyc01g005020.3;length=13404&#xA;SL4.0ch01   maker_ITAG  mRNA    46201   59605   .   -   .   ID=mRNA:Solyc01g005020.3.1;Parent=gene:Solyc01g005020.3;Name=Solyc01g005020.3.1;Note=Salt Overly Sensitive 1;_AED=0.25;_QI=241|1|1|1|0|0|23|311|1151;_eAED=0.25;**Ontology_term=GO:0006812,GO:0015299,GO:0016021,GO:0055085**&#xA;SL4.0ch01   maker_ITAG  three_prime_UTR 46201   46511   .   -   .   ID=three_prime_UTR:Solyc01g005020.3.1.0;Parent=mRNA:Solyc01g005020.3.1&#xA;SL4.0ch01   maker_ITAG  exon    46201   46844   .   -   .   ID=exon:Solyc01g005020.3.1.1;Parent=mRNA:Solyc01g005020.3.1&#xA;SL4.0ch01   maker_ITAG  CDS 46512   46844   .   -   0   ID=CDS:Solyc01g005020.3.1.1;Parent=mRNA:Solyc01g005020.3.1&#xA;SL4.0ch01   maker_ITAG  exon    47281   47585   .   -   .   ID=exon:Solyc01g005020.3.1.2;Parent=mRNA:Solyc01g005020.3.1&#xA;SL4.0ch01   maker_ITAG  CDS 47281   47585   .   -   2   ID=CDS:Solyc01g005020.3.1.2;Parent=mRNA:Solyc01g005020.3.1&#xA;SL4.0ch01   maker_ITAG  exon    48276   48543   .   -   .   ID=exon:Solyc01g005020.3.1.3;Parent=mRNA:Solyc01g005020.3.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to grab only the ones i have in my list gene text file (Alias names) and the corresponding &amp;quot;Ontology_term&amp;quot; next to it.&lt;/p&gt;&#xA;&lt;p&gt;Terminal is more convenient if there is a solution like grep or something.&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;&lt;p&gt;edit:&#xA;i would like to get this output, as an example&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Solyc01g005070  Ontology_term=GO:0005543&#xA;Solyc01g005230  Ontology_term=GO:0008168&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;i dont mind if this is included &amp;quot;Ontology_term=&amp;quot;, i would prefer not to be included though&lt;/p&gt;&#xA;&lt;p&gt;thanks again&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;grep -f list.txt data.gff&lt;/code&gt; doesn't work for me, because it grabs all the other as well.&lt;/p&gt;&#xA;" OwnerUserId="7172" LastEditorUserId="73" LastEditDate="2021-06-07T02:17:44.637" LastActivityDate="2024-03-06T10:15:32.787" Title="How to grep with a list of genes from a gff the GO terms" Tags="|grep|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15506" PostTypeId="1" CreationDate="2021-03-09T15:21:54.610" Score="0" ViewCount="134" Body="&lt;p&gt;Using Crowell 52.  When I do a glob to get matching files, Cromwell places a copy of each matching file in the glob-* directory.&lt;/p&gt;&#xA;&lt;p&gt;As these are multi-gigabyte Bam files, I'd really prefer for Cromwell to store soft links to the files, rather than full copies.&lt;/p&gt;&#xA;&lt;p&gt;The code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   output {&#xA;      Array[File] Outputs = glob(&amp;quot;*.fastq.gz&amp;quot;)&#xA;   }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is it possible to make this happen?&lt;/p&gt;&#xA;" OwnerUserId="9222" LastEditorUserId="9222" LastEditDate="2021-03-10T15:13:14.267" LastActivityDate="2023-08-08T19:12:24.533" Title="How do I keep a glob from copying a file in Cromwell / WDL" Tags="|wdl|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15507" PostTypeId="1" AcceptedAnswerId="15512" CreationDate="2021-03-09T20:15:22.070" Score="1" ViewCount="329" Body="&lt;p&gt;I came across references to &amp;quot;movie files&amp;quot; from PacBio sequencing in this paper:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.jimmunol.org/content/204/12/3434&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.jimmunol.org/content/204/12/3434&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Specifically:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Movie files used to generate results presented in this article have been submitted to the National Center for Biotechnology Information BioProject (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/bioproject/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/bioproject/&lt;/a&gt;) under accession number PRJNA389440.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Looking at the SRA entries, it looks like it's just the raw reads as I expected.  &lt;strong&gt;But what is meant by &amp;quot;movie files&amp;quot;?&lt;/strong&gt; (This is probably just my ignorance about PacBio as opposed to Illumina or others but I've never seen that term before and can't find much online.)&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2021-03-11T00:34:31.810" Title="What is a PacBio &quot;movie file&quot;?" Tags="|ngs|long-reads|pacbio|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15509" PostTypeId="1" AcceptedAnswerId="15510" CreationDate="2021-03-09T21:49:17.260" Score="1" ViewCount="95" Body="&lt;p&gt;I have two files master file.txt and sample.txt&lt;/p&gt;&#xA;&lt;p&gt;master.txt&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Name    GTEX.1117F.3226.SM.5N9CT    GTEX.111FC.3126.SM.5GZZ2    GTEX.111FC.3326.SM.5GZYV    GTEX.1128S.2726.SM.5H12C    GTEX.1128S.2826.SM.5N9DI    GTEX.117XS.3026.SM.5N9CA&#xA;ENSG00000223972 1   0   0   1   0   0&#xA;ENSG00000227232 298 168 197 106 221 184&#xA;ENSG00000278267 0   1   1   0   0   0&#xA;ENSG00000243485 0   2   0   1   0   1&#xA;ENSG00000237613 0   0   0   1   0   1&#xA;ENSG00000268020 1   0   0   1   0   1&#xA;ENSG00000240361 1   3   0   4   2   1&#xA;ENSG00000186092 3   1   1   1   1   2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Sample file.txt&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GTEX.1117F&#xA;GTEX.111FC  &#xA;GTEX.111XS&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Desired output&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Name    GTEX.1117F.3226.SM.5N9CT    GTEX.111FC.3126.SM.5GZZ2    GTEX.111FC.3326.SM.5GZYV    GTEX.117XS.3026.SM.5N9CA&#xA;ENSG00000223972 1   0   0   0&#xA;ENSG00000227232 298 168 197 184&#xA;ENSG00000278267 0   1   1   0&#xA;ENSG00000243485 0   2   0   1&#xA;ENSG00000237613 0   0   0   1&#xA;ENSG00000268020 1   0   0   1&#xA;ENSG00000240361 1   3   0   1&#xA;ENSG00000186092 3   1   1   2&#xA;ENSG00000238009 2   2   1   1&#xA;ENSG00000233750 0   3   3   0&#xA;ENSG00000268903 103 44  76  24&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried using the R-code form this link&#xA;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/15487/print-specific-columns-in-a-matrix-on-the-basis-of-sample-ids-in-the-header&quot;&gt;Print specific columns in a matrix on the basis of sample id&amp;#39;s in the header&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, I am not able to figure out how to extract columns matching only half part of ID (like for GTEX.111FC I want both GTEX.111FC.3126.SM.5GZZ2 and   GTEX.111FC.3326.SM.5GZYV)&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="4783" LastEditorUserId="4783" LastEditDate="2021-03-09T21:57:21.443" LastActivityDate="2021-03-09T22:40:52.877" Title="How to extract column from a matrix matching the another file (sample file)" Tags="|r|data-preprocessing|gtex|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15510" PostTypeId="2" ParentId="15509" CreationDate="2021-03-09T22:40:52.877" Score="1" Body="&lt;p&gt;Assuming your &amp;quot;master&amp;quot; files looks like this:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;df &amp;lt;- data.frame(Name = LETTERS[1:5],&#xA;                 Col1 = sample(1:10,5),&#xA;                 Col2 = sample(1:10,5),&#xA;                 Col11 = sample(1:10,5),&#xA;                 Col3 = sample(1:10,5))&#xA;&#xA;  Name Col1 Col2 Col11 Col3&#xA;1    A    5    4     3   10&#xA;2    B    1    6     5    6&#xA;3    C    3    1     9    4&#xA;4    D    9    9     8    9&#xA;5    E    8   10     1    1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For a sequence of pattern you want to extract, you can use base r function such as &lt;code&gt;grep&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;match_pat = c(&amp;quot;Col1&amp;quot;,&amp;quot;Col2&amp;quot;)&#xA;&#xA;m &amp;lt;- unlist(sapply(match_pat,function(x) grep(x,colnames(df))))&#xA;&#xA;Col11 Col12  Col2 &#xA;    2     4     3 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And then, extract your columns of interest by doing:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; df[,m]&#xA;  Col1 Col11 Col2&#xA;1    5     3    4&#xA;2    1     5    6&#xA;3    3     9    1&#xA;4    9     8    9&#xA;5    8     1   10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In your example, it will be something like:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;match_pat &amp;lt;- c(&amp;quot;GTEX.1117F&amp;quot;,&amp;quot;GTEX.111FC&amp;quot;,&amp;quot;GTEX.111XS&amp;quot;)&#xA;m &amp;lt;- unlist(sapply(match_pat,function(x) grep(x,colnames(df))))&#xA;&#xA;master[,m]&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6437" LastActivityDate="2021-03-09T22:40:52.877" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="15511" PostTypeId="1" CreationDate="2021-03-10T06:59:25.470" Score="-1" ViewCount="133" Body="&lt;p&gt;I tried running the following command&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/public/home/WHC_BBT/Softwares/bin/bin/denovo_map.pl --samples &lt;span class=&quot;math-container&quot;&gt;$RawData -O $&lt;/span&gt;Popmap -o &lt;span class=&quot;math-container&quot;&gt;$fullM2n3 -M 2 -n 3 -m 3 -T 15 &amp;amp;&amp;gt; $&lt;/span&gt;logfile&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the error I got&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;/opt/gridview//pbs/dispatcher/mom_priv/jobs/14923.admin.SC: line 6:&#xA;$logfile: ambiguous redirect&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I need help on how to resolve this.&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="9884" LastEditorUserId="3694" LastEditDate="2021-03-10T19:07:17.523" LastActivityDate="2021-09-10T05:04:10.420" Title="Denovo, Stacks: Getting an “ambiguous redirect” error" Tags="|snp|assembly|radseq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15512" PostTypeId="2" ParentId="15507" CreationDate="2021-03-10T07:23:59.967" Score="1" Body="&lt;p&gt;[disclaimer: not a PacBio person, I'm just vaguely aware of the technology]&lt;/p&gt;&#xA;&lt;p&gt;Movie files are the PacBio raw data format, representing observed signal intensities in each sequencing well over the course of the sequencing run. The movie files can be used to re-call sequences with updated statistical models, which can improve accuracy, or test out hypotheses about how epigenetics alters the rate of nucleotide incorporation (i.e. methylation).&lt;/p&gt;&#xA;&lt;p&gt;Illumina has a similar static version of this which is an image (or multiple images, depending on chemistry) of the flow cell after each synthesis cycle. Illumina's raw data format consumes tens of terabytes per run, and is of minimal value after base calling, so is almost always discarded after base calling.&lt;/p&gt;&#xA;&lt;p&gt;Anything uploaded to NCBI SRA must contain FASTQ information. While the original uploaded files can be kept (which may include time/signal information), the files are converted into a FASTQ-like format for NCBI's internal representation. As such, there's a chance that movie files could be uploaded to SRA, but those files would &lt;em&gt;also&lt;/em&gt; need to include the called FASTQ information (e.g. in a HDF5 container). In contrast to this, ENA &lt;em&gt;does&lt;/em&gt; allow uploading raw data files without called information.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-03-11T00:34:31.810" LastActivityDate="2021-03-11T00:34:31.810" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15513" PostTypeId="2" ParentId="15391" CreationDate="2021-03-10T08:20:08.960" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Check for mislabelled samples&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;For the mismatch you can check if there is a marker gene for these 3 samples for that particular cohort/tissue that you actually think should be expressed in this tissue only and not in other cohort.&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;strong&gt;Check for outliers&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;To check the logs of mapping reads of the samples that aren't clustering within expected tissue it might be due to low reads. You can use hierarchical clustering and project it in the form of a heatmap. Check if these two samples have varying expression levels to its corresponding biological replicates.&lt;/p&gt;&#xA;" OwnerUserId="10470" LastActivityDate="2021-03-10T08:20:08.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15514" PostTypeId="2" ParentId="15511" CreationDate="2021-03-10T08:33:28.227" Score="1" Body="&lt;p&gt;This most likely means your variable &lt;code&gt;$logfile&lt;/code&gt; does not exist.&lt;/p&gt;&#xA;" OwnerUserId="3818" LastActivityDate="2021-03-10T08:33:28.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15515" PostTypeId="1" CreationDate="2021-03-10T08:49:32.280" Score="2" ViewCount="654" Body="&lt;p&gt;I have (gave me) Illumina fastq files, which I want to use for variant calling, and I do not know if they are DNAseq or RNAseq data. How can I check this? I do not have any report or who to ask.&lt;/p&gt;&#xA;&lt;p&gt;Many thanks&lt;/p&gt;&#xA;" OwnerUserId="10712" LastEditorUserId="73" LastEditDate="2021-05-09T05:00:05.387" LastActivityDate="2021-05-09T05:00:05.387" Title="Fastq: how can I check if they are from DNA or RNAseq data?" Tags="|rna-seq|phylogenetics|fastq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15516" PostTypeId="1" CreationDate="2021-03-10T11:06:16.057" Score="0" ViewCount="251" Body="&lt;p&gt;I have RNA seq data and I need to use combat to remove the batch effects. Somehow when I run it, it isnt actually doing anything.&lt;/p&gt;&#xA;&lt;p&gt;The code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeq(dds, betaPrior=TRUE)&#xA;normalized_counts &amp;lt;- counts(dds, normalized=TRUE)&#xA;log2 = log2(normalized_counts+1)&#xA;&#xA;modcombat = model.matrix(~dex, metadata)# - with metadata being a variable containing treatment or control (under dex column) and batch and name of each patient.&#xA;&#xA;com&amp;lt;-ComBat(log2, metadata$Batch, mod = modcombat)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Its supposed to be 4 different batches, but in the com variable I can see that the values have stayed the same as in log2.&lt;/p&gt;&#xA;&lt;p&gt;What could be wrong? Would appreciate any help!&lt;/p&gt;&#xA;" OwnerUserId="10713" LastEditorUserId="3694" LastEditDate="2021-03-10T15:47:05.093" LastActivityDate="2021-03-10T15:47:05.093" Title="How to use combat in order to remove batch effects?" Tags="|rna-seq|batch-effects|combat|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15517" PostTypeId="2" ParentId="15515" CreationDate="2021-03-10T12:30:37.683" Score="7" Body="&lt;p&gt;There are a few quick'n'dirty ways depending on the type of data.&#xA;In any case you want to align your files to a reference genome and then check the distribution of reads, either on a genome browser or with tools such as &lt;a href=&quot;http://rseqc.sourceforge.net/#read-distribution-py&quot; rel=&quot;nofollow noreferrer&quot;&gt;RSEQC&lt;/a&gt;&#xA;which calcualtes the fraction of reads aligning to exon, intron, intergenic etc.&lt;/p&gt;&#xA;&lt;p&gt;RNA-seq, if you use a standard aligner such as bowtie2 or bwa, and you have a higher species that splices its RNA, then you should see most reads aligning to exons, and quite a fraction of reads that span a splice junction being unmapped (because the exon-intron-exon gap is large and non splice-aware tools do not &amp;quot;bridge&amp;quot; this without tremendous drop in mapping quality so the reads goes unmapped).&lt;/p&gt;&#xA;&lt;p&gt;DNA-seq (if whole genome) should have a somewhat even coverage across the genome on the global scale. Targeted assays such as ATAC-seq should give distinct peaks that you can easily see in a genome browser just by eye.&lt;/p&gt;&#xA;&lt;p&gt;If it is exome-seq (so targeted DNA-seq via exon pulldown) then it might get tricky because also most reads (like in RNA-seq) should align to exons, but you should not have spliced reads so the 5' part of the read starting in exon1 and 3' part ending in exon2. Instead many reads should also cover the exon-intron boundary.&lt;/p&gt;&#xA;&lt;p&gt;As said, I would simply align it and check on a browser, I could imagine one can make a sophisticated guess by just looking at the alignments by eye.&lt;/p&gt;&#xA;&lt;p&gt;A different topic is reliability, because I personally would not touch a dataset with unknown origin, because that also means you have no idea about the protocols that were used for library prep and what the sample actually is, so I wonder which question your analysis is going to answer.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2021-03-10T19:35:53.963" LastActivityDate="2021-03-10T19:35:53.963" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15521" PostTypeId="2" ParentId="6871" CreationDate="2021-03-10T15:32:56.317" Score="3" Body="&lt;h2&gt;Updated&lt;/h2&gt;&#xA;&lt;p&gt;As of 2021, the &lt;a href=&quot;https://alphafold.ebi.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;EBI alphaFold2&lt;/a&gt; is the best resource. There are some caveats, which I have outlined in &lt;a href=&quot;https://blog.matteoferla.com/2021/07/what-to-look-out-for-with-alphafold2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog post&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;But majorly, it contains only monomeric predictions and not oligomers and complexes. As a result, for protein with a close homologue with a solved structure, it may be more handy to use a threaded structure as found in the SwissModel database (see original answer below) because it may be oligomeric and/or one can copy over ligands or binding partners if deemed suitable.&lt;/p&gt;&#xA;&lt;p&gt;One can make one's own predictions with AlphaFold2, who have released a handy notebook. Also one can make complexes and oligomers with some tweaks or with the heavily improved notebooks found in &lt;a href=&quot;http://ColabFold%20GitHub%20repo&quot; rel=&quot;nofollow noreferrer&quot;&gt;ColabFold GitHub repo&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Original Answer&lt;/h2&gt;&#xA;&lt;p&gt;It is worth saying that for model organisms, Expasy &lt;a href=&quot;https://swissmodel.expasy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Swissmodel&lt;/a&gt; is the &amp;quot;classical&amp;quot; database of predicted proteome structures. It has several model organisms and gets updates regularly. The models are threaded only and the algorithm fell behind some time ago —their website however is modern and sleek. If using Rosetta afterwards, one should do a few cycles of FastRelax with a movemap preventing backbone movements or CA restrained, followed by a couple of cartesian bond/angle fixing cycles: the structure tends to blow up if energy minimised otherwise.&lt;/p&gt;&#xA;&lt;p&gt;However, if one had a specific protein in mind predicting with RosettaCM, the I-Tasser server or Phyre server is by the recommended option. And actually, the Phyre server has a &lt;a href=&quot;http://www.sbg.bio.ic.ac.uk/%7Emissense3d/dataset.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;dataset of the precomputed human proteome&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;About the PDB, the database has been purged of all &lt;em&gt;in&lt;/em&gt; &lt;em&gt;silico&lt;/em&gt; models and contains only empirical structures (NMR, X-ray, neutron diffraction and cryo-EM).&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2021-08-21T18:28:42.523" LastActivityDate="2021-08-21T18:28:42.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15522" PostTypeId="2" ParentId="5660" CreationDate="2021-03-10T17:43:00.800" Score="2" Body="&lt;p&gt;When studying demographics, you often need to account for the population size of a city or region (GDP per capita, is a good example).&#xA;Normalizing by &lt;span class=&quot;math-container&quot;&gt;$10^6$&lt;/span&gt; people is a nice round number, but lots of cities and regions don't have that many people.&#xA;So if we want to keep numbers in the range [1, 100], 1M is too big a denominator.&#xA;That's why a lot of studies use &lt;span class=&quot;math-container&quot;&gt;$10^5$&lt;/span&gt; people as a scaling factor.&lt;/p&gt;&#xA;&lt;p&gt;The same goes for RNA-seq and other sequencing measurements.&#xA;Your readout will typically contain &lt;span class=&quot;math-container&quot;&gt;$10^7 - 10^8$&lt;/span&gt; reads for a bulk sequencing run.&#xA;If your sample has thousands of cells, this means very few cells will have &amp;gt; &lt;span class=&quot;math-container&quot;&gt;$10^6$&lt;/span&gt; reads assigned to it.&lt;/p&gt;&#xA;&lt;p&gt;But the TPM unit was designed for bulk measurements and that's what most people are used to.&#xA;So we can keep the same idea of scaling by number of reads, but just use a different convenient scaling factor.&#xA;This way we get both scaling by sequencing depth and nice numbers to work with.&lt;/p&gt;&#xA;&lt;p&gt;As for your &lt;span class=&quot;math-container&quot;&gt;$\log(1+x)$&lt;/span&gt; comment, yes, if &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is in TP10K, the low read counts will be biased more than if &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; was in units of TPM.&#xA;But consistently lowly-expressed genes often aren't of interest.&#xA;So this won't be too much of a problem.&#xA;The bigger issue is the zero count over-inflation that is common in scRNA-seq measurements.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2021-03-10T17:43:00.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15523" PostTypeId="2" ParentId="7096" CreationDate="2021-03-10T17:49:14.883" Score="5" Body="&lt;h3&gt;How to re-analyze 10X BAM files?&lt;/h3&gt;&#xA;&lt;p&gt;This is a great question and honestly, I don't think there was an easy way to do this at the time the question was asked. The reason is that if you want to re-do the authors' analysis to get the gene expression (gene-cell / feature-barcode) matrix and they used &lt;code&gt;cellranger&lt;/code&gt; (the official 10X pipeline software) to do that, you will need FASTQ files as input - because &amp;quot;10x pipelines require sequencer FASTQs (with embedded barcodes) as input.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Therefore, to get FASTQ files from 10X BAM files, there's a relatively new tool provided by 10X Genomics: &lt;a href=&quot;https://support.10xgenomics.com/docs/bamtofastq&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;bamtofastq&lt;/code&gt;&lt;/a&gt;. Then you can simply use those FASTQ files as input to &lt;code&gt;cellranger&lt;/code&gt;.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;More info here: &lt;a href=&quot;https://support.10xgenomics.com/docs/bamtofastq&quot; rel=&quot;noreferrer&quot;&gt;https://support.10xgenomics.com/docs/bamtofastq&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h3&gt;How to correctly download 10X BAM files from NCBI?&lt;/h3&gt;&#xA;&lt;p&gt;The only caveat is that you need a BAM file generated directly by 10X's &lt;code&gt;cellranger&lt;/code&gt; (or the respective 10X pipeline, if not dealing with gene expression) - that means that a BAM file obtained by downloading an SRA from NCBI and converting to BAM won't work; you need to get the original BAM file directly (often found among the originally submitted files, under &amp;quot;Original format&amp;quot;).&lt;/p&gt;&#xA;&lt;p&gt;To give you some additional help here (because figuring this out is also not trivial): an original 10X BAM file (if originally submitted in this format) can be downloaded from NCBI using &lt;code&gt;prefetch&lt;/code&gt; from &lt;a href=&quot;https://github.com/ncbi/sra-tools&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;sra-tools&lt;/code&gt;&lt;/a&gt;, specifying the &lt;code&gt;--type&lt;/code&gt; option - for example:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;prefetch --type TenX --max-size 100000000 SRR5167880&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Because the file was larger than the default limit of 20GB, I also had to increase it by specifying the &lt;code&gt;--max-size&lt;/code&gt; option.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, you can follow the official 10X &lt;a href=&quot;https://support.10xgenomics.com/permalink/2CBLaOKISoo2m0eYAkcsaC&quot; rel=&quot;noreferrer&quot;&gt;Instructions to Download and Process BAM files of 1.3 Million Brain Cells&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;I know this is an old thread but I'm hoping my answer will be helpful to anyone who has the same question (like me).&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2021-03-10T17:49:14.883" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15525" PostTypeId="2" ParentId="9229" CreationDate="2021-03-10T18:10:51.370" Score="0" Body="&lt;p&gt;Edited version of &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/7230/namshin-kim&quot;&gt;Namshin's&lt;/a&gt; code this can handle if REF or ALT is '-' ( these records in vcf file are observed when there is insertion/deletion).&#xA;Added documentation in the script to explain code block I have modified.&#xA;Hope this useful.&#xA;Tested example with GRCh37&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;python fix_vcf.py -f genome.fa -o out_fixed.vcf input.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;example input.vcf:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.2&#xA;#CHROM  POS ID  REF ALT QUAL  FILTER  INFO&#xA;7 6027157 . - T 1 PASS  NS=1  GQ  10&#xA;9 98218616  . - C 1 PASS  NS=1  GQ  10&#xA;17  41245200  . AT  - 1 PASS  NS=1  GQ  10&#xA;13  32929213  . C - 1 PASS  NS=1  GQ  10&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;out_fixed.vcf:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.2&#xA;##FILTER=&amp;lt;ID=PASS,Description=&amp;quot;All filters passed&amp;quot;&amp;gt;&#xA;#CHROM  POS ID  REF ALT QUAL  FILTER  INFO&#xA;7 6027157 . T TT  1 PASS  NS=1&#xA;9 98218616  . A AC  1 PASS  NS=1&#xA;17  41245199  . GAT G 1 PASS  NS=1&#xA;13  32929212  . CC  C 1 PASS  NS=1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;# -*- coding: utf-8 -*-&#xA;import argparse&#xA;import sys&#xA;&#xA;import pysam&#xA;&#xA;&#xA;class Application:&#xA;    def __init__(self):&#xA;        self.args = Application.get_args()&#xA;        self.vcf = pysam.VariantFile(sys.stdin)&#xA;        self.fasta = pysam.FastaFile(self.args.fasta)&#xA;&#xA;    @staticmethod&#xA;    def get_args() -&amp;gt; argparse.Namespace:&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        Read command line arguments and set input and output sources.&#xA;&#xA;        :return: parsed command line arguments&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;        parser = argparse.ArgumentParser(prog=&amp;quot;fixvcf.py&amp;quot;, description=&amp;quot;Fixes deletion represented by -&amp;quot;)&#xA;        parser.add_argument('--version', action='version', version='%(prog)s 0.1')&#xA;&#xA;        parser.add_argument(&amp;quot;input&amp;quot;, help=&amp;quot;input file, use - to read from stdin&amp;quot;)&#xA;        parser.add_argument(&amp;quot;-f&amp;quot;, &amp;quot;--fasta&amp;quot;, help=&amp;quot;fasta reference file&amp;quot;, required=True)&#xA;        parser.add_argument(&amp;quot;-o&amp;quot;, &amp;quot;--output&amp;quot;, help=&amp;quot;output file&amp;quot;)&#xA;&#xA;        if len(sys.argv) == 1:&#xA;            parser.print_help(sys.stderr)&#xA;            sys.exit(1)&#xA;&#xA;        args = parser.parse_args()&#xA;&#xA;        if args.output:&#xA;            sys.stdout = open(args.output, &amp;quot;w&amp;quot;)&#xA;&#xA;        if args.input != &amp;quot;-&amp;quot;:&#xA;            sys.stdin = open(args.input, &amp;quot;r&amp;quot;)&#xA;&#xA;        return args&#xA;&#xA;    def fix_record(self, record: pysam.libcbcf.VariantRecord) -&amp;gt;     pysam.libcbcf.VariantRecord:&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        If the ALT column of the records contains a `-`, the variant position is shift one position to the left.&#xA;        For reaching this the reference base for the position is determined and prepended to the current REF bases.&#xA;        The ALT value `-` is replaced with the determined ref base on position before. In case there are multiple ALT&#xA;        values the determined ref base is prepended to them.&#xA;&#xA;        If the REF column of the records contains a `-`, the variant position will change.&#xA;        For reaching this the reference base for the position is determined and prepended to the current ALT bases.&#xA;        The REF value `-` is replaced with the determined ref base on position. In case there are multiple ALT&#xA;        values the determined ref base is prepended to them.&#xA;&#xA;        :param record: vcf file record&#xA;        :return: fixed vcf record&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;        if &amp;quot;-&amp;quot; in record.alts:&#xA;            ref = self.fasta.fetch(reference=record.chrom, start=record.pos - 2, end=record.pos -1)&#xA;&#xA;            record.pos = record.pos - 1&#xA;            record.ref = f&amp;quot;{ref}{record.ref}&amp;quot;&#xA;            record.alts = tuple(map(lambda alt: ref if alt == &amp;quot;-&amp;quot; else f&amp;quot;{ref}{alt}&amp;quot;, record.alts))&#xA;&#xA;        if &amp;quot;-&amp;quot; in record.ref:&#xA;            ref = self.fasta.fetch(reference=record.chrom, start=record.pos - 1, end=record.pos)&#xA;&#xA;            record.pos = record.pos&#xA;            record.ref = f&amp;quot;{ref}&amp;quot;&#xA;            record.alts = tuple(map(lambda alt: f&amp;quot;{ref}{alt}&amp;quot;, record.alts))&#xA;&#xA;        return record&#xA;&#xA;    def start(self):&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        Print the header of the vcf file. Afterwards the fix_record method is applied to each record in the vcf file.&#xA;&#xA;        For more information about python's map() see: &#xA;https://docs.python.org/3.6/library/functions.html#map&#xA;&#xA;        For more information about the unpacking mechanism using `*`see:&#xA;        https://docs.python.org/3/tutorial/controlflow.html#tut-unpacking-arguments&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;        print(self.vcf.header, end=&amp;quot;&amp;quot;)&#xA;        #print(*map(self.fix_record, self.vcf), sep=&amp;quot;&amp;quot;, end=&amp;quot;&amp;quot;)&#xA;        for line in self.vcf:&#xA;            print(self.fix_record(line), sep=&amp;quot;&amp;quot;, end=&amp;quot;&amp;quot;)&#xA;&#xA;if __name__ == &amp;quot;__main__&amp;quot;:&#xA;    app = Application()&#xA;    app.start()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10716" LastEditorUserId="10716" LastEditDate="2021-03-11T12:42:28.637" LastActivityDate="2021-03-11T12:42:28.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15527" PostTypeId="1" CreationDate="2021-03-10T18:30:52.637" Score="1" ViewCount="77" Body="&lt;p&gt;I am looking at using CD-HIT to efficiently &lt;strong&gt;filter my protein sequences dataset by a similarity threshold of 70% (cut-off).&lt;/strong&gt; More precisely, what I want to achieve is that for all the remaining sequences after filtering, all pairwise sequence similarity scores that can be computed are less than 70%.&lt;/p&gt;&#xA;&lt;p&gt;Does that mean that I &lt;strong&gt;should run CD-HIT on my dataset with similarity threshold=70%&lt;/strong&gt; (rest of the settings at default values), and &lt;strong&gt;then just keep only the &amp;quot;representative&amp;quot; sequences from the resulting cluster files&lt;/strong&gt;? I have read the guide (&lt;a href=&quot;http://weizhongli-lab.org/lab-wiki/doku.php?id=cd-hit-user-guide&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://weizhongli-lab.org/lab-wiki/doku.php?id=cd-hit-user-guide&lt;/a&gt;) but I am still not sure of how to use the output.&lt;/p&gt;&#xA;&lt;p&gt;Linking to the above, how would CD-HIT handle this scenario below or similar ones like this:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;assuming seq. similarity threshold=70%&lt;/li&gt;&#xA;&lt;li&gt;sim.(seqA, seqB) = 80%&lt;/li&gt;&#xA;&lt;li&gt;sim.(seqA,seqC) = 90%&lt;/li&gt;&#xA;&lt;li&gt;sim.(seqA,seqD) = 65%&lt;/li&gt;&#xA;&lt;li&gt;sim.(seqA, seqE) = 60%&lt;/li&gt;&#xA;&lt;li&gt;sim.(seqD, seqE) = 50%&lt;/li&gt;&#xA;&lt;li&gt;=&amp;gt; here, would the method keep just seqE, seqD in the output as sequences satisfying the threshold?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="10717" LastActivityDate="2021-03-10T18:30:52.637" Title="How to use CD-HIT to filter protein sequences dataset by certain similarity threshold?" Tags="|clustering|sequence|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15528" PostTypeId="1" AcceptedAnswerId="15564" CreationDate="2021-03-10T19:41:35.083" Score="1" ViewCount="191" Body="&lt;p&gt;in test.nf (dsl2) I have&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   ... &#xA;   workflow{&#xA;       a = Channel...&#xA;       processA(a)&#xA;       processB(processA.out.emitA)&#xA;       processC(processB.out.emitB)&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The pipeline completed successfully.&lt;/p&gt;&#xA;&lt;p&gt;Now I want to add &lt;code&gt;processC_2&lt;/code&gt; that takes the output of processB.&lt;/p&gt;&#xA;&lt;p&gt;How do I run the workflow to bypass processA execution?&lt;/p&gt;&#xA;&lt;p&gt;I added processC_2 to test.nf and then did:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;nextflow run test.nf -resume run-name&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But it doesn't run the newly add process.&lt;/p&gt;&#xA;&lt;p&gt;How do I achieve my goal?&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="2970" LastActivityDate="2021-03-17T13:13:05.067" Title="add task to nextflow and resume" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15529" PostTypeId="2" ParentId="15515" CreationDate="2021-03-10T20:33:53.050" Score="4" Body="&lt;p&gt;Honestly, ask who made them.  If the reads are from an organism without splicing, it will not be trivial to figure it out with 100% certainty.&lt;/p&gt;&#xA;&lt;p&gt;If you are calling variants, I don't think it matters much, except that higher organisms have such a small % of their genome transcribed, it should be obvious very quickly if you have the desired coverage across the entire genome, or just the transcripts.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2021-03-10T20:33:53.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15530" PostTypeId="1" CreationDate="2021-03-11T03:17:57.927" Score="1" ViewCount="46" Body="&lt;p&gt;The simple question is: how to store relative abundance data in a database? Which database to choose?&lt;/p&gt;&#xA;&lt;p&gt;I am talking here about metagenomics data, relative abundance of species or some features. Each feature has single abundance value. For each sample, there are 100's to 1000' features, and the number of unique features between samples keeps growing fast with new samples. Target number of samples ranging 1000-10000.&lt;/p&gt;&#xA;&lt;p&gt;The most lame way of handling that would be some big array (e.g. sampleID x featureID) but that would not be very convenient, as it will be holding mostly zeros (for features not found in given sample).&#xA;I wanted to organize it into some database, but Which one will be effective to store and query this kind of data? Until now I was working only with relational data bases (MySQL), but I guess that is not the solution here (growing number of samples and unique features). Or is it?&lt;/p&gt;&#xA;&lt;p&gt;Thanks for suggestions!&lt;/p&gt;&#xA;" OwnerUserId="10719" LastActivityDate="2021-03-11T03:17:57.927" Title="Which database is good to store (metagenomic) relative abundance data?" Tags="|database|metagenome|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15531" PostTypeId="2" ParentId="15484" CreationDate="2021-03-11T11:39:10.960" Score="0" Body="&lt;p&gt;Can you give a little more information about what tools and data sets you are using? How did you generate the OTU table etc?&lt;/p&gt;&#xA;&lt;p&gt;If the underlying data is nucleotide data and this is a phylogentic tree. It may well be a population genetic tree based for example in allele frequencies.&lt;/p&gt;&#xA;&lt;p&gt;For classical phylogenetics then FastTree is a very simple tool to generate a decent phylogeny. If you have the sequences of each OTU, you can align them in MegaX with muscle, or simply using muscle from the command line if you prefer. You can then generate the tree with this command (presuming you are on windows):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FastTree.exe -gtr -nt &amp;lt; aligned_sequences.fasta &amp;gt; otus_tree.tre &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I hope that helps!&lt;/p&gt;&#xA;" OwnerUserId="220" LastEditorUserId="3848" LastEditDate="2022-02-21T01:45:53.547" LastActivityDate="2022-02-21T01:45:53.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15532" PostTypeId="2" ParentId="11673" CreationDate="2021-03-11T18:08:07.297" Score="0" Body="&lt;p&gt;You can try &lt;a href=&quot;https://contignant.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;s-aligner&lt;/a&gt; (free for 15 days) which usually gets quite larger contigs and quite larger NG50 for metagenomic data containing viruses (also phages). Indeed metaSPAdes also gets larger contigs, despite shorter than s-aligner.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/qQKaY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qQKaY.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Ref. &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2021.02.02.429443v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;s-aligner: a greedy algorithm for non-greedy de novo genome assembly&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5190" LastActivityDate="2021-03-11T18:08:07.297" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15533" PostTypeId="2" ParentId="15391" CreationDate="2021-03-11T19:32:45.067" Score="1" Body="&lt;p&gt;Have you corrected for batch effects? This is usually a reason, as it can mean that the variation being observed from your PCA is due to other factors rather than tissue and/or treatment as you expect - of course, another can be they are incorrectly labeled !&lt;/p&gt;&#xA;&lt;p&gt;If the samples were sequenced at different times, or by different services, this can often happen. If that is the case, what you will usually find are samples being clustered based on the sequencing service, for example.&lt;/p&gt;&#xA;&lt;p&gt;Use &lt;code&gt;ComBat&lt;/code&gt; or &lt;code&gt;limma&lt;/code&gt; packages in R to fix variations in the data that may be due to this, and rerun the PCA. Best of luck.&lt;/p&gt;&#xA;" OwnerUserId="4001" LastActivityDate="2021-03-11T19:32:45.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15534" PostTypeId="1" CreationDate="2021-03-11T20:12:43.523" Score="2" ViewCount="103" Body="&lt;p&gt;I'm working with phage peptides libraries. Currently I'm analysing a massive data set from phage selection, NGS readout, preprocessed already (fasta file). I want to analyse the sequences to cluster peptides according to their consensus motifs. I found that Hammock may be helpful to do this.&lt;/p&gt;&#xA;&lt;p&gt;Here is the GitHub repo for Hammock: &lt;a href=&quot;https://github.com/krejciadam/hammock&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/krejciadam/hammock&lt;/a&gt;. I downloaded the master, unpacked it, but I don't know how to use it.&lt;/p&gt;&#xA;&lt;p&gt;I guess I need to compile, but I have no idea what files need to be compiled, how to do it, what software I need use to compile etc.&lt;/p&gt;&#xA;&lt;p&gt;I work with Mac/terminal.&lt;/p&gt;&#xA;" OwnerUserId="10722" LastEditorUserId="3694" LastEditDate="2021-04-12T12:38:51.517" LastActivityDate="2021-05-13T03:12:01.930" Title="How do I install hammock?" Tags="|clustering|software-installation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15535" PostTypeId="1" CreationDate="2021-03-12T02:38:48.907" Score="0" ViewCount="224" Body="&lt;p&gt;I am using RNA seq data to analyze genes via a volcano plot comparing differential gene expression of bacteria with and without antibiotic in R. After having created my plot, I am unsure why some of my values which read &amp;quot;0&amp;quot; TPM in one condition and a TPM value that's not &amp;quot;0&amp;quot; in the other condition were not determined to be differentially expressed. Some of the genes on my volcano plot have this difference in TPM and show up as significant on my plot while others with this &amp;quot;0&amp;quot; value difference are not considered to be significant according to my plot.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Sample of my data:&lt;/strong&gt;&#xA;(UO1_D712_##### represents the locus number, the top column represents the different replicates where &amp;quot;1&amp;quot; is untreated and &amp;quot;2&amp;quot; is treated with antibiotic, the letters next to the numbers represent a different technical replicate):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(`1` = c(&amp;quot;U01_D712_00001&amp;quot;, &#xA;&#xA;&amp;quot;U01_D712_00002&amp;quot;, &amp;quot;U01_D712_00003&amp;quot;, &#xA;&amp;quot;U01_D712_00004&amp;quot;, &amp;quot;U01_D712_00005&amp;quot;, &amp;quot;U01_D712_00006&amp;quot;, &amp;quot;U01_D712_00007&amp;quot;, &#xA;&amp;quot;U01_D712_00008&amp;quot;, &amp;quot;U01_D712_00009&amp;quot;, &amp;quot;U01_D712_00010&amp;quot;, &amp;quot;U01_D712_00011&amp;quot;, &#xA;&amp;quot;U01_D712_00012&amp;quot;, &amp;quot;U01_D712_00013&amp;quot;, &amp;quot;U01_D712_00014&amp;quot;, &amp;quot;U01_D712_00015&amp;quot;, &#xA;&amp;quot;U01_D712_00016&amp;quot;, &amp;quot;U01_D712_00017&amp;quot;, &amp;quot;U01_D712_00018&amp;quot;, &amp;quot;U01_D712_00019&amp;quot;, &#xA;&amp;quot;U01_D712_00020&amp;quot;, &amp;quot;U01_D712_00021&amp;quot;, &amp;quot;U01_D712_00022&amp;quot;, &amp;quot;U01_D712_00023&amp;quot;, &#xA;&amp;quot;U01_D712_00024&amp;quot;, &amp;quot;U01_D712_00025&amp;quot;, &amp;quot;U01_D712_00026&amp;quot;, &amp;quot;U01_D712_00027&amp;quot;, &#xA;&amp;quot;U01_D712_00028&amp;quot;, &amp;quot;U01_D712_00029&amp;quot;, &amp;quot;U01_D712_00030&amp;quot;), `1a` = c(&amp;quot;6.590456502&amp;quot;, &#xA;&amp;quot;6.741694688&amp;quot;, &amp;quot;7.110342585&amp;quot;, &amp;quot;6.299482433&amp;quot;, &amp;quot;2.77173982&amp;quot;, &amp;quot;2.470330508&amp;quot;, &#xA;&amp;quot;3.827125186&amp;quot;, &amp;quot;6.267229842&amp;quot;, &amp;quot;5.524708663&amp;quot;, &amp;quot;2.657913228&amp;quot;, &amp;quot;2.87209859&amp;quot;, &#xA;&amp;quot;7.479820548&amp;quot;, &amp;quot;4.980185572&amp;quot;, &amp;quot;4.210955199&amp;quot;, &amp;quot;0&amp;quot;, &amp;quot;4.492492822&amp;quot;, &#xA;&amp;quot;3.611091371&amp;quot;, &amp;quot;3.714270433&amp;quot;, &amp;quot;7.455036914&amp;quot;, &amp;quot;7.045203025&amp;quot;, &amp;quot;6.061860857&amp;quot;, &#xA;&amp;quot;2.925268313&amp;quot;, &amp;quot;6.544077039&amp;quot;, &amp;quot;5.747318013&amp;quot;, &amp;quot;9.97083444&amp;quot;, &amp;quot;9.22523089&amp;quot;, &#xA;&amp;quot;4.20205383&amp;quot;, &amp;quot;6.097040679&amp;quot;, &amp;quot;2.621192351&amp;quot;, &amp;quot;0&amp;quot;), `1b` = c(&amp;quot;6.544454427&amp;quot;, &#xA;&amp;quot;6.601489488&amp;quot;, &amp;quot;7.134224619&amp;quot;, &amp;quot;5.814043553&amp;quot;, &amp;quot;3.280958379&amp;quot;, &amp;quot;2.649180803&amp;quot;, &#xA;&amp;quot;3.860542083&amp;quot;, &amp;quot;6.256648363&amp;quot;, &amp;quot;5.380427766&amp;quot;, &amp;quot;2.581027705&amp;quot;, &amp;quot;3.016132165&amp;quot;, &#xA;&amp;quot;7.405329447&amp;quot;, &amp;quot;4.701503289&amp;quot;, &amp;quot;4.073814818&amp;quot;, &amp;quot;0&amp;quot;, &amp;quot;4.304196924&amp;quot;, &#xA;&amp;quot;3.515977329&amp;quot;, &amp;quot;3.843535649&amp;quot;, &amp;quot;7.342972625&amp;quot;, &amp;quot;6.966606769&amp;quot;, &amp;quot;6.122878624&amp;quot;, &#xA;&amp;quot;3.007522306&amp;quot;, &amp;quot;6.495797641&amp;quot;, &amp;quot;5.965431621&amp;quot;, &amp;quot;9.828050269&amp;quot;, &amp;quot;9.219563915&amp;quot;, &#xA;&amp;quot;4.065778989&amp;quot;, &amp;quot;6.105331066&amp;quot;, &amp;quot;3.061209408&amp;quot;, &amp;quot;0&amp;quot;), `1c` = c(&amp;quot;6.608196006&amp;quot;, &#xA;&amp;quot;6.743010138&amp;quot;, &amp;quot;7.102600793&amp;quot;, &amp;quot;6.146518601&amp;quot;, &amp;quot;3.555184202&amp;quot;, &amp;quot;2.364971542&amp;quot;, &#xA;&amp;quot;4.034053983&amp;quot;, &amp;quot;6.158523627&amp;quot;, &amp;quot;5.656051812&amp;quot;, &amp;quot;2.658660735&amp;quot;, &amp;quot;3.054717455&amp;quot;, &#xA;&amp;quot;7.392164473&amp;quot;, &amp;quot;4.950953264&amp;quot;, &amp;quot;4.277770477&amp;quot;, &amp;quot;0&amp;quot;, &amp;quot;4.507936666&amp;quot;, &#xA;&amp;quot;3.794842979&amp;quot;, &amp;quot;3.610794578&amp;quot;, &amp;quot;7.471646548&amp;quot;, &amp;quot;7.104792624&amp;quot;, &amp;quot;6.484767016&amp;quot;, &#xA;&amp;quot;3.071205184&amp;quot;, &amp;quot;6.584425715&amp;quot;, &amp;quot;6.20466015&amp;quot;, &amp;quot;9.986342122&amp;quot;, &amp;quot;9.282943758&amp;quot;, &#xA;&amp;quot;4.179958213&amp;quot;, &amp;quot;6.219551653&amp;quot;, &amp;quot;2.984738345&amp;quot;, &amp;quot;0&amp;quot;), `1d` = c(&amp;quot;6.547155382&amp;quot;, &#xA;&amp;quot;6.558328892&amp;quot;, &amp;quot;6.992501615&amp;quot;, &amp;quot;6.449558793&amp;quot;, &amp;quot;3.059801464&amp;quot;, &amp;quot;2.418800257&amp;quot;, &#xA;&amp;quot;3.96498952&amp;quot;, &amp;quot;6.013208538&amp;quot;, &amp;quot;5.279919645&amp;quot;, &amp;quot;2.893295085&amp;quot;, &amp;quot;1.750510471&amp;quot;, &#xA;&amp;quot;7.408735671&amp;quot;, &amp;quot;4.9425624&amp;quot;, &amp;quot;3.804549986&amp;quot;, &amp;quot;0&amp;quot;, &amp;quot;4.174835979&amp;quot;, &#xA;&amp;quot;3.806006888&amp;quot;, &amp;quot;3.570390524&amp;quot;, &amp;quot;7.641137006&amp;quot;, &amp;quot;6.976672494&amp;quot;, &amp;quot;6.363030106&amp;quot;, &#xA;&amp;quot;3.083061726&amp;quot;, &amp;quot;6.300910093&amp;quot;, &amp;quot;6.007490342&amp;quot;, &amp;quot;9.926316442&amp;quot;, &amp;quot;9.09671588&amp;quot;, &#xA;&amp;quot;4.320556917&amp;quot;, &amp;quot;6.153860107&amp;quot;, &amp;quot;2.877230446&amp;quot;, &amp;quot;0&amp;quot;), `1e` = c(&amp;quot;6.626724417&amp;quot;, &#xA;&amp;quot;6.577345176&amp;quot;, &amp;quot;7.156821278&amp;quot;, &amp;quot;6.296873411&amp;quot;, &amp;quot;3.618089702&amp;quot;, &amp;quot;2.394444986&amp;quot;, &#xA;&amp;quot;4.129376392&amp;quot;, &amp;quot;6.011715246&amp;quot;, &amp;quot;5.31197869&amp;quot;, &amp;quot;2.00754706&amp;quot;, &amp;quot;2.695493528&amp;quot;, &#xA;&amp;quot;7.538910448&amp;quot;, &amp;quot;4.606060035&amp;quot;, &amp;quot;3.909472643&amp;quot;, &amp;quot;0&amp;quot;, &amp;quot;4.346616047&amp;quot;, &#xA;&amp;quot;3.468681284&amp;quot;, &amp;quot;3.338231445&amp;quot;, &amp;quot;7.559599613&amp;quot;, &amp;quot;7.1527452&amp;quot;, &amp;quot;6.232923513&amp;quot;, &#xA;&amp;quot;3.108624209&amp;quot;, &amp;quot;6.535435309&amp;quot;, &amp;quot;6.12922864&amp;quot;, &amp;quot;10.13108497&amp;quot;, &amp;quot;9.310331313&amp;quot;, &#xA;&amp;quot;3.959568571&amp;quot;, &amp;quot;6.182335537&amp;quot;, &amp;quot;2.902736258&amp;quot;, &amp;quot;0&amp;quot;), `1f` = c(&amp;quot;6.419219179&amp;quot;, &#xA;&amp;quot;6.650459302&amp;quot;, &amp;quot;7.319125725&amp;quot;, &amp;quot;5.570869357&amp;quot;, &amp;quot;2.962845933&amp;quot;, &amp;quot;2.55903176&amp;quot;, &#xA;&amp;quot;4.087597573&amp;quot;, &amp;quot;5.995610538&amp;quot;, &amp;quot;5.386268651&amp;quot;, &amp;quot;2.750800859&amp;quot;, &amp;quot;2.416572678&amp;quot;, &#xA;&amp;quot;7.579148955&amp;quot;, &amp;quot;3.952633067&amp;quot;, &amp;quot;3.615227674&amp;quot;, &amp;quot;0&amp;quot;, &amp;quot;4.562838935&amp;quot;, &#xA;&amp;quot;3.76942104&amp;quot;, &amp;quot;3.920096905&amp;quot;, &amp;quot;7.935320749&amp;quot;, &amp;quot;7.501470652&amp;quot;, &amp;quot;6.13700099&amp;quot;, &#xA;&amp;quot;3.123910608&amp;quot;, &amp;quot;6.035971952&amp;quot;, &amp;quot;5.706235015&amp;quot;, &amp;quot;9.254751395&amp;quot;, &amp;quot;8.379630979&amp;quot;, &#xA;&amp;quot;4.51391973&amp;quot;, &amp;quot;5.6890651&amp;quot;, &amp;quot;2.43285316&amp;quot;, &amp;quot;0&amp;quot;), `1g` = c(&amp;quot;6.553221221&amp;quot;, &#xA;&amp;quot;6.633949852&amp;quot;, &amp;quot;7.182305386&amp;quot;, &amp;quot;5.769365973&amp;quot;, &amp;quot;2.721354972&amp;quot;, &amp;quot;1.668390466&amp;quot;, &#xA;&amp;quot;4.148367057&amp;quot;, &amp;quot;6.240883382&amp;quot;, &amp;quot;5.458877133&amp;quot;, &amp;quot;1.733842637&amp;quot;, &amp;quot;2.723803355&amp;quot;, &#xA;&amp;quot;7.522249899&amp;quot;, &amp;quot;4.149567197&amp;quot;, &amp;quot;3.780763096&amp;quot;, &amp;quot;0&amp;quot;, &amp;quot;4.496306813&amp;quot;, &#xA;&amp;quot;3.645643535&amp;quot;, &amp;quot;3.851001768&amp;quot;, &amp;quot;7.678552875&amp;quot;, &amp;quot;7.283411279&amp;quot;, &amp;quot;6.591585956&amp;quot;, &#xA;&amp;quot;2.879345378&amp;quot;, &amp;quot;6.389427003&amp;quot;, &amp;quot;5.911222165&amp;quot;, &amp;quot;9.851084493&amp;quot;, &amp;quot;9.084575304&amp;quot;, &#xA;&amp;quot;4.272587776&amp;quot;, &amp;quot;5.974762147&amp;quot;, &amp;quot;2.98852705&amp;quot;, &amp;quot;0&amp;quot;), `2a` = c(&amp;quot;9.769887737&amp;quot;, &#xA;&amp;quot;4.550226652&amp;quot;, &amp;quot;1.869021464&amp;quot;, &amp;quot;4.944848987&amp;quot;, &amp;quot;7.9678549&amp;quot;, &amp;quot;2.682865013&amp;quot;, &#xA;&amp;quot;8.495575559&amp;quot;, &amp;quot;2.234521659&amp;quot;, &amp;quot;3.667196316&amp;quot;, &amp;quot;9.180445037&amp;quot;, &amp;quot;3.210107621&amp;quot;, &#xA;&amp;quot;7.21523691&amp;quot;, &amp;quot;5.714579923&amp;quot;, &amp;quot;5.423986751&amp;quot;, &amp;quot;9.118981459&amp;quot;, &amp;quot;8.635701597&amp;quot;, &#xA;&amp;quot;2.742889473&amp;quot;, &amp;quot;3.712618983&amp;quot;, &amp;quot;8.006057144&amp;quot;, &amp;quot;4.999541279&amp;quot;, &amp;quot;10.54351774&amp;quot;, &#xA;&amp;quot;5.880978085&amp;quot;, &amp;quot;7.145433526&amp;quot;, &amp;quot;6.982416661&amp;quot;, &amp;quot;9.339651188&amp;quot;, &amp;quot;5.360835327&amp;quot;, &#xA;&amp;quot;4.699680905&amp;quot;, &amp;quot;3.423826225&amp;quot;, &amp;quot;6.408271885&amp;quot;, &amp;quot;5.038170992&amp;quot;), &#xA;    `2b` = c(&amp;quot;10.26397519&amp;quot;, &amp;quot;1.945664005&amp;quot;, &amp;quot;2.086763158&amp;quot;, &amp;quot;2.800904763&amp;quot;, &#xA;    &amp;quot;8.583418657&amp;quot;, &amp;quot;1.536094563&amp;quot;, &amp;quot;9.32057547&amp;quot;, &amp;quot;3.10685839&amp;quot;, &#xA;    &amp;quot;4.224502319&amp;quot;, &amp;quot;10.1252842&amp;quot;, &amp;quot;1.811175407&amp;quot;, &amp;quot;5.714439316&amp;quot;, &#xA;    &amp;quot;6.039142559&amp;quot;, &amp;quot;3.833361174&amp;quot;, &amp;quot;9.757360286&amp;quot;, &amp;quot;9.565906731&amp;quot;, &#xA;    &amp;quot;1.523640473&amp;quot;, &amp;quot;2.315033488&amp;quot;, &amp;quot;6.312524363&amp;quot;, &amp;quot;4.889986456&amp;quot;, &#xA;    &amp;quot;10.23020108&amp;quot;, &amp;quot;4.848727685&amp;quot;, &amp;quot;7.533256999&amp;quot;, &amp;quot;7.138160378&amp;quot;, &#xA;    &amp;quot;10.30380331&amp;quot;, &amp;quot;3.955469283&amp;quot;, &amp;quot;3.167940742&amp;quot;, &amp;quot;3.599655687&amp;quot;, &#xA;    &amp;quot;4.828945262&amp;quot;, &amp;quot;3.701043054&amp;quot;), `2c` = c(&amp;quot;9.478481216&amp;quot;, &amp;quot;2.789289131&amp;quot;, &#xA;    &amp;quot;3.393949527&amp;quot;, &amp;quot;3.754810933&amp;quot;, &amp;quot;8.710777154&amp;quot;, &amp;quot;1.806170784&amp;quot;, &#xA;    &amp;quot;9.150005253&amp;quot;, &amp;quot;2.612275457&amp;quot;, &amp;quot;4.961073313&amp;quot;, &amp;quot;9.802701699&amp;quot;, &#xA;    &amp;quot;2.933183115&amp;quot;, &amp;quot;6.532384958&amp;quot;, &amp;quot;6.919449225&amp;quot;, &amp;quot;4.432699799&amp;quot;, &#xA;    &amp;quot;9.715063475&amp;quot;, &amp;quot;9.265691356&amp;quot;, &amp;quot;1.412064593&amp;quot;, &amp;quot;3.330131873&amp;quot;, &#xA;    &amp;quot;6.665979896&amp;quot;, &amp;quot;5.158526421&amp;quot;, &amp;quot;9.417365584&amp;quot;, &amp;quot;4.899531204&amp;quot;, &#xA;    &amp;quot;8.173459354&amp;quot;, &amp;quot;7.271400938&amp;quot;, &amp;quot;9.813068613&amp;quot;, &amp;quot;4.384622077&amp;quot;, &#xA;    &amp;quot;3.700645365&amp;quot;, &amp;quot;4.457874829&amp;quot;, &amp;quot;5.649440022&amp;quot;, &amp;quot;3.531010379&amp;quot;&#xA;    ), `2d` = c(&amp;quot;8.199795497&amp;quot;, &amp;quot;2.565711524&amp;quot;, &amp;quot;2.202287889&amp;quot;, &#xA;    &amp;quot;3.856354444&amp;quot;, &amp;quot;7.380224849&amp;quot;, &amp;quot;2.192476466&amp;quot;, &amp;quot;8.14446837&amp;quot;, &#xA;    &amp;quot;1.144258862&amp;quot;, &amp;quot;3.31447122&amp;quot;, &amp;quot;8.713146629&amp;quot;, &amp;quot;2.697890381&amp;quot;, &#xA;    &amp;quot;6.304428859&amp;quot;, &amp;quot;5.745291803&amp;quot;, &amp;quot;4.898396114&amp;quot;, &amp;quot;9.173362747&amp;quot;, &#xA;    &amp;quot;8.339933849&amp;quot;, &amp;quot;3.159678152&amp;quot;, &amp;quot;4.094587234&amp;quot;, &amp;quot;7.608649692&amp;quot;, &#xA;    &amp;quot;5.280206424&amp;quot;, &amp;quot;10.34630403&amp;quot;, &amp;quot;5.098585806&amp;quot;, &amp;quot;7.262400625&amp;quot;, &#xA;    &amp;quot;6.150190905&amp;quot;, &amp;quot;9.316698845&amp;quot;, &amp;quot;5.073027993&amp;quot;, &amp;quot;4.695003229&amp;quot;, &#xA;    &amp;quot;2.485847024&amp;quot;, &amp;quot;5.545300465&amp;quot;, &amp;quot;4.350571411&amp;quot;), `2e` = c(&amp;quot;8.74935033&amp;quot;, &#xA;    &amp;quot;3.739489484&amp;quot;, &amp;quot;0.217205045&amp;quot;, &amp;quot;4.413657999&amp;quot;, &amp;quot;8.745525588&amp;quot;, &#xA;    &amp;quot;3.657060086&amp;quot;, &amp;quot;9.279834921&amp;quot;, &amp;quot;2.898951179&amp;quot;, &amp;quot;4.282874018&amp;quot;, &#xA;    &amp;quot;9.610485827&amp;quot;, &amp;quot;3.561102455&amp;quot;, &amp;quot;7.228334332&amp;quot;, &amp;quot;6.388491443&amp;quot;, &#xA;    &amp;quot;4.389908652&amp;quot;, &amp;quot;8.781086564&amp;quot;, &amp;quot;9.178866581&amp;quot;, &amp;quot;2.374603596&amp;quot;, &#xA;    &amp;quot;3.961037408&amp;quot;, &amp;quot;7.864369809&amp;quot;, &amp;quot;3.654728044&amp;quot;, &amp;quot;10.15284858&amp;quot;, &#xA;    &amp;quot;5.894439123&amp;quot;, &amp;quot;7.68020282&amp;quot;, &amp;quot;7.12243523&amp;quot;, &amp;quot;9.998637438&amp;quot;, &#xA;    &amp;quot;5.092174395&amp;quot;, &amp;quot;4.111530392&amp;quot;, &amp;quot;3.776835632&amp;quot;, &amp;quot;5.624523213&amp;quot;, &#xA;    &amp;quot;4.095011377&amp;quot;), `2f` = c(&amp;quot;7.648310926&amp;quot;, &amp;quot;4.345557215&amp;quot;, &amp;quot;1.986576876&amp;quot;, &#xA;    &amp;quot;4.99426288&amp;quot;, &amp;quot;7.087937177&amp;quot;, &amp;quot;2.810917253&amp;quot;, &amp;quot;7.77706637&amp;quot;, &#xA;    &amp;quot;2.62822773&amp;quot;, &amp;quot;3.581811188&amp;quot;, &amp;quot;8.470225989&amp;quot;, &amp;quot;3.335757437&amp;quot;, &#xA;    &amp;quot;7.416094847&amp;quot;, &amp;quot;5.208841128&amp;quot;, &amp;quot;5.536128034&amp;quot;, &amp;quot;8.255571138&amp;quot;, &#xA;    &amp;quot;7.993319997&amp;quot;, &amp;quot;1.9209089&amp;quot;, &amp;quot;3.573861828&amp;quot;, &amp;quot;7.318814519&amp;quot;, &#xA;    &amp;quot;5.233804806&amp;quot;, &amp;quot;11.05855833&amp;quot;, &amp;quot;6.247720809&amp;quot;, &amp;quot;6.673407583&amp;quot;, &#xA;    &amp;quot;6.029960625&amp;quot;, &amp;quot;8.806867591&amp;quot;, &amp;quot;5.459208493&amp;quot;, &amp;quot;4.001428729&amp;quot;, &#xA;    &amp;quot;3.609936979&amp;quot;, &amp;quot;5.876522973&amp;quot;, &amp;quot;4.652839671&amp;quot;), `2g` = c(&amp;quot;7.555235468&amp;quot;, &#xA;    &amp;quot;3.892899549&amp;quot;, &amp;quot;1.726443458&amp;quot;, &amp;quot;5.304546796&amp;quot;, &amp;quot;7.039588042&amp;quot;, &#xA;    &amp;quot;3.027235295&amp;quot;, &amp;quot;7.703852207&amp;quot;, &amp;quot;1.753183519&amp;quot;, &amp;quot;2.909288568&amp;quot;, &#xA;    &amp;quot;8.385169315&amp;quot;, &amp;quot;3.902707541&amp;quot;, &amp;quot;7.523315081&amp;quot;, &amp;quot;4.978364017&amp;quot;, &#xA;    &amp;quot;5.49103181&amp;quot;, &amp;quot;8.096218606&amp;quot;, &amp;quot;7.944822989&amp;quot;, &amp;quot;2.352609608&amp;quot;, &#xA;    &amp;quot;4.155433517&amp;quot;, &amp;quot;7.227355741&amp;quot;, &amp;quot;5.532668321&amp;quot;, &amp;quot;11.24946953&amp;quot;, &#xA;    &amp;quot;6.159185473&amp;quot;, &amp;quot;6.443203375&amp;quot;, &amp;quot;5.931761874&amp;quot;, &amp;quot;8.7421732&amp;quot;, &#xA;    &amp;quot;5.502000205&amp;quot;, &amp;quot;4.652883503&amp;quot;, &amp;quot;3.458323017&amp;quot;, &amp;quot;6.566487449&amp;quot;, &#xA;    &amp;quot;4.89790353&amp;quot;)), row.names = c(NA, 30L), class = &amp;quot;data.frame&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Design matrix:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(SampleID = c(&amp;quot;1a &amp;quot;, &amp;quot;1b&amp;quot;, &amp;quot;1c&amp;quot;, &#xA;&#xA;&amp;quot;1d&amp;quot;, &amp;quot;1e&amp;quot;, &amp;quot;1f&amp;quot;, &#xA;&amp;quot;1g&amp;quot;, &amp;quot;2a&amp;quot;, &amp;quot;2b&amp;quot;, &amp;quot;2c&amp;quot;, &amp;quot;2d&amp;quot;, &amp;quot;2e&amp;quot;, &amp;quot;2f&amp;quot;, &amp;quot;2g&amp;quot;), Group = c(1L, &#xA;1L, 1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L), Replicate = c(&amp;quot;a&amp;quot;, &#xA;&amp;quot;b&amp;quot;, &amp;quot;c&amp;quot;, &amp;quot;d&amp;quot;, &amp;quot;e&amp;quot;, &amp;quot;f&amp;quot;, &amp;quot;g&amp;quot;, &amp;quot;a&amp;quot;, &amp;quot;b&amp;quot;, &amp;quot;c&amp;quot;, &amp;quot;d&amp;quot;, &amp;quot;e&amp;quot;, &amp;quot;f&amp;quot;, &amp;quot;g&amp;quot;&#xA;)), row.names = c(NA, 14L), class = &amp;quot;data.frame&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here is how I wrote my code to sample the RNA seq data (note ~/Documents/VOLCANO/R10LB_0ugvs0.5ug.csv refers to the the sample of data above and ~/Documents/VOLCANO/R10LB_0vs0.5_designmatrix.csv refers to the above design matrix):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;R10LB_0vs0.5 &amp;lt;- read.csv2(&amp;quot;~/Documents/VOLCANO/R10LB_0ugvs0.5ug.csv&amp;quot;, sep=&amp;quot;,&amp;quot;, check.names = F)&#xA;R10LB_0vs0.5 &amp;lt;- janitor::remove_empty(R10LB_0vs0.5, which = &amp;quot;cols&amp;quot;)&#xA;R10LB_0vs0.5_design &amp;lt;- read.csv2(&amp;quot;~/Documents/VOLCANO/R10LB_0vs0.5_designmatrix.csv&amp;quot;, sep=&amp;quot;,&amp;quot;)&#xA;&#xA;rownames(R10LB_0vs0.5) &amp;lt;- R10LB_0vs0.5$`1`&#xA;R10LB_0vs0.5 &amp;lt;- R10LB_0vs0.5[,-1]&#xA;&#xA;#colnames(R10LB_0ugvs0.5ug) &amp;lt;- R10LB_0ugvs0.5ug[1,]&#xA;#R10LB_0ugvs0.5ug &amp;lt;- R10LB_0ugvs0.5ug[-1,]&#xA;&#xA;R10LB_0vs0.5 &amp;lt;- data.matrix(R10LB_0vs0.5)&#xA;&#xA;R10LB_0vs0.5 &amp;lt;- DGEList(counts = R10LB_0vs0.5, group = R10LB_0vs0.5_design$Group)&#xA;str(R10LB_0vs0.5)&#xA;&#xA;Group &amp;lt;- as.vector(as.character(R10LB_0vs0.5_design&lt;span class=&quot;math-container&quot;&gt;$Group))&#xA;Replicate &amp;lt;- as.vector((R10LB_0vs0.5_design$&lt;/span&gt;Replicate))&#xA;R10LB_0vs0.5_designmatrix &amp;lt;- model.matrix(~0+Group+Replicate)&#xA;&#xA;keep &amp;lt;- filterByExpr(R10LB_0vs0.5)&#xA;R10LB_0vs0.5_filter &amp;lt;- R10LB_0vs0.5[keep, keep.lib.sizes = FALSE]&#xA;&#xA;R10LB_0vs0.5_Disp &amp;lt;- estimateDisp(R10LB_0vs0.5_filter, R10LB_0vs0.5_designmatrix)&#xA;&#xA;CONTRASTS &amp;lt;- makeContrasts( Group1vs2 = Group1-Group2,&#xA;                            levels = R10LB_0vs0.5_designmatrix)&#xA;&#xA;R10LB_0vs0.5fit &amp;lt;- glmFit(R10LB_0vs0.5_Disp, contrast = CONTRASTS[,1])&#xA;R10LB_0vs0.5lrt &amp;lt;- glmLRT(R10LB_0vs0.5fit)&#xA;R10LB_0vs0.5TT &amp;lt;- topTags(R10LB_0vs0.5lrt, n=nrow(R10LB_0vs0.5_Disp))&#xA;write.csv(R10LB_0vs0.5TT, file = &amp;quot;R10LB_0vs0.5_comparison.csv&amp;quot;)&#xA;saveRDS(R10LB_0vs0.5TT, file = &amp;quot;R10LB_0vs0.5_comparison.RDS&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Volcano plot:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(data = R10LB_0vs0.5_comparison, aes(x = logFC, y = PValue)) + geom_point()&#xA;&#xA;p4 &amp;lt;- ggplot(data = R10LB_0vs0.5_comparison, aes(x = logFC, y = -log10(PValue))) + geom_point() + theme_minimal()&#xA;&#xA;#Add vertical lines for logFC thresholds, and one horizontal line for the p-value threshold&#xA;p5 &amp;lt;- p4 + geom_vline(xintercept = c(-0.6, 0.6), col = &amp;quot;red&amp;quot;) + geom_hline(yintercept = -log10(0.05), col = &amp;quot;red&amp;quot;)&#xA;&#xA;&#xA;# add a column of NAs&#xA;R10LB_0vs0.5_comparison&lt;span class=&quot;math-container&quot;&gt;$diffexpressed &amp;lt;- &quot;NO&quot;&#xA;# if logFC &amp;gt; 0.6 and PValue &amp;lt; 0.05, set as &quot;UP&quot; &#xA;R10LB_0vs0.5_comparison$&lt;/span&gt;diffexpressed[R10LB_0vs0.5_comparison&lt;span class=&quot;math-container&quot;&gt;$logFC &amp;gt; 0.6 &amp;amp; R10LB_0vs0.5_comparison$&lt;/span&gt;PValue &amp;lt; 0.05] &amp;lt;- &amp;quot;UP&amp;quot;&#xA;# if logFC &amp;lt; -0.6 and PValue &amp;lt; 0.05, set as &amp;quot;DOWN&amp;quot;&#xA;R10LB_0vs0.5_comparison&lt;span class=&quot;math-container&quot;&gt;$diffexpressed[R10LB_0vs0.5_comparison$&lt;/span&gt;logFC &amp;lt; -0.6 &amp;amp; R10LB_0vs0.5_comparison$PValue &amp;lt; 0.05] &amp;lt;- &amp;quot;DOWN&amp;quot;&#xA;&#xA;p4 &amp;lt;- ggplot(data=R10LB_0vs0.5_comparison, aes(x=logFC, y=-log10(PValue), col=diffexpressed)) + geom_point() + theme_minimal()&#xA;&#xA;p5 &amp;lt;- p4 + geom_vline(xintercept=c(-0.6, 0.6), col=&amp;quot;red&amp;quot;) + geom_hline(yintercept=-log10(0.05), col=&amp;quot;red&amp;quot;)&#xA;&#xA;p6 &amp;lt;- p5 + scale_color_manual(values=c(&amp;quot;blue&amp;quot;, &amp;quot;black&amp;quot;, &amp;quot;red&amp;quot;))&#xA;p6 &amp;lt;- p5 + scale_colour_manual(values = mycolors)&#xA;&#xA;R10LB_0vs0.5_comparison&lt;span class=&quot;math-container&quot;&gt;$delabel &amp;lt;- NA&#xA;R10LB_0vs0.5_comparison$&lt;/span&gt;delabel[R10LB_0vs0.5_comparison&lt;span class=&quot;math-container&quot;&gt;$diffexpressed != &quot;NO&quot;] &amp;lt;- R10LB_0vs0.5_comparison$&lt;/span&gt;gene_symbol[R10LB_0vs0.5_comparison$diffexpressed != &amp;quot;NO&amp;quot;]&#xA;&#xA;ggplot(data=R10LB_0vs0.5_comparison, aes(x=logFC, y=-log10(PValue), col=diffexpressed, label=delabel)) + geom_point() + theme_minimal() +geom_text()&#xA;&#xA;library(ggrepel)&#xA;ggplot(data=R10LB_0vs0.5_comparison, aes(x=logFC, y=-log10(PValue), col=diffexpressed, label=delabel)) +geom_point() + theme_minimal() +geom_text_repel() + scale_color_manual(values=c(&amp;quot;blue&amp;quot;, &amp;quot;black&amp;quot;, &amp;quot;red&amp;quot;)) + geom_vline(xintercept=c(-0.6, 0.6), col=&amp;quot;red&amp;quot;) +geom_hline(yintercept=-log10(0.05), col=&amp;quot;red&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the volcano plot I currently have with all of my data:&#xA;&lt;img src=&quot;https://i.stack.imgur.com/tidIM.png&quot; alt=&quot;volcano plot&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can you explain how some of these functions filtered my data (i.e. filterByExpr, estimateDisp, makeContrasts, glmFit, glmLRT)? Why are some of my data points that change from 0 TPM in one condition to some value in another condition showing up on my plot while others are not? Or should I accept that my data analysis is valid or &amp;quot;correct&amp;quot;?&lt;/p&gt;&#xA;" OwnerUserId="10720" LastEditorUserId="10720" LastEditDate="2021-03-12T20:57:06.257" LastActivityDate="2022-05-23T05:03:15.313" Title="Determining what RNAseq data is filtered on volcano plot" Tags="|r|rna-seq|bioconductor|edger|limma|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15536" PostTypeId="1" AcceptedAnswerId="15539" CreationDate="2021-03-12T03:39:13.400" Score="1" ViewCount="66" Body="&lt;p&gt;Is there a publicly available data set for humans SNPs, preferably together with the country of origin? If so, could someone please point me towards it.&lt;/p&gt;&#xA;&lt;p&gt;I've looked through other questions on the site and have been unable to find a similar one. I don't know how sensitive human SNP data is, so I'm not sure if this would be public.&lt;/p&gt;&#xA;" OwnerUserId="10725" LastActivityDate="2021-03-12T12:00:11.313" Title="Question about public availability of human SNP dataset with country of origin" Tags="|database|snp|public-databases|human-genome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15537" PostTypeId="2" ParentId="15535" CreationDate="2021-03-12T03:40:57.220" Score="0" Body="&lt;p&gt;&lt;em&gt;[disclaimer: I understand DESeq2, not EdgeR]&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Most likely, low expression values (as in raw counts, prior to any normalisation such as TPM). Expression that has high dispersion across all samples is less likely to be biologically significant.&lt;/p&gt;&#xA;&lt;p&gt;It's easier to work out what's going on with an MA plot, with average gene expression on the X axis, and log2 fold change on the Y axis.&lt;/p&gt;&#xA;&lt;p&gt;I'm used to DESeq2 results, so I'd guess the construction would be something like this (assuming that the 'baseMean' field exists in EdgeR results):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;R10LB_0vs0.5lrt %&amp;gt;%&#xA;    ggplot() +&#xA;    aes(y = logFC, col = padj, x=log(baseMean)) +&#xA;    geom_point()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It's possible that EdgeR doesn't have anything like baseMean in its default results. As a proxy for this, you could use the count totals from the input matrix (e.g. &lt;code&gt;R10LB_0vs0.5 %&amp;gt;% rowSums() %&amp;gt;% log2()&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-03-13T09:52:29.413" LastActivityDate="2021-03-13T09:52:29.413" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15538" PostTypeId="1" AcceptedAnswerId="15541" CreationDate="2021-03-12T04:10:45.773" Score="0" ViewCount="246" Body="&lt;p&gt;I have two FASTA sequences. Each is from a different species' mitochondrial genome. The sequences are pasted below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;speciesA_control_region_partial_sequence_mitochondrial&#xA;TTAAAGTGATGTAATCCTACATTAATGAAAAATCAAAAGTTATAAGAACTTAAATACATTATATCATCAAATAAATATGAAGGTAGACATAAACCACTAAACTTCAAACTCCATTAAATATGTTATAAAAATGACGATATTGAATTGTCCTATCATAACTCTCATCAGTCTAGATATACCAGGACTCACCACCTCTGCAAGTCAGAGTCAAATGTAGTAAGAGACCACCATCAGTTGATTTCTTAATGTACACGTTTATTGATGGTCAAGGACAAAAATCGTGGGGGTAGCACACAGTGAACTATTCCTGGCATTTGGTTCCTATTTCAGGTTCATCAATTGAATACATTCCCCTAACATTTCCTGACGCTTGCATAAGTTAATGGTGGAGTACATACTCCTCGTTACCCACCATGCCGAGCATTCACTCTATAGGGCTACTGGTATCTTTTTTCTATTTCCTTTCACTTGACATTTCAGAGTGCATACAGAAATGTTATATTAAGGTTGAACATTTCCTTGCCCGGCGGAAAATGTATGAATGTAATTAAACTTTTATTTATGAATTGCATAACTGATATCAAGAGCATAAAGTATTATTTTTTCCCCTGACTTTCCTGTCAAGCACCCCTCTCGGCTTTTACGGGCCAAAACCCCCCCTCCCCCCTAAACTCGTGAGATTCCATTGTGTCTGCAAACCCCCCGGAAACAGAGCAAATCTTACTAGTTTTATCCCTTCTCAAATTTTTGTTCACTTGTATTTTTA&#xA;&amp;gt;speciesB_control_region_partial_sequence_mitochondrial&#xA;TTAAAGTAATGTAACCCTACATTAATGAAAAATCAAAA-TTATAGGAACTTAAATACATTACACCATCAAATAAATATGAAGGTAGACATAAACCATTGAATTTTAAATTTCATTAAACATGTTAAAAAAATGACGATATTGAATTGCCCTATCACAACTCTCATCAGTCTAGATATACCAGGACTCACCACCTCTGCAAGTAAGAGTCAAATGCAGTAAGAGACCACCATCAGTTGATTTCTTAATGTACACGTTTATTGATGGTCACGGACAAAAATCGTGGGGGTCGCACACAGTGAACTATTCCTGGCATTTGGTTCCTATTTCAGGTTCATCAATTGAATACATTCCCCTAACATTTCTTGACGCTTGCATAAGTTAATGGTGGAGTACATACTCCTCGTTACCCACCATGCCGAGCCTTCACTCTATAGGGCTACTGGTATCTTTTTTCTATTTCCTTTCACTTGACATTTCAGAGTGCATACAGAAATGTTATATCAAGGTTGAACATTTCCTTGCCCGGCGGAAAATGTATGAATGTAATTAAACTTTTATTTATGAATTGCATAACTGATATCAAGAGCATAAAGTATCATTTTTTCCCCTGACTTTCCTATCAAGCACCCCTCTCGGCTTTTGCGGGCCAAAACCCCCCCTCCCCCCTAAACTCGTGAGATTCCATTGTGTCTGCAAACCCCCCGGAAACAGAGCAAATCTTACTAGTTTTATCCCTTCTCAAATTTTTGTTCACTTGTATTTTTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My goal is to extract all SNPs in these sequences and format them in SNP FASTA format, as shown below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;rs11540652&#xA;GAACC[A/G]GAGGCCC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I specifically need this format (rather than, say, VCF) so I can search for RFLP sites using the following website:&#xA;&lt;a href=&quot;http://bio.kuas.edu.tw/snp-rflping2/rflpUI.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://bio.kuas.edu.tw/snp-rflping2/rflpUI.jsp&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know of an existing program which will accomplish this? Also, what additional materials do I need to detect these SNPs? I presume that I need assembled mitochondrial reference genomes for each species - anything else?&lt;/p&gt;&#xA;" OwnerUserId="9118" LastActivityDate="2021-03-12T12:09:39.597" Title="Converting FASTA Sequences to SNP FASTA Format" Tags="|fasta|snp|genomics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15539" PostTypeId="2" ParentId="15536" CreationDate="2021-03-12T07:02:47.960" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://www.internationalgenome.org/data-portal/data-collection/30x-grch38&quot; rel=&quot;nofollow noreferrer&quot;&gt;1000 genomes&lt;/a&gt;, &lt;a href=&quot;https://www.internationalgenome.org/data-portal/data-collection/hgdp&quot; rel=&quot;nofollow noreferrer&quot;&gt;HGDP&lt;/a&gt; and &lt;a href=&quot;https://www.simonsfoundation.org/simons-genome-diversity-project/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SGDP&lt;/a&gt; all contain SNP data from individuals where the country of origin is known.&lt;/p&gt;&#xA;&lt;p&gt;They all have slightly different numbers of people from different populations (1000 genomes and HGDP have more individuals in each pop, but fewer pops, and SGDP is the opposite).&lt;/p&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="3694" LastEditDate="2021-03-12T12:00:11.313" LastActivityDate="2021-03-12T12:00:11.313" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15540" PostTypeId="2" ParentId="15534" CreationDate="2021-03-12T09:15:42.240" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;Download a release (Hammock_v_1.2.0.7z) from &lt;a href=&quot;https://github.com/krejciadam/hammock/releases&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/krejciadam/hammock/releases&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Unzip&lt;/li&gt;&#xA;&lt;li&gt;To run the first example:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;cd /your_path/Hammock_v_1.2.0/examples/MUSI/&#xA;java -jar ../../dist/Hammock.jar full -i musi.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As mentioned in the comments, it's in the manual and any further issues need more information (errors etc).&lt;/p&gt;&#xA;" OwnerUserId="10591" LastActivityDate="2021-03-12T09:15:42.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15541" PostTypeId="2" ParentId="15538" CreationDate="2021-03-12T12:09:39.597" Score="2" Body="&lt;p&gt;Your question is not very clear. Where does the flanking sequence in :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;rs11540652&#xA;GAACC[A/G]GAGGCCC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;come from? I'll assume its meant to be the reference sequence and that speciesA is the reference and that the first nuc in the [] is from the reference... I don't know an existing existing program but here's a bespoke one.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;speciesA='TTAAAGTGATGTAATCCTACATTAATGAAAAATCAAAAGTTATAAGAACTTAAATACATTATATCATCAAATAAATATGAAGGTAGACATAAACCACTAAACTTCAAACTCCATTAAATATGTTATAAAAATGACGATATTGAATTGTCCTATCATAACTCTCATCAGTCTAGATATACCAGGACTCACCACCTCTGCAAGTCAGAGTCAAATGTAGTAAGAGACCACCATCAGTTGATTTCTTAATGTACACGTTTATTGATGGTCAAGGACAAAAATCGTGGGGGTAGCACACAGTGAACTATTCCTGGCATTTGGTTCCTATTTCAGGTTCATCAATTGAATACATTCCCCTAACATTTCCTGACGCTTGCATAAGTTAATGGTGGAGTACATACTCCTCGTTACCCACCATGCCGAGCATTCACTCTATAGGGCTACTGGTATCTTTTTTCTATTTCCTTTCACTTGACATTTCAGAGTGCATACAGAAATGTTATATTAAGGTTGAACATTTCCTTGCCCGGCGGAAAATGTATGAATGTAATTAAACTTTTATTTATGAATTGCATAACTGATATCAAGAGCATAAAGTATTATTTTTTCCCCTGACTTTCCTGTCAAGCACCCCTCTCGGCTTTTACGGGCCAAAACCCCCCCTCCCCCCTAAACTCGTGAGATTCCATTGTGTCTGCAAACCCCCCGGAAACAGAGCAAATCTTACTAGTTTTATCCCTTCTCAAATTTTTGTTCACTTGTATTTTTA'&#xA;speciesB='TTAAAGTAATGTAACCCTACATTAATGAAAAATCAAAA-TTATAGGAACTTAAATACATTACACCATCAAATAAATATGAAGGTAGACATAAACCATTGAATTTTAAATTTCATTAAACATGTTAAAAAAATGACGATATTGAATTGCCCTATCACAACTCTCATCAGTCTAGATATACCAGGACTCACCACCTCTGCAAGTAAGAGTCAAATGCAGTAAGAGACCACCATCAGTTGATTTCTTAATGTACACGTTTATTGATGGTCACGGACAAAAATCGTGGGGGTCGCACACAGTGAACTATTCCTGGCATTTGGTTCCTATTTCAGGTTCATCAATTGAATACATTCCCCTAACATTTCTTGACGCTTGCATAAGTTAATGGTGGAGTACATACTCCTCGTTACCCACCATGCCGAGCCTTCACTCTATAGGGCTACTGGTATCTTTTTTCTATTTCCTTTCACTTGACATTTCAGAGTGCATACAGAAATGTTATATCAAGGTTGAACATTTCCTTGCCCGGCGGAAAATGTATGAATGTAATTAAACTTTTATTTATGAATTGCATAACTGATATCAAGAGCATAAAGTATCATTTTTTCCCCTGACTTTCCTATCAAGCACCCCTCTCGGCTTTTGCGGGCCAAAACCCCCCCTCCCCCCTAAACTCGTGAGATTCCATTGTGTCTGCAAACCCCCCGGAAACAGAGCAAATCTTACTAGTTTTATCCCTTCTCAAATTTTTGTTCACTTGTATTTTTA'&#xA;counter=0&#xA;for i, nuc in enumerate(speciesA):&#xA;    if nuc != speciesB[i]:&#xA;        counter+=1&#xA;        print ('&amp;gt;rs'+str(counter))&#xA;        print (f'{speciesA[i-5:i]}[{nuc}/{speciesB[i]}]{speciesA[i+1:i+6]}')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Gives:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;rs1&#xA;AAAGT[G/A]ATGTA&#xA;&amp;gt;rs2&#xA;TGTAA[T/C]CCTAC&#xA;&amp;gt;rs3&#xA;CAAAA[G/-]TTATA&#xA;&amp;gt;rs4&#xA;TTATA[A/G]GAACT&#xA;&amp;gt;rs5&#xA;CATTA[T/C]ATCAT&#xA;&amp;gt;rs6&#xA;TTATA[T/C]CATCA&#xA;&amp;gt;rs7&#xA;AACCA[C/T]TAAAC&#xA;&amp;gt;rs8&#xA;CCACT[A/G]AACTT&#xA;&amp;gt;rs9&#xA;CTAAA[C/T]TTCAA&#xA;&amp;gt;rs10&#xA;AACTT[C/T]AAACT&#xA;&amp;gt;rs11&#xA;TCAAA[C/T]TCCAT&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As for 'I presume that I need assembled mitochondrial reference genomes for each species - anything else?' These sequences have clearly been aligned. So there's that.&lt;/p&gt;&#xA;&lt;p&gt;I hope that helps.&lt;/p&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-03-12T12:09:39.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15542" PostTypeId="1" CreationDate="2021-03-12T19:59:42.267" Score="0" ViewCount="49" Body="&lt;p&gt;In molecular evolution, I know that the instantaneous rate of the rate matrix is the limit of the rate as time approaches 0, but what I don't understand is how this rate is established or what I can do with this number.&lt;/p&gt;&#xA;&lt;p&gt;For instance, if a discrete character is thought to have changed from 0 to 1 five times on a tree with a root age of 40 million years, is the proportion 5 / 40 million the same as the instantaneous rate of change? If not, how exactly are they different?&lt;/p&gt;&#xA;&lt;p&gt;From a different perspective, let's assume an (absolute) instantaneous rate of 0.0001 and let's say I have a tree with a root age of 2 million years. Can I multiply 0.0001 by two million to get the expected number of transitions of that character across the tree?&lt;/p&gt;&#xA;" OwnerUserId="2961" LastActivityDate="2021-03-12T19:59:42.267" Title="How is the instantaneous rate of an instantaneous rate matrix determined?" Tags="|phylogenetics|substitution-model|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15544" PostTypeId="1" CreationDate="2021-03-13T11:31:34.307" Score="1" ViewCount="35" Body="&lt;p&gt;I was executing the following command &lt;code&gt;plink.exe --file GWAS --check-sex --noweb&lt;/code&gt;&#xA;Expecthing to get a &lt;code&gt;plink.sexcheck&lt;/code&gt; file but I don't get anything. Not even an error. The output is always:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Skipping web check... [ --noweb ]&#xA;Writing this text to log file [ plink.log ]&#xA;Analysis started: Sat Mar 13 12:24:18 2021&#xA;&#xA;Options in effect:&#xA;        --file GWAS&#xA;        --check-sex&#xA;        --noweb&#xA;&#xA;75 (of 75) markers to be included from [ GWAS.map ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Do you know what could be the issue there? I have a &lt;code&gt;GWAS.map&lt;/code&gt; and &lt;code&gt;GWAS.ped&lt;/code&gt; file. Do I need any additional information for this to work?&lt;/p&gt;&#xA;" OwnerUserId="9131" LastActivityDate="2021-03-13T11:31:34.307" Title="plink doesn't generate a sexcheck-file" Tags="|plink|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15545" PostTypeId="1" AcceptedAnswerId="15569" CreationDate="2021-03-13T12:22:26.563" Score="1" ViewCount="114" Body="&lt;p&gt;Hi I am playing with Biopython and BioSQL,&lt;/p&gt;&#xA;&lt;p&gt;I manage to create an MySQL database using &lt;a href=&quot;https://biopython.org/wiki/BioSQL&quot; rel=&quot;nofollow noreferrer&quot;&gt;Managing local biological databases with the BioSQL module&lt;/a&gt; tutorial:&lt;/p&gt;&#xA;&lt;p&gt;and load 2909 fasta sequences in it through:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;db = server[&amp;quot;test_1&amp;quot;]&#xA;&#xA;count = db.load(SeqIO.parse(&amp;quot;Newtextfile-03.txt&amp;quot;, &amp;quot;fasta&amp;quot;))&#xA;        print(&amp;quot;Loaded %i records&amp;quot; % count)&#xA;        server.commit()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;from this file: &lt;a href=&quot;https://easyupload.io/us14gl&quot; rel=&quot;nofollow noreferrer&quot;&gt;Newtextfile-03.txt&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;problem is trying just to print the database index list with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in db:&#xA;    print(i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;works , while trying print element by element:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in db:&#xA;    print(i, db[i])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for key, record in db.items():&#xA;            print (&amp;quot;Key %r maps to a sequence record  %s&amp;quot; % (key, record))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;gives an error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;2557 Traceback (most recent call last):&#xA;  File &amp;quot;./try_001.py&amp;quot;, line 142, in &amp;lt;module&amp;gt;&#xA;    f()&#xA;  File &amp;quot;./try_001.py&amp;quot;, line 120, in f&#xA;    print(i, db[i])&#xA;  File &amp;quot;......../lib/python3.8/site-packages/Bio/SeqRecord.py&amp;quot;, line 652, in __str__&#xA;    lines.append(repr(self.seq))&#xA;  File &amp;quot;........//lib/python3.8/site-packages/Bio/Seq.py&amp;quot;, line 109, in __repr__&#xA;    return f&amp;quot;{self.__class__.__name__}({self._data!r})&amp;quot;&#xA;&#xA;AttributeError: 'DBSeq' object has no attribute '_data'&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;the error occurs for three records namely : [2164, 2522, 2557]&lt;/p&gt;&#xA;&lt;p&gt;inspecting the &lt;a href=&quot;https://easyupload.io/us14gl&quot; rel=&quot;nofollow noreferrer&quot;&gt;Newtextfile-03.txt&lt;/a&gt; file results in finding fasta sequences of less than&#xA;100 aminoacids.&lt;/p&gt;&#xA;&lt;p&gt;Is there any way to resolve the error ? It sounds strange to me that I can load the records into the database but not retrieving them. Is anything I am missing about the record load ? or similar ?&lt;/p&gt;&#xA;&lt;p&gt;I am working with the database using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; server = BioSeqDatabase.open_database(&#xA;    driver='MySQLdb' , &#xA;    user=password.userz,&#xA;    password=password.pazzword,&#xA;    host=&amp;quot;XXXXXXXX&amp;quot;,&#xA;    db=&amp;quot;bio-data-first&amp;quot;)&#xA;&#xA;db = server.new_database(&amp;quot;test_1&amp;quot;, description=&amp;quot;Just for testing&amp;quot;)&#xA;        server.commit()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;using  driver=&amp;quot;MySQLdb&amp;quot;, wasn't able to have driver='mysql.connector' to work,&lt;/p&gt;&#xA;&lt;p&gt;biopython is version 1.78&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2021-03-13T13:00:12.850" LastActivityDate="2021-03-17T18:33:20.867" Title="Biopython BioSQL error : AttributeError: 'DBSeq' object has no attribute '_data'" Tags="|fasta|python|database|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15547" PostTypeId="2" ParentId="15219" CreationDate="2021-03-15T09:37:51.027" Score="1" Body="&lt;p&gt;Permutational Multivariate Analysis of Variance (PERMANOVA) assumes no distribution which means it is a non-parameteric statistical test. According to the &lt;a href=&quot;https://archetypalecology.wordpress.com/2018/02/21/permutational-multivariate-analysis-of-variance-permanova-in-r-preliminary/#:%7E:text=Permutational%20multivariate%20analysis%20of%20variance%20(PERMANOVA)%20is%20a%20non-,are%20equivalent%20for%20all%20groups&quot; rel=&quot;nofollow noreferrer&quot;&gt;assumption of PERMANOVA test&lt;/a&gt;, each group must have same dispersion in the data. Before performing any statistical test, we must check the underlying assumptions. Variance partitioning (permutest) is a confirmatory test to check the assumption (similar dispersion). You can use &lt;a href=&quot;https://www.rdocumentation.org/packages/vegan/versions/2.4-2/topics/permutest.betadisper&quot; rel=&quot;nofollow noreferrer&quot;&gt;permutest.betadisper&lt;/a&gt; for permutest. After that you can perform PERMANOVA test using &lt;a href=&quot;https://www.rdocumentation.org/packages/vegan/versions/1.11-0/topics/adonis&quot; rel=&quot;nofollow noreferrer&quot;&gt;adonis()&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Interpretation of PERMANOVA test –&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If p value is small (&amp;lt;0.05 ) we reject the null hypothesis (No difference between the groups) otherwise we accept the null hypothesis (Significant difference between the groups).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Coefficient of determination, R-squared value indicates the variation in the distances being explained by groups under the study. Higher value indicates a better fit.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;You can further look into the confidence interval for each group.&lt;/p&gt;&#xA;&lt;p&gt;Hope this confirms the answer to your question.&lt;/p&gt;&#xA;" OwnerUserId="10470" LastActivityDate="2021-03-15T09:37:51.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15549" PostTypeId="2" ParentId="15504" CreationDate="2021-03-15T15:39:16.107" Score="2" Body="&lt;p&gt;@Jim Hat&lt;/p&gt;&#xA;&lt;p&gt;I took a stab at this given the example files you included.&lt;/p&gt;&#xA;&lt;p&gt;If the 'list.txt' file looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Solyc00g019980&#xA;Solyc01g005020&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the data.gff file looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SL4.0ch01   maker_ITAG  gene    46201   59605   .   -   .   ID=gene:Solyc01g005020.3;Alias=Solyc01g005020;Name=Solyc01g005020.3;length=13404&#xA;SL4.0ch01   maker_ITAG  mRNA    46201   59605   .   -   .   ID=mRNA:Solyc01g005020.3.1;Parent=gene:Solyc01g005020.3;Name=Solyc01g005020.3.1;Note=Salt Overly Sensitive 1;_AED=0.25;_QI=241|1|1|1|0|0|23|311|1151;_eAED=0.25;Ontology_term=GO:0006812,GO:0015299,GO:0016021,GO:0055085&#xA;SL4.0ch01   maker_ITAG  three_prime_UTR 46201   46511   .   -   .   ID=three_prime_UTR:Solyc01g005020.3.1.0;Parent=mRNA:Solyc01g005020.3.1&#xA;SL4.0ch01   maker_ITAG  exon    46201   46844   .   -   .   ID=exon:Solyc01g005020.3.1.1;Parent=mRNA:Solyc01g005020.3.1&#xA;SL4.0ch01   maker_ITAG  CDS 46512   46844   .   -   0   ID=CDS:Solyc01g005020.3.1.1;Parent=mRNA:Solyc01g005020.3.1&#xA;SL4.0ch01   maker_ITAG  exon    47281   47585   .   -   .   ID=exon:Solyc01g005020.3.1.2;Parent=mRNA:Solyc01g005020.3.1&#xA;SL4.0ch01   maker_ITAG  CDS 47281   47585   .   -   2   ID=CDS:Solyc01g005020.3.1.2;Parent=mRNA:Solyc01g005020.3.1&#xA;SL4.0ch01   maker_ITAG  exon    48276   48543   .   -   .   ID=exon:Solyc01g005020.3.1.3;Parent=mRNA:Solyc01g005020.3.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(Note I removed the **s, which I assumed were added for emphasis)&lt;/p&gt;&#xA;&lt;p&gt;You can do the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;aliasNames=&lt;span class=&quot;math-container&quot;&gt;$(grep -f list.txt data.gff | cut -d':' -f2 | cut -d';' -f1)&#xA;ontologyTerms=$&lt;/span&gt;(grep -f list.txt data.gff | cut -d';' -f 8 | cut -d'=' -f2)&#xA;paste &amp;lt;(echo &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$aliasNames&quot;) &amp;lt;(echo &quot;$&lt;/span&gt;ontologyTerms&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The first line filters the file based on your list and then extracts the alias names from the filtered GFF file. The second line filters the GFF and extracts the ontology terms (after the equal sign). And the third line joins the two as columns.&lt;/p&gt;&#xA;&lt;p&gt;The result that is printed to the screen is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Solyc01g005020.3&#xA;Solyc01g005020.3.1      GO:0006812,GO:0015299,GO:0016021,GO:0055085&#xA;Solyc01g005020.3.1.0&#xA;Solyc01g005020.3.1.1&#xA;Solyc01g005020.3.1.1&#xA;Solyc01g005020.3.1.2&#xA;Solyc01g005020.3.1.2&#xA;Solyc01g005020.3.1.3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Given the examples you included all of the lines of the GFF are included, but not all have gene ontology information.&lt;/p&gt;&#xA;" OwnerUserId="10742" LastActivityDate="2021-03-15T15:39:16.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15552" PostTypeId="2" ParentId="15485" CreationDate="2021-03-16T09:32:33.963" Score="1" Body="&lt;p&gt;Just a recommendation: Be careful when making your data sets.&lt;/p&gt;&#xA;&lt;p&gt;P31327_57, P31327_119, P31327_157 are all annotated as &amp;quot;By similarity&amp;quot; to the mouse entry in Swiss-Prot &lt;a href=&quot;https://www.uniprot.org/uniprot/P31327#ptm_processing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uniprot/P31327#ptm_processing&lt;/a&gt;, which means there is no direct experimental evidence in the human entry. If you do have publications that report experimental evidence for these PTMs, please don't hesitate to share them with the UniProt team.&lt;/p&gt;&#xA;&lt;p&gt;I also recommend reading this UniProt help page about using UniProt data in negative datasets if you haven't already done so: &lt;a href=&quot;https://www.uniprot.org/help/negative_datasets&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/help/negative_datasets&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1459" LastActivityDate="2021-03-16T09:32:33.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15554" PostTypeId="1" CreationDate="2021-03-16T11:55:34.927" Score="0" ViewCount="356" Body="&lt;p&gt;We designed an experiment to explore the potential role of carbon dioxide on algae physiology using RNA-Seq. We analyse the differential gene expression using DESeq2 but now we are interested into analyse a few genes in detail.&lt;/p&gt;&#xA;&lt;p&gt;My PI say to make the statistics over the ration response (expression_treatment/expression_control - 1) and my question is the following: Can I use TPM to calculate this ratio and make following statistics? I read in the documentation of DESeq2 that TPM we can't use this normalisation method to differential expression and my my other question is Why shouldn't?&lt;/p&gt;&#xA;&lt;p&gt;My advisor ask to re-do the statistical analysis using the ratio response (rr) so i need the rr for each sample. I think this doesnt make sense cause the result will be or should be the same that using DESeq2.&lt;/p&gt;&#xA;&lt;p&gt;Update: I'm using the fold-change calculated by DESeq2 and I'm not sure if I have transformed the data correctly. I took the two columns &lt;code&gt;log2FoldChange&lt;/code&gt; and &lt;code&gt;lfcSE&lt;/code&gt; (both are log-transformed variables) so I went back to the non-transformed variable as follow: &lt;code&gt;FC=2^log2FoldChange&lt;/code&gt; and &lt;code&gt;SE=lfcSE*2^log2FoldChange&lt;/code&gt;. I saw this approximation in &lt;a href=&quot;https://stats.stackexchange.com/questions/123514/calculating-standard-error-after-a-log-transform&quot;&gt;this post&lt;/a&gt;. Is that correct? I have obtained a very broad range of error/&lt;/p&gt;&#xA;" OwnerUserId="9701" LastEditorUserId="73" LastEditDate="2021-05-09T05:15:04.157" LastActivityDate="2021-05-09T05:15:04.157" Title="Doubt about using TPM for statistics" Tags="|rna-seq|normalization|tpm|fold-change|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15556" PostTypeId="1" AcceptedAnswerId="15557" CreationDate="2021-03-16T20:51:22.150" Score="6" ViewCount="85" Body="&lt;p&gt;I have a library of biological implementations (i.e. plasmids) of genetic circuits and their experimental data associated. I want to implement an &lt;em&gt;in silico&lt;/em&gt; version with SBOL, but to be consistent with what I have I'm my freezer, I need to identify not only the genetic circuit but also the biological replicate (i.e. the different colonies picked), as they have their own associated experimental data.&lt;/p&gt;&#xA;" OwnerUserId="10755" LastActivityDate="2021-03-16T21:09:20.260" Title="In SBOL, is there a way to distinguish biological replicates?" Tags="|sbol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15557" PostTypeId="2" ParentId="15556" CreationDate="2021-03-16T21:09:20.260" Score="6" Body="&lt;p&gt;Support for distinguishing replicates was added in SBOL 2.2, along with other support for linking data throughout the experiment cycle.  I will explain this in terms of SBOL 3, which is simpler, then give the SBOL 2 translation at the end.&lt;/p&gt;&#xA;&lt;p&gt;For SBOL 3, all designs, including those for a plasmid, a strain, etc, are specified in terms of the &lt;code&gt;Component&lt;/code&gt; class. This is just a &amp;quot;paper&amp;quot; design, however, and may or may not actually exist. When you have a real physical aliquot that you want to talk about, this is represented by an &lt;code&gt;Implementation&lt;/code&gt; object: this is your biological replicate.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;Implementation&lt;/code&gt; doesn't really have any content of its own; its nature is defined by linking back to the intended design with a &lt;code&gt;prov:wasDerivedFrom&lt;/code&gt; link. If you've verified that what you got is what you intended, you can supplement that with a &lt;code&gt;built&lt;/code&gt; link as well.&lt;/p&gt;&#xA;&lt;p&gt;One additional note, however: when you're representing a colony, you're probably not actually representing just a plasmid, but rather are representing a particular strain that has been transformed by a plasmid. The actual data you have may then be one level further removed, since the data has been gathered from a sample under particular growth conditions. SBOL can represent that as well, by describing the sample as a mixture of particular media, inducers, etc., with a transformed strain, which is itself a cell type containing your plasmid.&lt;/p&gt;&#xA;&lt;p&gt;Here is an example diagram showing just this sort of relationship, of data associated with three replicates of a sample.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ajATx.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ajATx.png&quot; alt=&quot;SBOL representation of three replicates of a sample&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can find this example in &lt;a href=&quot;https://github.com/SynBioDex/Community-Media/tree/master/2020/COMBINE20&quot; rel=&quot;noreferrer&quot;&gt;the SBOL3 tutorial from COMBINE 2020&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Finally, in SBOL 2, this works exactly the same way, except that instead of a &lt;code&gt;Component&lt;/code&gt; your implementation points to a &lt;code&gt;ComponentDefinition&lt;/code&gt; or a &lt;code&gt;ModuleDefinition&lt;/code&gt;, depending on the nature of the design you're representing. I won't go into that distinction, because it was a bad idea and there's a reason we got rid of it for SBOL 3.&lt;/p&gt;&#xA;" OwnerUserId="4869" LastActivityDate="2021-03-16T21:09:20.260" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15558" PostTypeId="1" AcceptedAnswerId="15560" CreationDate="2021-03-16T21:33:31.260" Score="4" ViewCount="97" Body="&lt;p&gt;I have a collection of biological parts and devices in a particular format that basically stores information of the sequence of the elements. My intention is to move to a more standardized data structure with a particular interest in Synthetic Biology. SBOL 3 looks like a good choice, then the question is if there is a way to kind of &amp;quot;import&amp;quot; data from my sequence-based database into SBOL?&lt;/p&gt;&#xA;" OwnerUserId="10755" LastActivityDate="2021-04-06T18:29:57.187" Title="Is there a good practice (or an easy one for a SBOL newbie) to set up a collection of biological parts and devices?" Tags="|sbol|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15559" PostTypeId="2" ParentId="15554" CreationDate="2021-03-16T21:37:00.123" Score="4" Body="&lt;p&gt;This is a common question, and the answer is while you &lt;em&gt;can&lt;/em&gt; calculate this, it's not statistically robust.&#xA;You're likely to arrive at false conclusions.&lt;/p&gt;&#xA;&lt;p&gt;Why is this?&#xA;Because while TPM is something you can calculate from a single sample, your question involves multiple replicates and those replicates vary between each other.&#xA;Counts in units of TPM are missing information about the variation between samples.&#xA;This extra information affects fold change estimates, and is why differential analysis of sequencing data is complicated.&lt;/p&gt;&#xA;&lt;p&gt;See &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2305/240&quot;&gt;this answer&lt;/a&gt; to a related question, and this &lt;a href=&quot;https://haroldpimentel.wordpress.com/2014/05/08/what-the-fpkm-a-review-rna-seq-expression-units/&quot; rel=&quot;nofollow noreferrer&quot;&gt;often-cited blog post&lt;/a&gt; for more information.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2021-03-16T21:37:00.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15560" PostTypeId="2" ParentId="15558" CreationDate="2021-03-16T21:59:52.420" Score="5" Body="&lt;p&gt;For SBOL 2, you can readily import data via the online converter tool at &lt;a href=&quot;https://converter.sbolstandard.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://converter.sbolstandard.org/&lt;/a&gt;&#xA;The converter basically just wraps the conversion function found in the SBOL libraries, so if you're coding, you can do the conversion directly from the libraries.&lt;/p&gt;&#xA;&lt;p&gt;For SBOL 3, there isn't yet an online tool, but James McLaughlin has a converter in his JavaScript library: &lt;a href=&quot;https://github.com/sboltools/sbolgraph&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/sboltools/sbolgraph&lt;/a&gt;&#xA;I believe he is working to bring it online shortly, so hopefully this answer will soon need to be updated or replaced by an answer from him.&lt;/p&gt;&#xA;&lt;p&gt;An anecdote on the difficulty level: I recently started using an SBOL3--&amp;gt;SBOL2--&amp;gt;GenBank conversion using pySBOL3 and pySBOL3. It took a couple of hours to setup the pipeline, and isn't perfect, but it's pretty doable (and ought to be generalized for others to use as well at some point). The other way would be easier, since GenBank contents are pretty simple.&lt;/p&gt;&#xA;" OwnerUserId="4869" LastEditorUserId="4869" LastEditDate="2021-03-17T18:40:53.163" LastActivityDate="2021-03-17T18:40:53.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15562" PostTypeId="1" AcceptedAnswerId="15567" CreationDate="2021-03-17T12:01:34.463" Score="0" ViewCount="21" Body="&lt;p&gt;I received a matrix of genomics features (KEGG annotated metagenomes) abundance from 6 samples belonging to 2 groups. My collaborator is interested in finding differences between the two groups.&lt;/p&gt;&#xA;&lt;p&gt;The matrix, after scaling and removing zero-variance, is roughly 4000 features. Via shapiro test I already know some 30% of the features are not normally distributed across the samples, so I assume PCA is out of option and any test with the normal distribution assumption.&lt;/p&gt;&#xA;&lt;p&gt;I already made some preliminary analysis using NMDS to cluster my samples and was thinking of trying also t-SNE and recursive feature elimination. However I am not sure how to find &amp;quot;significant&amp;quot; differences given the small sample size. Any suggestion? Would something like LEfSe work in this instance?&lt;/p&gt;&#xA;" OwnerUserId="9052" LastActivityDate="2021-03-17T17:21:31.207" Title="How do you find differences when your metagenome sample size is small?" Tags="|statistics|genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15564" PostTypeId="2" ParentId="15528" CreationDate="2021-03-17T13:13:05.067" Score="3" Body="&lt;p&gt;It is actually very hard to answer this using only the information you have provided. It'd be necessary to actually see your &lt;code&gt;test.nf&lt;/code&gt; file and read the processes. The execution seems right and I'd guess that the processB may be not emiting any output as shown in &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#outputs&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nextflow.io/docs/latest/process.html#outputs&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;It is also possible that when using the &lt;code&gt;-resume&lt;/code&gt; together with a specific run-name you may be recovering the state of that specific run as shown here &lt;a href=&quot;https://www.nextflow.io/blog/2019/demystifying-nextflow-resume.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nextflow.io/blog/2019/demystifying-nextflow-resume.html&lt;/a&gt;. You can try to execute it without the run-name or without the &lt;code&gt;-resume&lt;/code&gt; flag so you can check whether the problem is with the emition and channelling of outputs between processes, or with the &lt;code&gt;-resume&lt;/code&gt; flag.&lt;/p&gt;&#xA;" OwnerUserId="6767" LastActivityDate="2021-03-17T13:13:05.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15565" PostTypeId="1" AcceptedAnswerId="15566" CreationDate="2021-03-17T15:47:06.107" Score="0" ViewCount="232" Body="&lt;p&gt;I recently received some .bigwig files from a chipseq experiment.&#xA;I loaded the files into IGV.&lt;/p&gt;&#xA;&lt;p&gt;The interpretation is rather intuitive.&lt;/p&gt;&#xA;&lt;p&gt;My question is the following:&#xA;What exactly do the track heights represent?&lt;/p&gt;&#xA;&lt;p&gt;For IgG control, the track height is low ranging from 0~5.&lt;/p&gt;&#xA;&lt;p&gt;For positive controls, such as H3K4me3, I see clear peaks with height ~100.&lt;/p&gt;&#xA;&lt;p&gt;Do track heights simply mean the number of reads mapped to that genomic interval during peak calling?&#xA;Does it literally mean there are ~100 reads that aligned to that region in the H3K4me3 .bam file?&lt;/p&gt;&#xA;&lt;p&gt;Thanks so much.&lt;/p&gt;&#xA;" OwnerUserId="7056" LastEditorUserId="7056" LastEditDate="2021-03-17T20:36:26.660" LastActivityDate="2021-03-17T20:36:26.660" Title="How to interpret track height in Chipseq?" Tags="|chip-seq|peak-calling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15566" PostTypeId="2" ParentId="15565" CreationDate="2021-03-17T16:45:29.873" Score="2" Body="&lt;p&gt;Depends how the files were made. In the simplest case yes, the height represents the pipeup of reads from the BAM file that was used. For a direct (visual) comparison you have to normalize the files though as otherwise sequencing depth confounds the height of the peaks, like if file A has ten times more reads and the peak is 10 times higher than in B then this is probably only due to differences in total read number (=sequencing depth). I wrote down my preferred method to do that in this biostars post: &lt;a href=&quot;https://www.biostars.org/p/413626/#414440&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/413626/#414440&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Does this make sense to you?&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-03-17T16:45:29.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15567" PostTypeId="2" ParentId="15562" CreationDate="2021-03-17T17:21:31.207" Score="0" Body="&lt;p&gt;I think there are several ways to figure this out. I wouldn't worry too much about normality as most of the methods are either nonparametric or pretty robust to violations.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Since you are using KEGG, I would suggest working at the pathway or module level of annotation (if your features are e.g. KEGG Orthology groups). In principle, you can then combine information across features as well as across samples. Simplest I can think of, without even looking at the literature, would be to use permutations (shuffle row and column labels) to look for modules different in abundance between groups. (Many existing methods are doing some more sophisticated version of this.)&lt;/li&gt;&#xA;&lt;li&gt;I think that LEfSe in theory should work quite well here, it's been a while since I thought about it but I believe this is the problem that it is designed to solve, assuming that I understand your question.&lt;/li&gt;&#xA;&lt;li&gt;There are a lot of methods that are previously described for doing this kind of work. I would suggest reading some reviews of methods, for example the section &amp;quot;Multivariate differential abundance testing&amp;quot; in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6459172/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;, or &lt;a href=&quot;https://www.nature.com/articles/nbt.3935?foxtrotcallback=true&amp;amp;error=cookies_not_supported&amp;amp;code=c7b60d29-dbaf-46d9-8d2e-2bd0189867c4&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, looking for some inspiration. To some extent it depends on which questions you are interested in answering and what exactly your features represent.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-03-17T17:21:31.207" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15568" PostTypeId="2" ParentId="2644" CreationDate="2021-03-17T17:46:08.883" Score="0" Body="&lt;p&gt;Once you &lt;a href=&quot;https://github.com/ncbi/sra-tools/wiki/02.-Installing-SRA-Toolkit&quot; rel=&quot;nofollow noreferrer&quot;&gt;install SRA tools&lt;/a&gt;, you can grab a BioProject/SRP accession list (Send to - File - Accession List) and save it to default SRA tools /sra directory, then run (with /sra as cwd):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;prefetch --option-file SraAccList.txt&#xA;cat SraAccList.txt | xargs fasterq-dump --outdir &amp;quot;/your-directory/for-fastq&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;P.S. The text file can be any list of accessions, separated by return.&#xA;Tested in Bash&lt;/p&gt;&#xA;" OwnerUserId="10764" LastEditorUserId="10764" LastEditDate="2021-04-21T12:14:23.567" LastActivityDate="2021-04-21T12:14:23.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15569" PostTypeId="2" ParentId="15545" CreationDate="2021-03-17T18:33:20.867" Score="1" Body="&lt;p&gt;&lt;strong&gt;It's a bug in Biopython 1.78.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;What those three sequences have in common is that they're less than 61 characters long.  When you print the record it implicitly calls &lt;code&gt;__str__&lt;/code&gt; for the record, which calls &lt;code&gt;__repr__&lt;/code&gt; for the Seq.  DBSeq gets its method from Seq, which looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    def __repr__(self):&#xA;        &amp;quot;&amp;quot;&amp;quot;Return (truncated) representation of the sequence for debugging.&amp;quot;&amp;quot;&amp;quot;&#xA;        if len(self) &amp;gt; 60:&#xA;            # Shows the last three letters as it is often useful to see if&#xA;            # there is a stop codon at the end of a sequence.&#xA;            # Note total length is 54+3+3=60&#xA;            return f&amp;quot;{self.__class__.__name__}('{str(self[:54])}...{str(self[-3:])}')&amp;quot;&#xA;        else:&#xA;            return f&amp;quot;{self.__class__.__name__}({self._data!r})&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;...and then it only tries to access the nonexistent &lt;code&gt;_data&lt;/code&gt; for the short sequences.&lt;/p&gt;&#xA;&lt;p&gt;So there's a mismatch between DBSeq's lack of a &lt;code&gt;_data&lt;/code&gt; attribute and its use of &lt;code&gt;Seq.__repr__&lt;/code&gt;.  I don't see a DBSeq class at all in the latest Biopython, so maybe this is fixed already.  But there's no tag for 1.79 yet that I can see and I can't get the very latest code to import successfully so I'm not sure.  In the meantime if you avoid letting &lt;code&gt;repr(record)&lt;/code&gt; happen you should be able to work around it.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2021-03-17T18:33:20.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15572" PostTypeId="1" AcceptedAnswerId="15575" CreationDate="2021-03-18T22:09:29.810" Score="0" ViewCount="98" Body="&lt;p&gt;I have two questions, one is a direct question about some RNA expression data.  The other is more broad, but motivated by the first.  I have downloaded an expression set for a class project and I am unfamiliar with the naming convention, I recognize entrez ID but not this one.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    &amp;gt; rownames(Rao)[1:10]&#xA; [1] &amp;quot;0610007P14Rik&amp;quot; &amp;quot;0610009B22Rik&amp;quot; &amp;quot;0610009D07Rik&amp;quot; &amp;quot;0610009L18Rik&amp;quot; &amp;quot;0610009O20Rik&amp;quot; &amp;quot;0610010B08Rik&amp;quot; &amp;quot;0610010F05Rik&amp;quot;&#xA; [8] &amp;quot;0610010K14Rik&amp;quot; &amp;quot;0610011F06Rik&amp;quot; &amp;quot;0610012G03Rik&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So my first question is what is this gene naming convention.  The second is what are many other conventions I should recognize and what tools can I use for converting between them?  Lastly what kind of errors may crop up when I do that and how can I check for mismappings or non-unique mappings?&lt;/p&gt;&#xA;&lt;p&gt;I am primarily interested in R but methods in python would also be appreciated!  Its always good to have more tools.&lt;/p&gt;&#xA;" OwnerUserId="2482" LastActivityDate="2021-03-19T01:16:58.970" Title="Common gene naming conventions and how to convert between them" Tags="|r|rna-seq|python|gene-expression|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15573" PostTypeId="1" CreationDate="2021-03-19T00:57:02.713" Score="1" ViewCount="770" Body="&lt;p&gt;3 solutions that I had to unfortunately rule out.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Potential solution #1&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;As far as I know, the conventional way is to do this is by using CCDS ids (e.g. using biomart). However, the issue is that, as I see on the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/projects/CCDS/CcdsBrowse.cgi?REQUEST=SHOW_STATISTICS&quot; rel=&quot;nofollow noreferrer&quot;&gt;CCDS project page&lt;/a&gt;, the last release of CCDS ids was made almost 3 years ago (!) and it seems to be compatible with Ensembl release 92. Then how should one convert ids to the latest Ensembl release e.g. 100?&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;CCDS Release&lt;/th&gt;&#xA;&lt;th&gt;Species&lt;/th&gt;&#xA;&lt;th&gt;NCBI Annotation Release&lt;/th&gt;&#xA;&lt;th&gt;Ensembl Annotation Release&lt;/th&gt;&#xA;&lt;th&gt;Assembly Name&lt;/th&gt;&#xA;&lt;th&gt;Assembly ID&lt;/th&gt;&#xA;&lt;th&gt;Made Public&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;22&lt;/td&gt;&#xA;&lt;td&gt;Homo sapiens&lt;/td&gt;&#xA;&lt;td&gt;109&lt;/td&gt;&#xA;&lt;td&gt;92&lt;/td&gt;&#xA;&lt;td&gt;GRCh38.p12&lt;/td&gt;&#xA;&lt;td&gt;GCF_000001405.38&lt;/td&gt;&#xA;&lt;td&gt;06/14/2018&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;&lt;strong&gt;Potential solution #2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am also aware of Ensembl's &lt;a href=&quot;http://useast.ensembl.org/Homo_sapiens/Tools/IDMapper&quot; rel=&quot;nofollow noreferrer&quot;&gt;ID History Converter&lt;/a&gt;. However, I am looking for a programmatic approach. Also, I noticed that if I try to convert ~&amp;gt;20K ids with ID History Converter, in my experience, there is always some issue. e.g. an error I got when I tried to convert ~70K ensembl transcript ids.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/9Ie1t.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9Ie1t.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As a side note, I wonder why this ID conversion functionality is not available via Ensembl's REST API. To me, id conversion seems to be a basic enough task that almost every bioinformatician has to do. So it would be immensely helpful to be someday able to do this programmatically using Ensembl's mighty a REST API.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Potential solution #3&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Finally, as another potential solution, over at Ensembl's FTP website, located within the directory of the release to which I am interested in converting my ids to (release 100, target release), I saw that according to a &lt;a href=&quot;http://ftp.ensembl.org/pub/release-100/README&quot; rel=&quot;nofollow noreferrer&quot;&gt;README file&lt;/a&gt;, &amp;quot;ensembl_archive_&amp;quot; directory should contain &amp;quot;Data on historical Ensembl releases&amp;quot;. So, I wondered if this directory contains a map between ids from different releases. However, without any extra info about column names, contents, file formats etc, I couldn't really figure out how to use those files. So, I had to rule this one out too.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Index of /pub/release-100/mysql/ensembl_archive_100/&#xA;../&#xA;CHECKSUMS                                          26-Mar-2020 14:31                 132&#xA;ens_release.txt.gz                                 26-Mar-2020 14:31                1098&#xA;ensembl_archive_100.sql.gz                         26-Mar-2020 14:31                 943&#xA;release_species.txt.gz                             26-Mar-2020 14:31               39620&#xA;species.txt.gz                                     26-Mar-2020 14:31                6669&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Of note, I saw a similar &amp;quot;mysql&amp;quot; directory over at the directory of the release whose ids I want to convert (source release): &lt;code&gt;http://ftp.ensembl.org/pub/grch37/release-100/mysql/homo_sapiens_core_100_37/ &lt;/code&gt;. However, again, I do not understand its contents without any extra information.&lt;/p&gt;&#xA;&lt;p&gt;So, finally, I am not sure what is the best way to programmatically convert Ensembl ids from between different releases. As a test case scenario, I want to convert all the protein_coding transcript ids from GRCh37 (release 75) to GRCh38 (release 100). If anybody has any suggestions about this, please kindly let me know.&lt;/p&gt;&#xA;" OwnerUserId="274" LastEditorUserId="274" LastEditDate="2021-03-22T13:11:30.543" LastActivityDate="2021-03-25T16:15:02.477" Title="What is the best way to programmatically convert Ensembl ids from one release to other? e.g. transcript ids from release 75 to 100" Tags="|gene|ensembl|identifiers|conversion|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15575" PostTypeId="2" ParentId="15572" CreationDate="2021-03-19T01:16:58.970" Score="1" Body="&lt;p&gt;Biologists are familiar with &lt;code&gt;SYMBOL&lt;/code&gt; (for good reasons).&lt;/p&gt;&#xA;&lt;p&gt;Bioinformaticians like &lt;code&gt;ENTREZID&lt;/code&gt; &lt;code&gt;ENSEMBL&lt;/code&gt; &lt;code&gt;REFSEQ&lt;/code&gt; &lt;code&gt;GENBANK&lt;/code&gt; etc, I'm definitely missing some here.&lt;/p&gt;&#xA;&lt;p&gt;You don't need to know these by heart, just know that databases don't map 1:1.&lt;/p&gt;&#xA;&lt;p&gt;I've worked with mouse RNAseq data before, so I agree with user::RamRS comment that it looks like a mouse gene. What type of identifier is it? I had no idea.&lt;/p&gt;&#xA;&lt;p&gt;You can find this out at NCBI::GENE, by querying for, let's say &lt;code&gt;Rao[1]&lt;/code&gt; &lt;code&gt;&amp;quot;0610007P14Rik&amp;quot;&lt;/code&gt;. (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene/?term=0610007P14Rik&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/gene/?term=0610007P14Rik&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;NCBI::GENE tells you the &lt;code&gt;SYMBOL::Erg28&lt;/code&gt;, &lt;code&gt;ENTREZID::8520&lt;/code&gt;, and in the far right a column called &lt;code&gt;ALIAS::0610007P14Rik&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now that you know your identifier type (ALIAS), you can map between databases in R.&#xA;There are many packages to do this. Someone mentioned biomart in the comment.&lt;/p&gt;&#xA;&lt;p&gt;I use &lt;code&gt;AnnotationDbi&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(AnnotationDbi)&#xA;library(org.Mm.eg.db)&#xA;&#xA;Rao &amp;lt;- c(&amp;quot;0610007P14Rik&amp;quot;, &amp;quot;0610009B22Rik&amp;quot;, &amp;quot;0610009D07Rik&amp;quot;, &amp;quot;0610009L18Rik&amp;quot;, &amp;quot;0610009O20Rik&amp;quot;, &amp;quot;0610010B08Rik&amp;quot;, &amp;quot;0610010F05Rik&amp;quot;, &amp;quot;0610010K14Rik&amp;quot;, &amp;quot;0610011F06Rik&amp;quot;, &amp;quot;0610012G03Rik&amp;quot;)&#xA;&#xA;#use the function select to retrieve the column type &amp;quot;SYMBOL&amp;quot; with keys &amp;quot;RAO&amp;quot;.&#xA;#&amp;quot;RAO&amp;quot; is a keytype of &amp;quot;ALIAS&amp;quot;. And I want to search in the mice database &amp;quot;org.Mm.eg.db&amp;quot;&#xA;&#xA;AnnotationDbi::select(x = org.Mm.eg.db,&#xA;+                       keys = Rao,&#xA;+                       keytype = &amp;quot;ALIAS&amp;quot;,&#xA;+                       columns = &amp;quot;SYMBOL&amp;quot;)&#xA;'select()' returned 1:1 mapping between keys and columns&#xA;           ALIAS        SYMBOL&#xA;1  0610007P14Rik         Erg28&#xA;2  0610009B22Rik 0610009B22Rik&#xA;3  0610009D07Rik         Sf3b6&#xA;4  0610009L18Rik 0610009L18Rik&#xA;5  0610009O20Rik         Dele1&#xA;6  0610010B08Rik        Gm2026&#xA;7  0610010F05Rik 0610010F05Rik&#xA;8  0610010K14Rik 0610010K14Rik&#xA;9  0610011F06Rik       Mettl26&#xA;10 0610012G03Rik 0610012G03Rik&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In the cases when they don't map 1:1, the function will throw a warning, and you can access it by &lt;code&gt;warnings()&lt;/code&gt;. It is rarely important though.&lt;/p&gt;&#xA;&lt;p&gt;Hope this answers the question.&lt;/p&gt;&#xA;" OwnerUserId="7056" LastActivityDate="2021-03-19T01:16:58.970" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15576" PostTypeId="1" AcceptedAnswerId="15577" CreationDate="2021-03-19T06:44:32.993" Score="3" ViewCount="425" Body="&lt;p&gt;I have several bam files from unknown origin. I'd like to know what software was used to obtain those bam files, e.g. &lt;code&gt;bwa-mem&lt;/code&gt;. Is it possible to somehow check it?&lt;/p&gt;&#xA;" OwnerUserId="2869" LastEditorUserId="6251" LastEditDate="2021-03-20T11:33:32.823" LastActivityDate="2021-03-20T11:33:32.823" Title="How to check what software has been used to generate bam file?" Tags="|sam|ngs|read-mapping|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15577" PostTypeId="2" ParentId="15576" CreationDate="2021-03-19T08:03:04.353" Score="6" Body="&lt;p&gt;If you're lucky the information is stored in the BAM header in the lines starting with &lt;code&gt;@PG&lt;/code&gt; which stores the command lines run on the BAM, use &lt;code&gt;samtools view -H your.bam&lt;/code&gt; to display the header.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-03-19T08:03:04.353" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15578" PostTypeId="1" AcceptedAnswerId="15580" CreationDate="2021-03-19T08:55:07.030" Score="1" ViewCount="1081" Body="&lt;p&gt;Hi I was trying to use plink in nextflow and getting the error mentioned in the subject. Here is my code, any help what I am missing. I am new in nextflow, it was ok when input was a single file, but for grouped files, I am getting lost.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params.dir='data/'&#xA;params.publish=&amp;quot;output/&amp;quot;&#xA;Channel&#xA;.fromFilePairs(&amp;quot;${params.dir}*.{bed,fam,bim}&amp;quot;,size:3){file -&amp;gt; file.baseName}&#xA;.set {plink-data}&#xA;    &#xA;process test1{&#xA;publishDir params.publish&#xA;input:&#xA;set t1,file(plk1) from plink-data&#xA;output:&#xA;file &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${t1}.{bed,fam,bim}&quot; into test1_results&#xA;script:&#xA;output_file = &quot;$&lt;/span&gt;{t1}&amp;quot;&#xA;base = plk1[0].baseName&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;module load plink&#xA;plink --bfile &lt;span class=&quot;math-container&quot;&gt;$t1 --maf 0.05 --make-bed --out $&lt;/span&gt;{output_file}&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;process test2{&#xA;publishDir params.publish&#xA;input:&#xA;set file(bed), file(fam), file(bim) from test1_results.collect()&#xA;output:&#xA;set t2,file '&lt;span class=&quot;math-container&quot;&gt;${plk2}.{bed,fam,bim}' into test2_results&#xA;script:&#xA;output_file = &quot;{t2}&quot;&#xA;base = plk2[0].baseName&#xA;&quot;&quot;&quot;&#xA;plink --bfile $&lt;/span&gt;t2 --maf 0.01 --make-bed --out ${output_file}&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3839" LastActivityDate="2021-03-20T03:21:14.817" Title="Nextflow error: can not invoke method minus () on null object" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15580" PostTypeId="2" ParentId="15578" CreationDate="2021-03-19T13:33:33.857" Score="2" Body="&lt;p&gt;Channel names must consist only alphanumeric characters including underscores. Your channel called 'plink-data' contains a hyphen. This is problematic because it is interpreted as a minus() operation to remove 'data' from 'plink'. You get a NullPointerException since 'plink' hasn't been defined. You can test this easily with the following example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Channel&#xA;    .of(1..23, 'X', 'Y')&#xA;    .set { chr-names }&#xA;&#xA;chr-names.view()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Cannot invoke method minus() on null object&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The solution is to just use an underscore instead of a hyphen:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Channel&#xA;    .of(1..23, 'X', 'Y')&#xA;    .set { chr_names }&#xA;&#xA;chr_names.view()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Follow-up issues:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I think if you want to use the bfiles plink option (and I think this is reasonable), I think it makes the most sense to just keep your bfiles in a tuple:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Channel&#xA;    .fromFilePairs( &amp;quot;${params.input_dir}/*.{bed,bim,fam}&amp;quot;, size: 3 )&#xA;    .set { plink_data }&#xA;&#xA;&#xA;process test1{&#xA;&#xA;    publishDir params.publish_dir&#xA;&#xA;    module 'plink'&#xA;&#xA;    input:&#xA;    tuple val(sample_name), path(bfiles) from plink_data&#xA;&#xA;    output:&#xA;    tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_name}-t1&quot;), path(&quot;$&lt;/span&gt;{sample_name}-t1.{bed,bim,fam}&amp;quot;) into test1_results&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    plink \\&#xA;        --bfile &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_name}&quot; \\&#xA;        --maf 0.05 \\&#xA;        --make-bed \\&#xA;        --out &quot;$&lt;/span&gt;{sample_name}-t1&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process test2{&#xA;&#xA;    publishDir params.publish_dir&#xA;&#xA;    module 'plink'&#xA;&#xA;    input:&#xA;    tuple val(sample_name), path(bfiles) from test1_results&#xA;&#xA;    output:&#xA;    tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_name}-t2&quot;), path(&quot;$&lt;/span&gt;{sample_name}-t2.{bed,bim,fam}&amp;quot;) into test2_results&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    plink \\&#xA;        --bfile &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_name}&quot; \\&#xA;        --maf 0.01 \\&#xA;        --make-bed \\&#xA;        --out &quot;$&lt;/span&gt;{sample_name}-t2&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you need to reference each bfile separately, you could just unpack the tuple and reference each file accordingly:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Channel&#xA;    .fromFilePairs( &amp;quot;${params.input_dir}/*.{bed,bim,fam}&amp;quot;, size: 3 )&#xA;    .map { group_key, files -&amp;gt; tuple( group_key, *files ) }&#xA;    .set { plink_data }&#xA;&#xA;&#xA;process test1{&#xA;&#xA;    publishDir params.publish_dir&#xA;&#xA;    module 'plink'&#xA;&#xA;    input:&#xA;    tuple val(sample_name), path(bed), path(bim), path(fam) from plink_data&#xA;&#xA;    output:&#xA;    tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_name}-t1&quot;), path(&quot;$&lt;/span&gt;{sample_name}-t1.{bed,bim,fam}&amp;quot;) into test1_results&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    plink \\&#xA;        --bed &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${bed}&quot; \\&#xA;        --bim &quot;$&lt;/span&gt;{bim}&amp;quot; \\&#xA;        --fam &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${fam}&quot; \\&#xA;        --maf 0.05 \\&#xA;        --make-bed \\&#xA;        --out &quot;$&lt;/span&gt;{sample_name}-t1&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process test2{&#xA;&#xA;    publishDir params.publish_dir&#xA;&#xA;    module 'plink'&#xA;&#xA;    input:&#xA;    tuple val(sample_name), path(bed), path(bim), path(fam) from test1_results&#xA;&#xA;    output:&#xA;    tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_name}-t2&quot;), path(&quot;$&lt;/span&gt;{sample_name}-t2.{bed,bim,fam}&amp;quot;) into test2_results&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    plink \\&#xA;        --bed &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${bed}&quot; \\&#xA;        --bim &quot;$&lt;/span&gt;{bim}&amp;quot; \\&#xA;        --fam &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${fam}&quot; \\&#xA;        --maf 0.01 \\&#xA;        --make-bed \\&#xA;        --out &quot;$&lt;/span&gt;{sample_name}-t2&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that in the first example, &lt;code&gt;bfiles&lt;/code&gt; could be replaced with &lt;code&gt;&amp;quot;*&amp;quot;&lt;/code&gt; since no reference is made to the tuple of bfiles anyway. Also note that the list of files produced using the &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#fromfilepairs&quot; rel=&quot;nofollow noreferrer&quot;&gt;fromFilesPairs operator&lt;/a&gt; are returned in lexicographic order.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2021-03-20T03:21:14.817" LastActivityDate="2021-03-20T03:21:14.817" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="15581" PostTypeId="2" ParentId="13561" CreationDate="2021-03-20T00:28:52.787" Score="0" Body="&lt;p&gt;It depends on how many samples you have.&#xA;If you only compare one sample to the second, you can pool and call peaks and then directly calculate RPKM ratios for each peak.&#xA;It you have more samples, you may want to use IDR to score differential peaks.&#xA;You may also use packages such as Fseq or HOMER to treat one group as ChIP experiment while the other group as control to call peaks.&#xA;Alternatively, you could pool all the samples to call peaks, then calculate read counts in each peak for each sample, and then use DESeq2 or other differential tools to calculate significantly different peaks’ significance and effect size.&lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2021-03-20T00:28:52.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15582" PostTypeId="1" CreationDate="2021-03-20T05:15:45.887" Score="2" ViewCount="957" Body="&lt;p&gt;I have a .bam alignment file and a genome reference .fasta file. I am looking for a easy to use tool (that I can reference in a publication) to calculate the percentage coverage of the reference by the alignment and if possible also: average depth of reads that cover the gene, min/max depths. Running plot would be nice but not essential.&lt;/p&gt;&#xA;&lt;p&gt;I wrote some python code to calculate coverage percent, but I want to reference the tool I use in a paper and dont want to mess with making a git repo and linking to it in the paper just for this simple analysis.&lt;/p&gt;&#xA;" OwnerUserId="10383" LastActivityDate="2021-07-25T01:51:36.163" Title="Calculate genome coverage and depth from alignment" Tags="|rna-seq|sequence-alignment|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15583" PostTypeId="2" ParentId="15582" CreationDate="2021-03-21T03:45:49.677" Score="0" Body="&lt;p&gt;I've used &lt;a href=&quot;https://genome.sph.umich.edu/wiki/BamUtil&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genome.sph.umich.edu/wiki/BamUtil&lt;/a&gt; before. I think this has all the functionality you want and the Devs are responsive which is nice.&lt;/p&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-03-21T03:45:49.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15584" PostTypeId="1" AcceptedAnswerId="15589" CreationDate="2021-03-21T17:04:24.133" Score="3" ViewCount="1149" Body="&lt;p&gt;I was trying to use plink in nextflow. My first process will split the input (single bed|bim|fam file) into 22 chromosomes (each with bed|bim|fam files) which would be used in subsequent processes. My code and error log is below. Any help?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params.dir='data/'&#xA;params.publish=&amp;quot;output2/&amp;quot;&#xA;Channel&#xA;.fromFilePairs(&amp;quot;${params.dir}*.{bed,bim,fam}&amp;quot;,size:3)&#xA;.set {data}&#xA;&#xA;Channel&#xA;.of(1..22)&#xA;.set {chr_names}&#xA;    &#xA;process step1a {&#xA;publishDir params.publish&#xA;input:&#xA;tuple val(sample_name), path(bfiles) from data&#xA;output:&#xA;tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}&quot;), path(&quot;$&lt;/span&gt;{prefix}.step1a.{bed,bim,fam}&amp;quot;) into step1a_results&#xA;tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}&quot;), path(&quot;$&lt;/span&gt;{prefix}.step1b.{bed,bim,fam}&amp;quot;) into step1a_out&#xA;tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}&quot;), path(&quot;$&lt;/span&gt;{prefix}.step1b.bim&amp;quot;) into step1a_bims&#xA;path(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}.log&quot;) into step1a_logs&#xA;path(&quot;$&lt;/span&gt;{prefix}.step1a.dupvar&amp;quot;) into step1a_duplicates&#xA;path(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}.step1a.log&quot;) into step1a_log&#xA;path(&quot;$&lt;/span&gt;{prefix}.step1b.log&amp;quot;) into step1_log&#xA;script:&#xA;prefix=chr&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chr_names}&quot;&#xA;&quot;&quot;&quot;&#xA;plink --bfile &quot;$&lt;/span&gt;{sample_name}&amp;quot; --chr &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chr_names}&quot; --out &quot;$&lt;/span&gt;{prefix}&amp;quot; --make-bed&#xA;plink --bfile &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}&quot; --list-duplicate-vars ids-only suppress-first --out &quot;$&lt;/span&gt;{prefix}.step1a&amp;quot;&#xA;plink --bfile &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}&quot; --exclude &quot;$&lt;/span&gt;{prefix}.step1a.dupvar&amp;quot; --make-bed --out &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}.step1a&quot;&#xA;plink --bfile &quot;$&lt;/span&gt;{prefix}.step1a&amp;quot; --snps-only just-acgt --make-bed --out &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}.step1b&quot;&#xA;&quot;&quot;&quot;&#xA;}&#xA;process step1b {&#xA;publishDir params.publish&#xA;input:&#xA;tuple val(sample_name), path(bfiles) from step1a_out&#xA;tuple val(bims), path(&quot;${prefix}.step1b.bim&quot;) from step1a_bims&#xA;output:&#xA;tuple val(&quot;${sample_name}&quot;), path(&quot;${sample_name}.step1c.{bed,bim,fam}&quot;) into step1b_results&#xA;path(&quot;${sample_name}.step1c.log&quot;) into step1b_logs&#xA;path(&quot;${sample_name}.step1b.snplist.txt&quot;) into step1b_snplist&#xA;script:&#xA;&quot;&quot;&quot;&#xA;awk '{ if ((\\\$5==&quot;T&quot; &amp;amp;&amp;amp; \\\$6==&quot;A&quot;)||(\\\$5==&quot;A&quot; &amp;amp;&amp;amp; \\\$6==&quot;T&quot;)||(\\\$5==&quot;C&quot; &amp;amp;&amp;amp; \\\$6==&quot;G&quot;)||(\\\$5==&quot;G&quot; &amp;amp;&amp;amp; \\\$6==&quot;C&quot;)) print \\\$2, &quot;ambig&quot; ; else print \\\$2 ;}' &quot;${prefix}.step1b.bim&quot; | grep ambig &amp;gt; &quot;${prefix}.step1b.snplist.txt&quot;&#xA;plink --bfile &quot;${prefix}.step1b --exclude &quot;${prefix}.step1b.snplist.txt&quot; --make-bed --out &quot;$&lt;/span&gt;{prefix}.step1c&amp;quot;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The error message is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;N E X T F L O W  ~  version 20.10.0&#xA;Launching `t4.hf` [zen_picasso] - revision: b0ff8db413&#xA;[-        ] process &amp;gt; step1a -&#xA;[-        ] process &amp;gt; step1b -&#xA;Error executing process &amp;gt; 'step1a (1)'&#xA;&#xA;Caused by:&#xA;[-        ] process &amp;gt; step1a -&#xA;[-        ] process &amp;gt; step1b -&#xA;Error executing process &amp;gt; 'step1a (1)'&#xA;&#xA;Caused by:&#xA;  No signature of method: chr() is applicable for argument types: (org.codehaus.groovy.runtime.GStringImpl) values: [DataflowQueue(queue=[DataflowVariable(value=1), DataflowVariable(value=2), DataflowVariable(value=3), DataflowVariable(value=4), DataflowVariable(value=5), DataflowVariable(value=6), DataflowVariable(value=7), DataflowVariable(value=8), DataflowVariable(value=9), DataflowVariable(value=10), DataflowVariable(value=11), DataflowVariable(value=12), DataflowVariable(value=13), DataflowVariable(value=14), DataflowVariable(value=15), DataflowVariable(value=16), DataflowVariable(value=17), DataflowVariable(value=18), DataflowVariable(value=19), DataflowVariable(value=20), DataflowVariable(value=21), DataflowVariable(value=22), DataflowVariable(value=groovyx.gpars.dataflow.operator.PoisonPill@28c22659)])]&#xA;Possible solutions: run(), run(), any(), is(java.lang.Object), use([Ljava.lang.Object;), tap(groovy.lang.Closure)&#xA;&#xA;Source block:&#xA;  prefix=chr&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chr_names}&quot;&#xA;  &quot;&quot;&quot;&#xA;  plink --bfile &quot;$&lt;/span&gt;{sample_name}&amp;quot; --chr &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chr_names}&quot; --out &quot;$&lt;/span&gt;{prefix}&amp;quot; --make-bed&#xA;  plink --bfile &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}&quot; --list-duplicate-vars ids-only suppress-first --out &quot;$&lt;/span&gt;{prefix}.step1a&amp;quot;&#xA;  plink --bfile &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}&quot; --exclude &quot;$&lt;/span&gt;{prefix}.step1a.dupvar&amp;quot; --make-bed --out &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}.step1a&quot;&#xA;  plink --bfile &quot;$&lt;/span&gt;{prefix}.step1a&amp;quot; --snps-only just-acgt --make-bed --out &amp;quot;${prefix}.step1b&amp;quot;&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;Tip: when you have fixed the problem you can continue the execution adding the option `-resume` to the run command line&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If I use &amp;quot;prefix=chr&amp;quot;${chr_names}&amp;quot; &amp;quot; inside the code block (&amp;quot;&amp;quot;&amp;quot;) it shows: unknown variable 'prefix'&lt;/p&gt;&#xA;" OwnerUserId="3839" LastEditorUserId="3839" LastEditDate="2021-03-21T18:10:58.777" LastActivityDate="2021-03-22T01:43:30.723" Title="Nextflow: No signature of Method: chr() is applicable for argument types; unknown variable" Tags="|plink|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15586" PostTypeId="2" ParentId="15582" CreationDate="2021-03-21T19:23:10.480" Score="2" Body="&lt;p&gt;You could use &lt;code&gt;samtools coverage&lt;/code&gt; as explained in the &lt;a href=&quot;http://www.htslib.org/doc/samtools-coverage.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt; of samtoools.&lt;/p&gt;&#xA;&lt;p&gt;Here is a example which is also described on the manual site.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools coverage -r chr1:1M-12M input.bam&#xA;&#xA;#rname  startpos  endpos    numreads  covbases  coverage  meandepth  meanbaseq  meanmapq&#xA;chr1    1000000   12000000  528695    1069995   9.72723   3.50281    34.4       55.8&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3360" LastEditorUserId="3360" LastEditDate="2021-03-22T19:01:19.597" LastActivityDate="2021-03-22T19:01:19.597" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15587" PostTypeId="2" ParentId="3679" CreationDate="2021-03-21T19:41:00.850" Score="1" Body="&lt;p&gt;I wrote a simple script to do it. Not sure whether that's what you want.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;#CellrangerSummary.sh Sample1 Sample2&#xA;csvtool transpose &lt;span class=&quot;math-container&quot;&gt;$1/outs/*summary.csv | csvcut -c 1 &amp;gt; summary.csv&#xA;for sample in &quot;$&lt;/span&gt;@&amp;quot;&#xA;    do&#xA;        csvtool transpose &lt;span class=&quot;math-container&quot;&gt;$sample/outs/*summary.csv &amp;gt; tempfilesforcsvtool.$&lt;/span&gt;sample&#xA;        csvjoin -c 1 summary.csv tempfilesforcsvtool.&lt;span class=&quot;math-container&quot;&gt;$sample &amp;gt; temp2&#xA;    mv temp2 summary.csv&#xA;rm tempfilesforcsvtool.$&lt;/span&gt;sample&#xA;    done&#xA;&#xA;csvformat -T summary.csv &amp;gt; summary.tsv&#xA;rm summary.csv&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1852" LastEditorUserId="1852" LastEditDate="2021-03-22T22:53:48.450" LastActivityDate="2021-03-22T22:53:48.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15588" PostTypeId="2" ParentId="2644" CreationDate="2021-03-21T21:13:48.720" Score="0" Body="&lt;p&gt;This online tool creates download links for the less tech savy:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://sra-explorer.info/#&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sra-explorer.info/#&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Using PRJNA201245 as ID, it provides several codes including a bash script for downloading all files.&lt;/p&gt;&#xA;" OwnerUserId="10782" LastActivityDate="2021-03-21T21:13:48.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15589" PostTypeId="2" ParentId="15584" CreationDate="2021-03-22T01:43:30.723" Score="4" Body="&lt;p&gt;The documation re &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#conditional-scripts&quot; rel=&quot;nofollow noreferrer&quot;&gt;conditional scripts&lt;/a&gt; is useful here:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Process scripts can contain conditional statements by simply prefixing&#xA;the script block with the keyword &lt;code&gt;script:&lt;/code&gt;. By doing that the&#xA;interpreter will evaluate all the following statements as a code block&#xA;that must return the script string to be executed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But the statements don't necessarily need to be conditional. I.e. they can just be declarations like in your example. You get the error 'No signature of Method: chr()' because Nextflow interprets 'chr' as a method call when you use:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;prefix = chr&amp;quot;${chr_name}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If your intention is to simply add the 'chr' prefix to each chromosome name, you'd need:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;prefix = &amp;quot;chr${chr_name}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Notice that I changed your variable name. This is because in your code, you still need to feed in the channel of chromosome names. The easiest way to do this is to use the &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html?highlight=script#input-repeaters&quot; rel=&quot;nofollow noreferrer&quot;&gt;each qualifier&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process myprocess {&#xA;&#xA;    input:&#xA;    tuple val(sample_name), path(bfiles) from plink_data&#xA;    each chr_name from chr_names&#xA;&#xA;    script:&#xA;    prefix = &amp;quot;chr${chr_name}&amp;quot;&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    echo &amp;quot;My sample name is: &lt;span class=&quot;math-container&quot;&gt;${sample_name}&quot;&#xA;    echo &quot;My prefixed chromosome name is: $&lt;/span&gt;{prefix}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Unless you need chromosome names without the 'chr' prefix, another solution might be to simply add the prefix to each one initially and simply avoid having to prefix them later:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Channel&#xA;    .of(1..22)&#xA;    .map { &amp;quot;chr${it}&amp;quot; }&#xA;    .set { chr_names }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2021-03-22T01:43:30.723" CommentCount="14" ContentLicense="CC BY-SA 4.0" />
  <row Id="15590" PostTypeId="1" AcceptedAnswerId="15594" CreationDate="2021-03-22T12:04:38.850" Score="1" ViewCount="1294" Body="&lt;p&gt;I need to build a pipeline where 2nd process starts after completion of the 1st process and the 3rd after 2nd. As I understand nextflow's processes run parallel. But I need it to run in a order 1st&amp;gt;2nd&amp;gt;3rd processes and so on. Because each process runs on an array and produce multiple outputs which are used by the next. If they run paralleled, the 2nd process may look for input 20 whereas the 1st process produced output 10 and then the process stops, because the command in the 2nd process does not find the input. Is there any way to accomplish this. I tried as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process A{&#xA;input:&#xA;tuple val(some_name), path(some path) from some_channel&#xA;output:&#xA;tuple val(some val), path(some path) into A_some_ch&#xA;tuple va(some other va), path(some other path) into A_some_other_ch&#xA;val true into done_ch&#xA;script:&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;some command out some files&#xA;some other command out some other files&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;process B {&#xA;input:&#xA;val flag from done_ch&#xA;val (some val), path(some path) from A_some_ch&#xA;val (some other val), path(some other path) from A_some_other_ch&#xA;output:&#xA;some val and path&#xA;script:&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;some commands out some files&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I got the idea using of flag from here: &lt;a href=&quot;https://github.com/nextflow-io/patterns/blob/master/docs/mock-dependency.adoc&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nextflow-io/patterns/blob/master/docs/mock-dependency.adoc&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But if I use the flag, I got:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Operation 'set' does not allow more than one target name&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does it clarify the issue? Any Help?&#xA;Best, Zillur&lt;/p&gt;&#xA;" OwnerUserId="3839" LastActivityDate="2021-03-22T15:35:16.653" Title="Nextflow: ordering the execution of processes; outputs of process A are inputs of process B" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15591" PostTypeId="1" AcceptedAnswerId="15643" CreationDate="2021-03-22T13:13:43.003" Score="3" ViewCount="1409" Body="&lt;p&gt;I'm reading the Manta &lt;a href=&quot;https://github.com/Illumina/manta/blob/master/docs/userGuide/README.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;User Guide&lt;/a&gt;&#xA;→ it is a method to discover structural variants and indels from next generation sequencing data.&lt;/p&gt;&#xA;&lt;p&gt;The word 'breakend' is used in several contexts&lt;/p&gt;&#xA;&lt;p&gt;I have searched for a definition in &lt;code&gt;duckduckgo&lt;/code&gt;, &lt;code&gt;qwant&lt;/code&gt; and &lt;code&gt;google&lt;/code&gt; but have not found a concise definition.&lt;/p&gt;&#xA;&lt;p&gt;Thanks for you help&lt;/p&gt;&#xA;" OwnerUserId="10791" LastActivityDate="2023-11-08T21:26:45.220" Title="What is the exact definition of &quot;breakend&quot;?" Tags="|structural-variation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15592" PostTypeId="2" ParentId="15573" CreationDate="2021-03-22T13:51:44.683" Score="1" Body="&lt;p&gt;The ID history converter is available as a &lt;a href=&quot;https://github.com/Ensembl/ensembl-tools/tree/release/103/scripts/id_history_converter&quot; rel=&quot;nofollow noreferrer&quot;&gt;Perl script&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This script accesses these &lt;a href=&quot;http://www.ensembl.org/info/docs/api/core/core_schema.html#gene_archive&quot; rel=&quot;nofollow noreferrer&quot;&gt;MySQL tables&lt;/a&gt;, so you could query these tables directly.&lt;/p&gt;&#xA;&lt;p&gt;Note that the mapping session is only one release at a time, so you would need to run 25 queries to go from release 75 to 100.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastEditorUserId="1332" LastEditDate="2021-03-22T13:58:59.217" LastActivityDate="2021-03-22T13:58:59.217" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15593" PostTypeId="1" AcceptedAnswerId="15596" CreationDate="2021-03-22T14:49:09.353" Score="1" ViewCount="600" Body="&lt;p&gt;I am trying to use GATK IndexFeatureFile to create &amp;quot;.vcf.gz.tbi&amp;quot; file. But I have 480 files.&lt;/p&gt;&#xA;&lt;p&gt;For the pilot study I used:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;!./gatk IndexFeatureFile -I HG00096.chrom22.bam.10061.vcf.gz -O HG00096.chrom22.bam.10061.vcf.gz.tbi&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Doing this procedure for 480 files is not seem very practical. Is there any way to use all file as an input and it takes all files and gives &amp;quot;.tbi&amp;quot; files as an output?&lt;/p&gt;&#xA;&lt;p&gt;I thought&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;!./gatk IndexFeatureFile -I *.vcf.gz -O *.vcf.gz.tbi&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or maybe not using output argument (just input). I did not worked for me.&lt;/p&gt;&#xA;" OwnerUserId="1757" LastActivityDate="2021-03-22T17:48:08.870" Title="GATK 4.2 using IndexFeatureFile with multiple files" Tags="|vcf|gatk|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15594" PostTypeId="2" ParentId="15590" CreationDate="2021-03-22T15:35:16.653" Score="1" Body="&lt;p&gt;Nextflow uses &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;channels&lt;/em&gt;&lt;/a&gt; (asynchronous &lt;a href=&quot;https://en.wikipedia.org/wiki/FIFO_(computing_and_electronics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;FIFO&lt;/a&gt; queues) to communicate between processes. The interaction between these processes, and ultimately the pipeline execution flow itself, is implicitly defined by these input and output declarations.&lt;/p&gt;&#xA;&lt;p&gt;The issue with your code, I think, is the use of &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#understand-how-multiple-input-channels-work&quot; rel=&quot;nofollow noreferrer&quot;&gt;multiple input channels&lt;/a&gt;. Consider the following example that uses only a single &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#queue-channel&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;queue&lt;/em&gt;&lt;/a&gt; channel. To avoid declaring the inputs again in process A's output declaration, we can output a key to join on as the first element in the output tuple. In the case below, this is simply the 'some_name' variable. We can then join up the two channels using the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#join&quot; rel=&quot;nofollow noreferrer&quot;&gt;join operator&lt;/a&gt; as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Channel&#xA;    .fromFilePairs( './data/*.txt', size:1 )&#xA;    .map { group_key, files -&amp;gt; tuple( group_key, *files ) }&#xA;    .into { input_ch_A; input_ch_B }&#xA;&#xA;&#xA;process A {&#xA;&#xA;    input:&#xA;    tuple val(some_name), path(some_path) from input_ch_A&#xA;&#xA;    output:&#xA;    tuple val(some_name), val(&amp;quot;procA_&lt;span class=&quot;math-container&quot;&gt;${some_name}&quot;), path(&quot;procA_$&lt;/span&gt;{some_name}&amp;quot;) \&#xA;        into procA_results&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    touch &amp;quot;procA_${some_name}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process B {&#xA;&#xA;    input:&#xA;    tuple val(some_name), path(some_path), val(some_other_val), path(some_other_path) \&#xA;        from input_ch_B.join(procA_results)&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    true&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2021-03-22T15:35:16.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15596" PostTypeId="2" ParentId="15593" CreationDate="2021-03-22T17:48:08.870" Score="2" Body="&lt;p&gt;You could run loop over the files in your shell, running a separate command for each file. For example, in &lt;a href=&quot;https://www.gnu.org/software/bash/manual/html_node/Looping-Constructs.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bash&lt;/a&gt; (or another Bourne shell):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for file in *.vcf.gz; do ./gatk IndexFeatureFile -I &lt;span class=&quot;math-container&quot;&gt;${file} -O $&lt;/span&gt;{file}.tbi; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3351" LastActivityDate="2021-03-22T17:48:08.870" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15597" PostTypeId="1" CreationDate="2021-03-22T17:52:46.500" Score="0" ViewCount="783" Body="&lt;p&gt;How could I filter outputs of a process in the input of the next process? Filtering works fine in channel, but if I try to filter outputs I got compilation error. I tried as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Channel&#xA;.fromFilePairs(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${params.dir}*.{bed,bim,fam}&quot;,size:3)&#xA;.set {data}&#xA;    process step1a {&#xA;    input:&#xA;    tuple val(sample_name), path(bfiles) from data&#xA;    output:&#xA;    tuple val(&quot;${prefix}&quot;), path(&quot;${prefix}.{bed,bim,fam}&quot;) into (step1a_out,step1a_bim)&#xA;    script:&#xA;    &quot;&quot;&quot;&#xA;    plink --bfile &quot;${sample_name}&quot; --chr 1-22 --out &quot;${prefix}.step1b&quot; --make-bed&#xA;    &quot;&quot;&quot;&#xA;    }&#xA;    process step1b {&#xA;    input:&#xA;    tuple val(bim), path(bims) from step1a_bim.filter{bim, files -&amp;gt; name =~/*bim/}&#xA;    tuple val(sample_name), path(bfiles) from step1a_out&#xA;    output:&#xA;    path(&quot;${prefix}.step1b.snplist.txt&quot;)&#xA;    tuple val(&quot;${sample_name}&quot;), path(&quot;${sample_name}.step1c.{bed,bim,fam}&quot;) into step1b_results&#xA;    script:&#xA;    &quot;&quot;&quot;&#xA;    awk '{ if ((\\\$5==&quot;T&quot; &amp;amp;&amp;amp; \\\$6==&quot;A&quot;)||(\\\$5==&quot;A&quot; &amp;amp;&amp;amp; \\\$6==&quot;T&quot;)||(\\\$5==&quot;C&quot; &amp;amp;&amp;amp; \\\$6==&quot;G&quot;)||(\\\$5==&quot;G&quot; &amp;amp;&amp;amp; \\\$6==&quot;C&quot;)) print \\\$2, &quot;ambig&quot; ; else print \\\$2 ;}' &quot;${prefix}.step1b.bim&quot; | grep ambig &amp;gt; &quot;${prefix}.step1b.snplist.txt&quot;&#xA;    plink --bfile &quot;${prefix}.step1b&quot; --exclude &quot;${prefix}.step1b.snplist.txt&quot; --make-bed --out &quot;$&lt;/span&gt;{prefix}.step1c&amp;quot;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It gives me&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Compilation error&#xA;-cause: Unexpected input: '{'&#xA;process step1b {&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Step1a gives me chr1.step1b.bed|bim|fam, chr2.step1b.bed|bim|fam.......chr22.step1b.bed|bim|fam (22 plink files)&#xA;I need to access prefix of those outputs to run plink and other tools to generate chr1.step1c.bed|bim|fam ...chr22.step1c.bed|bim|fam (another 22 plink files). run awk to generate chr1.step1b.snplist.txt .... chr22.step1b.snplist.txt (22 txt files).&#xA;Any help to filter the inputs so that I could use only *bim files from step1a_bim? Also how to select prefix of files in the input of processes that are from step1a_out to run plink? Does it make sense now?&lt;/p&gt;&#xA;&lt;p&gt;Best Reagrds&#xA;Zillur&lt;/p&gt;&#xA;" OwnerUserId="3839" LastEditorUserId="3839" LastEditDate="2021-03-23T13:03:34.250" LastActivityDate="2021-12-18T15:05:05.470" Title="nextflow: Filter outputs of a process" Tags="|plink|nextflow|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15598" PostTypeId="1" AcceptedAnswerId="15605" CreationDate="2021-03-22T19:03:15.253" Score="1" ViewCount="639" Body="&lt;p&gt;I obtained a matrix of RNA-seq count data that has been normalized by DESeq2's median of ratio method.&lt;/p&gt;&#xA;&lt;p&gt;I know that DESeq2 wants to take in un-normalized counts, but I do not have access to those data.&lt;/p&gt;&#xA;&lt;p&gt;How do I best proceed here if I want to perform DEG analysis using DESeq2?&lt;/p&gt;&#xA;&lt;p&gt;I know I can always start from .fastq files, but that would be so much extra work.&lt;/p&gt;&#xA;&lt;p&gt;I don't think I can un-normalize the data because I don't know the scaling factors.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="7056" LastActivityDate="2021-03-23T17:31:40.187" Title="Un-normalize DESeq2 counts" Tags="|rna-seq|deseq2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15599" PostTypeId="2" ParentId="14083" CreationDate="2021-03-22T19:11:48.430" Score="0" Body="&lt;p&gt;Stringtie now does long reads too. We've found flair to be quite difficult to install and to crash often. That said, stringtie isn't very good about showing you its working (e.g. I have no idea how to get it to give the assembly it uses to figure out all the isoforms). Its values for TPM/FPKM also seem a little weird, in that they're perfectly correlated with each other, and for transcripts they're perfectly correlated with coverage too.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1910-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1910-1&lt;/a&gt;&#xA;&lt;a href=&quot;https://github.com/gpertea/stringtie&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/gpertea/stringtie&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;TALON is another tool that does isoform assembly. I'm not sure whether it does quantification though.&#xA;&lt;a href=&quot;https://github.com/mortazavilab/TALON#&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mortazavilab/TALON#&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3860" LastActivityDate="2021-03-22T19:11:48.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15600" PostTypeId="2" ParentId="15598" CreationDate="2021-03-22T21:37:54.213" Score="0" Body="&lt;p&gt;You might be able to cheat by giving DESeq a custom size factor matrix where all the values are 1.  Then the software will feel like it's done the normalization step, but the 'normalized' values will be what you want.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2021-03-22T21:37:54.213" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15601" PostTypeId="1" CreationDate="2021-03-23T04:30:38.193" Score="0" ViewCount="626" Body="&lt;p&gt;I am using a pipeline (bam -&amp;gt; featurecount-&amp;gt; EdgeR) to do some RNASeq analysis of several groups and sub-groups. For example, I have the following dataset with two types (T1 and T2) and T1 has three groups G1, G2 and G3, T2 has G5 and G6 groups. I would like to find TPM numbers, fold-change and FDR for all the genes and Groups.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Type Group SRA_Run#&#xA;T1   G1.   &amp;lt;some SRA ids e.g. SRR123456, SRR112233&amp;gt;&#xA;T1   G2.   &amp;lt;some SRA&amp;gt;&#xA;T1.  G3.   &amp;lt;some SRA&amp;gt;&#xA;T2.  G5.   &amp;lt;some SRA&amp;gt;&#xA;T2.  G6.   &amp;lt;some SRA&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have individual BAM files generated for each SRA Run. I also individual read counts from featureCount for each SRA Run. I was not sure if I need to merge individual read counts for each group or run featureCount using all bam files of a group at once. Does it make any difference in  the analysis that follows.&lt;/p&gt;&#xA;&lt;p&gt;I also would like to know if I can run EdgeR at once for all the types and groups at once for doing differential gene expression for each group of Type 1 (T1) against every other group of Type 2 (T2). I am bit stuck at this point how to proceed. Any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2594" LastEditorUserId="2594" LastEditDate="2021-03-23T04:48:41.940" LastActivityDate="2021-03-23T07:32:49.440" Title="RNASeq analysis using featureCount and EdgeR" Tags="|rna-seq|edger|featurecounts|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15602" PostTypeId="2" ParentId="15601" CreationDate="2021-03-23T07:32:49.440" Score="1" Body="&lt;p&gt;It makes no difference if you process the BAM files one at a time with featureCounts or all together, except that it changes how you have to read the files into R.&lt;/p&gt;&#xA;&lt;p&gt;You can supply edgeR with lists of contrasts to have it compute fold-changes and p-values for. Please have a look at the &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/vignettes/edgeR/inst/doc/edgeRUsersGuide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;edgeR user guide&lt;/a&gt; for examples.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-03-23T07:32:49.440" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15603" PostTypeId="2" ParentId="15597" CreationDate="2021-03-23T14:03:12.650" Score="1" Body="&lt;p&gt;Instead of outputting all files and filtering for only bim, output only bims:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;output:&#xA;tuple val(&amp;quot;${prefix}&amp;quot;), path(&amp;quot;${prefix}.{bed,bim,fam}&amp;quot;) into (step1a_out,step1a_bim)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Do&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;output:&#xA;tuple val(&amp;quot;${prefix}&amp;quot;), path(&amp;quot;${prefix}.{bed,bim,fam}&amp;quot;) into step1a_out&#xA;tuple val(&amp;quot;${prefix}&amp;quot;), path(&amp;quot;${prefix}.bim&amp;quot;) into step1a_bim&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So you can skip your complicated filter:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process step1b {&#xA;    input:&#xA;    tuple val(bim), path(bimfile) from step1a_bim&#xA;    output:&#xA;    path(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}.step1b.snplist.txt&quot;)&#xA;tuple val(&quot;$&lt;/span&gt;{sample_name}&amp;quot;), path(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_name}.step1c.{bed,bim,fam}&quot;) into &#xA;step1b_results&#xA;    script:&#xA;    &quot;&quot;&quot;&#xA;    awk '{ if ((\\\\\$5==&quot;T&quot; &amp;amp;&amp;amp; \\\\\$6==&quot;A&quot;)||(\\\\\$5==&quot;A&quot; &amp;amp;&amp;amp; \\\\\$6==&quot;T&quot;)||(\\\\\$5==&quot;C&quot; &amp;amp;&amp;amp; \\\\\$6==&quot;G&quot;)||(\\\\\$5==&quot;G&quot; &amp;amp;&amp;amp; \\\\\$6==&quot;C&quot;)) print \\\\\$2, &quot;ambig&quot; ; else print \\\\\$2 ;}' &quot;$&lt;/span&gt;bimfile&amp;quot; | grep ambig &amp;gt; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}.step1b.snplist.txt&quot;&#xA;    plink --bfile &quot;$&lt;/span&gt;{prefix}.step1b&amp;quot; --exclude &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}.step1b.snplist.txt&quot; --make-bed --out &quot;$&lt;/span&gt;{prefix}.step1c&amp;quot;&#xA;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3818" LastActivityDate="2021-03-23T14:03:12.650" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15605" PostTypeId="2" ParentId="15598" CreationDate="2021-03-23T17:31:40.187" Score="1" Body="&lt;p&gt;You can read in the normalized count table and don't normalize the data, but my advice here is not to do that. The rounding of the normalized matrix introduces some noise, but I think the larger issue is how sure are you that the table you are working with, is exactly a count table of normalized counts from DESeq2 ?&lt;/p&gt;&#xA;&lt;p&gt;It really doesn't take much time to align and get count table again, rather than going back and forth wondering whether you did the normalization correctly.&lt;/p&gt;&#xA;&lt;p&gt;I can show below with an example how it will look like, first we make an example dataset and obtain the normalized counts:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(DESeq2)&#xA;set.seed(111)&#xA;sz = runif(6,min=0.5,max=1.5)&#xA;x = makeExampleDESeqDataSet(sizeFactors=sz,m=6)&#xA;x = estimateSizeFactors(x)&#xA;ncounts = counts(x,normalize=TRUE)&#xA;x = estimateDispersions(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now suppose we feed this into a new DESeq2 object, but before that you need to do is check most of the size factors are close to 1:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;estimateSizeFactorsForMatrix(round(ncounts))&#xA; sample1  sample2  sample3  sample4  sample5  sample6 &#xA;1.039179 1.045991 1.048276 1.045872 1.032420 1.052006&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you see something above that suggest a deviation from 1, something is wrong from the normalization and do not even think about using it.&lt;/p&gt;&#xA;&lt;p&gt;Below you can use the normalized counts and you can see the difference is not huge:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dds = DESeqDataSetFromMatrix(round(ncounts),colData(x),~condition)&#xA;sizeFactors(dds) = 1&#xA;dds = estimateDispersions(dds)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Compare the new count table:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head(counts(dds,normalize=TRUE))&#xA;      sample1 sample2 sample3 sample4 sample5 sample6&#xA;gene1       0       2       0       1       0       0&#xA;gene2       9       8      16       6       1       2&#xA;gene3      23      40       8      38      11      41&#xA;gene4       0       0       0       6       8       0&#xA;gene5       3       2       0       5       1       2&#xA;gene6       0       1       3      19       4       9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And your input:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head(ncounts)&#xA;        sample1    sample2   sample3    sample4   sample5   sample6&#xA;gene1  0.000000  2.2658464  0.000000  0.9310803  0.000000  0.000000&#xA;gene2  8.693480  8.3081033 16.251453  5.5864817  1.121781  2.086619&#xA;gene3 23.472395 40.0299524  7.584012 38.1742919 11.217808 40.689076&#xA;gene4  0.000000  0.0000000  0.000000  5.5864817  7.852465  0.000000&#xA;gene5  3.477392  1.5105642  0.000000  4.6554014  1.121781  2.086619&#xA;gene6  0.000000  0.7552821  3.250291 18.6216058  4.487123  9.389787&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We can check the dispersion, for lowly expressed genes you see some difference, but on the whole difference&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with(elementMetadata(x),plot(log10(baseMean),dispGeneEst,pch=20,cex=0.7,col=&amp;quot;#FFC93C80&amp;quot;))&#xA;with(elementMetadata(dds),points(log10(baseMean),dispGeneEst,pch=20,cex=0.7,col=&amp;quot;#DBF6E980&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3hfZm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3hfZm.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Again, I would say I agree with @ATpoint comments that if you can, start from the fastq again.. don't work with data you have doubts about.&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2021-03-23T17:31:40.187" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15607" PostTypeId="1" AcceptedAnswerId="15608" CreationDate="2021-03-23T20:53:01.917" Score="4" ViewCount="89" Body="&lt;p&gt;Okay so I'm using grep to try and get a preview of some trimming operations that are not going as expected..&#xA;Lets say that my sequence in the FastQ file is: ATNGCNATCG&lt;/p&gt;&#xA;&lt;p&gt;What I want to do is..&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;grep &amp;quot;ATCGCTATCG&amp;quot; my.fastq&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;..and match the sequence given above&lt;/p&gt;&#xA;&lt;p&gt;Surely there is some way or some existing tool that I can use besides doing:&#xA;&lt;code&gt;grep &amp;quot;[A|N][T|N][C|N][G|N][C|N]...etc.&amp;quot;&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="10807" LastActivityDate="2021-03-30T17:18:45.377" Title="Telling grep to treat N as [ATCG]" Tags="|fastq|trimming|grep|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15608" PostTypeId="2" ParentId="15607" CreationDate="2021-03-24T01:39:57.780" Score="5" Body="&lt;p&gt;If you want to stick to &lt;code&gt;grep&lt;/code&gt;, use a scripting language such as Perl to generate the regex programmatically. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -le 'print join &amp;quot;&amp;quot;, map &amp;quot;[${_}N]&amp;quot;, split //, $ARGV[0];' ATCGCTATCG &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Prints:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[AN][TN][CN][GN][CN][TN][AN][TN][CN][GN]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can use it in &lt;code&gt;grep&lt;/code&gt; like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep '[AN][TN][CN][GN][CN][TN][AN][TN][CN][GN]' &amp;lt;&amp;lt;&amp;lt; ATNGCNATCG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Prints:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATNGCNATCG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If that works for you, you could make it into a little bash function that also runs the &lt;code&gt;grep&lt;/code&gt;. Add these lines to your &lt;code&gt;~/.bashrc&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;grepN(){&#xA;    seq=&amp;quot;$1&amp;quot;&#xA;    file=&amp;quot;$2&amp;quot;&#xA;    pattern=$(perl -le 'print join &amp;quot;&amp;quot;, map &amp;quot;[${_}N]&amp;quot;, split //, $ARGV[0];' &amp;quot;$seq&amp;quot;)&#xA;    grep &amp;quot;$pattern&amp;quot; &amp;quot;$file&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can now run&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grepN ATCGCTATCG my.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Of course, this is not a good idea since the sequence might be in different lines, but that's what you were doing originally.&lt;/p&gt;&#xA;" OwnerUserId="6251" LastEditorUserId="298" LastEditDate="2021-03-30T17:18:45.377" LastActivityDate="2021-03-30T17:18:45.377" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15609" PostTypeId="1" CreationDate="2021-03-24T09:47:10.937" Score="3" ViewCount="49" Body="&lt;p&gt;I am creating a pipeline for the identification of unknown transcripts. After a local BLASTn search of the transcripts, I have a large list of the respective hits with different genomes. I have the accession code of the respective genome, start/end coordinates, and the strand where the hit is found. Now, I would like to find if the genome where the hit was found is annotated within the coordinates returned by BLAST.&lt;/p&gt;&#xA;&lt;p&gt;I have found the possibility of using efetch to search for each genome via Biopython, but this only returns the whole annotation, even though I limit the search to the specific coordinates with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Entrez.efetch(db=&amp;quot;nuccore&amp;quot;, id=&amp;quot;CP054847.1&amp;quot;, strand=&amp;quot;2&amp;quot;, seq_start=&amp;quot;326053&amp;quot;, seq_end=&amp;quot;326786&amp;quot;, retmode=&amp;quot;xml&amp;quot;) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Of course, I could parse the XML and look for possible annotations within the given range, but not only would this be quite computationally intense since I expect around 10.000+ hits to be searched, but it also seems to return the same results for the plus/minus strand, even though the annotations should be strand-specific.&lt;/p&gt;&#xA;&lt;p&gt;I have also intended to use the standalone tool of efetch, but it would be easier to integrate the search to the pipeline using Biopython.&lt;/p&gt;&#xA;&lt;p&gt;Do you have any ideas why &lt;code&gt;Entrez.efetch&lt;/code&gt; is returning the whole &lt;code&gt;xml&lt;/code&gt; or is there any fast way of filtering the output? Do you have maybe another alternative/idea for this step? I would appreciate any suggestion!&lt;/p&gt;&#xA;" OwnerUserId="1104" LastActivityDate="2024-02-20T22:23:11.523" Title="Finding annotated counterpart after BLASTn with efetch (Biopython)" Tags="|python|sequence-annotation|blastn|entrez|efetch|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15610" PostTypeId="1" AcceptedAnswerId="15642" CreationDate="2021-03-24T10:30:02.560" Score="0" ViewCount="300" Body="&lt;p&gt;I have a large data-frame (excel file) of SNPs with genotyping data. I need to filter the data to get SNP information of a specific gene alone. The list is too long to do it manually. I was wondering of annotating the SNPs with the gene names so that I can filter using the gene name. Does anyone know of a good R package to perform the task?&lt;/p&gt;&#xA;" OwnerUserId="9234" LastActivityDate="2021-03-29T14:28:11.653" Title="Annotating gene names or gene IDs to a dataframe containing SNPs?" Tags="|r|gene|sequence-annotation|snp|filtering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15612" PostTypeId="1" CreationDate="2021-03-24T11:51:09.740" Score="-1" ViewCount="176" Body="&lt;p&gt;How do I remove rows in a VCF file on 1kGenome column, where Match=EXACT using bash ?&lt;/p&gt;&#xA;" OwnerUserId="10821" LastActivityDate="2021-06-11T15:58:22.827" Title="Filter rows of VCF file for Match=EXACT?" Tags="|vcf|ngs|bash|filtering|data-preprocessing|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15613" PostTypeId="1" AcceptedAnswerId="15615" CreationDate="2021-03-24T12:54:22.687" Score="0" ViewCount="69" Body="&lt;p&gt;Hi I am reading this paper and trying to analyse the same datasets in my work:&#xA;&lt;strong&gt;dynGENIE3: dynamical GENIE3 for the inference of gene networks from time series expression data&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this paper, the authors are working on several time course gene expression datasets, including &amp;quot;Drosophila melanogaster embryogenesis&amp;quot; dataset (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE6186&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE6186&lt;/a&gt;) . This dataset has the whole genome ( 14064 genes) of Drosophila melanogaster measured over 31 time-points.&lt;/p&gt;&#xA;&lt;p&gt;In the dynGenie3 paper, we analyse a subset of these  14064 genes, as they write:&#xA;&lt;em&gt;&amp;quot;We focused our analysis on the 1000 genes whose expression vary the most across the time series.&amp;quot;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want to know how do we measure the variance in expression across time series? What statistical tests might have been performed? I couldn't find any more information in the paper so I am assuming it might be something simple and frequently done.  Need some hints.&lt;/p&gt;&#xA;" OwnerUserId="10823" LastActivityDate="2021-03-24T15:43:43.543" Title="How to find genes whose expression vary the most in time-series?" Tags="|gene|gene-expression|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15614" PostTypeId="1" CreationDate="2021-03-24T14:04:34.347" Score="0" ViewCount="219" Body="&lt;p&gt;I am new to stacks for analyzing RAD-Seq data, I have run all the modules. I will like to know to get the total SNP from my sample after running the population module. Is there a command line to use and inside which file?&lt;/p&gt;&#xA;&lt;p&gt;command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;populations -P -M (denovo output) (popmap) -t 5 --vcf --fasta-samples --fasta-loci -O (pop-output) -r 0.8 -p 5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="9884" LastEditorUserId="2108" LastEditDate="2021-03-25T13:16:29.087" LastActivityDate="2021-03-25T13:18:39.360" Title="How to know the number of SNP in my samples after running the population module in stacks" Tags="|snp|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15615" PostTypeId="2" ParentId="15613" CreationDate="2021-03-24T15:43:43.543" Score="0" Body="&lt;p&gt;Maybe my thoughts are too simple, but wouldn't the computation of the variance per gene suffice for this? You really do not need a statistical test, since they are not talking about DEGs, just high-variant genes.&lt;/p&gt;&#xA;" OwnerUserId="1104" LastActivityDate="2021-03-24T15:43:43.543" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15616" PostTypeId="2" ParentId="15614" CreationDate="2021-03-24T16:36:15.127" Score="1" Body="&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Based on the newly provided command (which has &lt;code&gt;--vcf&lt;/code&gt;), there should be a VCF file with suffix &lt;code&gt;.vcf&lt;/code&gt; in the output directory.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Original answer&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It's pretty hard to answer without more information. For example, it's helpful to link to the software and the module that you are working from (I had never heard of Stacks). What do you mean by &amp;quot;total SNP&amp;quot;? Also, what is the actual command that you ran? There are a lot of possible options!&lt;/p&gt;&#xA;&lt;p&gt;But (&lt;a href=&quot;https://catchenlab.life.illinois.edu/stacks/comp/populations.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;based only on reading the docs for the command&lt;/a&gt;), it looks like there are a variety of formats in which the SNP data can be output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;File output options:&#xA;--ordered-export — if data is reference aligned, exports will be ordered; only a single representative of each overlapping site.&#xA;--fasta-loci — output locus consensus sequences in FASTA format..&#xA;--fasta-samples — output the sequences of the two haplotypes of each (diploid) sample, for each locus, in FASTA format.&#xA;--vcf — output SNPs and haplotypes in Variant Call Format (VCF).&#xA;--genepop — output results in GenePop format.&#xA;--structure — output results in Structure format.&#xA;--radpainter — output results in fineRADstructure/RADpainter format.&#xA;--plink — output genotypes in PLINK format.&#xA;--hzar — output genotypes in Hybrid Zone Analysis using R (HZAR) format.&#xA;--phylip — output nucleotides that are fixed-within, and variant among populations in Phylip format for phylogenetic tree construction.&#xA;--phylip-var — include variable sites in the phylip output encoded using IUPAC notation.&#xA;--phylip-var-all — include all sequence as well as variable sites in the phylip output encoded using IUPAC notation.&#xA;--treemix — output SNPs in a format useable for the TreeMix program (Pickrell and Pritchard).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would suggest examining the command that you used, specifically what you passed as the output format. Here are the example commands from the docs:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;populations -P dir -b batch_id [-O dir] [-M popmap] (filters) [--fstats] [-k [--window_size=150000] [--bootstrap [-N 100]]] (output formats) &#xA;populations -V vcf -O dir [-M popmap] (filters) [--fstats] [-k [--window_size=150000] [--bootstrap [-N 100]]] (output formats)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can you track down what you specified as output? That will contain the SNP data / genotypes.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Possibly you will need to rerun the command while specifying a different, preferred format.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2021-03-25T13:18:39.360" LastActivityDate="2021-03-25T13:18:39.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15617" PostTypeId="1" AcceptedAnswerId="15619" CreationDate="2021-03-24T17:18:04.860" Score="0" ViewCount="288" Body="&lt;p&gt;I am trying to understand how data that was uploaded to SRA (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra?LinkName=biosample_sra&amp;amp;from_uid=4510743&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/sra?LinkName=biosample_sra&amp;amp;from_uid=4510743&lt;/a&gt;) can be analyzed with the assumption that the FASTQ file should contain replicate information (3 replicates within the sample). I downloaded the file and ran it with Nextflow, and it was mapped to the human genome and determined to contain unstranded reads of 50bp, both directions.&lt;/p&gt;&#xA;&lt;p&gt;Which steps can I take to infer if the FASTQ file indeed is not demultiplexed?&lt;/p&gt;&#xA;&lt;p&gt;Many thanks,&#xA;Geo&lt;/p&gt;&#xA;&lt;p&gt;The header of the FASTQ looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@SRR3467208.1 1/1&#xA;NTCTCTATGTCCACTCTGGAGCCTTTAAGTGCCACTTGAGGGCCTTTAAC&#xA;+&#xA;#4:D;DDEHHFHHIJJIJJGHGHJJI@HFIIJJHJJIG?9CFHIIIHHHG&#xA;@SRR3467208.2 2/1&#xA;NGTGCTGTGTGTGCATGTGTGTGCGTGTGTGTGCTGTGCGTTTGTGTGTG&#xA;+&#xA;#1:B:BDDFFFFFFFBGFHIFIIFGFFGFGFGIIIIIIEFGGIIFFIFII&#xA;@SRR3467208.3 3/1&#xA;NGTCACTTGGCATCTGGCCATCGGGCTGGATGCCGTGTTCCAGGCAGTAG&#xA;+&#xA;#1:A=DDDB&amp;lt;&amp;lt;CFAFGI@FGG9A@68CEG;DB?&amp;lt;DGGGE49?BBF2B=FF&#xA;@SRR3467208.4 4/1&#xA;NTCCGGGGCTCCAGCAACCAGAAGAGGGAAAAGCTGTCTTCGGTGATGTC&#xA;+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10782" LastActivityDate="2021-03-24T20:43:28.097" Title="Demultiplexing FASTQ file without index information" Tags="|rna-seq|fastq|demultiplexing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15618" PostTypeId="1" AcceptedAnswerId="15622" CreationDate="2021-03-24T18:28:18.573" Score="1" ViewCount="59" Body="&lt;p&gt;One challenging aspect of modeling scRNA-seq data is data sparsity, that is, scRNA-seq measurements typically suffer from large fractions of observed zeros (i.e. &lt;em&gt;dropouts&lt;/em&gt;), where a given gene in a given cell has no unique molecular identifiers or reads mapping to it [&lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-020-1926-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;].&lt;/p&gt;&#xA;&lt;p&gt;What would be an appropriate dataset to benchmark supervised learning algorithms that operate on single-cell RNA-seq data? In particular, I am looking for challenging datasets (preferably with labels, i.e. cell type classification) where existing methods struggle due to the sparsity of the data. Additionally, does it ever happen that the &lt;em&gt;dropouts&lt;/em&gt; are confounders of the labels? (i.e. for a given cell type A the &lt;em&gt;dropout&lt;/em&gt; probability of a gene X is higher than for another cell type B).&lt;/p&gt;&#xA;&lt;p&gt;Any pointers are welcome.&lt;/p&gt;&#xA;" OwnerUserId="9072" LastActivityDate="2021-03-25T20:58:10.183" Title="Challenging benchmarks for supervised learning on sparse scRNA-seq data" Tags="|rna-seq|scrnaseq|single-cell|machine-learning|benchmarking|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15619" PostTypeId="2" ParentId="15617" CreationDate="2021-03-24T20:43:28.097" Score="1" Body="&lt;p&gt;I would suggest looking at sample metadata:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Library:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Name: TruSeq RNA&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Instrument: Illumina HiSeq 2000&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Strategy: RNA-Seq&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Source: TRANSCRIPTOMIC&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Selection: PolyA&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Layout: PAIRED&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You can see that it's paired. You can look at the spot descriptor at the page you link to see more information about the design.&lt;/p&gt;&#xA;&lt;p&gt;There is no multiplexing information available there. I would instead look at the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra?term=SRP074191&quot; rel=&quot;nofollow noreferrer&quot;&gt;overall project and its metadata&lt;/a&gt; to find the different replicates. It looks like there are 2 samples x 3 replicates each, deposited independently. So it is likely to already be demultiplexed, and you can just download them individually.&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Unstranded&amp;quot; isn't really a meaningful term for this, if I understand correctly. Forward and reverse read pairs don't correspond to forward and reverse on the chromosome, but to the way that sequencing-by-synthesis happens on the machine. So the mapping orientation is not really relevant.&lt;/p&gt;&#xA;&lt;p&gt;I infer that you are working from a single FASTQ file. If you are working from a single FASTQ file for paired-end out of SRA, then the FASTQ file is not yet resolved into F+R reads. I would suggest rerunning the FASTQ conversion with &lt;code&gt;--split-3&lt;/code&gt; or similar option.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-03-24T20:43:28.097" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15620" PostTypeId="2" ParentId="15612" CreationDate="2021-03-24T20:46:00.707" Score="1" Body="&lt;p&gt;Hard to say without seeing VCF rows, but generally speaking with text files you can use &lt;code&gt;grep&lt;/code&gt;, if I understand the question:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -vw &amp;quot;Match=EXACT&amp;quot; file.vcf &amp;gt; filtered.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2021-03-30T19:05:38.273" LastActivityDate="2021-03-30T19:05:38.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15622" PostTypeId="2" ParentId="15618" CreationDate="2021-03-25T10:28:35.267" Score="2" Body="&lt;p&gt;You could try the PBMC 3k data from the Satija lab:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://satijalab.org/seurat/articles/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/articles/pbmc3k_tutorial.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For well-annotated data, there's the Single-cell proteo-genomic reference map:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2021.03.18.435922v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/10.1101/2021.03.18.435922v1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Data for that are available &lt;a href=&quot;https://figshare.com/projects/Single-cell_proteo-genomic_reference_maps_of_the_human_hematopoietic_system/94469&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If these datasets are too large (i.e. too good), you can use the existing counts as weighting for a random removal of reads. Genes with high expression are likely to be present regardless of sequencing depth, hence the count-adjusted removal.&lt;/p&gt;&#xA;&lt;p&gt;FWIW, I don't think that supervised learning is a good fit for dealing with data sparsity in cDNA read counts. There are plenty of existing attempts to solve the issues by adjusting the assumed models, including research by the authors of commonly-used bulk differential expression tools (e.g. &lt;a href=&quot;https://doi.org/10.1186/s13059-018-1406-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-03-25T20:58:10.183" LastActivityDate="2021-03-25T20:58:10.183" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15623" PostTypeId="1" AcceptedAnswerId="15624" CreationDate="2021-03-25T11:09:53.087" Score="1" ViewCount="589" Body="&lt;p&gt;I was trying to develop a pipeline using nextflow and plink. My code runs without any error, but got only two output files instead of 22 (one for each chromosome). Here is part my code block, any help?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process step5 {&#xA;publishDir params.publish, mode: 'copy'&#xA;input:&#xA;val(hrc) from HRC_1000G_check_bim&#xA;val(hrc_site) from hrc_sites_file&#xA;tuple val(prefix), val(chr_name), path(bfiles) from step4_frq&#xA;output:&#xA;tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}&quot;), val(chr_name), path(&quot;LOG-$&lt;/span&gt;{prefix}&lt;span class=&quot;math-container&quot;&gt;${chr_name}.step4-HRC.txt&quot;) into step4_log&#xA;tuple val(&quot;$&lt;/span&gt;{prefix}&amp;quot;), val(chr_name), path(&amp;quot;Exclude-&lt;span class=&quot;math-container&quot;&gt;${prefix}$&lt;/span&gt;{chr_name}.step4-HRC.txt&amp;quot;) into step4_exclude&#xA;tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}&quot;), val(chr_name), path(&quot;Chromosome-$&lt;/span&gt;{prefix}&lt;span class=&quot;math-container&quot;&gt;${chr_name}.step4-HRC.txt&quot;) into step4_chromosome&#xA;tuple val(&quot;$&lt;/span&gt;{prefix}&amp;quot;), val(chr_name), path(&amp;quot;Position-&lt;span class=&quot;math-container&quot;&gt;${prefix}$&lt;/span&gt;{chr_name}.step4-HRC.txt&amp;quot;) into step4_position&#xA;tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}&quot;), val(chr_name), path(&quot;Strand-Flip-$&lt;/span&gt;{prefix}&lt;span class=&quot;math-container&quot;&gt;${chr_name}.step4-HRC.txt&quot;) into step4_strand&#xA;tuple val(&quot;$&lt;/span&gt;{prefix}&amp;quot;), val(chr_name), path(&amp;quot;Force-Allele1-&lt;span class=&quot;math-container&quot;&gt;${prefix}$&lt;/span&gt;{chr_name}.step4-HRC.txt&amp;quot;) into step4_allele&#xA;tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}&quot;), val(chr_name), path(&quot;ID-$&lt;/span&gt;{prefix}&lt;span class=&quot;math-container&quot;&gt;${chr_name}.step4-HRC.txt&quot;) into step4_id&#xA;script:&#xA;&quot;&quot;&quot;&#xA;perl &quot;$&lt;/span&gt;{hrc}&amp;quot; -b &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}$&lt;/span&gt;{chr_name}.step4.bim&amp;quot; -f &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}$&lt;/span&gt;{chr_name}.step4.frq&amp;quot; -r &amp;quot;${hrc_site}&amp;quot; -h&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output files are:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gsa-out/Chromosome-chr22.step4-HRC.txt     gsa-out/Force-Allele1-chr3.step4-HRC.txt  gsa-out/Position-chr22.step4-HRC.txt&#xA;gsa-out/Chromosome-chr3.step4-HRC.txt      gsa-out/ID-chr22.step4-HRC.txt            gsa-out/Position-chr3.step4-HRC.txt&#xA;gsa-out/Exclude-chr22.step4-HRC.txt        gsa-out/ID-chr3.step4-HRC.txt             gsa-out/Strand-Flip-chr22.step4-HRC.txt&#xA;gsa-out/Exclude-chr3.step4-HRC.txt         gsa-out/LOG-chr22.step4-HRC.txt           gsa-out/Strand-Flip-chr3.step4-HRC.txt&#xA;gsa-out/Force-Allele1-chr22.step4-HRC.txt  gsa-out/LOG-chr3.step4-HRC.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am not sure why I am getting only 2, instead of 22 output files. With similar output channel previously generated 22 output files in the same script. For eaxmple:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process step1b {&#xA;publishDir params.publish, mode: 'copy'&#xA;input:&#xA;tuple val(prefix), val(chr_name), path(bfiles) from step1a_out&#xA;output:&#xA;tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}&quot;), val(chr_name), path(&quot;$&lt;/span&gt;{prefix}&lt;span class=&quot;math-container&quot;&gt;${chr_name}.step1c.{bed,bim,fam}&quot;) into step1b_results&#xA;path(&quot;$&lt;/span&gt;{prefix}&lt;span class=&quot;math-container&quot;&gt;${chr_name}.step1c.log&quot;) into step1b_logs&#xA;path(&quot;$&lt;/span&gt;{prefix}${chr_name}.step1b.snplist.txt&amp;quot;) into step1b_snplist&#xA;script:&#xA;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;awk '{ if ((\\\$5==&amp;quot;T&amp;quot; &amp;amp;&amp;amp; \\\$6==&amp;quot;A&amp;quot;)||(\\\$5==&amp;quot;A&amp;quot; &amp;amp;&amp;amp; \\\$6==&amp;quot;T&amp;quot;)||(\\\$5==&amp;quot;C&amp;quot; &amp;amp;&amp;amp; \\\$6==&amp;quot;G&amp;quot;)||(\\\$5==&amp;quot;G&amp;quot; &amp;amp;&amp;amp; \\\$6==&amp;quot;C&amp;quot;)) print \\\$2, &amp;quot;ambig&amp;quot; ; else print \\\$2 ;}' &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${bfiles[1]&#xA;}&quot; | grep ambig &amp;gt; &quot;$&lt;/span&gt;{prefix}&lt;span class=&quot;math-container&quot;&gt;${chr_name}.step1b.snplist.txt&quot;&#xA;plink1.9 --bfile &quot;$&lt;/span&gt;{prefix}&lt;span class=&quot;math-container&quot;&gt;${chr_name}.step1b&quot; --exclude &quot;$&lt;/span&gt;{prefix}&lt;span class=&quot;math-container&quot;&gt;${chr_name}.step1b.snplist.txt&quot; --make-bed --out &quot;$&lt;/span&gt;{prefix}${chr_name}.step1c&amp;quot;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3839" LastActivityDate="2021-03-25T14:08:11.843" Title="Nextflow: Runs without error, produce few but not all output files into output channel" Tags="|plink|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15624" PostTypeId="2" ParentId="15623" CreationDate="2021-03-25T14:08:11.843" Score="1" Body="&lt;p&gt;I suspect your &lt;code&gt;HRC_1000G_check_bim&lt;/code&gt; and &lt;code&gt;hrc_sites_file&lt;/code&gt; channels are &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#queue-channel&quot; rel=&quot;nofollow noreferrer&quot;&gt;queue channels&lt;/a&gt; when they should be &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#value-channel&quot; rel=&quot;nofollow noreferrer&quot;&gt;value channels&lt;/a&gt;. Please again see &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#understand-how-multiple-input-channels-work&quot; rel=&quot;nofollow noreferrer&quot;&gt;how multiple channels work&lt;/a&gt;. Also, &lt;a href=&quot;https://www.nextflow.io/docs/latest/sharing.html#third-party-scripts&quot; rel=&quot;nofollow noreferrer&quot;&gt;third-party scripts&lt;/a&gt;, like your Perl script, would be better placed in a directory called &lt;code&gt;bin/&lt;/code&gt; at the root of your project repository. Nextflow adds this directory to your $PATH, so scripts in this directory will be automatically accessible in your pipeline without needing to localize them using a separate channel. This way also avoids having to specify an absolute path to invoke them. Just make sure your script is executable: &lt;code&gt;chmod +x your_script.pl&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;hrc_sites = file( params.hrc_sites_file )&#xA;&#xA;process step5 {&#xA;&#xA;    input:&#xA;    tuple val(prefix), val(chr_name), path(bfiles) from step4_frq&#xA;    path hrc_sites&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    your_script.pl \\&#xA;        -b &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}$&lt;/span&gt;{chr_name}.step4.bim&amp;quot; \\&#xA;        -f &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${prefix}$&lt;/span&gt;{chr_name}.step4.frq&amp;quot; \\&#xA;        -r &amp;quot;${hrc_sites}&amp;quot; \\&#xA;        -h&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2021-03-25T14:08:11.843" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="15626" PostTypeId="2" ParentId="15573" CreationDate="2021-03-25T16:15:02.477" Score="0" Body="&lt;p&gt;Here's one way to do it programmatically using the biomaRt package (your potential solution #1):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(biomaRt)&#xA;library(dplyr)&#xA;&#xA;mart37 &amp;lt;- useMart(biomart = &amp;quot;ENSEMBL_MART_ENSEMBL&amp;quot;,&#xA;                  host    = &amp;quot;grch37.ensembl.org&amp;quot;,&#xA;                  path    = &amp;quot;/biomart/martservice&amp;quot;,&#xA;                  dataset = &amp;quot;hsapiens_gene_ensembl&amp;quot;)&#xA;&#xA;mart38 &amp;lt;- useMart(biomart = &amp;quot;ENSEMBL_MART_ENSEMBL&amp;quot;,&#xA;                  path    = &amp;quot;/biomart/martservice&amp;quot;,&#xA;                  dataset = &amp;quot;hsapiens_gene_ensembl&amp;quot;)&#xA;&#xA;#pick all genes on chr22 from build 37 as a test dataset::&#xA;out37 &amp;lt;- getBM(attributes = c('hgnc_symbol', 'chromosome_name',&#xA;                              'ensembl_gene_id', 'ensembl_transcript_id', 'ccds'),&#xA;                  filters = c('chromosome_name'),&#xA;                  values = list(&amp;quot;22&amp;quot;),&#xA;                  mart = mart37) %&amp;gt;% #empty string CCDS ID's&#xA;  filter(ccds != &amp;quot;&amp;quot;)&#xA;&#xA;#NOTE:: to check which filters can be used to query mart37, run this command:&#xA;#View(listFilters(mart37))&#xA;&#xA;out38 &amp;lt;- getBM(attributes = c('hgnc_symbol', 'chromosome_name',&#xA;                              'ensembl_gene_id', 'ensembl_transcript_id', 'ccds'),&#xA;                        filters = c('ccds'),&#xA;                        values = list(out37$ccds),&#xA;                        mart = mart38)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What this does is query the build 38 DB via the CCDS ID's of build 37. You can include other variables in this &lt;code&gt;out38&lt;/code&gt; dataframe by simply adding to the attributes argument above. And in order to see all of the potential variables that can be included, you just have to run &lt;code&gt;View(listFilters(mart37))&lt;/code&gt; or &lt;code&gt;View(listFilters(mart38))&lt;/code&gt;, depending on which build you are interested in.&lt;/p&gt;&#xA;&lt;p&gt;Does this solve your problem?&lt;/p&gt;&#xA;" OwnerUserId="4001" LastActivityDate="2021-03-25T16:15:02.477" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15627" PostTypeId="1" CreationDate="2021-03-26T03:07:50.707" Score="2" ViewCount="174" Body="&lt;p&gt;(I am a complete newbie to bioinformatics, so please bear with me!).&lt;/p&gt;&#xA;&lt;p&gt;I recently used BLAST to compare a 500 bp nucleotide sequence in &lt;em&gt;S. cerevisiae&lt;/em&gt; to a bunch of other 500 bp nucleotide sequences in various fungi (these other sequences are homologs of the &lt;em&gt;S. cerevisiae&lt;/em&gt; gene). Ideally, when I click &amp;quot;BLAST&amp;quot;, I want to get nothing back. What I'm trying to double check is the assumption that my &lt;em&gt;S. cerevisiae&lt;/em&gt; sequence is NOT similar to my other input sequences.&lt;/p&gt;&#xA;&lt;p&gt;Here's what I ran on the computer, using the BLAST website:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hLcYh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hLcYh.png&quot; alt=&quot;BLAST input&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;After I run this query, I only get one sequence back:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zoYEd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zoYEd.png&quot; alt=&quot;BLAST output&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;which tells me that the sequence I got as an output needs to be excluded from my dataset, as it's too similar to the &lt;em&gt;S. cerevisiae&lt;/em&gt; sequence.&lt;/p&gt;&#xA;&lt;p&gt;The problem is, I need to run this type of query and analysis about 6000 more times for 6000 different &lt;em&gt;S. cerevisiae&lt;/em&gt; genes and their homologs. I don't know how to do this programmatically. The Biopython packages seem to only compare one sequence against the whole NCBI database, and I was really struggling to understand how to do this via the command line (was struggling with the package installations a bit, and it also seemed to only be for one sequence input).&lt;/p&gt;&#xA;&lt;p&gt;Is what I'm doing here a multiple sequence alignment? When I looked into programmatically doing an MSA, the output seemed to only be the alignment, but I want to see an output like the website gives me. I'm not really sure what I should be looking into.&lt;/p&gt;&#xA;&lt;p&gt;Any guidance would help!&lt;/p&gt;&#xA;" OwnerUserId="10849" LastActivityDate="2021-03-27T10:45:13.420" Title="What kind of BLAST do I need to do to accomplish this task?" Tags="|sequence-alignment|blast|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15628" PostTypeId="1" AcceptedAnswerId="15694" CreationDate="2021-03-26T15:52:03.233" Score="5" ViewCount="202" Body="&lt;p&gt;One of my students is working on the development of a SynBio tool to design and represent genetic circuits in a standardized format. I know there are some libraries out there.&lt;/p&gt;&#xA;&lt;p&gt;What I am interested in finding out is to have a hands-on tutorial how to use an SBOL library (say, pySBOL) to represent genetic circuits in SBOL data and visual representations.&lt;/p&gt;&#xA;" OwnerUserId="10851" LastEditorUserId="4869" LastEditDate="2021-03-28T18:24:15.327" LastActivityDate="2021-04-06T18:43:44.617" Title="How to use SBOL (Synthetic Biology Open Language)?" Tags="|sbol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15630" PostTypeId="2" ParentId="15627" CreationDate="2021-03-27T10:45:13.420" Score="6" Body="&lt;h2&gt;Clarification&lt;/h2&gt;&#xA;&lt;p&gt;Just making sure - you have ~6000 datasets - one for each &lt;em&gt;S.cerevisiae&lt;/em&gt; gene of interest - which are made up of homologues of each respective gene? And you want to filter out genes that are too similar?&lt;/p&gt;&#xA;&lt;p&gt;In that case, it would be helpful to know how you're defining a sequence as 'similar enough' for it to be filtered. Given they're all homologues, it's likely there's going to be a high baseline of sequence similarity. In your example, you consider 97.40% identity across the whole query as different enough to be filtered out, so I'm getting the feeling you specifically want to filter out identical sequences?&lt;/p&gt;&#xA;&lt;p&gt;Also, when you talk about genes here, do you want to filter out gene &lt;strong&gt;products&lt;/strong&gt; (i.e. proteins) that are too similar to the actual coding sequences (CDS)? The CDS  of two homologues might be X% different, but that difference might not be as high once translated into proteins due to how codons work. Proteins can go down to levels of similarity as low as ~30% identity and still have the same function, so depending on the context of your work (taxonomic, structural, functional) you might want to translate these CDS into protein, so further clarification would be helpful!&lt;/p&gt;&#xA;&lt;h2&gt;The actual bioinformatics question&lt;/h2&gt;&#xA;&lt;p&gt;To answer your question of programming, if the context of this comparison is always X &lt;em&gt;S.cerevisiae&lt;/em&gt; gene vs. Y dataset of homologues, a MSA is definitely not the answer (you would do that if you wanted to see how all of the homologues compare to each other, rather than to specifically X gene). Also, using the web-client for NCBI BLAST will take a lot of time, so that's also not recommended for the kind of throughput you need.&lt;/p&gt;&#xA;&lt;p&gt;Pairwise sequence alignments that you do on your local machine/server between X genes and all of its respective Y dataset sequences is probably the best way to go. I would suggest looking into &lt;strong&gt;&lt;a href=&quot;http://emboss.sourceforge.net/apps/release/6.6/emboss/apps/needleall.html&quot; rel=&quot;noreferrer&quot;&gt;needleall&lt;/a&gt;&lt;/strong&gt; - it allows you to compare X sequences to Y sequences in a pairwise way, though it performs a &lt;strong&gt;global alignment&lt;/strong&gt; rather than &lt;strong&gt;local&lt;/strong&gt; like BLAST, so it might be slower.&lt;/p&gt;&#xA;&lt;p&gt;I don't know what your computational resources are, so in case that might be too computationally heavy, the faster alternative would be to install the &lt;strong&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK279690/&quot; rel=&quot;noreferrer&quot;&gt;BLAST+ suite&lt;/a&gt;&lt;/strong&gt;, which allows you to run BLAST queries like you did on the web browser, except on your own machine. Specifically, you could make your own custom individual BLAST databases out of each Y dataset using &lt;code&gt;makeblastdb&lt;/code&gt; from the suite. Then you use &lt;code&gt;blastn&lt;/code&gt; (or &lt;code&gt;blastp&lt;/code&gt; if you decide to translate your sequences) on each dataset with its respective S.cerevisae sequence.&lt;/p&gt;&#xA;&lt;p&gt;BLAST+ also allows you to customise the actual output as well, making it easier to parse your results programatically for when you want to remove sequences that are too similar. There are many examples online of how to do this, but have a go at it on your own  first with the BLAST+ documentation I linked :)&lt;/p&gt;&#xA;" OwnerUserId="11856" LastActivityDate="2021-03-27T10:45:13.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15632" PostTypeId="1" CreationDate="2021-03-27T17:25:29.950" Score="3" ViewCount="94" Body="&lt;p&gt;I'm looking at a bacterial genome and trying to find specific PEGs (Protein Encoding Genes) by BLAST searching. I'm taking the genes found with the lowest E-value in order to psiBLAST them to check it codes for the same protein. I'm not exactly sure of how high the E-Value needs to be for me to conclude that the protein is not coded for by the subject gene.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have any advice on how to make this process a little more efficient/scientific?&#xA;Thank you.&#xA;(By the way, I'm using BLAST with a BLOUSUM62 matrix.)&lt;/p&gt;&#xA;" OwnerUserId="11861" LastEditorUserId="10756" LastEditDate="2021-04-05T00:04:13.363" LastActivityDate="2021-05-23T21:39:42.860" Title="How low does an E-Value need to be for a gene to code for the same protein as query?" Tags="|blast|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15633" PostTypeId="1" AcceptedAnswerId="15635" CreationDate="2021-03-28T08:41:40.617" Score="2" ViewCount="1964" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2dD9s.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2dD9s.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;a href=&quot;https://i.stack.imgur.com/A23k1.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/A23k1.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;I was trying to run a slurm job script inside nextflow process. Is it possible? For instance, I have a job script 'PRE_PHASING_QC.JOB' and outside nextflow it runs perfectly, I want to integrate it in a nextflow script. May be my example code could be an explanation of what I want.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;prephase='PRE_PHASING_QC.JOB'&#xA;process prephasing {&#xA;input:&#xA;file(prephasing) from prephase&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;sbatch &amp;quot;${prephasing}&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sbatch: error: This does not look like a batch script&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried to make my job script executable, put it in bin and then include it in the nextflow process according to @Steve. So, the script runs without any error but got no output. The original job script takes 3 files (bed,bim,fam) as input and produce multiple outputs. I guess that I have to declare outputs inside nextflow process. So far I tried as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process prephasing {&#xA;publishDir params.out, mode: 'copy'&#xA;output:&#xA;tuple val(chr),val(num),val(steps),path(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chr}$&lt;/span&gt;{num}.${steps}.{bed,bim,fam,log}&amp;quot;) into out1&#xA;script:&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;PRE_PHASING_QC.job&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It gives:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error executing process &amp;gt; 'prephasing'&#xA;Caused by:&#xA;Missing value declared as output parameter: chr&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My expected output would be chr{1-22}.step{1-10}.{bed,bim,fam,log}&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash &#xA;#SBATCH --job-name=prephase &#xA;#SBATCH --account=p697_norment &#xA;#SBATCH --time=1:00:00 &#xA;#SBATCH --ntasks=1 &#xA;#SBATCH --mem-per-cpu=7600M &#xA;#SBATCH --cpus-per-task=7 &#xA;#SBATCH --array=1-22 &#xA;&#xA;source /cluster/bin/jobsetup &#xA;&#xA;set -o errexit &#xA;&#xA;module load plink &#xA;&#xA;export prefix=chr${SLURM_ARRAY_TASK_ID} &#xA;export hrc_sites_file=/cluster/projects/p697/projects/moba_qc_imputation/resources/HRC/HRC.r1-1.GRCh37.wgs.mac5.sites.tab.gz &#xA;export HRC_1000G_check_bim=/cluster/projects/p697/github/norment/moba_qc_imputation/software/HRC-1000G-check-bim.pl&#xA;&#xA;# Step 1a (exclude duplicated), 1b (exclude indels and non-ACTG alleles), 1c (exclude ambiguous SNPs) &#xA;&#xA;plink --bfile hce-ec-eur-fin-pass-qc --chr &lt;span class=&quot;math-container&quot;&gt;${SLURM_ARRAY_TASK_ID} --out $&lt;/span&gt;prefix --make-bed || true&#xA;plink --bfile &lt;span class=&quot;math-container&quot;&gt;$prefix --list-duplicate-vars ids-only suppress first --out $&lt;/span&gt;prefix.stepla || true &#xA;plink --bfile &lt;span class=&quot;math-container&quot;&gt;$prefix --exclude $&lt;/span&gt;prefix.stepla.dupvar --make-bed --out &lt;span class=&quot;math-container&quot;&gt;$prefix.stepla || true &#xA;plink --bfile $&lt;/span&gt;prefix.stepla --snps-only just-acgt --make-bed --out $prefix.steplb || true &#xA;&#xA;awk '{ if ((&lt;span class=&quot;math-container&quot;&gt;$5==&quot;T&quot; &amp;amp;&amp;amp; $&lt;/span&gt;6==&amp;quot;A&amp;quot;)||(&lt;span class=&quot;math-container&quot;&gt;$5==&quot;A&quot; &amp;amp;&amp;amp; $&lt;/span&gt;6==&amp;quot;T&amp;quot;)ii(&lt;span class=&quot;math-container&quot;&gt;$5==&quot;C&quot; &amp;amp;&amp;amp; $&lt;/span&gt;6==&amp;quot;G&amp;quot;)||(&lt;span class=&quot;math-container&quot;&gt;$5==&quot;G&quot; &amp;amp;&amp;amp; $&lt;/span&gt;6==&amp;quot;C&amp;quot;)) print &lt;span class=&quot;math-container&quot;&gt;$2, &quot;ambig&quot; ; else print $&lt;/span&gt;2 ;}' &lt;span class=&quot;math-container&quot;&gt;$prefix. steplb.bim | grep ambig &amp;gt; $&lt;/span&gt;prefix.steplb.snplist.txt || true &#xA;plink --bfile &lt;span class=&quot;math-container&quot;&gt;$prefix.steplb --exclude $&lt;/span&gt;prefix.steplb.snplist.txt --make-bed --out $prefix.steplc || true &#xA;&#xA;# Step 2. Zero out Mendelian errors&#xA;plink --bfile &lt;span class=&quot;math-container&quot;&gt;$prefix.steplc --me 1 1 --set-me-missing --make-bed --out $&lt;/span&gt;prefix.step2 || true &#xA;&#xA;# Step 3. Missingness and HWE + Step 4. Compute allele frequencies &#xA;# NB! --mind 0.02 filter is removed from the next command, to avoid sample QC per chunk, which may leave inconsistent number of individuals avail able per chunk. &#xA;plink --bfile &lt;span class=&quot;math-container&quot;&gt;$prefix.step2 --hwe le-6 --geno 0.02 --make-bed --out $&lt;/span&gt;prefix.step3 || true &#xA;plink --bfile &lt;span class=&quot;math-container&quot;&gt;$prefix.step3 --freq --make-bed --out $&lt;/span&gt;prefix.step4 || true &#xA;&#xA;# Step 4-10. Run Oxford perl script and apply first 5 lines of its Run-plink.sh command, followed by '--update-name' command (update marker name s) &#xA;perl &lt;span class=&quot;math-container&quot;&gt;${HRC-IRC_1000G_check_bim} -b $&lt;/span&gt;prefix.step4.bim -f &lt;span class=&quot;math-container&quot;&gt;$prefix.step4.frq -r $&lt;/span&gt;{hrc_sites_file} -h || true &#xA;plink --bfile &lt;span class=&quot;math-container&quot;&gt;$prefix.step4 --exclude Exclude-$&lt;/span&gt;prefix.step4-HRC.txt --make-bed --out &lt;span class=&quot;math-container&quot;&gt;$prefix.step5 || true &#xA;plink --bfile $&lt;/span&gt;prefix.step5 --update-chr Chromosome-&lt;span class=&quot;math-container&quot;&gt;$prefix.step4-HRC.txt --make-bed --out $&lt;/span&gt;prefix.step6 || true &#xA;plink --bfile &lt;span class=&quot;math-container&quot;&gt;$prefix.step6 --update-map Position-$&lt;/span&gt;prefix.step4-HRC.txt --make-bed --out &lt;span class=&quot;math-container&quot;&gt;$prefix.step7 || true &#xA;plink --bfile $&lt;/span&gt;prefix.step7 --flip Strand-Flip-&lt;span class=&quot;math-container&quot;&gt;$prefix.step4-HRC.txt --make-bed --out $&lt;/span&gt;prefix.step8 || true &#xA;plink --bfile &lt;span class=&quot;math-container&quot;&gt;$prefix.step8 --a2-allele Force-Allele1-$&lt;/span&gt;prefix.step4-HRC.txt --make-bed --out &lt;span class=&quot;math-container&quot;&gt;$prefix.step9 || true &#xA;plink --bfile $&lt;/span&gt;prefix.step9 --update-name ID-&lt;span class=&quot;math-container&quot;&gt;$prefix.step4-HRC.txt --real-ref-alleles --make-bed --out $&lt;/span&gt;prefix.step10 || true&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3839" LastEditorUserId="10775" LastEditDate="2021-03-30T06:08:48.370" LastActivityDate="2021-03-30T14:15:10.390" Title="Nextflow: run sbatch (slurm job script) inside a process" Tags="|nextflow|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="15634" PostTypeId="1" CreationDate="2021-03-28T08:57:22.280" Score="1" ViewCount="44" Body="&lt;p&gt;I have a following fasta (produced by stacks populations):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;CLocus_10056_Sample_934_Locus_12529_Allele_0 [sample_934; groupI, 49712]&#xA;TGCAGGCCCCAGGCCACGCCGTCTGCGGCAGCGCTGGAAGGAGGCGGTGGAGGAGGCGGCCAACGGCTCCCTGCCCCAGAAGGCCGAGTTCACCG&#xA;&amp;gt;CLocus_10056_Sample_934_Locus_12529_Allele_1 [sample_934; groupI, 49712]&#xA;TGCAGGCCCCAGGCCACGCCGTCTGCGGCAGCGTTGGAAGGAGGCGGTGGAGGAGGCGGCCAACGGCTCCCTGCCCCAGAAGGCCGAGTTCACCG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;as both alleles are represented separetely, I want to combine both genes using IUPAC nucleotide code.&#xA;Is there a tool for it or is it more useful to do it via a custom script?&lt;/p&gt;&#xA;" OwnerUserId="8621" LastEditorUserId="3694" LastEditDate="2021-03-28T18:23:53.150" LastActivityDate="2021-03-28T18:23:53.150" Title="Combine two alleles to genotype locus?" Tags="|sequence-alignment|fasta|radseq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15635" PostTypeId="2" ParentId="15633" CreationDate="2021-03-28T12:28:23.003" Score="2" Body="&lt;p&gt;Using Nextflow to submit jobs this way makes little sense, since:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://slurm.schedmd.com/sbatch.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sbatch&lt;/a&gt; exits immediately after the script is successfully transferred&#xA;to the Slurm controller and assigned a Slurm job ID. The batch script&#xA;is not necessarily granted resources immediately, it may sit in the&#xA;queue of pending jobs for some time before its required resources&#xA;become available.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Assuming your job script is basically a shell script, make it executable (e.g. &lt;code&gt;chmod +x PRE_PHASING_QC.JOB&lt;/code&gt;) and move it into a folder called &lt;code&gt;bin&lt;/code&gt; in the root directory of your project repository. &lt;a href=&quot;https://www.nextflow.io/docs/latest/sharing.html#third-party-scripts&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nextflow automatically adds this folder to your $PATH&lt;/a&gt;, making scripts in this folder accessible in your pipeline without needing localize them and without needing to specify an absolute path to invoke them. Your code then becomes:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process prephasing {&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    PRE_PHASING_QC.JOB [options] &amp;lt;args&amp;gt;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Finally, add &lt;code&gt;process.executor = 'slurm'&lt;/code&gt; to your &lt;code&gt;nextflow.config&lt;/code&gt; file. You may need to configure some other options, like resource requests, but that should get your script integrated into your pipeline without much refactoring, if any.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Contents of &lt;code&gt;PRE_PHASING_QC.sh&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;set -euo pipefail&#xA;&#xA;hrc_sites_file=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${1}&quot;&#xA;input_prefix=&quot;$&lt;/span&gt;{2}&amp;quot;&#xA;chrom=&amp;quot;chr${3}&amp;quot;&#xA;&#xA;# Step 1a (exclude duplicated), 1b (exclude indels and non-ACTG alleles), 1c (exclude ambiguous SNPs)&#xA;&#xA;plink --bfile &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${input_prefix}&quot; --chr &quot;$&lt;/span&gt;{3}&amp;quot; --out &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}&quot; --make-bed&#xA;plink --bfile &quot;$&lt;/span&gt;{chrom}&amp;quot; --list-duplicate-vars ids-only suppress first --out &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}.step1a&quot;&#xA;plink --bfile &quot;$&lt;/span&gt;{chrom}&amp;quot; --exclude &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}.step1a.dupvar&quot; --make-bed --out &quot;$&lt;/span&gt;{chrom}.step1a&amp;quot;&#xA;plink --bfile &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}.step1a&quot; --snps-only just-acgt --make-bed --out &quot;$&lt;/span&gt;{chrom}.step1b&amp;quot;&#xA;&#xA;awkcmd='&#xA;(&lt;span class=&quot;math-container&quot;&gt;$5 == &quot;T&quot; &amp;amp;&amp;amp; $&lt;/span&gt;6 == &amp;quot;A&amp;quot;) || (&lt;span class=&quot;math-container&quot;&gt;$5 == &quot;A&quot; &amp;amp;&amp;amp; $&lt;/span&gt;6 == &amp;quot;T&amp;quot;) ||&#xA;(&lt;span class=&quot;math-container&quot;&gt;$5 == &quot;C&quot; &amp;amp;&amp;amp; $&lt;/span&gt;6 == &amp;quot;G&amp;quot;) || (&lt;span class=&quot;math-container&quot;&gt;$5 == &quot;G&quot; &amp;amp;&amp;amp; $&lt;/span&gt;6 == &amp;quot;C&amp;quot;)&#xA;{&#xA;    print &lt;span class=&quot;math-container&quot;&gt;$2, &quot;ambig&quot;&#xA;    next&#xA;}&#xA;{&#xA;    print $&lt;/span&gt;2&#xA;}'&#xA;&#xA;awk &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${awkcmd}&quot; &quot;$&lt;/span&gt;{chrom}.step1b&amp;quot; | grep &amp;quot;ambig&amp;quot; &amp;gt; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}.step1b.snplist.txt&quot;&#xA;plink --bfile &quot;$&lt;/span&gt;{chrom}.step1b&amp;quot; --exclude &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}.step1b.snplist.txt&quot; --make-bed --out &quot;$&lt;/span&gt;{chrom}.step1c&amp;quot;&#xA;&#xA;# Step 2. Zero out Mendelian errors&#xA;plink --bfile &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}.step1c&quot; --me 1 1 --set-me-missing --make-bed --out &quot;$&lt;/span&gt;{chrom}.step2&amp;quot;&#xA;&#xA;# Step 3. Missingness and HWE + Step 4. Compute allele frequencies&#xA;# NB! --mind 0.02 filter is removed from the next command, to avoid sample QC per chunk, which may leave inconsistent number of individuals avail able per chunk.&#xA;plink --bfile &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}.step2&quot; --hwe 1e-6 --geno 0.02 --make-bed --out &quot;$&lt;/span&gt;{chrom}.step3&amp;quot;&#xA;plink --bfile &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}.step3&quot; --freq --make-bed --out &quot;$&lt;/span&gt;{chrom}.step4&amp;quot;&#xA;&#xA;# Step 4-10. Run Oxford perl script and apply first 5 lines of its Run-plink.sh command, followed by '--update-name' command (update marker name s)&#xA;HRC-1000G-check-bim.pl -b &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}.step4.bim&quot; -f &quot;$&lt;/span&gt;{chrom}.step4.frq&amp;quot; -r &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${hrc_sites_file}&quot; -h&#xA;plink --bfile &quot;$&lt;/span&gt;{chrom}.step4&amp;quot; --exclude &amp;quot;Exclude-&lt;span class=&quot;math-container&quot;&gt;${chrom}.step4-HRC.txt&quot; --make-bed --out &quot;$&lt;/span&gt;{chrom}.step5&amp;quot;&#xA;plink --bfile &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}.step5&quot; --update-chr &quot;Chromosome-$&lt;/span&gt;{chrom}.step4-HRC.txt&amp;quot; --make-bed --out &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}.step6&quot;&#xA;plink --bfile &quot;$&lt;/span&gt;{chrom}.step6&amp;quot; --update-map &amp;quot;Position-&lt;span class=&quot;math-container&quot;&gt;${chrom}.step4-HRC.txt&quot; --make-bed --out &quot;$&lt;/span&gt;{chrom}.step7&amp;quot;&#xA;plink --bfile &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}.step7&quot; --flip &quot;Strand-Flip-$&lt;/span&gt;{chrom}.step4-HRC.txt&amp;quot; --make-bed --out &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}.step8&quot;&#xA;plink --bfile &quot;$&lt;/span&gt;{chrom}.step8&amp;quot; --a2-allele &amp;quot;Force-Allele1-&lt;span class=&quot;math-container&quot;&gt;${chrom}.step4-HRC.txt&quot; --make-bed --out &quot;$&lt;/span&gt;{chrom}.step9&amp;quot;&#xA;plink --bfile &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}.step9&quot; --update-name &quot;ID-$&lt;/span&gt;{chrom}.step4-HRC.txt&amp;quot; --real-ref-alleles --make-bed --out &amp;quot;${chrom}.step10&amp;quot;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Make &lt;code&gt;PRE_PHASING_QC.sh&lt;/code&gt; executable (&lt;code&gt;chmod +x PRE_PHASING_QC.sh&lt;/code&gt;), and replace your job script with the following. This file won't be needed by your Nextflow process, but shows you how the above shell script can be invoked:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;#SBATCH --job-name=prephase&#xA;#SBATCH --account=p697_norment&#xA;#SBATCH --time=1:00:00&#xA;#SBATCH --ntasks=1&#xA;#SBATCH --mem-per-cpu=7600M&#xA;#SBATCH --cpus-per-task=7&#xA;#SBATCH --array=1-22&#xA;&#xA;set -e&#xA;&#xA;. /cluster/bin/jobsetup&#xA;&#xA;export PATH=&amp;quot;/cluster/projects/p697/github/norment/moba_qc_imputation/software:${PATH}&amp;quot;&#xA;&#xA;module load plink&#xA;&#xA;cd /path/to/hce-ec-eur-fin-pass-qc&#xA;&#xA;/path/to/PRE_PHASING_QC.sh \&#xA;    /cluster/projects/p697/projects/moba_qc_imputation/resources/HRC/HRC.r1-1.GRCh37.wgs.mac5.sites.tab.gz \&#xA;    hce-ec-eur-fin-pass-qc \&#xA;    &amp;quot;${SLURM_ARRAY_TASK_ID}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Copy &lt;code&gt;PRE_PHASING_QC.sh&lt;/code&gt; and the Oxford Perl script, &lt;code&gt;HRC-1000G-check-bim.pl&lt;/code&gt;, into your project's &lt;code&gt;bin/&lt;/code&gt; directory. You may also need to make the Perl script executable. Then your Nextflow process becomes:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;hrc_sites_file = file(params.hrc_sites_file)&#xA;&#xA;Channel&#xA;    .fromFilePairs( &amp;quot;${params.input_dir}/*.{bed,bim,fam}&amp;quot;, size:3 )&#xA;    .set { plink_data }&#xA;&#xA;Channel&#xA;    .of(1..22)&#xA;    .set { chroms }&#xA;&#xA;&#xA;process prephasing {&#xA;&#xA;    input:&#xA;    tuple val(prefix), path(bfiles) from plink_data&#xA;    each chrom from chroms&#xA;    path hrc_sites_file&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    PRE_PHASING_QC.sh &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${hrc_sites_file}&quot; &quot;$&lt;/span&gt;{prefix}&amp;quot; &amp;quot;${chrom}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Once the above runs successfully, you can declare your process outputs. Please note that none of the above was tested; hopefully not too many typos, but YMMV.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2021-03-30T14:15:10.390" LastActivityDate="2021-03-30T14:15:10.390" CommentCount="18" ContentLicense="CC BY-SA 4.0" />
  <row Id="15636" PostTypeId="1" CreationDate="2021-03-28T12:39:56.913" Score="-2" ViewCount="53" Body="&lt;p&gt;Do proteins move inside the cell and how fast do they move or move? Please support your answer with articles or books.&lt;/p&gt;&#xA;" OwnerUserId="11870" LastActivityDate="2021-03-28T13:09:18.157" Title="Do proteins move inside the cell?" Tags="|proteins|" AnswerCount="1" CommentCount="1" ClosedDate="2021-03-29T17:43:42.067" ContentLicense="CC BY-SA 4.0" />
  <row Id="15637" PostTypeId="2" ParentId="15636" CreationDate="2021-03-28T13:09:18.157" Score="2" Body="&lt;p&gt;Proteins move in cells or get transported in the cell. You can go to NCBI (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/&lt;/a&gt;), and start searching for protein dynamics within cells. Also there is a technique called FRAP (&lt;a href=&quot;https://en.wikipedia.org/wiki/Fluorescence_recovery_after_photobleaching&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Fluorescence_recovery_after_photobleaching&lt;/a&gt;) which is used to measure kinetics and dynamics.&lt;br /&gt;&#xA;Also, this differs probably a lot between the cellular system you are asking for. You can imagine that some cells have a more active transport system because stretched out (neurons) vs others with maybe less active transport systems, because of a more static nature.&lt;/p&gt;&#xA;" OwnerUserId="11872" LastActivityDate="2021-03-28T13:09:18.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15638" PostTypeId="1" AcceptedAnswerId="15639" CreationDate="2021-03-29T05:07:26.237" Score="1" ViewCount="206" Body="&lt;p&gt;I am getting a rather strange error from topTable.    When I run my code I get &lt;code&gt;Error in fit$coefficients[, coef] : subscript out of bounds&lt;/code&gt; from topTable as if it is not recognizing the condition I am asking it to return the top ten DEGs of. I have provided a bit more context and some descriptions of my variables below.  Let me know if you need any more information or if anything is unclear.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Variables:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;dge&lt;/code&gt; is a DGEList object of RNA seq data.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;dge$counts&lt;/code&gt; is the raw counts.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;dge$samples$dge_meta&lt;/code&gt; is the metadata and log2 transformed counts.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; colnames(dge&lt;span class=&quot;math-container&quot;&gt;$samples$&lt;/span&gt;dge_meta)&#xA; [1] &amp;quot;Lab.sample.ID&amp;quot;                                              &#xA; [2] &amp;quot;Accession.Number&amp;quot;                                           &#xA; [3] &amp;quot;Alias.used.in.some.figures&amp;quot;                                 &#xA; [4] &amp;quot;Developmental.stage&amp;quot;                                        &#xA; [5] &amp;quot;Infected&amp;quot;                                                   &#xA; [6] &amp;quot;Batch&amp;quot;                                                      &#xA; [7] &amp;quot;Trimmed&amp;quot;                                                    &#xA; [8] &amp;quot;Number.of.reads.that.pass.Illumina.filter&amp;quot;                  &#xA; [9] &amp;quot;Total.number.of.reads.mapped&amp;quot;                               &#xA;[10] &amp;quot;%.of.total.reads.mapped&amp;quot;                                    &#xA;[11] &amp;quot;Reads.mapped.to.T..cruzi.Esmeraldo.haplotype&amp;quot;               &#xA;[12] &amp;quot;Reads.mapped.to.hg19&amp;quot;                                       &#xA;[13] &amp;quot;%.of.mapped.reads.belonging.to.T..cruzi.Esmeraldo.haplotype&amp;quot;&#xA;[14] &amp;quot;%.of.mapped.reads.belonging.to.hg19&amp;quot;                        &#xA;[15] &amp;quot;sample&amp;quot;                                                     &#xA;[16] &amp;quot;hpi&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Lastly &lt;code&gt;keep_it&lt;/code&gt; is a binary vector used to filter our zero count genes.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Code:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mm &amp;lt;- model.matrix(~~hpi*Infected, data = dge&lt;span class=&quot;math-container&quot;&gt;$samples$&lt;/span&gt;dge_meta) #making our design matrix&#xA;&#xA;more_than_1cpm_model &amp;lt;- voom(dge$counts[keep_it,], mm, plot = TRUE) #this one is filtered by the 25% greater than 1 cpm&#xA;&#xA;cutoff &amp;lt;-  1e-05&#xA;&#xA;empBayesfit &amp;lt;- eBayes(fit_gt1cpm)&#xA;&#xA;topTable(empBayesfit,&#xA;         coef = c(&amp;quot;Infected&amp;quot;),&#xA;         p.value = cutoff, number = 10)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Error message:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in fit$coefficients[, coef] : subscript out of bounds&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To understand the context better I ran rlang::last_error and rlang::last_trace as well.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; rlang::last_error()&#xA;&amp;lt;error/dplyr_error&amp;gt;&#xA;arrange() failed at implicit mutate() step. &#xA;* Problem with `mutate()` input `..1`.&#xA;x Input `..1` must be a vector, not a `standardGeneric` object.&#xA;i Input `..1` is `samples`.&#xA;Backtrace:&#xA;Run `rlang::last_trace()` to see the full context.&#xA;&amp;gt; rlang::last_trace()&#xA;&amp;lt;error/dplyr_error&amp;gt;&#xA;arrange() failed at implicit mutate() step. &#xA;* Problem with `mutate()` input `..1`.&#xA;x Input `..1` must be a vector, not a `standardGeneric` object.&#xA;i Input `..1` is `samples`.&#xA;Backtrace:&#xA;     x&#xA;  1. +-dplyr::arrange(x, samples)&#xA;  2. +-dplyr:::arrange.data.frame(x, samples)&#xA;  3. | \-dplyr:::arrange_rows(.data, dots)&#xA;  4. |   +-base::withCallingHandlers(...)&#xA;  5. |   +-dplyr::transmute(new_data_frame(.data), !!!quosures)&#xA;  6. |   \-dplyr:::transmute.data.frame(new_data_frame(.data), !!!quosures)&#xA;  7. |     +-dplyr::mutate(.data, ..., .keep = &amp;quot;none&amp;quot;)&#xA;  8. |     \-dplyr:::mutate.data.frame(.data, ..., .keep = &amp;quot;none&amp;quot;)&#xA;  9. |       \-dplyr:::mutate_cols(.data, ...)&#xA; 10. |         +-base::withCallingHandlers(...)&#xA; 11. |         \-mask$eval_all_mutate(quo)&#xA; 12. +-dplyr:::abort_glue(...)&#xA; 13. | +-rlang::exec(abort, class = class, !!!data)&#xA; 14. | \-(function (message = NULL, class = NULL, ..., trace = NULL, parent = NULL, ...&#xA; 15. |   \-rlang:::signal_abort(cnd)&#xA; 16. |     \-base::signalCondition(cnd)&#xA; 17. +-(function (e) ...&#xA; 18. | \-rlang::abort(...)&#xA; 19. |   \-rlang:::signal_abort(cnd)&#xA; 20. |     \-base::signalCondition(cnd)&#xA; 21. \-(function (cnd) ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2482" LastActivityDate="2021-03-29T08:12:17.493" Title="Toptable error, wont recognize condition" Tags="|r|rna-seq|limma|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15639" PostTypeId="2" ParentId="15638" CreationDate="2021-03-29T08:12:17.493" Score="1" Body="&lt;p&gt;The error is telling you that there isn't any Infected column on the model.matrix.&#xA;Check the column names of &lt;code&gt;mm&lt;/code&gt;. Also not sure that having two &lt;code&gt;~&lt;/code&gt; will work well.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2021-03-29T08:12:17.493" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15642" PostTypeId="2" ParentId="15610" CreationDate="2021-03-29T14:28:11.653" Score="2" Body="&lt;p&gt;I assume your SNP file contains genomic positions - please clarify if not. For example, you could have an rsID column, chromsome, and position column that contain &amp;quot;rs3088379&amp;quot;, &amp;quot;chr1&amp;quot;, &amp;quot;54225281&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Approach #1:&lt;/strong&gt; use R, as you suggest.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;ensembldb&lt;/code&gt; is an R package that may be what you're looking for: &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/ensembldb.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/html/ensembldb.html&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You could map the genomic positions of your SNPs to gene names by adapting their workflow: &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/ensembldb/inst/doc/coordinate-mapping.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/vignettes/ensembldb/inst/doc/coordinate-mapping.html&lt;/a&gt;.&#xA;After you've mapped the genomic position, you could filter for your gene name.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Approach #2:&lt;/strong&gt; use bedtools.&lt;/p&gt;&#xA;&lt;p&gt;You can convert your Excel file of genomic positions to a tab-delimited BED format. &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/overview.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedtools.readthedocs.io/en/latest/content/overview.html&lt;/a&gt; has an overview. So, your first SNP would turn into&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1 54225280 54225281 rs3088379&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;--&amp;gt; make sure you turn 1-based to 0-based coordinates!&#xA;You could then download the coordinates of your gene in BED format from your favorite source, like &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTables?command=start&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC Table Browser&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Then, you run &lt;code&gt;bedtools intersect&lt;/code&gt; with your SNP BED and your gene BED. What remains is just that gene's SNPs.&lt;/p&gt;&#xA;" OwnerUserId="11883" LastActivityDate="2021-03-29T14:28:11.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15643" PostTypeId="2" ParentId="15591" CreationDate="2021-03-29T14:45:48.943" Score="3" Body="&lt;p&gt;A structural variant is an inversion, deletion, duplication, or translocation. A &amp;quot;breakend&amp;quot; is one endpoint of a structural variant. This paper (Alkan et al.  &lt;em&gt;Nature Review Genetics&lt;/em&gt; 2011) may help you understand: &lt;a href=&quot;https://www.nature.com/articles/nrg2958&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/nrg2958&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The Manta paper and documentation seem sparse on diagrams. DELLY, another structural variant caller, has diagrams in their publication. Their paper may be more of use: &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/28/18/i333/245403&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://academic.oup.com/bioinformatics/article/28/18/i333/245403&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Specific example:&lt;/strong&gt; Say you're sequencing a chronic myeloid leukemia (CML) tumor. You will likely find reads that half map to the ABL1 gene on chr9 and half map to the BCR gene on chr22 (&lt;a href=&quot;https://en.wikipedia.org/wiki/Philadelphia_chromosome&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Philadelphia_chromosome&lt;/a&gt;). The genomic position of the location where chr9 turns to chr22 is called the breakpoint. The positions on chr9 and chr22 are the breakends.&lt;/p&gt;&#xA;" OwnerUserId="11883" LastActivityDate="2021-03-29T14:45:48.943" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15645" PostTypeId="1" AcceptedAnswerId="15650" CreationDate="2021-03-29T17:21:17.560" Score="4" ViewCount="746" Body="&lt;p&gt;I found that &lt;a href=&quot;https://en.wikipedia.org/wiki/Lipinski%27s_rule_of_five&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lipinski's rule of five&lt;/a&gt; states that Log P (octanol-water partition coefficient, lipophilicity measure) usually should not exceed 5.&lt;/p&gt;&#xA;&lt;p&gt;Many papers about drug discovery machine learning models tell about maximization of &amp;quot;penalized logP&amp;quot;, but the paper they refer in the end does not contain any information on it. Let me show that.&lt;/p&gt;&#xA;&lt;p&gt;Let's check &lt;a href=&quot;https://arxiv.org/pdf/1802.04364.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;, here we read about penalized log P:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The second task is to produce novel molecules with desired properties. Following (Kusner et al., 2017), our target chemical&#xA;property y(·)is octanol-water partition coefficients (logP) penalized&#xA;by the synthetic accessibility (SA)score and number of long cycles&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;y(m) = logP(m)−SA(m)−cycle(m) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where cycle(m)counts the number of rings that have more than six atoms.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Then we go into &lt;a href=&quot;https://arxiv.org/pdf/1703.01925.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kusner et al., 2017&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For the second optimization problem, we follow (Gómez-Bombarelli et al., 2016b) and optimize the drug properties of molecules. Our goal&#xA;is to maximize the water-octanol partition coefficient (logP), an&#xA;important metric in drug design that characterizes the drug-likeness&#xA;of a molecule.As in Gómez-Bombarelli et al. (2016b) we consider a&#xA;penalized logP score that takes into account other molecular&#xA;properties such as ring size and synthetic accessibility (Ertl &amp;amp;&#xA;Schuffenhauer, 2009).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here is &lt;a href=&quot;https://arxiv.org/pdf/1610.02415.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gómez-Bombarelli, 2016&lt;/a&gt; and it seems to contain nothing on the topic (I can be mistaken, please point me then), except&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The objective we chose to optimize was 5×QED−SAS, where QED is the Quantitative Estimation of Drug-likeness (QED),37 and SAS is the&#xA;Synthetic Accessibility score.36Thisobjective represents a rough&#xA;estimate of finding the most drug-like molecule that is also easy to&#xA;synthesize.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Other examples of the papers also leading to the same papers are &lt;a href=&quot;https://arxiv.org/pdf/1806.02473.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;, &lt;a href=&quot;https://www.nature.com/articles/s41598-019-47148-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;, etc.&lt;/p&gt;&#xA;&lt;p&gt;What does penalized logP mean for drug-likeliness and why it should be maximized?&lt;/p&gt;&#xA;" OwnerUserId="11885" LastActivityDate="2022-07-01T11:51:40.963" Title="Why do molecular generation models maximize “penalized logP” as a measure of drug-likeliness?" Tags="|drugs|small-molecules|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15646" PostTypeId="1" AcceptedAnswerId="15675" CreationDate="2021-03-29T18:21:12.447" Score="6" ViewCount="116" Body="&lt;p&gt;We currently work on a project that will generate a collection of recombinant plasmids targeting over 3500 genes of the cyanobacterium Synechocystis sp. PCC 6803. We would like to share these designs in SynBioHub but we are not sure how best to go about it. The plasmids have been designed and now they are starting to be produced in batches, checked and finally transformed into Synechocystis to induce a single gene deletion by homologous recombination.&#xA;During the project, we will have:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Stage 1 – the initial plasmids designs, waiting for production. They&#xA;are worth sharing as the community could influence the production&#xA;queue.&lt;/li&gt;&#xA;&lt;li&gt;Stage 2 – the assembled plasmids. In some cases they will have&#xA;different structure than initial designs from stage 1. If nothing&#xA;else their description should reflect the status change to&#xA;'produced'.&lt;/li&gt;&#xA;&lt;li&gt;Stage 3 – plasmids are verified by sequencing (also to&#xA;check the barcodes are correct) and sequences attached.&lt;/li&gt;&#xA;&lt;li&gt;Stage 4 – plasmids will be transformed into Synechocystis and checked by PCR to&#xA;see if i) the target gene has been knocked out and ii) if the strain&#xA;is segregated (i.e. all copies of the genome in the strain contain&#xA;the gene deletion).  This information should also be recorded with&#xA;the description.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Our initial plan was:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Create a collection with initial designs (component definitions) Stage 1&lt;/li&gt;&#xA;&lt;li&gt;As stage 2 progresses add new versions of udpated designs&lt;/li&gt;&#xA;&lt;li&gt;Stage 3, 4 update plasmids descriptions (synbio desc or notes) and attach additional files (e.g. from sequencing).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, our current understanding is the public collections are frozen so stage 2,3,4 cannot be accommodated.&lt;/p&gt;&#xA;&lt;p&gt;We are seeking advice how to plan for sharing this resource and some clarifications.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there a way of making collection readable by others without freezing its content?&lt;/li&gt;&#xA;&lt;li&gt;What operations are possible on public component definitions, can the free text descriptions/notes be updated or file attached?&lt;/li&gt;&#xA;&lt;li&gt;Can new component definitions or their versions be added to a published collection&lt;/li&gt;&#xA;&lt;li&gt;For 2) 3) who can perform these operations, the collection’s creator or SynBio curators?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Looking forward to input from SynBioHub community&lt;/p&gt;&#xA;" OwnerUserId="11887" LastActivityDate="2021-04-02T17:11:27.007" Title="How to share a collection of sbol designs in SynBioHub" Tags="|sbol|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15647" PostTypeId="1" CreationDate="2021-03-30T05:48:35.110" Score="5" ViewCount="2368" Body="&lt;p&gt;I used the following command to remove duplicate FASTA sequences based on the header sequence:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;paste -d &lt;span class=&quot;math-container&quot;&gt;$'\t' - - &amp;lt;fastaFileWithNoLinebreaksInSeq | sort -t $&lt;/span&gt;'\t' -uk1,1 | awk 'BEGIN{FS=&amp;quot;\t&amp;quot;;OFS=&amp;quot;\n&amp;quot;}{print &lt;span class=&quot;math-container&quot;&gt;$1,$&lt;/span&gt;2}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After this command, the output result is shown below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;_10|_10&#xA;AAAGCAAAACACAGAATTGACGGAAAGACTTACGTTATTAAACGTGTTAAATATAATAACGAGTTTGGCATGGATTTTAAAGAAATAGAATTAATTGGCTCAGGTGGATTTGGCCAAGTTTTC&#xA;&amp;gt;_10|_11&#xA;ATTATGATCCTGAGACCAGTGATGATTCTCTTGAGAGCAGTGATTATGATCCTGAGAACAGCAAAAATAGTTCAAGGTTTGGCATGGATTTTAAAGAAATAGAATTAATTGGCTCAGGTGGATTTGGCCAAGTTTTCAAAGCAAAACACA&#xA;&amp;gt;_10|_12&#xA;TTTGGAACTCTTTGAACAAATAACAAAAGGGGTGGATTATATACATTCAAAAAAATTAATTCATAGAGATCTTAAGGTTTGGCATGGATTTTAAAGAAATAGAATTAATTGGCTCAGGTGGATTTGGCCAAGTTTTCAAAGCAAAACACA&#xA;&amp;gt;_10|_13&#xA;ACTTGTAACATCTCTGAAAAATGATGGAAAGCGAACAAGGAGTAAGGGAACTTTGCGATACATGAGCCCAGAACAGGTTTGGCATGGATTTTAAAGAAATAGAATTAATTGGCTCAGGTGGATTTGGCCAAGTTTTCAAAGCAAAACACA&#xA;&amp;gt;_10|_14&#xA;AACGAATTTCTTCGCAAGACTATGGAAAGGAAGTGGACCTCTACGCTTTGGGGCTAATTCTTGCTGAACTTCTTCATGTTTGGCATGGATTTTAAAGAAATAGAATTAATTGGCTCAGGTGGATTTGGCCAAGTTTTCAAAGCAAAACAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is my actual FASTA file: I don't want to lose header information from my FASTA sequence after removing duplicate sequences based on header information.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;ENST00000642616_2|ENST00000642616_10&#xA;AAAAAAAATCTAGCATAAAGTTAATTTAGTTGGAGGAAAAGGAGGGAAGTGGATAAAGGGGCAGATGGAGGGAAGTAAAATGAAATCACAGGCGCATTACTGCCTTGTCTTGATGAGTCTCGTTTTGAAAATCTTGGAGTAAGCATGAAC&#xA;&amp;gt;ENST00000642616_2|ENST00000642616_10&#xA;AAAAAAAATCTAGCATAAAGTTAATTTAGTTGGAGGAAAAGGAGGGAAGTGGATAAAGGGGCAGATGGAGGGAAGTAAAATGAAATCACAGGCGCATTACTGCCTTGTCTTGATGAGTCTCGTTTTGAAAATCTTGGAGTAAGGAAGTTG&#xA;&amp;gt;ENST00000642616_2|ENST00000642616_10&#xA;AAAAAAAATCTAGCATAAAGTTAATTTAGTTGGAGGAAAAGGAGGGAAGTGGATAAAGGGGCAGATGGAGGGAAGTAAAATGAAATCACAGGCGCATTACTGCCTTGTCTTGATGAGTCTCGTTTTGAAAATCTTGGAGTAAGGGTGGGG&#xA;&amp;gt;ENST00000642616_2|ENST00000642616_10&#xA;AAAAAAAATCTAGCATAAAGTTAATTTAGTTGGAGGAAAAGGAGGGAAGTGGATAAAGGGGCAGATGGAGGGAAGTAAAATGAAATCACAGGCGCATTACTGCCTTGTCTTGATGAGTCTCGTTTTGAAAATCTTGGAGTAAGGTAATTA&#xA;&amp;gt;ENST00000642616_2|ENST00000642616_10&#xA;AAAAAAAATCTAGCATAAAGTTAATTTAGTTGGAGGAAAAGGAGGGAAGTGGATAAAGGGGCAGATGGAGGGAAGTAAAATGAAATCACAGGCGCATTACTGCCTTGTCTTGATGAGTCTCGTTTTGAAAATCTTGGAGTAAGGTCATGG&#xA;&amp;gt;ENST00000642616_2|ENST00000642616_10&#xA;AAAAAAAATCTAGCATAAAGTTAATTTAGTTGGAGGAAAAGGAGGGAAGTGGATAAAGGGGCAGATGGAGGGAAGTAAAATGAAATCACAGGCGCATTACTGCCTTGTCTTGATGAGTCTCGTTTTGAAAATCTTGGAGTAAGTTCCTTG&#xA;&amp;gt;ENST00000642616_8|ENST00000642616_10&#xA;AAAAAAAATCTAGCATAAAGTTAATTTAGTTGGAGGAAAAGGAGGGAAGTGGATAAAGGGGCAGATGGAGGGAAGTATTTGGTGCCTATCCTAAAACTTCCAGTGGGTTTTCTTGCCTTCCAGTGGCCATATAAAGGGCGTCCTGAAAAC&#xA;&amp;gt;ENST00000642616_7|ENST00000642616_10&#xA;AAAAAAAATCTAGCATAAAGTTAATTTAGTTGGAGGAAAAGGAGGGAAGTGGATAAAGGGGCAGATGGAGGGAAGTCATGAACAAGGCTCAGTTATGATGTTTGAGCACCTTATTAATTCTAATGGAATTAAGCCTGTCATGGAACAATA&#xA;&amp;gt;ENST00000642616_1|ENST00000642616_10&#xA;AAAAAAAATCTAGCATAAAGTTAATTTAGTTGGAGGAAAAGGAGGGAAGTGGATAAAGGGGCAGATGGAGGGAAGTGCGCCTGCGCGCGGGTCCGGCGCCGAGGTTCTTGACTGCTGTGCCGGACGCCAGGTGTAGCCATGCAGCGAGCC&#xA;&amp;gt;ENST00000642616_9|ENST00000642616_10&#xA;AAAAAAAATCTAGCATAAAGTTAATTTAGTTGGAGGAAAAGGAGGGAAGTGGATAAAGGGGCAGATGGAGGGAAGTGGACTGCCATCATCTTGGAATCCAAAATAATTTTGATTACAAGCGCTTTATTAAGTTTGCCCGTGTCTGTGAAG&#xA;&amp;gt;ENST00000642616_5|ENST00000642616_10&#xA;AAAAAAAATCTAGCATAAAGTTAATTTAGTTGGAGGAAAAGGAGGGAAGTGGATAAAGGGGCAGATGGAGGGAAGTGGTGGGGTATCTAGCAGGATGTCTAGTTCACGCACTGGGTGAAAAACAACCAGAGCTGCAGATAAGTGAACGAG&#xA;&amp;gt;ENST00000642616_4|ENST00000642616_10&#xA;AAAAAAAATCTAGCATAAAGTTAATTTAGTTGGAGGAAAAGGAGGGAAGTGGATAAAGGGGCAGATGGAGGGAAGTGTAATTAATGATCCTATCCATGGCCACATTGAGCTCCACCCTCTCCTCGTCCGAATCATTGATACACCTCAATT&#xA;&amp;gt;ENST00000642616_6|ENST00000642616_10&#xA;AAAAAAAATCTAGCATAAAGTTAATTTAGTTGGAGGAAAAGGAGGGAAGTGGATAAAGGGGCAGATGGAGGGAAGTGTCATGGGCCATTTTCTCACATGTTTGATGGACGATTTATTCCACTTGCTCGCCCGGAGGTGAAATGGACGATT&#xA;&amp;gt;ENST00000642616_6|ENST00000642616_10&#xA;AAAAAAAATCTAGCATAAAGTTAATTTAGTTGGAGGAAAAGGAGGGAAGTGGATAAAGGGGCAGATGGAGGGAAGTGTCATGGGCCATTTTCTCACATGTTTGATGGACGATTTATTCCACTTGCTCGCCCGGAGGTGAAATGGACGCAT&#xA;&amp;gt;ENST00000642616_6|ENST00000642616_10&#xA;AAAAAAAATCTAGCATAAAGTTAATTTAGTTGGAGGAAAAGGAGGGAAGTGGATAAAGGGGCAGATGGAGGGAAGTGTCATGGGCCATTTTCTCACATGTTTGATGGACGATTTATTCCACTTGCTCGCCCGGAGGTGAAATGGACGGAA&#xA;&amp;gt;ENST00000642616_3|ENST00000642616_10&#xA;AAAAAAAATCTAGCATAAAGTTAATTTAGTTGGAGGAAAAGGAGGGAAGTGGATAAAGGGGCAGATGGAGGGAAGTTTCCTTGGGGGAGAGGAAGAAGCTGCTTAGTTATATCCAGCGATTGGTTCAAATCCACGTTGATACAATGAAGA&#xA;&amp;gt;ENST00000642616_3|ENST00000642616_10&#xA;AAAAAAAATCTAGCATAAAGTTAATTTAGTTGGAGGAAAAGGAGGGAAGTGGATAAAGGGGCAGATGGAGGGAAGTTTCCTTGGGGGAGAGGAAGAAGCTGCTTAGTTATATCCAGCGATTGGTTCAAATCCACGTTGATACAATGAAGC&#xA;&amp;gt;ENST00000642616_3|ENST00000642616_10&#xA;AAAAAAAATCTAGCATAAAGTTAATTTAGTTGGAGGAAAAGGAGGGAAGTGGATAAAGGGGCAGATGGAGGGAAGTTTCCTTGGGGGAGAGGAAGAAGCTGCTTAGTTATATCCAGCGATTGGTTCAAATCCACGTTGATACAATGAAGG&#xA;&amp;gt;ENST00000462861_2|ENST00000462861_3&#xA;AAAAAACTCTTCTACAGAAATTACTCTCAAAGAAACCTGAGGATCGACCTAATTTTTCACAGACCTACGGGATGGCATCATCTCAGATATATTTGATAAAAAAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11890" LastEditorUserId="73" LastEditDate="2021-05-10T02:20:12.580" LastActivityDate="2021-05-10T02:20:12.580" Title="Removing duplicate FASTA sequences based on headers with Bash" Tags="|fasta|bash|linux|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15648" PostTypeId="2" ParentId="15647" CreationDate="2021-03-30T09:03:23.167" Score="9" Body="&lt;p&gt;You can use &lt;a href=&quot;https://bioinf.shenwei.me/seqkit/&quot; rel=&quot;noreferrer&quot;&gt;seqkit&lt;/a&gt; for this purpose.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;seqkit rmdup -n seqs.fa -o seqs_without_duplicate.fa&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2021-03-30T09:03:23.167" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15649" PostTypeId="2" ParentId="15647" CreationDate="2021-03-30T09:38:16.833" Score="10" Body="&lt;p&gt;If you want to avoid using extra libraries for any reason, you can just use a simple Python script (version 3.6 and above) to do this:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;fr = open(&amp;quot;dup_test.fasta&amp;quot;, &amp;quot;r&amp;quot;)&#xA;fw = open(&amp;quot;dup_edited.fasta&amp;quot;, &amp;quot;w&amp;quot;)&#xA;seq_dict = {}&#xA;curr_header = ''&#xA;&#xA;for line in fr:&#xA;   line = line.strip()&#xA;   if line[0] == '&amp;gt;':&#xA;        if line not in seq_dict:&#xA;          seq_dict[line] = ''&#xA;          curr_header = line&#xA;   else:&#xA;        if seq_dict[curr_header] == '':&#xA;          seq_dict[curr_header] = line&#xA;          fw.write(f&amp;quot;{curr_header}\n{line}\n&amp;quot;)&#xA;&#xA;fr.close()&#xA;fw.close()&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11856" LastActivityDate="2021-03-30T09:38:16.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15650" PostTypeId="2" ParentId="15645" CreationDate="2021-03-30T10:02:56.070" Score="2" Body="&lt;p&gt;&lt;strong&gt;A positive logP is good (hydrophobic)&lt;/strong&gt;, but high results in ADMET problems (cytochromes break overly hydrophobic compounds) and aggregation (gunk, tar, crud are words happily thrown around). However, in most cases the logP is not greater than +3 because good binding is result of hydrogen bonds etc —some bad algorithms will suggest a polyaromatic hydrocarbon that makes lots of pi-pi interactions but these are few.&lt;/p&gt;&#xA;&lt;p&gt;Confusingly, &lt;strong&gt;a high synthetic accessibility score is bad (hard)&lt;/strong&gt;. 3 is the average and 10 is basically something impossible. This score is how hard it is to make (i.e. pricier), which is one of the major drivers.&lt;/p&gt;&#xA;&lt;p&gt;The number of cycles is number of rings. Even if you get pi-pi, cation-pi and sulfur-pi bonds you also get aggregation due to stacking (a crud). So is somewhat bad.&lt;/p&gt;&#xA;&lt;p&gt;So basically that score is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;high=good minus high=bad minus high=bad&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If difference in relative Gibbs energy of binding is thrown it, negative ∆∆G is good, so that would be subtracted. As its atom count dependent ligand efficiency is better, but that ranges from 0 to -1.2 kcal/mol. So would need scaling.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2021-03-30T10:08:00.793" LastActivityDate="2021-03-30T10:08:00.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15651" PostTypeId="1" CreationDate="2021-03-30T10:31:38.887" Score="2" ViewCount="87" Body="&lt;p&gt;I'm trying to compare the output of 2 different algorithms of RNA structure prediction&#xA;(my implementation of Nussinov vs RNA-mfold algorithm) using the &lt;a href=&quot;https://www.tbi.univie.ac.at/RNA/RNAdistance.1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNAdistance&lt;/a&gt; algorithm that is part of &lt;a href=&quot;https://www.tbi.univie.ac.at/RNA/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ViennaRNA&lt;/a&gt; package.&#xA;I'm getting a distance score of 38 between Nussinov prediction and the actual NMR structure, and a score of 14 between the m-fold algorithm and the NMR structure and I'm trying to understand whether it is a big difference.&lt;/p&gt;&#xA;&lt;p&gt;In order to do that I have to look at the scoring table that is used by RNAdistance and found the following &lt;code&gt;score table&lt;/code&gt; in the &lt;a href=&quot;https://github.com/ViennaRNA/ViennaRNA/blob/ea2e71f98c3a6a98310fa5f25e9626e970f112d9/src/ViennaRNA/edit_cost.h&quot; rel=&quot;nofollow noreferrer&quot;&gt;source code&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;RIVATE CostMatrix  UsualCost =&#xA;{&#xA;&#xA;/*    Null,       U,        P,        H,        B,        I,        M,        S,        E,        R     */&#xA;&#xA;   {        0,        1,        2,        2,        2,        2,        2,        1,        1, DIST_INF},   /* Null replaced */&#xA;   {        1,        0,        1, DIST_INF, DIST_INF, DIST_INF, DIST_INF, DIST_INF, DIST_INF, DIST_INF},   /* U    replaced */&#xA;   {        2,        1,        0, DIST_INF, DIST_INF, DIST_INF, DIST_INF, DIST_INF, DIST_INF, DIST_INF},   /* P    replaced */&#xA;   {        2, DIST_INF, DIST_INF,        0,        2,        2,        2, DIST_INF, DIST_INF, DIST_INF},   /* H    replaced */&#xA;   {        2, DIST_INF, DIST_INF,        2,        0,        1,        2, DIST_INF, DIST_INF, DIST_INF},   /* B    replaced */&#xA;   {        2, DIST_INF, DIST_INF,        2,        1,        0,        2, DIST_INF, DIST_INF, DIST_INF},   /* I    replaced */&#xA;   {        2, DIST_INF, DIST_INF,        2,        2,        2,        0, DIST_INF, DIST_INF, DIST_INF},   /* M    replaced */&#xA;   {        1, DIST_INF, DIST_INF, DIST_INF, DIST_INF, DIST_INF, DIST_INF,        0, DIST_INF, DIST_INF},   /* S    replaced */&#xA;   {        1, DIST_INF, DIST_INF, DIST_INF, DIST_INF, DIST_INF, DIST_INF, DIST_INF,        0, DIST_INF},   /* E    replaced */&#xA;   { DIST_INF, DIST_INF, DIST_INF, DIST_INF, DIST_INF, DIST_INF, DIST_INF, DIST_INF, DIST_INF,        0},   /* R    replaced */&#xA;&#xA;};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also found the &lt;a href=&quot;https://www.tbi.univie.ac.at/RNA/ViennaRNA/doc/html/distance_measures.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;enter link description here&lt;/a&gt;following explanation, but &lt;strong&gt;I do not understand what is the meaning of the table letters: U,P,H,B,I,MS,E,R ?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;What is the meaning of the score of [H,B] = 2 for example?&#xA;If anyone can explain the table it will be a great help in my project, thank you!&lt;/p&gt;&#xA;" OwnerUserId="10109" LastActivityDate="2021-04-29T19:07:30.653" Title="Understanding ViennaRNA RNAdistance scoring table" Tags="|sequence-alignment|rna|rna-alignment|rna-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15652" PostTypeId="1" AcceptedAnswerId="15653" CreationDate="2021-03-30T16:48:00.990" Score="1" ViewCount="99" Body="&lt;p&gt;I need to set the substitution matrix used by command line CLUSTALW when comparing DNA sequences to:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; 0 -1 -1 -1&#xA;-1  0 -1 -1 &#xA;-1 -1  0 -1&#xA;-1 -1 -1  0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;from my understanding I need to pass the location of a file with the matrix to the &lt;code&gt;-dnamatrix&lt;/code&gt; parameter, however, I cannot find out what format the file is supposed to be in. Any idea?&lt;/p&gt;&#xA;" OwnerUserId="9815" LastActivityDate="2021-03-30T17:32:12.700" Title="File format of substitution matrix in clustalw" Tags="|sequence-alignment|multiple-sequence-alignment|substitution-model|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15653" PostTypeId="2" ParentId="15652" CreationDate="2021-03-30T17:02:42.523" Score="3" Body="&lt;p&gt;This &lt;a href=&quot;https://www.biostars.org/p/133912/&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt; seems to have information on this, looks like it should have this format:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   A   G   C   T   U   *&#xA;A  1   -1  -1  -1  -1  -1&#xA;G  -1  1   -1  -1  -1  -1&#xA;C  -1  -1  1   -1  -1  -1&#xA;T  -1  -1  -1  1   -1  -1&#xA;U  -1  -1  -1  -1  1   -1&#xA;*  -1  -1  -1  -1  -1  -1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, the page I linked is specifically about a bug regarding ClustalW &lt;strong&gt;ignoring&lt;/strong&gt; custom made matrices. It's a pretty old post so it might have been fixed since. However, if you encounter that bug, that same page thankfully does contain a workaround!&lt;/p&gt;&#xA;" OwnerUserId="11856" LastEditorUserId="11856" LastEditDate="2021-03-30T17:32:12.700" LastActivityDate="2021-03-30T17:32:12.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15654" PostTypeId="2" ParentId="15651" CreationDate="2021-03-30T18:18:24.413" Score="2" Body="&lt;p&gt;Ok, the answer was right &lt;a href=&quot;https://www.tbi.univie.ac.at/RNA/RNAdistance.1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and I missed it, posting it in case anyone else will do the same mistake as me in the future.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;U = unpaired&#xA;P = paired&#xA;H = hairpin&#xA;B = bulge&#xA;I = internal loop&#xA;M = multiloop&#xA;S = stem (or stack)&#xA;E = external loop&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10109" LastActivityDate="2021-03-30T18:18:24.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15655" PostTypeId="2" ParentId="15632" CreationDate="2021-03-30T20:37:08.613" Score="3" Body="&lt;p&gt;I assume that you mean &amp;quot;is homologous to&amp;quot; by &amp;quot;encodes the same protein as&amp;quot;. Identical (&amp;quot;same&amp;quot;) proteins should be fairly trivial to identify using a tool such as tblastx.&lt;/p&gt;&#xA;&lt;p&gt;You should probably not use E-value for this. E-value is similar to a p-value, in that it represents some aspect of the data (in this case a sequence and a database) compared to some null hypothesis (meaning in this case statistics on the database). Therefore, it depends both on the specifics of the data and the hypothesis, and there is no universal threshold that will work across all sequences and all databases. (See discussion of the meaning of &lt;a href=&quot;https://www.researchgate.net/post/What_is_E-value_exactly_means_and_what_is_it_1e-63_representing&quot; rel=&quot;nofollow noreferrer&quot;&gt;e-value&lt;/a&gt;.)&lt;/p&gt;&#xA;&lt;p&gt;E-value does not in any way represent the content of the sequence or its relationship to a sequence in the database. You could have E-values close to zero or E-values &amp;gt; 10 for a perfect sequence match or a very loose sequence match, depending on the input sequence and the database.&lt;/p&gt;&#xA;&lt;p&gt;However, &lt;a href=&quot;http://www.metagenomics.wiki/tools/blast/blastn-output-format-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;BLAST output&lt;/a&gt; does include other information that might be helpful. You could (for example) look at the length of the alignment and the number of mismatches/identities in the alignment. This will give you a pretty good guess at how similar the two sequences are; unless they are very divergent you would expect them to align across most of the length with somewhat high protein identity or similarity (&amp;gt;25% probably?). This process will still not work 100% of the time, but I think it is closer to your intent than any given approach involving E-values.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2021-03-30T21:48:11.583" LastActivityDate="2021-03-30T21:48:11.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15656" PostTypeId="1" CreationDate="2021-03-31T05:44:46.417" Score="4" ViewCount="255" Body="&lt;p&gt;This is may not be entirely technical question but rather a academic question. But the technique behind the application is within the scope of bioinformatics. So I would try to ask here that:&lt;/p&gt;&#xA;&lt;p&gt;In each cancer type, there have been tons of papers that describe RNA-Seq based subtyping resulting in molecularly distinct subgroups with survival difference. However, only very few of them gained acceptance at clinical setting, such as breast cancer (luminal subtype A, luminal subtype B, normal breast-like subtype, HER-2 overexpression subtype and basal-like subtype).&lt;/p&gt;&#xA;&lt;p&gt;My question is: why did this molecular subtyping of breast cancer gain much acceptance but not others? Did this one own some advantages that other did not?&lt;/p&gt;&#xA;" OwnerUserId="4905" LastActivityDate="2021-03-31T09:35:29.433" Title="Why did expression based subtypng of breast cancer gain much more acceptance than others" Tags="|rna-seq|gene-expression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15658" PostTypeId="2" ParentId="15656" CreationDate="2021-03-31T09:35:29.433" Score="6" Body="&lt;h2&gt;In two words: incidence and funding&lt;/h2&gt;&#xA;&lt;p&gt;I'm not an expert on this topic, but I assume it has something to do with the &lt;a href=&quot;https://www.cancer.org/cancer/breast-cancer/about/how-common-is-breast-cancer.html#:%7E:text=Breast%20cancer%20is%20the%20most,she%20will%20develop%20breast%20cancer.&quot; rel=&quot;noreferrer&quot;&gt;incidence of breast cancer itself&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Breast cancer is the most common cancer in American women, except for skin cancers. Currently, the average risk of a woman in the United States developing breast cancer sometime in her life is about 13%. This means there is a 1 in 8 chance she will develop breast cancer. This also means there is a 7 in 8 chance she will never have the disease.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This incidence most likely amplifies how big of a problem society/funders/governments consider breast cancer to be. For example, a study performed by &lt;a href=&quot;https://www.cancerhealth.com/article/cancers-better-funded-others&quot; rel=&quot;noreferrer&quot;&gt;Northwestern University’s Feinberg School of Medicine and Robert H. Lurie Comprehensive Cancer Center&lt;/a&gt; showed that breast cancer research is by and large one of the most well-funded areas of cancer research.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AjVlF.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AjVlF.png&quot; alt=&quot;How common breast cancer is has likely affected how much funding it gets&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This higher amount of funding leads to more and better resources and research into every aspect of breast cancer, from identification to treatment. I would wager better funding has had the effect of increasing the level of research performed in the usage of sub-typing for identifying people at risk of getting breast cancer. &lt;strong&gt;Given positive results and lots of literature backing up such a method as useful, its levels of acceptance in clinical settings increased.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="11856" LastActivityDate="2021-03-31T09:35:29.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15659" PostTypeId="1" CreationDate="2021-03-31T13:53:39.927" Score="1" ViewCount="315" Body="&lt;p&gt;I have a dataset that has two groups, perturbed vs control. Each group has 3 replicates. Each replicate has 8 time points. How do we do Differential gene expression analysis to find significantly expressed genes? Any packages in R or web tools etc?&lt;/p&gt;&#xA;" OwnerUserId="10823" LastActivityDate="2021-04-01T20:17:16.877" Title="Differential gene expression analysis of time series with replicates" Tags="|rna-seq|gene|differential-expression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15660" PostTypeId="1" CreationDate="2021-03-31T14:42:11.910" Score="1" ViewCount="47" Body="&lt;p&gt;I came across this &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S1097276520308285&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; as one of the examples from this paper, this one&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Figure 2. Host Protein Alterations in Infected iAT2s&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(B) Dysregulated (FDR &amp;lt; 0.05) functional modules, including (bottom) U7 snRNA and HDAC-associated complexes.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So how should be the data frame should be for my clustering and visualization.&#xA;If i have to replicate the same for lets say 4 different time point how to do that.What i have done so far.&lt;/p&gt;&#xA;&lt;p&gt;I have 4 condition C1,C2,C3,C4. I did pairwise comparison between C1vC2, C2vC3 , C3vC4.&lt;/p&gt;&#xA;&lt;p&gt;So now I did pathway enrichment with the comparison above. I have for C1vC2, then C2vC3 and C3vC4.&lt;/p&gt;&#xA;&lt;p&gt;Now I how to proceed what they have shown here. Do i intersect all the pathway that is common in all three? such that based on the any metric that is present I can cluster and show the differences or is there any simpler way.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SbPDT.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SbPDT.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2021-03-31T14:42:11.910" Title="Data format for pathway based clustering of samples" Tags="|r|rna-seq|complexheatmap|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15661" PostTypeId="1" CreationDate="2021-03-31T17:01:34.697" Score="1" ViewCount="51" Body="&lt;p&gt;I'm familiar with running Snakemake in a DRMAA-enabled grid environment. (Most often, I write Python programs as wrappers and invoke the workflow via Snakemake's Python API.) Typically, I'm running workflows with rules that perform large, multi-threaded tasks. The rules are usually designed to utilize all available processors on a node, and only a single task is executed on any given node at a time.&lt;/p&gt;&#xA;&lt;p&gt;But what if I have thousands of single-threaded tasks that I want to distribute among the requested grid nodes? If the corresponding rule(s) are marked &lt;code&gt;threads: 1&lt;/code&gt;, is Snakemake smart enough to run multiple tasks on each node? How does it know how many tasks can be run on an individual node—or how does the user specify a limit?&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2021-03-31T17:01:34.697" Title="Load balancing numerous single-threaded processes with Snakemake on the grid" Tags="|python|snakemake|hpc|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15662" PostTypeId="1" AcceptedAnswerId="15674" CreationDate="2021-03-31T21:43:58.353" Score="7" ViewCount="49" Body="&lt;p&gt;In our lab, we work with synthetic biology components with partially random sequences (similar to work in directed evolution). So, for example, we have a plasmid design with several components, including one protein coding sequence that will be created from a wild-type sequence via mutagenic PCR.&lt;/p&gt;&#xA;&lt;p&gt;At the design stage, we don't know the exact DNA sequence for any of the resulting variants of that protein coding sequence. We just know the wild-type sequence and the average number of mutations we'd like to apply.&#xA;How can we represent that design in SBOL (via the &lt;code&gt;Component&lt;/code&gt; class)?&lt;/p&gt;&#xA;&lt;p&gt;Then, when we implement the design, we end up with a single sample that contains plasmids with 10^5-10^6 (or more) different protein coding sequences. We then measure the complete DNA sequence for every plasmid variant, but I don't want to end up with &amp;gt;10^5 different SBOL definitions to describe the sample.&#xA;How can we best represent that sample in SBOL (via the &lt;code&gt;Implementation&lt;/code&gt; class)?&lt;/p&gt;&#xA;&lt;p&gt;Finally, we also end up measuring the performance of every plasmid variant, and we identify the specific protein coding sequences for useful and/or interesting variants. We then design and build specific, clonal plasmids with those useful/interesting protein coding sequences.&#xA;Representing those clonal plasmids as SBOL &lt;code&gt;Components&lt;/code&gt; seems pretty straight-forward, but is there a good way to capture the relationships with the original random-library plasmid design and the specific implementation of that design that we built and measured?&lt;/p&gt;&#xA;" OwnerUserId="11905" LastActivityDate="2021-04-02T11:54:33.340" Title="How to represent random sequence elements in SBOL?" Tags="|sbol|variants|library|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15663" PostTypeId="1" AcceptedAnswerId="15667" CreationDate="2021-04-01T02:38:37.587" Score="3" ViewCount="299" Body="&lt;p&gt;I would like to gather a list of ALL E coli NCBI (Entrez/Nucleotide) IDs so that I can download fna and faa files, through a function like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# python code #&#xA;from Bio import Entrez&#xA;from glob import glob&#xA;&#xA;def dl_genome(id, folder='genomes'):&#xA;    files=glob('%s/*.fna'%folder)&#xA;    out_file = '%s/%s.fna'%(folder, id)&#xA;&#xA;    if out_file in files:&#xA;        print (out_file, 'already downloaded')&#xA;        return&#xA;    else:&#xA;        print ('downloading %s from NCBI'%id)&#xA;        &#xA;    Entrez.email = &amp;quot;&amp;quot;&#xA;    handle = Entrez.efetch(db=&amp;quot;nucleotide&amp;quot;, id=id, rettype=&amp;quot;fna&amp;quot;, retmode=&amp;quot;text&amp;quot;)&#xA;    with open(out_file, 'w') as fout:&#xA;        fout.write(handle.read())&#xA;&#xA;for strain in targetStrainIDs:&#xA;    dl_genome(strain, folder='genomes')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My problem is that I don't have a good way to get the E coli strain names and their corresponding NCBI IDs for Entrez. Is there a way to get these NCBI IDs without manually searching for them?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Context&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have been trying to develop an E coli pan-genome for one of my projects. I initially just downloaded the relevant genomes (fna, faa, &amp;amp; features.tab files) from the PATRIC database. However, I've come to realize that the reference sequence on PATRIC has issues. Specifically, the reference sequence on PATRIC's fna/faa files for E coli K12 MG1655 notably do NOT list genes like b2031 or b2033, despite both being present in this strain.&lt;/p&gt;&#xA;&lt;p&gt;The NCBI reference genome DOES include these genes (with proper annotations), so I'd like to just download what I need from NCBI instead. However, I'd like a systematic way to download all the fna/faa files since my pan-genome is eventually going to be built from hundreds of of these fna/faa files.&lt;/p&gt;&#xA;&lt;p&gt;Any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="11907" LastActivityDate="2021-04-01T14:15:59.020" Title="How to obtain NCBI Nucleotide (Entrez) IDs for all E coli sequences?" Tags="|phylogenetics|entrez|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15665" PostTypeId="2" ParentId="15663" CreationDate="2021-04-01T11:25:41.290" Score="6" Body="&lt;p&gt;Do you mean the NCBI taxonomy IDs? If so, I've personally always preferred the way &lt;a href=&quot;https://www.uniprot.org/taxonomy/&quot; rel=&quot;noreferrer&quot;&gt;UniProt displays NCBI taxonomy&lt;/a&gt;, and it's easier to download IDs in mass.&lt;/p&gt;&#xA;&lt;p&gt;For example, if you find &lt;em&gt;E.coli&lt;/em&gt;'s entry: &lt;a href=&quot;https://www.uniprot.org/taxonomy/562&quot; rel=&quot;noreferrer&quot;&gt;https://www.uniprot.org/taxonomy/562&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can then click on &amp;quot;all lower taxonomy nodes&amp;quot;, which will show all &lt;em&gt;E.coli&lt;/em&gt; strains. If you click 'Preview first 10' before downloading, you'll see that the first column is the NCBI taxonomy ID of each strain of &lt;em&gt;E.coli&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For example, &lt;em&gt;E.coli&lt;/em&gt; strain 0.1197 is reported to have the ID &lt;strong&gt;1005443&lt;/strong&gt; from this preview page. Indeed, if we search this ID on NCBI's taxonomy page, we do get the correct entry: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?id=1005443&quot; rel=&quot;noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?id=1005443&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If this is what you want, click 'download all' to get a nice tab-separated file of all the strains of &lt;em&gt;E.coli&lt;/em&gt; and their NCBI taxonomy IDs. This file could then be read and processed before you send API requests to Entrez.&lt;/p&gt;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="11856" LastEditorUserId="11856" LastEditDate="2021-04-01T11:33:07.760" LastActivityDate="2021-04-01T11:33:07.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15666" PostTypeId="1" AcceptedAnswerId="15671" CreationDate="2021-04-01T13:08:40.520" Score="1" ViewCount="506" Body="&lt;p&gt;Is there a way we can get 2d/3d structural coordinates like in (x/y/z) manner from a pdb file.&lt;/p&gt;&#xA;&lt;p&gt;I have looked a believe that the coordinates are in below line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATOM   1067  NH1 ARG A 141     -10.147   7.455  -6.079  1.00 23.24           N  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but i am unable to make a sense from it as i don't know which one is (x/y/z) in (2d/3d) coordinate system.&lt;/p&gt;&#xA;&lt;p&gt;If anyone describe it, that'll be a great help.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="10761" LastActivityDate="2021-04-02T09:53:11.870" Title="How to get 2d/3d structural coordinates from a PDB file?" Tags="|proteins|protein-structure|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15667" PostTypeId="2" ParentId="15663" CreationDate="2021-04-01T14:15:59.020" Score="1" Body="&lt;p&gt;You should check out &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Datasets&lt;/a&gt; for this sort of thing. Since this is bacteria, you won't be able to use the web interface but there's an &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets/docs/datasets-api/&quot; rel=&quot;nofollow noreferrer&quot;&gt;API&lt;/a&gt; and a &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets/docs/command-line-start/&quot; rel=&quot;nofollow noreferrer&quot;&gt;command-line tool&lt;/a&gt; that you can use.&lt;/p&gt;&#xA;&lt;p&gt;For example, you can use the &lt;code&gt;datasets&lt;/code&gt; command line tool to download the E. coli RefSeq genome assemblies as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## see how many genomes are in scope&#xA;$ datasets summary genome taxon 562 --limit none --refseq&#xA;{&amp;quot;total_count&amp;quot;:22287}&#xA;&#xA;## download data excluding gff3, rna and genome seq files&#xA;## as an example but drop --exclude-seq to download genome seq as well&#xA;$ datasets download genome taxon 562 --refseq --exclude-gff3 --exclude-rna --exclude-seq --filename ecoli_genomes.zip&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2021-04-01T14:15:59.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15668" PostTypeId="2" ParentId="15659" CreationDate="2021-04-01T20:17:16.877" Score="1" Body="&lt;p&gt;R packages like &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/DESeq2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2&lt;/a&gt;, &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/edgeR.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;edgeR&lt;/a&gt;, and &lt;a href=&quot;https://github.com/pachterlab/sleuth&quot; rel=&quot;nofollow noreferrer&quot;&gt;sleuth&lt;/a&gt; will help you perform differential gene expression analysis.&#xA;These are typically used for control-vs-treatment experimental designs.&#xA;So you could use them like this and perform a bunch of pair-wise comparisons between the time points (i.e. what are the differentially expressed genes between timepoint &lt;span class=&quot;math-container&quot;&gt;$t_0$&lt;/span&gt; and timepoint &lt;span class=&quot;math-container&quot;&gt;$t_1$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$t_0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$t_2$&lt;/span&gt;, etc.).&lt;/p&gt;&#xA;&lt;p&gt;But all of these methods make use of &lt;a href=&quot;https://en.wikipedia.org/wiki/Generalized_linear_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;generalized linear models&lt;/a&gt; to perform the statistical inference on coefficients.&#xA;For the simple control-vs-treatment design, above, this looks something like&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$g \left(\mathbb{E} \left[Y_i | X \right] \right) = \mu_i + \beta_i X$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is the link function, &lt;span class=&quot;math-container&quot;&gt;$\mu_i$&lt;/span&gt; is the mean expression of gene &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$\beta_i$&lt;/span&gt; is the expression fold change estimate, and &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is the indicator variable saying if the sample is a treatment or control sample.&lt;/p&gt;&#xA;&lt;p&gt;You can modify this equation by adding a time component to it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$g \left(\mathbb{E} \left[Y_i | X \right] \right) = \mu_i + \beta_i^{(group)} X + \beta_i^{(time)} t$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you specify the timepoint of the sample in the experimental design matrix for DESeq2, edgeR, or Sleuth, you can get information about the effect of the perturbation as well as the effect over time.&lt;/p&gt;&#xA;&lt;p&gt;You can try more complicated designs, too, like a coefficient for the interaction between timepoint and perturbation status, but with 3 replicates in each, it's probably best to stick with simple, linear models.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2021-04-01T20:17:16.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15670" PostTypeId="1" AcceptedAnswerId="15672" CreationDate="2021-04-02T09:22:13.537" Score="2" ViewCount="398" Body="&lt;p&gt;I need to merge multiple channel to reuse them in two processes. So far I tried as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;channel.from([15,20,21]).into {k1;k2}&#xA;channel.from(['a','b','c']).into {m1;m2}&#xA;channel.from([1,2,3]).into {l1;l2}&#xA;para1=m1.merge(k1).merge(l1)&#xA;para2=m2.merge(k2).merge(l2)&#xA;process p1{&#xA;input:&#xA;each m, k, l from para1&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;echo &lt;span class=&quot;math-container&quot;&gt;$m, $&lt;/span&gt;k, &lt;span class=&quot;math-container&quot;&gt;$l&#xA;&quot;&quot;&quot;&#xA;}&#xA;process p2{&#xA;input:&#xA;each m, k, l from para2&#xA;&quot;&quot;&quot;&#xA;echo $m, $k, $&lt;/span&gt;l&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The error I am getting:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Unknown process directive: '_in_each'&#xA;&#xA;Did you mean of these?&#xA;    each&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If I use para1.view() I can see the merged elements as expected. Any help&lt;/p&gt;&#xA;" OwnerUserId="3839" LastActivityDate="2021-04-02T11:16:38.077" Title="Input from multiple channel, merge and use in nextflow" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15671" PostTypeId="2" ParentId="15666" CreationDate="2021-04-02T09:53:11.870" Score="3" Body="&lt;p&gt;Molecular structures are 3D.&#xA;The records (lines) in the PDB files are in so-called fixed column format.&lt;br /&gt;&#xA;The fields don't need to be separated with blank characters, but they must be in specified columns. For example, the X coordinate is in columns 31-38.&#xA;Here is the specification of the ATOM record from &lt;a href=&quot;https://www.wwpdb.org/documentation/file-format-content/format33/sect9.html#ATOM&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB v3.3&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;COLUMNS        DATA  TYPE    FIELD        DEFINITION&#xA;-------------------------------------------------------------------------------------&#xA; 1 -  6        Record name   &amp;quot;ATOM  &amp;quot;&#xA; 7 - 11        Integer       serial       Atom  serial number.&#xA;13 - 16        Atom          name         Atom name.&#xA;17             Character     altLoc       Alternate location indicator.&#xA;18 - 20        Residue name  resName      Residue name.&#xA;22             Character     chainID      Chain identifier.&#xA;23 - 26        Integer       resSeq       Residue sequence number.&#xA;27             AChar         iCode        Code for insertion of residues.&#xA;31 - 38        Real(8.3)     x            Orthogonal coordinates for X in Angstroms.&#xA;39 - 46        Real(8.3)     y            Orthogonal coordinates for Y in Angstroms.&#xA;47 - 54        Real(8.3)     z            Orthogonal coordinates for Z in Angstroms.&#xA;55 - 60        Real(6.2)     occupancy    Occupancy.&#xA;61 - 66        Real(6.2)     tempFactor   Temperature  factor.&#xA;77 - 78        LString(2)    element      Element symbol, right-justified.&#xA;79 - 80        LString(2)    charge       Charge  on the atom.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="266" LastActivityDate="2021-04-02T09:53:11.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15672" PostTypeId="2" ParentId="15670" CreationDate="2021-04-02T11:16:38.077" Score="2" Body="&lt;p&gt;I don't think you can unpack tuples using the &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#input-repeaters&quot; rel=&quot;nofollow noreferrer&quot;&gt;each&lt;/a&gt; qualifier like that. You can, however, unpack them in the script block:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process test {&#xA;    &#xA;    input:&#xA;    each triple from para1&#xA;    &#xA;    script: &#xA;    def (m, k, l) = triple&#xA;    &#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    echo &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${m} $&lt;/span&gt;{k} ${l}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Consider also that the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#merge&quot; rel=&quot;nofollow noreferrer&quot;&gt;merge&lt;/a&gt; operator is deprecated and will be removed from future Nextflow releases. Instead, you may need to &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#join&quot; rel=&quot;nofollow noreferrer&quot;&gt;join&lt;/a&gt; by index, for example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def indexedChannel( items ) {&#xA;    return Channel.from( items.withIndex() ).map { item, idx -&amp;gt; tuple( idx, item ) }&#xA;}&#xA;&#xA;k_ch = indexedChannel( [ 15, 20, 21 ] )&#xA;m_ch = indexedChannel( [ 'a', 'b', 'c' ] )&#xA;l_ch = indexedChannel( [ 1, 2, 3 ] )&#xA;&#xA;m_ch&#xA;    .join( k_ch )&#xA;    .join( l_ch )&#xA;    .map { idx, m, k, l -&amp;gt; tuple( m, k, l ) }&#xA;    .into { para1; para2 }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2021-04-02T11:16:38.077" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15674" PostTypeId="2" ParentId="15662" CreationDate="2021-04-02T11:54:33.340" Score="3" Body="&lt;p&gt;This sounds like a job for the &lt;code&gt;CombinatorialDerivation&lt;/code&gt; class, which is designed for representing libraries and the relationship between derived designs and the originals.&lt;/p&gt;&#xA;&lt;h3&gt;About &lt;code&gt;CombinatorialDerivation&lt;/code&gt;&lt;/h3&gt;&#xA;&lt;p&gt;In SBOL 3, a &lt;code&gt;CombinatorialDerivation&lt;/code&gt; takes a &lt;code&gt;Component&lt;/code&gt; in its &lt;code&gt;template&lt;/code&gt; property to represent the system that you intend to vary:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Some of the &lt;code&gt;Features&lt;/code&gt; of the template &lt;code&gt;Component&lt;/code&gt; are designated as the variables of the library via &lt;code&gt;VariableFeature&lt;/code&gt; child objects of the &lt;code&gt;CombinatorialDerivation&lt;/code&gt;, which point at the variables with their &lt;code&gt;variable&lt;/code&gt; property.&lt;/li&gt;&#xA;&lt;li&gt;The values that can be filled in for each variable are then designated via the &lt;code&gt;variant*&lt;/code&gt; properties of the &lt;code&gt;VariableFeature&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Finally, the distribution of values is determined by the &lt;code&gt;strategy&lt;/code&gt; property of the &lt;code&gt;CombinatorialDerivation&lt;/code&gt; and the &lt;code&gt;cardinality&lt;/code&gt; properties of each variable.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Representing Random Mutagenesis&lt;/h3&gt;&#xA;&lt;p&gt;In the case of random mutagenesis, if I'm understanding your design correctly, you would represent it with a &lt;code&gt;CombinatorialDerivation&lt;/code&gt; as follows:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &lt;code&gt;template&lt;/code&gt; is &lt;code&gt;Component&lt;/code&gt; of type DNA with the original sequence. Each site designated for mutagenesis can be designated with a &lt;code&gt;SequenceFeature&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;For each of the mutagenesis sites, create a &lt;code&gt;VariableFeature&lt;/code&gt; in the &lt;code&gt;CombinatorialDerivation&lt;/code&gt; with the corresponding &lt;code&gt;SequenceFeature&lt;/code&gt; as its &lt;code&gt;variable&lt;/code&gt; and a set of &lt;code&gt;variant&lt;/code&gt; values for each of the mutant options set by your primers. The cardinality is probably &lt;code&gt;one&lt;/code&gt; (if every site has to be hit by &lt;em&gt;something&lt;/em&gt; in your protocol) or &lt;code&gt;zeroOrOne&lt;/code&gt; (if sites might or might not be hit).&lt;/li&gt;&#xA;&lt;li&gt;The &lt;code&gt;strategy&lt;/code&gt; for the &lt;code&gt;CombinatorialDerivation&lt;/code&gt; is &lt;code&gt;sample&lt;/code&gt;, since only a few of the many possibilities will actually be realized.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Representing Samples and Characterized Constructs&lt;/h3&gt;&#xA;&lt;p&gt;Now, when you take it to the lab, you can represent your aliquot with an &lt;code&gt;Implementation&lt;/code&gt; whose &lt;code&gt;wasDerivedFrom&lt;/code&gt; points to the &lt;code&gt;CombinatorialDerivation&lt;/code&gt;, representing the fact that you've got a randomly generated library whose contents are not precisely known.&lt;/p&gt;&#xA;&lt;p&gt;Finally, once you've got your isolated products represented as &lt;code&gt;Component&lt;/code&gt;s, there is a recommended best practice to use &lt;code&gt;wasDerivedFrom&lt;/code&gt; links to connect the derived &lt;code&gt;Component&lt;/code&gt; and its varied &lt;code&gt;Feature&lt;/code&gt;s back to the &lt;code&gt;CombinatorialDerivation&lt;/code&gt; and its template. If you make a &lt;code&gt;Collection&lt;/code&gt; of your results, the &lt;code&gt;Collection&lt;/code&gt; should also link to the &lt;code&gt;CombinatorialDerivation&lt;/code&gt;.&#xA;This should then capture the relationship between all of the products, their associated data, and the original mutagenesis design.&lt;/p&gt;&#xA;" OwnerUserId="4869" LastActivityDate="2021-04-02T11:54:33.340" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15675" PostTypeId="2" ParentId="15646" CreationDate="2021-04-02T17:11:27.007" Score="9" Body="&lt;p&gt;There is a balancing act between allowing all edits, which defeats the purpose of having URIs point to static objects, and allowing no edits, which defeats the purpose of providing some updated information about an object.  SynBioHub resolves this issue by allowing a limited amount of edits after publication.  To answer your questions specifically:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Yes, you can share objects by generating a share link OR adding another owner to an object.  Sharing can be done by the original owner of the object.&lt;/li&gt;&#xA;&lt;li&gt;There are some mutable fields that any owner of the object can change, these include the description, notes, source, and references in the Details section.  You can also add attachments.&lt;/li&gt;&#xA;&lt;li&gt;Yes, when you publish (Make Public) a new object or collection, you have the option of making it public into an existing collection OR a new collection.  A person with curator privileges on the SynBioHub instance you are working on can perform this operation.&lt;/li&gt;&#xA;&lt;li&gt;Answered above.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="11918" LastActivityDate="2021-04-02T17:11:27.007" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15676" PostTypeId="1" CreationDate="2021-04-02T17:13:40.397" Score="2" ViewCount="781" Body="&lt;p&gt;I am trying to create a 3d structure from 3d coordinated from a pdb file.&lt;/p&gt;&#xA;&lt;p&gt;I am trying to draw 3d coordinates from pdb file &amp;quot;ATOM 1 N VAL A 14 22.692 -27.591 -39.402&amp;quot; and using webgl javascript.&lt;/p&gt;&#xA;&lt;p&gt;I've managed to get the 3d coordinates from a pdb file but i don't know the exact logic that is used to create 3d structure from the coordinates found in pdb file.&lt;/p&gt;&#xA;&lt;p&gt;I tried those coordinates and managed to create a cloud of 3d points but it is not anywhere near the structure that is found on pdb site.&lt;/p&gt;&#xA;&lt;p&gt;So wondering how the structure is created on the pdb website?&lt;/p&gt;&#xA;" OwnerUserId="10761" LastEditorUserId="73" LastEditDate="2021-04-28T00:41:00.293" LastActivityDate="2021-04-29T11:32:29.343" Title="How to draw a 3d structure using 3d coordinates from pdb file?" Tags="|pdb|3d-structure|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15677" PostTypeId="2" ParentId="15676" CreationDate="2021-04-03T10:42:08.107" Score="4" Body="&lt;p&gt;&lt;a href=&quot;http://jmol.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;JSmol&lt;/a&gt; sounds like what you want - it'll let you display 3D structures from PDB files onto webpages using JavaScript. If you just want to view the structure rather than display them on webpages, there's many programs that exist already like &lt;a href=&quot;https://spdbv.vital-it.ch/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Swiss-PdbViewer&lt;/a&gt; and &lt;a href=&quot;https://pymol.org/2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pymol&lt;/a&gt;. There's even web-interfaces that let you upload a PDB file and they display it for you like &lt;a href=&quot;http://jena3d.leibniz-fli.de/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jena3D Viewer&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="11856" LastActivityDate="2021-04-03T10:42:08.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15679" PostTypeId="2" ParentId="15676" CreationDate="2021-04-04T13:28:54.860" Score="4" Body="&lt;p&gt;If you want to share a PDB file, you could try &lt;a href=&quot;https://michelanglo.sgc.ox.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Michelanglo&lt;/a&gt;. It allows you to upload a PDB file (among other things) edit a description panel (which can feature special links that control the protein view and representation) and share the link with whomever —without needing any installations on either side.&#xA;Here is the page of shared protein: &lt;a href=&quot;https://michelanglo.sgc.ox.ac.uk/gallery&quot; rel=&quot;nofollow noreferrer&quot;&gt;gallery&lt;/a&gt;&#xA;(disclaimer: I am the admin).&lt;/p&gt;&#xA;&lt;p&gt;If you are interested in implementing a molecular viewer on your site, I'd suggest NGL. The PDB website uses NGL too, but is recently switching to MOL* (&amp;quot;molstar&amp;quot;), which is a new development, so less documented, but fixes some corner cases and is marginally faster and is a collaboration with a PDBe. NGL does not depend on any technology that will be depracated mind you, so it actually is a very solid choice for a JS molecular viewer.&lt;/p&gt;&#xA;&lt;p&gt;The PDB format is a working format, not a deposition format, as is mmCIF. As a result NGL and Mol* have some issues when it comes to PDB files, such as lack of secondary structure records (HELIX/SHEET) results in anaemic sheets, fancier linkages (LINK/SSBOND reconds) are not handled only CONECT records.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2021-04-08T11:55:23.873" LastActivityDate="2021-04-08T11:55:23.873" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15680" PostTypeId="1" AcceptedAnswerId="15681" CreationDate="2021-04-05T10:24:01.593" Score="2" ViewCount="2536" Body="&lt;p&gt;I need to merge multiple channels each emits multiple file. So far I did something as below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process prephasing {&#xA;publishDir params.out, mode:'copy'&#xA;input:&#xA;tuple val(prefix), path(bfiles) from plink_data&#xA;each chrom from chroms&#xA;path hrc_sites_file&#xA;output:&#xA;tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}&quot;),path(&quot;chr$&lt;/span&gt;{chrom}*.{bed,bim,fam}&amp;quot;) into qc1_chr&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;PRE_PASHING_QC2.sh &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${hrc_sites_file}&quot; &quot;$&lt;/span&gt;{prefix}&amp;quot; &amp;quot;${chrom}&amp;quot;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;channel.fromPath('/some/path/genetic_map_chr*_combined_b37.txt').into {shape_map;i&#xA;mp_map}&#xA;channel.fromPath('/some/path/*hrc.r1-1.ega.grch*.hap.gz').into {shape_ref;&#xA;imp_ref}&#xA;channel.fromPath('/some/path/*hrc.r1-1.ega.grch37.chr*.legend.gz').into {s&#xA;hape_legend;imp_legend}&#xA;channel.fromPath('/some/path/*hrc.r1-1.ega.grch37.chr*.samples').into {sha&#xA;pe_sample;imp_sample}&#xA;qc1_chr.filter{name, bfiles -&amp;gt; bfiles.collect {it.getBaseName()}.every {it.endsWith(&amp;quot;step10&amp;quot;)}}.into {shape_in;imp_in}&#xA;&#xA;shape_para=shape_in.merge(shape_map).merge(shape_ref).merge(shape_legend).merge(shape_sample)&#xA;    &#xA;process phasing {&#xA;publishDir params.out, mode:'copy'&#xA;input:&#xA;each file(penta) from shape_para&#xA;output:&#xA;path(&amp;quot;*step10*&amp;quot;) into phased&#xA;script:&#xA;def (in, map, ref, legend, sample) = penta&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;shapeit2 -T 32 --input-bed &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${in}&quot; --input-map &quot;$&lt;/span&gt;{map}&amp;quot; --input-ref &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${ref}&quot; &quot;$&lt;/span&gt;{legend}&amp;quot; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample}&quot; --duohmm -O &quot;$&lt;/span&gt;{in}.phased&amp;quot; --seed 54321&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &amp;quot;prephasing&amp;quot; process works fine and produce expected output. But the &amp;quot;phasing&amp;quot; process does not produce any output nor any error. Any help?&lt;/p&gt;&#xA;&lt;p&gt;I have tried according to &lt;code&gt;@Steve&lt;/code&gt; as below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;channel.fromFilePairs('/cluster/projects/p697/projects/moba_qc_imputation/resources/1000Genomes/genetic_map_chr*_combined_b37.txt',size:1).map {group_key, file_list -&amp;gt; tuple(group_key.replaceFirst(/^genetic_map_/,&amp;quot;&amp;quot;),file_list.first())}.set {genetic_map}&#xA;&#xA;channel.fromFilePairs('/cluster/projects/p697/projects/moba_qc_imputation/resources/HRC/haplegendsample/*hrc.r1-1.ega.grch37.chr*.haplotypes.{hap.gz,legend.gz,samples}',size:3).map{group_key,file_list -&amp;gt; tuple(group_key.replaceFirst(/^*HRC\.r1-1\.ega\.grch37\./,&amp;quot;&amp;quot;),file_list)}.set {ref_panel}&#xA;&#xA;qc1_chr.filter{chrom, bfiles -&amp;gt; bfiles.collect {it.getBaseName()}.every {it.endsWith(&amp;quot;step10&amp;quot;)}}.join(ref_panel).join(genetic_map).into {shape_in;imp_in}&#xA;&#xA;process phasing {&#xA;publishDir params.out, mode:'copy'&#xA;input:&#xA;tuple val(chrom), path(input_bed), path(intput_ref), path(input_map) from shape_in&#xA;output:&#xA;tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}&quot;),path(&quot;$&lt;/span&gt;{chrom}.{phased.haps,phased.sample,log}&amp;quot;) into phased&#xA;script:&#xA;def (bed, bim, fam) = input_bed&#xA;def (haplotype, legend, sample) = input_ref&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;shapeit2 --thread &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${task.cpus}&quot; --input-bed &quot;$&lt;/span&gt;{bed}&amp;quot; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${bim}&quot; &quot;$&lt;/span&gt;{fam}&amp;quot; --input-map &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${input_map}&quot; --input-ref &quot;$&lt;/span&gt;{haplotype}&amp;quot; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${legend}&quot; &quot;$&lt;/span&gt;{sample}&amp;quot; --duohmm --output-max &amp;quot;${chrom}.phased&amp;quot; --seed 54321&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But getting no output so far, (attached image&lt;a href=&quot;https://i.stack.imgur.com/B1PtL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/B1PtL.png&quot; alt=&quot;phasing process&quot; /&gt;&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;I have tried to check individual channel separately using channel.view(), identified the problems, modified the code and checked again. Using the code below I can see individual channel emits 22 lines [chr1-22,[file_list]]. After merging I can see [chr1-22,[joined_file_list].&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;channel&#xA;    .fromFilePairs( &amp;quot;output3/*.{bed,bim,fam}&amp;quot;, size:3 )&#xA;    .set { plink_data }&#xA;plink_data.filter{chrom, bfiles -&amp;gt; bfiles.collect {it.getBaseName()}.every {it.endsWith(&amp;quot;step10&amp;quot;)}}.map{group_key,file_list -&amp;gt; tuple (group_key.&#xA;replaceFirst(/^*\.step10/,&amp;quot;&amp;quot;), file_list)}.set {in_pl}&#xA;&#xA;channel.fromFilePairs('/cluster/projects/p697/projects/moba_qc_imputation/resources/1000Genomes/genetic_map_chr*_combined_b37.txt',size:1).map {&#xA;group_key, file_list -&amp;gt; tuple(group_key.replaceFirst(/^*genetic_map_/,&amp;quot;&amp;quot;),file_list.first())}.set {genetic_map}&#xA;//genetic_map.view()&#xA;&#xA;channel.fromFilePairs('/cluster/projects/p697/projects/moba_qc_imputation/resources/HRC/haplegendsample/*hrc.r1-1.ega.grch37.chr*.haplotypes.{ha&#xA;p.gz,legend.gz,samples}',size:3).map {group_key,file_list -&amp;gt; tuple(group_key.replaceFirst(/^.*.hrc\.r1-1\.ega\.grch37\./,&amp;quot;&amp;quot;),file_list)}.set {re&#xA;f_panel}&#xA;//ref_panel.view()&#xA;&#xA;&#xA;&#xA;in_pl.join(ref_panel).join(genetic_map).into {shape_in;imp_in}&#xA;process phasing {&#xA;//publishDir params.out, mode:'copy'&#xA;input:&#xA;tuple val(chrom), path(input_bed), path(intput_ref), path(input_map) from shape_in&#xA;output:&#xA;tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}&quot;),path(&quot;$&lt;/span&gt;{chrom}.{phased.haps,phased.sample,log}&amp;quot;) in&#xA;to phased&#xA;script:&#xA;def (bed, bim, fam) = input_bed&#xA;def (haplotype, legend, sample) = input_ref&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;//shapeit=&amp;quot;/cluster/projects/p697/projects/moba_qc_imputation/software/shap&#xA;eit&amp;quot;&#xA;shapeit --thread &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${task.cpus}&quot; --input-bed &quot;$&lt;/span&gt;{bed}&amp;quot; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${bim}&quot; &quot;$&lt;/span&gt;{fam}&amp;quot; --input-map &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${input_map}&quot; --input-ref &quot;$&lt;/span&gt;{haplotype}&amp;quot; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${legend}&quot; &quot;$&lt;/span&gt;{samp&#xA;le}&amp;quot; --duohmm --output-max &amp;quot;${chrom}.phased&amp;quot; --seed 54321&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CR2sW.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CR2sW.jpg&quot; alt=&quot;joined channel example view&quot; /&gt;&lt;/a&gt;&#xA;But now I get error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;No such variable: chrom&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I decalred &amp;quot;chrom&amp;quot; in the input&lt;/p&gt;&#xA;&lt;p&gt;The input bed|bim|fam files from channel from FilePairs works but from a output channel of previous process does not. If I run &lt;code&gt;phasing&lt;/code&gt; solo and all the inputs from channel from FilePairs it works. But if I run &lt;code&gt;prephasing&lt;/code&gt; and &lt;code&gt;phasing&lt;/code&gt; together, the &lt;code&gt;prephasing&lt;/code&gt; process works but for &lt;code&gt;phasing&lt;/code&gt; process not output and no error; Not sure what I am doing wrong.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;hrc_sites_file = file(params.hrc_sites_file)&#xA;&#xA;Channel.fromFilePairs( &amp;quot;${params.input_dir}/*.{bed,bim,fam}&amp;quot;, size:3 ).set { plink_data }&#xA;&#xA;Channel.of(1..22).set { chroms }&#xA;params.out='output/'&#xA;&#xA;process prephasing {&#xA;publishDir params.out, mode:'copy'&#xA;input:&#xA;tuple val(prefix), path(bfiles) from plink_data&#xA;each chrom from chroms&#xA;path hrc_sites_file&#xA;output:&#xA;tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}&quot;),path(&quot;chr$&lt;/span&gt;{chrom}*.{bed,bim,fam}&amp;quot;) into qc1_chr&#xA;tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}&quot;),path(&quot;chr$&lt;/span&gt;{chrom}*log&amp;quot;) into qc1_log&#xA;tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}&quot;),path(&quot;*-chr$&lt;/span&gt;{chrom}.step4-HRC.txt&amp;quot;) into qc1_hrc&#xA;path(&amp;quot;chr&lt;span class=&quot;math-container&quot;&gt;${chrom}.step1b.snplist.txt&quot;) into qc1_snps&#xA;path(&quot;chr$&lt;/span&gt;{chrom}.step1a.dupvar&amp;quot;) into qc1_dups&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;PRE_PASHING_QC2.sh &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${hrc_sites_file}&quot; &quot;$&lt;/span&gt;{prefix}&amp;quot; &amp;quot;${chrom}&amp;quot;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;channel.fromFilePairs('/cluster/projects/p697/projects/moba_qc_imputation/resources/1000Genomes/genetic_map_chr*_combined_b37.txt',size:1).map {group_key, file_list -&amp;gt; tuple(group_key.replaceFirst(/^*gen&#xA;etic_map_/,&amp;quot;&amp;quot;),file_list.first())}.set {genetic_map}&#xA;&#xA;channel.fromFilePairs('/cluster/projects/p697/projects/moba_qc_imputation/resources/HRC/haplegendsample/*hrc.r1-1.ega.grch37.chr*.haplotypes.{hap.gz,legend.gz,samples}',size:3).map{group_key,file_list -&amp;gt;&#xA; tuple(group_key.replaceFirst(/^.*.hrc\.r1-1\.ega\.grch37\./,&amp;quot;&amp;quot;),file_list)}.set {ref_panel}&#xA;&#xA;qc1_chr.filter{chrom, bfiles -&amp;gt; bfiles.collect {it.getBaseName()}.every {it.endsWith(&amp;quot;step10&amp;quot;)}}.map{group_key,file_list -&amp;gt; tuple (group_key.replaceFirst(/^*\.step10/,&amp;quot;&amp;quot;), file_list)}.join(ref_panel).joi&#xA;n(genetic_map).into {shape_in;imp_in}&#xA;imp_in.view()&#xA;process phasing {&#xA;publishDir params.out, mode:'copy'&#xA;input:&#xA;tuple val(chrom), path(input_bed), path(input_ref), path(input_map) from shape_in&#xA;output:&#xA;tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}&quot;),path(&quot;$&lt;/span&gt;{chrom}.*&amp;quot;) into phased&#xA;script:&#xA;def (bed, bim, fam) = input_bed&#xA;def (haplotype, legend, sample) = input_ref&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;shapeit --thread &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${task.cpus}&quot; --input-bed &quot;$&lt;/span&gt;{bed}&amp;quot; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${bim}&quot; &quot;$&lt;/span&gt;{fam}&amp;quot; --input-map &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${input_map}&quot; --input-ref &quot;$&lt;/span&gt;{haplotype}&amp;quot; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${legend}&quot; &quot;$&lt;/span&gt;{sample}&amp;quot; --duohmm --output-max &amp;quot;${chrom}.step10&amp;quot; --seed 54321&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bjxQJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bjxQJ.png&quot; alt=&quot;prephasing works but phasing does not&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3839" LastEditorUserId="3839" LastEditDate="2021-04-06T18:52:49.430" LastActivityDate="2021-04-07T23:18:54.153" Title="Merge multiple channels each emits multiple files in nextflow" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15681" PostTypeId="2" ParentId="15680" CreationDate="2021-04-05T14:40:11.413" Score="3" Body="&lt;p&gt;The &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#merge&quot; rel=&quot;nofollow noreferrer&quot;&gt;merge operator&lt;/a&gt; is deprecated and will be removed from future Nextflow releases. You'll need instead the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#join&quot; rel=&quot;nofollow noreferrer&quot;&gt;join operator&lt;/a&gt; to join by chromosome. You can use the &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#fromfilepairs&quot; rel=&quot;nofollow noreferrer&quot;&gt;fromFilePairs operator&lt;/a&gt; to quickly get the group key, formatting it as required:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;channel&#xA;    .fromFilePairs(&amp;quot;/some/path/HRC.r1-1.EGA.GRCh37.chr*.{hap.gz,legend.gz,samples}&amp;quot;, size: 3)&#xA;    .map { group_key, file_list -&amp;gt;&#xA;        tuple( group_key.replaceFirst(/^HRC\.r1-1\.EGA\.GRCh37\./, &amp;quot;&amp;quot;), file_list )&#xA;    }&#xA;    .set { ref_panel_files }&#xA;&#xA;&#xA;channel&#xA;    .fromFilePairs(&amp;quot;/some/path/genetic_map_chr*_combined_b37.txt&amp;quot;, size: 1)&#xA;    .map { group_key, file_list -&amp;gt;&#xA;        tuple( group_key.replaceFirst(/^genetic_map_/, &amp;quot;&amp;quot;), file_list.first() )&#xA;    }&#xA;    .set { genetic_map_files }&#xA;&#xA;&#xA;qc1_chr&#xA;    .map { chrom, bfiles -&amp;gt;&#xA;        tuple( chrom, bfiles.findAll { it.baseName.endsWith(&amp;quot;step10&amp;quot;) } )&#xA;    }&#xA;    .join( ref_panel_files )&#xA;    .join( genetic_map_files )&#xA;    .set { phasing_inputs }&#xA;&#xA;&#xA;process phasing {&#xA;&#xA;    input:&#xA;    tuple val(chrom), path(input_bed), path(input_ref), path(input_map) from phasing_inputs&#xA;&#xA;    script:&#xA;    def (bed, bim, fam) = input_bed&#xA;    def (haplotypes, legend, sample) = input_ref&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    shapeit2 \\&#xA;        --thread &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${task.cpus}&quot; \\&#xA;        --input-bed &quot;$&lt;/span&gt;{bed}&amp;quot; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${bim}&quot; &quot;$&lt;/span&gt;{fam}&amp;quot; \\&#xA;        --input-ref &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${haplotypes}&quot; &quot;$&lt;/span&gt;{legend}&amp;quot; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample}&quot; \\&#xA;        --input-map &quot;$&lt;/span&gt;{input_map}&amp;quot; \\&#xA;        --duohmm \\&#xA;        --output-max &amp;quot;${chrom}.phased&amp;quot; \\&#xA;        --seed 54321&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2021-04-07T23:18:54.153" LastActivityDate="2021-04-07T23:18:54.153" CommentCount="19" ContentLicense="CC BY-SA 4.0" />
  <row Id="15682" PostTypeId="2" ParentId="15647" CreationDate="2021-04-05T15:26:36.417" Score="4" Body="&lt;p&gt;One way using &lt;a href=&quot;/questions/tagged/awk&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; rel=&quot;tag&quot;&gt;awk&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '/^&amp;gt;/ { f = !a[$0]++ } f' seqs.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Explanation:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The above is basically the same as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '/^&amp;gt;/ { f = !($0 in a); a[$0]++ } f' seqs.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That is, for each header line (lines that start with a '&amp;gt;' character), set a flag, 'f' to true if the current line is not in an array, 'a'. Then add the (header) line to the array. Note that in this form, it's not necessary to increment the value for each element - the following would be sufficient:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '/^&amp;gt;/ { f = !($0 in a); a[$0] } f' seqs.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The default action is to print, and we print lines only when 'f' is true. This is the same as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '/^&amp;gt;/ { f = !($0 in a); a[$0] } f==1 { print $0 }' seqs.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2021-04-06T00:04:16.323" LastActivityDate="2021-04-06T00:04:16.323" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15683" PostTypeId="1" AcceptedAnswerId="15689" CreationDate="2021-04-06T05:52:32.870" Score="1" ViewCount="795" Body="&lt;p&gt;I'm a newbie to bioinfo. I'm trying to use galaxy.networkanalyst.ca for the analysis of RNA-Seq data and for that I need to convert &lt;em&gt;&lt;strong&gt;txt.gz files into fastq.gz files&lt;/strong&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/r4FeB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/r4FeB.png&quot; alt=&quot;PIc1&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Link: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM4273445&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSM4273445&lt;/a&gt; is one sample and the whole series is from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE143735&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I've Googled to find out if there are online tools to convert it, but there aren't any. Besides, there is no discussion addressing this format conversion on researchgate.&lt;/p&gt;&#xA;&lt;p&gt;I intend to use this RNA-Seq data and investigate the relative expression of some specific transcripts of interest. I have been introduced to 'galaxy.networkanalyst.ca' where the very first step is to upload fastq files. In addition to this I was aware of ncbi GEOdatasets as source of open RnA-Seq data.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone suggest a method or tool for this?&lt;/p&gt;&#xA;" OwnerUserId="555" LastEditorUserId="73" LastEditDate="2021-05-09T05:05:22.217" LastActivityDate="2021-05-09T05:05:22.217" Title="How to convert txt.gz into fastq.gz?" Tags="|fastq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15685" PostTypeId="1" AcceptedAnswerId="15716" CreationDate="2021-04-06T10:20:30.857" Score="1" ViewCount="166" Body="&lt;p&gt;I mapped raw reads obtained from an E. coli genome (.fastq file) to my reference genome. Next, I exported the .vcf (variant calling file), which looks something like the header example shown below (total &amp;gt;50 lines):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  Variants:_LGC19-XL01_S63_L001_R_001_(trimmed)&#xA;CFC381_K12_Bw25113  360287  .   GGGAAT  AATTCC  1952.76 .   NS=1;VF=0.380;SB=0.533;SB50=0.60;SB65=1.0;TYPE=Substitution;AVQUAL=30   DP:AO   231:89&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;How can I back-fill or quickly assign the missing GTs to the &lt;code&gt;.vcf&lt;/code&gt; ?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;For my downstream process (&lt;code&gt;plink --pca&lt;/code&gt;, via &lt;code&gt;bfctools merge&lt;/code&gt;), I need that there are GT entries in every &lt;code&gt;.vcf&lt;/code&gt;, so I can merge them and the respective GT is shown in the merged &lt;code&gt;.vcf&lt;/code&gt;. I only found solutions that need the &lt;code&gt;.bam&lt;/code&gt; file, but that is laborious and I don't think I need the &lt;code&gt;.bam&lt;/code&gt; in the case of haploid data.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;This method needs &lt;code&gt;.bam&lt;/code&gt;: &lt;a href=&quot;https://www.biostars.org/p/319304/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/319304/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;This method needs &lt;code&gt;.fastq&lt;/code&gt;: &lt;a href=&quot;https://training.galaxyproject.org/training-material/topics/variant-analysis/tutorials/non-dip/tutorial.html#examining-the-results&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://training.galaxyproject.org/training-material/topics/variant-analysis/tutorials/non-dip/tutorial.html#examining-the-results&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, the&lt;code&gt;.fastq&lt;/code&gt; is actually from a mixed culture, so I wonder do I really need to follow the &lt;code&gt;.bam&lt;/code&gt;-strategy? Suggestion for a tool? But in that case, why did it not export the GTs in the fist place (I exported from &amp;quot;geneious&amp;quot; sequence analysis program)&lt;/p&gt;&#xA;" OwnerUserId="11900" LastEditorUserId="11900" LastEditDate="2021-04-07T02:37:55.773" LastActivityDate="2021-04-28T14:40:22.693" Title="Why is there no GT-entry in my .vcf file from bacteria" Tags="|variant-calling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15687" PostTypeId="1" CreationDate="2021-04-06T13:03:22.970" Score="0" ViewCount="135" Body="&lt;p&gt;I've installed GEOquery with&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;install.packages(&amp;quot;BiocManager&amp;quot;)&#xA;BiocManager::install(&amp;quot;GEOquery&amp;quot;)&#xA;library(GEOquery)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;then I ran the command&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;GSE68808 &amp;lt;- getGEO(&amp;quot;GSE68808&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but it resulted in..&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;Found 1 file(s)&#xA;GSE68808_series_matrix.txt.gz&#xA;trying URL 'https://ftp.ncbi.nlm.nih.gov/geo/series/GSE68nnn/GSE68808/matrix/GSE68808_series_matrix.txt.gz'&#xA;Content type 'application/x-gzip' length 4138819 bytes (3.9 MB)&#xA;downloaded 3.9 MB&#xA;&#xA;error: 'C:\Users\(myname)\AppData\Local\Temp\RtmpUp5Sv7\file8e844a7a59bc' does not exist.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;ㅜㅜ&lt;/p&gt;&#xA;&lt;p&gt;How can I solve this? thanks.&lt;/p&gt;&#xA;" OwnerUserId="11947" LastEditorUserId="650" LastEditDate="2023-09-15T21:42:54.040" LastActivityDate="2024-02-12T22:06:40.750" Title="Error downloading GSE dataset" Tags="|geoquery|gse|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15688" PostTypeId="2" ParentId="15494" CreationDate="2021-04-06T14:09:06.083" Score="0" Body="&lt;p&gt;FIRST: I wasn't able to find a way to pass a SeqRecord object from a list to a file without saving it to a temp file first. That is without:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i , batch in enumerate(batch_iterator(data_input_iter , 1)):&#xA;        record_iter = batch &#xA;......&#xA;&#xA;SeqIO.write(record_iter, &amp;quot;temp_file&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;            &#xA;record_iter = SeqIO.read(&amp;quot;temp_file&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;the line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; if record_iter.seq != record_output.seq  :&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;will throw:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;AttributeError: 'list' object has no attribute 'seq'&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After spending some timde on it I realized that:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print('\n@@@@@@@@@@@@@@\n',type(record_iter),'\n',record_iter)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;is returning:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;class 'list'&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but jus using :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i , batch in enumerate(batch_iterator(data_input_iter , 1)):&#xA;        record_iter = batch[0] &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;now :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print('\n@@@@@@@@@@@@@@\n',type(record_iter),'\n',record_iter)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;is returning:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;class 'Bio.SeqRecord.SeqRecord'&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;no need to save and read the Bio.SeqRecord.SeqRecord' anymore&lt;/p&gt;&#xA;&lt;p&gt;so this is just half the question answered so no need to save and read the record on file&lt;/p&gt;&#xA;&lt;p&gt;but even if speed increases the memory usage goes up :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data_input_iter :  48 &#xA;&#xA;Run time is 1.442115306854248&#xA;Maximum memory usage: 110.09765625&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2021-11-02T22:19:03.440" LastActivityDate="2021-11-02T22:19:03.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15689" PostTypeId="2" ParentId="15683" CreationDate="2021-04-06T15:07:01.170" Score="3" Body="&lt;p&gt;I've had a quick look at the file you linked to and this is a featureCounts file - this list the number of reads for each gene/transcript (COUTNS.txt in the file name pretty much gave it away that it was going to be a file of, well counts).&lt;/p&gt;&#xA;&lt;p&gt;I can't help you with Galaxy, but if you want to do an RNA-seq analysis, the count data + metadata is likely all you'll need.&lt;/p&gt;&#xA;&lt;p&gt;If you really want to do the preprocessing and mapping yourself, the fastq files can be downloaded from the SRA archive (SRP242087) - click on the SRA link in your whole series link.&lt;/p&gt;&#xA;" OwnerUserId="8919" LastActivityDate="2021-04-06T15:07:01.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15691" PostTypeId="2" ParentId="15687" CreationDate="2021-04-06T16:33:19.293" Score="0" Body="&lt;p&gt;There seems to be some problem with your &amp;quot;platform specific tempdir&amp;quot; that is used by getGEO's &lt;code&gt;destdir&lt;/code&gt; parameter. See &lt;code&gt;?getGEO&lt;/code&gt; for more info on the &lt;code&gt;destdir&lt;/code&gt; parameter.&lt;/p&gt;&#xA;&lt;p&gt;Try&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GSE68808 &amp;lt;- getGEO(&amp;quot;GSE68808&amp;quot;, destdir = getwd())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastActivityDate="2021-04-06T16:33:19.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15692" PostTypeId="1" AcceptedAnswerId="15745" CreationDate="2021-04-06T17:58:38.713" Score="5" ViewCount="42" Body="&lt;p&gt;Like the propagation delay and threshold value parameters in electronic circuits, does SBOL format support any such parameters/glyph to represent the &lt;strong&gt;propagation delay&lt;/strong&gt; (time to trigger the output protein once the input is provided) and &lt;strong&gt;threshold value&lt;/strong&gt; (minimum input concentration required to trigger the model's output) of genetic models?&lt;/p&gt;&#xA;&lt;p&gt;If currently not supported, can these parameters be incorporated in the SBOL visual representation which would give reader an idea (just by looking at the schematic diagram) about the timing behavior of a model?&lt;/p&gt;&#xA;" OwnerUserId="10851" LastActivityDate="2021-04-16T13:54:57.303" Title="Does SBOL support timing and threshold value parameters?" Tags="|sbol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15693" PostTypeId="2" ParentId="15558" CreationDate="2021-04-06T18:29:57.187" Score="3" Body="&lt;p&gt;Further to jakebeal's answer there is also an &lt;a href=&quot;https://github.com/SynBioDex/Excel-to-SBOL&quot; rel=&quot;nofollow noreferrer&quot;&gt;excel2sbol library&lt;/a&gt; (still in alpha) that can take input to a standard template and convert to SBOL2.&lt;/p&gt;&#xA;&lt;p&gt;For SBOL3 there is also a &lt;a href=&quot;https://github.com/SynBioDex/pySBOL3&quot; rel=&quot;nofollow noreferrer&quot;&gt;python library&lt;/a&gt;, but using this to create SBOL3 is more involved.&lt;/p&gt;&#xA;" OwnerUserId="11955" LastActivityDate="2021-04-06T18:29:57.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15694" PostTypeId="2" ParentId="15628" CreationDate="2021-04-06T18:43:44.617" Score="3" Body="&lt;p&gt;For pySBOL there is a tutorial you can follow in the &lt;a href=&quot;https://pysbol2.readthedocs.io/en/latest/getting_started.html#&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;. Additionally, there is a &lt;a href=&quot;https://www.youtube.com/watch?v=GtDttQX4_z4&quot; rel=&quot;nofollow noreferrer&quot;&gt;youtube video&lt;/a&gt; walking through a similar tutorial, note the tutorial starts at about 1:09:00.&lt;/p&gt;&#xA;&lt;p&gt;There are lots of additional tools and libraries out there such as &lt;a href=&quot;https://sbolcanvas.org/canvas/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SBOL Canvas&lt;/a&gt; (for visualisation), &lt;a href=&quot;https://goksel.github.io/libSBOLj3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;libsbolj&lt;/a&gt; (a java library for SBOL), and other &lt;a href=&quot;https://sbolstandard.org/libraries/&quot; rel=&quot;nofollow noreferrer&quot;&gt;libraries&lt;/a&gt; and &lt;a href=&quot;https://sbolstandard.org/applications/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tools&lt;/a&gt; listed on the SBOL website.&lt;/p&gt;&#xA;" OwnerUserId="11955" LastActivityDate="2021-04-06T18:43:44.617" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15695" PostTypeId="1" AcceptedAnswerId="15696" CreationDate="2021-04-07T09:17:20.107" Score="1" ViewCount="1863" Body="&lt;p&gt;Simple question but I am stumped. (I am still learning Nextflow)&lt;/p&gt;&#xA;&lt;p&gt;I have a process which takes as input a channel of files and processes each file separately.&lt;/p&gt;&#xA;&lt;p&gt;There is a second process which needs to wait until all files are complete, so it can merge the outputs into a single collated file.&lt;/p&gt;&#xA;&lt;p&gt;How can I do this? I have tried subscribing to &amp;quot;onComplete&amp;quot; event to no avail as I cannot specify a process as the function to trigger.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;files = Channel.from params.the_files_list&#xA;&#xA;process process_files {&#xA;&#xA;    input:&#xA;    val file_path from files&#xA;&#xA;    output:&#xA;    stdout result&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    python ./process_one_file.py $file_path&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process merge_results {&#xA;    &#xA;    input:&#xA;    ????&#xA;&#xA;    output:&#xA;    ????&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    echo &amp;quot;How to trigger this process once, after all individual files are processed?&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;        &#xA;}&#xA;&#xA;??? Want to do something like this&#xA;result.subscribe onComplete: merge_results&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11959" LastActivityDate="2022-05-30T21:59:06.200" Title="How to trigger next process on completion of previous process?" Tags="|nextflow|workflow-management|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15696" PostTypeId="2" ParentId="15695" CreationDate="2021-04-07T09:28:36.203" Score="2" Body="&lt;p&gt;So from this page...&#xA;&lt;a href=&quot;https://groups.google.com/g/nextflow/c/jt77_-uApMs&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://groups.google.com/g/nextflow/c/jt77_-uApMs&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is the &amp;quot;scatter-gather&amp;quot; pattern. I think the answer is to collect() the channel as follows - interested to hear if there is a better way&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;files = Channel.from params.the_files_list&#xA;&#xA;process process_files {&#xA;&#xA;    input:&#xA;    val file_path from files&#xA;&#xA;    output:&#xA;    stdout result&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    python ./process_one_file.py $file_path&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process merge_results {&#xA;    &#xA;    input:&#xA;    val go from result.collect()&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    echo &amp;quot;Triggered once after all files complete!&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;        &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11959" LastActivityDate="2021-04-07T09:28:36.203" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15697" PostTypeId="2" ParentId="15408" CreationDate="2021-04-07T17:51:45.127" Score="1" Body="&lt;p&gt;I would suggest reading a little on the topic; my understanding is that distance measures can be pretty robust to proportion/compositional data. &lt;a href=&quot;https://www.umass.edu/landeco/teaching/multivariate/readings/McCune.and.Grace.2002.chapter6.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is a relevant textbook chapter&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Other distance measures assume that the data are proportions ranging from zero to one, inclusive.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Among these are indeed Bray-Curtis as well as Jaccard distance (and others). I think that you will want to use one of the more traditional distances rather than a Pearson/Spearman, though they are getting at the same problem. A transformation might help, I suppose, but I would prefer to choose a simple but predictable distance measure rather than a correction on a possibly unsuitable distance that might have unpredictable behavior.&lt;/p&gt;&#xA;&lt;p&gt;It's a little hard to tell from your question, but it seems that you are wanting to in some way account for indirect correlations using a distance metric. I would not suggest trying to make your distance measure do all this work. Instead, I would use a simple but clear distance method, and then &lt;a href=&quot;https://pjbartlein.github.io/GeogDataAnalysis/lec18.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;use a multidimensional analysis such as principal coordinate analysis (or similar) to understand the correlation structure of your compositional variables&lt;/a&gt;. This is a fairly standard workflow in metagenomics, for instance, which &lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fmicb.2017.02224/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;relies heavily on high-dimensional compositional data&lt;/a&gt;. I would recommend looking more into metagenomics statistical applications for inspiration if needed.&lt;/p&gt;&#xA;&lt;p&gt;The choice does also depend to a certain extent on the specific application. In some fields certain methods are familiar and trusted even if there is not a strong empirical/theoretical rule.&lt;/p&gt;&#xA;&lt;p&gt;You've asked a lot of questions about Pearson/Spearman correlations. Much has been written on the use of correlation analysis on compositional data, most of it &lt;a href=&quot;https://link.springer.com/article/10.1007/s11004-008-9196-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;cautioning against doing this kind of analysis&lt;/a&gt; without special treatment. This leads to &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-019-2915-1#:%7E:text=SparCC%20is%20an%20iterative%20approximation,are%20large%2Dscale%20and%20sparse.&quot; rel=&quot;nofollow noreferrer&quot;&gt;difficulties in building meaningful association models&lt;/a&gt; for different variables. One method that I'm familiar with in metagenomics is sparse correlation, e.g. &lt;a href=&quot;https://web.mit.edu/almlab/sparcc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SPARCC&lt;/a&gt;. But there are newer methods that may be better, for example &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-019-2915-1#:%7E:text=SparCC%20is%20an%20iterative%20approximation,are%20large%2Dscale%20and%20sparse.&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/35/6/1064/5086389&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-04-07T17:51:45.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15698" PostTypeId="1" CreationDate="2021-04-08T07:57:36.803" Score="0" ViewCount="604" Body="&lt;p&gt;[![example csv file][1]][1]&lt;/p&gt;&#xA;&lt;p&gt;First I want to start by parsing and doing some operation on a text/csv file. How could I read a csv/text file in nextflow?&lt;/p&gt;&#xA;&lt;p&gt;Suppose, I have above csv file with 3 columns. Lets say I want to create a channel which emits an array from minimum of column 2 (7.9e5) to maximum of column 2 (8.8e5) with an increment of 1e4. How could I create the following channel and later access it in the script? Any help? The&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;array=([1]=&amp;quot;7.9e5 8.0e5&amp;quot; [2]=&amp;quot;8.0e5 8.1e5&amp;quot; [3]=&amp;quot;8.1e5 8.2e5&amp;quot; [4]=&amp;quot;8.2e5 8.3e5&amp;quot; [5]=&amp;quot;8.3e5 8.4e5&amp;quot; [6]=&amp;quot;8.4e5 8.5e5&amp;quot; [7]=&amp;quot;8.5e5 8.6e5&amp;quot; [8]=&amp;quot;8.6e5 8.7e5&amp;quot; [9]=&amp;quot;8.7e5 8.8e5&amp;quot; [10]=&amp;quot;8.8e5 8.9e5&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I need to do some operations with this array in the script which will generate multiple files for each element in the array. These multiple files need to be merged in a single file.&#xA;Sorry, I could not provide any nextflow script that I tried, because I not sure how create an array with &lt;code&gt;splitCsv&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have tried in python:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df1=pd.read_csv('data1/omni-ec-eur-fin-pass-qc.bim',sep='\t',header=None)&#xA;arr1=np.arange(df1.iloc[:,3].min()-1,df1.iloc[:,3].max()+1,3000000)&#xA;diffs = []&#xA;for x in zip(arr1[0::],arr1[1::]):&#xA;    diffs.append(x)&#xA;arr2=np.arange(1,len(diffs)+1,1)&#xA;dict(zip(arr2,diffs))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which gave me:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{1: (12343, 3012343),&#xA; 2: (3012343, 6012343),&#xA; 3: (6012343, 9012343),&#xA; 4: (9012343, 12012343),...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I need something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;([1]=&amp;quot;12343 3012343&amp;quot; [2]=&amp;quot;3012343 6012343&amp;quot; [3]=&amp;quot;6012343   9012343&amp;quot; [4]=&amp;quot;9012343 12012343&amp;quot;....)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It could be arrays (22 arrays for 22 chromosome) or a csv file containing arrays for each chromosome which could be used as an input channel. There are 22 bim files (one for each chromosome) stored in the output channel which would be used to create the above mentioned array.&#xA;The original data inside the bim looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    0   1   2   3   4   5&#xA;0   1   rs11240777  0   798959  A   G&#xA;1   1   rs4475691   0   846808  A   G&#xA;2   1   rs7537756   0   854250  G   A&#xA;3   1   rs13302982  0   861808  A   G&#xA;4   1   rs1110052   0   873558  C   A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The 4th column would be used to create the arrays.&#xA;[1]: &lt;a href=&quot;https://i.stack.imgur.com/cxqyl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://i.stack.imgur.com/cxqyl.png&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3839" LastEditorUserId="3839" LastEditDate="2021-04-12T03:40:35.987" LastActivityDate="2021-04-12T03:40:35.987" Title="parse csv/text file to create a channel with array in nextflow" Tags="|nextflow|" AnswerCount="0" CommentCount="18" ContentLicense="CC BY-SA 4.0" />
  <row Id="15701" PostTypeId="1" AcceptedAnswerId="15708" CreationDate="2021-04-08T12:43:38.210" Score="1" ViewCount="121" Body="&lt;p&gt;I trying to locally reproduce a &lt;a href=&quot;https://github.com/Illumina/manta&quot; rel=&quot;nofollow noreferrer&quot;&gt;Manta&lt;/a&gt; run as seen in the &lt;em&gt;log&lt;/em&gt; of an illumina basespace &lt;a href=&quot;https://emea.illumina.com/products/by-type/informatics-products/basespace-sequence-hub/apps/rna-seq-alignment.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNA-Seq Alignment tool v1.1.1&lt;/a&gt; run I've done. The log reads as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[LOG][2021-03-12 20:58:56.377]03/12/2021,20:58:56.193,Launch process:  \&#xA;/path_to_RNA_Tools/bin/python  \&#xA;'/path_to_RNA_Tools/Manta/bin/configManta.py'  \&#xA;--rna --normalBam='path_to_alignments/alignments.bam' \&#xA;--runDir='path_to_samples/CTR/replicates/CTR/MantaFusions/manta.run'  \&#xA;--referenceFasta='/path_to_genomes/genome.fa' \&#xA;--config '/opt/illumina/Isis/2.6.25.18/Workflows/WholeGenomeRnaSeqWorker/scripts/configManta.py.ini'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The file shown in the line above &lt;code&gt;configManta.py.ini&lt;/code&gt; is however no were to be found.&#xA;The &lt;a href=&quot;https://github.com/Illumina/manta/blob/master/docs/userGuide/README.md#advanced-configuration-options&quot; rel=&quot;nofollow noreferrer&quot;&gt;Manta user guide&lt;/a&gt; states in the linked section&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Options listed in the file: &lt;code&gt;${MANTA_INSTALL_PATH}/bin/configManta.py.ini&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The &lt;code&gt;configManta.py&lt;/code&gt; is in the expected directory, only the &lt;code&gt;configManta.py.ini&lt;/code&gt; is absent.&lt;/p&gt;&#xA;&lt;p&gt;I am using &lt;a href=&quot;https://anaconda.org/bioconda/manta&quot; rel=&quot;nofollow noreferrer&quot;&gt;Manta version  1.6.0 from bioconda&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Your insights will be much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="10791" LastEditorUserId="10791" LastEditDate="2021-04-09T12:20:01.297" LastActivityDate="2021-04-09T12:20:01.297" Title="Manta SV caller: I can't find a file in the path specified by the user guide" Tags="|fusions|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15702" PostTypeId="2" ParentId="14101" CreationDate="2021-04-08T13:57:06.427" Score="4" Body="&lt;p&gt;You can also use the SeqIO module from Biopython &lt;a href=&quot;https://biopython.org/docs/1.75/api/Bio.SeqIO.PdbIO.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biopython.org/docs/1.75/api/Bio.SeqIO.PdbIO.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and do something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;for record in SeqIO.parse(target, &amp;quot;pdb-atom&amp;quot;):&#xA;    print(record.seq)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where target is the path to your pdb file&#xA;the parser has multiple input/output functions, and pdb-atom is the one that will read out of the coordinates itself.&lt;/p&gt;&#xA;" OwnerUserId="11969" LastActivityDate="2021-04-08T13:57:06.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15704" PostTypeId="1" CreationDate="2021-04-08T16:04:30.033" Score="1" ViewCount="134" Body="&lt;p&gt;I have some SARS-CoV-2 sequencing data that I'm trying to annotate with SnpEff, however SnpEff doesn't appear to be recognizing multiple adjacent SNPs within the same codon and correctly calling them as a single MNP (multiple-nucleotide polymorphism).&lt;/p&gt;&#xA;&lt;p&gt;I'm using the GenBank reference MN908947.3. Here are the relevant lines of the annotated VCF file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MN908947.3  28280   .   G   C   2911.06 PASS    AC=2;AF=1.00;AN=2;DP=65;ExcessHet=3.0103;FS=0.000;MLEAC=2;MLEAF=1.00;MQ=60.00;QD=31.56;SOR=8.380;ANN=C|missense_variant|MODERATE|N|Gene_28273_29532|transcript|QHD43423.2|protein_coding|1/1|c.7G&amp;gt;C|p.Asp3His|7/1260|7/1260|3/419||,C|upstream_gene_variant|MODIFIER|ORF10|Gene_29557_29673|transcript|QHI42199.1|protein_coding||c.-1278G&amp;gt;C|||||1278|,C|downstream_gene_variant|MODIFIER|S|Gene_21562_25383|transcript|QHD43416.1|protein_coding||c.*2896G&amp;gt;C|||||2896|,C|downstream_gene_variant|MODIFIER|ORF3a|Gene_25392_26219|transcript|QHD43417.1|protein_coding||c.*2060G&amp;gt;C|||||2060|,C|downstream_gene_variant|MODIFIER|E|Gene_26244_26471|transcript|QHD43418.1|protein_coding||c.*1808G&amp;gt;C|||||1808|,C|downstream_gene_variant|MODIFIER|M|Gene_26522_27190|transcript|QHD43419.1|protein_coding||c.*1089G&amp;gt;C|||||1089|,C|downstream_gene_variant|MODIFIER|ORF6|Gene_27201_27386|transcript|QHD43420.1|protein_coding||c.*893G&amp;gt;C|||||893|,C|downstream_gene_variant|MODIFIER|ORF7a|Gene_27393_27758|transcript|QHD43421.1|protein_coding||c.*521G&amp;gt;C|||||521|,C|downstream_gene_variant|MODIFIER|ORF8|Gene_27893_28258|transcript|QHD43422.1|protein_coding||c.*21G&amp;gt;C|||||21| GT:AD:DP:GQ:PL  1/1:0,65:65:99:2925,196,0&#xA;MN908947.3  28281   .   A   T   2912.06 PASS    AC=2;AF=1.00;AN=2;DP=78;ExcessHet=3.0103;FS=0.000;MLEAC=2;MLEAF=1.00;MQ=60.00;QD=26.91;SOR=8.380;ANN=T|missense_variant|MODERATE|N|Gene_28273_29532|transcript|QHD43423.2|protein_coding|1/1|c.8A&amp;gt;T|p.Asp3Val|8/1260|8/1260|3/419||,T|upstream_gene_variant|MODIFIER|ORF10|Gene_29557_29673|transcript|QHI42199.1|protein_coding||c.-1277A&amp;gt;T|||||1277|,T|downstream_gene_variant|MODIFIER|S|Gene_21562_25383|transcript|QHD43416.1|protein_coding||c.*2897A&amp;gt;T|||||2897|,T|downstream_gene_variant|MODIFIER|ORF3a|Gene_25392_26219|transcript|QHD43417.1|protein_coding||c.*2061A&amp;gt;T|||||2061|,T|downstream_gene_variant|MODIFIER|E|Gene_26244_26471|transcript|QHD43418.1|protein_coding||c.*1809A&amp;gt;T|||||1809|,T|downstream_gene_variant|MODIFIER|M|Gene_26522_27190|transcript|QHD43419.1|protein_coding||c.*1090A&amp;gt;T|||||1090|,T|downstream_gene_variant|MODIFIER|ORF6|Gene_27201_27386|transcript|QHD43420.1|protein_coding||c.*894A&amp;gt;T|||||894|,T|downstream_gene_variant|MODIFIER|ORF7a|Gene_27393_27758|transcript|QHD43421.1|protein_coding||c.*522A&amp;gt;T|||||522|,T|downstream_gene_variant|MODIFIER|ORF8|Gene_27893_28258|transcript|QHD43422.1|protein_coding||c.*22A&amp;gt;T|||||22| GT:AD:DP:GQ:PL  1/1:0,65:65:99:2926,196,0&#xA;MN908947.3  28282   .   T   A   2912.06 PASS    AC=2;AF=1.00;AN=2;DP=78;ExcessHet=3.0103;FS=0.000;MLEAC=2;MLEAF=1.00;MQ=60.00;QD=28.71;SOR=8.380;ANN=A|missense_variant|MODERATE|N|Gene_28273_29532|transcript|QHD43423.2|protein_coding|1/1|c.9T&amp;gt;A|p.Asp3Glu|9/1260|9/1260|3/419||,A|upstream_gene_variant|MODIFIER|ORF10|Gene_29557_29673|transcript|QHI42199.1|protein_coding||c.-1276T&amp;gt;A|||||1276|,A|downstream_gene_variant|MODIFIER|S|Gene_21562_25383|transcript|QHD43416.1|protein_coding||c.*2898T&amp;gt;A|||||2898|,A|downstream_gene_variant|MODIFIER|ORF3a|Gene_25392_26219|transcript|QHD43417.1|protein_coding||c.*2062T&amp;gt;A|||||2062|,A|downstream_gene_variant|MODIFIER|E|Gene_26244_26471|transcript|QHD43418.1|protein_coding||c.*1810T&amp;gt;A|||||1810|,A|downstream_gene_variant|MODIFIER|M|Gene_26522_27190|transcript|QHD43419.1|protein_coding||c.*1091T&amp;gt;A|||||1091|,A|downstream_gene_variant|MODIFIER|ORF6|Gene_27201_27386|transcript|QHD43420.1|protein_coding||c.*895T&amp;gt;A|||||895|,A|downstream_gene_variant|MODIFIER|ORF7a|Gene_27393_27758|transcript|QHD43421.1|protein_coding||c.*523T&amp;gt;A|||||523|,A|downstream_gene_variant|MODIFIER|ORF8|Gene_27893_28258|transcript|QHD43422.1|protein_coding||c.*23T&amp;gt;A|||||23| GT:AD:DP:GQ:PL  1/1:0,65:65:99:2926,196,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These are the positions corresponding to codon 3 on the N gene, they should be annotated as a single MNP, p.Asp3Leu, but istead they've been annotated seperately, as three individual SNPs.&lt;/p&gt;&#xA;&lt;p&gt;I'm using the latest version of SnpEff (v5), and according to the manual SnpEff should have this functionality, what am I doing wrong?&lt;/p&gt;&#xA;&lt;p&gt;You can see an example of how it's supposed to work &lt;a href=&quot;https://pcingola.github.io/SnpEff/se_inputoutput/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1456" LastActivityDate="2021-04-08T16:04:30.033" Title="SnpEff not correctly annotating multiple-nucleotide polymorphisms" Tags="|sequence-annotation|variant-calling|snpeff|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="15705" PostTypeId="2" ParentId="10777" CreationDate="2021-04-08T18:54:40.870" Score="1" Body="&lt;p&gt;The other answer is incorrect. The original poster is asking about paired-end sequencing data, not paired treatment/control experimental design.&lt;/p&gt;&#xA;&lt;p&gt;Here is how you analyze paired-end sequencing data (this was an answer I gave on Biostars last year):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/303656/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/303656/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10379" LastActivityDate="2021-04-08T18:54:40.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15708" PostTypeId="2" ParentId="15701" CreationDate="2021-04-08T23:45:09.043" Score="2" Body="&lt;p&gt;The &lt;code&gt;--config&lt;/code&gt; file option for Manta can be used to override the defaults in the global config file. If you type &lt;code&gt;configManta.py -h&lt;/code&gt;, you'll see where the global config file has been installed. For me, it's here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/home/steve/miniconda3/envs/manta/share/manta-1.6.0-1/bin/configManta.py.ini&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So the command in your log file takes a specific config file installed into:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/opt/illumina/Isis/2.6.25.18/...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I suspect 'Isis' is a reference to Illumina's analysis tool 'IsisRNA'. Papers that mention Illumina's 'RnaReadCounter' tool, say that it is part of the &lt;em&gt;internal&lt;/em&gt; analysis tool 'IsisRNA'. Because it's one of Illumina's in-house tools, I think you'd need to contact Illumina support to get a copy. I would only do this though if you need to reproduce &lt;em&gt;exactly&lt;/em&gt; as per the command in your log file. If you only need to roughly reproduce the output, the defaults in the global config file are probably fine.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2021-04-08T23:45:09.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15710" PostTypeId="1" CreationDate="2021-04-09T15:33:45.177" Score="0" ViewCount="185" Body="&lt;p&gt;I have a snp information in an excel file and I used SNPstat to do some analysis. But I also need a plot of LD for these snps. But my file is not formatted as the most programs required (.ped, hapmap, Plink...). Here is a screen shot of file: &lt;a href=&quot;https://i.stack.imgur.com/6Vja9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6Vja9.png&quot; alt=&quot;Data file&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have also chromosome numbers and rs information for this snps. Due to this data format problem I couldn't use Haploview or other programs. Can you suggest me a solution to create a LD graph?&lt;/p&gt;&#xA;" OwnerUserId="1757" LastActivityDate="2021-04-09T15:33:45.177" Title="How to plot a graph of Linkage disequilibrium with limited information" Tags="|snp|haplotypes|graphs|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15713" PostTypeId="1" CreationDate="2021-04-12T06:42:55.317" Score="3" ViewCount="1326" Body="&lt;p&gt;I download a BAM file from &lt;a href=&quot;ftp://ftp-trace.ncbi.nlm.nih.gov//giab/ftp/data/NA12878/10Xgenomics_ChromiumGenome_LongRanger2.0_06202016/NA12878_GRCh38.bam&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp-trace.ncbi.nlm.nih.gov:/giab/ftp/data/NA12878/10Xgenomics_ChromiumGenome_LongRanger2.0_06202016/NA12878_GRCh38.bam&lt;/a&gt; (171G), and I am trying to view the comparison result of the BAM file using the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[yyn@localhost All]$ samtools flagstat NA12878_GRCh38.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The result shows that the file is truncated, the result as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[E::bgzf_uncompress] Inflate operation failed: progress temporarily not possible, or in() / out() returned an error&#xA;[E::bgzf_read] Read block operation failed with error 1 after 0 of 4 bytes&#xA;[bam_flagstat_core] Truncated file? Continue anyway.&#xA;72639 + 0 in total (QC-passed reads + QC-failed reads)&#xA;896 + 0 secondary&#xA;0 + 0 supplementary&#xA;2529 + 0 duplicates&#xA;72639 + 0 mapped (100.00% : N/A)&#xA;71743 + 0 paired in sequencing&#xA;36865 + 0 read1&#xA;34878 + 0 read2&#xA;48600 + 0 properly paired (67.74% : N/A)&#xA;69949 + 0 with itself and mate mapped&#xA;1794 + 0 singletons (2.50% : N/A)&#xA;10233 + 0 with mate mapped to a different chr&#xA;2869 + 0 with mate mapped to a different chr (mapQ&amp;gt;=5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The total number of reads is only 72639, Obviously, this is the wrong result. And I checked the file again with the following command&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[yyn@localhost All]$ samtools quickcheck -qv NA12878_GRCh38.bam &amp;amp;&amp;amp; echo 'all ok' || echo 'failed check'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The result is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;all ok&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I checked the contents of the file again, and the result showed that there was no problem with the first 72620 lines in the file, but there was a problem with the first 72630 lines, so the file should be truncated in the 72620 to 72630 lines.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[yyn@localhost All]$ samtools view NA12878_GRCh38.bam | head -n 72610 &amp;gt;&amp;gt; head72610.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That's ok.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[yyn@localhost All]$ samtools view NA12878_GRCh38.bam | head -n 72620 &amp;gt;&amp;gt; head72620.sam&#xA;[E::bgzf_uncompress] Inflate operation failed: progress temporarily not possible, or in() / out() returned an error&#xA;[E::bgzf_read] Read block operation failed with error 1 after 0 of 4 bytes&#xA;[main_samview] truncated file.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;samtools sort&lt;/code&gt; and &lt;code&gt;samtools index&lt;/code&gt; are the same error results as above. Does anyone know what's going on? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="12006" LastEditorUserId="298" LastEditDate="2021-04-12T09:32:41.673" LastActivityDate="2021-04-12T09:32:41.673" Title="Inflate operation failed: progress temporarily not possible, or in() / out() returned an error" Tags="|sam|samtools|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15715" PostTypeId="2" ParentId="15713" CreationDate="2021-04-12T09:25:50.797" Score="1" Body="&lt;p&gt;My best guess would be a corruption during download as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools quickcheck -qvvvv ftp://ftp-trace.ncbi.nlm.nih.gov:/giab/ftp/data/NA12878/10Xgenomics_ChromiumGenome_LongRanger2.0_06202016/NA12878_GRCh38.bam                &#xA;verbosity set to 4&#xA;checking ftp://ftp-trace.ncbi.nlm.nih.gov:/giab/ftp/data/NA12878/10Xgenomics_ChromiumGenome_LongRanger2.0_06202016/NA12878_GRCh38.bam&#xA;opened ftp://ftp-trace.ncbi.nlm.nih.gov:/giab/ftp/data/NA12878/10Xgenomics_ChromiumGenome_LongRanger2.0_06202016/NA12878_GRCh38.bam&#xA;ftp://ftp-trace.ncbi.nlm.nih.gov:/giab/ftp/data/NA12878/10Xgenomics_ChromiumGenome_LongRanger2.0_06202016/NA12878_GRCh38.bam is sequence data&#xA;ftp://ftp-trace.ncbi.nlm.nih.gov:/giab/ftp/data/NA12878/10Xgenomics_ChromiumGenome_LongRanger2.0_06202016/NA12878_GRCh38.bam has 196 targets in header.&#xA;ftp://ftp-trace.ncbi.nlm.nih.gov:/giab/ftp/data/NA12878/10Xgenomics_ChromiumGenome_LongRanger2.0_06202016/NA12878_GRCh38.bam has good EOF block.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Plus I can successfully run &lt;code&gt;idxstats&lt;/code&gt; from remote and fetch individual chromosomes without error via &lt;code&gt;samtools view&lt;/code&gt;. Try redownloading before digging deeper.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-04-12T09:25:50.797" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15716" PostTypeId="2" ParentId="15685" CreationDate="2021-04-12T09:51:22.320" Score="2" Body="&lt;p&gt;The representation for a variant present in a haploid genome should be &lt;code&gt;1&lt;/code&gt;. A heterozygote in a diploid genome would be &lt;code&gt;0/1&lt;/code&gt; (or &lt;code&gt;0|1&lt;/code&gt; or &lt;code&gt;1|0&lt;/code&gt;  if phased). So, to add a &lt;code&gt;1&lt;/code&gt; as the genotype, you would need to first insert a &lt;code&gt;GT&lt;/code&gt; to the beginning of each INFO line and then the &lt;code&gt;1&lt;/code&gt; at the beginning of the last field. But, this should happen only on lines that are not headers (do not start with &lt;code&gt;#&lt;/code&gt;). This should do it:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ awk -F'\t' -v OFS=&amp;quot;\t&amp;quot; '{ if(/^#/){print}else{$9=&amp;quot;GT:&amp;quot;$9;$10=&amp;quot;1:&amp;quot;$10;print}}' file.vcf &#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  Variants:_LGC19-XL01_S63_L001_R_001_(trimmed)&#xA;CFC381_K12_Bw25113  360287  .   GGGAAT  AATTCC  1952.76 .   NS=1;VF=0.380;SB=0.533;SB50=0.60;SB65=1.0;TYPE=Substitution;AVQUAL=30   GT:DP:AO    1:231:89&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So just redirect the output to a fixed file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ awk -F'\t' -v OFS=&amp;quot;\t&amp;quot; '{ if(/^#/){print}else{$9=&amp;quot;GT:&amp;quot;$9;$10=&amp;quot;1:&amp;quot;$10;print}}' file.vcf &amp;gt; fixed.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, in this specific case, you could also do something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ sed -i.bak '/^[^#]/{s/\&amp;lt;DP:AO\t/GT:DP:AO\t1:/}' file.vcf &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But that assumes that all lines will have the exact same INFO field of &lt;code&gt;DP:AO&lt;/code&gt; which is not a safe assumption since each variant can have different INFO fields.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2021-04-28T14:40:22.693" LastActivityDate="2021-04-28T14:40:22.693" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15717" PostTypeId="1" CreationDate="2021-04-12T10:09:28.003" Score="0" ViewCount="41" Body="&lt;p&gt;I'm currently working on a program which will determine the activity difference between an original enzyme and a variant with just 1 or 2 variants. Of course, I'm not talking about the &amp;quot;kinetic activity&amp;quot;, but some indications of a probable activity difference should be given. The system has to be sensitive enough to determine the probable activity difference between unthought-of variants.&lt;/p&gt;&#xA;&lt;p&gt;From the information that I've got from my research, there is two option. I can use one of 3 quantitive enzyme reaction analysis tools (such as EVB, FEB and LRA) or their hybrids.&lt;/p&gt;&#xA;&lt;p&gt;EVB =&amp;gt; Empirical Valence Bond&lt;/p&gt;&#xA;&lt;p&gt;FEB =&amp;gt; Free Energy Perturbation&lt;/p&gt;&#xA;&lt;p&gt;LRA =&amp;gt; Linear Response Approximation&lt;/p&gt;&#xA;&lt;p&gt;Source:&#xA;&lt;a href=&quot;https://www.researchgate.net/post/Can-the-activity-of-proteins-be-studied-using-molecular-dynamic-simulations&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.researchgate.net/post/Can-the-activity-of-proteins-be-studied-using-molecular-dynamic-simulations&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, my questions are: - Does these application be used for such purpose? - For an accurate prediction of  enzyme activity difference, are these programs good enough or require developement? - Which one of them can give the best result? - And lastly, are QM/MM methods a must for an accurate answer?&lt;/p&gt;&#xA;&lt;p&gt;I'm a high school student so sorry if my questions sound dumb.&#xA;If you can answer my inquiry's I would appreciate it:)&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="12009" LastActivityDate="2021-04-16T13:35:12.523" Title="Holistic enzyme activity determination with computation" Tags="|proteins|protein-structure|proteomics|structural-biology|" AnswerCount="1" CommentCount="0" ClosedDate="2021-04-28T00:45:11.017" ContentLicense="CC BY-SA 4.0" />
  <row Id="15719" PostTypeId="2" ParentId="7337" CreationDate="2021-04-12T15:31:45.030" Score="1" Body="&lt;p&gt;TLDR: Harmony can work on 106~ cell samples but has less versatility then methods like BATMAN. BATMAN is useful if you need your data for differential gene expression and single-cell eQTL and can work on ~200 cell data.&lt;/p&gt;&#xA;&lt;p&gt;Well their certainly is a minimum which comes down to several factors. As many of these rely on algorithms like KNN and other clustering algorithms for integration the threshold is dependent on the quality of the data and the  algorithms abilities to predict reasonable clusters from that data. As well as the downstream uses of the data...&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S2589004220303709&quot; rel=&quot;nofollow noreferrer&quot;&gt;BATMAN highest performance on simulated set 200 cells and 1000 cells&lt;/a&gt;&lt;br /&gt;&#xA;&lt;strong&gt;With 200 cells from one batch and and 1000 in the other only BATMAN succeeded in clustering the high dimensionality simulated data well. If you need your data for differential gene expression and single-cell eQTL, BATMAN seems to be the correct choice, for small sets.&lt;/strong&gt; &amp;quot;The downside of these methods is that they operate in latent space, which limits their interpretability and use in downstream analyses such as differential gene expression and single-cell eQTL analyses.&amp;quot;&#xA;&amp;quot;A shows that the two original datasets become better mixed with each other. BATMAN has the highest performance not only when considering the top two principal components but also when considering more top principal components. It is the only method that manages to efficiently maintain a high iLISI score in a larger number of dimensions.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.researchgate.net/publication/328742713_Fast_sensitive_and_flexible_integration_of_single_cell_data_with_Harmony&quot; rel=&quot;nofollow noreferrer&quot;&gt;Harmony is best if... differential gene expression and single-cell eQTL is not necessary&lt;/a&gt;&#xA;&lt;strong&gt;&amp;quot;Moreover, we show that Harmony requires dramatically fewer computational resources. It is the only available algorithm that makes the integration of ~106 cells feasible on a personal computer.&amp;quot;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7338326/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Computing time benchmarks-section&lt;/a&gt;&#xA;It seems as the runtime of these algorithms is less than 2 minutes, using a set with less than ~2000 cells will not benefit your computational time much.&#xA;&amp;quot;When processing a small data set of ~ 2000 cells, all four methods took less than 2 min.&amp;quot;&#xA;&amp;quot;To obtain such datasets, we downsampled the MCA and TM datasets to obtain a total of 9 sets of data containing between ~ 2000 and ~ 140,000 cells, while the number of highly variable genes (HVGs) was controlled in a range from ~ 2000 to ~ 3000 (Table S1).&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="6487" LastActivityDate="2021-04-12T15:31:45.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15720" PostTypeId="1" CreationDate="2021-04-13T00:01:18.277" Score="0" ViewCount="1929" Body="&lt;p&gt;I am working on integrating a labelled single cell RNA seq cell atlas with an unlabelled one. I am wondering how do I determine the value of max.features when integrating two single cell RNA seq datasets?&lt;/p&gt;&#xA;&lt;p&gt;Below is the code I am using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;epithelial = Read10X(data.dir=&amp;quot;/projects/b1025/sdi0596/Covid19_Single_Cell_RNASeq_project/filtered_matrices_count/epithelial/filtered_feature_bc_matrix/&amp;quot;,gene.column=1,cell.column=1)&#xA;habermann = Read10X(data.dir=&amp;quot;/projects/b1025/sdi0596/Covid19_Single_Cell_RNASeq_project/filtered_matrices_count/Habermann_Single_Cell_Data/&amp;quot;,gene.column=1,cell.column=1)&#xA;&#xA;data.list=c(epithelial,habermann)&#xA;&#xA;normalize and identify variable features for each dataset independently&#xA;ifnb.list &amp;lt;- lapply(X = data.list, FUN = function(x) {&#xA;x &amp;lt;- NormalizeData(x)&#xA;x &amp;lt;- FindVariableFeatures(x, selection.method = &amp;quot;vst&amp;quot;, nfeatures = 2000)&#xA;})&#xA;&#xA;Perform integration&#xA;immune.achors= FindIntegrationAnchors(object.list = ifnb.list,max.features = 200,verbose = TRUE)&#xA;&#xA;Error in FindIntegrationAnchors(object.list = ifnb.list, max.features = 200, :&#xA;Max dimension too large: objects 1, 2 contain fewer than 30 cells.&#xA;Please specify a maximum dimensions that is less than the number of cells in any object (4).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Regarding the dimension of these objects, the epithelial one is 33538 x 49109 and the habermann is 33694 x 220213.&#xA;Any insights would be highly helpful.&lt;/p&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="4307" LastActivityDate="2023-04-20T01:40:57.027" Title="Error in FindIntegrationAnchors- Seurat package" Tags="|scrnaseq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15722" PostTypeId="1" AcceptedAnswerId="15725" CreationDate="2021-04-13T06:16:50.333" Score="3" ViewCount="1354" Body="&lt;p&gt;I have set of ~5 processes which perform very similar tasks such that their differences can be parameterized. Rather than putting 5 items through some channel (which seems complicated), or defining 5 separate processes, it appears to be possible in Nextflow to declare the processes in a for loop, and output into an array of channels.&lt;/p&gt;&#xA;&lt;p&gt;Below is a prototypical example and it seems to work as intended. (I understand this could be done differently with some channel cross product operator but I am just giving this as an example of a process in a for loop.)&lt;/p&gt;&#xA;&lt;p&gt;Is there anything wrong with doing this? Is there a better way to do it?&lt;/p&gt;&#xA;&lt;p&gt;Bonus question - the name of the process below is &amp;quot;foo&amp;quot;. I would like to have the index reflected in the name i.e. &amp;quot;foo_$i&amp;quot; but this doesn't seem possible by default - any way to do this??&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;some_inputs = ['prot','dna', 'rna']&#xA;&#xA;some_parameters = [5,3,2,7,1]&#xA;&#xA;// Set up an array of channels to output to&#xA;receivers = new ArrayList&amp;lt;Channel&amp;gt;()&#xA;for (i = 0; i &amp;lt; 5; i++) {&#xA;    receivers.add(Channel.create())&#xA;}&#xA;&#xA;// Iterate 5 times to create 5 processes each with different parameters&#xA;for (i = 0; i &amp;lt; 5; i++) {&#xA;&#xA;    // WOW - process can be defined inside a for loop!!&#xA;    process &amp;quot;foo&amp;quot; {&#xA;&#xA;      // Extract parameters here so correct $i value is used&#xA;      p = some_parameters[i]&#xA;&#xA;      input:&#xA;      val x from Channel.from(some_inputs)&#xA;&#xA;      output:&#xA;      val &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$x $&lt;/span&gt;p&amp;quot; into receivers[i] // Note that correct index $i seems to be used here&#xA;&#xA;      &amp;quot;&amp;quot;&amp;quot;&#xA;      &amp;quot;&amp;quot;&amp;quot;&#xA;    }&#xA;}&#xA;&#xA;// Confirm that each of the processes outputted independently with the correct parameter&#xA;receivers[0].view { &amp;quot;Received[0]: &lt;span class=&quot;math-container&quot;&gt;$it&quot; }&#xA;receivers[1].view { &quot;Received[1]: $it&quot; }&#xA;receivers[2].view { &quot;Received[2]: $it&quot; }&#xA;receivers[3].view { &quot;Received[3]: $it&quot; }&#xA;receivers[4].view { &quot;Received[4]: $&lt;/span&gt;it&amp;quot; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output - note that the parameter is consistent within each input channel i.e. working correctly:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;N E X T F L O W  ~  version 20.10.0&#xA;Launching `play.nf` [fabulous_keller] - revision: c89e2b2281&#xA;executor &amp;gt;  local (15)&#xA;[10/b477ec] process &amp;gt; foo (3) [100%] 3 of 3 ✔&#xA;[16/8282ba] process &amp;gt; foo (2) [100%] 3 of 3 ✔&#xA;[db/f7327e] process &amp;gt; foo (1) [100%] 3 of 3 ✔&#xA;[fb/886f6f] process &amp;gt; foo (1) [100%] 3 of 3 ✔&#xA;[46/c34ed3] process &amp;gt; foo (1) [100%] 3 of 3 ✔&#xA;Received[1]: rna 3&#xA;Received[1]: prot 3&#xA;Received[4]: rna 1&#xA;Received[2]: rna 2&#xA;Received[0]: dna 5&#xA;Received[2]: dna 2&#xA;Received[2]: prot 2&#xA;Received[0]: prot 5&#xA;Received[3]: rna 7&#xA;Received[0]: rna 5&#xA;Received[1]: dna 3&#xA;Received[3]: dna 7&#xA;Received[4]: dna 1&#xA;Received[3]: prot 7&#xA;Received[4]: prot 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11959" LastActivityDate="2021-04-14T14:00:36.763" Title="Nextflow: Can I parameterize similar processes inside a for loop?" Tags="|nextflow|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15723" PostTypeId="1" CreationDate="2021-04-13T08:14:02.497" Score="1" ViewCount="95" Body="&lt;p&gt;I have 6 folders. Each one contains 7 datasets of a specific type of cancer (RNA-Seq) and 7 datasets of normal tissue (healthy control for that type of cancer). A total of 84 datasets.&lt;/p&gt;&#xA;&lt;p&gt;I want to investigate my gene of interest in each type of cancer.&lt;/p&gt;&#xA;&lt;p&gt;I believe the best strategy is to merge the FPKM columns and perform correlation analyses of my gene of interest (GOI) versus all other genes in both cancer and normal tissues. As a result, I will have 14 correlation tables, one for each type of cancer and one for each control tissue. Then, I can investigate the pathways related to the genes most correlated with my GOI in each type of cancer and write my biological interpretation.&lt;/p&gt;&#xA;&lt;p&gt;Am I in the right direction? Is this a good strategy to answer to my question?&lt;/p&gt;&#xA;&lt;p&gt;I could also compare the control and cancer samples to investigate the significant DEGs (fold change &amp;gt; 1) and the pathways activated by these DEGs, but my gene of interest doesn't have a strong pattern of gene expression, so I'm afraid it won't be in the list of statistically significant DEGs, that's why I'm preferring the correlation analysis approach.&lt;/p&gt;&#xA;&lt;p&gt;Any other idea?&lt;/p&gt;&#xA;&lt;p&gt;The controls were collected from the same patient and processed at the same time. I have the gene counts and can use EdgeR, but what should I do if my GOI doesn't appear among the statistically significant DEGs (fold change &amp;gt; 1)? How can I investigate the role of one single gene in a RNA-Seq dataset if it is not differentially expressed? It can have a role in cancer, even if it is not an important DEG. That's why I thought correlation analyses could give me interesting information.&lt;/p&gt;&#xA;" OwnerUserId="6509" LastEditorUserId="73" LastEditDate="2021-05-14T09:30:21.630" LastActivityDate="2021-05-14T09:30:53.883" Title="correlation coefficient versus DEGs analysis: what's the best approach for low expressed genes?" Tags="|r|rna-seq|correlation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15724" PostTypeId="6" CreationDate="2021-04-13T10:23:40.003" Score="0" Body="&lt;h1&gt;Kamil S Jaron&lt;/h1&gt;&#xA;&lt;p&gt;I am ...&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;loyally active in this community for years now&lt;/li&gt;&#xA;&lt;li&gt;trying my best to be kind and welcoming to everyone&lt;/li&gt;&#xA;&lt;li&gt;a fairly incompetent bioinformatian, so I am also quite patient&lt;/li&gt;&#xA;&lt;li&gt;know how hard is to maintain scientific software&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The only real reason I run for a mod is to  get more power on my crusaide for increased bioinformatics accessiblity. I am not sure how being a moderator will help me achieve that goal, but I do consider doubting my abilities as a sign of strenght, so I hope I will earn some symphaties for it.&lt;/p&gt;&#xA;&lt;p&gt;Is it allowed to give people doughnuts? Did I mention that although I do take this site seriously, I do like horsing around a lot?!&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2021-04-13T10:23:40.003" LastActivityDate="2021-04-13T10:23:40.003" CommentCount="0" CommunityOwnedDate="2021-04-13T10:23:40.003" ContentLicense="CC BY-SA 4.0" />
  <row Id="15725" PostTypeId="2" ParentId="15722" CreationDate="2021-04-13T11:25:08.340" Score="4" Body="&lt;p&gt;Use channels, this is what nextflow's entire design is based on. I'm honestly surprised that spawning 5 processes with the same name in a loop even works.&lt;/p&gt;&#xA;&lt;p&gt;Turn your parameters into regular channels, use the &lt;code&gt;combine&lt;/code&gt; operator (&lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#combine&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nextflow.io/docs/latest/operator.html#combine&lt;/a&gt;) to transform the 2 parameter channels into a channel that has each combination of params/inputs. Feed this channel into the  &lt;code&gt;foo&lt;/code&gt;  process. Afterward use &lt;code&gt;branch&lt;/code&gt; (&lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#branch&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nextflow.io/docs/latest/operator.html#branch&lt;/a&gt;) to split your &lt;code&gt;foo&lt;/code&gt; output channel into 5 different channels.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;typechan = Channel.from(5,3,2,7,1)&#xA;numchan = Channel.from('prot','dna', 'rna')&#xA;&#xA;cartesian = numchan.combine(typechan)&#xA;&#xA;process foo {&#xA;  input:&#xA;  val numtype from cartesian&#xA;  output:&#xA;  val numtype into bar &#xA;  script:&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  echo $numtype&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;bar.branch {&#xA;  one: it[1] == 1&#xA;  two: it[1] == 2&#xA;  three: it[1] == 3 &#xA;  five: it[1] == 5&#xA;  seven : it[1] == 7&#xA;}.set {result}&#xA;result.one.view()&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3818" LastActivityDate="2021-04-13T11:25:08.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15727" PostTypeId="2" ParentId="15722" CreationDate="2021-04-13T14:28:38.817" Score="3" Body="&lt;p&gt;You don't need to create processes in a loop when you have channels. The &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#input-repeaters&quot; rel=&quot;nofollow noreferrer&quot;&gt;each qualifier&lt;/a&gt; allows you to repeat the execution of a process for each item in a collection. For example, the contents of 'test.nf':&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;some_inputs = Channel.of( 'prot', 'dna', 'rna' )&#xA;some_parameters = Channel.of( 5, 3, 2, 7, 1 )&#xA;&#xA;process foo {&#xA;&#xA;    tag { &amp;quot;task: &lt;span class=&quot;math-container&quot;&gt;${task.index} - $&lt;/span&gt;{x},${y}&amp;quot; }&#xA;&#xA;    input:&#xA;    val x from some_inputs&#xA;    each y from some_parameters&#xA;&#xA;    output:&#xA;    tuple val(x), val(y) into receivers&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    echo &amp;quot;x: &lt;span class=&quot;math-container&quot;&gt;${x}; y: $&lt;/span&gt;{y}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;results = receivers.branch { x, y -&amp;gt;&#xA;    &#xA;    five: y == 5&#xA;        return tuple( x, y )&#xA;&#xA;    three: y == 3&#xA;        return tuple( x, y )&#xA;&#xA;    two: y == 2&#xA;        return tuple( x, y )&#xA;&#xA;    seven: y == 7&#xA;        return tuple( x, y )&#xA;    &#xA;    one: y == 1&#xA;        return tuple( x, y )&#xA;}&#xA;&#xA;results.five.view { &amp;quot;My result for param 5 is: &lt;span class=&quot;math-container&quot;&gt;$it&quot; }&#xA;results.three.view { &quot;My result for param 3 is: $it&quot; }&#xA;results.two.view { &quot;My result for param 2 is: $it&quot; }&#xA;results.seven.view { &quot;My result for param 7 is: $it&quot; }&#xA;results.one.view { &quot;My result for param 1 is: $&lt;/span&gt;it&amp;quot; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then running like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;nextflow run -ansi-log false test.nf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;N E X T F L O W  ~  version 20.10.0&#xA;Launching `test.nf` [fabulous_franklin] - revision: 14c3290d2b&#xA;[6e/c12ce7] Submitted process &amp;gt; foo (task: 7 - dna,3)&#xA;[93/616f93] Submitted process &amp;gt; foo (task: 4 - prot,7)&#xA;[27/0bc3c4] Submitted process &amp;gt; foo (task: 3 - prot,2)&#xA;[0d/0d0b51] Submitted process &amp;gt; foo (task: 9 - dna,7)&#xA;[d4/f56d18] Submitted process &amp;gt; foo (task: 6 - dna,5)&#xA;[b0/d1535f] Submitted process &amp;gt; foo (task: 8 - dna,2)&#xA;[13/0a660a] Submitted process &amp;gt; foo (task: 5 - prot,1)&#xA;[f0/1c2f57] Submitted process &amp;gt; foo (task: 2 - prot,3)&#xA;[3f/3e893d] Submitted process &amp;gt; foo (task: 1 - prot,5)&#xA;[19/2921de] Submitted process &amp;gt; foo (task: 10 - dna,1)&#xA;My result for param 3 is: [dna, 3]&#xA;My result for param 7 is: [prot, 7]&#xA;My result for param 7 is: [dna, 7]&#xA;[a7/953a2b] Submitted process &amp;gt; foo (task: 11 - rna,3)&#xA;[1c/0fd4ef] Submitted process &amp;gt; foo (task: 12 - rna,5)&#xA;My result for param 1 is: [prot, 1]&#xA;[39/d21adf] Submitted process &amp;gt; foo (task: 14 - rna,7)&#xA;My result for param 5 is: [dna, 5]&#xA;My result for param 2 is: [prot, 2]&#xA;My result for param 2 is: [dna, 2]&#xA;[f6/8509a7] Submitted process &amp;gt; foo (task: 15 - rna,1)&#xA;My result for param 5 is: [prot, 5]&#xA;[6f/cee0ae] Submitted process &amp;gt; foo (task: 13 - rna,2)&#xA;My result for param 3 is: [rna, 3]&#xA;My result for param 1 is: [dna, 1]&#xA;My result for param 2 is: [rna, 2]&#xA;My result for param 3 is: [prot, 3]&#xA;My result for param 1 is: [rna, 1]&#xA;My result for param 5 is: [rna, 5]&#xA;My result for param 7 is: [rna, 7]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The answer to your bonus question is to use the &lt;code&gt;task.index&lt;/code&gt; variable in your script. I used this to &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#tag&quot; rel=&quot;nofollow noreferrer&quot;&gt;tag&lt;/a&gt; your job, but it can be used anywhere in the process declaration. You likely won't need this variable though.&lt;/p&gt;&#xA;&lt;p&gt;Usually, using the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#branch&quot; rel=&quot;nofollow noreferrer&quot;&gt;branch operator&lt;/a&gt; is enough for most use cases. However,it's possible to implement your own branching function. The following uses the new DSL, DSL2, to automatically fork the 'receivers' channel:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;nextflow.enable.dsl=2&#xA;&#xA;process foo {&#xA;&#xA;    tag { &amp;quot;task: &lt;span class=&quot;math-container&quot;&gt;${task.index} - $&lt;/span&gt;{x},${y}&amp;quot; }&#xA;&#xA;    input:&#xA;    val x&#xA;    each y&#xA;&#xA;    output:&#xA;    tuple val(x), val(y)&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    echo &amp;quot;x: &lt;span class=&quot;math-container&quot;&gt;${x}; y: $&lt;/span&gt;{y}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;def branchByElement( input_ch, elements, by=0 ) {&#xA;&#xA;    def targets = [:]&#xA;    for( element in elements ) {&#xA;        def key = element&#xA;        targets[ key ] = { input_ch.filter { it[by] == key } }&#xA;    }&#xA;    return targets&#xA;}&#xA;&#xA;workflow {&#xA;&#xA;    some_inputs = [ 'prot', 'dna', 'rna' ]&#xA;    some_parameters = [ 5, 3, 2, 7, 1 ]&#xA;&#xA;    inputs_ch = Channel.fromList( some_inputs )&#xA;    parameters_ch = Channel.fromList( some_parameters )&#xA;&#xA;    foo( inputs_ch, parameters_ch )&#xA;&#xA;    parameter_results = branchByElement( foo.out, some_parameters, by=1 )&#xA;    input_results = branchByElement( foo.out, some_inputs, by=0 )&#xA;&#xA;    parameter_results[7].call().view { &amp;quot;Param 7: &lt;span class=&quot;math-container&quot;&gt;$it&quot; }&#xA;    input_results.dna().view { &quot;Input dna: $&lt;/span&gt;it&amp;quot; }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;N E X T F L O W  ~  version 20.10.0&#xA;Launching `test.nf` [distraught_meninsky] - revision: 04df251752&#xA;[bb/32325f] Submitted process &amp;gt; foo (task: 4 - prot,7)&#xA;[a0/19fdf5] Submitted process &amp;gt; foo (task: 8 - dna,2)&#xA;[e9/4c1d09] Submitted process &amp;gt; foo (task: 6 - dna,5)&#xA;[a3/dcb002] Submitted process &amp;gt; foo (task: 2 - prot,3)&#xA;[db/7e612e] Submitted process &amp;gt; foo (task: 5 - prot,1)&#xA;[cd/dfa84e] Submitted process &amp;gt; foo (task: 9 - dna,7)&#xA;[d4/6a85f2] Submitted process &amp;gt; foo (task: 7 - dna,3)&#xA;[f0/8e8bf4] Submitted process &amp;gt; foo (task: 1 - prot,5)&#xA;[89/f20a28] Submitted process &amp;gt; foo (task: 3 - prot,2)&#xA;Input dna: [dna, 2]&#xA;Param 7: [prot, 7]&#xA;Input dna: [dna, 5]&#xA;[53/056fe4] Submitted process &amp;gt; foo (task: 10 - dna,1)&#xA;[f3/a61abf] Submitted process &amp;gt; foo (task: 12 - rna,3)&#xA;Input dna: [dna, 7]&#xA;[f9/a568a8] Submitted process &amp;gt; foo (task: 13 - rna,2)&#xA;Param 7: [dna, 7]&#xA;[06/72bb6c] Submitted process &amp;gt; foo (task: 11 - rna,5)&#xA;Input dna: [dna, 3]&#xA;[a3/6d80d8] Submitted process &amp;gt; foo (task: 14 - rna,7)&#xA;[ac/acdb5a] Submitted process &amp;gt; foo (task: 15 - rna,1)&#xA;Param 7: [rna, 7]&#xA;Input dna: [dna, 1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2021-04-14T14:00:36.763" LastActivityDate="2021-04-14T14:00:36.763" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15728" PostTypeId="1" AcceptedAnswerId="15731" CreationDate="2021-04-13T14:58:06.350" Score="0" ViewCount="625" Body="&lt;p&gt;I am trying to analyze the scRNAseq data from &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0092867418307931?via%3Dihub&quot; rel=&quot;nofollow noreferrer&quot;&gt;this study&lt;/a&gt;. In their Method section they write:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The accession number for the RNA and DNaseq data reported in this&#xA;paper is GEO:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE116237&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSE116237&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;When I go ahead and try to pull this data in R using &lt;code&gt;GEOquery&lt;/code&gt;, I am faced with a data frame with no features (i.e. no genes). The following code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(&amp;quot;GEOquery&amp;quot;)&#xA;gse &amp;lt;- getGEO(&amp;quot;GSE116237&amp;quot;)&#xA;eset1 &amp;lt;- gse[[1]]&#xA;eset2 &amp;lt;- gse[[2]]&#xA;&#xA;print(dim(eset1))&#xA;print(dim(eset2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;yields&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Features  Samples &#xA;       0      184 &#xA;Features  Samples &#xA;       0      681 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also didn't manage to download the data manually; I get an error message when trying to unzip the &lt;code&gt;.zip&lt;/code&gt; files or &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Traces/study/?acc=PRJNA477841&amp;amp;o=acc_s%3Aa&quot; rel=&quot;nofollow noreferrer&quot;&gt;get redirected to SRA Run Selector&lt;/a&gt; from which I also didn't manage to figure out how to download stuff.&lt;/p&gt;&#xA;&lt;p&gt;Am I missing something or are these files simply corrupted? The paper is published in Cell, so I would assume this data is, to some extend, legit...&lt;/p&gt;&#xA;" OwnerUserId="6339" LastActivityDate="2021-04-13T19:03:08.507" Title="Why am I getting empty expression data from GEO?" Tags="|scrnaseq|gene-expression|sratoolkit|geoquery|geo|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15729" PostTypeId="2" ParentId="15723" CreationDate="2021-04-13T15:27:19.050" Score="2" Body="&lt;p&gt;First of all, RNASeq is extremely sensitive to batch effects.  Are these matched controls processed at the same time as the tumors?  IF they are not, lots of your differences will be caused by batch effect, and not cancer.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I believe the best strategy is to merge the FPKM columns and perform&#xA;correlation analyses of my gene of interest (GOI) versus all other&#xA;genes in both cancer and normal tissues.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What you propose isn't publishable.  Use the software that people have been using and validating for years.  Use either DESeq2 or EdgeR, or limma if you don't have access to raw gene counts.&lt;/p&gt;&#xA;&lt;p&gt;If the underlying counts are low, the correlation won't mean much, because your real values could be quite off. If your gene has 2 counts in one sample, and 3 counts in another, that does not mean it really correlates with a gene that has 500 reads in the one sample and 750 in the other. I don't know that you can trick your way around it; you need more reads, or more samples for more power.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastEditorUserId="73" LastEditDate="2021-05-14T09:30:53.883" LastActivityDate="2021-05-14T09:30:53.883" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15730" PostTypeId="6" CreationDate="2021-04-13T17:14:45.127" Score="0" Body="&lt;h1&gt;Ram RS&lt;/h1&gt;&#xA;&lt;p&gt;I've been an active user on SE for many years, and have been active on bioinformatics SE since its conception. I love the grunt work of getting posts up to spec (formatting, asking for more information etc.) so contributors can help askers more easily.&lt;/p&gt;&#xA;&lt;p&gt;I spent a few formative years of my career as a software developer/architect/educator, so helping people understand how computers speak and work is one of my strengths. As someone who moved from software to bioinformatics, I am also aware of the many quirks that bioinformatics software takes for granted, and I strive to introduce more code-data separation/compartmentalization wherever I can.&lt;/p&gt;&#xA;&lt;p&gt;I have been a moderator on BioStars for a few years now, and I am happy to contribute more here along the same lines - both on subject matter when I can and on moderating content/user activity when required. While I am welcoming to new users, I do expect a decent level of effort from all posts. I do have a bit of a reputation as the bad cop in the good cop-bad cop dynamic as I insist on quality, effort and self learning.&lt;/p&gt;&#xA;&lt;p&gt;And speaking of cops, I'm more than happy to take @Kamil up on the donuts offer (as long as they're eggless!)&lt;/p&gt;&#xA;" OwnerUserId="650" LastEditorUserId="650" LastEditDate="2021-04-13T17:14:45.127" LastActivityDate="2021-04-13T17:14:45.127" CommentCount="0" CommunityOwnedDate="2021-04-13T17:14:45.127" ContentLicense="CC BY-SA 4.0" />
  <row Id="15731" PostTypeId="2" ParentId="15728" CreationDate="2021-04-13T19:03:08.507" Score="1" Body="&lt;p&gt;I am sure the data is legit, you are just approaching it incorrectly. getGEO is an application for microarray data, not for digital count data such as (sc)RNA-seq, therefore what you aim to do is simply not possible by design. Unless you want to start from the raw reads why not taking the at the file named &lt;code&gt;GSE116237_scRNAseq_expressionMatrix.txt.gz&lt;/code&gt; provided at the bottom of &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE116237&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE116237&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-04-13T19:03:08.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15732" PostTypeId="1" AcceptedAnswerId="15833" CreationDate="2021-04-14T09:22:52.730" Score="2" ViewCount="1265" Body="&lt;p&gt;I am experiencing timeout problems when downloading the NCBI nr preformatted blast database using the update_blastdb script (version 504861).&lt;/p&gt;&#xA;&lt;p&gt;I run the script with the following paramters&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;update_blastdb --decompress --passive --verbose nr&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and I get the following error message (in verbose mode)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Downloading nr (45 volumes) ...&#xA;Downloading nr.00.tar.gz...Net::FTP=GLOB(0x5610fb59b8f8)&amp;gt;&amp;gt;&amp;gt; PASV&#xA;Net::FTP=GLOB(0x5610fb59b8f8)&amp;lt;&amp;lt;&amp;lt; 227 Entering Passive Mode (165,112,9,229,195,144).&#xA;Net::FTP=GLOB(0x5610fb59b8f8)&amp;gt;&amp;gt;&amp;gt; RETR nr.00.tar.gz&#xA;Net::FTP=GLOB(0x5610fb59b8f8)&amp;lt;&amp;lt;&amp;lt; 150 Opening BINARY mode data connection for nr.00.tar.gz (18745730730 bytes)&#xA;Net::FTP: Net::Cmd::getline(): timeout at /usr/share/perl/5.26/Net/FTP/dataconn.pm line 82.&#xA;Unable to close datastream at /usr/bin/update_blastdb line 202.&#xA;Net::FTP=GLOB(0x5610fb59b8f8)&amp;gt;&amp;gt;&amp;gt; PASV&#xA;Net::FTP: Net::Cmd::getline(): unexpected EOF on command channel: Connection reset by peer at /usr/bin/update_blastdb line 203.&#xA;Failed to download nr.00.tar.gz.md5!&#xA;Net::FTP: Net::Cmd::_is_closed(): unexpected EOF on command channel: Connection reset by peer at /usr/bin/update_blastdb line 101.&#xA;Net::FTP: Net::Cmd::_is_closed(): unexpected EOF on command channel: Connection reset by peer at /usr/bin/update_blastdb line 101.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The timeout happens after ~35 minutes and a file that is approximately 18GB big is being downloaded, which matches the expected filesize. The checksum file (nr.00.tar.gz.md5) is not downloaded. So I'm not sure which of the two files is actually the problem.&lt;/p&gt;&#xA;&lt;p&gt;I tested downloading the nt database and everything seems to work fine, so I don't think the script is the problem. For comparison, this is the nt download output for the first file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Downloading nt.00.tar.gz...Net::FTP=GLOB(0x562575a73168)&amp;gt;&amp;gt;&amp;gt; PASV&#xA;Net::FTP=GLOB(0x562575a73168)&amp;lt;&amp;lt;&amp;lt; 227 Entering Passive Mode (165,112,9,229,196,51).&#xA;Net::FTP=GLOB(0x562575a73168)&amp;gt;&amp;gt;&amp;gt; RETR nt.00.tar.gz&#xA;Net::FTP=GLOB(0x562575a73168)&amp;lt;&amp;lt;&amp;lt; 150 Opening BINARY mode data connection for nt.00.tar.gz (4065912989 bytes)&#xA;Net::FTP=GLOB(0x562575a73168)&amp;lt;&amp;lt;&amp;lt; 226 Transfer complete&#xA;Net::FTP=GLOB(0x562575a73168)&amp;gt;&amp;gt;&amp;gt; PASV&#xA;Net::FTP=GLOB(0x562575a73168)&amp;lt;&amp;lt;&amp;lt; 227 Entering Passive Mode (165,112,9,229,195,107).&#xA;Net::FTP=GLOB(0x562575a73168)&amp;gt;&amp;gt;&amp;gt; RETR nt.00.tar.gz.md5&#xA;Net::FTP=GLOB(0x562575a73168)&amp;lt;&amp;lt;&amp;lt; 150 Opening BINARY mode data connection for nt.00.tar.gz.md5 (47 bytes)&#xA;Net::FTP=GLOB(0x562575a73168)&amp;lt;&amp;lt;&amp;lt; 226 Transfer complete&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="4441" LastActivityDate="2021-04-30T07:40:38.877" Title="Timeout when downloading the ncbi nr blast database" Tags="|blast|phylogenetics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15736" PostTypeId="2" ParentId="15720" CreationDate="2021-04-15T08:38:59.767" Score="2" Body="&lt;p&gt;There's a few problems with your code, first, when you do &lt;code&gt;Read10X()&lt;/code&gt; it returns you a sparse matrix, and you need to put this into a &lt;code&gt;Seurat&lt;/code&gt; object with meta data, before doing the integration.&lt;/p&gt;&#xA;&lt;p&gt;So for example, i create some example data that is similar to your output from &lt;code&gt;Read10X()&lt;/code&gt; :&lt;/p&gt;&#xA;&lt;p&gt;library(Matrix)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;epithelial = Matrix(matrix(rnbinom(100000,mu=200,size=1),ncol=50),sparse=TRUE)&#xA;rownames(epithelial) = paste0(&amp;quot;gene&amp;quot;,1:2000)&#xA;colnames(epithelial) = paste0(&amp;quot;e&amp;quot;,1:50)&#xA;&#xA;habermann = epithelial + rnbinom(100000,mu=20,size=1)&#xA;colnames(habermann) = paste0(&amp;quot;h&amp;quot;,1:50)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then convert to a seurat object:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data.list=c(epithelial = CreateSeuratObject(epithelial),&#xA;            habermann = CreateSeuratObject(habermann))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then find variable features:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data.list &amp;lt;- lapply(data.list, FUN = function(x) {&#xA;x &amp;lt;- NormalizeData(x)&#xA;x &amp;lt;- FindVariableFeatures(x, selection.method = &amp;quot;vst&amp;quot;, nfeatures = 2000)&#xA;})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Should look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$epithelial&#xA;An object of class Seurat &#xA;2000 features across 50 samples within 1 assay &#xA;Active assay: RNA (2000 features, 2000 variable features)&#xA;&#xA;$habermann&#xA;An object of class Seurat &#xA;2000 features across 50 samples within 1 assay &#xA;Active assay: RNA (2000 features, 2000 variable features)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now perform integration, below I have to reduce &lt;code&gt;k.filter&lt;/code&gt; because I have very little cells in this example. In your case, you can simply use the default settings.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;features &amp;lt;- SelectIntegrationFeatures(object.list = data.list)&#xA;anchors= FindIntegrationAnchors( data.list,max.features = 200,&#xA;k.filter=50,k.anchor = 3,verbose = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2021-04-15T08:38:59.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15737" PostTypeId="1" AcceptedAnswerId="15748" CreationDate="2021-04-15T08:47:32.150" Score="1" ViewCount="193" Body="&lt;p&gt;I have got two files large files like this in Tab-delimited format, trying to merge in R, do we need to sort the file before merging?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; otu &amp;lt;- read.table(&amp;quot;asv.txt&amp;quot;, sep=&amp;quot;\t&amp;quot;, header=T, blank.lines.skip=F, check.names=F)&#xA;&amp;gt; head(otu)&#xA;                ID                   Soil.9.S42 Soil-27.S33.L001 Soil-45.S54.L001&#xA;1 d2ec9f3b77975c0f457e4b7413b217ff         34               51               84&#xA;2 3147790f0d5a78316fb9dd64f53b9473         28               32               52&#xA;3 97aecc1f35cc1f50db507ad71dd22367         33                0                7&#xA;4 bfad6370d28182cc6304844e9bec7fb6        545              237              127&#xA;5 5fa2a987221a1d9ca416148570c18086          0                0                0&#xA;6 8e60d301122d7aa359eb6b0b00f37f62        124              143              128&#xA;&#xA;&#xA;&amp;gt; taxa &amp;lt;- read.table(&amp;quot;taxonomy.txt&amp;quot;, sep=&amp;quot;\t&amp;quot;, header=F ,stringsAsFactors=F,quote=&amp;quot;&amp;quot;)&#xA;&amp;gt; head(taxa)&#xA;                  ID                     V2             V3                  V4&#xA;d2ec9f3b77975c0f457e4b7413b217ff Bacteria  Cyanobacteria    Oxyphotobacteria&#xA;3147790f0d5a78316fb9dd64f53b9473 Bacteria Proteobacteria Alphaproteobacteria&#xA;97aecc1f35cc1f50db507ad71dd22367 Bacteria Proteobacteria Gammaproteobacteria&#xA;bfad6370d28182cc6304844e9bec7fb6 Bacteria Proteobacteria Alphaproteobacteria&#xA;5fa2a987221a1d9ca416148570c18086 Bacteria Proteobacteria Alphaproteobacteria&#xA;8e60d301122d7aa359eb6b0b00f37f62 Bacteria Proteobacteria Deltaproteobacteria&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now having issue with sorting file but merge command work perfectly;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;newotu &amp;lt;- otu[order(ID),]&#xA;Error in order(ID) : object 'ID' not found&#xA;&amp;gt; total &amp;lt;- merge(otu,taxa,by=&amp;quot;ID&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thanks for your help.&lt;/p&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="1008" LastEditDate="2021-04-15T11:16:58.117" LastActivityDate="2021-04-17T19:52:32.987" Title="how to sort and join two files based on First Column Id" Tags="|r|bash|linux|" AnswerCount="1" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="15738" PostTypeId="1" CreationDate="2021-04-15T11:40:59.490" Score="3" ViewCount="55" Body="&lt;p&gt;I'm looking for databases and/or raw data of protein mass spectrometry experiments.&lt;/p&gt;&#xA;&lt;p&gt;After reviewing &lt;a href=&quot;http://xlinkdb.gs.washington.edu/xlinkdb/&quot; rel=&quot;nofollow noreferrer&quot;&gt;xlinkdb&lt;/a&gt; and &lt;a href=&quot;https://proxl-ms.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Proxl&lt;/a&gt;, I was curious to learn about other alternatives and options that allow users to run the database locally and browse raw experimental data. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/protein/?term=xl-ms&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI&lt;/a&gt; and &lt;a href=&quot;http://proteomecentral.proteomexchange.org/cgi/GetDataset&quot; rel=&quot;nofollow noreferrer&quot;&gt;proteomexchange.org&lt;/a&gt; don't seem to have enough data either.&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2021-07-09T13:33:52.927" LastActivityDate="2021-07-09T13:33:52.927" Title="How to access XL-MS raw data and databases?" Tags="|protein-protein-interaction|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15739" PostTypeId="1" AcceptedAnswerId="15743" CreationDate="2021-04-15T13:48:12.513" Score="3" ViewCount="2085" Body="&lt;p&gt;How should I do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;if ($(params.aligner) == &amp;quot;bowtie&amp;quot;) {&#xA;    align_bowtie2(get_fq_info.out.fq_info)&#xA;}&#xA;else {&#xA;    align_bwa(get_fq_info.out.fq_info)&#xA;}&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2970" LastActivityDate="2021-04-16T03:15:39.450" Title="Use conditional in workflow in Nextflow dsl2" Tags="|nextflow|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15740" PostTypeId="1" AcceptedAnswerId="16322" CreationDate="2021-04-15T14:01:58.673" Score="3" ViewCount="768" Body="&lt;p&gt;I have a csv file which describes number of chunks per chromosome, like;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chrom,chunks&#xA;chr1,&amp;quot;CHUNKS=([1]=&amp;quot;&amp;quot;0e6 3e6&amp;quot;&amp;quot; [2]=....)&amp;quot;&#xA;chr2,&amp;quot;CHUNKS=([1]=&amp;quot;&amp;quot;0e6 3e6&amp;quot;&amp;quot; [2]=....)&amp;quot;&#xA;chr3,&amp;quot;CHUNKS=([1]=&amp;quot;&amp;quot;0e6 3e6&amp;quot;&amp;quot; [2]=....)&amp;quot;&#xA;.....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I could crate a channel as follows,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;channel&#xA;    .fromPath('chunks3.csv')&#xA;    .splitCsv(header:true)&#xA;    .map{row -&amp;gt; tuple(row.chrom,row.chunks)}&#xA;    .set{chunk_ch}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It gives me chromosome with respective chunk definition.&lt;/p&gt;&#xA;&lt;p&gt;But when I try to join this channel with another channel, it does not work.&lt;/p&gt;&#xA;&lt;p&gt;For example I have a channel &amp;quot;phased&amp;quot;. It is a output channel created from a process.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;channel&#xA;    .fromFilePairs('/some/path/genetic_map_chr*_combined_b37.txt',size:1)&#xA;    .map {group_key, file_list -&amp;gt; tuple(group_key.replaceFirst(/^*genetic_map_/,&amp;quot;&amp;quot;),file_list.first())}&#xA;    .set {genetic_map}&#xA;&#xA;channel&#xA;    .fromFilePairs('/some/path/*hrc.r1-1.ega.grch37.chr*.{hap.gz,legend.gz,samples}',size:3)&#xA;    .map {group_key,file_list -&amp;gt; tuple(group_key.replaceAll(/^.*.hrc\.r1-1\.ega\.grch37\.|\.haplotypes\.noibd|\.haplotypes/,&amp;quot;&amp;quot;),file_list)}&#xA;    .set {ref_panel}&#xA;&#xA;qc1_chr&#xA;    .map{chrom,bfiles -&amp;gt; tuple(chrom,bfiles.findAll{it.baseName.endsWith('step10')})}&#xA;    .join(ref_panel).join(genetic_map)&#xA;    .into {shape_in;imp_in}&#xA;&#xA;process phasing {&#xA;    publishDir params.out, mode:'copy'&#xA;    &#xA;    input:&#xA;    tuple val(chrom), path(input_bed), path(input_ref), path(input_map) from shape_in&#xA;    &#xA;    output:&#xA;    tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chrom}&quot;),path(&quot;$&lt;/span&gt;{chrom}.*&amp;quot;) into phased&#xA;&#xA;    script:&#xA;    def (bed, bim, fam) = input_bed&#xA;    def (haplotype, legend, sample) = input_ref&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    shapeit --thread &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${task.cpus}&quot; --input-bed &quot;$&lt;/span&gt;{bed}&amp;quot; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${bim}&quot; &quot;$&lt;/span&gt;{fam}&amp;quot; --input-map &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${input_map}&quot; --input-ref &quot;$&lt;/span&gt;{haplotype}&amp;quot; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${legend}&quot; &quot;$&lt;/span&gt;{sample}&amp;quot; --duohmm --&#xA;    output-max &amp;quot;${chrom}.step10&amp;quot; --seed 54321&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which emits something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;phased.view()&#xA;[chr1,[file1]&#xA;chr2,[file2]&#xA;chr3,[file3]....]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If I try &lt;code&gt;chunk_ch.join(phased)&lt;/code&gt; it gives nothing.&lt;/p&gt;&#xA;&lt;p&gt;Any help?&lt;/p&gt;&#xA;" OwnerUserId="3839" LastEditorUserId="3252" LastEditDate="2021-07-06T10:03:26.437" LastActivityDate="2021-07-07T15:24:11.227" Title="splitCsv map with text, join with another channel, nextflow" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15741" PostTypeId="2" ParentId="15739" CreationDate="2021-04-15T18:28:48.163" Score="2" Body="&lt;p&gt;Is this inside the &lt;code&gt;main.nf&lt;/code&gt; file or inside the &lt;code&gt;process.nf&lt;/code&gt; file? Anyway, the &lt;code&gt;param.anything&lt;/code&gt; variable should already be available in the &lt;code&gt;nextflow&lt;/code&gt; context when executed, thus, you could simply do:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;if (params.aligner == &amp;quot;bowtie&amp;quot;) {&#xA;    align_bowtie2(get_fq_info.out.fq_info)&#xA;} else {&#xA;    align_bwa(get_fq_info.out.fq_info)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Check out this &lt;a href=&quot;https://github.com/fmalmeida/bacannot/blob/master/main.nf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;main.nf&lt;/code&gt;&lt;/a&gt; of a nextflow pipeline of mine, it contains some working if/else snippets.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Here it is &lt;a href=&quot;https://stackoverflow.com/questions/50443545/conditional-pipeline-in-nextflow&quot;&gt;another StackOverflow question on conditional&lt;/a&gt; nextflow pipelines that might help.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Hope it helps.&lt;/p&gt;&#xA;" OwnerUserId="6767" LastEditorUserId="6767" LastEditDate="2021-04-15T18:33:55.127" LastActivityDate="2021-04-15T18:33:55.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15742" PostTypeId="2" ParentId="15732" CreationDate="2021-04-15T19:43:27.457" Score="1" Body="&lt;p&gt;Same here. I used axel to download it a week ago, but fail to get finished. Also, the downloaded taxa zipped file (60Mb) doesn't match md5sum. Apparently, they update the FTP files every 30 minutes or 1 hour.&lt;/p&gt;&#xA;" OwnerUserId="9975" LastActivityDate="2021-04-15T19:43:27.457" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15743" PostTypeId="2" ParentId="15739" CreationDate="2021-04-16T03:15:39.450" Score="2" Body="&lt;p&gt;You can move that into your &lt;a href=&quot;https://www.nextflow.io/docs/latest/dsl2.html#workflow-definition&quot; rel=&quot;nofollow noreferrer&quot;&gt;workflow definition&lt;/a&gt;. There's an example in the docs of how best to use &lt;a href=&quot;https://www.nextflow.io/docs/latest/dsl2.html#workflow-parameters&quot; rel=&quot;nofollow noreferrer&quot;&gt;workflow parameters&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A workflow component can access any variable and parameter defined in&#xA;the outer scope:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params.data = '/some/data/file'&#xA;&#xA;workflow my_pipeline {&#xA;    if( params.data )&#xA;        bar(params.data)&#xA;    else&#xA;        bar(foo()) }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Your example workflow would look like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow {&#xA;&#xA;    fq_info = get_fq_info.out.fq_info&#xA;&#xA;    if( params.aligner == &amp;quot;bowtie&amp;quot; ) {&#xA;        align_bowtie2( fq_info )&#xA;    }&#xA;    else {&#xA;        align_bwa( fq_info )&#xA;    }&#xA;&#xA;    ...&#xA;}&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2021-04-16T03:15:39.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15744" PostTypeId="2" ParentId="15717" CreationDate="2021-04-16T12:09:51.217" Score="1" Body="&lt;p&gt;Your title says &lt;code&gt;holistic&lt;/code&gt;. This is a tad problematic as there's layers upon layers. Say, post-translation regulation, inhibiting metabolites, interacting protein etc. That is why when talking of an enzyme inhibitor, at the biochemical level one speaks in terms of k_i (inhibition constant), while at the cellular level (&amp;quot;holistic&amp;quot;) one talks of IC50.&lt;/p&gt;&#xA;&lt;p&gt;This could be a biocatalysis/synthetic biology problem (make an enzyme better) or translational medicine problem (why is this mutation pathogenic).&lt;/p&gt;&#xA;&lt;p&gt;In terms of &lt;em&gt;in silico&lt;/em&gt; enzyme design, one can have all sorts of issues that need to be addressed:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;catalysis&lt;/li&gt;&#xA;&lt;li&gt;side reactions&lt;/li&gt;&#xA;&lt;li&gt;stability&lt;/li&gt;&#xA;&lt;li&gt;aggregation&lt;/li&gt;&#xA;&lt;li&gt;solubility&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Also there's the expression problems, which may be due to subpar promoter, insufficient cofactors and poor codon usage.&lt;/p&gt;&#xA;&lt;p&gt;For clinical variant assessment, a variant may:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;destabilise leaving to degradation&lt;/li&gt;&#xA;&lt;li&gt;remove a post-translational modification&lt;/li&gt;&#xA;&lt;li&gt;disrupt the active site, affecting catalysis&lt;/li&gt;&#xA;&lt;li&gt;lose a binding partner&lt;/li&gt;&#xA;&lt;li&gt;result in incorrect localisation&lt;/li&gt;&#xA;&lt;li&gt;result in solubility issues&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So force-field scores are not the sole solution. Furthermore, a forcefield method gives one a ∆∆G (difference in relative Gibbs free energy), not a K_M (Michaelis constant) and a k_cat (catalytic rate), although it is somewhat possible to calculate these, or a relative variant of these.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;a variant with just 1 or 2 variants&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;A variant protein can have one or two &lt;em&gt;mutations&lt;/em&gt; (e.g. G10R) in its sequence. A &lt;em&gt;mutant&lt;/em&gt; and a &lt;em&gt;variant&lt;/em&gt; are near synonyms, but variant is generally intended to be a mutant in a population.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Does these application be used for such purpose?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;For enzyme design MD simulations are not generally the first step. Rosetta pmut_scan for example is a nice starting step as the mutational landscape of a protein is N^19, while a MD simulation takes weeks and 6+ core per single variant.&#xA;This calculates only an increase in stability (relative difference in Gibbs folding energy), not reactivity. For that you can use a transition state, whose results are inaccurate, but majorly affected by the set-up, i.e. junk-in junk-out plays a big part. There's a saying 'a week in the lab saves you an hour at the library': this is backwards for sarcasm as it happens too often, namely that is background research is always a lifesaver. An QM-MD simulation on the wild type reaction definitely can help in elucidating things if it is not clear.&lt;/p&gt;&#xA;&lt;h2&gt;Online tools&lt;/h2&gt;&#xA;&lt;p&gt;Generally, however, there are resources out there to do many stability calculations already.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;SynBio&lt;/strong&gt; If you are interested in picking a residue or two to make an enzyme more stable, without resorting to a thermophilic enzyme, &lt;a href=&quot;https://pross.weizmann.ac.il/step/pross-terms/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PROSS&lt;/a&gt; is a very nice tool.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Clinical&lt;/strong&gt; (Caveat: If your variant is a variant that affects you, a friend or a relative, you/they should speak to a genetic councillor and not do research on it yourself) There are several tools that can describe a variant at the protein level. Do remember that ∆∆G is a potential so more negative is more stable —except in SDM where the score is inverted.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.sbg.bio.ic.ac.uk/%7Emissense3d/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Missense3D&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://venus.sgc.ox.ac.uk&quot; rel=&quot;nofollow noreferrer&quot;&gt;Venus&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://miscast.broadinstitute.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Miscast&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www-cryst.bioc.cam.ac.uk/%7Esdm/sdm.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;SDM&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2021-04-16T13:35:12.523" LastActivityDate="2021-04-16T13:35:12.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15745" PostTypeId="2" ParentId="15692" CreationDate="2021-04-16T13:54:57.303" Score="3" Body="&lt;p&gt;&lt;em&gt;Note: I will use SBOL3 in my descriptions; the answer can be equivalently translated to SBOL2&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;At present, SBOL doesn't have any explicit recommendations for how to represent these types of timing parameters. This is because there is not yet a clear scientific consensus on whether or not such parameters are a good way to represent genetic circuits.&lt;/p&gt;&#xA;&lt;p&gt;That said, one of the advantages of using SBOL is that its extensible RDF-based model allows additional information like this to be added into a model. Two good ways of doing so are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If the parameters are simple and can stand alone as measurements, then they can be added to designs as a &lt;code&gt;Measure&lt;/code&gt; object with appropriate units and type. For example, if you have biological devices with a 90 minute propagation delay, then on the &lt;code&gt;Component&lt;/code&gt; representing the device you can add a &lt;code&gt;Measure&lt;/code&gt; with numerical value 90, units of &lt;code&gt;om:minute&lt;/code&gt;.  You'd likely want two types, one of &lt;code&gt;SBO:0000225 (Delay)&lt;/code&gt;, and a custom type for your own notion of propagation delay, since I don't think that's included in any ontology yet.&lt;/li&gt;&#xA;&lt;li&gt;If the parameters are more complex and/or inter-related, e.g., a full data-sheet, then you'd likely want to instead create one or more extension classes that are used to represent the information. A library that makes this easy is &lt;a href=&quot;https://github.com/SynBioDex/sbol_factory&quot; rel=&quot;nofollow noreferrer&quot;&gt;sbol_factory&lt;/a&gt;, though one can do it with any of the SBOL libraries.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This becomes more complicated if you want to talk about behavior in different environmental contexts, but the basic principles are still the same.&lt;/p&gt;&#xA;&lt;p&gt;Finally, as for SBOL visual: again, there's no explicit representation, but the model is extensible, and this information can be added as any sort of text or custom glyph as long as it doesn't conflict with ones already in the standard.&lt;/p&gt;&#xA;" OwnerUserId="4869" LastActivityDate="2021-04-16T13:54:57.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15746" PostTypeId="1" AcceptedAnswerId="15755" CreationDate="2021-04-16T18:53:35.937" Score="1" ViewCount="785" Body="&lt;p&gt;From the bedtools intersect man (for version 2.30.0):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-a BAM/BED/GFF/VCF file “A”. Each feature in A is compared to B in search of overlaps. &#xA;   Use “stdin” if passing A with a UNIX pipe.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The stdin part, however, does not apply to version 2.27.1. The help on this version gives no advice for piping.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bam mem ref fq1.fastq fq2.fastq | samtools view -b | bedtools intersect -a stdin -b blacklist.bed -v &amp;gt; align_black.sam&#xA;Error: unable to open file or unable to determine types for file stdin&#xA;&#xA;- Please ensure that your file is TAB delimited (e.g., cat -t FILE).&#xA;- Also ensure that your file has integer chromosome coordinates in the&#xA;  expected columns (e.g., cols 2 and 3 for BED).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also tried:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(map to bam) | bedtools intersect -a &amp;quot;stdin&amp;quot; -b blacklist.bed -v &amp;gt; align_black.sam&#xA;(map to bam) | bedtools intersect -a -b blacklist.bed -v &amp;gt; align_black.sam&#xA;(map to bam) | bedtools intersect -b blacklist.bed -v &amp;gt; align_black.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But these don't work either, either for the same or a different reason. Here is a site that tells me to use the above syntax: &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/example-usage.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedtools.readthedocs.io/en/latest/content/example-usage.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What is the solution to this if I want to pipe everything together for my users?&lt;/p&gt;&#xA;" OwnerUserId="9521" LastActivityDate="2021-05-09T05:03:14.517" Title="Unexpected error using &quot;stdin&quot; in bedtools intersect as a piped command" Tags="|bedtools|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15747" PostTypeId="2" ParentId="15746" CreationDate="2021-04-16T21:48:40.363" Score="1" Body="&lt;p&gt;Here's a way to skip conversion to BAM and pipe between tools, which performs the same set operations:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ bam mem ref fq1.fastq fq2.fastq | sam2bed - | bedops -e 1 - blacklist.bed &amp;gt; blacklisted_reads.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With this toolkit, the hyphen (&lt;code&gt;-&lt;/code&gt;) follows a Unix convention of serving as a placeholder for stdin.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2021-04-16T21:48:40.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15748" PostTypeId="2" ParentId="15737" CreationDate="2021-04-16T22:00:56.770" Score="3" Body="&lt;p&gt;You should check &lt;code&gt;left_join()&lt;/code&gt; method in &lt;code&gt;dplyr&lt;/code&gt; package. In this way you do not need to sort the data frames, as the function automatically take care of this.&#xA;For example: &lt;code&gt;merged_df &amp;lt;- left_join(otu, taxa, by = &amp;quot;ID&amp;quot;)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2021-04-16T22:00:56.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15749" PostTypeId="1" CreationDate="2021-04-17T06:52:08.010" Score="1" ViewCount="1374" Body="&lt;p&gt;I have 3 VCF files.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A1.vcf&#xA;A2.vcf&#xA;A3.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to get the common SNPs that are present in all these three files. And output must be in vcf format.&lt;/p&gt;&#xA;&lt;p&gt;output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;common_A.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6504" LastEditorUserId="3694" LastEditDate="2021-04-18T13:07:20.413" LastActivityDate="2023-05-23T16:47:32.210" Title="Find common SNPs in multiple VCF files" Tags="|vcf|snp|variants|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15751" PostTypeId="6" CreationDate="2021-04-17T13:37:28.813" Score="0" Body="&lt;h1&gt;gringer&lt;/h1&gt;&#xA;&lt;p&gt;Hi everyone, my educational background is in computer science, biology, and mathematics, and I have worked for many years as a professional bioinformatician, trying to encourage more people to dip their feet into the ocean of bioinformatics.&lt;/p&gt;&#xA;&lt;p&gt;I would love to join the moderation team for Bioinformatics Stack Exchange. I was part of the team of early adopters on this site, bringing the site from commitment (advertising) through to private and now public beta.&lt;/p&gt;&#xA;&lt;p&gt;Other sites I visit for bioinfomatics:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://twitter.com/gringene_bio&quot; rel=&quot;nofollow noreferrer&quot;&gt;Twitter&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://community.nanoporetech.com/profile/david-eccles&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nanopore Community Forums&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.reddit.com/user/gringer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reddit&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://gringer.gitlab.io/presentation-notes/2018/10/25/nanopore-notes/#email--message-discussions&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitLab Pages&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I'm also apparently popular enough here that someone &lt;a href=&quot;https://bioinformatics.meta.stackexchange.com/questions/243/why-does-gringer-have-a-wiki-tag&quot;&gt;added a wiki tag for me&lt;/a&gt;!&lt;/p&gt;&#xA;&lt;p&gt;I look forward to seeing how this community will develop and grow to become the primary source of help for bioinformatics in the future.&lt;/p&gt;&#xA;&lt;p&gt;Ngā mihi nui [Māori: many thanks],&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;David Eccles (gringer)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-04-17T13:37:28.813" LastActivityDate="2021-04-17T13:37:28.813" CommentCount="0" CommunityOwnedDate="2021-04-17T13:37:28.813" ContentLicense="CC BY-SA 4.0" />
  <row Id="15755" PostTypeId="2" ParentId="15746" CreationDate="2021-04-18T10:34:45.997" Score="3" Body="&lt;p&gt;With v2.27.1 (or any version I ever used) instead of &lt;code&gt;stdin&lt;/code&gt; use &lt;code&gt;-&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(...) | bedtools intersect -a - -b blacklist.bed -v &amp;gt; out.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Be aware that the output of this is a BAM, not SAM file. It would be easier though to simply make a list of regions you want to keep, e.g. the complement of the backlist file with the entire genome (bedtools complement), and then use &lt;code&gt;samtools view -L&lt;/code&gt; option to only keep overlapping reads. That saves you the run through &lt;code&gt;bedtools&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2021-04-18T13:05:48.073" LastActivityDate="2021-04-18T13:05:48.073" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15757" PostTypeId="2" ParentId="15746" CreationDate="2021-04-19T14:04:09.990" Score="1" Body="&lt;p&gt;While I have chosen ATpoint as the official answer, another possible syntax to use is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bedtools intersect -a stdin -b blacklist.bed -v &amp;lt; `(map to bam)` &amp;gt; align_black.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The main issue with this syntax is that it obfuscates the direction of flow for the pipeline, piping step 2 after step 1 is better in the official answer.&lt;/p&gt;&#xA;" OwnerUserId="9521" LastEditorUserId="73" LastEditDate="2021-05-09T05:03:14.517" LastActivityDate="2021-05-09T05:03:14.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15758" PostTypeId="2" ParentId="2341" CreationDate="2021-04-19T18:56:58.603" Score="0" Body="&lt;p&gt;You can use &lt;code&gt;agat_sp_extract_sequences.pl&lt;/code&gt; from &lt;a href=&quot;https://github.com/NBISweden/AGAT&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGAT&lt;/a&gt;.&#xA;See here for all option: &lt;a href=&quot;https://agat.readthedocs.io/en/latest/tools/agat_sp_extract_sequences.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://agat.readthedocs.io/en/latest/tools/agat_sp_extract_sequences.html&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Below example of use:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vygC4.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vygC4.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2486" LastActivityDate="2021-04-19T18:56:58.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15759" PostTypeId="2" ParentId="14328" CreationDate="2021-04-19T22:08:00.103" Score="1" Body="&lt;p&gt;Bit of an old post sorry, but I think you might be getting a little confused here. The nf-core pipelines come with their own software already, so you don't need to install or bootstrap optitype yourself. If it's used in the pipeline, it comes in the pipeline container(s) (or conda environment or whatever packaging system you use). So you should never have to touch &lt;code&gt;docker://fred2/optitype&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;When you run the pipeline with the &lt;code&gt;-profile singularity&lt;/code&gt; argument, it will automatically pull the image from Docker Hub and convert to Singularity for you &lt;em&gt;(some of the newer pipelines pull pre-build Singularity images directly)&lt;/em&gt;. You can also use the &lt;code&gt;nf-core download hlatyping --singularity&lt;/code&gt; command from the helper tools to pull the image ahead of time.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Docker images for this pipeline: &lt;a href=&quot;https://hub.docker.com/r/nfcore/hlatyping&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hub.docker.com/r/nfcore/hlatyping&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Documentation for this pipeline: &lt;a href=&quot;https://nf-co.re/hlatyping/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nf-co.re/hlatyping/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;nf-core/tools download docs: &lt;a href=&quot;https://nf-co.re/tools/#downloading-pipelines-for-offline-use&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nf-co.re/tools/#downloading-pipelines-for-offline-use&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I hope this helps! Come and &lt;a href=&quot;https://nf-co.re/join&quot; rel=&quot;nofollow noreferrer&quot;&gt;join the nf-core Slack organisation&lt;/a&gt; if you need more help. We have a channel for each pipeline, including &lt;code&gt;#hlatyping&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="224" LastActivityDate="2021-04-19T22:08:00.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15760" PostTypeId="2" ParentId="15749" CreationDate="2021-04-20T00:22:17.707" Score="3" Body="&lt;p&gt;A potential solution is to compress and index the files using &lt;a href=&quot;http://www.htslib.org/doc/bgzip.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bgzip&lt;/a&gt; and &lt;a href=&quot;http://www.htslib.org/doc/tabix.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tabix&lt;/a&gt;, then use &lt;a href=&quot;https://samtools.github.io/bcftools/bcftools.html#isec&quot; rel=&quot;nofollow noreferrer&quot;&gt;bcftools isec&lt;/a&gt; to find SNPs common to all three files, e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for files in A1.vcf A2.vcf A3.vcf&#xA;  do&#xA;  bgzip &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$files&quot;&#xA;  tabix &quot;$&lt;/span&gt;files&amp;quot;&#xA;done&#xA;&#xA;bcftools isec -n=3 -c all -o all_common_SNPs.vcf A1.vcf.gz A2.vcf.gz A3.vcf.gz&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9098" LastActivityDate="2021-04-20T00:22:17.707" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15762" PostTypeId="1" AcceptedAnswerId="15778" CreationDate="2021-04-20T08:43:26.567" Score="3" ViewCount="332" Body="&lt;p&gt;I'm writing the code from scratch and not using libraries for the actual indexing/search since it's a project for school. Any blackbox advice is fairly useless.&lt;/p&gt;&#xA;&lt;p&gt;I'm planning on aligning some short reads with the &lt;em&gt;D. melanogaster&lt;/em&gt; genome but, since I was thrown into this project with very little info/ experience I'm stuck on a few things.&lt;/p&gt;&#xA;&lt;p&gt;Firstly, for preprocessing the fasta file:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;What should I do with Ns (ambiguous bases), I've seen some people removing them or replacing them by gaps but this does not seem like the same information? Should they be left in?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As far as I understand, the lowercase letters stand for potential repeats. However, If I just ignore this, it is possible my sequences might not align to the repeat because it is not actually stated as a repeat. E.g: &amp;quot;ATATATAT&amp;quot; will not align to 'at'.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;As far as I understand the difference between using suffix array and suffix tree: suffix tree will search in linear time (but is memory intensive), while suffix array will be space efficient (but searches through ALL sequences each time). I'm leaning towards suffix array because I'm working on my laptop and memory is kind of limited. I'm also thinking that, since you sort the array lexicographically, you really only have to check the sequences if they start with the same first base as the read you're searching for?&lt;/p&gt;&#xA;&lt;p&gt;Another question on the arrays: if my genome has 100M base pairs, the array will have 100M entries, some of them being close to 100M bases long? Should I be using the seed and extend method?&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, since the fruit fly genome is one of the smaller genomes, maybe it would be feasible to use a tree?&lt;/p&gt;&#xA;&lt;p&gt;Is there any way to validate the alignment quickly at the end? How much error should I be allowing?&lt;/p&gt;&#xA;&lt;p&gt;This were a lot of question, I've been trying to read up on things but I can mostly only find applications using biopython etc.&lt;/p&gt;&#xA;&lt;p&gt;Any help would be hugely appreciated!&lt;/p&gt;&#xA;" OwnerUserId="12019" LastEditorUserId="29" LastEditDate="2021-04-21T09:45:46.863" LastActivityDate="2021-04-21T19:04:57.713" Title="Sequence allignment with suffix array?" Tags="|sequence-alignment|fasta|homework|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15763" PostTypeId="2" ParentId="15762" CreationDate="2021-04-20T12:11:58.610" Score="2" Body="&lt;p&gt;Suffix arrays vs Suffix tree is a problem quite often discussed in bioinformatics classes, but it's not that much used in practice. Most of people use already optimized mappers &lt;a href=&quot;http://bowtie-bio.sourceforge.net/bowtie2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;bowtie2&lt;/a&gt; or &lt;a href=&quot;https://github.com/lh3/bwa&quot; rel=&quot;nofollow noreferrer&quot;&gt;bwa-mem&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I would advise you against your intention of mapping the reads using your own implementation of anything. You don't need to reinvent the wheel, just use an existing solution.&lt;/p&gt;&#xA;&lt;p&gt;By the way, the Ns in the reference are either hard-masked nucleotides (that would be repeats of some sorts) or (more often) simply unknown nucleotides. So there is no way to map your reads there anyway (and the mentioned mappers will handle the Ns for you).&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2021-04-21T10:32:38.993" LastActivityDate="2021-04-21T10:32:38.993" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15764" PostTypeId="2" ParentId="11150" CreationDate="2021-04-20T12:41:43.390" Score="1" Body="&lt;p&gt;You can access these data from a Seurat object as columns of &lt;code&gt;object@meta.data&lt;/code&gt; and tabulate them.&lt;/p&gt;&#xA;&lt;p&gt;There are functions such as “propellor” which will also implement this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/Oshlack/speckle&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Oshlack/speckle&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note that a Seurat vignette also explores this topic:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://satijalab.org/seurat/archive/v3.0/interaction_vignette.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/archive/v3.0/interaction_vignette.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You need to be careful when doing this to ensure different proportions are biological as single cell data is prone to technical errors.&lt;/p&gt;&#xA;" OwnerUserId="1855" LastActivityDate="2021-04-20T12:41:43.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15765" PostTypeId="1" CreationDate="2021-04-20T14:27:54.167" Score="1" ViewCount="824" Body="&lt;p&gt;I have RNA samples sequenced with TruSeq Stranded Total RNA kit protocol in Illumina HiSeq (2x125bp) and NovaSeq platforms (2x150bp) - almost 100 samples altogether. I have to use the samples data for expression analysis. Question is how do we normalize samples from different runs and of read length? What is best way (with available packages) we can normalize the count (to overcome the bias) and do a DE analysis? ANy suggestion would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="12038" LastActivityDate="2021-04-28T07:57:53.840" Title="Normalize RNA seq data from multiple runs for expression analysis" Tags="|rna-seq|differential-expression|gene-expression|normalization|featurecounts|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15766" PostTypeId="1" CreationDate="2021-04-20T14:36:52.150" Score="2" ViewCount="320" Body="&lt;p&gt;My data consists of two groups control and stim. I'm concerned that my clustering in the stim group is dominated by the unregulated genes. I can remove these genes and recluster but is there a way to then project the removed genes onto the new umap space so see if they are indeed all clustered together. They are no longer in the seurat object so I can't think of how to do this. I hope this makes sense.&lt;/p&gt;&#xA;" OwnerUserId="12039" LastActivityDate="2022-10-14T06:04:00.530" Title="Clustering with Seurat" Tags="|r|seurat|clustering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15767" PostTypeId="2" ParentId="15765" CreationDate="2021-04-20T17:37:11.240" Score="0" Body="&lt;p&gt;I don't think RNASEq is that sensitive to running different things on different instruments.  Differing read lengths will change things.  You could always trim the reads down so they all match.  Or include read length as a factor in the design when looking for DE genes.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2021-04-20T17:37:11.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15768" PostTypeId="1" AcceptedAnswerId="15772" CreationDate="2021-04-21T01:52:49.640" Score="2" ViewCount="1668" Body="&lt;p&gt;I'm trying to analyze data from a genotype-phenotype association study. The genotype data is stored in csv files that record for each variant site and each subject, the alleles that the subject has at that site. A quick online search reveals that 0 means the reference allele, 1 means the most common alternate allele, 2 means the 2nd most common alternate allele, etc.&lt;/p&gt;&#xA;&lt;p&gt;But I noticed that at some variant sites, most (&amp;gt;90%) subjects have 1/1, i.e. two copies of the alternate allele. I asked around and some suggested I just swap the roles of reference and alternate alleles whenever I find that the alternate allele frequency &amp;gt; 0.5. My questions are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Can there be a good reason to classify a less common allele as the reference allele? I thought that the reference allele was defined as the most common one in the population.&lt;/li&gt;&#xA;&lt;li&gt;Is swapping the roles of reference and alternate alleles whenever I find that the alternate allele frequency &amp;gt; 0.5 a standard practice?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="12044" LastEditorUserId="3694" LastEditDate="2021-04-22T10:14:14.780" LastActivityDate="2021-04-22T10:14:14.780" Title="Can an alternate allele be more common than a reference allele?" Tags="|variant-calling|gwas|allele-frequency|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15769" PostTypeId="2" ParentId="15765" CreationDate="2021-04-21T05:23:08.627" Score="1" Body="&lt;p&gt;There are a number of methods. If you're doing DE, you have ComBatSeq, SVAseq, RUVseq, BUSseq. You could also try Z-score normalization (&lt;span class=&quot;math-container&quot;&gt;$(x - \overline{x}) / \sigma$&lt;/span&gt;) or even quantile transformation. For the latter two, make sure you work on each batch individually, not the whole dataset at once. See more on that &lt;a href=&quot;https://www.nature.com/articles/s41598-020-72664-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To visually compare if transformation works, plotting PCA/UMAP/t-SNE on raw and transformed data can perhaps be of some insight.&lt;/p&gt;&#xA;" OwnerUserId="200" LastActivityDate="2021-04-21T05:23:08.627" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15770" PostTypeId="2" ParentId="15768" CreationDate="2021-04-21T05:52:52.933" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Yes, depending on how the reference allele was determined. If, for example, you want to refer to hg38 as the reference genome, then it's important to use the hg38 allele as reference.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Without stating the reference allele convention? Not that I'm aware of. However many people refer to minor/major alleles rather than non-reference alleles, in which case it is assumed that the major allele is the most common allele.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;As long as you state what your reference genome and/or population is, then it's fine to choose your own convention about reference alleles. But please be consistent and explicit about what you've done. One of my biggest struggles in GWAS has been working out the correct strand orientation for stated variants (important when merging datasets), and it doesn't help for complementary variants (i.e. A/T, C/G) when the ref/alt alleles are arbitrarily chosen.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-04-21T23:39:29.983" LastActivityDate="2021-04-21T23:39:29.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15771" PostTypeId="2" ParentId="15766" CreationDate="2021-04-21T06:04:42.217" Score="1" Body="&lt;p&gt;Yes, this can be done, particularly if your concern is with the clustering / UMAP rather than the transformation process; I've done this recently with some Rhapsody single cell data that we generated. The main trick is adding &lt;em&gt;'return.only.var.genes = FALSE'&lt;/em&gt;, as well as calculating a variable set that is a bit larger than the desired set. Depending on the number of cells, this will balloon the size of the resulting Seurat object pretty significantly.&lt;/p&gt;&#xA;&lt;p&gt;Note that UMAP is a technique to place &lt;em&gt;cells&lt;/em&gt;, rather than genes. If you have the information on where the cells should go, then plotting gene expression on top of that via FeaturePlot should work without any reprojection needed.&lt;/p&gt;&#xA;&lt;p&gt;Here's what I did (with experiment-specific variables excluded):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sct.data &amp;lt;- SCTransform(sct.data, vars.to.regress=&amp;quot;percent.mt&amp;quot;,&#xA;                        variable.features.n = 3000 + length(genesToExclude),&#xA;                        return.only.var.genes = FALSE);&#xA;&#xA;&#xA;sct.data &amp;lt;- RunPCA(sct.data, verbose = TRUE,&#xA;                   features=setdiff(VariableFeatures(sct.data), genesToExclude));&#xA;sct.data &amp;lt;- RunUMAP(sct.data);&#xA;sct.data &amp;lt;- FindNeighbors(sct.data);&#xA;sct.data &amp;lt;- FindClusters(sct.data);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;All transformed genes should be available for querying in the results.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-05-17T05:20:04.350" LastActivityDate="2022-05-17T05:20:04.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15772" PostTypeId="2" ParentId="15768" CreationDate="2021-04-21T08:56:40.447" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You are mixing up a bit the terminology here. One thing is a reference allele and another is a major allele. The first refers to the genotype present at a given position in the &amp;quot;reference&amp;quot; genome (e.g. if you work with human data this can be the reference version hg19 or hg38, etc). The second refers to the most common allele in a population. This definition can lead to problems since what is the most common allele can differ among different populations. In this respect, it is perfectly fine to have a reference allele with a lower frequency than another alternate allele.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;No for the reasons above. If you mean whether swapping major to minor is standard practice whenever you find out the major allele is actually minor as per the definition (less frequent), then I guess you can but this will lead to inconsistencies whenever you try to merge your data with other data sets, or other people merge their data with your data. Since this for sure happens and issues about strandness are well known, one has to make sure that you make explicit how you treat your data once you release it, and also one has to make sure that major/minor are defined the same way when merging data, and do the fix whenever needed.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="475" LastEditorUserId="475" LastEditDate="2021-04-21T12:46:00.857" LastActivityDate="2021-04-21T12:46:00.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15773" PostTypeId="2" ParentId="15765" CreationDate="2021-04-21T09:18:10.810" Score="0" Body="&lt;p&gt;Normalization and batch-effect correction are two different things.&lt;/p&gt;&#xA;&lt;p&gt;You always have to do normalization of RNAseq data in order to be able to compare the runs and do DE analysis, for instance. Plenty of methods and you have to search for them as normalization methods indeed.&lt;/p&gt;&#xA;&lt;p&gt;What you are asking and concerned about is &amp;quot;batch-effects&amp;quot; in your data as a function of having used two different Illumina machines. This is a valid concern, but before you try to correct for it or mitigate it, you first have to assess whether it is actually there. Maybe is not at all. To figure out if you have batch-effects you have to first do data exploration, this is why the use of PCA/UMAP/t-SNE has been already suggested. In this plots you will be trying to see whether your data cluster together by machine type instead of by the biology it represents. Read about how to assess the presence of batch effects.&lt;/p&gt;&#xA;&lt;p&gt;If you do observe it or anyway want to correct for its presence, then indeed you can try one of the methods also already mentioned: ComBatSeq, SVAseq, RUVseq, BUSseq.&lt;/p&gt;&#xA;&lt;p&gt;Rather than the two different machines, the batch-effect might actually be due to different people doing the lib preparation, experimental settings if experiments were done at different moments, etc.&lt;/p&gt;&#xA;" OwnerUserId="475" LastEditorUserId="475" LastEditDate="2021-04-23T14:10:18.647" LastActivityDate="2021-04-23T14:10:18.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15774" PostTypeId="2" ParentId="13521" CreationDate="2021-04-21T09:34:31.553" Score="0" Body="&lt;p&gt;I had this issue as well. It looks like all zero-length genes also have zero expression across all samples. The code below makes sure only to exclude genes that have both zero length and zero expression.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;txi = tximport(file_paths, type = &amp;quot;rsem&amp;quot;, txIn = TRUE, txOut = TRUE)&#xA;&#xA;zero_length_and_unexpressed = (apply(txi&lt;span class=&quot;math-container&quot;&gt;$abundance, 1, max) == 0) &amp;amp;&#xA;                              (apply(txi$&lt;/span&gt;length, 1, min) == 0)&#xA;&#xA;txi&lt;span class=&quot;math-container&quot;&gt;$length = txi$&lt;/span&gt;length[!zero_length_and_unexpressed,]&#xA;txi&lt;span class=&quot;math-container&quot;&gt;$abundance = txi$&lt;/span&gt;abundance[!zero_length_and_unexpressed,]&#xA;txi&lt;span class=&quot;math-container&quot;&gt;$counts = txi$&lt;/span&gt;counts[!zero_length_and_unexpressed,]&#xA;&#xA;dds = DESeqDataSetFromTximport(txi, sampleTable, ~ 1)&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="492" LastActivityDate="2021-04-21T09:34:31.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15775" PostTypeId="1" CreationDate="2021-04-21T10:07:12.180" Score="1" ViewCount="131" Body="&lt;p&gt;Dear Orthofinder users and BUSCO users,&lt;/p&gt;&#xA;&lt;p&gt;Recently, I have assessed 30 genome annotataions &lt;code&gt;using BUSCO with lineage&lt;/code&gt; dataset &lt;code&gt;ascomycota_odb10&lt;/code&gt;, I got 1100+ complete and single-copy genes. But the &lt;code&gt;orthofinder&lt;/code&gt;  only identified 300+ single-copy genes. Aneone else have encountered this similar problem?&lt;/p&gt;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;&lt;p&gt;Yanpeng&lt;/p&gt;&#xA;" OwnerUserId="8757" LastActivityDate="2021-04-29T12:37:56.593" Title="Number of sing-copy genes identified by orthofinder is far less than BUSCO assessment?" Tags="|orthologues|busco|orthofinder|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15777" PostTypeId="1" CreationDate="2021-04-21T12:48:27.747" Score="1" ViewCount="454" Body="&lt;p&gt;Problem Statement&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The GFF3 format is a commonly-used one in bioinformatics for&#xA;representing sequence  annotation. You can find the specification&#xA;here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.sequenceontology.org/gff3.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.sequenceontology.org/gff3.shtml&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The genome and annotation for Saccharomyces cerevisiae S288C is on the&#xA;class server here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/home/jorvis1/Saccharomyces_cerevisiae_S288C.annotation.gff&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that this same file has both the annotation feature table and the&#xA;FASTA sequence for the  molecules referenced. (See the '##FASTA'&#xA;directive in the specification.) Within the feature table another&#xA;column of note is the 9 th , where we can store any key=value  pairs&#xA;relevant to that row's feature such as ID, Ontology_term or Note. Your&#xA;task is to write a GFF3 feature exporter. A user should be able to run&#xA;your script like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ export_gff3_feature.py --source_gff=/path/to/some.gff3 --type=gene --attribute=ID --value=YAR003W&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There are 4 arguments here that correspond to values in the GFF3&#xA;columns. In this case, your  script should read the path to a GFF3&#xA;file, find any gene (column 3) which has an ID=YAR003W (column 9).&#xA;When it finds this, it should use the coordinates for that feature&#xA;(columns 4, 5 and 7)  and the FASTA sequence at the end of the&#xA;document to return its FASTA sequence. Your script should work&#xA;regardless of the parameter values passed, warning the user if no&#xA;features were found that matched their query. (It should also check&#xA;and warn if more than one  feature matches the query.)&lt;/p&gt;&#xA;&lt;p&gt;The output should just be printed on STDOUT (no writing to a file is&#xA;necessary.) It should have a header which matches their query, like&#xA;this:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;gene:ID:YAR003W .... sequence here ...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As an extra challenge, you can format the sequence portion of the&#xA;FASTA output as 60-characters per line, which follows the standard.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So far I have this below. I'm not even sure if this is correct to start with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;for line in open(&amp;quot;Saccharomyces_cerevisiae_S288C.annotation.gff&amp;quot;):&#xA;   line1 = line.rstrip()&#xA;   if line.startswith(&amp;quot;#&amp;quot;):&#xA;       continue&#xA;   column = line1.split(&amp;quot;\t&amp;quot;)&#xA;   if len(column) != 9:&#xA;       continue&#xA;   id = column[8]&#xA;   type = column[2]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can anyone help me with this? I'm totally lost. Thank you!&lt;/p&gt;&#xA;" OwnerUserId="12047" LastEditorUserId="3694" LastEditDate="2022-05-17T15:03:06.073" LastActivityDate="2024-02-06T18:09:06.130" Title="Create GFF3 feature exporter" Tags="|gff3|homework|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15778" PostTypeId="2" ParentId="15762" CreationDate="2021-04-21T19:04:57.713" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What should I do with Ns (ambiguous bases), I've seen some people removing them or replacing them by gaps but this does not seem like the same information? Should they be left in?&lt;/li&gt;&#xA;&lt;li&gt;As far as I understand, the lowercase letters stand for potential repeats. However, If I just ignore this, it is possible my sequences might not align to the repeat because it is not actually stated as a repeat. E.g: &amp;quot;ATATATAT&amp;quot; will not align to 'at'.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Good questions. I think this depends on the scope of your class project. Frankly, I think it would be best to just start with unambiguous sequences (only ATGC) and not worry about potential repeats. It's also a good software engineering exercise to solve the easy case first in a way that's flexible and allows you to expand to the additional complexities above. For example, the original Bowtie software considered references with ambiguous characters invalid. You can see for yourself &lt;a href=&quot;http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml#ambiguous-characters&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; how Bowtie2 handles it.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;As far as I understand the difference between using suffix array and suffix tree: suffix tree will search in linear time (but is memory intensive), while suffix array will be space efficient (but searches through ALL sequences each time). I'm leaning towards suffix array because I'm working on my laptop and memory is kind of limited. I'm also thinking that, since you sort the array lexicographically, you really only have to check the sequences if they start with the same first base as the read you're searching for?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, the most naive implementation of a search with a suffix array will be &lt;span class=&quot;math-container&quot;&gt;$O(m\log n)$&lt;/span&gt; time, where &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; is the read length and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the genome length. However, with &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S1570866703000650?via%3Dihub&quot; rel=&quot;nofollow noreferrer&quot;&gt;enhanced suffix arrays&lt;/a&gt; you can also obtain the same &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt; search complexity of suffix trees. There is also the use of suffix arrays to construct &lt;a href=&quot;https://en.wikipedia.org/wiki/Burrows%E2%80%93Wheeler_transform&quot; rel=&quot;nofollow noreferrer&quot;&gt;Burrows-Wheeler transforms&lt;/a&gt; and even further compression with the &lt;a href=&quot;https://en.wikipedia.org/wiki/FM-index&quot; rel=&quot;nofollow noreferrer&quot;&gt;FM index&lt;/a&gt;. There are open source implementations of all of these data structures that may be interesting to you.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Another question on the arrays: if my genome has 100M base pairs, the array will have 100M entries, some of them being close to 100M bases long? Should I be using the seed and extend method?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Suffix arrays store the indices of the start of each suffix in order to save space, so each entry is a single integer.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;On the other hand, since the fruit fly genome is one of the smaller genomes, maybe it would be feasible to use a tree?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Its worth trying to compute this yourself (: You can determine the number of bytes required per base pair in your input and see how that scales with your available memory. For example, if your suffix tree  implementation required &lt;span class=&quot;math-container&quot;&gt;$20n$&lt;/span&gt; bytes, you'd end up w/ roughly 2Gb needed for a fruit fly genome.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is there any way to validate the alignment quickly at the end? How much error should I be allowing?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Again, this depends on the scope of the project. Allowing for errors in mapping reads is an active area of research. See the supplemental of the Bowtie2 paper for a dynamic programming algorithm that handles a certain number of mismatches.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This were a lot of question, I've been trying to read up on things but I can mostly only find applications using biopython etc.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;All good questions, and I think many answers depend on the scope of the project. Reimplementing a read mapper is no small feat, especially when you start considering all of the different complexities such as sequencing error and ambiguous bases. I would strongly consider starting with looking at just the pure string problem of read mapping: Only consider ATCG in both reference and reads, and don't even consider reads who do not have exact matches to the reference. From there, time permitting, you can start adding on to your solution. As far as reading goes, there's a deep rabbit whole of these topics, from tools like bwa-mem, bowtie2, mini-map, to the more theory-oriented papers discussing optimized versions of the data structures. I can attach some here if you would like. Hope these answers helped!&lt;/p&gt;&#xA;" OwnerUserId="6628" LastActivityDate="2021-04-21T19:04:57.713" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15779" PostTypeId="2" ParentId="15415" CreationDate="2021-04-21T19:29:19.160" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Why is this approach not used? Are there any algorithms that use it?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As shown in &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0304397597002405&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, the MEDIAN STRING problem is NP-Complete on alphabets of carnality greater than 3. While the problem isn't used directly in approaches due to the complexity, the idea of trying to create a consensus sequence between pairs of input and using that as building blocks is common. These are considered &amp;quot;progressive alignment methods&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="6628" LastActivityDate="2021-04-21T19:29:19.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15780" PostTypeId="1" CreationDate="2021-04-22T03:36:35.383" Score="0" ViewCount="126" Body="&lt;p&gt;I'm trying to study fetal fraction by using a seqff script from &lt;a href=&quot;https://obgyn.onlinelibrary.wiley.com/doi/abs/10.1002/pd.4615&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The instruction says that I need a headless sam file to work so after alignment with bwa, I process my sam file by using samtools:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -S -q 20 -F 4 target.sam &amp;gt; treated.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This process also remove the @SG header since there is no reference file. I also use grep to remove header of the target.sam got after alignment for comparison. However, when I compare, I noticed that ENet calculation on the treated.sam returned with NA results, so half of the method to calculate fetal fraction of this script was gone. I have tested it in multiple cases and it seems that without using samtools, the script worked well but with samtools Enet calculaton not worked. Can someone tell me what is wrong with the script? I'm using R 4.0 and all modification I made to the script is on the args since it cannot recognize args I input with anymore.&lt;/p&gt;&#xA;&lt;p&gt;p/s: this is few lines of headless target.sam made by using grep&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;W2003120038L1C001R0050000009    0   chr10   6490952 0   50M *   0   0   GAGCAGAAAAACTGGAAACTCTAAAAAGCAGAGTGCCTCTCCTCCTCCAA  IBGIBAB&amp;lt;ABBCA9IB@BHDIA=ABBB8IB9A;DGII@I8IIAII&amp;lt;IDCA  XT:A:R  NM:i:0  X0:i:151    XM:i:0  XO:i:0  XG:i:0  MD:Z:50&#xA;W2003120038L1C001R0050000037    0   chr5    172722817   37  50M *   0   0   GTCCTAAGACAAAGACCTCAGTGCTTATAATGTAAGTGGTCACTGGGCTG  ICIICBB:BIBBBIAICDIAIAIICDBDAABIDBBICEIAIAFDH?IIAI  XT:A:U  NM:i:1  X0:i:1  X1:i:0  XM:i:1  XO:i:0  XG:i:0  MD:Z:7T42&#xA;W2003120038L1C001R0050000062    0   chr5    84724587    37  50M *   0   0   TAAGAATTGTGCATAACCAGTAGCCCATGTTATCTTGCAGAAGCATTTGT  ABBI@BDDIDIIADABHEAIC@IIIBBAID@ADIDCA:BC=BIIAAAAIC  XT:A:U  NM:i:0  X0:i:1  X1:i:0  XM:i:0  XO:i:0  XG:i:0  MD:Z:50&#xA;W2003120038L1C001R0050000067    16  chr5    45648067    37  50M *   0   0   TTTATTTAAATACAAGCCAATTAAATGATTTTATTTTACATTTTCTAACC  AABAAABCAAABH@CFHIDDB@BCDAID@BBACBB@BBIDB@2BIBC@II  XT:A:U  NM:i:0  X0:i:1  X1:i:0  XM:i:0  XO:i:0  XG:i:0  MD:Z:50&#xA;W2003120038L1C001R0050000184    16  chr4    103629659   37  50M *   0   0   CAGTGCTAAGAGGGAAGTTTATAGCAACAGATACTTACATCCAAAAAGTA  FDFBIIADAFDICICCIAABDBAFIADIDFDABIBA@I@CI3?DCCDIBC  XT:A:U  NM:i:0  X0:i:1  X1:i:0  XM:i:0  XO:i:0  XG:i:0  MD:Z:50&#xA;W2003120038L1C001R0050000211    0   chr3    157734199   37  50M *   0   0   TCTGAAATTGATCTTTATCCCACATCCACGTATCCTAAAATAATAAAACA  AFDBBBB*CIBDFACABDIIBB&amp;lt;1CI=BFIDBAFEABBAADBBABAABIB  XT:A:U  NM:i:2  X0:i:1  X1:i:0  XM:i:2  XO:i:0  XG:i:0  MD:Z:7G15T26&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And this is from treated.sam, the sam file be processed by samtools&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;W2003120038L1C001R0050000037    0   chr5    172722817   37  50M *   0   0   GTCCTAAGACAAAGACCTCAGTGCTTATAATGTAAGTGGTCACTGGGCTG  ICIICBB:BIBBBIAICDIAIAIICDBDAABIDBBICEIAIAFDH?IIAI  XT:A:U  NM:i:1  X0:i:1  X1:i:0  XM:i:1  XO:i:0  XG:i:0  MD:Z:7T42&#xA;W2003120038L1C001R0050000062    0   chr5    84724587    37  50M *   0   0   TAAGAATTGTGCATAACCAGTAGCCCATGTTATCTTGCAGAAGCATTTGT  ABBI@BDDIDIIADABHEAIC@IIIBBAID@ADIDCA:BC=BIIAAAAIC  XT:A:U  NM:i:0  X0:i:1  X1:i:0  XM:i:0  XO:i:0  XG:i:0  MD:Z:50&#xA;W2003120038L1C001R0050000067    16  chr5    45648067    37  50M *   0   0   TTTATTTAAATACAAGCCAATTAAATGATTTTATTTTACATTTTCTAACC  AABAAABCAAABH@CFHIDDB@BCDAID@BBACBB@BBIDB@2BIBC@II  XT:A:U  NM:i:0  X0:i:1  X1:i:0  XM:i:0  XO:i:0  XG:i:0  MD:Z:50&#xA;W2003120038L1C001R0050000184    16  chr4    103629659   37  50M *   0   0   CAGTGCTAAGAGGGAAGTTTATAGCAACAGATACTTACATCCAAAAAGTA  FDFBIIADAFDICICCIAABDBAFIADIDFDABIBA@I@CI3?DCCDIBC  XT:A:U  NM:i:0  X0:i:1  X1:i:0  XM:i:0  XO:i:0  XG:i:0  MD:Z:50&#xA;W2003120038L1C001R0050000211    0   chr3    157734199   37  50M *   0   0   TCTGAAATTGATCTTTATCCCACATCCACGTATCCTAAAATAATAAAACA  AFDBBBB*CIBDFACABDIIBB&amp;lt;1CI=BFIDBAFEABBAADBBABAABIB  XT:A:U  NM:i:2  X0:i:1  X1:i:0  XM:i:2  XO:i:0  XG:i:0  MD:Z:7G15T26&#xA;W2003120038L1C001R0050000212    0   chr4    157022243   37  50M *   0   0   AAAAATAAAAAAGCATTCTGCCCAGATCTCTCAACATTATGTTATTTCTT  4&amp;lt;AA?D2,AAA@IF@DDIDIIDIA2AD=DBAF@BIADA&amp;lt;DICD@DADFD@  XT:A:U  NM:i:0  X0:i:1  X1:i:0  XM:i:0  XO:i:0  XG:i:0  MD:Z:50&#xA;W2003120038L1C001R0050000258    0   chr13   106093727   37  50M *   0   0   AATGGTTACTTTATGTTTATATTTCAACAGGGAATATTTGAAATATGGCA  97?IIDA*FDAD&amp;lt;@H@AD&amp;lt;@&amp;lt;?AAG8??,I;I@:DADDAG7@@7(;IH6?  XT:A:U  NM:i:1  X0:i:1  X1:i:0  XM:i:1  XO:i:0  XG:i:0  MD:Z:7C42&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12051" LastEditorUserId="12051" LastEditDate="2021-04-22T07:34:45.110" LastActivityDate="2021-04-22T07:34:45.110" Title="seqff script got trouble after samtools treatment" Tags="|r|samtools|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15781" PostTypeId="2" ParentId="15777" CreationDate="2021-04-22T05:37:11.940" Score="0" Body="&lt;p&gt;Your answer looks like a good start. You'll probably want to use sys.argv, argparse, or another argument parsing tool to read in options from the command line, then compare them to the fields (e.g. the &amp;quot;type&amp;quot; argument gets compared with the type column).&lt;/p&gt;&#xA;&lt;p&gt;Given that attributes can be arranged in an arbitrary order, you'll probably need to read all of them from each line and store in a dict in order to determine whether the target is present. A regular expression search is another way to do that.&lt;/p&gt;&#xA;&lt;p&gt;There are a few libraries available (e.g. from biopython) for reading and subsetting FASTA records.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-04-22T05:37:11.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15782" PostTypeId="1" CreationDate="2021-04-22T08:35:55.337" Score="0" ViewCount="167" Body="&lt;p&gt;I have RNA samples but with different read lengths (Eg, HiSeq 2x125 and NovaSeq 2x150bp data). I would like to do DE analysis on these samples. What do you recommend? Do you recommend to trim the reads to 125bp for the NovaSeq to compare with the other batches from HiSeq? Or any other suggestions.&lt;/p&gt;&#xA;" OwnerUserId="12038" LastActivityDate="2021-04-22T12:07:24.860" Title="RNAseq DE comparison with samples of different read length" Tags="|rna-seq|" AnswerCount="2" CommentCount="1" ClosedDate="2021-04-28T00:27:24.890" ContentLicense="CC BY-SA 4.0" />
  <row Id="15783" PostTypeId="2" ParentId="15782" CreationDate="2021-04-22T09:20:07.030" Score="0" Body="&lt;p&gt;You don't need to do anything, this is taken care of by the linear model - if you don't have any confounding effects.&lt;/p&gt;&#xA;&lt;p&gt;If you do have confounding effects, e.g. all control samples on HiSeq and all treated on NovaSeq there's not a lot you can do anyway, other than be honest when you present your results.&lt;/p&gt;&#xA;&lt;p&gt;Taken care of by the model means adding an extra 2 level factor for sequencer by sample, you need to do this anyway to take care of the batch effect between sequencer runs.&lt;/p&gt;&#xA;" OwnerUserId="8919" LastActivityDate="2021-04-22T09:20:07.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15784" PostTypeId="2" ParentId="15782" CreationDate="2021-04-22T09:32:09.870" Score="0" Body="&lt;p&gt;Yes, I would simply trim the longer one to the shorter read length, then map the data as usual. I doubt though that it makes a big difference on the global scale but the processing step is simple and it makes sure you have no mappability bias. If you do not have any other confounding effects then you can do a DE analysis without any extra covariates, I think it is always desirable to keep things as simple as possible and trimming is the simplest solution.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2021-04-22T12:07:24.860" LastActivityDate="2021-04-22T12:07:24.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15786" PostTypeId="1" AcceptedAnswerId="15788" CreationDate="2021-04-22T15:15:12.803" Score="2" ViewCount="121" Body="&lt;p&gt;What are the units of force, velocity, and coordinates output by gromacs trajectory? In &lt;code&gt;.mdp&lt;/code&gt;, they are specified by &lt;code&gt;nstfout&lt;/code&gt;,&lt;code&gt;nstvout&lt;/code&gt;,&lt;code&gt;nstxout&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9890" OwnerDisplayName="BoKuToTsuZenU" LastEditorUserId="9890" LastEditDate="2021-04-22T15:58:02.113" LastActivityDate="2021-04-24T20:33:20.347" Title="What are the units for the force output by nstfout, the velocity output by nstvout, and the coordinates output by nstxout in gromacs?" Tags="|simulated-data|gromacs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15787" PostTypeId="1" CreationDate="2021-04-22T18:34:47.780" Score="2" ViewCount="316" Body="&lt;p&gt;I have 3 VCF files, I want to make CIRCOS plot just like the following:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/s41598-020-70527-8/figures/4&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41598-020-70527-8/figures/4&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I try to follow the &lt;a href=&quot;http://circos.ca/documentation/tutorials/2d_tracks/tiles/images&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; but could not understand. The genome is of Zea mays and the reference files used in making VCF files are downloaded from &lt;a href=&quot;ftp://ftp.ensemblgenomes.org/pub/plants/release-50/fasta/zea_mays/dna/Zea_mays.B73_RefGen_v4.dna.toplevel.fa.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl plant&lt;/a&gt;. How I can make karyotype and config files?&lt;/p&gt;&#xA;" OwnerUserId="6504" LastActivityDate="2021-04-22T18:34:47.780" Title="How to make CIRCOS plot of VCF file?" Tags="|vcf|snp|circos|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15788" PostTypeId="2" ParentId="15786" CreationDate="2021-04-22T20:23:11.407" Score="3" Body="&lt;p&gt;&lt;code&gt;nstxout&lt;/code&gt;, &lt;code&gt;nstvout&lt;/code&gt; and &lt;code&gt;nstfout&lt;/code&gt; are the number of &lt;strong&gt;steps&lt;/strong&gt; between coordinates, speed and forces written in the output trajectory (&lt;a href=&quot;https://manual.gromacs.org/current/user-guide/mdp-options.html?highlight=nstxout#output-control&quot; rel=&quot;nofollow noreferrer&quot;&gt;Output control documentation&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;As for the units present in your trajectory files (say a &lt;code&gt;.trr&lt;/code&gt;):&#xA;coordinates are in &lt;strong&gt;nm&lt;/strong&gt;, speeds in &lt;strong&gt;nm.μs⁻¹&lt;/strong&gt; (so in &lt;strong&gt;mm.m⁻¹&lt;/strong&gt;) and forces in &lt;strong&gt;kJ.mol⁻¹.nm⁻¹&lt;/strong&gt;. These should be consistent through all GROMACS files (&lt;a href=&quot;https://manual.gromacs.org/current/reference-manual/definitions.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Definitions and Units documentation&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Note that &lt;code&gt;.xtc&lt;/code&gt; files multiply coordinates by 1000 to compress them, so they are stored in &lt;strong&gt;pm&lt;/strong&gt; (see &lt;a href=&quot;https://manual.gromacs.org/current/reference-manual/file-formats.html#xtc&quot; rel=&quot;nofollow noreferrer&quot;&gt;GROMACS xtc documentation&lt;/a&gt;). Given &lt;a href=&quot;https://userguide.mdanalysis.org/1.0.0/formats/reference/xtc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MDAnalysis documentation&lt;/a&gt; on the subject, I believe &lt;code&gt;.xtc&lt;/code&gt; files can &lt;em&gt;only&lt;/em&gt; store coordinates, no speeds or forces.&lt;/p&gt;&#xA;&lt;p&gt;I would be wary of softwares used to analyze your trajectory results (MDAnalysis, VMD, etc), as they may automatically do the conversion of these units, and for example show you coordinates in Å.&lt;/p&gt;&#xA;" OwnerUserId="12054" LastEditorUserId="12054" LastEditDate="2021-04-24T20:33:20.347" LastActivityDate="2021-04-24T20:33:20.347" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15789" PostTypeId="5" CreationDate="2021-04-22T21:49:17.953" Score="0" Body="&lt;p&gt;&lt;strong&gt;GROMACS&lt;/strong&gt; is a free open-source package to perform molecular dynamics, i.e. simulate the Newtonian equations of motion for systems with hundreds to millions of particles.&lt;/p&gt;&#xA;&lt;p&gt;It is primarily designed for biochemical molecules like proteins, lipids and nucleic acids that have a lot of complicated bonded interactions, though many groups also use it for research on non-biological systems, e.g. polymers. Like most molecular dynamics software, its engine allows parallelization on CPU and GPU.&lt;/p&gt;&#xA;&lt;p&gt;There is no scripting language - all programs use a simple interface with command line options for input and output files. GROMACS also comes with a large selection of flexible tools for trajectory analysis. It also comprises &lt;code&gt;gmxapi&lt;/code&gt; Python Package, which allows molecular simulation and analysis work to be staged and run from Python. All these features are referenced and explained in &lt;a href=&quot;https://manual.gromacs.org/documentation/current/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GROMACS online documentation&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Example:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Starting from a Protein Data Bank (pdb) file, the whole process of solvation, ionization and minimization of the system through command lines would be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gmx pdb2gmx -f 1AKI.pdb -o 1AKI_processed.gro -water spce&#xA;gmx editconf -f 1AKI_processed.gro -o 1AKI_newbox.gro -c -d 1.0 -bt cubic&#xA;gmx solvate -cp 1AKI_newbox.gro -cs spc216.gro -o 1AKI_solv.gro -p topol.top&#xA;gmx grompp -f ions.mdp -c 1AKI_solv.gro -p topol.top -o ions.tpr&#xA;gmx genion -s ions.tpr -o 1AKI_solv_ions.gro -p topol.top -pname NA -nname CL -neutral&#xA;gmx grompp -f minim.mdp -c 1AKI_solv_ions.gro -p topol.top -o em.tpr&#xA;gmx mdrun -v -deffnm em&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(From &lt;a href=&quot;http://www.mdtutorials.com/gmx/lysozyme/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GROMACS Tutorial&lt;/a&gt; by Dr. Justin A. Lemkul)&lt;/p&gt;&#xA;" OwnerUserId="12054" LastEditorUserId="12054" LastEditDate="2021-04-26T20:50:11.450" LastActivityDate="2021-04-26T20:50:11.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15790" PostTypeId="4" CreationDate="2021-04-22T21:49:17.953" Score="0" Body="GROMACS is a molecular dynamics package mainly designed for simulations of proteins, lipids, and nucleic acids. Its command line interface and python API are designed to make simulations on such molecules easy to setup, run, and analyze." OwnerUserId="12054" LastEditorUserId="12054" LastEditDate="2021-04-26T20:50:19.107" LastActivityDate="2021-04-26T20:50:19.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15791" PostTypeId="1" AcceptedAnswerId="15793" CreationDate="2021-04-22T22:21:32.787" Score="1" ViewCount="61" Body="&lt;p&gt;I am trying to write a program that will count the repeats of a given DNA strand:&lt;/p&gt;&#xA;&lt;p&gt;I am using a Counter library, something like this code shown below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from collections import Counter&#xA;&#xA;        s=&amp;quot;XXXXXAGCCTGCCAAGCAAACTTCACTGGAGXXXXXTGTGCGTAGCATGCTAGTAACTGCAXXXXXTCTGAATCTTTCAGCTGCTTGTXXXXXTGGGCCTCTCACAAGGCAGAGTGTCTTCATGGGACTTTGATATTTATTTTTGTACAACCTAAGAGGAACAAATCCTTTGACACTGACAAATTGGCTTCCATATTTTATACCTTAATCATCTCCATGTTGAATTCATTGATCAACAGTTTAAGAAAAAAAGATGTAAAAATGCTTTTAGAAAGAGAGGCAAAGTTATGCACAATAACTTCTCATGAAGTCACAGTTTGTTAAAAGTTGCCTTAGTTCACAATAAATAATTATGTATGCTCTATAATTTCAGTGA&amp;quot;&#xA;&#xA;&#xA;def chunk_string(s, n):&#xA;    return [s[i:i+n] for i in range(len(s)-n+1)]&#xA;&#xA;counter = Counter(chunk_string(s, 5))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Repeat supposed to be &amp;quot;declared&amp;quot; as an identical sequence on the same strand (like &lt;code&gt;XXXXX&lt;/code&gt; here that occurs 4 times). However, I can also have this repeat (for ex. &lt;code&gt;XXXXX&lt;/code&gt; in this particular test case) on the OPPOSITE strand.&lt;/p&gt;&#xA;&lt;p&gt;How am I suppose to intake the opposite strand in case of using &lt;code&gt;.fasta&lt;/code&gt; files? As I know for today - fasta is presented only in a single strand way. Or am I completely mixed the terms?&lt;/p&gt;&#xA;" OwnerUserId="6470" LastActivityDate="2021-04-23T02:56:06.017" Title="How to input the opposite strand of DNA?" Tags="|python|fasta|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15792" PostTypeId="2" ParentId="15791" CreationDate="2021-04-22T23:01:45.737" Score="0" Body="&lt;p&gt;What's going to be simplest is to generate the reverse complement of your short search sequence, and search for that too.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2021-04-22T23:01:45.737" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15793" PostTypeId="2" ParentId="15791" CreationDate="2021-04-23T02:56:06.017" Score="1" Body="&lt;p&gt;Yes, you're going to want to take the reverse complement of your string in order to analyze the other strand, that is the standard.&lt;/p&gt;&#xA;&lt;p&gt;Also, it is worth pointing out that you may want to use a generator expression instead of a list expression for your &lt;code&gt;chunk_string&lt;/code&gt; function:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def chunk_string(s, n):&#xA;    return (s[i:i+n] for i in range(len(s)-n+1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will result in a generator that yields kmers as the &lt;code&gt;Counter&lt;/code&gt; constructor iterates through them, which means that you don't need to first construct and store the entire list of kmers first and then pass to &lt;code&gt;Counter&lt;/code&gt; but instead can feed them directly to it. Not essential for small strings, of course, but a good practice and useful for larger strings. See &lt;a href=&quot;https://stackoverflow.com/questions/47789/generator-expressions-vs-list-comprehensions&quot;&gt;this SO post&lt;/a&gt; for a better explanation.&lt;/p&gt;&#xA;&lt;p&gt;You can also use Biopython's &lt;a href=&quot;https://biopython.org/wiki/Seq#complement-and-reverse-complement&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;reverse_complement&lt;/code&gt;&lt;/a&gt; function in order to easily obtain the reverse complement.&lt;/p&gt;&#xA;" OwnerUserId="6628" LastActivityDate="2021-04-23T02:56:06.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15794" PostTypeId="1" AcceptedAnswerId="15795" CreationDate="2021-04-23T05:08:13.037" Score="0" ViewCount="1453" Body="&lt;p&gt;I tried to use these python codes to retrieve a .fasta file from gene database, but the output were the parameters of the report&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez&#xA;Entrez.email = &amp;quot;send_helps@earth.com&amp;quot;&#xA;handle = Entrez.efetch(db = &amp;quot;gene&amp;quot;,id = &amp;quot;682&amp;quot;,rettype=&amp;quot;fasta&amp;quot;, retmode=&amp;quot;xml&amp;quot;)&#xA;print(Entrez.read(handle))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;i also read the documentation on &lt;a href=&quot;http://www.ncbi.nlm.nih.gov/books/NBK25499/#chapter4.EFetch&quot; rel=&quot;nofollow noreferrer&quot;&gt;this site&lt;/a&gt; about efetch() and seems like this function can only retrieve certain kinds of data for each database. So i can't take fasta file from gene database !?&lt;/p&gt;&#xA;&lt;p&gt;I don't want to use the NCBI web UI. What's wrong with my code? Is there a way i can guarantee to retreive a .fasta file through python from NCBI ? Am i misundertanding something severly ?&lt;/p&gt;&#xA;&lt;p&gt;Sorry for bad English. thank you.&lt;/p&gt;&#xA;" OwnerUserId="8611" LastActivityDate="2021-04-23T11:38:11.627" Title="Using Entrez.efetch() to retrive .fasta file from any NCBI database?" Tags="|fasta|python|phylogenetics|entrez|efetch|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15795" PostTypeId="2" ParentId="15794" CreationDate="2021-04-23T11:38:11.627" Score="2" Body="&lt;p&gt;In order to download the FASTA sequence of transcript variants using Entrez eutilities, you will have to use &lt;code&gt;Entrez.elink&lt;/code&gt; to get a list of transcript accessions for the gene of interest and then use &lt;code&gt;Entrez.efetch&lt;/code&gt; to fetch the sequence(s). Just to give you an idea, you can use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez Direct&lt;/a&gt; for this as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;elink -db gene -id 682 -target nuccore -name gene_nuccore_refseqrna \&#xA;  | efetch -format fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can find more Entrez Link descriptions &lt;a href=&quot;https://eutils.ncbi.nlm.nih.gov/entrez/query/static/entrezlinks.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Note, there is a &lt;code&gt;gene_fasta&lt;/code&gt; option for &lt;code&gt;efetch&lt;/code&gt; but it still returns only the transcript sequence (i.e., no intron sequence).&lt;/p&gt;&#xA;&lt;p&gt;If you need the full gene sequence (including the intronic regions), you can use the commands mentioned &lt;a href=&quot;https://github.com/NCBI-Hackathons/EDirectCookbook/issues/47&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, you can use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Datasets&lt;/a&gt; for this. You can use the web interface, command line tool or the python API. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets/new/docs/how-tos/genes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This page&lt;/a&gt; with the how-to articles has the information you seek.&lt;/p&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2021-04-23T11:38:11.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15796" PostTypeId="2" ParentId="8612" CreationDate="2021-04-23T12:40:46.987" Score="0" Body="&lt;p&gt;There was a &lt;a href=&quot;https://github.com/satijalab/seurat/issues/2201&quot; rel=&quot;nofollow noreferrer&quot;&gt;discussion&lt;/a&gt; about this in Seurat developer's github and someone actually made a package to easily add several color bars using some DoHeamap-like function. Check it &lt;a href=&quot;https://github.com/elliefewings/DoMultiBarHeatmap&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="5139" LastActivityDate="2021-04-23T12:40:46.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15797" PostTypeId="2" ParentId="4101" CreationDate="2021-04-23T16:17:35.080" Score="0" Body="&lt;p&gt;The inverse relationship between variance and mean is a consequence of the stochastic nature of gene expression, which can be modeled as a discrete birth-death process. One of the simplest models assumes that the gene is expressed constitutively. Then, using the chemical master equation, mRNA production can be shown to evolve as a discrete Poisson process. The steady-state distribution is Poisson. For the Poisson distribution, coefficient of variation (&lt;span class=&quot;math-container&quot;&gt;$\sigma^2/\mu^2$&lt;/span&gt;) is equal to &lt;span class=&quot;math-container&quot;&gt;$1/\mu$&lt;/span&gt;. Further, if protein is translated in bursts from mRNA (translation burst), protein noise can be written as&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;\begin{equation}\label{eq1}\tag{1}&#xA;\eta^2_P = \frac{\sigma_P^2}{\mu_P^2} = \frac{B}{\mu_P},&#xA;\end{equation}&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$\eta_P^2, \sigma_P^2,\mu_P,$&lt;/span&gt; and B are the coefficient of variation, variance, mean and burst size for protein. From \eqref{eq1}, the inverse relationship is evident.&lt;/p&gt;&#xA;&lt;p&gt;Gene expression models can be made more detailed by assuming different states for the promoter. The promoter switches between these states, and each of these states can have different rates of transcription. For these models, &lt;span class=&quot;math-container&quot;&gt;$\eta_P^2$&lt;/span&gt; can again be shown to follow an inverse relationship with &lt;span class=&quot;math-container&quot;&gt;$\mu_P^2$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Reference:&lt;/p&gt;&#xA;&lt;p&gt;Paulsson, Johan. &amp;quot;&lt;a href=&quot;https://doi.org/10.1016/j.plrev.2005.03.003&quot; rel=&quot;nofollow noreferrer&quot;&gt;Models of stochastic gene expression.&lt;/a&gt;&amp;quot; Physics of life reviews 2.2 (2005): 157-175.&lt;/p&gt;&#xA;" OwnerUserId="12057" LastActivityDate="2021-04-23T16:17:35.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15798" PostTypeId="1" CreationDate="2021-04-23T21:30:40.927" Score="3" ViewCount="54" Body="&lt;p&gt;I have been trying to estimate Tajima-Nei distance for my data (link below). The output is s phylogenetics distance (pairwise) matrix which accommodates a detailed transition transversion and the Jukes-Cantor correction.&lt;/p&gt;&#xA;&lt;p&gt;I´m following this protocol from BioPerl: &lt;a href=&quot;https://metacpan.org/pod/Bio::Align::DNAStatistics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://metacpan.org/pod/Bio::Align::DNAStatistics&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have 314 sequences in a fasta file and another file with the list of IDs.&#xA;Fasta:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;AVP78031.1&#xA;----atgttgtttttcttgtttcttcagttcgccttagtaaactc---------------&#xA;------------------------------------ccagtgtgttaacttgacaggcag&#xA;a----------------accccactcaatcccaattat--actaattcttcacaaagagg&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;IDs:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;AVP78031.1&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I´m using this Perl script to calculate the Tajima-Nei Distance in a pairwise comparison (314 * 314):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;use strict;&#xA;use warnings;&#xA;use Bio::AlignIO;&#xA;use Bio::Align::DNAStatistics;&#xA;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$file = $&lt;/span&gt;ARGV[0];&#xA;my &lt;span class=&quot;math-container&quot;&gt;$idfile = $&lt;/span&gt;ARGV[1];&#xA;&#xA;if (&lt;span class=&quot;math-container&quot;&gt;$file eq &quot;&quot; ) {&#xA;  $file = &quot;NT_MSA_S_protein.fasta&quot;;&#xA;} elsif ($&lt;/span&gt;idfile eq &amp;quot;&amp;quot; ) {&#xA;  $idfile = &amp;quot;NT_ID_S_protein.csv&amp;quot;;&#xA;}&#xA;&#xA;&#xA;#### Considerando un archivo&#xA;my @contentIDS;&#xA;&#xA;open (LIST, &lt;span class=&quot;math-container&quot;&gt;$idfile) or die;&#xA;while (my $&lt;/span&gt;l = &amp;lt;LIST&amp;gt;) {&#xA;  &lt;span class=&quot;math-container&quot;&gt;$l =~ s/\n//g; # delete newline&#xA;    $&lt;/span&gt;l =~ s/\r//g; # delete CR&#xA;  next if (length(&lt;span class=&quot;math-container&quot;&gt;$l) &amp;lt; 1);&#xA;  push @contentIDS, $&lt;/span&gt;l;&#xA;}&#xA;close LIST;&#xA;&#xA;#### .... IDs list &#xA;my &lt;span class=&quot;math-container&quot;&gt;$stats = Bio::Align::DNAStatistics-&amp;gt;new();&#xA;my $&lt;/span&gt;alignin = Bio::AlignIO-&amp;gt;new(-format =&amp;gt; 'fasta', -file   =&amp;gt; &lt;span class=&quot;math-container&quot;&gt;$file);  ### $&lt;/span&gt;file: MSA file&#xA;while (my &lt;span class=&quot;math-container&quot;&gt;$aln = $&lt;/span&gt;alignin-&amp;gt;next_aln) {&#xA;  #print &amp;quot;reading...A\n&amp;quot;; ### DIAG&#xA;  my &lt;span class=&quot;math-container&quot;&gt;$matrix = $&lt;/span&gt;stats-&amp;gt;distance(-align =&amp;gt; &lt;span class=&quot;math-container&quot;&gt;$aln, -method =&amp;gt; 'Tajima-Nei');&#xA;  #print &quot;reading...B\n&quot;; ### DIAG&#xA;  ### Obtaining values for each pair (DISTANCE!)&#xA;  WL1:&#xA;  foreach my $&lt;/span&gt;aaa (@contentIDS) { ### ID #1&#xA;    WL2:&#xA;    foreach my &lt;span class=&quot;math-container&quot;&gt;$baa (@contentIDS) { ### ID #2&#xA;    next (WL2) if ($aaa eq $baa);&#xA;  my $data =  $matrix-&amp;gt;get_entry($aaa, $baa);&#xA;  #($data = 0) if ($data &amp;lt; 0);&#xA;    print &quot;DISTANCE\t$aaa\t$baa\t$&lt;/span&gt;data\n&amp;quot;;&#xA;    } # END WL2&#xA;  } # END WL1&#xA;}&#xA;&#xA;exit;&#xA;&#xA;&#xA;#&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This script work it when I tried with small data, however, when I try with my real data this is the error message&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MSG: Must provide a DNA alignment to Bio::Align::DNAStatistics, you provided a protein&#xA;---------------------------------------------------&#xA;Can't locate object method &amp;quot;get_entry&amp;quot; via package &amp;quot;0&amp;quot; (perhaps you forgot to load &amp;quot;0&amp;quot;?) at Tajima-Nei_Distance_NV.pl lin$&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is weird because I review my data for ambiguous characters and the characters are in majority &amp;quot;atcg&amp;quot; and on some occasions &amp;quot;n&amp;quot;, at least that there are other ambiguous characters (maybe) that represent protein sequence.&#xA;I really don´t understand the message because the fasta file is clearly a nucleotide sequence.&lt;/p&gt;&#xA;&lt;p&gt;Anyone with experience using Bioperl and estimation Tajima-Nei distance?&lt;/p&gt;&#xA;&lt;p&gt;Link: &lt;a href=&quot;https://github.com/MauriAndresMU1313/Example_Tajima-Nei_Distance_Bioperl/tree/main&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/MauriAndresMU1313/Example_Tajima-Nei_Distance_Bioperl/tree/main&lt;/a&gt;&#xA;Any comment or help is welcome!!&#xA;Thank!&lt;/p&gt;&#xA;" OwnerUserId="10557" LastEditorUserId="3848" LastEditDate="2022-02-23T09:27:15.540" LastActivityDate="2022-02-23T09:33:29.450" Title="Tajima-Nei Distance estimate with BioPerl" Tags="|statistics|phylogeny|phylogenetics|perl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15799" PostTypeId="2" ParentId="15765" CreationDate="2021-04-24T13:05:52.923" Score="0" Body="&lt;p&gt;I agree with &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/2388/swbarnes2&quot;&gt;swbarnes&lt;/a&gt; that you don't need to explicitly model for different read length, but you need to normalize for different production of sequencing runs.&#xA;Among the several available packages, &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/DESeq2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DEseq2&lt;/a&gt; is one of the most widely used.&#xA;The normalization method computes the geometric mean of expression of each gene across samples; then, expression of the gene in each sample is divided by the geometric mean. Once this has been don for all genes, in each sample it computes the median of values, and this is the &amp;quot;size factor&amp;quot;. Ok, it looks convoluted, but it is indeed easy, and it is implemented in DESeq2 and in other software packages.&#xA;You can find methodological details &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4302049/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&#xA;Finally, DESeq2 also offer the possibility to account for additional experimental details (e.g. read length) and this should reduce the risk of spurious results.&lt;/p&gt;&#xA;" OwnerUserId="1140" LastEditorUserId="1140" LastEditDate="2021-04-24T13:12:05.420" LastActivityDate="2021-04-24T13:12:05.420" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15800" PostTypeId="1" AcceptedAnswerId="15826" CreationDate="2021-04-24T13:54:51.643" Score="3" ViewCount="123" Body="&lt;p&gt;I am using GROMACS to perform MD simulations of proteins. So, I tried to record the simulation results &lt;code&gt;nstxout&lt;/code&gt;, &lt;code&gt;nstvout&lt;/code&gt;, and &lt;code&gt;nstfout&lt;/code&gt; with the same step size (&lt;code&gt;nstxout 1, nstvout 1, nstfout 1&lt;/code&gt;). Then, I calculated the coordinate (step n) of an arbitrary step of the trajectory (step n) from the velocity of step n - 1, step n, and calculated the velocity of n - 1/2, and used that velocity to calculate the coordinate (n-1) of step n using the velocity of n - 1/2. LeapFrog was used in the simulation.&lt;/p&gt;&#xA;&lt;p&gt;How can I get the coordinates, velocity, and force to be consistent?&lt;/p&gt;&#xA;" OwnerUserId="9890" LastEditorUserId="57" LastEditDate="2021-04-29T15:34:53.263" LastActivityDate="2021-04-29T15:34:53.263" Title="GROMACS Coordinates and velocity and force trajectories do not match accuracy" Tags="|simulated-data|gromacs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15804" PostTypeId="1" AcceptedAnswerId="15806" CreationDate="2021-04-26T08:35:16.410" Score="1" ViewCount="233" Body="&lt;p&gt;I am doing metaanalysis with forestplot from ggforestplot package. I have 7 different datasets from 7 different countries. After metaanalysis, I have 7 Effect sizes and a Meta-analysis effect size, i.e., total of 8 effect sizes. Now, I have generated a forest plot using this code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Forestplot&#xA;my_plot &amp;lt;- forestplot(&#xA;  df = final,&#xA;  estimate = estimate,&#xA;  logodds = FALSE,&#xA;  colour = country==&amp;quot;metaanalysis&amp;quot;,&#xA;  shape = country,&#xA;  title = &amp;quot;Associations to disease accross populations&amp;quot;,&#xA;  xlab = &amp;quot;Effect Size&amp;quot;,&#xA;  #xlim = c(-1.5, 1.5),&#xA;  xtickbreaks = c(-1.0, -0.5, -0.45, -0.4, 0, 0.4, 0.45,0.5, 1.0)&#xA;)+&#xA;  # You may also want to add a manual shape scale to mark meta-analysis with a&#xA;  # diamond shape&#xA;  ggplot2::scale_shape_manual(&#xA;    values = c(5L, 0L, 1L, 2L, 8L, 19L, 4L, 3L),&#xA;    &#xA;  )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But, the shapes denoting the countries and the metanalysis seem very small. How can I make them larger for better visualization? Can anyone please help me?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Representative data for replication:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;final &amp;lt;- data.frame(&#xA;  stringsAsFactors = FALSE,&#xA;              name = c(&amp;quot;RIBOSYN2-PWY__flavin_biosynthesis_I_(bacteria_and_plants)&amp;quot;,&#xA;                       &amp;quot;PWY-6385__peptidoglycan_biosynthesis_III_(mycobacteria)&amp;quot;,&#xA;                       &amp;quot;PWY-6151__S-adenosyl-L-methionine_cycle_I&amp;quot;,&#xA;                       &amp;quot;PWY-6387__UDP-N-acetylmuramoyl-pentapeptide_biosynthesis_I_(meso-diaminopimelate_containing)&amp;quot;,&#xA;                       &amp;quot;PWY-6700__queuosine_biosynthesis&amp;quot;,&#xA;                       &amp;quot;PEPTIDOGLYCANSYN-PWY__peptidoglycan_biosynthesis_I_(meso-diaminopimelate_containing)&amp;quot;,&#xA;                       &amp;quot;PWY-6386__UDP-N-acetylmuramoyl-pentapeptide_biosynthesis_II_(lysine-containing)&amp;quot;,&#xA;                       &amp;quot;PWY-5667__CDP-diacylglycerol_biosynthesis_I&amp;quot;,&#xA;                       &amp;quot;PWY0-1319__CDP-diacylglycerol_biosynthesis_II&amp;quot;,&#xA;                       &amp;quot;PWY-6122__5-aminoimidazole_ribonucleotide_biosynthesis_II&amp;quot;,&#xA;                       &amp;quot;PWY-6277__superpathway_of_5-aminoimidazole_ribonucleotide_biosynthesis&amp;quot;,&#xA;                       &amp;quot;PWY-6163__chorismate_biosynthesis_from_3-dehydroquinate&amp;quot;,&#xA;                       &amp;quot;TRPSYN-PWY__L-tryptophan_biosynthesis&amp;quot;,&#xA;                       &amp;quot;PANTOSYN-PWY__pantothenate_and_coenzyme_A_biosynthesis_I&amp;quot;,&#xA;                       &amp;quot;GLYCOCAT-PWY__glycogen_degradation_I_(bacterial)&amp;quot;,&#xA;                       &amp;quot;P161-PWY__acetylene_degradation&amp;quot;,&#xA;                       &amp;quot;FUC-RHAMCAT-PWY__superpathway_of_fucose_and_rhamnose_degradation&amp;quot;,&#xA;                       &amp;quot;P108-PWY__pyruvate_fermentation_to_propanoate_I&amp;quot;,&#xA;                       &amp;quot;PWY-7328__superpathway_of_UDP-glucose-derived_O-antigen_building_blocks_biosynthesis&amp;quot;,&#xA;                       &amp;quot;TCA__TCA_cycle_I_(prokaryotic)&amp;quot;,&#xA;                       &amp;quot;PWY-6901__superpathway_of_glucose_and_xylose_degradation&amp;quot;,&amp;quot;PWY66-400__glycolysis_VI_(metazoan)&amp;quot;,&#xA;                       &amp;quot;FUCCAT-PWY__fucose_degradation&amp;quot;,&#xA;                       &amp;quot;PRPP-PWY__superpathway_of_histidine,_purine,_and_pyrimidine_biosynthesis&amp;quot;,&#xA;                       &amp;quot;PWY-7234__inosine-5'-phosphate_biosynthesis_III&amp;quot;,&#xA;                       &amp;quot;TRPSYN-PWY__L-tryptophan_biosynthesis&amp;quot;,&#xA;                       &amp;quot;PWY-6122__5-aminoimidazole_ribonucleotide_biosynthesis_II&amp;quot;,&#xA;                       &amp;quot;PWY-6277__superpathway_of_5-aminoimidazole_ribonucleotide_biosynthesis&amp;quot;,&#xA;                       &amp;quot;RIBOSYN2-PWY__flavin_biosynthesis_I_(bacteria_and_plants)&amp;quot;,&#xA;                       &amp;quot;PWY-6163__chorismate_biosynthesis_from_3-dehydroquinate&amp;quot;,&#xA;                       &amp;quot;PWY-5667__CDP-diacylglycerol_biosynthesis_I&amp;quot;,&#xA;                       &amp;quot;PWY0-1319__CDP-diacylglycerol_biosynthesis_II&amp;quot;,&#xA;                       &amp;quot;PANTOSYN-PWY__pantothenate_and_coenzyme_A_biosynthesis_I&amp;quot;,&#xA;                       &amp;quot;RIBOSYN2-PWY__flavin_biosynthesis_I_(bacteria_and_plants)&amp;quot;,&#xA;                       &amp;quot;PWY-6386__UDP-N-acetylmuramoyl-pentapeptide_biosynthesis_II_(lysine-containing)&amp;quot;,&#xA;                       &amp;quot;RIBOSYN2-PWY__flavin_biosynthesis_I_(bacteria_and_plants)&amp;quot;,&#xA;                       &amp;quot;PANTOSYN-PWY__pantothenate_and_coenzyme_A_biosynthesis_I&amp;quot;,&#xA;                       &amp;quot;PWY-6385__peptidoglycan_biosynthesis_III_(mycobacteria)&amp;quot;,&#xA;                       &amp;quot;PWY-6387__UDP-N-acetylmuramoyl-pentapeptide_biosynthesis_I_(meso-diaminopimelate_containing)&amp;quot;,&amp;quot;PWY-6700__queuosine_biosynthesis&amp;quot;,&#xA;                       &amp;quot;PWY-6151__S-adenosyl-L-methionine_cycle_I&amp;quot;,&#xA;                       &amp;quot;PWY0-1319__CDP-diacylglycerol_biosynthesis_II&amp;quot;,&#xA;                       &amp;quot;PWY-5667__CDP-diacylglycerol_biosynthesis_I&amp;quot;,&#xA;                       &amp;quot;PEPTIDOGLYCANSYN-PWY__peptidoglycan_biosynthesis_I_(meso-diaminopimelate_containing)&amp;quot;,&#xA;                       &amp;quot;PWY-6387__UDP-N-acetylmuramoyl-pentapeptide_biosynthesis_I_(meso-diaminopimelate_containing)&amp;quot;,&#xA;                       &amp;quot;PWY-6385__peptidoglycan_biosynthesis_III_(mycobacteria)&amp;quot;,&#xA;                       &amp;quot;PANTOSYN-PWY__pantothenate_and_coenzyme_A_biosynthesis_I&amp;quot;,&#xA;                       &amp;quot;PEPTIDOGLYCANSYN-PWY__peptidoglycan_biosynthesis_I_(meso-diaminopimelate_containing)&amp;quot;,&amp;quot;PWY-5667__CDP-diacylglycerol_biosynthesis_I&amp;quot;,&#xA;                       &amp;quot;PWY0-1319__CDP-diacylglycerol_biosynthesis_II&amp;quot;),&#xA;          estimate = c(0.492903365186722,&#xA;                       0.490116714405755,0.470681520902659,0.465726944992238,&#xA;                       0.461786439528495,0.455469435286562,0.449174326308655,&#xA;                       0.433755737833203,0.433670791448977,0.412009484469875,&#xA;                       0.412009484469875,0.409849212159568,0.407818381473059,&#xA;                       0.406187518931948,-0.405573831841341,-0.407806403622317,&#xA;                       -0.418351108232386,-0.419107645158175,-0.420781760133428,&#xA;                       -0.441930843508614,-0.444723954813865,-0.451224756237618,&#xA;                       -0.504642907784859,-0.515843696383127,-0.523823484265933,&#xA;                       1.01203740887784,0.860911808112254,0.860911808112254,&#xA;                       0.782824395343453,0.779871038531055,0.763412247455786,&#xA;                       0.763412247455786,0.744965434413444,0.735086809796288,&#xA;                       0.733132943355509,0.731855318166865,0.730523378560539,&#xA;                       0.728356885227315,0.694487466356475,0.694335496605963,&#xA;                       0.68469489116557,0.680753784497535,0.680732580641012,&#xA;                       0.668909750729078,0.665107737526883,0.662352062210176,&#xA;                       0.643767077562848,0.640204416487105,0.639460661647805,&#xA;                       0.637911093993277),&#xA;                se = c(0.10858902830706,&#xA;                       0.101375498952004,0.101193730594653,0.101213265518358,&#xA;                       0.101165178275411,0.101149604926044,0.101114528367103,&#xA;                       0.101177617463096,0.101176196339295,0.10102421433221,&#xA;                       0.10102421433221,0.100910703206783,0.101046445558438,&#xA;                       0.12265509469116,0.10486856029419,0.103267024648877,0.100867566762125,&#xA;                       0.11467836793049,0.10790947678297,0.100997454994136,&#xA;                       0.101684644439116,0.103275688295495,0.101358559584742,&#xA;                       0.101485340026678,0.131050876787633,0.12672527384678,&#xA;                       0.0911987727585976,0.0911987727585976,0.0473284627336248,&#xA;                       0.120947206065826,0.120594420271743,0.120594420271743,&#xA;                       0.120207965256539,0.12000489747134,0.119965054342127,&#xA;                       0.0890575546896603,0.0468935746714197,&#xA;                       0.0890043416222531,0.119198789457309,0.0466102635462743,&#xA;                       0.119011209638725,0.0883056120686039,0.0883053113523379,&#xA;                       0.0881390968551948,0.0880862640207957,0.118593197976584,&#xA;                       0.0877953030586674,0.118192523540133,0.0462081105569776,&#xA;                       0.0461972348759043),&#xA;           country = c(&amp;quot;metaanalysis&amp;quot;,&amp;quot;metaanalysis&amp;quot;,&#xA;                       &amp;quot;metaanalysis&amp;quot;,&amp;quot;metaanalysis&amp;quot;,&amp;quot;metaanalysis&amp;quot;,&#xA;                       &amp;quot;metaanalysis&amp;quot;,&amp;quot;metaanalysis&amp;quot;,&amp;quot;metaanalysis&amp;quot;,&amp;quot;metaanalysis&amp;quot;,&#xA;                       &amp;quot;metaanalysis&amp;quot;,&amp;quot;metaanalysis&amp;quot;,&amp;quot;metaanalysis&amp;quot;,&#xA;                       &amp;quot;metaanalysis&amp;quot;,&amp;quot;metaanalysis&amp;quot;,&amp;quot;metaanalysis&amp;quot;,&amp;quot;metaanalysis&amp;quot;,&#xA;                       &amp;quot;metaanalysis&amp;quot;,&amp;quot;metaanalysis&amp;quot;,&amp;quot;metaanalysis&amp;quot;,&#xA;                       &amp;quot;metaanalysis&amp;quot;,&amp;quot;metaanalysis&amp;quot;,&amp;quot;metaanalysis&amp;quot;,&amp;quot;metaanalysis&amp;quot;,&#xA;                       &amp;quot;metaanalysis&amp;quot;,&amp;quot;metaanalysis&amp;quot;,&amp;quot;Swedish&amp;quot;,&amp;quot;Kazakh&amp;quot;,&amp;quot;Kazakh&amp;quot;,&#xA;                       &amp;quot;Japanese&amp;quot;,&amp;quot;Swedish&amp;quot;,&amp;quot;Swedish&amp;quot;,&amp;quot;Swedish&amp;quot;,&amp;quot;Swedish&amp;quot;,&#xA;                       &amp;quot;Swedish&amp;quot;,&amp;quot;Swedish&amp;quot;,&amp;quot;Kazakh&amp;quot;,&amp;quot;Japanese&amp;quot;,&amp;quot;Kazakh&amp;quot;,&#xA;                       &amp;quot;Swedish&amp;quot;,&amp;quot;Japanese&amp;quot;,&amp;quot;Swedish&amp;quot;,&amp;quot;Kazakh&amp;quot;,&amp;quot;Kazakh&amp;quot;,&#xA;                       &amp;quot;Kazakh&amp;quot;,&amp;quot;Kazakh&amp;quot;,&amp;quot;Swedish&amp;quot;,&amp;quot;Kazakh&amp;quot;,&amp;quot;Swedish&amp;quot;,&amp;quot;Japanese&amp;quot;,&#xA;                       &amp;quot;Japanese&amp;quot;)&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Output:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OmtLi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OmtLi.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="7427" LastEditorUserId="7427" LastEditDate="2021-04-26T10:02:35.420" LastActivityDate="2021-04-26T14:53:47.247" Title="How to increase the size of the shapes that denotes effect sizes of different datasets and metaanalysis?" Tags="|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15805" PostTypeId="1" CreationDate="2021-04-26T14:01:42.683" Score="2" ViewCount="1067" Body="&lt;p&gt;I am aware of &lt;a href=&quot;https://biology.stackexchange.com/questions/64860/what-is-the-length-of-gene-when-calculating-tpm-transcripts-per-million&quot;&gt;this&lt;/a&gt; similar question. But the accepted answer there answers how to calculate TPM given a mapping from gene name to gene length.&lt;/p&gt;&#xA;&lt;p&gt;My question is, given an annotation file of genes (for example from Ensembl &lt;a href=&quot;ftp://ftp.ensembl.org/pub/current_gtf/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), how do you calculate gene length for the purpose of calculating TPM on a gene count matrix coming from RNA seq data. That is, I understand the actualy calculation of TPM given gene lengths, but I do not understand what the gene lengths are.&lt;/p&gt;&#xA;&lt;p&gt;From what I understand, to calculate gene lengths from annotation file for RNA seq data, you only consider the exons of each gene, but even so, many exons overlap. What is the standard method of dealing with this?&lt;/p&gt;&#xA;&lt;p&gt;For example, I have read that using the length of the union of all exons of the gene is acceptable, but from what I understand TPM is meant to represent the &amp;quot;expected number of reads coming from gene X when randomly selecting 1e6 transcripts&amp;quot;. So taking the union of all exons will be an estimate of this but it might under-estimate this.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: From what I can tell this question is not a duplicate of &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/10845/rnaseq-normalization-stabilization-gene-length-and-rlog&quot;&gt;RNASeq: Normalization, stabilization, gene length and rlog&lt;/a&gt;&#xA;That question does discuss how to perform normalization that takes gene length into account, but it does not ask &amp;quot;what is gene length? How does one determine the length of each gene?&amp;quot; Possible answers could be like what I already mentioned: length of the union of all exons in the gene, or another possible answer is one from the linked question &amp;quot;length of the longest transcript isoform&amp;quot; but I feel that it is not really discussed in that answer/question.&lt;/p&gt;&#xA;" OwnerUserId="9095" LastEditorUserId="9095" LastEditDate="2021-04-28T08:17:12.050" LastActivityDate="2021-04-28T09:38:12.710" Title="What are common ways to calculate gene length for TPM calculations?" Tags="|rna-seq|gene-expression|tpm|" AnswerCount="1" CommentCount="0" ClosedDate="2021-04-28T10:22:23.540" ContentLicense="CC BY-SA 4.0" />
  <row Id="15806" PostTypeId="2" ParentId="15804" CreationDate="2021-04-26T14:53:47.247" Score="1" Body="&lt;p&gt;A quick look at the code for forestplot shows that geom_effect is what needs to be modified. If you add a value for fatten changes the size of the shapes in the the figure (but not in the legend), or a value for size will change the shape, line and legend sizes (add to not geom_effect not to the aes layer)&lt;/p&gt;&#xA;&lt;p&gt;This may be easier said than done as forestplot also seems to call an unexposed function (function quo_is_null). A couple of simple hacks are possible; copy and make your own forestplot in the default environment and remove the call to the function (it seems to just check if pvalue is set), or change the environment of the &amp;quot;new&amp;quot; forestplot to ggforestplot&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;envorionment(forestplot) &amp;lt;- envornment(ggforestplot::forestplot)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There may be other less awkward methods, but this will work. Fatten changed to size 6 which is massive.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;forestplot &amp;lt;- &#xA;function (df, name = name, estimate = estimate, se = se, pvalue = NULL, &#xA;          colour = NULL, shape = NULL, logodds = FALSE, psignif = 0.05, &#xA;          ci = 0.95, ...) &#xA;{&#xA;  stopifnot(is.data.frame(df))&#xA;  stopifnot(is.logical(logodds))&#xA;  name &amp;lt;- enquo(name)&#xA;  estimate &amp;lt;- enquo(estimate)&#xA;  se &amp;lt;- enquo(se)&#xA;  pvalue &amp;lt;- enquo(pvalue)&#xA;  colour &amp;lt;- enquo(colour)&#xA;  shape &amp;lt;- enquo(shape)&#xA;  args &amp;lt;- list(...)&#xA;  const &amp;lt;- stats::qnorm(1 - (1 - ci)/2)&#xA;  df &amp;lt;- df %&amp;gt;% dplyr::mutate(`:=`(!!name, factor(!!name, levels = !!name %&amp;gt;% &#xA;                                                   unique() %&amp;gt;% rev(), ordered = TRUE)), .xmin = !!estimate - &#xA;                               const * !!se, .xmax = !!estimate + const * !!se, .filled = TRUE, &#xA;                             .label = sprintf(&amp;quot;%.2f&amp;quot;, !!estimate))&#xA;  if (logodds) {&#xA;    df &amp;lt;- df %&amp;gt;% mutate(.xmin = exp(.data&lt;span class=&quot;math-container&quot;&gt;$.xmin), .xmax = exp(.data$&lt;/span&gt;.xmax), &#xA;                        `:=`(!!estimate, exp(!!estimate)))&#xA;  }&#xA;  if (!quo_is_null(pvalue)) {&#xA;    df &amp;lt;- df %&amp;gt;% dplyr::mutate(.filled = !!pvalue &amp;lt; !!psignif)&#xA;  }&#xA;  g &amp;lt;- ggplot2::ggplot(df, aes(x = !!estimate, y = !!name))&#xA;  if (logodds) {&#xA;    if (&amp;quot;xtickbreaks&amp;quot; %in% names(args)) {&#xA;      g &amp;lt;- g + scale_x_continuous(trans = &amp;quot;log10&amp;quot;, breaks = args&lt;span class=&quot;math-container&quot;&gt;$xtickbreaks)&#xA;    }&#xA;    else {&#xA;      g &amp;lt;- g + scale_x_continuous(trans = &quot;log10&quot;, breaks = scales::log_breaks(n = 7))&#xA;    }&#xA;  }&#xA;  g &amp;lt;- g + theme_forest() + scale_colour_ng_d() + scale_fill_ng_d() + &#xA;    geom_stripes() + geom_vline(xintercept = ifelse(test = logodds, &#xA;                                                    yes = 1, no = 0), linetype = &quot;solid&quot;, size = 0.4, colour = &quot;black&quot;)&#xA;  g &amp;lt;- g + geom_effect(ggplot2::aes(xmin = .data$&lt;/span&gt;.xmin, xmax = .data&lt;span class=&quot;math-container&quot;&gt;$.xmax, &#xA;                                    colour = !!colour, shape = !!shape, filled = .data$&lt;/span&gt;.filled), &#xA;                       position = ggstance::position_dodgev(height = 0.5),&#xA;                       fatten=6) + &#xA;    ggplot2::scale_shape_manual(values = c(21L, 22L, 23L, &#xA;                                           24L, 25L)) + guides(colour = guide_legend(reverse = TRUE), &#xA;                                                               shape = guide_legend(reverse = TRUE))&#xA;  if (&amp;quot;title&amp;quot; %in% names(args)) {&#xA;    g &amp;lt;- g + labs(title = args&lt;span class=&quot;math-container&quot;&gt;$title)&#xA;  }&#xA;  if (&quot;subtitle&quot; %in% names(args)) {&#xA;    g &amp;lt;- g + labs(subtitle = args$subtitle)&#xA;  }&#xA;  if (&quot;caption&quot; %in% names(args)) {&#xA;    g &amp;lt;- g + labs(caption = args$caption)&#xA;  }&#xA;  if (&quot;xlab&quot; %in% names(args)) {&#xA;    g &amp;lt;- g + labs(x = args$xlab)&#xA;  }&#xA;  if (!&quot;ylab&quot; %in% names(args)) {&#xA;    args$ylab &amp;lt;- &quot;&quot;&#xA;  }&#xA;  g &amp;lt;- g + labs(y = args$&lt;/span&gt;ylab)&#xA;  if (&amp;quot;xlim&amp;quot; %in% names(args)) {&#xA;    g &amp;lt;- g + coord_cartesian(xlim = args&lt;span class=&quot;math-container&quot;&gt;$xlim)&#xA;  }&#xA;  if (&quot;ylim&quot; %in% names(args)) {&#xA;    g &amp;lt;- g + ylim(args$ylim)&#xA;  }&#xA;  if (&quot;xtickbreaks&quot; %in% names(args) &amp;amp; !logodds) {&#xA;    g &amp;lt;- g + scale_x_continuous(breaks = args$&lt;/span&gt;xtickbreaks)&#xA;  }&#xA;  g&#xA;}&#xA;environment(forestplot) &amp;lt;- environment(ggforestplot::forestplot)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8919" LastActivityDate="2021-04-26T14:53:47.247" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15808" PostTypeId="2" ParentId="15805" CreationDate="2021-04-26T17:09:54.340" Score="1" Body="&lt;p&gt;I think the most common way is to use something like Kallisto which figures out what proportion of reads are likely assigned to each transcript variant, and calculated TPM  based on that.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2021-04-26T17:09:54.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15810" PostTypeId="1" AcceptedAnswerId="15811" CreationDate="2021-04-26T20:56:54.800" Score="2" ViewCount="74" Body="&lt;p&gt;I have a text file that contains a list of IDs (314 sequences):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;AVP78031.1&#xA;AVP78042.1&#xA;ATO98108.1&#xA;ATO98120.1&#xA;ATO98132.1&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My goal is to make a script (maybe using Python or Perl) to check in the list if all the IDs are nucleotide or protein sequences.&#xA;For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;AVP78031.1 -&amp;gt; protein (this is a nucleotide sequence, I change nucleotide for protein to show an example).&#xA;AVP78042.1 -&amp;gt; nucleotide&#xA;ATO98108.1 -&amp;gt; nucleotide&#xA;ATO98120.1 -&amp;gt; nucleotide&#xA;ATO98132.1 -&amp;gt; nucleotide&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any idea to do a script?&lt;/p&gt;&#xA;&lt;p&gt;Thank everybody!&lt;/p&gt;&#xA;" OwnerUserId="10557" LastActivityDate="2021-04-27T11:34:00.910" Title="How to know if ID correspond to nucleotide sequence or protein with a script" Tags="|python|perl|sequence|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15811" PostTypeId="2" ParentId="15810" CreationDate="2021-04-26T21:25:12.763" Score="2" Body="&lt;p&gt;If these are all GenBank or RefSeq accessions, you can use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez Direct&lt;/a&gt; for this as shown below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat accs.txt &#xA;ATO98108.1&#xA;ATO98120.1&#xA;ATO98132.1&#xA;AVP78031.1&#xA;AVP78042.1&#xA;$ cat accs.txt | epost -db nuccore | efetch -format acc&#xA;## no output because none of them are nucleotide accessions&#xA;$ cat accs.txt | epost -db protein -format acc | efetch -format acc&#xA;AVP78042.1&#xA;AVP78031.1&#xA;ATO98132.1&#xA;ATO98120.1&#xA;ATO98108.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;NOTE&lt;/strong&gt;: This will work only if the accessions are currently live because &lt;code&gt;epost&lt;/code&gt; does not find any suppressed accessions. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat accs.txt &#xA;NM_002826.3&#xA;NM_002826.4&#xA;NM_002826.5&#xA;$ cat accs.txt | epost -db nuccore -format acc | efetch -format acc&#xA;NM_002826.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here, all three accessions are valid nucleotide accessions but only the last one, NM_002826.5, is alive.&lt;/p&gt;&#xA;&lt;p&gt;An alternate way is to use the accession prefixes defined &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genbank/acc_prefix/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and come up with an appropriate regular expression query.&lt;/p&gt;&#xA;" OwnerUserId="3577" LastEditorUserId="3577" LastEditDate="2021-04-27T11:34:00.910" LastActivityDate="2021-04-27T11:34:00.910" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15812" PostTypeId="1" CreationDate="2021-04-27T13:44:37.240" Score="3" ViewCount="761" Body="&lt;p&gt;I just tried to use Vina for calculating binding affinity.&#xA;I used the same protein, the same ligand, the same options, but there are different results.&#xA;Why do I get different results?&#xA;Is it the limit Point of docking tools?&#xA;Then why we use it?&lt;/p&gt;&#xA;" OwnerUserId="10090" LastActivityDate="2023-11-29T15:06:11.503" Title="Why Do I Get Different Results Each Time From AutoDock Vina?" Tags="|docking|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15813" PostTypeId="1" AcceptedAnswerId="15818" CreationDate="2021-04-28T02:22:13.703" Score="2" ViewCount="618" Body="&lt;p&gt;I have a group of about 1050 sequences that have been aligned using blastn. I am now trying to create a cluster plot of the sequences in order to identify similar sequences, something like this:&#xA;&lt;img src=&quot;https://www.datanovia.com/en/wp-content/uploads/2020/06/k-means-clustering-visualization-in-r-plot-k-means-in-r-1.png&quot; alt=&quot;Text&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, I am having trouble finding a tool to do this. My professor recommended using ClustalW, and so I have spent plenty of time searching for a tool that can take a sequence alignment and plot them such as in the linked image. However, I have not been successful in finding what I need and am beginning to think I am missing something or something is going over my head. If anyone could assist with finding the tool I need I would very much appreciate it. Thanks so much!&lt;/p&gt;&#xA;" OwnerUserId="12090" LastEditorUserId="73" LastEditDate="2021-04-28T23:37:34.640" LastActivityDate="2021-04-29T15:52:54.400" Title="How do I plot clusters based on DNA sequence alignment?" Tags="|visualization|clustering|multiple-sequence-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15814" PostTypeId="2" ParentId="14740" CreationDate="2021-04-28T03:14:10.780" Score="1" Body="&lt;p&gt;It's possible to do 5' cap enrichment sequencing (e.g. see &lt;a href=&quot;https://elifesciences.org/articles/49658#s2-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), and presumably you could look at differences between an enriched vs non-enriched sequencing run to determine whether or not there's a substantial number of sequences with the cap.&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, because the cap is right at the end of the sequence, it zips through a nanopore very quickly, so may be quite difficult to distinguish from any other base at the end. There are also no official models that have been trained to specifically detect the cap, so you'd be initially restricted to looking for signal deviations. If you could find a way to extend the sequence for 20 or so based beyond the cap site, it might be possible to detect it at a single-molecule level.&lt;/p&gt;&#xA;&lt;p&gt;Despite all that, this is something that UCSC is working on; see their repository here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/mitenjain/dRNA_capping_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mitenjain/dRNA_capping_analysis&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-04-28T03:14:10.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15815" PostTypeId="2" ParentId="15812" CreationDate="2021-04-28T04:57:29.510" Score="2" Body="&lt;p&gt;Vina uses a non-deterministic docking algorithm. See the &lt;a href=&quot;http://vina.scripps.edu/manual.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The docking algorithm is non-deterministic. Even though with this receptor-ligand pair, the minimum of the scoring function corresponds to the correct conformation, the docking algorithm sometimes fails to find it. Try several times and see for yourself. Note that the probability of failing to find the mininum may be different with a different system.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The manual states that you can set the random seed in order to obtain reproducibility.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I changed something, and now the docking results are different. Why?&#xA;Firstly, had you not changed anything, some results could have been different anyway, due to the non-deterministic nature of the search algorithm. Exact reproducibility can be assured by supplying the same random seed to both calculations, but only if all other inputs and parameters are the same as well. Even minor changes to the input can have an effect similar to a new random seed. What does make sense discussing are the statistical properties of the calculations: e.g. &amp;quot;with the new protonation state, Vina is much less likely to find the correct docked conformation&amp;quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="6628" LastActivityDate="2021-04-28T04:57:29.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15816" PostTypeId="2" ParentId="15765" CreationDate="2021-04-28T07:57:53.840" Score="0" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If you have different read length then trim all samples to the shortest length across batches, that will avoid mappability bias. Including it into the DE design is possible but as you can completely eliminate this bias in silico by trimming this is what I'd do.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sequencing on different machines is usually not much of a confounding factor, but the fact that two machines were involved suggest different batches of experiments. I hope that batch is not confounded by group so each batch should contain samples of all groups to avoid a general batch effect that cannot be corrected for. Library prep should (in a well-designed experiment) be fully identical.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use any of the established methods, be it RLE or TMM (they usually perform ery similar), and then perform PCA based on the top variable log2-transformed counts to explore whether there are batch effects that are worth correcting, so whether you see clustering or differences based on batch rather than experimental design in the early PCs. If so (given it is not confounded by experimental groups) you can include it into the DE design as a covariate.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;As said by others, normalization and batch correction are independent processes. Also as I said in on top, you can eliminate read length bias (even though I doubt it is a major factor here) in silico which is what I'd recommend so you do not have to bother with it downstream.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-04-28T07:57:53.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15817" PostTypeId="2" ParentId="15812" CreationDate="2021-04-28T09:09:42.927" Score="1" Body="&lt;p&gt;The answer by @Throckmorton is 100% correct in that it's not deterministic. I just wanted to address a few things —so this is a long comment.&lt;/p&gt;&#xA;&lt;p&gt;All too often the box is set as the whole protein, this is bound to make it trip up on local minima. Sure, with native ligands, binding pockets close to the active site increases the dissociation constant of the complex, but docking is not the way to calculate these. Generally this is done by tethered MD, where the ligand is pulled out.&lt;/p&gt;&#xA;&lt;p&gt;Docking returns generally either a &lt;strong&gt;binding potential&lt;/strong&gt; (∆∆G_bind) as a kcal/mol (Autodock) or kJ/mol, where negative is good, or a rank. This is not the typically binding affinity measurements, e.g. an IC50 or dissociation constant (K_D) —The Eyring equation converted in a relative form (ratio makes constants disappear) allows you do some brute conversions to a k-on for the site, but this is highly not recommended.&lt;/p&gt;&#xA;&lt;p&gt;Docking is not perfect and, worse, gets abused a lot. However I'd flip the OP's statement around and say the limitation by far is wet lab side, where doing an ITC properly is hard and crystallising a protein with a ligand is even worse...&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2023-11-29T15:06:11.503" LastActivityDate="2023-11-29T15:06:11.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15818" PostTypeId="2" ParentId="15813" CreationDate="2021-04-28T10:01:12.440" Score="2" Body="&lt;p&gt;I think one of the reasons you struggle is that clustering DNA sequences is not a clearly defined task. In general intention of clustering is to reconstruct, or approximate the relatidness of the DNA seqeunces.&lt;/p&gt;&#xA;&lt;p&gt;If all these sequences are homologous you can do a multiple sequence alignment (using &lt;a href=&quot;https://mafft.cbrc.jp/alignment/software/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MAFFT&lt;/a&gt; or Clustal) and build a phylogenetic tree (using something like &lt;a href=&quot;http://www.iqtree.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IQ tree&lt;/a&gt;). If your prof suggested ClustalW, this is probably what they meant. Evolutionary speaking, that's the optimal method for getting relations of sequences, but they must be &amp;quot;alignable&amp;quot;. Also with this many sequences, it will be computationally quite heavy.&lt;/p&gt;&#xA;&lt;p&gt;For within species relatedness, phylogenetic reconstruction is sometimes impossible, because recombination breaks down the linkage of individual alleles. However, if all of the sequences are long enough, you can do a principal component analysis on SNPs, for example using something like &lt;a href=&quot;https://speciationgenomics.github.io/pca/&quot; rel=&quot;nofollow noreferrer&quot;&gt;plink&lt;/a&gt;. The plot you showed actually looks a lot like PCA (it even has the explained variability on axes).&lt;/p&gt;&#xA;&lt;p&gt;Finally, regardless of their relationship, you can always do a dendrogram based on a similarity matrix, but for that, you need to define similarity. That will be usually something kmer-based. I am not following this field very closely lately, but there are some recent papers about kmer-based clusterings, like &lt;a href=&quot;https://www.nature.com/articles/s41598-019-40452-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;When looking for the paper I also run into &lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fgene.2019.00234/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;, which proposes genome clustering using &amp;quot;Inter-nucleotide Covariance&amp;quot;. No idea how useful/sound it is. Just found the abstract quite interesting.&lt;/p&gt;&#xA;&lt;p&gt;Long story short, it really depends on what you are after and what are your sequences! If you can build a reliable tree, it is the most powerful way to determine the relationships among sequences.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2021-04-29T15:52:54.400" LastActivityDate="2021-04-29T15:52:54.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15819" PostTypeId="2" ParentId="15676" CreationDate="2021-04-28T11:24:33.637" Score="2" Body="&lt;p&gt;As others said, there already exist javascript libraries to visualize a protein. The most complete and recent one would probably be &lt;a href=&quot;https://molstar.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mol*&lt;/a&gt;, which is what is used on RCSB PDB website. Mol* actually works using &lt;a href=&quot;https://nglviewer.org/#ngl&quot; rel=&quot;nofollow noreferrer&quot;&gt;NGL&lt;/a&gt;, a WebGL engine. But probably the oldest library used on RCSB PDB to visualize proteins would be &lt;a href=&quot;https://jsmol.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;JSMol&lt;/a&gt;. You can actually switch between these viewers on RCSB PDB 3D view using &amp;quot;Select a different viewer&amp;quot; in the bottom right (&lt;a href=&quot;https://www.rcsb.org/3d-view/1ubq&quot; rel=&quot;nofollow noreferrer&quot;&gt;try it out on 1UBQ&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;Now to address the first exercise: there are several things going on after getting the atom coordinates. A PDB file may or may not contain a CONECT part, where bonds between atoms will be clearly stated (contrarily to a CIF file which I believe always contain the bonds). Henceforth these packages may or may not have to guess atom bonds. Given that a PDB file also contains the residues of each atom it's fairly easy to do, less so when dealing with small molecules.&lt;/p&gt;&#xA;&lt;p&gt;However what you see most of the time is a cartoon representation of proteins, where each secondary structure is represented differently. The library will compute the &lt;a href=&quot;https://en.wikipedia.org/wiki/Protein_secondary_structure&quot; rel=&quot;nofollow noreferrer&quot;&gt;secondary structure&lt;/a&gt; of each residue based on the dihedral angles of the backbone, then represent them. As of why they look like this, check the work of &lt;a href=&quot;http://pdb101.rcsb.org/sci-art/geis-archive/irving-geis&quot; rel=&quot;nofollow noreferrer&quot;&gt;Irving Geis&lt;/a&gt; and &lt;a href=&quot;https://en.wikipedia.org/wiki/Jane_S._Richardson&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jane Richardson&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So if you want to have a website showing protein structures I would strongly advise to use an already existing library, although the exercise of coding your own can always be interesting. If you want to visualize structures on your own computer without using a web interface I can only recommend &lt;a href=&quot;https://pymol.org/2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyMOL&lt;/a&gt;, &lt;a href=&quot;https://www.ks.uiuc.edu/Research/vmd/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VMD&lt;/a&gt; and &lt;a href=&quot;https://www.cgl.ucsf.edu/chimerax/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ChimeraX&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="12054" LastEditorUserId="12054" LastEditDate="2021-04-29T08:34:20.220" LastActivityDate="2021-04-29T08:34:20.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15820" PostTypeId="1" CreationDate="2021-04-28T17:17:45.093" Score="0" ViewCount="52" Body="&lt;p&gt;I am looking for a probability distribution of a number of reads mapped to a particular gene in metagenomic sequencing (NGS, shotgun, likely illumina).&lt;/p&gt;&#xA;&lt;p&gt;Naively one could model it via a binomial (or Poisson) distribution as:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;P_n = {N\choose n}\pi^n(1-\pi)^{N-n},&#xA;$$&lt;/span&gt;&#xA;where&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is the number of reads mapped to the gene&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is the sequencing depth (total number of reads)&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$\pi\propto l$&lt;/span&gt; is the probability of read mapping to the particular gene, proportioal to the gene length (possibly adjusted for the read length)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This is however complicated by such intricacies of sequencing as:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;finite length of the fragments, which can be longer or shorter than the gene length&lt;/li&gt;&#xA;&lt;li&gt;paired-end sequencing, which means one has to be careful when counting evens where only one or both reads map to a gene&lt;/li&gt;&#xA;&lt;li&gt;something else?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Is there an established model for this case? Could you recommend reading on the subject?&lt;/p&gt;&#xA;" OwnerUserId="1813" LastActivityDate="2021-04-28T23:28:19.220" Title="Modeling number of reads mapped to a gene" Tags="|ngs|read-mapping|reads|illumina|genome-sequencing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15821" PostTypeId="1" CreationDate="2021-04-28T19:33:41.680" Score="3" ViewCount="88" Body="&lt;p&gt;Is there a method or tool to convert a .smp file from CDD (Protein Domain File) into a HMM file? I downloaded the CDD files from the FTP site and am more familiar with domain searching with the HMMer suite.&lt;/p&gt;&#xA;" OwnerUserId="1833" LastEditorUserId="15806" LastEditDate="2022-07-21T22:25:34.953" LastActivityDate="2022-07-21T22:25:34.953" Title="Convert .smp downloaded from NCBI CDD into HMM(.hmm) file" Tags="|proteins|phylogenetics|hmmer|domains|" AnswerCount="0" CommentCount="0" ClosedDate="2021-06-25T11:08:24.517" ContentLicense="CC BY-SA 4.0" />
  <row Id="15822" PostTypeId="1" CreationDate="2021-04-28T21:37:56.190" Score="2" ViewCount="136" Body="&lt;p&gt;I have shallow whole genome sequence data that I used the tool iChorCNA to find the mean copy number segmentation per region. The tool produces a seg file and can be seen here: &lt;a href=&quot;https://github.com/broadinstitute/ichorCNA&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/broadinstitute/ichorCNA&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have also run these same samples to higher coverage on a targeted panel and used the tool VisCap Cancer to find the mean copy number segmentation. The tool can be found here and also produces a seg file: &lt;a href=&quot;https://github.com/pughlab/VisCapCancer&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/pughlab/VisCapCancer&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am looking to compare the outputs of each tool to see how close they overlap with each other. The problem is that iChorCNA produces outputs in bins that span the genome per million bases, and VisCap Cancer is only looking at the sites for targetted probes. These probes are anywhere from 1.3 million basepairs to 260 million basepairs in length.&lt;/p&gt;&#xA;&lt;p&gt;What I am looking to do is segment the VisCap output into bins that match the bins used by iChorCNA. I want to then take the sum of the mean value given by iChorCNA at each bin that overlaps with the VisCapCancer results, and divide it by the total number of bins that fall in the range of the results to find the average iChorCNA reading per VisCap reading. Then, I can do a t-test, ANOVA or other measure to see how closely these two numbers are to each other.&lt;/p&gt;&#xA;&lt;p&gt;I was wondering if anyone has directly compared copy number results from two data sources such as this before, such as shallow whole genome sequencing against a targeted panel, and if they have any advice as to how to go about this most effectively - including if it would be better to do the comparison using R, python or another language, and best practices for setting up the comparison most effectively.&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your time&lt;/p&gt;&#xA;" OwnerUserId="12098" LastActivityDate="2021-04-28T21:37:56.190" Title="What are the best practices for comparing .seg copy number outputs from iChorCNA to VisCapCancer?" Tags="|r|statistics|copy-number|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15823" PostTypeId="2" ParentId="15820" CreationDate="2021-04-28T23:28:19.220" Score="2" Body="&lt;p&gt;This is quite a similar question to &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/10845/rnaseq-normalization-stabilization-gene-length-and-rlog&quot;&gt;this one&lt;/a&gt;, but I notice that you're specifically asking about a &lt;em&gt;probability distribution&lt;/em&gt; rather than differential expression. That sounds like gene dispersion and/or shot noise; a good reference for me on that has been the documentation for DESeq2:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The gene dispersion issue is discussed in more detail in their 2014 paper:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1186/s13059-014-0550-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1186/s13059-014-0550-8&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We model read counts K ij as following a negative binomial distribution (sometimes also called a gamma-Poisson distribution) with mean μ ij and dispersion α i . The mean is taken as a quantity q ij, proportional to the concentration of cDNA fragments from the gene in the sample, scaled by a normalization factor s ij , i.e., μ ij =s ij q ij . For many applications, the same constant s j can be used for all genes in a sample, which then accounts for differences in sequencing depth between samples. To estimate these size factors, the DESeq2 package offers the median-of-ratios method already used in DESeq. However, it can be advantageous to calculate gene-specific normalization factors s ij to account for further sources of technical biases such as differing dependence on GC content, gene length or the like, using published methods, and these can be supplied instead.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-04-28T23:28:19.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15825" PostTypeId="1" CreationDate="2021-04-29T08:53:23.563" Score="1" ViewCount="214" Body="&lt;p&gt;I want to use rna seq data to later perform functional tests on fusion genes.&#xA;so before that I need to filter the &amp;quot;best results&amp;quot; (of rnaseq) for deciding which candidates I actually want to try in the lab.&#xA;one criterium is the split read. (among coverage, discordant mate and confidence)&lt;/p&gt;&#xA;&lt;p&gt;In some cases the split read (or discordant mate/coverage) of one of the two genes that fusioned is &amp;quot;0&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Does that automatically mean that the gene is no longer a candidate for functional testing?&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately I only got the results in an excel data sheet and don't know which tool was used to gain data.&lt;/p&gt;&#xA;" OwnerUserId="12100" LastEditorUserId="73" LastEditDate="2021-04-29T21:41:01.823" LastActivityDate="2021-09-26T22:04:10.347" Title="What is the meaning of split read?" Tags="|rna-seq|sequence-alignment|read-mapping|rna-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15826" PostTypeId="2" ParentId="15800" CreationDate="2021-04-29T10:04:16.103" Score="4" Body="&lt;p&gt;From GROMACS documentation about &lt;a href=&quot;https://manual.gromacs.org/current/user-guide/managing-simulations.html#reproducibility&quot; rel=&quot;nofollow noreferrer&quot;&gt;reproducibility&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The following factors affect the reproducibility of a simulation, and&#xA;thus its output:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Precision (mixed / double) with double giving “better” reproducibility.&#xA;Number of cores, due to different order in which forces are accumulated. For instance (a+b)+c is not necessarily binary identical&#xA;to a+(b+c) in floating-point arithmetic.&lt;/li&gt;&#xA;&lt;li&gt;Type of processors. Even within the same processor family there can be slight differences.&lt;/li&gt;&#xA;&lt;li&gt;Optimization level when compiling.&lt;/li&gt;&#xA;&lt;li&gt;Optimizations at run time: e.g. the FFTW library that is typically used for fast Fourier transforms determines at startup which version&#xA;of their algorithms is fastest, and uses that for the remainder of the&#xA;calculations. Since the speed estimate is not deterministic, the&#xA;results may vary from run to run.&lt;/li&gt;&#xA;&lt;li&gt;Dynamic linking to different versions of shared libraries (e.g. for FFTs)&lt;/li&gt;&#xA;&lt;li&gt;Dynamic load balancing, since particles are redistributed to processors based on elapsed wallclock time, which will lead to (a+b)+c&#xA;!= a+(b+c) issues as above&lt;/li&gt;&#xA;&lt;li&gt;Number of PME-only ranks (for parallel PME simulations)&lt;/li&gt;&#xA;&lt;li&gt;MPI reductions typically do not guarantee the order of the operations, and so the absence of associativity for floating-point&#xA;arithmetic means the result of a reduction depends on the order&#xA;actually chosen&lt;/li&gt;&#xA;&lt;li&gt;On GPUs, the reduction of e.g. non-bonded forces has a non-deterministic summation order, so any fast implementation is&#xA;non-reprodudible by design.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I would add that in your case of hard coding everything by hand you may have to look a bit into your barostat and thermostat (although their influence should be found on speed of the atoms). Maybe a sample code with coordinates and speed of 4 atoms would help to give a more precise answer.&lt;/p&gt;&#xA;&lt;p&gt;You can also try having a look at &lt;a href=&quot;https://github.com/gromacs/gromacs&quot; rel=&quot;nofollow noreferrer&quot;&gt;gromacs source code&lt;/a&gt;. I believe &lt;a href=&quot;https://github.com/openmm/openmm&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenMM&lt;/a&gt; provides a bit more control and clarity on each step, if for some unfathomable reason you are not doing this only for curiosity sake but because you actually have to make a code that reproduces these results.&lt;/p&gt;&#xA;" OwnerUserId="12054" LastEditorUserId="12054" LastEditDate="2021-04-29T13:46:45.113" LastActivityDate="2021-04-29T13:46:45.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15827" PostTypeId="2" ParentId="15676" CreationDate="2021-04-29T11:17:42.027" Score="2" Body="&lt;p&gt;I am not good as you are and needed to use a parser for the pdb file&lt;/p&gt;&#xA;&lt;p&gt;see &lt;a href=&quot;https://mmcif.wwpdb.org/docs/software-resources.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://mmcif.wwpdb.org/docs/software-resources.html&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;# -*- coding: utf-8 -*-&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;Created on Thu Nov  7 17:40:03 2020&#xA;&#xA;@author: Pietro&#xA;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;&#xA;&#xA;import matplotlib.pyplot as plt  #matplotlib&#xA;&#xA;from pdbx.reader.PdbxReader import PdbxReader #PDB file parser&#xA;&#xA;import numpy as np #numpy&#xA;&#xA;&#xA;openz = open('./1msc.cif')&#xA;&#xA;print('opened')&#xA;&#xA;# Initialize a PdbxReader object with the input file handle&#xA;&#xA;pRd = PdbxReader(openz)&#xA;&#xA;# Initialize a list to be propagated with DataContainer (and/or DefinitionContainer) objects (of the DataContainer class, which inherits from ContainerBase) parsed from the CIF file, where data blocks map to DataContainer objects&#xA;&#xA;data = []&#xA;&#xA;print(data)&#xA;&#xA;# Call the read(self, containerList) method with your list&#xA;&#xA;pRd.read(data)&#xA;&#xA;&#xA;print ('data ###################################################################\n' , data)&#xA;       &#xA;print('data lenght' , len(data))&#xA;&#xA;# Your list is now propagated with one or more DataContainer objects, which represent data blocks. To get the first data block, just use list notation:&#xA;&#xA;block = data[0]&#xA;&#xA;print(' block ##############################################################\n',block)&#xA;      &#xA;&#xA;pippo=block.getObjNameList()&#xA;&#xA;print(pippo)&#xA;&#xA;# To retrieve a category object, use the getObj(self, name) method&#xA;&#xA;atomsite = block.getObj(&amp;quot;atom_site&amp;quot;)&#xA;&#xA;&#xA;&#xA;&#xA;print('atom site : ', atomsite)&#xA;&#xA;&#xA;atom = atomsite.getValue(&amp;quot;group_PDB&amp;quot;,0)&#xA;&#xA;&#xA;&#xA;# print('atom sites id :', atom)&#xA;&#xA;&#xA;atomlist=[]&#xA;&#xA;i=0&#xA;while True:&#xA;    atom =  atomsite.getValue(&amp;quot;group_PDB&amp;quot;,i)&#xA;    atomid = atomsite.getValue('label_atom_id',i)&#xA;    if atom =='ATOM':&#xA;        # print(atomsite.getValue(&amp;quot;group_PDB&amp;quot;,i))&#xA;        if atomid == 'CA':                         #select only carbon alpha (CA) to work with&#xA;&#xA;            atomlist.append([float(atomsite.getValue('Cartn_x',i)),\&#xA;                           float(atomsite.getValue('Cartn_y',i)),float(atomsite.getValue('Cartn_z',i))])&#xA;&#xA;    &#xA;        i +=1&#xA;    else:&#xA;        print('loop ends')&#xA;        False&#xA;        break&#xA;&#xA;&#xA;&#xA;atomlist=np.array(atomlist)&#xA;&#xA;print('end')&#xA;&#xA;print(atomlist)&#xA;&#xA;print('lunghezza : ', len(atomlist))&#xA;&#xA;&#xA;&#xA;&#xA;fig = plt.figure()&#xA;ax = plt.axes(projection=&amp;quot;3d&amp;quot;)&#xA;&#xA;&#xA;&#xA;for p in atomlist:       #print CA as dots&#xA;    ax.scatter3D(p[0], p[1], p[2])&#xA;    print(p)&#xA;&#xA;    &#xA;for i in range(0,len(atomlist)-1):   #connect with a line CA n with n+1&#xA;    x,y,z = [atomlist[i+1][0],atomlist[i][0]],[atomlist[i+1][1],atomlist[i][1]],[atomlist[i+1][2],atomlist[i][2]]&#xA;    ax.plot(x,y,z, color='red')&#xA;&#xA;&#xA;plt.show()&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;my input is &lt;a href=&quot;https://files.rcsb.org/download/1MSC.cif&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://files.rcsb.org/download/1MSC.cif&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Result is just the CAs (carbon alpha) shown as dots and connected to form the main chain:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2bmRT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2bmRT.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Didn't manage how to show the dimer though&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2021-04-29T11:32:29.343" LastActivityDate="2021-04-29T11:32:29.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15828" PostTypeId="1" CreationDate="2021-04-29T12:03:13.993" Score="5" ViewCount="53" Body="&lt;p&gt;I've got annotations for splicing variants in a format like this (this is one variant):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;Variant:&lt;/em&gt; NM_004092.3:c.88+5G&amp;gt;A&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Effect:&lt;/em&gt; Retention; r.87_88ins1_88+10:p.(Ala31Glufs*23)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I want to extract which acceptor/donor sites are lost or gained (in this case lost causing intron retention). In other words, the genomic location of the sequence that is retained in mRNA.&lt;/p&gt;&#xA;&lt;p&gt;I parsed the variant HGVS ID using ensembl VEP (like &lt;a href=&quot;https://rest.ensembl.org/vep/human/hgvs/NM_004092.3:c.88+5G%3EA?content-type=application/json&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;) without problems, giving me the genomic location of this variant. However, VEP seems to not like &amp;quot;r.&amp;quot; annotations for the effect: I tried ensembl VEP again with &lt;a href=&quot;https://rest.ensembl.org/vep/human/hgvs/NM_004092.3:r.87_88ins1_88+10?content-type=application/json&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNA&lt;/a&gt;, &lt;a href=&quot;https://rest.ensembl.org/vep/human/hgvs/NM_004092.3:p.(Ala31Glufs*23)?content-type=application/json&quot; rel=&quot;nofollow noreferrer&quot;&gt;protein&lt;/a&gt;, and &lt;a href=&quot;https://rest.ensembl.org/vep/human/hgvs/NM_004092.3:r.87_88ins1_88+10:p.(Ala31Glufs*23)?content-type=application/json&quot; rel=&quot;nofollow noreferrer&quot;&gt;both&lt;/a&gt; annotations, none of which worked.&lt;/p&gt;&#xA;&lt;p&gt;Is there a tool that can parse these annotations for me? I've got too many to do it by hand.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note on RNA splicing&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;RNA splicing and acceptor and donor sites:&lt;/p&gt;&#xA;&lt;p&gt;An introns contains: a 5' donor site, a branch site located near the 3' site and a 3' acceptor site. Both the donor and acceptor  are required for splicing along with the branch site.&lt;/p&gt;&#xA;" OwnerUserId="12101" LastEditorUserId="3848" LastEditDate="2022-03-24T14:36:09.887" LastActivityDate="2022-03-24T14:36:09.887" Title="Parse RNA variant effect annotations (&quot;r.&quot; format)" Tags="|rna|variation|rna-splicing|variants|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15829" PostTypeId="2" ParentId="15775" CreationDate="2021-04-29T12:37:56.593" Score="1" Body="&lt;p&gt;It's hard to say for sure without seeing the exact commands you used. However, Orthofinder identifies/defines single-copy orthologues as those genes present in a single copy in all species being compared in the analysis. If a BUSCO orthologue is present in more than one copy in one of the species being compared (could be multi-copy, could be fragmented etc.), then it won't be recovered as single copy by Orthofinder.&lt;/p&gt;&#xA;" OwnerUserId="12102" LastActivityDate="2021-04-29T12:37:56.593" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15830" PostTypeId="2" ParentId="15825" CreationDate="2021-04-29T21:39:48.270" Score="1" Body="&lt;p&gt;For discordant mapped reads, a position of 0 for the pair of a mapped read usually means that the pair couldn't be mapped anywhere. This would be expected for fusion events because some reads could originate from across the fusion site (i.e. not being fully contained in either &amp;quot;unfused&amp;quot; gene), leading to a sequence that doesn't match anything in the reference.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-04-29T21:39:48.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15831" PostTypeId="1" CreationDate="2021-04-29T22:49:05.543" Score="0" ViewCount="35" Body="&lt;p&gt;I want to do &lt;strong&gt;meta-analysis of alpha diversity&lt;/strong&gt; between test and control groups across 10 different populations. I want to see what is the summed effect (pool effect) of intervention between the two groups for alpha diversity. For that, first I need the intervention effect on test group's alpha diversity for individual population dataset. How should I calculate this? Should I calculate this by difference in the log transformed mean values of the groups (&lt;strong&gt;log2FC&lt;/strong&gt;) or should I calculate &lt;strong&gt;SMD&lt;/strong&gt; for each study?&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="7427" LastActivityDate="2021-04-29T22:49:05.543" Title="How to decide between standardized mean difference (SMD) and log2fold change (log2FC) before doing meta-analysis?" Tags="|statistics|meta-analysis|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15833" PostTypeId="2" ParentId="15732" CreationDate="2021-04-30T07:40:38.877" Score="1" Body="&lt;p&gt;The answer of the support stated that it is likely that the size of the first download is the problem and that I should try using another tool like rsync. Because I'm unfamiliar with rsync I decided to write a python script that did the job.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import urllib, tarfile, json&#xA;    &#xA;base_url = &amp;quot;https://ftp.ncbi.nlm.nih.gov/blast/db&amp;quot;&#xA;&#xA;# download manifest file to get the filenames&#xA;manifest_file = 'blastdb-manifest.json'&#xA;urllib.request.urlretrieve(f&amp;quot;{base_url}/{manifest_file}&amp;quot;, manifest_file)&#xA;&#xA;with open(manifest_file) as f:&#xA;    manifest_data = json.load(f)&#xA;&#xA;# download everything&#xA;for file in manifest_data['nr']['files']:&#xA;    # download checksum file&#xA;    checksum_file = f&amp;quot;{file}.md5&amp;quot;&#xA;    urllib.request.urlretrieve(f&amp;quot;{base_url}/{checksum_file}&amp;quot;, checksum_file)&#xA;    &#xA;    # download the archive file&#xA;    urllib.request.urlretrieve(f&amp;quot;{base_url}/{file}&amp;quot;, file)&#xA;    &#xA;    # check that the checksums match&#xA;    calculated_checksum = get_md5_for_file(file)&#xA;    &#xA;    with open(checksum_file) as f:&#xA;        for line in f:  &#xA;            line = line.strip()&#xA;            if line.endswith(file):&#xA;                checksum = re.split(&amp;quot;\s+&amp;quot;, line)[0]&#xA;                if checksum == calculated_checksum:&#xA;                    checksum_matches = True&#xA;                else:&#xA;                    raise Exception(f&amp;quot;Checksum doesn't match expected for file {file}&amp;quot;)&#xA;    &#xA;    # unpack the archive&#xA;    tar = tarfile.open(archive_file)&#xA;    tar.extractall(file)&#xA;    tar.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The function to calculate the md5 checksum is the following (borrowed from &lt;a href=&quot;https://stackoverflow.com/questions/3431825/generating-an-md5-checksum-of-a-file&quot;&gt;this&lt;/a&gt; question)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def get_md5_for_file(file):&#xA;    hash_md5 = hashlib.md5()&#xA;    with open(file, 'rb') as f:&#xA;        for chunk in iter(lambda: f.read(4096), b&amp;quot;&amp;quot;):&#xA;            hash_md5.update(chunk)&#xA;    &#xA;    return hash_md5.hexdigest()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that this is not the exact script I use, so I can't guarantee it works exactly like this, but I tried to include all the important stuff.&lt;/p&gt;&#xA;&lt;p&gt;In my version I include checks to whether my existing blast database needs to be updated (I keep the manifest files and compare the manifest['nr']['last_updated'] values) and I also try to skip already unpacked archives while iterating (check if the nr.xx.phd file exists) to save time in case the script fails due to problems in the download.&lt;/p&gt;&#xA;" OwnerUserId="4441" LastActivityDate="2021-04-30T07:40:38.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15834" PostTypeId="1" AcceptedAnswerId="15835" CreationDate="2021-04-30T08:07:14.950" Score="1" ViewCount="101" Body="&lt;p&gt;I have issues working with pybedtools due to strand information being lost after extracting and merging transcript coordinates from a gtf file.&lt;/p&gt;&#xA;&lt;p&gt;It seems that the solution to preserve strand information depends on the version of bedtools, so I would like to have my code detect bedtools version and adapt its behaviour accordingly.&lt;/p&gt;&#xA;&lt;p&gt;On a machine with bedtools v2.27.0:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;ids = ['WBGene00022277', 'WBGene00007068']&#xA;&#xA;def in_ids(itvl):&#xA;    return itvl[2] == &amp;quot;transcript&amp;quot; and itvl[&amp;quot;gene_id&amp;quot;] in ids&#xA;&#xA;all_genes = BedTool(&amp;quot;genes.gtf&amp;quot;)&#xA;my_genes = all_genes.filter(in_ids).saveas()&#xA;print(my_genes)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;I       ensembl transcript      4119    10230   .       -       .       gene_biotype &amp;quot;protein_coding&amp;quot;;gene_id &amp;quot;WBGene00022277&amp;quot;;gene_name &amp;quot;homt-1&amp;quot;;gene_source &amp;quot;ensembl&amp;quot;;gene_version &amp;quot;1&amp;quot;;p_id &amp;quot;P18791&amp;quot;;transcript_biotype &amp;quot;protein_coding&amp;quot;;transcript_id &amp;quot;Y74C9A.3&amp;quot;;transcript_name &amp;quot;Y74C9A.3&amp;quot;;transcript_source &amp;quot;ensembl&amp;quot;;transcript_version &amp;quot;1&amp;quot;;tss_id &amp;quot;TSS21161&amp;quot;;&#xA;X       ensembl transcript      17714968        17718714        .       +       .       gene_biotype &amp;quot;protein_coding&amp;quot;;gene_id &amp;quot;WBGene00007068&amp;quot;;gene_name &amp;quot;cTel55X.1&amp;quot;;gene_source &amp;quot;ensembl&amp;quot;;gene_version &amp;quot;1&amp;quot;;p_id &amp;quot;P8395&amp;quot;;transcript_biotype &amp;quot;protein_coding&amp;quot;;transcript_id &amp;quot;cTel55X.1b&amp;quot;;transcript_name &amp;quot;cTel55X.1b&amp;quot;;transcript_source &amp;quot;ensembl&amp;quot;;transcript_version &amp;quot;1&amp;quot;;tss_id &amp;quot;TSS18767&amp;quot;;&#xA;X       ensembl transcript      17714968        17718717        .       +       .       gene_biotype &amp;quot;protein_coding&amp;quot;;gene_id &amp;quot;WBGene00007068&amp;quot;;gene_name &amp;quot;cTel55X.1&amp;quot;;gene_source &amp;quot;ensembl&amp;quot;;gene_version &amp;quot;1&amp;quot;;p_id &amp;quot;P8874&amp;quot;;transcript_biotype &amp;quot;protein_coding&amp;quot;;transcript_id &amp;quot;cTel55X.1a&amp;quot;;transcript_name &amp;quot;cTel55X.1a&amp;quot;;transcript_source &amp;quot;ensembl&amp;quot;;transcript_version &amp;quot;1&amp;quot;;tss_id &amp;quot;TSS18767&amp;quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;merged = my_genes.sort().merge(s=True).saveas()&#xA;print(merged)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;I       4118    10230&#xA;X       17714967        17718717&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(&lt;strong&gt;Missing strand information&lt;/strong&gt;)&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;merged = my_genes.sort().merge(s=True, c=7, o=&amp;quot;distinct&amp;quot;).saveas()&#xA;print(merged)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;I       4118    10230   -&#xA;X       17714967        17718717        +&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;On a machine with bedtools v2.26.0:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;ids = ['WBGene00022277', 'WBGene00007068']&#xA;&#xA;def in_ids(itvl):&#xA;    return itvl[2] == &amp;quot;transcript&amp;quot; and itvl[&amp;quot;gene_id&amp;quot;] in ids&#xA;&#xA;all_genes = BedTool(&amp;quot;genes.gtf&amp;quot;)&#xA;my_genes = all_genes.filter(in_ids).saveas()&#xA;print(my_genes)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;I       ensembl transcript      4119    10230   .       -       .       gene_biotype &amp;quot;protein_coding&amp;quot;;gene_id &amp;quot;WBGene00022277&amp;quot;;gene_name &amp;quot;homt-1&amp;quot;;gene_source &amp;quot;ensembl&amp;quot;;gene_version &amp;quot;1&amp;quot;;p_id &amp;quot;P18791&amp;quot;;transcript_biotype &amp;quot;protein_coding&amp;quot;;transcript_id &amp;quot;Y74C9A.3&amp;quot;;transcript_name &amp;quot;Y74C9A.3&amp;quot;;transcript_&#xA;source &amp;quot;ensembl&amp;quot;;transcript_version &amp;quot;1&amp;quot;;tss_id &amp;quot;TSS21161&amp;quot;;&#xA;X       ensembl transcript      17714968        17718714        .       +       .       gene_biotype &amp;quot;protein_coding&amp;quot;;gene_id &amp;quot;WBGene00007068&amp;quot;;gene_name &amp;quot;cTel55X.1&amp;quot;;gene_source &amp;quot;ensembl&amp;quot;;gene_version &amp;quot;1&amp;quot;;p_id &amp;quot;P8395&amp;quot;;transcript_biotype &amp;quot;protein_coding&amp;quot;;transcript_id &amp;quot;cTel55X.1b&amp;quot;;transcript_name &amp;quot;c&#xA;Tel55X.1b&amp;quot;;transcript_source &amp;quot;ensembl&amp;quot;;transcript_version &amp;quot;1&amp;quot;;tss_id &amp;quot;TSS18767&amp;quot;;&#xA;X       ensembl transcript      17714968        17718717        .       +       .       gene_biotype &amp;quot;protein_coding&amp;quot;;gene_id &amp;quot;WBGene00007068&amp;quot;;gene_name &amp;quot;cTel55X.1&amp;quot;;gene_source &amp;quot;ensembl&amp;quot;;gene_version &amp;quot;1&amp;quot;;p_id &amp;quot;P8874&amp;quot;;transcript_biotype &amp;quot;protein_coding&amp;quot;;transcript_id &amp;quot;cTel55X.1a&amp;quot;;transcript_name &amp;quot;c&#xA;Tel55X.1a&amp;quot;;transcript_source &amp;quot;ensembl&amp;quot;;transcript_version &amp;quot;1&amp;quot;;tss_id &amp;quot;TSS18767&amp;quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;merged = my_genes.sort().merge(s=True).saveas()&#xA;print(merged)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;I       4118    10230   -&#xA;X       17714967        17718717        +&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;merged = my_genes.sort().merge(s=True, c=7, o=&amp;quot;distinct&amp;quot;).saveas()&#xA;print(merged)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;I       4118    10230   -       -&#xA;X       17714967        17718717        +       +&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(&lt;strong&gt;Duplicate strand information&lt;/strong&gt;)&lt;/p&gt;&#xA;&lt;p&gt;Given that I work on linux systems, &lt;strong&gt;how can I detect bedtools version from within python&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is there a better way&lt;/strong&gt; to do what I want?&lt;/p&gt;&#xA;&lt;p&gt;(Ideally, I would like to have my resulting bed consist in chrom, start, stop, name, score, strand fields, where the score would just be &amp;quot;.&amp;quot; and the name would be the content of &lt;code&gt;itvl[&amp;quot;gene_id&amp;quot;]&lt;/code&gt;)&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Edit (05/05/2021): Regarding my last comment above, I found out that it is easy to convert gff into bed using &lt;a href=&quot;https://daler.github.io/pybedtools/pybedtools.featurefuncs.gff2bed.html#pybedtools.featurefuncs.gff2bed&quot; rel=&quot;nofollow noreferrer&quot;&gt;pybedtools.featurefuncs.gff2bed&lt;/a&gt; (tested with bedtools 2.30):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;from pybedtools import BedTool&#xA;from pybedtools.featurefuncs import gff2bed&#xA;&#xA;ids = ['WBGene00022277', 'WBGene00007068']&#xA;&#xA;def in_ids(itvl):&#xA;    return itvl[2] == &amp;quot;transcript&amp;quot; and itvl[&amp;quot;gene_id&amp;quot;] in ids&#xA;&#xA;all_genes = BedTool(&amp;quot;genes.gtf&amp;quot;)&#xA;my_genes = all_genes.filter(in_ids).each(gff2bed, &amp;quot;gene_id&amp;quot;).saveas()&#xA;print(my_genes)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;I       4118    10230   WBGene00022277  .       -&#xA;X       17714967        17718714        WBGene00007068  .       +&#xA;X       17714967        17718717        WBGene00007068  .       +&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;merged = my_genes.sort().merge(s=True, c=[4, 5, 6], o=&amp;quot;distinct&amp;quot;).saveas()&#xA;print(merged)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;I       4118    10230   WBGene00022277  .       -&#xA;X       17714967        17718717        WBGene00007068  .       +&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This ensures proper bed format.&lt;/p&gt;&#xA;&lt;p&gt;However, with bedtools 2.26, &lt;code&gt;s=True&lt;/code&gt; makes strand information appear as a 4th column, so it is more complicated to obtain correct column order after merging.&lt;/p&gt;&#xA;" OwnerUserId="292" LastEditorUserId="292" LastEditDate="2021-05-05T08:19:59.720" LastActivityDate="2021-05-05T08:19:59.720" Title="How to detect bedtools version used by pybedtools (in order to correctly preserve strand information when merging gtf records)" Tags="|python|bedtools|pybedtools|strandedness|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15835" PostTypeId="2" ParentId="15834" CreationDate="2021-04-30T09:13:41.583" Score="3" Body="&lt;p&gt;pybedtools assumes that bedtools is in your path and bedtools itself will return the version with &lt;code&gt;bedtools --version&lt;/code&gt;. So:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import subprocess&#xA;subprocess.check_output(['bedtools', '--version'], text=True).strip().split()[1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For me that returns &lt;code&gt;'v2.30.0'&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-04-30T09:13:41.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15837" PostTypeId="1" CreationDate="2021-05-01T06:09:38.017" Score="2" ViewCount="150" Body="&lt;p&gt;Steps I have done&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I have two cell types to compare one is stem cell and leukemic stem cell&lt;/li&gt;&#xA;&lt;li&gt;Two sets of data for each condition RNA seq and ATAC seq.&lt;/li&gt;&#xA;&lt;li&gt;Did differential expression as well as differential accessibility analysis.&lt;/li&gt;&#xA;&lt;li&gt;From the differential accessibility analysis I'm getting the regions which I used for motif analysis using homer.&lt;/li&gt;&#xA;&lt;li&gt;The result of de novo motif enrichment&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AbXLJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AbXLJ.png&quot; alt=&quot;motif enrichment &quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;So this results shows possible transcription factor binding motif. Now how do i find the target genes of this putative transcription factor. Do i need to see the literature or what other ways to do it. As of now I don;t have chip seq data. So is it possible to use &lt;strong&gt;expression&lt;/strong&gt; and &lt;strong&gt;accessibility&lt;/strong&gt; data to find out the possible targets of this putative transcription factors, which in turn can give me lead for my downstream validation.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion or help would be really appreciated&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2021-05-01T06:09:38.017" Title="Finding targets genes of putative transcription factor after homer motif enrichment" Tags="|atac-seq|homer|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15839" PostTypeId="1" CreationDate="2021-05-02T23:41:04.720" Score="4" ViewCount="147" Body="&lt;p&gt;One standard deviation from the mean is commonly used to calculate a polygenic risk score for GWAS, e.g. human genetic disease.&lt;/p&gt;&#xA;&lt;p&gt;Why is this a common metric, for example why not 2-SD or 1.96 SD as in the normal distribution and what is the formula?&lt;/p&gt;&#xA;&lt;p&gt;Here's a figure taken from this paper (&lt;a href=&quot;https://genomemedicine.biomedcentral.com/track/pdf/10.1186/s13073-020-00742-5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;10.1186/s13073-020-00742-5&lt;/a&gt;) to help guide the explanation:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/D2U8d.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D2U8d.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1503" LastEditorUserId="3848" LastEditDate="2022-02-20T02:05:28.960" LastActivityDate="2023-06-14T21:21:20.570" Title="How is the odds ratio of disease risk conferred by a 1-standard deviation increase in PRS calculated?" Tags="|gwas|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15840" PostTypeId="1" CreationDate="2021-05-03T00:57:13.403" Score="3" ViewCount="61" Body="&lt;p&gt;Just learning the basics of bioinformatics and bash shell scripting.&lt;/p&gt;&#xA;&lt;p&gt;Is there a correct way to search for a specific allele with one or even multiple sequencing runs? I am searching for prevalence of the C2416T allele in the SARS-CoV-2 genome within the ORF1a RNA polymerase gene as a mini project.&lt;/p&gt;&#xA;&lt;p&gt;I downloaded a bunch of SARS-COV-2 sequencing runs, I think I need to convert them to bam file type than I am not sure how to call for one specific SNP, especially in a loop for 5-7 runs. Can this be done with bash?&lt;/p&gt;&#xA;&lt;p&gt;I was looking into variant calling from a bam file type using vcf and I can't find a way to call for one specific snp on multiple bam files at once.&lt;/p&gt;&#xA;&lt;p&gt;Ideally I would want to filter the variant out of the .bcf files or use grep since I sort of understand those the most and I’m not working with a large amount of data or a human genome.&lt;/p&gt;&#xA;" OwnerUserId="12116" LastEditorUserId="3848" LastEditDate="2022-02-24T03:43:53.130" LastActivityDate="2022-02-24T03:43:53.130" Title="Calling for one specific SNP from multiple sequencing runs" Tags="|variant-calling|bash|covid-19|sars-cov-2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15841" PostTypeId="2" ParentId="15840" CreationDate="2021-05-03T13:15:20.363" Score="4" Body="&lt;p&gt;Getting SNPs from whole-genome data is usually done via mapping sequencing reads on the reference (bwa-mem or bowtie are two popular mappers) and then using one of variant calling tools (like freebayes, samtools, or GATK). However, I would also look up if there is something virus-specific.&lt;/p&gt;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://www.biostars.org/p/93673/#93676&quot; rel=&quot;nofollow noreferrer&quot;&gt;this answer&lt;/a&gt;, you could get variant calls of a specific region using samtools if you give it a bed file that contains the region of interest (three columns &lt;code&gt;chr    start    stop&lt;/code&gt; format). For example using human reference&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools mpileup -uDl regions.bed -f hg19.fa file1.sorted.bam file2.sorted.bam | bcftools view -bvcg - &amp;gt; RAL_samtools.raw.bcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You could also specify a bunch of files, not just two and it would do the variant calling on them all.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2021-05-03T13:15:20.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15842" PostTypeId="1" AcceptedAnswerId="15844" CreationDate="2021-05-03T16:48:50.637" Score="3" ViewCount="249" Body="&lt;p&gt;I am working in c++ using the htslib library to parse different vcf files as well as fasta files.&lt;/p&gt;&#xA;&lt;p&gt;My goal is to read in a specific region from a fastafile and then extract all the VCF sites within that region from a seperate vcf file, in this case only containing chr14. I can extract the fasta region in several ways, one of which is to use the function faidx_fetch_seq() and define a region of chr14:20000000-22000000.&lt;/p&gt;&#xA;&lt;p&gt;Then for the bcf files i am able to iterate through the entire file and extracting the different information regarding the reference, alternative allele and so on. With a simple example shown below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;void faVCF(const char* fastafile,FILE *fp){&#xA;  htsFile *test_bcf = NULL;&#xA;  bcf_hdr_t *test_header = NULL;&#xA;&#xA;  test_bcf = bcf_open(&amp;quot;chr14.vcf&amp;quot;, &amp;quot;r&amp;quot;);&#xA;  test_header = bcf_hdr_read(test_bcf);&#xA;  bcf_hdr_set_samples(test_header, &amp;quot;20000000-22000000&amp;quot;, 0);&#xA;  bcf1_t *test_record = bcf_init();&#xA;  while(bcf_read(test_bcf, test_header, test_record) == 0) {&#xA;              std::cout &amp;lt;&amp;lt; bcf_hdr_id2name(test_header, test_record-&amp;gt;rid) &amp;lt;&amp;lt; &amp;quot;\t&amp;quot; &amp;lt;&amp;lt;&#xA;                          test_record-&amp;gt;pos &amp;lt;&amp;lt; &amp;quot;\t&amp;quot; &amp;lt;&amp;lt;&#xA;                          test_record-&amp;gt;n_allele &amp;lt;&amp;lt; &amp;quot;\t&amp;quot; &amp;lt;&amp;lt;&#xA;                          std::endl;&#xA;          }  &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, i doesn't work to extract all the variant sites in a specific region. I tried bcf_hdr_set_samples with the argument &amp;quot;20000000-22000000&amp;quot;, to extract that given region from my chromosome. As far as I understand this should be able to subsample both samples and region queries using the header information, and then upon initiation extract that region, however this doesnt work.&lt;/p&gt;&#xA;&lt;p&gt;So my question is: Can anyone help me with what function I can use to extract all the vcf sites from the region chr14:20000000-22000000.?&lt;/p&gt;&#xA;" OwnerUserId="8630" LastActivityDate="2021-05-04T09:22:25.260" Title="Use Htslib to extract specific region of vcf file" Tags="|c++|htslib|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15843" PostTypeId="1" CreationDate="2021-05-03T18:26:47.620" Score="1" ViewCount="17" Body="&lt;p&gt;I'm using R and bioconductor in order to conduct some gene analysis on an Affymetrix dataset (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE173360&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSE173360&lt;/a&gt;) but I'm having trouble at filtering genes. I'm trying to create a dataset called small.eset by filtering my dataset but small.eset returns NULL dimensions. I was wondering if I'm doing some obvious mistake?&lt;/p&gt;&#xA;&lt;p&gt;The dataset I'm using is GSE173360 and I'm sharing my condensed code (without plots and quality checks).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Library and Data Imports ------------------------------------------------&#xA;library(affy)&#xA;library(affyPLM)&#xA;library(simpleaffy)&#xA;library(limma)&#xA;library(RColorBrewer)&#xA;library(genefilter) &#xA;setwd(&amp;quot;/Users/Documents/R Scripts/biostat_geo/data&amp;quot;)&#xA;Data &amp;lt;- ReadAffy() #read all CEL files in working directory&#xA;&#xA;# log2 transformation -----------------------------------------------------&#xA;&#xA;ex &amp;lt;- exprs(Data)&#xA;qx &amp;lt;- as.numeric(quantile(ex, c(0., 0.25, 0.5, 0.75, 0.99, 1.0), na.rm=T))&#xA;LogC &amp;lt;- (qx[5] &amp;gt; 100) ||&#xA;  (qx[6]-qx[1] &amp;gt; 50 &amp;amp;&amp;amp; qx[2] &amp;gt; 0)&#xA;if (LogC) { ex[which(ex &amp;lt;= 0)] &amp;lt;- NaN&#xA;exprs(Data) &amp;lt;- log2(ex) }&#xA;&#xA;#background correction&#xA;&#xA;eset &amp;lt;- expresso(Data, bgcorrect.method = &amp;quot;mas&amp;quot;,&#xA;             normalize.method = &amp;quot;loess&amp;quot;, &#xA;             pmcorrect.method = &amp;quot;pmonly&amp;quot;,&#xA;             summary.method = &amp;quot;mas&amp;quot;)&#xA;&#xA;# filtering&#xA;&#xA;esetunlog &amp;lt;- 2^exprs(eset)&#xA;ffun &amp;lt;- filterfun(pOverA(0.20,10))&#xA;t.fil &amp;lt;- genefilter(esetunlog,ffun)&#xA;small.eset &amp;lt;- (log2(esetunlog[t.fil,]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12125" LastEditorUserId="57" LastEditDate="2021-05-05T09:24:56.223" LastActivityDate="2021-05-05T09:24:56.223" Title="Bioconductor, genefilter() returns NULL dimensions, is my filtering process wrong?" Tags="|r|bioconductor|microarray|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15844" PostTypeId="2" ParentId="15842" CreationDate="2021-05-03T19:30:48.487" Score="4" Body="&lt;p&gt;For iterating over reads in a region in a single BCF file, you want to use the iterator interface: &lt;code&gt;bcf_itr_queryi()&lt;/code&gt; or &lt;code&gt;bcf_itr_querys()&lt;/code&gt;, together with &lt;code&gt;bcf_itr_next()&lt;/code&gt;. For a VCF file, it appears you need to use the similar TBX iterator interface: &lt;code&gt;tbx_itr_queryi()&lt;/code&gt; or &lt;code&gt;tbx_itr_querys()&lt;/code&gt;, together with &lt;code&gt;tbx_itr_next()&lt;/code&gt;. See HTSlib's &lt;em&gt;tabix.c&lt;/em&gt; for reasonably straightforward examples of both.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively you could use the routines in &lt;em&gt;htslib/synced_bcf_reader.h&lt;/em&gt;, which encapsulates both and is what bcftools mostly uses.&lt;/p&gt;&#xA;" OwnerUserId="134" LastEditorUserId="134" LastEditDate="2021-05-04T09:22:25.260" LastActivityDate="2021-05-04T09:22:25.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15846" PostTypeId="1" AcceptedAnswerId="15848" CreationDate="2021-05-03T23:29:18.067" Score="3" ViewCount="46" Body="&lt;p&gt;I am looking for a software that can identify long sequences (1500-2000 bp) that occur more than once in my genome of interest. A k-mer counter like KMC could have worked, but KMC has a max limit of 256 bp. If I can club contiguous 256-mers from KMC, that would work too, but I don't know how to do that. Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="12127" LastActivityDate="2021-05-04T03:38:13.037" Title="Finding 2000 bp sequences that occur more than once in the genome" Tags="|sequence-alignment|k-mer|repeat-elements|sequence-homology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15848" PostTypeId="2" ParentId="15846" CreationDate="2021-05-04T03:38:13.037" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://mummer4.github.io/manual/manual.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MUMmer&lt;/a&gt; includes a &lt;code&gt;repeat-match&lt;/code&gt; tool which can do exactly that:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ repeat-match&#xA;USAGE:  repeat-match  [options]  &amp;lt;genome-file&amp;gt;&#xA;&#xA;Find all maximal exact matches in &amp;lt;genome-file&amp;gt;&#xA;&#xA;Options:&#xA; -E    Use exhaustive (slow) search to find matches&#xA; -f    Forward strand only, don't use reverse complement&#xA; -n #  Set minimum exact match length to #&#xA; -t    Only output tandem repeats&#xA; -V #  Set level of verbose (debugging) printing to #&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that a &amp;quot;maximal exact match&amp;quot; is just a maximal repeat.&lt;/p&gt;&#xA;" OwnerUserId="6628" LastActivityDate="2021-05-04T03:38:13.037" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15850" PostTypeId="1" CreationDate="2021-05-04T10:29:03.023" Score="1" ViewCount="175" Body="&lt;p&gt;I want to submit some phage sequences to NCBI. So I want to transform a genbank file into a tbl file. Can someone help me please?&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="2686" LastActivityDate="2021-05-04T10:29:03.023" Title="Convert genbank file into tbl file" Tags="|python|phylogenetics|genbank|" AnswerCount="0" CommentCount="2" ClosedDate="2021-05-05T12:49:29.657" ContentLicense="CC BY-SA 4.0" />
  <row Id="15851" PostTypeId="1" AcceptedAnswerId="15856" CreationDate="2021-05-04T12:20:32.500" Score="2" ViewCount="1016" Body="&lt;p&gt;&lt;em&gt;Note: this question has also been asked on &lt;a href=&quot;https://support.bioconductor.org/p/9136832/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioconductor Support&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I know there are a lot of questions asking similar things here on this forum and I checked the vingette and did a lot of other research but I still have a hard time wrapping my head around it.&lt;/p&gt;&#xA;&lt;p&gt;In my position, data is unfortunately just put in front of me and I'm asked for pretty pictures. Even making it clear to the wetlab people in the lab that we need replicates is an issue.&lt;/p&gt;&#xA;&lt;p&gt;I have multiple cell lines, multiple time points and multiple treatments:&lt;/p&gt;&#xA;&lt;p&gt;Cell lines: CL1, CL2, CL3 Time points: 6h, 24h Treatments: T1, T2, T3, Control&lt;/p&gt;&#xA;&lt;p&gt;For each cell line and each time point, there are 3 different treatments plus a control. 3 replicates for each sample -&amp;gt; 72 samples&lt;/p&gt;&#xA;&lt;p&gt;What I want is actually quite simple; I want to measure control vs each treatment at each time point in each cell line. No testing across time points or across cell lines.&lt;/p&gt;&#xA;&lt;p&gt;My first thought was to separate the data into 6 different data sets (CL1, 6h | CL1, 24h | CL2, 6h | CL2 24h | CL3, 6h | CL3, 24h) and simply do the DE analysis separately, but I read that this is not the way to go. In the end it would also be nice to get a normalized count matrix with all data normalized together, for PCAs and similar.&lt;/p&gt;&#xA;&lt;p&gt;I hope I made it understandable. How do I design the DESeqDataSet?&lt;/p&gt;&#xA;" OwnerUserId="12129" LastEditorUserId="73" LastEditDate="2021-05-06T09:27:03.030" LastActivityDate="2021-05-06T09:27:03.030" Title="DESeq2 multiple treatments, multiple time points, multiple cell lines" Tags="|deseq2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15852" PostTypeId="1" CreationDate="2021-05-04T14:30:33.983" Score="1" ViewCount="403" Body="&lt;p&gt;I have a question about the normalization for ChIP-seq. I used CPM to normalize my bam files of each IP and Input. Then I calculate the coverage of gene bodies for all genes on the genome. I have WT and different mutants. Take H3K4me1 as an example, I first calculate the matrix over gene bodies for H3K4me1 IP and Input. Then I used the matrix of H3K4me1 divided by the matrix of input. then log2(H3K4me1/Input) and plot this value over entire gene bodies. I found all the regions are lower than 0 and it seems that my IP does not work. In fact, this is not the case since I have checked the antibody's efficiency via ChIP-qPCR and it works well. Moreover, the distribution of H3K4me1 is consistent with the published data. But the problem is the profile of log2(H3K4me1/Input) is lower than 0.  I don't know how to deal with such problems? I would appreciate it if someone can give me some suggestions on that.  Thanks!&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Xxlab.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Xxlab.png&quot; alt=&quot;prolfile of H3K4me1 over gene bodies&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12132" LastActivityDate="2021-05-05T09:29:14.663" Title="Problems about value log2(IP/Input) less than zero in ChIP-seq?" Tags="|normalization|chip-seq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15853" PostTypeId="1" CreationDate="2021-05-04T17:47:27.203" Score="1" ViewCount="62" Body="&lt;p&gt;I have 10 transcripts where I need to do 5 peptide sequence alignments (2 transcripts per alignment). I need to do this in order to indicate which amino acids lead to which domains.&lt;/p&gt;&#xA;&lt;p&gt;My questions are two:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a tool I can use from my laptop that aligns peptide sequences? Because results from the clustal omega peptide alignments web service only last two weeks, they're not ideal for replicability in a paper.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The ultimate aim is to use those alignments and domain classifications to a project I am writing and I am trying to make it as presentable and scientific as possible. Any suggestions on how to make peptide alignments and annotations on peptide alignments (which indicate protein domains) look presentable for a paper or if a paper has already shown peptide alignments in a presentable way that would be highly appreciated.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="12135" OwnerDisplayName="mk894" LastEditorUserId="73" LastEditDate="2021-05-05T06:29:15.690" LastActivityDate="2021-05-05T06:29:15.690" Title="Alignment tools on desktop for peptide alignment sequence and how to make annotations on alignments look presentable?" Tags="|sequence-alignment|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15854" PostTypeId="2" ParentId="15851" CreationDate="2021-05-04T17:58:50.317" Score="1" Body="&lt;p&gt;To compare a subset of samples to another subset of samples, make one column of column data which will distinguish them all the way you want (so probably join cell line, treatment, timepoint), and make that column your design, and then when you call results, you specify the pair you want to contrast, one pair at a time.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2021-05-04T17:58:50.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15855" PostTypeId="2" ParentId="15852" CreationDate="2021-05-05T06:10:19.433" Score="0" Body="&lt;p&gt;This is a common issue caused by the normalization method not being particularly robust (most likely it's using the mean signal for normalization, which will be a bit upward skewed when you have regions of high enrichment in the ChIP). The basic ways to fix this are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use a more robust normalization (e.g., the median signal, or even just manually select un-enriched regions to calculate the normalization factors)&lt;/li&gt;&#xA;&lt;li&gt;Manually tweak the normalization factors so the background (i.e., log2 ratio in regions where you don't expect any enrichment) is 1. That will generally make the regions you're plotting more biologically realistic.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Note that there can be a negative enrichment in some areas, due to things like exclusion of histones, so don't be overly zealous if you manually tweak the normalization factors.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-05-05T06:10:19.433" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="15856" PostTypeId="2" ParentId="15851" CreationDate="2021-05-05T06:26:25.413" Score="1" Body="&lt;p&gt;As &lt;a href=&quot;https://support.bioconductor.org/p/9136832/#9136833&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mike Love&lt;/a&gt; alluded to on the Bioconductor pages, it's best to have specialised help for datasets that are this complex; doing analysis on anything which has multiple compounding factors is not simple. It's also a good idea to talk through the analysis &lt;em&gt;before&lt;/em&gt; doing sequencing, so you can discuss things like having six replicates per sample (including controls).&lt;/p&gt;&#xA;&lt;p&gt;In any case, what I did for something similar to this (eventually, after lots of discussion with biologists about possible models) was to combine all the non-tested factor columns into a single factor. Something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;meta.combined.df &amp;lt;- meta.df %&amp;gt;%&#xA;  mutate(lineTime = paste0(cellLine, &amp;quot;_&amp;quot;, timePoint))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And then subset the data based on this statistic:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;counts.sub.mat &amp;lt;- &#xA;   counts.sub.mat[,meta.df$lineTime == desired.lineTime]&#xA;meta.sub.df &amp;lt;-&#xA;   subset(meta.combined.df, lineTime == desired.lineTime)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then use &lt;code&gt;~ Treatment&lt;/code&gt; as the design formula (assuming &amp;quot;Treatment&amp;quot; is a field containing either the treatment type or &amp;quot;control&amp;quot;).&lt;/p&gt;&#xA;&lt;p&gt;But... this is explicitly treating each line/time point as a separate experiment, which may not be what you want. There are complications associated with batch effects and outliers that need to be talked through, because you could end up identifying &amp;quot;differential expression&amp;quot; that is not reproducible.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-05T06:26:25.413" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15858" PostTypeId="2" ParentId="15852" CreationDate="2021-05-05T09:29:14.663" Score="1" Body="&lt;p&gt;Seconding what Devon says:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use a proper normalization technique such as TMM from &lt;code&gt;edgeR&lt;/code&gt;. For applying this to bigwig files you could follow &lt;a href=&quot;https://www.biostars.org/p/413626/#414440&quot; rel=&quot;nofollow noreferrer&quot;&gt;my tutorial at biostars.org&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My two cents beyond that:&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;I would not use enrichment over input as input is hard to normalize and can be sparse depending on sequencing depth. Therefore, you might get biased enrichments that do not well reflect reality, especially if counts in IP or input (of both) are low. I personally only use input for peak calling, and for plots like this then compare IPs among each other.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;So the strategy would be to first normalize your bigwigs (I guess you use these for the plots?) using the strategy as linked above, and then use these robust CPMs to make the plots without any input samples, only using the IPs. You can use MA-plots to explore whether samples are properly normalized. I made a little code suggestion &lt;a href=&quot;https://www.biostars.org/p/9465851/#9465854&quot; rel=&quot;nofollow noreferrer&quot;&gt;here in this biostars post&lt;/a&gt; on how to do this starting basically from a matrix of raw counts using &lt;code&gt;edgeR&lt;/code&gt;. This is basically (from the TMM normalization standpoint) the same as in the first link, but then doing pairwise MA-plots to see whether TMM properly centers the majority of points along y-axis being somewhat zero. Normalization is key in any experiment, therefore I always recommend to use these kinds of plots for diagnostics.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-05-05T09:29:14.663" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15859" PostTypeId="2" ParentId="15840" CreationDate="2021-05-05T13:06:20.420" Score="2" Body="&lt;p&gt;&lt;em&gt;Answer from @atpoint [transferred from comment]:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, you can use &lt;code&gt;samtools view&lt;/code&gt; to only retain reads that overlap the region you are interested in and then feed these into the mpileup. IIRC there was something with that &lt;code&gt;-l&lt;/code&gt; option in mpileup that caused trouble but I do not remember what it was. I think it still called all variants based on all reads and then simply filtered for the target regions in &lt;code&gt;-l&lt;/code&gt;. The &lt;code&gt;view&lt;/code&gt; approach would probably be much faster. Just keep that in mind as an alternative.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-05T13:06:20.420" CommentCount="0" CommunityOwnedDate="2021-05-05T13:06:20.420" ContentLicense="CC BY-SA 4.0" />
  <row Id="15860" PostTypeId="1" AcceptedAnswerId="15864" CreationDate="2021-05-05T15:07:51.123" Score="2" ViewCount="171" Body="&lt;p&gt;I have been working with Shannon's entropy (SE) and DNA lately, and referring to the formula and concept of SE, I just wondered whether one should in the case of DNA use each nucleotide as having a probability to show up as 0,25 or if it was better to use the probability based on the frequency in the genome (let's say the human genome).&lt;/p&gt;&#xA;&lt;p&gt;In the case of the latter being the better, is it interesting to have those frequencies at the scale of the genome, the chromosome, the sub segment ?&lt;/p&gt;&#xA;&lt;p&gt;I'm interested in calculating the entropy relative to a short window (the length of an Illumina read, about 125-150bases), using the proportion of each base on a given sequence length. The idea is &amp;quot;how much information is given&amp;quot; in this sequence, and by information I mean how much different bases do I have.&lt;/p&gt;&#xA;&lt;p&gt;It might that I'm missing some basic mathematics knowledge here, don't hesitate to point it out so I can understand it better&lt;/p&gt;&#xA;&lt;p&gt;cheers,&lt;/p&gt;&#xA;" OwnerUserId="12141" LastEditorUserId="73" LastEditDate="2021-05-09T21:16:07.473" LastActivityDate="2021-05-09T21:16:07.473" Title="Shannon's Entropy and DNA" Tags="|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15861" PostTypeId="1" AcceptedAnswerId="15862" CreationDate="2021-05-05T15:33:21.483" Score="0" ViewCount="57" Body="&lt;p&gt;&lt;strong&gt;Short question:&lt;/strong&gt;&lt;br /&gt;&#xA;where can I find what &lt;code&gt;F&lt;/code&gt; and &lt;code&gt;J&lt;/code&gt; and &lt;code&gt;&amp;lt;&lt;/code&gt; and &lt;code&gt;-&lt;/code&gt; and &lt;code&gt;7&lt;/code&gt; mean in a Sequence file?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Longer version:&lt;/strong&gt;&lt;br /&gt;&#xA;I was looking at this sequence file I got from NCBI (it's an sra file? if that is the correct terminology) and I used the &lt;code&gt;sratoolkit&lt;/code&gt; to print out a few &lt;code&gt;spot&lt;/code&gt;s&lt;/p&gt;&#xA;&lt;p&gt;looking at them I expected to see ACGU (this was meant to be an RNA sequence), but what I see are like&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;AAFFFJJJFJJJJJJJJJJJJJJJJJJJJJFJJF&amp;lt;JJ77AJJJJ-JJJJJFF7FA--F7FAJJ7&amp;lt;AJ-FJA&amp;lt;J&amp;lt;JJFA&amp;lt;&amp;lt;77F-JJJJJJF&amp;lt;JF&amp;lt;7-FF-A-----A-77JJ7FJJF-FFFAJ&amp;lt;&amp;lt;-FJJFAJAFFAFFFFF&amp;lt;7A7JFJJ7AAFFFJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJFJJJFJJJJJJJJJJJJAA&amp;lt;JJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJJAAFFJJFJJJJJJJJJJJJJJFJ-FJF&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Where can I find a reference to help me &amp;quot;understand&amp;quot; this?&lt;br /&gt;&#xA;I mean I assume &lt;code&gt;-&lt;/code&gt; means missed or unknown? or is it meant to be already aligned and these are deletions?&lt;/p&gt;&#xA;&lt;p&gt;I appreciate any pointer to get started. as you must have observed I am a total noob.&lt;/p&gt;&#xA;&lt;p&gt;I got the file from: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra/SRX8156420%5Baccn%5D&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/sra/SRX8156420[accn]&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then I did a&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; ./fastq-dump -X 5 -Z ERR4557575&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Follow-up question:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Where do you look for the quality scores ranges and such, I guess as @winnie2k suggested I should convert this to FASTQ and make my life easier, but still wanted to understand the file better.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="12142" OwnerDisplayName="Ali" LastEditorUserId="73" LastEditDate="2021-05-06T00:37:56.113" LastActivityDate="2021-05-06T00:37:56.113" Title="Base abbreviations other than ACTGU in sequence file" Tags="|rna-seq|" AnswerCount="1" CommentCount="1" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15862" PostTypeId="2" ParentId="15861" CreationDate="2021-05-05T16:10:43.943" Score="3" Body="&lt;p&gt;I usually convert SRA files to FASTQ before doing any further processing, but the tooling for converting SRA files is cumbersome.&lt;/p&gt;&#xA;&lt;p&gt;To simplify things, I recommend you download the FASTQ file directly. The ENA currently has a nicer interface for doing these things. Here is a link to download the FASTQ files for the SRA sequencing run associated with the SRA experiment you are describing: &lt;a href=&quot;https://www.ebi.ac.uk/ena/browser/view/SRR11588968&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/ena/browser/view/SRR11588968&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;A description of the FASTQ format can be found here: &lt;a href=&quot;https://en.wikipedia.org/wiki/FASTQ_format&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/FASTQ_format&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2021-05-05T16:10:43.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15863" PostTypeId="1" CreationDate="2021-05-05T23:25:04.217" Score="1" ViewCount="163" Body="&lt;p&gt;I am having trouble creating a 10 basepair sliding window that goes through &lt;code&gt;seq&lt;/code&gt; and trim them all 10 basepairs that have an average quality score (&lt;code&gt;qual&lt;/code&gt;) less than 15. I am trying to approach this by reading in the file and seperating the sequences and quality scores, then read in the 10 basepair window in the sequence.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/envpython&#xA;with open('fastaQ.fq') as f:&#xA;    lines = [line.rstrip() for line in f]&#xA;    seq= []&#xA;    qual= []&#xA;    tenFrame= []&#xA;&#xA;    #Seperates the sequences and the quality score of the file&#xA;    for i in range(len(lines)):&#xA;        if lines[i].startswith('@'):&#xA;            seq.append(lines[i+1])&#xA;            qual.append(lines[i+3])&#xA;            i=i+4&#xA;&#xA;    #&#xA;    for i in range(0,len(seq)):&#xA;        tenFrame.append(seq[i][i:i+10])&#xA;        if len(tenFrame)%10 == 0:&#xA;            print(tenFrame)&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12146" LastEditorUserId="3541" LastEditDate="2021-05-07T19:16:55.757" LastActivityDate="2021-05-07T19:16:55.757" Title="10bp sliding window quality score" Tags="|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15864" PostTypeId="2" ParentId="15860" CreationDate="2021-05-06T00:35:26.953" Score="1" Body="&lt;p&gt;I think that genome-wide probabilities (i.e. not 0.25 for each base) should be fine. For a given segment of DNA, the chance that the sequence doesn't appear anywhere else in the genome is fairly high.&lt;/p&gt;&#xA;&lt;p&gt;There is lots of patchy non-randomness (e.g. at centromeres, STRs, and other tandem repeat regions), but there's not much point in modelling that because it's not random.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-05-06T19:54:50.237" LastActivityDate="2021-05-06T19:54:50.237" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15865" PostTypeId="2" ParentId="15863" CreationDate="2021-05-06T00:46:15.280" Score="1" Body="&lt;p&gt;I'm not sure what you mean by &amp;quot;trim&amp;quot;. The code below will discard any reads that have a 10bp window with a mean phred score below your cutoff. I'll ammend the post to also include a solution that &amp;quot;trims&amp;quot; basepairs on both ends until the window achieves an average score of above the cutoff. It would be ideal if you could provide a minimal reproducible example that shows what your code does and what you'd like it to do.&lt;/p&gt;&#xA;&lt;p&gt;As far as the code goes, I'd first suggest using a library like Biopython to parse the Fastq file. It will make your life a lot easier. For your code, it looks like &lt;code&gt;seq&lt;/code&gt; actually contains all the sequences, so &lt;code&gt;len(seq)&lt;/code&gt; is the number of reads in the file. However, when you iterate over &lt;code&gt;i in range(len(seq))&lt;/code&gt;, it looks like you're using &lt;code&gt;i&lt;/code&gt; to both index which read to iterate over as well as get the 10 base pair window. You need an additional index variable to obtain the window:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for window_start in range(len(seq[i]) - window_len) + 1)&#xA;    window = seq[i][window_start:window_start + window_len]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Additionally, some pythonic tips for the future:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If a range starts from 0, don't specify it as an arugment i.e. use &lt;code&gt;range(x)&lt;/code&gt; as opposed to &lt;code&gt;range(0, x)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Try to avoid anything along the lines of &lt;code&gt;range(len(x))&lt;/code&gt;. In this case, you could just iterate over the sequences directly: &lt;code&gt;for s in seq:&lt;/code&gt; or, if you wanted the index, you could enumerate the sequences via &lt;code&gt;for seq_idx, s in enumerate(seq)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Use good variable names. &lt;code&gt;seq&lt;/code&gt; implies a single sequence, yet is a list of sequences. Same with &lt;code&gt;qual&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Here is some code that will retain only sequences that have all windows with a passing mean phred score&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;read_file = &amp;quot;/file/here.fq&amp;quot;&#xA;w = 10&#xA;cutoff = 15&#xA;valid_seqs = []&#xA;with open(fq_file) as fq_fd:&#xA;    seqs = SeqIO.parse(fq_fd, &amp;quot;fastq&amp;quot;)&#xA;    for seq in seqs:&#xA;        scores = seq.letter_annotations[&amp;quot;phred_quality&amp;quot;]&#xA;        mean_scores = (np.mean(scores[i:i+w]) for i in range(len(scores) - w))&#xA;        if any(mean_score &amp;lt; cutoff for mean_score in mean_scores):&#xA;            print(f&amp;quot;{seq.id} does not pass the filter!&amp;quot;)&#xA;            continue&#xA;        valid_seqs.append(seq)&#xA;    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6628" LastActivityDate="2021-05-06T00:46:15.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15866" PostTypeId="1" AcceptedAnswerId="15867" CreationDate="2021-05-06T01:42:09.263" Score="0" ViewCount="1044" Body="&lt;p&gt;I am trying to run cellranger but I get fastq permission denied error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fi1d18@RBGO-Server2:/opt&lt;span class=&quot;math-container&quot;&gt;$ cellranger-atac count --id test --reference refdata-cellranger-arc-GRCh38-2020-A-2.0.0 --fastqs /data/fi1d18/Downloads/FASTQ_scATAC/ --chemistry ARC-v1&#xA;error: couldn't open MRO file&#xA;    Caused by: Permission denied (os error 13)&#xA;fi1d18@RBGO-Server2:/opt$&lt;/span&gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have added the path by&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;export PATH=/opt/cellranger-atac-2.0.0:$PATH&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And by vim I have also added the path&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fi1d18@RBGO-Server2:/opt$ vim ~/.bashrc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any help please?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2021-05-06T09:38:14.880" Title="Cellranger gives error" Tags="|single-cell|bash|linux|10x-genomics|cellranger|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15867" PostTypeId="2" ParentId="15866" CreationDate="2021-05-06T09:38:14.880" Score="1" Body="&lt;p&gt;Two of those command line parameters should be accessible locations (see &lt;a href=&quot;https://support.10xgenomics.com/single-cell-atac/software/pipelines/latest/using/count#header&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). Could you please add the output of the following commands:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;ls -l /data/fi1d18/Downloads/FASTQ_scATAC/ | head&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;ls -l refdata-cellranger-arc-GRCh38-2020-A-2.0.0 | head&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I suspect that the refdata link should be an absolute path, rather than a relative path, i.e. something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cellranger-atac count --id test --reference /opt/refdata-cellranger-arc-GRCh38-2020-A-2.0.0 --fastqs /data/fi1d18/Downloads/FASTQ_scATAC/ --chemistry ARC-v1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-06T09:38:14.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15868" PostTypeId="2" ParentId="2694" CreationDate="2021-05-06T09:52:18.730" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @terdon converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;$&lt;/code&gt; are often used to signify that something is a variable. So I guess they mean you need to use a name that you've already used. I mean, the same name as used for something else in your pipeline, so that then the program will see the file with the same name and know to apply it. Sorry, I don't have any experience with chimera, so I have no idea if what I'm suggesting is reasonable. It's just what I get from the &lt;code&gt;$&lt;/code&gt; you mentioned.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Answer from @marcin converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;this question was answered on chimera-users:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://plato.cgl.ucsf.edu/pipermail/chimera-users/2017-October/014045.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://plato.cgl.ucsf.edu/pipermail/chimera-users/2017-October/014045.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Answer from @joe-healey converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;You are probably looking for the “Morph Conformations” function within Chimera. It’ll move and superimpose 2 structures on to one another.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-06T09:52:18.730" CommentCount="0" CommunityOwnedDate="2021-05-06T09:52:18.730" ContentLicense="CC BY-SA 4.0" />
  <row Id="15870" PostTypeId="2" ParentId="7339" CreationDate="2021-05-06T10:09:19.373" Score="1" Body="&lt;p&gt;Short template DNA is a big problem with nanopore sequencing; it doesn't easily show up on a gel if it's well distributed across different lengths (as appears to be the case here from the read length plot), and sucks up both adapter and sequencing capacity.&lt;/p&gt;&#xA;&lt;p&gt;The usual way to deal with it (for reads in the size ranges seen here) is to reduce the bead concentration when doing magnetic bead cleanup. Try something like 0.6X beads (e.g. 30μl of Ampure XP beads added to 50μl template DNA solution), which should still allow most of the long template sequences to bind, but leave shorter reads in solution.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-06T10:09:19.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15871" PostTypeId="2" ParentId="10885" CreationDate="2021-05-06T10:15:35.020" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @matteo-ferla converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I assume you mean sequence (MSA), not structure alignment. Despite being a structural bioinformatician, I would slap a big caveat against structure based aligners. TCoffee:Espresso is probably the best structure-based aligner (browser and download and API), but the results are not as solid as some non-structural approaches such as training Psi-blast and aligning with that matrix. I am not familiar with MSA of TM regions, however, so this group may be an exception.&lt;/p&gt;&#xA;&lt;p&gt;Given that helices are a repeat of 2 hydrophobic residues and 1.5 either polar (core-wards) or also hydrophobic (depending on how the helices pack), a pattern based aligner ought to do well from an evolutionary perspective. If you really want from a structure perspective and don't want to know about the cases where a loss of a turn at the top is compensated by the bottom, say, then TCoffee is definitely worth a try&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Answer from @m converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;PSI-blast is a useful tool for deep alignment patterns, albeit it has its critics. I agree the vast majority of in vivo structures lack TM proteins, for different reasons&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-06T10:15:35.020" CommentCount="0" CommunityOwnedDate="2021-05-06T10:15:35.020" ContentLicense="CC BY-SA 4.0" />
  <row Id="15872" PostTypeId="2" ParentId="10716" CreationDate="2021-05-06T10:23:01.930" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @stupidwolf converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would try something like this, for every gene you have a logFC of RNAPII in control versus treatment. Do this for other 2 proteins, and you correlate the logFC. In very rough terms, correlation is the scaled form of covariance, so this will work.&lt;/p&gt;&#xA;&lt;p&gt;If you have 1000 genes, do correlation between 1000 logFC from RNAPII against 1000 logFC from other proteins.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Answer from @m converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;You use a covariation stat and then perform Pearson's correlation to standardise the response between -1 to 1. An answer of &amp;gt;0.7 is considered robust and &amp;gt;0.8 highly robust. ITs an interesting question and you are right covariation is the way to answerr it&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-06T10:23:01.930" CommentCount="0" CommunityOwnedDate="2021-05-06T10:23:01.930" ContentLicense="CC BY-SA 4.0" />
  <row Id="15873" PostTypeId="2" ParentId="10547" CreationDate="2021-05-06T10:25:50.830" Score="1" Body="&lt;p&gt;&lt;em&gt;solution from @sören:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I came up with this solution for my problem:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.KEGG import REST as kegg&#xA;&#xA;def _get_kegg(kegg_id):&#xA;    kegg_output = kegg.kegg_get(kegg_id).read()&#xA;    results = {}&#xA;    for line in kegg_output.split('\n'):&#xA;        splits = line.split()&#xA;        if not line.startswith(' '):    &#xA;            if len(splits) &amp;gt; 0:&#xA;                key = splits[0]&#xA;                value = ' '.join(splits[1:])&#xA;                results[key] = value&#xA;        else:&#xA;            results[key] += ' '.join(splits)&#xA;    return pd.DataFrame(results, index=[kegg_id])&#xA;&#xA;&#xA;_get_kegg_v = np.vectorize(_get_kegg)&#xA;&#xA;def get_kegg_info(kegg_ids):&#xA;    if isinstance(kegg_ids, str):&#xA;        kegg_ids = [kegg_ids]&#xA;    return pd.concat(_get_kegg_v(kegg_ids), sort=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-06T10:25:50.830" CommentCount="0" CommunityOwnedDate="2021-05-06T10:25:50.830" ContentLicense="CC BY-SA 4.0" />
  <row Id="15874" PostTypeId="1" CreationDate="2021-05-06T10:52:15.627" Score="1" ViewCount="15" Body="&lt;p&gt;I am using PhyloNet to compute the MAST of two unrooted binary phylogenetic trees but I am not getting the output that I should. I think the issue might be with the way I am writing the trees in the input, but  I don't know.&lt;/p&gt;&#xA;&lt;p&gt;For example: if I input &lt;code&gt;T1=((((1,5),6), (2,4)),3)&lt;/code&gt; and &lt;code&gt;T2=((1,2), (((3,5),6),4))&lt;/code&gt;, the MAST returned is &lt;code&gt;((1,2),3)&lt;/code&gt;, whereas if I draw the trees, I get the MAST &lt;code&gt;((3,6),(4,2))&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I think this might happen because somehow it assumes a root, but I have no idea how to fix it or if that is really the issue. Any guidelines for computing MASTs of unrooted trees are appreciated.&lt;/p&gt;&#xA;" OwnerUserId="12150" LastEditorUserId="650" LastEditDate="2021-05-06T18:16:16.930" LastActivityDate="2021-05-06T18:16:16.930" Title="Computing Maximum Agreement Subtree of two unrooted phylogenetic trees with PhyloNet?" Tags="|phylogenetics|phylogeny|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15875" PostTypeId="1" AcceptedAnswerId="15908" CreationDate="2021-05-06T13:38:34.237" Score="1" ViewCount="187" Body="&lt;p&gt;TL;DR Can anyone point me in the direction of a tool that can be used to input different viral substrain reference sequences (fasta) as well as sequence data (fastq/bam/vcf) and find the most likely substrain of the sequenced data?&lt;/p&gt;&#xA;&lt;p&gt;I am doing multiple sequence alignment of different viral subtype references and would like to get the exact positions and base each time there is a mismatch. I have tried using different tools. For example Clustal Omega can almost give me what I seek, but I have trouble getting the exact position and nucleotide for the mismatches (the full thing is ~8000bp long):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ACTACAATAATCCATGTATAAAACTAAGGGCGTAACCGAAATCGGTTGAACCGAAACCGG&#xA;ACTACAATAATTCATGTATAAAACTAAGGGTGTAACCGAAATCGGTTGAACCGAAACCGG&#xA;ACTACAATAATTCATGTATAAAACTAAGGGTGTAACCGAAATCGGTTGAACCGAAACCGG&#xA;ACTACAATAATTCATGTATAAAACTAAGGGCGTAACCGAAATCGGTTGAACCGAAACCGG&#xA;ACTACAATAATTCATGTATAAAACTAAGGGCGTAACCGAAATCGGTTGAACCGAAACCGG&#xA;ACTACAATAATTCATGTATAAAACTAAGGGTGTAACCGAAATCGGTTGAACCGAAACCGG&#xA;ACTACAATAATTCATGTATAAAATTAAGGGCGTAACCGAAATCGGTTGAACCGAAACCGG&#xA;ACTACAATAATTCATGTATAAAACTAAGGGCGTAACCGAAATCGGTTGAACCGAAACCGG&#xA;ACTACAATAATTCATGTATAAAACTAAGGGCGTAACCGAAATCGGTTGAACCGAAACCGG&#xA;ACTACAATAATTCATGTATAAAACTAAGGGCGTAACCGAAATCGGTTGAACCGAAACCGG&#xA;&#xA;*********** *********** ****** ***************************** &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;E.g I would like something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Pos 12:&#xA;C&#xA;T&#xA;T&#xA;T&#xA;T&#xA;T&#xA;T&#xA;T&#xA;T&#xA;T&#xA;T&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For each position with a mismatch.&lt;/p&gt;&#xA;&lt;p&gt;A problem is that the sequences are slightly different lengths so something like a variant call format on the &amp;quot;main&amp;quot; strain might work, but I'm not sure how I can do this on multiple files of this length.&lt;/p&gt;&#xA;&lt;p&gt;The end goal is actually to be able to call which substrain is present from NGS data. I imagine using variant calling on the &amp;quot;Main&amp;quot; strain, but getting from there to calling which substrain is in the data is a bit of a long way for me. If someone have any suggestions on tools for this, please enlighten me :)&lt;/p&gt;&#xA;&lt;p&gt;Thank you very much!&lt;/p&gt;&#xA;" OwnerUserId="12151" LastEditorUserId="12151" LastEditDate="2021-05-07T07:44:13.363" LastActivityDate="2021-05-12T06:40:03.517" Title="Getting exact mismatch data from multiple sequence alignment" Tags="|multiple-sequence-alignment|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15876" PostTypeId="1" CreationDate="2021-05-06T20:26:03.723" Score="0" ViewCount="40" Body="&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; What are some instances of polythetic datasets in biology? In particular, I am looking for a dataset to benchmark a machine learning algorithm optimized via episodic training.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;On polythetic classification&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The conventional definition of a conceptual or monothetic class is that its members must possess certain properties in common [&lt;a href=&quot;https://www.jstor.org/stable/pdf/2799807.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;]. For example, in the MNIST database of handwritten digits, images can be classified into 10 distinct classes (i.e. digits) where all instances of any given class possess the same set of attributes (e.g. zeros are typically written as ellipses).&lt;/p&gt;&#xA;&lt;p&gt;In contrast, a polythetic class encompasses members that might share a number of characteristics, but none of these attributes is essential for membership of that group or class. For example, one could group green zeros together with red ones in one polythetic class, and red zeros together with green ones in another one, as shown below.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oOS3G.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oOS3G.png&quot; alt=&quot;Example polythetic classification&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Are there any instances of biological datasets with similar properties? I am particularly thinking of genetics data (e.g. single nucleotide polymorphisms) where perhaps a certain mutation A counteracts the effect of mutation B? Any pointers are welcome.&lt;/p&gt;&#xA;" OwnerUserId="9072" LastActivityDate="2021-05-09T18:35:50.410" Title="Biological datasets for polythetic classification" Tags="|snp|machine-learning|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15877" PostTypeId="1" AcceptedAnswerId="15890" CreationDate="2021-05-07T04:10:44.110" Score="2" ViewCount="254" Body="&lt;p&gt;I was doing a Virtual Screening Search using Pharmer software. The .sdf output file contains a column in mol format and another column with Root Mean Square Deviations. I can see some bond order and connectivity problems in the mol column. I do have a mol2 file that contains the original molecules and I was wondering if I can use these mol2 files, to fix this. Can rdkit work in this case?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hwEFI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hwEFI.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;a href=&quot;https://i.stack.imgur.com/hRzUM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hRzUM.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12153" LastEditorUserId="12153" LastEditDate="2021-05-08T03:08:57.857" LastActivityDate="2021-05-09T22:10:12.047" Title="Is it possible to correct bond order and conectivity problems in an sdf file according to mol2 files?" Tags="|rdkit|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15878" PostTypeId="1" CreationDate="2021-05-07T06:25:02.063" Score="1" ViewCount="30" Body="&lt;p&gt;I'm trying to predict targets for miRNA sequences, and so far after reading some papers for other tools that do this, I've found that I should be using FASTA's &lt;code&gt;ssearch36&lt;/code&gt; for this.&lt;/p&gt;&#xA;&lt;p&gt;I'm starting with a small file of 500 miRNA sequences, searching a reference DB of 50k sequences. When using &lt;code&gt;ssearch&lt;/code&gt;, I get the following output at the very bottom after ~85 minutes of running:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Total Scan time: 294.000 Total Display time: 4681.000&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am piping the output to a file, and the result file is ~13 Gb. It is taking this long even when I pipe the output to &lt;code&gt;&amp;gt;nul&lt;/code&gt; so I don't think it's my disk speed.&lt;/p&gt;&#xA;&lt;p&gt;I'm wondering if there's any way to speed this up, since it seems very unreasonable to me that the actual search itself only takes ~5 minutes but yet taking over an hour to simply generate output. Is it a bug or am I interpreting it incorrectly?&lt;/p&gt;&#xA;" OwnerUserId="12154" LastActivityDate="2021-05-07T06:25:02.063" Title="How can I speed up FASTA ssearch output?" Tags="|fasta|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15879" PostTypeId="2" ParentId="3275" CreationDate="2021-05-07T09:51:09.300" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @heathobrien converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;looks like that link has been dead for years:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/16763/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/16763/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Answer from @jason-aller converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;The biostars article mentions this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://baderlab.org/BINDTranslation&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://baderlab.org/BINDTranslation&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-07T09:51:09.300" CommentCount="0" CommunityOwnedDate="2021-05-07T09:51:09.300" ContentLicense="CC BY-SA 4.0" />
  <row Id="15880" PostTypeId="1" CreationDate="2021-05-07T09:55:26.050" Score="1" ViewCount="628" Body="&lt;p&gt;I did &lt;code&gt;gmx trjconv&lt;/code&gt; on GROMACS &lt;code&gt;trr&lt;/code&gt; file to generate a &lt;code&gt;.xtc&lt;/code&gt; file, and then used &lt;code&gt;gmx dump&lt;/code&gt; and saved its standard output in a file. Then I tried to make a movie with VMD using the file. As a result, it does not work properly. The error in the terminal is as follows.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gromacsplugin) Cannot read header fromm '/Users/kondouakira/mount/Lab/1ns_sim/md_0_1_noPBC3.trj', no error&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I use following command.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gmx trjconv -s md.tpr -f md.trr -o md_0.trr -pbc mol -ur compact -b 1 -force&#xA;gmx dump -f md_0.trr &amp;gt; md_0.trj&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9890" LastEditorUserId="12054" LastEditDate="2021-05-31T17:34:00.760" LastActivityDate="2021-05-31T17:34:00.760" Title="How to make a video of a gromacs gmx dump standard output with VMD" Tags="|gromacs|molecular-dynamics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15881" PostTypeId="2" ParentId="12866" CreationDate="2021-05-07T09:59:12.273" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from StupidWolf, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;There is a function call Read10X in seurat, you can see how to use it with the example in &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/rkit&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/rkit&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You place the three files into a directory / folder, then you specify Read10X()... hey how big is your data? you might need to run this on a machine with enough RAM.&lt;/p&gt;&#xA;&lt;p&gt;You can try downloading the data following instructions here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://satijalab.org/seurat/v3.1/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/v3.1/pbmc3k_tutorial.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-07T09:59:12.273" CommentCount="0" CommunityOwnedDate="2021-05-07T09:59:12.273" ContentLicense="CC BY-SA 4.0" />
  <row Id="15882" PostTypeId="2" ParentId="15875" CreationDate="2021-05-07T11:35:33.660" Score="0" Body="&lt;p&gt;I have partly answered this for know! For anyone interested i use kalign to align all 10 strains to a fasta format, then I use snp-sites to output a vcf file that can show each mismatch in accordance to the first strain in the fasta. New problem is that the alignment &amp;quot;stretches&amp;quot; the length of this first strain, because some of the other strains are a bit longer. This means that the position of variants are with reference to the longest strain of the 10 and not the strain I want. It can work for now, but any solution to this, would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="12151" LastEditorUserId="12151" LastEditDate="2021-05-07T11:56:34.353" LastActivityDate="2021-05-07T11:56:34.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15883" PostTypeId="1" CreationDate="2021-05-07T13:09:59.043" Score="1" ViewCount="19" Body="&lt;p&gt;I'm learning to build Bayesian Network Based Modeling of Gene Regulatory Network of Cancerous Cellular Cells. So far, I have learned to form a Bayesian Network based on nature of genes(i.e Functional Genes and TFs). However, I cannot figure out how to give probability to each node/vertex. Is it possibile to somehow iterpret the centrality score of each vertex an associate the meaning with the probabilistic model in terms of cause and effect?&lt;/p&gt;&#xA;&lt;p&gt;or can you atleast refer me a relevant research paper?&lt;/p&gt;&#xA;&lt;p&gt;I believe I would be worth mentioning that I'm fairly new to this.&lt;/p&gt;&#xA;&lt;p&gt;Here's my data of TFs and Sources.&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;TF&lt;/th&gt;&#xA;&lt;th&gt;Target&lt;/th&gt;&#xA;&lt;th&gt;Model of Regulation&lt;/th&gt;&#xA;&lt;th&gt;References&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;ABL1&lt;/td&gt;&#xA;&lt;td&gt;BCL2&lt;/td&gt;&#xA;&lt;td&gt;Repression&lt;/td&gt;&#xA;&lt;td&gt;11753601&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;AHR&lt;/td&gt;&#xA;&lt;td&gt;FOS&lt;/td&gt;&#xA;&lt;td&gt;Unknown&lt;/td&gt;&#xA;&lt;td&gt;10478842&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;APC&lt;/td&gt;&#xA;&lt;td&gt;BIRC5&lt;/td&gt;&#xA;&lt;td&gt;Unknown&lt;/td&gt;&#xA;&lt;td&gt;11751382&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;AR&lt;/td&gt;&#xA;&lt;td&gt;ERBB2&lt;/td&gt;&#xA;&lt;td&gt;Activation&lt;/td&gt;&#xA;&lt;td&gt;21613411&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;AR&lt;/td&gt;&#xA;&lt;td&gt;ERBB2&lt;/td&gt;&#xA;&lt;td&gt;Unknown&lt;/td&gt;&#xA;&lt;td&gt;21741601&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;Following is the sample of my data for network analysis, mostly containing centrality scores some graph properties.&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;SUID&lt;/th&gt;&#xA;&lt;th&gt;name&lt;/th&gt;&#xA;&lt;th&gt;BetweennessCentrality&lt;/th&gt;&#xA;&lt;th&gt;ClosenessCentrality&lt;/th&gt;&#xA;&lt;th&gt;ClusteringCoefficient&lt;/th&gt;&#xA;&lt;th&gt;Eccentricity&lt;/th&gt;&#xA;&lt;th&gt;EdgeCount&lt;/th&gt;&#xA;&lt;th&gt;Indegree&lt;/th&gt;&#xA;&lt;th&gt;IsSingleNode&lt;/th&gt;&#xA;&lt;th&gt;NeighborhoodConnectivity&lt;/th&gt;&#xA;&lt;th&gt;Outdegree&lt;/th&gt;&#xA;&lt;th&gt;PartnerOfMultiEdgedNodePairs&lt;/th&gt;&#xA;&lt;th&gt;selected&lt;/th&gt;&#xA;&lt;th&gt;SelfLoops&lt;/th&gt;&#xA;&lt;th&gt;shared name&lt;/th&gt;&#xA;&lt;th&gt;Stress&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;7228&lt;/td&gt;&#xA;&lt;td&gt;TP53&lt;/td&gt;&#xA;&lt;td&gt;0.06611272&lt;/td&gt;&#xA;&lt;td&gt;0.46456693&lt;/td&gt;&#xA;&lt;td&gt;0.04141865&lt;/td&gt;&#xA;&lt;td&gt;4&lt;/td&gt;&#xA;&lt;td&gt;75&lt;/td&gt;&#xA;&lt;td&gt;50&lt;/td&gt;&#xA;&lt;td&gt;FALSE&lt;/td&gt;&#xA;&lt;td&gt;14.40625&lt;/td&gt;&#xA;&lt;td&gt;25&lt;/td&gt;&#xA;&lt;td&gt;11&lt;/td&gt;&#xA;&lt;td&gt;FALSE&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;TP53&lt;/td&gt;&#xA;&lt;td&gt;39572&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;6788&lt;/td&gt;&#xA;&lt;td&gt;AR&lt;/td&gt;&#xA;&lt;td&gt;0.03605041&lt;/td&gt;&#xA;&lt;td&gt;0.40410959&lt;/td&gt;&#xA;&lt;td&gt;0.03164983&lt;/td&gt;&#xA;&lt;td&gt;4&lt;/td&gt;&#xA;&lt;td&gt;60&lt;/td&gt;&#xA;&lt;td&gt;46&lt;/td&gt;&#xA;&lt;td&gt;FALSE&lt;/td&gt;&#xA;&lt;td&gt;13.38181818&lt;/td&gt;&#xA;&lt;td&gt;14&lt;/td&gt;&#xA;&lt;td&gt;5&lt;/td&gt;&#xA;&lt;td&gt;FALSE&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;AR&lt;/td&gt;&#xA;&lt;td&gt;20498&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;7029&lt;/td&gt;&#xA;&lt;td&gt;MYC&lt;/td&gt;&#xA;&lt;td&gt;0.03342125&lt;/td&gt;&#xA;&lt;td&gt;0.35014837&lt;/td&gt;&#xA;&lt;td&gt;0.03280053&lt;/td&gt;&#xA;&lt;td&gt;5&lt;/td&gt;&#xA;&lt;td&gt;81&lt;/td&gt;&#xA;&lt;td&gt;74&lt;/td&gt;&#xA;&lt;td&gt;FALSE&lt;/td&gt;&#xA;&lt;td&gt;12.93589744&lt;/td&gt;&#xA;&lt;td&gt;7&lt;/td&gt;&#xA;&lt;td&gt;3&lt;/td&gt;&#xA;&lt;td&gt;FALSE&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;MYC&lt;/td&gt;&#xA;&lt;td&gt;19140&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;" OwnerUserId="12155" LastActivityDate="2021-05-07T13:09:59.043" Title="How can I associate centrality score of genes(vertices) in a GRN with the probabilistic model in terms of cause and effect?" Tags="|gene|gene-expression|networks|transcriptome-regulation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15884" PostTypeId="2" ParentId="4873" CreationDate="2021-05-07T18:58:03.647" Score="1" Body="&lt;p&gt;GEO provides a GUI for this (just click on the &amp;quot;Analyze with GEO2 button&amp;quot;), you can even check/download the R/Bioconductor code used for the analyses.&lt;/p&gt;&#xA;&lt;p&gt;Basically you will create groups of samples to compare/contrast and R/Bioconductor will run behind the scenes. Here is a nice &lt;a href=&quot;https://www.youtube.com/watch?v=EUPmGWS8ik0&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2021-05-07T18:58:03.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15885" PostTypeId="1" CreationDate="2021-05-07T20:15:20.513" Score="1" ViewCount="148" Body="&lt;p&gt;I am analyzing Sanger sequencing data to verify whether or not the correct protein coding sequence was inserted during cloning.&lt;/p&gt;&#xA;&lt;p&gt;For each sample, I have both forward and reverse Sanger sequencing reads that cover the coding sequence.&lt;/p&gt;&#xA;&lt;p&gt;Would it be better if I first align the forward and reverse reads to get a consensus, and then align the consensus to the reference?&#xA;Or should I just align each read directly to the reference?&lt;/p&gt;&#xA;" OwnerUserId="11905" LastActivityDate="2021-05-08T01:23:24.613" Title="Alignment to verify plasmid from forward and reverse Sanger sequencing reads" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15886" PostTypeId="2" ParentId="15885" CreationDate="2021-05-08T01:23:24.613" Score="2" Body="&lt;p&gt;Align all reads from a sample at the same time IMO. This will give you all of the information at once, and it should not be difficult. It should be very obvious whenever there's a disagreement between reads or between reads and reference.&lt;/p&gt;&#xA;&lt;p&gt;I would also recommend inspecting chromatograms directly at all sites of interest (such disagreements, insertion sites, possible errors or mutations).&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-05-08T01:23:24.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15887" PostTypeId="1" CreationDate="2021-05-08T15:51:27.203" Score="3" ViewCount="55" Body="&lt;p&gt;I am an undergrad working on a set of genotype matrix of dogs, it's in csv file format with information on only the each sample as rows, chromosome and its position as columns, and the rest of the matrix is filled with 0,1,2 and missing values.&lt;/p&gt;&#xA;&lt;p&gt;So I am wondering what would be the best method to impute the genotype with all the data I get? I looked up Impute2, but it seems to require the reference data(recombination map) which I don't have...&lt;/p&gt;&#xA;&lt;p&gt;Really appreciate whoever answers my question!!&lt;/p&gt;&#xA;" OwnerUserId="12162" LastActivityDate="2021-06-09T07:13:54.703" Title="Imputation of dog genotype" Tags="|imputation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15888" PostTypeId="2" ParentId="15887" CreationDate="2021-05-09T12:06:19.767" Score="2" Body="&lt;p&gt;Impute2 is a great, if dated choice. Another great choice is &lt;a href=&quot;http://faculty.washington.edu/browning/beagle/beagle.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;beagle&lt;/a&gt;. The advantage of beagle in this case is that it is not as strict about requiring a recombination map. Without a map, results will be poorer, but perhaps good enough.&lt;/p&gt;&#xA;&lt;p&gt;Do make sure that your sample size including a reference panel is at least 200 in any case.&lt;/p&gt;&#xA;" OwnerUserId="492" LastEditorUserId="492" LastEditDate="2021-06-09T07:13:54.703" LastActivityDate="2021-06-09T07:13:54.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15889" PostTypeId="2" ParentId="15876" CreationDate="2021-05-09T18:35:50.410" Score="1" Body="&lt;p&gt;I may not be understanding exactly what you are talking about, but it seems to me that nearly any set of classes of interest must be polythetic. Most groups of comparable things differ in some characteristics but also have characteristics in common, and none of those characters is 100% identical with membership in the class.&lt;/p&gt;&#xA;&lt;p&gt;Some examples off the top of my head:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Biological species / taxonomic groups (classify according to phenotypic characters)&lt;/li&gt;&#xA;&lt;li&gt;Biological species / taxonomic groups (classify according to molecular sequence e.g. DNA)&lt;/li&gt;&#xA;&lt;li&gt;Cell types (classify according to cell epitopes)&lt;/li&gt;&#xA;&lt;li&gt;Cell types (classify according to transcriptional profile)&lt;/li&gt;&#xA;&lt;li&gt;Ecosystems (classify according to species profile)&lt;/li&gt;&#xA;&lt;li&gt;Disease state (classify according to any one of many molecular profiles)&lt;/li&gt;&#xA;&lt;li&gt;Members of population groups (classify according to genetic variation)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;etc.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-05-09T18:35:50.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15890" PostTypeId="2" ParentId="15877" CreationDate="2021-05-09T22:10:12.047" Score="2" Body="&lt;p&gt;RDKit has a one line command to fix the bond order (&lt;a href=&quot;https://www.rdkit.org/docs/source/rdkit.Chem.AllChem.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;docs&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; mol = Chem.MolFromMolFile('wonky.mol')&#xA; template = Chem.MolFromMol2File('template.mol2')&#xA; newMol = AllChem.AssignBondOrdersFromTemplate(template, mol)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The problem is that the two need to be identical. I see a carboxylate in your screenshots, so you most likely did the correct thing and docked with the appropriate protonation for pH 7, i.e. as the conjugate base with a negative formal charge. If one does not then RDKit will fail.&lt;/p&gt;&#xA;&lt;p&gt;Also, RDKit is &amp;quot;not great&amp;quot; with mol2 files. If you have the original SMILES that is better. If the problem is atom names, using a PDB is a better call. If you want to assign atom names, this &lt;a href=&quot;https://blog.matteoferla.com/2020/03/atom-names-purely-in-rdkit.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt; of mine may help.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-05-09T22:10:12.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15891" PostTypeId="2" ParentId="4196" CreationDate="2021-05-10T05:55:25.607" Score="1" Body="&lt;p&gt;It's likely that the caller is being overly sensitive to deletions, which are the most common error mode for nanopore reads, even three years later in 2021.&lt;/p&gt;&#xA;&lt;p&gt;What I'd guess is happening is that the caller is seeing a string of consecutive 1bp deletions at high frequency, and treating these as a consensus deletion covering many more bases. Looking at the pileup calls for the surrounding bases may help to diagnose this.&lt;/p&gt;&#xA;&lt;p&gt;My general advice to people doing variant calling with nanopore is to be suspicious of deletions, especially if they include homopolymers. That usually means trying to confirm any important variants with another technology.&lt;/p&gt;&#xA;&lt;p&gt;Newer, more accurate, base calling models are being created all the time by ONT, so if you still have the raw FAST5 data, it might be worth it to run through the most recent base caller.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-10T05:55:25.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15892" PostTypeId="1" CreationDate="2021-05-10T10:30:31.893" Score="2" ViewCount="78" Body="&lt;p&gt;I need to annotate human genome with different types of information, amongst them: transposable elements.&#xA;I read about repeatmasker, but what I understood is that it estimates the probability of a sequence to be a transposon based on the sequence and not based on curated database.&#xA;My university does not have access to RepBase (GIRI). I was hoping finding some possibility with biomart but it seems not possible.&lt;/p&gt;&#xA;&lt;p&gt;What would be a way to access curated annotation of all kinds of transposable elements (SINE, Alu seq..) within a given genome/genome coordinates?&#xA;My goal is to estimate the link between some measurements we have made on the genome (binned) ( and Shannon entropy is one of them, and probably the less interesting regarding TE). Eventually, I'd like to be able to estimate that measurement X has a correlation with the number of Alu repeats present in the neighbourhood.&lt;/p&gt;&#xA;&lt;p&gt;What would be the best way to achieve this?&lt;/p&gt;&#xA;" OwnerUserId="12141" LastEditorUserId="57" LastEditDate="2021-05-10T11:08:48.660" LastActivityDate="2021-05-10T11:13:43.347" Title="Annotatation of transposable elements / transposons / SINE sequences in human" Tags="|repeat-elements|repeatmasker|transposable-elements|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15893" PostTypeId="2" ParentId="15892" CreationDate="2021-05-10T11:13:43.347" Score="3" Body="&lt;p&gt;repeatmasker is meant to annotate/mask a genome given a library of known transposable elements. From the project description, I assume you have not done any sequencing on your own. Then, instead of annotating, I think you should find already computed annotations of TEs.&lt;/p&gt;&#xA;&lt;p&gt;The alternative to subscription-based repbase is &lt;a href=&quot;https://www.dfam.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dfam&lt;/a&gt;. And Just looking at their webpage they seem to have an annotation of all individual TEs in &lt;a href=&quot;https://www.dfam.org/releases/Dfam_3.3/annotations/hg38/&quot; rel=&quot;nofollow noreferrer&quot;&gt;hg38&lt;/a&gt;. I bet my shoes there will be plenty of TE annotations in human in genome databases too.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2021-05-10T11:13:43.347" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15895" PostTypeId="1" CreationDate="2021-05-10T19:38:16.560" Score="2" ViewCount="49" Body="&lt;p&gt;I want to differentiate between classical class I and non classical class I MHC molecules in a model organism using well conserved structural features within classical MHC I molecules (eg intradomain disulfide bridges within class I a1 domain). To do this I need to align the peptide sequences of transcripts of that model organism with transcripts of classical MHC I human genes and see which genes from those transcripts have those structural features. So I am looking for a tool to align and annotate those structural features in a replicable manner&lt;/p&gt;&#xA;&lt;p&gt;Additionally if you have any other bioinformatic ideas /computational analyses on how to find classical MHC Class I molecules on an organism which is poorly annotated that would highly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="12135" LastActivityDate="2023-06-01T00:00:30.197" Title="Differentiating molecules based on peptide sequence? How to annotate?" Tags="|sequence-alignment|protein-structure|features|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15896" PostTypeId="2" ParentId="15880" CreationDate="2021-05-11T08:57:33.543" Score="2" Body="&lt;p&gt;No need to use &lt;code&gt;gmx dump&lt;/code&gt;, you can simply load a &lt;code&gt;xtc&lt;/code&gt; or &lt;code&gt;trr&lt;/code&gt; trajectory in VMD using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;vmd path/to/gro_file.gro other/path/to/trajectory_file.xtc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As always you can also use the GUI to first open your &lt;code&gt;gro&lt;/code&gt; file, then right click on the molecule and select &lt;code&gt;Load data into molecule&lt;/code&gt; before selecting your &lt;code&gt;xtc&lt;/code&gt; or &lt;code&gt;trr&lt;/code&gt; file.&lt;/p&gt;&#xA;&lt;p&gt;After putting the representations as you see fit, if you want to export this as a movie (GIF, MP4, others...) you can go in Extensions -&amp;gt; Visualization -&amp;gt; Movie Maker. See &lt;a href=&quot;http://www.ks.uiuc.edu/Training/Tutorials/vmd/tutorial-html/node3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trajectories and Movie Making&lt;/a&gt; VMD official tutorial for more details.&lt;/p&gt;&#xA;" OwnerUserId="12054" LastActivityDate="2021-05-11T08:57:33.543" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15899" PostTypeId="2" ParentId="15895" CreationDate="2021-05-11T18:25:02.450" Score="1" Body="&lt;p&gt;Knowing almost nothing about the biology, I think that the most straightforward way to do this is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;train profile HMMs on your different classes of protein sequences (in human maybe, but ideally including MHCs from a large number of organisms)&lt;/li&gt;&#xA;&lt;li&gt;translate your transcripts into proteins and run them against your trained profile HMMs&lt;/li&gt;&#xA;&lt;li&gt;confirm that the profiles adequately discriminate the two classes and adjust if needed.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;For an example of how to do this (on MHC), you can see &lt;a href=&quot;https://www.nature.com/articles/s41598-019-50932-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;These approaches are then easily extensible to further organisms, as you have the profile HMMs.&lt;/p&gt;&#xA;&lt;p&gt;It's possible that a simpler approach would consist of just finding hits against human MHC and then just checking for the existence of sequence patterns corresponding to those structural motifs, but I doubt that would work well in any significantly divergent organism.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-05-11T18:25:02.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15901" PostTypeId="1" AcceptedAnswerId="15903" CreationDate="2021-05-11T20:13:12.513" Score="1" ViewCount="95" Body="&lt;p&gt;I have two lists of dataframes. List &lt;code&gt;x&lt;/code&gt; and list &lt;code&gt;my_list&lt;/code&gt;. Each having four dataframes and their names are same for the two lists, i.e., &lt;code&gt;bray-curtis, Chao1, FB&lt;/code&gt;, and &lt;code&gt;Shannon&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The dataframes within the two lists have same element names, but they contain different information. This is because list &lt;code&gt;x&lt;/code&gt; has been generated after some operations on list &lt;code&gt;my_list&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now I want to write a function and run that on list &lt;code&gt;x&lt;/code&gt; with `lapply. the command I want to write as a function is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;forest(x&lt;span class=&quot;math-container&quot;&gt;$Chao1, slab = paste(my_list$&lt;/span&gt;Chao1$StudyID))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here, the first &lt;code&gt;Chao1&lt;/code&gt; is coming from list &lt;code&gt;x&lt;/code&gt; while the second  &lt;code&gt;Chao1&lt;/code&gt; is coming from the list &lt;code&gt;my_list&lt;/code&gt;. It works for a single dataframe but I am unable to write the codes with &lt;code&gt;lapply&lt;/code&gt; so that it runs across all the dataframes of the lists.&lt;/p&gt;&#xA;&lt;p&gt;The desired outputs are four plots from four dataframes of list &lt;code&gt;x&lt;/code&gt;.&#xA;Can anyone please help me?&lt;/p&gt;&#xA;&lt;p&gt;Update: @gringer's approach works, but it saves all the plots in a single list. When I tried to save the images with following command it just saved the last image:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tiff(filename = &amp;quot;~/Desktop/image.tiff&amp;quot;, width = 30, height = 30, units = &amp;quot;cm&amp;quot;, res = 1200)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there a way to save all them individually?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="7427" LastEditorUserId="73" LastEditDate="2021-05-13T00:12:24.143" LastActivityDate="2021-05-13T00:12:24.143" Title="How to create this function for lapply?" Tags="|r|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15902" PostTypeId="1" CreationDate="2021-05-11T22:27:40.807" Score="1" ViewCount="266" Body="&lt;p&gt;I have 5 sets of Log2 Fold Change values pulled off DualSeqDB. Each of the 5 groups corresponds to a different microbial taxon. Within the 5 groups, there are many Log2 Fold Change values, each corresponding to a gene that was significantly affected by the presence of the microbe.&lt;/p&gt;&#xA;&lt;p&gt;My question is: is there any way to get the average Log2 Fold Change for each group? How could I calculate that?&lt;/p&gt;&#xA;&lt;p&gt;The end goal is to compare each microbe's effect on gene expression, hopefully to prove that the phenotype of the microbe is in some way related to how much it can change the gene expression in its host.&lt;/p&gt;&#xA;&lt;p&gt;I have considered using p-values to compare each taxa's effect on gene expression, but because all the datasets are significant, the p-values I got using fisher's method are all 0, so I don't know how I could go ahead and compare that to categorical variables (like presence of a certain phenotype).&lt;/p&gt;&#xA;&lt;p&gt;Thanks so much to anyone who can help!&lt;/p&gt;&#xA;" OwnerUserId="12208" LastActivityDate="2021-05-11T22:27:40.807" Title="How to compare Log2 Fold Change values" Tags="|statistics|microbial-genomics|fold-change|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15903" PostTypeId="2" ParentId="15901" CreationDate="2021-05-12T01:33:03.443" Score="2" Body="&lt;p&gt;Assuming the lists are named, and you want to fetch the element with the same name from each list, I'm guessing something like this would work (or at least get you closer to a solution):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;v &amp;lt;- lapply(names(x), function(var){&#xA;   forest(x[[var]], slab=(my_list[[var]])$StudyID)&#xA;})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When creating images that don't have a multi-page format (i.e. anything except PDF), and not changing the image name with each save, you need to add in a format specifier to tell R what to do with new created images. For example, this will add a three digit zero-padded number for each new TIFF image:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tiff(filename = &amp;quot;~/Desktop/image_%03d.tiff&amp;quot;, width = 30, height = 30, units = &amp;quot;cm&amp;quot;, res = 1200)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-05-12T21:36:04.500" LastActivityDate="2021-05-12T21:36:04.500" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15904" PostTypeId="2" ParentId="4677" CreationDate="2021-05-12T02:11:27.020" Score="1" Body="&lt;p&gt;My recommendation would be to run &lt;em&gt;Trinity&lt;/em&gt; using the &lt;code&gt;--long_reads&lt;/code&gt; option, which allows you to provide an error-corrected fastq file for anchoring reads to a transcript isoform. The reads are then clustered per-isoform, and assembled, in a similar fashion to what's done with genome-guided Trinity:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/trinityrnaseq/trinityrnaseq/wiki/Running-Trinity&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/trinityrnaseq/trinityrnaseq/wiki/Running-Trinity&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Trinity --seqType fq --max_memory 50G --long_reads corrected_pacbio_reads.fa --left reads_1.fq  --right reads_2.fq --CPU 6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;Disclaimer: I have contributed code to Trinity at some time in the distant past&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-12T02:11:27.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15905" PostTypeId="2" ParentId="3270" CreationDate="2021-05-12T02:18:06.593" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @llrs, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Probably is a question for the maintainer but I would &lt;em&gt;guess&lt;/em&gt; that is a variance of the gene in the control dataset. So, if in normal samples the gene has a standard deviation of 0.5 this is the expected too in the tumoral cells. calculate the sd only on the control samples. So if the healthy/control samples have such variation the altered ones should have these or more according to the article (if I understand it correctly).&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;std &amp;lt;- apply(data[, controls], 1, sd, na.rm = TRUE); quantile(std, 0.25)&lt;/code&gt; should be much faster and efficient&lt;/p&gt;&#xA;&lt;p&gt;But  I would check it in &lt;a href=&quot;https://support.bioconductor.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;support.bioconductor&lt;/a&gt; with the authors/maintainers.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-12T02:18:06.593" CommentCount="0" CommunityOwnedDate="2021-05-12T02:18:06.593" ContentLicense="CC BY-SA 4.0" />
  <row Id="15906" PostTypeId="2" ParentId="3097" CreationDate="2021-05-12T02:20:47.020" Score="2" Body="&lt;p&gt;&lt;em&gt;Answer from @llrs, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;You could try to download RNAseq data from this bacteria and see if some of the RNA align to your sequence. You could try to blast your gene and look for the sequences that have low identity similarity (or classify them by type and then look what is each hit).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-12T02:20:47.020" CommentCount="1" CommunityOwnedDate="2021-05-12T02:20:47.020" ContentLicense="CC BY-SA 4.0" />
  <row Id="15907" PostTypeId="2" ParentId="2519" CreationDate="2021-05-12T02:22:04.593" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @fabio-marroni, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;WARNING! Self-promotion! We published a freely-available tool for detection of LOH (and also redeuction fo homozygosity). Here you find the link to the paper and therein will find the link to the software. We compared with other tools that you might also try to see which one is the best performing. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5466433/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5466433/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-12T02:22:04.593" CommentCount="0" CommunityOwnedDate="2021-05-12T02:22:04.593" ContentLicense="CC BY-SA 4.0" />
  <row Id="15908" PostTypeId="2" ParentId="15875" CreationDate="2021-05-12T06:40:03.517" Score="0" Body="&lt;p&gt;I found a solution.&#xA;For variant calling on multple sequence alignment I can recommend snp-sites.&#xA;For anyone facing a similar problem of wanting to find substrains, here is an overview of useful tools:&#xA;&amp;quot;Computational Methods for Strain-Level Microbial Detection in Colony and Metagenome Sequencing Data&amp;quot;&#xA;(&lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fmicb.2020.01925/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.frontiersin.org/articles/10.3389/fmicb.2020.01925/full&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="12151" LastActivityDate="2021-05-12T06:40:03.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15909" PostTypeId="1" AcceptedAnswerId="15910" CreationDate="2021-05-12T11:10:21.050" Score="2" ViewCount="356" Body="&lt;p&gt;With HISAT2, after the alignment of fastq files you get an alignment summary like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;20000 reads; of these:&#xA;  20000 (100.00%) were unpaired; of these:&#xA;    1247 (6.24%) aligned 0 times&#xA;    18739 (93.69%) aligned exactly 1 time&#xA;    14 (0.07%) aligned &amp;gt;1 times&#xA;93.77% overall alignment rate&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is it possible to get this information with Salmon?&lt;/p&gt;&#xA;" OwnerUserId="10386" LastActivityDate="2021-05-12T11:23:51.500" Title="Get Salmon mapping/alignment summary" Tags="|rna-seq|sequence-alignment|salmon|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15910" PostTypeId="2" ParentId="15909" CreationDate="2021-05-12T11:23:51.500" Score="6" Body="&lt;p&gt;The &lt;code&gt;salmon&lt;/code&gt; output log contains a line with &lt;code&gt;[info] Mapping rate&lt;/code&gt; which allows to calculate how many reads went mapped or unmapped. Is that sufficient? It should be in the &lt;code&gt;logs&lt;/code&gt; folder of the directory &lt;code&gt;salmon&lt;/code&gt; creates per sample.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-05-12T11:23:51.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15912" PostTypeId="2" ParentId="15534" CreationDate="2021-05-13T03:12:01.930" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @ram-rs, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;There's a Hammock.jar file - there' no need to compile Hammock, although they recommend compiling dependencies. Have you tried running java -jar dist/Hammock.jar --help? Follow their steps from Example 1 - it should work.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-13T03:12:01.930" CommentCount="0" CommunityOwnedDate="2021-05-13T03:12:01.930" ContentLicense="CC BY-SA 4.0" />
  <row Id="15913" PostTypeId="1" CreationDate="2021-05-13T10:50:49.503" Score="1" ViewCount="220" Body="&lt;p&gt;I have paired-end fastq files some of which seem to be in a weird format (from a collaborator, not a public database). When I &lt;code&gt;cat&lt;/code&gt; the file I get what seem to me to be binary syntax but not sure?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CvKUz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CvKUz.png&quot; alt=&quot;weird output&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However when I &lt;code&gt;cat&lt;/code&gt; into the other pair-end the output seems fine&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/N9ouH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/N9ouH.png&quot; alt=&quot;correct output&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Any help/suggestion/solution would be greatly appreciated.&lt;/p&gt;&#xA;&lt;p&gt;It does seem to be compressed with a missing suffix:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;file pair1.fastq&#xA;   pair1.fastq: data.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12219" LastEditorUserId="73" LastEditDate="2021-05-14T09:33:20.950" LastActivityDate="2021-05-14T09:33:20.950" Title="fastq file format unknown" Tags="|sequence-alignment|ngs|trimming|bowtie2|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15914" PostTypeId="2" ParentId="15913" CreationDate="2021-05-13T13:03:34.673" Score="3" Body="&lt;p&gt;This indeed looks like some sort of binary file. I have seen bunch of those when people renamed &lt;code&gt;.fastq.gz&lt;/code&gt; to &lt;code&gt;.fastq&lt;/code&gt; without actually uncompressing them. However, as you say, this is not your case, as &lt;code&gt;file pair1.fastq&lt;/code&gt; would identify the file as &lt;code&gt;gzip compressed data&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Looking at possible return values of &lt;code&gt;file&lt;/code&gt; comments &lt;a href=&quot;https://unix.stackexchange.com/questions/151008/linux-file-command-classifying-files&quot;&gt;here&lt;/a&gt;, &lt;code&gt;data&lt;/code&gt; basically mean the command could not figure out anything about it beside being some sort of binary data file, but I suspect it probably somehow broken file. Honestly, at this point I think you should check the source of the data. If it's from a public database, re-download the data. If it's your own sequencing, contact the sequencing provider and check the data with them. I don't see any way to rescue this file (which does not necerasrily mean there is nothing you can do).&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2021-05-13T13:03:34.673" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15915" PostTypeId="1" CreationDate="2021-05-13T19:20:46.053" Score="4" ViewCount="50" Body="&lt;p&gt;&lt;em&gt;Note: this question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/9470102/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am currently trying to create a theoretical argument that a microbe's phenotype can affect gene expression in their host. I have 5 species of microbes, each with a different COG (Cluster of Orthologous Genes) profile.  I've been using python and a large database (&lt;a href=&quot;http://www.tartaglialab.com/dualseq/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DualSeqDB&lt;/a&gt;) of significant changes in gene expression to compare gene expression in a host and the COG profiles of my 5 species.&lt;/p&gt;&#xA;&lt;p&gt;This is an example of the dataframe I'm working off of:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3iNxk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3iNxk.png&quot; alt=&quot;Where columns are counts of COGs present in a microbial species and rows are species&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;columns are counts of COGs present in a microbial species and rows are species&lt;/p&gt;&#xA;&lt;p&gt;I was wondering if there is any way to compare the species and their COG profiles. My end goal is to draw parallels between how prevalent a certain phenotype is and how much that species can affect a host's gene expression.&lt;/p&gt;&#xA;&lt;p&gt;If this seems infeasible, would a better approach be to BLAST the species and compare their genome sequences phylogenetically? (this would change the nature of my argument, so I'm hesitant)&lt;/p&gt;&#xA;&lt;p&gt;I'm not the best with statistics so I'm sorry if there's a really obvious answer I'm not thinking of. If anyone knows of any resources I could use to read up on this stuff more on my own, I would really appreciate that as well.&lt;/p&gt;&#xA;" OwnerUserId="12208" LastEditorUserId="73" LastEditDate="2021-05-14T09:22:38.887" LastActivityDate="2021-05-14T09:22:38.887" Title="How can I use statistics to compare microbial phenotypes?" Tags="|python|microbial-genomics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15916" PostTypeId="1" AcceptedAnswerId="15917" CreationDate="2021-05-14T04:46:41.527" Score="3" ViewCount="314" Body="&lt;p&gt;&lt;code&gt;vcf2maf&lt;/code&gt; uses &lt;code&gt;VEP&lt;/code&gt; to annotate variants, and I believe selects the default Ensembl transcript to use for annotation. Sometimes the transcript that &lt;code&gt;VEP&lt;/code&gt; selects is not the transcript I'm interested in, usually because the selected transcript is not the most highly expressed transcript in my tissue of interest (skin). &lt;code&gt;vcf2maf&lt;/code&gt; allows you to provide a transcript override list so that VEP annotates the variant using the specified transcripts instead.&lt;/p&gt;&#xA;&lt;p&gt;I have several skin samples sequenced with bulk RNA-Seq. I want to estimate the average abundance for each transcript across all samples and then use these abundances to rank transcripts from most to least abundant. Then I will use the most abundant transcript as the default VEP transcript. I plan to use &lt;code&gt;salmon&lt;/code&gt; or &lt;code&gt;kallisto&lt;/code&gt; to quantify transcript abundance. Should I use TPM or normalized counts to calculate average expression?&lt;/p&gt;&#xA;&lt;p&gt;My initial thought is to use normalized counts (generated by DESeq2 from raw counts). Are there any problems with this approach? GTEx displays transcript abundance with average TPM, but I thought TPM was inappropriate to use across samples because it doesn't account for between sample differences.&lt;/p&gt;&#xA;&lt;p&gt;Update: I forgot to mention I also tried using TPM ranks like @ATpoint describes. I haven't fully compared how this compares to transcripts identified by normalized counts, but the initial genes I checked showed good concordance between methods&lt;/p&gt;&#xA;" OwnerUserId="6674" LastEditorUserId="73" LastEditDate="2021-05-14T20:27:29.203" LastActivityDate="2021-05-14T20:27:29.203" Title="Calculating most abundant transcript from RNA-Seq data" Tags="|rna-seq|deseq2|transcriptome|normalization|salmon|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15917" PostTypeId="2" ParentId="15916" CreationDate="2021-05-14T07:37:31.873" Score="5" Body="&lt;p&gt;Transcript abundance quantification is a tricky topic since a read often could belong to several transcripts, so any &amp;quot;count&amp;quot; is a best guess as to which transcript it actually originates from. That being said, there are tools that can help you here:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;salmon&lt;/code&gt; (as you mentioned) to quanitfy. Run it with &lt;code&gt;--numGibbsSamples 50&lt;/code&gt; (or higher if your computer has the hardware. Also run with &lt;code&gt;-d&lt;/code&gt; for the next tool&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;terminus&lt;/code&gt; is a tool that also originates from the COMBINE-lab, It's relatively experimental, but it applies well to your use case. It will take the salmon output and assign counts to transcript groups. If a set of transcripts can be clearly quantified above a certain confidence, that transcript will stay unique. If it is confounded with other transcripts, those will be placed in a transcript group and their counts will be summarised. More info &lt;a href=&quot;https://github.com/COMBINE-lab/terminus&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;You are right in that TPM, FPKM etc. are not appropriate measures to compare across samples. TMM, or DESeq2's median ratio method (what you get with &lt;code&gt;counts(dds, normalized=TRUE)&lt;/code&gt; are a step in the right direction, but are still inappropriate across samples. I'd suggest this &lt;a href=&quot;https://haroldpimentel.wordpress.com/2014/05/08/what-the-fpkm-a-review-rna-seq-expression-units/&quot; rel=&quot;noreferrer&quot;&gt;often posted blog post&lt;/a&gt; in the topic.&lt;/p&gt;&#xA;&lt;p&gt;There are no perfect solutions here since RNA-Seq is inherently a relative quanitifcation method so this will always be an issue. IMO, use &lt;code&gt;DESeq2::varianceStabilizingTransformation()&lt;/code&gt;, &lt;code&gt;DESeq2::rlog()&lt;/code&gt; or &lt;code&gt;limma::voom()&lt;/code&gt; after terminus and rank your transcripts that way.&lt;/p&gt;&#xA;&lt;p&gt;Just keep the inherent transcript incertainties in mind.&lt;/p&gt;&#xA;" OwnerUserId="9693" LastActivityDate="2021-05-14T07:37:31.873" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15918" PostTypeId="2" ParentId="15916" CreationDate="2021-05-14T12:20:03.213" Score="0" Body="&lt;p&gt;If I read you correctly, you want to find the gene with the highest expression?&lt;/p&gt;&#xA;&lt;p&gt;With RNA-seq data from multiple samples, I'd find this by ranking each gene by TPM/FPKM and taking the mean rank across samples. This method will tend to mitigate any large differences in a gene's expression between samples.&lt;/p&gt;&#xA;&lt;p&gt;As you are comparing (RNA-seq) expression across genes you must take into account transcript length.&lt;/p&gt;&#xA;" OwnerUserId="8919" LastActivityDate="2021-05-14T12:20:03.213" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15919" PostTypeId="2" ParentId="15405" CreationDate="2021-05-14T13:46:01.700" Score="1" Body="&lt;p&gt;There are a lot of possible ways to count the number of perfect matches in an alignment, and the details will depend on which alignment tool you are using.&lt;/p&gt;&#xA;&lt;p&gt;But, if you use the built-in pairwise aligner in BioPython (&lt;code&gt;Bio.Align.PairwiseAligner()&lt;/code&gt;, &lt;a href=&quot;https://biopython.org/docs/1.75/api/Bio.Align.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biopython.org/docs/1.75/api/Bio.Align.html&lt;/a&gt;), then the easiest way is to use the formatted string representation of the alignment, which uses the pipe/vertical-bar character ('|') to indicate perfect matches.&lt;/p&gt;&#xA;&lt;p&gt;Example code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Align&#xA;&#xA;aligner = Align.PairwiseAligner()&#xA;alignments = aligner.align(&amp;quot;GAACT&amp;quot;, &amp;quot;GAT&amp;quot;)&#xA;max_matches = 0&#xA;for i, a in enumerate(alignments):&#xA;    align_str = f'{a}'&#xA;    align_str = align_str.split('\n')&#xA;    match_str = align_str[1]&#xA;    match_count = match_str.count('|')&#xA;    mismatch_count = match_str.count('.')&#xA;    print(f'Number of matches: {match_count}')&#xA;    print(f'Number of mismatches: {mismatch_count}')&#xA;    if match_count&amp;gt;max_matches:&#xA;        max_matches = match_count&#xA;        max_ind = i&#xA;&#xA;best_alignment = alignments[max_ind]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In this example, both alignments have the same number of perfect patches (3), but that may not always be the case, depending on the input sequences and the scoring parameters used. The example just uses the default scoring parameters, but in general, you probably want to choose parameters appropriate for your type of alignment.&lt;/p&gt;&#xA;" OwnerUserId="11905" LastActivityDate="2021-05-14T13:46:01.700" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15920" PostTypeId="2" ParentId="4887" CreationDate="2021-05-14T14:11:13.477" Score="1" Body="&lt;p&gt;As suggested in the comments, the built-in BioPython pairwise aligner should work for this. You can find a short sequence match in a longer sequence using a global alignment with the right score parameters (&lt;code&gt;query_end_... = 0&lt;/code&gt;). The other scoring parameters are values that I've found to be good for aligning similar sequences.&lt;/p&gt;&#xA;&lt;p&gt;Example code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Align&#xA;&#xA;aligner = Align.PairwiseAligner()&#xA;aligner = Align.PairwiseAligner()&#xA;aligner.match_score = 5&#xA;aligner.mismatch_score = -9&#xA;aligner.mode = 'global'&#xA;aligner.target_internal_open_gap_score = -12&#xA;aligner.target_internal_extend_gap_score = -3&#xA;aligner.target_end_open_gap_score = -12&#xA;aligner.target_end_extend_gap_score = -3&#xA;aligner.query_internal_open_gap_score = -12&#xA;aligner.query_internal_extend_gap_score = -3&#xA;&#xA;# Set the query_end penalties to zero to effectively implement a 'glocal' alignment as described in the question&#xA;# These are the default values, so you could skip the next two lines, &#xA;#    but this is relevant to the question, so I'm including them in the example.&#xA;aligner.query_end_open_gap_score = 0&#xA;aligner.query_end_extend_gap_score = 0&#xA;    &#xA;alignments = aligner.align(&amp;quot;GAACTCGCATCGATGATCCGTGCGCTGAGATCCGCTGACATGATC&amp;quot;, &amp;quot;GATCCG&amp;quot;)&#xA;&#xA;for a in alignments:&#xA;    print(a)&#xA;print()&#xA;print()&#xA;&#xA;alignments = aligner.align(&amp;quot;GAACTCGCATCGATGATCCGTGCGCTGAGATCCGCTGACATGATC&amp;quot;, &amp;quot;GATACG&amp;quot;)&#xA;&#xA;for a in alignments:&#xA;    print(a)&#xA;print()&#xA;print()&#xA;&#xA;alignments = aligner.align(&amp;quot;GAACTCGCATCGATGATCCGTGCGCTGAGATCCGCTGACATGATC&amp;quot;, &amp;quot;GATACCG&amp;quot;)&#xA;&#xA;for a in alignments:&#xA;    print(a)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11905" LastActivityDate="2021-05-14T14:11:13.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15921" PostTypeId="1" CreationDate="2021-05-15T02:06:09.780" Score="1" ViewCount="21" Body="&lt;p&gt;I'm trying to find the frequency of UA dinucleotide in a bed file. I want to convert the bed file to FASTA, count all the UA dinucleotides up, then divide by the total length of the FASTA file.&lt;/p&gt;&#xA;&lt;p&gt;Also, I probably need to compare the value I get above to the background, but I'm not sure how to go about this?&lt;/p&gt;&#xA;&lt;p&gt;I understand homer should do the counting for me and also create a background distribution for me. It looks like I need to build a basic motif file and give homer a bedfile of locations and motif file to find. However, I'm unsure of how to set this up.&lt;/p&gt;&#xA;&lt;p&gt;Other suggestions are welcome.&lt;/p&gt;&#xA;" OwnerUserId="12230" LastActivityDate="2021-05-15T02:06:09.780" Title="Nucleotide Frequency - HOMER?" Tags="|bash|homer|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15922" PostTypeId="1" CreationDate="2021-05-15T23:53:59.473" Score="1" ViewCount="85" Body="&lt;p&gt;I've recently learned about the reference bias issue - inability to properly map NGS reads in certain genomic regions caused by the fact that our standard genomic references are linear and do not fully represent the genomic variation present in the population. According to what I read this has multiple consequences: missing structural variants or variants in highly variable regions like HLA for humans.&lt;/p&gt;&#xA;&lt;p&gt;Now I wonder what is the best way to address this. Of course, &amp;quot;best&amp;quot; is context-dependent but I don't have any specific context so general pros and cons will be highly appreciated. I know about two approaches but probably there are more. The first one is an alt-aware mapping that's used in DRAGEN by Illumina. As far as I understand, they just keep multiple linear alternatives for some regions and try doing a &amp;quot;smarter&amp;quot; mapping for those regions. The second approach that I've found is using graph references where the reference is not linear anymore but rather a graph and mapping is taking into account this graph structure.&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="12236" LastEditorUserId="492" LastEditDate="2021-05-17T07:27:52.560" LastActivityDate="2021-05-24T23:08:17.203" Title="Solutions to reference bias issue" Tags="|variant-calling|read-mapping|reference-genome|structural-variation|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15923" PostTypeId="1" CreationDate="2021-05-16T09:48:05.557" Score="3" ViewCount="577" Body="&lt;p&gt;I have conducted a large-scale GWAS study and got a few significantly associated SNPs. I used &lt;code&gt;GEMMA&lt;/code&gt; with &lt;code&gt;-lmm 1&lt;/code&gt; options to run the GWAS and obtain the &lt;code&gt;beta&lt;/code&gt; and &lt;code&gt;standard-error&lt;/code&gt; estimates. I want to estimate the percent phenotypic variation explained by each of the significant SNPs. I used the following procedure for estimating the variance explained in &lt;code&gt;R&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fit &amp;lt;- lm (Phenotypic_value ~ SNP_data, data = a)&#xA;summary(fit)$adj.r.squared&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here, the datafile &lt;code&gt;a&lt;/code&gt; contains three columns namely, &lt;code&gt;sample_ID&lt;/code&gt;, &lt;code&gt;Phenotypic_value&lt;/code&gt; for each sample, and the biallelic &lt;code&gt;SNP_data&lt;/code&gt;. I got a value which is 0.43 meaning 43% phenotypic variation explained by the SNP.&lt;/p&gt;&#xA;&lt;p&gt;Again, I used another formula which is: &lt;code&gt;2*f*(1-f)*b.alt^2&lt;/code&gt;. Here, &lt;code&gt;f&lt;/code&gt; is the minor allele frequency and &lt;code&gt;b.alt&lt;/code&gt; is the effect size i.e. &lt;code&gt;beta estimate&lt;/code&gt; obtained from &lt;code&gt;GEMMA&lt;/code&gt;. This gives me a value of 0.03 meaning 3% variation explained which seems reasonable to me.&lt;/p&gt;&#xA;&lt;p&gt;My question is that which of the above method is correct? Why do I obtain such a huge difference between the two approaches? Is there any other way to estimate the percent variation explained?&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, from the GEMMA google group, I have got this formula &lt;code&gt;pve &amp;lt;- var(x) * (beta^2 + se^2)/var(y)&lt;/code&gt;. But I do not understand how can I obtain the value of &lt;code&gt;var(x)&lt;/code&gt; and &lt;code&gt;var(y)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I borrowed the first and the last formula from GEMMA google group discussion. The first one is obviously a normal linear model equation in statistics. The second formula is from &lt;a href=&quot;https://www.mv.helsinki.fi/home/mjxpirin/GWAS_course/material/GWAS3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;. Also, the data structure is three columns, 1 is with individual, 2 is the genetic information (individual carrying A or T) and 3 is the phenotype.&#xA;In the last formula, &lt;code&gt;var(x)&lt;/code&gt; is the variance of the genotype vector as they said in GEMMA group and the &lt;code&gt;beta&lt;/code&gt; is the effect size estimate and &lt;code&gt;se&lt;/code&gt; is the standard error from GEMMA output.&lt;/p&gt;&#xA;&lt;p&gt;It will be great to receive some feedback on this. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="12238" LastEditorUserId="73" LastEditDate="2021-05-19T07:35:11.577" LastActivityDate="2021-05-19T07:38:17.500" Title="How to estimate the phenotypic variation explained by top SNPs from a GWAS study?" Tags="|r|snp|gwas|variation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15924" PostTypeId="2" ParentId="15922" CreationDate="2021-05-17T04:27:18.600" Score="2" Body="&lt;p&gt;&lt;em&gt;Answer from @throckmorton, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is a recent approach to this problem, using multiple linear references:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-020-02229-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-020-02229-3&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-17T04:27:18.600" CommentCount="0" CommunityOwnedDate="2021-05-17T04:27:18.600" ContentLicense="CC BY-SA 4.0" />
  <row Id="15925" PostTypeId="2" ParentId="13701" CreationDate="2021-05-17T04:42:03.467" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @m, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Personally I'd build a tree. Whats your question patients or the relationship between RNAseq 'signatures'? Its not the definitive analysis, but its good mining.&lt;/p&gt;&#xA;&lt;p&gt;Pearson is doable but how many Pearson's? You'll end up with a 2x2 matrix. How will you resolve the matrix clustering? A maximum likelihood phylogeny solution would work here making use of the binary format. There are other approaches.&lt;/p&gt;&#xA;&lt;p&gt;Spearman correlation would be better than Pearson's you end up with a 2x2 matrix.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Answer from @peter-langfelder, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can use Pearson correlation, it is equivalent to carrying out an equal-variance t-test for the module eigengene between the two groups specified by the binary variable.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-17T04:42:03.467" CommentCount="0" CommunityOwnedDate="2021-05-17T04:42:03.467" ContentLicense="CC BY-SA 4.0" />
  <row Id="15926" PostTypeId="1" CreationDate="2021-05-17T08:11:23.373" Score="1" ViewCount="128" Body="&lt;p&gt;I have one FASTA file with sequences and headers, and one OTUs table.&#xA;What I want to do is replace the current headers in the FASTA file with just the OTU IDs from the .txt file. The files look like something like this:&lt;/p&gt;&#xA;&lt;p&gt;FASTA file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;9d4544fa-322a-4fac-bcf5-f8ca410c7e8a runid=b48dcb2029ef589b8cdb55fe529d93b3baadbc7c sampleid=28012021COVID3 read=57150 ch=310 start_time=2021-01-28T18:23:35Z&#xA;TCAGTTACGTATTGCTGGTGCTGAAGAAAGTTGTCGGTGTCTTTGTGTTAACCTTTCTGTTGGTGCTGATATTGCAGTCGCCCCCACCTCCTTGGATCCGCCAGGTTAAACACCCAAGCAGACGCCGAAGATAGAGCGACAGGCAAGTAGGTTAACACAAAGACACCGACAACTTTTCTTCCAGCACCAGCAGCCCGTAACT&#xA;&amp;gt;a8d321b9-97f1-42aa-a17a-fa1c4609e60e runid=b48dcb2029ef589b8cdb55fe529d93b3baadbc7c sampleid=28012021COVID3 read=46034 ch=83 start_time=2021-01-28T18:24:16Z&#xA;CGTTCGGTGCGTATTGCTGGTGCTGAAGAAAGTTGTCCGGTGTCTTTGTGTTAACCTTTCTGTTGGTGCTGATATTGCGGCGTCTGGCAGGTGTTAACCTGGCCTCGAGAGAGTTTGATCCTGGCTGGGATGAACGCTGACGTGCCTAATACATGCAAGTAG&#xA;&amp;gt;6d97cca8-6795-4c59-b4d1-05880735b957 runid=b48dcb2029ef589b8cdb55fe529d93b3baadbc7c sampleid=28012021COVID3 read=46363 ch=227 start_time=2021-01-28T18:24:08Z&#xA;GTTCAGTGCATATTGCTGTGGTGCTGCGAAAGTTTGTGGTGTCTTTGTGTTAACGCTTTCTGTTGGTGCTGATATTGCGGCGGCGTCTGCGCGGTGTTTAACCTGGCGGATCAAGGAGGTGTTCAGCCGCACCTTCCGTTTTGACTACGCCATTACGACTGCCCAATCTGTTTTACCCTAGGCCCGATCCTTGCGGTCACGGACTTCAGGCACCCCCGGCTTTCATGGCTTGACAGGCGGTGTGTACAAGGCCCGGGAACATTCACCCGCGCCATGGCTGATGCGCGATTACTAGCGAATCCCAGCTTCACGGGTCGGGTTGCA&#xA;&amp;gt;772e0bef-fad4-4767-b0e6-d88e6f16b2ed runid=b48dcb2029ef589b8cdb55fe529d93b3baadbc7c sampleid=28012021COVID3 read=49763 ch=349 start_time=2021-01-28T18:24:10Z&#xA;CGTTCAGTTACGTATTGCTGGTGCTGAAGAAAGTTGTCGGTGTCTTTAGGTTAACCTTTCTGTTGGTGCTGATATTGCGGCGTCTGCTTGGGTGTTTAACCTGGCCTCGAGAAGTTTGATCCTGGCTCAGGATGAACGCTAGCTTCTGAGCTTAACATGCAGAGTCCAGGGCAGCATGGAAGAAACTGCTTCTTCTGATGGCGACAACGCACGGGTGCATGCGCGTATCAAACCTGCCTCATA&#xA;&amp;gt;2b673c48-69c9-48a4-995b-4ad630e23502 runid=b48dcb2029ef589b8cdb55fe529d93b3baadbc7c sampleid=28012021COVID3 read=71966 ch=124 start_time=2021-01-28T18:24:11Z&#xA;TTCGTTCAGTTACGTATTGCTGGTGCTGAAAGTTGTCGGTGTCTTTGTGTTAACCTTTCTGTTGGTGCTGATATTGCGGCTCTGCTTGGGTGTTTAACCTGGCGGATCAAGGAGTGTTAGCCGCACCTTCCGGTACCAACTACCTTGTTACGACTTAGCCCC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;OTUs table:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;OTU ID  IA1 IA2 IA3 IA4 IA5 IS6&#xA;otu500  7   10  3   0   5   0&#xA;otu502  7   8   4   0   11  1&#xA;otu504  4   5   6   2   4   2&#xA;otu505  4   4   6   2   8   3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and my desired output is that the .fasta file would look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;otu500&#xA;tggggaatcttagacaatgggcgcaagcctgatctagccatgccg&#xA;&amp;gt;otu502&#xA;tgaggaatattggtcaatggaggcaactctgaaccagccatgccttggtcaatggaggc&#xA;&amp;gt;otu504&#xA;gcgaacaggattagataccctggtagtccacgccgtaaacttggtcaatggaggc&#xA;&amp;gt;otu505&#xA;ttggtcaatggaggcttggtcaatggaggctaccctggttaccctggt&#xA;&amp;gt;otu506&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9854" LastEditorUserId="650" LastEditDate="2021-05-17T18:44:51.143" LastActivityDate="2021-05-17T18:44:51.143" Title="How can I extract sequences from FASTA file using OTUs ID?" Tags="|fasta|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15927" PostTypeId="1" AcceptedAnswerId="15932" CreationDate="2021-05-17T09:06:14.920" Score="2" ViewCount="486" Body="&lt;p&gt;I recently did some sequencing on a MiniSeq and unexpectedly the &lt;code&gt;Undetermined_S0_L001_R1_001.fastq.gz&lt;/code&gt; file contained quite a lot of data (about 40% of total). I would like to know which indices are present in this file and ideally get a frequency of all index sequences in this file. How can I do this?&lt;/p&gt;&#xA;&lt;p&gt;This is the output of &lt;code&gt;head&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@MN00148:128:000H2YCW2:1:11102:12116:1065 1:N:0:0&#xA;GNAGAGGATTCAGGGAGCCAGCGCGCATATATCAG&#xA;+&#xA;F#FFFFFFFFFFFFFFFFFFA=/A///////////&#xA;@MN00148:128:000H2YCW2:1:11102:11354:1065 1:N:0:0&#xA;GNACCCGTGCGGCTGGAGAGACCACTAGAAAGAAT&#xA;+&#xA;A#AFFFFFFFFFFFFFFFFF///////FF=////6&#xA;@MN00148:128:000H2YCW2:1:11102:4103:1066 1:N:0:0&#xA;ANGTTGGTGGTACATCAGGGACGCGGAGATCTCAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10386" LastEditorUserId="10386" LastEditDate="2021-05-17T10:01:31.663" LastActivityDate="2021-05-18T14:02:17.940" Title="Get frequency of index sequences of fastq file" Tags="|ngs|sequence-analysis|fastq|demultiplexing|indexing|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15929" PostTypeId="2" ParentId="15927" CreationDate="2021-05-17T17:11:07.090" Score="1" Body="&lt;p&gt;I don't know that you can get them from that file.  I've noticed that when the Miseq is left to run its version of fastq generation software, there are no indices in the undetermined file.  I usually run bcl2fastq myself if I want that information.&lt;/p&gt;&#xA;&lt;p&gt;They recently came out with a new version of software for the MiSeq, so things might be different now.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2021-05-17T17:11:07.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15930" PostTypeId="1" CreationDate="2021-05-17T19:46:01.653" Score="4" ViewCount="535" Body="&lt;p&gt;Just wondering whether anyone can point me to a research paper which describes how the scoring values are generated by the Bruker software for species identification of bacteria in MALDI-TOF MS. For example, there are countless papers describing the Bruker output score as green (species-level confidence), yellow (genus-level), or red (neither species- nor genus-level confidence). However, none of the papers that I see are discussing how these numbers are actually calculated within the Bruker system. Even the &lt;a href=&quot;https://www.bruker.com/en/products-and-solutions/microbiology-and-diagnostics/microbial-identification/_jcr_content/root/contentpar/linklist.download-asset.pdf/primarylist/item0/1884563-brochure-maldi-biotyper-ruo-gp-bruker-md-12-2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bruker brochure&lt;/a&gt; glosses over the point: &amp;quot;After the acquisition of the spectral data has been completed, a report is generated. The result for each sample is clearly listed under ‘Organism (best match)’ accompanied by the resulting score and an appropriate ‘traffic light’ color scheme.&amp;quot; Can anyone point me in the direction of a paper or at least some online information such as wikipedia describing how Bruker came up with these numbers? It's not just an arbitrary number, right?&lt;/p&gt;&#xA;&lt;p&gt;Another generic description of the scores is in &lt;a href=&quot;https://idmicro.files.wordpress.com/2020/06/maldi-tof-elias-16jun2020.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this presentation&lt;/a&gt;: &amp;quot;Higher the log (score), higher the similarity between mass spectrum of isolate &amp;amp; the database entry in the reference library.&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="12245" LastEditorUserId="57" LastEditDate="2021-05-19T11:39:55.450" LastActivityDate="2021-05-25T17:48:51.877" Title="Bruker MALDI-TOF bacteria species identification scoring algorithm" Tags="|bacteria|mass-spectrometry|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15931" PostTypeId="2" ParentId="935" CreationDate="2021-05-18T08:20:46.717" Score="1" Body="&lt;p&gt;Use assembly-stats!&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ assembly-stats barcode13_filtered.fastq &#xA;stats for barcode13_filtered.fastq&#xA;sum = 2080834976, n = 656192, ave = 3171.08, largest = 15321&#xA;N50 = 3598, n = 225863&#xA;N60 = 3263, n = 286569&#xA;N70 = 2920, n = 353904&#xA;N80 = 2548, n = 429982&#xA;N90 = 2081, n = 519671&#xA;N100 = 500, n = 656192&#xA;N_count = 0&#xA;Gaps = 0&#xA;&#xA;real    0m7.418s&#xA;user    0m5.976s&#xA;sys     0m1.275s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Works on any fasta/fastq files, even those that aren't assemblies. And it's pretty fast!&lt;/p&gt;&#xA;&lt;p&gt;Download from here: &lt;a href=&quot;https://anaconda.org/bioconda/assembly-stats&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://anaconda.org/bioconda/assembly-stats&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7373" LastEditorUserId="7373" LastEditDate="2021-05-18T10:10:12.057" LastActivityDate="2021-05-18T10:10:12.057" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15932" PostTypeId="2" ParentId="15927" CreationDate="2021-05-18T14:02:17.940" Score="1" Body="&lt;p&gt;I actually went back to the output folder on the MiniSeq and found this file: &lt;code&gt;DemultiplexSummaryF1L1&lt;/code&gt;, which contains demultiplexing information of the whole run. Half-way through this file I found what I was looking for:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;### Most Popular Index Sequences                        &#xA;&#xA;### Columns: Sequence ReverseComplement HitCount                        &#xA;&#xA;Index                       &#xA;TGACCA  TGGTCA  2455964             &#xA;CGATGT  ACATCG  2320579             &#xA;GGCCCC  GGGGCC  2023652             &#xA;CGCGGG  CCCGCG  1983605             &#xA;CCGTCC  GGACGG  1641274         &#xA;CAGATC  GATCTG  1514252             &#xA;CCGGCC  GGCCGG  1319383             &#xA;CCGCGC  GCGCGG  1109940             &#xA;ACAGTG  CACTGT  891344      &#xA;CCCGGG  CCCGGG  706083              &#xA;TGCCCA  TGGGCA  168294              &#xA;CAGCTC  GAGCTG  164091  &#xA;.... [data continues]       &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10386" LastActivityDate="2021-05-18T14:02:17.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15934" PostTypeId="1" CreationDate="2021-05-19T00:56:44.893" Score="1" ViewCount="91" Body="&lt;p&gt;I would like to average (and maybe also additional operations such as subtraction) 2 or more DataTracks and plot the result using the Gviz (&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/Gviz.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/html/Gviz.html&lt;/a&gt;) package.&lt;/p&gt;&#xA;&lt;p&gt;Say I have:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr=&amp;quot;chr2&amp;quot;&#xA;gen=&amp;quot;mm10&amp;quot;&#xA;from=1000&#xA;to=5000&#xA;&#xA;dTrep1 &amp;lt;- DataTrack(range = path/to/rep1.bw, genome = gen, &#xA;             name = &amp;quot;bigWig&amp;quot;,chromosome = chr)&#xA;dTrep2 &amp;lt;- DataTrack(range = path/to/rep2.bw, genome = gen, &#xA;             name = &amp;quot;bigWig&amp;quot;,chromosome = chr)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Instead of:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plotTracks(list(dTrep1,dTrep2),type=&amp;quot;polygon&amp;quot;,from=from,to=to)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plotTracks(mean(dTrep1,dTrep2),type=&amp;quot;polygon&amp;quot;,from=from,to=to)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plotTracks(dTrep1-dTrep2,type=&amp;quot;polygon&amp;quot;,from=from,to=to)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;See posted also at: &lt;a href=&quot;https://support.bioconductor.org/p/9137211/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.bioconductor.org/p/9137211/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="12259" LastEditorUserId="12259" LastEditDate="2021-05-19T17:13:30.880" LastActivityDate="2021-05-19T17:13:30.880" Title="Combining dataTracks in Gviz by averaging or subtraction" Tags="|r|gviz|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15935" PostTypeId="2" ParentId="15923" CreationDate="2021-05-19T07:38:17.500" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @maximilian-press, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;At a guess, the GEMMA linear mixed model is controlling for population structure and other SNPs, which your R example is not (it only fits 1 SNP?), but it's hard to be sure with this information. It would be helpful to see data samples. There is probably more context and usage information available in those resources beyond just the equations. It is especially difficult to evaluate with no terms defined as in your last example (defining the terms in the earlier ones is a good start).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-19T07:38:17.500" CommentCount="0" CommunityOwnedDate="2021-05-19T07:38:17.500" ContentLicense="CC BY-SA 4.0" />
  <row Id="15936" PostTypeId="1" CreationDate="2021-05-19T20:25:39.330" Score="1" ViewCount="122" Body="&lt;p&gt;I have data from different countries from two groups of samples (control and test). And, calculated three different indices alpha, beta and gamma. I have a large dataframe containing all the data. Here is a reproducible example of my dataframe (df):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df &amp;lt;- structure(list(&#xA;    SampleID = c(&amp;quot;SRR490_profile&amp;quot;, &amp;quot;SRR467268_profile&amp;quot;,&#xA;                 &amp;quot;SRR467265_profile&amp;quot;, &amp;quot;SRR467264_profile&amp;quot;,&#xA;                 &amp;quot;SRR467263_profile&amp;quot;, &amp;quot;SRR467258_profile&amp;quot;,&#xA;                 &amp;quot;SRR467250_profile&amp;quot;, &amp;quot;SRR467244_profile&amp;quot;,&#xA;                 &amp;quot;SRR467242_profile&amp;quot;, &amp;quot;SRR467227_profile&amp;quot;,&#xA;                 &amp;quot;ERR710432_profile&amp;quot;, &amp;quot;ERR710430_profile&amp;quot;,&#xA;                 &amp;quot;ERR710429_profile&amp;quot;, &amp;quot;SRR232646_profile&amp;quot;,&#xA;                 &amp;quot;SRR232622_profile&amp;quot;, &amp;quot;SRR232619_profile&amp;quot;,&#xA;                 &amp;quot;SRR232617_profile&amp;quot;, &amp;quot;SRR232616_profile&amp;quot;,&#xA;                 &amp;quot;SRR232614_profile&amp;quot;, &amp;quot;SRR232612_profile&amp;quot;,&#xA;                 &amp;quot;HJ4386_profile&amp;quot;, &amp;quot;HJ4382_profile&amp;quot;,&#xA;                 &amp;quot;HJ4380_profile&amp;quot;, &amp;quot;HJ4379_profile&amp;quot;,&#xA;                 &amp;quot;HJ4377_profile&amp;quot;, &amp;quot;HJ4375_profile&amp;quot;,&#xA;                 &amp;quot;HJ4374_profile&amp;quot;, &amp;quot;HJ4373_profile&amp;quot;,&#xA;                 &amp;quot;HJ4372_profile&amp;quot;, &amp;quot;HJ4370_profile&amp;quot;,&#xA;                 &amp;quot;KLM3246730_profile&amp;quot;, &amp;quot;KLM3246729_profile&amp;quot;,&#xA;                 &amp;quot;KLM3246725_profile&amp;quot;, &amp;quot;KLM3246722_profile&amp;quot;,&#xA;                 &amp;quot;KLM3246721_profile&amp;quot;, &amp;quot;KLM3246714_profile&amp;quot;,&#xA;                 &amp;quot;KLM3246713_profile&amp;quot;, &amp;quot;KLM3246712_profile&amp;quot;,&#xA;                 &amp;quot;KLM3246711_profile&amp;quot;, &amp;quot;KLM3246705_profile&amp;quot;),&#xA;    type = c(&amp;quot;test&amp;quot;, &amp;quot;control&amp;quot;, &amp;quot;control&amp;quot;, &amp;quot;control&amp;quot;,&#xA;             &amp;quot;control&amp;quot;, &amp;quot;test&amp;quot;, &amp;quot;test&amp;quot;, &amp;quot;test&amp;quot;, &amp;quot;control&amp;quot;,&#xA;             &amp;quot;control&amp;quot;, &amp;quot;control&amp;quot;, &amp;quot;test&amp;quot;, &amp;quot;control&amp;quot;,&#xA;             &amp;quot;control&amp;quot;, &amp;quot;control&amp;quot;, &amp;quot;control&amp;quot;, &amp;quot;control&amp;quot;,&#xA;             &amp;quot;test&amp;quot;, &amp;quot;control&amp;quot;, &amp;quot;test&amp;quot;, &amp;quot;control&amp;quot;, &amp;quot;control&amp;quot;,&#xA;             &amp;quot;control&amp;quot;, &amp;quot;control&amp;quot;, &amp;quot;control&amp;quot;, &amp;quot;test&amp;quot;,&#xA;             &amp;quot;control&amp;quot;, &amp;quot;test&amp;quot;, &amp;quot;test&amp;quot;, &amp;quot;control&amp;quot;, &amp;quot;control&amp;quot;,&#xA;             &amp;quot;test&amp;quot;, &amp;quot;test&amp;quot;, &amp;quot;test&amp;quot;, &amp;quot;control&amp;quot;, &amp;quot;test&amp;quot;,&#xA;             &amp;quot;control&amp;quot;, &amp;quot;test&amp;quot;, &amp;quot;test&amp;quot;, &amp;quot;test&amp;quot;),&#xA;    variable = c(&amp;quot;gamma&amp;quot;, &amp;quot;alpha&amp;quot;, &amp;quot;beta&amp;quot;, &amp;quot;gamma&amp;quot;, &amp;quot;alpha&amp;quot;, &amp;quot;beta&amp;quot;,&#xA;                 &amp;quot;gamma&amp;quot;, &amp;quot;alpha&amp;quot;, &amp;quot;beta&amp;quot;, &amp;quot;gamma&amp;quot;, &amp;quot;alpha&amp;quot;, &amp;quot;alpha&amp;quot;,&#xA;                 &amp;quot;beta&amp;quot;, &amp;quot;gamma&amp;quot;, &amp;quot;alpha&amp;quot;, &amp;quot;beta&amp;quot;, &amp;quot;gamma&amp;quot;, &amp;quot;alpha&amp;quot;,&#xA;                 &amp;quot;beta&amp;quot;, &amp;quot;gamma&amp;quot;, &amp;quot;alpha&amp;quot;, &amp;quot;alpha&amp;quot;, &amp;quot;alpha&amp;quot;, &amp;quot;beta&amp;quot;,&#xA;                 &amp;quot;beta&amp;quot;, &amp;quot;gamma&amp;quot;, &amp;quot;alpha&amp;quot;, &amp;quot;beta&amp;quot;, &amp;quot;gamma&amp;quot;, &amp;quot;alpha&amp;quot;,&#xA;                 &amp;quot;beta&amp;quot;, &amp;quot;gamma&amp;quot;, &amp;quot;alpha&amp;quot;, &amp;quot;beta&amp;quot;, &amp;quot;gamma&amp;quot;, &amp;quot;alpha&amp;quot;,&#xA;                 &amp;quot;beta&amp;quot;, &amp;quot;gamma&amp;quot;, &amp;quot;alpha&amp;quot;, &amp;quot;beta&amp;quot;),&#xA;    value = c(95, 90, 109, 112, 128, 109, 90, 87, 90, 99, 129, 101, 113,&#xA;              100, 111, 102, 99, 105, 85, 94, 133, 117, 116, 83, 113, 126,&#xA;              90, 128, 107, 121, 88, 109, 82, 42, 67, 100, 88, 101, 127,&#xA;              115),&#xA;    StudyID = c(&amp;quot;swiss&amp;quot;, &amp;quot;swiss&amp;quot;, &amp;quot;swiss&amp;quot;, &amp;quot;swiss&amp;quot;, &amp;quot;swiss&amp;quot;, &amp;quot;swiss&amp;quot;,&#xA;                &amp;quot;swiss&amp;quot;, &amp;quot;swiss&amp;quot;, &amp;quot;swiss&amp;quot;, &amp;quot;swiss&amp;quot;, &amp;quot;australian&amp;quot;,&#xA;                &amp;quot;australian&amp;quot;, &amp;quot;australian&amp;quot;, &amp;quot;australian&amp;quot;, &amp;quot;australian&amp;quot;,&#xA;                &amp;quot;australian&amp;quot;, &amp;quot;australian&amp;quot;, &amp;quot;australian&amp;quot;, &amp;quot;australian&amp;quot;,&#xA;                &amp;quot;australian&amp;quot;, &amp;quot;thai&amp;quot;, &amp;quot;thai&amp;quot;, &amp;quot;thai&amp;quot;, &amp;quot;thai&amp;quot;, &amp;quot;thai&amp;quot;,&#xA;                &amp;quot;thai&amp;quot;, &amp;quot;thai&amp;quot;, &amp;quot;thai&amp;quot;, &amp;quot;thai&amp;quot;, &amp;quot;thai&amp;quot;, &amp;quot;english&amp;quot;,&#xA;                &amp;quot;english&amp;quot;, &amp;quot;english&amp;quot;, &amp;quot;english&amp;quot;, &amp;quot;english&amp;quot;, &amp;quot;english&amp;quot;,&#xA;                &amp;quot;english&amp;quot;, &amp;quot;english&amp;quot;, &amp;quot;english&amp;quot;, &amp;quot;english&amp;quot;)),&#xA;    row.names = c(NA, -40L),&#xA;    class = c(&amp;quot;tbl_df&amp;quot;, &amp;quot;tbl&amp;quot;, &amp;quot;data.frame&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, I want an image for each index containing barplots for two groups (control and test) and Y-axis showing the StudyID names (i.e., country names) like the following (this image shows plots for two indices alpha and beta):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/AeKTY.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AeKTY.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;Is this possible from my data? Can anyone give me a solution?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="12271" LastEditorUserId="73" LastEditDate="2021-05-20T07:26:08.907" LastActivityDate="2021-11-17T06:08:25.850" Title="How can I create a combined barplot like this image?" Tags="|r|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15937" PostTypeId="1" AcceptedAnswerId="15950" CreationDate="2021-05-19T23:09:00.297" Score="1" ViewCount="32" Body="&lt;p&gt;&lt;a href=&quot;https://vows.sbolstandard.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Synthetic Biology Open Language's Visual Ontology Webservice&lt;/a&gt; docs have example URLs but they all return a github pages 404.&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&#xA;The SBOL-VOWS has site has been fixed but the &lt;a href=&quot;https://sbolstandard.org/sbol-visual-ontology/&quot; rel=&quot;nofollow noreferrer&quot;&gt;section on the SBOL website&lt;/a&gt; still has links with a 404.&lt;/p&gt;&#xA;" OwnerUserId="12216" LastEditorUserId="12216" LastEditDate="2021-05-25T17:17:36.730" LastActivityDate="2021-05-25T17:17:36.730" Title="Is SBOL-VOWS operational?" Tags="|visualization|sbol|ontology|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15938" PostTypeId="1" CreationDate="2021-05-20T07:37:16.693" Score="0" ViewCount="91" Body="&lt;p&gt;I am trying to run Illumina CANVAS cnv caller for Somatic-WGS. There is an option &amp;quot;--ploidy-vcf&amp;quot; which is mandatory to supply, but don't know what exactly that mean. I had supplied the CNV.vcf file from the germline WGS (matched normal run on Canvas smallPedigree), but it has given an error that says &amp;quot;Reference copy number &amp;gt; 2 is not supported&amp;quot; at the last stage of run.&lt;/p&gt;&#xA;&lt;p&gt;I would like to know your experience when you used CANVAS on somatic WGS samples. What ploidy-file is appropriate here? Should it be only chrX and Y in it? Your suggestions please.&lt;/p&gt;&#xA;" OwnerUserId="12038" LastActivityDate="2021-08-23T16:57:24.333" Title="polidy VCF file for Canvas somatic-wgs cnv calling" Tags="|wgs|illumina|cnv|somatic|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15940" PostTypeId="2" ParentId="15936" CreationDate="2021-05-20T10:19:30.693" Score="2" Body="&lt;p&gt;If you're not wedded to a dot and whiskers plot as the one show, I'd probably plot this dataset as a boxplot with ggplot:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(df,aes(x=value,y=StudyID,colour=type)) +&#xA;geom_boxplot() +&#xA;facet_wrap(~variable)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8919" LastEditorUserId="3694" LastEditDate="2021-05-20T12:46:07.467" LastActivityDate="2021-05-20T12:46:07.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15941" PostTypeId="1" AcceptedAnswerId="15943" CreationDate="2021-05-20T11:39:31.673" Score="2" ViewCount="1045" Body="&lt;p&gt;i am trying to calculate mean Phred scores for my sequencing data, but i feel not very comfortable about it. There are actually two ways of calculating. (I just use an existing sample)&#xA;giving: 3 reads with a mean phred score of 20, 10 and 3 (meaning Error rates of 1%, 10% and 50%, respectively)&lt;/p&gt;&#xA;&lt;p&gt;Way 1 to calculate: Just take the arithmetic mean of the q scores:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;(20+10+3)/3 = 11&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Way 2:  Take the arithmetic mean of the error rates:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;(0,5+0,1+0,01)/3 ~= 0.2&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;And then calculate the q score by&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;-10*log10(0.2) = 6.99&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;(&lt;a href=&quot;https://gigabaseorgigabyte.wordpress.com/2017/06/26/averaging-basecall-quality-scores-the-right-way/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gigabaseorgigabyte.wordpress.com/2017/06/26/averaging-basecall-quality-scores-the-right-way/&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;The Author of this Blog claims, that Way 2 is the correct version, but i still do not get why.&lt;/p&gt;&#xA;&lt;p&gt;Is somebody able to answer this question?&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot&lt;/p&gt;&#xA;" OwnerUserId="10677" LastEditorUserId="3360" LastEditDate="2021-05-21T06:27:14.947" LastActivityDate="2021-05-21T06:27:14.947" Title="What is the right way of calculating a Phred score by hand?" Tags="|ngs|reads|quality-control|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15942" PostTypeId="1" AcceptedAnswerId="15954" CreationDate="2021-05-20T11:45:31.540" Score="3" ViewCount="884" Body="&lt;p&gt;I want to introduce point mutations to a protein PDB file and automatically predict the 3D coordinates of a new amino acid residue in the mutated protein.&lt;/p&gt;&#xA;&lt;p&gt;I used interactive Schrodinger's Maestro tool to mutate residues and minimize them to avoid clashes, however this time i want to automate the task and use some open source tools.&lt;/p&gt;&#xA;&lt;p&gt;Is there any free software for this task?&lt;/p&gt;&#xA;" OwnerUserId="7150" LastEditorUserId="73" LastEditDate="2021-05-22T04:09:33.187" LastActivityDate="2021-05-22T04:09:33.187" Title="How to introduce point mutations in a protein structure?" Tags="|protein-structure|point-mutation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15943" PostTypeId="2" ParentId="15941" CreationDate="2021-05-20T12:40:44.160" Score="5" Body="&lt;p&gt;Not sure if my explanation is any good but let's try...&lt;/p&gt;&#xA;&lt;p&gt;First, let's convince ourselves that converting phred to probability is the right thing to do (&lt;em&gt;i.e.&lt;/em&gt; your Way 2).&lt;/p&gt;&#xA;&lt;p&gt;Imagine a read of length 10000 bp with half of the bases with Q=10 (or P=0.1) and the other half with Q=20 (or P=0.01). The number of wrong calls you expect is therefore 550:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;n = 10000&#xA;(n * 0.01 * 0.5) + (n * 0.1 * 0.5) # -&amp;gt; 550 wrong calls&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So the overall error rate is &lt;code&gt;550/10000 = 0.055&lt;/code&gt; which corresponds to Q=12.6. If you follow your Way 2 you do get this answer:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;((n * 0.5 * 0.01) + (n * 0.5 * 0.1))/n = 0.055&#xA;-10*log10(0.055) = 12.6 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you follow your Way 1 you get Q=15, which means an overoptimistic error rate of ~0.03 or ~316 wrong calls. Why is that?&lt;/p&gt;&#xA;&lt;p&gt;Because phred is on a non-linear, logarithmic, scale: a unit change  towards the high values is less important than a unit change towards the lower end. For example, across 10000 bases a change from Q=30 to Q=31 means a change from 10 to ~8 wrong calls - a difference of 2 only. But a change from  Q=10 to Q=11 means 1000 to ~800 wrong calls: the difference is a substantial ~200 calls.&lt;/p&gt;&#xA;&lt;p&gt;Having said this, the straight average of phred scores is not &amp;quot;wrong&amp;quot;. It measures the overall phred score but it does not reflect the expected number of wrong calls, which is probably what you want in this case.&lt;/p&gt;&#xA;" OwnerUserId="339" LastActivityDate="2021-05-20T12:40:44.160" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15944" PostTypeId="1" CreationDate="2021-05-20T14:31:49.823" Score="1" ViewCount="471" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cdZ52.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cdZ52.png&quot; alt=&quot;Integrated Genome Viewer image&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;IGV (Integrated Genome Viewer) is a popular open-source tool for viewing alignment files. In my BAM file in IGV, some deletions have black lines running through them, and others don't. What causes this? Short black lines also appear adjacent to some mutations and insertions.&lt;/p&gt;&#xA;&lt;p&gt;I generated the CIGAR for the SAM file myself, and am wondering if I made an error. I didn't find anything in IGV's documentation about this.&lt;/p&gt;&#xA;" OwnerUserId="6332" LastActivityDate="2021-06-23T14:07:15.677" Title="What does presence/absence of black lines mean in IGV?" Tags="|sam|igv|cigar|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="15945" PostTypeId="2" ParentId="12824" CreationDate="2021-05-20T15:57:44.453" Score="9" Body="&lt;p&gt;For the most modern CHM13 assembly (v2.0; Complete T2T reconstruction of a human genome; &lt;a href=&quot;https://github.com/marbl/CHM13&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/marbl/CHM13&lt;/a&gt;), the best known compression without any use of a reference is 567 MB (1.45 bits/bp; 1.80 bits/distinct canonical 31-mer).&lt;/p&gt;&#xA;&lt;p&gt;Upper estimates of genome compressibility can be made using standard general or specialized compressors (see, e.g., &lt;a href=&quot;https://academic.oup.com/gigascience/article/9/7/giaa072/5867695&quot; rel=&quot;nofollow noreferrer&quot;&gt;this comparison&lt;/a&gt;). In my experience, &lt;code&gt;xz&lt;/code&gt; can achieve great compression ratios on genomic sequences, especially when used with the maximum compression level (&lt;code&gt;-9&lt;/code&gt;) and a single thread (&lt;code&gt;-T1&lt;/code&gt;). The compression performance can be further boosted by using a large dictionary (&lt;code&gt;-e --lzma2=preset=9,dict=512MiB&lt;/code&gt;). Importantly, the original FASTA file needs to be cleaned – linebreaks hamper compression as they act as noise; the reformatting can be done by &lt;code&gt;seqtk seq -U&lt;/code&gt; (which also converts all nucleotides to the upper case). It the case of the CHM13v2 assembly, &lt;code&gt;xz&lt;/code&gt; compresses the human genome to 634 MB.&lt;/p&gt;&#xA;&lt;p&gt;However, even better performance can be achieved with specialized compressors, especially if they have particular support for reverse complements and mutational models. Among the best ones is GeCo3, which has been shown to compress CHM13v2, with a particular combination of parameters, to 567 MB &lt;a href=&quot;https://github.com/pratas/HumanGenome&quot; rel=&quot;nofollow noreferrer&quot;&gt;3&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Lower estimates are impossible in general (we can't reject the existence of a program that would create the whole sequence from small amount of starting conditions), but can be done for particular compression techniques, e.g., particular variants of dictionary compressors. However, this leads to intractable optimization problems (see e.g. the problem of minimal string attractors).&lt;/p&gt;&#xA;&lt;p&gt;A useful auxiliary measure for estimating the compressibility of single genomes is the number of distinct k-mers for a &amp;quot;reasonably&amp;quot; selected k (e.g., k=31). The lower bound on the compressibility of k-mer sets can then be roughly approximated by &lt;a href=&quot;https://arxiv.org/abs/1401.5383&quot; rel=&quot;nofollow noreferrer&quot;&gt;2 bits per distinct k-mer&lt;/a&gt;, however, this is limited by the embedded structural assumptions, e.g., that single genomes are in principle random strings with occasional (possibly distant) repeats. For instance, the CHM13v2 human genome assembly contains 2,512,390,070 distinct canonical 31-mers, which would then correspond to 628 MB assuming 2bits/kmers, while the best available specialized compressors can compress it down to 567 MB, i.e., by approx 10% better.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Updates&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;May 27, 2022 – updated the upper bound based on the latest results with CHM13v2 (xz &amp;amp; GeCo3), changed the lower bound wording to an auxiliary estimate as the assumptions have shown not to hold&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="425" LastEditorUserId="425" LastEditDate="2022-05-27T15:43:05.637" LastActivityDate="2022-05-27T15:43:05.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15946" PostTypeId="2" ParentId="9104" CreationDate="2021-05-21T08:57:08.707" Score="1" Body="&lt;p&gt;I'm late to the party but simply run &lt;code&gt;cellranger mkfastq&lt;/code&gt; twice, with different arguments for the mask and with &lt;code&gt;--filter-dual-index&lt;/code&gt; for the double-indexed samples. In a second step you then have to disentangle the dual A+B reads from single-index A reads based on the readID's that are found in both of the demultiplexings. This is cumbersome and time+diskspace consuming, but it can be done. I would use some Unix tools for that but there are probably smarter ways.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: I mistakenly said &lt;code&gt;bcl2fastq&lt;/code&gt;, this must be &lt;code&gt;cellranger mkfastq&lt;/code&gt;. (thanks AmadeusDrZaius)&lt;/p&gt;&#xA;" OwnerUserId="9846" LastEditorUserId="9846" LastEditDate="2021-05-26T19:58:00.250" LastActivityDate="2021-05-26T19:58:00.250" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15947" PostTypeId="1" CreationDate="2021-05-21T09:35:12.147" Score="2" ViewCount="712" Body="&lt;p&gt;Using DSL2.&lt;/p&gt;&#xA;&lt;p&gt;I have processes A, B, C which I want to run in sequence - B waits for A to finish, C waits for B to finish.&lt;/p&gt;&#xA;&lt;p&gt;They are not communicating via channels so I can't do A.out.collect().&lt;/p&gt;&#xA;&lt;p&gt;I have tried making dummy channels but this is problematic and a hack. The onComplete subscription doesn't seem to let me trigger another process.&lt;/p&gt;&#xA;&lt;p&gt;What is the correct way to run a sequence of processes which are not communicating or triggered by channels?&lt;/p&gt;&#xA;&lt;p&gt;P.S. I am aware that this is not really the intended use case of Nextflow (which is oriented around channel communication/triggers... but... I just need to get this done.)&lt;/p&gt;&#xA;" OwnerUserId="11959" LastActivityDate="2022-01-26T16:24:37.903" Title="How to execute several processes sequentially, without channels?" Tags="|nextflow|workflow-management|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15948" PostTypeId="1" AcceptedAnswerId="15951" CreationDate="2021-05-21T11:23:21.670" Score="-4" ViewCount="185" Body="&lt;p&gt;I have downloaded &lt;code&gt;log2(TPM/10+1)&lt;/code&gt; values of &lt;code&gt;11,548 genes&lt;/code&gt; and &lt;code&gt;9609 cells&lt;/code&gt; from &lt;code&gt;GSE146026&lt;/code&gt; (10x) in &lt;code&gt;tsv format&lt;/code&gt; as the raw data is not available&lt;/p&gt;&#xA;&lt;p&gt;I see patient IDs, cell barcodes, genes, even assigned clusters are here&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dput(head(a[1:5,1:5]))&#xA;structure(list(Cell_ID = c(&amp;quot;10x_barcode&amp;quot;, &amp;quot;patient&amp;quot;, &amp;quot;time&amp;quot;, &#xA;&amp;quot;sample_ID&amp;quot;, &amp;quot;clst&amp;quot;), X10x_1 = c(&amp;quot;10x_3288_t1_AAACATACCTTCCG-1&amp;quot;, &#xA;&amp;quot;5&amp;quot;, &amp;quot;1&amp;quot;, &amp;quot;3288.1&amp;quot;, &amp;quot;1&amp;quot;), X10x_2 = c(&amp;quot;10x_3288_t1_AAACATACTCCTAT-1&amp;quot;, &#xA;&amp;quot;5&amp;quot;, &amp;quot;1&amp;quot;, &amp;quot;3288.1&amp;quot;, &amp;quot;1&amp;quot;), X10x_3 = c(&amp;quot;10x_3288_t1_AAACATTGAACTGC-1&amp;quot;, &#xA;&amp;quot;5&amp;quot;, &amp;quot;1&amp;quot;, &amp;quot;3288.1&amp;quot;, &amp;quot;1&amp;quot;), X10x_4 = c(&amp;quot;10x_3288_t1_AAACATTGCTGACA-1&amp;quot;, &#xA;&amp;quot;5&amp;quot;, &amp;quot;1&amp;quot;, &amp;quot;3288.1&amp;quot;, &amp;quot;2&amp;quot;)), row.names = c(NA, 5L), class = &amp;quot;data.frame&amp;quot;)&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to make a Seurat object of that but really I don't know how to do that&lt;/p&gt;&#xA;&lt;p&gt;I have tried this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    CreateSeuratObject(getwd(),gene.column = 2,cell.column = 1,unique.features = TRUE,strip.suffix = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But get the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in CreateAssayObject(counts = counts, min.cells = min.cells, min.features = min.features) : &#xA;  No cell names (colnames) names present in the input matrix&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Anybody have ever dealt with such a case to help me please?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="73" LastEditDate="2021-05-23T12:19:51.840" LastActivityDate="2021-05-23T12:19:51.840" Title="Making Seurat object from a processed file" Tags="|r|seurat|10x-genomics|" AnswerCount="1" CommentCount="0" ClosedDate="2021-05-27T06:41:23.493" ContentLicense="CC BY-SA 4.0" />
  <row Id="15949" PostTypeId="1" CreationDate="2021-05-21T11:59:16.397" Score="0" ViewCount="38" Body="&lt;p&gt;I'm trying to evaluate a method that I'm not familiar with. They used SNPRelate. I'm wondering if this is the best / only choice... what other tools would you recommend?&lt;/p&gt;&#xA;&lt;p&gt;Many thanks,&lt;/p&gt;&#xA;" OwnerUserId="8875" LastActivityDate="2021-05-21T11:59:16.397" Title="Is SNPRelate the goto package for PCA of SNP data in R?" Tags="|r|snp|plink|phylogenetics|pca|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15950" PostTypeId="2" ParentId="15937" CreationDate="2021-05-21T13:00:38.910" Score="3" Body="&lt;p&gt;Many thanks for raising this issue. The web service is now working. Only the glyph related web service calls were affected due to restructuring folders at the SBOL-Visual GitHub repository. I now revived the glyphs using the sbol-visual-ontology GitHub repository. Over the summer, we will develop the web service further so that you can request different versions of glyphs. Currently, the web service works with the SBOL Visual v2.3 glyphs and we will provide support for SBOL Visual v3 glyphs soon.&lt;/p&gt;&#xA;&lt;p&gt;The web service is available at:&#xA;&lt;a href=&quot;https://vows.sbolstandard.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://vows.sbolstandard.org/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Please let me know if there is anything I can clarify.&lt;/p&gt;&#xA;" OwnerUserId="12286" LastActivityDate="2021-05-21T13:00:38.910" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15951" PostTypeId="2" ParentId="15948" CreationDate="2021-05-21T13:44:08.397" Score="2" Body="&lt;p&gt;The function you need is &lt;code&gt;CreateSeuratObject()&lt;/code&gt; and not &lt;code&gt;Read10X()&lt;/code&gt; as you start from TPMs.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2021-05-21T13:44:08.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15952" PostTypeId="1" CreationDate="2021-05-21T13:54:34.643" Score="1" ViewCount="201" Body="&lt;p&gt;I can easily grab SNP-Chip data from NCBI's GEO database for a given Illumina SNP-Chip array, e.g. from here:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE52147&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE52147&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;GEO provides data as &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/info/soft.html#format&quot; rel=&quot;nofollow noreferrer&quot;&gt;SOFT formatted family file(s)&lt;/a&gt;, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/info/MINiML.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MINiML formatted family file(s)&lt;/a&gt; or Series Matrix File(s).&lt;/p&gt;&#xA;&lt;p&gt;I'm looking for a way to convert these into PED and MAP files for use in &lt;a href=&quot;https://doi.org/10.1186/s13742-015-0047-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;PLINK&lt;/a&gt;...&lt;/p&gt;&#xA;&lt;p&gt;Or does PLINK read these formats natively?&lt;/p&gt;&#xA;" OwnerUserId="8875" LastActivityDate="2021-06-11T18:25:58.100" Title="Convert SNP data from GEO into PED and MAP files?" Tags="|plink|format-conversion|geo|snp-chip|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15953" PostTypeId="1" CreationDate="2021-05-21T16:57:10.027" Score="1" ViewCount="431" Body="&lt;p&gt;I have a &lt;code&gt;MainDirectory/hundreds_of_subdirectories/thousands_of_fastas&lt;/code&gt; structure. For each fasta in the bottom level, I am trying to change the file name as well as the first line in the file to retain only some parts of the file.&lt;/p&gt;&#xA;&lt;p&gt;So that the following text, which makes up both the file name and the first name of the file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;947-length-1150-cov-1000|contig:JAECWU010000213.1|slice:817050-818200|uce:uce-1452|match:817550-817700|orient:{'+'}|probes:4.unaligned.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;becomes just:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;uce-1452.unaligned.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For editing the first line in the file, I have tried&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sed '1 s/^[^|uce-]*(|uce-[0-9]).*/\1/' hundreds_of_subdirectories/*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but this isn't the ticket..&lt;/p&gt;&#xA;&lt;p&gt;Then messing around with some kind of loop...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in *\*.unaligned.fasta&#xA;do&#xA;  sed -E 's/^[^|uce-]*(|uce-[0-9]).*/\1/'&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For the file name, it was suggested that I use rename instead of sed, but with sed I have tried something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for x in hundreds_of_subdirectories/thousands_of_fastas*unaligned.fasta&#xA;do&#xA;  echo $x | sed -r 's/^[^|uce-]*(|uce-[0-9]).*/mv' -v &amp;quot;\0&amp;quot; &amp;quot;|uce-[0-9].*/\1/.unaligned.fasta&amp;quot;/&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Am I even barking up the right tree? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="12291" LastEditorUserId="650" LastEditDate="2021-05-21T18:17:41.177" LastActivityDate="2021-05-24T16:11:56.703" Title="Retain only part of a file name and fasta header in fasta directory" Tags="|fasta|bash|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15954" PostTypeId="2" ParentId="15942" CreationDate="2021-05-21T17:48:45.503" Score="3" Body="&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://github.com/openmm/pdbfixer&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDBFixer&lt;/a&gt;&lt;/strong&gt; and &lt;strong&gt;&lt;a href=&quot;http://mmb.irbbarcelona.org/gitlab/BioExcel/pyMutate&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyMutate&lt;/a&gt;&lt;/strong&gt; are simple, lightweight, free and open-source. Both don't do any optimizing, though, so you may have to run some molecular dynamics afterward to get what would be the right &amp;quot;orientation&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;a href=&quot;https://www.ks.uiuc.edu/Research/vmd/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VMD&lt;/a&gt;&lt;/strong&gt; would work for sure, I believe &lt;strong&gt;&lt;a href=&quot;https://pymol.org/2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyMOL&lt;/a&gt;&lt;/strong&gt; too. They are both graphical interfaces but you can start them in text only mode (or just &lt;code&gt;import pymol&lt;/code&gt; in a python script and use the Python API). They are free for particulars and academics, but given their renown you may already be using them.&lt;/p&gt;&#xA;&lt;p&gt;If you're looking for something a tad more complete and that may adapt the side-chain to the surrounding environment I'd go with &lt;strong&gt;&lt;a href=&quot;https://salilab.org/modeller/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Modeller&lt;/a&gt;&lt;/strong&gt;, which provides two example scripts to do point mutations (&lt;a href=&quot;https://salilab.org/modeller/8v0/manual/node125.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://salilab.org/modeller/wiki/Mutate%20model&quot; rel=&quot;nofollow noreferrer&quot;&gt;there&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;If you don't want to just get the side-chain of this amino-acid right but actually strive to see some general backbone adaptation too then use &lt;strong&gt;&lt;a href=&quot;http://www.pyrosetta.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyRosetta&lt;/a&gt;&lt;/strong&gt;. It can probably do what all the others do, too, but given it is 1 GB big it may be overkill. Any of the package coupled with molecular dynamics would also do the trick.&lt;/p&gt;&#xA;" OwnerUserId="12054" LastActivityDate="2021-05-21T17:48:45.503" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15956" PostTypeId="2" ParentId="15953" CreationDate="2021-05-22T04:54:12.940" Score="1" Body="&lt;p&gt;I think you're working too hard to capture the &lt;code&gt;uce:&lt;/code&gt; expression.&lt;/p&gt;&#xA;&lt;p&gt;Try&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sed -Ee '1s/^.+uce:(uce-[0-9]+).+/\1.unaligned.fasta/' file_glob&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to replace the first line content. You may want to add the in-place flag after testing the expression on a few files.&lt;/p&gt;&#xA;&lt;p&gt;For the &lt;code&gt;mv&lt;/code&gt;, use command substitution, which runs the command in a sub-shell and uses the output value as the parameter:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for f in file_glob&#xA;do    &#xA;    mv -- &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$f&quot; &quot;$&lt;/span&gt;(echo \&amp;quot;$f\&amp;quot; | sed '...')&amp;quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastEditorUserId="650" LastEditDate="2021-05-24T16:11:56.703" LastActivityDate="2021-05-24T16:11:56.703" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="15958" PostTypeId="1" CreationDate="2021-05-22T12:01:39.730" Score="4" ViewCount="666" Body="&lt;p&gt;This is my workflow:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pre_align()&#xA;pre_align.out.single_fastqs.view()&#xA;get_fq_info(pre_align.out.single_fastqs)&#xA;&#xA;align_bwa(get_fq_info.out.fq_info)&#xA;&#xA;align_bowtie2(get_fq_info.out.fq_info)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where I want to use the same output from &lt;code&gt;get_fq_info&lt;/code&gt; as input for either &lt;code&gt;align_bwa&lt;/code&gt; and align_bowtie2`.&lt;/p&gt;&#xA;&lt;p&gt;As it's shown here &lt;strong&gt;only align_bwa is executed&lt;/strong&gt;. If I comment &lt;code&gt;align_bwa&lt;/code&gt; THEN &lt;code&gt;align_bowtie2&lt;/code&gt; gets executed.&lt;/p&gt;&#xA;&lt;p&gt;How can I fix this?&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="2970" LastActivityDate="2022-01-10T17:58:25.413" Title="use same output in two processes in nextflow dsl2" Tags="|nextflow|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15959" PostTypeId="1" CreationDate="2021-05-22T13:22:53.100" Score="2" ViewCount="188" Body="&lt;p&gt;I have a question about using FDRtool. In the below code (on RNA seq data whose p values were acquired using Deseq2), the FDRtool was first used and thereafter p.adjust using the benjamini hochberg method. I didn't write this, its taken from someone else's code found online, in addition it is also the way the it is written in the pipeline at my university. Therefore, I assumed that this is how fdrtool is used and in my eyes it looks like they both adjust the p value with the same method. So that's why I asked, as I want to understand why you would need both these lines, which I've seen many people do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FDR.ddsRes &amp;lt;- fdrtool(ddsRes$stat, statistic= &amp;quot;normal&amp;quot;, plot = T)&#xA;ddsRes[,&amp;quot;padj&amp;quot;]  &amp;lt;- p.adjust(FDR.ddsRes$pval, method = &amp;quot;BH&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Dont they both correct for false discovery rate? Why do you need to run p.adjust after using fdrtool?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!!&lt;/p&gt;&#xA;" OwnerUserId="12295" LastEditorUserId="5077" LastEditDate="2021-05-24T08:46:53.020" LastActivityDate="2021-05-24T09:09:47.190" Title="How does FDRtool work?" Tags="|r|rna-seq|deseq2|rna|p-values|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15960" PostTypeId="1" AcceptedAnswerId="15961" CreationDate="2021-05-22T18:53:59.367" Score="0" ViewCount="200" Body="&lt;p&gt;I am indexing a structure by the code below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;PDB_CODE = '1lve'&#xA;RES_CHAIN = 'A'&#xA;RES_NUM = 27    &#xA;&#xA;structure = PDBParser(is_pqr=True).get_structure(PDB_CODE, PDB_CODE+&amp;quot;.pqr&amp;quot;)&#xA;residue = structure[0][RES_CHAIN][RES_NUM]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, the 1lve contains residues with the same RES_NUM but different icodes 'A' to 'F' so my code either ignores them or throws an error if I pass '27A'. Does biopython allow this kind of structure indexing directly?&lt;/p&gt;&#xA;" OwnerUserId="7150" LastActivityDate="2021-05-22T19:13:02.233" Title="How to index residues by icode in biopython?" Tags="|python|protein-structure|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15961" PostTypeId="2" ParentId="15960" CreationDate="2021-05-22T19:13:02.233" Score="0" Body="&lt;p&gt;Okay, I found the answer myself. It is possible to index not only by RES_NUM, but by its id (check with residue.get_id()), so the below example would work:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#structure[0][RES_CHAIN][het, resseq, icode]&#xA;structure[0][RES_CHAIN][' ', 27, 'A']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7150" LastActivityDate="2021-05-22T19:13:02.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15962" PostTypeId="2" ParentId="14134" CreationDate="2021-05-23T12:42:34.537" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @ram-rs, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;For the &lt;code&gt;Group&lt;/code&gt;, try using it without the &lt;code&gt;anno_simple&lt;/code&gt;. Just &lt;code&gt;Top_ha = HeatmapAnnotation(Group=top$Group, &amp;quot;Total Mutation Burden&amp;quot; = ...)&lt;/code&gt;, nothing else for &lt;code&gt;Group&lt;/code&gt;. See if that generates a legend at all. We can customize later.hing else for Group. See if that generates a legend at all. We can customize later.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-23T12:42:34.537" CommentCount="0" CommunityOwnedDate="2021-05-23T12:42:34.537" ContentLicense="CC BY-SA 4.0" />
  <row Id="15963" PostTypeId="2" ParentId="14125" CreationDate="2021-05-23T12:59:58.853" Score="1" Body="&lt;p&gt;A &lt;em&gt;gene set enrichment analysis&lt;/em&gt; (GSEA) tests for &lt;em&gt;enrichment&lt;/em&gt; of a gene set within a ranked list of genes.&lt;/p&gt;&#xA;&lt;p&gt;The primary outcome of the analysis is &lt;em&gt;enrichment&lt;/em&gt; or &lt;em&gt;no enrichment&lt;/em&gt;. Gene sets frequently include correlated genes, and that correlation can be positive (enhancers or co-expressors) or negative (e.g repressors). It's reasonable to include negatively-correlated genes within gene sets because some pathways involve both enhancement of a particular biochemical pathway, and suppression of a pathway that carries out opposite things.&lt;/p&gt;&#xA;&lt;p&gt;I don't know about the KRAS pathway you're comparing against here, but a statement I would make based on interpreting that GSEA graph would be something like, &amp;quot;The KRAS.600_UP.V1_DN gene set is enriched when comparing the expression of responders vs non-responders.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;I'm not quite sure how you separated up and down-regulated genes (code would be helpful with that). Your down regulated gene plot looks like it's ranked in the opposite direction to the combined plot, which is not what I expect. I think that determining an enrichment score separately for upregulated and downregulated genes (within your ranked list of genes) is a good idea, but it shouldn't change the direction of the enrichment curve within each category. It will, however, alter the form of the curve, by stretching or compressing values vertically and altering the start/end offset.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-05-23T13:09:18.437" LastActivityDate="2021-05-23T13:09:18.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15964" PostTypeId="2" ParentId="13341" CreationDate="2021-05-23T13:17:53.187" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @stupidwolf, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;You cannot use grepl with a vector, it must be a pattern.&lt;/p&gt;&#xA;&lt;p&gt;I can suggest two things and you just put them together i hope, to know if something is found in a vector, you can use &lt;code&gt;data$Sample %in% c(&amp;quot;TCGA-IC-A6RE&amp;quot;, &amp;quot;TCGA-IG-A4QS&amp;quot;)&lt;/code&gt; etc ..&lt;/p&gt;&#xA;&lt;p&gt;Of course &lt;code&gt;data$Sample&lt;/code&gt; will not be in there, you need a bit of manipulation, so since your samples follow a certain pattern, you can use &lt;code&gt;substr(data$Sample,....)&lt;/code&gt; . If you do &lt;code&gt;substr(data$Sample,....) %in% c(.....) &lt;/code&gt;, this will give you the selection.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-23T13:17:53.187" CommentCount="0" CommunityOwnedDate="2021-05-23T13:17:53.187" ContentLicense="CC BY-SA 4.0" />
  <row Id="15965" PostTypeId="2" ParentId="13242" CreationDate="2021-05-23T13:21:41.230" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @atpoint, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;There are dozens of threads in GitHub, SE and other support platforms that clearly explain what to do. (e.g. &lt;a href=&quot;https://github.com/velocyto-team/velocyto.R/issues/2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/velocyto-team/velocyto.R/issues/2&lt;/a&gt;) The issue is that the compiler does not support openMP, so you have to install one that does.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Answer from @devon-ryan, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;In conda on OSX we solve this by installing llvm-openmp, which provides the -fopenmp flag. Maybe the same is possible outside of conda (no clue). Don't compile it, find an easier way to install it.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-23T13:21:41.230" CommentCount="0" CommunityOwnedDate="2021-05-23T13:21:41.230" ContentLicense="CC BY-SA 4.0" />
  <row Id="15966" PostTypeId="1" AcceptedAnswerId="15977" CreationDate="2021-05-23T18:27:48.207" Score="0" ViewCount="1089" Body="&lt;p&gt;Recently, I had to re-install all my R packages (R version 4.1.0 (2021-05-18) -- &amp;quot;Camp Pontanezen&amp;quot;) and it turns out that &lt;code&gt;&amp;quot;RDAVIDWebService&amp;quot;&lt;/code&gt; is not available anymore on Bioconductor:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;code&gt;package ‘RDAVIDWebService’ is not available for Bioconductor version '3.13'&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Is there an another way to install it (couldn't find the files on the Bioconductor site to install the package manually) or is there another R package that allows to use the &lt;a href=&quot;https://david.ncifcrf.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DAVID tool&lt;/a&gt;?&lt;/p&gt;&#xA;" OwnerUserId="10386" LastActivityDate="2021-05-24T19:51:54.640" Title="RDAVIDWebService (R package) manual installation, or alternative tool" Tags="|r|bioconductor|gene-ontology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15967" PostTypeId="1" CreationDate="2021-05-23T21:35:17.603" Score="1" ViewCount="17" Body="&lt;p&gt;Does anybody know what units the mirna expression data on ccle is in? I saw the pipeline in the original paper but it doesn't mention units.&lt;/p&gt;&#xA;&lt;p&gt;Original paper: &lt;a href=&quot;https://www.nature.com/articles/s41586-019-1186-3?proof=t&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41586-019-1186-3?proof=t&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Data: &lt;a href=&quot;https://portals.broadinstitute.org/ccle/data&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://portals.broadinstitute.org/ccle/data&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12299" LastActivityDate="2021-05-23T21:35:17.603" Title="Ccle miRna data units" Tags="|rna-seq|public-databases|cancer|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15968" PostTypeId="1" CreationDate="2021-05-23T22:21:56.560" Score="2" ViewCount="46" Body="&lt;p&gt;I'm new to bioinformatics, and I'm looking for a tool that takes two FASTA files, one containing a reference, let's call it &lt;code&gt;A.fasta&lt;/code&gt;, the other another sequence, let's call it &lt;code&gt;B.fasta&lt;/code&gt;, and outputs just the mutations that &lt;code&gt;B&lt;/code&gt; contains with respect to &lt;code&gt;A&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I'm open as to the output format. Is there a standard for this?&lt;/p&gt;&#xA;&lt;p&gt;Background: I'm trying to analyse Sars-CoV-2 sequences and identify mutations with respect to the reference. I've done a google search for a tool that does what I want to but couldn't find something straightaway. Is there no standard tool for this?&lt;/p&gt;&#xA;" OwnerUserId="12272" LastEditorUserId="3848" LastEditDate="2022-02-25T20:54:34.180" LastActivityDate="2022-02-25T20:54:34.180" Title="Python package or CLI tool outputting mutations of sequence with respect to reference" Tags="|sequence-alignment|fasta|samtools|covid-19|sars-cov-2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15969" PostTypeId="1" CreationDate="2021-05-24T04:55:29.510" Score="2" ViewCount="500" Body="&lt;p&gt;I am doing a course on graph algorithms in genome sequencing and the current assignment involves building an overlap graph from a bunch of k-mers. Formally the problem is as follows:&lt;br&gt;&lt;br&gt;&#xA;&lt;strong&gt;Input:&lt;/strong&gt; A collection of Patterns of k-mers.&lt;br&gt;&#xA;&lt;strong&gt;Output:&lt;/strong&gt; The overlap graph in the form of an adjacency list.&lt;br&gt;&lt;br&gt;&#xA;I have a working algorithm that handles most cases except for self-loops. The problem is that I don't know when/why I should be adding self-loops. Any help on this would be greatly appreciated. Below are some examples illustrating my question and my current solution. Thank you!&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Example 1&lt;/em&gt;:&lt;br&gt;&#xA;Input:&lt;/p&gt;&#xA;&lt;p&gt;ACT &lt;br&gt;&#xA;CTT &lt;br&gt;&#xA;TTT &lt;br&gt;&lt;/p&gt;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&lt;p&gt;TTT-&amp;gt;TTT &lt;br&gt;&#xA;ACT-&amp;gt;CTT &lt;br&gt;&#xA;CTT-&amp;gt;TTT &lt;br&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Example 2&lt;/em&gt;:&lt;br&gt;&#xA;Input:&lt;/p&gt;&#xA;&lt;p&gt;CCCC &lt;br&gt;&#xA;Output:&lt;/p&gt;&#xA;&lt;p&gt;CCCC-&amp;gt;CCCC &lt;br&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Example 3&lt;/em&gt;: &lt;br&gt;&#xA;Input:&lt;/p&gt;&#xA;&lt;p&gt;CT &lt;br&gt;&#xA;TG &lt;br&gt;&#xA;TG &lt;br&gt;&#xA;TC &lt;br&gt;&#xA;TT &lt;br&gt;&#xA;TC &lt;br&gt;&#xA;Output:&lt;/p&gt;&#xA;&lt;p&gt;TC-&amp;gt;CT &lt;br&gt;&#xA;CT-&amp;gt;TC,TG,TT &lt;br&gt;&#xA;TT-&amp;gt;TC,TG,TT &lt;br&gt;&lt;/p&gt;&#xA;&lt;p&gt;The goal here is to represent a collection of kmers as an overlap graph where unique kmer will be represented as a node in the graph. Directed edges will connect nodes whose last &lt;span class=&quot;math-container&quot;&gt;$k-1$&lt;/span&gt; characters overlap with the other node's first &lt;span class=&quot;math-container&quot;&gt;$k-1$&lt;/span&gt; characters. Therefore, repeated kmers that overlap with a other kmer should be reflected as self loops. That's what I understood, however I am looking for a more general rule/solution.&lt;/p&gt;&#xA;&lt;p&gt;Here is another example to illustrate this.&lt;/p&gt;&#xA;&lt;p&gt;Input:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CT&#xA;TT&#xA;TT&#xA;TT&#xA;TT&#xA;TT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CT-&amp;gt;TT&#xA;TT-&amp;gt;TT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Current code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/*&#xA; * Creates an overlap graph from a list of k-mers&#xA; *&#xA; * \param patterns      a vector of string k-mers&#xA; *&#xA; * \return a string containing an adjacency list representation of the&#xA; * overlap graph as described in the problem specification&#xA; * &#xA; * Output Format. The overlap graph Overlap(Patterns), in the form of an&#xA; * adjacency list. Specifically, each line of the output represents all&#xA; * edges (u,v) leaving a given node u in the format “u -&amp;gt; v”, where u and v&#xA; * are both k-mers. If a given node u has multiple edges leaving it (e.g. v and w),&#xA; * the destination nodes are comma-separated in any order. For example, if&#xA; * there exist nodes ACG, CGT, and CGG, the resulting line of the adjacency&#xA; * list would be: ACG -&amp;gt; CGT,CGG&#xA; */&#xA;&#xA;&#xA;#include &amp;lt;algorithm&amp;gt;&#xA;#include &amp;lt;iostream&amp;gt;&#xA;#include &amp;lt;map&amp;gt;&#xA;#include &amp;lt;vector&amp;gt;&#xA;#include &amp;lt;set&amp;gt;&#xA;#include &amp;lt;string&amp;gt;&#xA;&#xA;&#xA;std::string overlap_graph(const std::vector&amp;lt;std::string&amp;gt;&amp;amp; patterns) {&#xA;    // Overlap Graph of the k-mers collection patterns, in which a vertex exists for&#xA;    // each unique k-mer, and an edge (u,v) exists if the last k-1 characters of u are&#xA;    // equal to the first k-1 characters of v.&#xA;&#xA;    if (patterns.size() == 1) return patterns[0] + &amp;quot;-&amp;gt;&amp;quot; + patterns[0];&#xA;    // Build adjancency list O(n^2)&#xA;    std::map &amp;lt;std::string, std::vector&amp;lt;std::string&amp;gt;&amp;gt; adj;&#xA;    for (const auto&amp;amp; kmer_u : patterns) {&#xA;        // Add kmer as key if it doesn't exist in the map and find its neighbours.&#xA;        if (adj.find(kmer_u) == adj.end()) {&#xA;            adj[kmer_u] = {};&#xA;            for (const auto&amp;amp; kmer_v : patterns) {&#xA;                if ((kmer_u != kmer_v) &amp;amp;&amp;amp;&#xA;                    (kmer_u.substr(1, kmer_u.length() - 1) == kmer_v.substr(0, kmer_v.length() - 1)) &amp;amp;&amp;amp; &#xA;                    (std::find(adj[kmer_u].begin(), adj[kmer_u].end(), kmer_v) == adj[kmer_u].end())) {&#xA;                    adj[kmer_u].push_back(kmer_v);&#xA;                }&#xA;            }&#xA;        } // Otherwise continue.&#xA;    }&#xA;&#xA;    // Convert to string O(n).&#xA;    std::string ans;&#xA;    for (auto const&amp;amp; key_value : adj) {&#xA;        std::string kmer_u{ key_value.first };&#xA;        std::vector&amp;lt;std::string&amp;gt; kmers_v{ key_value.second };&#xA;        if (!kmers_v.empty()) {&#xA;            ans.append(kmer_u + &amp;quot;-&amp;gt;&amp;quot; + kmers_v[0]);&#xA;            for (std::size_t kmer_v{ 1 }; kmer_v &amp;lt; kmers_v.size(); kmer_v++) {&#xA;                ans.append(',' + kmers_v[kmer_v]);&#xA;            }&#xA;            ans.push_back('\n');&#xA;        }&#xA;    }&#xA;    return ans;&#xA;}&#xA;&#xA;&#xA;void tests() {&#xA;    /* Sample Dataset */&#xA;    std::vector&amp;lt;std::string&amp;gt; patterns{ {&amp;quot;AAG&amp;quot;}, {&amp;quot;AGA&amp;quot;}, {&amp;quot;ATT&amp;quot;}, {&amp;quot;CTA&amp;quot;}, {&amp;quot;CTC&amp;quot;}, {&amp;quot;GAT&amp;quot;},&#xA;                                       {&amp;quot;TAC&amp;quot;}, {&amp;quot;TCT&amp;quot;}, {&amp;quot;TCT&amp;quot;}, {&amp;quot;TTC&amp;quot;} };&#xA;    std::string answer{ &amp;quot;TTC-&amp;gt;TCT\nCTA-&amp;gt;TAC\nAAG-&amp;gt;AGA\nCTC-&amp;gt;TCT\nTCT-&amp;gt;CTA,CTC\nAGA-&amp;gt;GAT\nGAT-&amp;gt;ATT\nATT-&amp;gt;TTC\n&amp;quot; };&#xA;    std::cout &amp;lt;&amp;lt; answer &amp;lt;&amp;lt; std::endl;&#xA;    // std::cout &amp;lt;&amp;lt; (overlap_graph(patterns) == answer) &amp;lt;&amp;lt; std::endl;&#xA;    std::cout &amp;lt;&amp;lt; overlap_graph(patterns) &amp;lt;&amp;lt; std::endl;&#xA;}&#xA;&#xA;&#xA;void solution() {&#xA;    std::vector&amp;lt;std::string&amp;gt; patterns;&#xA;&#xA;    std::string pattern;&#xA;    while (std::cin &amp;gt;&amp;gt; pattern) {&#xA;        patterns.push_back(pattern);&#xA;    }&#xA;    std::cout &amp;lt;&amp;lt; overlap_graph(patterns) &amp;lt;&amp;lt; std::endl;&#xA;}&#xA;&#xA;&#xA;int main() {&#xA;    //tests();&#xA;    solution();&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12302" LastEditorUserId="6628" LastEditDate="2021-05-26T08:12:02.327" LastActivityDate="2021-06-25T09:04:50.450" Title="Construct the Overlap Graph of a Collection of k-mers" Tags="|ngs|k-mer|homework|c++|graphs|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="15970" PostTypeId="2" ParentId="15968" CreationDate="2021-05-24T06:15:45.043" Score="2" Body="&lt;p&gt;If it's just a comparison of two sequences that you're looking for, then &lt;em&gt;diffseq&lt;/em&gt; from the &lt;a href=&quot;http://emboss.sourceforge.net/what/&quot; rel=&quot;nofollow noreferrer&quot;&gt;emboss toolkit&lt;/a&gt; should be okay:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.bioinformatics.nl/cgi-bin/emboss/diffseq&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioinformatics.nl/cgi-bin/emboss/diffseq&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I think this tool requires the sequences to be oriented in the same direction. If you discover a &lt;em&gt;lot&lt;/em&gt; more variants than you expect, try reverse-complementing one of the sequences.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-24T06:15:45.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15971" PostTypeId="2" ParentId="15959" CreationDate="2021-05-24T09:09:47.190" Score="1" Body="&lt;p&gt;Something like this &lt;a href=&quot;https://web.stanford.edu/class/bios221/book/Chap-Testing.html#the-false-discovery-rate&quot; rel=&quot;nofollow noreferrer&quot;&gt;book chapter&lt;/a&gt; might help on understanding FDR. The packages or functions you call differ in their estimation of pi0 or the proportion of null hypothesis.&lt;/p&gt;&#xA;&lt;p&gt;Basically, the &lt;code&gt;p.adjust(..method = &amp;quot;BH&amp;quot;)&lt;/code&gt; is a more conservative method with Benjamini-Hochberg, you are assuming the proportion of null features to be 1.&lt;/p&gt;&#xA;&lt;p&gt;When you use fdrtool, you are estimating the proportion of null hypothesis and from there, the local FDR. This in theory gives you more power.&lt;/p&gt;&#xA;&lt;p&gt;However, I have to note that in your code, you are not using &lt;code&gt;fdrtool&lt;/code&gt; to estimate the fdr as explained above, because:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FDR.ddsRes &amp;lt;- fdrtool(ddsRes$stat, statistic= &amp;quot;normal&amp;quot;, plot = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With this, the raw p-values are stored in &lt;code&gt;FDR.ddsRes$pval&lt;/code&gt; and the q value, which is the corrected p-value, is stored in &lt;code&gt;FDR.ddsRes$qval&lt;/code&gt;. In your code you are basically taking the raw p-values, and applying Benjamin-Hochberg:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;p.adjust(FDR.ddsRes$pval, method = &amp;quot;BH&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You should apply a two tailed test. From the wald stats, it should be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;derived_p = 2*pnorm(-abs(ddsRes$stat))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In your code, you are running it wrongly by passing the test statistic into &lt;code&gt;fdrtools&lt;/code&gt;, we can compare the raw p-values:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head(data.frame(stat = ddsRes&lt;span class=&quot;math-container&quot;&gt;$stat,derived = derived_p,&#xA;DESeq2=ddsRes$&lt;/span&gt;pvalue,fdrtools=FDR.ddsRes$pval))&#xA;&#xA;        stat    derived     DESeq2  fdrtools&#xA;1  1.2405796 0.21476110 0.21476110 0.1216358&#xA;2  1.2314421 0.21815755 0.21815755 0.1244054&#xA;3 -0.3292516 0.74196552 0.74196552 0.6811997&#xA;4 -0.4964574 0.61957174 0.61957174 0.5356140&#xA;5 -0.4101163 0.68172066 0.68172066 0.6088427&#xA;6 -1.8007943 0.07173531 0.07173531 0.0246428&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To summarize, use the adjusted p-value returned by DESeq2. The code you have above is wrong unless there's a good reason to do a one-tail test.&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2021-05-24T09:09:47.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15972" PostTypeId="1" CreationDate="2021-05-24T13:19:27.653" Score="4" ViewCount="244" Body="&lt;p&gt;Some programs (e.g. shapeit4) automatically annotate an &lt;code&gt;INFO&lt;/code&gt; tag into a .vcf file which gives the cumulative genetic distance in cM between each SNP:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.2&#xA;##FILTER=&amp;lt;ID=PASS,Description=&amp;quot;All filters passed&amp;quot;&amp;gt;&#xA;##fileDate=10/11/2020 - 21:29:39&#xA;##source=G2H&#xA;##contig=&amp;lt;ID=9&amp;gt;&#xA;##INFO=&amp;lt;ID=AF,Number=A,Type=Float,Description=&amp;quot;Allele Frequency&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=AC,Number=1,Type=Integer,Description=&amp;quot;Allele count&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=CM,Number=A,Type=Float,Description=&amp;quot;Interpolated cM position&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&amp;quot;Phased genotypes&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=AN,Number=1,Type=Integer,Description=&amp;quot;Total number of alleles in called genotypes&amp;quot;&amp;gt;&#xA;##bcftools_viewVersion=1.9-207-g2299ab6+htslib-1.9-271-g6738132&#xA;##bcftools_viewCommand=view -s -1_-1 ukb_cal_chr9_v2.HumanOrigins.common_snps.HumanOriginsInds.fiftyPercentEthnicUKBInds.PHASED.bgz.vcf.gz; Date=Mon May 24 14:15:51 2021&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  -1_-1&#xA;9   257034  rs573860    A   G   .   .   AC=0;AF=0.388699;CM=0;AN=2  GT  0|0&#xA;9   270321  rs17720310  A   G   .   .   AC=0;AF=0.0820595;CM=0.007994;AN=2  GT  0|0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, this program is really used for phasing not for .vcf annotation.&lt;/p&gt;&#xA;&lt;p&gt;Ideally I'd like a simple command line program which takes input the .vcf and a genetic map and interpolates the genetic distance and adds a cumulative map. This could be done manually with a custom script, but quite frankly I can't be bothered to do that.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know of a tool that exists which will do this?&lt;/p&gt;&#xA;&lt;p&gt;I &lt;a href=&quot;https://www.biostars.org/p/488447/&quot; rel=&quot;nofollow noreferrer&quot;&gt;posted&lt;/a&gt; this to biostars a while ago and received one reply which linked to &lt;a href=&quot;https://www.biostars.org/p/16236/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt;, which includes links to tables containing cM locations for different rs numbers.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: I am interested in doing this for human data (although the same would be the case for any diploid organism).&lt;/p&gt;&#xA;&lt;p&gt;Sample vcf:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://pastebin.com/raw/YSbuU0qM&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/raw/YSbuU0qM&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="3694" LastEditDate="2021-05-31T15:07:36.537" LastActivityDate="2021-05-31T15:07:36.537" Title="Annotating a .vcf with centimorgan information" Tags="|vcf|sequence-annotation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15973" PostTypeId="2" ParentId="15944" CreationDate="2021-05-24T13:31:26.837" Score="1" Body="&lt;p&gt;Black line represents an indel.  Read was lengthened to fit consensus.&lt;/p&gt;&#xA;&lt;p&gt;No black line (blank), no read/sequence.  Read likely ended or was truncated.&lt;/p&gt;&#xA;" OwnerUserId="6343" LastActivityDate="2021-05-24T13:31:26.837" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15975" PostTypeId="2" ParentId="15953" CreationDate="2021-05-24T15:21:12.893" Score="1" Body="&lt;p&gt;If you are using &lt;code&gt;bash&lt;/code&gt;, you can enable the &lt;code&gt;globstar&lt;/code&gt; option (&lt;code&gt;shopt -s globstar&lt;/code&gt;) which allows &lt;code&gt;**&lt;/code&gt; to match recursively. That way, &lt;code&gt;**/*fasta&lt;/code&gt; will find &lt;em&gt;all&lt;/em&gt; files whose name ends with &lt;code&gt;fastq&lt;/code&gt; in this directory and all of its subdirectories. So that will let you easily iterate over all the target names. Next, if you have the perl &lt;code&gt;rename&lt;/code&gt; command (called &lt;code&gt;rename&lt;/code&gt; on Debian and Ubuntu and similar systems, &lt;code&gt;prename&lt;/code&gt; or &lt;code&gt;perl-rename&lt;/code&gt; on others), you can do it quite easily.&lt;/p&gt;&#xA;&lt;p&gt;Note that &lt;code&gt;perl-rename&lt;/code&gt; has the &lt;code&gt;-n&lt;/code&gt; option that makes it print out the renaming it would do without actually renaming anything. So I suggest you first run this to test what you will be doing:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;shopt -s globstar&#xA;for file in **/*.fasta; do&#xA;   echo perl -i.bak -pe 's/^\s*&amp;gt;.*uce:(uce-\d+).*(\.unaligned.fasta)/$1$2/' &amp;quot;$file&amp;quot;&#xA;   rename -n 's/^.*uce:(uce-\d+).*(\.unaligned.fasta)/$1$2/' &amp;quot;$file&amp;quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That will &lt;code&gt;echo&lt;/code&gt; the perl command, so you can try it manually first and make sure it works, and will print out the &lt;code&gt;rename&lt;/code&gt; operations without actually doing anything. If you are satisfied it will work, run again without the &lt;code&gt;echo&lt;/code&gt; or &lt;code&gt;-n&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for file in **/*fasta; do&#xA;    perl -i.bak -pe 's/^\s*&amp;gt;.*uce:(uce-\d+).*(\.unaligned.fasta)/$1$2/' &amp;quot;$file&#xA;    rename 's/^.*uce:(uce-\d+).*(\.unaligned.fasta)/$1$2/' &amp;quot;$file&amp;quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or, if you don't have &lt;code&gt;perl-rename&lt;/code&gt;, you can use the shell's string manipulation for this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for file in **/*.fasta; do&#xA;    perl -i.bak -pe 's/^\s*&amp;gt;.*uce:(uce-\d+).*(\.unaligned.fasta)/$1$2/' &amp;quot;$file&#xA;    tmpFileName=&amp;quot;${file#*uce:}&amp;quot;&#xA;    mv -- &amp;quot;$file&amp;quot; &amp;quot;${tmpFileName/|*/.unaligned.fasta}&amp;quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3&gt;Explanation&lt;/h3&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;shopt -s globstar&lt;/code&gt;: this enables the &lt;code&gt;globstar&lt;/code&gt; bash option for the current shell session. It only needs to be run once, its effects will last until you close the terminal or start a new shell session. As for what it does, see &lt;code&gt;man bash&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;globstar&lt;/p&gt;&#xA;&lt;p&gt;If set, the pattern &lt;code&gt;**&lt;/code&gt; used in a pathname expansion context will match all files and zero or more directories and subdirectories. If the pattern is followed by a &lt;code&gt;/&lt;/code&gt;, only directories and subdirectories match.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;for file in **/*fasta; do ... ; done&lt;/code&gt;: iterate over all files (and directories) whose name ends in &lt;code&gt;.fasta&lt;/code&gt;, saving each file name as &lt;code&gt;$file&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;perl -i.bak -pe 's/^\s*&amp;gt;.*(uce-\d+).*(\.unaligned.fasta)/$1$2/' &amp;quot;$file&lt;/code&gt;: this changes the fasta header. The &lt;code&gt;-i.bak&lt;/code&gt; option tells &lt;code&gt;perl&lt;/code&gt; to edit the file in place, but to save the original file with a &lt;code&gt;.bak&lt;/code&gt; extension. This can help you revert any changes if there is a mistake. Once you have finished, you could run &lt;code&gt;rm **/*fasta.bak&lt;/code&gt; to delete the backups. Alternatively, just remove the &lt;code&gt;-i.bak&lt;/code&gt; if you don't want to keep backups.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;-p&lt;/code&gt; option tells &lt;code&gt;perl&lt;/code&gt; to process its input line by line and print every line after applying whatever script you give it with &lt;code&gt;-e&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;s/old/new/&lt;/code&gt; is the substitution operator, same as &lt;code&gt;sed&lt;/code&gt;. The regular expression is looking for 0 or more whitespace characters and then a &lt;code&gt;&amp;gt;&lt;/code&gt; at the beginning of the line (&lt;code&gt;^\s*&amp;gt;&lt;/code&gt;). It then allows for 0 or more characters (&lt;code&gt;.*&lt;/code&gt;) until it finds the string &lt;code&gt;uce:&lt;/code&gt; followed by &lt;code&gt;uce-&lt;/code&gt; and one or more digits (&lt;code&gt;uce:(uce-\d+)&lt;/code&gt;). The parentheses &amp;quot;capture&amp;quot; the pattern, storing it as &lt;code&gt;$1&lt;/code&gt;. Next, we want 0 or more characters again until we reach &lt;code&gt;.unaligned.fasta&lt;/code&gt; (&lt;code&gt;.*(\.unaligned.fasta)&lt;/code&gt;). We capture the &lt;code&gt;.unaligned.fasta&lt;/code&gt; as &lt;code&gt;$2&lt;/code&gt; (second set of parentheses) only to avoid typing it out again. Note that this assumes you will never have anything after the &lt;code&gt;.unaligned.fasta&lt;/code&gt;. Finally, we replace everything matched above with just &lt;code&gt;$1&lt;/code&gt; (the &lt;code&gt;uce-numbers&lt;/code&gt;) and &lt;code&gt;$2&lt;/code&gt; (&lt;code&gt;.unaligned.fasta&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;rename 's/^.*uce:(uce-\d+).*(\.unaligned.fasta)/$1$2/' &amp;quot;$file&amp;quot;&lt;/code&gt;: the rename opration. This will match everything from the beginning of the file name until the last occurrence of &lt;code&gt;uce:uce-NNNN&lt;/code&gt;, capture the &lt;code&gt;uce-NNNN&lt;/code&gt; as &lt;code&gt;$1&lt;/code&gt; then skip the remaining characters until &lt;code&gt;.unaligned.fasta&lt;/code&gt; and finally rename as just &lt;code&gt;$1$2&lt;/code&gt; (the capturing logic is the same as above).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;tmpFileName=&amp;quot;${file#*uce:}&amp;quot;&lt;/code&gt;: The &lt;code&gt;${var#pattern}&lt;/code&gt; syntax means &amp;quot;print the value of &lt;code&gt;$var&lt;/code&gt;, but remove the shortest match of &lt;code&gt;pattern&lt;/code&gt; from the front of the variable's value&amp;quot;. The &lt;code&gt;${var//pattern/replacement}&lt;/code&gt; syntax means &amp;quot;print the value of &lt;code&gt;$var&lt;/code&gt; and replace the first occurrence of &lt;code&gt;pattern&lt;/code&gt; with &lt;code&gt;replacement&lt;/code&gt;. Here, the pattern is &lt;code&gt;|*&lt;/code&gt; which means &amp;quot;anything after a &lt;code&gt;|&lt;/code&gt;&amp;quot; (note that this is shell stuff and is using globs and not regular expressions). This is probably easier to understand with an example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ echo &amp;quot;$file&amp;quot;&#xA;947-length-1150-cov-1000|contig:JAECWU010000213.1|slice:817050-818200|uce:uce-1452|match:817550-817700|orient:{'+'}|probes:4.unaligned.fasta&#xA;$ tmpFileName=&amp;quot;${file#*uce:}&amp;quot;&#xA;$ echo &amp;quot;$tmpFileName&amp;quot;&#xA;uce-1452|match:817550-817700|orient:{'+'}|probes:4.unaligned.fasta&#xA;$ echo &amp;quot;${tmpFileName/|*/.unaligned.fasta}&amp;quot;&#xA;uce-1452.unaligned.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, given &lt;code&gt;tmpFileName=&amp;quot;${file#*uce:}&amp;quot;&lt;/code&gt;, &lt;code&gt;&amp;quot;${tmpFileName/|*/.unaligned.fasta}&amp;quot;&lt;/code&gt; is the desired file name and this is what we pass to &lt;code&gt;mv&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2021-05-24T15:33:58.200" LastActivityDate="2021-05-24T15:33:58.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15976" PostTypeId="1" CreationDate="2021-05-24T16:57:59.730" Score="2" ViewCount="76" Body="&lt;p&gt;&lt;em&gt;Note: this question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/9471832/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am currently trying to complete a Breast Cancer Classification task using Neural Networks. I have experimented with using my full dataset of gene expressions(1000+) and managed to improve my results by using only the PAM50 genes. I was wondering though what other genes I could add to the list to even better improve my results.&lt;/p&gt;&#xA;&lt;p&gt;I have tried looking for papers about extending the PAM50, but could not find anything. I would greatly appreciate any advice.&lt;/p&gt;&#xA;" OwnerUserId="12307" LastEditorUserId="73" LastEditDate="2021-06-28T09:14:00.567" LastActivityDate="2021-06-28T09:14:00.567" Title="Important genes beyond PAM50 for breast cancer classification" Tags="|gene|cancer|deep-learning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15977" PostTypeId="2" ParentId="15966" CreationDate="2021-05-24T19:51:54.640" Score="1" Body="&lt;p&gt;That package was deprecated in Bioconductor version 3.13 after several attempts to contact the package author/maintainer.&lt;/p&gt;&#xA;&lt;p&gt;Your options include reverting back to Bioconductor version 3.12 or using one of the other options for GO/pathway enrichment analyses, e.g. &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/topGO.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;topGO&lt;/a&gt;, &lt;a href=&quot;https://cran.r-project.org/web/packages/enrichR/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;enrichR&lt;/a&gt;, or &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/clusterProfiler.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;clusterProfiler&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1269" LastActivityDate="2021-05-24T19:51:54.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15978" PostTypeId="2" ParentId="15922" CreationDate="2021-05-24T23:08:17.203" Score="2" Body="&lt;p&gt;Slightly older than @gringer 's reference, but still from Richard Durbin himself&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Variation graph toolkit improves read mapping by representing genetic variation in the reference&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/30125266/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pubmed.ncbi.nlm.nih.gov/30125266/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12308" LastActivityDate="2021-05-24T23:08:17.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15979" PostTypeId="2" ParentId="15969" CreationDate="2021-05-25T03:12:08.787" Score="0" Body="&lt;p&gt;The usual way to avoid loops in linked lists is to keep a record of the things that have already been used / visited, and stop when an already visited thing has been seen again (or when there is nothing more to visit).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-25T03:12:08.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15981" PostTypeId="1" CreationDate="2021-05-25T07:44:29.523" Score="1" ViewCount="146" Body="&lt;p&gt;I have some transcriptomic (Whole) sequencing data that I should analyse. I would like to do raw data alignment to a reference genome taking into account the multi mapped reads and uniquely mapped reads. I would like to know the consequences on my transcriptomic analysis results : 1. if I should only focus on uniqueMapping reads (mapped to one loci) and neglect MultiMapping read (mapped to several loci or variants)! 2. or take them all together&lt;/p&gt;&#xA;&lt;p&gt;So my questions are :&lt;/p&gt;&#xA;&lt;p&gt;1- Considering unique and multi-aligned reads, wouldn't this be a source of bias in my results interpretation?&lt;/p&gt;&#xA;&lt;p&gt;2- By hiding the multi-mapped reads, could I lose information on the expression of important genes?&lt;/p&gt;&#xA;&lt;p&gt;thank you in advance&lt;/p&gt;&#xA;" OwnerUserId="6694" LastEditorUserId="73" LastEditDate="2021-05-27T11:47:16.860" LastActivityDate="2021-05-27T11:47:16.860" Title="RNA_Seq data aligned used uniquely or multi mapped reads impact on result interpretation" Tags="|rna-seq|sam|ngs|genomics|transcriptome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15982" PostTypeId="1" AcceptedAnswerId="15987" CreationDate="2021-05-25T10:03:51.027" Score="0" ViewCount="170" Body="&lt;p&gt;I am currently trying to understand how the traceback algorithm is supposed to work for the smith-waterman algorithm as my current understanding breaks down in case of a large alignment gap.&lt;/p&gt;&#xA;&lt;p&gt;Assume the sequences&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;3GA12CA&#xA;GA56CA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The obvious best alignment would be (or some other variation thereof)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;3GA12-CA&#xA;-GA--6CA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, assuming an affine gap penalty function of &lt;code&gt;x =&amp;gt; x + 2&lt;/code&gt; and a binary similarity score (i.e., if equal then 1 else -1) would lead to a scoring matrix&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    3 G A 1 2 C A&#xA;  0 0 0 0 0 0 0 0&#xA;G 0 0 1 0 0 0 0 1&#xA;A 0 0 0 2 0 0 0 0&#xA;6 0 0 0 0 0 0 0 0&#xA;C 0 0 0 0 0 0 1 0&#xA;A 0 0 0 1 0 0 0 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When doing the traceback I would start with the largest value towards the lower right corner of the matrix&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;(6,6)&lt;/li&gt;&#xA;&lt;li&gt;(5,5)&lt;/li&gt;&#xA;&lt;li&gt;Now I do not know how to find the next best alignment (in this example at (3,2)) as neither going up nor going to the left nor going diagonal is going to lead to a score greater 0 meaning I have no next best value to go on.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;How is the traceback function supposed to continue here?&lt;/p&gt;&#xA;&lt;p&gt;Additionally, I read that the traceback should start at the highest score in the matrix itself. However, if we would extend the first subsequence match length by one, this would mean we miss the right alignment option completely.&lt;/p&gt;&#xA;" OwnerUserId="12311" LastEditorUserId="12311" LastEditDate="2021-05-25T10:22:32.370" LastActivityDate="2021-05-26T09:25:03.057" Title="Smith-Waterman traceback with large gaps" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15983" PostTypeId="2" ParentId="15981" CreationDate="2021-05-25T15:15:18.960" Score="3" Body="&lt;p&gt;There are several methods/tools that take into account multi-mappers and deal with them in reasonable ways (e.g. pseudo-alignment methods like &lt;a href=&quot;https://combine-lab.github.io/salmon/&quot; rel=&quot;nofollow noreferrer&quot;&gt;salmon&lt;/a&gt; and more traditional alignment-based methods like &lt;a href=&quot;https://deweylab.github.io/RSEM/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RSEM&lt;/a&gt;). If you completely ignore multi-mappers, then yes, you will be losing information, which may or may not be valuable to you.&lt;/p&gt;&#xA;&lt;p&gt;Both of those methods utilize the &lt;a href=&quot;https://en.wikipedia.org/wiki/Expectation%E2%80%93maximization_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;expectation-maximization algorithm&lt;/a&gt; (or a variant thereof) to determine the relative contributions of multi-mapping reads to the various mapping locations. With salmon, you can forego alignment completely and end up with very accurate gene quantifications in hand in like 20 minutes if you want.&lt;/p&gt;&#xA;" OwnerUserId="1269" LastActivityDate="2021-05-25T15:15:18.960" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15984" PostTypeId="2" ParentId="15930" CreationDate="2021-05-25T17:48:51.877" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;For identification scoring, the software correlates signal&#xA;intensities of matched signals of mass spectra. The three scores&#xA;obtained from such a procedure are multiplied and normalized to a&#xA;value of 1,000 and its common logarithm. Log scores over 2 are&#xA;considered as reliable identification of a bacterial species (Table&#xA;4), whereas log scores over 1.7 generally indicate reliable&#xA;identification of bacterial genera. Log scores of 3 are obtained when&#xA;spectra are matched with themselves.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Sauer, S., Freiwald, A., Maier, T., Kube, M., Reinhardt, R., Kostrzewa, M., &amp;amp; Geider, K. (2008). Classification and identification of bacteria by mass spectrometry and&#xA;computational analysis. &lt;em&gt;PloS one, 3&lt;/em&gt;(7), e2843.&lt;/p&gt;&#xA;&lt;p&gt;Going further, from a random thread on ResearchGate:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.researchgate.net/post/Does_anyone_know_how_is_the_p-value_calculated_by_Bruker_Biotyper_during_MALDI-MS_fingerprinting_of_bacteria&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.researchgate.net/post/Does_anyone_know_how_is_the_p-value_calculated_by_Bruker_Biotyper_during_MALDI-MS_fingerprinting_of_bacteria&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Number of peaks matched in your unknown sample divided by the number&#xA;of peaks in the unknown sample, this is given a score (out of 10)&lt;/p&gt;&#xA;&lt;p&gt;Number of peaks matched in your unknown sample divided by the number&#xA;of peaks in the reference sample, this is given a score (out of 10)&lt;/p&gt;&#xA;&lt;p&gt;Score of relative intensities of matching peaks (out of 10)&lt;/p&gt;&#xA;&lt;p&gt;Total score out of 1000&lt;/p&gt;&#xA;&lt;p&gt;Log10 score out of 3&lt;/p&gt;&#xA;&lt;p&gt;There is no relationship of the score with a p-value nor you can&#xA;calculate the statistics because the bruker database is not open and&#xA;you can not know the values for what the score was calculated to do&#xA;the statistical analysis&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="12245" LastActivityDate="2021-05-25T17:48:51.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15985" PostTypeId="2" ParentId="15969" CreationDate="2021-05-26T01:22:23.790" Score="1" Body="&lt;p&gt;You will want to create a self loop for a &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-mer &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; whenever &lt;span class=&quot;math-container&quot;&gt;$s_1,...,s_{k-1} = s_2,...,s_k$&lt;/span&gt;. This will only happen when all of the characters of the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt;-mer are identical, as &lt;span class=&quot;math-container&quot;&gt;$s_1=s_2, s_2=s_3,...,s_{k-1}=s_k$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6628" LastActivityDate="2021-05-26T01:22:23.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15986" PostTypeId="2" ParentId="15958" CreationDate="2021-05-26T04:38:06.793" Score="2" Body="&lt;p&gt;I have solved this problem in the past by &lt;a href=&quot;https://www.nextflow.io/docs/latest/faq.html#how-do-i-use-the-same-channel-multiple-times&quot; rel=&quot;nofollow noreferrer&quot;&gt;duplicating the output channel&lt;/a&gt; of the desired process in order to feed the same output into multiple downstream processes.&lt;/p&gt;&#xA;" OwnerUserId="6628" LastActivityDate="2021-05-26T04:38:06.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15987" PostTypeId="2" ParentId="15982" CreationDate="2021-05-26T09:25:03.057" Score="1" Body="&lt;p&gt;This question is based on a misunderstanding. The Smith-Waterman algorithm calculates the best &lt;code&gt;local&lt;/code&gt; segment. You have to start at the &lt;span class=&quot;math-container&quot;&gt;$max(H_{i,j})$&lt;/span&gt; point and run the traceback as described. It will (most likely) return only a local segment that, however, has the maximum value in terms of alignment score.&lt;/p&gt;&#xA;&lt;p&gt;To get a &lt;code&gt;global&lt;/code&gt; alignment, another algorithm has to be used. Needleman-Wunsch would be an example.&lt;/p&gt;&#xA;&lt;p&gt;The difference between &lt;code&gt;local&lt;/code&gt; and &lt;code&gt;global&lt;/code&gt; is that a &lt;code&gt;local&lt;/code&gt; alignment only alignes a subsequence of the provided sequences whereas a &lt;code&gt;global&lt;/code&gt; alignment gives the best alignment of the whole sequences.&lt;/p&gt;&#xA;" OwnerUserId="12311" LastActivityDate="2021-05-26T09:25:03.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15989" PostTypeId="1" CreationDate="2021-05-26T13:13:27.633" Score="2" ViewCount="103" Body="&lt;p&gt;I'm trying to download PubMed Central Open Access articles from &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_bulk&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_bulk&lt;/a&gt;. However, I am not able to decompress the archives without errors.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;tar&lt;/code&gt; always gives me the error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gzip: data stream error&#xA;tar: Unexpected EOF in archive&#xA;tar: Error is not recoverable: exiting now&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does PubMed provide corrupted archives or does downloading corrupt the archive?&lt;/p&gt;&#xA;" OwnerUserId="12328" LastActivityDate="2021-06-03T05:25:33.113" Title="PubMed Central Open Access archives are corrupted" Tags="|public-databases|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="15990" PostTypeId="1" CreationDate="2021-05-26T14:27:27.817" Score="2" ViewCount="56" Body="&lt;p&gt;I have a fastq file from Nanopore sequencing data that contains reads from &lt;strong&gt;both&lt;/strong&gt; the UK and South Africa variants of SARS-CoV-2.&lt;/p&gt;&#xA;&lt;p&gt;The variants are identified by three key mutations in the receptor-binding domain of Spike.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Aim&lt;/strong&gt;&#xA;Firstly, I want to determine the number of reads then belongs to UK and South Africa variant. Then, I will create a bam file from the fastq file using the SARS-CoV-2 reference genome.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How do I determine the reads belonging to each variants?&lt;/li&gt;&#xA;&lt;li&gt;Are there any tools available to classify the reads based on the pre-defined mutation criteria?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1684" LastEditorUserId="3848" LastEditDate="2022-02-25T21:28:19.920" LastActivityDate="2022-02-25T21:28:19.920" Title="Detecting SARS-CoV-2 variants from the mixed virus population" Tags="|sam|variant-calling|nanopore|covid-19|sars-cov-2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15991" PostTypeId="1" AcceptedAnswerId="16016" CreationDate="2021-05-26T14:30:33.070" Score="2" ViewCount="75" Body="&lt;p&gt;How long does it take for the latest Bioconductor to end up on Conda? I think I saw a blog post about the challenges with this process, but I cannot find it. For example, Bioconductor 3.13 was released a week ago, but is still not available. For comparison, many CRAN packages tend to show up within a day.&lt;/p&gt;&#xA;&lt;p&gt;Additionally, is there a way to keep track of that progress?&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="35" LastEditDate="2021-05-26T15:39:28.600" LastActivityDate="2021-05-31T14:41:32.393" Title="Bioconductor Conda release delay" Tags="|bioconductor|conda|bioconda|anaconda|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="15992" PostTypeId="2" ParentId="15991" CreationDate="2021-05-26T15:15:38.047" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/users/77/devon-ryan&quot;&gt;Devon&lt;/a&gt; will probably add a more precise answer, until then I just link his Tweet from last week:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://twitter.com/dpryan79/status/1394714988720308226&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://twitter.com/dpryan79/status/1394714988720308226&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Edit: 31.5.21&lt;/p&gt;&#xA;&lt;p&gt;Rollout started yesterday, should take a week to complete:&lt;br /&gt;&#xA;&lt;a href=&quot;https://twitter.com/dpryan79/status/1399273322601459712&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://twitter.com/dpryan79/status/1399273322601459712&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2021-05-31T08:14:29.267" LastActivityDate="2021-05-31T08:14:29.267" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15994" PostTypeId="1" CreationDate="2021-05-26T15:57:41.840" Score="1" ViewCount="57" Body="&lt;p&gt;The following&#xA;(thx to &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2634/retrieve-detailed-gene-descriptions&quot;&gt;these answers&lt;/a&gt;)&#xA;retrieves the info, in particular a long summary (such as &amp;quot;The membrane-associated protein encoded by this gene is a member of the superfamily of ATP-binding cassette... etc.&amp;quot;), for a few mouse genes.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(dplyr)&#xA;&#xA;org.Mm.eg.db::org.Mm.egSYMBOL2EG %&amp;gt;% &#xA;  as.data.frame() %&amp;gt;% &#xA;  pull(gene_id) %&amp;gt;% &#xA;  head() %&amp;gt;%  # !!!&#xA;  rentrez::entrez_summary(id = ., db = &amp;quot;gene&amp;quot;) %&amp;gt;%&#xA;  { do.call(&amp;quot;rbind&amp;quot;, .) } %&amp;gt;%&#xA;  as.data.frame() %&amp;gt;%&#xA;  select(name, description, summary)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note the &lt;code&gt;head&lt;/code&gt;. The protocol/server does not allow large queries. How can I get the same information (incl. the long summary) for all the mouse genes at once?&lt;/p&gt;&#xA;" OwnerUserId="12324" LastActivityDate="2021-06-02T14:00:57.233" Title="Long-form gene summary" Tags="|r|phylogenetics|entrez|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15995" PostTypeId="2" ParentId="7401" CreationDate="2021-05-26T16:10:40.643" Score="3" Body="&lt;p&gt;I had the same problem as you, and so I wrote my own code and sharing it here. No guarantee that it's bug-free ;-)&lt;/p&gt;&#xA;&lt;p&gt;This code takes advantage of the cigar string. It returns None if the base has been deleted or clipped, or if the bam file did not store the sequence.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from typing import Optional&#xA;import pysam&#xA;&#xA;&#xA;&#xA;def find_base_in_alignment(alignment: pysam.AlignedSegment,&#xA;                           pos: int, &#xA;                           bam_stores_revcomp: bool = False) \ &#xA;        -&amp;gt; Optional[str]:&#xA;    idx_q = 0&#xA;    idx_r = pos - alignment.reference_start&#xA;    if bam_stores_revcomp:&#xA;        seq = alignment.query_sequence&#xA;    else:&#xA;        seq = alignment.get_forward_sequence()&#xA;    &#xA;    if seq is None:&#xA;        return None&#xA;    &#xA;    for op, l in alignment.cigartuples:&#xA;        ref_consumed = op in {0, 2, 3, 7, 8}&#xA;        query_consumed = op in {0, 1, 4, 7, 8}&#xA;        &#xA;        if ref_consumed:&#xA;            idx_r -= l&#xA;        if query_consumed:&#xA;            idx_q += l&#xA;        &#xA;        if idx_r &amp;lt; 0:&#xA;            if query_consumed:&#xA;                # base is in query between idx_q-l , idx_q&#xA;                base = seq[idx_q + idx_r - 1]&#xA;                return base&#xA;            else:&#xA;                # position has been deleted&#xA;                return None&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Let's say you want to genotype all reads by looking at the base at chromosome &lt;code&gt;chrom&lt;/code&gt; and position &lt;code&gt;pos&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bamfile = pysam.AlignmentFile(filename, &amp;quot;r&amp;quot; )&#xA;for alignment in bamfile.fetch(chrom, pos-1, pos+1):&#xA;    base = find_base_in_alignment(alignment, pos)&#xA;    # ... do something with it&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that the parameter &lt;code&gt;bam_stores_revcomp&lt;/code&gt; is important. Some aligners will store the original query sequence in the bam, and other aligners (like minimap2) will store the forward sequence (i.e. the reverse complement whenever a read was aligned to the reverse strand). If you use the former, set &lt;code&gt;bam_stores_revcomp&lt;/code&gt; to &lt;code&gt;False&lt;/code&gt;, otherwise set it to &lt;code&gt;True&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="12344" LastEditorUserId="12344" LastEditDate="2021-05-27T09:13:49.007" LastActivityDate="2021-05-27T09:13:49.007" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15996" PostTypeId="2" ParentId="15990" CreationDate="2021-05-27T12:07:44.807" Score="2" Body="&lt;p&gt;Based on a quick request from someone else, I wrote some code a couple of years ago for assigning read groups based on variants at particular locations; this was set up for HIV variant splitting, so I expect it should also work on SARS-CoV-2 nanopore reads as well:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://gitlab.com/gringer/bioinfscripts/-/blob/master/samVarSplitter.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gitlab.com/gringer/bioinfscripts/-/blob/master/samVarSplitter.pl&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It processes a SAM stream [i.e. in text mode] using Perl, so might be a bit slow, but should be fine for a few hundred thousand sequences. You'd use it as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h mapped_reads.bam | \&#xA;  samVarSplitter.pl [-ref &amp;lt;refName&amp;gt;] [-pos &amp;lt;int&amp;gt;] | \&#xA;  samtools sort &amp;gt; grouped_reads.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then the reads can be split into different files based on their group using &lt;code&gt;samtools split&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Usage: samtools split [-u &amp;lt;unaccounted.bam&amp;gt;] [-h &amp;lt;unaccounted_header.sam&amp;gt;]&#xA;                      [-f &amp;lt;format_string&amp;gt;] [-v] &amp;lt;merged.bam&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-27T12:07:44.807" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="15997" PostTypeId="1" AcceptedAnswerId="16000" CreationDate="2021-05-27T12:26:27.403" Score="0" ViewCount="693" Body="&lt;p&gt;I have a 1000 genomes vcf file for chromosome 14. And I want to remove variants in repetitive regions flagged by repeatMasker. I have a bed file for those repeats. I downloaded that file from UCSC. How can I eliminate/remove those sites in vcf file using repeat masker's bed file? Insights will be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="12351" LastActivityDate="2021-06-10T20:56:06.057" Title="Remove repetitive region vcf file using repeatmasker bed file" Tags="|vcf|snp|bedtools|repeatmasker|variants|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ClosedDate="2021-06-11T02:39:27.727" ContentLicense="CC BY-SA 4.0" />
  <row Id="15998" PostTypeId="2" ParentId="15997" CreationDate="2021-05-27T13:19:59.010" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://vcftools.github.io/man_latest.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;vcftools&lt;/code&gt;&lt;/a&gt; with the &lt;code&gt;--exclude-bed&lt;/code&gt; option or &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;bedtools&lt;/code&gt;&lt;/a&gt; with the &lt;code&gt;subtract&lt;/code&gt; command should do the trick.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2021-05-27T13:19:59.010" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="15999" PostTypeId="2" ParentId="15972" CreationDate="2021-05-27T15:09:22.177" Score="2" Body="&lt;p&gt;I think a useful tool to address your question is &lt;a href=&quot;https://www.well.ox.ac.uk/%7Egav/qctool_v2/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;QCTOOL v2&lt;/a&gt;. Particularly, the option you are looking for is &lt;a href=&quot;https://www.well.ox.ac.uk/%7Egav/qctool_v2/documentation/alphabetical_options.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;-annotate-genetic-map&lt;/a&gt;. This tool has been developed originally for GWAS datasets. It has &lt;a href=&quot;https://www.well.ox.ac.uk/%7Egav/qctool_v2/documentation/genotype_file_formats.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;multiple inputs&lt;/a&gt; data format.&lt;/p&gt;&#xA;&lt;p&gt;If you could provide input data as an example, I will try working on it to provide the necessary steps to be done.&lt;/p&gt;&#xA;&lt;p&gt;Hope this answers your question.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="4413" LastEditDate="2021-05-27T15:21:42.967" LastActivityDate="2021-05-27T15:21:42.967" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16000" PostTypeId="2" ParentId="15997" CreationDate="2021-05-28T06:13:29.053" Score="2" Body="&lt;p&gt;A couple fast alternatives:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ bedops --chrom chr14 -n 1 &amp;lt;(vcf2bed &amp;lt; variants.vcf) &amp;lt;(sort-bed repeats.bed) &amp;gt; answer.bed&#xA;$&lt;/span&gt; bedops --chrom chr14 -n 1 &amp;lt;(vcf2bed &amp;lt; variants.vcf) &amp;lt;(rmsk2bed &amp;lt; repeats.rmsk) &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Using &lt;code&gt;--chrom chr14&lt;/code&gt; focuses set operations on that chromosome.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2021-05-28T06:13:29.053" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16001" PostTypeId="1" AcceptedAnswerId="16002" CreationDate="2021-05-28T08:44:31.450" Score="1" ViewCount="506" Body="&lt;p&gt;Where or How can I find contigs? I am trying to learn Bioinformatics and I want to make a reference-based gene search. I also want to align contigs with a given reference sequence. From NCBI other databases I can find only reads and complete genome sequences. Please tell me how can I find contig sequences of a species?&lt;/p&gt;&#xA;" OwnerUserId="12357" LastEditorUserId="57" LastEditDate="2021-05-28T10:09:41.213" LastActivityDate="2021-05-28T14:15:30.533" Title="where to find sample contig data" Tags="|sequence-alignment|gene|assembly|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16002" PostTypeId="2" ParentId="16001" CreationDate="2021-05-28T10:18:14.093" Score="2" Body="&lt;p&gt;Most of the reference genomes on NCBI are made of contigs or scaffolds (which is just a fancy name for contigs glued together by a bunch of unknown nucleotides). Hence, by download the whole genome, you also download all the contigs.&lt;/p&gt;&#xA;&lt;p&gt;If you search for a specific contig (presumably a known locus) of a specific species, you can search NCBI for the species name + locus / scaffold / contig name and if it was sequenced it will pop up in the &amp;quot;genes&amp;quot; section of NCBI. For example, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene/112206411&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is a search for &lt;em&gt;Pan troglodites&lt;/em&gt; contig NC_036898.1 (I typed literally Pan troglodites NC_036898.1 in the search bar). You can even just type a species name, and click on &amp;quot;genes&amp;quot; and see loads of all the individual contigs sequenced.&lt;/p&gt;&#xA;&lt;p&gt;--- edit ---&lt;/p&gt;&#xA;&lt;p&gt;There is a bunch of ways to generate something we could call contig. However, usually what we mean is &amp;quot;continuous sequence assembled from sequencing reads&amp;quot; and it's generated from whole-genome sequencing reads. We sequence a genome, assemble the reads and what we get is a set of contigs as a representation of a haploid reference genome. Some of them will be short fragments of nucleotides with unknown functions, some of them can be long and contain hundreds of genes. The important part is that it's a continuous sequence.&lt;/p&gt;&#xA;&lt;p&gt;With a bit more relaxed definition of &amp;quot;contig&amp;quot;, we could also accept as contigs outputs of targeted sequencing (including old-fashioned sanger). It's a continuous stretch of nucleotides, usually bordered by a pair of primers. I am not 100% sure, but my best guess that most of the &amp;quot;gene&amp;quot; sequences in NCBI will be sanger-based (but I should disclaim I never investigated too much where all the NCBI sequences come from).&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2021-05-28T14:15:30.533" LastActivityDate="2021-05-28T14:15:30.533" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16003" PostTypeId="1" CreationDate="2021-05-28T13:45:01.043" Score="1" ViewCount="900" Body="&lt;p&gt;I need to mask all sites in a vcf file flagged by the 1000 Genomes Project as being unfit for population genetic analyses. The sites for all chromosomes are available at:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000_genomes_project/working/20160622_genome_mask_GRCh38/PilotMask/&quot; rel=&quot;nofollow noreferrer&quot;&gt;1000Genomes masked sites&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;From above mentioned link you can get both bed files and Fasta files.&lt;/p&gt;&#xA;&lt;p&gt;I wanted to mask all sites using VCFtools but VCFtools is not longer being updated and therefore, can't me used on the my vcf file because you can only VCFtools upto VCF file format v.2. And file format for my file is v.3. Can someone suggest another tool that I can use. Insights will be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="12351" LastActivityDate="2021-06-15T01:11:29.730" Title="Masking sites in a vcf file" Tags="|vcf|bedtools|vcftools|bcftools|1000genomes|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16004" PostTypeId="1" CreationDate="2021-05-28T15:45:21.933" Score="2" ViewCount="98" Body="&lt;p&gt;To be specific, we have different lineages of E. coli. I wondered if the ortholog genes among them should all have the same name in all lineages or can they also have different names?&lt;/p&gt;&#xA;" OwnerUserId="7444" LastActivityDate="2021-06-28T18:00:45.110" Title="Can ortholog genes have different names among different species?" Tags="|phylogenetics|sequence-homology|orthologues|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16006" PostTypeId="2" ParentId="15976" CreationDate="2021-05-28T19:56:13.810" Score="3" Body="&lt;p&gt;There is a &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5921990/bin/12885_2018_4388_MOESM2_ESM.csv&quot; rel=&quot;nofollow noreferrer&quot;&gt;CSV table&lt;/a&gt; in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5921990/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; with 33 sets of genes.&lt;/p&gt;&#xA;" OwnerUserId="12324" LastActivityDate="2021-05-28T19:56:13.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16007" PostTypeId="2" ParentId="16003" CreationDate="2021-05-29T10:46:49.000" Score="0" Body="&lt;p&gt;Have a look at &lt;a href=&quot;https://samtools.github.io/bcftools/bcftools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BCFtools&lt;/a&gt;, which I think does most of the stuff that VCFtools does. I notice there's a &lt;code&gt;--mask&lt;/code&gt; option in &lt;code&gt;BCFtools consensus&lt;/code&gt;, which may do what you want:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-m, --mask FILE&#xA; BED file or TAB file with regions to be replaced with N&#xA; (the default) or as specified by the next --mask-with&#xA; option. See discussion of --regions-file in Common&#xA; Options for file format details&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-05-29T10:46:49.000" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16008" PostTypeId="2" ParentId="16004" CreationDate="2021-05-29T17:53:34.710" Score="3" Body="&lt;p&gt;Gene naming is pretty subjective and even a single gene can have many names. If you're lucky orthologs will be named the same in different lineages, but you can't rely on it. Better to use gene ids (e.g. from Ensembl) and an &lt;a href=&quot;https://questfororthologs.org/orthology_databases&quot; rel=&quot;nofollow noreferrer&quot;&gt;ortholog db&lt;/a&gt; to map the orthologs. However, if you're looking at different strains, this is more tricky since most db don't go below the species level&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2021-05-29T17:53:34.710" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16009" PostTypeId="1" AcceptedAnswerId="16069" CreationDate="2021-05-29T21:49:17.747" Score="4" ViewCount="166" Body="&lt;p&gt;I'm coding a pipeline where I make a lot of pairwise alignments, and I end up with raw alignment scores. But, I really need to look at my results in terms of bit scores.&lt;/p&gt;&#xA;&lt;p&gt;I know that the formula is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$′ = (ƛ* − ()) / (2)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;But, I don't know what &lt;code&gt;ƛ&lt;/code&gt; or &lt;code&gt;K&lt;/code&gt; would be. I'm currently using &lt;strong&gt;BLOSUM62&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I've looked around online for how to look up (and even empirically deduce) lambda and k, and I all I've come across were convoluted explanations.&lt;/p&gt;&#xA;&lt;p&gt;I also know that this &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/5145/calculating-bit-score-how-do-you-find-lambda-and-k&quot;&gt;question&lt;/a&gt; from almost 3 years ago attempts to address the same topic, but the best response only leads to an NCBI link that requires login credentials.&lt;/p&gt;&#xA;&lt;p&gt;Any help would be much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="11917" LastEditorUserId="11917" LastEditDate="2021-06-05T16:41:14.367" LastActivityDate="2021-06-07T01:06:43.017" Title="How do you convert Raw Alignment Score to Bit Score?" Tags="|python|sequence-alignment|bitscore|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16010" PostTypeId="1" CreationDate="2021-05-30T13:36:19.787" Score="1" ViewCount="210" Body="&lt;p&gt;I am using Bowtie to remove non-coding RNAs (tRNA, snRNA, rRNA) by rfam and also maping our microRNA-seq data with mirbase using following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bowtie -m 10  -l 18 -n 1 -v 2 -e 80 --best --strata -a   genome.index.file    inputfastqfile   --un unaligned.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Unfortunately I dont have any references or index for that code.&#xA;Are there any other settings and options to improve this code?&lt;/p&gt;&#xA;" OwnerUserId="12371" LastEditorUserId="3541" LastEditDate="2021-05-30T13:45:14.297" LastActivityDate="2021-05-30T13:45:14.297" Title="using Bowtie to map miRNA-seq data to rfam and also reference genome" Tags="|genomics|read-mapping|bowtie2|non-coding|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16011" PostTypeId="1" AcceptedAnswerId="16030" CreationDate="2021-05-30T13:40:24.740" Score="2" ViewCount="529" Body="&lt;p&gt;I'm trying to use Entrez (through Biopython) to download the sequence of a TMV replicase gene.&lt;/p&gt;&#xA;&lt;p&gt;I have the following code:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;# Lookup ID&#xA;search = Entrez.esearch(db='gene', term='Tobacco mosaic virus[Orgn] AND replicase')&#xA;read = Entrez.read(search)&#xA;idlist = read[&amp;quot;IdList&amp;quot;]&#xA;&#xA;# Fetch sequence&#xA;search = Entrez.efetch(db='gene', id=idlist[0], retmode='text', rettype='gb')&#xA;read = SeqIO.read(search, &amp;quot;genbank&amp;quot;)&#xA;tmv_replicase = read.seq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But when I run it it throws the exception on the &lt;code&gt;SeqIO.read&lt;/code&gt; line: &lt;code&gt;ValueError: No records found in handle&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The IDs provided in idlist are: &lt;code&gt;['1494081', '1494082']&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="12370" LastEditorUserId="3848" LastEditDate="2022-03-08T20:55:35.450" LastActivityDate="2022-03-08T20:55:35.450" Title="Entrez (Biopython) esearch and efetch not returning sequence as expected" Tags="|python|covid-19|genbank|entrez|virus|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16012" PostTypeId="1" AcceptedAnswerId="16017" CreationDate="2021-05-30T14:03:37.817" Score="0" ViewCount="482" Body="&lt;p&gt;I would like to calculate the rmsd between re-docked complex and co-crystallized complex for docking validation. How do I calculate the rmsd using Pymol? I tried the following command&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;align re-docked_complex, co-crystallized_complex&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But the above command gives the rmsd of two proteins. I would like to get the rmsd of re-docked and co-crystallized ligand. How can I achieve this using pymol?&lt;/p&gt;&#xA;" OwnerUserId="12372" LastEditorUserId="3541" LastEditDate="2021-05-31T07:19:14.110" LastActivityDate="2021-05-31T15:13:14.757" Title="rmsd between re-docked complex and co-crystallized complex" Tags="|docking|pymol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16013" PostTypeId="2" ParentId="15880" CreationDate="2021-05-30T18:26:35.473" Score="0" Body="&lt;p&gt;I don't think that the output of &lt;code&gt;gmx dump&lt;/code&gt; can be read by VMD: the goal of this command is to provide a human-readable output, not to convert one trajectory format into another.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;trj&lt;/code&gt; files are &lt;strong&gt;NOT&lt;/strong&gt; the output of &lt;code&gt;gmx dump&lt;/code&gt;. &lt;code&gt;trj&lt;/code&gt; files were used in &amp;quot;old&amp;quot; versions of GROMACS to store trajectories (see &lt;a href=&quot;https://manual.gromacs.org/archive/5.0/online/trj.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;trj file format&lt;/a&gt; documentation of GROMACS 5), and this is what VMD is awaiting, not the output of &lt;code&gt;gmx dump&lt;/code&gt;. This is also why it tells you the header is wrong.&lt;/p&gt;&#xA;&lt;p&gt;In recent versions of GROMACS &lt;code&gt;trj&lt;/code&gt; files are no longer supported, probably as they were not in a binary format, hence took too much disk space (see &lt;a href=&quot;https://manual.gromacs.org/current/reference-manual/file-formats.html#trajectory-files&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trajectory files formats&lt;/a&gt; in GROMACS 2021 documentation). Converting a &lt;code&gt;.trr&lt;/code&gt; in a &lt;code&gt;.trj&lt;/code&gt; was done by &lt;code&gt;gmx trjconv&lt;/code&gt;, but I don't think this is possible anymore (see &lt;code&gt;gmx trjconv&lt;/code&gt; supported trajectory formats in &lt;a href=&quot;https://manual.gromacs.org/archive/5.0.3/programs/gmx-trjconv.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GROMACS 5&lt;/a&gt; vs &lt;a href=&quot;https://manual.gromacs.org/current/onlinehelp/gmx-trjconv.html?highlight=trjconv#options&quot; rel=&quot;nofollow noreferrer&quot;&gt;GROMACS 2021&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Why would you absolutely need &lt;code&gt;gmx dump&lt;/code&gt; output to be read by VMD? The only reason I can see is a handmade script doing modifications to your trajectory. In this case I would recommend to instead look into some molecular dynamics python libraries, like &lt;a href=&quot;https://www.mdanalysis.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MDAnalysis&lt;/a&gt;, &lt;a href=&quot;http://amber-md.github.io/pytraj/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyTraj&lt;/a&gt; or &lt;a href=&quot;https://mdtraj.org/1.9.4/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MDTraj&lt;/a&gt;, which will also allow you to make minute modifications to the trajectory file while still reading it in its original format. You can also look harder into GROMACS tools, as they are quite complete.&lt;/p&gt;&#xA;" OwnerUserId="12054" LastActivityDate="2021-05-30T18:26:35.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16014" PostTypeId="2" ParentId="13581" CreationDate="2021-05-30T19:02:46.603" Score="0" Body="&lt;p&gt;I found my answer &lt;a href=&quot;https://www.biostars.org/p/444909/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. It uses &lt;code&gt;getGeneLengthAndGCContent&lt;/code&gt; but changes a small part of the function.&lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2021-05-30T19:02:46.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16016" PostTypeId="2" ParentId="15991" CreationDate="2021-05-31T14:41:32.393" Score="4" Body="&lt;p&gt;I started the roll-out on the 30th in the evening. There's not much of a delay for the October Bioconductor release, but for the summer release we first have to build the newest R version and then rebuild all conda CRAN packages. It's probably not widely known, but Bioconductor packages are only compatible with a single R version. For example, the ones being released now are not compatible with R 4.0 and will not be compatible with R 4.2, but will instead only ever work with R 4.1. This is regardless of what the metadata provided by Bioconductor package authors might say, the Bioconductor core team has requested this (there's actually not even a guarantee that packages built under R 4.1.0 will work with 4.1.1, but such issues are exceedingly rare). This is why there is a multi-week delay every summer.&lt;/p&gt;&#xA;&lt;p&gt;On the plus side, at least we're not waiting for the first bug-fix release of R to start the migration anymore.&lt;/p&gt;&#xA;&lt;p&gt;For Bioconda we've never gotten around to making a migration tracker. If you have a local clone of Bioconda, you can run &lt;code&gt;./scripts/bioconductor/rootNodes.py --sinceNDays 2 --rootNodes&lt;/code&gt; to get a count of the packages completed in the past 2 days and a list of the root nodes on the dependency DAG that still need to be handled. That's how I estimate the fraction finished (currently about 600 packages).&lt;/p&gt;&#xA;&lt;p&gt;For tracking the status on conda-forge you can see their status page: &lt;a href=&quot;https://conda-forge.org/status/#r410&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://conda-forge.org/status/#r410&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-05-31T14:41:32.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16017" PostTypeId="2" ParentId="16012" CreationDate="2021-05-31T15:13:14.757" Score="1" Body="&lt;p&gt;Welcome.&lt;/p&gt;&#xA;&lt;p&gt;The Protein-Ligand-complex is the complete protein with Ligand!&lt;/p&gt;&#xA;&lt;p&gt;If you only want to compare the two ligands, you can delete the rest of the proteins, or specify the input so only the ligand is choosen. (select Ligand and save selection. then use the function on that selections)&lt;/p&gt;&#xA;&lt;p&gt;But be aware that for Docking comparison you probably don't want to align! but rather calculate the RMSD directly (because the absolute position and not the relative position is of interest?)&lt;/p&gt;&#xA;&lt;p&gt;Maybe use: &amp;quot;rms_cur&amp;quot; (&lt;a href=&quot;https://pymolwiki.org/index.php/Rms_cur&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pymolwiki.org/index.php/Rms_cur&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="12181" LastActivityDate="2021-05-31T15:13:14.757" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16018" PostTypeId="1" CreationDate="2021-05-31T15:39:12.683" Score="2" ViewCount="196" Body="&lt;p&gt;I am generating a consensus sequence for SARS-Cov-2. I am visually inspecting every candidate SNP in a pileup.&lt;/p&gt;&#xA;&lt;p&gt;Sometimes I find that there are two consistent variants for the same position, but I can only generate one consensus sequence.&lt;/p&gt;&#xA;&lt;p&gt;Which variant should I target:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The majority variant&lt;/li&gt;&#xA;&lt;li&gt;The minoritarian one if I find that this SNP is under-reported in other public samples?&lt;/li&gt;&#xA;&lt;li&gt;Are both required&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;What one is best practice?&lt;/p&gt;&#xA;" OwnerUserId="5190" LastEditorUserId="3848" LastEditDate="2022-02-26T23:07:24.750" LastActivityDate="2022-02-26T23:07:24.750" Title="Consensus sequence from vcf file" Tags="|vcf|variant-calling|covid-19|sars-cov-2|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16019" PostTypeId="1" CreationDate="2021-05-31T16:05:50.907" Score="0" ViewCount="58" Body="&lt;p&gt;I couldn't find any information on this online. Which of the following sequencing technologies are considered &lt;strong&gt;short read&lt;/strong&gt; technologies? Illumina, Oxford Nanopore, Ion Torrent, Pacific Biosciences&lt;/p&gt;&#xA;" OwnerUserId="12382" LastEditorUserId="57" LastEditDate="2021-06-02T11:28:35.837" LastActivityDate="2021-06-02T11:28:35.837" Title="Which sequencing technologies are considered short read" Tags="|ngs|homework|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16020" PostTypeId="1" CreationDate="2021-05-31T16:32:46.670" Score="0" ViewCount="71" Body="&lt;p&gt;I wanted information regarding which format is considered the most raw from an illumina sequencer ? &lt;Br&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;fasta&lt;/li&gt;&#xA;&lt;li&gt;fastq&lt;/li&gt;&#xA;&lt;li&gt;bcl&lt;/li&gt;&#xA;&lt;li&gt;bam &lt;br&gt;&#xA;As per my research it should be &lt;strong&gt;bcl&lt;/strong&gt; but I am not sure.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="12383" LastActivityDate="2021-06-01T08:08:48.397" Title="Which format is the most raw format" Tags="|ngs|file-formats|illumina|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16022" PostTypeId="2" ParentId="16018" CreationDate="2021-06-01T06:05:10.320" Score="4" Body="&lt;p&gt;Making a consensus sequence tends to involve some arbitrary decisions. In this case, the common practices are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use the ancestral allele (this isn't always apparent)&lt;/li&gt;&#xA;&lt;li&gt;Use the most common allele&lt;/li&gt;&#xA;&lt;li&gt;Randomly choose&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I think 1 and 2 are generally preferred, with 2 being easier to implement.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-06-01T06:05:10.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16023" PostTypeId="2" ParentId="16020" CreationDate="2021-06-01T08:08:48.397" Score="1" Body="&lt;p&gt;BCL (binary base call) files are the most raw data that you will likely get. As a matter of fact, the images taken by the machine each cycle are even more raw but it makes no sense to interact with this data directly. The base calls are derived from counting the spots (clusters) on the images.&lt;/p&gt;&#xA;&lt;p&gt;For any useful application you would convert them to fastq (e.g. using illumina &lt;code&gt;bcl2fastq&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;What is the goal of your question? Do you suspect some problems with your data?&lt;br /&gt;&#xA;Adding this information will make it easier to interpret what kind of answer you are looking for - and less likely that your question is downvoted.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2021-06-01T08:08:48.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16024" PostTypeId="1" AcceptedAnswerId="16025" CreationDate="2021-06-01T10:18:28.350" Score="2" ViewCount="300" Body="&lt;p&gt;I have some RNA-seq data with two very obvious batches as you can see in the PCA:&lt;br /&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Mv1r7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Mv1r7.png&quot; alt=&quot;PCA prior to normalization&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The samples of interest (A - H) are from tumor tissues. In addition, there are data for two cell lines I and J. The cell lines should be the same across the batches, while for the tumor samples there may be some additional biological difference.&lt;/p&gt;&#xA;&lt;p&gt;My question is, if there is a way to use the cell line data to estimate the technical part of the batch information since there should very little biological difference here. Then I would like to use this to correct the technical aspect of the batch effect in the tumor samples.&lt;/p&gt;&#xA;&lt;p&gt;When I run normal batch correction like &lt;code&gt;ComBat&lt;/code&gt; or add the batch to the DE model the batch will be estimated over all samples including biological differences.&lt;/p&gt;&#xA;&lt;h2&gt;Update&lt;/h2&gt;&#xA;&lt;p&gt;I figured I could determine which genes are differentially expressed between the batches in the cell lines only. As mentioned before, the cell lines are highly similar biologically. Thus these DE genes should represent the technical aspect of the difference.&lt;/p&gt;&#xA;&lt;p&gt;The plot below shows the PCA when after removing genes DE in the cell lines. Obviously, the differences for the cell lines are greatly reduced. Also the tumor samples are now less clearly separated.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/crxYc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/crxYc.png&quot; alt=&quot;PCA after normalization&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Most of the removed genes were from shorter RNA species and a look at sequencing QC later confirmed that in fact the fragment size in the second batch was lower. Therefore I'm positive that I could identify and remove the technical component of the batch effect.&lt;/p&gt;&#xA;&lt;h2&gt;tl;dr&lt;/h2&gt;&#xA;&lt;p&gt;My data contains a strong difference between batches even for samples that are biologically the same (untreated cell lines). I looked at differentially expressed genes in the cell lines. This revealed to me that the second batch contained a lot more short RNAs due to lower fragment size of the library.&lt;/p&gt;&#xA;&lt;p&gt;In the end I'm still wondering if there is an elegant solution for this requiring less footwork&lt;/p&gt;&#xA;" OwnerUserId="4885" LastEditorUserId="4885" LastEditDate="2021-06-15T16:18:11.890" LastActivityDate="2021-06-15T16:18:11.890" Title="Batch effect correction using a subset of samples - using DE genes" Tags="|rna-seq|batch-effects|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16025" PostTypeId="2" ParentId="16024" CreationDate="2021-06-01T10:57:23.417" Score="1" Body="&lt;p&gt;The batch effect is expected to be the same across all samples - therefore you want to estimate its effect across all of the samples. Just add batch as an additional parameter to the linear model (this is presuming samples sequenced in the two batches have been randomised).&lt;/p&gt;&#xA;&lt;p&gt;You can show the effect of &amp;quot;batch effect&amp;quot; removal with a PCA plot by plotting the residual error, e.g. in R if using DESeq:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mypca &amp;lt;-  prcomp(t(assay(varianceStabilizingTransformation(dds)))&#xA;pc.resid &amp;lt;- resid(aov(mypca$x~batch.info))&#xA;ggplot(pc.resid,aes(x=PC1,y=PC2) + geom_point() &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(I haven't checked the code, but it will be something like this).&lt;/p&gt;&#xA;" OwnerUserId="8919" LastActivityDate="2021-06-01T10:57:23.417" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16027" PostTypeId="2" ParentId="15994" CreationDate="2021-06-01T13:32:05.930" Score="1" Body="&lt;p&gt;You could wrap the call too &lt;code&gt;entrez_summary&lt;/code&gt; in a loop.&lt;br /&gt;&#xA;For example to cut the data into chunks of five genes:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gene.vector # this would be the genes you are interested in&#xA;gene.chunks &amp;lt;- ceiling(length(gene.vector) / 5)&#xA;# initializing an empty list&#xA;summary.list &amp;lt;- vector(mode = &amp;quot;list&amp;quot;, length = gene.chunks)&#xA;for(i in gene.chunks){&#xA;   # select the ith set of five genes from your gene vector&#xA;   goi &amp;lt;- gene.vector[(i - 1) * 5 + 1:5]&#xA;   # if there are less than five elements there will be NAs that need to be removed&#xA;   goi &amp;lt;- goi[!is.na(goi)]&#xA;   # adding your code &#xA;   summary.list[[i]] &amp;lt;- org.Mm.eg.db::org.Mm.egSYMBOL2EG %&amp;gt;% &#xA;     as.data.frame() %&amp;gt;% &#xA;     filter(gene_id %in% goi) %&amp;gt;% # here the current five genes are selected&#xA;     pull(gene_id) %&amp;gt;% &#xA;     rentrez::entrez_summary(id = ., db = &amp;quot;gene&amp;quot;) %&amp;gt;%&#xA;     { do.call(&amp;quot;rbind&amp;quot;, .) } %&amp;gt;%&#xA;     as.data.frame() %&amp;gt;%&#xA;     select(name, description, summary)&#xA;  &#xA;   Sys.sleep(10) # wait for 10 seconds&#xA;}&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There will also be a way to do this using &lt;code&gt;dplyr&lt;/code&gt; &lt;code&gt;group&lt;/code&gt;. In this case the loop is a little easier to debug.&lt;/p&gt;&#xA;&lt;p&gt;I added &lt;code&gt;Sys.sleep&lt;/code&gt; at the end to reduce the frequency with which you are making requests from entrez. If you are posting too many requests in a short time this will be like a DDOS attack. You (and potentially your whole IP range including other people at your institution) could get banned from the service for a couple of hours. Maybe try it with a subset first.&lt;/p&gt;&#xA;&lt;p&gt;If you really want to get the information for all mouse genes this probably will be rather slow. Alternatively, you can go to the entrez webpage and try to download the information as a list. Unfortunately, I don't know exactly where you will find this information.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2021-06-01T13:32:05.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16030" PostTypeId="2" ParentId="16011" CreationDate="2021-06-02T09:23:23.740" Score="2" Body="&lt;p&gt;Did you try to fetch from the database &lt;code&gt;nucleotide&lt;/code&gt; instead of &lt;code&gt;gene&lt;/code&gt; ?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# load modules&#xA;from Bio import SeqIO&#xA;from Bio import Entrez&#xA;&#xA;# Lookup ID&#xA;search = Entrez.esearch(db='nucleotide', term='Tobacco mosaic virus[Orgn] AND replicase')&#xA;read = Entrez.read(search)&#xA;idlist = read[&amp;quot;IdList&amp;quot;]&#xA;&#xA;# Fetch sequence&#xA;search = Entrez.efetch(db='nucleotide', id=idlist[0], retmode='text', rettype='gb')&#xA;read = SeqIO.read(search, &amp;quot;genbank&amp;quot;)&#xA;tmv_replicase = read.seq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It returns&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; read&#xA;SeqRecord(seq=Seq('GTATTTTTACAACAATTACCAACAACAACAAACAACAAACAACATTACAATTAC...GGG'), id='MN912489.1', name='MN912489', description='Tobacco mosaic virus isolate TMV:BR:SC:02:01, complete genome', dbxrefs=[])&#xA;&amp;gt;&amp;gt;&amp;gt; tmv_replicase&#xA;Seq('GTATTTTTACAACAATTACCAACAACAACAAACAACAAACAACATTACAATTAC...GGG')&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4992" LastActivityDate="2021-06-02T09:23:23.740" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16031" PostTypeId="2" ParentId="16019" CreationDate="2021-06-02T11:26:38.497" Score="1" Body="&lt;p&gt;As mentioned by Greg, list of short-read sequencing platforms is &lt;a href=&quot;https://en.wikipedia.org/wiki/DNA_sequencing#Short-read_sequencing_methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;listed on wikipedia&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Of those you mentioned Illumina and Ion Torrent are short-read sequencing platforms.&lt;/p&gt;&#xA;&lt;p&gt;Usually what makes short reads short is a PCR step in the library prep or directly on the flowcell to amplify the signal, which is the limitation for how long stretches of DNA are possible to sequence. Long read sequencing technologies sequence directly isolated single molecules.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2021-06-02T11:26:38.497" CommentCount="0" CommunityOwnedDate="2021-06-02T11:26:38.497" ContentLicense="CC BY-SA 4.0" />
  <row Id="16033" PostTypeId="2" ParentId="15994" CreationDate="2021-06-02T14:00:57.233" Score="1" Body="&lt;p&gt;The following bash script downloads the summaries into an XML file.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env bash&#xA;&#xA;# This requires the ncbi tool `gene2xml`&#xA;# sudo apt install ncbi-tools-bin&#xA;&#xA;DIR=&amp;quot;download_cache&amp;quot;&#xA;mkdir -p &amp;quot;$DIR&amp;quot;&#xA;&#xA;URL=&amp;quot;ftp://ftp.ncbi.nlm.nih.gov/gene/DATA/ASN_BINARY/Mammalia/Mus_musculus.ags.gz&amp;quot;&#xA;wget -S -O - &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$URL&quot; | gzip -d | gene2xml -b T -l T | gzip -9 &amp;gt; &quot;$&lt;/span&gt;DIR/mm.xml.gz&amp;quot;&#xA;&#xA;URL=&amp;quot;ftp://ftp.ncbi.nlm.nih.gov/gene/DATA/ASN_BINARY/Mammalia/Homo_sapiens.ags.gz&amp;quot;&#xA;wget -S -O - &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$URL&quot; | gzip -d | gene2xml -b T -l T | gzip -9 &amp;gt; &quot;$&lt;/span&gt;DIR/hs.xml.gz&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These XML files can be converted to tables along &lt;a href=&quot;https://github.com/hsiaoyi0504/gene_dictionary/blob/4d84c5c4e4acb6103c6d22f8cc0e101e9535ab85/extract.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;these lines&lt;/a&gt;, viz.:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;&#xA;import xml.etree.ElementTree as ET&#xA;import gzip&#xA;&#xA;from pathlib import Path&#xA;&#xA;from tcga.utils import mkdir, unlist1, relpath&#xA;&#xA;&#xA;def parse(src):&#xA;    def find_text(e, path):&#xA;        e = e.findall(path)&#xA;        if e:&#xA;            return unlist1(e).text&#xA;        else:&#xA;            return &amp;quot;&amp;quot;&#xA;&#xA;    yield ('gene_id', 'description', 'summary')&#xA;&#xA;    with gzip.open(src, mode='r') as fd:&#xA;        e: ET.Element&#xA;        for (x, e) in ET.iterparse(fd, events=['end']):&#xA;            if e.tag == &amp;quot;Entrezgene&amp;quot;:&#xA;                gene_id = find_text(e, &amp;quot;**/Gene-track_geneid&amp;quot;)&#xA;                descptn = find_text(e, &amp;quot;**/Gene-ref_desc&amp;quot;)&#xA;                summary = find_text(e, &amp;quot;Entrezgene_summary&amp;quot;)&#xA;&#xA;                yield (gene_id, descptn, summary)&#xA;&#xA;                e.clear()&#xA;&#xA;&#xA;def main():&#xA;    for src in Path(__file__).parent.glob(&amp;quot;download_cache/*.xml.gz&amp;quot;):&#xA;        trg = mkdir(Path(__file__).with_suffix('')) / Path(src.stem).with_suffix('.tsv.gz')&#xA;        if trg.is_file():&#xA;            print(&amp;quot;Skipping existing file:&amp;quot;, relpath(trg))&#xA;        else:&#xA;            print(&amp;quot;Writing:&amp;quot;, relpath(trg))&#xA;            with gzip.open(trg, mode='wt') as fd:&#xA;                for t in parse(src):&#xA;                    print(*t, sep='\t', file=fd)&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Btw., a table of vertebrate gene homology is available &lt;a href=&quot;http://www.informatics.jax.org/downloads/reports/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (it's just a tab-separated table), and can be used to impute summaries by homology.&lt;/p&gt;&#xA;" OwnerUserId="12324" LastActivityDate="2021-06-02T14:00:57.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16035" PostTypeId="1" AcceptedAnswerId="16040" CreationDate="2021-06-02T19:12:21.217" Score="0" ViewCount="115" Body="&lt;p&gt;I want to do differential expression analysis with DESEQ2. I have three read counts files downloaded from GEO (small RNAseq based) where the number of miRNAs and id is nearly the same. These studies have used the same tool for alignment but different tools for counting aligned reads.&lt;/p&gt;&#xA;&lt;p&gt;My question is&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Can I combine these read count files&lt;/li&gt;&#xA;&lt;li&gt;Is there any recommended meta-analysis based tool for this purpose&lt;/li&gt;&#xA;&lt;li&gt;Is there any other way to do this other then redoing RNA-seq analysis.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="4783" LastActivityDate="2021-06-03T07:34:50.160" Title="Combining read counts from three separate GEO studies" Tags="|rna-seq|reads|meta-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16038" PostTypeId="2" ParentId="14990" CreationDate="2021-06-03T06:50:14.753" Score="2" Body="&lt;p&gt;I had the same error and I solved by removing &lt;code&gt;defined&lt;/code&gt; and writing only &lt;code&gt;if (%$rref)&lt;/code&gt; in the mummerplot code. You have to do this for all the lines that you have it. There should be six lines to correct in this way .&lt;/p&gt;&#xA;&lt;p&gt;You can find an explanation of why &lt;a href=&quot;https://perldoc.perl.org/functions/defined&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, that tells that the function &lt;code&gt;defined&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Returns a Boolean value telling whether EXPR has a value other than the undefined value undef. If EXPR is not present, $_ is checked.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And then:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Use of &lt;code&gt;defined&lt;/code&gt; on aggregates (hashes and arrays) is no longer supported. It used to report whether memory for that aggregate had ever been allocated. You should instead use a simple test for size:&#xA;&lt;code&gt;if (%a_hash)   { print &amp;quot;has hash members\n&amp;quot;   } &lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="10594" LastEditorUserId="10594" LastEditDate="2021-06-03T12:13:50.243" LastActivityDate="2021-06-03T12:13:50.243" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16039" PostTypeId="1" AcceptedAnswerId="16260" CreationDate="2021-06-03T07:01:07.367" Score="2" ViewCount="290" Body="&lt;p&gt;I am doing genome sequence alignment using MUMmer, in particular I want to do a dotplot with mummerplot. So the passages that I did are:&lt;/p&gt;&#xA;&lt;p&gt;1.create a file .mums with the following command line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mummer -mum -b -c H_pylori26695_Eslice.fasta H_pyloriJ99_Eslice.fasta &amp;gt; mummer.mums&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;2.and then plot all the MUMs with MUMmerplot:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mummerplot -x &amp;quot;[0,275287]&amp;quot; -y &amp;quot;[0,265111]&amp;quot; -postscript -p mummer mummer.mums&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The example is taken &lt;a href=&quot;https://mummer4.github.io/tutorial/tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The plot works, but I would like to understand several things to have more consciousness of what I am doing: the file .mums has three columns with many rows. What do they represent ? I know that it is a little explained in the link given before:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This command will find all maximal unique matches (-mum) between the reference and query on both the forward and reverse strands (-b) and report all the match positions relative to the forward strand (-c). Output is to stdout, so we will redirect it into a file named mummer.mums. This file lists all of the MUMs of the default length or greater between the two input sequences.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So the file should report all the Maximal Unique Matches between the two sequences but I do not understand what are these three columns...&lt;/p&gt;&#xA;&lt;p&gt;Moreover, can someone explain in simple words the reason of the term &amp;quot;unique&amp;quot; in &amp;quot;Maximal Unique Matches&amp;quot; ?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance .&lt;/p&gt;&#xA;" OwnerUserId="10594" LastActivityDate="2021-06-29T17:11:17.577" Title="What is the file .mums created by mummer?" Tags="|sequence-alignment|genome|phylogenetics|file-formats|software-usage|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16040" PostTypeId="2" ParentId="16035" CreationDate="2021-06-03T07:34:50.160" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;Yes, but there will be a batch effect.&lt;/li&gt;&#xA;&lt;li&gt;Not that I know if, you'll need to use sva or something along those lines to handle the batch effect.&lt;/li&gt;&#xA;&lt;li&gt;Crossing your fingers and hoping the batch effect is small is pretty much your only other option if the data isn't setup in a way that you can use sva or similar. So yeah, redoing the experiment may be the only real option.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-06-03T07:34:50.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16041" PostTypeId="1" AcceptedAnswerId="16042" CreationDate="2021-06-03T09:23:22.653" Score="2" ViewCount="43" Body="&lt;p&gt;And thanks in advance. I am debugging a text file which is the one the two inputs on the program RGmatch &lt;a href=&quot;https://bitbucket.org/pfurio/rgmatch/src/master/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bitbucket.org/pfurio/rgmatch/src/master/&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The code works. Is that some records give division by zero.&lt;/p&gt;&#xA;&lt;p&gt;I am doing it by hand.&lt;/p&gt;&#xA;&lt;p&gt;But is there a way by making a bash script, to see which records show the problem?&lt;/p&gt;&#xA;&lt;p&gt;I do not want to touch the python code.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;python rgmatch.py -g ../../hg38.ncbiRefSeq.gtf -b ../../test.bed -o myassoSort.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have tried with classical debugging, but my actual question is how to automatize that?&lt;/p&gt;&#xA;&lt;p&gt;Also there are no empty records. The bed file was created based on refseq.txt from the latest human assembly. And also the GTF file. On the refseq.txt I processed to obtain a bed file.&lt;/p&gt;&#xA;&lt;p&gt;I have checked if this works on a single record, it does, but not in some specific ones. The error I get is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  File &amp;quot;rgmatch.py&amp;quot;, line 1311, in &amp;lt;module&amp;gt;&#xA;    main()&#xA;  File &amp;quot;rgmatch.py&amp;quot;, line 287, in main&#xA;    run(gtf, dhs, outputfile)&#xA;  File &amp;quot;rgmatch.py&amp;quot;, line 1153, in run&#xA;    pctg_area      = (float(region_overlap)/intron_length)*100&#xA;ZeroDivisionError: float division by zero&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The record I found so far is this one.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;chr19    14087850    14090751&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I did grep on both GTF files and Refseq Files and in both files there is information about the record. Because the program search for areas between region of interest, If I put inside just the record only in GTF file and bed Files no error occurs.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;grep &amp;quot;14090751&amp;quot; refseq.bed&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;692  NM_138352.3 chr19   -   14087850    14090751    14088416    14090420    6   14087850,14088670,14088879,14089067,14089784,14090084,  14088575,14088782,14088976,14089362,14090084,14090751,  0   SAMD1   cmpl    cmpl    0,2,1,0,0,0,&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;grep &amp;quot;14090751&amp;quot; hg38.ncbiRefSeq.gtf&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr19   ncbiRefSeq.2021-02-10   exon    14090085    14090751    .   -   .   gene_id &amp;quot;SAMD1&amp;quot;; transcript_id &amp;quot;NM_138352.3&amp;quot;; exon_number &amp;quot;1&amp;quot;; exon_id &amp;quot;NM_138352.3.1&amp;quot;; gene_name &amp;quot;SAMD1&amp;quot;;&#xA;chr19   ncbiRefSeq.2021-02-10   5UTR    14090421    14090751    .   -   .   gene_id &amp;quot;SAMD1&amp;quot;; transcript_id &amp;quot;NM_138352.3&amp;quot;; exon_number &amp;quot;1&amp;quot;; exon_id &amp;quot;NM_138352.3.1&amp;quot;; gene_name &amp;quot;SAMD1&amp;quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10768" LastEditorUserId="10768" LastEditDate="2021-06-03T10:11:23.700" LastActivityDate="2021-06-03T10:11:23.700" Title="Automatize classical debugging in a bash script? (normal debugging by hand)" Tags="|python|bash|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16042" PostTypeId="2" ParentId="16041" CreationDate="2021-06-03T09:45:44.573" Score="1" Body="&lt;p&gt;If you want to just start incrementally adding lines from the bed file until you hit the one that reproduces the problem, you can do something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in &lt;span class=&quot;math-container&quot;&gt;$(seq 20 30 $&lt;/span&gt;(wc -l &amp;lt; ../../test.bed)); do&#xA;    head -n &lt;span class=&quot;math-container&quot;&gt;$i ../../test.bed &amp;gt; slice.bed&#xA;    if python rgmatch.py -g ../../hg38.ncbiRefSeq.gtf -b slice.bed -o slice.out&#xA;    then&#xA;        true&#xA;    else&#xA;        echo &quot;Failed when running with the $&lt;/span&gt;i first lines.&amp;quot;&#xA;        break&#xA;    fi&#xA;done&#xA;    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Explanation&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;seq 20 30 $(wc -l &amp;lt; ../../test.bed)&lt;/code&gt; will iterate over all numbers between 20 and the number of lines in your bed file with increments of 30. Next, the &lt;code&gt;head -n $i ../../test.bed &amp;gt; slice.bed&lt;/code&gt; will take the first &lt;code&gt;$i&lt;/code&gt; lines of the bed file. This way, the loop will run the &lt;code&gt;rgmatch&lt;/code&gt; command on the first 20 lines of the file, then the first 50, then the first 80 and so on as long as the command runs successfully. When the &lt;code&gt;rgmatch&lt;/code&gt; command fails, it will report the value of &lt;code&gt;$i&lt;/code&gt; to help you find the line that caused the problem.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2021-06-03T09:45:44.573" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16045" PostTypeId="1" CreationDate="2021-06-03T10:41:22.000" Score="1" ViewCount="68" Body="&lt;p&gt;I am battling the submission of scaffolded assembly to EBI, somehow I can not get AGP gap file right.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;background&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;The only option seems to be using the &lt;a href=&quot;https://ena-docs.readthedocs.io/en/latest/submit/assembly.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Webin-CLI submission interface&lt;/a&gt;. For the scaffolded assembly submission, &lt;a href=&quot;https://ena-docs.readthedocs.io/en/latest/submit/assembly/genome.html#scaffold-assembly&quot; rel=&quot;nofollow noreferrer&quot;&gt;one needs&lt;/a&gt; the manifest file, the sequence (fasta or flat file), and AGP file, which denotes how contigs are glued together into scaffolds. I got already that the sequence file should contain only contigs, and EBI will glue them together using the AGP file.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;the problem&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;So far so good, I checked &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly/agp/AGP_Specification/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the specification&lt;/a&gt; and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly/agp/AGP_Specification/scaffold_from_contig_WGS.agp.v2.0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the example provided&lt;/a&gt; and generated similarly looking files for my assembly, but then when I tried to validate the files using&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ena-webin-cli -context genome -userName XXX -password XXX -manifest manifest_file -outputDir asm01_submission -inputDir ena_submission_input -ascp -validate&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I got tons of errors with type&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ERROR: Scaffold &amp;quot;NODE_667656&amp;quot; has only &amp;quot;1&amp;quot; component, minimum two components expected.&#xA;ERROR: Scaffold &amp;quot;NODE_353389&amp;quot; has only &amp;quot;1&amp;quot; component, minimum two components expected.&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;supposedly for each contig that actually was not scaffolded. I am quite confused about this, as the example file contains tons of scaffold with one component only (for example the first one &lt;code&gt;EG1_scaffold1&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;What am I missing?&lt;/p&gt;&#xA;&lt;p&gt;Sample of my AGP file (incl 1 scaffold and 3 contigs):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Path_2003       1       1672    1       W       NODE_32694_length_1672_cov_36.024451    1       1672    +&#xA;Path_2003       1673    1772    2       U       100     scaffold        yes     align_trnscpt&#xA;Path_2003       1773    2521    3       W       NODE_61726_length_749_cov_25.233631     1       749     +&#xA;Path_2003       2522    2621    4       U       100     scaffold        yes     align_trnscpt&#xA;Path_2003       2622    5705    5       W       NODE_20424_length_3084_cov_26.290988    1       3084    +&#xA;NODE_667656     1       86      1       W       NODE_667656_length_86_cov_136.111111    1       86      +&#xA;NODE_353389     1       155     1       W       NODE_353389_length_155_cov_25.230769    1       155     +&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="57" LastActivityDate="2021-06-03T22:00:36.213" Title="How to specify the list of contigs in AGP gap file used for ENA submissions" Tags="|database|public-databases|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16047" PostTypeId="1" CreationDate="2021-06-03T13:32:34.090" Score="1" ViewCount="24" Body="&lt;p&gt;I have a very basic and quick question: does the complete sequence of nucleotides of an organism (for example &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NC_003413.1?report=fasta&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; one that I am considering in my homework of Pyrococcus Fuoriosus) reported in NCBI represent the sequence of nucleotides of the forward strand (5' to 3') of DNA?&lt;/p&gt;&#xA;" OwnerUserId="10594" LastEditorUserId="57" LastEditDate="2021-06-03T21:31:49.060" LastActivityDate="2021-06-03T21:31:49.060" Title="How to interpret complete genome sequence in NCBI" Tags="|database|phylogenetics|sequence|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16048" PostTypeId="1" CreationDate="2021-06-03T14:13:01.553" Score="1" ViewCount="12" Body="&lt;p&gt;I am a bioinformatics beginner.&#xA;I want to draw line graphs such that for a gene, say gene_A, intronic regions are (20,100), (500,600) and a conserved domain was found in (400-550), then I should get base position/sequence length on the x axis and gene name on the y axis.&#xA;Is there any software that can help me do this?&lt;/p&gt;&#xA;&lt;p&gt;I used matplotlib but would like better visuals, any suggestions would be helpful. Thanks!&#xA;&lt;a href=&quot;https://i.stack.imgur.com/JeIqD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JeIqD.png&quot; alt=&quot;Sample result from Matplotlib&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12409" LastActivityDate="2021-06-03T14:13:01.553" Title="How to graph sequence ranges, for example intron sequences or conserved domains" Tags="|sequence|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16049" PostTypeId="1" CreationDate="2021-06-03T14:30:50.673" Score="1" ViewCount="205" Body="&lt;p&gt;I would like to ask how to overcome the problem of mismatched boxes and labels (pic). Is there an option focused on this?&#xA;thanks&lt;/p&gt;&#xA;&lt;p&gt;Radim&lt;a href=&quot;https://i.stack.imgur.com/Dor4l.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Dor4l.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12412" LastActivityDate="2021-06-04T11:00:58.520" Title="Label and labelbox mismatched in Pymol after ray" Tags="|visualization|pymol|3d-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16050" PostTypeId="1" CreationDate="2021-06-03T17:45:56.357" Score="0" ViewCount="79" Body="&lt;p&gt;I'm attempting to come up with a statistical model to determine the p-value for the similarity value &lt;span class=&quot;math-container&quot;&gt;$ 0 \le s \le 1 $&lt;/span&gt; of two DNA contigs.&#xA;The idea is to get a numerical measure to estimate if two contigs are homologous.&lt;/p&gt;&#xA;&lt;p&gt;I want to account for base composition (and/or other structural features) if possible, but not necessarily because I might want to do the analysis only from the information about the length of the sequences/contigs and largest overlaps. Maybe something inspired from other &lt;a href=&quot;https://academic.oup.com/bib/article/7/1/2/262762&quot; rel=&quot;nofollow noreferrer&quot;&gt;statistical significance in bioinformatics &lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Assume we have two DNA sequences: &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; with the basepair lengths &lt;span class=&quot;math-container&quot;&gt;$a_{\text{len}}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$b_{\text{len}}$&lt;/span&gt;. Their overlap &lt;span class=&quot;math-container&quot;&gt;$o_{a-b}$&lt;/span&gt; is the number of contiguous bases that appears in both sequences. If we define the similarity &lt;span class=&quot;math-container&quot;&gt;$s = \frac{o_{a-b}}{\min{(a_{\text{len}}, b_{\text{len}})}}$&lt;/span&gt; how can we define a p-value for &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2021-06-08T06:06:46.273" LastActivityDate="2021-06-08T06:06:46.273" Title="How to compute a p-value for the similarity measurement of two DNA sequences/contigs?" Tags="|sequence-analysis|phylogenetics|p-values|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16051" PostTypeId="1" CreationDate="2021-06-03T18:07:50.187" Score="1" ViewCount="855" Body="&lt;p&gt;I am very new to micro RNA analysis. I have been using H. sapiens, GRCh38 + major index as given in the Bowtie Website to align with my trimmed FASTQ file .&#xA;The command I am using to make very sensitive alignments with Bowtie2 is given below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;module load  Bowtie2; &#xA;bowtie2 --local -p 8 -q --phred33 -D 20 -R 3 -N 0 -L 8 \&#xA;    -i S,1,0.50 -x grch38_1kgmaj \&#xA;    -U /gpfs/ycga/scratch60/nicoli/ag2646/HumanmicroRNAbowtie_verysensitive_1.06.2021./3kPa_HDF_miRNA_1bbduk.fastq \&#xA;    -S /gpfs/ycga/scratch60/nicoli/ag2646/HumanmicroRNAbowtie_verysensitive_1.06.2021./3kPa_HDF_miRNA_1bbduk.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then I am using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools sort -bS 3kPa_HDF_miRNA_1bbduk.sam &amp;gt; 3kPa_HDF_miRNA_1bbduk.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to convert sam files into bam files.&lt;/p&gt;&#xA;&lt;p&gt;However, I am hitting the error described below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[W::sam_read1_sam] Parse error at line 46371&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I grepped the specific line it shows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SRR8248790.46371        4       *       0       0       *       *       0       0       CCTGGATGATGATAAGCAAATGCTGACTGAACATGAAGGTCTTAATTAGCTCTAACTGACTAGATCGGAAGAGCAC   BBADAGEHIFHHIIIIIGGFIHIIIIIIIHHHHHIHHHEHEHEHFEHIIIEHIIIIIEHIIHIIIHGHHHIIII1C   YT:Z:UU&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any specific help troubleshooting this error will be very useful.&lt;/p&gt;&#xA;" OwnerUserId="9659" LastEditorUserId="298" LastEditDate="2021-06-03T18:34:13.433" LastActivityDate="2021-06-03T18:34:13.433" Title="Conversion of SAM to BAM files" Tags="|samtools|genomics|format-conversion|bowtie2|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="16052" PostTypeId="2" ParentId="16045" CreationDate="2021-06-03T22:00:36.213" Score="1" Body="&lt;p&gt;Oh you silly sausage! Don't use AGP gap files.&lt;/p&gt;&#xA;&lt;p&gt;This is not a direct answer but a workaround. However, if your scaffolded genome is annotated, you might as well submit a flat file (.embl) instead of fasta and the gap file. The thing is, making that file is actually a lot more straightforward thanks to amazing &lt;a href=&quot;https://github.com/NBISweden/EMBLmyGFF3&quot; rel=&quot;nofollow noreferrer&quot;&gt;EMBLmyGFF3&lt;/a&gt;. You will also need the locus tag (a string used as prefix for loci in your genome) you must pre-register in the web interface (it's among &amp;quot;Project&amp;quot; options). Note that although the script does handle some EBI specifics (like automatic filtering of gff3 tags they don't like, or filtering sequences &amp;lt; 100 bases), some there are things that need to be taken care of. For example, I had to remove duplicates (using &lt;code&gt;agat_sp_fix_features_locations_duplicated.pl&lt;/code&gt;) and overlapping genes (using &lt;code&gt;agat_sp_fix_overlaping_genes.pl&lt;/code&gt;), both available in &lt;a href=&quot;https://github.com/NBISweden/AGAT&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGAT collection&lt;/a&gt;. This might be different for genomes annotated with a different tool (mine was &lt;code&gt;braker2&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Anyway, once I was advised to use this tool, it took like 2 hours for 400Mbp genome including the uploading time, so I certainly recommend it over wrestling with AGP files.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2021-06-03T22:00:36.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16055" PostTypeId="2" ParentId="16049" CreationDate="2021-06-04T11:00:58.520" Score="1" Body="&lt;p&gt;It is a glitch in PyMOL.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Caveat&lt;/strong&gt;. I would say that, whereas &lt;code&gt;cmd.label&lt;/code&gt; is great for adding labels en-mass for internal figures, it is not great for figures for dissemination (which require few &lt;em&gt;strategically&lt;/em&gt; placed labels possibly with a faint white outer glow —cf. your D330): most figures in papers I would say are Powerpoint or Photoshop labelled.&lt;/p&gt;&#xA;&lt;p&gt;You called it 'labelbox', this is not a normal term, so you may be using a custom script. The box is normally called a &lt;code&gt;bounding box&lt;/code&gt; and &lt;code&gt;connector line&lt;/code&gt;. To add it you need to set a few values. You may have done the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;label name CA and resi 44, 'Foo'&#xA;set label_outline_color, 0xff8c69  # font colour to salmon&#xA;set label_connector_color, 0xff8c69 # connect or bounding box&#xA;set label_bg_outline, true  # adds a border (lowercase bool)&#xA;set label_connector, true   # adds a line&#xA;set label_connector_mode, 1 # how the line looks label_position&#xA;set label_position, [5, 5, 5]  # offset the label&#xA;set ray_label_connector_flat, true # cylinders as line? No thanks.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The issue is the &lt;code&gt;set label_position, [5, 5, 5]&lt;/code&gt;. If you give the camera-based reference vector &lt;code&gt;[0,0,5]&lt;/code&gt; you get no line as Z is depth. This ray traces fine. Otherwise the glitch you found happens. My guess is that the box (but not the label) gets calculated based on the viewport window size, not the ray trace window size. So my guess is that there is a resolution and ray trace window size combination just right that they do align... but on my machine at least this is low res.&lt;/p&gt;&#xA;&lt;p&gt;To make matters worse, the shadow is horrendous and in a random location and &lt;code&gt;label_shadow_mode&lt;/code&gt; does not stop it. Let's not even discuss &lt;code&gt;ray_label_connector_flat&lt;/code&gt;. So the best way to avoid this is to simply label in Photoshop, which is much more convenient anyway.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-06-04T11:00:58.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16056" PostTypeId="1" CreationDate="2021-06-04T15:25:48.673" Score="1" ViewCount="221" Body="&lt;p&gt;I am currently working with fastq files which have hundreds of thousands of lines of text. However, not all of them are sequences I am interested in. My sequences are in one line and have a fixed length of 272 characters.&lt;/p&gt;&#xA;&lt;p&gt;I possess a list of smaller sequences, around 2 million text strings in the following formats (which are fragments from the various fastq files I have):&lt;/p&gt;&#xA;&lt;p&gt;Format 1&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATCGATCG&#xA;ATCGATCGATCG&#xA;ACGATCGTGTTACG&#xA;ATTTACGTACGTA&#xA;AATCG&#xA;ACGATACGATACG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or I also have them in&#xA;Format 2&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATCGATCGATACGAT|ATACGTGTTACGAT|ATCGATACG|ATACGATGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(I just wrote random sequences just to give an idea of how the pattern.txt files look like, I know Format 1 and 2 are not the same in this post, however, the files are in the same order - only the syntax is changed.).&lt;/p&gt;&#xA;&lt;p&gt;What I've been trying to do is use grep in a Linux environment and use the pattern.txt files to search for the bigger sequences in my fastq files where the pattern is present, and then output it into a .txt file to further work on.&lt;/p&gt;&#xA;&lt;p&gt;I've used this command so far (with the Format 2 of patterns):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -E -f patterns.txt target.fastq &amp;gt; output.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Now, this works.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;However,&lt;/strong&gt; I have to limit the number of patterns in my key file and the number of fastq files I can search at once, otherwise I get &amp;quot;grep: memory exhausted&amp;quot; error.&lt;/p&gt;&#xA;&lt;p&gt;I also want to print out all the sequences up to and including my pattern, but not past it (I don't care what's after my pattern sequence).&lt;/p&gt;&#xA;&lt;p&gt;And for that I have &lt;strong&gt;successfully&lt;/strong&gt; used the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -oE &amp;quot;\S+singlepattern&amp;quot; target.fastq &amp;gt; output.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, I can only manage to get this to work if I manually write in the command the pattern sequence I want to look for (thus I can only use a couple). I can't seem to get it to work with the file list.&lt;/p&gt;&#xA;&lt;p&gt;So I thought of using a while loop.&#xA;This is what I've written so far in the terminal (it's one continuous line, but I formatted it here for simplicity on the eyes):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;while read line; do&#xA;grep $line target.fastq &amp;gt; output.txt; &#xA;done &amp;lt; patterns.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;However,&lt;/strong&gt; this does absolutely nothing and just spits out a blank output.txt&lt;/p&gt;&#xA;&lt;p&gt;I have tried changing the syntax, with quotations on arguments, no quotations, I tried the various -E -f -o -F, egrep, etc but it's all the same. No error message, just blank file.&lt;/p&gt;&#xA;&lt;p&gt;I've also tried implementing the \S to the loop as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;while read line; do&#xA;grep -oE &amp;quot;\S+$line&amp;quot; target.fastq &amp;gt; output.txt; &#xA;done &amp;lt; patterns.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Yet, the result is the same.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestions on how I can fix this? Or even other methods to get the cut-down sequences (up to and including my pattern) from the fastq files into an output.txt file?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="12419" LastEditorUserId="73" LastEditDate="2021-06-07T00:59:48.373" LastActivityDate="2021-06-07T01:01:58.577" Title="Finding smaller sequences from within larger sequences" Tags="|fastq|linux|grep|" AnswerCount="3" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="16057" PostTypeId="2" ParentId="16056" CreationDate="2021-06-04T15:50:27.347" Score="0" Body="&lt;p&gt;The reason your loop is giving a blank file is because you are using &lt;code&gt;&amp;gt;&lt;/code&gt; which will overwrite whatever was previously in the file. This means that your &lt;code&gt;output.txt&lt;/code&gt; will only have the result of the last &lt;code&gt;grep&lt;/code&gt; and if that last &lt;code&gt;grep&lt;/code&gt; has no results, you'll get an empty file.&lt;/p&gt;&#xA;&lt;p&gt;Now, you're on the right track here. You could use a loop, although it will be horribly, horribly slow (by the way, you can use this exact format in the terminal as well, it doesn't need to be on the same line):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;## create output.txt if it doesn't exist and empty it if it does exist&#xA;&amp;gt; output.txt &#xA;&#xA;while read line; do&#xA;    ## Use &amp;gt;&amp;gt; instead of &amp;gt; so you append to the file&#xA;    grep -oE &amp;quot;\S+$line&amp;quot; target.fastq &amp;gt;&amp;gt; output.txt; &#xA;done &amp;lt; patterns.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Your attempt with the &lt;code&gt;-f&lt;/code&gt; option is the most reasonable. You might get it to work if you tell &lt;code&gt;grep&lt;/code&gt; to treat the input as fixed strings and not regular expressions (&lt;code&gt;-F&lt;/code&gt;). Also, you don't need the &lt;code&gt;-E&lt;/code&gt; (and that might be causing issues), so try this in case it works:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -Ff patterns.txt target.fastq &amp;gt; output.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If that doesn't work, just split your &lt;code&gt;patterns.txt&lt;/code&gt; file into smaller files and then use grep in a loop with each of them:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;split -l300000 patterns.txt pattern_chunk&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That will split &lt;code&gt;patterns.txt&lt;/code&gt; into multiple files of 3000000 lines each, named &lt;code&gt;pattern_chunkaa&lt;/code&gt;, &lt;code&gt;pattern_chunkab&lt;/code&gt; etc. For example, this is the result of running that command on a file with 5 million lines:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;$ wc -l patterns.txt &#xA;5000000 patterns.txt&#xA;$ split -l300000 patterns.txt pattern_chunk&#xA;$ ls&#xA;pattern_chunkaa  pattern_chunkad  pattern_chunkag  pattern_chunkaj  pattern_chunkam  pattern_chunkap&#xA;pattern_chunkab  pattern_chunkae  pattern_chunkah  pattern_chunkak  pattern_chunkan  pattern_chunkaq&#xA;pattern_chunkac  pattern_chunkaf  pattern_chunkai  pattern_chunkal  pattern_chunkao  patterns.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, you can now use each of those as the patterns:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;## create output.txt if it doesn't exist and empty it if it does exist&#xA;&amp;gt; output.txt &#xA;for chunk in pattern_chunk*; do&#xA;    grep -f &amp;quot;$chunk&amp;quot; target.fastq &amp;gt;&amp;gt; output.txt; &#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This doesn't help you get the sequence only up until the matched pattern. For that, you can add the &lt;code&gt;\S+&lt;/code&gt; to the beginning of each pattern in &lt;code&gt;patterns.txt&lt;/code&gt; (this assumes you are using the file which has one pattern per line) and then split the file and loop over the chunks again:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;## add the \S+&#xA;sed -i 's/^/\\S+/' patterns.txt&#xA;## create output.txt if it doesn't exist and empty it if it does exist&#xA;&amp;gt; output.txt &#xA;&#xA;for chunk in pattern_chunk*; do&#xA;    grep -Ef &amp;quot;$chunk&amp;quot; target.fastq &amp;gt;&amp;gt; output.txt; &#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note how I keep using &lt;code&gt;&amp;gt; output.txt&lt;/code&gt;. This is not actually necessary, you can use &lt;code&gt;command &amp;gt; file&lt;/code&gt; even if &lt;code&gt;file&lt;/code&gt; doesn't exist. However, it is useful here to make sure that the &lt;code&gt;output.txt&lt;/code&gt; file doesn't have the result of a previous run since we are now appending (&lt;code&gt;&amp;gt;&amp;gt;&lt;/code&gt;) instead of overwriting (&lt;code&gt;&amp;gt;&lt;/code&gt;) its contents.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2021-06-04T16:03:57.147" LastActivityDate="2021-06-04T16:03:57.147" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16058" PostTypeId="1" CreationDate="2021-06-05T09:37:46.713" Score="2" ViewCount="573" Body="&lt;p&gt;Process A runs a python script which generates a CSV file in some absolute file path say &amp;quot;/mnt/x/blah.csv&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Process B needs to consume the CSV file line by line, after process A has finished.&lt;/p&gt;&#xA;&lt;p&gt;The trouble is that the channel to input to B is initialized at the beginning before &amp;quot;/mnt/x/blah.csv&amp;quot; is created by A so it fails. This occurs even if I collect the result of process A as input to process B (because channel is initialized regardless)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;nextflow.enable.dsl=2&#xA;&#xA;&#xA;def get_the_csv_records() {&#xA;    Channel&#xA;        .fromPath(&amp;quot;/mnt/x/blah.csv&amp;quot;)&#xA;        .splitCsv(header:true, quote: '\&amp;quot;')&#xA;        .map{ it.TheFieldIWant }&#xA;}&#xA;&#xA;process process_a() {&#xA;    &#xA;    output:&#xA;    val(&amp;quot;I'm done!&amp;quot;)&#xA;&#xA;    '''&#xA;    python.exe ./make_the_csv_file.py --path=/mnt/x/blah.csv&#xA;    '''&#xA;}&#xA;&#xA;process process_b() {&#xA;&#xA;    input:&#xA;    val(previous_step_is_done_dummy_variable)&#xA;    val(the_field)&#xA;&#xA;    '''&#xA;    echo ${the_field}&#xA;    '''&#xA;&#xA;}&#xA;&#xA;workflow do_a_then_b{&#xA;    process_a()&#xA;    process_b(process_a.out.collect(), get_the_csv_records())&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This fails immediately with &amp;quot;No such file: /mnt/x/blah.csv&amp;quot; because the channel is initialized before process_a has even started.&lt;/p&gt;&#xA;&lt;p&gt;Same behavior occurs when using concat:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    process_b(process_a.out.collect().concat(get_the_csv_records()))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Question:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I make process_b() only read the input channel from the CSV file after it is created?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Alternatively, can I make process_b() trigger when process_a() is finished some other way?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Alternatively, can I add an intermediate step which reads process_a.out.collect(), then reads /mnt/x/blah.csv and outputs it into some existing channel which process_b() takes as input?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Another approach - what is the way to create a channel which outputs the results of splitCsv, when the file is not known at compile time?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="11959" LastEditorUserId="11959" LastEditDate="2021-06-05T10:18:27.483" LastActivityDate="2022-03-15T18:01:15.093" Title="Nextflow: How to initialize channel from CSV file but only after CSV file created?" Tags="|nextflow|workflow-management|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16059" PostTypeId="2" ParentId="16056" CreationDate="2021-06-05T10:29:16.930" Score="0" Body="&lt;p&gt;I would recommend not do this using grep. There is a program within the &lt;a href=&quot;https://github.com/TGAC/KAT&quot; rel=&quot;nofollow noreferrer&quot;&gt;KAT&lt;/a&gt; suite called KAT filter for filtering reads by a set of kmers (that might require all your sequences to be of constant length, but I am not sure). Another software you might want to check out is &lt;a href=&quot;https://github.com/ad3002/Cookiecutter&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cookiecutter&lt;/a&gt;. These tools specialised in subsetting sequencing files are usually a lot more efficient than generic regular expression tools.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2021-06-05T10:29:16.930" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16061" PostTypeId="1" CreationDate="2021-06-05T15:27:04.273" Score="6" ViewCount="806" Body="&lt;p&gt;I am working with bam files and I have to check if reads of a specific position or their mates are soft clipped. So, I am looking for a fast way to extract the read pairs from a bam file in python. So far, I use pysam and fetch reads of a given position. From those reads, I have access to the CIGAR string and can check them for soft clipping. My problem is that I cannot access the CIGAR of the read mates. Is there a way to access information of read mates with pysam or another tool, especially CIGAR strings, without iterating over the whole .bam file?&lt;/p&gt;&#xA;&lt;p&gt;Unlike in this question &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/3838/extracting-the-cigar-string-from-a-bam-via-python&quot;&gt;Extracting the CIGAR string from a BAM via Python?&lt;/a&gt; I am looking for the CIGAR of both reads of a pair.&lt;/p&gt;&#xA;" OwnerUserId="12426" LastActivityDate="2021-08-12T03:55:49.087" Title="Is there an efficient way to extract CIGAR strings for read pairs from bam files with python?" Tags="|sam|pysam|cigar|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16062" PostTypeId="2" ParentId="16061" CreationDate="2021-06-05T19:00:06.703" Score="5" Body="&lt;p&gt;I think that the easiest way is to work with query-name sorted groups of reads. In that case, mates will be adjacent in the sort and you can use that to extract the paired CIGARs.&lt;/p&gt;&#xA;&lt;p&gt;If you are depending on position sorting to extract the alignments of interest efficiently, you can maybe do the following:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;extract alignments in the region of interest plus a lot of padding to ensure that you are also getting the mates (this will obviously only work with standard short-insert shotgun libraries).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Query-name sort the list iterator using built-in &lt;code&gt;sort()&lt;/code&gt; or &lt;code&gt;sorted()&lt;/code&gt; with a key function that returns the query name.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Iterate over the pairs using adjacency in the iterator to extract paired CIGARs. Discard unpaired reads (first reads whose query name does not match supposed mates).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Code would look something like this (I have not run or tested this in any way!):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pysam&#xA;&#xA;def get_query_key(read):&#xA;  &amp;quot;&amp;quot;&amp;quot;read is pysam.AlignedSegment&amp;quot;&amp;quot;&amp;quot;&#xA;  return read.query_name&#xA;&#xA;def get_paired_cigars(bam, chr, start, end, padding=1000):&#xA;  &amp;quot;&amp;quot;&amp;quot;bam is pysam.AlignmentFile&amp;quot;&amp;quot;&amp;quot;&#xA;  reads = list(bam.fetch(bam, chr, start-padding, end+padding))  # may fail at chr ends&#xA;  reads.sort(key=get_query_key)&#xA;  first_cigar = None&#xA;  mate_cigar = None&#xA;  cigars = list()&#xA;  qn = &amp;quot;&amp;quot;&#xA;  for read in reads:&#xA;    if first_cigar is None:&#xA;      first_cigar = read.cigarstring&#xA;      qn = read.query_name&#xA;    elif mate_cigar is None:&#xA;      if read.query_name != qn:&#xA;        first_cigar = read.cigarstring&#xA;        qn = read.query_name&#xA;      else:&#xA;        mate_cigar = read.cigarstring&#xA;    if first_cigar is not None and mate_cigar is not None:&#xA;      cigars.append((first_cigar, mate_cigar))&#xA;      first_cigar = None&#xA;      mate_cigar = None&#xA;  return cigars&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I agree that I would like this to be simpler! Unfortunately BAM files are not terribly well-designed for using pairing information at that level of detail IMO, I have always had to use something like this. I would be interested to see a straightforward alternative to this approach from someone with more experience.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2021-08-12T03:55:49.087" LastActivityDate="2021-08-12T03:55:49.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16063" PostTypeId="1" CreationDate="2021-06-06T08:53:39.877" Score="1" ViewCount="75" Body="&lt;p&gt;I need to use NetworkAnalyzer to analyse the nodes of a PPI network on Cytoscape, but the pre-installed app is not displayed on the Tools menu.&lt;/p&gt;&#xA;&lt;p&gt;I checked the App Manager, but it shows &amp;quot;Installed&amp;quot; beside the app name and also it is not in the Currently installed apps list, which is why I'm unable to perform uninstall-reinstall to fix it.&lt;/p&gt;&#xA;&lt;p&gt;I've once replaced the app by installing it manually, but despite of that there's no change in the Tools menu.&lt;/p&gt;&#xA;&lt;p&gt;There's an option &amp;quot;Analyse Network&amp;quot; in the Tools, probably a substitute of NetworkAnalyzer, but it is not showing the usual attributes (Degree, BetweenessCentrality etc) which are displayed on using Network Analyzer app.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/r9q9j.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/r9q9j.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="555" LastActivityDate="2021-06-06T08:53:39.877" Title="How to activate an installed plugin (NetworkAnalyzer) on Cytoscape?" Tags="|cytoscape|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16064" PostTypeId="1" CreationDate="2021-05-23T12:25:15.337" Score="1" ViewCount="83" Body="&lt;p&gt;I have an RNAseq count matrix consisting of 2 groups (high, low) with 6 timepoints per group (T1,T2,...,T6) and 3 replicates per timepoint (rep1, rep2, rep3). So a 2-factor design with 36 samples in total (2x6x3=36).&lt;/p&gt;&#xA;&lt;p&gt;I want to perform a varianceStabilizingTransformation (with blind=FALSE) of the matrix to later perform rhythm analysis (over the timepoints, but separately for each group). Now I wonder how to create the DESeqDataSet that the varianceStabilizingTransformation command needs as input and how to appropriately define the design formula associated with the DESeqDataSet.&lt;/p&gt;&#xA;&lt;p&gt;Here a scheme of my count table (the numbers are random), which I would save in a .txt or .csv (named &amp;quot;matrix&amp;quot;) and import with &lt;code&gt;matrix &amp;lt;- read.table(FILE.txt, header=TRUE, row.names=1)&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ID            high_T1_rep1   high_T1_rep2   high_T1_rep3   high_T2_rep1 ...&#xA;transcript1   1.23           1.45           1.67           1.89 &#xA;transcript2   5.32           5.54           4.76           5.98&#xA;transcript3   3.22           3.44           3.66           3.88&#xA;transcript4   7.33           7.55           7.77           7.99&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For the design I would now create a table (also txt. or .csv) as follows and import it with &lt;code&gt;designdata &amp;lt;- read.table(FILE.txt, header=TRUE, row.names=1)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sample         group   timepoint&#xA;high_T1_rep1   high    T1&#xA;high_T1_rep2   high    T1&#xA;high_T1_rep3   high    T1&#xA;high_T2_rep1   high    T2&#xA;...            ...     ...&#xA;low_T5_rep3    low     T5&#xA;low_T6_rep1    low     T6&#xA;low_T6_rep2    low     T6&#xA;low_T6_rep3    low     T6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I went through several DESeq2 manuals, but still do not understand how to find the right definition for the &lt;code&gt;design=...&lt;/code&gt; term when creating the DESeqDataSet. The varianceStabilizingTransformation seems to use the design information (if &lt;code&gt;blind=FALSE&lt;/code&gt;), and thus I want to make sure, I provide it correctly.&lt;/p&gt;&#xA;&lt;p&gt;From what I found out so far, my command for the DESeqDataSet would be something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData=matrix, colData=designdata, design=???)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As options for the design I thought of e.g. &amp;quot;~ group + timepoint&amp;quot; or &amp;quot;~ group + timepoint + group:timepoint&amp;quot;, but I have no idea how one or the other option would affect the following varianceStabilizingTransformation.&lt;/p&gt;&#xA;&lt;p&gt;So while I have at least some rough idea what to do, my big concern is handling the data incorrectly and thereby distorting it without even noticing.&#xA;Any help you could give me is greatly appreciated. Thanks a lot in advance!&lt;/p&gt;&#xA;" OwnerUserId="12499" OwnerDisplayName="colormashed" LastActivityDate="2021-06-06T09:39:59.007" Title="How to create a DESeqDataSet and define experiment design before varianceStabilizingTransformation?" Tags="|r|rna-seq|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="16065" PostTypeId="2" ParentId="16056" CreationDate="2021-06-06T11:59:29.313" Score="1" Body="&lt;p&gt;Thank you all for your help. I managed to fix it in the end. The error was with how my pattern.txt file was formatted.&lt;/p&gt;&#xA;&lt;p&gt;I initially exported the .txt from R Studio by specifying the EOL as \r\n. However, I think this was the issue as then from my laptop to the computer cluster the files seemed to have a windows EOL.&lt;/p&gt;&#xA;&lt;p&gt;Thus on the linux server I did&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dos2unix patterns.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And since then all of the suggested loops, both while read and the for chunk work beautifully and I get a nice &lt;code&gt;\S+$line&lt;/code&gt; exported into the desired output.txt.&lt;/p&gt;&#xA;&lt;p&gt;Thanks again all for your advice and help!&lt;/p&gt;&#xA;" OwnerUserId="12419" LastEditorUserId="73" LastEditDate="2021-06-07T01:01:58.577" LastActivityDate="2021-06-07T01:01:58.577" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16066" PostTypeId="1" CreationDate="2021-06-06T22:29:06.020" Score="0" ViewCount="16" Body="&lt;p&gt;These are the facts: I am working with some morphological features. Each trait can have one of the following three states: 0, 1, 2.&lt;/p&gt;&#xA;&lt;p&gt;State 0 is always independent.&lt;/p&gt;&#xA;&lt;p&gt;But if there is a trait with state 1, then there must be, in another column of the same row, a state 2 (and vice versa, obviously).&lt;/p&gt;&#xA;&lt;p&gt;The traits can't be encoded separately because, eg, in the &lt;code&gt;row_01&lt;/code&gt;, at the &lt;code&gt;trait_01&lt;/code&gt;, the state 1 is ligated to the &lt;code&gt;trait_20&lt;/code&gt; with state 2. But in another row (&lt;code&gt;row_02&lt;/code&gt;), the &lt;code&gt;trait_01&lt;/code&gt; with state 1 could be ligated to another trait (&lt;code&gt;trait_33&lt;/code&gt;) with state 2. My data behaves like this because I'm working with morphology in... molecules.&lt;/p&gt;&#xA;&lt;p&gt;So far, I know that the data of an array in phylogeny must be independent, so I am wondering if any of you know of any way to approach this problem (paper, algorithm, your own propose, etc.).&lt;/p&gt;&#xA;&lt;p&gt;solving conditions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I can't discard the traits with state 1 or 2.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Cheers!&lt;/p&gt;&#xA;" OwnerUserId="729" LastEditorUserId="73" LastEditDate="2021-06-07T00:53:44.933" LastActivityDate="2021-06-07T00:53:44.933" Title="How to run a phylogenetic analysis with non-independent data?" Tags="|phylogenetics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16067" PostTypeId="1" AcceptedAnswerId="16072" CreationDate="2021-06-07T00:06:50.490" Score="2" ViewCount="1259" Body="&lt;p&gt;I looking at a particular protein structure called &lt;a href=&quot;https://www.rcsb.org/structure/2LY4&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;2LY4&lt;/strong&gt;&lt;/a&gt; accessible from &lt;a href=&quot;https://www.rcsb.org/structure/2LY4&quot; rel=&quot;nofollow noreferrer&quot;&gt;RSCB PDB&lt;/a&gt; website.&#xA;The corresponding fasta sequence for that structure is &lt;a href=&quot;https://www.rcsb.org/fasta/entry/2LY4&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;2LY4_1|Chain A|High mobility group protein B1|Homo sapiens (9606)&#xA;GKGDPKKPRGKMSSYAFFVQTCREEHKKKHPDASVNFSEFSKKCSERWKTMSAKEKGKFEDMAKADKARYEREMKTYIPPKGE&#xA;&amp;gt;2LY4_2|Chain B|Cellular tumor antigen p53|Homo sapiens (9606)&#xA;MEEPQSDPSVEPPLSQETFSDLWKLLPENNVLSPLPSQAMDDLMLSPDDIEQWFTEDPGPDEAPRMPEAAPPVAPAPAAPTPAAPAPAPSWPL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the PDB format file can be downloaded &lt;a href=&quot;https://files.rcsb.org/download/2LY4.pdb&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (1.7MB). The entire is to large to be pasted here.&lt;/p&gt;&#xA;&lt;p&gt;What I want to do is to extract the subset of PDB format based on the subsequence in fasta above.&#xA;Namely &lt;code&gt;Chain A&lt;/code&gt; starting  from 1st residue to 30th residue.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GKGDPKKPRGKMSSYAFFVQTCREEHKKKH&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I do that in R or Python?&lt;/p&gt;&#xA;" OwnerUserId="12433" LastActivityDate="2021-06-07T14:04:53.660" Title="How to extract subset of protein structure (PDB format) file based on a subsequence of that protein" Tags="|r|python|fasta|protein-structure|pdb|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16068" PostTypeId="2" ParentId="16050" CreationDate="2021-06-07T00:10:38.313" Score="1" Body="&lt;p&gt;Well, here my thoughts:&lt;/p&gt;&#xA;&lt;p&gt;The equation &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; somehow tries to represent the ratio &lt;span class=&quot;math-container&quot;&gt;$\frac{equality}{equality+inequality}$&lt;/span&gt;. But &lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; returns only a value. And to assing the p-value to a single value... is complicated.&lt;/p&gt;&#xA;&lt;p&gt;How could you approach this?&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Using some kind of Hamming distance for each base pair at the position &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; (&lt;span class=&quot;math-container&quot;&gt;$1 \leqslant n \leqslant len$&lt;/span&gt;) between the &lt;code&gt;sequence i&lt;/code&gt; (&lt;span class=&quot;math-container&quot;&gt;$s_{i}$&lt;/span&gt;) and &lt;code&gt;sequence j&lt;/code&gt; (&lt;span class=&quot;math-container&quot;&gt;$s_{j}$&lt;/span&gt;) in order to get a set of binary data:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$s_{ij} = \begin{Bmatrix} 1\ if\ seq_{i_{n}}=seq_{j_{n}} \\ 0\ if\ seq_{i_{n}}\neq seq_{j_{n}}   \end{Bmatrix}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;With this, you get a set of binomial data (eg &lt;span class=&quot;math-container&quot;&gt;$\{0,1,0,1,0,1,1,1,0,1,1,1,0,...\}$&lt;/span&gt;). Now, you should be able to perform some (Binary?) test and get the p-value.&lt;/p&gt;&#xA;&lt;p&gt;PD: Consider aligning the two sequences if there is interest in discussing base pair homology.&lt;/p&gt;&#xA;" OwnerUserId="729" LastEditorUserId="729" LastEditDate="2021-06-07T00:16:56.253" LastActivityDate="2021-06-07T00:16:56.253" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16069" PostTypeId="2" ParentId="16009" CreationDate="2021-06-07T01:06:43.017" Score="2" Body="&lt;p&gt;&lt;em&gt;Answer from @terdon, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;The link just points to the NCBI ftp site where you can download the source package and look at the code. There is no need for login credentials, you can log in as anonymous. It's just that the version has changed so that answer's URL no longer works. Try this instead:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;wget https://ftp.ncbi.nlm.nih.gov/blast/executables/blast+/LATEST/ncbi-blast-2.11.0+-src.tar.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then, extract the files and look at the file &lt;code&gt;c++/src/algo/blast/core/blast_stat.c&lt;/code&gt; which, again according to the answer you linked to, should have the lamda and k tables at line 589.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-06-07T01:06:43.017" CommentCount="0" CommunityOwnedDate="2021-06-07T01:06:43.017" ContentLicense="CC BY-SA 4.0" />
  <row Id="16070" PostTypeId="2" ParentId="14178" CreationDate="2021-06-07T02:06:03.973" Score="1" Body="&lt;p&gt;Yes, you can use the second one for volcano plots, but it might help to understand what it's implying.&lt;/p&gt;&#xA;&lt;p&gt;The difference between these formulas is in the &lt;code&gt;mean&lt;/code&gt; calculation. The following equations are identical:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;log(a/b)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;log(a) - log(b)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, in your example, the first equation calculates the mean prior to doing the log, whereas the second example does the log first (i.e. &lt;code&gt;log(mean(Group1))&lt;/code&gt; vs &lt;code&gt;mean(log(Group1))&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;As a consequence, the equations are &lt;em&gt;similar&lt;/em&gt;, but not identical, and that difference depends on the underlying assumptions of the expression model.&lt;/p&gt;&#xA;&lt;p&gt;In the first equation (what you call 'Actual'), the genes are assumed to follow a normal distribution in linear space, whereas in the second equation they're assumed to follow a normal distribution in &lt;em&gt;log&lt;/em&gt; space.&lt;/p&gt;&#xA;&lt;p&gt;Which one of these is correct is a subject of debate. It's pretty much established that gene expression for &lt;em&gt;all genes together&lt;/em&gt; is log-normal (or, at least, much closer to log-normal than to linear), but whether this applies at an individual gene level is less clear.&lt;/p&gt;&#xA;&lt;p&gt;In most cases, the results will be similar when looking at individual genes, and the equations can be used interchangeably. Where they differ substantially, there will be large expression differences between genes, so it's unlikely to alter the interpretation of results.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-06-07T02:06:03.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16071" PostTypeId="2" ParentId="16067" CreationDate="2021-06-07T06:42:26.870" Score="2" Body="&lt;p&gt;If you need to do it only once, for this one PDB file,&#xA;it's easier to do it manually.&#xA;When you open the file you can see it has 10 models from NMR.&#xA;Do you want to extract a subset of one model or of all 10?&#xA;In this file the residues are numbered according to the sequence.&#xA;So you find the line where residue 31 begins in the first model:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATOM    500  N   PRO A  31      18.844  -9.699  -4.620  1.00  0.00           N&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and delete all lines until&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ENDMDL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then you do the same for other models, or you just remove other models, depending on what you need.&#xA;You may also want to remove headers – from the beginning of the file until the &lt;code&gt;CRYST1&lt;/code&gt; record.&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2021-06-07T06:42:26.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16072" PostTypeId="2" ParentId="16067" CreationDate="2021-06-07T07:07:04.823" Score="4" Body="&lt;p&gt;In general, if you simply want to extract that part of the PDB file, you could loop over it (it's plain text) and check the fields you're interested in:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;with open('2ly4.pdb') as pdb:&#xA;    for line in pdb:&#xA;        if line[:4] == 'ATOM':&#xA;            chain = line[21]&#xA;            res_idx = int(line[23:26])&#xA;            if chain == 'A' and 1 &amp;lt;= res_idx &amp;lt;= 30:&#xA;                print(line[:-1])&#xA;        else:&#xA;            print(line[:-1]) # alternatively, skip with continue&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, this doesn't quite work with the file you've linked because it actually contains ten different conformations (models). This is better handled with a proper parser like &lt;a href=&quot;http://prody.csb.pitt.edu/tutorials/structure_analysis/pdbfiles.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;prody&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import prody&#xA;&#xA;model = prody.parsePDB('2ly4.pdb', model=1)&#xA;extract = model.select('chain A resnum 1to30')&#xA;prody.writePDB('extract.pdb', extract)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will also allow you to do more complex atom selection.&lt;/p&gt;&#xA;" OwnerUserId="12439" LastActivityDate="2021-06-07T07:07:04.823" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16073" PostTypeId="1" AcceptedAnswerId="16079" CreationDate="2021-06-07T11:35:58.020" Score="2" ViewCount="320" Body="&lt;p&gt;I'm in a little bit of a bind with targeted single-cell sequencing. I'm trying to match up our reads to the targeted amplicon panel (418 targets), and all but one have matched successfully with Salmon Alevin. I have a transcript that produces many 75bp reads that only match the first 28bp of a target, and Alevin is ignoring those reads as a match (the 28bp sequence is genome-unique).&lt;/p&gt;&#xA;&lt;p&gt;Here's an example read, with matched sequence:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;qual FFFFFFFFFFFFFFFFFFFFFFFFFFFF,,,::,,:,:,,,,,,,,,,,,,,,,,:,,,,,,,,:,,,,,:,:&#xA;read GCTGCATAATCTCTTCATTCCGAGGAGCACCCCACGCGCGCCGCGCCCCCCCGCCCCCGGGGGGGGGGGGGGG&#xA;SNPs ...................................G.G.G..G.GC.....CG...CCGGGGGGGGG..GG..&#xA; ref GCTGCATAATCTCTTCATTCCGAGGAGCACCCCACCCCCACCCCCACCCCCACCCCATTCTTAAATTGTTTGG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you may be able to see from the above sequence, the reads for this particular transcript get up to 28bp (sometimes with one or two errors), then the wheels start coming off the sequencer, showing an abundance of Gs. This was a NovaSeq run, and a polyG read is the same as no signal using its two-dye process, so I suspect that the sequencing stops after ~35bp, rather than actually synthesising Gs from thin air.&lt;/p&gt;&#xA;&lt;p&gt;I have lots of these reads; there are about 400k reads that match the reference transcript (Cxcr4, just in case it matters), and about 250k of those reads have a G homopolymer that's at least 8bp in length.&lt;/p&gt;&#xA;&lt;p&gt;Any ideas on what to do?&lt;/p&gt;&#xA;&lt;p&gt;FWIW, the above match was taken from a Bowtie2 alignment I did on the reads. I could use Alevin for cell barcode correction only and do the UMI deduplication manually, but it seems a shame to do that when it's only one transcript (from 418) that's spoiling a fully-automated process.&lt;/p&gt;&#xA;&lt;p&gt;I did try adding in an additional transcript that included the full sequence of a short-match read, but there seem to be too many errors in the non-matching sequence for any reads to count (at least... I &lt;em&gt;assume&lt;/em&gt; that's why the counts are not coming through; I'm not quite sure how I can work out what's going on behind the scenes).&lt;/p&gt;&#xA;&lt;p&gt;Is there any way to tweak Salmon Alevin to get it to accept these shorter matches? If I can change the kmer size, how do I do that?&lt;/p&gt;&#xA;&lt;p&gt;End trimming is unlikely to help, because that's going to shorten the reads and make them less likely to map to the target sequence. I want a way to &lt;em&gt;include&lt;/em&gt; these reads as valid hits to the target sequence.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-06-08T04:28:13.043" LastActivityDate="2021-06-08T04:28:13.043" Title="How do I increase the sensitivity of Salmon Alevin?" Tags="|scrnaseq|transcriptome|salmon|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16074" PostTypeId="2" ParentId="13160" CreationDate="2021-06-07T14:00:14.003" Score="4" Body="&lt;p&gt;I think what you're trying to do could be achieved by adding parenthesis to the conditions and by specifying the format of the input file by using the flag &lt;code&gt;--format vcf&lt;/code&gt;, as specified in the &lt;a href=&quot;https://www.ensembl.org/info/docs/tools/vep/script/vep_filter.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;.&#xA;The final command should look like this:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;filter_vep -i &amp;quot;$input&amp;quot; --format vcf --filter &amp;quot;(SIFT = deleterious) and (PolyPhen match damaging)&amp;quot; -o &amp;quot;$output&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9584" LastActivityDate="2021-06-07T14:00:14.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16075" PostTypeId="2" ParentId="16067" CreationDate="2021-06-07T14:04:53.660" Score="3" Body="&lt;p&gt;Here is an example how to do it using another Python library with another selection syntax.&lt;br /&gt;&#xA;Requires &lt;code&gt;pip install gemmi&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import gemmi&#xA;&#xA;st = gemmi.read_structure('2ly4.pdb')&#xA;subset = gemmi.parse_cid('/*/A/1-30').copy_structure_selection(st)&#xA;subset.write_minimal_pdb('output.pdb')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="266" LastActivityDate="2021-06-07T14:04:53.660" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16076" PostTypeId="1" CreationDate="2021-06-07T14:57:10.840" Score="0" ViewCount="727" Body="&lt;p&gt;My VLS run completed yielding millions of ligands and I would like to extract top-scored 20000 ligands in a separate file. Incorporating the .maegz output in Maestro is not an option as it fills up all the memory and keeps crashing. Are there any command-line options for this task?&lt;/p&gt;&#xA;&lt;p&gt;PS: I tried using rdkit in python to parse the output file after converting it to .sdf, however, rdkit skipped some ligands that it could not read.&lt;/p&gt;&#xA;" OwnerUserId="7150" LastActivityDate="2021-11-06T10:03:35.373" Title="How to extract docked ligands from .maegz docking results file sorted by glide gscores in Schrodinger Maestro?" Tags="|format-conversion|docking|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16077" PostTypeId="1" AcceptedAnswerId="16078" CreationDate="2021-06-07T20:31:33.640" Score="0" ViewCount="68" Body="&lt;p&gt;I am reading a blog and it says:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The numbers for identities and replacements used for calculating the overall alignment score in the expression above are usually presented in the form of a 20 x 20 matrix (20 is the number of the most common amino acids). In total there are 210 possible replacement pairs (residues replacing each other) of amino acids, which includes 190 pairs of different amino acid substitutions + 20 pairs of identical substitutions&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;My question is how are there 210 replacement pairs. What is the formula being used here? Insights will be appreciated. The link to blog is :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://proteinstructures.com/sequence/amino-acid-substitutions/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://proteinstructures.com/sequence/amino-acid-substitutions/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12351" LastActivityDate="2021-06-08T01:39:50.273" Title="Total pairs of amino acid substitution" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16078" PostTypeId="2" ParentId="16077" CreationDate="2021-06-08T00:54:10.937" Score="1" Body="&lt;p&gt;There are 190 pairs of different amino acids:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$ {20\choose 2} = 190$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and 20 pairs of identical amino acids (AA, TT, WW, etc). Combining these, you get all 210 pairs referenced in the article. Another way of seeing this is dividing the 20x20 matrix into 3 parts: the diagonal, all values above the diagonal, and all values below. We have 20 values on the diagonal with 380 values to split evenly across the upper and lower parts, leaving 190 for each. Therefore, the lower (or upper) part plus the diagonal comes out to 210.&lt;/p&gt;&#xA;" OwnerUserId="6628" LastEditorUserId="6628" LastEditDate="2021-06-08T01:39:50.273" LastActivityDate="2021-06-08T01:39:50.273" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16079" PostTypeId="2" ParentId="16073" CreationDate="2021-06-08T03:14:08.700" Score="3" Body="&lt;p&gt;So, I think there are a few potential options here.  Alevin is using selective-alignment internally to determine the mappings for the reads.  So, even if you have a k-mer supporting the mapping, if you have gibberish for the rest of the read, the mapping score is going to be very poor and you're not going to recover that mapping locus.  There are some parameters you can mess with to try to increase the sensitivity of mapping, though you should always be aware that upping the sensitivity a lot will mean there is less aggressive filtering early in the mapping process which can slow things down.  The main parameter you should tweak to allow lower alignment scores is &lt;code&gt;--minScoreFraction&lt;/code&gt;; the smaller this value the lower (worse) will be the permitted scores for valid mappings.  Additionally, you can combine changes to the &lt;code&gt;--minScoreFraction&lt;/code&gt; with the &lt;code&gt;--softclip&lt;/code&gt; option to simply score the non-matching end of read as 0 (rather than a long deletion or large set of mismatches); this will lower the effect of bad read ends.&lt;/p&gt;&#xA;&lt;p&gt;Another alternative you might consider is to try out &lt;a href=&quot;https://alevin-fry.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;alevin-fry&lt;/a&gt; for processing this data, and mapping the reads with the &lt;code&gt;--sketch&lt;/code&gt; flag.  This will only check certain structural constraints of the mapping and won't force a particular minimum alignment score.  The caveat here is that dropping the requirement for alignment scoring can increase the rate of spurious mappings (reads deriving from elsewhere that spuriously map to some indexed target simply by virtue of sharing some sequence).  In transcriptome wide analyses, &lt;a href=&quot;https://combine-lab.github.io/alevin-fry-tutorials/2021/improving-txome-specificity/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this can be greatly mitigated by building an appropriate index and using a UMI resolution strategy that complements this expanded index&lt;/a&gt;. We've not examined this in detail for targeted single-cell sequencing.  However, if you want to give it a try and run into any issues, feel free to drop an issue over on &lt;a href=&quot;https://github.com/COMBINE-lab/alevin-fry/issues&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt; and we'd be happy to help work through it.&lt;/p&gt;&#xA;" OwnerUserId="491" LastActivityDate="2021-06-08T03:14:08.700" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16080" PostTypeId="2" ParentId="16073" CreationDate="2021-06-08T03:25:20.700" Score="0" Body="&lt;p&gt;@ATPoint commented that &lt;em&gt;the default kmer size in Salmon indexing is 31&lt;/em&gt;, which led me to a solution that involved altering the index generation step:&lt;/p&gt;&#xA;&lt;p&gt;Instead of:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;salmon index -t merged_salmon_&lt;span class=&quot;math-container&quot;&gt;${targetName}.fasta -i salmonIndex_$&lt;/span&gt;{targetName}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I now have:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;salmon index -t merged_salmon_&lt;span class=&quot;math-container&quot;&gt;${targetName}.fasta -i salmonIndex_k11_$&lt;/span&gt;{targetName} -k 11&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In other words, I changed the kmer size from its default of 31 to 11, which matches the BD-recommended seed substring length for the standard Bowtie2 approach (&amp;quot;-L 10&amp;quot;, but I increased that to 11 because Salmon needs odd kmer sizes). I'm pretty sure that my 28bp-mapping reads won't be picked up with a kmer size of 31. Because this is targeted sequencing, I don't expect that a reduced kmer size will be an issue, but I'll keep a watch out for that.&lt;/p&gt;&#xA;&lt;p&gt;I'm currently re-running the analysis. If it still doesn't work, then I'll see about adding in the tweaks suggested by @nomad.&lt;/p&gt;&#xA;&lt;p&gt;Update: still not working, so I'll shift onto score tweaks.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-06-08T03:56:19.840" LastActivityDate="2021-06-08T03:56:19.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16081" PostTypeId="1" CreationDate="2021-06-08T08:00:44.720" Score="1" ViewCount="128" Body="&lt;p&gt;I am working on network analysis of differentially expressed genes, but the problem I am facing is that the gene set is very small (around 16 genes only), so when I'm trying to build an interaction network with these genes using STRING, the nodes starts to dissociate from the interaction when I set the confidence score to 0.400 or above (as the genes are related to &lt;em&gt;Homo sapiens&lt;/em&gt;).&lt;/p&gt;&#xA;&lt;p&gt;DE gene network has to be made in cytoscape, for that STRING ppi interactions are needed, that interaction network will tell us co-expression, homology etc and provide us a combined score.&lt;/p&gt;&#xA;&lt;p&gt;So, to build an interaction network between these genes, I have to decrease the confidence score to 0.150 since it is only with this score that I am finding interactors.&lt;/p&gt;&#xA;&lt;p&gt;So, I want to ask &lt;em&gt;&lt;strong&gt;whether this low confidence score is valid for this short gene set related to Homo sapiens&lt;/strong&gt;&lt;/em&gt; or, if not, what should I do in this case?&lt;/p&gt;&#xA;" OwnerUserId="12443" LastEditorUserId="73" LastEditDate="2021-06-08T21:13:40.287" LastActivityDate="2021-06-08T21:13:40.287" Title="Network analysis: confidence score (STRING database) validation" Tags="|networks|stringdb|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16082" PostTypeId="1" AcceptedAnswerId="16086" CreationDate="2021-06-08T13:22:44.787" Score="4" ViewCount="653" Body="&lt;p&gt;I previously &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/13845/programmatically-retrieve-accession-list-from-ncbi&quot;&gt;asked a question&lt;/a&gt; about how to retrieve the Accession List associated with a SRA project. The answer was:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;esearch -db sra -query 'PRJNA491191[bioproject]' | efetch -format runinfo&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;code&gt;PRJNA491191&lt;/code&gt; is the bioproject that I'm interested in in this example. How can I do the same for the &lt;code&gt;metadata&lt;/code&gt; file (the arrow in the picture below is pointing to accession list. It should point to metadata, which is what I'm interested in now)?&#xA;&lt;a href=&quot;https://i.stack.imgur.com/3hySD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3hySD.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2021-06-09T17:22:01.563" Title="Programmatically retrieve Metadata from SRA Run Selector" Tags="|phylogenetics|sra|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16085" PostTypeId="2" ParentId="14082" CreationDate="2021-06-08T13:59:42.270" Score="0" Body="&lt;p&gt;Signal and noise are best determined at the count level, rather than after normalisation. I don't think there's an easy way to get a signal level from normalised data.&lt;/p&gt;&#xA;&lt;p&gt;When I wanted to establish the noise level of counts from Illumina reads from lots of different samples, I looked at the total count distribution for genes that had counts in only one sample, and used the maximum value of that (or possibly an elbow of that distribution) for my signal threshold.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-06-08T13:59:42.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16086" PostTypeId="2" ParentId="16082" CreationDate="2021-06-08T14:05:13.500" Score="3" Body="&lt;p&gt;Depending on exactly which information you seek, you can try:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db sra -query 'PRJNA491191[bioproject]' | efetch -format native -mode xml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will return an XML with additional information that is not present in the &lt;code&gt;runinfo&lt;/code&gt; table. You can use the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez Direct&lt;/a&gt; tool &lt;code&gt;xtract&lt;/code&gt; to parse the output XML and extract the fields of interest into a tabular format.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, you can try:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db sra -query 'PRJNA491191[bioproject]' | esummary&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to retrieve a summary output that includes fewer fields than the &lt;code&gt;efetch&lt;/code&gt; option shown above. Perhaps all of the fields that are of interest to you are present in the summary output... it will save you time and effort in parsing the XML and retrieving the information you want.&lt;/p&gt;&#xA;&lt;p&gt;For example, you can use the following to retrieve some fields using &lt;code&gt;xtract&lt;/code&gt; from the &lt;code&gt;esummary&lt;/code&gt; output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ esearch -db sra -query 'PRJNA491191[bioproject]' \&#xA;  | esummary \&#xA;  | xtract -pattern DocumentSummary -element Summary/Title,Statistics@total_size,Study@acc &#xA;GSM3391430: hsf-1(OE) rep3; Caenorhabditis elegans; RNA-Seq  731402855  SRP161795&#xA;GSM3391429: hsf-1(OE) rep2; Caenorhabditis elegans; RNA-Seq  703822418  SRP161795&#xA;GSM3391428: hsf-1(OE) rep1; Caenorhabditis elegans; RNA-Seq  715482490  SRP161795&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3577" LastEditorUserId="3577" LastEditDate="2021-06-09T17:22:01.563" LastActivityDate="2021-06-09T17:22:01.563" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16087" PostTypeId="1" CreationDate="2021-06-08T15:31:33.117" Score="1" ViewCount="359" Body="&lt;p&gt;&lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/fastq-input&quot; rel=&quot;nofollow noreferrer&quot;&gt;10x Genomics&lt;/a&gt; data are stored in three FASTQ files, besides the standard R1 and R2 reads, there is also a I1 file with some metadata. Sometimes however they are shipped in a single bam/cram file (e.g. the data from the Darwin Tree of Life). How can I convert the BAM or CRAM files to FASTQ?&lt;/p&gt;&#xA;&lt;p&gt;-- edit --&lt;/p&gt;&#xA;&lt;p&gt;Apparently, the I3 file is not 10x specific thing. It is an index generated when demultiplexing any Illumina data. See &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/5178/what-is-the-index-fastq-file-sample-i-fastq-gz-generated-when-demultiplexing&quot;&gt;this question&lt;/a&gt; for a nice explanation.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2021-06-08T22:48:56.707" LastActivityDate="2022-03-02T21:39:51.300" Title="How to convert CRAM file with 10x data in three fastq files" Tags="|sam|fastq|format-conversion|cram|" AnswerCount="2" CommentCount="2" CommunityOwnedDate="2021-06-10T09:39:58.440" ContentLicense="CC BY-SA 4.0" />
  <row Id="16088" PostTypeId="2" ParentId="16087" CreationDate="2021-06-08T15:31:33.117" Score="1" Body="&lt;p&gt;All can be done with &lt;a href=&quot;http://www.htslib.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools&lt;/a&gt;. This is how &lt;a href=&quot;https://github.com/darwintreeoflife/darwintreeoflife.data/issues/2#event-4859159249&quot; rel=&quot;nofollow noreferrer&quot;&gt;the Darwin Tree of Life folks convert it&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools fastq -@4 -i \&#xA;  -1 &lt;span class=&quot;math-container&quot;&gt;${sample}_S$&lt;/span&gt;{tag}_L%03s_R1_001.fastq.gz \&#xA;  -2 &lt;span class=&quot;math-container&quot;&gt;${sample}_S$&lt;/span&gt;{tag}_L%03s_R2_001.fastq.gz \&#xA;  --i1 &lt;span class=&quot;math-container&quot;&gt;${sample}_S$&lt;/span&gt;{tag}_L%03s_I1_001.fastq.gz \&#xA;  --index-format i8 ${lane}.cram&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2021-06-08T22:42:02.747" LastActivityDate="2021-06-08T22:42:02.747" CommentCount="4" CommunityOwnedDate="2021-06-10T09:39:45.473" ContentLicense="CC BY-SA 4.0" />
  <row Id="16089" PostTypeId="1" AcceptedAnswerId="16099" CreationDate="2021-06-08T23:46:37.597" Score="1" ViewCount="2990" Body="&lt;p&gt;I'm a biologist, not a programmer so please be gentle.&lt;/p&gt;&#xA;&lt;p&gt;So I have a dataset that looks like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Genes  Patient1   Patient2   Patient3&#xA;A          324      433         343&#xA;B          431       342        124&#xA;Z          232       234        267&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;then I have the sample sheet where it contains sample info like:&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;pre&gt;&lt;code&gt;Patient1 - Healthy&#xA;Patient2 - Disease&#xA;Patient3 - Healthy &#xA;(110 samples)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(ggfortify)&#xA;df &amp;lt;- dataset&#xA;pca_res &amp;lt;- prcomp(df, scale. = TRUE)&#xA;&#xA;autoplot(pca_res)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then I want to do&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;autoplot(pca_res, data = ?, colour = '?')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I wish to use the info from the sample sheet to colour my PCA based on the state (healthy/disease) using the autoplot function. Is there a way to do this?&lt;/p&gt;&#xA;" OwnerUserId="9788" LastEditorUserId="650" LastEditDate="2021-06-09T16:15:29.570" LastActivityDate="2021-06-09T16:20:46.683" Title="PCA plot in R coloured by sample type" Tags="|r|ggplot2|pca|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16090" PostTypeId="2" ParentId="16076" CreationDate="2021-06-09T08:48:41.250" Score="1" Body="&lt;p&gt;It sounds like you are saying your file is already sorted? Is this correct?&lt;/p&gt;&#xA;&lt;p&gt;RDKit Mae supplier is an iterator as is the gzip reader, so does not load all in memory, so that is the best option given that loading all 1e6 into memory crashes your system. What is the fraction of unreadable compounds in RDKit? And does it matter? These are likely failing sanitisation —the &lt;code&gt;sanitize=False&lt;/code&gt; flag. If for some reason none can be ignored, circumventing this is hard. The &lt;code&gt;removeHs=True&lt;/code&gt; flag is rarely/never the cause even though the description of several aromatics require protons, e.g. an indole ring is &lt;code&gt;c1ccc2c(c1)cc[nH]2&lt;/code&gt;, the proton removal happens after loading. If you have an identifier and a list of SMILES you can correct dodgy bond orders etc. But making an unsanitisable molecule sanitary is tricky and rarely worth it as the output molecule is likely rubbish anyway.&#xA;Plus loading in RDKit will allow you do some additional calculations, such as ligand efficiency (best for sorting anyway), partition coefficient etc etc. As opposed to simply slicing your sorted file.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, I believe Open Babel does not load all into memory and has a &lt;code&gt;splitinto&lt;/code&gt; argument to make the file smaller and can convert gzipped Mae file to SDF.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-06-09T08:48:41.250" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16091" PostTypeId="1" CreationDate="2021-06-09T09:56:58.513" Score="2" ViewCount="1134" Body="&lt;p&gt;I need to identify the sequence of a gene in the complete genome sequence . I thought it was simple, instead it is not a straightforward task !&lt;/p&gt;&#xA;&lt;p&gt;My method was the following: I downloaded the FASTA format file of the complete genome sequence of the archaea organism that I am studying, that is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/CP003685.1?report=fasta&quot; rel=&quot;nofollow noreferrer&quot;&gt;CP003685&lt;/a&gt;, and then I downloaded the CDS file from NCBI by clicking on &amp;quot;&amp;gt;Send to&amp;gt;Coding Sequencing&amp;gt;FASTA Nucleotide&amp;gt;Create File&amp;quot;. In this way I have a txt file (always FASTA format file) in which I find a sequence for each gene (if I am not wrong).&lt;/p&gt;&#xA;&lt;p&gt;So for example, I should find, in the complete genome sequence, the sequences of the CDS file (the last that I downloaded), right?&lt;/p&gt;&#xA;&lt;p&gt;For example, if I consider:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;lcl|CP003685.1_cds_AFN03983.1_1007 [locus_tag=PFC_05180] [protein=DNA protection protein DPS] [protein_id=AFN03983.1] [location=complement(940023..940580)] [gbkey=CDS]&#xA;ATGCCAGAGCATAATAGGAGATTAGTTGAAAGGACAGGAATAGATGTAGAGAAGCTGTTGGAACTTCTCA&#xA;TTAAAGCGGCTGCAGCAGAGTTCACAACGTATTATTACTACACTATACTTAGAAACCATGCAACAGGTCT&#xA;GGAAGGAGAAGCAATTAAGGAGATTATCGAAGATGCTAGACTTGAGGACAGAAACCATTTTGAAGCCCTC&#xA;GTGCCAAGAATATACGAGCTTGGAGGAGAATTACCAAGGGATATTCGGGAATTTGCAGATCTAGCTTCTT&#xA;GTAGAGATGCCTACCTACCAGAAGAGCCCACTATAGAGAACATTCTAAAAGTCCTACTCGAAGCTGAGAG&#xA;ATGTGCAGTTGGAGTCTATACAGAAATATGCAACTACACTTTTGGAAAGGATCCAAGGACTTATGATTTG&#xA;GCCCTTGCCATATTACATGAAGAGATAGAGCATGAAGCATGGTTTGAAGAGCTCTTGACTGGAAAGCCGA&#xA;GTGGTCACTTTAGAAGAGGAAAGCCTGGGGAAAGTCCATATGTCTCAAAGTTTTTGAAAACTAGATAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I should find in the complete genome sequence, this sequence, right ? But this is not. Why?&lt;/p&gt;&#xA;&lt;p&gt;I suspect that I miss very important passages.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="10594" LastActivityDate="2023-07-26T05:03:39.020" Title="How to identify the DNA sequence of a gene in the complete genome sequence FASTA format file?" Tags="|fasta|genome|sequence-analysis|phylogenetics|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16092" PostTypeId="2" ParentId="16091" CreationDate="2021-06-09T10:00:56.080" Score="5" Body="&lt;p&gt;In your sequence header you will see:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[location=complement(940023..940580)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It's a reverse strand gene. Reverse complement the sequence, then search the genome sequence.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2021-06-09T10:00:56.080" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16093" PostTypeId="2" ParentId="16091" CreationDate="2021-06-09T10:34:44.593" Score="6" Body="&lt;p&gt;I would just &lt;a href=&quot;https://anaconda.org/bioconda/blast&quot; rel=&quot;nofollow noreferrer&quot;&gt;blast&lt;/a&gt; it. When blasting locally, you need to first make a database from your genome, so assuming you got the command-line version of blast installed you can do something like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;makeblastdb -in my_study_genome.fa -dbtype nucl&#xA;blastn -max_target_seqs 10 -db my_study_genome.fa -evalue 1e-10 -outfmt 6 -query my_downloaded_gene_of_interest.fasta -out gene_in_my_study_genome.blast&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;blast search for anything that is &amp;quot;similar enough&amp;quot; and also considers both strands, so for example, if there is an SNP in the middle of the gene. Hence even if the searched gene and the reference genome are from different sequencing runs / individuals, blast will still find it. Blast will also handle linebreaks for you (in fasta files the nucleotide sequence is quite often wrapped by 60 or 100 characters a line). Long story short, there is not much that can go wrong with blast (while with full text search you can easily shoot yourself in foot).&lt;/p&gt;&#xA;&lt;p&gt;-- edit 2 --&lt;/p&gt;&#xA;&lt;p&gt;As @terdon suggested, potentially if you would locating a transcript sequence (excluding introns), you would do better with software like &lt;a href=&quot;https://www.ebi.ac.uk/about/vertebrate-genomics/software/exonerate-user-guide&quot; rel=&quot;nofollow noreferrer&quot;&gt;exonerate&lt;/a&gt; or &lt;a href=&quot;https://www.ebi.ac.uk/Tools/psa/genewise/&quot; rel=&quot;nofollow noreferrer&quot;&gt;genewise&lt;/a&gt;, because they model exon/intron boundaries. However, even in those cases blast will give you an approximate position of the gene.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2021-06-09T15:51:14.787" LastActivityDate="2021-06-09T15:51:14.787" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16094" PostTypeId="2" ParentId="16089" CreationDate="2021-06-09T10:54:12.243" Score="2" Body="&lt;p&gt;Can't say I've ever used autoplot, but this is fairly easy to achieve using &amp;quot;base&amp;quot; ggplot.&lt;/p&gt;&#xA;&lt;p&gt;One thing autoplot does is put the axis on the same scale reflecting the percentage variation captured by each PC - I think this is a good thing, but most PCA plots you'll see don't do this.&lt;/p&gt;&#xA;&lt;p&gt;As you haven't given us any info on your metadata, I'll just make a few assumptions - lets make it a data.frame called &amp;quot;colData&amp;quot; with a &amp;quot;Status&amp;quot; column.&lt;/p&gt;&#xA;&lt;p&gt;prcomp stores the PC scores in &amp;quot;pca_res$x&amp;quot;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# generate some metadata&#xA;colData &amp;lt;- data.frame(row.names=rownames(pca_res&lt;span class=&quot;math-container&quot;&gt;$x),status=as.factor(sample(c(&quot;diseased&quot;,&quot;healthy&quot;),nrow(pca_res$&lt;/span&gt;x),replace = T)))&#xA;&#xA;# get % variation for each PC &#xA;pca_res&lt;span class=&quot;math-container&quot;&gt;$percentVar &amp;lt;- pca_res$&lt;/span&gt;sdev^2/sum(pca_res$sdev^2)&#xA;&#xA;# change scale to reflect % variation&#xA;d &amp;lt;- t(data.frame(t(pca_res&lt;span class=&quot;math-container&quot;&gt;$x)*pca_res$&lt;/span&gt;percentVar))&#xA;&#xA;# merge metadata and modified PC scores by row.names (by=0)&#xA;d &amp;lt;- merge(d,colData,by=0) &#xA;&#xA;# make the base plot (PC1 vs PC2)&#xA;g &amp;lt;- ggplot(d,aes(x=PC1,y=PC2,colour=status))&#xA;# add the points&#xA;g &amp;lt;- g + geom_point(size=2)&#xA;# add a &amp;quot;wow&amp;quot; factor to the points&#xA;geom_point(size=4,alpha=0.5)&#xA;# display the plot&#xA;g &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There may be a way to do this with autoplot - e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;autoplot(pca_res,colour=as.numeric(colData$status))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;gets you part of the way there, but I think using the standard ggplot syntax is easy enough anyway.&lt;/p&gt;&#xA;" OwnerUserId="8919" LastActivityDate="2021-06-09T10:54:12.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16095" PostTypeId="1" AcceptedAnswerId="16115" CreationDate="2021-06-09T13:02:28.750" Score="1" ViewCount="307" Body="&lt;p&gt;I detected all markers for each cluster and checked 20 top markers for my clusters one by one. Is there any way that I figure out the selective markers automatically from all the clusters?&lt;/p&gt;&#xA;&lt;p&gt;For instance, in seurat tutorial, I found comparing one cluster two or three other clusters, but I want to see for instance what are the 10 top markers or genes that are expressed only in cluster of interest.&lt;/p&gt;&#xA;" OwnerUserId="12472" LastEditorUserId="57" LastEditDate="2021-06-11T12:34:23.543" LastActivityDate="2021-06-11T12:34:23.543" Title="Find selective markers for one cluster automatically" Tags="|seurat|clustering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16096" PostTypeId="1" AcceptedAnswerId="16097" CreationDate="2021-06-09T14:06:05.977" Score="2" ViewCount="3621" Body="&lt;p&gt;I am currently asking myself how to rename samples in a vcf file. This is necessary, because the naming of several samples included in thr vcf file was unfortunately not be chosen by me.&lt;/p&gt;&#xA;&lt;p&gt;I am loading the vcf file via &lt;code&gt;readVcf()&lt;/code&gt; from the &lt;code&gt;VariantAnnotation&lt;/code&gt; package in R.&lt;/p&gt;&#xA;&lt;p&gt;To check the sample names, I can use &lt;code&gt;samples(header(vcf.file))&lt;/code&gt;, which will give me the names as characters.&lt;/p&gt;&#xA;&lt;p&gt;It looks like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nSsBq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nSsBq.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and so on..&lt;/p&gt;&#xA;&lt;p&gt;(I am actually very sorry that I cannot give a reproducible example here :( )&lt;/p&gt;&#xA;&lt;p&gt;I also have a vector with all the new names. For example&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;new.names &amp;lt;- c(&amp;quot;1&amp;quot;,&amp;quot;2&amp;quot;,&amp;quot;3&amp;quot;,&amp;quot;4&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried to exchange the &amp;quot;old names&amp;quot; with the new.names vector:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;samples(header(vcf.file)) &amp;lt;- new.names&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which leads to the error&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;Error in samples(header(data.vcf)) &amp;lt;- names.sample : &#xA;  could not find function &amp;quot;samples&amp;lt;-&amp;quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also tried to use the &lt;code&gt;rutilstimflutre&lt;/code&gt; package, which has a already designed function for this, but unfortunately the package is outdated and not available for the newest R version.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to do this? Perhaps a better way to assign the new names to the old ones?&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot.&lt;/p&gt;&#xA;" OwnerUserId="10677" LastEditorUserId="650" LastEditDate="2023-06-16T14:11:17.313" LastActivityDate="2023-06-16T14:11:35.043" Title="Renaming samples in vcf file" Tags="|r|vcf|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16097" PostTypeId="2" ParentId="16096" CreationDate="2021-06-09T14:20:58.320" Score="4" Body="&lt;p&gt;Easiest to use &lt;code&gt;bcftools&lt;/code&gt;, if &lt;code&gt;samples.txt&lt;/code&gt; has the new sample names:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools reheader -s samples.txt -o new.bcf old.bcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2021-06-09T14:20:58.320" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16098" PostTypeId="1" CreationDate="2021-06-09T15:38:16.253" Score="1" ViewCount="627" Body="&lt;p&gt;I am learning about the marker genes and clustering by Seurat in scRNA-seq datasets. However, I am confused about the term positive marker and negative marker. What I understand is the marker genes of a cluster is the genes which are highly expressed (high counts) in the cells of that cluster. Let's say cluster A has 200 genes. My questions are:&lt;/p&gt;&#xA;&lt;p&gt;Can I say that those 200 genes are marker genes for cluster A?&#xA;How to determine which marker genes are positive marker genes and which are negative marker genes?&lt;/p&gt;&#xA;" OwnerUserId="12477" LastEditorUserId="104" LastEditDate="2021-12-21T21:55:01.097" LastActivityDate="2021-12-21T21:55:01.097" Title="Positive marker and negative marker in scRNA-seq datasets" Tags="|scrnaseq|gene|seurat|clustering|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16099" PostTypeId="2" ParentId="16089" CreationDate="2021-06-09T16:12:59.887" Score="1" Body="&lt;p&gt;I've found this the way to work with autoplot (ggfortify's prcomp autoplot) - Use a df that has the field you need for color as the first column (to make things easy), then do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;autoplot(prcomp(df[,-1], data=df, color='column1_name')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If the color-by field is not the first column, do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;autoplot(prcomp(df[,!(colnames(df) == &amp;quot;color_by_column_name&amp;quot;)], data = df, color = &amp;quot;color_by_column_name&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&lt;p&gt;That being said, you're going to need to process your data so patients are along rows instead of column. Transpose your dataset so rows are columns and columns become rows; change patient ID to rownames and Gene name to colnames, then &lt;code&gt;merge&lt;/code&gt; the patient category dataset with the expression dataset. Once you merge, your dataset should look like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;         Category    A      B      Z&#xA;Patient1 Healthy     324    431    232&#xA;Patient2 Disease     433    342    124&#xA;Patient3 Healthy     343    124    267&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And then,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;autoplot(prcomp(df[,!(colnames(df) == &amp;quot;Category&amp;quot;)]), data = df, color = &amp;quot;Category&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastEditorUserId="650" LastEditDate="2021-06-09T16:20:46.683" LastActivityDate="2021-06-09T16:20:46.683" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16100" PostTypeId="1" AcceptedAnswerId="16101" CreationDate="2021-06-09T16:24:22.210" Score="1" ViewCount="430" Body="&lt;p&gt;I want to get the array of keys from a map data structure, so I can perform array-based functions against the map in subsequent tasks. I am limited to &lt;code&gt;draft-2&lt;/code&gt; Workflow Description Language (WDL), so I don't have access to the newer spec's extraction functions. I thought I could achieve this by writing a task:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;task extract_keys {&#xA;  Map[String, String] map&#xA;  &#xA;  command {&#xA;    cut -f1 ${write_map(map)}&#xA;  }&#xA;&#xA;  output {&#xA;    Array[String] keys = read_lines(stdout())&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This task works just fine and, as far as I can tell, returns the array of map keys. However, when I try to feed this array into subsequent tasks, I get an error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow test {&#xA;  Map[String, String] my_map = {&#xA;    &amp;quot;a&amp;quot;: &amp;quot;foo&amp;quot;,&#xA;    &amp;quot;b&amp;quot;: &amp;quot;bar&amp;quot;,&#xA;    &amp;quot;c&amp;quot;: &amp;quot;quux&amp;quot;&#xA;  }&#xA;&#xA;  call extract_keys { input: map = my_map }&#xA;&#xA;  scatter (k in extract_keys.keys) {&#xA;    call task_foo { input: k = k, s = my_map[k] }&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Call input and runtime attributes evaluation failed for task_foo&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't have access to the full error message or the logs, unfortunately. The frontend I'm using hides these away.&lt;/p&gt;&#xA;&lt;p&gt;(Incidentally, I'm aware that you can scatter over maps, although it's undocumented, per &lt;a href=&quot;https://github.com/openwdl/wdl/issues/106#issuecomment-356047538&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/openwdl/wdl/issues/106#issuecomment-356047538&lt;/a&gt;. This code is just illustrative; I wish to do other things, such as &lt;code&gt;length&lt;/code&gt; and &lt;code&gt;cross&lt;/code&gt;, that &lt;em&gt;only&lt;/em&gt; accept arrays.)&lt;/p&gt;&#xA;&lt;p&gt;Is it not possible to scatter over the output of a previous task?&lt;/p&gt;&#xA;" OwnerUserId="7482" LastEditorUserId="57" LastEditDate="2021-06-11T12:25:54.123" LastActivityDate="2021-06-11T12:25:54.123" Title="Extracting WDL map keys as a task" Tags="|wdl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16101" PostTypeId="2" ParentId="16100" CreationDate="2021-06-09T16:52:39.277" Score="4" Body="&lt;p&gt;I seem to have solved this myself by virtue of the linked undocumented feature (scattering over a map). It turns out you don't need a separate task at all. This works:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow test {&#xA;  Map[String, String] my_map = {&#xA;    &amp;quot;a&amp;quot;: &amp;quot;foo&amp;quot;,&#xA;    &amp;quot;b&amp;quot;: &amp;quot;bar&amp;quot;,&#xA;    &amp;quot;c&amp;quot;: &amp;quot;quux&amp;quot;&#xA;  }&#xA;&#xA;  scatter (pairs in my_map) {&#xA;    String keys = pairs.left&#xA;    # String values = pairs.right&#xA;  }&#xA;&#xA;  scatter (k in keys) {&#xA;    call task_foo { input: k = k, s = my_map[k] }&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7482" LastActivityDate="2021-06-09T16:52:39.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16102" PostTypeId="1" AcceptedAnswerId="16107" CreationDate="2021-06-09T17:09:05.047" Score="1" ViewCount="105" Body="&lt;p&gt;Hello every one i need help in editting my .bim file. So my .bim file looked like following. I want to convert the ids with chromosomal position from &lt;code&gt;chr1_867635_C_T&lt;/code&gt; to &lt;code&gt;chr1_867635&lt;/code&gt; while others remained as it is. I want to remove all the alleles from my SNP IDs, not just specific ones.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1       rs2880024             866893  A       G&#xA;1       chr1_867635_C_T       867635  A       G&#xA;1       chr1_869303_C_T       869303  A       G&#xA;1       chr1_873558_G_T       873558  C       A&#xA;1       exm2275405           879439  A       G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8654" LastEditorUserId="73" LastEditDate="2021-06-10T12:16:30.820" LastActivityDate="2021-06-10T15:20:15.210" Title="How do I remove allele annotations from SNP Ids in .bim file?" Tags="|snp|plink|linux|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16104" PostTypeId="2" ParentId="16098" CreationDate="2021-06-09T18:49:18.947" Score="1" Body="&lt;p&gt;In scRNA-seq clusters are groups of cells with similar transcriptomes typically defined by graph based methods (e.g. leiden, louvain) or more rarely k-means.&lt;/p&gt;&#xA;&lt;p&gt;Marker genes could be genes that are known to be highly expressed (or better specifically expressed) in a particular cell type (e.g. CD3E is a T cell marker). You can visualize the expression of such marker genes on a UMAP/t-SNE without doing any clustering.&lt;/p&gt;&#xA;&lt;p&gt;When talking about positive and negative markers, I assume the reference is too differentially expressed genes between different clusters. Typically simple differential expression tests are conducted (e.g. Wilcoxon rank sum) between each individual cluster and the other collective clusters. The upregulated differentially expressed genes could be considered positive markers and downregulated genes negative markers.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2021-06-09T18:49:18.947" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16105" PostTypeId="2" ParentId="16098" CreationDate="2021-06-10T01:45:05.993" Score="0" Body="&lt;p&gt;A positive marker is a gene that is overexpressed in a cell population compared to other populations.&lt;/p&gt;&#xA;&lt;p&gt;A negative marker is a gene that is underexpressed in a cell population compared to other populations.&lt;/p&gt;&#xA;&lt;p&gt;I think the Seurat FindMarkers function has an option to output Log2FoldChange as well as the marker names. In this case the positive markers will have a positive Log2FoldChange, and the negative markers will have a negative Log2FoldChange.&lt;/p&gt;&#xA;&lt;p&gt;Both positive and negative markers can be used to describe a cell population. In the research that I've done in the past, there's a particular type of dendritic cell that is only distinguished from another dendritic cell type based on cell surface markers that it &lt;em&gt;doesn't&lt;/em&gt; have; our own name for these DCs was &amp;quot;Triple negative&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-06-10T01:45:05.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16107" PostTypeId="2" ParentId="16102" CreationDate="2021-06-10T08:54:06.520" Score="4" Body="&lt;p&gt;The simplest approach is to remove the first occurrence of a &lt;code&gt;_&lt;/code&gt; followed by a capital letter, another &lt;code&gt;_&lt;/code&gt; and another capital letter from each line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ sed 's/_[A-Z]_[A-Z]//' file &#xA;1       rs2880024             866893  A       G&#xA;1       chr1_867635       867635  A       G&#xA;1       chr1_869303       869303  A       G&#xA;1       chr1_873558       873558  C       A&#xA;1       exm2275405           879439  A       G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alternatively, find the first occurrence of &lt;code&gt;chr&lt;/code&gt; followed by numbers, then an underscore and more numbers and remove all non-whitespace characters after the last numbers (note that this assumes GNU &lt;code&gt;sed&lt;/code&gt;, the default on Linux systems):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ sed -E 's/(\schr[0-9]+_[0-9]+)_\S+/\1/' file &#xA;1       rs2880024             866893  A       G&#xA;1       chr1_867635       867635  A       G&#xA;1       chr1_869303       869303  A       G&#xA;1       chr1_873558       873558  C       A&#xA;1       exm2275405           879439  A       G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If your file is more varied than you show, you can use this to ensure you only make the change on the 2nd field:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ awk -F'[ ]' '{ sub(/_[A-Z]_[A-Z]/,&amp;quot;&amp;quot;,$2)}1' file &#xA;1       rs2880024             866893  A       G&#xA;1       chr1_867635_C_T       867635  A       G&#xA;1       chr1_869303_C_T       869303  A       G&#xA;1       chr1_873558_G_T       873558  C       A&#xA;1       exm2275405           879439  A       G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or, the same but only matching A, C, T or G:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ awk -F'[ ]' '{ sub(/_[ACTG]_[ACTG]/,&amp;quot;&amp;quot;,$2)}1' file &#xA;1       rs2880024             866893  A       G&#xA;1       chr1_867635_C_T       867635  A       G&#xA;1       chr1_869303_C_T       869303  A       G&#xA;1       chr1_873558_G_T       873558  C       A&#xA;1       exm2275405           879439  A       G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2021-06-10T15:20:15.210" LastActivityDate="2021-06-10T15:20:15.210" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16109" PostTypeId="1" CreationDate="2021-06-10T12:01:11.160" Score="2" ViewCount="580" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9474791/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Consider this minimal VCF file which contains a multi-allelic call made by a somatic variant caller on a human tumor sample:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.3&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&amp;quot;Genotype&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=AD,Number=R,Type=Integer,Description=&amp;quot;Allelic depths for the ref and alt alleles in the order listed&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=AF,Number=A,Type=Float,Description=&amp;quot;Allele fractions of alternate alleles in the tumor&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=DP,Number=1,Type=Integer,Description=&amp;quot;Approximate read depth (reads with MQ=255 or with bad mates are filtered)&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=F1R2,Number=R,Type=Integer,Description=&amp;quot;Count of reads in F1R2 pair orientation supporting each allele&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=F2R1,Number=R,Type=Integer,Description=&amp;quot;Count of reads in F2R1 pair orientation supporting each allele&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=SB,Number=4,Type=Integer,Description=&amp;quot;Per-sample component statistics which comprise the Fisher's Exact Test to detect strand bias.&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=DP,Number=1,Type=Integer,Description=&amp;quot;Approximate read depth; some reads may have been filtered&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=clustered_events,Description=&amp;quot;Clustered events observed in the tumor&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=multiallelic,Description=&amp;quot;Site filtered because too many alt alleles pass tumor LOD&amp;quot;&amp;gt;&#xA;#CHROM  POS     ID      REF     ALT     QUAL    FILTER  INFO    FORMAT  foo&#xA;chr10   43613843        .       GC      TC,CC,TT        .       clustered_events;multiallelic   DP=5180 GT:AD:AF:DP:F1R2:F2R1:SB        0/1/2/3:0,5154,11,15:0.994,0.00229,0.00306:5180:0,2507,4,8:0,2576,6,5:0,0,2585,2595&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, we have a site with three ALT alleles and a genotype of &lt;code&gt;0/1/2/3&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VCF specification&lt;/a&gt; has this to say about the genotype field:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;GT (String):  Genotype, encoded as allele values separated by either of/or|.  The allele values are 0 for the reference allele (what is in the REF field),  1 for the first allele listed in ALT, 2 for the second allele list in ALT and so on.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As I understand the VCF specification, given the ALT field &lt;code&gt;TC,CC,TT&lt;/code&gt;, the genotype of &lt;code&gt;0/1/2/3&lt;/code&gt; is telling me that the reference sequence is absent but there are heterozygous calls for each of the three variants in ALT. This makes sense because this is a tumor sample, so presumably what we have here is three populations of cells, some with the GC=&amp;gt;TC change, some with GC=&amp;gt;CC and some with GC=&amp;gt;TT. Given the very high support for the GC=&amp;gt;TC change, I would assume this is actually a germline variant and there are small populations of tumor cells in my sample with either the GC=&amp;gt;CC or the GC=&amp;gt;TT variant. Alternatively, and quite likely, the latter two variants are sequencing artefacts.&lt;/p&gt;&#xA;&lt;p&gt;Regardless, I need to split these into one variant per line. As I understand the specification, after splitting, I should be left with three variants and these specific genotypes (I am showing only the genotype here, for simplicity):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr10   43613843   .   GC    TC   GT 1/0&#xA;chr10   43613843   .   GC    CC   GT 1/0&#xA;chr10   43613843   .   GC    TT   GT 1/0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is this actually correct? Unfortunately, standard tools like &lt;code&gt;bcftools&lt;/code&gt; produce something different:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools norm -m -any foo.vcf 2&amp;gt;/dev/null | grep chr10 | awk '{print $1,$2,$3,$4,$5,$9,$10}'&#xA;chr10 43613843 . GC TC GT:AD:AF:DP:F1R2:F2R1:SB 0/1/0/0:0,5154:0.994:5180:0,2507:0,2576:0,0,2585,2595&#xA;chr10 43613843 . GC CC GT:AD:AF:DP:F1R2:F2R1:SB 0/0/1/0:0,11:0.00229:5180:0,4:0,6:0,0,2585,2595&#xA;chr10 43613843 . GC TT GT:AD:AF:DP:F1R2:F2R1:SB 0/0/0/1:0,15:0.00306:5180:0,8:0,5:0,0,2585,2595&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here, we have sites with a single ALT variant but which have multiple genotypes and that doesn't make sense to me. How can we have a &lt;code&gt;0/0/0/1&lt;/code&gt; genotype and only one ALT allele? Is &lt;code&gt;bcftools&lt;/code&gt; assuming this is a triploid genome?  Is this a bug in &lt;code&gt;bcftools&lt;/code&gt; or am I misunderstanding the VCF specification? What is the correct way to display the genotype after splitting this kind of multi-allelic call?&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="73" LastEditDate="2022-06-21T09:34:25.053" LastActivityDate="2022-06-21T09:34:25.053" Title="What is the correct way of setting the genotype after splitting multi-allelic sites in a VCF file?" Tags="|vcf|genotyping|bcftools|somatic|multi-allelic|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16110" PostTypeId="1" AcceptedAnswerId="16111" CreationDate="2021-06-10T12:36:54.903" Score="1" ViewCount="1966" Body="&lt;p&gt;I am trying to update my rsIDs in .bim file by using --update-name flag in plink but getting this error message. what could be the reason of it?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plink --bfile updated_ff  --update-name updated_ref.csv --make-bed --out test_updatedrs&#xA;&#xA;PLINK v1.90b6.21 64-bit (19 Oct 2020)&#xA;Options in effect:&#xA;  --bfile updated_ff&#xA;  --make-bed&#xA;  --out test_updatedrs&#xA;  --update-name updated_ref.csv&#xA;&#xA;Hostname: login1&#xA;Working directory: /gpfs/home/bsc08/bsc08856/phs001273_dbgap&#xA;Start time: Thu Jun 10 14:23:44 2021&#xA;&#xA;Random number seed: 1623327824&#xA;96310 MB RAM detected; reserving 48155 MB for main workspace.&#xA;Allocated 15236 MB successfully, after larger attempt(s) failed.&#xA;Error: Line 1 of updated_ref.csv has fewer tokens than expected.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My updated_ref.csv file looks like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Name,chrpos&#xA;rs3793692,chr10_100008436&#xA;rs1983865,chr10_100016339&#xA;newrs150271620,chr10_100017402&#xA;rs10736126,chr10_100025924&#xA;chr10_100044928_A_G,chr10_100044928&#xA;rs7905091,chr10_100046078&#xA;rs6584182,chr10_100055644&#xA;chr10_100058710_C_T,chr10_100058710&#xA;rs4360594,chr10_10007547&#xA;rs927739,chr10_100077732&#xA;rs4917817,chr10_100085888&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;while my .bim looks like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1 rs3094315 0 752566 G A&#xA;1 rs3131972 0 752721 A G&#xA;1 chr1770377 0 770377 T A&#xA;1 rs11240777 0 798959 A G&#xA;1 chr1834753 0 834753 C G&#xA;1 rs72890788 0 836924 A G&#xA;1 rs4970383 0 838555 A C&#xA;1 chr1840753 0 840753 G A&#xA;&#xA;1 rs4475691 0 846808 A G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8654" LastEditorUserId="3694" LastEditDate="2021-06-10T20:06:05.573" LastActivityDate="2021-06-10T20:06:05.573" Title="Error: &quot;fewer tokens than expected&quot; while using --update-name flag in plink" Tags="|plink|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16111" PostTypeId="2" ParentId="16110" CreationDate="2021-06-10T12:49:39.123" Score="2" Body="&lt;p&gt;The reason is (likely) that plink is expecting a space separated file for the &lt;code&gt;--update-name&lt;/code&gt; argument:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;--update-name &amp;lt;filename&amp;gt; [new ID col. number] [old ID col.] [skip]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You've given it a comma-separated file, so it thinks there is only one column, hence the error. You also don't want column names.&lt;/p&gt;&#xA;&lt;p&gt;Change the sep by:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sed 's/,/ /g' updated_ref.csv &amp;gt; updated_ref.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or, edit the file in place, and keep a backup of the original named &lt;code&gt;updated_ref.csv.bak&lt;/code&gt; :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sed -i.bak 's/,/ /g' updated_ref.csv &amp;gt; updated_ref.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="298" LastEditDate="2021-06-10T13:19:48.300" LastActivityDate="2021-06-10T13:19:48.300" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16112" PostTypeId="1" CreationDate="2021-06-10T13:17:50.737" Score="1" ViewCount="672" Body="&lt;p&gt;I have a bed file of coordinates and would like to clip anything outside these regions in my aligned bam files. However, trying programs such as bedtools intersect and samtools ampliconclip (on complementary bed file), they always let some reads slip through outside the ranges. I tried using hard-clip and both-ends options for ampliconclip aswell.&lt;/p&gt;&#xA;&lt;p&gt;Example (top is clipped bam, middle is unclipped bam and bottom is bed file with regions I would like to keep):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VUC23.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VUC23.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Although most reads outside the designed amplicons are removed, there are still reads outside the regions in the Designed_pool1 file.&#xA;The problem is that the bed file contains amplicons and outside the amplicons are primers where I do not want to call variants. I cannot get the exact primer sequences because I don't know their length and the manufacturer does not state them, so I cannot use programs that require primer sequences.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have some experience with this and know why the programs behave this way?&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&lt;p&gt;The exact commands I have used are:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bedtools intersect -a $BamFile -b $BedFile &amp;gt; $BamOut&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools ampliconclip --hard-clip --both-ends -b $ComplBedFile $BamFile &amp;gt; $BamOut&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;as well as without --hard-clip and --both-ends options. All methods let some sequences stay out of the regions in bed files.&lt;/p&gt;&#xA;&lt;p&gt;I am not completely familiar with the bedpe format, but I am not using paired end data. I cannot get primer start and stop positions, only amplicon positions.&lt;/p&gt;&#xA;" OwnerUserId="12151" LastEditorUserId="12151" LastEditDate="2021-06-11T06:36:38.810" LastActivityDate="2022-04-02T00:05:15.480" Title="Precisely clipping bam file to bed coordinates" Tags="|base-clipping|" AnswerCount="2" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16115" PostTypeId="2" ParentId="16095" CreationDate="2021-06-10T19:04:57.067" Score="2" Body="&lt;p&gt;Directly copy-pasting from &lt;a href=&quot;https://satijalab.org/seurat/articles/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;one of the Seurat vignettes&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# find markers for every cluster compared to all remaining cells, report only the positive ones&#xA;pbmc.markers &amp;lt;- FindAllMarkers(pbmc, only.pos = TRUE, min.pct = 0.25, logfc.threshold = 0.25)&#xA;pbmc.markers %&amp;gt;%&#xA;    group_by(cluster) %&amp;gt;%&#xA;    top_n(n = 2, wt = avg_log2FC)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Change &lt;code&gt;n&lt;/code&gt; to 10 in the &lt;code&gt;top_n()&lt;/code&gt; call and filter for your cluster of interest and you will get what you want.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2021-06-10T19:04:57.067" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16116" PostTypeId="2" ParentId="16096" CreationDate="2021-06-10T20:41:00.293" Score="0" Body="&lt;p&gt;Here is an example in python how to rename samples using &lt;code&gt;pyvcf&lt;/code&gt; &lt;a href=&quot;https://sbslee-fuc.readthedocs.io/en/latest/api.html#module-fuc.api.pyvcf&quot; rel=&quot;nofollow noreferrer&quot;&gt;submodule&lt;/a&gt; I wrote:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; from fuc import pyvcf&#xA;&amp;gt;&amp;gt;&amp;gt; data = {&#xA;...     'CHROM': ['chr1', 'chr2'],&#xA;...     'POS': [100, 101],&#xA;...     'ID': ['.', '.'],&#xA;...     'REF': ['G', 'T'],&#xA;...     'ALT': ['A', 'C'],&#xA;...     'QUAL': ['.', '.'],&#xA;...     'FILTER': ['.', '.'],&#xA;...     'INFO': ['.', '.'],&#xA;...     'FORMAT': ['GT', 'GT'],&#xA;...     'Steven': ['0/1', '1/1'],&#xA;...     'Sara': ['0/0', '0/1'],&#xA;... }&#xA;&amp;gt;&amp;gt;&amp;gt; vf = pyvcf.VcfFrame.from_dict([], data)&#xA;&amp;gt;&amp;gt;&amp;gt; # vf = pyvcf.VcfFrame.from_file('your_file.vcf')&#xA;&amp;gt;&amp;gt;&amp;gt; vf.df&#xA;  CHROM  POS ID REF ALT QUAL FILTER INFO FORMAT Steven Sara&#xA;0  chr1  100  .   G   A    .      .    .     GT    0/1  0/0&#xA;1  chr2  101  .   T   C    .      .    .     GT    1/1  0/1&#xA;&amp;gt;&amp;gt;&amp;gt; vf.df = vf.df.rename(columns={'Steven': 'John', 'Sara': 'Rachel'})&#xA;&amp;gt;&amp;gt;&amp;gt; vf.df&#xA;  CHROM  POS ID REF ALT QUAL FILTER INFO FORMAT John Rachel&#xA;0  chr1  100  .   G   A    .      .    .     GT  0/1    0/0&#xA;1  chr2  101  .   T   C    .      .    .     GT  1/1    0/1&#xA;&amp;gt;&amp;gt;&amp;gt; # vf.to_file('renamed.vcf')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12502" LastEditorUserId="57" LastEditDate="2021-06-11T12:39:16.257" LastActivityDate="2021-06-11T12:39:16.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16117" PostTypeId="2" ParentId="16003" CreationDate="2021-06-10T20:52:45.430" Score="1" Body="&lt;p&gt;If you are familiar with Python, check out the &lt;code&gt;pyvcf.VcfFrame.filter_bed()&lt;/code&gt; &lt;a href=&quot;https://sbslee-fuc.readthedocs.io/en/latest/api.html#fuc.api.pyvcf.VcfFrame.filter_bed&quot; rel=&quot;nofollow noreferrer&quot;&gt;method&lt;/a&gt; I wrote:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from fuc import pyvcf&#xA;vf = pyvcf.VcfFrame.from_file('your_vcf.vcf')&#xA;filtered_vf = vf.filter_bed('your_bed.bed', opposite=True)&#xA;filtered_vf.to_file('filtered.vcf')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12502" LastEditorUserId="12502" LastEditDate="2021-06-10T20:59:48.583" LastActivityDate="2021-06-10T20:59:48.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16118" PostTypeId="1" CreationDate="2021-06-10T20:55:21.527" Score="1" ViewCount="62" Body="&lt;p&gt;I am trying to query metadata from the Genomic Data Commons (GDC) Data Portal and used the sample code from its website:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;import json&#xA;&#xA;cases_endpt = 'https://api.gdc.cancer.gov/cases'&#xA;&#xA;# The 'fields' parameter is passed as a comma-separated string of single names.&#xA;fields = [&amp;quot;submitter_id&amp;quot;,&#xA;          &amp;quot;case_id&amp;quot;,&#xA;          &amp;quot;primary_site&amp;quot;,&#xA;          &amp;quot;disease_type&amp;quot;,&#xA;          &amp;quot;diagnoses.vital_status&amp;quot;&#xA;         ]&#xA;&#xA;fields = ','.join(fields)&#xA;&#xA;params = {&amp;quot;fields&amp;quot;: fields,&#xA;          &amp;quot;format&amp;quot;: &amp;quot;TSV&amp;quot;,&#xA;          &amp;quot;size&amp;quot;: &amp;quot;100&amp;quot;&#xA;         }&#xA;&#xA;response = requests.get(cases_endpt, params = params)&#xA;&#xA;print(response.content)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, I am not getting a tsv file in return. I am ultimately getting an FPKM instead. Any help is appreciated!&lt;/p&gt;&#xA;" OwnerUserId="12503" LastEditorUserId="57" LastEditDate="2021-06-11T12:27:29.860" LastActivityDate="2021-06-11T12:27:29.860" Title="querying metadata (GDC Data Portal) in python3" Tags="|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16120" PostTypeId="2" ParentId="15612" CreationDate="2021-06-10T21:01:55.400" Score="2" Body="&lt;p&gt;You can filter records in Python for example using &lt;code&gt;pyvcf.VcfFrame.filter_flagany()&lt;/code&gt; &lt;a href=&quot;https://sbslee-fuc.readthedocs.io/en/latest/api.html#fuc.api.pyvcf.VcfFrame.filter_flagany&quot; rel=&quot;nofollow noreferrer&quot;&gt;method&lt;/a&gt; I wrote:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; from fuc import pyvcf&#xA;&amp;gt;&amp;gt;&amp;gt; data = {&#xA;...     'CHROM': ['chr1', 'chr1', 'chr1', 'chr1'],&#xA;...     'POS': [100, 101, 102, 103],&#xA;...     'ID': ['.', '.', '.', '.'],&#xA;...     'REF': ['G', 'T', 'A', 'C'],&#xA;...     'ALT': ['A', 'C', 'T', 'A'],&#xA;...     'QUAL': ['.', '.', '.', '.'],&#xA;...     'FILTER': ['.', '.', '.', '.'],&#xA;...     'INFO': ['DB', 'DB;H2;Match=EXACT', 'DB;H2;Match=EXACT', '.'],&#xA;...     'FORMAT': ['GT', 'GT', 'GT', 'GT'],&#xA;...     'Steven': ['0/0', '0/1', '0/1', '0/0'],&#xA;... }&#xA;&amp;gt;&amp;gt;&amp;gt; vf = pyvcf.VcfFrame.from_dict([], data)&#xA;&amp;gt;&amp;gt;&amp;gt; # vf = pyvcf.VcfFrame.from_file('your_vcf.vcf')&#xA;&amp;gt;&amp;gt;&amp;gt; vf.df&#xA;  CHROM  POS ID REF ALT QUAL FILTER               INFO FORMAT Steven&#xA;0  chr1  100  .   G   A    .      .                 DB     GT    0/0&#xA;1  chr1  101  .   T   C    .      .  DB;H2;Match=EXACT     GT    0/1&#xA;2  chr1  102  .   A   T    .      .  DB;H2;Match=EXACT     GT    0/1&#xA;3  chr1  103  .   C   A    .      .                  .     GT    0/0&#xA;&amp;gt;&amp;gt;&amp;gt; vf.filter_flagany(['Match=EXACT'], opposite=True).df&#xA;  CHROM  POS ID REF ALT QUAL FILTER INFO FORMAT Steven&#xA;0  chr1  100  .   G   A    .      .   DB     GT    0/0&#xA;1  chr1  103  .   C   A    .      .    .     GT    0/0&#xA;&amp;gt;&amp;gt;&amp;gt; # vf.to_file('filtered.vcf')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12502" LastEditorUserId="57" LastEditDate="2021-06-11T15:58:22.827" LastActivityDate="2021-06-11T15:58:22.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16121" PostTypeId="2" ParentId="12824" CreationDate="2021-06-11T02:04:12.087" Score="5" Body="&lt;p&gt;There's a benchmark that compared dozens of compressors on DNA data, including human genome: &lt;a href=&quot;http://kirr.dyndns.org/sequence-compression-benchmark/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://kirr.dyndns.org/sequence-compression-benchmark/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In particular, this table shows file size of compressed human genome, produced by the strongest setting of each compressor: &lt;a href=&quot;http://kirr.dyndns.org/sequence-compression-benchmark/?d=Homo+sapiens+GCA_000001405.28+%283.31+GB%29&amp;amp;cs=1&amp;amp;cg=1&amp;amp;com=yes&amp;amp;src=all&amp;amp;nt=4&amp;amp;only-best=1&amp;amp;bn=1&amp;amp;bm=size&amp;amp;sm=same&amp;amp;tm0=name&amp;amp;tm1=size&amp;amp;button=Show+table&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://kirr.dyndns.org/sequence-compression-benchmark/?d=Homo+sapiens+GCA_000001405.28+%283.31+GB%29&amp;amp;cs=1&amp;amp;cg=1&amp;amp;com=yes&amp;amp;src=all&amp;amp;nt=4&amp;amp;only-best=1&amp;amp;bn=1&amp;amp;bm=size&amp;amp;sm=same&amp;amp;tm0=name&amp;amp;tm1=size&amp;amp;button=Show+table&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The best result is currently 657 MB on that particular genome build (GCA_000001405.28), when compressed with fastqz.&lt;/p&gt;&#xA;&lt;p&gt;By the way, xz (mentioned in another answer) is currently number 20 in that table. (Though the genome is un-modified straight from the database, not the &amp;quot;cleaned&amp;quot; form.)&lt;/p&gt;&#xA;&lt;p&gt;Disclosure: I worked on that benchmark.&lt;/p&gt;&#xA;&lt;p&gt;Note: What is being compressed here is a FASTA file containing a particular assembly of a human genome. Compared to an actual genome it has this extra content: Sequence names and brakes between contigs, end of line characters, N characters representing unknown bases, and soft-masked repeats (represented as lowercase). On the other hand, it also misses some information from the actual genome: 1. It's an unphased haploid sequence, while the real genome is diploid (two copies of each chromosome). 2. It includes no epigenetic information (DNA methylation, histone modifications), nucleosome positioning, 3D structure of packed DNA, etc. So, it's just a digital model of a genome with a certain degree of accuracy, that is useful,  simple, and relatively affordable to obtain. It's a format commonly used for representing genomes in databases.&lt;/p&gt;&#xA;" OwnerUserId="12506" LastEditorUserId="12506" LastEditDate="2021-06-17T21:30:11.577" LastActivityDate="2021-06-17T21:30:11.577" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="16122" PostTypeId="1" AcceptedAnswerId="16123" CreationDate="2021-06-11T07:53:52.167" Score="1" ViewCount="739" Body="&lt;p&gt;I have a list of ko IDs from which I want to get the pathway names. How can I get that? I have tried &lt;a href=&quot;https://www.kegg.jp/kegg/tool/map_pathway.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt;, but it does not give me any result. What to do?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  ko00010   ko00030 ko00061 ko00120 ko00121 ko00190 ko00195 ko00230 ko00250 ko00253 ko00270 ko00300 ko00330 ko00400 ko00401 ko00410 ko00430 ko00450 ko00460 ko00471 ko00472 ko00500 ko00511 ko00520 ko00521 ko00523 ko00531 ko00550 ko00562 ko00604 ko00620 ko00625 ko00640&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="12271" LastActivityDate="2021-06-11T08:41:08.000" Title="How to get KEGG pathway names from a list of 'ko' IDs?" Tags="|kegg|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16123" PostTypeId="2" ParentId="16122" CreationDate="2021-06-11T08:41:08.000" Score="2" Body="&lt;p&gt;You can use the KEGG API:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ko_list=&amp;quot;ko00010 ko00030 ko00061 ko00120 ko00121&#xA;         ko00190 ko00195 ko00230 ko00250 ko00253&#xA;         ko00270 ko00300 ko00330 ko00400 ko00401&#xA;         ko00410 ko00430 ko00450 ko00460 ko00471&#xA;         ko00472 ko00500 ko00511 ko00520 ko00521&#xA;         ko00523 ko00531 ko00550 ko00562 ko00604&#xA;         ko00620 ko00625 ko00640&amp;quot;&#xA;&#xA;for i in &lt;span class=&quot;math-container&quot;&gt;${ko_list}&#xA;do&#xA;    hit=$&lt;/span&gt;(curl -# http://rest.kegg.jp/list/pathway | grep -w &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${i/ko/map}&quot;)&#xA;    echo -e &quot;$&lt;/span&gt;{i}\t${hit}&amp;quot; | cut -f1,3 &amp;gt;&amp;gt; ko_pathways.txt&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or if your list is too long you can change the input and use a file with a list of ids.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;while read line&#xA;do&#xA;    hit=&lt;span class=&quot;math-container&quot;&gt;$(curl -# http://rest.kegg.jp/list/pathway | grep -w &quot;$&lt;/span&gt;{line/ko/map}&amp;quot;)&#xA;    echo -e &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${line}\t$&lt;/span&gt;{hit}&amp;quot; | cut -f1,3 &amp;gt;&amp;gt; ko_pathways.txt&#xA;done &amp;lt; ko_list.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The result will be something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; ko00010    Glycolysis / Gluconeogenesis&#xA; ko00030    Pentose phosphate pathway&#xA; ko00061    Fatty acid biosynthesis&#xA; ko00120    Primary bile acid biosynthesis&#xA; ko00121    Secondary bile acid biosynthesis&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2021-06-11T08:41:08.000" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16124" PostTypeId="2" ParentId="16118" CreationDate="2021-06-11T09:54:14.120" Score="1" Body="&lt;p&gt;You can use &lt;code&gt;str()&lt;/code&gt; to convert bytes to strings and &lt;code&gt;print()&lt;/code&gt; with the proper separator.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;from io import StringIO&#xA;import pandas as pd&#xA;&#xA;cases_endpt = 'https://api.gdc.cancer.gov/cases'&#xA;&#xA;fields = [&amp;quot;submitter_id&amp;quot;,&#xA;          &amp;quot;case_id&amp;quot;,&#xA;          &amp;quot;primary_site&amp;quot;,&#xA;          &amp;quot;disease_type&amp;quot;,&#xA;          &amp;quot;diagnoses.vital_status&amp;quot;&#xA;          ]&#xA;&#xA;fields = ','.join(fields)&#xA;&#xA;params = {&amp;quot;fields&amp;quot;: fields,&#xA;          &amp;quot;format&amp;quot;: &amp;quot;TSV&amp;quot;,&#xA;          &amp;quot;size&amp;quot;: &amp;quot;100&amp;quot;&#xA;          }&#xA;&#xA;response = requests.get(cases_endpt, params=params)&#xA;&#xA;content = str(response.content, 'utf-8')&#xA;&#xA;print(content, sep='\t', end='\r')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you want to convert to a data frame you can use &lt;code&gt;StringIO()&lt;/code&gt; to load a data frame from an input string directly to a pandas data frame.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data = StringIO(content)&#xA;df = pd.read_csv(data, sep='\t')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;case_id&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;disease_type&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;id&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;primary_site&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;submitter_id&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;375436b3-66ac-4d5e-b495-18a96d812a69&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Adenomas and Adenocarcinomas&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;375436b3-66ac-4d5e-b495-18a96d812a69&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Rectum&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;TCGA-F5-6810&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;38f93ca8-4971-4932-b592-0a17eaece2ad&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Ductal and Lobular Neoplasms&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;38f93ca8-4971-4932-b592-0a17eaece2ad&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Pancreas&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;C3L-02115&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;2&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;5ea1361a-050d-4fa0-be59-b530c456b6e3&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Ductal and Lobular Neoplasms&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;5ea1361a-050d-4fa0-be59-b530c456b6e3&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Pancreas&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;C3L-00928&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;3&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;690378f3-e38a-4b25-8489-05a3b4bd40b6&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Adenomas and Adenocarcinomas&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;690378f3-e38a-4b25-8489-05a3b4bd40b6&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Kidney&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;C3N-01175&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;4&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;6b4f086e-4ef9-4f30-874e-8a155b2e3bab&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Squamous Cell Neoplasms&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;6b4f086e-4ef9-4f30-874e-8a155b2e3bab&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Other and ill-defined sites&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;C3N-04276&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;" OwnerUserId="4282" LastActivityDate="2021-06-11T09:54:14.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16125" PostTypeId="2" ParentId="15952" CreationDate="2021-06-11T18:25:58.100" Score="2" Body="&lt;p&gt;Sadly the 'contact us' email at GEO didn't help at all.&lt;/p&gt;&#xA;&lt;p&gt;I ended up probably re-inventing the wheel with this script:&#xA;&lt;a href=&quot;https://gist.github.com/CholoTook/60968e3ab6d90cb8fd19be55a25592f1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gist.github.com/CholoTook/60968e3ab6d90cb8fd19be55a25592f1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;YMMV&lt;/p&gt;&#xA;" OwnerUserId="8875" LastActivityDate="2021-06-11T18:25:58.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16126" PostTypeId="1" AcceptedAnswerId="16148" CreationDate="2021-06-12T01:42:28.073" Score="2" ViewCount="48" Body="&lt;p&gt;This is an open question that came to my mind recently, though it might be that it is not a common use case. The purpose is to find common patterns in a collection of sequences by representing the patterns as PSSMs. However, some of the bases have sequencing errors and are thus missing. Is there a way of integrating these missing bases in the PSSM calculation?&lt;/p&gt;&#xA;&lt;p&gt;For a collection of sequences, deriving a sequence logo is relatively straightforward. For the case:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ACTAGCGT&#xA;ACGTACTC&#xA;ACGATCTC&#xA;ACGTCAGT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Taking the frequencies of each letter at each position would yield the following (pseudocount-corrected) Position Frequency Matrix:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;      A       G     T       C&#xA;0  0.85  0.0375  0.05  0.0625&#xA;1  0.05  0.0375  0.05  0.8625&#xA;2  0.05  0.6375  0.25  0.0625&#xA;3  0.45  0.0375  0.45  0.0625&#xA;4  0.25  0.2375  0.25  0.2625&#xA;5  0.25  0.0375  0.05  0.6625&#xA;6  0.05  0.4375  0.45  0.0625&#xA;7  0.05  0.0375  0.45  0.4625&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And by including background frequencies, we can derive the total Infomation Content for each position, and sum them up to compute the total Infomation Content of the matrix.&lt;/p&gt;&#xA;&lt;p&gt;But what would happen if the analyzed sequences had experimented errors in the sequencing and some of the letters were missing?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ACNNGCGT&#xA;ACGTACTC&#xA;ACGATNTC&#xA;ACNTCAGT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Would you consider &lt;code&gt;N&lt;/code&gt; a new letter of the alphabet and follow the same procedure? (counting ocurrences per position, calculating background frequencies, include it in the sequence logo...)&lt;/li&gt;&#xA;&lt;li&gt;Would you ignore them and normalize each column's frequencies to the number of non-N bases?&lt;/li&gt;&#xA;&lt;li&gt;Would you penalize  positions/sequences bearing &lt;code&gt;N&lt;/code&gt;'s when calculating the IC?&lt;/li&gt;&#xA;&lt;li&gt;Other?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I wanted to start with the case of &lt;code&gt;N&lt;/code&gt; since it's the most generic wildcard, and therefore could be easier to address; but if you have ideas on incorporating other more specific wildcards, feel free to add it :)&lt;/p&gt;&#xA;" OwnerUserId="12308" LastEditorUserId="73" LastEditDate="2021-06-13T14:20:00.757" LastActivityDate="2021-06-15T14:02:20.063" Title="How to deal with wildcards (N) in sequence logos" Tags="|sequence-analysis|motifs|pssm|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16127" PostTypeId="1" CreationDate="2021-06-12T08:05:15.230" Score="1" ViewCount="58" Body="&lt;p&gt;I'm trying to understand why RPKM is not an appropriate way to normalize for RNA seq (I understand the general idea, but I'd like to gain a deeper understanding). So I'm reading the &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/22872506/&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt; by Wagner et al 2012 that suggests TPM as an alternative. However, I'm struggling to understand the following paragraph, and was wondering if anyone can shed some light on what the authors are trying to say:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The reason for the inconsistency of RPKM across samples arises from the normalization by the total number of reads. While rmc as well as qPCR results are ratios of transcript concentrations, the RPKM normalizes a proxy for transcript number by &lt;span class=&quot;math-container&quot;&gt;$r_g \times 10^3 / fl_g$&lt;/span&gt; the number of sequencing reads in millions, &lt;span class=&quot;math-container&quot;&gt;$R / 10^6$&lt;/span&gt;. The latter, however, is not a measure of total transcript number. The relationship between &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; and the total number of transcripts sampled depends on the size distribution of RNA transcripts, which can differ between samples. In a sample with, on average, longer transcripts the same number of reads represents fewer transcripts.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;One part that I'm particularly confused by is: &amp;quot;The RPKM normalizes a proxy for transcript number by &lt;span class=&quot;math-container&quot;&gt;$r_g \times 10^3/fl_g$&lt;/span&gt; the number of sequencing reads in millions, &lt;span class=&quot;math-container&quot;&gt;$R/10^6$&lt;/span&gt;.&amp;quot; What is the &amp;quot;proxy&amp;quot; here?&lt;/p&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="73" LastEditDate="2021-07-13T08:49:36.160" LastActivityDate="2021-07-13T08:49:36.160" Title="Understanding &quot;Measurement of mRNA abundance using RNA-seq data&quot; by Wagner et al 2012" Tags="|rna-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16128" PostTypeId="1" CreationDate="2021-06-12T10:55:47.717" Score="2" ViewCount="401" Body="&lt;p&gt;I have perform a ligand-protein docking using &lt;a href=&quot;http://vina.scripps.edu&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Autodock Vina&lt;/strong&gt;&lt;/a&gt;.&#xA;The result of the docking looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;WARNING: The search space volume &amp;gt; 27000 Angstrom^3 (See FAQ)&#xA;Detected 8 CPUs&#xA;Reading input ... done.&#xA;Setting up the scoring function ... done.&#xA;Analyzing the binding site ... done.&#xA;Using random seed: -1553787135&#xA;Performing search ... done.&#xA;Refining results ... done.&#xA;&#xA;mode |   affinity | dist from best mode&#xA;     | (kcal/mol) | rmsd l.b.| rmsd u.b.&#xA;-----+------------+----------+----------&#xA;   1         -5.9      0.000      0.000.  Pose 1&#xA;   2         -5.7     22.945     25.492.  Pose 2&#xA;   3         -5.5      1.426      2.046.  Pose 3&#xA;   4         -5.5     23.669     25.616&#xA;   5         -5.4     25.783     29.152.  .....&#xA;   6         -5.3     21.146     23.357&#xA;   7         -5.2     20.323     22.545&#xA;   8         -5.2     23.864     26.064&#xA;   9         -5.1     23.422     26.585.  Pose 9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As far as I understand from these statistics Mode 1(Pose 1) is the best.&#xA;However when I actually visualize them in Pymol, Pose 1 has no hydrogen bonding at all&#xA;but Pose 2 has.&lt;/p&gt;&#xA;&lt;p&gt;My question is how can we judge if which of those two Pose is the best to use?&lt;/p&gt;&#xA;&lt;p&gt;Note in figure below Pose 2 has dashed line (Hydrogen bond).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8gY6p.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8gY6p.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12433" LastEditorUserId="12433" LastEditDate="2021-06-12T13:21:24.930" LastActivityDate="2021-06-13T18:41:09.387" Title="How to tell if our ligand-protein docking is good from AutoDock Vina's result" Tags="|protein-structure|docking|structural-biology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16129" PostTypeId="2" ParentId="13160" CreationDate="2021-06-12T12:18:46.790" Score="2" Body="&lt;p&gt;Here is another way to achieve your goal using the &lt;code&gt;fuc vcf_vep&lt;/code&gt; &lt;a href=&quot;https://sbslee-fuc.readthedocs.io/en/latest/cli.html#vcf-vep&quot; rel=&quot;nofollow noreferrer&quot;&gt;command&lt;/a&gt; I wrote:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ fuc vcf_vep in.vcf 'SIFT.str.contains(&amp;quot;deleterious&amp;quot;) and PolyPhen.str.contains(&amp;quot;damaging&amp;quot;)' &amp;gt; out.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For getting help:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ fuc vcf_vep -h&#xA;usage: fuc vcf_vep [-h] [--opposite] [--as_zero] vcf expr&#xA;&#xA;This command will filter a VCF file annotated by Ensemble VEP. It essentially wraps the `pandas.DataFrame.query` method. For details on query expression, please visit the method's documentation page (https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.query.html#pandas-dataframe-query).&#xA;&#xA;examples:&#xA;  &lt;span class=&quot;math-container&quot;&gt;$ fuc vcf_vep in.vcf 'SYMBOL == &quot;TP53&quot;' &amp;gt; out.vcf&#xA;  $&lt;/span&gt; fuc vcf_vep in.vcf 'SYMBOL != &amp;quot;TP53&amp;quot;' &amp;gt; out.vcf&#xA;  &lt;span class=&quot;math-container&quot;&gt;$ fuc vcf_vep in.vcf 'SYMBOL == &quot;TP53&quot;' --opposite &amp;gt; out.vcf&#xA;  $&lt;/span&gt; fuc vcf_vep in.vcf 'Consequence in [&amp;quot;splice_donor_variant&amp;quot;, &amp;quot;stop_gained&amp;quot;]' &amp;gt; out.vcf&#xA;  &lt;span class=&quot;math-container&quot;&gt;$ fuc vcf_vep in.vcf '(SYMBOL == &quot;TP53&quot;) and (Consequence.str.contains(&quot;stop_gained&quot;))' &amp;gt; out.vcf&#xA;  $&lt;/span&gt; fuc vcf_vep in.vcf 'gnomAD_AF &amp;lt; 0.001' &amp;gt; out.vcf&#xA;  $ fuc vcf_vep in.vcf 'gnomAD_AF &amp;lt; 0.001' --as_zero &amp;gt; out.vcf&#xA;&#xA;positional arguments:&#xA;  vcf         Ensemble VEP-annotated VCF file&#xA;  expr        query expression to evaluate&#xA;&#xA;optional arguments:&#xA;  -h, --help  show this help message and exit&#xA;  --opposite  use this flag to return records that don’t meet the said criteria&#xA;  --as_zero   use this flag to treat missing values as zero instead of NaN&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can also do above with Python:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from fuc import pyvcf, pyvep&#xA;vf = pyvcf.VcfFrame.from_file('in.vcf')&#xA;expr = 'SIFT.str.contains(&amp;quot;deleterious&amp;quot;) and PolyPhen.str.contains(&amp;quot;damaging&amp;quot;)'&#xA;filtered_vf = pyvep.filter_query(vf, expr)&#xA;filtered_vf.to_file('out.vcf')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12502" LastEditorUserId="12502" LastEditDate="2021-06-12T12:24:08.290" LastActivityDate="2021-06-12T12:24:08.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16130" PostTypeId="2" ParentId="16127" CreationDate="2021-06-12T20:36:04.560" Score="1" Body="&lt;p&gt;That is why: &lt;a href=&quot;https://www.biostars.org/p/9465851/#9465854&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/9465851/#9465854&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;All these naive per-million scaling methods may fail to correct for library composition differences.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-06-12T20:36:04.560" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16132" PostTypeId="1" CreationDate="2021-06-13T10:02:03.457" Score="2" ViewCount="38" Body="&lt;p&gt;I'm doing my master's thesis on drug repositioning for SARS‑CoV‑2 using AI more specifically network medicine,&lt;/p&gt;&#xA;&lt;p&gt;I found only one human protein-protein interactions (PPIs) map from &lt;a href=&quot;https://www.biorxiv.org/content/early/2020/03/27/2020.03.22.002386&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gordon et al.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The manuscript doesn't include the ace2 protein which the SARS‑CoV‑2 spike protein directly binds with. AVE2 is a host cell surface receptor facilitating virus entry and thereafter replication.&lt;/p&gt;&#xA;&lt;p&gt;Thus, I'm looking for PPI map that includes ACE2 protein but i can't find any&lt;/p&gt;&#xA;&lt;p&gt;Secondly, is there a reason why it's not been included ?&lt;/p&gt;&#xA;&lt;p&gt;PS: I have very basic knowledge of this domain&lt;/p&gt;&#xA;" OwnerUserId="12535" LastEditorUserId="3694" LastEditDate="2022-02-27T15:56:30.233" LastActivityDate="2022-02-27T15:56:30.233" Title="Why isn’t ace2 included in SARS‑CoV‑2 PPI map?" Tags="|proteins|covid-19|sars-cov-2|protein-protein-interaction|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16134" PostTypeId="2" ParentId="16128" CreationDate="2021-06-13T18:41:09.387" Score="2" Body="&lt;p&gt;It is best to contextualise the numbers. -1 kcal/mol is about the potential energy gained from a hydrogen bond —technically described in the r^6 part of the Lenard–Jones term, it is also the average collision energy of a water molecule at 37°C as that is RT (&lt;span class=&quot;math-container&quot;&gt;$\frac{k_b\cdot T}{N_A}$&lt;/span&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/KT_(energy)&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki&lt;/a&gt;)under a &lt;a href=&quot;https://en.wikipedia.org/wiki/Maxwell%E2%80%93Boltzmann_distribution&quot; rel=&quot;nofollow noreferrer&quot;&gt;Maxwell–Boltzmann distribution&lt;/a&gt;. A salt bridge –2 kcal/mol (Columbic force term).&#xA;So your scores are not very low, hence why you are counting two hydrogen bonds. Although you can also see a lovely sulfur-pi interaction which is –1 to –2 kcal/mol, so those bonds alone are probably making a –4 kcal/mol contribution, so I am guessing that some terms may be horrendous, such as repulsion forces etc.&#xA;A nice metric is doing a conversion to &lt;a href=&quot;https://en.wikipedia.org/wiki/Ligand_efficiency&quot; rel=&quot;nofollow noreferrer&quot;&gt;ligand efficiency&lt;/a&gt;, which weeds out affinities driven by size of the molecule... in this case most of the molecule is doing nothing.&#xA;Also, most programs have an accuracy of 1 kcal/mol or higher.&lt;/p&gt;&#xA;&lt;p&gt;So one cannot say what ∆∆G is the best with the data at hand due to noise, but one can say that the ∆∆G is not low enough... Sorry.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-06-13T18:41:09.387" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="16135" PostTypeId="1" CreationDate="2021-06-13T19:34:31.053" Score="1" ViewCount="27" Body="&lt;p&gt;I am using Terminal to try to run Chimera MD simulations on a remote server/computer. I have been able to successfully connect using the command ssh username@detroit05 (and then entering the password). However, I want to be able to send a command to a remote computer to improve the computational time especially when running multiple MD simulations; right now, I am running MD simulations on the same computer. I was first thinking of a sequence of commands below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Ctrl+Z&#xA;bg&#xA;ssh -y username@detroit05&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I know that the Ctrl+Z command temporarily pauses a command, but I believe that the jobs command is first needed to list the current running jobs (that Ctrl+Z can be performed on). However, the running jobs are not listed when I type “jobs”. After using the “top” command to list all of the processes, I have verified that the Kill command works by typing these in succession:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Kill -STOP PID#&#xA;Kill -CONT PID#&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can you walk me through how to first have a listing of jobs running in Chimera and then sending it to a remote computer?&lt;/p&gt;&#xA;" OwnerUserId="12537" LastEditorUserId="73" LastEditDate="2021-06-14T09:54:25.713" LastActivityDate="2021-06-14T09:54:25.713" Title="Sending chimera MD to remote computer" Tags="|molecular-dynamics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16136" PostTypeId="1" CreationDate="2021-06-14T08:24:21.593" Score="1" ViewCount="1277" Body="&lt;p&gt;Originally- I was looking if in at least one of these genes were expressed:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;test1 &amp;lt;- subset(myData,&#xA;                  (KLRC1 &amp;gt; 0) | (KIR2DL1 &amp;gt; 0) | &#xA;                (KIR2DL3 &amp;gt; 0) | (KIR2DL4 &amp;gt; 0) |&#xA;                (KIR3DL1 &amp;gt; 0) | (KIR3DL2 &amp;gt; 0) | (KIR3DL3 &amp;gt; 0), slot = &amp;quot;data&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now- I would like to subset if expressed in at least two of these genes- is this possible with the subset function?&lt;/p&gt;&#xA;" OwnerUserId="12540" LastEditorUserId="73" LastEditDate="2021-06-14T09:52:58.063" LastActivityDate="2021-06-14T13:44:06.607" Title="Seurat - subsetting by genes expressed" Tags="|seurat|single-cell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16137" PostTypeId="1" CreationDate="2021-06-14T08:37:24.873" Score="1" ViewCount="770" Body="&lt;p&gt;Can you please help me in a technical issue? We sent some samples to single cell sequencing The company says&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Many of the reads were not assigned to cell-associated barcodes. This&#xA;could be caused by high levels of ambient RNA or by a significant&#xA;population of cells with a low RNA content, which the algorithm did&#xA;not call as cells. The latter case can be addressed by inspecting the&#xA;data to determine the appropriate cell count and using --force-cells.&lt;/p&gt;&#xA;&lt;p&gt;This would happen if the cells aren’t transcriptionally active – but&#xA;as these are PBMCs this is unusual .&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Have you ever heard about such a problem? Any solution in this case? sending more sample of each patient for sequencing or comparing single cell with matched bulk RNA-seq which one help?&lt;/p&gt;&#xA;&lt;p&gt;10X say they would do deep sequencing (covering the cost) to get something for us but I don't know why my boss says we should compare single cell with matched bulk RNA-seq.&lt;/p&gt;&#xA;" OwnerUserId="9449" LastEditorUserId="73" LastEditDate="2021-06-15T10:18:17.127" LastActivityDate="2022-03-19T14:06:30.570" Title="10X single cell sequencing issue" Tags="|10x-genomics|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16138" PostTypeId="2" ParentId="16136" CreationDate="2021-06-14T13:44:06.607" Score="1" Body="&lt;p&gt;With a little bit of workaround:&lt;/p&gt;&#xA;&lt;p&gt;i) Add a new column to the data slot (only because your original subset() call  does so but it can be raw counts or any other data matrix in your Seurat object). &lt;code&gt;rowSums()&lt;/code&gt; determines how many non-zero counts you have.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df&lt;span class=&quot;math-container&quot;&gt;$non_zero &amp;lt;- rowSums(cbind(df$&lt;/span&gt;gene1 &amp;gt; 0, df&lt;span class=&quot;math-container&quot;&gt;$gene2 &amp;gt; 0, df$&lt;/span&gt;gene3 &amp;gt; 0, ...)) &amp;gt; 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Change df with the your data slot, should be sth like &lt;code&gt;YourSeuratObject@...$data&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;ii) Subset the newly modified data slot, basically treating the new column as another gene. Be careful not to use this column in preprocessing steps like normalization.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;YourSeuratObject &amp;lt;- subset(YourSeuratObject, non_zero &amp;gt; 1), slot = &amp;quot;data&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2021-06-14T13:44:06.607" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16140" PostTypeId="1" AcceptedAnswerId="16141" CreationDate="2021-06-14T18:35:02.747" Score="1" ViewCount="125" Body="&lt;p&gt;I have been trying to access the GDC metadata with python through the filter below. However, this code I have been using always yields a blank tsv file. Any help on fixing the code is appreciated!&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;import json&#xA;&#xA;cases_endpt = 'https://api.gdc.cancer.gov/cases'&#xA;&#xA;# The 'fields' parameter is passed as a comma-separated string of single names.&#xA;fields = [&#xA;    &amp;quot;sample_id&amp;quot;,&#xA;    &amp;quot;sample_uuid&amp;quot;,&#xA;    &amp;quot;sample_type&amp;quot;,&#xA;    &amp;quot;sample_type_id&amp;quot;,&#xA;    &amp;quot;tissue_type&amp;quot;,&#xA;    &amp;quot;tumor_code&amp;quot;,&#xA;    &amp;quot;tumor_code_id&amp;quot;,&#xA;    &amp;quot;oct_embedded&amp;quot;,&#xA;    &amp;quot;shortest_dimension&amp;quot;,&#xA;    &amp;quot;intermediate dimension&amp;quot;,&#xA;    &amp;quot;longest dimension&amp;quot;,&#xA;    &amp;quot;is_ffpe&amp;quot;,&#xA;    &amp;quot;pathology_report_uuid&amp;quot;,&#xA;    &amp;quot;tumor_descriptor&amp;quot;,&#xA;    &amp;quot;current weight&amp;quot;,&#xA;    &amp;quot;initial weight&amp;quot;,&#xA;    &amp;quot;composition&amp;quot;,&#xA;    &amp;quot;time_between_clamping_and_freezing&amp;quot;,&#xA;    &amp;quot;time_between_excision_and_freezing&amp;quot;,&#xA;    &amp;quot;days_to_sample_procurement&amp;quot;,&#xA;    &amp;quot;freezing_method&amp;quot;,&#xA;    &amp;quot;preservation_method&amp;quot;,&#xA;    &amp;quot;days_to_collection&amp;quot;,&#xA;    &amp;quot;portions&amp;quot;&#xA;    ]&#xA;&#xA;    fields = ','.join(fields)&#xA;&#xA;filters = {&#xA;    &amp;quot;op&amp;quot;: &amp;quot;in&amp;quot;,&#xA;    &amp;quot;content&amp;quot;: {&#xA;        &amp;quot;field&amp;quot;: &amp;quot;project_id&amp;quot;,&#xA;        &amp;quot;value&amp;quot;: [&amp;quot;TCGA-BRCA&amp;quot;]&#xA;    }&#xA; }&#xA;&#xA;params = {&#xA;    &amp;quot;filters&amp;quot;: json.dumps(filters),&#xA;    &amp;quot;fields&amp;quot;: fields,&#xA;    &amp;quot;format&amp;quot;: &amp;quot;TSV&amp;quot;,&#xA;    &amp;quot;size&amp;quot;: &amp;quot;10000&amp;quot;&#xA;    }&#xA;&#xA;response = requests.get(cases_endpt, params=params)&#xA;&#xA;file = open(&amp;quot;query10.tsv&amp;quot;, &amp;quot;w&amp;quot;)&#xA;file.write(response.text)&#xA;file.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12503" LastEditorUserId="4282" LastEditDate="2021-06-14T23:08:51.810" LastActivityDate="2021-06-14T23:08:51.810" Title="Querying metadata (GDC) using a filter" Tags="|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16141" PostTypeId="2" ParentId="16140" CreationDate="2021-06-14T20:33:51.117" Score="1" Body="&lt;p&gt;I think you are using an outdated version of the API, you need to change the &lt;code&gt;&amp;quot;field&amp;quot;&lt;/code&gt; with &lt;code&gt;cases.project.project_id&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;import json&#xA;&#xA;cases_endpt = &amp;quot;https://api.gdc.cancer.gov/cases&amp;quot;&#xA;&#xA;fields = [&#xA;    &amp;quot;sample_id&amp;quot;,&#xA;    &amp;quot;sample_uuid&amp;quot;,&#xA;    &amp;quot;sample_type&amp;quot;,&#xA;    &amp;quot;sample_type_id&amp;quot;,&#xA;    &amp;quot;tissue_type&amp;quot;,&#xA;    &amp;quot;tumor_code&amp;quot;,&#xA;    &amp;quot;tumor_code_id&amp;quot;,&#xA;    &amp;quot;oct_embedded&amp;quot;,&#xA;    &amp;quot;shortest_dimension&amp;quot;,&#xA;    &amp;quot;intermediate dimension&amp;quot;,&#xA;    &amp;quot;longest dimension&amp;quot;,&#xA;    &amp;quot;is_ffpe&amp;quot;,&#xA;    &amp;quot;pathology_report_uuid&amp;quot;,&#xA;    &amp;quot;tumor_descriptor&amp;quot;,&#xA;    &amp;quot;current weight&amp;quot;,&#xA;    &amp;quot;initial weight&amp;quot;,&#xA;    &amp;quot;composition&amp;quot;,&#xA;    &amp;quot;time_between_clamping_and_freezing&amp;quot;,&#xA;    &amp;quot;time_between_excision_and_freezing&amp;quot;,&#xA;    &amp;quot;days_to_sample_procurement&amp;quot;,&#xA;    &amp;quot;freezing_method&amp;quot;,&#xA;    &amp;quot;preservation_method&amp;quot;,&#xA;    &amp;quot;days_to_collection&amp;quot;,&#xA;    &amp;quot;portions&amp;quot;,&#xA;]&#xA;&#xA;fields = &amp;quot;,&amp;quot;.join(fields)&#xA;&#xA;filters = {&#xA;    &amp;quot;op&amp;quot;: &amp;quot;in&amp;quot;,&#xA;    &amp;quot;content&amp;quot;: {&amp;quot;field&amp;quot;: &amp;quot;cases.project.project_id&amp;quot;,&#xA;                &amp;quot;value&amp;quot;: [&amp;quot;TCGA-BRCA&amp;quot;]},&#xA;}&#xA;&#xA;params = {&amp;quot;filters&amp;quot;: json.dumps(filters), &#xA;          &amp;quot;format&amp;quot;: &amp;quot;TSV&amp;quot;, &#xA;          &amp;quot;size&amp;quot;: &amp;quot;10000&amp;quot;}&#xA;&#xA;response = requests.get(cases_endpt, params=params)&#xA;content = str(response.content, &amp;quot;utf-8&amp;quot;)&#xA;&#xA;with open(&amp;quot;query10.tsv&amp;quot;, &amp;quot;w&amp;quot;) as file:&#xA;    file.write(content)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And if you use &lt;em&gt;with open&lt;/em&gt;, you don't need to use &lt;em&gt;close&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2021-06-14T20:33:51.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16143" PostTypeId="2" ParentId="16003" CreationDate="2021-06-15T00:56:05.093" Score="2" Body="&lt;p&gt;I've tried many ways to do this with bcftools but don't think it's possible to exclude a bed file. What you want is&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;bedtools intersect -v -a sample.vcf -b mask.bed -wa -header&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;-v&lt;/code&gt; option gives you only the parts of file a that are &lt;em&gt;not&lt;/em&gt; in file b. &lt;code&gt;-wa&lt;/code&gt; only keeps entries from file a, and &lt;code&gt;-header&lt;/code&gt; preserves the header from file a.&lt;/p&gt;&#xA;" OwnerUserId="1152" LastEditorUserId="1152" LastEditDate="2021-06-15T01:11:29.730" LastActivityDate="2021-06-15T01:11:29.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16147" PostTypeId="2" ParentId="16137" CreationDate="2021-06-15T10:21:46.657" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @devon-ryan, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Comparing with matched bulk RNA-seq isn't reasonably related to this, unless you want to ensure the pseudo-bulk is similarish to the actual bulk. Of course they'll be different, but I suppose if you had enough samples and enough background knowledge about what to expect maybe you could get an idea whether the pseudo-bulk way reasonable, which would hit at whether the scRNA-seq was OK. But realistically you'd need a lot of experience with these exact cells in bulk and scRNA-seq to do that. Otherwise, it's a waste of time.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-06-15T10:21:46.657" CommentCount="0" CommunityOwnedDate="2021-06-15T10:21:46.657" ContentLicense="CC BY-SA 4.0" />
  <row Id="16148" PostTypeId="2" ParentId="16126" CreationDate="2021-06-15T14:02:20.063" Score="3" Body="&lt;p&gt;Since the Position Frequency Matrix is an estimate for the probability to find a particular letter at each position, I think the best way to deal with sequencing errors that result in &lt;code&gt;N&lt;/code&gt; is to ignore that base for that position and normalize based on the number of non-&lt;code&gt;N&lt;/code&gt; bases.&lt;/p&gt;&#xA;&lt;p&gt;The Information Content at each position is just a function of the (estimated) probability distribution. So, once you have the 'best' estimate for the Position Frequency Matrix, I think you should use it directly with no penalty when calculating the IC.&lt;/p&gt;&#xA;&lt;p&gt;If there is a position with a lot of &lt;code&gt;N&lt;/code&gt;s, though, you might get a biased estimate of the IC for that position - since using observed frequencies to estimate entropy/information is strongly biased when the number of observations is small.&lt;/p&gt;&#xA;" OwnerUserId="11905" LastActivityDate="2021-06-15T14:02:20.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16150" PostTypeId="1" CreationDate="2021-06-16T08:39:23.130" Score="0" ViewCount="542" Body="&lt;p&gt;I'm trying to generate boxplot from Beta diversity boxplot between two conditions from a phyloseq object.&lt;/p&gt;&#xA;&lt;p&gt;I have a Morisita-horn distance generated by vegdist fonction :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;vegdist&amp;lt;-vegdist(t(otu_table(physeq_genus_3a)), method=&amp;quot;horn&amp;quot;)#matrice distance Morisita-horn&#xA;vegdist&amp;lt;-as.matrix(vegdist)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now I have a matrix and I would like to somewhat create a mean distance boxplot of the Morisita-horn index winthin two condition like the graph below.&lt;/p&gt;&#xA;&lt;p&gt;I wanted to do a simple boxplot with ggplot but beacause it is a matrix and not a dataframe, I can't do it.&lt;/p&gt;&#xA;&lt;p&gt;Any help or suggestions would be more than welcome.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8bUl6.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8bUl6.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12549" LastActivityDate="2021-06-17T10:38:50.197" Title="How to create Beta Diversity distance boxplot" Tags="|r|statistics|microbiology|graphs|phyloseq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16151" PostTypeId="1" CreationDate="2021-06-16T09:15:13.163" Score="1" ViewCount="12" Body="&lt;p&gt;&lt;a href=&quot;https://doi.org/10.2307/2412519&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; paper introduces an approach to estimate a convergence from discord between phylogenetic species trees and gene trees, using Gene Duplication and Expression events.&lt;/p&gt;&#xA;&lt;p&gt;A &lt;strong&gt;Gene Duplication (GD)&lt;/strong&gt; event refers to when a bifurcation of the phylogenetic gene tree occurs creating two duplicate gene loci. The first image marks them in solid white diamonds.&lt;/p&gt;&#xA;&lt;p&gt;A &lt;strong&gt;Gene Expression (GE)&lt;/strong&gt; event refers to the &lt;strong&gt;loss&lt;/strong&gt; or &lt;strong&gt;Regulatory Mutation (RM)&lt;/strong&gt; of genes. The types are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Gene Loss (marked by the cross in the first image) refers to when a locus is lost/deleted along the lineage of an extant species.&lt;/li&gt;&#xA;&lt;li&gt;Gene Inactivation (marked by the unfilled downward caret in the first image) refers to a &lt;em&gt;Regulatory Mutation&lt;/em&gt; where a gene is rendered inactive.&lt;/li&gt;&#xA;&lt;li&gt;Gene Reactivation (marked by the unfilled upward caret in the first image) refers to a &lt;em&gt;Regulatory Mutation&lt;/em&gt; where a previously inactivated gene is reactivated and sampled in extant species.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Now, in the event of a discord between a gene tree and a species tree, we can attempt to establish concordance through assumptions about hypothetical duplication and expression events. The gene tree with a minimum count of (GD + GE) is the most likely assumption for concordance.&lt;/p&gt;&#xA;&lt;p&gt;In the first image, the &lt;strong&gt;right locus&lt;/strong&gt; resulting from the &lt;strong&gt;second bifurcation&lt;/strong&gt; of the gene tree is sampled in extant species C and D and undergoes a &lt;strong&gt;loss&lt;/strong&gt; event for species E. I would assume that it would also undergo a &lt;strong&gt;loss&lt;/strong&gt; event for species A and B but the paper merges those two loss events into one.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, based on the second image from &lt;a href=&quot;https://doi.org/10.1093/sysbio/46.3.523&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper, I've simulated the Gene Duplication/Expression events on the gene tree and found that if such loss events are merged into one, the Duplication/Expression (D/E in the table) count I calculate is lower than what is mentioned in the table.&lt;/p&gt;&#xA;&lt;p&gt;I'm confused as to whether there is an appropriate approach to this. The author of the &lt;a href=&quot;https://doi.org/10.2307/2412519&quot; rel=&quot;nofollow noreferrer&quot;&gt;first&lt;/a&gt; paper mentions that such assumptions regarding Gene Duplication and Expression events are not at all representative of real Gene Duplication and Expression events (established through observed evidence) and serve as a mechanism to approach likely solutions to discordance between species trees and gene trees.&lt;/p&gt;&#xA;&lt;p&gt;My question is, how do I arrive at the minimum cost tree?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pu5ds.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pu5ds.png&quot; alt=&quot;First Image&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/P5dbh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/P5dbh.png&quot; alt=&quot;Second Image&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12555" LastEditorUserId="12555" LastEditDate="2021-06-16T09:22:00.787" LastActivityDate="2021-06-16T09:22:00.787" Title="How to decide whether to assume 2 Gene Losses or 1 to find minimum cost trees for Gene Duplication/Expression events?" Tags="|phylogenetics|phylogeny|gene-expression|" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16152" PostTypeId="1" CreationDate="2021-06-16T15:07:45.920" Score="-1" ViewCount="50" Body="&lt;p&gt;I have two .csv files&lt;/p&gt;&#xA;&lt;p&gt;One for raw read counts of each gene and the other 3114 single cells for four patients which should be the row names of the first file&lt;/p&gt;&#xA;&lt;p&gt;As single cells per patients seems duplicated I am getting error and don't know how to read them together&lt;/p&gt;&#xA;&lt;p&gt;Any way this is the files&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/slkhspz7lgdgph6/ascites.csv?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/slkhspz7lgdgph6/ascites.csv?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.dropbox.com/s/iq8lt7qrh7hdtq8/ascitesRownames.csv?dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/s/iq8lt7qrh7hdtq8/ascitesRownames.csv?dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I tried&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;a=read.csv(&amp;quot;ascites.csv&amp;quot;)&#xA;Warning message:&#xA;In doTryCatch(return(expr), name, parentenv, handler) :&#xA;  restarting interrupted promise evaluation&#xA;b=read.csv(&amp;quot;ascitesRownames.csv&amp;quot;)&#xA;View(b)&#xA;View(a)&#xA;rownames(a)=b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error in &lt;code&gt;.rowNamesDF&amp;lt;-&lt;/code&gt;(x, value = value) :    duplicate 'row.names'&#xA;are not allowed In addition: Warning message: non-unique value when&#xA;setting 'row.names': ‘logical(0)’&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Any help in having a matrix in row names are 3114 single cells and genes in columns?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="3694" LastEditDate="2021-06-16T16:11:19.130" LastActivityDate="2021-06-16T16:48:55.360" Title="Unifying these files together" Tags="|r|single-cell|" AnswerCount="2" CommentCount="1" ClosedDate="2021-06-17T19:54:40.287" ContentLicense="CC BY-SA 4.0" />
  <row Id="16153" PostTypeId="1" CreationDate="2021-06-16T15:51:38.087" Score="3" ViewCount="431" Body="&lt;p&gt;I am trying to create haplotype networks in &lt;a href=&quot;http://popart.otago.ac.nz/index.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;PopART&lt;/a&gt; (Population genetics with Reticulated Trees*) which I have successfully done many times in the past.&lt;/p&gt;&#xA;&lt;p&gt;Two weeks ago the program was working fine for me but recently I am unable to load files. What happens is that when I go to load a nexus file through the &amp;quot;nex&amp;quot; button, or through the &amp;quot;file/import/alignment&amp;quot; or &amp;quot;file/open&amp;quot; options nothing loads.&lt;/p&gt;&#xA;&lt;p&gt;There is no error message, instead nothing happens. This occurs with both files I have recently created and with files that I have successfully loaded in the past. I've sent files to other people and everyone has been able to load the files using PopART on their computers. I am running macOS 10.15.6 and have tried turning my computer off and on as well as uninstalling and reinstalling PopART. I also thought that maybe Java wasn't up to date on my computer but I don't think PopArt runs with Java. Any help would be very appreciated!&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;PopArt builds reticulated HAplotype networks such shown below&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zrUwM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zrUwM.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12558" OwnerDisplayName="Gabriella Berman" LastEditorUserId="3848" LastEditDate="2022-03-16T14:19:25.653" LastActivityDate="2024-04-04T19:04:55.557" Title="Can't load files into PopART" Tags="|phylogenetics|phylogeny|haplotypenetwork|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16154" PostTypeId="2" ParentId="16152" CreationDate="2021-06-16T16:12:41.473" Score="0" Body="&lt;p&gt;&lt;code&gt;read.csv&lt;/code&gt; produces a data.frame. You'll need &lt;code&gt;rownames(a) &amp;lt;- b$col_name&lt;/code&gt;, not &lt;code&gt;rownames(a) &amp;lt;- b&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2021-06-16T16:12:41.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16155" PostTypeId="2" ParentId="16137" CreationDate="2021-06-16T16:13:55.477" Score="1" Body="&lt;p&gt;From what was described, mostly likely they ran cellranger and usually you can get &lt;a href=&quot;https://cf.10xgenomics.com/samples/cell-exp/4.0.0/Parent_NGSC3_DI_PBMC/Parent_NGSC3_DI_PBMC_web_summary.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;a summary report like this&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;One of the metric would be &lt;code&gt;Fraction Reads in Cells&lt;/code&gt; and I think this is what they are referring to this.&lt;/p&gt;&#xA;&lt;p&gt;My suggestion would be to ask for the report from cellranger and check the metrics and the different plots. If what you have are indeed pbmcs, the reason is most likely not a lot of cells made it into droplets.&lt;/p&gt;&#xA;&lt;p&gt;Submitting another sample will not help unless you can ensure the cells are of good enough quality, otherwise you will not get much cells back&lt;/p&gt;&#xA;&lt;p&gt;Sequencing it deeper does not help as well because you are basically sequencing RNA of so called ambient DNA, and it does not give you useful information.&lt;/p&gt;&#xA;" OwnerUserId="5077" LastActivityDate="2021-06-16T16:13:55.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16156" PostTypeId="2" ParentId="16152" CreationDate="2021-06-16T16:48:55.360" Score="1" Body="&lt;p&gt;There are a couple of problems:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;As Ram RS said you get a data frame when you load that second CSV.  Since the information is stored in a row rather than a column you need a little more work to extract it too.&lt;/li&gt;&#xA;&lt;li&gt;R will silently mangle your column names from the first data frame and will also interpret the second file as having column names (and then mangle those too) unless you give extra arguments to &lt;code&gt;read.csv&lt;/code&gt; to prevent it.&lt;/li&gt;&#xA;&lt;li&gt;The fact that you only have four identifiers (not 3114) in the second file means you can't use those as row names.  Your rows in &lt;code&gt;a&lt;/code&gt; are cells, not patients, right?  You could make up something unique based on both patient and cell info but I'd suggest avoiding that.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;First you could use something like this (going straight from Dropbox here) to read the files without losing information:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;a &amp;lt;- read.csv(&amp;quot;http://dl.dropboxusercontent.com/s/slkhspz7lgdgph6/ascites.csv&amp;quot;, check.names = FALSE)&#xA;b &amp;lt;- read.csv(&amp;quot;http://dl.dropboxusercontent.com/s/iq8lt7qrh7hdtq8/ascitesRownames.csv&amp;quot;, header = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then, since you want to associate more information than can easily fit in a &amp;quot;wide-format&amp;quot; (matrix-like) data frame, and row and column names are so easily mangled by R, I'd suggest using a &amp;quot;long-format&amp;quot; data frame instead.  The package reshape2 is very helpful for this.  Here you can put the patient IDs as an additional column in your first data frame, explicitly label the cells, and then make a new data frame where each row is a cell and gene combination for a particular patient:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;a &amp;lt;- cbind(patient = as.character(b[1, ]), cell_id = 1:nrow(a), a)&#xA;gene_counts &amp;lt;- reshape2::melt(&#xA;  a, id.vars = 1:2, variable.name = &amp;quot;gene&amp;quot;, value.name = &amp;quot;count&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So you get something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(gene_counts)&#xA;  patient cell_id gene count&#xA;1   7873M       1 A1BG     1&#xA;2   7873M       2 A1BG     0&#xA;3   7873M       3 A1BG     2&#xA;4   7873M       4 A1BG     0&#xA;5   7873M       5 A1BG     1&#xA;6   7873M       6 A1BG     0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(Note the gene column is a factor there.)&lt;/p&gt;&#xA;&lt;p&gt;I'd be very careful with row and column names since R is so prone to altering them without warning, and in general with assuming things will match up in the right order like the rows of the one table and the columns of the other.  Hadley Wickham has a good summary of the problems with row names in particular:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://adv-r.hadley.nz/vectors-chap.html#rownames&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://adv-r.hadley.nz/vectors-chap.html#rownames&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2021-06-16T16:48:55.360" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16157" PostTypeId="2" ParentId="4849" CreationDate="2021-06-16T19:14:55.767" Score="0" Body="&lt;p&gt;They may be B cells, did you analysis if they have b cell marker?&lt;/p&gt;&#xA;" OwnerUserId="12560" LastActivityDate="2021-06-16T19:14:55.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16158" PostTypeId="2" ParentId="16150" CreationDate="2021-06-16T19:41:27.543" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You can use &lt;code&gt;as.data.frame()&lt;/code&gt; or &lt;code&gt;as_tibble()&lt;/code&gt; to convert/coerce your matrix to a data frame or a tibble respectively (Technically a tibble is also a data frame). From there you should be fine with &lt;code&gt;ggplot()&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I would not use a &amp;quot;dynamite plot&amp;quot; (this is a box plot with error bars like the one in your question), that type of plots conceal the data! &lt;a href=&quot;https://simplystatistics.org/2019/02/21/dynamite-plots-must-die/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This post&lt;/a&gt; explains why and remedies.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="73" LastEditDate="2021-06-17T10:38:50.197" LastActivityDate="2021-06-17T10:38:50.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16159" PostTypeId="1" CreationDate="2021-06-17T07:41:07.220" Score="0" ViewCount="652" Body="&lt;p&gt;I have a list of genes in a mouse model&lt;/p&gt;&#xA;&lt;p&gt;How I can find the equivalent gene symbol/name for human from them?&lt;/p&gt;&#xA;&lt;p&gt;This is my list&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CD45&#xA;MHC II&#xA;CD11b&#xA;Ly6C&#xA;Ly6G&#xA;F4/80&#xA;CD11c&#xA;CD38&#xA;Arg1&#xA;SiglecF&#xA;CD206&#xA;CD62L&#xA;CD103&#xA;iNOS&#xA;PD-L1&#xA;TNFa&#xA;CD64&#xA;TCRgd&#xA;Foxp3&#xA;RORgt&#xA;CD8α&#xA;Tbet&#xA;CD25&#xA;IFN-γ&#xA;CD44&#xA;CD86&#xA;CD80&#xA;PD-1&#xA;B220&#xA;NK1.1&#xA;CD19&#xA;CD4&#xA;TCR β&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From these I only find two homologous&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.ibb.co/vL36YnX/Screenshot-2021-06-17-at-13-20-16.png&quot; alt=&quot;enter link description here&quot; /&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="650" LastEditDate="2021-06-17T13:19:19.517" LastActivityDate="2021-06-17T13:19:19.517" Title="Converting gene symbols between species" Tags="|gene|sequence-homology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16160" PostTypeId="2" ParentId="16159" CreationDate="2021-06-17T07:45:20.337" Score="3" Body="&lt;p&gt;Easiest way is &lt;a href=&quot;http://www.ensembl.org/biomart/martview&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioMart&lt;/a&gt;. Help video to get you started &lt;a href=&quot;https://www.youtube.com/watch?v=QvGT2G0-hYA&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Use the mouse genes dataset and filter by the list of gene names, get the mouse gene name and the human gene name (listed under homologues) as attributes.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2021-06-17T07:45:20.337" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16161" PostTypeId="1" CreationDate="2021-06-17T08:52:17.513" Score="1" ViewCount="415" Body="&lt;p&gt;I need to perform differential expression analysis using the &lt;a href=&quot;https://rdrr.io/bioc/scDD/f/inst/doc/scDD.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;scDD package&lt;/a&gt; from R, but I am not able to since I miss the &lt;strong&gt;normcounts&lt;/strong&gt; assay in my SCE object (of course in the example they show, the assay is already there).&lt;/p&gt;&#xA;&lt;p&gt;As I found out in the preprocessing steps, one can use &lt;a href=&quot;https://rdrr.io/bioc/scDD/man/preprocess.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this function&lt;/a&gt; to generate the missing assay, except that it does not work:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# this is the SCE object&#xA;&amp;gt; cTSubSet&#xA;class: SingleCellExperiment &#xA;dim: 19767 149 &#xA;metadata(2): Samples Samples&#xA;assays(2): counts logcounts&#xA;rownames(19767): Xkr4 Gm19938 ... CAAA01147332.1 AC149090.1&#xA;rowData names(4): ID Symbol Type chrLoc&#xA;colnames(149): AAAGGATGTTCACGAT-1 AACCATGTCTTTCGAT-1 ...&#xA;  TGTGGCGTCTCTTAAC-1 TTTGGTTAGACGCTCC-1&#xA;colData names(17): Sample Barcode ... slm_1 slm_1.5&#xA;reducedDimNames(0):&#xA;mainExpName: NULL&#xA;altExpNames(0):&#xA;&#xA;# setting parameters as suggested in the vignette linked above&#xA;&amp;gt; paR &amp;lt;- list(alpha=0.01, mu0=0, s0=0.01, a0=0.01, b0=0.01)&#xA;# setting zero.thresh to zero since I already removed unwanted genes&#xA;&amp;gt; cTSubSet &amp;lt;- preprocess(cTSubSet, zero.thresh=0, scran_norm=TRUE)&#xA;&#xA;# the not so useful error:&#xA;Performing scran Normalization&#xA;Error in t.default(Data) : argument is not a matrix&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I was also looking for other possible ways of obtaining such &lt;strong&gt;normcounts&lt;/strong&gt; assay and I found &lt;a href=&quot;https://bioconductor.org/packages/devel/bioc/vignettes/SingleCellExperiment/inst/doc/intro.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this description&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;normcounts: Normalized values on the same scale as the original&#xA;counts. For example, counts divided by cell-specific size factors that&#xA;are centred at unity.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Still, I can't figure out what this may mean. Is there a way of obtaining such &lt;strong&gt;normcounts&lt;/strong&gt;?&lt;/p&gt;&#xA;" OwnerUserId="3119" LastActivityDate="2021-06-18T12:41:34.557" Title="how to get normcounts for singlecellexperiment object?" Tags="|r|rna-seq|scrnaseq|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16162" PostTypeId="1" CreationDate="2021-06-17T09:12:52.407" Score="3" ViewCount="703" Body="&lt;p&gt;I'm trying to dock a ligand to several hundred PDB files (receptors).&#xA;I thus need to prepare the those PDB files like removing water and adding hydrogen.&lt;/p&gt;&#xA;&lt;p&gt;I can do that manually using Autodock or Pymol. But it's too troublesome.&#xA;Is there a way to do it programmatically with Python or R or any command line procedure?&lt;/p&gt;&#xA;" OwnerUserId="12433" LastEditorUserId="12433" LastEditDate="2021-06-17T10:29:51.557" LastActivityDate="2021-06-17T20:47:58.463" Title="Programmatically adding hydrogen and remove water to multiple PDB files" Tags="|python|protein-structure|docking|pymol|structural-biology|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="16163" PostTypeId="1" CreationDate="2021-06-17T13:08:38.487" Score="1" ViewCount="282" Body="&lt;p&gt;Can someone help me with comparing the cost per run of Illumina, PacBio and Nanopore sequencing technologies?&lt;/p&gt;&#xA;" OwnerUserId="12565" LastEditorUserId="12565" LastEditDate="2021-06-20T12:06:11.380" LastActivityDate="2021-06-21T12:36:03.513" Title="Illumina vs PacBio vs Nanopore price" Tags="|ngs|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16164" PostTypeId="1" AcceptedAnswerId="16165" CreationDate="2021-06-17T13:10:43.300" Score="-2" ViewCount="332" Body="&lt;p&gt;I have a Seurat object made by human single cells&lt;/p&gt;&#xA;&lt;p&gt;I am mapping some genes on that but no sign of expression&lt;/p&gt;&#xA;&lt;p&gt;When I GOOGLE for those genes I see the genes have different names&lt;/p&gt;&#xA;&lt;p&gt;How I know Seurat uses which gene name please&lt;/p&gt;&#xA;&lt;p&gt;This is my list&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CD45&#xA;MHC II&#xA;CD11b&#xA;Ly6C&#xA;Ly6G&#xA;F4/80&#xA;CD11c&#xA;CD38&#xA;Arg1&#xA;SiglecF&#xA;CD206&#xA;CD62L&#xA;CD103&#xA;iNOS&#xA;PD-L1&#xA;TNFa&#xA;CD64&#xA;TCRgd&#xA;Foxp3&#xA;RORgt&#xA;CD8α&#xA;Tbet&#xA;CD25&#xA;IFN-γ&#xA;CD44&#xA;CD86&#xA;CD80&#xA;PD-1&#xA;B220&#xA;NK1.1&#xA;CD19&#xA;CD4&#xA;TCR β&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From this list only 6 genes being mapped on the Seurat data&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2021-06-17T17:36:03.173" LastActivityDate="2021-06-17T17:36:03.173" Title="Getting conventional gene symbol for Seurat" Tags="|scrnaseq|gene|seurat|10x-genomics|" AnswerCount="1" CommentCount="1" ClosedDate="2021-06-17T21:28:15.263" ContentLicense="CC BY-SA 4.0" />
  <row Id="16165" PostTypeId="2" ParentId="16164" CreationDate="2021-06-17T16:18:42.923" Score="1" Body="&lt;p&gt;You might want to use Biomart to figure out the official names for those genes based on their aliases.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2021-06-17T16:18:42.923" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16166" PostTypeId="1" CreationDate="2021-06-17T19:26:35.590" Score="1" ViewCount="46" Body="&lt;p&gt;I have been trying to access metadata from the Genomic Data Commons; however, not all of the information I want is showing up in the resulting tsv file. How do I get sample type and analyte type to show up in the tsv file? I have tried altering what I pass as a field (towards the top of the code), or adding a different filter, but nothing seems to change what shows up in the tsv file. Any help is appreciated!&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;import json&#xA;&#xA;cases_endpt = &amp;quot;https://api.gdc.cancer.gov/cases&amp;quot;&#xA;&#xA;# The 'fields' parameter is passed as a comma-separated string of single names.&#xA;fields = [&#xA;    &amp;quot;sample_ids&amp;quot;,&#xA;    &amp;quot;submitter_sample_ids&amp;quot;,&#xA;    &amp;quot;files.cases.sample_ids&amp;quot;,&#xA;    &amp;quot;files.cases.submitter_sample_ids&amp;quot;,&#xA;    &amp;quot;files.cases.samples.sample_id&amp;quot;,&#xA;    &amp;quot;files.cases.samples.sample_type&amp;quot;,&#xA;    &amp;quot;files.cases.samples.sample_type_id&amp;quot;,&#xA;    &amp;quot;files.cases.samples.submitter_id&amp;quot;,&#xA;    &amp;quot;samples.sample_id&amp;quot;,&#xA;    &amp;quot;samples.sample_type&amp;quot;,&#xA;    &amp;quot;samples.sample_type_id&amp;quot;,&#xA;    &amp;quot;tissue_source_site.bcr_id&amp;quot;,&#xA;    &amp;quot;tissue_source_site.code&amp;quot;,&#xA;    &amp;quot;tissue_source_site.name&amp;quot;,&#xA;    &amp;quot;tissue_source_site.project&amp;quot;,&#xA;    &amp;quot;tissue_source_site.tissue_source_site_id&amp;quot;,&#xA;    &amp;quot;diagnoses.classification_of_tumor&amp;quot;,&#xA;    &amp;quot;diagnoses.tumor_stage&amp;quot;,&#xA;    &amp;quot;diagnoses.tumor_grade&amp;quot;,&#xA;    &amp;quot;files.cases.diagnoses.tumor_grade&amp;quot;,&#xA;    &amp;quot;files.cases.diagnoses.tumor_stage&amp;quot;,&#xA;    &amp;quot;files.cases.diagnoses.classification_of_tumor&amp;quot;,&#xA;    &amp;quot;files.cases.samples.tumor_code&amp;quot;,&#xA;    &amp;quot;files.cases.samples.tumor_code_id&amp;quot;,&#xA;    &amp;quot;files.cases.samples.tumor_descriptor&amp;quot;,&#xA;    &amp;quot;files.cases.samples.portions.slides.percent_tumor_cells&amp;quot;,&#xA;    &amp;quot;files.cases.samples.portions.slides.percent_tumor_nuclei&amp;quot;,&#xA;    &amp;quot;samples.tumor_code&amp;quot;,&#xA;    &amp;quot;samples.tumor_code_id&amp;quot;,&#xA;    &amp;quot;samples.tumor_descriptor&amp;quot;,&#xA;    &amp;quot;samples.portions.slides.percent_tumor_cells&amp;quot;,&#xA;    &amp;quot;samples.portions.slides.percent_tumor_nuclei&amp;quot;,&#xA;    &amp;quot;files.cases.samples.oct_embedded&amp;quot;,&#xA;    &amp;quot;samples.oct_embedded&amp;quot;,&#xA;    &amp;quot;files.cases.samples.intermediate_dimension&amp;quot;,&#xA;    &amp;quot;files.cases.samples.longest_dimension&amp;quot;,&#xA;    &amp;quot;files.cases.samples.shortest_dimension&amp;quot;,&#xA;    &amp;quot;samples.intermediate_dimension&amp;quot;,&#xA;    &amp;quot;samples.longest_dimension&amp;quot;,&#xA;    &amp;quot;samples.shortest_dimension&amp;quot;,&#xA;    &amp;quot;files.cases.samples.portions.is_ffpe&amp;quot;,&#xA;    &amp;quot;samples.is_ffpe&amp;quot;,&#xA;    &amp;quot;samples.portions.is_ffpe&amp;quot;,&#xA;    &amp;quot;files.cases.samples.pathology_report_uuid&amp;quot;,&#xA;    &amp;quot;samples.pathology_report_uuid&amp;quot;,&#xA;    &amp;quot;exposures.weight&amp;quot;,&#xA;    &amp;quot;files.cases.exposures.weight&amp;quot;,&#xA;    &amp;quot;files.cases.samples.current_weight&amp;quot;,&#xA;    &amp;quot;files.cases.samples.initial_weight&amp;quot;,&#xA;    &amp;quot;files.cases.samples.portions.weight&amp;quot;,&#xA;    &amp;quot;samples.current_weight&amp;quot;,&#xA;    &amp;quot;samples.initial_weight&amp;quot;,&#xA;    &amp;quot;samples.portions.weight&amp;quot;,&#xA;    &amp;quot;files.cases.samples.composition&amp;quot;,&#xA;    &amp;quot;samples.composition&amp;quot;,&#xA;    &amp;quot;files.cases.samples.freezing_method&amp;quot;,&#xA;    &amp;quot;files.cases.samples.time_between_clamping_and_freezing&amp;quot;,&#xA;    &amp;quot;files.cases.samples.time_between_excision_and_freezing&amp;quot;,&#xA;    &amp;quot;samples.freezing_method&amp;quot;,&#xA;    &amp;quot;samples.time_between_clamping_and_freezing&amp;quot;,&#xA;    &amp;quot;samples.time_between_excision_and_freezing&amp;quot;,&#xA;    &amp;quot;files.cases.samples.days_to_sample_procurement&amp;quot;,&#xA;    &amp;quot;samples.days_to_sample_procurement&amp;quot;,&#xA;    &amp;quot;files.cases.samples.preservation_method&amp;quot;,&#xA;    &amp;quot;samples.preservation_method&amp;quot;,&#xA;    &amp;quot;samples.days_to_collection&amp;quot;,&#xA;    &amp;quot;files.cases.samples.days_to_collection&amp;quot;,&#xA;    &amp;quot;files.cases.samples.portions.analytes.analyte_type&amp;quot;,&#xA;    &amp;quot;files.cases.samples.portions.analytes.analyte_type_id&amp;quot;,&#xA;    &amp;quot;files.cases.samples.portions.analytes.aliquots.analyte_type&amp;quot;,&#xA;    &amp;quot;files.cases.samples.portions.analytes.aliquots.analyte_type_id&amp;quot;,&#xA;    &amp;quot;samples.portions.analytes.analyte_type&amp;quot;,&#xA;    &amp;quot;samples.portions.analytes.analyte_type_id&amp;quot;,&#xA;    &amp;quot;samples.portions.analytes.aliquots.analyte_type&amp;quot;,&#xA;    &amp;quot;samples.portions.analytes.aliquots.analyte_type_id&amp;quot;,&#xA;    ]&#xA;&#xA;fields = ','.join(fields)&#xA;&#xA;filters = {&#xA;    &amp;quot;op&amp;quot;: &amp;quot;in&amp;quot;,&#xA;    &amp;quot;content&amp;quot;: {&amp;quot;field&amp;quot;: &amp;quot;cases.project.project_id&amp;quot;,&#xA;                &amp;quot;value&amp;quot;: [&amp;quot;TCGA-BRCA&amp;quot;]},&#xA;}&#xA;&#xA;params = {&amp;quot;filters&amp;quot;: json.dumps(filters),&#xA;          &amp;quot;format&amp;quot;: &amp;quot;TSV&amp;quot;,&#xA;          &amp;quot;size&amp;quot;: &amp;quot;12000&amp;quot;}&#xA;&#xA;response = requests.get(cases_endpt, params=params)&#xA;content = str(response.content, &amp;quot;utf-8&amp;quot;)&#xA;&#xA;with open(&amp;quot;addingsampletype5.tsv&amp;quot;, &amp;quot;w&amp;quot;) as file:&#xA;    file.write(content)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12503" LastActivityDate="2024-03-05T17:08:46.057" Title="querying gdc metadata issues with resulting fields" Tags="|data-retrieval|api|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16167" PostTypeId="2" ParentId="16162" CreationDate="2021-06-17T20:28:45.387" Score="2" Body="&lt;p&gt;I believe &lt;a href=&quot;https://pdb2pqr.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB2PQR&lt;/a&gt; CLI will do the work wonderfully. Don't let the name trick you: PQR files are organized like PDB ones. Under the hood it runs &lt;a href=&quot;https://propka.readthedocs.io/en/latest/quickstart.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;propka&lt;/a&gt;, which is state-of-the-art for predicting a protein residues protonation state. The best part is that you can use &lt;a href=&quot;https://server.poissonboltzmann.org/pdb2pqr&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB2PQR web server&lt;/a&gt; and they will give you the corresponding CLI arguments for each option you choose. It also automatically removes water (and other non-protein molecules).&lt;/p&gt;&#xA;&lt;p&gt;You can also try looking into tutorials on how to setup proteins for dynamics on GROMACS, AMBER or OpenMM and stumble on their own tools for protonating stuff (respectively &lt;code&gt;pdb2gmx&lt;/code&gt; (CLI), &lt;code&gt;protonate&lt;/code&gt; (CLI), and &lt;code&gt;modeller.addHydrogen&lt;/code&gt; (python API)), but as far as I know propka computes residues pKi in a knowledge-based manner and out-performs physics-based (if not just totally automatic, without guess) approaches like these.&lt;/p&gt;&#xA;&lt;p&gt;For some context: acidic or basic amino-acids in proteins may have different pKi depending on their local environment (i.e. if they are in contact with solvent or not, and residues surrounding them), which is why &amp;quot;adding hydrogen to proteins crystallographic structures at a constant pH&amp;quot; is not as easy as it sounds.&lt;/p&gt;&#xA;" OwnerUserId="12054" LastEditorUserId="12054" LastEditDate="2021-06-17T20:47:58.463" LastActivityDate="2021-06-17T20:47:58.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16168" PostTypeId="1" AcceptedAnswerId="16201" CreationDate="2021-06-17T20:55:36.007" Score="1" ViewCount="420" Body="&lt;p&gt;Recently, our lab ran a 192 sample experiment through our ATACseq pipeline. In doing so, HOMER's &lt;code&gt;mergePeaks&lt;/code&gt; told us that our 512 GB RAM server had too little memory to process all of the samples at once. So here we are trying to replace it with an alternative.&lt;/p&gt;&#xA;&lt;p&gt;It seems that &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/merge.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;bedTools merge&lt;/code&gt;&lt;/a&gt; would be the way to go to replace it. However, going into this step, each sample has its own bed file of peaks called by &lt;code&gt;MACS2&lt;/code&gt;. The &lt;code&gt;mergeBed&lt;/code&gt; only takes in 1 input file that must be sorted by chromosome and peak start position before we run it. I can run a &lt;code&gt;cat&lt;/code&gt; call on all of the samples followed by a &lt;code&gt;sort&lt;/code&gt;, but those individual sample bed files are already sorted. Is there a way to run &lt;code&gt;mergeBed&lt;/code&gt; with multiple input files at once? I tried &lt;code&gt;mergeBed -i 1.bed -i 2.bed &amp;gt; out.bed&lt;/code&gt;, but that is equivalent to &lt;code&gt;mergeBed -i 1.bed &amp;gt; out.bed&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In image format, I want the leftmost example, where the three peaks come from three separate files instead of all being described in the same file: &lt;a href=&quot;https://i.stack.imgur.com/HVrE5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HVrE5.png&quot; alt=&quot;mergeBed&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Additionally, bedtools does have a poorly documented function called &lt;code&gt;multiIntersectBed&lt;/code&gt;that appears to run the &lt;code&gt;cat&lt;/code&gt; and &lt;code&gt;sort&lt;/code&gt; together, but is both slower and doesn't merge bookended sections--eg files look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1 4658186 4658322&#xA;chr1 4658322 4658775&#xA;chr1 4658775 4658777&#xA;chr1 4671049 4671351&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When they should look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1 4658186 4658777 # This was 3 lines before&#xA;chr1 4671049 4671351&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In summary, I have sorted bed files &lt;code&gt;1.bed&lt;/code&gt; and &lt;code&gt;2.bed&lt;/code&gt; from the same experiment, and I want to merge all bookended and overlapping peak sections between them. How do I do so in 1 step that doesn't involve HOMER's &lt;code&gt;mergePeaks&lt;/code&gt;? Thank you!&lt;/p&gt;&#xA;&lt;p&gt;EDIT: Here's the speed of my current solutions at n=192:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;HOMER: Unstable&#xA;&#xA;`multiIntersectBed | mergeBed`&#xA;&#xA;real    3m53.230s&#xA;user    6m28.166s&#xA;sys     0m14.912s&#xA;&#xA;`cat | sort | mergeBed`&#xA;&#xA;real    0m24.860s&#xA;user    0m30.939s&#xA;sys     0m2.946s&#xA;&#xA;`cat &amp;gt; tmp1; sort &amp;gt; tmp2; mergeBed &amp;gt; out; rm tmp?`&#xA;&#xA;real    0m11.824s&#xA;user    0m20.710s&#xA;sys     0m2.021s&#xA;&#xA;`sort -m | mergeBed`&#xA;&#xA;real    0m17.905s&#xA;user    0m25.112s&#xA;sys     0m1.816s&#xA;&#xA;`sort -m --batch-size=192 | mergeBed`&#xA;&#xA;real    0m14.687s&#xA;user    0m25.589s&#xA;sys     0m0.747s&#xA;&#xA;`sort -m &amp;gt; tmp ; mergeBed -i tmp ; rm tmp`&#xA;&#xA;real    0m17.472s&#xA;user    0m15.251s&#xA;sys     0m2.212s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So... Followup questions, why does &lt;code&gt;sort -m&lt;/code&gt; take longer than &lt;code&gt;sort | cat&lt;/code&gt; especially when the sys time is indeed smaller? And why does piping take more time than creating temporary files?&lt;/p&gt;&#xA;" OwnerUserId="9521" LastEditorUserId="9521" LastEditDate="2021-06-20T12:55:39.247" LastActivityDate="2021-06-22T01:13:10.837" Title="alternative to HOMER's mergePeaks" Tags="|bedtools|homer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16169" PostTypeId="1" CreationDate="2021-06-18T00:13:00.077" Score="3" ViewCount="411" Body="&lt;p&gt;I'm working on nsSNPs(Non-synonymous single nucleotide polymorphisms) of protein AKAP13 whose PDB not available.&#xA;I did one mutation in this protein and I want to show the effect by 3d modeling without using a PDB but don't know how to do.&#xA;Can anyone help Please!!&lt;/p&gt;&#xA;" OwnerUserId="12568" LastActivityDate="2021-06-22T10:45:52.110" Title="How to make 3d model of a protein that not exist in PDB?" Tags="|python|3d-structure|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16170" PostTypeId="2" ParentId="16169" CreationDate="2021-06-18T07:30:41.457" Score="6" Body="&lt;h2&gt;No PDB&lt;/h2&gt;&#xA;&lt;p&gt;Getting a crystal structure is hard work for crystallographers, even with a high throughput systems, so they make constructs with only known domains or regions of particular interest. What happened here is that only two parts have been cloned, expressed and crystallised. The rest may be crystallisable or solvable by cryoEM, but may be too disordered.&lt;/p&gt;&#xA;&lt;h2&gt;Homology modelling&lt;/h2&gt;&#xA;&lt;p&gt;If the protein has a close homologue it can be threaded, that is it's sequences is &amp;quot;threaded&amp;quot; over the structure of the homologue. Two site provide these ready-made.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://swissmodel.expasy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SwissModel&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://modbase.compbio.ucsf.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ModBase&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The former is very clear, has an NGL viewer, a handy circular map and the assessment is clear. For &lt;a href=&quot;https://swissmodel.expasy.org/repository/uniprot/Q12802&quot; rel=&quot;nofollow noreferrer&quot;&gt;your protein&lt;/a&gt; it is showing four existing crystal structures, but these cover only a small part of it. The homology modelled section is small and not good here, but handy as the model has two numbers Qmean of –4 and identity of 23%, two red flags. So the next place to check are the following servers. They thread against different structures and hybridised. Missing parts are predicted ab initio (from the scratch). The latter parts look like spaghetti and cannot be trusted.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.sbg.bio.ic.ac.uk/phyre2/html/page.cgi?id=index&quot; rel=&quot;nofollow noreferrer&quot;&gt;Phyre2&lt;/a&gt;. &amp;lt;1 day turnover. Do pay attention to the colour rails. If your region of interest in low confidence and is based on nothing, then do not trust it at all. Also there it is good to double check keep an eye on the sequence alignments of your position as misalignments can happen.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://zhanglab.dcmb.med.umich.edu/I-TASSER/&quot; rel=&quot;nofollow noreferrer&quot;&gt;I-Tasser&lt;/a&gt;. 3-5 day turnover. The CAPS server winner year after year. You 5 models representing each cluster. If they look very different and the metric is bad for the top, then you have very low confidence and must look at the all if even.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://evcouplings.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;EVFold&lt;/a&gt;. The amino acid co-ocurrence is rather interesting&lt;/li&gt;&#xA;&lt;li&gt;AlphaFold... Google keeps it secret. Nobody except them can use it.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Here is an example of a spaghetti model (not your protein):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cZ7np.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cZ7np.png&quot; alt=&quot;spaghetti&quot; /&gt;&lt;/a&gt;&#xA;The except for the red bar, which is modelled against homologues and forms a nice domain within the spaghetti ball at the bottom, all of the rest is &lt;em&gt;ab initio&lt;/em&gt; and should not be trusted as the warning in the &amp;quot;Multi-template and ab initio information&amp;quot; section says.&lt;/p&gt;&#xA;&lt;p&gt;A nice addendum to a model is &lt;a href=&quot;https://consurf.tau.ac.il/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Consurf&lt;/a&gt;, which maps conservation to the models as B-factors. In a crystal structure, the core is conserved and the surface is not. Metrics are nice, but human insight is better.&lt;/p&gt;&#xA;&lt;p&gt;A big problem are protein that are part of a complex, because alone they are not structured and their structure is dependant on a neighbour. Models are great if they are threaded against a model that forms the same complex, eg. mouse to human, but not if they are hybridised.&lt;/p&gt;&#xA;&lt;p&gt;In the case of multidomain protein that are large. It is best to split the protein up. Furtehrmore, I-Tasser and Phyre have length limits because the longer the protein to predict is, the higher the chance of error.&#xA;One can determine where the domain boundaries may be —a Blast or a few iterations of Psi-blast will show conservation boundaries nicely. PFam has a lot to say about the protein architecture: &lt;a href=&quot;http://pfam.xfam.org/protein/Q12802&quot; rel=&quot;nofollow noreferrer&quot;&gt;enter link description here&lt;/a&gt;&#xA;Regions of high disorder cannot be and should not be modelled as they do not form a structure free in solution —unless wrapped around another protein as mentioned.&lt;/p&gt;&#xA;&lt;h2&gt;Wisdom to step away&lt;/h2&gt;&#xA;&lt;p&gt;However, the real question is not &lt;code&gt;how to&lt;/code&gt; but &lt;code&gt;should one&lt;/code&gt;. If one only get &lt;em&gt;ab initio&lt;/em&gt; results, do not trust them at all —count your losses as opposed to embarking on a castle in the sky. For a SNV, check if it is close to a post-translational modification in PhosphoSitePlus or if there is a linear motif in ELM.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2021-06-21T09:52:43.853" LastActivityDate="2021-06-21T09:52:43.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16171" PostTypeId="2" ParentId="16161" CreationDate="2021-06-18T12:41:34.557" Score="2" Body="&lt;p&gt;I do not know this package but it seems to call &lt;code&gt;scran&lt;/code&gt; so why bothering and not call &lt;code&gt;scran&lt;/code&gt; directly, see for an extended discussion + code:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioconductor.org/books/release/OSCA/normalization.html#normalization-by-deconvolution&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/books/release/OSCA/normalization.html#normalization-by-deconvolution&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-06-18T12:41:34.557" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16172" PostTypeId="2" ParentId="16058" CreationDate="2021-06-18T15:47:09.740" Score="1" Body="&lt;p&gt;Assuming you are using latest version of Nextflow which support the DSL 2 language, and assuming your python process output the file blah.csv, then the easiest way to move forward will be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process process_a() {&#xA;    output:&#xA;    path(&amp;quot;blah.csv&amp;quot;)&#xA;&#xA;    '''&#xA;    python.exe ./make_the_csv_file.py --path=blah.csv&#xA;    '''&#xA;}&#xA;&#xA;process process_b() {&#xA;&#xA;    input:&#xA;    val(the_field)&#xA;&#xA;    '''&#xA;    echo ${the_field}&#xA;    '''&#xA;&#xA;}&#xA;&#xA;workflow do_a_then_b{&#xA;    process_a() \&#xA;        | splitCsv \&#xA;        | map{ it -&amp;gt; it.TheFieldIWant } \&#xA;        | process_b&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can also refer to the nextflow google group.&lt;/p&gt;&#xA;" OwnerUserId="12574" LastActivityDate="2021-06-18T15:47:09.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16173" PostTypeId="1" AcceptedAnswerId="16176" CreationDate="2021-06-18T17:08:54.840" Score="1" ViewCount="595" Body="&lt;p&gt;It used to be easy to download off a results page of a PSI-Blast from NCBI the PSSM (scorematrix). Now I cannot seem to find an &lt;code&gt;asn&lt;/code&gt; file that is a scorematrix —&lt;code&gt;Download&amp;gt;ASN.1&lt;/code&gt; is the output of the search. Whereas before it was in &amp;quot;format results&amp;quot; IIRC.&#xA;I was after this to check something in the PSSM viewer or do further cycles of PSI-Blast with different settings —the latter especially handy given that at present the exclude taxa checkbox is troublesome. Where is the option gone?&#xA;&lt;a href=&quot;https://i.stack.imgur.com/gn3Ph.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gn3Ph.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-06-19T09:04:50.910" Title="Downloading PSSM file from NCBI PSI-Blast results after the UI update" Tags="|phylogenetics|psi-blast|pssm|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16176" PostTypeId="2" ParentId="16173" CreationDate="2021-06-19T08:52:12.260" Score="1" Body="&lt;p&gt;Have you tried clicking on the &amp;quot;Back to Traditional Results Page&amp;quot; link in the top right corner? If you're expecting a different interface, that might help work out where to find the right information.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ubpHt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ubpHt.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="6322" LastEditDate="2021-06-19T09:04:50.910" LastActivityDate="2021-06-19T09:04:50.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16177" PostTypeId="1" CreationDate="2021-06-19T11:07:01.453" Score="1" ViewCount="52" Body="&lt;p&gt;I have extracted residue 45 to 88 from a PDB file (&lt;strong&gt;&lt;a href=&quot;https://www.rcsb.org/structure/2yrq&quot; rel=&quot;nofollow noreferrer&quot;&gt;2YRQ&lt;/a&gt;&lt;/strong&gt;).&#xA;With that I have a new smaller PDB file. Let's call it 45_88.pdb&#xA;And I want to dock it to some other full protein.&lt;/p&gt;&#xA;&lt;p&gt;My question is what's the best way to dock it. I'm considering &lt;strong&gt;&lt;a href=&quot;http://vina.scripps.edu&quot; rel=&quot;nofollow noreferrer&quot;&gt;Autodock Vina&lt;/a&gt;&lt;/strong&gt;.&#xA;But I'm not sure if it's appropriate since it is for ligand-protein docking.&lt;/p&gt;&#xA;&lt;p&gt;What's the definition of a ligand? Can a protein with only 42 residues be considered a ligand?&lt;/p&gt;&#xA;" OwnerUserId="6199" LastEditorUserId="12433" LastEditDate="2021-06-19T11:18:41.813" LastActivityDate="2021-12-02T00:36:36.967" Title="Best way to dock subset of protein PDB and definition of a ligand" Tags="|pdb|docking|3d-structure|structural-biology|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16178" PostTypeId="2" ParentId="16153" CreationDate="2021-06-19T11:49:12.953" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @user438383, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Hi, this seems like quite a niche problem that you are facing, perhaps a bug in the GUI of some kind. Although it's on-topic, I think you're unlikely to find somewhere here who can help (unless the popart devs lurk here). I would suggest emailing the authors, but they point towards the github which mostly seems to be ignored.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-06-19T11:49:12.953" CommentCount="1" CommunityOwnedDate="2021-06-19T11:49:12.953" ContentLicense="CC BY-SA 4.0" />
  <row Id="16179" PostTypeId="2" ParentId="16166" CreationDate="2021-06-19T12:02:40.067" Score="0" Body="&lt;p&gt;When I visit the &lt;code&gt;cases_endpt&lt;/code&gt; page you linked &lt;a href=&quot;https://api.gdc.cancer.gov/cases&quot; rel=&quot;nofollow noreferrer&quot;&gt;directly&lt;/a&gt; on Firefox v88, I get a filterable JSON file. Looking for &amp;quot;type&amp;quot; only comes up with &amp;quot;disease_type&amp;quot;, not sample or analyte:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data    &#xA; hits   &#xA;  0 &#xA;   disease_type &amp;quot;Adenomas and Adenocarcinomas&amp;quot;&#xA;  1 &#xA;   disease_type &amp;quot;Ductal and Lobular Neoplasms&amp;quot;&#xA;  2 &#xA;   disease_type &amp;quot;Ductal and Lobular Neoplasms&amp;quot;&#xA;  3 &#xA;   disease_type &amp;quot;Adenomas and Adenocarcinomas&amp;quot;&#xA;  4 &#xA;   disease_type &amp;quot;Squamous Cell Neoplasms&amp;quot;&#xA;  5 &#xA;   disease_type &amp;quot;Adenomas and Adenocarcinomas&amp;quot;&#xA;  6 &#xA;   disease_type &amp;quot;Squamous Cell Neoplasms&amp;quot;&#xA;  7 &#xA;   disease_type &amp;quot;Adenomas and Adenocarcinomas&amp;quot;&#xA;  8 &#xA;   disease_type &amp;quot;Squamous Cell Neoplasms&amp;quot;&#xA;  9 &#xA;   disease_type &amp;quot;Ductal and Lobular Neoplasms&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-06-19T12:02:40.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16180" PostTypeId="1" CreationDate="2021-06-19T12:05:01.253" Score="1" ViewCount="89" Body="&lt;p&gt;I really appreciate @Matteo Ferla's detailed answer to &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/16170/73&quot;&gt;my previous question&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I've already tested several tools to know if a mutation can cause a modification in the function of protein or not. There are some tools such as SIFT, PolyPhen 2, PROVEAN, SNPs&amp;amp;GO, I-Mutant, and so on. These tools give us the possibility to check if this mutation is deleterious or tolerant, So after using my own data on these tools I found that most of the tools give me ‘deleterious ‘ meaning that the structure of my mutated protein will be changed. So, my problem is about the mutated protein that does not have a PDB Code which I can use in Pymol, mcSm, SDM, DUET…. to visualize the 3d structure I tried to use BioPython but it requires a PDB code.&lt;/p&gt;&#xA;&lt;p&gt;You talked about the swissmodel, I had tried to use this server but it gives me the same PDB code for the original protein and the mutated protein with a similarity of 100% and QMEAN approximately the same, as shown in the following figures:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zXlZA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zXlZA.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;After that, I had used this PDB code in PyMol but I didn't find the position 2700 in which the mutation occurs.&lt;/p&gt;&#xA;&lt;p&gt;The ModeBase displays the same PDB code as Swiss-Model in which there is no 2700 position.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OckHY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OckHY.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/D5KkY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D5KkY.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The I-TASSER and EVFold require a small amount of data, since I am working on position 2700 these tools are not useful for me.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KqBwv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KqBwv.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9ZHoE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9ZHoE.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There is no method to visualize a 3d model with just a FASTA file or how to do my own PDB file based on a FASTA file using Python or R or any other language?&lt;/p&gt;&#xA;&lt;p&gt;I have Fasta format data (not PDB code) which I would like to visualize in 3d structure using R or Python or anything that can help me because I can't use tools of 3d structure(i have not PDB code).&lt;/p&gt;&#xA;&lt;p&gt;I've already tested several tools to know if a mutation can cause a modification in the function of protein or not. There are some tools such as SIFT, PolyPhen 2, PROVEAN, SNPs&amp;amp;GO, I-Mutant, and so on. These tools give us the possibility to check if this mutation is deleterious or tolerant, So after using my own data on these tools I found that most of the tools give me ‘deleterious ‘ meaning that the structure of my mutated protein will be changed. So, my problem is about the mutated protein that does not have a PDB Code which I can use in Pymol, mcSm, SDM, DUET…. to visualize the 3d structure I tried to use BioPython but it requires a PDB code.&lt;/p&gt;&#xA;&lt;p&gt;As long as the mutation is not known yet, I couldn't use these tools because they need the PDB code that I haven't.&lt;/p&gt;&#xA;&lt;p&gt;I would be grateful to those who will help me!!&lt;/p&gt;&#xA;" OwnerUserId="12568" LastEditorUserId="73" LastEditDate="2021-06-20T04:18:24.327" LastActivityDate="2021-06-20T04:20:27.353" Title="3d model of a protein via nucleotide sequence" Tags="|r|python|3d-structure|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16181" PostTypeId="2" ParentId="15936" CreationDate="2021-06-20T04:10:25.393" Score="2" Body="&lt;p&gt;The ggplot2 function &lt;code&gt;stat_summary&lt;/code&gt; or &lt;code&gt;stat_summary_bin&lt;/code&gt; should get you most of the way there, something like this [not tested]:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse)&#xA;library(ggplot2)&#xA;&#xA;df %&amp;gt;%&#xA;  ggplot() +&#xA;  aes(x=value, y=StudyID, colour=type) +&#xA;  stat_summary() +&#xA;  facet_wrap(~variable)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;See &lt;code&gt;?stat_summary&lt;/code&gt; for more information.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-06-20T04:10:25.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16183" PostTypeId="1" CreationDate="2021-06-20T11:23:26.280" Score="2" ViewCount="52" Body="&lt;p&gt;I have been trying to understand the Seurat for analysing scRNA-seq data. It comes to my mind that the main data is organised in the Seurat object with rows as genes and columns as the cells, and the entries are the gene read count.&lt;/p&gt;&#xA;&lt;p&gt;I am confused with the statement, &amp;quot;the connection between the detected genes in the cells and the read counts&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;What is exactly the meaning of the detected genes in the cells biologically and its connection to the read counts? Can we say if the read count of gene A is 10 it means this gene is detected 10 times in a cell? How to interpret the detected genes biologically?&lt;/p&gt;&#xA;" OwnerUserId="12477" LastEditorUserId="73" LastEditDate="2021-07-04T14:13:06.510" LastActivityDate="2021-07-04T14:13:07.313" Title="Connection between Detected Genes and The Read Counts" Tags="|scrnaseq|gene|seurat|reads|featurecounts|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16184" PostTypeId="1" CreationDate="2021-06-20T11:49:03.460" Score="1" ViewCount="2967" Body="&lt;p&gt;I have a fasta file with 18 sequences inside. I want to extract the length of each sequence with header using the len function. My file looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;DR-1&#xA;ATGTATGAGGGCAAGAAGACGAAGAACATGTTCCTGACCCGGGCCCTGGAGAAGATTCTG&#xA;GCCGACAAGGAAGTGAAGAAGGCGCATCACTCCCAGCTGCGCAAAGCGTGCGAGGTGGCC&#xA;CTAGAGGAAATAAAAGTTGAAACTGAGAAACAAAGCCCTCCTCATGGAGAAGCAAAAGCT&#xA;GGATCAGGCACCCTTCCACCAGTGAAATCAAAGACAAATTTTATTGAAGCAGACAAGTAC&#xA;TTTTTACCCTTTGAGTTGGCATGCCAGTCCAAATGTCCCCGCATAGTTAGTACATCTCTA&#xA;GATTGTTTGCAGAAACTTATTGCTTATGGGCACTTGACTGGCAGGGCTCCAGATAGTACG&#xA;ACACCAGGCAAAAAATTAATTGATAGAATTATTGAAACAATATGTGGCTGCTTCCAAGGT&#xA;CCTCAAACAGATGAAGGAGTTCAGTTGCAGATAATAAAGGCTTTACTTACCGCAGTGACA&#xA;TCACAACACATAGAAATCCATGAAGGAACTGTGCTGCAAGCTGTGAGGACATGCTACAAT&#xA;ATCTATTTAGCCAGCAAAAATCTCATCAATCAGACAACAGCCAAAGCTACTCTTACTCAA&#xA;ATGCTGAACGTTATCTTTGCACGCATGGAAAACCAAGCATTACAGGAGGCCAAACAAATG&#xA;GAAAGAGAAAGACATCGGCAGCAGCAGCATTTGTTACAGTCTCCAGTAAGCCATCATGAG&#xA;CCCGAGTCACCTCATCTTAGATATTTGCCACCTCAGACTGTTGATCATATAAACCAAGAG&#xA;CATGAAGGGGATCTTGAGCCCCAAACACATGATGTGGATAAAAGTCTCCAAGATGATACA&#xA;GAGCCTGAAAACGGATCTGATATTTCCAGTGCAGAAAATGAACAGACAGAAGCTGATCAG&#xA;GCAACTGCAGCTGAAACATTATCTAAAAATGATATCTTGTATGATGGAGACTATGAAGAA&#xA;AAGCCACTTGATATTGTCCAGAGCATTGTAGAAGAAATGGTGAACATCATTGTTGGAGAC&#xA;ATGGGAGAAGGGATGGCTATCAGTGCAAGTACAGAAGGCAACACTGGAACTGTAGAAGAT&#xA;GGTAGTGACAGTGAAAACATCCAAGCAAATGGAATCCCAGGAACACCCATTTCTGTTGCA&#xA;TATACACCATCGTTACCTGACGACAGGTTATCTGTCTCCTCCAATGATACTCAGGAATCT&#xA;GGAAATTCTTCAGGACCTTCACCTGGTGCTAAGTTTTCCCACATTTTACAAAAGGATGCC&#xA;&amp;gt;SK-1&#xA;TTTCTAGTGTTCAGGTCATTGTGTAAGCTGTCAATGAAACCACTGTCAGATGGACCACCA&#xA;GATCCAAAGTCTCATGAGTTACGGTCCAAGATTCTTTCATTGCAGTTGCTTCTGTCCATT&#xA;CTGCAGAATGCAGGACCTGTTTTCAGGACAAATGAGATGTTTATTAATGCTATCAAGCAG&#xA;TATCTATGTGTTGCACTCTCAAAAAACGGAGTCTCATCTGTTCCAGAGGTTTTTGAGCTT&#xA;TCTCTTTCCATATTTCTTACTTTGTTGTCAAACTTCAAGACACATCTGAAGATGCAAATT&#xA;GAGGTATTCTTTAAAGAAATCTTCCTGTACATTTTGGAAACGTCTACAAGCTCATTTGAC&#xA;CATAAGTGGATGGTTATTCAGACACTGACAAGGATCTGTGCAGATGCTCAGAGTGTGGTG&#xA;GACATTTATGTAAATTATGACTGTGACCTAAATGCAGCAAATATATTTGAAAGACTAGTA&#xA;AATGACCTGTCGAAAATTGCCCAAGGAAGGGGCAGCCAAGAACTTGGTATGAGTAATGTT&#xA;CAGGAATTGAGCCTGCGAAAAAAGGGGTTAGAATGTTTAGTTTCAATTTTGAAATGTATG&#xA;GTTGAATGGAGCAAGGATCAGTATGTGAACCCCAACTCTCAAACTACCCTTGGTCAGGAA&#xA;AAACCCTCAGAGCAAGAGATAAGTGAAGTTAAACACCCAGAGACTATAAACAGATATGGA&#xA;AGTTTAAACTCCTTAGAATCCACATCATCATCAGGAATTGGCAGCTACAGTACACAGATG&#xA;TCTGGCACTGATAATCCAGAACAATTTGAGGTCCTAAAGCAGCAAAAAGAAATAATAGAA&#xA;CAAGGGATTGATTTATTTAATAAGAAACCAAAGAGAGGAATACAATACCTCCAAGAACAA&#xA;GGGATGCTTGGCACCACACCCGAAGATATAGCTCAGTTCTTACATCAAGAAGAAAGATTA&#xA;GACTCTACTCAGGTGGGTGAGTTCCTGGGAGATAATGATAAATTCAATAAAGAAGTCATG&#xA;&amp;gt;SC-1&#xA;TATGCATATGTAGACCAACATGACTTTTCAGGAAAAGACTTTGTTTCAGCTCTTCGTCTA&#xA;TTTCTGGAAGGATTCCGGCTTCCAGGAGAAGCTCAGAAAATTGATCGATTAATGGAAAAA&#xA;TTTGCTGCAAGATACCTAGAGTGCAACCAAGGACAAACCCTTTTTGCTAGTGCAGATACT&#xA;GCTTATGTTTTGGCCTACTCAATTATCATGTTGACTACAGACCTTCACAGTCCACAGGTT&#xA;AAAAACAAAATGACAAAAGAACAGTACATTAAAATGAATCGTGGTATCAATGACAGTAAA&#xA;GATCTCCCTGAAGAGTATCTATCAGCCATTTATAACGAAATAGCTGGGAAAAAAATATCA&#xA;ATGAAAGAAACAAAAGAACTAACAATCCCTACAAAGTCAACTAAACAGAATGTAGCCAGT&#xA;GAAAAGCAAAGACGACTTCTGTATAACTTAGAAATGGAACAGATGGCCAAGACGGCTAAA&#xA;GCACTCATGGAAGCTGTGAGCCATGTTCAGGCACCTTTTACAAGTGCAACACACTTGGAG&#xA;CACGTGAGGCCGATGTTTAAGTTGGCTTGGACACCTTTTCTGGCTGCATTCAGTGTGGGT&#xA;CTCCAAGACTGTGATGATACTGAAGTAGCCTCTCTTTGCCTAGAAGGGATAAGATGTGCT&#xA;ATCAGGATTGCATGCATTTTCAGCATTCAGCTGGAAAGAGATGCATATGTCCAAGCACTA&#xA;GCCAGATTTACCTTACTTACAGTGAGTTCTGGTATTACTGAAATGAAACAGAAGAATATC&#xA;GACACAATAAAAACACTCATCACGGTGGCTCACACTGATGGAAATTATTTAGGAAATTCG&#xA;TGGCATGAGATCCTGAAGTGCATCAGTCAGTTAGAGCTGGCACAACTTATAGGAACTGGA&#xA;GTGAAGCCTCGGTACATTTCTGGGACTGTGAGAGGCAGAGAAGGGTCTCTCACTGGGACA&#xA;AAAGATCAGGCTCCAGATGAATTTGTTGGTCTAGGGCTAGTTGGAGGAAATGTGGACTGG&#xA;AAGCAGATAGCCAGTATTCAGGAGTCTATTGGAGAAACCAGCTCTCAAAGTGTGGTAGTT&#xA;GCTGTGGATAGAATATTCACCGGATCCACAAGGCTAGATGGAAATGCTATTGTTGATTTT&#xA;GTCCGTTGGCTATGTGCTGTGTCTATGGATGAATTGCTTTCTACTACACACCCAAGAATG&#xA;TTTAGTCTACAAAAAATAGTAGAAATATCATACTATAACATGGGAAGAATAAGACTGCAG&#xA;TGGTCTCGGATTTGGGAAGTTATTGGAGATCATTTTAATAAGGTTGGTTGCAATCCTAAT&#xA;GAAGATGTAGCCATATTTGCAGTAGACTCTCTGAGGCAGCTATCAATGAAGTTTTTAGAG&#xA;&amp;gt;SC-2&#xA;AAAGGAGAGCTTGCAAACTTCAGATTCCAAAAGGATTTCTTACGACCTTTTGAACATATA&#xA;ATGAAACGAAACAGGTCTCCAACAATTCGAGATATGGTTGTACGGTGTATAGCACAAATG&#xA;GTTAATTCTCAGGCTGCAAATATTCGTTCAGGATGGAAGAACATTTTCTCAGTATTCCAT&#xA;CTAGCTGCATCAGACCAAGATGAAAGCATAGTAGAACTTGCATTTCAGACAACAGGGCAC&#xA;ATTGTCACTCTTGTATTTGAAAAACACTTTCCGGCCACCATTGATTCTTTCCAGGATGCA&#xA;GTGAAGTGTTTGTCTGAATTCGCATGTAATGCAGCTTTCCCAGACACAAGTATGGAAGCA&#xA;ATCCGACTTATTCGCCATTGTGCAAAATATGTGTCTGATAGACCGCAGGCTTTCAAGGAG&#xA;TACACAAGTGATGACATGAGCGTGGCACCTGAAGACAGGGTGTGGGTGAGAGGGTGGTTC&#xA;CCGATCCTCTTTGAGTTGTCTTGTATCATCAATAGATGCAAATTAGATGTAAGAACCAGG&#xA;GGCTTAACAGTAATGTTTGAAATAATGAAAACATATGGCCACACTTATGAAAAACATTGG&#xA;TGGCAGGACTTATTCAGAATTGTTTTCAGAATCTTTGACAACATGAAATTGCCAGAACAG&#xA;CAGACAGAGAAAGCTGAATGGATGACAACAACTTGTAACCATGCCCTTTATGCAATATGT&#xA;GATGTATTCACCCAGTATTTAGAAGTTCTCAGCGATGTACTTTTGGATGATATTTTTGCC&#xA;CAGCTTTACTGGTGTGTACAACAAGACAATGAGCAGTTAGCACGATCTGGTACAAACTGC&#xA;TTAGAAAACGTGGTTATTCTGAATGGTGAAAAGTTTACTCTTGAAATCTGGGATAAAACG&#xA;TGTAACTGCACCCTGGATATCTTCAAAACCACAATTCCTCACGCGCTTCTGACTTGGCGT&#xA;&amp;gt;TA-1&#xA;CCCACCTCTGGCGAGGCCGAACCCCCATCTCCATCAGCTGTGAGTGAAAAGCCGTTGGAT&#xA;GCAATATCACAAAAATCTGTAGACATACATGATTCTATTCAACCAAGATCTTCAGATAAC&#xA;AGACAGCAAGCACCACTGGTTTCTGTTTCTACTGTGAGTGAAGAAGTTAGCAAAGTGAAA&#xA;TCCACAGCAAAATTCCCAGAACAAAAATTGTTTGCTGCCCTGTTGATTAAATGTGTTGTG&#xA;CAGCTGGAACTCATCCAGACCATCGACAACATTGTCTTCTTCCCAGCCACAAGCAAGAAG&#xA;GAGGATGCAGAGAACTTAGCTGCAGCCCAGAGAGATGCAGTAGACTTCGATGTTCGAGTT&#xA;GATACTCAAGACCAAGGAATGTACCGCTTTTTAACATCACAACAACTGTTCAAGCTACTG&#xA;GACTGCTTATTAGAGTCGCATAGATTTGCAAAAGCATTTAATTCCAACAATGAACAGAGG&#xA;ACTGCTCTGTGGAAAGCAGGCTTCAAAGGCAAGTCCAAGCCCAATCTTCTAAAACAGGAG&#xA;ACAAGCAGCCTGGCGTGTGGGCTGCGCATTCTCTTCCGGATGTACATGGACGAGAGCAGA&#xA;GTCAGTGCCTGGGAAGAAGTCCAGCAGAGGCTTTTAAACGTCTGCAGGGAAGCGCTGAGC&#xA;TACTTCCTCACTCTAACATCCGAAAGTCACCGGGAAGCCTGGACTAACTTACTGTTACTG&#xA;TTCCTCACTAAAGTTCTGAAGATCAGTGACAGCAGGTTTAAAGCTCACGCATCATTCTAC&#xA;TACCCTCTCTTATGTGAAATTATGCAATTTGACTTGATTCCGGAACTGCGTGCTGTTCTT&#xA;AGAAGATTTTTCCTGCGAATTGGAATAGTATTTCAAATATCACAACCACCTGAACAGGAA&#xA;CTTGGGATAAACAGGCAATGA&#xA;&amp;gt;DR-2&#xA;ATGCAGCCACCACCTCAGGCAGTCCCGTCTGGGGTGGCTGGACCACCCCCAGCTGGGAAT&#xA;CCTCGGAGCATGTTCTGGGCCAACAGCCCTTACAGGAAGCCAGCCAATAATGCACCAGTG&#xA;GCTCCCATAACACGCCCATTGCAGCCAGTAACGGATCCGTTTGCTTTTAATAGACAGACT&#xA;CTTCAAAACACACCAGTAGGAAGTTCATCCAAAAGCAGCCTGCCCAATTTGCCTGGCCCA&#xA;GCCCTCTCTGTTTTTTCTCAGTGGCCTGGTTTGCCTGTGACTCCCACAAATGCTGGGGAT&#xA;AGCTCCACAGGATTGCATGAGCCTCTGTCAGGAACTCTGTCACAGCCTAGAGCAGATGCC&#xA;AGTCTATTTCCTCCTGCATCTACCCCTTCTTCACTCCCTGGGCTGGAGGTGAGCAGGAAT&#xA;GCTGAGGCTGATCCCAGCTCAGGACATGAAGTCCAGATGCTGCCACATTCAGCTCACTAC&#xA;ATTCCAGGAGTGGGTCCTGAGCAGCCTCTCGGGGGCCAAATGAATGACAGTGGGTCCGGG&#xA;CCTGACCAACCCATGAATAGGCATGCCCCCCATGATGGTGCCGTGACCCACGCAGCATCT&#xA;CCTTTCCTCCCTCAGCCACAGATGCCAGGTCAGTGGGGGCCTGCACAAGGAGGCCCTCAG&#xA;CCCTCCTATCAGCATCACTCACCCTACCTGGAGGGACCTGTTCAGAACATGGGGCTCCAG&#xA;GCTGCCAGCTTGCCCCATTTTCCCCCTCCATCTAGTCTACACCAGGGACCTGGCCATGAG&#xA;TCTCATGCCCCACAAACATTTACTCCAGCATCCTTGGCCAGTGGTGAAGGGAATGAGATA&#xA;GTTCACCAGCAAAGTAAAAATCACCCATTGAGTAGCTTCCCCCCTAAACACACTTTTGAG&#xA;CAGAATTCCAGAATTGGGAATATGTGGGCAAGCCCAGAGTTGAAGCAGAATCCGGGAGTG&#xA;AATAAAGAGCATCTGCTAGACCCTGCCCATGTCAATCCCTTCACTCAGGGAAATAGTCCT&#xA;GAAAACCAGGCGCATCATCCCCCAGTGGCAGCAACTAACCATGCCCTGCAAGAAGCAGCC&#xA;TCTGGAGCTCTCTCAATGTTTTTTCAAGGGGAAGAGACAGAGAATGAGGAAAATCTCTCA&#xA;TCTGAAAAAGCAGGCCTCGATAAGAGATTGAACTTGGATAGCTTTTCTTCTACCTCCAGA&#xA;CTGGGCCATCCACCACCACCTGGAGCTAGTGGTGTCTACCAGGCCTTTCCTAGAGGCCCT&#xA;AGCAGTGAGGCTGCGCAGGAAGGAGATGCGCAACCTTATTTTTCGCAGTCTGTGGGCGTC&#xA;CGTCTTGACAAACAGAGCACTGTGCCCCCTGCTAATGATGCATGGGGTGATGTTCCGGGG&#xA;ACCGGGACCCGCTGTGCCAGTGGGCCACAGTGTGAGAATGTTGAGAACTTAGAATTTGTT&#xA;CAGAATCAAGAAGTTTTGCCCCGTGAGACCCTGAGTGTGGATCCCTTCCCCCTGAGTGAT&#xA;CAGATCCGATATGGGCCCCTTCCTGGGCCAGCTGCCTCCAGGCCTGCTACTGTGGGCCTC&#xA;ACCAGAGGTGGGGGGCTAAATCTTGAGGCCCCAGACACACCACTGCACCCTACACGACCT&#xA;GACAGTGTGTCATCCAGCTACAGCAGCCATAGCCACAGAAGTCCTCCTGGGTCAGCCAGA&#xA;CCCCAAGAATTGGTGGGCACATTCATTCAGCAAGAAGTTGGGAAACTTGAAGATGATACT&#xA;TCAGGGAGTTTTTTTAAGCAAATTGATTCTTCTCCTGTTGGAGGTGAAACAGATGAGGTG&#xA;ACTGGGAGCCAAAATTGCTGCAGCAGTCTGTCCCAGCCTTCAACCCCAAGCCCTCCAAAA&#xA;CCTACAGGAGTATTCCAAACAAGTGCAAACAGTTCTTTTGAGCCAGTAAAATCCCACTTG&#xA;GTTGGAGTGAAACCAGTTGAAGCAGATCGTGCCAACATGGTGGTTGAAGTGAGAGGGACC&#xA;CAATACTGTCCGAAGAAGCGCAGGGCAGCTGTTGCACCACCTGATGCTACCTCAGGAAAC&#xA;CTGGAACAGCCACCAGACAACATGGAGACCCCGTGTGCACCTCAGGCCTGCCCTTTGCCT&#xA;CTTAGCACCACTGGAGAGGCTGGGCAACTGGTTTCAAACACGGCAGGGACACCCCTGGAT&#xA;ACTGTGCGCCCAGTGCCTGATAAGAGGCCCTCAGCCAGGGCCCAGGGACCTGTGAAGTGT&#xA;GAGAGCCCAGCAACAACTCTGTGGGCACAAAATGAGCTGCCAGATTTTGGTGGCAATGTC&#xA;CTTCTAGCCCCTGCTGCCCCTGCACTTTATGTTCCTGTAAAACCAAAACCTTCAGAGGTT&#xA;GTTCATCATCCAGAAAAGGGAATGTCTGGACAGAAGGCCTGGAAACAAGGGTCTGTGCCA&#xA;&amp;gt;TA-2&#xA;CCTTTGCAGAACCAAGACCCTCCTGGTGCTTCTGAGAACCTTGAGAACCCTCCCAAGGTA&#xA;GGAGAAGAGGAGGCCCTTCCAGTGCAGGCAAGTTCTGGTTATGCGAGTTTGCTTTCTTCG&#xA;CCACCCACTGAATCTTTGCACAATCAACCTGTCCTGATTGCTCAGCCTGATCAAAGCTAT&#xA;AATTTGGCTCAGCCTATTAATTTTTCTGTGTCCTTACTGAATCCTAATGAGAAGAATCAG&#xA;TCCTGGGGAGATGCTGTGGTGGGGGAGAGATCCATAGTGAGCAACAACTGGGCTCTCGGT&#xA;GGTGATCCTGAAGAACGTGCTGCTCTGTCTGGAGTTCCAGCCAGTGCTGTCACTGGCGCG&#xA;TCTTTGCCTAGCAGTATCCCCCAGAATTGTGCCCCTCAAGGTTCGGGGTCTTCTGAAATG&#xA;ATTGCTAGCCAGTCTGCCAGTTGGTTGGTTCAACAGCTATCCCCTCAAACCCCTCAAAGT&#xA;CCGCATCCAAATGCAGAAAAGGGCCCTTCCGAGTTTGTTAGTAGCCCTGCTGGCAATACA&#xA;AGTGTGATGCTAGTTCCACCTGCAAGCAGTACTTTGGTTCCTAACAGTAATAAGGCAAAG&#xA;CACTCTAGTAACCAGGAAGAAGCTGTTGGCGCCCTAGACTTCACATTAAATAGAACTTTG&#xA;GAAAATCCTGTGAGAATGTACAGCCCATCTCCTTCAGATGGCCCAGCTTCTCAGCAGCCT&#xA;CTTCCCAATCATCCCAGACAATCTGGGCCTGGGCTGCATAACCAGGATCATTTCTACCAG&#xA;CAGGTAACAAAAGATGCTCAGGACCAGCATAGACTAGAAAGAGCCCAGCCAGAGCTGGTG&#xA;CCTCCTCGGCCACAGAATTCTCCCCAAGTGCCCCAAGCATCGTGTCCAGAGCCTTCAAAT&#xA;CCTGAAAGCCCACCCACACAAGGACAGTCTGAAAGCTTGGCCCAGCCACCAGCAAGTCCA&#xA;GCTTCAGTCAACACAGGCCAGCTGCTGCCTCAGCCACCACAGGCTTCCAGTGCATCCGTT&#xA;ACATCTACCAACTCCAGCCAGGCAGCTGTGCGGTCAGAGCAGCTGTGGCTACACCCACCA&#xA;CCTCCTAATACTTTTGGCCCAGCACCTCAAGACTTGGCATCCTACTACTATTACAGACCC&#xA;CTGTATGATGCCTACCAGTCTCAGTACCCTTCACCGTACCCGTCAGATCCTGGGACAGCC&#xA;TCGCTGTACTACCAGGATATGTATGGCCTGTATGAGCCTCGTTACAGACCCTATGACAGC&#xA;TCAGCATCTGCTTATGCCGAGAACCACCGCTATTCTGAGCCTGAGCGGCCCAGCTCCCGT&#xA;GCAAGTCACTATTCCGACCAGCTTGCTCCCAGGCAAGGATATCCTGAAGGATACTATAAT&#xA;TCCAAGAGTGGATGGAGCAGTCACAGTGATTACTATGCAAATTATTACTCTGGCCAGTAT&#xA;GATTATGGAGACCCCAGCCGCTGGGATCGTTACTATGGGTCTCGCCTTAGGGATCCTCGC&#xA;ACCTGGGACCGGAGGTACTGGTATGACTCTGAACATGACCCATACAGGAAGGACCACTAT&#xA;GCTTACAGTGACAGGCCTGAGAAATGTGATGATCACTGGAGGTATGACCCTCGCTTCACT&#xA;GGGAGCTTCGACGATGACGCTGAGATCCACAGGGACCCCTATGGAGAAGAAGCAGACAGA&#xA;CGCAGCATCCACAGTGAGCACTCGGCACGGAGCCTGCGCAGCACTCACAGTCTGCCCAGC&#xA;CGCCGCAGCAGCCTCAGCTCCCATTCACACCAGAGTCAGATTTACAGAAGCCACCATGTG&#xA;ACTGGCGGTTCCTTTGAGGCCCCTCATGCCCCAGGCTCATTTCATGGTGATTATGCCTAT&#xA;GGCACATATGCCAGCAATTTCAGCGGTGCCCATGGTTTTCCAGAGTACAGCTACCCTGCA&#xA;GACACCTCCTGGCCTGCTGTGGAGCAAGTTCCATCAAGACCGACCTCTCCTGAGAAGTTT&#xA;...&#xA;...&#xA;...&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want the output to look like :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;DR-1&#xA;1260 &#xA;&amp;gt;SK-1&#xA;1020&#xA;&amp;gt;SC-1&#xA;1380&#xA;&amp;gt;SC-2&#xA;960&#xA;&amp;gt;TA-1&#xA;921&#xA;&amp;gt;DR-2&#xA;2400&#xA;TA-2&#xA;&amp;gt;1980&#xA;...&#xA;...&#xA;...&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am new to learning python for molecular analysis, so am having trouble with the logic and syntax behind it. I'm looking forward to a warm response from you. Thanks again.&lt;/p&gt;&#xA;" OwnerUserId="12579" LastEditorUserId="73" LastEditDate="2021-06-20T20:48:16.510" LastActivityDate="2021-06-21T16:58:06.477" Title="Count of each sequence length from a fasta file with header using len function of python" Tags="|python|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16186" PostTypeId="2" ParentId="16184" CreationDate="2021-06-20T12:59:37.820" Score="3" Body="&lt;p&gt;Replace &amp;quot;string protein sequences.fa&amp;quot; with &amp;quot;your_fasta_file.fa&amp;quot; and your file should be at the same path that you created your code.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import Bio.SeqIO as IO&#xA;record_dict = IO.to_dict(IO.parse(&amp;quot;string_protein_sequences.fa&amp;quot;, &amp;quot;fasta&amp;quot;))&#xA;for key in record_dict.items():&#xA;    print(key[0],&amp;quot;\n &amp;quot;,len(key[1].seq))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;9606.ENSP00000354718 &#xA;  2817&#xA;9606.ENSP00000383118 &#xA;  191&#xA;9606.ENSP00000275364 &#xA;  381&#xA;9606.ENSP00000265563 &#xA;  404&#xA;9606.ENSP00000265717 &#xA;  418&#xA;9606.ENSP00000348461 &#xA;  211&#xA;9606.ENSP00000249071 &#xA;  192&#xA;9606.ENSP00000304283 &#xA;  192&#xA;9606.ENSP00000400175 &#xA;  193&#xA;9606.ENSP00000285735 &#xA;  193&#xA;9606.ENSP00000339467 &#xA;  191&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I hope that's what you need, good luck&lt;/p&gt;&#xA;" OwnerUserId="12568" LastEditorUserId="12568" LastEditDate="2021-06-21T16:09:12.327" LastActivityDate="2021-06-21T16:09:12.327" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16187" PostTypeId="1" CreationDate="2021-06-20T16:15:40.857" Score="5" ViewCount="171" Body="&lt;p&gt;Using the &lt;a href=&quot;https://doi.org/doi:10.18129/B9.bioc.flowCore&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioconductor flowCore package&lt;/a&gt;, I'm applying two parallel and non-overlapping gates to a gatingSet directly under &lt;code&gt;&amp;quot;root&amp;quot;&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(flowCore)&#xA;&#xA;# Import data&#xA;file.name &amp;lt;-system.file(&amp;quot;extdata&amp;quot;,&amp;quot;0877408774.B08&amp;quot;,package=&amp;quot;flowCore&amp;quot;)&#xA;x &amp;lt;-read.FCS(file.name, transformation=FALSE)&#xA;&#xA;# Create flowSet&#xA;fs &amp;lt;-as(list(x,x), &amp;quot;flowSet&amp;quot;)&#xA;&#xA;# Gate&#xA;library(flowWorkspace)&#xA;gs &amp;lt;- GatingSet(fs)&#xA;&#xA;gate1 &amp;lt;- rectangleGate(filterId = &amp;quot;Gate1&amp;quot;, list(&amp;quot;FSC-H&amp;quot; = c(300,600), &amp;quot;SSC-H&amp;quot; = c(50,300)))&#xA;gs_pop_add(gs, gate1, parent = &amp;quot;root&amp;quot;)&#xA;&#xA;gate2 &amp;lt;- rectangleGate(filterId = &amp;quot;Gate2&amp;quot;, list(&amp;quot;FSC-H&amp;quot; = c(700,800), &amp;quot;SSC-H&amp;quot; = c(500,600)))&#xA;gs_pop_add(gs, gate2, parent = &amp;quot;root&amp;quot;)&#xA;&#xA;recompute(gs)&#xA;&#xA;gs_get_pop_paths(gs)&#xA;# [1] &amp;quot;root&amp;quot; &amp;quot;/Gate1&amp;quot; &amp;quot;/Gate2&amp;quot; &#xA;&#xA;# Plot&#xA;library(ggcyto)&#xA;autoplot(gs, gs_get_leaf_nodes(gs))&#xA;&#xA;# Extract gated data&#xA;gs_pop_get_data(gs, &amp;quot;Gate1&amp;quot;) # Works&#xA;gs_pop_get_data(gs, &amp;quot;Gate2&amp;quot;) # Works&#xA;gs_pop_get_data(gs, c(&amp;quot;Gate1&amp;quot;, &amp;quot;Gate2&amp;quot;)) # Does not work&#xA;gs_pop_get_data(gs, &amp;quot;Gate1&amp;quot;, &amp;quot;Gate2&amp;quot;) # Does not work&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to get the data from the &lt;a href=&quot;https://en.wikipedia.org/wiki/Union_(set_theory)&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;union&lt;/strong&gt;&lt;/a&gt; of gates 1 and 2 but I can't figure out how. The flowCore documentation touches on this topic, but the examples shown are for filters, and I haven't figured out how to get it to work with gates.&lt;/p&gt;&#xA;&lt;p&gt;If there is a more straightforward way of doing this using another package that would also be of interest, as long as it is compatible with flowSet/flowCore objects.&lt;/p&gt;&#xA;" OwnerUserId="11919" LastEditorUserId="11919" LastEditDate="2022-04-03T01:51:10.983" LastActivityDate="2022-04-03T01:51:10.983" Title="Simultaneously get data from multiple applied gates in flowCore" Tags="|r|bioconductor|flow-cytometry|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16188" PostTypeId="1" AcceptedAnswerId="16265" CreationDate="2021-06-20T17:27:30.930" Score="1" ViewCount="78" Body="&lt;p&gt;In the &lt;a href=&quot;https://satijalab.org/seurat/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seurat&lt;/a&gt; analysis, if we suppose that Xg and Xr denote the random variables that associate to the expression level of the gene g and of the gene r, respectively. Let Y and Yv represent the responses for Xg and Xr, respectively.&lt;/p&gt;&#xA;&lt;p&gt;Is it true that biologically, the random variables Xg and Xr represent the distributions of the gene expression measure per cell and the gene expression measure per spot, respectively and the responses Y and Yv represent the range of the random variable Xg and Xr, respectively. The values for Y and Yv are the cell types?&lt;/p&gt;&#xA;&lt;p&gt;Reference:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://projecteuclid.org/journals/statistical-science/volume-18/issue-1/Multiple-Hypothesis-Testing-in-Microarray-Experiments/10.1214/ss/1056397487.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://projecteuclid.org/journals/statistical-science/volume-18/issue-1/Multiple-Hypothesis-Testing-in-Microarray-Experiments/10.1214/ss/1056397487.full&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;[Method section 2.1]&lt;/p&gt;&#xA;&lt;p&gt;I try to understand what is the Random variable and the response in this context biologically?&lt;/p&gt;&#xA;&lt;p&gt;I am confused to make a biological statements about these response. Please help. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="12477" LastEditorUserId="73" LastEditDate="2021-06-24T01:56:44.743" LastActivityDate="2021-06-29T23:05:48.833" Title="The biological meaning of the random variables and the responses in Seurat analysis" Tags="|gene|seurat|differential-expression|gene-expression|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16189" PostTypeId="2" ParentId="16183" CreationDate="2021-06-20T20:38:09.567" Score="2" Body="&lt;p&gt;&lt;em&gt;Answer from @ATpoint, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;A detected gene is one that has a count &amp;gt; 0, and generally the deeper you sequence the more genes you detect until you reach a saturation which is being influenced by the number of biologically-expressed genes and the single-cell technology you're using, e.g. droplet vs plate-based and the library setup, e.g. end-tagged vs full length.&lt;/p&gt;&#xA;&lt;p&gt;Counts cannot influence the nature of gene translation. A count is an observation; a translated gene sequence is a physical thing. Whether or not a gene can be translated into protein has no relationship to how often it is observed in existence.&lt;/p&gt;&#xA;&lt;p&gt;Even with count zero (i.e. no detection in scRNA-seq), it does not mean that the gene is not present in the cell. NGS technology is a relative quantification, I doubt it is meaningful to try to &lt;strong&gt;directly&lt;/strong&gt; make statements about the connection between read counts and biological meaning in the cell.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-06-24T01:54:02.517" LastActivityDate="2021-06-24T01:54:02.517" CommentCount="0" CommunityOwnedDate="2021-06-20T20:38:09.567" ContentLicense="CC BY-SA 4.0" />
  <row Id="16191" PostTypeId="1" CreationDate="2021-06-20T21:38:51.673" Score="2" ViewCount="79" Body="&lt;p&gt;I want to retrieve an annotation object (as GRanges) containing different UCSC annotation tracks such as CpG Islands, TSS, TFBS and promoters for hg38 Human reference genome. But &amp;quot;UCSC Genes&amp;quot; was phased out for hg38. What could be the best resource for this ?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&lt;p&gt;RT&lt;/p&gt;&#xA;" OwnerUserId="3656" LastActivityDate="2021-11-19T22:02:00.140" Title="Hg38 annotation tracks retrieval" Tags="|sequence-annotation|ucsc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16192" PostTypeId="1" CreationDate="2021-06-21T06:46:15.223" Score="1" ViewCount="507" Body="&lt;p&gt;I have the following PDB file that looks like this.&#xA;It is downloadable &lt;a href=&quot;https://bit.ly/3xAlF5X&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;here&lt;/strong&gt;&lt;/a&gt;. This PDB already contains ligand and receptor together.&lt;/p&gt;&#xA;&lt;p&gt;The red molecule is the ligand and the yellow is the receptor.&#xA;They're already docked with &lt;a href=&quot;https://cluspro.bu.edu&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;ClusPro&lt;/strong&gt;&lt;/a&gt;.&#xA;My question is how to calculate the RMSD and binding energy from it?&#xA;Any existing tools (with or without command line) that can help?&lt;/p&gt;&#xA;&lt;p&gt;I don't have any data to try it out at hand but have a look at the documentation I linked. There you can find ways of constructing an Ensemble which can then be superposed and RMSDs calculated, even pairwise.&lt;/p&gt;&#xA;&lt;p&gt;Reference is the red one (receptor). I have hundreds of other complexed ligands. I want to calculate how good are those complexes. What's the reasonable measure to do that?&lt;/p&gt;&#xA;&lt;p&gt;Example code using Prody? please :-)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YuHMK.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YuHMK.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12433" LastEditorUserId="12054" LastEditDate="2021-07-09T11:54:37.623" LastActivityDate="2023-04-25T20:59:08.053" Title="How to calculate RMSD and binding energy from the already docked ligand-receptor" Tags="|proteins|protein-structure|pdb|docking|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16195" PostTypeId="2" ParentId="16169" CreationDate="2021-06-21T10:13:02.983" Score="2" Body="&lt;p&gt;&lt;em&gt;Answer from @o-laprevote, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Your protein includes intrinsically disordered regions (&lt;a href=&quot;https://www.uniprot.org/uniprot/Q12802#family_and_domains&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uniprot/Q12802#family_and_domains&lt;/a&gt;). Showing a predicted structure may give a wrong idea of what it looks like: I'd consider simply showing the ordered part (of which a structure already exists) plus a &amp;quot;spaghetti&amp;quot; starting from its N-terminal residue.&lt;/p&gt;&#xA;&lt;p&gt;In this case the ordered regions would be in the &amp;quot;interaction with ESR1&amp;quot; domain (1968-2338), which as you can see has been solved experimentally (PDB 4D0N), and the sequence spanning from 965-1431 (part of it has been solved, PDB 2DRN). What I mean by &amp;quot;spaghetti&amp;quot; is basically: render these two regions then draw curvy arabesques/lines between, before and after them to show it is disordered (starting from their N-terminal or C-terminal).&lt;/p&gt;&#xA;&lt;p&gt;Such a cartoon is best done in a program that is shows it utterly differently —to not confuse the viewer as this is for display purposes only. Namely, render the parts in PyMOL, but then add the connecting lines in Photoshop or Powerpoint or MS-Paint/Preview.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-06-22T10:45:52.110" LastActivityDate="2021-06-22T10:45:52.110" CommentCount="1" CommunityOwnedDate="2021-06-21T10:13:02.983" ContentLicense="CC BY-SA 4.0" />
  <row Id="16196" PostTypeId="1" CreationDate="2021-06-21T11:14:25.730" Score="1" ViewCount="31" Body="&lt;p&gt;I am planning to perform multiple routine NGS analyses for a hospital lab on patients' samples on an everyday basis.&lt;/p&gt;&#xA;&lt;p&gt;Before I used a new tuxedo approach with &lt;code&gt;ballgown&lt;/code&gt; etc and was rather satisfied with it.&lt;/p&gt;&#xA;&lt;p&gt;I wonder if for the above-mentioned job it's ok to use a ballgown or, perhaps, that is not the best choice?&lt;/p&gt;&#xA;" OwnerUserId="6470" LastActivityDate="2021-06-21T11:14:25.730" Title="Is New Tuxedo approach really better than the earlier ones?" Tags="|bioconductor|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16197" PostTypeId="2" ParentId="16184" CreationDate="2021-06-21T14:30:37.697" Score="0" Body="&lt;p&gt;Don't forget to change the file name and write the valid path&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with open(&amp;quot;string_protein_sequences.fa&amp;quot;) as file_content:&#xA;    liste=[]&#xA;    j=-1&#xA;    for seqs in file_content:&#xA;        if seqs.startswith('&amp;gt;'):&#xA;            j+=1&#xA;            print(seqs,lis[j])&#xA;            some=0&#xA;            i=0&#xA;        else:&#xA;            i+=1&#xA;            some+=len(seqs) &#xA;        liste.append(some-i)&#xA;    lis=[]&#xA;    for j in range(1,len(liste)):&#xA;        if liste[j]==0:&#xA;            lis.append(liste[j-1])&#xA;    lis.append(liste[-1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; &amp;gt;9606.ENSP00000354718 (AKAP13)&#xA;  2817&#xA; &amp;gt;9606.ENSP00000383118 (CDC42)&#xA;  191&#xA; &amp;gt;9606.ENSP00000275364 (GNA12)&#xA;  381&#xA; &amp;gt;9606.ENSP00000265563 (PRKAR2A)&#xA;  404&#xA; &amp;gt;9606.ENSP00000265717 (PRKAR2B)&#xA;  418&#xA; &amp;gt;9606.ENSP00000348461 (RAC1)&#xA;  211&#xA; &amp;gt;9606.ENSP00000249071 (RAC2)&#xA;  192&#xA; &amp;gt;9606.ENSP00000304283 (RAC3)&#xA;  192&#xA; &amp;gt;9606.ENSP00000400175 (RHOA)&#xA;  193&#xA; &amp;gt;9606.ENSP00000285735 (RHOC)&#xA;  193&#xA; &amp;gt;9606.ENSP00000339467 (RHOG)&#xA;  191&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12568" LastEditorUserId="12568" LastEditDate="2021-06-21T16:11:32.770" LastActivityDate="2021-06-21T16:11:32.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16198" PostTypeId="2" ParentId="16184" CreationDate="2021-06-21T16:58:06.477" Score="0" Body="&lt;p&gt;you can use a dictionary as well:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fasta = {}&#xA;with open(&amp;quot;test.fasta&amp;quot;, &amp;quot;r&amp;quot;) as file:&#xA;    for line in file:&#xA;        line = line.strip()&#xA;        if line[0] == &amp;quot;&amp;gt;&amp;quot;:&#xA;            header = line&#xA;            fasta[header] = &amp;quot;&amp;quot;&#xA;        else:&#xA;            data = line&#xA;            fasta[header] += data&#xA;&#xA;for k in fasta.keys():&#xA;    print(f&amp;quot;{k}\n{len(fasta[k])}&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2021-06-21T16:58:06.477" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16199" PostTypeId="1" AcceptedAnswerId="16202" CreationDate="2021-06-21T20:52:19.693" Score="2" ViewCount="298" Body="&lt;p&gt;I am using &lt;code&gt;Monocle3&lt;/code&gt; to run pseudo time analysis on samples at two different conditions (samples are paired). My question is, is it valid to combine all the cells from samples at specific condition when clustering, learning graph and ordering cells? or should I pick up only one sample per condition and run the analysis on?&lt;/p&gt;&#xA;&lt;p&gt;My understanding is that pseudotime analysis uses the expression data of cells to try to infer the trajectory of cells, so I believe it might not be correct when I pool all the cells that might not have a true trajectory?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="12592" LastActivityDate="2021-06-22T10:05:19.590" Title="Pseudotime analysis on multiple samples" Tags="|scrnaseq|monocle|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16200" PostTypeId="1" CreationDate="2021-06-21T22:41:45.170" Score="1" ViewCount="19" Body="&lt;p&gt;Sorry if this question has been asked. Post batch correction and integration of two datasets, where cells of multiple different tissue types are present, I want to run clustering on cells of corresponding tissue types between the two datasets. Should I simply subset the integrated object to get each tissue type, scale the data and run PCA and then run clustering on it?&lt;/p&gt;&#xA;&lt;p&gt;Thanks so much for the clarifications.&lt;/p&gt;&#xA;" OwnerUserId="4307" LastActivityDate="2021-06-21T22:41:45.170" Title="Clustering by individual tissue population post integration of two datasets" Tags="|scrnaseq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16201" PostTypeId="2" ParentId="16168" CreationDate="2021-06-21T23:27:20.747" Score="1" Body="&lt;p&gt;Via BEDOPS, you could combine &lt;code&gt;bedops --merge&lt;/code&gt; with process substitutions sorting &lt;code&gt;N&lt;/code&gt; files:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops --merge &amp;lt;(sort-bed 1.bed) &amp;lt;(sort-bed 2.bed) ... &amp;lt;(sort-bed N.bed) &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The peak memory usage of a typical &lt;code&gt;bedops --merge&lt;/code&gt; run should be around 10MB, if I remember correctly, as a result of using sorted inputs.&lt;/p&gt;&#xA;&lt;p&gt;The memory usage of each &lt;code&gt;sort-bed&lt;/code&gt; process can take up the size of each file. You can specify &lt;code&gt;--max-mem&lt;/code&gt; and &lt;code&gt;--tmpdir&lt;/code&gt; options to instead use disk space to sort files. You may want to do this if the sum of file sizes is equal to or greater than ~512GB, or however much system memory is available (typically less).&lt;/p&gt;&#xA;&lt;p&gt;If your inputs are already sorted, or if your ATACseq pipeline is set up to create sorted files, then &lt;code&gt;sort-bed&lt;/code&gt; does not need to be run again, and you can just use the merge operation on its own:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops --merge 1.bed 2.bed ... N.bed &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The memory usage is then only what is needed for merging, as discussed above.&lt;/p&gt;&#xA;&lt;p&gt;Using &lt;code&gt;bedops --merge&lt;/code&gt; will merge adjacent (&amp;quot;bookended&amp;quot;) intervals, so long as they are in sort-order, &lt;em&gt;e.g.&lt;/em&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ echo -e &amp;quot;chr1\t4658186\t4658322\nchr1\t4658322\t4658775&amp;quot; | bedops --merge -&#xA;chr1    4658186 4658775&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Notes:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The file &lt;code&gt;answer.bed&lt;/code&gt; will be sorted per &lt;code&gt;sort-bed&lt;/code&gt; for downstream operations, such as &lt;code&gt;bedmap&lt;/code&gt; association of merged regions to peaks. Additional sorting is not needed.&lt;/li&gt;&#xA;&lt;li&gt;Using &lt;code&gt;sort-bed&lt;/code&gt; will often be faster than Unix &lt;code&gt;sort&lt;/code&gt; on BED files.&lt;/li&gt;&#xA;&lt;li&gt;The limit of &lt;code&gt;N&lt;/code&gt; on most platforms will be 1021 files. Most OSes have a limit of 1024 file descriptors, and three are reserved for standard IO. This soft limit can be &lt;a href=&quot;https://serverfault.com/questions/48717/practical-maximum-open-file-descriptors-ulimit-n-for-a-high-volume-system&quot;&gt;dialed upwards&lt;/a&gt; on some hosts where the user has admin access.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;References:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/summary.html#bedops&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedops.readthedocs.io/en/latest/content/summary.html#bedops&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/set-operations/bedops.html#merge-m-merge&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedops.readthedocs.io/en/latest/content/reference/set-operations/bedops.html#merge-m-merge&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/file-management/sorting/sort-bed.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedops.readthedocs.io/en/latest/content/reference/file-management/sorting/sort-bed.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2021-06-22T01:13:10.837" LastActivityDate="2021-06-22T01:13:10.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16202" PostTypeId="2" ParentId="16199" CreationDate="2021-06-22T10:05:19.590" Score="1" Body="&lt;p&gt;I would try it both ways (i.e. run pseudotime analysis on individual and combined samples). If you have the same transcriptional trajectories in multiple samples then you might be able to capture this via combining samples, and arguably the most interesting signatures are generalizable across samples. On the other hand, there might be batch effects that disrupt the signal of a combined analysis.&lt;/p&gt;&#xA;&lt;p&gt;I think I understand your concern is that different samples have independent cells and so they should not form part of one trajectory. However, even within a sample, we are not capturing the true trajectory as this would require sampling the same individual cell multiple times at different time points, which is not currently possible because scRNA-seq requires lysing the cells (see also &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2021.03.24.436752v1.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;live-seq&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2021-06-22T10:05:19.590" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16203" PostTypeId="1" CreationDate="2021-06-22T11:48:15.303" Score="4" ViewCount="119" Body="&lt;p&gt;I want to use &lt;a href=&quot;https://pachterlab.github.io/kallisto/about&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kallisto&lt;/a&gt; for sequence alignment in Galaxy. Its description is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;a program for quantifying abundances of transcripts from bulk and&#xA;single-cell RNA-Seq data, or more generally of target sequences using&#xA;high-throughput sequencing reads.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However, I found this field empty:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;there is no option available for the reference transcriptome.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How I use a reference sequence?&lt;/li&gt;&#xA;&lt;li&gt;Can I leave the other configuration by default?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;this is the link of the tool:  &lt;a href=&quot;https://usegalaxy.org/root?tool_id=toolshed.g2.bx.psu.edu/repos/iuc/kallisto_quant/kallisto_quant/0.43.1.3&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://usegalaxy.org/root?tool_id=toolshed.g2.bx.psu.edu/repos/iuc/kallisto_quant/kallisto_quant/0.43.1.3&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Xn5I1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Xn5I1.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12578" LastEditorUserId="3848" LastEditDate="2022-02-17T21:14:59.693" LastActivityDate="2022-02-18T16:33:30.263" Title="use Kallisto in galaxy" Tags="|sequence-alignment|transcriptome|galaxy|kallisto|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16204" PostTypeId="1" CreationDate="2021-06-22T12:00:23.387" Score="-1" ViewCount="81" Body="&lt;p&gt;Are there any databases or tools that contain a list of all RefSeq protein IDs for the interest organism/microorganism, e.g. get a specific microorganism name and return all RefSeq Protein IDs that have been identified?&lt;/p&gt;&#xA;" OwnerUserId="12595" LastActivityDate="2021-08-09T22:35:10.010" Title="A database for RefSeq protein accession IDs" Tags="|database|proteomics|refseq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16207" PostTypeId="2" ParentId="16204" CreationDate="2021-06-22T13:27:09.120" Score="0" Body="&lt;p&gt;How about using RefSeq itself? Say, you are interested in all E. coli proteins, you can use the query &lt;code&gt;&amp;quot;Escherichia coli&amp;quot;[Organism] AND RefSeq[Filter]&lt;/code&gt; &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/protein&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you know the Taxonomy identifier for the specific organism of interest to you, it can also be used. For example, if you are interested in only &amp;quot;Escherichia coli str. K-12 substr. DH10B&amp;quot;, you can use the taxonomy identifier 316385 (obtained from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?id=316385&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) in the query &lt;code&gt;txid316385[Organism] AND RefSeq[Filter]&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To extract a list of accessions and save them to a file on disk, use the &amp;quot;Send To&amp;quot; menu at the top right corner, choose &amp;quot;File&amp;quot; as destination followed by &amp;quot;Accession List&amp;quot; as the format and download the file. Alternatively, you can use E-utilities API or the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;EntrezDirect&lt;/a&gt; tool on the Unix command line to download these data programmatically.&lt;/p&gt;&#xA;&lt;p&gt;To address the question about E. coli proteins shared between different species... There's no need to parse FTP files to get all the RefSeq proteins for a bacteria species. While it's true that some RefSeq WP proteins have taxids above species level, like Enterobacterales WP_003861789.1 where the identical protein is found in genomes from multiple species of the order Enterobacterales, those proteins are still retrieved by a species-level organism search. NCBI implemented some extra handling for the RefSeq WP proteins to treat the MULTISPECIES WPs like they belong to all of the species with genomes on which they're annotated. So of the 56947 WPs assigned as &amp;quot;Bacteria&amp;quot;, 7401 are found by &lt;code&gt;&amp;quot;Escherichia coli&amp;quot;[Organism]&lt;/code&gt;, and the species-level search will give you the full set of proteins.&lt;/p&gt;&#xA;&lt;p&gt;You could also use the &lt;code&gt;AutonomousProtein2Genomic.gz&lt;/code&gt; file from the &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/refseq/release/release-catalog/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RefSeq FTP&lt;/a&gt; path which has a mapping of how all WPs map to specific genomic sequences, but it's a really big file and it would take a long time to grep for &amp;quot;Escherichia coli&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="3577" LastEditorUserId="3577" LastEditDate="2021-08-09T22:35:10.010" LastActivityDate="2021-08-09T22:35:10.010" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16209" PostTypeId="2" ParentId="16191" CreationDate="2021-06-22T20:52:13.537" Score="1" Body="&lt;p&gt;The UCSC &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgIntegrator&quot; rel=&quot;nofollow noreferrer&quot;&gt;Data Integrator&lt;/a&gt; can combine multiple tracks (CpG Islands, TFBS, cCREs, etc.) into one output annotation file, and it works with any assembly you choose. You can select which output fields you want, then download the tab-delimited file.&lt;/p&gt;&#xA;&lt;p&gt;From there, you can use R, Python, and/or GNU tools such as &lt;code&gt;awk&lt;/code&gt; and &lt;code&gt;sed&lt;/code&gt; to coerce the data into a format which is useful for your purposes. You can then load it into R as a data frame and use the columns as arguments in the GRanges constructor to create a GRanges object.&lt;/p&gt;&#xA;" OwnerUserId="12598" LastActivityDate="2021-06-22T20:52:13.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16212" PostTypeId="2" ParentId="10980" CreationDate="2021-06-23T17:56:19.397" Score="0" Body="&lt;p&gt;I solved this problem but UBCG.JAR does not work with java 11 . It is necessary to install openjdk-8-jre and switch the default java.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Install openjdk-8-jre&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ sudo apt-cache policy openjdk-8-jre&#xA;$&lt;/span&gt; sudo apt install openjdk-8-jre=1.8.0_292&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Switch the default java&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ sudo update-alternatives --config java&#xA;*            /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java   1081  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Checking the default java&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    $ java -version&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;openjdk version &amp;quot;1.8.0_292&amp;quot;&#xA;OpenJDK Runtime Environment (build 1.8.0_292-8u292-b10-0ubuntu1~18.04-b10)&#xA;OpenJDK 64-Bit Server VM (build 25.292-b10, mixed mode)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Run the UBGC.jar&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ java -jar UBCG.jar &#xA;      ###############################&#xA;      #         UBCG ver. 3.0       #&#xA;      ###############################&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12606" LastEditorUserId="650" LastEditDate="2021-06-23T20:50:30.350" LastActivityDate="2021-06-23T20:50:30.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16213" PostTypeId="1" AcceptedAnswerId="16216" CreationDate="2021-06-23T22:39:31.067" Score="0" ViewCount="730" Body="&lt;p&gt;Whats the role of &amp;quot;variable.features.n&amp;quot; in SCTransform function of Seurat? I try setting &amp;quot;variable.features.n&amp;quot; at three different numbers (2000, 5000, 10000). In the downstream cells are clustering in different groups?&lt;/p&gt;&#xA;" OwnerUserId="4951" LastActivityDate="2021-06-24T07:17:45.483" Title="&quot;variable.features.n&quot; in SCTransform" Tags="|seurat|sctransform|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16215" PostTypeId="1" AcceptedAnswerId="16217" CreationDate="2021-06-24T03:06:04.017" Score="0" ViewCount="73" Body="&lt;p&gt;I have a very big matrix I want to put colnames as cell1, cell2, et,.. but I get error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; colnames(new.dat)&#xA;NULL&#xA;&amp;gt; View(new.dat)&#xA;&amp;gt; dim(new.dat)&#xA;[1]     34 263086&#xA;&amp;gt; colnames(new.dat)=make.names(c(1: 263086, &amp;quot;cell&amp;quot;))&#xA;Error in dimnames(x) &amp;lt;- dn : &#xA;  length of 'dimnames' [2] not equal to array extent&#xA;&amp;gt; colnames(new.dat) &amp;lt;- paste0(&amp;quot;cell&amp;quot;, seq_along(new.dat))&#xA;Error in dimnames(x) &amp;lt;- dn : &#xA;  length of 'dimnames' [2] not equal to array extent&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0RXFO.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0RXFO.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;any help?&lt;/p&gt;&#xA;" OwnerUserId="9448" LastEditorUserId="9448" LastEditDate="2021-06-24T03:16:57.977" LastActivityDate="2021-06-24T08:44:14.500" Title="Making colnames by number and character" Tags="|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16216" PostTypeId="2" ParentId="16213" CreationDate="2021-06-24T07:17:45.483" Score="1" Body="&lt;p&gt;&lt;code&gt;variable.features.n&lt;/code&gt; sets the number of features (you can think of genes in the case of scRNA-seq) you would like to use for the downstream steps such as clustering. Basically, using a gene that is expressed more or less at similar levels across different cell types would not be informative in terms of differentiating (for example via clustering) these cell types, this parameter helps you to choose the &amp;quot;most informative&amp;quot; (most variable across the whole data set) features.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2021-06-24T07:17:45.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16217" PostTypeId="2" ParentId="16215" CreationDate="2021-06-24T08:44:14.500" Score="3" Body="&lt;p&gt;You should really test your code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;make.names(c(1: 5, &amp;quot;cell&amp;quot;))&#xA;[1] &amp;quot;X1&amp;quot;   &amp;quot;X2&amp;quot;   &amp;quot;X3&amp;quot;   &amp;quot;X4&amp;quot;   &amp;quot;X5&amp;quot;   &amp;quot;cell&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is obviously wrong.&lt;/p&gt;&#xA;&lt;p&gt;Better:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;colnames(new.dat) &amp;lt;- paste0(&amp;quot;cell&amp;quot;, 1:ncol(new.dat))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-06-24T08:44:14.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16218" PostTypeId="1" AcceptedAnswerId="16222" CreationDate="2021-06-24T12:57:45.117" Score="1" ViewCount="227" Body="&lt;p&gt;have a folder with roughly 1000 vcf files which have divided the human genome into chunks, the folder looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;- main_programme_aggregated_chr1_1_194789237.vcf.gz&#xA;....&#xA;- main_programme_aggregated_chrX_14235_8759493845.vcf.gz&#xA;etc &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I will divide this file name into:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;*_chr[1-22, X,Y]_A_B.* &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have a separate tab divided file with the names of genes and their genomic coordinates (all the genes in the human genome on build 38) - file1:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1 19853 90835 WASH7P ENSG0000022454&#xA;...&#xA;chr10 38732 390853 JHBF ENSG0000578382&#xA;..&#xA;chrX 9532 908032 LOTY ENSG00005847&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;etc&lt;/p&gt;&#xA;&lt;p&gt;I would like to use the gene name from the 4th column of file1 to find the relevant chunked file it falls within. So I would need to take column 1 of file 1, match it to chr[1-22, X or Y] and then find the chunk were the range of columns2+3 fall between (A-B) in the file name.&lt;/p&gt;&#xA;&lt;p&gt;I do not need to go into the files themselves, just use their names.&lt;/p&gt;&#xA;&lt;p&gt;I have been just manually looking up the gene coordinates and the relevant chunk but am sure this can be automated.&lt;/p&gt;&#xA;&lt;p&gt;An example would be that I want to get the chunk file that the gene WASH7P falls under, so the relevant file would be main_programme_aggregated_chr1_1_194789237.vcf.gz.&lt;/p&gt;&#xA;&lt;p&gt;Ideally I could do this from the command line and the relevant chunk file then be inputted as a variable into some downstream workflows I have.&lt;/p&gt;&#xA;&lt;p&gt;Would anyone be able to help? Many thanks for your time&lt;/p&gt;&#xA;" OwnerUserId="9990" LastActivityDate="2021-06-28T23:19:41.007" Title="Pulling a numbered chromosome range file given a gene location from a lookup table ideally from command line or R" Tags="|r|linux|shell|genomicranges|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16219" PostTypeId="1" CreationDate="2021-06-24T16:02:33.850" Score="0" ViewCount="56" Body="&lt;p&gt;I am new in R sorry for the naive question. I want to generate the seqlogo for each position using R. Can anyone guide me? Thank you so much in advance.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1  GATGGGGTGGGCAACTTAGAACTT&#xA;2  ATAACTGT&#xA;3  CAGAGGCGAACT&#xA;4  GGGCTAGGG&#xA;5  GGGGGGTGGCAGCACTAGC&#xA;6  GGGTAATATG&#xA;7  GTCGAGCGGT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Best&lt;/p&gt;&#xA;" OwnerUserId="12608" LastEditorUserId="73" LastEditDate="2021-06-25T10:39:47.050" LastActivityDate="2021-06-25T10:39:47.050" Title="How do I generate sequence logo of ATGC against each position in R?" Tags="|r|sequence|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16220" PostTypeId="2" ParentId="16219" CreationDate="2021-06-24T16:15:55.913" Score="3" Body="&lt;p&gt;It's good practice to google before asking on online forums. Searching for &amp;quot;r seqlogo&amp;quot; pointed me to the seqLogo BioConductor package, which might be really helpful to you: &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/seqLogo/inst/doc/seqLogo.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/vignettes/seqLogo/inst/doc/seqLogo.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You will need your sequences aligned though.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2021-06-24T16:15:55.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16221" PostTypeId="2" ParentId="16218" CreationDate="2021-06-24T20:54:05.577" Score="2" Body="&lt;p&gt;Below is a Python solution:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;&#xA;files = {}&#xA;&#xA;for root, dirs, files in os.walk('/path/to/vcf/files'):&#xA;    for file in files:&#xA;        # Example: main_programme_ver2_chr22_2874210_2947732_VEPannot.vcf.gz&#xA;        chrom = file.split('_')[3]&#xA;        start = int(file.split('_')[4])&#xA;        end = int(file.split('_')[5])&#xA;        files[os.path.abspath(file)] = {'chrom': chrom, 'start': start, 'end': end}&#xA;    break&#xA;&#xA;genes = {}&#xA;&#xA;with open(file1) as f:&#xA;    for line in f:&#xA;        fields = line.strip().split('\t')&#xA;        chrom = fields[0]&#xA;        start = int(fields[1])&#xA;        end = int(fields[2])&#xA;        gene = fields[3]&#xA;        genes[gene] = {'chrom': chrom, 'start': start, 'end': end}&#xA;&#xA;mapping = {}&#xA;&#xA;for gene in genes:&#xA;    for file in files:&#xA;        if (genes[gene]['chrom'] == files[file]['chrom'] and&#xA;            genes[gene]['start'] &amp;gt;= files[file]['start'] and&#xA;            genes[gene]['end'] &amp;lt;= files[file]['end']):&#xA;            mapping[gene] = file&#xA;            break&#xA;&#xA;with open('mappings.txt', 'w') as f:&#xA;    for gene in mapping:&#xA;        f.write(gene + '\t' + file + '\n')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12502" LastEditorUserId="12502" LastEditDate="2021-06-28T23:19:41.007" LastActivityDate="2021-06-28T23:19:41.007" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16222" PostTypeId="2" ParentId="16218" CreationDate="2021-06-24T23:00:35.760" Score="2" Body="&lt;p&gt;Here is a solution with &lt;code&gt;bash&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;# 1 - create an array of coordinate files&#xA;&#xA;# use nullglob in case there are no matching files,&#xA;# not that important for this toy case&#xA;shopt -s nullglob&#xA;&#xA;# make a(n) array/list of vcf file in the current (./) directory&#xA;coordinate_files=(./*.vcf.gz)&#xA;&#xA;# going through each line of &amp;quot;file1&amp;quot;&#xA;while read line; do&#xA;    &#xA;    # 2 - get the gene related info from &amp;quot;file1&amp;quot;&#xA;    # 2.1 - change line to an array (line split by whitespace)&#xA;    line_array=( &lt;span class=&quot;math-container&quot;&gt;$line )&#xA;    # 2.2 - get the 4th column, gene name and chr, the first column&#xA;    gene=$&lt;/span&gt;{line_array[3]}&#xA;    chr=&lt;span class=&quot;math-container&quot;&gt;${line_array[0]}&#xA;    # 2.3 get coordinates&#xA;    start=$&lt;/span&gt;{line_array[1]}&#xA;    end=${line_array[2]}&#xA;    &#xA;    # loop through each file in the array created at step 1&#xA;    for ((i=0; i&amp;lt;${#coordinate_files[@]}; i++)); do&#xA;        &#xA;        # parse the info hardcoded in the filename&#xA;        chr_in_filename=&lt;span class=&quot;math-container&quot;&gt;$(echo $&lt;/span&gt;{coordinate_files[&lt;span class=&quot;math-container&quot;&gt;$i]} | awk -F_ '{print $4}')&#xA;        start_in_filename=$&lt;/span&gt;(echo &lt;span class=&quot;math-container&quot;&gt;${coordinate_files[$i]} | awk -F_ '{print $5}')&#xA;        end_in_filename=$&lt;/span&gt;(echo &lt;span class=&quot;math-container&quot;&gt;${coordinate_files[$i]} | awk -F_ '{print $6}')&#xA;        end_in_filename=$&lt;/span&gt;{end_in_filename%.vcf.gz}&#xA;&#xA;        if [ &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$chr&quot; == &quot;$&lt;/span&gt;chr_in_filename&amp;quot; ]&#xA;        then&#xA;            if [ &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$start&quot; -ge &quot;$&lt;/span&gt;start_in_filename&amp;quot; ] &amp;amp;&amp;amp; [ &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$end&quot; -le &quot;$&lt;/span&gt;end_in_filename&amp;quot; ]&#xA;            then&#xA;                echo &lt;span class=&quot;math-container&quot;&gt;$gene&#xA;                echo $&lt;/span&gt;{coordinate_files[$i]}&#xA;            fi&#xA;        fi&#xA;    done&#xA;    &#xA;done &amp;lt;file1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The input is the two example coordinate files + &amp;quot;file1&amp;quot; in the question and the output is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;WASH7P&#xA;chr1 19853 90835 WASH7P ENSG0000022454&#xA;./main_programme_aggregated_chr1_1_194789237.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2021-06-25T10:29:06.773" LastActivityDate="2021-06-25T10:29:06.773" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16223" PostTypeId="2" ParentId="13472" CreationDate="2021-06-25T11:11:45.950" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @user1271772, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Try on this site: &lt;a href=&quot;https://mattermodeling.stackexchange.com/questions/tagged/gromacs&quot;&gt;https://mattermodeling.stackexchange.com/questions/tagged/gromacs&lt;/a&gt;. There's an entire GROMACS tag there.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Answer from @vishal-kumar-sahu, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Try running/resuming using &lt;code&gt;-cpi&lt;/code&gt; the simulation again, I got NaN error and rerun kept continued.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-06-25T11:11:45.950" CommentCount="0" CommunityOwnedDate="2021-06-25T11:11:45.950" ContentLicense="CC BY-SA 4.0" />
  <row Id="16224" PostTypeId="2" ParentId="9175" CreationDate="2021-06-25T11:12:54.260" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @merv, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Perhaps it's an insert size issue. Can you get the post library construction QC info? You might also see this in the BAMs, with the first showing a full bell-shaped peaks, and the latter truncating the peaks at the cleavage site. (Most 10X datasets I've seen have very few reads traversing the 3' end cleavage site.)&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-06-25T11:12:54.260" CommentCount="0" CommunityOwnedDate="2021-06-25T11:12:54.260" ContentLicense="CC BY-SA 4.0" />
  <row Id="16225" PostTypeId="2" ParentId="8770" CreationDate="2021-06-25T11:17:26.477" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @cephbirk, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;What I ended up doing was to choose the read coverage for the lowest covered nucleotide in each codon as each residue's read coverage.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-06-25T11:17:26.477" CommentCount="0" CommunityOwnedDate="2021-06-25T11:17:26.477" ContentLicense="CC BY-SA 4.0" />
  <row Id="16226" PostTypeId="2" ParentId="112" CreationDate="2021-06-25T13:18:25.937" Score="2" Body="&lt;p&gt;I'm a bit late to answer this question, but we've developed pretty much exactly the system you describe, based on Git and git-annex.&#xA;Its called DataLad, free and open source software, and currently mostly known within the neurosciences.&lt;/p&gt;&#xA;&lt;p&gt;DataLad's core data structure are DataLad datasets: Git repositories with an optional annex for large or private data. Within a dataset, code and data can be version controlled in conjunction.&#xA;You can stick to Git and git-annex commands for handling the dataset, but DataLad also offers a core API that aims to harmonize and simplify version control operations. For example, in a dataset configured according to your preferences, a &lt;code&gt;datalad save&lt;/code&gt; commits a file based on name (glob), file type, size, or location into either Git or git-annex (instead of &lt;code&gt;git add&lt;/code&gt;, &lt;code&gt;git annex add&lt;/code&gt;, and &lt;code&gt;git commit&lt;/code&gt;). And likewise, a &lt;code&gt;datalad push&lt;/code&gt; can push Git history and annexed data to your clones (instead of &lt;code&gt;git push&lt;/code&gt; and &lt;code&gt;git-annex sync&lt;/code&gt;). Everything is completely decentral, and DataLad integrates via git-annex special remote mechanism with a variety of hosting providers (S3, OSF, DropBox, ...), where you could store data if you like. A high level overview is here: &lt;a href=&quot;http://handbook.datalad.org/r.html?about&quot; rel=&quot;nofollow noreferrer&quot;&gt;handbook.datalad.org/r.html?about&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Unfortunately it seems it does not work well with the large number of files. Often I have more that 10k small files in my analysis. There are some tricks to improve indexing, but it doesn't solve the issue. What I need is one symlink representing the full contents of directory.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;DataLad overcomes this issue by using Git's submodule mechanishm. Datasets can nested, and operated on recursively (to preserver a monorepo-like feel). With this mechanism, we've for example exposed the human connectome project dataset (80TB, 15 million files) to GitHub: &lt;a href=&quot;https://github.com/datalad-datasets/human-connectome-project-openaccess&quot; rel=&quot;nofollow noreferrer&quot;&gt;github.com/datalad-datasets/human-connectome-project-openaccess&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12616" LastActivityDate="2021-06-25T13:18:25.937" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16227" PostTypeId="1" CreationDate="2021-06-25T13:28:17.400" Score="2" ViewCount="153" Body="&lt;p&gt;I am trying to learn how to analyse normalised DIA-MS data and I am struggling with it ://&lt;/p&gt;&#xA;&lt;p&gt;The original dataset I got is &lt;strong&gt;(6 conditions (2 samples each)) with 3 technical replicates (total: 36 sample columns x 1128 proteins in rows)&lt;/strong&gt;. I removed the proteins with more than 80% of NA, and impute with kNN the rest of missing values.&lt;/p&gt;&#xA;&lt;p&gt;Then I collapse the replicates doing the average of the 3 replicates (is this correct?)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;So now, I have a dataframe with 12 columns x 1072 rows.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The question is... I am not sure how to perform the differential analysis from here (assuming I did well the previous steps lol)&lt;/p&gt;&#xA;&lt;p&gt;Could someone help me with the tools or code to compare all the 6 conditions between them?&lt;/p&gt;&#xA;&lt;p&gt;I read something about Limma package, but not sure if it can be used in this DIA-ms data and neither sure of how to use limma for it (I already check the vignette with no succeed)&lt;/p&gt;&#xA;&lt;p&gt;UPDATE: I tried to use Limma as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fit1 &amp;lt;- lmFit(norm.data_collapsed, design)&#xA;fit2 &amp;lt;- contrasts.fit(fit1, contrasts = contrast)&#xA;fit3 &amp;lt;- eBayes(fit2)&#xA;limma.res.01 &amp;lt;- topTable(fit3, coef = comb.contrast[1], n= Inf)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This returns me a non-sense values for logFC or significance:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Sl8xX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Sl8xX.png&quot; alt=&quot;head of the results topTable&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Any help would be very welcome,&lt;/p&gt;&#xA;&lt;p&gt;Best regards&lt;/p&gt;&#xA;" OwnerUserId="12617" LastEditorUserId="12617" LastEditDate="2021-06-25T15:15:28.607" LastActivityDate="2024-03-14T17:09:06.297" Title="Help with DIA-Mass Spectrometry data analysis with several conditions (limma?)" Tags="|differential-expression|proteomics|limma|mass-spectrometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16228" PostTypeId="2" ParentId="2514" CreationDate="2021-06-25T15:21:37.867" Score="2" Body="&lt;p&gt;As you answered yourself @Death Metal, &lt;code&gt;voom&lt;/code&gt; will by default not perform additional normalization.&lt;br /&gt;&#xA;However in virtually all cases you would want to do some kind of &lt;a href=&quot;https://dx.doi.org/10.1093%2Fbib%2Fbbx008&quot; rel=&quot;nofollow noreferrer&quot;&gt;normalization&lt;/a&gt; at least to correct for differences in sequenced reads between the samples. This is why in the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/limma/inst/doc/usersguide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt; (page 71 right at the top) the &lt;code&gt;calcNormFactors&lt;/code&gt; function from  &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/html/edgeR.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;edgeR&lt;/code&gt;&lt;/a&gt; is used. This does a global &lt;code&gt;TMM&lt;/code&gt; normalization for differences in library size.&lt;/p&gt;&#xA;&lt;p&gt;Basically &lt;code&gt;voom&lt;/code&gt; assumes that you did a normalization and will not do a second one because this could cause all kinds of problems.&lt;/p&gt;&#xA;&lt;p&gt;In some cases you may prefer normalization other than &lt;code&gt;TMM&lt;/code&gt;, for example if the number of genes detected in your samples is widely different. In this case is could be better to do &lt;code&gt;quantile&lt;/code&gt; normalization that is available through the &lt;code&gt;normalizeBetweenArrays&lt;/code&gt; function in &lt;code&gt;voom&lt;/code&gt;.&lt;br /&gt;&#xA;However, this may over-correct your data and get rid of many interesting differences. In these cases I have found &lt;a href=&quot;https://dx.doi.org/10.1093/biostatistics/kxx028&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;qsmooth&lt;/code&gt;&lt;/a&gt; quite helpful. Here quantile normalization is only applied across groups if the local distributions are similar enough.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2021-06-25T15:21:37.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16229" PostTypeId="1" AcceptedAnswerId="16231" CreationDate="2021-06-25T19:57:53.020" Score="1" ViewCount="1057" Body="&lt;p&gt;I am sequencing long read genomic sequences to assemble MHC region haplotypes in a non model organism using Cas9 Sequencing Kit (SQK-CS9109) using flongle adaptors in the minION from Oxford Nanopore Technologies (ONT). However the error rate in genomic sequences results from ONT has been over 14%.&lt;/p&gt;&#xA;&lt;p&gt;I know that there is error correction using Illumina sequencing and I was hoping anyone knows more about how this is accomplished?&lt;/p&gt;&#xA;&lt;p&gt;Additionally, there are bioinformatic tools to error correct these errors? Has anyone accomplished diminishing this big error rate using bioinformatic tools?&lt;/p&gt;&#xA;" OwnerUserId="12135" LastEditorUserId="73" LastEditDate="2021-06-26T10:45:47.263" LastActivityDate="2021-06-28T12:32:20.580" Title="What are some ways to error correct Oxford Nanopore long read sequencing?" Tags="|nanopore|illumina|errors|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16230" PostTypeId="1" AcceptedAnswerId="16232" CreationDate="2021-06-25T22:36:35.130" Score="0" ViewCount="35" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/9477217/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;The position that I have to visualize is in the 2700 position and this position exists in two ranges in &lt;a href=&quot;https://www.uniprot.org/uniprot/Q12802#family_and_domains&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uniprot/Q12802#family_and_domains&lt;/a&gt; the first range is Interaction with ESR1 and the second region is disordered, the first question is which range I should be based on? and the second is Why &lt;strong&gt;intrinsically disordered protein(IDP)&lt;/strong&gt; can't be visualized in 3D?&lt;/p&gt;&#xA;" OwnerUserId="12568" LastEditorUserId="73" LastEditDate="2021-06-30T12:08:52.077" LastActivityDate="2021-06-30T12:08:52.077" Title="Intrinsically disordered protein" Tags="|3d-structure|structural-biology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16231" PostTypeId="2" ParentId="16229" CreationDate="2021-06-25T22:42:25.053" Score="4" Body="&lt;p&gt;How are you evaluating sequencing error rate? My most recent re-calls of 2017 sequences are demonstrating median single-read accuracy over 96%.&lt;/p&gt;&#xA;&lt;p&gt;Before considering Illumina, it'd be worth it to do an initial correction using nanopore-only reads. This will make sure that the best results are obtained from the nanopore signal. First, make sure that the reads have been recalled using the latest highest-accuracy basecaller (currently guppy v5.0.11):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;guppy_basecaller -c dna_r9.4.1_450bps_sup.cfg ... # for 9.4.1 reads&#xA;guppy_basecaller -c dna_r10.3_450bps_sup.cfg ... # for 10.3 reads&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you don't have any reference sequences available, reads can be corrected using the other reads by using &lt;a href=&quot;https://canu.readthedocs.io/en/latest/quick-start.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;canu&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;canu -correct -p nonModel -d nonModel genomeSize=10m -nanopore reads.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you &lt;em&gt;do&lt;/em&gt; have reference sequences available, then reads can be corrected using &lt;a href=&quot;https://github.com/nanoporetech/megalodon#getting-started&quot; rel=&quot;nofollow noreferrer&quot;&gt;megalodon&lt;/a&gt;. I don't have any experience with megalodon, but have seen a couple of papers demonstrating that it has good accuracy, especially for methylation calling.&lt;/p&gt;&#xA;&lt;p&gt;After all that, if you're still not happy with the accuracy, you can use &lt;a href=&quot;https://github.com/broadinstitute/pilon&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pilon&lt;/a&gt; for read correction using the Illumina reads. Pilon is used for correcting a &lt;em&gt;reference&lt;/em&gt; sequence, so you'll need to create your haplotype assembly first before using Pilon. Even when using Pilon, I'd still recommend only using the Illumina reads for &lt;a href=&quot;https://github.com/broadinstitute/pilon/wiki/Methods-of-Operation#finding-and-fixing-small-indels&quot; rel=&quot;nofollow noreferrer&quot;&gt;local INDEL correction&lt;/a&gt;. Pileup nanopore reads are very accurate for substitutions, especially when considering [lack of] strand bias when evaluating variants.&lt;/p&gt;&#xA;&lt;p&gt;Update:&lt;/p&gt;&#xA;&lt;p&gt;Given that you're interested in determining haplotypes across the MHC, which is something that performs poorly for short reads even for well-annotated genomes, I think you're better of sticking with reference-free correction, and categorising haplotypes while taking taking this into account. If you just need to be able to distinguish haplotypes, I recommend doing homopolymer compression on the resultant sequences, as hompolymer inaccuracies are the most common error mode for nanopore reads. Even if that doesn't get you all the way to perfect haplotypes, it might get close enough that something like &lt;a href=&quot;http://weizhong-lab.ucsd.edu/cdhit_suite/cgi-bin/index.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;CDHIT&lt;/a&gt; can be used for binning similar sequences.&lt;/p&gt;&#xA;&lt;p&gt;HLA &lt;em&gt;typing&lt;/em&gt; in humans can be done from nanopore reads (e.g. see &lt;a href=&quot;https://doi.org/10.3389/fgene.2018.00152&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;a href=&quot;https://doi.org/10.1016/j.humimm.2021.02.005&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), but don't expect perfect base-level accuracy. What I'm suggesting is something similar to a typing process, but without an established reference, i.e. binning reads based on their similarity to each other.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-06-28T12:32:20.580" LastActivityDate="2021-06-28T12:32:20.580" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16232" PostTypeId="2" ParentId="16230" CreationDate="2021-06-25T22:48:54.413" Score="3" Body="&lt;p&gt;Asking how to visualise the disordered region of a protein is a bit like asking how to visualise the location of an electron.&lt;/p&gt;&#xA;&lt;p&gt;An &lt;em&gt;intrinsically disordered protein&lt;/em&gt;, by definition, has regions that are disordered. In other words, their location is variable under observation, so they cannot be precisely placed in a 3D model.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-06-25T22:48:54.413" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16233" PostTypeId="1" AcceptedAnswerId="16234" CreationDate="2021-06-26T01:19:12.553" Score="0" ViewCount="37" Body="&lt;p&gt;I am trying to start a side project where I verify the claim that human and mice/monkey share 99% of their genes. I hope to learn the basics of doing genomic analysis.&lt;/p&gt;&#xA;&lt;p&gt;I would like guidance about where I can find the sequences I need, and programming languages or applications that I should use. For the sequences, I would like for example, a list of all genes for each species. In the case that the programming language or application is foreign, some basic guidance would be helpful.&lt;/p&gt;&#xA;&lt;p&gt;I have programming experience with matlab and python, and have learnt R previously, though forgotten. Would be able to learn a new language.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, I think this may be a very large subject. If there is a link to a tutorial/lectures it will also be helpful.&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="12623" LastEditorUserId="73" LastEditDate="2021-06-26T02:10:13.703" LastActivityDate="2021-06-26T18:11:06.570" Title="Comparing the genetic similarity of human genome and mice/monkey genome - where can I find a list of genes?" Tags="|genome|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16234" PostTypeId="2" ParentId="16233" CreationDate="2021-06-26T02:20:59.003" Score="1" Body="&lt;p&gt;For downloading lists of genes, together with associated features, I like using Ensembl Biomart:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://ensembl.org/biomart/martview/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ensembl.org/biomart/martview/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this case, you can &amp;quot;CHOOSE DATABASE -&amp;gt; Ensembl Genes&amp;quot;, then &amp;quot;CHOOSE DATASET -&amp;gt; Human Genes&amp;quot; or &amp;quot;CHOOSE DATASET -&amp;gt; Chimpanzee genes&amp;quot; to get to a table selection.&lt;/p&gt;&#xA;&lt;p&gt;Clicking on &amp;quot;Attributes&amp;quot; on the left hand sidebar brings you to a selector for different fields to add to the table.&lt;/p&gt;&#xA;&lt;p&gt;Clicking on the &amp;quot;Results&amp;quot; button at the top left brings you to a form for choosing how results should be output.&lt;/p&gt;&#xA;&lt;p&gt;Bear in mind that gene names are not likely to match. Finding the intersection of gene sets between species is a complicated process, usually requiring knowledge about gene copies, paralogs, orthologs, and sequence comparison.&lt;/p&gt;&#xA;&lt;p&gt;It's also likely that the human genes are more annotated than the chimpanzee genes, and there might be genes present in the human list that are also in the chimpanzee &lt;em&gt;genome&lt;/em&gt;, but have not yet been discovered.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-06-26T02:20:59.003" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16239" PostTypeId="1" AcceptedAnswerId="16241" CreationDate="2021-06-27T13:14:28.053" Score="4" ViewCount="695" Body="&lt;p&gt;I have a list of UniProt ids that I want to get Gene Ontology annotations for. I need this information because I want this high-level information as an input to a neural network. The model I wish to develop is inspired by this paper: &lt;a href=&quot;https://www.ijcai.org/Proceedings/2018/0468.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ijcai.org/Proceedings/2018/0468.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can anyone advise how I can programmatically access this information in the&#xA;Gene Ontology Annotation (GOA) Database? &lt;a href=&quot;https://www.ebi.ac.uk/GOA/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/GOA/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12641" LastEditorUserId="298" LastEditDate="2021-06-29T16:17:34.713" LastActivityDate="2021-06-29T16:19:26.737" Title="How do I get GO annotations for a list of UniProt IDs?" Tags="|proteins|uniprot|gene-ontology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16240" PostTypeId="1" AcceptedAnswerId="16261" CreationDate="2021-06-27T13:18:56.217" Score="4" ViewCount="181" Body="&lt;p&gt;I am trying to open .bam files in C++ to extract reads occurring at specific scaffolds and loci. I essentially want to call &amp;quot;samtools view sample.bam -o sample.sam scaffold:pos-pos&amp;quot; from C++. I have tried system(&amp;quot;samtools view sample.bam -o sample.sam scaffold:pos-pos&amp;quot;) and can explain the issues if you want to know, but for the sake of conciseness I will leave the details out of this post. So, I am using SeqAn. When I run my script, I get the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;terminate called after throwing an instance of 'seqan::UnknownExtensionError'&#xA;  what():  Unknown file extension of copyPolly_SRR6511930.bam: iostream error&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am running my program on my institution's supercomputer clusters (Texas A&amp;amp;M Terra cluster), to which I do &lt;strong&gt;not&lt;/strong&gt; have sudo access. I have loaded the following modules:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GCC/9.3.0&#xA;SeqAn/2.4.0-GCCcore-9.3.0&#xA;zlib/1.2.11-GCCcore-9.3.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here is my minimal script:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;seqan/bam_io.h&amp;gt;&#xA;#include &amp;lt;zlib.h&amp;gt;&#xA;&#xA;using namespace seqan;&#xA;&#xA;int main(){&#xA;    CharString bamFileName = &amp;quot;copyPolly_SRR6511930.bam&amp;quot;;&#xA;// Open input file.&#xA;    BamFileIn bamFileIn(toCString(bamFileName));&#xA;    if (!open(bamFileIn, toCString(bamFileName))){&#xA;        std::cerr &amp;lt;&amp;lt; &amp;quot;ERROR: Could not open &amp;quot; &amp;lt;&amp;lt; bamFileName &amp;lt;&amp;lt; std::endl;&#xA;        return 1;&#xA;    }&#xA;&#xA;    unsigned numUnmappedReads = 0;&#xA;&#xA;    try{&#xA;        // Read header.&#xA;        BamHeader header;&#xA;        readHeader(header, bamFileIn);&#xA;&#xA;        // Read records.&#xA;        BamAlignmentRecord record;&#xA;        while (!atEnd(bamFileIn)){&#xA;            readRecord(record, bamFileIn);&#xA;            if (hasFlagUnmapped(record))&#xA;                numUnmappedReads += 1;&#xA;        }&#xA;    }&#xA;    catch (Exception const &amp;amp; e)&#xA;    {&#xA;        std::cout &amp;lt;&amp;lt; &amp;quot;ERROR: &amp;quot; &amp;lt;&amp;lt; e.what() &amp;lt;&amp;lt; std::endl;&#xA;        return 1;&#xA;    }&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have tried the following solutions:&lt;/p&gt;&#xA;&lt;p&gt;-Adding &lt;code&gt;#define SEQAN_HAS_ZLIB (1)&lt;/code&gt; to my script's header&lt;/p&gt;&#xA;&lt;p&gt;-Adding &lt;code&gt;-D SEQAN_HAS_ZLIB&lt;/code&gt; to my compilation such that it reads &lt;code&gt;g++ script.cpp -D SEQAN_HAS_ZLIB&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;-Adding &lt;code&gt;-lz&lt;/code&gt; to my compilation such that it reads &lt;code&gt;g++ script.cpp -lz&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am not dead-set on using SeqAn for this project, so if anyone knows of another C++ library which  extracts specific reads from .bam files, please share.&lt;/p&gt;&#xA;&lt;p&gt;Update: I've never used HTSlib and it seems to have a fairly steep learning curve, so if anyone has an answer for the SeqAn error, that would still be very much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="9118" LastEditorUserId="9118" LastEditDate="2021-06-29T16:04:05.523" LastActivityDate="2023-01-01T08:45:26.457" Title="Unable to open .bam file in C++ using SeqAn due to 'seqan::UnknownExtensionError'" Tags="|sam|samtools|c++|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16241" PostTypeId="2" ParentId="16239" CreationDate="2021-06-27T23:53:33.193" Score="4" Body="&lt;p&gt;The link to the FTP for the &lt;a href=&quot;ftp://ftp.ebi.ac.uk/pub/databases/GO/goa/UNIPROT/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GOA database files&lt;/a&gt; is listed on the &lt;a href=&quot;https://www.ebi.ac.uk/GOA/downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;GOA Downloads page&lt;/a&gt;. The file containing the mapping info you seek, &lt;code&gt;goa_uniprot.all&lt;/code&gt;, comes in two formats, &lt;code&gt;.gaf&lt;/code&gt; and &lt;code&gt;.gpa&lt;/code&gt;. The README in the FTP directory linked provides details on the structure of the files.&lt;/p&gt;&#xA;&lt;p&gt;Below is the first 20 lines of the &lt;code&gt;.gaf&lt;/code&gt; file after uncompressing. Column 2 contains the UniProtID and column 5 contains the associated GO term. (Note that column 4 is empty in the snippet below)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;$ head goa_uniprot_all.gaf  -n 20&#xA;!gaf-version: 2.2&#xA;!&#xA;!The set of protein accessions included in this file is based on UniProt reference proteomes, which provide one protein per gene.&#xA;!They include the protein sequences annotated in Swiss-Prot or the longest TrEMBL transcript if there is no Swiss-Prot record.&#xA;!&#xA;!date-generated: 2021-02-12 21:14&#xA;!generated-by: UniProt&#xA;!go-version: http://purl.obolibrary.org/obo/go/releases/2021-02-08/extensions/go-plus.owl&#xA;!&#xA;UniProtKB   A0A009GUA7  J508_4179       GO:0003677  GO_REF:0000002  IEA InterPro:IPR006119|InterPro:IPR036162   F   Resolvase/invertase-type recombinase catalytic domain-containing protein    J508_4179   protein taxon:1310609   20201128    InterPro        &#xA;UniProtKB   A0A009GUA7  J508_4179       GO:0006310  GO_REF:0000002  IEA InterPro:IPR006119|InterPro:IPR036162   P   Resolvase/invertase-type recombinase catalytic domain-containing protein    J508_4179   protein taxon:1310609   20201128    InterPro        &#xA;UniProtKB   A0A009GUA7  J508_4179       GO:0000150  GO_REF:0000002  IEA InterPro:IPR006119|InterPro:IPR036162   F   Resolvase/invertase-type recombinase catalytic domain-containing protein    J508_4179   protein taxon:1310609   20201128    InterPro        &#xA;UniProtKB   A0A009HCR2  J517_0313       GO:0005886  GO_REF:0000044  IEA UniProtKB-SubCell:SL-0039   C   Biopolymer transport ExbD/TolR family protein   J517_0313   protein taxon:1310618   20201128    UniProt     &#xA;UniProtKB   A0A009HCR2  J517_0313       GO:0016020  GO_REF:0000043  IEA UniProtKB-KW:KW-0472    C   Biopolymer transport ExbD/TolR family protein   J517_0313   protein taxon:1310618   20201128    UniProt     &#xA;UniProtKB   A0A009HCR2  J517_0313       GO:0016021  GO_REF:0000043  IEA UniProtKB-KW:KW-0812    C   Biopolymer transport ExbD/TolR family protein   J517_0313   protein taxon:1310618   20201128    UniProt     &#xA;UniProtKB   A0A009HCR2  J517_0313       GO:0015031  GO_REF:0000043  IEA UniProtKB-KW:KW-0653    P   Biopolymer transport ExbD/TolR family protein   J517_0313   protein taxon:1310618   20201128    UniProt     &#xA;UniProtKB   A0A009HCR2  J517_0313       GO:0005886  GO_REF:0000043  IEA UniProtKB-KW:KW-1003    C   Biopolymer transport ExbD/TolR family protein   J517_0313   protein taxon:1310618   20201128    UniProt     &#xA;UniProtKB   A0A009HCR2  J517_0313       GO:0055085  GO_REF:0000002  IEA InterPro:IPR003400  P   Biopolymer transport ExbD/TolR family protein   J517_0313   protein taxon:1310618   20201128    InterPro        &#xA;UniProtKB   A0A009HCR2  J517_0313       GO:0022857  GO_REF:0000002  IEA InterPro:IPR003400  F   Biopolymer transport ExbD/TolR family protein   J517_0313   protein taxon:1310618   20201128    InterPro        &#xA;UniProtKB   A0A009HLK9  J512_2607       GO:0006355  GO_REF:0000002  IEA InterPro:IPR000847|InterPro:IPR037402   P   HTH lysR-type domain-containing protein J512_2607   protein taxon:1310613   20201128    InterPro&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6628" LastEditorUserId="298" LastEditDate="2021-06-29T16:19:26.737" LastActivityDate="2021-06-29T16:19:26.737" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16242" PostTypeId="2" ParentId="16240" CreationDate="2021-06-28T09:16:04.920" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/samtools/htslib&quot; rel=&quot;nofollow noreferrer&quot;&gt;Htslib&lt;/a&gt; is the official library to access alignment files. It supports more formats. The typical way to read through a SAM/BAM/CRAM:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-cpp prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;quot;htslib/sam.h&amp;quot;&#xA;int main(int argc, char *argv[])&#xA;{&#xA;  samFile *fp = sam_open(argv[1], &amp;quot;r&amp;quot;);&#xA;  bam_hdr_t *h = sam_hdr_read(fp);&#xA;  bam1_t *b = bam_init1();&#xA;  while (sam_read1(fp, h, b) &amp;gt;= 0) {&#xA;    if (b-&amp;gt;core.tid &amp;lt; 0) continue;&#xA;    int end_pos = bam_endpos(b);&#xA;    printf(&amp;quot;%s\t%d\t%d\n&amp;quot;, h-&amp;gt;target_name[b-&amp;gt;core.tid], b-&amp;gt;core.pos, end_pos);&#xA;  }&#xA;  bam_destroy1(b);&#xA;  bam_hdr_destroy(h);&#xA;  sam_close(fp);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;PS: the above started as a comment for &amp;quot;relevant but minor or transient information to&amp;quot; the original question. I didn't consider htslib a direct answer to the question, but because this had been changed to an answer, I added some details anyway. By the way, seqan2 has largely been discontinued. Use seqan3.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorDisplayName="user3051" LastEditDate="2021-06-30T10:31:31.487" LastActivityDate="2021-06-30T10:31:31.487" CommentCount="1" CommunityOwnedDate="2021-06-28T09:16:04.920" ContentLicense="CC BY-SA 4.0" />
  <row Id="16243" PostTypeId="2" ParentId="16227" CreationDate="2021-06-28T12:09:25.370" Score="1" Body="&lt;p&gt;Regarding the big FC values, I just realise the Limma lmfit function requires as input a log2 matrix... so I just transformed my data and now makes more sense. Still I have the doubt regarding the collapsing technical replicates and and if some one can verify the pipeline would be like that.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!!!&lt;/p&gt;&#xA;&lt;p&gt;Fer&lt;/p&gt;&#xA;" OwnerUserId="12617" LastActivityDate="2021-06-28T12:09:25.370" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16244" PostTypeId="2" ParentId="9175" CreationDate="2021-06-28T13:48:20.520" Score="-1" Body="&lt;p&gt;I actually figured out what was going on some time ago:&lt;/p&gt;&#xA;&lt;p&gt;The over-represented sequence is the &lt;a href=&quot;https://teichlab.github.io/scg_lib_structs/methods_html/10xChromium3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;template switching oligo (TSO)&lt;/a&gt; that is used for the synthesis of the second strand (the example is for the v2 chemistry but the principle is the same for v3 and later).&lt;br /&gt;&#xA;This is a simplified illustration from the 10x manual:&lt;br /&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/2936b.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2936b.png&quot; alt=&quot;template switching and second strand synthesis&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you can see the cDNA insert should be between the polyA capture probe and the TSO. In the second sample there are many cases with no insert and the TSO is followed by the poly A sequence. Actually the TSO should not be present in the final read after addition of Illumina adapters. But without any insert the fragmentation does not work and many reads still have this technical sequence.&lt;br /&gt;&#xA;So @merv was correct. Many reads contain no insert.&lt;/p&gt;&#xA;&lt;h2&gt;Why did this happen?&lt;/h2&gt;&#xA;&lt;p&gt;Why were there many reads without insert in this experiment and why are some samples more heavily affected than others? As I mentioned, this were eosinophil granulozytes and these fellas contain many proteases, DNAses and RNAses to destroy invading pathogens. Unfortunately, they also destroy mRNA in single cell experiments.&lt;br /&gt;&#xA;Interestingly, the eosinophily in the second sample received an activating treatment and probably contained even more or more active RNAses.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2021-06-28T13:48:20.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16245" PostTypeId="1" AcceptedAnswerId="16266" CreationDate="2021-06-28T15:21:21.837" Score="1" ViewCount="80" Body="&lt;p&gt;I am investigating the rate of synonymous non synonymous substitutions and would like to find the Ka/Ks value also known as dn and ds between multiple alleles? I want to investigate multiple alleles because I am examining the MHC region which is highly polymorphic. Any suggestions would be very helpful!&lt;/p&gt;&#xA;" OwnerUserId="12135" LastActivityDate="2021-06-30T07:59:45.313" Title="Is there a way to calculate the Ka/Ks value in R/unix?" Tags="|r|snp|bash|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16246" PostTypeId="1" AcceptedAnswerId="16258" CreationDate="2021-06-28T16:43:39.620" Score="0" ViewCount="52" Body="&lt;p&gt;I have a list of mutated genes from a VCF. Does it make sense to perform a pathway enrichment analysis using &lt;a href=&quot;http://www.pantherdb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PANTHER&lt;/a&gt; or &lt;a href=&quot;https://string-db.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;STRING&lt;/a&gt; as examples to know which pathway is affected?&lt;/p&gt;&#xA;&lt;p&gt;EDIT: to reply to @terdon&lt;/p&gt;&#xA;&lt;p&gt;We sequence mostly human genome data using Illumina sequencers. As for the type of data, we perform mostly Whole-Exome Sequencing (WES), Whole-Genome Sequencing (WGS).&lt;/p&gt;&#xA;&lt;p&gt;At the end, after performing all pre-processing, we obtain the VCF files from HaplotypeCaller using GATK. Then we proceed to annotate using VEP.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="4413" LastEditDate="2021-06-29T16:24:03.240" LastActivityDate="2021-06-29T16:24:03.240" Title="Does it make sense to perform pathway enrichment analysis on a list of mutated genes from a VCF?" Tags="|pathway|go-enrichment|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="16247" PostTypeId="1" AcceptedAnswerId="16252" CreationDate="2021-06-28T16:49:54.203" Score="7" ViewCount="1098" Body="&lt;p&gt;This question is to be read in the context of data protection. It seems to be common understanding that the whole genome sequence of an individual is personally identifiable, non-anonymizable information: if you possess the whole genome sequence, you can without any doubt identify an individual. You also cannot anonymize it, because you would need to obstruct the genome sequence in some way, making it useless for any diagnostics or research.&lt;/p&gt;&#xA;&lt;p&gt;But how about Variant Calling files (specifically referring to the &lt;a href=&quot;https://www.internationalgenome.org/wiki/Analysis/vcf4.0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VCF file format&lt;/a&gt;)? You reference against a known genome, but you neglect structural changes such as translocations, duplications and deletions. Can you still identify a person based on a Variant Calling file?&lt;/p&gt;&#xA;" OwnerUserId="12645" LastEditorUserId="73" LastEditDate="2021-06-29T09:20:59.233" LastActivityDate="2021-06-29T17:30:49.313" Title="Are variant calling files personally identifiable information?" Tags="|vcf|variant-calling|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16248" PostTypeId="2" ParentId="13716" CreationDate="2021-06-28T19:36:44.013" Score="0" Body="&lt;p&gt;Looking at this again, I have now worked out (and should probably have noticed previously) that the 'source Gene ID' field from the 'gene_presence_absence.csv' file originates from the ID field of the .gff file for the sequence in question.&lt;/p&gt;&#xA;&lt;p&gt;You can therefore easily tie the results from the 'gene_presence_absence.csv' file back to the annotation from the sequence in question, and therefore to the location of the nucleotide sequence for the gene in question.&lt;/p&gt;&#xA;&lt;p&gt;So for instance, if you take a 'source Gene ID' from the gene_presence_absence file - say &lt;code&gt;GEICCBEJ_00950&lt;/code&gt; - and use it in a grep command referencing the genome from which it came, &lt;code&gt;grep &amp;quot;GEICCBEJ_00950&amp;quot; my_annotation_file.gff&lt;/code&gt;, then you should see the line from the annotation corresponding to this sequence.&lt;/p&gt;&#xA;&lt;p&gt;The 'source Gene ID' is also referenced in the pan_genome_reference.fa output from ROARY, which gives &amp;quot;single representative nucleotide sequence from each of the clusters in the pan genome (core and accessory)&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="6738" LastActivityDate="2021-06-28T19:36:44.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16249" PostTypeId="2" ParentId="16247" CreationDate="2021-06-28T20:23:00.687" Score="1" Body="&lt;p&gt;There should be someone at your workplace you can ask about this, but I would say, yes, since there can be SNPs in there that are private, or even a combination of public SNPs might be identifying, it should be treated as potentially identifying.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2021-06-28T20:23:00.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16250" PostTypeId="2" ParentId="16247" CreationDate="2021-06-28T22:30:44.087" Score="8" Body="&lt;p&gt;Yes, definitely identifiable. The combination of ~80 unlinked common SNPs can be fairly unique in the entire human population, let alone the whole VCF file.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: 30 in the original answer is an underestimate. We need ~80 unlinked common SNPs to uniquely identify every individual to a low false positive rate. Here is the derivation based on the approximate solution to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Birthday_problem#Approximations&quot; rel=&quot;nofollow noreferrer&quot;&gt;birthday paradox&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For simplicity, we assume each person has a haploid genome. Then the genotype at each SNP can only be 0 or 1. Suppose there are &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; people in the world and we have &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; sites. On the condition that &lt;span class=&quot;math-container&quot;&gt;$n^2\ll 2^{m+1}$&lt;/span&gt;, the probability of seeing any two people having the same binary genotypes across all &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; sites is&#xA;&lt;span class=&quot;math-container&quot;&gt;$$p(n,m)\approx 1-e^{-n^2/2^{m+1}}\approx n^2/2^{m+1}$$&lt;/span&gt;&#xA;There are &lt;span class=&quot;math-container&quot;&gt;$n\approx2^{33}$&lt;/span&gt; people. Then &lt;span class=&quot;math-container&quot;&gt;$p(2^{33},72)&amp;lt;1\%$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In reality, we are diploid, so each genotype has three states instead of two. This would lead to a smaller &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; at the same false positive rate. On the other hand, selected SNPs are not totally random. This would lead to a larger &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;. It is safer to choose a larger &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2021-06-29T17:30:49.313" LastActivityDate="2021-06-29T17:30:49.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16251" PostTypeId="2" ParentId="16240" CreationDate="2021-06-28T23:12:37.507" Score="1" Body="&lt;p&gt;I'm not quite sure why assigning a string to a variable would lead to a complaint about file extensions. From the &lt;a href=&quot;https://seqan.readthedocs.io/en/master/Tutorial/InputOutput/SamAndBamIO.html#tutorial-io-sam-bam-io&quot; rel=&quot;nofollow noreferrer&quot;&gt;SeqAn BAM tutorial&lt;/a&gt;, it looks like you need to use the &lt;em&gt;BamFileIn&lt;/em&gt; object for reading BAM/SAM files:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#include &amp;lt;seqan/bam_io.h&amp;gt;&#xA;using namespace seqan;&#xA;&#xA;int main(){&#xA;    CharString bamFileName = &amp;quot;copyPolly_SRR6511930.bam&amp;quot;;&#xA;    // Open input file, BamFileIn can read SAM and BAM files.&#xA;    BamFileIn bamFileIn;&#xA;    if (!open(bamFileIn, toCString(bamFileName))) {&#xA;        std::cerr &amp;lt;&amp;lt; &amp;quot;ERROR: Could not open &amp;quot; &amp;lt;&amp;lt; bamFileName &amp;lt;&amp;lt; std::endl;&#xA;        return 1;&#xA;    }&#xA;    return 0;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Edit: after seeing the updated code... I've got nothing, sorry. It all seems like reasonable code that shouldn't be generating errors.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-06-30T10:32:27.130" LastActivityDate="2021-06-30T10:32:27.130" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16252" PostTypeId="2" ParentId="16247" CreationDate="2021-06-29T07:43:24.023" Score="9" Body="&lt;p&gt;TLDR: yes! be careful with someone's genomic data!&lt;/p&gt;&#xA;&lt;p&gt;There are two aspects to this question: can I find a match for a random VCF in a database of genomes (YES) or can I identify a subject who is not in a (public) database.&lt;/p&gt;&#xA;&lt;p&gt;But even when it may not possible to identify the subject itself, using DTC genetic testing sites may enable you to identify (close or distant) relatives (e.g. read about the &lt;a href=&quot;https://en.wikipedia.org/wiki/Joseph_James_DeAngelo#Arrest_and_trial&quot; rel=&quot;noreferrer&quot;&gt;Golden State Killer&lt;/a&gt;). Identifying relatives brings you close to identifying the individual, and also those relatives may not be pleased with learning that &lt;a href=&quot;https://en.wikipedia.org/wiki/Henrietta_Lacks#Consent_issues_and_privacy_concerns&quot; rel=&quot;noreferrer&quot;&gt;the variants of someone with who they share (a small or larger part) of their genome are accessible by anyone&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;We are further ahead for some traits than others, but for more and more human characteristics we will be able to predict e.g. eye color, height based on genome data, or at least estimate to some degree. Genetic ancestry and biological sex are trivial. Often genomic data is from a patient with a (rare) disorder, further narrowing down the options.&lt;/p&gt;&#xA;&lt;p&gt;In addition, there is a difference between &amp;quot;can someone be identified today&amp;quot; (which is definitely possible but maybe tough) and &amp;quot;given that the variants are out there (online) forever can someone be identified within 10 years&amp;quot; and the answer to the latter part is a loud yes.&lt;/p&gt;&#xA;" OwnerUserId="681" LastEditorUserId="681" LastEditDate="2021-06-29T11:45:56.667" LastActivityDate="2021-06-29T11:45:56.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16253" PostTypeId="2" ParentId="16039" CreationDate="2021-06-29T07:56:14.893" Score="0" Body="&lt;p&gt;found this in the manual, so I guess the first two columns are the reference and query positions and the third is the length. I am not 100% sure either, so let me know if you find out more.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://mummer.sourceforge.net/manual/#mummerplot&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mummer.sourceforge.net/manual/#mummerplot&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Just a simple three column list, noting the position and length of every maximal exact match. Note that for reverse complement matches (produced with the -r option), the query start positions will reference the reverse complement of the query input sequence.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="12651" LastActivityDate="2021-06-29T07:56:14.893" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16254" PostTypeId="1" CreationDate="2021-06-29T08:43:32.240" Score="0" ViewCount="122" Body="&lt;p&gt;I'd like to a follow up question to &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/414/12652&quot;&gt;this question&lt;/a&gt; related to merging fastq files for ChIP-seq.&lt;/p&gt;&#xA;&lt;p&gt;Let's assume we have one sample library that is re-sequenced two or three times in order to achieve a desired read coverage. The resulting output of the sequencing is 2 or 3 fastq files for one individual sample.&lt;/p&gt;&#xA;&lt;p&gt;If one has to mark duplicates (for example using &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360037052812-MarkDuplicates-Picard-&quot; rel=&quot;nofollow noreferrer&quot;&gt;Picard's MarkDuplicates&lt;/a&gt;) should the sub-samples be merged at the fastq level or at the bam file level (post alignment) after flagging duplicates before the merge?&lt;/p&gt;&#xA;&lt;p&gt;If one takes the 2-3 sub-fastq files align them and then mark the duplicates in the bam files if the sequencing is shallow one could end up with not calling reads that are actually PCR duplicates of the library preparation. While in the other scenario, a fastq merge might result in reads to be marked as &amp;quot;duplicates&amp;quot; just because they appear 2-3 times, but maybe that read is a single reads that I sequenced 2-3 times, individually in each run.&lt;/p&gt;&#xA;&lt;p&gt;My point is that I would expect this 2 different approached to lead to different outcomes. Probably this is a negligible effect in a overall analysis, but maybe it might actually have an impact in certain biological conditions (I'm thinking of heterochromatin silencing of repeat elements in the genome). I'm going to try it out as soon as I have a change, but I was wondering if this was discussed already somewhere else (I couldn't find much relevant) or if someone has some ideas on how to tackle this.&lt;/p&gt;&#xA;&lt;p&gt;Let's assume that the reads are 50bp single end.&lt;/p&gt;&#xA;" OwnerUserId="12652" LastActivityDate="2021-06-29T08:50:42.197" Title="Impact of merging ChIP-seq runs of the same sample on PCR duplicates identification?" Tags="|sam|fastq|chip-seq|merge|markduplicate|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16255" PostTypeId="2" ParentId="16254" CreationDate="2021-06-29T08:50:42.197" Score="1" Body="&lt;p&gt;If you are sequencing same library multiple times (as you state in the question), any read that appears in  multiple sequencing runs is likely to have been generated via PCR duplication from the same original biological molecule, and therefore needs to be removed. Indeed, the chance of the read being a duplicate is the same if it was sequenced twice in the same lane, or it was sequenced in two different lanes, as long as it comes from the same library.&lt;/p&gt;&#xA;&lt;p&gt;Thus, in this situation, you should either:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Merge the fastqs, map and MarkDuplicates or&lt;/li&gt;&#xA;&lt;li&gt;Map the separate fastqs, merge the BAM files, and then MarkDuplicates.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;You should definitely merge before doing either MarkDuplicates or peak calling.&lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2021-06-29T08:50:42.197" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16256" PostTypeId="2" ParentId="16247" CreationDate="2021-06-29T11:06:19.487" Score="3" Body="&lt;p&gt;Not only are they theoretically identifiable based on the combination of SNPs as others have said (you are more or less identifiable from ~20 well chosen microsats) - but it's possible to be actually identified from your data using a long-range familial search, based on genetics and a bit of sleuthing.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://science.sciencemag.org/content/362/6415/690.abstract&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://science.sciencemag.org/content/362/6415/690.abstract&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2021-06-29T11:06:19.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16257" PostTypeId="1" AcceptedAnswerId="16267" CreationDate="2021-06-29T12:28:25.573" Score="-1" ViewCount="289" Body="&lt;p&gt;I have a big single-cell RNA seq data&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dput(head(new.dat[,1:10]))&#xA;structure(list(cell1 = c(0.793763840992639, 0, 1.96843530982957, &#xA;0.461736429639991, 0.717968540649498, 0), cell2 = c(3.61741696702738, &#xA;0.231662370550224, 0, 0, 0, 0), cell3 = c(4.14348883366621, 0.118161316317251, &#xA;0.08074552209482, 2.27968429766934, 0.0470313356296409, 0), cell4 = c(1.34783143327084, &#xA;0.0094666040612932, 1.14392942941128, 0.652535826921119, 0.357542816432864, &#xA;0.149587369334621), cell5 = c(1.27104023273899, 1.55185229643731, &#xA;0, 0, 0, 0.0117525723115277), cell6 = c(1.92307653575663, 0, &#xA;0, 0.319156642478379, 0, 0), cell7 = c(3.9343015424917, 0.132824589520901, &#xA;0.119679885703561, 0.772516422897241, 0.0236884909844904, 0), &#xA;    cell8 = c(3.74969491678643, 0.103404975609384, 0.0354753982873036, &#xA;    0, 0, 0), cell9 = c(1.19084857532713, 3.9213265721495, 0, &#xA;    0.0341973245272891, 0.0419122921627454, 0), cell10 = c(4.1224255501566, &#xA;    0.301871669274068, 0.0633536200981225, 0.389959552469879, &#xA;    0, 0.0405296102106492)), row.names = c(&amp;quot;PTPRC&amp;quot;, &amp;quot;MHC-II&amp;quot;, &#xA;&amp;quot;ITGAM&amp;quot;, &amp;quot;Ly6C&amp;quot;, &amp;quot;Ly6G&amp;quot;, &amp;quot;EMR1&amp;quot;), class = &amp;quot;data.frame&amp;quot;)&#xA;&amp;gt; &#xA;&#xA;&amp;gt; dim(new.dat)&#xA;[1]     33 263086&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How I &lt;code&gt;remove every columns&lt;/code&gt; which are &lt;code&gt;zero&lt;/code&gt; for one gene, let's say &lt;code&gt;PTPRC&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="9448" LastEditorUserId="57" LastEditDate="2021-07-01T08:08:34.863" LastActivityDate="2021-07-01T08:08:34.863" Title="Removing cells zero for a gene from a scRNA-seq data" Tags="|r|scrnaseq|10x-genomics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16258" PostTypeId="2" ParentId="16246" CreationDate="2021-06-29T14:12:27.483" Score="1" Body="&lt;p&gt;The concept of looking for enriched pathways makes sense, but you can't do it using the same tools you would use for gene expression data because the biases are different - longer genes are far more likely to contain variants by chance than short ones, and this needs to be accounted for in any enrichment analysis.&lt;/p&gt;&#xA;" OwnerUserId="235" LastEditorUserId="298" LastEditDate="2021-06-29T16:11:46.997" LastActivityDate="2021-06-29T16:11:46.997" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16259" PostTypeId="2" ParentId="16257" CreationDate="2021-06-29T14:29:37.633" Score="0" Body="&lt;p&gt;This is basic R subsetting operation. Most subset operations give you a subset of rows based on a column, while you're doing the opposite here - but the logic is the same.&lt;/p&gt;&#xA;&lt;p&gt;This gives you a subset of rows when the expression is based on a column&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df[logical_expression_that_subsets_rows,]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Why not try this - it should give you a subset of columns based on a row value. (Untested, but you should be able to build on this):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df[, logical_expression_that_subsets_columns]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The logical expression for the first could be &lt;code&gt;df$cell1 &amp;lt; 0.5&lt;/code&gt; (or the equivalent &lt;code&gt;df[,&amp;quot;cell1&amp;quot;] &amp;lt; 0.5&lt;/code&gt;. You should be able to get a logical expression for the second based on this.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2021-06-29T14:29:37.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16260" PostTypeId="2" ParentId="16039" CreationDate="2021-06-29T17:04:07.147" Score="2" Body="&lt;h2&gt;Maximal Unique Matches&lt;/h2&gt;&#xA;&lt;p&gt;To answer the question about MUMs, there are two important definitions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A &amp;quot;match&amp;quot; is &lt;em&gt;maximal&lt;/em&gt; if it cannot be extended in either direction and still be a match i.e, for two strings &lt;span class=&quot;math-container&quot;&gt;$s,r$&lt;/span&gt;, a match&#xA;&lt;span class=&quot;math-container&quot;&gt;$s_i,...,s_{i+k}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$r_{j},...,r_{j+k}$&lt;/span&gt; is maximal if &lt;span class=&quot;math-container&quot;&gt;$s_i,...,s_{i+k+1}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$r_{j},...,r_{j+k+1}$&lt;/span&gt; is not a match and &lt;span class=&quot;math-container&quot;&gt;$s_{i-1},...,s_{i+k}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$r_{j-1},...,r_{j+k}$&lt;/span&gt; is also not a match.&lt;/li&gt;&#xA;&lt;li&gt;A &amp;quot;match&amp;quot; is &lt;em&gt;unique&lt;/em&gt; if the matched substring is not present anywhere else in either genome. i.e, for two strings &lt;span class=&quot;math-container&quot;&gt;$s,r$&lt;/span&gt;, a match&#xA;&lt;span class=&quot;math-container&quot;&gt;$s_i,...,s_{i+k}$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$r_{j},...,r_{j+k}$&lt;/span&gt; is unique if the substring &lt;span class=&quot;math-container&quot;&gt;$s_i,...,s_{i+k}$&lt;/span&gt; is present only once in each genome.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;With those definitions in hand, we can easily define a Maximal Unique Match (MUM) as an exact match that is both maximal and unique. Note that MUMmer also provides support for Maximal Exact Matches (MEMs), which do not require uniqueness of matches.&lt;/p&gt;&#xA;&lt;h2&gt;MUMmer output&lt;/h2&gt;&#xA;&lt;p&gt;As far as the output goes:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://mummer.sourceforge.net/manual/&quot; rel=&quot;nofollow noreferrer&quot;&gt;For each match, the three columns list the position in the reference sequence, the position in the query sequence, and the length of the match respectively. Reverse complemented query positions are reported relative to the reverse of the query sequence unless the &lt;code&gt;-c&lt;/code&gt; option was used&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;For example, the row:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1      1120       299&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;denotes a MUM of length 299 starting at the first base in the first input file and the 1120th base in the second input file. For matches on the reverse complement strand, you'll notice a second part of the output that contains the matches on the reverse complemented strand.&lt;/p&gt;&#xA;&lt;p&gt;Here is a sample output I have on my device:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;$ $  mummer -mum -b -c England1.fna Indiana-USA-1_Saudi_Arabia_2014.fna&#xA;&amp;gt; gi|633896549|gb|KJ813439.1|&#xA;       1      1120       299&#xA;     301      1420       174&#xA;     476      1595        65&#xA;     542      1661        81&#xA;     624      1743        43&#xA;     668      1787       264&#xA;     933      2052      1042&#xA;    1976      3095      1299&#xA;    3276      4395       330&#xA;    3607      4726        69&#xA;    3677      4796       885&#xA;    4563      5682       812&#xA;    5376      6495       792&#xA;    6169      7288       831&#xA;    7001      8120       731&#xA;    7733      8852      1963&#xA;    9697     10816      1176&#xA;   10874     11993       617&#xA;   11492     12611        41&#xA;   11534     12653       164&#xA;   11708     12827      3007&#xA;   14716     15835      1040&#xA;   15757     16876       489&#xA;   16247     17366       412&#xA;   16660     17779       337&#xA;   16998     18117       463&#xA;   17462     18581       325&#xA;   17788     18907       686&#xA;   18475     19594       599&#xA;   19075     20194       342&#xA;   19418     20537       179&#xA;   19598     20717       532&#xA;   20131     21250       152&#xA;   20284     21403       935&#xA;   21220     22339       348&#xA;   21569     22688       764&#xA;   22334     23453       351&#xA;   22686     23805        73&#xA;   22760     23879        29&#xA;   22790     23909       180&#xA;   22971     24090       532&#xA;   23504     24623       143&#xA;   23648     24767       107&#xA;   23756     24875       434&#xA;   24191     25310        59&#xA;   24251     25370        47&#xA;   24299     25418       214&#xA;   24515     25634       224&#xA;   24740     25859      1386&#xA;   26127     27246        39&#xA;   26167     27286        63&#xA;   26231     27350       114&#xA;   26346     27465       102&#xA;   26449     27568      1399&#xA;   27849     28968       207&#xA;   28057     29176       938&#xA;   28996     30115       726&#xA;   29743     30868       369&#xA;&amp;gt; gi|633896549|gb|KJ813439.1| Reverse&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, if you compute the reverse complement of the input and rerun, you'll notice the matches appear in reverse in the second section denoted by &lt;code&gt;&amp;gt; gi|633896549|gb|KJ813439.1| Reverse&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="6628" LastEditorUserId="6628" LastEditDate="2021-06-29T17:11:17.577" LastActivityDate="2021-06-29T17:11:17.577" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="16261" PostTypeId="2" ParentId="16240" CreationDate="2021-06-29T18:24:05.813" Score="3" Body="&lt;p&gt;The helpdesk for my institution's supercomputers showed me the following flags:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;g++ seqan_test.cpp -DSEQAN_HAS_ZLIB -lz -lpthread&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This fixed the error. I'm now running into different errors and am suspicious that I am getting them because I am using SeqAn2 rather than SeqAn3. I'm going to try HTSlib.&lt;/p&gt;&#xA;" OwnerUserId="9118" LastActivityDate="2021-06-29T18:24:05.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16262" PostTypeId="1" AcceptedAnswerId="16263" CreationDate="2021-06-29T19:07:15.357" Score="2" ViewCount="175" Body="&lt;p&gt;I am using the &amp;quot;htslib/sam.h&amp;quot; header in C++. I need to access the SEQ field of bam reads and store each sequence in a vector of strings. I also need to access the MAPQ field of each read to check if the read passes my threshold quality of 10 (which I have stored as an integer). Does anyone know which htslib/sam.h functions can accomplish these tasks?&lt;/p&gt;&#xA;" OwnerUserId="9118" LastActivityDate="2021-06-29T19:58:18.587" Title="How do I access sequence and MAPQ from bam using HTSlib in C++?" Tags="|sam|c++|htslib|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16263" PostTypeId="2" ParentId="16262" CreationDate="2021-06-29T19:58:18.587" Score="3" Body="&lt;p&gt;The MAPQ is stored in &lt;code&gt;b-&amp;gt;core.qual&lt;/code&gt;, given a &lt;code&gt;bam1_t *b&lt;/code&gt;. You can access the sequence with &lt;code&gt;bam_seqi(bam_get_seq(b), some_offset)&lt;/code&gt;(see &lt;a href=&quot;https://github.com/samtools/htslib/blob/develop/htslib/sam.h#L317&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/samtools/htslib/blob/develop/htslib/sam.h#L317&lt;/a&gt;). This will get you a base at a time, so you'd need to iterate over it (the length of the sequence being in &lt;code&gt;b-&amp;gt;core.l_qseq&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-06-29T19:58:18.587" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16264" PostTypeId="2" ParentId="16245" CreationDate="2021-06-29T21:09:11.233" Score="1" Body="&lt;p&gt;A while back, I used &lt;a href=&quot;http://abacus.gene.ucl.ac.uk/software/paml.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;PAML&lt;/code&gt;&lt;/a&gt; for this purpose for a phylogenetics project. It has Windows and macOS executables but for *nix, you will need to compile it from source code.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2021-06-30T07:59:45.313" LastActivityDate="2021-06-30T07:59:45.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16265" PostTypeId="2" ParentId="16188" CreationDate="2021-06-29T23:05:48.833" Score="4" Body="&lt;p&gt;In the linked article the authors formalize microarray analysis as the study of the joint distributions of &lt;span class=&quot;math-container&quot;&gt;$\overrightarrow{X}_i$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$Y_i$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$\overrightarrow{X}_i$&lt;/span&gt; is a vector of random variables, the distribution of each of which (i.e. each of &lt;span class=&quot;math-container&quot;&gt;$X_{ji}$&lt;/span&gt;) is determined by the level of expression of gene j in sample i, and &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; is some response or covariate of interest.&lt;/p&gt;&#xA;&lt;p&gt;The response/covariate is the easiest to interpret in terms of biology. It is the condition. So it might be treated or untreated if we were interested in genes that respond to drug treatment, or it might be diseased vs healthy - this would be covariates. Or it might be &amp;quot;time to death&amp;quot;, &amp;quot;body mass index&amp;quot;, etc. - these would be responses. The difference between a co-variate and a response is which is the dependent and which is the independent variable. In differential expression we have &amp;quot;gene expression ~  disease&amp;quot; (i.e. X ~ covariate), where &lt;code&gt;~&lt;/code&gt; is pronounced &amp;quot;is predicted by&amp;quot;. In the case of survival analysis &amp;quot;time to death ~ gene expression&amp;quot; (i.e. response ~ X).The malleability of the relationship between X and Y is what is meant by&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;The population and sampling mechanism will depend on&#xA;the particular application&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;A random variable is any variable that we cannot know the value of until it has been measured. Note, this is &lt;em&gt;not&lt;/em&gt; &amp;quot;do not&amp;quot; know the value of, its &amp;quot;can not&amp;quot; know the value of.  So for example, in this case of RNA-seq, X is not the gene expression, it is the number of reads. An all-knowing daemon could know the gene expression for a gene in a sample before we measured it, but they couldn't know how many reads we will get because of the randomness introduced by sampling.&lt;/p&gt;&#xA;&lt;p&gt;Random variables don't &lt;em&gt;represent&lt;/em&gt; distributions, they &lt;em&gt;have&lt;/em&gt; distributions. That is, if you were to realize many trials of a random variable, the values you obtained would form a distribution.&lt;/p&gt;&#xA;&lt;p&gt;So the number of reads you get mapping to a particular gene in a sequencing experiment is a random variable. As is the brightness of a spot on a microarray. Its an interesting question to ask if &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; is a random variable. The answer is &amp;quot;it depends&amp;quot; - and what it depends on is how good we are at controlling an expriment. You might argue that &amp;quot;drug treatment&amp;quot; is not a random variable because we treat the cells or not - we know, but that, say &amp;quot;time to death&amp;quot; or even &amp;quot;age&amp;quot; is a random variable, because we don't determine it ahead of the experiment. Either way, this subtlety is covered by the above quoted sentence.&lt;/p&gt;&#xA;&lt;p&gt;So, in a traditional bulk sequencing experiment we have a matrix of random variables associated with gene expression &lt;span class=&quot;math-container&quot;&gt;$X_{ji}$&lt;/span&gt; - the number of reads for gene j in sample i, and &lt;span class=&quot;math-container&quot;&gt;$Y_i$&lt;/span&gt; the condition for sample i. Note that &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; is indexed by the sample, not the gene - every gene in the sample uses the same value of &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt;. And for each gene j we test the model &lt;code&gt;X_i ~ Y_i&lt;/code&gt; - the expression in sample i is predicted by the condition of sample i.&lt;/p&gt;&#xA;&lt;p&gt;In a single cell sequencing our matrix of random variables associated with gene expression is actaully a tensor, that is, it has three indexes, not two: &lt;span class=&quot;math-container&quot;&gt;$X_{jki}$&lt;/span&gt; - that is the number of reads from gene j in cell k in sample i. In a scarily large number of single cell sequencing experiments, i only has one value (i.e. there is only one sample). And in most single cell experiments I've seen &lt;span class=&quot;math-container&quot;&gt;$Y_i$&lt;/span&gt; only takes one value, even if there are multiple samples. Even if &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; takes multiple values, its not clear what sort of model you'd fit to the structure &lt;code&gt;X_{ki} ~ Y_i&lt;/code&gt; - i.e predicting a matrix of many cells by a single vector of covariate values.&lt;/p&gt;&#xA;&lt;p&gt;I think one analysis you can do with Seurat is to treat each cell as a separate sample, and each cluster as a condition, and thus try to find gene DE between cell types. I'm not convinced how great this sort of analysis, but here, the i in &lt;span class=&quot;math-container&quot;&gt;$X_{ji}$&lt;/span&gt; would represent which cell a measurement was from and &lt;span class=&quot;math-container&quot;&gt;$Y_{i}$&lt;/span&gt; would represent which cluster. But this sort of analysis seems prone to all sorts of date-peeking and training set leakage problems.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;if we suppose that Xg and Xr denote the random variables that associate to the expression level of the gene g and of the gene r, respectively. Let Y and Yv represent the responses for Xg and Xr, respectively.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;First, &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; is a vector of which &lt;span class=&quot;math-container&quot;&gt;$Y_v$&lt;/span&gt; is a component, so &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; contains &lt;span class=&quot;math-container&quot;&gt;$Y_v$&lt;/span&gt;. And in the formulation presented in the linked paper (and in almost all RNAseq analysis literature), the response is the same for two different genes, so there is only one &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; across both genes g and r.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;the random variables Xg and Xr represent the distributions of the gene expression measure per cell and the gene expression measure per spot, respectively&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I'm not quite sure what you mean by &amp;quot;per spot&amp;quot;. Spots are a microarray term, and don't really have any counterpart in RNA-seq analysis. They were, literally, the spots of DNA oligos printed onto the glass slides used in microarray analysis.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x_{gi}$&lt;/span&gt; is the number of reads mapped to gene g in cell i, and has a distribution determined by the level of expression of gene g in the cell i.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x_{ri}$&lt;/span&gt; is the number of reads mapped to gene r in cell i, and has a distribution determined by the level of expression of gene r in cell i.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;and the responses Y and Yv represent the range of the random variable Xg and Xr, respectively&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No. The response &lt;span class=&quot;math-container&quot;&gt;$Y$&lt;/span&gt; represents some biological dicotomy you would like to associate with gene expression.  You could use cell type, but this is neither response, nor covariate, it is not measured independently of X, but is simply a phantom of the correlation structure in &lt;span class=&quot;math-container&quot;&gt;$X_{ji}$&lt;/span&gt;, so I can see why the &amp;quot;response&amp;quot; type language would be confusing.&lt;/p&gt;&#xA;" OwnerUserId="235" LastActivityDate="2021-06-29T23:05:48.833" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16266" PostTypeId="2" ParentId="16245" CreationDate="2021-06-30T00:20:16.193" Score="2" Body="&lt;p&gt;I have also used the &lt;a href=&quot;https://cran.r-project.org/web/packages/seqinr/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqinr&lt;/a&gt; package for &lt;a href=&quot;https://www.rdocumentation.org/packages/seqinr/versions/4.2-8/topics/kaks&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ka/Ks&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12660" LastEditorUserId="3541" LastEditDate="2021-06-30T07:48:20.327" LastActivityDate="2021-06-30T07:48:20.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16267" PostTypeId="2" ParentId="16257" CreationDate="2021-06-30T13:19:02.697" Score="-3" Body="&lt;pre&gt;&lt;code&gt;WhichCells(seurat_object made by this matrix, slot = 'counts', expression = PTPRC &amp;gt; 0 )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;By Seurat R package this is possible&lt;/p&gt;&#xA;" OwnerUserId="9448" LastActivityDate="2021-06-30T13:19:02.697" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16268" PostTypeId="2" ParentId="933" CreationDate="2021-06-30T17:00:18.183" Score="-1" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/KalinNonchev/gnomAD_MAF&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/KalinNonchev/gnomAD_MAF&lt;/a&gt; would solve your problem I think.&#xA;It is in python and you can annotate fast variants with their allele frequencies.&lt;/p&gt;&#xA;" OwnerUserId="4805" LastActivityDate="2021-06-30T17:00:18.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16269" PostTypeId="1" CreationDate="2021-06-30T17:57:02.960" Score="2" ViewCount="729" Body="&lt;p&gt;I am learning the Seurat algorithms to cluster the scRNA-seq datasets. I found this explanation, but am confused. Can someone explain it to me, &amp;quot;The FindClusters function implements the procedure, and contains a resolution parameter that sets the ‘granularity’ of the downstream clustering, with increased values leading to a greater number of clusters. We find that setting this parameter between 0.6-1.2 typically returns good results for single cell datasets of around 3K cells&amp;quot;. My questions are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What is &amp;quot;granularity&amp;quot; of the downstream analysis?&lt;/li&gt;&#xA;&lt;li&gt;How does the parameter &amp;quot;resolution&amp;quot; work?&lt;/li&gt;&#xA;&lt;li&gt;How to know the optimum value for parameter &amp;quot;resolution&amp;quot;?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I found this paper about the resolution that might help, but not clear to me yet. &lt;a href=&quot;https://iopscience.iop.org/article/10.1088/1742-5468/2008/10/P10008/pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://iopscience.iop.org/article/10.1088/1742-5468/2008/10/P10008/pdf&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I think the higher the resolution, the more clusters produced meaning the smaller clusters obtained? The smaller the clusters the more specific cell types we get, is that correct?&lt;/p&gt;&#xA;" OwnerUserId="12477" LastEditorUserId="104" LastEditDate="2021-12-21T21:53:39.867" LastActivityDate="2021-12-21T21:53:39.867" Title="Seurat clustering Methods-resolution parameter explanation" Tags="|scrnaseq|seurat|single-cell|clustering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16270" PostTypeId="1" AcceptedAnswerId="16280" CreationDate="2021-06-30T18:03:06.470" Score="1" ViewCount="391" Body="&lt;p&gt;I am using htslib/sam.h to write a C++ program. As part of this program, I must extract reads occurring on specific scaffolds in specific regions from a bam file. Essentially, I want to perform the following samtools command in C++:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view name.bam -o name.sam scaffold:pos-pos&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Since I must extract reads from thousands of regions, I do not want to iterate through the whole bam file each time I extract reads from a region. I am wondering if HTSlib can extract reads occurring in a specified scaffold and range without iterating through the whole bam. I found the function bam_itr_querys() in the htslib/sam.h documentation on GitHub, but don't know how to format the inputs for this function, or what the &amp;quot;idx&amp;quot; input is. I also saw somewhere that I would need to pair this function with bam_itr_next(), but don't know the format to do so.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: My bam files are sorted and indexed, if that matters.&lt;/p&gt;&#xA;" OwnerUserId="9118" LastEditorUserId="9118" LastEditDate="2021-07-03T19:21:57.580" LastActivityDate="2021-07-03T20:21:55.200" Title="Can HTSlib extract bam reads occurring in a specific region without iterating through the whole file?" Tags="|sam|samtools|c++|htslib|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16271" PostTypeId="1" AcceptedAnswerId="16277" CreationDate="2021-06-30T18:40:04.010" Score="0" ViewCount="182" Body="&lt;p&gt;I implemented the following &lt;strong&gt;find_neighbors_with_expected_hamming_distance&lt;/strong&gt; function that generates all k-mers of Hamming distance at most d from the given Pattern.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from .Hamming_Distance import find_hamming_distance&#xA;&#xA;dp_map_that_stores_all_neighbors_with_expected_hamming_distance = {}&#xA;&#xA;unique_nucleotides = {'A', 'C', 'G', 'T'}&#xA;&#xA;&#xA;def suffix(pattern):&#xA;    if len(pattern) &amp;lt;= 1:&#xA;        return None&#xA;    else:&#xA;        return pattern[1: (len(pattern))]&#xA;&#xA;&#xA;def first_symbol(pattern):&#xA;    if len(pattern) == 0:&#xA;        return None&#xA;    else:&#xA;        return pattern[0]&#xA;&#xA;&#xA;def find_neighbors_with_expected_hamming_distance(pattern, expected_hamming_distance):&#xA;    if expected_hamming_distance == 0:&#xA;        return {pattern}&#xA;&#xA;    if len(pattern) == 1:&#xA;        return {'A', 'C', 'G', 'T'}&#xA;&#xA;    if (pattern, expected_hamming_distance) in dp_map_that_stores_all_neighbors_with_expected_hamming_distance.keys():&#xA;        return dp_map_that_stores_all_neighbors_with_expected_hamming_distance[(pattern, expected_hamming_distance)]&#xA;&#xA;    neighborhood_set = set()&#xA;&#xA;    suffix_neighbors_set = find_neighbors_with_expected_hamming_distance(suffix(pattern), expected_hamming_distance)&#xA;&#xA;    for suffix_neighbor in suffix_neighbors_set:&#xA;&#xA;        if find_hamming_distance(suffix(pattern), suffix_neighbor) &amp;lt; expected_hamming_distance:&#xA;&#xA;            for nucleotide in unique_nucleotides:&#xA;                neighborhood_set.add(nucleotide + suffix_neighbor)&#xA;&#xA;        else:&#xA;            neighborhood_set.add(first_symbol(pattern) + suffix_neighbor)&#xA;&#xA;    if (pattern, expected_hamming_distance) not in dp_map_that_stores_all_neighbors_with_expected_hamming_distance.keys():&#xA;        dp_map_that_stores_all_neighbors_with_expected_hamming_distance[&#xA;            (pattern, expected_hamming_distance)] = neighborhood_set&#xA;&#xA;    return neighborhood_set&#xA;    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;I want your suggestion on my time complexity calculation:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;For a given pattern of length &lt;strong&gt;N&lt;/strong&gt; and &lt;strong&gt;D&lt;/strong&gt; as maximum expected hamming distance, I calculated time complexity as -&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\sum_{n=1}^N$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$\sum_{d=1}^D$&lt;/span&gt; &lt;span class=&quot;math-container&quot;&gt;$n\choose d$&lt;/span&gt; * &lt;span class=&quot;math-container&quot;&gt;$3^i$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Reasoning:&lt;/strong&gt; For any length &lt;strong&gt;n&lt;/strong&gt;, we will choose any &lt;strong&gt;d&lt;/strong&gt; letters (nucleotide in this case) to be replaced with the other 3 nucleotides that is different from the nucleotide present at that particular position.&lt;/p&gt;&#xA;&lt;p&gt;For example: Pattern = ATGCAT and &lt;strong&gt;D&lt;/strong&gt; = 2, then suppose for one of the case, I select 2nd and 4th positions to be replaced.&lt;/p&gt;&#xA;&lt;p&gt;Then, at the second position (current nucleotide = 'T'), I can substitute any nucleotide from the set {'A', 'C', 'G'} and&lt;/p&gt;&#xA;&lt;p&gt;for fourth position (current nucleotide = 'C'), I can substitute any nucleotide from the set {'A', 'T', 'G'}.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Kindly tell me if this approach is correct?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have provided the pseudocode for the &lt;strong&gt;Neighbors&lt;/strong&gt; function as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Neighbors(Pattern, d)&#xA;    if d = 0&#xA;        return {Pattern}&#xA;    if |Pattern| = 1 &#xA;        return {A, C, G, T}&#xA;    Neighborhood ← an empty set&#xA;    SuffixNeighbors ← Neighbors(Suffix(Pattern), d)&#xA;    for each string Text from SuffixNeighbors&#xA;        if HammingDistance(Suffix(Pattern), Text) &amp;lt; d&#xA;            for each nucleotide x&#xA;                add x • Text to Neighborhood&#xA;        else&#xA;            add FirstSymbol(Pattern) • Text to Neighborhood&#xA;    return Neighborhood&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The symbol • means concatenation of 2 strings. For example, 'S' • 'V' = 'SV'.&lt;/p&gt;&#xA;&lt;p&gt;If we remove the first symbol of Pattern (denoted FirstSymbol(Pattern)), then we will obtain a&lt;/p&gt;&#xA;&lt;p&gt;(k − 1)-mer that we denote by Suffix(Pattern).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Illustration:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Given pattern = 'ACG'&lt;/p&gt;&#xA;&lt;p&gt;Expected Hamming distance (D) = 1&lt;/p&gt;&#xA;&lt;p&gt;Answer = {'CCG', 'TCG', 'GCG', 'AAG', 'ATG', 'AGG', 'ACA', 'ACC', 'ACT', 'ACG'}&lt;/p&gt;&#xA;&lt;p&gt;Note that the Answer is a set of all strings who have hamming distance of 1 with respect to the given pattern 'ACG'.&lt;/p&gt;&#xA;" OwnerUserId="12425" LastEditorUserId="12425" LastEditDate="2021-07-01T12:52:06.307" LastActivityDate="2021-07-01T17:55:24.620" Title="Time complexity of the Neighbors function" Tags="|python|k-mer|algorithms|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16272" PostTypeId="2" ParentId="16269" CreationDate="2021-06-30T18:54:47.543" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What is &amp;quot;granularity&amp;quot; of the downstream analysis?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Low granularity implies basic cell types (e.g. T cells, B cells), implies few larger clusters.&lt;/p&gt;&#xA;&lt;p&gt;High granularity implies more refined cell types (e.g. CD4+ Central Memory T, Treg, Memory B cell etc.), implies more smaller clusters&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How does the parameter &amp;quot;resolution&amp;quot; work?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;A high resolution parameter value produces more smaller clusters, i.e. higher granularity. If you want to know how modularity clustering works, I recommend the Leiden algorithm paper &lt;a href=&quot;https://www.nature.com/articles/s41598-019-41695-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41598-019-41695-z&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How to know the optimum value for parameter &amp;quot;resolution&amp;quot;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Tricky, and it depends on what questions you want to answer. Normally it's a subjective iterative process of trying a clustering resolution, checking if the clusters make sense, re-trying etc. but I also quite like the clustree approach &lt;a href=&quot;https://cran.r-project.org/web/packages/clustree/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cran.r-project.org/web/packages/clustree/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The smaller the clusters the more specific cell types we get, is that correct?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes correct&lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="73" LastEditDate="2021-07-02T12:59:24.723" LastActivityDate="2021-07-02T12:59:24.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16273" PostTypeId="1" CreationDate="2021-06-30T20:19:55.107" Score="1" ViewCount="41" Body="&lt;p&gt;I am trying to use an external file with a list of gene names to query the ENSEMBL IDs using the perl API. This script works:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/local/bin/perl&#xA;&#xA;use strict;&#xA;use warnings;&#xA;use Bio::Ensembl::Registry;&#xA;&#xA;my $registry = 'Bio::EnsEMBL::Registry';&#xA;&#xA;$registry-&amp;gt;load_registry_from_db(&#xA;    -host =&amp;gt; 'useastdb.ensembl.org',&#xA;    -port =&amp;gt; 5306,&#xA;    -user =&amp;gt; 'anonymous'&#xA;);&#xA;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$gene_adaptor = $&lt;/span&gt;registry-&amp;gt;get_adaptor( 'Human', 'Core', 'Gene' );&#xA;&#xA;my @genes;&#xA;@genes = ('CCL2', 'CCL5');&#xA;&#xA;foreach &lt;span class=&quot;math-container&quot;&gt;$name(@genes){&#xA;    my $gene = $gene_adaptor -&amp;gt; fetch_by_display_label($name);&#xA;    print &quot;GENE &quot;, $&lt;/span&gt;gene-&amp;gt;stable_id(), &amp;quot;,&amp;quot;;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However I can't figure out how to get this to work with an external file that contains a list of genes like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;genes.txt&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CCL2&#xA;CCL5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here's what I tried:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/local/bin/perl&#xA;&#xA;use strict;&#xA;use warnings;&#xA;use Bio::Ensembl::Registry;&#xA;&#xA;my $registry = 'Bio::EnsEMBL::Registry';&#xA;&#xA;$registry-&amp;gt;load_registry_from_db(&#xA;    -host =&amp;gt; 'useastdb.ensembl.org',&#xA;    -port =&amp;gt; 5306,&#xA;    -user =&amp;gt; 'anonymous'&#xA;    );&#xA;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$gene_adaptor = $&lt;/span&gt;registry-&amp;gt;get_adaptor( 'Human', 'Core', 'Gene' );&#xA;&#xA;open (IDs, '&amp;lt;', &amp;quot;genes.txt&amp;quot;) or die &amp;quot;can't read input file&amp;quot;;&#xA;while (&amp;lt;IDs&amp;gt;){&#xA;    my &lt;span class=&quot;math-container&quot;&gt;$gene = $&lt;/span&gt;gene_adaptor -&amp;gt; fetch_by_display_label(&lt;span class=&quot;math-container&quot;&gt;$_);&#xA;    print &quot;GENE &quot;, $&lt;/span&gt;gene-&amp;gt;stable_id(), &amp;quot;,&amp;quot;;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I'm getting this error:&#xA;&lt;code&gt;Can't call method &amp;quot;stable_id&amp;quot; on an undefined value at ./get_ensembl_ids.pl line 22, &amp;lt;IDs&amp;gt; line 1.&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I was trying to follow this &lt;a href=&quot;https://www.perltutorial.org/perl-read-file/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; for reading txt files in perl, but what am I missing for getting perl to actually fetch the ENSEMBL ids with each of the .txt file lines?&lt;/p&gt;&#xA;" OwnerUserId="12664" LastActivityDate="2021-07-01T19:03:23.877" Title="Using txt file to query Ensembl API" Tags="|ensembl|perl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16274" PostTypeId="2" ParentId="16273" CreationDate="2021-07-01T03:52:18.457" Score="2" Body="&lt;p&gt;Similar to reading files in other languages, it is necessary to trim the newlines from each line. If you add &lt;a href=&quot;https://perldoc.perl.org/functions/chomp&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;chomp;&lt;/code&gt;&lt;/a&gt; to the first line of the &lt;code&gt;while&lt;/code&gt; loop, the code runs just fine (note that I also print a newline at the end of the loop).&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;open (IDs, '&amp;lt;', &amp;quot;genes.txt&amp;quot;) or die &amp;quot;can't read input file&amp;quot;;&#xA;while (&amp;lt;IDs&amp;gt;){ &#xA;    chomp;&#xA;    my $gene = $gene_adaptor -&amp;gt; fetch_by_display_label($_);&#xA;    print &amp;quot;GENE &amp;quot;, $gene-&amp;gt;stable_id(), &amp;quot;,&amp;quot;;&#xA;}&#xA;print '\n';&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The new loop produces the following result:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;$ perl query.pl &#xA;GENE ENSG00000108691,GENE ENSG00000271503,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Additionally, you may need to &lt;a href=&quot;https://stackoverflow.com/a/4006339/7092484&quot;&gt;catch any exceptions/errors&lt;/a&gt; thrown if you have an invalid gene in your input file:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;open (IDs, '&amp;lt;', &amp;quot;genes.txt&amp;quot;) or die &amp;quot;can't read input file&amp;quot;;&#xA;while (&amp;lt;IDs&amp;gt;){ &#xA;    chomp;&#xA;    eval {&#xA;        my $gene = $gene_adaptor -&amp;gt; fetch_by_display_label($_);&#xA;        print &amp;quot;GENE &amp;quot;, $gene-&amp;gt;stable_id(), &amp;quot;,&amp;quot;;&#xA;    };&#xA;    if ($@) {&#xA;        warn &amp;quot;Error:\t$@\n&amp;quot;&#xA;    }&#xA;}&#xA;print '\n';&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6628" LastEditorUserId="6628" LastEditDate="2021-07-01T19:03:23.877" LastActivityDate="2021-07-01T19:03:23.877" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16275" PostTypeId="2" ParentId="16270" CreationDate="2021-07-01T06:04:04.937" Score="0" Body="&lt;p&gt;Have you considered using a sorted and indexed bam file? e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools sort in.bam &amp;gt; out.sorted.bam&#xA;samtools index out.sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This makes arbitrary region lookups trivial. I am more familiar with pysam (HTSlib for python), which has the &lt;code&gt;AlignmentFile.fetch()&lt;/code&gt; functionality.&lt;/p&gt;&#xA;&lt;p&gt;Surely there is an analog in HTSlib API, though I don't know what to search to get it. It is in samtools as demonstrated &lt;a href=&quot;https://www.biostars.org/p/151403/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/151053/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This answer&lt;/a&gt; seems to put the components together that you are interested in, among them &lt;code&gt;sam_index_load()&lt;/code&gt;, &lt;code&gt;sam_itr_querys()&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2021-07-01T06:49:24.857" LastActivityDate="2021-07-01T06:49:24.857" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16276" PostTypeId="1" CreationDate="2021-07-01T14:24:39.813" Score="0" ViewCount="145" Body="&lt;p&gt;I have helped a lab sequence a mitochondrial genome. I used then MITOS to annotate the genome (warning them that I did it just for curiosity as I am not experienced). They submitted the sequence and annotation to genbank but it was refused for different reasons.&lt;/p&gt;&#xA;&lt;p&gt;It's the genome of a species for which there is no reference. It's a jumping spider and the genome is almost identical to the one below. I used MITOS web server and I got an annotation almost identical to this one (&lt;a href=&quot;http://mitos.bioinf.uni-leipzig.de/result.py?hash=bJuyJw4b&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mitos.bioinf.uni-leipzig.de/result.py?hash=bJuyJw4b&lt;/a&gt;) which corresponds to this sequence (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/MH891570&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/MH891570&lt;/a&gt; ). But the submitted annotation to Genbank is slightly different so I guess they manually edited it or they used another software. I am not sure I can share more specific information without the permission of the lab.&lt;/p&gt;&#xA;&lt;p&gt;You can download a GFF file or the raw sequence at the page in the link (for the similar sequence).&lt;/p&gt;&#xA;&lt;p&gt;My questions are as follows:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;after applying MITOS, is there any manual or automatic review that should be applied?&lt;/li&gt;&#xA;&lt;li&gt;Is it common that the annotation returned by MITOS is not good enough?&lt;/li&gt;&#xA;&lt;li&gt;Or could that indicate that maybe there is a problem in the assembled genome? (looks unlikely except for a few SNPs as three assemblers return almost the same sequence)&lt;/li&gt;&#xA;&lt;li&gt;Should we edit the annotation ourselves or there is a software that does it itself?&lt;/li&gt;&#xA;&lt;li&gt;How often using MITOS is enough or how often manually editing is needed?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The sequence corresponds to an arachnid. The reasons given by Genbank are:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Some or all of the sequences contain incorrect annotation and/or&#xA;contain little similarity to other sequences in the database based on&#xA;BLAST searches. Please also review your nucleotide sequences for&#xA;issues such as low quality data or frameshifts.&lt;/p&gt;&#xA;&lt;p&gt;Incorrect annotation is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;coding regions lack stop codons&lt;/li&gt;&#xA;&lt;li&gt;CDS features do not have the correct nucleotide spans or location&#xA;for example, nad5 is longer at the amino terminus compared to&#xA;similar proteins in the database and extends into the flanking tRNA.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The first part of their reply is apparently generic as I have seen the same answer in another submission in which I didn't participate. I am not sure it's even verified as there are quite good BLAST matches.&#xA;I have no idea about the other part. I just applied MITOS.&lt;/p&gt;&#xA;&lt;p&gt;I used the invertebrate mitochondrial translation table option in MITOS. I guess it's the correct one as the result quite makes sense except for small adjustments.&lt;/p&gt;&#xA;" OwnerUserId="5190" LastEditorUserId="73" LastEditDate="2021-07-04T20:57:56.727" LastActivityDate="2021-07-09T12:34:10.053" Title="Genome annotation" Tags="|assembly|sequence-annotation|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16277" PostTypeId="2" ParentId="16271" CreationDate="2021-07-01T16:25:07.613" Score="2" Body="&lt;p&gt;Let's first establish a lower bound on any solution:&lt;/p&gt;&#xA;&lt;p&gt;Given a hamming distance &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; and an input pattern &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt; of length &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;, there are &lt;span class=&quot;math-container&quot;&gt;$\sum_{k=0}^D{N \choose k} 3^k$&lt;/span&gt; other strings within hamming distance &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; of &lt;span class=&quot;math-container&quot;&gt;$P$&lt;/span&gt;. Of course in the worst case, when &lt;span class=&quot;math-container&quot;&gt;$D=N$&lt;/span&gt;, there are &lt;span class=&quot;math-container&quot;&gt;$4^n$&lt;/span&gt; possible strings.&#xA;We can also get a closed bound on the number of solutions. We can show that the sum of the first &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; binomial coefficients for a fixed &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; is bounded by &lt;span class=&quot;math-container&quot;&gt;${N \choose D} {N-(D-1) \over N-(2D-1)}$&lt;/span&gt; as shown &lt;a href=&quot;https://mathoverflow.net/a/17236&quot;&gt;here&lt;/a&gt;. Therefore we have the following closed upper bound on the number of strings within hamming distance &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; of some pattern of length &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\sum_{k=0}^D{N \choose k} 3^k \leq {N \choose D}  {3^D (N-(D-1)) \over N-(2D-1)} \leq \left(\frac{3N}{D}\right)^D {N-(D-1) \over N-(2D-1)}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, for your approach, you are going through recursively, so when the subproblem is on strings within &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt; of the suffix of length &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;, you have &lt;span class=&quot;math-container&quot;&gt;$\sum_{k=0}^D{n \choose k} 3^k$&lt;/span&gt;  of these. Given that you have &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; subproblems (suffixes), at first glance it does indeed look like you have roughly &lt;span class=&quot;math-container&quot;&gt;$\sum_{n=1}^N\sum_{k=0}^D{n \choose k} 3^k$&lt;/span&gt; iterations of the innermost for loop. However, in that &lt;code&gt;for&lt;/code&gt; loop, you are also computing the Hamming Distance function, which is &lt;span class=&quot;math-container&quot;&gt;$O(n)$&lt;/span&gt;. This means that you have roughly&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;\sum_{n=1}^Nn\sum_{k=0}^D{n \choose k} 3^k&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;computations.&lt;/p&gt;&#xA;&lt;p&gt;I would suggest finding a way to get rid of having to compute the hamming function. Since you are constructing the sequences yourself, you can keep track of each sequences hamming distance from the pattern during construction.&lt;/p&gt;&#xA;" OwnerUserId="6628" LastEditorUserId="6628" LastEditDate="2021-07-01T17:55:24.620" LastActivityDate="2021-07-01T17:55:24.620" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16278" PostTypeId="1" AcceptedAnswerId="16284" CreationDate="2021-07-01T17:33:43.797" Score="1" ViewCount="30" Body="&lt;p&gt;I've done a differential expression analysis with transcriptomic information (microarrays and RNAseq) from different datasets by applying for some of them t-test and for the rest Wald-test. I've got many coincidences for these significant genes so I want to recalculate my p-values (or my adjusted p-values, obtained by the BH method) using all these datasets. What is the proper method to do this (if possible)? I am completely new in this field so any additional information would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="12360" LastActivityDate="2021-07-02T13:18:50.267" Title="P value recalculation from multiple analysis (Transcriptomics)" Tags="|statistics|microarray|p-values|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16279" PostTypeId="1" CreationDate="2021-07-01T20:24:46.603" Score="0" ViewCount="32" Body="&lt;p&gt;I have a group of L. plantarum genomes. I identified the pangenome using Roary. From the pangenome, I was able to identify unique 'genes' for some of the strains. One of these (actually several were, but I'm only interested in one for now) was tagged as 'investigate' in the QC column of the gene_presence_absence matrix, so I did some investigating. Looking at this sequence, I saw it was a hypothetical protein of around 4k bases, and when I ran it through BLAST on the NCBI website, far from being unique, it gave 100% matches to many other L. plantarum strains. Further investigation suggests that this sequence was present on a contig not much larger than the 'gene' its self.&lt;/p&gt;&#xA;&lt;p&gt;I have two questions.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;any ideas why this was identified as unique by Roary but ubiquitous in other members of the species on BLAST (accepting that it's possible it genuinely is unique among my 30 or so strains, but common in the species more generally - it just seems unlikely) - maybe something to do with poorly called open reading frame in prodigal (genome was annotated by prokka)? Or something to do with Roary separating paralogs?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;what is the simplest way of confirming if this 4k sequence is/isn't present in my group of 30 genomes?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="6738" LastActivityDate="2021-07-01T20:24:46.603" Title="Search for specific sequence in group of bacterial genomes" Tags="|genome|bacteria|sequence-annotation|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16280" PostTypeId="2" ParentId="16270" CreationDate="2021-07-01T23:56:18.093" Score="2" Body="&lt;pre class=&quot;lang-js prettyprint-override&quot;&gt;&lt;code&gt;#include &amp;lt;stdio.h&amp;gt;&#xA;#include &amp;quot;htslib/sam.h&amp;quot;&#xA;int main(int argc, char *argv[])&#xA;{&#xA;  samFile *fp = sam_open(argv[1], &amp;quot;r&amp;quot;);&#xA;  hts_idx_t *idx = sam_index_load(fp, argv[1]);&#xA;  bam_hdr_t *h = sam_hdr_read(fp);&#xA;  bam1_t *b = bam_init1();&#xA;  hts_itr_t *itr = sam_itr_querys(idx, h, argc &amp;lt; 3? &amp;quot;.&amp;quot; : argv[2]);&#xA;  while (sam_itr_next(fp, itr, b) &amp;gt;= 0) {&#xA;    if (b-&amp;gt;core.tid &amp;lt; 0) continue;&#xA;    printf(&amp;quot;%s\t%d\t%d\n&amp;quot;, h-&amp;gt;target_name[b-&amp;gt;core.tid], b-&amp;gt;core.pos,&#xA;      b-&amp;gt;core.pos + bam_cigar2rlen(b-&amp;gt;core.n_cigar, bam_get_cigar(b)));&#xA;  }&#xA;  bam_destroy1(b);&#xA;  hts_itr_destroy(itr);&#xA;  bam_hdr_destroy(h);&#xA;  sam_close(fp);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The usage would then be &lt;code&gt;./a.out alignments.bam chr1:123-456&lt;/code&gt;, assuming your BAM file is sorted an indexed. Importantly this is making use of the index via &lt;code&gt;sam_itr_querys()&lt;/code&gt; so it's not iterating over the entire file.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="77" LastEditDate="2021-07-03T20:21:55.200" LastActivityDate="2021-07-03T20:21:55.200" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16281" PostTypeId="1" CreationDate="2021-07-02T09:09:11.380" Score="1" ViewCount="429" Body="&lt;p&gt;I was trying to make a dotplot to visualize genome-genome sequence alignment by using the MUMmer 3 software (it is typically used to compare whole genome sequences of bacteria). I runned the same command lines that I wrote in this question post &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/16039/what-is-the-file-mums-created-by-mummer&quot;&gt;What is the file .mums created by mummer?&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I obtain a plot that is consistent with what I expected but I do not understand why it does not show the axis scales (or labels ? I do not know how to explain but I report below my plot vs the plot made in the MUMmer page example &lt;a href=&quot;https://mummer4.github.io/tutorial/tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;2.1. mummer&lt;/a&gt; so that you can understand what I mean). There could be two solutions:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;1&lt;/strong&gt;. Since my command lines are the same as the one of the example that I found on web, I think that I should modify the mummerplot code to fix this problem.&lt;/p&gt;&#xA;&lt;p&gt;However, the code is written in C/C++, so not really easy for me. I tried to modify some lines of code but with no success. If you want to have a look at the code and give me some suggestions, it is available on &lt;a href=&quot;https://github.com/marbl/MUMmer3/blob/master/scripts/mummerplot.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;2&lt;/strong&gt;. But I wonder also if it is the only way to adjust the mummerplot plot. I found in &lt;a href=&quot;http://mummer.sourceforge.net/manual/#coords&quot; rel=&quot;nofollow noreferrer&quot;&gt;the MUMmer 3 manual&lt;/a&gt; this:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The mummerplot script outputs three files, .gp .fplot .rplot, when run with standard parameters. The first of which is the gnuplot script. This script contains the commands necessary to generate the plot, and refers to the two data files which contain the forward and reverse matches respectively.  [...] &lt;strong&gt;Line thickness, color, and many other options can be added or removed from the plot by hand editing the gnuplot script&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Indeed I noticed that my .gp file has some differences with respect to the .gp file of the example that I linked above (the one &lt;a href=&quot;https://mummer4.github.io/tutorial/tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). So I could simply modify my .gp file writing it as the one of the example and obtain the plot as I want it, as also the manual suggests. But how can I do it if running mummerplot to generate my plot I obtain the .ps image of the plot and the .gp file at the same time ? The .gp file is not something that I can create before running the mummerplot command...&lt;/p&gt;&#xA;&lt;p&gt;Below I report, respectively, my plot and the plot of the example (whose aesthetics I would like to imitate):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ZiZFy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZiZFy.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Hvrkd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Hvrkd.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thus, I ask if you can give me any hint by answering to one of the two questions or even both or even if there is a third way to solve that I ignore. Thank you in advance !&lt;/p&gt;&#xA;" OwnerUserId="10594" LastEditorUserId="10594" LastEditDate="2021-07-02T13:02:44.600" LastActivityDate="2021-07-02T13:02:44.600" Title="How to modify dot plot in MUMmer 3 for bacteria comparative genomics?" Tags="|sequence-alignment|genomics|visualization|software-usage|bacteria|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16282" PostTypeId="1" AcceptedAnswerId="16295" CreationDate="2021-07-02T10:51:28.210" Score="2" ViewCount="104" Body="&lt;p&gt;I am trying to understand in detail how does the gapped BLAST seed extension DP algorithm works. Despite searching for days now, I could not find anywhere a pseudocode that would allow implementing this in python. Not being a C programmer, it would be impossible for me to grasp this from the BLAST source code.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone help me with this?&lt;/p&gt;&#xA;&lt;p&gt;Thank you all. Best,&lt;/p&gt;&#xA;&lt;p&gt;Daniel&lt;/p&gt;&#xA;" OwnerUserId="12672" LastActivityDate="2021-07-05T13:32:02.417" Title="Pseudocode for gapped BLAST seed extension" Tags="|blast|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16283" PostTypeId="2" ParentId="16278" CreationDate="2021-07-02T11:39:50.707" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What is the proper method to do this (if possible)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Please use dedicated software for RNA-seq and microarray analysis and &lt;strong&gt;not&lt;/strong&gt; naive tests such as the t-test. For microarray one commonly uses &lt;code&gt;limma&lt;/code&gt; and for RNA-seq either &lt;code&gt;DESeq2&lt;/code&gt;, &lt;code&gt;edgeR&lt;/code&gt; or &lt;code&gt;limma-voom&lt;/code&gt;, all are available as Bioconductor packages in R.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-07-02T11:39:50.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16284" PostTypeId="2" ParentId="16278" CreationDate="2021-07-02T13:18:50.267" Score="0" Body="&lt;p&gt;For cDNA-derived data have a look at the &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#quick-start&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2 analysis guide&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The general process is to convert all your input data into an integer &lt;em&gt;count&lt;/em&gt; matrix (not expression matrix), with genes in rows and samples in columns, and pair it with a metadata data frame that has one line per sample (ordered in the same way as the columns), and indicates the conditions for each sample (including sequencing batch). By incorporating the sequencing batch into the statistical model, it's possible to reduce any technical error associated with consistent linear changes in expression based on the batch (e.g. different runs giving different numbers of total reads).&lt;/p&gt;&#xA;&lt;p&gt;Microarrays &lt;em&gt;can&lt;/em&gt; be combined, bearing in mind that microarray results are often inconsistently presented as normalised / non-normalised, or linear / log. I've found that normalising within-batch then converting to log space usually seems to work well. There is a verbose Users Guide for &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/vignettes/limma/inst/doc/usersguide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Limma Voom&lt;/a&gt;, but I've found it to be tricky to navigate - it's less friendly to new users.&lt;/p&gt;&#xA;&lt;p&gt;Combining microarrays and RNASeq is complicated, because the data have different dynamic ranges, and different biases. I think the best thing to do is to process the microarray data separately from the RNASeq data and generate two lists of biologically-interesting genes, then look at the set relationships (e.g. intersection, exclusion).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-02T13:18:50.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16286" PostTypeId="1" CreationDate="2021-07-02T17:46:06.893" Score="1" ViewCount="20" Body="&lt;p&gt;I'm trying to use &lt;a href=&quot;https://github.com/Taiji-pipeline/Taiji&quot; rel=&quot;nofollow noreferrer&quot;&gt;Taiji&lt;/a&gt; to run a pagerank on our combined bulk RNAseq and ATACseq dataset.&lt;/p&gt;&#xA;&lt;p&gt;Here's how we run it:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;taiji run --config config.yml -n 10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here's &lt;code&gt;config.yml&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;input: &amp;quot;input.yml&amp;quot;&#xA;output_dir: &amp;quot;output2/&amp;quot;&#xA;assembly: &amp;quot;mm10&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here's a snippet of &lt;code&gt;input.yml&lt;/code&gt; (the real one has 8 conditions):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATAC-Seq:&#xA;  - id: cond1_ATAC&#xA;    group: cond1&#xA;    replicates:&#xA;      - rep: 1&#xA;        files:&#xA;          - path: p1.cond1.narrowPeak&#xA;            format: NarrowPeak&#xA;      - rep: 2&#xA;        files:&#xA;          - path: p2.cond1.narrowPeak&#xA;            format: NarrowPeak&#xA;      - rep: 3&#xA;        files:&#xA;          - path: p3.cond1.narrowPeak&#xA;            format: NarrowPeak&#xA;&#xA;  - id: cond2_ATAC&#xA;    group: cond2&#xA;    replicates:&#xA;      - rep: 1&#xA;        files:&#xA;          - path: p1.cond2.narrowPeak&#xA;            format: NarrowPeak&#xA;      - rep: 2&#xA;        files:&#xA;          - path: p2.cond2.narrowPeak&#xA;            format: NarrowPeak&#xA;      - rep: 3&#xA;        files:&#xA;          - path: p3.cond2.narrowPeak&#xA;            format: NarrowPeak&#xA;&#xA;RNA-Seq:&#xA;  - id: cond1_RNA&#xA;    group: cond1&#xA;    replicates:&#xA;      - rep: 1&#xA;        files:&#xA;          - path: m1.cond1.tsv&#xA;            tags: ['GeneQuant']&#xA;      - rep: 2&#xA;        files:&#xA;          - path: m2.cond1.tsv&#xA;            tags: ['GeneQuant']&#xA;      - rep: 3&#xA;        files:&#xA;          - path: m3.cond1.tsv&#xA;            tags: ['GeneQuant']&#xA;&#xA;  - id: cond2_RNA&#xA;    group: cond2&#xA;    replicates:&#xA;      - rep: 1&#xA;        files:&#xA;          - path: m1.cond2.tsv&#xA;            tags: ['GeneQuant']&#xA;      - rep: 2&#xA;        files:&#xA;          - path: m2.cond2.tsv&#xA;            tags: ['GeneQuant']&#xA;      - rep: 3&#xA;        files:&#xA;          - path: m3.cond2.tsv&#xA;            tags: ['GeneQuant']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We've checked that all the files are there. The NarrowPeak files were created by macs2, the tsv files are 2 column tsv's where column 1 is all EntrezID's we were measuring and column 2 is the expression of that gene.&lt;/p&gt;&#xA;&lt;p&gt;The output of the pipeline is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATACseq&#xA;|-Download (empty directory&#xA;|-TFBS&#xA;| |-all narrowpeak files copied over in bed.gz format&#xA;| |-9 motif_sites_part.bed7411-?.gz (where ? is [0-8])&#xA;|-openChromatin.bed.gz (all peaks in the dataset)&#xA;GENOME (contains the mm10 genome)&#xA;RNAseq&#xA;|-All tsv files copied over&#xA;|-expression_profile.tsv (pasted all txv files together)&#xA;|-Download (empty directory)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://taiji-pipeline.github.io/documentation/tutorial.html#introduction&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual linked here&lt;/a&gt; says at the bottom there should also be two other files in the output directory, &lt;code&gt;GeneRanks_PValues.tsv&lt;/code&gt; and &lt;code&gt;GeneRanks.tsv&lt;/code&gt;. These are the only files we were looking for. Anybody have a clue why we don't see anything? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="9521" LastActivityDate="2021-07-02T17:46:06.893" Title="Taiji doesn't run Pagerank" Tags="|rna-seq|atac-seq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16287" PostTypeId="1" CreationDate="2021-07-02T21:13:03.620" Score="1" ViewCount="842" Body="&lt;p&gt;I want to draw the standard plot of GSEA on Rstudio. I have a data frame that consists of a list of DEGs as below.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/D4kdb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D4kdb.png&quot; alt=&quot;Structure&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Actually, I put the DEG list directly to PANTHER that resulted in the GO or pathway analysis data. However, I want to use GSEA.&lt;/p&gt;&#xA;&lt;p&gt;Meanwhile, I found nice online site and follow their codes for the GSEA analysis(&lt;a href=&quot;https://bioinformaticsbreakdown.com/how-to-gsea/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioinformaticsbreakdown.com/how-to-gsea/&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;And, here is the my code.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tmp = read.csv(&amp;quot;/data/directory/DEGs_HCvsJSS_DC.csv&amp;quot;, header = TRUE) %&amp;gt;% filter(Gene.Symbol !=&amp;quot;&amp;quot;)&#xA;&#xA;GO_file = &amp;quot;/blue/zhangw/my.kim/HCvsJSS/c5.all.v7.4.symbols.gmt&amp;quot; # got this from http://www.gsea-msigdb.org/gsea/msigdb/collections.jsp&#xA;&#xA;gene_list = tmp$avg_log2FC&#xA;&#xA;names(gene_list) = tmp$Gene.Symbol&#xA;&#xA;gene_list = sort(gene_list, decreasing = TRUE)&#xA;&#xA;gene_list = gene_list[!duplicated(names(gene_list))]&#xA;&#xA;head(gene_list)&#xA;&#xA;res = GSEA(gene_list, GO_file, pval = 0.05) # this caused the error as follows. Error in build_Anno(TERM2GENE, TERM2NAME) :  argument &amp;quot;TERM2GENE&amp;quot; is missing, with no default&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It's really dissapointing because I found this relatively easy protocol for GSEA, but found the error, lol.&lt;/p&gt;&#xA;&lt;p&gt;Do you guys have any suggests for this troubles? or any recommendations for GSEA analysis that can be done with DEG list generated from seruat scRNAseq data?&lt;/p&gt;&#xA;" OwnerUserId="8657" LastEditorUserId="3694" LastEditDate="2021-07-04T15:07:11.230" LastActivityDate="2021-12-03T17:00:35.127" Title="How to run GSEA analysis on R studio using DEG file list generated from scRNAseq analysis" Tags="|r|scrnaseq|gsea|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16288" PostTypeId="1" CreationDate="2021-07-02T22:50:22.723" Score="0" ViewCount="286" Body="&lt;p&gt;BowTie2 puts out summary info to the terminal but doesn't allow me to save each to a separate file. How can I go about saving the output of BowTie2 so that MultiQC can read it and create a website from it?&lt;/p&gt;&#xA;" OwnerUserId="12676" LastActivityDate="2021-07-03T09:26:57.783" Title="How to export summary data from Bowtie2 for MultiQC to read?" Tags="|bowtie2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16289" PostTypeId="2" ParentId="16288" CreationDate="2021-07-02T22:53:26.967" Score="1" Body="&lt;p&gt;After trying on my own this is the solution I came up with.&#xA;My solution was to use a pipe of the stderr into a new file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bowtie2 --local --threads 4 -x reference_genome.fa -f -U inputdata.fa -S OutputData.sam --no-hd --no-sq --no-unal -k 5 2&amp;gt; 'bowtie2_summary/inputdata.txt'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12676" LastEditorUserId="73" LastEditDate="2021-07-03T09:26:57.783" LastActivityDate="2021-07-03T09:26:57.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16290" PostTypeId="2" ParentId="16276" CreationDate="2021-07-03T09:18:56.997" Score="2" Body="&lt;p&gt;Any regions annotated as protein sequences must have a complete protein sequence, including start and stop codons. It's somewhat common for mitochondrial genomes to require polyA extension for the stop codons to be completed; I expect that NCBI is asking that the submissions make this explicit. See, for example, our lab's Nippostrongylus brasiliensis mtDNA submission:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NC_033886.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/NC_033886.1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As an example in this genome annotation, the ND5 gene requires extension, as mentioned in the note:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;3858..5439&#xA;/gene=&amp;quot;ND5&amp;quot;&#xA;/locus_tag=&amp;quot;B2K98_mgp10&amp;quot;&#xA;/note=&amp;quot;TAA stop codon is completed by the addition of 3' A&#xA;residues to the mRNA&amp;quot;&#xA;/codon_start=1&#xA;/transl_except=(pos:5439,aa:TERM)&#xA;/transl_table=5&#xA;/product=&amp;quot;NADH dehydrogenase subunit 5&amp;quot;&#xA;/protein_id=&amp;quot;YP_009347353.1&amp;quot;&#xA;/db_xref=&amp;quot;GeneID:31081492&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It doesn't surprise me that an automated tool is not perfect. The mitochondrial genome is small enough that manual verification can be feasibly done. You should at least look at the annotated information to make sure it makes sense; the response from NCBI suggests that it doesn't (e.g. nad5 extends into an annotated tRNA sequence).&lt;/p&gt;&#xA;&lt;p&gt;As a direct / explicit response to your questions, just in case my answers are useful:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;after applying MITOS, is there any manual or automatic review that should be applied?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't know. You should at least look at the annotated information to make sure it makes sense.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is it common that the annotation returned by MITOS is not good enough?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't know; I have never used MITOS.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Or could that indicate that maybe there is a problem in the assembled genome? (looks unlikely except for a few SNPs as three assemblers return almost the same sequence)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't know; you haven't provided the assembled gene sequence or annotation. The response from NCBI suggests that there is a problem with either the sequence or annotation (e.g. nad5 extends into an annotated tRNA sequence).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Should we edit the annotation ourselves or there is a software that does it itself?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If there is an error in the annotation through automated methods, you should fix it manually.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How often using MITOS is enough or how often manually editing is needed?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't know; I have never used MITOS.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-07-06T20:31:36.453" LastActivityDate="2021-07-06T20:31:36.453" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16292" PostTypeId="1" CreationDate="2021-07-03T10:22:41.210" Score="1" ViewCount="23" Body="&lt;p&gt;I want to work string-db to read the protein-protein interaction from it and I look up disease association from disgenet but the Ids do not seem to match-up and I do not know what to do.&lt;/p&gt;&#xA;&lt;p&gt;for example for protein &lt;code&gt;INSR&lt;/code&gt; which is mostly afflicted with diabetes and similar diseases I found that from 'disgenet' but I do not find the ids in string, is the a convertion table some where?&lt;/p&gt;&#xA;&lt;pre&gt;&#xA;&lt;code&gt;protein1 protein2 combined_score&#xA;9606.ENSP00000000233 9606.ENSP00000272298 490&#xA;&lt;/code&gt;&#xA;&lt;/pre&gt;&#xA;&lt;p&gt;PS: I think this question may sound stupid to you but I am an absolute noob and would be glad to help me out.&lt;/p&gt;&#xA;" OwnerUserId="12677" LastActivityDate="2021-07-03T10:22:41.210" Title="How to find disease association of disgenet with string db in homo sapiens" Tags="|proteins|networks|stringdb|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16293" PostTypeId="1" CreationDate="2021-07-03T12:14:47.243" Score="1" ViewCount="3456" Body="&lt;p&gt;I downloaded some of my data in fastq format instead of fastq.gz. I want to upload my data on a server now.&lt;/p&gt;&#xA;&lt;p&gt;I was wondering if there is a way to convert my fastq files to fastq.gz on windows (I read some stuff about python, but I'm not quite sure) before uploading them on the server, since it takes a lot of time uploading the fastq files.&lt;/p&gt;&#xA;" OwnerUserId="12678" LastEditorUserId="96" LastEditDate="2022-06-27T14:51:35.833" LastActivityDate="2023-09-26T08:07:27.913" Title="Compressing read data (converting fastq to fastq.gz) on Windows" Tags="|fasta|fastq|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16294" PostTypeId="2" ParentId="16293" CreationDate="2021-07-03T13:37:31.723" Score="7" Body="&lt;p&gt;&lt;code&gt;7zip&lt;/code&gt; is file archiver capable of running &lt;code&gt;gzip&lt;/code&gt;-style compression, and runs on Windows: &lt;a href=&quot;https://www.7-zip.org/&quot; rel=&quot;noreferrer&quot;&gt;https://www.7-zip.org/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6332" LastActivityDate="2021-07-03T13:37:31.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16295" PostTypeId="2" ParentId="16282" CreationDate="2021-07-03T21:10:32.483" Score="3" Body="&lt;p&gt;So it turns out the &lt;a href=&quot;https://academic.oup.com/nar/article/25/17/3389/1061651&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gapped Blast paper&lt;/a&gt; doesn't actually give the extension algorithm. It gives a high level overview and refers to an article in preparation at the time:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The central idea is to consider only cells for which the optimal local alignment score falls no more than &lt;span class=&quot;math-container&quot;&gt;$X_g$&lt;/span&gt; below the best alignment score yet found. Starting from a single aligned pair of residues, called the seed, the dynamic programming proceeds both forward and backward through the path graph (Zheng Zhang et al., manuscript in preparation) (Figs 3a and 4). The advantage of this approach is that the region of the path graph explored adapts to the alignment being constructed. The alignment can wander arbitrarily many diagonals away from the seed, but the number of cells expanded on each row tends to remain limited, and may even shrink to zero before a boundary of the path graph is encountered (Fig. 4). The &lt;span class=&quot;math-container&quot;&gt;$X_g$&lt;/span&gt; parameter serves a similar function to the band-width parameter of the earlier heuristic, but the region of the path graph it implicitly specifies be explored is in general more productively chosen.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The motivation for this approach is to be a &amp;quot;smarter&amp;quot; version of the banded alignment approach. The issue with banded alignment approaches is that they implicitly won't allow for large insertions/deletions within the alignment, even if the areas around the alignment are high scoring. Increasing the band with to allow for this reduces the speedup gained by the band constraint. This is where the idea of &amp;quot;&lt;a href=&quot;https://www.bx.psu.edu/miller_lab/dist/lowscore.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alignments Without Low-Scoring Regions&lt;/a&gt;&amp;quot; comes into play. This paper is the one cited as &amp;quot;in-prep&amp;quot; by the gapped blast paper, and it describes an early version of the gapped extension approach. However, that  approach seems to have been superseded by the one described &lt;a href=&quot;http://pipmaker.bx.psu.edu/dist/greedy.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; 2 years later by the same lead author (the former paper in '98 having 67 citations and the latter in 2000 having 4700+).&lt;/p&gt;&#xA;&lt;p&gt;For completeness, I've pasted some pseudocode from the more popular x-drop paper from 2000. Note that &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; is a user specified parameter, with larger &lt;span class=&quot;math-container&quot;&gt;$X$&lt;/span&gt; allowing for a larger search space but slower speed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/craWo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/craWo.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6628" LastEditorUserId="6628" LastEditDate="2021-07-05T13:32:02.417" LastActivityDate="2021-07-05T13:32:02.417" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="16296" PostTypeId="2" ParentId="5124" CreationDate="2021-07-04T04:19:24.237" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @user172818, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;My understanding is that you can't compute Jaccard similarity given two count-min sketches. You can with two histosketches, though I don't know exactly how yet.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-04T04:19:24.237" CommentCount="0" CommunityOwnedDate="2021-07-04T04:19:24.237" ContentLicense="CC BY-SA 4.0" />
  <row Id="16297" PostTypeId="2" ParentId="4996" CreationDate="2021-07-04T04:21:12.437" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @devon-ryan, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Try a UCSC gene ID instead. For those genes they'd be: &lt;code&gt;uc001zni.3&lt;/code&gt; (INO80), &lt;code&gt;uc002vaz.4&lt;/code&gt; (INO80D), &lt;code&gt;uc004eup.4&lt;/code&gt; or &lt;code&gt;uc004eun.4&lt;/code&gt; (SMARCA1) and &lt;code&gt;uc001coj.2&lt;/code&gt;, &lt;code&gt;uc010olq.2&lt;/code&gt;, &lt;code&gt;uc010olr.2&lt;/code&gt;, &lt;code&gt;uc001coi.2&lt;/code&gt; or &lt;code&gt;uc001col.2&lt;/code&gt; (NASP). You might have to omit the versions (the &lt;code&gt;.4&lt;/code&gt; part, for example).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-04T04:21:12.437" CommentCount="0" CommunityOwnedDate="2021-07-04T04:21:12.437" ContentLicense="CC BY-SA 4.0" />
  <row Id="16298" PostTypeId="2" ParentId="4900" CreationDate="2021-07-04T04:27:14.133" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @Kozolovska, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Found the problem, some of ALT were structural variant, so the list of vcf2@info$ALT was a charterer list and not a 'DNAStringSetList' after removing that variant it worked. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-04T04:27:14.133" CommentCount="0" CommunityOwnedDate="2021-07-04T04:27:14.133" ContentLicense="CC BY-SA 4.0" />
  <row Id="16299" PostTypeId="2" ParentId="3045" CreationDate="2021-07-04T04:33:10.100" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @Llrs, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would say that the problem is defining each steps on your folding path, it is a continum, I don't know if you have seen one of those videos/gifs of a protein folding, they are quite illustrative on this aspect. Usually there isn't a clear step with a pause unless it reaches a local minimum, it could be possible to save that minimums with its folding but usually they take time and resources to calculates the folding process for a long time, so probably one ends up with just a few steps of the folding.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-04T04:33:10.100" CommentCount="0" CommunityOwnedDate="2021-07-04T04:33:10.100" ContentLicense="CC BY-SA 4.0" />
  <row Id="16300" PostTypeId="2" ParentId="2964" CreationDate="2021-07-04T04:40:39.627" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @mjoppich, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Late answer, but it looks like you are running out of memory, hence the intptr is invalid.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-04T04:40:39.627" CommentCount="0" CommunityOwnedDate="2021-07-04T04:40:39.627" ContentLicense="CC BY-SA 4.0" />
  <row Id="16301" PostTypeId="2" ParentId="3991" CreationDate="2021-07-04T04:52:32.073" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @llrs, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I found &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3695771/&quot; rel=&quot;nofollow noreferrer&quot;&gt;an article&lt;/a&gt; where they said that they could do GWAS in 6000 samples in 6 hours in a regular user computer. But I couldn't find any other explanation about how does it take with other kind of data. It wasn't with plink too and I think it was a bit outdated.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Answer from @christopher-chang, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Incidentally, you may want to use plink 2.0 rather than 1.9 for this. It automatically makes good use of all your cores; in contrast, you’d need to start ~24 copies of plink 1.9 to saturate a 24-core machine.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-04T04:52:32.073" CommentCount="0" CommunityOwnedDate="2021-07-04T04:52:32.073" ContentLicense="CC BY-SA 4.0" />
  <row Id="16302" PostTypeId="1" AcceptedAnswerId="17891" CreationDate="2021-07-04T15:08:14.087" Score="3" ViewCount="65" Body="&lt;p&gt;I am working on a project proposal for an exam, it involves a budget of 1M euros and I got to design a genomic study. I chose a GWAS to assess altered gene expression after covid19 that leads to long-COVID. I thought about 3 cohorts:&lt;/p&gt;&#xA;&lt;p&gt;Case cohort1: Long-Covid patients (N: 3,000)&lt;/p&gt;&#xA;&lt;p&gt;Case cohort2: Fully recovered patients (N: 3000)&lt;/p&gt;&#xA;&lt;p&gt;Control cohort: healthy patients. (N: 6000)&lt;/p&gt;&#xA;&lt;p&gt;Would it make sense to run two GWAS for the two case cohorts against the control cohort (cohort1/controls and cohort2/controls) then do a miami plot to see if there are some hits that differs from the two cohorts, and pick the ones only presented in the Long-covid for further assessment?&lt;/p&gt;&#xA;&lt;p&gt;The study will also be supported by a validation step for both gwas. What do you think about this idea? Would it make sense or the fully-recovered patients against control cohorts is totally useless? Thank you in advance&lt;/p&gt;&#xA;" OwnerUserId="12682" LastActivityDate="2021-10-19T15:31:27.360" Title="Long-Covid (LC) GWAS project proposal: should I include a non-LC GWAS?" Tags="|covid-19|gwas|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16303" PostTypeId="1" CreationDate="2021-07-04T16:32:06.327" Score="6" ViewCount="538" Body="&lt;p&gt;I have the files of 2 persons that got whole genome sequencing, one by nebula genomics and the other by dante labs.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I want to compare their genomes to see if they are father and son.&lt;/li&gt;&#xA;&lt;li&gt;I want to see exactly which genetic sequences they share.&lt;/li&gt;&#xA;&lt;li&gt;In the shared sequences, I want to know what mutations exist only in the son.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Which program should I use?&lt;/p&gt;&#xA;&lt;p&gt;Each company provide fastq and vcf files to the genome. so I have each genome in fastq and vcf. which is better to use?&lt;/p&gt;&#xA;&lt;p&gt;(I have the files on my hard drive, so is it better to use offline program?)&lt;/p&gt;&#xA;" OwnerUserId="12683" LastEditorUserId="73" LastEditDate="2021-07-05T01:11:01.390" LastActivityDate="2021-07-05T01:27:08.983" Title="How to compare 2 whole genome sequences" Tags="|wgs|human-genome|" AnswerCount="2" CommentCount="3" ClosedDate="2021-07-14T12:08:34.727" ContentLicense="CC BY-SA 4.0" />
  <row Id="16304" PostTypeId="2" ParentId="16112" CreationDate="2021-07-04T17:57:40.373" Score="1" Body="&lt;p&gt;I am curious whether your pileup visualization tool is respecting the clipping. It is possible that there is a subtlety of samtools ampliconclip that your visualization is not understanding, with regard to how it treats e.g. supplementary alignments or something.&lt;/p&gt;&#xA;&lt;p&gt;Perhaps more importantly, &lt;strong&gt;I notice that the reads that are not being properly clipped in your visualization tend to have one end anchored in each of your BED regions.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Looking at the samtools ampliconclip docs, it seems to be assuming that &lt;strong&gt;reads will only be clipped from the ends&lt;/strong&gt;. They will not be clipped if the region outside the BED is in the middle, if I read it correctly. I think that you may need to include an additional step in which you either trim reads (so that they are not long enough to bridge your regions) or you somehow postprocess your alignments to explicitly clip in the middle.&lt;/p&gt;&#xA;&lt;p&gt;If true, this seems like a significant enough oversight of samtools that it may be worth &lt;a href=&quot;https://github.com/samtools/samtools/issues&quot; rel=&quot;nofollow noreferrer&quot;&gt;logging an issue with the samtools maintenance team&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Also, not clear if it's relevant but reading the samtools ampliconclip docs:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;adjustments to the left most mapping position (POS) will mean that &lt;strong&gt;coordinate sorted files will need resorting. In such cases the sorting order in the header is set to unknown. Clipping of reads results in template length (TLEN) being incorrect.&lt;/strong&gt; This can be corrected by samtools fixmates. Any &lt;em&gt;MD and NM aux tags will also be incorrect&lt;/em&gt;*, which can be fixed by samtools calmd.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-07-04T17:57:40.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16305" PostTypeId="2" ParentId="16303" CreationDate="2021-07-04T23:37:16.287" Score="2" Body="&lt;p&gt;I think you can't do (2) and (3) without the mother.&lt;/p&gt;&#xA;&lt;p&gt;(1) is doable to a very high accuracy. It is better to start with fastq. You need to generate a multi-sample VCF first. Here is a high-level overview: a) align reads with bwa-mem; b) sort alignments with samtools and then index the resulting BAM; c) call variants &lt;em&gt;jointly&lt;/em&gt; with freebayes (or GATK if you like challenges). After you get the two-sample VCF, you go through every site where the father is homozygous and it is a &lt;em&gt;high-quality&lt;/em&gt; site in both samples (e.g. average coverage, high mapping quality, not in difficult regions, etc). Among these sites, check if the child has the paternal allele. If you have a father-son pair, the child is expected to have all the paternal alleles in theory. In practice, if &amp;gt;99% of paternal homozygotes are present in the child, the two samples will almost certainly be a father-son pair. The actual threshold may depend on how you define &amp;quot;high-quality&amp;quot; sites. Errors may lead to apparent mendelian violations.&lt;/p&gt;&#xA;&lt;p&gt;In principle, it is possible to do a similar analysis with the two provided VCFs, but you need to be very careful about various technical artifacts.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2021-07-04T23:37:16.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16306" PostTypeId="2" ParentId="16303" CreationDate="2021-07-05T01:27:08.983" Score="0" Body="&lt;p&gt;Assuming you've got genotyped VCF files from the same type of data, and they are similarly normalised, I expect that the VCF files will be easier to work with initially.&lt;/p&gt;&#xA;&lt;p&gt;That's a lot of assumptions, so until there's more information about the input data, I'll just point you to &lt;a href=&quot;https://samtools.github.io/bcftools/bcftools.html#merge&quot; rel=&quot;nofollow noreferrer&quot;&gt;bcftools merge&lt;/a&gt;, which I think would be a reasonable first step given those assumptions:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools merge -o merged.vcf.gz A.vcf.gz B.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will produce a merged VCF file that contains genotypes from both files on each line. To look for similarities and/or differences, you would then inspect each line to check whether the genotypes were the same or different (e.g. &lt;code&gt;same: 1/1 1/1&lt;/code&gt;; &lt;code&gt;different: 0/1 1/1&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-05T01:27:08.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16307" PostTypeId="1" AcceptedAnswerId="16313" CreationDate="2021-07-05T04:01:45.240" Score="3" ViewCount="245" Body="&lt;p&gt;I am absolutely new to Structural Bioinformatics (I only started last week). I am working on the secondary structure assignment/prediction (actually I am not sure) of proteins using ML.&lt;/p&gt;&#xA;&lt;p&gt;When it comes to protein analysis, most of the books or research papers talk about protein structure &lt;strong&gt;Prediction&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bip.weizmann.ac.il/course/structbioinfo/pdf/SecondaryStructurePrediction_WIS.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bip.weizmann.ac.il/course/structbioinfo/pdf/SecondaryStructurePrediction_WIS.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Only a handful of books or articles talk about protein structure &lt;strong&gt;Assignments&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2021.02.02.429329v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/10.1101/2021.02.02.429329v1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can anyone tell me what is the basic difference between Assignment and Prediction?&lt;/p&gt;&#xA;&lt;p&gt;Kindly explain with examples.&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="12686" LastEditDate="2023-04-10T18:53:15.323" LastActivityDate="2023-04-10T18:53:15.323" Title="What is protein secondary structure assignment?" Tags="|proteins|protein-structure|secondary-structure|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16308" PostTypeId="2" ParentId="8880" CreationDate="2021-07-05T04:19:37.633" Score="0" Body="&lt;p&gt;The file names mean this isn't Nanopore data. Read the bit just above these file names:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Approximately 50x of data was generated on a NovaSeq instrument.&#xA;Based on the summary output of Supernova, there are 1.2 billion reads with 41x effective coverage.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;That linked github page has Nanopore, Illumina/10x, BioNano, HiC, and PacBio data. For the nanopore files, you should be looking at &lt;a href=&quot;https://github.com/nanopore-wgs-consortium/chm13#downloads-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; section.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-05T04:19:37.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16309" PostTypeId="2" ParentId="13277" CreationDate="2021-07-05T04:27:10.650" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @fabio-marroni, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I suggest that you align to the full genome and after this, you only extract the SNPs in the CDS. This procedure is commonly used in all exome sequencing projects, so you can find a lot of instructions software. You can choose between extracting the regions from the alignment (using samtools), or keeping everything, calling the SNPs and then only select those in CDS using vcftools.&lt;/p&gt;&#xA;&lt;p&gt;For looking at the CDS region, you could try bedtools, as suggested &lt;a href=&quot;https://www.biostars.org/p/310492/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-05T04:27:10.650" CommentCount="0" CommunityOwnedDate="2021-07-05T04:27:10.650" ContentLicense="CC BY-SA 4.0" />
  <row Id="16310" PostTypeId="2" ParentId="8988" CreationDate="2021-07-05T04:46:53.230" Score="1" Body="&lt;p&gt;Short answer for short reads: you can't. Not reliably, at least.&lt;/p&gt;&#xA;&lt;p&gt;There's a known error mode with aligning short reads to reference genomes where it's quite good at mis-calling reads within repetitive regions. See, for example, &lt;a href=&quot;https://doi.org/10.1093/gigascience/giz104&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, which compares Illumina vs Nanopore for detecting bacterial strains.&lt;/p&gt;&#xA;&lt;p&gt;If you know that a region is likely to contain a segmental duplication and/or deletion (e.g. due to unexpected changes in read coverage), then it'd be a good idea to double-check any discovered variants (or unexpected non-variants) using a different technology that does not have the same coverage issues. I recommend a long-read sequencing technology that can handle regions at least as large as the segmental duplication of interest (e.g. PacBio up to ~15kb, Nanopore for anything longer), so that you have a high chance of mapping reads uniquely.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-05T04:46:53.230" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16312" PostTypeId="2" ParentId="16307" CreationDate="2021-07-05T12:50:41.890" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @jan-wilamowski, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;That second paper seems to distinguish between the known data that is used for ML training (i.e. amino acid sequences are assigned some secondary structure) and the attempt to replicate this natural assignment by an &amp;quot;artificial&amp;quot; prediction.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-05T12:50:41.890" CommentCount="0" CommunityOwnedDate="2021-07-05T12:50:41.890" ContentLicense="CC BY-SA 4.0" />
  <row Id="16313" PostTypeId="2" ParentId="16307" CreationDate="2021-07-05T14:16:29.310" Score="7" Body="&lt;p&gt;To make it crystal clear (and make than pun):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;Assignment&lt;/code&gt;: a 3D structure is known and the residues are &lt;em&gt;assigned&lt;/em&gt; a secondary structure&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;Prediction&lt;/code&gt;: an algorithm predicts from linear (primary) sequence what the secondary structure may be —often incorrectly. This can be homology based, covariance, deep learning etc. But most commonly just the vector norm of window residue composition, e.g. proline is never in a helix and helices and sheets have different periodicity (two for a sheet and three and a bit for a helix).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-07-05T14:16:29.310" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16314" PostTypeId="1" AcceptedAnswerId="16326" CreationDate="2021-07-05T21:53:28.770" Score="1" ViewCount="149" Body="&lt;p&gt;I got this question as a homework. Does anyone know how the it can be solved?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Consider a short string &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; as a read from a string &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt; is converted from a substring of &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; by at most &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; swaps at random locations, where a swap is an operation that switch two adjacent characters, i.e., string &lt;span class=&quot;math-container&quot;&gt;$s_1s_2...s_i s_{i+1}...s_n$&lt;/span&gt; is one swap away from the string &lt;span class=&quot;math-container&quot;&gt;$s_1s_2...s_{i+1} s_{i}...s_n$&lt;/span&gt;. Two reads are called overlapping if the substrings to which they are converted are overlapping. Given the input of &lt;span class=&quot;math-container&quot;&gt;$k &amp;lt;&amp;lt; |r|$&lt;/span&gt;, devise an algorithm to reconstruct a target string &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; from a large set of overlapping reads of &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The question doesn't explicitly mention which of the possible superstrings of reads should be returned, so I think it's asking for &lt;em&gt;any&lt;/em&gt; superstring.&lt;/p&gt;&#xA;" OwnerUserId="12692" LastEditorUserId="73" LastEditDate="2021-07-06T20:51:24.323" LastActivityDate="2021-07-08T03:41:23.213" Title="How to reconstruct a target string from a large set of overlapping reads" Tags="|algorithms|homework|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16315" PostTypeId="2" ParentId="16112" CreationDate="2021-07-06T00:00:38.343" Score="1" Body="&lt;p&gt;For single-end reads, generically:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ bam2bed &amp;lt; reads.bam &amp;gt; reads.bed&#xA;$ bedmap --echo --fraction-ref 1 reads.bed roi.bed &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The BED file &lt;code&gt;answer.bed&lt;/code&gt; contains filtered reads which entirely overlap intervals in &lt;code&gt;roi.bed&lt;/code&gt;. It contains fields required for backconversion to SAM/BAM.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2021-07-06T00:00:38.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16316" PostTypeId="2" ParentId="16287" CreationDate="2021-07-06T15:57:44.223" Score="1" Body="&lt;p&gt;I'd probably just use &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/fgsea.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;fgsea&lt;/a&gt;. Your code won't have to change much, it also just uses a ranked list.&lt;/p&gt;&#xA;" OwnerUserId="1269" LastActivityDate="2021-07-06T15:57:44.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16317" PostTypeId="1" AcceptedAnswerId="16319" CreationDate="2021-07-06T16:23:22.507" Score="1" ViewCount="437" Body="&lt;p&gt;I recently got fastQ files from a MinION sequencer. I would like to demultiplex these files.&lt;/p&gt;&#xA;&lt;p&gt;Is there any other software to demultiplex (no porechop) in or outside Artic protocol? I would like to learn options different from porechop.&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="12707" LastActivityDate="2021-07-06T20:44:45.397" Title="How to demultiplex MinION fastQ files without Porechop?" Tags="|r|minion|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16318" PostTypeId="1" CreationDate="2021-07-06T17:07:39.837" Score="2" ViewCount="58" Body="&lt;p&gt;I am currently doing a project regarding gene-based analysis and gene-set analysis given a certain GWAS dataset, I only know 2 well known gene-based analysis tool which is &lt;a href=&quot;https://github.com/bulik/ldsc&quot; rel=&quot;nofollow noreferrer&quot;&gt;LDSC&lt;/a&gt; and &lt;a href=&quot;https://ctg.cncr.nl/software/magma&quot; rel=&quot;nofollow noreferrer&quot;&gt;MAGMA&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As of now, I'm familiar with how MAGMA works, but not really sure about LDSC, I did read LDSC Wiki briefly, and I am leaning towards MAGMA atm, but it's probably just because I'm more familiar with MAGMA.&lt;/p&gt;&#xA;&lt;p&gt;So, which one do you think is more robust? or at least what is the pros and cons of each software?&lt;/p&gt;&#xA;" OwnerUserId="12708" LastEditorUserId="57" LastEditDate="2021-07-10T18:07:36.583" LastActivityDate="2021-07-10T18:07:36.583" Title="Which GWGAS Tool is 'better', MAGMA or LDSC?" Tags="|gene|software-recommendation|gwas|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16319" PostTypeId="2" ParentId="16317" CreationDate="2021-07-06T17:09:14.293" Score="2" Body="&lt;p&gt;The Guppy basecaller itself now has the ability to internally perform multiplexing. That is currently the preferred option.&lt;/p&gt;&#xA;&lt;p&gt;If for some reason, you still want a stand-alone application to demultiplex already basecalled reads, then &lt;code&gt;qcat&lt;/code&gt; is an option: &lt;a href=&quot;https://github.com/nanoporetech/qcat&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nanoporetech/qcat&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6332" LastActivityDate="2021-07-06T17:09:14.293" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16320" PostTypeId="2" ParentId="16317" CreationDate="2021-07-06T20:44:45.397" Score="1" Body="&lt;p&gt;I have written up my own semi-manual method for demultiplexing Nanopore reads. As distinct from the other methods that are commonly used, it searches the &lt;em&gt;entire&lt;/em&gt; read for potential barcode sequences, and splits out identified chimeric reads (containing multiple barcode sequences) into a separate file:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://dx.doi.org/10.17504/protocols.io.7vmhn46&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://dx.doi.org/10.17504/protocols.io.7vmhn46&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-06T20:44:45.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16321" PostTypeId="1" AcceptedAnswerId="16337" CreationDate="2021-07-07T08:37:03.180" Score="2" ViewCount="2347" Body="&lt;p&gt;I have a Seurat object of 8 patients. I want to add metadata to that so that I have origin of each cell. At the moment UMAP just shows a bunch of cells while I want to color clusters by sample&lt;/p&gt;&#xA;&lt;p&gt;This is my metadata&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; str(head(metadata))&#xA;'data.frame':   6 obs. of  3 variables:&#xA; &lt;span class=&quot;math-container&quot;&gt;$ barcodes: chr  &quot;AAACAGCCAAACTGCC-5&quot; &quot;AAACAGCCAACAGGAT-1&quot; &quot;AAACAGCCAAGATTCT-1&quot; &quot;AAACAGCCAAGGTATA-1&quot; ...&#xA; $&lt;/span&gt; libcodes: Factor w/ 8 levels &amp;quot;1&amp;quot;,&amp;quot;2&amp;quot;,&amp;quot;3&amp;quot;,&amp;quot;4&amp;quot;,..: 5 1 1 1 5 7&#xA; $ samples : chr  &amp;quot;PN0252_0005&amp;quot; &amp;quot;PN0252_0001&amp;quot; &amp;quot;PN0252_0001&amp;quot; &amp;quot;PN0252_0001&amp;quot; ...&#xA;&amp;gt; &#xA;&#xA;&amp;gt; pbmc&#xA;An object of class Seurat &#xA;36601 features across 26473 samples within 1 assay &#xA;Active assay: RNA (36601 features, 0 variable features)&#xA;&amp;gt; &#xA;&#xA;&amp;gt; dim(metadata)&#xA;[1] 26473     3&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2021-07-07T08:48:06.080" LastActivityDate="2023-09-16T22:08:18.357" Title="Adding metadata to Seurat object" Tags="|scrnaseq|seurat|10x-genomics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16322" PostTypeId="2" ParentId="15740" CreationDate="2021-07-07T15:24:11.227" Score="1" Body="&lt;p&gt;Below is a reproducible example. I tried to simulate the same channels using &lt;code&gt;chr1&lt;/code&gt; as an example.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-java prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env nextflow&#xA;&#xA;// Simulate the chunk channel&#xA;channel&#xA;    .fromPath('chunk.csv')&#xA;    .splitCsv(header:true)&#xA;    .map{row -&amp;gt; tuple(row.chrom,row.chunks)}&#xA;    .set{chunk_ch}&#xA;&#xA;// Simulate chr and chr_path as a single entry list&#xA;channel.from([&amp;quot;chr1&amp;quot;, &amp;quot;chr1.txt&amp;quot;]).toList().map{tuple(it[0], it[1])}.set{shape_in}&#xA;&#xA;// uncomment to view the content&#xA;// shape_in.view()&#xA;&#xA;// process of phased_ch. I modified it to output the chr and chr_path as given in your example&#xA;process phased_ch {&#xA;  echo true&#xA;  input:&#xA;  tuple val(chr), val(chr_path) from shape_in&#xA;  output:&#xA;  tuple val(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chr}&quot;), path(&quot;$&lt;/span&gt;{chr_path}&amp;quot;) into phased&#xA;  script:&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  echo &lt;span class=&quot;math-container&quot;&gt;${chr}&#xA;  touch $&lt;/span&gt;{chr_path}&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;// joining the two channels to produce the correct join&#xA;chunk_ch.join(phased).view()&#xA;&#xA;// will produce &#xA;// [chr1, &amp;quot;CHUNKS=([1]=&amp;quot;&amp;quot;0e6 3e6&amp;quot;&amp;quot; [2]=....)&amp;quot;, /media/Documents/Nextflow/work/8e/2fcd9a00b04f0f5d44711068d68a0f/chr1.txt]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;chunk.csv&lt;/code&gt; contains&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chrom,chunks&#xA;chr1,&amp;quot;CHUNKS=([1]=&amp;quot;&amp;quot;0e6 3e6&amp;quot;&amp;quot; [2]=....)&amp;quot;&#xA;chr2,&amp;quot;CHUNKS=([1]=&amp;quot;&amp;quot;0e6 3e6&amp;quot;&amp;quot; [2]=....)&amp;quot;&#xA;chr3,&amp;quot;CHUNKS=([1]=&amp;quot;&amp;quot;0e6 3e6&amp;quot;&amp;quot; [2]=....)&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In your case, you should verify what are the contents of &lt;code&gt;chunk_ch&lt;/code&gt; and &lt;code&gt;phased&lt;/code&gt; on output. The first key of &lt;code&gt;join()&lt;/code&gt; should be similar otherwise it won't give anything. From the documentation of &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#join&quot; rel=&quot;nofollow noreferrer&quot;&gt;join&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The join operator creates a channel that joins together the items&#xA;emitted by two channels for which exists a matching key. The key is&#xA;defined, by default, as the first element in each item emitted.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2021-07-07T15:24:11.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16323" PostTypeId="2" ParentId="16192" CreationDate="2021-07-07T15:34:13.817" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;NB. This is a long comment as opposed to an answer which requires a colossal working chunk of code.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h2&gt;Control&lt;/h2&gt;&#xA;&lt;p&gt;Science is all about controls failing. A great control for docking is the barnase-barnstar complex (PDB:1BRS): both small, but form a strong interactions. If this is done, the RMSD against the original structure is very meaningful and useful.&lt;/p&gt;&#xA;&lt;h2&gt;RMSF&lt;/h2&gt;&#xA;&lt;p&gt;A RMSD is a metric of how much two structures differ. In an ensemble, the metric of how much a structure and the ensemble average is called a RMSF (root-mean square fluctuation). Here &lt;em&gt;the RMSF is sought&lt;/em&gt; as the RMSD from first/top pose or worse the manually thrown together pre-docking pose does not mean much.&lt;/p&gt;&#xA;&lt;h2&gt;Clusters&lt;/h2&gt;&#xA;&lt;p&gt;The decoys (PBDs trails) need to be clustered into different ensembles first, before a RMSF can be calculated.&lt;/p&gt;&#xA;&lt;h2&gt;∆∆G and water&lt;/h2&gt;&#xA;&lt;p&gt;For the calculation of difference in Gibbs free energy between the docked minus the docked poses, the structures need to be energy minimised with the forcefield scorefunction that is to be used for scoring.&#xA;Water is very important for protein-protein interactions, but adds a lot of complication. Say Gromacs is used, &lt;code&gt;gmx energy&lt;/code&gt; will return the single state (conformer) Gibbs free energy, but it requires a water box. For the difference, the water box of undocked needs to be the same size and preferably with the same amount of molecules. A much simpler solution is using an implicit solvent model, e.g. Rosetta. As a bonus, the energy minimisation (&lt;code&gt;relax&lt;/code&gt;) is relative fast. And specific explicit waters can be added. However, calculating ∆∆G for every decoy will take hundreds of CPU-hours and is a bit nonsensical.&lt;/p&gt;&#xA;&lt;p&gt;A better approach is to look at residue conservation (&lt;a href=&quot;https://consurf.tau.ac.il/&quot; rel=&quot;nofollow noreferrer&quot;&gt;consurf&lt;/a&gt;), cross-linking mass spec data if available, and if human post-translational modifications (&lt;a href=&quot;https://www.phosphosite.org/homeAction.action&quot; rel=&quot;nofollow noreferrer&quot;&gt;phosphosite&lt;/a&gt;) and healthy variants (&lt;a href=&quot;https://gnomad.broadinstitute.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gnomAD&lt;/a&gt;). But ultimately wet lab validation by ITC of different mutants/labelling based upon the docking results &lt;em&gt;etc.&lt;/em&gt; is required.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-07-07T15:34:13.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16325" PostTypeId="2" ParentId="13531" CreationDate="2021-07-08T03:32:18.667" Score="0" Body="&lt;p&gt;I'm having a bit of trouble understanding the question, but we've used DESeq2 for doing both between-sample and between-transcript comparisons.&lt;/p&gt;&#xA;&lt;p&gt;When comparing transcripts for short reads I use the variant-stabilised transformation, adjusted for transcript length. This is good for a visual overview of results (e.g. in a heatmap).&lt;/p&gt;&#xA;&lt;p&gt;When comparing samples I use the variance-adjusted DESeq2 test, producing normalised &lt;span class=&quot;math-container&quot;&gt;$\textrm{log}_2FC$&lt;/span&gt; values. This works well for quantitative results.&lt;/p&gt;&#xA;&lt;p&gt;See &lt;a href=&quot;https://doi.org/10.1084/jem.20160470&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, where we use both of these approaches (e.g. Figure 2 shows both in a side-by-side view).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-08T03:32:18.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16326" PostTypeId="2" ParentId="16314" CreationDate="2021-07-08T03:41:23.213" Score="2" Body="&lt;p&gt;Unless it's not obvious, the problem has no unique solution. An obvious example is a string which uses entirely one character; the problem is then to determine its &lt;em&gt;length&lt;/em&gt;, which you can only do statistically.&lt;/p&gt;&#xA;&lt;p&gt;As you've correctly surmised, this is more or less the short-read &lt;em&gt;de novo&lt;/em&gt; assembly problem, with one modification: the &amp;quot;read errors&amp;quot; are all swaps.&lt;/p&gt;&#xA;&lt;p&gt;The usual way that we solve this problem is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Pick a length (which is customarily called &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; in bioinformatics, but the problem confusingly already used that, so we'll use &lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt; instead), and extract all &lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt;-length substrings of the reads. These substrings will, from here on, be referred to as &lt;a href=&quot;https://en.wikipedia.org/wiki/K-mer&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt;-mers&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Construct a subgraph of the &lt;a href=&quot;https://en.wikipedia.org/wiki/De_Bruijn_graph&quot; rel=&quot;nofollow noreferrer&quot;&gt;De Bruijn graph&lt;/a&gt;, where the &lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt;-mers are the nodes, and there is an edge between two nodes if two adjacent &lt;span class=&quot;math-container&quot;&gt;$\kappa$&lt;/span&gt;-mers are connected in any read. Label the edges with the number of times this happens in all reads.&lt;/li&gt;&#xA;&lt;li&gt;Prune the graph of any edges that have low frequency. In your example, the kinds of errors are known, so you could in fact &amp;quot;correct&amp;quot; the pruned edges, or even correct the reads, based on the higher-consensus path.&lt;/li&gt;&#xA;&lt;li&gt;An &lt;a href=&quot;https://en.wikipedia.org/wiki/Eulerian_path&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eulerian superpath&lt;/a&gt; through the graph corresponds to the desired string.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I'm not saying that this is the best approach for your homework question, but this is how we do it in practice.&lt;/p&gt;&#xA;&lt;p&gt;To understand the method in depth, here are some suitable references. Some of these are tool papers, and others explain the data structures and algorithms well.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Pevzner et al, &lt;a href=&quot;https://www.pnas.org/content/98/17/9748&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;An Eulerian path approach to DNA fragment assembly&lt;/em&gt;&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Zerbino and Birney, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2336801/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;Velvet: Algorithms for de novo short read assembly using de Bruijn graphs&lt;/em&gt;&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Sohn and Nam, &lt;a href=&quot;https://academic.oup.com/bib/article/19/1/23/2339783&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;The present and future of de novo whole-genome assembly&lt;/em&gt;&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9536" LastActivityDate="2021-07-08T03:41:23.213" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16327" PostTypeId="1" CreationDate="2021-07-08T06:20:55.717" Score="0" ViewCount="88" Body="&lt;p&gt;I have the following docking results:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Dock_1&lt;/code&gt; has better energy score than &lt;code&gt;Dock_2&lt;/code&gt;.&#xA;What I want to do is to show with Pymol that &lt;code&gt;Dock_1&lt;/code&gt; is better visually.&lt;/p&gt;&#xA;&lt;p&gt;Here I use hydrogen bond, highlighted with Yellow (&lt;code&gt;Dock_1&lt;/code&gt;) and Gray (&lt;code&gt;Dock_2&lt;/code&gt;).&#xA;But as you can see, with that I can't show that &lt;code&gt;Dock_1&lt;/code&gt; is clearly better.&#xA;That's why I'm looking for better alternative than hydrogen bond.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/E1YcH.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/E1YcH.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6199" LastEditorUserId="12054" LastEditDate="2021-07-09T11:54:09.557" LastActivityDate="2021-07-09T11:54:09.557" Title="How to show that one docking is better than another with Pymol visually" Tags="|protein-structure|pdb|docking|pymol|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16328" PostTypeId="2" ParentId="16327" CreationDate="2021-07-08T09:13:07.757" Score="1" Body="&lt;p&gt;maybe you can use the &amp;quot;&lt;strong&gt;per-residue interaction scores&lt;/strong&gt;&amp;quot; (wich is available via Schrödinger).&lt;/p&gt;&#xA;&lt;p&gt;On the other hand,to show if a Docking Pose is really &amp;quot;better&amp;quot; you could show specific interactions rather than the complete big molecule.. e.g. an important salt bridge is missing. (hydrogen bonds are not that important, due to the fact that they can be bridged with water molecules)&lt;/p&gt;&#xA;&lt;p&gt;Sorry, this is not really a solution, but I am not allowed to comment jet -.-&lt;/p&gt;&#xA;" OwnerUserId="12181" LastActivityDate="2021-07-08T09:13:07.757" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="16331" PostTypeId="1" CreationDate="2021-07-08T12:44:57.333" Score="2" ViewCount="774" Body="&lt;p&gt;I'm working on gut microbiota data. I want to perform a biplot showing my 2 enterotypes and the major OTU explaining the variance between my two enterotypes.&lt;/p&gt;&#xA;&lt;p&gt;Something like : &lt;a href=&quot;https://i.stack.imgur.com/7sz8B.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7sz8B.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt; from &lt;a href=&quot;https://doi.org/10.1038/srep08397&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1038/srep08397&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm using the package ade4 to plot the enterotypes which gives me a nice plot but I can't find how to add the OTU loading&lt;/p&gt;&#xA;&lt;p&gt;My script :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(ade4)&#xA;#dist_jsd_3a = distance&#xA;#sample_data(physeq_genus_3a_epi)&lt;span class=&quot;math-container&quot;&gt;$cluster_3a_fullname) = enterotypes&#xA;#obs.pcoa$&lt;/span&gt;li = pcoa result from dudi.pco function&#xA;&#xA;obs.pcoa=dudi.pco(cailliez(dist_jsd_3a), scannf=F, nf=2)&#xA;s.class(obs.pcoa&lt;span class=&quot;math-container&quot;&gt;$li, fac=as.factor(sample_data(physeq_genus_3a_epi)$&lt;/span&gt;cluster_3a_fullname), grid=F, col = c(&amp;quot;#0073C2FF&amp;quot;,&amp;quot;#EFC000FF&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;this code gives me this plot : &lt;a href=&quot;https://i.stack.imgur.com/RqACP.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RqACP.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I also run this following code using the envfit function from vegan package which allowed me to plot the OTU but the resulting plot is not as pleasant as the one above with the s.class function from ade4 package.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(vegan)&#xA;#dist_jsd_3a = distance&#xA;#groups = enterotypes&#xA;#pcoa = pcoa result from cmdscale(dis)&#xA;&#xA;dis&amp;lt;-dist_jsd_3a&#xA;groups &amp;lt;- sample_data(physeq_genus_3a_epi)$cluster_3a&#xA;pcoa &amp;lt;- cmdscale(dis)&#xA;efit &amp;lt;- envfit(pcoa, t(otu_table(physeq_genus_3a_epi)))&#xA;plot(pcoa, col = c(&amp;quot;#0073C2FF&amp;quot;, &amp;quot;#EFC000FF&amp;quot;)[groups], pch = c(21,24)[groups])&#xA;##Ordiellipse##&#xA;## I use confidence level of 0.95) ##&#xA;ordiellipse(pcoa, groups, col = c(&amp;quot;#0073C2FF&amp;quot;, &amp;quot;#EFC000FF&amp;quot;),&#xA;            kind=&amp;quot;sd&amp;quot;,lty=c(1), conf = 0.95, alpha = 0.05, lwd = 2)&#xA;&#xA;plot(efit, col = &amp;quot;red&amp;quot;, cex = 0.9)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;See the result here : &lt;a href=&quot;https://i.stack.imgur.com/SqcpL.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SqcpL.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want to know if anyone already plotted a biplot using ade4 package and have been able to add arrows for OTU. If yes, any tips would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="12549" LastActivityDate="2021-07-08T12:44:57.333" Title="How to plot a PCOA biplot with OTU loadings as arrows" Tags="|r|microbiome|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16332" PostTypeId="1" AcceptedAnswerId="16333" CreationDate="2021-07-08T19:56:33.760" Score="2" ViewCount="2217" Body="&lt;p&gt;I have a Seurat object&lt;/p&gt;&#xA;&lt;p&gt;I want to have both cluster numbers and coloured cells by sample names like this figure (from a Nature paper)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ENABR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ENABR.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have tried &lt;code&gt;group.by&lt;/code&gt; argument in Seurat but only samples come not cluster numbers like below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; DimPlot(pbmc, reduction = &amp;quot;umap&amp;quot;,group.by = &amp;quot;samples&amp;quot;)&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;[![enter image description here][2]][2]&lt;/p&gt;&#xA;&lt;p&gt;Any help that to get something like the first figure? Having cluster numbers and coloured by sample names please&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="9449" LastEditDate="2021-07-27T10:45:44.943" LastActivityDate="2021-07-27T10:45:44.943" Title="color by clusters and sampled in Seurat" Tags="|scrnaseq|seurat|10x-genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16333" PostTypeId="2" ParentId="16332" CreationDate="2021-07-08T20:20:20.187" Score="2" Body="&lt;p&gt;Here is a solution that makes use of &lt;code&gt;LabelClusters()&lt;/code&gt; from &lt;code&gt;Seurat&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# creating a plot and assigning it to the &amp;quot;plot&amp;quot; variable&#xA;# coloring by sample id and turning labels off&#xA;plot &amp;lt;- &#xA;DimPlot(&#xA;  pbmc,&#xA;  reduction = &amp;quot;umap&amp;quot;,&#xA;  group.by = &amp;quot;some_random_sample_id&amp;quot;,&#xA;  label = FALSE,&#xA;)&#xA;&#xA;# here is the trick, adding cluster labels to the &amp;quot;data&amp;quot; variable of the &amp;quot;plot&amp;quot;&#xA;# without this, the &amp;quot;plot&amp;quot; object would not have the clustering information but only embeddings and sample id&#xA;plot&lt;span class=&quot;math-container&quot;&gt;$data$&lt;/span&gt;seurat_clusters&amp;lt;- pbmc@meta.data$seurat_clusters&#xA;&#xA;# and here is a sanity check to see if cluster labels are transferred to the &amp;quot;right places&amp;quot;&#xA;all(rownames(plot$data) == rownames(pbmc@meta.data))&#xA;&#xA;LabelClusters(plot, id = &amp;quot;seurat_clusters&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/A65yx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/A65yx.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2021-07-10T22:15:12.437" LastActivityDate="2021-07-10T22:15:12.437" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16334" PostTypeId="2" ParentId="5315" CreationDate="2021-07-08T21:30:45.047" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @fridaymeetssunday, converted from comments:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;If python-reading is correct, it looks like that is an internally defined module (class) which is in &lt;a href=&quot;https://github.com/DovetailGenomics/HiRise_July2015_GR/blob/3cdda0a1a44585b52274c01d7e9e02268e2d337f/scripts/bamtags.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. So basically, it's there.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Response from poster:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thank you, guys, so helpful! Yes, that's it. I wasn't aware of the simplicity of import function. I thought it only worked with builtin or pip/conda installed modules. I just had to add the scripts folder to my PATH variable or have the functions in the working directory.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-08T21:30:45.047" CommentCount="0" CommunityOwnedDate="2021-07-08T21:30:45.047" ContentLicense="CC BY-SA 4.0" />
  <row Id="16335" PostTypeId="2" ParentId="5277" CreationDate="2021-07-08T21:32:53.707" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @llrs, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Perhaps I'm not understanding correctly what you are doing. But these &lt;a href=&quot;https://stats.stackexchange.com/a/11702/105234&quot;&gt;stability tests&lt;/a&gt; are used when the number of clusters is unknown, so you test the resolution, or other metrics for a range of numbers of clusters. Then you choose the best one (either the one that is better by the score, or the one that increasing the number of clusters don't improve your metric much more)&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-08T21:32:53.707" CommentCount="0" CommunityOwnedDate="2021-07-08T21:32:53.707" ContentLicense="CC BY-SA 4.0" />
  <row Id="16336" PostTypeId="2" ParentId="16321" CreationDate="2021-07-08T21:35:48.810" Score="2" Body="&lt;p&gt;&lt;em&gt;Answer from @user438383, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Did you try &lt;a href=&quot;https://www.rdocumentation.org/packages/Seurat/versions/3.1.4/topics/AddMetaData&quot; rel=&quot;nofollow noreferrer&quot;&gt;the AddMetaData function&lt;/a&gt;? &lt;a href=&quot;https://stackoverflow.com/questions/42279766/add-metadata-to-seurat-object&quot;&gt;This post&lt;/a&gt; may also be useful.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cluster_letters &amp;lt;- LETTERS[Idents(object = pbmc_small)]&#xA;names(cluster_letters) &amp;lt;- colnames(x = pbmc_small)&#xA;pbmc_small &amp;lt;- AddMetaData(&#xA;  object = pbmc_small,&#xA;  metadata = cluster_letters,&#xA;  col.name = 'letter.idents'&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-09-16T22:08:18.357" LastActivityDate="2023-09-16T22:08:18.357" CommentCount="0" CommunityOwnedDate="2021-07-08T21:35:48.810" ContentLicense="CC BY-SA 4.0" />
  <row Id="16337" PostTypeId="2" ParentId="16321" CreationDate="2021-07-09T02:44:28.763" Score="2" Body="&lt;p&gt;If you have single-dimension per-cell metadata, and it's arranged identically to the cell order in the Seurat object, I find it easier to use the double bracket notation to add metadata to a Seurat object. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;metadata&lt;span class=&quot;math-container&quot;&gt;$barcodes -&amp;gt; pbmc[[&quot;barcodes&quot;]]&#xA;metadata$&lt;/span&gt;libcodes -&amp;gt; pbmc[[&amp;quot;libcodes&amp;quot;]]&#xA;metadata$samples -&amp;gt; pbmc[[&amp;quot;samples&amp;quot;]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-09T02:44:28.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16338" PostTypeId="1" AcceptedAnswerId="16339" CreationDate="2021-07-09T06:33:30.783" Score="0" ViewCount="47" Body="&lt;p&gt;I want to get all the variants (SNPs) in a given region of the human genome. Is there a way on how to do this with the ensembl-API? It should be a representation similar to this: &lt;a href=&quot;https://i.stack.imgur.com/HChBc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HChBc.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There I added dbSNP to the region details. Having this data in a JSON or XML-Format would be really helpful! &lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Location/View?db=core;r=11:108290588-108290689;source=dbSNP;v=rs1491549802;vdb=variation;vf=595960779;time=1625753581&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt;'s the link to the image.&lt;/p&gt;&#xA;&lt;p&gt;Edit: I know that the export-function on ensembl is a thing. But then I only get the rs-Numbers. To get all the info I need I would have to run (potentially thousands) all the rs-Numbers through some additional API. Getting that info in bulk right away would be a huge time-saver.&lt;/p&gt;&#xA;" OwnerUserId="9131" LastEditorUserId="9131" LastEditDate="2021-07-09T07:19:02.460" LastActivityDate="2021-07-09T07:19:21.663" Title="Getting all variants in a region from ensembl" Tags="|ensembl|variation|api|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16339" PostTypeId="2" ParentId="16338" CreationDate="2021-07-09T07:19:21.663" Score="1" Body="&lt;p&gt;&lt;a href=&quot;http://rest.ensembl.org/documentation/info/overlap_region&quot; rel=&quot;nofollow noreferrer&quot;&gt;REST API overlap region endpoint&lt;/a&gt;. &lt;a href=&quot;https://rest.ensembl.org/overlap/region/human/11:108290588-108290689?feature=variation;content-type=application/json&quot; rel=&quot;nofollow noreferrer&quot;&gt;For example&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2021-07-09T07:19:21.663" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16341" PostTypeId="1" AcceptedAnswerId="16396" CreationDate="2021-07-09T07:34:29.563" Score="1" ViewCount="276" Body="&lt;p&gt;The following is a valid .fasta file content:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;&amp;gt;HSBGPG Human gene for bone gla protein (BGP) GGCAGATTCCCCCTAGACCCGCCCGCACCATGGTCAGGCATGCCCCTCCTCATCGCTGGGCACAGCCCAGAGGGT ATAAACAGTGCTGGAGGCTGGCGGGGCAGGCCAGCTGAGTCCTGAGCAGCAGCCCAGCGCAGCCACCGAGACACC ATGAG.&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is this also?&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;&amp;gt;Arbirary_Name_iJustCameUP_with_and_other_local_identifiers GGCAGATTCCCCCTAGACCCGCCCGCACCATGGTCAGGCATGCCCCTCCTCATCGCTGGGCACAGCCCAGAGGGT ATAAACAGTGCTGGAGGCTGGCGGGGCAGGCCAGCTGAGTCCTGAGCAGCAGCCCAGCGCAGCCACCGAGACACC ATGAG.&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="6511" LastActivityDate="2021-07-15T01:57:47.940" Title="Can the header line contain arbitrary text in FASTA format?" Tags="|fasta|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16342" PostTypeId="2" ParentId="13531" CreationDate="2021-07-09T10:01:47.310" Score="0" Body="&lt;p&gt;As mentioned by @gringer DESeq2 is a popular choice for normalization of counts between samples. However, because each gene is adjusted individually it is not well suited to compare within a sample.&lt;br /&gt;&#xA;For this a global normalization like &lt;code&gt;TMM&lt;/code&gt; from &lt;code&gt;edgeR&lt;/code&gt; would be more appropriate.&lt;/p&gt;&#xA;&lt;p&gt;You can find more information in this &lt;a href=&quot;https://hbctraining.github.io/DGE_workshop_salmon_online/lessons/02_DGE_count_normalization.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; from the hbc at Harvard.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2021-07-09T10:01:47.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16343" PostTypeId="2" ParentId="16341" CreationDate="2021-07-09T10:10:51.397" Score="4" Body="&lt;p&gt;I am not aware of an official fasta format description. &lt;strike&gt;The only constraint that I know of is that there should not be any whitespace after &lt;code&gt;&amp;gt;&lt;/code&gt;&lt;/strike&gt;. Apart from that particular position, whitespaces are allowed in the header line.&lt;/p&gt;&#xA;&lt;p&gt;So, regarding your particular example, both are valid fasta sequences. When in doubt, you can use &lt;a href=&quot;https://biopython.org/wiki/SeqIO&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;SeqIO&lt;/code&gt;&lt;/a&gt; from &lt;code&gt;Biopython&lt;/code&gt;, if you can parse your file with the following code, it &lt;strike&gt; is&lt;/strike&gt; should be a valid fasta file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;with open(&amp;quot;example.fasta&amp;quot;) as handle:&#xA;    for record in SeqIO.parse(handle, &amp;quot;fasta&amp;quot;):&#xA;        print(record.id)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Edit per @Chris_Rands' comment&lt;/p&gt;&#xA;&lt;p&gt;The code below does the same as above, meaning that &lt;code&gt;SeqIO.parse()&lt;/code&gt; takes care of opening and closing the file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;for record in SeqIO.parse(&amp;quot;example.fasta&amp;quot;, &amp;quot;fasta&amp;quot;):&#xA;    print(record.id)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2021-07-12T18:32:49.510" LastActivityDate="2021-07-12T18:32:49.510" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="16344" PostTypeId="2" ParentId="2068" CreationDate="2021-07-09T10:13:02.447" Score="0" Body="&lt;p&gt;I'm quite sure it will work with &lt;code&gt;agat_convert_sp_gff2gtf.pl&lt;/code&gt; from &lt;a href=&quot;https://github.com/NBISweden/AGAT&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGAT&lt;/a&gt;.&lt;br /&gt;&#xA;See the review of the different solution I made here:&#xA;&lt;a href=&quot;https://agat.readthedocs.io/en/latest/gff_to_gtf.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://agat.readthedocs.io/en/latest/gff_to_gtf.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;AGAT it the only one you can decide wich version of GTF you require as output.&lt;/p&gt;&#xA;" OwnerUserId="2486" LastActivityDate="2021-07-09T10:13:02.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16345" PostTypeId="1" CreationDate="2021-07-09T10:45:36.803" Score="1" ViewCount="60" Body="&lt;p&gt;I'm using plink (1.9b5) to do allelic association. My problem is that my output does not write the &lt;code&gt;rsID&lt;/code&gt; (SNP), it just writes a dot.&lt;/p&gt;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CHR SNP  BP   A1  F_A      F_U      A2  P       OR           &#xA;1   .    1014 A   0.18     0.07     C   0.22    3.03&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My script:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plink --bed file.bed --bim file.bim --fam file.fam --make-bed --out output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can someone please help me see what I'm doing wrong?&lt;/p&gt;&#xA;" OwnerUserId="12734" LastEditorUserId="57" LastEditDate="2021-07-10T18:04:42.303" LastActivityDate="2022-12-05T10:06:05.897" Title="Output of allelic association doesn't write the rsID" Tags="|snp|phylogenetics|ngs|plink|" AnswerCount="1" CommentCount="1" ClosedDate="2022-12-05T16:28:28.263" ContentLicense="CC BY-SA 4.0" />
  <row Id="16346" PostTypeId="2" ParentId="16276" CreationDate="2021-07-09T12:34:10.053" Score="2" Body="&lt;p&gt;You might be luckier by submitting to INSDC by the ENA gate instead of NCBI.&lt;/p&gt;&#xA;&lt;p&gt;To do so you first have to convert your GFF into the EMBL format used by ENA with the help of &lt;a href=&quot;https://github.com/NBISweden/EMBLmyGFF3&quot; rel=&quot;nofollow noreferrer&quot;&gt;EMBLmyGFF3&lt;/a&gt;. During the conversion you can use the &lt;code&gt;--tranlate&lt;/code&gt; option that can help to see if there is anything wrong in the ORFs (e.g. frameshifts).&lt;/p&gt;&#xA;&lt;p&gt;Incomplete genes is supposed to be supported by INSDC databases, so I don't get why NCBI complains about it. Either their pipeline does not take care of this, or it is something specific to Mitochondrion annotations. Using &lt;code&gt;EMBLmyGFF3&lt;/code&gt; incomplete CDS will be handled properly and the location field in the resulting EMBL flat file will have the information that it is incomplete (see &lt;code&gt;3.4.3 Location examples&lt;/code&gt; in the documentation &lt;a href=&quot;http://ftp.ebi.ac.uk/pub/databases/embl/doc//FT_current.html#3.4&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ftp.ebi.ac.uk/pub/databases/embl/doc//FT_current.html#3.4&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;Then once you have your EMBL flat file you can check by yourself if there is anything not accepted by the DB using the ENA webin-cli (&lt;a href=&quot;https://github.com/enasequence/webin-cli/releases/tag/v3.7.0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/enasequence/webin-cli/releases/tag/v3.7.0&lt;/a&gt;)&#xA;It there are errors they will be listed with details and you will have to fix your file accordingly. When everything is fixed you can submit to ENA. Once accepted by ENA it will be within 24 hours within NCBI.&lt;/p&gt;&#xA;" OwnerUserId="2486" LastActivityDate="2021-07-09T12:34:10.053" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16347" PostTypeId="2" ParentId="13531" CreationDate="2021-07-09T13:03:48.607" Score="0" Body="&lt;p&gt;I found a recent technique that allows for both, called &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-018-2246-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;geTMM&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2021-07-09T13:03:48.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16348" PostTypeId="1" CreationDate="2021-07-09T18:16:45.013" Score="3" ViewCount="99" Body="&lt;p&gt;I need to retrieve COVID-19 metadata (such as date of collection of sequence data, location, etc.) for a batch of 3953 GISAID accessions from the GISAID database.&lt;/p&gt;&#xA;&lt;p&gt;I am not very comfortable with GISAID. I would prefer an NCBI database solution.&lt;/p&gt;&#xA;&lt;p&gt;Any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="12736" LastEditorUserId="3848" LastEditDate="2022-02-25T19:33:18.533" LastActivityDate="2022-02-25T19:36:28.183" Title="Retrieval of metadata of a batch of 3953 sequence data from GISAID database" Tags="|sequence-analysis|public-databases|phylogenetics|covid-19|sars-cov-2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16349" PostTypeId="2" ParentId="3169" CreationDate="2021-07-09T18:57:40.437" Score="2" Body="&lt;p&gt;I had the same problem. I'm pretty sure you can calculate it yourself using the following:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The length of your sequence (seq_len)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The value from the &lt;code&gt;&amp;lt;Hsp_query-from&amp;gt;&lt;/code&gt; tag from the hit (&lt;code&gt;&amp;lt;Hsp_query-from&amp;gt;&lt;/code&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The value from the &lt;code&gt;&amp;lt;Hsp_query-to&amp;gt;&lt;/code&gt; tag from the hit (&lt;code&gt;&amp;lt;Hsp_query-to&amp;gt;&lt;/code&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;It can be calculated by:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;query_cover = (&amp;lt;Hsp_query-to&amp;gt; - &amp;lt;Hsp_query-from&amp;gt; + 1) / seq_len * 100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Please note that I did not get this from any documentation. I figured it out by staring at the xml output returned from the Biopython &lt;code&gt;NCBIWWW.qblast()&lt;/code&gt; function. This solution seems to match with the results I get by blasting the same sequence on the interactive web portal.&lt;/p&gt;&#xA;" OwnerUserId="12737" LastEditorUserId="57" LastEditDate="2021-07-10T18:02:26.130" LastActivityDate="2021-07-10T18:02:26.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16351" PostTypeId="1" AcceptedAnswerId="16371" CreationDate="2021-07-10T06:48:23.497" Score="2" ViewCount="92" Body="&lt;p&gt;The following paragraph comes from the GSEA paper that explains how to obtain an enrichment score for a gene set. Note that the enrichment score is computed &lt;strong&gt;up to a position i&lt;/strong&gt; in L. There are 2 things I don’t understand about this:&lt;/p&gt;&#xA;&lt;p&gt;1- first, the following image illustrating the method seems to suggest that that we score a gene set across the whole list of ranked genes, which conflicts with the idea of “up to a position i” in L&lt;/p&gt;&#xA;&lt;p&gt;2- second, if we’re only considering genes up to a position i, then how are we going to capture the genes that are underexpressed (appear at the bottom of the ranked list) for phenotype A?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/guDWR.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/guDWR.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Wh35F.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Wh35F.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="8601" LastEditDate="2021-07-13T13:08:47.497" LastActivityDate="2021-07-13T13:09:00.567" Title="GSEA: Computing an Enrichment Score (as described in the original paper)" Tags="|gsea|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16352" PostTypeId="2" ParentId="16348" CreationDate="2021-07-10T11:39:19.970" Score="1" Body="&lt;p&gt;You can download the full db and and put out anything you want with your favourite scripting language.&lt;/p&gt;&#xA;&lt;p&gt;To do so, you need to request an account first. Once you log in, scroll down to the bottom of the download options. There are 2 meta-data download options, pick what you need.&lt;/p&gt;&#xA;" OwnerUserId="7153" LastEditorUserId="57" LastEditDate="2021-07-11T12:15:25.187" LastActivityDate="2021-07-11T12:15:25.187" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16353" PostTypeId="1" CreationDate="2021-07-11T03:58:09.427" Score="2" ViewCount="192" Body="&lt;p&gt;Is there a python equivalent to &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/ensembldb.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ensembldb&lt;/a&gt;? I want to get genome coordinates for a transcript (like the function &lt;code&gt;transcriptToGenome&lt;/code&gt;) but need to do it in python. I wrote a R script that I'm calling from python but I was hoping there was a more direct way to do this.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="12743" LastActivityDate="2021-07-11T10:07:27.920" Title="ensembldb equivalent in python" Tags="|bioconductor|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16354" PostTypeId="2" ParentId="16353" CreationDate="2021-07-11T10:07:27.920" Score="4" Body="&lt;p&gt;&lt;code&gt;pyensembl&lt;/code&gt; should do the job:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; from pyensembl import EnsemblRelease&#xA;&amp;gt;&amp;gt;&amp;gt; data = EnsemblRelease(76)&#xA;&amp;gt;&amp;gt;&amp;gt; data.transcript_by_id(&amp;quot;ENST00000506751&amp;quot;)&#xA;Transcript(..., contig='5', start=140861224, end=140863521, strand='+', genome='GRCh38')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2021-07-11T10:07:27.920" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16355" PostTypeId="1" CreationDate="2021-07-12T14:25:14.823" Score="0" ViewCount="1408" Body="&lt;p&gt;I want to rerun a Snakemake workflow, ignoring all previously produced intermediary outputs.&lt;/p&gt;&#xA;&lt;p&gt;Snakemake has about 100 different flags. I just want to make it easier to find an answer to this. What's the syntax/CLI flag I need to add?&lt;/p&gt;&#xA;" OwnerUserId="12272" LastEditorUserId="73" LastEditDate="2021-08-05T09:08:03.890" LastActivityDate="2021-09-04T10:05:20.603" Title="How can I make Snakemake ignore all previously produced/cached output files?" Tags="|snakemake|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16356" PostTypeId="2" ParentId="16355" CreationDate="2021-07-12T14:25:14.823" Score="1" Body="&lt;p&gt;If you want Snakemake to ignore the previous output of all rules, add the flag &lt;code&gt;--forceall&lt;/code&gt;. Otherwise, if you just want to ignore the output of a specific rule to rerun that rule, add &lt;code&gt;--force&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;snakemake --forceall -c1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The command line interface documentation is helpful as a reference: &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/executing/cli.html#all-options&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://snakemake.readthedocs.io/en/stable/executing/cli.html#all-options&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12272" LastEditorUserId="73" LastEditDate="2021-08-05T09:09:20.513" LastActivityDate="2021-08-05T09:09:20.513" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16357" PostTypeId="1" CreationDate="2021-07-12T15:34:53.763" Score="0" ViewCount="627" Body="&lt;p&gt;I'm wondering how Snakemake deals with output to &lt;code&gt;stdout&lt;/code&gt; and &lt;code&gt;stderr&lt;/code&gt; that is generated when running a rule with a &lt;code&gt;shell:&lt;/code&gt; command.&lt;/p&gt;&#xA;&lt;p&gt;I ran the following rule and only got informed that there was a non-zero exit code. However, I did not get a stack trace. To get access to the stack trace I had to run the shell command manually:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-yaml prettyprint-override&quot;&gt;&lt;code&gt;# Shortened for clarity&#xA;rule parse:&#xA;    input:&#xA;        sequences = rules.join_downloads.output&#xA;    output:&#xA;        sequences = &amp;quot;data/sequences.fasta&amp;quot;&#xA;    shell:&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        augur parse&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Why is that? And can I somehow make Snakemake output more detailed information when something goes wrong to help with debugging?&lt;/p&gt;&#xA;" OwnerUserId="12272" LastActivityDate="2021-07-12T17:41:49.467" Title="Where does Snakemake send output that's generated by shell commands?" Tags="|snakemake|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16358" PostTypeId="1" CreationDate="2021-07-12T15:54:23.187" Score="1" ViewCount="73" Body="&lt;p&gt;I have multiple .GFF files, each in a single directory per annotated whole genome, that I wish to copy into a single directory so that I can perform a core genome alignment.&lt;/p&gt;&#xA;&lt;p&gt;However the following code does not seem to work:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for pathname in &amp;quot;~/ST_0075/annotations/*/*.gff&amp;quot;; do cp &amp;quot;$pathname&amp;quot; ~/ST_0075/alignment; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The directory structure is the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Home&#xA;-ST_0075&#xA;---Annotations&#xA;-----Sample_001&#xA;----------001.gff&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The goal is to copy from multiple Sample_XXX subdirectories within &lt;code&gt;Annotations&lt;/code&gt; the XXX.gff files into another directory named &lt;code&gt;alignment&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="12317" LastEditorUserId="650" LastEditDate="2021-07-12T17:21:35.590" LastActivityDate="2021-07-12T17:21:35.590" Title="Copying files from multiple directories to a single" Tags="|linux|data-management|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16359" PostTypeId="2" ParentId="16358" CreationDate="2021-07-12T16:12:29.480" Score="3" Body="&lt;p&gt;Tinkered a bit and realized the solution did not neccistate a for loop.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mkdir ~/ST_0075/alignment; cp ~/ST_0075/annotations/*/*.gff ~/ST_0075/alignment&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12317" LastActivityDate="2021-07-12T16:12:29.480" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16360" PostTypeId="1" AcceptedAnswerId="17557" CreationDate="2021-07-12T16:43:31.850" Score="0" ViewCount="147" Body="&lt;p&gt;The student who was working on scRNA seq of KO and WT lines has made a mistake and he mixed both lines and generate the final sequencing data. Now, we are having gene expression data but don't know which is KO and which is WT.&lt;/p&gt;&#xA;&lt;p&gt;It's a genetic KO. Looking at the presence of absence of the KO gene might not be helpful here as its not deep sequenced.&lt;/p&gt;&#xA;&lt;p&gt;Cells were extracted from two stable line of animal; one wild type and other Chd2 gene knocked out. As a sequencing output I am having one fastq file in the format 28+10+10+90 . I can generate a count matrix out of it; but I dont know which barcode from WT and with one from KO.&lt;/p&gt;&#xA;&lt;p&gt;Is there any way to segregate the output based on the KO gene expression?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4951" LastEditorUserId="73" LastEditDate="2021-07-14T10:30:24.983" LastActivityDate="2021-08-13T13:38:35.063" Title="10X scRNAseq: Sample mix-up" Tags="|scrnaseq|single-cell|gene-expression|10x-genomics|cellranger|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16361" PostTypeId="2" ParentId="16357" CreationDate="2021-07-12T17:41:49.467" Score="1" Body="&lt;p&gt;You usually define a log file like below, and redirect the stdout and/or stderr to it. Stdout is &lt;code&gt;&amp;gt;&lt;/code&gt;, stderr is &lt;code&gt;2&amp;gt;&lt;/code&gt; and both of them combined is &lt;code&gt;&amp;amp;&amp;gt;&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule parse:&#xA;    input:&#xA;        sequences = rules.join_downloads.output&#xA;    output:&#xA;        sequences = &amp;quot;data/sequences.fasta&amp;quot;&#xA;    log:&#xA;        my_output/my_log.log&#xA;    shell:&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        augur parse 2&amp;gt; {log}&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="681" LastActivityDate="2021-07-12T17:41:49.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16362" PostTypeId="1" AcceptedAnswerId="16364" CreationDate="2021-07-12T21:31:48.097" Score="1" ViewCount="2296" Body="&lt;p&gt;I have a matrix in R. When I run &lt;code&gt;class(my.matrix)&lt;/code&gt; it returns:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;matrix&amp;quot; &amp;quot;array&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Why do I get &amp;quot;array&amp;quot; when in fact I want a plain matrix? The same &lt;code&gt;class&lt;/code&gt; output is returned when running &lt;code&gt;as.matrix(my.matrix))&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Why is that?&lt;/p&gt;&#xA;" OwnerUserId="9449" LastEditorDisplayName="user3051" LastEditDate="2021-07-13T14:21:31.117" LastActivityDate="2021-07-13T14:21:31.117" Title="Converting a matrix array to matrix" Tags="|r|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16363" PostTypeId="1" CreationDate="2021-07-13T02:26:44.253" Score="1" ViewCount="471" Body="&lt;p&gt;When you combine samples for de-novo transcriptome assembly with Trinity, do you suggest limiting the number of reads for each sample? I had read one of Matthew MacManes' papers awhile back suggesting 40 million reads was a good number for Trinity assemblies of most metazoans.&lt;/p&gt;&#xA;&lt;p&gt;However, I have 28 samples across 3 treatments that I want to use for DESeq and so, if I follow the advice &lt;a href=&quot;https://www.biostars.org/p/17826/&quot; rel=&quot;nofollow noreferrer&quot;&gt;at BioStars here&lt;/a&gt;, that's 28x40 million reads = 1.1 billion reads going into a Trinity assembler... This seems like a troublesome amount. How is it suggested to combine these samples in situations like this?&lt;/p&gt;&#xA;" OwnerUserId="2727" LastActivityDate="2021-07-13T08:52:03.067" Title="Trinity assembly from many samples" Tags="|assembly|transcriptome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16364" PostTypeId="2" ParentId="16362" CreationDate="2021-07-13T07:35:30.820" Score="5" Body="&lt;p&gt;&lt;code&gt;class()&lt;/code&gt; on a matrix will always return:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;matrix&amp;quot; &amp;quot;array&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Simple example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; class(as.matrix(1))&#xA;[1] &amp;quot;matrix&amp;quot; &amp;quot;array&amp;quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is because an array in R is a vector with multiple dimensions, and the matrix is a special type of array with exactly two dimensions, so every matrix is an array and this is what the &lt;code&gt;class&lt;/code&gt; command on a matrix is telling us here, see &lt;a href=&quot;http://adv-r.had.co.nz/Data-structures.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://adv-r.had.co.nz/Data-structures.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2021-07-13T14:12:56.797" LastActivityDate="2021-07-13T14:12:56.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16365" PostTypeId="2" ParentId="16363" CreationDate="2021-07-13T08:42:11.580" Score="1" Body="&lt;p&gt;Assuming these reads are from the same type of thing - and they should be if you're planning to use DESeq[2] on them - then it's fine to combine all the reads into a single Trinity assembly run.&lt;/p&gt;&#xA;&lt;p&gt;All recent versions of Trinity will perform in silico normalization by default (based on kmer counts), so you shouldn't need to do a manual subsampling of reads. Read normalisation can be done separately; details on that can be found here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/trinityrnaseq/trinityrnaseq/wiki/Trinity-Insilico-Normalization&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/trinityrnaseq/trinityrnaseq/wiki/Trinity-Insilico-Normalization&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;My vague memory is that the kmer counts are only used to get an idea of the total transcriptome size (i.e. for determining the average transcriptome coverage). If you want to get a more complete assembly, then consider only choosing samples that are most likely to have different overall transcript expression (e.g. different tissue types, treatments, etc.). If you've got 28 samples, you shouldn't need to worry about replicates; more variation is better.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-07-13T08:52:03.067" LastActivityDate="2021-07-13T08:52:03.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16366" PostTypeId="2" ParentId="16362" CreationDate="2021-07-13T08:55:15.197" Score="2" Body="&lt;p&gt;From the return value of the &lt;code&gt;class&lt;/code&gt; function, it looks like this already is a matrix. It just happens that it is considered by R to be both a matrix &lt;em&gt;and&lt;/em&gt; an array.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-13T08:55:15.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16367" PostTypeId="2" ParentId="16345" CreationDate="2021-07-13T09:01:51.397" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @user438383, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;It seems likely your input .bim file probably doesn't have RSIDs either - is that the case?&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-13T09:01:51.397" CommentCount="0" CommunityOwnedDate="2021-07-13T09:01:51.397" ContentLicense="CC BY-SA 4.0" />
  <row Id="16369" PostTypeId="1" CreationDate="2021-07-13T11:55:53.690" Score="1" ViewCount="31" Body="&lt;p&gt;I am looking for a database or different databases where I can find information on different gene variants for a gene. As an example if I enter PPARG, I could be able to see SNPs, CNV, InDels and MSI.&lt;/p&gt;&#xA;&lt;p&gt;In my understanding the information on SNPs can be downloaded from dbSNPs, MSI from ClinVar, CNV &amp;amp; InDels from dbVar. Please correct me if I am wrong.&lt;/p&gt;&#xA;&lt;p&gt;Is there a database that houses all of this information at under one roof?&lt;/p&gt;&#xA;&lt;p&gt;Your suggestions are very welcome.&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="5062" LastActivityDate="2021-07-14T10:25:01.233" Title="A unified database for CNV, SNP, Indel and MSI" Tags="|snp|mutations|cnv|indel|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16370" PostTypeId="1" CreationDate="2021-07-13T12:34:16.843" Score="1" ViewCount="419" Body="&lt;p&gt;I have used the TMM method in CSAW package to normalize the composition bias for my different ChIP-seq samples and also got the normalized bigwig files. Next step I want to analyze the differential binding peaks between several mutant with WT for specific IP (for example, H3K4me1). But I was confused about how to choose the proper way to do the differential binding peaks analysis. What I know now, there exist two basic strategies I could use:&lt;/p&gt;&#xA;&lt;p&gt;option 1. I use the DiffBind package to do that. The initial input file is bam files and a series of bed files generated by MACS2 call peaks (such as narrowPeak format). In this package, I will generate the normalization factors for the raw bam files before starting the differential binding analysis.&#xA;option 2. I could also use the strategy in the CSAW package to filter out the low abundance region and only keep the higher abundance region for multiple tests between different samples. (as shown in the second example mentioned in this paper(&lt;a href=&quot;https://f1000research.com/articles/4-1080/v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://f1000research.com/articles/4-1080/v1&lt;/a&gt;). In this example, I do not need to do peak calling beforehand and I just by calculating the counts by setting the bin width as a narrow or wide range.&lt;/p&gt;&#xA;&lt;p&gt;Based on these, I have several questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;since I want to use TMM composition bias normalization to normalize all bam files for specific IP, if I choose the DiffBind package to get the differential binding peaks based on raw bam files, will it cause some argument since in both TMM composition bias normalization in CSAW and DiffBind will generate their own normalization factors.&lt;/li&gt;&#xA;&lt;li&gt;is the peak calling by MACS2 is an essential step before the differential binding peak analysis? can I omit the peak calling step, just use the generate matrix generated by TMM composition bias normalization to do the differential binding peak analysis?&lt;/li&gt;&#xA;&lt;li&gt;if choose the DiffBind to analyze differential binding peaks, for the bed files of two biological replicates, should I use the common regions or the pooling regions of two biological replicate to do the analysis?&lt;/li&gt;&#xA;&lt;li&gt;If I pool the coordinate of peaks of all samples (macs2 callpeak) and calculate the count matrix based on the TMM composition bias-normalized bigwig files, is it possible to import the matrix directly into DiffBind to do the differential binding peak analysis?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I would appreciate it if someone would like to give me some suggestions on that! Many thanks.&lt;/p&gt;&#xA;" OwnerUserId="12132" LastActivityDate="2023-11-23T23:34:53.363" Title="How to choose the proper way to analyze the differential binding peaks among different samples for ChIP-seq? DiffBind or CSAW?" Tags="|chip-seq|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16371" PostTypeId="2" ParentId="16351" CreationDate="2021-07-13T13:09:00.567" Score="2" Body="&lt;p&gt;Actually I believe I get it now. i is not a constant. We vary it from i=1 to N and consider the i for which &lt;span class=&quot;math-container&quot;&gt;$P_{hit} - P_{miss}$&lt;/span&gt;  is largest in magnitude&lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2021-07-13T13:09:00.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16372" PostTypeId="1" CreationDate="2021-07-13T16:23:34.757" Score="3" ViewCount="2059" Body="&lt;p&gt;It is shown that TPM  values are not suitable for DEG analysis but good for within-sample comparison since TPM normalized the gene length. My question is first: if TPM is not suitable for across sample comparison, then why are some heatmaps using TPM values?  In what scenario are TPM values really useful? Should downstream analysis use TPM or raw count matrix?&lt;/p&gt;&#xA;&lt;p&gt;When I analyzed sc-RNA seq data, the Seurat package uses normalized counts (which is CPM) for DEG, but TPM is not advised for differential testing. I'm not sure what I'm missing in the understanding. It seems very convoluted to me.&lt;/p&gt;&#xA;&lt;p&gt;I just don't get the point that TPM is commonly used as an input for DEG testing by Seurat (Seurat findmarker function uses &amp;quot;data&amp;quot; slot, which is normalized counts), but TPM is not recommended for DEG testing for bulk.&lt;/p&gt;&#xA;" OwnerUserId="6398" LastEditorUserId="7340" LastEditDate="2021-11-02T17:24:46.630" LastActivityDate="2021-11-02T17:24:46.630" Title="The confusion of using TPM (transcripts per million)" Tags="|rna-seq|scrnaseq|tpm|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16373" PostTypeId="1" CreationDate="2021-07-13T18:09:58.843" Score="1" ViewCount="45" Body="&lt;p&gt;I'm using &lt;a href=&quot;https://beast.community/tip_dates&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEAUti&lt;/a&gt; as part of my pipeline and the sequence names include &lt;code&gt;|&lt;/code&gt; to separate the metadata fields - with culture date being at the very end.&lt;/p&gt;&#xA;&lt;p&gt;When I attempt to auto-configure the tip dates on the sequences however, some of the sequences will not update their date(raw value) and if I double-click and manually change the date it will then reverse back to 0.&lt;/p&gt;&#xA;&lt;p&gt;Any idea what may be preventing me from manually correcting the dates or auto-config to ignore the culture date after the last &lt;code&gt;|&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="12317" LastEditorUserId="650" LastEditDate="2021-07-13T18:44:02.420" LastActivityDate="2022-04-10T02:02:00.280" Title="BEAUti not accepting taxa tip dates" Tags="|multiple-sequence-alignment|beast|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16374" PostTypeId="2" ParentId="16372" CreationDate="2021-07-13T18:17:15.420" Score="4" Body="&lt;p&gt;Use raw counts as much as you can. Add in various relevant factors as covariates in DESeq2. RNAseq metrics have come a long way but are still misused by people because it's more convenient to compare some sort of normalized metric that give out relevant confounding factors, I guess.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2021-07-13T18:17:15.420" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16375" PostTypeId="2" ParentId="16372" CreationDate="2021-07-13T21:13:40.610" Score="1" Body="&lt;p&gt;For scRNA-seq, you don't want to normalize for gene length because the most popular 10X technology only sequences a the 3' or 5' end of the transcript. Hence CPM, which normalizes for sequencing depth alone, is popular for single cell data.&lt;/p&gt;&#xA;&lt;p&gt;This is different from your average bulk RNA-seq, where you typically get full length transcripts (unless you're doing something like DRUG-seq).&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2021-07-13T21:13:40.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16376" PostTypeId="1" CreationDate="2021-07-13T21:35:18.903" Score="1" ViewCount="126" Body="&lt;p&gt;We were looking for genetic diversity in the exon level of an MHC transcript and analysing the data under &lt;a href=&quot;http://www.popgen.dk/angsd/index.php/ANGSD&quot; rel=&quot;nofollow noreferrer&quot;&gt;ANGSD&lt;/a&gt; (Analysis of next generation Sequencing Data) written in C++&lt;/p&gt;&#xA;&lt;p&gt;We used ANGSD to calculate Thetas and Tajima D neutrality tests and one of the thetas was a pairwise theta, however I am unsure what this means and how it differs from Tajima's D?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.popgen.dk/angsd/index.php/Thetas,Tajima,Neutrality_tests&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.popgen.dk/angsd/index.php/Thetas,Tajima,Neutrality_tests&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;We then divided the tP over length of each exon to find genetic diversity in the exon level. However, as I am very interested in statistics, if someone could explain what the tP statistic means exactly it would be highly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="12135" LastEditorUserId="3848" LastEditDate="2022-02-26T23:04:43.067" LastActivityDate="2022-02-26T23:04:43.067" Title="What is the tP statistic from ANGSD?" Tags="|statistics|phylogeny|phylogenetics|exon|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16377" PostTypeId="2" ParentId="16372" CreationDate="2021-07-13T22:33:09.430" Score="3" Body="&lt;p&gt;The naive per-million scaling methods do not properly correct for the compositional bias between samples. This is especially true if the groups you compare are expected to be very different, e.g. different organs, see here demonstrated on some GTEx data: &lt;a href=&quot;https://www.biostars.org/p/9465851/#9465854&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/9465851/#9465854&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Why do people use flawed metrics? Well, because some are very common. Per-million is easy to compute and does not change when new samples are added. The latter is convenient, and sometimes per-million might be good enough for visualization. I never do it though, I always use normalized (or vst) counts from DESeq2 or edgeR.&lt;/p&gt;&#xA;&lt;p&gt;For differential analysis of bulk data one commonly uses raw counts which are then normalized internally by the established frameworks such as DESeq2, edgeR or limma-voom.&lt;/p&gt;&#xA;&lt;p&gt;Single-cell data, if you consider each cell a replicate, often can go with simpler stats as the large sample such allows tests such as the T- or Wilcox test. Again, plain per-million scaling is probably bad, and alternative methods do exist, e.g. the deconvulution/sum-factor strategy in the &lt;code&gt;scran&lt;/code&gt; package at Bioconductor. I cannot speak for Seurat, as I do not use it.&lt;/p&gt;&#xA;&lt;p&gt;For most downstream applications such as visualization and clustering you probably want to use the properly normalized counts on the log2 scale. For something like heatmaps one would scale the data (Z-score) before plotting to show relative differences. I see no application for raw counts beyond feeding them into a DE testing framework that then normalizes them. The only applications that use raw countsbeyond the aforementioned DE testing frameworks that come to my mind are Combat-Seq (batch correction for bulk data) and sctransform (varaince stab/normalization/regression) for single-cell data.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-07-13T22:33:09.430" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16378" PostTypeId="1" AcceptedAnswerId="16379" CreationDate="2021-07-13T23:56:23.767" Score="0" ViewCount="466" Body="&lt;p&gt;I want to remove sequence VRE32514 – it doesn’t belong and thus is the reason it lacks additional metadata. However I tried implementing this &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/13126/remove-sequences-from-fasta-file-matching-a-string-in-the-header&quot;&gt;code&lt;/a&gt; from a similar question:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN{while((getline&amp;lt;&amp;quot;&amp;gt;VRE32514&amp;quot;)&amp;gt;0)l[&amp;quot;&amp;gt;&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$1]=1}/^&amp;gt;/{f=!l[$&lt;/span&gt;1]}f' *.fasta | grep -v &amp;quot;&amp;gt;VRE32514&amp;quot; *.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With no success and the proposed solution in R using &lt;code&gt;seqinr&lt;/code&gt; doesn't seem to work either for myself.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#load package&#xA;library(seqinr)&#xA;#load file containing sequences&#xA;data&amp;lt;-read.fasta(&amp;quot;test.fasta&amp;quot;)&#xA;#create a vector containing species names: these are the last part of the string&#xA;vec.names&amp;lt;-unlist(lapply(strsplit(names(data), &amp;quot;|&amp;quot;), function(x)x[length(x)]))&#xA;#find names to keep: indices which are not in the species to remove&#xA;species.to.remove&amp;lt;-c(&amp;quot;&amp;gt;VRE32514&amp;quot;)&#xA;vec.tokeep&amp;lt;-which(! vec.names %in%  species.to.remove)&#xA;#write the final output&#xA;write.fasta(sequences=data[vec.tokeep], names=names(data)[vec.tokeep], file.out=&amp;quot;output.fasta&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Below is an example of what my Multi-FASTA appears as and the desired output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;VRE32491|736|PUH-10C|Blood|2016-12-07&#xA;ATGAGATCAGAAAAAGAAATGATGGATTTAGTACTTTCTTTAGCAGAACAGGATGAACGT&#xA;ATTCGAATTGTGACCCTTGAGGGGTCACGCGCAAATATTAATATACCTAAAGATGAATTT&#xA;&amp;gt;VRE32493|1471|PUH-10N|Tissue/Surgical|2016-12-08&#xA;CAGGATTATGATATTACATATTTTGTAAGTGATATAGAACCGTTTATATCTAATGATGAC&#xA;TGGCTTAATCAATTTGGGAATATAATAATGATGCAAAAGCCGGAGGATATGGAATTATTC&#xA;CCACCTGAAGAAAAGGGATTTTCCTATCTTATGCTATTTGATGATTACAATAAAATTGAT&#xA;&amp;gt;VRE32503|1471|PUH-11N|Wound|2017-01-05&#xA;CTTACCTTATTGCCCTTGGAAGAGTTAGATAATTACCTAAAGGGCGATAAATTAATAAAG&#xA;GTTCTAATTGATAAAGATTGTAGAATTAAAAGGGACATAGTTCCGACTGATATAGATTAT&#xA;&amp;gt;VRE32504|1471|PUH-EMEP|Blood|2017-01-10&#xA;CATGTAAGAAAGCCAAGCGCAAGGGAGTATGATGATTGCTGCAATGAATTTTGGAATGTA&#xA;ACACCTTATGTTATTAAAGGATTGTGCCGTAAGGAAATTTTATTTGCTATTGATCATTTT&#xA;&amp;gt;VRE32514&#xA;AATCAGATTGTTCGCCATGAGCTGCTGAGAATGATATCATGGAAGGGCGGCATCGAAACA&#xA;GGCTTTAAATTAAGTGTAGGCAAGAACTATAAGTTTATTGAAAGGTATATATCCGAGGAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Desired Output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;VRE32491|736|PUH-10C|Blood|2016-12-07&#xA;ATGAGATCAGAAAAAGAAATGATGGATTTAGTACTTTCTTTAGCAGAACAGGATGAACGT&#xA;ATTCGAATTGTGACCCTTGAGGGGTCACGCGCAAATATTAATATACCTAAAGATGAATTT&#xA;&amp;gt;VRE32493|1471|PUH-10N|Tissue/Surgical|2016-12-08&#xA;CAGGATTATGATATTACATATTTTGTAAGTGATATAGAACCGTTTATATCTAATGATGAC&#xA;TGGCTTAATCAATTTGGGAATATAATAATGATGCAAAAGCCGGAGGATATGGAATTATTC&#xA;CCACCTGAAGAAAAGGGATTTTCCTATCTTATGCTATTTGATGATTACAATAAAATTGAT&#xA;&amp;gt;VRE32503|1471|PUH-11N|Wound|2017-01-05&#xA;CTTACCTTATTGCCCTTGGAAGAGTTAGATAATTACCTAAAGGGCGATAAATTAATAAAG&#xA;GTTCTAATTGATAAAGATTGTAGAATTAAAAGGGACATAGTTCCGACTGATATAGATTAT&#xA;&amp;gt;VRE32504|1471|PUH-EMEP|Blood|2017-01-10&#xA;CATGTAAGAAAGCCAAGCGCAAGGGAGTATGATGATTGCTGCAATGAATTTTGGAATGTA&#xA;ACACCTTATGTTATTAAAGGATTGTGCCGTAAGGAAATTTTATTTGCTATTGATCATTTT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12317" LastEditorUserId="12317" LastEditDate="2021-07-14T00:14:51.903" LastActivityDate="2021-07-22T14:47:37.860" Title="Remove from Multi-FASTA by Sequence ID" Tags="|fasta|linux|shell|awk|multiple-sequence-alignment|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16379" PostTypeId="2" ParentId="16378" CreationDate="2021-07-14T00:30:08.790" Score="2" Body="&lt;p&gt;You can use &lt;code&gt;samtools faidx&lt;/code&gt; to first index your file and then retrieve only those sequences you specify. For this &lt;code&gt;grep&lt;/code&gt; all the sequence headers you want and then remove the unwanted ones, followed by retrieval of the fasta entries with &lt;code&gt;faidx&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#/ index:&#xA;samtools faidx your.fa&#xA;&#xA;#/ run:&#xA;grep '^&amp;gt;' test.fa \             # grep names&#xA;| grep -v '&amp;gt;VRE32514' \         # remove unwanted&#xA;| tr -d '&amp;gt;' \                   # remove &amp;quot;&amp;gt;&amp;quot; from name&#xA;| xargs samtools faidx your.fa  # get the entries you want&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;VRE32491|736|PUH-10C|Blood|2016-12-07&#xA;ATGAGATCAGAAAAAGAAATGATGGATTTAGTACTTTCTTTAGCAGAACAGGATGAACGT&#xA;ATTCGAATTGTGACCCTTGAGGGGTCACGCGCAAATATTAATATACCTAAAGATGAATTT&#xA;&amp;gt;VRE32493|1471|PUH-10N|Tissue/Surgical|2016-12-08&#xA;CAGGATTATGATATTACATATTTTGTAAGTGATATAGAACCGTTTATATCTAATGATGAC&#xA;TGGCTTAATCAATTTGGGAATATAATAATGATGCAAAAGCCGGAGGATATGGAATTATTC&#xA;CCACCTGAAGAAAAGGGATTTTCCTATCTTATGCTATTTGATGATTACAATAAAATTGAT&#xA;&amp;gt;VRE32503|1471|PUH-11N|Wound|2017-01-05&#xA;CTTACCTTATTGCCCTTGGAAGAGTTAGATAATTACCTAAAGGGCGATAAATTAATAAAG&#xA;GTTCTAATTGATAAAGATTGTAGAATTAAAAGGGACATAGTTCCGACTGATATAGATTAT&#xA;&amp;gt;VRE32504|1471|PUH-EMEP|Blood|2017-01-10&#xA;CATGTAAGAAAGCCAAGCGCAAGGGAGTATGATGATTGCTGCAATGAATTTTGGAATGTA&#xA;ACACCTTATGTTATTAAAGGATTGTGCCGTAAGGAAATTTTATTTGCTATTGATCATTTT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In &lt;code&gt;R&lt;/code&gt; one could use &lt;code&gt;Biostrings&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(Biostrings)&#xA;&#xA;fasta &amp;lt;- readDNAStringSet(&amp;quot;path/to/your.fa&amp;quot;)&#xA;fasta_new &amp;lt;- fasta[!names(fasta) %in% c(&amp;quot;VRE32514&amp;quot;)]&#xA;&#xA;#/ save to disk:&#xA;writeXStringSet(fasta_new, &amp;quot;path/to/outputfile.fa&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-07-14T00:30:08.790" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16380" PostTypeId="2" ParentId="16373" CreationDate="2021-07-14T01:37:04.010" Score="1" Body="&lt;p&gt;I found the solution, one of my Seq IDs had a &lt;code&gt;,&lt;/code&gt; in one of the meta-data fields, this appears to have thrown off the software and all subsequent Seq IDs could not be auto-configured.&lt;/p&gt;&#xA;&lt;p&gt;Simply removing the &lt;code&gt;,&lt;/code&gt; solved the issue.&lt;/p&gt;&#xA;" OwnerUserId="12317" LastActivityDate="2021-07-14T01:37:04.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16381" PostTypeId="2" ParentId="16378" CreationDate="2021-07-14T01:59:47.017" Score="0" Body="&lt;p&gt;Here's a solution using Python:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;sequences = SeqIO.parse('test.fasta', 'fasta')&#xA;filtered = [seq for seq in sequences if 'VRE32514' not in seq.id]&#xA;&#xA;with open('output.fasta', 'wt') as output:&#xA;    SeqIO.write(filtered, output, 'fasta')&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12439" LastActivityDate="2021-07-14T01:59:47.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16382" PostTypeId="2" ParentId="16369" CreationDate="2021-07-14T10:25:01.233" Score="1" Body="&lt;p&gt;To Answer my own question, one can use variation viewer to get all the information :)&lt;/p&gt;&#xA;&lt;p&gt;DisGeNet is another resource.&lt;/p&gt;&#xA;" OwnerUserId="5062" LastActivityDate="2021-07-14T10:25:01.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16383" PostTypeId="2" ParentId="16360" CreationDate="2021-07-14T10:32:07.470" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @haci, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;If your mutation of interest is &amp;quot;covered&amp;quot; by the &amp;quot;sample read&amp;quot;, you should be able to assign which cell comes from which line. By &amp;quot;sampe read&amp;quot;, I mean some 90+ bp read that is used for sequencing the insert (no the read for the cell barcode &amp;amp; UMI).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-14T10:32:07.470" CommentCount="0" CommunityOwnedDate="2021-07-14T10:32:07.470" ContentLicense="CC BY-SA 4.0" />
  <row Id="16384" PostTypeId="2" ParentId="12981" CreationDate="2021-07-14T11:02:28.433" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from PPK, converted from comments:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;The memory warning is only letting you know that the upper limit you set on memory (112 gb) was higher than the available memory. Do you share the server with other people? because the minimum amount of memory for Cell Ranger is 64 GB. In your initial run there may have been less than 64 GB free.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-14T11:02:28.433" CommentCount="0" CommunityOwnedDate="2021-07-14T11:02:28.433" ContentLicense="CC BY-SA 4.0" />
  <row Id="16385" PostTypeId="2" ParentId="12953" CreationDate="2021-07-14T11:03:49.303" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @devon-ryan, converted from comments:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Look in Biomart to see if there's SNP rsids for wheat.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-14T11:03:49.303" CommentCount="0" CommunityOwnedDate="2021-07-14T11:03:49.303" ContentLicense="CC BY-SA 4.0" />
  <row Id="16386" PostTypeId="2" ParentId="12820" CreationDate="2021-07-14T11:08:14.203" Score="3" Body="&lt;p&gt;&lt;em&gt;Answer from @alex-reynolds, converted from comments:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;As with any compression scheme, if you have to store a reference genome, along with the 4 Mb &amp;quot;diff&amp;quot;, is it really compressed?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://mattmahoney.net/dc/dce.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mattmahoney.net/dc/dce.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For sequencing data, you might also look at the CRAM format, which also uses diffs against a reference genome to store reduced datasets. Labs are switching from BAM to CRAM to manage storage of increased numbers of sequencing datasets, with associated changes to pipelines and toolkits to use this format. For performance reasons, BAM may be preferred for analysis, and CRAM for long-term storage.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035890791-SAM-or-BAM-or-CRAM-Mapped-sequence-data-formats&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360035890791-SAM-or-BAM-or-CRAM-Mapped-sequence-data-formats&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ga4gh.org/news/cram-compression-for-genomics/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ga4gh.org/news/cram-compression-for-genomics/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-14T11:08:14.203" CommentCount="0" CommunityOwnedDate="2021-07-14T11:08:14.203" ContentLicense="CC BY-SA 4.0" />
  <row Id="16387" PostTypeId="2" ParentId="12768" CreationDate="2021-07-14T11:14:23.450" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @user438383, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would recommend checking out &lt;a href=&quot;https://odelaneau.github.io/shapeit4/&quot; rel=&quot;nofollow noreferrer&quot;&gt;shapeit4&lt;/a&gt; as it is able to integrate statistical phasing and read-based phasing like you are after. The manual page for WhatsHap might also be useful for you:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://whatshap.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://whatshap.readthedocs.io/en/latest/index.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-14T11:14:23.450" CommentCount="0" CommunityOwnedDate="2021-07-14T11:14:23.450" ContentLicense="CC BY-SA 4.0" />
  <row Id="16388" PostTypeId="2" ParentId="11573" CreationDate="2021-07-14T11:27:19.127" Score="1" Body="&lt;p&gt;&lt;em&gt;comment discussion between @krushnach-chandra and @stupidwolf (in chronical order):&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;SW:&lt;/strong&gt; No you don't need gene expression data.. You can just count the reads from polII chip (normalized by some method of your choice) and perform the analysis&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;KC:&lt;/strong&gt; are you suggesting I can compare the Pol II chip between control and Knock out with their respect IP data?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;SW:&lt;/strong&gt; more or less yes...if you are interested in genomewide effects.. For example, if the treatment causes the polymerase to stall, you will see more reads in the promoter compared to the gene body&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;KC:&lt;/strong&gt; so naive question do i need those Input and IgG at all for the comparison?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;SW:&lt;/strong&gt; good question.. depends on how you analyze it. If you always normalize within the gene, for example, you calculate a coverage normalized to that in the gene, or some meta gene thing, you don't need.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;SW:&lt;/strong&gt; If you do a differential gene expression based analysis, in theory you don't really need that, because you will always compare between your two conditions right. The IgG is good for checking you don't have any odd stuff... You can see whether there are genes where the control is enriched over your samples&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;KC:&lt;/strong&gt; &amp;quot;calculate a coverage normalized to that in the gene, or some meta gene thing&amp;quot; any example can you cite me..so that i can follow and proceed ,as it looks like binding data are really complicated compared to rna seq&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;KC:&lt;/strong&gt; &amp;quot;If you do a differential gene expression based analysis,&amp;quot; you saying i can take out counts and do diff analysis for chip seq data using deseq2? but how do i get the information regarding if Pol II activity is going or stalled ? and how do i find the occupancy from the differential result&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;SW:&lt;/strong&gt; welcome to the world of epigenomics!!! maybe something like &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/vignettes/ChIPseeker/inst/doc/ChIPseeker.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;SW:&lt;/strong&gt; As for differential gene expression, yes. If you do this separately for promoters and gene body, you can see that the fold change will be different for a gene's promoter vs gene body, in situations where there's stalling, or reduced elongation etc.. I think it's a good way to account for the variation&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;KC:&lt;/strong&gt; your insights are good start for me to run the analysis&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;KC:&lt;/strong&gt; mundane question how do i get promoter and gene body information is it after annotation which should be done after running analysis ?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;SW:&lt;/strong&gt; Meaning? I normally align the reads, then for TSS and gene body you have to define it yourself using the transcript or gene annotation. Normally like +/- 200 bp around TSS for promoter and gene body is rest of it?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;KC:&lt;/strong&gt; &amp;quot;Normally like +/- 200 bp around TSS for promoter and gene body is rest of it&amp;quot; so i can do that in both chipseeker as well as homer?&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-14T11:27:19.127" CommentCount="0" CommunityOwnedDate="2021-07-14T11:27:19.127" ContentLicense="CC BY-SA 4.0" />
  <row Id="16389" PostTypeId="2" ParentId="11690" CreationDate="2021-07-14T11:32:17.673" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @fabio-marroni, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;You might try to use &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/MutationalPatterns/inst/doc/Introduction_to_MutationalPatterns.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;mutationalPatterns&lt;/a&gt; and see if the  responders cluster together and separated from nonresponders.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-14T11:32:17.673" CommentCount="0" CommunityOwnedDate="2021-07-14T11:32:17.673" ContentLicense="CC BY-SA 4.0" />
  <row Id="16390" PostTypeId="2" ParentId="11413" CreationDate="2021-07-14T11:40:04.423" Score="2" Body="&lt;p&gt;&lt;em&gt;Answer from @devon-ryan, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Both, that's why NAT5 is no longer a human gene symbol.&lt;/p&gt;&#xA;&lt;p&gt;NAT5 does not equal NAA20, it's an out-dated name for it. Names change over time as people realize that there are more genes for something than originally thought.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-14T11:40:04.423" CommentCount="0" CommunityOwnedDate="2021-07-14T11:40:04.423" ContentLicense="CC BY-SA 4.0" />
  <row Id="16391" PostTypeId="2" ParentId="11143" CreationDate="2021-07-14T11:45:52.277" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @haci, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can use the &lt;code&gt;pch&lt;/code&gt; parameter in &lt;code&gt;anno_simple()&lt;/code&gt; but you can only use a single character or a digit. It is not limited to symbols only,  you can provide a vector to &lt;code&gt;pch&lt;/code&gt;, but only the first character would be printed. If by chance your values are [0-9] you can use &lt;code&gt;pch&lt;/code&gt; as kind of hack for your purposes above.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-14T11:45:52.277" CommentCount="0" CommunityOwnedDate="2021-07-14T11:45:52.277" ContentLicense="CC BY-SA 4.0" />
  <row Id="16392" PostTypeId="2" ParentId="10877" CreationDate="2021-07-14T11:56:24.953" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @devon-ryan, converted from comments:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;The common presumption in such cases is that there's a feedback loop on most genes to control their expression level. It doesn't have to be specific to arm-level expression.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-14T11:56:24.953" CommentCount="0" CommunityOwnedDate="2021-07-14T11:56:24.953" ContentLicense="CC BY-SA 4.0" />
  <row Id="16393" PostTypeId="2" ParentId="11413" CreationDate="2021-07-14T11:57:48.040" Score="5" Body="&lt;p&gt;At Ensembl, we categorise synonyms as anything that a gene might also be known as. This includes older names for them, since those names will be in the literature, including where a gene has been split in two.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2021-07-14T11:57:48.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16394" PostTypeId="2" ParentId="10721" CreationDate="2021-07-14T12:06:00.757" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @m, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;It's definitely not supposed to happen. There appears to be a deep rooted classification bug, where matk is misclassified as ITS. Unpicking this will be hard.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-14T12:06:00.757" CommentCount="0" CommunityOwnedDate="2021-07-14T12:06:00.757" ContentLicense="CC BY-SA 4.0" />
  <row Id="16396" PostTypeId="2" ParentId="16341" CreationDate="2021-07-15T01:39:25.793" Score="2" Body="&lt;p&gt;NCBI's FASTA format description:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genbank/fastaformat/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/genbank/fastaformat/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;NCBI's BLAST page describing valid FASTA input:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&amp;amp;PAGE_TYPE=BlastDocs&amp;amp;DOC_TYPE=BlastHelp&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&amp;amp;PAGE_TYPE=BlastDocs&amp;amp;DOC_TYPE=BlastHelp&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;NCBI's SNP page describing FASTA format:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/projects/SNP/snp_legend.cgi?legend=fasta&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/projects/SNP/snp_legend.cgi?legend=fasta&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;UniProt's FASTA-header description:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.uniprot.org/help/fasta-headers&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/help/fasta-headers&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Wikipedia reference (indicating format origin):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/FASTA_format&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/FASTA_format&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;FASTA program (origin of format, per Wikipedia):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://fasta.bioch.virginia.edu/wrp_fasta/fasta_guide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://fasta.bioch.virginia.edu/wrp_fasta/fasta_guide.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Harvard &lt;em&gt;PolyPhen&lt;/em&gt; page describing FASTA format:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://genetics.bwh.harvard.edu/pph/FASTA.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genetics.bwh.harvard.edu/pph/FASTA.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;FASTA file format (from Pacific Biosciences):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://pacbiofileformats.readthedocs.io/en/3.0/FASTA.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pacbiofileformats.readthedocs.io/en/3.0/FASTA.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9384" LastEditorUserId="9384" LastEditDate="2021-07-15T01:57:47.940" LastActivityDate="2021-07-15T01:57:47.940" CommentCount="0" CommunityOwnedDate="2021-07-15T01:39:25.793" ContentLicense="CC BY-SA 4.0" />
  <row Id="16397" PostTypeId="1" AcceptedAnswerId="16398" CreationDate="2021-07-15T10:18:04.453" Score="1" ViewCount="1037" Body="&lt;p&gt;In Seurat metadata I have assigned cells to some cell types with different resolutions&lt;/p&gt;&#xA;&lt;p&gt;I have added the cluster identities to the object via &lt;code&gt;Idents&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; Idents(control) &amp;lt;- a$predicted.celltype.l1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xoWPp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xoWPp.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I then added another clustering in the same way:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; Idents(control) &amp;lt;- a$predicted.celltype.l2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Dimplot will now show the new identities:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XpReN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XpReN.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;How I can ask Dimplot to plot the clustering I want for example if I want to show &lt;code&gt;predicted.celltype.l1&lt;/code&gt; again (something like changing the default)?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="73" LastEditDate="2021-07-15T11:59:49.150" LastActivityDate="2021-07-15T23:05:39.110" Title="Seurat Dimplot with different clustering IDs" Tags="|scrnaseq|seurat|10x-genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16398" PostTypeId="2" ParentId="16397" CreationDate="2021-07-15T12:08:41.263" Score="1" Body="&lt;p&gt;The thing to tweak is the &lt;code&gt;group.by&lt;/code&gt; argument of the &lt;code&gt;Dimplot&lt;/code&gt; function.&lt;/p&gt;&#xA;&lt;p&gt;It looks like you've already tried this in your attempts at solving &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/16332/color-by-clusters-and-sampled-in-seurat&quot;&gt;a different problem&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The main thing you need to be aware of is that each new clustering needs to be saved as a separate metadata variable in order to do this. As I mentioned in one of my &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/16337/73&quot;&gt;previous answers&lt;/a&gt;, I like to use the &lt;code&gt;[[X]]&lt;/code&gt; notation for this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;control[[&amp;quot;predicted.celltype.l1&amp;quot;]] &amp;lt;- a&lt;span class=&quot;math-container&quot;&gt;$predicted.celltype.l1&#xA;control[[&quot;predicted.celltype.l2&quot;]] &amp;lt;- a$&lt;/span&gt;predicted.celltype.l2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then you should be able to use both in the &lt;code&gt;group.by&lt;/code&gt; argument:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;DimPlot(control, reduction = &amp;quot;umap&amp;quot;, group.by = &amp;quot;predicted.celltype.l1&amp;quot;)&#xA;DimPlot(control, reduction = &amp;quot;umap&amp;quot;, group.by = &amp;quot;predicted.celltype.l2&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-07-15T23:05:39.110" LastActivityDate="2021-07-15T23:05:39.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16399" PostTypeId="1" AcceptedAnswerId="16401" CreationDate="2021-07-15T19:04:43.610" Score="1" ViewCount="80" Body="&lt;p&gt;I have extensively read about &lt;code&gt;percent mito&lt;/code&gt; in Seurat but I got more and more confused&lt;/p&gt;&#xA;&lt;p&gt;Let's say we want to keep cells with &lt;code&gt;maximum 20%&lt;/code&gt; of their expressed genes are mitochondrial genes, so shall I set &lt;code&gt;percent.mito &amp;lt; 20&lt;/code&gt; or &lt;code&gt;percent.mito &amp;lt; 0.2&lt;/code&gt; or &lt;code&gt;percent.mito &amp;lt; 80&lt;/code&gt; ?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2021-07-15T20:57:29.390" Title="What does this Seurat argument mean" Tags="|scrnaseq|seurat|10x-genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16401" PostTypeId="2" ParentId="16399" CreationDate="2021-07-15T20:57:29.390" Score="2" Body="&lt;p&gt;Checkout the first &lt;a href=&quot;https://satijalab.org/seurat/articles/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seurat tutorial&lt;/a&gt;. To filter cells with &amp;gt;20% mitochondrial counts:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;obj[[&amp;quot;percent.mito&amp;quot;]] &amp;lt;- PercentageFeatureSet(obj, pattern = &amp;quot;^MT-&amp;quot;)&#xA;obj &amp;lt;- subset(obj, subset = percent.mito &amp;lt; 20)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastActivityDate="2021-07-15T20:57:29.390" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16402" PostTypeId="1" CreationDate="2021-07-16T05:37:51.433" Score="0" ViewCount="562" Body="&lt;p&gt;I metadata slot of Seurat object I have mapping score of each cell to a reference PBMC data like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pbmc &amp;lt;- subset(object, subset = object@meta.data$mapping.score &amp;gt; 0.5)&#xA;Error in FetchData(object = object, vars = unique(x = expr.char[vars.use]),  : &#xA;  None of the requested variables were found: &#xA;&amp;gt; &#xA;&amp;gt; &#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to only keep cells with mapping score &amp;gt; 0.5&lt;/p&gt;&#xA;&lt;p&gt;Any help please?&lt;/p&gt;&#xA;" OwnerUserId="9449" LastActivityDate="2021-07-16T10:56:42.823" Title="Filtering cells from values in metadata" Tags="|scrnaseq|seurat|10x-genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16403" PostTypeId="1" AcceptedAnswerId="16404" CreationDate="2021-07-16T07:51:35.967" Score="1" ViewCount="66" Body="&lt;p&gt;I have a file with cluster number of a scRNA-seq and corresponding annotated cell type like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dput(head(anno))&#xA;structure(list(cluster = 1:6, celltype = c(&amp;quot;T:CD4+NAIVE&amp;quot;, &amp;quot;T:CD4+NAIVE&amp;quot;, &#xA;&amp;quot;T:CD8+NAIVE&amp;quot;, &amp;quot;NK:CD56+16+3+NKT&amp;quot;, &amp;quot;T:CD8+NAIVE&amp;quot;, &amp;quot;B:&amp;quot;)), row.names = c(NA, &#xA;6L), class = &amp;quot;data.frame&amp;quot;)&#xA;&#xA;&#xA;&amp;gt; unique(anno&lt;span class=&quot;math-container&quot;&gt;$cluster)&#xA; [1]  1  2  3  4  5  6  8  9 11 12 15 16 19 20 21 22 23 24 25 27 28 29 30 31 35 36 37 38 39 40 41 42 43 44 45 46&#xA;[37] 47 48 49 50 51 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 80 81 82 83 84&#xA;[73] 85 86 87 88 89 90 91 92 93 94 95&#xA;&amp;gt; unique(anno$&lt;/span&gt;celltype)&#xA; [1] &amp;quot;T:CD4+NAIVE&amp;quot;        &amp;quot;T:CD8+NAIVE&amp;quot;        &amp;quot;NK:CD56+16+3+NKT&amp;quot;   &amp;quot;B:&amp;quot;                 &amp;quot;NKT&amp;quot;               &#xA; [6] &amp;quot;T:CD8+EM&amp;quot;           &amp;quot;T:CD4+CM&amp;quot;           &amp;quot;mDC:&amp;quot;               &amp;quot;MACROPHAGE:&amp;quot;        &amp;quot;MONOCYTE:precursor&amp;quot;&#xA;[11] &amp;quot;NEUTROPHIL:&amp;quot;        &amp;quot;NK:CD56+16+3-&amp;quot;      &amp;quot;MAST:&amp;quot;              &amp;quot;MONOCYTE:&amp;quot;          &amp;quot;pDC:&amp;quot;              &#xA;[16] &amp;quot;T:Reg&amp;quot;  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and another data frame with cell IDs and corresponding cluster number of each cell&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dput(head(df2))&#xA;structure(list(cluster = c(1L, 1L, 1L, 1L, 1L, 1L), cellid = c(&amp;quot;cell17203&amp;quot;, &#xA;&amp;quot;cell17205&amp;quot;, &amp;quot;cell17206&amp;quot;, &amp;quot;cell17207&amp;quot;, &amp;quot;cell17208&amp;quot;, &amp;quot;cell17209&amp;quot;&#xA;)), row.names = 15498:15503, class = &amp;quot;data.frame&amp;quot;)&#xA;&#xA;&amp;gt; unique(df2$cluster)&#xA; [1]  1  2  3  4  5  6  8 12 16 19 21 23 27 29 30 35 36 40 53 55 92 11 24 38 41 46 90  9 15 20 47 74 57 39 42 25&#xA;[37] 56 64&#xA;&amp;gt; &#xA;&amp;gt; dim(df2)&#xA;[1] 16056     2&#xA;&amp;gt;   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to create a cell type column in the second data frame (df2) in which relate cell type from the first data frame to cluster number for instance &lt;code&gt;cluster 1&lt;/code&gt; is &lt;code&gt;T:CD4+NAIVE&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have tried these&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; df2[anno, on = &amp;quot;cluster&amp;quot;]&#xA;Error in `[.data.frame`(df2, anno, on = &amp;quot;cluster&amp;quot;) : &#xA;  unused argument (on = &amp;quot;cluster&amp;quot;)&#xA;&amp;gt; df2[c, on = anno&lt;span class=&quot;math-container&quot;&gt;$cluster]&#xA;Error in `[.data.frame`(df2, anno, on = c$&lt;/span&gt;cluster) : &#xA;  unused argument (on = anno&lt;span class=&quot;math-container&quot;&gt;$cluster)&#xA;&amp;gt; df2[c, on = df2$&lt;/span&gt;cluster]&#xA;Error in `[.data.frame`(df2, anno, on = df2&lt;span class=&quot;math-container&quot;&gt;$cluster) : &#xA;  unused argument (on = df2$&lt;/span&gt;cluster)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2021-07-16T22:51:26.180" Title="Merging two dataframes in R" Tags="|r|scrnaseq|seurat|10x-genomics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16404" PostTypeId="2" ParentId="16403" CreationDate="2021-07-16T08:02:51.277" Score="1" Body="&lt;p&gt;You used &lt;code&gt;data.table&lt;/code&gt; notation without actually having data.table:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(data.table)&#xA;setDT(anno)&#xA;setDT(df2)&#xA;df2[anno, on = &amp;quot;cluster&amp;quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;    cluster    cellid         celltype&#xA; 1:       1 cell17203      T:CD4+NAIVE&#xA; 2:       1 cell17205      T:CD4+NAIVE&#xA; 3:       1 cell17206      T:CD4+NAIVE&#xA; 4:       1 cell17207      T:CD4+NAIVE&#xA; 5:       1 cell17208      T:CD4+NAIVE&#xA; 6:       1 cell17209      T:CD4+NAIVE&#xA; 7:       2      &amp;lt;NA&amp;gt;      T:CD4+NAIVE&#xA; 8:       3      &amp;lt;NA&amp;gt;      T:CD8+NAIVE&#xA; 9:       4      &amp;lt;NA&amp;gt; NK:CD56+16+3+NKT&#xA;10:       5      &amp;lt;NA&amp;gt;      T:CD8+NAIVE&#xA;11:       6      &amp;lt;NA&amp;gt;               B:&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2021-07-16T08:02:51.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16407" PostTypeId="2" ParentId="16402" CreationDate="2021-07-16T10:56:42.823" Score="1" Body="&lt;pre&gt;&lt;code&gt;your_new_object &amp;lt;- subset(your_object, subset = mapping.score &amp;gt; 0.5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2021-07-16T10:56:42.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16410" PostTypeId="1" CreationDate="2021-07-16T20:31:56.183" Score="0" ViewCount="425" Body="&lt;p&gt;I tried to add a &amp;quot;gather_results&amp;quot; task to a workflow that takes in an array of Files and moves output files to a new folder of a specified name.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Cannot interpolate Array[File?]+ into a command string with attribute set wdl PlaceholderAttributeSet(None,None,None,Some( ))]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The wdl file passes womtools and I have no idea what a PlaceholderAttributeSet is, which makes debugging quite difficult.&lt;/p&gt;&#xA;&lt;p&gt;--&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;gather files task&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;task gather_result{&#xA;  input{&#xA;    String dir&#xA;    String? subDir&#xA;    Array[File?]+ files&#xA;  }&#xA;&#xA;  command&#xA;  {&#xA;    mkdir --parents &lt;span class=&quot;math-container&quot;&gt;${dir}/$&lt;/span&gt;{default=&amp;quot;&amp;quot; subDir}; /bin/mv -t &lt;span class=&quot;math-container&quot;&gt;${dir}/$&lt;/span&gt;{default=&amp;quot;&amp;quot; subDir} ${sep=&amp;quot; &amp;quot; files}&#xA;  }&#xA;  runtime&#xA;  {&#xA;    docker_image: &amp;quot;pipeline&amp;quot;&#xA;    memory: &amp;quot;2 GB&amp;quot;&#xA;  }&#xA;  output&#xA;  {&#xA;    String out = stdout()&#xA;  }&#xA;}&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How it's called:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  call gather_result as gather_files {&#xA;      input: files=[create_hic.output_hic[1],&#xA;                    dedup.deduped_bam,&#xA;                    calculate_stats.stats[0],&#xA;                    calculate_stats.stats_hists[0],&#xA;                    calculate_stats.stats_json[0]],&#xA;            dir=OutputDir&#xA;    }&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12775" LastActivityDate="2021-07-19T20:17:39.510" Title="Error when trying to move multiple file types in wdl" Tags="|wdl|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16411" PostTypeId="2" ParentId="9228" CreationDate="2021-07-16T21:38:26.423" Score="2" Body="&lt;p&gt;MRNM stands for &amp;quot;Mate reference index&amp;quot;. So Picard found something in the RNEXT field which should be set only for paired-end reads but the rest of the file looks like single-end.&lt;/p&gt;&#xA;&lt;p&gt;The problematic line in your code is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;line.next_reference_id = 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This sets the RNEXT SAM field to whatever Pysam stores as a reference with index 0 (&lt;code&gt;next_reference_id&lt;/code&gt;). This is &lt;strong&gt;not&lt;/strong&gt; the same as reference name (&lt;code&gt;next_reference_name&lt;/code&gt;). In single-end SAM file this field should be set to &lt;code&gt;*&lt;/code&gt;. You cannot set &lt;code&gt;*&lt;/code&gt; with &lt;code&gt;next_reference_id&lt;/code&gt; but you can set it with &lt;code&gt;next_reference_name&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;line.next_reference_name = &amp;quot;*&amp;quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would also recommend to change &lt;code&gt;next_reference_start&lt;/code&gt; line to:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;line.next_reference_start = -1 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;just to be sure Pysam handles this field as not set (I tried to put there &lt;code&gt;0&lt;/code&gt; Pysam changed it to &lt;code&gt;1&lt;/code&gt; for some reason). I would also add line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;line.template_length = -1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;because this field should also be empty in single-end SAM.&lt;/p&gt;&#xA;&lt;p&gt;This worked in my case. If Picard keeps complaining you can add &lt;code&gt;VALIDATION_STRINGENCY=LENIENT&lt;/code&gt; or &lt;code&gt;SILENT&lt;/code&gt; to your Picard command to suppress exiting of the command at errors.&lt;/p&gt;&#xA;" OwnerUserId="486" LastEditorUserId="486" LastEditDate="2021-07-20T19:07:38.047" LastActivityDate="2021-07-20T19:07:38.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16412" PostTypeId="2" ParentId="16403" CreationDate="2021-07-16T22:51:26.180" Score="5" Body="&lt;p&gt;For merging dataframes, I find it easiest to use the tidyverse / dplyr functions inner/full/left/right_join. See the &amp;quot;Data Transformation Cheatsheet&amp;quot; on &lt;a href=&quot;https://www.rstudio.com/resources/cheatsheets/&quot; rel=&quot;noreferrer&quot;&gt;this page&lt;/a&gt;. For the merge that @user438383 has mentioned, this would be a &lt;em&gt;left_join&lt;/em&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse)&#xA;anno &amp;lt;- data.frame(cluster = 1:6, celltype = c(&amp;quot;T:CD4+NAIVE&amp;quot;, &amp;quot;T:CD4+NAIVE&amp;quot;, &#xA;                   &amp;quot;T:CD8+NAIVE&amp;quot;, &amp;quot;NK:CD56+16+3+NKT&amp;quot;, &amp;quot;T:CD8+NAIVE&amp;quot;, &amp;quot;B:&amp;quot;))&#xA;&#xA;df2 &amp;lt;- data.frame(cluster = c(1L, 1L, 1L, 1L, 1L, 1L),&#xA;                  cellid = c(&amp;quot;cell17203&amp;quot;, &amp;quot;cell17205&amp;quot;, &amp;quot;cell17206&amp;quot;,&#xA;                             &amp;quot;cell17207&amp;quot;, &amp;quot;cell17208&amp;quot;, &amp;quot;cell17209&amp;quot;),&#xA;                  row.names = 15498:15503)&#xA;&#xA;&amp;gt; left_join(anno, df2)&#xA;Joining, by = &amp;quot;cluster&amp;quot;&#xA;   cluster         celltype    cellid&#xA;1        1      T:CD4+NAIVE cell17203&#xA;2        1      T:CD4+NAIVE cell17205&#xA;3        1      T:CD4+NAIVE cell17206&#xA;4        1      T:CD4+NAIVE cell17207&#xA;5        1      T:CD4+NAIVE cell17208&#xA;6        1      T:CD4+NAIVE cell17209&#xA;7        2      T:CD4+NAIVE      &amp;lt;NA&amp;gt;&#xA;8        3      T:CD8+NAIVE      &amp;lt;NA&amp;gt;&#xA;9        4 NK:CD56+16+3+NKT      &amp;lt;NA&amp;gt;&#xA;10       5      T:CD8+NAIVE      &amp;lt;NA&amp;gt;&#xA;11       6               B:      &amp;lt;NA&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-16T22:51:26.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16413" PostTypeId="1" CreationDate="2021-07-17T07:56:19.057" Score="2" ViewCount="84" Body="&lt;p&gt;I would like to use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3276614/&quot; rel=&quot;nofollow noreferrer&quot;&gt;low pass sequencing&lt;/a&gt; to replace a &lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fpls.2019.00544/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;genotyping chip&lt;/a&gt; to be able to detect &lt;a href=&quot;https://en.wikipedia.org/wiki/Variant_(biology)&quot; rel=&quot;nofollow noreferrer&quot;&gt;variants&lt;/a&gt; up to 0.1 % &lt;a href=&quot;https://en.wikipedia.org/wiki/Allele&quot; rel=&quot;nofollow noreferrer&quot;&gt;allele&lt;/a&gt; frequency in available population data. What is the minimum &lt;a href=&quot;https://www.nature.com/articles/nrg3642&quot; rel=&quot;nofollow noreferrer&quot;&gt;depth&lt;/a&gt; I can opt for to retain the &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/8188283/&quot; rel=&quot;nofollow noreferrer&quot;&gt;confidence&lt;/a&gt; of &lt;a href=&quot;https://en.wikipedia.org/wiki/Base_calling&quot; rel=&quot;nofollow noreferrer&quot;&gt;base calls&lt;/a&gt;? Can &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATK&lt;/a&gt; be used to analyse this data?&lt;/p&gt;&#xA;" OwnerUserId="12777" LastEditorDisplayName="user12779" LastEditDate="2021-07-19T15:50:16.170" LastActivityDate="2021-07-19T15:50:16.170" Title="Low pass sequencing has been reported to detect common variants. How low can one go and get reliable data? Is 2X pass sequencing analysis possible?" Tags="|ngs|gwas|gatk|coverage|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16414" PostTypeId="2" ParentId="16413" CreationDate="2021-07-17T11:35:27.690" Score="2" Body="&lt;p&gt;This is a good idea - low pass sequencing has shown to be better than genotype arrays for things such as GWAS or QTL mapping.&lt;/p&gt;&#xA;&lt;p&gt;The primary way of processing low-pass sequencing data is to use imputation - partciularly methods such as GLIMPSE or QUILT and a large reference panel such as the HRC or TopMed in order to refine the genotype calls.&lt;/p&gt;&#xA;&lt;p&gt;This is a figure from &lt;a href=&quot;https://www.nature.com/articles/s41588-020-00756-0.pdf?proof=t%C2%A0&quot; rel=&quot;nofollow noreferrer&quot;&gt;the GLIMPSE paper&lt;/a&gt; showing how you can retain variants of different frequencies at different levels of sequencing depth.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CBbN0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CBbN0.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And a similar image from &lt;a href=&quot;https://www.nature.com/articles/s41588-021-00877-0.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the QUILT paper&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6yTqT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6yTqT.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You might also want to check out some of the methodology in this paper &lt;a href=&quot;http://m.genome.cshlp.org/content/31/4/529&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://m.genome.cshlp.org/content/31/4/529&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="3694" LastEditDate="2021-07-17T14:32:38.147" LastActivityDate="2021-07-17T14:32:38.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16415" PostTypeId="1" CreationDate="2021-07-18T04:56:21.450" Score="1" ViewCount="87" Body="&lt;p&gt;I have googled a &amp;quot;lot&amp;quot;, couldn't find any specific answer to the question. So, I am here seeking for your guidance. My question is similar to &lt;a href=&quot;https://www.biostars.org/p/400176/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;. I have several metagenome (n=30). But for simplicity’s sake, let's assume only one metagenome. I wanted to quantify lasA genes in the metagenome. So far, what I have done (not sure if it's the right approach) is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Download the lasA genes from &lt;a href=&quot;https://www.uniprot.org/uniprot/?query=gene:lasa&amp;amp;format=fasta&amp;amp;force=true&amp;amp;sort=score&quot; rel=&quot;nofollow noreferrer&quot;&gt;uniport&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;I then made the multiple sequence alignment (lasA.msa) of the lasA gene downloaded in the previous step.&lt;/li&gt;&#xA;&lt;li&gt;The protein profile was generated &lt;code&gt;hmmbuild res.hmm lasA.msa&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Genome assembly (contigs) were done with megahit &lt;code&gt;megahit -1 r1.fastq.gz -2 r2.fastq.gz --min-contig-len 1000 -m 0.85 -o ASSEMBLY/ -t 20&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;ORFs were predicted and extracted using &lt;code&gt;getorf&lt;/code&gt; from the contigs (from step 4)&lt;/li&gt;&#xA;&lt;li&gt;The profile (from step 3) was used to search the ORFs using &lt;code&gt;hmmsearch&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Several hits were obtained from &lt;code&gt;hmmsearch&lt;/code&gt; indicating that the lasA gene is present in the metagenome&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Now, my question is, how do I count the number of reads affiliated to the lasA gene? I feel that I need to map the reads to the assembly, but I don't know how to go about that.&lt;/p&gt;&#xA;&lt;p&gt;NOTE: I have used MG-RAST, Humann3, and eggNOG-mapper, but none of them showed the presence of lasA gene, however the above-mentioned hmmsearch was able to find the gene, what could be the reason? Problem with my approach? Or hmmsearch using the protein profile is more sensitive?&lt;/p&gt;&#xA;&lt;p&gt;Thanks and regards&lt;/p&gt;&#xA;" OwnerUserId="12781" LastEditorUserId="12781" LastEditDate="2021-07-18T05:15:47.697" LastActivityDate="2022-09-12T20:41:40.907" Title="How to quantifiy of specific genes from shotgun metagenome?" Tags="|genome|assembly|metagenome|bowtie2|hmmer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16417" PostTypeId="1" CreationDate="2021-07-18T21:47:21.290" Score="1" ViewCount="880" Body="&lt;p&gt;I want to make a python script that can get as input sequence of N amino acids, and the output is a matrix of Nx6 containing 6 features.&#xA;The features are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Computed volume&lt;/li&gt;&#xA;&lt;li&gt;Hydrophobicity&lt;/li&gt;&#xA;&lt;li&gt;Surface charge&lt;/li&gt;&#xA;&lt;li&gt;Secondary structure (SS)&lt;/li&gt;&#xA;&lt;li&gt;Go, InterPro and Pfam type&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I have developed this script :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#Importing Libraries&#xA;import numpy as np&#xA;import pandas as pd&#xA;from Bio.SeqUtils.ProtParam import ProteinAnalysis&#xA;from Bio.SeqUtils.ProtParam import ProtParamData&#xA;from quantiprot.metrics.aaindex import get_aa2volume, get_aa2hydropathy&#xA;from quantiprot.metrics.basic import average&#xA;&#xA;#Input amino acid&#xA; my_seq = input(&amp;quot;Enter Your Sequence : &amp;quot;)&#xA; SEQ = my_seq.upper()&#xA;&#xA;#Analysis&#xA;analysed_seq = ProteinAnalysis(SEQ)&#xA;MW=  analysed_seq.molecular_weight()&#xA;Gravity = analysed_seq.gravy()&#xA;aa_composition = analysed_seq.count_amino_acids()&#xA;aa_percentage=analysed_seq.get_amino_acids_percent()&#xA;HP = analysed_seq.protein_scale(window=7, param_dict=ProtParamData.kd)&#xA;Sec_Str = analysed_seq.secondary_structure_fraction()&#xA;&#xA; #Amino Acid Compsoition Calculation&#xA;  print(&amp;quot;\n&amp;quot; ,&amp;quot;AA count:&amp;quot;,aa_composition,&amp;quot;\n&amp;quot;)&#xA;&#xA;  #Molecular Weight Calculation&#xA;   print(&amp;quot;Molecular Weight : &amp;quot;,MW)&#xA;&#xA;&#xA;   #Gravity Calculation&#xA;    print(&amp;quot;\n&amp;quot;,&amp;quot;Gravity:&amp;quot;,&amp;quot;\n\n&amp;quot;,Gravity)&#xA;    #Hydrophobicty Calculation by using kd scale&#xA;    print(&amp;quot;\n&amp;quot;,&amp;quot;Hydrophobicity:&amp;quot;,&amp;quot;\n\n&amp;quot;,HP)&#xA;    #kd → Kyte &amp;amp; Doolittle Index of Hydrophobicity&#xA;   #Flex → Normalized average flexibility parameters (B-values)&#xA;   #hw → Hopp &amp;amp; Wood Index of Hydrophilicity&#xA;   #em → Emini Surface fractional probability&#xA;&#xA;  #Amino Acid Percent Calculation&#xA;&#xA; print(&amp;quot;\n&amp;quot;,&amp;quot;Percentage of Amino Acids in Protein:&amp;quot;,&amp;quot;\n\n&amp;quot;,aa_composition)&#xA;&#xA;&#xA; #Volume Calculation by using quantiprot library&#xA; vol =  get_aa2volume(analysed_seq)&#xA;print(&amp;quot;\n&amp;quot;,&amp;quot;Volume of amino acids in protein:&amp;quot;,&amp;quot;\n\n&amp;quot;,str(vol))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have calculated volume, Hydrophobicity, and SS but I don't know how can I find the&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Go, InterPro and Pfam type&lt;/li&gt;&#xA;&lt;li&gt;polarity, i.e. surface charge&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9269" LastEditorUserId="3848" LastEditDate="2022-08-05T05:26:10.073" LastActivityDate="2022-08-05T05:26:10.073" Title="Python script to calculate properties of amino acids and storing them in matrix" Tags="|python|proteins|protein-structure|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="16418" PostTypeId="2" ParentId="16410" CreationDate="2021-07-19T20:17:39.510" Score="1" Body="&lt;p&gt;I think you'll need to convert your &lt;code&gt;Array[File?]+&lt;/code&gt; to a &lt;code&gt;Array[File]&lt;/code&gt; before you can use &lt;code&gt;~{sep=&amp;quot; &amp;quot; files}&lt;/code&gt; in the command section.&lt;/p&gt;&#xA;&lt;p&gt;You can use the &lt;code&gt;select_all&lt;/code&gt; function for that, which will only keep valid files.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/openwdl/wdl/blob/main/versions/development/SPEC.md#arrayx-select_allarrayx&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/openwdl/wdl/blob/main/versions/development/SPEC.md#arrayx-select_allarrayx&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="81" LastActivityDate="2021-07-19T20:17:39.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16419" PostTypeId="1" AcceptedAnswerId="17457" CreationDate="2021-07-20T09:43:27.243" Score="3" ViewCount="76" Body="&lt;p&gt;What data structure is used in representing protein structures in computers so that we can apply algorithms?&lt;/p&gt;&#xA;&lt;p&gt;Matrix or Graph or tree?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;P.S.&lt;/strong&gt; I am absolutely new to Bioinformatics.&lt;/p&gt;&#xA;" OwnerUserId="12686" LastActivityDate="2021-07-29T12:32:54.403" Title="How are tertiary protein structures represented in computers?" Tags="|protein-structure|" AnswerCount="1" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16420" PostTypeId="1" CreationDate="2021-07-20T12:44:36.350" Score="2" ViewCount="221" Body="&lt;p&gt;I need to download some of the genomes of human gut microbiome in FASTA format. Anybody has any script to download?&lt;/p&gt;&#xA;" OwnerUserId="12798" LastEditorUserId="650" LastEditDate="2021-07-20T16:16:59.273" LastActivityDate="2021-07-20T16:16:59.273" Title="How to download the whole genome of an organism from NCBI using R or Python?" Tags="|computational-biochemistry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16421" PostTypeId="2" ParentId="16420" CreationDate="2021-07-20T13:12:20.150" Score="3" Body="&lt;p&gt;I'd suggest the &lt;code&gt;bit-dl-ncbi-assemblies&lt;/code&gt; from the &lt;a href=&quot;https://github.com/AstrobioMike/bit&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioinformatics Tools&lt;/a&gt; package.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ bit-dl-ncbi-assemblies&#xA;&#xA; --------------------------------  HELP INFO  --------------------------------- &#xA;&#xA;  This program downloads assembly files for NCBI genomes. It takes as input&#xA;  assembly accessions (either GCA_* or GCF_*) and optionally a specification of&#xA;  which format to download (currently: genbank, fasta, protein, or gff). For&#xA;  version info, run `bit-version`.&#xA;&#xA;    Required input:&#xA;&#xA;        - [-w &amp;lt;file&amp;gt;] single-column file of NCBI assembly accessions&#xA;&#xA;    Optional arguments include:&#xA;&#xA;        - [-f &amp;lt;str&amp;gt;] default: genbank&#xA;                  Specify the desired format. Available options currently&#xA;                  include: genbank, fasta, protein, gff, feature_tab, report, stats.&#xA;&#xA;        - [-j &amp;lt;int&amp;gt; ] default: 1&#xA;                  The number of downloads you'd like to run in parallel. Write speeds&#xA;                  can become problematic generally with around 15 or more, so a max&#xA;                  of 12 will be used to be safe even if more are requested.&#xA;&#xA;    Example usage:&#xA;&#xA;     bit-dl-ncbi-assemblies -w ncbi_accessions.txt -f protein -j 4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alternatively, you can use this code from the &lt;a href=&quot;http://biopython.org/DIST/docs/tutorial/Tutorial.html#sec149&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biopython cookbook&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import os&#xA;from Bio import SeqIO&#xA;from Bio import Entrez&#xA;&#xA;Entrez.email = &amp;quot;A.N.Other@example.com&amp;quot;  # Always tell NCBI who you are&#xA;&#xA;seq_ids = [&amp;quot;EU490707&amp;quot;] # Your accessions go in this list&#xA;for sid in seq_ids:&#xA;    filename = f&amp;quot;{sid}.fasta&amp;quot;&#xA;    if not os.path.isfile(filename):&#xA;        # Downloading...&#xA;        net_handle = Entrez.efetch(&#xA;            db=&amp;quot;nucleotide&amp;quot;, id=sid, rettype=&amp;quot;fasta&amp;quot;, retmode=&amp;quot;text&amp;quot;&#xA;        )&#xA;        out_handle = open(filename, &amp;quot;w&amp;quot;)&#xA;        out_handle.write(net_handle.read())&#xA;        out_handle.close()&#xA;        net_handle.close()&#xA;        print(&amp;quot;Saved&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6628" LastActivityDate="2021-07-20T13:12:20.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16423" PostTypeId="1" CreationDate="2021-07-20T17:25:12.567" Score="1" ViewCount="40" Body="&lt;p&gt;I have done gene set enrichment analysis which is now straight forward. Such as taking gene set and making a query. For microbiome data it is not that straight forward which I found out.&lt;/p&gt;&#xA;&lt;p&gt;Step done so far..&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Ran Kraken2&lt;/li&gt;&#xA;&lt;li&gt;Generated OTU table&lt;/li&gt;&#xA;&lt;li&gt;Downstream analysis using phyloseq.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Now if I have to the functional enrichment from start I would need to use qiime2 which I didn't do earlier.&lt;/p&gt;&#xA;&lt;p&gt;Now the other option after reading through paper I found kegg can also perform the microbiome functional enrichment&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;ISSUE&lt;/strong&gt;&lt;br /&gt;&#xA;Unlike other database I'm not sure where do I put query in KEGG for the bacterial genus level whatever found.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion how to proceed would be really helpful&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="650" LastEditDate="2021-07-20T19:33:09.993" LastActivityDate="2021-07-20T19:33:09.993" Title="Doing functional enrichment of microbiome data" Tags="|nanopore|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16424" PostTypeId="2" ParentId="13412" CreationDate="2021-07-21T10:27:54.600" Score="1" Body="&lt;p&gt;There are two things that I notice from these plots that are unusual for GSEA analyses that people do:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The zero cross is at the extreme end of the plot, meaning that &lt;em&gt;all&lt;/em&gt; selected genes have a positive test statistic. I think it's a good idea to separate out the positive and negative test statistics (because then the zero point cross is at zero on the Y axis), but other people disagree with me, preferring that the plots show &lt;em&gt;all&lt;/em&gt; genes, rather than positive or negative genes. In any case, GSEA seems to work best when the ranked gene list is unfiltered.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The &amp;quot;negative phenotype&amp;quot; plot has an unusual shape, with high enrichment score boosts for what should be low-ranked genes (and, now that I look at it, the first &amp;quot;positive phenotype&amp;quot; plot has similar issues). Most often, I see GSEA plots that have gradually decreasing enrichment score boosts as the graph moves further down the list of genes. The &amp;quot;negative phenotype&amp;quot; plot also doesn't have any sign changes; the bottom bits should be blue, not red, if the flip has been done correctly.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I suspect what's happened here is something like that there is no test statistic in the list of ranked genes, and the gene rank is being used as a distinguishing statistic. For the GSEA to look &amp;quot;normal&amp;quot;, the gene list should have a field that indicates signed difference (e.g. mean expression of mutant minus mean expression of control). With a list presented in this fashion with signed difference, it doesn't really make sense to compare mutant and control phenotypes separately, as the test statistic incorporates both at once.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-21T10:27:54.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16425" PostTypeId="1" AcceptedAnswerId="16438" CreationDate="2021-07-21T14:07:08.373" Score="1" ViewCount="436" Body="&lt;p&gt;In DESeq2 you can identify your differentially expressed (DE) genes using the results function. I noticed people in my lab using the results() function with the minimum number of arguments supplied (below) and then converting the resulting object to a data frame filtering for the genes they considered DE (where they considered a fold change of +/- 2 and a p-value of 0.05 to be DE).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;res1 &amp;lt;- results(dds, contrast(condition, treated, untreated))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alternatively, I noticed that the results function allows you to set both the P-value and minimum fold change when you use the 'altHypothesis' argument (below). The resulting list from using this method is considerably shorter than the list from manually filtering for the same P-value and fold change. I am aware that that by using the 'altHypothesis' argument it comparisons are made using a Wald test rather than a LRT test, however, I am completely unfamiliar with these two tests and I was not able to understand their significant differences in the context of RNA-seq analysis.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;res1 &amp;lt;- results(dds, contrast(condition, treated, untreated), alpha = 0.1, lfcThreshold = 1, altHypothesis = &amp;quot;greaterAbs&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Due to the difference in size of the lists, and the goal of being consistent moving forward for the lab, does anyone have any insight as too which one is better? Neither myself or my lab are particularly used to this analysis but we would like to try to follow best practices as much as possible.&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="12803" LastEditorUserId="650" LastEditDate="2021-07-21T16:32:13.497" LastActivityDate="2023-03-06T18:43:16.523" Title="DESeq2. Which is better: using the altHypothesis argument in the results function or manually filtering for your desired P-value and Log2FoldChange?" Tags="|rna-seq|bioconductor|deseq2|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16427" PostTypeId="1" CreationDate="2021-07-22T12:06:42.033" Score="2" ViewCount="2511" Body="&lt;p&gt;My workflow for downloading data from SRA has been the following:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Access &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Traces/study/?&quot; rel=&quot;nofollow noreferrer&quot;&gt;SRA Run Selector&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Enter the accession number for the project of interest.&lt;/li&gt;&#xA;&lt;li&gt;Download &amp;quot;Accession List&amp;quot; for the &amp;quot;Total&amp;quot; data. The accession list is basically, as the name says, a &lt;code&gt;.txt&lt;/code&gt; file listing all accession numbers for each run in a given bioproject.&lt;/li&gt;&#xA;&lt;li&gt;For each accession number in the accession list I run the following command:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ fastq-dump --accession &amp;lt;accesion_number&amp;gt; --outdir .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How to download data from SRA in Linux systems via the command line? What are fast and scalable ways of achieving this? What should one know before downloading data?&lt;/p&gt;&#xA;&lt;p&gt;Although there are some similar discussions in this StackExchange (eg. &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/13802/trying-to-retrieve-sra-data-using-sra-explorer&quot;&gt;Trying to retrieve sra data using sra explorer&lt;/a&gt;, &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/11166/downloading-dataset-from-sra-solid-platform&quot;&gt;Downloading dataset from SRA (SOLiD Platform)&lt;/a&gt;, &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/12937/downloading-sra-files-from-aws&quot;&gt;Downloading SRA Files from AWS&lt;/a&gt;, etc).&lt;/p&gt;&#xA;&lt;p&gt;The discussion on &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/411/whats-the-best-way-to-download-data-from-the-sra-is-it-really-this-slow?rq=1&quot;&gt;What's the best way to download data from the SRA? Is it really this slow?&#xA;&lt;/a&gt; seems like a good starting point, but I wasn't able to find a straightforward guide on how to download data from SRA, e believe that a well thought and complete answer would help beginners alike. I am preparing an answer to the problem myself, but I thought that the community would probably give some interesting insights (:&lt;/p&gt;&#xA;" OwnerUserId="12644" LastEditorUserId="12644" LastEditDate="2021-07-23T09:52:35.703" LastActivityDate="2021-07-24T20:18:56.370" Title="How to download data from SRA in Linux systems via the command line?" Tags="|genomics|public-databases|transcriptome|data-download|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16428" PostTypeId="2" ParentId="16427" CreationDate="2021-07-22T12:39:23.367" Score="2" Body="&lt;p&gt;So far, &lt;strong&gt;grabseqs&lt;/strong&gt; is the easiest option, a wrapper for &lt;code&gt;fastq_dump&lt;/code&gt; and &lt;code&gt;fasterq-dump&lt;/code&gt;, you can install it with conda, I recommend you to use an environment:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda create -n grabseqs -c louiejtaylor -c bioconda -c conda-forge grabseqs&#xA;&#xA;conda activate grabseqs&#xA;&#xA;grabseqs sra SRRNNNNNNNN&#xA;&#xA;conda deactivate&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And download directly projects, runs, and bioprojects from SRA with &lt;code&gt;grabseqs sra SRRNNNNNNNN&lt;/code&gt;. Check it out here: &lt;a href=&quot;https://github.com/louiejtaylor/grabseqs&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/louiejtaylor/grabseqs&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can install in a different way just in case the first one doesn't work:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda create -n grabseqs -yc bioconda parallel-fastq-dump pigz &#xA;conda activate grabseqs &#xA;pip install grabseqs&#xA;&#xA;grabseqs sra SRRNNNNNNNN&#xA;&#xA;conda deactivate&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4282" LastEditorUserId="4282" LastEditDate="2021-07-24T20:18:56.370" LastActivityDate="2021-07-24T20:18:56.370" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="16433" PostTypeId="1" CreationDate="2021-07-23T07:22:25.710" Score="0" ViewCount="950" Body="&lt;p&gt;I am new to bioinformatics and programming. I tried to convert my fastq files to fasta file, but got this. What could be wrong?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(base) jszheng@Greater:~/seqtk$ seqtk seq -a KF0124_1_1.fastq &amp;gt; KF0124_1_1.fasta&#xA;[E::stk_seq] failed to open the input file/stream.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12817" LastEditorUserId="3541" LastEditDate="2021-07-23T09:26:04.327" LastActivityDate="2022-08-18T01:07:55.857" Title="Converting Fastq files to Fasta files on Ubuntu" Tags="|fasta|genome|sequence-analysis|fastq|linux|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16434" PostTypeId="1" CreationDate="2021-07-23T12:33:00.870" Score="0" ViewCount="91" Body="&lt;p&gt;I have a pdb file of a protein structure, and I would like to programmatically select certain peptides and label them in an annotated pdb file output, so that when the output pdb file is opened again by one of the 3D viewers, one can highlight the annotated peptides.&lt;/p&gt;&#xA;&lt;p&gt;I can do this interactively using UCSF Chimera or &lt;a href=&quot;https://www.rcsb.org/3d-view&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/3d-view&lt;/a&gt;,&lt;/p&gt;&#xA;&lt;p&gt;and with Chimera, I can even save a session which is a python file that contains the lines of my manual annotations as lists of coordinates.&lt;/p&gt;&#xA;&lt;p&gt;whereas in RCSB, saving a session produces a molx compressed file that includes hashes such as this one, where &lt;code&gt;myseq&lt;/code&gt; is defined by a range within the file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;              {&#xA;                &amp;quot;parent&amp;quot;: &amp;quot;yNaD8QxiUNHaHCvxrC5rew&amp;quot;,&#xA;                &amp;quot;transformer&amp;quot;: &amp;quot;ms-plugin.structure-component&amp;quot;,&#xA;                &amp;quot;params&amp;quot;: {&#xA;                  &amp;quot;type&amp;quot;: {&#xA;                    &amp;quot;name&amp;quot;: &amp;quot;bundle&amp;quot;,&#xA;                    &amp;quot;params&amp;quot;: {&#xA;                      &amp;quot;hash&amp;quot;: 1673297064,&#xA;                      &amp;quot;elements&amp;quot;: [&#xA;                        {&#xA;                          &amp;quot;groupedUnits&amp;quot;: [&#xA;                            [&#xA;                              0&#xA;                            ]&#xA;                          ],&#xA;                          &amp;quot;set&amp;quot;: [],&#xA;                          &amp;quot;ranges&amp;quot;: [&#xA;                            1669,&#xA;                            1777&#xA;                          ]&#xA;                        }&#xA;                      ]&#xA;                    }&#xA;                  },&#xA;                  &amp;quot;nullIfEmpty&amp;quot;: true,&#xA;                  &amp;quot;label&amp;quot;: &amp;quot;myseq2&amp;quot;&#xA;                },&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So I want to do the same programmatically, e.g.:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;annotate_sequences --input file.pdb --sequences 1:ARNDC,2:CDNRA,foo:VYWTSP &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I expect an output file where these three peptides are annotated with the labels &lt;code&gt;1&lt;/code&gt;, &lt;code&gt;2&lt;/code&gt;, and &lt;code&gt;foo&lt;/code&gt;, so when someone opens the file in a 3D structure viewer, then can select/deselect the annotations.&lt;/p&gt;&#xA;&lt;p&gt;Any ideas on how to do this?&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="180" LastEditDate="2021-07-26T15:04:54.617" LastActivityDate="2021-07-26T15:04:54.617" Title="programmatically select and label peptides within pdb file" Tags="|protein-structure|pdb|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16435" PostTypeId="2" ParentId="16434" CreationDate="2021-07-23T15:46:38.393" Score="2" Body="&lt;p&gt;There is no consensus on how different programs do representations (&lt;a href=&quot;https://github.com/matteoferla/MichelaNGLo-transpiler/blob/master/docs/conversion.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;example of NGL vs. PyMOL&lt;/a&gt;) or what they are called so there is not official way to store selections.&lt;/p&gt;&#xA;&lt;p&gt;Annotations can be &lt;code&gt;REMARK&lt;/code&gt; lines, which have a &lt;a href=&quot;https://www.wwpdb.org/documentation/file-format-content/format33/remarks1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;specific format&lt;/a&gt; or simply lines after a hash at the bottom of the file get ignored by most viewers but that specific program may read. Here is a scored Rosetta PDB file for example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;HEADER                                            14-JUN-21   XXXX              &#xA;EXPDTA    THEORETICAL MODEL                                                     &#xA;REMARK 220                                                                      &#xA;REMARK 220 EXPERIMENTAL DETAILS                                                 &#xA;REMARK 220  EXPERIMENT TYPE                : THEORETICAL MODELLING              &#xA;REMARK 220  DATE OF DATA COLLECTION        : 14-JUN-21                          &#xA;REMARK 220                                                                      &#xA;REMARK 220 REMARK: MODEL GENERATED BY ROSETTA                                   &#xA;REMARK 220  VERSION 2020.49+release.fac69dd    &#xA;ATOM      1  N   MET A   1     107.793 134.398   6.022  1.00  0.00           N  &#xA;ATOM      2  CA  MET A   1     108.468 134.162   7.299  1.00  0.00           C  &#xA;ATOM      3  C   MET A   1     108.644 135.405   8.219  1.00  0.00           C  &#xA;ATOM      4  O   MET A   1     108.050 135.416   9.303  1.00  0.00           O  &#xA;ATOM      5  CB  MET A   1     109.782 133.398   7.080  1.00  0.00           C  &#xA;ATOM      6  CG  MET A   1     110.650 133.262   8.335  1.00  0.00           C  &#xA;ATOM      7  SD  MET A   1     112.224 132.454   8.016  1.00  0.00           S  &#xA;...&#xA;# All scores below are weighted scores, not raw scores.&#xA;#BEGIN_POSE_ENERGIES_TABLE &#xA;label fa_atr fa_rep fa_sol fa_intra_rep fa_intra_sol_xover4 lk_ball_wtd fa_elec pro_close hbond_sr_bb hbond_lr_bb hbond_bb_sc hbond_sc dslf_fa13 omega fa_dun p_aa_pp yhh_planarity ref rama_prepro total&#xA;weights 1 0.55 1 0.005 1 1 1 1.25 1 1 1 1 1.25 0.4 0.7 0.6 0.625 1 0.45 NA&#xA;pose -17624.7 2135.24 10828.2 38.8161 591.762 -521.596 -5634.4 9.63107 -1337.76 -585.258 -616.219 -453.443 0 197.59 3291.97 -602.3 2.74143 1088.19 99.1601 -9092.4&#xA;MET:NtermProteinFull_1 -2.63316 0.99093 2.70981 0.01206 0.04738 -0.17936 0.08728 0.00151 0 0 0 0 0 0.35192 1.9909 0 0 1.65735 0 5.0366&#xA;                 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3 id=&quot;hacked-b-factors-2yn8&quot;&gt;Hacked b-factors&lt;/h3&gt;&#xA;&lt;p&gt;Another way is to store numerical data as b-factors. The AlphaFold2 data for example stores pLDDT in the b-factor column, while consurf stores conservation score there. So a boolean 1=show, 0=hide could be done for example, which would allow one to select these, e.g. in PyMOL:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# store chosen selection...&#xA;alter chosen, &amp;quot;b=1&amp;quot;&#xA;sort&#xA;save foo.pdb&#xA;&#xA;# load it back...&#xA;select chosen, b&amp;lt;0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Although this makes no sense as one can save a session file...&#xA;Also note, that for multiple selections, you'd need create different models, but whatever happens the other non-coordinate column, occupancy, should not be touched.&lt;/p&gt;&#xA;&lt;h3 id=&quot;sharing-buuz&quot;&gt;Sharing&lt;/h3&gt;&#xA;&lt;p&gt;If the problem is that you want to share a protein with someone and want to let the recipient know about features of interest but are using different software than you, you may want to consider &lt;a href=&quot;https://michelanglo.sgc.ox.ac.uk&quot; rel=&quot;nofollow noreferrer&quot;&gt;Michelanglo&lt;/a&gt;, a website I wrote that allows one to share a webpage with an interactive protein viewer (using NGL.js, hence the name) and a side description with custom annotations that control the protein representation (&lt;a href=&quot;https://michelanglo.sgc.ox.ac.uk/r/basics&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2021-07-24T08:58:27.440" LastActivityDate="2021-07-24T08:58:27.440" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16436" PostTypeId="1" AcceptedAnswerId="16446" CreationDate="2021-07-23T18:18:58.143" Score="0" ViewCount="29" Body="&lt;p&gt;I am a statistics student trying to find data suitable for a particular method. From what I have read so far, it is of interest to determine if variants within a chromosome are associated with a phenotype. I would like to understand this more and if possible, find an example of this application. In particular, what test statistic is used when associating each chromosome variant to a phenotype? Also, is it possible to gain access to examples of SNP Chip data fitting this context?&lt;/p&gt;&#xA;" OwnerUserId="12822" LastActivityDate="2021-07-25T22:25:39.100" Title="Where can I find a basic example for testing association between a phenotype and each of the variants of a chromosome?" Tags="|statistics|database|public-databases|snp-chip|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16437" PostTypeId="2" ParentId="16433" CreationDate="2021-07-23T21:47:52.403" Score="2" Body="&lt;p&gt;&lt;em&gt;Answer from @haci, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;You should run your command within that directory (the directory where the fastq file is), this does not seem to be the case as far as I see. Alternatively you can provide the absolute path of your fastq file.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;cd&lt;/code&gt; into the directory where the fastq file is (you should be able to see your file of interest when you type &lt;code&gt;ls&lt;/code&gt;) and then run the command in your question:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cd ./genome_diirt&#xA;seqtk seq -a KF0124_1_1.fastq &amp;gt; KF0124_1_1.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alternatively you can provide the absolute path of your fastq file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqtk seq -a ./genome_diirt/KF0124_1_1.fastq &amp;gt; ./genome_diirt/KF0124_1_1.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-23T21:47:52.403" CommentCount="0" CommunityOwnedDate="2021-07-23T21:47:52.403" ContentLicense="CC BY-SA 4.0" />
  <row Id="16438" PostTypeId="2" ParentId="16425" CreationDate="2021-07-23T22:08:40.000" Score="1" Body="&lt;p&gt;If you have an existing understanding of a fold change that you would consider to be biologically relevant, it will be better to use the lfcThreshold for comparisons. Working out that fold change is the hard bit.&lt;/p&gt;&#xA;&lt;p&gt;When using the Wald test, the p-value is associated specifically with that threshold, rather than the difference from zero.&lt;/p&gt;&#xA;&lt;p&gt;This is explained in the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#tests-of-log2-fold-change-above-or-below-a-threshold&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2 Analysis Vignette&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;It is also possible to provide thresholds for constructing Wald tests of significance. Two arguments to the &lt;em&gt;results&lt;/em&gt; function allow for threshold-based Wald tests: &lt;code&gt;lfcThreshold&lt;/code&gt;, which takes a numeric of a non-negative threshold value, and &lt;code&gt;altHypothesis&lt;/code&gt;, which specifies the kind of test. Note that the alternative hypothesis is specified by the user, i.e. those genes which the user is interested in finding, and the test provides &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; values for the null hypothesis, the complement of the set defined by the alternative. The &lt;code&gt;altHypothesis&lt;/code&gt; argument can take one of the following four values, where &lt;span class=&quot;math-container&quot;&gt;$β$&lt;/span&gt; is the log2 fold change specified by the &lt;code&gt;name&lt;/code&gt; argument, and &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is the &lt;code&gt;lfcThreshold&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;greaterAbs - &lt;span class=&quot;math-container&quot;&gt;$|β|&amp;gt;x$&lt;/span&gt; - tests are two-tailed&lt;/li&gt;&#xA;&lt;li&gt;lessAbs - &lt;span class=&quot;math-container&quot;&gt;$|β|&amp;lt;x$&lt;/span&gt; - p values are the maximum of the upper and lower tests&lt;/li&gt;&#xA;&lt;li&gt;greater - &lt;span class=&quot;math-container&quot;&gt;$β&amp;gt;x$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;less - &lt;span class=&quot;math-container&quot;&gt;$β&amp;lt;−x$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-23T22:08:40.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16439" PostTypeId="1" CreationDate="2021-07-24T15:18:26.177" Score="2" ViewCount="932" Body="&lt;p&gt;Running SCTransform on my seurat object produces the warning:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Warning in theta.ml(y = y, mu = fit$fitted) : iteration limit reached&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What does this warning mean? Can I trust the output?&lt;/p&gt;&#xA;&lt;p&gt;SCTransform runs through the data producing this warning until the progress bar reaches 100%. The resulting seurat object &lt;strong&gt;does&lt;/strong&gt; have an SCT assay and I have successfully done downstream analysis with it. I am concerned that my SCTransform has not successfully done what it is supposed to do. Is there a way to cross-check?&lt;/p&gt;&#xA;" OwnerUserId="9088" LastEditorUserId="650" LastEditDate="2021-07-26T18:21:39.193" LastActivityDate="2021-07-27T04:47:35.553" Title="SCTransform Warning: in theta.ml(y = y, mu = fit$fitted) : iteration limit reached" Tags="|scrnaseq|seurat|sctransform|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16440" PostTypeId="2" ParentId="16439" CreationDate="2021-07-24T16:07:48.157" Score="2" Body="&lt;p&gt;The explanation can be found in link posted by &lt;strong&gt;ATpoint&lt;/strong&gt; in his comment:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I get at least 10 posts on GitHub, GitMemory and Biostars just on the top page.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/satijalab/seurat/issues/1378&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/satijalab/seurat/issues/1378&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/satijalab/seurat/issues/2578&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/satijalab/seurat/issues/2578&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/satijalab/seurat/issues/1378&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/satijalab/seurat/issues/1378&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.biostars.org/p/450394/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/450394/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The first author of the SCTransform paper, Christoph Hafemeister had posted this answer in a similar query on the satijalab GitHub page &lt;a href=&quot;https://github.com/satijalab/seurat/issues/1378&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/satijalab/seurat/issues/1378&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;These warnings are related to the estimation of the theta parameter of the Negative Binomial regression model used to describe the molecule count distribution of each gene. When a gene has very few non-zero observations estimation of that parameter is not reliable and that's what the message indicates. This is not a problem, since the estimated theta parameters are not used directly for normalization but are regularized (sharing information across genes and thus lowering uncertainty).&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="9088" LastEditorUserId="73" LastEditDate="2021-07-27T04:47:35.553" LastActivityDate="2021-07-27T04:47:35.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16446" PostTypeId="2" ParentId="16436" CreationDate="2021-07-25T22:20:20.107" Score="0" Body="&lt;p&gt;There are a very large number of such methods. I would recommend googling &amp;quot;GWAS&amp;quot; or &amp;quot;genome wide association analysis&amp;quot; and you will find many many results.&lt;/p&gt;&#xA;&lt;p&gt;One relatively simple example would be &lt;a href=&quot;https://easygwas.ethz.ch/down/dataset/download/1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this plant genotype-phenotype dataset&lt;/a&gt;, associated with &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/20336072/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this publication&lt;/a&gt;. Some basic summary description of the GWAS dataset can be found &lt;a href=&quot;https://easygwas.ethz.ch/data/public/species/view/1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. By now the methods in this paper are very old and naive seeming, but it gives an idea for how people approached this problem in the golden age of GWAS. The data are small and simple and full of strong associations. Additionally, the phenotype measurements are based on isogenic inbred lines, which are more or less homozygous, making the analysis much easier, interpretation a lot simpler.&lt;/p&gt;&#xA;&lt;p&gt;Another point in favor of using the plant data is that there is actually some strong genotype-phenotype association with a really strong mechanistic basis, like the various flowering time associations. For better or for worse, such unambiguous associations are often absent in e.g. human phenotypes of interest.&lt;/p&gt;&#xA;&lt;p&gt;For some background on the general problem of controlling for population structure in GWAS, see &lt;a href=&quot;https://www.genetics.org/content/178/3/1709&quot; rel=&quot;nofollow noreferrer&quot;&gt;the old EMMA paper&lt;/a&gt;. A much more recent example of current statistical methodology that has been making the rounds on Twitter can be found &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2020.08.04.236703v2.abstract&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For very large catalogs of GWAS data in humans, I can point you &lt;a href=&quot;https://www.ebi.ac.uk/gwas/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; or &lt;a href=&quot;https://atlas.ctglab.nl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. &lt;a href=&quot;https://www.ukbiobank.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The UK Biobank dataset&lt;/a&gt; in particular is very commonly used and quite high quality.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2021-07-25T22:25:39.100" LastActivityDate="2021-07-25T22:25:39.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16447" PostTypeId="1" AcceptedAnswerId="16455" CreationDate="2021-07-26T11:04:22.267" Score="1" ViewCount="337" Body="&lt;p&gt;I have &lt;em&gt;&lt;strong&gt;four PBMC samples from 10X scRNA-seq&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; cancer&#xA;An object of class Seurat &#xA;36601 features across 18338 samples within 1 assay &#xA;Active assay: RNA (36601 features, 0 variable features)&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And this is Seurat QC plot&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Hqj48.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Hqj48.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If I am not wrong, samples were of relatively low quality, with gene expression data revealing the presence of mitochondrial genes, as well as MALAT-1, which are suggestive of poor sample quality (dead/dying cells).&lt;/p&gt;&#xA;&lt;p&gt;Any way by this plot I filtered cells to remove data that had more than &lt;code&gt;20 &amp;quot;percent&amp;quot; mitochondrial expression&lt;/code&gt; , &lt;code&gt;&amp;gt; 2000 features&lt;/code&gt; or &lt;code&gt;&amp;lt; 100 features&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cancer &amp;lt;- subset(cancer, subset = nFeature_RNA &amp;gt; 100 &amp;amp; nFeature_RNA &amp;lt; 2000 &amp;amp; percent.mt &amp;lt; 20)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;By this I lost most of cells&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; cancer&#xA;    An object of class Seurat &#xA;    36601 features across 6,883 samples within 1 assay &#xA;    Active assay: RNA (36601 features, 0 variable features)&#xA;    &amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Please, can somebody experienced in scRNA-seq inspect this plot and my thresholds and tell me if I am wrong, any suggestion, if losing this number of cells is normal&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="9449" LastActivityDate="2021-07-28T00:30:47.550" Title="Help with Seurat QC ambiguity" Tags="|scrnaseq|seurat|10x-genomics|qc|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16448" PostTypeId="2" ParentId="16447" CreationDate="2021-07-26T14:18:37.000" Score="1" Body="&lt;p&gt;From the graphs, you're losing most of the cells through the &lt;code&gt;percent.mitochondrial&lt;/code&gt; threshold. The distribution is so broad on that mitochondrial plot that I probably wouldn't filter for mitochondrial expression (except possibly at ~95%). If I did have a lower threshold, I'd put it at around 50% rather than 20%, because that's where there's a change in density on the mitochondrial plot.&lt;/p&gt;&#xA;&lt;p&gt;This suggests that there's a very large mitochondria-rich population in the PBMC dataset. I don't have a good understanding of biology and cell populations, but if I were presented with these data I'd probably try clipping out those cells with 65-95% mtDNA and see if there is a strong enrichment for markers that correspond to any particular cell population (e.g. platelets).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-26T14:18:37.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16449" PostTypeId="1" CreationDate="2021-07-26T17:10:33.627" Score="1" ViewCount="41" Body="&lt;p&gt;I  am trying to look for miRNA targets using a file called &lt;code&gt;Conservedfamily.txt&lt;/code&gt; from the Zebrafish target scan fish website. I have written a python program to extract each of the miRNAs which works fine. However, for a big list of microRNAs, I must provide a name every time in line 22 and get one individual output file. I am thinking is there any option to parallelize the process so that somehow, I can provide a list of all microRNAs and obtain all my targets at the same time. The python code is attached below. I have &lt;a href=&quot;https://www.biostars.org/p/9481491/&quot; rel=&quot;nofollow noreferrer&quot;&gt;posted the question in biostars&lt;/a&gt; but go no reply from them for 4 days. Hence I am posting the question in Stack exchange. The python code is attached below.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/python&#xA;&#xA;#Import relevant modules&#xA;&#xA;import sys&#xA;&#xA;conservedfamily = open(sys.argv[1], 'r')&#xA;targetgenelist = open(sys.argv[2], 'w')&#xA;targetgeneid = open(sys.argv[3], 'w')&#xA;&#xA;geneid = []&#xA;&#xA;targetgenelist.write('miR family' + '\t' + 'Gene ID' + '\t' + 'Gene Symbol' + '\t' + 'Transcript ID' + '\t' + 'Species ID' + '\t' + 'UTR start' + '\t' + 'UTR end' + '\t' + 'MSA start' + '\t' + 'MSA end' + '\t' + 'Seed match' + '\n')&#xA;&#xA;for line in conservedfamily:&#xA;  splitline = line.strip().split('\t')&#xA;&#xA;  if splitline[0] == 'miR-210':&#xA;    targetgenelist.write(line)&#xA;    if not splitline[1] in geneid:&#xA;      geneid.append(splitline[1])&#xA;      targetgeneid.write(splitline[1] + '\n')&#xA;&#xA;conservedfamily.close()&#xA;targetgenelist.close()&#xA;targetgeneid.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;python miR99target.py 'Conserved_Family_Info.txt' 'miR-210targetlist.txt' 'miR-210targetid.txt'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9659" LastEditorUserId="650" LastEditDate="2021-07-26T18:24:15.300" LastActivityDate="2022-04-23T05:06:35.577" Title="Parallelizing microRNA targets" Tags="|rna-seq|genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16450" PostTypeId="2" ParentId="16449" CreationDate="2021-07-27T02:42:59.217" Score="2" Body="&lt;p&gt;I'd suggest loading in all of the miRNA tags as a &lt;code&gt;set()&lt;/code&gt;. Then, you can just check the set membership each time.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/python&#xA;&#xA;#Import relevant modules&#xA;&#xA;import sys&#xA;&#xA;&#xA;conservedfamily = open(sys.argv[1], 'r')&#xA;targetgenelist = open(sys.argv[2], 'w')&#xA;targetgeneid = open(sys.argv[3], 'w')&#xA;&#xA;&#xA;# Create a set of all miRNA tags. Each tag should be on its own line.&#xA;with open(sys.argv[4], 'r') as miRNAlist:&#xA;    miRNA_set = {line.strip() for line in miRNAlist}&#xA;&#xA;geneid = []&#xA;&#xA;targetgenelist.write('miR family' + '\t' + 'Gene ID' + '\t' + 'Gene Symbol' + '\t' + 'Transcript ID' + '\t' + 'Species ID' + '\t' + 'UTR start' + '\t' + 'UTR end' + '\t' + 'MSA start' + '\t' + 'MSA end' + '\t' + 'Seed match' + '\n')&#xA;&#xA;for line in conservedfamily:&#xA;  splitline = line.strip().split('\t')&#xA;&#xA;  # Check if current tag is in the desired set&#xA;  if splitline[0] in miRNA_set:&#xA;    targetgenelist.write(line)&#xA;    if not splitline[1] in geneid:&#xA;      geneid.append(splitline[1])&#xA;      targetgeneid.write(splitline[1] + '\n')&#xA;&#xA;conservedfamily.close()&#xA;targetgenelist.close()&#xA;targetgeneid.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;One additional comment:&#xA;I'd suggest using the &lt;a href=&quot;https://docs.python.org/3/library/csv.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;python module for csv&lt;/a&gt; reading/writing, even for quick and dirty scripts as well as using &lt;a href=&quot;https://stackoverflow.com/a/36559849/7092484&quot;&gt;context managers&lt;/a&gt; for reading/writing files.&lt;/p&gt;&#xA;" OwnerUserId="6628" LastEditorUserId="6628" LastEditDate="2021-07-27T02:49:01.303" LastActivityDate="2021-07-27T02:49:01.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16451" PostTypeId="2" ParentId="14803" CreationDate="2021-07-27T09:53:17.830" Score="1" Body="&lt;p&gt;Here is a &lt;a href=&quot;https://github.com/tidymodels/corrr/issues/29&quot; rel=&quot;nofollow noreferrer&quot;&gt;suggested workaround&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Generate a p-value table using one of these tools:&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;RcmdrMisc::rcorr.adjust()&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;psych::corr.test()&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Then use &lt;code&gt;corrr:as_cordf()&lt;/code&gt; to generate the tidy dataframe with the same structure as a regular corrr correlation table.&lt;/p&gt;&#xA;&lt;p&gt;Personally, I have used the &lt;code&gt;Hmisc::rcorr()&lt;/code&gt; package, and it works well:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Hmisc::rcorr() %&amp;gt;% `[[`('P') %&amp;gt;% corrr::as_cordf()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12844" LastEditorUserId="12844" LastEditDate="2021-08-18T14:26:11.633" LastActivityDate="2021-08-18T14:26:11.633" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="16452" PostTypeId="2" ParentId="11557" CreationDate="2021-07-27T10:44:00.697" Score="1" Body="&lt;p&gt;Both &lt;a href=&quot;https://biopython.org/docs/1.75/api/Bio.SeqUtils.MeltingTemp.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bio.SeqUtils.MeltingTemp&lt;/a&gt; of Biopython and &lt;a href=&quot;https://primer3.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Primer3&lt;/a&gt; offers different methods/algorithms to calculate the Tm value.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2021-07-27T10:44:00.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16453" PostTypeId="1" AcceptedAnswerId="17459" CreationDate="2021-07-27T11:49:50.110" Score="0" ViewCount="2331" Body="&lt;p&gt;I have a Seurat object of four cancers and four controls&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; colnames(pbmc@meta.data)&#xA; [1] &amp;quot;orig.ident&amp;quot;       &amp;quot;nCount_RNA&amp;quot;       &amp;quot;nFeature_RNA&amp;quot;     &amp;quot;samples&amp;quot;         &#xA; [5] &amp;quot;Condition&amp;quot;        &amp;quot;Gender&amp;quot;           &amp;quot;percent.mt&amp;quot;       &amp;quot;nCount_SCT&amp;quot;      &#xA; [9] &amp;quot;nFeature_SCT&amp;quot;     &amp;quot;SCT_snn_res.0.75&amp;quot; &amp;quot;seurat_clusters&amp;quot;  &amp;quot;SCT_snn_res.0.5&amp;quot; &#xA;&amp;gt; unique(pbmc@meta.data$Condition)&#xA;[1] &amp;quot;control&amp;quot; &amp;quot;cancer&amp;quot; &#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TbG8d.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TbG8d.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want to have a heat map of differentially expressed between cancers and controls like &lt;code&gt;figure 3A&lt;/code&gt; in this paper&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fgene.2020.610682/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.frontiersin.org/articles/10.3389/fgene.2020.610682/full&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZVaJh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZVaJh.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Do you have any idea please how I can do that? I've looked at the &lt;a href=&quot;https://satijalab.org/seurat/articles/integration_introduction.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;integration tutorial&lt;/a&gt;, but can't find the heatmap in it.&lt;/p&gt;&#xA;" OwnerUserId="9449" LastEditorUserId="73" LastEditDate="2021-07-29T23:17:45.060" LastActivityDate="2021-07-29T23:17:45.060" Title="Seurat heatmap for two conditions" Tags="|scrnaseq|seurat|differential-expression|10x-genomics|heatmap|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="16454" PostTypeId="1" CreationDate="2021-07-27T14:50:04.007" Score="2" ViewCount="86" Body="&lt;p&gt;I am running splice variant analysis. I wanted to use NCBI genome but the program works better with ensembl. I am a bit confused on primary vs. top level to use as my reference genome. I also do not know which annotation file to use. And I am not seeing a reference rna download. If anyone can help that would be immensely appreciated.&lt;/p&gt;&#xA;" OwnerUserId="12852" LastActivityDate="2021-09-04T13:01:40.317" Title="Splice variant analysis with ensembl genome/annotation/rna. Which files do I use?" Tags="|sequence-annotation|transcriptome|reference-genome|ensembl|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="16455" PostTypeId="2" ParentId="16447" CreationDate="2021-07-28T00:30:47.550" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @haci, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I think you should not be using a cut-off higher than 20 or 25 for the reasons I explained above.&lt;/p&gt;&#xA;&lt;p&gt;I would advise against using cells with more than 20-25% percent.mito. This metric is used as a proxy for cell death and there is not much point keeping these cells if the cell death is so severe to the point where let's say half the reads map to the mitochondrial genome (with a 50% cut-off). There will be many caveats one being apoptotic cells from different lineages clustering together, not interesting unless you are studying apoptosis related phenomena. In my opinion 6000 good quality cells are better than 10000 apoptotic cells.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-28T00:30:47.550" CommentCount="0" CommunityOwnedDate="2021-07-28T00:30:47.550" ContentLicense="CC BY-SA 4.0" />
  <row Id="17457" PostTypeId="2" ParentId="16419" CreationDate="2021-07-29T12:32:54.403" Score="4" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;comments from above fleshed out.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Both: A graph network is a handy way to represent a molecule, but a matrix makes for fast maths.&lt;/p&gt;&#xA;&lt;p&gt;A graph network is a list of &lt;code&gt;nodes&lt;/code&gt; (atoms), a list of &lt;code&gt;edges&lt;/code&gt;, the connections between nodes and a list of &lt;code&gt;properties&lt;/code&gt; associated with the nodes (symbol, name, index, residue index, residue number, chain, coordinates etc.) and occasionally a list of those of the edges (single, double etc.).&lt;/p&gt;&#xA;&lt;p&gt;A protein is a big molecule. Same rules apply.&#xA;A small compound can be represented with a string, i.e. SMILES, which is a form of graph network even if it looks nothing like one.&#xA;&lt;img src=&quot;https://upload.wikimedia.org/wikipedia/commons/thumb/0/00/SMILES.png/300px-SMILES.png&quot; alt=&quot;SMILES&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;In a mol file and its sdf/mol2 derivatives, a tabular representation is used to store coordinates (&lt;code&gt;properties&lt;/code&gt;) of the atoms of the small molecule (top part below), while the connectivity has an edge-node relationship, i.e. are edges. Note that the index of the properties in the first block is the id of the atom.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;     RDKit          3D&#xA;&#xA; 12 12  0  0  0  0  0  0  0  0999 V2000&#xA;   -5.5310   24.4560   -8.2060 O   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    0.9560   21.4190   -7.6300 O   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -5.4170   23.9250   -6.8750 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    0.2770   19.3290   -8.7330 O   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -3.9460   23.5670   -6.5510 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -0.3270   19.7590   -6.4400 O   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -3.4590   22.5590   -7.5870 N   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -3.5890   23.1360   -8.9920 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -5.0600   23.5190   -9.2240 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -2.0390   22.1150   -7.2780 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -1.6090   21.0030   -8.2230 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -0.0430   20.3530   -7.7350 S   0  0  1  0  0  0  0  0  0  0  0  0&#xA;  1  3  1  0&#xA;  1  9  1  0&#xA;  3  5  1  0&#xA;  9  8  1  0&#xA; 12  2  1  1&#xA; 12  4  1  0&#xA; 12  6  1  0&#xA; 12 11  1  0&#xA;  5  7  1  0&#xA;  7  8  1  0&#xA;  7 10  1  0&#xA; 10 11  1  0&#xA;M  END&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A PDB, CIF, etc. file is the same. It has a per-atom tabular representation of the properties, with an optional connectivity part (&lt;code&gt;CONECT&lt;/code&gt; entries) for non-standard residues (not AAs and DNA).&lt;/p&gt;&#xA;&lt;p&gt;Now when these are loaded into memory, they become an array of atoms and bonds, each with their properties —graph network.&lt;/p&gt;&#xA;&lt;p&gt;However, the coordinates within get often converted into a matrix for speedy operations.&#xA;For example to calculate an atom to atom distance matrix, converting the coordinates with the array of atoms to a matrix can be very easily be converted into a distance matrix via simple matrix calculations.&lt;/p&gt;&#xA;&lt;p&gt;So both matrix representations and graph network representations are used.&lt;/p&gt;&#xA;&lt;p&gt;For 3D visualisation, a pre-existing library is used and bonds and atoms become primitive shapes, whose splines may be connected to each other (in knots/vertices) or simply the bonds shapes clip through the atoms. That is there is not graph relationship.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-07-29T12:32:54.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17458" PostTypeId="1" CreationDate="2021-07-29T16:10:12.913" Score="1" ViewCount="505" Body="&lt;p&gt;I have number of cells in three cell types T, B, M in a Seurat object&lt;/p&gt;&#xA;&lt;p&gt;For two groups of patients, cancers and controls like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; table(Idents(pbmc), pbmc$samples)&#xA;   &#xA;    PN0252_0001 PN0252_0002 PN0252_0003 PN0252_0004 PN0252_0005 PN0252_0006 PN0252_0008&#xA;  T        3550         568         136         268        1835           2        1678&#xA;  B         136         129          11          25         133           0         203&#xA;  M         120          22          18          26         159           0          59&#xA;&amp;gt; dput(table(Idents(pbmc), pbmc$samples))&#xA;structure(c(3550L, 136L, 120L, 568L, 129L, 22L, 136L, 11L, 18L, &#xA;268L, 25L, 26L, 1835L, 133L, 159L, 2L, 0L, 0L, 1678L, 203L, 59L&#xA;), .Dim = c(3L, 7L), .Dimnames = structure(list(c(&amp;quot;T&amp;quot;, &amp;quot;B&amp;quot;, &amp;quot;M&amp;quot;&#xA;), c(&amp;quot;PN0252_0001&amp;quot;, &amp;quot;PN0252_0002&amp;quot;, &amp;quot;PN0252_0003&amp;quot;, &amp;quot;PN0252_0004&amp;quot;, &#xA;&amp;quot;PN0252_0005&amp;quot;, &amp;quot;PN0252_0006&amp;quot;, &amp;quot;PN0252_0008&amp;quot;)), .Names = c(&amp;quot;&amp;quot;, &#xA;&amp;quot;&amp;quot;)), class = &amp;quot;table&amp;quot;)&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Patients &lt;code&gt;PN0252_0001&lt;/code&gt;, &lt;code&gt;PN0252_0002&lt;/code&gt;, &lt;code&gt;PN0252_0003&lt;/code&gt;, &lt;code&gt;PN0252_0004&lt;/code&gt; are controls&lt;/p&gt;&#xA;&lt;p&gt;and &lt;code&gt;PN0252_0005&lt;/code&gt;, &lt;code&gt;PN0252_0006&lt;/code&gt;, &lt;code&gt;PN0252_0008&lt;/code&gt; are cancers&lt;/p&gt;&#xA;&lt;p&gt;I want to look at the difference of the proportion of cells in each cell type between cancer and control&lt;/p&gt;&#xA;&lt;p&gt;But I am confused for getting proportion should I divide the number of cells on total cells in the object or total cells in each group&lt;/p&gt;&#xA;&lt;p&gt;Basically I need a box plot like although here numbers are not correct&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ht4k6.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ht4k6.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9449" LastEditorUserId="650" LastEditDate="2021-07-29T19:40:00.703" LastActivityDate="2021-07-29T19:40:00.703" Title="Making a box plot of the proportion of cells in each cell types in two groups" Tags="|scrnaseq|seurat|10x-genomics|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="17459" PostTypeId="2" ParentId="16453" CreationDate="2021-07-29T23:15:46.260" Score="1" Body="&lt;p&gt;See the &lt;code&gt;DoHeatmap&lt;/code&gt; function in Seurat, which seems to be what that paper has used:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://satijalab.org/seurat/articles/pbmc3k_tutorial.html#cb50&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/articles/pbmc3k_tutorial.html#cb50&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; DoHeatmap(pbmc, features = myGeneList, group.by=&amp;quot;Condition&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-07-29T23:15:46.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17460" PostTypeId="1" AcceptedAnswerId="17461" CreationDate="2021-07-30T09:39:49.167" Score="0" ViewCount="533" Body="&lt;p&gt;I have 2 bam files that belong to the same sample and I merged them with &lt;code&gt;samtools merge&lt;/code&gt;. And after merging, I realized that the merged version is a bit smaller than the sum of the other two separate files' size. I also wonder if I merged &amp;quot;the fastq&amp;quot; files of this sample first and convert it to bam would the result be the same?&lt;/p&gt;&#xA;" OwnerUserId="13878" LastActivityDate="2021-07-30T10:39:44.283" Title="Is it normal to have a smaller bam file after merging bam files?" Tags="|sam|samtools|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17461" PostTypeId="2" ParentId="17460" CreationDate="2021-07-30T09:48:36.053" Score="1" Body="&lt;p&gt;If you merge a lot of BAM files you lose the overhead of the header which depending on the size of your BAM files this can be a significant difference or not.&lt;/p&gt;&#xA;&lt;p&gt;With FASTQ files there should be less difference (as they don't have an overall header). However, the compression ratio can change depending on how you zip your files. [EDIT: I just tested it merging two fastq.gz that were 357MB and 562MB - the result was fastq.gz that was smaller than the sum at only 826MB]&lt;/p&gt;&#xA;&lt;p&gt;No, the BAM header information would be lost in the conversion from BAM -&amp;gt; FASTQ -&amp;gt; BAM.&lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="383" LastEditDate="2021-07-30T10:39:44.283" LastActivityDate="2021-07-30T10:39:44.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17462" PostTypeId="1" CreationDate="2021-07-30T11:58:06.080" Score="0" ViewCount="140" Body="&lt;p&gt;I have some .bam files and I also got featureCounts by Subread. But after getting the featureCounts I realized that among my bam files 2 of them actually belong to the same sample. So I wonder if I can remove the result of these two and after merging bam files add them again into featureCounts without doing all the featureCounts for other samples again.&lt;/p&gt;&#xA;" OwnerUserId="13878" LastActivityDate="2021-07-30T11:58:06.080" Title="Is it possible to add one more sample to the result of featureCounts?" Tags="|subread|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17463" PostTypeId="2" ParentId="15361" CreationDate="2021-07-30T23:17:19.863" Score="3" Body="&lt;p&gt;If you go to the wwPDB ftp site &lt;a href=&quot;https://ftp.wwpdb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.wwpdb.org/&lt;/a&gt; in your browser you can find the biological unit structures (and anything else available in the PDB). The caveat of course is that a biounit structure has to have been generated for your structure.&lt;/p&gt;&#xA;&lt;p&gt;By way of some examples, to get all biological units you could do (note the &lt;code&gt;ftp&lt;/code&gt; instead of &lt;code&gt;https&lt;/code&gt; in front of the URL):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;wget ftp://ftp.wwpdb.org/pub/pdb/data/biounit/PDB/all/*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or to get a version organized in directories:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;wget ftp://ftp.wwpdb.org/pub/pdb/data/biounit/PDB/divided/*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that this would take a few days to download everything, though. After the first download, you can add a &lt;code&gt;-N&lt;/code&gt; to the &lt;code&gt;wget&lt;/code&gt; command to just download new files.&lt;/p&gt;&#xA;&lt;p&gt;For getting just one file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;wget ftp://ftp.wwpdb.org/pub/pdb/data/biounit/PDB/divided/00/100d.pdb1.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="13881" LastActivityDate="2021-07-30T23:17:19.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17464" PostTypeId="1" CreationDate="2021-07-31T03:48:29.087" Score="0" ViewCount="255" Body="&lt;p&gt;I just ran an AlphaFold job for PduK from the &lt;a href=&quot;https://en.wikipedia.org/wiki/Bacterial_microcompartment#PDU_BMCs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pdu BMC&lt;/a&gt; operon and got very strange results including a large circular loop of random coil. According to &lt;a href=&quot;https://www.nature.com/articles/srep36899&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mayer &lt;em&gt;et al.&lt;/em&gt;&lt;/a&gt;, PduK should be semi-triangular protein that forms hexagonal homohexamers. To fix this issue, I am interested in performing homology modeling of PduK with other Pdu proteins (namely PduA, PduB, PduJ, PduK, PduU, and PduT, all of which have crystal structures in the Protein Data Bank) as templates. I was only able to figure out how to use one template .pdb file in &lt;a href=&quot;https://zhanglab.dcmb.med.umich.edu/I-TASSER/&quot; rel=&quot;nofollow noreferrer&quot;&gt;I-TASSER&lt;/a&gt; with its &lt;a href=&quot;https://zhanglab.dcmb.med.umich.edu/I-TASSER/download/README5.1.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;-restraint4&amp;quot; parameter&lt;/a&gt; and was not able to get &lt;a href=&quot;https://salilab.org/modeller/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MODELLER&lt;/a&gt; working for this purpose.&lt;/p&gt;&#xA;&lt;p&gt;My PduK residue sequence is: VKQSLGLLEVSGLALAISCADVMAKAASITLVGLEKTNGSGWMVIKIIGDVASVQAAISTGVSFADQRDGLVAHKVISRPGDGILSHSVTPESESEPAPAPTPVVPHEEIPEDHAAPEAPQDAELISCNLCLDPACPRQKGEPRSLCLHSGKRGEA&lt;/p&gt;&#xA;" OwnerUserId="13882" LastActivityDate="2023-07-21T04:58:46.673" Title="How to use multiple .pdb files as templates for homology modelling?" Tags="|protein-structure|modelling|sequence-homology|computation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17465" PostTypeId="1" AcceptedAnswerId="17466" CreationDate="2021-07-31T10:07:23.283" Score="0" ViewCount="35" Body="&lt;p&gt;I am reading &lt;a href=&quot;https://www.nature.com/articles/s41592-019-0576-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper, where the authors mention&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We apply SAUCIE to the batch correcting, denoising and clustering of&#xA;an 11-million cell mass cytometry dataset with 180 samples from 40&#xA;subjects in a study of the dengue flavivirus&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What exactly is the difference between a “sample” and a “subject” in scRNA-Seq?&lt;/p&gt;&#xA;&lt;p&gt;PS: here’s a possibly relevant visual from the paper, but I still couldn’t understand the difference&#xA;&lt;a href=&quot;https://i.stack.imgur.com/DEUUU.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DEUUU.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="8601" LastEditDate="2021-07-31T10:14:03.150" LastActivityDate="2021-07-31T12:24:28.950" Title="Sample versus subject in scRNA-seq" Tags="|scrnaseq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17466" PostTypeId="2" ParentId="17465" CreationDate="2021-07-31T12:24:28.950" Score="1" Body="&lt;p&gt;A subject is an individual or donor. You can take multiple samples from the same subject, e.g. from different tissues, different timepoints etc.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2021-07-31T12:24:28.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17467" PostTypeId="1" CreationDate="2021-07-31T14:06:11.513" Score="0" ViewCount="844" Body="&lt;p&gt;I am trying to run DESeq2 analysis on my featureCounts output counts table.&lt;/p&gt;&#xA;&lt;p&gt;I ran the following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(&amp;quot;DESeq2&amp;quot;)&#xA;&#xA;samplefile&amp;lt;-&amp;quot;C:/Users/RGB King/Documents/R/~/R/DESeq2_analysis/sample_table.txt&amp;quot;&#xA;&#xA;coldata&amp;lt;-read.table(file=samplefile, sep=&amp;quot;\t&amp;quot;, header=FALSE, col.names = as.factor(c(&amp;quot;&amp;quot;,&amp;quot;treatment&amp;quot;)), row.names=1)&#xA;&#xA;countfile&amp;lt;-&amp;quot;C:/Users/RGB King/Documents/R/~/R/DESeq2_analysis/feature_counts_test_AvABC.txt&amp;quot;&#xA;countdata&amp;lt;-read.table(file=countfile, sep=&amp;quot;\t&amp;quot;, header=FALSE, row.names = 1)&#xA;&#xA;countdata&amp;lt;-countdata[-c(1:5)]&#xA;&#xA;dds &amp;lt;- DESeqDataSetFromMatrix(&#xA;    countData = countdata,&#xA;    colData = coldata,&#xA;    design = ~ treatment&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then get the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Warning message:&#xA;&#xA;In DESeqDataSet(se, design = design, ignoreRank) :&#xA;&#xA;  some variables in design formula are characters, converting to factors&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The countdata table looks as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;|------|Variable1|Variable2|Variable3|Variable4|Variable5|Variable6|Variable7|Variable8|&#xA;|GeneID|A_1.bam--|A_2.bam--|A_3.bam--|A_4.bam--|AB_1.bam-|AB_2.bam-|AB_3.bam-|AB_4.bam-|&#xA;|THI3--|:12:-----|:34:-----|:8:------|:15:-----|:6:------|:22:-----|:41:-----|:33:-----|&#xA;|THI7--|:17:-----|:78:-----|:9:------|:23:-----|:11:-----|:54:-----|:19:-----|:7:------|&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The coldata looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;|--------|treatment|&#xA;|A_1.bam-|:A:------|&#xA;|A_2.bam-|:A:------|&#xA;|A_3.bam-|:A:------|&#xA;|A_4.bam-|:A:------|&#xA;|AB_1.bam|:B:------|&#xA;|AB_2.bam|:B:------|&#xA;|AB_3.bam|:B:------|&#xA;|AB_4.bam|:B:------|&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So after I got the error I found posts on a forum that stated that DESeq2 only recognises &amp;quot;integer&amp;quot; class input values for the analyses. Therefore, I ran the following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sapply(countdata,class)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This then showed me that the data in the countdata table was in &amp;quot;character&amp;quot; class, so to fix this I ran the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chars &amp;lt;- sapply(countdata, is.character)&#xA;countdata[ , chars] &amp;lt;- as.data.frame(apply(countdata[ , chars], 2, as.integer))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But this then caused the values in the GeneID row to become &amp;quot;NA&amp;quot;. So my questions are how can I change the class of just the GeneID row back to what it was? Will this then allow the analysis to run correctly? And in my code for the countdata I do specify &amp;quot;row.names =1&amp;quot; so why does this not keep my values in the GeneID row separate from the rest of the table?&lt;/p&gt;&#xA;" OwnerUserId="13885" LastEditorUserId="3694" LastEditorDisplayName="user3051" LastEditDate="2022-08-18T08:15:44.890" LastActivityDate="2022-08-18T08:15:44.890" Title="Beginner with DESeq2 having issue with analysis" Tags="|rna-seq|differential-expression|deseq2|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="17468" PostTypeId="1" AcceptedAnswerId="17469" CreationDate="2021-08-01T00:10:46.543" Score="1" ViewCount="212" Body="&lt;p&gt;I am trying to build a shiny dashboard for the visualization of different RNAseq data sets.&lt;/p&gt;&#xA;&lt;p&gt;However, I am encountering problem with reactive input, the code is always using the second dataset that I have loaded/read in. Anyone has faced a similar issue? If yes, how did you resolve it.&lt;/p&gt;&#xA;&lt;p&gt;My code is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    library(shinydashboard)&#xA;library(uwot)&#xA;library(DESeq2)&#xA;library(gridExtra)&#xA;library(tidyverse)&#xA;library(RColorBrewer)&#xA;library(DESeq2)&#xA;library(pheatmap)&#xA;library(DEGreport)&#xA;library(vsn)&#xA;library(RColorBrewer)&#xA;library(&amp;quot;genefilter&amp;quot;)&#xA;library(org.Hs.eg.db)&#xA;library(dplyr)&#xA;library(tidyverse)&#xA;library(fgsea)&#xA;library(clusterProfiler)&#xA;library(ggplot2)&#xA;&#xA;&#xA;set_1&amp;lt;-load(&amp;quot;C:/Users/abn/Documents/Shiny/DashBoardTutorial/TeData2.RData&amp;quot;)&#xA;set_2&amp;lt;-load(&amp;quot;C:/Users/abn/Documents/Shiny/DashBoardTutorial/TeData1.RData&amp;quot;)&#xA;data_list = list(set_1=set_1,set_2=set_2)&#xA;&#xA;ui &amp;lt;- dashboardPage(&#xA;    dashboardHeader(title = &amp;quot;Data Visualizer&amp;quot;, titleWidth = 300),&#xA;    dashboardSidebar(&#xA;        width = 300,&#xA;        sidebarMenu(&#xA;&#xA;            menuItem(&amp;quot;Datasets&amp;quot;,  icon = icon(&amp;quot;cog&amp;quot;),&#xA;                     selectInput(&amp;quot;Datasets&amp;quot;, &amp;quot;Datasets:&amp;quot;, choices = list(&amp;quot;sample1&amp;quot; = &amp;quot;set_1&amp;quot;, &amp;quot;sample2&amp;quot; = &amp;quot;set_2&amp;quot;),&#xA;                                 selected = &amp;quot;set_1&amp;quot;)),&#xA;            menuItem(&amp;quot;Quality Control&amp;quot;, tabName = &amp;quot;widgets&amp;quot;, icon = icon(&amp;quot;th&amp;quot;)),&#xA;            menuItem(&amp;quot;Differential Genes&amp;quot;, tabName = &amp;quot;widgets2&amp;quot;, icon = icon(&amp;quot;th&amp;quot;)),&#xA;            menuItem(&amp;quot;Downstream&amp;quot;, tabName = &amp;quot;widgets3&amp;quot;, icon = icon(&amp;quot;th&amp;quot;))&#xA;            &#xA;         )&#xA;    ),&#xA;    ## Body content&#xA;    dashboardBody(&#xA;        tabItems(&#xA;            # First tab content&#xA;            # Second tab content&#xA;            tabItem(tabName=&amp;quot;widgets&amp;quot;,&#xA;                   h2(&amp;quot;widgets&amp;quot;),&#xA;                   plotOutput(&amp;quot;widgets&amp;quot;),&#xA;                    &#xA;            ),&#xA;            tabItem(tabName = &amp;quot;widgets2&amp;quot;,&#xA;                    h2(&amp;quot;Widgets2 tab content&amp;quot;),&#xA;                    &#xA;            ),&#xA;            tabItem(tabName = &amp;quot;widgets3&amp;quot;,&#xA;                    h2(&amp;quot;Widgets3 tab content&amp;quot;),&#xA;                    plotOutput(&amp;quot;widgets3&amp;quot;),&#xA;                    &#xA;            )&#xA;&#xA;        )&#xA;    )&#xA;)&#xA;&#xA;&#xA;server &amp;lt;- function(input, output) {&#xA;&#xA;    &#xA;    datasetInput &amp;lt;- reactive({&#xA;        df &amp;lt;- data_list[[input$Datasets]]&#xA;        })&#xA;    &#xA;        output$widgets &amp;lt;- renderPlot({&#xA;            datasetInput()&#xA;            par(mfrow=c(1,2))&#xA;            &#xA;            boxplot(counts(dds, normalized=F), outline=F, col=dds&lt;span class=&quot;math-container&quot;&gt;$condition, medcol = &quot;white&quot;, cex.axis=0.6, main=&quot;Without Normalization&quot;)&#xA;            boxplot(counts(dds, normalized=T), outline=F, col=dds$&lt;/span&gt;condition, medcol = &amp;quot;white&amp;quot;, cex.axis=0.6, main=&amp;quot;Normalized&amp;quot;)&#xA;            &#xA;            })&#xA;        &#xA;        }&#xA;&#xA;shinyApp(ui, server)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5062" LastActivityDate="2021-08-01T12:22:38.597" Title="shiny dashboard to visualise RNA seq data" Tags="|r|rna-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17469" PostTypeId="2" ParentId="17468" CreationDate="2021-08-01T12:22:38.597" Score="1" Body="&lt;p&gt;There are two issues that I can see.&lt;/p&gt;&#xA;&lt;p&gt;The first is that the results are loaded via &lt;code&gt;RData&lt;/code&gt; files, which include information about the variable names. Assuming these work, I expect they create a variable within the R environment called &lt;code&gt;dds&lt;/code&gt; which has the DESeq2 results. The second load overwrites that variable:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;set_1&amp;lt;-load(&amp;quot;C:/Users/abn/Documents/Shiny/DashBoardTutorial/TeData2.RData&amp;quot;)&#xA;set_2&amp;lt;-load(&amp;quot;C:/Users/abn/Documents/Shiny/DashBoardTutorial/TeData1.RData&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What should probably be done is storing the results as an R Dataset that doesn't include variable name information:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# create set_1 DESeq2 results called 'dds'&#xA;saveRDS(dds, &amp;quot;C:/Users/abn/Documents/Shiny/DashBoardTutorial/TeData2.rds&amp;quot;)&#xA;# create set_2 DESeq2 results called 'dds'&#xA;saveRDS(dds, &amp;quot;C:/Users/abn/Documents/Shiny/DashBoardTutorial/TeData1.rds&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then those can be loaded into the R environment using the assignment you expect:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;set_1&amp;lt;-readRDS(&amp;quot;C:/Users/abn/Documents/Shiny/DashBoardTutorial/TeData2.RData&amp;quot;)&#xA;set_2&amp;lt;-readRDS(&amp;quot;C:/Users/abn/Documents/Shiny/DashBoardTutorial/TeData1.RData&amp;quot;)&#xA;data_list = list(set_1=set_1,set_2=set_2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The second issue is that the plot output displays &lt;code&gt;dds&lt;/code&gt;, rather than the &lt;code&gt;datasetInput&lt;/code&gt; variable, and there is no assignment to change the definition of &lt;code&gt;dds&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    output$widgets &amp;lt;- renderPlot({&#xA;        dds &amp;lt;- datasetInput() ## Note: added assignment&#xA;        par(mfrow=c(1,2))&#xA;        &#xA;        boxplot(counts(dds, normalized=F), outline=F, col=dds&lt;span class=&quot;math-container&quot;&gt;$condition, medcol = &quot;white&quot;, cex.axis=0.6, main=&quot;Without Normalization&quot;)&#xA;        boxplot(counts(dds, normalized=T), outline=F, col=dds$&lt;/span&gt;condition, medcol = &amp;quot;white&amp;quot;, cex.axis=0.6, main=&amp;quot;Normalized&amp;quot;)&#xA;        &#xA;        })&#xA;    &#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-08-01T12:22:38.597" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17470" PostTypeId="2" ParentId="17464" CreationDate="2021-08-01T16:30:53.973" Score="1" Body="&lt;h2&gt;AlphaFold2&lt;/h2&gt;&#xA;&lt;p&gt;You are asking about &lt;strong&gt;how to hybridise different threaded models&lt;/strong&gt;, but I would not give up on AF just yet as it can do complexes.&lt;/p&gt;&#xA;&lt;p&gt;AlphaFold2 is conceptually amazing, but does have some issues to such an extent that I &lt;a href=&quot;https://blog.matteoferla.com/2021/07/what-to-look-out-for-with-alphafold2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;blogged about common problems I was getting so many questions&lt;/a&gt;. Specifically, it models one protein chain in isolation.&lt;/p&gt;&#xA;&lt;p&gt;In the &lt;a href=&quot;https://science.sciencemag.org/content/early/2021/07/19/science.abj8754&quot; rel=&quot;nofollow noreferrer&quot;&gt;RoseTTAFold paper&lt;/a&gt; David Baker and co. mimicked the AlphaFold2 algorithm hinted at from the CASP14 competition. They discovered that if multiple chains were given as a protein concatenation, with suitably long polyglycine linkers, a good oligomer or hetero complex could be resolved. Quickly after the public release of the &lt;a href=&quot;https://colab.research.google.com/github/deepmind/alphafold/blob/main/notebooks/AlphaFold.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Colabs notebooks of AlphaFold2&lt;/a&gt; in the Twittersphere there were reports of homo/hetero-oligomers being resolved nicely. Of particular interest is &lt;a href=&quot;https://twitter.com/Ag_smith/status/1417335144243859457&quot; rel=&quot;nofollow noreferrer&quot;&gt;the report that if you put a spurious pairing you get bad results&lt;/a&gt; —i.e. a control of sorts.&lt;/p&gt;&#xA;&lt;p&gt;This strategy many be applicable to your homohexamer.&lt;/p&gt;&#xA;&lt;p&gt;Your long random loop could be binding something else, so if you know what you may also then want to try with that something else!&lt;/p&gt;&#xA;&lt;h2&gt;Threading, ab initio and hybridisation&lt;/h2&gt;&#xA;&lt;p&gt;Given a template PDB and an aligned sequence a threaded model of the sequence can be made by replacing the residues where required. SwissModel, ModBase and Rosetta partial_thread app does this in the open, while Phyre2 and I-Tasser do it hidden in the pipeline.&#xA;If a span is absent, SwissModel and others will try to predict from scratch (ab initio) the span.&#xA;Features of different threaded models can be stuck together (hybridised) manually or automatically as do Phyre2 and I-Tasser. The &lt;a href=&quot;https://new.rosettacommons.org/docs/latest/application_documentation/structure_prediction/RosettaCM&quot; rel=&quot;nofollow noreferrer&quot;&gt;RosettaCM (comparative modelling) toolkit&lt;/a&gt; is really handy in that one can customise how different parts are treated by adding Rosetta constraints (restraints in crystallography parlance), ligands etc etc. It should be noted that threaded models are faithful to the template, so can be converted into the same complexes as the template, but as soon as they get hybridised they loose this.&lt;/p&gt;&#xA;&lt;p&gt;You currently have various threaded models against a panel of homologues of your choosing and are looking to hybridise them based on your requirements —I assume to circumvent a mistake the standard I-Tasser server makes, such as messing up oligomers.&#xA;So the hybridisation from RosettaCM (skipping the threading) is probably a good choice, however, requires familiarity with Rosetta.&lt;/p&gt;&#xA;&lt;p&gt;The model would need to be &amp;quot;symmetric&amp;quot; (oligomer) before starting the hybridisation (Step 2 in the manual). But that can simply be a case of getting the oligomeric template and superimposing that many copies of your threaded model onto it. Then as your threaded model is from a different framework, it needs to be adapted to the forcefield in Rosetta which differs from I-Tasser, so needs to be minimised, using &lt;a href=&quot;https://new.rosettacommons.org/docs/latest/application_documentation/structure_prediction/relax&quot; rel=&quot;nofollow noreferrer&quot;&gt;Relax&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Another minor note is that a weakness is the pairwise alignment for all of these, which for distant variants can be problematically erroneous —hence why Phyre2 and I-Tasser run a MSA and from there extract the pairwise, which is a lot more accurate. I do not know how the restraint4 parameter works.&lt;/p&gt;&#xA;&lt;h2&gt;Edit: SwissModel&lt;/h2&gt;&#xA;&lt;p&gt;I realised just now that maybe SwissModel was not tried —it is an old threader, but does oligomers nicely. In fact, I tried &lt;a href=&quot;https://swissmodel.expasy.org/interactive/8RE9fB/models/&quot; rel=&quot;nofollow noreferrer&quot;&gt;your sequence&lt;/a&gt; and the hexamer looks decent and the internal metric is pretty good...&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2021-08-02T08:52:35.323" LastActivityDate="2021-08-02T08:52:35.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17471" PostTypeId="1" AcceptedAnswerId="17472" CreationDate="2021-08-01T16:52:04.690" Score="0" ViewCount="300" Body="&lt;p&gt;I have downloaded my FASTA format files, that have the DNA sequences of the coding region of the genes and the DNA sequence of the complete genome, from NCBI. How can I recognize if these sequences have been assembled or not ?&lt;/p&gt;&#xA;&lt;p&gt;In particular, I read this paper &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3416535/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genome Sequencing of a Genetically Tractable Pyrococcus furiosus Strain Reveals a Highly Dynamic Genome&lt;/a&gt; and I downloaded the sequences that they indicate in &lt;strong&gt;Nucleotide sequence accession number&lt;/strong&gt; where they say:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The fully assembled DNA sequence of COM1 has been deposited in GenBank (accession number &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/CP003685&quot; rel=&quot;nofollow noreferrer&quot;&gt;CP003685&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I clicked on the link &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/CP003685&quot; rel=&quot;nofollow noreferrer&quot;&gt;CP003685&lt;/a&gt; and I downloaded the DNA sequences that I mentioned at the beginning by clicking on &amp;quot;Send to&amp;quot;, then &amp;quot;Complete Record&amp;quot;,then &amp;quot;File&amp;quot; (in Choose Destination), then FASTA (in Format). For the cds file I made the same steps but clicking on &amp;quot;Coding Sequences&amp;quot; and I could do the same to have the sequences of the genes clicking on &amp;quot;Gene Features&amp;quot;. But are these sequences assembled? By reading the extract of the article I would say yes... but is there a way to know this from the FASTA files that I downloaded or the NCBI website ?&lt;/p&gt;&#xA;&lt;p&gt;Moreover, why is it important to know how a sequence has been assembled (so which software has been used) ?&lt;/p&gt;&#xA;&lt;p&gt;I apologize for my very basic questions and I thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="10594" LastActivityDate="2022-06-27T10:05:16.760" Title="How to know if the DNA sequence has been assembled and why is it important to know how it was assembled?" Tags="|fasta|genome|assembly|sequence-analysis|phylogenetics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17472" PostTypeId="2" ParentId="17471" CreationDate="2021-08-01T21:42:35.190" Score="2" Body="&lt;p&gt;Genomes are always assemblies. Assembly basically means making a consensus sequence from many partially overlapping readouts. As there is currently no sequencing technology that allows to sequence an entire genome or even an entire chromosome with a single super high fidelity read that is without errors we have to combine several reads to form the consensus which is then the assembly.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Moreover, why is it important to know how a sequence has been assembled (so which software has been used) ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Tbh, I never cared on how a genome I used was assembled. To be fair I only work with mouse or human, and for people who work with genomes that are not as well curated as these it might indeed matter, but others would need to comment on that, not really my field.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-08-01T21:42:35.190" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17473" PostTypeId="2" ParentId="17467" CreationDate="2021-08-01T21:44:51.933" Score="1" Body="&lt;p&gt;There is nothing wrong with your colData. It is encoded as character rather than factor, and DESeq2 is turning it into factors, that is all that the warning (warning != error) tells. I never needed to convert my matrix to load featureCounts output into DESeq2. Just load the output with the header, use &lt;code&gt;rownames()&lt;/code&gt; to move the gene column to rownames, remove all the non-relevant columns (that is usually 1-7) and then feed the remaining one into DESeq2. I have no featureCounts output at hand right now. If you need code then please provide suitable example data via the &lt;code&gt;dput&lt;/code&gt; function for easy copy/pasting.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-08-01T21:44:51.933" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17475" PostTypeId="1" AcceptedAnswerId="17476" CreationDate="2021-08-02T17:41:26.520" Score="0" ViewCount="86" Body="&lt;p&gt;Article of interest: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3023908/#SD1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3023908/#SD1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Referring to Figure 2b of that article, if I understand correctly, the x-axis refers to the position of each variant in a chromosome while the y-axis is a log transform of the raw p-value of association of each variant with some variable of interest.&lt;/p&gt;&#xA;&lt;p&gt;My questions is, is there practical importance in finding strong association (high log transform p-value) around one particular position versus finding it on some other position in the same chromosome?&lt;/p&gt;&#xA;&lt;p&gt;My follow up question is, are these log transformed p-values publicly available?&lt;/p&gt;&#xA;&lt;p&gt;Context: I am a statistics student studying multiple testing. My biology background is very limited.&lt;/p&gt;&#xA;" OwnerUserId="12822" LastActivityDate="2021-08-04T05:18:52.333" Title="Question about the practical relevance of chromosome position and p-value" Tags="|statistics|gwas|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17476" PostTypeId="2" ParentId="17475" CreationDate="2021-08-02T19:59:54.010" Score="1" Body="&lt;p&gt;In the specific case that you are talking about, the answer is very much yes that the specific position is important. Genes occur at (fairly) fixed positions along chromosomes.&lt;/p&gt;&#xA;&lt;p&gt;The point of that figure, as indicated in the legend, is that the very low p-values are around the gene &lt;em&gt;RPM1&lt;/em&gt;. That gene is known to have important natural genetic variation associated with the phenotype in question (the one for whichg they are looking for genetic associations). That figure is therefore an important validation of the methods of the authors.&lt;/p&gt;&#xA;&lt;p&gt;In terms of finding such data, a fun account on twitter is the &lt;a href=&quot;https://twitter.com/SbotGwa&quot; rel=&quot;nofollow noreferrer&quot;&gt;GWASbot&lt;/a&gt;, that just posts random manhattan plots (which is the informal name for the plot you are talking about), along with links to source data. I believe that that source data includes the p-values for each ascertained position, along with other summary statistics. One example is &lt;a href=&quot;https://t.co/thzvlqF8ma?amp=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you are specifically interested in the topic of multiple test correction, an important related concept in GWAS is that of &lt;a href=&quot;https://en.wikipedia.org/wiki/Genome-wide_significance&quot; rel=&quot;nofollow noreferrer&quot;&gt;genome-wide significance&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;To unzip and look at the file, use &lt;a href=&quot;http://www.htslib.org/doc/bgzip.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bgzip&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ bgzip -d ~/Downloads/continuous-20491-both_sexes.tsv.bgz&#xA;$ head ~/Downloads/continuous-20491-both_sexes.tsv&#xA;# prints the following:&#xA;chr pos ref alt af_meta beta_meta   se_meta pval_meta   pval_heterogeneity  af_AFR  af_CSA  af_EUR  beta_AFR    beta_CSA    beta_EUR    se_AFR  se_CSA  se_EUR  pval_AFR    pval_CSA    pval_EUR    low_confidence_AFR  low_confidence_CSA  low_confidence_EUR&#xA;1   11063   T   G   2.088e-02   2.247e-02   2.072e-01   9.137e-01   0.000e+00   2.088e-02   NA  5.212e-05   2.247e-02   NA  1.287e-01   2.072e-01   NA  5.080e-01   9.137e-01   NA  8.000e-01   false   NA  true&#xA;1   13259   G   A   2.805e-04   9.840e-02   8.909e-02   2.694e-01   1.000e+00   NA  NA  2.805e-04   NA  NA  9.840e-02   NA  NA  8.909e-02   NA  NA  2.694e-01   NA  NA  false&#xA;1   17641   G   A   8.908e-04   -1.244e-02  4.922e-02   8.005e-01   1.000e+00   NA  NA  8.908e-04   NA  NA  -1.244e-02  NA  NA  4.922e-02   NA  NA  8.005e-01   NA  NA  false&#xA;1   30741   C   A   NA  NA  NA  NA  NA  9.400e-04   NA  NA  3.935e-03   NA  NA  1.297e+00   NA  NA  9.976e-01   NA  NA  true    NA  NA&#xA;1   51427   T   G   NA  NA  NA  NA  NA  1.127e-03   NA  NA  8.780e-01   NA  NA  9.366e-01   NA  NA  3.486e-01   NA  NA  true    NA  NA&#xA;1   57222   T   C   7.078e-04   -4.153e-03  5.656e-02   9.415e-01   1.000e+00   NA  NA  7.078e-04   NA  NA  -4.153e-03  NA  NA  5.656e-02   NA  NA  9.415e-01   NA  NA  false&#xA;1   58396   T   C   2.735e-04   3.011e-02   8.221e-02   7.142e-01   1.000e+00   2.622e-03   NA  2.735e-04   -1.069e-01  NA  3.011e-02   7.079e-01   NA  8.221e-02   8.800e-01   NA  7.142e-01   true    NA  false&#xA;1   62745   C   G   NA  NA  NA  NA  NA  2.701e-03   NA  NA  5.524e-01   NA  NA  5.529e-01   NA  NA  3.178e-01   NA  NA  true    NA  NA&#xA;1   63668   G   A   NA  NA  NA  NA  NA  NA  1.129e-03   3.673e-05   NA  2.205e-01   -4.462e-02  NA  9.738e-01   2.544e-01   NA  8.209e-01   8.608e-01   NA  true    true&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2021-08-03T00:26:41.043" LastActivityDate="2021-08-03T00:26:41.043" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="17478" PostTypeId="2" ParentId="3045" CreationDate="2021-08-03T09:02:41.597" Score="1" Body="&lt;p&gt;One way to get one of the many statistically possible paths is to start from the end and work backwards. This works with the maze puzzles on kids' menus, but also with proteins. To find how a ligand enters an active site, dynamic undocking uses tethered MD to pull a bound form out. Likewise for a protein folding starting from a folded state and lowering the attraction term of Lenard-Jones or gradually increasing the reference temperature used by the Berendsen thermostat in an MD run (as done for DNA annealing simulations).&lt;/p&gt;&#xA;&lt;p&gt;This assumes that the system is reversible (not true for protein with a maturation step like GFP), but it is avoids getting stuck in local minima or dead-end paths in the case of maze puzzles.&#xA;It also assumes that peptides start folding only once they are finished being synthesised, which is not true and actually circular permutations are an interesting case as many the permutation windows/spokes are unfolded.&lt;/p&gt;&#xA;&lt;p&gt;However, the path down a folding funnel is more of a hike/roll down a canyon than a maze puzzles on paper as there are many possible routes to the solution. So having a bunch of trajectories makes a nice video, but does not say much...&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EV4SV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EV4SV.png&quot; alt=&quot;funnel&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;A seemingly similar concept, but very different, is the pathway reconstruction for the evolution trajectory that leads to a multi-mutation variant of interest. This is well studied because the intermediates are not transient but have to persevere in the population until another mutation appears. If they have a too high fitness cost they will not be around long enough for a compensation mutation (cf. epistasis).&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-08-03T09:02:41.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17479" PostTypeId="1" CreationDate="2021-08-03T11:42:00.607" Score="0" ViewCount="187" Body="&lt;p&gt;I'd like to perform a motif analysis of all promoter regions of the gene ENSMUSG00000020538 in mus musculus.&#xA;To do so, I wanted to use Biomart :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library('biomaRt')&#xA;mart &amp;lt;- useMart(&amp;quot;ensembl&amp;quot;)&#xA;mart &amp;lt;- useDataset(&amp;quot;mmusculus_gene_ensembl&amp;quot;, mart = mart)&#xA;all.genes &amp;lt;- getSequence(id = 'ENSMUSG00000020538', &#xA;                         type=&amp;quot;ensembl_gene_id&amp;quot;,&#xA;                         seqType=&amp;quot;coding_gene_flank&amp;quot;,&#xA;                         upstream=100, &#xA;                         mart=mart)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But this provide me only the flanking region of the sepcified gene, not all its promoter regions. How could I do ?&lt;/p&gt;&#xA;" OwnerUserId="13908" LastActivityDate="2024-01-14T00:08:05.977" Title="Getting all promoters of a specific gene" Tags="|ensembl|biomart|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17480" PostTypeId="1" CreationDate="2021-08-03T16:26:39.017" Score="2" ViewCount="63" Body="&lt;p&gt;I need some help getting started on this project.&lt;/p&gt;&#xA;&lt;p&gt;To simplify we want to be able to quantify the occurrence of 3 variants on each sequencing read in an alignment file as a proxy measurement for recombination.&lt;/p&gt;&#xA;&lt;p&gt;The possible combinations we have are A, B or AB.&lt;/p&gt;&#xA;&lt;p&gt;For example, variant A is at position 100 and variant B is at position 300 of the reference and AB would have both.&lt;/p&gt;&#xA;&lt;p&gt;We have alignment files against a reference sequence, and then I get stuck. I could technically look by eye in tablet or IGV for example but is there a way to extract each read then query by expected base at the reference position then bin these reads into the categories; A, B, AB and unclassified then calculate the total occurrences of each?&lt;/p&gt;&#xA;&lt;p&gt;Thanks for your time and suggestions!&lt;/p&gt;&#xA;" OwnerUserId="13909" LastEditorUserId="2108" LastEditDate="2021-08-29T01:19:11.193" LastActivityDate="2021-08-29T22:20:38.900" Title="Counting the co-occurrence of variants A and B in an aligned sequencing read" Tags="|sam|samtools|mutations|variants|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17481" PostTypeId="2" ParentId="5021" CreationDate="2021-08-03T17:14:10.437" Score="1" Body="&lt;p&gt;If your goal is to simply &amp;quot;to quantify the change in the amount of molecule,&amp;quot; then you can do that at the population-average level by simply taking the average antibody signal for the different cell populations (stimulated vs. not stimulated). In that case, you don't have to throw out samples with noise above a threshold - since they are still part of the population and so should be used when calculating average.&lt;/p&gt;&#xA;&lt;p&gt;If you want to quantify &amp;quot;the change in the amount of molecule&amp;quot; at the single-cell level, then things are a bit more complicated, and the best answer depends in part on things like the shapes of the distribution of signal from the stimulated and non-stimulated populations.&#xA;And related to that, when you say &amp;quot;noise above the 1%,&amp;quot; do you mean that the Coefficient of Variation (CV) is greater than 1%? Or something else?&lt;/p&gt;&#xA;" OwnerUserId="11905" LastActivityDate="2021-08-03T17:14:10.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17482" PostTypeId="1" CreationDate="2021-08-03T18:14:23.987" Score="1" ViewCount="129" Body="&lt;p&gt;I'm using the PositionScan function in FoldX 5.0 to calculate delta delta G for some mutants. I'm working with the command prompt in Windows 10. For some reason, this function will only return output for residues 1-9 of my protein - any residues in the double and triple digits don't work.&lt;/p&gt;&#xA;&lt;p&gt;For example, my protein only has one chain (A) and it has Leu at position 9 and Lys at position 10. For whatever reason, the following command will work:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FoldX --command=PositionScan --pdb=protein.pdb --positions=LA9a &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;While this one won't:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FoldX --command=PositionScan --pdb=protein.pdb --positions=KA10a&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have triple checked the PDB file itself to make sure I'm not typing in the wrong letter codes and positions. The format of the PDB file seems to be fine, when I open it in a structure viewer nothing seems out of the ordinary.&lt;/p&gt;&#xA;&lt;p&gt;What's going on here?&lt;/p&gt;&#xA;&lt;p&gt;Thanks for any help in advance! I'm new at this stuff.&lt;/p&gt;&#xA;" OwnerUserId="13910" LastEditorUserId="73" LastEditDate="2021-08-04T13:34:54.903" LastActivityDate="2021-08-04T13:34:54.903" Title="FoldX PositionScan functions only works for certain position inputs" Tags="|protein-structure|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17483" PostTypeId="1" AcceptedAnswerId="17487" CreationDate="2021-08-03T18:35:20.807" Score="2" ViewCount="167" Body="&lt;p&gt;I need idea, intuition, suggestion please&lt;/p&gt;&#xA;&lt;p&gt;I have &lt;code&gt;10X scRNA-seq&lt;/code&gt; of &lt;code&gt;PBMC (multiome's 3' poly-A capture)&lt;/code&gt;, can I capture &lt;code&gt;TCR/BCR&lt;/code&gt; from my scRNA-seq or I need &lt;code&gt;TCRseq&lt;/code&gt; ? I googled and likely TCR and BCR immune profiling require 5' transcript capture and the V(D)J variable sequence is on the 5' end of the gene. So the 3' capture typically yields the C constant region of TCR and BCR transcripts. can you help me please? Thank you&lt;/p&gt;&#xA;" OwnerUserId="9449" LastActivityDate="2023-01-26T16:27:11.983" Title="TCR-seq or scRNA-seq" Tags="|scrnaseq|seurat|10x-genomics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17484" PostTypeId="2" ParentId="17475" CreationDate="2021-08-03T21:41:01.733" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Are these log transformed p-values publicly available?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;For human population data, it's pretty common for the association statistics to be either included with the research paper, or uploaded to a public database (e.g. see curated compilations like &lt;a href=&quot;https://www.gwascentral.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GWAS Central&lt;/a&gt; or &lt;a href=&quot;https://www.ebi.ac.uk/gwas/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NHGRI GWAS Catalog&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;For plant and other non-human data, there doesn't seem to have been as much drive to include GWAS data with the research paper, and it's often up to the authors to decide to do that. I find this a little odd, given that there should be fewer ethical barriers to releasing non-human public data.&lt;/p&gt;&#xA;&lt;p&gt;If you're interested in a particular dataset, you can contact the authors, but I acknowledge that such an approach can be too high of a barrier for most interested researchers.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is there practical importance in finding strong association (high&#xA;log transform p-value) around one particular position versus&#xA;finding it on some other position in the same chromosome?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It's common for statistical tests for association to assume independence of variant locations. With this in mind, two associated regions that are one base apart should not be any more interesting than two associated regions that are 50 megabases apart.&lt;/p&gt;&#xA;&lt;p&gt;The problem with this assumption is that DNA variation is &lt;em&gt;not&lt;/em&gt; statistically independent. Recombination of DNA (and associated reassortment of genetic variation) between two locations is more common when the locations are close, and this &lt;em&gt;linkage&lt;/em&gt; should really be accounted for when doing association tests. One way to do this is to combine variants that share the same ancestral origin into a haplotype, then do association tests on the &lt;em&gt;haplotype block&lt;/em&gt; rather than the single-base &lt;em&gt;genotype&lt;/em&gt;. Even if the particular observed variants are not directly causative, there may be another untyped causative variant (or combination of variants) on the same haplotype block, meaning that the block becomes a proxy for the causative variant.&lt;/p&gt;&#xA;&lt;p&gt;But this haplotype-based approach becomes quickly computationally intractable when looking at larger regions of chromosomes, and is very difficult to use for comparing populations with substantially different ancestral backgrounds. The workaround (if it could be called that) is to ignore the linkage aspect of genetic variation, and return to those same statistical tests, but declare regions that have peaks of close association to be statistically interesting, and zoom into them to look in more detail at the region.&lt;/p&gt;&#xA;&lt;p&gt;That zoom in would be a great opportunity to use more complex / accurate haplotype-based tests for confirming an associative signal around a particular region. But for the most part, as with the paper you've indicated, the tests are identical.&lt;/p&gt;&#xA;&lt;p&gt;However, even ignoring the haplotype issue... &lt;em&gt;&amp;lt;begin rant&amp;gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;The most common manhattan plots represent &lt;em&gt;only&lt;/em&gt; p-value on the Y-axis. I think this is a mistake, because the p-value only represents the confidence in a statistic, and the important thing for association is the statistic itself, rather than the confidence in that statistic.&lt;/p&gt;&#xA;&lt;p&gt;I have a few specific concerns about this pure p-value approach:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;there may be values that have less confidence, but a much higher associative statistic, and contribute more to the trait of interest (e.g. &lt;a href=&quot;https://twitter.com/gringene_bio/status/1207380005249929216&quot; rel=&quot;nofollow noreferrer&quot;&gt;see here&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;values with a low association statistic are more sensitive to genotyping error, leading to spurious associations.&lt;/li&gt;&#xA;&lt;li&gt;the p-value doesn't indicate the direction of association&lt;/li&gt;&#xA;&lt;li&gt;p-values should not be used for ranking, and this ranking is implicitly done when using the p-value only manhattan plot&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In addition, the generated p-values often don't make sense, and I haven't seen any commentary on this. Anything smaller than &lt;span class=&quot;math-container&quot;&gt;$10^{-40}$&lt;/span&gt; doesn't represent any reasonable natural random occurrence, even on galactic timescales, but often these astronomically-small p-values can become insignificant through a few random mutations in one of the populations.&lt;/p&gt;&#xA;&lt;p&gt;As a quick-fix using the existing statistics that are typically presented in GWAS results, where single points are desired, I recommend that people instead use the standard deviation to determine the minimum expected beta score:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kGJku.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kGJku.png&quot; alt=&quot;beta_min scores for Heel bone mineral density&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;see &lt;a href=&quot;https://twitter.com/gringene_bio/status/1207630454033014788&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for more details.&lt;/p&gt;&#xA;&lt;p&gt;More rigorously, I think that people should consider doing their own internal validation of associations via bootstrap sub-sampling, and report a non-parametric statistic: the best observed &lt;em&gt;rank&lt;/em&gt; after multiple bootstrap sub-samples. This requires access to genotype-level data, so cannot be done on existing GWAS results. I explore this in more detail in &lt;a href=&quot;https://doi.org/10.1101/104497&quot; rel=&quot;nofollow noreferrer&quot;&gt;this preprint&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Yes, I'm aware this goes against the grain of how GWAS is typically done. I have been emboldened in my position on this after reading the ASA principles about the p-value, and seeing that many of the principles are in direct opposition to the typical manhattan plot:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.tandfonline.com/doi/full/10.1080/00031305.2016.1154108#_i31&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tandfonline.com/doi/full/10.1080/00031305.2016.1154108#_i31&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-08-04T05:18:52.333" LastActivityDate="2021-08-04T05:18:52.333" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17485" PostTypeId="1" AcceptedAnswerId="17489" CreationDate="2021-08-03T23:05:52.760" Score="0" ViewCount="38" Body="&lt;p&gt;Trying to read in this file but saved as a .txt file: &lt;a href=&quot;http://www.informatics.jax.org/downloads/reports/HOM_MouseHumanSequence.rpt&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.informatics.jax.org/downloads/reports/HOM_MouseHumanSequence.rpt&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;gene_conversion = read.table(&amp;quot;HOM_MouseHumanSequence.rpt.txt&amp;quot;,header=T, fill=T)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;but the column names are splitting weirdly.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(gene_conversion)&#xA;        DB  Class        Key Common Organism       Name        NCBI Taxon    ID                      Symbol&#xA;1 38710983 mouse, laboratory  10090     Gdnf      14573  MGI:107430 Chr15   3.8                          cM&#xA;2 38710983  human       9606   GDNF     2668  HGNC:4232 OMIM:600837  Chr5 p13.2   Chr5:37812677-37840044(-)&#xA;3 38710984 mouse, laboratory  10090     Rrm1      20133   MGI:98180  Chr7 54.72                          cM&#xA;4 38710984  human       9606   RRM1     6240 HGNC:10451 OMIM:180410 Chr11 p15.4    Chr11:4094685-4138925(+)&#xA;5 38710985 mouse, laboratory  10090   Hoxd12      15432   MGI:96204  Chr2 44.13                          cM&#xA;6 38710985  human       9606 HOXD12     3238  HGNC:5135 OMIM:142988  Chr2 q31.1 Chr2:176099802-176100760(+)&#xA;                                                            EntrezGene&#xA;1                                             Chr15:7840327-7867056(+)&#xA;2 NM_199231,NM_001278098,NM_199234,NM_001190469,NM_001190468,NM_000514&#xA;3                                          Chr7:102090902-102118978(+)&#xA;4                     NM_001033,NM_001318065,NM_001330193,NM_001318064&#xA;5                                            Chr2:74505357-74508049(+)&#xA;6                                                            NM_021193&#xA;                                                                                  ID.1&#xA;1                                     NM_001301357,NM_010275,NM_001301333,NM_001301332&#xA;2 NP_001265027,NP_000505,NP_001177397,NP_001177398,XP_016864826,XP_011512332,NP_954701&#xA;3                                                                            NM_009103&#xA;4                                     NP_001024,NP_001317122,NP_001304994,NP_001304993&#xA;5                                                                            NM_008274&#xA;6                                                                            NP_067016&#xA;                                             Mouse    MGI ID.2 HGNC ID.3 OMIM Gene ID.4 Genetic Location Genomic&#xA;1 NP_001288286,NP_001288262,NP_034405,NP_001288261 P48540   NA   NA   NA   NA   NA   NA      NA       NA      NA&#xA;2                                           P39905          NA   NA   NA   NA   NA   NA      NA       NA      NA&#xA;3                                        NP_033129 P07742   NA   NA   NA   NA   NA   NA      NA       NA      NA&#xA;4                                           P23921          NA   NA   NA   NA   NA   NA      NA       NA      NA&#xA;5                                        NP_032300 P23812   NA   NA   NA   NA   NA   NA      NA       NA      NA&#xA;6                                           P35452          NA   NA   NA   NA   NA   NA      NA       NA      NA&#xA;  Coordinates X.mouse. X. human. X..1 Nucleotide RefSeq IDs Protein RefSeq.1 IDs.1 SWISS_PROT IDs.2&#xA;1          NA       NA NA     NA   NA         NA     NA  NA      NA       NA    NA         NA    NA&#xA;2          NA       NA NA     NA   NA         NA     NA  NA      NA       NA    NA         NA    NA&#xA;3          NA       NA NA     NA   NA         NA     NA  NA      NA       NA    NA         NA    NA&#xA;4          NA       NA NA     NA   NA         NA     NA  NA      NA       NA    NA         NA    NA&#xA;5          NA       NA NA     NA   NA         NA     NA  NA      NA       NA    NA         NA    NA&#xA;6          NA       NA NA     NA   NA         NA     NA  NA      NA       NA    NA         NA    NA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6827" LastEditorUserId="73" LastEditDate="2021-08-04T13:01:37.750" LastActivityDate="2021-08-04T13:07:51.783" Title="loading in .txt files with different spacing in column names into R" Tags="|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17486" PostTypeId="1" AcceptedAnswerId="17497" CreationDate="2021-08-04T00:17:58.143" Score="3" ViewCount="6597" Body="&lt;p&gt;I'm trying to convert mouse genes from PTX data to human genes in order to do a comparison with patient data to see what genes are being conserved. I'm using this file for the orthologs.&#xA;&lt;strong&gt;&lt;a href=&quot;http://www.informatics.jax.org/downloads/reports/HOM_MouseHumanSequence.rpt&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.informatics.jax.org/downloads/reports/HOM_MouseHumanSequence.rpt&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I read the table with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gene_conversion &amp;lt;- read.table(&amp;quot;HOM_MouseHumanSequence.rpt.txt&amp;quot;, sep=&amp;quot;\t&amp;quot;, header=T, fill=T)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have subsetted the mouse genes from the PTX seurat object to only include the mouse genes with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mouse_only &amp;lt;- subset(scData, features = mouse_1)&#xA;head(rownames(mouse_only))&#xA;[1] &amp;quot;Xkr4&amp;quot;    &amp;quot;Gm37180&amp;quot; &amp;quot;Gm37363&amp;quot; &amp;quot;Gm19938&amp;quot; &amp;quot;Sox17&amp;quot;   &amp;quot;Gm6085&amp;quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now I'm not sure how to convert the mouse genes in my mouse-only seurat object to match the human genes in the file.&lt;/p&gt;&#xA;&lt;p&gt;Any help would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="6827" LastEditorUserId="6827" LastEditDate="2021-08-04T14:12:59.400" LastActivityDate="2024-01-09T06:53:33.173" Title="converting mouse genes to human genes" Tags="|r|seurat|single-cell|conversion|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17487" PostTypeId="2" ParentId="17483" CreationDate="2021-08-04T11:52:06.397" Score="5" Body="&lt;p&gt;The 3' gene expression protocol will capture TCR and BCR mRNAs but this may not be very helpful to you. As you already mentioned only the 3' end will be sequenced, which are the constant regions. With this data you may potentially be able to discriminate &lt;code&gt;TRAC&lt;/code&gt; from &lt;code&gt;TRBC1&lt;/code&gt; and so forth.&lt;/p&gt;&#xA;&lt;p&gt;However, in most cases researchers are interested in clonotypes (combinations of variable regions) and these you can only obtain by using the &lt;a href=&quot;https://support.10xgenomics.com/single-cell-vdj&quot; rel=&quot;nofollow noreferrer&quot;&gt;dedicated TCR or BCR kits&lt;/a&gt; from 10X. They contain specific sets of primers for these genes to cover the whole TCR or BCR gene body and not only the 3' end. This way you can see the different clonotypes in your population&lt;/p&gt;&#xA;&lt;p&gt;The human TCR loci can be found for example at &lt;a href=&quot;https://www.genenames.org/data/genegroup/#!/group/370&quot; rel=&quot;nofollow noreferrer&quot;&gt;HUGO&lt;/a&gt;. The last gene at each locus is the constant and is identified by a C at the end. TRAC = T cell receptor alpha constant (I did not use the correct naming convention in the original answer, sorry)&lt;/p&gt;&#xA;&lt;p&gt;B-cell receptors basically are membrane bound immunoglobins (IG). Therefore you will likely find the constant regions of the heavy chain in your data. The different &lt;a href=&quot;https://www.genenames.org/data/genegroup/#!/group/886&quot; rel=&quot;nofollow noreferrer&quot;&gt;heavy chains&lt;/a&gt; are what identifies the IG. For example IGHA1 is the heavy chain of an alpha IG. If the secreted and membrane bound form of the IGs is always the same per B-cell, unfortunately, is beyond my knowledge of B-cells.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastEditorUserId="4885" LastEditDate="2021-08-05T11:04:37.700" LastActivityDate="2021-08-05T11:04:37.700" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17488" PostTypeId="1" CreationDate="2021-08-04T12:27:08.137" Score="1" ViewCount="271" Body="&lt;p&gt;I'm trying to look for a gene in a group of SRA files. Web BLAST can't cope with them, so I assume they're too large. I've tried tblastn_vdb from the SRA toolkit, as follows. It had previously worked well for me for WGS files.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;module load anaconda3/personal&#xA;source activate sratoolkit&#xA;&#xA;cd ~/sra-tools&#xA;PATH=$PATH:~/sra-tools/sratoolkit.2.10.5-ubuntu64/bin&#xA;tblastn_vdb -db &amp;quot;SRR6256958&amp;quot; -query ~/sra-tools/query/query.txt -out ~/sra-tools/out/results.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get nothing in my output file except for 'TBLASTN 2.6.1+'. My error file goes as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Loading anaconda3/personal&#xA;  Loading requirement: fix_unwritable_tmp fix_setxattr&#xA;VDB: 2021-07-16T12:03:22 . debug: KSysDirResolvePath_v1 = '/[path]/sra-tools/sratoolkit.2.10.5-ubuntu64/schema'&#xA;VDB: 2021-07-16T12:03:22 . debug: KSysDirResolvePath_v1 = '/tmp/pbs.4070476.pbs'&#xA;VDB: 2021-07-16T12:03:22 . debug: KSysDirResolvePath_v1 = '/tmp/pbs.4070476.pbs'&#xA;BLAST engine error: NCBI C++ Exception:&#xA;    T0 &amp;quot;/panfs/pan1/sra-test/vdb3/git/blast_vdb/sratoolkit_2_9_0/c++/src/algo/blast/api/prelim_stage.cpp&amp;quot;, line 304: Error: BLAST::ncbi::blast::CBlastPrelimSearch::Run() - search cannot proceed due to errors retrieving sequences from databases&#xA;&#xA;/var/spool/PBS/mom_priv/jobs/4070476.pbs.SC: line 7: 4032551 Aborted [then it lists the original code again]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does anyone know what I am doing wrong? Help would be much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2021-08-04T12:27:08.137" Title="How to use SRA Toolkit to blast SRA accessions?" Tags="|blast|sratoolkit|sra|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17489" PostTypeId="2" ParentId="17485" CreationDate="2021-08-04T13:07:51.783" Score="0" Body="&lt;p&gt;I see spaces in the headings, which suggests this might be a tab-separated file, rather than a space-separated file. If that's the case, you should be using &lt;code&gt;sep=&amp;quot;\t&amp;quot;&lt;/code&gt; in your &lt;code&gt;read.table&lt;/code&gt; command, or alternatively using &lt;code&gt;read.delim&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gene_conversion &amp;lt;-&#xA;  read.table(&amp;quot;HOM_MouseHumanSequence.rpt.txt&amp;quot;, sep=&amp;quot;\t&amp;quot;, header=T, fill=T)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gene_conversion &amp;lt;- read.delim(&amp;quot;HOM_MouseHumanSequence.rpt.txt&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-08-04T13:07:51.783" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17490" PostTypeId="1" AcceptedAnswerId="17491" CreationDate="2021-08-04T15:02:27.990" Score="0" ViewCount="255" Body="&lt;p&gt;I have a list of sequences:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CAGGTAGCC&#xA;CCGGTCAGA&#xA;AGGGTTTGA&#xA;TTGGTGAGG&#xA;CAAGTATGA&#xA;ACTGTATGC&#xA;CTGGTAACC&#xA;    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Each sequence is nine nucleotides long. I want to calculate the dinucleotide frequency of AA, CG ..etc for each of the dinucleotide positions.&#xA;Similar problem shared with one nucleotide in this &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/3089/calculating-nucleotide-frequency-per-column&quot;&gt;post&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For example, 1st two column will have these bases CA,CC,AG,TT etc, and second two should be AG,CG,GG,TG and so on. Output should be something like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;AA   0.25    0.34    0.56    0.43    0.00    0.90    0.45    0.34    0.31&#xA;CC   0.45    0.40    0.90    0.00    0.40    0.90    0.30    0.25    0.2&#xA;GC   0.00    0.00    0.34    1.00    0.30    0.30    0.35    0.90    0.1&#xA;TC   0.24    0.56    0.00    0.00    1.00    0.34    0.45    0.35    0.36&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;AA, CC, GC and TC could be any dinucleotide combination.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What I tried from the last post: mainly taken from @Devon&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import itertools&#xA;hl = []&#xA;&#xA;#making all possible dinucleotide combinations&#xA;bases=['A','T','G','C']&#xA;k = 2&#xA;&#xA;kmers = [''.join(p) for p in itertools.product(bases, repeat=k)]&#xA;kmers = dict.fromkeys(kmers, 0)&#xA;print(kmers)&#xA;for i in range(9):&#xA;    hl.append(kmers)&#xA;&#xA;# my seq file&#xA;f = open(&amp;quot;H3K27ac.seq&amp;quot;)&#xA;nLines = 0&#xA;&#xA;for line in f:&#xA;    id = 0&#xA;# trying to read dinucleotide at each loop&#xA;    for (fst, sec) in zip(line[0::2], line[1::2]):&#xA;        id = id + 1&#xA;        hl[id][fst+sec] += 1&#xA;    nLines += 1&#xA;f.close()&#xA;&#xA;nLines = float(nLines)&#xA;for char in ['AA', 'TT', 'CC', 'GG', 'CG', 'GC']:   &#xA;    print(&amp;quot;{}\t{}&amp;quot;.format(char, &amp;quot;\t&amp;quot;.join([&amp;quot;{:0.2f}&amp;quot;.format(x[char]/nLines) for x in hl])))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;By somehow it's estimation the total count of all dinucleotide in whole file.&#xA;I am trying to make it more general so that it can be used for tri- and any further number of nucleotides.&lt;/p&gt;&#xA;" OwnerUserId="4492" LastActivityDate="2021-08-04T20:36:20.887" Title="calculating kmer nucleotide frequency per column" Tags="|python|sequence-analysis|k-mer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17491" PostTypeId="2" ParentId="17490" CreationDate="2021-08-04T20:36:20.887" Score="2" Body="&lt;p&gt;This should do the trick. The following code works for any aligned character set, but assumes that all sequences should be the same length.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import itertools&#xA;&#xA;      &#xA;with open(&amp;quot;f.txt&amp;quot;) as seqs:&#xA;    sequences = list(line.strip() for line in seqs.readlines())&#xA;&#xA;# Just in case sequence lengths are different&#xA;n = min(len(s) for s in sequences)&#xA;if any(n != len(s) for s in sequences):&#xA;    print(&amp;quot;Sequence lengths are different!&amp;quot;)&#xA;&#xA;# Get all bases. &#xA;bases = set(c for seq in sequences for c in seq)&#xA;&#xA;k = 2 &#xA;kmers = [''.join(p) for p in itertools.product(bases, repeat=k)]&#xA;pos_to_freq = []&#xA;&#xA;for i in range(n - k + 1):&#xA;    kmers_dict = dict.fromkeys(kmers, 0)&#xA;    for s in sequences:&#xA;        kmers_dict[s[i:i+k]] += 1/len(sequences)&#xA;    pos_to_freq.append(kmers_dict)&#xA;&#xA;for kmer in kmers:&#xA;    print(&amp;quot;\t&amp;quot;.join([kmer] + [str(round(d[kmer], 3)) for d in pos_to_freq]))&#xA;        &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output for your example is below (I apologize for the formatting):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;AA  0   0.143   0   0   0   0.143   0   0&#xA;AT  0   0   0   0   0   0.286   0   0&#xA;AC  0.143   0   0   0   0   0   0.143   0&#xA;AG  0.143   0.143   0.143   0   0   0.143   0.286   0&#xA;TA  0   0   0   0   0.571   0   0   0&#xA;TT  0.143   0   0   0   0.143   0.143   0   0&#xA;TC  0   0   0   0   0.143   0   0   0&#xA;TG  0   0.286   0.143   0   0.143   0   0.429   0&#xA;CA  0.286   0   0   0   0   0.143   0   0&#xA;CT  0.143   0.143   0   0   0   0   0   0&#xA;CC  0.143   0   0   0   0   0   0   0.286&#xA;CG  0   0.143   0   0   0   0   0   0&#xA;GA  0   0   0   0   0   0.143   0   0.429&#xA;GT  0   0   0   1.0 0   0   0   0&#xA;GC  0   0   0   0   0   0   0.143   0.143&#xA;GG  0   0.143   0.714   0   0   0   0   0.143&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6628" LastActivityDate="2021-08-04T20:36:20.887" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17492" PostTypeId="2" ParentId="3943" CreationDate="2021-08-04T22:37:10.183" Score="0" Body="&lt;p&gt;The accepted solution is probably the best for older objects of type &lt;code&gt;seurat&lt;/code&gt; created with Seurat package v2. For newer &lt;code&gt;Seurat Objects&lt;/code&gt;, there is a new tool designed specifically for this purpose, called &lt;a href=&quot;https://mojaveazure.github.io/seurat-disk/articles/convert-anndata.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;SeuratDisk&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There is a nicely documented vignette about the &lt;a href=&quot;https://mojaveazure.github.io/seurat-disk/articles/convert-anndata.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seurat &amp;lt;-&amp;gt; AnnData conversion&lt;/a&gt;. In short:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In R, save the Seurat object as an h5Seurat file:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(Seurat)&#xA;library(SeuratDisk)&#xA;&#xA;SaveH5Seurat(SeuratObject, filename = &amp;quot;file.h5Seurat&amp;quot;)&#xA;Convert(&amp;quot;file.h5Seurat&amp;quot;, dest = &amp;quot;h5ad&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;In python, simply load the file:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import scanpy&#xA;&#xA;adata = scanpy.read_h5ad(&amp;quot;file.h5Seurat&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4446" LastActivityDate="2021-08-04T22:37:10.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17493" PostTypeId="1" CreationDate="2021-08-05T02:56:42.640" Score="0" ViewCount="256" Body="&lt;p&gt;According to GATK best practice, it is recommended that different VQSR models be built for SNPs and INDELs, because the annotations for high-quality SNPs and INDELs are systematically different (if I understand it correctly). Since annotations for good variants on autosomes could be different from those on X chromosome, e.g., DP for good variants on X chromosome could be substantially smaller than DP on autosomes due to having large number of male samples, it seems reasonable to build VQSR models separately for X chromosome and autosomes. However, no such advice is proposed on GATK website.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My question is&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;Should we build VQSR models separately for autosomes and X chromosome? If yes, we will have 4 VQSR models: Auto SNPs, Auto INDELs, X SNPs and X INDELs.&lt;/p&gt;&#xA;&lt;p&gt;Cross-posting: &lt;a href=&quot;https://www.biostars.org/p/9483110/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/9483110/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10056" LastEditorDisplayName="user3051" LastEditDate="2021-08-05T09:44:10.717" LastActivityDate="2021-08-13T07:39:47.297" Title="Best practice for running GATK VQSR on X chromosome" Tags="|wgs|gatk|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="17494" PostTypeId="2" ParentId="17479" CreationDate="2021-08-05T08:58:47.560" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @ATpoint, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;You need to extract the TSS of all transcripts of that gene, not the gene borders. Query biomaRt for that gene, then set options to extract all transcripts, then get the sequence for the TSS coordinates of those (with a window).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-08-05T08:58:47.560" CommentCount="0" CommunityOwnedDate="2021-08-05T08:58:47.560" ContentLicense="CC BY-SA 4.0" />
  <row Id="17495" PostTypeId="1" CreationDate="2021-08-05T11:12:08.490" Score="1" ViewCount="77" Body="&lt;p&gt;I want to do a verification of the result of an RNAseq analysis of human cancer samples by GWAS, I want to use rrBLUP package in R, so I need three input file genotype, phenotype and mapping file, so I am confused about how I can get these files.&lt;/p&gt;&#xA;" OwnerUserId="12578" LastEditorUserId="12578" LastEditDate="2021-08-05T11:29:46.853" LastActivityDate="2021-08-05T11:29:46.853" Title="GWAS : input files for rrBLUP package in R" Tags="|gwas|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17496" PostTypeId="2" ParentId="16454" CreationDate="2021-08-05T11:41:49.573" Score="1" Body="&lt;p&gt;Ensembl provides gene annotation files in both GTF and GFF3 format via the FTP site:&#xA;&lt;a href=&quot;http://ftp.ensembl.org/pub/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ftp.ensembl.org/pub/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The GTF and GFF3 files are named according to whether they contain data relating to the primary assembly, or on all top-level sequences including unplaced scaffolds, patches and haplotypes. More information can be found in the README files within each folder. Which genome sequence file and annotation file you use will depend on the details of your study and have been discussed in detail on other posts.&lt;/p&gt;&#xA;&lt;p&gt;Ensembl also provides reference RNAseq data aligned to the reference genome sequence as BAM files with corresponding .bai index files on the FTP site:&#xA;&lt;a href=&quot;http://ftp.ensembl.org/pub/current_bamcov/homo_sapiens/genebuild/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ftp.ensembl.org/pub/current_bamcov/homo_sapiens/genebuild/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;More information can be found in the README:&#xA;&lt;a href=&quot;http://ftp.ensembl.org/pub/current_bamcov/homo_sapiens/genebuild/README&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ftp.ensembl.org/pub/current_bamcov/homo_sapiens/genebuild/README&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6500" LastActivityDate="2021-08-05T11:41:49.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17497" PostTypeId="2" ParentId="17486" CreationDate="2021-08-05T14:11:57.497" Score="3" Body="&lt;p&gt;Mice genes are usually the same as the human ones, but lower case with only the first character as upper (&lt;code&gt;Sox17&lt;/code&gt; -&amp;gt; &lt;code&gt;SOX17&lt;/code&gt;).&#xA;&lt;code&gt;toupper(rownames(mouse_only))&lt;/code&gt; will do the trick.&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, there are exceptions, and it is always better to relay on information from databases. The function bellow takes a vector of human genes, and uses &lt;code&gt;biomaRt&lt;/code&gt; to return the corresponding genes in mouse.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# from Human to Mouse&#xA;convertHumanGeneList &amp;lt;- function(x){&#xA;&#xA;   library(&amp;quot;biomaRt&amp;quot;)&#xA;   human &amp;lt;- useMart(&amp;quot;ensembl&amp;quot;, dataset = &amp;quot;hsapiens_gene_ensembl&amp;quot;)&#xA;   mouse &amp;lt;- useMart(&amp;quot;ensembl&amp;quot;, dataset = &amp;quot;mmusculus_gene_ensembl&amp;quot;)&#xA;&#xA;   genesV2 &amp;lt;- getLDS(attributes = c(&amp;quot;hgnc_symbol&amp;quot;), filters = &amp;quot;hgnc_symbol&amp;quot;, &#xA;                 values = x , mart = human, attributesL = c(&amp;quot;mgi_symbol&amp;quot;), martL = mouse, uniqueRows=T)&#xA;&#xA;   humanx &amp;lt;- unique(genesV2[, 2])&#xA;&#xA;   return(humanx)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You simply need to invert the 2 and you should be able to convert your vector &lt;code&gt;rownames(mouse_only)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;#---- Update using SOX17 as example&#xA;&lt;a href=&quot;https://i.stack.imgur.com/adHUN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/adHUN.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3631" LastEditorUserId="3631" LastEditDate="2021-08-05T21:29:40.720" LastActivityDate="2021-08-05T21:29:40.720" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17498" PostTypeId="1" CreationDate="2021-08-05T14:49:36.233" Score="0" ViewCount="67" Body="&lt;p&gt;I have a data frame of raw read counts, genes in rows and samples in columns&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dput(head(oc))&#xA;structure(list(sample1 = c(13L, 0L, 1L, 42L, 15L, 1137L), sample2 = c(21L, &#xA;2L, 20L, 51L, 5L, 203L), sample3 = c(25L, 2L, 6L, 71L, 36L, 1346L&#xA;), sample4 = c(59L, 3L, 8L, 58L, 42L, 3428L), sample5 = c(19L, &#xA;4L, 8L, 52L, 1L, 201L)), row.names = c(&amp;quot;WASH7P&amp;quot;, &amp;quot;MIR6859-1&amp;quot;, &#xA;&amp;quot;CICP27&amp;quot;, &amp;quot;WASH9P&amp;quot;, &amp;quot;MTND1P23&amp;quot;, &amp;quot;MTND2P28&amp;quot;), class = &amp;quot;data.frame&amp;quot;)&#xA;&amp;gt; &#xA;&#xA;&amp;gt; head(oc)&#xA;          sample1 sample2 sample3 sample4 sample5&#xA;WASH7P         13      21      25      59      19&#xA;MIR6859-1       0       2       2       3       4&#xA;CICP27          1      20       6       8       8&#xA;WASH9P         42      51      71      58      52&#xA;MTND1P23       15       5      36      42       1&#xA;MTND2P28     1137     203    1346    3428     201&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I just want to keep genes with at least 10 reads in each sample, I mean even if a gene has less than 10 reads in one sample and the rest of four samples have 10 reads, this gene should be removed. For instance here the second, third and fifth rows should be removed&lt;/p&gt;&#xA;&lt;p&gt;I tied this but all I obtained zero&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; df=t(oc)&#xA;&amp;gt; cutoff &amp;lt;- 10&#xA;&amp;gt; &#xA;&amp;gt; # encode the data based on the cutoff&#xA;&amp;gt; df[df &amp;gt;= cutoff] &amp;lt;- 1&#xA;&amp;gt; df[df &amp;lt; cutoff] &amp;lt;- 0&#xA;&amp;gt; View(df)&#xA;&amp;gt; rowSums(df)&#xA;sample1 sample2 sample3 sample4 sample5 &#xA;      0       0       0       0       0 &#xA;&amp;gt; &#xA;&#xA;&amp;gt; apply(df &amp;gt;= 5, 1, sum)&#xA;sample1 sample2 sample3 sample4 sample5 &#xA;      0       0       0       0       0 &#xA;&amp;gt; apply(df &amp;gt;= 1, 1, sum)&#xA;sample1 sample2 sample3 sample4 sample5 &#xA;      0       0       0       0       0 &#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any suggestion please?&lt;/p&gt;&#xA;" OwnerUserId="9449" LastEditorUserId="9449" LastEditDate="2021-08-05T17:22:47.210" LastActivityDate="2021-08-05T17:22:47.210" Title="Keeping rows with a cut-off in all columns" Tags="|r|rna-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17499" PostTypeId="1" AcceptedAnswerId="17500" CreationDate="2021-08-05T16:14:13.230" Score="0" ViewCount="223" Body="&lt;p&gt;I have a Seurat object&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; pbmc&#xA;An object of class Seurat &#xA;48608 features across 11709 samples within 2 assays &#xA;Active assay: SCT (22220 features, 3000 variable features)&#xA; 1 other assay present: RNA&#xA; 2 dimensional reductions calculated: pca, umap&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to plot TCRVδ2&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; FeaturePlot(pbmc,&amp;quot;TCRVδ2&amp;quot;)&#xA;Error: None of the requested features were found: TCRVδ2 in slot data&#xA;In addition: Warning message:&#xA;In FetchData(object = object, vars = c(dims, &amp;quot;ident&amp;quot;, features),  :&#xA;  The following requested variables were not found: TCRVδ2&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This data base is about human TCR genes&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.genenames.org/data/genegroup/#!/group/370&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.genenames.org/data/genegroup/#!/group/370&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But I even can not find this gene here&lt;/p&gt;&#xA;&lt;p&gt;Do you know what is the corresponding symbol for this gene used in Seurat?&lt;/p&gt;&#xA;" OwnerUserId="9449" LastEditorUserId="3694" LastEditDate="2021-08-05T17:03:38.773" LastActivityDate="2021-08-05T17:03:38.773" Title="Find corresponding symbol for gene used in Seurat" Tags="|scrnaseq|gene|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17500" PostTypeId="2" ParentId="17499" CreationDate="2021-08-05T16:44:56.467" Score="1" Body="&lt;p&gt;&lt;code&gt;TCRVδ2&lt;/code&gt; is not a gene, but it defines a specific subset of gamma-delta TCRs. You can look at &lt;code&gt;TRDC&lt;/code&gt; (delta chain) and &lt;code&gt;TRGC1&lt;/code&gt; and &lt;code&gt;TRGC2&lt;/code&gt; (gamma chains). &lt;code&gt;TCRVδ2&lt;/code&gt; should have the combo &lt;code&gt;TRDC-TRGC1&lt;/code&gt;, while &lt;code&gt;TCRVδ1&lt;/code&gt; should express &lt;code&gt;TRDC-TRGC2&lt;/code&gt;. You may need to increase the resolution of your clustering, to avoid clustering the gamma-delta T cells with the NK cells.&lt;/p&gt;&#xA;" OwnerUserId="3631" LastActivityDate="2021-08-05T16:44:56.467" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17501" PostTypeId="1" CreationDate="2021-08-05T16:57:45.763" Score="0" ViewCount="2390" Body="&lt;p&gt;I have a Seurat object&lt;/p&gt;&#xA;&lt;p&gt;I want to plot a list of genes but I got a warning saying most of genes removed&lt;/p&gt;&#xA;&lt;p&gt;I have tried with both &lt;em&gt;&lt;strong&gt;my genes of interest&lt;/strong&gt;&lt;/em&gt; and &lt;em&gt;&lt;strong&gt;all variable genes&lt;/strong&gt;&lt;/em&gt; but the results is very similar&lt;/p&gt;&#xA;&lt;p&gt;Does this mean these genes are basically are not expressed in my data?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   &amp;gt; t&lt;span class=&quot;math-container&quot;&gt;$V1&#xA;  [1] &quot;IGH&quot;          &quot;IGHV1-2&quot;      &quot;IGHV1-3&quot;      &quot;IGHV1-12&quot;     &quot;IGHV1-8&quot;     &#xA;  [6] &quot;IGHV1-14&quot;     &quot;IGHV1-17&quot;     &quot;IGHV1-18&quot;     &quot;IGHV1-24&quot;     &quot;IGHV1-45&quot;    &#xA; [11] &quot;IGHV1-46&quot;     &quot;IGHV1-58&quot;     &quot;IGHV1-67&quot;     &quot;IGHV1-68&quot;     &quot;IGHV1-69&quot;    &#xA; [16] &quot;IGHV1-69D&quot;    &quot;IGHV1-38-4&quot;   &quot;IGHV1-69-2&quot;   &quot;IGHV2-5&quot;      &quot;IGHV2-10&quot;    &#xA; [21] &quot;IGHV2-26&quot;     &quot;IGHV2-70&quot;     &quot;IGHV2-70D&quot;    &quot;IGHV3-6&quot;      &quot;IGHV3-7&quot;     &#xA; [26] &quot;IGHV3-9&quot;      &quot;IGHV3-11&quot;     &quot;IGHV3-13&quot;     &quot;IGHV3-15&quot;     &quot;IGHV3-16&quot;    &#xA; [31] &quot;IGHV3-19&quot;     &quot;IGHV3-20&quot;     &quot;IGHV3-21&quot;     &quot;IGHV3-22&quot;     &quot;IGHV3-23&quot;    &#xA; [36] &quot;IGHV3-25&quot;     &quot;IGHV3-29&quot;     &quot;IGHV3-30&quot;     &quot;IGHV3-30-2&quot;   &quot;IGHV3-30-3&quot;  &#xA; [41] &quot;IGHV3-30-5&quot;   &quot;IGHV3-32&quot;     &quot;IGHV3-33&quot;     &quot;IGHV3-33-2&quot;   &quot;IGHV3-35&quot;    &#xA; [46] &quot;IGHV3-36&quot;     &quot;IGHV3-37&quot;     &quot;IGHV3-38&quot;     &quot;IGHV3-41&quot;     &quot;IGHV3-42&quot;    &#xA; [51] &quot;IGHV3-43&quot;     &quot;IGHV3-43D&quot;    &quot;IGHV3-47&quot;     &quot;IGHV3-48&quot;     &quot;IGHV3-49&quot;    &#xA; [56] &quot;IGHV3-50&quot;     &quot;IGHV3-52&quot;     &quot;IGHV3-53&quot;     &quot;IGHV3-54&quot;     &quot;IGHV3-57&quot;    &#xA; [61] &quot;IGHV3-60&quot;     &quot;IGHV3-62&quot;     &quot;IGHV3-63&quot;     &quot;IGHV3-64&quot;     &quot;IGHV3-65&quot;    &#xA; [66] &quot;IGHV3-66&quot;     &quot;IGHV3-71&quot;     &quot;IGHV3-72&quot;     &quot;IGHV3-73&quot;     &quot;IGHV3-74&quot;    &#xA; [71] &quot;IGHV3-75&quot;     &quot;IGHV3-76&quot;     &quot;IGHV3-79&quot;     &quot;IGHV3-38-3&quot;   &quot;IGHV3-69-1&quot;  &#xA; [76] &quot;IGHV4-4&quot;      &quot;IGHV4-28&quot;     &quot;IGHV4-30-1&quot;   &quot;IGHV4-30-2&quot;   &quot;IGHV4-30-4&quot;  &#xA; [81] &quot;IGHV4-31&quot;     &quot;IGHV4-34&quot;     &quot;IGHV4-39&quot;     &quot;IGHV4-55&quot;     &quot;IGHV4-59&quot;    &#xA; [86] &quot;IGHV4-61&quot;     &quot;IGHV4-80&quot;     &quot;IGHV4-38-2&quot;   &quot;IGHV5-51&quot;     &quot;IGHV5-78&quot;    &#xA; [91] &quot;IGHV5-10-1&quot;   &quot;IGHV6-1&quot;      &quot;IGHV7-4-1&quot;    &quot;IGHV7-40&quot;     &quot;IGHV7-27&quot;    &#xA; [96] &quot;IGHV7-56&quot;     &quot;IGHV7-34-1&quot;   &quot;IGHV7-81&quot;     &quot;IGHVII-1-1&quot;   &quot;IGHVII-15-1&quot; &#xA;[101] &quot;IGHVII-20-1&quot;  &quot;IGHVII-22-1&quot;  &quot;IGHVII-26-2&quot;  &quot;IGHVII-28-1&quot;  &quot;IGHVII-30-1&quot; &#xA;[106] &quot;IGHVII-30-21&quot; &quot;IGHVII-31-1&quot;  &quot;IGHVII-33-1&quot;  &quot;IGHVII-40-1&quot;  &quot;IGHVII-43-1&quot; &#xA;[111] &quot;IGHVII-44-2&quot;  &quot;IGHVII-46-1&quot;  &quot;IGHVII-49-1&quot;  &quot;IGHVII-51-2&quot;  &quot;IGHVII-53-1&quot; &#xA;[116] &quot;IGHVII-60-1&quot;  &quot;IGHVII-62-1&quot;  &quot;IGHVII-65-1&quot;  &quot;IGHVII-67-1&quot;  &quot;IGHVII-74-1&quot; &#xA;[121] &quot;IGHVII-78-1&quot;  &quot;IGHVIII-2-1&quot;  &quot;IGHVIII-5-1&quot;  &quot;IGHVIII-5-2&quot;  &quot;IGHVIII-11-1&quot;&#xA;[126] &quot;IGHVIII-13-1&quot; &quot;IGHVIII-16-1&quot; &quot;IGHVIII-22-2&quot; &quot;IGHVIII-25-1&quot; &quot;IGHVIII-26-1&quot;&#xA;[131] &quot;IGHVIII-38-1&quot; &quot;IGHVIII-44&quot;   &quot;IGHVIII-47-1&quot; &quot;IGHV8-51-1&quot;   &quot;IGHVIII-67-2&quot;&#xA;[136] &quot;IGHVIII-67-3&quot; &quot;IGHVIII-67-4&quot; &quot;IGHVIII-76-1&quot; &quot;IGHVIII-82&quot;   &quot;IGHVIV-44-1&quot; &#xA;[141] &quot;IGHD1-1&quot;      &quot;IGHD1-7&quot;      &quot;IGHD1-14&quot;     &quot;IGHD1-20&quot;     &quot;IGHD1-26&quot;    &#xA;[146] &quot;IGHD2-2&quot;      &quot;IGHD2-8&quot;      &quot;IGHD2-15&quot;     &quot;IGHD2-21&quot;     &quot;IGHD3-3&quot;     &#xA;[151] &quot;IGHD3-9&quot;      &quot;IGHD3-10&quot;     &quot;IGHD3-16&quot;     &quot;IGHD3-22&quot;     &quot;IGHD4-4&quot;     &#xA;[156] &quot;IGHD4-11&quot;     &quot;IGHD4-17&quot;     &quot;IGHD4-23&quot;     &quot;IGHD5-5&quot;      &quot;IGHD5-12&quot;    &#xA;[161] &quot;IGHD5-18&quot;     &quot;IGHD5-24&quot;     &quot;IGHD6-6&quot;      &quot;IGHD6-13&quot;     &quot;IGHD6-19&quot;    &#xA;[166] &quot;IGHD6-25&quot;     &quot;IGHD7-27&quot;     &quot;IGHJ1&quot;        &quot;IGHJ1P&quot;       &quot;IGHJ2&quot;       &#xA;[171] &quot;IGHJ2P&quot;       &quot;IGHJ3&quot;        &quot;IGHJ3P&quot;       &quot;IGHJ4&quot;        &quot;IGHJ5&quot;       &#xA;[176] &quot;IGHJ6&quot;        &quot;IGHA1&quot;        &quot;IGHA2&quot;        &quot;IGHG1&quot;        &quot;IGHG2&quot;       &#xA;[181] &quot;IGHG3&quot;        &quot;IGHG4&quot;        &quot;IGHGP&quot;        &quot;IGHD&quot;         &quot;IGHE&quot;        &#xA;[186] &quot;IGHEP1&quot;       &quot;IGHM&quot;        &#xA;&amp;gt; pbmc=ScaleData(&#xA;+     pbmc,&#xA;+     features = t$&lt;/span&gt;V1)&#xA;Centering and scaling data matrix&#xA;  |================================================================================| 100%&#xA;&amp;gt; DoHeatmap(pbmc,t&lt;span class=&quot;math-container&quot;&gt;$V1,group.by = &quot;Condition&quot;)&#xA;Warning message:&#xA;In DoHeatmap(pbmc, t$&lt;/span&gt;V1, group.by = &amp;quot;Condition&amp;quot;) :&#xA;  The following features were omitted as they were not found in the scale.data slot for the SCT assay: IGHEP1, IGHE, IGHJ6, IGHJ5, IGHJ4, IGHJ3P, IGHJ3, IGHJ2P, IGHJ2, IGHJ1P, IGHJ1, IGHD7-27, IGHD6-25, IGHD6-19, IGHD6-13, IGHD6-6, IGHD5-24, IGHD5-18, IGHD5-12, IGHD5-5, IGHD4-23, IGHD4-17, IGHD4-11, IGHD4-4, IGHD3-22, IGHD3-16, IGHD3-10, IGHD3-9, IGHD3-3, IGHD2-21, IGHD2-15, IGHD2-8, IGHD2-2, IGHD1-26, IGHD1-20, IGHD1-14, IGHD1-7, IGHD1-1, IGHVIV-44-1, IGHVIII-82, IGHVIII-76-1, IGHVIII-67-4, IGHVIII-67-3, IGHVIII-67-2, IGHV8-51-1, IGHVIII-47-1, IGHVIII-44, IGHVIII-38-1, IGHVIII-26-1, IGHVIII-25-1, IGHVIII-22-2, IGHVIII-16-1, IGHVIII-13-1, IGHVIII-11-1, IGHVIII-5-2, IGHVIII-5-1, IGHVIII-2-1, IGHVII-78-1, IGHVII-74-1, IGHVII-67-1, IGHVII-65-1, IGHVII-62-1, IGHVII-60-1, IGHVII-53-1, IGHVII-51-2, IGHVII-49-1, IGHVII-46-1, IGHVII-44-2, IGHVII-43-1, IGHVII-40-1, IGHVII-33-1, IGHVII-31-1, IGHVII-30-21, IGHVII-30-1, IGHVII-28-1, IGHVII-26-2, IGHVII-22-1, IGHVII-20-1, IGHVII-15-1, IGHVII-1-1, IGH [... truncated]&#xA;&amp;gt; pbmc=ScaleData(&#xA;+     pbmc,&#xA;+     features =rownames(pbmc))&#xA;Centering and scaling data matrix&#xA;  |================================================================================| 100%&#xA;&amp;gt; rm(pbmc)&#xA;&amp;gt; pbmc=readRDS(&amp;quot;pbmc.rds&amp;quot;)&#xA;&amp;gt; pbmc=ScaleData(&#xA;+     pbmc,&#xA;+     features =rownames(pbmc))&#xA;Centering and scaling data matrix&#xA;  |================================================================================| 100%&#xA;&amp;gt; DoHeatmap(pbmc,t&lt;span class=&quot;math-container&quot;&gt;$V1,group.by = &quot;Condition&quot;)&#xA;Warning message:&#xA;In DoHeatmap(pbmc, t$&lt;/span&gt;V1, group.by = &amp;quot;Condition&amp;quot;) :&#xA;  The following features were omitted as they were not found in the scale.data slot for the SCT assay: IGHEP1, IGHE, IGHJ6, IGHJ5, IGHJ4, IGHJ3P, IGHJ3, IGHJ2P, IGHJ2, IGHJ1P, IGHJ1, IGHD7-27, IGHD6-25, IGHD6-19, IGHD6-13, IGHD6-6, IGHD5-24, IGHD5-18, IGHD5-12, IGHD5-5, IGHD4-23, IGHD4-17, IGHD4-11, IGHD4-4, IGHD3-22, IGHD3-16, IGHD3-10, IGHD3-9, IGHD3-3, IGHD2-21, IGHD2-15, IGHD2-8, IGHD2-2, IGHD1-26, IGHD1-20, IGHD1-14, IGHD1-7, IGHD1-1, IGHVIV-44-1, IGHVIII-82, IGHVIII-76-1, IGHVIII-67-4, IGHVIII-67-3, IGHVIII-67-2, IGHV8-51-1, IGHVIII-47-1, IGHVIII-44, IGHVIII-38-1, IGHVIII-26-1, IGHVIII-25-1, IGHVIII-22-2, IGHVIII-16-1, IGHVIII-13-1, IGHVIII-11-1, IGHVIII-5-2, IGHVIII-5-1, IGHVIII-2-1, IGHVII-78-1, IGHVII-74-1, IGHVII-67-1, IGHVII-65-1, IGHVII-62-1, IGHVII-60-1, IGHVII-53-1, IGHVII-51-2, IGHVII-49-1, IGHVII-46-1, IGHVII-44-2, IGHVII-43-1, IGHVII-40-1, IGHVII-33-1, IGHVII-31-1, IGHVII-30-21, IGHVII-30-1, IGHVII-28-1, IGHVII-26-2, IGHVII-22-1, IGHVII-20-1, IGHVII-15-1, IGHVII-1-1, IGH [... truncated]&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9449" LastEditorUserId="9449" LastEditDate="2021-08-05T18:06:04.197" LastActivityDate="2021-08-05T18:06:04.197" Title="Avoiding the warning 'The following features were omitted as they were not found in the scale.data slot for the SCT assay' in Seurat" Tags="|scrnaseq|seurat|10x-genomics|heatmap|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17502" PostTypeId="2" ParentId="17498" CreationDate="2021-08-05T17:14:18.113" Score="1" Body="&lt;p&gt;&lt;code&gt;apply(my_df &amp;gt;= 10, 1, sum)&lt;/code&gt; would give you that how many of the samples satisfy your criteria. Basically you &lt;code&gt;apply()&lt;/code&gt; the function &lt;code&gt;sum()&lt;/code&gt; to the rows (denoted as &lt;code&gt;1&lt;/code&gt;). The &lt;code&gt;sum()&lt;/code&gt; does the trick that @Devon Ryan suggested above.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2021-08-05T17:14:18.113" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17503" PostTypeId="2" ParentId="17501" CreationDate="2021-08-05T17:51:07.570" Score="1" Body="&lt;p&gt;Specify your genes of interest using the &lt;code&gt;features&lt;/code&gt; parameter of the &lt;code&gt;ScaleData()&lt;/code&gt; function. If this parameter is not specified, only &amp;quot;variable genes&amp;quot; are scaled and that makes sense, genes that do not demonstrate variability across conditions is not interesting in general. Anyhow, since the  &lt;code&gt;DoHeatmap()&lt;/code&gt; would be looking at the &amp;quot;scaled data slot&amp;quot;, it will not be able to find your genes of interest unless they show variation.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2021-08-05T17:51:07.570" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17504" PostTypeId="1" CreationDate="2021-08-05T20:54:39.900" Score="1" ViewCount="663" Body="&lt;p&gt;I have a folder of pdb ligand structures that I'd like to test in some docking experiments with a certain protein using AutoDock Vina.&#xA;I am not familiar with Python so I've always been using the interactive procedure on AutoDockTools to prepare the pdbqt ligand files instead of the &lt;a href=&quot;http://autodock.scripps.edu/faqs-help/how-to/how-to-prepare-a-ligand-file-for-autodock4&quot; rel=&quot;nofollow noreferrer&quot;&gt;prepare_ligand4.py script&lt;/a&gt;.&#xA;Could you please help me understand how to iterate this python script in a way that it can be applied to a set of pdb ligand files in order to create the corresponding set of pdbqt ligand files?&#xA;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="13927" LastActivityDate="2022-01-01T01:00:35.050" Title="Python script to simultaneously generate multiple pdbqt files for AutoDockTools?" Tags="|python|pdb|docking|structural-biology|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17506" PostTypeId="1" CreationDate="2021-08-05T22:04:15.647" Score="1" ViewCount="208" Body="&lt;p&gt;I'm trying to do a cross-species comparison between the patient TME vs. the PTX TME to understand gene expression conservation. I have already converted between mouse and human genes giving me a resultant gene expression matrix that is just one set of (homologous) genes X cells for all mouse and human cells.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;musGenes &amp;lt;- rownames(mouse_only)&#xA;# Basic function to convert mouse to human gene names&#xA;convertMouseGeneList &amp;lt;- function(x){&#xA;require(&amp;quot;biomaRt&amp;quot;)&#xA;h = useMart(&amp;quot;ensembl&amp;quot;, dataset = &amp;quot;hsapiens_gene_ensembl&amp;quot;)&#xA;m = useMart(&amp;quot;ensembl&amp;quot;, dataset = &amp;quot;mmusculus_gene_ensembl&amp;quot;)&#xA;genesV2 = getLDS(attributes = c(&amp;quot;mgi_symbol&amp;quot;), filters = &amp;quot;mgi_symbol&amp;quot;, values = x , mart = m, attributesL = c(&amp;quot;hgnc_symbol&amp;quot;), martL = h, uniqueRows=T)&#xA;humanx &amp;lt;- unique(genesV2[, 2])&#xA;# Print the first 6 genes found to the screen&#xA;print(head(humanx))&#xA;return(humanx)&#xA;}&#xA;genes &amp;lt;- convertMouseGeneList(musGenes)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now how do I create a new Seurat object with the converted mouse gene list, such as rename the matrix, so that the rownames/colnames are human symbols, rather than mouse symbols?&lt;/p&gt;&#xA;" OwnerUserId="6827" LastEditorUserId="6827" LastEditDate="2021-08-20T15:11:05.027" LastActivityDate="2021-10-11T16:09:13.357" Title="Creating new seurat object with new matrix" Tags="|r|scrnaseq|seurat|single-cell|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17509" PostTypeId="1" AcceptedAnswerId="17516" CreationDate="2021-08-06T10:42:33.187" Score="3" ViewCount="2717" Body="&lt;p&gt;The R package &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/limma.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;limma&lt;/code&gt;&lt;/a&gt; is ideal to perform differential expression analysis. Is there any &lt;code&gt;limma&lt;/code&gt; alternative in Python?&lt;/p&gt;&#xA;&lt;p&gt;I'm trying to use &lt;a href=&quot;https://www.statsmodels.org/stable/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;statsmodels&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;https://scikit-learn.org/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;scikitlearn&lt;/code&gt;&lt;/a&gt; in conjunction with some other cool tools (such as &lt;a href=&quot;https://github.com/epigenelabs/pyComBat&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;pycombat&lt;/code&gt;&lt;/a&gt;) to get &lt;code&gt;limma&lt;/code&gt;-like workflows. However, I'm missing many capabilities from the &lt;code&gt;limma&lt;/code&gt; toolbox.&lt;/p&gt;&#xA;" OwnerUserId="12603" LastActivityDate="2021-08-11T01:37:57.120" Title="R limma alternatives in Python" Tags="|r|python|differential-expression|limma|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17510" PostTypeId="1" CreationDate="2021-08-06T13:57:25.867" Score="0" ViewCount="747" Body="&lt;p&gt;I have a Seurat object which has a high expression of &lt;code&gt;mitochondrial genes&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; pbmc1&#xA;An object of class Seurat &#xA;36601 features across 18338 samples within 1 assay &#xA;Active assay: RNA (36601 features, 0 variable features)&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cgUpB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cgUpB.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want to use &lt;code&gt;SCTransform in Seurat&lt;/code&gt;, I don't know if still I should define the mitochondrial percentage or not, if not, how SCTransform function knows which percentage suits for my data?&lt;/p&gt;&#xA;&lt;p&gt;I used SCTransform without clipping &lt;code&gt;MT genes&lt;/code&gt; and I noticed I have the same number of cells&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; pbmc_SCTransform &amp;lt;- SCTransform(pbmc, method = &amp;quot;glmGamPoi&amp;quot;, vars.to.regress = &amp;quot;percent.mt&amp;quot;, verbose = FALSE)&#xA;&amp;gt; pbmc_SCTransform&#xA;An object of class Seurat &#xA;58629 features across 18338 samples within 2 assays &#xA;Active assay: SCT (22028 features, 3000 variable features)&#xA; 1 other assay present: RNA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is it fine or I should clip &lt;code&gt;MT genes&lt;/code&gt; before running SCTransform function?&lt;/p&gt;&#xA;&lt;p&gt;I am not too sure about the input composition but from 10X websummary two of my samples likely suffer from poor sample prep or poor beginning cell health. The gene expression median genes per cell are 41 and 50 and differential expression show only mitochondrial (MT) and MALAT1 genes. This is indicative of subpar lysis conditions.&lt;/p&gt;&#xA;" OwnerUserId="9449" LastEditorUserId="73" LastEditDate="2021-08-08T12:26:12.967" LastActivityDate="2021-08-08T12:26:12.967" Title="sctransform - mitochondrial expression filtering / transformation" Tags="|scrnaseq|seurat|10x-genomics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17511" PostTypeId="1" AcceptedAnswerId="17512" CreationDate="2021-08-06T17:37:09.303" Score="1" ViewCount="626" Body="&lt;p&gt;I'm inputting a .csv file and then plotting a barplot based on the colnames but when I go plot the order the names are being alphabetized rather than get the order that is in the .csv file. How to I prevent that from happening?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;p2 = ggplot(data=clusters_per_sample, aes(x=sample, y=Perc, fill=cluster)) +&#xA;  geom_bar(stat=&amp;quot;identity&amp;quot;)+&#xA;  scale_fill_manual(values = getPalette(22))+&#xA;  theme_minimal() + labs(y=&amp;quot;percentage of total # cells in the sample&amp;quot;) + theme(axis.text.x = element_text(angle = 90, face = &amp;quot;bold&amp;quot;)) + theme(axis.text.x = element_text(size=12, color=&amp;quot;black&amp;quot;)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried this to get rid of the alphabetizing but my bars are very thin when doing so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;p2 = ggplot(data=clusters_per_sample_2, aes(x=sample, y=Perc, fill=cluster)) +&#xA;geom_bar(stat=&amp;quot;identity&amp;quot;)+ scale_x_discrete(limits=clusters_per_sample_2$sample) +&#xA;heme_minimal() + labs(y=&amp;quot;percentage of total # cells in the sample&amp;quot;) + theme(axis.text.x = element_text(angle = 90, face = &amp;quot;bold&amp;quot;)) + theme(axis.text.x = element_text(size=12, color=&amp;quot;black&amp;quot;)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="6827" LastActivityDate="2021-08-06T17:42:59.077" Title="Barplot without alphabetizing x-axis" Tags="|r|ggplot2|graphs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17512" PostTypeId="2" ParentId="17511" CreationDate="2021-08-06T17:42:59.077" Score="0" Body="&lt;p&gt;ggplot2 uses factors on the underlying data frame. To have the data plotted in a custom order, apply a custom ordered factor on the underlying dataset.&lt;/p&gt;&#xA;&lt;p&gt;For example, if &lt;code&gt;clusters_per_sample$sample&lt;/code&gt; is your custom-ordered vector, do this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;clusters_per_sample&lt;span class=&quot;math-container&quot;&gt;$sample &amp;lt;- factor(clusters_per_sample$&lt;/span&gt;sample, levels = unique(clusters_per_sample$sample), ordered = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now when you plot &lt;code&gt;sample&lt;/code&gt; as an aesthetic, ggplot has a factor component to work with already, so it won't try and make a new one (this new factor would have been in alphabetical order, which we circumvent with the &lt;code&gt;ordered=TRUE&lt;/code&gt; parameter in our custom factor).&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2021-08-06T17:42:59.077" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17513" PostTypeId="2" ParentId="17510" CreationDate="2021-08-06T23:55:55.733" Score="1" Body="&lt;p&gt;Mitochondrial gene filtering is done on a case-by-case basis, which is why there's a bit of discussion about choosing appropriate thresholds in the &lt;a href=&quot;https://satijalab.org/seurat/articles/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seurat PBMC3k vignette&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In your specific case, there are a lot of cells with very high mitochondrial gene content. It would be a good idea to work out why that is before doing any more filtering.&lt;/p&gt;&#xA;&lt;p&gt;However, you're asking specifically about &lt;code&gt;sctransform&lt;/code&gt;, and &lt;a href=&quot;https://satijalab.org/seurat/articles/sctransform_vignette.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the SCtransform vignette&lt;/a&gt; does account for mitochondrial gene content in the transformation:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;During normalization, we can also remove confounding sources of variation, for example, mitochondrial mapping percentage&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;# store mitochondrial percentage in object meta data&#xA;pbmc &amp;lt;- PercentageFeatureSet(pbmc, pattern = &amp;quot;^MT-&amp;quot;, col.name = &amp;quot;percent.mt&amp;quot;)&#xA;&#xA;# run sctransform&#xA;pbmc &amp;lt;- SCTransform(pbmc, vars.to.regress = &amp;quot;percent.mt&amp;quot;, verbose = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There's no filtering in that vignette, but there's also no mention of how clean the data was initially. It's still a good idea to look at your data for outliers, try to understand why they are outliers, and have a think about whether or not to exclude them.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-08-06T23:55:55.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17516" PostTypeId="2" ParentId="17509" CreationDate="2021-08-07T18:08:00.623" Score="2" Body="&lt;p&gt;&lt;strong&gt;August 2021&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;To date, it seems that the response to this question is &lt;strong&gt;&amp;quot;No&amp;quot;&lt;/strong&gt;. However, there is a GitHub repository called &lt;a href=&quot;https://github.com/r-bioinformatics/edgePy&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;edgePy&lt;/code&gt;&lt;/a&gt;, aiming to &lt;em&gt;&amp;quot;become an implementation of &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/edgeR.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;edgeR&lt;/code&gt;&lt;/a&gt; for differential expression analysis in the Python language&amp;quot;&lt;/em&gt; (&lt;code&gt;edgeR&lt;/code&gt; is somewhat similar to &lt;code&gt;limma&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;PS: Many thanks to @ATpoint90 (the Twitter &lt;a href=&quot;https://twitter.com/ATpoint90/status/1423669033702940678&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;hive mind&amp;quot; summoner&lt;/a&gt;) and @sridnona6 (&lt;code&gt;edgePy&lt;/code&gt; developer).&lt;/p&gt;&#xA;" OwnerUserId="12603" LastEditorUserId="73" LastEditDate="2021-08-11T01:37:57.120" LastActivityDate="2021-08-11T01:37:57.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17517" PostTypeId="1" CreationDate="2021-08-07T19:28:06.633" Score="0" ViewCount="303" Body="&lt;p&gt;I am working with a high performance cluster computer containing 112 threads. I am trying to align PacBio transcriptome reads against the genome to count the gene number. For pair end read i used the following workflow:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#convert gff to gtf&#xA;/home/software/cufflinks-2.2.1/gffread xxx.gff -T -o xxx.gtf&#xA;#build index&#xA;/home/software/hisat2-2.2.1/hisat2_extract_exons.py xxx.gtf &amp;gt; xxx.exon&#xA;/home/software/hisat2-2.2.1/hisat2_extract_splice_sites.py xxx.gtf &amp;gt; xxx.ss&#xA;/home/software/hisat2-2.2.1/hisat2-build -p 20 xxx.fa --ss xxx.ss --exon xxx.exon xxx_tran&#xA;#align reads&#xA;/home/software/hisat2-2.2.1/hisat2 -p 80 -x ./xxx_tran -1 R1.fq -2 R2.fq -S gonad.sam&#xA;&#xA;#convert to bam and sort by read name&#xA;samtools view -bS gonad.sam -o gonad.bam&#xA;&#xA;samtools sort -@ 80 gonad.bam -o gonad.sorted.bam&#xA;&#xA;#count gene&#xA;htseq-count -f bam --strand=no gonad.sorted.bam xxx.all.gtf &amp;gt; gonad-counts.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But in case of Long read i am not sure what i need to do during the alignment stage. I have tried with minimap2 like as following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#align reads&#xA;/home/software/minimap2-2.17_x64-linux/minimap2 -ax splice:hq -uf yyy.fa gill.fq &amp;gt; gill.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;in following, i stucked in counting stage:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#count gene&#xA;htseq-count -f bam --strand=no gill.sorted.bam yyy.gtf &amp;gt; gill-counts.txt&#xA;error txt:&#xA;[E::idx_find_and_load] Could not retrieve index file for 'gill.sorted.bam'&#xA;100000 GFF lines processed.&#xA;200000 GFF lines processed.&#xA;300000 GFF lines processed.&#xA;400000 GFF lines processed.&#xA;438395 GFF lines processed.&#xA;[E::idx_find_and_load] Could not retrieve index file for 'gill.sorted.bam'&#xA;Error occured when processing input (record #79 in file gill.sorted.bam):&#xA;  'NoneType' object has no attribute 'encode'&#xA;  [Exception type: AttributeError, raised in _HTSeq.pyx:1379]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My question is that during counting it requires sorted bam and genome gtf file that was generated succesfully during previous staps, why it is getting failed even though the same pipeline is absolutely fine for pair end reads? Hisat2 can align long read? Is there any other way do that? As a novice i will be always grateful to you. Thanks.&lt;/p&gt;&#xA;&lt;p&gt;Update: I tried generating an index:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# generate index&#xA;samtools index gill.sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the output was a .bai file like &amp;quot;gill.sorted.bam.bai&amp;quot;. And I tried again to count as written above. But then I found the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;100000 GFF lines processed.&#xA;200000 GFF lines processed.&#xA;300000 GFF lines processed.&#xA;400000 GFF lines processed.&#xA;438395 GFF lines processed.&#xA;Error occured when processing input (record #79 in file gill.sorted.bam):&#xA;  'NoneType' object has no attribute 'encode'&#xA;  [Exception type: AttributeError, raised in _HTSeq.pyx:1379]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there any way?&lt;/p&gt;&#xA;&lt;p&gt;Update 2:&lt;/p&gt;&#xA;&lt;p&gt;Here is what the BAM file looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SRR6418045.89565    2048    PYXC01000001.1  9893    60  5794H11M1I42M1I3M2I19M1D20M1I15M1D20M3D43M1I11M3D8M1I19M3I24M1I7M1I4M1I5M1D6M1D14M1I45M1D2M1D3M1D4M1I8M1I25M1D21M1D62M1D3M1D25M2D21M2I30M1I25M1D2M1D5M2I4M1D13M1D12M1D10M1I59M21564H    *   0   0   GAATACATAAACTGGATAGCGCACTGTTAATATTTACAGTGAAGAGAGTCTCTTCGAATTTTCCTCCTCAATCATCTTATTAGAAAGATGTTGAGTGGGCAGCAGCCAAATAAAATACTGTTCAGTGATCACTTCCAATGATGCTGTGCACTTACAGAAGCATGTATGGTACACAGTGTTATGCCTAATTTAAAAAACTAATGTAAATTATTTAAAAGTTTTTTTTTTTTTTATGAACTTTTTAGCCACATTGAATGGCTTCAGAATATCTACAGCCTAATGTATTTTCTTAGTAGCCTAATGAAGAAAGAATATTACCTTAAGTTACCATAGATTGACGCCAAAACTCGTCAAATCTCAAACCCCTAACATTTGTTCAACTGTTGTATACATGCTCATACACAGTGTTTCTTGTGTTCCTTAACCCACCGGGTAAAAAGTGGAGGTCCATGTCTGATACGTAGTTATCAATGTACTGTTGCCCATTTAGAGAGCACTTTGGTTTTCCCTCACCAGATAAGGTTTTTTCTCACTTTCAGTGTAAACCTGAGACTTGTTTGCACTTAATATTGGGGATTGTGTAACTCTTATTCAAGGAATTGCCTGTTTTCCAATTTGGTCTGATTACACTAACCTTTGTGTAAAATTATGGTTCAAATAAACTGTATCACA    -)/////,+$/&amp;quot;/..+//-/-+/.-/-.*,./.,)//'//.+/./..-/./../&amp;quot;/*/.),#)*.)-+)*-(///&amp;amp;.//.*'/,*////*-,./+.++#&amp;quot;,.///,/,*,/-,)/,/)/.-)/.///./-..-*++)-(/*,../..)+//.////+*...////./...(././,/.,*.-'*/)(.-*)&amp;amp;(#'&amp;amp;,(#'....').,,$*')+#&amp;amp;,,)%%%$&amp;amp;,)*)*(*,..,'/-.+*)*,)+',-).-)*//,-&amp;quot;,-+./)///'-.//,-+,.*//.-/%,**)-..../.*/,*'/.*//,*/.,(&amp;amp;.,+-)-'-*/*-/.-/-&amp;amp;+$&amp;amp;,'.'.&amp;amp;,.*,)/-.&amp;quot;/,.,*/-//.,)/*,%.+*)/+.,--,./*--/.-.//'///)/--/$&amp;amp;//-//.//.-)/),,./-)+&amp;amp;../*,*,..-/+,)/,,,-*,/-,/.(.(///./(////./&amp;amp;-./-/+//*...////,&amp;amp;%+$&amp;amp;%#%$$--.././/.-+',-,#(.&amp;amp;%(-/,*/.///*..--++*)/-.*..*)*(,$*/--*.+./'//...-..)-.&amp;amp;.-)&amp;amp;))+.-,&amp;amp;#'-'--..'/'-//',//-*.)/+-*....%../.'(/*-$./,(.(////.#/..+/+%/'-'%...//*$+#--+/&amp;amp;-,-/,*-.-*/-.&amp;amp;../-,%/    NM:i:55 ms:i:343    AS:i:343    nn:i:0  ts:A:+  tp:A:P  cm:i:77 s1:i:460    s2:i:0  de:f:0.0656 SA:Z:PYXC01000001.1,9893,+,4147S672M2I23209S,60,37;PYXC01000001.1,9893,+,847S669M26I26488S,60,42;PYXC01000001.1,9893,+,26975S673M3I379S,60,44;PYXC01000001.1,9910,-,10980S654M9I16387S,60,41;PYXC01000001.1,9900,-,20736S661M9I6624S,60,43;PYXC01000001.1,9893,+,7409S673M15I19933S,60,49;PYXC01000001.1,9893,-,27284S667M12I67S,60,52;PYXC01000001.1,9893,-,19099S673M3I8255S,60,48;PYXC01000001.1,9893,-,7656S673M1652I18049S,4,1688;PYXC01000001.1,9908,+,2517S649M17I24847S,60,51;PYXC01000001.1,9893,+,23715S672M1D3643S,60,51;PYXC01000001.1,9893,-,25633S669M22I1706S,60,53;PYXC01000001.1,9893,-,17479S668M1I9882S,60,53;PYXC01000001.1,9893,-,14214S673M25I13118S,22,55;PYXC01000001.1,9899,+,13933S663M13I13421S,60,55;PYXC01000001.1,9900,-,22352S658M21I4999S,60,55;PYXC01000001.1,9901,+,15556S665M7I11802S,60,60;PYXC01000001.1,9893,+,12284S668M9I15069S,60,58;PYXC01000001.1,9909,-,24008S657M16I3349S,60,57;PYXC01000001.1,9902,+,25358S661M10I2001S,60,60;PYXC01000001.1,9893,+,10659S671M2D16700S,60,60;PYXC01000001.1,9893,+,9039S668M10I18313S,60,51;PYXC01000001.1,9893,-,2790S673M21I24546S,60,65;PYXC01000001.1,9899,+,22114S669M8I5239S,60,59;PYXC01000001.1,9893,-,6025S673M17I21315S,60,64;PYXC01000001.1,9908,-,12604S645M13I14768S,60,64;PYXC01000001.1,9893,-,4418S673M3I22936S,60,71;PYXC01000001.1,9893,-,1162S673M15I26180S,60,70;PYXC01000001.1,9898,+,17178S641M1I10210S,60,63;PYXC01000001.1,9893,+,20477S673M13I6867S,60,67;PYXC01000001.1,10390,-,64S160M2D27806S,60,8;   rl:i:2011&#xA;SRR6418045.89565    2048    PYXC01000001.1  9893    60  12284H12M1D27M1D9M1I15M1I26M1D19M2I16M2I2M1I17M1I32M1I31M2D8M3I25M1I6M1I19M2I12M1D32M2D15M1I6M2I7M1D4M2D17M1I10M1I32M1I36M1D4M1D2M1D20M1I59M1I11M1D10M1D10M1I24M1I9M2D2M1D28M1I24M1I11M15069H   *   0   0   GAATACATAAATGATAGCGCACTGTTAATATTTACAGTGAGAGAGTCTACTTGAATTCCTCCTCAAATCATCTTATTTAGAAAGATGTTGATGGGAGCAACCAAATAAAAATCTACTGTTCAGTGATCCAACTTTCACAAAATGATGCTGTTGCACTTACAGAAGCATGTATGGTACACAGTGTTATGACTATTTTAATAAAAAACAATGTAAATTTTAAAAGGTTTTTTTTTTTTTATGAACTTTTTAGCAACATTGAATGGTTCAGAAATATCTTAGCAAGCCTAATGTATTCTTAGTAGCCTAATGAAGAAAGAATATTACTAAGTTACCAGTAGGAATCTGAACGCCAAAATGTCATCTCAAACCCCTAACATTTTGGTTCAATCTGTTGTATACATGCATCATACACAGTGTTTCTTTGTGTTCCTTTAACCACCGGGTAAAAAGTGGAGGTCATGCTATTACTGTAGTTATCAATGTAACTGTTGCCCACTTTTAGAGAGCACTTTGGTTTTCTCACCAGATAAGGTTTTTTCTCACTTTTTATGTAAACTGAGACTTGTTGCACTGTACTATATTGAATATGTGTAACTCTTATTTTCAAGGATGCCTGTTTTCCATTTGGTCTATTAACACTAACACTTTGTGTAAACTTATGGTTCAAAATAAACTGT   -*//..//-*./%/,/./,,///.,/+//*/,////..,*..+/..-,(*.,.%&amp;amp;&amp;quot;.*./%-..((//$///...)+-&amp;amp;.+'/////.+*$(),,../$.'.,)//-++$..&amp;quot;//,-.,/////'//,/$...*//./-()./+//,///,...%/-//,*.*/.//&amp;amp;/.//...///./..(-.///#/.+$(#.'..((%*(.)(-+-..%.*,((,,(..(-&amp;amp;+*)*******..+/$-..,++..--$-..'-.)//..-)/.,,*...//./.#.&amp;amp;/)*/)*(/./*,.&amp;amp;-(+////*/.).).+//-*//*)/*.*..-(//-/&amp;amp;,/-//..)*//.+.$/.',-+,(+(&amp;amp;//..//.,).%')./*///.--)--*.'(/&amp;quot;//...//.././/-)'(/,'..)%/./..&amp;amp;..$*,+-.-/*/++-%.+--+.),-,#,&amp;amp;(//.-,).-/./-.'./..///....-(*-/)/.#-$.//.%$*(*..())+/--/.../..(,+,..*).-/#.*'.+/-(/...&amp;amp;'+)/--.+'-))&amp;amp;$,'//*'/.-/(/.).,/,-,*.,/.*.&amp;quot;.)..)./*/*/+/.'/%/&amp;amp;-),/..,)).(/--/&amp;amp;,*/..,*)-#+/.,-,./-**./%.//+/*)().(+-,,-.%'//../--.--/)*#..,*///.-   NM:i:58 ms:i:325    AS:i:325    nn:i:0  ts:A:+  tp:A:P  cm:i:65 s1:i:397    s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12579" LastEditorUserId="73" LastEditDate="2021-08-15T02:26:42.557" LastActivityDate="2021-08-15T02:26:42.557" Title="Hisat2 compatibility for long reads (Pacbio)" Tags="|gene|transcriptome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17518" PostTypeId="1" AcceptedAnswerId="17522" CreationDate="2021-08-07T20:53:43.707" Score="1" ViewCount="183" Body="&lt;p&gt;Coming from devops background, working on automating usual genomics pipelines, I am benchmarking existing pipelines in order to choose cloud resources properly.&lt;/p&gt;&#xA;&lt;p&gt;Past week most of work was on testing &lt;code&gt;bwa mem&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I've noticed that &lt;code&gt;bwa&lt;/code&gt; takes about 10 minutes to start processing data. When monitoring &lt;code&gt;bwa&lt;/code&gt; resource usage I've noticed that RAM is slowly building up to about 6GB. That is amount of reference indexed data.&lt;/p&gt;&#xA;&lt;p&gt;After bwa loads the reference first time, any subsequent run is fast. But why this preloading is happening so slow? My reference data set is already indexed with &lt;code&gt;bwa index&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;When I monitor disk activity with &lt;code&gt;iotop&lt;/code&gt; I noticed that &lt;code&gt;disk read&lt;/code&gt; is only &lt;code&gt;5MB/s&lt;/code&gt;, when I run &lt;code&gt;bwa mem&lt;/code&gt; alignment (only first time), which is very slow. Performance of the drive is lot more than that and a simple &lt;code&gt;copy&lt;/code&gt; test confirms that.&lt;/p&gt;&#xA;&lt;p&gt;Is it a &amp;quot;limitation&amp;quot; of &lt;code&gt;bwa&lt;/code&gt;, is this slow preloading expected to be like that and I should just accept it and move on forward or there is something I don't understand and maybe someone can help me understand what could I do?&lt;/p&gt;&#xA;&lt;p&gt;The same &amp;quot;slow preload&amp;quot; happens when I run &lt;code&gt;bwa shm /path/to/ref&lt;/code&gt;.&#xA;Then &lt;code&gt;bwa mem&lt;/code&gt; starts immediately. But again slow preload is something that is bothering me and I am trying to understand what can I do about it.&lt;/p&gt;&#xA;" OwnerUserId="13937" LastEditorUserId="13937" LastEditDate="2021-08-08T07:55:46.167" LastActivityDate="2021-08-09T10:36:49.630" Title="Slow bwa shm &quot;preloading&quot; into memory" Tags="|bwa|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17519" PostTypeId="2" ParentId="5277" CreationDate="2021-08-07T23:56:19.410" Score="-1" Body="&lt;p&gt;If you have both cluster assignments, you can check for consistency of assignment by looking at pairwise counts using &lt;code&gt;table&lt;/code&gt; in R:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cons.mat &amp;lt;- table(clusters.1, clusters.2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The result of this will be a matrix showing the number of cells / samples that have the observed cluster assignments via each method. If the two cluster assignments are similar, then samples from one cluster via one method should be mostly assigned to one cluster in the other method. You can check this by comparing table cell values with their row/column sum:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## show consistency along the first dimension&#xA;apply(cons.mat, 1, function(x){max(x / sum(x))})&#xA;## show consistency along the second dimension&#xA;apply(cons.mat, 2, function(x){max(x / sum(x))})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-08-07T23:56:19.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17520" PostTypeId="2" ParentId="17517" CreationDate="2021-08-08T00:00:27.297" Score="1" Body="&lt;p&gt;This error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[E::idx_find_and_load] Could not retrieve index file for 'gill.sorted.bam'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Means that you have not generated an index for a BAM file. This needs to be done for most downstream processing of BAM files:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# generate index&#xA;samtools index gill.sorted.bam&#xA;# count genes&#xA;htseq-count -f bam --strand=no gill.sorted.bam yyy.gtf &amp;gt; gill-counts.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Update (after second error):&lt;/p&gt;&#xA;&lt;p&gt;I've got no idea what's going wrong; this is not an error that I would expect to see, and doesn't look related to PacBio input. I wonder if the error is related to the number of threads used in the sort command, given that it happens at record 79, and you've specified 80 threads. The next things that I would do would be to look at the input files around where the error is to try to work out what's going wrong:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view gill.sorted.bam | head -n 85 | tail -n 10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-08-08T12:23:47.187" LastActivityDate="2021-08-08T12:23:47.187" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17521" PostTypeId="2" ParentId="17518" CreationDate="2021-08-08T12:31:58.507" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @user438383, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;BWA has slow indexing due to the indexing size; this can be improved by using bwa-mem2:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/bwa-mem2/bwa-mem2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/bwa-mem2/bwa-mem2&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We are happy to announce that the index size on disk is down by 8 times and in memory by 4 times due to moving to only one type of FM-index (2bit.64 instead of 2bit.64 and 8bit.32) and 8x compression of suffix array. For example, for human genome, index size on disk is down to ~10GB from ~80GB and memory footprint is down to ~10GB from ~40GB. &lt;em&gt;&lt;strong&gt;There is a substantial reduction in index IO time due to the reduction&lt;/strong&gt;&lt;/em&gt; and hardly any performance impact on read mapping.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-08-08T23:25:33.580" LastActivityDate="2021-08-08T23:25:33.580" CommentCount="1" CommunityOwnedDate="2021-08-08T12:31:58.507" ContentLicense="CC BY-SA 4.0" />
  <row Id="17522" PostTypeId="2" ParentId="17518" CreationDate="2021-08-09T00:24:24.010" Score="1" Body="&lt;p&gt;I'll add an answer to my own question, which was asked out of lack of experience with cloud (AWS) EC2 service.&lt;/p&gt;&#xA;&lt;p&gt;The way I was testing &lt;code&gt;bwa&lt;/code&gt; is from a custom AMI image created for our pipelines.&lt;/p&gt;&#xA;&lt;p&gt;The image has &amp;quot;preloaded&amp;quot; reference data that we are using for alignement so that we spare some time downloading reference data from &lt;code&gt;S3&lt;/code&gt; and spare some &lt;code&gt;S3&lt;/code&gt; api calls since we have a lot of files used as reference data.&lt;/p&gt;&#xA;&lt;p&gt;Simply our image is single 200GB EBS volume used with &lt;code&gt;AWS ECS AMI&lt;/code&gt; where we stored our reference data and created a custom image for Batch out of it. Then we use that AMI for AWS Batch jobs.&lt;/p&gt;&#xA;&lt;p&gt;Once I used &lt;code&gt;iotop&lt;/code&gt; (after few days of testing) to monitor disk activity I've noticed that &lt;code&gt;bwa&lt;/code&gt; loads index into memory very slow, cca &lt;code&gt;5MB/s&lt;/code&gt;. Syncing files from S3 was faster then using this &lt;code&gt;snapshot&lt;/code&gt; approach.&lt;/p&gt;&#xA;&lt;p&gt;For benchmarking I've used the same AMI manually launched. In the AWS documentation there is part which explains why and how you should optimize EBS when AMI is launched from a snapshot.&#xA;&lt;a href=&quot;https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-initialize.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-initialize.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;After spining up fresh EC2, syncing reference data and running &lt;code&gt;bwa&lt;/code&gt; everything was clear. My EBS volume was slow and had to be initialized first.&lt;/p&gt;&#xA;&lt;p&gt;That is all. My lack of EBS storage knowledge was the issue.&lt;/p&gt;&#xA;&lt;p&gt;This helped me understand that I should not use custom AMI with ref data stored and will probably use EFS or simply syncing the data from S3 inside Docker entrypoint.&lt;/p&gt;&#xA;" OwnerUserId="13937" LastEditorUserId="13937" LastEditDate="2021-08-09T10:36:49.630" LastActivityDate="2021-08-09T10:36:49.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17523" PostTypeId="1" AcceptedAnswerId="17528" CreationDate="2021-08-09T02:59:45.910" Score="0" ViewCount="985" Body="&lt;p&gt;So I have a file, let's call it pcawg.tsv. It's formatted like the one down below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Tumor sample ID  Normal sample ID  Mechanism&#xA;a                1                 deletion&#xA;b                2                 ecDNA&#xA;c                3                 ecDNA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have a file of tumor sample ID's with the ecDNA mechanism called ecDNA_samples.tsv. It's formatted like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Tumor sample ID&#xA;a&#xA;b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To filter and extract the lines of &lt;code&gt;pcawg.tsv&lt;/code&gt; that contain the tumor sample ID's in &lt;code&gt;ecDNA_samples.tsv&lt;/code&gt; into a new file, I used this line of code below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -f ecDNA_samples.tsv pcawg.tsv &amp;gt; ecDNA.vcf.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Would this be the right code to use?&lt;/p&gt;&#xA;" OwnerUserId="12859" LastEditorUserId="73" LastEditDate="2021-08-09T23:51:19.687" LastActivityDate="2021-08-09T23:58:22.937" Title="Using grep to get lines from a file (.tsv) that contain specific samples listed in a sample file (.tsv)" Tags="|linux|cancer|filtering|awk|grep|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17525" PostTypeId="1" CreationDate="2021-08-09T13:27:52.503" Score="1" ViewCount="100" Body="&lt;p&gt;I have been doing single cell analysis based on seurat and I realized after each clustering and reclustering when I run SCTransform, in my code there is vars.to.regress = percent.mt and I was running this everytime. I removed MT- contaminated cells before.&lt;/p&gt;&#xA;&lt;p&gt;My clustering seems fine though.&lt;/p&gt;&#xA;&lt;p&gt;Do you think this is very critical thing and I have to analyze everything from scratch?&lt;/p&gt;&#xA;" OwnerUserId="12472" LastActivityDate="2022-05-08T03:03:30.357" Title="Seurat - single cell analysis" Tags="|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17526" PostTypeId="1" AcceptedAnswerId="17527" CreationDate="2021-08-09T17:31:24.090" Score="3" ViewCount="495" Body="&lt;p&gt;I have a file with a 3-column list of pairwise distances:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A B distance&#xA;A C distance&#xA;B C distance&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And I would like to calculate a phylogenetic tree for all the nodes in the list. I've seen &lt;code&gt;phylip neighbor&lt;/code&gt; and similar options, but it seems like the length of the ids is a limit, which I rather not have to deal with. What would be a straightforward way of getting a newick tree output out of an input as above?&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2021-08-10T14:04:31.100" Title="calculating phylogenetic tree from pairwise distances" Tags="|phylogeny|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17527" PostTypeId="2" ParentId="17526" CreationDate="2021-08-09T18:36:34.813" Score="3" Body="&lt;p&gt;I believe there are a number of ways to construct a tree metric from a distance metric. A very straightforward method, neighbor joining, is available in the &lt;a href=&quot;http://scikit-bio.org/docs/latest/generated/skbio.tree.nj.html?highlight=neighbor%20join&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sciki-Bio package&lt;/a&gt;. A less straightforward option with more freedom is by using the &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/cluster.hierarchy.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy.cluster.hierarchy&lt;/a&gt; module to obtain a linkage matrix, then using the &lt;code&gt;to_tree()&lt;/code&gt; method to obtain a scipy &lt;code&gt;Tree&lt;/code&gt; object, and finally writing a script to &lt;a href=&quot;https://stackoverflow.com/questions/28222179/save-dendrogram-to-newick-format&quot;&gt;convert a scipy Tree object to a newick string&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I've copied the example from scikit-bio below:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; from skbio import DistanceMatrix&#xA;&amp;gt;&amp;gt;&amp;gt; from skbio.tree import nj&#xA;&amp;gt;&amp;gt;&amp;gt;&#xA;&amp;gt;&amp;gt;&amp;gt; data = [[0,  5,  9,  9,  8],&#xA;...         [5,  0, 10, 10,  9],&#xA;...         [9, 10,  0,  8,  7],&#xA;...         [9, 10,  8,  0,  3],&#xA;...         [8,  9,  7,  3,  0]]&#xA;&amp;gt;&amp;gt;&amp;gt; ids = list('abcde')&#xA;&amp;gt;&amp;gt;&amp;gt;&#xA;&amp;gt;&amp;gt;&amp;gt; dm = DistanceMatrix(data, ids)&#xA;&amp;gt;&amp;gt;&amp;gt; # Or from a tsv file with header, no column 0&#xA;&amp;gt;&amp;gt;&amp;gt; reader = csv.reader(open(args.inputfile), delimiter=&amp;quot;\t&amp;quot;)&#xA;&amp;gt;&amp;gt;&amp;gt; x = list(reader)                                         &#xA;&amp;gt;&amp;gt;&amp;gt; dm = DistanceMatrix(x[1:],x[0])                          &#xA;&amp;gt;&amp;gt;&amp;gt;&#xA;&amp;gt;&amp;gt;&amp;gt; newick_str = nj(dm, result_constructor=str)&#xA;&amp;gt;&amp;gt;&amp;gt; print(newick_str[:55], &amp;quot;...&amp;quot;)&#xA;(d:2.000000, (c:4.000000, (b:3.000000, a:2.000000):3.00 ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Note: Both methods require you to supply the input as a numpy distance matrix&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6628" LastEditorUserId="180" LastEditDate="2021-08-10T14:04:31.100" LastActivityDate="2021-08-10T14:04:31.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17528" PostTypeId="2" ParentId="17523" CreationDate="2021-08-09T19:12:31.583" Score="1" Body="&lt;p&gt;This may help:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat ecDNA_samples.tsv | xargs -I % grep '^%\s' pcawg.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The command above executes &lt;code&gt;grep '^&amp;lt;line&amp;gt;\s' pcawg.tsv&lt;/code&gt;, for each line of ecDNA_samples.tsv. Which matches the lines taken from ecDNA_samples.tsv to the sequences of characters between line start and a white-space in pcawg.tsv.&lt;/p&gt;&#xA;" OwnerUserId="13945" LastEditorUserId="13945" LastEditDate="2021-08-09T22:16:07.147" LastActivityDate="2021-08-09T22:16:07.147" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17529" PostTypeId="2" ParentId="17523" CreationDate="2021-08-09T23:58:22.937" Score="0" Body="&lt;p&gt;Just in case an R solution is acceptable as well, this should be fairly robust code because it properly parses the TSV files and matches up column names:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env Rscript&#xA;&#xA;library(tidyverse)&#xA;read_tsv(&amp;quot;pcawg.tsv&amp;quot;) %&amp;gt;%&#xA;   inner_join(read_tsv(&amp;quot;ecDNA_samples.tsv&amp;quot;)) %&amp;gt;%&#xA;   write_tsv(&amp;quot;filtered_pcawg.tsv&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-08-09T23:58:22.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17531" PostTypeId="1" CreationDate="2021-08-10T12:08:27.870" Score="0" ViewCount="271" Body="&lt;p&gt;I'm running FindAllMarkers&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;markers &amp;lt;- FindAllMarkers(scData_scTransform_PTX_only_res04_20PC_6000g_30mt_sb_removal_20210713_ann_exp0_counts, min.pct = 0.25, logfc.threshold = 0.25)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But was wondering if there is a function in seurat to get the raw (non-normalized) expression data from each cell type cluster&lt;/p&gt;&#xA;" OwnerUserId="6827" LastEditorUserId="6827" LastEditDate="2021-08-19T18:08:06.253" LastActivityDate="2021-08-19T18:08:06.253" Title="functions in seurat to calculate the gene count per cluster" Tags="|r|gene|seurat|single-cell|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17532" PostTypeId="1" CreationDate="2021-08-10T13:18:42.587" Score="1" ViewCount="730" Body="&lt;p&gt;I am working with single cell data and using seurat to analyze the results. Often in manuscript, we see the dotplots showing the expression of the marker genes or genes of interest across the different clusters. My question here is:&#xA;a. When comparing expression of a gene (lets say geneA) across cluster, for example, lets say for cluster 0, the average expression for geneA that is plotted, is it calculated considering all the cells in the sample or all the cells in the cluster0? If it is calculating the average expresison based on total number cells present in cluster 0 (here), is it correct to compare the expression of geneA across all the clusters?&#xA;Because among the different clusters, the number of cells are different, and the number of cells that are expressing geneA can vary alot across the clusters.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="12804" LastActivityDate="2021-08-10T13:18:42.587" Title="Understanding the dot plot from seurat" Tags="|scrnaseq|seurat|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17533" PostTypeId="1" CreationDate="2021-08-10T14:09:04.220" Score="1" ViewCount="25" Body="&lt;p&gt;I am a computational genomics student who has recently joined a research lab as an RA. We often work with oncoprints (comuts/heatmaps), and we visualize molecular signatures by reorganizing the comut. Although this is the first-line approach, what are some statistical ways to identify molecular signatures? I have heard of NNF, clustering, etc, but I also heard a colleague mention using cox analysis (which I thought was for survival analysis?).&lt;/p&gt;&#xA;&lt;p&gt;My Background:&#xA;While introductory statistics has been hammered into my head, I have less experience with higher level computational techniques, like PCA, clustering. I am familiar with the general purpose of the statistical approaches I've mentioned, but not confident in when or how to use them. For example, I know the general purpose of PCA, but I did not know that if you have categorical binary data and want to do unsupervised clustering, you should do logisticPCA. I would appreciate any advice, general or relevant to the question at hand. I am eager to be competent, confident, and contribute to my lab! Thank you :)&lt;/p&gt;&#xA;" OwnerUserId="13949" LastActivityDate="2021-08-10T14:09:04.220" Title="General Question: Statistical Alternatives to Oncoprint" Tags="|statistics|genomics|clustering|heatmap|statsmodels|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17534" PostTypeId="1" CreationDate="2021-08-10T14:15:25.507" Score="1" ViewCount="109" Body="&lt;p&gt;I have &lt;code&gt;10X scRNA-seq multiome's 3' poly-A capture (scRNA-seq+ATAC-seq)&lt;/code&gt; from PBMC&lt;/p&gt;&#xA;&lt;p&gt;Using Seuratreference mapping, I mapped my scRNA-seq part on reference PBMC (for cell type annotation)&lt;/p&gt;&#xA;&lt;p&gt;I have also tried Seurat zimuth shiny app&lt;/p&gt;&#xA;&lt;p&gt;But likely just a few of my cells has been mapped on the reference&lt;/p&gt;&#xA;&lt;p&gt;Please look at this&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UEGmA.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UEGmA.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Do you think this because my data is multiome or something else is going wrong?&lt;/p&gt;&#xA;&lt;p&gt;Please any body has some experience help me&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2023-04-18T15:52:55.760" Title="Problem with Seurat reference mapping" Tags="|scrnaseq|seurat|sequence-annotation|10x-genomics|cellranger|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17535" PostTypeId="1" AcceptedAnswerId="17537" CreationDate="2021-08-10T15:07:48.287" Score="-1" ViewCount="792" Body="&lt;p&gt;I have a file containing protein sequence information of 100 odd proteins. Each sequence starts with a header that looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;sp|Q9UDW1|QCR9_HUMAN Cytochrome b-c1 complex subunit 9 OS=Homo sapiens OX=9606 GN=UQCR10 PE=1 SV=3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to edit these header to look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Q9UDW1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have tried the following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;with open ('WT.fasta', 'r+') as f:&#xA;    contents = f.readlines()&#xA;    for i in range(len(contents)-1):&#xA;        target = re.compile(r'&amp;gt;..\|(.*)\|.*', contents[i])    &#xA;        target.sub(r'\1', contents[i])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I think I am not being able to call the right methods to do this. I have been trying to read the documentation: &lt;a href=&quot;https://docs.python.org/3/library/re.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.python.org/3/library/re.html&lt;/a&gt;. I am not been able to understand it and feeling overwhelmed by the details. I would really appreciate if anyone could help or point me in the right direction. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="7168" LastActivityDate="2021-08-10T21:16:55.313" Title="How to read and edit a FASTA file with python using regular expression?" Tags="|python|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="17536" PostTypeId="1" CreationDate="2021-08-10T18:06:04.140" Score="1" ViewCount="106" Body="&lt;p&gt;I am currently learning to perform Differential Analysis via DESEQ2 R Package, and I believe I've made progress, able to format the data correctly [maybe] for DDS(). When I run the results function to see the output, the data seems fine, as can be seen below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;log2 fold change (MLE): Group RA vs C  Wald test p-value: Group RA vs&#xA;C  DataFrame with 6 rows and 6 columns&#xA;       baseMean log2FoldChange     lfcSE      stat    pvalue      padj&#xA;      &amp;lt;numeric&amp;gt;      &amp;lt;numeric&amp;gt; &amp;lt;numeric&amp;gt; &amp;lt;numeric&amp;gt; &amp;lt;numeric&amp;gt; &amp;lt;numeric&amp;gt; Gnai3   95.1978      0.3458366  0.275366  1.255916  0.209146  0.999383&#xA;Cdc45   21.9942     -0.0545177  0.346199 -0.157475  0.874870  0.999383&#xA;H19     29.0133     -1.4954220  1.516390 -0.986173  0.324048  0.999383&#xA;Narf    16.6967      0.2571059  0.370880  0.693232  0.488164  0.999383&#xA;Cav2    63.6356      0.5060019  0.354969  1.425480  0.154018  0.999383&#xA;Klf6    65.8469      0.2851175  0.265755  1.072858  0.283335  0.999383&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, when I run summary() or any other function, they do not seem to behave correctly, not counting any stats. Same with MAplot not coloring genes that should be marked red:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;out of 13184 with nonzero total read count adjusted p-value &amp;lt; 0.1 LFC&#xA;&amp;gt; 0 (up)       : 0, 0% LFC &amp;lt; 0 (down)     : 0, 0% outliers [1]       : 150, 1.1% low counts [2]     : 0, 0% (mean count &amp;lt; 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm confused as to why this is occurring and was wondering if anyone had ideas.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: Added More Code for context&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sergey &amp;lt;- (read.csv(&amp;quot;cleanmatrix25+.csv&amp;quot;, sep = &amp;quot;\t&amp;quot;, header = T,&#xA;    check.names = F, row.names = &amp;quot;Gene_names&amp;quot;)) &#xA;&#xA;meta &amp;lt;-(read.csv(&amp;quot;metadata25+.csv&amp;quot;, sep = &amp;quot;\t&amp;quot;, row.names = 1))&#xA; &#xA;dds &amp;lt;- DESeqDataSetFromMatrix(countData = sergey, colData = meta,&#xA;    design = ~Group)&#xA;&#xA;keep &amp;lt;- rowSums(counts(dds)) &amp;gt;= 10 &#xA;dds &amp;lt;- dds[keep,]&#xA;&#xA;dds&lt;span class=&quot;math-container&quot;&gt;$Group &amp;lt;- relevel(dds$&lt;/span&gt;Group, ref = &amp;quot;C&amp;quot;)&#xA;&#xA;dds &amp;lt;- DESeq(dds)&#xA;&#xA;res &amp;lt;- results(dds)&#xA;&#xA;head(res)&#xA;&#xA;summary(res)&#xA;&#xA;plotMA(res, ylim=c(-2,2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;EDIT2:&lt;/p&gt;&#xA;&lt;p&gt;I sorted result by p-value, so all of these should be red on the following MAplot due to having a p-value &amp;lt; 0.1&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QhLwY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QhLwY.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/klZPh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/klZPh.png&quot; alt=&quot;plotMA&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="13950" LastEditorUserId="13950" LastEditDate="2021-08-10T19:40:06.360" LastActivityDate="2021-08-11T08:55:34.577" Title="DESEQ2 Question about results()" Tags="|r|deseq2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17537" PostTypeId="2" ParentId="17535" CreationDate="2021-08-10T19:45:01.620" Score="3" Body="&lt;p&gt;In general, I usually recommend Biopython for any sort of fasta parsing, but the code below should work as well.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;with open ('WT.fasta', 'r+') as in_f, open(&amp;quot;WT_out.fasta&amp;quot;, 'w') as out_f:&#xA;    for line in in_f:&#xA;        if line[0] == &amp;quot;&amp;gt;&amp;quot;:&#xA;            out_f.write(&amp;quot;&amp;gt;&amp;quot; + line.split('|')[1] + &amp;quot;\n&amp;quot;)&#xA;        else:&#xA;            out_f.write(line)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6628" LastActivityDate="2021-08-10T19:45:01.620" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17538" PostTypeId="2" ParentId="17535" CreationDate="2021-08-10T21:16:55.313" Score="3" Body="&lt;p&gt;You could do this with &lt;code&gt;awk&lt;/code&gt; by processing the header:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ awk -v FS=&amp;quot;|&amp;quot; '{ if ($0 ~ /^&amp;gt;/) { printf(&amp;quot;&amp;gt;%s\n&amp;quot;, $2) } else print $0 }' in.fa &amp;gt; out.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;% echo -e &amp;quot;&amp;gt;sp|Q9UDW1|QCR9_HUMAN Cytochrome b-c1 complex subunit 9 OS=Homo sapiens OX=9606 GN=UQCR10 PE=1 SV=3\nACCGT&amp;quot; | awk -v FS=&amp;quot;|&amp;quot; '{ if ($0 ~ /^&amp;gt;/) { printf(&amp;quot;&amp;gt;%s\n&amp;quot;, $2) } else print $0 }' &#xA;&amp;gt;Q9UDW1&#xA;ACCGT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="776" LastActivityDate="2021-08-10T21:16:55.313" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17539" PostTypeId="2" ParentId="17536" CreationDate="2021-08-10T21:27:01.533" Score="2" Body="&lt;p&gt;You do not have any significantly differential genes after FDR correction according to padj, simple as that, which means that either there are none or your sample size is too small (underpowered) given the observed variability of your data. As the MA-plot shows some logFCs that could qualify as large enough to be called DEG it probably comes down to a too small sample size and quite some variability between the replicates of the groups. A PCA can help diagnosing whether there is simply lots of per-group variability or other factors such as batch effects that need to be taken care of. Please see the DESeq2 vignette for code examples or Bioc packages such as &lt;code&gt;PCAtools&lt;/code&gt; for this.&lt;/p&gt;&#xA;&lt;p&gt;Edit: &lt;code&gt;plotMA&lt;/code&gt; by the way colors based on padj, not p-value alone.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2021-08-11T08:55:34.577" LastActivityDate="2021-08-11T08:55:34.577" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17540" PostTypeId="2" ParentId="17534" CreationDate="2021-08-11T01:24:48.413" Score="0" Body="&lt;p&gt;As I mentioned in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/17513/73&quot;&gt;my&lt;/a&gt; &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/16448/73&quot;&gt;answers&lt;/a&gt; to similar questions, there are a lot of cells with very high mitochondrial gene content. It would be a good idea to work out why that is before doing any more downstream processing.&lt;/p&gt;&#xA;&lt;p&gt;These cells will, of course, be filtered out by the thresholds you have set.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-08-11T01:24:48.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17541" PostTypeId="2" ParentId="17525" CreationDate="2021-08-11T01:34:52.363" Score="1" Body="&lt;p&gt;I think it would be a good idea to at least look at the distribution of mitochondrial gene fraction in your cells. If they mostly have low fractions (e.g. less than 15%), then I wouldn't expect the clustering to be influenced too much by them.&lt;/p&gt;&#xA;&lt;p&gt;With SCTransform regressing on the percent.mt, I think it's okay to do post-transform filtering, but it'd be a good idea to exclude any cells that have high mitochondrial fraction prior to doing clustering (unless, for whatever reason, those are the cells you're interested in).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-08-11T01:34:52.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17542" PostTypeId="2" ParentId="15511" CreationDate="2021-08-11T01:44:56.510" Score="0" Body="&lt;p&gt;At a guess, the variable &lt;code&gt;$logfile&lt;/code&gt; is not properly defined (i.e. it hasn't been defined at all). What is the output of the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;echo &amp;quot;logfile: ${logfile}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-08-11T01:44:56.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17543" PostTypeId="1" AcceptedAnswerId="17544" CreationDate="2021-08-11T12:09:48.300" Score="1" ViewCount="444" Body="&lt;p&gt;I'd like to know if there is a simple way to check the concordance between a GTF file and a fasta file (that I use as mapping reference) ?&lt;/p&gt;&#xA;&lt;p&gt;It may be a dumb question, but I suspect discrepancies between my 2 files and I don't want to check this manually&lt;/p&gt;&#xA;" OwnerUserId="12767" LastActivityDate="2021-08-12T14:53:00.147" Title="How to check if a fasta file and a GTF file fit and form a valid pair?" Tags="|fasta|gtf|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17544" PostTypeId="2" ParentId="17543" CreationDate="2021-08-11T12:47:23.113" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @ATpoint, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Check whether chromosome identifiers are the same and both files contain the same chromosomes. You can also check that the end coordinate of the GTF never exceeds the length of the respective chromosome. Maybe you can also check whether the first three nucleotides of every CDS indeed start with a start codon. If this is all true then I guess I would trust this pair.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-08-11T12:47:23.113" CommentCount="0" CommunityOwnedDate="2021-08-11T12:47:23.113" ContentLicense="CC BY-SA 4.0" />
  <row Id="17547" PostTypeId="1" CreationDate="2021-08-11T19:41:31.360" Score="-1" ViewCount="2533" Body="&lt;p&gt;What does the padj really mean? I believe it's a BH method, where it attempts to quantify the likelihood of a &amp;quot;false positive&amp;quot; but I am not sure. Also, what does it mean if the resulting calculations nearly all have the same padj values despite having different p-values?&lt;/p&gt;&#xA;" OwnerUserId="13950" LastActivityDate="2021-09-11T04:07:20.587" Title="Adjusted P-value from DESeq2" Tags="|statistics|deseq2|p-values|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17548" PostTypeId="1" CreationDate="2021-08-11T20:10:24.040" Score="0" ViewCount="859" Body="&lt;p&gt;I am new to PyMOL but have a very specific task that I need to do.&lt;/p&gt;&#xA;&lt;p&gt;I have a PDB structure file of protein homo-oligomer, and I want to use PyMol to determine polar contacts between chain A and all other chains.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;However, I not only want to visualize these polar contacts (which is easy to do via PyMOL GUI), but I also want to be able to store/save pairs of interacting residues&lt;/strong&gt; in a dictionary or whatever may be the most convenient data structure to do this. The end goal to create a sort of &amp;quot;interaction map&amp;quot; in a csv or text file where each interacting pair of residues has its own line.&lt;/p&gt;&#xA;&lt;p&gt;I would ideally like to do this via a Python script where I call the PyMol API or through a PyMol script - whichever is easier.&lt;/p&gt;&#xA;&lt;p&gt;I have seen a &lt;a href=&quot;https://stackoverflow.com/questions/49633809/pymol-select-residues-for-which-i-have-found-hydrogen-bonds&quot;&gt;similar question&lt;/a&gt; on Stack Overflow but found it unhelpful because I not only want to select and visualize these polar contacts but also want to save/record them in a list or dictionary (and then to a text or csv file), which the above question does not address.&lt;/p&gt;&#xA;" OwnerUserId="13953" LastEditorUserId="57" LastEditDate="2021-08-13T18:30:45.500" LastActivityDate="2021-08-16T14:32:00.093" Title="PyMol: Output Polar Contacts Between Chains to Text File" Tags="|python|protein-structure|visualization|pymol|protein-protein-interaction|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17549" PostTypeId="1" AcceptedAnswerId="17553" CreationDate="2021-08-11T21:30:18.343" Score="0" ViewCount="404" Body="&lt;p&gt;What kind of multiple hypothesis testing correction does &lt;code&gt;method=&amp;quot;global&amp;quot;&lt;/code&gt; do in Limma's &lt;code&gt;decideTests&lt;/code&gt; function? According to the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/manuals/limma/man/limma.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;method=&amp;quot;global&amp;quot; will treat the entire matrix of t-statistics as a&#xA;single vector of unrelated tests.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;method=&amp;quot;global&amp;quot; is useful when it is important that the same&#xA;t-statistic cutoff should correspond to statistical significance for&#xA;all the contrasts.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What does this really mean? Does it use a Bonferroni correction?&lt;/p&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="8601" LastEditDate="2021-08-12T07:32:45.383" LastActivityDate="2021-08-12T19:05:30.987" Title="Limma decideTests function: what kind of multiple hypothesis testing correction does parameter &quot;method&quot; involve?" Tags="|rna-seq|limma|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17550" PostTypeId="2" ParentId="17547" CreationDate="2021-08-12T03:50:14.363" Score="1" Body="&lt;p&gt;If we look at &lt;a href=&quot;https://hbctraining.github.io/DGE_workshop/lessons/05_DGE_DESeq2_analysis2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation of DESEQ2&lt;/a&gt; and search &amp;quot;adjusted p-value&amp;quot;, we find the section &amp;quot;Multiple test correction&amp;quot;. In this section they discuss Benjamini-Hochberg (&amp;quot;BH&amp;quot;) false discovery rate (FDR) correction procedures:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;FDR/Benjamini-Hochberg: Benjamini and Hochberg (1995) defined the concept of FDR and created an algorithm to control the expected FDR below a specified level given a list of independent p-values. &lt;strong&gt;An interpretation of the BH method for controlling the FDR is implemented in DESeq2 in which we rank the genes by p-value, then multiply each ranked p-value by m/rank.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Earlier, they write:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;m = total number of tests&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;They further add:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;So what does FDR &amp;lt; 0.05 mean? By setting the FDR cutoff to &amp;lt; 0.05, we’re saying that the proportion of false positives we expect amongst our differentially expressed genes is 5%. For example, if you call 500 genes as differentially expressed with an FDR cutoff of 0.05, you expect 25 of them to be false positives.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I would suggest reading Benjamini and Hochberg (1995) to learn more details if desired.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-08-12T03:50:14.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17551" PostTypeId="2" ParentId="17548" CreationDate="2021-08-12T09:42:06.703" Score="2" Body="&lt;p&gt;So you can install PyMOL as a standalone, but you can also install it as a bona fide Python 3 module via conda:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c schrodinger -c conda-forge pymol-bundle&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It can be installed in other ways —apt-get or brew or even compiled, but the latter is excruciatingly painful.&lt;/p&gt;&#xA;&lt;p&gt;In your python notebook you can do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pymol2&#xA;with pymol2.PyMOL() as pymol:&#xA;    pymol.cmd.load('structure_file.pdb', 'foo') # file&#xA;    # pymol.cmd.fetch('PDB_CODE') # PDB code&#xA;    # pymol.cmd.read_pdbstr(foo_str, 'foo') # string&#xA;    ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where the ellipsis can be any API cmd.&lt;/p&gt;&#xA;&lt;p&gt;This context manager approach is to avoid the problem of namespace pollution as the session (e.g. loading structures without removing the old ones etc.) is the same but you don't see what is happening. Also PyMOL() is parallel the parallel mode (so mutagenesis wizard does not work), while in singleton mode (&lt;code&gt;pymol2.SingletonPyMOL&lt;/code&gt;), you get issues if you try to re-initialise etc.&lt;/p&gt;&#xA;&lt;p&gt;Anyway. To &amp;quot;select&amp;quot; all interacting atoms at the interface you can do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;n = pymol.cmd.select('chain A around 3')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;Expand&lt;/code&gt; includes the selection. &lt;code&gt;Around&lt;/code&gt; does not.&lt;/p&gt;&#xA;&lt;p&gt;This adds a selection to the session. So what we actually want is a pythonic instance to play with. So:&lt;/p&gt;&#xA;&lt;p&gt;atoms = pymol.cmd.get_model('chain A around 3')&lt;/p&gt;&#xA;&lt;p&gt;where atoms is a list of &lt;code&gt;chempy.Atom&lt;/code&gt; objects. With attributes such as resi, resn, chain etc. Do dir on an atom for more...&lt;/p&gt;&#xA;&lt;p&gt;One attribute is &lt;code&gt;coord&lt;/code&gt;, which contains a length 3 list of floats.&lt;/p&gt;&#xA;&lt;p&gt;NB. The above selection gives not a full residue, whereas 'byres chain A around 3' does. So if you want to make a list of all interacting residues:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;interactors = [{'chain': atom.chain, 'index': atom.resi, 'number': atom.resn} for atom in pymol.cmd.get_model('name CA and byres chain A around 3')]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;An iteration can give the atom to atom distances.&lt;/p&gt;&#xA;&lt;p&gt;Do note that this is all proximity, so you will have to screen out hydrogens and carbons to get actual polar contacts.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="650" LastEditDate="2021-08-16T14:32:00.093" LastActivityDate="2021-08-16T14:32:00.093" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17552" PostTypeId="2" ParentId="17543" CreationDate="2021-08-12T14:53:00.147" Score="0" Body="&lt;p&gt;You might use &lt;code&gt;agat_sq_filter_feature_from_fasta.pl&lt;/code&gt; from &lt;a href=&quot;https://github.com/NBISweden/AGAT&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGAT&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; DESCRIPTION &#xA; This script is a kind of annotation filter by sequence name. It goes through the gff annotation features and remove those&#xA; that are not linked to a sequence from the fasta file provided. The&#xA; match between sequence name in the fasta file and the 1st column of&#xA; the gff3 file is case sensitive.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2486" LastActivityDate="2021-08-12T14:53:00.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17553" PostTypeId="2" ParentId="17549" CreationDate="2021-08-12T19:05:30.987" Score="1" Body="&lt;p&gt;More context from the docs, looking at the arguments to &lt;code&gt;decideTests()&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;method: character string specifying how genes and contrasts are to be combined in the&#xA;multiple testing scheme. Choices are &amp;quot;separate&amp;quot;, &amp;quot;global&amp;quot;, &amp;quot;hierarchical&amp;quot;&#xA;or &amp;quot;nestedF&amp;quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;adjust.method: character string specifying p-value adjustment method. Possible values are &amp;quot;none&amp;quot;,&#xA;&amp;quot;BH&amp;quot;, &amp;quot;fdr&amp;quot; (equivalent to &amp;quot;BH&amp;quot;), &amp;quot;BY&amp;quot; and &amp;quot;holm&amp;quot;. See p.adjust for details.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;These are two different options.&lt;/p&gt;&#xA;&lt;p&gt;One option (&lt;code&gt;adjust.method&lt;/code&gt;) chooses the multiple test correction, for which the default is BH/FDR.&lt;/p&gt;&#xA;&lt;p&gt;The other option (&lt;code&gt;method&lt;/code&gt;) decides how the multiple test correction is applied, for example can it be applied separately to different genes or across contrasts. &lt;code&gt;method = global&lt;/code&gt; in this context just means that every single test statistic is considered at the same time. An alternative to this would be to consider a nested design.&lt;/p&gt;&#xA;&lt;p&gt;More intuitively, &lt;code&gt;method = separate&lt;/code&gt; applies the correction to each column of your matrix independently, such that in a &lt;span class=&quot;math-container&quot;&gt;$m x n$&lt;/span&gt; matrix of statistics, the &amp;quot;number of tests&amp;quot; is &lt;span class=&quot;math-container&quot;&gt;$m*n$&lt;/span&gt; for &lt;code&gt;global&lt;/code&gt; and &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; for &lt;code&gt;separate&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For understanding the effect of the number of tests on FDR, I suggest consulting the &lt;a href=&quot;https://en.wikipedia.org/wiki/False_discovery_rate&quot; rel=&quot;nofollow noreferrer&quot;&gt;FDR wiki page&lt;/a&gt;. Basically, using a &lt;code&gt;method&lt;/code&gt; other than &lt;code&gt;global&lt;/code&gt; will increase power.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-08-12T19:05:30.987" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17554" PostTypeId="2" ParentId="17493" CreationDate="2021-08-13T07:39:47.297" Score="2" Body="&lt;p&gt;You are correct that differences in ploidy between autosomes (always ploidy=2) and X chromosome mean that the same VQSR model cannot be applied to both cases equally.&lt;/p&gt;&#xA;&lt;p&gt;However, this is the case only for male X chromosomes (in human species).&lt;/p&gt;&#xA;&lt;p&gt;Therefore, yes, I'd recommend building a separate VQSR model when it comes to male X chromosomes. Importantly, the processing has to be different not only at the VQSR step, but from the very beginning at the HaplotypeCaller step (if you are using GATK also for variant calling), which has to be used with the option ploidy = 1.&lt;/p&gt;&#xA;&lt;p&gt;So,&lt;/p&gt;&#xA;&lt;p&gt;On one hand -&amp;gt; Processing of autosomes (females and males) and x chromosomes (females only) can be done altogether with no differences all the way to VQSR (SNPs + INDELs as in best practices)&lt;/p&gt;&#xA;&lt;p&gt;On the other hand -&amp;gt; Processing of X chromosomes (only males) goes in separate, from variant calling to VQSR (SNPs + INDELs as in best practices)&lt;/p&gt;&#xA;" OwnerUserId="475" LastActivityDate="2021-08-13T07:39:47.297" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17556" PostTypeId="2" ParentId="12820" CreationDate="2021-08-13T13:35:31.273" Score="1" Body="&lt;p&gt;I saw this paper a few weeks back that claims to offer 4x more compression than typical formats for both fastqs and BAM/CRAM files.&lt;/p&gt;&#xA;&lt;p&gt;Genozip - A Universal Extensible Genomic Data Compressor&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://academic.oup.com/bioinformatics/advance-article/doi/10.1093/bioinformatics/btab102/6135077&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://academic.oup.com/bioinformatics/advance-article/doi/10.1093/bioinformatics/btab102/6135077&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2021-08-13T13:35:31.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17557" PostTypeId="2" ParentId="16360" CreationDate="2021-08-13T13:38:35.063" Score="1" Body="&lt;p&gt;If the samples were from different individual organisms you could try looking for mitochondrial variants that differentiate between them. You should have reasonable coverage of the entire mitochondrial mRNA transcriptome even in 10x datasets.&lt;/p&gt;&#xA;&lt;p&gt;This tool might help:&#xA;&lt;a href=&quot;https://bioconductor.org/packages/devel/bioc/html/mitoClone2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/devel/bioc/html/mitoClone2.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2021-08-13T13:38:35.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17558" PostTypeId="1" CreationDate="2021-08-13T16:06:24.357" Score="0" ViewCount="1957" Body="&lt;p&gt;I'm trying to subset my seurat object based on colnames. I have gone ahead and labeled each cluster and now I want to subset all the colnames that are in Cancer_human for human_colnames and all the barcodes that are not in Cancer_human for mouse_colnames but get an error.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;human_colnames = colnames(scData[,scData&lt;span class=&quot;math-container&quot;&gt;$cell_ann == &quot;Cancer_human&quot;])&#xA;mouse_colnames = colnames(scData[,!scData$&lt;/span&gt;cell_ann == &amp;quot;Cancer_human&amp;quot;])&#xA;&#xA;Error in `[.Seurat`(scData, , scData$cell_ann == &amp;quot;Cancer_human&amp;quot;) : Incorrect number of logical values provided to subset cells&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm filtering on colnames not rownames. I want all the barcodes from the Cancer_human.&lt;/p&gt;&#xA;" OwnerUserId="6827" LastEditorUserId="650" LastEditDate="2024-02-28T22:06:51.643" LastActivityDate="2024-03-29T23:00:55.117" Title="Subsetting a Seurat object based on colnames" Tags="|r|seurat|single-cell|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17559" PostTypeId="1" CreationDate="2021-08-14T03:16:07.913" Score="0" ViewCount="156" Body="&lt;p&gt;I have RNA seq data of 12 samples (paired end, so 24 files)  and I ran tophat2 for alignment on all of them which gave me one bam file for all the samples. Now I need separate bam files for all the samples. Can I spilt them from the one I have or I have to do tophat2 for all the samples separately? If yes, how to do it.&lt;/p&gt;&#xA;&lt;p&gt;I need them for FRASER (splicing detction) I would also appreciate if anyone used FRASER before could help me a bit to get started. I am working with plant data for the first time.&lt;/p&gt;&#xA;&lt;p&gt;The command I used&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tophat2 -p 40 \&#xA;    -o tophat_output_final bowtie_indexes/Medicago1/Medicago \&#xA;    NRNSA_S103_R1_001.fastq.gz,NRNSB_S107_R1_001.fastq.gz,\&#xA;      NRNSC_S111_R1_001.fastq.gz,NRSA_S104_R1_001.fastq.gz,\&#xA;      NRSB_S108_R1_001.fastq.gz,NRSC_S112_R1_001.fastq.gz,\&#xA;      RNSA_S105_R1_001.fastq.gz,RNSB_S109_R1_001.fastq.gz,\&#xA;      RNSC_S113_R1_001.fastq.gz,SRAB_S106_R1_001.fastq.gz,\&#xA;      SRB_S110_R1_001.fastq.gz,SRC_S114_R1_001.fastq.gz \&#xA;    NRNSA_S103_R2_001.fastq.gz,NRNSB_S107_R2_001.fastq.gz,\&#xA;      NRNSC_S111_R2_001.fastq.gz,NRSA_S104_R2_001.fastq.gz,\&#xA;      NRSB_S108_R2_001.fastq.gz,NRSC_S112_R2_001.fastq.gz,\&#xA;      RNSA_S105_R2_001.fastq.gz,RNSB_S109_R2_001.fastq.gz,\&#xA;      RNSC_S113_R2_001.fastq.gz,SRAB_S106_R2_001.fastq.gz,\&#xA;      SRB_S110_R2_001.fastq.gz,SRC_S114_R2_001.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="13985" LastEditorUserId="3694" LastEditDate="2021-08-17T13:21:02.163" LastActivityDate="2021-08-17T13:21:02.163" Title="Splitting .bam files into separate samples for tophat2" Tags="|rna-seq|sam|multiple-sequence-alignment|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17560" PostTypeId="1" CreationDate="2021-08-16T14:22:35.273" Score="1" ViewCount="68" Body="&lt;p&gt;I am planning some experiments to investigate how mono cultures of microbes respond to different nutrient concentrations. I want to use an -omics based approach to understand how, on a molecular level, the microbes utilise these nutrients by looking for up regulation of associated genes. With the idea looking out for upregulation in mind, I was going to go for transcriptomics. But now I am beginning to realise that regulatory changes in genes does not necessarily translate to the molecular utilisation of a nutrient by a cell. To that end, perhaps I would be better using quantitative proteomics to capture changes in the concentration of functional proteins which are actually interacting with the nutrients. However, I worry that by relying on proteomics I might miss important regulatory changes for microbe species where the proteome is less well defined.&lt;/p&gt;&#xA;&lt;p&gt;I should say that it's an either/or situation, there is only budget for one.&#xA;I'd appreciate any thoughts on the best approach.&#xA;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="13996" LastActivityDate="2021-08-16T14:22:35.273" Title="Transcriptomics vs Proteomics for understanding molecular responses" Tags="|transcriptome|proteomics|microbiology|" AnswerCount="0" CommentCount="6" ClosedDate="2023-04-13T03:56:02.113" ContentLicense="CC BY-SA 4.0" />
  <row Id="17561" PostTypeId="2" ParentId="5292" CreationDate="2021-08-17T09:22:19.867" Score="-1" Body="&lt;p&gt;You can use following command from the parent folder to run fastqc on &lt;code&gt;fastq.gz&lt;/code&gt; files in subdirectories&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;find -name '*.gz' | xargs fastqc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="5044" LastEditorUserId="5044" LastEditDate="2021-08-20T14:40:31.417" LastActivityDate="2021-08-20T14:40:31.417" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17562" PostTypeId="1" CreationDate="2021-08-17T13:17:53.720" Score="1" ViewCount="194" Body="&lt;p&gt;So, I am using cutadapt to remove two primers from some paired end reads.&lt;/p&gt;&#xA;&lt;p&gt;I have 2 files for each sample, i.e. sample_R1.fastq and sample_R2.fastq.&lt;/p&gt;&#xA;&lt;p&gt;I have &lt;strong&gt;primer_1&lt;/strong&gt; and &lt;strong&gt;primer_2&lt;/strong&gt; and both of them can be found in &lt;strong&gt;sample_R1.fastq&lt;/strong&gt; and &lt;strong&gt;sample_R2.fastq&lt;/strong&gt;. In other words, if I search for &lt;strong&gt;primer_1&lt;/strong&gt; and &lt;strong&gt;primer_2&lt;/strong&gt; in &lt;strong&gt;sample_R1.fastq&lt;/strong&gt;, the sum of the matches is the same as the number of the reads in &lt;strong&gt;sample_R1.fastq&lt;/strong&gt;. Same for R2.&lt;/p&gt;&#xA;&lt;p&gt;To properly trim them what I do is the following, through R:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;setOne &amp;lt;- paste(&amp;quot;-g&amp;quot;, primer_1, &amp;quot;-a&amp;quot;, primer_1)&#xA;setTwo &amp;lt;- paste(&amp;quot;-G&amp;quot;, primer_2, &amp;quot;-A&amp;quot;, primer_2)&#xA;system2(cutadapt, args = c(setOne, setTwo, &amp;quot;-n&amp;quot;, 2, &#xA;                           &amp;quot;-o&amp;quot;, outOne, &amp;quot;-p&amp;quot;, outTwo,&#xA;                            inputOne, inputTwo, &amp;quot;--minimum-length&amp;quot;, 1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The command works fine except for one thing. On the one hand &lt;code&gt;cutadapt&lt;/code&gt; should remove those sequences which, after the trimming, will have a &lt;code&gt;length &amp;lt; 1&lt;/code&gt;, as specified by the command. However, some reads are removed even though they are longer than 1 and I would like to know why this happens, since I can't find a specific reference mentioning that &lt;code&gt;cutadapt&lt;/code&gt; removes sequences.&lt;/p&gt;&#xA;&lt;p&gt;If I am getting this straight, &lt;code&gt;cutadapt&lt;/code&gt; only removes the primer sequence from the beginning of the read, then removes the equivalent characters in the quality field of the read that was trimmed and, finally, it returns the trimmed read with the quality field adjusted to the new length. If the length is lower than the threshold, the read is discarded. If it is discarded in R1, it will be discarded in R2 and viceversa.&lt;/p&gt;&#xA;&lt;p&gt;I give an example of read that is removed, this one from R1:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@some:sample:number 1:N:0:1&#xA;ACCGTCACGTCATCAGCAGCATCGACTCCTGTTTGATCCCCGTGCCTGTACTCAAGTCAACCAGTACACGCTTTTCAAGCTGCCTTCTGCCGTATTTCACACTCAGCGCATTTGCAATTGGTGTTCTTTGCACCACATATCTATCAGTTAAGCCACCGCTACTCCGCCTACCTCGTACTGACTTAATCGACCGCCTACGCACCCTTTAAACCCAATTCGAGACCGTCAATTCAATCATACTTTAGTAAGTG&#xA;+&#xA;BBBBBBBBBBBBGGGGHEGHFHA@EGGGHHHHHHHHHGHHHHHHHHGGGGGHGHGHGGGHHHHHGDGGGFHGGGGHGGGGGGHGHGFHHHHHHHHHHGGGHHHGHGHHHHHHHGFFGGGGGGGGGGGGGGGGGFFBDFDFFFFFFFBFFFFFFFFFHGHGGHHHHHEGGGGGHHHHGHHHHHGHHHHHHHHHHHHGGGHGHHGGGGEFGGHHHHHHHHHHHHHGHHHHHHHHHHHFHFGHHHHGGGGGHHG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;this one from R2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@some:sample:number 2:N:0:1&#xA;CTGCTGCTATCACTAGCTAAAGCTTTAACGTAGGGCAATTGACGAAGGTGAAATGCATAGATATGTGATGACGGTGTAGCGTTGGCGCTGGGGTTTAAAGGGTGCGTAGTGACTTCACAGAACACCAATTGGCAGCTTACTAAAGCATGAGTACAGACGAGGTAGGCGGAATTGCGGTCGATTAAGTCAGTTGTGAAATACGGCAGCTTAACTGTTGGCAAGCGTTGTCCGGCACTGGTTTTATTCGAGGT&#xA;+&#xA;AAAAA@BFFFFFGFGCGGFDDC/CDHF2FGFHHG//&amp;lt;&amp;lt;CCC1FGHH1&amp;lt;1&amp;gt;11C&amp;lt;FF/0BFFFFFFF/:DEEEFFHGG0AEG/B9B./-.@9GHGEFC&amp;gt;EE&amp;lt;FHEHEGHDHGGFHGGBGGGHGGDGH0CDDGFFHHCC--.:CGCFG??GGAEGGHFHHBFFFEFB./B..;/;:/999=AAA../BB/;//////9/EFFCBDAA..990ADD=99/;/BFGGGGGGGGGFHHHHHHGHDGHHFFDHHHHG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and these are the primers that I am removing:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ACCGTCACGTCATCAGCAGCATCGAC&lt;/li&gt;&#xA;&lt;li&gt;CTGCTGCTATCACTAGCTAAAGCTTTA&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;matching the reads I provided. Clearly, if I trim them manually, the reads length is way greater than 1 so, why are these reads removed from the final &lt;code&gt;cutadapt&lt;/code&gt; output file?&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&lt;p&gt;I might have solved the issue, but I asked on the &lt;code&gt;cutadapt&lt;/code&gt; &lt;a href=&quot;https://github.com/marcelm/cutadapt/issues/556&quot; rel=&quot;nofollow noreferrer&quot;&gt;github issues page&lt;/a&gt;, just to make sure.&lt;/p&gt;&#xA;" OwnerUserId="3119" LastEditorUserId="3119" LastEditDate="2021-08-18T07:07:34.640" LastActivityDate="2021-08-18T07:07:34.640" Title="Why is cutadapt discarding some paired end reads?" Tags="|trimming|cutadapt|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17565" PostTypeId="1" CreationDate="2021-08-18T09:45:07.793" Score="1" ViewCount="684" Body="&lt;p&gt;Hi I am trying to convert &lt;code&gt;bam&lt;/code&gt; files generated from Ion Torrent Proton sequencing to &lt;code&gt;fastq&lt;/code&gt; format so that I can upload them to KBase for analysis.&lt;/p&gt;&#xA;&lt;p&gt;The files are named as follows:&#xA;&lt;code&gt;01_thu159_IX_RNA_001.bam 01_thu163_IX_RNA_001.bam 01_thu164_IX_RNA_001.bam 01_thu190_IX_RNA_001.bam&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;So I tried:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;samtools fastq *.bam &amp;gt; *.fastq&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, that did not work. But, I know it is possible to use a placeholder for the name in code by using something like &lt;code&gt;$f&lt;/code&gt; or I have also seen &lt;code&gt;%N%&lt;/code&gt;. I am just not sure how to basically use those placeholders to be able to quickly apply a &lt;code&gt;for&lt;/code&gt; loop, which will convert the files to fastq format without having to do each one individually. I have 28 samples each containing 4 &lt;code&gt;bam&lt;/code&gt; files. Any advice on how I can do this would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="13885" LastActivityDate="2021-08-18T13:05:25.787" Title="How to convert multiple single-end bam files to fastq using samtools" Tags="|sam|samtools|fastq|format-conversion|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17567" PostTypeId="2" ParentId="17565" CreationDate="2021-08-18T13:05:25.787" Score="2" Body="&lt;p&gt;Using GNU parallel:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ls *.bam | parallel -j 2 &amp;quot;samtools fastq {} &amp;gt; {.}.fastq&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Use &lt;code&gt;-j&lt;/code&gt; to control the number of parallel jobs. That only works for single-end data though as paired-end data must first be sorted or collated by name.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-08-18T13:05:25.787" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17568" PostTypeId="1" AcceptedAnswerId="17596" CreationDate="2021-08-18T13:08:55.080" Score="1" ViewCount="278" Body="&lt;p&gt;I am looking for any public available databases where I can download Sanger &lt;a href=&quot;http://www.dnabaser.com/help/samples/what%20is%20a%20chromatogram.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;chromatography&lt;/a&gt; data ideally in &lt;a href=&quot;https://www.reviversoft.com/en/file-extensions/ab1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ab1&lt;/a&gt; or &lt;a href=&quot;http://staden.sourceforge.net/manual/formats_unix_2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SCF&lt;/a&gt; file format. I need extremely large amounts of this, since I want to use it for machine learning.&lt;/p&gt;&#xA;" OwnerUserId="12126" LastEditorUserId="3694" LastEditDate="2021-08-19T09:51:38.463" LastActivityDate="2021-08-25T14:42:21.503" Title="Where to find publicly available Sanger chromatography data?" Tags="|ngs|sequence-analysis|public-databases|data-download|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17569" PostTypeId="1" CreationDate="2021-08-18T14:06:18.617" Score="0" ViewCount="89" Body="&lt;p&gt;I am creating a bioinformatics pipeline using Snakemake doing everying from adapter trimming of raw sequencing data in .fq to mapping, SNP calling and SNP filtering.&lt;/p&gt;&#xA;&lt;p&gt;The issue i am having is when i try to filter specific chromosomes of the bcf files, using the --regions. The rule i have created below works and does create a filtered bcf file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule bcftools_filter:&#xA;    input:&#xA;        bcf = &amp;quot;/dir/{sample}_aligned_call.bcf&amp;quot;,&#xA;    output:&#xA;        &amp;quot;/dir/{sample}_aligned_call_filter.bcf&amp;quot;&#xA;    threads: 20&#xA;    shell:&#xA;        &amp;quot;bcftools view -Ob --threads {threads} -m2 -M2 -v snps {input.bcf} -o {output}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However i would like to add the --regions parameter to the bcftools command. I have tried the shell part of the rule to several things&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view -Ob --threads {threads} -m2 -M2 -v snps {input.bcf} --regions &amp;lt;chr1,chr2&amp;gt; -o {output}&#xA;bcftools view -Ob --threads {threads} -m2 -M2 -v snps {input.bcf} --regions {chr1,chr2} -o {output}&#xA;bcftools view -Ob --threads {threads} -m2 -M2 -v snps {input.bcf} --regions {{chr1,chr2}} -o {output}&#xA;bcftools view -Ob --threads {threads} -m2 -M2 -v snps {input.bcf} --regions {{chr1} {chr2}} &#xA; {{chr2}} -o {output}&#xA;bcftools view -Ob --threads {threads} -m2 -M2 -v snps {input.bcf} --regions ['chr1','chr2'] -o {output}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But it doesn't work, and i am having trouble finding any suggestions to this specific issue.&lt;/p&gt;&#xA;&lt;p&gt;I know it works if i just run bcftools filter in my command line, so using --regions should work.&lt;/p&gt;&#xA;&lt;p&gt;Could anyone have any suggestions how to add --regions to the snakemake rule?&lt;/p&gt;&#xA;" OwnerUserId="8630" LastActivityDate="2022-06-07T05:00:05.050" Title="Snakemake rule issue with bcftools view filtering" Tags="|snp|snakemake|bcftools|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17570" PostTypeId="1" AcceptedAnswerId="17584" CreationDate="2021-08-18T17:19:13.500" Score="0" ViewCount="77" Body="&lt;ul&gt;&#xA;&lt;li&gt;Given a multi-fasta multiple sequence alignment file, is there a quick way/tool to calculate average pairwise sequence identity?&lt;/li&gt;&#xA;&lt;li&gt;Even better, given a whole genome alignment file (&lt;code&gt;.maf&lt;/code&gt; or even &lt;code&gt;.hal&lt;/code&gt;) is there a quick way/tool to calculate average pairwise sequence identity across the individual local multiple sequence alignments?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6628" LastActivityDate="2021-08-23T08:31:59.077" Title="Alignment statistics" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17571" PostTypeId="2" ParentId="17569" CreationDate="2021-08-18T19:06:59.737" Score="1" Body="&lt;p&gt;Something like the following would work:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule bcftools_filter:&#xA;    input:&#xA;        bcf = &amp;quot;/dir/{sample}_aligned_call.bcf&amp;quot;,&#xA;    output:&#xA;        &amp;quot;/dir/{sample}_aligned_call_filter.bcf&amp;quot;&#xA;    params:&#xA;        regions=&amp;quot;chr1,chr2&amp;quot;&#xA;    threads: 20&#xA;    shell:&#xA;        &amp;quot;bcftools view -Ob --threads {threads} -m2 -M2 --regions {params.regions} -v snps {input.bcf} -o {output}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You could also include the regions as part of the output file name, rather than specifying it as a param.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-08-18T19:06:59.737" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17572" PostTypeId="1" AcceptedAnswerId="17575" CreationDate="2021-08-19T11:37:21.657" Score="-1" ViewCount="216" Body="&lt;p&gt;What does removing a batch effect (e.g. with &lt;code&gt;limma::removeBatchEffect&lt;/code&gt;) assume about the batch effect?&lt;/p&gt;&#xA;&lt;p&gt;Does it assume simply a constant batch effect for each level?&#xA;E.g. if the levels are&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Date1&#xA;Date2&#xA;Date3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then one constant will be subtracted from all samples on Date3, another constant from all samples of Date2 and so on?&lt;/p&gt;&#xA;&lt;p&gt;Or is it more complicated?&lt;/p&gt;&#xA;" OwnerUserId="6317" LastActivityDate="2021-08-19T21:59:55.880" Title="Assumptions of batch effect removal" Tags="|limma|batch-effects|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17573" PostTypeId="1" CreationDate="2021-08-19T12:24:09.863" Score="-1" ViewCount="126" Body="&lt;p&gt;I'm trying to find the rat genome to carry out STAR alignment for several FASTQ files.  Where would I find these files?&lt;/p&gt;&#xA;" OwnerUserId="14003" LastActivityDate="2021-08-19T13:36:22.327" Title="Where to find rat genome for STAR alignment?" Tags="|sequence-alignment|genome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17574" PostTypeId="2" ParentId="17573" CreationDate="2021-08-19T13:36:22.327" Score="2" Body="&lt;p&gt;For example from Ensembl: &lt;a href=&quot;https://www.ensembl.org/Rattus_norvegicus/Info/Index&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ensembl.org/Rattus_norvegicus/Info/Index&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can find other links via a search engine of choice.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-08-19T13:36:22.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17575" PostTypeId="2" ParentId="17572" CreationDate="2021-08-19T21:59:55.880" Score="1" Body="&lt;p&gt;You're basically subtracting a constant per-gene per-level. The relevant portion of the code is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fit &amp;lt;- lmFit(x,cbind(design,X.batch),...)&#xA;beta &amp;lt;- fit$coefficients[,-(1:ncol(design)),drop=FALSE]&#xA;beta[is.na(beta)] &amp;lt;- 0&#xA;as.matrix(x) - beta %*% t(X.batch)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where &lt;code&gt;x&lt;/code&gt; is your input matrix, &lt;code&gt;design&lt;/code&gt; is the design matrix and &lt;code&gt;X.batch&lt;/code&gt; is the matrix of batch and covariates. So you're fitting your data with the combined batch/design matrix (line 1), extracting out the batch coefficients (line 2), and subtracting out the expected effect (line 4). This is simple enough and remarkably effective for its intended use. You can get a bit fancier by using PCA or something else to instead allow individual samples to be more/less affected by a batch-effect, but added complexity has its own issues.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-08-19T21:59:55.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17576" PostTypeId="1" CreationDate="2021-08-20T13:08:47.533" Score="0" ViewCount="23" Body="&lt;p&gt;I'm an undergraduate biology student and my thesis is on designing a deep learning architecture to predict whether two proteins interact or not given their primary sequences. I have read some papers with different approaches to the problem including the paper [https://academic.oup.com/bioinformatics/article/35/14/i305/5529260] uses &lt;strong&gt;PIPR&lt;/strong&gt; model which is the state of the art model for this problem in 2019 and recently in 2021 the paper [https://www.biorxiv.org/content/10.1101/2021.01.22.427866v1] uses &lt;strong&gt;D-SCRIPT&lt;/strong&gt; which surpass &lt;strong&gt;PIPR&lt;/strong&gt; with the aspect of the small dataset and has better generalization feature for many model organisms.&lt;/p&gt;&#xA;&lt;p&gt;For my problem, I have two questions&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;can you suggest to me the approach to build a model for my thesis. Should I follow these papers to build a model, how my model will be different from the models in the papers?&lt;/li&gt;&#xA;&lt;li&gt;Which architecture or model recently that the literature considers the most proper for the protein-protein interaction prediction approach?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I very much appreciate any help from everyone!&lt;/p&gt;&#xA;" OwnerUserId="14006" LastActivityDate="2021-08-20T13:08:47.533" Title="Build protein-protein interaction prediction deep learning model" Tags="|machine-learning|protein-protein-interaction|deep-learning|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17577" PostTypeId="1" CreationDate="2021-08-20T21:38:01.783" Score="1" ViewCount="282" Body="&lt;p&gt;Let's assume I have a BAM file and several positions that I would like to examine more closely in this alignment. My goal is to find out whether these positions are on the same reads and which nucleotides each read has in these two positions.&#xA;So my output should be a table of all reads that occur in the positions and list the associated nucleotides.&lt;/p&gt;&#xA;&lt;p&gt;The output should look like this (with 980, 1350 and 1400 the first three positions; I made up the read names):&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;read name&lt;/th&gt;&#xA;&lt;th&gt;980&lt;/th&gt;&#xA;&lt;th&gt;1350&lt;/th&gt;&#xA;&lt;th&gt;1400&lt;/th&gt;&#xA;&lt;th&gt;...&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;ReAdN4m1L03&lt;/td&gt;&#xA;&lt;td&gt;A&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;td&gt;...&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;ReAdN4m2L02&lt;/td&gt;&#xA;&lt;td&gt;A&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;G&lt;/td&gt;&#xA;&lt;td&gt;...&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;ReAdN4m3L01&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;G&lt;/td&gt;&#xA;&lt;td&gt;...&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;...&lt;/td&gt;&#xA;&lt;td&gt;...&lt;/td&gt;&#xA;&lt;td&gt;...&lt;/td&gt;&#xA;&lt;td&gt;...&lt;/td&gt;&#xA;&lt;td&gt;...&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;At the end I would like to have a table that links the nucleotides via the reads.&lt;/p&gt;&#xA;&lt;p&gt;So far, I have tried to read in the Bam file at the desired positions:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#Positions of interest&#xA;pos &amp;lt;- c(980, 1350, 1400)&#xA;&#xA;#path to BAM file&#xA;path &amp;lt;- pathToBamFile&#xA;&#xA;#name of reference used for mapping&#xA;refname &amp;lt;- &amp;quot;myreference&amp;quot;&#xA;&#xA;#loading the bam file&#xA;bamFile &amp;lt;- BamFile(path)&#xA;gr &amp;lt;- GRanges(seqnames = refname, ranges = IRanges(start = pos, end = pos))&#xA;params &amp;lt;- ScanBamParam(which = gr, what = scanBamWhat())&#xA;aln &amp;lt;- scanBam(bamFile, param = params)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With the following code I can access the reads, but how do I get the corresponding nucleotides in the individual positions?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pos.no &amp;lt;- c(1:length(pos))&#xA;&#xA;i &amp;lt;- 1&#xA;aln[[pos.no[i]]]$qname&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have already tried to work with the CIGAR strings &lt;code&gt;aln[[i]]$cigar&lt;/code&gt; with the sequences &lt;code&gt;aln[[i]]$seq&lt;/code&gt; in order to extract to the nucleotides manually, but that led to more and more problems.&lt;/p&gt;&#xA;&lt;p&gt;Is there any easy way to extract the nucleotides?&lt;/p&gt;&#xA;&lt;p&gt;I have searched for a long time now on the internet for an easy solution but without success. Hopefully someone will help me here! Many thanks for your time in advance.&lt;/p&gt;&#xA;" OwnerUserId="14008" LastActivityDate="2021-08-20T21:38:01.783" Title="Extract read names and the associated nucleotides on specific positions from a BAM file (in R)" Tags="|r|sam|snp|reads|cigar|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17578" PostTypeId="2" ParentId="6871" CreationDate="2021-08-20T21:46:43.747" Score="4" Body="&lt;p&gt;By far the most accurate model right now should be AlphaFold, and there are open structures across several proteomes available at &lt;a href=&quot;https://alphafold.ebi.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://alphafold.ebi.ac.uk/&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="14009" LastActivityDate="2021-08-20T21:46:43.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17579" PostTypeId="1" CreationDate="2021-08-21T13:55:22.753" Score="0" ViewCount="131" Body="&lt;p&gt;I am attempting to edit my &lt;code&gt;fastq&lt;/code&gt; sequence files with an &lt;code&gt;awk&lt;/code&gt; command from a published article. The sequence files are all contained in one directory where I am running the &lt;code&gt;awk&lt;/code&gt; command from the terminal in a Linux environment. There are 4 fastq files in the directory&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;01_thu159_IX_RNA_001.fastq&#xA;01_thu163_IX_RNA_001.fastq&#xA;01_thu164_IX_RNA_001.fastq&#xA;01_thu190_IX_RNA_001.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to trim all these files at once with the &lt;code&gt;awk&lt;/code&gt; command so I wrote this &lt;code&gt;for&lt;/code&gt; loop:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for filename in ./*.fastq*&#xA;do&#xA;awk -v s=10 -v e= 40 ' {if (NR%2 == 0) print substr (&lt;span class=&quot;math-container&quot;&gt;$0, s+1, length ($&lt;/span&gt;0)-s-e); else print &lt;span class=&quot;math-container&quot;&gt;$0;} ' ./$&lt;/span&gt;{filename} &amp;gt;  ./${filename}_trimmed&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, I am getting the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk: fatal: cannot open file ` {if (NR%2 == 0) print substr (&lt;span class=&quot;math-container&quot;&gt;$0, s+1, length ($&lt;/span&gt;0)-s-e); else print $0;} ' for reading (No such file or directory)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then thought maybe it was an error with the &lt;code&gt;for&lt;/code&gt; loop so I tried the command as per the instructions in the article&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk -v s=10 -v e= 40 '{if (NR%2 == 0) print substr (&lt;span class=&quot;math-container&quot;&gt;$0, s+1, length ($&lt;/span&gt;0)-s-e); else print $0;}' ./01_thu159_IX_RNA_001.fastq &amp;gt; ./Trim_01_thu159_IX_RNA_001.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But this also produced the same error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk: fatal: cannot open file `{if (NR%2 == 0) print substr (&lt;span class=&quot;math-container&quot;&gt;$0, s+1, length ($&lt;/span&gt;0)-s-e); else print $0;}' for reading (No such file or directory)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have checked the syntax for the &lt;code&gt;awk&lt;/code&gt; command and it should be correct. Could anyone please give me advice as to what I am doing wrong and how to fix this issue?&lt;/p&gt;&#xA;" OwnerUserId="13885" LastEditorUserId="650" LastEditDate="2021-08-21T22:28:47.703" LastActivityDate="2021-08-21T22:28:47.703" Title="Error when using awk command to trim sequence files" Tags="|rna-seq|fastq|linux|awk|read-correction|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17580" PostTypeId="2" ParentId="17579" CreationDate="2021-08-21T15:36:49.883" Score="3" Body="&lt;p&gt;You have a whitespace here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;e= 40&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;awk thinks that 40 is your rule and the rest of the line is a filename.&lt;/p&gt;&#xA;" OwnerUserId="14015" LastActivityDate="2021-08-21T15:36:49.883" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17581" PostTypeId="1" AcceptedAnswerId="17636" CreationDate="2021-08-22T04:33:53.883" Score="0" ViewCount="75" Body="&lt;p&gt;Suppose, the following is a diagram of a protein's polypeptide chain:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HWIPy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HWIPy.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What does this diagram represent?&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What are the letters A, E, M, W, L, N, S, etc. represent?  (I suppose these are amino acid codes. Right?)&lt;/li&gt;&#xA;&lt;li&gt;Which one is the c-alpha atom here?&lt;/li&gt;&#xA;&lt;li&gt;What are the sequence numbers &lt;code&gt;i-2&lt;/code&gt;, &lt;code&gt;i-1&lt;/code&gt;, &lt;code&gt;i&lt;/code&gt;, &lt;code&gt;i+1&lt;/code&gt;, &lt;code&gt;i+2&lt;/code&gt;, etc. represent? Why there are negative and positive increments from &lt;code&gt;i&lt;/code&gt;?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The following is a related diagram:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QthLx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QthLx.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12686" LastActivityDate="2021-08-31T12:57:42.630" Title="What does the following diagram represent?" Tags="|proteins|protein-structure|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17582" PostTypeId="2" ParentId="15290" CreationDate="2021-08-22T17:49:59.417" Score="2" Body="&lt;p&gt;This is late but I am putting it here for future reference. You can use the &lt;code&gt;vc_nonsyn&lt;/code&gt; argument in the &lt;code&gt;read.maf&lt;/code&gt; function to manually list the variant classifications you want to plot later. It will look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df.maf &amp;lt;- read.maf(maf = df, vc_nonSyn = c(&amp;quot;Frame_Shift_Del&amp;quot;,&amp;quot;Missense_Mutation&amp;quot;,&amp;quot;Nonstop_Mutation&amp;quot;,&amp;quot;Silent&amp;quot;,&amp;quot;Splice_Region&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14020" LastEditorUserId="57" LastEditDate="2021-08-25T22:03:57.983" LastActivityDate="2021-08-25T22:03:57.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17584" PostTypeId="2" ParentId="17570" CreationDate="2021-08-23T08:31:59.077" Score="2" Body="&lt;p&gt;There are tons of methods that do something similar to this. Is it necessary that it be &lt;em&gt;alignment&lt;/em&gt; rather than a heuristic such as k-mer hashing?&lt;/p&gt;&#xA;&lt;p&gt;If k-mer hashing is ok, then I would suggest &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-0997-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;mash&lt;/a&gt; or &lt;a href=&quot;https://sourmash.readthedocs.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;sourmash&lt;/a&gt;. There are additionally newer &lt;a href=&quot;https://academic.oup.com/nargab/article/3/1/lqab001/6125549&quot; rel=&quot;nofollow noreferrer&quot;&gt;fancy techniques&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For just an old-school program that computes pairwise identities, &lt;a href=&quot;https://evolution.genetics.washington.edu/phylip.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PHYLIP&lt;/a&gt; has programs DNADIST and PROTDIST that exactly compute identities. &lt;a href=&quot;https://www.biostars.org/p/301422/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Similar questions on biostars&lt;/a&gt; suggest that PHYLIP's utilities are recommended even relatively recently elsewhere. But it looks like &lt;a href=&quot;https://www.ebi.ac.uk/Tools/msa/clustalo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clustal Omega&lt;/a&gt; also computes identities for free.&lt;/p&gt;&#xA;&lt;p&gt;For microbial genomes a common statistic is &lt;a href=&quot;https://www.sciencedirect.com/science/article/abs/pii/S0580951714000087&quot; rel=&quot;nofollow noreferrer&quot;&gt;average nucleotide identity&lt;/a&gt; (ANI), which may be what you are looking for. There are a variety of tools to do this, here is &lt;a href=&quot;https://www.nature.com/articles/s41467-018-07641-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;one&lt;/a&gt;, and &lt;a href=&quot;http://enve-omics.ce.gatech.edu/ani/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here is a webserver&lt;/a&gt; that will do it for you.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-08-23T08:31:59.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17585" PostTypeId="1" CreationDate="2021-08-23T09:09:51.163" Score="0" ViewCount="96" Body="&lt;p&gt;So, I am analysing 18S amplicon data (fastq files) to be able to eventually investigate the taxa composition.&lt;/p&gt;&#xA;&lt;p&gt;After removing primers (using &lt;code&gt;cutadapt&lt;/code&gt;) from both R1 and R2 (assuming primers are found at the beginning of my sequences), I am still able to detect, apparently at random, a few reads that contain my primers (i.e. a part of the sequence matching the primer) in, again apparently random, positions.&lt;/p&gt;&#xA;&lt;p&gt;I am able to detect almost all my primers (16 out of 24, both forward and reverse) in at least one sequence and I am wondering how common (or how weird) this is. I guess it is important to consider that my primers contains a few, i.e. from 1 to 3, wild-card bases that may match multiple other bases (apparently called &lt;a href=&quot;https://www.biosearchtech.com/support/faqs/custom-oligonucleotides-modifications/what-are-wobbles&quot; rel=&quot;nofollow noreferrer&quot;&gt;whobble bases&lt;/a&gt;), which makes it easier to match a random sequence (there are multiple possibilities for an actual match).&lt;/p&gt;&#xA;&lt;p&gt;Problem is that the presence of primers in the sequences may affect downstream analyses (e.g. with DADA2 &lt;a href=&quot;https://benjjneb.github.io/dada2/tutorial_1_8.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;they are very clear&lt;/a&gt; about the fact that primers have to be removed) in several ways and I was wondering if these sequences I am finding may be interfering somehow of if I can live with it.&lt;/p&gt;&#xA;" OwnerUserId="3119" LastEditorUserId="3119" LastEditDate="2021-08-23T09:25:25.273" LastActivityDate="2021-08-23T09:25:25.273" Title="How likely is it to find primers sequences in already-trimmed reads?" Tags="|sequence-alignment|sequence-analysis|phylogenetics|fastq|primer|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17587" PostTypeId="2" ParentId="14739" CreationDate="2021-08-23T10:51:33.643" Score="-1" Body="&lt;p&gt;show sphere, A/1/C&lt;/p&gt;&#xA;&lt;p&gt;set sphere_color, red, A/1/C&lt;/p&gt;&#xA;" OwnerUserId="14024" LastActivityDate="2021-08-23T10:51:33.643" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17588" PostTypeId="1" AcceptedAnswerId="17590" CreationDate="2021-08-23T16:34:12.333" Score="1" ViewCount="27" Body="&lt;p&gt;I want to encode two different DNAs using pySBOL3. First dna is an engineered region usually composed by a transcriptional unit (TU) or a set of TUs. The second is vector which is a basically a plasmid that contains a set of the previously described dna. This is for differentiate between engineered on different plasmids.&lt;/p&gt;&#xA;&lt;p&gt;Now for dna I used as type: &lt;code&gt;sbol3.SBO_DNA&lt;/code&gt; and role: &lt;code&gt;sbol3.SO_ENGINEERED_REGION&lt;/code&gt; and for vector I used as type: &lt;code&gt;sbol3.SBO_DNA&lt;/code&gt; and role: &lt;code&gt;SO:0000155&lt;/code&gt; (plasmid).&lt;/p&gt;&#xA;&lt;p&gt;I would like to have feedback on this choice. For example the plasmid could also be an engineered plasmid &lt;code&gt;SO:0000637&lt;/code&gt; or circular plasmid &lt;code&gt;SO:0002211&lt;/code&gt; or just circular &lt;code&gt;SO:0000988&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here is my actual implementation:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for dna_id in params.get('dna'):&#xA;    #this needs to be handlded by vector&#xA;    dnas = Dna.object().filter(dna_id)&#xA;    dna = sbol3.Component(dnas.name(), sbol3.SBO_DNA)&#xA;    dna.roles = sbol3.SO_ENGINEERED_REGION&#xA;    dna.name = dnas.name&#xA;    dna.description = dnas.description&#xA;    if dnas == None:&#xA;        print('No dnas specified')&#xA;&#xA;for vector_id in params.get('vector'):&#xA;    vectors =Vectors.object().filter(vector_id)&#xA;    vector = sbol3.Component(vectors.name(), sbol3.SBO_DNA)&#xA;    vector.roles = SO:0000155&#xA;    vector.name =vectors.name&#xA;    vector.description = vectors.description()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14025" LastEditorUserId="650" LastEditDate="2021-08-23T16:38:38.527" LastActivityDate="2021-08-23T17:31:48.820" Title="Encoding different DNAs in pySBOL3?" Tags="|phylogenetics|sbol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17589" PostTypeId="2" ParentId="15938" CreationDate="2021-08-23T16:57:24.333" Score="1" Body="&lt;p&gt;It seems you need to supply vcf files specifying the ploidy of the sex chromosomes.&lt;/p&gt;&#xA;&lt;p&gt;See &lt;a href=&quot;https://github.com/Illumina/canvas/issues/89&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Illumina/canvas/issues/89&lt;/a&gt; for explanation and example vcf files.&lt;/p&gt;&#xA;" OwnerUserId="14026" LastActivityDate="2021-08-23T16:57:24.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17590" PostTypeId="2" ParentId="17588" CreationDate="2021-08-23T17:31:48.820" Score="1" Body="&lt;p&gt;Your choices match the same choices that I've made when designated types for inserts and vectors.&lt;/p&gt;&#xA;&lt;p&gt;The more specific plasmid types are reasonable to use as well, but choosing just &amp;quot;plasmid&amp;quot; is certainly safe. That's why I have preferred to use that in my code because (1) my code might sometimes be applied to a different sort of plasmid, (2) I don't currently have a purpose for making the finer distinction, and (3) circular DNA would be indicated by SO.circular (SO:0000988) in any case.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Side note: I would suggest using the &lt;a href=&quot;https://github.com/SynBioDex/tyto&quot; rel=&quot;nofollow noreferrer&quot;&gt;tyto&lt;/a&gt; package so that your code can use names like &lt;code&gt;SO.plasmid&lt;/code&gt; in your code instead of opaque symbols like &lt;code&gt;SO:0000155&lt;/code&gt;.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="4869" LastActivityDate="2021-08-23T17:31:48.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17591" PostTypeId="1" CreationDate="2021-08-23T20:55:25.600" Score="0" ViewCount="206" Body="&lt;p&gt;I have microRNA (miRNA) expression data in RPM. I would like to do differential gene expression between two groups.&lt;/p&gt;&#xA;&lt;p&gt;How can I do this?&lt;/p&gt;&#xA;&lt;p&gt;I have considered edgeR and DESeq2 in R, but it looks like they require counts and I cannot use RPM in these. Or can I convert RPM to counts?&lt;/p&gt;&#xA;&lt;p&gt;Can anyone help?&lt;/p&gt;&#xA;" OwnerUserId="7354" LastEditorUserId="7354" LastEditDate="2021-08-24T03:36:55.657" LastActivityDate="2021-08-24T03:36:55.657" Title="Differential miRNA expression using RPM" Tags="|r|differential-expression|deseq2|gene-expression|edger|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17592" PostTypeId="1" CreationDate="2021-08-24T05:48:33.753" Score="2" ViewCount="155" Body="&lt;p&gt;It appears that AWS and GCP host SRA data and it's beneficial to grab the data from that source when running on GCP for example.&lt;/p&gt;&#xA;&lt;p&gt;Given an SRR accession like SRR1929796 &lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR1929796&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR1929796&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is there a metadata API / source that can tell me which google cloud bucket to retrieve the data from like they show on the website?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gs://sra-pub-crun-4/SRR1929796/SRR1929796.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It also sounds like fastq-dump and fasterq-dump can be configured with Google Cloud credentials to download that for me.&lt;/p&gt;&#xA;&lt;p&gt;I'm just wondering is there a way to determine the google cloud storage bucket path it's in &lt;code&gt;gs://sra-pub-crun-4&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="14028" LastEditorUserId="3848" LastEditDate="2022-03-29T19:53:59.460" LastActivityDate="2023-12-20T01:05:15.700" Title="How to determine NCBI's SRS google cloud bucket or AWS bucket" Tags="|fastq|sratoolkit|sra|cloud|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17593" PostTypeId="1" AcceptedAnswerId="18906" CreationDate="2021-08-24T11:30:57.643" Score="5" ViewCount="89" Body="&lt;h1&gt;The vertebrate genome project (VGP)&lt;/h1&gt;&#xA;&lt;p&gt;has a lot of interesting publications such as &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2020.05.22.110833v1.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The rough pipeline is outlined below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://www.rockefeller.edu/research/uploads/www.rockefeller.edu/sites/8/2018/01/DNAnexus.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here the pipeline in more detail:&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://training.galaxyproject.org/training-material//topics/assembly/images/vgp_assembly/VGP_Pipeline.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;While the paper describes all the steps of the iterative assembly pipeline, I cannot find information on the &lt;strong&gt;runtime&lt;/strong&gt; of their &lt;strong&gt;assembly-pipeline&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I was excited to check out their github page to find out more in the documentation. Unfortunately did I not find any information on the runtime in the &lt;a href=&quot;https://github.com/VGP/vgp-assembly&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/VGP/vgp-assembly&lt;/a&gt; repo.&lt;/p&gt;&#xA;&lt;p&gt;I'have written some e-mails asking collaborators of VGP but I have still not gotten any answers.&lt;/p&gt;&#xA;&lt;p&gt;Did any of you apply this assembly-pipeline?&#xA;Could anyone tell me the runtime of the assembly-pipeline?&lt;/p&gt;&#xA;&lt;p&gt;I hope this question is in the appropriate stackexchange, if not please point me to what you think fits best. Thanks for your suggestions, please be gentle I'm new to all of this.&lt;/p&gt;&#xA;&lt;p&gt;For more info on the VGP check out &lt;a href=&quot;https://www.rockefeller.edu/research/vertebrate-genomes-project/technology-pipeline-and-policies/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10791" LastEditorUserId="10791" LastEditDate="2021-09-09T19:06:27.857" LastActivityDate="2022-04-09T22:10:55.887" Title="Can someone help me estimating the runtime of the pipeline applied by the vertebrate genome project?" Tags="|assembly|genome-sequencing|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17595" PostTypeId="1" AcceptedAnswerId="17611" CreationDate="2021-08-24T16:19:03.353" Score="0" ViewCount="240" Body="&lt;p&gt;Is there any way that I can read &lt;a href=&quot;http://staden.sourceforge.net/manual/formats_unix_2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;SCF&lt;/a&gt; file in python like in R using &lt;a href=&quot;https://rdrr.io/bioc/sangerseqR/man/read.scf.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sangerseqR&lt;/a&gt;, I have tried with &lt;a href=&quot;https://biopython.org/wiki/SeqIO&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biopython&lt;/a&gt;, it seems it does not support this format.&lt;/p&gt;&#xA;" OwnerUserId="12126" LastEditorUserId="12126" LastEditDate="2021-08-24T16:26:46.843" LastActivityDate="2021-08-26T13:34:56.010" Title="How to Read SCF file in Python?" Tags="|python|ngs|sequence-analysis|data-preprocessing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17596" PostTypeId="2" ParentId="17568" CreationDate="2021-08-24T18:21:23.993" Score="2" Body="&lt;h2&gt;&lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/trace/trace.cgi?view=list_arrivals&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Trace Archive&lt;/a&gt;&lt;/h2&gt;&#xA;&lt;p&gt;The NCBI Trace Archive is a permanent repository of DNA sequence chromatograms (traces), base calls, and quality estimates for single-pass reads from various large-scale sequencing projects. Contaning more than 2 billion traces.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/z95Hu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/z95Hu.png&quot; alt=&quot;![![enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12126" LastEditorUserId="12126" LastEditDate="2021-08-25T14:42:21.503" LastActivityDate="2021-08-25T14:42:21.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17597" PostTypeId="1" CreationDate="2021-08-24T18:38:48.663" Score="0" ViewCount="90" Body="&lt;p&gt;I am performing differential gene expression analysis to microarray data for type 2 diabetes donors and nondiabetic donors. When I run the code I get some different results in each time (about 50 or so different genes in each run). Is this normal? I am submitting a project and it could be very problematic if my results change in each run, since I will perform downstream analysis for the DE genes and use web tools such as DAVID for further analysis.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to keep the results from changing in each run?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;g &amp;lt;- getGEO(&amp;quot;GSE20966&amp;quot;,GSEMatrix = TRUE)&#xA;g&amp;lt;- g[[1]]&#xA;exprs(g)&#xA;exprs(g) &amp;lt;- log2(exprs(g))&#xA;&#xA;#Diffrential gene exp&#xA;&#xA;&#xA;library(limma)&#xA;design &amp;lt;- model.matrix(~0+sampleInfo$characteristics_ch1.1)&#xA;&#xA;colnames(design) &amp;lt;- c(&amp;quot;NonDiabetic&amp;quot;,&amp;quot;T2D&amp;quot;)&#xA;&#xA;summary(exprs(g))&#xA;&#xA;exp &amp;lt;- na.omit(exprs(g))&#xA;cutoff &amp;lt;- median(exp)&#xA;&#xA;## TRUE or FALSE for whether each gene is &amp;quot;expressed&amp;quot; in each sample&#xA;is_expressed &amp;lt;- exprs(g) &amp;gt; cutoff&#xA;&#xA;## Identify genes expressed in more than 2 samples&#xA;&#xA;keep &amp;lt;- rowSums(is_expressed) &amp;gt; 2&#xA;&#xA;## check how many genes are removed / retained.&#xA;table(keep)&#xA;keep&amp;lt;- na.omit(keep)&#xA;&#xA;## subset to just those expressed genes&#xA;g&amp;lt;- g[keep,]&#xA;fit &amp;lt;- lmFit(exp, design)&#xA;&#xA;head(fit&lt;span class=&quot;math-container&quot;&gt;$coefficients)&#xA;contrasts &amp;lt;- makeContrasts(T2D - NonDiabetic, levels=design)&#xA;fit2 &amp;lt;- contrasts.fit(fit, contrasts)&#xA;fit2 &amp;lt;- eBayes(fit2)&#xA;topTable(fit2)&#xA;topTable(fit2, coef=1)&#xA;decideTests(fit2)&#xA;table(decideTests(fit2))&#xA;aw &amp;lt;- arrayWeights(exp,design)&#xA;aw&#xA;fit &amp;lt;- lmFit(exp, design,&#xA;            weights = aw)&#xA;contrasts &amp;lt;- makeContrasts(T2D - NonDiabetic, levels=design)&#xA;fit2 &amp;lt;- contrasts.fit(fit, contrasts)&#xA;fit2 &amp;lt;- eBayes(fit2)&#xA;g &amp;lt;- na.omit(g)&#xA;anno &amp;lt;- fData(g)&#xA;anno&#xA;anno &amp;lt;- select(anno,`Gene Symbol`,ENTREZ_GENE_ID)&#xA;fit2$&lt;/span&gt;genes &amp;lt;- anno&#xA;topTable(fit2)&#xA;full_results &amp;lt;- topTable(fit2, number=Inf)&#xA;full_results &amp;lt;- tibble::rownames_to_column(full_results,&amp;quot;ID&amp;quot;)&#xA;library(ggplot2)&#xA;dev.off()&#xA;ggplot(full_results,aes(x = logFC, y=B)) + geom_point()&#xA;p_cutoff &amp;lt;- 0.05&#xA;fc_cutoff &amp;lt;- 1&#xA;&#xA;full_results %&amp;gt;% &#xA;  mutate(Significant = adj.P.Val &amp;lt; p_cutoff, abs(logFC) &amp;gt; fc_cutoff ) &#xA; %&amp;gt;% &#xA;  mutate(Rank = 1:n(), Label = ifelse(Rank &amp;lt; topN, anno$`Gene &#xA; Symbol`,&amp;quot;&amp;quot;)) %&amp;gt;% &#xA;  ggplot(aes(x = logFC, y = B, col=Significant,label=Label)) + &#xA;geom_point() + geom_text_repel(col=&amp;quot;black&amp;quot;)&#xA;&#xA; p_cutoff &amp;lt;- 0.05&#xA; fc_cutoff &amp;lt;- 1&#xA;&#xA;filter(full_results, adj.P.Val &amp;lt; 0.05, abs(logFC) &amp;gt; 1)&#xA; library(pheatmap)&#xA;topN &amp;lt;- 20&#xA;&#xA;top&amp;lt;-150&#xA;gene_names &amp;lt;- mutate(full_results, Rank = 1:n()) %&amp;gt;% &#xA;  filter(Rank &amp;lt; top) %&amp;gt;% &#xA;  pull(Gene.Symbol) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12396" LastEditorUserId="3694" LastEditDate="2023-02-03T11:25:12.883" LastActivityDate="2023-02-03T11:25:12.883" Title="Different results in differential expression analysis of microarray data" Tags="|r|bioconductor|differential-expression|gene-expression|microarray|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17598" PostTypeId="1" AcceptedAnswerId="17605" CreationDate="2021-08-24T20:33:47.373" Score="1" ViewCount="298" Body="&lt;p&gt;I am trying to find hub genes for modules made using WGCNA:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;hub = chooseTopHubInEachModule(datExpr, moduleColors)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I am getting error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in hubs[m] &amp;lt;- colnames(adj)[hub] : replacement has length zero&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thank you for the help1&lt;/p&gt;&#xA;" OwnerUserId="14034" LastActivityDate="2023-01-17T21:18:17.830" Title="WGCNA hub genes error" Tags="|wgcna|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17599" PostTypeId="1" AcceptedAnswerId="17600" CreationDate="2021-08-25T00:33:31.973" Score="1" ViewCount="434" Body="&lt;p&gt;Recently I've been tasked to count the numbers of gene isoforms for each locus in a &lt;code&gt;.gff3&lt;/code&gt; file. I'm still doing my first steps in biology and bioinformatics, so I still struggle with some concepts and methodologies.&lt;/p&gt;&#xA;&lt;p&gt;According to my understanding a gene isoform is a mRNA located in the same locus but with different start positions, so I did the following procedures:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Selected the entries in the &lt;code&gt;.gff3&lt;/code&gt; that are classified as mRNA.&lt;/li&gt;&#xA;&lt;li&gt;In my &lt;code&gt;.gff3&lt;/code&gt; there is the additional information of the locus of the mRNA, so I checked for each locus how many unique values there were for the Start values and subtracted one.&lt;/li&gt;&#xA;&lt;li&gt;Assigned each value from step 2. to their respective locus, resulting in the desired number of isoforms for each locus.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My question is &lt;em&gt;is this method correct&lt;/em&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Why do I believe it could be wrong or inaccurate:&#xA;I don't have a complete solid understanding of a gene isoform, and researching some more I've seen that not only the start point of the mRNA is relevant but other things such as the protein coding DNA sequence (which I believe could be identified in the file under the CDS tag) and untranslated regions (which I don't know if I can identify in my file), and in the method I described above I've only taken in consideration the Start position, what appears to be incomplete.&lt;/p&gt;&#xA;&lt;p&gt;Additionally if the method is incorrect how can I take in consideration the additional circumstances, and if it's correct why  didn't those circumstances influence the count so much?&lt;/p&gt;&#xA;" OwnerUserId="14035" LastEditorUserId="7340" LastEditDate="2021-11-02T13:56:19.880" LastActivityDate="2021-11-02T13:56:19.880" Title="Counting the number of gene isoforms in a GFF3, is this method correct?" Tags="|rna-seq|genomics|gff3|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17600" PostTypeId="2" ParentId="17599" CreationDate="2021-08-25T01:30:55.353" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;In my .gff3 there is the additional information of the locus of the mRNA, so I checked for each locus how many unique values there were for the Start values and subtracted one;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Different isoforms of mRNA can share the same start coordinate. For example, they could share the same start coordinate but one skips an exon. MSpaint illustration below to help. Generally for GFF3 I would simply count e.g. how many mRNA have Parent=GENE0001234 to get the number of mRNA for GENE0001234, and you can expand that for all instances of GENENNNNNNNN&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RluRS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RluRS.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;See also for example Wikipedia for Alternative splicing &lt;a href=&quot;https://en.wikipedia.org/wiki/Alternative_splicing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Alternative_splicing&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="60" LastActivityDate="2021-08-25T01:30:55.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17601" PostTypeId="2" ParentId="17599" CreationDate="2021-08-25T01:49:06.273" Score="0" Body="&lt;p&gt;Gene isoforms are transcripts derived from the same gene that vary in their exons (blocks of the gene that make up the final transcript). This can alter the start position, end position, and/or internal sequence of the transcript. GTF/GFF files account for this by having multiple transcripts per gene (if applicable) that capture the various isoforms of the gene.&lt;/p&gt;&#xA;&lt;p&gt;Here's a little example in R to count the number of transcripts for a few human genes found in an ENSEMBL reference GFF3 file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(&amp;quot;GenomicFeatures&amp;quot;)&#xA;&#xA;gff3 &amp;lt;- makeTxDbFromGFF(&amp;quot;http://ftp.ensembl.org/pub/release-104/gff3/homo_sapiens/Homo_sapiens.GRCh38.104.chr.gff3.gz&amp;quot;)&#xA;&#xA;&amp;gt; stack(vapply(transcriptsBy(gff3, &amp;quot;gene&amp;quot;)[1:10], length, 1L))&#xA;   values             ind&#xA;1       5 ENSG00000000003&#xA;2       2 ENSG00000000005&#xA;3      16 ENSG00000000419&#xA;4       5 ENSG00000000457&#xA;5       9 ENSG00000000460&#xA;6       7 ENSG00000000938&#xA;7       6 ENSG00000000971&#xA;8       3 ENSG00000001036&#xA;9      14 ENSG00000001084&#xA;10      2 ENSG00000001167&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9471" LastActivityDate="2021-08-25T01:49:06.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17602" PostTypeId="1" AcceptedAnswerId="17603" CreationDate="2021-08-25T06:35:41.227" Score="1" ViewCount="376" Body="&lt;p&gt;This is RMSF for protein from GROMACS after 100ns simulation-&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/H0QPP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/H0QPP.png&quot; alt=&quot;RMSF Result&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have few questions-&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Residue numbers are starting from 1650 to 1850, has it ignore LIG&#xA;and HOH as residues?&lt;/li&gt;&#xA;&lt;li&gt;There are three lines, so are there three chains and their residue number is same?&lt;/li&gt;&#xA;&lt;li&gt;Are the straight lines connecting these chains or this is kind of loop something?&lt;/li&gt;&#xA;&lt;li&gt;Since I am seeing 2nm or 20A fluctuation, is the protein quite disturbed?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14037" LastActivityDate="2021-09-24T13:00:50.070" Title="GROMACS RMSF Result Analysis Problem" Tags="|protein-structure|gromacs|molecular-dynamics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17603" PostTypeId="2" ParentId="17602" CreationDate="2021-08-25T12:38:01.347" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;The residue numbering in a PDB file mainly depend on the person who deposited it. There are usually reasons for how things are numbered, but ultimately you should look at PDB file, and check the details on how the structure was obtained. To give you an example, during my PhD I worked on a protein that had some missing residues at the N-terminus. That was because the N-term was very unstable and they didn't manage to crystallise it properly.&lt;/li&gt;&#xA;&lt;li&gt;You need to check the PDB, but it is possible that the 3 chains have the same residue numbering (A1-200, B1-200). This is common in structures where you have multiple identical sub-units (e.g. dimers).&lt;/li&gt;&#xA;&lt;li&gt;The lines are merely an artefact. Those lines are the attempt of the engine to draw a single line for all the chains. You go from residue 1850 of chain 1 to residue 1650 of chain 2, and so on. Some tools handle this behaviour, others don't. I would suggest you to parse the PDB yourself and plot (e.g. &lt;code&gt;ggplot&lt;/code&gt;) each chain as a layer of the same figures .&lt;/li&gt;&#xA;&lt;li&gt;It is indeed, but only you can tell if this is good or bad. It really depends on how you calculate the RMSF. Was it on the backbone or on the side-chains? How is your dynamics? High temperature? Does you protein partially unfold? are the chains separating?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3631" LastActivityDate="2021-08-25T12:38:01.347" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17604" PostTypeId="1" CreationDate="2021-08-25T12:52:23.693" Score="0" ViewCount="342" Body="&lt;p&gt;I have started using snakemake 6.5.2 to align fastq files with reference file. I have pasted the error below in this question. How to allocate memory in the snakefile and read the header from samfile, '-'. This is the snakefile (wrapper for running alignment):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; rule bwa_mem2_mem:&#xA;        input:&#xA;            reads=[&amp;quot;/scicore/home/cichon/GROUP/test_workflow/samples/{sample}.1.fq&amp;quot;, &amp;quot;/scicore/home/cichon/GROUP/test_workflow/samples/{sample}.2.fq&amp;quot;]&#xA;        output:&#xA;            &amp;quot;/scicore/home/cichon/GROUP/test_workflow/output/{sample}.bam&amp;quot;&#xA;        log:&#xA;            &amp;quot;/scicore/home/cichon/GROUP/test_workflow/output/{sample}.log&amp;quot;&#xA;        params:&#xA;            index=&amp;quot;/scicore/home/cichon/GROUP/test_workflow/data/gch38.fa&amp;quot;,&#xA;            extra=r&amp;quot;-R '@RG\tID:{sample}\tSM:{sample}'&amp;quot;,&#xA;            sort=&amp;quot;none&amp;quot;,             # Can be 'none', 'samtools' or 'picard'.&#xA;            sort_order=&amp;quot;coordinate&amp;quot;, # Can be 'coordinate' (default) or 'queryname'.&#xA;            sort_extra=&amp;quot;&amp;quot;     &#xA;&#xA;        threads: 8&#xA;        wrapper:&#xA;            &amp;quot;0.77.0/bio/bwa-mem2/mem&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I used this command to execute the above snakemake file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;snakemake --use-conda --snakefile Snakefile --cores -p /scicore/home/cichon/GROUP/test_workflow/output/DE98NGSUKBD117612_1.bam /scicore/home/cichon/GROUP/test_workflow/output/DE71NGSUKBD117613_3.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Error from log file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Looking to launch executable &amp;quot;/scicore/home/cichon/GROUP/test_workflow/.snakemake/conda/0ecf6ba401c70d7501c73bb1c733a060/bin/bwa-mem2.avx&amp;quot;, simd = .avx&#xA;Launching executable &amp;quot;/scicore/home/cichon/GROUP/test_workflow/.snakemake/conda/0ecf6ba401c70d7501c73bb1c733a060/bin/bwa-mem2.avx&amp;quot;&#xA;-----------------------------&#xA;Executing in AVX mode!!&#xA;-----------------------------&#xA;* SA compression enabled with xfactor: 8&#xA;* Ref file: /scicore/home/cichon/GROUP/test_workflow/data/gch38.fa&#xA;* Entering FMI_search&#xA;* Index file found. Loading index from /scicore/home/cichon/GROUP/test_workflow/data/gch38.fa.bwt.2bit.64&#xA;* Reference seq len for bi-index = 6544178411&#xA;* sentinel-index: 2782712944&#xA;* Count:&#xA;0,  1&#xA;1,  1914743810&#xA;2,  3272089206&#xA;3,  4629434602&#xA;4,  6544178411&#xA;&#xA;* Reading other elements of the index from files /scicore/home/cichon/GROUP/test_workflow/data/gch38.fa&#xA;* Index prefix: /scicore/home/cichon/GROUP/test_workflow/data/gch38.fa&#xA;* Read 0 ALT contigs&#xA;[fread] Cannot allocate memory&#xA;[main_samview] fail to read the header from &amp;quot;-&amp;quot;.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&lt;p&gt;Priya&lt;/p&gt;&#xA;" OwnerUserId="14040" LastActivityDate="2021-08-25T12:52:23.693" Title="BWA-MEM2 alignment-Snakemake" Tags="|ngs|snakemake|python|workflow-management|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17605" PostTypeId="2" ParentId="17598" CreationDate="2021-08-25T17:37:35.290" Score="1" Body="&lt;p&gt;The most likely culprit here is that your expression data (the datExpr input to chooseTopHubInEachModule) has no colnames. Check that your data do have appropriate column names. Also check that the module color vector you supply has at least actual module (type table(moduleColors) and check that you have at least one module apart from &amp;quot;grey&amp;quot; or 0).&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2021-08-25T17:37:35.290" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17606" PostTypeId="1" CreationDate="2021-08-25T20:07:00.490" Score="2" ViewCount="488" Body="&lt;p&gt;I am still new to this.&lt;/p&gt;&#xA;&lt;p&gt;I am trying to download the GRch37 dbSNP data from the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/?term=%20rs141575513&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI dbSNP Browser&lt;/a&gt; using the &amp;quot;Send to:&amp;quot; button. But whenever I select&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Send to:&amp;quot; &amp;gt; &amp;quot;File&amp;quot; &amp;gt; &amp;quot;Tab&amp;quot; &amp;gt; &amp;quot;Default order&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;The coordinates of my positions will be GRch38 based. Does anyone knows how can I download the dbSNP data with GRch37 coordinates?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&lt;p&gt;Ken&#xA;&lt;a href=&quot;https://i.stack.imgur.com/BJ6wO.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BJ6wO.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tdsMM.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tdsMM.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14043" LastEditorUserId="650" LastEditDate="2021-08-26T21:05:31.237" LastActivityDate="2021-08-26T21:05:31.237" Title="dbSNP GRch37 Download" Tags="|public-databases|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17609" PostTypeId="2" ParentId="17606" CreationDate="2021-08-26T10:08:09.613" Score="2" Body="&lt;p&gt;I see no option to specify the reference version using the NCBI dbSNP browser, it is set to download the latest built information. I think this is not the standard tool to download dbSNP data across many sites, but rather to browse it and visualize it for specific variation.&lt;/p&gt;&#xA;&lt;p&gt;If you are only interested in getting info for a smaller range of positions, I recommend you to use the UCSC browser, go to &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTables?hgsid=1148861511_dmErwL9heyBIHik5fOe7yXaTP76i&quot; rel=&quot;nofollow noreferrer&quot;&gt;Table Browser&lt;/a&gt;, -&amp;gt; assembly GRch37, -&amp;gt; group variation, -&amp;gt; track all SNPs, -&amp;gt; table snp51 -&amp;gt; have a look at &amp;quot;describe table schema&amp;quot; to see what will you get, select your genomic range, and click on get output.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively and if you have a lot of space available in a server, simply go to the &lt;a href=&quot;https://ftp.ncbi.nih.gov/snp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI dbSNP FTP site&lt;/a&gt;, and download the latest GRch37 dataset (GCF_000001405.25) with all the entries. Make sure you read the README for further info about the files and data.&lt;/p&gt;&#xA;" OwnerUserId="475" LastActivityDate="2021-08-26T10:08:09.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17611" PostTypeId="2" ParentId="17595" CreationDate="2021-08-26T13:34:56.010" Score="1" Body="&lt;p&gt;It was done using &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/sangerseqR.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sangerseqR&lt;/a&gt; with &lt;a href=&quot;https://rpy2.github.io/doc/latest/html/overview.html#&quot; rel=&quot;nofollow noreferrer&quot;&gt;rpy2&lt;/a&gt; in python&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    import rpy2.robjects as r&#xA;    from rpy2.robjects.packages import importr&#xA;    utils = importr('utils')&#xA;    utils.install_packages('sangerseqR', repos=&amp;quot;https://git.bioconductor.org/packages/sangerseqR&amp;quot;)&#xA;    utils.chooseBioCmirror(ind=1) # select the first mirror in the list&#xA;    utils.install_packages('sangerseqR')&#xA;&#xA;&#xA;    sangerseqR = importr('sangerseqR')&#xA;    Trace = sangerseqR.readsangerseq( file=&amp;quot;1I1_F_P1815443_047.scf&amp;quot;) #scf sequence file is now loaded&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/vignettes/sangerseqR/inst/doc/sangerseq_walkthrough.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;sangerseqR Documentation&lt;/a&gt;(pdf)&lt;/p&gt;&#xA;" OwnerUserId="12126" LastActivityDate="2021-08-26T13:34:56.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17613" PostTypeId="2" ParentId="13237" CreationDate="2021-08-26T13:44:53.957" Score="1" Body="&lt;p&gt;Recently I have done something similar for my work, I am using &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/sangerseqR.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sangerseqR&lt;/a&gt; which is s Bioconductor package in Python Environment using &lt;a href=&quot;https://rpy2.github.io/doc/latest/html/overview.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;rpy2&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    import rpy2.robjects as r&#xA;    from rpy2.robjects.packages import importr&#xA;    utils = importr('utils')&#xA;    utils.install_packages('sangerseqR', repos=&amp;quot;https://git.bioconductor.org/packages/sangerseqR&amp;quot;)&#xA;    utils.chooseBioCmirror(ind=1) # select the first mirror in the list&#xA;    utils.install_packages('sangerseqR')&#xA;&#xA;    sangerseqR = importr('sangerseqR')&#xA;&#xA;   #now using the sangerseqR package to the read the sequcence&#xA;   Trace = sangerseqR.readsangerseq( file=&amp;quot;1I1_F_P1815443_047.scf&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12126" LastEditorUserId="12126" LastEditDate="2021-08-30T10:18:45.940" LastActivityDate="2021-08-30T10:18:45.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17619" PostTypeId="1" CreationDate="2021-08-27T14:54:01.910" Score="1" ViewCount="72" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9486373/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have a list of GO ids and I want to find out a unique term description such that if I provide say 200 GO IDs I will give 200 specific GO terms. The code snippet I am using is given below. Although it kind of works, for one given  GO ID I get several terms like for my 200 GO ids I get 2577 terms whereas I am interested in a specific GO term. The output that I want is like this GO:0007254 = JNK cascade.&#xA;I will like to mention that I have also cross-posted the query in biostars and got no reply for the last 2 days and hence I am posting the query on Stack exchange bioinformatics. Any help will be useful.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(biomaRT)&#xA;mart &amp;lt;- useMart(biomart = &amp;quot;ENSEMBL_MART_ENSEMBL&amp;quot;, dataset = &amp;quot;drerio_gene_ensembl&amp;quot;, host = &amp;quot;asia.ensembl.org&amp;quot;)  &#xA;v1 &amp;lt;-c(target)   # The list of GO ids that I want to pass as a list.&#xA;go_list &amp;lt;- getBM(attributes=c(&amp;quot;go_id&amp;quot;, &amp;quot;name_1006&amp;quot;, &amp;quot;namespace_1003&amp;quot;),&#xA;              filters = &amp;quot;go&amp;quot;,&#xA;              values = c(&amp;quot;v1&amp;quot;),&#xA;              mart=mart, uniqueRows = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9659" LastEditorUserId="73" LastEditDate="2022-10-13T23:39:43.763" LastActivityDate="2022-10-13T23:39:43.763" Title="Mapping unique GO term description given a specific GO ids" Tags="|genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17620" PostTypeId="2" ParentId="17619" CreationDate="2021-08-27T16:35:57.087" Score="2" Body="&lt;pre&gt;&lt;code&gt;library(biomaRt)&#xA;mart &amp;lt;- biomaRt:::useMart(biomart = &amp;quot;ENSEMBL_MART_ENSEMBL&amp;quot;, dataset = &amp;quot;drerio_gene_ensembl&amp;quot;, host = &amp;quot;asia.ensembl.org&amp;quot;)  &#xA;&#xA;v1 &amp;lt;- c(&amp;quot;GO::0009055&amp;quot;,&amp;quot;GO:0016020&amp;quot;,&amp;quot;GO:0016491&amp;quot;,&amp;quot;GO:0006122&amp;quot;,&amp;quot;GO:0008121&amp;quot;)&#xA;&#xA;go_list &amp;lt;- biomaRt:::getBM(attributes=c(&amp;quot;go_id&amp;quot;, &amp;quot;name_1006&amp;quot;, &amp;quot;namespace_1003&amp;quot;),&#xA;              filters = &amp;quot;go&amp;quot;,&#xA;              values = v1,&#xA;              mart=mart, uniqueRows = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;so it is likely the &amp;quot;v1&amp;quot; that should not be in quotes&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head(go_list)&#xA;        go_id                                                   name_1006&#xA;1 GO:0016020                                                    membrane&#xA;2 GO:0005739                                               mitochondrion&#xA;3 GO:0005743                                mitochondrial inner membrane&#xA;4 GO:0070469                                                 respirasome&#xA;5 GO:0006122 mitochondrial electron transport, ubiquinol to cytochrome c&#xA;6 GO:0001525                                                angiogenesis&#xA;      namespace_1003&#xA;1 cellular_component&#xA;2 cellular_component&#xA;3 cellular_component&#xA;4 cellular_component&#xA;5 biological_process&#xA;6 biological_process&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="383" LastEditorUserId="383" LastEditDate="2021-08-30T09:59:14.690" LastActivityDate="2021-08-30T09:59:14.690" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17621" PostTypeId="1" CreationDate="2021-08-27T19:06:01.070" Score="0" ViewCount="485" Body="&lt;p&gt;When converting vcf to tbed/tfam format, I am getting error like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;unrecognized flag&amp;lt;'--transpose'&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Code I used:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plink2 --vcf sample_vcf_file.vcf --recode --transpose --out outputfile&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14054" LastEditorUserId="650" LastEditDate="2021-08-27T19:25:13.070" LastActivityDate="2021-08-29T15:24:11.880" Title="How to convert vcf to tbed/tfam and again convert it back to tbed/tfam to vcf using plink" Tags="|plink|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17623" PostTypeId="1" AcceptedAnswerId="17626" CreationDate="2021-08-28T11:27:59.563" Score="0" ViewCount="35" Body="&lt;p&gt;I have done the differential accessibility analysis then annotated the region with chipseeker.&lt;/p&gt;&#xA;&lt;p&gt;my output for one of the many region and question regarding this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Peak log2FoldChange         padj seqnames    start      end width strand        annotation geneChr geneStart&#xA;105  Peak32946      -1.992474 9.197721e-19    chr11 14078815 14079558   744      * Distal Intergenic      11  13962689&#xA;5445 Peak32938      -2.473959 4.878995e-04    chr11 13961467 13961818   352      *          Promoter      11  13962689&#xA;      geneEnd geneLength geneStrand geneId      transcriptId distanceToTSS         ENSEMBL SYMBOL  GENENAME&#xA;105  14267884     305196          1  10418 ENST00000576479.3        116126 ENSG00000262655  SPON1 spondin 1&#xA;5445 14267884     305196          1  10418 ENST00000576479.3          -871 ENSG00000262655  SPON1 spondin 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So the same region is marked as Distal Intergenic and Promoter. Next I have the same comparison RNA seq expression data. So now my objective is to find those genes+region which are expressed as well as deferentially accessible.&lt;/p&gt;&#xA;&lt;p&gt;So for example here in this case in RNA seq data there is only one &lt;code&gt;SPON1 gene&lt;/code&gt; so if I have to show it associated region how to put that,shall I consider promoter or the distal intergenic or consider which is more significant.&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2021-08-28T19:38:11.343" Title="Same region/region but different annotation" Tags="|r|atac-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17624" PostTypeId="2" ParentId="14795" CreationDate="2021-08-28T17:24:27.033" Score="1" Body="&lt;p&gt;Cromwell will send jobs to the AWS Batch queue ARN defined in the &lt;code&gt;cromwell.conf&lt;/code&gt; file in the Cromwell server. To use Spot instances you can create an AWS Batch Compute environment configured to use Spot instances and create a queue for that compute environment. You can then set the arn of that queue in the configuration file or override the configured value using the standard mechanisms of overriding Cromwell’s configuration (workflow options, runtime parameters etc). If you change the configuration file you will need to restart the server.&lt;/p&gt;&#xA;&lt;p&gt;Batch queues can point to more than one compute environment. Each will have a priority. Jobs in a queue will be sent to the highest priority environment until that environment has reached its max vCPU capacity, at that point jobs will be sent to the next environment.&lt;/p&gt;&#xA;" OwnerUserId="8985" LastEditorUserId="8985" LastEditDate="2021-08-28T17:29:53.207" LastActivityDate="2021-08-28T17:29:53.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17625" PostTypeId="1" AcceptedAnswerId="17627" CreationDate="2021-08-28T18:03:30.150" Score="1" ViewCount="967" Body="&lt;p&gt;Suppose, we have a protein (&lt;code&gt;16PK&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;We are considering a 5-residue segment/window.&lt;/p&gt;&#xA;&lt;p&gt;If we assume &lt;code&gt;ASN10&lt;/code&gt; to be the &lt;code&gt;i&lt;/code&gt;th residue, how can I calculate the following distances in &lt;strong&gt;Python&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;from [i+2] to [i-2]&lt;/li&gt;&#xA;&lt;li&gt;from [i+1] to [i-2]&lt;/li&gt;&#xA;&lt;li&gt;from [i+2] to [i-1]&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Next major question is, what would be the distances like if my segment/window is 7 or 9 residues long?&lt;/p&gt;&#xA;&lt;img src=&quot;https://i.stack.imgur.com/ft9jZ.png&quot; width=&quot;450&quot;&gt;&#xA;" OwnerUserId="12686" LastActivityDate="2021-08-29T00:02:56.630" Title="How can I calculate the distances between two specific residues of a protein from a PDB file?" Tags="|python|proteins|pdb|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17626" PostTypeId="2" ParentId="17623" CreationDate="2021-08-28T19:38:11.343" Score="2" Body="&lt;p&gt;You have two different peaks at different positions with different annotations. This makes complete sense, you're just misreading it.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-08-28T19:38:11.343" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17627" PostTypeId="2" ParentId="17625" CreationDate="2021-08-29T00:02:56.630" Score="2" Body="&lt;p&gt;Your &lt;span class=&quot;math-container&quot;&gt;$[i,j,k]$&lt;/span&gt; coordinates in the PDB file are coordinates. You can readily compute &lt;a href=&quot;http://%20Euclidean%20distance%20-%20Wikipediahttps://en.wikipedia.org%20%E2%80%BA%20wiki%20%E2%80%BA%20Euclidean_distance&quot; rel=&quot;nofollow noreferrer&quot;&gt;Euclidean distances&lt;/a&gt; between any pair of coordinates as described in the wiki page:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$d(p,q)=\sqrt{(p_1-q_1)^2 + (p_2-q_2)^2 + (p_3-q_3)^2}.&#xA;$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://warwick.ac.uk/fac/sci/moac/people/students/peter_cock/python/protein_contact_map/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This resource&lt;/a&gt; suggests one such approach (code reproduced below in case that site stops existing):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import Bio.PDB&#xA;import numpy&#xA;&#xA;def calc_residue_dist(residue_one, residue_two) :&#xA;    &amp;quot;&amp;quot;&amp;quot;Returns the C-alpha distance between two residues&amp;quot;&amp;quot;&amp;quot;&#xA;    diff_vector  = residue_one[&amp;quot;CA&amp;quot;].coord - residue_two[&amp;quot;CA&amp;quot;].coord&#xA;    return numpy.sqrt(numpy.sum(diff_vector * diff_vector))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-08-29T00:02:56.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17628" PostTypeId="2" ParentId="17480" CreationDate="2021-08-29T00:39:22.003" Score="1" Body="&lt;p&gt;Related question: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/7401/access-base-aligned-to-particular-reference-position&quot;&gt;Access base aligned to particular reference position&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;After an initial effort posted here, I ended up rewriting this and testing it a little, and posted the script &lt;a href=&quot;https://github.com/maximilianpress/teaching_code/blob/master/variant_cooccurrence.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. It still has some drawbacks (handling indels intelligently) but now it more or less works.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2021-08-29T22:20:38.900" LastActivityDate="2021-08-29T22:20:38.900" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17629" PostTypeId="1" CreationDate="2021-08-29T03:16:04.453" Score="1" ViewCount="740" Body="&lt;p&gt;I am new in bioinformatic field. I would like to know a way to generate a consensus sequence from nanopore fastq files (fastq files demultiplexed).&lt;/p&gt;&#xA;&lt;p&gt;I usually generate a consensus sequence with &amp;quot;MEDAKA&amp;quot; of artic protocol, but it is used to generate a SARS CoV 2 sequence only.&lt;/p&gt;&#xA;&lt;p&gt;I would like to know a differente way to do it with another program, R, or python.&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="12707" LastActivityDate="2021-08-29T03:16:04.453" Title="How to get a consensus sequence from a nanopore fastq files?" Tags="|r|python|bioconductor|software-recommendation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17630" PostTypeId="2" ParentId="17621" CreationDate="2021-08-29T15:24:11.880" Score="1" Body="&lt;p&gt;Correct command is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plink2 --vcf sample_vcf_file.vcf --recode transpose --out outputfile&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;vcftools also can convert vcf into tped/tfam&lt;/p&gt;&#xA;" OwnerUserId="7439" LastActivityDate="2021-08-29T15:24:11.880" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17631" PostTypeId="1" CreationDate="2021-08-30T10:51:49.630" Score="0" ViewCount="212" Body="&lt;p&gt;I am analysing Pyrococcus Furiosus DNA sequencing data by considering data published &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NC_003413.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; in NCBI. When I click on &amp;quot;Send to&amp;quot;&amp;gt;&amp;quot;Gene Features&amp;quot;&amp;gt;&amp;quot;FASTA format&amp;quot; I download a file that has the sequences of genes of this organism but I realized that this file has some sequences that are doubled (that is the sequence is present twice with different identifiers)... is there a way to get a file with all genes annotated and their respective DNA sequences without double sequences and with all sequences in the 5' to 3' direction without reverse complement in a well &amp;quot;ordered&amp;quot; way ? In NCBI (in the link I reported before) it indicates 2,128 genes so I would like a file with all these 2,128 genes annotated and their respective DNA sequences in FASTA format. Do you know if there is an other website or an other place in NCBI in which I can search to get this kind of file?&lt;/p&gt;&#xA;&lt;p&gt;EDIT: if you follow the path &amp;quot;Send to&amp;quot;&amp;gt;&amp;quot;Gene Features&amp;quot;&amp;gt;&amp;quot;FASTA format&amp;quot;, now it seems that there are just 2,128 genes so this problem is vanished in this case. Anyway, I would appreciate the answer for the future and for the reverse complement gene sequences that are still there.&lt;/p&gt;&#xA;&lt;p&gt;Thank you very much.&lt;/p&gt;&#xA;" OwnerUserId="10594" LastEditorUserId="10594" LastEditDate="2021-09-20T14:53:49.997" LastActivityDate="2021-09-20T14:53:49.997" Title="How to get a FASTA format file with the DNA sequences of all annotated genes?" Tags="|fasta|sequence-analysis|public-databases|phylogenetics|public-dataset|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17633" PostTypeId="1" AcceptedAnswerId="17634" CreationDate="2021-08-30T17:15:42.093" Score="1" ViewCount="234" Body="&lt;p&gt;I'm working with paired-end NGS reads from an Illumina platform. The sequences I have are all of the same gene, but have one or more substitution mutations each.&lt;/p&gt;&#xA;&lt;p&gt;Here is a rough workflow for generating the reads:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Expression of a library of sequence variants for a heterogeneous protein of interest in phage.&lt;/li&gt;&#xA;&lt;li&gt;Selection of high-fitness variants according to a custom assay.&lt;/li&gt;&#xA;&lt;li&gt;Extraction of the variant pool via amplification of the variable region of the gene of interest.&lt;/li&gt;&#xA;&lt;li&gt;Preparation of Illumina sequencing library from the amplicon pool.&lt;/li&gt;&#xA;&lt;li&gt;Deep sequencing of the amplicon pool.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I've already combined the paired-end reads with USearch and manually removed sequences with subpar quality scores, but I now need to translate all of the remaining DNA sequences into amino acid sequences. I'm uncertain of where to begin addressing this issue, and I was hoping someone could point me in the direction of an application or python package that could assist me.&lt;/p&gt;&#xA;&lt;p&gt;My goal: &lt;strong&gt;Count the abundances of each sequence variant within the Illumina data to identify which of the &lt;em&gt;protein&lt;/em&gt; sequences are the most fit.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="14001" LastEditorUserId="14001" LastEditDate="2021-09-01T16:08:08.643" LastActivityDate="2021-09-01T16:08:08.643" Title="Translate all reads in a .fastq into protein sequence from deep mutational scanning experiment" Tags="|python|proteins|phylogenetics|fastq|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="17634" PostTypeId="2" ParentId="17633" CreationDate="2021-08-30T21:17:35.547" Score="2" Body="&lt;p&gt;Based on back and forth in the comments, I think that you might be interested in a deep mutational scanning workflow, for example &lt;a href=&quot;https://jbloomlab.github.io/dms_tools2/examples.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;dms_tools2&lt;/code&gt;&lt;/a&gt; or &lt;a href=&quot;https://github.com/FowlerLab/Enrich2&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;enrich2&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;These are built specifically for the kind of experiment that you are interested in, if I understand correctly. See attached figure from dms_tools2:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/07tZ9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/07tZ9.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-08-30T21:17:35.547" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17635" PostTypeId="1" CreationDate="2021-08-31T04:13:10.493" Score="-1" ViewCount="62" Body="&lt;p&gt;I tried to identify intramodular hub genes in &lt;code&gt;WGCNA&lt;/code&gt; analysis. I used:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;hub = chooseTopHubInEachModule(datExpr, moduleColors)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But when I checked the hub genes for the modules, I found that the hub genes are expressed only in less than 5 out of 300 samples. I did not filter the gene expression data for any low expressed genes in certain number of samples. How does this make biological relevance if the hub genes are not expressed in most of the samples?&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="14034" LastActivityDate="2022-01-28T19:02:50.213" Title="no expression for hub genes identified in wgcna" Tags="|wgcna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17636" PostTypeId="2" ParentId="17581" CreationDate="2021-08-31T12:57:42.630" Score="2" Body="&lt;ol&gt;&#xA;&lt;li&gt;Yes, the letters are amino acids&lt;/li&gt;&#xA;&lt;li&gt;It's a schematic with a whole residue as a circle —possibly from Voet and Voet.&lt;/li&gt;&#xA;&lt;li&gt;Residue &lt;code&gt;i&lt;/code&gt; is the one in focus, the reference. &lt;code&gt;i-1&lt;/code&gt; is the preceding residue. Have a gander at &lt;a href=&quot;http://en.wikipedia.org/wiki/Turn_(biochemistry)&quot; rel=&quot;nofollow noreferrer&quot;&gt;en.wikipedia.org/wiki/Turn_(biochemistry)&lt;/a&gt; where this type of relative notation is important.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The diagram shows how secondary structure (helices, sheets etc.) affect the distance between the residues. In a helix a residue 3 positions down will be close, but not in a sheet.&lt;/p&gt;&#xA;&lt;p&gt;However, for assignment of secondary structure it may be easier to look at the φ and ψ angles:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;α helix: phi=-57.8, psi=-47.0&lt;/li&gt;&#xA;&lt;li&gt;π helix: phi=-57.1, psi=-69.7&lt;/li&gt;&#xA;&lt;li&gt;3.10 helix: phi=-74.0, psi=-4.0&lt;/li&gt;&#xA;&lt;li&gt;β sheet: phi=-139, psi=+135&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Although these have a range and you can get π bulges within α helices —important for conformation switching for pore opening (capsaicin binding to TRPV1 is a classic example).&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-08-31T12:57:42.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17637" PostTypeId="2" ParentId="17635" CreationDate="2021-08-31T17:28:51.237" Score="1" Body="&lt;p&gt;It could be that your modules group together genes (over)expressed in only certain small groups of samples that may be biologically relevant. You would need to check whether the samples in which the hubs (and presumably the modules) are up share some sort of a characteristic.&lt;/p&gt;&#xA;&lt;p&gt;In general though, there are good reasons why authors of WGCNA recommend filtering out non- and low-expressed genes - correlations tend to not be a good co-expression measure when counts are very low, and technical effects can be much more pronounced. It is also recommended to spend a bit of time on preprocessing to make sure the data set does not have strong global drivers that are technical or uninteresting.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2021-08-31T17:28:51.237" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17638" PostTypeId="1" AcceptedAnswerId="17639" CreationDate="2021-09-01T09:56:50.380" Score="1" ViewCount="233" Body="&lt;p&gt;I want to automate some scripts which should run after a MinION run completes. Is there a file which is produced by MinKNOW which will enable me to tell that the run has finished?&lt;/p&gt;&#xA;" OwnerUserId="993" LastActivityDate="2021-09-01T09:56:50.380" Title="How do I tell that a MinION run has finished based on the files produced?" Tags="|nanopore|minion|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17639" PostTypeId="2" ParentId="17638" CreationDate="2021-09-01T09:56:50.380" Score="1" Body="&lt;p&gt;You can just look for the presence of a file called &amp;quot;final_summary_*.txt&amp;quot; in the run folder. From &lt;a href=&quot;https://twitter.com/mattloose?lang=en&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matt Loose&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="993" LastActivityDate="2021-09-01T09:56:50.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17641" PostTypeId="1" CreationDate="2021-09-01T16:25:21.847" Score="1" ViewCount="386" Body="&lt;p&gt;I used &lt;code&gt;rma&lt;/code&gt; from the oligo package to normalize CEL files, returning an object of class &lt;code&gt;ExpressionSet&lt;/code&gt;. How do I access the actual expression data?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; data = read.celfiles(names)&#xA;&amp;gt; data2 = oligo::rma(data)&#xA;class(data2)&#xA;[1] &amp;quot;ExpressionSet&amp;quot;&#xA;attr(,&amp;quot;package&amp;quot;)&#xA;[1] &amp;quot;Biobase&amp;quot;&#xA;&amp;gt; data3 = as.matrix(data2)&#xA;Error in as.vector(data) : &#xA;  no method for coercing this S4 class to a vector&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14091" LastEditorDisplayName="user3051" LastEditDate="2021-09-02T08:42:35.033" LastActivityDate="2021-09-03T10:54:33.140" Title="Accessing expression data in an ExpressionSet" Tags="|bioconductor|microarray|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17643" PostTypeId="2" ParentId="17631" CreationDate="2021-09-02T02:15:13.223" Score="1" Body="&lt;p&gt;From &lt;a href=&quot;https://stackoverflow.com/a/61374803/7092484&quot;&gt;this SO post&lt;/a&gt;. You can use &lt;a href=&quot;https://bioinf.shenwei.me/seqkit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqkit&lt;/a&gt; to remove duplicate sequences with the command below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqkit rmdup -s &amp;lt; sequences.txt &amp;gt; out.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The &lt;code&gt;rmdup&lt;/code&gt; option removes duplicates, and the &lt;code&gt;-s&lt;/code&gt; option calls duplicates on the basis of sequence, ignoring differences in headers.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="6628" LastActivityDate="2021-09-02T02:15:13.223" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="17644" PostTypeId="2" ParentId="17641" CreationDate="2021-09-02T07:27:26.447" Score="2" Body="&lt;p&gt;The class of your object which the &lt;code&gt;rma&lt;/code&gt; function returns is an &lt;code&gt;ExpressionSet&lt;/code&gt; which is a standardized data structure in Bioconductor/R from the Biobase package.&lt;/p&gt;&#xA;&lt;p&gt;It stores both expression (a numeric matrix) data as well as column- and row annotation data.&#xA;You have to use the getter function &lt;code&gt;exprs()&lt;/code&gt; to access the expression data slot, so &lt;code&gt;exprs(data2)&lt;/code&gt; which is then of class &lt;code&gt;matrix&lt;/code&gt;, link to &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/Biobase/inst/doc/ExpressionSetIntroduction.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorUserId="48" LastEditDate="2021-09-03T10:54:33.140" LastActivityDate="2021-09-03T10:54:33.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17645" PostTypeId="1" CreationDate="2021-09-02T08:04:47.950" Score="4" ViewCount="231" Body="&lt;p&gt;I constructed a phylogenetic tree using a gene (example - &lt;a href=&quot;https://www.uniprot.org/uniprot/P10408&quot; rel=&quot;nofollow noreferrer&quot;&gt;secA&lt;/a&gt;). I had to gather the same gene sequence for all the required species from public database-NCBI and then constructed the tree after multiple sequence alignment. I used the MEGA X software.&lt;/p&gt;&#xA;&lt;p&gt;I now want to build a single phylogenetic tree using concatenated multiple genes, around 20.&lt;/p&gt;&#xA;&lt;p&gt;How do I go about it.&#xA;Do I,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Concatenate all gene sequences for each species and do a multiple alignment or&lt;/li&gt;&#xA;&lt;li&gt;perform an alignment for each gene and finally concatenate the alignments to get the tree?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Could this be done using MEGA X?&lt;/p&gt;&#xA;" OwnerUserId="12357" LastEditorUserId="3848" LastEditDate="2022-02-19T23:11:59.567" LastActivityDate="2022-02-19T23:11:59.567" Title="How to get phylogenetic tree from multiple genes?" Tags="|sequence-alignment|phylogenetics|phylogeny|multiple-sequence-alignment|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17646" PostTypeId="1" CreationDate="2021-09-02T12:35:15.000" Score="0" ViewCount="78" Body="&lt;p&gt;I have an issue running a module trying to detect circRNA from RNAseq data. The error is a java error and I have not knowledge of java so I don't understand the issues and googling hasn't helped.&#xA;the issue arises when I try to run this module of the algorithm&#xA;&lt;a href=&quot;https://ciri-cookbook.readthedocs.io/en/latest/CIRI-full.html#the-merge-module&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ciri-cookbook.readthedocs.io/en/latest/CIRI-full.html#the-merge-module&lt;/a&gt;&#xA;as I try to run the module I get the following output and error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Merge module start&#xA;&#xA;Loading Annotation...&#xA;Annotation Loaded&#xA;Loading CIRI_AS output...&#xA;CIRI_AS output Loaded&#xA;Loading CIRI output...&#xA;CIRI output Loaded&#xA;Loading CIRI_RO output...&#xA;CIRI_RO output Loaded&#xA;Combine AS and RO output&#xA;Combine completed, 71 reads are used&#xA;Exception in thread &amp;quot;main&amp;quot; java.lang.reflect.InvocationTargetException&#xA;        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)&#xA;        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)&#xA;        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)&#xA;        at java.lang.reflect.Method.invoke(Method.java:483)&#xA;        at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)&#xA;Caused by: java.lang.NullPointerException&#xA;        at Exon_gtf.get_exon(Exon_gtf.java:16)&#xA;        at Merge3.getmerge(Merge3.java:784)&#xA;        at CIRI_Full2.main(CIRI_Full2.java:551)&#xA;        ... 5 more&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;All prior steps preceding this one seem to have run properly and I was able to generate all the files to run the Merge module.&#xA;any help or explanation as to what the issue could be would be greatly appreciated. As I mentioned before I have no experience in java.&#xA;kind regards&lt;/p&gt;&#xA;" OwnerUserId="14100" LastActivityDate="2023-01-11T20:49:46.923" Title="java error when running CIRI-full.jar Merge module" Tags="|rna-seq|java|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17647" PostTypeId="1" AcceptedAnswerId="17653" CreationDate="2021-09-02T12:36:01.293" Score="1" ViewCount="2107" Body="&lt;p&gt;Is there a way to generate a BED file of a list of gene symbols for a given genome?&lt;/p&gt;&#xA;" OwnerUserId="10386" LastActivityDate="2021-09-03T08:45:15.253" Title="Create BED file from list with gene symbols" Tags="|fasta|genome|bed|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17648" PostTypeId="1" CreationDate="2021-09-02T13:19:36.777" Score="2" ViewCount="31" Body="&lt;p&gt;Parts of sequences are given below-&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Reference sequence (pre-alignment):&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATTAAAGGTTTATACCTTCCCAGGTAACAAACCAACCAACTTTCGATCTCTTGTAGATCTGTTCTCTAAACGAACTTTAAAATCTGTGT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Reference sequence (post-alignment)&lt;/strong&gt; and below it is &lt;strong&gt;Sample sequence (post-alignment)&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;--------------------------------------------------------------------------------------------------attaaa---------ggtt------------------tataccttc---------ccaggtaacaaa-------------ccaacc-----aactttcgatctcttgtagatctgttctctaaacgaactttaaaatctgtgt&#xA;--------------------------------------------------------------------------------------------------------------------------------------------------------------taacaaa-------------ccaacc-----aactttcgatctcttgtagatctgttctctaaacgaactttaaaatctgtgt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm adding a simpler to interpret example-&lt;/p&gt;&#xA;&lt;p&gt;Say the ref seq is &lt;code&gt;aattaaatttgggggtttt&lt;/code&gt; and the sample seq is &lt;code&gt;ttaaggggttaaatttgggggt--t&lt;/code&gt;. Then post-alignment, they will be like-&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;--aa----ttaaatttgggggtttt &#xA;ttaaggggttaaatttgggggt--t&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, since my ref seq was&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;0    1    2    3    4    5    6    7    8    9    10   11   12   13   14   15   16   17   18&#xA;a    a    t    t    a    a    a    t    t    t    g    g    g    g    g    t    t    t    t&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want that post-alignment also, the indexing should be conserved-&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;           0    1                        2    3    4    5    6    7    8    9    10   11   12   13   14   15   16   17   18&#xA; -    -    a    a    -    -    -    -    t    t    a    a    a    t    t    t    g    g    g    g    g    t    t    t    t&#xA; t    t    a    a    g    g    g    g    t    t    a    a    a    t    t    t    g    g    g    g    g    t    -    -    t &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to keep the indexing of the reference sequence conserved(i.e. the first base in ref seq post-alignment is &lt;code&gt;a&lt;/code&gt;, second is &lt;code&gt;t&lt;/code&gt;, third is &lt;code&gt;t&lt;/code&gt;, etc.), like they do in standard softwares, and then I want to run some quick analysis on it, say to check for conservation of a (mono/di)-nucleotide at some positions. If anybody has some insight on how to do it the most efficient way(memory-wise and time-wise), then that'd be great. I use &lt;em&gt;Python&lt;/em&gt; for my work.&lt;/p&gt;&#xA;" OwnerUserId="2883" LastEditorUserId="2883" LastEditDate="2021-09-02T16:43:32.930" LastActivityDate="2021-09-02T16:43:32.930" Title="Alignment with inserts and keeping the indexing of ref seq intact" Tags="|sequence-alignment|sequence-analysis|mutations|multiple-sequence-alignment|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17649" PostTypeId="1" CreationDate="2021-09-02T14:46:37.550" Score="2" ViewCount="363" Body="&lt;p&gt;I've got an annotated draft genome assembly made using short and long read strategies. I've also done optical mapping to stitch some of the contigs together, with the goal being chromsome-length assemblies. How do I annotate the output .fasta from the optical mapping assembly using the .gff of the draft assembly that was used as input? I also have a .agp file that describes what sequence fragments ended up where in the output.&lt;/p&gt;&#xA;" OwnerUserId="9810" LastActivityDate="2021-09-02T23:23:34.543" Title="Transfer annotations from one genome assembly to another" Tags="|fasta|sequence-annotation|genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17650" PostTypeId="1" CreationDate="2021-09-02T17:11:27.973" Score="2" ViewCount="1351" Body="&lt;p&gt;I am trying to model how a specific point mutation would affect the protein structure.  So far, I've figured out how to create the point mutation in PyMOL using the mutagenesis wizard.  I'm not sure how to take the next step toward actually visualizing the consequences of the mutation though (i.e. would it disrupt a helix, etc).  Is there a way to do this in PyMOL, or would I need other software?  Thank you!&lt;/p&gt;&#xA;" OwnerUserId="14105" LastActivityDate="2022-02-16T15:06:40.693" Title="How to see effect of point mutation in PyMOL?" Tags="|pymol|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17651" PostTypeId="2" ParentId="17650" CreationDate="2021-09-02T20:21:52.703" Score="1" Body="&lt;p&gt;You could run a molecular dynamics simulation for example with Gromacs or Charm. The problem with pymol is that the structure is static and you don't know how much tension that point mutation creates. First of all you would run an energy minimization to remove bad non-physical overlaps between atoms, then add solvent molecules, and then simulate your system first with position restraints for equilibration and then without for 1ns to 100ns+.&lt;/p&gt;&#xA;&lt;p&gt;Even then you cannot be sure that what comes out is the 'final' result as sometimes an induced fold change can take much longer. It depends on the system and the energy barriers included and the time scales that you are interested in. If the protein is large it may have many different physiologically relevant conformations and the point mutation might only change the equilibrium distribution of those states or totally prevent a transition.&lt;/p&gt;&#xA;&lt;p&gt;In case you have more than one crystal structure (in different conformations) you may introduce these point mutations in all of them and simulate them for a while. But it is not that trivial to setup such a simulation you should contact an expert. There are many things that can go wrong. Other software you could use are Modeller and Rosetta, if I am not mistaken.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2021-09-02T20:21:52.703" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17652" PostTypeId="2" ParentId="17649" CreationDate="2021-09-02T23:23:34.543" Score="2" Body="&lt;p&gt;You should use a liftover approach. You could either do this based on exact changes, which might be a bit tedious given the presumably large number of join operations you've done (especially if you've e.g. broken contigs).&lt;/p&gt;&#xA;&lt;p&gt;Therefore it might be more convenient to do it on the basis of homology.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/agshumate/Liftoff&quot; rel=&quot;nofollow noreferrer&quot;&gt;liftoff&lt;/a&gt; is one simple tool for doing this with a gff file and your 2 fasta files. It seems to mostly work ok.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-09-02T23:23:34.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17653" PostTypeId="2" ParentId="17647" CreationDate="2021-09-02T23:51:18.683" Score="4" Body="&lt;p&gt;Via Gencode and BEDOPS &lt;code&gt;convert2bed&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ wget -qO- ftp://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_28/gencode.v28.annotation.gff3.gz \&#xA;    | gunzip --stdout - \&#xA;    | awk '$&lt;/span&gt;3 == &amp;quot;gene&amp;quot;' - \&#xA;    | convert2bed -i gff - \&#xA;    &amp;gt; genes.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Replace the FTP path with that needed for your genome of interest. Feel free to edit your question if you need help with that, by specifying your given genome.&lt;/p&gt;&#xA;&lt;p&gt;You can modify the &lt;code&gt;awk&lt;/code&gt; statement to get exons, by replacing &lt;code&gt;gene&lt;/code&gt; with &lt;code&gt;exon&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;BEDOPS: &lt;a href=&quot;https://github.com/bedops/bedops&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/bedops/bedops&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Once you have the genes in BED format, you can &lt;code&gt;grep&lt;/code&gt; the file for them, using a text file of matches:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -Fwf genes_of_interest.txt genes.bed &amp;gt; genes_of_interest.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2021-09-03T03:24:09.560" LastActivityDate="2021-09-03T03:24:09.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17654" PostTypeId="2" ParentId="17647" CreationDate="2021-09-03T08:45:15.253" Score="3" Body="&lt;p&gt;You could also get your genes from &lt;a href=&quot;https://genome-euro.ucsc.edu/cgi-bin/hgTables&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC&lt;/a&gt;. Here's how:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hxe6O.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hxe6O.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2021-09-03T08:45:15.253" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17655" PostTypeId="2" ParentId="17650" CreationDate="2021-09-03T10:00:33.683" Score="3" Body="&lt;p&gt;A common metric used to assess if a mutation destabilises the structure is the difference in Gibbs free energy. PyMOL does not give a ∆∆G (difference in Gibbs free energy, kcal/mol), but a very crude steric clash nor does it repack neighbouring sidechains out of the way, or alter the backbone, so unfortunately is not the way forward. However, if one must there is a way (see SIMBAI below).&lt;/p&gt;&#xA;&lt;h2&gt;Beyond destabilisation&lt;/h2&gt;&#xA;&lt;p&gt;Destabilisation predictors have a ~60% or lower success at predicting if something is pathogenic (&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/32958805/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ref&lt;/a&gt;). This is because the effect a mutation has on a protein is rather complicated as you can have destabilisation, but also aggregation, loss of regulation, mislocalisation, loss of binding to another protein, loss of binding to ligand, loss of activity (catalysis, inability to change conformation) etc etc. Then you can get curious subcases, such as loss of activity that result in the sequestration of a binding partner or homologues in a complex resulting in a worse phenotype than destabilisation or truncation.&lt;/p&gt;&#xA;&lt;h2&gt;Modelling&lt;/h2&gt;&#xA;&lt;p&gt;The behaviour of a protein can be assess by the energy of the system using &lt;a href=&quot;https://en.wikipedia.org/wiki/Force_field_(chemistry)&quot; rel=&quot;nofollow noreferrer&quot;&gt;forcefields&lt;/a&gt;, which are multi-term equations that describes the atomic interactions (attraction, repulsion etc.). The result is the Gibbs free energy mentioned above, which is a potential, so negative is good. Kinetic energy is release when the unfolded protein &amp;quot;rolls down&amp;quot; down the potential energy funnel —&lt;a href=&quot;https://academic.oup.com/nar/article/49/D1/D420/5983626&quot; rel=&quot;nofollow noreferrer&quot;&gt;ProTherm dataset&lt;/a&gt; (an empirical dataset of ∆∆Gs) and the &lt;a href=&quot;http://marid.bioc.cam.ac.uk/sdm2&quot; rel=&quot;nofollow noreferrer&quot;&gt;tool SDM&lt;/a&gt; give this value inverted as the negative can be confusing.&lt;/p&gt;&#xA;&lt;p&gt;An uphill change in the Gibbs free energy (∆∆G &amp;gt; 0) generally correlates with a destabilisation of a protein —aggregation and solubility being the caveat in the &amp;quot;generally&amp;quot;. The destabilisation a single state (static conformation) can be assessed. &lt;code&gt;gmx energy&lt;/code&gt; in Gromacs, FoldX or Rosetta can be used for this.&lt;/p&gt;&#xA;&lt;p&gt;For altered ability to switch conformation steered MD simulations are required, but generally the ∆∆G at different states can be used.&lt;/p&gt;&#xA;&lt;p&gt;For the non-destabilisation options, &lt;a href=&quot;https://www.phosphosite.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.phosphosite.org/&lt;/a&gt; and &lt;a href=&quot;http://elm.eu.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://elm.eu.org/&lt;/a&gt; are very useful.&lt;/p&gt;&#xA;&lt;p&gt;I have been working on a server that gives a ∆∆G value and note if there are any nearby post-translational modifications, gnomAD or changes in potential linear motifs etc.: &lt;a href=&quot;https://venus.sgc.ox.ac.uk/venus&quot; rel=&quot;nofollow noreferrer&quot;&gt;Venus&lt;/a&gt; (unpublished). However, there are many ∆∆G calculation servers out there and are compared in many papers (&lt;a href=&quot;https://pubs.acs.org/doi/10.1021/acs.jcim.0c00591&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt;), but it should be said that the dataset used in these comparisons is always ultra-biased despite how much is tried. &lt;a href=&quot;http://missense3d.bc.ic.ac.uk/missense3d/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Missense&lt;/a&gt; by the Phyre2 group (Michael Sternberg) give detailed information of the neighbourhood changes the mutation causes.&#xA;In terms of extra info, &lt;a href=&quot;http://miscast.broadinstitute.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Miscast&lt;/a&gt; gives Uniprot info.&lt;/p&gt;&#xA;&lt;h2&gt;SIMBA-I via PyMOL&lt;/h2&gt;&#xA;&lt;p&gt;In terms of PyMOL, the &lt;a href=&quot;https://pubs.acs.org/doi/abs/10.1021/acs.jcim.1c00201&quot; rel=&quot;nofollow noreferrer&quot;&gt;SIMBA-I algorithm&lt;/a&gt; can be used by getting the RSA from it —I would not recommend it as it may underestimate the severity of unusual cases. But if one must, in PyMOL one can paste the following into the console with the wild type variant:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# defined by you so change accordingly&#xA;from_residue = '' # single letter&#xA;to_residue = '' # single letter&#xA;resi = &#xA;chain = ''&#xA;    &#xA;## data&#xA;maxASA = {'A': 121.0, 'R': 265.0, 'N': 187.0, 'D': 187.0, 'C': 148.0, 'E': 214.0, 'Q': 214.0, 'G': 97.0, 'H': 216.0,\&#xA;          'I': 195.0, 'L': 191.0, 'K': 230.0, 'M': 203.0, 'F': 228.0, 'P': 154.0, 'S': 143.0, 'T': 163.0,\&#xA;              'W': 264.0, 'Y': 255.0, 'V': 165.0}&#xA;volumes = {'A': 88.6, 'R': 173.4, 'N': 114.1, 'D': 111.1, 'C': 108.5, 'Q': 143.8, 'E': 138.4, 'G': 60.1,\&#xA;               'H': 153.2, 'I': 166.7, 'L': 166.7, 'K': 168.6, 'M': 162.9, 'F': 189.9, 'P': 112.7, 'S': 89.0,\&#xA;               'T': 116.1, 'W': 227.8, 'Y': 193.6, 'V': 140.0}&#xA;hydrophobicities = {'A': 8.1, 'R': 10.5, 'N': 11.6, 'D': 13.0, 'C': 5.5, 'Q': 10.5, 'E': 12.3, 'G': 9.0,\&#xA;                    'H': 10.4, 'I': 5.2, 'L': 4.9, 'K': 11.3, 'M': 5.7, 'F': 5.2, 'P': 8.0, 'S': 9.2, 'T': 8.6,\&#xA;                    'W': 5.4, 'Y': 6.2, 'V': 5.9}&#xA;cmd.set('dot_solvent', 1)&#xA;cmd.set('dot_density', 3)&#xA;sasa = cmd.get_area(f'resi {resi} and chain {chain}')&#xA;rsa = sasa / maxASA[from_residue]&#xA;offset = -1.6&#xA;rsa_factor = +2.2&#xA;dV_factor = +0.59&#xA;dH_factor = -0.29&#xA;rsa_dV_factor = -0.54&#xA;rsa_dH_factor = +0.49&#xA;# calculate&#xA;dV = (volumes[to_residue] - volumes[from_residue]) / 100&#xA;dH = (hydrophobicities[to_residue] - hydrophobicities[from_residue])&#xA;simba = offset + rsa_factor * rsa + dV_factor * dV + dH_factor * dH + rsa_dV_factor * rsa * dV + rsa_dH_factor * rsa * dH&#xA;print -simba  # negative stabilised, positive destabilised&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Human reasoning&lt;/h2&gt;&#xA;&lt;p&gt;However, the major problem with all the tools mentioned is that human reasoning is often required to read the literature and make a solid hypothesis.&lt;/p&gt;&#xA;&lt;p&gt;A resource I cannot stress enough, for human mutations, its utility is &lt;a href=&quot;https://gnomad.broadinstitute.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gnomAD&lt;/a&gt;, which contains &lt;em&gt;phenotypically neutral&lt;/em&gt; mutations: for example, one really ought to not claim a homozygous missense is pathogenic because it is causing a loss of function by destabilising the protein, yet there are homozygous truncations roaming around in the human population.&lt;/p&gt;&#xA;&lt;p&gt;The paper describing the crystal structure used and those of the homologues may point out key features which would missed by automated servers, such as secondary-structure elements that are speculated to perform certain roles.&#xA;In fact, without doing any calculations seeing in PyMOL if a mutation causes a clash in the core of a protein, alters the active site, or changes a residue to a smaller one in a region known to alter conformation upon a given condition, is a lot more informative even if not quantitative...&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2021-09-19T14:11:34.780" LastActivityDate="2021-09-19T14:11:34.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17656" PostTypeId="1" CreationDate="2021-09-03T15:55:21.890" Score="2" ViewCount="172" Body="&lt;p&gt;I am predicting genes using &lt;code&gt;GeneMark-ES&lt;/code&gt;, I managed to generate a &lt;code&gt;gtf&lt;/code&gt; file, but when I tried to extract the nt/protein sequence using &lt;code&gt;get_sequence_from_GTF.pl&lt;/code&gt;, I am running into an error in the format of the gtf file, which I find puzzing as it was the gtf file was generated using &lt;code&gt;gmes_petap.pl&lt;/code&gt; (an script in the &lt;code&gt;GeneMark-ES&lt;/code&gt; package).&lt;/p&gt;&#xA;&lt;p&gt;Here is the command I used and the error message which also prints the first line of the genemark.gtf file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ get_sequence_from_GTF.pl genemark.gtf DT_CAJQTU010000037_1.fasta&#xA;&#xA;error in file format GTF: CAJQTU010000037.1 Desoria tigrina genome assembly, contig: utg000040l, whole genome shotgun sequence     GeneMark.hmm3   gene    2099    4039    .       +       .       gene_id &amp;quot;1_g&amp;quot;; transcript_id &amp;quot;1_t&amp;quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I think I have found the line in &lt;code&gt;get_sequence_from_GTF.pl&lt;/code&gt; that is causing the error, but not being a perl monk, I am having troubles to read figure out what is wrong with my &lt;code&gt;gtf&lt;/code&gt; file and more importantly figure out how to fix it.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;if ( /^(\S+)\t\S+\t(\S+)\t(\d+)\t(\d+)\t.\t([\+-])\t([012.])\tgene_id \&amp;quot;(\S+)\&amp;quot;/ )&#xA;    {&#xA;    ... (some stuff)&#xA;    }&#xA;else&#xA;{&#xA;  if ( /^\s+&lt;span class=&quot;math-container&quot;&gt;$/ ) { next };&#xA;  if ( /^#/ ) { next };&#xA;  die( &quot;error in file format GTF: $&lt;/span&gt;_\n&amp;quot; );&#xA;}&#xA;... &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14110" LastActivityDate="2022-03-22T14:16:07.557" Title="GTF file format causing GeneMark-ES error" Tags="|gtf|" AnswerCount="2" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17657" PostTypeId="1" AcceptedAnswerId="17658" CreationDate="2021-09-04T19:10:23.783" Score="2" ViewCount="1622" Body="&lt;p&gt;I have written a Nextflow script with three process:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The first process takes a pair of &lt;code&gt;fastq&lt;/code&gt; files and aligns with reference genome. The process writes the resulting &lt;code&gt;SAM&lt;/code&gt; file into &lt;code&gt;sam channel&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Second process takes input from the &lt;code&gt;sam channel&lt;/code&gt; and creates a &lt;code&gt;BAM&lt;/code&gt; file from it, and writes it into &lt;code&gt;bam channel&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Final process reads from &lt;code&gt;bam channel&lt;/code&gt; and sorts the &lt;code&gt;BAM&lt;/code&gt; files.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Following is the entire code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!~/bin nextflow&#xA;&#xA;reads_ch = Channel.fromFilePairs('raw/*_{1,2}.fastq.gz', flat: true)&#xA;ref_genome = file('reference/genome/index/ref_genome')&#xA;&#xA;process mapToRef {&#xA;    memory '20 GB'&#xA;    cpus 16&#xA;    input:&#xA;    tuple val(sample_id), file(read1), file(read2) from reads_ch&#xA;    &#xA;    output:&#xA;    tuple val(sample_id), file(&amp;quot;${sample_id}.sam&amp;quot;) into sam_ch&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    ~/biotools/bwa/bwa mem -t 60 &lt;span class=&quot;math-container&quot;&gt;${ref_genome} $&lt;/span&gt;{read1} &lt;span class=&quot;math-container&quot;&gt;${read2} &amp;gt; $&lt;/span&gt;{sample_id}.sam&#xA;    &amp;quot;&amp;quot;&amp;quot;   &#xA;}&#xA;&#xA;process samToBam {&#xA;    memory '20 GB'&#xA;    cpus 16&#xA;    input:&#xA;    tuple val(sample_id), file(&amp;quot;${sample_id}.sam&amp;quot;) from sam_ch&#xA;&#xA;    output:&#xA;    tuple val(sample_id), file(&amp;quot;${sample_id}.bam&amp;quot;) into bam_ch&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    ~/biotools/SAMTOOLS/samtools-1.12/samtools view -S -b &lt;span class=&quot;math-container&quot;&gt;${sample_id}.sam &amp;gt; $&lt;/span&gt;{sample_id}.bam &#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process sortBam {&#xA;    memory '20 GB'&#xA;    cpus 16&#xA;    input:&#xA;    tuple val(sample_id), file(&amp;quot;${sample_id}.bam&amp;quot;) from bam_ch&#xA;&#xA;    output:&#xA;    tuple val(sample_id), file(&amp;quot;${sample_id}.sorted.bam&amp;quot;) into sorted_bam_ch&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    ~/biotools/SAMTOOLS/samtools-1.12/samtools sort &lt;span class=&quot;math-container&quot;&gt;${sample_id}.bam -o $&lt;/span&gt;{sample_id}.sorted.bam&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;All three processes successfully execute their commands. However, they are executed sequentially. For example, the first process &lt;code&gt;mapToRef&lt;/code&gt; runs &lt;strong&gt;sequentially&lt;/strong&gt; for all read pairs, followed by &lt;code&gt;samToBam&lt;/code&gt;, which too, runs sequentially after all &lt;code&gt;mapToRef&lt;/code&gt;s have finished execution. I was under impression that parallelization would &amp;quot;work&amp;quot; out of the box, but I must be doing something wrong. I am a complete beginner in Nextflow and would greatly appreciate any help regarding this.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Output of &lt;code&gt;reads_ch.view()&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[ERR2512393, /home/work/raw/ERR2512393_1.fastq.gz, /home/work/raw/ERR2512393_2.fastq.gz]&#xA;[ERR2512394, /home/work/raw/ERR2512394_1.fastq.gz, /home/work/raw/ERR2512394_2.fastq.gz]&#xA;[ERR2512391, /home/work/raw/ERR2512391_1.fastq.gz, /home/work/raw/ERR2512391_2.fastq.gz]&#xA;[ERR2512392, /home/work/raw/ERR2512392_1.fastq.gz, /home/work/raw/ERR2512392_2.fastq.gz]&#xA;[ERR2512390, /home/work/raw/ERR2512390_1.fastq.gz, /home/work/raw/ERR2512390_2.fastq.gz]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12217" LastEditorUserId="4413" LastEditDate="2021-09-04T20:11:34.950" LastActivityDate="2022-02-12T13:48:15.850" Title="Why are my Nextflow processes not executing in parallel?" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17658" PostTypeId="2" ParentId="17657" CreationDate="2021-09-04T19:42:41.057" Score="4" Body="&lt;p&gt;I have tested it on my end with the following script:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-java prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env nextflow&#xA;&#xA;reads_ch = Channel.fromFilePairs('/root/test_nextflow/*_{1,2}.fastq.gz', flat: true)&#xA;&#xA;&#xA;process mapToRef {&#xA;  memory '1 GB'&#xA;  echo true&#xA;  cpus 1&#xA;&#xA;  input:&#xA;  tuple val(sample_id), file(r1), file(r2) from reads_ch&#xA;&#xA;  output:&#xA;  tuple val(sample_id), file(&amp;quot;${sample_id}.sam&amp;quot;) into sam_ch&#xA;&#xA;  script:&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  echo bwa mem &lt;span class=&quot;math-container&quot;&gt;${r1} {r2}&#xA;  if [ $&lt;/span&gt;{sample_id} == &amp;quot;ERR2512393&amp;quot; ]; then sleep 10; fi&#xA;  touch ${sample_id}.sam&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process samToBam {&#xA;  memory '1 GB'&#xA;  cpus 1&#xA;  echo true&#xA;&#xA;  input:&#xA;  tuple val(sample_id), file(sam) from sam_ch&#xA;&#xA;  output:&#xA;  tuple val(sample_id), file(&amp;quot;${sample_id}.bam&amp;quot;) into bam_ch&#xA;&#xA;  script:&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  echo samtools view -S -b &lt;span class=&quot;math-container&quot;&gt;${sample_id}.sam&#xA;  touch $&lt;/span&gt;{sample_id}.bam&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process sortBam {&#xA;  memory '1 GB'&#xA;  cpus 1&#xA;  input:&#xA;  tuple val(sample_id), file(bam) from bam_ch&#xA;&#xA;  output:&#xA;  tuple val(sample_id), file(&amp;quot;${sample_id}.sorted.bam&amp;quot;) into sorted_bam_ch&#xA;&#xA;  script:&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  echo samtools sort &lt;span class=&quot;math-container&quot;&gt;${sample_id}.bam&#xA;  touch $&lt;/span&gt;{sample_id}.sorted.bam&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It's producing the following result:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/w9Ng8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/w9Ng8.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you can see, the two processes are running in parallel (&lt;code&gt;2 of 2&lt;/code&gt;)&lt;/p&gt;&#xA;&lt;p&gt;It could be that yours are limited by the resources? If your PC doesn't have 40GB of memory and 32 CPUs, it will run them sequentially.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: By adding &lt;code&gt;sleep 10&lt;/code&gt; to &lt;code&gt;mapToRef&lt;/code&gt;, we can clearly see both of them are running in parallel now:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VaMTj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VaMTj.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT 2&lt;/strong&gt;: Added condition to &amp;quot;sleep&amp;quot; on one of the samples:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YwBmX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YwBmX.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Final EDIT&lt;/strong&gt;: be mindful of the resources you allocate to your processes. You could use &lt;code&gt;${task.cpus}&lt;/code&gt; and &lt;code&gt;${task.memory.toGiga()}&lt;/code&gt; to facilitate the configuration of your processes so that they are not different from what you have defined and what is actually used. See &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#cpus&quot; rel=&quot;nofollow noreferrer&quot;&gt;CPUs&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="12644" LastEditDate="2022-02-12T13:48:15.850" LastActivityDate="2022-02-12T13:48:15.850" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="17660" PostTypeId="1" CreationDate="2021-09-05T20:38:25.330" Score="-1" ViewCount="49" Body="&lt;p&gt;Transcription factors regulate the genes in different cells. That is, different cells have different sequence of genes that are transcribed (set on or off) by the RNA polymerase. This will help different types of cells to perform their own unique operation.&lt;/p&gt;&#xA;&lt;p&gt;So, why do the cells have to look different physically, if the their function is determined by the on/off sequence?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;NOTE:&lt;/strong&gt; I am a neophyte in Bioinformatics. Any help on my building my conceptual knowledge is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="12425" LastEditorUserId="4413" LastEditDate="2021-09-09T15:45:01.267" LastActivityDate="2021-09-09T15:45:01.267" Title="If regulatory proteins decide the gene regulation, is it necessary for every cell to have a different look?" Tags="|transcription-factors|" AnswerCount="1" CommentCount="1" ClosedDate="2021-09-21T23:34:31.163" ContentLicense="CC BY-SA 4.0" />
  <row Id="17661" PostTypeId="1" CreationDate="2021-09-06T05:29:45.277" Score="-1" ViewCount="29" Body="&lt;p&gt;I read the following line in the research paper - &lt;a href=&quot;http://cole-trapnell-lab.github.io/pdfs/papers/trapnell-cacchiarelli-monocle.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The dynamics and regulators of cell fate decisions are revealed by pseudotemporal ordering of single cells&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;During differentiation, for example, each cell makes independent fate decisions by integrating a wide array of signals from other cells and executing a complex choreography of gene-regulatory changes. Thus individual cells can execute&#xA;the same sequence of transcriptional changes over highly varying&#xA;time scales.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Can anyone explain in simple terms what is meant by &lt;strong&gt;&amp;quot;same sequence of transcriptional changes&amp;quot;&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;p&gt;How can it be the same sequence if during differentiation, different types of cells are formed?&lt;/p&gt;&#xA;" OwnerUserId="12425" LastActivityDate="2021-09-06T09:05:24.780" Title="What is meant by transcriptional changes executed by the cell over a time period?" Tags="|gene|single-cell|phylogenetics|rna|transcriptome-regulation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17662" PostTypeId="2" ParentId="17650" CreationDate="2021-09-06T08:58:04.300" Score="1" Body="&lt;p&gt;You could give &lt;a href=&quot;http://www.bioinf.org.uk/mutations/saapdap/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAAPpred&lt;/a&gt; a try.&#xA;This tool uses machine learning (in the form of &lt;code&gt;random forest&lt;/code&gt;) to predict the &lt;strong&gt;local&lt;/strong&gt; effects of a single point mutation on the protein structure.&lt;/p&gt;&#xA;&lt;p&gt;I did my PhD in the group and I used SAAPpred to find the most (predicted) damaging mutations of G6PD, to then ran extensive MD simulations to evaluate the final stability of those mutants.&lt;/p&gt;&#xA;&lt;p&gt;Keep in mind that the tools doesn't return a structure, but just the possible/likely effects of that substitution.&lt;/p&gt;&#xA;" OwnerUserId="3631" LastActivityDate="2021-09-06T08:58:04.300" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17663" PostTypeId="2" ParentId="17661" CreationDate="2021-09-06T09:05:24.780" Score="1" Body="&lt;p&gt;Not all cells will perform the same sequence of transcriptional changes. The paragraph you quote simply states that differentiation isn't a step change with all cells differentiating at the exact same moment. Rather, it's a more gradual change with cells each working on their own time scale.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-09-06T09:05:24.780" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17665" PostTypeId="1" CreationDate="2021-09-06T10:59:55.533" Score="0" ViewCount="41" Body="&lt;p&gt;I am looking into different methods to test the long-branch attraction problem. In &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/B9780444536327011084&quot; rel=&quot;nofollow noreferrer&quot;&gt;this chapter&lt;/a&gt; they talk about the SAW method - excluding one and the other taxa from the dataset and see if they change their position within the tree. From the description of the method, I was not sure, if the bunch of sequences with excluded samples should be realigned, or if I should simply exclude the rows in the alignment.&lt;/p&gt;&#xA;&lt;p&gt;I tried to figure out it by looking at the &lt;a href=&quot;https://onlinelibrary.wiley.com/doi/10.1111/j.1096-0031.1999.tb00391.x&quot; rel=&quot;nofollow noreferrer&quot;&gt;Siddall and Whiting 1995&lt;/a&gt; paper, which was referred to as the original description of this method in the chapter mentioned above. The best I found is&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Using their alignment (&lt;a href=&quot;http://biology.unm.edu/esloker/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://biology.unm.edu/esloker/&lt;/a&gt;), when Polypodium is excluded, the most parsimonious tree groups the myxozoans as sister to the Bilateria (Fig. 1a).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;which makes me think that we should just exclude the taxa from the final alignment, but as there is still a space for interpretation, I thought I'll ask.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2021-09-06T10:59:55.533" Title="Does the alignment need to be redone when using SAW method to test long branch attraction?" Tags="|sequence-alignment|phylogenetics|phylogeny|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17666" PostTypeId="1" AcceptedAnswerId="17668" CreationDate="2021-09-06T13:36:17.693" Score="3" ViewCount="1206" Body="&lt;p&gt;I have a GTF file (in 'test.gtf' file):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1    HAVANA  gene    10069   18412   .   +   .   gene_id &amp;quot;ENSG00000223972.4&amp;quot;; transcript_id &amp;quot;ENSG00000223972.4&amp;quot;; gene_type &amp;quot;pseudogene&amp;quot;; gene_status &amp;quot;KNOWN&amp;quot;; gene_name &amp;quot;DDX11L1&amp;quot;; transcript_type &amp;quot;pseudogene&amp;quot;; transcript_status &amp;quot;KNOWN&amp;quot;; transcript_name &amp;quot;DDX11L1&amp;quot;; level 2; havana_gene &amp;quot;OTTHUMG00000000961.2&amp;quot;;&#xA;chr2    HAVANA  gene    11869   14412   .   +   .   gene_id &amp;quot;ENSG0000023172.1&amp;quot;; transcript_id &amp;quot;ENSG000023172.1&amp;quot;; gene_type &amp;quot;pseudogene&amp;quot;; gene_status &amp;quot;KNOWN&amp;quot;; gene_name &amp;quot;PCNA&amp;quot;; transcript_type &amp;quot;pseudogene&amp;quot;; transcript_status &amp;quot;KNOWN&amp;quot;; transcript_name &amp;quot;PCNA&amp;quot;; level 2; havana_gene &amp;quot;OTTHUMG00000001931.2&amp;quot;;&#xA;chr3    HAVANA  gene    15769   19412   .   +   .   gene_id &amp;quot;ENSG00000433972.1&amp;quot;; transcript_id &amp;quot;ENSG000023172.1&amp;quot;; gene_type &amp;quot;pseudogene&amp;quot;; gene_status &amp;quot;KNOWN&amp;quot;; gene_name &amp;quot;USP21&amp;quot;; transcript_type &amp;quot;pseudogene&amp;quot;; transcript_status &amp;quot;KNOWN&amp;quot;; transcript_name &amp;quot;USP21&amp;quot;; level 2; havana_gene &amp;quot;OTTHUMG00000002361.2&amp;quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have loaded it as a pyranges object and I want to subset it so that it only contains gene from &lt;code&gt;test_list&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pyranges as pr&#xA;import pandas as pd&#xA;&#xA;gtf_file = &amp;quot;test.gtf&amp;quot;&#xA;test_list = [&amp;quot;PCNA&amp;quot;,&amp;quot;USP21&amp;quot;, &amp;quot;USP1&amp;quot;]&#xA;&#xA;gtf = pr.read_gtf(gtf_file)&#xA;&#xA;sub_gtf = gtf&#xA;&#xA;sub_gtf.df = gtf.df[gtf.df.gene_name.isin(test_list)]&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But the last line throws this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;File &amp;quot;&amp;lt;ipython-input-24-af0b4b5c7576&amp;gt;&amp;quot;, line 1, in &amp;lt;module&amp;gt;&#xA;    irongtf.df = irongtf.df[irongtf.df.gene_name.isin(test_list)]&#xA;&#xA;  File &amp;quot;/home/niek/.local/lib/python3.8/site-packages/pyranges/pyranges.py&amp;quot;, line 314, in __setattr__&#xA;    _setattr(self, column_name, column)&#xA;&#xA;  File &amp;quot;/home/niek/.local/lib/python3.8/site-packages/pyranges/methods/attr.py&amp;quot;, line 48, in _setattr&#xA;    df.insert(pos, column_name, column)&#xA;&#xA;  File &amp;quot;/home/niek/.local/lib/python3.8/site-packages/pandas/core/frame.py&amp;quot;, line 4413, in insert&#xA;    value = self._sanitize_column(value)&#xA;&#xA;  File &amp;quot;/home/niek/.local/lib/python3.8/site-packages/pandas/core/frame.py&amp;quot;, line 4504, in _sanitize_column&#xA;    com.require_length_match(value, self.index)&#xA;&#xA;  File &amp;quot;/home/niek/.local/lib/python3.8/site-packages/pandas/core/common.py&amp;quot;, line 531, in require_length_match&#xA;    raise ValueError(&#xA;&#xA;ValueError: Length of values (2) does not match length of index (1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What would be the right way to subset the gtf file so that it only contains the rows where &lt;code&gt;gene_name&lt;/code&gt; contains the gene symbols from &lt;code&gt;test_list&lt;/code&gt;?&#xA;Or is there another tool that I could use to do this?&lt;/p&gt;&#xA;" OwnerUserId="10386" LastEditorUserId="10386" LastEditDate="2021-09-07T06:36:43.273" LastActivityDate="2021-09-07T18:26:26.147" Title="Subset GTF file for specific genes" Tags="|python|gtf|pyranges|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17667" PostTypeId="2" ParentId="17660" CreationDate="2021-09-07T07:15:38.987" Score="2" Body="&lt;p&gt;There are many more contributors to the cell differentiation that the regulators. But one that you will encounter at an higher level is the chromatin status: if it is in the open status it allows transcripts to be on and being regulated (with all the further levels of regulations such as methylation, glycosylation etc). If the chromatin is off (the state is very condensed) very few transcript can be made from that region.&#xA;Check this wikipedia page &lt;a href=&quot;https://en.wikipedia.org/wiki/Heterochromatin&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="11884" LastEditorDisplayName="user3051" LastEditDate="2021-09-08T09:00:21.870" LastActivityDate="2021-09-08T09:00:21.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17668" PostTypeId="2" ParentId="17666" CreationDate="2021-09-07T09:21:45.087" Score="1" Body="&lt;p&gt;It turned out to be a lot easier if I used &lt;code&gt;grep&lt;/code&gt; to do this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -w -f genes.txt gencode.v19.annotation.gtf &amp;gt; sub_set.gtf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;genes.txt&lt;/code&gt; contains gene symbols on new lines.&lt;/p&gt;&#xA;" OwnerUserId="10386" LastActivityDate="2021-09-07T09:21:45.087" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="17669" PostTypeId="2" ParentId="17666" CreationDate="2021-09-07T14:21:54.620" Score="2" Body="&lt;p&gt;If you have your gtf file like a DataFrame you can use:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; df[df['gene_name'].str.contains('|'.join(test_list))]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2021-09-07T14:21:54.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17670" PostTypeId="2" ParentId="17666" CreationDate="2021-09-07T18:26:26.147" Score="3" Body="&lt;p&gt;Following up on zorbax's answer, you could read in and filter the GTF file in this way, among others:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;&#xA;import gtfparse as gp&#xA;&#xA;gtf_file = &amp;quot;test.gtf&amp;quot;&#xA;test_list = [&amp;quot;PCNA&amp;quot;, &amp;quot;USP21&amp;quot;, &amp;quot;USP1&amp;quot;]&#xA;&#xA;df = gp.read_gtf(gtf_file)&#xA;subset = df[df['gene_name'].str.contains('|'.join(test_list))]&#xA;print(subset)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You could further cull this subset by other attributes or columns, e.g.:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;genes_only = subset[subset[&amp;quot;feature&amp;quot;] == &amp;quot;gene&amp;quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;etc.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;gtfparse&lt;/code&gt; library is available via PyPi: &lt;a href=&quot;https://pypi.org/project/gtfparse/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pypi.org/project/gtfparse/&lt;/a&gt; or Bioconda: &lt;a href=&quot;https://bioconda.github.io/recipes/gtfparse/README.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconda.github.io/recipes/gtfparse/README.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2021-09-07T18:26:26.147" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17673" PostTypeId="1" AcceptedAnswerId="17674" CreationDate="2021-09-09T20:21:25.977" Score="0" ViewCount="190" Body="&lt;p&gt;I am reading the &lt;a href=&quot;https://www.pacb.com/wp-content/uploads/Application-Brief-Whole-genome-sequencing-for-de-novo-assembly-Best-Practices.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Whole Genome Sequencing for &lt;em&gt;de novo&lt;/em&gt; Assembly Best Practices&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use the Sequel II or IIe System and SMRT® Cell 8M to sequence to desired coverage depth for complexity&#xA;of genome&lt;/li&gt;&#xA;&lt;li&gt;10- to 15-fold coverage per haplotype recommended&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;First of all Wikipedia shows different definitions for the term &lt;a href=&quot;https://en.wikipedia.org/wiki/Haplotype&quot; rel=&quot;nofollow noreferrer&quot;&gt;haplotype&lt;/a&gt;. Which one applies here?&lt;/p&gt;&#xA;&lt;p&gt;Second while the recommendations state to use &lt;code&gt;10- to 15-fold&lt;/code&gt; coverage per haplotype I find myself confused about how I would know the &lt;a href=&quot;https://en.wikipedia.org/wiki/Haplotype&quot; rel=&quot;nofollow noreferrer&quot;&gt;haplotype&lt;/a&gt; beforehand.&lt;/p&gt;&#xA;&lt;p&gt;From what I understand of &lt;a href=&quot;https://en.wikipedia.org/wiki/Haplotype_estimation&quot; rel=&quot;nofollow noreferrer&quot;&gt;haplotype-estimation&lt;/a&gt; I would need the sequenced data first.&lt;/p&gt;&#xA;&lt;p&gt;How would I go about this when wanting to do a &lt;em&gt;de novo&lt;/em&gt; assembly of a species that has no published assembly of any close relatives? That means the only information I would have at that point is the C-value of the species to be sequenced.&lt;/p&gt;&#xA;&lt;p&gt;Or better, how does the Sequel II figure out the number of haplotypes for achieving the desired fold-coverage?&lt;/p&gt;&#xA;&lt;p&gt;I hope I'm in the right SE.&lt;/p&gt;&#xA;" OwnerUserId="10791" LastActivityDate="2021-09-11T02:09:57.937" Title="How do you set the coverage in PacBio's Sequel II?" Tags="|assembly|coverage|pacbio|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17674" PostTypeId="2" ParentId="17673" CreationDate="2021-09-09T20:52:56.500" Score="2" Body="&lt;p&gt;They just want to know the ploidy of the sample. In a diploid sample, e.g. humans, there are two haplotypes. This is mostly important for haplotype resolution (i.e. get the two full genomes in a diploid).&lt;/p&gt;&#xA;&lt;p&gt;Note that elsewhere in the brochure that you link it writes:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Fast and efficient assembly of even&#xA;the largest genomes with haplotype&#xA;resolution of complex polyploids&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If you are working with a diploid species, that translates to 20-30X coverage. That seems low to me for the older tech of CLR/Sequel sequencing, but they are the experts.&lt;/p&gt;&#xA;&lt;p&gt;Different recommendations posted &lt;a href=&quot;https://pacbio.gs.washington.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; by a service provider suggest 40-50X for structural variant analysis, which seems more reasonable.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-09-09T20:52:56.500" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17675" PostTypeId="1" AcceptedAnswerId="17680" CreationDate="2021-09-09T21:48:46.753" Score="-1" ViewCount="154" Body="&lt;p&gt;I am confused about the genes at the top of the volcano plot. There are a lot of genes at the top, what's wrong with it? If that's normal what's the interpretation of it?&lt;/p&gt;&#xA;&lt;p&gt;I used volcano plot tools from  galaxy with  p-value=0.05.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/02dny.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/02dny.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12578" LastEditorUserId="77" LastEditDate="2021-09-10T13:59:23.263" LastActivityDate="2021-09-10T15:33:40.223" Title="Why does my volcano plot show genes at the extremities of the graph?" Tags="|visualization|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17677" PostTypeId="1" AcceptedAnswerId="17678" CreationDate="2021-09-10T05:58:58.973" Score="1" ViewCount="107" Body="&lt;p&gt;I'm trying to compare proteins generated with prokka (from a genome) with blastp using a virulence factor database &lt;a href=&quot;http://www.mgc.ac.cn/VFs/download.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;VFDB&lt;/a&gt;, and I want to use some parameters obtained from this &lt;a href=&quot;https://journals.asm.org/doi/epdf/10.1128/mBio.02852-18&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;blastp option, top hit, ≥50% identity, ≥50% alignment length, E-value of &amp;lt; 10 - 5&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;How can I specify those parameters on the command line?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# create DB&#xA;makeblastdb -in VFDB.fas -dbtype prot&#xA;&#xA;# run blast&#xA;blastp -query my.faa -db VFDB.fas -out results.txt .....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;any suggestion?&lt;/p&gt;&#xA;" OwnerUserId="14140" LastEditorUserId="57" LastEditDate="2021-09-11T13:30:49.407" LastActivityDate="2021-09-11T13:30:49.407" Title="blastp option to compare with a personal database" Tags="|blastp|makeblastdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17678" PostTypeId="2" ParentId="17677" CreationDate="2021-09-10T07:59:33.120" Score="5" Body="&lt;p&gt;You can use these options:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastp -query my.faa -db VFDB.fas \&#xA;       -perc_identity 50 -outfmt 6 \&#xA;       -evalue 10e-5 -out results.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then select the 5th column, &lt;em&gt;qlen&lt;/em&gt;, and the 13th columns, &lt;em&gt;slen&lt;/em&gt;, and get the percentage higher than 50.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk  -F'\t' '{ if (&lt;span class=&quot;math-container&quot;&gt;$NF &amp;gt;= 50) printf &quot;%s\t%.2f\n&quot;, $&lt;/span&gt;0 , (&lt;span class=&quot;math-container&quot;&gt;$2/$&lt;/span&gt;5)*100 }' result.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Check the documentation for the full &lt;code&gt;-outfmt&lt;/code&gt; column names with &lt;code&gt;blast -help&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="4282" LastEditorUserId="298" LastEditDate="2021-09-11T12:24:37.607" LastActivityDate="2021-09-11T12:24:37.607" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17679" PostTypeId="1" CreationDate="2021-09-10T08:13:09.457" Score="1" ViewCount="83" Body="&lt;p&gt;I am receiving imputed data from UK Biobank to conduct a GWAS on.  Previously I have carried out GWAS on genotype data, which I have QC'd for missingness per individual and per SNP, sex discrepancy, MAF filter &amp;lt; 0.05, heterozygosity rate, cryptic relatedness and population stratification.&lt;/p&gt;&#xA;&lt;p&gt;However, from reading several papers about QC of imputed data, I do not carry out any of these QC checks on the imputed data.  What QC procedures should I expect to carry out on UKBB data? Just to filter by info score?&lt;/p&gt;&#xA;&lt;p&gt;Or would it be better to carry out the initial QC checks on the genotype data and then impute myself?&lt;/p&gt;&#xA;" OwnerUserId="10083" LastEditorUserId="10083" LastEditDate="2021-09-10T09:29:34.053" LastActivityDate="2021-09-10T09:29:34.053" Title="What is the best QC to do on imputed UK Biobank data?" Tags="|gwas|imputation|qc|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17680" PostTypeId="2" ParentId="17675" CreationDate="2021-09-10T15:33:40.223" Score="0" Body="&lt;p&gt;Either your graph or your data is capped at 300.  Probably the p-values themselves; whatever calculated those p-values literally can't represent a number tinier than 1x10^-300&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2021-09-10T15:33:40.223" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17683" PostTypeId="2" ParentId="11158" CreationDate="2021-09-10T18:36:48.957" Score="1" Body="&lt;p&gt;The HiFi de novo assembly tool that &lt;strong&gt;PacBio&lt;/strong&gt; has been developing is called &lt;strong&gt;IPA&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;It’s available on &lt;a href=&quot;https://github.com/PacificBiosciences/pbipa&quot; rel=&quot;nofollow noreferrer&quot;&gt;pbbioconda&lt;/a&gt; for command line use, and as a push-button pipeline in their SMRT Link software.&lt;/p&gt;&#xA;&lt;p&gt;I looked at a few examples of runtimes of &lt;strong&gt;IPA&lt;/strong&gt; for different genome sizes:&lt;/p&gt;&#xA;&lt;p&gt;3 Gb human genome – ~600 CPU hours&#xA;800 Mb complex plant – ~430 CPU hours&#xA;400 Mb plant - ~100 CPU hours&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/chhylp123/hifiasm&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hifiasm&lt;/a&gt; seems to be more efficient as far as I can tell. In &lt;a href=&quot;https://doi.org/10.1038/s41592-020-01056-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;their paper&lt;/a&gt; they have a few nice tables comparing it to HiCanu, Peregrine and Falcon.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QCPit.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QCPit.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10791" LastActivityDate="2021-09-10T18:36:48.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17684" PostTypeId="1" AcceptedAnswerId="17688" CreationDate="2021-09-10T20:42:19.220" Score="0" ViewCount="29" Body="&lt;p&gt;From the &lt;a href=&quot;https://en.wikipedia.org/wiki/Protein_Data_Bank&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia entry&lt;/a&gt; on the Protein Data Bank (PDB) (emphasis mine):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Most structures are determined by X-ray diffraction, but about 10% of structures are determined by protein NMR. When using X-ray diffraction, approximations of the coordinates of the atoms of the protein are obtained, &lt;strong&gt;whereas using NMR, the distance between pairs of atoms of the protein is estimated&lt;/strong&gt;. The final conformation of the protein is obtained from NMR by solving a distance geometry problem.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I am currently working on an algorithm to solve distance geometry problems in the context of molecular conformations. I would prefer to test this algorithm on real data, so does the PDB contain information about the distances between pairs of atoms that were estimated from NMR spectroscopy measurements?&lt;/p&gt;&#xA;&lt;p&gt;I checked the PDB file for the &lt;a href=&quot;https://www.rcsb.org/structure/2KB7&quot; rel=&quot;nofollow noreferrer&quot;&gt;2KB7&lt;/a&gt; protein, but I only found the xyz coordinates of the atoms obtained &lt;em&gt;after&lt;/em&gt; solving the distance geometry problem. Is it possible to get access to the estimated distances that were used to generate these conformations?&lt;/p&gt;&#xA;" OwnerUserId="14148" LastEditorUserId="6322" LastEditDate="2021-09-11T08:11:55.503" LastActivityDate="2021-09-11T08:11:55.503" Title="Does the Protein Data Bank contain the estimated distances obtained using NMR spectroscopy?" Tags="|protein-structure|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17685" PostTypeId="1" AcceptedAnswerId="17689" CreationDate="2021-09-10T23:40:41.330" Score="1" ViewCount="486" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/9488897/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I ran Kallisto with the --pseudobam option. How do I extract all the short reads that are mapped to a single transcript (e.g. ENST00000367969.8)?  As a person without any previous SAM/BAM experience, I tried the following things without success. Really appreciate some help here.&lt;/p&gt;&#xA;&lt;p&gt;I first tried:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -S pseudoalignments.bam ENST00000367969.8&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but got the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[E::idx_find_and_load] Could not retrieve index file for 'pseudoalignments.bam'&#xA;[main_samview] random alignment retrieval only works for indexed BAM or CRAM files.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried to index the file using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools index pseudoalignments.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but get the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[E::hts_idx_push] NO_COOR reads not in a single block at the end 79729 -1&#xA;[E::sam_index] Read 'A00742:184:HLTF2DSXY:4:1101:4417:1000' with ref_name='ENST00000556431.1', ref_length=4142, flags=83, pos=2098 cannot be indexed&#xA;samtools index: failed to create index for &amp;quot;pseudoalignments.bam&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14149" LastEditorUserId="73" LastEditDate="2021-12-02T00:31:57.797" LastActivityDate="2021-12-02T00:31:57.797" Title="How to extract all sequences mapped to a transcript from Kallisto output" Tags="|sam|samtools|kallisto|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17686" PostTypeId="2" ParentId="17673" CreationDate="2021-09-11T02:09:57.937" Score="1" Body="&lt;p&gt;Unless you want to reconstruct all haplotypes, you can sequence to ~30X regardless of ploidy.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2021-09-11T02:09:57.937" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17687" PostTypeId="1" CreationDate="2021-09-11T06:30:06.340" Score="1" ViewCount="348" Body="&lt;p&gt;I'm using PyTorch (1.7.1), PyTorch Geometric (1.6.3), NVIDIA Cuda (11.2).&lt;/p&gt;&#xA;&lt;p&gt;I need to make a neural network reproducible for a competition.&lt;/p&gt;&#xA;&lt;p&gt;However, when I try:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;device = torch.device('cuda:0')&#xA;rand = 123&#xA;torch.manual_seed(rand)&#xA;torch.cuda.manual_seed(rand)&#xA;torch.cuda.manual_seed_all(rand)&#xA;torch.backends.cudnn.deterministic = True&#xA;torch.backends.cudnn.benchmark = False&#xA;np.random.seed(rand)&#xA;random.seed(rand)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;the results are different each time I run the code. How can I fix it?&lt;/p&gt;&#xA;&lt;p&gt;(For a reference, it always comes out the same in &lt;code&gt;device = torch.device('cpu')&lt;/code&gt;.)&lt;/p&gt;&#xA;" OwnerUserId="12688" LastEditorUserId="57" LastEditDate="2021-09-11T13:35:57.743" LastActivityDate="2021-09-13T16:35:06.100" Title="Reproducible PyTorch Model" Tags="|machine-learning|deep-learning|pytorch|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17688" PostTypeId="2" ParentId="17684" CreationDate="2021-09-11T08:11:09.107" Score="2" Body="&lt;p&gt;The data can be found here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bmrb.io/search/instant.php?term=2KB7&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bmrb.io/search/instant.php?term=2KB7&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There is no link directly in the PBD as it is a one-to-many relationship.&#xA;Clicking on the first value (a shifts dataset) you get somewhere in the middle a link to the actual dataset, which is a CIF like file with the following table:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;                              vvvvvvv     &#xA;    1 . 1 1  2  2 MET HA H  1   4.070 . . 1 . . . A  2 MET HA . 18256 1 &#xA;    2 . 1 1  2  2 MET C  C 13 176.670 . . 1 . . . A  2 MET C  . 18256 1 &#xA;                              ^^^^^^^ &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Each row has the spectral shift in ppm (chevrons added) of an &lt;em&gt;assigned&lt;/em&gt; hydrogen or carbon atom. There are also anisotropic chemical shifts and dipolar couplings datasets etc. etc. Each with a particular piece of data...&lt;/p&gt;&#xA;&lt;p&gt;Chemical structures are &amp;quot;elucidated&amp;quot; (=solved) in different ways depending on the type of experiment.&lt;/p&gt;&#xA;&lt;p&gt;Say you wanted only the distanced calculated by &lt;a href=&quot;https://en.wikipedia.org/wiki/Nuclear_Overhauser_effect&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nuclear Overhauser effect&lt;/a&gt; (r_NOE), you would need to find a NOE NMR experiment.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-09-11T08:11:09.107" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17689" PostTypeId="2" ParentId="17685" CreationDate="2021-09-11T09:13:07.630" Score="1" Body="&lt;p&gt;Sort the BAM file again with &lt;code&gt;samtools sort&lt;/code&gt; and then run &lt;code&gt;samtools index&lt;/code&gt; on the result. That should fix the issue.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-09-11T09:13:07.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17690" PostTypeId="1" AcceptedAnswerId="17693" CreationDate="2021-09-11T11:22:37.547" Score="0" ViewCount="294" Body="&lt;p&gt;Dipeptide composition of a protein sequence is the number of times a particular dipeptide (e.g. Arginine-Histidine) occurs in a sequence divided by the total number of dipeptides in the sequence (which is the length of the sequence - 1)&lt;/p&gt;&#xA;&lt;p&gt;I have found the following code for this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;&#xA;def DPC(fastas):&#xA;    AA = 'ACDEFGHIKLMNPQRSTVWY'&#xA;    encodings = []&#xA;    diPeptides = [aa1 + aa2 for aa1 in AA for aa2 in AA]&#xA;    header = ['#'] + diPeptides&#xA;    encodings.append(header)&#xA;&#xA;    AADict = {}&#xA;    for i in range(len(AA)):&#xA;        AADict[AA[i]] = i&#xA;&#xA;    for i in fastas:&#xA;        name, sequence = i[0], re.sub('-', '', i[1])&#xA;        code = [name]&#xA;        tmpCode = [0] * 400&#xA;        for j in range(len(sequence) - 2 + 1):&#xA;            tmpCode[AADict[sequence[j]] * 20 + AADict[sequence[j+1]]] = tmpCode[AADict[sequence[j]] * 20 + AADict[sequence[j+1]]] +1&#xA;        if sum(tmpCode) != 0:&#xA;            tmpCode = [i/sum(tmpCode) for i in tmpCode]&#xA;        code = code + tmpCode&#xA;        encodings.append(code)&#xA;    return encodings&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There is another file (main) from where the arguments &lt;code&gt;fastas&lt;/code&gt; come from which is basically a list of strings where every string is a protein sequence. I am finding it really difficult to understand the nested loops that were used:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    for i in fastas:&#xA;        name, sequence = i[0], re.sub('-', '', i[1])&#xA;        code = [name]&#xA;        tmpCode = [0] * 400&#xA;        for j in range(len(sequence) - 2 + 1):&#xA;            tmpCode[AADict[sequence[j]] * 20 + AADict[sequence[j+1]]] = tmpCode[AADict[sequence[j]] * 20 + AADict[sequence[j+1]]] +1&#xA;        if sum(tmpCode) != 0:&#xA;            tmpCode = [i/sum(tmpCode) for i in tmpCode]&#xA;        code = code + tmpCode&#xA;        encodings.append(code)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have spent 4-5 hours on this but I am still finding it really difficult to follow. I would really appreciate if anyone can explain the steps involved in these loops. Thank you!&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: The github link to the files are :&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/Superzchen/iFeature/blob/master/codes/DPC.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;DPC.py&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/Superzchen/iFeature/blob/master/iFeature.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;iFeature.py&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The github link to the python toolkit: &lt;a href=&quot;https://github.com/Superzchen/iFeature&quot; rel=&quot;nofollow noreferrer&quot;&gt;iFeature&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="7168" LastEditorUserId="7168" LastEditDate="2021-09-11T14:12:10.160" LastActivityDate="2021-09-11T15:18:02.433" Title="Help to understand the code for dipeptide composition calculation (in python)" Tags="|python|featurecounts|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17691" PostTypeId="2" ParentId="17685" CreationDate="2021-09-11T12:28:15.980" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/a/17689/298&quot;&gt;What Devon said.&lt;/a&gt; Alternatively, you could also just convert the bam to sam and then look for lines where the 3rd field (the reference the sequence is aligned to) is what you want:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view pseudoalignments.bam | awk '$3==&amp;quot;ENST00000367969.8&amp;quot;'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But really, it's far better to sort and index the file as Devon suggested so  you can use dedicated tools in future.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2021-10-01T22:25:05.390" LastActivityDate="2021-10-01T22:25:05.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17692" PostTypeId="2" ParentId="17656" CreationDate="2021-09-11T12:50:41.457" Score="2" Body="&lt;p&gt;I'm afraid this is a clear bug in the tool you are using. The regular expression requires that all fields in the GTF file contain no whitespace. The &lt;code&gt;\S+&lt;/code&gt; means &amp;quot;one or more non-whitespace characters&amp;quot; but your &amp;quot;source&amp;quot; field, the second field in the GTF file, contains spaces: &lt;code&gt;Desoria tigrina genome assembly, contig: utg000040l, whole genome shotgun sequence&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The thing is, that is a perfectly valid field for a GTF file! GTF fields can contain spaces, they just can't contain tabs. I tried to find a repository for the software to suggest a fix, but found only &lt;a href=&quot;https://github.com/gatech-genemark/GeneMark-ES&quot; rel=&quot;nofollow noreferrer&quot;&gt;an empty one&lt;/a&gt;. In any case, try changing the regex you show to this:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;#if ( /^(\S+)\t\S+\t(\S+)\t(\d+)\t(\d+)\t.\t([\+-])\t([012.])\tgene_id \&amp;quot;(\S+)\&amp;quot;/ )&#xA;if ( /^([^\t]+)\t[^\t]+\t([^\t]+)\t(\d+)\t(\d+)\t.\t([\+-])\t([012.])\tgene_id \&amp;quot;(\S+)\&amp;quot;/ )&#xA;    {&#xA;    ... (some stuff)&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I can't guarantee that this will work as expected, please test! The details will depend on what they do with the captured groups (the &lt;code&gt;some stuff&lt;/code&gt;), but at least that regex will match and I &lt;em&gt;think&lt;/em&gt; I have kept the numbering of the capture groups.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2021-09-11T12:59:15.520" LastActivityDate="2021-09-11T12:59:15.520" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17693" PostTypeId="2" ParentId="17690" CreationDate="2021-09-11T15:00:09.310" Score="3" Body="&lt;p&gt;A few additional hints that may help clarify the algorithm:&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;20&lt;/code&gt; magic number is because there are twenty amino acids.  The &lt;code&gt;400&lt;/code&gt; magic number is because there are &lt;code&gt;20 * 20&lt;/code&gt; potential dipeptides.&lt;/p&gt;&#xA;&lt;p&gt;These peptides are then hashed by converting their letter to a number in the range [0, 20). You can then multiple one of them by 20 and add the other to get a number between [0, 400).  In the case of this function the first amino acid &amp;quot;code&amp;quot; is multiplied by 20. This number can be converted back to the dipeptide sequence when outputting the data.  The translation table they use is generated here &lt;code&gt;diPeptides = [aa1 + aa2 for aa1 in AA for aa2 in AA]&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The nested loops are to loop through each entry and then each sequence to get every dipeptide sequence, which can be done in linear time.&lt;/p&gt;&#xA;&lt;p&gt;Just as a quick note on hashing DNA and peptide sequences.  You can generally convert them to an array index by using increasing powers of the alphabet size multiplied by the letter index of the alphabet.  For example, for DNA you can convert &lt;code&gt;A=0, C=1, G=2, T=3&lt;/code&gt;, and you can hash fourmers by using &lt;code&gt;4^4 * a[i] + 4^3 * a[i + 1] + 4^2 * a[i + 2] + 4^1 * a[i + 3]&lt;/code&gt;. In code you may see this performed with bitshifts for DNA since the alphabet size is a power of 2.&lt;/p&gt;&#xA;&lt;p&gt;The advantage of this hashing scheme is. thateach sequence is guaranteed to have a unique index.  The problem with this method is that as if you wish to use longer k-mers the memory usage will grow exponentially.  For example, for 16-mers you need 4 billion combinations, which is the size of a 32 bit unsigned integer.&lt;/p&gt;&#xA;" OwnerUserId="64" LastEditorUserId="64" LastEditDate="2021-09-11T15:18:02.433" LastActivityDate="2021-09-11T15:18:02.433" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17694" PostTypeId="1" CreationDate="2021-09-11T17:52:51.993" Score="1" ViewCount="29" Body="&lt;p&gt;I'm an interested layman reading the paper &lt;a href=&quot;https://arxiv.org/pdf/1408.3421.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/pdf/1408.3421.pdf&lt;/a&gt; (Hsu, Stephen, On the genetic architecture of intelligence and other quantitative traits, 2014). I'm trying to understand Hsu's claim (my possibly incomplete/faulty interpretation; for his words see pages 13 and 16):&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Alleles affecting a quantitative phenotype trait that has been strongly selected for (such as height or intelligence for humans) will have their frequencies pushed close to 0 or 1 by evolution.&lt;/li&gt;&#xA;&lt;li&gt;If there are two possible alleles at each of &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; sites contributing to the quantitative trait, say with each minor allele having frequency &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, then linear contributions of minor alleles would number &lt;span class=&quot;math-container&quot;&gt;$O(p)$&lt;/span&gt; whereas non-linear effects would number &lt;span class=&quot;math-container&quot;&gt;$O(p^2)$&lt;/span&gt;. By the previous point, &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; is small, so the linear terms dominate the total contribution.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Two things confuse me with this:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Might not the larger number of non-linear interactions (at least &lt;span class=&quot;math-container&quot;&gt;$\binom N2)$&lt;/span&gt; vs &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt;) compensate for the extra factor &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;?&lt;/li&gt;&#xA;&lt;li&gt;Shouldn't we expect alleles only involved in non-linear contributions to experience less selective pressure, hence making the &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; for such minor alleles larger than for those contributing linearly, again compensating for the differing orders in &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14152" LastEditorUserId="7340" LastEditDate="2021-11-02T17:24:37.823" LastActivityDate="2021-11-02T17:24:37.823" Title="Reason for linear effects dominating the difference in a quantitative trait between individual genomes" Tags="|gwas|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17695" PostTypeId="1" CreationDate="2021-09-11T22:40:38.383" Score="1" ViewCount="100" Body="&lt;p&gt;I have a contigs file that I generated using Minia and then I have used Minimap2 to map these contigs again to the reference genome. Now I want to plot the gap distribution i.e. gap distance of the set of contigs with respect to the positions in the reference genome. For example: something like , if there are m contigs obtained from the Minia, then X axis will show 0 to m-1 gap points (in an ideal case where there is no overlap between the contigs and the contigs are aligned back to back without space).&lt;/p&gt;&#xA;&lt;p&gt;The file from minimap2 looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;0   17679   0   17679   +   NZ_BCML01000091.1   39426   18736   36415   17679   17679   60  NM:i:0  ms:i:35358  AS:i:35358  nn:i:0  tp:A:P  cm:i:3274   s1:i:17675  s2:i:49 de:f:0  rl:i:75 cg:Z:17679M&#xA;1   16011   0   16011   +   NZ_BCML01000021.1   105137  86      16097   16011   16011   60  NM:i:0  ms:i:32022  AS:i:32022  nn:i:0  tp:A:P  cm:i:3008   s1:i:15996  s2:i:0  de:f:0  rl:i:15 cg:Z:16011M&#xA;2   2779    0   2779    -   NZ_BCML01000048.1   72397   14902   17681   2779    2779    60  NM:i:0  ms:i:5558   AS:i:5558   nn:i:0  tp:A:P  cm:i:519    s1:i:2751   s2:i:0  de:f:0  rl:i:45 cg:Z:2779M&#xA;4   177     0   177     +   NZ_BCML01000150.1   16425   15710   15887   177     177     60  NM:i:0  ms:i:354    AS:i:354    nn:i:0  tp:A:P  cm:i:30     s1:i:160    s2:i:0  de:f:0  rl:i:15 cg:Z:177M    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where the 1st column is the contig ID and 8th and 9th column show the start and end position of the contig on the reference genome.&lt;/p&gt;&#xA;&lt;p&gt;I have used the samtools for obtaining a length distribution plot, but not understanding how to do a gap distribution plot. I can use awk on linux or python or R. Anything that is going to work. Thank you for all the help.&lt;/p&gt;&#xA;" OwnerUserId="14155" LastActivityDate="2021-09-18T23:25:19.803" Title="How to plot the gap distribution of contigs wrt to reference genome" Tags="|r|python|fasta|ggplot2|minimap2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17696" PostTypeId="2" ParentId="17695" CreationDate="2021-09-11T23:33:04.870" Score="2" Body="&lt;h2&gt;a better solution&lt;/h2&gt;&#xA;&lt;p&gt;I wasn't too happy with the previous answer that I gave, so I was fiddling around, and I realized that &lt;code&gt;bedtools genomecov&lt;/code&gt; &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/genomecov.html#bg-reporting-genome-coverage-in-bedgraph-format&quot; rel=&quot;nofollow noreferrer&quot;&gt;will give you this for free&lt;/a&gt; with the following workflow:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 -ax asm10 ref.fasta contigs.fasta | \&#xA;samtools sort --output_fmt BAM &amp;gt; contigs_to_ref.bam&#xA;# -bga means output in bedgraph format, including all positions&#xA;bedtools genomecov -ibam contigs_to_ref.bam -bga &amp;gt; contig_cov.bg&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you look at the resulting file, it will report all regions of identical coverage:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head contig_cov.bg&#xA;# prints the following:&#xA;# reference chr/contig, start position, end position, coverage&#xA;NODE_1_length_442924_cov_598.199    0   8151    0&#xA;NODE_1_length_442924_cov_598.199    8151    8392    1&#xA;NODE_1_length_442924_cov_598.199    8392    20203   0&#xA;NODE_1_length_442924_cov_598.199    20203   20463   1&#xA;NODE_1_length_442924_cov_598.199    20463   44953   0&#xA;NODE_1_length_442924_cov_598.199    44953   45216   1&#xA;NODE_1_length_442924_cov_598.199    45216   52633   0&#xA;NODE_1_length_442924_cov_598.199    52633   52899   1&#xA;NODE_1_length_442924_cov_598.199    52899   57698   0&#xA;NODE_1_length_442924_cov_598.199    57698   58207   1&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can simply filter this file for all regions wherein the fourth column is equal to zero, and take the first 3 columns, and that is a list of gaps in the assembly relative to the reference:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '&lt;span class=&quot;math-container&quot;&gt;$4 == &quot;0&quot; { print $&lt;/span&gt;0 }' contig_cov.bg | cut -f1-3&#xA;# prints&#xA;NODE_1_length_442924_cov_598.199    0   8151&#xA;NODE_1_length_442924_cov_598.199    8392    20203&#xA;NODE_1_length_442924_cov_598.199    20463   44953&#xA;NODE_1_length_442924_cov_598.199    45216   52633&#xA;NODE_1_length_442924_cov_598.199    52899   57698&#xA;NODE_1_length_442924_cov_598.199    58207   73398&#xA;NODE_1_length_442924_cov_598.199    73733   78601&#xA;NODE_1_length_442924_cov_598.199    78959   80147&#xA;NODE_1_length_442924_cov_598.199    80419   86967&#xA;NODE_1_length_442924_cov_598.199    87223   93872&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You could even do this as a one-liner probably though that's getting a little convoluted.&lt;/p&gt;&#xA;&lt;h2&gt;Original answer&lt;/h2&gt;&#xA;&lt;p&gt;It's a bit circuitous, but one way to do this would be to follow &lt;a href=&quot;http://seqanswers.com/forums/showthread.php?t=12338&quot; rel=&quot;nofollow noreferrer&quot;&gt;this answer's suggestion&lt;/a&gt; and generate a pileup for the alignment of your contigs to the reference.&lt;/p&gt;&#xA;&lt;p&gt;You can get minimap2 to give SAM/BAM output with the &lt;code&gt;-a&lt;/code&gt; flag. You then use the SAM/BAM to generate a pileup or a coverage file, using &lt;code&gt;samtools mpileup&lt;/code&gt; or &lt;code&gt;samtools depth -a&lt;/code&gt;. You can then use a heuristic to find contiguous regions with zero coverage of contigs. You could for example loop through a depth file, here is some python code that gives the basic idea:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python&#xA;&#xA;# minimap2 -ax asm10 ref.fasta contigs.fasta | \&#xA;# samtools sort --output_fmt BAM &amp;gt; contigs_to_ref.bam&#xA;# samtools depth contigs_to_ref.bam &amp;gt; depth.txt&#xA;depth_file = &amp;quot;depth.txt&amp;quot;&#xA;&#xA;with open(depth_file) as dep_handle:&#xA;  gap_sizes = list()&#xA;  start = None&#xA;  pos = None&#xA;  for line in dep_handle:&#xA;    # needs also logic to handle multiple chromosomes in reference &#xA;    # but I'm lazy&#xA;    fields = line.split()&#xA;    if int(fields[2]) == 0:&#xA;      pos = int(fields[2])&#xA;      if start is None:&#xA;        start = int(fields[1])&#xA;      &#xA;    elif fields[2] &amp;gt;0:&#xA;      if start is None:&#xA;        continue&#xA;      else:&#xA;        gap_sizes.append(pos - start)&#xA;        start = None&#xA;&#xA;print(gap_sizes)&#xA;        &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Probably there is a simpler way to do this but I don't know it.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2021-09-18T23:25:19.803" LastActivityDate="2021-09-18T23:25:19.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17697" PostTypeId="2" ParentId="3931" CreationDate="2021-09-13T05:00:49.723" Score="0" Body="&lt;p&gt;Firstly, you have to install &lt;code&gt;biopython&lt;/code&gt; using the cmd &lt;code&gt;pip install biopython&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Suppose you have two files &lt;code&gt;id.txt&lt;/code&gt; and &lt;code&gt;sequence.fasta&lt;/code&gt;. The solution file name is &lt;code&gt;seq_rm.py&lt;/code&gt; which contains the following codes.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import argparse&#xA;from Bio import SeqIO&#xA;from Bio.SeqRecord import SeqRecord&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    parser = argparse.ArgumentParser()&#xA;    parser.add_argument('--id_file', required=True,&#xA;                        help='file path of removable ids')&#xA;    parser.add_argument('--input', required=True,&#xA;                        help='path of fasta sequence')&#xA;    parser.add_argument('--output', required=True,&#xA;                        help='path of selected fasta sequence')&#xA;&#xA;    args = parser.parse_args()&#xA;&#xA;    with open(args.id_file, mode='r', encoding='utf-8') as file:&#xA;        ids = set(file.read().splitlines())&#xA;&#xA;    records = [SeqRecord(record.seq, id=record.id, description='', name='') for record in SeqIO.parse(args.input, 'fasta')&#xA;               if record.id not in ids]&#xA;    SeqIO.write(records, args.output, &amp;quot;fasta&amp;quot;)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Run &lt;code&gt;cmd&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;python3 seq_rm.py --id_file id.txt --input sequence.fasta --output selected.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It will generate an output file named &lt;code&gt;selected.fasta&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="14162" LastEditorUserId="14162" LastEditDate="2021-09-18T08:14:24.903" LastActivityDate="2021-09-18T08:14:24.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17699" PostTypeId="2" ParentId="17687" CreationDate="2021-09-13T16:35:06.100" Score="2" Body="&lt;p&gt;There is this nice function from Kaggle:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def seed_everything(seed=1234):&#xA;    random.seed(seed)&#xA;    os.environ['PYTHONHASHSEED'] = str(seed)&#xA;    np.random.seed(seed)&#xA;    torch.manual_seed(seed)&#xA;    torch.cuda.manual_seed(seed)&#xA;    torch.backends.cudnn.deterministic = True&#xA;&#xA;seed_everything()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There are several version of this function for torch, tensorflow and combinations. This specific one is from &lt;a href=&quot;https://www.kaggle.com/bminixhofer/deterministic-neural-networks-using-pytorch&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Run this before you run your model. But not only the fitting function. It needs to be before the validation split function as well.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2021-09-13T16:35:06.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17700" PostTypeId="1" CreationDate="2021-09-13T17:21:05.060" Score="0" ViewCount="35" Body="&lt;p&gt;Maybe I'm having a brain-freeze so excuse me if this is a waste of everyone's time...&lt;/p&gt;&#xA;&lt;p&gt;I am working with an organism with ~700Mb genome.&lt;/p&gt;&#xA;&lt;p&gt;I have bam files of two individuals that are whole genome sequenced with the average depth of ~150.&lt;/p&gt;&#xA;&lt;p&gt;I would like to count the occurrences when both of the individuals are homozygote for the reference allele. Preferably without any hassle or space, I have no need to create a vcf file for the task or save the information in any form. Something like '&lt;code&gt;gatk or mpileup... argument that only retains sites when all are homozygote reference... delete tmp file...'&lt;/code&gt;. So the question is: what would be the &amp;quot;cheapest&amp;quot;/fastest way to count where these individuals are homozygote reference?&lt;/p&gt;&#xA;&lt;p&gt;Come to think of it... I'm asking for a line count value on an &amp;quot;inverted variant file&amp;quot;/&amp;quot;gVCF minus variation&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="5182" LastActivityDate="2021-09-13T17:21:05.060" Title="Fastest way to count where two bam files are homozygote reference (inverted variant calling)" Tags="|sam|variant-calling|reference-genome|wgs|multiple-sequence-alignment|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17701" PostTypeId="1" CreationDate="2021-09-14T07:12:54.293" Score="1" ViewCount="605" Body="&lt;p&gt;I need to call variants on a large number of reads using both &lt;code&gt;VarScan2&lt;/code&gt; and &lt;code&gt;BCFTools&lt;/code&gt;. Both of these variant calling tools use &lt;code&gt;mpileup&lt;/code&gt; file as input to call the variants. &lt;code&gt;VarScan2&lt;/code&gt; command is straightforward, as we can specify the &lt;code&gt;mpileup&lt;/code&gt; file as a command parameter. The problem is, &lt;code&gt;BCFTools&lt;/code&gt; expects &lt;code&gt;mpileup&lt;/code&gt; to be piped from another command and I cannot find an input  parameter to specify a &lt;code&gt;mpileup&lt;/code&gt; file that already exists (the one used with &lt;code&gt;VarScan2&lt;/code&gt;). It seems horribly inefficient to run the same command twice instead of reusing the file obtained from previous step (especially because generating &lt;code&gt;mpileup&lt;/code&gt; takes a substantial amount of time on my computer).&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to reuse an existing &lt;code&gt;mpileup&lt;/code&gt; file with &lt;code&gt;BCFTools&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;I am using &lt;code&gt;Nextflow&lt;/code&gt; to run my pre-processing pipeline, so could it be possible that I can pipe an input channel with &lt;code&gt;mpileup&lt;/code&gt; into the &lt;code&gt;BCFTools&lt;/code&gt;'s &lt;code&gt;call&lt;/code&gt; command?&lt;/p&gt;&#xA;&lt;p&gt;&lt;s&gt;I also tried doing&#xA;&lt;code&gt;cat file.mpileup | bcftools call ...&lt;/code&gt; but this does not seem to work.&lt;/s&gt;&lt;/p&gt;&#xA;&lt;p&gt;Edit: &lt;code&gt;cat file.mpileup | bcftools call ...&lt;/code&gt; does work, which would be a solution to this question, however, answers offer other, even more elegant, solutions.&lt;/p&gt;&#xA;" OwnerUserId="12217" LastEditorUserId="57" LastEditDate="2021-09-19T12:27:17.503" LastActivityDate="2021-09-19T12:27:17.503" Title="How do I call variants using BCFTools without piping mpileup file?" Tags="|variant-calling|nextflow|bcftools|mpileup|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17702" PostTypeId="2" ParentId="17701" CreationDate="2021-09-14T07:42:01.233" Score="2" Body="&lt;p&gt;Why not use &lt;code&gt;tee&lt;/code&gt;? Please see this &lt;a href=&quot;https://stackoverflow.com/questions/418896/how-to-redirect-output-to-a-file-and-stdout&quot;&gt;post&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You can write &lt;code&gt;mpileup&lt;/code&gt; to file while also piping it to another process. For example, &lt;code&gt;samtools mpileup ... | tee file.mpileup | bcftools call ...&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;the &lt;code&gt;bcftools&lt;/code&gt; will run while generating the &lt;code&gt;file.mpileup&lt;/code&gt;. After it's done, you can use &lt;code&gt;file.mpileup&lt;/code&gt; for your &lt;code&gt;VarScan2&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You can test it with the following:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;touch helloworld.txt&#xA;ls | tee file.txt | grep helloworld    # output helloworld.txt&#xA;cat file.txt | grep helloworld         # output helloworld.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="298" LastEditDate="2021-09-16T13:55:20.810" LastActivityDate="2021-09-16T13:55:20.810" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17703" PostTypeId="1" AcceptedAnswerId="17709" CreationDate="2021-09-14T08:38:50.223" Score="2" ViewCount="36" Body="&lt;p&gt;I am looking for a sequence alignment software capable of pairwise alignment of a large number of protein sequences (10^7).&lt;/p&gt;&#xA;&lt;p&gt;I tried to go for mpiblast: &lt;a href=&quot;https://wiki.canterbury.ac.nz/display/UCHPC/Installing+mpiBLAST&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://wiki.canterbury.ac.nz/display/UCHPC/Installing+mpiBLAST&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;but seems their website was down.&lt;/p&gt;&#xA;&lt;p&gt;Do you know any alternatives to mpiblast which is also highly efficient in protein sequence alignment?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="14169" LastActivityDate="2021-09-14T19:34:05.993" Title="mpiblast alternatives?" Tags="|sequence-alignment|blast|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17704" PostTypeId="1" AcceptedAnswerId="17707" CreationDate="2021-09-14T08:48:12.960" Score="1" ViewCount="67" Body="&lt;p&gt;I have a pdb file for a protein (3D structure predicted by a tool) and around hundreds of known point mutations in the protein. I want to visualize the protein with all the mutations to see the hub of or core where the mutations are mostly. Is there are tool that does this? I would give the structure in a pdb file and a list of point mutations myself.&lt;/p&gt;&#xA;" OwnerUserId="14170" LastActivityDate="2021-09-14T14:51:32.870" Title="Site Mutations in a 3D protein structure to locate the core of mutations" Tags="|protein-structure|3d-structure|point-mutation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17705" PostTypeId="1" CreationDate="2021-09-14T09:11:01.373" Score="1" ViewCount="69" Body="&lt;p&gt;Let's say I have a candidate gene and I believe that in an individual sample, the genome sequence differs from the reference which then interferes with alignment.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way for me to do a &amp;quot;transcriptome assembly&amp;quot; with a specific read or sequence as a starting point? And only do the k-mer assembly from that single sequence?&lt;/p&gt;&#xA;&lt;p&gt;I could probably code something myself but I feel like a tool to do this must already exist?&lt;/p&gt;&#xA;&lt;p&gt;The goal here is to avoid the significant overhead of assembling a whole transcriptome when I'm only interested in one gene.&lt;/p&gt;&#xA;&lt;p&gt;NOTE: In this case the data is RNA-seq&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2023-06-06T20:08:51.267" Title="Assembling all transcripts for an individual gene? (using single sequence to seed the assembly)" Tags="|rna-seq|assembly|k-mer|stringtie|trinity|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17706" PostTypeId="2" ParentId="3931" CreationDate="2021-09-14T12:33:55.353" Score="0" Body="&lt;p&gt;One way using &lt;a href=&quot;/questions/tagged/awk&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; rel=&quot;tag&quot;&gt;awk&lt;/a&gt; without &lt;a href=&quot;https://awk.freeshell.org/AllAboutGetline&quot; rel=&quot;nofollow noreferrer&quot;&gt;getline&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'FNR==NR { a[$0]; next } /^&amp;gt;/ { f = !(substr($0, 2) in a) }f' ids.txt seqs.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2021-09-14T12:33:55.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17707" PostTypeId="2" ParentId="17704" CreationDate="2021-09-14T14:51:32.870" Score="0" Body="&lt;p&gt;Clustering mutations could reveal lots about the protein providing one has the functional impact of each mutation (this cluster are benign, this cluster causes misfolding etc). Many tools are emerging to explore this and there are plenty of established tools that can handle this.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://academic.oup.com/bioinformatics/article/35/22/4854/5514476?login=true&quot; rel=&quot;nofollow noreferrer&quot;&gt;VarMap&lt;/a&gt; is a tool that is able to map genomic variant coordinates to the best available structure.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://pymol.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyMol&lt;/a&gt; can open PDB files and you can programmatically or manually mutate each position using the &lt;a href=&quot;https://pymolwiki.org/index.php/Mutagenesis&quot; rel=&quot;nofollow noreferrer&quot;&gt;mutagenesis tool&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="747" LastActivityDate="2021-09-14T14:51:32.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17708" PostTypeId="2" ParentId="17705" CreationDate="2021-09-14T16:21:38.407" Score="1" Body="&lt;p&gt;In essence what you're looking for is targeted assembly. There aren't a lot of tools out there for that that I'm aware of, but on of them is &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-13-48&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mapsembler&lt;/a&gt;. The general idea is to input a seed region and then let the program perform iterative extension of it. That sort of strategy should largely match your needs.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-09-14T16:21:38.407" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17709" PostTypeId="2" ParentId="17703" CreationDate="2021-09-14T19:34:05.993" Score="4" Body="&lt;p&gt;The &lt;a href=&quot;https://github.com/bbuchfink/diamond&quot; rel=&quot;nofollow noreferrer&quot;&gt;Diamond protein aligner&lt;/a&gt; is a very popular tool for pairwise protein alignment which is much faster than blastp while having comparable sensitivity. Diamond also supports the use of using multiple compute nodes (similar to mpiBLAST). You can view the details of the new Diamond 2.0 release in &lt;a href=&quot;https://www.nature.com/articles/s41592-021-01101-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; recent Nature Methods paper (source of the image as well).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;...we aligned all 281 million protein sequences from the NCBI nr database against the UniRef50 database, which consists of 39 million sequences, using DIAMOND (v2.0.7) in ultra-sensitive mode...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/73rdB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/73rdB.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6628" LastActivityDate="2021-09-14T19:34:05.993" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17712" PostTypeId="2" ParentId="3634" CreationDate="2021-09-15T14:20:21.950" Score="2" Body="&lt;p&gt;If reads were independently sampled from a population with given, fixed fractions of genes, the read counts would follow a multinomial distribution, which can be approximated by the Poisson distribution. However, the Poisson distribution assumes the mean equals the variance, which is usually not satisfied by the data. And for the RNA sequencing data, the variance is usually larger than the mean. So we can assume the data follows a &lt;span class=&quot;math-container&quot;&gt;$Y|\lambda\sim Poisson(\lambda), \lambda\sim Gamma(r_0,b_0)$&lt;/span&gt; hierarchy.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\begin{align}&#xA;P(Y=y)&amp;amp;=\int_{0}^{\infty}P(Y=y|\lambda)f(\lambda)d\lambda\\&#xA;&amp;amp;=\int_{0}^{\infty}\frac{e^{-\lambda}\lambda^y}{y!}\frac{b_0^{r_0}}{\Gamma(r_0)}\lambda^{r_0-1}e^{-b_0\lambda}d\lambda\\&#xA;&amp;amp;=\frac{b_0^{r_0}}{\Gamma(r_0)y!}\int_{0}^{\infty}e^{-\lambda}\lambda^y\lambda^{r_0-1}e^{-b_0\lambda}d\lambda\\&#xA;&amp;amp;=\frac{b_0^{r_0}}{\Gamma(r_0)y!}\int_{0}^{\infty}e^{-\lambda(b_0+1)}\lambda^{y+r_0-1}d\lambda\\&#xA;&amp;amp;=\frac{\Gamma(r_0+y)}{\Gamma(r_0)y!}\frac{b_0^{r_0}}{(b_0+1)^{r_0+y}}\int_{0}^{\infty}\frac{1}{\Gamma(r_0+y)}e^{-\lambda(b_0+1)}\lambda^{y+r_0-1}(b_0+1)^{r_0+y}d\lambda\\&#xA;&amp;amp;=\frac{\Gamma(r_0+y)}{\Gamma(r_0)y!}\frac{b_0^{r_0}}{(b_0+1)^{r_0+y}}\int_{0}^{\infty}\frac{1}{\Gamma(r_0+y)}e^{-\lambda(b_0+1)}[\lambda(b_0+1)]^{r_0+y}\frac{1}{\lambda}d\lambda\\&#xA;&amp;amp;=\frac{\Gamma(r_0+y)}{\Gamma(r_0)y!}\frac{b_0^{r_0}}{(b_0+1)^{r_0+y}}\\&#xA;&amp;amp;=\frac{\Gamma(r_0+y)}{\Gamma(r_0)y!}\left(\frac{1}{b_0+1}\right)^y\left(\frac{b_0}{b_0+1}\right)^{r_0}\\&#xA;\end{align}$$&lt;/span&gt; which is negative binomial distribution.&lt;/p&gt;&#xA;&lt;p&gt;Negative binomial distribution depicts the number of failures &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; before &lt;span class=&quot;math-container&quot;&gt;$r$&lt;/span&gt;th success of a series of independent Bernoulli experiments, with success probability &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$f(k;r,p)\equiv Pr(X=k)=\frac{\Gamma(k+r)}{k!\Gamma(r)}p^r(1-p)^k\quad k=0,1,2,\cdots$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mu=\mathbb E(X)=\frac{r(1-p)}{p}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$\sigma^2=\mathbb V(X)=\frac{r(1-p)}{p^2}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then&#xA;&lt;span class=&quot;math-container&quot;&gt;$$p=\frac{\mu}{\sigma^2}$$&lt;/span&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$r=\frac{\mu^2}{\sigma^2-\mu}$$&lt;/span&gt;&#xA;The probability mass function (PMF) is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$f(k;r,p)\equiv Pr(X=k)={k+\frac{\mu^2}{\sigma^2-\mu}-1\choose{\frac{\mu^2}{\sigma^2-\mu}-1}}\left(\frac{\sigma^2-\mu}{\sigma^2}\right)^k\left(\frac{\mu}{\sigma^2}\right)^{\frac{\mu^2}{\sigma^2-\mu}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The &lt;span class=&quot;math-container&quot;&gt;$\mu$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\sigma^2$&lt;/span&gt; of each gene can be estimated from data.&lt;/p&gt;&#xA;&lt;p&gt;Assume that the number of reads in sample &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt; that are assigned to gene &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; can be modeled by a negative binomial (NB) distribution,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$K_{ij}\sim NB(\mu_{ij},\sigma^2_{ij})\tag1$$&lt;/span&gt; which has two parameters, the mean &lt;span class=&quot;math-container&quot;&gt;$\mu_{ij}$&lt;/span&gt; and the variance &lt;span class=&quot;math-container&quot;&gt;$\sigma^2_{ij}$&lt;/span&gt;. The read counts &lt;span class=&quot;math-container&quot;&gt;$K_{ij}$&lt;/span&gt; are non-negative integers. &lt;span class=&quot;math-container&quot;&gt;$q_{i, \rho(j)}$&lt;/span&gt; is proportional to the expectation value of the true (but unknown) concentration of fragments from gene &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; under condition &lt;span class=&quot;math-container&quot;&gt;$\rho(j)$&lt;/span&gt;. The size factor &lt;span class=&quot;math-container&quot;&gt;$s_j$&lt;/span&gt; represents the coverage, or sampling depth, of library &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;First, the mean parameter &lt;span class=&quot;math-container&quot;&gt;$\mu_{ij}$&lt;/span&gt; , that is, the expectation value of the observed counts for gene &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; in sample &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;, is the product of a condition-dependent per-gene value &lt;span class=&quot;math-container&quot;&gt;$q_{i, \rho(j)}$&lt;/span&gt;(where &lt;span class=&quot;math-container&quot;&gt;$\rho(j)$&lt;/span&gt; is the experimental condition of sample &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;) and a size factor &lt;span class=&quot;math-container&quot;&gt;$s_j$&lt;/span&gt;,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\mu_{ij} = q_{i, \rho(j)}s_j\tag2$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Second, the variance &lt;span class=&quot;math-container&quot;&gt;$\sigma^2_{ij}$&lt;/span&gt; is the sum of a shot noise term and a raw variance term,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\sigma^2_{ij}=\mu_{ij}+s_j^2v_{i,\rho(j)}\tag3$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Third, we assume that the per-gene raw variance parameter &lt;span class=&quot;math-container&quot;&gt;$v_{i,\rho(j)}$&lt;/span&gt; is a smooth function of &lt;span class=&quot;math-container&quot;&gt;$q_{i, \rho(j)}$&lt;/span&gt;,&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$v_{i,ρ(j)}=v_ρ(q_{i, \rho(j)}) \tag4$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;This assumption is needed because the number of replicates is typically too low to get a precise estimate of the variance for gene &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt; from just the data available for this gene. This assumption allows us to pool the data from genes with similar expression strength for the purpose of variance estimation (see &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/gb-2010-11-10-r106&quot; rel=&quot;nofollow noreferrer&quot; title=&quot;article&quot;&gt;article&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="9698" LastEditorUserId="9698" LastEditDate="2021-09-17T13:59:11.043" LastActivityDate="2021-09-17T13:59:11.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17713" PostTypeId="1" CreationDate="2021-09-16T11:41:57.963" Score="0" ViewCount="44" Body="&lt;p&gt;From the FASTA header I get the following information:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gi|1543850138|ref|NC_040192.1| Desmodium mottle virus isolate UG5 segment DNA-B, complete sequence&#xA;['gi', '1543850138', 'ref', 'NC_040192.1', ' Desmodium mottle virus isolate UG5 segment DNA-B, complete sequence']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there a better way to create a dictonary as I did with the below script?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;organismInfo = {}&#xA;for seq_record in SeqIO.parse(&amp;quot;data/landplantsviruses.fasta&amp;quot;,&amp;quot;fasta&amp;quot;):&#xA;    print(seq_record.description)&#xA;    temp = seq_record.description.split('|')&#xA;    print(temp)&#xA;    organismID = temp[3]&#xA;    organismName = temp[4].lstrip()&#xA;    print(organismID, organismName)&#xA;    organismInfo[organismID] = organismName&#xA;print(organismInfo)&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3329" LastActivityDate="2021-09-16T12:13:19.983" Title="creating a dictonary in Python" Tags="|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17714" PostTypeId="2" ParentId="17713" CreationDate="2021-09-16T12:13:19.983" Score="2" Body="&lt;p&gt;You may notice that Uniprot and NCBI have different standard: so your solution is not universal, which is why BioPython does not split the description for you —it could know different 'dialects' of fasta files but that is a tedious minefield.&lt;/p&gt;&#xA;&lt;p&gt;About coding, one write something in a &lt;a href=&quot;https://en.wikipedia.org/wiki/Code_golf&quot; rel=&quot;nofollow noreferrer&quot;&gt;code golf&lt;/a&gt; format, but this is problematic as it is hard to interpret and adapt, i.e. it's a bad practice. Having said that if made a list of keys you could use &lt;code&gt;zip&lt;/code&gt; with your list of elements as the list of keys may be more intelligible.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;header = 'gi|1543850138|ref|NC_040192.1| Desmodium mottle virus isolate UG5 segment DNA-B, complete sequence'&#xA;keys = 'db|id|type|id2|wordy.'.split('|')&#xA;organism_info = dict( zip(keys, map(str.strip, header.split('|')) ))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Also, for variables in Python, camel case is for Classes, not instances. Even if you have to send data to JS, using a constistent style, i.e. the PIP guideline one, avoids errors.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-09-16T12:13:19.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17715" PostTypeId="1" AcceptedAnswerId="17718" CreationDate="2021-09-16T12:22:35.047" Score="0" ViewCount="128" Body="&lt;p&gt;I am trying to edit the &amp;quot;end&amp;quot; of my sequences in a GFF3 that contains 10000 sequences (so I do not want to do it manually). I want to add 30 residues to every single fragment. How can I do this?&lt;/p&gt;&#xA;&lt;p&gt;Currently, I have a file that looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##gff-version 3&#xA;sp_P45452_MMP13_HUMAN   SignalP-5.0 signal_peptide  1   19  0.994300    .   .   .&#xA;sp_Q8R2H5_PHLD_RAT      SignalP-5.0 signal_peptide  1   23  0.986089    .   .   .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to edit it so that it looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##gff-version 3&#xA;sp_P45452_MMP13_HUMAN   SignalP-5.0 signal_peptide  1   49  0.994300    .   .   .&#xA;sp_Q8R2H5_PHLD_RAT      SignalP-5.0 signal_peptide  1   53  0.986089    .   .   .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="14187" LastActivityDate="2021-09-16T13:42:33.933" Title="Edit end column in GFF3 file" Tags="|bedtools|gff3|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17717" PostTypeId="1" AcceptedAnswerId="17719" CreationDate="2021-09-16T13:31:35.360" Score="2" ViewCount="875" Body="&lt;p&gt;I have written a rule for CombineGVCFs in gatk4. The rule is as follow&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;all_gvcf = get_all_gvcf_list()&#xA;&#xA;rule cohort:&#xA;  input:&#xA;    all_gvcf_list = all_gvcf,&#xA;    ref=&amp;quot;/data/refgenome/hg38.fa&amp;quot;,&#xA;    interval_list = prefix+&amp;quot;/bedfiles/hg38.interval_list&amp;quot;,&#xA;  params:&#xA;    extra = &amp;quot;--variant&amp;quot;,&#xA;  output:&#xA;    prefix+&amp;quot;/vcf/cohort.g.vcf&amp;quot;,&#xA;  shell:&#xA;    &amp;quot;gatk CombineGVCFs -R {input.ref} {params.extra} {input.all_gvcf_list} -O {output} --tmp-dir=/data/tmp -L {input.interval_list}&amp;quot;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;all_gvcf is the dataset for all gvcf files which will be combined. But the problem is I need to add --variant parameter before every input. Command I'm getting right now is as follows&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gatk CombineGVCFs -R /data/refgenome/hg38.fa --variant /data/prjna644607/vcf/SRR12165216_HC.g.vcf /data/prjna644607/vcf/SRR12165217_HC.g.vcf /data/prjna644607/vcf/SRR12165218_HC.g.vcf /data/prjna644607/vcf/SRR12165219_HC.g.vcf -O /data/prjna644607/vcf/cohort.g.vcf --tmp-dir=/data/tmp -L /data/prjna644607/bedfiles/hg38.interval_list&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The command I want to achieve is as follows&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gatk CombineGVCFs -R /data/refgenome/hg38.fa --variant /data/prjna644607/vcf/SRR12165216_HC.g.vcf --variant /data/prjna644607/vcf/SRR12165217_HC.g.vcf --variant /data/prjna644607/vcf/SRR12165218_HC.g.vcf --variant /data/prjna644607/vcf/SRR12165219_HC.g.vcf -O /data/prjna644607/vcf/cohort.g.vcf --tmp-dir=/data/tmp -L /data/prjna644607/bedfiles/hg38.interval_list&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I add this extra &amp;quot;--variant&amp;quot; tag before every input? I have added it in the get_all_gvcf_list() function. But then snakmake gives me inputfiles not found problem.&lt;/p&gt;&#xA;" OwnerUserId="10296" LastEditorUserId="10296" LastEditDate="2021-09-16T14:11:26.383" LastActivityDate="2022-07-26T05:01:30.233" Title="snakemake multiple parameters for multiple input and single output in snakemake. ConbineGVCFs gatk problem" Tags="|python|snakemake|gatk|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17718" PostTypeId="2" ParentId="17715" CreationDate="2021-09-16T13:42:33.933" Score="3" Body="&lt;p&gt;That's what awk excels at! Simply set the input (&lt;code&gt;-F '\t'&lt;/code&gt;) and output (&lt;code&gt;-v OFS='\t'&lt;/code&gt;) record field separators to tab, and add 30 to the value of the fifth field on lines that don't start with a &lt;code&gt;#&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ awk -F'\t' -v OFS='\t' '/^[^#]/{$5+=30}1' file.gtf &#xA;##gff-version 3&#xA;sp_P45452_MMP13_HUMAN   SignalP-5.0 signal_peptide  1   19  30.9943 .   .   .&#xA;sp_Q8R2H5_PHLD_RAT  SignalP-5.0 signal_peptide  1   23  30.9861 .   .   .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2021-09-16T13:42:33.933" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17719" PostTypeId="2" ParentId="17717" CreationDate="2021-09-16T17:19:47.027" Score="5" Body="&lt;p&gt;Found out the problem. Turns out I can write a lambda function as follows&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params:&#xA;        extra=lambda wildcards, input: ' -V '.join(input.all_gvcf_list)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and add '-V' before {params.extra} in the &amp;quot;shell&amp;quot; sction. That solves the problem&lt;/p&gt;&#xA;" OwnerUserId="10296" LastEditorUserId="10296" LastEditDate="2021-09-20T14:49:11.760" LastActivityDate="2021-09-20T14:49:11.760" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17720" PostTypeId="1" CreationDate="2021-09-17T04:00:06.810" Score="4" ViewCount="5360" Body="&lt;p&gt;Coming from a CS background. Reading through the wikipedia page, these all sounds like the &lt;strong&gt;same&lt;/strong&gt; thing: Variant, Allele, and SNP.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Variant/Allele/SNP:&lt;/strong&gt; Some gene locus that differs from the idea human.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;For example, if 99% of humans have a T at some locus, and you have a G, C, or A, then you have a Variant/Allele/SNP at that locus.&lt;/p&gt;&#xA;&lt;p&gt;Is there intuitive way to explain the difference?&lt;/p&gt;&#xA;" OwnerUserId="9488" LastActivityDate="2021-09-22T14:26:41.850" Title="Variant vs Allele vs SNP" Tags="|genetics|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17721" PostTypeId="2" ParentId="17720" CreationDate="2021-09-17T07:19:49.560" Score="4" Body="&lt;p&gt;Variant = a locus in the genome where there are differences between individuals&lt;/p&gt;&#xA;&lt;p&gt;Allele = one of the possible bases/sequences that can occur at the variant&lt;/p&gt;&#xA;&lt;p&gt;SNP = single nucleotide polymorphism, a type of variant. Here the alleles are just single bases, but other variants can be insertions/deletions of one or more bases.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2021-09-17T07:19:49.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17722" PostTypeId="2" ParentId="17656" CreationDate="2021-09-17T08:01:29.300" Score="1" Body="&lt;p&gt;You might use &lt;code&gt;agat_sp_extract_sequences.pl&lt;/code&gt; from &lt;a href=&quot;https://github.com/NBISweden/AGAT&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGAT&lt;/a&gt; instead.&lt;/p&gt;&#xA;&lt;p&gt;To extract and translate the coding regions:&lt;br /&gt;&#xA;&lt;code&gt;agat_sp_extract_sequences.pl -g infile.gxf -f infile.fasta -p&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;see here for all possibilities with that script: &lt;a href=&quot;https://agat.readthedocs.io/en/latest/tools/agat_sp_extract_sequences.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://agat.readthedocs.io/en/latest/tools/agat_sp_extract_sequences.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Otherwise with the modification suggested by @terdon, your script should work fine.&lt;/p&gt;&#xA;" OwnerUserId="2486" LastActivityDate="2021-09-17T08:01:29.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17723" PostTypeId="1" AcceptedAnswerId="17724" CreationDate="2021-09-17T08:22:46.367" Score="1" ViewCount="2006" Body="&lt;p&gt;I have three bulk RNA-seq samples (test1, test2, test3) without replicates.&lt;/p&gt;&#xA;&lt;p&gt;And I noticed that DEG analysis tools such as DESeq2/edgeR/etc cannot be applied for data with no replicates.&lt;/p&gt;&#xA;&lt;p&gt;So, I just drew a heatmap with expression value (TPM) of genes of interest, using NMF (aheatmap).&lt;/p&gt;&#xA;&lt;p&gt;But now I want to perform DEG analysis across the three samples. Is there anything I can try for the analysis?&lt;/p&gt;&#xA;&lt;p&gt;My TPM data (.csv) looks like below.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9nqEK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9nqEK.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thank you so much.&lt;/p&gt;&#xA;" OwnerUserId="14192" LastEditorUserId="57" LastEditDate="2021-09-19T12:19:46.153" LastActivityDate="2021-09-19T12:19:46.153" Title="Is it possible to do DEG analysis without replicates?" Tags="|r|rna-seq|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17724" PostTypeId="2" ParentId="17723" CreationDate="2021-09-17T10:58:28.853" Score="2" Body="&lt;p&gt;According to &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#can-i-use-deseq2-to-analyze-a-dataset-without-replicates&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2&lt;/a&gt;, you cannot analyze your data without having replicates.&lt;/p&gt;&#xA;&lt;p&gt;In short, you have an experiment where you compare A to B because, in your theory, A is more expressed than B. However, you do the experiment only once. Without repeating the same experiment again under the same circumstances, there's no way to confirm you will have the same results and that your theory is correct.&lt;/p&gt;&#xA;&lt;p&gt;You can still compare the ratio of &lt;code&gt;A/B&lt;/code&gt; but that will not be meaningful in any way since there's not enough statistical power behind it. In your case, doing the ratio of &lt;code&gt;test1/test2&lt;/code&gt;, &lt;code&gt;test2/test3&lt;/code&gt; and &lt;code&gt;test1/test3&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2021-09-17T10:58:28.853" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17725" PostTypeId="2" ParentId="17701" CreationDate="2021-09-17T11:24:38.430" Score="1" Body="&lt;p&gt;Bcftools and samtools both follow the very common &lt;a href=&quot;https://en.wikipedia.org/wiki/Filter_(software)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unix filter&lt;/a&gt; pattern: they read from all input files specified (as non-option plain arguments), or if there are none they read from standard input.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;bcftools call&lt;/code&gt; is no exception:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools call&#xA;&#xA;About:   SNP/indel variant calling from VCF/BCF.&#xA;         To be used in conjunction with bcftools mpileup.&#xA;         [etc]&#xA;Usage:   bcftools call [options] &amp;lt;in.vcf.gz&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So you can write either of the following equivalently:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat file.mpileup | bcftools call [options]&#xA;bcftools call [options] file.mpileup&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This assumes that the file is VCF/BCF as produced by &lt;code&gt;bcftools mpileup&lt;/code&gt;, and not actually textual “mpileup output” as suggested by the filename. If it is the latter that VarScan2 is expecting, you won't be able to share the input files between the two after all.&lt;/p&gt;&#xA;&lt;p&gt;See &lt;a href=&quot;http://www.htslib.org/workflow/#variant-calling&quot; rel=&quot;nofollow noreferrer&quot;&gt;this workflow example&lt;/a&gt;, which demonstrates &lt;code&gt;bcftools call&lt;/code&gt; both with a pipe and a previously made input file.&lt;/p&gt;&#xA;" OwnerUserId="134" LastActivityDate="2021-09-17T11:24:38.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17727" PostTypeId="2" ParentId="17723" CreationDate="2021-09-17T13:18:48.070" Score="3" Body="&lt;p&gt;In this situation, you can apply the strategy in section 2.12 in the &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/vignettes/edgeR/inst/doc/edgeRUsersGuide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;edgeR manual&lt;/a&gt; which basically &amp;quot;makes up&amp;quot; a dispersion estimate and then run the normal DE strategy. This is obviously neither reliable, nor publishable but at least it gives you a list of genes you can use for validation. Treat results with care, focus on genes with decent counts (not low on both conditions of the contrast) and select those with large effect sizes (fod changes). Alternatively, you can run &lt;code&gt;vst&lt;/code&gt; from DESeq2 and then rank them by fold change, again focusing on genes with decent (non-low) counts. Both is not really reliable, so be careful with the results. As you have no information on the variability of the groups the fold changes are of limited value, but if you are forced to go with these data it is better than nothing.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2021-09-17T13:34:44.843" LastActivityDate="2021-09-17T13:34:44.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17729" PostTypeId="1" AcceptedAnswerId="17730" CreationDate="2021-09-17T21:00:02.750" Score="3" ViewCount="111" Body="&lt;p&gt;I would like to define a bedfile for chromosome X, to target only basepairs outside of pseudo-autosomal regions (and going forward other regions, too).&lt;/p&gt;&#xA;&lt;p&gt;First, I wasn't able to find an already prepared bedfile in any of the online repositories (but let me know if it is available somewhere).&lt;/p&gt;&#xA;&lt;p&gt;Second, I compiled the bedfile for all basepairs in chromosome X as a starting point (by looking at the latest assembly at &lt;a href=&quot;https://genome.ucsc.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;genome browser&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chrX    0       156040895       # as defined at https://genome.ucsc.edu/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The genome browser is 1-based and the last base is inclusive, while the bedfile is 0-based and last base is not inclusive, therefore I had to subtract 1 from the bed &lt;code&gt;start&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Afterwards, I wanted to exclude the pseudo-autosomal regions as defined at &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/grc/human&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt;. However, I am not sure if basepair &amp;quot;Start&amp;quot; and &amp;quot;Stop&amp;quot; are 0- or 1-based and inclusive/not inclusive. I might use the same convention used on the genome browser, but I am not sure if that's correct in this case. I wasn't able to find this information on their website or online.&lt;/p&gt;&#xA;&lt;p&gt;Assuming that the NCBI &amp;quot;Start&amp;quot; and &amp;quot;Stop&amp;quot; field follow the UCSC parameters (1-based and inclusive), the resulting bed (PAR regions excluded) might be the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chrX    0       10000&#xA;chrX    2781479 155701382&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: the section on the centromeres has the &amp;quot;Length&amp;quot; parameter, from which I could infer that &amp;quot;Start&amp;quot; and &amp;quot;Stop&amp;quot; base-pairs are both inclusive. At the same time, the &amp;quot;Stop&amp;quot; base-pair of PAR#2 in chromosome X is base-pair 156,030,895, which is also the last base-pair number if we were considering the UCSC genome browser notation (1-based and inclusive). Therefore, it seems to me clear that the same notation is used, and the proposed bed-file should be correct.&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2021-09-17T21:33:22.857" LastActivityDate="2021-09-20T17:55:07.337" Title="Defining a bedfile for chromosome X without pseudo-autosomal regions" Tags="|bed|chromosomes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17730" PostTypeId="2" ParentId="17729" CreationDate="2021-09-17T21:39:26.910" Score="2" Body="&lt;p&gt;The UCSC Genome Browser renders data visually with a one-based, closed index. The underlying data are often stored and presented as files worked with on the command-line, using zero-based, half-open coordinates.&lt;/p&gt;&#xA;&lt;p&gt;This difference in indexing can be important when using the website to view data or when using the download tools to grab a dataset for offline calculations, such as files in BED format.&lt;/p&gt;&#xA;&lt;p&gt;There is a &lt;a href=&quot;https://www.biostars.org/p/84686/&quot; rel=&quot;nofollow noreferrer&quot;&gt;good overview of indexing&lt;/a&gt; on Biostars.&lt;/p&gt;&#xA;&lt;p&gt;To solve this in a general way:&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;http://hgdownload.soe.ucsc.edu/admin/exe/&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC Kent Utilities&lt;/a&gt; &lt;code&gt;fetchChromSizes&lt;/code&gt; tool can be used to retrieve the sizes of chromosomes for your genome assembly of interest, e.g. &lt;code&gt;hg38&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Using this assembly, to build a sorted BED file for &lt;a href=&quot;https://github.com/bedops/bedops&quot; rel=&quot;nofollow noreferrer&quot;&gt;set operations&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ fetchChromSizes hg38 | grep -v '_*_' | awk -v FS=&quot;\t&quot; -v OFS=&quot;\t&quot; '{ print $1, &quot;0&quot;, $&lt;/span&gt;2 }' | sort-bed - &amp;gt; hg38.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Replace &lt;code&gt;hg38&lt;/code&gt; with your assembly of interest.&lt;/p&gt;&#xA;&lt;p&gt;To then filter this for &lt;code&gt;chrX&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ awk '($&lt;/span&gt;1==&amp;quot;chrX&amp;quot;)' hg38.bed &amp;gt; hg38.chrX.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The intervals in &lt;code&gt;hg38.bed&lt;/code&gt; and &lt;code&gt;hg38.chrX.bed&lt;/code&gt; use a zero-based, half-open coordinate system.&lt;/p&gt;&#xA;&lt;p&gt;To then get regions of &lt;code&gt;chrX&lt;/code&gt; without autosomal regions, you could define a sorted BED file containing those regions, which also use zero-based, half-open coordinates:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ echo -e 'chrX\t10000\t2781479\nchrX\t155701382\t156030895' &amp;gt; pseudoautosomal.chrX.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am assuming that the coordinates on Ensembl and NCBI use one-based indexing. Converting to a zero-based index requires subtracting one base from the start positions of these intervals.&lt;/p&gt;&#xA;&lt;p&gt;It would be ideal if these groups were to clarify the indexing they use. I am making this assumption based on &lt;a href=&quot;https://uswest.ensembl.org/info/genome/genebuild/human_PARS.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;how Ensembl defines regions&lt;/a&gt; on &lt;code&gt;chrY&lt;/code&gt;, which start at position &lt;code&gt;1&lt;/code&gt; and not &lt;code&gt;0&lt;/code&gt;, as one would expect to see with a one-based coordinate system.&lt;/p&gt;&#xA;&lt;p&gt;The numbers reported on Ensembl and NCBI are otherwise identical, so I also assume that NCBI's numbers use the same indexing and require the same correction to the start base.&lt;/p&gt;&#xA;&lt;p&gt;In any case, to &lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/set-operations/bedops.html#difference-d-difference&quot; rel=&quot;nofollow noreferrer&quot;&gt;calculate the difference&lt;/a&gt; between these regions and &lt;code&gt;chrX&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops --difference hg38.chrX.bed pseudoautosomal.chrX.bed &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is what my output looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ more answer.bed&#xA;chrX    0   10000&#xA;chrX    2781479 155701382&#xA;chrX    156030895   156040895&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These coordinates are in BED format and so use a &lt;a href=&quot;https://www.biostars.org/p/84686/&quot; rel=&quot;nofollow noreferrer&quot;&gt;zero-based, half-open&lt;/a&gt; coordinate system, by convention.&lt;/p&gt;&#xA;&lt;p&gt;To turn these into one-based, closed coordinates — to query signals over exact regions in the UCSC Genome Browser, for instance — you would add one base to the start positions of each genomic interval.&lt;/p&gt;&#xA;&lt;p&gt;(Edit: Fixed PAR#2 start coordinate per comment.)&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2021-09-20T17:55:07.337" LastActivityDate="2021-09-20T17:55:07.337" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17731" PostTypeId="2" ParentId="17720" CreationDate="2021-09-18T01:58:53.937" Score="1" Body="&lt;p&gt;Just to add on to previous answers, it is important that allele is one of the possible versions or sequences that can happen at a given position.&lt;/p&gt;&#xA;&lt;p&gt;Take SNP. Most SNPs have two alleles, meaning that in the population we only see two possible nucleotides at this position. For instance, everybody either has an A or a G at this position. This type of SNPs are called &lt;strong&gt;biallelic&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In rare cases, a SNP can have three different alleles, like A/G/C. These are called &lt;strong&gt;triallelic&lt;/strong&gt;. So already you can see that &lt;strong&gt;allele&lt;/strong&gt; is different from &lt;strong&gt;SNP&lt;/strong&gt;. They are actually completely different concepts.&lt;/p&gt;&#xA;&lt;p&gt;Also, the word allele has been used for a long time. Scientists first used this to refer to a certain version of genetic variation at one place in a genome. This happened even before we know what is DNA. On the contrary, SNP was only discovered after we can sequence human DNA, which is basically in the past 20-30 years. Therefore sometimes it is hard to distinguish the subtle differences between these terms, because they can carry both original meaning based on primitive understand of heredity as well as newer meaning in the era molecular genetics. It might be helpful to think about what these term are referring to in different time and contexts.&lt;/p&gt;&#xA;" OwnerUserId="4766" LastActivityDate="2021-09-18T01:58:53.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17733" PostTypeId="1" CreationDate="2021-09-18T15:57:25.643" Score="0" ViewCount="692" Body="&lt;p&gt;I have the following settings in PyCharm:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WbEgo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WbEgo.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;BioPython is properly installed:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LVf7F.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LVf7F.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;dssp_test.py&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.PDB import PDBParser&#xA;from Bio.PDB.DSSP import DSSP&#xA;p = PDBParser()&#xA;structure = p.get_structure(&amp;quot;16PK&amp;quot;, &amp;quot;16pk.pdb&amp;quot;)&#xA;model = structure[0]&#xA;dssp = DSSP(model, &amp;quot;16pk.pdb&amp;quot;)&#xA;a_key = list(dssp.keys())[2]&#xA;print(dssp[a_key])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;C:\ProgramData\Miniconda3\python.exe C:/Users/pc/source/repos/DSSP_algoritm_test/dssp_test.py&#xA;Traceback (most recent call last):&#xA;  File &amp;quot;C:\Users\pc\source\repos\DSSP_algoritm_test\dssp_test.py&amp;quot;, line 6, in &amp;lt;module&amp;gt;&#xA;    dssp = DSSP(model, &amp;quot;16pk.pdb&amp;quot;)&#xA;  File &amp;quot;C:\ProgramData\Miniconda3\lib\site-packages\Bio\PDB\DSSP.py&amp;quot;, line 437, in __init__&#xA;    dssp_dict, dssp_keys = dssp_dict_from_pdb_file(in_file, dssp)&#xA;  File &amp;quot;C:\ProgramData\Miniconda3\lib\site-packages\Bio\PDB\DSSP.py&amp;quot;, line 231, in dssp_dict_from_pdb_file&#xA;    p = subprocess.Popen(&#xA;  File &amp;quot;C:\ProgramData\Miniconda3\lib\subprocess.py&amp;quot;, line 947, in __init__&#xA;    self._execute_child(args, executable, preexec_fn, close_fds,&#xA;  File &amp;quot;C:\ProgramData\Miniconda3\lib\subprocess.py&amp;quot;, line 1416, in _execute_child&#xA;    hp, ht, pid, tid = _winapi.CreateProcess(executable, args,&#xA;FileNotFoundError: [WinError 2] The system cannot find the file specified&#xA;&#xA;Process finished with exit code 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Why am I unable to execute DSSP algorithm from BioPython?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="12686" LastEditDate="2021-09-20T14:40:51.353" LastActivityDate="2022-06-07T10:41:01.247" Title="Unable to execute DSSP algorithm from BioPython" Tags="|python|secondary-structure|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17734" PostTypeId="1" AcceptedAnswerId="17747" CreationDate="2021-09-19T04:16:20.020" Score="1" ViewCount="131" Body="&lt;p&gt;I encountered this question in a textbook (source: &lt;a href=&quot;https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.721.2540&amp;amp;rep=rep1&amp;amp;type=pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.721.2540&amp;amp;rep=rep1&amp;amp;type=pdf&lt;/a&gt;) and was trying to workout the solution for it. Was wondering for the last question, wouldn't the probability be 0?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WaTVy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WaTVy.png&quot; alt=&quot;Question&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;My working:&lt;/p&gt;&#xA;&lt;p&gt;START -&amp;gt; M1 -&amp;gt; M2 -&amp;gt; M3 -&amp;gt; M4 -&amp;gt; M5 -&amp;gt; M6 -&amp;gt; M7 -&amp;gt; END&lt;/p&gt;&#xA;&lt;p&gt;Probability Vector [A, T, C, G]&lt;/p&gt;&#xA;&lt;p&gt;M1 : [0.6, 0.4, 0, 0]&lt;/p&gt;&#xA;&lt;p&gt;M2 : [0.6, 0.4, 0, 0]&lt;/p&gt;&#xA;&lt;p&gt;M3 : [0.4, 0.4, 0.2, 0]&lt;/p&gt;&#xA;&lt;p&gt;M4 : [0.6, 0.4, 0, 0]&lt;/p&gt;&#xA;&lt;p&gt;M5 : [0.4, 0.6, 0, 0]&lt;/p&gt;&#xA;&lt;p&gt;M6 : [1, 0, 0, 0]&lt;/p&gt;&#xA;&lt;p&gt;M7 : [0, 0, 0, 1]&lt;/p&gt;&#xA;&lt;p&gt;To generate CTAAAAG = 0 * 0.4 * 0.4 * 0.6 * 0.4 * 1 * 1 = 0&lt;/p&gt;&#xA;" OwnerUserId="14197" LastEditorUserId="73" LastEditDate="2021-09-29T04:46:03.187" LastActivityDate="2021-09-29T04:46:48.247" Title="Generating sequences from HMM" Tags="|homework|hidden-markov-models|multiple-sequence-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17737" PostTypeId="1" CreationDate="2021-09-19T14:47:16.327" Score="0" ViewCount="71" Body="&lt;p&gt;So basically I have RNA-seq reads that were generated from Illumina and Ion Torrent platforms for yeast species. I have seen an article where they compared liver cells of a rat that were sequenced with these two platforms, but they made use of PORT bioinformatics tool for the normalization of the data. This requires input data to be in the form of the Ensembl database. This database only has Saccharomyces cerevisiae annotation files available on it, whereas I have other species that I need annotation files for as well, which are available in RefSeq format from NCBI.&lt;/p&gt;&#xA;&lt;p&gt;So, I am still brand new to doing RNA-expression analyses and have never done anything remotely at this scale. Please could anyone give me advice on the steps involved in analyzing the paired-end and single-end reads together as comparable data sets and then carrying out a complete differential expression analysis on this data. Any advice or guidance would really be appreciated, I am honestly not sure how to approach this at all.&lt;/p&gt;&#xA;" OwnerUserId="13885" LastActivityDate="2021-09-19T14:47:16.327" Title="How to perform a meta-analysis using data consisting of paired-end and single-end reads generated from Illumina and Ion Torrent?" Tags="|rna-seq|differential-expression|normalization|illumina|meta-analysis|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17741" PostTypeId="2" ParentId="17733" CreationDate="2021-09-20T10:09:19.370" Score="1" Body="&lt;p&gt;This looks like a python problem and not a “biopython”problem.&lt;/p&gt;&#xA;&lt;p&gt;Check your file structures. Try giving a full path for 16pk.pdb&lt;/p&gt;&#xA;&lt;p&gt;These might help. &lt;a href=&quot;https://stackoverflow.com/questions/17658856/why-am-i-getting-a-filenotfounderror&quot;&gt;https://stackoverflow.com/questions/17658856/why-am-i-getting-a-filenotfounderror&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/18757127/windowserror-error-2-the-system-cannot-find-the-file-specified-is-not-resol&quot;&gt;https://stackoverflow.com/questions/18757127/windowserror-error-2-the-system-cannot-find-the-file-specified-is-not-resol&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12424" LastActivityDate="2021-09-20T10:09:19.370" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17742" PostTypeId="1" CreationDate="2021-09-20T20:02:09.900" Score="5" ViewCount="87" Body="&lt;p&gt;&lt;code&gt;ATTAAAGGTT TATACCTTCC CAGGTAACAA ACCAACCAAC TTTCGAT...&lt;/code&gt; is part of the 5'UTR of genome of an RNA virus SARS-Cov-2.&lt;/p&gt;&#xA;&lt;p&gt;RNA contains letters &lt;code&gt;C&lt;/code&gt;+&lt;code&gt;G&lt;/code&gt;+&lt;code&gt;A&lt;/code&gt;+&lt;code&gt;U&lt;/code&gt;.&lt;br /&gt;&#xA;DNA contains letters &lt;code&gt;C&lt;/code&gt;+&lt;code&gt;G&lt;/code&gt;+&lt;code&gt;A&lt;/code&gt;+&lt;code&gt;T&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Why are there letters &lt;code&gt;T&lt;/code&gt;, in the SARS-Cov-2 genome, instead of &lt;code&gt;U&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="14210" LastEditorUserId="3848" LastEditDate="2022-02-23T17:49:52.963" LastActivityDate="2022-02-23T17:49:52.963" Title="Why does the SARS-Cov2 genome has letter t" Tags="|genome|phylogenetics|rna|sars-cov-2|covid-19|" AnswerCount="1" CommentCount="1" ClosedDate="2022-02-23T18:03:59.130" ContentLicense="CC BY-SA 4.0" />
  <row Id="17744" PostTypeId="2" ParentId="17742" CreationDate="2021-09-20T21:12:03.940" Score="10" Body="&lt;p&gt;We sequence and therefore typically report assemblies as DNA sequences, even if they're actually RNA.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-09-20T21:12:03.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17746" PostTypeId="2" ParentId="11265" CreationDate="2021-09-21T09:51:40.440" Score="1" Body="&lt;p&gt;The best way to approach it would be to take trimmed reads map them to the bins (i.e. with bowtie2) and estimate the abundance of each bin with samtools by getting the depth of coverage:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.metagenomics.wiki/tools/samtools/breadth-of-coverage&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.metagenomics.wiki/tools/samtools/breadth-of-coverage&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;or you can use QUANT_BIN module from the METAWRAP to do it for you. Follow this for the METAWRAP approach:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/bxlab/metaWRAP/issues/40&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/bxlab/metaWRAP/issues/40&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14212" LastActivityDate="2021-09-21T09:51:40.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17747" PostTypeId="2" ParentId="17734" CreationDate="2021-09-21T12:23:40.867" Score="1" Body="&lt;p&gt;&lt;em&gt;[Comments from other post migrated as answer]&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;The parenthetical statement in the first bullet point says &lt;strong&gt;without corrections&lt;/strong&gt;. In which case, if an unseen nucleotide has a probability of zero, the probability of the sequence is zero. If the unseen nucleotides had the probability of NaN, then the sequence has a probability of NaN, which is a more thorough solution.&lt;/p&gt;&#xA;&lt;p&gt;The correction would be adding a small iota value to the probabilities to be non zero —&lt;strong&gt;fudging&lt;/strong&gt; against the zero outcome you otherwise find, which may be the result of the training being too small. You have 5 training sequences, so the iota would be &amp;lt;1/5. Let's hypothetically assume that if you were to have picked a 6th MSA sequence it would have had the missing value, so you could set the iota at 1/6. The assumption that you may find that element were you to sample twice as many is more statistically sensible, so iota to 1/10.&lt;/p&gt;&#xA;&lt;p&gt;I used the letter iota, which is common parlance for something small, but iota is a really bad in maths as it looks like a 1 with English handwriting or in print: you can call the fudge value anything.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="73" LastEditDate="2021-09-29T04:46:48.247" LastActivityDate="2021-09-29T04:46:48.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17749" PostTypeId="1" CreationDate="2021-09-21T18:15:12.047" Score="2" ViewCount="83" Body="&lt;p&gt;I am working on the UKBB data and trying to specify cases and controls by converting ICD9 codes to PheCodes. I have input data (icdt) where the first column (f.eid) is patient IDs and the remaining columns are ICD9 codes. I am using CreatePhewasTable function in R for the conversion. The code seems working well, But I am not getting enough 'TRUE' values for the PheCode 250.1 and 250.2, which represent PheCode for T1D and T2D. I am not sure what I am overlooking because I am getting most of the 'FALSE' values for T1D and T2D columns (250.1 and 250.2). I am sharing my R script&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(data.table)&#xA;library(dplyr)&#xA;library(PheWAS)&#xA;icdt &amp;lt;- fread(&amp;quot;icd_codes.tab&amp;quot;,header=T,sep=&amp;quot;\t&amp;quot;)&#xA;icdt &amp;lt;- as.data.frame(icdt)&#xA;&#xA;for (i in 215:261) {&#xA;  icd9 &amp;lt;- data.frame(matrix(nrow = dim(icdt)[1], ncol = 2))&#xA;  icd9[,1] &amp;lt;- icdt[,1]&#xA;  colnames(icd9)[1] &amp;lt;- 'f.eid'&#xA;  colnames(icd9)[2] &amp;lt;- 'DiagnosisCodes'&#xA;  icd9[,2] &amp;lt;- icdt[,i]&#xA;  icd9 &amp;lt;- icd9[!rowSums((is.na(icd9))),]&#xA;  icd.codes &amp;lt;- icd9 %&amp;gt;% mutate(DiagnosisCodes = (paste0(substr(DiagnosisCodes,1,3),&amp;quot;.&amp;quot;,substr(DiagnosisCodes,4,nchar(DiagnosisCodes)))))&#xA;  icd.count &amp;lt;-  as.data.frame(count(icd.codes, f.eid, DiagnosisCodes))&#xA;  phenotypes &amp;lt;- createPhewasTable(icd.count, min.code.count = 1, add.exclusions = T, translate=T)&#xA;  phenotypes &amp;lt;- as.data.frame(phenotypes)&#xA;  assign(paste('Phecodes',i,sep=''),phenotypes)&#xA;  myfile &amp;lt;- paste0( &amp;quot;ALL_Phecodes&amp;quot;, i, &amp;quot;.txt&amp;quot;)&#xA;  write.table(phenotypes, file = myfile, sep = &amp;quot;\t&amp;quot;, row.names = FALSE, col.names = TRUE, quote = FALSE, append = FALSE)  &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="14215" LastEditorUserId="3061" LastEditDate="2022-05-09T18:40:36.913" LastActivityDate="2022-05-09T18:40:36.913" Title="Mapping ICD9 codes in UKBB data to PheCodes using PheWAS package" Tags="|r|database|public-databases|data-preprocessing|icd-codes|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17750" PostTypeId="1" CreationDate="2021-09-21T20:36:51.110" Score="1" ViewCount="50" Body="&lt;p&gt;It appers when a gene has a negative strand then I need to reverse the sequence. Unfortunately, I got the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print(&amp;quot;1&amp;quot; + '&amp;quot;' + str(fastaIdx) + '&amp;quot;' + '&amp;quot;' + str(mrna_id) + &amp;quot;'&amp;quot; + db.strand())&#xA;AttributeError: 'FeatureDB' object has no attribute 'strand'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;with the following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import gffutils&#xA;import pyfaidx&#xA;from Bio.Seq import Seq&#xA;from Bio.Alphabet import IUPAC&#xA;&#xA;def getPeptideSeq(fastaIDX, db, mrna_id):&#xA;    print(&amp;quot;Petide Sequence&amp;quot;)&#xA;    return Seq(getCdsSeq(fastaIDX, db, db[mrna_id]), IUPAC.unambiguous_dna).translate()&#xA;&#xA;if __name__ == '__main__':&#xA;    db = gffutils.FeatureDB('annotation.db', keep_order=True)&#xA;    fastaIdx = pyfaidx.Fasta('assembly.fasta')&#xA;    print(&amp;quot;1&amp;quot; + '&amp;quot;' + str(fastaIdx) + '&amp;quot;' + '&amp;quot;' + str(mrna_id) + &amp;quot;'&amp;quot; + db.strand())&#xA;&#xA;        seq = getPeptideSeq(fastaIdx, db, mrna_id)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What did I miss?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance,&lt;/p&gt;&#xA;&lt;p&gt;Best wishes,&lt;/p&gt;&#xA;" OwnerUserId="3329" LastActivityDate="2021-09-21T20:36:51.110" Title="AttributeError: 'FeatureDB' object has no attribute 'strand'" Tags="|python|gffutils|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17751" PostTypeId="1" AcceptedAnswerId="17761" CreationDate="2021-09-22T02:56:21.260" Score="3" ViewCount="192" Body="&lt;p&gt;I'm starting to use Limma to find differential expressed sites within my data and would like to ask if my approach was correct.&lt;/p&gt;&#xA;&lt;p&gt;The data has a control and a treatment set, each containing data of 48 patients, which were measured before treatment (t0) and after treatment (t1). So I have Control-t0 and Control-t1 as well as Treatment-t0 and Treatment-t1, whereas Control-t0 and Treatment-t0 should not show any effect.&#xA;Now I would like to know if there is a significant change in Treatment compared to Control. Therefore, I think, I have to adjust for the baseline (t0), right?&lt;/p&gt;&#xA;&lt;p&gt;The code below is probably the most verbose option Limma provides, but I wanted to use the long method first before I use the shortcuts.&#xA;So would the Interaction-coefficient I defined within my cont-matrix be the difference between Control and Treatment adjusted for the baseline?&#xA;Also, I had to turn of adjusting for multiple testing here as my example data is random&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;&#xA;#Data.frame with columns representing samples and rows different sites/genes&#xA;data_df &amp;lt;- data.frame(replicate(8, sample(100,1000, rep=TRUE)))&#xA;colnames(data_df) &amp;lt;- c(&amp;quot;Patient_01_t0&amp;quot;, &amp;quot;Patient_01_t1&amp;quot;,&#xA;                       &amp;quot;Patient_02_t0&amp;quot;, &amp;quot;Patient_02_t1&amp;quot;,&#xA;                       &amp;quot;Patient_03_t0&amp;quot;, &amp;quot;Patient_03_t1&amp;quot;,&#xA;                       &amp;quot;Patient_04_t0&amp;quot;, &amp;quot;Patient_04_t1&amp;quot;)&#xA;                     &#xA;#Metadata for data_df&#xA;meta &amp;lt;- data.frame(Sample= colnames(data_df),&#xA;                   ID= c(rep(&amp;quot;Patient_01&amp;quot;,2),&#xA;                         rep(&amp;quot;Patient_02&amp;quot;,2),&#xA;                         rep(&amp;quot;Patient_03&amp;quot;,2),&#xA;                         rep(&amp;quot;Patient_04&amp;quot;,2)),&#xA;                   Time= c(rep(c(&amp;quot;t0&amp;quot;,&amp;quot;t1&amp;quot;),4)),&#xA;                   Group=c(rep(&amp;quot;Control&amp;quot;,4),&#xA;                           rep(&amp;quot;Treatment&amp;quot;,4))&#xA;)&#xA;                         &#xA;#Create levels for model-matrix&#xA;time &amp;lt;- factor(meta&lt;span class=&quot;math-container&quot;&gt;$Time, labels=c('t0','t1') )&#xA;group&amp;lt;- factor(meta$&lt;/span&gt;Group, labels=c('Control','Treatment') )&#xA;lev=paste(group,&amp;quot;.&amp;quot;,time,sep=&amp;quot;&amp;quot;)&#xA;lev&#xA;&#xA;&#xA;design &amp;lt;- model.matrix(~0+lev)&#xA;design&#xA;&#xA;#Create contrast matrix with all factors of interest&#xA;cont_matrix &amp;lt;- makeContrasts( t1_VS_t0_Ctrl = levControl.t1 - levControl.t0,&#xA;                              t1_VS_t0_Treat = levTreatment.t1 - levTreatment.t0,&#xA;                              Ctrl_vs_Treat_at_t0= levControl.t0 - levTreatment.t0,&#xA;                              Ctrl_vs_Treat_at_t1= levControl.t1 - levTreatment.t1,&#xA;                              Interaction = (levControl.t1-levControl.t0)-(levTreatment.t1-levTreatment.t0), &#xA;                              levels=design)&#xA;&#xA;#Calculate different expressed sites&#xA;fit &amp;lt;- lmFit(data_df, design) &#xA;fit &amp;lt;- contrasts.fit(fit, cont_matrix) &#xA;fit.eBayes &amp;lt;- eBayes(fit, robust=TRUE)&#xA;&#xA;summary(decideTests(fit.eBayes, adjust.method=&amp;quot;none&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14218" LastActivityDate="2021-09-23T11:38:28.033" Title="Help with Limma-model" Tags="|r|differential-expression|limma|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17752" PostTypeId="1" AcceptedAnswerId="17766" CreationDate="2021-09-22T03:01:17.063" Score="2" ViewCount="58" Body="&lt;p&gt;I am new to bioinformatics and am working on viral metagenomic (virome) datasets and so would like to use &lt;a href=&quot;https://github.com/feargalr/Demovir&quot; rel=&quot;nofollow noreferrer&quot;&gt;Demovir*&lt;/a&gt; for doing taxonomic annotations on my viral contigs.&lt;/p&gt;&#xA;&lt;p&gt;First, I installed all prerequisites in my macOS but I found that Usearch 32-bit version could not run on macOS Big Sur, so I installed all and re-ran &lt;code&gt;demovir.sh my.viral.contig.fasta threads&lt;/code&gt; in ubuntu VM. I could therefore successfully obtained trembl_ublast.viral.u.contigID.txt as an input for R, but I would always get an empty output after performing the demovir.R command of demovir.sh in ubuntu VM.&lt;/p&gt;&#xA;&lt;p&gt;Thus, I took these back to my macOS, used conda to create a virtual environment for R-3.5.1 and installed R using &lt;code&gt;conda install -c r r=3.5.1&lt;/code&gt;, then ran &lt;code&gt;Rscript demovir.R trembl_ublast.viral.u.contigID.txt&lt;/code&gt;. Again, I got an empty output with only showing the headers of five columns.&lt;/p&gt;&#xA;&lt;p&gt;I also tried to modify the demovir.sh script by removing all previous parts I've completed and retaining the last command &lt;code&gt;Rscript $DIR/demovir.R&lt;/code&gt;then ran &lt;code&gt;modified.demvir.sh my.viral.contigs.fasta threads&lt;/code&gt;again, but still got the same result.&lt;/p&gt;&#xA;&lt;p&gt;Hopefully anyone can do me a favour to check and resolve this issue.&lt;/p&gt;&#xA;&lt;p&gt;Many thanks.&lt;/p&gt;&#xA;&lt;p&gt;Ernie&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Quoting the GitHub repository:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Democratic taxonomic classification of viral contigs to Order and&#xA;Family level&lt;/p&gt;&#xA;&lt;p&gt;When performing metagenomic sequencing of Viral-Like Particle (VLP),&#xA;the majority of returned sequences often bare little to no homology to&#xA;reference sequences - Viral Dark Matter. Frequently it may be useful&#xA;to know which viral taxonomic group these novel viruses are likely to&#xA;belong to as this will give information about nucleic acid type, size&#xA;and behaviour.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="14074" LastEditorUserId="3848" LastEditDate="2022-02-28T15:21:42.237" LastActivityDate="2022-02-28T15:21:42.237" Title="Demovir produces an empty output" Tags="|sequence-annotation|covid-19|metagenome|taxonomy|virus|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17753" PostTypeId="1" AcceptedAnswerId="17770" CreationDate="2021-09-22T14:23:27.430" Score="3" ViewCount="53" Body="&lt;p&gt;Imagine we have an experiment for which we would like to minimize the number of purchased primers. Let's look at the layout of a single primer below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/M4Tyz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/M4Tyz.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Assume we have a precomputed set of all necessary &lt;a href=&quot;https://en.wikipedia.org/wiki/Amplicon&quot; rel=&quot;nofollow noreferrer&quot;&gt;amplicons&lt;/a&gt; and we are trying to reduce the costs and find the minimum set of amplicons that tile all regions of interest (ROIs) with some flanking.&lt;/p&gt;&#xA;&lt;p&gt;The algorithm seems to work, but I am not sure it is ideal, so I would like to know how to assert it finds the minimal covering set of amplicons and how to improve the runtime complexity.&lt;/p&gt;&#xA;&lt;p&gt;First, we define the underline representation of a primer/amplicon (I  use the terms interchangeably), and an auxiliary function for maintaining the same colors among different plotting sessions.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from dataclasses import dataclass&#xA;import numpy as np&#xA;&#xA;def rand_color():&#xA;   return &amp;quot;#&amp;quot; + &amp;quot;%06x&amp;quot; % np.random.randint(0, 0xFFFFFF)&#xA;&#xA;@dataclass(eq=True,frozen=True)&#xA;class Primer:&#xA;    left_end:  int      &#xA;    right_end: int&#xA;    left_roi: int&#xA;    right_roi: int&#xA;    color: str&#xA;    level: int&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We then create a random set of primers:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;N = 25; max_coor = 2_000&#xA;min_primer_len, max_primer_len = 200, 500&#xA;min_required_flank_len, max_required_flank_len = 10, 50&#xA;roi_len = 20&#xA;primers = []&#xA;for i in range(N):&#xA;  left_end = np.random.randint(0,max_coor-max_primer_len)&#xA;  primer_length = np.random.randint(min_primer_len,max_primer_len)&#xA;  required_roi_flank = np.random.randint(min_required_flank_len,max_required_flank_len)&#xA;  left_roi = np.random.randint(left_end+required_roi_flank,left_end+primer_length-required_roi_flank-roi_len-1)&#xA;  right_roi = left_roi + roi_len&#xA;  primers.append(Primer(left_end, left_end+primer_length, left_roi, right_roi, color=rand_color(), level=i))&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The algorithm for finding the minimal primer covering set is as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;primers.sort(key=lambda p:p.left_end)&#xA;&#xA;def does_a_cover_b(a,b, flank=0):&#xA;  return a.left_end&amp;lt; b.left_roi-flank and a.right_end&amp;gt; b.right_roi+flank  &#xA;&#xA;p_tmp = primers.copy()&#xA;&#xA;cover = set()&#xA;for p in p_tmp:&#xA;  add_to_cover_set = True&#xA;  for c in cover:&#xA;    if does_a_cover_b(c,p):&#xA;      add_to_cover_set = False&#xA;  &#xA;  if add_to_cover_set:&#xA;    cover.add(p)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now &lt;code&gt;cover&lt;/code&gt; is supposed to be a minimal subset of primers that covers all ROIs. To visualize, we use &lt;code&gt;plot_primer_set&lt;/code&gt; twice:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;from shapely.geometry import Polygon&#xA;&#xA;def plot_primer_set(primer_set):&#xA;  plt.figure(figsize=(4,4), dpi=120)&#xA;  for c in primer_set:&#xA;    p_all = Polygon([(c.left_end,c.level),(c.left_roi,c.level),(c.right_roi,c.level),(c.right_end,c.level)])&#xA;    x,y = p_all.exterior.xy&#xA;    plt.plot(x,y, markersize=7, marker='o', linewidth=5, color=c.color)&#xA;    plt.yticks(np.arange(0, N, 1))&#xA;    axes = plt.axes()&#xA;    axes.set_xlim([-100, max_coor+100])&#xA;&#xA;plot_primer_set(primers)&#xA;plot_primer_set(cover)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Kjzjy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Kjzjy.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/1cnJn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1cnJn.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is this the optimal and correct minimal set?&lt;/li&gt;&#xA;&lt;li&gt;My complexity is &lt;span class=&quot;math-container&quot;&gt;$O(N^2)$&lt;/span&gt;, but I expected it to be &lt;span class=&quot;math-container&quot;&gt;$O(N \log{N})$&lt;/span&gt;, what am I missing?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2021-09-23T10:53:33.593" LastActivityDate="2021-09-26T09:36:25.863" Title="How to optimize the number of amplicons ordered for a PCR wet experiment with several genomic ROIs?" Tags="|python|algorithms|pcr|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17754" PostTypeId="2" ParentId="17720" CreationDate="2021-09-22T14:26:41.850" Score="2" Body="&lt;p&gt;First some basic biology, taking human as an example. Humans have 22 chromosomes (&lt;a href=&quot;https://en.wikipedia.org/wiki/Chromosome#/media/File:NHGRI_human_male_karyotype.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;image source&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4NHfOm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4NHfOm.png&quot; alt=&quot;human karyotype&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you can see (if you squint) in the image, we have two copies of each of the 22. With the exception of the &lt;a href=&quot;https://en.wikipedia.org/wiki/Sex_chromosome&quot; rel=&quot;nofollow noreferrer&quot;&gt;sex chromosomes&lt;/a&gt;, X and Y (seen in the bottom right of the image above), you have the same complement of genes on each of the copies of the chromosome. Traditionally, the term &amp;quot;allele&amp;quot; refers to the sequence of one of the two copies.&lt;/p&gt;&#xA;&lt;p&gt;We inherit one copy of each chromosome from each of our parents. So, if you have inherited a mutation at position &lt;span class=&quot;math-container&quot;&gt;$Ψ$&lt;/span&gt; of chromosome 21 from one of your parents, you will be &lt;a href=&quot;https://en.wikipedia.org/wiki/Zygosity#Heterozygous&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;heterozygous&amp;quot;&lt;/a&gt; for that mutation. This means that of the two copies of chromosome 21 in your cells, one will have the mutant &lt;em&gt;allele&lt;/em&gt; at position &lt;span class=&quot;math-container&quot;&gt;$Ψ$&lt;/span&gt;, and the other the wild-type (i.e. &amp;quot;normal&amp;quot;, although that's possibly a misnomer) &lt;em&gt;allele&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;SNP stands for &amp;quot;single nucleotide polymorphism&amp;quot;. This means that it describes a difference &lt;em&gt;of a single nucleotide&lt;/em&gt; between the genome you are looking at and the &lt;a href=&quot;https://en.wikipedia.org/wiki/Reference_genome&quot; rel=&quot;nofollow noreferrer&quot;&gt;reference genome&lt;/a&gt;. For example, if the reference genome has a G at position Ψ of chromosome 21 and you have a T, then this is a SNP.&lt;/p&gt;&#xA;&lt;p&gt;The term SNP is also commonly used to refer to polymorphic positions. This means specific, single base positions of the genome where variation is common in the human population. Having differences from the reference genome in such polymorphic sites is usually not problematic. The specific use of the term varies a little between subfields. If you're working on population genomics, you are likely to use SNP only for such known polymorphic sites, while others may use &lt;em&gt;SNP&lt;/em&gt; to refer to any single nucleotide variant.&lt;/p&gt;&#xA;&lt;p&gt;Which brings us to &lt;em&gt;variant&lt;/em&gt;. This is a blanket term encompassing any and all forms of variation in a genome. SNPs are variants, but so are larger insertions or deletions (indels), copy number variants, large transposed sequences, the list goes on. You can basically think of &lt;em&gt;variant&lt;/em&gt; as the catch all term used when you don't want or need to use anything more specific.&lt;/p&gt;&#xA;&lt;p&gt;To sum up, we have:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Allele: one of the two copies of a genomic sequence present in (diploid) cells.&lt;/li&gt;&#xA;&lt;li&gt;Variant: a site where the genome you are looking at differs from a reference genome of that species.&lt;/li&gt;&#xA;&lt;li&gt;SNP: a single nucleotide variant often, but not always, in a position of the genome where a significant proportion of the species' individuals show variation.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="298" LastActivityDate="2021-09-22T14:26:41.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17755" PostTypeId="1" AcceptedAnswerId="17757" CreationDate="2021-09-22T14:43:41.893" Score="6" ViewCount="685" Body="&lt;p&gt;BAM files are, at least as far as I know, simply binary compressed versions of SAM files. They have the exact same information and are used in the same way. Why then does the SAM format use a 1-based coordinate system while BAM uses a 0-based one?&lt;/p&gt;&#xA;&lt;p&gt;This is the relevant section of the &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAM format specification&lt;/a&gt; (emphasis mine):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;1-based coordinate system&lt;/strong&gt; A coordinate system where the first base of&#xA;a sequence is one. In this coordinate system, a region is specified by&#xA;a closed interval. For example, the region between the 3rd and the 7th&#xA;bases inclusive is [3, 7]. &lt;strong&gt;The SAM, VCF, GFF and Wiggle formats are&#xA;using the 1-based coordinate system&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;0-based coordinate system&lt;/strong&gt; A&#xA;coordinate system where the first base of a sequence is zero. In this&#xA;coordinate system, a region is specified by a half-closed-half-open&#xA;interval. For example, the region between the 3rd and the 7th bases&#xA;inclusive is [2, 7). &lt;strong&gt;The BAM, BCFv2, BED, and PSL formats are using&#xA;the 0-based coordinate system&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The two coordinate systems are a very common source of error in bioinformatics, so there must be some reason why these two largely equivalent formats use different systems. For that matter, why do the VCF and BCF formats, whose relationship is equivalent to that between SAM and BAM, also do this? I am guessing there is a good reason and it is equally applicable to both pairs of sister formats.&lt;/p&gt;&#xA;&lt;p&gt;Note that the &lt;a href=&quot;http://www.htslib.org/doc/samtools-view.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;samtools&lt;/code&gt; manual&lt;/a&gt; states that &amp;quot;all position coordinates are 1-based&amp;quot; when giving regions, so it must handle the conversion internally. Indeed, I can confirm that this command returns identical md5sums for both inputs:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;md5sum &amp;lt;(samtools view -b sample.bam chr1:36931633-36931871 | samtools view) \ &#xA;       &amp;lt;(samtools view sample.bam chr1:36931633-36931871) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2022-10-13T12:35:29.707" LastActivityDate="2022-10-13T12:37:53.413" Title="Why do SAM and BAM use different coordinate systems?" Tags="|sam|file-formats|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17756" PostTypeId="1" CreationDate="2021-09-22T18:29:57.043" Score="0" ViewCount="1579" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kuE63.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kuE63.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;I'm trying to set the x-axis in my Violin Plot to go by each patient rather than it being scattered.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;VlnPlot(obj, features = c(&amp;quot;classical_basal&amp;quot;), pt.size = 0) + NoLegend() + theme(axis.text.x = element_text(angle = 90, hjust = 1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above code gives me the x-axis but there is no pattern or order.&#xA;I would like the x-axis to be grouped with the HTB, for example ALL of the HTB2867 first, followed by HTB2876, etc...&lt;/p&gt;&#xA;" OwnerUserId="6827" LastEditorUserId="6827" LastEditDate="2021-09-28T19:03:19.130" LastActivityDate="2021-12-04T05:58:45.827" Title="ViolinPlot x-axis" Tags="|r|seurat|single-cell|ggplot2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17757" PostTypeId="2" ParentId="17755" CreationDate="2021-09-22T18:42:16.717" Score="12" Body="&lt;p&gt;Arithmetic on a zero-based coordinate system is less complicated than that on a one-based system, so it appears zero-based is often (not exclusively) used for binary data formats, like BAM or bigBed, or text formats like BED, where computers are used to more efficiently calculate lengths of or set operations on intervals.&lt;/p&gt;&#xA;&lt;p&gt;A more complete answer on the benefits of zero-based accounting is available &lt;a href=&quot;https://cseducators.stackexchange.com/a/5026&quot;&gt;here&lt;/a&gt;, which references the &lt;a href=&quot;https://www.cs.utexas.edu/users/EWD/transcriptions/EWD08xx/EWD831.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;original note&lt;/a&gt; (EWD831) on this subject from Dijkstra.&lt;/p&gt;&#xA;&lt;p&gt;When reading other textual data formats like SAM or VCF, a one-based index is perhaps more &amp;quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/Natural_number#Notation&quot; rel=&quot;nofollow noreferrer&quot;&gt;natural&lt;/a&gt;&amp;quot; for humans used to counting digits on their hands. If you visually inspect an annotation on the closed interval &lt;code&gt;[3,7]&lt;/code&gt; on a line that denotes some abstract chromosome and starts at position &lt;code&gt;1&lt;/code&gt;, it is clear to you where those data points are positioned. Colloquially, you understand that the interval starts on the &lt;em&gt;third&lt;/em&gt; position and ends on the &lt;em&gt;seventh&lt;/em&gt;. Or, perhaps, people would typically label the initial item in this set the &lt;em&gt;first&lt;/em&gt; item, and not the &lt;em&gt;zeroth&lt;/em&gt; item.&lt;/p&gt;&#xA;&lt;p&gt;A one-based index therefore seems to get used often with formats used for visual inspection or presentation of genomic intervals or sequences, such as in genome browsers — or via CLI tools like &lt;code&gt;samtools&lt;/code&gt;, when exporting reads to human-readable SAM.&lt;/p&gt;&#xA;&lt;p&gt;The language used for working with and organizing data can determine choice of indexing, as well. For programs written in C and offshoot languages, including an API like &lt;code&gt;htslib&lt;/code&gt;, binary data is typically worked with as chunks of memory accessed via pointers or arrays, and it is convention to iterate over that data in increments that use a zero-based index. Contrariwise, R's roots in FORTRAN and focus on linear algebraic operations (and matrix indexing) may perhaps explain why that language uses one-based indices for its lists, matrices, and vectors.&lt;/p&gt;&#xA;&lt;p&gt;The designer of the BAM and SAM formats, Heng Li, comments below(*):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I think 1-based is easier for biologists and&#xA;mathematicians while 0-based is easier for C/C++/Java programmers.&#xA;SAM/VCF is for biologists to read. BAM/BCF is for a small number of&#xA;programmers. I wrote the initial specs of sam, bam and bcf2. PS: when&#xA;sam and vcf are stored in memory by htslib, everything is 0-based in&#xA;accordance with C conventions. bam and bcf are largely direct dumps of&#xA;their in-memory representations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;Note: Original comments were unfortunately deleted from answer by someone, but the quotes were taken verbatim at the time they were written.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2021-10-01T22:57:44.393" LastActivityDate="2021-10-01T22:57:44.393" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17758" PostTypeId="2" ParentId="17756" CreationDate="2021-09-23T01:43:58.997" Score="0" Body="&lt;p&gt;It is difficult to say exactly what you would like to achieve without an example.&lt;/p&gt;&#xA;&lt;p&gt;But if you would like an individual axis based on a factor like patient maybe you want to use facet_wrap for the Patient variable like in the example below?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(ggplot2)&#xA;&#xA;data &amp;lt;- iris&#xA;&#xA;ggplot(data, aes(x=Sepal.Width, y=Sepal.Length, fill=Species)) +&#xA;  geom_violin() +&#xA;  facet_wrap(~Species, scales=&amp;quot;free_x&amp;quot;)&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14218" LastActivityDate="2021-09-23T01:43:58.997" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17759" PostTypeId="1" AcceptedAnswerId="17760" CreationDate="2021-09-23T07:06:57.963" Score="3" ViewCount="40" Body="&lt;p&gt;I have a dataset about adundance of urine proteins from cancer patients. I would like to compare these abundance levels to healthy patients' urine proteins. However I don't have any data with healthy patients. Is there any database about it?&lt;/p&gt;&#xA;" OwnerUserId="14227" LastActivityDate="2021-09-23T09:02:31.347" Title="Is there any database about the abundance of proteins in urine of healthy patients?" Tags="|proteins|database|cancer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17760" PostTypeId="2" ParentId="17759" CreationDate="2021-09-23T09:02:31.347" Score="3" Body="&lt;p&gt;If you have access to the UK Biobank dataset, then they have urine data for healthy individuals.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/s41588-020-00757-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41588-020-00757-z&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2021-09-23T09:02:31.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17761" PostTypeId="2" ParentId="17751" CreationDate="2021-09-23T11:38:28.033" Score="3" Body="&lt;p&gt;It looks good to me. One think I could suggest is to see the distribution of your data/model (with limma/voom).&lt;/p&gt;&#xA;&lt;p&gt;After you get the list of your genes with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;top.table &amp;lt;- topTable(fit.eBayes, sort.by = &amp;quot;P&amp;quot;, n = Inf)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Maybe you can check your multidimensional scaling (MDS):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plotMDS(your_data, col = as.numeric(group))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then you can also check with voom&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;y &amp;lt;- voom(data, model.matrix, plot = T)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I hope this helps&lt;/p&gt;&#xA;" OwnerUserId="11884" LastActivityDate="2021-09-23T11:38:28.033" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17762" PostTypeId="1" CreationDate="2021-09-23T12:11:52.727" Score="1" ViewCount="41" Body="&lt;p&gt;I wonder about using The Synthetic Biology Open Language (SBOL) for a project involving the development of a web application to track the lineage of recombinant DNA molecules in a laboratory collection. I am wondering whether SBOL can support this use case, and I could really use some help from community members.&lt;/p&gt;&#xA;&lt;h3&gt;Use case&lt;/h3&gt;&#xA;&lt;p&gt;The ultimate goal is to develop a web application to keep an audit trail of recombinant DNA in a laboratory collection, where sequences are traceable through cloning steps up to their entry point into the collection.&lt;/p&gt;&#xA;&lt;p&gt;For instance, we can imagine we want to document the generation of a new plasmid by combining fragments of two existing plasmids by restriction and ligation. The web application would provide an interface where you can select the input sequences, select the restriction sites and the assembly, and would generate the child sequence. This information would then be stored in a relational database, and would also be exportable to a text file. Essentially, something like the history of a sequence in SnapGene, but using a relational database instead of the file system, and having an open standard to export the information.&lt;/p&gt;&#xA;&lt;h3&gt;Implementation (roughly)&lt;/h3&gt;&#xA;&lt;p&gt;This requires the following steps:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Generating a schema to document genetic engineering steps:&lt;/strong&gt; The most abstract version of a step would just have inputs and output, but there would have to be a specific implementation for each operation. For the restriction and ligation described above, there would be three steps&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Digestion plasmid A: which enzyme was used, and where it cut.&lt;/li&gt;&#xA;&lt;li&gt;Digestion of plasmid B: which enzyme was used, and where it cut.&lt;/li&gt;&#xA;&lt;li&gt;Ligation of pieces: Order of the pieces of the assembly&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;Plasmid A -[restriction]-&amp;gt; Restriction Fragment A&#xA;                                                |-[Ligation]-&amp;gt;Final product&#xA;Plasmid B -[restriction]-&amp;gt; Restriction Fragment B&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Building an API that reads the inputs and description of a step and returns the output:&lt;/strong&gt; For this, I had thought of using the library &lt;a href=&quot;https://pydna.readthedocs.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pydna&lt;/a&gt;, since it seems to cover the initial use cases I could think of.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Building a visual interface to generate this documentation:&lt;/strong&gt; A mockup with minimal functionality can be found &lt;a href=&quot;https://www.genestorian.org/html/web_interface/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Developing a web application managing a database based on the schema mentioned above&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h3&gt;Will SBOL ever cover this use case?&lt;/h3&gt;&#xA;&lt;p&gt;As far as I understand it, SBOL currently provides the most abstract layer of the schema described in the previous section, supported by the &lt;code&gt;&amp;lt;prov:&amp;gt;&lt;/code&gt; tags, so each entity could be related to its ancestors through an &lt;code&gt;Activity&lt;/code&gt; that represents the ligation, the restriction. etc. However, from looking at the documentation there seems to be no schema for any concrete implementation of &lt;code&gt;Activity&lt;/code&gt;. For example, to describe a restriction operation. Restriction is relatively simple, but for PCR there would be the need to also define a schema for the alignment of primers, with potential missmatches etc. Provenances should also support importing entries from public resources (such as genome databases). These concrete &lt;code&gt;Activity&lt;/code&gt; schemas should be readable by an API that would take the inputs and return the output. For cloning steps, this would be performed with &lt;a href=&quot;https://pydna.readthedocs.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pydna&lt;/a&gt; or something similar. For sequences imported from public resources, the provenance would have the url of a web request, and the API would request the sequence, and parse it.&lt;/p&gt;&#xA;&lt;p&gt;From the documentation of &lt;code&gt;prov:Activity&lt;/code&gt;: &lt;em&gt;to capture how an entity was derived, it is expected that any additional information needed will be attached as annotations&lt;/em&gt;. From this, I wanted to ask whether specific schemas representing the cloning steps will ever be part of the SBOL, or should be provided as external files.&lt;/p&gt;&#xA;&lt;h3&gt;How to approach implementation?&lt;/h3&gt;&#xA;&lt;p&gt;My original plan (before realising about &lt;code&gt;&amp;lt;prov&amp;gt;&lt;/code&gt; tags in SBOL) was to do the schema in json, since it's the easiest way to go for a web application. I suppose that the json schema would be easily convertible into xml to eventually be used in SBOL.&lt;/p&gt;&#xA;&lt;p&gt;I still think it would be best to start with json, since I don't think for the nature of this task I can leverage any of the features of SBOL beyond the &lt;code&gt;&amp;lt;prov&amp;gt;&lt;/code&gt; tag. The sequences themselves will be stored in the original import format, and the only new aspect that I will introduce would be the schemas for cloning steps. Once I have a minimal prototype of the schemas and I have tested the application with users, I could come back to these forums and discuss the schema and potential changes that can be made to it, if it is to be included in SBOL.&lt;/p&gt;&#xA;&lt;p&gt;If you have some thoughts in any of these points, I would really appreciate any feedback.&lt;/p&gt;&#xA;" OwnerUserId="14223" LastEditorUserId="57" LastEditDate="2021-09-26T16:59:09.707" LastActivityDate="2021-09-29T04:17:25.750" Title="Using SBOL for lineages of recombinant DNA molecules and strains" Tags="|sbol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17763" PostTypeId="2" ParentId="17762" CreationDate="2021-09-23T15:22:08.903" Score="1" Body="&lt;p&gt;SBOL 3 is well suited for an application like this, though you may wish to consider some of the specifics (e.g., whether to use a conventional database or an RDF triple store), and has been used for similar (though not exactly the same) workflows in other projects.&lt;/p&gt;&#xA;&lt;p&gt;To some of your specific questions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In addition to tracking the provenance relationships, SubComponent relations can be used to track the inclusion of parts within your final build. The sourceLocation field can be used to express the difference between the original part and a post-assembly trimmed version when there are changes in the reaction.&lt;/li&gt;&#xA;&lt;li&gt;The assembly plans and reactions can be represented not just with activities, but using SBOL Interaction objects.&lt;/li&gt;&#xA;&lt;li&gt;SBOL Implementation objects can be used for tracking specific aliquots as well as the designs that are intended to be included in them.&lt;/li&gt;&#xA;&lt;li&gt;If you still need to enhance Activity records, you can either use the base extension capabilities in pySBOL3 or (better) use &lt;a href=&quot;https://github.com/SynBioDex/sbol_factory/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SBOL Factory&lt;/a&gt; to create extension classes via an ontology.&lt;/li&gt;&#xA;&lt;li&gt;With regards to serialization format, since SBOL 3 is pure RDF, you can use any RDF serialization you want, including &lt;a href=&quot;https://json-ld.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;JSON-LD&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;There's a lot to unpack here, so I would suggest that if you want to dig deeper into any of these specific responses that you ask new questions focused on individual aspects.&lt;/p&gt;&#xA;&lt;p&gt;Here is an example in prototype work on protocols:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/SD2E/paml/blob/main/paml/paml.ttl&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/SD2E/paml/blob/main/paml/paml.ttl&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4869" LastEditorUserId="73" LastEditDate="2021-09-29T04:17:25.750" LastActivityDate="2021-09-29T04:17:25.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17765" PostTypeId="1" AcceptedAnswerId="17785" CreationDate="2021-09-24T19:38:55.720" Score="0" ViewCount="258" Body="&lt;p&gt;I'm trying to run the Non-Negative Matrix Factorization(NMF) for my Gene Expression dataset which was originally in matrix form. But it throws errors as given below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;expr &amp;lt;- read_excel(&amp;quot;./expr.xlsx&amp;quot;)&#xA;expr&amp;lt;- as.matrix(expr)&#xA;&#xA;#removing the first column containing gene symbols before clustering&#xA;expr_rm &amp;lt;- expr[,-1]&#xA;class(expr_rm)&#xA;[1] &amp;quot;matrix&amp;quot; &amp;quot;array&amp;quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So converted these expr_rm to numeric:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;expr_rm &amp;lt;- as.numeric(expr_rm)&#xA;&#xA;#removing negative values for running NMF&#xA;eps &amp;lt;- .Machine$double.eps&#xA;expr_rm[expr_rm&amp;lt;=0] = eps&#xA;&#xA;estimate &amp;lt;- NMF::nmf(expr_rm, 3, nrun = 10, method = &amp;quot;lee&amp;quot;, seed = 123)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error: NMF::nmf - 10/10 fit(s) threw an error.&#xA;Error(s) thrown:&#xA;run #1: 'x' must be numeric&#xA;Timing stopped at: 1.95 0.14 4.4*&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;cluster &amp;lt;- cbind(names(NMF::predict(estimate)), NMF::predict(estimate))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error in h(simpleError(msg, call)) :&#xA;error in evaluating the argument 'object' in selecting a method for function 'predict': object 'estimate' not found&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;result &amp;lt;- list(estimate, cluster)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error: object 'estimate' not found&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;For clarity, my expr_rm has row indices as 1, 2, 3, 4,.. which indicates different genes and column headings as samples.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Please suggest a solution for this soon!&lt;/p&gt;&#xA;&lt;p&gt;Additional information (requested from comments):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;summary(expr_rm) - before conversion &#xA; Sample_C44_0001.genes.results Sample_C44_0002.genes.results Sample_C44_0003.genes.results Sample_C44_0005.genes.results &#xA; Length:145   Length:145 Length:145  Length:145  Class :character&#xA;&#xA; &amp;gt; summary(expr_rm) - after numeric type conversion&#xA;   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. &#xA;  0.000   1.808   4.264   4.220   6.333  16.246&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If in expr_rm is data.matrix() also, it gives the error like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; median_centered_data&amp;lt;- expr_rm - med&#xA;Error in expr_rm - med : non-numeric argument to binary operator &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14209" LastEditorUserId="73" LastEditDate="2021-09-29T04:35:23.277" LastActivityDate="2021-09-29T04:36:00.260" Title="NMF clustering not giving results" Tags="|r|genomics|gene-expression|clustering|modelling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17766" PostTypeId="2" ParentId="17752" CreationDate="2021-09-25T11:35:02.923" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;maybe you can open the demovir.R file with Rstudio and execute line by line to check where the failure is. - @zorbax&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Thanks for your suggestion. I eventually got a correct final output already.&lt;/p&gt;&#xA;&lt;p&gt;The key issue was Prodigal would add extra strings after my original contig IDs, so it would cause R not recongnising my viral contigs. Therefore, I added a flag &lt;code&gt;-trunclabels&lt;/code&gt; in Usearch (after e-value flag) to truncate my contig names. Finally, I re-ran the demovir.sh in ubuntu VM and it has worked well.&lt;/p&gt;&#xA;&lt;p&gt;Ernie&lt;/p&gt;&#xA;" OwnerUserId="14074" LastEditorUserId="73" LastEditDate="2021-09-29T04:27:48.183" LastActivityDate="2021-09-29T04:27:48.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17767" PostTypeId="1" CreationDate="2021-09-25T14:22:35.857" Score="3" ViewCount="84" Body="&lt;p&gt;Hi there I have generated a counts table of samples I need to compare by differential expression analysis.&lt;/p&gt;&#xA;&lt;p&gt;The layout of the counts table is as follows:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;Gene_id&lt;/th&gt;&#xA;&lt;th&gt;Sample_A_r1&lt;/th&gt;&#xA;&lt;th&gt;Sample_A_r2&lt;/th&gt;&#xA;&lt;th&gt;Sample_B_r1&lt;/th&gt;&#xA;&lt;th&gt;Sample_B_r2&lt;/th&gt;&#xA;&lt;th&gt;Sample_AB_r1&lt;/th&gt;&#xA;&lt;th&gt;Sample_AB_r2&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;KLTH001&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;td&gt;5&lt;/td&gt;&#xA;&lt;td&gt;8&lt;/td&gt;&#xA;&lt;td&gt;3&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;KLTH002&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;5&lt;/td&gt;&#xA;&lt;td&gt;3&lt;/td&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;7&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;YL99812&lt;/td&gt;&#xA;&lt;td&gt;6&lt;/td&gt;&#xA;&lt;td&gt;4&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;12&lt;/td&gt;&#xA;&lt;td&gt;14&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;YL99813&lt;/td&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;7&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;15&lt;/td&gt;&#xA;&lt;td&gt;19&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;So here, for the differential expression analysis, I only want to compare for example the genes starting with &amp;quot;YL-&amp;quot;. So in a new file I want the rows including the first row which is &amp;quot;Gene_id&amp;quot; and the sample names e.g. &amp;quot;Sample_A_r1&amp;quot; and then only the rows with genes starting with &amp;quot;YL-&amp;quot; in their names. This will allow me to only compare the changes between Sample_A alone versus when it was combined with Sample_B, or in the table Sample_AB.&lt;/p&gt;&#xA;&lt;p&gt;If anyone can please give me an indication of how to do this in linux I would really appreciate it. I have seen other similar posts, involving &lt;code&gt;grep&lt;/code&gt; and &lt;code&gt;awk&lt;/code&gt; I am just not sure of exactly how to do this. I am still quite new to linux. Any help will be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="13885" LastEditorUserId="13885" LastEditDate="2021-09-25T14:33:07.960" LastActivityDate="2021-09-26T21:24:47.470" Title="How to copy only certain counts for genes in tsv file to new file in linux" Tags="|linux|filtering|awk|grep|featurecounts|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17768" PostTypeId="2" ParentId="14738" CreationDate="2021-09-25T15:07:46.940" Score="2" Body="&lt;p&gt;Consider &lt;strong&gt;mmJSON&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Each of the three PDB sites had the same problem: using mmCIF files in web-based viewers is inefficient. To solve this problem three new file formats were introduced:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;MMTF at RCSB,&lt;/li&gt;&#xA;&lt;li&gt;BinaryCIF at PDBe,&lt;/li&gt;&#xA;&lt;li&gt;mmJSON at PDBj.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;MMTF&lt;/strong&gt; has the smallest files – it's aggressively optimized for size.&#xA;It is based on MessagePack, but on top of it uses a number of custom compression schemes, which make the format rather complex.&#xA;Usually it's not a problem – MMTF developers made an effort to provide libraries for reading MMTF files in the most popular programming languages. But you need to install these libraries.&#xA;To make the file smaller, MMTF doesn't contain all the metadata from mmCIF, which may be a blocker in some cases.&#xA;What is nice is that the MMTF developers reached out to other developers in this field, organized some gathering and they were taking comments and suggestions from the outside.&#xA;As the result, a few popular programs such as PyMOL and JMol can also read MMTF.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;BinaryCIF&lt;/strong&gt; is clearly based on (or at least inspired by) MMTF.&#xA;So it's also built on top of MessagePack with a number of custom compression schemes.&#xA;It includes all the mmCIF metadata, which makes the files slightly bigger, but I think it's a good trade-off.&#xA;Initially there was only TypeScript code for reading it, now I see that there are also implementations in Java in Python.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;mmJSON&lt;/strong&gt; is 1:1 equivalent of mmCIF, but uses JSON instead of CIF.&#xA;There is no extra compression added, but having the data from mmCIF re-ordered column-wise makes gzip compression much more efficient (this was also noted by MMTF developers).&#xA;If the file is stored as json.gz or downloaded in web browser (with the usual compression between the server and the client) its size is 30-40% of gzipped mmCIF. It's not as small as MMTF and BinaryCIF (which are several times smaller than mmCIF) and this is the only downside.&lt;/p&gt;&#xA;&lt;p&gt;The advantage is that you can read in it any language like all other JSON files.&#xA;No need to install/update MessagePack + MMTF/BinaryCIF libraries.&lt;/p&gt;&#xA;&lt;p&gt;A few years ago I compared reading MMTF and mmJSON in Python – reading mmJSON was order of magnitude faster. It may be very different with libraries in other languages, but generally JSON parsers are fast.&#xA;The fastest ones parse GBs per second. Reading binary formats can be faster, but decompression schemes can also make it slower.&#xA;Anyway, all the three formats are fast enough to read.&lt;/p&gt;&#xA;&lt;p&gt;Sadly, I don't think that mmJSON is as popular as it should be.&#xA;Perhaps I'm the only person using it outside of PDBj.&#xA;That's why I'm writing this answer.&lt;/p&gt;&#xA;&lt;p&gt;The mmJSON files are available at:&#xA;&lt;a href=&quot;https://pdbj.org/help/mmjson&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pdbj.org/help/mmjson&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="266" LastEditorUserId="266" LastEditDate="2021-09-25T15:17:40.513" LastActivityDate="2021-09-25T15:17:40.513" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17769" PostTypeId="2" ParentId="17767" CreationDate="2021-09-25T17:42:13.390" Score="3" Body="&lt;p&gt;Something like this should do job:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# creating a new file with the first line of the original    &#xA;head -1 foo.txt &amp;gt; bar.txt&#xA;&#xA;# &amp;quot;&amp;gt;&amp;gt;&amp;quot; appends to the file on the right hand side&#xA;# &amp;quot;^&amp;quot; dictates that the search term should be at the beginning of a line&#xA;grep &amp;quot;^YL&amp;quot; foo.txt &amp;gt;&amp;gt; bar.txt &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2021-09-26T15:52:59.473" LastActivityDate="2021-09-26T15:52:59.473" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17770" PostTypeId="2" ParentId="17753" CreationDate="2021-09-25T18:55:28.173" Score="1" Body="&lt;p&gt;I ended up implementing the following greedy algorithm that should work fairly good for any &lt;a href=&quot;https://en.wikipedia.org/wiki/Binary_relation&quot; rel=&quot;nofollow noreferrer&quot;&gt;binary relation&lt;/a&gt; even if it is not transitive nor symmetric.&lt;/p&gt;&#xA;&lt;p&gt;Assume you have the following binary relation (you may plug in any function you want):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def does_a_cover_b(a,b, min_dist=30, max_dist=130):&#xA;  return min_dist &amp;lt; b.left_roi - a.left_end &amp;lt; max_dist or min_dist &amp;lt; a.right_end - b.right_roi &amp;lt; max_dist&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Using the auxiliary functions to define the binary relation, draw the graph, and build the graph:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import networkx as nx&#xA;&#xA;def does_a_cover_b(a,b, min_dist=30, max_dist=130):&#xA;  return min_dist &amp;lt; b.left_roi - a.left_end &amp;lt; max_dist or min_dist &amp;lt; a.right_end - b.right_roi &amp;lt; max_dist&#xA;&#xA;def draw(G):&#xA;  pos = nx.spring_layout(G,scale=2)&#xA;  nx.draw(G, pos, with_labels=True, font_weight='bold', node_size=750, width=2)&#xA;&#xA;def build_a_graph(primer_list):&#xA;  G = nx.DiGraph()&#xA;&#xA;  for i,p_i in enumerate(primer_list):&#xA;    G.add_node(i, data=p_i)&#xA;&#xA;  for i,p_i in enumerate(primer_list):&#xA;    for j,p_j in enumerate(primer_list):&#xA;      if i == j:&#xA;        continue&#xA;      if does_a_cover_b(p_i, p_j):&#xA;        G.add_edge(i,j)&#xA;  return G&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;we may implement the following algorithm:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;p_set = set(primers)&#xA;cover_set = set()&#xA;covered_primers = set()&#xA;while len(covered_primers)&amp;lt;N and len(p_set)&amp;gt;0:&#xA;  G = build_a_graph(p_set)&#xA;  nodes =sorted(G.out_degree, key=lambda item: item[1], reverse=True)&#xA;  node = nodes[0][0]&#xA;  absorber = G.nodes[node]['data']&#xA;  p_set.remove(absorber)&#xA;  cover_set.add(absorber)&#xA;  covered_primers.add(absorber)&#xA;&#xA;  for edge in G.out_edges(node):&#xA;    primer = G.nodes[edge[1]]['data']&#xA;    p_set.remove(primer)&#xA;    covered_primers.add(primer)&#xA;&#xA;print(f&amp;quot;Used a cover set of length {len(cover_set)} primers to cover the total primer set with {len(covered_primers)} primers&amp;quot;)&#xA;plot_primer_set(primers)&#xA;plot_primer_set(cover_set)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ePf7n.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ePf7n.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3qHvc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3qHvc.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2021-09-26T09:36:25.863" LastActivityDate="2021-09-26T09:36:25.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17772" PostTypeId="2" ParentId="17767" CreationDate="2021-09-26T10:42:19.977" Score="1" Body="&lt;p&gt;One way using &lt;a href=&quot;/questions/tagged/awk&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; rel=&quot;tag&quot;&gt;awk&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'NR==1 || /^YL/ { print $0 }' file.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The default action is to print the line/record, so all you need is the condition:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'NR==1 || /^YL/' file.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2021-09-26T10:42:19.977" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17773" PostTypeId="1" AcceptedAnswerId="17786" CreationDate="2021-09-26T15:51:18.377" Score="1" ViewCount="37" Body="&lt;p&gt;I want to do a GWAS analysis using the differentially expressed genes that I got from a differential analysis. The purpose is the identification of biomarkers.&lt;/p&gt;&#xA;&lt;p&gt;It's my first time using GWAS, can you please suggest to me a pipeline to use?&lt;/p&gt;&#xA;" OwnerUserId="12578" LastEditorUserId="73" LastEditDate="2021-09-29T04:19:10.340" LastActivityDate="2021-09-29T16:30:24.487" Title="Gwas analysis studies using the result of differential analysis" Tags="|gwas|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17774" PostTypeId="1" AcceptedAnswerId="17775" CreationDate="2021-09-26T16:52:45.010" Score="1" ViewCount="132" Body="&lt;p&gt;When looking for repeat patterns in a DNA sequence, we can look for the pattern and its reverse compliment with up to &lt;code&gt;d&lt;/code&gt; mismatches. However, why do we look for reverse compliments if we're analyzing a single DNA strand?&lt;/p&gt;&#xA;&lt;p&gt;I thought about it and my guess is that this lets us analyze both strands of DNA. For example, if we find the reverse compliment at position &lt;code&gt;x&lt;/code&gt;, that means that the other strand of DNA has the pattern at position &lt;code&gt;x - pattern.length&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Is that correct?&lt;/p&gt;&#xA;" OwnerUserId="14160" LastActivityDate="2021-09-26T17:43:00.057" Title="Why do we count reverse compliments when counting kmers in a single DNA strand?" Tags="|k-mer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17775" PostTypeId="2" ParentId="17774" CreationDate="2021-09-26T17:43:00.057" Score="3" Body="&lt;p&gt;When we sequence genomes, the individual reads come from both strands, and that also applies for assembled sequences* (contigs / scaffolds). Therefore for most of the genome assemblies out there, strandeness of individual scaffolds is actually an arbitrary property, and reverse complementing any sequence will generate an equally valid representation (with reverted coordinates).&lt;/p&gt;&#xA;&lt;p&gt;The coordinates of the reverse complementary k-mer matches depend on the notation. Usually, the notations are still in the coordinates of the reference, for example, blast gives for reverse complimentary strand matches the &lt;code&gt;from&lt;/code&gt; subject coordinate greater than the &lt;code&gt;to&lt;/code&gt; coordinate. Some others are denoting &amp;quot;the leftmost&amp;quot; position and strand encoded in a different column (e.g. &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;sam&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;*perhaps with the exception of chromosomal level assemblies&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2021-09-26T17:43:00.057" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17776" PostTypeId="2" ParentId="17767" CreationDate="2021-09-26T21:24:47.470" Score="1" Body="&lt;p&gt;Alternatively, you can use &lt;code&gt;grep&lt;/code&gt; like so to remove all lines starting with &lt;code&gt;KLTH&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -v '^KLTH' in_file &amp;gt; out_file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;echo &amp;quot;Gene_id\nKLTH001\nYL99812&amp;quot; | grep -v '^KLTH'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Prints:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Gene_id&#xA;YL99812&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here, GNU &lt;a href=&quot;https://www.gnu.org/software/grep/manual/grep.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;grep&lt;/code&gt;&lt;/a&gt; uses option:&lt;br /&gt;&#xA;&lt;code&gt;-v&lt;/code&gt; : Print lines that do &lt;strong&gt;not&lt;/strong&gt; match.&lt;/p&gt;&#xA;&lt;p&gt;In the regular expression (regex):&lt;br /&gt;&#xA;&lt;code&gt;^&lt;/code&gt; : Match the beginning of the line.&lt;/p&gt;&#xA;" OwnerUserId="6251" LastActivityDate="2021-09-26T21:24:47.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17778" PostTypeId="1" CreationDate="2021-09-27T21:59:11.430" Score="1" ViewCount="357" Body="&lt;p&gt;Given two sequences, I know we can determine optimal alignment of sequence with the Needleman–Wunsch algorithm. But is it possible that two alignments have multiple optimal alignment (I guess it would mean that multiple alignement have the same score)?&lt;/p&gt;&#xA;&lt;p&gt;And if possible, how would you find them via an algorithm? Just looking for a general idea on how to go about it, I can't see how you could do it beside by checking for all possible combinations.&lt;/p&gt;&#xA;" OwnerUserId="14258" LastEditorUserId="73" LastEditDate="2021-09-29T04:02:14.890" LastActivityDate="2021-09-29T04:02:14.890" Title="Given two ADN sequence, can there be multiple optimal alignement? And how to optimally find them?" Tags="|sequence-alignment|algorithms|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17779" PostTypeId="1" CreationDate="2021-09-27T22:03:03.427" Score="2" ViewCount="19" Body="&lt;p&gt;Is there a basic example anywhere that explains how to create a no-frills view&#xA;config to view the contact map for a non-human genome assembly using the&#xA;server-based viewer at &lt;a href=&quot;http://higlass.io&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://higlass.io&lt;/a&gt;?&lt;/p&gt;&#xA;&lt;p&gt;I believe I have a valid .mcool file for the contact map. But looking at the&#xA;description of view config files it seems clear those are not created by hand.&#xA;I haven't been able to find any place that describes how to create one.&lt;/p&gt;&#xA;&lt;p&gt;Any help would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="14257" LastActivityDate="2021-09-27T22:03:03.427" Title="How can I create a no-frills view config file for HiGlass?" Tags="|genomics|visualization|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17780" PostTypeId="1" CreationDate="2021-09-28T01:59:18.183" Score="2" ViewCount="64" Body="&lt;p&gt;Is there an appropriate statistical or performance test to perform when I want to compare genomic intervals between 2 samples. Sample A is the the benchmark of comparing the intervals and sample B is the sample in question. What I am really looking for is not only how many intervals from sample B did indeed overlap with sample A but additionally what intervals from sample B has intervals that don't overlap with sample A and also those intervals in sample A that don't overlap with B and being able to measure the performance of how accurate sample B intervals overlap with sample A. This in some sense, can be translated into the following:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;True positive: intervals from sample B that overlap with sample A.&lt;/li&gt;&#xA;&lt;li&gt;False positive: intervals from sample B that don't overlap with sample A.&lt;/li&gt;&#xA;&lt;li&gt;False negative: intervals from sample A that don't overlap with sample B.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;If I consider the above schema, then I will not be able to do any of the performance and statistical test (ROC curve or accuracy) because it is not possible to know what the true negative and the true negative are needed parameters. Then, what would should I be using to be able to incorporate the above schema and still apply performance and statistical testing.&lt;/p&gt;&#xA;&lt;p&gt;I highly appreciate any help or pointers in advance.&lt;/p&gt;&#xA;" OwnerUserId="9407" LastActivityDate="2023-06-23T22:07:20.857" Title="Statistical or performance test for genomic intervals between 2 samples" Tags="|statistics|genomics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17782" PostTypeId="1" CreationDate="2021-09-28T15:48:45.310" Score="0" ViewCount="429" Body="&lt;p&gt;I have a CS background, but am a bioinformatics neophyte&lt;/p&gt;&#xA;&lt;p&gt;I did a full genome sequence which provided me with ~100GB of files (SNP VCF, Indel VCF, BAM, Indel TBI, SNP TBI, BAM BAI, CNV VCF, CNV TBI, CV VCF, SV TBI, FASTQ R1, and FASTQ R2).&lt;/p&gt;&#xA;&lt;p&gt;I'd like to search the genome for a pathogenic variant of a known gene. What tools do I need to do this, or where do I start?&lt;/p&gt;&#xA;" OwnerUserId="14262" LastEditorUserId="7340" LastEditDate="2021-11-02T17:24:31.303" LastActivityDate="2021-11-02T17:24:31.303" Title="If I have data from a full genome sequence, how can I search for a known pathogenic variant of a known gene?" Tags="|vcf|software-recommendation|genetics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17783" PostTypeId="2" ParentId="17778" CreationDate="2021-09-28T20:08:21.377" Score="1" Body="&lt;p&gt;As Needleman-Wunsch is a Dynamic Programming algorithm you do need to complete the entire matrix to find the optimal solution and it is possible to have multiple best alignments.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The algorithm assigns a score to every possible alignment, and the purpose of the algorithm is to find all possible alignments having the highest score.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Needleman%E2%80%93Wunsch_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Needleman%E2%80%93Wunsch_algorithm&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="964" LastActivityDate="2021-09-28T20:08:21.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17784" PostTypeId="2" ParentId="17756" CreationDate="2021-09-29T03:57:59.503" Score="2" Body="&lt;p&gt;Seurat does violin plots using ggplot, which orders based on the factor levels. You need to re-factor your grouping category so that the levels matches your preferred order:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# default level sort order is alphabetical&#xA;obj[[&amp;quot;grouping&amp;quot;]] &amp;lt;- factor(unlist(obj[[&amp;quot;grouping&amp;quot;]])) # refactor&#xA;VlnPlot(obj, group.by = &amp;quot;grouping&amp;quot;, features = c(&amp;quot;classical_basal&amp;quot;),&#xA;        pt.size = 0) +&#xA;  NoLegend() + theme(axis.text.x = element_text(angle = 90, hjust = 1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-12-04T05:58:45.827" LastActivityDate="2021-12-04T05:58:45.827" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17785" PostTypeId="2" ParentId="17765" CreationDate="2021-09-29T04:36:00.260" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @maximilian-press, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Sometimes casting to &lt;code&gt;as.numeric()&lt;/code&gt; will convert from two-dimensional matrix to one-dimensional vector. Based on the output you put into comments (which would be helpful in the question), I wonder if this has occurred and caused an issue. For &lt;code&gt;data.matrix&lt;/code&gt;, I suggest reading docs for that function- it should have returned only numeric, such that your &lt;code&gt;med&lt;/code&gt; variable is likely not numeric, or there is a problem with your data. Possibly the tidyverse &lt;code&gt;tibble&lt;/code&gt; created by &lt;code&gt;read_excel&lt;/code&gt; might be the issue. I don't know enough tidyverse to say what exactly is going on.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-09-29T04:36:00.260" CommentCount="0" CommunityOwnedDate="2021-09-29T04:36:00.260" ContentLicense="CC BY-SA 4.0" />
  <row Id="17786" PostTypeId="2" ParentId="17773" CreationDate="2021-09-29T16:30:24.487" Score="0" Body="&lt;p&gt;I am not an expert on recent packages but I would suggest using &lt;a href=&quot;http://www.bios.unc.edu/research/genomic_software/Matrix_eQTL/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Matrix eQTL&lt;/a&gt; for purposes of the statistical analysis.&lt;/p&gt;&#xA;&lt;p&gt;As mentioned in the comments I think that it is worth testing &lt;em&gt;all&lt;/em&gt; transcripts for eQTL associations. This will additionally give some idea of the false discovery rate. It is possible that testing &amp;quot;candidate gene&amp;quot; differentially expressed genes will lead to a lot of false positives relative to the background level of associations.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-09-29T16:30:24.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17787" PostTypeId="1" CreationDate="2021-09-29T16:39:36.443" Score="0" ViewCount="139" Body="&lt;p&gt;Our lab has been using bcl2fastq v2.20.0.422 to demultiplex RNA-seq data sequenced on an Illumina Novaseq machine on a beefy EC2 instance and we've run into the strange problem: namely that while demultiplexing is very fast, generating the stats files are unbearably slow.&lt;/p&gt;&#xA;&lt;p&gt;In a recent example, we demultiplexed one lane of a Novaseq run (total Fastq.gz files were 464 GB) on a m5.24xlarge instance (with 96 vCPUs) with the data being read and written to the same EFS mount. The demultiplexing took only ~70 minutes, while generating the Stats files took over 4 hours (14,557 seconds to be exact)!&lt;/p&gt;&#xA;&lt;p&gt;Running with log level INFO, the demultiplexing progressed quickly as expected, all Fastq files were generated, and the bcl2fastq output the message:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;INFO: Created InterOp file '&amp;quot;./InterOp/IndexMetricsOut.bin&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and sat for hours using &amp;lt;1% CPU writing to &lt;code&gt;IndexMetricsOut.bin&lt;/code&gt; &lt;em&gt;very slowly&lt;/em&gt;. Tracking the size of the &lt;code&gt;IndexMetricsOut.bin&lt;/code&gt; file showed that it was being continuously written to at the rate of about 4kb per second. It did eventually finish (final size 49M) and bcl2fastq reported that it completed successfully, but it seems crazy to me that this stage could take so much longer than actually demultiplexing almost half a TB of data.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have any ideas &lt;em&gt;why&lt;/em&gt; the stats files are taking so long to generate, and what I could do about it? Additionally, since the fastq files seem to be completed, is it safe to just move on to alignment without waiting for bcl2fastq to finish?&lt;/p&gt;&#xA;&lt;p&gt;NOTE: this question was crossposted on Biostars: &lt;a href=&quot;http://biostars.org/p/9491612&quot; rel=&quot;nofollow noreferrer&quot;&gt;biostars.org/p/9491612&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3573" LastEditorUserId="3573" LastEditDate="2021-09-29T18:37:34.253" LastActivityDate="2021-09-29T18:37:34.253" Title="Why is bcl2fastq2 taking so long to calculate stats?" Tags="|rna-seq|ngs|illumina|demultiplexing|aws|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17788" PostTypeId="1" CreationDate="2021-09-30T03:27:23.600" Score="0" ViewCount="638" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vr0D4.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vr0D4.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Did anyone know how to find the differential expressed genes within pseudotime trajectory with Seurat cluster? In monocle tutorial, we can use BEAM to find the differential expressed genes between the two branches even with the states difference. But how can I find the differential expressed genes with seurat culster enven with the genotype diffence? Thank you so much!&lt;/p&gt;&#xA;" OwnerUserId="3171" LastActivityDate="2023-02-24T00:08:00.900" Title="How to find differential expressed genes within pseudotime trajectory with Seurat cluster?" Tags="|single-cell|monocle|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17789" PostTypeId="2" ParentId="17788" CreationDate="2021-09-30T09:41:53.307" Score="1" Body="&lt;p&gt;You probably want approaches such as &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/tradeSeq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tradeSeq&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For Seurat to tradeSeq see also this &lt;a href=&quot;https://github.com/statOmics/tradeSeq/issues/45&quot; rel=&quot;nofollow noreferrer&quot;&gt;issue&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-09-30T09:41:53.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17790" PostTypeId="2" ParentId="17788" CreationDate="2021-09-30T21:47:52.520" Score="0" Body="&lt;p&gt;Not sure if it helps, but there's a tutorial for how to switch between Seurat and monocle &lt;a href=&quot;https://satijalab.org/signac/articles/monocle.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The basic approach is to convert it to a &lt;code&gt;cell_data_set&lt;/code&gt; object, then process that object with monocle:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;erythroid.cds &amp;lt;- as.cell_data_set(erythroid)&#xA;erythroid.cds &amp;lt;- cluster_cells(cds = erythroid.cds, reduction_method = &amp;quot;UMAP&amp;quot;)&#xA;erythroid.cds &amp;lt;- learn_graph(erythroid.cds, use_partition = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Pseudotime values can be returned back to the seurat object after processing with monocle via AddMetaData:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;erythroid &amp;lt;- AddMetaData(&#xA;  object = erythroid,&#xA;  metadata = erythroid.cds@principal_graph_aux@listData&lt;span class=&quot;math-container&quot;&gt;$UMAP$&lt;/span&gt;pseudotime,&#xA;  col.name = &amp;quot;monocle_pseudotime&amp;quot;&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Thank you for your reply. So I need to do the code you sent from erythroid.cds &amp;lt;- as.cell_data_set(erythroid) ...to ...col.name = &amp;quot;monocle_pseudotime&amp;quot;), then use the findmarker function of Seurat to find the differential expressed genes? It's that correct?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It's unclear what you're asking, but what you describe here will produce a list of differentially expressed genes based on the cluster definitions of monocle. As far as I'm aware, these clusters don't depend on the pseudotime values.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-10-01T20:26:50.100" LastActivityDate="2021-10-01T20:26:50.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17791" PostTypeId="2" ParentId="17780" CreationDate="2021-10-01T19:02:39.887" Score="1" Body="&lt;p&gt;I would recommend looking into BEDtools, assuming that you are talking about genomic intervals, which I &lt;em&gt;think&lt;/em&gt; you are based on the question. Possibly a diagram would help.&lt;/p&gt;&#xA;&lt;p&gt;BEDtools has a suite of statistical tools for evaluating overlaps, including &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/fisher.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fisher Exact Tests&lt;/a&gt; and also approaches based on the &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/reldist.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;relative distance&lt;/a&gt; of 2 sets of intervals. So as long as your genomic intervals are defined by BED files and a genome file, this should be quite easy, e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bedtools fisher -a A.bed -b B.bed -g genome_file.genome&#xA;# will print p-values for left, right, and two-tailed tests&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-10-01T19:02:39.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17792" PostTypeId="1" CreationDate="2021-10-02T03:53:41.673" Score="1" ViewCount="263" Body="&lt;p&gt;I need suggestions in creating index and dictionary with vcf files. For the base recalibration step, I downloaded Homo_sapiens_assembly38.known_indels.vcf.gz from the given link: &lt;a href=&quot;https://console.cloud.google.com/storage/browser/genomics-public-data/resources/broad/hg38/v0;tab=objects?prefix=&amp;amp;forceOnObjectsSortingFiltering=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://console.cloud.google.com/storage/browser/genomics-public-data/resources/broad/hg38/v0;tab=objects?prefix=&amp;amp;forceOnObjectsSortingFiltering=false&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am using GATK version 4.2.2.0&lt;/p&gt;&#xA;&lt;p&gt;When I try to create index and dictoionary of the vcf file, I get the following error: no suitable codecs found. I also tried to check the vcf file using ValidateVariants and I get the same error: no suitable codecs found.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone suggest which indels vcf file is used for masking sites in the base recalibration step. Has anyone come across this above error and how did they resolve the issue.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="14040" LastActivityDate="2022-03-02T10:07:55.767" Title="Dictionary and index of vcf for base recalibration step" Tags="|vcf|ngs|gatk|data-preprocessing|variants|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17793" PostTypeId="2" ParentId="17792" CreationDate="2021-10-03T08:12:44.907" Score="1" Body="&lt;p&gt;I have also encountered this error before in cases where there was something wrong with the VCF file I was supplying.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Double check your command to ensure that you are feeding it the file you believed you were and verify that file is a vcf (gzipped or otherwise)&lt;/li&gt;&#xA;&lt;li&gt;If you know how, please compare MD5 or SHA checksums of your copy vs. the original to ensure you have an uncorrupted copy.&lt;/li&gt;&#xA;&lt;li&gt;If this is something you haven't dealt with before, a less effective version would be to check and make sure that the file size on the server is the same as the file size on your machine.&lt;/li&gt;&#xA;&lt;li&gt;Additionally, you may want to just use the &lt;code&gt;less&lt;/code&gt; command to look inside the file and see if anything appears to be corrupted (you may need to decompress it for that purpose).&lt;/li&gt;&#xA;&lt;li&gt;It may also be worth trying to &lt;code&gt;gunzip&lt;/code&gt; or &lt;code&gt;gzip -d&lt;/code&gt; commands to decompress the file (assuming you have the drive space).  This will both ensure that decompressing the file on the fly is not causing any grief to GATK (which is unlikely) and that the file decompresses without error (which would also be an indication of corruption).&lt;/li&gt;&#xA;&lt;li&gt;Lastly, you could try downloading a fresh version.&#xA;Please let me know if this helps and you discover the source of the error.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14277" LastActivityDate="2021-10-03T08:12:44.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17794" PostTypeId="2" ParentId="17782" CreationDate="2021-10-03T09:08:07.727" Score="4" Body="&lt;p&gt;I'm going to try and keep this answer from going overboard while still helping you find your feet. The good news is that it sounds like you have started off with some of this work already done for you, and based upon the files you have above, I will try to indicate what you have already and also where you might be able to go with this. The term you should be entering into Google as the generally accepted term for this pipeline is &lt;code&gt;variant calling&lt;/code&gt; or possibly &lt;code&gt;variant curation&lt;/code&gt; for the downstream steps.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;This process starts off with one or more files containing raw observed sequences from a sequencing device (often, but not always, Illumina)  This files will usually be in the FASTQ format, but historically I have seen another format called QSEQ. You will likely never see this format and I feel old just bringing it up, but if you ever encounter it, just convert your files to FASTQ immediately.  (Files entering here: FASTQ, possibly QSEQ if it was generated well before Carly Rae Jepson suggested that we call her, maybe)&lt;/li&gt;&#xA;&lt;li&gt;Although not a hard requirement, I recommend running something like &lt;code&gt;fastqc&lt;/code&gt; or &lt;code&gt;multiqc&lt;/code&gt; or some other form of quality control on your reads and making sure that the reads are of satisfactory quality with no anomalous metrics on them. You may or may not decide that you need to do some clean-up on the reads if you determine that there might be adapter contamination or that some reads are partially/entirely garbage quality. (Files entering here: FASTQ, reference genome in FASTA. Files generated: a quality metrics report and possibly a cleaned up FASTQ)&lt;/li&gt;&#xA;&lt;li&gt;Once you have your raw reads and consider them to be of satisfactory quality, they will need to be aligned to a reference genome. Read alignment is a critical, computationally intensive process that is required for many bioinformatics pipelines, meaning that there are many aligners out there that have been optimized for many different situations. If you are running a variant calling pipeline on Illumina short reads from genomic DNA, a current version of BWA should be a solid choice for you here.  (Files entering here: FASTQ.  Files generated: SAM and/or BAM files with aligned reads)&lt;/li&gt;&#xA;&lt;li&gt;Now that alignment is complete, we can start getting reads for the real variant calling process by doing any preprocessing needed. The most popular tool(s) for this historically were Picard Tools and GATK, but those two tool sets were used together for so long that they were just begging to be merged. The recent versions of GATK have made it so and Picard tools are now rolled into and called directly from GATK. If you are using an older GATK version, at a minimum you will want to recalibrate the base quality scores to try correcting biases that your sequencer had when assigning confidence scores to individual base calls (they all have this, trust me) and realigning reads around detected insertion/deletion (indel) sites. Newer versions of GATK don't need the indel realignment steps due to the haplotype calling algorithm going even further and reassembling reads around these complex variants. Those are just the minimal preprocessing steps, you may also need to assign read groups if they aren't already in the BAM files and possibly some other stuff (most likely GATK will let you know what's missing as you progress).  (Files in: BAM, reference genome and &amp;quot;gold standard&amp;quot; variants in VCF format. Files out: preprocessed BAM)&lt;/li&gt;&#xA;&lt;li&gt;Actual variant calling can happen now, and this is likely going to happen using HaplotypeCaller (mutect2, if you're dealing with cancer/somatic changes). If this is your intention, you can look up the latest and greatest practices at &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035535932-Germline-short-variant-discovery-SNPs-Indels-&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360035535932-Germline-short-variant-discovery-SNPs-Indels-&lt;/a&gt; for the most current guidance. (Files in: preprocessed BAM, reference genome.  Files out: variant calls in VCF, possibly an active sites BAM file to visualize reassemblies around active indels)&lt;/li&gt;&#xA;&lt;li&gt;Filtering of variants should now happen. At this stage, you may be very application dependent, but essentially this process should assign some variants to being high confidence and others to being potentially technical artifacts. I have seen people skip this step either due to lack of interest or concern that it will filter out a real variant of interest, but I don't recommend that approach and have seen more people missing real variants due to being inundated with low confidence variants that should have been otherwise filtered out.&lt;/li&gt;&#xA;&lt;li&gt;Variant curation can now happen where one takes their high confidence variants and figures out (or usually has a computer program figure this out) which genes (if any) are being affected by the variant, what the chance will be to the protein coding as well as if the variant might affect splicing or some aspect of gene regulation or structure (such as an out of frame indel). It is often also useful here to compare to some database of mutation frequencies to filter out common SNP variants as potential cause for rare conditions.&lt;/li&gt;&#xA;&lt;li&gt;Identification of candidate mutations based upon the curation results. Identifying the mutations that are in genes that could plausibly control the observed phenotype, are uncommon, and are potentially causing pathogenic changes to the gene or its product (protein).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;It looks like you already have VCF files, so at least the initial phase of variant calling has been finished for you already.  I would recommend checking out something like Annovar or Varient Effect Predictor (VEP) that can run over your (hopefully filtered) VCF and assign a predicted gene and effect to a variant. Check both the SNP and Indel VCF files, as indels, while more rare, can have profound effects on a gene.&lt;/p&gt;&#xA;&lt;p&gt;From your description, it sounds like you wanted to check a few genes in particular, and after the variant effect preduction, you should have the needed information to do this.&lt;/p&gt;&#xA;" OwnerUserId="14277" LastActivityDate="2021-10-03T09:08:07.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17797" PostTypeId="1" CreationDate="2021-10-04T02:54:21.827" Score="1" ViewCount="1200" Body="&lt;p&gt;I have seven fasta files, one per gene, with more than 400 fasta entries per file, like this:&#xA;Input:&lt;/p&gt;&#xA;&lt;p&gt;Gene1.fasta&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1721_1 gene name  &#xA;ATG   &#xA;&amp;gt;10_14 gene name  &#xA;GTT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Gene2.fasta&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1721_1 gene name  &#xA;TGA  &#xA;&amp;gt;10_14 gene name   &#xA;GAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;1721_1  &#xA;ATGTGA  &#xA;&amp;gt;10_14   &#xA;GTTGAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What I want to do is create one fasta (continuously) file containing seven genes per sample. As you can see, the sample name is not the same before the _ in the fasta headers.&lt;/p&gt;&#xA;&lt;p&gt;Is there any way to do this with awk or other command-line tools? Thank you very much.&lt;/p&gt;&#xA;" OwnerUserId="14278" LastEditorUserId="2108" LastEditDate="2021-10-05T11:54:12.607" LastActivityDate="2022-12-05T11:54:19.270" Title="Concatenate multiple fasta files into one file for MLST" Tags="|fasta|linux|awk|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17799" PostTypeId="1" CreationDate="2021-10-04T12:23:46.103" Score="3" ViewCount="66" Body="&lt;p&gt;I am involved in a Drosophila transcriptomics project in which we have RNAseq data for small RNAs (for microRNAs) in addition to total RNA (for mRNAs).&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The total RNA reads have been analysed satisfactorily using the &lt;a href=&quot;https://www.nature.com/articles/nprot.2012.016.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tuxedo pipeline&lt;/a&gt; using TopHat, Cufflinks, Cuffquant and Cuffdiff. (I run this locally on a fast desktop Mac.)&lt;/li&gt;&#xA;&lt;li&gt;Initially we applied the Tuxedo pipeline to the short RNA reads, but abandoned this because of an incidence (1 in 10) of anomalously low or high replicates that was inconsistent with the bigwig plots from the bam stage, displayed in the UCSC browser.&lt;/li&gt;&#xA;&lt;li&gt;We there switched to analysing the short RNA reads using the &lt;a href=&quot;https://bmcgenomics.biomedcentral.com/articles/10.1186/1471-2164-15-423&quot; rel=&quot;nofollow noreferrer&quot;&gt;CAP-miRSeq pipeline&lt;/a&gt; on Unix, which initially — though very slow — gave consistent results. However recent attempts to process later samples by this method failed as the pipeline can no longer find some dependencies it requires.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I see no point in spending further time on either of these methods.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Has anyone &lt;em&gt;experience&lt;/em&gt; of a &lt;em&gt;more modern&lt;/em&gt; analysis method for microRNA RNAseq data, preferably one that would run on MacOS?&lt;/p&gt;&#xA;&lt;p&gt;I have no trouble in running the Tuxedo pipeline and get good results with total RNA. The problem there is not my computer. Unless someone who has analysed microRNA using this package understands the problem I have encountered, there is no rationale basis for trying this again. As regards Cap-miRSeq, it does not run on the Mac, only on Unix. A colleague set it up on a virtual box. We are our support. Is there anyone working on RNAseq of microRNA?&lt;/p&gt;&#xA;" OwnerUserId="1255" LastEditorUserId="73" LastEditDate="2022-02-13T21:41:24.487" LastActivityDate="2022-03-17T20:06:15.763" Title="Alternatives to CAP-miRSeq for microRNA processing" Tags="|rna-seq|transcriptome|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17800" PostTypeId="1" CreationDate="2021-10-04T17:48:38.080" Score="1" ViewCount="38" Body="&lt;p&gt;Noob here. I get that I should  deduce what characteristics the ideal new enzyme should meet, and then use tools such as PDB and blast to compare to the old enzymes, and use other tools such as pymol, but I have no idea on how to approach this. Where should I look for tutorials?&lt;/p&gt;&#xA;" OwnerUserId="14284" LastActivityDate="2023-09-18T16:00:51.237" Title="How do you search for new enzymes that are more stable for handling, immobilization?" Tags="|sequence-alignment|blast|pdb|phylogenetics|pymol|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17801" PostTypeId="1" CreationDate="2021-10-04T17:49:59.817" Score="0" ViewCount="189" Body="&lt;p&gt;I  am working on an iterative analysis that uses orthologous pipelines that require mpileup.txt files as input for a visualization step. This requires me to convert VCF files to mpileup.txt.&lt;/p&gt;&#xA;&lt;p&gt;This experiment uses error-prone-pcr and follows a somatic variant discovery workflow that was designed by a 3rd party. I have written and implemented several orthologous pipelines with small differences from the original workflow in order to produce a truth set (variants identified consistently by all of the pipelines). The 3rd party used a workflow that calls Samtools to produce mpileup.txt files as the final output and necessitated the development of a custom visualization script. My orthologous pipelines use mutect2 to produce vcf files that need to be compared to the mpileup.txt files.&lt;/p&gt;&#xA;&lt;p&gt;If I could convert my vcf files to mpileup.txt files without loss of information it would save me significant time cost associated with writing a similar visualization script for vcf files and prevent bias involved in converting the 3rd parties output to vcf format. There are plenty of examples for using samtools and bcftools to produce an intermediate mpileup file that is then converted to a finalized fitered vcf file. But I can't find much direction for converting a vcf file to an mpileup. Can anybody provide direction for this task?&lt;/p&gt;&#xA;" OwnerUserId="4507" LastActivityDate="2021-10-06T01:27:31.457" Title="Convert VCF file to mpileup.txt" Tags="|rna-seq|samtools|variant-calling|gatk|bcftools|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17802" PostTypeId="1" AcceptedAnswerId="17803" CreationDate="2021-10-05T01:58:08.780" Score="1" ViewCount="186" Body="&lt;p&gt;Order of alleles in genotype field (on &lt;a href=&quot;https://www.internationalgenome.org/wiki/Analysis/vcf4.0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VCFs&lt;/a&gt;) can be used &lt;a href=&quot;https://www.biostars.org/p/5298/&quot; rel=&quot;nofollow noreferrer&quot;&gt;to infer haplotypes&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, what about unphased genotypes? I have a VCF file where I only have unphased genotypes:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;0/1&#xA;1/0&#xA;1/1&#xA;1/2&#xA;2/1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, I have, for example, both &lt;code&gt;0/1&lt;/code&gt; and &lt;code&gt;1/0&lt;/code&gt;. Does the order of alleles have some different meaning with unphased genotypes, or genotypes with same alleles (e.g., &lt;code&gt;0/1&lt;/code&gt; and &lt;code&gt;1/0&lt;/code&gt;) can be considered synonymous?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2021-10-06T09:38:41.483" Title="Meaning of order of alleles in VCF GT field for unphased genotypes" Tags="|vcf|variant-calling|genotyping|phasing|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17803" PostTypeId="2" ParentId="17802" CreationDate="2021-10-05T07:25:02.313" Score="3" Body="&lt;p&gt;In the case of unphased VCF &lt;code&gt;0/1&lt;/code&gt; and &lt;code&gt;1/0&lt;/code&gt; are exactly equivalent and the order has no meaning.&lt;/p&gt;&#xA;&lt;p&gt;Throughout the VCF specification, it is always encouraged to use numerical precedence in order to order the genotypes, e.g. &lt;code&gt;0/0&lt;/code&gt;, &lt;code&gt;0/1&lt;/code&gt;, &lt;code&gt;1/1&lt;/code&gt; would be the preferred representations for diploid bi-allelic genotypes.&lt;/p&gt;&#xA;&lt;p&gt;However it is not strictly enforced in the VCF specification as for now and you could have &lt;code&gt;1/0&lt;/code&gt;, &lt;code&gt;2/0&lt;/code&gt;, etc, and bcftools will not complain parsing them. The presence of two representation in the same VCF might indicate that the data comes from multiple variant callers or has been through manual editing, since for example variant callers will always use one representation only (numerical precedence usually)&lt;/p&gt;&#xA;&lt;p&gt;Also, the order does impact how you read some of the sample information fields, for example the AD (allelic depth), for which the values are read following the order present in the GT field.&lt;/p&gt;&#xA;&lt;p&gt;I had a look at the VCF &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;4.2&lt;/a&gt; and &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;4.3&lt;/a&gt; spec, but indeed is not so precise there on these details.&lt;/p&gt;&#xA;" OwnerUserId="475" LastEditorUserId="475" LastEditDate="2021-10-06T09:38:41.483" LastActivityDate="2021-10-06T09:38:41.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17804" PostTypeId="2" ParentId="17800" CreationDate="2021-10-05T10:09:14.667" Score="1" Body="&lt;p&gt;From your title, you are after better immobilisation, i.e. your enzyme behaves poorly when bound.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Poor thermostability: Engineering enzymes for better thermostability, involves predicting mutations that improve its Gibbs free energy (∆G, negative potential = better). Finding mutations that improve a protein by going through the mutational landscape is called scanning.&#xA;Rosetta &lt;code&gt;pmut_scan&lt;/code&gt; is a command line tool to do this, while &lt;a href=&quot;https://pross.weizmann.ac.il/step/pross-terms/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pross&lt;/a&gt; is a really nice webapp that also does it.&lt;/li&gt;&#xA;&lt;li&gt;Poor catalysis: if your attachment is close to the active site, that is a design flaw and you need to attach on the other terminus or within a loop. Circular permutation may be an option.&lt;/li&gt;&#xA;&lt;li&gt;Aggregation: your enzyme aggregates due to pI close to 7 (see &lt;a href=&quot;https://web.expasy.org/protparam/&quot; rel=&quot;nofollow noreferrer&quot;&gt;protparam&lt;/a&gt;) or has a face that is hydrophobic (see the APBS plugin shipped within PyMOL). Rosetta &lt;code&gt;supercharge&lt;/code&gt; is great at suggesting mutations to decrease the pI, but if doing it crudely by changing Gln -&amp;gt; Gln &amp;amp; Asn -&amp;gt; Asp, do check for surface exposure.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2023-09-18T16:00:51.237" LastActivityDate="2023-09-18T16:00:51.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17805" PostTypeId="1" CreationDate="2021-10-05T18:28:28.883" Score="1" ViewCount="105" Body="&lt;p&gt;I cannot find in the &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VCF specs&lt;/a&gt; what is the default value of Genotype Quality &lt;code&gt;GQ&lt;/code&gt; for haploid calls. In such case, is the default the missing value &lt;code&gt;.&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;From the specs:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If any of the fields is missing, it is replaced with the MISSING value. For example if the FORMAT is GT:GQ:DP:HQ&#xA;then 0 | 0 : . : 23 : 23, 34 indicates that GQ is missing. If a field contains a list of missing values, it can be represented&#xA;either as a single MISSING value (‘.’) or as a list of missing values (e.g. ‘.,.,.’ if the field was Number=3). Trailing&#xA;fields can be dropped, with the exception of the GT field, which should always be present if specified in the FORMAT&#xA;field.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;From this I assume that there is no default value, &lt;code&gt;GQ&lt;/code&gt; might be dropped from the &lt;code&gt;FORMAT&lt;/code&gt; field, correct?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2021-10-06T17:08:57.930" LastActivityDate="2021-10-06T17:08:57.930" Title="Genotype quality in VCF for haploid calls" Tags="|vcf|genotyping|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17806" PostTypeId="1" CreationDate="2021-10-05T22:17:04.490" Score="0" ViewCount="60" Body="&lt;p&gt;Could I ask, is anyone familiar with the Drugebility score provided by Chembl?&lt;/p&gt;&#xA;&lt;p&gt;I wanted to add a druggability score to a set of human targets, to understand the likelihood that they are druggable.&lt;/p&gt;&#xA;&lt;p&gt;I found Chembl's drugebility score &lt;a href=&quot;https://chembl.gitbook.io/chembl-interface-documentation/downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, but I don't fully understand which file I'm meant to use that adds a drugebility score to each human protein? In particular, should I be looking at the drugebility.dmp.gz file (that was created in 2019) or the site_drugebility.txt file that was created in 2013 (seems very long ago)?&lt;/p&gt;&#xA;&lt;p&gt;I'm unclear in drugebility.dmp how do understand what ensembl/NCBI/uniprot ID each score is assigned to?&lt;/p&gt;&#xA;&lt;p&gt;Or does someone have a better suggestion for how to link ideally Ensembl/UniProt IDs to a score indicating the likelihood that the protein is druggable?&lt;/p&gt;&#xA;" OwnerUserId="10328" LastActivityDate="2021-10-05T22:17:04.490" Title="Best resource to link a gene/protein to a druggability score?" Tags="|ensembl|uniprot|drugs|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17807" PostTypeId="2" ParentId="17801" CreationDate="2021-10-06T01:27:31.457" Score="3" Body="&lt;p&gt;Going VCF to mpileup is not really something one does or can do. The mpileup should be generated from a BAM or SAM file or something else that has raw, unfiltered read alignments in it.  The VCF just has data on variant sites, and usually just variant sites that passed a certain likelihood threshold at that.  With the VCF, the best you can do is generate a very limited pileup based upon the VCF-reported counts of reads at each variant locus with no information in between reported variant loci. This would not be a true mpileup and would most likely cause an error with the downstream process or worse, not cause an error and give a result that is divorced from the reality of your sample.&lt;/p&gt;&#xA;" OwnerUserId="14277" LastActivityDate="2021-10-06T01:27:31.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17808" PostTypeId="2" ParentId="17797" CreationDate="2021-10-06T16:26:46.877" Score="0" Body="&lt;p&gt;I think it can be done using python dictionary. You can split the file using the header and then make the fasta id as your key and the sequence as your value. Then using simple loop and conditions you can concatenate them.&lt;/p&gt;&#xA;" OwnerUserId="14290" LastActivityDate="2021-10-06T16:26:46.877" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17809" PostTypeId="1" AcceptedAnswerId="19037" CreationDate="2021-10-07T04:34:08.483" Score="1" ViewCount="36" Body="&lt;p&gt;As a practice data analysis I am trying to train a convolutional neural network (CNN) on &lt;a href=&quot;https://www.kaggle.com/xhlulu/recursion-2019-load-resize-and-save-images/data&quot; rel=&quot;nofollow noreferrer&quot;&gt;some cellular images&lt;/a&gt; made publicly available on Kaggle in 2019 by the company Recursion via supervised deep learning.&lt;/p&gt;&#xA;&lt;p&gt;I want to increase the predictive power using transfer learning at the lower layers of my CNN preferably from a public imaging model.&lt;/p&gt;&#xA;&lt;p&gt;Are there any appropriate models available?&lt;/p&gt;&#xA;" OwnerUserId="2482" LastEditorUserId="3848" LastEditDate="2022-05-06T00:57:20.407" LastActivityDate="2022-05-06T02:25:35.670" Title="Transfer learning for a Convolutional NN for Recursion dataset" Tags="|machine-learning|deep-learning|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17810" PostTypeId="1" CreationDate="2021-10-07T05:48:46.713" Score="2" ViewCount="382" Body="&lt;p&gt;Anyone can answer, and should be in simple way&lt;/p&gt;&#xA;" OwnerUserId="14293" LastActivityDate="2021-10-07T09:46:05.877" Title="Difference between bioinformatics and cheminformatics" Tags="|genomics|cheminformatics|" AnswerCount="1" CommentCount="1" ClosedDate="2021-10-20T17:51:49.787" ContentLicense="CC BY-SA 4.0" />
  <row Id="17811" PostTypeId="2" ParentId="17810" CreationDate="2021-10-07T09:46:05.877" Score="1" Body="&lt;p&gt;There are four terms to consider, all are computational&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;cheminformatics — &lt;a href=&quot;https://en.wikipedia.org/wiki/Big_data&quot; rel=&quot;nofollow noreferrer&quot;&gt;Big data&lt;/a&gt; to explore relationship between small molecules (e.g. enumeration and machine learning stuff)&lt;/li&gt;&#xA;&lt;li&gt;computational chemistry (&amp;quot;compchem&amp;quot;) - thermodynamics-based methods to explore small molecules (e.g. docking)&lt;/li&gt;&#xA;&lt;li&gt;bioinformatics — big data primarily for exploring -omics data (genomics, proteomics, transcriptomics, metabolomics)&lt;/li&gt;&#xA;&lt;li&gt;computational biochemistry - thermodynamics-based methods to study structure of protein&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-10-07T09:46:05.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17812" PostTypeId="1" CreationDate="2021-10-07T13:32:54.007" Score="1" ViewCount="159" Body="&lt;p&gt;I applied three multiple sequence alignment methods of muscle, mafft and clistal omega on 30 sequences and 400 sequences . In the case of 30 seq, the output of alignment from the 3 methods was in the same length but in the case of larger data with different length of sequences (input data) the output from alignment methods was different length clustal length is different than the two others. My question is how can i assure that the length from MSA should be equal or different?&lt;/p&gt;&#xA;" OwnerUserId="14295" LastActivityDate="2023-04-02T21:05:10.623" Title="Multiple Sequence Alignment" Tags="|variation|multiple-sequence-alignment|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17813" PostTypeId="1" AcceptedAnswerId="17820" CreationDate="2021-10-07T20:36:25.043" Score="2" ViewCount="45" Body="&lt;p&gt;In my work I've found that in circumstances where there is a mismatch near the edges of the query sequence, blastn prefers to return a shorter contiguous alignment, rather than allowing for a mismatch near the 5' or 3' end of the query sequence.&lt;/p&gt;&#xA;&lt;p&gt;For example, consider the subject sequence (bold indicates alignment with mismatch in brackets):&lt;/p&gt;&#xA;&lt;p&gt;GACCAACCTG&lt;strong&gt;CTGGATCTGCTGATCGGCG[G]T&lt;/strong&gt;TTGCGAAATA&lt;/p&gt;&#xA;&lt;p&gt;and query sequence:&lt;/p&gt;&#xA;&lt;p&gt;CTGGATCTGCTGATCGGCG[A]T&lt;/p&gt;&#xA;&lt;p&gt;Blasting this will return the 19bp contiguous alignment upstream of the mismatch, rather than returning a 21bp alignment with a single mismatch at the G/A base at position 20 in the query.&lt;/p&gt;&#xA;&lt;p&gt;I'm using the command line version of blastn, and I've tried messing with the penalties as well as setting &lt;code&gt;-task blastn-short&lt;/code&gt;, but nothing seems to fix this issue.&lt;/p&gt;&#xA;&lt;p&gt;I speculate that the various gap and mismatch penalties are working out such that a shorter contiguous alignment is preferred over a slightly longer alignment with a mismatch, but I haven't been able to figure out how to modify the penalties in such a way as to change this preference.&lt;/p&gt;&#xA;" OwnerUserId="1456" LastEditorUserId="1456" LastEditDate="2021-10-08T17:17:38.883" LastActivityDate="2021-10-08T17:17:38.883" Title="blastn returning an inferior alignment" Tags="|sequence-alignment|blast|blastn|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17815" PostTypeId="1" CreationDate="2021-10-08T10:05:54.960" Score="1" ViewCount="397" Body="&lt;p&gt;I am trying to predict transmembrane protein using &lt;code&gt;tmhmm-2.0c&lt;/code&gt;. According to Package instruction, I have changed the path of perl inside the program like as following:&lt;/p&gt;&#xA;&lt;p&gt;First i checked the location of perl&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;which perl&#xA;/usr/bin/perl&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then I checked the version also to make sure it is above 5.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -v &#xA;This is perl 5, version 16, subversion 3 (v5.16.3) built for x86_64-linux-thread-multi (with 44 registered patches, see perl -V for more detail)&#xA;&#xA;Copyright 1987-2012, Larry Wall&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have pasted the path (&lt;code&gt;/usr/bin/perl&lt;/code&gt;) in the &lt;code&gt;tmhmm&lt;/code&gt; and &lt;code&gt;tmhmmformat.pl&lt;/code&gt;&#xA;Run the cmd:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./tmhmm --short &amp;lt; /mnt/genome3/Lab_Users/Kishor/DISK_3/mnspt1/blood/rsem_outdir/blood.Trinity.RSEM.retained.clustered.fasta.transdecoder.pep &amp;gt; tmhmm.out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I only received a blank folder. It will be great for me if I get any suggestions to solve the problem.&lt;/p&gt;&#xA;" OwnerUserId="12579" LastEditorUserId="57" LastEditDate="2021-10-10T16:45:59.680" LastActivityDate="2021-12-15T05:37:19.467" Title="tmhmm installation and running" Tags="|linux|software-installation|perl|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="17816" PostTypeId="1" CreationDate="2021-10-08T10:42:56.573" Score="1" ViewCount="1022" Body="&lt;p&gt;I have a little pipeline that I want to execute using Snakemake.&lt;/p&gt;&#xA;&lt;p&gt;One of my rules (let's call it &lt;code&gt;create_somefile&lt;/code&gt;) creates a file &lt;code&gt;somefile_{n_dims}.json&lt;/code&gt;, and when running the pipeline can specify that I want to run the pipeline for a pre-defined set of values for &lt;code&gt;n_dims&lt;/code&gt;, which is why I specified this as a wildcard.&lt;/p&gt;&#xA;&lt;p&gt;Now I want to have another rule, that takes as input &lt;strong&gt;any of&lt;/strong&gt; these files. It can take either &lt;code&gt;somefile_1.json&lt;/code&gt; or &lt;code&gt;somefile_2.json&lt;/code&gt; or &lt;code&gt;somefile_32402348.json&lt;/code&gt;, it doesn't matter which one as long as any of these exists. The rule then creates a file that does not depend on the number of dimensions, and thus doesn't want and need the n_dims in its output.&lt;/p&gt;&#xA;&lt;p&gt;When I try to have the wildcard only in the input, like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule do_stuff:&#xA;    input:&#xA;        &amp;quot;somefile_{n_dims}.json&amp;quot;&#xA;    output:&#xA;        &amp;quot;general_file.json&amp;quot;&#xA;    shell:&#xA;        &amp;quot;echo {input} &amp;amp;&amp;amp;&amp;quot;&#xA;        &amp;quot;touch {output}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Snakemake gives me the error &lt;code&gt;Wildcards in input files cannot be determined from output files: n_dims&lt;/code&gt;.&#xA;I don't want the output file to contain &lt;code&gt;n_dims&lt;/code&gt; whatsoever, and unfortunately it's also impossible in snakemake to just create an additional throwaway &lt;code&gt;temporary(..)&lt;/code&gt;-output that does depend on the n_dims just so snakemake is happy (error message then: &lt;code&gt;ot all output, log and benchmark files of rule do_stuff contain the same wildcards. This is crucial though, in order to avoid that two or more jobs write to the same file.&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Is there any way in snakemake to specify a rule's inputs as &amp;quot;take any of these&amp;quot;, without wildcards?&lt;/p&gt;&#xA;&lt;p&gt;Looking at the documentation, the only thing I found is the concept of &lt;code&gt;checkpoints&lt;/code&gt; (&lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#data-dependent-conditional-execution&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#data-dependent-conditional-execution&lt;/a&gt;), however that doesn't seem to work as well - I tried making the rule &lt;code&gt;create_somefile&lt;/code&gt; a checkpoint and changing the &lt;code&gt;do_stuff&lt;/code&gt; rule to&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule do_stuff:&#xA;    input:&#xA;        lambda wildcards: checkpoints.create_somefile.get(**wildcards).output&#xA;    output:&#xA;        &amp;quot;general_file.json&amp;quot;,&#xA;    shell:&#xA;        &amp;quot;echo {input} &amp;amp;&amp;amp;&amp;quot;&#xA;        &amp;quot;touch {output}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;however then I get the error &lt;code&gt;WorkflowError: Missing wildcard values for n_dims&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Is there any concept in snakemake to allow me to do what I want? To have &amp;quot;wildcards&amp;quot; in inputs, that just stand for &amp;quot;execute this rule as soon as the input for any value of the wildcard is fulfilled, execute only once and be done with it&amp;quot;?&lt;/p&gt;&#xA;" OwnerUserId="10588" LastActivityDate="2021-10-10T16:23:50.743" Title="Snakemake: Accept any of a possible set of inputs (any one being sufficient)" Tags="|python|snakemake|" AnswerCount="1" CommentCount="2" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17817" PostTypeId="1" CreationDate="2021-10-08T14:29:24.340" Score="1" ViewCount="313" Body="&lt;p&gt;I need help with some clarifications please&lt;/p&gt;&#xA;&lt;p&gt;I have control versus treatment in two time points like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; y&#xA;                             X condition time&#xA;1       CTRL_24_hrs_replicate1   control   24&#xA;2       CTRL_24_hrs_replicate2   control   24&#xA;3       CTRL_24_hrs_replicate3   control   24&#xA;4  treatment_24_hrs_replicate1        t   24&#xA;5  treatment_24_hrs_replicate2        t   24&#xA;6  treatment_24_hrs_replicate3        t   24&#xA;7       CTRL_48_hrs_replicate1   control   48&#xA;8       CTRL_48_hrs_replicate2   control   48&#xA;9       CTRL_48_hrs_replicate3   control   48&#xA;10 treatment_48_hrs_replicate1        t   48&#xA;11 treatment_48_hrs_replicate2        t   48&#xA;12 treatment_48_hrs_replicate3        t   48&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to see the difference of treatment versus control from 24 hours to 48 hours, so this is not just the difference of treatment versus control or difference between 24 versus 48 hours rather the treatment versus control from 24 hours to 48 hours&lt;/p&gt;&#xA;&lt;p&gt;This is what I have done so far&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;  dds &amp;lt;- DESeqDataSetFromMatrix(countData=count, &#xA;+                                 colData=design, &#xA;+                                 design=~condition, tidy = FALSE)&#xA;Warning message:&#xA;In DESeqDataSet(se, design = design, ignoreRank) :&#xA;  some variables in design formula are characters, converting to factors&#xA;&amp;gt;   dds&lt;span class=&quot;math-container&quot;&gt;$group &amp;lt;- factor(paste0(dds$&lt;/span&gt;time, dds$condition))&#xA;&amp;gt;   design(dds) &amp;lt;- ~ group&#xA;&amp;gt;   dds &amp;lt;- DESeq(dds)&#xA;estimating size factors&#xA;estimating dispersions&#xA;gene-wise dispersion estimates&#xA;mean-dispersion relationship&#xA;final dispersion estimates&#xA;fitting model and testing&#xA;&amp;gt;   resultsNames(dds)&#xA;&amp;quot;Intercept&amp;quot;                    &amp;quot;group_24IT_vs_24control&amp;quot;         &amp;quot;group_48control_vs_24control&amp;quot; &amp;quot;group_48IT_vs_24control&amp;quot;     &#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How to find the difference at time point 24 vs 48 in condition ?&lt;/p&gt;&#xA;&lt;p&gt;I searched and I found a similar thing but I don't know how to adopt that to my data&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;results(dds, contrast=list(c(&amp;quot;time_4_vs_0&amp;quot;, &amp;quot;conditionLD.time4&amp;quot;)), test=&amp;quot;Wald&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried this and I got error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; results(dds, contrast=list(c(&amp;quot;time_48_vs_24&amp;quot;, &amp;quot;conditionIT.time24&amp;quot;)), test=&amp;quot;Wald&amp;quot;)&#xA;Error in checkContrast(contrast, resNames) : &#xA;  all elements of the contrast as a list of length 2 should be elements of 'resultsNames(object)'&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can you help me in adopting this to my data please?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2021-10-27T12:14:56.500" LastActivityDate="2022-12-21T19:00:45.113" Title="DEseq2 contrast in two time points" Tags="|deseq2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17819" PostTypeId="2" ParentId="17817" CreationDate="2021-10-08T17:09:24.410" Score="1" Body="&lt;p&gt;You can't figure out how to adapt this to your data?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#contrasts&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#contrasts&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Ignore resultsNames.  You don't need them for simple comparisons like you want to do.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2021-10-08T17:09:24.410" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17820" PostTypeId="2" ParentId="17813" CreationDate="2021-10-08T17:17:27.337" Score="0" Body="&lt;p&gt;Here's my working solution,&lt;/p&gt;&#xA;&lt;p&gt;So I found a choice of parameters that resolves this, &lt;code&gt;-reward 3 -gapopen 5 -gapextend 5&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1456" LastActivityDate="2021-10-08T17:17:27.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17821" PostTypeId="2" ParentId="17812" CreationDate="2021-10-08T17:49:44.317" Score="1" Body="&lt;p&gt;The length of the alignment will depend on the model. For example, high gap costs will typically result in shorter alignments, whereas low gap-costs will do the opposite.&lt;/p&gt;&#xA;&lt;p&gt;Consider the sequences below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;AAAAA&#xA;TTTTT&#xA;GGGGG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;with a very large gap cost, these sequences will result in the following MSA&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;AAAAA&#xA;|||||&#xA;TTTTT&#xA;|||||&#xA;GGGGG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but with lower gap costs, we'd observe&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;AAAAA----------&#xA;-----TTTTT-----&#xA;----------GGGGG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Of course, this is an extreme case, but it goes to show that the alignment model will vary how long the resulting alignment is.&lt;/p&gt;&#xA;" OwnerUserId="6628" LastEditorUserId="6628" LastEditDate="2022-03-08T17:13:53.603" LastActivityDate="2022-03-08T17:13:53.603" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17822" PostTypeId="2" ParentId="15177" CreationDate="2021-10-08T18:11:10.547" Score="2" Body="&lt;p&gt;I will give an answer that is specific to closely-related microbial genomes. Core-SNPs, SNPs which are within conserved and likely orthologous sequences, have been shown to be relatively informative for constructing phylogenies of large sequence sets&lt;a href=&quot;http://scholar.google.com/scholar_lookup?&amp;amp;title=Genomics%20and%20outbreak%20investigation%3A%20from%20sequence%20to%20consequence&amp;amp;journal=Genome%20Med&amp;amp;volume=5&amp;amp;publication_year=2013&amp;amp;author=Robinson%2CER&amp;amp;author=Walker%2CTM&amp;amp;author=Pallen%2CMJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;[1]&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-014-0524-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;Parsnp&lt;/a&gt; is the most popular tool for constructing core-genome alignments on microbial genomes, and extracts the core SNPs from the alignments. Parsnp then uses raxml or FastTree in order to produce a phylogeny from the core SNPs. I will say, though, that 20k sequences may be too many for Parsnp to run on (depending on how large the sequences are and your available resources).&lt;/p&gt;&#xA;&lt;p&gt;Disclaimer: I am the maintainer of Parsnp (:&lt;/p&gt;&#xA;" OwnerUserId="6628" LastEditorUserId="6628" LastEditDate="2021-10-10T13:23:14.887" LastActivityDate="2021-10-10T13:23:14.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17824" PostTypeId="1" CreationDate="2021-10-09T07:23:31.400" Score="2" ViewCount="37" Body="&lt;p&gt;I have a list of UniProt codes and I am trying to fetch the genomic coordinates through the API from programmatic access to UniProtKB (&lt;a href=&quot;https://www.ebi.ac.uk/proteins/api/doc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/proteins/api/doc/&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Moreover, I read that UniProt in collaboration with Ensembl have mapped the protein sequences on the GRCh38 genome (doi: 10.1093/nar/gkw1099)&lt;/p&gt;&#xA;&lt;p&gt;However, it is true that the number of protein codes in UniProt is updated more frequently than the GRCh38 assembly.&lt;/p&gt;&#xA;&lt;p&gt;Considering only the protein genomic coordinates data:&#xA;It is possible that a new submission in UniProt (in TrEMBL for instance) does not have any correspondence with Ensembl, due to the possibility that the genomic coordinates of a new protein do not match any gene model stored in GRCh38?&lt;/p&gt;&#xA;&lt;p&gt;If not:&#xA;It is possible to say that all the protein genomic coordinates in UniProt have been mapped on the GRCh38 (in connection with Ensembl) and it is impossible that a new submission in UniProt lacks of this data?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="14310" LastActivityDate="2022-01-25T09:02:39.020" Title="It is possible to have a UniProt protein entry that does not have any genomic coordinates, because lacks of an Ensembl link?" Tags="|ensembl|uniprot|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17825" PostTypeId="1" AcceptedAnswerId="17826" CreationDate="2021-10-09T17:45:07.967" Score="1" ViewCount="2181" Body="&lt;p&gt;I'm trying to remove columns from the meta.data from a Seurat object. Is there a way to do that or would I have to re-create the Seurat object without that column?&lt;/p&gt;&#xA;" OwnerUserId="6827" LastActivityDate="2021-10-09T22:49:04.010" Title="Removing columns from meta.data in Seurat" Tags="|r|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17826" PostTypeId="2" ParentId="17825" CreationDate="2021-10-09T22:49:04.010" Score="2" Body="&lt;p&gt;Set the column you don't want to NULL&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;x$remove.column &amp;lt;- NULL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12114" LastActivityDate="2021-10-09T22:49:04.010" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17827" PostTypeId="1" CreationDate="2021-10-10T10:09:15.320" Score="1" ViewCount="36" Body="&lt;p&gt;I've calculated the cophenetic coefficients for the NMF of gene expression data, but is giving error when performing the clustering information step as given below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;clustered_data &amp;lt;- NMF_estimate[[num.opt - 1]][[2]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error in NMF_estimate[[num.opt - 1]] : subscript out of bounds&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here num.opt is &lt;code&gt;num.opt &amp;lt;- coph[which(coph[,2]==max(coph[-1,2])),1]&lt;/code&gt; which gives 2, 5 as outputs, and NMF Estimate is a list of 6 elements.&lt;/p&gt;&#xA;&lt;p&gt;Please suggest a solution as soon as possible.&lt;/p&gt;&#xA;" OwnerUserId="14209" LastActivityDate="2021-10-10T10:09:15.320" Title="NMF - Subscript out of bounds" Tags="|r|genomics|gene-expression|clustering|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17828" PostTypeId="2" ParentId="17816" CreationDate="2021-10-10T16:23:50.743" Score="1" Body="&lt;p&gt;I don't think Snakemake can follow a &amp;quot;use any of these&amp;quot; pattern with a unified dependency graph and without additional assumptions, but you can do it if you break the workflow into two pieces.&lt;/p&gt;&#xA;&lt;p&gt;This will take whatever &lt;code&gt;somefile_*.json&lt;/code&gt; shows up first, or throw an exception if there aren't any:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from pathlib import Path&#xA;&#xA;def input_for_do_stuff(wildcards):&#xA;    try:&#xA;        return next(Path(&amp;quot;.&amp;quot;).glob(&amp;quot;somefile_*.json&amp;quot;))&#xA;    except StopIteration as err:&#xA;        raise Exception(&amp;quot;No n_dims JSON files found&amp;quot;) from err&#xA;&#xA;rule do_stuff:&#xA;    input: input_for_do_stuff&#xA;    output:&#xA;        &amp;quot;general_file.json&amp;quot;,&#xA;    shell:&#xA;        &amp;quot;echo {input} &amp;amp;&amp;amp;&amp;quot;&#xA;        &amp;quot;touch {output}&amp;quot;&#xA;&#xA;rule create_somefile:&#xA;    output: &amp;quot;somefile_{n_dims}.json&amp;quot;&#xA;    shell: &amp;quot;touch {output}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For example if you have &lt;code&gt;somefile_2.json&lt;/code&gt; already there:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ snakemake -nrp&#xA;Building DAG of jobs...&#xA;Job stats:&#xA;job         count    min threads    max threads&#xA;--------  -------  -------------  -------------&#xA;do_stuff        1              1              1&#xA;total           1              1              1&#xA;&#xA;&#xA;[Sun Oct 10 12:22:14 2021]&#xA;rule do_stuff:&#xA;    input: somefile_2.json&#xA;    output: general_file.json&#xA;    jobid: 0&#xA;    reason: Missing output files: general_file.json&#xA;    resources: tmpdir=/tmp&#xA;&#xA;echo somefile_2.json &amp;amp;&amp;amp;touch general_file.json&#xA;Job stats:&#xA;job         count    min threads    max threads&#xA;--------  -------  -------------  -------------&#xA;do_stuff        1              1              1&#xA;total           1              1              1&#xA;&#xA;This was a dry-run (flag -n). The order of jobs does not reflect the order of execution.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With that, you'll never be able to run the workflow by just requesting the final output, which kind of breaks Snakemake's way of thinking about things.  But at least it'll work like you want if you run it in two pieces.  If having a default makes sense for your case, you could also do something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def input_for_do_stuff(wildcards):&#xA;    try:&#xA;        return next(Path(&amp;quot;.&amp;quot;).glob(&amp;quot;somefile_*.json&amp;quot;))&#xA;    except StopIteration as err:&#xA;        return &amp;quot;somefile_1.json&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That way, even if no files exist to start with, Snakemake can come up with a way to make the output you ask for.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2021-10-10T16:23:50.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17829" PostTypeId="1" AcceptedAnswerId="17830" CreationDate="2021-10-10T20:09:53.660" Score="5" ViewCount="357" Body="&lt;p&gt;The BioPython documentation shows the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;11.6.3 Measuring torsion angles Use the vector representation of the atomic coordinates, and the calc dihedral function from the Vector module:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; vector1 = atom1.get_vector()&#xA;&amp;gt;&amp;gt;&amp;gt; vector2 = atom2.get_vector()&#xA;&amp;gt;&amp;gt;&amp;gt; vector3 = atom3.get_vector()&#xA;&amp;gt;&amp;gt;&amp;gt; vector4 = atom4.get_vector()&#xA;&amp;gt;&amp;gt;&amp;gt; angle = calc_dihedral(vector1, vector2, vector3, vector4)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;We know that a torsion angle between N-C-alpha and C-alpha-C planes.&lt;/p&gt;&#xA;&lt;p&gt;So, the minimum number of points required to form a plane is 3.&lt;/p&gt;&#xA;&lt;p&gt;Then, Why does BioPython require 4 vectors?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastActivityDate="2021-10-12T22:39:54.313" Title="Why does it require 4 vectors to calculate a torsion angle?" Tags="|python|" AnswerCount="2" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17830" PostTypeId="2" ParentId="17829" CreationDate="2021-10-11T03:54:05.840" Score="5" Body="&lt;p&gt;From &lt;a href=&quot;https://en.wikipedia.org/wiki/Dihedral_angle&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia&lt;/a&gt; on torsion angles:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A dihedral angle is the angle between two intersecting planes or half-planes. In chemistry, it is the clockwise angle between half-planes through two sets of three atoms, having two atoms in common.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Two sets of three with two in common makes four total atoms.&lt;/p&gt;&#xA;&lt;p&gt;You are defining not one plane but two half-planes or planes.&lt;/p&gt;&#xA;&lt;p&gt;Wikipedia includes the following as an illustrative image:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/N5x9R.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/N5x9R.png&quot; alt=&quot;Angle between two half-planes (α, β, green) in a third plane (pink) which cuts the line of intersection at right angles&#xA;&quot; /&gt;&lt;/a&gt;&#xA;&amp;quot;Angle between two half-planes (α, β, green) in a third plane (pink) which cuts the line of intersection at right angles&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;In this image I interpret the atoms to be at the vertices of the planes, e.g. the two atoms that are in common between the two planes are those at the points of intersection of the planes, and the two other atoms extend the planes &lt;span class=&quot;math-container&quot;&gt;$\alpha$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2021-10-12T22:39:54.313" LastActivityDate="2021-10-12T22:39:54.313" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17831" PostTypeId="1" CreationDate="2021-10-11T06:04:14.770" Score="1" ViewCount="156" Body="&lt;p&gt;I am trying to convert chromosome positions to rsIDs of a GWAS summary statistics file. I used &lt;code&gt;bedtools intersect&lt;/code&gt; to merge the reference genome GRCh37 and the summary statistics based on the same chromosome positions.&lt;/p&gt;&#xA;&lt;p&gt;But, INDELs regions resulted in several rsIDs. Some regions counting +1 base pairs regions resulted in duplicate rsIDs. In some cases, a single SNP related to many positions were not included in the summary statistics file.&lt;/p&gt;&#xA;&lt;p&gt;I used this command &lt;code&gt;intersectBed -sorted -a sumstats.bed -b 00-All.vcf -wb&lt;/code&gt; but it did not work. Please let me know if anybody who had a similar experience and solved it before.&lt;/p&gt;&#xA;" OwnerUserId="12699" LastEditorUserId="4413" LastEditDate="2021-10-11T09:40:09.157" LastActivityDate="2021-10-11T09:40:09.157" Title="How to convert the chromosome position for insertions/deletions to rsIDs in a GWAS summary statistics?" Tags="|sequence-alignment|gwas|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17832" PostTypeId="1" CreationDate="2021-10-11T09:15:32.147" Score="1" ViewCount="31" Body="&lt;p&gt;I have a list of site-specific amino acid substitutions from a mutagenesis experiment which I am visualising by colouring the structure of the protein in PyMOL. I would like a way to colour the substitutions by some approximation of the biochemical (dis)similarity between them.&lt;/p&gt;&#xA;&lt;p&gt;So far, I have tried to create a colour gradient of scores from an alignment scoring matrix (e.g. BLOSUM62, therefore a 16-shade white to red gradient; a substitution of A to R for example gets a -1). However, I am fairly certain this is not the best interpretation of of the log-odds scores in the BLOSUM matrix, and I wondered if anyone had any thoughts on how I might better accomplish this?&lt;/p&gt;&#xA;&lt;p&gt;Note that other colour schemes such as hydrophobicity, Tailor etc. aren't suitable for my purpose.&lt;/p&gt;&#xA;" OwnerUserId="2841" LastActivityDate="2021-10-11T09:15:32.147" Title="Colouring amino acid substitutions by scoring matrix" Tags="|sequence-alignment|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17834" PostTypeId="2" ParentId="10825" CreationDate="2021-10-11T09:31:39.667" Score="1" Body="&lt;p&gt;Check &lt;a href=&quot;https://nf-co.re/&quot; rel=&quot;nofollow noreferrer&quot;&gt;nf-core&lt;/a&gt; for all the pipelines they have, including &lt;a href=&quot;https://nf-co.re/smrnaseq&quot; rel=&quot;nofollow noreferrer&quot;&gt;small-RNA seq&lt;/a&gt;. To run them, you have to use &lt;a href=&quot;https://nextflow.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nextflow&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2021-10-11T09:31:39.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17836" PostTypeId="1" CreationDate="2021-10-11T11:57:47.650" Score="1" ViewCount="226" Body="&lt;p&gt;I have a treatment and control in two time points like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; &amp;gt; design&#xA;                                 X condition time&#xA;    1       CTRL_24_hrs_replicate1   control   24&#xA;    2       CTRL_24_hrs_replicate2   control   24&#xA;    3       CTRL_24_hrs_replicate3   control   24&#xA;    4  treatment_24_hrs_replicate1        t   24&#xA;    5  treatment_24_hrs_replicate2        t   24&#xA;    6  treatment_24_hrs_replicate3        t   24&#xA;    7       CTRL_48_hrs_replicate1   control   48&#xA;    8       CTRL_48_hrs_replicate2   control   48&#xA;    9       CTRL_48_hrs_replicate3   control   48&#xA;    10 treatment_48_hrs_replicate1        t   48&#xA;    11 treatment_48_hrs_replicate2        t   48&#xA;    12 treatment_48_hrs_replicate3        t   48&#xA;    &amp;gt; &#xA;&#xA;                                               &#xA;    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to test between between treatment and control considering time point 2 hours to 4 hours&lt;/p&gt;&#xA;&lt;p&gt;I have done like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData=a,colData=design, design=~time + condition + time:condition)&#xA;&#xA;ddsTC &amp;lt;- DESeq(dds, test=&amp;quot;LRT&amp;quot;, reduced = ~ time + time:condition)&#xA;&#xA;&amp;gt; resultsNames(ddsTC)&#xA;[1] &amp;quot;Intercept&amp;quot;                &amp;quot;condition_t_vs_control&amp;quot;  &#xA;[3] &amp;quot;time&amp;quot;                     &amp;quot;conditiont.time&amp;quot; &#xA;&#xA; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What is the difference of the results of these codes&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;results(ddsTC, name=&amp;quot;condition_t_vs_control&amp;quot;, test=&amp;quot;Wald&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;versus&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;results(ddsTC, name=&amp;quot;conditiont.time&amp;quot;, test=&amp;quot;Wald&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Actually I want to know what &lt;code&gt;condition_t_vs_control&lt;/code&gt; gives and what &lt;code&gt;conditiont.time&lt;/code&gt; gives&lt;/p&gt;&#xA;&lt;p&gt;Thank you so much for any intuition&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2021-10-27T12:20:23.763" LastActivityDate="2021-10-27T12:20:23.763" Title="Different contrasts in DESeq2" Tags="|rna-seq|differential-expression|deseq2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17837" PostTypeId="1" CreationDate="2021-10-11T12:04:30.700" Score="1" ViewCount="119" Body="&lt;p&gt;I tried to install all the needed and related packages but I still did not know what the problem is, Can anyone please help if anything else I can do??&#xA;I always get this over and over:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Warning: Error in : Navigation containers expect a collection of `bslib::nav()`/`shiny::tabPanel()`s and/or `bslib::nav_menu()`/`shiny::navbarMenu()`s. Consider using `header` or `footer` if you wish to place content above (or below) every panel's contents.&#xA;  85: stop&#xA;  84: FUN&#xA;  83: lapply&#xA;  82: buildTabset&#xA;  81: tabsetPanel_&#xA;  80: func&#xA;  78: tabsetPanel&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14317" LastEditorUserId="3694" LastEditDate="2023-09-07T11:49:42.267" LastActivityDate="2023-09-07T11:49:42.267" Title="I can't launch FastqCleaner I always get a warning message and the application never starts" Tags="|r|bioconductor|fastq|rstudio|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17839" PostTypeId="1" CreationDate="2021-10-11T15:25:06.067" Score="5" ViewCount="227" Body="&lt;p&gt;The unweighted pair group method with arithmetic mean (UPGMA) is a hierarchical clustering method, for example used in phylogeny. In a phylogenetic tree it would  result in a global molecular clock for taxa under consideration.&lt;/p&gt;&#xA;&lt;p&gt;Two clusters &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; with the minimum distance are selected and merged to form a new cluster &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;, where the distance between &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; = &lt;span class=&quot;math-container&quot;&gt;$A \cup B$&lt;/span&gt;. Assuming another &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, we can compute:&#xA;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;d(C, D) = \frac{1}{|C| \cdot |D|} \sum_{x \in A \cup B,\, y \in D} d(x, y).&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;I think &lt;span class=&quot;math-container&quot;&gt;$d(x,y)$&lt;/span&gt; is the distance between objects &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; in the input matrix.&lt;/p&gt;&#xA;&lt;p&gt;Given the information, is there a way to prove/disprove the following statement?&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;d(C, D) = \frac{1}{|A| + |B|} \Big( |A| d(A, D) + |B| d(B, D) \Big).&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="14197" LastEditorUserId="17320" LastEditDate="2023-09-13T23:35:47.373" LastActivityDate="2024-02-11T01:07:55.950" Title="UPGMA computation" Tags="|phylogenetics|phylogeny|statistics|computation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17842" PostTypeId="1" CreationDate="2021-10-11T22:05:57.897" Score="2" ViewCount="27" Body="&lt;p&gt;I want to see the correlation between % mouse reads per sample vs % mouse cells per sample. I've already calculated the % mouse reads per sample but I'm stuck on calculating the % of mouse cells per sample and creating a scatterplot.&lt;/p&gt;&#xA;&lt;p&gt;I calculated the % mouse reads per sample by doing the following loop:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sampleid &amp;lt;- levels(df_combined&lt;span class=&quot;math-container&quot;&gt;$orig.ident)&#xA;df_fractions &amp;lt;- as.data.frame(sampleid)&#xA;for (i in 1:nrow(df_fractions)) {&#xA;sample &amp;lt;- df_fractions[i,1]&#xA;sample_tmp &amp;lt;- subset(df_combined,orig.ident == sample)&#xA;tmp_mouse_total_UMI = sum(sample_tmp$mouse_gene_UMI)&#xA;tmp_human_total_UMI = sum(sample_tmp$human_gene_UMI)&#xA;tmp_Total_UMI &amp;lt;- tmp_mouse_total_UMI+tmp_human_total_UMI&#xA;df_fractions$fraction_mouse[i] = tmp_mouse_total_UMI/tmp_Total_UMI&#xA;df_fractions$&lt;/span&gt;fraction_human[i] = tmp_human_total_UMI/tmp_Total_UMI &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any help is greatly appreciated it.&lt;/p&gt;&#xA;" OwnerUserId="6827" LastActivityDate="2021-10-11T22:05:57.897" Title="Calculating percent mouse cells per sample" Tags="|r|scrnaseq|seurat|ggplot2|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17843" PostTypeId="1" AcceptedAnswerId="17844" CreationDate="2021-10-12T12:17:39.477" Score="1" ViewCount="64" Body="&lt;p&gt;&amp;quot;&lt;a href=&quot;https://ciri-cookbook.readthedocs.io/en/latest/CIRIquant_1_installation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;CIRIquant&lt;/a&gt;&amp;quot; is a Python package for detecting Circular RNA. I installed &amp;quot;CIRIquant&amp;quot; through the below command in terminal:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;pip install CIRIquant&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;My Linux distribution is Ubuntu 16.04 and my python version is Python 2.7.12. After installing, I got the below message:&#xA;“Installing collected packages: numpy, scipy, scikit-learn, pysam, PyYAML, argparse, numexpr, CIRIquant&#xA;Successfully installed CIRIquant-1.1.2 PyYAML-5.1.1 argparse-1.2.1 numexpr-2.6.9 numpy-1.16.4 pysam-0.15.2 scikit-learn-0.20.3 scipy-1.2.2”&lt;/p&gt;&#xA;&lt;p&gt;So, When I call CIRIquant for running its function in Terminal, I got the below message in Terminal:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CIRIquant: command not found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, I don't know what is my problem?&#xA;I appreciate it if anybody shares his/her comment with me.&#xA;Best,&lt;/p&gt;&#xA;" OwnerUserId="4915" LastEditorUserId="4915" LastEditDate="2021-10-12T12:28:50.513" LastActivityDate="2021-10-12T15:30:59.983" Title="Error to running &quot;CIRIquant&quot; function in CIRIquant package at python Language" Tags="|rna-seq|python|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17844" PostTypeId="2" ParentId="17843" CreationDate="2021-10-12T15:30:59.983" Score="3" Body="&lt;p&gt;When I try this as a regular user on our Ubuntu 16 LTS system, I end up with &lt;code&gt;~/.local/bin/CIRIquant&lt;/code&gt; from the pip install, so maybe just add that directory to your path:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ export PATH=&amp;quot;$HOME/.local/bin:$PATH&amp;quot;&#xA;$ CIRIquant -h&#xA;usage: CIRIquant [-h] [--config FILE] [-1 MATE1] [-2 MATE2] [-o DIR]&#xA;                 [-p PREFIX] [-t INT] [-a INT] [-l INT] [-v] [--version]&#xA;                 [-e LOG] [--bed FILE] [--circ FILE] [--tool TOOL]&#xA;                 [--RNaseR FILE] [--bam BAM] [--no-gene]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2021-10-12T15:30:59.983" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17845" PostTypeId="1" AcceptedAnswerId="17846" CreationDate="2021-10-12T16:33:19.900" Score="1" ViewCount="57" Body="&lt;p&gt;a. Is there a SAM flag that specifies all reads?&#xA;b. Is there a SAM flag that specifies none of the reads?&lt;/p&gt;&#xA;&lt;p&gt;So that if I run samtools view -f (a) -F (b)&#xA;the result will be all reads of the file (as if omitting the specification)  ?&lt;/p&gt;&#xA;" OwnerUserId="6317" LastActivityDate="2021-10-12T20:31:25.147" Title="Is there a sam flag for all&amp;none of the reads?" Tags="|samtools|sam|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17846" PostTypeId="2" ParentId="17845" CreationDate="2021-10-12T16:48:29.003" Score="3" Body="&lt;p&gt;Instead of doing this kind of tweaking I would write my script in a way that these options are passed as strings that can be empty. Something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#/ dummy syntax just for illustration:&#xA;if(you_want_filtering){&#xA;  filter_f='-f something'&#xA;  filter_F='-F something_else'&#xA;} else {&#xA;  filter_f=filter_F=''&#xA;}&#xA;&#xA;samtools view (...) &lt;span class=&quot;math-container&quot;&gt;${filter_f} $&lt;/span&gt;{filter_F}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So if you want all reads then make these strings empty. That is far more generic than your idea because it can well be that the flag values you need do not exist.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2021-10-12T20:31:25.147" LastActivityDate="2021-10-12T20:31:25.147" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17847" PostTypeId="2" ParentId="17836" CreationDate="2021-10-12T20:22:38.123" Score="3" Body="&lt;p&gt;&lt;code&gt;condition_t_vs_control&lt;/code&gt; gives the effect of condition, &lt;code&gt;conditiont.time&lt;/code&gt; is the interaction of condition and time. If you wanted to test the effect of time, use &lt;code&gt;name=&amp;quot;time&amp;quot;&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="77" LastEditorUserId="4595" LastEditDate="2021-10-27T12:13:37.103" LastActivityDate="2021-10-27T12:13:37.103" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="17848" PostTypeId="1" CreationDate="2021-10-12T22:15:54.040" Score="1" ViewCount="19" Body="&lt;p&gt;yeastgenome.org has releases called R64-1-1, R64-2-1, etc, with 3 &amp;quot;levels&amp;quot; of versioning but in a few places I have seen a &amp;quot;fourth level&amp;quot;. For example, Ensemble calls their hosted version of R64-1-1 &amp;quot;version 51&amp;quot; (&lt;a href=&quot;http://ftp.ensemblgenomes.org/pub/fungi/release-51/fasta/saccharomyces_cerevisiae/dna/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ftp.ensemblgenomes.org/pub/fungi/release-51/fasta/saccharomyces_cerevisiae/dna/&lt;/a&gt;). And the pre-created databases available with snpEff 4.3 include &amp;quot;R64-1-1.75&amp;quot; and &amp;quot;R64-1-1.99&amp;quot;. I assume the &amp;quot;75&amp;quot; and &amp;quot;99&amp;quot; here correspond to Ensembl's &amp;quot;51&amp;quot;. What is this fourth number and what does it mean?&lt;/p&gt;&#xA;" OwnerUserId="14328" LastActivityDate="2021-10-12T22:15:54.040" Title="What is the 4th &quot;level&quot; of version number for the S. cerevisiae reference genome?" Tags="|reference-genome|snpeff|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17849" PostTypeId="1" AcceptedAnswerId="17855" CreationDate="2021-10-12T23:28:58.020" Score="0" ViewCount="104" Body="&lt;p&gt;I am using DEseq2 and trying to understand the results obtained using different models.&lt;/p&gt;&#xA;&lt;p&gt;I have a data design with 2 genotypes and 2 time points.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    sample genotype time&#xA;1   WT_S1       WT   T1&#xA;2   WT_S2       WT   T1&#xA;3   WT_S3       WT   T1&#xA;4   WT_S4       WT   T2&#xA;5   WT_S5       WT   T2&#xA;6   WT_S6       WT   T2&#xA;7   KO_S1       KO   T1&#xA;8   KO_S2       KO   T1&#xA;9   KO_S3       KO   T1&#xA;10  KO_S4       KO   T2&#xA;11  KO_S5       KO   T2&#xA;12  KO_S6       KO   T2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;I want to know the differences in results obtained from genotype coefficient and time coefficient when using different models.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Model1) ~ genotype&lt;/p&gt;&#xA;&lt;p&gt;Since it is comparing the differences in genotype regardless of time , samples 1-6 vs 7-12 are being compared.&lt;/p&gt;&#xA;&lt;p&gt;Model2) ~ time&lt;/p&gt;&#xA;&lt;p&gt;This model compares time T2 vs T1 regardless of genotypes. So it is comparing samples (1-3 + 7-9) vs (4-6 +10-12), &lt;strong&gt;is this correct?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Model3) ~ genotype + time&lt;/p&gt;&#xA;&lt;p&gt;My understanding is that this model assumes the genotype effect is the same at both time points and so it adds a time effect to both genotypes.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Does that mean results(obj3, name=&amp;quot;genotype_KO_vs_WT&amp;quot;) give differences in genotypes by comparing samples 4-6 vs 10-12 ?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What samples are being compared in the results obtained from results(obj3, name=&amp;quot;time_T2_vs_T1&amp;quot;)? How is it different from model 2?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Model 4) ~ genotype + time + genotype:time&lt;/p&gt;&#xA;&lt;p&gt;Here I understand results(obj4, name=&amp;quot;genotype_KO_vs_WT&amp;quot;) gives the differences in genotypes at reference levels ie, samples 1-3 vs 7-9&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;results(obj4, name=&amp;quot;time_T2_vs_T1&amp;quot;). What samples are compared here? How is it different from ‘time_T2_vs_T1’ results in model 2 or model3?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The interaction term as I understand is giving the specific effect due to KO at time T2 controlling for the baseline differences in genotypes.&#xA;results(obj5, name=&amp;quot;genotypeKO.timeT2&amp;quot;)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What samples are being compared from the model matrix?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="14329" LastActivityDate="2021-10-13T17:16:38.440" Title="Understanding the different designs in DESeq2" Tags="|r|statistics|differential-expression|deseq2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17852" PostTypeId="1" AcceptedAnswerId="17872" CreationDate="2021-10-13T08:14:07.237" Score="0" ViewCount="103" Body="&lt;p&gt;I am a complete rookie in shotgun analysis, having only performed marker gene analysis before. I have a pipeline that I want to try but I can't wrap my head around how it all works.&lt;/p&gt;&#xA;&lt;p&gt;Please correct me if I'm wrong:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I start with a paired-end Illumina short reads fastq files (.R1,.R2)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I will use FASTQC to trim according to quality. I assume that since data has been demultiplexed, the primers and and adaptors are removed as well.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;? First question, a lot of people advise to merge all the samples together and then do the co-assembly since apparently it helps with the assembly. Would this be necessary if I also have nanopore reads and only 9 samples? If if I did this, how do I then separate the samples from each other?&lt;/p&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Because I also have nanopore, I will use a hybrid assembler to get contigs. The output will be a file with contigs.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I then plan to use a binning tool (Metawrap) to create bins.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;? I believe the output of these will be a file containing bins per sample. How do I estimate the abundance of each bin in the sample? And how do I factor in the depth of sequencing to normalise them? This is my biggest problem, I can't wrap my head around it.&lt;/p&gt;&#xA;&lt;p&gt;?Can some explain VERY plainly what is sequencing depth and how do I factor it into my analysis? I've seen some people using it for normalisation of the bin/contigs abundance but again, this is idea is not intuitive to me at all.&lt;/p&gt;&#xA;&lt;ol start=&quot;5&quot;&gt;&#xA;&lt;li&gt;From then on, taxonomy and functional annotation is pretty straightforward.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;#Update&lt;/p&gt;&#xA;&lt;p&gt;So a little background on the project. I have samples sequenced from a bioreactor containing a small community of bacteria (no host removal required). The samples derive from 6 different reactors = 6 samples as well as one reactor will be samples three times (time series) so I have 9 samples in total. The shotgun Illumina will be combined with nanopore. The idea is to get the composition of the community (what kind of bacteria are there?), abundance (what is their relative quantity?) but also functions (what do they do?).&lt;/p&gt;&#xA;&lt;p&gt;I'm sorry for a very generic question, I would really appreciate somebody answering very plainly without too much jargon. Hopefully it will help me and other people who are just starting out with the shotgun.&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="14212" LastEditorUserId="14212" LastEditDate="2021-10-14T11:55:14.873" LastActivityDate="2022-03-14T23:03:55.417" Title="Shotgun analysis: Where and how to start?" Tags="|ngs|metagenome|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17854" PostTypeId="1" AcceptedAnswerId="17857" CreationDate="2021-10-13T15:43:22.603" Score="0" ViewCount="850" Body="&lt;p&gt;I used AlphaFold and Swiss-Model to get the structure of a protein (KiSS1R) and then compare them. The structures don't match and  I was wondering why the softwares produce models that are different in some regions?&lt;/p&gt;&#xA;&lt;p&gt;Thankful for all help!&lt;/p&gt;&#xA;" OwnerUserId="14299" LastActivityDate="2021-10-14T10:32:32.117" Title="Different structures of same protein from two softwares" Tags="|proteins|database|protein-structure|software-quality|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17855" PostTypeId="2" ParentId="17849" CreationDate="2021-10-13T17:16:38.440" Score="0" Body="&lt;p&gt;I strongly suggest that you take your normalized counts, get the averages of different groups, and compare those ratios to the different designs.&lt;/p&gt;&#xA;&lt;p&gt;That said, you might not be able to replicate the fold changes from ~ genotype + time well in Excel.  What that design does is basically say &amp;quot;Lots of the variance within each genotype/time point is due to the time point/genotype.  Model that so that those differences are taken into account.&amp;quot;  Whereas a design of just ~ time, the software will go &amp;quot;Oh, there's a lot of variance within each time point&amp;quot;.  You won't be able to replicate that well in Excel...but that's why you are using DESeq.&lt;/p&gt;&#xA;&lt;p&gt;That design is probably better than just time or genotype alone, if you want to look at all the KO versus all the WT&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Does that mean results(obj3, name=&amp;quot;genotype_KO_vs_WT&amp;quot;) give&#xA;differences in genotypes by comparing samples 4-6 vs 10-12 ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No.  The simplest way to do that comparison, and comparisons like it is to make a third data column that's genotype_time, and use contrasts to specify what subgroup to compare to what subgroup.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Here I understand results(obj4, name=&amp;quot;genotype_KO_vs_WT&amp;quot;) gives the&#xA;differences in genotypes at reference levels ie, samples 1-3 vs 7-9&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes.  But since that's super confusing, I'd recommend against it.  The way I outlined above is far more readable, and should work the same.&lt;/p&gt;&#xA;&lt;p&gt;The interaction design is best used when you want to see of the changes caused by time are different in the KO than the WT.  In theory, you could just generate the two timepoint fold changes, and subtract one from the other, but you'd have no p-value.  Using a design with interactions will get you that fold change difference with a p-value.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2021-10-13T17:16:38.440" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17856" PostTypeId="1" AcceptedAnswerId="17866" CreationDate="2021-10-13T18:06:01.457" Score="1" ViewCount="1153" Body="&lt;p&gt;I want to use a file available on a database called &lt;code&gt;DepMap&lt;/code&gt; (&lt;a href=&quot;https://depmap.org/portal/download/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://depmap.org/portal/download/&lt;/a&gt;). This file called &lt;code&gt;CRISPR_gene_effect.csv&lt;/code&gt; contains gene ids in the format &lt;code&gt;HUGO (Entrez)&lt;/code&gt; eg. &lt;code&gt;TP53 (7157)&lt;/code&gt;, without having any complementary information regarding which genome assembly (GRCh37/38) the Entrez ids (and also HUGO symbols) belong to.&lt;/p&gt;&#xA;&lt;p&gt;I want to convert these gene ids in the &lt;code&gt;HUGO (Entrez)&lt;/code&gt; format to  GRCh38 Ensembl ids (specifically to Ensembl release 100), so that I can map the data to Ensembl's annotations. I would like to get suggestions on this.&lt;/p&gt;&#xA;&lt;p&gt;I feel that figuring out the genome assembly version (GRCh37/38) of the Entrez ids would be critical in this conversion. Although, I am not aware if Entrez ids change from one genome assembly to other.&lt;/p&gt;&#xA;" OwnerUserId="274" LastEditorUserId="274" LastEditDate="2021-10-15T13:28:31.600" LastActivityDate="2021-10-16T15:36:13.890" Title="Convert Entrez gene ids and HUGO symbols, whose genome assembly is unknown, to GRCh38 Ensembl gene ids" Tags="|ensembl|identifiers|entrez|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17857" PostTypeId="2" ParentId="17854" CreationDate="2021-10-13T21:03:42.437" Score="2" Body="&lt;h2&gt;Superposition&lt;/h2&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://michelanglo.sgc.ox.ac.uk/data/d6e427bb-a0cb-4a65-b988-b08237f5054a&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://michelanglo.sgc.ox.ac.uk/data/d6e427bb-a0cb-4a65-b988-b08237f5054a&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h2&gt;SwissModel&lt;/h2&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;https://swissmodel.expasy.org/repository/uniprot/Q969F8&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;SwissModel models are threaded against a template. In this case the homologue has 31% identity, which isn't great. The template is 6TP3 which has a rather high R_free of 0.32 and it's solved at 3 Å, but it's a membrane structure so I personally would forgive it —others may not. SwissModel has switched primary metric to its QMEANDisCo, which is present as global (0.6 in this case) and per residue (saved in the PDB file as the temperature factor column), which is roughly similar conceptually to AlphaFold2's pLDDT (vide infra), i.e. it tells you what resides to trust. 0.6 not great, not terrible. A nice thing in SwissModel is that, by virtue of being threaded, it's easy to steal ligands and other protein and it shows models of oligomers.&lt;/p&gt;&#xA;&lt;h2&gt;AlphaFold2&lt;/h2&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;https://alphafold.ebi.ac.uk/entry/Q969F8&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;AlphaFold2 is by far the best modelling algorithm as see in the latest CASP competition (#14). However it natively doesn't do oligomers —&lt;a href=&quot;https://github.com/sokrypton/ColabFold&quot; rel=&quot;nofollow noreferrer&quot;&gt;ColabFold notebook&lt;/a&gt; can.&lt;/p&gt;&#xA;&lt;p&gt;AlphaFold2 gives a per residue score (plDDT). AlphaFold2 models the whole sequence, which results in some nasty &amp;quot;spaghetti loops&amp;quot;: these have really bad scores, so need to be ignored. These are not trash, but are likely unstructured in isolation, but bind other protein etc. (&lt;a href=&quot;https://blog.matteoferla.com/2021/07/what-to-look-out-for-with-alphafold2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;more info in this blog post&lt;/a&gt;). This structure has N- and C-terminus loops that need to be ignored. The N in particular is the signal that would be cleaved off —to get the membrane embedding dots, one can use the &lt;a href=&quot;https://opm.phar.umich.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OPM server&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Choice&lt;/h2&gt;&#xA;&lt;p&gt;The difference in the two models only involves low certainly regions and the support for the SwissModel is problematic. So the AlphaFold2 model is better, but requires removal of the C- and N-terminal loops.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2021-10-14T10:32:32.117" LastActivityDate="2021-10-14T10:32:32.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17858" PostTypeId="1" CreationDate="2021-10-14T07:00:33.833" Score="0" ViewCount="88" Body="&lt;p&gt;&amp;quot;&lt;a href=&quot;https://ciri-cookbook.readthedocs.io/en/latest/CIRIquant_2_quantification.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;CIRIquant&lt;/a&gt;&amp;quot; is a Python package for detecting Circular RNA. I ran &amp;quot;CIRIquant&amp;quot; individually and it was run. consequently, I ran CIRIquant through the below command and parameters in the terminal:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; ~/.local/bin/CIRIquant -t  4&#xA;&#xA;     -1 ./SRR1176949_1.fastq.gz     &#xA;     -2 ./SRR1176949_2.fastq.gz &#xA;     --config ./CIRIquant-Config.yml              &#xA;     -o ./SRR1176949   &#xA;     -p  SRR1176949&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I got the below message:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &amp;quot;/home/genetic/.local/bin/CIRIquant&amp;quot;, line 11, in &amp;lt;module&amp;gt;&#xA;    sys.exit(main())&#xA;  File &amp;quot;/home/genetic/.local/lib/python2.7/site-packages/CIRIquant/main.py&amp;quot;, line 89, in main&#xA;    config = check_config(check_file(args.config_file))&#xA;  File &amp;quot;/home/genetic/.local/lib/python2.7/site-packages/CIRIquant/utils.py&amp;quot;, line 82, in check_config&#xA;    globals()[i.upper()] = check_file(config['tools'][i])&#xA;  File &amp;quot;/home/genetic/.local/lib/python2.7/site-packages/CIRIquant/utils.py&amp;quot;, line 49, in check_file&#xA;    raise ConfigError('File: {}, not found'.format(file_name))&#xA;CIRIquant.utils.ConfigError: File: /home/genetic/ncbi/dbGaP_26368/bwa_sam_CIRI/bwa, not found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;###########&#xA;My &amp;quot;CIRIquant-Config.yml&amp;quot; content is as below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tools:&#xA;  bwa: /home/genetic/ncbi/dbGaP_26368/bwa_sam_CIRI/bwa&#xA;  hisat2: /usr/bin/hisat2-2.2.1&#xA;  stringtie: /usr/bin/stringtie&#xA;  samtools: /usr/bin/samtools-1.9&#xA;&#xA;reference:&#xA;  fasta: /home/genetic/ncbi/dbGaP-26368/fasta_file/hg19.fa&#xA;  gtf: /home/genetic/ncbi/dbGaP-26368/gtf_file/gencode.v19.annotation.gtf&#xA;  bwa_index: /home/genetic/ncbi/dbGaP-26368/bwa-sam-CIRI/bwa-0.7.17/hg19&#xA;  hisat_index: /usr/bin/hisat2-2.2.1/hg19&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I apprecite it if anyone share his/her comments with me.&lt;/p&gt;&#xA;" OwnerUserId="4915" LastActivityDate="2021-10-14T07:00:33.833" Title="Error in running CIRIquant package for 2 RNA-seq in Ubuntu 16.04" Tags="|python|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="17860" PostTypeId="1" CreationDate="2021-10-14T09:46:15.440" Score="0" ViewCount="75" Body="&lt;p&gt;I have a vcf with multi-allelic SNPs, e.g. &lt;code&gt;REF: A&lt;/code&gt; and &lt;code&gt;ALT: G, C&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I would like to combine all reference alleles to the most frequent reference alleles. This should reflect both in the REF field and in the genotypes themselves.&lt;/p&gt;&#xA;&lt;p&gt;Can this be done with an existing tool?&lt;/p&gt;&#xA;" OwnerUserId="14341" LastActivityDate="2021-10-14T09:46:15.440" Title="Combine all alternate alleles" Tags="|vcf|snp|bcftools|multi-allelic|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17861" PostTypeId="1" CreationDate="2021-10-14T11:14:30.820" Score="2" ViewCount="89" Body="&lt;p&gt;I am aligning a fastq file as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cutadapt -j 8 --quality-base 33 -l 20 \&#xA;     -o - /home/user/Documents/analyses/CRISPR-screens/screen3/D16.fq.gz | &#xA;       ~/bowtie2-2.4.3-linux-x86_64/bowtie2 --no-hd -p 8 \&#xA;         -t -N 0 -x ~/moffat_tko3 - | &#xA;           sed '/XS:/d' | cut -f3 | sort | uniq -c &amp;gt; ~/D16.guidecounts.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The expected output is similar to this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head D16.guidecounts.txt &#xA;329 A1BG_sgA1BG_1&#xA;669 A1BG_sgA1BG_2&#xA;65 A1BG_sgA1BG_3&#xA;397 A1BG_sgA1BG_4&#xA;100 A1CF_sgA1CF_1&#xA;731 A1CF_sgA1CF_2&#xA;396 A1CF_sgA1CF_3&#xA;536 A1CF_sgA1CF_4&#xA;427 A2ML1_sgA2ML1_1&#xA;978 A2ML1_sgA2ML1_2&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, there is no output from this command as &lt;code&gt;bowtie2&lt;/code&gt; gets stuck. &lt;code&gt;cutadapt&lt;/code&gt; works fine, but when that step is done, nothing happens for hours and it should only take a few minutes.&#xA;I have tried to take a subset of reads using &lt;code&gt;head&lt;/code&gt; and &lt;code&gt;tail&lt;/code&gt; as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zcat D16.fq.gz | head -10000 | gzip -c &amp;gt; D16head.fq.gz&#xA;zcat D16.fq.gz | tail -10000 | gzip -c &amp;gt; D16tail.fq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What confuses me is that these files don't freeze &lt;code&gt;bowtie2&lt;/code&gt; and align well (&amp;gt; 90% alignment rates). Other files from the same sequencing run also align fine (NovaSeq SP flow cell 50 bp paired-end).&lt;/p&gt;&#xA;&lt;p&gt;As far as I can tell the original fastq file looks fine:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zcat D16.fq.gz | head -12&#xA;@A00489:1088:HKN2TDRXY:2:2101:1334:1172 1:N:0:GCCAAT&#xA;GCCCAGTGACCCTTTCACGGGTTTTAGAGCTAGAAATAGCAAGTTAAAAT&#xA;+&#xA;FFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF,,FFFF&#xA;@A00489:1088:HKN2TDRXY:2:2101:1606:1172 1:N:0:GCCAAT&#xA;AAAGTTCTATCTGTTGGGCGGTTTTAGAGCTAGAAATAGCAAGTTCAAGT&#xA;+&#xA;FFFFFFFFFFFFFFFFFFFFF:FFFFFFF:FFFFF:FFFFFFFFF,FFFF&#xA;@A00489:1088:HKN2TDRXY:2:2101:1624:1172 1:N:0:GCCAAT&#xA;GCGACGGTTCATCTCCGCAAGTTTTAGAGCTAGAAATAGCAAGTTAAAAT&#xA;+&#xA;FFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFF:FFFFFFFF,FFFF&#xA;&#xA;&#xA;zcat D16.fq.gz | tail -12&#xA;@A00489:1088:HKN2TDRXY:2:2278:30644:37059 1:N:0:GCCAAT&#xA;AGGTCATCCACTCTAGACCTGTTTTAGAGCTAGAAATAGCAAGTTAAAAT&#xA;+&#xA;FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF&#xA;@A00489:1088:HKN2TDRXY:2:2278:30915:37059 1:N:0:GCCAAT&#xA;ATGAGGAGGGTGATGTTCCGGAGTTTTAGAGCTAGAAATAGCAAGTTAAA&#xA;+&#xA;FF:FFFFFFFFFFFFFFFFFFFFFFF,FFFFFF:FFFFFFFFFFFFFFFF&#xA;@A00489:1088:HKN2TDRXY:2:2278:31638:37059 1:N:0:GCCAAT&#xA;GAGTGTGGTTGAGATCCCAAGTTTTAGGCTAGAAATAGCAAGTTAAAATA&#xA;+&#xA;FF:FF:F:FFFFFFFF,FFFFFF:FF::FFFFFFFFFFFFFFFFFFFFFF&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does anyone know what could be wrong?&lt;/p&gt;&#xA;" OwnerUserId="10386" LastEditorUserId="298" LastEditDate="2021-10-15T12:31:08.740" LastActivityDate="2021-10-15T12:31:08.740" Title="Bowtie2 gets stuck on alignment" Tags="|sequence-alignment|fastq|bowtie2|cutadapt|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17865" PostTypeId="1" CreationDate="2021-10-10T23:49:40.217" Score="0" ViewCount="57" Body="&lt;p&gt;This question is more on the theoretical-encoding side. Given the known kinds of genetic mutation, transposition, deletion, and such, what can be said about genes that would be left untouched by the mutation, left invariant by the permutation, if you will? Are there any such sequences? Would they be more common in a population, if not detrimental?&lt;/p&gt;&#xA;" OwnerUserId="14376" OwnerDisplayName="Elkin Montoya" LastActivityDate="2021-10-17T15:08:44.577" Title="Is there any genetic sequence, such that, for all the kinds of mutation, it still has a 100% probability to be passed on?" Tags="|genetics|molecular-genetics|phylogenetics|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="17866" PostTypeId="2" ParentId="17856" CreationDate="2021-10-15T09:00:19.390" Score="2" Body="&lt;p&gt;Hugo names don't really have an associated assembly. They are just the official name of the gene in human. You will find them in all assemblies (except if they hadn't been annotated/identified in one of the older ones). Entrez entries are also not tied to a specific genome assembly. They usually have mappings to multiple ones. For example, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene/?term=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hugo gene name A1BG, with Entrez ID 1&lt;/a&gt; has:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Yj7Wd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Yj7Wd.png&quot; alt=&quot;Screenshot showing the Entrez entry and the hg19 and hg38 coordinates&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, if you want to get Ensembl gene IDs for a list of Hugo gene names, you can use &lt;a href=&quot;https://www.ensembl.org/info/data/biomart/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioMart&lt;/a&gt; to convert. First, get the list of target IDs. You can use either Hugo or Entrez. I found the Entrez simpler to parse out, so I went for those (note that this assumes GNU &lt;code&gt;grep&lt;/code&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -oP '(?&amp;lt;=\()\d+(?=\))' CRISPR_gene_effect.csv &amp;gt; entrez.ids&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then went to BioMart and selected &amp;quot;Ensembl genes 104&amp;quot; (note that I could only get Ensembl version 104, I assume you're not adamant about getting version 100 given that it doesn't seem to be current) and &amp;quot;Human genes GRCh38.p13&amp;quot; :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wHKly.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wHKly.png&quot; alt=&quot;BioMart screenshot1&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Next, I set the filters to &amp;quot;NCBI gene (formerly Entrezgene) ID(s)&amp;quot; and uploaded my file:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DQFII.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DQFII.png&quot; alt=&quot;BioMart screenshot2&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And, finally, selected the Attributes I want to see, namely Ensembl gene ID, HGNC (Hugo) symbol and Entrez ID so I can map everything to each other:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/fykjh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/fykjh.png&quot; alt=&quot;BioMart screenshot3&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/cJ4vw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cJ4vw.png&quot; alt=&quot;BioMart screenshot4&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I then clicked on &amp;quot;Results&amp;quot; and got:&lt;/p&gt;&#xA;&lt;p&gt;[&lt;img src=&quot;https://i.stack.imgur.com/LfrE0.png&quot; alt=&quot;BioMart screenshot58&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can now download this to a file and have your mapping. You might even be able to use the one I created: &lt;a href=&quot;https://www.ensembl.org/biomart/martview/352aae8e0b5c774cbbee73d67634d577&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ensembl.org/biomart/martview/352aae8e0b5c774cbbee73d67634d577&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2021-10-16T15:36:13.890" LastActivityDate="2021-10-16T15:36:13.890" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17867" PostTypeId="1" CreationDate="2021-10-15T12:29:02.373" Score="1" ViewCount="82" Body="&lt;p&gt;I am using positive selection analysis for each leaves in a phylogenetic tree.&lt;/p&gt;&#xA;&lt;p&gt;In the routine calculation via &lt;code&gt;codeml&lt;/code&gt;, I will be running model zero for all leaves and then model 2 for each leaves as foreground (marking leaves as #1 in phylogenetic tree), then calculate selection using a likelihood ratio test.&lt;/p&gt;&#xA;&lt;p&gt;Is there a method for using &lt;a href=&quot;https://gitlab.sib.swiss/phylo/fastcodeml&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;FastCodeML&lt;/code&gt;&lt;/a&gt; for each leaves as foreground instead of the branch?&lt;/p&gt;&#xA;" OwnerUserId="14351" LastEditorUserId="3848" LastEditDate="2022-08-18T13:08:23.577" LastActivityDate="2024-02-09T18:04:10.503" Title="Selection analysis based on each leaves as foreground with FastCodeML" Tags="|phylogenetics|phylogeny|positive-selection|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17868" PostTypeId="1" AcceptedAnswerId="17869" CreationDate="2021-10-15T13:47:32.860" Score="3" ViewCount="93" Body="&lt;p&gt;Some websites mention amino acid mutations as ORF1ab, most however use ORF1a/b instead.&lt;/p&gt;&#xA;&lt;p&gt;So how can I translate from one to the other? What do I need to subtract from say &lt;code&gt;ORF1ab:4588I&lt;/code&gt; to get the corresponding &lt;code&gt;N&lt;/code&gt; in &lt;code&gt;ORF1b:&amp;lt;N&amp;gt;I&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="12272" LastEditorUserId="3848" LastEditDate="2022-02-22T14:27:39.440" LastActivityDate="2022-11-28T16:25:43.773" Title="How to convert Sars-CoV-2 ORF1ab codon positions to ORF1b positions?" Tags="|sars-cov-2|covid-19|codon|nomenclature|orf|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17869" PostTypeId="2" ParentId="17868" CreationDate="2021-10-15T13:59:03.590" Score="2" Body="&lt;p&gt;&lt;code&gt;ORF1ab:4588I&lt;/code&gt; corresponds to &lt;code&gt;ORF1b:187I&lt;/code&gt; so one needs to subtract &lt;code&gt;4401&lt;/code&gt; to go from &lt;code&gt;1ab&lt;/code&gt; to &lt;code&gt;1b&lt;/code&gt; and add &lt;code&gt;4401&lt;/code&gt; to go the other way.&lt;/p&gt;&#xA;" OwnerUserId="12272" LastEditorUserId="12272" LastEditDate="2022-02-28T13:33:29.513" LastActivityDate="2022-02-28T13:33:29.513" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17871" PostTypeId="1" CreationDate="2021-10-15T17:02:15.027" Score="0" ViewCount="465" Body="&lt;p&gt;I am a beginner and trying to read a bam file in Python.&lt;/p&gt;&#xA;&lt;p&gt;The lines below throw the error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;OSError: [Errno 8] could not open alignment file `mybam.bam`: Exec format error&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What could be the possible solution?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;user@server:~/Documents$ python3&#xA;Python 3.8.10 (default, Jun  2 2021, 10:49:15)&#xA;[GCC 9.4.0] on linux&#xA;Type &amp;quot;help&amp;quot;, &amp;quot;copyright&amp;quot;, &amp;quot;credits&amp;quot; or &amp;quot;license&amp;quot; for more information.&#xA;&#xA;&amp;gt;&amp;gt;&amp;gt; import pysam&#xA;&amp;gt;&amp;gt;&amp;gt; samfile = pysam.AlignmentFile(&amp;quot;mybam.bam&amp;quot;, &amp;quot;rb&amp;quot;)&#xA;&#xA;[E::hts_hopen] Failed to open file mybam.bam&#xA;[E::hts_open_format] Failed to open file &amp;quot;mybam.bam&amp;quot; : Exec format error&#xA;Traceback (most recent call last):&#xA;  File &amp;quot;&amp;lt;stdin&amp;gt;&amp;quot;, line 1, in &amp;lt;module&amp;gt;&#xA;  File &amp;quot;pysam/libcalignmentfile.pyx&amp;quot;, line 742, in pysam.libcalignmentfile.AlignmentFile.__cinit__&#xA;  File &amp;quot;pysam/libcalignmentfile.pyx&amp;quot;, line 941, in pysam.libcalignmentfile.AlignmentFile._open&#xA;OSError: [Errno 8] could not open alignment file `mybam.bam`: Exec format error&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14354" LastEditorUserId="650" LastEditDate="2021-10-15T17:27:32.187" LastActivityDate="2021-10-15T17:27:32.187" Title="pysam &quot;Exec format&quot; error" Tags="|python|sequence-alignment|sam|file-formats|pysam|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17872" PostTypeId="2" ParentId="17852" CreationDate="2021-10-15T22:58:24.013" Score="1" Body="&lt;p&gt;&lt;strong&gt;Question 1&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;For co-assembly vs individual assembly, it is sometimes advisable to do them separately, perhaps if you expect samples to be very different or if you have huge amounts of data. For &amp;lt;10 species I imagine that you will have quite high coverage, especially with nanopore.&lt;/p&gt;&#xA;&lt;p&gt;Some metagenomic binners (e.g. &lt;a href=&quot;https://peerj.com/articles/7359&quot; rel=&quot;nofollow noreferrer&quot;&gt;MetaBAT2&lt;/a&gt;, a standard from a few years ago) can even incorporate information across samples, which might argue for maintaining at least some separation.&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;How do I separate the samples from each other&amp;quot; I guess the point is that you &lt;strong&gt;don't&lt;/strong&gt;, for assembly at least. You can then go in post hoc with your sequencing data from each sample and try to figure out with abundance analysis which of your genomes are in each sample.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;For abundance analysis/depth, there are strong statistical parallels in RNA-seq analysis, &lt;a href=&quot;https://liorpachter.wordpress.com/2015/10/27/straining-metagenomics/&quot; rel=&quot;nofollow noreferrer&quot;&gt;as Lior Pachter is fond of saying&lt;/a&gt;, if that is helpful. Any number of metagenomic shotgun abundance computations can be used to estimate abundance of genomes, including Pachter's metakallisto (linked therein). I found metakallisto fairly straightforward to use on contigs resulting from assembly.&lt;/p&gt;&#xA;&lt;p&gt;Basically, you have some set of DNAs in your sample with some stoichiometry (relative abundance with respect to each other). The idea is to use DNA sequencing to estimate that stoichiometry (absolute abundance is much harder so most people don't bother).&lt;/p&gt;&#xA;&lt;p&gt;Here is a broad view of the workflow:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;You come up with some estimate of the DNAs present in the community. This can be a shotgun assembly, or some set of reference genomes, or whatever.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You count how many sequencing reads derive from each of those DNAs, usually via alignment or pseudoalignment.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;YOu do some numerical corrections for systematic covariates such as DNA length. (These can look like RPKMs or TPMs from RNA-seq; metakallisto literally uses &lt;a href=&quot;https://liorpachter.wordpress.com/tag/tpm/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TPM calculations&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The resulting counts should be abundances, more or less.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;You don't have to figure this stuff out yourself. There are various end to end pipelines that do this.&lt;/p&gt;&#xA;&lt;p&gt;You &lt;em&gt;will&lt;/em&gt; almost certainly need to compute abundances of contigs if you want to do metagenomic binning.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-10-15T22:58:24.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17873" PostTypeId="1" CreationDate="2021-10-16T08:51:45.000" Score="1" ViewCount="46" Body="&lt;p&gt;I have a set of NCBI txids (of varying ranks) that I want to sort in some way (e.g. a table or a tree, the way it is sorted isn't super important), based on the current NCBI data for phylogenetic relatedness.&lt;/p&gt;&#xA;&lt;p&gt;I don't have so many txids that I can map phylogenetic relationships at a really high resolution, but if, say, the txids for &lt;code&gt;plants, mammals, chordates, humans&lt;/code&gt; could be sorted so that humans are nested within mammals which is nested within chordates, which is separated from plants.&lt;/p&gt;&#xA;&lt;p&gt;Is there a tool that can do this?&lt;/p&gt;&#xA;" OwnerUserId="14358" LastActivityDate="2021-10-16T08:51:45.000" Title="Tool that can sort NCBI taxonomy ids based on phylogeny" Tags="|phylogeny|phylogenetics|taxonomy|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17874" PostTypeId="2" ParentId="17865" CreationDate="2021-10-17T09:51:04.657" Score="1" Body="&lt;p&gt;Not in the way you suggest: a sequence of letters with the property of being immutable by any type of permutation on its constituent bases. I would be interested if anything of the likes exists in mathematics.&lt;/p&gt;&#xA;&lt;p&gt;Having said that, there are examples of sequences that seem to have such property but only in the light of evolution and the biological processes that come with it:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Ultra-conserved_element&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ultra-conserved regions&lt;/a&gt; have been described as sequences of about 200 base pairs that appear to be exactly the same from humans to mice. They have not experience change in about ~300 million years of evolution (!) and exposure to the various forces of mutation.&lt;/p&gt;&#xA;&lt;p&gt;The reason for this is not an underlying 'invariant to permutation' property, but rather strong negative selection that prevents individuals that acquire changes on the sequence from being able to pass the changes over to their progeny, thus effectively maintaining only the original version in the population.&lt;/p&gt;&#xA;&lt;p&gt;The exact mechanisms, functions and properties of these regions are under research. You might want to see if the invariant properties you have in mind have anything to do with these interesting regions.&lt;/p&gt;&#xA;" OwnerUserId="475" LastEditorUserId="475" LastEditDate="2021-10-17T15:08:44.577" LastActivityDate="2021-10-17T15:08:44.577" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17876" PostTypeId="1" CreationDate="2021-10-10T20:12:13.140" Score="0" ViewCount="17" Body="&lt;p&gt;The BioPython documentation shows the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;11.6.3 Measuring torsion angles Use the vector representation of the atomic coordinates and the calc dihedral function from the Vector module:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; vector1 = atom1.get_vector()&#xA;&amp;gt;&amp;gt;&amp;gt; vector2 = atom2.get_vector()&#xA;&amp;gt;&amp;gt;&amp;gt; vector3 = atom3.get_vector()&#xA;&amp;gt;&amp;gt;&amp;gt; vector4 = atom4.get_vector()&#xA;&amp;gt;&amp;gt;&amp;gt; angle = calc_dihedral(vector1, vector2, vector3, vector4)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;We know that a torsion angle between N-Cα and Cα-C planes.&lt;/p&gt;&#xA;&lt;p&gt;So, the minimum number of points required to form a plane is 3.&lt;/p&gt;&#xA;&lt;p&gt;Then, Why does BioPython require 4 vectors?&lt;/p&gt;&#xA;" OwnerUserId="12686" OwnerDisplayName="user366312" LastActivityDate="2021-10-17T12:57:57.983" Title="Why does it require four vectors to calculate a torsion angle?" Tags="|python|" AnswerCount="0" CommentCount="0" FavoriteCount="0" ClosedDate="2021-10-17T21:20:03.260" ContentLicense="CC BY-SA 4.0" />
  <row Id="17877" PostTypeId="2" ParentId="17829" CreationDate="2021-10-13T11:39:30.447" Score="4" Body="&lt;p&gt;To compute dihedral angles you require 4 atoms and hence for vectors to determine two planes (&lt;a href=&quot;https://foldit.fandom.com/wiki/Torsion_angle&quot; rel=&quot;nofollow noreferrer&quot;&gt;see this figure here&lt;/a&gt;) whose angle you want to measure - dihedral angle.&lt;/p&gt;&#xA;&lt;p&gt;You can also check Fig 55 from the GROMACS documentation&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://manual.gromacs.org/documentation/current/reference-manual/analysis/bond-angle-dihedral.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://manual.gromacs.org/documentation/current/reference-manual/analysis/bond-angle-dihedral.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Definition of (virtual) Cα backbone bond (κ) and torsion (τ) angles.&#xA;from &lt;a href=&quot;https://journals.plos.org/plosone/article/figure?id=10.1371/journal.pone.0215141.g002&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://journals.plos.org/plosone/article/figure?id=10.1371/journal.pone.0215141.g002&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/vfp3S.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vfp3S.png&quot; alt=&quot;from https://journals.plos.org/plosone/article/figure?id=10.1371/journal.pone.0215141.g002&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerDisplayName="Broncio Aguilar-Sanjuan" LastActivityDate="2021-10-15T14:43:31.633" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17878" PostTypeId="1" CreationDate="2021-10-18T08:39:08.930" Score="0" ViewCount="321" Body="&lt;p&gt;So, I have limited knowledge of R but I need to do a PCA analysis of 3 different datasets of gene expression as a result of combined growth or mono-culture growth.&lt;/p&gt;&#xA;&lt;p&gt;The 3 different datasets I performed DESeq2 analysis on are as follows:&lt;/p&gt;&#xA;&lt;p&gt;Dataset 1: This was sequenced by Ion Torrent and it was single-end reads. Samples that were compared were biological quadruplicates of yeast species A grown as pure culture versus species A grown in direct contact with yeast species B. This was all done for 1 time point @ 24h.&lt;/p&gt;&#xA;&lt;p&gt;Dataset 2: This was sequenced by Illumina MiSeq and paired-end reads were generated, but this dataset had two parts to it.&lt;/p&gt;&#xA;&lt;p&gt;Samples that were compared were biological quadruplicates of yeast species A grown as pure culture versus species A grown in direct contact with yeast species B. This dataset contains 2 time points @ 2h and 24h.&lt;/p&gt;&#xA;&lt;p&gt;For the other part of this, samples that were compared were biological quadruplicates of yeast species A grown as pure culture versus species A grown via indirect (metabolic exchange possible) contact with yeast species B. This dataset contains 2 time points @ 2h and 24h.&lt;/p&gt;&#xA;&lt;p&gt;Dataset 3: This was sequenced by Illumina MiSeq and paired-end reads were generated, but this dataset had two parts to it.&lt;/p&gt;&#xA;&lt;p&gt;Here, two biological repeats of samples of yeast species A grown as pure culture versus two biological repeats of species A grown in direct contact with yeast species B, under AEROBIC conditions were performed. This dataset contains 1 time point @ 48h.&lt;/p&gt;&#xA;&lt;p&gt;Then one biological repeat of sample of yeast species A grown as pure culture versus two biological repeats of species A grown in direct contact with yeast species B, under ANAEROBIC conditions were performed. This dataset contains 1 time point @ 48h.&lt;/p&gt;&#xA;&lt;p&gt;I wrote the results out of the normalized counts per sample combined with the DESeq2 results as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;res &amp;lt;- results(dds)&#xA;table(res$padj&amp;lt;0.05)&#xA;res &amp;lt;- res[order(res$padj), ]&#xA;resdata &amp;lt;- merge(as.data.frame(res), as.data.frame(counts(dds, normalized=TRUE)), by=&amp;quot;row.names&amp;quot;, sort=FALSE)&#xA;names(resdata)[1] &amp;lt;- &amp;quot;Gene&amp;quot;&#xA;head(resdata)&#xA;write.csv(resdata, file=&amp;quot;diffexpr-results.csv&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So the issue I am having is, because these are different datasets and conditions differ across them there are different numbers of genes expressed in each dataset that are not present across all datasets. So I need to combine all the datasets to only compare the genes found in all the datasets mentioned and then perform a PCA analysis on those genes only.&lt;/p&gt;&#xA;&lt;p&gt;An example of the setup of the excel files I have, without the other info that you get with DESeq2 analysis e.g. columns &lt;code&gt;baseMean&lt;/code&gt;, &lt;code&gt;log2FoldChange&lt;/code&gt;, &lt;code&gt;lfcSE&lt;/code&gt;, &lt;code&gt;stat&lt;/code&gt;, &lt;code&gt;pvalue&lt;/code&gt; and &lt;code&gt;padj&lt;/code&gt;, is as follows:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;Gene_ID&lt;/th&gt;&#xA;&lt;th&gt;Sample1_PC&lt;/th&gt;&#xA;&lt;th&gt;Sample2_PC&lt;/th&gt;&#xA;&lt;th&gt;Sample3_PC&lt;/th&gt;&#xA;&lt;th&gt;Sample4_PC&lt;/th&gt;&#xA;&lt;th&gt;Sample5_Mixed&lt;/th&gt;&#xA;&lt;th&gt;Sample6_Mixed&lt;/th&gt;&#xA;&lt;th&gt;Sample7_Mixed&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;YL09980&lt;/td&gt;&#xA;&lt;td&gt;1240,016&lt;/td&gt;&#xA;&lt;td&gt;759,683&lt;/td&gt;&#xA;&lt;td&gt;985,322&lt;/td&gt;&#xA;&lt;td&gt;1025,855&lt;/td&gt;&#xA;&lt;td&gt;5002,654&lt;/td&gt;&#xA;&lt;td&gt;4200,963&lt;/td&gt;&#xA;&lt;td&gt;4002,855&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;YLW9980&lt;/td&gt;&#xA;&lt;td&gt;589,456&lt;/td&gt;&#xA;&lt;td&gt;452,236&lt;/td&gt;&#xA;&lt;td&gt;630,228&lt;/td&gt;&#xA;&lt;td&gt;523,821&lt;/td&gt;&#xA;&lt;td&gt;2325,556&lt;/td&gt;&#xA;&lt;td&gt;2566,987&lt;/td&gt;&#xA;&lt;td&gt;1800,633&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;So these tables are just the normalized counts of the genes per sample.&lt;/p&gt;&#xA;&lt;p&gt;I was told that the data in the table has to be transposed basically, so that the genes are the variables and the samples are the rows. I am just not sure how to do this in R.&lt;/p&gt;&#xA;&lt;p&gt;I was also told there is a good chance that the noise resulting from the use of different platforms would likely cause these datasets to separate from each other based on that, making this analysis meaningless or not allowing for comparison of gene expression that is conserved across the datasets as a result of mixing the two species during growth.&lt;/p&gt;&#xA;&lt;p&gt;However, I just need to do this analysis and find that out as soon as possible, as this is a bottleneck to my progress at this point. Please any assistance would be greatly appreciated. I am currently teaching myself more about &amp;quot;tidyverse&amp;quot; in an effort to try and do this, but I am on a deadline with this.&lt;/p&gt;&#xA;&lt;p&gt;Edit: So I did the PCA plot using the following function:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data_total.pca &amp;lt;- prcomp(data_total[,c(2:32)], center = TRUE, scale. = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and this was the output I got:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Importance of components:&#xA;                          PC1    PC2    PC3    PC4     PC5     PC6     PC7     PC8     PC9    PC10    PC11    PC12    PC13    PC14&#xA;Standard deviation     4.2180 2.5400 1.9188 1.2866 0.65393 0.47799 0.40826 0.38879 0.35585 0.34915 0.27036 0.18549 0.14335 0.11830&#xA;Proportion of Variance 0.5739 0.2081 0.1188 0.0534 0.01379 0.00737 0.00538 0.00488 0.00408 0.00393 0.00236 0.00111 0.00066 0.00045&#xA;Cumulative Proportion  0.5739 0.7820 0.9008 0.9542 0.96800 0.97537 0.98074 0.98562 0.98971 0.99364 0.99600 0.99711 0.99777 0.99822&#xA;                          PC15    PC16    PC17    PC18    PC19    PC20    PC21    PC22    PC23    PC24    PC25    PC26    PC27    PC28&#xA;Standard deviation     0.10153 0.09239 0.08407 0.07765 0.07536 0.05978 0.04883 0.04556 0.04406 0.03960 0.03764 0.03378 0.03167 0.02897&#xA;Proportion of Variance 0.00033 0.00028 0.00023 0.00019 0.00018 0.00012 0.00008 0.00007 0.00006 0.00005 0.00005 0.00004 0.00003 0.00003&#xA;Cumulative Proportion  0.99855 0.99883 0.99906 0.99925 0.99943 0.99955 0.99963 0.99969 0.99976 0.99981 0.99985 0.99989 0.99992 0.99995&#xA;                          PC29    PC30    PC31&#xA;Standard deviation     0.02695 0.02126 0.02085&#xA;Proportion of Variance 0.00002 0.00001 0.00001&#xA;Cumulative Proportion  0.99997 0.99999 1.00000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So according to the tutorial I am following a general rule of thumb is that the first 3 PCs should explain at least 70% of the variance. The issue is my PCA plot looks weird all of the points are squished up at the x = 0; y = 0 axes of my PCA plot. So I think I need to transpose the data so that the genes are the columns (variables) and the samples are the rows (objects). So I tried this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data_total_t &amp;lt;- transpose(data_total)&#xA;rownames(data_total_t) &amp;lt;- colnames(data_total)&#xA;colnames(data_total_t) &amp;lt;- rownames(data_total)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And this worked, but now I need to exclude row 1 from the function I used to plot the PCA as the PCA cannot have character (gene name) info in it.&lt;/p&gt;&#xA;&lt;p&gt;Any help on how I can do this please?&lt;/p&gt;&#xA;" OwnerUserId="13885" LastEditorUserId="3694" LastEditDate="2022-11-27T09:13:35.243" LastActivityDate="2022-11-27T09:13:35.243" Title="Help with performing PCA analysis on data from 3 different datasets of normalized counts data for RNA-seq experiment" Tags="|r|rna-seq|deseq2|pca|meta-analysis|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17879" PostTypeId="1" CreationDate="2021-10-18T15:12:25.843" Score="1" ViewCount="468" Body="&lt;p&gt;I'm using &lt;a href=&quot;https://hub.docker.com/r/jupyter/r-notebook&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Jupyter notebook&lt;/a&gt; to work on R... How do I do the equivalent of &lt;code&gt;?someFun&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;When I run that in the notebook, I get:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Warning message in seq_len(head.end.idx):&#xA;“first element used of 'length.out' argument”&#xA;ERROR while rich displaying an object: Error in seq_len(head.end.idx): argument must be coercible to non-negative integer&#xA;&#xA;Traceback:&#xA;1. FUN(X[[i]], ...)&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3&gt;Update&lt;/h3&gt;&#xA;&lt;p&gt;When I execute &lt;code&gt;docker run jupyter/r-notebook&lt;/code&gt; I notice that it &lt;strong&gt;does&lt;/strong&gt; pop up the widget (after throwing the error). Not it isn't a popup window, but a panel at the bottom of the workbook. My notebook is 'Trusted', which I think is significant, but I don't know how to turn it off.&lt;/p&gt;&#xA;&lt;p&gt;When I execute &lt;code&gt;docker run -e JUPYTER_ENABLE_LAB=yes jupyter/r-notebook&lt;/code&gt; I get the error and the widget &lt;strong&gt;does not&lt;/strong&gt; pop up. I'm not sure how to set or inspect the JS trust in that environment. I suspect it's related. I don't see any blocked popups in the browser.&lt;/p&gt;&#xA;&lt;p&gt;Strange that even when it works (the widget opens at the bottom of the browser) I still get an error in the output.&lt;/p&gt;&#xA;" OwnerUserId="8875" LastEditorUserId="8875" LastEditDate="2021-10-21T09:55:58.263" LastActivityDate="2021-11-12T07:43:19.423" Title="R help in Jupyter notebook?" Tags="|r|jupyter|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17880" PostTypeId="2" ParentId="17878" CreationDate="2021-10-18T18:00:01.200" Score="1" Body="&lt;p&gt;I think you are making this harder than you have to.  Why can't you follow the code used in the vignette for making PCA plots?  It doesn't even involve getting all the way to the DE genes.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2021-10-18T18:00:01.200" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17881" PostTypeId="2" ParentId="578" CreationDate="2021-10-18T19:18:05.657" Score="0" Body="&lt;p&gt;Here is a code on github that can generate a csv file as well as a python dictionary.&#xA;&lt;a href=&quot;https://github.com/rahulnutron/scientific_name_to_common-name_converter&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/rahulnutron/scientific_name_to_common-name_converter&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14374" LastActivityDate="2021-10-18T19:18:05.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17882" PostTypeId="1" CreationDate="2021-10-18T22:26:59.777" Score="1" ViewCount="39" Body="&lt;p&gt;I am prototyping with data visualization of DGE results, and I would like to work on the analysis pipeline before the real data is available. Currently, I only have 3 samples for  wild type and 1 sample for a mutant. I would like to simulate 2 extras maples of the mutant with the same variance as the wild type sample. How can I simulate data to make up for the absence of replicates in the comparison group?&lt;/p&gt;&#xA;&lt;p&gt;My idea was to use pandas and SciPy to import the count tables, use these tools to simulate the replicates and then run DESeq2 on it. Still, I am not sure which functions I should look for once I am not so proficient with SciPy and pandas yet.&lt;/p&gt;&#xA;" OwnerUserId="12644" LastActivityDate="2021-10-18T22:26:59.777" Title="How to simulate replicates for DGE analysis?" Tags="|rna-seq|deseq2|transcriptome|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17883" PostTypeId="1" CreationDate="2021-10-19T05:57:55.147" Score="1" ViewCount="524" Body="&lt;p&gt;Following the &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/getting_started/installation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;docs&lt;/a&gt; just won't work for me.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -n base -c conda-forge mamba&#xA;conda activate base&#xA;mamba create -c conda-forge -c bioconda -n snakemake snakemake&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the Dockerfile I use:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FROM ubuntu:20.04&#xA;ENV PATH /opt/conda/bin:${PATH}&#xA;ENV LANG C.UTF-8&#xA;ENV SHELL /bin/bash&#xA;COPY aws/docker/Miniconda3-py38_4.10.3-Linux-x86_64.sh /tmp/&#xA;RUN bash /tmp/Miniconda3-py38_4.10.3-Linux-x86_64.sh -b -p /opt/conda&#xA;RUN conda install -y -c conda-forge mamba&#xA;RUN mamba create -q -y -c conda-forge -c bioconda -n snakemake snakemake snakemake-minimal --only-deps&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The error is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;WARNING Extracted package cache '/opt/conda/pkgs/_openmp_mutex-4.5-1_gnu' has invalid size&#xA;WARNING Extracted package cache '/opt/conda/pkgs/_openmp_mutex-4.5-1_gnu' has invalid SHA-256 checksum&#xA;WARNING Package tarball '/opt/conda/pkgs/_openmp_mutex-4.5-1_gnu.tar.bz2' is invalid&#xA;  Package                              Version  Build                Channel                     Size&#xA;───────────────────────────────────────────────────────────────────────────────────────────────────────&#xA;  Install:&#xA;───────────────────────────────────────────────────────────────────────────────────────────────────────&#xA;&#xA;  ...Package list omitted&#xA;&#xA;  Ignored:&#xA;───────────────────────────────────────────────────────────────────────────────────────────────────────&#xA;&#xA;  = snakemake                            6.9.1  hdfd78af_0           bioconda/noarch          Ignored&#xA;  = snakemake-minimal                    6.9.1  pyhdfd78af_0         bioconda/noarch          Ignored&#xA;&#xA;  Summary:&#xA;&#xA;  Install: 215 packages&#xA;  Ignored: 2 packages&#xA;&#xA;  Total download: 256 MB&#xA;&#xA;───────────────────────────────────────────────────────────────────────────────────────────────────────&#xA;&#xA;ERROR   File not valid: SHA256 sum doesn't match expectation &amp;quot;/opt/conda/pkgs/gettext-0.19.8.1-h73d1719_1008.tar.bz2&amp;quot;&#xA;        Expected: 1bb53c99b4943d210c881aad9158fb0235b348498bad1a7076d1f2bef6671922&#xA;        Actual: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855&#xA;&#xA;ERROR   File not valid: SHA256 sum doesn't match expectation &amp;quot;/opt/conda/pkgs/gtk2-2.24.33-h539f30e_1.tar.bz2&amp;quot;&#xA;        Expected: 2a0aef09a947e3768a426b0ea414ddd1e015c9804a46de619a5e67a689ea1dec&#xA;        Actual: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855&#xA;&#xA;ERROR   File not valid: SHA256 sum doesn't match expectation &amp;quot;/opt/conda/pkgs/botocore-1.21.64-pyhd8ed1ab_0.tar.bz2&amp;quot;&#xA;        Expected: ee5501c74eefd0396f141593933ef2cc3419ad21624ed140bbc1e04944af6e31&#xA;        Actual: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855&#xA;&#xA;ERROR   File not valid: SHA256 sum doesn't match expectation &amp;quot;/opt/conda/pkgs/libopenblas-0.3.18-pthreads_h8fe5266_0.tar.bz2&amp;quot;&#xA;        Expected: 6d67f0f7bda1503af83659ca45d8c8ba34d94091fa4cd5b1ecd85332c14db3cc&#xA;        Actual: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855&#xA;&#xA;ERROR   File not valid: SHA256 sum doesn't match expectation &amp;quot;/opt/conda/pkgs/scipy-1.7.1-py39hee8e79c_0.tar.bz2&amp;quot;&#xA;        Expected: 536500b79a5ccc91f6059fcaa6d7e959bddb37a0ed7a930797375c1ed51a6af1&#xA;        Actual: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855&#xA;&#xA;&#xA;# &amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt; ERROR REPORT &amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&#xA;&#xA;    Traceback (most recent call last):&#xA;      File &amp;quot;/opt/conda/lib/python3.8/site-packages/conda/exceptions.py&amp;quot;, line 1079, in __call__&#xA;        return func(*args, **kwargs)&#xA;      File &amp;quot;/opt/conda/lib/python3.8/site-packages/mamba/mamba.py&amp;quot;, line 926, in exception_converter&#xA;        raise e&#xA;      File &amp;quot;/opt/conda/lib/python3.8/site-packages/mamba/mamba.py&amp;quot;, line 920, in exception_converter&#xA;        exit_code = _wrapped_main(*args, **kwargs)&#xA;      File &amp;quot;/opt/conda/lib/python3.8/site-packages/mamba/mamba.py&amp;quot;, line 878, in _wrapped_main&#xA;        result = do_call(args, p)&#xA;      File &amp;quot;/opt/conda/lib/python3.8/site-packages/mamba/mamba.py&amp;quot;, line 757, in do_call&#xA;        exit_code = create(args, parser)&#xA;      File &amp;quot;/opt/conda/lib/python3.8/site-packages/mamba/mamba.py&amp;quot;, line 660, in create&#xA;        return install(args, parser, &amp;quot;create&amp;quot;)&#xA;      File &amp;quot;/opt/conda/lib/python3.8/site-packages/mamba/mamba.py&amp;quot;, line 609, in install&#xA;        downloaded = transaction.prompt(repos)&#xA;    RuntimeError: Found incorrect download: libopenblas. Aborting&#xA;&#xA;`$ /opt/conda/bin/mamba create -q -y -c conda-forge -c bioconda -n snakemake snakemake snakemake-minimal --only-deps`&#xA;&#xA;  environment variables:&#xA;                 CIO_TEST=&amp;lt;not set&amp;gt;&#xA;               CONDA_ROOT=/opt/conda&#xA;           CURL_CA_BUNDLE=&amp;lt;not set&amp;gt;&#xA;                     PATH=/opt/conda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin&#xA;                          :/bin&#xA;       REQUESTS_CA_BUNDLE=&amp;lt;not set&amp;gt;&#xA;            SSL_CERT_FILE=&amp;lt;not set&amp;gt;&#xA;&#xA;     active environment : None&#xA;       user config file : /root/.condarc&#xA; populated config files :&#xA;          conda version : 4.10.3&#xA;    conda-build version : not installed&#xA;         python version : 3.8.10.final.0&#xA;       virtual packages : __linux=5.10.16.3=0&#xA;                          __glibc=2.31=0&#xA;                          __unix=0=0&#xA;                          __archspec=1=x86_64&#xA;       base environment : /opt/conda  (writable)&#xA;      conda av data dir : /opt/conda/etc/conda&#xA;  conda av metadata url : None&#xA;           channel URLs : https://conda.anaconda.org/conda-forge/linux-64&#xA;                          https://conda.anaconda.org/conda-forge/noarch&#xA;                          https://conda.anaconda.org/bioconda/linux-64&#xA;                          https://conda.anaconda.org/bioconda/noarch&#xA;                          https://repo.anaconda.com/pkgs/main/linux-64&#xA;                          https://repo.anaconda.com/pkgs/main/noarch&#xA;                          https://repo.anaconda.com/pkgs/r/linux-64&#xA;                          https://repo.anaconda.com/pkgs/r/noarch&#xA;          package cache : /opt/conda/pkgs&#xA;                          /root/.conda/pkgs&#xA;       envs directories : /opt/conda/envs&#xA;                          /root/.conda/envs&#xA;               platform : linux-64&#xA;             user-agent : conda/4.10.3 requests/2.25.1 CPython/3.8.10 Linux/5.10.16.3-microsoft-standard-WSL2 ubuntu/20.04.3 glibc/2.31&#xA;                UID:GID : 0:0&#xA;             netrc file : None&#xA;           offline mode : False&#xA;&#xA;&#xA;An unexpected error has occurred. Conda has prepared the above report.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="13937" LastEditorUserId="3848" LastEditDate="2022-03-18T13:50:50.397" LastActivityDate="2022-03-18T14:53:45.460" Title="Installing snakemake following official docs just won't work. Help needed" Tags="|snakemake|docker|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="17884" PostTypeId="1" CreationDate="2021-10-19T07:27:15.580" Score="-1" ViewCount="44" Body="&lt;p&gt;Can anyone explain what is cheminformatics and how does it help in the future?&lt;/p&gt;&#xA;&lt;p&gt;I need to write an article that should be understandable for 14-20 yr age group. Please help with references, I don't know anything.&lt;/p&gt;&#xA;" OwnerUserId="14293" LastEditorUserId="57" LastEditDate="2021-10-24T18:59:27.277" LastActivityDate="2021-10-24T18:59:27.277" Title="general cheminformatics references" Tags="|computational-biochemistry|cheminformatics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17885" PostTypeId="2" ParentId="17884" CreationDate="2021-10-19T08:41:47.017" Score="3" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;The definition of cheminformatics has been covered in &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/17810/difference-between-bioinformatics-and-cheminformatics/17811#17811&quot;&gt;Difference between bioinformatics and cheminformatics&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For a reference, simply citing &lt;a href=&quot;https://www.amazon.co.uk/Essentials-Computational-Chemistry-Christopher-Cramer/dp/0470091827&quot; rel=&quot;nofollow noreferrer&quot;&gt;Essentials of Computational Chemistry by C. Cramer&lt;/a&gt; will do it. I do not know if the newer Clayden (nitrile cover) deals with computational methods, but the old one (water+formaldehyde on cover) did not. In generally, a textbook is a very good way to get familiar with a field and Cramer's book is good.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2021-10-19T08:47:00.993" LastActivityDate="2021-10-19T08:47:00.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17886" PostTypeId="1" CreationDate="2021-10-19T10:14:21.920" Score="1" ViewCount="29" Body="&lt;p&gt;I am running &lt;code&gt;CIRIquant&lt;/code&gt; on Ubuntu 16.04 LTS following the instruction which indicated on &lt;a href=&quot;https://ciriquant-cookbook.readthedocs.io/en/latest/de.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ciriquant-cookbook.readthedocs.io/en/latest/de.html&lt;/a&gt;. I got .GTF file that contains detailed information of BSJ and FSJ reads of circRNAs and annotation of circRNA back-spliced regions.&#xA;In the following, I added a record of Circular RNA detected in .GTF file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Chrom: chr1 &#xA;&#xA;Type: circRNA&#xA;    &#xA;start: 1289228&#xA;    &#xA;End:  1289448&#xA;    &#xA;score:0.02  &#xA;&#xA;Strand Information: -&#xA;    &#xA;circ_id: chr1:1289228|1289448   &#xA;&#xA;circ_type   :exon&#xA;    &#xA;bsj: 1000&#xA;&#xA;fsj: 515.000&#xA;&#xA;junc_ratio: 0.004&#xA;    &#xA;gene_id: ENSG00000162576.12&#xA;&#xA;gene_name: MXRA8&#xA;&#xA;gene_type: protein_coding&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I need to know which column of .GTF file indicates the expression count of circular detected?&lt;/p&gt;&#xA;&lt;p&gt;I appreciate it if you share your comment with me.&lt;/p&gt;&#xA;&lt;p&gt;Best Regards,&lt;/p&gt;&#xA;" OwnerUserId="4915" LastEditorUserId="73" LastEditDate="2021-10-26T22:32:06.610" LastActivityDate="2021-10-26T22:32:06.610" Title="In CIRIquant output, which column shows the expression level of circular?" Tags="|python|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17887" PostTypeId="2" ParentId="17879" CreationDate="2021-10-19T10:28:00.120" Score="0" Body="&lt;p&gt;I think error is not related to &lt;code&gt;?&lt;/code&gt;. This is what I see when I try &lt;code&gt;?seq&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/D5Tc8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D5Tc8.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="131" LastActivityDate="2021-10-19T10:28:00.120" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17888" PostTypeId="1" CreationDate="2021-10-19T11:35:40.683" Score="0" ViewCount="13083" Body="&lt;p&gt;I've just dowloaded entrez direct on my Mac and I was trying to perform some simple queries like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db pubmed -query &amp;quot;tn3 transposition immunity&amp;quot; | efetch -format medline&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but I keep getting this error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;curl: (22) The requested URL returned error: 400 Bad Request&#xA; ERROR:  curl command failed ( Mar 19 Ott 2021 13:02:26 CEST ) with: 22 &#xA;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi -d db=pubmed&amp;amp;id=Unable%2Cto%2Clocate%2Cxtract%2Cexecutable.%2CPlease%2Cexecute%2Cthe%2Cfollowing%2Cnquire%2Cdwn%2Cftp.ncbi.nlm.nih.gov%2Centrez%2Centrezdirect%2Cxtract.Darwin.gz%2Cgunzip%2Cf%2Cxtract.Darwin.gz%2Cchmod%2Cx%2Cxtract.Darwin&amp;amp;rettype=medline&amp;amp;retmode=text&amp;amp;tool=edirect&amp;amp;edirect=16.1&amp;amp;edirect_os=Darwin&amp;amp;email=mariocatalano%40MBP-di-Mario.lan&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm using bash scripts on my Mac's terminal&lt;/p&gt;&#xA;&lt;p&gt;How should I fix this?&lt;/p&gt;&#xA;" OwnerUserId="14388" LastEditorUserId="298" LastEditDate="2021-10-19T12:28:04.723" LastActivityDate="2021-10-19T19:02:12.003" Title="curl: (22) The requested URL returned error: 400 Bad Request when using entrez direct" Tags="|bash|entrez|curl|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17889" PostTypeId="2" ParentId="17879" CreationDate="2021-10-19T11:40:11.460" Score="3" Body="&lt;p&gt;If you launch like &lt;code&gt;docker run -p 8888:8888 jupyter/r-notebook&lt;/code&gt; you will get the help page like a widget and a warning in your cell output. Check your browser, maybe is blocking pop-up elements.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KRqZI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KRqZI.png&quot; alt=&quot;r-notebook&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2021-10-19T11:40:11.460" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17890" PostTypeId="2" ParentId="17883" CreationDate="2021-10-19T11:46:41.673" Score="1" Body="&lt;p&gt;I select the Miniconda installer directly from the official repository:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FROM ubuntu:20.04&#xA;ENV LANG C.UTF-8&#xA;SHELL [&amp;quot;/bin/bash&amp;quot;, &amp;quot;-c&amp;quot;]&#xA;RUN apt update -qq &amp;amp;&amp;amp; apt -qq -y install curl ca-certificates --no-install-recommends&#xA;RUN ( cd /tmp/; curl -LO &amp;quot;http://repo.anaconda.com/miniconda/Miniconda3-py38_4.10.3-Linux-x86_64.sh&amp;quot;)&#xA;RUN bash /tmp/Miniconda3-py38_4.10.3-Linux-x86_64.sh -b -p /opt/conda&#xA;ENV PATH=/opt/conda/bin:&lt;span class=&quot;math-container&quot;&gt;${PATH}&#xA;RUN conda install -y -c conda-forge mamba&#xA;RUN mamba create -q -y -c conda-forge -c bioconda -n snakemake snakemake snakemake-minimal --only-deps&#xA;RUN echo &quot;source activate snakemake&quot; &amp;gt; ~/.bashrc&#xA;ENV PATH /opt/conda/envs/snakemake/bin:$&lt;/span&gt;{PATH}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2021-10-19T11:46:41.673" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17891" PostTypeId="2" ParentId="16302" CreationDate="2021-10-19T15:31:27.360" Score="2" Body="&lt;p&gt;In the end I manage to do 2 cohorts&#xA;1: Long Covid Patients (1300N)&#xA;2: NON Long Covid Patients that had Covid (1300).&lt;/p&gt;&#xA;&lt;p&gt;It was the most rational thing to do, and professor liked it. So that's how I manage in th end.&lt;/p&gt;&#xA;" OwnerUserId="12682" LastActivityDate="2021-10-19T15:31:27.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17892" PostTypeId="1" AcceptedAnswerId="17894" CreationDate="2021-10-19T17:49:43.560" Score="0" ViewCount="63" Body="&lt;p&gt;Suppose, I have a protein (say, 16PK).&lt;/p&gt;&#xA;&lt;p&gt;and, I want to calculate&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the distance between one of its alpha-carbons and the nearest beta-sheet.&lt;/li&gt;&#xA;&lt;li&gt;the distance between two strands in a beta-sheet (i, j).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;How should I proceed to achieve this?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastActivityDate="2021-10-20T09:27:26.560" Title="How can I calculate the distance w.r.t. a beta sheet?" Tags="|python|proteins|protein-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17893" PostTypeId="1" AcceptedAnswerId="17903" CreationDate="2021-10-19T17:49:51.307" Score="2" ViewCount="204" Body="&lt;p&gt;I'm using cutadapt 3.5 to trim adapters and perform some filtering on paired-end data.&#xA;Both R1 and R2 sequences have 3' adapters that &lt;em&gt;might&lt;/em&gt; be found depending on the sequence length, but R1 also has an anchored 5' adapter that I want to require for the pair to be kept in the output.  For example, if that required R1 5' adapter is AAAAAAAA, the R1 3' adapter is TTTTTTTT, and the R2 3' adapter is GGGGGGGG, and I have this input:&lt;/p&gt;&#xA;&lt;p&gt;R1:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@read1 should be kept&#xA;AAAAAAAACGTCCTGGGATTTGTAATAATATTTTAGTTCTGAGCGACAAGTAAGGGATAATTTTTTTT&#xA;+&#xA;FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF&#xA;@read2 should be filtered&#xA;TACCAACTACATTTAGCTTCAGGCTAGTGATGCCCGCCGTCGGCACACTGGACACATGGCTTTTTTTT&#xA;+&#xA;FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF&#xA;@read3 should be kept&#xA;AAAAAAAACAATTTTTACTCTAGAAATGTCTGTGCTCATCACCCCGACACCGAATAGCTATGACTCAC&#xA;+&#xA;FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and R2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@read1&#xA;AGGCTCACCGCCACTGTTGTACCTTCCTATCGCCACTCTAAGATCTATGTAACCTCTCCCGGGGGGGG&#xA;+&#xA;FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF&#xA;@read2&#xA;TGAATAATACTGCGGTTTAATCGATATATTCGGGATTATGCAAGACACCCTACGTACTTAGGGGGGGG&#xA;+&#xA;FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF&#xA;@read3&#xA;TACTGGAGGGTACAAGCCCGGACTATCCACGGCGTCAGCTGGCTTAGCATTGAAGGTCGGCGGTGTGT&#xA;+&#xA;FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm aiming to get this R1 out:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@read1 should be kept&#xA;CGTCCTGGGATTTGTAATAATATTTTAGTTCTGAGCGACAAGTAAGGGATAA&#xA;+&#xA;FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF&#xA;@read3 should be kept&#xA;CAATTTTTACTCTAGAAATGTCTGTGCTCATCACCCCGACACCGAATAGCTATGACTCAC&#xA;+&#xA;FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And this R2 out:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@read1&#xA;AGGCTCACCGCCACTGTTGTACCTTCCTATCGCCACTCTAAGATCTATGTAACCTCTCCC&#xA;+&#xA;FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF&#xA;@read3&#xA;TACTGGAGGGTACAAGCCCGGACTATCCACGGCGTCAGCTGGCTTAGCATTGAAGGTCGGCGGTGTGT&#xA;+&#xA;FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This approach doesn't work as it discards read3:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cutadapt -a '^AAAAAAAA...TTTTTTTT' -A 'GGGGGGGG' example.R{1,2}.fastq --discard-untrimmed -o out.R1.fastq -p out.R2.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I expect that's because the required/optional behavior only applies to the linked adapter for R1, and R2 is considered untrimmed for read3, which then filters the whole pair because of the &lt;a href=&quot;https://cutadapt.readthedocs.io/en/stable/guide.html#filtering-paired-end-reads&quot; rel=&quot;nofollow noreferrer&quot;&gt;default&lt;/a&gt; &lt;code&gt;--pair-filter=any&lt;/code&gt; behavior (if &lt;em&gt;either&lt;/em&gt; read is untrimmed, the pair is discarded).  I found I can get the filtering I want with &lt;code&gt;--pair-filter first&lt;/code&gt;, but then the other filtering criteria like quality and length will only apply to R1, too.&lt;/p&gt;&#xA;&lt;p&gt;My question is, can I get the trimmed/untrimmed behavior I'm looking for (read1 and read3 trimmed, read2 untrimmed) while also applying cutadapt's filtering to both reads in each pair? In other words, can I specify to cutadapt that R1's anchored adapter is required and the other two optional, without resorting to changing the pair filter setting?&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2021-10-20T17:33:08.843" Title="When using cutadapt, can I specify an R2 adapter as optional when I have a required R1 adapter?" Tags="|ngs|fastq|trimming|cutadapt|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17894" PostTypeId="2" ParentId="17892" CreationDate="2021-10-19T18:43:08.127" Score="1" Body="&lt;p&gt;The documentation for the PDB module in biopython, which you have tagged, is very clear about how distances are measured between two given Cα atoms.&lt;/p&gt;&#xA;&lt;p&gt;To find out the secondary structure and interactions between strands of a sheet, you may want to look at the &lt;strong&gt;header&lt;/strong&gt; of the PDB. This is the stuff before the &lt;code&gt;ATOM&lt;/code&gt;/&lt;code&gt;HETATM&lt;/code&gt; entries. &lt;strong&gt;&lt;code&gt;SHEET&lt;/code&gt; entries&lt;/strong&gt; are what you are after. Please read the &lt;a href=&quot;https://www.wwpdb.org/documentation/file-format-content/format33/sect5.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB format specifications for secondary structure&lt;/a&gt;. Biopython PDB does not parse headers well, but reading the file and finding the SHEET entry and positionally splicing out the strings of interest from the entry is beyond straightforward.&lt;/p&gt;&#xA;&lt;p&gt;Example take from the above link:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;         1         2         3         4         5         6         7         8&#xA;12345678901234567890123456789012345678901234567890123456789012345678901234567890&#xA;SHEET    1   A 5 THR A 107  ARG A 110  0&#xA;SHEET    2   A 5 ILE A  96  THR A  99 -1  N  LYS A  98   O  THR A 107&#xA;SHEET    3   A 5 ARG A  87  SER A  91 -1  N  LEU A  89   O  TYR A  97&#xA;SHEET    4   A 5 TRP A  71  ASP A  75 -1  N  ALA A  74   O  ILE A  88&#xA;SHEET    5   A 5 GLY A  52  PHE A  56 -1  N  PHE A  56   O  TRP A  71&#xA;SHEET    1   B 5 THR B 107  ARG B 110  0&#xA;SHEET    2   B 5 ILE B  96  THR B  99 -1  N  LYS B  98   O  THR B 107&#xA;SHEET    3   B 5 ARG B  87  SER B  91 -1  N  LEU B  89   O  TYR B  97&#xA;SHEET    4   B 5 TRP B  71  ASP B  75 -1  N  ALA B  74   O  ILE B  88&#xA;SHEET    5   B 5 GLY B  52  ILE B  55 -1  N  ASP B  54   O  GLU B  73&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2021-10-20T09:27:26.560" LastActivityDate="2021-10-20T09:27:26.560" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17895" PostTypeId="2" ParentId="17888" CreationDate="2021-10-19T19:02:12.003" Score="2" Body="&lt;p&gt;It's not a curl problem.  (The &lt;a href=&quot;https://curl.se/libcurl/c/libcurl-errors.html#CURLEHTTPRETURNEDERROR&quot; rel=&quot;nofollow noreferrer&quot;&gt;22 error&lt;/a&gt; is just passing along the server-side HTTP 400 error, which is because it dumped an error message into a URL.)  I think terdon is right and there's something wrong with your edirect installation.  You have esearch and efetch on your PATH but apparently not xtract and/or xtract.Darwin.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;More detail:&lt;/p&gt;&#xA;&lt;p&gt;Look toward the end of the error message, at the text being passed to curl's &lt;code&gt;-d&lt;/code&gt; argument:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;db=pubmed&amp;amp;id=Unable%2Cto%2Clocate%2Cxtract%2Cexecutable.%2CPlease%2Cexecute%2Cthe%2Cfollowing%2Cnquire%2Cdwn%2Cftp.ncbi.nlm.nih.gov%2Centrez%2Centrezdirect%2Cxtract.Darwin.gz%2Cgunzip%2Cf%2Cxtract.Darwin.gz%2Cchmod%2Cx%2Cxtract.Darwin&amp;amp;rettype=medline&amp;amp;retmode=text&amp;amp;tool=edirect&amp;amp;edirect=16.1&amp;amp;edirect_os=Darwin&amp;amp;email=mariocatalano%40MBP-di-Mario.lan&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It's putting this in the &lt;code&gt;id=&lt;/code&gt; part:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Unable%2Cto%2Clocate%2Cxtract%2Cexecutable.%2CPlease%2Cexecute%2Cthe%2Cfollowing%2Cnquire%2Cdwn%2Cftp.ncbi.nlm.nih.gov%2Centrez%2Centrezdirect%2Cxtract.Darwin.gz%2Cgunzip%2Cf%2Cxtract.Darwin.gz%2Cchmod%2Cx%2Cxtract.Darwin&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which is maybe something like (trying to guess which characters got turned into &lt;code&gt;%2C&lt;/code&gt;):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Unable to locate xtract executable. Please execute the following: nquire -dwn ftp.ncbi.nlm.nih.gov entrez/entrezdirect xtract.Darwin.gz; gunzip -f xtract.Darwin.gz; chmod +x xtract.Darwin&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But you should already have xtract and xtract.Darwin from the same edirect install that gave you esearch and efetch.&lt;/p&gt;&#xA;&lt;p&gt;For what it's worth the official Entrez Direct install on Mac OS looks a little goofy.  When I try it myself it ends with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Entrez Direct has been successfully downloaded and installed.&#xA;&#xA;-e In order to complete the configuration process, please execute the following:&#xA;&#xA;  echo &amp;quot;export PATH=\\\${PATH}:&amp;quot; &amp;gt;&amp;gt; &lt;span class=&quot;math-container&quot;&gt;$HOME/.bashrc&#xA;  echo &quot;export PATH=\\\${PATH}:&quot; &amp;gt;&amp;gt; $&lt;/span&gt;HOME/.zshrc&#xA;&#xA;or manually edit the PATH variable assignments in your .bashrc and .zshrc files.&#xA;&#xA;Would you like to do that automatically now? [y/N]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Those commands don't look right (what's getting added to the path? an empty string?) and if I say y, I get this at the end of .bashrc and .zshrc which definitely isn't right:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;export PATH=${PATH}:&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It seems like an empty string at the end of PATH behaves like &lt;code&gt;.&lt;/code&gt; (current directory), so I can go into ~/edirect and run your command and it &lt;em&gt;does&lt;/em&gt; actually work.  So this isn't exactly your problem... but still, that seems wrong.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2021-10-19T19:02:12.003" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="17896" PostTypeId="1" CreationDate="2021-10-20T09:15:38.533" Score="0" ViewCount="761" Body="&lt;p&gt;Suppose I have a list of genes and I want to find their genomic coordinates. To be specific and to keep things consistent with the rest of my project, I'm using hg38 on the website genome.ucsc.edu. So far, I've been typing one gene symbol at a time into the gene browser and clicking the correct match. This sends me to the gene's page, which shows me the start and end genome coordinates for the gene's transcript and coding region. That's what I'm looking for.&lt;/p&gt;&#xA;&lt;p&gt;This is a tedious process when I have hundreds of genes to go through, so I was wondering if I can input a list and get out a list instead. I know about the page &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTables&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genome.ucsc.edu/cgi-bin/hgTables&lt;/a&gt;. As an example, suppose I click &amp;quot;paste list&amp;quot; and just type the following two lines:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CASZ1&#xA;MTHFR&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here's a screenshot of the output (click to see full sized image):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ibb.co/MZthX6N&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.ibb.co/4KXfQdN/temp.jpg&quot; alt=&quot;&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I think this page is generally intended to go the opposite direction (genomic coordinates -&amp;gt; genes)?&lt;/p&gt;&#xA;&lt;p&gt;Edit: Ok, here's an example of what I want. I'd like to just enter the gene symbols and get the transcript and coding regions shown.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CASZ1, Transcript chr1:10,636,604-10,796,646, Coding Region chr1:10,638,942-10,693,889&#xA;MTHFR, Transcript chr1:11,785,723-11,805,964, Coding Region chr1:11,790,680-11,803,116&#xA;ST7L, Transcript chr1:112,525,883-112,618,928, Coding Region chr1:112,526,013-112,598,043&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I obtained these by searching through genome.ucsc.edu one by one manually.&lt;/p&gt;&#xA;" OwnerUserId="12044" LastEditorUserId="650" LastEditDate="2021-10-25T19:40:24.473" LastActivityDate="2021-11-24T20:01:25.130" Title="How to Find List of Genomic Coordinates from List of Genes using UCSC Website?" Tags="|genome|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="17897" PostTypeId="2" ParentId="3943" CreationDate="2021-10-20T10:37:24.413" Score="0" Body="&lt;p&gt;Since both Seurat and Scanpy are evolving rapidly, the safest way is to&#xA;&lt;a href=&quot;https://github.com/brianpenghe/Seurat3-plus/blob/master/Seurat_to_MM10X.R&quot; rel=&quot;nofollow noreferrer&quot;&gt;save your Seurat object into MatrixMarket format + metadata csv&lt;/a&gt;, and&#xA;&lt;a href=&quot;https://scanpy.readthedocs.io/en/stable/generated/scanpy.read_10x_mtx.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;import them into Scanpy&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2021-10-20T10:37:24.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17898" PostTypeId="2" ParentId="11562" CreationDate="2021-10-20T14:47:36.877" Score="2" Body="&lt;p&gt;A very basic solution in python:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio import SeqIO&#xA;seq_dict = {i : rec.seq for i,rec in enumerate(SeqIO.parse(&amp;quot;your_msa.aln&amp;quot;, &amp;quot;fasta&amp;quot;))}&#xA;for pos in range(len(seq_dict[0])):&#xA;    pos_bases = ([seqio[i] for seqio in [*seq_dict.values()]])&#xA;    # If position contains more than one unique base&#xA;    if len([*set(pos_bases)])&amp;gt;1:&#xA;        print(pos, pos_bases)        # prints the position and alternative bases there&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you need something more useful, just leave a comment here and I'll develop this further.&lt;/p&gt;&#xA;" OwnerUserId="2459" LastEditorUserId="96" LastEditDate="2021-10-20T18:03:39.557" LastActivityDate="2021-10-20T18:03:39.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17901" PostTypeId="1" AcceptedAnswerId="17902" CreationDate="2021-10-20T16:39:34.700" Score="1" ViewCount="25" Body="&lt;p&gt;I am developing software to store experiments or studies using SBOL3. In the regular synthetic biology workflow we will find the experimental components Strain and Media. A safe way to implement them is using &lt;code&gt;tyto.SBO.functional_entity&lt;/code&gt; as type for both.&lt;/p&gt;&#xA;&lt;p&gt;Other attempts to implement media (&lt;a href=&quot;https://github.com/Bioprotocols/paml/blob/main/examples/growth_curve.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Bioprotocols/paml/blob/main/examples/growth_curve.py&lt;/a&gt;)looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SC_media = sbol3.Component('SC_Media', 'TBD', name='Synthetic Complete Media')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Therefore there is a need for standardise these types on SBOL3 to provide more accurate representations. The Experimental Factor Ontology (EFO) have ontology terms for Strain &lt;a href=&quot;http://www.ebi.ac.uk/efo/EFO_0005135&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ebi.ac.uk/efo/EFO_0005135&lt;/a&gt; and Media &lt;a href=&quot;http://www.ebi.ac.uk/efo/EFO_0000579&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.ebi.ac.uk/efo/EFO_0000579&lt;/a&gt; that can be used as type. Then draft SBOL components looks as follow:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;strain = sbol3.Component(fj_strain.name(), http://www.ebi.ac.uk/efo/EFO_0005135)&#xA;strain.name =fj_strain.name&#xA;strain.description = fj_strain.description()&#xA;&#xA;&#xA;media = sbol3.Component(fj_media.name(), http://www.ebi.ac.uk/efo/EFO_0000579)&#xA;media.name =fj_media.name&#xA;media.description = fj_media.description()&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The problem with this is the lack of readability on the code given the obscure ontology URIs. This problem has been solved for other ontologies using tyto, but EFO is not supported out-of-the-box and the configuration of new ontologies is not working. On tyto repository there is an example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from tyto import EBIOntologyLookupService, Ontology&#xA;KISAO = Ontology(uri='http://www.biomodels.net/kisao/KISAO_FULL#', endpoints=[EBIOntologyLookupService])&#xA;KISAO.Gillespie_direct_algorithm&#xA;'http://www.biomodels.net/kisao/KISAO#KISAO_0000029'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I copied and runed that code but it raise this error:&#xA;[ERROR] tyto.py 70: Ontology &lt;a href=&quot;http://www.biomodels.net/kisao/KISAO_FULL#&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.biomodels.net/kisao/KISAO_FULL#&lt;/a&gt; is not available at EBI Ontology Lookup Service&lt;/p&gt;&#xA;&lt;p&gt;What I would like to do is something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from tyto import EBIOntologyLookupService, Ontology&#xA;efo = Ontology(uri='http://www.ebi.ac.uk/efo', endpoints=[EBIOntologyLookupService])&#xA;efo.media&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But this raises the same error.&#xA;How can I add EFO to tyto? What is the URI that should be provided? Should I raise an issue in tyto to ask for EFO out-of-the-box support?&lt;/p&gt;&#xA;&lt;p&gt;Do you agree with the ontology terms for Strain and Media? Any feedback?&lt;/p&gt;&#xA;" OwnerUserId="14025" LastActivityDate="2021-10-20T17:00:02.573" Title="Creating Experimental Components on SBOL" Tags="|sbol|ontology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17902" PostTypeId="2" ParentId="17901" CreationDate="2021-10-20T17:00:02.573" Score="0" Body="&lt;p&gt;Section 7.9 of the SBOL3 specification already provides recommended terms for experimental elements.  Among these, Section 7.9.2 specifically addresses strain and media, reading in pat:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The roles of reagents may vary in context: for example, arabinose may serve as an inducer or as a media carbon source. As such, contextual role SHOULD be indicated by an NCI Thesaurus (NCIT) term in a role property of the Feature. Examples include:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;NCIT:C64356: PositiveControl&lt;/li&gt;&#xA;&lt;li&gt;NCIT:C48694: Cell&lt;/li&gt;&#xA;&lt;li&gt;NCIT:C85504: Media&lt;/li&gt;&#xA;&lt;li&gt;NCIT:C14419: Strain&lt;/li&gt;&#xA;&lt;li&gt;NCIT:C120268: Inducer&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;NCIT is one of the ontologies that is built-in for tyto:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tyto.NCIT.Strain&#xA;Out[4]: 'https://identifiers.org/ncit:C95501'&#xA;tyto.NCIT.Media&#xA;Out[5]: 'https://identifiers.org/ncit:C19442'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4869" LastActivityDate="2021-10-20T17:00:02.573" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17903" PostTypeId="2" ParentId="17893" CreationDate="2021-10-20T17:33:08.843" Score="0" Body="&lt;p&gt;This is not possible in a single cutadapt call, but can be done fairly seamlessly with a pipe between two calls and with cutadapt's interleaving feature.  (Thanks to the author for &lt;a href=&quot;https://github.com/marcelm/cutadapt/issues/576&quot; rel=&quot;nofollow noreferrer&quot;&gt;help with this&lt;/a&gt;.)&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Giving the R2 3' adapter to one cutadapt command and interleaving the read pairs to piped output, a second cutadapt command can take that interleaved output and use both of R1's adapters as a linked adapter:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cutadapt -A GGGGGGGG --interleaved example.R{1,2}.fastq | \&#xA;cutadapt -a &amp;quot;^AAAAAAAA...TTTTTTTT&amp;quot; --interleaved --discard-untrimmed -o out.R1.fastq -p out.R2.fastq -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now I can add other filters to the second call.&lt;/p&gt;&#xA;&lt;p&gt;An additional read pair that should be filtered:&lt;/p&gt;&#xA;&lt;p&gt;R1:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@read4 would be kept but R2 is too short&#xA;AAAAAAAACAATTTTTACTCTAGAAATGTCTGTGCTCATCACCCCGACACCGAATAGCTATGACTCAC&#xA;+&#xA;FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;R2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@read4&#xA;CTACTAGACATATACCCTGGGGGGGGTTCCCCGTGAAGACGGTTACGATGTGGACAAGCACTCCAGCG&#xA;+&#xA;FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I can now give a minimum length that applies to both in each pair:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cutadapt -A GGGGGGGG --interleaved example.R{1,2}.fastq | \&#xA;cutadapt -a &amp;quot;^AAAAAAAA...TTTTTTTT&amp;quot; --interleaved --discard-untrimmed --minimum-length 30 -o out.R1.fastq -p out.R2.fastq -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With that I get read1 and read3 in the output, as intended.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2021-10-20T17:33:08.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17905" PostTypeId="1" CreationDate="2021-10-21T09:30:53.157" Score="0" ViewCount="454" Body="&lt;p&gt;My idea is to align whole-genome sequencing data (as fastq files after, 30× coverage, gDNA) to the reference mouse genome (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly/GCF_000001635.27&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI&lt;/a&gt;), extract the immunoglobulin locus and compare it to the reference. I think the question is too broad but maybe a hint how to start ? (I'm new to the field.)&lt;/p&gt;&#xA;&lt;p&gt;Task is: to check for difference between reference genome and a sample for 100-200 genes.&lt;/p&gt;&#xA;&lt;p&gt;I checked these: &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/16303/14406&quot;&gt;question&lt;/a&gt; (but it's closed), &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/11396/14406&quot;&gt;answer&lt;/a&gt; (gave a hint about whole reference mapping not only the locus of interest), &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/10633/14406&quot;&gt;question&lt;/a&gt; (not answered), &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/10633/14406&quot;&gt;question&lt;/a&gt; (useful I guess but not sure).&lt;/p&gt;&#xA;&lt;p&gt;Additional info:&#xA;Instrument Platform: ILLUMINA&lt;br /&gt;&#xA;Instrument Model: HiSeq X Ten&lt;br /&gt;&#xA;Library Layout: PAIRED&lt;br /&gt;&#xA;Library Strategy: WGS&lt;br /&gt;&#xA;Library Source: GENOMIC&lt;/p&gt;&#xA;&lt;p&gt;Typical sequence length distribution from FastQC: 150nt.&lt;/p&gt;&#xA;&lt;p&gt;I have Win10 with &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/13245&quot;&gt;WSL 1&lt;/a&gt; installed.&lt;/p&gt;&#xA;" OwnerUserId="14406" LastEditorUserId="14406" LastEditDate="2021-11-13T16:11:31.487" LastActivityDate="2021-11-21T00:50:50.803" Title="How to compare sequences of genes obtained after whole-genome sequencing to reference genome?" Tags="|sequence-alignment|fastq|reference-genome|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17906" PostTypeId="1" CreationDate="2021-10-21T12:37:54.330" Score="0" ViewCount="89" Body="&lt;p&gt;I have performed a phylogenetic analysis using CLUSTALW and obtained a phylogenetic tree. I know how to read a phylogenetic tree and how to relate the sequence similarity between two sequences by looking at the tree, but what I am concerned about is how one can decipher the father-daughter relationship by looking at the tree? Also, if I want to isolate the sequence of the father corresponding to a daughter sequence how can I do that?&#xA;I have attached the corresponding image. In the image, I know that sequences &amp;quot;QXN18196&amp;quot; and &amp;quot;QXN18436&amp;quot; are closely related. But will it be safe to say that sequence &amp;quot;QXN18520&amp;quot; is the father sequence for both the sequences? Or if I want to establish a father-daughter relationship for the whole tree, what will be the relation between the sequences?&lt;a href=&quot;https://i.stack.imgur.com/4zKeX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4zKeX.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14408" LastActivityDate="2021-10-21T15:03:47.620" Title="How to identify Father-Daughter Relationship from Phylogenetic analysis?" Tags="|phylogenetics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17907" PostTypeId="2" ParentId="17906" CreationDate="2021-10-21T15:03:47.620" Score="1" Body="&lt;p&gt;Unless you have wildly differing mutation rates on different branches, you can't say that QXN18520 has the same sequence as the most recent common ancestor of QXN18196 and QXN18436, because QXN18520 was collected today, not back whenever QXN18196 and QXN18436 shared a common ancestor. All that you can say from this phylogeny is that there probably existed a common ancestor of QXN18196 and QXN18436, marked here by 5e-9[86.8], and that this common ancestor and QXN18520 shared a common ancestor even further back, marked here by 5e-9[0].&lt;/p&gt;&#xA;&lt;p&gt;(General observation: Note also that for a number of reasons, phylogenies often don't reflect reality -- they are simply a maximum likelihood estimate of the actual ancestral pedigree, often under some very unrealistic assumptions, and/or according to an incomplete or incorrect view of reality -- e.g. the samples that you have collected may have a lot of allele matches due to identity by state (IBS) rather than identity by descent (IBD). In other words, let phylogenies inform you, but don't assume they are the absolute truth.)&lt;/p&gt;&#xA;" OwnerUserId="14409" LastActivityDate="2021-10-21T15:03:47.620" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17908" PostTypeId="2" ParentId="17905" CreationDate="2021-10-21T20:39:10.097" Score="1" Body="&lt;p&gt;This is very easy. First map all reads to the reference sequence for the organism. If they are short reads use BWA mem. If long reads use minimap2 or ngmlr. Once mapped view the locus of interest in igv (GUI). If you need to extract the reads at the locus pass the coordinates to samtools view (cli).&lt;/p&gt;&#xA;" OwnerUserId="7153" LastActivityDate="2021-10-21T20:39:10.097" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17909" PostTypeId="1" CreationDate="2021-10-21T23:59:15.513" Score="1" ViewCount="33" Body="&lt;p&gt;I would like to train Augustus, SNAP and GlimmerHMM. I found protein sequences in GenBank and orthodb.org. Furthermore, I found HMM files on busco-data.ezlab.org.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ wget -c https://busco-data.ezlab.org/v5/data/lineages/viridiplantae_odb10.2020-09-10.tar.gz&#xA;$&lt;/span&gt; wget -c https://v100.orthodb.org/download/odb10_plants_fasta.tar.gz&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Are there any instructions on how to train those annotations tools?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance&lt;/p&gt;&#xA;" OwnerUserId="3329" LastActivityDate="2021-10-21T23:59:15.513" Title="How to train annotations tools" Tags="|gene|sequence-annotation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17911" PostTypeId="1" CreationDate="2021-10-22T07:31:09.737" Score="2" ViewCount="33" Body="&lt;p&gt;How can I get all proteins coded by mitochondrial genome or plastid genome from UniProt. I see UniProt has multiple filtering and search parameters but I could not distinguish between proteins/genes coded by nuclear DNA and mitochondrial or plastid genomes.&lt;/p&gt;&#xA;&lt;p&gt;I specifically want UniProt (rather than e.g. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genome/organelle/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Organelle Genome Resources&lt;/a&gt;) because usually annotation of proteins uses UniProt ids.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="12357" LastEditorUserId="73" LastEditDate="2021-10-26T22:25:11.920" LastActivityDate="2022-01-25T08:35:43.100" Title="Organelle genome genes" Tags="|proteins|genome|uniprot|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17912" PostTypeId="1" CreationDate="2021-10-22T12:14:40.157" Score="0" ViewCount="525" Body="&lt;p&gt;I would like to perform &lt;em&gt;de novo&lt;/em&gt; genome assembly on a diploid microalgal strain.&lt;/p&gt;&#xA;&lt;p&gt;I have two datasets:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;PacBio CCS/HiFi reads, low coverage.&lt;/li&gt;&#xA;&lt;li&gt;Illumina PE 2x150 (standard shotgun)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Does anybody have any software recommendations for how best to use both in a hybrid &lt;em&gt;de novo&lt;/em&gt; assembly?&lt;/p&gt;&#xA;&lt;p&gt;Most options out there are set up for Pacbio CLR reads, not Pacbio CCS/HiFi, for using in hybrid assembly. Perhaps I should assemble short read contigs first then combine with &lt;a href=&quot;https://github.com/adigenova/wengan#pacccs-experimental&quot; rel=&quot;nofollow noreferrer&quot;&gt;wengan&lt;/a&gt;, or go back to the PacBio subreads instead of CCS to use in a hybrid method?&lt;/p&gt;&#xA;" OwnerUserId="2907" LastEditorUserId="2108" LastEditDate="2021-10-24T18:39:37.300" LastActivityDate="2021-10-24T18:39:37.300" Title="How to de novo hybrid assemble with Pacbio CCS and Illumina PE reads" Tags="|assembly|software-recommendation|illumina|pacbio|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17913" PostTypeId="1" AcceptedAnswerId="18048" CreationDate="2021-10-22T13:03:16.863" Score="0" ViewCount="214" Body="&lt;p&gt;Suppose, I want to find the neighboring Hydrogen atoms of a specific C-alpha atom.&lt;/p&gt;&#xA;&lt;p&gt;The following was my effort. However, it produces a bunch of empty lists.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from typing import List&#xA;from Bio.PDB import Chain, Atom, NeighborSearch, PDBParser&#xA;&#xA;&#xA;def get_hydrogen_atoms(c_alpha_atom):&#xA;    ns = NeighborSearch([c_alpha_atom])&#xA;    return ns.search_all(190.0, &amp;quot;A&amp;quot;)&#xA;&#xA;p = PDBParser()&#xA;structure = p.get_structure('16pk', '16pk.pdb')&#xA;&#xA;c_alpha_list = [atom for atom in structure.get_atoms()]&#xA;&#xA;for c_alpha in c_alpha_list:&#xA;    if c_alpha.name == &amp;quot;CA&amp;quot;:&#xA;        hydrogen_list = get_hydrogen_atoms(c_alpha)&#xA;        print(hydrogen_list)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I correct this?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="298" LastEditDate="2023-01-24T11:50:42.333" LastActivityDate="2023-01-24T11:50:42.333" Title="How can I get a list of neighboring Hydrogen atoms of an alpha-carbon?" Tags="|python|protein-structure|biopython|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17914" PostTypeId="1" CreationDate="2021-10-22T13:44:42.377" Score="0" ViewCount="35" Body="&lt;p&gt;I am running the script from de Distance examples (&lt;a href=&quot;http://examples.distancesampling.org/Distance-cues/cuecounts-distill.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://examples.distancesampling.org/Distance-cues/cuecounts-distill.html&lt;/a&gt;) to estimate density using function dht2 of Distance library but I have an error.&lt;/p&gt;&#xA;&lt;p&gt;The script is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library (Distance)&#xA;data(wren_cuecount)&#xA;conversion.factor &amp;lt;- convert_units(&amp;quot;meter&amp;quot;, NULL, &amp;quot;hectare&amp;quot;)&#xA;wren_cuecount$Effort &amp;lt;- wren_cuecount$Search.time&#xA;wrensong.hr &amp;lt;- ds(wren_cuecount, transect=&amp;quot;point&amp;quot;, key=&amp;quot;hr&amp;quot;, adjustment=NULL, &#xA;                  truncation=100)&#xA;cuerate &amp;lt;- unique(wren_cuecount[ , c(&amp;quot;Cue.rate&amp;quot;,&amp;quot;Cue.rate.SE&amp;quot;)])&#xA;names(cuerate) &amp;lt;- c(&amp;quot;rate&amp;quot;, &amp;quot;SE&amp;quot;)&#xA;(mult &amp;lt;- list(creation=cuerate))&#xA;wren.estimate &amp;lt;- dht2(wrensong.hr, flatfile=wren_cuecount, strat_formula=~1,&#xA;                      multipliers=mult, convert_units=conversion.factor)&#xA;print(wren.estimate, report=&amp;quot;density&amp;quot;)&amp;quot;&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I run the dht2 function I get:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error in apply(grps, 1, function(x) any(is.na(x))) :    dim(X) must&#xA;have a positive length&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How can I solve this?&lt;/p&gt;&#xA;" OwnerUserId="14419" LastEditorUserId="3694" LastEditDate="2021-10-23T17:24:20.727" LastActivityDate="2021-10-23T17:24:20.727" Title="Error in &quot;apply(grps, 1, function(x) any(is.na(x))) : dim(X) must have a positive length&quot; at running dht2 function in R" Tags="|r|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17916" PostTypeId="1" AcceptedAnswerId="17921" CreationDate="2021-10-22T22:21:03.803" Score="2" ViewCount="1898" Body="&lt;p&gt;I need to programmatically add Hydrogen atoms to a PDB file (say, 16PK.pdb) using BioPython.&lt;/p&gt;&#xA;&lt;p&gt;What algorithm should I use for this?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastActivityDate="2023-12-24T23:06:27.157" Title="How can I programmatically add Hydrogen to a PDB structure using BioPython?" Tags="|python|proteins|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17917" PostTypeId="1" CreationDate="2021-10-22T22:33:28.163" Score="1" ViewCount="590" Body="&lt;p&gt;I have some &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5142217/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PCR-CLIP&lt;/a&gt; data. PAR-CLIP identifies targets for RNA binding proteins. I am interested to build metagene profile so that transcript wide binding pattern can be compared. What I am mostly interested to plot:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;5UTR-CDS-3UTR&lt;/li&gt;&#xA;&lt;li&gt;Exon-Intron-Exon&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;With deepTools I can make a combined profile where metagene profile for whole transcript length is found.&lt;/p&gt;&#xA;&lt;p&gt;For 5UTR-CDS-3UTR, what I did:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;create bigwig files (from bam files) for signal&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;used GTF as region of interest&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The code goes as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;computeMatrix scale-regions&#xA;--metagene&#xA;-m 1000&#xA;-S *bw&#xA;-R *gtf&#xA;-a 1000&#xA;-b 1000&#xA;--transcriptID transcript&#xA;--transcript_id_designator transcript_id&#xA;-o outMat.mtx&#xA;&#xA;plotProfile -m outMat.mtx -out outFig.png --perGroup&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Generated image, given below, where my understanding is&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;-1.0:TSS is 5' UTR,&lt;/li&gt;&#xA;&lt;li&gt;TSS:TES is CDS+3'UTR&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/t1Nlu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/t1Nlu.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If that is so, I am missing 3' UTR region.&#xA;So what I did, running &lt;code&gt;computeMatrix reference-point&lt;/code&gt; by centering at TSS, center and TES with +/- 1000bp. The plot is as follows:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/VsPw8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VsPw8.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;With some operations I converted it to:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/N0Oj7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/N0Oj7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Beside this kind of manual fix, I believe there are some systematic way to render such metagene profile where 5UTR-CDS-3UTR would be side by side for comparison.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;1. For plotting such 5UTR-CDS-3UTR metagene profile systematically, how should I proceed with deepTools&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In addition to that if I am interested to plot such metagene profile for 5Exon-Intron-3Exon, like the example given below.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/AD7zo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AD7zo.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;2. For generating Exon-Intron-Exon metagene profile how could I proceed with deepTools?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="14423" LastActivityDate="2021-10-22T22:33:28.163" Title="Metagene analysis with deepTools; 5UTR-CDS-3UTR and Exon-Intron-Exon" Tags="|chip-seq|deeptools|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17918" PostTypeId="1" CreationDate="2021-10-23T01:25:44.113" Score="1" ViewCount="176" Body="&lt;p&gt;I downloaded and extract blast nt.24 database, but when I run :&lt;/p&gt;&#xA;&lt;p&gt;blastn -task blastn-short -query consulta.fa -db &lt;strong&gt;nt.24&lt;/strong&gt; -out misalida.txt&lt;/p&gt;&#xA;&lt;p&gt;it says: mdb_env_open: No such file or directory&lt;/p&gt;&#xA;&lt;p&gt;and if a try with nt instead of nt.24 it says:&lt;/p&gt;&#xA;&lt;p&gt;BLAST Database error: Could not find volume or alias file (nt.00) referenced in alias file (/root/my-documents/blastdb/nt/nt&#xA;So my question is :&lt;/p&gt;&#xA;&lt;p&gt;Do I have to download all nt.xx files?&lt;/p&gt;&#xA;&lt;p&gt;And : Will it show the organisms in the output result?&lt;/p&gt;&#xA;" OwnerUserId="14425" LastActivityDate="2021-11-24T01:03:30.427" Title="Do I have to download all nt.xx files to perform a search in blastn 2.12.0" Tags="|blastn|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17919" PostTypeId="2" ParentId="17912" CreationDate="2021-10-23T03:45:34.477" Score="1" Body="&lt;p&gt;If you have reasonable coverage with HiFi (30X per haplotype), then there usually isn't any reason to also use Illumina. My understanding is that Illumina can make the assembly worse if you try to polish with it, because it can erroneously induce haplotype switching errors.&lt;/p&gt;&#xA;&lt;p&gt;I would use &lt;a href=&quot;https://genome.cshlp.org/content/early/2020/08/14/gr.263566.120.full.pdf+html&quot; rel=&quot;nofollow noreferrer&quot;&gt;HiCanu&lt;/a&gt; or &lt;a href=&quot;https://www.nature.com/articles/s41592-020-01056-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;HiFiAsm&lt;/a&gt;, depending on whether you expect significant heterozygosity, and also assuming that you have non-trivial HiFi coverage.&lt;/p&gt;&#xA;&lt;p&gt;If your HiFi coverage is very low (&amp;lt;15X per haplotype, maybe), you might consider going back to subreads, assembling those, and polishing with Pilon and your Illumina reads, or one of the workflows that you have found. But otherwise I don't think the Illumina will be very useful. If you had Hi-C reads you could use them directly in HiFiAsm to assist phasing and scaffolding, but I'm assuming that your Illumina is shotgun.&lt;/p&gt;&#xA;&lt;p&gt;I know it's a PB promo material, but you see that PB promotes HiFi as &lt;a href=&quot;https://www.pacb.com/wp-content/uploads/Trade-Up-to-HiFi-Sequencing-on-the-Sequel-IIe-System_North-America-Sales.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;not needing polishing&lt;/a&gt;, as &lt;a href=&quot;https://academic.oup.com/gigascience/article/9/12/giaa123/6034784&quot; rel=&quot;nofollow noreferrer&quot;&gt;HiFi has accuracy &amp;gt;99%, so shotgun Illumina is not going to do much&lt;/a&gt;. There is some argument for still polishing with your original long reads.&lt;/p&gt;&#xA;&lt;p&gt;I can recommend any of the linked papers for ideas or further explanation.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-10-23T03:45:34.477" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17921" PostTypeId="2" ParentId="17916" CreationDate="2021-10-23T16:45:53.687" Score="4" Body="&lt;p&gt;Not actually and answer as you cannot do it with &lt;code&gt;biopython.PDB&lt;/code&gt;. But there are may other tools.&lt;/p&gt;&#xA;&lt;p&gt;Several are online, for example: &lt;a href=&quot;https://zhanggroup.org/HAAD/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://zhanggroup.org/HAAD/&lt;/a&gt; by the folk who wrote I-Tasser.&#xA;Nearly every MD pipeline has their own protonation algorithm. The only hydrogens that are an issue are histidine protonation —&lt;code&gt;HIE&lt;/code&gt; vs. &lt;code&gt;HID&lt;/code&gt;— and protonation states of catalytic residues (&lt;code&gt;CYZ&lt;/code&gt; vs. &lt;code&gt;CYS&lt;/code&gt; etc).&lt;/p&gt;&#xA;&lt;p&gt;The PyMOL python module is really good and can do a lot. Only issue is that the easiest way to install is via conda.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c conda-forge -c schrodinger pymol-bundle&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;with which one can:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pymol2&#xA;pymol2.PyMOL() as pymol:&#xA;    pymol.cmd.load('.pdb', 'myprotein') # file&#xA;    # pymol.cmd.fetch('', 'myprotein') # PDB code&#xA;    pymol.cmd.h_add()&#xA;    pymol.cmd.save('.pdb')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;One can find neighbours to a given atom too (see &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/17548/pymol-output-polar-contacts-between-chains-to-text-file/17551&quot;&gt;this Q&amp;amp;A&lt;/a&gt;), select by secondary structure, define secondary structure (&lt;code&gt;pymol.cmd.dss()&lt;/code&gt;) etc. A caveat is that, bar for some commands like &lt;code&gt;pymol.cmd.get_model(selection).atom&lt;/code&gt;, one does not interact directly with the atoms and everything is done via selections.&lt;/p&gt;&#xA;&lt;p&gt;However, as this Q follows on from &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/17913/how-can-i-get-a-list-of-neighboring-hydrogen-atoms-of-an-alpha-carbon&quot;&gt;this unanswerable one&lt;/a&gt; and in turn from several others about SS. I am guessing it is to determine the hydrogen bond distances between different sheets. These have a backbone amide nitrogen (name &lt;code&gt;N&lt;/code&gt;) donating a hydrogen (&lt;code&gt;H&lt;/code&gt;, not &lt;code&gt;HA&lt;/code&gt; etc.) to an accepting backbone amide oxygen of another sheet (&lt;code&gt;O&lt;/code&gt;). Unless you want an exact distance, you could simply use the distance between these heavy atoms, which would be slightly larger than the &lt;code&gt;H-N&lt;/code&gt; bond plus the &lt;code&gt;H•O&lt;/code&gt; hydrogen bond and the oxygen may not be coplanar with the planar amide.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-10-23T16:45:53.687" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17922" PostTypeId="2" ParentId="17918" CreationDate="2021-10-24T00:32:13.077" Score="1" Body="&lt;p&gt;In general, yes, you do need to download and extract all (nt.00 - nt.46 as of now) of the nt portions in order to do a comprehensive BLAST against all of nt. The database has been split up for convenience (to make smaller individual files), so all are necessary to be comprehensive.&lt;/p&gt;&#xA;&lt;p&gt;Once you download all the pieces, you’ll want to use &lt;code&gt;nt&lt;/code&gt; as the database parameter to blastn.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;blastn -task blastn-short -query consulta.fa -db nt -out misalida.txt&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;As for the second question - this depends on how you run the BLAST. Each accession is associated with a taxid, which will map to its taxonomy. This can be mapped back to the organism, but you may need to do some interpretation in order to convert the BLAST hits to the proper organism - this depends highly on what kind of sequences you are aligning.&lt;/p&gt;&#xA;" OwnerUserId="92" LastEditorUserId="92" LastEditDate="2021-10-24T01:25:25.517" LastActivityDate="2021-10-24T01:25:25.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17923" PostTypeId="1" AcceptedAnswerId="17944" CreationDate="2021-10-24T01:20:06.473" Score="2" ViewCount="2542" Body="&lt;p&gt;I am doing scRNAseq analysis with Seurat.&lt;/p&gt;&#xA;&lt;p&gt;I clustered the cells using the FindClusters() function.&lt;/p&gt;&#xA;&lt;p&gt;What I want to do is to export information about which cells belong to which clusters to a CSV file.&lt;/p&gt;&#xA;&lt;p&gt;In a Seurat object, we can show the cluster IDs by using Idents(・), but I have no idea how to export this to CSV files.&lt;/p&gt;&#xA;&lt;p&gt;I would be grateful if you could show this by using the PMBC data (&lt;a href=&quot;https://satijalab.org/seurat/articles/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/articles/pbmc3k_tutorial.html&lt;/a&gt;) as an example.&lt;/p&gt;&#xA;" OwnerUserId="14429" LastEditorUserId="73" LastEditDate="2021-10-26T22:07:26.663" LastActivityDate="2021-10-27T09:21:28.803" Title="In Seurat, how can I export cluster IDs to csv files" Tags="|seurat|single-cell|clustering|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17924" PostTypeId="2" ParentId="694" CreationDate="2021-10-24T05:17:41.950" Score="2" Body="&lt;p&gt;You can use PyMol to change the PDB to mmtf or other format. First open the PDB file and then choose file-&amp;gt;export molecule-&amp;gt; choose which format you want to save as&lt;/p&gt;&#xA;" OwnerUserId="14431" LastActivityDate="2021-10-24T05:17:41.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17925" PostTypeId="2" ParentId="17918" CreationDate="2021-10-25T00:18:59.727" Score="0" Body="&lt;p&gt;Thanks, I downloaded nt.00 (by http firefox because wget via ftp always returned a corrupted file when decompressing it) and it works fine locally.&#xA;The command is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;blastn -task blastn-short -query consulta.fa -db nt.00 -out misalida.txt&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I also added a first line &amp;gt;Seq1 to the query file consulta.fa (if not didn't work):&lt;strong&gt;&amp;gt;Seq1&#xA;GTGCC...&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;And the results in misalida.txt first print the taxid with the organism name and chromosome. At the bottom it printed the positions details.&#xA;What I'm doing is a genome editor, so when you enter some nucleotids(like 14 to 20 or more) it will search with blastn possible next nucleotids paths from many organisms with  blastdbcmd.&lt;/p&gt;&#xA;" OwnerUserId="14425" LastActivityDate="2021-10-25T00:18:59.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17926" PostTypeId="2" ParentId="17896" CreationDate="2021-10-25T13:35:08.463" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://www.gencodegenes.org/human/&quot; rel=&quot;nofollow noreferrer&quot;&gt;current release&lt;/a&gt; of Gencode for &lt;code&gt;hg38&lt;/code&gt; is v38. This is one way you could get a list of Gencode gene annotations:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ wget -qO- ftp://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_38/gencode.v38.annotation.gff3.gz \&#xA;    | gunzip --stdout - \&#xA;    | awk '$3 == &amp;quot;gene&amp;quot;' - \&#xA;    | convert2bed -i gff - \&#xA;    &amp;gt; genes.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To get &lt;code&gt;convert2bed&lt;/code&gt;, you could install a toolkit called BEDOPS, which is available via &lt;a href=&quot;https://github.com/bedops/bedops&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github&lt;/a&gt; or package managers. Refer to the &lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/installation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;installation instructions&lt;/a&gt; for more detail.&lt;/p&gt;&#xA;&lt;p&gt;Once you have &lt;code&gt;genes.bed&lt;/code&gt;, you don't need to make it again, unless Gencode publishes a new set of annotations.&lt;/p&gt;&#xA;&lt;p&gt;You could create a text file called &lt;code&gt;genes_of_interest.txt&lt;/code&gt; that contains your HGNC names of interest, e.g.:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat &amp;gt; genes_of_interest.txt&#xA;CASZ1&#xA;MTHFR&#xA;ST7L&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Press &lt;kbd&gt;Ctrl&lt;/kbd&gt; + &lt;kbd&gt;D&lt;/kbd&gt; at the end to close the file.&lt;/p&gt;&#xA;&lt;p&gt;To search &lt;code&gt;genes.bed&lt;/code&gt; for entries where there are matches to your genes of interest, you can use &lt;code&gt;grep&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -Fwf genes_of_interest.txt genes.bed&#xA;chr1    10636603    10796650    ENSG00000130940.14  .   -   HAVANA  gene    .   ID=ENSG00000130940.14;gene_id=ENSG00000130940.14;gene_type=protein_coding;gene_name=CASZ1;level=1;havana_gene=OTTHUMG00000002035.5&#xA;chr1    11785722    11806920    ENSG00000177000.11  .   -   HAVANA  gene    .   ID=ENSG00000177000.11;gene_id=ENSG00000177000.11;gene_type=protein_coding;gene_name=MTHFR;level=2;havana_gene=OTTHUMG00000002277.5&#xA;chr1    112523517   112620825   ENSG00000007341.18  .   -   HAVANA  gene    .   ID=ENSG00000007341.18;gene_id=ENSG00000007341.18;gene_type=protein_coding;gene_name=ST7L;level=2;havana_gene=OTTHUMG00000011753.7&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above is just sample output. Your result may differ.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2021-10-25T13:35:08.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17927" PostTypeId="1" CreationDate="2021-10-25T13:57:39.823" Score="1" ViewCount="1547" Body="&lt;p&gt;I have an amino acids string &amp;quot;MKGK&amp;quot; and I would like to convert it into &lt;a href=&quot;https://en.wikipedia.org/wiki/Simplified_molecular-input_line-entry_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;SMILES&lt;/a&gt; format.&lt;/p&gt;&#xA;&lt;p&gt;I tried two approaches.&lt;/p&gt;&#xA;&lt;p&gt;First with &lt;a href=&quot;https://www.novoprolabs.com/tools/convert-peptide-to-smiles-string&quot; rel=&quot;nofollow noreferrer&quot;&gt;PepSMI&lt;/a&gt; which gives me:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;N[C@@]([H])(CCSC)C(=O)N[C@@]([H])(CCCCN)C(=O)NCC(=O)N[C@@]([H])(CCCCN)C(=O)O&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Second, using ChemAxon Marvin's &lt;a href=&quot;https://docs.chemaxon.com/display/docs/molconvert.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;molconvert&lt;/strong&gt;&lt;/a&gt; command&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/Applications/MarvinSuite/bin/molconvert --peptide MKGK smiles&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[H]N[C@@H](CCSC)C(=O)N[C@@H](CCCCN)C(=O)NCC(=O)N[C@@H](CCCCN)C(O)=O&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Both of them give different result. Are they chemically the same?&#xA;If not, which result should I trust?&lt;/p&gt;&#xA;" OwnerUserId="12433" LastActivityDate="2021-10-26T12:29:30.787" Title="How to convert amino acid sequence to SMILES format" Tags="|computational-biochemistry|conversion|cheminformatics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17928" PostTypeId="1" CreationDate="2021-10-25T17:11:51.013" Score="0" ViewCount="87" Body="&lt;p&gt;I'm using RepeatMasker on human sequences. When running RepeatMasker this way:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;RepeatMasker -e rmblast -species human -dir . myFile.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;RepeatMasker builds some libraries:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Building general libraries in: /path/to/share/RepeatMasker/Libraries/CONS-Dfam_3.3/general&#xA;Building species libraries in: /path/to/share/RepeatMasker/Libraries/CONS-Dfam_3.3/human&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(Paths edited)&#xA;On subsequent runs, this does not happen.&#xA;I want to check if these libraries exist beforehand (which I can easily do with &lt;code&gt;ls /path/to/share/RepeatMasker/Libraries/CONS-Dfam_3.3/&lt;/code&gt;), and if they don't, build them.&lt;/p&gt;&#xA;&lt;p&gt;I could do a run with a decoy fasta to trigger that step, but it's a bit hacky. Is there a better way to force Repeat Masker to build those libraries?&lt;/p&gt;&#xA;&lt;p&gt;(I mention that I'm using bioconda in the title because I'm working on a remote server with restrictions, so that should be taken into account, no because there is already a solution for other installation methods).&lt;/p&gt;&#xA;" OwnerUserId="14438" LastActivityDate="2022-03-24T22:03:55.313" Title="Build RepeatMasker default libraries beforehand (bioconda installation)" Tags="|repeatmasker|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17929" PostTypeId="1" CreationDate="2021-10-25T18:26:41.980" Score="1" ViewCount="112" Body="&lt;p&gt;when I run a short search in blastn locally with less than 16 nucleotides with nt.00 database I don't get any found match, but if I increase the searched string 2 nucleotides it finds matches, so the 14 string was in the BD. Also, when I run the same 14  nucleotides search in a BD created with homo-sapiens Y chromosome it finds matches. This BD was created with the command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;makeblastdb -in homoy.fasta -out homoydb -dbtype 'nucl' -parse_seqids -hash_index&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you see it includes &lt;code&gt;-parse_seqids&lt;/code&gt; &lt;code&gt;-hash_index&lt;/code&gt; parameters.&#xA;The command for search is (replacing homoydb with nt.00 in the first case):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastn -task blastn-short -query myqueryin.fa -db homoydb -out myout.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So why blast nt preformatted databases don't find sequences shorter than 16 if they are in the BD and with my own DB find them OK?&lt;/p&gt;&#xA;" OwnerUserId="14425" LastEditorUserId="4282" LastEditDate="2021-10-26T13:56:37.443" LastActivityDate="2021-10-26T13:56:37.443" Title="Blastn locally nt database search length &lt; 16" Tags="|blastn|makeblastdb|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17930" PostTypeId="2" ParentId="17928" CreationDate="2021-10-25T19:14:52.100" Score="1" Body="&lt;p&gt;Something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;db_dir=$CONDA_PREFIX/share/RepeatMasker/Libraries/CONS-Dfam_3.3/&#xA;&#xA;if [ -d &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${db_dir}&quot; ] &amp;amp;&amp;amp; [ -n &quot;$&lt;/span&gt;(ls -A ${db_dir})&amp;quot; ]; then&#xA;    echo &amp;quot;CONS-Dfam folder is not empty&amp;quot;&#xA;    # ... do something&#xA;else&#xA;    echo &amp;gt; sequence.fasta&#xA;    RepeatMasker -e rmblast -species human -dir . sequence.fasta&#xA;fi&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If the folder exists and is not empty, otherwise build it using an empty file.&lt;/p&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2021-10-25T19:14:52.100" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17931" PostTypeId="1" CreationDate="2021-10-25T22:58:57.010" Score="1" ViewCount="2782" Body="&lt;p&gt;I asked this question in biology.stack exchange and was referred to this community. I am going to request migration for this question.&#xA;&lt;a href=&quot;https://biology.stackexchange.com/q/105024/67415&quot;&gt;https://biology.stackexchange.com/q/105024/67415&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want a PDB file of DNA sequence I am interested in (10 base pairs) for docking. I will appreciate if any of you can please clarify some of my doubts.&lt;/p&gt;&#xA;&lt;p&gt;Do you think, I can generate PDB file by using this link &lt;a href=&quot;http://www.scfbio-iitd.res.in/software/drugdesign/bdna.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.scfbio-iitd.res.in/software/drugdesign/bdna.jsp&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;How can I get the pdb file of the DNA sequence I want from protein data bank, I did search it but did not find the one I want. Is it a way to extract the PDB file?&lt;/p&gt;&#xA;&lt;p&gt;What are other methods I can use to generate PDB file of B-DNA sequence?&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your time.&lt;/p&gt;&#xA;&lt;p&gt;Please delete if it is against the group for posting again and redirect me to where I can request migration of this question.&lt;/p&gt;&#xA;" OwnerUserId="14439" LastActivityDate="2021-12-04T17:11:45.433" Title="Generating PDB file for custom DNA sequence" Tags="|phylogenetics|pdb|docking|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17932" PostTypeId="1" CreationDate="2021-10-26T01:04:03.637" Score="0" ViewCount="108" Body="&lt;p&gt;Once again, I am trying to upload reads to ENA using aspera. Following their instructions I used several times in past (successfully) to upload my reads&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ascp -QT -l300M -L- *.fastq.gz Webin-XXXXX@webin.ebi.ac.uk:.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;it starts as usual, uploads by the ~300M/s speed. But when it gets to ~30 - 40% complete transfer, it gets aborted from the server site:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;RR PEER ABORT: Received sesserr notification from peer, reason FASP_ABORT_LOCAL&#xA;ERR PEER ABORTED: Shutting down source/transmitter, reason FASP_ABORT_LOCAL&#xA;ERR Cannot write to management socket localhost:35666, Connection reset by peer (server)&#xA;LOG Sender bl t/o/r/dl/c/h=2672819/2672819/0/0/2668562/2668562 ctl bm/bs=0/0 rex b/r/n=0/0/0 r_ott q/s/n=0/0/1001 rate t/m/c=300000000/0/300000000 prog t/f/e=3762237952/376223795&#xA;2/104125014 xmtD=0&#xA;LOG Sender DS Qs ds/re/a/d/ao/eo/n/ap/rq/rd/ru/cl/di/do=1/0/0/0/0/0/0/0/0/0/1/0/0/0 Rs o/i=1/1 mgmt backlog i/s/n =&#xA;LOG Sender bl t/o/r/dl/c/h=2672946/2672946/0/0/2668562/2668562 ctl bm/bs=0/0 rex b/r/n=0/0/0 r_ott q/s/n=0/0/1001 rate t/m/c=300000000/0/300000000 prog t/f/e=3762237952/376223795&#xA;2/104125014 xmtD=0&#xA;LOG Sender DS Qs ds/re/a/d/ao/eo/n/ap/rq/rd/ru/cl/di/do=1/0/0/0/0/0/0/0/0/0/1/0/0/0 Rs o/i=1/1 mgmt backlog i/s/n =&#xA;ERR FASP sender had error, errcode 19 errstr [Cannot write to management socket localhost:35666, Connection reset by peer (server)]&#xA;ERR Peer FASP receiver had error, errcode 74 errstr [Cannot write to management socket localhost:35666, Connection reset by peer]&#xA;ERR Cannot write to management socket localhost:35666, Connection reset by peer (server), Cannot write to management socket localhost:35666, Connection reset by peer&#xA;              &#xA;Session Stop  (Error: Cannot write to management socket localhost:35666, Connection reset by peer (server), Cannot write to management socket localhost:35666, Connection reset by&#xA; peer)      &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does anyone have an idea what went wrong?&lt;/p&gt;&#xA;&lt;p&gt;I also asked ENA helpdesk, if I will get an answer from them I will forward it here.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2021-10-26T01:04:03.637" Title="uploading reads to ENA using aspera" Tags="|database|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17933" PostTypeId="1" CreationDate="2021-10-26T07:18:17.030" Score="1" ViewCount="163" Body="&lt;p&gt;When I execute:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;&amp;gt; FastqCleaner:::launch_fqc()()&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I get the following output&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Listening on http://127.0.0.1:4863&#xA;Warning: Navigation containers expect a collection of `bslib::nav()`/`shiny::tabPanel()`s and/or `bslib::nav_menu()`/`shiny::navbarMenu()`s. Consider using `header` or `footer` if you wish to place content above (or below) every panel's contents.&#xA;Warning: Error in : Navigation containers expect a collection of `bslib::nav()`/`shiny::tabPanel()`s and/or `bslib::nav_menu()`/`shiny::navbarMenu()`s. Consider using `header` or `footer` if you wish to place content above (or below) every panel's contents.&#xA;  85: stop&#xA;  84: FUN&#xA;  83: lapply&#xA;  82: buildTabset&#xA;  81: tabsetPanel_&#xA;  80: func&#xA;  78: tabsetPanel&#xA;Warning: Navigation containers expect a collection of `bslib::nav()`/`shiny::tabPanel()`s and/or `bslib::nav_menu()`/`shiny::navbarMenu()`s. Consider using `header` or `footer` if you wish to place content above (or below) every panel's contents.&#xA;Warning: Error in : Navigation containers expect a collection of `bslib::nav()`/`shiny::tabPanel()`s and/or `bslib::nav_menu()`/`shiny::navbarMenu()`s. Consider using `header` or `footer` if you wish to place content above (or below) every panel's contents.&#xA;  85: stop&#xA;  84: FUN&#xA;  83: lapply&#xA;  82: buildTabset&#xA;  81: tabsetPanel_&#xA;  80: func&#xA;  78: tabsetPanel&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Have I done something wrong or is the function broken in some way?&lt;/p&gt;&#xA;" OwnerUserId="14444" LastEditorUserId="3694" LastEditDate="2023-09-07T11:49:34.750" LastActivityDate="2023-09-07T11:49:34.750" Title="fastqCleaner failing to launch in RStudio" Tags="|r|bioconductor|fastq|fastqc|rstudio|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17934" PostTypeId="2" ParentId="17931" CreationDate="2021-10-26T09:15:28.527" Score="1" Body="&lt;p&gt;I normally use PyMOL. In PyMOL you can generate a model of a sequence of DNA with the command &lt;a href=&quot;https://pymolwiki.org/index.php/Fnab&quot; rel=&quot;nofollow noreferrer&quot;&gt;fnab&lt;/a&gt; — a sister command to the better known fab, which makes a protein sequence of a given secondary structure. I've not encountered that site, but it says it uses experimental data, which is a better approach.&lt;/p&gt;&#xA;&lt;h2&gt;Side note 1&lt;/h2&gt;&#xA;&lt;p&gt;OP is asking for DNA, but if anyone stumbles across this, do note that RNA is a totally different ballgame and requires more advanced solutions (&lt;a href=&quot;https://www.rosettacommons.org/docs/latest/application_documentation/rna/rnp-modeling&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;h2&gt;Side note 2&lt;/h2&gt;&#xA;&lt;p&gt;I should say docking a known sequence is fine, but not docking to guess the sequence is not.&lt;/p&gt;&#xA;&lt;p&gt;For docking, if you have a zinc finger, helix-loop-helix, leucine zipper, MADS box or similar domain which binds DNA in a consistent manner then it is straightforward. However, predicting the bound sequence is far from trivial or accurate.&#xA;For example, in &lt;a href=&quot;https://michelanglo.sgc.ox.ac.uk/r/mef2c&quot; rel=&quot;nofollow noreferrer&quot;&gt;MEF2C the N-terminal amine&lt;/a&gt; there are three reader-head dictating specificity and one is a floppy terminus, which most forcefield-based methods would struggle to get right.&lt;/p&gt;&#xA;&lt;h2&gt;Testing&lt;/h2&gt;&#xA;&lt;p&gt;So let's say we want to test PyMOL's fnab in the GUI. We could pick a protein, MEF2A example from above:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fetch 1EGW, type=pdb1&#xA;print cmd.get_fastastr('chain E') # to get the sequence&#xA;fnab AAAGCTATTATTAGCTT&#xA;align obj01 and chain A, 1EGW and chain E&#xA;show sticks, chain E+F&#xA;hide cartoon&#xA;remove solvent&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The RMSD is 1.6 Å which is not great, but pretty decent considering that it is with a generated sequence that does not take into account backbone distortion from the bound protein.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/m8uEL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m8uEL.png&quot; alt=&quot;DNAs&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2021-10-26T15:46:36.977" LastActivityDate="2021-10-26T15:46:36.977" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="17935" PostTypeId="1" CreationDate="2021-10-26T11:40:43.560" Score="1" ViewCount="92" Body="&lt;p&gt;I want to download exon-wise gene sequence of around 200 genes from NCBI. Is there anyway to do this from terminal? I have the transcript id for each sequence and I am using efetch command to download their CDS.&lt;/p&gt;&#xA;&lt;p&gt;eg: &lt;code&gt;efetch -db nuccore -format fasta_cds_na -id NM_000125.4&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, I need their &lt;strong&gt;exon-wise&lt;/strong&gt; sequence as well (i.e. I need the sequence of each exon separately). It will be better if the exon starts with start codon and ends with stop codon (the untranslated regions of those exons completely removed) but I can modify the sequences accordingly once I have all the sequences (UTR included).&lt;/p&gt;&#xA;&lt;p&gt;Is there any way in which we can give the NCBI transcript id as an input and get exon-wise sequence as an output?&lt;/p&gt;&#xA;" OwnerUserId="14446" LastEditorUserId="73" LastEditDate="2021-10-26T22:12:14.363" LastActivityDate="2021-10-26T22:12:14.363" Title="Downloading exon-wise gene sequence using commandline" Tags="|sequence-analysis|phylogenetics|exon|efetch|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17936" PostTypeId="2" ParentId="17927" CreationDate="2021-10-26T12:29:30.787" Score="3" Body="&lt;p&gt;SMILES are meant for machines, but reading up on &lt;a href=&quot;https://en.wikipedia.org/wiki/Simplified_molecular-input_line-entry_system&quot; rel=&quot;nofollow noreferrer&quot;&gt;the format&lt;/a&gt; is very handy and one totally can read and write them —I do it often and could not recommend it enough.&lt;/p&gt;&#xA;&lt;p&gt;You can have SMILES written differently and they give the same result —hence why you should never compare SMILES, but Inchi or better still a parsed molecule —e.g. in RDKit.&lt;/p&gt;&#xA;&lt;p&gt;Looking at them, I can see that&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;they are the same except for one explicit proton on the N-terminus.&lt;/li&gt;&#xA;&lt;li&gt;Neither are canonical SMILES —which is fine.&lt;/li&gt;&#xA;&lt;li&gt;Both are written N to C like biologist write peptides —chemists write peptides C to N due to solid phase synthesis.&lt;/li&gt;&#xA;&lt;li&gt;they are L-amino acids: the pattern &lt;code&gt;N[C@@H](*)C(=O)O&lt;/code&gt; is the give away. Technically, the implicit hydrogen in the Cα [C@@H] has &lt;code&gt;@@&lt;/code&gt;, which means counter-clockwise  —Voet &amp;amp; Voet taught everyone to read C(=O)RN counter-clockwise with the N-C (the first bond) sticking out.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The first has rather oddly declared chiral protons, the second has an unnecessary N-terminus proton, which is not adding a charge: &lt;code&gt;[H]N*&lt;/code&gt; is the same as &lt;code&gt;[NH2]*&lt;/code&gt; as it is assumed to be &lt;code&gt;[H][NH]*&lt;/code&gt;. So the correct version &lt;strong&gt;at pH 7&lt;/strong&gt; should be:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;[NH3+][C@@H](CCSC)C(=O)N[C@@H](CCCCN)C(=O)NCC(=O)N[C@@H](CCCCN)C([O-])=O&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-10-26T12:29:30.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17937" PostTypeId="1" CreationDate="2021-10-26T13:30:22.057" Score="1" ViewCount="49" Body="&lt;p&gt;I have a treatment and control in two time points like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; design&#xA;                                  condition time&#xA;    1       CTRL_24_hrs_replicate1   control   24&#xA;    2       CTRL_24_hrs_replicate2   control   24&#xA;    3       CTRL_24_hrs_replicate3   control   24&#xA;    4  treatment_24_hrs_replicate1        t   24&#xA;    5  treatment_24_hrs_replicate2        t   24&#xA;    6  treatment_24_hrs_replicate3        t   24&#xA;    7       CTRL_48_hrs_replicate1   control   48&#xA;    8       CTRL_48_hrs_replicate2   control   48&#xA;    9       CTRL_48_hrs_replicate3   control   48&#xA;    10 treatment_48_hrs_replicate1        t   48&#xA;    11 treatment_48_hrs_replicate2        t   48&#xA;    12 treatment_48_hrs_replicate3        t   48&#xA;    &amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to test between &lt;strong&gt;treatment&lt;/strong&gt; and &lt;strong&gt;control&lt;/strong&gt; considering time point 24 hours to 48 hours&lt;/p&gt;&#xA;&lt;p&gt;I have done like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData=a,colData=design, design=~time + condition + time:condition)&#xA;&#xA;But at this part I get error, although I am trying different things&#xA;&#xA;&amp;gt; ddsTC &amp;lt;- DESeq(dds, test=&amp;quot;LRT&amp;quot;, reduced = ~ time + time:condition)&#xA;estimating size factors&#xA;estimating dispersions&#xA;gene-wise dispersion estimates&#xA;mean-dispersion relationship&#xA;final dispersion estimates&#xA;fitting model and testing&#xA;Error in nbinomLRT(object, full = full, reduced = reduced, quiet = quiet,  : &#xA;  less than one degree of freedom, perhaps full and reduced models are not in the correct order&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; ddsTC &amp;lt;- DESeq(dds, test=&amp;quot;LRT&amp;quot;, reduced = ~time:condition)&#xA;estimating size factors&#xA;estimating dispersions&#xA;gene-wise dispersion estimates&#xA;mean-dispersion relationship&#xA;final dispersion estimates&#xA;fitting model and testing&#xA;Error in nbinomLRT(object, full = full, reduced = reduced, quiet = quiet,  : &#xA;  less than one degree of freedom, perhaps full and reduced models are not in the correct order&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried these with no error although I am not certain if this makes sense at all while the results of both is the exactly the same&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; ddsTC &amp;lt;- DESeq(dds, test=&amp;quot;LRT&amp;quot;, reduced = ~ time + condition)&#xA;estimating size factors&#xA;estimating dispersions&#xA;gene-wise dispersion estimates&#xA;mean-dispersion relationship&#xA;final dispersion estimates&#xA;fitting model and testing&#xA;&amp;gt; &#xA;&#xA;&amp;gt; ddsTC &amp;lt;- DESeq(dds, test=&amp;quot;LRT&amp;quot;, reduced = ~ time)&#xA;estimating size factors&#xA;estimating dispersions&#xA;gene-wise dispersion estimates&#xA;mean-dispersion relationship&#xA;final dispersion estimates&#xA;fitting model and testing&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Al I want is getting the difference of treatment versus control but &lt;em&gt;&lt;strong&gt;considering&lt;/strong&gt;&lt;/em&gt; time goes from 24 hours to 48 hours&lt;/p&gt;&#xA;&lt;p&gt;Thanks for any help&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2021-10-27T15:22:53.663" LastActivityDate="2021-10-27T15:22:53.663" Title="The correct design for time series experiment" Tags="|r|differential-expression|deseq2|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17938" PostTypeId="1" CreationDate="2021-10-25T17:00:57.860" Score="1" ViewCount="230" Body="&lt;p&gt;I want a PDB file of DNA sequence I am interested in (10 base pairs) for docking. I will appreciate if any of you can please clarify some of my doubts.&lt;/p&gt;&#xA;&lt;p&gt;Do you think, I can generate PDB file by using this link &lt;a href=&quot;http://www.scfbio-iitd.res.in/software/drugdesign/bdna.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.scfbio-iitd.res.in/software/drugdesign/bdna.jsp&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I did generate the PDB file using the above site. However, when I am using that PDB file for docking it do not give accurate results. It just fits to the sequence and it lacks the information the PDB file downloaded from protein data bank has.&lt;/p&gt;&#xA;&lt;p&gt;How can I get the pdb file of the DNA sequence I want from protein data bank, I did search it but did not find the one I want. Is it a way to extract the PDB file?&lt;/p&gt;&#xA;&lt;p&gt;What are other methods I can use to generate PDB file of B-DNA sequence?&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your time.&lt;/p&gt;&#xA;" OwnerUserId="14439" OwnerDisplayName="dnalectronics" LastEditorUserId="73" LastEditDate="2021-10-26T22:14:16.730" LastActivityDate="2024-01-15T12:17:20.960" Title="getting PDB file of interested sequence" Tags="|phylogenetics|pdb|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17939" PostTypeId="2" ParentId="17923" CreationDate="2021-10-26T22:04:25.873" Score="2" Body="&lt;p&gt;I've recently done this with my own Seurat data; here's the process:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# load tidyverse packages&#xA;library(tidyverse)&#xA;# create cluster ID table&#xA;tibble(cellID = colnames(pbmc), clusterID = Idents(pbmc)) %&amp;gt;%&#xA;  # write out to a file, with today's date&#xA;  write_csv(file = sprintf(&amp;quot;cluster_mappings_%s.csv&amp;quot;, Sys.Date()))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-10-26T22:04:25.873" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17940" PostTypeId="1" CreationDate="2021-10-26T22:50:31.740" Score="0" ViewCount="38" Body="&lt;p&gt;I've got gene sequences of mouse genes. Each sequence contains variants from the reference genome. I need to find out whether these SNPs/variations are in functionally significant regions (i.e. exons).&lt;/p&gt;&#xA;&lt;p&gt;I also need to see if these SNPs/mutations are conserved across species.&lt;/p&gt;&#xA;&lt;p&gt;I'm not very proficient in this, but would BLAST or ENSEMBL be useful here? Is there an easy way to search this? I have the sequences as .txt files. I'm proficient in Python if that's useful as well.&lt;/p&gt;&#xA;" OwnerUserId="14450" LastActivityDate="2021-10-26T22:50:31.740" Title="Identifying Variant Conservation Across Species" Tags="|blast|ensembl|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17941" PostTypeId="2" ParentId="6694" CreationDate="2021-10-27T00:50:57.207" Score="1" Body="&lt;p&gt;We had the same issue. Turned out, our tdTomato transgene was followed up by WPRE and BGH polyA signals. These sequences are quite large, and their presence precludes sequencing into the tdTomato coding sequence when using the 10x genomics platform. Incorporating these into our build, allowed us to detect tdTomato+ cells.&lt;/p&gt;&#xA;" OwnerUserId="14451" LastActivityDate="2021-10-27T00:50:57.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17943" PostTypeId="2" ParentId="17923" CreationDate="2021-10-27T09:10:46.427" Score="0" Body="&lt;p&gt;The clusters are saved in the field &lt;code&gt;object$seurat_clusters&lt;/code&gt; where &lt;code&gt;object&lt;/code&gt; represents your Seurat object.&lt;/p&gt;&#xA;&lt;p&gt;You can also get them running the command &lt;code&gt;Idents(object)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;NOTE&lt;/strong&gt;: if you run the command &lt;code&gt;FindClusters&lt;/code&gt; on different SNN graphs, or different resolutions, you need to specify the clusters by &lt;code&gt;object$graph_name_resolution&lt;/code&gt;.&#xA;Every time you run &lt;code&gt;FindClusters&lt;/code&gt;, the idents of the cells are updated to the latest results, same for &lt;code&gt;seurat_clusters&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For instance, let's say that you have run the command &lt;code&gt;FindNeighbors&lt;/code&gt; on the &lt;code&gt;RNA&lt;/code&gt; assay (default value).&lt;br /&gt;&#xA;Then you run &lt;code&gt;FindClusters&lt;/code&gt; with &lt;code&gt;graph = &amp;quot;RNA_snn&amp;quot;&lt;/code&gt; and &lt;code&gt;resolution=1&lt;/code&gt;.&lt;br /&gt;&#xA;In that case, the clusters are in &lt;code&gt;object$RNA_snn_1&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Once you have the clusters, you can save them as a CSV:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;clusters &amp;lt;-  &amp;lt;here you use one of the described methods&amp;gt;&#xA;# data.frame&#xA;clust.df &amp;lt;- data.frame(cellID = colnames(object), clusters = clusters)&#xA;write.csv(clust.df, &amp;lt;path.of.the.csv.file&amp;gt;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14454" LastEditorUserId="14454" LastEditDate="2021-10-27T09:16:21.410" LastActivityDate="2021-10-27T09:16:21.410" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17944" PostTypeId="2" ParentId="17923" CreationDate="2021-10-27T09:21:28.803" Score="2" Body="&lt;p&gt;The clusters information, together with additional information on each cells (%mito, sample name, group, orig.ident,...) are stored inside &lt;code&gt;seuratObject@meta.data&lt;/code&gt;.&#xA;&lt;code&gt;write.csv(seuratObject@meta.data,&amp;quot;./seurat_metadata.csv&amp;quot;)&lt;/code&gt; will dump everything to a file.&lt;/p&gt;&#xA;&lt;p&gt;The method @osmoc suggested, is using built in seurat functions to interact with &lt;code&gt;seuratObject@meta.data&lt;/code&gt;.&#xA;&lt;code&gt;seuratObject$graph_name_resolution&lt;/code&gt; points to &lt;code&gt;seuratObject@meta.data$graph_name_resolution&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I personally prefer to use the metadata table, as I have more control on how I interact with the data.&lt;/p&gt;&#xA;" OwnerUserId="3631" LastActivityDate="2021-10-27T09:21:28.803" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17945" PostTypeId="2" ParentId="4511" CreationDate="2021-10-27T11:04:17.727" Score="1" Body="&lt;p&gt;You can use the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Datasets&lt;/a&gt; to download the protein sequences of all annotated protein-coding genes for an organism starting with the organism name. There is a web-interface that allows exploring by taxonomy or specific organism search as well as a command line utility and an API. For example, the command line utility can be used as follows:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$ datasets download genome taxon 'trichoderma harzianum' \&#xA;    --annotated --exclude-genomic-cds --exclude-gff3 \&#xA;    --exclude-rna --exclude-seq --reference&#xA;Collecting 1 genome accessions [================================================] 100% 1/1&#xA;Downloading: ncbi_dataset.zip    3.92MB done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This downloads a zip archive containing the &lt;code&gt;protein.faa&lt;/code&gt; file for the &lt;em&gt;Trichoderma harzianum&lt;/em&gt; reference assembly.&lt;/p&gt;&#xA;&lt;p&gt;Note, unlike the Entrez method described above, this will not download all proteins for a given organism from the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/protein&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Protein&lt;/a&gt; portal. Thus, &lt;em&gt;Lophodermium seditiosum&lt;/em&gt; does not return any results as there is no available genome for this assembly in the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Assembly&lt;/a&gt; database.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, the NCBI Datasets method will be significantly faster and return cleaner results. I would argue that if you are interested in obtaining the 'proteomes' of a given set of organisms, this is a better choice as it includes only those cases where a complete proteome is available based on the annotation of the genome and does not include cases where only a few proteins from an organism are available.&lt;/p&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2021-10-27T11:04:17.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17946" PostTypeId="1" AcceptedAnswerId="17950" CreationDate="2021-10-27T11:35:35.637" Score="0" ViewCount="210" Body="&lt;p&gt;I know how to traverse through structures, models, residues, and atoms in a protein chain using BioPython (which is very easy).&lt;/p&gt;&#xA;&lt;p&gt;How can I identify donor and acceptor atoms in a chain?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastActivityDate="2021-10-28T10:27:27.520" Title="How can I identify acceptor and donor atoms using BioPython?" Tags="|python|proteins|small-molecules|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="17948" PostTypeId="1" CreationDate="2021-10-27T23:32:01.053" Score="0" ViewCount="119" Body="&lt;p&gt;I am analyzing ukbb data and I have patient IDs for both cases and controls. Now I want the genetic data of all patients based on their IDs.&#xA;I am trying to test with chromosome 5 as I have both bgen and sample file. I am using plink 2 for the conversion of bgen and sample file&lt;br /&gt;&#xA;&lt;code&gt;plink2 --bgen ukb_chr5_v3.bgen ref-first --sample ukb22828_c5_b0_v3_s487239.sample --make-bed --out testfile&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am getting three files as output like .pgen .psam .pvar.&#xA;This is how .psam looks like,&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;FID&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: center;&quot;&gt;IID&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;sex&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;482828&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;482282&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;and .pvar file has five columns, chromosome, position, ID, ref allele, alt allele.&#xA;I am still confused which IDs have the corresponding alleles/genotypes.&#xA;Please guide me in further analysis. My goal is find genetic data of specfic IDs which I am looking for.&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="14215" LastActivityDate="2021-10-27T23:32:01.053" Title="Linking UK biobank patient IDs with their corresponding genetic data" Tags="|r|statistics|plink|phylogenetics|epidemiology|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17949" PostTypeId="1" AcceptedAnswerId="17953" CreationDate="2021-10-28T03:31:32.873" Score="0" ViewCount="1019" Body="&lt;p&gt;I am now drawing a weighted directed graph using Pygraphviz.&lt;/p&gt;&#xA;&lt;p&gt;The adjacency matrix A of the weighted directed graph is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A = pd.DataFrame({'X': [0.0,0.1,-0.8], 'Y': [-0.7,0.0,-0.1], 'Z': &#xA;    [0.4,0,-0.1]} ,index=[&amp;quot;X&amp;quot;, &amp;quot;Y&amp;quot;, &amp;quot;Z&amp;quot;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tpEZ4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tpEZ4.png&quot; alt=&quot;A&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;where A_{i,j} indicates the edge weight from the node i to the node j.&lt;/p&gt;&#xA;&lt;p&gt;I was able to draw the graph without edge color.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import networkx as nx&#xA;from networkx.drawing.nx_agraph import to_agraph&#xA;from IPython.display import SVG, display&#xA;&#xA;G_nx = nx.from_pandas_adjacency(A,create_using=nx.DiGraph())&#xA;G_nx.graph['edge'] = {'arrowsize': '1.0', 'splines': 'curved'}&#xA;G_nx.graph['graph'] = {'scale': '3'}&#xA;&#xA;&#xA;Agraph_eg = to_agraph(G_nx)                                                              &#xA;Agraph_eg .node_attr[&amp;quot;height&amp;quot;] = 0.3&#xA;Agraph_eg .node_attr[&amp;quot;width&amp;quot;] = 0.3&#xA;Agraph_eg .node_attr[&amp;quot;shape&amp;quot;] = &amp;quot;circle&amp;quot;&#xA;Agraph_eg .node_attr[&amp;quot;fixedsize&amp;quot;] = &amp;quot;true&amp;quot;&#xA;Agraph_eg .node_attr[&amp;quot;fontsize&amp;quot;] = 8&#xA;Agraph_eg.layout(prog=&amp;quot;neato&amp;quot;)                                                                     &#xA;Agraph_eg.draw('graph_eg1.png')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/k7jpY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/k7jpY.png&quot; alt=&quot;graph_eg1.png&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But I have no idea how to assign edge colors and widths according to the following two rules.&lt;/p&gt;&#xA;&lt;p&gt;#1 If weights are more than 0, edge color should be red. If weight are less than 0, edge color should be blue.&lt;/p&gt;&#xA;&lt;p&gt;#2 Edge widths should be proportional to the absolute values of weights.&lt;/p&gt;&#xA;&lt;p&gt;Do you have any idea how to do this??&lt;/p&gt;&#xA;" OwnerUserId="14429" LastEditorUserId="14429" LastEditDate="2021-10-28T03:45:19.927" LastActivityDate="2021-10-28T20:02:34.657" Title="In Pygraphviz, How can I assign edge colors according to edge weights?" Tags="|python|networks|graphs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17950" PostTypeId="2" ParentId="17946" CreationDate="2021-10-28T10:27:27.520" Score="3" Body="&lt;p&gt;A hydrogen bond donor heavy atom is one that is protonated, while the acceptor has a lone pair. If you talking about amino acids, then it's simple as they are fixed.&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;atom name&lt;/th&gt;&#xA;&lt;th&gt;description&lt;/th&gt;&#xA;&lt;th&gt;role&lt;/th&gt;&#xA;&lt;th&gt;note&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;&lt;code&gt;N&lt;/code&gt;&lt;/td&gt;&#xA;&lt;td&gt;backbone nitrogen&lt;/td&gt;&#xA;&lt;td&gt;donor&lt;/td&gt;&#xA;&lt;td&gt;SS&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;&lt;code&gt;CA&lt;/code&gt;&lt;/td&gt;&#xA;&lt;td&gt;backbone Cα&lt;/td&gt;&#xA;&lt;td&gt;apolar&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;&lt;code&gt;C&lt;/code&gt;&lt;/td&gt;&#xA;&lt;td&gt;backbone carboxyl C&lt;/td&gt;&#xA;&lt;td&gt;sterically hindered and electrophilic&lt;/td&gt;&#xA;&lt;td&gt;nucleophilic attack&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;&lt;code&gt;O&lt;/code&gt;&lt;/td&gt;&#xA;&lt;td&gt;backbone carboxyl O&lt;/td&gt;&#xA;&lt;td&gt;acceptor&lt;/td&gt;&#xA;&lt;td&gt;SS&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;&lt;code&gt;CB&lt;/code&gt; etc.&lt;/td&gt;&#xA;&lt;td&gt;sidechain carbons&lt;/td&gt;&#xA;&lt;td&gt;apolar&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;&lt;code&gt;OXT&lt;/code&gt;&lt;/td&gt;&#xA;&lt;td&gt;C-terminal oxygen&lt;/td&gt;&#xA;&lt;td&gt;acceptor&lt;/td&gt;&#xA;&lt;td&gt;may be protonated especially if a membrane embedded C-terminus&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Serine's &lt;code&gt;OG&lt;/code&gt; / Threonine's &lt;code&gt;OG1&lt;/code&gt;&lt;/td&gt;&#xA;&lt;td&gt;sidechain hydroxyl&lt;/td&gt;&#xA;&lt;td&gt;donor&lt;/td&gt;&#xA;&lt;td&gt;SS: may form ST turns. Fairly uncommon: may be a donor&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Asp &lt;code&gt;OD1&lt;/code&gt;/&lt;code&gt;OD2&lt;/code&gt;/Asn &lt;code&gt;OD1&lt;/code&gt;&lt;/td&gt;&#xA;&lt;td&gt;sidechain carboxyl/amide O&lt;/td&gt;&#xA;&lt;td&gt;acceptor&lt;/td&gt;&#xA;&lt;td&gt;SS: may form Asx turns&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Glu &lt;code&gt;OG1&lt;/code&gt;/&lt;code&gt;OG2&lt;/code&gt;/Gln &lt;code&gt;OG1&lt;/code&gt;&lt;/td&gt;&#xA;&lt;td&gt;sidechain carboxyl/amide O&lt;/td&gt;&#xA;&lt;td&gt;acceptor&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Lysine &lt;code&gt;NZ&lt;/code&gt;&lt;/td&gt;&#xA;&lt;td&gt;sidechain amine&lt;/td&gt;&#xA;&lt;td&gt;donor&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Asn &lt;code&gt;ND2&lt;/code&gt; / Gln &lt;code&gt;NG2&lt;/code&gt;&lt;/td&gt;&#xA;&lt;td&gt;terminal admide N&lt;/td&gt;&#xA;&lt;td&gt;donor&lt;/td&gt;&#xA;&lt;td&gt;the direction of an Asn/Gln may be accidentally filled if manually assigned as N/O have the same density in X-ray crystals&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;Water can be both a donor and an acceptor.&#xA;Two charged residues form a salt bridge more so than a hydrogen bond, because the interaction is driven by the Coulomb electrostatic forces of the two heavy atoms.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-10-28T10:27:27.520" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="17951" PostTypeId="1" CreationDate="2021-10-28T10:55:57.030" Score="2" ViewCount="149" Body="&lt;p&gt;I have &amp;gt;100 qPCR experiments that I'd like to analyze together, each containing the same set of genes (10 genes of interest and 2 reference genes).&#xA;I have four different samples (untreated &amp;amp; 3 treatments) from the same cell culture. Our setup allows two of them to be run at once, i.e. the samples from the same cell culture are split to two qPCR runs.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I would like to do statistics with the dCq values after normalizing to the reference genes, but the dataset doesn't pass any normality tests.&#xA;What methods are suitable for finding valid differences between treatments (or any other features like age)?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If the same amount of cDNA was used and reference genes were added to every run, is it still necessary to normalize between runs or between cell cultures?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="12596" LastActivityDate="2021-10-29T05:38:26.430" Title="What statistics can I use on qPCR data composed of several runs if all normality tests fail on the dataset?" Tags="|statistics|normalization|qpcr|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17952" PostTypeId="1" CreationDate="2021-10-28T13:28:53.913" Score="0" ViewCount="35" Body="&lt;p&gt;I want to carry out the simulation of drug-dna complex by placing the drug molecule at particular site in my DNA sequence (intercalating site). I have PDB files of both drug and DNA sequence. I will truly appreciate if anyone could guide me how to place my drug at the desired location in the DNA sequence using Amber.&lt;/p&gt;&#xA;" OwnerUserId="14439" LastEditorUserId="96" LastEditDate="2021-10-28T13:53:10.733" LastActivityDate="2021-10-28T13:53:10.733" Title="Drug-DNA complex simulation using AMBER" Tags="|ngs|phylogenetics|docking|drugs|molecular-dynamics|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17953" PostTypeId="2" ParentId="17949" CreationDate="2021-10-28T20:02:34.657" Score="2" Body="&lt;p&gt;This might help you&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for edge in G_nx.edges(data=True):&#xA;    color = &amp;quot;black&amp;quot;&#xA;    weight = edge[2][&amp;quot;weight&amp;quot;]&#xA;    if weight &amp;gt; 0:&#xA;        color = &amp;quot;red&amp;quot;&#xA;    elif weight &amp;lt; 0:&#xA;        color = &amp;quot;blue&amp;quot;&#xA;    edge[2][&amp;quot;color&amp;quot;] = color&#xA;    edge[2][&amp;quot;penwidth&amp;quot;] = abs(weight)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This gives you the following graph:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xa2yg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xa2yg.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Depending on your network size this solution might take some time. &amp;quot;penwidth&amp;quot; is the graphviz information for the edge width.&lt;/p&gt;&#xA;" OwnerUserId="3360" LastActivityDate="2021-10-28T20:02:34.657" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17954" PostTypeId="2" ParentId="17951" CreationDate="2021-10-29T05:38:26.430" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;A wilcoxon test is what you're looking for.&lt;/li&gt;&#xA;&lt;li&gt;Yes, since there will inevitably be some small batch effects to compensate for.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-10-29T05:38:26.430" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17956" PostTypeId="1" CreationDate="2021-10-29T14:06:22.777" Score="2" ViewCount="26" Body="&lt;p&gt;I am interested in constructing a phylogenetic tree containing the classical HLA α1 and α2 domains and potential classical MHC α1 and α2 domains of a different organism. I am using the amino acid sequences for the tree.&lt;/p&gt;&#xA;&lt;p&gt;However, I am unsure of what bioinformatic tools and software to use and what parameters to fill in.&lt;/p&gt;&#xA;&lt;p&gt;Lastly any advice on how to read a phylogenetic tree and how to relate the sequence similarity between two sequences by looking at the tree would be very helpful.&lt;/p&gt;&#xA;&lt;p&gt;My aim is to investigate the classical MHC genes of a non human organism.&lt;/p&gt;&#xA;&lt;p&gt;Any help and advice would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="12135" LastEditorUserId="734" LastEditDate="2021-11-01T05:06:27.530" LastActivityDate="2021-11-01T05:06:27.530" Title="Constructing a phylogenetic tree for classical MHC genes from different organisms" Tags="|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17958" PostTypeId="1" AcceptedAnswerId="17959" CreationDate="2021-10-29T21:27:20.013" Score="1" ViewCount="67" Body="&lt;p&gt;Hi guys I would like to run wordcount from Emboss in a bunch of fasta files. I tried this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;import sys&#xA;import subprocess&#xA;&#xA;&#xA;&#xA;filenames = ['Downloads/Data/GCF_000016725.1_chr.fna']&#xA;for filename in filenames:&#xA;    name = filename.split('/')[2]&#xA;    print(filename)&#xA;    subprocess.call (['wordcount', 'Input sequence(s): ' + filename, 'Word size [4]: ' + sys.argv[1]])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I got error messages:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Count and extract unique words in molecular sequence(s)&#xA;Error: Failed to open filename 'Input sequence(s): Downloads/Data/GCF_000016725.1_chr.fna'&#xA;Error: Unable to read sequence 'Input sequence(s): Downloads/Data/GCF_000016725.1_chr.fna'&#xA;Died: wordcount terminated: Bad value for '-sequence' and no prompt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But if a use shell with Downloads/Data/GCF_000016725.1_chr.fna  it works fine!&lt;/p&gt;&#xA;&lt;p&gt;Any tip would be helpful!&#xA;I am using python, but I would have a method to compare my results!&lt;/p&gt;&#xA;&lt;p&gt;Thank you by your time!&lt;/p&gt;&#xA;&lt;p&gt;Paulo&lt;/p&gt;&#xA;" OwnerUserId="4222" LastActivityDate="2021-10-30T01:17:43.837" Title="Using python to call Emboss wordcount" Tags="|python|emboss|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17959" PostTypeId="2" ParentId="17958" CreationDate="2021-10-30T00:59:19.513" Score="3" Body="&lt;p&gt;You have to use command line options, not interactive prompts, like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;subprocess.call(['wordcount', '-sequence', filename, '-wordsize', sys.argv[1]], '-outfile', filename + '.wordcount.txt')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or, using the &lt;a href=&quot;https://docs.python.org/reference/lexical_analysis.html#f-strings&quot; rel=&quot;nofollow noreferrer&quot;&gt;f-strings or formatted string literals&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;subprocess.call(f'wordcount -sequence {filename} -wordsize {sys.argv[1]} -outfile {filename}.wordcount.txt')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that I had to add &lt;code&gt;-outfile&lt;/code&gt; option to your system call, otherwise &lt;code&gt;wordcount&lt;/code&gt; prompts for the output file. See &lt;a href=&quot;https://www.bioinformatics.nl/cgi-bin/emboss/help/wordcount&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;wordcount&lt;/code&gt; manual&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6251" LastEditorUserId="6251" LastEditDate="2021-10-30T01:17:43.837" LastActivityDate="2021-10-30T01:17:43.837" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17961" PostTypeId="1" CreationDate="2021-10-30T23:07:49.067" Score="1" ViewCount="703" Body="&lt;p&gt;I have the following Pymol visual,&#xA;The green molecule is receptor and red/yellow is the ligand.&#xA;In the ligand I highlighted Lysine (K) residue as yellow.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GJy8Kl.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GJy8Kl.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What I want to do is to highlight and draw the electro static contact of the ligand especially with Lysine residue. Like this in Pymol:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/M8lCum.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/M8lCum.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;How can I achieve that?&lt;/p&gt;&#xA;" OwnerUserId="12433" LastActivityDate="2021-10-31T11:00:58.910" Title="How to show electrostatic interactions in Pymol on the wanted residue" Tags="|protein-structure|docking|pymol|structural-biology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17962" PostTypeId="1" CreationDate="2021-10-31T03:29:02.617" Score="0" ViewCount="284" Body="&lt;p&gt;I want to draw directed graphs using networkx, followed by manual tuning of the visualization with the use of NETWOLF.&lt;/p&gt;&#xA;&lt;p&gt;Here is the example of a network.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;G =nx.DiGraph()&#xA;G.add_edges_from([(1, 2), (1, 3),(1, 4),(2, 3)])&#xA;network, config = nw.visualize(G,config={'zoom':3})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, the graph shown in NETWOLF was undirected graph.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xvKti.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xvKti.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Do you know if you can style network visualizations of directed graphs using NETWULF?&lt;/p&gt;&#xA;" OwnerUserId="14429" LastEditorUserId="14429" LastEditDate="2021-10-31T03:40:07.990" LastActivityDate="2022-06-10T16:09:47.780" Title="Can you style network visualizations of directed graphs using NETWULF?" Tags="|python|networks|graphs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17963" PostTypeId="2" ParentId="17961" CreationDate="2021-10-31T11:00:58.910" Score="4" Body="&lt;p&gt;Depends whether you work with named selections which is preferable and helpful if you want to use this Pymol session for future images. In this case, use the distance command for your two selections, one being the N of the K in your ligand and the other the O of the E of your protein.&lt;/p&gt;&#xA;&lt;p&gt;Full syntax and examples here: &lt;a href=&quot;https://pymolwiki.org/index.php/Distance&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pymolwiki.org/index.php/Distance&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you need a quick solution: in the menu Wizard, click on Measurement. Then click on the two atoms where you want the dotted line to appear in between.&lt;/p&gt;&#xA;" OwnerUserId="6240" LastActivityDate="2021-10-31T11:00:58.910" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17964" PostTypeId="2" ParentId="17956" CreationDate="2021-10-31T16:32:30.317" Score="1" Body="&lt;p&gt;This is a very unusual situation as a main effort is finding the outgroup where these two domains aren't duplicated.&lt;/p&gt;&#xA;&lt;h2&gt;General situation&lt;/h2&gt;&#xA;&lt;p&gt;In animals a gene tree nearly always follows a species tree, with the exception of gene duplication events, which frequently stem from genome duplication events. Making a tree with lots of genes gets hard to follow and actually becomes pointlessly problematic when you get a branch that does not quite follow the species phylogram —say, you have rabbit, mouse and human genes and the inferred tree, due to poor signal, comes out with mouse+human together as opposed to rabbit+mouse. As a result a nice tree needs to present the species before the genome duplication, often sea lamprey —I wrote &lt;a href=&quot;https://blog.matteoferla.com/2021/10/multiple-sequence-alignments.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;a blog post&lt;/a&gt; recently were I discuss nice species to pick for a multiple sequence alignment.&lt;/p&gt;&#xA;&lt;p&gt;In this simple case, one can get the protein sequence of interest and &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;blastp&lt;/a&gt; it against human, sea lamprey and fruit fly, see how divergent are the paralogues in human and if the split is more recent that the latter two, if so, the latter will be an outgroup and after getting a few representative species in between, one can make a MSA with &lt;a href=&quot;https://www.ebi.ac.uk/Tools/msa/muscle/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Muscle&lt;/a&gt;, which contains also a tool to make a neighbour joining tree.&lt;/p&gt;&#xA;&lt;h2&gt;Domain duplication&lt;/h2&gt;&#xA;&lt;p&gt;In your case you have something more complicated in that you have a series of paralogues, each gene encoding a protein (e.g. &lt;a href=&quot;https://www.uniprot.org/uniprot/P04439#family_and_domains&quot; rel=&quot;nofollow noreferrer&quot;&gt;HLA-A&lt;/a&gt;) containing two globular domains, which &lt;a href=&quot;http://pfam.xfam.org/family/MHC_I&quot; rel=&quot;nofollow noreferrer&quot;&gt;form the MHC-1 fold&lt;/a&gt;, and you want make a tree of this... which means you want find out when the two domains duplicated (i.e. outgroup)? If the domains work together (correct?) and have the same fold (as you alluded to, correct?), this would mean that before the duplication event of the domains, the ancestral protein may have worked as a homodimer.&lt;/p&gt;&#xA;&lt;p&gt;I am not familiar with the literature, so do not know if this has already been looked at —but I suspect it has.&lt;/p&gt;&#xA;&lt;p&gt;If not, clicking on the sequence of one of the two domains in the Uniprot entry of one of the paralogues and doing a few iterations of psi-blast against human and a few other distant species (such as zebra fish, sea urchins, which are basal to vertebrates, and fruit fly, which I do not believe have a MHC etc.) most of these will have two hits per protein... but one might not. This is the outgroup!&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-10-31T16:32:30.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17965" PostTypeId="1" CreationDate="2021-10-31T18:58:26.750" Score="2" ViewCount="606" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8ncih.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8ncih.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you can see in the picture I have two files from the paired end reads. I would assume READ1 would be forward reads and READ2 would be reverse reads, but my lab colleague told me that that might not be the case. How do I confirm?&lt;/p&gt;&#xA;" OwnerUserId="9373" LastEditorUserId="9373" LastEditDate="2021-11-02T17:27:26.550" LastActivityDate="2021-11-02T17:27:26.550" Title="I have two fasta files READ1 and READ 2. How to identify which one is the reverse and the forward?" Tags="|fasta|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17966" PostTypeId="1" CreationDate="2021-10-31T20:48:27.123" Score="1" ViewCount="556" Body="&lt;p&gt;I know the algorithm for creating a Hydrogen atom and adding to a residue:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Point3d  create_hydrogen(Point3d C, Point3d N, Point3d CA, Point3d H) &#xA;{&#xA;  H.set(N);&#xA;  H -= C;&#xA;  H.norm();&#xA;&#xA;  Point3d tmp2(N);&#xA;  tmp2 -= CA;&#xA;  tmp2.norm();&#xA;&#xA;  H += tmp2;&#xA;  H.norm(N_H_BOND_LENGTH);&#xA;  H += N;&#xA;&#xA;  return H;&#xA;}&#xA;&#xA;Point3d  create_hydrogen(Residue prev_residue, Residue residue, Point3d H) &#xA;{&#xA;  return create_hydrogen(prev_residue.find_atom(&amp;quot; C  &amp;quot;), &#xA;                              residue.find_atom(&amp;quot; N  &amp;quot;),&#xA;                              residue.find_atom(&amp;quot; CA &amp;quot;), &#xA;                                   H);&#xA;}&#xA;&#xA;Atom create_hydrogen(Residue prev_residue, Residue residue) &#xA;{&#xA;  Atom H = make_shared_atom(0, &amp;quot; H  &amp;quot;, 1);&#xA;  create_hydrogen(prev_residue, residue, H);&#xA;  residue.add(H);&#xA;  return H;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I didn't find this implementation in BioPython.&lt;/p&gt;&#xA;&lt;p&gt;Is this already implemented in BioPython? If YES, in which module?&lt;/p&gt;&#xA;&lt;p&gt;If NO, what would be the implementation like in BioPython?&lt;/p&gt;&#xA;&lt;p&gt;Added image of peptide bond too try to depict the problem at hand, I believe the hydrogen in cause is the one on the peptide bond plane(pale yellow) indentifiedin purple&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IUpqe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IUpqe.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="10049" LastEditDate="2023-10-13T14:37:26.913" LastActivityDate="2023-10-15T16:06:15.363" Title="How can I programmatically add a Hydrogen 'Atom' to a 'Residue' object?" Tags="|python|protein-structure|" AnswerCount="3" CommentCount="20" ContentLicense="CC BY-SA 4.0" />
  <row Id="17967" PostTypeId="2" ParentId="17965" CreationDate="2021-11-01T05:31:03.813" Score="-1" Body="&lt;p&gt;If this is RNASeq, then the question might have an answer (or it might be nonsense), but you have to know what kind of library prep was used.&lt;/p&gt;&#xA;&lt;p&gt;Or just align and inspect the results to figure it out.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastEditorUserId="73" LastEditDate="2021-11-02T01:24:55.103" LastActivityDate="2021-11-02T01:24:55.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17968" PostTypeId="2" ParentId="17965" CreationDate="2021-11-01T10:21:22.033" Score="3" Body="&lt;p&gt;Unless this is RNA-seq prepared with a directional library, or something else that you're sure is directional, it is irrelevant.&lt;/p&gt;&#xA;&lt;p&gt;DNA is complementary, so a DNA molecule can be turned into a sequence-able fragment with either strand given any pair of Illumina adapters and indices.&lt;/p&gt;&#xA;&lt;p&gt;As far as Illumina lingo goes, R1 is the first read that is read off the sequencer, or what people call the &amp;quot;forward read&amp;quot;. R2 is the second pass of the same clusters, and some people call it the &amp;quot;reverse read&amp;quot;. I recommend just calling them R1 and R2 reads to avoid confusion. Sequencing by synthesis produces the cleanest reads in the first pass, so R1 reads usually have better Q values. You can check this easily by comparing the quality scores with &lt;a href=&quot;https://www.bioinformatics.babraham.ac.uk/projects/fastqc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;fastqc&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You can learn more about Illumina sequencing chemistry &lt;a href=&quot;http://nextgen.mgh.harvard.edu/IlluminaChemistry.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, or check out some other answers on the bioinformatics SE or biostars.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2021-11-01T10:21:22.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17969" PostTypeId="2" ParentId="17965" CreationDate="2021-11-01T13:26:43.210" Score="0" Body="&lt;p&gt;I think @conchoecia answer is correct but maybe not clear enough for someone making himself that question for the first time (like was my case recently).&lt;/p&gt;&#xA;&lt;p&gt;I think the simple way to explain this is that in F_NL_READ1.fq you have random reads, in mixed directions, and in F_NL_READ2.fq you have the paired read in the opposed direction for every read in F_NL_READ1.fq, but not necessarily and almost certainly, not every read in every file will go in the same sense (forward or reverse).&lt;/p&gt;&#xA;&lt;p&gt;At least for paired-end reads from Illumina.&lt;/p&gt;&#xA;" OwnerUserId="5190" LastActivityDate="2021-11-01T13:26:43.210" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17970" PostTypeId="1" CreationDate="2021-11-01T13:41:08.713" Score="0" ViewCount="123" Body="&lt;p&gt;I have two assemblies obtained from two different softwares for the same run data. In one of them, I get a BUSCO score of 69,4%. In the other one, I get 68,5%. The first assembly covers a 99,7% fraction of the reference while the second one covers 100%.&lt;/p&gt;&#xA;&lt;p&gt;The relevant thing is that the reference genome also gets 68,5% (less than the first assembly). How can I interpret this situation?&lt;/p&gt;&#xA;&lt;p&gt;I identify these options:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The first assembly is showing a different strain.&lt;/li&gt;&#xA;&lt;li&gt;BUSCO is imprecise&lt;/li&gt;&#xA;&lt;li&gt;The first assembly is merging DNA from another organism&lt;/li&gt;&#xA;&lt;li&gt;The first assembly is a misassembly, repeating part of the sequence and BUSCO interprets that as new genes&lt;/li&gt;&#xA;&lt;li&gt;The sample really is an organism that has more genes than the reference.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The organism is a bacteria and I am using bacteria_odb10 as the lineage dataset.&lt;/p&gt;&#xA;&lt;p&gt;This is the BUSCO result for the reference (LN554883.1 Chlamydophila abortus strain 1H):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;C:68.5%[S:68.5%,D:0.0%],F:8.1%,M:23.4%,n:124       &#xA;85  Complete BUSCOs (C)            &#xA;85  Complete and single-copy BUSCOs (S)    &#xA;0   Complete and duplicated BUSCOs (D)     &#xA;10  Fragmented BUSCOs (F)              &#xA;29  Missing BUSCOs (M)             &#xA;124 Total BUSCO groups searched&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This for the first assembly:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;C:69.4%[S:69.4%,D:0.0%],F:8.9%,M:21.7%,n:124       &#xA;86  Complete BUSCOs (C)            &#xA;86  Complete and single-copy BUSCOs (S)    &#xA;0   Complete and duplicated BUSCOs (D)     &#xA;11  Fragmented BUSCOs (F)              &#xA;27  Missing BUSCOs (M)             &#xA;124 Total BUSCO groups searched     &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And this for the second assembly:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;C:68.5%[S:0.0%,D:68.5%],F:8.1%,M:23.4%,n:124       &#xA;85  Complete BUSCOs (C)            &#xA;0   Complete and single-copy BUSCOs (S)    &#xA;85  Complete and duplicated BUSCOs (D)     &#xA;10  Fragmented BUSCOs (F)              &#xA;29  Missing BUSCOs (M)             &#xA;124 Total BUSCO groups searched&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I see now that the first assembly also contains a lot of contigs with human DNA. I guess BUSCO finds a match for some genes in the human DNA?&lt;/p&gt;&#xA;" OwnerUserId="5190" LastEditorUserId="73" LastEditDate="2021-11-02T21:03:20.860" LastActivityDate="2021-11-02T21:05:57.510" Title="BUSCO results apparently inconsistent" Tags="|assembly|busco|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17971" PostTypeId="2" ParentId="17966" CreationDate="2021-11-01T15:40:23.270" Score="2" Body="&lt;p&gt;Not sure but here &lt;a href=&quot;https://biopython.org/docs/1.75/api/Bio.PDB.Residue.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biopython.org/docs/1.75/api/Bio.PDB.Residue.html&lt;/a&gt;   Bio.PDB.Residue module Your can find :&lt;/p&gt;&#xA;&lt;p&gt;add(self, atom) :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;      Add an Atom object.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Checks for adding duplicate atoms, and raises a PDBConstructionException if so.&lt;/p&gt;&#xA;&lt;p&gt;Can’t try it just now but who knows it may work&lt;/p&gt;&#xA;&lt;p&gt;Googling around I found out that if you plan to add Hydrogens to PDB files [I believe nowadays PDBx is the repository grade standard] you could do thar following PDB format rules. I am copying them from UCSF Chimera website (&lt;a href=&quot;https://www.cgl.ucsf.edu/chimera/docs/UsersGuide/tutorials/pdbintro.html#hydrogens&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cgl.ucsf.edu/chimera/docs/UsersGuide/tutorials/pdbintro.html#hydrogens&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;Hydrogen Atoms&#xA;&#xA;In brief, conventions for hydrogen atoms in version 3.0 PDB format are as follows:&#xA;&#xA;Hydrogen atom records follow the records of all other atoms of a particular residue.&#xA;A hydrogen atom name starts with H. The next part of the name is based on the name of the connected nonhydrogen atom. For example, in amino acid residues, H is followed by the remoteness indicator (if any) of the connected atom, followed by the branch indicator (if any) of the connected atom; if more than one hydrogen is connected to the same atom, an additional digit is appended so that each hydrogen atom will have a unique name. Hydrogen atoms in standard nucleotides and amino acids (other than the rarely seen HXT) are named according to the IUPAC recommendations (Markley et al., Pure Appl Chem 70:117 (1998)). Names of hydrogen atoms in HETATM residues are determined in a similar fashion.&#xA;If the name of a hydrogen has four characters, it is left-justified starting in column 13; if it has fewer than four characters, it is left-justified starting in column 14. &#xA;&#xA;In the following excerpt from entry 1vm3, atom H is attached to atom N. Atom HA is attached to atom CA; the remoteness indicator A is the same for these atoms. Two hydrogen atoms are connected to CB, one is connected to CG, three are connected to CD1, and three are connected to CD2.&#xA;&#xA;ATOM     10  N   LEU A   2       4.595   6.365   3.756  1.00  0.00           N&#xA;ATOM     11  CA  LEU A   2       4.471   5.443   2.633  1.00  0.00           C&#xA;ATOM     12  C   LEU A   2       5.841   5.176   2.015  1.00  0.00           C&#xA;ATOM     13  O   LEU A   2       6.205   4.029   1.755  1.00  0.00           O&#xA;ATOM     14  CB  LEU A   2       3.526   6.037   1.578  1.00  0.00           C&#xA;ATOM     15  CG  LEU A   2       2.790   4.919   0.823  1.00  0.00           C&#xA;ATOM     16  CD1 LEU A   2       3.803   3.916   0.262  1.00  0.00           C&#xA;ATOM     17  CD2 LEU A   2       1.817   4.196   1.769  1.00  0.00           C&#xA;ATOM     18  H   LEU A   2       4.169   7.246   3.704  1.00  0.00           H&#xA;ATOM     19  HA  LEU A   2       4.063   4.514   2.992  1.00  0.00           H&#xA;ATOM     20  HB2 LEU A   2       2.804   6.675   2.065  1.00  0.00           H&#xA;ATOM     21  HB3 LEU A   2       4.099   6.623   0.873  1.00  0.00           H&#xA;ATOM     22  HG  LEU A   2       2.234   5.353   0.004  1.00  0.00           H&#xA;ATOM     23 HD11 LEU A   2       4.648   4.447  -0.148  1.00  0.00           H&#xA;ATOM     24 HD12 LEU A   2       3.334   3.331  -0.516  1.00  0.00           H&#xA;ATOM     25 HD13 LEU A   2       4.137   3.260   1.052  1.00  0.00           H&#xA;ATOM     26 HD21 LEU A   2       0.941   3.892   1.216  1.00  0.00           H&#xA;ATOM     27 HD22 LEU A   2       1.522   4.860   2.568  1.00  0.00           H&#xA;ATOM     28 HD23 LEU A   2       2.296   3.323   2.188  1.00  0.00           H&#xA;&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2021-11-01T21:52:10.323" LastActivityDate="2021-11-01T21:52:10.323" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17972" PostTypeId="2" ParentId="17966" CreationDate="2021-11-01T16:05:00.680" Score="2" Body="&lt;p&gt;The other answer is correct. But I thought I'd give a few pointers in how one can understand how to do something with a given Python module —teach a man to fish kind of thing...&lt;/p&gt;&#xA;&lt;p&gt;In a Jupyter Notebook or IPython shell (don't), you can do:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;help(function)&lt;/code&gt; print out the docstring&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;dir(object)&lt;/code&gt; shows all the public and magic attributes and methods —note a class will not show the instance methods&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;type(object)&lt;/code&gt; or &lt;code&gt;object.__class__&lt;/code&gt; gives you what the class is, but &lt;code&gt;object.__class__.__mro__&lt;/code&gt; tells you what it inherits.&lt;/li&gt;&#xA;&lt;li&gt;in the standard library module &lt;code&gt;inspect&lt;/code&gt; is the function &lt;code&gt;inspect.getsource(function)&lt;/code&gt;, which for a notebook is messy due to newlines so needs to  be &lt;code&gt;print_code = lambda fun: print(inspect.getsource(fun))&lt;/code&gt; and &lt;code&gt;print_code(fun)&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In your case you read a file...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import PDB&#xA;parser = PDB.PDBParser()&#xA;structure : PDB.Structure.Structure = parser.get_structure(&amp;quot;tryp-cage&amp;quot;, &amp;quot;1L2Y.first.pdb&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So &lt;code&gt;parser.get_structure&lt;/code&gt; &amp;quot;magically&amp;quot; makes a structure, eh? This means it can add stuff! As mentioned &lt;code&gt;print_code(parser.get_structure)&lt;/code&gt; will reveal its secrets, or within it call some &lt;code&gt;self._private_fun()&lt;/code&gt;, which can be inspected (&lt;code&gt;self&lt;/code&gt; = &lt;code&gt;parser&lt;/code&gt;) all the way down the rabbit hole. Until you find a nice constructor that declares nicely an atom instance or whatever you ever fancy searching for —that is the great thing of open source code.&lt;/p&gt;&#xA;&lt;p&gt;In the specific case of &lt;code&gt;PDB&lt;/code&gt; module, everything inherits &lt;code&gt;Entity&lt;/code&gt;, which has the attribute &lt;code&gt;child_list&lt;/code&gt;, which is what the iterators (&lt;code&gt;__iter__&lt;/code&gt; is the topmost) read without making a copy. The end of the above rabbit hole may just add a child &lt;code&gt;Entity&lt;/code&gt;-inheriting class instance to this list, but it also do other things that is worth keeping an eye out.&lt;/p&gt;&#xA;&lt;p&gt;In the case of &lt;code&gt;residue.add(atom)&lt;/code&gt;, creating the &lt;code&gt;Bio.PDB.Atom.Atom&lt;/code&gt; instance is slightly problematic as there's no typehinting or &lt;code&gt;:type xxx:&lt;/code&gt; declarations in the docstring. But doing &lt;code&gt;type(atom.bfactor)&lt;/code&gt; etc. on the attributes of an existing atom will reveal how to do it. One the serial number is a PDB thing, which is sequential for the model.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;last_serial = list(structure.get_residues())[-1].child_list[-1].serial_number + 1&#xA;residue.add(PDB.Atom.Atom(name=' H  ', coord=np.array([1,1,1]), bfactor=0., occupancy=1., altloc=' ', fullname=' H  ', serial_number=last_serial+1,element='H'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The coordinates will not be &lt;code&gt;[1,1,1]&lt;/code&gt;, but the point that you have calculate. Say for &lt;code&gt;H&lt;/code&gt;, this would be coplanar with &lt;code&gt;C&lt;/code&gt;, &lt;code&gt;CA&lt;/code&gt; and &lt;code&gt;N&lt;/code&gt;, that is 1 and smidge Å away from &lt;code&gt;N&lt;/code&gt; at an angle not quite in the middle as found in amide bonds.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2021-11-01T16:58:43.740" LastActivityDate="2021-11-01T16:58:43.740" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="17973" PostTypeId="2" ParentId="17970" CreationDate="2021-11-01T18:59:11.800" Score="1" Body="&lt;p&gt;The core idea behind BUSCO is that there are a number of genes that are Unique Single-Copy Orthologues (USCOs) that should be present in most genomes and should only appear &lt;em&gt;once&lt;/em&gt; in the genome. This is only a guess, however, and not a universal rule.&lt;/p&gt;&#xA;&lt;p&gt;Sometimes genomes can have multiple copies of these USCOs, and these duplicated genes reduce the overall score (which is based on the proportion of complete &lt;em&gt;single-copy&lt;/em&gt; genes that are present as a fraction of the entire USCO set).&lt;/p&gt;&#xA;&lt;p&gt;Regarding human DNA, bacteria are a curiously diverse group, with genomes that have more similarities in function than in form. Given this diversity, it wouldn't surprise me if contaminating DNA could increase the BUSCO score.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-11-02T21:05:57.510" LastActivityDate="2021-11-02T21:05:57.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17975" PostTypeId="1" CreationDate="2021-11-02T00:29:00.200" Score="4" ViewCount="105" Body="&lt;p&gt;I have a lot of fasta files, each one with thousand of reads containing the hexameric motif &amp;quot;CCCTCT&amp;quot;. The hexameric motif is highly continuous in most cases but interruptions may occur. I need to count the hexameric motif keeping the read ID and identify their structure (CCCTCT)n. These interruptions are real and do not come from Sequencing error&lt;/p&gt;&#xA;&lt;p&gt;Example Sequence:(interruption in bold)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;ACATTTTTTTTCCACATCTGATGTGGAAAAAAAAAAAAATGAAATAGCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCT&lt;strong&gt;CCT&lt;/strong&gt;CCCTCTCCCTCTCCCCACGGTCTCCCTCTCATGCGGAGCCAAAGCTGGACTGTACTGCT&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Strategy 1: Count the CCCTCT motif&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -on &amp;quot;CCCTCT&amp;quot; fasta.fa | cut -f 1 -d &amp;quot;:&amp;quot; | sort | uniq -c&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Output: &lt;code&gt;23 CCCTCT&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As you can see we have one interruption (&lt;code&gt;CCT&lt;/code&gt;) in this sequence which the codes doesn't detect.&lt;/p&gt;&#xA;&lt;p&gt;Strategy 2: Identify reads with the same structure and count repeats in each read&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio import SeqIO&#xA;from collections import defaultdict&#xA;&#xA;dedup_records = defaultdict(list)&#xA;for record in SeqIO.parse(&amp;quot;fasta.fa&amp;quot;, &amp;quot;fasta&amp;quot;):&#xA;    # Use the sequence as the key and then have a list of id's as the value&#xA;    dedup_records[str(record.seq)].append(record.id)&#xA;with open(&amp;quot;Output.fasta&amp;quot;, 'w') as output:&#xA;    for seq, ids in sorted(dedup_records.items(), key=lambda t: len(t[1]), reverse=True):&#xA;        # Join the ids and write them out as the fasta&#xA;        output.write(&amp;quot;&amp;gt;{}_counts{}\n&amp;quot;.format(ids[0], len(ids)))&#xA;        output.write(seq + &amp;quot;\n&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output: list ranking of reads with identical structures but does not show the structure&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;ID/ccs_counts2&#xA;CCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCTCCCTCTCCCTCT.....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can you please give me an idea of how to obtain the following output? example of a sequence of 23 hexamers with interruptions.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(CCCTCT)20 [INTERRUPTION] (CCCTCT)3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have used expansion hunter, repeatFinder and repeatAnalysis-tool by PacBio but they were not particularly sensitive for this case. Any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="14495" LastEditorUserId="96" LastEditDate="2021-11-02T14:24:33.820" LastActivityDate="2022-04-06T04:07:00.653" Title="Counting hexamers in fasta sequence and identify its structure (and interruptions)" Tags="|fasta|python|phylogenetics|structural-variation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17977" PostTypeId="1" CreationDate="2021-11-02T15:59:35.773" Score="1" ViewCount="287" Body="&lt;p&gt;I have an assignment for a project to build a tetrapeptides library to screen against some target enzymes &lt;em&gt;in silicon&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I am looking for a script to construct about 160000 tetrapeptide from the 20Amino acids permutations.&lt;/p&gt;&#xA;&lt;p&gt;I am quite familiar with Bash, but am struggling to get a function to working.&lt;/p&gt;&#xA;" OwnerUserId="14503" LastEditorUserId="3848" LastEditDate="2023-08-16T02:02:41.880" LastActivityDate="2023-08-16T02:02:41.880" Title="Permutations of 4 amino acid sequence oligopeptide" Tags="|python|phylogenetics|phylogeny|protein-structure|bash|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17978" PostTypeId="1" CreationDate="2021-11-02T17:35:44.283" Score="0" ViewCount="40" Body="&lt;p&gt;I have a population in the hundreds and did an Illumina pool-seq (ended up with ~100bp reads for the whole population).&lt;/p&gt;&#xA;&lt;p&gt;I have the reference genome for the species and now need to align all the population reads to the reference genome, figure out which are SNPs and finally determine which ones are changing from one generation to the next (about 20generations difference).&lt;/p&gt;&#xA;&lt;p&gt;Right now I'm trying to use popoolation2 &lt;a href=&quot;https://sourceforge.net/p/popoolation2/wiki/Main/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sourceforge.net/p/popoolation2/wiki/Main/&lt;/a&gt;&#xA;but it seems to be a little old as it has not been updated in ~10 years.&lt;/p&gt;&#xA;&lt;p&gt;In Galaxy, I couldn't find any pipelines dedicated to determine population SNP variation through generation.&lt;/p&gt;&#xA;&lt;p&gt;So two questions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;what software to use to determine significant variation in SNPs in a population (in poolSeq)?&lt;/li&gt;&#xA;&lt;li&gt;can you suggest me a pipeline for this? if one exists.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9373" LastActivityDate="2021-11-08T12:32:28.993" Title="What to use to determine SNPs in a population? How to find significant SNP variation between generations?" Tags="|snp|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17979" PostTypeId="1" CreationDate="2021-11-02T17:45:57.837" Score="1" ViewCount="140" Body="&lt;p&gt;We have an RNA seq taken from a the same tissue in two different developmental stages. We have one gene, with the number of reads from stage A and stage B, and we want to see which genes are differentially expressed.&lt;/p&gt;&#xA;&lt;p&gt;After performing RNA-seq, I am left with a CSV file containing a list of genes and the amount of reads in two different cell types. We have one list of genes, with the number of reads in Stage A in one column and the number of reads in Stage B in another column. We are looking to see what types of genes are differentially expressed in the two stages.&lt;/p&gt;&#xA;&lt;p&gt;I have used DESeq2, but I'm not sure if this is the best/correct to analyze what I am looking for.&lt;/p&gt;&#xA;&lt;p&gt;What is the best package to analyze the data for significant differential gene expression?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="14504" LastEditorUserId="73" LastEditDate="2021-12-04T06:08:22.030" LastActivityDate="2021-12-04T06:09:26.820" Title="Best R program to use for RNA seq analysis" Tags="|rna-seq|deseq2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17980" PostTypeId="1" AcceptedAnswerId="17986" CreationDate="2021-11-02T21:51:25.530" Score="0" ViewCount="1710" Body="&lt;p&gt;sorry for the question, I'm trying to extract the proteins sequences from a genbank file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; gene            complement(516466..532086)&#xA;                 /gene=&amp;quot;rtxA&amp;quot;&#xA;                 /locus_tag=&amp;quot;VV1_RS17390&amp;quot;&#xA;                 /old_locus_tag=&amp;quot;VV2_0479&amp;quot;&#xA; CDS             complement(516466..532086)&#xA;                 /gene=&amp;quot;rtxA&amp;quot;&#xA;                 /locus_tag=&amp;quot;VV1_RS17390&amp;quot;&#xA;                 /old_locus_tag=&amp;quot;VV2_0479&amp;quot;&#xA;                 /inference=&amp;quot;COORDINATES: similar to AA&#xA;                 sequence:RefSeq:WP_011081430.1&amp;quot;&#xA;                 /note=&amp;quot;Derived by automated computational analysis using&#xA;                 gene prediction method: Protein Homology.&amp;quot;&#xA;                 /codon_start=1&#xA;                 /transl_table=11&#xA;                 /product=&amp;quot;MARTX multifunctional-autoprocessing&#xA;                 repeats-in-toxin holotoxin RtxA&amp;quot;&#xA;                 /protein_id=&amp;quot;WP_011081430.1&amp;quot;&#xA;                 /translation=&amp;quot;MGKPFWRSVEYFFTGNYSADDGNNSIVAIGFGGEIHAYGGDDHV&#xA;                 TVGSIGATVYTGSGNDTVVGGSAYLRVEDTTGHLSVKGAAGYADINKSGDGNVSFAGA&#xA;                 AGGVSIDHLGNHGDVSYGGAAAYNGITRKGLSGNVTFKGAGGYNALWHETNQGNLSFA&#xA;                 GAGAGNKLDRTWFNRYQGSRGDVTFDGAGAANSISSRVETGNITFRGAGADNHLVRKG&#xA;                 KEGNHTANLANEDISSANGYHSMGKGGYSLSDLHYSVNAVRSTSETVADIDEYTDQTL&#xA;                 FKPATDSGESSGDVRFNGAGGGNVIKSNVTRGNVYFNGGGIANVILHSSQFGHTEFNG&#xA;                 GGAANVIVKSGEEGDLTFRGAGLANVLVHQSKQGKMDVYAGGAVNVLVRIGDGQYLAH&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm using a code from a previous answer to extract the protein and the locus_tag, but I want some extra features like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;locus_tag|old_locus_tag|gene|product|complement|protein_id|chromosome|Accession|organism|strain&#xA;MGKPFWRSVEYFFTGNYSADDG ..........&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;the result will be like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;VV1_RS17390|VV2_0479|rtxA|MARTX multifunctional-autoprocessing repeats-in-toxin holotoxin RtxA&amp;quot;|516466..532086|WP_011081430.1|Chromosome-II|NC_004459.3|Vibrio vulnificus|CMCP6&#xA;MGKPFWRSVEYFFTGNYSADDG ..........&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;this is the code that I'm using, how to add those features, or an other way using R ???&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;file_name = 'CMCP6.gb'&#xA;# stores all the CDS entries&#xA;all_entries = []&#xA;with open(file_name, 'r') as GBFile:&#xA;    GBcds = SeqIO.InsdcIO.GenBankCdsFeatureIterator(GBFile)&#xA;&#xA;    for cds in GBcds:&#xA;        if cds.seq is not None:&#xA;            cds.id = cds.name&#xA;            cds.description = ''&#xA;            all_entries.append(cds)&#xA;&#xA;&#xA;# write file&#xA;SeqIO.write(all_entries, '{}.fasta'.format(file_name[:-3]), 'fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But the result from the code only add the locus_tag and the sequence:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;VV1_RS17380&#xA;MFIKNLSIGKKIAAAFSIIAVINIAFGIFLSTELNTVKSELLNYTEDTLPAMEKVDAVRD&#xA;KISYWRRTQFAVFAMNDENQIKQTITRNEGIRREIETELAAYGKSVWPGEEEQTYNRLMS&#xA;LWSGYLSTMDKFNDALLAGDKDAAYPILTNSLSTFESIETEVNKLVMILKGAMDSNKNQI&#xA;LSSVNGLNTTAVISNIAIFAVMVIMTLLLTRIICGPLNIVVRQANAIARGDLSQDLDRNA&#xA;IGNDELGELADATIKMQDDLRQVIDNVIAAVTQLSSAVEEMTQISEMSASGMKDQQMQVT&#xA;LVATAMTEMKAAVADVARNTEDSASQAYDANRRTQDGAKETHQMVVSIEEVADIIAKAGD&#xA;TVAELEAQSNQINVVVDVIRGIADQTNLLALNAAIEAARAGESGRGFAVVADEVRTLAGR&#xA;TQDSTGEITSIIEKLQDLAKQAKHATEDSRTSIGACVEQGNNAQELMGSIEKSIANIADM&#xA;GAQIATACGQQDSVAEELSRNIENIHMASQEVAQGSQQTAQACRELTQLSVSLQDVMSRF&#xA;KLN&#xA;&amp;gt;VV1_RS17385&#xA;MNFKKTLLSIAIASASLTPAFSYSAPLLLDNTVHQTSQIAGANAWLEISLGQFKSNIEQF&#xA;KSHIAPQTKICAVMKADAYGNGIRGLMPTILEQQIPCVAIASNAEAKLVRESGFEGELIR&#xA;VRSASTSEIEQALSLDIEELIGSEQQARELASLAEKYSKTIKVHLALNDGGMGRNGIDMS&#xA;TERGPKEAVAIATHPSVAVVGIMTHFPNYNAEDVRTKLKSFNQHAQWLMESAGLKREEIT&#xA;LHVANSYTALNVPEAQLDMVRPGGVLYGDLPTNPEYPSIVAFKTRVASLHSLPAGSTVGY&#xA;DSTFTTANDAVMANLTVGYSDGYPRKMGNKAQVLINGQRANVVGVASMNTTMVDVSNIKG&#xA;VLPGDEVTLFGAQKNQHISVGEMEENAEVIFPELYTIWGTSNPRFYVK&#xA;&amp;gt;VV1_RS17390&#xA;MGKPFWRSVEYFFTGNYSADDGNNSIVAIGFGGEIHAYGGDDHVTVGSIGATVYTGSGND&#xA;TVVGGSAYLRVEDTTGHLSVKGAAGYADINKSGDGNVSFAGAAGGVSIDHLGNHGDVSYG&#xA;GAAAYNGITRKGLSGNVTFKGAGGYNALWHETNQGNLSFAGAGAGNKLDRTWFNRYQGSR&#xA;GDVTFDGAGAANSISSRVETGNITFRGAGADNHLVRKGKVGDITLQGAGASNRIERTRQA&#xA;EDVYAQTRGNIRFEGVGGYNSLYSDVAHGDIHFSGGGAYNTITRKGSGSSFDAQGMEYAK&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thanks so much&lt;/p&gt;&#xA;" OwnerUserId="14140" LastEditorUserId="10049" LastEditDate="2023-01-24T11:50:19.100" LastActivityDate="2023-01-24T11:50:19.100" Title="How to extract the protein sequences of a genbank file using R or biopython" Tags="|r|python|biopython|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17982" PostTypeId="2" ParentId="17979" CreationDate="2021-11-03T06:07:38.290" Score="1" Body="&lt;p&gt;DESeq2 works for that.  Unless you literally have only two samples.&lt;/p&gt;&#xA;&lt;p&gt;There is &lt;strong&gt;no&lt;/strong&gt; statistically sound way of analyzing DE genes with literally only two samples.  This is a bad experimental design, suitable for only the roughest of exploratory purposes.  Only the most obvious differences (that you probably knew already) are likely to be borne out in a legitimate experiment.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastEditorUserId="73" LastEditDate="2021-12-04T06:09:26.820" LastActivityDate="2021-12-04T06:09:26.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17983" PostTypeId="1" AcceptedAnswerId="17997" CreationDate="2021-11-03T11:10:06.627" Score="1" ViewCount="72" Body="&lt;p&gt;I've been trying without success to phase the variants of about 30 samples in a 5Mb non-recombining region. I have a phased genome assembly of the region but so far I have not been successful in using that to inform the phasing of the samples which have been sequenced to high coverage with short reads. Non-reference panel phasing using Beagle 5.2 and Shapeit4 have produced haplotypes that are composed of many small phased blocks which switch phase between them, far to many to manually correct.&lt;/p&gt;&#xA;&lt;p&gt;I have created a &amp;quot;reference panel&amp;quot; of SNPs of known phase for the region and want to use it to guide the rest of the phasing of the non-&amp;quot;reference panel&amp;quot; SNPs but every method I have found only uses the intersection of the reference panel and the samples for analysis. Is there some technique that I can use to phase the genotypes in the samples? I believe this is a tractable problem since recombination is suppressed in this region so haplotype blocks should be large and contiguous.&lt;/p&gt;&#xA;" OwnerUserId="14509" LastActivityDate="2021-11-07T06:49:46.280" Title="Using a phased reference assembly to phase sample genotypes" Tags="|haplotypes|phasing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17984" PostTypeId="1" AcceptedAnswerId="17985" CreationDate="2021-11-03T12:12:40.680" Score="3" ViewCount="179" Body="&lt;p&gt;What would be the total size of all PDB files in RCSB?&lt;/p&gt;&#xA;&lt;p&gt;Can I traverse/enumerate them one at a time using a Python script without downloading all of them to my local disk?&lt;/p&gt;&#xA;&lt;p&gt;If YES, what would be the Python script like?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="3848" LastEditDate="2023-01-24T00:37:54.770" LastActivityDate="2023-01-24T00:37:54.770" Title="Can I traverse/enumerate PDB one at a time using a Python script without downloading all of them to my local disk?" Tags="|python|pdb|biopython|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17985" PostTypeId="2" ParentId="17984" CreationDate="2021-11-03T14:49:23.830" Score="6" Body="&lt;p&gt;The page &lt;a href=&quot;https://www.wwpdb.org/ftp/pdb-ftp-sites&quot; rel=&quot;noreferrer&quot;&gt;https://www.wwpdb.org/ftp/pdb-ftp-sites&lt;/a&gt; gives the addresses of interest. PDB has an FTP site downloadable via rsync and the latter has a dry run mode.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rsync --port=33444 -rlpt -v -z --dry-run --stats rsync.rcsb.org::ftp_data/structures/divided/pdb/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You get a bunch of folders names and the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Number of files: 182479&#xA;Number of files transferred: 0&#xA;Total file size: 38382243431 bytes&#xA;Total transferred file size: 0 bytes&#xA;Literal data: 0 bytes&#xA;Matched data: 0 bytes&#xA;File list size: 3777395&#xA;File list generation time: 52.909 seconds&#xA;File list transfer time: 0.000 seconds&#xA;Total bytes sent: 16&#xA;Total bytes received: 3777651&#xA;&#xA;sent 16 bytes  received 3777651 bytes  66861.36 bytes/sec&#xA;total size is 38382243431  speedup is 10160.30&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So there are 182,479 *.ent.gz in various folders. The total is 38,382,243,431 bytes, which is 38 GB, so not too much. In Python it is easy to iterate across gzip files with the &lt;a href=&quot;https://docs.python.org/3/library/gzip.html&quot; rel=&quot;noreferrer&quot;&gt;gzip module&lt;/a&gt;, so uncompressing them is generally a thing to avoid as it will bloat up greatly. The problem arise when dealing with the density maps, which do take up a lot of space...&lt;/p&gt;&#xA;&lt;h2&gt;Download without saving&lt;/h2&gt;&#xA;&lt;p&gt;The PDBe is easier to for downloading things than the RCSB PDB and the API are much more comprehensive.&lt;/p&gt;&#xA;&lt;p&gt;Given a list of PDB codes one could&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pdbcode : str = ''&#xA;&#xA;import requests&#xA;r = requests.get(f'https://www.ebi.ac.uk/pdbe/entry-files/download/pdb{pdbcode.lower()}.ent', stream=True)&#xA;assert r.status_code == 200, f'something ({r.status_code} is wrong: {r.text}'&#xA;pdb_code = r.text&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2021-11-04T08:47:33.540" LastActivityDate="2021-11-04T08:47:33.540" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="17986" PostTypeId="2" ParentId="17980" CreationDate="2021-11-04T09:20:37.383" Score="3" Body="&lt;p&gt;ok tried a little bit, not sure is the most elegant/fastest way of doing it&lt;/p&gt;&#xA;&lt;p&gt;starting from file from : &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NC_004460.2?report=gbwithparts&amp;amp;log$=seqview&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/NC_004460.2?report=gbwithparts&amp;amp;log$=seqview&lt;/a&gt;        {GenBank(Full)}&lt;/p&gt;&#xA;&lt;p&gt;here my code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;from Bio.Seq import Seq&#xA;from Bio.SeqRecord import SeqRecord&#xA;&#xA;file_name = 'CMCP6.gb'&#xA;&#xA;&#xA;# stores all the CDS entries&#xA;all_entries = []&#xA;&#xA;&#xA;gb_record = SeqIO.read(open(file_name,&amp;quot;r&amp;quot;), &amp;quot;genbank&amp;quot;)&#xA;&#xA;&#xA;gb_feature = gb_record.features&#xA;&#xA;&#xA;accession = gb_record.annotations['accessions'][0]+'.'+str(gb_record.annotations['sequence_version'])&#xA;&#xA;# accession = gb_record.annotations['accessions']&#xA;# print(gb_record.annotations['accessions'])&#xA;&#xA;for seq_feature in gb_feature:&#xA;    if seq_feature.type==&amp;quot;source&amp;quot;:&#xA;            organism = str(seq_feature.qualifiers['organism'][0]).rsplit(&amp;quot; &amp;quot;, 1)[0]&#xA;            strain = seq_feature.qualifiers['strain'][0]&#xA;            chromosome = seq_feature.qualifiers['chromosome'][0]&#xA;&#xA;&#xA;&#xA;for seq_feature in gb_feature:&#xA;    if seq_feature.type==&amp;quot;CDS&amp;quot;:&#xA;            if 'translation' in seq_feature.qualifiers:&#xA;                if seq_feature.qualifiers['translation'][0] != '':&#xA;                    if 'gene' in seq_feature.qualifiers:&#xA;                        gene = seq_feature.qualifiers['gene'][0]&#xA;                &#xA;                    else:&#xA;                    &#xA;                        gene = 'not defined'&#xA;                    &#xA;                    if 'product' in seq_feature.qualifiers:&#xA;                        product = seq_feature.qualifiers['product'][0]&#xA;                    &#xA;                    else:&#xA;&#xA;                        product = 'not defined'&#xA;                        &#xA;                    if 'protein_id' in seq_feature.qualifiers:&#xA;                          protein_id = seq_feature.qualifiers['protein_id'][0]&#xA;                     &#xA;                    else:&#xA;&#xA;                          protein_id = 'not defined'&#xA;                         &#xA;                         &#xA;                    if 'old_locus_tag' in seq_feature.qualifiers:&#xA;                          old_locus_tag = seq_feature.qualifiers['old_locus_tag'][0]&#xA;                     &#xA;                    else:&#xA;&#xA;                          old_locus_tag = 'not defined'&#xA;                         &#xA;                    if 'locus_tag' in seq_feature.qualifiers:&#xA;                          locus_tag = seq_feature.qualifiers['locus_tag'][0]&#xA;                     &#xA;                    else:&#xA;&#xA;                          locus_tag = 'not defined'&#xA;            &#xA;                    if seq_feature.location.strand == 1:&#xA;                          complement = 'no complement'&#xA;&#xA;                    if seq_feature.location.strand == -1:&#xA;                        complement = str(seq_feature.location).strip('[]').split('(-)')[0]&#xA;                        &#xA;                        &#xA;                        &#xA;                    pippo = SeqRecord(&#xA;                                Seq(seq_feature.qualifiers['translation'][0]),&#xA;                                &#xA;                                id = seq_feature.qualifiers['locus_tag'][0] ,&#xA;                                &#xA;                                description = ('|'+str(old_locus_tag)+'|'+str(gene)+'|'+str(product)\&#xA;                                                +'|'+str(complement)+'|'+str(protein_id) +'|Chromosome-'+str(chromosome)\&#xA;                                                +'|'+str(accession).strip(&amp;quot;[]&amp;quot;).replace(&amp;quot;'&amp;quot;, &amp;quot;&amp;quot;)+'|'+str(organism)\&#xA;                                                    +'|'+str(strain))&#xA;                                )&#xA;                    all_entries.append(pippo)&#xA;                                &#xA;# print(all_entries)&#xA;# write file&#xA;SeqIO.write(all_entries, '{}.fasta'.format(file_name[:-3]), 'fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output for your example line is :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;VV1_RS17390 |VV2_0479|rtxA|MARTX multifunctional-autoprocessing repeats-in-toxin holotoxin RtxA|516465:532086]|WP_011081430.1|Chromosome-II|NC_004460.2|Vibrio vulnificus|CMCP6&#xA;MGKPFWRSVEYFFTGNYSADDGNNSIVAIGFGGEIHAYGGDDHVTVGSIGATVYTGSGND.....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I got extra space between locus_tag and | , I dont know how to get rid of it&lt;/p&gt;&#xA;&lt;p&gt;Its not clear to me what you mean by Accessions, tried two ways (second one is past # in code) and got &lt;code&gt;NC_004460.2&lt;/code&gt; instead of &lt;code&gt;NC_004459.3&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="10049" LastActivityDate="2021-11-04T09:20:37.383" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17987" PostTypeId="2" ParentId="11336" CreationDate="2021-11-04T15:05:15.530" Score="1" Body="&lt;p&gt;There is a trade off between read length and the limit of the indel that can be accurately mapped. Intermediate-size indels specifically are better detected with longer read lengths.&lt;/p&gt;&#xA;" OwnerUserId="14514" LastActivityDate="2021-11-04T15:05:15.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17988" PostTypeId="1" AcceptedAnswerId="18014" CreationDate="2021-11-04T18:12:50.713" Score="1" ViewCount="99" Body="&lt;p&gt;I will appreciate if you can please clarify some of my doubts about drug-DNA complex.&lt;/p&gt;&#xA;&lt;p&gt;I want to study the drug-DNA simulation using AMBER. I did go through all the tutorials video on youtube but everything is about protein-ligand using GROMACS. I can also use GROMACS if that gives me the results.&lt;/p&gt;&#xA;&lt;p&gt;My DNA sequences very small (12-mer) and I want to to intercalate it with a drug molecule.&#xA;I generated the DNA sequence in Chimera. My questions are as follows.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it possible to insert my drug at specific sequence and then do the binding energy calculation using AMBER?&#xA;(1) How can I insert it and make a complex?&#xA;(2) If I have to create a complex in AMBER how can I do that---reference/tutorials link will be     great.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The publications I read, they downloaded the PDB file with the drug molecule and extracted the  crystallographic information where the drug is attached and then they extended the sequence from there using NAB tools in AMBER. Can somebody please explain how can we do that.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thank you so much for your time.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I want to intercalate the drug molecule at the desired sequence I want and calculate the binding energy.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14439" LastActivityDate="2021-11-09T12:16:18.300" Title="building complex drug-dna for AMBER software" Tags="|phylogenetics|docking|cancer|simulated-data|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17990" PostTypeId="1" CreationDate="2021-11-05T07:48:00.993" Score="1" ViewCount="76" Body="&lt;p&gt;I have a long list of NCBI taxonomic ids for organisms and I want to write a script to get their corresponding translation tables. Is there a way to do this through Entrez?&lt;/p&gt;&#xA;&lt;p&gt;(packages in R, Python etc. would be fine too)&lt;/p&gt;&#xA;" OwnerUserId="14517" LastActivityDate="2022-12-04T18:37:56.807" Title="Getting translation table of organisms through Entrez direct (or other CLIs)" Tags="|phylogenetics|entrez|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17992" PostTypeId="1" CreationDate="2021-11-05T14:34:49.257" Score="1" ViewCount="65" Body="&lt;p&gt;I think Eagle or haploSep will do the job of reconstructing haplotype data, but I'm not sure what the 'standard' or best practice tool to use is.&lt;/p&gt;&#xA;&lt;p&gt;I have a VCF/PLINK format file of ~8,000 individuals with ~150,000 SNPs each (from across a population with several sub-populations).&lt;/p&gt;&#xA;&lt;p&gt;I want to use the resulting data to place new samples into one or more populations (e.g. ancestry analysis).&lt;/p&gt;&#xA;&lt;p&gt;Before I go trying to make one of these tools work / look at accuracy, etc., am I doing it right in the first place?&lt;/p&gt;&#xA;" OwnerUserId="8875" LastEditorUserId="3694" LastEditDate="2022-12-02T11:40:59.520" LastActivityDate="2022-12-02T11:40:59.520" Title="Given SNP-Chip data for a population, what tool should I use to reconstruct haplotypes?" Tags="|snp|phylogenetics|haplotypes|phasing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="17993" PostTypeId="2" ParentId="17990" CreationDate="2021-11-05T14:37:46.273" Score="0" Body="&lt;p&gt;You can find them here at least:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Taxonomy/Utils/wprintgc.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/Taxonomy/Utils/wprintgc.cgi&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can download the complete list of codes (using FTP) &lt;a href=&quot;ftp://ftp.ncbi.nih.gov/entrez/misc/data/gc.prt&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, to link a specific organism to a specific codon usage table, you'll need to do a bit of extra work. Perhaps the &lt;a href=&quot;https://github.com/sleyn/get_taxonomy_by_NCBI_TaxID&quot; rel=&quot;nofollow noreferrer&quot;&gt;script here&lt;/a&gt; can give you a starting point?&lt;/p&gt;&#xA;" OwnerUserId="8875" LastEditorUserId="8875" LastEditDate="2021-11-08T12:14:48.660" LastActivityDate="2021-11-08T12:14:48.660" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17994" PostTypeId="2" ParentId="17992" CreationDate="2021-11-05T15:57:29.493" Score="1" Body="&lt;p&gt;HaploSep uses allele frequency data - I presume you have genotype data.&lt;/p&gt;&#xA;&lt;p&gt;In that case, I would suggest looking at either &lt;a href=&quot;https://faculty.washington.edu/browning/beagle/beagle.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Beagle5.2&lt;/a&gt;, &lt;a href=&quot;https://odelaneau.github.io/shapeit4/&quot; rel=&quot;nofollow noreferrer&quot;&gt;shapeit4&lt;/a&gt; and &lt;a href=&quot;https://alkesgroup.broadinstitute.org/Eagle/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eagle2&lt;/a&gt;. I believe they more or less give the same accuracy (Eagle2 is slightly less accurate than the other two) and Beagle 5.2 is faster than the rest. Depending on what you plan on doing with the haplotypes, I don't think that the choice of phasing algorithm will affect downstream results a great deal.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2021-11-05T15:57:29.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17995" PostTypeId="2" ParentId="17797" CreationDate="2021-11-05T17:59:23.960" Score="0" Body="&lt;p&gt;Assuming your files are small enough to allow you to fit one of them into memory, you can do this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ awk '{ &#xA;        if(/^&amp;gt;/){ name=$0 }&#xA;        else{ &#xA;            if(NR==FNR){seq[name]=$0 } &#xA;            else{&#xA;                print name; &#xA;                printf &amp;quot;%s%s\n&amp;quot;, seq[name],$0&#xA;            }&#xA;        }&#xA;      }' file1.fa file2.fa &#xA;&amp;gt;1721_1 gene name&#xA;TGA  &#xA;&amp;gt;10_14 gene name&#xA;GTTGAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you cannot fit the files in memory, you can try this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ join -o1.1,1.2,2.2 -t $'\t' &amp;lt;(FastaToTbl file1.fa ) &amp;lt;(FastaToTbl file2.fa) | sed 's/\t//2' | TblToFasta &#xA;&amp;gt;1721_1 gene name &#xA;ATGTGA&#xA;&amp;gt;10_14 gene name &#xA;GTTGAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that this approach requires the &lt;code&gt;FastaToTbl&lt;/code&gt; and &lt;code&gt;TblToFasta&lt;/code&gt; scripts I have posted here: &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2651/298&quot;&gt;https://bioinformatics.stackexchange.com/a/2651/298&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Or, if brevity is more your thing, the same approach but more compact:&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2021-11-05T17:59:23.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17996" PostTypeId="1" CreationDate="2021-11-05T19:14:08.787" Score="1" ViewCount="54" Body="&lt;p&gt;As you can see in the image, the Kmer distribution field is empty: &lt;strong&gt;&amp;quot;No options available&amp;quot;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;What am I missing?&lt;/p&gt;&#xA;&lt;p&gt;Is there another tool I must use first to generate a Kmer distribution?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/u7RpL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/u7RpL.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9373" LastActivityDate="2021-11-05T19:14:08.787" Title="In Galaxy, using the bracken tool, how do I get a Kmer distribution?" Tags="|galaxy|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="17997" PostTypeId="2" ParentId="17983" CreationDate="2021-11-07T02:44:34.687" Score="0" Body="&lt;p&gt;The fact that the reference assembly is phased doesn't do anything to help actually phase the 30 other samples. There is no way of knowing what the combination of variants on single haplotypes are in the 30 samples from the one reference genome + 30-sample VCF alone. With this info, it is possible to come up with a measure of &amp;quot;HaplotypeA from sample 1 is more similar to HaplotypeII than HaplotypeI from my phased reference&amp;quot;, though.&lt;/p&gt;&#xA;&lt;p&gt;To actually phase the 30 other samples it will take long reads and/or proximity ligation data to phase the variants over a 5Mb region. Hi-C, Chicago, or DNAse Hi-C/Dovetail Omni-C reads will be necessary to correctly phase all of the variants over a region that is as large as 5Mbp.&lt;/p&gt;&#xA;&lt;p&gt;I recommend using the tool &lt;a href=&quot;https://github.com/vibansal/HapCUT2&quot; rel=&quot;nofollow noreferrer&quot;&gt;HapCUT2&lt;/a&gt; to use the reads to phase the variants already called for each sample. When mapping the reads to your genome, only map the reads to a fasta file that contains a single haploid copy of the genome (2 homologous chromosomes/haplotypes  represented by 1 scaffold), not a fasta file that contains both haplotypes from the phased assembly (2 homologous chromosomes/haplotypes represented by 2 scaffolds).&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2021-11-07T06:49:46.280" LastActivityDate="2021-11-07T06:49:46.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="17998" PostTypeId="2" ParentId="17975" CreationDate="2021-11-07T03:31:30.897" Score="1" Body="&lt;p&gt;I was going to write some clever code counting &lt;em&gt;k-mer&lt;/em&gt;s, but this problem can easily be solved using the handy sequentially-operating feature of python's str.replace() method.&lt;/p&gt;&#xA;&lt;p&gt;This code doesn't work if you need to look at the motif occurring on the other strand. Just the forward strand.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;mystring = &amp;quot;CCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCC\&#xA;TCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCT\&#xA;CCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCCTCTCCTCCCTCTCCCTCT&amp;quot;&#xA;&#xA;motif = &amp;quot;CCCTCT&amp;quot;&#xA;&#xA;# This replaces the motif with 1s&#xA;newstring = mystring.replace(motif, &amp;quot;1&amp;quot;)&#xA;# results in 1111111111111111111111111111111111111CCT11&#xA;# silly, right?&#xA;&#xA;# This whole block just parses that string and prints everything&#xA;#  the way that OP wanted.&#xA;done = False&#xA;space=&amp;quot;&amp;quot;&#xA;accumulator = 0&#xA;while not done:&#xA;    if len(newstring) == 0:&#xA;        done = True&#xA;        break&#xA;    else:&#xA;        if newstring[0] == &amp;quot;1&amp;quot;:&#xA;            accumulator += 1&#xA;        else:&#xA;            if accumulator != 0:&#xA;                print(&amp;quot;{}({}){} &amp;quot;.format(space, motif, accumulator), end = &amp;quot;&amp;quot;)&#xA;                accumulator = 0&#xA;            print(newstring[0], end = &amp;quot;&amp;quot;)&#xA;            space = &amp;quot; &amp;quot;&#xA;    newstring = newstring[1::]&#xA;&#xA;# This is our check after exiting the while loop.&#xA;#  This only fires if we stepped off cleanly at the&#xA;#  end of a motif.&#xA;if accumulator &amp;gt; 0:&#xA;    print(&amp;quot;{}({}){} &amp;quot;.format(space, motif, accumulator), end = &amp;quot;&amp;quot;)&#xA;print()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Prints, &lt;code&gt;(CCCTCT)37 CCT (CCCTCT)2&lt;/code&gt;, matching OP's spec.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2021-11-07T03:31:30.897" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="17999" PostTypeId="2" ParentId="15292" CreationDate="2021-11-07T05:35:19.920" Score="2" Body="&lt;p&gt;No, that is not quite right. specifically, this part:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;And by &amp;quot;hit&amp;quot; I mean an aligned sequence fragment that includes that coordinate.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;To get data on how many reads contributed to calling a specific allele at a given position, one would use the DP field, for example, which isn't included in the Kaviar VCF.&lt;/p&gt;&#xA;&lt;p&gt;The AC, AN, and AF fields in a VCF file are meant to be used to show the allele's frequency in the context of all of the samples used when making that VCF file.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Investigation process&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is one line from the Kaviar vcf file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1       10230   rs200279319     AC      AA,A    .       .       AF=0.0002654,0.0048525;AC=7,128;AN=26378;END=10231&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If we look at the README attached with the Kaviar vcf file, it says,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Kaviar is a compilation of SNVs, indels, and comlex variants observed in humans, designed to facilitate testing for the novelty and frequency of observed variants.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How many humans though? If we go to the Kaviar website it says:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Kaviar contains 162 million SNV sites (including 25M not in dbSNP) and incorporates data from 35 projects encompassing 77,781 individuals (13.2K whole genome, 64.6K exome).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If we look at the Kaviar VCF header we see that the fields are defined:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##INFO=&amp;lt;ID=AF,Number=A,Type=Float,Description=&amp;quot;Allele Frequency&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=AC,Number=A,Type=Integer,Description=&amp;quot;Allele Count&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=AN,Number=1,Type=Integer,Description=&amp;quot;Total number of alleles in data sources&amp;quot;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If we look at these fields again, &lt;code&gt;AF=0.0002654,0.0048525;AC=7,128;AN=26378&lt;/code&gt;, the AN=26378 tells us that for this locus, the Kaviar dataset contains information from 26378/2=13189 humans. That's the 13.2K whole genomes the Kaviar website was talking about. This site I'm looking at must be a non-coding region, because there is no exome data (protein-coding regions).&lt;/p&gt;&#xA;&lt;p&gt;For this same site, AC=7,128 means that out of 13189 humans (26378 haplotypes), we saw the variants 7 and 128 times. Out of 26378 haplotypes, this is a frequency of AF=0.0002654,0.0048525.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2021-11-07T06:51:46.557" LastActivityDate="2021-11-07T06:51:46.557" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18000" PostTypeId="2" ParentId="11512" CreationDate="2021-11-07T06:35:27.907" Score="0" Body="&lt;p&gt;Yes, it looks odd, but you might need to assemble the sequences to tell what is happening. The fact that the peaks are symmetrical around the presumably &lt;em&gt;Aspergillus&lt;/em&gt; peak around 50% GC is strange. Usually contamination appears as one peak or shoulder on either side of the host DNA peak rather than symmetrical peaks.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2021-11-07T06:35:27.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18001" PostTypeId="1" CreationDate="2021-11-07T11:21:47.363" Score="1" ViewCount="47" Body="&lt;p&gt;I'm doing an exercise that asks for two files:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Input 1: &lt;code&gt;A target file&lt;/code&gt; (.bed format) contains multiple regions from&#xA;&lt;code&gt;chr7:40000000-50000000&lt;/code&gt; of human reference genome GRCh37 (hg19)&lt;/li&gt;&#xA;&lt;li&gt;Input 2: &lt;code&gt;Refseq exon list file&lt;/code&gt; (.bed format) for all human coding genes&#xA;(hg19 position)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The final goal is:&lt;/p&gt;&#xA;&lt;p&gt;For all genes located in &lt;code&gt;chr7:40000000-50000000&lt;/code&gt;, get the summary statistics of the target file coverage. (For each gene, get the fraction of exonic bases that was covered by the target file).&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I believe what they refer in this exercise is that the target file&#xA;should be something like the whole chr sizes of hg19 &lt;code&gt;hg19.chr.sizes&lt;/code&gt; and &lt;code&gt;refseq_exon_list&lt;/code&gt; the list of exons from reqseq database. Both can be&#xA;downloaded from tools like &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTables?hgsid=1207958047_1HLrWSX8jM6O6nujrnQsrG0p1Da8&quot; rel=&quot;nofollow noreferrer&quot;&gt;table browser&lt;/a&gt; .Is that correct ?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I'm not sure which files I should download here to perform this task.&#xA;Once downloaded the file I believe what I need to do is to restrict&#xA;the &lt;code&gt;refseq_exon_list&lt;/code&gt; by the requested region and then perform&#xA;coverage with something like &lt;code&gt;bedtools&lt;/code&gt;. Something on those lines&#xA;&lt;code&gt;bedtools coverage -a hg19.chr.sizes -b reqseq_exon_list&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Am i right here ? Any input is appreciated, thanks.&lt;/p&gt;&#xA;" OwnerUserId="10309" LastActivityDate="2021-11-07T17:54:45.847" Title="Understanding exercise on file coverage with question on summary statistics" Tags="|bedtools|data-download|coverage|refseq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18002" PostTypeId="2" ParentId="18001" CreationDate="2021-11-07T15:49:43.830" Score="0" Body="&lt;p&gt;This sounds like homework - can you post what you have tried so far + a few lines of the input files + your code? Also, please add the &amp;quot;homework&amp;quot; tag if this is homework, or a task from a learning exercise.&lt;/p&gt;&#xA;&lt;p&gt;Without giving it away think about the information in your &lt;code&gt;Refseq exon list file&lt;/code&gt;. Is it a 3-field bed file with just coordinates, &lt;a href=&quot;https://en.wikipedia.org/wiki/BED_(file_format)&quot; rel=&quot;nofollow noreferrer&quot;&gt;or is it the expanded file format with fields 4-12?&lt;/a&gt; Do you really need to download the RefSeq annotation, or can you parse the information in your bed file to answer your question?&lt;/p&gt;&#xA;&lt;p&gt;It sounds like this is your ultimate goal:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For each gene, get the fraction of exonic bases that was covered by the target file.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What information from the &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/coverage.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;output of &lt;code&gt;bedtools coverage&lt;/code&gt;&lt;/a&gt; will give you this answer? Does this information exactly match the spec you provided? Are there other tools in &lt;code&gt;bedtools&lt;/code&gt; you could use to achieve the same thing?&lt;/p&gt;&#xA;&lt;p&gt;Side note: It isn't 100% clear what you are asking and what your inputs are. I think the policy here isn't to outright answer homework questions without seeing evidence of some attempts. Someone can provide a better answer if you can update your question.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastActivityDate="2021-11-07T15:49:43.830" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18003" PostTypeId="2" ParentId="17905" CreationDate="2021-11-07T17:03:58.743" Score="2" Body="&lt;p&gt;Surprisingly there aren't actually many helpful one-liners to map Illumina WGS reads on the internet - everyone just sort of assumes that people can figure it out from the manual. Many people end up doing it in a not-so-ideal way, in that they stop at the &lt;code&gt;.sam&lt;/code&gt; files that &lt;code&gt;bwa mem&lt;/code&gt; makes that take up a lot of disk space.&lt;/p&gt;&#xA;&lt;p&gt;I like this incantation:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# index the reference&#xA;bwa index reference_genome.fasta&#xA;&#xA;# map your reads to the reference (bwa mem)&#xA;# compress that output to a bam file (samtools view -hb)&#xA;# sort the bam file (samtools sort)&#xA;&#xA;bwa mem -t &amp;lt;number_of_threads&amp;gt; reference_genome.fasta your_sample.R1.fastq.gz your_sample.R2.fastq.gz | \&#xA;samtools view -hb - | \&#xA;samtools sort - &amp;gt; your_reads_to_reference.sorted.bam&#xA;&#xA;# index the bamfile. makes a .bai file&#xA;&#xA;samtools index your_reads_to_reference.sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With this sorted &lt;code&gt;.bam&lt;/code&gt;, the &lt;code&gt;.bai&lt;/code&gt;, and the reference genome you can now use &lt;a href=&quot;https://software.broadinstitute.org/software/igv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IGV&lt;/a&gt; to directly look at the reads mapped to your genome.&lt;/p&gt;&#xA;&lt;p&gt;Also with the bam, you can generate a VCF file using &lt;a href=&quot;https://github.com/freebayes/freebayes&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;freebayes&lt;/code&gt;&lt;/a&gt; or another tool like &lt;a href=&quot;https://www.biostars.org/p/63429/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;samtools mpileup&lt;/code&gt;&lt;/a&gt;. This will give you a file of how your sample is different from the reference genome. You can use &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/intersect.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;bedtools intersect&lt;/code&gt;&lt;/a&gt; to get the variants only from your gene/regions of choice.&lt;/p&gt;&#xA;" OwnerUserId="2085" LastEditorUserId="2085" LastEditDate="2021-11-21T00:50:50.803" LastActivityDate="2021-11-21T00:50:50.803" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="18004" PostTypeId="2" ParentId="18001" CreationDate="2021-11-07T17:54:45.847" Score="1" Body="&lt;p&gt;Based on the description in your question, I don't think this assumption is correct:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I believe what they refer in this exercise is that the target file should be something like the whole chr sizes of hg19 hg19.chr.sizes [. . .]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I understand that the question is asking you to simply produce a random selection of sub-regions from chromosome 7, all of which fall within the larger region 40000000-50000000. It can't be the list of chromosome sizes since, first of all, the question is asking you to get regions from chr17 only, and second because that just wouldn't make sense: only chr17 is relevant here, so the rest would be useless and, in any case, the length of the chromosome itself isn't helpful information.&lt;/p&gt;&#xA;&lt;p&gt;The idea here is to demonstrate that given a set of genomic regions and a list of start and end coordinates of exons, you can calculate how many of your exons overlap the target regions. Therefore, the steps to follow would be:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Generate random intervals in the 40000000-50000000 range.&lt;/li&gt;&#xA;&lt;li&gt;Get the bed file for all human coding genes on hg19. You should indeed be able to download this from the UCSC genome browser, among other places.&lt;/li&gt;&#xA;&lt;li&gt;Use a tool, &lt;code&gt;bedtools&lt;/code&gt; or build your own using &lt;code&gt;awk&lt;/code&gt;, or anything else that takes your fancy and for each gene found in the chr7:40000000-50000000 range, calculate how many bases of its exons fall within your randomly chosen selection of regions.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Of course, the best thing to do would be to ask your teacher to clarify.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2021-11-07T17:54:45.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18005" PostTypeId="1" CreationDate="2021-11-08T04:44:45.293" Score="2" ViewCount="154" Body="&lt;p&gt;I have the following GFF3 file which I would like to convert to &lt;a href=&quot;https://github.com/KorfLab/SNAP#parameter-estimation&quot; rel=&quot;nofollow noreferrer&quot;&gt;ZFF&lt;/a&gt; file with the below script:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ head -n 20 traingenes.gff3 &#xA;##gff-version 3&#xA;# gffread v0.12.7&#xA;# gffread -E traingenes.gtf -o traingenes.gff3&#xA;NbLab330C00 GeneMark.hmm3   mRNA    74501   76501   .   +   .   ID=1_t;geneID=1_g&#xA;NbLab330C00 GeneMark.hmm3   CDS 74501   74512   .   +   0   Parent=1_t&#xA;NbLab330C00 GeneMark.hmm3   CDS 75568   75999   .   +   0   Parent=1_t&#xA;NbLab330C00 GeneMark.hmm3   CDS 76151   76501   .   +   0   Parent=1_t&#xA;NbLab330C00 GeneMark.hmm3   mRNA    76535   77079   .   +   .   ID=2_t;geneID=2_g&#xA;NbLab330C00 GeneMark.hmm3   CDS 76535   76639   .   +   0   Parent=2_t&#xA;NbLab330C00 GeneMark.hmm3   CDS 76702   77079   .   +   0   Parent=2_t&#xA;NbLab330C00 GeneMark.hmm3   mRNA    93763   100703  .   -   .   ID=3_t;geneID=3_g&#xA;NbLab330C00 GeneMark.hmm3   CDS 93763   93837   .   -   0   Parent=3_t&#xA;NbLab330C00 GeneMark.hmm3   CDS 93915   94031   .   -   0   Parent=3_t&#xA;NbLab330C00 GeneMark.hmm3   CDS 94351   94430   .   -   2   Parent=3_t&#xA;NbLab330C00 GeneMark.hmm3   CDS 95483   95589   .   -   1   Parent=3_t&#xA;NbLab330C00 GeneMark.hmm3   CDS 95697   95746   .   -   0   Parent=3_t&#xA;NbLab330C00 GeneMark.hmm3   CDS 100464  100703  .   -   0   Parent=3_t&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env perl&#xA;use strict;&#xA;# &#xA;# source: https://biowize.wordpress.com/2012/06/01/training-the-snap-ab-initio-gene-predictor/ &#xA;#&#xA;my @exons;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$gene_count = 0;&#xA;my $&lt;/span&gt;current_seq = &amp;quot;&amp;quot;;&#xA;while (my &lt;span class=&quot;math-container&quot;&gt;$line = &amp;lt;STDIN&amp;gt;) {&#xA;  if ($line =~ m/^###/) {&#xA;    flush(\@exons);&#xA;    next;&#xA;  }&#xA;  my @fields = split(/\t/, $line);&#xA;  if ($fields[2] eq &quot;mRNA&quot;) {&#xA;    flush(\@exons);&#xA;  } elsif ($fields[2] eq &quot;exon&quot;) {&#xA;    if ($fields[0] ne $current_seq) {&#xA;      $current_seq = $fields[0];&#xA;      printf(&quot;&amp;gt;%s\n&quot;, $&lt;/span&gt;current_seq);&#xA;    }&#xA;    push(@exons, \@fields);&#xA;  }&#xA;}&#xA;flush();&#xA; &#xA;sub flush&#xA;{&#xA;  my &lt;span class=&quot;math-container&quot;&gt;$num_exons = scalar(@exons);&#xA;  return if($&lt;/span&gt;num_exons == 0);&#xA;   &#xA;  my &lt;span class=&quot;math-container&quot;&gt;$group = sprintf(&quot;%s.%d&quot;, $&lt;/span&gt;exons[0]-&amp;gt;[0], &lt;span class=&quot;math-container&quot;&gt;$gene_count);&#xA;  $&lt;/span&gt;gene_count++;&#xA;   &#xA;  if (&lt;span class=&quot;math-container&quot;&gt;$num_exons == 1) {&#xA;my($start, $end) = ($exons[0]-&amp;gt;[3], $exons[0]-&amp;gt;[4]);&#xA;if ($exons[0]-&amp;gt;[6] eq &quot;-&quot;) {&#xA;  ($start, $end) = ($exons[0]-&amp;gt;[4], $exons[0]-&amp;gt;[3]);&#xA;}&#xA;printf(&quot;Esngl\t%lu\t%lu\t%s\n&quot;, $start, $end, $group);&#xA;  } else {&#xA;@exons = reverse(@exons) if($exons[0]-&amp;gt;[6] eq &quot;-&quot;);&#xA;for (my $i = 0; $i &amp;lt; $num_exons; $i++) {&#xA;  my $exon_type = &quot;Exon&quot;;&#xA;  if ($i == 0) {&#xA;    $exon_type = &quot;Einit&quot;;&#xA;  } elsif ($i == $num_exons - 1) {&#xA;    $&lt;/span&gt;exon_type = &amp;quot;Eterm&amp;quot;;&#xA;      }&#xA;       &#xA;      my(&lt;span class=&quot;math-container&quot;&gt;$start, $&lt;/span&gt;end) = (&lt;span class=&quot;math-container&quot;&gt;$exons[$&lt;/span&gt;i]-&amp;gt;[3], &lt;span class=&quot;math-container&quot;&gt;$exons[$&lt;/span&gt;i]-&amp;gt;[4]);&#xA;      if (&lt;span class=&quot;math-container&quot;&gt;$exons[0]-&amp;gt;[6] eq &quot;-&quot;) {&#xA;    ($start, $end) = ($exons[$i]-&amp;gt;[4], $exons[$&lt;/span&gt;i]-&amp;gt;[3]);&#xA;      }&#xA;       &#xA;      printf(&amp;quot;%s\t%lu\t%lu\t%s\n&amp;quot;, &lt;span class=&quot;math-container&quot;&gt;$exon_type, $&lt;/span&gt;start, &lt;span class=&quot;math-container&quot;&gt;$end, $&lt;/span&gt;group);&#xA;    }&#xA;  }&#xA;  @exons = ();&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, when I run it like &lt;code&gt;perl gff3_to_zff.pl &amp;lt; traingenes.gff3 &amp;gt;&amp;gt; traingenes.ann &lt;/code&gt; an empty file get generated.&lt;/p&gt;&#xA;&lt;p&gt;What did I miss?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance,&lt;/p&gt;&#xA;&lt;p&gt;Michal&lt;/p&gt;&#xA;" OwnerUserId="2477" LastEditorUserId="298" LastEditDate="2021-11-15T17:04:44.207" LastActivityDate="2023-08-07T19:03:45.527" Title="Perl convert GFF3 file" Tags="|perl|gff3|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18006" PostTypeId="1" AcceptedAnswerId="18026" CreationDate="2021-11-08T11:06:42.357" Score="0" ViewCount="83" Body="&lt;p&gt;I'd like to perform multiple alignments, where a gene from each sample was read in from fasta files. The fasta file represented one sample and had multiple genes. I have read in each sample fasta file and now have a dictionary of genes and their samples and sequences.&#xA;Here is a small example.&lt;/p&gt;&#xA;&lt;p&gt;The main key is the gene ID for another dictionary as the value. In the dictionary 'R??' is the sample name and key,&#xA;followed by the annotation and sequence in a list.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{'PRELSG_MIT_v1:5894-6032': {'R47': ['not annotated',&#xA;                                     Seq('ATAAAAATAGTTACCATAGCTGTTGATGGATGCTTCGATCATAAAATATGTTTG...GGA')],&#xA;                             'R48': ['not annotated',&#xA;                                     Seq('ATAAAAATAGTTACCATAGCTGTTGATGGATGCTTCGATCATAAAATATGTTTG...GGA')],&#xA;                             'R49': ['not annotated',&#xA;                                     Seq('ATAAAAATAGTTACCATAGCTGTTGATGGATGCTTCGATCATAAAATATGTTTG...GGA')],&#xA;                             'R50': ['not annotated',&#xA;                                     Seq('ATAAAAATAGTTACCATAGCTGTTGATGGATGCTTCGATCATAAAATATGTTTG...GGA')],&#xA;                             'R51': ['not annotated',&#xA;                                     Seq('ATAAAAATAGTTACCATAGCTGTTGATGGATGCTTCGATCATAAAATATGTTTG...GGA')],&#xA;                             'R52': ['not annotated',&#xA;                                     Seq('ATAAAAATAGTTACCATAGCTGTTGATGGATGCTTCGATCATAAAATATGTTTG...GGA')]},&#xA; 'exon_PRELSG_MIT00300.1-E1': {'R47': ['cytochrome b',&#xA;                                       Seq('TCCAGATAATGCTATTACAGTAGATAGATATGCTACTCCTTTACATATTGTTCC...TTC')],&#xA;                               'R48': ['cytochrome b',&#xA;                                       Seq('TCCAGATAATGCTATTACAGTAGATAGATATGCTACTCCTTTACATATTGTTCC...TTC')],&#xA;                               'R49': ['cytochrome b',&#xA;                                       Seq('TCCAGATAATGCTATTACAGTAGATAGATATGCTACTCCTTTACATATTGTTCC...TTC')],&#xA;                               'R50': ['cytochrome b',&#xA;                                       Seq('TCCAGATAATGCTATTACAGTAGATAGATATGCTACTCCTTTACATATTGTTCC...TTC')],&#xA;                               'R51': ['cytochrome b',&#xA;                                       Seq('TCCAGATAATGCTATTACAGTAGATAGATATGCTACTCCTTTACATATTGTTCC...TTC')],&#xA;                               'R52': ['cytochrome b',&#xA;                                       Seq('TCCAGATAATGCTATTACAGTAGATAGATATGCTACTCCTTTACATATTGTTCC...TTC')]}}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'd like to align each gene and write out those alignments to file without any intermediate files written to disk.&#xA;How do I do that?&lt;/p&gt;&#xA;" OwnerUserId="10553" LastEditorUserId="10553" LastEditDate="2021-11-08T12:06:50.757" LastActivityDate="2021-11-11T10:23:06.013" Title="Use biopython to align SeqRecords stored in dict" Tags="|python|multiple-sequence-alignment|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18007" PostTypeId="2" ParentId="17984" CreationDate="2021-11-08T12:11:57.843" Score="0" Body="&lt;p&gt;Not Python, but you can do this with &lt;a href=&quot;https://biojulia.net/BioStructures.jl/stable&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioStructures.jl&lt;/a&gt; in Julia:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;using BioStructures&#xA;pdblist = pdbentrylist()&#xA;for pdbid in pdblist&#xA;    # Download a file and delete it after use&#xA;    downloadpdb(pdbid, format=MMCIF) do fp&#xA;        s = read(fp, MMCIF)&#xA;        # Do something with the file, e.g. print the cysteine fraction&#xA;        nres = countresidues(s, standardselector)&#xA;        if nres &amp;gt; 0&#xA;            frac = countresidues(s, standardselector, x -&amp;gt; resname(x) == &amp;quot;CYS&amp;quot;) / nres&#xA;            println(pdbid, &amp;quot;  &amp;quot;, round(frac; digits=2))&#xA;        end&#xA;    end&#xA;end&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you want to get all the PDB entries you should use the mmCIF format, though it does lead to larger file sizes.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2021-11-08T12:11:57.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18008" PostTypeId="2" ParentId="18006" CreationDate="2021-11-08T12:19:23.333" Score="0" Body="&lt;p&gt;This &lt;a href=&quot;https://biopython.org/docs/1.75/api/Bio.Align.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioPython document&lt;/a&gt; should get you started.&lt;/p&gt;&#xA;&lt;p&gt;A specific example is given &lt;a href=&quot;https://stackoverflow.com/questions/45581807/muscle-multiple-sequence-alignment-with-biopython&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="8875" LastActivityDate="2021-11-08T12:19:23.333" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="18009" PostTypeId="2" ParentId="18005" CreationDate="2021-11-08T12:23:08.967" Score="1" Body="&lt;p&gt;Your file looks like it is &lt;code&gt;CDS&lt;/code&gt; feature based, not &lt;code&gt;exon&lt;/code&gt; feature based, which &lt;em&gt;seems&lt;/em&gt; to be what the script expects.&lt;/p&gt;&#xA;&lt;p&gt;There is a tool to convert CDS based GFF to exon based that I've used successfully in the past...&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Could be &lt;a href=&quot;https://ccb.jhu.edu/software/stringtie/gff.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;GFF utilities&lt;/a&gt;?&lt;/li&gt;&#xA;&lt;li&gt;Actually, I think it's &lt;a href=&quot;http://genometools.org/tools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GenomeTools&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="8875" LastActivityDate="2021-11-08T12:23:08.967" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18010" PostTypeId="2" ParentId="17978" CreationDate="2021-11-08T12:32:28.993" Score="1" Body="&lt;p&gt;First focus on calling SNPs accurately. See the &lt;a href=&quot;http://bio-bwa.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BWA&lt;/a&gt;, &lt;a href=&quot;https://www.htslib.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAMtools&lt;/a&gt;, &lt;a href=&quot;https://samtools.github.io/bcftools/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BCFtools&lt;/a&gt; pipeline (as &lt;a href=&quot;https://www.ebi.ac.uk/sites/ebi.ac.uk/files/content.ebi.ac.uk/materials/2014/140217_AgriOmics/dan_bolser_snp_calling.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;described here&lt;/a&gt;) or &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATK4&lt;/a&gt; (as &lt;a href=&quot;https://gencore.bio.nyu.edu/variant-calling-pipeline-gatk4/&quot; rel=&quot;nofollow noreferrer&quot;&gt;described here&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Once you have the [BV]CFs, then just code up the analysis to see changes over generations.&lt;/p&gt;&#xA;" OwnerUserId="8875" LastActivityDate="2021-11-08T12:32:28.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18013" PostTypeId="2" ParentId="15248" CreationDate="2021-11-08T23:33:05.107" Score="2" Body="&lt;p&gt;The &lt;a href=&quot;https://github.com/higlass/clodius&quot; rel=&quot;nofollow noreferrer&quot;&gt;clodius&lt;/a&gt; tool puts tab-delimited data from a BED file into an &lt;a href=&quot;https://www.sqlite.org/rtree.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;rtree&lt;/a&gt; in a sqlite3 database:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/higlass/clodius/blob/c98bb16ade93402fcea3b749d705c52ea165b609/clodius/cli/aggregate.py#L457-L777&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/higlass/clodius/blob/c98bb16ade93402fcea3b749d705c52ea165b609/clodius/cli/aggregate.py#L457-L777&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Data are stored in this database at multiple &amp;quot;zoom levels&amp;quot;, which allows for fast queries at wide genomic extents:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/higlass/clodius/blob/develop/clodius/tiles/beddb.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/higlass/clodius/blob/develop/clodius/tiles/beddb.py&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Aggregation (input) and querying (output) seem efficient in practice, but efficiency may depend on aspects of your specific dataset, i.e., whether you mean efficiency in execution time of &lt;code&gt;INSERT&lt;/code&gt; or &lt;code&gt;SELECT&lt;/code&gt; statements, or in space efficiency in the overall database size.&lt;/p&gt;&#xA;&lt;p&gt;Other data can be converted to BED via &lt;code&gt;vcf2bed&lt;/code&gt;, &lt;code&gt;sam2bed&lt;/code&gt;, &lt;code&gt;gff2bed&lt;/code&gt;, etc. and then aggregated. This is not the only way to do things, of course, but hopefully this can suggest directions to explore.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2021-11-09T00:02:30.307" LastActivityDate="2021-11-09T00:02:30.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18014" PostTypeId="2" ParentId="17988" CreationDate="2021-11-09T12:16:18.300" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;the tutorials video on youtube&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Gromacs has great written documentation, so I'd suggest referring to that directly.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I can also use GROMACS if that gives me the results.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Gromacs works fine with a DNA-ligand system. There are different Amber forcefields, proteins are generally used with &lt;code&gt;AMBER99SB-ILDN&lt;/code&gt; simply because it has the longest name and is one of the inbuilt ones. Technically, the Amber forcefield tailored towards DNA is &lt;code&gt;OL15&lt;/code&gt; (recommended here) and &lt;code&gt;gaff2&lt;/code&gt; is for a generalised ff for ligands. These aren't natively within Gromacs, but been ported to Gromacs.&lt;/p&gt;&#xA;&lt;p&gt;Whereas pi-pi iterations are somewhat covered by the Lenard-Jones term, the orientation will be driven not only by dipole, but also quadrupole moments, which aren't covered by classical mechanics forcefields. Likewise S-pi and cation-pi interactions will not be covered. Most DNA intercalating small molecules have multiple fused rings, e.g. doxorubicin, that pi stack with the DNA. &lt;strong&gt;Consequently, a simulation under a classical mechanics forcefield is likely going to be gibberish.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is it possible to insert my drug at specific sequence&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;See your previous question and the easiest option of dragging it in PyMOL&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Can somebody please explain how can we do that.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Read the AmberTools21 documentation.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-11-09T12:16:18.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18016" PostTypeId="2" ParentId="18005" CreationDate="2021-11-09T14:41:16.827" Score="1" Body="&lt;p&gt;The easiest is to use &lt;code&gt;agat_convert_sp_gff2zff.pl&lt;/code&gt; from &lt;a href=&quot;https://github.com/NBISweden/AGAT&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGAT&lt;/a&gt;&lt;br /&gt;&#xA;(&lt;strong&gt;Edit&lt;/strong&gt; Actually you will have to first standardize your file because you do not have gene features, neither gene_id or Parent atttributes in your mRNA features. AGAT will be confused. So, in your case, prior the &lt;em&gt;gff2zff&lt;/em&gt; conversion you will have to run &lt;code&gt;agat_convert_sp_gxf2gxf.pl -c geneID --gff file.gff -o file_clean.gff&lt;/code&gt;)&lt;/p&gt;&#xA;&lt;p&gt;Otherwise you can use your script but you need to add the exon as suggested by @Dan.&lt;br /&gt;&#xA;To do so you can just replace the CDS by exon using a sed command, it should work.&lt;br /&gt;&#xA;You can also use &lt;code&gt;agat_convert_sp_gxf2gxf.pl&lt;/code&gt; (with parameter &lt;code&gt;-c geneID&lt;/code&gt;) it will add the gene and exon features.&lt;/p&gt;&#xA;" OwnerUserId="2486" LastEditorUserId="2486" LastEditDate="2021-11-09T14:47:00.663" LastActivityDate="2021-11-09T14:47:00.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18017" PostTypeId="2" ParentId="15248" CreationDate="2021-11-09T19:52:39.137" Score="1" Body="&lt;p&gt;You might also take a look at this genomics-focused sqlite extension: &lt;a href=&quot;https://github.com/mlin/GenomicSQLite&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mlin/GenomicSQLite&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As described in the repository README:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This SQLite3 loadable extension adds features to the ubiquitous&#xA;embedded RDBMS supporting applications in genome bioinformatics:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;genomic range indexing for overlap queries &amp;amp; joins&lt;/li&gt;&#xA;&lt;li&gt;in-SQL utility functions, e.g. reverse-complement DNA, parse &amp;quot;chr1:2,345-6,789&amp;quot;&lt;/li&gt;&#xA;&lt;li&gt;automatic streaming storage compression (also available standalone)&lt;/li&gt;&#xA;&lt;li&gt;reading directly from HTTP(S) URLs (also available standalone)&lt;/li&gt;&#xA;&lt;li&gt;pre-tuned settings for &amp;quot;big data&amp;quot;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="776" LastActivityDate="2021-11-09T19:52:39.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18018" PostTypeId="1" AcceptedAnswerId="18023" CreationDate="2021-11-09T19:59:26.597" Score="2" ViewCount="92" Body="&lt;p&gt;I'm looking for a little bit of guidance.&lt;/p&gt;&#xA;&lt;p&gt;My question is regarding the simulation of DNA sequences with a fix substitution rate.&lt;/p&gt;&#xA;&lt;p&gt;The majority of the programs for simulating sequences use Continuous Markov Chain Models, with different instantaneous rate matrices that can be modified according to the free parameters of each model (JC69, HKY, GTR, etc..). For example, in HKY the free parameters are the transition/transversion (kappa) rate and nucleotide frequencies (pi).&lt;/p&gt;&#xA;&lt;p&gt;In these programs (for example INDELible, Seq-Gen, Pyvolve), the substitution rate is expected to be represented by the branch length of the phylogenetic tree used to create the simulations.&lt;/p&gt;&#xA;&lt;p&gt;INDELible paper.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&amp;quot;..Rate matrices are rescaled by INDELible such that the branch lengths represent the expected number of substitutions per site (or the average expected number of substitutions per site under a heterogeneous-sites model).&amp;quot;&lt;/em&gt; (P2 - Simulation of substitutions)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/19423664/&quot; rel=&quot;nofollow noreferrer&quot;&gt;INDELible: a flexible simulator of biological sequence evolution&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&amp;quot;...Each branch length is assumed to denote the mean number of nucleotide substitutions per site that will be simulated along that branch...&amp;quot;&lt;/em&gt; (P2 - Algorithm)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/9183526/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seq-Gen: an application for the Monte Carlo simulation of DNA sequence evolution along phylogenetic trees&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let's say I want to simulate sequences with a mean substitution rate of 2.5 x 10^-7 per year. How would I prepare a tree that can accurately represent that substitution rate and be used in the simulation? In other words, how can the length be represented as a substitution rate accurately?&lt;/p&gt;&#xA;&lt;p&gt;Any insight is appreciated!.&#xA;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="6777" LastActivityDate="2021-11-10T20:43:11.443" Title="Simulation of DNA sequences through substitution rates" Tags="|phylogeny|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18020" PostTypeId="1" CreationDate="2021-11-09T21:39:24.833" Score="0" ViewCount="88" Body="&lt;p&gt;I'm trying to create a python program that follows the codon translation and turn mRNA sequence into amino acids (see the dictionary). The system has to return an error of the string is not a multiplier of 3, and the the function should not produce any more amino acids once a stop codon (keys with a value of X in the dictionary) is found, and should not display 'X' in the output. I do not know how to incorporate the stop codon to stop the programming entirely, as well as the multiplier of 3 aspect in the code. Can someone give me any advice? Thank you so much!&#xA;example: protfromRNA('GCCAACGCACCCACUUUUUUUUAGAAA') should return ANAPTFF&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def protfromRNA(Seq):&#xA;    GeneticCode = {'AAA':'K', 'AAC':'N', 'AAG':'K', 'AAU':'N',&#xA;               'ACA':'T', 'ACC':'T', 'ACG':'T', 'ACU':'T',&#xA;               'AGA':'R', 'AGC':'S', 'AGG':'R', 'AGU':'S',&#xA;               'AUA':'I', 'AUC':'I', 'AUG':'M', 'AUU':'I',&#xA;               'CAA':'Q', 'CAC':'H', 'CAG':'Q', 'CAU':'H',&#xA;               'CCA':'P', 'CCC':'P', 'CCG':'P', 'CCU':'P',&#xA;               'CGA':'R', 'CGC':'R', 'CGG':'R', 'CGU':'R',&#xA;               'CUA':'L', 'CUC':'L', 'CUG':'L', 'CUU':'L',&#xA;               'GAA':'E', 'GAC':'D', 'GAG':'E', 'GAU':'D',&#xA;               'GCA':'A', 'GCC':'A', 'GCG':'A', 'GCU':'A',&#xA;               'GGA':'G', 'GGC':'G', 'GGG':'G', 'GGU':'G',&#xA;               'GUA':'V', 'GUC':'V', 'GUG':'V', 'GUU':'V',&#xA;               'UAA':'X', 'UAC':'Y', 'UAG':'X', 'UAU':'Y',&#xA;               'UCA':'S', 'UCC':'S', 'UCG':'S', 'UCU':'S',&#xA;               'UGA':'X', 'UGC':'C', 'UGG':'W', 'UGU':'C',&#xA;               'UUA':'L', 'UUC':'F', 'UUG':'L', 'UUU':'F'}&#xA;    return ''.join(GeneticCode[i] if GeneticCode[i]!=X else '' for i in Seq)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14540" LastActivityDate="2021-11-09T22:23:24.283" Title="list comprehension and codon translation" Tags="|python|codon|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18021" PostTypeId="2" ParentId="18020" CreationDate="2021-11-09T22:23:24.283" Score="1" Body="&lt;p&gt;You need to iterate over the sequence 3 characters at a time and you need to stop when you hit the stop codon (or before for an invalid codon, e.g. of length 2). Here is a memory efficient implementation using iterators (requires python &amp;gt;= 3.8):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from itertools import islice&#xA;&#xA;def protfromRNA(Seq):&#xA;    it = iter(Seq)&#xA;    lst = []&#xA;    while codon := &amp;quot;&amp;quot;.join(islice(it, 3)):&#xA;        aa = GeneticCode.get(codon)&#xA;        if aa is None:&#xA;            raise ValueError(f&amp;quot;invalid codon: {codon}&amp;quot;)&#xA;        if aa == &amp;quot;X&amp;quot;:&#xA;            break&#xA;        lst.append(aa)&#xA;    return &amp;quot;&amp;quot;.join(lst)&#xA;&#xA;print(protfromRNA('GCCAACGCACCCACUUUUUUUUAGAAA'))&#xA;# ANAPTFF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastActivityDate="2021-11-09T22:23:24.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18023" PostTypeId="2" ParentId="18018" CreationDate="2021-11-10T11:26:02.120" Score="2" Body="&lt;h2&gt;UPDATE based on comment&lt;/h2&gt;&#xA;&lt;p&gt;Simplest answer: just rescale your substitution rate from years to units of branch length. (Stop thinking in terms of years.)&lt;/p&gt;&#xA;&lt;h2&gt;original answer&lt;/h2&gt;&#xA;&lt;p&gt;The short answer is that you have to calibrate the branch lengths of the tree to yield the substitution rates per year you want. (Leave the topology alone.)&lt;/p&gt;&#xA;&lt;p&gt;Tree branch lengths are usually agnostic to chronological time (e.g. years). Instead it is very focused on whatever the &lt;a href=&quot;https://en.wikipedia.org/wiki/Substitution_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;model used to construct the tree&lt;/a&gt; is. This can be simple substitution counts (e.g. Jukes-Cantor), or a sophisticated model that takes into account a lot of information about relative rates of different changes, possibly learned from the dataset itself (e.g. GTR).&lt;/p&gt;&#xA;&lt;p&gt;You yourself will have to decide what the conversion factor is between your branch lengths (presumably derived from such a substitution model or a &lt;a href=&quot;https://en.wikipedia.org/wiki/Distance_matrices_in_phylogeny&quot; rel=&quot;nofollow noreferrer&quot;&gt;distance metric&lt;/a&gt;) and units of chronological time. In some trees, it may already be calibrated to units of chronological time. But those will be rare.&lt;/p&gt;&#xA;&lt;p&gt;So what I'd do is to make sure that the branch lengths of the tree are of a length that will, on average, yield the number of substitutions that you expect based on how long (in years) a given branch on the tree is supposed to be. For example, if you have a tree of primates, then you want root to tip distances to be on the order of tens of millions of years, probably. If you have a tree of all bacteria, you want the root-to-tip distances to be on the order of billions of years.&lt;/p&gt;&#xA;&lt;p&gt;Note please that this is identical to re-scaling the substitution matrix to substitutions per year. What is crucial is that you know what you mean by &amp;quot;substitutions per year&amp;quot; and how you are defining that in terms of the tree.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2021-11-10T20:43:11.443" LastActivityDate="2021-11-10T20:43:11.443" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18024" PostTypeId="1" AcceptedAnswerId="18030" CreationDate="2021-11-11T00:47:56.080" Score="0" ViewCount="359" Body="&lt;p&gt;I am trying to check the quality of RNA-Seq data from Illumina using fastqc in snakemake in a conda environment. I get the error &amp;quot;Multiple run or shell keywords in rule run_fastqc&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Snakefile:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule run_fastqc:&#xA;    input:&#xA;        &amp;quot;/[RNA fastq file path]/well01_S001_L001_R1_001.fastq.gz&amp;quot;&#xA;    output:&#xA;        &amp;quot;./[out path]/well01_S001_L001_R1_001.fastqc.html&amp;quot;,&#xA;        &amp;quot;./[out path]/well01_S001_L001_R1_001.fastqc.zip&amp;quot;&#xA;    wrapper:&#xA;        &amp;quot;0.79.0/bio/fastqc&amp;quot;&#xA;    shell:&#xA;        &amp;quot;fastqc {input} --outdir=./[out path]/&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've also tried:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;    shell:&#xA;        &amp;quot;fastqc {input} -o ./[out path]/&amp;quot;&#xA;...&#xA;    shell:&#xA;        &amp;quot;fastqc -o ./[out path]/ {input}&amp;quot;&#xA;...&#xA;    shell:&#xA;        &amp;quot;fastqc {input}&amp;quot;&#xA;        &amp;quot;fastqc -o ./[out path]/&amp;quot;&#xA;...&#xA;    shell:&#xA;        &amp;quot;fastqc {input} &amp;gt; -o ./[out path]/&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;snakemake --cores 1 well01_S001_L001_R1_001.fastqc.html well01_S001_L001_R1_001.fastqc.zip&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SyntaxError in line 9 of /[path to Snakefile]/Snakefile:&#xA;Multiple run or shell keywords in rule run_fastqc. (Snakefile, line 9)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I run fastqc alone (not in snakemake), it works and I get the two output files. How can I get snakemake to work?&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your time and help!&lt;/p&gt;&#xA;" OwnerUserId="14547" LastEditorUserId="14547" LastEditDate="2021-11-12T00:05:36.033" LastActivityDate="2021-11-12T00:05:36.033" Title="Snakemake Fastqc: &quot;Multiple run or shell keywords in rule run_fastqc.&quot;" Tags="|rna-seq|python|snakemake|quality-control|fastqc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18025" PostTypeId="1" AcceptedAnswerId="18038" CreationDate="2021-11-11T07:06:15.367" Score="0" ViewCount="27" Body="&lt;p&gt;I have to peptide output sample output i would give here ..&lt;/p&gt;&#xA;&lt;p&gt;Sample A&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dput(head(B5_ILS_Mus_PeptideSummary))&#xA;structure(list(N = c(1, 1, 1, 1, 1, 1), Unused = c(47.67, 47.67, &#xA;47.67, 47.67, 47.67, 47.67), Total = c(47.67, 47.67, 47.67, 47.67, &#xA;47.67, 47.67), `%Cov` = c(98.0000019073486, 98.0000019073486, &#xA;98.0000019073486, 98.0000019073486, 98.0000019073486, 98.0000019073486&#xA;), `%Cov(50)` = c(86.0000014305115, 86.0000014305115, 86.0000014305115, &#xA;86.0000014305115, 86.0000014305115, 86.0000014305115), `%Cov(95)` = c(86.0000014305115, &#xA;86.0000014305115, 86.0000014305115, 86.0000014305115, 86.0000014305115, &#xA;86.0000014305115), Accessions = c(&amp;quot;sp|P20065|TYB4_MOUSE&amp;quot;, &amp;quot;sp|P20065|TYB4_MOUSE&amp;quot;, &#xA;&amp;quot;sp|P20065|TYB4_MOUSE&amp;quot;, &amp;quot;sp|P20065|TYB4_MOUSE&amp;quot;, &amp;quot;sp|P20065|TYB4_MOUSE&amp;quot;, &#xA;&amp;quot;sp|P20065|TYB4_MOUSE&amp;quot;), Names = c(&amp;quot;Thymosin beta-4 OS=Mus musculus OX=10090 GN=Tmsb4x PE=1 SV=1&amp;quot;, &#xA;&amp;quot;Thymosin beta-4 OS=Mus musculus OX=10090 GN=Tmsb4x PE=1 SV=1&amp;quot;, &#xA;&amp;quot;Thymosin beta-4 OS=Mus musculus OX=10090 GN=Tmsb4x PE=1 SV=1&amp;quot;, &#xA;&amp;quot;Thymosin beta-4 OS=Mus musculus OX=10090 GN=Tmsb4x PE=1 SV=1&amp;quot;, &#xA;&amp;quot;Thymosin beta-4 OS=Mus musculus OX=10090 GN=Tmsb4x PE=1 SV=1&amp;quot;, &#xA;&amp;quot;Thymosin beta-4 OS=Mus musculus OX=10090 GN=Tmsb4x PE=1 SV=1&amp;quot;&#xA;), Used = c(NA, NA, NA, NA, NA, NA), Annotation = c(NA, NA, NA, &#xA;NA, NA, NA), Contrib = c(2, 2, 2, 2, 2, 2), Conf = c(99.0000009536743, &#xA;58.2899987697601, 97.5199997425079, 99.0000009536743, 98.9300012588501, &#xA;73.7299978733063), Sequence = c(&amp;quot;EKNPLPSKETIEQEKQAGES&amp;quot;, &amp;quot;KKTETQEKNPLPSKETIEQEKQAGES&amp;quot;, &#xA;&amp;quot;KNPLPSKETIEQEKQAGES&amp;quot;, &amp;quot;KTETQEKNPLPSKETIEQEKQAGES&amp;quot;, &amp;quot;LKKTETQEKNPLPSKETIEQEKQAGES&amp;quot;, &#xA;&amp;quot;PLPSKETIEQEKQAGES&amp;quot;), Modifications = c(NA_character_, NA_character_, &#xA;NA_character_, NA_character_, NA_character_, NA_character_), &#xA;    ProteinModifications = c(NA_character_, NA_character_, NA_character_, &#xA;    NA_character_, NA_character_, NA_character_), Cleavages = c(NA, &#xA;    NA, NA, NA, NA, NA), dMass = c(-0.0625222995877266, 0.180465996265411, &#xA;    -0.00865099020302296, -0.0651467964053154, -0.107698999345303, &#xA;    -0.0186963994055986), `Obs MW` = c(2241.05004882813, 2956.67944335938, &#xA;    2112.06127929688, 2828.3388671875, 3069.47534179688, 1869.91320800781&#xA;    ), `Obs m/z` = c(748.0239, 740.1771, 529.0226, 708.092, 768.3761, &#xA;    624.3117), `Theor MW` = c(2241.11254882813, 2956.4990234375, &#xA;    2112.06982421875, 2828.40405273438, 3069.5830078125, 1869.93200683594&#xA;    ), `Theor m/z` = c(748.044738769531, 740.132019042969, 529.024719238281, &#xA;    708.108276367188, 768.403015136719, 624.317932128906), `Theor z` = c(3, &#xA;    4, 4, 4, 4, 3), Sc = c(15, 8, 11, 21, 12, 10), Spectrum = c(&amp;quot;1.1.1.372.12&amp;quot;, &#xA;    &amp;quot;1.1.1.339.6&amp;quot;, &amp;quot;1.1.1.358.3&amp;quot;, &amp;quot;1.1.1.355.16&amp;quot;, &amp;quot;1.1.1.346.14&amp;quot;, &#xA;    &amp;quot;1.1.1.384.8&amp;quot;), `Acq Time` = c(4.2116, 3.539133, 3.91795, &#xA;    3.867617, 3.68345, 4.45125), `Intensity (Peptide)` = c(3775.35, &#xA;    110.58, 4374.93, 7285.52, 3073.51, 5776.43), PrecursorIntensityAcquisition = c(3626.33, &#xA;    37.12, 4301.05, 7124.23, 2706.13, 5533.12), `Apex Time (Peptide)` = c(4.2, &#xA;    3.59, 3.91, 3.85, 3.67, 4.44), `Elution Peak Width (Peptide)` = c(0.1, &#xA;    0.06, 0.08, 0.1, 0.08, 0.08), MS2Counts = c(3077.868, 477.5326, &#xA;    2262.827, 4694.357, 1759.267, 2618.842)), row.names = c(NA, &#xA;-6L), class = c(&amp;quot;tbl_df&amp;quot;, &amp;quot;tbl&amp;quot;, &amp;quot;data.frame&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Sample B&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dput(head(B6_ILS_mus_PeptideSummary))&#xA;structure(list(N = c(1, 1, 1, 1, 1, 1), Unused = c(82.41, 82.41, &#xA;82.41, 82.41, 82.41, 82.41), Total = c(82.41, 82.41, 82.41, 82.41, &#xA;82.41, 82.41), `%Cov` = c(100, 100, 100, 100, 100, 100), `%Cov(50)` = c(87.9999995231628, &#xA;87.9999995231628, 87.9999995231628, 87.9999995231628, 87.9999995231628, &#xA;87.9999995231628), `%Cov(95)` = c(86.0000014305115, 86.0000014305115, &#xA;86.0000014305115, 86.0000014305115, 86.0000014305115, 86.0000014305115&#xA;), Accessions = c(&amp;quot;sp|P20065|TYB4_MOUSE&amp;quot;, &amp;quot;sp|P20065|TYB4_MOUSE&amp;quot;, &#xA;&amp;quot;sp|P20065|TYB4_MOUSE&amp;quot;, &amp;quot;sp|P20065|TYB4_MOUSE&amp;quot;, &amp;quot;sp|P20065|TYB4_MOUSE&amp;quot;, &#xA;&amp;quot;sp|P20065|TYB4_MOUSE&amp;quot;), Names = c(&amp;quot;Thymosin beta-4 OS=Mus musculus OX=10090 GN=Tmsb4x PE=1 SV=1&amp;quot;, &#xA;&amp;quot;Thymosin beta-4 OS=Mus musculus OX=10090 GN=Tmsb4x PE=1 SV=1&amp;quot;, &#xA;&amp;quot;Thymosin beta-4 OS=Mus musculus OX=10090 GN=Tmsb4x PE=1 SV=1&amp;quot;, &#xA;&amp;quot;Thymosin beta-4 OS=Mus musculus OX=10090 GN=Tmsb4x PE=1 SV=1&amp;quot;, &#xA;&amp;quot;Thymosin beta-4 OS=Mus musculus OX=10090 GN=Tmsb4x PE=1 SV=1&amp;quot;, &#xA;&amp;quot;Thymosin beta-4 OS=Mus musculus OX=10090 GN=Tmsb4x PE=1 SV=1&amp;quot;&#xA;), Used = c(NA, NA, NA, NA, NA, NA), Annotation = c(NA, NA, NA, &#xA;NA, NA, NA), Contrib = c(2, 2, 2, 2, 2, 2), Conf = c(99.0000009536743, &#xA;99.0000009536743, 99.0000009536743, 99.0000009536743, 94.4599986076355, &#xA;99.0000009536743), Sequence = c(&amp;quot;DKPDMAEIEKFDKSKLK&amp;quot;, &amp;quot;EKNPLPSKETIEQEKQAGES&amp;quot;, &#xA;&amp;quot;ETQEKNPLPSKETIEQEKQAGES&amp;quot;, &amp;quot;IEKFDKSKLKKTETQEKNPLPSKETIEQEKQAGES&amp;quot;, &#xA;&amp;quot;KKTETQEKNPLPSKETIEQEKQAGES&amp;quot;, &amp;quot;KNPLPSKETIEQEK&amp;quot;), Modifications = c(NA_character_, &#xA;NA_character_, NA_character_, NA_character_, NA_character_, NA_character_&#xA;), ProteinModifications = c(NA_character_, NA_character_, NA_character_, &#xA;NA_character_, NA_character_, NA_character_), Cleavages = c(NA, &#xA;NA, NA, NA, NA, NA), dMass = c(0.0129736997187138, -0.0542825013399124, &#xA;-0.0299271997064352, -0.107947997748852, -0.0191929005086422, &#xA;-0.0046052997931838), `Obs MW` = c(2021.06335449219, 2241.05834960938, &#xA;2599.2314453125, 4045.01391601563, 2956.47973632813, 1639.87341308594&#xA;), `Obs m/z` = c(506.2731, 748.0267, 650.8151, 810.0101, 592.3032, &#xA;547.6318), `Theor MW` = c(2021.05029296875, 2241.11254882813, &#xA;2599.26123046875, 4045.12182617188, 2956.4990234375, 1639.87805175781&#xA;), `Theor m/z` = c(506.269866943359, 748.044738769531, 650.822631835938, &#xA;810.031677246094, 592.307067871094, 547.63330078125), `Theor z` = c(4, &#xA;3, 4, 5, 5, 3), Sc = c(11, 12, 12, 26, 10, 10), Spectrum = c(&amp;quot;1.1.1.858.2&amp;quot;, &#xA;&amp;quot;1.1.1.621.5&amp;quot;, &amp;quot;1.1.1.640.9&amp;quot;, &amp;quot;1.1.1.643.4&amp;quot;, &amp;quot;1.1.1.597.3&amp;quot;, &amp;quot;1.1.1.604.4&amp;quot;&#xA;), `Acq Time` = c(8.74735, 4.069217, 4.43505, 4.4987, 3.615083, &#xA;3.7369), `Intensity (Peptide)` = c(1798.16, 2856.36, 1278.57, &#xA;5833.81, 758.71, 599.21), PrecursorIntensityAcquisition = c(1603.78, &#xA;443.55, 1104.72, 5265.07, 484.03, 423.02), `Apex Time (Peptide)` = c(8.72, &#xA;4.19, 4.39, 4.45, 3.55, 3.78), `Elution Peak Width (Peptide)` = c(0.14, &#xA;0.12, 0.12, 0.37, 0.1, 0.08), MS2Counts = c(456.3906, 313.663, &#xA;712.3438, 2748.186, 351.0362, 228.9387)), row.names = c(NA, -6L&#xA;), class = c(&amp;quot;tbl_df&amp;quot;, &amp;quot;tbl&amp;quot;, &amp;quot;data.frame&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Unlike genes expression where it can be intersected since i know the gene name or ensemble ID are consistent.&lt;/p&gt;&#xA;&lt;p&gt;Here My objective is how many of the sequence are common or unique to these two condition.&lt;/p&gt;&#xA;&lt;p&gt;In the &lt;strong&gt;accession&lt;/strong&gt; column many ID are same but the differences are in the sequence column.&lt;/p&gt;&#xA;&lt;p&gt;Is there any programmatic way of finding the elements that are common or different in both the conditions&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2021-11-12T18:03:34.393" Title="How to match peptide sequence from two different condition" Tags="|proteomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18026" PostTypeId="2" ParentId="18006" CreationDate="2021-11-11T10:23:06.013" Score="0" Body="&lt;p&gt;I don't believe this is possible. So I'm answering this to close it, since I don't have enough reputation to close it myself.&lt;/p&gt;&#xA;" OwnerUserId="10553" LastActivityDate="2021-11-11T10:23:06.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18027" PostTypeId="1" CreationDate="2021-11-11T13:10:19.530" Score="0" ViewCount="38" Body="&lt;p&gt;I have 7 &lt;code&gt;RefSeq&lt;/code&gt; accessions which I performed &lt;code&gt;multiple sequence aligment&lt;/code&gt; on it using &lt;code&gt;CLUSTALO&lt;/code&gt;. These are the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;NC_026431.1&#xA;NC_007363.1&#xA;NC_004907.1&#xA;NC_007377.1&#xA;NC_007367.1&#xA;NC_002016.1&#xA;NC_026427.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I input the fasta sequence of those 7 accessions on &lt;code&gt;CLUSTALO&lt;/code&gt; under this &lt;a href=&quot;https://www.ebi.ac.uk/Tools/msa/clustalo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;. I get the result in either the &lt;code&gt;msf&lt;/code&gt; format or &lt;code&gt;clustal&lt;/code&gt; format with characters. The result looks like the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CLUSTAL O(1.2.4) multiple sequence alignment&#xA;&#xA;&#xA;NC_002016.1      -------AGCGAAAGCAGGTAGATATTGAAAGATGAGTCTTCTAACCGAGGTCGAAACGT   53&#xA;NC_007377.1      -------AGCAAAAGCAGGTAGATATTGAAAGATGAGCCTTCTAACCGAGGTCGAAACGT   53&#xA;NC_007367.1      -------AGCAAAAGCAGGTAGATATTGAAAGATGAGCCTTCTAACCGAGGTCGAAACGT   53&#xA;NC_004907.1      GGGGAATTCCAAAAGCAGGTAGATATTGAAAGATGAGTCTTCTAACCGAGGTCGAAACGT   60&#xA;NC_026427.1      --------------------------------ATGAGTCTTCTAACCGAGGTCGAAACGT   28&#xA;NC_007363.1      -------AGCAAAAGCAGGTAGATATTGAAAAATGAGTCTTCTAACCGAGGTCGAAACGT   53&#xA;NC_026431.1      --------------------------------ATGAGTCTTCTAACCGAGGTCGAAACGT   28&#xA;                                                 ***** **********************&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm aware that the &lt;code&gt;*&lt;/code&gt; symbols means a highly conserved region and that's exatcly what I'm looking for , however I need the highly conserved regions relative to a specific accession, namely &lt;code&gt;NC_007367.1&lt;/code&gt;. I'm not sure how to do that. I'm also not sure what the &lt;code&gt;-&lt;/code&gt; represents.&lt;/p&gt;&#xA;" OwnerUserId="10309" LastActivityDate="2021-11-14T19:48:00.750" Title="find conservative regions relative to a specific accession" Tags="|multiple-sequence-alignment|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18028" PostTypeId="1" CreationDate="2021-11-11T16:29:08.707" Score="1" ViewCount="326" Body="&lt;p&gt;I got this error during execution, pls guide me.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;vep --cache -i xxxx.vcf -o xxxxx_new.vcf&#xA;&#xA;Possible precedence issue with control flow operator at /software/xxxx_pkgs/envs/ensemble-vep/lib/site_perl/5.26.2/Bio/DB/IndexedBase.pm line 805.&#xA;DBI connect('host=ensembldb.ensembl.org;port=3306','anonymous',...) failed: Can't connect to MySQL server on 'ensembldb.ensembl.org' (110) at /software/xxxx_pkgs/envs/ensemble-vep/share/ensembl-vep-88.9-1/Bio/EnsEMBL/Registry.pm line 1771.&#xA;&#xA;-------------------- EXCEPTION --------------------&#xA;MSG: Cannot connect to the Ensembl MySQL server at ensembldb.ensembl.org:3306; check your settings &amp;amp; DBI error message: Can't connect to MySQL server on 'ensembldb.ensembl.org' (110)&#xA;STACK Bio::EnsEMBL::Registry::load_registry_from_db /software/xxxx_pkgs/envs/ensemble-vep/share/ensembl-vep-88.9-1/Bio/EnsEMBL/Registry.pm:1771&#xA;STACK Bio::EnsEMBL::VEP::BaseVEP::registry /software/xxxx_pkgs/envs/ensemble-vep/share/ensembl-vep-88.9-1/modules/Bio/EnsEMBL/VEP/BaseVEP.pm:163&#xA;STACK Bio::EnsEMBL::VEP::BaseRunner::setup_db_connection /software/xxxx_pkgs/envs/ensemble-vep/share/ensembl-vep-88.9-1/modules/Bio/EnsEMBL/VEP/BaseRunner.pm:80&#xA;STACK Bio::EnsEMBL::VEP::Runner::init /software/xxxx_pkgs/envs/ensemble-vep/share/ensembl-vep-88.9-1/modules/Bio/EnsEMBL/VEP/Runner.pm:72&#xA;STACK Bio::EnsEMBL::VEP::Runner::run /software/xxxx_pkgs/envs/ensemble-vep/share/ensembl-vep-88.9-1/modules/Bio/EnsEMBL/VEP/Runner.pm:111&#xA;STACK toplevel /software//envs/ensemble-vep/bin/vep:209&#xA;Ensembl API version = 88&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14551" LastEditorUserId="650" LastEditDate="2021-11-11T19:13:28.950" LastActivityDate="2021-11-12T04:15:36.827" Title="Ensembl Variant Effect Predictor (VEP) issue during execution" Tags="|vcf|sequence-annotation|ensembl|errors|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18030" PostTypeId="2" ParentId="18024" CreationDate="2021-11-11T19:16:57.397" Score="1" Body="&lt;p&gt;From a quick look at the manual, it looks like you are supposed to use just the &lt;code&gt;wrapper&lt;/code&gt;, not combine the &lt;code&gt;wrapper&lt;/code&gt; with a &lt;code&gt;shell&lt;/code&gt;. Can you show me an example of such combined usage?&lt;/p&gt;&#xA;&lt;p&gt;The error seems to suggest that the wrapper already has a &lt;code&gt;shell&lt;/code&gt; directive that you &lt;code&gt;shell&lt;/code&gt; conflicts with.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2021-11-11T19:16:57.397" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18031" PostTypeId="1" CreationDate="2021-11-11T23:42:12.360" Score="1" ViewCount="431" Body="&lt;p&gt;I am trying to run Fastqc on multiple files using Snakemake but am receiving an ambiguous SyntaxError past the last line of code in the Snakefile.&lt;/p&gt;&#xA;&lt;p&gt;Snakefile:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;SAMPLE_TABLE=pd.read_csv('./SampleTable.txt',sep='\t')&#xA;SAMPLES=SAMPLE_TABLE[&amp;quot;sample_name&amp;quot;]&#xA;SAMPLES=list(SAMPLES)&#xA;DIRECTORY=list(SAMPLE_TABLE[&amp;quot;FolderName&amp;quot;])&#xA;&#xA;rule run_fastqc:&#xA;    input:&#xA;        expand(&amp;quot;/PathToInputsFolder/{dir}/{sample}_{{Rnumber}}_001.fastq.gz&amp;quot;, zip, dir=DIRECTORY, sample=SAMPLES)&#xA;    output:&#xA;        expand(&amp;quot;{sample}_{{Rnumber}}_001.fastqc.html&amp;quot;, sample=SAMPLES)&#xA;        expand(&amp;quot;{sample}_{{Rnumber}}_001.fastqc.zip&amp;quot;, sample=SAMPLES)&#xA;    shell:&#xA;        &amp;quot;fastqc {input} --outdir=./qc/&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;snakemake --cores 1 {.+}.fastqc.html {.+}.fastqc.zip&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SyntaxError in line 16 of /PathToSnakefile/Snakefile:&#xA;invalid syntax (Snakefile, line 16)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How should I figure out where the error is? Thank you for your time!&lt;/p&gt;&#xA;" OwnerUserId="14547" LastActivityDate="2021-11-16T19:16:22.277" Title="SyntaxError after end of Snakefile code?" Tags="|rna-seq|python|snakemake|quality-control|fastqc|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18032" PostTypeId="2" ParentId="18028" CreationDate="2021-11-12T04:15:36.827" Score="2" Body="&lt;p&gt;You might prefer to run &lt;a href=&quot;https://m.ensembl.org/info/docs/tools/vep/script/vep_options.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;VEP&lt;/a&gt; in offline mode:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Using &lt;code&gt;--cache&lt;/code&gt; (without &lt;code&gt;--offline&lt;/code&gt;) uses the local cache on disk to&#xA;fetch most annotations, but allows database connections for some&#xA;features (see &lt;a href=&quot;https://m.ensembl.org/info/docs/tools/vep/script/vep_cache.html#limitations&quot; rel=&quot;nofollow noreferrer&quot;&gt;cache limitations&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If you need HGVS names in your output, you'll also need to supply a FASTA file using &lt;code&gt;--fasta&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://m.ensembl.org/info/docs/tools/vep/script/vep_options.html#opt_offline&quot; rel=&quot;nofollow noreferrer&quot;&gt;--offline&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Enable offline mode. No database connections will be made, and a cache file or GFF/GTF file is required for annotation. Add&#xA;--refseq to use the refseq cache (if installed). Not used by default.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://m.ensembl.org/info/docs/tools/vep/script/vep_options.html#opt_fasta&quot; rel=&quot;nofollow noreferrer&quot;&gt;--fasta [file|dir]&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Specify a FASTA file or a directory containing FASTA files to use to look up reference sequence. The first time you run VEP with this parameter an index will be built which can take a few minutes. This is required if fetching HGVS annotations (--hgvs) or checking reference sequences (--check_ref) in offline mode (--offline), and optional with some performance increase in cache mode (--cache). See documentation for more details. Not used by default.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2021-11-12T04:15:36.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18033" PostTypeId="2" ParentId="17879" CreationDate="2021-11-12T07:43:19.423" Score="1" Body="&lt;p&gt;I met the same problem. R after v4.1.1 seem to have problems with &lt;code&gt;repr_html&lt;/code&gt;.&#xA;I fixed the problem by following the quotes.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This tiny commit solves a long standing problem while trying to read help from inside jupyterlab.&lt;/p&gt;&#xA;&lt;p&gt;In order not to wait for the PR to be merged, you may just install from @renkun-ken 's fork:&#xA;&lt;code&gt;devtools::install_github(&amp;quot;renkun-ken/repr&amp;quot;, ref = &amp;quot;fix-html-filter&amp;quot;)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h4&gt;Reference&lt;/h4&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/IRkernel/repr/pull/144#event-5589475824&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/IRkernel/repr/pull/144#event-5589475824&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/IRkernel/repr/issues/143&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/IRkernel/repr/issues/143&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14553" LastActivityDate="2021-11-12T07:43:19.423" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18034" PostTypeId="1" CreationDate="2021-11-12T11:09:05.170" Score="0" ViewCount="146" Body="&lt;p&gt;I want to look at an RNA-seq data-set on 1 condition to look for expressed genes. As there is only one condition, I can’t perform differential analysis.&lt;/p&gt;&#xA;&lt;p&gt;The only thing I can think to do is set a threshold (i.e. all replicates must have &amp;gt; 10 gene counts) then say these are expressed. But I don’t think this is a very good way of doing it.&lt;/p&gt;&#xA;&lt;p&gt;Do you know of any better ways that we can do this?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="3694" LastEditDate="2022-10-09T17:47:57.597" LastActivityDate="2022-10-09T17:47:57.597" Title="Gene expression changes in one condition" Tags="|rna-seq|differential-expression|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18035" PostTypeId="2" ParentId="18031" CreationDate="2021-11-12T11:25:54.257" Score="2" Body="&lt;p&gt;How to figure out what error messages are trying to tell you is one of the great mysteries… :-)&lt;/p&gt;&#xA;&lt;p&gt;What version of Snakemake are you using? I get slightly different messages. With an ancient version (3.13.3) on one machine:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SyntaxError in line 14 of /path/to/Snakefile:&#xA;invalid syntax&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With a more up-to-date version (6.9.1, running on Python 3.9):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SyntaxError in line 16 of /path/to/Snakefile:&#xA;invalid syntax. Perhaps you forgot a comma? (Snakefile, line 16)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Both of these are useful hints. So one piece of advice would be to update your Snakemake and/or Python if it is old.&lt;/p&gt;&#xA;&lt;p&gt;Another useful technique when faced with a mysterious error like this is to temporarily remove parts of your &lt;em&gt;Snakefile&lt;/em&gt; until the error goes away. The complicated part of your script towards the end of the file is the &lt;code&gt;output:&lt;/code&gt; stanza, so you might try removing one of the &lt;code&gt;expand(…)&lt;/code&gt; lines and seeing what happens. If the error goes away, then you have localised the problem and can put the line back and carefully check the syntax in that particular part of the &lt;em&gt;Snakefile&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="134" LastActivityDate="2021-11-12T11:25:54.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18036" PostTypeId="1" CreationDate="2021-11-12T12:34:00.290" Score="2" ViewCount="145" Body="&lt;p&gt;I failed to install bioperl on debian in the following way:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ sudo cpanm bioperl&#xA;! Finding bioperl on cpanmetadb failed.&#xA;! Finding bioperl () on mirror http://www.cpan.org failed.&#xA;! Couldn't find module or a distribution bioperl&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thank you in advance,&lt;/p&gt;&#xA;" OwnerUserId="2477" LastEditorUserId="6251" LastEditDate="2021-11-14T10:24:18.470" LastActivityDate="2021-11-14T10:24:18.470" Title="Finding bioperl on cpanmetadb failed" Tags="|software-installation|perl|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18037" PostTypeId="2" ParentId="18031" CreationDate="2021-11-12T14:01:20.677" Score="1" Body="&lt;p&gt;&lt;strong&gt;You're missing a comma between the two &lt;code&gt;output&lt;/code&gt; lines.&lt;/strong&gt;  (John Marshall's more detailed syntax error message was spot-on.)&lt;/p&gt;&#xA;&lt;p&gt;With a simplified version of your Snakefile:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SAMPLES=[&amp;quot;s1&amp;quot;, &amp;quot;s2&amp;quot;, &amp;quot;s3&amp;quot;]&#xA;rule run_fastqc:&#xA;    output:&#xA;        expand(&amp;quot;{sample}_{{Rnumber}}_001.fastqc.html&amp;quot;, sample=SAMPLES)&#xA;        expand(&amp;quot;{sample}_{{Rnumber}}_001.fastqc.zip&amp;quot;, sample=SAMPLES)&#xA;    shell:&#xA;        &amp;quot;echo {output}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You get the syntax error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ snakemake -nrp s1_R1_001.fastqc.html&#xA;SyntaxError in line 6 of /data/home/jesse/Snakefile:&#xA;invalid syntax (Snakefile, line 6)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With the comma:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SAMPLES=[&amp;quot;s1&amp;quot;, &amp;quot;s2&amp;quot;, &amp;quot;s3&amp;quot;]&#xA;rule run_fastqc:&#xA;    output:&#xA;        expand(&amp;quot;{sample}_{{Rnumber}}_001.fastqc.html&amp;quot;, sample=SAMPLES), # &amp;lt;-- here&#xA;        expand(&amp;quot;{sample}_{{Rnumber}}_001.fastqc.zip&amp;quot;, sample=SAMPLES)&#xA;    shell:&#xA;        &amp;quot;echo {output}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now it's OK:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ snakemake -nrp s1_R1_001.fastqc.html&#xA;Building DAG of jobs...&#xA;Job stats:&#xA;job           count    min threads    max threads&#xA;----------  -------  -------------  -------------&#xA;run_fastqc        1              1              1&#xA;total             1              1              1&#xA;&#xA;&#xA;[Fri Nov 12 08:54:54 2021]&#xA;rule run_fastqc:&#xA;    output: s1_R1_001.fastqc.html, s2_R1_001.fastqc.html, s3_R1_001.fastqc.html, s1_R1_001.fastqc.zip, s2_R1_001.fastqc.zip, s3_R1_001.fastqc.zip&#xA;    jobid: 0&#xA;    reason: Missing output files: s1_R1_001.fastqc.html&#xA;    wildcards: Rnumber=R1&#xA;    resources: tmpdir=/tmp&#xA;&#xA;echo s1_R1_001.fastqc.html s2_R1_001.fastqc.html s3_R1_001.fastqc.html s1_R1_001.fastqc.zip s2_R1_001.fastqc.zip s3_R1_001.fastqc.zip&#xA;Job stats:&#xA;job           count    min threads    max threads&#xA;----------  -------  -------------  -------------&#xA;run_fastqc        1              1              1&#xA;total             1              1              1&#xA;&#xA;This was a dry-run (flag -n). The order of jobs does not reflect the order of execution.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Each &lt;code&gt;expand()&lt;/code&gt; makes a list, so at first Snakemake sees two lists back-to-back and doesn't know what to do with them.  If you use a comma &lt;em&gt;and&lt;/em&gt; give output names you could handle the two lists separately in your rule, but if you don't need to, you could also just us a single &lt;code&gt;expand&lt;/code&gt; call, like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;output:&#xA;    expand(&amp;quot;{sample}_{{Rnumber}}_001.fastqc.{ext}&amp;quot;, sample=SAMPLES, ext=[&amp;quot;html&amp;quot;, &amp;quot;zip&amp;quot;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Other suggestions for troubleshooting:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If the above simplified Snakefile works but the real one still doesn't, edit them incrementally to be more and more similar and you'll hone in on whatever problems are left.&lt;/li&gt;&#xA;&lt;li&gt;I find trimming away complexity until the problem reveals itself is often easiest, so I'd keep simplifying the real case until that happens.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;expand&lt;/code&gt; is just a Python function that Snakemake provides you, so you can call it yourself outside of a rule definition and check what it returns.&lt;/li&gt;&#xA;&lt;li&gt;You can even comment out &lt;em&gt;all&lt;/em&gt; the Snakemake-specific syntax and leave just Python code when debugging, to sanity-check the rest of the code (e.g. your inputs and outputs from &lt;code&gt;expand&lt;/code&gt;).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4980" LastEditorUserId="4980" LastEditDate="2021-11-16T19:16:22.277" LastActivityDate="2021-11-16T19:16:22.277" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18038" PostTypeId="2" ParentId="18025" CreationDate="2021-11-12T18:03:34.393" Score="1" Body="&lt;p&gt;&lt;code&gt;merge(a, b, by = &amp;quot;Sequence&amp;quot;)&lt;/code&gt; would &amp;quot;merge&amp;quot; the two tables (as the name suggests) based on the &lt;code&gt;Sequence&lt;/code&gt; column. The caveat is that sequences have to be exactly the same to be considered &amp;quot;common to both samples&amp;quot;, so even 1 aa difference would be missed.&lt;/p&gt;&#xA;&lt;p&gt;Simplified output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1       EKNPLPSKETIEQEKQAGES   1    47.67   47.67     98         86         86 sp|P20065|TYB4_MOUSE&#xA;2 KKTETQEKNPLPSKETIEQEKQAGES   1    47.67   47.67     98         86         86 sp|P20065|TYB4_MOUSE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2021-11-12T18:03:34.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18039" PostTypeId="2" ParentId="18034" CreationDate="2021-11-12T18:27:38.933" Score="2" Body="&lt;p&gt;If you've got enough replicates (ideally at least 6, but can be done with 3), you should be able to do a Variance-stabilised Transformation using DESeq2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;vsd &amp;lt;- vst(dds, blind=FALSE)&#xA;rld &amp;lt;- rlog(dds, blind=FALSE)&#xA;head(assay(vsd), 3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;More details on this can be found in the DESeq2 Vignette:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#extracting-transformed-values&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#extracting-transformed-values&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-11-12T18:27:38.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18040" PostTypeId="2" ParentId="10738" CreationDate="2021-11-13T00:14:32.777" Score="0" Body="&lt;p&gt;You can define your breaks with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;p + scale_size(breaks = c(0, 25, 50, 75))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can adjust the size of the points with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;p + scale_size(range = c(1,8))     # Note: c(1,6) is the default&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For more info, see &lt;code&gt;??scale_size&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="14557" LastEditorUserId="14557" LastEditDate="2021-11-13T00:24:52.593" LastActivityDate="2021-11-13T00:24:52.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18042" PostTypeId="1" CreationDate="2021-11-14T00:50:39.157" Score="0" ViewCount="15" Body="&lt;p&gt;I'm running the following command which uses the NCBI e-utilities:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;esearch -db bioproject -query &amp;quot;PRJNA198476 [PRJA]&amp;quot; | esummary&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;This returns:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;?xml version=&amp;quot;1.0&amp;quot; encoding=&amp;quot;UTF-8&amp;quot; ?&amp;gt;&#xA;&amp;lt;!DOCTYPE DocumentSummarySet PUBLIC &amp;quot;-//NLM//DTD esummary bioproject 20140903//EN&amp;quot; &amp;quot;https://eutils.ncbi.nlm.nih.gov/eutils/dtd/20140903/esummary_bioproject.dtd&amp;quot;&amp;gt;&#xA;&#xA;&amp;lt;DocumentSummarySet status=&amp;quot;OK&amp;quot;&amp;gt;&#xA;&amp;lt;DbBuild&amp;gt;Build211111-1430.1&amp;lt;/DbBuild&amp;gt;&#xA;&#xA;&amp;lt;DocumentSummary&amp;gt;&amp;lt;Id&amp;gt;198476&amp;lt;/Id&amp;gt;&#xA;    &amp;lt;TaxId&amp;gt;6239&amp;lt;/TaxId&amp;gt;&#xA;    &amp;lt;Project_Id&amp;gt;198476&amp;lt;/Project_Id&amp;gt;&#xA;    &amp;lt;Project_Acc&amp;gt;PRJNA198476&amp;lt;/Project_Acc&amp;gt;&#xA;    &amp;lt;Project_Type&amp;gt;Primary submission&amp;lt;/Project_Type&amp;gt;&#xA;    &amp;lt;Project_Data_Type&amp;gt;Transcriptome or Gene expression&amp;lt;/Project_Data_Type&amp;gt;&#xA;    &amp;lt;Sort_By_ProjectType&amp;gt;476331&amp;lt;/Sort_By_ProjectType&amp;gt;&#xA;    &amp;lt;Sort_By_DataType&amp;gt;472932&amp;lt;/Sort_By_DataType&amp;gt;&#xA;    &amp;lt;Sort_By_Organism&amp;gt;51608&amp;lt;/Sort_By_Organism&amp;gt;&#xA;    &amp;lt;Project_Subtype&amp;gt;&amp;lt;/Project_Subtype&amp;gt;&#xA;    &amp;lt;Project_Target_Scope&amp;gt;Multiisolate&amp;lt;/Project_Target_Scope&amp;gt;&#xA;    &amp;lt;Project_Target_Material&amp;gt;Transcriptome&amp;lt;/Project_Target_Material&amp;gt;&#xA;    &amp;lt;Project_Target_Capture&amp;gt;Whole&amp;lt;/Project_Target_Capture&amp;gt;&#xA;    &amp;lt;Project_MethodType&amp;gt;Sequencing&amp;lt;/Project_MethodType&amp;gt;&#xA;    &amp;lt;Project_Method&amp;gt;&amp;lt;/Project_Method&amp;gt;&#xA;    &amp;lt;Project_Objectives_List&amp;gt;&#xA;        &amp;lt;Project_Objectives_Struct&amp;gt;&#xA;            &amp;lt;Project_ObjectivesType&amp;gt;Expression&amp;lt;/Project_ObjectivesType&amp;gt;&#xA;            &amp;lt;Project_Objectives&amp;gt;&amp;lt;/Project_Objectives&amp;gt;&#xA;        &amp;lt;/Project_Objectives_Struct&amp;gt;&#xA;    &amp;lt;/Project_Objectives_List&amp;gt;&#xA;    &amp;lt;Registration_Date&amp;gt;2013/04/22 00:00&amp;lt;/Registration_Date&amp;gt;&#xA;    &amp;lt;Project_Name&amp;gt;Caenorhabditis elegans&amp;lt;/Project_Name&amp;gt;&#xA;    &amp;lt;Project_Title&amp;gt;Deep sequencing of endogenous mRNA from Caenorhabditis elegans in the presence and absence of arsenite&amp;lt;/Project_Title&amp;gt;&#xA;    &amp;lt;Project_Description&amp;gt;Background: Arsenite is one of the most toxic chemical substances known and is assumed to exert detrimental effects on viability even at lowest concentrations. By contrast and unlike higher concentrations, we here find that exposure to low-dose arsenite promotes growth of cultured mammalian cells. In the nematode C. elegans, low-dose arsenite promotes resistance against thermal and chemical stressors, and extends lifespan of this metazoan, whereas higher concentrations reduce longevity. While arsenite causes a transient increase in reactive oxygen species (ROS) levels in C. elegans, co-exposure to ROS scavengers prevents the lifespan-extending capabilities of arsenite, indicating that transiently increased ROS levels act as transducers of arsenite effects on lifespan, a process known as mitohormesis. The RNA-seq data comprises 2 biological replicates for worms exposed to 100nM Arsenite 48h after L4 and 2 biological replicates of the same age as controls Jena Centre for Systems Biology of Ageing - JenAge (www.jenage.de) Overall design: 4 samples: 2 mRNA profiles of C.elegans 48h after L4 exposed to Arsenite; 2 mRNA profiles of C.elegans 48h after L4 as controls (H20). The N2 wild type (var. Bristol) strain was used.&amp;lt;/Project_Description&amp;gt;&#xA;    &amp;lt;Keyword&amp;gt;&amp;lt;/Keyword&amp;gt;&#xA;    &amp;lt;Relevance_Agricultural&amp;gt;&amp;lt;/Relevance_Agricultural&amp;gt;&#xA;    &amp;lt;Relevance_Medical&amp;gt;&amp;lt;/Relevance_Medical&amp;gt;&#xA;    &amp;lt;Relevance_Industrial&amp;gt;&amp;lt;/Relevance_Industrial&amp;gt;&#xA;    &amp;lt;Relevance_Environmental&amp;gt;&amp;lt;/Relevance_Environmental&amp;gt;&#xA;    &amp;lt;Relevance_Evolution&amp;gt;&amp;lt;/Relevance_Evolution&amp;gt;&#xA;    &amp;lt;Relevance_Model&amp;gt;Yes&amp;lt;/Relevance_Model&amp;gt;&#xA;    &amp;lt;Relevance_Other&amp;gt;&amp;lt;/Relevance_Other&amp;gt;&#xA;    &amp;lt;Organism_Name&amp;gt;Caenorhabditis elegans&amp;lt;/Organism_Name&amp;gt;&#xA;    &amp;lt;Organism_Strain&amp;gt;&amp;lt;/Organism_Strain&amp;gt;&#xA;    &amp;lt;Organism_Label&amp;gt;&amp;lt;/Organism_Label&amp;gt;&#xA;    &amp;lt;Sequencing_Status&amp;gt;SRA/Trace&amp;lt;/Sequencing_Status&amp;gt;&#xA;    &amp;lt;Submitter_Organization&amp;gt;Leibniz Institute for Age Research - Fritz Lipmann Institute&amp;lt;/Submitter_Organization&amp;gt;&#xA;    &amp;lt;Submitter_Organization_List&amp;gt;&#xA;        &amp;lt;string&amp;gt;Leibniz Institute for Age Research - Fritz Lipmann Institute&amp;lt;/string&amp;gt;&#xA;    &amp;lt;/Submitter_Organization_List&amp;gt;&#xA;    &amp;lt;Supergroup&amp;gt;Eukaryotes&amp;lt;/Supergroup&amp;gt;&#xA;&amp;lt;/DocumentSummary&amp;gt;&#xA;&#xA;&amp;lt;/DocumentSummarySet&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm trying to extract the &lt;code&gt;Submitter_Organization&lt;/code&gt; using:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;esearch -db bioproject -query &amp;quot;PRJNA198476 [PRJA]&amp;quot; | esummary | xtract -pattern DocummentSummary -element Submitter_Organization&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, this returns nothing, and I can't figure out why&lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2021-11-14T00:50:39.157" Title="xtract from eutilities not exctracting the information I need" Tags="|phylogenetics|e-utilities|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18043" PostTypeId="2" ParentId="18027" CreationDate="2021-11-14T19:48:00.750" Score="2" Body="&lt;p&gt;You can opt for a similarity matrix as output (&lt;a href=&quot;https://biology.stackexchange.com/questions/46049/how-to-interpret-percent-identity-matrix-created-by-clustal-omega&quot;&gt;here is an example&lt;/a&gt;). The fact that you would like to judge conservation &amp;quot;relative to a given sequence&amp;quot;, you can use this matrix to rank similarities of different sequences with respect to any of these sequences.&lt;/p&gt;&#xA;&lt;p&gt;I would advise visualizing these similarities, for example in terms of a heatmap, and seeing if anything stands out (with respect to your sequence of interest). And I would advise against selecting an arbitrary threshold (i.e. 90% or 95%), this would trigger questions based on the selection criteria of such a cutoff and this is understandable, such arbitrary cut-offs often do not mean much in biology.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2021-11-14T19:48:00.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18044" PostTypeId="1" CreationDate="2021-11-15T02:10:25.660" Score="3" ViewCount="200" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/k7HvH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/k7HvH.png&quot; alt=&quot;Genome Circular Map&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know what tools are used to construct this beautiful genome map?&#xA;The research paper didn't mention how to construct this map.&#xA;Link to paper: &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S2052297515000773?via%3Dihub&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sciencedirect.com/science/article/pii/S2052297515000773?via%3Dihub&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14560" LastActivityDate="2021-11-17T02:47:55.960" Title="Does anyone know what tools are used to construct this beautiful genome map?" Tags="|sequence-annotation|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18045" PostTypeId="2" ParentId="18044" CreationDate="2021-11-15T06:52:30.860" Score="6" Body="&lt;p&gt;Circos: &lt;a href=&quot;http://circos.ca/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://circos.ca/&lt;/a&gt; (or a copy of it written for another language, depending on what the paper's authors used — probably best to check with them).&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2021-11-15T17:36:15.910" LastActivityDate="2021-11-15T17:36:15.910" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18046" PostTypeId="2" ParentId="2684" CreationDate="2021-11-15T11:27:19.953" Score="0" Body="&lt;p&gt;From the top bar:&#xA;View -&amp;gt; DNA&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MdU7t.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MdU7t.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1852" LastActivityDate="2021-11-15T11:27:19.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18047" PostTypeId="1" CreationDate="2021-11-15T12:19:00.433" Score="1" ViewCount="25" Body="&lt;p&gt;I have a list of accessions for the the entire mitochondrial genomes for big cats. I need to download the COX1 genes for each of these accessions.&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/KP202272&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is one accession&lt;/a&gt; and here is a &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/KP202272.1?from=5994&amp;amp;to=7538&amp;amp;report=gbwithparts&quot; rel=&quot;nofollow noreferrer&quot;&gt;link to its COX1&lt;/a&gt; gene, which I found manually on that page.&lt;/p&gt;&#xA;&lt;p&gt;I have downloaded all the accessions as an XML genbank file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;from Bio import Entrez&#xA;from Bio import GenBank&#xA;from Bio import SeqIO&#xA;import os&#xA;&#xA;&#xA;if os.path.exists('cats.xml') ==False:&#xA;    with open('accessions.txt', 'r') as fin:&#xA;        #for line&#xA;        accs = [line.rstrip() for line in fin ]&#xA;&#xA;    print(accs)&#xA;&#xA;    Entrez.email =  &amp;quot;first.last@example.se&amp;quot;&#xA;    rec = Entrez.efetch(db='nuccore', id=accs, rettype=&amp;quot;gb&amp;quot;, retmode=&amp;quot;xml&amp;quot;)&#xA;    with open('cats.xml', 'w') as out:&#xA;        print(rec.read().decode(&amp;quot;utf-8&amp;quot;), file=out)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then I tried to parse this file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with open('cats.xml', 'rb') as xin:&#xA;    for record  in Entrez.parse(xin):&#xA;        #print(dir(record))&#xA;        #print(record.keys())&#xA;        pass&#xA;print(record['GBSeq_feature-table'][0])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But it's a mess of nested lists and dictionaries. Help!&lt;/p&gt;&#xA;" OwnerUserId="10553" LastEditorUserId="298" LastEditDate="2021-11-15T16:51:57.867" LastActivityDate="2021-11-15T16:51:57.867" Title="Download COX1 (COI) gene via biopython using accessions for entire mitochondrial genomes" Tags="|python|entrez|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18048" PostTypeId="2" ParentId="17913" CreationDate="2021-11-15T12:54:16.513" Score="2" Body="&lt;p&gt;You posted similar question here: &lt;a href=&quot;https://stackoverflow.com/questions/69676943/how-can-i-get-a-list-of-neighboring-hydrogen-atoms-of-an-alpha-carbon/69815509#69815509&quot;&gt;https://stackoverflow.com/questions/69676943/how-can-i-get-a-list-of-neighboring-hydrogen-atoms-of-an-alpha-carbon/69815509#69815509&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;ok got some code :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.PDB import Chain, Atom, NeighborSearch, PDBParser, Selection&#xA;&#xA;&#xA;def get_hydrogen_atoms(c_alpha_atom: Atom.Atom, po):&#xA;    ns = NeighborSearch(po)&#xA;    return [h for h in ns.search( c_alpha_atom.get_coord(), 10.0, 'A') if h.element == 'H']&#xA;&#xA;p = PDBParser()&#xA;# structure = p.get_structure('16pk', '16pk.pdb')&#xA;structure = p.get_structure('16pk', 'pH8_5_1bq9_short.pdb')&#xA;&#xA;c_alpha_list = [atom for atom in structure.get_atoms() if atom.name == &amp;quot;CA&amp;quot;]&#xA;&#xA;print('c_alpha_list[0]  : ', c_alpha_list[0], c_alpha_list[0].id , c_alpha_list[0].get_serial_number(),'\n')&#xA;&#xA;po = Selection.unfold_entities(structure, 'A')&#xA;                               &#xA;hydrogen_list = get_hydrogen_atoms(c_alpha_list[0], po)&#xA;&#xA;print('hydrogen_list : \n',hydrogen_list)&#xA;&#xA;print('\nSelection (po) : \n', po)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;had to use a different PDB file because I needed the H atoms,&lt;/p&gt;&#xA;&lt;p&gt;pH8_5_1bq9_short.pdb :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;HETATM    1  N   FME A   1      23.447  -5.765   5.119  1.00 19.07           N&#xA;HETATM    2  CN  FME A   1      23.308  -6.776   4.291  1.00 20.46           C&#xA;HETATM    3  O1  FME A   1      22.504  -6.713   3.357  1.00 24.75           O&#xA;HETATM    4  CA  FME A   1      22.488  -4.745   5.410  1.00 18.15           C&#xA;HETATM    5  CB  FME A   1      22.433  -4.342   6.827  1.00 21.15           C&#xA;HETATM    6  CG  FME A   1      22.198  -5.618   7.751  1.00 22.84           C&#xA;HETATM    7  SD  FME A   1      20.445  -6.032   7.772  1.00 23.40           S&#xA;HETATM    8  CE  FME A   1      20.476  -7.587   8.656  1.00 28.76           C&#xA;HETATM    9  C   FME A   1      22.699  -3.530   4.487  1.00 17.12           C&#xA;HETATM   10  O   FME A   1      23.804  -3.248   4.077  1.00 20.25           O&#xA;HETATM    0 3HE  FME A   1      19.574  -7.938   8.726  1.00 28.76           H   new&#xA;HETATM    0 2HG  FME A   1      22.713  -6.369   7.418  1.00 22.84           H   new&#xA;HETATM    0 2HE  FME A   1      21.032  -8.221   8.177  1.00 28.76           H   new&#xA;HETATM    0 2HB  FME A   1      23.260  -3.901   7.077  1.00 21.15           H   new&#xA;HETATM    0 1HG  FME A   1      22.508  -5.438   8.652  1.00 22.84           H   new&#xA;HETATM    0 1HE  FME A   1      20.838  -7.447   9.545  1.00 28.76           H   new&#xA;HETATM    0 1HB  FME A   1      21.717  -3.701   6.961  1.00 21.15           H   new&#xA;HETATM    0  HN  FME A   1      24.199  -5.711   5.532  1.00 19.07           H   new&#xA;HETATM    0  HCN FME A   1      23.841  -7.577   4.415  1.00 20.46           H   new&#xA;HETATM    0  HA  FME A   1      21.619  -5.138   5.231  1.00 18.15           H   new&#xA;ATOM     11  N   ALA A   2      21.633  -2.939   4.043  1.00 15.50           N&#xA;ATOM     12  CA  ALA A   2      21.644  -1.884   3.062  1.00 15.08           C&#xA;ATOM     13  C   ALA A   2      20.899  -0.661   3.519  1.00 13.72           C&#xA;ATOM     14  O   ALA A   2      20.278  -0.702   4.584  1.00 13.49           O&#xA;ATOM     15  CB  ALA A   2      21.083  -2.339   1.710  1.00 17.86           C&#xA;ATOM      0  H   ALA A   2      20.842  -3.144   4.312  1.00 15.50           H   new&#xA;ATOM      0  HA  ALA A   2      22.579  -1.652   2.951  1.00 15.08           H   new&#xA;ATOM      0  HB1 ALA A   2      21.110  -1.599   1.083  1.00 17.86           H   new&#xA;ATOM      0  HB2 ALA A   2      21.618  -3.072   1.368  1.00 17.86           H   new&#xA;ATOM      0  HB3 ALA A   2      20.166  -2.634   1.823  1.00 17.86           H   new&#xA;ATOM     16  N   LYS A   3      21.005   0.412   2.774  1.00 12.48           N&#xA;ATOM     17  CA  LYS A   3      20.341   1.658   3.029  1.00 12.61           C&#xA;ATOM     18  C   LYS A   3      19.415   1.991   1.879  1.00 11.53           C&#xA;ATOM     19  O   LYS A   3      19.784   1.753   0.701  1.00 11.66           O&#xA;ATOM     20  CB  LYS A   3      21.338   2.806   3.273  1.00 13.38           C&#xA;ATOM     21  CG  LYS A   3      22.062   2.721   4.593  1.00 16.70           C&#xA;ATOM     22  CD  LYS A   3      23.062   3.847   4.814  1.00 18.05           C&#xA;ATOM     23  CE  LYS A   3      23.806   3.576   6.136  1.00 20.42           C&#xA;ATOM     24  NZ  LYS A   3      24.878   4.503   6.403  1.00 22.91           N&#xA;ATOM      0  H   LYS A   3      21.494   0.433   2.067  1.00 12.48           H   new&#xA;ATOM      0  HA  LYS A   3      19.821   1.557   3.842  1.00 12.61           H   new&#xA;ATOM      0  HB2 LYS A   3      21.992   2.812   2.556  1.00 13.38           H   new&#xA;ATOM      0  HB3 LYS A   3      20.862   3.650   3.230  1.00 13.38           H   new&#xA;ATOM      0  HG2 LYS A   3      21.411   2.732   5.312  1.00 16.70           H   new&#xA;ATOM      0  HG3 LYS A   3      22.527   1.871   4.644  1.00 16.70           H   new&#xA;ATOM      0  HD2 LYS A   3      23.689   3.892   4.076  1.00 18.05           H   new&#xA;ATOM      0  HD3 LYS A   3      22.606   4.702   4.851  1.00 18.05           H   new&#xA;ATOM      0  HE2 LYS A   3      23.170   3.611   6.868  1.00 20.42           H   new&#xA;ATOM      0  HE3 LYS A   3      24.166   2.676   6.116  1.00 20.42           H   new&#xA;ATOM      0  HZ1 LYS A   3      25.266   4.294   7.176  1.00 22.91           H   new&#xA;ATOM      0  HZ2 LYS A   3      25.479   4.458   5.748  1.00 22.91           H   new&#xA;ATOM      0  HZ3 LYS A   3      24.551   5.330   6.449  1.00 22.91           H   new&#xA;ATOM     25  N   TRP A   4      18.274   2.598   2.165  1.00 10.75           N&#xA;ATOM     26  CA  TRP A   4      17.283   3.019   1.181  1.00 10.69           C&#xA;ATOM     27  C   TRP A   4      16.875   4.472   1.502  1.00 11.37           C&#xA;ATOM     28  O   TRP A   4      16.812   4.872   2.651  1.00 17.19           O&#xA;ATOM     29  CB  TRP A   4      16.012   2.143   1.352  1.00 11.20           C&#xA;ATOM     30  CG  TRP A   4      16.164   0.734   0.997  1.00 11.80           C&#xA;ATOM     31  CD1 TRP A   4      16.913  -0.218   1.669  1.00 12.50           C&#xA;ATOM     32  CD2 TRP A   4      15.535   0.036  -0.079  1.00 11.53           C&#xA;ATOM     33  NE1 TRP A   4      16.806  -1.435   1.050  1.00 12.76           N&#xA;ATOM     34  CE2 TRP A   4      15.959  -1.302  -0.013  1.00 12.61           C&#xA;ATOM     35  CE3 TRP A   4      14.642   0.421  -1.079  1.00 11.88           C&#xA;ATOM     36  CZ2 TRP A   4      15.531  -2.258  -0.929  1.00 13.18           C&#xA;ATOM     37  CZ3 TRP A   4      14.217  -0.527  -1.989  1.00 12.54           C&#xA;ATOM     38  CH2 TRP A   4      14.668  -1.858  -1.894  1.00 14.26           C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;c_alpha_list[0]  :  &amp;lt;Atom CA&amp;gt; CA 4 &#xA;&#xA;hydrogen_list : &#xA; [&amp;lt;Atom HB1&amp;gt;, &amp;lt;Atom HB3&amp;gt;, &amp;lt;Atom H&amp;gt;, &amp;lt;Atom 2HE&amp;gt;, &amp;lt;Atom 3HE&amp;gt;, &amp;lt;Atom 1HE&amp;gt;, &amp;lt;Atom HB3&amp;gt;, &amp;lt;Atom HA&amp;gt;, &amp;lt;Atom HG2&amp;gt;, &amp;lt;Atom HB2&amp;gt;, &amp;lt;Atom HCN&amp;gt;, &amp;lt;Atom HA&amp;gt;, &amp;lt;Atom HN&amp;gt;, &amp;lt;Atom 1HB&amp;gt;, &amp;lt;Atom 2HB&amp;gt;, &amp;lt;Atom 2HG&amp;gt;, &amp;lt;Atom 1HG&amp;gt;, &amp;lt;Atom H&amp;gt;, &amp;lt;Atom HB2&amp;gt;, &amp;lt;Atom HA&amp;gt;, &amp;lt;Atom HD2&amp;gt;, &amp;lt;Atom HG3&amp;gt;, &amp;lt;Atom HD3&amp;gt;, &amp;lt;Atom HZ2&amp;gt;, &amp;lt;Atom HE3&amp;gt;, &amp;lt;Atom HE2&amp;gt;, &amp;lt;Atom HZ1&amp;gt;]&#xA;&#xA;Selection (po) : &#xA; [&amp;lt;Atom N&amp;gt;, &amp;lt;Atom CN&amp;gt;, &amp;lt;Atom O1&amp;gt;, &amp;lt;Atom CA&amp;gt;, &amp;lt;Atom CB&amp;gt;, &amp;lt;Atom CG&amp;gt;, &amp;lt;Atom SD&amp;gt;, &amp;lt;Atom CE&amp;gt;, &amp;lt;Atom C&amp;gt;, &amp;lt;Atom O&amp;gt;, &amp;lt;Atom 3HE&amp;gt;, &amp;lt;Atom 2HG&amp;gt;, &amp;lt;Atom 2HE&amp;gt;, &amp;lt;Atom 2HB&amp;gt;, &amp;lt;Atom 1HG&amp;gt;, &amp;lt;Atom 1HE&amp;gt;, &amp;lt;Atom 1HB&amp;gt;, &amp;lt;Atom HN&amp;gt;, &amp;lt;Atom HCN&amp;gt;, &amp;lt;Atom HA&amp;gt;, &amp;lt;Atom N&amp;gt;, &amp;lt;Atom CA&amp;gt;, &amp;lt;Atom C&amp;gt;, &amp;lt;Atom O&amp;gt;, &amp;lt;Atom CB&amp;gt;, &amp;lt;Atom H&amp;gt;, &amp;lt;Atom HA&amp;gt;, &amp;lt;Atom HB1&amp;gt;, &amp;lt;Atom HB2&amp;gt;, &amp;lt;Atom HB3&amp;gt;, &amp;lt;Atom N&amp;gt;, &amp;lt;Atom CA&amp;gt;, &amp;lt;Atom C&amp;gt;, &amp;lt;Atom O&amp;gt;, &amp;lt;Atom CB&amp;gt;, &amp;lt;Atom CG&amp;gt;, &amp;lt;Atom CD&amp;gt;, &amp;lt;Atom CE&amp;gt;, &amp;lt;Atom NZ&amp;gt;, &amp;lt;Atom H&amp;gt;, &amp;lt;Atom HA&amp;gt;, &amp;lt;Atom HB2&amp;gt;, &amp;lt;Atom HB3&amp;gt;, &amp;lt;Atom HG2&amp;gt;, &amp;lt;Atom HG3&amp;gt;, &amp;lt;Atom HD2&amp;gt;, &amp;lt;Atom HD3&amp;gt;, &amp;lt;Atom HE2&amp;gt;, &amp;lt;Atom HE3&amp;gt;, &amp;lt;Atom HZ1&amp;gt;, &amp;lt;Atom HZ2&amp;gt;, &amp;lt;Atom HZ3&amp;gt;, &amp;lt;Atom N&amp;gt;, &amp;lt;Atom CA&amp;gt;, &amp;lt;Atom C&amp;gt;, &amp;lt;Atom O&amp;gt;, &amp;lt;Atom CB&amp;gt;, &amp;lt;Atom CG&amp;gt;, &amp;lt;Atom CD1&amp;gt;, &amp;lt;Atom CD2&amp;gt;, &amp;lt;Atom NE1&amp;gt;, &amp;lt;Atom CE2&amp;gt;, &amp;lt;Atom CE3&amp;gt;, &amp;lt;Atom CZ2&amp;gt;, &amp;lt;Atom CZ3&amp;gt;, &amp;lt;Atom CH2&amp;gt;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;the selection list contains both ATOM and HETATM&lt;/p&gt;&#xA;&lt;p&gt;have a look at : &lt;a href=&quot;https://www.biostars.org/p/269579/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/269579/&lt;/a&gt; How to use biopython neighborsearch?&lt;/p&gt;&#xA;&lt;p&gt;and Bio.PDB.NeighborSearch module :&lt;a href=&quot;https://biopython.org/docs/1.75/api/Bio.PDB.NeighborSearch.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biopython.org/docs/1.75/api/Bio.PDB.NeighborSearch.html&lt;/a&gt; ;&lt;/p&gt;&#xA;&lt;p&gt;Bio.PDB.Atom module :&lt;a href=&quot;https://biopython.org/docs/1.75/api/Bio.PDB.Atom.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biopython.org/docs/1.75/api/Bio.PDB.Atom.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10049" LastActivityDate="2021-11-15T12:54:16.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18049" PostTypeId="1" CreationDate="2021-11-15T17:47:56.623" Score="1" ViewCount="142" Body="&lt;p&gt;I am trying to extract the protein sequence of specific genes from gene bank like format file obtained from antismash part of which looks like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;LOCUS       scaffold_10            47160 bp    DNA     linear   UNK 01-JAN-1980&#xA;DEFINITION  scaffold_10.&#xA;ACCESSION   scaffold_10&#xA;VERSION     scaffold_10&#xA;KEYWORDS    .&#xA;SOURCE&#xA;  ORGANISM&#xA;            .&#xA;COMMENT     ##antiSMASH-Data-START##&#xA;            Version      :: 6.0.1-25e43a7(changed)&#xA;            Run date     :: 2021-11-05 17:39:45&#xA;            NOTE: This is a single cluster extracted from a larger record!&#xA;            Orig. start  :: 2049841&#xA;            Orig. end    :: 2097001&#xA;            ##antiSMASH-Data-END##&#xA;FEATURES             Location/Qualifiers&#xA;     protocluster    1..47160&#xA;                     /aStool=&amp;quot;rule-based-clusters&amp;quot;&#xA;                     /contig_edge=&amp;quot;False&amp;quot;&#xA;                     /core_location=&amp;quot;join{[2069841:2070128](+),&#xA;                     [2070175:2070449](+), [2070507:2071973](+),&#xA;                     [2072023:2074178](+), [2074232:2074784](+),&#xA;                     [2074838:2077001](+)}&amp;quot;&#xA;                     /cutoff=&amp;quot;20000&amp;quot;&#xA;                     /detection_rule=&amp;quot;cds(PKS_AT and (PKS_KS or ene_KS or mod_KS&#xA;                     or hyb_KS or itr_KS or tra_KS))&amp;quot;&#xA;                     /neighbourhood=&amp;quot;20000&amp;quot;&#xA;                     /product=&amp;quot;T1PKS&amp;quot;&#xA;                     /protocluster_number=&amp;quot;1&amp;quot;&#xA;                     /tool=&amp;quot;antismash&amp;quot;&#xA;     proto_core      join(20001..20287,20335..20608,20667..22132,22183..24337,&#xA;                     24392..24943,24998..27160)&#xA;                     /aStool=&amp;quot;rule-based-clusters&amp;quot;&#xA;                     /tool=&amp;quot;antismash&amp;quot;&#xA;                     /cutoff=&amp;quot;20000&amp;quot;&#xA;                     /detection_rule=&amp;quot;cds(PKS_AT and (PKS_KS or ene_KS or mod_KS&#xA;                     or hyb_KS or itr_KS or tra_KS))&amp;quot;&#xA;                     /neighbourhood=&amp;quot;20000&amp;quot;&#xA;                     /product=&amp;quot;T1PKS&amp;quot;&#xA;                     /protocluster_number=&amp;quot;1&amp;quot;&#xA;     cand_cluster    1..47160&#xA;                     /SMILES=&amp;quot;CC(=O)C(=O)O&amp;quot;&#xA;                     /candidate_cluster_number=&amp;quot;1&amp;quot;&#xA;                     /contig_edge=&amp;quot;False&amp;quot;&#xA;                     /detection_rules=&amp;quot;cds(PKS_AT and (PKS_KS or ene_KS or&#xA;                     mod_KS or hyb_KS or itr_KS or tra_KS))&amp;quot;&#xA;                     /kind=&amp;quot;single&amp;quot;&#xA;                     /product=&amp;quot;T1PKS&amp;quot;&#xA;                     /protoclusters=&amp;quot;1&amp;quot;&#xA;                     /tool=&amp;quot;antismash&amp;quot;&#xA;     region          1..47160&#xA;                     /candidate_cluster_numbers=&amp;quot;1&amp;quot;&#xA;                     /contig_edge=&amp;quot;False&amp;quot;&#xA;                     /product=&amp;quot;T1PKS&amp;quot;&#xA;                     /region_number=&amp;quot;1&amp;quot;&#xA;                     /rules=&amp;quot;cds(PKS_AT and (PKS_KS or ene_KS or mod_KS or&#xA;                     hyb_KS or itr_KS or tra_KS))&amp;quot;&#xA;                     /tool=&amp;quot;antismash&amp;quot;&#xA;     CDS             1899..2720&#xA;                     /ID=&amp;quot;CDS_13110&amp;quot;&#xA;                     /NRPS_PKS=&amp;quot;Domain: PKS_KR (26-149). E-value: 7.6e-08.&#xA;                     Score: 24.2. Matches aSDomain:&#xA;                     nrpspksdomains_jgi.p_FusspF11_1_766668_PKS_KR.1&amp;quot;&#xA;                     /NRPS_PKS=&amp;quot;type: other&amp;quot;&#xA;                     /gene=&amp;quot;jgi.p_FusspF11_1_766668&amp;quot;&#xA;                     /gene_functions=&amp;quot;biosynthetic-additional (smcogs)&#xA;                     SMCOG1001:short-chain dehydrogenase/reductase SDR (Score:&#xA;                     161.6; E-value: 3.9e-49)&amp;quot;&#xA;                     /gene_kind=&amp;quot;biosynthetic-additional&amp;quot;&#xA;                     /phase=&amp;quot;0&amp;quot;&#xA;                     /source=&amp;quot;prediction&amp;quot;&#xA;                     /transl_table=1&#xA;                     /translation=&amp;quot;MTPPPKSALESDPAVGAQRRFSVTGNAVVTGGAGVLGLHACDALL&#xA;                     EHGLEGLMILDVNPAQSQGQITSLQNKFPRAKIMALKVDVTDENVVNAAMEETARVLGS&#xA;                     IDTLICFVGVVGCVETLDMPVPQWRKILDINTTGSFICAQAAARQMVKRGRGGSIVFVA&#xA;                     SISAHRVNYPQPQAAYNVSKSALLMLKSCLAAEWARYGIRTNSISPGYMDTILNEGDGI&#xA;                     AEHRKIWAEHNPSGRMGAPSELTGTVVLLASSAGSYINGADIVVDGGGIVL&amp;quot;&#xA;     aSDomain        1977..2345&#xA;                     /aSDomain=&amp;quot;PKS_KR&amp;quot;&#xA;                     /aSTool=&amp;quot;nrps_pks_domains&amp;quot;&#xA;                     /database=&amp;quot;nrpspksdomains.hmm&amp;quot;&#xA;                     /detection=&amp;quot;hmmscan&amp;quot;&#xA;                     /domain_id=&amp;quot;nrpspksdomains_jgi.p_FusspF11_1_766668_PKS_KR.1&#xA;                     &amp;quot;&#xA;                     /evalue=&amp;quot;7.60E-08&amp;quot;&#xA;                     /label=&amp;quot;jgi.p_FusspF11_1_766668_PKS_KR.1&amp;quot;&#xA;                     /locus_tag=&amp;quot;jgi.p_FusspF11_1_766668&amp;quot;&#xA;                     /protein_end=&amp;quot;149&amp;quot;&#xA;                     /protein_start=&amp;quot;26&amp;quot;&#xA;                     /score=&amp;quot;24.2&amp;quot;&#xA;                     /specificity=&amp;quot;KR activity: inactive&amp;quot;&#xA;                     /specificity=&amp;quot;KR stereochemistry: C1&amp;quot;&#xA;                     /tool=&amp;quot;antismash&amp;quot;&#xA;                     /translation=&amp;quot;AVVTGGAGVLGLHACDALLEHGLEGLMILDVNPAQSQGQITSLQN&#xA;                     KFPRAKIMALKVDVTDENVVNAAMEETARVLGSIDTLICFVGVVGCVETLDMPVPQWRK&#xA;                     ILDINTTGSFICAQAAARQ&amp;quot;&#xA;     aSModule        1977..2345&#xA;                     /domains=&amp;quot;nrpspksdomains_jgi.p_FusspF11_1_766668_PKS_KR.1&amp;quot;&#xA;                     /incomplete&#xA;                     /locus_tags=&amp;quot;jgi.p_FusspF11_1_766668&amp;quot;&#xA;                     /tool=&amp;quot;antismash&amp;quot;&#xA;                     /type=&amp;quot;pks&amp;quot;&#xA;     CDS_motif       1980..2042&#xA;                     /aSTool=&amp;quot;nrps_pks_domains&amp;quot;&#xA;                     /database=&amp;quot;abmotifs&amp;quot;&#xA;                     /detection=&amp;quot;hmmscan&amp;quot;&#xA;                     /domain_id=&amp;quot;nrpspksmotif_jgi.p_FusspF11_1_766668_0001&amp;quot;&#xA;                     /evalue=&amp;quot;1.90E-05&amp;quot;&#xA;                     /label=&amp;quot;PKSI-KR_m1&amp;quot;&#xA;                     /locus_tag=&amp;quot;jgi.p_FusspF11_1_766668&amp;quot;&#xA;                     /protein_end=&amp;quot;48&amp;quot;&#xA;                     /protein_start=&amp;quot;27&amp;quot;&#xA;                     /score=&amp;quot;17.3&amp;quot;&#xA;                     /tool=&amp;quot;antismash&amp;quot;&#xA;                     /translation=&amp;quot;VVTGGAGVLGLHACDALLEHG&amp;quot;&#xA;     CDS             join(3252..3263,3321..3443,3493..4428)&#xA;                     /ID=&amp;quot;CDS_13111&amp;quot;&#xA;                     /NRPS_PKS=&amp;quot;Domain: PKS_ER (28-277). E-value: 6.4e-13.&#xA;                     Score: 40.2. Matches aSDomain:&#xA;                     nrpspksdomains_jgi.p_FusspF11_1_794272_PKS_ER.1&amp;quot;&#xA;                     /NRPS_PKS=&amp;quot;type: other&amp;quot;&#xA;                     /gene=&amp;quot;jgi.p_FusspF11_1_794272&amp;quot;&#xA;                     /gene_functions=&amp;quot;biosynthetic-additional (smcogs)&#xA;                     SMCOG1040:alcohol dehydrogenase (Score: 275.7; E-value:&#xA;                     7.6e-84)&amp;quot;&#xA;                     /gene_kind=&amp;quot;biosynthetic-additional&amp;quot;&#xA;                     /phase=&amp;quot;0&amp;quot;&#xA;                     /source=&amp;quot;prediction&amp;quot;&#xA;                     /transl_table=1&#xA;                     /translation=&amp;quot;MSQTNLSCVLYGPGKARFENRPVPSLKDAHDVIIRISYVGVCGSD&#xA;                     VHFWTDGGFARKVSEDQPLVMGHEASGIVRSIGPDVTLLKPGDRVAIEPGFSCRRCKQC&#xA;                     KDGRYNLCPKMKFAADPPLTQGTLSRFFSIPEDFAYKIPDSLSLEEAVLVEPLAVAVHG&#xA;                     IRLAGLEVGQRVLVQGSGTIGLLTAAVAKAYGAKQVYITDVNLDKIKFAKKYLECSAFI&#xA;                     PDLGSTPEENAARFKTETGLDDGVDAVIECTGVEASTQTGLLALSAGGVLVQVGLGKPV&#xA;                     QAIPIHAMSEKEIVLKTSFRYGPGDYEIALELLESGKVSVRPLISSITPFEKATEAWEK&#xA;                     TRKGKGIKNLIRGVQD&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now from this I want to extract the protein sequence of gens having PKS domain (PKS_ER/PKS_AT/PKS_KS ...). So far I have tried the following biopython code but it is giving me error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;from Bio.SeqFeature import SeqFeature&#xA;&#xA;gbk = &amp;quot;myfile_predicted_SMclusters.gbk&amp;quot;&#xA;fa = &amp;quot;myfile_PKS_protein.fa&amp;quot;&#xA;input_handle = open(gbk, &amp;quot;r&amp;quot;)&#xA;output_handle = open(fa, &amp;quot;w&amp;quot;)&#xA;&#xA;for record in SeqIO.parse(input_handle, &amp;quot;genbank&amp;quot;):&#xA;    features = [feature for feature in record.features if feature.type == &amp;quot;CDS&amp;quot;]&#xA;    for feature in features:&#xA;        if feature.qualifiers[&amp;quot;NRPS_PKS&amp;quot;][1] == &amp;quot;PKS&amp;quot;:&#xA;        assert len(seq_feature.qualifiers['translation'])==1&#xA;        output_handle.write(&amp;quot;&amp;gt;%s %s\n%s\n&amp;quot; % (&#xA;            record.description,&#xA;            feature.location,&#xA;            seq_feature.qualifiers['translation'][0]))&#xA;&#xA;output_handle.close()&#xA;input_handle.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The error is as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;if feature.qualifiers[&amp;quot;NRPS_PKS&amp;quot;][1] == &amp;quot;PKS&amp;quot;:&#xA;KeyError: 'NRPS_PKS'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The link to the sample inputfile is below:&#xA;&lt;a href=&quot;https://antismash.secondarymetabolites.org/upload/bacteria-ab76f3b0-768f-495c-beb0-885651eb2439/Necha_masked_scaffolds.gbk&quot; rel=&quot;nofollow noreferrer&quot;&gt;sample my_file.gbk&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm using python 3.8.8&lt;/p&gt;&#xA;&lt;p&gt;Please help!!!&lt;/p&gt;&#xA;" OwnerUserId="14569" OwnerDisplayName="ambika" LastEditorUserId="73" LastEditDate="2021-11-16T21:56:35.967" LastActivityDate="2023-04-11T01:03:20.373" Title="Error while parsing gene bank file using Biopython" Tags="|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18050" PostTypeId="2" ParentId="18049" CreationDate="2021-11-15T20:16:24.613" Score="1" Body="&lt;p&gt;I see 2 problems.&lt;/p&gt;&#xA;&lt;p&gt;First, you need to check if the qualifier your are searching for exists in the given CDS feature.&lt;/p&gt;&#xA;&lt;p&gt;Second, the qualifier holds more then just the string &lt;code&gt;PKS&lt;/code&gt; for example &lt;code&gt;Domain: PKS_AT (537-846). E-value: 1.5e-98&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You could modify your code like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;feature_type = &amp;quot;CDS&amp;quot;&#xA;feature_qualifier = &amp;quot;NRPS_PKS&amp;quot;&#xA;target = &amp;quot;PKS&amp;quot;&#xA;&#xA;def has_pks_value(feature, qualifier, part):&#xA;    return any(part in target for target in feature.qualifiers[qualifier])&#xA;&#xA;for record in SeqIO.parse(input_handle, &amp;quot;genbank&amp;quot;):&#xA;    for feature in record.features:&#xA;        if feature.type == feature_type and feature_qualifier in feature.qualifiers.keys() and has_pks_value(feature, feature_qualifier, target) and len(feature.qualifiers['translation'])==1:&#xA;            output_handle.write(f&amp;quot;&amp;gt;{record.description} {feature.location.start} {feature.location.end} {feature.location.strand} \n {SeqFeature(feature.location).extract(record.seq)}&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've created some variables which for your string. This might prevent you from making some typos.&lt;/p&gt;&#xA;&lt;p&gt;Also instead of a list comprehension, I just run over all features and checked with a single &lt;code&gt;if&lt;/code&gt; statement if the asked qualifier exists and contains the given value.&lt;/p&gt;&#xA;&lt;p&gt;I also changed the string formatting to f-string. But I think you should create the Fasta Files using the &lt;code&gt;SeqIO&lt;/code&gt; module of Biopython.&lt;/p&gt;&#xA;&lt;p&gt;As result I got 37 sequences&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Update - I added a missing method to check for the correct value of the qualifier&lt;/p&gt;&#xA;" OwnerUserId="3360" LastEditorUserId="3360" LastEditDate="2021-11-15T21:34:04.480" LastActivityDate="2021-11-15T21:34:04.480" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18051" PostTypeId="1" AcceptedAnswerId="18054" CreationDate="2021-11-16T04:54:07.413" Score="2" ViewCount="51" Body="&lt;pre&gt;&lt;code&gt;dict1 = {'chr1:1000-2000': ['A', 'B', 'C'], 'chr2:999-2000': ['A', 'X', 'Y'], 'chr3:1200-3000': ['B', 'A', 'C']}&#xA;&#xA;list1 = [('A', 'B'), ('A', 'C'), ('A', 'X'), ('A', 'Y'), ('B', 'C'), ('B', 'X'), ('B', 'Y'), ('C', 'X'), ('C', 'Y'), ('X', 'Y')]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to map the items in list1 eg('A','B') both in dict1 values and return keys in wich this group occur&lt;/p&gt;&#xA;" OwnerUserId="9397" LastEditorUserId="9397" LastEditDate="2023-09-20T09:01:51.823" LastActivityDate="2023-09-20T09:01:51.823" Title="How to map list items which are in groups in the dictionary values and return keys corrosponding it?" Tags="|python|sequence-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18052" PostTypeId="2" ParentId="18044" CreationDate="2021-11-16T14:10:50.837" Score="2" Body="&lt;p&gt;I tried to used Circos for my MSc Thesis but didn't have enough time to actually learn how to use so I managed to find a less fancy but also cool version of it called &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/26227146/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ClicO.FS&lt;/a&gt; which is an interactive web-based service of Circos (as the title of the article below says).&lt;/p&gt;&#xA;" OwnerUserId="14575" LastActivityDate="2021-11-16T14:10:50.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18054" PostTypeId="2" ParentId="18051" CreationDate="2021-11-16T18:04:43.897" Score="1" Body="&lt;p&gt;I am assuming you only want keys where &lt;em&gt;both&lt;/em&gt; tuple items are present. If so, this should work:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;no_match = []&#xA;for item in list1:&#xA;    item1, item2 = item&#xA;    wanted_keys = [&#xA;        key for key in dict1.keys() if&#xA;        item1 in dict1[key] and&#xA;        item2 in dict1[key]&#xA;    ]&#xA;    if len(wanted_keys) &amp;gt; 0:&#xA;        print(&amp;quot;%s:  %s&amp;quot; % (item, wanted_keys))&#xA;    else:&#xA;        no_match.append(item)&#xA;print(&amp;quot;No matches found for items:&amp;quot;, no_match)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;('A', 'B'):  ['chr1:1000-2000', 'chr3:1200-3000']&#xA;('A', 'C'):  ['chr1:1000-2000', 'chr3:1200-3000']&#xA;('A', 'X'):  ['chr2:999-2000']&#xA;('A', 'Y'):  ['chr2:999-2000']&#xA;('B', 'C'):  ['chr1:1000-2000', 'chr3:1200-3000']&#xA;('X', 'Y'):  ['chr2:999-2000']&#xA;No matches found for items: [('B', 'X'), ('B', 'Y'), ('C', 'X'), ('C', 'Y')]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There is probably a better, more elegant way of doing this, I am not a python person, but this is the best I could come up with.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2021-11-16T18:04:43.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18055" PostTypeId="1" CreationDate="2021-11-16T19:47:52.100" Score="0" ViewCount="30" Body="&lt;p&gt;I am trying to do a synteny test for MHC genes in human and another mammalian organism&lt;/p&gt;&#xA;&lt;p&gt;For this reason, I need to download all paralogs of rabbits from humans and then construct position the coordinates of paralog to test the synteny. However, I am unsure how to download the paralogs from Ensembl and how to perform this test?&lt;/p&gt;&#xA;&lt;p&gt;My main aim is to research synteny between MHC genes of humans, rabbits and mice.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestions on how to do this or alternative bioinformatic methods on how to perform a synteny test would be highly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="12135" LastActivityDate="2021-11-17T09:52:10.000" Title="Synteny test and tools to use for mammalian organisms?" Tags="|ensembl|human-genome|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18056" PostTypeId="2" ParentId="18044" CreationDate="2021-11-17T02:47:55.960" Score="2" Body="&lt;p&gt;Possibly BRIG (blast ring image generator) was used&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://brig.sourceforge.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://brig.sourceforge.net/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I also catalog other tools here if interested &lt;a href=&quot;https://cmdcolin.github.io/awesome-genome-visualization/?latest=true&amp;amp;selected=%23BRIG&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cmdcolin.github.io/awesome-genome-visualization/?latest=true&amp;amp;selected=%23BRIG&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="60" LastActivityDate="2021-11-17T02:47:55.960" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18057" PostTypeId="2" ParentId="18055" CreationDate="2021-11-17T09:52:10.000" Score="1" Body="&lt;p&gt;This is not an easy ask. Firstly, very few species (I think only human and mouse) have alternative haplotypes annotated for MHC. Most species only have the primary assembly, only one possible haplotype of this locus. This means that the haplotype in the primary assembly is not necessarily the same haplotype as you'll find in another species. Since the MHC haplotypes can contain completely different genes, you will necessarily be able to find orthologues and synteny.&lt;/p&gt;&#xA;&lt;p&gt;Secondly, we in Ensembl do not calculate homologues of the genes on the alternative loci, so we only have homologues of the genes that just happen to be on the primary assembly.&lt;/p&gt;&#xA;&lt;p&gt;There is already pre-calculated synteny in Ensembl. &lt;a href=&quot;https://www.ensembl.org/Homo_sapiens/Location/Synteny?db=core&amp;amp;r=6%3A28655199-33350119&amp;amp;otherspecies=Oryctolagus_cuniculus&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's the human MHC locus compared to Rabbit.&lt;/a&gt; This includes all the homologous genes across this locus. However, this is just one arbitrary MHC vs another arbitrary MHC.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2021-11-17T09:52:10.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18058" PostTypeId="1" AcceptedAnswerId="18060" CreationDate="2021-11-17T10:25:38.377" Score="1" ViewCount="58" Body="&lt;p&gt;I'm trying to replicate the figure given in this &lt;a href=&quot;https://journals.plos.org/plosgenetics/article?id=10.1371/journal.pgen.1004085#pgen.1004085.s010&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; using this &lt;a href=&quot;https://doi.org/10.1371/journal.pgen.1004085.s010&quot; rel=&quot;nofollow noreferrer&quot;&gt;table&lt;/a&gt; which they have given.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xwABU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xwABU.png&quot; alt=&quot;Figure 5&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So what i understand from the figure is they have calculated the zscore between two condition and plotted the same.&lt;/p&gt;&#xA;&lt;p&gt;Now How is the length of each bar which is the peptide here is variable is it the zscore or the actual peptide length?&lt;/p&gt;&#xA;&lt;p&gt;Small subset of the data&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dput(head(d2))&#xA;structure(list(Peptide = structure(1:6, .Label = c(&amp;quot;AAHLPAEFTPAVH&amp;quot;, &#xA;&amp;quot;AAHLPAEFTPAVHASLD&amp;quot;, &amp;quot;AAWGKVG&amp;quot;, &amp;quot;AEFTPAVHA&amp;quot;, &amp;quot;AEFTPAVHASLDKFLASVSTVLTSKYR&amp;quot;, &#xA;&amp;quot;AFSDGLAH&amp;quot;, &amp;quot;AFSDGLAHLD&amp;quot;, &amp;quot;AFSDGLAHLDNLK&amp;quot;, &amp;quot;AGEYG&amp;quot;, &amp;quot;AHAGEYG&amp;quot;, &#xA;&amp;quot;AHAGEYGAEALE&amp;quot;, &amp;quot;AHAGEYGAEALERMF&amp;quot;, &amp;quot;AHAGEYGAEALERMFLSFPTTK&amp;quot;, &#xA;&amp;quot;AHGKKVLG&amp;quot;, &amp;quot;AHLDNLK&amp;quot;, &amp;quot;AHLPAEFTPAVH&amp;quot;, &amp;quot;AHLPAEFTPAVHASLD&amp;quot;, &amp;quot;AHLPAEFTPAVHASLDKF&amp;quot;, &#xA;&amp;quot;AHVDDMPNA&amp;quot;, &amp;quot;AHVDDMPNALS&amp;quot;, &amp;quot;ALSDLH&amp;quot;, &amp;quot;ALWGKVNVDEVG&amp;quot;, &amp;quot;ALWGKVNVDEVGGEA&amp;quot;, &#xA;&amp;quot;ALWGKVNVDEVGGEALG&amp;quot;, &amp;quot;ALWGKVNVDEVGGEALGRLL&amp;quot;, &amp;quot;ANALAHKYH&amp;quot;, &amp;quot;ASLDKF&amp;quot;, &#xA;&amp;quot;ASLDKFLA&amp;quot;, &amp;quot;ASLDKFLASVS&amp;quot;, &amp;quot;ASLDKFLASVSTVLT&amp;quot;, &amp;quot;ASLDKFLASVSTVLTS&amp;quot;, &#xA;&amp;quot;ASLDKFLASVSTVLTSKYR&amp;quot;, &amp;quot;ATLS&amp;quot;, &amp;quot;AVAHVDD&amp;quot;, &amp;quot;AVAHVDDMPN&amp;quot;, &amp;quot;AVAHVDDMPNA&amp;quot;, &#xA;&amp;quot;AVAHVDDMPNAL&amp;quot;, &amp;quot;AVAHVDDMPNALS&amp;quot;, &amp;quot;AVAHVDDMPNALSA&amp;quot;, &amp;quot;AYQKVVAGVANALAHK&amp;quot;, &#xA;&amp;quot;DALT&amp;quot;, &amp;quot;DALTN&amp;quot;, &amp;quot;DDMPNA&amp;quot;, &amp;quot;DDMPNALS&amp;quot;, &amp;quot;DEVG&amp;quot;, &amp;quot;DEVGGEA&amp;quot;, &amp;quot;DEVGGEALG&amp;quot;, &#xA;&amp;quot;DEVGGEALGRLL&amp;quot;, &amp;quot;DGLA&amp;quot;, &amp;quot;DGLAH&amp;quot;, &amp;quot;DGLAHLD&amp;quot;, &amp;quot;DGLAHLDNLK&amp;quot;, &amp;quot;DKLHVD&amp;quot;, &#xA;&amp;quot;DKLHVDPEN&amp;quot;, &amp;quot;DKLHVDPENF&amp;quot;, &amp;quot;DKLHVDPENFR&amp;quot;, &amp;quot;DKLHVDPENFRL&amp;quot;, &amp;quot;DKLHVDPENFRLL&amp;quot;, &#xA;&amp;quot;DKLHVDPENFRLLG&amp;quot;, &amp;quot;DKTNVK&amp;quot;, &amp;quot;DLH&amp;quot;, &amp;quot;DLS&amp;quot;, &amp;quot;DLSTPD&amp;quot;, &amp;quot;DLSTPDA&amp;quot;, &#xA;&amp;quot;DMPNALS&amp;quot;, &amp;quot;DPENF&amp;quot;, &amp;quot;DPENFR&amp;quot;, &amp;quot;DPVNF&amp;quot;, &amp;quot;DPVNFK&amp;quot;, &amp;quot;EFTPAVH&amp;quot;, &amp;quot;EFTPAVHASLD&amp;quot;, &#xA;&amp;quot;EFTPAVHASLDKFLASVSTVLTSKYR&amp;quot;, &amp;quot;EFTPPVQ&amp;quot;, &amp;quot;EFTPPVQA&amp;quot;, &amp;quot;EFTPPVQAA&amp;quot;, &#xA;&amp;quot;ESFG&amp;quot;, &amp;quot;ESFGDL&amp;quot;, &amp;quot;ESFGDLSTPDAVM&amp;quot;, &amp;quot;ESFGDLSTPDAVMG&amp;quot;, &amp;quot;EVGG&amp;quot;, &#xA;&amp;quot;EVGGEALG&amp;quot;, &amp;quot;FDLS&amp;quot;, &amp;quot;FESF&amp;quot;, &amp;quot;FESFG&amp;quot;, &amp;quot;FESFGDLS&amp;quot;, &amp;quot;FESFGDLSTPDAVMGNPKVK&amp;quot;, &#xA;&amp;quot;FGDL&amp;quot;, &amp;quot;FGDLS&amp;quot;, &amp;quot;FPHF&amp;quot;, &amp;quot;FPHFD&amp;quot;, &amp;quot;FPHFDL&amp;quot;, &amp;quot;FPHFDLS&amp;quot;, &amp;quot;FPT&amp;quot;, &#xA;&amp;quot;FPTT&amp;quot;, &amp;quot;FPTTKT&amp;quot;, &amp;quot;FPTTKTY&amp;quot;, &amp;quot;FPTTKTYFPHF&amp;quot;, &amp;quot;FSDGLA&amp;quot;, &amp;quot;FSDGLAH&amp;quot;, &#xA;&amp;quot;FSDGLAHLD&amp;quot;, &amp;quot;FSDGLAHLDNLK&amp;quot;, &amp;quot;FTPAVH&amp;quot;, &amp;quot;FTPAVHASLD&amp;quot;, &amp;quot;FTPAVHASLDKFLASVSTVLT&amp;quot;, &#xA;&amp;quot;FTPPVQ&amp;quot;, &amp;quot;FTPPVQA&amp;quot;, &amp;quot;FTPPVQAA&amp;quot;, &amp;quot;GAFSDGLA&amp;quot;, &amp;quot;GAFSDGLAH&amp;quot;, &amp;quot;GAFSDGLAHLDNLK&amp;quot;, &#xA;&amp;quot;GEALG&amp;quot;, &amp;quot;GEYG&amp;quot;, &amp;quot;GGEAL&amp;quot;, &amp;quot;GKKVL&amp;quot;, &amp;quot;GKKVLG&amp;quot;, &amp;quot;GKVNVD&amp;quot;, &amp;quot;GKVNVDE&amp;quot;, &#xA;&amp;quot;GKVNVDEVG&amp;quot;, &amp;quot;GKVNVDEVGGEA&amp;quot;, &amp;quot;GKVNVDEVGGEAL&amp;quot;, &amp;quot;GKVNVDEVGGEALG&amp;quot;, &#xA;&amp;quot;GKVNVDEVGGEALGRL&amp;quot;, &amp;quot;GKVNVDEVGGEALGRLL&amp;quot;, &amp;quot;GLAHLDNLK&amp;quot;, &amp;quot;GNPKV&amp;quot;, &#xA;&amp;quot;GNPKVK&amp;quot;, &amp;quot;GNVL&amp;quot;, &amp;quot;GRLL&amp;quot;, &amp;quot;GTFAT&amp;quot;, &amp;quot;GVANALAHKYH&amp;quot;, &amp;quot;HAGEYG&amp;quot;, &amp;quot;HAGEYGAEALE&amp;quot;, &#xA;&amp;quot;HAGEYGAEALERMF&amp;quot;, &amp;quot;HFDLS&amp;quot;, &amp;quot;HHFG&amp;quot;, &amp;quot;HKYH&amp;quot;, &amp;quot;HLD&amp;quot;, &amp;quot;HLDNLK&amp;quot;, &amp;quot;HLPAE&amp;quot;, &#xA;&amp;quot;HLPAEF&amp;quot;, &amp;quot;HLPAEFTPAVH&amp;quot;, &amp;quot;HLPAEFTPAVHASLD&amp;quot;, &amp;quot;HLTPEEK&amp;quot;, &amp;quot;HVDD&amp;quot;, &#xA;&amp;quot;HVDDMP&amp;quot;, &amp;quot;HVDDMPN&amp;quot;, &amp;quot;HVDDMPNA&amp;quot;, &amp;quot;HVDDMPNAL&amp;quot;, &amp;quot;HVDDMPNALS&amp;quot;, &amp;quot;HVDDMPNALSA&amp;quot;, &#xA;&amp;quot;HVDDMPNALSALS&amp;quot;, &amp;quot;HVDP&amp;quot;, &amp;quot;HVDPEN&amp;quot;, &amp;quot;HVDPENF&amp;quot;, &amp;quot;HVDPENFR&amp;quot;, &amp;quot;HVDPENFRL&amp;quot;, &#xA;&amp;quot;HVDPENFRLLG&amp;quot;, &amp;quot;KAAWGKVG&amp;quot;, &amp;quot;KEFTPPVQ&amp;quot;, &amp;quot;KEFTPPVQA&amp;quot;, &amp;quot;KEFTPPVQAA&amp;quot;, &#xA;&amp;quot;KEFTPPVQAAYQ&amp;quot;, &amp;quot;KLHVDPENFRLLG&amp;quot;, &amp;quot;KTYFP&amp;quot;, &amp;quot;KTYFPH&amp;quot;, &amp;quot;KTYFPHF&amp;quot;, &#xA;&amp;quot;KTYFPHFD&amp;quot;, &amp;quot;KVNVDEVGG&amp;quot;, &amp;quot;KVNVDEVGGEA&amp;quot;, &amp;quot;KVNVDEVGGEALG&amp;quot;, &amp;quot;LDNL&amp;quot;, &#xA;&amp;quot;LDNLK&amp;quot;, &amp;quot;LGAFSDGLA&amp;quot;, &amp;quot;LHVDP&amp;quot;, &amp;quot;LHVDPENF&amp;quot;, &amp;quot;LHVDPENFRLLG&amp;quot;, &amp;quot;LKGTFATL&amp;quot;, &#xA;&amp;quot;LLVVYPWTQRF&amp;quot;, &amp;quot;LPAE&amp;quot;, &amp;quot;LPAEF&amp;quot;, &amp;quot;LPAEFTPAVH&amp;quot;, &amp;quot;LPAEFTPAVHASLD&amp;quot;, &#xA;&amp;quot;LPAEFTPAVHASLDK&amp;quot;, &amp;quot;LPAEFTPAVHASLDKF&amp;quot;, &amp;quot;LSALS&amp;quot;, &amp;quot;LSFPTTK&amp;quot;, &amp;quot;LSFPTTKT&amp;quot;, &#xA;&amp;quot;LSFPTTKTY&amp;quot;, &amp;quot;LSFPTTKTYFPHF&amp;quot;, &amp;quot;LSFPTTKTYFPHFDL&amp;quot;, &amp;quot;LSFPTTKTYFPHFDLS&amp;quot;, &#xA;&amp;quot;LSFPTTKTYFPHFDLSHG&amp;quot;, &amp;quot;LSTPDA&amp;quot;, &amp;quot;LTPEEK&amp;quot;, &amp;quot;LTPEEKSAVT&amp;quot;, &amp;quot;LTPEEKSAVTA&amp;quot;, &#xA;&amp;quot;LTPEEKSAVTAL&amp;quot;, &amp;quot;LTPEEKSAVTALW&amp;quot;, &amp;quot;LTPEEKSAVTALWG&amp;quot;, &amp;quot;LTPEEKSAVTALWGKVNVD&amp;quot;, &#xA;&amp;quot;LTPEEKSAVTALWGKVNVDEVG&amp;quot;, &amp;quot;LTPEEKSAVTALWGKVNVDEVGGEA&amp;quot;, &amp;quot;LTPEEKSAVTALWGKVNVDEVGGEALG&amp;quot;, &#xA;&amp;quot;LVVYPWTQRF&amp;quot;, &amp;quot;LVVYPWTQRFF&amp;quot;, &amp;quot;LWGKVNVDEVGGEALG&amp;quot;, &amp;quot;NAVAHVDD&amp;quot;, &#xA;&amp;quot;NAVAHVDDMPNA&amp;quot;, &amp;quot;NAVAHVDDMPNAL&amp;quot;, &amp;quot;NAVAHVDDMPNALS&amp;quot;, &amp;quot;NAVAHVDDMPNALSA&amp;quot;, &#xA;&amp;quot;NAVAHVDDMPNALSALS&amp;quot;, &amp;quot;NAVAHVDDMPNALSALSDL&amp;quot;, &amp;quot;NAVAHVDDMPNALSALSDLHAHKLR&amp;quot;, &#xA;&amp;quot;NVDEVG&amp;quot;, &amp;quot;NVDEVGG&amp;quot;, &amp;quot;NVDEVGGEA&amp;quot;, &amp;quot;NVDEVGGEALG&amp;quot;, &amp;quot;PADKTNVK&amp;quot;, &#xA;&amp;quot;PAEF&amp;quot;, &amp;quot;PAEFTPAVH&amp;quot;, &amp;quot;PAEFTPAVHASLD&amp;quot;, &amp;quot;PAVH&amp;quot;, &amp;quot;PDAV&amp;quot;, &amp;quot;PDAVM&amp;quot;, &#xA;&amp;quot;PDAVMG&amp;quot;, &amp;quot;PEE&amp;quot;, &amp;quot;PEEK&amp;quot;, &amp;quot;PEEKSAVT&amp;quot;, &amp;quot;PEEKSAVTAL&amp;quot;, &amp;quot;PENF&amp;quot;, &amp;quot;PENFR&amp;quot;, &#xA;&amp;quot;PENFRLLG&amp;quot;, &amp;quot;PHF&amp;quot;, &amp;quot;PHFD&amp;quot;, &amp;quot;PHFDL&amp;quot;, &amp;quot;PHFDLS&amp;quot;, &amp;quot;PKVK&amp;quot;, &amp;quot;PNAL&amp;quot;, &#xA;&amp;quot;PNALS&amp;quot;, &amp;quot;PPVQ&amp;quot;, &amp;quot;PPVQA&amp;quot;, &amp;quot;PPVQAA&amp;quot;, &amp;quot;PVNF&amp;quot;, &amp;quot;PVNFK&amp;quot;, &amp;quot;PVQ&amp;quot;, &amp;quot;PVQA&amp;quot;, &#xA;&amp;quot;PWTQ&amp;quot;, &amp;quot;PWTQRF&amp;quot;, &amp;quot;PWTQRFF&amp;quot;, &amp;quot;QVKGHGKKVADALTN&amp;quot;, &amp;quot;RVDPVNF&amp;quot;, &amp;quot;SDGLA&amp;quot;, &#xA;&amp;quot;SDGLAHLDN&amp;quot;, &amp;quot;SDGLAHLDNLK&amp;quot;, &amp;quot;SDLH&amp;quot;, &amp;quot;SFGD&amp;quot;, &amp;quot;SFGDL&amp;quot;, &amp;quot;SFGDLS&amp;quot;, &#xA;&amp;quot;SFGDLSTPDAVMG&amp;quot;, &amp;quot;SLDKF&amp;quot;, &amp;quot;SPADK&amp;quot;, &amp;quot;SPADKTNVK&amp;quot;, &amp;quot;SVSTVLT&amp;quot;, &amp;quot;TKTYFPHF&amp;quot;, &#xA;&amp;quot;TNVK&amp;quot;, &amp;quot;TPAVH&amp;quot;, &amp;quot;TPAVHASLD&amp;quot;, &amp;quot;TPAVHASLDKFLASVSTVLT&amp;quot;, &amp;quot;TPDAVM&amp;quot;, &#xA;&amp;quot;TPDAVMG&amp;quot;, &amp;quot;TPDAVMGNPKVK&amp;quot;, &amp;quot;TPEEKSAVT&amp;quot;, &amp;quot;TPEEKSAVTAL&amp;quot;, &amp;quot;TPEEKSAVTALW&amp;quot;, &#xA;&amp;quot;TPPVQ&amp;quot;, &amp;quot;TSKY&amp;quot;, &amp;quot;TVLT&amp;quot;, &amp;quot;TYFP&amp;quot;, &amp;quot;TYFPH&amp;quot;, &amp;quot;TYFPHF&amp;quot;, &amp;quot;TYFPHFD&amp;quot;, &#xA;&amp;quot;TYFPHFDLS&amp;quot;, &amp;quot;VAHVDD&amp;quot;, &amp;quot;VAHVDDMPNA&amp;quot;, &amp;quot;VAHVDDMPNALS&amp;quot;, &amp;quot;VAHVDDMPNALSA&amp;quot;, &#xA;&amp;quot;VDDMP&amp;quot;, &amp;quot;VDDMPN&amp;quot;, &amp;quot;VDDMPNA&amp;quot;, &amp;quot;VDDMPNAL&amp;quot;, &amp;quot;VDDMPNALS&amp;quot;, &amp;quot;VDEVG&amp;quot;, &#xA;&amp;quot;VDEVGG&amp;quot;, &amp;quot;VDEVGGEA&amp;quot;, &amp;quot;VDEVGGEALG&amp;quot;, &amp;quot;VDPENF&amp;quot;, &amp;quot;VDPENFR&amp;quot;, &amp;quot;VDPENFRL&amp;quot;, &#xA;&amp;quot;VDPENFRLL&amp;quot;, &amp;quot;VDPENFRLLG&amp;quot;, &amp;quot;VDPVN&amp;quot;, &amp;quot;VDPVNF&amp;quot;, &amp;quot;VDPVNFK&amp;quot;, &amp;quot;VDPVNFKL&amp;quot;, &#xA;&amp;quot;VDPVNFKLL&amp;quot;, &amp;quot;VDPVNFKLLS&amp;quot;, &amp;quot;VHL&amp;quot;, &amp;quot;VHLT&amp;quot;, &amp;quot;VHLTP&amp;quot;, &amp;quot;VHLTPE&amp;quot;, &#xA;&amp;quot;VHLTPEE&amp;quot;, &amp;quot;VHLTPEEK&amp;quot;, &amp;quot;VHLTPEEKSAVT&amp;quot;, &amp;quot;VHLTPEEKSAVTAL&amp;quot;, &amp;quot;VHLTPEEKSAVTALWGKVNVD&amp;quot;, &#xA;&amp;quot;VHLTPEEKSAVTALWGKVNVDE&amp;quot;, &amp;quot;VHLTPEEKSAVTALWGKVNVDEVG&amp;quot;, &amp;quot;VHLTPEEKSAVTALWGKVNVDEVGGEA&amp;quot;, &#xA;&amp;quot;VHLTPEEKSAVTALWGKVNVDEVGGEALG&amp;quot;, &amp;quot;VHLTPEEKSAVTALWGKVNVDEVGGEALGRLL&amp;quot;, &#xA;&amp;quot;VKAAWGKVGAHAGEY&amp;quot;, &amp;quot;VLSP&amp;quot;, &amp;quot;VLSPA&amp;quot;, &amp;quot;VLSPAD&amp;quot;, &amp;quot;VLSPADK&amp;quot;, &amp;quot;VLSPADKT&amp;quot;, &#xA;&amp;quot;VLSPADKTN&amp;quot;, &amp;quot;VLSPADKTNVK&amp;quot;, &amp;quot;VLSPADKTNVKAA&amp;quot;, &amp;quot;VLSPADKTNVKAAWGKVG&amp;quot;, &#xA;&amp;quot;VNVDE&amp;quot;, &amp;quot;VNVDEVG&amp;quot;, &amp;quot;VNVDEVGG&amp;quot;, &amp;quot;VNVDEVGGEA&amp;quot;, &amp;quot;VNVDEVGGEAL&amp;quot;, &#xA;&amp;quot;VNVDEVGGEALG&amp;quot;, &amp;quot;VNVDEVGGEALGRL&amp;quot;, &amp;quot;VVAGVANALA&amp;quot;, &amp;quot;VVYPWTQRF&amp;quot;, &#xA;&amp;quot;VVYPWTQRFF&amp;quot;, &amp;quot;VVYPWTQRFFE&amp;quot;, &amp;quot;VVYPWTQRFFESFGDLS&amp;quot;, &amp;quot;VYPWTQRF&amp;quot;, &#xA;&amp;quot;VYPWTQRFF&amp;quot;, &amp;quot;VYPWTQRFFE&amp;quot;, &amp;quot;WGKVG&amp;quot;, &amp;quot;WGKVNVD&amp;quot;, &amp;quot;WGKVNVDE&amp;quot;, &amp;quot;WGKVNVDEVG&amp;quot;, &#xA;&amp;quot;WGKVNVDEVGGEA&amp;quot;, &amp;quot;WGKVNVDEVGGEAL&amp;quot;, &amp;quot;WGKVNVDEVGGEALG&amp;quot;, &amp;quot;WGKVNVDEVGGEALGRL&amp;quot;, &#xA;&amp;quot;WGKVNVDEVGGEALGRLL&amp;quot;, &amp;quot;YFPHFDLS&amp;quot;, &amp;quot;YPWT&amp;quot;, &amp;quot;YPWTQ&amp;quot;, &amp;quot;YPWTQR&amp;quot;, &#xA;&amp;quot;YPWTQRF&amp;quot;, &amp;quot;YPWTQRFF&amp;quot;, &amp;quot;YPWTQRFFE&amp;quot;), class = &amp;quot;factor&amp;quot;), C2 = c(-1.13005927266738, &#xA;-1.15451123584162, -0.771975424114207, -0.891503793435265, 0.686928914375861, &#xA;-1.15469327326147), C4 = c(0.770515719059141, 0.559148858503814, &#xA;1.12965461223673, 1.08129555353272, 0.46033682419062, 0.58089396147208&#xA;), C6 = c(0.359543553608241, 0.595362377337809, -0.357679188122518, &#xA;-0.18979176009746, -1.14726573856648, 0.573799311789392)), row.names = c(NA, &#xA;6L), class = &amp;quot;data.frame&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any suggestion or help would be really appreciated&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2021-11-17T11:06:55.437" Title="Peptide data visualization" Tags="|visualization|proteomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18059" PostTypeId="1" AcceptedAnswerId="18064" CreationDate="2021-11-17T10:48:40.287" Score="1" ViewCount="2203" Body="&lt;p&gt;I have been working on FindMarkers function for identifying significant genes in the cluster. But some Significant genes have very low p values, so they are returned as 0 in the output.Any value less than the .machine$double.xmin limit on R(that is 1e-305). So this output cannot be used for further plots like Volcano plots for visualization.&#xA;Any solution to get p values for all the genes?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cd4.markers &amp;lt;- FindMarkers(sc.combined, ident.1 = &amp;quot;CD4+ T cell&amp;quot;, min.pct = 0.25)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Part of the cd4.markers output&#xA;&lt;a href=&quot;https://i.stack.imgur.com/8Q9cr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8Q9cr.png&quot; alt=&quot;Output sample for cd4.markers&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14576" LastActivityDate="2021-12-11T19:31:39.760" Title="FindMarkers from Seurat returns p values as 0 for highly significant genes" Tags="|r|scrnaseq|seurat|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18060" PostTypeId="2" ParentId="18058" CreationDate="2021-11-17T11:06:55.437" Score="2" Body="&lt;p&gt;The length of the bars and their placement on the x-axis represents the length of the peptide and mapping to the source protein, respectively. The coloring represents the average difference in peptide MS intensity between the control and two experiments.&lt;/p&gt;&#xA;" OwnerUserId="14577" LastActivityDate="2021-11-17T11:06:55.437" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18063" PostTypeId="1" CreationDate="2021-11-17T15:19:31.840" Score="1" ViewCount="33" Body="&lt;p&gt;I got a rejection of publication which the reviewers mainly point out as not following a reasonable evolutionary criterion for choice of species used for phylogeny creation. In our dataset, a list of 100+ eukaryotic species was used to avoid bias. What exactly is a good &amp;quot;reasonable evolutionary criterion&amp;quot;? We are studying an ancient protein (receptor).phylogeny is created based on protein sequences.Can someone provide insight into this? Any suggestions would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="11882" LastEditorUserId="11882" LastEditDate="2021-11-18T12:56:35.807" LastActivityDate="2021-11-18T12:56:35.807" Title="What exactly is a reasonable evolutionary criterion?" Tags="|phylogenetics|phylogeny|molecular-genetics|" AnswerCount="0" CommentCount="0" ClosedDate="2021-11-17T21:22:37.203" ContentLicense="CC BY-SA 4.0" />
  <row Id="18064" PostTypeId="2" ParentId="18059" CreationDate="2021-11-17T21:30:44.650" Score="1" Body="&lt;p&gt;I prefer to apply a threshold when showing Volcano plots, displaying any points with extreme / impossible p-values (e.g. &amp;lt; 10e-20) with a different symbol at the top of the graph.&lt;/p&gt;&#xA;&lt;p&gt;However, a better approach is to avoid using p-values as quantitative / rankable results in plots; they're not meant to be used in that way. You can show results as a MA plot instead, plotting log2 fold change vs average expression:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pIE45.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pIE45.jpg&quot; alt=&quot;MA plot of log2 fold change vs mean normalised expression&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It's a bit trickier to get expression values out of Seurat because they're not currently calculated in the FindMarkers results tables, so you'll need to manually subset the cells and calculate mean expression on a per-marker basis.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-11-17T21:30:44.650" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18065" PostTypeId="1" CreationDate="2021-11-17T23:08:37.877" Score="3" ViewCount="526" Body="&lt;p&gt;I'm working on a project where I am analyzing the performance of an alignment workflow. My goal is to find regions in the resulting BAM file where there are outstanding discrepancies or anything that indicates my assembly/alignment has &amp;quot;mistakes&amp;quot;. I'm working with human genomes from the 1000 genomes project, they are high coverage Illumina paired end reads, each read is 100bp.&lt;/p&gt;&#xA;&lt;p&gt;My workflow so far:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;input paired end FASTQ files (human) into SPAdes to create contigs.fasta file&lt;/li&gt;&#xA;&lt;li&gt;align paired end FASTQ files to contigs.fasta file with minimap2 to get SAM file&lt;/li&gt;&#xA;&lt;li&gt;convert SAM to BAM with samtools&lt;/li&gt;&#xA;&lt;li&gt;sort and index BAM file&lt;/li&gt;&#xA;&lt;li&gt;load BAM file into IGV and load contigs.fasta as the reference&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In the IGV GUI, I have colored alignments by insert size and pair orientation. This has revealed a bunch of insertions, tandem duplications, and inversions.&lt;/p&gt;&#xA;&lt;p&gt;I am wondering:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Do these colored reads indicate that the assembler made &amp;quot;mistakes&amp;quot; or could they be there just by chance?&lt;/li&gt;&#xA;&lt;li&gt;Do SNPs indicate anything about how SPAdes/minimap2/samtools performed?&lt;/li&gt;&#xA;&lt;li&gt;Is there a better way to sort/color the alignments based on what I'm trying to do?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Thank you so much to anyone who has any advice!&lt;/p&gt;&#xA;&lt;p&gt;EDIT: here is a screenshot of the alignment. For clarification, there are 8 genomes of interest, hence 8 tracks&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BVvW7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BVvW7.png&quot; alt=&quot;image&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&#xA;Integrative genome viewer (IGV) is a well known Broad Institute tool and available &lt;a href=&quot;https://software.broadinstitute.org/software/igv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The most recent papers are:&#xA;James T. Robinson, Helga Thorvaldsdóttir, Aaron M. Wenger, Ahmet Zehir, Jill P. Mesirov; Variant Review with the Integrative Genomics Viewer. Cancer Res 1 November 2017; 77 (21): e31–e34. &lt;a href=&quot;https://doi.org/10.1158/0008-5472.CAN-17-0337&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1158/0008-5472.CAN-17-0337&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;igv.js: an embeddable JavaScript implementation of the Integrative Genomics Viewer (IGV)&#xA;James T. Robinson, Helga Thorvaldsdóttir, Douglass Turner, Jill P. Mesirov&#xA;bioRxiv 2020.05.03.075499; doi: &lt;a href=&quot;https://doi.org/10.1101/2020.05.03.075499&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1101/2020.05.03.075499&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12208" LastEditorUserId="3848" LastEditDate="2022-03-24T14:46:48.893" LastActivityDate="2022-03-24T14:46:48.893" Title="How to analyze an IGV alignment" Tags="|samtools|visualization|igv|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="18066" PostTypeId="2" ParentId="15406" CreationDate="2021-11-18T08:38:59.527" Score="2" Body="&lt;p&gt;I don't know if it's too late, but I met same question and got an answer.&#xA;You can use &lt;code&gt;add_proteins_description()&lt;/code&gt;. It works in same way with &lt;code&gt;map()&lt;/code&gt;. If you put a data frame having &lt;code&gt;STRING_id&lt;/code&gt; column, it adds &lt;code&gt;preferred_name&lt;/code&gt;, &lt;code&gt;protein_size&lt;/code&gt; and &lt;code&gt;annotation&lt;/code&gt;. You may find your answer on &lt;code&gt;preferred_name&lt;/code&gt; column.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;require(STRINGdb)&#xA;string_db &amp;lt;- STRINGdb&lt;span class=&quot;math-container&quot;&gt;$new()&#xA;string_db$&lt;/span&gt;add_proteins_description(data.frame(STRING_id = [vector containing string ids]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14585" LastEditorUserId="181" LastEditDate="2021-11-20T06:31:38.230" LastActivityDate="2021-11-20T06:31:38.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18067" PostTypeId="1" CreationDate="2021-11-18T17:24:53.297" Score="0" ViewCount="35" Body="&lt;p&gt;I'm a computer scientist who is now getting into designing new molecules, with the goal that they dock well to a given target. While I'm used to coding up machine learning algorithms in python, I'm finding things a bit murky in terms of the how-to of docking. I lack pretty much any (medicinal) chemistry/biomedical engineering background and so understanding how to get things to work has been a real struggle.&lt;/p&gt;&#xA;&lt;p&gt;In particular I've been looking at two githubs: &lt;a href=&quot;https://github.com/cieplinski-tobiasz/smina-docking-benchmark&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one using smina&lt;/a&gt; and &lt;a href=&quot;https://github.com/cstein/GB-GA/tree/feature-glide_docking&quot; rel=&quot;nofollow noreferrer&quot;&gt;this one using glide&lt;/a&gt;. Recently, my collaborators have steered me toward rDock/RxDoc.  Near as I can tell the basic ingredients for ANY docking software are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;a target file (rna or protein) in a .pdbdt,.pdb, or .mol2 file with&#xA;it docked to any ligand&lt;/li&gt;&#xA;&lt;li&gt;a ligand file (mol2)&lt;/li&gt;&#xA;&lt;li&gt;a target file with the ligand removed&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;But where I get lost is in any intuition on how to prepare these files, how to find them, and any intuition &lt;em&gt;at all&lt;/em&gt; about how to make these work. I've seen suggestions of openbabel, etc, but there seem to be no tutorials online.&lt;/p&gt;&#xA;&lt;p&gt;So, my question is: Suppose I have a molecule in &lt;code&gt;rdkit&lt;/code&gt; and a protein or an RNA that I want to dock it to, accessed from the protein databank, say. What is the ``right'' way to do the following:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Download the receptor from pdb&lt;/li&gt;&#xA;&lt;li&gt;Process the receptor to get file of the receptor with and without ligand&lt;/li&gt;&#xA;&lt;li&gt;Process a molecule from &lt;code&gt;rdkit&lt;/code&gt; so that the docking software can dock a (new) molecule to the receptor and give a docking score?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Ideally this should work for `any' receptor, but if the steps above need to be based on the specifics of the receptor, that would be helpful for me to know.&lt;/p&gt;&#xA;&lt;p&gt;I'm shocked there isn't a beginners guide on this out there that's more easy to find, and I worry about going down this route without any intuition.&lt;/p&gt;&#xA;" OwnerUserId="9487" LastActivityDate="2021-11-18T17:24:53.297" Title="Basic Docking Work Flow" Tags="|docking|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18068" PostTypeId="1" CreationDate="2021-11-18T21:04:13.750" Score="1" ViewCount="82" Body="&lt;p&gt;I have a list of miRNAs IDs (2000-2500) that I want to find miRBase IDs for them. For example:&lt;/p&gt;&#xA;&lt;p&gt;hsa-miR-106a ---&amp;gt; hsa-miR-106a-5p&#xA;hsa-miR-373* --&amp;gt; hsa-miR-373-5p&#xA;hsa-miR-33 --&amp;gt; hsa-miR-33a-5p&#xA;I used many R packages and websites such as (miRBaseConverter, miRNAmeConverter, MiEAA, miRSystem,miRandola) but non of them give me the comprehensive result.&lt;/p&gt;&#xA;&lt;p&gt;e.g. For &amp;quot;hsa-mir-30c-1&amp;quot; miRNAmeConverter did not give miRBase ID but if you search it in miRBase it hsa two matures IDs: hsa-miR-30c-5p &amp;amp; hsa-miR-30c-1-3p.&lt;/p&gt;&#xA;&lt;p&gt;miRBAseConverter is also like the mentioned one: It can not find the mature one for many miRNAs e.g. &amp;quot;hsa-mir-34&amp;quot; has to mature IDs: &amp;quot;hsa-miR-34a-5p&amp;quot; &amp;amp; &amp;quot;hsa-miR-34a-3p&amp;quot; but despite having them miRBaseConverter could not retrieve them.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/X88g9.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/X88g9.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can anyone suggest any tools that might help to do this?&lt;/p&gt;&#xA;" OwnerUserId="14588" LastActivityDate="2021-11-18T21:04:13.750" Title="Converting miRNA names to miRBase version IDs" Tags="|r|python|bioconductor|computational-biochemistry|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18069" PostTypeId="1" CreationDate="2021-11-18T23:16:04.117" Score="0" ViewCount="1104" Body="&lt;p&gt;I'm getting an error when creating the counts from a single cell experiment.I'm identifying shared barcodes as well as lowUMI barcodes with the follow code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;shared = colnames(cts[[i]])[which(colnames(cts[[i]]) %in% lb[[libID]])]&#xA;lowUMI = colnames(cts[[i]])[colSums(cts[[i]])&amp;lt;1000]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but my code is erroring at&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;lowUMI = colnames(cts[[i]])[colSums(cts[[i]])&amp;lt;1000]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;with this error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in colSums(cts[[i]]) : 'x' must be an array of at least two dimensions&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My full code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cts=list()&#xA;for (i in 1:length(mf$LibraryID)){&#xA;  # Read in all four input expression matrices&#xA;  message(i)&#xA;  message(mf$LibraryID[i])&#xA;  libID = mf$LibraryID[i]&#xA;  #lb[[libID]] = gsub(&amp;quot;-1&amp;quot;, &amp;quot;&amp;quot;, lb[[libID]])&#xA;  lb[[libID]] = unique(lb[[libID]])&#xA;  ## by default we point to the 10X dir&#xA;  #data.dir = paste0(wd, mf$GSM[i], &amp;quot;/cellranger_out/&amp;quot;,mf$LibraryID[i],&amp;quot;/filtered_feature_bc_matrix/&amp;quot;) &#xA;  data.dir = paste0(wd2, mf$GSM[i], &amp;quot;/metadata/cellranger_ASI_run/&amp;quot;,mf$LibraryID[i],&amp;quot;/filtered_feature_bc_matrix/&amp;quot;)&#xA;      if (!dir.exists(data.dir)) data.dir = paste0(wd3, mf$GSM[i], &amp;quot;/&amp;quot;, mf$LibraryID[i],&amp;quot;/outs/filtered_feature_bc_matrix/&amp;quot;)&#xA;  if (!dir.exists(data.dir)) data.dir = paste0(wd2, mf$GSM[i], &amp;quot;/metadata/cellranger_ASI_run/&amp;quot;,mf$LibraryID[i],&amp;quot;/outs/filtered_feature_bc_matrix/&amp;quot;)&#xA;  if (dir.exists(data.dir)){&#xA;    message(data.dir)&#xA;    cts[[i]] &amp;lt;- Read10X(data.dir)&#xA;    #colnames(cts[[i]]) = gsub(&amp;quot;-1&amp;quot;, &amp;quot;&amp;quot;, colnames(cts[[i]]))&#xA;    shared = colnames(cts[[i]])[which(colnames(cts[[i]]) %in% lb[[libID]])]&#xA;    lowUMI = colnames(cts[[i]])[colSums(cts[[i]])&amp;lt;1000]&#xA;    to_filter = intersect(shared, lowUMI)&#xA;    #to_filter = shared&#xA;    ok = colnames(cts[[i]])[!(colnames(cts[[i]]) %in% to_filter)]&#xA;    cts[[i]] = cts[[i]][, ok]&#xA;    colnames(cts[[i]])=paste0(mf$LibraryID[i], &amp;quot;_&amp;quot;, colnames(cts[[i]]))&#xA;    if (i==1) {&#xA;      cts_all=cts[[1]] &#xA;    } else {&#xA;      cts_all = cbind(cts_all, cts[[i]])&#xA;    } &#xA;  }&#xA;}  &#xA;dim(cts_all)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6827" LastActivityDate="2021-11-18T23:16:04.117" Title="Error in colSums(cts[[i]]) : 'x' must be an array of at least two dimensions" Tags="|r|seurat|single-cell|10x-genomics|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18070" PostTypeId="1" CreationDate="2021-11-19T05:08:24.630" Score="1" ViewCount="172" Body="&lt;p&gt;I have into a dictionary some sequences. I would like to replace the sequences name for new names contained into a dataframe.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#my sequences into a dictionary&#xA;sequence = {&amp;quot;seq01&amp;quot;: &amp;quot;agatcatggctaactatgcacgtgca&amp;quot;,             &#xA;           &amp;quot;seq02&amp;quot;: &amp;quot;atatcatggctaactatgcacgtgca&amp;quot;,&#xA;           &amp;quot;seq03&amp;quot;: &amp;quot;tgatcatggctaactatgcacgtgca&amp;quot;}&#xA;&#xA;#my data frame containing old names and new names&#xA;data = {&amp;quot;old_name&amp;quot;: [&amp;quot;seq01&amp;quot;, &amp;quot;seq02&amp;quot;,&amp;quot;seq03&amp;quot;], &amp;quot;new_name&amp;quot;: [&amp;quot;sample1&amp;quot;, &amp;quot;sample2&amp;quot;, &amp;quot;sample3&amp;quot;]}&#xA;data = pd.DataFrame(data)&#xA;print(data)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried to replace the old names with a for loop, but it was imposible to do it.&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="12707" LastActivityDate="2021-11-22T06:40:22.617" Title="How do you replace multiple sequence names with new names using python?" Tags="|fasta|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18071" PostTypeId="2" ParentId="18070" CreationDate="2021-11-19T12:34:33.557" Score="4" Body="&lt;p&gt;I am pretty sure there would be more elegant, in-place solutions but here is a quick solution to your problem. Should be OK if your &lt;code&gt;sequence&lt;/code&gt; dictionary is not way too large.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seq_names = zip(data['old_name'], data['new_name']) # brings together old and new names in tuples&#xA;renamed_sequence = {}&#xA;for i in seq_names:&#xA;    renamed_sequence[i[1]] = sequence[i[0]]&#xA;renamed_sequence&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;EDIT: The section below is not quite accurate, please see the comments below.&lt;/p&gt;&#xA;&lt;p&gt;And regarding your comment:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I tried to replace the old names with a for loop, but it was imposible&#xA;to do it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You are not supposed to loop over and at the same time modify an element at the same time anyway (there might be side effects that you would not think of), hence a brand new dictionary in my code.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2021-11-22T06:40:22.617" LastActivityDate="2021-11-22T06:40:22.617" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18073" PostTypeId="1" CreationDate="2021-11-19T15:52:51.590" Score="0" ViewCount="41" Body="&lt;p&gt;hi we have transgenic mice with human gene inserted. if we profom rnaseq for the mice, how can we get the expression value of the human gene inserted to the mice? and use this expression to compare with other mice gene expression?&lt;/p&gt;&#xA;&lt;p&gt;the mice ref genome doesn't have this piece of human gene sequence. anybody knows how?&lt;/p&gt;&#xA;" OwnerUserId="14594" LastActivityDate="2022-04-18T17:06:10.570" Title="how can I get expression of an inserted foreign genes?" Tags="|rna-seq|transcriptome|reference-genome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18074" PostTypeId="1" AcceptedAnswerId="18129" CreationDate="2021-11-19T16:16:44.803" Score="1" ViewCount="42" Body="&lt;p&gt;I am using Lexogen's Quantitative Sequencing Pool Analysis tool (here: &lt;a href=&quot;https://github.com/Lexogen-Tools/quantseqpool_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Lexogen-Tools/quantseqpool_analysis&lt;/a&gt;) to analyze R1 and R2 files.&lt;/p&gt;&#xA;&lt;p&gt;I have been able to successfully run this tool. However, I need to downsample and then normalize these results so I need to do this all again.&lt;/p&gt;&#xA;&lt;p&gt;To downsample, I put the R1s (the ones generated after idemultiplexing) through the SeqTK_sample tool on UseGalaxy. Afterwards, I ran the QSPA tool with the new R1s and old R2s. I skipped the idemultiplexing part by # out the code. This is generating errors (screenshots are attached). I don't think these errors are from me # out the code but rather from the new R1s with the old R2s. I didn't receive any errors the first time I ran this tool.&lt;/p&gt;&#xA;&lt;p&gt;I will be normalizing my data with DESeq2 using the summary_unique.tsv file.&lt;/p&gt;&#xA;&lt;p&gt;If anyone has any tips/suggestions it would be much appreciated. I am sure there is a better way to do this. If you require anything from me just ask!&lt;/p&gt;&#xA;&lt;p&gt;Cheers,&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/L7UtS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/L7UtS.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Cqyex.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Cqyex.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14593" LastActivityDate="2021-12-01T19:24:50.243" Title="How should I downsample and normalize R1s and R2s and incorporate this into Lexogen's QSPA tool" Tags="|sequence-analysis|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18075" PostTypeId="2" ParentId="18073" CreationDate="2021-11-19T16:24:36.013" Score="1" Body="&lt;p&gt;You will need to create (and probably index) a &amp;quot;custom&amp;quot; genome/transcriptome, mouse + whatever human genes inserted, depending on the alignment method of choice. And how to do so would depend on the your tool of choice.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2021-11-19T16:24:36.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18076" PostTypeId="1" CreationDate="2021-11-19T17:20:21.730" Score="0" ViewCount="128" Body="&lt;p&gt;Can anybody please explain the below line to me.&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;We took the structure and coordinates of nogalamycin&#xA;from the X-ray structure determined in PDB code = 1D17&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;What do I need to download from there....a pdb file or what?&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="14439" LastActivityDate="2022-04-21T19:02:45.257" Title="Help me in understanding the PDB file" Tags="|protein-structure|phylogenetics|pdb|drugs|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="18078" PostTypeId="1" AcceptedAnswerId="18080" CreationDate="2021-11-19T21:16:57.870" Score="1" ViewCount="41" Body="&lt;p&gt;I have a binary file which looks like this,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Orthogroup  A   B   C   D   E   F&#xA;OG0000000   1   1   1   0   0   0&#xA;OG0000001   1   1   1   0   0   0&#xA;OG0000002   1   1   1   0   0   0&#xA;OG0000003   1   1   1   0   0   0&#xA;OG0000004   0   0   0   1   1   0&#xA;OG0000005   0   0   0   1   1   0&#xA;OG0000006   0   0   0   1   1   0&#xA;OG0000007   0   0   0   1   1   0&#xA;OG0000008   1   0   0   0   0   0&#xA;OG0000009   1   0   0   0   0   0&#xA;OG0000010   1   0   0   0   0   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to use awk to print the first column if column D and E is 1 and rest is 0.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;OG0000004&#xA;OG0000005&#xA;OG0000006&#xA;OG0000007&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried to do this with individual files of each column  using the awk command&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'NR==FNR{arr[&lt;span class=&quot;math-container&quot;&gt;$0];next} $&lt;/span&gt;0 in arr' file1 file2 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but doing this I had to generate multiple files and I thought using a binary file will be a more clean way.&lt;/p&gt;&#xA;&lt;p&gt;Please Help,&lt;/p&gt;&#xA;" OwnerUserId="14569" LastEditorUserId="14569" LastEditDate="2021-11-19T21:47:20.270" LastActivityDate="2021-11-19T21:47:20.270" Title="Print first colum is the condition is true using awk" Tags="|linux|awk|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18079" PostTypeId="2" ParentId="18078" CreationDate="2021-11-19T21:34:20.717" Score="2" Body="&lt;p&gt;Straightforwardly&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;awk '$2==0 &amp;amp;&amp;amp; $3==0 &amp;amp;&amp;amp; $4==0 &amp;amp;&amp;amp; $5==1 &amp;amp;&amp;amp; $6==1 &amp;amp;&amp;amp; $7==0 {print $1}' bin.file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A little spiffier&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;awk '$5 &amp;amp;&amp;amp; $6 &amp;amp;&amp;amp; !($2 || $3 || $4 || $7) {print $1}' bin.file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2131" LastActivityDate="2021-11-19T21:34:20.717" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18080" PostTypeId="2" ParentId="18078" CreationDate="2021-11-19T21:36:44.713" Score="2" Body="&lt;p&gt;I would do something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat test.tsv &#xA;groupA  1       1       1       0       0       0&#xA;groupB  1       1       0       0       0       0&#xA;groupC  1       0       1       0       0       0&#xA;groupD  1       1       1       0       0       0&#xA;groupE  0       0       0       1       1       1&#xA;groupF  0       0       0       1       1       0&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;$ awk '($&lt;/span&gt;5==1 &amp;amp;&amp;amp; &lt;span class=&quot;math-container&quot;&gt;$6==1 &amp;amp;&amp;amp; $&lt;/span&gt;2+&lt;span class=&quot;math-container&quot;&gt;$3+$&lt;/span&gt;4+&lt;span class=&quot;math-container&quot;&gt;$7==0){print $&lt;/span&gt;1}'&#xA;groupF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2021-11-19T21:36:44.713" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18082" PostTypeId="1" CreationDate="2021-11-21T14:54:32.443" Score="1" ViewCount="226" Body="&lt;p&gt;I have some RNA-seq data from a stranded paired end library prep, with dUTP and UDG preparation, so the orientation should be RF (confirmed with sequencing provider). I assembled the reads with Trinity, specifying RF for the strandedness, and I am mapping the reads to the transcripts with bwa-mem (I am aware bowtie2 or STAR might be better for this, but bwa is what I presently have installed, so just using it to check).&lt;/p&gt;&#xA;&lt;p&gt;In the bwa output, I see lines &lt;code&gt;# candidate unique pairs for (FF, FR, RF, RR): (0, 5372, 2, 0)&lt;/code&gt;, which seems to contradict the expected orientation. Is there a reason that this is expected, or does it indicate a problem?&lt;/p&gt;&#xA;" OwnerUserId="2841" LastActivityDate="2021-11-23T14:47:59.860" Title="Why does BWA MEM orientation contradict my library prep method" Tags="|rna-seq|sequence-alignment|bwa|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18083" PostTypeId="1" CreationDate="2021-11-22T00:35:32.180" Score="2" ViewCount="1333" Body="&lt;p&gt;I have an MSA (protein sequence) and I have used various programmes (Clustal, Aliview, MEGA11 etc) to align. However, in all programmes I get many gaps which is not ideal as I am trying to construct a tree from this alignment.&lt;/p&gt;&#xA;&lt;p&gt;My question is are there any tools to delete gaps in alignments?&lt;/p&gt;&#xA;&lt;p&gt;Your help would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="12135" LastActivityDate="2021-11-30T15:00:01.443" Title="Remove gaps from alignment?" Tags="|sequence-alignment|multiple-sequence-alignment|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18084" PostTypeId="2" ParentId="18083" CreationDate="2021-11-22T04:05:31.787" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://academic.oup.com/mbe/article/17/4/540/1127654&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gblocks&lt;/a&gt; is a tool for removing gaps/poorly conserved positions from alignments. Available &lt;a href=&quot;http://molevol.cmima.csic.es/castresana/Gblocks.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. &lt;a href=&quot;http://molevol.cmima.csic.es/castresana/GBLOCKS_server.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is a webserver&lt;/a&gt; that runs it for you.&lt;/p&gt;&#xA;&lt;p&gt;It's a bit old, possibly there are others at this point.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2021-11-22T04:11:41.880" LastActivityDate="2021-11-22T04:11:41.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18085" PostTypeId="1" CreationDate="2021-11-22T08:31:54.980" Score="0" ViewCount="167" Body="&lt;p&gt;I have a PDB file which contain two molecules (receptor and ligand).&#xA;Each molecule will have its own header. All in &lt;strong&gt;ONE&lt;/strong&gt; PDB file.&lt;/p&gt;&#xA;&lt;p&gt;‌The header of receptor section looks like this (line 1-6 of the PDB file):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;HEADER rec.pdb&#xA;REMARK original generated coordinate pdb file&#xA;ATOM      1  N   GLY A   1     -51.221 -13.970  37.091  1.00  0.00      RA0  N&#xA;ATOM      2  H   GLY A   1     -50.383 -13.584  37.482  1.00  0.00      RA0  H&#xA;ATOM      3  CA  GLY A   1     -50.902 -15.071  36.197  1.00  0.00      RA0  C&#xA;ATOM      4  C   GLY A   1     -49.525 -15.659  36.443  1.00  0.00      RA0  C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and this ligand section (line   11435 to 11440) of the PDB file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;HEADER lig.000.00.pdb&#xA;ATOM      1  N   MET A   1      27.318 -26.957  12.663  1.00  0.00      LA0  N&#xA;ATOM      2  H   MET A   1      27.313 -27.570  11.870  1.00  0.00      LA0  H&#xA;ATOM      3  CA  MET A   1      28.374 -27.102  13.668  1.00  0.00      LA0  C&#xA;ATOM      4  CB  MET A   1      28.531 -28.564  14.090  1.00  0.00      LA0  C&#xA;ATOM      5  CG  MET A   1      27.224 -29.154  14.628  1.00  0.00      LA0  C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that the receptor and ligand section also contain the string &lt;code&gt;RA0&lt;/code&gt; and &lt;code&gt;LA0&lt;/code&gt;&#xA;at the 11th column of PDB file.&lt;/p&gt;&#xA;&lt;p&gt;What I want to do is to rename the chain of receptor as chain A and ligand as chain B.&#xA;In order to do that I intended to extract both part into two different objects first,&#xA;then rename the chain and finally put them together again.&lt;/p&gt;&#xA;&lt;p&gt;I tried this code with &lt;a href=&quot;http://thegrantlab.org/bio3d/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bio3D&lt;/a&gt;.&#xA;But doesn't work:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(bio3d)&#xA;pdb_infile &amp;lt;- &amp;quot;myfile.pdb&amp;quot;&#xA;&#xA;pdb &amp;lt;- read.pdb(pdb_infile)&#xA;receptor_segment.sele &amp;lt;- atom.select(pdb, segid = &amp;quot;RA0&amp;quot;,  verbose = TRUE)&#xA;receptor_pdb &amp;lt;- trim.pdb(pdb, receptor_segment.sele)&#xA;ligand_segment.sele &amp;lt;- atom.select(pdb, segid = &amp;quot;LA0&amp;quot;,  verbose = TRUE)&#xA;ligand_pdb &amp;lt;- trim.pdb(pdb, ligand_segment.sele) # showed no entry&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What's the way to do it?&#xA;I'm open to solution with R or Python.&lt;/p&gt;&#xA;" OwnerUserId="12433" LastActivityDate="2021-11-22T10:24:18.280" Title="How to extract part of protein complex separated with HEADER with two molecules from one PDB file" Tags="|r|python|protein-structure|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18086" PostTypeId="2" ParentId="18076" CreationDate="2021-11-22T09:33:36.010" Score="1" Body="&lt;p&gt;well wasnt able to figure out how to make openapigenerator work with rcsb pdb ModelServer &lt;a href=&quot;https://models.rcsb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://models.rcsb.org/&lt;/a&gt;, I ended up with this code&#xA;copying the webserver manual input results for ligands:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;&#xA;&#xA;def main():&#xA;   &#xA;  risposta = requests.get('https://models.rcsb.org/v1/1d17/ligand?encoding=cif&amp;amp;copy_all_categories=false&amp;amp;download=true&amp;amp;filename=out',&#xA;                          &#xA;                          allow_redirects=True)&#xA;  &#xA;  open('out.cif', 'wb').write(risposta.content)&#xA;&#xA;  print(risposta.status_code)&#xA;&#xA;if __name__ == '__main__':      &#xA;   &#xA;   main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;unfortunately the server gives you the wrong ligand (try it out)&lt;/p&gt;&#xA;&lt;p&gt;it says :' ​/v1​/{id}​/ligand&#xA;Coordinates of the first group satisfying the given criteria.'&lt;/p&gt;&#xA;&lt;p&gt;tried with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;risposta = requests.get('https://models.rcsb.org/v1/1d17/ligand?label_atom_id=NGM&amp;amp;encoding=cif&amp;amp;copy_all_categories=false&amp;amp;download=true&amp;amp;filename=out',&#xA;                       &#xA;                       allow_redirects=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but no luck again, think problem is in downloaded file :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;_model_server_params.name     atom_site &#xA;_model_server_params.value    '{&amp;quot;label_atom_id&amp;quot;:&amp;quot;NGM&amp;quot;}'  ###### here&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but not an expert of APis and openAPI at all&lt;/p&gt;&#xA;&lt;p&gt;EDITED:&lt;/p&gt;&#xA;&lt;p&gt;eventually I got it right using in the code above:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;risposta = requests.get('https://models.rcsb.org/v1/1d17/ligand?label_comp_id=NGM&amp;amp;encoding=cif&amp;amp;copy_all_categories=false&amp;amp;download=true&amp;amp;filename=out',&#xA;                      &#xA;                      allow_redirects=True) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;so I get a out.cif file containing NGM&lt;/p&gt;&#xA;&lt;p&gt;using another database and FTP :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;# -*- coding: utf-8 -*-&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;Created on Fri Nov 19 19:29:30 2021&#xA;&#xA;@author: Pietro&#xA;&#xA;&#xA;https://bioinformatics.stackexchange.com/questions/18076/help-me-in-understanding-the-pdb-file#comment25289_18076&#xA;&#xA;Ligand extract:&#xA;    &#xA;https://stackoverflow.com/questions/61390035/how-to-save-each-ligand-from-a-pdb-file-separately-with-bio-pdb&#xA;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;from ftplib import FTP&#xA;&#xA;import gzip&#xA;&#xA;import os&#xA;&#xA;from Bio.PDB import PDBParser, PDBIO, Select&#xA;&#xA;&#xA;def is_het(residue):&#xA;    res = residue.id[0]&#xA;    return res != &amp;quot; &amp;quot; and res != &amp;quot;W&amp;quot;&#xA;&#xA;&#xA;class ResidueSelect(Select):&#xA;    def __init__(self, chain, residue):&#xA;        self.chain = chain&#xA;        self.residue = residue&#xA;&#xA;    def accept_chain(self, chain):&#xA;        return chain.id == self.chain.id&#xA;&#xA;    def accept_residue(self, residue):&#xA;        &amp;quot;&amp;quot;&amp;quot; Recognition of heteroatoms - Remove water molecules &amp;quot;&amp;quot;&amp;quot;&#xA;        return residue == self.residue and is_het(residue)&#xA;&#xA;&#xA;def extract_ligands(path):&#xA;    &amp;quot;&amp;quot;&amp;quot; Extraction of the heteroatoms of .pdb files &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;    for pfb_file in os.listdir(path):&#xA;        i = 1&#xA;        if pfb_file.endswith('.pdb') and not pfb_file.startswith(&amp;quot;lig_&amp;quot;):&#xA;            pdb_code = pfb_file[:-4]&#xA;            pdb = PDBParser().get_structure(pdb_code, path + pfb_file)&#xA;            io = PDBIO()&#xA;            io.set_structure(pdb)&#xA;            for model in pdb:&#xA;                for chain in model:&#xA;                    for residue in chain:&#xA;                        if not is_het(residue):&#xA;                            continue&#xA;                        print(f&amp;quot;saving {chain} {residue}&amp;quot;)&#xA;                        io.save(f&amp;quot;lig_{pdb_code}_{i}.pdb&amp;quot;, ResidueSelect(chain, residue))&#xA;                        i += 1&#xA;&#xA;&#xA;&#xA;&#xA;def getFile(ftp, filename):&#xA;    try:&#xA;        # ftp.retrbinary(&amp;quot;RETR&amp;quot;, filename ,open(code+'.gz', 'wb').write)&#xA;        ftp.retrbinary(&amp;quot;RETR &amp;quot; + filename ,open(code+'.gz', 'wb').write)&#xA;    except:&#xA;        print(&amp;quot;Error somewhere !!!!&amp;quot;)&#xA;&#xA;&#xA;code = '1D17'&#xA;filename = 'pdb'+code.lower()+'.ent.gz'                 ## pdb1D17.ent.gz    &#xA;&#xA;print(filename)&#xA;&#xA;&#xA;&#xA;&#xA;ftp = FTP('ftp.wwpdb.org')   # connect to host, default port&#xA;&#xA;# ftp.login()               # user anonymous, passwd anonymous@&#xA;print(ftp.login())&#xA;&#xA;ftp.cwd('/pub/pdb/data/structures/all/pdb/')  &#xA;&#xA;print(ftp.pwd()  )&#xA;&#xA;getFile(ftp, filename)&#xA;&#xA;zipped = gzip.open(code+'.gz', 'rb')&#xA;&#xA;unzipped = zipped.read()&#xA;&#xA;# print(unzipped)&#xA;&#xA;&#xA;saved = open(code+'.pdb', 'wb')&#xA;&#xA;saved.write(unzipped)&#xA;saved.close()&#xA;&#xA;&#xA;path = './'&#xA;&#xA;extract_ligands(path)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;copying from : &lt;a href=&quot;https://stackoverflow.com/questions/61390035/how-to-save-each-ligand-from-a-pdb-file-separately-with-bio-pdb&quot;&gt;https://stackoverflow.com/questions/61390035/how-to-save-each-ligand-from-a-pdb-file-separately-with-bio-pdb&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and using FTP from ftplib on &lt;a href=&quot;http://ftp.wwpdb.org/pub/pdb/data/structures/all/pdb/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ftp.wwpdb.org/pub/pdb/data/structures/all/pdb/&lt;/a&gt;   : PDB - FTP Archive over HTTP&lt;/p&gt;&#xA;&lt;p&gt;NOGALAMYCIN is NGM third ligand downloaded (shuold be the only one according to &lt;a href=&quot;https://www.rcsb.org/structure/1d17&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/structure/1d17&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;In any case I am failing at ubderstandung *,-)  how &lt;a href=&quot;https://models.rcsb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://models.rcsb.org/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Parameters are related to &lt;a href=&quot;https://pdb101.rcsb.org/learn/guide-to-understanding-pdb-data/beginner%E2%80%99s-guide-to-pdb-structures-and-the-pdbx-mmcif-format&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pdb101.rcsb.org/learn/guide-to-understanding-pdb-data/beginner%E2%80%99s-guide-to-pdb-structures-and-the-pdbx-mmcif-format&lt;/a&gt; , the mmcif format&lt;/p&gt;&#xA;&lt;p&gt;but as I told before I can't read inside openapi.json OpenAPI Specification&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2021-11-22T17:34:06.490" LastActivityDate="2021-11-22T17:34:06.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18087" PostTypeId="2" ParentId="18085" CreationDate="2021-11-22T10:24:18.280" Score="2" Body="&lt;p&gt;Since this is not a correct PDB file, it can be interpreted differently by different PDB reading libraries. The most reliable way to change it would be to read it as plain text and change the chain ID to &lt;code&gt;B&lt;/code&gt; in the lines that have segment ID &lt;code&gt;LA0&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Note: Strings RA0 and LA0 start at column 73. Columns in the PDB format correspond to characters. This is not a whitespace-delimited file.&lt;/p&gt;&#xA;&lt;p&gt;Chain ID is in column 22.&lt;br /&gt;&#xA;Segment ID is in columns 73-76.&lt;/p&gt;&#xA;&lt;p&gt;As a side note, segment ID was only briefly part of the official PDB spec (around 1996), nevertheless it's supported by many programs.&lt;/p&gt;&#xA;&lt;p&gt;Of course, if you have only one file that you want to modify, the fastest way is to do it manually in a text editor.&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2021-11-22T10:24:18.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18088" PostTypeId="2" ParentId="18083" CreationDate="2021-11-22T17:18:38.273" Score="3" Body="&lt;p&gt;You may also find useful the BMGE tool (stands for Block Mapping and Gathering with Entropy). BMGE selects regions in a multiple sequence alignment that are suitable for phylogenetic inference by computing their entropy-like score. This allows to distinguish among biologically expected and unexpected variability for each aligned character. BMGE also implements different types of trimming (for instance, deletes gaps) to minimize phylogeny reconstruction artifacts.&lt;/p&gt;&#xA;&lt;p&gt;Available from &lt;a href=&quot;https://research.pasteur.fr/en/software/bmge-block-mapping-and-gathering-with-entropy/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and also installable from &lt;a href=&quot;https://anaconda.org/bioconda/bmge&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioconda&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2574" LastActivityDate="2021-11-22T17:18:38.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18089" PostTypeId="1" CreationDate="2021-11-22T18:32:44.817" Score="0" ViewCount="19" Body="&lt;p&gt;Thank you for your help.&lt;/p&gt;&#xA;&lt;p&gt;Can anybody please tell me how I can generate an intercalation site in DNA at the base pairs I am interested on. I appreciate if you can please guide me to any tutorials or videos which shows how to do that. Actually, I want to intercalate at particular site and I did Autodock to but it shows that the ligand is not at all intercalated.&lt;/p&gt;&#xA;" OwnerUserId="14439" LastActivityDate="2021-11-22T18:32:44.817" Title="Generating intercalation site in DNA for custom sequence" Tags="|phylogenetics|docking|drugs|gromacs|molecular-dynamics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18090" PostTypeId="2" ParentId="18083" CreationDate="2021-11-22T21:11:00.473" Score="2" Body="&lt;p&gt;If you want an ungapped alignment, you could &lt;em&gt;do&lt;/em&gt; an ungapped alignment (the older read aligners were ungapped, e.g. bowtie). However, I suggest that you don't want an ungapped alignment, you just want a stricter one, here's a few options...&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Remove/trim poorly aligning sequences. You could try &lt;a href=&quot;https://academic.oup.com/mbe/article/27/8/1759/988930&quot; rel=&quot;nofollow noreferrer&quot;&gt;GUIDANCE&lt;/a&gt;, which has an explicit model or GBLOCKS for a more simple filtering criteria&lt;/li&gt;&#xA;&lt;li&gt;Tune your alignment parameters for ClustalOmega/MAFFT etc. to make it stricter&lt;/li&gt;&#xA;&lt;li&gt;Finally, consider that your sequences must be homologous otherwise you'll trim away sequences until you have a random alignment that appears good&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="104" LastActivityDate="2021-11-22T21:11:00.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18091" PostTypeId="1" CreationDate="2021-11-23T14:04:57.453" Score="0" ViewCount="170" Body="&lt;p&gt;I am looking at genomic data (HG19). I have many SNPs, their chromosomes and positions.&#xA;I want to look at certain SNP (suppose its chromosome is 1 and position 77,226,919), and extract all SNPs in distance 1mb from that SNP.&lt;/p&gt;&#xA;&lt;p&gt;Are the HG19 position in terms of base-pairs? Meaning that I need to take all SNPs in position &lt;span class=&quot;math-container&quot;&gt;$77,226,919 \pm100,000$&lt;/span&gt;? Or is it something else?&lt;/p&gt;&#xA;" OwnerUserId="3039" LastActivityDate="2021-11-23T16:27:39.170" Title="HG19 Position meaning" Tags="|snp|gwas|hg19|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18092" PostTypeId="2" ParentId="18091" CreationDate="2021-11-23T14:24:56.377" Score="2" Body="&lt;p&gt;Yes, they are base pair positions&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2021-11-23T14:24:56.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18093" PostTypeId="2" ParentId="18082" CreationDate="2021-11-23T14:47:59.860" Score="2" Body="&lt;p&gt;Not familiar with the protocol, but I guess RF means different things here. If I am right, with your RF protocol, the first read comes from the reverse strand of the transcript and the second read from the forward strand, but the pair orientation is still FR:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-------------------&amp;gt;  transcript&#xA;  R2 ----&amp;gt;   &amp;lt;---- R1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The RF pair orientation should be&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;----   ----&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can confirm this by checking the mapping strand of read1.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2021-11-23T14:47:59.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18094" PostTypeId="2" ParentId="18091" CreationDate="2021-11-23T16:27:39.170" Score="1" Body="&lt;p&gt;HG19 refers to the &lt;em&gt;build&lt;/em&gt; used as the reference genome. To extend on what @Emily_Ensembl said, base pair positions may change between different builds.&lt;/p&gt;&#xA;&lt;p&gt;Please look at the following link for a more detailed explanation: &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035890951-Human-genome-reference-builds-GRCh38-or-hg38-b37-hg19&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360035890951-Human-genome-reference-builds-GRCh38-or-hg38-b37-hg19&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4001" LastActivityDate="2021-11-23T16:27:39.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18095" PostTypeId="1" AcceptedAnswerId="18097" CreationDate="2021-11-24T01:15:24.080" Score="0" ViewCount="351" Body="&lt;p&gt;I have downloaded a dataset containing RNA-Seq RPKM (reads per kilobase per million) values for 52376 genes (in humans). To find RPKM values for the gene KIR2DL2 in the dataset, I have to search for it using its Ensembl ID.&lt;/p&gt;&#xA;&lt;p&gt;After searching for this gene using the Ensembl browser, I obtain &lt;a href=&quot;https://asia.ensembl.org/Homo_sapiens/Search/Results?q=KIR2DL2;site=ensembl;facet_species=Human;page=1;perpage=10&quot; rel=&quot;nofollow noreferrer&quot;&gt;many results&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pYc28.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pYc28.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For all the results in the first page, it says &lt;em&gt;Human Alternative sequence Gene&lt;/em&gt; after the gene name. When I have searched for other genes using their Ensembl ID I get only one result:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tilON.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tilON.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What does &lt;em&gt;Human Alternative sequence Gene&lt;/em&gt; mean in Ensembl? I am not sure how to tell which is the correct Ensembl ID for the KIR2DL2 gene.&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="2531" LastEditorUserId="2531" LastEditDate="2021-11-24T01:27:21.780" LastActivityDate="2021-11-24T09:19:50.517" Title="What does 'Human Alternative sequence Gene' mean in Ensembl?" Tags="|sequence-annotation|ensembl|human-genome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18097" PostTypeId="2" ParentId="18095" CreationDate="2021-11-24T09:19:50.517" Score="4" Body="&lt;p&gt;These are genes on the patches and haplotypes. Patches are repairs in the genome, where they have discovered the sequence or assembly is incorrect, so there is an alternative sequence that is created on top of the reference assembly (the reason they don't change the reference is that then there's a new reference and this causes a massive headache for everyone). Haplotypes are alternative versions of genomic loci that exist in some individuals, such as the MHC locus, which are different and common enough to require a proper assembly change, more than just genetic variants.&lt;/p&gt;&#xA;&lt;p&gt;The genes that are found on the alt sequences are listed as &lt;em&gt;Human Alternative sequence Gene&lt;/em&gt;. They are both real genes and correct for the gene name.&lt;/p&gt;&#xA;&lt;p&gt;If the gene is on a patch (the location will include the word PATCH in its name) you should probably use the patch gene because the primary assembly gene may be dodgy.&lt;/p&gt;&#xA;&lt;p&gt;It gets more complicated if you're working with haplotypes, as all the versions are correct. Most people take the easy option of using just the primary assembly, but if you're variant calling, it may be worth trying to determine which haplotype the individual has and calling against that. This is particularly true for highly variable regions like MHC, where there are genes only found on the alternative haplotypes and not found on the primary assembly.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.ensembl.org/info/genome/genebuild/haplotypes_patches.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;There's more information about alternative sequences from Ensembl here&lt;/a&gt;. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/grc/help/faq/#difference-between-alternate-loci-and-novel-patch&quot; rel=&quot;nofollow noreferrer&quot;&gt;And here from the GRC&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1332" LastActivityDate="2021-11-24T09:19:50.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18098" PostTypeId="1" AcceptedAnswerId="18101" CreationDate="2021-11-24T15:26:31.297" Score="1" ViewCount="60" Body="&lt;p&gt;I am a software engineer and not a bioinformatician. I am looking at some code where they are looking at aligned proteins that are in the A2M alignment format (&lt;a href=&quot;https://compbio.soe.ucsc.edu/a2m-desc.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://compbio.soe.ucsc.edu/a2m-desc.html&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;According to the page, the&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Uppercase characters and &amp;quot;-&amp;quot; represent alignment columns, &#xA;and there must be exactly the same number of alignment columns in each sequence&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, the code I am looking at is doing something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sequence = sequence.replace(&amp;quot;.&amp;quot;,&amp;quot;-&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So replacing the dots with hyphen and I am not sure what the purpose of that might be.&lt;/p&gt;&#xA;&lt;p&gt;For anyone interested, the code is here (See line 182): &lt;a href=&quot;https://github.com/debbiemarkslab/DeepSequence/blob/master/DeepSequence/helper.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/debbiemarkslab/DeepSequence/blob/master/DeepSequence/helper.py&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14621" LastActivityDate="2021-11-24T18:12:36.893" Title="replacing periods with hyphen" Tags="|sequence-alignment|data-preprocessing|sequence-annotation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18100" PostTypeId="1" CreationDate="2021-11-24T17:32:46.037" Score="0" ViewCount="50" Body="&lt;p&gt;I am using Trinity from last couple-up months or so. Initially, using same species in different condition was worked fine for me. Now i am trying to do a DE analysis using multiple species. Actually, i have 3 fish performs similar behaviour  and another 3 performs similar strategy. Here i considered 3 similar species as replicate for one type (and totally i have two type) . &lt;strong&gt;Though these are 6 different species but belong to same family&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I have already made a combine assembly according to trinity protocol (--samples_file). After assembly i have checked the separate alignment rate for each species and it was above 85%. And lastly i checked heat-map and it gave me a result that somehow match with my expectation.&lt;/p&gt;&#xA;&lt;p&gt;But i am a bit confused about the possible generation of &amp;quot;assembly chimera&amp;quot;. In most of the cases i found people do DE analysis with same species. Also didn't find any reference article that did multiple species DE analysis using Trinity pipeline. So it will be a great help for me if i get some suggestion that either i should go with existing result or i need to find another way.&lt;/p&gt;&#xA;" OwnerUserId="12579" LastEditorUserId="9476" LastEditDate="2021-12-29T08:38:40.053" LastActivityDate="2021-12-29T08:38:40.053" Title="Trinity Multi-species compatibility" Tags="|trinity|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18101" PostTypeId="2" ParentId="18098" CreationDate="2021-11-24T18:12:36.893" Score="2" Body="&lt;p&gt;In A2M format, upper case letters represent matches, lower case letters represent inserts, dashes represent deletions, and dots (or spaces outside the identifier lines) represent gaps aligned to inserts.&#xA;So, both &amp;quot;-&amp;quot; and &amp;quot;.&amp;quot; are essentially gaps, but assumed to have different origin. This information is supplementary, and most MSA programs ignore dots and spaces or treat them incorrectly.&lt;/p&gt;&#xA;&lt;p&gt;The code you're looking at, in lines 187–197, removes sequences that contain characters not from &lt;code&gt;self.alphabet&lt;/code&gt; (defined in lines 74–86) by checking &lt;code&gt;if letter not in alphabet_set and letter != &amp;quot;-&amp;quot;&lt;/code&gt; (line 192). Therefore, just for the sake of parsimony, it looks natural to transform &amp;quot;.&amp;quot; into &amp;quot;-&amp;quot; beforehand.&lt;/p&gt;&#xA;" OwnerUserId="2574" LastActivityDate="2021-11-24T18:12:36.893" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18102" PostTypeId="1" AcceptedAnswerId="18103" CreationDate="2021-11-24T18:44:29.170" Score="2" ViewCount="64" Body="&lt;p&gt;I am new to genetics and I know humans share many genes with mice for instance but that there are slight differences in conserved nucleotide sequences.  Is there a community consensus around at what point a gene in one species is a different gene?  How are regulatory elements like enhancers factored into this definition?&lt;/p&gt;&#xA;" OwnerUserId="2482" LastActivityDate="2021-11-24T19:16:54.047" Title="At what point is a gene different between species that we call it a different gene?" Tags="|phylogenetics|genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18103" PostTypeId="2" ParentId="18102" CreationDate="2021-11-24T19:16:54.047" Score="3" Body="&lt;p&gt;From an evolutionary angle, you are describing an ortholog: &amp;quot;Orthologs are genes in different species that evolved from a common ancestral gene by speciation, and, in general, orthologs retain the same function during the course of evolution.&amp;quot; quoting &lt;a href=&quot;https://www.sciencedirect.com/topics/biochemistry-genetics-and-molecular-biology/orthology&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. A lot of effort has been made to construct ortholog databases (e.g. see &lt;a href=&quot;https://questfororthologs.org/orthology_databases&quot; rel=&quot;nofollow noreferrer&quot;&gt;quest for orthologs&lt;/a&gt;). They are distinguished from paralogs (approximating duplicated genes) and lateral gene transfer events, there is however, no magic cutoffs for defining orthology.&lt;/p&gt;&#xA;&lt;p&gt;Your 2nd point about regulatory elements, well they could fit in the same framework, any sequence can be clustered. What is a gene anyway? Is a noncoding RNA a gene? How do we go from a collection of transcripts to this unit we term gene? There is some ambiguity...&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2021-11-24T19:16:54.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18104" PostTypeId="2" ParentId="17931" CreationDate="2021-11-24T19:20:58.323" Score="1" Body="&lt;p&gt;There is a tool for making DNA segments of any curvature. It exports as a PDB, and the people behind the tool are quite committed and helpful.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.lifeexplorer.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.lifeexplorer.info/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is a curved DNA segment I generated using their tool; they can make much more dramatic bends.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/t3xQE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/t3xQE.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I regularly use the site you cited (scfbio...) to generate idealized straight DNA segments for illustration purposes. It is not sequence-dependent. For an actual solved structure you can extract specific chains from existing structures containing DNA (e.g. RNA polymerase II structures by the Cramer group). These structures will have their own sequence, not your specific one, and will be distorted by protein-DNA interactions. Other than that, there likely won't be a PDB structure of your specific sequence. And even if there was, the conditions likely would differ from the ones your structure was acquired at. I may be entirely off, but it sounds like you should be fitting individual bases, sugars and phosphates, not continuous poly-d-nucleotides.&lt;/p&gt;&#xA;&lt;p&gt;Shown below is the pymol visualization of a 100bp dsDNA input for B-DNA from the resource you cited&lt;a href=&quot;https://i.stack.imgur.com/NOwZN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NOwZN.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt; It does not take into account the DNA flexibility (persistence length etc)&lt;/p&gt;&#xA;" OwnerUserId="14622" LastEditorUserId="14622" LastEditDate="2021-12-04T17:11:45.433" LastActivityDate="2021-12-04T17:11:45.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18105" PostTypeId="1" CreationDate="2021-11-25T14:52:56.453" Score="-1" ViewCount="73" Body="&lt;p&gt;I have a spreadsheet of CPM values&lt;/p&gt;&#xA;&lt;p&gt;I want to visualize the top 20 genes expressed across my samples&lt;/p&gt;&#xA;&lt;p&gt;This is how my data look like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(b[1:5,1:5])&#xA;                 30566-001 - CPM 30566-002 - CPM 30566-003 - CPM 30566-004 - CPM 30566-005 - CPM&#xA;hsa-miR-548ap-3p               0               0               0               0               0&#xA;hsa-miR-548t-3p                0               0               0               0               0&#xA;hsa-miR-548e-3p                0               0               0               0               0&#xA;hsa-miR-548az-3p               0               0               0               0               0&#xA;hsa-miR-548f-3p                0               0               0               0               0&#xA;&amp;gt; dim(b)&#xA;[1] 2632   42&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How I can do that?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2021-11-26T12:06:39.157" Title="Visualizing top expressed genes" Tags="|r|differential-expression|deseq2|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18106" PostTypeId="1" CreationDate="2021-11-25T14:59:54.683" Score="1" ViewCount="42" Body="&lt;p&gt;I have a SNP file(it is 96 obs. of 1178 variables with -1, 1 and 0 and in some places NA) and I need to calculate Jaccard genetic distance from them in R. Is there any way to convert these 3 values to 0 and 1? I tried to change from -1 to 2 before for a different distance, but the Jaccard distance is binary and you cannot assign 2. Is there a way to re-code Jaccard to take into account -1 and give a good distance matrix? (Jaccard must remain binary and contain only 0 and 1). I will need it later to divide the subpopulation into 2 groups.&lt;/p&gt;&#xA;&lt;p&gt;Here is an example input data file:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://pbgworks.org/sites/pbgworks.org/files/snp.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pbgworks.org/sites/pbgworks.org/files/snp.txt&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14626" LastEditorUserId="73" LastEditDate="2021-11-26T01:15:14.213" LastActivityDate="2021-11-26T01:15:14.213" Title="How to split genotypes into two groups from a SNP txt file contains -1, 0 and 1 in R?" Tags="|r|snp|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18108" PostTypeId="2" ParentId="18100" CreationDate="2021-11-26T01:11:46.990" Score="2" Body="&lt;p&gt;Given that you're using multiple species / samples, the results from this are going to be quite weird.&lt;/p&gt;&#xA;&lt;p&gt;For doing such a comparison, I think the best approach is to do a combined assembly (as it looks like you've done) to generate a single chimeric transcriptome (which will not be a good representation of any of the component species), then do a differential expression analysis using that combined assembly as the reference.&lt;/p&gt;&#xA;&lt;p&gt;Ideally you would have multiple replicates for each different species, and then add the species and conditions into the statistical model, so that any species-specific variation can be identified and/or filtered out.&lt;/p&gt;&#xA;&lt;p&gt;Six separate assemblies that are later merged will be less useful for a differential expression, because it will bias towards differences in the assemblies. This is a similar rationale to that used for generating UMAPs from multiple single-cell datasets where a combined UMAP is generated for the full dataset with identical parameters, so that sample-specific differences in the UMAP parameters are less of an issue.&lt;/p&gt;&#xA;&lt;p&gt;I think it will be okay to go ahead with the combined assembly, bearing in mind that the lack of replicates for the differential expression study is going to make it more difficult to identify biological significance. Using split assemblies will not make that better; if anything, it'll make it worse.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-11-28T22:50:36.687" LastActivityDate="2021-11-28T22:50:36.687" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18109" PostTypeId="2" ParentId="18105" CreationDate="2021-11-26T12:06:39.157" Score="2" Body="&lt;p&gt;Gene selection is entirely on you and on the question you ask. Most expressed genes would be selected by &lt;code&gt;rowMeans&lt;/code&gt; or &lt;code&gt;rowMedians&lt;/code&gt; or even &lt;code&gt;rowSums&lt;/code&gt; but this is not very informative as digital expression levels (=counts) is influenced by many technical factors and there is (to my knowledge) no direct link to biological relevance.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, you can select by genes that vary a lot between experimental groups. Naively that could be the rowwise variance (&lt;code&gt;rowVars&lt;/code&gt;) or standard deviation of the logCPMs (use logscale, otherwise variance is a function of the expression level) or the output of a differential analysis. Ideally plug the raw counts into DESeq2 or similar frameworks. If you only have CPMs then you probably want to feed the logCPMs into limma-trend pipeline, see for a discussion on ways to get going with normalized counts for differential expression in this &lt;a href=&quot;https://support.bioconductor.org/p/125144/#125156&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioconductor thread&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Visualization is then again on you. Either make a heatmap (see for example ComplexHeatmap package) or aggregate the genes into a PCA (see for example PCAtools package at Bioconductor), all mentioned packages contain extensive example code and background information which is fully sufficient to run such an analysis.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-11-26T12:06:39.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18110" PostTypeId="2" ParentId="17817" CreationDate="2021-11-26T15:29:31.317" Score="1" Body="&lt;p&gt;you can do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data&lt;span class=&quot;math-container&quot;&gt;$condition_new &amp;lt;- as.factor(paste0(data$&lt;/span&gt;condition, &amp;quot; - &amp;quot;, data$time))&#xA;dds &amp;lt;- DESeqDataSetFromMatrix(countData=count, &#xA;                             colData=design, &#xA;                             design=~condition_new&#xA;                             , tidy = FALSE)&#xA;enter code here&#xA;results(dds, contrast=c(&amp;quot;condition_new&amp;quot;, &amp;quot;control - 24&amp;quot;, &amp;quot;condition - 4&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I hope it helps.&lt;/p&gt;&#xA;" OwnerUserId="11884" LastActivityDate="2021-11-26T15:29:31.317" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18111" PostTypeId="1" CreationDate="2021-11-26T18:09:23.950" Score="0" ViewCount="35" Body="&lt;p&gt;I have a list of &lt;em&gt;A. thaliana&lt;/em&gt; proteins and I'd like to know their location in the cell. I've found &lt;a href=&quot;https://suba.plantenergy.uwa.edu.au/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SUBA&lt;/a&gt; but they seem not to share their dataset and only allow queries through their website. It's inconvenient for me, because I need to locate 1.6k of proteins and process the results further. Is there a dataset or an API I could use instead?&lt;/p&gt;&#xA;" OwnerUserId="14629" LastActivityDate="2021-11-27T12:14:21.117" Title="Where can I get info about A. thaliana proteins' locations?" Tags="|proteins|database|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18112" PostTypeId="2" ParentId="18111" CreationDate="2021-11-27T12:14:21.117" Score="1" Body="&lt;p&gt;Given the fact that it is a simple query, in my opinion, the best approach would be an advanced search on &lt;a href=&quot;http://amigo.geneontology.org/amigo/search/annotation&quot; rel=&quot;nofollow noreferrer&quot;&gt;AmigoGO&lt;/a&gt;. There is no need to bother with complex methods like programmatic access APIs, as you basically have only two criteria to meet: organism and cellular localization.&#xA;After you specify those, you can download the file using the appropriate button, or mess a bit with the columns that you are interested in.&#xA;Once you have the file, you can then manipulate it further to your specific need with any tool of your preference, for example, to retain only proteins of your interest.&#xA;Hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="9572" LastActivityDate="2021-11-27T12:14:21.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18115" PostTypeId="1" CreationDate="2021-11-29T11:28:13.577" Score="0" ViewCount="54" Body="&lt;p&gt;I'm looking for publicly available genotype data (eg. 1000 Genomes or HapMap) that also has associated phenotypes (any traits).  I'm wanting to use it as a test run for a GWAS pipeline. Does anyone know if this exists anywhere?&lt;/p&gt;&#xA;" OwnerUserId="14638" LastActivityDate="2021-11-29T22:45:44.770" Title="Publicly available genotype / phenotype dataset?" Tags="|gwas|plink|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18116" PostTypeId="1" AcceptedAnswerId="18118" CreationDate="2021-11-29T12:15:54.673" Score="2" ViewCount="8218" Body="&lt;p&gt;I plotted a heatmap&lt;/p&gt;&#xA;&lt;p&gt;I want cancer to be red and healthy to be blue&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   pheatmap(heat1, annotation_col=df,color=colorRampPalette(c(&amp;quot;navy&amp;quot;, &amp;quot;white&amp;quot;, &amp;quot;red&amp;quot;))(50))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You are seeing cancer is in blue in annotation, how I can change that&lt;a href=&quot;https://i.stack.imgur.com/iHsFn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iHsFn.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2021-11-29T14:16:43.093" LastActivityDate="2022-12-19T09:31:37.643" Title="Changing the order of colors in pheatmap" Tags="|r|heatmap|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18117" PostTypeId="1" CreationDate="2021-11-29T13:33:02.767" Score="3" ViewCount="33" Body="&lt;p&gt;I am using RNA seq data and have been using QuSage in R in hopes to try to identify different samples and figure out which cell it belongs to.&lt;/p&gt;&#xA;&lt;p&gt;I am trying to visualize and utilize the results but I am having trouble doing that. Does anyone have an idea?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/23921631/&quot; rel=&quot;nofollow noreferrer&quot;&gt;QuSAGE&lt;/a&gt;&#xA;Quantitative Set Analysis for Gene Expression method&#xA;&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/qusage.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/html/qusage.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14640" LastEditorUserId="3848" LastEditDate="2022-04-12T15:20:36.667" LastActivityDate="2022-04-12T15:20:36.667" Title="How to identify in a RNA seq data in which sample is in which cell using R using qusage" Tags="|r|scrnaseq|bioconductor|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18118" PostTypeId="2" ParentId="18116" CreationDate="2021-11-29T13:35:22.607" Score="5" Body="&lt;p&gt;It is possible with &lt;code&gt;annotation_colors&lt;/code&gt; argument :&lt;/p&gt;&#xA;&lt;p&gt;First create a list with your conditions and then add the argument :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;annot_colors=list(HealthStatus=c(Cancer=&amp;quot;#F0978D&amp;quot;,Healthy=&amp;quot;#63D7DE&amp;quot;))&#xA;    &#xA;pheatmap(heat1,&#xA;         annotation_col=df,&#xA;         color=colorRampPalette(c(&amp;quot;navy&amp;quot;, &amp;quot;white&amp;quot;, &amp;quot;red&amp;quot;))(50),&#xA;         annotation_colors=annot_colors)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14639" LastActivityDate="2021-11-29T13:35:22.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18119" PostTypeId="1" AcceptedAnswerId="18120" CreationDate="2021-11-29T15:25:56.760" Score="4" ViewCount="625" Body="&lt;p&gt;I'm trying to convert gff3 and fasta into a gbk file for usage in Mauve. I've found a &lt;a href=&quot;https://www.biostars.org/p/2492/&quot; rel=&quot;nofollow noreferrer&quot;&gt;solution&lt;/a&gt; but the code is outdated:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;&amp;quot;&amp;quot;Convert a GFF and associated FASTA file into GenBank format.&#xA;&#xA;Usage:&#xA;gff_to_genbank.py &amp;lt;GFF annotation file&amp;gt; &amp;lt;FASTA sequence file&amp;gt;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;import sys&#xA;import os&#xA;&#xA;from Bio import SeqIO&#xA;from Bio.Alphabet import generic_dna&#xA;from BCBio import GFF&#xA;&#xA;def main(gff_file, fasta_file):&#xA;    out_file = &amp;quot;%s.gb&amp;quot; % os.path.splitext(gff_file)[0]&#xA;    fasta_input = SeqIO.to_dict(SeqIO.parse(fasta_file, &amp;quot;fasta&amp;quot;, generic_dna))&#xA;    gff_iter = GFF.parse(gff_file, fasta_input)&#xA;    SeqIO.write(gff_iter, out_file, &amp;quot;genbank&amp;quot;)&#xA;&#xA;if __name__ == &amp;quot;__main__&amp;quot;:&#xA;    main(*sys.argv[1:])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But the Bio.Alphabet module is removed now. I tried to read the documentation on these changes and how to update the code (&lt;a href=&quot;https://biopython.org/wiki/Alphabet&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biopython.org/wiki/Alphabet&lt;/a&gt;) but I'm still confused about how to update the code. If I remove the Bio.Alphabet and the generic_dna, the code will return ValueError&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ValueError: missing molecule_type in annotations&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I know that I have to type the molecule_type = &amp;quot;DNA&amp;quot; but I'm confused as to where should I put it because the code directly inputs the fasta_file into the SeqIO.parse() function.&lt;/p&gt;&#xA;&lt;p&gt;Anyone knows how to update code related to SeqIO.parse() with the removal of Bio.Alphabet? Or anyone knows how to convert GFF3 + Fasta into gbk files?&lt;/p&gt;&#xA;&lt;p&gt;Note: I can't use the seqret because my files is just too big for it, same problem as &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/11115/existing-tool-for-converting-gff3-to-genbank-gbk&quot;&gt;this&lt;/a&gt; question.&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="14560" LastActivityDate="2021-11-29T21:01:55.253" Title="Updating the GFF3 + Fasta to GeneBank code" Tags="|python|parsing|seqio|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18120" PostTypeId="2" ParentId="18119" CreationDate="2021-11-29T21:01:55.253" Score="4" Body="&lt;p&gt;You can set the molecule type on the existing annotations attribute of the record objects just before they get to &lt;code&gt;SeqIO.write&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There might be an even leaner way to do it, but if open the output file (so that you can do multiple writes to one file handle) and then iterate over each record from the GFF iterator, you can do it like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with open(out_file, &amp;quot;wt&amp;quot;) as f_out:&#xA;    for record in GFF.parse(gff_file, fasta_input):&#xA;        record.annotations[&amp;quot;molecule_type&amp;quot;] = &amp;quot;DNA&amp;quot;&#xA;        SeqIO.write(record, f_out, &amp;quot;genbank&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(The documentation is vague on that point but if you check you'll see that &lt;code&gt;record.annotations&lt;/code&gt; is a dictionary that you can modify to set the molecule type.)&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2021-11-29T21:01:55.253" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18121" PostTypeId="2" ParentId="18115" CreationDate="2021-11-29T22:40:14.653" Score="1" Body="&lt;p&gt;See openSNP here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://opensnp.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://opensnp.org/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is a web site for people to upload their own genetic datasets, providing freely-accessible public data with annotated phenotypes.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;openSNP allows customers of direct-to-customer genetic tests to publish their test results, find others with similar genetic variations, learn more about their results by getting the latest primary literature on their variations, and help scientists find new associations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The large collaborative research genetic data projects typically have very strict access constraints due to the realisation that genetic data is personally-identifiable data, and the expectation of anonymity. That means there's a low likelihood that a large research project will make their phenotype + genotype data publicly available.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-11-29T22:45:44.770" LastActivityDate="2021-11-29T22:45:44.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18122" PostTypeId="1" CreationDate="2021-11-30T02:59:12.767" Score="3" ViewCount="499" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9499558/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am working on miRNA alignment using bowtie. But it seems to fail to map. Here are the commands:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bowtie-build -f miRNA.fa miRNA&#xA;bowtie --norc -k 1 -v 0 /public/home/wschen_gibh/cpa-seq/ref/miRNA/miRNA -1 SRR16643593_1.NEW.trim.fq -2 SRR16643593_2.NEW.trim.fq -S test.sam &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and here is the result:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/FpPCI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FpPCI.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and the output sam file:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bFd3I.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bFd3I.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14097" LastEditorUserId="73" LastEditDate="2022-07-12T01:22:26.513" LastActivityDate="2022-07-12T01:22:26.513" Title="bowtie No alignments" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18123" PostTypeId="1" CreationDate="2021-11-30T08:50:12.443" Score="2" ViewCount="119" Body="&lt;p&gt;I am to phase my genotyped data with Shapeit (from ped or bed files) and I keep on getting this error message:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;ERROR: Wrong ordering in physical positions curr_pos=13915&#xA;prev=157299737&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Any idea on how to fix this?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;From the website:&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;SHAPEIT is a fast and accurate method for estimation of haplotypes (aka phasing) from genotype or sequencing data. &amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://mathgen.stats.ox.ac.uk/genetics_software/shapeit/shapeit.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://mathgen.stats.ox.ac.uk/genetics_software/shapeit/shapeit.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14645" LastEditorUserId="3848" LastEditDate="2022-04-13T23:15:06.263" LastActivityDate="2022-04-13T23:15:06.263" Title="Phasing genotyped data using shapeit: ERROR: Wrong ordering in physical positions" Tags="|phasing|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18124" PostTypeId="2" ParentId="18083" CreationDate="2021-11-30T15:00:01.443" Score="1" Body="&lt;p&gt;This new tool, CIAlign, talks about some automated removal of gaps in multiple sequence alignments&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://twitter.com/thekatybrown1/status/1306215132670889984&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://twitter.com/thekatybrown1/status/1306215132670889984&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/KatyBrown/CIAlign&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/KatyBrown/CIAlign&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="60" LastActivityDate="2021-11-30T15:00:01.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18125" PostTypeId="2" ParentId="18122" CreationDate="2021-11-30T21:37:08.137" Score="2" Body="&lt;p&gt;You should be using Bowtie 2 instead of bowtie:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Bowtie 1 was released in 2009 and was geared toward aligning the relatively short sequencing reads (up to 25-50 nucleotides) prevalent at the time. Since then, technology has improved both sequencing throughput (more nucleotides produced per sequencer per day) and read length (more nucleotides per read).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml#how-is-bowtie-2-different-from-bowtie-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://bowtie-bio.sourceforge.net/bowtie2/manual.shtml#how-is-bowtie-2-different-from-bowtie-1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In particular, Bowtie 2 supports local alignment, which is &lt;em&gt;essential&lt;/em&gt; when mapping microRNA. All microRNA sequences are short, which means that there will be adapter read-through in the sequenced reads. This adapter sequence will not map to the genome, and that will cause global aligners (like Bowtie) to fail.&lt;/p&gt;&#xA;&lt;p&gt;An alternative approach would be to trim off &lt;a href=&quot;http://www.usadellab.org/cms/?page=trimmomatic&quot; rel=&quot;nofollow noreferrer&quot;&gt;palindromic adapter sequences&lt;/a&gt; prior to aligning (which it is possible you've tried to do given the file name). Because of adapter read through, standard adapter trimming will not be sufficient. What needs to be trimmed off is the &lt;em&gt;reverse-complement&lt;/em&gt; of the adapter sequences. I have no experience using &lt;em&gt;fastp&lt;/em&gt;, so don't know if it is capable of doing that.&lt;/p&gt;&#xA;&lt;p&gt;However, even after trimming there will still be some additional short sequences left on some reads, so a local alignment would still be a good idea.&lt;/p&gt;&#xA;&lt;p&gt;Doing a local alignment with Bowtie 2 is much easier and faster than trying to troubleshoot adapter read-through issues. The command line you would run is similar, but not identical:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bowtie2-build -f miRNA.fa miRNA&#xA;bowtie2 --local -x miRNA -1 SRR16643593_1.NEW.trim.fq -2 SRR16643593_2.NEW.trim.fq | samtools sort &amp;gt; test.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-11-30T21:43:18.167" LastActivityDate="2021-11-30T21:43:18.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18126" PostTypeId="1" AcceptedAnswerId="18648" CreationDate="2021-12-01T00:28:49.763" Score="1" ViewCount="56" Body="&lt;p&gt;Reading from &lt;a href=&quot;https://www.nejm.org/doi/full/10.1056/NEJMoa2024534&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The remaining 9 new copy-number alterations showed altered coverage patterns on whole-genome sequencing but could not be confirmed by orthogonal methods because of their small size, &lt;strong&gt;low abundance&lt;/strong&gt;, or both (Fig. 2C, Fig. S2C, and Table S5)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the supplementary file, CNA abundance as percentage is defined as:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$Abundance = \frac{2^{L2R} - 1}{\frac{CN}{2} - 1}*100$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where L2R is the log&lt;span class=&quot;math-container&quot;&gt;$_{2}$&lt;/span&gt; normalized coverage ratio vs. a panel of normals and CN is the estimated copy number for the event.&lt;/p&gt;&#xA;&lt;p&gt;I haven't found a better definition/explanation behind &amp;quot;CNA abundance&amp;quot;, is it defined elsewhere, or is it a common term in CNA analysis?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2022-11-19T21:02:10.287" Title="Definition of Copy Number Abundance" Tags="|copy-number|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18127" PostTypeId="1" CreationDate="2021-12-01T01:47:50.017" Score="1" ViewCount="489" Body="&lt;p&gt;Which tool would be good to polish PacBio or ONT with Illumina?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance&lt;/p&gt;&#xA;" OwnerUserId="2477" LastActivityDate="2023-12-30T05:33:31.300" Title="Polishing PacBio or ONT with Illumina" Tags="|nanopore|illumina|pacbio|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18128" PostTypeId="1" CreationDate="2021-12-01T02:11:03.387" Score="0" ViewCount="140" Body="&lt;p&gt;I am doing some RNA analysis and am having issues trying to generate count data. I mapped my reads to a reference genome fasta file (genbank fasta file from ncbi) using bbmap and .sam files as the output.&lt;/p&gt;&#xA;&lt;p&gt;I am now trying to use featurecounts to generate count data but am running into issues with my gtf file. The gtf file is also from ncbi and corresponds to the alignment fasta file used for mapping. This is it's structure:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#gtf-version 2.2&#xA;#!genome-build ASM14559v1&#xA;#!genome-build-accession NCBI_Assembly:GCA_000145595.1&#xA;CP002120.1      Genbank gene    1       1362    .       +       .       gene_id &amp;quot;SAA6008_00001&amp;quot;; transcript_id &amp;quot;&amp;quot;; gbkey &amp;quot;Gene&amp;quot;; gene &amp;quot;dnaA&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; locus_tag &amp;quot;SAA6008_00001&amp;quot;;&#xA;CP002120.1      Genbank CDS     1       1359    .       +       0       gene_id &amp;quot;SAA6008_00001&amp;quot;; transcript_id &amp;quot;unassigned_transcript_1&amp;quot;; gbkey &amp;quot;CDS&amp;quot;; gene &amp;quot;dnaA&amp;quot;; locus_tag &amp;quot;SAA6008_00001&amp;quot;; product &amp;quot;chromosomal replication initiator protein DnaA&amp;quot;; protein_id &amp;quot;ADL64089.1&amp;quot;; transl_table &amp;quot;11&amp;quot;; exon_number &amp;quot;1&amp;quot;;&#xA;CP002120.1      Genbank start_codon     1       3       .       +       0       gene_id &amp;quot;SAA6008_00001&amp;quot;; transcript_id &amp;quot;unassigned_transcript_1&amp;quot;; gbkey &amp;quot;CDS&amp;quot;; gene &amp;quot;dnaA&amp;quot;; locus_tag &amp;quot;SAA6008_00001&amp;quot;; product &amp;quot;chromosomal replication initiator protein DnaA&amp;quot;; protein_id &amp;quot;ADL64089.1&amp;quot;; transl_table &amp;quot;11&amp;quot;; exon_number &amp;quot;1&amp;quot;;&#xA;CP002120.1      Genbank stop_codon      1360    1362    .       +       0       gene_id &amp;quot;SAA6008_00001&amp;quot;; transcript_id &amp;quot;unassigned_transcript_1&amp;quot;; gbkey &amp;quot;CDS&amp;quot;; gene &amp;quot;dnaA&amp;quot;; locus_tag &amp;quot;SAA6008_00001&amp;quot;; product &amp;quot;chromosomal replication initiator protein DnaA&amp;quot;; protein_id &amp;quot;ADL64089.1&amp;quot;; transl_table &amp;quot;11&amp;quot;; exon_number &amp;quot;1&amp;quot;;&#xA;CP002120.1      Genbank gene    1640    2773    .       +       .       gene_id &amp;quot;SAA6008_00002&amp;quot;; transcript_id &amp;quot;&amp;quot;; gbkey &amp;quot;Gene&amp;quot;; gene &amp;quot;dnaN&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; locus_tag &amp;quot;SAA6008_00002&amp;quot;;&#xA;CP002120.1      Genbank CDS     1640    2770    .       +       0       gene_id &amp;quot;SAA6008_00002&amp;quot;; transcript_id &amp;quot;unassigned_transcript_2&amp;quot;; gbkey &amp;quot;CDS&amp;quot;; gene &amp;quot;dnaN&amp;quot;; locus_tag &amp;quot;SAA6008_00002&amp;quot;; product &amp;quot;DNA-directed DNA polymerase beta subunit&amp;quot;; protein_id &amp;quot;ADL64090.1&amp;quot;; transl_table &amp;quot;11&amp;quot;; exon_number &amp;quot;1&amp;quot;;&#xA;CP002120.1      Genbank start_codon     1640    1642    .       +       0       gene_id &amp;quot;SAA6008_00002&amp;quot;; transcript_id &amp;quot;unassigned_transcript_2&amp;quot;; gbkey &amp;quot;CDS&amp;quot;; gene &amp;quot;dnaN&amp;quot;; locus_tag &amp;quot;SAA6008_00002&amp;quot;; product &amp;quot;DNA-directed DNA polymerase beta subunit&amp;quot;; protein_id &amp;quot;ADL64090.1&amp;quot;; transl_table &amp;quot;11&amp;quot;; exon_number &amp;quot;1&amp;quot;;&#xA;CP002120.1      Genbank stop_codon      2771    2773    .       +       0       gene_id &amp;quot;SAA6008_00002&amp;quot;; transcript_id &amp;quot;unassigned_transcript_2&amp;quot;; gbkey &amp;quot;CDS&amp;quot;; gene &amp;quot;dnaN&amp;quot;; locus_tag &amp;quot;SAA6008_00002&amp;quot;; product &amp;quot;DNA-directed DNA polymerase beta subunit&amp;quot;; protein_id &amp;quot;ADL64090.1&amp;quot;; transl_table &amp;quot;11&amp;quot;; exon_number &amp;quot;1&amp;quot;;&#xA;CP002120.1      Genbank gene    3163    3399    .       +       .       gene_id &amp;quot;SAA6008_00003&amp;quot;; transcript_id &amp;quot;&amp;quot;; gbkey &amp;quot;Gene&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; locus_tag &amp;quot;SAA6008_00003&amp;quot;;&#xA;CP002120.1      Genbank CDS     3163    3396    .       +       0       gene_id &amp;quot;SAA6008_00003&amp;quot;; transcript_id &amp;quot;unassigned_transcript_3&amp;quot;; gbkey &amp;quot;CDS&amp;quot;; locus_tag &amp;quot;SAA6008_00003&amp;quot;; product &amp;quot;conserved hypothetical protein&amp;quot;; protein_id &amp;quot;ADL64091.1&amp;quot;; transl_table &amp;quot;11&amp;quot;; exon_number &amp;quot;1&amp;quot;;&#xA;CP002120.1      Genbank start_codon     3163    3165    .       +       0       gene_id &amp;quot;SAA6008_00003&amp;quot;; transcript_id &amp;quot;unassigned_transcript_3&amp;quot;; gbkey &amp;quot;CDS&amp;quot;; locus_tag &amp;quot;SAA6008_00003&amp;quot;; product &amp;quot;conserved hypothetical protein&amp;quot;; protein_id &amp;quot;ADL64091.1&amp;quot;; transl_table &amp;quot;11&amp;quot;; exon_number &amp;quot;1&amp;quot;;&#xA;CP002120.1      Genbank stop_codon      3397    3399    .       +       0       gene_id &amp;quot;SAA6008_00003&amp;quot;; transcript_id &amp;quot;unassigned_transcript_3&amp;quot;; gbkey &amp;quot;CDS&amp;quot;; locus_tag &amp;quot;SAA6008_00003&amp;quot;; product &amp;quot;conserved hypothetical protein&amp;quot;; protein_id &amp;quot;ADL64091.1&amp;quot;; transl_table &amp;quot;11&amp;quot;; exon_number &amp;quot;1&amp;quot;;&#xA;CP002120.1      Genbank gene    3396    4508    .       +       .       gene_id &amp;quot;SAA6008_00004&amp;quot;; transcript_id &amp;quot;&amp;quot;; gbkey &amp;quot;Gene&amp;quot;; gene &amp;quot;recF&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; locus_tag &amp;quot;SAA6008_00004&amp;quot;;&#xA;CP002120.1      Genbank CDS     3396    4505    .       +       0       gene_id &amp;quot;SAA6008_00004&amp;quot;; transcript_id &amp;quot;unassigned_transcript_4&amp;quot;; gbkey &amp;quot;CDS&amp;quot;; gene &amp;quot;recF&amp;quot;; locus_tag &amp;quot;SAA6008_00004&amp;quot;; note &amp;quot;Required for DNA replication; binds preferentially to single-stranded, linear DNA&amp;quot;; product &amp;quot;DNA repair and genetic recombination protein&amp;quot;; protein_id &amp;quot;ADL64092.1&amp;quot;; t$&#xA;CP002120.1      Genbank start_codon     3396    3398    .       +       0       gene_id &amp;quot;SAA6008_00004&amp;quot;; transcript_id &amp;quot;unassigned_transcript_4&amp;quot;; gbkey &amp;quot;CDS&amp;quot;; gene &amp;quot;recF&amp;quot;; locus_tag &amp;quot;SAA6008_00004&amp;quot;; note &amp;quot;Required for DNA replication; binds preferentially to single-stranded, linear DNA&amp;quot;; product &amp;quot;DNA repair and genetic recombination protein&amp;quot;; protein_id &amp;quot;ADL640$&#xA;CP002120.1      Genbank stop_codon      4506    4508    .       +       0       gene_id &amp;quot;SAA6008_00004&amp;quot;; transcript_id &amp;quot;unassigned_transcript_4&amp;quot;; gbkey &amp;quot;CDS&amp;quot;; gene &amp;quot;recF&amp;quot;; locus_tag &amp;quot;SAA6008_00004&amp;quot;; note &amp;quot;Required for DNA replication; binds preferentially to single-stranded, linear DNA&amp;quot;; product &amp;quot;DNA repair and genetic recombination protein&amp;quot;; protein_id &amp;quot;ADL640$&#xA;CP002120.1      Genbank gene    4518    6452    .       +       .       gene_id &amp;quot;SAA6008_00005&amp;quot;; transcript_id &amp;quot;&amp;quot;; gbkey &amp;quot;Gene&amp;quot;; gene &amp;quot;gyrB&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; locus_tag &amp;quot;SAA6008_00005&amp;quot;;&#xA;CP002120.1      Genbank CDS     4518    6449    .       +       0       gene_id &amp;quot;SAA6008_00005&amp;quot;; transcript_id &amp;quot;unassigned_transcript_5&amp;quot;; gbkey &amp;quot;CDS&amp;quot;; gene &amp;quot;gyrB&amp;quot;; locus_tag &amp;quot;SAA6008_00005&amp;quot;; note &amp;quot;TIGRFAM: DNA gyrase, B subunit PFAM: DNA gyrase, subunit B domain protein; ATP-binding region, ATPase domain protein domain protein; TOPRIM&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I try to run featurecounts in R using this code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;featureCounts(&amp;quot;s1_mapped.sam&amp;quot;, annot.ext = &amp;quot;GCA_000145595.1_ASM14559v1_genomic.gtf&amp;quot;, isGTFAnnotationFile = T)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;The chromosome name contains unexpected characters: &amp;quot;CP002120.1 Staphylococcus aureus subsp. aureus str. JKD6008, complete genome&amp;quot; (77 chars)&#xA;featureCounts has to stop running&#xA;&#xA;FATAL Error: The program has to terminate and no counting file is generated.&#xA;&#xA;Error in featureCounts(&amp;quot;s1_mapped.sam&amp;quot;, annot.ext = &amp;quot;GCA_000145595.1_ASM14559v1_genomic.gtf&amp;quot;,  : &#xA;  No counts were generated.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Do I need to format my gtf file in some way? I have also tried the code with GTF.featureType = &amp;quot;gene&amp;quot; but I get the same error.&lt;/p&gt;&#xA;" OwnerUserId="14648" LastEditorUserId="14648" LastEditDate="2021-12-01T02:23:00.943" LastActivityDate="2021-12-01T02:23:00.943" Title="Error generating count data using featurecounts in R" Tags="|r|rna|featurecounts|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18129" PostTypeId="2" ParentId="18074" CreationDate="2021-12-01T19:24:50.243" Score="0" Body="&lt;p&gt;The issue was that we downsampled with useGalaxy's SeqTK tool prior to running  UMI-tools and cutadapt (the 2nd and 3rd process, respectively, of Quantitative Sequencing Pool Analysis).&lt;/p&gt;&#xA;&lt;p&gt;I tried downsampling with the R1 data, available after cutadapt (located in the &amp;quot;trimmed&amp;quot; folder), and everything ran fine. I believe the issues were caused by the barcode sequences being removed by the SeqTK tool prior to running UMI-tools.&lt;/p&gt;&#xA;&lt;p&gt;You can (somewhat) customize the QSPA script by #'ing out parts of the script that you don't want to run. In this case, I #'d out Star Aligner, UMI-tools (deduplication), and Feature Counts so that the script would stop after cutadapt. After I downsampled with SeqTK, I ran the script using only the parts that I had previously #'d out.&lt;/p&gt;&#xA;" OwnerUserId="14593" LastActivityDate="2021-12-01T19:24:50.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18130" PostTypeId="2" ParentId="17504" CreationDate="2021-12-02T00:33:26.357" Score="1" Body="&lt;p&gt;Answer from @matteo-ferla, converted from comment:&lt;/p&gt;&#xA;&lt;p&gt;If command line interface is more suitable for you, you may consider OpenBabel, which can predict Gasteiger charges, protonate at pH 7, accept a smi file of multiple SMILES or a multi-entry sdf file and can &lt;a href=&quot;http://openbabel.org/docs/current/FileFormats/AutoDock_PDQBT_format.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;output in PDQBT format&lt;/a&gt;, but is not quite the same as the python script allows merging and various settings.&#xA;But to use the script iteratively (a horrendous thing to do as it loads up the python kernel each time) check out &lt;a href=&quot;https://www.cyberciti.biz/faq/bash-loop-over-file/&quot; rel=&quot;nofollow noreferrer&quot;&gt;how to loop files via bash&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-12-02T00:33:26.357" CommentCount="0" CommunityOwnedDate="2021-12-02T00:33:26.357" ContentLicense="CC BY-SA 4.0" />
  <row Id="18131" PostTypeId="2" ParentId="16177" CreationDate="2021-12-02T00:36:36.967" Score="1" Body="&lt;p&gt;Answer from @pippo1980, converted from comment:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://pubs.acs.org/doi/10.1021/acs.jctc.9b01208&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pubs.acs.org/doi/10.1021/acs.jctc.9b01208&lt;/a&gt; Comprehensive Evaluation of Fourteen Docking Programs on Protein–Peptide Complexes&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-12-02T00:36:36.967" CommentCount="0" CommunityOwnedDate="2021-12-02T00:36:36.967" ContentLicense="CC BY-SA 4.0" />
  <row Id="18132" PostTypeId="2" ParentId="15143" CreationDate="2021-12-02T00:39:39.217" Score="1" Body="&lt;p&gt;Answer from @devon-ryan, converted from comment:&lt;/p&gt;&#xA;&lt;p&gt;They meant the nucleotide database.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-12-02T00:39:39.217" CommentCount="0" CommunityOwnedDate="2021-12-02T00:39:39.217" ContentLicense="CC BY-SA 4.0" />
  <row Id="18133" PostTypeId="2" ParentId="18127" CreationDate="2021-12-02T02:38:41.857" Score="1" Body="&lt;p&gt;PacBio and ONT are very different technologies; it would be helpful to be more specific. Regardless, I can only give my understanding as it relates to ONT reads....&lt;/p&gt;&#xA;&lt;p&gt;TLDR: use &lt;a href=&quot;https://github.com/nanoporetech/medaka&quot; rel=&quot;nofollow noreferrer&quot;&gt;medaka&lt;/a&gt;. Don't use Illumina reads.&lt;/p&gt;&#xA;&lt;p&gt;Nanopore reads called using the most recent super-accuracy basecallers have a very high consensus accuracy (q40-q50) for SNPs and non-homopolymeric INDELs after nanopore-only correction. ONT provide their own tool, &lt;a href=&quot;https://github.com/nanoporetech/medaka&quot; rel=&quot;nofollow noreferrer&quot;&gt;medaka&lt;/a&gt;, which uses assembly-mapped nanopore reads to correct the assembly. Medaka uses the base calling model from Guppy to predict the most likely sequence, given the input reads and the assembly.&lt;/p&gt;&#xA;&lt;p&gt;If you're using Illumina reads to polish a nanopore assembly, it should &lt;em&gt;only&lt;/em&gt; be used to correct INDEL errors (and ideally, only homopolymer errors). The two most commonly-used tools that can correct nanopore reads using Illumina reads are &lt;a href=&quot;https://github.com/lbcb-sci/racon&quot; rel=&quot;nofollow noreferrer&quot;&gt;Racon&lt;/a&gt; and &lt;a href=&quot;https://github.com/broadinstitute/pilon/wiki&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pilon&lt;/a&gt;. Based on the recency of the commit histories, I'd guess that Racon would be the most likely to work well. It looks like Racon can do both assembly correction &lt;em&gt;and&lt;/em&gt; read correction.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-12-02T02:38:41.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18134" PostTypeId="1" AcceptedAnswerId="18159" CreationDate="2021-12-02T10:49:39.830" Score="0" ViewCount="150" Body="&lt;p&gt;I am trying to calculate the dn/ds ratio between the human genes and c.elegans. However, I am getting NA values in the output file&lt;/p&gt;&#xA;&lt;p&gt;I used the following script&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(biomaRt)&#xA;&#xA;## This gives the full list of all marts, including archives&#xA;archives &amp;lt;- listEnsemblArchives()&#xA;&#xA;## For the c.elegans data&#xA;ensemblcelegans=useMart(host = 'jan2020.archive.ensembl.org', biomart = &#xA;'ENSEMBL_MART_ENSEMBL', &#xA;                 dataset = &amp;quot;celegans_gene_ensembl&amp;quot;)&#xA;&#xA;filters = listFilters(ensemblcelegans)&#xA;attributes = listAttributes(ensemblcelegans)&#xA;&#xA;Celegans_GFList &amp;lt;- scan(&amp;quot;c.elegans.tsv&amp;quot;, what = character())&#xA;Celegans_GFListValues&amp;lt;-getBM(attributes = c('ensembl_gene_id', &#xA;'hsapiens_homolog_ensembl_gene',  'hsapiens_homolog_dn', &#xA;                                    'hsapiens_homolog_ds', 'hsapiens_homolog_orthology_type'), &#xA;                     filters = 'ensembl_gene_id', &#xA;                     values = Celegans_GFList, &#xA;                     mart = ensemblcelegans)&#xA;&#xA;Celegans_GFListValues1 &amp;lt;- subset(Celegans_GFListValues, &#xA;Celegans_GFListValues&lt;span class=&quot;math-container&quot;&gt;$hsapiens_homolog_orthology_type == 'ortholog_one2one')&#xA;Celegans_GFListValues1$&lt;/span&gt;dnds_ratios &amp;lt;- &#xA;Celegans_GFListValues1&lt;span class=&quot;math-container&quot;&gt;$hsapiens_homolog_dn/Celegans_GFListValues1$&lt;/span&gt;hsapiens_homolog_ds&#xA;write.csv(Celegans_GFListValues1, file = &amp;quot;Celegans_GFListValues.csv&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10684" LastActivityDate="2021-12-07T11:54:08.340" Title="Calculating dn/ds ratio between two model organisms" Tags="|r|biomart|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18135" PostTypeId="1" CreationDate="2021-12-02T11:22:00.567" Score="0" ViewCount="99" Body="&lt;p&gt;I have two genomes with high similarity. But I found out that one genome has a longer sequence of about 200 kbp. I try to align the sequence with Mauve. How can I extract this unique 200 kbp?&lt;/p&gt;&#xA;" OwnerUserId="14560" LastActivityDate="2021-12-13T06:44:49.227" Title="Extracting Sequence Unique to a Certain Genome" Tags="|sequence-alignment|genome|mauve|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18136" PostTypeId="1" CreationDate="2021-12-02T22:00:02.120" Score="1" ViewCount="578" Body="&lt;p&gt;I have just started the QC steps from the dada2 pipeline, and have failed to find a detailed explanation of what the maxEE argument entails.&lt;/p&gt;&#xA;&lt;p&gt;I have surfed many forums, as well as the details given by the creators of the package, but they are lacking in detail. All it says is that:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;After truncation, reads with higher than maxEE &amp;quot;expected errors&amp;quot; will be discarded. Expected &#xA;errors are calculated from the nominal definition of the quality score: EE = sum(10^(-Q/10))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What is Q in this instance? Is it the Quality score obtained from the &lt;code&gt;plotQualityProfile&lt;/code&gt; function?&lt;/p&gt;&#xA;&lt;p&gt;In the tutorial, it states that:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;The standard filtering parameters are starting points, not set in stone. If you want to speed up &#xA;downstream computation, consider tightening maxEE. If too few reads are passing the filter, &#xA;consider relaxing maxEE, perhaps especially on the reverse reads (eg. MaxEE = c (2,5)), and &#xA;reducing the truncLen to remove low quality tails. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What is 2, and 5 in this instance? What does an expected error of 2, or 5, actually mean?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4001" LastActivityDate="2021-12-20T15:26:34.383" Title="maxEE parameter from DADA2 package in R" Tags="|r|microbiome|16rrna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18137" PostTypeId="1" AcceptedAnswerId="18170" CreationDate="2021-12-03T09:56:02.053" Score="3" ViewCount="262" Body="&lt;p&gt;Is there any annotation file providing correspondence between &lt;a href=&quot;https://clades.nextstrain.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nextclade&lt;/a&gt; and &lt;a href=&quot;https://cov-lineages.org/resources/pangolin.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pangolin&lt;/a&gt; variant nomenclature/annotations, to annotate some SARS-CoV-2 genomes from &lt;a href=&quot;https://www.epicov.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gisaid&lt;/a&gt; with both these?&lt;/p&gt;&#xA;&lt;p&gt;For the moment I could only find:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/nextstrain/ncov/blob/master/defaults/clades.tsv&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nextclade's clade definitions&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://raw.githubusercontent.com/cov-lineages/pango-designation/master/lineages.csv&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pangolin's lineage designations&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;What I'm looking for is an annotation as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;lineage    clade&#xA;AY.43      21J &#xA;AY.4       21J&#xA;...        ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The idea is to generate a table as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;genome                                 lineage    clade&#xA;hCoV-19/Germany/BW-RKI-I-195742/2021   AY.43      21J&#xA;...                                    ...        ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The way I proceed now is the following:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Parse the Pangolin's lineage designation file to find a target genome&lt;/li&gt;&#xA;&lt;li&gt;Look for the target genome in Gisaid (for quality check)&lt;/li&gt;&#xA;&lt;li&gt;Parse the metadata associated to the selected genome to get the lineage for verification (only Pangolin lineage is reported)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;These steps allow me to build the first part of the table:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;genome                                 lineage    &#xA;hCoV-19/Germany/BW-RKI-I-195742/2021   AY.43      &#xA;...                                    ...  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Paste the genome to Nextclade, wait for the analysis to be completed&lt;/li&gt;&#xA;&lt;li&gt;Report the calculated clade&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;With this information, I can then build the table:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;genome                                 lineage    clade&#xA;hCoV-19/Germany/BW-RKI-I-195742/2021   AY.43      21J&#xA;...                                    ...        ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The idea behind this is that both lineages and clades can be used for &amp;quot;tagging&amp;quot; a genome, depending on the aim (a given clade e.g. 21J includes many lineages e.g. AY.43 and AY.4 in the example above) and that I don't enjoy manual and error-prone procedures;) Thanks for your input.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The GISAID nomenclature via Nextstrain and overall tree is presented &lt;a href=&quot;https://www.gisaid.org/phylodynamics/global/nextstrain/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;(Crossposted on &lt;a href=&quot;https://discussion.nextstrain.org/t/correspondance-of-sars-cov-2-annotations-nextclade-pangolin/883&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nextstrain's discord forum&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="294" LastEditorUserId="12272" LastEditDate="2023-02-22T19:10:42.923" LastActivityDate="2023-02-22T19:10:42.923" Title="Correspondence of SARS-CoV-2 annotations (Nextstrain clades - Pango lineages)" Tags="|phylogenetics|sequence-annotation|covid-19|variants|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18138" PostTypeId="1" CreationDate="2021-12-03T11:45:03.840" Score="1" ViewCount="55" Body="&lt;p&gt;I use the CGView Server for constructing circular genome maps. It is really a convenient tool for me to explore circular genome maps because it's interactive. I can search a certain region and it can automatically hover to that region. However, the job running on CGView Server will be deleted after 7 days. Are there any alternatives to CGView Server? Preferably something that's offline so I can save it for longer than 7 days. I've checked the website on CGView, but I think the java package for CGView is not interactive. Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="14560" LastActivityDate="2021-12-06T08:20:45.873" Title="Offline Interactive Genome Map" Tags="|genome|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18139" PostTypeId="1" AcceptedAnswerId="18145" CreationDate="2021-12-03T14:10:14.790" Score="0" ViewCount="101" Body="&lt;p&gt;I am new to metagenomic binning. I've used CheckM in order to estimate completeness and contamination values, and most of my bins of interest appear to have good values. My workflow was pretty straightforward (BBDuk for quality control &amp;gt; Megahit for assembly &amp;gt; vamb for binning). What else could I do / where do I go from here? (especially regarding quality control)&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2021-12-05T00:14:21.850" Title="What are some ways to check metagenomic bin quality?" Tags="|assembly|metagenome|quality-control|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18140" PostTypeId="1" CreationDate="2021-12-03T14:47:54.653" Score="1" ViewCount="666" Body="&lt;p&gt;I'm trying to read in an external single cell dataset from &lt;a href=&quot;https://www.nature.com/articles/s41467-020-16164-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41467-020-16164-1&lt;/a&gt;, but I am having trouble reading in the count matrix.&lt;/p&gt;&#xA;&lt;p&gt;counts found here: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE154778&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE154778&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is how I've been reading in the raw counts.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cts = read.table(&amp;quot;GSE154778_RAW.tar&amp;quot;, sep=',', header=T, stringsAsFactors=F, check.names = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output is this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(test)&#xA;                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                          GSM4679532_K16733_barcodes.tsv.gz&#xA;1                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                     xVZ\xbe\xc6pܧj\x95I\xed\030\xee\xfe\xf8ޔ\x93\xa7\xbc\xf2\023\xed{\x8e\x98)w\\\xef:ܣN\xdfrr\xc7\xf5\f\xaa\xcb\001\xfa[\002\021@|a\x91\024\xab\xc5^\017\xc0O\xbd`7\x97&amp;amp;m\xb9\x81\xe8R\x97{\x9f\023@\x9a\xf2\x81I\0368\x80N\027\xfa\xf8\x89fǾ6V\x98h\x9aF\023\x80\xef\x8f\xf2;f\xb1%@\xffif\004(4\xf4\037\020\xaf\x8d\032\037Y\021 1\xdd\xd9V\xc0\xde{\xf0U\017&amp;lt;?\xd6\xe0D\x9b\x96\x8e\xc1\xd4s\xdb\024 &#xA;2                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               *\x9b\xee5\xbb\xb3g\xab%\xa5r\xd7?\x82\x89V\xe5\177\xd5L4g\xe7?\xa2\x96\x8b\xaa\xf0\0215D\xd4\xe1\x87~D\xd9ou\xda\xe7?~\xab&#xA;3                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         e\xb1\xe9ky\xb0w0Qy\xd7\xdd\xc6uaZ\xcb8\025-N\x96\xb6Y\xac\0015h\xeb\xe5\nL\xeb\xc4\177\x9cX\xbdU\016\xde\xcc\xdeAE\xa7\\w\xfcE_\001\xbb\026\xba\xe7\xf7ܳS'\x96\u038bI\xfe\x9a\xef\xfdm\xa2\xbf_瘺a\030\037\xec\xa3\032\x97\xa2\xea\x9e\xf3\xf40\x8d\xfb#\n\xa9\xb8vE\x94\x9b:\xef\xcb\xe5\xd5\xc69n\xe1\xbd\xcab\025\xc7\xfb|\fT\xb5\xfa\xea4lK\xb2\037)\xdfL\xfc\x8f\xf5yz\xa6D}\001/\xaak?̑\xe9\003\xf1\031\x94\xed~\xfdӳ\xc7B]\xca/\\\xfb\xa3%n&#xA;4 \xef\xaa\xc4B\xe0]\xe5\xac6\x9e\xea\xbbe\xf9\x8a\xecch/nV5\xd1p_O\xb6N\034\xa9\xf3x\034\x98&amp;amp;\025\xd75\xa7b\021\xb3v\xd6P\xf7D4~u\xb3\xab(\xf9\x9d\xb29^,\xd8\xf3v`B)\xc3c|&amp;gt;'\xc36Z\016\xb4\0257x8\u07b5fb\xd1(\xb9\xbe\xe7WT.x\xaa5\xe4\xc1\xd6\xe6ny\f֟\fYST.\\\xd0\xf2h\xf6\x8d\xc5Q\x90o\xbc\xadG\004\xb1\x9el\xcb\xd8\037\xef\xd4\024Y\xf1Q.\xe3\x81FGZ\016\026t\xa4\xcaY\xf3\xb7\x95\017,\au\xe8\037\xcf\a\xb3\ab\001\x8d.\aճ\x8f\xe6\xc4h\xe5n8\xf7Sdg\xc5Z\xef\xaa\xcb!\xd6\xd0ӳ\x92\xa2\xc4Wݥ?\xcc\xc4\016\031\023\xaa\xe7\xb6-H^X\005\xb1ӡ\x9b\xe6\xfb\xf0\x9b\x9d\027q\xcbq\x9aEm%\036\xb3\xe9&lt;span class=&quot;math-container&quot;&gt;$f\xc8\xd6\005\xdb\xee\x8b\xec\xf5\xb9\xef'\xaa~\xad\xb3\xf9\x8fg\xb3\025\xe5h\xc9\xe9_\xee7v\x87\xa3\xf5\177,ǌ\x9c\016Z\f\xe6\xf353\027\0367=\xc0\002{\x89\037,\017\xd3Œ\030\034D/\xeeb\x8ct\033\xe5UD\xedϧ\xb9\xf1\xbe\xe2\034L\xa7o\xea\xff\xb9\xecŭ\x9b\036Õ\xdbM+&amp;gt;Su$\\\xe0\xda2\xdbl?\xa2\xcc\v\xf4rS\xe4\027\x8aEdR9f\xb9N\022\xb4И@,e\xfe&amp;lt;\x95gN\xb4Ć\x9b\xc7\xeb\032\xc7P~\xa2\034\xd8]\x98\032\te\xff\xd8!vr\xb0\xa6\x92K\xf0\xfd`*\xb3\xe8\aq\x90\x9e'\xe6FI\x84\xa3&amp;lt;B\xeddE\xefpg\001V[\xbc\xb7u\xb04\003\xe3K\xb6h\xc0ild`\x92\025W\xe5#J\177\xa6\x95\xabZ.\x96\xec\016w\x93d÷+]Di\xbaJ\xa2\003\xf5\021m\xa4\x80J\022\f\027QqE\021r\x94\xf0\xbd\037\xd6ye\x970Fm\xa1ˎO\035\xfe\xe5\xb8\xf2/\x89\x9bX$\xa9b\026\037B~\xe2\023k\033\xbaB\xbcɪ\xa7\xe4f\xf2\027\x8f\xa2t\xa1\xee\xbd\xec\x80\xe5;\037\xbc\xfeC\x94\xed\x91UQ\x81~\xbe^N&amp;lt;\003&amp;amp;\xceDJE\xab\031\xa7\xf9\x9a\xa2\xf2%\024}\x9e.g\x9a0J&amp;amp;lj\xd2\032\xc55~\022\xa3ޭ\xd3|\xd1,k#\017ֶ\xfbg\026CL\016\x9bx\023Y\x89\xffI\x85\x8c\x89&amp;gt;U=\024|í\xb4\xf2\006\xfa#ʎ\032v\xc6Hps\034\fI\xef`E\xb1\xec\xcb\xf3\xc6\xf2E\xe5ӹ\xee1σc\x8b\x89\xe5\xf2\xd0\xe3\xf2\xeb\xac\xe5\xfb,&amp;amp;N\002\x9b\xe14i\xc5\024y\xc8^Ď\xe73\n\006\022\x94TM\xc5p\xbf(\xb8w~ES\x8d&amp;amp;N\xafr\xefL\f\xbf\x816~$&lt;/span&gt;\x99\021A\xfe7m\tƘhX\xe0\xa2\xc2&amp;gt;&amp;amp;\xe5s\xa0\x99\t\004\x88\u07bd\xa34\xf6q\xe4\xf1ٕ\x9b\xea\003\xd7frq(~\xe7\024\035\xcf&#xA;5                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 \x91\xb7U&#xA;6                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                               \x9f-\xcf\025\xa7\xeb6&amp;lt;\x89\xf55\xc6+\xb9K\xb5\xbaa-[.\xda \xca]&amp;lt;M\xea\xbd%&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any suggestions?&lt;/p&gt;&#xA;&lt;p&gt;Update:&lt;/p&gt;&#xA;&lt;p&gt;I unzipped the file in HPC but now I'm having trouble reading in the counts as it's all the counts per sample.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Data_Processing.nb.html            GSM4679535_T3_genes.tsv.gz     GSM4679540_T9_barcodes.tsv.gz      GSM4679544_Y00014_matrix.mtx.gz&#xA;Data_Processing.Rmd                GSM4679535_T3_matrix.mtx.gz    GSM4679540_T9_genes.tsv.gz         GSM4679545_Y00016_barcodes.tsv.gz&#xA;GSE154778_dgeMtx.csv.gz            GSM4679536_T4_barcodes.tsv.gz  GSM4679540_T9_matrix.mtx.gz        GSM4679545_Y00016_genes.tsv.gz&#xA;GSE154778_RAW.tar                  GSM4679536_T4_genes.tsv.gz     GSM4679541_T10_barcodes.tsv.gz     GSM4679545_Y00016_matrix.mtx.gz&#xA;GSM4679532_K16733_barcodes.tsv.gz  GSM4679536_T4_matrix.mtx.gz    GSM4679541_T10_genes.tsv.gz        GSM4679546_Y00019_barcodes.tsv.gz&#xA;GSM4679532_K16733_features.tsv.gz  GSM4679537_T5_barcodes.tsv.gz  GSM4679541_T10_matrix.mtx.gz       GSM4679546_Y00019_genes.tsv.gz&#xA;GSM4679532_K16733_matrix.mtx.gz    GSM4679537_T5_genes.tsv.gz     GSM4679542_Y00008_barcodes.tsv.gz  GSM4679546_Y00019_matrix.mtx.gz&#xA;GSM4679533_Y00006_barcodes.tsv.gz  GSM4679537_T5_matrix.mtx.gz    GSM4679542_Y00008_genes.tsv.gz     GSM4679547_Y00027_barcodes.tsv.gz&#xA;GSM4679533_Y00006_genes.tsv.gz     GSM4679538_T6_barcodes.tsv.gz  GSM4679542_Y00008_matrix.mtx.gz    GSM4679547_Y00027_genes.tsv.gz&#xA;GSM4679533_Y00006_matrix.mtx.gz    GSM4679538_T6_genes.tsv.gz     GSM4679543_Y00013_barcodes.tsv.gz  GSM4679547_Y00027_matrix.mtx.gz&#xA;GSM4679534_T2_barcodes.tsv.gz      GSM4679538_T6_matrix.mtx.gz    GSM4679543_Y00013_genes.tsv.gz     Lin_et_al_GMedicine_metadata.csv&#xA;GSM4679534_T2_genes.tsv.gz         GSM4679539_T8_barcodes.tsv.gz  GSM4679543_Y00013_matrix.mtx.gz&#xA;GSM4679534_T2_matrix.mtx.gz        GSM4679539_T8_genes.tsv.gz     GSM4679544_Y00014_barcodes.tsv.gz&#xA;GSM4679535_T3_barcodes.tsv.gz      GSM4679539_T8_matrix.mtx.gz    GSM4679544_Y00014_genes.tsv.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've created this loop to read through all the samples.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;counts_list &amp;lt;- list()&#xA;for (sample in samples){&#xA;    print(sample)&#xA;    filename_header &amp;lt;- sample&#xA;    barcode_file &amp;lt;- file.path(wd3, paste(filename_header, &amp;quot;_barcodes.tsv&amp;quot;, sep=&amp;quot;&amp;quot;))&#xA;    gene_file &amp;lt;- file.path(wd3, paste(filename_header, &amp;quot;_genes.tsv&amp;quot;, sep=&amp;quot;&amp;quot;))&#xA;    matrix_file &amp;lt;- file.path(wd3, paste(filename_header, &amp;quot;_matrix.mtx&amp;quot;, sep=&amp;quot;&amp;quot;))&#xA;    counts &amp;lt;- Matrix::readMM(matrix_file)&#xA;    barcodes &amp;lt;- read.csv(barcode_file, sep=&amp;quot;\t&amp;quot;, header=F)&#xA;    barcodes &amp;lt;- barcodes$V1&#xA;    colnames(counts) &amp;lt;- barcodes&#xA;    genes &amp;lt;- read.csv(gene_file, sep=&amp;quot;\t&amp;quot;, header=F)&#xA;    genes &amp;lt;- genes[,-3]&#xA;    colnames(genes) &amp;lt;- c(&amp;quot;ENSG&amp;quot;, &amp;quot;Name&amp;quot;)&#xA;    rownames(counts) &amp;lt;- genes$ENSG&#xA;    genes &amp;lt;- genes[!duplicated(genes$Name),]&#xA;    counts &amp;lt;- counts[genes$ENSG,]&#xA;    rownames(counts) &amp;lt;- genes$Name&#xA;    counts_list[[sample]] &amp;lt;- counts&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Receive this error when I run:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;counts &amp;lt;- do.call(cbind, counts_list)&#xA;Error: Matrices must have same number of rows in cbind2(argl[[i]], r)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I added:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;genes &amp;lt;- genes[,-3]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to ensure that all the gene file has only 2 columns as I noticed that one sample had 3 variables but I'm still erroring.&lt;/p&gt;&#xA;" OwnerUserId="6827" LastEditorUserId="6827" LastEditDate="2021-12-04T01:31:58.623" LastActivityDate="2021-12-04T10:54:13.813" Title="Reading in external single cell data" Tags="|r|seurat|single-cell|data-preprocessing|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18142" PostTypeId="2" ParentId="18140" CreationDate="2021-12-03T16:29:47.880" Score="3" Body="&lt;p&gt;Tar is an &lt;a href=&quot;https://en.wikipedia.org/wiki/Tar_(computing)&quot; rel=&quot;nofollow noreferrer&quot;&gt;archive format&lt;/a&gt; so as you figured out by now you have to unpack it.&lt;/p&gt;&#xA;&lt;p&gt;The files you now see are as follows:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;mtx.gz&lt;/code&gt; are compressed count matrices which you load into R using the &lt;code&gt;Matrix::readMM()&lt;/code&gt; function&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;genes.tsv.gz&lt;/code&gt; are the associated rowwise data so the &amp;quot;genes&amp;quot; which you can load with &lt;code&gt;read.delim&lt;/code&gt; and then use &lt;code&gt;rownames()&lt;/code&gt; to assign them to the matrix generated by &lt;code&gt;readMM()&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;barcodes.tsv.gz&lt;/code&gt; are the associated column data so the samples or &amp;quot;cells&amp;quot;, use &lt;code&gt;colnames()&lt;/code&gt; to assign them to the matrix&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I would use &lt;code&gt;list.files&lt;/code&gt; to create a vector of files and then either a loop or &lt;code&gt;lapply&lt;/code&gt; to iterate over them for the loading. Does that make sense to you?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit after OP received this dimensions error&lt;/strong&gt;: This then means that these files come from different experiments and have different upstream processing. THere is nothing you can do about it but figuring out which files come from which toplevel experiment and then only combine those of one experiment, respectively. The procedure towards reading data is as I lined out above.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2021-12-04T10:54:13.813" LastActivityDate="2021-12-04T10:54:13.813" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18143" PostTypeId="2" ParentId="18140" CreationDate="2021-12-03T17:10:20.667" Score="1" Body="&lt;p&gt;It looks like it is 10X data and I strongly suggest a dedicated single cell analysis package such as Seurat. You will most likely end up using one of these anyway for data analysis.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://satijalab.org/seurat/reference/read10x&quot; rel=&quot;nofollow noreferrer&quot;&gt;Read10X()&lt;/a&gt; can be a good start. I don't remember whether it requires dedicated folders per sample though. Even if this is the case, you can create individual sample folders with a simple bash script, can be done within R as well.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2021-12-03T17:10:20.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18144" PostTypeId="1" AcceptedAnswerId="18391" CreationDate="2021-12-04T00:56:35.757" Score="1" ViewCount="715" Body="&lt;p&gt;I am running an analysis in &lt;a href=&quot;https://zzz.bwh.harvard.edu/plink/&quot; rel=&quot;nofollow noreferrer&quot;&gt;plink&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This is my first attempt at a QC and I keep encountering the same message when I try to go through with my sex check. I've done the SNP missingness step, it's this step that keeps stumping me. I'm assuming it has something to do with the chromosome naming but I'm a bit lost as to how I should tackle that problem. For reference, I attempted running the --check-sex function on the files (which are in bim, bed, fam) however, I'm met with this error.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;431911 variants loaded from .bim file.&#xA;13635 people (7170 males, 6465 females) loaded from .fam.&#xA;13635 phenotype values loaded from .fam.&#xA;Using 1 thread (no multithreaded calculations invoked).&#xA;Before main variant filters, 13635 founders and 0 nonfounders present.&#xA;Calculating allele frequencies... done.&#xA;Total genotyping rate is 0.997824.&#xA;431911 variants and 13635 people pass filters and QC.&#xA;Among remaining phenotypes, 4457 are cases and 9178 are controls.&#xA;**Error: --check-sex/--impute-sex requires at least one polymorphic X chromosome&#xA;locus.**&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does anybody know what I should do to fix this?&lt;/p&gt;&#xA;&lt;p&gt;This is the command I used: plink --bfile ppmissingness2 --check-sex&lt;/p&gt;&#xA;" OwnerUserId="14657" LastEditorUserId="14657" LastEditDate="2022-01-19T14:47:56.783" LastActivityDate="2022-02-22T13:07:05.687" Title="Plink error: --check-sex/--impute-sex requires at least one polymorphic X chromosome locus" Tags="|gwas|plink|bed|qc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18145" PostTypeId="2" ParentId="18139" CreationDate="2021-12-04T15:27:53.583" Score="1" Body="&lt;p&gt;Some papers check 5S, 16S, 23S and ≥18 tRNA genes for completeness. For example, you can see &lt;a href=&quot;https://www.nature.com/articles/s41467-021-22203-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2021-12-04T15:27:53.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18146" PostTypeId="2" ParentId="18139" CreationDate="2021-12-05T00:14:21.850" Score="1" Body="&lt;p&gt;There are many other tools. If you want to get nice statistics and graphs I recommend Amber, there is also Anvio for manual refinement of the bins.&lt;/p&gt;&#xA;" OwnerUserId="14212" LastActivityDate="2021-12-05T00:14:21.850" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18147" PostTypeId="2" ParentId="18127" CreationDate="2021-12-05T00:17:33.427" Score="1" Body="&lt;p&gt;Why don’t you use a hybrid assembly which pretty much does it by default? Tools like OPERA-MS assemble the short reads with MEGAHIT and then uses nanopore to build bigger contgs. Otherwise, I recommend pilon which probably the most popular tool.&lt;/p&gt;&#xA;" OwnerUserId="14212" LastActivityDate="2021-12-05T00:17:33.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18150" PostTypeId="1" CreationDate="2021-12-05T16:18:05.590" Score="1" ViewCount="28" Body="&lt;p&gt;I am running a GWAS on a large dataset. I am using the H3aBioNet QC workflow (&lt;a href=&quot;https://github.com/h3abionet/h3agwas/blob/master/qc/README.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/h3abionet/h3agwas/blob/master/qc/README.md&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;I have run the QC on initial parameters:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;MAF &amp;lt; 0.01&lt;/li&gt;&#xA;&lt;li&gt;GENO &amp;gt; 0.05&lt;/li&gt;&#xA;&lt;li&gt;HWE 0.0000001&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My question is now that I have the output, do I need to change these parameters to a better estimate?&lt;/p&gt;&#xA;&lt;p&gt;I have the following graphs:&#xA;Figure 1 SNP missingness: For each level of missingness specied on the x axis, the corresponding y-value shows the proportion of SNPs which have missingness less than this.&lt;/p&gt;&#xA;&lt;p&gt;Figure 2 Missingness per indvididual: For each level of missingness specied on the x axis, the corresponding y-value shows the proportion of individuals which have missingness less than this.&lt;/p&gt;&#xA;&lt;p&gt;Figure 3 Cumulative frequency of SNPs. For a frequency shown on the x-axis, the corresponding y-value shows the proportion of SNPs with frequency at least this frequency; that is, it shows the proportion of SNPs which will remain if the MAF lter of this x-value is chosen.&lt;/p&gt;&#xA;&lt;p&gt;Figure 4 HWE distribution. For an HWE-value shown on the x-axis, the corresponding y-value shows the proportion of SNPs with HWE p-value at least this frequency; that is, it shows the proportion of SNPs which will be removed if the HWE lter of this x-value is chosen.&lt;/p&gt;&#xA;&lt;p&gt;Figure 5 QQ plot for Hardy-Weinberg scores.&lt;/p&gt;&#xA;&lt;p&gt;Please let me know which one is the best to attach to the question.&lt;/p&gt;&#xA;" OwnerUserId="5066" LastActivityDate="2021-12-05T16:18:05.590" Title="How to chose values for QC for a GWAS?" Tags="|gwas|qc|maf|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18151" PostTypeId="2" ParentId="18138" CreationDate="2021-12-06T07:15:46.230" Score="1" Body="&lt;p&gt;SnapGene Viewer: &lt;a href=&quot;https://www.snapgene.com/features/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.snapgene.com/features/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;MacVector: &lt;a href=&quot;https://macvector.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://macvector.com/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;ApE: &lt;a href=&quot;https://jorgensen.biology.utah.edu/wayned/ape/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://jorgensen.biology.utah.edu/wayned/ape/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2021-12-06T07:15:46.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18152" PostTypeId="2" ParentId="18138" CreationDate="2021-12-06T08:20:45.873" Score="1" Body="&lt;p&gt;I've been using Unipro Ugene and Snapgene Viewer (not for circular genomes, but they both have the option).&lt;/p&gt;&#xA;&lt;p&gt;Snapgene Viewer is more user-friendly, but I think Ugene might have more tools that are free (if you want to copy-paste pieces of DNA, for instance, you can't do that in Snapgene Viewer, it would require the paid or demo version of Snapgene)&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2021-12-06T08:20:45.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18154" PostTypeId="1" AcceptedAnswerId="18156" CreationDate="2021-12-06T11:26:59.130" Score="2" ViewCount="118" Body="&lt;p&gt;So far I haven't done any variant calling as such. Nanopore I have used for 16s microbiome data.&lt;/p&gt;&#xA;&lt;p&gt;Now My question/doubt so how do I proceed for nano-pore virus sequencing data&lt;/p&gt;&#xA;&lt;p&gt;Steps:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I get fast5 files&lt;/li&gt;&#xA;&lt;li&gt;The i perform base-calling using guppy&lt;/li&gt;&#xA;&lt;li&gt;Get fastq files&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Now after doing the above is it possible to use variant calling that is used for Ilumina once I get the fastq files?&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2021-12-06T20:47:49.377" Title="Nanopore variant calling" Tags="|variant-calling|nanopore|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18155" PostTypeId="2" ParentId="18154" CreationDate="2021-12-06T19:08:38.120" Score="-1" Body="&lt;p&gt;Sure it is possible, but I guess it won't be more reliable than a method specifically designed for Nanopore. But if you use Nanopore I guess accuracy is not your main concern.&lt;/p&gt;&#xA;" OwnerUserId="5190" LastActivityDate="2021-12-06T19:08:38.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18156" PostTypeId="2" ParentId="18154" CreationDate="2021-12-06T20:47:49.377" Score="1" Body="&lt;p&gt;Tools for illumina are not really suitable for long read sequencing. For nanopore data I would suggest &lt;a href=&quot;https://github.com/pjedge/longshot&quot; rel=&quot;nofollow noreferrer&quot;&gt;LongShot&lt;/a&gt;, &lt;a href=&quot;https://github.com/HKU-BAL/Clair3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clair3&lt;/a&gt; or &lt;a href=&quot;https://github.com/kishwarshafin/pepper&quot; rel=&quot;nofollow noreferrer&quot;&gt;DeepVariant-PEPPER&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="681" LastActivityDate="2021-12-06T20:47:49.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18157" PostTypeId="2" ParentId="9111" CreationDate="2021-12-07T00:10:40.640" Score="1" Body="&lt;p&gt;I don't know about the BLAST server you're referring to unfortunately.&lt;/p&gt;&#xA;&lt;p&gt;But another BLAST server, SequenceServer looks quite nice too. And has a bunch of helpful visualisations that you can download directly in png or svg. Source code and installation instructions for this one are at &lt;a href=&quot;http://sequenceserver.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://sequenceserver.com&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;My opinion is biased because &lt;a href=&quot;https://wurmlab.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;my lab&lt;/a&gt; makes SequenceServer. But there is also social proof: dozens and dozens of community websites use it, and it get &lt;a href=&quot;https://scholar.google.com/scholar?oi=bibs&amp;amp;hl=en&amp;amp;cites=1988041843760551302,5980610465470832900,4588167011929671798,10707512076571248813&quot; rel=&quot;nofollow noreferrer&quot;&gt;cited regularly&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Apologies for spamming a bunch of screenshots. I think they're helpful to understand what SequenceServer can do &amp;amp; I'm proud of what we've built.&lt;/p&gt;&#xA;&lt;p&gt;(it's open source so you can install it on your own server, or use our &lt;a href=&quot;https://sequenceserver.com/cloud&quot; rel=&quot;nofollow noreferrer&quot;&gt;cloud BLAST hosting service&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;h2&gt;Overview of how hits align to query&lt;/h2&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://sequenceserver.com/img/visualization/BLAST-hits-align-to-different-regions-of-query-sequence.png&quot; alt=&quot;Aligning segments&quot; /&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Lengths of queries and hits&lt;/h2&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://sequenceserver.com/img/visualization/blast-hit-protein-length-distribution.png&quot; alt=&quot;Lengths of queries and hits&quot; /&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Kablammo pairwise comparison of hit-query alignment&lt;/h2&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://sequenceserver.com/img/visualization/pairwise-overview-of-HSPs-aligning-between-BLAST-query-and-hit-sequence.png&quot; alt=&quot;Kablammo pairwise comparison of hit-query alignment&quot; /&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Circos overview of BLAST results&lt;/h2&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://sequenceserver.com/img/visualization/circos-plot-of-blast-hits-sequenceserver.gif&quot; alt=&quot;Circos/circoletto overview&quot; /&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Search interface (automatically detects query type &amp;amp; chooses BLAST algorithm)&lt;/h2&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://sequenceserver.com/img/overview/search.png&quot; alt=&quot;BLAST search submission interface&quot; /&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Example BLAST result overview&lt;/h2&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://sequenceserver.com/img/blast-setup/normal-blast-result.png&quot; alt=&quot;BLAST result&quot; /&gt;&lt;/p&gt;&#xA;" OwnerUserId="2271" LastEditorUserId="2271" LastEditDate="2021-12-08T19:12:35.433" LastActivityDate="2021-12-08T19:12:35.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18158" PostTypeId="2" ParentId="2486" CreationDate="2021-12-07T11:19:46.247" Score="0" Body="&lt;p&gt;depending on your goals you could possibly use tools like &lt;a href=&quot;https://github.com/mkirsche/Jasmine&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mkirsche/Jasmine&lt;/a&gt; or simple bedtools intersect on the vcf entries could help (e.g. harmonize after the calling)&lt;/p&gt;&#xA;&lt;p&gt;or if you have a cohort maybe possibly doing joint-CNV calling or &amp;quot;genotyping CNV&amp;quot; in a cohort is useful &lt;a href=&quot;http://penncnv.openbioinformatics.org/en/latest/user-guide/joint/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://penncnv.openbioinformatics.org/en/latest/user-guide/joint/&lt;/a&gt; or &lt;a href=&quot;https://github.com/abyzovlab/CNVpytor#merging-calls-form-multiple-samples&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/abyzovlab/CNVpytor#merging-calls-form-multiple-samples&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="60" LastEditorUserId="60" LastEditDate="2021-12-07T11:29:35.060" LastActivityDate="2021-12-07T11:29:35.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18159" PostTypeId="2" ParentId="18134" CreationDate="2021-12-07T11:54:08.340" Score="0" Body="&lt;p&gt;Ensembl discontinued our calculation of dN/dS scores for orthologue pairs in release 100 (&lt;a href=&quot;https://apr2020.archive.ensembl.org/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://apr2020.archive.ensembl.org/index.html&lt;/a&gt;). However, even when we did perform these calculations in Ensembl 99 and earlier, we only did this for pairs of orthologues that diverged within the last ∼100 million years, namely between pairs of mammalian, avian or tetraodontiforme genomes. More information can be found here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://academic.oup.com/database/article/doi/10.1093/database/bav096/2630091&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://academic.oup.com/database/article/doi/10.1093/database/bav096/2630091&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6500" LastActivityDate="2021-12-07T11:54:08.340" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18161" PostTypeId="1" AcceptedAnswerId="18163" CreationDate="2021-12-07T18:01:06.843" Score="0" ViewCount="403" Body="&lt;p&gt;The attached figure is generated based on Illumina reads from multiple individuals compared to genome assembly. Looks like there are a lot of kmers are reads only (grey colored). Also, a blue peak (2x kmers), does it represents high heterozygosity? GenomeScope gives 1.5% of heterozygosity result. But it seems the blue peak normally should be near the 70x on the x-axis&lt;a href=&quot;https://i.stack.imgur.com/IpLZN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IpLZN.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="9975" LastEditorUserId="57" LastEditDate="2021-12-07T18:06:09.337" LastActivityDate="2021-12-07T18:18:47.163" Title="Understand this Kmer plot from Merqury?" Tags="|genome|assembly|illumina|k-mer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18162" PostTypeId="1" AcceptedAnswerId="18166" CreationDate="2021-12-07T18:13:41.877" Score="1" ViewCount="672" Body="&lt;p&gt;I trying to write a pythonic PyMol script that loads a PDB, cmd.iterates through the PDB and stores the residues (three letter code) into an array.  I then want to loop through this array and calculate the center of masses (COM) of each aa's sidechains (except gly) and add it into a second array.  I am able to calculate the COM of a residue as a whole, i.e. including backbone atoms.  But what I want to calculate is the COM of the sidechain.  Please point me in the right direction.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;path = 'Path to PDB dir'&#xA;for filename in glob.glob(os.path.join(path, '*.PDB')):&#xA;    cmd.load(filename)  &#xA;    stored_residues = []&#xA;    cmd.iterate('name ca', 'stored_residues.append(resn)')&#xA;&#xA;    stored_residues_COM = []&#xA;&#xA;    for i in stored_residues:&#xA;         stored_residues_sidechain_COM.append(centerOfMass('resn %s' % i))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14684" LastEditorUserId="6322" LastEditDate="2021-12-08T13:12:32.650" LastActivityDate="2022-01-07T14:00:35.077" Title="Pymol python script for selecting a residue's sidechain and calculating its center of mass" Tags="|python|pymol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18163" PostTypeId="2" ParentId="18161" CreationDate="2021-12-07T18:18:47.163" Score="1" Body="&lt;p&gt;Assuming a diploid genome, usually, we aim to build a monoploid reference, representing a single haplotype.&lt;/p&gt;&#xA;&lt;p&gt;Practically speaking, we need to include only one of the two k-mers that are representing heterozygous sites, however, we would desire all the homozygous k-mers to be included exactly once.&lt;/p&gt;&#xA;&lt;p&gt;Looking at your plot, I would guess 1n coverage is somewhere around 18x, and 2n about 36x. If that's right, your plot is suggesting that your assembly contains lots of uncollapsed haplotypes, which is quite understandable given very high heterozygosity you estimated with genomescope. This is also supported by many homozygous k-mers are there twice (the blue bump). These are probably regions in between heterozygous sites, or flanking them.&lt;/p&gt;&#xA;&lt;p&gt;Anyway, I would recommend something like &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-018-2485-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;purge haplotigs&lt;/a&gt; to collapse haplotypes into single monoploid reference and redo the plot. The blue bump should disappear and the black and red distributions should have about the same height around 18x, while nearly all the 36x k-mers should be red (once).&lt;/p&gt;&#xA;&lt;p&gt;Also, if the interpretation of the plot still would not be clear, you can check this &lt;a href=&quot;https://youtu.be/XJW-mgmkgk0?list=PL-_gg7O4EMWbFLTdqr501tjNJ1xq43BqA&amp;amp;t=764&quot; rel=&quot;nofollow noreferrer&quot;&gt;part of a lecture by Lucía Campos-Dominguez&lt;/a&gt; on the topic (it's a different software called KAT she talks about, but it is using the same principle as Mercury).&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2021-12-07T18:18:47.163" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18165" PostTypeId="2" ParentId="3931" CreationDate="2021-12-08T06:46:41.377" Score="1" Body="&lt;p&gt;Shortest way...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '!/Id2|Id3/' RS=&amp;quot;&amp;gt;&amp;quot; ORS=&amp;quot;&amp;gt;&amp;quot; fasta.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;!&lt;/code&gt;= Not match the pattern&#xA;&lt;code&gt;Note:&lt;/code&gt; Sequence doesn't need to be in a single line!&lt;/p&gt;&#xA;" OwnerUserId="4472" LastActivityDate="2021-12-08T06:46:41.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18166" PostTypeId="2" ParentId="18162" CreationDate="2021-12-08T13:26:09.923" Score="1" Body="&lt;p&gt;I am guessing you used the command &lt;code&gt;pymol.cmd.centerofmass&lt;/code&gt;? If so the selector is the ticket to doing it by selecting everything of the residue bar the backbone atoms, e.g. f'resi {r} not name N+CA+C+O+H'&lt;/p&gt;&#xA;&lt;p&gt;Parenthetically, the &lt;code&gt;cmd.iterate&lt;/code&gt; command is really ugly and counter-intuitive —it's heavily used because it is older. The command &lt;code&gt;get_model(selection)&lt;/code&gt; in pymol2 module is so much nicer.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for atom in pymol.cmd.get_model('name CA').atom:&#xA;     atom.resi #note it is type str due to insertion codes!&#xA;     atom.resn&#xA;     atom.coords # List[float]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Second parenthesis. A &lt;code&gt;and not element H&lt;/code&gt; is not needed or beneficial to &lt;code&gt;pymol.cmd.centerofmass&lt;/code&gt; as this takes account of mass and occupancy. There is no need to select only heavy atoms as a proton weighs nothing. The barycentre and centroid are the same in geometry where there is no mass to the points, but here mass is present.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2021-12-08T13:32:01.143" LastActivityDate="2021-12-08T13:32:01.143" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18168" PostTypeId="1" CreationDate="2021-12-08T14:49:36.183" Score="1" ViewCount="839" Body="&lt;p&gt;I have 10,000 controls and 3000 cases and the .bgen file has genotype of 200,000 individuals. I would like to extract only those individuals from the .bgen file that are either cases or controls using plink for downstream analyses. How can i do that?&lt;/p&gt;&#xA;" OwnerUserId="14687" LastEditorUserId="650" LastEditDate="2021-12-14T21:58:25.480" LastActivityDate="2021-12-14T21:58:25.480" Title="How to extract a subset of individuals from a .bgen file?" Tags="|file-formats|plink|filtering|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18169" PostTypeId="1" CreationDate="2021-12-08T15:06:04.977" Score="2" ViewCount="56" Body="&lt;p&gt;I am trying to understand the WDL script for the DRAGEN-GATK whole genome analysis and I am faced with this input section inside the workflow definition which I really don't get:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# WORKFLOW DEFINITION&#xA;workflow WholeGenomeGermlineSingleSample {&#xA;&#xA;  String pipeline_version = &amp;quot;3.0.0&amp;quot;&#xA;&#xA;      input {&#xA;        SampleAndUnmappedBams sample_and_unmapped_bams&#xA;        DNASeqSingleSampleReferences references&#xA;        DragmapReference? dragmap_reference&#xA;        VariantCallingScatterSettings scatter_settings&#xA;        PapiSettings papi_settings&#xA;    &#xA;        File? fingerprint_genotypes_file&#xA;        File? fingerprint_genotypes_index&#xA;    &#xA;        File wgs_coverage_interval_list&#xA;    &#xA;        Boolean provide_bam_output = false&#xA;        Boolean use_gatk3_haplotype_caller = true&#xA;    &#xA;        Boolean dragen_functional_equivalence_mode = false&#xA;        Boolean dragen_maximum_quality_mode = false&#xA;    &#xA;        Boolean run_dragen_mode_variant_calling = false&#xA;        Boolean use_spanning_event_genotyping = true&#xA;        Boolean unmap_contaminant_reads = true&#xA;        Boolean perform_bqsr = true&#xA;        Boolean use_bwa_mem = true&#xA;        Boolean allow_empty_ref_alt = false&#xA;        Boolean use_dragen_hard_filtering = false&#xA;      }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What do the first 5 lines under &lt;code&gt;input&lt;/code&gt; mean? Are they Strings? Files? Or what exactly?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance for your help&lt;/p&gt;&#xA;&lt;p&gt;Nawar&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;GATK&lt;/strong&gt; is Broad's VCF&#xA;&lt;strong&gt;DRAGEN&lt;/strong&gt; is Dynamic Read Analysis for GENomics to facilitates hardware-acceleration of analytics and processes including conversion, compression, mapping, alignment, sorting, duplicate marking, haplotype variant calling and joint genotyping.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360045944831-DRAGEN-GATK&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360045944831-DRAGEN-GATK&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2396" LastEditorUserId="3848" LastEditDate="2022-04-13T23:12:54.197" LastActivityDate="2022-04-13T23:12:54.197" Title="About workflow input section in DRAGEN-GATK v1 whole genome analysis script" Tags="|wgs|gatk|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18170" PostTypeId="2" ParentId="18137" CreationDate="2021-12-08T16:25:26.693" Score="3" Body="&lt;p&gt;A comment from a Nextclade's user satisfies my actual needs, I will post it here for future reference:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Hi - one possible source is the metadata file for UCSC’s daily-updated tree of public SARS-CoV-2 sequences:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;https://hgdownload.soe.ucsc.edu/goldenPath/wuhCor1/UShER_SARS-CoV-2/public-latest.metadata.tsv.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;It’s a big file – almost 3 million lines – because there are about 3M sequences in GenBank &amp;amp; COG-UK at this point and that number will probably grow. The last two columns of the file (10th &amp;amp; 11th) are the Nextstrain clade and Pango lineage assigned to each sequence based on its placement in the UCSC/UShER tree and the tree’s annotation of where each clade/lineage starts.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This command will make a file relating each Nextstrain clade to at least one lineage (often to a lot of lineages, especially for Delta):&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;gunzip -c public-latest.metadata.tsv.gz | cut -f 10,11 | sort -u &amp;gt; cladeToLineages.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Some Pango lineages have no public sequences (only GISAID sequences, which can’t be shared at a public URL), so those lineages will be missing from that file. Some older lineages are not monophyletic on the UCSC/UShER tree, so those lineages are not annotated and will be missing. (In total, ~200 / ~1500 are missing.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="294" LastActivityDate="2021-12-08T16:25:26.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18171" PostTypeId="2" ParentId="15481" CreationDate="2021-12-08T19:44:47.390" Score="2" Body="&lt;p&gt;It seems the sequencing is single-end, and there is only one fastq per sample. If so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;staridx=/path/to/star/genome/index&#xA;&#xA;mkdir /plates/star/&#xA;cd /plates/star/&#xA;&#xA;files=$(find /plates/plate_1/rawdata/ -name &amp;quot;*_R1.fastq&amp;quot;)&#xA;&#xA;for i in ${files}&#xA;do&#xA;&#xA;  sample=&lt;span class=&quot;math-container&quot;&gt;$(basename $&lt;/span&gt;i &amp;quot;_R1.fastq&amp;quot;)&#xA;&#xA;  STAR --genomeDir &lt;span class=&quot;math-container&quot;&gt;$staridx --runThreadN 8 --outFileNamePrefix $&lt;/span&gt;{sample}. \&#xA;    --readFilesIn $i --outSAMtype BAM&#xA;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;All fastq files will be mapped individually, and all bam files will be saved to &lt;code&gt;/plates/star/&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1642" LastActivityDate="2021-12-08T19:44:47.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18172" PostTypeId="2" ParentId="10615" CreationDate="2021-12-09T03:25:34.087" Score="3" Body="&lt;p&gt;The headers are defined by the basecaller (guppy, Albacore, bonito, etc) during basecalling. They have changed many times over the years, but generally they start with a readID of &lt;code&gt;@+UUIDv4&lt;/code&gt; followed by multiple entries separated by spaces, of form key=value&lt;/p&gt;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@945da76d-b7ef-4c65-91a1-e69c85832185 runid=2cf9e7c4dd1888d9a1090ebc394dd5b7cb2fe3f0 read=126 ch=181 start_time=2017-10-17T06:43:19Z barcode=barcode01&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is still valid, however, there is some movement to add sam tags to parse data through to sam files when aligned.&lt;/p&gt;&#xA;" OwnerUserId="14692" LastActivityDate="2021-12-09T03:25:34.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18173" PostTypeId="2" ParentId="18168" CreationDate="2021-12-09T16:04:27.803" Score="2" Body="&lt;p&gt;You can use &lt;a href=&quot;https://www.well.ox.ac.uk/%7Egav/qctool_v2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;qctool&lt;/a&gt; for this. If you create a list of IDs in &lt;code&gt;list.txt&lt;/code&gt; (one ID per line), you can filter the .bgen file(s) by running&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; qctool -g yourdata.bgen -s yourdata.sample -incl-samples list.txt -og yourdata_filtered.bgen&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12128" LastActivityDate="2021-12-09T16:04:27.803" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="18176" PostTypeId="1" AcceptedAnswerId="18181" CreationDate="2021-12-09T23:02:44.863" Score="1" ViewCount="998" Body="&lt;p&gt;I have recently received some metagenomic data from 16S rrna sequencing. The sequencing company claim to have removed primers, however not adapter sequences. Please note that the files have been demultiplexed.&lt;/p&gt;&#xA;&lt;p&gt;In the platform, we have forward and reverse strand fastq files. For each fastq file, there is information regarding the adapter used, for each strand.&lt;/p&gt;&#xA;&lt;p&gt;However, when I run the following command:&#xA;&lt;code&gt;cat myFastQ_forward.fastq | grep 'adapter_forward'&lt;/code&gt;, it returns nothing. My question is, shouldn't the adapters be identified successfully simply by looking for an exact match in the fastq files? If not, why?&lt;/p&gt;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="4001" LastEditDate="2021-12-10T14:02:39.227" LastActivityDate="2021-12-10T16:53:55.427" Title="Searching for adapter sequences in FASTQ files - metgenomics" Tags="|linux|metagenome|trimming|cutadapt|16rrna|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18177" PostTypeId="1" AcceptedAnswerId="18223" CreationDate="2021-12-09T23:16:42.553" Score="1" ViewCount="485" Body="&lt;p&gt;I have created a Blast database using a reference genome. Then, I have performed a local blast search in command line using a gene of interest. I have obtained some hits with the usual Blasting information. Now, I want to extract the exact matching sequence from the blast search with their corresponding start and stop position. I have a &lt;code&gt;.gff&lt;/code&gt; file for the genes as well for the reference genome. Can you give me any suggestions on how to do that? Thanks a lot.&lt;/p&gt;&#xA;" OwnerUserId="12238" LastActivityDate="2021-12-19T01:08:22.137" Title="How to retrieve fasta sequence after local blast?" Tags="|gene|ngs|blast|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18179" PostTypeId="1" CreationDate="2021-12-10T09:41:39.623" Score="2" ViewCount="87" Body="&lt;p&gt;According to the Chou-Fasman method, when there is an overlapping region on the amino acid sequence, we compare the scores of alpha-helix and beta-sheet and chose the highest one.&#xA;Let's say I have this amino acid sequence: &lt;strong&gt;ABC&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As a result of computations, &lt;strong&gt;AB&lt;/strong&gt; is identified as alpha-helix, and &lt;strong&gt;BC&lt;/strong&gt; is identified as beta-sheet.&#xA;This means, I have an overlapping region, which is &lt;strong&gt;B&lt;/strong&gt; in this case.&#xA;The algorithm says that we need to compare the scores of the overlapping region, and select the secondary structure with the higher score, let's say alpha-helix in this case.&lt;/p&gt;&#xA;&lt;p&gt;Would the resulting secondary structure be &lt;span class=&quot;math-container&quot;&gt;$\alpha$$\alpha$$\beta$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$\alpha$$\alpha$&lt;/span&gt;&lt;em&gt;_&lt;/em&gt;?&lt;/p&gt;&#xA;&lt;p&gt;In other words, do we cancel out the secondary structure identifications when we encounter overlapping regions?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="14697" LastActivityDate="2022-01-09T13:05:10.530" Title="Chou-Fasman Method Overlapping Regions Problem - Secondary Structure Prediction" Tags="|protein-structure|secondary-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18180" PostTypeId="2" ParentId="18179" CreationDate="2021-12-10T12:22:21.563" Score="1" Body="&lt;p&gt;Sure, an isolated residue has a ψ and φ angles, so you could technically have an isolated residue in a given SS conformation, but that does not mean much if its neighbours' backbones aren't in that same conformation. Also, sure, in Rosetta design algorithms residues are thrown into ABEGO bins based on ψ and φ angles alone and as a result one can get isolated α-helix bin residues (in what is a loop): but that's a design algorithm that will want to correct this.&lt;/p&gt;&#xA;&lt;p&gt;In your particular example, IIRC that algorithm uses a window of 4 or 6 residues for SS prediction. So it will surely be (αα)αα_.&#xA;But I am pretty certain you cannot have a ααααββββ switch without a loop or turn in nature —I might even hazard a guess that for design it is not Koga-compliant even if not specifically addressed. And with the Chou-Fasman algorithm you'd need to have the windows spanning the switch score above the threshold in the correct way. I am sure there's a combination that allows it as after all we're talking of an ancient 70s prediction method based on probabilities alone...&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2021-12-10T12:22:21.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18181" PostTypeId="2" ParentId="18176" CreationDate="2021-12-10T16:53:55.427" Score="0" Body="&lt;p&gt;It depends on the read layout they used, but typically when I hear anyone talk about removing adapters in paired-end sequencing, it's the occurrences of adapter sequences at the &lt;em&gt;ends&lt;/em&gt; of each read from the &lt;em&gt;complementary strand's&lt;/em&gt; adapter.  For example see the figures on &lt;a href=&quot;https://www.idtdna.com/pages/education/decoded/article/16s-rrna-indexed-primers-amplify-phylogenic-markers-for-microbiome-sequencing-analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;this IDT page about 16S sequencing&lt;/a&gt;.  If a given sequence is short enough/number of sequencing cycles high enough, you'll end up seeing &amp;quot;past the ends&amp;quot; on each read.&lt;/p&gt;&#xA;&lt;p&gt;For example, with a reverse read reverse-complemented and aligned with a forward read, you could maybe see something like this off the sequencer:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;                       |--&amp;gt;&#xA;Forward:               BarcodefwdAmpliconBarcoderevAdapterrev...&#xA;Reverse:  ...AdapterfwdBarcodefwdAmpliconBarcoderev&#xA;                                               &amp;lt;--|&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And then if the company already removed barcodes and primers at the start of the reads, maybe they gave you:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;                                 |--&amp;gt;&#xA;Forward:                         AmpliconBarcoderevAdapterrev...&#xA;Reverse:  ...AdapterfwdBarcodefwdAmplicon&#xA;                                     &amp;lt;--|&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So the material you'd want to trim is on the ends, and reverse-complemented.  How much you see depends on the lengths involved so maybe you'll see partial adapters or none at all, and there may be sequencing error, too.  See &lt;a href=&quot;https://cutadapt.readthedocs.io/en/stable/guide.html#regular-3-adapters&quot; rel=&quot;nofollow noreferrer&quot;&gt;cutadapt's documentation about trimming 3' adapters&lt;/a&gt; and &lt;a href=&quot;https://cutadapt.readthedocs.io/en/stable/guide.html#trimming-paired-end-reads&quot; rel=&quot;nofollow noreferrer&quot;&gt;paired-end reads&lt;/a&gt; for more on that.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2021-12-10T16:53:55.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18182" PostTypeId="1" CreationDate="2021-12-10T17:49:59.603" Score="0" ViewCount="211" Body="&lt;p&gt;I am trying to incorporate Methylation data (Illumina 450K) into my pipeline. I was provided with two versions of the same data: one (dataset &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt;) normalized with ComBat and the other (dataset &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt;) normalized using the residuals method (i.e., taking the residuals of the models fitted on the covariates). Looking at the distribution of the beta values, I noticed that:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The values for dataset &lt;span class=&quot;math-container&quot;&gt;$a$&lt;/span&gt; have a typical bimodal distribution with peaks at around &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;The values for dataset &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; are roughly normally distributed with the mean at around &lt;span class=&quot;math-container&quot;&gt;$0.5$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I am not that familiar with Methylation data, so maybe this is expected. Is it? I did not find any paper showing similar results.&lt;/p&gt;&#xA;" OwnerUserId="2862" LastActivityDate="2022-09-09T16:50:01.740" Title="Methylation data: beta values are normally distributed" Tags="|methylation|multi-omics|combat|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18183" PostTypeId="1" AcceptedAnswerId="18188" CreationDate="2021-12-11T04:48:12.303" Score="0" ViewCount="55" Body="&lt;p&gt;I have sequenced a bacterial genome and after I checked the ANI scores, the scores against similar bacterias were 79 and lower. How can I confirm if this bacteria is a novel species/strain? Is there any threshold value for ANI score? I'm pretty sure if the ANI score is above 95, it indicates the same species.&lt;/p&gt;&#xA;&lt;p&gt;Should I double-check this result by using other methods such as GGDC?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="14560" LastActivityDate="2021-12-12T22:39:55.953" Title="Identifying Novel Species with ANI Taxonomy" Tags="|genome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18185" PostTypeId="1" CreationDate="2021-12-12T08:15:04.963" Score="0" ViewCount="96" Body="&lt;p&gt;I am new to programming and I am given the task to extract structural data of soluble proteins from PDB and then shortlist the soluble enzymes from the extracted data set for further classification based shortlisting of soluble non membrane enzymes. Since I do not have much expertise in programming I am finding it difficult to figure out how to solve the problem. Any help would be really appreciated.&lt;/p&gt;&#xA;" OwnerUserId="14705" LastActivityDate="2021-12-12T08:15:04.963" Title="Extracting and analyzing structure data of multiple proteins from PDB using Python" Tags="|fasta|protein-structure|pdb|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18186" PostTypeId="1" CreationDate="2021-12-12T12:03:35.420" Score="2" ViewCount="3172" Body="&lt;p&gt;I am testing a 10x fastq dataset , but cellranger count complains &amp;quot;An extremely low rate of correct barcodes was observed for all the candidate chemistry choices for the input&amp;quot;, I have tried all the chemistry recommendation codes on the 10x genomics website, and I still get the same result, how can I solve this problem, thanks.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ra0XA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ra0XA.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/FChZu.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FChZu.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14706" LastEditorUserId="14706" LastEditDate="2021-12-13T01:26:49.590" LastActivityDate="2023-08-29T08:02:55.320" Title="10X low rate of correct barcodes was observed for the candidate chemistry choices for the input" Tags="|single-cell|fastq|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18188" PostTypeId="2" ParentId="18183" CreationDate="2021-12-12T22:39:55.953" Score="2" Body="&lt;h2&gt;Short answer:&lt;/h2&gt;&#xA;&lt;p&gt;Yes.&lt;/p&gt;&#xA;&lt;h2&gt;Long answer:&lt;/h2&gt;&#xA;&lt;p&gt;I suggest 4 methods for checking the genome.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use minimap2 or a similar aligner to perform alignments with the most similar genomes, then generate a dot plot or other visualization of the regions that align to figure out what is going on. &lt;a href=&quot;https://cube.univie.ac.at/gepard&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is one tool, I haven't used it, &lt;a href=&quot;https://peerj.com/articles/4958/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is another. I generally use &lt;a href=&quot;https://github.com/lh3/minimap2&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimap2&lt;/a&gt; to very quickly align, and then a quick plot in R using the PAF file.&lt;/li&gt;&#xA;&lt;li&gt;Use k-mer hashing methods such as &lt;a href=&quot;https://mash.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;mash&lt;/a&gt; or sourmash to quickly compare to a large database. &lt;a href=&quot;https://mash.readthedocs.io/en/latest/data.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Large precomputed databases are available for download&lt;/a&gt; on the linked mash site. You can also use it for one-by-one comparisons.&lt;/li&gt;&#xA;&lt;li&gt;Use &lt;a href=&quot;https://github.com/ablab/quast&quot; rel=&quot;nofollow noreferrer&quot;&gt;QUAST&lt;/a&gt; to estimate more informative summary statistics for comparisons to the more similar genomes. QUAST has a rich set of possible visualizations that you can use. It also tries to identify the most closely related public genome using rDNA sequences, though that has been hit or miss for me.&lt;/li&gt;&#xA;&lt;li&gt;Use a quality assessment tool such as &lt;a href=&quot;https://ecogenomics.github.io/CheckM/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CheckM&lt;/a&gt; or &lt;a href=&quot;https://busco.ezlab.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BUSCO&lt;/a&gt; to estimate the quality of your genome. THese also have utilities for taxonomic placement of your genome in case something bad happened (e.g. sample swap).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2021-12-12T22:39:55.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18189" PostTypeId="2" ParentId="18177" CreationDate="2021-12-12T23:51:20.477" Score="1" Body="&lt;p&gt;My usual approach for extracting sequence from a reference genome is to use &lt;code&gt;samtools faidx&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# create genome index; slow; only do once per reference genome&#xA;samtools faidx reference_genome.fa&#xA;# extract sequence for a specific match&#xA;samtools faidx reference_genome.fa chromosome_name1:posStart1-posEnd1 &amp;gt; match1.fa&#xA;samtools faidx reference_genome.fa chromosome_name2:posStart2-posEnd2 &amp;gt; match2.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You haven't mentioned how many BLAST results you have, or what format they're in, so I'm not sure if it's practical to do this manually, or if it needs to be automated in some way.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-12-12T23:51:20.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18190" PostTypeId="2" ParentId="18182" CreationDate="2021-12-12T23:55:56.883" Score="0" Body="&lt;p&gt;It's not clear from your question what the statistical design of the experiment is, so I can't judge whether or not a distribution around 0.5 would be expected from the residuals after fitting.&lt;/p&gt;&#xA;&lt;p&gt;Given that dataset &lt;em&gt;a&lt;/em&gt; looks like what you expect, I'd be inclined to use that (and perhaps ask the person who gave you this data why dataset &lt;em&gt;b&lt;/em&gt; looks so different, and why it ended up with a 0.5 mean).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-12-12T23:55:56.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18191" PostTypeId="2" ParentId="18186" CreationDate="2021-12-12T23:57:07.210" Score="0" Body="&lt;p&gt;Answer from @maximilian-press, converted from comments:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I can't comment on 10X data specifically, however, I suggest writing a&#xA;simple script or command that counts the frequency of each barcode,&#xA;and reports back the N (10? 20? 100?) most frequent ones, which will&#xA;allow you to investigate in more detail. This is what I have done in&#xA;the case of suspected bardode swaps or reverse complement issues.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-12-12T23:57:07.210" CommentCount="0" CommunityOwnedDate="2021-12-12T23:57:07.210" ContentLicense="CC BY-SA 4.0" />
  <row Id="18193" PostTypeId="2" ParentId="18186" CreationDate="2021-12-13T03:24:41.060" Score="2" Body="&lt;p&gt;I have solve this problem, cause the combined barcode is differernt from the common 10x barcode. when mapping the genome, need to add-- chemistry ARC-v1 when cellranger count.&lt;/p&gt;&#xA;" OwnerUserId="14706" LastActivityDate="2021-12-13T03:24:41.060" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18194" PostTypeId="2" ParentId="18135" CreationDate="2021-12-13T04:48:44.023" Score="1" Body="&lt;p&gt;I've found the answer.&lt;/p&gt;&#xA;&lt;p&gt;I've aligned these genomes (gbk files) with Mauve Contig Mover, and then I exported the Gap File into a csv file. Then I used python pandas to analyze the region that is unique to each strand from the csv file.&lt;/p&gt;&#xA;" OwnerUserId="14560" LastEditorUserId="57" LastEditDate="2021-12-13T06:44:49.227" LastActivityDate="2021-12-13T06:44:49.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18196" PostTypeId="1" CreationDate="2021-12-13T09:32:22.843" Score="1" ViewCount="40" Body="&lt;p&gt;In &lt;a href=&quot;https://github.com/biocommons/hgvs&quot; rel=&quot;nofollow noreferrer&quot;&gt;HGVS&lt;/a&gt; package, how to create more complex variants ?&lt;/p&gt;&#xA;&lt;p&gt;I just wonder how can I parse more complex insertions such as those :&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;NM_004006.2:c.940_941ins[903_940inv;851_885inv]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;NC_000006.11:g.8897754_8897755ins[(543);8897743_8897754]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;NC_000006.11:g.10791926_10791927ins[NC_000004.11:g.106370094_106370420;A[26]]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;(All taken from the &lt;a href=&quot;https://varnomen.hgvs.org/recommendations/DNA/variant/insertion/&quot; rel=&quot;nofollow noreferrer&quot;&gt;varnomen website&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;When I tried I got parsing errors (mainly because of the brackets that represent the inserted sequences)&lt;/p&gt;&#xA;&lt;p&gt;Whole traceback :&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;In [6]: var_g = hp.parse_hgvs_variant('NM_004006.2:c.940_941ins[903_940inv;851_885inv]')&#xA;---------------------------------------------------------------------------&#xA;ParseError                                Traceback (most recent call last)&#xA;~/playground/python/hgvs/venv/lib/python3.9/site-packages/hgvs/parser.py in rule_fxn(s)&#xA;    129                 try:&#xA;--&amp;gt; 130                     return self._grammar(s).__getattr__(rule_name)()&#xA;    131                 except ometa.runtime.ParseError as exc:&#xA;&#xA;~/playground/python/hgvs/venv/lib/python3.9/site-packages/parsley.py in invokeRule(*args, **kwargs)&#xA;     97                                      [[&amp;quot;message&amp;quot;, &amp;quot;expected EOF&amp;quot;]], err.trail)&#xA;---&amp;gt; 98             raise err&#xA;     99         return invokeRule&#xA;&#xA;~/playground/python/hgvs/venv/lib/python3.9/site-packages/ometa/runtime.py in _or(self, fns)&#xA;    597                 m = self.input&#xA;--&amp;gt; 598                 ret, err = f()&#xA;    599                 errors.append(err)&#xA;&#xA;/pymeta_generated_code/pymeta_grammar__Grammar.py in _G_or_212()&#xA;    803                 self._trace('| dna_in', (5502, 5510), self.input.position)&#xA;--&amp;gt; 804                 _G_apply_213, lastError = self._apply(self.rule_dna_ins, &amp;quot;dna_ins&amp;quot;, [])&#xA;    805                 self.considerError(lastError, None)&#xA;&#xA;~/playground/python/hgvs/venv/lib/python3.9/site-packages/ometa/runtime.py in _apply(self, rule, ruleName, args)&#xA;    494                 memoRec = self.input.setMemo(ruleName,&#xA;--&amp;gt; 495                                          [rule(), self.input])&#xA;    496             except ParseError as e:&#xA;&#xA;/pymeta_generated_code/pymeta_grammar__Grammar.py in rule_dna_ins(self)&#xA;    980                 return (_G_many1_275, self.currentError)&#xA;--&amp;gt; 981             _G_consumedby_276, lastError = self.consumedby(_G_consumedby_272)&#xA;    982             self.considerError(lastError, 'dna_ins')&#xA;&#xA;~/playground/python/hgvs/venv/lib/python3.9/site-packages/ometa/runtime.py in consumedby(self, expr)&#xA;    675         oldInput = self.input&#xA;--&amp;gt; 676         _, e = expr()&#xA;    677         slice = oldInput.data[oldInput.position:self.input.position]&#xA;&#xA;/pymeta_generated_code/pymeta_grammar__Grammar.py in _G_consumedby_272()&#xA;    977                     return (_G_apply_274, self.currentError)&#xA;--&amp;gt; 978                 _G_many1_275, lastError = self.many(_G_many1_273, _G_many1_273())&#xA;    979                 self.considerError(lastError, None)&#xA;&#xA;/pymeta_generated_code/pymeta_grammar__Grammar.py in _G_many1_273()&#xA;    974                     self._trace('&amp;lt;dn', (5909, 5912), self.input.position)&#xA;--&amp;gt; 975                     _G_apply_274, lastError = self._apply(self.rule_dna, &amp;quot;dna&amp;quot;, [])&#xA;    976                     self.considerError(lastError, None)&#xA;&#xA;~/playground/python/hgvs/venv/lib/python3.9/site-packages/ometa/runtime.py in _apply(self, rule, ruleName, args)&#xA;    494                 memoRec = self.input.setMemo(ruleName,&#xA;--&amp;gt; 495                                          [rule(), self.input])&#xA;    496             except ParseError as e:&#xA;&#xA;/pymeta_generated_code/pymeta_grammar__Grammar.py in rule_dna(self)&#xA;   2667             self._trace('= dna_iupa', (11894, 11904), self.input.position)&#xA;-&amp;gt; 2668             _G_apply_809, lastError = self._apply(self.rule_dna_iupac, &amp;quot;dna_iupac&amp;quot;, [])&#xA;   2669             self.considerError(lastError, 'dna')&#xA;&#xA;~/playground/python/hgvs/venv/lib/python3.9/site-packages/ometa/runtime.py in _apply(self, rule, ruleName, args)&#xA;    494                 memoRec = self.input.setMemo(ruleName,&#xA;--&amp;gt; 495                                          [rule(), self.input])&#xA;    496             except ParseError as e:&#xA;&#xA;/pymeta_generated_code/pymeta_grammar__Grammar.py in rule_dna_iupac(self)&#xA;   2682                 return (_G_python_813, self.currentError)&#xA;-&amp;gt; 2683             _G_pred_814, lastError = self.pred(_G_pred_811)&#xA;   2684             self.considerError(lastError, 'dna_iupac')&#xA;&#xA;~/playground/python/hgvs/venv/lib/python3.9/site-packages/ometa/runtime.py in pred(self, expr)&#xA;    648         if not val:&#xA;--&amp;gt; 649             raise e&#xA;    650         else:&#xA;&#xA;ParseError: &#xA;NM_004006.2:c.940_941ins[903_940inv;851_885inv]&#xA;                        ^&#xA;Parse error at line 1, column 24: Syntax error. trail: [dna_iupac dna dna_ins dna_edit c_posedit c_variant hgvs_variant]&#xA;&#xA;&#xA;During handling of the above exception, another exception occurred:&#xA;&#xA;HGVSParseError                            Traceback (most recent call last)&#xA;&amp;lt;ipython-input-6-9d483e266162&amp;gt; in &amp;lt;module&amp;gt;&#xA;----&amp;gt; 1 var_g = hp.parse_hgvs_variant('NM_004006.2:c.940_941ins[903_940inv;851_885inv]')&#xA;&#xA;~/playground/python/hgvs/venv/lib/python3.9/site-packages/hgvs/parser.py in rule_fxn(s)&#xA;    130                     return self._grammar(s).__getattr__(rule_name)()&#xA;    131                 except ometa.runtime.ParseError as exc:&#xA;--&amp;gt; 132                     raise HGVSParseError(&amp;quot;{s}: char {exc.position}: {reason}&amp;quot;.format(&#xA;    133                         s=s, exc=exc, reason=exc.formatReason()))&#xA;    134 &#xA;&#xA;HGVSParseError: NM_004006.2:c.940_941ins[903_940inv;851_885inv]: char 24: Syntax error&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14707" LastActivityDate="2021-12-13T09:32:22.843" Title="HGVS python package : How to parse complex insertions?" Tags="|python|variants|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18198" PostTypeId="1" CreationDate="2021-12-14T03:51:40.900" Score="1" ViewCount="106" Body="&lt;p&gt;I have two vcf files, one containing samples with TP53 mutations, and one containing samples with no TP53 mutations. The vcf without the mutations is the control and the vcf with the mutations is the case. How can I combine these two files into something that can be processed by the --assoc function in plink 1.9.0? Do I need to make them into one vcf? Or is it not necessary, and I can just use --assoc on each file after converting each individually?&lt;/p&gt;&#xA;" OwnerUserId="12859" LastActivityDate="2021-12-15T11:00:04.307" Title="Converting Control and Case vcf files into a format that can be processed by plink 1.9.0" Tags="|vcf|plink|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18199" PostTypeId="2" ParentId="18198" CreationDate="2021-12-14T19:10:19.680" Score="1" Body="&lt;p&gt;Both &lt;a href=&quot;http://www.cog-genomics.org/plink/1.9/input#vcf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Plink 1.9&lt;/a&gt; [uses &lt;code&gt;--assoc&lt;/code&gt;] and &lt;a href=&quot;http://www.cog-genomics.org/plink/2.0/input#vcf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Plink 2&lt;/a&gt; [uses &lt;code&gt;--glm&lt;/code&gt;] can process VCF files directly; no conversion needed.&lt;/p&gt;&#xA;&lt;p&gt;I think that Plink will combine multiple files specified on the command line into one file, but if this is not the case then &lt;a href=&quot;https://www.htslib.org/doc/1.1/bcftools.html#merge&quot; rel=&quot;nofollow noreferrer&quot;&gt;bcftools&lt;/a&gt; can be used to merge them.&lt;/p&gt;&#xA;&lt;p&gt;As described in those input specifications, you can provide Plink with a &lt;code&gt;.fam&lt;/code&gt; file (via &lt;code&gt;--fam&lt;/code&gt;) to add in phenotype data.&lt;/p&gt;&#xA;&lt;p&gt;Information about the &lt;code&gt;.fam&lt;/code&gt; formats can be found &lt;a href=&quot;http://www.cog-genomics.org/plink/1.9/formats#fam&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. It's a 6-column headerless space-separated format containing the following fields:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Family ID ('FID')&lt;/li&gt;&#xA;&lt;li&gt;Within-family ID ('IID'; cannot be '0')&lt;/li&gt;&#xA;&lt;li&gt;Within-family ID of father ('0' if father isn't in dataset)&lt;/li&gt;&#xA;&lt;li&gt;Within-family ID of mother ('0' if mother isn't in dataset)&lt;/li&gt;&#xA;&lt;li&gt;Sex code ('1' = male, '2' = female, '0' = unknown)&lt;/li&gt;&#xA;&lt;li&gt;Phenotype value ('1' = control, '2' = case, '-9'/'0'/non-numeric = missing data if case/control)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The phenotype column can be quantitative as well; which is triggered when a numeric value is seen that is not one of the above qualitative codes.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2021-12-15T11:00:04.307" LastActivityDate="2021-12-15T11:00:04.307" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18200" PostTypeId="1" AcceptedAnswerId="18203" CreationDate="2021-12-15T04:38:05.333" Score="1" ViewCount="170" Body="&lt;p&gt;Attached are two kmer plots from two closely related species. Is that safe to say that the one on the left has higher homozygosity than the one in the right k-mer plot, due to a low to almost flat blue peak (2x) in the left plot? My concern is that the blue peaks in the two plots are not in the same position: left plot at 4n, while right plot at 2n.&#xA;I am aware of the grey area in the right plot indicating missing content in assembly, likely due to reads from multiple individuals (correct me if I am wrong).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uymiG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uymiG.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Model for kmer plot on the left&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Formula: y_transform ~ x^transform_exp * length * predict2_0(r1, k, d, &#xA;    kmercov, bias, x)&#xA;&#xA;Parameters:&#xA;         Estimate Std. Error t value Pr(&amp;gt;|t|)    &#xA;d       4.019e-02  1.262e-03   31.84   &amp;lt;2e-16 ***&#xA;r1      1.014e-02  1.664e-04   60.94   &amp;lt;2e-16 ***&#xA;kmercov 1.996e+01  2.452e-02  814.12   &amp;lt;2e-16 ***&#xA;bias    1.900e+00  2.495e-02   76.17   &amp;lt;2e-16 ***&#xA;length  4.553e+08  8.993e+05  506.34   &amp;lt;2e-16 ***&#xA;---&#xA;Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1&#xA;&#xA;Residual standard error: 4024000 on 1995 degrees of freedom&#xA;&#xA;Number of iterations to convergence: 8 &#xA;Achieved convergence tolerance: 1.49e-08&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Model for kmer plot on the right&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Formula: y_transform ~ x^transform_exp * length * predict2_0(r1, k, d, &#xA;    kmercov, bias, x)&#xA;&#xA;Parameters:&#xA;         Estimate Std. Error t value Pr(&amp;gt;|t|)    &#xA;d       4.399e-02  1.407e-03   31.27   &amp;lt;2e-16 ***&#xA;r1      1.519e-02  1.002e-04  151.64   &amp;lt;2e-16 ***&#xA;kmercov 2.069e+01  2.416e-02  856.53   &amp;lt;2e-16 ***&#xA;bias    1.697e+00  2.099e-02   80.88   &amp;lt;2e-16 ***&#xA;length  5.069e+08  1.056e+06  480.10   &amp;lt;2e-16 ***&#xA;---&#xA;Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1&#xA;&#xA;Residual standard error: 4878000 on 1995 degrees of freedom&#xA;&#xA;Number of iterations to convergence: 7 &#xA;Achieved convergence tolerance: 1.49e-08&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Both the GenomeScope estimated genome sizes are smaller than the assembly (both have microorganisms filtered). The genome size of the left species is ~100Mb smaller than the one on the right, consistently in GenomeScope and the assembly.&lt;/p&gt;&#xA;&lt;h2&gt;GenomeScope Plots:&lt;/h2&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eNjLE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eNjLE.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9975" LastEditorUserId="57" LastEditDate="2022-01-04T12:12:43.840" LastActivityDate="2022-01-04T12:12:43.840" Title="Comparing homozygosity of k-mer plots" Tags="|genome|assembly|k-mer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18203" PostTypeId="2" ParentId="18200" CreationDate="2021-12-15T08:15:57.820" Score="1" Body="&lt;p&gt;I would say yes, but it is not that easy to see it on a histogram decomposed to overlapping distributions. While the decomposition to different ditributions is assembly dependent, the overall k-mer histogram is actually directly derived from the sequencing data.&lt;/p&gt;&#xA;&lt;p&gt;And these k-mer histograms can be used to estimate heterozygosity. For example, using genomescope. You can &lt;a href=&quot;https://github.com/tbenavi1/genomescope2.0&quot; rel=&quot;nofollow noreferrer&quot;&gt;install the tool locally&lt;/a&gt;, or fit the model on &lt;a href=&quot;http://qb.cshl.edu/genomescope/genomescope2.0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;a webserver&lt;/a&gt; (which seems to be down at the moment, but it should be back on at some point). For more details check &lt;a href=&quot;https://www.nature.com/articles/s41467-020-14998-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper&lt;/a&gt;. If you get the tool running, you can estimate heterozygosity in the two samples to see how much more/less heterozygous they are.&lt;/p&gt;&#xA;&lt;p&gt;P.S. I am always thinking about heterozygosity, which is supposed an equivalent of 1 - homozygosity, so I hope the answer will make sense to you.&lt;/p&gt;&#xA;&lt;p&gt;--- edit ---&lt;/p&gt;&#xA;&lt;p&gt;so, looking at the gneomescope models and knowing that genome size estimate matches the assembly size, I think it's fair to conclude the right genome is more heterozygous than the left one. However, keep in mind that it is not super great sequencing dataset, as the error peak and heterozygous peak are not clearly separated, hence the estimates of genomic features might be less accurate.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2021-12-22T20:01:25.467" LastActivityDate="2021-12-22T20:01:25.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18205" PostTypeId="1" CreationDate="2021-12-15T13:18:29.630" Score="1" ViewCount="55" Body="&lt;p&gt;I have some data indicating there might be a splicing variant of the Arabidopsis Thaliana protein I'm studying that has not been identified. Is there a database of i.e. RNA sequences (transcriptome?) that I could search for it, knowing a part of its sequence?&lt;/p&gt;&#xA;" OwnerUserId="14629" LastActivityDate="2023-09-07T21:07:15.880" Title="How to find an unidentified splicing variant of a protein?" Tags="|rna-seq|transcriptome|rna-splicing|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18207" PostTypeId="1" CreationDate="2021-12-15T16:16:36.777" Score="1" ViewCount="189" Body="&lt;p&gt;I am currently in the process of removing adapters and primers from some 16S data acquired, and have a conceptual question regarding this pre-processing step:&lt;/p&gt;&#xA;&lt;p&gt;So there are a series of different primer sequences (4 for the forward and 4 for the reverse strands), as well as two adapter sequences (1 for each strand) overhanging these primers.&lt;/p&gt;&#xA;&lt;p&gt;To test this process I am currently using &lt;code&gt;cutadapt&lt;/code&gt;. Just to give you an example of what one of my primers vs. adapters look like:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;&amp;quot;ACACTCTTTCCCTACACGACGCTCTTCCGATCTGTGCCAGCMGCCGCGGTAA&amp;quot; -&amp;gt; fp&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;&amp;quot;ACACTCTTTCCCTACACGACGCTCTTCCGATCT&amp;quot; -&amp;gt; a1&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;So my question is: Given that the adapters are overhanging these primers (a1 is a an exact match with first N base pairs in fp), could we just use &lt;code&gt;cutadapt&lt;/code&gt; once, in order to remove the primers + adapter sequences in one go?&lt;/p&gt;&#xA;" OwnerUserId="4001" LastActivityDate="2021-12-15T16:16:36.777" Title="Removing adapters + primers from fastq files" Tags="|fastq|linux|metagenome|qc|cutadapt|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18210" PostTypeId="1" AcceptedAnswerId="18215" CreationDate="2021-12-16T03:08:45.390" Score="0" ViewCount="77" Body="&lt;p&gt;Why are there shorter contigs from transcriptome assembly than from a whole genome assembly? I know the difference between transcriptome and genome, but don't really understand what contigs are in the context of sequencing in bioinformatics, and why are the contigs shorter in transcriptome assembly than in whole genome assembly. Anyone mind explaining? Much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="14721" LastEditorUserId="298" LastEditDate="2021-12-16T17:24:42.810" LastActivityDate="2021-12-16T21:59:20.680" Title="Length of Contigs in Transcriptome and Whole Genome Assembly" Tags="|assembly|transcriptome|genome-sequencing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18213" PostTypeId="2" ParentId="18205" CreationDate="2021-12-16T16:01:19.990" Score="1" Body="&lt;p&gt;There's not going to be a specific database that's particularly thorough for this. The best you can do is one of the following:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Search through TAIR (&lt;a href=&quot;https://www.arabidopsis.org/servlets/Search?action=new_search&amp;amp;type=protein&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.arabidopsis.org/servlets/Search?action=new_search&amp;amp;type=protein&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Blast SRA with the full length sequence ( &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PROGRAM=blastn&amp;amp;BLAST_PROGRAMS=megaBlast&amp;amp;PAGE_TYPE=BlastSearch&amp;amp;BLAST_SPEC=SRA&amp;amp;SHOW_DEFAULTS=on&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PROGRAM=blastn&amp;amp;BLAST_PROGRAMS=megaBlast&amp;amp;PAGE_TYPE=BlastSearch&amp;amp;BLAST_SPEC=SRA&amp;amp;SHOW_DEFAULTS=on&lt;/a&gt;). This will unfortunately pull up a LOT of related hits for every other related isoform.&lt;/li&gt;&#xA;&lt;li&gt;Download some public RNA-seq data from SRA/ENA and see if it has supporting data.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;You're probably best off doing all of these, where you use blast to find relevant datasets that you then align. My guess is that TAIR won't have everything, since this has probably been seen before but not reported since the experiment wasn't intending to find new proteins.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2021-12-16T16:01:19.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18215" PostTypeId="2" ParentId="18210" CreationDate="2021-12-16T19:27:35.017" Score="3" Body="&lt;p&gt;The longest possible transcriptome contig reflects the longest possible transcript. In the human genome, that's possibly &lt;a href=&quot;https://en.wikipedia.org/wiki/Titin&quot; rel=&quot;nofollow noreferrer&quot;&gt;Titin&lt;/a&gt;, which is ~35K aa (that's ~105Kb) and will be longer pre-splicing.&lt;/p&gt;&#xA;&lt;p&gt;The longest possible genome contig reflects the longest chromosome. In the human genome that's chr1 at ~250Mb.&lt;/p&gt;&#xA;&lt;p&gt;In practise your contigs/scaffolds will be a lot shorter for various reasons (e.g. hard to sequence the telomeres and repeats) but it illustrates the difference since all the chromosomes consist of contiguous DNA but many separate RNA transcripts (and not all DNA is transcribed).&lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="104" LastEditDate="2021-12-16T21:59:20.680" LastActivityDate="2021-12-16T21:59:20.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18218" PostTypeId="1" AcceptedAnswerId="18434" CreationDate="2021-12-17T10:16:42.750" Score="2" ViewCount="115" Body="&lt;p&gt;I'm trying to visualize mutations in SARS-CoV-2 sequences, somewhat like it's done on outbreak.info:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ziBPR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ziBPR.png&quot; alt=&quot;An overview of mutations in the B.1.617.2 lineage, shown as dots labeled with amino acids on a horizontal bar divided by genes.&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;but for individual sequences, probably on the base of the vcf files generated by the &lt;a href=&quot;https://github.com/artic-network/fieldbioinformatics&quot; rel=&quot;nofollow noreferrer&quot;&gt;ARTIC pipeline&lt;/a&gt;. A pipeline focused on virus nanopore output.&lt;/p&gt;&#xA;&lt;p&gt;The ready-made tools I've found for visualization of mutations so far were specifically tailored to human sequences and/or for visualizing aggregated data for multiple genomes, or were web tools, which can't be used for medical data due to confidentiality.&lt;/p&gt;&#xA;&lt;p&gt;Is there any existing offline tool that&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;is Linux-based&lt;/li&gt;&#xA;&lt;li&gt;can be integrated into an automated pipeline&lt;/li&gt;&#xA;&lt;li&gt;is tailored towards viral genomes or is &amp;quot;organism-agnostic&amp;quot;&lt;/li&gt;&#xA;&lt;li&gt;is based on comparing an individual sequence to a reference sequence&lt;/li&gt;&#xA;&lt;li&gt;shows the sequence as &amp;quot;blocks&amp;quot; of genes with the mutations highlighted&lt;/li&gt;&#xA;&lt;li&gt;(ideally plays well with MultiQC for integration into an existing report, but that's a nice-to-have)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;?&lt;/p&gt;&#xA;" OwnerUserId="12755" LastEditorUserId="3848" LastEditDate="2022-02-27T02:55:11.617" LastActivityDate="2022-02-27T02:55:11.617" Title="Visualization of mutations in (part of) viral genome - existing tools?" Tags="|vcf|visualization|covid-19|software-recommendation|sars-cov-2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18219" PostTypeId="1" CreationDate="2021-12-17T18:40:51.207" Score="1" ViewCount="137" Body="&lt;p&gt;I seem to have determined that there are &lt;a href=&quot;https://gist.github.com/lancejpollard/afd70e3a84b8545f5196adafac41b474&quot; rel=&quot;nofollow noreferrer&quot;&gt;287,614 unique Latin taxonomic names&lt;/a&gt; used in the biological taxonomy according to the &lt;a href=&quot;https://www.gbif.org/dataset/9ca92552-f23a-41a8-a140-01abaa31c931#description&quot; rel=&quot;nofollow noreferrer&quot;&gt;Integrated Taxonomic Information System (ITIS) taxonomy taxon.txt file&lt;/a&gt;. First of all, that is an enormous number of unique names, I would have hoped they could be reused to a greater degree but I guess not. There are 90k some odd genera names, the rest are species names, or certain names might be duplicated throughout the complete taxonomy.&lt;/p&gt;&#xA;&lt;p&gt;What I would like to do is take the taxonomy Latin names and convert them into their English equivalents. Not their &amp;quot;common names&amp;quot;, as that can't be done by looking at the Latin name alone. But by taking their Latin name and making it into the equivalent English literal meaning. Wikipedia has a nice list on &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_commonly_used_taxonomic_affixes&quot; rel=&quot;nofollow noreferrer&quot;&gt;commonly used taxonomic affixes&lt;/a&gt;, and another on &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_Latin_and_Greek_words_commonly_used_in_systematic_names&quot; rel=&quot;nofollow noreferrer&quot;&gt;Latin and Greek words commonly used in systematic names&lt;/a&gt;, however a quick glance at the list and they both are missing &lt;code&gt;-bacter-&lt;/code&gt; as one infix, for a single example, or &lt;code&gt;vorans&lt;/code&gt;, or many many others. The Wikipedia list seems to only contain a small fraction of the names used in the actual complete taxonomy.&lt;/p&gt;&#xA;&lt;p&gt;What I'm wondering is if there exist anywhere online in text format (so it can be copy-pasted) a complete list of all the Latin affixes (prefixes, infixes, suffixes, etc.) so I can accomplish my goal. The goal being, programmatically convert the Latin names to their literal English equivalent. I would split apart the Latin name based on some database of Latin affixes, and then plop out the English word matching the affix. It should take only a few seconds for a computer to do this transformation, assuming the list mapping Latin names to English names exists already.&lt;/p&gt;&#xA;&lt;p&gt;Does anything like this exist anywhere on the web?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.learnthat.org/pages/view/roots.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;This&lt;/a&gt; calls itself the &amp;quot;worlds largest affix directory&amp;quot;, yet it still doesn't have these common ones... &lt;a href=&quot;https://en.wikipedia.org/wiki/List_of_Greek_and_Latin_roots_in_English/P%E2%80%93Z&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here's&lt;/a&gt; another Wikipedia page, perhaps this is as good as it is going to get?&lt;/p&gt;&#xA;" OwnerUserId="9787" LastActivityDate="2021-12-17T20:09:55.203" Title="List of all Latin word parts used in Biological Taxonomy?" Tags="|database|taxonomy|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18220" PostTypeId="2" ParentId="18219" CreationDate="2021-12-17T20:09:55.203" Score="1" Body="&lt;p&gt;Maybe you can use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/taxonomy&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Taxonomy&lt;/a&gt; instead, which is a full hierarchical taxonomy with &amp;gt;700K taxa. You can use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK25497/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez utilities&lt;/a&gt; to programmatically query NCBI taxonomy to retrieve Latin or common names as well as other metadata.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2021-12-17T20:09:55.203" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18221" PostTypeId="1" AcceptedAnswerId="18250" CreationDate="2021-12-18T05:53:00.363" Score="1" ViewCount="84" Body="&lt;p&gt;Two-parted question:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Where is a cheatsheet or &lt;em&gt;complete&lt;/em&gt; list of the &amp;quot;formal&amp;quot; IUPAC naming conventions?&lt;/li&gt;&#xA;&lt;li&gt;Where is a cheatsheet or complete list of the &amp;quot;trivial&amp;quot; or &amp;quot;informal&amp;quot; chemical names?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Does the &lt;a href=&quot;https://en.wikipedia.org/wiki/IUPAC_nomenclature_of_organic_chemistry&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia page&lt;/a&gt; have &lt;em&gt;all&lt;/em&gt; of the naming conventions, or are some missing? Then &lt;a href=&quot;https://www.acdlabs.com/iupac/nomenclature/93/r93_671.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page titled &amp;quot;Trivial and semisystematic names retained for naming organic compounds&amp;quot;&lt;/a&gt; seems to link to some of the trivial names, but it is in a terrible format for a computer to read (I would like a spreadsheet or some sort of structured data format so I can use these in code) and I'm not sure it's complete. I have searched around on the web but don't seem to find anything that is complete and ideally as structured data. Does anything like this exist? If not structured data, what is the easiest document to read that is complete that I could manually convert into structured data? Also, any of this should ideally be a free resource.&lt;/p&gt;&#xA;" OwnerUserId="9787" LastActivityDate="2021-12-25T14:47:11.300" Title="Complete List of IUPAC Chemical Naming Conventions for Formal and Trivial Names?" Tags="|database|cheminformatics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18222" PostTypeId="1" CreationDate="2021-12-18T23:58:49.863" Score="0" ViewCount="46" Body="&lt;p&gt;I have a large vcf file with both case and control samples in the file. I am planning to input the vcf into the --assoc function of plink with the --fam parameter that contains a .fam file that specifies which samples are case and which are control. I want to make a .fam file with the case and control samples labelled in the phenotype value column. How can I assign case/control to different samples for the .fam of my merged vcf? My big vcf file doesn't indicate whether the samples are case or control. I have two tsv files, one with a list of control samples and one with a list of case samples. Can I use these two files to specify in the .fam file which samples are case and which are control?&lt;/p&gt;&#xA;&lt;p&gt;Could I also set the specific gene mutation as the case and samples without the mutation as the control, and run an association test based on those parameters?&lt;/p&gt;&#xA;" OwnerUserId="12859" LastActivityDate="2021-12-18T23:58:49.863" Title="Assigning case and control label to different samples of large merged file to input into a plink 1.9.0 association test" Tags="|vcf|plink|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18223" PostTypeId="2" ParentId="18177" CreationDate="2021-12-19T01:08:22.137" Score="0" Body="&lt;p&gt;FWIW, In the &lt;a href=&quot;https://sequenceserver.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sequenceserver BLAST software&lt;/a&gt;, you can just click on &amp;quot;download alignment&amp;quot; to extract the aligning region.&lt;/p&gt;&#xA;" OwnerUserId="2271" LastActivityDate="2021-12-19T01:08:22.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18224" PostTypeId="2" ParentId="14113" CreationDate="2021-12-19T01:22:07.263" Score="2" Body="&lt;p&gt;BLAST isn't appropriate for multiple sequence alignment.&#xA;BLAST generates pairwise alignments. Furthermore, the aligning regions shown in a BLAST report only include subsets of the hit sequences.&lt;/p&gt;&#xA;&lt;p&gt;To create multiple sequence alignment of hits you want to:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;run blast&lt;/li&gt;&#xA;&lt;li&gt;extract the hit sequences&lt;/li&gt;&#xA;&lt;li&gt;perform multiple-sequence-alignment (e.g., using MAFFT)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Extracting hit sequences can be done using the command-line. Check the docs for &lt;code&gt;blastdbcmd&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you prefer a point-and-click interface, you can also do this using my lab's &lt;a href=&quot;https://sequenceserver.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;SequenceServer BLAST software&lt;/a&gt; by simply selecting the hits that you want to retain, and in the &lt;strong&gt;Download&lt;/strong&gt; section, clicking &amp;quot;FASTA of selected hits&amp;quot;. Or if you want all of them, click &amp;quot;FASTA of all hits&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://sequenceserver.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://sequenceserver.com/img/blast-setup/normal-blast-result.png&quot; alt=&quot;Download hits from BLAST report &quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2271" LastEditorUserId="2271" LastEditDate="2021-12-21T19:40:59.833" LastActivityDate="2021-12-21T19:40:59.833" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18226" PostTypeId="1" CreationDate="2021-12-19T22:55:12.333" Score="2" ViewCount="262" Body="&lt;p&gt;I have been trying to follow the GATK Best Practice Workflow for 'Data pre-processing for variant discovery' (&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035535912&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360035535912&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;This has all been run on Windows Subsystem for Linux 2 on the Bash shell.&lt;/p&gt;&#xA;&lt;p&gt;I started off with FASTQ files from IGSR (&lt;a href=&quot;https://www.internationalgenome.org/data-portal&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.internationalgenome.org/data-portal&lt;/a&gt;) and performed alignment with Bowtie2 (instead of BWA).&lt;/p&gt;&#xA;&lt;p&gt;This produced a SAM file which I converted to a BAM file using SAMTOOLS.&lt;/p&gt;&#xA;&lt;p&gt;I have been trying to use MarkDuplicatesSpark (&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/4409897162139-MarkDuplicatesSpark&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/4409897162139-MarkDuplicatesSpark&lt;/a&gt;) but am getting an output that I can't make sense of.&lt;/p&gt;&#xA;&lt;p&gt;I am running the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gatk MarkDuplicatesSpark -I HG00102_hGRCH38_exome_aignment.bam -O HG00102_hGRCH38_exome_aignment.marked_duplicates.bam --remove-sequencing-duplicates --spark-master local[*]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am getting the following output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Using GATK jar /mnt/c/Users/angus/Documents/Bioinformatics/GATK4/gatk-4.2.4.0/gatk-package-4.2.4.0-local.jar&#xA;Running:&#xA;    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /mnt/c/Users/angus/Documents/Bioinformatics/GATK4/gatk-4.2.4.0/gatk-package-4.2.4.0-local.jar MarkDuplicatesSpark -I HG00102_hGRCH38_exome_aignment.bam -O HG00102_hGRCH38_exome_aignment.marked_duplicates.bam --remove-sequencing-duplicates --spark-master local[*]&#xA;&#xA;22:45:08.487 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/mnt/c/Users/angus/Documents/Bioinformatics/GATK4/gatk-4.2.4.0/gatk-package-4.2.4.0-local.jar!/com/intel/gkl/native/libgkl_compression.so&#xA;Dec 19, 2021 10:45:08 PM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine&#xA;INFO: Failed to detect whether we are running on Google Compute Engine.&#xA;22:45:08.748 INFO  MarkDuplicatesSpark - ------------------------------------------------------------&#xA;22:45:08.748 INFO  MarkDuplicatesSpark - The Genome Analysis Toolkit (GATK) v4.2.4.0&#xA;22:45:08.748 INFO  MarkDuplicatesSpark - For support and documentation go to https://software.broadinstitute.org/gatk/&#xA;22:45:08.748 INFO  MarkDuplicatesSpark - Executing as *removed* on Linux v4.19.128-microsoft-standard amd64&#xA;22:45:08.749 INFO  MarkDuplicatesSpark - Java runtime: OpenJDK 64-Bit Server VM v11.0.9.1+1-Ubuntu-0ubuntu1.18.04&#xA;22:45:08.749 INFO  MarkDuplicatesSpark - Start Date/Time: December 19, 2021 at 10:45:08 PM GMT&#xA;22:45:08.749 INFO  MarkDuplicatesSpark - ------------------------------------------------------------&#xA;22:45:08.749 INFO  MarkDuplicatesSpark - ------------------------------------------------------------&#xA;22:45:08.750 INFO  MarkDuplicatesSpark - HTSJDK Version: 2.24.1&#xA;22:45:08.750 INFO  MarkDuplicatesSpark - Picard Version: 2.25.4&#xA;22:45:08.751 INFO  MarkDuplicatesSpark - Built for Spark Version: 2.4.5&#xA;22:45:08.751 INFO  MarkDuplicatesSpark - HTSJDK Defaults.COMPRESSION_LEVEL : 2&#xA;22:45:08.751 INFO  MarkDuplicatesSpark - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false&#xA;22:45:08.751 INFO  MarkDuplicatesSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true&#xA;22:45:08.751 INFO  MarkDuplicatesSpark - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false&#xA;22:45:08.751 INFO  MarkDuplicatesSpark - Deflater: IntelDeflater&#xA;22:45:08.751 INFO  MarkDuplicatesSpark - Inflater: IntelInflater&#xA;22:45:08.751 INFO  MarkDuplicatesSpark - GCS max retries/reopens: 20&#xA;22:45:08.752 INFO  MarkDuplicatesSpark - Requester pays: disabled&#xA;22:45:08.752 INFO  MarkDuplicatesSpark - Initializing engine&#xA;22:45:08.752 INFO  MarkDuplicatesSpark - Done initializing engine&#xA;Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties&#xA;21/12/19 22:45:09 WARN Utils: Your hostname, *hostname removed* resolves to a loopback address: *IP removed*; using 172.27.147.65 instead (on interface eth0)&#xA;21/12/19 22:45:09 WARN Utils: Set SPARK_LOCAL_IP if you need to bind to another address&#xA;WARNING: An illegal reflective access operation has occurred&#xA;WARNING: Illegal reflective access by org.apache.spark.unsafe.Platform (file:/mnt/c/Users/angus/Documents/Bioinformatics/GATK4/gatk-4.2.4.0/gatk-package-4.2.4.0-local.jar) to method java.nio.Bits.unaligned()&#xA;WARNING: Please consider reporting this to the maintainers of org.apache.spark.unsafe.Platform&#xA;WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations&#xA;WARNING: All illegal access operations will be denied in a future release&#xA;21/12/19 22:45:10 INFO SparkContext: Running Spark version 2.4.5&#xA;21/12/19 22:45:10 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable&#xA;21/12/19 22:45:10 INFO SparkContext: Submitted application: MarkDuplicatesSpark&#xA;21/12/19 22:45:10 INFO SecurityManager: Changing view acls to: abgane&#xA;21/12/19 22:45:10 INFO SecurityManager: Changing modify acls to: abgane&#xA;21/12/19 22:45:10 INFO SecurityManager: Changing view acls groups to:&#xA;21/12/19 22:45:10 INFO SecurityManager: Changing modify acls groups to:&#xA;21/12/19 22:45:10 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users  with view permissions: Set(abgane); groups with view permissions: Set(); users  with modify permissions: Set(abgane); groups with modify permissions: Set()&#xA;21/12/19 22:45:10 INFO Utils: Successfully started service 'sparkDriver' on port 35319.&#xA;21/12/19 22:45:10 INFO SparkEnv: Registering MapOutputTracker&#xA;21/12/19 22:45:11 INFO SparkEnv: Registering BlockManagerMaster&#xA;21/12/19 22:45:11 INFO BlockManagerMasterEndpoint: Using org.apache.spark.storage.DefaultTopologyMapper for getting topology information&#xA;21/12/19 22:45:11 INFO BlockManagerMasterEndpoint: BlockManagerMasterEndpoint up&#xA;21/12/19 22:45:11 INFO DiskBlockManager: Created local directory at /tmp/blockmgr-1e5e323f-dd90-4093-980c-0ceda43349c2&#xA;21/12/19 22:45:11 INFO MemoryStore: MemoryStore started with capacity 3.6 GB&#xA;21/12/19 22:45:11 INFO SparkEnv: Registering OutputCommitCoordinator&#xA;21/12/19 22:45:11 INFO Utils: Successfully started service 'SparkUI' on port 4040.&#xA;21/12/19 22:45:11 INFO SparkUI: Bound SparkUI to 0.0.0.0, and started at http://172.27.147.65:4040&#xA;21/12/19 22:45:11 INFO Executor: Starting executor ID driver on host localhost&#xA;21/12/19 22:45:11 INFO Utils: Successfully started service 'org.apache.spark.network.netty.NettyBlockTransferService' on port 33189.&#xA;21/12/19 22:45:11 INFO NettyBlockTransferService: Server created on 172.27.147.65:33189&#xA;21/12/19 22:45:11 INFO BlockManager: Using org.apache.spark.storage.RandomBlockReplicationPolicy for block replication policy&#xA;21/12/19 22:45:11 INFO BlockManagerMaster: Registering BlockManager BlockManagerId(driver, 172.27.147.65, 33189, None)&#xA;21/12/19 22:45:11 INFO BlockManagerMasterEndpoint: Registering block manager 172.27.147.65:33189 with 3.6 GB RAM, BlockManagerId(driver, 172.27.147.65, 33189, None)&#xA;21/12/19 22:45:11 INFO BlockManagerMaster: Registered BlockManager BlockManagerId(driver, 172.27.147.65, 33189, None)&#xA;21/12/19 22:45:11 INFO BlockManager: Initialized BlockManager: BlockManagerId(driver, 172.27.147.65, 33189, None)&#xA;22:45:11.804 INFO  MarkDuplicatesSpark - Spark verbosity set to INFO (see --spark-verbosity argument)&#xA;21/12/19 22:45:11 INFO GoogleHadoopFileSystemBase: GHFS version: 1.9.4-hadoop3&#xA;21/12/19 22:45:12 INFO MemoryStore: Block broadcast_0 stored as values in memory (estimated size 172.9 KB, free 3.6 GB)&#xA;21/12/19 22:45:13 INFO MemoryStore: Block broadcast_0_piece0 stored as bytes in memory (estimated size 35.5 KB, free 3.6 GB)&#xA;21/12/19 22:45:13 INFO BlockManagerInfo: Added broadcast_0_piece0 in memory on 172.27.147.65:33189 (size: 35.5 KB, free: 3.6 GB)&#xA;21/12/19 22:45:13 INFO SparkContext: Created broadcast 0 from newAPIHadoopFile at PathSplitSource.java:96&#xA;21/12/19 22:45:13 INFO MemoryStore: Block broadcast_1 stored as values in memory (estimated size 172.9 KB, free 3.6 GB)&#xA;21/12/19 22:45:13 INFO MemoryStore: Block broadcast_1_piece0 stored as bytes in memory (estimated size 35.5 KB, free 3.6 GB)&#xA;21/12/19 22:45:13 INFO BlockManagerInfo: Added broadcast_1_piece0 in memory on 172.27.147.65:33189 (size: 35.5 KB, free: 3.6 GB)&#xA;21/12/19 22:45:13 INFO SparkContext: Created broadcast 1 from newAPIHadoopFile at PathSplitSource.java:96&#xA;21/12/19 22:45:13 INFO FileInputFormat: Total input files to process : 1&#xA;21/12/19 22:45:13 INFO SparkUI: Stopped Spark web UI at http://172.27.147.65:4040&#xA;21/12/19 22:45:13 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped!&#xA;21/12/19 22:45:13 INFO MemoryStore: MemoryStore cleared&#xA;21/12/19 22:45:13 INFO BlockManager: BlockManager stopped&#xA;21/12/19 22:45:13 INFO BlockManagerMaster: BlockManagerMaster stopped&#xA;21/12/19 22:45:13 INFO OutputCommitCoordinator&lt;span class=&quot;math-container&quot;&gt;$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped!&#xA;21/12/19 22:45:13 INFO SparkContext: Successfully stopped SparkContext&#xA;22:45:13.851 INFO  MarkDuplicatesSpark - Shutting down engine&#xA;[December 19, 2021 at 10:45:13 PM GMT] org.broadinstitute.hellbender.tools.spark.transforms.markduplicates.MarkDuplicatesSpark done. Elapsed time: 0.09 minutes.&#xA;Runtime.totalMemory()=419430400&#xA;java.lang.IllegalArgumentException: Unsupported class file major version 55&#xA;    at org.apache.xbean.asm6.ClassReader.&amp;lt;init&amp;gt;(ClassReader.java:166)&#xA;    at org.apache.xbean.asm6.ClassReader.&amp;lt;init&amp;gt;(ClassReader.java:148)&#xA;    at org.apache.xbean.asm6.ClassReader.&amp;lt;init&amp;gt;(ClassReader.java:136)&#xA;    at org.apache.xbean.asm6.ClassReader.&amp;lt;init&amp;gt;(ClassReader.java:237)&#xA;    at org.apache.spark.util.ClosureCleaner$&lt;/span&gt;.getClassReader(ClosureCleaner.scala:49)&#xA;        at org.apache.spark.util.FieldAccessFinder&lt;span class=&quot;math-container&quot;&gt;$$anon$3$$&lt;/span&gt;anonfun&lt;span class=&quot;math-container&quot;&gt;$visitMethodInsn$&lt;/span&gt;2.apply(ClosureCleaner.scala:517)&#xA;        at org.apache.spark.util.FieldAccessFinder&lt;span class=&quot;math-container&quot;&gt;$$anon$3$$&lt;/span&gt;anonfun&lt;span class=&quot;math-container&quot;&gt;$visitMethodInsn$&lt;/span&gt;2.apply(ClosureCleaner.scala:500)&#xA;        at scala.collection.TraversableLike&lt;span class=&quot;math-container&quot;&gt;$WithFilter$$anonfun$&lt;/span&gt;foreach&lt;span class=&quot;math-container&quot;&gt;$1.apply(TraversableLike.scala:733)&#xA;    at scala.collection.mutable.HashMap$$anon$&lt;/span&gt;1&lt;span class=&quot;math-container&quot;&gt;$$anonfun$foreach$2.apply(HashMap.scala:134)&#xA;    at scala.collection.mutable.HashMap$$&lt;/span&gt;anon&lt;span class=&quot;math-container&quot;&gt;$1$$anonfun$&lt;/span&gt;foreach&lt;span class=&quot;math-container&quot;&gt;$2.apply(HashMap.scala:134)&#xA;    at scala.collection.mutable.HashTable$&lt;/span&gt;class.foreachEntry(HashTable.scala:236)&#xA;        at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40)&#xA;        at scala.collection.mutable.HashMap&lt;span class=&quot;math-container&quot;&gt;$$anon$1.foreach(HashMap.scala:134)&#xA;    at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732)&#xA;    at org.apache.spark.util.FieldAccessFinder$$&lt;/span&gt;anon&lt;span class=&quot;math-container&quot;&gt;$3.visitMethodInsn(ClosureCleaner.scala:500)&#xA;    at org.apache.xbean.asm6.ClassReader.readCode(ClassReader.java:2175)&#xA;    at org.apache.xbean.asm6.ClassReader.readMethod(ClassReader.java:1238)&#xA;    at org.apache.xbean.asm6.ClassReader.accept(ClassReader.java:631)&#xA;    at org.apache.xbean.asm6.ClassReader.accept(ClassReader.java:355)&#xA;    at org.apache.spark.util.ClosureCleaner$$anonfun$&lt;/span&gt;org&lt;span class=&quot;math-container&quot;&gt;$apache$&lt;/span&gt;spark&lt;span class=&quot;math-container&quot;&gt;$util$&lt;/span&gt;ClosureCleaner&lt;span class=&quot;math-container&quot;&gt;$$clean$14.apply(ClosureCleaner.scala:307)&#xA;    at org.apache.spark.util.ClosureCleaner$$&lt;/span&gt;anonfun&lt;span class=&quot;math-container&quot;&gt;$org$&lt;/span&gt;apache&lt;span class=&quot;math-container&quot;&gt;$spark$&lt;/span&gt;util&lt;span class=&quot;math-container&quot;&gt;$ClosureCleaner$$clean$&lt;/span&gt;14.apply(ClosureCleaner.scala:306)&#xA;        at scala.collection.immutable.List.foreach(List.scala:392)&#xA;        at org.apache.spark.util.ClosureCleaner&lt;span class=&quot;math-container&quot;&gt;$.org$&lt;/span&gt;apache&lt;span class=&quot;math-container&quot;&gt;$spark$&lt;/span&gt;util&lt;span class=&quot;math-container&quot;&gt;$ClosureCleaner$$clean(ClosureCleaner.scala:306)&#xA;    at org.apache.spark.util.ClosureCleaner$&lt;/span&gt;.clean(ClosureCleaner.scala:162)&#xA;        at org.apache.spark.SparkContext.clean(SparkContext.scala:2326)&#xA;        at org.apache.spark.SparkContext.runJob(SparkContext.scala:2100)&#xA;        at org.apache.spark.SparkContext.runJob(SparkContext.scala:2126)&#xA;        at org.apache.spark.rdd.RDD&lt;span class=&quot;math-container&quot;&gt;$$anonfun$collect$1.apply(RDD.scala:990)&#xA;    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)&#xA;    at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)&#xA;    at org.apache.spark.rdd.RDD.withScope(RDD.scala:385)&#xA;    at org.apache.spark.rdd.RDD.collect(RDD.scala:989)&#xA;    at org.apache.spark.RangePartitioner$.sketch(Partitioner.scala:309)&#xA;    at org.apache.spark.RangePartitioner.&amp;lt;init&amp;gt;(Partitioner.scala:171)&#xA;    at org.apache.spark.RangePartitioner.&amp;lt;init&amp;gt;(Partitioner.scala:151)&#xA;    at org.apache.spark.rdd.OrderedRDDFunctions$$&lt;/span&gt;anonfun&lt;span class=&quot;math-container&quot;&gt;$sortByKey$&lt;/span&gt;1.apply(OrderedRDDFunctions.scala:62)&#xA;        at org.apache.spark.rdd.OrderedRDDFunctions$$anonfun$sortByKey$1.apply(OrderedRDDFunctions.scala:61)&#xA;        at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)&#xA;        at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)&#xA;        at org.apache.spark.rdd.RDD.withScope(RDD.scala:385)&#xA;        at org.apache.spark.rdd.OrderedRDDFunctions.sortByKey(OrderedRDDFunctions.scala:61)&#xA;        at org.apache.spark.api.java.JavaPairRDD.sortByKey(JavaPairRDD.scala:936)&#xA;        at org.broadinstitute.hellbender.utils.spark.SparkUtils.sortUsingElementsAsKeys(SparkUtils.java:165)&#xA;        at org.broadinstitute.hellbender.utils.spark.SparkUtils.sortReadsAccordingToHeader(SparkUtils.java:143)&#xA;        at org.broadinstitute.hellbender.utils.spark.SparkUtils.querynameSortReadsIfNecessary(SparkUtils.java:306)&#xA;        at org.broadinstitute.hellbender.tools.spark.transforms.markduplicates.MarkDuplicatesSpark.mark(MarkDuplicatesSpark.java:206)&#xA;        at org.broadinstitute.hellbender.tools.spark.transforms.markduplicates.MarkDuplicatesSpark.mark(MarkDuplicatesSpark.java:270)&#xA;        at org.broadinstitute.hellbender.tools.spark.transforms.markduplicates.MarkDuplicatesSpark.runTool(MarkDuplicatesSpark.java:350)&#xA;        at org.broadinstitute.hellbender.engine.spark.GATKSparkTool.runPipeline(GATKSparkTool.java:546)&#xA;        at org.broadinstitute.hellbender.engine.spark.SparkCommandLineProgram.doWork(SparkCommandLineProgram.java:31)&#xA;        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:140)&#xA;        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:192)&#xA;        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:211)&#xA;        at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)&#xA;        at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)&#xA;        at org.broadinstitute.hellbender.Main.main(Main.java:289)&#xA;21/12/19 22:45:13 INFO ShutdownHookManager: Shutdown hook called&#xA;21/12/19 22:45:13 INFO ShutdownHookManager: Deleting directory /tmp/spark-f49df209-c946-40e3-96ca-9d8125c3a733&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As an alternative, I am trying to sort the BAM file and use MarkDuplicates instead (&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/4409924785691-MarkDuplicates-Picard-&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/4409924785691-MarkDuplicates-Picard-&lt;/a&gt;).This appears to work fine. I obviously want to make use of multiprocessing though so ideally would like to get the Spark implementation going.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know what might be causing this command to fail?&lt;/p&gt;&#xA;&lt;p&gt;Any ideas for a fix?&lt;/p&gt;&#xA;&lt;p&gt;Any alternative software that might be useful?&lt;/p&gt;&#xA;&lt;p&gt;Thanks for your time,&lt;/p&gt;&#xA;&lt;p&gt;Angus&lt;/p&gt;&#xA;" OwnerUserId="14741" LastActivityDate="2023-06-11T15:02:20.310" Title="MarkDuplicatesSpark failing with cryptic error message. MarkDuplicates succeeds" Tags="|sequence-alignment|gatk|markduplicate|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18228" PostTypeId="2" ParentId="18136" CreationDate="2021-12-20T15:26:34.383" Score="1" Body="&lt;p&gt;The previous paragraph in the tutorial provides a link to the paper.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We’ll use standard filtering parameters: maxN=0 (DADA2 requires no Ns), truncQ=2, rm.phix=TRUE and maxEE=2. The maxEE parameter sets the maximum number of “expected errors” allowed in a read, which is &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/31/21/3476/194979&quot; rel=&quot;nofollow noreferrer&quot;&gt;a better filter than simply averaging quality scores.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="14745" LastActivityDate="2021-12-20T15:26:34.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18231" PostTypeId="1" CreationDate="2021-12-21T07:03:31.357" Score="1" ViewCount="61" Body="&lt;p&gt;I had to do a file transfer for in our system to run from fastqs and I came across the term &amp;quot;dummy fastq&amp;quot;, not sure what exactly it means and what is the purpose of it in the workflow. Can someone answer?&lt;/p&gt;&#xA;" OwnerUserId="14749" LastActivityDate="2021-12-21T11:22:12.230" Title="What are Dummy fastqs?" Tags="|ngs|linux|cancer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18232" PostTypeId="2" ParentId="18231" CreationDate="2021-12-21T09:57:58.587" Score="2" Body="&lt;p&gt;Probably some made-up data solely for the sake of testing a workflow but actually impossible to know what random files are sitting on your servers.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2021-12-21T11:22:12.230" LastActivityDate="2021-12-21T11:22:12.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18234" PostTypeId="2" ParentId="11042" CreationDate="2021-12-21T18:02:55.693" Score="0" Body="&lt;p&gt;You can try bioinfokit (&lt;a href=&quot;https://github.com/reneshbedre/bioinfokit&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/reneshbedre/bioinfokit&lt;/a&gt;) in python.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from bioinfokit.analys import gff&#xA;&#xA;gff.gff_to_gtf(file=&amp;quot;yourfile.gff3&amp;quot;) &#xA;# The converted gtf file will be saved in the same working directory.&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14755" LastEditorUserId="650" LastEditDate="2021-12-23T20:11:09.687" LastActivityDate="2021-12-23T20:11:09.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18235" PostTypeId="1" CreationDate="2021-12-21T21:29:03.700" Score="0" ViewCount="314" Body="&lt;p&gt;In certain UMAP visualization, it appears that the visualizations show the first UMAP &amp;quot;principal component&amp;quot; plotted against the second. For example, like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/m1UOS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/m1UOS.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Apparently, this lets us use the first and second UMAP &amp;quot;principal components&amp;quot; to tell apart the different cell clusterings in a dataset. I'm curious about if we were to use e.g. the second and the &lt;em&gt;third&lt;/em&gt; UMAP &amp;quot;principal components&amp;quot; instead - would this let us tell apart different cell clusterings according to the second and third features that explain those clusterings' largest sources of variation? If someone could explain this to me, I would be very grateful, and if you would like to correct whatever misunderstandings I have about UMAP as well, that would be great too - I'd be the first one to tell you that I don't understand it well!&lt;/p&gt;&#xA;" OwnerUserId="14758" LastEditorUserId="104" LastEditDate="2021-12-21T21:54:10.960" LastActivityDate="2022-05-20T23:02:50.260" Title="Using different &quot;principal components&quot; in UMAP visualizations" Tags="|scrnaseq|seurat|visualization|umap|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18236" PostTypeId="2" ParentId="18235" CreationDate="2021-12-21T21:50:44.733" Score="1" Body="&lt;p&gt;You can plot UMAP3 too in &lt;a href=&quot;https://github.com/Dragonmasterx87/Interactive-3D-Plotting-in-Seurat-3.0.0&quot; rel=&quot;nofollow noreferrer&quot;&gt;3 dimensions&lt;/a&gt;, but that's about the limit for viz. You could plot UMAP2 vs. UMAP3, UMAP3 vs. UMAP4 etc. but I'm not sure how informative it is. Instead, you can focus on other analyses like clustering, differential expression, cell type annotation, marker gene analysis, and other machine learning approaches etc. Ultimately, UMAP/t-SNE are nice tools for viz but they can be over interpreted- it is inherently impossible to summarise very highly dimensional data in just a few dimensions without losing information- see &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2021.08.25.457696v3&quot; rel=&quot;nofollow noreferrer&quot;&gt;here for an (overly) provocative take&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="104" LastEditorUserId="104" LastEditDate="2021-12-21T21:57:19.740" LastActivityDate="2021-12-21T21:57:19.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18238" PostTypeId="1" CreationDate="2021-12-22T01:40:58.997" Score="1" ViewCount="48" Body="&lt;p&gt;I have a gene tree file of 436 orthologue genes from 6 species. I want to remove unwanted extensions as it looks massy after visualization. My file looks like:&lt;/p&gt;&#xA;&lt;p&gt;(TRINITY_Clupea_DN5452_c0_g1_i1.p1:0.0824467436,TRINITY_Engraulis_DN43599_c0_g1_i1.p1:0.1634781085)100:0.0876433106,TRINITY_Sardina_DN15766_c0_g1_i2.p1:0.0164132018)..................&lt;/p&gt;&#xA;&lt;p&gt;What i need:&lt;/p&gt;&#xA;&lt;p&gt;(Clupea_DN5452:0.0824467436,Engraulis_DN43599:0.1634781085)100:0.0876433106,Sardina_DN15766:0.0164132018)..................&lt;/p&gt;&#xA;&lt;p&gt;As &amp;quot;TRINITY&amp;quot; is identicial, i can remove it using sed. But after the species name ids are not identical. And i only need the 2nd and 3rd part of the identifier.&lt;/p&gt;&#xA;&lt;p&gt;It will be helpful for me if i get some suggestion. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="12579" LastEditorUserId="73" LastEditDate="2021-12-23T21:16:02.120" LastActivityDate="2021-12-23T21:22:08.100" Title="Remove variable sequence component within a tree text file" Tags="|phylogenetics|identifiers|trinity|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18239" PostTypeId="2" ParentId="18238" CreationDate="2021-12-22T02:12:43.133" Score="1" Body="&lt;p&gt;Lots of ways to do this, but here's one way using &lt;a href=&quot;/questions/tagged/sed&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;sed&amp;#39;&quot; rel=&quot;tag&quot;&gt;sed&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sed 's/TRINITY_\([^_]\+\)_\([^_]\+\)_[^:]\+/\1_\2/g' file.tree&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above applies a substitution to capture the first two fields using parentheses (which need to be escaped). Each field captures any character not an underscore one or more times (using the &amp;quot;+&amp;quot; symbol, which also needs to be escaped). Finally, we also match any character not a colon one or more times. The replacement string can be just the first two capture groups separated with an underscore.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2021-12-22T05:01:20.483" LastActivityDate="2021-12-22T05:01:20.483" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18240" PostTypeId="1" AcceptedAnswerId="18243" CreationDate="2021-12-22T02:57:41.610" Score="1" ViewCount="449" Body="&lt;p&gt;I'm looking for intuition for why a randomized motif search works. My current thinking is as follows:&lt;/p&gt;&#xA;&lt;p&gt;We are selecting many random kmers from our DNA sequences. The chosen kmers will bias the profile matrix to selecting kmers like them.&lt;/p&gt;&#xA;&lt;p&gt;Given any particular k-mer chosen, there are two possibilities:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;We've selected a meaningless kmer, so the profile will bias selecting random (ie high entropy) kmers and the score will stop improving soon.&lt;/li&gt;&#xA;&lt;li&gt;We've captured a portion of the true motif. Since the motif isn't random (ie doesn't have high entropy) subsequent iterations using this profile matrix will pick motifs closer to the real motif and the score will keep improving.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Since we give this algorithm many tries and at some point it will eventually arrive at possibility 2 and come to a good conclusion.&lt;/p&gt;&#xA;&lt;p&gt;Is this a correct understanding of why randomized motif search works? Is there a more rigorous way to prove this?&lt;/p&gt;&#xA;" OwnerUserId="14160" LastEditorUserId="2108" LastEditDate="2021-12-23T22:01:16.553" LastActivityDate="2022-01-01T20:26:37.417" Title="Why does randomized motif search work?" Tags="|sequence-analysis|k-mer|algorithms|motifs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18242" PostTypeId="2" ParentId="18238" CreationDate="2021-12-23T21:22:08.100" Score="0" Body="&lt;p&gt;If you want to remove component/gene/isoform parts from a Trinity ID, you can do &lt;code&gt;s/_c[0-9]+_.*?:/:/&lt;/code&gt; to remove anything between the first c with only numbers after it up to the next colon. Combining this with the TRINITY removal at the start, it'd be something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sed 's/TRINITY_\(.*?\)_c[0-9]+_.*?:/\1:/g' file.tree&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-12-23T21:22:08.100" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18243" PostTypeId="2" ParentId="18240" CreationDate="2021-12-23T22:10:46.620" Score="2" Body="&lt;h2&gt;Update&lt;/h2&gt;&#xA;&lt;p&gt;With regard to comment, the proof of effectiveness of Monte Carlo algorithms like this is a little thorny- if you look at &lt;a href=&quot;https://en.wikipedia.org/wiki/Monte_Carlo_method&quot; rel=&quot;nofollow noreferrer&quot;&gt;the wikipedia page&lt;/a&gt;, you will see that a proof of statistical consistency for any Monte Carlo algorithms took a long time:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;From 1950 to 1996, all the publications on Sequential Monte Carlo methodologies, including the pruning and resample Monte Carlo methods introduced in computational physics and molecular chemistry, present natural and heuristic-like algorithms applied to different situations without a single proof of their consistency, nor a discussion on the bias of the estimates and on genealogical and ancestral tree based algorithms. The mathematical foundations and the first rigorous analysis of these particle algorithms were written by Pierre Del Moral in 1996.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://projecteuclid.org/journals/annals-of-applied-probability/volume-8/issue-2/Measure-valued-processes-and-interacting-particle-systems-Application-to-nonlinear/10.1214/aoap/1028903535.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is a possibly more general reference by Del Moral in 1998.&lt;/a&gt; The paper is somewhat dense and I am not an expert in the field so I don't think I'll do it justice, but I think that we could probably figure out a similar mathematical proof for random motif search given time and the right problem formulation.&lt;/p&gt;&#xA;&lt;p&gt;I sense that your concern might be with Monte Carlo algorithms generally rather than random motif search specifically, so hopefully that gets close to your question.&lt;/p&gt;&#xA;&lt;p&gt;It does look like &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC312096/pdf/nar00079-0138.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;people have done heuristic analysis of similar algorithms&lt;/a&gt;, but they are not formal proofs.&lt;/p&gt;&#xA;&lt;h2&gt;Original answer&lt;/h2&gt;&#xA;&lt;p&gt;I believe that you are more or less correct.&lt;/p&gt;&#xA;&lt;p&gt;For example, &lt;a href=&quot;http://bix.ucsd.edu/bioalgorithms/presentations/Ch12_RandAlgs.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this presentation's&lt;/a&gt; walkthrough of the algorithm (slides 35-36) specifically refers to &lt;strong&gt;Greedy&lt;/strong&gt; randomized profile motif searches. By making the greediness of the algorithm explicit, we can see that there is no claim of finding an optimal solution in any particular iteration, thus the need for multiple retries to get somewhere close to approximate solution.&lt;/p&gt;&#xA;&lt;p&gt;One additional point made in that presentation is that this algorithm works best when coupled to approaches such as Gibbs sampling for targeting the sampled k-mer chain in regions with higher scores (less entropic k-mers).&lt;/p&gt;&#xA;&lt;p&gt;A different learning resource with similar content (and a little more intuition) is available &lt;a href=&quot;https://www.bioinformaticsalgorithms.org/bioinformatics-chapter-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2022-01-01T20:26:37.417" LastActivityDate="2022-01-01T20:26:37.417" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18244" PostTypeId="2" ParentId="17597" CreationDate="2021-12-24T01:22:13.060" Score="2" Body="&lt;p&gt;DE results do not change from one run to another. The code given in your question will give identical results each time you run it.&lt;/p&gt;&#xA;" OwnerUserId="9042" LastEditorUserId="9042" LastEditDate="2021-12-24T06:32:27.523" LastActivityDate="2021-12-24T06:32:27.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18245" PostTypeId="2" ParentId="15535" CreationDate="2021-12-24T03:52:00.557" Score="1" Body="&lt;p&gt;edgeR functions do not do any filtering, i.e., they do not remove any genes from the results or from the plots. The only filtering was that which you did explicitly yourself by:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;keep &amp;lt;- filterByExpr(R10LB_0vs0.5)&#xA;R10LB_0vs0.5_filter &amp;lt;- R10LB_0vs0.5[keep, keep.lib.sizes = FALSE]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For your data, &lt;code&gt;filterByExpr&lt;/code&gt; will keep any gene that is expressed at a minimal level in at least 7 samples.&#xA;You could relax that requirement, to allow genes to be expressed in only 5 or 6 samples say, by reducing the &lt;code&gt;large.n&lt;/code&gt; argument to &lt;code&gt;filterByExpr&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9042" LastEditorUserId="9042" LastEditDate="2021-12-24T04:04:15.170" LastActivityDate="2021-12-24T04:04:15.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18247" PostTypeId="1" AcceptedAnswerId="18264" CreationDate="2021-12-24T22:30:10.293" Score="1" ViewCount="178" Body="&lt;p&gt;Hi I'm new to bioinformatics and have just completed my first run on the MinION (long read sequencing Oxford Nanopore Technologies). I was hoping someone could direct me towards R packages, workflow, tutorials or guides that will help me identify species that are present in my sample mainly for fungi as my amplicons had ITS1 and ITS2 regions.&lt;/p&gt;&#xA;&lt;p&gt;I have been watching Youtube tutorials on RStudio for over a week and I feel like I'm no closer than I was when I started. I have both fastq and fast5 files and I tried to run the fastq files using EPI2ME, however, the database is lacking in fungi. I have tried using NCBI's BLAST but I can only run &amp;lt;5kb reads and most of my sequencing reads are &amp;gt;6kb. Also, I've tried using Galaxy to convert my fastq files to fasta but I'm not sure how to run my long reads for species identification using Galaxy or if that is even possible.&lt;/p&gt;&#xA;&lt;p&gt;Any help would be greatly appreciated. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="14146" LastActivityDate="2021-12-31T03:05:12.750" Title="Help with MinION sequencing data species identification" Tags="|sequence-alignment|sequence-analysis|minion|galaxy|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18248" PostTypeId="1" AcceptedAnswerId="18258" CreationDate="2021-12-24T22:45:17.473" Score="1" ViewCount="92" Body="&lt;p&gt;I'm trying to run a command using snakemake and a TSV config file. Here is my sample config file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sample_id       FQ1_id  FQ2_id&#xA;MyCleanName-1       MyFqName-1_R1.fastq.gz        MyFqName-1_R2.fastq.gz&#xA;MyCleanName-2       MyFqName-2_R1.fastq.gz        MyFqName-2_R2.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here is my snakefile:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;samples = pd.read_table(&amp;quot;samples.tsv&amp;quot;).set_index(&amp;quot;sample_id&amp;quot;, drop = False)&#xA;&#xA;rule all:&#xA;    input:&#xA;        expand(&amp;quot;{sample}_{num}.fastq.gz&amp;quot;, sample = samples.sample_id, num = [1,2])&#xA;&#xA;rule my_job:&#xA;    input:&#xA;        fq1 = expand(&amp;quot;{fq1_id}&amp;quot;, fq1_id = samples.FQ1_id),&#xA;        fq2 = expand(&amp;quot;{fq2_id}&amp;quot;, fq2_id = samples.FQ2_id),&#xA;        logdir = expand(&amp;quot;{sample}.logs&amp;quot;, sample = samples.sample_id)&#xA;    output:&#xA;        &amp;quot;{sample}_1.fastq.gz&amp;quot;,&#xA;        &amp;quot;{sample}_2.fastq.gz&amp;quot;,&#xA;    message:&#xA;        &amp;quot;Running MyCmd on {wildcards.sample}&amp;quot;&#xA;    shell:&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        MyCmd --sample-name {wildcards.sample} -i1 {input.fq1} -i2 {input.fq2}&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;&#xA;rule create_logdir:&#xA;    output:&#xA;        directory(&amp;quot;{sample}.logs&amp;quot;)&#xA;    message:&#xA;        &amp;quot;Creating log directories&amp;quot;&#xA;    shell:&#xA;        &amp;quot;mkdir -pv {wildcards.sample}.logs/{{MyCmd,MyOtherCmd}}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I run &lt;code&gt;snakemake -np MyCleanName-1.fastq.gz&lt;/code&gt;, I get the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Building DAG of jobs...&#xA;Job counts:&#xA;        count   jobs&#xA;        2       create_logdir&#xA;        1       my_job&#xA;        3&#xA;&#xA;[Fri Dec 24 16:38:54 2021]&#xA;Job 1: Creating log directories&#xA;&#xA;mkdir -pv MyCleanName-1.logs/{MyCmd,MyOtherCmd}&#xA;&#xA;[Fri Dec 24 16:38:54 2021]&#xA;Job 2: Creating log directories&#xA;&#xA;mkdir -pv MyCleanName-2.logs/{MyCmd,MyOtherCmd}&#xA;&#xA;[Fri Dec 24 16:38:54 2021]&#xA;Job 0: Running MyCmd on MDA-IBC-3&#xA;&#xA;&#xA;        MyCmd         --sample-name MyCleanName-1 -i1 MyFqName-1_R1.fastq.gz MyFqName-2_R1.fastq.gz -i2 MyFqName-1_R2.fastq.gz MyFqName-2_R2.fastq.gz&#xA;&#xA;Job counts:&#xA;        count   jobs&#xA;        2       create_logdir&#xA;        1       my_job&#xA;        3&#xA;This was a dry-run (flag -n). The order of jobs does not reflect the order of execution.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I restrict this so the run only creates log files for the Clean Name being used and uses only the corresponding FASTQs? I feel like I'm missing something trivial here. Should I not be using the config file variable &lt;code&gt;samples&lt;/code&gt; directly in the rule &lt;code&gt;my_job&lt;/code&gt;, but somehow wiring up the corresponding &lt;code&gt;sample_id&lt;/code&gt; and &lt;code&gt;FQ_id&lt;/code&gt;s on the &lt;code&gt;all&lt;/code&gt; rule?&lt;/p&gt;&#xA;" OwnerUserId="650" LastEditorUserId="650" LastEditDate="2021-12-24T23:10:17.060" LastActivityDate="2021-12-28T20:09:36.070" Title="Pick matching entry from snakemake config table" Tags="|snakemake|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18250" PostTypeId="2" ParentId="18221" CreationDate="2021-12-25T14:40:07.097" Score="2" Body="&lt;h2&gt;IUPAC Reports&lt;/h2&gt;&#xA;&lt;p&gt;The International Union of Pure and Applied Chemistry (IUPAC) produces &lt;a href=&quot;https://iupac.org/what-we-do/nomenclature/&quot; rel=&quot;nofollow noreferrer&quot;&gt;recommendations&lt;/a&gt; on nomenclature for chemical compounds. They produce complete reports, and &lt;a href=&quot;https://iupac.org/what-we-do/nomenclature/brief-guides/&quot; rel=&quot;nofollow noreferrer&quot;&gt;brief guides&lt;/a&gt;. However, creating a systematic name from an arbitrary structure is not simple.&lt;/p&gt;&#xA;&lt;p&gt;It probably depends what 'space' of structures you want to deal with. For simple organic compounds (branched, fused rings, non-bridged) it might be feasible, but there are many edge cases and tricky topics, such as stereochemistry, ligand priority ordering, multiple bridges, and so on.&lt;/p&gt;&#xA;&lt;p&gt;For example, this is the 'Blue Book' : &lt;a href=&quot;https://iupac.qmul.ac.uk/BlueBook/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://iupac.qmul.ac.uk/BlueBook/&lt;/a&gt; which is a set of PDF docs. Converting these to formal rules would be time consuming, to say the least!&lt;/p&gt;&#xA;&lt;h2&gt;Informal Names&lt;/h2&gt;&#xA;&lt;p&gt;A resource like &lt;a href=&quot;http://www.chemspider.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ChemSpider&lt;/a&gt; provides multiple informal names for compounds, with an API to retrieve data. Seems like (from a quick test) you can get the 'CommonName'  field from the details. For example, for a random recordId:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://api.rsc.org/compounds/v1/records/325/details&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://api.rsc.org/compounds/v1/records/325/details&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;you get this return value:&lt;/p&gt;&#xA;&lt;p&gt;{&amp;quot;id&amp;quot;: 325,&amp;quot;commonName&amp;quot;: &amp;quot;2-Methoxy-4-vinylphenol&amp;quot;}&lt;/p&gt;&#xA;&lt;p&gt;so it gives you one common name. Seemingly also for benzene (id 236).&lt;/p&gt;&#xA;" OwnerUserId="745" LastEditorUserId="745" LastEditDate="2021-12-25T14:47:11.300" LastActivityDate="2021-12-25T14:47:11.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18251" PostTypeId="1" AcceptedAnswerId="18252" CreationDate="2021-12-27T08:48:51.837" Score="1" ViewCount="1260" Body="&lt;p&gt;I am analysing RNA-Seq data for a set of genes from different human brain regions using data from the Allen Brain Atlas. I have made a heatmap using &lt;a href=&quot;https://www.graphpad.com/scientific-software/prism/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;GraphPad Prism&lt;/strong&gt;&lt;/a&gt;. In this heatmap, I show the average RPKM value for each gene of interest for each brain region (the average is taken as there are multiple samples for each brain region).&lt;/p&gt;&#xA;&lt;p&gt;I am new to creating heatmaps but I know that you can use different methods to cluster the heatmap so that the data is more easy to interpret. I want to cluster the heatmap so that the brain regions which have the highest level of gene expression overall are grouped together.&lt;/p&gt;&#xA;&lt;p&gt;I have seen online that &lt;code&gt;R&lt;/code&gt; can also be used to create a heatmap, and I have had previous experience with &lt;code&gt;R&lt;/code&gt;. Does anyone have any recommendations on any good tutorials (that are not too difficult) on creating heat-maps in &lt;code&gt;R&lt;/code&gt; as well as clustering the data?&lt;/p&gt;&#xA;" OwnerUserId="2531" LastEditorUserId="2531" LastEditDate="2022-02-09T09:54:39.873" LastActivityDate="2022-02-09T09:54:39.873" Title="Best methods for clustering heatmap based on regions where the genes are most highly expressed?" Tags="|r|heatmap|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18252" PostTypeId="2" ParentId="18251" CreationDate="2021-12-27T11:49:00.303" Score="3" Body="&lt;p&gt;For visualization of expression values it usually makes sense to standardize the data first. The problem with plain expression values is that they differ a lot between genes and as such highly-expressed genes dominate the color scale. As you see in your heatmap, the first and last rows barely show any difference because the color scale is dominated by a single gene CHL1. Therefore the clustering would be due to expression level and not due to differences between columns. For this we standardize the data. Assuming your expression matrix had RPKMs (or similar values) we first log2-transform it and then standardize:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&#xA;mat_log2 &amp;lt;- log2(mat+1)&#xA;&#xA;#/ standardize (Z = Z-scored)&#xA;mat_Z &amp;lt;- t(scale(t(mat_log2)))&#xA;&#xA;#/ cluster rows:&#xA;hclust_row &amp;lt;- hclust(dist(mat_Z), method=&amp;quot;ward.D2&amp;quot;)&#xA;&#xA;#/ cluster columns (optional)&#xA;hclust_col &amp;lt;- hclust(dist(t(mat_Z)), method=&amp;quot;ward.D2&amp;quot;)&#xA;&#xA;#/ plot heatmap:&#xA;library(ComplexHeatmap)&#xA;&#xA;hm &amp;lt;- &#xA;  Heatmap(matrix=mat_Z,&#xA;          cluster_rows=hclust_row, &#xA;          cluster_columns=hclust_col,&#xA;          name=&amp;quot;relative expression&amp;quot;)&#xA;draw(hm)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;See the &lt;a href=&quot;https://jokergoo.github.io/ComplexHeatmap-reference/book/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ComplexHeatmap&lt;/a&gt; documentation for details on the package itself, it is very powerful but also monolithic. This is just an example, any other heatmap package will do fine as well.&lt;/p&gt;&#xA;&lt;p&gt;Important is to standardize data to put emphasis on differences between columns rather than on absolute expression levels.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorUserId="3541" LastEditDate="2021-12-29T12:33:10.420" LastActivityDate="2021-12-29T12:33:10.420" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18253" PostTypeId="1" AcceptedAnswerId="18254" CreationDate="2021-12-27T15:34:11.173" Score="0" ViewCount="317" Body="&lt;p&gt;I am new to low-pass whole genome sequencing and have the basic idea of phasing and imputation. I have .vcf file after calling &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360037225632-HaplotypeCaller&quot; rel=&quot;nofollow noreferrer&quot;&gt;haplotypecaller&lt;/a&gt; tools from GATK. After searching the phasing and imputation tools I've encountered some tools like Beagle, Minimac, Shapeit, Glimpse, Eagle. The problem is that I didn't find enough hands-on material on how to process a .vcf from unphased to phased and then to an imputed one.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone suggests any hands-on tutorial regarding phasing and imputation?&lt;/p&gt;&#xA;" OwnerUserId="10296" LastEditorUserId="3694" LastEditDate="2021-12-29T12:34:15.410" LastActivityDate="2021-12-29T12:34:15.410" Title="Tutorials on phasing and imputing low-coverage sequencing data" Tags="|vcf|imputation|phasing|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18254" PostTypeId="2" ParentId="18253" CreationDate="2021-12-27T16:59:23.767" Score="3" Body="&lt;p&gt;This is all relevant for data 0.5x-1x coverage.&lt;/p&gt;&#xA;&lt;p&gt;Assuming you have genotype likelihoods data, if you want to phase low-coverage data, the most suitable options are GLIMPSE, Beagle4 and QUILT. Of the three, GLIMSPE and QUILT are the most recent and similar in performance. shapeit, eagle and minimac aren't suitable for low-coverage data.&lt;/p&gt;&#xA;&lt;p&gt;I have found &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/33414550/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GLIMPSE&lt;/a&gt; to be fast and memory efficient, but it requires several different steps; breaking the genome into chunks, imputing, reforming the chunks and then sampling from the haplotypes to phase the data. It has a fairly good tutorial &lt;a href=&quot;https://odelaneau.github.io/GLIMPSE/tutorial_b38.html#run_preliminaries&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/s41588-021-00877-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;QUILT&lt;/a&gt; has fewer stages and faster, but is unlikely to make much of a difference unless you are working on very large datasets. It has a tutorial &lt;a href=&quot;https://github.com/rwdavies/QUILT&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Which one you use probably doesn't matter a whole lot, but I would probably suggest using QUILT as it has fewer stages and is slightly more recent.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="3694" LastEditDate="2021-12-27T19:15:06.760" LastActivityDate="2021-12-27T19:15:06.760" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="18255" PostTypeId="1" AcceptedAnswerId="18260" CreationDate="2021-12-28T10:38:48.167" Score="1" ViewCount="395" Body="&lt;p&gt;I have a set of 100 amino acid sequences and I want to perform a BLASTP sesrch against the &lt;code&gt;refseq_protein&lt;/code&gt; database. Accordingly I had set up the standalone version of BLAST (Version 2.11.0+) and downloaded the refseq_protein database from NCBI using the following code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;wget ftp://ftp.ncbi.nlm.nih.gov/refseq/release/complete/*.faa.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The database gets downloaded as 3027 zipped files containing FASTA sequences. I unzipped all these files and concatenated them into a single file &lt;code&gt;refseq_protein.faa&lt;/code&gt; (which is around 95 GB in size). Now when I run the following Python code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.Blast.Applications import NcbimakeblastdbCommandline&#xA;from Bio.Blast.Applications import NcbiblastpCommandline&#xA;&#xA;cline = NcbimakeblastdbCommandline(dbtype = &amp;quot;prot&amp;quot;, input_file = &amp;quot;D:\\refseq_protein.faa&amp;quot;, out &#xA;= &amp;quot;refseq_protein&amp;quot;)&#xA;&#xA;blastp_cline = NcbiblastpCommandline(query = &amp;quot;D:\\DEP_sequences.fasta', db = &#xA;&amp;quot;refseq_protein&amp;quot;, evalue = 0.01, outfmt = &amp;quot;7 sseqid evalue qcovs pident&amp;quot;)&#xA;&#xA;cline()&#xA;&#xA;response = blastp_cline()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;the &lt;code&gt;NcbimakeblastdbCommandline&lt;/code&gt; function keeps creating multiple &lt;code&gt;.phr&lt;/code&gt;, &lt;code&gt;.pin&lt;/code&gt;, &lt;code&gt;.psq&lt;/code&gt; etc files which take up a lot of space (In a demo run it had created ~30GB of these files and was still running). I'm afraid this will exhaust the entire space available on my internal hard drive. I'm wondering if there is a way to estimate the total size of the files which &lt;code&gt;NcbimakeblastdbCommandline&lt;/code&gt; would create. This will help me in deciding whether or not to switch to an external storage to perform the BLASTP search.&lt;/p&gt;&#xA;&lt;p&gt;I am aware of the fact that pre-formatted refseq_protein database exists but I'm not sure what value is to be passed in the &lt;code&gt;db&lt;/code&gt; parameter of the &lt;code&gt;NcbiblastpCommandline&lt;/code&gt; function, because it asks for the name of the database against which the BLASTP search is to be performed. In the approach that I chose, I had the liberty to set the name of the database.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestions on how to solve this issue would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="14724" LastEditorUserId="14724" LastEditDate="2021-12-28T11:54:53.047" LastActivityDate="2021-12-29T06:44:53.267" Title="makeblastdb creating multiple files of unexpectedly large sizes" Tags="|fasta|python|refseq|blastp|makeblastdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18256" PostTypeId="1" AcceptedAnswerId="18262" CreationDate="2021-12-28T17:04:02.573" Score="0" ViewCount="74" Body="&lt;p&gt;What are the major differences between the use cases of a shot-gun genome and an mRNA capture transcriptome? Especially when it comes to downstream analyses such as looking for orthology, and selection analysis.&lt;/p&gt;&#xA;&lt;p&gt;From what I can visualize in my head, I can come up with the following points:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The genome has the structural data, information on chromosomes, and haplotypes while the transcriptome lacks this information.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The genome records the SNPs more directly than transcriptomes. I might be thinking of this wrong but the genomes would have the actual bases and we can then compare the bases between the genomes but for transcriptomes differences in expression would show up as presence/absence variations that have no way of being traced back?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The transcriptomes should be easier to work with when it comes to computational requirements since they tend to be 25 to 50 times smaller than genomes.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I could be completely wrong about the points I listed and could be missing something major. What do you guys think?&lt;/p&gt;&#xA;" OwnerUserId="12107" LastActivityDate="2021-12-29T21:12:42.310" Title="Pros and cons between genomes and transcriptomes?" Tags="|transcriptome|genome-sequencing|orthologues|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18257" PostTypeId="1" CreationDate="2021-12-28T19:48:38.757" Score="0" ViewCount="58" Body="&lt;p&gt;I have read &lt;a href=&quot;http://www.sthda.com/english/wiki/cox-proportional-hazards-model&quot; rel=&quot;nofollow noreferrer&quot;&gt;STDHA&lt;/a&gt; and the documentation for the coxph() function in R. I understand that the cox PH model explores how different covariates simultaneously impact survival. Based on the Lung data example in STDHA, a covariate can seem significantly influential in univariate analysis, but not in multivariate analysis. Following the lung data example, &amp;quot;age&amp;quot; is no longer significant in the multivariate cox model.  Quite literally, this means that age does not impact survival when you account for sex and ph.ecog. However, how does the cox PH model evaluate and decide this? Under the hood, does the cox ph model stratify groups by sex, then evaluate if age is still impactful on survival (and then repeat for ph.ecog)? My intuition says no, because I don't think you can do that with continuous variables (so, in turn, how would the model stratify by age and then test if sex still impacts survival)? I don't need a deeply mathematical explanation, as I'm just a coder!&lt;/p&gt;&#xA;&lt;p&gt;Ultimately, I want to understand if I can run coxph() on the female and male group separately, or if these results would be redundant since we have already added &amp;quot;sex&amp;quot; as a covariate when analyzing all patients.&lt;/p&gt;&#xA;&lt;p&gt;This is a conceptual question, but please let me know if I can provide any code. If it means anything, I am using surv &amp;amp; survminer in R.&lt;/p&gt;&#xA;" OwnerUserId="13949" LastActivityDate="2022-03-03T17:06:46.930" Title="How Does Cox Proportional Hazard Model Describe Interaction between Covariates" Tags="|r|statistics|statsmodels|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18258" PostTypeId="2" ParentId="18248" CreationDate="2021-12-28T20:09:36.070" Score="1" Body="&lt;p&gt;Using pointers provided by @bli, I made a few changes and the Snakefile started working as expected.&lt;/p&gt;&#xA;&lt;p&gt;Instead of using &lt;code&gt;expand&lt;/code&gt; for the rule's input, I switched to a function that accepts the wildcard sample as input and returns a dict with the actual inputs.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; # Could be a one-liner but I wrote it in a verbose&#xA; # manner so it's easier to read&#xA; def get_input_files(in_sample_id):&#xA;     fq1 = samples.loc[str(in_sample_id), 'FQ1_id']&#xA;     fq2 = samples.loc[str(in_sample_id), 'FQ2_id']&#xA;     logdir = &amp;quot;{in_sample}.logs&amp;quot;.format(in_sample=str(in_sample_id))&#xA;     return_dict = { 'fq1' : fq1, 'fq2' : fq2, 'logdir' : logdir }&#xA;     return(return_dict)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the rule &lt;code&gt;my_job&lt;/code&gt; is now changed (just the &lt;code&gt;input&lt;/code&gt; part):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule my_job:&#xA;    input:&#xA;        unpack(get_input_files)&#xA;    output:&#xA;        &amp;quot;{sample}_1.fastq.gz&amp;quot;,&#xA;        &amp;quot;{sample}_2.fastq.gz&amp;quot;,&#xA;...&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Hope this helps someone looking for a similar solution in the future!&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2021-12-28T20:09:36.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18259" PostTypeId="2" ParentId="18247" CreationDate="2021-12-28T23:57:16.183" Score="0" Body="&lt;p&gt;I've found &lt;a href=&quot;http://ccb.jhu.edu/software/centrifuge/manual.shtml#obtaining-centrifuge&quot; rel=&quot;nofollow noreferrer&quot;&gt;Centrifuge&lt;/a&gt; and &lt;a href=&quot;https://github.com/DerrickWood/kraken2/wiki/Manual&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kraken2&lt;/a&gt; to be useful for species classification. There are pre-prepared databases that include Fungi, e.g. see &lt;a href=&quot;https://benlangmead.github.io/aws-indexes/k2&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for the Kraken 2 &amp;quot;PF&amp;quot; database, which is the Standard database plus protozoa &amp;amp; fungi.&lt;/p&gt;&#xA;&lt;p&gt;Can't really provide much more help than that with more information about what you've done.&lt;/p&gt;&#xA;&lt;p&gt;You'll need a computer with around 64 GB of memory in order to be able to run these programs, and it's probably not the best idea to jump right into command line compilations when starting out... but it looks like you're up for a challenge given that your first MinION run has involved jumping into the deep end with a fungi metagenomic project.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2021-12-28T23:57:16.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18260" PostTypeId="2" ParentId="18255" CreationDate="2021-12-29T06:25:57.580" Score="2" Body="&lt;p&gt;Welcome to the Bioinformatics site! Hopefully you find the below helpful.&lt;/p&gt;&#xA;&lt;p&gt;Regarding the creation of a RefSeq protein database locally...as you mentioned, there are pre-made RefSeq BLAST databases available, and looking at the &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/blast/db/&quot; rel=&quot;nofollow noreferrer&quot;&gt;website&lt;/a&gt;, the current DB (as of 12/27/2021) is roughly 62.9GB compressed - and is probably ~2x larger decompressed, so I’d guess that the database you are constructing manually will be at least 125GB in size - and likely more.&lt;/p&gt;&#xA;&lt;p&gt;The above is regarding the technical issue you are currently facing - but I'd back up and look closer at what you are trying to do - align the 100 amino acid sequences to RefSeq. Here are some alternatives that may be -much- easier.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;BLAST your sequences on the &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PROGRAM=blastp&amp;amp;PAGE_TYPE=BlastSearch&amp;amp;LINK_LOC=blasthome&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI website&lt;/a&gt;. This may not be as satisfying as building out your local tools and databases, but may get the job done more efficiently. This of course depends on whether this is a one-off thing, or something you plan to do frequently, in which case a local database may make sense.&lt;/li&gt;&#xA;&lt;li&gt;Download the premade RefSeq BLAST database. You mentioned you were having issues with how to refer to this database (which may be a useful question in and of itself), but generally an alias file exists within the premade database (entitled &lt;code&gt;refseq_protein.pal&lt;/code&gt;), and calling this file will allow you to BLAST all the separate chunks. So, the command line might be something like this:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastp -db refseq_protein -query amino_acid_100.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There are a ton more options you may want to use (try &lt;code&gt;blastp -help&lt;/code&gt;) , but the above should work.&lt;/p&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Use the &lt;code&gt;-remote&lt;/code&gt; option in blastp to run from the command line against the NCBI server. The below line should perform the BLAST alignment against the NCBI refseq protein database.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastp -remote -db refseq_protein -query amino_acid_100.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="92" LastEditorUserId="92" LastEditDate="2021-12-29T06:44:53.267" LastActivityDate="2021-12-29T06:44:53.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18261" PostTypeId="1" AcceptedAnswerId="18266" CreationDate="2021-12-29T14:05:05.773" Score="0" ViewCount="433" Body="&lt;p&gt;I have a problem with my nextflow pipeline.&lt;/p&gt;&#xA;&lt;p&gt;My workflow looks like this:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Uncompress the hisat2 genotpye index grch38_genome.tar.gz from &lt;a href=&quot;https://genome-idx.s3.amazonaws.com/hisat/grch38_genome.tar.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genome-idx.s3.amazonaws.com/hisat/grch38_genome.tar.gz&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Align .fastq.gz files from /https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR1806626 and &lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR1806627&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR1806627&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;However, in process HISAT2 I get command exit status 255:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Command error: &amp;quot;grch38&amp;quot; does not exist&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The command executed is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;hisat2 --very-sensitive --no-spliced-alignment -x grch38 -U&#xA;SRR1806626.fastq.gz &amp;gt; SRR1806626.fastq.sam&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Can someone please explain to me what I'm doing wrong?&lt;/p&gt;&#xA;&lt;p&gt;My script.nf file for reproducing the error looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env nextflow&#xA;&#xA;nextflow.enable.dsl=2&#xA;&#xA;params.reads = &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$baseDir/data/*{1,2}*.fastq.gz&quot; //https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR1806626, https://trace.ncbi.nlm.nih.gov/Traces/sra/?run=SRR1806627&#xA;params.hisat2_index = &quot;$&lt;/span&gt;baseDir/data/*_genome.tar.gz&amp;quot; // https://genome-idx.s3.amazonaws.com/hisat/grch38_genome.tar.gz&#xA;&#xA;log.info &amp;quot;&amp;quot;&amp;quot;\&#xA;         reads:        &lt;span class=&quot;math-container&quot;&gt;${params.reads}&#xA;         hisat2_index: $&lt;/span&gt;{params.hisat2_index}&#xA;         &amp;quot;&amp;quot;&amp;quot;&#xA;         .stripIndent()&#xA;&#xA;process UNCOMPRESS_GENOTYPE_INDEX {&#xA;&#xA;  tag &amp;quot;$hisat2_index_ch.baseName&amp;quot;&#xA;&#xA;  input:&#xA;  path(hisat2_index_ch)&#xA;&#xA;  output:&#xA;  path('*')&#xA;&#xA;  script:&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  tar xvzf $hisat2_index_ch&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;}&#xA;&#xA;process HISAT2 {&#xA;&#xA;  tag &amp;quot;$reads.baseName&amp;quot;&#xA;&#xA;  input:&#xA;  path(hisat2_indices)&#xA;  path(reads)&#xA;&#xA;  output:&#xA;  path('*.sam')&#xA;&#xA;  script:&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  hisat2 --very-sensitive \&#xA;         --no-spliced-alignment \&#xA;         -x &lt;span class=&quot;math-container&quot;&gt;$hisat2_indices \&#xA;         -U $&lt;/span&gt;reads \&#xA;         &amp;gt; ${reads.baseName}.sam&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;}&#xA;&#xA;workflow {&#xA;&#xA;  Channel&#xA;    .fromPath( params.reads )&#xA;    .ifEmpty { exit 1, &amp;quot;reads was empty - no input files supplied&amp;quot; }&#xA;    .set { reads_ch }&#xA;&#xA;  Channel.fromPath( params.hisat2_index )&#xA;    .ifEmpty { exit 1, &amp;quot;hisat2_index was empty - no input file supplied&amp;quot; }&#xA;    .set { hisat2_index_ch }&#xA;&#xA;  UNCOMPRESS_GENOTYPE_INDEX(hisat2_index_ch)&#xA;&#xA;  HISAT2(UNCOMPRESS_GENOTYPE_INDEX.out, reads_ch)&#xA;&#xA;  //Command error:&#xA;  //(ERR): &amp;quot;grch38&amp;quot; does not exist&#xA;  //Exiting now ...&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Kind regards,&#xA;Thomas&lt;/p&gt;&#xA;" OwnerUserId="14784" LastActivityDate="2021-12-30T12:13:06.337" Title="Nextflow HISAT2 command exit status 255 with prebuild index" Tags="|rna-seq|nextflow|rna-alignment|errors|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18262" PostTypeId="2" ParentId="18256" CreationDate="2021-12-29T21:12:42.310" Score="0" Body="&lt;p&gt;It really depends on your use case. If this is human data, where we already have a fairly polished reference genome, then genome sequencing adds value for the genetic variation, e.g. rare disease genetics, GWAS or cancer genomics. Transcriptome sequencing allow exploration of tissue (or cell type) specific patterns and RNA biomarkers etc. Calling variants from RNA can be interesting in the context of splicing and RNA editing.&lt;/p&gt;&#xA;&lt;p&gt;For a non-model organism where there isn't currently a reference genome, then the genome creates a reference for future studies to map genomic/transcriptomic reads to, so it's usual to do the genome first.&lt;/p&gt;&#xA;&lt;p&gt;Since you mention ortholgy and selection studies, well traditional comparative genomics studies have done these analyses from the genome only, but the transcriptome can add value depending on your questions.&lt;/p&gt;&#xA;&lt;p&gt;Finally, I wouldn't consider computational requirements as you stated: it depends on sequencing depth/coverage and other factors, and is likely not to drive the choice between the two assays&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2021-12-29T21:12:42.310" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18263" PostTypeId="1" AcceptedAnswerId="18274" CreationDate="2021-12-29T22:06:59.227" Score="1" ViewCount="534" Body="&lt;p&gt;I want to specifically find the common ancestor between a lobster and a humans. I suspect it was an aquatic worm of some description. But I want to know about the nervous system of this common ancestor.&lt;/p&gt;&#xA;" OwnerUserId="3823" LastActivityDate="2022-01-05T06:13:41.407" Title="What is the most appropriate way to find the most recent common ancestor between two distantly related species" Tags="|sequence-alignment|phylogenetics|genome|phylogeny|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18264" PostTypeId="2" ParentId="18247" CreationDate="2021-12-30T00:05:58.727" Score="1" Body="&lt;p&gt;There are a number of useful metagenomics tools out there, including Centrifuge and Kraken2, as @gringer has mentioned.&lt;/p&gt;&#xA;&lt;p&gt;Also, FWIW, the &lt;a href=&quot;https://epi2me.nanoporetech.com/workflow-docs-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;WIMP tool from ONT&lt;/a&gt;, a Centrifuge based metageonomic classifier, does include fungus sequences derived from RefSeq, though it may not include the species you are looking for.&lt;/p&gt;&#xA;&lt;p&gt;If you want to use NCBI BLAST, you could split your long reads into pieces that are &amp;lt;5kb (there are a number of tools out there for FASTQ manipulation - my current favorite tool for this is &lt;a href=&quot;https://bioinf.shenwei.me/seqkit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqkit&lt;/a&gt;, where you could do something like this):&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;seqkit sliding -s 4900 -W 4900 test1.fastq&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The above will split your FASTQ reads into chunks of 4.9kb. You can adjust the &lt;code&gt;-s&lt;/code&gt; parameter if you instead want overlapping chunks.&lt;/p&gt;&#xA;&lt;p&gt;From a GUI standpoint, you can use Galaxy for the above by making a workflow with the following steps:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Trim Sequences (enter reasonable numbers here, to trim each of your reads to &amp;lt;5kb)&lt;/li&gt;&#xA;&lt;li&gt;FASTQ to FASTA converter&lt;/li&gt;&#xA;&lt;li&gt;NCBI BLAST+&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Another possibility is QIIME2, which can be used with the UNITE fungal training set. See &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7032056/&quot; rel=&quot;nofollow noreferrer&quot;&gt;recent reference&lt;/a&gt; (looking at the ITS1/ITS2 regions), &lt;a href=&quot;https://unite.ut.ee/repository.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;UNITE&lt;/a&gt;, and &lt;a href=&quot;https://docs.qiime2.org/2021.11/&quot; rel=&quot;nofollow noreferrer&quot;&gt;QIIME2&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Long term, you may want to look into using the command line, as it is much more powerful and customizable, but if BLAST gives you reasonable results for your reads, the above process in Galaxy may work for you.&lt;/p&gt;&#xA;" OwnerUserId="92" LastEditorUserId="92" LastEditDate="2021-12-31T03:05:12.750" LastActivityDate="2021-12-31T03:05:12.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18265" PostTypeId="1" CreationDate="2021-12-30T08:18:29.667" Score="0" ViewCount="73" Body="&lt;p&gt;I'm using &lt;a href=&quot;https://alkesgroup.broadinstitute.org/Eagle/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eagle 2.4.1&lt;/a&gt; for phasing a vcf.gz file. Whenever I give the command it gives me the following error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Phasing samples 1-1&#xA;WARNING: Sample 1 (1-indexed) has a het count of 0&#xA;ERROR: Failed to allocate 18446744073709551596 bytes&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The command to reproduce the error is as follows,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;eagle --vcf myprofile.vcf.gz  --geneticMapFile genetic_map_hg38_withX.txt.gz --outPrefix chr2 --chrom 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Link for the vcf file &lt;a href=&quot;https://github.com/shafayetrahat/test/blob/main/NA20298_chr2.vcf.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/shafayetrahat/test/blob/main/NA20298_chr2.vcf.gz&lt;/a&gt;&#xA;Any help will be apreciated.&lt;/p&gt;&#xA;" OwnerUserId="10296" LastEditorUserId="10296" LastEditDate="2021-12-30T17:54:57.720" LastActivityDate="2021-12-30T17:54:57.720" Title="ERROR: Failed to allocate 18446744073709551596 bytes" Tags="|errors|imputation|phasing|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="18266" PostTypeId="2" ParentId="18261" CreationDate="2021-12-30T12:13:06.337" Score="0" Body="&lt;p&gt;The problem is probably that the tar process creates a folder with the genome files but the input for &lt;code&gt;-x&lt;/code&gt; is not the folder with the index files but the basename of the index files itself.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-12-30T12:13:06.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18268" PostTypeId="1" CreationDate="2021-12-30T19:22:26.487" Score="0" ViewCount="1436" Body="&lt;p&gt;The basemean is described as the &amp;quot;mean of normalized counts of all samples&amp;quot;. My question is, how would we interpret differences in basemean between genes, pertaining to reliability and such? For example, if two genes that have similar p-values and l2fchanges, but one has a much higher basemean, would that gene's data be more dependable?&lt;/p&gt;&#xA;&lt;p&gt;Sorry if this is a trivial question!&lt;/p&gt;&#xA;" OwnerUserId="13950" LastActivityDate="2021-12-30T22:05:28.873" Title="Use of &quot;baseMean&quot; in DeSEQ2" Tags="|deseq2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18269" PostTypeId="2" ParentId="18268" CreationDate="2021-12-30T22:05:28.873" Score="2" Body="&lt;p&gt;I could write a long story about this but actually I will just link &lt;a href=&quot;https://hbctraining.github.io/DGE_workshop/lessons/04_DGE_DESeq2_analysis.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this excellent resource from HBCtraining&lt;/a&gt; which dissects the individual steps DESeq2 does, starting from normalization over dispersion estimation (this is where the baseMean comes into play) over model fitting and testing. I think it will clarify the role of the baseMean in DESeq2.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2021-12-30T22:05:28.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18270" PostTypeId="1" AcceptedAnswerId="18271" CreationDate="2022-01-01T11:03:48.593" Score="1" ViewCount="316" Body="&lt;p&gt;I'm following &lt;a href=&quot;https://odelaneau.github.io/GLIMPSE/tutorial_b38.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; tutorial of GLIMPSE for learning. I was expecting some extra SNPS coming from the 1000 genome reference in the resulting .vcf file. Though I understand the phasing in the output resulting .vcf, what I don't understand is the imputed result. Apparently the .vcf before and after has the same number of SNPs. I ran the scripts from &lt;a href=&quot;http://github.com/odelaneau/GLIMPSE/tree/master/tutorial&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I'm including the &lt;a href=&quot;https://drive.google.com/drive/folders/1-Jc6b6wMK1RJVtgnLNam-QCJoTjWrTcL?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;vcf files&lt;/a&gt; before and after imputation.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone explain me where the imputed SNPs are in the output v.cf file. If my concept is wrong can anyone tell me what should we expect in result.&lt;/p&gt;&#xA;" OwnerUserId="10296" LastEditorUserId="3694" LastEditDate="2022-01-31T19:47:38.497" LastActivityDate="2022-01-31T19:47:38.497" Title="Interpreting imputation result from GLIMPSE" Tags="|vcf|imputation|phasing|1000genomes|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18271" PostTypeId="2" ParentId="18270" CreationDate="2022-01-01T11:33:21.030" Score="2" Body="&lt;p&gt;It seems that you didn't use the following flag:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; --impute-reference-only-variants   Allows imputation at variants only present&#xA;                                     in the reference panel. The use of this &#xA;                                     option is intended only to allow &#xA;                                     imputation at sporadic missing variants. &#xA;                                     If the number of missing variants is &#xA;                                     non-sporadic, please re-run the genotype &#xA;                                     likelihood computation at all reference &#xA;                                     variants and avoid using this option, &#xA;                                     since data from the reads should be used. &#xA;                                     A warning is thrown if reference-only &#xA;                                     variants are found.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You were expecting additional SNPs which were present in the reference panel to be imputed into your targets, but you didn't include the flag &lt;code&gt;--impute-reference-only-variants&lt;/code&gt;, so only individual missing genotypes were imputed. Individual missing genotypes &lt;code&gt;./.&lt;/code&gt; are always imputed if they are present in both the target and reference, but if the SNP is present in the reference and not the target, it won't be unless you use that flag.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="3694" LastEditDate="2022-01-01T13:02:42.970" LastActivityDate="2022-01-01T13:02:42.970" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18272" PostTypeId="1" CreationDate="2022-01-01T17:05:14.763" Score="0" ViewCount="48" Body="&lt;p&gt;I have a CRAN package say &lt;code&gt;myPack&lt;/code&gt;, that imports two CRAN packages, say &lt;code&gt;cranA&lt;/code&gt; and &lt;code&gt;cranB&lt;/code&gt;. These two packages import Bioconductor packages, say &lt;code&gt;biocA&lt;/code&gt; and &lt;code&gt;biocB&lt;/code&gt;. When I try installation of my package with &lt;code&gt;install.packages(&amp;quot;myPack&amp;quot;)&lt;/code&gt; this ends up with error. I have to install &lt;code&gt;biocA&lt;/code&gt; and &lt;code&gt;biocB&lt;/code&gt; that are imported from both &lt;code&gt;cranA&lt;/code&gt; and &lt;code&gt;cranB&lt;/code&gt; in order to successfully install &amp;quot;myPack&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have any idea how to do in order to successfully install &lt;code&gt;myPack&lt;/code&gt; without having to install beforehand &lt;code&gt;biocA&lt;/code&gt; and &lt;code&gt;biocB&lt;/code&gt; from Bioconductor?&lt;/p&gt;&#xA;&lt;p&gt;P.S. I have seen answers if you add &lt;code&gt;biocViews&lt;/code&gt; to the &lt;code&gt;DESCRIPTION&lt;/code&gt; file of &lt;code&gt;myPack&lt;/code&gt;, but I think this is not sufficient for successful installation with &lt;code&gt;install.packages()&lt;/code&gt;. The installation is successful if in addition you set the repositories with &lt;code&gt;setRepositories()&lt;/code&gt; and you choose &amp;quot;CRAN&amp;quot; and &amp;quot;BioC software&amp;quot; and then you use &lt;code&gt;install.packages()&lt;/code&gt; where the package inside &lt;code&gt;install.packages()&lt;/code&gt; contains  &lt;code&gt;biocViews&lt;/code&gt; to the &lt;code&gt;DESCRIPTION&lt;/code&gt; file. So it seems that &lt;code&gt;biocViews&lt;/code&gt; to the &lt;code&gt;DESCRIPTION&lt;/code&gt; file does not automatically connect to Bioconductor repository.&lt;/p&gt;&#xA;" OwnerUserId="14789" LastEditorUserId="650" LastEditDate="2022-01-03T21:15:32.517" LastActivityDate="2022-01-03T21:15:32.517" Title="install a CRAN package that imports a CRAN package that imports a Bioconductor package" Tags="|r|bioconductor|software-installation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18273" PostTypeId="1" CreationDate="2022-01-01T22:14:04.773" Score="2" ViewCount="135" Body="&lt;p&gt;I'd like to try a contact prediction method (like TripletRes, CCMpred etc) using a custom set of sequences.  There are about 1.5M sequences, all very similar to the canonical sequence; their distribution reflects the occurence of mutations &amp;quot;in the wild&amp;quot; (they're not clustered or anything like that).  How should I make a MSA in this case?&lt;/p&gt;&#xA;&lt;p&gt;For example in the original &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6851483/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;TripletRes paper&lt;/strong&gt;&lt;/a&gt; it looks like the MSA comes from DeepMSA - this searches UniClust30 and UniRef90 using &lt;a href=&quot;https://www.openstructure.org/docs/2.2/bindings/hhblits/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;hhblits&lt;/strong&gt;&lt;/a&gt; and &lt;a href=&quot;http://hmmer.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;jackhmmer&lt;/strong&gt;&lt;/a&gt;.  I don't see any way to use DeepMSA to make a MSA from a fully custom set of sequences, and also it's not clear how to combine a custom set with the UniClust/UniRef search results.  I can make an alignment of just my data with jackhmmer, but I don't know if that's what the other tools expect.  For one thing, the alignment is pretty huge compared to the typical inputs (~million vs typical DeepMSA outputs of a few hundred); for another, it is a lot less variable (most sequences differ in just 1-2 aa, vs completely different species).&lt;/p&gt;&#xA;" OwnerUserId="14472" LastEditorUserId="9476" LastEditDate="2022-01-02T06:13:40.340" LastActivityDate="2022-01-07T04:19:39.203" Title="How to make MSA for contact prediction?" Tags="|protein-structure|multiple-sequence-alignment|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18274" PostTypeId="2" ParentId="18263" CreationDate="2022-01-01T23:58:34.473" Score="4" Body="&lt;p&gt;Because I've now posted several comments, I'll just roll them all up.&lt;/p&gt;&#xA;&lt;p&gt;For background on the approaches used to identify most recent common ancestors and a high-level look at how animal taxonomy has been inferred, I suggest &lt;a href=&quot;https://onlinelibrary.wiley.com/doi/epdf/10.1111/j.1558-5646.1999.tb03768.x&quot; rel=&quot;noreferrer&quot;&gt;Lynch 1999&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I think that there are 2 interpretations of this question. If you are interested in &lt;strong&gt;just looking up a single MRCA of well-defined clades&lt;/strong&gt;, such as lobster and human, here are some approaches:&lt;/p&gt;&#xA;&lt;h2&gt;Easy way:&lt;/h2&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Look at a tree diagram, e.g. this:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iqoml.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iqoml.png&quot; alt=&quot;Animal phylogeny diagram&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Find the tips that correspond to your species of interest (arthropods for lobster, chordata for humans).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Find where they join together in the diagram (the branch labeled &amp;quot;true coelom&amp;quot;).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;You have your answer, the MRCA is the group of organisms with a true coelom,  coelomates.&lt;/p&gt;&#xA;&lt;h2&gt;A more involved way using a database&lt;/h2&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Go to &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?mode=Undef&amp;amp;id=33208&amp;amp;lvl=3&amp;amp;lin=f&amp;amp;keep=1&amp;amp;srchmode=1&amp;amp;unlock&quot; rel=&quot;noreferrer&quot;&gt;this website&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Find the group of species 1 (arthropods, protostomes, etc. for lobster, chordata, deuterostomes etc. for human)&lt;/li&gt;&#xA;&lt;li&gt;navigate around until you see the group containing the two groups (in this case listed as &amp;quot;bilateria&amp;quot;). In this case you are looking for the bilaterian common ancestor.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h2&gt;another database&lt;/h2&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Go to &lt;a href=&quot;http://lifemap-ncbi.univ-lyon1.fr/&quot; rel=&quot;noreferrer&quot;&gt;this website&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Point and click your way to a view where you see your 2 clades of interest (arthropods, chordates in this case). See figure.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/0y23C.jpg&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0y23C.jpg&quot; alt=&quot;A view of a phylogenetic website&quot; /&gt;&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Find where they join (in this case, it is less certain about the existence of a coelomate common ancestor, so it just says &amp;quot;&lt;strong&gt;bilaterians&lt;/strong&gt;&amp;quot;).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h2&gt;In R&lt;/h2&gt;&#xA;&lt;p&gt;We might also want a programmatic procedure in case we need to automate it. The following gives a series of commands in R that do this.&lt;/p&gt;&#xA;&lt;p&gt;Get a newick/nexus tree from somewhere e.g. &lt;a href=&quot;https://itol.embl.de/tree/129215302173073111930481660&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; of animals (go &amp;quot;control panel --&amp;gt; export, format = &amp;quot;newick tree&amp;quot; --&amp;gt; download).&lt;/p&gt;&#xA;&lt;p&gt;For this tree, we can approximate an answer as following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(ape)&#xA;it = read.tree(&amp;quot;~/Downloads/zUtVMYn3wJQBoP7E8fygTw_newick.txt&amp;quot;)  # autogenerated filename from the website.&#xA;getMRCA(it, c(&amp;quot;Homo_sapiens&amp;quot;, &amp;quot;Drosophila_melanogaster&amp;quot;))  # drosophila is one of only 2 arthropods in this tree, but arthropods all have an arthropod common ancestor&#xA;# prints &amp;quot;200&amp;quot;, the node index of the MRCA.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can use that node index to do things like look up a name in the &lt;code&gt;node.label&lt;/code&gt; attribute of the tree (&lt;code&gt;it&lt;/code&gt;) structure, or to do other operations on the tree.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2022-01-02T04:33:05.707" LastActivityDate="2022-01-02T04:33:05.707" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18275" PostTypeId="1" AcceptedAnswerId="18277" CreationDate="2022-01-03T12:13:00.297" Score="1" ViewCount="649" Body="&lt;p&gt;I am making a heatmap using RNA-Seq data in R. The heatmap shows gene expression values (RPKM) in different brain regions. I have the following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse)&#xA;&#xA;library(pheatmap)&#xA;&#xA;library(matrixStats)&#xA;&#xA;read_csv(&amp;quot;prenatal_heatmap_data.csv&amp;quot;) -&amp;gt; all_data&#xA;&#xA;all_data %&amp;gt;%&#xA;&#xA;   column_to_rownames(&amp;quot;Brain Region&amp;quot;) -&amp;gt; heatmap_data&#xA;&#xA;heatmap_data %&amp;gt;%&#xA;&#xA;   pheatmap()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which generates the following heatmap:&lt;/p&gt;&#xA;&lt;p&gt;I want to do a log2 transformation of the RPKM values so that I can do the Z-score standardisation of the data. I have log2-transformed the values using the following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;heatmap_data %&amp;gt;%&#xA;&#xA;   log2() -&amp;gt; heatmap_data_log2&#xA;&#xA;heatmap_data_log2 %&amp;gt;%&#xA;&#xA;   pheatmap()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, when I try to create a heatmap using the log2-transformed values I get the following error:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ixnfd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ixnfd.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have looked at the log2-transformed data and I know that the reason I am getting this error is because in the original dataset, some of the RPKM values were 0. When these 0 values get log-transformed they become -Inf:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hT7l7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hT7l7.png&quot; alt=&quot;Original Data&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;^ Original Data&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MJ0B7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MJ0B7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;^ log2 transformed data&lt;/p&gt;&#xA;&lt;p&gt;I am not sure how to overcome this issue. I was wondering if it is necessary to do the log2 transformation of the RPKM values before doing the Z-score standardisation? I have seen that it is conventional to do the log2-transformation of the data.&lt;/p&gt;&#xA;&lt;p&gt;Any advice is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2531" LastEditorUserId="2531" LastEditDate="2022-02-09T09:55:33.190" LastActivityDate="2022-02-09T09:55:33.190" Title="RNA-Seq Data Heatmap: Is it necessary to do a log2 transformation of RPKM values before doing the Z-score standardisation?" Tags="|r|heatmap|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18276" PostTypeId="1" CreationDate="2022-01-03T13:12:42.203" Score="3" ViewCount="261" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9504085/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am using Rcppml package in R for my NMF analysis, I have a matrix from single-cell analysis. I have cells scores and I wanted to know how to extract genes instead of the components.&lt;/p&gt;&#xA;&lt;p&gt;i used the following codes.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;t1.cpm&amp;lt;- LogNormalize(data = as.matrix(t1@assays$RNA@counts)) # Log converting the data form seurat obj&#xA;&#xA;t1.model &amp;lt;- RcppML::nmf(as.matrix(t1.cpm), 30, verbose = F, seed = 1234)&#xA;&#xA;t1.w &amp;lt;- t1.model&lt;span class=&quot;math-container&quot;&gt;$w&#xA;t1.d &amp;lt;- t1.model$&lt;/span&gt;d&#xA;t1.h &amp;lt;- t1.model$h&#xA;&#xA;dim(t1.w)&#xA;rownames(t1.w)&amp;lt;- rownames(t1.cpm)&#xA;colnames(t1.w)&amp;lt;- paste0(&amp;quot;component&amp;quot;, 1:30)&#xA;rownames(t1.h)&amp;lt;- paste0(&amp;quot;component&amp;quot;, 1:30)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and my output is as follows&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;        T_1_TACCTATCAGATGGCA T_1_CCTATTATCAGGCAAG&#xA;component1             0.000000000            0.0000000000&#xA;component2             0.000000000            0.0023590277&#xA;component3             0.001512931            0.0000000000&#xA;component4             0.000000000            0.0021574559&#xA;component5             0.001175799            0.0020222144&#xA;component6             0.000000000            0.0004463346&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It would be helpful if you can let me know how to extract Gene names instead of components.&lt;/p&gt;&#xA;&lt;p&gt;Thanks,&#xA;Dave.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Rcppml is a machine learning library.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/zdebruine/RcppML&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/zdebruine/RcppML&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;NMF is Non-negative matrix factorization which is used in bioinformatics and AI, which can examine the latent relationships in experimental data sets.&lt;/p&gt;&#xA;" OwnerUserId="9817" LastEditorUserId="73" LastEditDate="2023-04-13T03:35:17.683" LastActivityDate="2023-04-13T03:35:17.683" Title="Gene names from components in NMF analysis" Tags="|scrnaseq|machine-learning|nmf|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18277" PostTypeId="2" ParentId="18275" CreationDate="2022-01-03T13:28:54.097" Score="1" Body="&lt;p&gt;Add a pseudocount of one as in the code suggestion I gave you earlier: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/18251/best-methods-for-clustering-heatmap-based-on-regions-where-the-genes-are-most-hi&quot;&gt;Best methods for clustering heatmap based on regions where the genes are most highly expressed?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Log of zero is not defined.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2022-01-03T13:28:54.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18280" PostTypeId="2" ParentId="18273" CreationDate="2022-01-04T19:25:03.650" Score="1" Body="&lt;p&gt;That's a LOT of sequences! I'm doubtful you'd be able to get TripletRes or CCMPred to run at all with an MSA depth of 1.5 million. I suggest pruning your dataset to remove redundant sequences (sequences which don't add much information, i.e. the ones which differ only 1 or 2 AA). I'm curious if your Neff (normalized number of effective sequences--as defined in the ResTriplet paper you link) is anywhere close to your number of sequences.&lt;/p&gt;&#xA;&lt;p&gt;Also, I think you've misunderstood the use of DeepMSA. DeepMSA's main use is to find sequences to include in an MSA. I believe it uses much simpler programs to construct the MSA itself. So you'd use DeepMSA if you wanted to do 1 query sequence --&amp;gt; an MSA of several hundred sequences. Since you already have the (very complete) set of sequences, it wouldn't make sense to use DeepMSA. You could instead just use something like &lt;a href=&quot;http://www.clustal.org/omega/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clustal Omega&lt;/a&gt; to generate the MSA. There's also a pretty easy-to-use &lt;a href=&quot;https://www.ebi.ac.uk/Tools/msa/clustalo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;web server&lt;/a&gt; but it's limited to 4k sequences.&lt;/p&gt;&#xA;" OwnerUserId="14806" LastActivityDate="2022-01-04T19:25:03.650" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="18281" PostTypeId="2" ParentId="18263" CreationDate="2022-01-05T06:13:41.407" Score="3" Body="&lt;p&gt;Good answer.&lt;/p&gt;&#xA;&lt;p&gt;The common ancestor is not known due to the paucity of the fossil record at this point in evolutionary time. A reconstruction can be made using coalescent theory which is what @MaximillanPress describes. The level of uncertainty at that depth of calculation requires significant expertise. I would re-evaluate the comparison with related organisms that are heavily studied.&lt;/p&gt;&#xA;&lt;p&gt;Lobsters are decapods (Decapoda) and closely related to insects, moreover they are monophyletic with the Order Insecta. A lot is known about a lot of insects, many being model organisms, such as Drosophila melanogaster. Both decapods and insects exist within the arthropods (Arthropoda), along with spiders and the like. Their ancestors were marine arthropods, trilobites are arthropods. The major diversity of the arthropods was generated in the Cambrian Explosion and a lot of work has gone into identifying these fossils.&lt;/p&gt;&#xA;&lt;p&gt;Traversing between decapods and insects is as far as I'd go, which is likely the late stages of the Cambrian explosion. There are likely to be putative ancestral fossils - not that you can get any molecular sequence from them however. The closest living ancestor on the insect side is Colembola (beach hoppers).&lt;/p&gt;&#xA;&lt;p&gt;There are serious difficulties in major molecular calculations between decapods and mammals.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-01-05T06:13:41.407" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18282" PostTypeId="1" CreationDate="2022-01-05T15:00:16.800" Score="1" ViewCount="98" Body="&lt;p&gt;I am trimming the aligned peptide sequences and back translating them to codon using Trimal.&lt;/p&gt;&#xA;&lt;p&gt;Here i am using following cmd's:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;trimal -in  OG0008206.fa.aln.out -out OG0008206.fa.aln.out_1st.trim -automated1 -backtrans OG0008206.fa.codon&#xA;trimal -out OG0008206.fa.aln.out_2nd.trim -phylip_paml -nogaps&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But i am confused about, should i remove the gaps fisrt and then backtransleted the peptides or Backtransleted first and then remove the gaps. Because i am getting different  results (codon) in two attempts. which one i should do fisrt?&#xA;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="12579" LastEditorUserId="3848" LastEditDate="2023-07-02T16:08:34.287" LastActivityDate="2023-07-02T16:08:34.287" Title="Steps for trimming aligned peptides using Trimal" Tags="|trimming|codon|" AnswerCount="0" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18285" PostTypeId="1" CreationDate="2022-01-05T18:04:14.017" Score="2" ViewCount="39" Body="&lt;p&gt;I have run into the following problem&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error: failed to find 'gcloud' binary; set option or environment variable 'GCLOUD_SDK_PATH'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;when executing this line of R code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;avfiles_ls( recursive = FALSE,&#xA;            namespace = avworkspace_namespace('bdcat-cohort3-stanislawski'),&#xA;            name = avworkspace_name('SOLGeneticTopMed')&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This works when I run it in a notebook in a cloud environment but not when I am running an R script inside a wdl on Terra.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to more fully quality where the namespace is?&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&#xA;Liz&lt;/p&gt;&#xA;" OwnerUserId="14810" LastEditorUserId="650" LastEditDate="2022-01-05T19:53:55.543" LastActivityDate="2022-01-08T12:48:06.873" Title="'GCLOUD_SDK_PATH' Error" Tags="|python|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18286" PostTypeId="1" AcceptedAnswerId="18288" CreationDate="2022-01-05T19:22:03.033" Score="2" ViewCount="275" Body="&lt;p&gt;I would like to implement on a web page a tool to visualize 3D structures like it is for AlphaFold and RCSB.&#xA;A couple of examples:&#xA;&lt;a href=&quot;https://www.rcsb.org/3d-view/7KX9&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/3d-view/7KX9&lt;/a&gt;&#xA;&lt;a href=&quot;https://alphafold.com/entry/Q5VSL9&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://alphafold.com/entry/Q5VSL9&lt;/a&gt;&#xA;As you can see, if you hover over a part of the structure or on a residue on the RNA/amino-acid sequence, that residue is underlined and different details pop up on the screen.&lt;/p&gt;&#xA;&lt;p&gt;What is the javascript library that they use?&#xA;Do you know also where to find the documentation?&#xA;Best regards!&lt;/p&gt;&#xA;" OwnerUserId="9142" LastActivityDate="2022-01-06T14:01:14.670" Title="What tool RCSB and AlphaFold use to visualize 3D structures?" Tags="|protein-structure|3d-structure|rna-structure|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18288" PostTypeId="2" ParentId="18286" CreationDate="2022-01-06T14:01:14.670" Score="5" Body="&lt;p&gt;Different criteria give different rankings.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;Mol*&lt;/code&gt; (read molstar with trilled rhotic R according to the given IPA) is the newest, is used by the RCSB PDB and can support huge complexes. It is less implemented and has a tricker documentation.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;NGL&lt;/code&gt; is the former viewer from RCSB PDB. It is good and the switch to Mol∗ was driven by an effort for uniformity with the PDBe. Both are written by Alex Rose. There is more documentation about NGL —in the form of a gallery of codepens and various GitHub repos using it. There is a package for Python notebooks called nglview. It has issues such as no SS predictions (i.e. requires sheets and helix entries to be defined in the file for proper rendering).&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;3DMol&lt;/code&gt; is simpler and is of note as the Python module nglview does not work in colab notebooks, but the Python module py3DMol does.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2022-01-06T14:01:14.670" CommentCount="4" CommunityOwnedDate="2022-01-13T11:41:52.527" ContentLicense="CC BY-SA 4.0" />
  <row Id="18289" PostTypeId="2" ParentId="774" CreationDate="2022-01-06T17:00:55.717" Score="0" Body="&lt;p&gt;I understand that this post is rather old but there has been some development in the field and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/labs/pmc/articles/PMC6178912.2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this 2019 paper&lt;/a&gt; might be a good resource for interested individuals.&lt;/p&gt;&#xA;" OwnerUserId="7307" LastActivityDate="2022-01-06T17:00:55.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18290" PostTypeId="1" AcceptedAnswerId="18311" CreationDate="2022-01-06T20:11:37.410" Score="0" ViewCount="149" Body="&lt;p&gt;Does anyone know a good, free-to-use VCF file inspector app (Mac OS, with GUI) where I can browse the content of a VCF file &lt;a href=&quot;https://mybiosoftware.com/bamseek-20110724-browse-large-bam-sam-alignment-files.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;similarly to bamseek&lt;/a&gt;*?&lt;/p&gt;&#xA;&lt;p&gt;I would like to browse SNP annotation split by columns, possibly even filter it.&lt;/p&gt;&#xA;&lt;p&gt;'* Bamseek does not work for me, fails at open file. It anyways seems to be an abandonware.&lt;/p&gt;&#xA;&lt;p&gt;Example table view from an expensive commercial software:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/yzAwe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yzAwe.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1365" LastEditorUserId="3694" LastEditDate="2022-01-07T04:07:53.350" LastActivityDate="2022-01-11T16:28:58.827" Title="VCF file inspector app with GUI" Tags="|vcf|snp|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18291" PostTypeId="2" ParentId="18290" CreationDate="2022-01-06T20:32:52.860" Score="2" Body="&lt;p&gt;OK it's a command line program, not a GUI, but &lt;code&gt;bcftools query&lt;/code&gt; and &lt;code&gt;bcftools view&lt;/code&gt; can easily be used to to view elements of a vcf/bcf file and are (probably) a lot more flexible and scalable than a GUI. Examples from &lt;a href=&quot;https://samtools.github.io/bcftools/howtos/query.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools query -f '%CHROM %POS %REF %ALT\n' file.bcf | head -3&#xA;1 13380 C G&#xA;1 16071 G A&#xA;1 16141 C T&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools query -f '%CHROM %POS  GTs:[ %GT]\t PLs:[ %PL]\n' file.bcf | head -3&#xA;1 10234  GTs: 1/1 1/1    PLs: 28,3,0 29,3,0&#xA;1 10291  GTs: ./. 1/1    PLs: 0,0,0 28,3,0&#xA;1 14907  GTs: 0/1 0/1    PLs: 8,0,17 26,0,48&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2022-01-06T20:32:52.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18292" PostTypeId="2" ParentId="18285" CreationDate="2022-01-07T04:01:19.527" Score="2" Body="&lt;p&gt;I assume that you are running Terra on Google Cloud Platform (GCP)?&lt;/p&gt;&#xA;&lt;p&gt;It is weird, because Terra is supposed to run on GCP - that is what its intended purpose - and finding gcloud is automatic, you don't need to configure it.&lt;/p&gt;&#xA;&lt;p&gt;Three suggestions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If the cloud is GCP it is appears to could be a bug with the Broad Institutes platform and should be reported directly on their website. They will know immediately, so if they don't respond immediately its a user problem. This is because they will prioritise system bugs, but user errors they may not respond to.&lt;/li&gt;&#xA;&lt;li&gt;At a guess the error suggests you just need to point to gcloud within .bashrc or .profile, but you don't need to do this on GCP because gcloud is its modus operandi*.&lt;/li&gt;&#xA;&lt;li&gt;Run WDL directly outside Terra.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;em&gt;The simplest approach is to install miniconda and use it to install gcloud-sdk package and then install Terra from within conda. Assuming Terra is present on &amp;quot;-c bioconda&amp;quot;. At this point Terra is forced to see gcloud&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, it feels like Terra is being installed on a cloud outside GCP and hence the error.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;To those who don't know GCP gcloud = google cloud.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-01-08T12:48:06.873" LastActivityDate="2022-01-08T12:48:06.873" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18293" PostTypeId="2" ParentId="18273" CreationDate="2022-01-07T04:19:39.203" Score="1" Body="&lt;p&gt;You might have some luck with &lt;a href=&quot;https://github.com/marbl/parsnp&quot; rel=&quot;nofollow noreferrer&quot;&gt;Parsnp&lt;/a&gt;. From the description:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Parsnp was designed to align the core genome of hundreds to thousands of bacterial genomes within a few minutes to few hours. Input can be both draft assemblies and finished genomes, and output includes variant (SNP) calls, core genome phylogeny and multi-alignments. Parsnp leverages contextual information provided by multi-alignments surrounding SNP sites for filtration/cleaning, in addition to existing tools for recombination detection/filtration and phylogenic reconstruction.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The tldr is that Parsnp is meant for aligning large sets of very closely related microbial sequences. For example, others in my lab have used parsnp to align over a million SARS-CoV-2 genomes. The only potential concern is that the Parsnp workflow also finishes with a non-optional phylogeny estimation step that takes quite long on such large datasets. Someone should really make it so that you can opt out of that step...&lt;/p&gt;&#xA;&lt;p&gt;Spoiler/disclaimer, as the current maintainer of Parsnp, that someone is probably me :grimace:&lt;/p&gt;&#xA;&lt;p&gt;The output of Parsnp is in &lt;a href=&quot;http://darlinglab.org/mauve/user-guide/files.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;XMFA&lt;/a&gt;. You're probably looking for a fasta alignment, so you can use &lt;a href=&quot;https://github.com/marbl/harvest-tools&quot; rel=&quot;nofollow noreferrer&quot;&gt;HarvestTools&lt;/a&gt; to convert them:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;harvesttools -x sars_align/parsnp.xmfa -M sars_align/parsnp.maf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Both parsnp and harvest-tools are available on bioconda, and I'd highly recommend installing them that way.&lt;/p&gt;&#xA;" OwnerUserId="6628" LastActivityDate="2022-01-07T04:19:39.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18294" PostTypeId="1" CreationDate="2022-01-07T11:06:00.990" Score="2" ViewCount="27" Body="&lt;p&gt;I'd like to implement a graph-based algorithm for DNA comparison, more especially by solving a problem that can be formulated as a Maximum Weighted Independent Set problem.&lt;/p&gt;&#xA;&lt;p&gt;I have found the article in reference [1], especially part 2.2.2 &amp;quot;The Independent Subset of Rectangles (IR) Problem&amp;quot;, but it is not very explicit on the way to build the graph G given 2 DNA sequence&lt;/p&gt;&#xA;&lt;p&gt;So I'm looking for any indication, article, or description on the way to construct the conflict graph cited in article reference [1] and [2].&lt;/p&gt;&#xA;&lt;p&gt;I'm not from the biology world but the computer world.&lt;/p&gt;&#xA;&lt;p&gt;Thank you very much for your help.&lt;/p&gt;&#xA;&lt;p&gt;[1] Efficient combinatorial algorithms&#xA;for DNA sequence processing: &lt;a href=&quot;https://www.cs.uic.edu/%7Edasgupta/resume/publ/papers/book-chapter-dasgupta-kao.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cs.uic.edu/~dasgupta/resume/publ/papers/book-chapter-dasgupta-kao.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;[2] Nonoverlapping local alignments (weighted independent&#xA;sets of axis-parallel rectangles) : &lt;a href=&quot;https://core.ac.uk/download/pdf/82057963.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://core.ac.uk/download/pdf/82057963.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14816" LastActivityDate="2023-02-01T17:05:25.640" Title="Independent Subset of Rectangles Problem for Nonoverlapping local alignments" Tags="|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18296" PostTypeId="2" ParentId="17558" CreationDate="2022-01-07T14:48:36.670" Score="0" Body="&lt;p&gt;Did you tried the &lt;code&gt;subset&lt;/code&gt; function?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# assuming that scData@meta.data looks like this&#xA;#         orig.ident  ...  cell_ann&#xA;# ATTCAA  sample1          Cancer_human&#xA;# ATTTAT  sample1          Cancer_human&#xA;# ...&#xA;# AAATCA  sampleX          Cancer_mouse&#xA;# ATTTAA  sampleX          Cancer_mouse&#xA;&#xA;Idents(scData) &amp;lt;- &amp;quot;cell_ann&amp;quot;&#xA;human_subset &amp;lt;- subset(scData, idents = &amp;quot;Cancer_human&amp;quot;)&#xA;mouse_subset &amp;lt;- subset(scData, idents = &amp;quot;Cancer_mouse&amp;quot;) &#xA;mouse_subset &amp;lt;- subset(scData, idents = &amp;quot;Cancer_human&amp;quot;, invert = TRUE) # alternative way &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;P.S. It is not clear what you are trying to do. You should add to your post how your data looks like.&lt;/p&gt;&#xA;" OwnerUserId="3631" LastEditorUserId="3631" LastEditDate="2022-01-07T15:15:22.237" LastActivityDate="2022-01-07T15:15:22.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18298" PostTypeId="2" ParentId="18294" CreationDate="2022-01-07T15:47:16.347" Score="1" Body="&lt;p&gt;Consider the below image from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6450745/&quot; rel=&quot;nofollow noreferrer&quot;&gt;a recent review on whole genome alignment&lt;/a&gt;:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/XroLC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XroLC.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The lines in the figure are local alignments (sometimes referred to as &lt;em&gt;anchors&lt;/em&gt; in literature). These lines can be maintained a number of ways, but often via pairwise genome alignment methods like LastZ, MUMmer, and Last (there are many many more).&lt;/p&gt;&#xA;&lt;p&gt;Often times, one wants to pull out the &amp;quot;best set&amp;quot; of local alignments. In some cases, algorithms will use the &amp;quot;single copy&amp;quot; heuristic, which constrains that each nucleotide in the query can be aligned to at most one nucleotide from the reference. In this case, you can create a bounding rectangle along each of the lines, and two rectangles are independent if there is no point within either rectangle that is on the same &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; coordinate as any point in the other rectangle. You can use this to set your constraint graph.&lt;/p&gt;&#xA;&lt;p&gt;However, it is worth noting that two rectangles may overlap just a tiny bit. In this case, it would be ideal to chop the overlapping part off of either rectangle, that way you can select both and not just one. This requires a bit more methodology, but a crude way would be to just look for such cases and for each one, chop the overlap off of the less scoring rectangle.&lt;/p&gt;&#xA;" OwnerUserId="6628" LastActivityDate="2022-01-07T15:47:16.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18299" PostTypeId="1" CreationDate="2022-01-07T15:20:51.020" Score="7" ViewCount="566" Body="&lt;p&gt;I have a project to write in Python that requires to write a function that given  degenerate DNA sequence (for example: KKGTACACCAG) sequence and a molecular weight interval, returns a list of all unambiguous sequences represented by the given sequence (encoding based on IUPAC alphabet: &lt;em&gt;&lt;strong&gt;&lt;a href=&quot;http://www.bioinformatics.org/sms/iupac.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.bioinformatics.org/sms/iupac.html&lt;/a&gt;&lt;/strong&gt;&lt;/em&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Is there any function in Biopython to convert a DNA sequence from degenerate to unambiguous base pairs?&lt;/p&gt;&#xA;&lt;p&gt;Also, Is it possible to use the molecular weight function from Bio.SeqUtils library for an ambiguous DNA?&lt;/p&gt;&#xA;" OwnerUserId="14830" OwnerDisplayName="Ke Keiss" LastEditorUserId="73" LastEditDate="2022-02-18T23:13:59.800" LastActivityDate="2023-08-16T16:26:04.057" Title="Is there any function in Biopython to convert a DNA sequence from ambiguous to unambiguous?" Tags="|python|homework|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18300" PostTypeId="1" CreationDate="2022-01-07T16:42:17.823" Score="0" ViewCount="12" Body="&lt;p&gt;I'm sorry for having such a juvenile question, but I'm still new in bioinformatics, and I was wondering whether there is any tool able to perform the task I'm dealing with now.&#xA;Here is a short description (please let me know if more information should be provided from my side):&lt;/p&gt;&#xA;&lt;p&gt;I started with 3 genes of &lt;em&gt;Arabidopsis thaliana&lt;/em&gt; (let me call them geneA, geneB, geneC). All of them are well annotated and they're known as transcription factors, but each is coming from different signaling pathway.&lt;/p&gt;&#xA;&lt;p&gt;My aim is to do this:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;To find all the signaling pathways related to each of these 3 genes (this is to identify which genes are the most upstream ones in each pathway*).&lt;/li&gt;&#xA;&lt;li&gt;Afterwards, let's say from all the found signaling pathways of the geneA, I only want to pick those that are sharing genes with the signaling pathways of the geneB and with the signaling pathways of the geneC.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;At the end, I would like to get a graphical visualization** of whether and how these signaling pathways are interconnected and what are the particular genes that are shared by those pathways.&lt;/p&gt;&#xA;&lt;p&gt;*) I cannot use the very popular tool called &amp;quot;String&amp;quot; from string-db.org because it &lt;strong&gt;doesn't show the sequence/order of the signaling&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;**) I know that KEGG is a database full of graphs of different metabolic patways, but in case of interconnections between &lt;strong&gt;signaling pathways in plants&lt;/strong&gt;, it is either very poor tool, or I'm not that much able to find such a option there to do the job for me (even my colleagues didn't manage finding it).&lt;/p&gt;&#xA;&lt;p&gt;All of this would be probably more of a literature-mining type of work, but I'm quite sure that, at the end, my task will be to process not only 3 genes, but maybe even more than 10-20 genes. Thus, I would appreciate any more convenient way rather than just digging the info about each gene from the literature and drawing my graphs manually by hand. Automation in this case matters!&lt;/p&gt;&#xA;&lt;p&gt;Thanks for any help!&lt;/p&gt;&#xA;" OwnerUserId="10053" LastActivityDate="2022-01-07T16:42:17.823" Title="Plant signaling pathway interconnections" Tags="|interactions|graphs|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18302" PostTypeId="1" CreationDate="2022-01-08T22:54:45.317" Score="2" ViewCount="3702" Body="&lt;p&gt;I've analyzed my scRNA-seq data and have a couple of Seurat clusters that show more than one cell type in each cluster. (for example, cluster 9 shows both NK and CD4 cells)&#xA;How can I split a cluster into subclusters, based on the gene expression?&lt;/p&gt;&#xA;" OwnerUserId="14822" LastActivityDate="2023-10-10T16:40:49.040" Title="How to split a Seurat cluster in several subclusters?" Tags="|scrnaseq|seurat|single-cell|gene-expression|clustering|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18303" PostTypeId="2" ParentId="18302" CreationDate="2022-01-09T13:11:35.483" Score="1" Body="&lt;p&gt;Increase the clustering &lt;code&gt;resolution&lt;/code&gt; parameter to generate more (smaller) clusters, see &lt;a href=&quot;https://satijalab.org/seurat/reference/findclusters&quot; rel=&quot;nofollow noreferrer&quot;&gt;FindClusters in the Seurat docs&lt;/a&gt;. Whether or not this will neatly, split your clusters into subclusters depends on your data, but normally one can easily separate CD4 and NK cells from PBMCs. See also the &lt;a href=&quot;https://lazappi.github.io/clustree/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clustree&lt;/a&gt; approach for determining the optimal resolution.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively/additionally use a cell type annotation approach that does not require clustering, e.g. Seurat V4 permits &lt;a href=&quot;https://satijalab.org/seurat/articles/multimodal_reference_mapping.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;reference mapping&lt;/a&gt; and there are many other methods too&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2022-01-09T13:11:35.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18304" PostTypeId="2" ParentId="18302" CreationDate="2022-01-09T20:20:14.167" Score="1" Body="&lt;p&gt;I've done sub-clustering a few times on my Seurat data sets. The approach I take is to subset the clusters that need to be clustered (i.e. using &lt;code&gt;subset&lt;/code&gt;), carry out a clustering of only those cells, then transfer the subcluster labels back to the original dataset. Here's some rough code, which will need to be modified for your specific situation and code preferences:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;data.sc[[&amp;quot;main.cluster&amp;quot;]] &amp;lt;- Idents(data.sc);&#xA;clusterToSubset &amp;lt;- &amp;quot;B cells&amp;quot;;&#xA;data.sub &amp;lt;- subset(data.sc, subset = main.cluster == clusterToSubset);&#xA;## ** [Cluster data.sub] **&#xA;Idents(data.sc) &amp;lt;- &amp;quot;main.cluster&amp;quot;;&#xA;## This should work as long as cell order (i.e. column names) isn't changed&#xA;Idents(data.sc, cells=colnames(data.sc) %in% colnames(data.sub)) &amp;lt;- &#xA;    paste0(&amp;quot;Sub_&amp;quot;, Idents(data.sub));&#xA;data.sc[[&amp;quot;merged.cluster&amp;quot;]] &amp;lt;- Idents(data.sc);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This approach will work with multiple sub-clusters as well by changing the label that is pasted onto the sub-cluster idents:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;data.sc[[&amp;quot;main.cluster&amp;quot;]] &amp;lt;- Idents(data.sc);&#xA;clusterToSubset1 &amp;lt;- &amp;quot;B cells&amp;quot;;&#xA;data.sub.1 &amp;lt;- subset(data.sc, subset = main.cluster == clusterToSubset1);&#xA;## ** [Cluster data.sub.1] **&#xA;clusterToSubset2 &amp;lt;- &amp;quot;T cells&amp;quot;;&#xA;data.sub.2 &amp;lt;- subset(data.sc, subset = main.cluster == clusterToSubset2);&#xA;## ** [Cluster data.sub.2] **&#xA;Idents(data.sc) &amp;lt;- &amp;quot;main.cluster&amp;quot;;&#xA;## This should work as long as cell order (i.e. column names) isn't changed&#xA;Idents(data.sc, cells=colnames(data.sc) %in% colnames(data.sub.1)) &amp;lt;- &#xA;    paste0(&amp;quot;Sub1_&amp;quot;, Idents(data.sub.1));&#xA;Idents(data.sc, cells=colnames(data.sc) %in% colnames(data.sub.2)) &amp;lt;- &#xA;    paste0(&amp;quot;Sub2_&amp;quot;, Idents(data.sub.2));&#xA;data.sc[[&amp;quot;merged.cluster&amp;quot;]] &amp;lt;- Idents(data.sc);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="650" LastEditDate="2023-10-10T15:00:48.447" LastActivityDate="2023-10-10T15:00:48.447" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18305" PostTypeId="1" CreationDate="2022-01-10T15:21:19.103" Score="-1" ViewCount="35" Body="&lt;p&gt;Evolutionary speaking, some mutations lead to better fitness of an organism and its adaptation to the environment changes, but I was wondering if there are some constraints on having new mutations. For example, if a gene is needed for survival, I expect to see it more resistant to having new mutations. So, my question is what are these constraints, especially for bacteria?&lt;/p&gt;&#xA;" OwnerUserId="7444" LastActivityDate="2022-01-11T14:36:58.757" Title="Evolutionary speaking, what are the constraints on having new mutations?" Tags="|genome|bacteria|positive-selection|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18306" PostTypeId="1" CreationDate="2022-01-10T15:45:12.213" Score="1" ViewCount="72" Body="&lt;p&gt;I am doing RNAseq analysis and I am using ballgown procedure for it.&lt;/p&gt;&#xA;&lt;p&gt;There is an option in R to calculate differentially expressed genes and use FPKM in calculating differential gene expression, which is the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;results_genes = stattest(bg_filt, feature=&amp;quot;gene&amp;quot;, covariate=&amp;quot;state&amp;quot;, getFC=TRUE, meas=&amp;quot;FPKM&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When accessing head(results_genes) there is a column named &lt;code&gt;id&lt;/code&gt; that contain gene names.&lt;/p&gt;&#xA;&lt;p&gt;While some genes contain more or less readable names, others look like &lt;code&gt;gene-LOC100037748, gene-LOC100037751, gene-LOC100037760&lt;/code&gt; etc...&lt;/p&gt;&#xA;&lt;p&gt;Can I create a column with real gene names that correspond to these? Is there a sort of API or something maybe?&lt;/p&gt;&#xA;" OwnerUserId="6470" LastActivityDate="2022-01-10T15:45:12.213" Title="How can I get a column with real gene names in my ballgown analysis?" Tags="|r|rna-seq|differential-expression|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18307" PostTypeId="2" ParentId="18305" CreationDate="2022-01-10T17:44:14.327" Score="0" Body="&lt;p&gt;This is an undergraduate/graduate lecture type question and the answer varies: traditional Darwinists believed a reasonable proportion of mutations would result in beneficial adaptation, neutral theory however would state the vast majority of mutations have no effect on phenotype and only an absolute minority result in beneficial adaptation. It depends on which bacteria are the focus of investigation and the selective constraint (I would assume its antibiotics).&lt;/p&gt;&#xA;&lt;p&gt;Once the bacterial species and selective constraint are described its a straight-forward empirical investigation using molecular mathematics because the mutations driving antibiotic resistance are all known.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-01-11T14:36:58.757" LastActivityDate="2022-01-11T14:36:58.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18308" PostTypeId="2" ParentId="15958" CreationDate="2022-01-10T17:50:05.060" Score="1" Body="&lt;p&gt;As far as I know, DSL2 should be able to handle this. Have you specified an output qualifier for your process &lt;code&gt;get_fq_info&lt;/code&gt; (e.g. &lt;code&gt;path&lt;/code&gt;)? I ran into a similar issue when I forgot to specify it.&lt;/p&gt;&#xA;&lt;p&gt;For example, this will result in the error &lt;code&gt;process align_bowtie2 declares 1 input channel but 0 were specified&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process get_fq_info {&#xA;   input:&#xA;   path reads&#xA;&#xA;   output:&#xA;   &amp;quot;output.txt&amp;quot;&#xA;&#xA;   shell:&#xA;   &amp;quot;&amp;quot;&amp;quot;&#xA;   myprocess.sh ${reads} &amp;gt; output.txt&#xA;   &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And is probably solved by this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process get_fq_info {&#xA;   input:&#xA;   path reads&#xA;&#xA;   output:&#xA;   path &amp;quot;output.txt&amp;quot;&#xA;&#xA;   shell:&#xA;   &amp;quot;&amp;quot;&amp;quot;&#xA;   myprocess.sh ${reads} &amp;gt; output.txt&#xA;   &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14828" LastEditorUserId="14828" LastEditDate="2022-01-10T17:58:25.413" LastActivityDate="2022-01-10T17:58:25.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18310" PostTypeId="1" CreationDate="2022-01-11T16:21:07.773" Score="0" ViewCount="90" Body="&lt;p&gt;I want to install annovar but in one particular part I always get this error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;i1d18@RBGO-Server2:/data2/RNASeq/Angel/annovar&lt;span class=&quot;math-container&quot;&gt;$ perl annotate_variation.pl -downdb -webfrom annovar avsnp147 humandb/&#xA;NOTICE: The --buildver is set as 'hg18' by default&#xA;NOTICE: Web-based checking to see whether ANNOVAR new version is available ... Done&#xA;NOTICE: Downloading annotation database http://www.openbioinformatics.org/annovar/download/hg18_avsnp147.txt.gz ... Failed&#xA;NOTICE: Downloading annotation database http://www.openbioinformatics.org/annovar/download/hg18_avsnp147.txt.idx.gz ... Failed&#xA;WARNING: Some files cannot be downloaded, including http://www.openbioinformatics.org/annovar/download/hg18_avsnp147.txt.gz, http://www.openbioinformatics.org/annovar/download/hg18_avsnp147.txt.idx.gz&#xA;fi1d18@RBGO-Server2:/data2/RNASeq/Angel/annovar$&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Googling did not return any clue&lt;/p&gt;&#xA;&lt;p&gt;Any help please?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="57" LastEditDate="2022-01-12T14:27:49.567" LastActivityDate="2022-01-12T14:27:49.567" Title="An error with annovar" Tags="|vcf|sequence-annotation|maf|wes|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18311" PostTypeId="2" ParentId="18290" CreationDate="2022-01-11T16:28:58.827" Score="1" Body="&lt;p&gt;I wrote vcf2table: &lt;a href=&quot;http://lindenb.github.io/jvarkit/VcfToTable.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://lindenb.github.io/jvarkit/VcfToTable.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view in.bcf |  java -jar dist/vcf2table.jar | more&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9Tc4w.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9Tc4w.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and &lt;a href=&quot;http://lindenb.github.io/jvarkit/SwingVcfView.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://lindenb.github.io/jvarkit/SwingVcfView.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UVKJ7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UVKJ7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar dist/swingvcfview.jar indexed.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="71" LastActivityDate="2022-01-11T16:28:58.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18313" PostTypeId="1" CreationDate="2022-01-11T23:35:49.137" Score="1" ViewCount="815" Body="&lt;p&gt;I have a scRNA-seq Seurat object I've analyzed, and I noticed that for some of the clusters, there's more than one cell type.&lt;/p&gt;&#xA;&lt;p&gt;I've created a subset which and run FindClusters again to label the cell more efficiently, and now I want to &amp;quot;paste&amp;quot; the Idents I've assigned in the subcluster to the original object.&lt;/p&gt;&#xA;&lt;p&gt;How do I do it?&lt;/p&gt;&#xA;" OwnerUserId="14823" LastActivityDate="2023-07-13T16:06:10.757" Title="Assigning subcluster idents to original object" Tags="|scrnaseq|seurat|single-cell|clustering|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18314" PostTypeId="1" CreationDate="2022-01-12T01:44:27.637" Score="1" ViewCount="46" Body="&lt;p&gt;I want to map NCBI taxID, specifically a pair of Genbank sequences, to the divergence time estimate of their common ancestor. For example, using a coalescence theory approach predominantly used in phylogenetics.&#xA;Do we have a database for such correlation?&lt;/p&gt;&#xA;" OwnerUserId="14169" LastEditorUserId="3848" LastEditDate="2022-01-22T04:29:09.793" LastActivityDate="2022-01-22T04:29:09.793" Title="Mapping NCBI taxID to divergence time?" Tags="|phylogenetics|phylogeny|taxonomy|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18315" PostTypeId="2" ParentId="3482" CreationDate="2022-01-12T03:53:13.553" Score="0" Body="&lt;p&gt;You may be interested in a Python library I developed called &lt;a href=&quot;https://github.com/SACGF/pyreference&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyReference&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Install&#xA;python3 -m pip install pyreference&#xA;# Process GTF into a JSON file&#xA;pyreference_gff_to_json.py --gtf gencode.hg38.gtf&#xA;# Create a setup file (see website for details)&#xA;# Run against your BAM file&#xA;pyreference_biotype.py ${BAM_FILE}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This generates a graph:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Tsjr3.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Tsjr3.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As well as a CSV of read counts per biotype for each length. You can sum up the read lengths for a total.&lt;/p&gt;&#xA;" OwnerUserId="1713" LastEditorUserId="1713" LastEditDate="2022-01-13T03:54:04.760" LastActivityDate="2022-01-13T03:54:04.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18317" PostTypeId="2" ParentId="18314" CreationDate="2022-01-12T18:07:31.483" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;I want to map NCBI taxID to their evolution time. Do we have a database for such correlation?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It would be best to edit your question to clarify what you mean by &amp;quot;evolution time&amp;quot; since this is not a well-defined term in molecular evolution. Even so, I think we can provide a clear answer to this question.&lt;/p&gt;&#xA;&lt;p&gt;Estimating branch lengths between divergent taxa is an area of active research—there is no &lt;em&gt;one&lt;/em&gt; method that works best in all scenarios. As far as taxonomy is concerned, resources like the NCBI Taxonomy database are under continuous refinement by the community.&lt;/p&gt;&#xA;&lt;p&gt;Computing estimated divergence times and mapping those times to taxids would be a non-trivial task even for a single clade. Actively maintaining a database of divergence times across the entire taxonomy would be incredibly challenging, and it's doubtful that any such resource exists.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2022-01-12T18:07:31.483" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18319" PostTypeId="1" AcceptedAnswerId="18325" CreationDate="2022-01-12T19:45:46.660" Score="0" ViewCount="266" Body="&lt;p&gt;&lt;strong&gt;Goal:&lt;/strong&gt; To ensure &amp;quot;the sum of the counts for [my samples] is the same as the counts in the [samples] columns in &lt;code&gt;ddsColl&lt;/code&gt;&amp;quot; after collapsing replicates using DESeq2.&lt;/p&gt;&#xA;&lt;p&gt;Before performing differential expression (DE) analysis on my RNA-seq data stored in a DESeqDataSet object, I need to collapse the read counts observed for each of my technical replicates. This should be simple enough using the DESeq2 &lt;a href=&quot;https://bioconductor.org/packages/devel/bioc/manuals/DESeq2/man/DESeq2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt; and &lt;a href=&quot;http://bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;vignettes&lt;/a&gt; available as reference.&lt;/p&gt;&#xA;&lt;p&gt;This is how you collapse replicates:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ddsColl &amp;lt;- collapseReplicates(dds, dds&lt;span class=&quot;math-container&quot;&gt;$sample, dds$&lt;/span&gt;run)&#xA;# examine the colData of the collapsed data&#xA;colData(ddsColl)&#xA;# check that the sum of the counts for &amp;quot;sample1&amp;quot; is the same&#xA;# as the counts in the &amp;quot;sample1&amp;quot; column in ddsColl&#xA;matchFirstLevel &amp;lt;- dds&lt;span class=&quot;math-container&quot;&gt;$sample == levels(dds$&lt;/span&gt;sample)[1]&#xA;stopifnot(all(rowSums(counts(dds[,matchFirstLevel])) == counts(ddsColl[,1])))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, when I try this, I get the following error with no apparent resolution reported in the manual or on online forums:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error: all(rowSums(counts(dds[, matchFirstLevel])) == counts(ddsColl[,  .... is not TRUE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;How might I circumvent this error and continue on with the analysis?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Input files:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(cts)&#xA;            KO1_P1_L001.Counts KO1_P1_L002.Counts KO1_P2_L001.Counts KO1_P2_L002.Counts KO1_P3_L001.Counts KO1_P3_L002.Counts KO2_P1_L001.Counts KO2_P1_L002.Counts KO2_P2_L001.Counts KO2_P2_L002.Counts KO2_P3_L001.Counts KO2_P3_L002.Counts WT_P1_L001.Counts WT_P1_L002.Counts WT_P2_L001.Counts WT_P2_L002.Counts WT_P3_L001.Counts WT_P3_L002.Counts&#xA;DDX11L1                      1                  0                  1                  1                  2                  1                  0                  1                  2                  1                  3                  1                 7                 2                 1                 1                 0                 0&#xA;WASH7P                     124                144                128                151                102                118                 39                 41                103                 87                105                106               125               120                13                26               104               136&#xA;MIR6859-1                    8                  6                  4                  5                  4                  6                  2                  2                  7                  2                  7                  6                 6                 6                 2                 2                 2                 9&#xA;MIR1302-2HG                  1                  4                  0                  0                  0                  2                  2                  2                  2                  2                  0                  4                 1                 2                 0                 3                 2                 2&#xA;MIR1302-2                    0                  0                  0                  0                  0                  0                  0                  0                  0                  0                  0                  0                 0                 0                 0                 0                 0                 0&#xA;FAM138A                      0                  0                  0                  0                  0                  0                  0                  0                  0                  0                  0                  0                 0                 0                 0                 0                 0                 0&#xA;&#xA;&amp;gt; print(coldata)&#xA;                   condition sample           run&#xA;KO1_P1_L001.Counts    longKO     l1  long_P1_L001&#xA;KO1_P1_L002.Counts    longKO     l1  long_P1_L002&#xA;KO1_P2_L001.Counts    longKO     l2  long_P2_L001&#xA;KO1_P2_L002.Counts    longKO     l2  long_P2_L002&#xA;KO1_P3_L001.Counts    longKO     l3  long_P3_L001&#xA;KO1_P3_L002.Counts    longKO     l3  long_P3_L002&#xA;KO2_P1_L001.Counts   shortKO     s1 short_P1_L001&#xA;KO2_P1_L002.Counts   shortKO     s1 short_P1_L002&#xA;KO2_P2_L001.Counts   shortKO     s2 short_P2_L001&#xA;KO2_P2_L002.Counts   shortKO     s2 short_P2_L002&#xA;KO2_P3_L001.Counts   shortKO     s3 short_P3_L001&#xA;KO2_P3_L002.Counts   shortKO     s3 short_P3_L002&#xA;WT_P1_L001.Counts         WT     w1    wt_P1_L001&#xA;WT_P1_L002.Counts         WT     w1    wt_P1_L002&#xA;WT_P2_L001.Counts         WT     w2    wt_P2_L001&#xA;WT_P2_L002.Counts         WT     w2    wt_P2_L002&#xA;WT_P3_L001.Counts         WT     w3    wt_P3_L001&#xA;WT_P3_L002.Counts         WT     w3    wt_P3_L002&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;My workflow:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dds &amp;lt;- DESeqDataSetFromMatrix(countData = cts, colData = coldata, design = ~ condition)&#xA;&amp;gt; ddsColl &amp;lt;- collapseReplicates(dds, dds&lt;span class=&quot;math-container&quot;&gt;$sample, dds$&lt;/span&gt;run)&#xA;&amp;gt; colData(ddsColl)&#xA;DataFrame with 9 rows and 4 columns&#xA;   condition      sample           run               runsCollapsed&#xA;    &amp;lt;factor&amp;gt; &amp;lt;character&amp;gt;   &amp;lt;character&amp;gt;                 &amp;lt;character&amp;gt;&#xA;l1    longKO          l1  long_P1_L001   long_P1_L001,long_P1_L002&#xA;l2    longKO          l2  long_P2_L001   long_P2_L001,long_P2_L002&#xA;l3    longKO          l3  long_P3_L001   long_P3_L001,long_P3_L002&#xA;s1   shortKO          s1 short_P1_L001 short_P1_L001,short_P1_L002&#xA;s2   shortKO          s2 short_P2_L001 short_P2_L001,short_P2_L002&#xA;s3   shortKO          s3 short_P3_L001 short_P3_L001,short_P3_L002&#xA;w1        WT          w1    wt_P1_L001       wt_P1_L001,wt_P1_L002&#xA;w2        WT          w2    wt_P2_L001       wt_P2_L001,wt_P2_L002&#xA;w3        WT          w3    wt_P3_L001       wt_P3_L001,wt_P3_L002&#xA;&#xA;&amp;gt; matchFirstLevel &amp;lt;- dds&lt;span class=&quot;math-container&quot;&gt;$sample == levels(dds$&lt;/span&gt;sample)[1]&#xA;&amp;gt; matchFirstLevel&#xA;logical(0)&#xA;&#xA;&amp;gt; stopifnot(all(rowSums(counts(dds[,matchFirstLevel])) == counts(ddsColl[,1])))&#xA;Error: all(rowSums(counts(dds[, matchFirstLevel])) == counts(ddsColl[,  .... is not TRUE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From the output of &lt;code&gt;dds&lt;/code&gt; and &lt;code&gt;ddsColl&lt;/code&gt;, it seems the number of columns decreased from 18 to 9, as expected. The only difference I see between my DESeq2 workflow output is the result of &lt;code&gt;matchFirstLevel&lt;/code&gt; (see below for expected result). Why might this be for my dataset, but not the example dataset?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Example workflow:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; dds &amp;lt;- makeExampleDESeqDataSet(m=12)&#xA;&amp;gt; dds&lt;span class=&quot;math-container&quot;&gt;$sample &amp;lt;- factor(sample(paste0(&quot;sample&quot;,rep(1:9, c(2,1,1,2,1,1,2,1,1)))))&#xA;&amp;gt; dds$&lt;/span&gt;run &amp;lt;- paste0(&amp;quot;run&amp;quot;,1:12)&#xA;&amp;gt; ddsColl &amp;lt;- collapseReplicates(dds, dds&lt;span class=&quot;math-container&quot;&gt;$sample, dds$&lt;/span&gt;run)&#xA;&amp;gt; colData(ddsColl)&#xA;DataFrame with 9 rows and 4 columns&#xA;        condition   sample         run runsCollapsed&#xA;         &amp;lt;factor&amp;gt; &amp;lt;factor&amp;gt; &amp;lt;character&amp;gt;   &amp;lt;character&amp;gt;&#xA;sample1         B  sample1       run10   run10,run11&#xA;sample2         A  sample2        run5          run5&#xA;sample3         A  sample3        run6          run6&#xA;sample4         A  sample4        run1     run1,run4&#xA;sample5         B  sample5        run7          run7&#xA;sample6         B  sample6        run9          run9&#xA;sample7         A  sample7        run3    run3,run12&#xA;sample8         B  sample8        run8          run8&#xA;sample9         A  sample9        run2          run2&#xA;&#xA;&amp;gt; matchFirstLevel &amp;lt;- dds&lt;span class=&quot;math-container&quot;&gt;$sample == levels(dds$&lt;/span&gt;sample)[1]&#xA;&amp;gt; matchFirstLevel&#xA; [1] FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE FALSE  TRUE  TRUE FALSE&#xA;&#xA;&amp;gt; stopifnot(all(rowSums(counts(dds[,matchFirstLevel])) == counts(ddsColl[,1])))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9007" LastEditorUserId="9007" LastEditDate="2022-01-13T20:32:46.513" LastActivityDate="2022-01-14T07:48:56.517" Title="How to remedy a DESeq2 collapsing technical replicates error?" Tags="|r|rna-seq|deseq2|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18321" PostTypeId="1" CreationDate="2022-01-13T20:38:52.170" Score="2" ViewCount="556" Body="&lt;p&gt;I am trying to find a hack for getting a process to run until all emissions in a channel are used up. The problem is that the process also takes as input a second channel that only has a single emission. Therefore, the process only runs once, but I need it to repeatedly use the single emission channel for all the emissions in the other channel. I imagine this is not too uncommon a problem, but I have not found a solution.&lt;/p&gt;&#xA;&lt;p&gt;To be more clear, here is the actual bioinformatics I am trying to do: I have split fastqs into files for multiple samples and emit these files into a channel. I copy this channel because I want to use the files again later. I then build an assembly from the combined files which I index with bwa. I next pass the indexed assembly (index channel) into a new process that also takes as input ALL of the original fastq files (the ones I put into a duplicate channel earlier). The goal is to align each of the fastq files against the genome. However, there are only a few emissions in the index channel so the process doesn't repeat for all the items in my fastq channel.&lt;/p&gt;&#xA;&lt;p&gt;To aid you all here is a simple example that may help, but take note that my situation doesn't use value channels so is a bit trickier to deal with.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;exampleChannel_multiplethings = Channel.from( [1, 3, 5, 7, 9] )&#xA;exampleChannel_onething = Channel.from( [1] )&#xA;&#xA;process foo {&#xA;  input:&#xA;  val x from exampleChannel_multiplethings&#xA;  val y from exampleChannel_onething&#xA;  &#xA;  output:&#xA;  file 'x.txt' into foo_out&#xA;&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  echo &lt;span class=&quot;math-container&quot;&gt;$y $&lt;/span&gt;x &amp;gt; x.txt&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This outputs &amp;quot;1 1&amp;quot; into the x.txt file. This is expected behavior, but I need to circumvent it somehow so that the process is repeated for all the items in the exampleChannel_multiplethings channel. Important note, if I was dealing with single values I could probably use one of the combine operators, but in reality I am trying to pass a handful of files for an indexed genome into the process. I haven't found a combine operator that works, but hopefully I have missed something!&lt;/p&gt;&#xA;&lt;p&gt;Thanks for any assistance&lt;/p&gt;&#xA;&lt;p&gt;Actual code (overly lengthy to paste here) can be found at: &lt;a href=&quot;https://github.com/JHarrisonEcoEvo/nextflow_gbs/blob/main/gbs_nextflow&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/JHarrisonEcoEvo/nextflow_gbs/blob/main/gbs_nextflow&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14844" LastEditorUserId="57" LastEditDate="2022-01-13T23:41:14.350" LastActivityDate="2022-01-13T23:41:14.350" Title="How fo force nextflow to repeat a process until all values in a particular channels are used up BUT a single value from another channel is needed" Tags="|bwa|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18323" PostTypeId="2" ParentId="18321" CreationDate="2022-01-13T22:24:16.413" Score="4" Body="&lt;p&gt;I discovered a solution using a combination of &lt;code&gt;.collect()&lt;/code&gt; and &lt;code&gt;each&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If one collects from a path the result is a value channel of paths. Value channels can be used multiple times, whereas queue channels cannot.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ch_collected = Channel.fromPath( &amp;quot;somepath&amp;quot; ).collect()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;ch_collected&lt;/code&gt; can be used multiple times but if one tries to input it to a process using the usual file..from syntax one will get all the files input to the process at once. Instead, one must use the &lt;code&gt;each&lt;/code&gt; option during input. E.g.,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  input:&#xA;    each fastq_file from ch_collected&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I hope this helps someone.&lt;/p&gt;&#xA;" OwnerUserId="14844" LastEditorUserId="57" LastEditDate="2022-01-13T23:39:41.467" LastActivityDate="2022-01-13T23:39:41.467" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18324" PostTypeId="2" ParentId="10386" CreationDate="2022-01-14T04:30:48.027" Score="1" Body="&lt;p&gt;I've written a Python library called &lt;a href=&quot;https://github.com/SACGF/pyreference&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyReference&lt;/a&gt; which converts a GTF / GFF (RefSeq and Ensembl) into a gzipped JSON file.&lt;/p&gt;&#xA;&lt;p&gt;You can just use this if you like but there's also a Python wrapper around the JSON, which allows you to write genomics code more naturally.&lt;/p&gt;&#xA;&lt;p&gt;Python isn't known for being fast but the library function for reading a JSON file is highly optimised - the following takes less than 4 seconds on my laptop:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import pyreference&#xA;&#xA;reference = pyreference.Reference()&#xA;&#xA;my_gene_ids = [&amp;quot;MSN&amp;quot;, &amp;quot;GATA2&amp;quot;, &amp;quot;ZEB1&amp;quot;]&#xA;for gene in reference[my_gene_ids]:&#xA;    average_length = np.mean([t.length for t in gene.transcripts])&#xA;    print(&amp;quot;%s average length = %.2f&amp;quot; % (gene, average_length))&#xA;    print(gene.iv)&#xA;    for transcript in gene.transcripts:&#xA;        if transcript.is_coding:&#xA;            threep_utr = transcript.get_3putr_sequence()&#xA;            print(&amp;quot;%s end of 3putr: %s&amp;quot; % (transcript.get_id(), threep_utr[-20:]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1713" LastActivityDate="2022-01-14T04:30:48.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18325" PostTypeId="2" ParentId="18319" CreationDate="2022-01-14T07:48:56.517" Score="1" Body="&lt;p&gt;&lt;code&gt;sample&lt;/code&gt; is a character vector, so &lt;code&gt;levels(dds$sample)&lt;/code&gt; is returning &lt;code&gt;null&lt;/code&gt;. Presumably you mean to use &lt;code&gt;dds$sample[1]&lt;/code&gt; here instead, which will likely solve the problem.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2022-01-14T07:48:56.517" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18326" PostTypeId="1" CreationDate="2022-01-14T14:44:21.053" Score="1" ViewCount="56" Body="&lt;p&gt;Two years ago, I started working on a project uses both RNAseq and ATACseq. It's supposed to be a simple Healthy Control (HC) vs disease study. The sample collection was done in 2 phases, and it was clear that there was a batch effect that we could adjust for.&#xA;However, recently, we received additional metadata. Plotting a PCA plot showed that there was another, larger batch effect that we didn't account for--location of sample donation. There are 4 different locations with the disease samples being from any of the 4, but &lt;strong&gt;ALL&lt;/strong&gt; HC came from only one of the locations.&lt;/p&gt;&#xA;&lt;p&gt;I reran the count data through DESeq2 with this design formula: phase + location + disease. It didn't fuss about collinearity like it often likes to do and then pooped out a big list of DEG in the RNAseq.&lt;/p&gt;&#xA;&lt;p&gt;I could probably run the DEG through GSEA to see if the results match the disease's previously known signatures for biological validity, but what statistical worries should I have about this design matrix? What justification would I need for this statistical asymmetry? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="9521" LastActivityDate="2022-01-14T14:44:21.053" Title="Batch Effects that are Close to Collinear with Main Effect" Tags="|deseq2|batch-effects|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18327" PostTypeId="1" AcceptedAnswerId="18338" CreationDate="2022-01-14T20:43:19.747" Score="1" ViewCount="587" Body="&lt;p&gt;I have one more question related to genotype file. I submitted the job in michigan imputation server.&lt;/p&gt;&#xA;&lt;p&gt;and my imputed dosage for chr 22 file is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;22 16050435 22:16050435:T:C T C . PASS&#xA;AF=0.00098;MAF=0.00098;R2=0.23769;IMPUTED GT:DS:HDS:GP&#xA;0|0:0.002:0.001,0.000:0.998,0.002,0.000 0|0:0:0,0:1,0,0&#xA;0|0:0:0,0:1,0,0 0|0:0.001:0.000,0.001:0.999,0.001,0.000&#xA;0|0:0:0,0:1,0,0 0|0:0.001:0.001,0.000:0.999,0.001,0.000&#xA;0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0&#xA;0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0&#xA;0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0&#xA;0|0:0.001:0.001,0.000:0.999,0.001,0.000&#xA;0|0:0.002:0.000,0.002:0.998,0.002,0.000 0|0:0:0,0:1,0,0&#xA;0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0&#xA;0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0&#xA;0|0:0.001:0.000,0.001:0.999,0.001,0.000 0|0:0:0,0:1,0,0&#xA;0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0&#xA;0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0&#xA;0|0:0:0,0:1,0,0 0|0:0.002:0.000,0.002:0.998,0.002,0.000&#xA;0|0:0:0,0:1,0,0 0|0:0.001:0.001,0.000:0.999,0.001,0.000&#xA;0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0 0|0:0:0,0:1,0,0&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Its coming like this.&lt;/p&gt;&#xA;&lt;p&gt;wherein the test data provided in your predixcan tutorial looks like this:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;22_16494187_C_A_b37 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0&#xA;0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0&#xA;0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 1&#xA;0 0 0 0 0 0 0&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Also it states that the imputed dosage should be encoded on a 0-2 scale representing the number imputed or number of the effect alleles the sample posseses. Do anyone know should I convert my vcf files to the desired input genotype file to train the model. I don't understand in my vcf files which column is representing dosage values Thank you.&lt;/p&gt;&#xA;" OwnerUserId="14859" LastActivityDate="2022-01-16T12:06:05.853" Title="imputed dosage values for vcf files" Tags="|vcf|snp|genotyping|imputation|chromosomes|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18328" PostTypeId="1" AcceptedAnswerId="18335" CreationDate="2022-01-14T21:27:57.347" Score="0" ViewCount="83" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pVxA6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pVxA6.png&quot; alt=&quot;This is how my input file looks like. I am not able to show the header of the file so I am showing the one with 22_160549_T_C_b37 one&quot; /&gt;&lt;/a&gt;&lt;a href=&quot;https://i.stack.imgur.com/ij2dh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ij2dh.png&quot; alt=&quot;This is how the output file should look like. This is not mine. But a test data image&quot; /&gt;&lt;/a&gt;I have an imputed dosage for chr 22 file. its content is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#[1]CHROM       [2]POS  [3]REF  [4]ALT  [5]HG00096_HG00096      [6]HG00097_HG00097      [7]HG00099_HG00099      [8]HG00100_HG00100     &#xA;22      16051249        T       C       0.0     1.0     1.0     0.0    0.0     1.0     0.0     0.0     1.0     0.0     &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to remove &lt;code&gt;[]&lt;/code&gt; and remove anything coming after &lt;code&gt;_&lt;/code&gt; from this file as well as instead of separate &lt;code&gt;22&lt;/code&gt; &lt;code&gt;16051249&lt;/code&gt; &lt;code&gt;T&lt;/code&gt; and &lt;code&gt;C&lt;/code&gt; i want to write:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;22_16051249_T_C_b37&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So the final file should look like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Id      HG00096 HG00097 HG00099 HG00100 &#xA;22_16051249_T_C_b37     0       1       1       0       1       0       0       1       0       &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can anyone please help me out with this.&lt;/p&gt;&#xA;" OwnerUserId="14859" LastEditorUserId="57" LastEditDate="2022-01-16T17:46:00.107" LastActivityDate="2022-01-16T17:46:00.107" Title="How to format my file using linux tools?" Tags="|file-formats|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="18329" PostTypeId="1" AcceptedAnswerId="18333" CreationDate="2022-01-15T09:37:18.553" Score="2" ViewCount="385" Body="&lt;p&gt;I have a pipeline, which is written in &lt;code&gt;WDL&lt;/code&gt; &amp;amp; runs with the help of &lt;code&gt;Cromwell&lt;/code&gt;. I use &lt;code&gt;Docker&lt;/code&gt; for containerization.&lt;br /&gt;&#xA;However, my container is dependent on huge (&amp;gt;100GB databases), which cannot be shipped with the image.&lt;br /&gt;&#xA;Is there an option to mount a directory from the host filesystem to a &lt;code&gt;Docker&lt;/code&gt; while running &lt;code&gt;Cromwell&lt;/code&gt; workflow? I found this functionality in &lt;code&gt;snakemake&lt;/code&gt;, therefore I assume it is common, but I haven't found it in the docs yet.&lt;/p&gt;&#xA;" OwnerUserId="14512" LastActivityDate="2022-01-15T23:06:00.910" Title="Mounting a folder from the host to a Docker, while running it as Cromwell runtime" Tags="|wdl|workflow-management|docker|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18330" PostTypeId="1" CreationDate="2022-01-15T14:40:55.007" Score="-1" ViewCount="136" Body="&lt;p&gt;I am performing  multiple sequence alignment using Clustal Omega for 600 RNA covid-sequences, i.e. cDNA on Genbank, with input characters around 30574 characters for each sequence.&lt;/p&gt;&#xA;&lt;p&gt;I am running it on windows cmd.&#xA;I defined parameters in input of maxseqle = 37000, however, the output each time gives duplicated length around 75000 charcters per each sequence.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;clustalo.exe -i Allseq.fasta --is-profile --use-kimura --seqtype DNA --maxseqlen 37000 --threads 8 -o myclustalv3.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I solve this problem, what will I define in input parameters. This problem occurs with large no of sequences 600 sequences. In test of 40 seq, it gave me normal output length around 32000 characters but when align all 600 sequences together, it gave this duplicated length results.&lt;/p&gt;&#xA;" OwnerUserId="14295" LastEditorUserId="3848" LastEditDate="2022-01-18T15:38:21.513" LastActivityDate="2022-01-19T19:24:59.223" Title="ClustalOmega alignment" Tags="|sequence-alignment|multiple-sequence-alignment|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18331" PostTypeId="1" AcceptedAnswerId="18334" CreationDate="2022-01-15T17:56:13.947" Score="1" ViewCount="3399" Body="&lt;p&gt;I want to create a dataframe in Python starting from a FASTA format file. Given the toy FASTA file that I am attaching, I built this program in Python that returns four colums corresponding to id, sequence length, sequence, animal name and rows corresponding to all the data available.&lt;/p&gt;&#xA;&lt;p&gt;However, I am trying to understand how to modify this code in order to create a dataframe in which classes Human and Dog have the same number of data. For example, I want to say to Python: &amp;quot;Append to record (that is the empty list) id, sequence length, sequence and animal for Human, but do it a number of times that is equal to the number of data of the class with minimum number of data (that is Dog)&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I think that a &lt;code&gt;while&lt;/code&gt; loop is needed but I have a bit troubles to understand how to do it. Any suggestion ?&lt;/p&gt;&#xA;&lt;p&gt;Below the Python code I wrote and the FASTA format file I used.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;import re&#xA;def read_fasta(file_path, columns) :&#xA;    from Bio.SeqIO.FastaIO import SimpleFastaParser &#xA;    with open(&amp;quot;Proof.txt&amp;quot;) as fasta_file :  &#xA;        records = [] # create empty list&#xA;        for title, sequence in SimpleFastaParser(fasta_file): #SimpleFastaParser Iterate over Fasta records as string tuples. For each record a tuple of two strings is returned, the FASTA title line (without the leading ‘&amp;gt;’ character), and the sequence (with any whitespace removed). &#xA;            record = []&#xA;            title_splits=re.findall(r&amp;quot;[\w']+&amp;quot;, title) # Data cleaning is needed&#xA;          &#xA;                 &#xA;            &#xA;            record.append(title_splits[0])  #First values are ID (Append adds element to a list)&#xA;            record.append(len(sequence)) #Second values are sequences lengths&#xA;            sequence = &amp;quot; &amp;quot;.join(sequence) #It converts into one line&#xA;            record.append(sequence)#Third values are sequences&#xA;                &#xA;            #Fourth column will contain the species&#xA;            if &amp;quot;Human&amp;quot; in title_splits:&#xA;                    record.append(&amp;quot;Human&amp;quot;)    &#xA;            else:&#xA;                    record.append(&amp;quot;Dog&amp;quot;)&#xA;                &#xA;           &#xA;              &#xA;            records.append(record)&#xA;    return pd.DataFrame(records, columns = columns) #We have created a function that returns a dataframe&#xA;&#xA;#Now let's use this function by inserting in the first argument the file name (or file path if your working directory is different from where the fasta file is)        &#xA;#And in the second one the names of columns&#xA;data = read_fasta(&amp;quot;Proof.txt&amp;quot;, columns=[&amp;quot;id&amp;quot;,&amp;quot;sequence_length&amp;quot;, &amp;quot;sequence&amp;quot;, &amp;quot;animal&amp;quot;])&#xA;data&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The FASTA format file is this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Numer|Human|Hearth&#xA;HSSFIEIVNIEHVIEHIVK&#xA;&amp;gt;Numer|Human|Hearth&#xA;HSSFIEIVNIEHVIEHIVK&#xA;&amp;gt;Numer|Human|Hearth&#xA;HSSFIEIVNIEHVIEHIVK&#xA;&amp;gt;Numer|Human|Hearth&#xA;HSSFIEIVNIEHVIEHIVK&#xA;&amp;gt;Numer|Dog|Hearth&#xA;HSSFIEIVNIEHVIEHIVK&#xA;&amp;gt;Numer|Dog|Hearth&#xA;HSSFIEIVNIEHVIEHIVK&#xA;&amp;gt;Numer|Dog|Hearth&#xA;HSSFIEIVNIEHVIEHIVK&#xA;&amp;gt;Numer|Dog|Hearth&#xA;HSSFIEIVNIEHVIEHIVK&#xA;&amp;gt;Numer|Human|Hearth&#xA;HSSFIEIVNIEHVIEHIVK&#xA;&amp;gt;Numer|Human|Hearth&#xA;HSSFIEIVNIEHVIEHIVK&#xA;&amp;gt;Numer|Human|Hearth&#xA;HSSFIEIVNIEHVIEHIVK&#xA;&amp;gt;Numer|Human|Hearth&#xA;HSSFIEIVNIEHVIEHIVK&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My code prints a dataframe like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;       id  sequence_length                               sequence animal&#xA;0   Numer               19  H S S F I E I V N I E H V I E H I V K  Human&#xA;1   Numer               19  H S S F I E I V N I E H V I E H I V K  Human&#xA;2   Numer               19  H S S F I E I V N I E H V I E H I V K  Human&#xA;3   Numer               19  H S S F I E I V N I E H V I E H I V K  Human&#xA;4   Numer               19  H S S F I E I V N I E H V I E H I V K    Dog&#xA;5   Numer               19  H S S F I E I V N I E H V I E H I V K    Dog&#xA;6   Numer               19  H S S F I E I V N I E H V I E H I V K    Dog&#xA;7   Numer               19  H S S F I E I V N I E H V I E H I V K    Dog&#xA;8   Numer               19  H S S F I E I V N I E H V I E H I V K  Human&#xA;9   Numer               19  H S S F I E I V N I E H V I E H I V K  Human&#xA;10  Numer               19  H S S F I E I V N I E H V I E H I V K  Human&#xA;11  Numer               19  H S S F I E I V N I E H V I E H I V K  Human&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I would like that the number of rows for Human is the same for Dog (because, in other words, I would like the same number of data for each class).&lt;/p&gt;&#xA;&lt;p&gt;Hoping to have been clear, I thank you in advance.&lt;/p&gt;&#xA;&lt;p&gt;P.S.: My program is an adapted version of &lt;a href=&quot;https://github.com/aws-samples/amazon-sagemaker-protein-classification/blob/main/sm-protein-classification.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; notebook .&lt;/p&gt;&#xA;" OwnerUserId="10594" LastEditorUserId="10594" LastEditDate="2022-01-18T16:56:42.523" LastActivityDate="2022-01-27T22:27:30.187" Title="Help to create a dataframe in Python from a FASTA file" Tags="|python|pandas|dataframe|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18332" PostTypeId="1" CreationDate="2022-01-15T20:11:10.257" Score="0" ViewCount="311" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/9505729/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have 2 directories that contains multiple VCF files.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/Irq3f.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Irq3f.png&quot; alt=&quot;dir&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want to write code in python that will process all vcf files and create an AFD (allele frequency difference) plot for each VCF file present in a folder&lt;/p&gt;&#xA;&lt;p&gt;I have started with this code, which attempts to convert the VCF files into a pandas dataframe:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Import Module&#xA;import os&#xA;import pandas as pd&#xA;# Folder Path&#xA;path1 = &amp;quot;C://Users//USER//Desktop//isiah/VCFs_1/&amp;quot;&#xA;path2 = &amp;quot;C://Users//USER//Desktop//isiah/VCFs_2/&amp;quot;&#xA;#os.chdir(path1)&#xA;&#xA;# Read text File&#xA;&#xA;&#xA;def read_text_file(file_path1,file_path2):&#xA;    with open(file_path, 'r') as f:&#xA;        print(f.read())&#xA;&#xA;&#xA;# iterate through all file&#xA;for file in os.listdir():&#xA;    # Check whether file is in text format or not&#xA;    if file.endswith(&amp;quot;.vcf&amp;quot;):&#xA;        file_path1 = f&amp;quot;{path1}\{file}&amp;quot;&#xA;        file_path2 = f&amp;quot;{path2}\{file}&amp;quot;&#xA;        print(file_path1,&amp;quot;\n\n&amp;quot;,file_path2)&#xA;&#xA;# call read text file function&#xA;#data = read_text_file(path1,path2)&#xA;#print(read_text_file(path1,path2))&#xA;Final= pd.DataFrame(read_text_file(path1,path2))&#xA;print(Final)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and its giving me output without calling function and also gives me empty dataframe. Also i am not sure if result is showing all vcf files record or not.&lt;/p&gt;&#xA;&lt;p&gt;Kindly help me to modify this code. I need my all result in dataframe so i can make plot easily&lt;/p&gt;&#xA;&lt;p&gt;Folder 1 contains 3 test vcfs whereas folder 2 contains 4 test vcfs:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QHERb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QHERb.png&quot; alt=&quot;directories&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is test 1 vcf file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.2&#xA;##ALT=&amp;lt;ID=NON_REF,Description=&amp;quot;Represents any possible alternative allele at this location&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=FS,Description=&amp;quot;FS&amp;gt;60.0&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=HS,Description=&amp;quot;HaplotypeScore&amp;lt;13.0&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=LowQual,Description=&amp;quot;Low quality&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=MQ,Description=&amp;quot;MQ&amp;lt;40.0&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=MQRS,Description=&amp;quot;MQRankSum &amp;lt; -12.5&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=Mask,Description=&amp;quot;Doesn't overlap a user-input mask&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=PASS,Description=&amp;quot;All filters passed&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=QD,Description=&amp;quot;QD&amp;lt;2.0&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=RPRS,Description=&amp;quot;ReadPosRankSum &amp;lt; -8.0&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=AD,Number=R,Type=Integer,Description=&amp;quot;Allelic depths for the ref and alt alleles in the order listed&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=DP,Number=1,Type=Integer,Description=&amp;quot;Approximate read depth (reads with MQ=255 or with bad mates are filtered)&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GQ,Number=1,Type=Integer,Description=&amp;quot;Genotype Quality&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&amp;quot;Genotype&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=PGT,Number=1,Type=String,Description=&amp;quot;Physical phasing haplotype information, describing how the alternate alleles are phased in relation to one another&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=PID,Number=1,Type=String,Description=&amp;quot;Physical phasing ID information, where each unique ID within a given sample (but not across samples) connects records within a phasing group&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=PL,Number=G,Type=Integer,Description=&amp;quot;Normalized, Phred-scaled likelihoods for genotypes as defined in the VCF specification&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=RGQ,Number=1,Type=Integer,Description=&amp;quot;Unconditional reference genotype confidence, encoded as a phred quality -10*log10 p(genotype call is wrong)&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=SB,Number=4,Type=Integer,Description=&amp;quot;Per-sample component statistics which comprise the Fisher's Exact Test to detect strand bias.&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=AC,Number=A,Type=Integer,Description=&amp;quot;Allele count in genotypes, for each ALT allele, in the same order as listed&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=AF,Number=A,Type=Float,Description=&amp;quot;Allele Frequency, for each ALT allele, in the same order as listed&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=AN,Number=1,Type=Integer,Description=&amp;quot;Total number of alleles in called genotypes&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=BaseQRankSum,Number=1,Type=Float,Description=&amp;quot;Z-score from Wilcoxon rank sum test of Alt Vs. Ref base qualities&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=ClippingRankSum,Number=1,Type=Float,Description=&amp;quot;Z-score From Wilcoxon rank sum test of Alt vs. Ref number of hard clipped bases&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=DP,Number=1,Type=Integer,Description=&amp;quot;Approximate read depth; some reads may have been filtered&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=DS,Number=0,Type=Flag,Description=&amp;quot;Were any of the samples downsampled?&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=ExcessHet,Number=1,Type=Float,Description=&amp;quot;Phred-scaled p-value for exact test of excess heterozygosity&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=FS,Number=1,Type=Float,Description=&amp;quot;Phred-scaled p-value using Fisher's exact test to detect strand bias&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=HaplotypeScore,Number=1,Type=Float,Description=&amp;quot;Consistency of the site with at most two segregating haplotypes&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=InbreedingCoeff,Number=1,Type=Float,Description=&amp;quot;Inbreeding coefficient as estimated from the genotype likelihoods per-sample when compared against the Hardy-Weinberg expectation&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=MLEAC,Number=A,Type=Integer,Description=&amp;quot;Maximum likelihood expectation (MLE) for the allele counts (not necessarily the same as the AC), for each ALT allele, in the same order as listed&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=MLEAF,Number=A,Type=Float,Description=&amp;quot;Maximum likelihood expectation (MLE) for the allele frequency (not necessarily the same as the AF), for each ALT allele, in the same order as listed&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=MQ,Number=1,Type=Float,Description=&amp;quot;RMS Mapping Quality&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=MQRankSum,Number=1,Type=Float,Description=&amp;quot;Z-score From Wilcoxon rank sum test of Alt vs. Ref read mapping qualities&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=QD,Number=1,Type=Float,Description=&amp;quot;Variant Confidence/Quality by Depth&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=RAW_MQ,Number=1,Type=Float,Description=&amp;quot;Raw data for RMS Mapping Quality&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=ReadPosRankSum,Number=1,Type=Float,Description=&amp;quot;Z-score from Wilcoxon rank sum test of Alt vs. Ref read position bias&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=SOR,Number=1,Type=Float,Description=&amp;quot;Symmetric Odds Ratio of 2x2 contingency table to detect strand bias&amp;quot;&amp;gt;&#xA;##reference=file:///Users/sian_bray/Dropbox/Yant/Cochlearia/02_Data/References/pseudohap_cochlearia_135M_3kb.fasta&#xA;##source=SelectVariants&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  ALO_006 ALO_007 ALO_013 ALO_017 BEA_002 BEA_004 BEA_010 BRI_002 BRI_005 BRI_006 BRI_009 ELI_001 ELI_002 ELI_003 ELI_004 FRE_013 JON_001 KVA_002 KVA_003 KVA_009 KVA_010 LAB_004 LAB_300 LAB_400 LAB_500 LNL_001 LNL_002 LNL_003 LNL_008 MEL_001 MEL_002 NEN_001 NEN_003 NEN_200 NEN_300 ROT_004 ROT_006 ROT_007 ROT_013 RUZ_001 SAL_003 SAL_004 SAL_006 SKF_002 SKF_003 SKF_005 SKF_009 SKI_004 SKI_005 SKI_008 SKN_001 SKN_002 SKN_005 SKN_008 SPU_006 SPU_008 SPU_009 SPU_010 TET_002 TET_004 TET_006 TET_008 TRO_001 TRO_003 TRO_005 TRO_009 VAG_003 VAG_004 VAG_007 VAG_102 VEG_003 VEG_004 WOL_002 WOL_006 WOL_009 WOL_010&#xA;237 50041   .   G   T   4099.26 PASS    AC=35;AF=0.133;AN=264;BaseQRankSum=0.00;ClippingRankSum=0.00;DP=1228;ExcessHet=4.3562;FS=0.644;InbreedingCoeff=-0.0953;MQ=57.84;MQRankSum=0.00;QD=25.94;ReadPosRankSum=-1.570e-01;SOR=0.768 GT:AD:DP:GQ:PL  0/0/0/0:11,0:11:14:0,14,33,66,425   0/0/0/0:11,1:12:0:0,0,0,27,369  0/0/0/0:13,3:16:0:0,0,0,0,355   0/0/0/0:41,0:41:50:0,50,120,241,1800    0/0/1/1:5,8:13:1:293,15,0,1,170 0/0/0/1:5,3:8:0:104,0,0,10,191  0/0/0/0:8,0:8:10:0,10,24,48,272 0/1/1/1:1,8:9:11:318,33,11,0,26 ./././.:0,0 1/1/1/1:0,2:2:2:87,12,6,2,0 0/0/0/0:5,2:7:0:0,0,0,0,120 0/0/0/0:16,2:18:0:0,0,0,21,515  0/0/0/0:8,1:9:0:0,0,0,13,270    0/0/0/0:15,3:18:0:0,0,0,0,459   0/0/0/0:10,1:11:0:0,0,0,21,363  0/0/0/0/0/0:15,0:15:12:0,12,26,45,71,116,589    0/0/0/0/0/0:7,2:9:0:0,0,0,0,0,0,151 ./././.:0,0 0/0/0/0:14,0:14:14:0,14,33,66,495   0/0/0/0:1,0:1:1:0,1,3,6,42  0/0/0/0:9,0:9:10:0,10,24,48,360 0/0:21,2:23:14:0,14,726 0/0:14,2:16:11:0,11,452 0/0:14,2:16:0:0,0,467   0/0:23,9:32:0:0,0,527   0/0/0/0:10,0:10:12:0,12,30,60,359   0/0/0/0:11,1:12:0:0,0,0,27,404  0/0/0/0:13,0:13:16:0,16,39,78,526   0/0/0/0:20,3:23:0:0,0,0,2,655   1/1/1/1:0,4:4:5:155,24,12,5,0   0/0/0/0:2,3:5:0:0,0,0,0,8   0/0:23,2:25:0:0,0,798   0/0:12,2:14:0:0,0,381   0/0:14,0:14:36:0,36,540 0/0:24,0:24:63:0,63,945 0/0/0/0:8,1:9:0:0,0,0,12,245    0/0/0/0:27,0:27:32:0,32,78,157,1170 0/0/0/0:18,5:23:0:0,0,0,0,474   0/0/0/0:22,1:23:15:0,15,50,114,736  0/0:10,0:10:27:0,27,405 0/0:8,0:8:24:0,24,322   0/0:13,0:13:36:0,36,540 0/0:4,0:4:12:0,12,155   0/0/0/0/0/0:5,2:7:0:0,0,0,0,0,0,125 0/0/0/0/0/0:19,1:20:0:0,0,0,18,50,106,731   0/0/0/0/0/0:28,3:31:0:0,0,0,0,1,82,978  0/0/0/0/0/0:53,10:63:0:0,0,0,0,0,0,1843 0/0/0/0:1,1:2:0:0,0,0,0,4   1/1/1/1:0,22:22:27:917,132,66,27,0  ./././.:0,0 0/0/0/0:15,0:15:17:0,17,42,84,630   0/0/0/0:14,2:16:0:0,0,0,27,421  0/0/0/0:17,0:17:21:0,21,51,102,670  0/0/0/0:17,7:24:0:0,0,0,0,323   0/0:11,2:13:0:0,0,380&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9269" LastEditorUserId="73" LastEditDate="2022-01-23T12:59:27.850" LastActivityDate="2022-01-23T13:00:13.923" Title="How to process multiple VCF files present in a 2 folders and create afd plots" Tags="|python|vcf|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18333" PostTypeId="2" ParentId="18329" CreationDate="2022-01-15T23:06:00.910" Score="1" Body="&lt;p&gt;Not an expert of WDL but looking at &lt;a href=&quot;https://cromwell.readthedocs.io/en/stable/tutorials/Containers/#configuration-in-detail&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cromwell documentation&lt;/a&gt;, there is a block in the configuration of &lt;code&gt;Cromwell&lt;/code&gt; called &lt;code&gt;submit-docker&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;you'll need to modify the config block&#xA;inside backend.providers in your Cromwell configuration. In&#xA;particular, this block contains a key called submit-docker&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In this &lt;a href=&quot;https://github.com/broadinstitute/cromwell/blob/develop/cromwell.example.backends/cromwell.examples.conf#L420&quot; rel=&quot;nofollow noreferrer&quot;&gt;example file&lt;/a&gt;, on line &lt;code&gt;420&lt;/code&gt;, it is possible to specify the mount option which is &lt;code&gt;-v /path/you/need/to/mount:/path/in/docker/container&lt;/code&gt; (usually they are the same) to Docker command line. Setting it like that should work.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2022-01-15T23:06:00.910" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18334" PostTypeId="2" ParentId="18331" CreationDate="2022-01-15T23:52:21.943" Score="2" Body="&lt;p&gt;Assuming what I said in the comments is true. Here's my attempt, there may be an easier solution but this should do it:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;import re&#xA;&#xA;# a function to return the count number of animals in the fasta in a dictionary&#xA;def count_animals(file_path, separator, animal_index):&#xA;  animal_dictionary={}&#xA;  with open(&amp;quot;Proof.txt&amp;quot;) as fasta_file:&#xA;    fasta_contents = fasta_file.readlines()&#xA;    for line in fasta_contents:&#xA;      if line.startswith(&amp;quot;&amp;gt;&amp;quot;):&#xA;        animal = line.split(separator)[animal_index]&#xA;        if animal not in animal_dictionary:&#xA;          animal_dictionary[animal] = 1&#xA;        else:&#xA;          animal_dictionary[animal] = animal_dictionary[animal] + 1&#xA;  return animal_dictionary&#xA;&#xA;def read_fasta(file_path, columns) :&#xA;  counter = 0  # counter used to print no more than number of times 'Dog'&#xA;  animal_d = count_animals(file_path, '|', 1) # execute function&#xA;  max_animal =  max(animal_d.values()) # get max number of animals&#xA;  min_animal =  min(animal_d.values()) # get min number of animals&#xA;&#xA;  from Bio.SeqIO.FastaIO import SimpleFastaParser &#xA;  with open(&amp;quot;Proof.txt&amp;quot;) as fasta_file :&#xA;    records = [] # create empty list&#xA;    for title, sequence in SimpleFastaParser(fasta_file): #SimpleFastaParser Iterate over Fasta records as string tuples. For each record a tuple of two strings is returned, the FASTA title line (without the leading ‘&amp;gt;’ character), and the sequence (with any whitespace removed). &#xA;      record = []&#xA;      title_splits=re.findall(r&amp;quot;[\w']+&amp;quot;, title) # Data cleaning is needed&#xA;&#xA;      record.append(title_splits[0])  #First values are ID (Append adds element to a list)&#xA;      record.append(len(sequence)) #Second values are sequences lengths&#xA;      sequence = &amp;quot; &amp;quot;.join(sequence) #It converts into one line&#xA;      record.append(sequence)#Third values are sequences&#xA;      # if 'Human' exists in title and the counter is less than min_animal('Dog'), add the record&#xA;      if 'Human' in title and counter &amp;lt; min_animal:&#xA;        counter += 1 # add +1 to counter&#xA;&#xA;        #Fourth column will contain the species&#xA;        record.append(&amp;quot;Human&amp;quot;)  &#xA;          &#xA;        records.append(record)&#xA;      elif 'Dog' in title:&#xA;        record.append(&amp;quot;Dog&amp;quot;)  &#xA;          &#xA;        records.append(record)&#xA;&#xA;    # print(records)&#xA;  return pd.DataFrame(records, columns = columns) #We have created a function that returns a dataframe&#xA;&#xA;#Now let's use this function by inserting in the first argument the file name (or file path if your working directory is different from where the fasta file is)    &#xA;#And in the second one the names of columns&#xA;data = read_fasta(&amp;quot;Proof.txt&amp;quot;, columns=[&amp;quot;id&amp;quot;,&amp;quot;sequence_length&amp;quot;, &amp;quot;sequence&amp;quot;, &amp;quot;animal&amp;quot;])&#xA;&#xA;print(data)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Should give:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;      id  sequence_length                               sequence animal&#xA;0  Numer               19  H S S F I E I V N I E H V I E H I V K  Human&#xA;1  Numer               19  H S S F I E I V N I E H V I E H I V K  Human&#xA;2  Numer               19  H S S F I E I V N I E H V I E H I V K  Human&#xA;3  Numer               19  H S S F I E I V N I E H V I E H I V K  Human&#xA;4  Numer               19  H S S F I E I V N I E H V I E H I V K    Dog&#xA;5  Numer               19  H S S F I E I V N I E H V I E H I V K    Dog&#xA;6  Numer               19  H S S F I E I V N I E H V I E H I V K    Dog&#xA;7  Numer               19  H S S F I E I V N I E H V I E H I V K    Dog&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2022-01-15T23:52:21.943" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18335" PostTypeId="2" ParentId="18328" CreationDate="2022-01-16T00:35:09.903" Score="0" Body="&lt;p&gt;Assuming that the third line is the continuation of the second line. Here's a one (long) liner:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;cat &amp;lt;( \&#xA;      paste -d '\t' \&#xA;         &amp;lt;(echo &amp;quot;Id&amp;quot;) \&#xA;         &amp;lt;(head -1 test.tsv \&#xA;      | cut -f5- \&#xA;      | sed -e '$s/\[[[:digit:]]\+\]//g; s/_HG[[:digit:]]\+//g') ) \&#xA;    &amp;lt;( \&#xA;      paste -d '\t' \&#xA;        &amp;lt;(awk 'NR&amp;gt;1 {print $1&amp;quot;_&amp;quot;$2&amp;quot;_&amp;quot;$3&amp;quot;_&amp;quot;$4&amp;quot;_b37&amp;quot;}' test.tsv) \&#xA;        &amp;lt;(awk 'NR&amp;gt;1 {print $0}' test.tsv | cut -f5- | sed 's/\.[[:digit:]]\+//g')) \&#xA;&amp;gt; output.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2022-01-16T00:35:09.903" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18336" PostTypeId="1" CreationDate="2022-01-16T08:40:17.387" Score="2" ViewCount="107" Body="&lt;p&gt;I have a subset of high-quality MAGs from different environments, sequenced on different runs. I want to know, for any particular MAG, if it (or something similar to it) is present in any particular sample.&lt;/p&gt;&#xA;&lt;p&gt;I've recently started learning anvi'o for metagenomic visualization. One of its perks is that it can nicely represent coverages for either contigs or bins (tutorial example: &lt;a href=&quot;https://merenlab.org/images/miscellaneous/2017-05-11-anvi-refine-by-veronika/Image_1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://merenlab.org/images/miscellaneous/2017-05-11-anvi-refine-by-veronika/Image_1.png&lt;/a&gt; , the circles marked Sample 07 and 08).&#xA;&lt;img src=&quot;https://merenlab.org/images/miscellaneous/2017-05-11-anvi-refine-by-veronika/Image_1.png&quot; alt=&quot;Example_plot&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;The coverage options are defined as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Mean coverage&#xA;Average depth of coverage across contig.&#xA;Add up the coverage of each nucleotide in a contig, and divide by the length of the contig. This is the default view in most cases.&#xA;&#xA;Mean coverage Q2Q3&#xA;Average depth of coverage across contig excluding nucleotide positions with coverage values falling outside of the interquartile range for that contig.&#xA;Calculated the same as mean coverage, except only incorporating those nucleotide coverage values that fall within 2nd and 3rd quartiles of the distribution of nucleotide coverages for that contig. This can help smooth out the mean coverage visualization by removing nucleotide coverage values from the equation that may be outliers due to non-specific mapping.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Additional representation options:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Abundance&#xA;Mean coverage of a contig divided by that sample’s overall mean coverage.&#xA;Abundance for a contig is constrained to within one sample. In a sense this view is telling you that those contigs with larger abundance values are more represented in that sample (i.e. recruited more reads) than those contigs with smaller abundance values. And it does this by providing the ratio of that contig’s mean coverage to that sample’s overall mean coverage incorporating all contigs. So if your abundance value is 2, that contig’s mean coverage is twice that of the mean for all contigs in that sample.&#xA;&#xA;Relative abundance&#xA;Proportion of reads recruited to a contig out of the total reads recruited to that contig across all samples.&#xA;Relative abundance considers one contig across all samples. This will tell you in which sample a particular contig recruited the most reads. Since it is normalized to the total number of reads recruited to contig across all samples, values from all samples for a given contig will always sum to 1.&#xA;&#xA;Max-normalized ratio&#xA;Proportion of reads recruited to a contig out of the maximum number of reads recruited to that contig in any sample.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This also considers one contig across all samples. But in this case the value is normalized to the single maximum value for that contig (as opposed to the sum as in relative abundance). Because of this the sample containing the contig that contributed the max value will always equal 1, and the value for that contig in the other samples will be the fraction of that max.&lt;/p&gt;&#xA;&lt;p&gt;I'm trying to cross-map the reads between samples to the best MAGs (+within the same sample). The reason I'm trying to map to MAGs as opposed to genomes, as usual, is that some of my MAGs of interest represent genera (at the least) without available genomes, so it feels like I'd lose considerable information that way. But then, I'm learning this as I go along, so for all I know it might be a bad idea.&lt;/p&gt;&#xA;&lt;p&gt;From what I understood, these settings generally make sense / are used for samples that have been co-assembled. However, if I've been assembling samples independently and co-binning (due to wanting to avoid chimeras) or I want to compare some of  the metagenomic composition of different environments (which really wouldn't make sense to either co-assemble or co-bin), would it still make sense to use coverages as a proxy for presence / absence / abundance?&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your time!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastEditorUserId="73" LastEditDate="2022-01-23T12:50:22.993" LastActivityDate="2022-01-23T12:50:22.993" Title="Does it make sense to compare the mean relative coverages for contigs / MAGs from samples that were not co-assembled?" Tags="|visualization|metagenome|coverage|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18337" PostTypeId="2" ParentId="18330" CreationDate="2022-01-16T09:35:06.790" Score="2" Body="&lt;p&gt;Have you had a look at your alignment? I personally use Jalview but any alignment viewer should work. Colour &amp;gt; Percentage Identity (or Nucleotide) should help with visualization.&lt;/p&gt;&#xA;&lt;p&gt;Does the alignment look reasonable (and gaps are just inserted at the end for some reason), or is it pretty messy? If it looks messy, is it just a few outlier sequences that cause the problem, or does it look like you have two main groups of sequences that align between themselves but not with the other? In the latter case, could it be possible that some of your initial sequences were inputed in the reverse orientation? I'm not sure about the command line implementation of Clustal Omega, but I know that at least the versions I use don't automatically reverse-complement sequences to be aligned.&lt;/p&gt;&#xA;" OwnerUserId="4401" LastEditorUserId="3848" LastEditDate="2022-01-18T15:32:38.640" LastActivityDate="2022-01-18T15:32:38.640" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18338" PostTypeId="2" ParentId="18327" CreationDate="2022-01-16T12:06:05.853" Score="1" Body="&lt;p&gt;When i was still working in the field of genotype imputation, a dosage was the expectation (average) of the (posterior) genotype probabilities, coded as GP in the OP's VCF. I once contributed code to bcftools to convert genotype probabilities stored in the GP field to dosages. It looks like that code may now live in a bcftools plugin called &amp;quot;dosage&amp;quot;: &lt;a href=&quot;https://samtools.github.io/bcftools/howtos/plugins.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://samtools.github.io/bcftools/howtos/plugins.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2022-01-16T12:06:05.853" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18339" PostTypeId="1" CreationDate="2022-01-16T18:10:55.747" Score="1" ViewCount="23" Body="&lt;p&gt;Gene Ontology provides many genes annotated as taking part in various morphogenesis processes but I want to get a list of all morphogen coding genes specifically. Uniprot  does not have &amp;quot;morphogen&amp;quot; keyword.&lt;/p&gt;&#xA;&lt;p&gt;I need some gene annotation database that has genes labeled as &amp;quot;morphogen&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="4971" LastActivityDate="2022-01-16T18:10:55.747" Title="How to get a list of all human morphogens?" Tags="|gene|database|sequence-annotation|ontology|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18340" PostTypeId="1" AcceptedAnswerId="18350" CreationDate="2022-01-17T04:26:36.523" Score="2" ViewCount="2662" Body="&lt;p&gt;I am trying to understand the source code of the &lt;a href=&quot;https://vina.scripps.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AutoDock Vina&lt;/a&gt;, but as I am new to the field of Bioinformatics it is a little hard to understand. It seems like they are using a tree data structure to represent the ligand and the receptor. I cannot understand what nodes and branches are in this data structure (&lt;a href=&quot;https://github.com/ccsb-scripps/AutoDock-Vina/blob/ea68a7abd8d90d2f8ab296521c0d07ee828f9d46/src/lib/tree.h#L187-L220&quot; rel=&quot;nofollow noreferrer&quot;&gt;Heterotree&lt;/a&gt;). I think in order to understand it I should first understand what information the PDBQT file contains. Can you tell me what branch and root is in PDBQT file format?&lt;/p&gt;&#xA;" OwnerUserId="14875" LastEditorUserId="57" LastEditDate="2022-01-17T11:43:28.867" LastActivityDate="2022-01-18T10:17:43.107" Title="PDBQT file format" Tags="|proteins|pdb|docking|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18341" PostTypeId="1" AcceptedAnswerId="18393" CreationDate="2022-01-17T09:42:44.503" Score="0" ViewCount="53" Body="&lt;p&gt;What does it mean by &amp;quot;PC&amp;quot; in this article?&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/24995959/&quot; rel=&quot;nofollow noreferrer&quot;&gt;a fast and efficient Cα-based method for accurately assigning protein secondary structure elements&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/h7JCE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/h7JCE.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12686" LastActivityDate="2022-01-23T12:41:17.663" Title="What does it mean by &quot;PC&quot; in this article?" Tags="|protein-structure|algorithms|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18342" PostTypeId="1" AcceptedAnswerId="18345" CreationDate="2022-01-17T09:58:16.207" Score="1" ViewCount="62" Body="&lt;p&gt;In the following article:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/24995959/&quot; rel=&quot;nofollow noreferrer&quot;&gt;a fast and efficient Cα-based method for accurately assigning protein secondary structure elements&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;What are the minimum and maximum cutoff distances in this algorithm?&lt;/p&gt;&#xA;&lt;p&gt;Also, what does it mean by:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CWKAX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CWKAX.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I.e. where does (2x43)x3 come from?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="12686" LastEditDate="2022-01-17T10:13:07.747" LastActivityDate="2022-01-17T16:42:34.093" Title="What is the cutoff distance in this algorithm?" Tags="|proteins|algorithms|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18344" PostTypeId="1" AcceptedAnswerId="18366" CreationDate="2022-01-17T16:29:33.347" Score="2" ViewCount="141" Body="&lt;p&gt;So I know Consensus CDS has the genomic locations of exons, however, only of the exons in the CDS. Everything before or after is not included.&lt;/p&gt;&#xA;&lt;p&gt;NIH variation viewer has those (kinda-) radio buttons next to the gene name, and when you hover over them, you do get to see the genomic locations of the exons. But this method isn't very scalable.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know an easy, scalable way of retrieving the genomic locations of exon for a certain gene?&lt;/p&gt;&#xA;" OwnerUserId="14882" LastEditorUserId="57" LastEditDate="2022-01-17T18:45:32.893" LastActivityDate="2022-02-22T16:08:55.963" Title="Find the genomic locations of exons of certain human gene" Tags="|genome|human-genome|sequence-annotation|exon|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18345" PostTypeId="2" ParentId="18342" CreationDate="2022-01-17T16:42:34.093" Score="2" Body="&lt;p&gt;The 43 features are listed in the supplementary materials on page 4. For example,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Feature Type Atoms Involved &#xA;1 Distance i, i-5 &#xA;2 Distance i, i-4 &#xA;3 Distance i, i-3 &#xA;4 Distance i, i-2  &#xA;...&#xA;11 Angle i-1, i, i+1 &#xA;12 Dihedral i, i+1, i+2, i+3 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Are some of them; so Feature 11 is the angle between atom(i-1), atom(i), and atom(i+1). Since it says these were calculated for both the C-alphas and pseudo-centers, this gives 2 * 43. Then this is multiplied by three as mentioned in your quoted paragraph.&lt;/p&gt;&#xA;&lt;p&gt;These features (measurements, almost) are numbers, not booleans - so I do not think that they use distance cutoffs as such, although the random forest method is presumably learning various cutoffs and encodes those in the structure of the forest.&lt;/p&gt;&#xA;" OwnerUserId="745" LastActivityDate="2022-01-17T16:42:34.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18346" PostTypeId="2" ParentId="18331" CreationDate="2022-01-17T18:41:15.187" Score="1" Body="&lt;p&gt;An example of contructing a pandas dataframe for fasta on Stack Exchange is &lt;strong&gt;&lt;a href=&quot;https://stackoverflow.com/questions/69829981/how-to-convert-fasta-file-into-a-pandas-dataframe&quot;&gt;here&lt;/a&gt;&lt;/strong&gt; . This is clean code deploying &lt;code&gt;zip&lt;/code&gt; outside Biopython. Constructing a pandas dataframe directly via dictionary.&lt;/p&gt;&#xA;&lt;p&gt;Are you intending to use &lt;code&gt;split&lt;/code&gt; method directly on the sequence, such that each residue is a separate column in a dataframe?&lt;/p&gt;&#xA;&lt;p&gt;Anyway,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;title_splits=re.findall(r&amp;quot;[\w']+&amp;quot;, title) # Data cleaning is needed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is a bad idea because you will remove the Genbank unique identifier (its an integer) and this should not leave the sequence.&lt;/p&gt;&#xA;&lt;p&gt;You will need to switch out of the Fasta object. Your approacha and @user324810's use of&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sequence = &amp;quot; &amp;quot;.join(sequence)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;is okayish. However, I think it should be '', or &amp;quot;&amp;quot; (no space character). I spotted manually removing fasta, well the method should do this automatically. I am not familiar with Fastaparser, but AlignIO and SeqIO will both automatically process fasta without the need deal with &amp;quot;&amp;gt;&amp;quot;. Biopython will have a length method and whilst you have the Biopython object it is preferable to use Biopython (OOP) methods.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.SeqIO.FastaIO import SimpleFastaParser &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This should be at the top of the code.&lt;/p&gt;&#xA;&lt;p&gt;Overall, you manually open the data, pass it through the Bioparser then immediately dump the object. I do agree to write a pandas dataframe from a Biopython object these conversion to string, but Biopython will provide a method to print the object (pass it to a string) without all the parsing. Some of the parsing is removing key data (Genbank ID).&lt;/p&gt;&#xA;&lt;p&gt;Personally I would look at the Stackoverflow example and base your code around this.&lt;/p&gt;&#xA;&lt;p&gt;You clearly have an understanding of pandas, but I get the impression Biopython is something your not familiar with. My suspicion is that Biopython would infact answer the entirety of your question (whatever it might be) without passing to pandas.&lt;/p&gt;&#xA;&lt;p&gt;pandas is cool and powerful for non-molecular data, but unless your using machine/deep learning (where a dataframe is necessary input) I think Biopython will provide most of the answers.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Important Terminology&lt;/strong&gt;&#xA;The class is FastaIO and the method is SimpleFastaParser. Humans and dogs are instances of this method and are therefore object(s).&lt;/p&gt;&#xA;&lt;p&gt;Please don't write 'class' of dogs and humans in a formal report because it is incorrect OOP and will cause alarm if your reviewer is a coder. Your code is functional programming rather than OOP. You could say the human/dog fasta object, or just refer to it as human/dog dataframes and avoid any OOP terminology.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-01-18T00:21:25.653" LastActivityDate="2022-01-18T00:21:25.653" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18347" PostTypeId="1" AcceptedAnswerId="18349" CreationDate="2022-01-17T18:48:30.967" Score="0" ViewCount="665" Body="&lt;p&gt;Comparing read counts from an RNA-seq experiment for two select genes &lt;em&gt;before&lt;/em&gt; and &lt;em&gt;after&lt;/em&gt; using DESeq2's &lt;code&gt;collapseReplicates()&lt;/code&gt; and &lt;code&gt;plotCounts()&lt;/code&gt; functions yields interesting results:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Before &lt;code&gt;collapseReplicates()&lt;/code&gt; and &lt;code&gt;plotCounts()&lt;/code&gt;:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Geneid&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;foo1.1&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;foo1.2&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;foo2.1&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;foo2.2&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;bar1.1&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;bar1.2&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;bar2.1&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;bar2.2&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;baz1.1&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;baz1.2&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;baz2.1&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;baz2.2&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;baz3.1&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;baz3.2&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;WASH7P&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;6&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;5&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;2&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;8&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;5&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;SOX3&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1880&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1861&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1950&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;2055&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1189&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1181&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;2415&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;2482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;3887&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;3810&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1851&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1738&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;3217&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;3406&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;&lt;strong&gt;After &lt;code&gt;collapseReplicates()&lt;/code&gt; and &lt;code&gt;plotCounts()&lt;/code&gt;:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Geneid&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;foo1&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;foo2&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;bar1&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;bar2&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;baz1&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;baz2&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;baz3&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;WASH7P&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;9.877191&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;2.279384&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;3.478891&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;11.613875&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0.500000&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0.500000&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0.500000&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;SOX3&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;3189.598&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;3563.717&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;3530.486&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;4187.011&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;6473.122&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;7991.460&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;5390.721&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; In the above comparison, there are two (2) technical replicates (1.1 + 1.2, 2.1 + 2.2, etc.) for each biological replicate (foo1, foo2, etc.) for each of three (3) conditions (foo, bar, and baz).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Comparing the tables above, it appears as though DESeq2 is &lt;em&gt;NOT&lt;/em&gt; taking the average &lt;em&gt;or&lt;/em&gt; sum of columns being collapsed.&lt;/p&gt;&#xA;&lt;p&gt;It is also curious - and mildly concerning - that some very low expression (i.e., 0.5 counts) is reported for genes in the matrix of collapsed replicates when, in the original count matrix, zero (0) reads were counted as 'mapped to that gene'.&lt;/p&gt;&#xA;&lt;p&gt;So, how does &lt;code&gt;collapseReplicates()&lt;/code&gt; &amp;quot;combine counts into single columns of the count matrix&amp;quot; as is described in the &lt;a href=&quot;http://bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#collapsing-technical-replicates&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2 vignette&lt;/a&gt;?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Here is the code to collapse replicates and retrieve the number of read counts for a specific gene (e.g., &lt;em&gt;WASH7P&lt;/em&gt;) in a dataframe to be used in a &lt;a href=&quot;http://bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#plot-counts&quot; rel=&quot;nofollow noreferrer&quot;&gt;count plot&lt;/a&gt;, getting the &amp;quot;after&amp;quot; results shown above:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData = cts, colData = coldata, design = ~ condition)&#xA;ddsColl &amp;lt;- collapseReplicates(dds, dds&lt;span class=&quot;math-container&quot;&gt;$sample, dds$&lt;/span&gt;run)&#xA;keep &amp;lt;- rowSums(counts(ddsColl)) &amp;gt;= 10&#xA;dds &amp;lt;- ddsColl[keep,]&#xA;dds&lt;span class=&quot;math-container&quot;&gt;$condition &amp;lt;- relevel(dds$&lt;/span&gt;condition, ref = &amp;quot;baz&amp;quot;)&#xA;dds &amp;lt;- DESeq(dds)&#xA;countsdf &amp;lt;- plotCounts(dds, gene=&amp;quot;WASH7P&amp;quot;, intgroup=&amp;quot;condition&amp;quot;, returnData=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9007" LastEditorUserId="9007" LastEditDate="2022-02-24T19:17:19.153" LastActivityDate="2022-02-24T19:17:19.153" Title="How does DESeq2 &quot;collapseReplicates()&quot; function work on read counts data?" Tags="|rna-seq|deseq2|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18349" PostTypeId="2" ParentId="18347" CreationDate="2022-01-18T08:09:21.917" Score="1" Body="&lt;p&gt;The DESeq2 function &lt;code&gt;collapseReplicates&lt;/code&gt; sums the counts for the technical replicates. &lt;br&gt;Here is the code reference:&#xA;github.com/mikelove/DESeq2/blob/master/R/helper.R#L186 &lt;br&gt;&lt;/p&gt;&#xA;&lt;p&gt;OPs actual confusion was with the DESeq2 function &lt;code&gt;plotCounts&lt;/code&gt; which by default normalizes count data and adds a pseudocount of 0.5 for plotting on log2 scale.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2022-01-18T08:09:21.917" CommentCount="0" CommunityOwnedDate="2022-01-18T08:09:21.917" ContentLicense="CC BY-SA 4.0" />
  <row Id="18350" PostTypeId="2" ParentId="18340" CreationDate="2022-01-18T10:17:43.107" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;what information the PDBQT file contains&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The PDBQT format is described here: &lt;a href=&quot;https://autodock.scripps.edu/wp-content/uploads/sites/56/2021/10/AutoDock4.2.6_UserGuide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://autodock.scripps.edu/wp-content/uploads/sites/56/2021/10/AutoDock4.2.6_UserGuide.pdf&lt;/a&gt; —ignore the expired SSL certificate warning as I am pretty sure the Scripps Institute is not a criminal organisation.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;what branch and root is in PDBQT file format&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is rather self evident but with a twist and as you have guessed it has to do with trees in graph networks. Lines of text &amp;lt;-&amp;gt; graph networks describing a molecule is a common nuisance in compchemistry. Normally the aim is connectivity, but here it is torisons. But the former is a nice example, a small molecules can be represented with SMILES (strings), where say isopropanol is &lt;code&gt;CC(O)C&lt;/code&gt;, in which you have a branch going off in round brackets, whereas everything else is assumed connected sequentially. In the case of the PDBQT format you are describing what atom subsets of the molecule rotate together and actually not how they connect as given away by the fact that you don't have a special syntax for ring closures (in a SMILES you write cyclopropane as &lt;code&gt;C1CC1&lt;/code&gt;, where the third carbon connects to C1, which was declared first. This is because the &lt;strong&gt;topology&lt;/strong&gt; is dealt with elsewhere.&lt;/p&gt;&#xA;&lt;h3&gt;Resources&lt;/h3&gt;&#xA;&lt;p&gt;AutoDock 4 and AutoDock Vina are divergent pieces of software, but they have several common elements like the PDBQT format, so you get double the documentation (and papers to read)!&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2022-01-18T10:17:43.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18351" PostTypeId="1" AcceptedAnswerId="18355" CreationDate="2022-01-18T12:08:46.880" Score="2" ViewCount="1263" Body="&lt;p&gt;I have PBMCs of some patients who have recovered from cancer, and I want to look at their immune landscapes.&lt;/p&gt;&#xA;&lt;p&gt;I got confused when looking at available kits: can I use 5' end kits from 10x instead of 3' end based?&lt;/p&gt;&#xA;&lt;p&gt;I tried to search for information on this, but I am not clear if 5' can substitute for 3' or if they are just too different.&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="73" LastEditDate="2022-02-18T23:30:30.720" LastActivityDate="2023-01-27T10:59:16.820" Title="Can 5' end 10x be used instead of 3' end" Tags="|scrnaseq|single-cell|10x|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18352" PostTypeId="2" ParentId="18330" CreationDate="2022-01-18T15:30:52.653" Score="0" Body="&lt;p&gt;Firstly its good you've done some proofing of your output wherein you spotted,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;75000&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The alignment output is approximately double the genome size, just over double the input.&lt;/p&gt;&#xA;&lt;p&gt;An off-question point, I assume the computer you are working on has at least 8-cores because thats what you are requesting.&lt;/p&gt;&#xA;&lt;p&gt;Clustal omega is a very solid algorhithm and has been around for a long time in its current omega version. The error you are encountering is an input sequence bug. @Laura's suggest is correct and is formal practice: you must manually verify bioinformatics output and in this case an alignment viewer is the correct proceedure.&lt;/p&gt;&#xA;&lt;p&gt;Thereafter I suspect you would find the answer, such as omitting carriage returns in the fasta format, seeing taxa names caught up post 30K.&lt;/p&gt;&#xA;&lt;p&gt;If you run,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;wc ./myclustalv3.fasta # output file&#xA;wc ./Allseq.fasta  # input file&#xA;wc ./testgenome.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You will probably find that the number lines per file does not agree between the input and output format OR the number of lines of both input and output is &amp;lt;&amp;lt;600 or &amp;lt;&amp;lt;1200 (600 x2 ), i.e. the expected number of sequences. This points to parsing errors within the input data. Fasta has a line for the sequence identifier and a line for the sequence. It is using &amp;gt; for the sequence identifier, but will use a carriage return for sequence.&lt;/p&gt;&#xA;&lt;p&gt;Again manually assessing an alignment is baseline proceedure regardless of output.&lt;/p&gt;&#xA;&lt;p&gt;An example (guess) about what might have happened is that every other input has omitted &amp;quot;&amp;gt;&amp;quot;, omega will then see the first &amp;quot;&amp;gt;&amp;quot; then read everything until it sees another &amp;quot;&amp;gt;&amp;quot; as sequence data. Thus its doubled the sequence length and dragged a sequence identifier exactly in the centre of the doubled up genomes.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;An issue with fasta is there is no internal proofing of the format. In other formats the number of sequences and their length is specified within the file, so it is easy to internally verfiy the data prior a calculation. Code-wise it is known as 'setting' and you can't do that with fasta.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-01-19T19:24:59.223" LastActivityDate="2022-01-19T19:24:59.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18353" PostTypeId="2" ParentId="2637" CreationDate="2022-01-18T16:02:07.133" Score="1" Body="&lt;p&gt;You can try a hierarchical clustering tool HGC (hierarchical graph clustering).&lt;/p&gt;&#xA;&lt;p&gt;You may get cell population at different hierarchy by choosing multiple cutting heights. See &lt;a href=&quot;https://academic.oup.com/bioinformatics/article-abstract/37/21/3964/6294402?redirectedFrom=fulltext&quot; rel=&quot;nofollow noreferrer&quot;&gt;the paper&lt;/a&gt; for more details.&lt;/p&gt;&#xA;" OwnerUserId="2471" LastEditorUserId="57" LastEditDate="2022-01-19T10:02:06.957" LastActivityDate="2022-01-19T10:02:06.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18354" PostTypeId="2" ParentId="18313" CreationDate="2022-01-18T17:14:47.520" Score="1" Body="&lt;p&gt;Since you haven't shared any data or code so I will show it using the example given below,&lt;/p&gt;&#xA;&lt;p&gt;First you need to assign a specific cluster to the cells which are having another type like C_new.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cells_df &amp;lt;- data.frame(row.names=paste0(&amp;quot;Cell&amp;quot;, 1:10, sep = &amp;quot;&amp;quot;),as.vector(paste0(&amp;quot;C&amp;quot;,1:10,sep=&amp;quot;&amp;quot;)))&#xA;&#xA;colnames(cells_df) &amp;lt;- &amp;quot;clusters&amp;quot;&#xA;&#xA;&#xA;cluster_new &amp;lt;- c(&amp;quot;Cell6&amp;quot;,&amp;quot;Cell2&amp;quot;)&#xA;&#xA;&#xA;cells_df&lt;span class=&quot;math-container&quot;&gt;$clusters &amp;lt;- as.character(cells_df$&lt;/span&gt;clusters)&#xA;&#xA;for(i in 1:length(cluster_new)) {&#xA;j &amp;lt;- cluster_new[i] &#xA;cells_df[j,] &amp;lt;- &amp;quot;C_new&amp;quot;&#xA;}&#xA;&#xA;head(cells_df)&#xA;         clusters&#xA; Cell1       C1&#xA; Cell2    C_new&#xA; Cell3       C3&#xA; Cell4       C4&#xA; Cell5       C5&#xA; Cell6    C_new&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And if you have already assigned new clusters to the cells then you can skip the above step. Now save the new cluster belongings into the seurat object using the &lt;code&gt;Idents&lt;/code&gt; function&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Idents(seurat_obj) &amp;lt;- cells_df$clusters&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then you can perform the differential gene expression as follows, for example if you want to find the differentially expressed genes between cluster &lt;code&gt;C7&lt;/code&gt; vs &lt;code&gt;C_new&lt;/code&gt; you can do it as follows,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FindMarkers(seurat_obj,ident.1 = &amp;quot;C7&amp;quot;, ident.2 = &amp;quot;C_new&amp;quot; )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1194" LastActivityDate="2022-01-18T17:14:47.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18355" PostTypeId="2" ParentId="18351" CreationDate="2022-01-18T19:45:08.060" Score="3" Body="&lt;p&gt;I'm not entirely clear on your question, but if it is about the 10X 3' vs. 5' solutions then either can be used to sequence PBMCs. The principle difference is that 3' captures the 3' end of the transcript and 5' captures the 5' end. The differences are more important for multi-modal single cell data, e.g. only the 5' can be combined with VDJ sequencing and for CITE-seq, different antibody panels are compatible with each tech. There are also some more subtle differences and technical nuances.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2022-01-18T19:45:08.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18356" PostTypeId="1" CreationDate="2022-01-18T22:15:13.707" Score="2" ViewCount="3231" Body="&lt;p&gt;I am getting the following error message when I tried to load .TXT data file to GSEA 4.2.1.  I tried different way to change the file name as .GCT but not working.  Any suggestion? Many thanks.&lt;/p&gt;&#xA;&lt;p&gt;Full Error Message:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;There were errors: ERROR(S) #:1&#xA;Parsing trouble&#xA;java.lang.NumberFormatException: ...&#xA;&#xA;---- Stack Trace ----&#xA;# of exceptions: 1&#xA;------For input string: &amp;quot;description&amp;quot;------&#xA;java.lang.NumberFormatException: For input string: &amp;quot;description&amp;quot;&#xA;    at java.base/jdk.internal.math.FloatingDecimal.readJavaFormatString(Unknown Source)&#xA;    at java.base/jdk.internal.math.FloatingDecimal.parseFloat(Unknown Source)&#xA;    at java.base/java.lang.Float.parseFloat(Unknown Source)&#xA;    at org.gsea_msigdb.gsea/edu.mit.broad.genome.parsers.AbstractParser.parseStringToFloat(AbstractParser.java:266)&#xA;    at org.gsea_msigdb.gsea/edu.mit.broad.genome.parsers.AbstractParser.parseFieldsIntoFloatArray(AbstractParser.java:371)&#xA;    at org.gsea_msigdb.gsea/edu.mit.broad.genome.parsers.AbstractParser.parseTextMatrixToDataset(AbstractParser.java:293)&#xA;    at org.gsea_msigdb.gsea/edu.mit.broad.genome.parsers.TxtDatasetParser.parse(TxtDatasetParser.java:109)&#xA;    at org.gsea_msigdb.gsea/edu.mit.broad.genome.parsers.ParserFactory.readDatasetTXT(ParserFactory.java:201)&#xA;    at org.gsea_msigdb.gsea/edu.mit.broad.genome.parsers.ParserFactory.read(ParserFactory.java:760)&#xA;    at org.gsea_msigdb.gsea/edu.mit.broad.genome.parsers.ParserFactory.read(ParserFactory.java:737)&#xA;    at org.gsea_msigdb.gsea/edu.mit.broad.genome.parsers.ParserWorker.doInBackground(ParserWorker.java:52)&#xA;    at java.desktop/javax.swing.SwingWorker&lt;span class=&quot;math-container&quot;&gt;$1.call(Unknown Source)&#xA;    at java.base/java.util.concurrent.FutureTask.run(Unknown Source)&#xA;    at java.desktop/javax.swing.SwingWorker.run(Unknown Source)&#xA;    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)&#xA;    at java.base/java.util.concurrent.ThreadPoolExecutor$&lt;/span&gt;Worker.run(Unknown Source)&#xA;    at java.base/java.lang.Thread.run(Unknown Source)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14887" LastEditorUserId="57" LastEditDate="2022-01-19T09:57:22.753" LastActivityDate="2024-01-11T14:11:59.117" Title="#1 Parsing trouble java.lang.NumberFormatException in GSEA 4.2.1" Tags="|gsea|parsing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18357" PostTypeId="2" ParentId="18356" CreationDate="2022-01-18T22:43:08.970" Score="2" Body="&lt;p&gt;I just figured out the error. My .TXT was not correctly formated, following the instruction from &lt;a href=&quot;https://software.broadinstitute.org/cancer/software/gsea/wiki/index.php/Data_formats#Expression_Data_Formats&quot; rel=&quot;nofollow noreferrer&quot;&gt;their manual&lt;/a&gt; under .TXT format worked.&lt;/p&gt;&#xA;" OwnerUserId="14887" LastEditorUserId="57" LastEditDate="2022-01-19T10:00:24.590" LastActivityDate="2022-01-19T10:00:24.590" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18359" PostTypeId="1" AcceptedAnswerId="18361" CreationDate="2022-01-19T11:01:18.263" Score="3" ViewCount="1190" Body="&lt;p&gt;I have made a heatmap using RPKM values from a RNA-Seq dataset using the pheatmap() function in R. I have log2-transformed the data before performing Z-score standardisation of the data. I have also clustered the rows and columns of the heatmap. I have the following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;heatmap_data_log2 %&amp;gt;% pheatmap(color = colorRampPalette(c(&amp;quot;blue2&amp;quot;,&amp;quot;white&amp;quot;,&amp;quot;red&amp;quot;))(100), scale = &amp;quot;column&amp;quot;, cluster_cols = T, clustering_method = &amp;quot;ward.D2&amp;quot;, angle_col = 45, fontsize_row = 7.5, fontsize_col = 8, border_color = NA, cutree_rows = 4)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have seen in a &lt;a href=&quot;https://www.youtube.com/watch?v=pTeTH9bz-_s&amp;amp;list=LL&amp;amp;index=14&amp;amp;t=714s&amp;amp;ab_channel=BabrahamBioinf&quot; rel=&quot;nofollow noreferrer&quot;&gt;video tutorial&lt;/a&gt; that the pheatmap() function does the Z-score scaling before doing the clustering of the rows and columns. However, in this &lt;a href=&quot;https://hbctraining.github.io/GCC-BOSC-2018/lessons/data_visualization.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;online article&lt;/a&gt;, it says that &lt;em&gt;&amp;quot;The Z-scores are computed after the clustering, so that it only affects the graphical aesthetics and the color visualization is improved&amp;quot;&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;As these two sources are giving contradictory information, I was wondering which is better. Should you do Z-score scaling of the gene expression values before doing the clustering or after? Any advice is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="2531" LastEditorUserId="2531" LastEditDate="2022-02-09T09:56:04.743" LastActivityDate="2022-02-09T09:56:04.743" Title="In a RNA-Seq heatmap should you do Z-score standardisation before clustering the rows/columns or after?" Tags="|r|heatmap|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18360" PostTypeId="2" ParentId="18313" CreationDate="2022-01-19T12:36:08.987" Score="1" Body="&lt;p&gt;Assuming &lt;code&gt;parentObj&lt;/code&gt; is the original seurat object, and &lt;code&gt;tmp_subset&lt;/code&gt; is your subset, you can&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;new_metadata &amp;lt;- parentObj@meta.data &#xA;tmp_subset_annotation &amp;lt;- tmp_subset@meta.data &#xA;&#xA;# new_metadata&#xA;           original_annotation&#xA;ATTCGGA-1   TCD4&#xA;AAATAAA-1   TCD4-naive&#xA;...&#xA;&#xA;# tmp_subset_annotation&#xA;           new_cellType&#xA;ATTCGGA-1   TCD4-naive&#xA;AAATAAA-1   TCD4-naive&#xA;...&#xA;&#xA;new_metadata$refined_annotation &amp;lt;- sapply(rownames(new_metadata), &#xA;     function(ita) ifelse(ita %in% rownames(tmp_subset_annotation),&#xA;            tmp_subset_annotation[match(ita,rownames(tmp_subset_annotation)),&amp;quot;new_cellType&amp;quot;],&#xA;            new_metadata[match(ita,rownames(new_metadata)),&amp;quot;original_annotation&amp;quot;]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Basically you iterate each rownames of the original object, and assign the new annotation, if the UMI matches a UMI in the subset, otherwise you keep the original annotation.&lt;/p&gt;&#xA;" OwnerUserId="3631" LastEditorUserId="650" LastEditDate="2023-06-13T15:09:33.190" LastActivityDate="2023-06-13T15:09:33.190" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18361" PostTypeId="2" ParentId="18359" CreationDate="2022-01-19T12:45:43.193" Score="3" Body="&lt;p&gt;Looking at the &lt;a href=&quot;https://rdrr.io/cran/pheatmap/src/R/pheatmap.r&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt; for &lt;code&gt;pheatmap&lt;/code&gt;, there is a function called &lt;code&gt;scale_mat&lt;/code&gt; that is used to preprocess and normalize the input matrix, depending on the value of &lt;code&gt;scale&lt;/code&gt;, which specifies one of either none, row, or column normalization options. Separate and similar functions are used for color value scaling downstream.&lt;/p&gt;&#xA;&lt;p&gt;I have not watched the video or read the linked article, so I'm not sure this is contradictory, but it may be just small confusion about how normalization is applied (and to what).&lt;/p&gt;&#xA;&lt;p&gt;Normalization is usually done before clustering to focus cluster construction on signal rows or columns that are not some multiple or scale factor of another — and in referring back to the source code, this is what &lt;code&gt;pheatmap&lt;/code&gt; does, as well.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2022-01-19T20:43:38.077" LastActivityDate="2022-01-19T20:43:38.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18363" PostTypeId="1" AcceptedAnswerId="18364" CreationDate="2022-01-19T22:59:12.910" Score="2" ViewCount="440" Body="&lt;p&gt;I would like to add the SRA toolkit to my path under macOS 12.1.&lt;/p&gt;&#xA;&lt;p&gt;After downloading the SRA directory, decompressing, and adding to the path, the directory does not seem to be available e.g. I'm attempting to add it to my path with&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;export path=(/Users/ray/Documents/School/bioinformatics/sratoolkit.2.11.2-mac64/bin)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and it does not work. For example if I type&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; `ray@dynamic-xx-xxx-xxx-xx ~ % which prefetch&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;prefetch not found` macOS gives me prefetch not found.  The result of my path is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;`ray@dynamic-10-125-206-18 ~ % echo $PATH&#xA;/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin` &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there a way to add it to my path so it works?&lt;/p&gt;&#xA;" OwnerUserId="14894" LastEditorUserId="3848" LastEditDate="2022-01-20T01:03:10.153" LastActivityDate="2022-01-20T01:05:54.527" Title="How do I set my path for the SRA toolkit under macOS 12.1?" Tags="|sratoolkit|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18364" PostTypeId="2" ParentId="18363" CreationDate="2022-01-20T00:35:52.723" Score="0" Body="&lt;p&gt;Firstly, you are trying to set a path into the Documents directory? Thats like for documents.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Anyway there could be a permission issue. The quickest way is simply set a path within .bashrc or .profile whatever into ~/bin&#xA;and then use&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ln -si /Users/ray/Documents/School/bioinformatics/sratoolkit.2.11.2-mac64/bin/prefetch ~/bin&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... for each compiled program.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ prefetch &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;should then work.&#xA;This creates a symbolic link in &lt;code&gt;~/bin&lt;/code&gt; so when you type the program in your commandline prompt it will fire, without wiring it via a custom bin. If there is a permission issue a permission error will be quickly flagged.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The best way is to use Anaconda because its within the &lt;a href=&quot;https://anaconda.org/bioconda/sra-tools&quot; rel=&quot;nofollow noreferrer&quot;&gt;Anaconda collection&lt;/a&gt; and simplifies everything on three counts, viz:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;in answer to your question all configurations are automated&lt;/li&gt;&#xA;&lt;li&gt;it is easy to automate all updates&lt;/li&gt;&#xA;&lt;li&gt;it offers a virtual environment where code can't mess with your system&lt;/li&gt;&#xA;&lt;li&gt;whenever you want to download other packages its the same approach&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Thus all you have to do is download and install &lt;a href=&quot;https://www.anaconda.com/products/individual&quot; rel=&quot;nofollow noreferrer&quot;&gt;Anaconda&lt;/a&gt; or &lt;a href=&quot;https://docs.conda.io/en/latest/miniconda.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;miniconda&lt;/a&gt;. From this point everything is done using conda, either:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c bioconda sra-tools&#xA;conda install -c bioconda/label/cf201901 sra-tools&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If one fails the other one will work. Then simply,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda activate &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;SRA is now fully configured and ready to go. To get out of the conda virtual environment,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda deactivate &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you use Anaconda alot it is better to make a specific environment for SRA and not use the core environment, but thats another question.&lt;/p&gt;&#xA;&lt;p&gt;Anaconda installs in two different locations depending on whether you install the GUI version, which makes maintenance and environments a bit easier. In the later case I think its &lt;code&gt;/opt&lt;/code&gt; (but forget), otherwise the commandline version is &lt;code&gt;~/anadonda3&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I see your using a virtual environment via &lt;code&gt;homebrew&lt;/code&gt;. Anaconda is separate, its the same idea, but its better.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-01-20T01:05:54.527" LastActivityDate="2022-01-20T01:05:54.527" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18366" PostTypeId="2" ParentId="18344" CreationDate="2022-01-20T03:32:20.973" Score="2" Body="&lt;p&gt;Perhaps you could use this (assuming &lt;code&gt;hg38&lt;/code&gt;) as a lookup table:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ wget -qO- https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_39/gencode.v39.annotation.gff3.gz \&#xA;    | gunzip -c \&#xA;    | grep -wF exon \&#xA;    | convert2bed --input=gff --attribute-key=&amp;quot;gene_name&amp;quot; - \&#xA;    &amp;gt; gencode.v39.exons.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you prefer to search for an Ensembl accession ID, you can remove the &lt;code&gt;--attribute-key=&amp;quot;gene_name&amp;quot;&lt;/code&gt; option. Otherwise, this puts the HGNC symbol into the ID column.&lt;/p&gt;&#xA;&lt;p&gt;The preceding step takes about 30 seconds. Once you have made this lookup file, you can search it for exons of any gene of interest. Using &lt;code&gt;MYC&lt;/code&gt; as an example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ awk -v FS=&amp;quot;\t&amp;quot; -v OFS=&amp;quot;\t&amp;quot; '($5==&amp;quot;MYC&amp;quot;)' gencode.v39.exons.bed &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can then further filter that by transcript or other attributes, etc.&lt;/p&gt;&#xA;" OwnerUserId="776" LastEditorUserId="776" LastEditDate="2022-02-22T16:08:55.963" LastActivityDate="2022-02-22T16:08:55.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18367" PostTypeId="1" CreationDate="2022-01-20T10:27:42.010" Score="0" ViewCount="147" Body="&lt;p&gt;I am trying to find out how one can using gene expression data can infer gene regulatory network applying graph theory concepts. But I could not find a proper reference that&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;explain how one can get the adjacency matrix for a gene regulatory network? like what are nodes what are the edges? which kinds of parameters one should infer?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;how one can use gene expression data for this modeling?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Any reference or help regarding these questions is highly appreciated if even it is going to be a keyword.&lt;/p&gt;&#xA;" OwnerUserId="14890" LastActivityDate="2022-01-22T00:05:30.120" Title="what are the nodes and edges values in gene regulatory networks?" Tags="|genomics|gene-expression|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18368" PostTypeId="2" ParentId="18367" CreationDate="2022-01-20T10:36:40.573" Score="3" Body="&lt;p&gt;This is an incomplete answer since it's not my field of expertise, but it sounds like Weighted correlation network analysis is what you are referring to.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-9-559&quot; rel=&quot;nofollow noreferrer&quot;&gt;There is a paper&lt;/a&gt; which describes how it can be applied to gene expression data.&lt;/p&gt;&#xA;&lt;p&gt;Langfelder, P., Horvath, S. WGCNA: an R package for weighted correlation network analysis. BMC Bioinformatics 9, 559 (2008). &lt;a href=&quot;https://doi.org/10.1186/1471-2105-9-559&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1186/1471-2105-9-559&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2022-01-20T10:36:40.573" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18370" PostTypeId="1" CreationDate="2022-01-20T15:21:09.723" Score="1" ViewCount="469" Body="&lt;p&gt;So, I have downloaded some .txt and .csv files containing the raw count data (I am supposed to find out differentially expressed genes and see which genes and up- and down-regulated). I have also downloaded both the .family.soft and the series matrix files as they mentioned they contained metadata - however I'm facing several errors while trying to import them.&lt;/p&gt;&#xA;&lt;p&gt;And all the tutorials I am trying to follow seem to have premade metadata files and there is no explanation on how this metadata is made.&lt;/p&gt;&#xA;&lt;p&gt;Please help me in trying to understand what I am doing wrong and how to proceed. I will try to give as much information as I can to the best of my abilities, as I have barely any experience in this field.&lt;/p&gt;&#xA;" OwnerUserId="14897" LastEditorUserId="3694" LastEditDate="2023-09-07T11:49:23.037" LastActivityDate="2023-09-07T11:49:23.037" Title="How to input data and metadata from NCBI for RNA-Seq analysis in R" Tags="|r|rna-seq|deseq2|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18371" PostTypeId="2" ParentId="18367" CreationDate="2022-01-20T19:55:58.240" Score="1" Body="&lt;p&gt;There are many different kinds of &amp;quot;gene regulatory networks&amp;quot;, so how to model them with gene expression data depends on what you're trying to model and what type of regulatory mechanisms you want to study.&lt;/p&gt;&#xA;&lt;p&gt;Pick a random gene in the human genome and use a resource database like &lt;a href=&quot;https://stringdb.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;StringDB&lt;/a&gt;.&#xA;Let's take the &lt;a href=&quot;https://string-db.org/cgi/network?taskId=bfdm466sjcp0&amp;amp;sessionId=bEa7ENjHHZ2u&quot; rel=&quot;nofollow noreferrer&quot;&gt;TET1&lt;/a&gt; protein for example.&#xA;Some edges in that graph represent co-expression that you'd be able to find from protein expression data (e.g. NANOG, DNMT3B, and more are linked to TET1 this way).&#xA;This type of graph tries to model how different proteins interact with each other inside a cell.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, you can look at 3D chromatin interactions from Hi-C data, for example.&#xA;Nodes in that graph could be regulatory elements like enhancers and promoters, and the edges could be how frequently these elements interact with each other.&#xA;This type of graph tries to model how DNA elements interact with each other to control expression of nearby genes.&#xA;This is a different purpose to the one above, so it needs to be modelled and interpreted differently.&lt;/p&gt;&#xA;&lt;p&gt;Each of these are their own type of &amp;quot;gene regulatory network&amp;quot; which require their own methods for constructing adjacency matrices.&#xA;For example, you could construct an adjacency matrix by looking at the correlation of expression for each pair of proteins (or transcripts, or genes) across a large set of samples.&#xA;Or in the promoter-enhancer interaction case, a Hi-C contact matrix is itself a type of adjacency matrix.&lt;/p&gt;&#xA;&lt;p&gt;Different graph models (and thus different adjacency matrices) will be useful for different purposes.&#xA;So it might be worth trying to refine what you mean by a &amp;quot;gene regulatory network&amp;quot;, what aspect of biology you're trying to study using this network, and how different types of gene expression data may be useful to construct these networks.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.hsph.harvard.edu/profile/john-quackenbush/&quot; rel=&quot;nofollow noreferrer&quot;&gt;John Quackenbush&lt;/a&gt;'s and &lt;a href=&quot;https://califano.c2b2.columbia.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andrea Califano&lt;/a&gt;'s labs have produced a lot of papers modelling different types of gene regulatory networks, like &lt;a href=&quot;https://netzoo.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Network Zoo&lt;/a&gt; and &lt;a href=&quot;https://califano.c2b2.columbia.edu/aracne&quot; rel=&quot;nofollow noreferrer&quot;&gt;ARACNe&lt;/a&gt;.&#xA;These mostly use gene expression data from RNA-seq measurements, but you can read up on some of these to get some ideas on how these types of models may work.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2022-01-20T19:55:58.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18372" PostTypeId="2" ParentId="18370" CreationDate="2022-01-20T20:06:08.093" Score="1" Body="&lt;p&gt;A metadata file isn't that large or complicated.  If you want to do DE, you must have two treatments?  Or two timepoints?  Two or more something, and you are comparing samples of one type to another?&lt;/p&gt;&#xA;&lt;p&gt;It's just a data table where the rownames are the samples, and then a column for, say, treatment.  If you have some samples treated, some not, some at day 0, and some at day 5, then you have a column for Day and another column for Treatment.  (Depending on what you want to compare to what, you might want another column of Day_Treatment)&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2022-01-20T20:06:08.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18373" PostTypeId="2" ParentId="18344" CreationDate="2022-01-20T20:07:36.180" Score="1" Body="&lt;p&gt;I'd use biomart on ensembl.org&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2022-01-20T20:07:36.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18374" PostTypeId="2" ParentId="3365" CreationDate="2022-01-21T10:21:49.897" Score="2" Body="&lt;p&gt;I have a counter example to other answers. BiocViews is not needed to install Bioconductor dependencies on CRAN. I have a &lt;a href=&quot;https://cran.r-project.org/package=BaseSet&quot; rel=&quot;nofollow noreferrer&quot;&gt;package on CRAN&lt;/a&gt; that suggests Bioconductor packages and they are found without using BiocViews.&lt;/p&gt;&#xA;&lt;p&gt;If you are installing a package from github and it has dependencies on Bioconductor you can use directly &lt;code&gt;BiocManager::install(&amp;quot;user/package&amp;quot;)&lt;/code&gt; and it will use &lt;code&gt;remotes::install_github&lt;/code&gt; internally and pull also packages from Bioconductor (from the right release).&lt;/p&gt;&#xA;&lt;p&gt;BiocViews is/was used to detect that a package is for Bioconductor by remotes, devtools and sessioninfo among other packages that help the development and testing of packages.&#xA;In addition Bioconductor doesn't add a &lt;code&gt;Repository: Bioconductor&lt;/code&gt; on the provided DESCRIPTION field of the packages, while CRAN adds &lt;code&gt;Repository: CRAN&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2022-01-21T10:21:49.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18375" PostTypeId="1" AcceptedAnswerId="18389" CreationDate="2022-01-21T10:36:04.993" Score="0" ViewCount="182" Body="&lt;p&gt;I have a question related to processing of Single cell RNA-seq data originating from non 10x platform. I have worked with data that originated from 10x platform and I parse it through cell ranger pipeline and then put it through Seurat.&lt;/p&gt;&#xA;&lt;p&gt;As an example the dataset in this GSE156644 repository are not being processed through Cell ranger pipeline and I get an error that indicates &amp;quot;single-end read found&amp;quot; another example is of this GSE115235, in the later I have bunch of files that look like bulk RNA-seq.&lt;/p&gt;&#xA;&lt;p&gt;If you check the GSE115235 it has bunch of files that are not cellranger compatible. However, they are from single cell experiment. I find it strange that NCBI accepts such submission and the reviewers don't even pay attention to what has been submitted.&lt;/p&gt;&#xA;&lt;p&gt;I guess, that these are SE RNA seq data that can be aligned using STAR or Bowtie and then merged in some way to get per cell per sample expression.&lt;/p&gt;&#xA;&lt;p&gt;Along similar lines, how do I deal with 2 fastq files for a scRNA seq run? example &lt;a href=&quot;https://www.ebi.ac.uk/ena/browser/view/PRJNA602526&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/ena/browser/view/PRJNA602526&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This link has scRNA seq files that look paired end (R1 &amp;amp; R2 but NO I1). I am bit confused as cell ranger count option requires the folder to have 3 files (I1,R1,R2). How can I run cell ranger with R1 &amp;amp; R2? Or do I need to use another aligner?&lt;/p&gt;&#xA;&lt;p&gt;I will send an email to sra and also to the authors, but any other help to navigate through these datasets and process them is deeply appreciated.&lt;/p&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="5062" LastEditorUserId="73" LastEditDate="2022-01-24T04:51:15.290" LastActivityDate="2022-01-24T04:51:15.290" Title="Acquiring &amp; Processing single cell data (not 10x)" Tags="|scrnaseq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18377" PostTypeId="1" CreationDate="2022-01-21T12:51:56.773" Score="0" ViewCount="69" Body="&lt;p&gt;I have more of a conceptual question. I have two genome assemblies from the same plant, one from Dovetail technology (~998 Gb) and another is PacBio HiFi assembly (~1.1 Gb). The Dovetail assembly is more contiguous but has lower base quality whereas the PacBio has higher base quality but is more fragmented. It is a diploid organism with relatively high heterozygosity.&lt;/p&gt;&#xA;&lt;p&gt;I tried to use &lt;code&gt;RagTag&lt;/code&gt; &lt;code&gt;patch&lt;/code&gt; to improve the assembly. But no improvement was obtained.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to use both assemblies and produce a hybrid assembly with high contiguity and base quality?&lt;/p&gt;&#xA;" OwnerUserId="12238" LastEditorUserId="57" LastEditDate="2022-01-21T23:58:49.677" LastActivityDate="2022-01-23T12:27:27.050" Title="How to improve a genome assembly using Dovetail and PacBio assembly?" Tags="|sequence-analysis|assembly|reference-genome|pacbio|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18378" PostTypeId="1" AcceptedAnswerId="18381" CreationDate="2022-01-21T15:32:54.583" Score="5" ViewCount="231" Body="&lt;p&gt;I don't know if I'm in the right SE, if not let me know and I'll delete it.&lt;/p&gt;&#xA;&lt;p&gt;I am reading the &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/29/1/15/272537&quot; rel=&quot;noreferrer&quot;&gt;original publication of the STAR RNA-Sequence aligner&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Given the quoted text below I wonder if the algorithm for the seed search is actually an implementation of th Knut-Morris-Pratt algorithm?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The central idea of the STAR seed finding phase is the sequential search for a Maximal Mappable Prefix (MMP). MMP is similar to the Maximal Exact (Unique) Match concept used by the large-scale genome alignment tools Mummer (Delcher et al., 1999, 2002; Kurtz et al.) and MAUVE (Darling et al., 2004, 2010). Given a read sequence R, read location i and a reference genome sequence G, the MMP(R,i,G) is defined as the longest substring (Ri, Ri+1, … , Ri+MML−1) that matches exactly one or more substrings of G, where MML is the maximum mappable length. We will explain this concept using a simple example of a read that contains a single splice junction and no mismatches (Fig. 1a). In the first step, the algorithm finds the MMP starting from the first base of the read. Because the read in this example comprises a splice junction, it cannot be mapped contiguously to the genome, and thus the first seed will be mapped to a donor splice site. Next, the MMP search is repeated for the unmapped portion of the read, which, in this case, will be mapped to an acceptor splice site. Note that this &lt;strong&gt;sequential application of MMP search only to the unmapped portions of the read&lt;/strong&gt; makes the STAR algorithm extremely fast and distinguishes it from Mummer and MAUVE, which find all possible Maximal Exact Matches.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is not cited in the references but it might be considered 'common knowledge' by now which is why the citation is omitted...&lt;/p&gt;&#xA;" OwnerUserId="10791" LastActivityDate="2022-01-24T18:56:45.723" Title="Is the algorithm of the STAR RNA Seq-Aligner similar to the Knuth-Morris-Pratt string matching algorithm?" Tags="|algorithms|star|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18379" PostTypeId="1" CreationDate="2022-01-21T15:35:15.990" Score="2" ViewCount="85" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/9507213/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;How do I convert the Copy Number (CN) file into useful information such as HOMDEL and AMP?&lt;/p&gt;&#xA;&lt;p&gt;This is an already normalized file with pseudo count + 1. In this file there are values from 0 to 1 for each cancer cell line, in wide format.&lt;/p&gt;&#xA;&lt;p&gt;I have already looked at the &lt;a href=&quot;https://forum.depmap.org/t/what-is-relative-copy-number-copy-number-ratio/104/9&quot; rel=&quot;nofollow noreferrer&quot;&gt;forum post&lt;/a&gt; about the file, but it's not helpful for me.&lt;/p&gt;&#xA;&lt;p&gt;This is what the file looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;,DDX11L1 (100287102),WASH7P (653635),FAM138A (645520),OR4F5 (79501)&#xA;ACH-000621,1.0166833616923812,1.0166833616923812,1.0166833616923812,1.0166833616923812&#xA;ACH-002512,2.9659871795507433,2.9659871795507433,2.9659871795507433,2.9659871795507433&#xA;ACH-001188,0.9881267634909964,0.9881267634909964,0.9881267634909964,0.9881267634909964&#xA;ACH-000471,0.8053837915961594,0.8053837915961594,0.8053837915961594,0.8053837915961594&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thank you for your help&lt;/p&gt;&#xA;" OwnerUserId="11884" LastEditorUserId="73" LastEditDate="2022-01-24T18:51:32.207" LastActivityDate="2023-10-16T22:06:45.830" Title="How to interpret the Copy Number from pseudocount depmap portal file?" Tags="|cancer|cell-line|copy-number|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18380" PostTypeId="2" ParentId="18314" CreationDate="2022-01-21T15:39:02.727" Score="1" Body="&lt;p&gt;Frontiers in Ecology and Evolution did a focused and succinct review looking at 'the Tree of Life' in 2017 &lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fevo.2017.00042/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. They particularly highlight the importance of integrating a fossil record within molecular trees. The upshot is simply that increasingly tighter integration between palaeontology and molecular dating is the ongoing research frontier.&lt;/p&gt;&#xA;&lt;p&gt;The best approach in context is to look at relevant literature. The key challenge in my opinion is the underlying population dynamic because this does heavily affect temporal estimates.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-01-21T15:39:02.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18381" PostTypeId="2" ParentId="18378" CreationDate="2022-01-21T21:24:44.627" Score="5" Body="&lt;p&gt;No, STAR isn't using the KMP algorithm or a modification of it.  The KMP algorithm is an online exact pattern matching algorithm.  It does (linear time) pre-processing on the &lt;em&gt;query&lt;/em&gt; and then finds all exact occurrences of the query in the string (as you may be alluding, you can also modify KMP to report maximal exactly matching prefixes of the query).  This process takes time of order N+M where N is the length of the reference and M the length of the query.&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, STAR pre-computes a &lt;a href=&quot;https://courses.cs.washington.edu/courses/cse590q/00au/papers/manber-myers_soda90.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;suffix array&lt;/a&gt; on the &lt;em&gt;reference&lt;/em&gt;, and then uses this to answer exact prefix match queries across &lt;em&gt;many&lt;/em&gt; different queries.  This is a very different approach.  Building the suffix array takes time linear in the reference, and storing the suffix array requires space linear in the reference (though the constant factor is pretty big).  However, it allows enumerating all occurrences of a query &lt;em&gt;Q&lt;/em&gt; in the reference &lt;em&gt;R&lt;/em&gt; in time &lt;span class=&quot;math-container&quot;&gt;$O(k + lg(\left|R\right|)+ \left|Q\right|)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; is the number of occurences of &lt;em&gt;Q&lt;/em&gt; in &lt;em&gt;R&lt;/em&gt;.  This holds not just for the whole pattern, but for any maximal prefix as well.  Also, in practice, STAR does this much faster by building a lookup table that maps small prefixes (say, all length 12 prefixes) to the contiguous suffix array interval where they occur.  In practice, this gives very fast lookup of queries.  Again, a &lt;em&gt;key&lt;/em&gt; difference between this kind of approach and KMP is that the suffix array pre-processes the &lt;em&gt;reference&lt;/em&gt; and therefore builds an index that works for many different queries (the sequencing reads), while KMP pre-processes the query to find all of its occurrences on the reference.  Another &lt;em&gt;key&lt;/em&gt; difference is that, while the KMP algorithm is always linear in the length of the reference to return all matches in the worst case, the suffix array can be considerably faster.&lt;/p&gt;&#xA;&lt;p&gt;In the original STAR publication, this is mentioned in the supplementary material in section 1.2 &amp;quot;Pre-indexing of suffix arrays&amp;quot;.  This idea (along with others) is also covered in quite some depth in &lt;a href=&quot;https://arxiv.org/pdf/1607.08176.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper.  Basically, the look up table has been a folk technique for a long time, but it's not clear there is an &amp;quot;original&amp;quot; source for it.&lt;/p&gt;&#xA;" OwnerUserId="491" LastEditorUserId="73" LastEditDate="2022-01-24T18:56:45.723" LastActivityDate="2022-01-24T18:56:45.723" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18382" PostTypeId="2" ParentId="18370" CreationDate="2022-01-22T00:00:30.387" Score="1" Body="&lt;p&gt;I don't expect the soft or matrix files will be useful for you, although it might be possible to pull metadata out of those files if absolutely necessary. I usually only see those for microarray data, so it might be worth double-checking that you do actually have RNASeq data.&lt;/p&gt;&#xA;&lt;p&gt;I've found the best &amp;quot;quick start&amp;quot; explanation on how to carry out differential expression analysis to be the one for &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#quick-start&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2&lt;/a&gt;. As input, this requires three things:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;A count matrix, with samples in columns, genes in rows, and matrix entries being the raw read counts of reads mapped to genes for each sample&lt;/li&gt;&#xA;&lt;li&gt;A metadata data frame, with rows arranged exactly as in the columns of the count matrix, and fields describing features of each sample&lt;/li&gt;&#xA;&lt;li&gt;A statistical model / equation that describes the relationship between samples&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I have found the equation to be the most complicated to understand, and unfortunately it's very situation dependent. In its simplest form, setting up a DESeq2 data structure would look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData = count.mat,&#xA;                              colData = metadata.df,&#xA;                              design= ~ Condition)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This would be for a situation where all samples were sequenced in the same sequencing run, and only a single characteristic were being compared (e.g. with vs without treatment). The metadata data frame might look something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;metadata.df &amp;lt;- data.frame(row.names = colnames(count.mat),&#xA;                          Condition = c(rep(&amp;quot;treated&amp;quot;, 6),&#xA;                                        rep(&amp;quot;untreated&amp;quot;, 6))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;e.g.:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;          Condition&#xA;Sample01    treated&#xA;Sample02    treated&#xA;Sample03    treated&#xA;Sample04    treated&#xA;Sample05    treated&#xA;Sample06    treated&#xA;Sample07  untreated&#xA;Sample08  untreated&#xA;Sample09  untreated&#xA;Sample010 untreated&#xA;Sample011 untreated&#xA;Sample012 untreated&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-01-22T00:00:30.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18383" PostTypeId="2" ParentId="18379" CreationDate="2022-01-22T00:12:15.857" Score="1" Body="&lt;p&gt;I'm not familiar with this file structure, but have you looked at the &lt;a href=&quot;https://forum.depmap.org/t/what-is-relative-copy-number-copy-number-ratio/104/9&quot; rel=&quot;nofollow noreferrer&quot;&gt;forum post&lt;/a&gt; about relative copy number linked to in the FAQ?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Since we do not have matched normals, the output is a “copy ratio” or relative copy number. It is relative to the rest of the genome for that cell line. E.g. if the cell line is tetraploid we would not be able to see it from the relative copy number. These values are reported as log2(relative CN + 1) in the portal.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-01-22T00:12:15.857" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18384" PostTypeId="1" CreationDate="2022-01-22T21:45:53.813" Score="3" ViewCount="81" Body="&lt;p&gt;I created a basic CNN architecture using Tensorflow to classify transcription factor binding sites. My aim is to somehow extract and visualize sequence logos from the convolutional kernels. The model looks like this:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;model = tf.keras.models.Sequential()&#xA;model.add(tf.keras.layers.Conv2D(128, (1, 24), padding='same', input_shape=(1, 101, 4), activation='relu'))&#xA;model.add(tf.keras.layers.GlobalMaxPool2D())&#xA;# ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am using the supplement &lt;a href=&quot;http://cnn.csail.mit.edu/motif_discovery/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dataset&lt;/a&gt; from &lt;a href=&quot;https://www.researchgate.net/publication/304002757_Convolutional_neural_network_architectures_for_predicting_DNA-protein_binding&quot; rel=&quot;nofollow noreferrer&quot;&gt;this publication&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What I tried so far is that I have trained the model and created a new model out of it which has only an input layer and an output layer that contains the weights of the original convolution kernels:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;test_model = tf.keras.models.Model(inputs=model.inputs, outputs=model.layers[0].output)&#xA;feature_maps = test_model.predict(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have ran a prediction on a sequence and plotted out a convolution kernel:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/j7CKEl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/j7CKEl.png&quot; alt=&quot;Convolution kernel plot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;At this point I got a bit stuck as I am not sure how can I move forward with this in the direction of visualizing the sequence logos.&lt;/p&gt;&#xA;" OwnerUserId="14911" LastActivityDate="2024-03-13T16:02:20.383" Title="How to extract and visualize sequence logos from CNN kernels?" Tags="|python|machine-learning|tensorflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18385" PostTypeId="2" ParentId="18384" CreationDate="2022-01-22T22:46:23.220" Score="0" Body="&lt;p&gt;What you are missing is the accuracy of the prediction, this is the next step and must be &amp;gt;80%, preferably &amp;gt;85% and &amp;gt;90% is good in biological systems. If its lower than 80% you'll need to adjust the training. Following this there are issues with false positives and false negatives.&lt;/p&gt;&#xA;&lt;p&gt;The method of using seems fine for mapping sequence motifs to transcription factors, which is my interpretation of your question. There must have been vectorisation of the sequence data and this is not trivial.&lt;/p&gt;&#xA;&lt;p&gt;To answer your question, which you might be asking, you cannot produce an automated heatmap of sequence motif in relation to the transcriptor factor binding. You simply input a sequence and the associated transcription factors are revealed. Unless you vectorised your transcription factors somehow and associated this with a sequence motif.&lt;/p&gt;&#xA;&lt;p&gt;To produce a heatmap, which again is what I suspect you are requesting, you do this via iteration. So you need a bit of coding. Thus you take all relevant sequence motifs - particularly those that have not been trained - and input them into deep learning algorithm and map which motifs associate with which binding factors and assign a numerical score to given groups of sequence motifs. Any graphics program can then be used to convert this to a heat map. Deciding 'taxonomic groups' for the sequence motifs is subject to your discretion. If you've given the transcription factors and trained this onto the sequence motifs, then classification of transcription factors I suspect would be much easier.&lt;/p&gt;&#xA;&lt;p&gt;Heatmaps are produced in other machine learning approaches, i.e. which exclude deep learning, but your next step is percentage accuracy regardless of approach.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="73" LastEditDate="2022-02-22T11:24:04.883" LastActivityDate="2022-02-22T11:24:04.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18387" PostTypeId="1" CreationDate="2022-01-22T23:30:49.373" Score="3" ViewCount="120" Body="&lt;p&gt;After running PSI-BLAST one obtains a profile matrix with this header:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; A   R   N   D   C   Q   E   G   H   I   L   K   M   F   P   S   T   W   Y   V&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How does PSI-BLAST handles X and B &amp;quot;residues&amp;quot; in the fasta file then?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;X residue being an amino acid that can't be assigned due to the presence of nucleotide ambiguities which result in the codon representing more than one amino acid.&lt;/li&gt;&#xA;&lt;li&gt;B (beta) residues, with the exception of B-alanine are singly generated &lt;em&gt;in vitro&lt;/em&gt; and being explored for their putative antibacterial properties.&lt;/li&gt;&#xA;&lt;li&gt;PSSM is the position specific score matrix&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14030" LastEditorUserId="3848" LastEditDate="2022-03-19T16:35:12.067" LastActivityDate="2022-03-19T16:35:12.067" Title="PSSM Matrix in PSI-BLAST" Tags="|sequence-alignment|proteins|blast|pssm|psi-blast|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18388" PostTypeId="1" AcceptedAnswerId="18396" CreationDate="2022-01-23T09:31:30.723" Score="1" ViewCount="255" Body="&lt;p&gt;I'm using STAR for an internship (fusion-genes in ALL cancer) and I'd like to understand what exactly it is doing. The people at the internship don't know either how it works, given their biology backgrounds. They told me &amp;quot;its enough to know that it works&amp;quot;. That was not an satisfactory answer for me which is why I am fighting my way trough the suplementary materials trying to wrap my head around the algo. My background is bio as well but I've spent reading a book about algos and algos in BI this year.&lt;/p&gt;&#xA;&lt;p&gt;I am trying to understand the section &lt;em&gt;Pre-indexing of suffix arrays&lt;/em&gt; from &lt;a href=&quot;https://oup.silverchair-cdn.com/oup/backfile/Content_public/Journal/bioinformatics/29/1/10.1093_bioinformatics_bts635/2/bts635_Supplementary_Data.zip?Expires=1645401704&amp;amp;Signature=jkgvDWNJutWbvNaZ9RiZbgkSBswvI19rq-8NkA35l-ruzy5w0W4RBIRvhtpncSGvAd1iiCi3PHZCeDmq1Jgr1N79wDz%7E2bnb8KSoQcSh%7EObsOwCI-G7bWuZiQRh3T1jbjOL-1VW9irjTnVO4icYY%7EgGvASDXRxVvxTsOl1m3%7EdQrGjcvUU4QXFQG8opHedFwmY0NaC9e%7E8WfqrdGLLFPXbMbZarL396dLCIVgk7DhgaeIUktXIPxXqyEeJ6fRH4FCnUl63uuAU45GJnE8aObOGwXKM5IDVlEz9AcSiB-QuHQN76LghnZNc6iDQZVCmBAZTLnvH0D0huPHuHUvGaEMg__&amp;amp;Key-Pair-Id=APKAIE5G5CRDK6RD3PGA&quot; rel=&quot;nofollow noreferrer&quot;&gt;supplementary materials&lt;/a&gt; of the &lt;em&gt;Dobin et al.&lt;/em&gt; &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/29/1/15/272537&quot; rel=&quot;nofollow noreferrer&quot;&gt;STAR RNA seq-aligner Publication&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;While suffix array search is theoretically fast owing to its binary nature, in practice it may suffer from non-locality resulting in persistent cache misses which deteriorate the performance. To alleviate this problem we developed a pre-indexing strategy. After the SA is generated, we find the locations of all possible &lt;span class=&quot;math-container&quot;&gt;$L-mers$&lt;/span&gt; in the SA, &lt;span class=&quot;math-container&quot;&gt;$L&amp;lt;=L_{max}$&lt;/span&gt;, where L_{max} is user defined and is typically 12-15. Since the nucleotide alphabet contains only four letters, there are NL=22L different L-mers for which the SA locations have to be stored.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The authors also state:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Suffix Array (SA) of the whole genome is utilized to find the Maximum Mappable Prefixes (MMP).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I am confused by the use of the term &lt;span class=&quot;math-container&quot;&gt;$L-mer$&lt;/span&gt;. When googleing I only keep finding &lt;span class=&quot;math-container&quot;&gt;$k-mers$&lt;/span&gt; in combination with suffix arrays. I did however find &lt;a href=&quot;https://github.com/mariusmni/radixSA64&quot; rel=&quot;nofollow noreferrer&quot;&gt;this GitHub page&lt;/a&gt; mentioning &lt;span class=&quot;math-container&quot;&gt;$L-mers$&lt;/span&gt; in the following way:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This program computes either the suffix array or the L-mer array of a string. The suffix array gives the lexicographical order of all the suffixes of a string. The L-mer array gives the lexicographical order of all the L-mers in the string (sorts the suffixes with respect to only the first L characters).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;→ Can I assume that &lt;span class=&quot;math-container&quot;&gt;$L-mer$&lt;/span&gt; was used instead of &lt;span class=&quot;math-container&quot;&gt;$k-mer$&lt;/span&gt; to stress the lexicographical ordering? But as far as I understood &lt;em&gt;all&lt;/em&gt; suffix arrays contain the suffixes' index based on its lexicographic ordering as illustrated by &lt;a href=&quot;https://en.wikipedia.org/wiki/Suffix_array#Correspondence_to_suffix_trees&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia's page on suffix arrays&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/sDHuo.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Do I have to imagine the entire genome being a single string that is converted into one single suffix array? Could you provide sources fit for a beginner with biology background and very little idea about algorithms?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="10791" LastEditorUserId="73" LastEditDate="2022-01-24T18:54:34.667" LastActivityDate="2022-01-24T18:54:34.667" Title="How should I imagine the pre-indexing of suffix arrays in the STAR RNA Seq-aligner?" Tags="|algorithms|star|indexing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18389" PostTypeId="2" ParentId="18375" CreationDate="2022-01-23T12:24:11.970" Score="2" Body="&lt;p&gt;&lt;em&gt;Modified from @devon-ryan's comments:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;The couple of samples I looked at have the sequence from read 1 embedded in their header. I1 isn't needed, they're already demultiplexed. You could feed those into STAR or similar tools, but processing them will be more of a pain since most tools expect the original unprocessed version of what they uploaded.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Modified from atpoint's comments:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Single-cell data is not really useful as &amp;quot;bulk&amp;quot; as the notable amount of PCR during the library prep creates quite a bias that without the UMI information cannot be corrected. I'd email the authors and ask for the R1 files. Single-cell uploads at NCBI tend to be a mess, that's unfortunately how it is. Without R1 data are almost useless.&lt;/p&gt;&#xA;&lt;p&gt;Those file format issues are strange, but it is what it is. Reviewers do not pay attention to method sections, choice of tools and data submission most of the time in my experience, especially not when the paper is biology-focused rather than a computational-focused one. Even more, you often do not find the suitable metadata for a proper analysis such as information about batches etc. The only choice you have is emailing the authors as noted above.&lt;/p&gt;&#xA;&lt;p&gt;If by &amp;quot;two files&amp;quot; you mean that the same library (=sample) was sequenced over multiple lanes (very common to do that since required depth is often larger than what a single lane can provide) then merge the fastq files, so all R1s into one and all R2 into one file.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-01-24T04:48:37.067" LastActivityDate="2022-01-24T04:48:37.067" CommentCount="0" CommunityOwnedDate="2022-01-23T12:24:11.970" ContentLicense="CC BY-SA 4.0" />
  <row Id="18390" PostTypeId="2" ParentId="18377" CreationDate="2022-01-23T12:27:27.050" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @maximilian-press, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am confused when you talk about base quality, because those technologies are largely about ordering and orienting existing contigs, not assembling contigs in the first place. If you have Hi-C reads, you can read them into juicebox and actually visualize the junctions to see if the assembly makes sense: aidenlab.org/juicebox. It would also be helpful to see the actual numbers, not just &amp;quot;more&amp;quot; or &amp;quot;lower&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-01-23T12:27:27.050" CommentCount="6" CommunityOwnedDate="2022-01-23T12:27:27.050" ContentLicense="CC BY-SA 4.0" />
  <row Id="18391" PostTypeId="2" ParentId="18144" CreationDate="2022-01-23T12:37:47.660" Score="1" Body="&lt;p&gt;Without X chromosome variation, sex cannot be reliably determined from input genotypes.&lt;/p&gt;&#xA;&lt;p&gt;The error is telling you that there are no polymorphic X chromosome loci in the provided data. Either your input files don't include any X chromosome information (this could happen due to the increased statistical challenges involved in X chromosome data), or all the X chromosome &amp;quot;variants&amp;quot; are invariant / homozygous.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-01-23T12:37:47.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18392" PostTypeId="1" CreationDate="2022-01-23T12:40:25.880" Score="1" ViewCount="234" Body="&lt;p&gt;In a custom package which imports a package from Bioconductor, what is the right DESCRIPTION file?&lt;/p&gt;&#xA;&lt;p&gt;I added in my DESCRIPTION a line with Remote like so&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Remotes: &#xA;    bioc::3.14/dada2&#xA; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;as suggested by &lt;a href=&quot;https://cran.r-project.org/web/packages/remotes/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cran.r-project.org/web/packages/remotes/index.html&lt;/a&gt; vignette, but it still failed the R &lt;code&gt;check()&lt;/code&gt; with error &amp;quot;Packages required not available&amp;quot;. I am very confused about what is the right way to solve this problem, since multiple sources propose a different solution:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Package-Dependencies&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Package-Dependencies&lt;/a&gt; mention Additional_repository key in the DESCRIPTION&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;mention Remotes, suggesting one add a Remotes entry to DESCRIPTION with a specific syntax for Bioconductor dependencies&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The following post suggest using blockViews &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/3365/r-package-development-how-does-one-automatically-install-bioconductor-packages/3375#3375&quot;&gt;R package development: How does one automatically install Bioconductor packages upon package installation?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;What is the appropriate way to declare dependencies on packages on Bioconductor? The remotes support helped me satisfy a dependency on a package from Github, but not for bioconductor&lt;/p&gt;&#xA;" OwnerUserId="14913" LastActivityDate="2022-01-23T20:12:24.707" Title="How to consume non-CRAN packages in custom package?" Tags="|bioconductor|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18393" PostTypeId="2" ParentId="18341" CreationDate="2022-01-23T12:41:17.663" Score="3" Body="&lt;p&gt;&lt;em&gt;Answer from @user438383, @matteo-ferla and @m, converted from comments [anyone, feel free to edit this answer to improve it]:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;According to the intro it means 'pseudocenter'. All measurements are relative to an assigned point because a protein doesn't have a 'centre' they assigned it as a pseudocentre.&lt;/p&gt;&#xA;&lt;p&gt;Note that PC normally means principal component, but that does not seem to be the case here.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-01-23T12:41:17.663" CommentCount="0" CommunityOwnedDate="2022-01-23T12:41:17.663" ContentLicense="CC BY-SA 4.0" />
  <row Id="18394" PostTypeId="2" ParentId="18332" CreationDate="2022-01-23T13:00:13.923" Score="2" Body="&lt;p&gt;&lt;em&gt;Answer from @user324810, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;There is an alternative than doing it with Python unless it is absolutely necessary. If you want to merge multiple samples into one, use &lt;a href=&quot;http://samtools.github.io/bcftools/bcftools.html#merge&quot; rel=&quot;nofollow noreferrer&quot;&gt;bcftools merge&lt;/a&gt;. If you want simply to concatenate multiple VCFs, use &lt;a href=&quot;https://samtools.github.io/bcftools/bcftools.html#concat&quot; rel=&quot;nofollow noreferrer&quot;&gt;bcftools concatenate&lt;/a&gt;. Once you obtain the output file, you can load the dataframe and plot it as you wish.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-01-23T13:00:13.923" CommentCount="0" CommunityOwnedDate="2022-01-23T13:00:13.923" ContentLicense="CC BY-SA 4.0" />
  <row Id="18396" PostTypeId="2" ParentId="18388" CreationDate="2022-01-23T19:06:31.390" Score="2" Body="&lt;p&gt;You can take a look at slide 40 of the &lt;a href=&quot;https://rob-p.github.io/CMSC423_F21/static_files/presentations/CMSC423_F21_5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;lectures slides&lt;/a&gt; I use when I teach suffix arrays (reproduced here)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/aNdIr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aNdIr.png&quot; alt=&quot;pre-indexing of the suffix array&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;An L-mer &lt;em&gt;is&lt;/em&gt; a K-mer — STAR builds an explicit lookup table from all length 14 (by default) strings that maps the length 14 string to interval of the suffix array that contains &lt;strong&gt;all suffixes that begin with this length 14 prefix&lt;/strong&gt;.  This means that, to find a MMP, instead of having to do a binary search on the entire suffix array, you only have to do a binary search on the sub-interval that begins with the length 14 prefix that your query string starts with.  If all substrings suffixes began with length 14 prefixes with equal frequency, then this would reduce the size of the interval you have to look at by a factor of &lt;span class=&quot;math-container&quot;&gt;$268435456 = 4^{14}$&lt;/span&gt;.  Of course, in biological sequence, all prefixes are not equally likely, so the speedup is more variable in practice.  Nonetheless, it makes lookup in the suffix array much, much faster.&lt;/p&gt;&#xA;&lt;p&gt;Of course, in addition to the core algorithm, STAR contains many heuristics not all of which are likely described in the paper (especially since the tool has been and continues to be under active development and improvement since the paper's publication).  Nonetheless, this is the idea behind the suffix array pre-indexing.  Since STAR still has the whole suffix array, if there is no length 14 exact prefix match, it is still possible to find a smaller match using a more standard binary search in the suffix array.  However, I do not know (and am not sure where it is documented) exactly when/if STAR will fall back to a more naive binary search if a prefix L-mer is not found.  If you are truly curious about this, I'd recommend asking Alex Dobin (the STAR author and maintainer) over on the STAR GitHub repository.  He is generally very responsive to user queries there.&lt;/p&gt;&#xA;" OwnerUserId="491" LastActivityDate="2022-01-23T19:06:31.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18397" PostTypeId="2" ParentId="18392" CreationDate="2022-01-23T20:12:24.707" Score="2" Body="&lt;p&gt;As &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/18374/48&quot;&gt;I answered&lt;/a&gt; on the linked question, the right way to declare packages from repositories recognized by R is to just declare normally on the Imports, Depends, Enhances and LinkingTo.&lt;/p&gt;&#xA;&lt;p&gt;Bioconductor is an accepted repository so it doesn't need to be added on Additional_repository field.&#xA;BiocViews is a required field from Bioconductor. As CRAN accepts other fields on the DESCRIPTION it could be used by CRAN packages.&lt;/p&gt;&#xA;&lt;p&gt;You don't mention it but if you found this problem and it wasn't on your computer were you are developing the package it means you are using a continuous integration test. The problem with CI like Github actions, travis, appveyor or other is that dependencies must be installed before (as they should on your computer).&lt;/p&gt;&#xA;&lt;p&gt;To install these dependencies before that depends on the CI you use and the strategy used there (install from ppa, use RSPM, use &lt;code&gt;bspm::install&lt;/code&gt;, ...) . However, I recommend to do it via &lt;code&gt;BiocManager::install&lt;/code&gt; because this way you can install all packages from both CRAN and Bioconductor and their dependencies.&lt;/p&gt;&#xA;&lt;p&gt;Even if the package is on CRAN, github or just Bioconductor with &lt;code&gt;BiocManager::install&lt;/code&gt; you'll be able to install all the dependencies and continue to check your package normally on the CI.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Additional_repositories is for other non standard repositories like a personal one or other a specific of a project. It is used by only ~100  packages of the 18765 packages on CRAN. But there are many more dependencies from CRAN to Bioconductor (and vice versa) .&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2022-01-23T20:12:24.707" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="18398" PostTypeId="1" AcceptedAnswerId="18402" CreationDate="2022-01-24T12:15:09.843" Score="0" ViewCount="296" Body="&lt;p&gt;I have run a GWAS and used PRSice (&lt;a href=&quot;https://www.prsice.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.prsice.info/&lt;/a&gt;) to calculate polygenic risk scores on the data collected from the GWAS. I need to check whether alcohol dependence can predict gambling disorder, through the PRS. However I am having trouble with how exactly to do that. I have a GWAS on alcohol disorder from the one population and a GWAS on gambling and alcohol disorder from the other population, then I calculated PRS for the three GWAS datasets, but how do I use the one to predict in the other? Through the literature it was suggested to use linear mixed models, which I have done but I still feel as if I am doing it wrong as I only did the model on the one dataset and not the two combined.&lt;/p&gt;&#xA;" OwnerUserId="5066" LastEditorUserId="5066" LastEditDate="2022-01-24T12:33:20.363" LastActivityDate="2022-01-24T20:40:01.500" Title="Polygenic Risk Scores" Tags="|python|gwas|polygenic-risk-score|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18401" PostTypeId="1" AcceptedAnswerId="18421" CreationDate="2022-01-24T19:43:11.693" Score="4" ViewCount="373" Body="&lt;p&gt;I recently had a script fail due to poor handling of BLAST output. The BLAST &lt;code&gt;-outfmt staxids&lt;/code&gt; field usually returns a single taxid, but occasionally it returns two or more taxids separated by a semicolon, such as &lt;code&gt;556514;701533&lt;/code&gt;. Fixing the script to handle this should be fairly straightforward. But the original failure happened when I was searching a local copy of the NCBI nt database, and I'd like to create a test case with a small query and small database that reproduces this issue more rapidly.&lt;/p&gt;&#xA;&lt;p&gt;I am familiar with the &lt;code&gt;-taxid&lt;/code&gt; and &lt;code&gt;-taxid_map&lt;/code&gt; flags of the &lt;code&gt;makeblastdb&lt;/code&gt; command, but I have been unable to reproduce this behavior with BLAST; that is, make BLAST return multiple taxids for a sequence.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If I provide &lt;code&gt;556514;701533&lt;/code&gt; or &lt;code&gt;556514,701533&lt;/code&gt; as a sequence's taxid with &lt;code&gt;-taxid&lt;/code&gt; or &lt;code&gt;-taxid_map&lt;/code&gt;, the &lt;code&gt;makeblastdb&lt;/code&gt; command fails citing an integer conversion error.&lt;/li&gt;&#xA;&lt;li&gt;If I provide &lt;code&gt;556514 701533&lt;/code&gt; as the taxid in the &lt;code&gt;taxid_map&lt;/code&gt; file, the database will be indexed successfully but BLAST will return the invalid value &lt;code&gt;0&lt;/code&gt; for &lt;code&gt;staxids&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;If I duplicate each entry in the &lt;code&gt;-taxid_map&lt;/code&gt; file so that there is an entry for each taxid (see below), only a single taxid is reported by BLAST.&#xA;&lt;pre&gt;&lt;code&gt;seq1 556514&#xA;seq1 701533&#xA;seq2 556514&#xA;seq2 701533&#xA;seq3 556514&#xA;seq3 701533&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;When constructing a custom local BLAST database, how do I associated a database sequence with more than one taxid?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;: The offending entry (or one such entry) appears to be a duplicate or merged entry in the nt database. When I use &lt;code&gt;blastdbcmd&lt;/code&gt; to retrieve the subject sequence by accession, nt returns a sequence with two deflines on single line, like so.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;accession1 plain text description &amp;gt;accession2 other description&#xA;ACGT...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Using either of the two accessions, &lt;code&gt;blastdbcmd&lt;/code&gt; will return the exact same entry, and the two staxids shown above correspond to the two distinct accessions.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;So part of the issue seems to be how to construct a BLAST database such that multiple accessions can point to a single entry/subject.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="96" LastEditorUserId="96" LastEditDate="2022-01-26T18:15:52.233" LastActivityDate="2022-01-26T19:34:50.797" Title="Assign multiple taxids to a sequence when constructing a local BLAST database" Tags="|blast|taxonomy|makeblastdb|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18402" PostTypeId="2" ParentId="18398" CreationDate="2022-01-24T20:40:01.500" Score="1" Body="&lt;p&gt;Before doing anything else, you should split your dataset into training and test groups, and [this is important] don't touch the test dataset until you're fully confident that your approach is correct: no post-test modification.&lt;/p&gt;&#xA;&lt;p&gt;A split that's considered acceptable in Computer Science is a 10%/90% split, i.e. use 10% of the data for training / algorithm generation and the remaining 90% for testing. This will give you good confidence that the algorithm works.&lt;/p&gt;&#xA;&lt;p&gt;I personally think it's a good idea to do population sub-sampling (i.e. further subsample that 10%) to determine an informative marker set prior to any risk calculations. What you should be looking for are markers that are consistently linked with the trait of interest (i.e. in your case alcohol dependence) in multiple sub-sampled populations. I go into a lot of detail about how to do this &lt;a href=&quot;https://doi.org/10.1101/104497&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; [bearing in mind it's an old pre-print, and doesn't do the 10/90 split I've recommended above].&lt;/p&gt;&#xA;&lt;p&gt;After you're happy with the marker set, calculate the risk scores &lt;em&gt;only using that set&lt;/em&gt; (hopefully PRSice can do this), then carry that risk score [only] through to test against the dependent trait of interest in the training dataset (i.e. in your case gambling disorder). If your dependent trait is a binary trait, this could be done using ROC analysis (as in my preprint) to calculate area under the curve. Or use linear mixed models, whatever works for you.&lt;/p&gt;&#xA;&lt;p&gt;Make any changes you think you need to improve the test result (e.g. changing the composition of the marker set).&lt;/p&gt;&#xA;&lt;p&gt;After convincing yourself that the approach is correct, do the same risk score calculation (using the same set of markers) on the test set, find out how well it predicts the dependent trait, and report on the results. Don't change anything.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-01-24T20:40:01.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18405" PostTypeId="2" ParentId="17911" CreationDate="2022-01-25T08:35:43.100" Score="2" Body="&lt;p&gt;You can use the UniProt Advanced search for this.&lt;/p&gt;&#xA;&lt;p&gt;If you look at this help page &lt;a href=&quot;https://www.uniprot.org/help/advanced_search&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/help/advanced_search&lt;/a&gt;, in particular the first example screenshot, you can type &amp;quot;encoded&amp;quot; where the screenshot has &amp;quot;structure&amp;quot;. Then you can choose your organelle. For Plastid, for example, this will translate into the query syntax organelle:plastid :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.uniprot.org/uniprot/?query=organelle%3APlastid&amp;amp;sort=score&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uniprot/?query=organelle%3APlastid&amp;amp;sort=score&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Several other organelles are available, of course, e.g. Mitochondrion :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.uniprot.org/uniprot/?query=organelle%3AMitochondrion&amp;amp;sort=score&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uniprot/?query=organelle%3AMitochondrion&amp;amp;sort=score&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Please don't hesitate to contact the UniProt helpdesk. We can unfortunately not guarantee to monitor all available forums as closely as we would like to, and can answer much more promptly when contacted directly.&lt;/p&gt;&#xA;" OwnerUserId="1459" LastActivityDate="2022-01-25T08:35:43.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18406" PostTypeId="2" ParentId="17824" CreationDate="2022-01-25T09:02:39.020" Score="1" Body="&lt;p&gt;If you look at this query&#xA;&lt;a href=&quot;https://www.uniprot.org/uniprot/?query=NOT%20database:(type:ensembl)&amp;amp;fil=organism:%22Homo%20sapiens%20(Human)%20%5B9606%5D%22&amp;amp;sort=score&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uniprot/?query=NOT%20database:(type:ensembl)&amp;amp;fil=organism:%22Homo%20sapiens%20(Human)%20[9606]%22&amp;amp;sort=score&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and restrict it to the reference proteome (by using the link on the left hand side):&#xA;&lt;a href=&quot;https://www.uniprot.org/uniprot/?query=NOT%20database:(type:ensembl)&amp;amp;fil=proteome%3AUP000005640+AND+organism%3A%22Homo+sapiens+%28Human%29+%5B9606%5D%22&amp;amp;sort=score&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uniprot/?query=NOT%20database:(type:ensembl)&amp;amp;fil=proteome%3AUP000005640+AND+organism%3A%22Homo+sapiens+%28Human%29+%5B9606%5D%22&amp;amp;sort=score&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;you will see that there are &amp;quot;only&amp;quot; 1100 entries, and that they are all in UniProtKB/Swiss-Prot, but none in UniProtKB/TrEMBL, the unreviewed section.&lt;/p&gt;&#xA;&lt;p&gt;The manual review of sequences for human and other proteomes is one of&#xA;our most important and crucial tasks but as you can imagine this is an&#xA;ongoing process. First because it represents an immense work, second&#xA;because knowledge is still evolving, the human reference genome being&#xA;for instance constantly improved by the Genome Reference Consortium. For&#xA;the human proteome, there are still a small but non negligible number of&#xA;UniProtKB/Swiss-Prot entries for which the sequence is not identical or&#xA;is not represented at all by Ensembl sequences. We are constantly&#xA;collaborating with Ensembl and other genome annotation resources trying&#xA;to agree on identical sequences to fill the gaps. UniProtKB human&#xA;entries are automatically mapped to the latest Ensembl release but you&#xA;have to take into account that UniProtKB and Ensembl releases are not&#xA;synchronized.&lt;/p&gt;&#xA;&lt;p&gt;Starting with release 2022_01, we will have cross-references from relevant UniProtKB/Swiss-Prot entries to MANE-Select (Matched Annotation from NCBI and EMBL-EBI, see &lt;a href=&quot;https://www.ensembl.org/info/genome/genebuild/mane.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ensembl.org/info/genome/genebuild/mane.html&lt;/a&gt; and&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/refseq/MANE/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/refseq/MANE/&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="1459" LastActivityDate="2022-01-25T09:02:39.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18409" PostTypeId="1" CreationDate="2022-01-25T15:30:09.490" Score="1" ViewCount="16" Body="&lt;p&gt;I am running a variant calling pipeline to detect solid tumours and haematological alterations. The variant calling is done with &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360037593851-Mutect2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mutect2&lt;/a&gt;, whereas I am using an R script to perform some filtering on the VCF produced by Mutect&lt;/p&gt;&#xA;&lt;p&gt;I noticed, however, that the output of the &lt;code&gt;header&lt;/code&gt; command (&lt;code&gt;VariantAnnotation&lt;/code&gt; library, R programming language) is not the same considering two versions (1.24.5 vs. 1.32.0).&lt;/p&gt;&#xA;&lt;p&gt;When using v 1.32.0 I get&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; header(input)&#xA;class: VCFHeader &#xA;samples(1): Panel-STHT-T210986-PJR&#xA;meta(6): MutectVersion fileformat ... GATKCommandLine contig&#xA;fixed(1): FILTER&#xA;info(16): CONTQ DP ... STR TLOD&#xA;geno(13): GT AD ... SAAF SAPP&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Whereas when using v 1.24.5 I get &lt;code&gt;fixed(0):&lt;/code&gt; instead of &lt;code&gt;fixed(1): FILTER&lt;/code&gt;, as shown above:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; header(input)&#xA;class: VCFHeader &#xA;samples(1): Panel-STHT-T210986-PJR&#xA;meta(3): META GATKCommandLine contig&#xA;fixed(0):&#xA;info(16): CONTQ DP ... STR TLOD&#xA;geno(13): GT AD ... SAAF SAPP&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also noticed that if I continue with the filtering that the R script does (eventually writing a vcf with &lt;code&gt;writeVcf&lt;/code&gt;), with &lt;code&gt;VariantAnnotation&lt;/code&gt; v 1.32.0 I get an extra line in the vcf header: &lt;code&gt;##FILTER=All filters passed&lt;/code&gt;, which is upsetting another downstream program (the error message that I get is &lt;code&gt;.vcf: Invalid FILTER line&lt;/code&gt;). And it appears to me that this &lt;code&gt;##FILTER=All filters passed&lt;/code&gt; line is actually coming from the extra line of the &lt;code&gt;header&lt;/code&gt; command.&lt;/p&gt;&#xA;&lt;p&gt;The easiest solution for me would be to downgrade from 1.32.0 to 1.24.5: However, I would like to understand where this extra line comes from&lt;/p&gt;&#xA;&lt;p&gt;Session information (when using v1.32.0 of &lt;code&gt;VariantAnnotation&lt;/code&gt; is shown below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;R version 3.6.1 (2019-07-05)&#xA;Platform: x86_64-redhat-linux-gnu (64-bit)&#xA;Running under: CentOS Linux 8 (Core)&#xA;&#xA;Matrix products: default&#xA;BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so&#xA;&#xA;locale:&#xA; [1] LC_CTYPE=en_GB.UTF-8          LC_NUMERIC=C                 &#xA; [3] LC_TIME=de_BE.UTF-8           LC_COLLATE=en_GB.UTF-8       &#xA; [5] LC_MONETARY=de_BE.UTF-8       LC_MESSAGES=en_GB.UTF-8      &#xA; [7] LC_PAPER=de_BE.UTF-8          LC_NAME=de_BE.UTF-8          &#xA; [9] LC_ADDRESS=de_BE.UTF-8        LC_TELEPHONE=de_BE.UTF-8     &#xA;[11] LC_MEASUREMENT=de_BE.UTF-8    LC_IDENTIFICATION=de_BE.UTF-8&#xA;&#xA;attached base packages:&#xA;[1] stats4    parallel  stats     graphics  grDevices utils     datasets &#xA;[8] methods   base     &#xA;&#xA;other attached packages:&#xA; [1] httr_1.4.1                  stringr_1.4.0              &#xA; [3] readr_1.3.1                 xlsx_0.6.3                 &#xA; [5] VariantAnnotation_1.32.0    Rsamtools_2.2.3            &#xA; [7] Biostrings_2.54.0           XVector_0.26.0             &#xA; [9] SummarizedExperiment_1.16.1 DelayedArray_0.12.2        &#xA;[11] BiocParallel_1.20.1         matrixStats_0.55.0         &#xA;[13] Biobase_2.46.0              GenomicRanges_1.38.0       &#xA;[15] GenomeInfoDb_1.22.0         IRanges_2.20.2             &#xA;[17] S4Vectors_0.24.3            BiocGenerics_0.32.0        &#xA;[19] optparse_1.6.4             &#xA;&#xA;loaded via a namespace (and not attached):&#xA; [1] Rcpp_1.0.2               lattice_0.20-38          prettyunits_1.0.2       &#xA; [4] xlsxjars_0.6.1           assertthat_0.2.1         zeallot_0.1.0           &#xA; [7] digest_0.6.15            BiocFileCache_1.10.2     R6_2.2.2                &#xA;[10] backports_1.1.4          RSQLite_2.1.1            pillar_1.4.2            &#xA;[13] zlibbioc_1.32.0          rlang_0.4.0              GenomicFeatures_1.38.2  &#xA;[16] progress_1.2.2           curl_4.2                 blob_1.1.1              &#xA;[19] Matrix_1.2-17            RCurl_1.98-1.1           bit_1.1-14              &#xA;[22] biomaRt_2.42.0           compiler_3.6.1           rtracklayer_1.46.0      &#xA;[25] pkgconfig_2.0.3          askpass_1.1              openssl_1.4.1           &#xA;[28] tidyselect_0.2.5         tibble_2.1.3             GenomeInfoDbData_1.2.2  &#xA;[31] XML_3.98-1.11            crayon_1.3.4             dplyr_0.8.3             &#xA;[34] dbplyr_1.4.2             GenomicAlignments_1.22.1 bitops_1.0-6            &#xA;[37] rappdirs_0.3.1           grid_3.6.1               DBI_1.0.0               &#xA;[40] magrittr_1.5             stringi_1.4.3            getopt_1.20.3           &#xA;[43] vctrs_0.2.0              tools_3.6.1              bit64_0.9-7             &#xA;[46] BSgenome_1.54.0          glue_1.3.1               purrr_0.3.2             &#xA;[49] hms_0.5.1                AnnotationDbi_1.48.0     memoise_1.1.0           &#xA;[52] rJava_0.9-11&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3452" LastActivityDate="2022-01-25T15:30:09.490" Title="Different output on same command and same input file, different library version" Tags="|r|bioconductor|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18411" PostTypeId="1" CreationDate="2022-01-25T16:32:51.430" Score="1" ViewCount="107" Body="&lt;p&gt;I am trying to run my VCF's through Topmed Imputation (&lt;a href=&quot;https://imputation.biodatacatalyst.nhlbi.nih.gov/#!pages/home&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://imputation.biodatacatalyst.nhlbi.nih.gov/#!pages/home&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;I am trying to make sure that I have the correct requirements for my file to be able to run through the software. These requirements are:&lt;/p&gt;&#xA;&lt;p&gt;Create a separate vcf.gz file for each chromosome.&lt;/p&gt;&#xA;&lt;p&gt;Variations must be sorted by genomic position.&lt;/p&gt;&#xA;&lt;p&gt;GRCh37 or GRCh38 coordinates are required.&lt;/p&gt;&#xA;&lt;p&gt;I have managed to split my VCF into each chromosome but I am unsure how to check which coordinates are being used and if my file is sorted by genomic position. Are there unix commands I can use to check, or do I just look in the file, Ive looked and not noticed anything.&lt;/p&gt;&#xA;" OwnerUserId="5066" LastActivityDate="2022-01-25T21:05:10.263" Title="Topmed File Requirements" Tags="|vcf|imputation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18412" PostTypeId="1" AcceptedAnswerId="18413" CreationDate="2022-01-25T18:56:14.597" Score="0" ViewCount="36" Body="&lt;p&gt;I can't cope with reworking the result. If id is the same I want to stick those items.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with open(x,'r') as rf:&#xA;    l = fr.readlines()&#xA;    previous_line_id = ''&#xA;    corrected_lines = []&#xA;&#xA;    for i in l:&#xA;       i_split = i.split('\t')&#xA;       id_ = i_split[0]&#xA;       if id_ == previous_line&#xA;           xxxxxxxxxx&#xA;       else:&#xA;           corrected_lines .append(i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;result:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    WP_XXXXXXXX2.1  PF06325 Ribosomal protein L11 methyltransferase (PrmA)  5&#xA;    WP_192XXXXXX.1  PF13489 Methyltransferase domain    4&#xA;    XXXXXXX45.1 PF02310 B12 binding domain  12  &amp;lt;-----------  DOUBLE IDS&#xA;    XXXXXXX45.1 PF04055 Radical SAM superfamily 5 &amp;lt;-----------&#xA;    WP_XXXXXXX29.1  PF13489 Methyltransferase domain    4&#xA;    WP_0XXXXXXXX.1  PF13489 Methyltransferase domain    4&#xA;    WP_XXXXXXX90.1  PF06325 Ribosomal protein L11 methyltransferase (PrmA)  5  &amp;lt;--------------   DOUBLE ID&#xA;    WP_XXXXXXX90.1  PF06325 Ribosomal protein L11 methyltransferase (PrmA)  7  &amp;lt;--------------&#xA;    WP_0XXXXXX62.1  PF13489 Methyltransferase domain    4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;result I need:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;WP_XXXXXXXX2.1  PF06325 Ribosomal protein L11 methyltransferase (PrmA)  5&#xA;WP_192XXXXXX.1  PF13489 Methyltransferase domain    4&#xA;XXXXXXX45.1 PF02310 B12 binding domain  12; PF04055 Radical SAM superfamily 5 &amp;lt;----------- STICKED SAME ID,OTHER VALUES&#xA;WP_XXXXXXX29.1  PF13489 Methyltransferase domain    4&#xA;WP_0XXXXXXXX.1  PF13489 Methyltransferase domain    4&#xA;WP_XXXXXXX90.1  PF06325 Ribosomal protein L11 methyltransferase (PrmA)  5; PF06325  Ribosomal protein L11 methyltransferase (PrmA)  7  &amp;lt;--------- STICKED STICKED SAME ID,OTHER VALUES&#xA;WP_0XXXXXX62.1  PF13489 Methyltransferase domain    4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1224" LastEditorUserId="1224" LastEditDate="2022-01-25T20:06:01.153" LastActivityDate="2022-01-26T06:15:03.587" Title="connect same ids with values in id rows (result of effectiveELD)" Tags="|python|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18413" PostTypeId="2" ParentId="18412" CreationDate="2022-01-25T20:43:28.733" Score="2" Body="&lt;p&gt;Assuming that the file &lt;code&gt;test.txt&lt;/code&gt; is tab-separated values with these fields:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;WP_XXXXXXXX2.1  PF06325     Ribosomal protein L11 methyltransferase (PrmA)      5&#xA;WP_192XXXXXX.1  PF13489     Methyltransferase domain                            4&#xA;XXXXXXX45.1     PF02310     B12 binding domain                                  12 &#xA;XXXXXXX45.1     PF04055     Radical SAM superfamily                             5&#xA;WP_XXXXXXX29.1  PF13489     Methyltransferase domain                            4&#xA;WP_0XXXXXXXX.1  PF13489     Methyltransferase domain                            4&#xA;WP_XXXXXXX90.1  PF06325     Ribosomal protein L11 methyltransferase (PrmA)      5&#xA;WP_XXXXXXX90.1  PF06325     Ribosomal protein L11 methyltransferase (PrmA)      7&#xA;WP_0XXXXXX62.1  PF13489     Methyltransferase domain                            4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can proceed first by reading your file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with open(&amp;quot;test.txt&amp;quot;,'r') as rf:&#xA;    l = rf.readlines() # all the lines are stored in variable 'l'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You should create a dictionary with your ID as keys and the remaining line as values:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# create empty dictionary&#xA;dict_id = {}&#xA;&#xA;for i in l:                                                               # for each line&#xA;    key_id = i.split(&amp;quot;\t&amp;quot;)[0]                                             # get the id&#xA;    if key_id not in dict_id.keys():                                      # if id as key is not found in dictionary&#xA;        value = &amp;quot;\t&amp;quot;.join(i.split(&amp;quot;\t&amp;quot;)[1:]).rstrip()                     # parse the value&#xA;        dict_id[key_id] = [value]                                         # add the id as key to dictionary with its value&#xA;    else:                                                                 # else if id as key is already in dictionary&#xA;        dict_id[key_id].append(&amp;quot;\t&amp;quot;.join(i.split(&amp;quot;\t&amp;quot;)[1:]).rstrip())     # append to id as key&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And now, you can print the result:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for key, value in dict_id.items():         # for each key, values in dictionary&#xA;    print(key + &amp;quot;: &amp;quot; + &amp;quot;; &amp;quot;.join(value))   # print the key and their values separated by '; '&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And you should obtain:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;WP_XXXXXXXX2.1: PF06325 Ribosomal protein L11 methyltransferase (PrmA)  5&#xA;WP_192XXXXXX.1: PF13489 Methyltransferase domain        4&#xA;XXXXXXX45.1: PF02310    B12 binding domain      12; PF04055     Radical SAM superfamily 5&#xA;WP_XXXXXXX29.1: PF13489 Methyltransferase domain        4&#xA;WP_0XXXXXXXX.1: PF13489 Methyltransferase domain        4&#xA;WP_XXXXXXX90.1: PF06325 Ribosomal protein L11 methyltransferase (PrmA)  5; PF06325      Ribosomal protein L11 methyltransferase (PrmA)  7&#xA;WP_0XXXXXX62.1: PF13489 Methyltransferase domain        4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The final script:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with open(&amp;quot;test.txt&amp;quot;,'r') as rf:&#xA;    l = rf.readlines()&#xA;&#xA;dict_id = {}&#xA;for i in l:&#xA;    key_id = i.split(&amp;quot;\t&amp;quot;)[0]&#xA;    if key_id not in dict_id.keys():&#xA;        value = &amp;quot;\t&amp;quot;.join(i.split(&amp;quot;\t&amp;quot;)[1:]).rstrip()&#xA;        dict_id[key_id] = [value] &#xA;    else:&#xA;        dict_id[key_id].append(&amp;quot;\t&amp;quot;.join(i.split(&amp;quot;\t&amp;quot;)[1:]).rstrip())&#xA;&#xA;for key, value in dict_id.items():&#xA;    print(key + &amp;quot;: &amp;quot; + &amp;quot;; &amp;quot;.join(value))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="4413" LastEditDate="2022-01-26T06:15:03.587" LastActivityDate="2022-01-26T06:15:03.587" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18414" PostTypeId="2" ParentId="18411" CreationDate="2022-01-25T21:05:10.263" Score="3" Body="&lt;p&gt;There are tools that allows you to sort your VCF file by genomic position like &lt;a href=&quot;http://samtools.github.io/bcftools/bcftools.html#sort&quot; rel=&quot;nofollow noreferrer&quot;&gt;bcftools sort&lt;/a&gt; or &lt;a href=&quot;https://vcftools.github.io/perl_module.html#vcf-sort&quot; rel=&quot;nofollow noreferrer&quot;&gt;vcf-sort&lt;/a&gt;. But if you don't want to install these software, you could always do with bash tools for one vcf:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;# get the header of your VCF and write to file 'sorted.vcf'&#xA;# if your file has the extension .gz, use zgrep instead&#xA;grep &amp;quot;^#&amp;quot; your.vcf &amp;gt; sorted.vcf&#xA;&#xA;# get the rest of the lines not starting with '#', sort them first by chromosome, then position and add them to file 'sorted.vcf'&#xA;grep -v &amp;quot;^#&amp;quot; your.vcf | sort -V -k1,1 -k2,2n &amp;gt;&amp;gt; sorted.vcf   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But since you have one file for each chromosome and assuming that your files look like &lt;code&gt;sample_chrZ.vcf.gz&lt;/code&gt;, where &lt;code&gt;Z&lt;/code&gt; is the chromosome number, you will have to do a loop:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;mkdir sorted_vcf&#xA;for chr in *.vcf.gz; do&#xA;   filename=&amp;quot;${chr%.*}&amp;quot;&#xA;   zgrep &amp;quot;^#&amp;quot; &amp;quot;$chr&amp;quot; &amp;gt; sorted_vcf/&amp;quot;$filename&amp;quot;&#xA;   zgrep -v &amp;quot;^#&amp;quot; &amp;quot;$chr&amp;quot; | sort -V -k1,1 -k2,2n  &amp;gt;&amp;gt; sorted_vcf/&amp;quot;$filename&amp;quot;&#xA;   bgzip sorted_vcf/&amp;quot;$filename&amp;quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And if you want to know whether your VCF has been aligned against GRCh38 or GRCh37, check the header in the VCF file with a line starting with &lt;code&gt;##reference=&lt;/code&gt;. A command-line like &lt;code&gt;grep &amp;quot;##reference=&amp;quot; your.vcf&lt;/code&gt; should give you the result. However, if you don't have the reference mentioned in your VCF file, this is going to be harder to guess. This &lt;a href=&quot;https://www.biostars.org/p/146171/&quot; rel=&quot;nofollow noreferrer&quot;&gt;post&lt;/a&gt; should help you figure it out.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2022-01-25T21:05:10.263" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18415" PostTypeId="1" CreationDate="2022-01-26T07:34:18.557" Score="2" ViewCount="47" Body="&lt;p&gt;Do we have a publication database or search engine that offers geneID or UniProtID correlation?&lt;/p&gt;&#xA;&lt;p&gt;For example, I search &amp;quot;IPS cells&amp;quot; -&amp;gt; I want to know what genes or proteins are under active research in IPS cells.&lt;/p&gt;&#xA;" OwnerUserId="14169" LastEditorUserId="12644" LastEditDate="2022-02-01T18:28:34.533" LastActivityDate="2022-02-01T22:19:35.393" Title="Is there a publication database or search engine offering geneID or UniProtID correlation?" Tags="|proteins|gene|database|phylogenetics|uniprot|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18417" PostTypeId="1" AcceptedAnswerId="18419" CreationDate="2022-01-26T14:43:11.193" Score="-3" ViewCount="608" Body="&lt;p&gt;I have a zipped vcf file of dbSNP hg38 version&lt;/p&gt;&#xA;&lt;p&gt;No space left on my device to unzip that&lt;/p&gt;&#xA;&lt;p&gt;I want to extract a column from that&lt;/p&gt;&#xA;&lt;p&gt;I have tried this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cut -f 3 00-All.vcf.gz | sed 's/[\t]/,/g' &amp;gt; cols.csv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But that needs my vcf to be unzipped&lt;/p&gt;&#xA;&lt;p&gt;Do you know any solution please?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2022-01-26T16:05:22.977" Title="Extracting a column from a vcf file" Tags="|vcf|snp|wes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18418" PostTypeId="1" AcceptedAnswerId="18423" CreationDate="2022-01-26T15:27:29.730" Score="0" ViewCount="71" Body="&lt;p&gt;I have a list of protein sequence ids (names). I want to sort the lines of a table, checking if the id is equal to the first column of the table. I split the table on lines and search for the id via regex:&lt;/p&gt;&#xA;&lt;p&gt;code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def map_line_to_ids(id):&#xA;    id= id.strip('\n')&#xA;    for s in lines:&#xA;        if re.search(id,s):&#xA;            return s #&amp;lt;------------ why always return else not if(s) when takes empty 'def'&#xA;        else:&#xA;            return id + '\n'&#xA;&#xA;names = ['XXXXXXX45.1', 'YP_343434.1', 'WP_XXXXXXXX2.1', 'WP_XXXXXXX90.1',&#xA;         'WP_0XXXXXX62.1', 'XP_222222222.2', 'WP_192XXXXXX.1',&#xA;         'WP_XXXXXXX29.1', 'WP_0XXXXXXXX.1']&#xA;&#xA;sorted_lines = list(map(map_line_to_ids, names))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;names (order):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;XXXXXXX45.1&#xA;YP_343434.1&#xA;WP_XXXXXXXX2.1&#xA;WP_XXXXXXX90.1&#xA;WP_0XXXXXX62.1&#xA;XP_222222222.2&#xA;WP_192XXXXXX.1&#xA;WP_XXXXXXX29.1&#xA;WP_0XXXXXXXX.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Input table:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;WP_XXXXXXXX2.1  PF06325 Ribosomal protein L11 methyltransferase (PrmA)  5&#xA;WP_192XXXXXX.1  PF13489 Methyltransferase domain    4&#xA;XXXXXXX45.1 PF02310 B12 binding domain  12; PF04055 Radical SAM superfamily 5 &#xA;WP_XXXXXXX29.1  PF13489 Methyltransferase domain    4&#xA;WP_0XXXXXXXX.1  PF13489 Methyltransferase domain    4&#xA;WP_XXXXXXX90.1  PF06325 Ribosomal protein L11 methyltransferase (PrmA)  5; PF06325  Ribosomal protein L11 methyltransferase (PrmA)  7  &#xA;WP_0XXXXXX62.1  PF13489 Methyltransferase domain    4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Expected output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;XXXXXXX45.1 PF02310 B12 binding domain  12; PF04055 Radical SAM superfamily 5&#xA;YP_343434.1&#xA;WP_XXXXXXXX2.1  PF06325 Ribosomal protein L11 methyltransferase (PrmA)  5&#xA;WP_XXXXXXX90.1  PF06325 Ribosomal protein L11 methyltransferase (PrmA)  5; PF06325  Ribosomal protein L11 methyltransferase (PrmA)  7&#xA;XP_222222222.2  &#xA;WP_0XXXXXX62.1  PF13489 Methyltransferase domain    4&#xA;WP_192XXXXXX.1  PF13489 Methyltransferase domain    4&#xA;WP_XXXXXXX29.1  PF13489 Methyltransferase domain    4&#xA;WP_0XXXXXXXX.1  PF13489 Methyltransferase domain    4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1224" LastEditorUserId="73" LastEditDate="2022-01-27T03:06:19.830" LastActivityDate="2022-01-27T03:06:19.830" Title="if, else statment gives back else value" Tags="|python|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18419" PostTypeId="2" ParentId="18417" CreationDate="2022-01-26T15:32:54.530" Score="3" Body="&lt;p&gt;Either the &lt;code&gt;zcat&lt;/code&gt; command or &lt;code&gt;gunzip -c&lt;/code&gt; will decompress the file in a streaming fashion to the standard output without writing the decompressed data to disk. Pipe the output to &lt;code&gt;cut&lt;/code&gt;, &lt;code&gt;sed&lt;/code&gt;, etc.&lt;/p&gt;&#xA;&lt;p&gt;Try&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zcat 00-All.vcf.gz | cut -f 3 | sed 's/[\t]/,/g' &amp;gt; cols.csv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gunzip -c 00-All.vcf.gz | cut -f 3 | sed 's/[\t]/,/g' &amp;gt; cols.csv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But if you just want a single 'column' (I presume you mean field), then you are better off just doing &lt;code&gt;bcftools query&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="96" LastEditDate="2022-01-26T16:05:22.977" LastActivityDate="2022-01-26T16:05:22.977" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18420" PostTypeId="2" ParentId="15947" CreationDate="2022-01-26T16:24:37.903" Score="1" Body="&lt;p&gt;I would guess it is not possible without channels unless you are using &lt;code&gt;beforeScript&lt;/code&gt; and &lt;code&gt;afterScript&lt;/code&gt;. I came up with this solution:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env nextflow&#xA;&#xA;nextflow.enable.dsl=2&#xA;&#xA;workflow {&#xA;  // C(B(A(&amp;quot;&amp;quot;)))&#xA;&#xA;  next = &amp;quot;&amp;quot;&#xA;  next = A(next)&#xA;  next = B(next)&#xA;  next = C(next)&#xA;}&#xA;&#xA;&#xA;process A{&#xA;  echo true&#xA;  input: val _&#xA;  output: val &amp;quot;&amp;quot;&#xA;&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  echo $task.process&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process B{&#xA;  echo true&#xA;  input: val _&#xA;  output: val &amp;quot;&amp;quot;&#xA;&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  echo $task.process&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process C{&#xA;  echo true&#xA;  input: val _&#xA;  output: val &amp;quot;&amp;quot;&#xA;&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  echo $task.process&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;By using workflows you can change the execution order. But it is still a hack and I would prefer something like a &lt;code&gt;next:&lt;/code&gt; keyword to specify the next process.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;And for those not interested in DSL2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process A {&#xA;  echo true&#xA;&#xA;  output:&#xA;  val &amp;quot;&amp;quot; into B&#xA;  &#xA;&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  echo &amp;quot;$task.process&amp;quot;&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process B {&#xA;  echo true&#xA;&#xA;  input:&#xA;  val _ from B&#xA;&#xA;  output:&#xA;  val &amp;quot;&amp;quot; into C&#xA;&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  echo &amp;quot;$task.process&amp;quot;&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process C {&#xA;  echo true&#xA;&#xA;  input:&#xA;  val _ from C&#xA;&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  echo &amp;quot;$task.process&amp;quot;&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Also, I am quite curious to see other solutions.&lt;/p&gt;&#xA;" OwnerUserId="14931" LastActivityDate="2022-01-26T16:24:37.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18421" PostTypeId="2" ParentId="18401" CreationDate="2022-01-26T19:34:50.797" Score="4" Body="&lt;p&gt;I inquired with the NCBI BLAST help desk today, and within an hour they provided a very helpful response.&lt;/p&gt;&#xA;&lt;p&gt;To summarize, when redundant records in the database are collapsed, the deflines (accessions + descriptions) of the original entries are placed on a single line separated by an invisible control character (Ctrl-A or &lt;code&gt;\x01&lt;/code&gt;). The &lt;code&gt;makeblastdb&lt;/code&gt; program will recognize this and associate multiple accessions (from the combined defline) and taxids (from the taxid_map file) to a single entry. If you are retrieving entries from e.g. nt or nr with &lt;code&gt;blastdbcmd&lt;/code&gt;, you can use the &lt;code&gt;-ctrl_a&lt;/code&gt; flag to format the data with these invisible control characters—by default the &lt;code&gt;&amp;gt;&lt;/code&gt; character is used.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;tl;dr&lt;/strong&gt; I can reproduce the behavior described in the original question by:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;creating a FASTA file containing an entry with multiple accessions separated by the invisible &lt;code&gt;\x01&lt;/code&gt; character&lt;/li&gt;&#xA;&lt;li&gt;create a taxid_map file mapping the accessions to different taxids&lt;/li&gt;&#xA;&lt;li&gt;format the FASTA file with &lt;code&gt;makeblastdb -taxid_map ...&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;perform a BLAST search with a query that matches the entry in question; &lt;code&gt;-outfmt &amp;quot;6 staxids&amp;quot;&lt;/code&gt; will return a string with multiple taxids.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="96" LastActivityDate="2022-01-26T19:34:50.797" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18422" PostTypeId="2" ParentId="18415" CreationDate="2022-01-26T19:47:37.637" Score="4" Body="&lt;p&gt;I think this could be close to what you're looking for: &lt;a href=&quot;http://glad4u.zhang-lab.org/index.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://glad4u.zhang-lab.org/index.php&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Paper: &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/23282288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GLAD4U: deriving and prioritizing gene lists from PubMed literature&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It works with gene names though, so I am not sure it's a 100% fit.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2022-01-26T19:47:37.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18423" PostTypeId="2" ParentId="18418" CreationDate="2022-01-26T20:29:35.410" Score="3" Body="&lt;p&gt;Following on your &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/18412/connect-same-ids-with-values-in-id-rows-result-of-effectiveeld/18413#18413&quot;&gt;last post&lt;/a&gt;, your initial file &lt;code&gt;test.txt&lt;/code&gt; looks like that:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;WP_XXXXXXXX2.1  PF06325 Ribosomal protein L11 methyltransferase (PrmA)  5&#xA;WP_192XXXXXX.1  PF13489 Methyltransferase domain    4&#xA;XXXXXXX45.1 PF02310 B12 binding domain  12&#xA;XXXXXXX45.1 PF04055 Radical SAM superfamily 5&#xA;WP_XXXXXXX29.1  PF13489 Methyltransferase domain    4&#xA;WP_0XXXXXXXX.1  PF13489 Methyltransferase domain    4&#xA;WP_XXXXXXX90.1  PF06325 Ribosomal protein L11 methyltransferase (PrmA)  5&#xA;WP_XXXXXXX90.1  PF06325 Ribosomal protein L11 methyltransferase (PrmA)  7&#xA;WP_0XXXXXX62.1  PF13489 Methyltransferase domain    4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And you have another file &lt;code&gt;order.txt&lt;/code&gt; that contains your ordered IDs:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;XXXXXXX45.1&#xA;YP_343434.1&#xA;WP_XXXXXXXX2.1&#xA;WP_XXXXXXX90.1&#xA;WP_0XXXXXX62.1&#xA;XP_222222222.2&#xA;WP_192XXXXXX.1&#xA;WP_XXXXXXX29.1&#xA;WP_0XXXXXXXX.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Since you want to merge by the first field in &lt;code&gt;test.txt&lt;/code&gt; and sort them according to &lt;code&gt;order.txt&lt;/code&gt;, you have to read first both of the files:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# read 'test.txt'&#xA;with open(&amp;quot;test.txt&amp;quot;,'r') as rf:&#xA;    l = rf.readlines()&#xA;&#xA;# read sorted IDs 'order.txt'&#xA;with open(&amp;quot;order.txt&amp;quot;,'r') as rf:&#xA;    l_order = rf.readlines()&#xA;&#xA;# remove newlines from each element in list&#xA;l_order = [i.strip() for i in l_order]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My previous answers describe how we obtain the merged fields in the file, so I will not go over it again. Since we need to sort the dictionary according to the list, we simply have to do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# for each element in ordered list&#xA;for i in l_order:&#xA;    if i in dict_id.keys():                        # if element is in my dictionary's keys&#xA;        print(i + &amp;quot;\t&amp;quot; + &amp;quot;; &amp;quot;.join(dict_id[i]))    # print the key and the value&#xA;    else:&#xA;        print(i)                                   # else prints the element&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This should print:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;XXXXXXX45.1     PF02310 B12 binding domain      12; PF04055     Radical SAM superfamily 5&#xA;YP_343434.1&#xA;WP_XXXXXXXX2.1  PF06325 Ribosomal protein L11 methyltransferase (PrmA)  5&#xA;WP_XXXXXXX90.1  PF06325 Ribosomal protein L11 methyltransferase (PrmA)  5; PF06325      Ribosomal protein L11 methyltransferase (PrmA)  7&#xA;WP_0XXXXXX62.1  PF13489 Methyltransferase domain        4&#xA;XP_222222222.2&#xA;WP_192XXXXXX.1  PF13489 Methyltransferase domain        4&#xA;WP_XXXXXXX29.1  PF13489 Methyltransferase domain        4&#xA;WP_0XXXXXXXX.1  PF13489 Methyltransferase domain        4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Full code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with open(&amp;quot;test.txt&amp;quot;,'r') as rf:&#xA;    l = rf.readlines()&#xA;&#xA;with open(&amp;quot;order.txt&amp;quot;,'r') as rf:&#xA;    l_order = rf.readlines()&#xA;&#xA;l_order = [i.strip() for i in l_order]&#xA;&#xA;dict_id = {}&#xA;for i in l:&#xA;    key_id = i.split(&amp;quot;\t&amp;quot;)[0]&#xA;    if key_id not in dict_id.keys():&#xA;        value = &amp;quot;\t&amp;quot;.join(i.split(&amp;quot;\t&amp;quot;)[1:]).rstrip()&#xA;        dict_id[key_id] = [value] &#xA;    else:&#xA;        dict_id[key_id].append(&amp;quot;\t&amp;quot;.join(i.split(&amp;quot;\t&amp;quot;)[1:]).rstrip())&#xA;&#xA;for i in l_order:&#xA;    if i in dict_id.keys():&#xA;        print(i + &amp;quot;\t&amp;quot; + &amp;quot;; &amp;quot;.join(dict_id[i]))&#xA;    else:&#xA;        print(i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;PS: please for further pure programming questions, do post your questions in StackOverflow instead of Bioinformatics.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="4413" LastEditDate="2022-01-26T20:34:41.910" LastActivityDate="2022-01-26T20:34:41.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18424" PostTypeId="1" AcceptedAnswerId="18426" CreationDate="2022-01-27T05:53:05.893" Score="0" ViewCount="44" Body="&lt;p&gt;I am trying to reporduce the the following article  &lt;a href=&quot;https://www.nature.com/articles/s41586-020-2905-5#Sec7&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chen, M. B., Jiang, X., Quake, S. R., &amp;amp; Südhof, T. C. (2020). Persistent transcriptional programmes are associated with remote memory. Nature, 587(7834), 437-442.&lt;/a&gt;.  In the methods section &lt;strong&gt;Bioinformatics and data analysis - Mapping to the genome&lt;/strong&gt;  they say &amp;quot;Next, the ‘canonical correlation analysis’ function from the Seurat package27 was used to align raw data from multiple experiments. &amp;quot;  But they had previously aligned the data and generated gene counts.  So by this do they mean anchor?  I'm confused why would some one run CCA on the raw unaligned data.  They only provided gene count data not raw data, but they use the CCA to restrict the gene's used from ~23000 down to just ~2500 of the more positively and negatively correlated (to the resulting CCA generated principle components) without providing the exact details of which genes they included or excluded before running their analysis.&lt;/p&gt;&#xA;&lt;p&gt;Can I just do the CCA on the count data and hope for the same results?  Have I misunderstood their filtering and alignment?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;  For clarity for future readers, my question boiled down to the the term alignment. I was confused because I thought alignment specifically referred to aligning sequenced fragments to a reference genome, here alignment basically refers to correcting for batch effects in scRNA-seq.&lt;/p&gt;&#xA;" OwnerUserId="2482" LastEditorUserId="2482" LastEditDate="2022-01-27T19:56:04.923" LastActivityDate="2022-01-27T19:56:04.923" Title="Integrating scRNA-seq data using raw data" Tags="|scrnaseq|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18426" PostTypeId="2" ParentId="18424" CreationDate="2022-01-27T06:30:12.860" Score="1" Body="&lt;p&gt;I believe when you say alignment, you mean aligning reads to a genome (sometimes to transcriptome) and count these to get count matrices. In the aforementioned paper, however, what is meant is &amp;quot;bringing different data sets to a level where they can be compared/integrated/...&amp;quot;. Basically scRNA-seq data are heavily prone to batch effetcs and if you try to integrate different data sets (let's say data generated with 10x 3' and 10x 5' chemistries), there is a good chance is most of the signal would come from the technical differences rather than biology, for example resulting in clustering by scRNA-seq technology and not cell type/stat, hence there are many batch-effect-removal/data-integration/data-alignment tools for scRNA-seq.&lt;/p&gt;&#xA;&lt;p&gt;Seurat has a nice CCA tutorial, you would not need the raw data, count data would be fine but you will need to be able to specify what data are coming from what source, the authors must have provided this information.&lt;/p&gt;&#xA;&lt;p&gt;When it comes to what an &amp;quot;anchor&amp;quot; is, you would need to read about the CCA technique but in a rather naïve way, you can think of these as commonalities between data sets or some kind of a reference and are used to &amp;quot;align&amp;quot; these different data sets.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, reducing the number of genes from tens of thousands to thousands is just fine and is not really specific to CCA: not all genes are expected to be expressed at all times in all cell types and hence are not that informative and also scRNA-seq is not able to pick up lowly-expressed genes, causing these to show up with 0 counts in the count matrix, so again not that informative/interesting for the downstream analysis steps.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2022-01-27T12:49:26.807" LastActivityDate="2022-01-27T12:49:26.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18430" PostTypeId="1" AcceptedAnswerId="18446" CreationDate="2022-01-27T10:13:48.250" Score="3" ViewCount="153" Body="&lt;p&gt;I am looking to establish the total number of complete phage genomes available on NCBI.&lt;/p&gt;&#xA;&lt;p&gt;I am not looking for any specific type, but want to understand the total diversity available.&lt;/p&gt;&#xA;&lt;p&gt;How can I perform the search?&lt;/p&gt;&#xA;" OwnerUserId="12644" LastEditorUserId="3848" LastEditDate="2022-02-04T23:40:21.860" LastActivityDate="2022-02-04T23:40:21.860" Title="How to get the number of complete phage genomes available on ncbi?" Tags="|genome|public-databases|phylogenetics|virus|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18431" PostTypeId="1" AcceptedAnswerId="18432" CreationDate="2022-01-27T11:37:27.983" Score="1" ViewCount="1140" Body="&lt;p&gt;I have a vcf file containing an ID field with a snps and a list of human SNPs from dbSNP.I want to remove whatever in dbSNP from vcf file. The text file is like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rs978760828&#xA;rs1008829651&#xA;rs1052373574&#xA;rs1326880612&#xA;rs768019142&#xA;rs892501864&#xA;rs1010989343&#xA;rs1489251879&#xA;rs1022805358&#xA;rs1322538365&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How to remove the common snps from dbSNP and my vcf?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="3694" LastEditDate="2022-01-27T12:33:21.717" LastActivityDate="2022-01-27T12:33:21.717" Title="Filtering a vcf with a text file of SNP rsIDs" Tags="|vcf|snp|bcftools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18432" PostTypeId="2" ParentId="18431" CreationDate="2022-01-27T12:12:00.847" Score="1" Body="&lt;p&gt;I'd advise against using &lt;code&gt;grep&lt;/code&gt;, &lt;code&gt;sed&lt;/code&gt;, &lt;code&gt;awk&lt;/code&gt; for processing vcfs unless you really know what you're doing - there's almost always an easier way using &lt;code&gt;bcftools&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;bcftools view -i'ID=@snplist.txt' in.vcf&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2022-01-27T12:12:00.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18433" PostTypeId="1" CreationDate="2022-01-27T12:55:59.463" Score="1" ViewCount="35" Body="&lt;p&gt;have a list of pdbid and I want to separate the dihedral angles of some sequences in protein .&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pdbid      st          en&#xA;101m    4.A.GLU 15.A.ALA&#xA;102m    4.A.GLU 15.A.ALA&#xA;103m    4.A.GLU 15.A.ALA&#xA;104m    4.A.GLU 15.A.ALA&#xA;105m    4.A.GLU 15.A.ALA&#xA;106m    4.A.GLU 15.A.ALA&#xA;107m    4.A.GLU 15.A.ALA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;st is start of sequences   en is end of sequences.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;And after separating the dihedral angles, I want to write the&#xA;corresponding pdbid in front of it.&lt;/p&gt;&#xA;&lt;p&gt;I wrote the following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  library(bio3d)&#xA;&#xA;  # PDB structure object as obtained from function read.pdb.&#xA; pdb=read.pdb(df$pdbid[1])&#xA;&#xA;  #Calculate all torsion angles for a given protein PDB structure object&#xA;tor=torsion.pdb(pdb)&#xA; &#xA;            phi        psi&#xA;1.A.VAL -144.91395  137.473333&#xA;2.A.LEU -95.02097   149.7834&#xA;3.A.SER -82.68263   166.676942&#xA;4.A.GLU -62.53142   -36.263406&#xA;5.A.GLY -59.64632   -42.035533&#xA;6.A.GLU -68.69144   -40.23538&#xA;7.A.TRP -63.52969   -36.769049&#xA;8.A.GLN -64.80557   -38.512821&#xA;9.A.LEU -64.82472   -42.143097&#xA;10.A.VAL    -63.77383   -50.465974&#xA;11.A.LEU    -69.74235   -25.698665&#xA;12.A.HIS    -70.86863   -41.810178&#xA;13.A.VAL    -74.99987   -33.340633&#xA;14.A.TRP    -63.20999   -32.690374&#xA;15.A.ALA    -61.13164   -30.030673&#xA;16.A.LYS    -75.94714   -43.271738&#xA;17.A.VAL    -54.76125   -43.181171&#xA;18.A.GLU    -64.33161   -12.649873&#xA;19.A.ALA    -70.38802   -22.502227&#xA;&#xA;   &#xA;    &#xA;#separating phi angles from &amp;quot;4.A.GLU&amp;quot; to &amp;quot;15.A.ALA&amp;quot;&#xA;phi=tor&lt;span class=&quot;math-container&quot;&gt;$phi[df$&lt;/span&gt;st[1]:df$en[1]]&#xA;&#xA;#separating psi angles&#xA;psi=tor&lt;span class=&quot;math-container&quot;&gt;$psi[df$&lt;/span&gt;st[1]:df$en[1]]&#xA;print(cbind(phi,psi))&#xA;&#xA;#Put the codes in the loop&#xA;for(i in 2:nrow(df))&#xA;{&#xA;  pdb=read.pdb(df&lt;span class=&quot;math-container&quot;&gt;$pdbid[i])&#xA;  tor=torsion.pdb(pdb)&#xA;  psi=tor$&lt;/span&gt;psi[df&lt;span class=&quot;math-container&quot;&gt;$st[i]:df$&lt;/span&gt;en[i]]&#xA;  phi=tor&lt;span class=&quot;math-container&quot;&gt;$phi[df$&lt;/span&gt;st[i]:df$en[i]]&#xA;  print(cbind(phi,psi))&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I get the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in df$st[1]:df$en[1] : NA/NaN argument&#xA;In addition: Warning messages:&#xA;1: NAs introduced by coercion &#xA;2: NAs introduced by coercion&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;str(df)&#xA;'data.frame':   9 obs. of  3 variables:&#xA; &lt;span class=&quot;math-container&quot;&gt;$ pdbid: chr  &quot;101m&quot; &quot;102m&quot; &quot;103m&quot; &quot;104m&quot; ...&#xA; $&lt;/span&gt; st   : chr  &amp;quot;4.A.GLU&amp;quot; &amp;quot;4.A.GLU&amp;quot; &amp;quot;4.A.GLU&amp;quot; &amp;quot;4.A.GLU&amp;quot; ...&#xA; &lt;span class=&quot;math-container&quot;&gt;$ en   : chr  &quot;15.A.ALA&quot; &quot;15.A.ALA&quot; &quot;15.A.ALA&quot; &quot;15.A.ALA&quot; ...&#xA;str(tor)&#xA;List of 11&#xA; $&lt;/span&gt; psi   : num [1:295] 158.1 137.5 149.8 166.7 -36.3 ...&#xA; &lt;span class=&quot;math-container&quot;&gt;$ phi   : num [1:295] NA -144.9 -95 -82.7 -62.5 ...&#xA; $&lt;/span&gt; omega : num [1:295] -179 179 177 179 179 ...&#xA; &lt;span class=&quot;math-container&quot;&gt;$ chi1  : num [1:295] 54.3 175.7 -81 62.3 175.1 ...&#xA; $&lt;/span&gt; chi2  : num [1:295] -171.5 NA 72.7 NA 64.4 ...&#xA; &lt;span class=&quot;math-container&quot;&gt;$ chi3  : num [1:295] -72.1 NA NA NA 17 ...&#xA; $&lt;/span&gt; chi4  : num [1:295] NA NA NA NA NA NA NA NA NA NA ...&#xA; &lt;span class=&quot;math-container&quot;&gt;$ chi5  : logi [1:295] NA NA NA NA NA NA ...&#xA; $&lt;/span&gt; alpha : num [1:295] NA -139.4 -123.2 -108.3 58.3 ...&#xA; &lt;span class=&quot;math-container&quot;&gt;$ coords: num [1:3, 1:17, 1:295] 24.28 8.37 -9.85 24.4 9.86 ...&#xA;  ..- attr(*, &quot;dimnames&quot;)=List of 3&#xA;  .. ..$&lt;/span&gt; xyz: chr [1:3] &amp;quot;x&amp;quot; &amp;quot;y&amp;quot; &amp;quot;z&amp;quot;&#xA;  .. ..&lt;span class=&quot;math-container&quot;&gt;$ atm: chr [1:17] &quot;N&quot; &quot;CA&quot; &quot;C&quot; &quot;O&quot; ...&#xA;  .. ..$&lt;/span&gt; res: chr [1:295] &amp;quot;  0.A.MET&amp;quot; &amp;quot;  1.A.VAL&amp;quot; &amp;quot;  2.A.LEU&amp;quot; &amp;quot;  3.A.SER&amp;quot; ...&#xA; &lt;span class=&quot;math-container&quot;&gt;$ tbl   : num [1:295, 1:7] NA -144.9 -95 -82.7 -62.5 ...&#xA;  ..- attr(*, &quot;dimnames&quot;)=List of 2&#xA;  .. ..$&lt;/span&gt; : chr [1:295] &amp;quot;  0.A.MET&amp;quot; &amp;quot;  1.A.VAL&amp;quot; &amp;quot;  2.A.LEU&amp;quot; &amp;quot;  3.A.SER&amp;quot; ...&#xA;  .. ..$ : chr [1:7] &amp;quot;phi&amp;quot; &amp;quot;psi&amp;quot; &amp;quot;chi1&amp;quot; &amp;quot;chi2&amp;quot; ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14941" LastEditorUserId="48" LastEditDate="2022-01-28T11:29:40.823" LastActivityDate="2022-01-28T11:29:40.823" Title="Separation of dihedral angles of the desired sequence in the protein" Tags="|r|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18434" PostTypeId="2" ParentId="18218" CreationDate="2022-01-27T13:46:19.433" Score="3" Body="&lt;p&gt;I ended up using &lt;code&gt;lolliplots&lt;/code&gt; from the &lt;a href=&quot;https://bioconductor.org/packages/devel/bioc/vignettes/trackViewer/inst/doc/trackViewer.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;trackViewer R package&lt;/a&gt;; it can do VCF-based visualization &lt;em&gt;and&lt;/em&gt; is flexible enough I can feed it data from other sources (I wound up parsing Nextclade mutation calls for the amino acids since I had these anyway).&lt;/p&gt;&#xA;" OwnerUserId="12755" LastActivityDate="2022-01-27T13:46:19.433" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18435" PostTypeId="1" AcceptedAnswerId="18443" CreationDate="2022-01-27T16:36:08.533" Score="-1" ViewCount="451" Body="&lt;p&gt;So I am trying to compute diversity indices for a dataset I have but the math doesn't work out and I don't really understand how to translate Shannon and/or Simpson diversity indices into something that is understandable. Maybe I got everything wrong, this is certainly a possibility. However, this is what I have up to now:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# load libraries&#xA;library(&amp;quot;phyloseq&amp;quot;)&#xA;data(&amp;quot;GlobalPatterns&amp;quot;)&#xA;&#xA;# load data&#xA;globp = GlobalPatterns&#xA;&#xA;# compute transformation according to link below&#xA;simpson_transformed &amp;lt;- 1/(1-estimate_richness(globp, measures=&amp;quot;Simpson&amp;quot;))&#xA;shannon_transformed &amp;lt;- exp(estimate_richness(globp, measures=&amp;quot;Shannon&amp;quot;))&#xA;&#xA;# print side by side&#xA;print(cbind.data.frame(simpson_transformed, shannon_transformed))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;strong&gt;comparing_indices&lt;/strong&gt; columns are computed according to what I found &lt;a href=&quot;http://www.loujost.com/Statistics%20and%20Physics/Diversity%20and%20Similarity/Diversity%20of%20a%20single%20community.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;at this link&lt;/a&gt; (italic is mine):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;They &lt;em&gt;[i.e. the values]&lt;/em&gt; should be converted to effective number of species,&#xA;which are the true diversities. By referring to Table 1 we see that&#xA;species richness is already a true diversity, so the true diversity&#xA;according to this index is 5.000 species. Table 1 shows that the&#xA;Shannon entropy or Shannon-Wiener index is converted to effective&#xA;number of species or true diversity by taking the exponential:&#xA;exp(1.609) is 5.000, so the true diversity according to the Shannon&#xA;entropy is also 5.000 species. The Gini-Simpson index is converted to&#xA;a  true diversity by subtracting it from unity and inverting:&#xA;1/(1-0.8) = 5.000 species also. So in fact all these indices agree&#xA;that the diversity of this community is 5.000 species.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I also verified how &lt;strong&gt;phyloseq&lt;/strong&gt; is computing the indices. I believe the transformation for Shannon should be the same mentioned above. I have doubts about the Simpson, since they refer to the Gini-Simpson index and not the Simpson itself. However, the &lt;a href=&quot;https://www.rdocumentation.org/packages/vegan/versions/2.4-2/topics/diversity&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; for &lt;strong&gt;vegan&lt;/strong&gt;'s &lt;code&gt;diversity()&lt;/code&gt; function, which is the function that (I believe) is called by &lt;strong&gt;phyloseq&lt;/strong&gt;'s &lt;code&gt;estimate_richness()&lt;/code&gt; (see &lt;a href=&quot;https://www.rdocumentation.org/packages/phyloseq/versions/1.16.2/topics/estimate_richness&quot; rel=&quot;nofollow noreferrer&quot;&gt;docs&lt;/a&gt;), mention that the Simpson index is returned as &lt;code&gt;1-D&lt;/code&gt; &lt;a href=&quot;https://en.wikipedia.org/wiki/Diversity_index#Gini%E2%80%93Simpson_index&quot; rel=&quot;nofollow noreferrer&quot;&gt;which seems to be the Gini-Simpson index&lt;/a&gt; (see the &lt;code&gt;1-lambda&lt;/code&gt;). In other words, it looks like the transformation I linked above should be exactly the one that are used in the code I provided. But, I was expecting the same values in the two columns. But they are not there and I wonder why this happens. What did I do wrong?&lt;/p&gt;&#xA;" OwnerUserId="3119" LastActivityDate="2022-01-28T11:38:24.847" Title="Translating Simpson and Shannon diversity indices?" Tags="|phyloseq|" AnswerCount="1" CommentCount="0" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18437" PostTypeId="2" ParentId="16376" CreationDate="2022-01-27T18:46:16.687" Score="1" Body="&lt;p&gt;While I can't explain it to you directly, you might check out the section on theta estimators in &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-14-289&quot; rel=&quot;nofollow noreferrer&quot;&gt;Korneliussen et al. (2013)&lt;/a&gt; which is cited on the above ANGSD wiki page. The equation for this estimator in this paper traces back to the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1203831/pdf/ge1233585.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;original Tajima (1989).&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2907" LastActivityDate="2022-01-27T18:46:16.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18438" PostTypeId="1" CreationDate="2022-01-27T19:18:13.173" Score="0" ViewCount="194" Body="&lt;p&gt;I have a PacBio HiFi assembly of 1.1 Gb from a heterozygous species. I have aligned this assembly against a reference genome which is around 0.9 Gb. I can see that there are quite a few INDELs, Duplications, Inverted regions in my Query sequence on the Y-axis (PacBio Hifi) compared to the reference genome.&lt;/p&gt;&#xA;&lt;p&gt;I want to check if these structural variations are real or only resulting from sequencing errors or not. So, I want to manually curate the PacBio assembly. Because I can see at which contig the structural variations are.&lt;/p&gt;&#xA;&lt;p&gt;Can you please suggest any tools for manual curation of a genome assembly to investigate whether these structural variations are sequencing errors or real sequence variation?&lt;/p&gt;&#xA;&lt;p&gt;I have uploaded the dot-plot below. Thank you.&lt;a href=&quot;https://i.stack.imgur.com/owOVb.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/owOVb.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12238" LastActivityDate="2022-01-27T19:18:13.173" Title="How to manually curate a genome assembly for sequence variation or error?" Tags="|genome|assembly|reference-genome|structural-variation|pacbio|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18440" PostTypeId="2" ParentId="18331" CreationDate="2022-01-27T22:21:50.133" Score="0" Body="&lt;p&gt;Just to add, usually the way to move a sequence object to a neat, decluttered string in Biopython is not to do this manually as done here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;title_splits=re.findall(r&amp;quot;[\w']+&amp;quot;, title)&#xA;sequence = &amp;quot; &amp;quot;.join(sequence) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;both being from &lt;code&gt;SimpleFastaParser(fastafile)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;It should be possible and, if so, better to state,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; str(title)&#xA; str(sequence)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you can issue a &lt;code&gt;print&lt;/code&gt; command and the string appears nice and neat on the screen &lt;code&gt;str()&lt;/code&gt; will work.&lt;/p&gt;&#xA;&lt;p&gt;This becomes important for complex sequence/alignment objects, such as SeqIO and AlignIO. The reason it works is &lt;code&gt;__str__&lt;/code&gt; has been declared or available through OOP inheritance.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-01-27T22:27:30.187" LastActivityDate="2022-01-27T22:27:30.187" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18441" PostTypeId="1" AcceptedAnswerId="18447" CreationDate="2022-01-28T00:46:28.667" Score="2" ViewCount="508" Body="&lt;p&gt;I would like to download all the PDB files which have crystal structure information in them. But the query builder seems to require me to enter the PDB IDs manually.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to get a list of those PDB IDs at RCSB PDB?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="14948" LastActivityDate="2023-01-12T13:03:11.937" Title="Is there a way to get a list of all PDB IDs for proteins with crystal structure information?" Tags="|pdb|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18442" PostTypeId="1" CreationDate="2022-01-28T08:22:05.637" Score="1" ViewCount="728" Body="&lt;p&gt;Hope anyone can help a beginner here. I'm building a proof-of concept tensorflow classifier for DNA sequences. However, the NN model does not let through train and test vectors saying the matrix size is incompatible. Where could be the mistake in my implementation?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import pandas as pd&#xA;&#xA;df=pd.DataFrame(columns=['sequences', 'label'])&#xA;a='ttttccagaattctcttagttt gtgatgtctttattgcttctattt'&#xA;b='ctcctgcttgctttttttcttg ggtttctgatattctttaaaggat'&#xA;c='tcctgcttgctttttttcttgg gtttctgatattctttaaaggatt'&#xA;df.sequences=[a,b,c]&#xA;df.label=[1,1,1]&#xA;&#xA;from sklearn.preprocessing import LabelEncoder, OneHotEncoder&#xA;integer_encoder = LabelEncoder()&#xA;one_hot_encoder = OneHotEncoder(categories='auto')&#xA;input_features = []&#xA;sequences=df.sequences&#xA;for sequence in sequences:&#xA;     integer_encoded = integer_encoder.fit_transform(list(sequence))&#xA;     integer_encoded = np.array(integer_encoded).reshape(-1, 1)&#xA;     one_hot_encoded = one_hot_encoder.fit_transform(integer_encoded)&#xA;     input_features.append(one_hot_encoded.toarray())&#xA;&#xA;np.set_printoptions(threshold=40)&#xA;input_features = np.stack(input_features)&#xA;print(&amp;quot;Example sequence\n-----------------------&amp;quot;)&#xA;print('DNA Sequence #1:\n',sequences[0][:10],'...',sequences[0][-10:])&#xA;print('One hot encoding of Sequence #1:\n',input_features[0].T)&#xA;&#xA;labels=df.label&#xA;one_hot_encoder = OneHotEncoder(categories='auto')&#xA;labels = np.array(labels).reshape(-1, 1)&#xA;input_labels = one_hot_encoder.fit_transform(labels).toarray()&#xA;print('Labels:\n',labels.T)&#xA;print('One-hot encoded labels:\n',input_labels.T)&#xA;&#xA;from sklearn.model_selection import train_test_split&#xA;train_features, test_features, train_labels, test_labels = train_test_split(input_features, input_labels, test_size=0.25, random_state=42)&#xA;&#xA;&#xA;def get_batch(x_data, y_data, batch_size):&#xA;     idxs = np.random.randint(0, len(y_data), batch_size)&#xA;     return x_data[idxs,:,:], y_data[idxs]&#xA;&#xA;epochs = 10&#xA;batch_size = 100&#xA;&#xA;x_train = train_features&#xA;x_test = train_labels&#xA;x_train = x_train / 255.0&#xA;x_test = x_test / 255.0&#xA;x_test = tf.Variable(x_test)&#xA;&#xA;W1 = tf.Variable(tf.random.normal([784, 300], stddev=0.03), name='W1')&#xA;b1 = tf.Variable(tf.random.normal([300]), name='b1')&#xA;W2 = tf.Variable(tf.random.normal([300, 10], stddev=0.03), name='W2')&#xA;b2 = tf.Variable(tf.random.normal([10]), name='b2')&#xA;&#xA;def nn_model(x_input, W1, b1, W2, b2): &#xA;     x_input = tf.reshape(x_input, (x_input.shape[0], -1))&#xA;     x = tf.add(tf.matmul(tf.cast(x_input, tf.float32), W1), b1) #ERROR FROM HERE?&#xA;     x = tf.nn.relu(x)&#xA;     logits = tf.add(tf.matmul(x, W2), b2)&#xA;     return logits&#xA;&#xA;def loss_fn(logits, labels):&#xA;     cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=labels,logits=logits))&#xA;     return cross_entropy&#xA;&#xA;optimizer = tf.keras.optimizers.Adam()&#xA;total_batch = int(len(y_train) / batch_size)&#xA;&#xA;for epoch in range(epochs):&#xA;    avg_loss = 0&#xA;    for i in range(total_batch):&#xA;        batch_x, batch_y = get_batch(x_train, y_train, batch_size=batch_size)&#xA;        # create tensors&#xA;        batch_x = tf.Variable(batch_x)&#xA;        batch_y = tf.Variable(batch_y)&#xA;        # create a one hot vector&#xA;        batch_y = tf.one_hot(batch_y, 10)&#xA;        with tf.GradientTape() as tape:&#xA;            logits = nn_model(batch_x, W1, b1, W2, b2)&#xA;            loss = loss_fn(logits, batch_y)&#xA;        gradients = tape.gradient(loss, [W1, b1, W2, b2])&#xA;        optimizer.apply_gradients(zip(gradients, [W1, b1, W2, b2]))&#xA;        avg_loss += loss / total_batch&#xA;    test_logits = nn_model(x_test, W1, b1, W2, b2) ##ERROR LOG APPEARS HERE&#xA;    max_idxs = tf.argmax(test_logits, axis=1)&#xA;    test_acc = np.sum(max_idxs.numpy() == y_test) / len(y_test)&#xA;    print(f&amp;quot;Epoch: {epoch + 1}, loss={avg_loss:.3f}, test set accuracy = {test_acc*100:.3f}%&amp;quot;)&#xA;print(&amp;quot;\nTraining complete!&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Traceback (most recent call last): File &amp;quot;&amp;quot;, line 16, in File &amp;quot;&amp;quot;, line&#xA;4, in nn_model File&#xA;&amp;quot;/usr/local/lib/python3.8/dist-packages/tensorflow/python/util/traceback_utils.py&amp;quot;,&#xA;line 153, in error_handler raise e.with_traceback(filtered_tb) from&#xA;None File&#xA;&amp;quot;/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py&amp;quot;,&#xA;line 7107, in raise_from_not_ok_status raise&#xA;core._status_to_exception(e) from None # pylint:&#xA;disable=protected-access&#xA;tensorflow.python.framework.errors_impl.InvalidArgumentError: Matrix&#xA;size-incompatible: In[0]: [2,1], In[1]: [784,300] [Op:MatMul]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="14354" LastActivityDate="2024-02-18T21:02:20.293" Title="tensorflow nn_model for DNA sequences: Matrix size-incompatible: In[0]: [2,1], In[1]: [784,300]" Tags="|python|genome|sequence-analysis|encode|tensorflow|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18443" PostTypeId="2" ParentId="18435" CreationDate="2022-01-28T11:38:24.847" Score="2" Body="&lt;p&gt;You are trying to obtain Hill numbers, sometimes called true diversity, and you have different results because you have uneven species frequencies.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Vegan&lt;/code&gt; calculates Shannon index like:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ H' = -\sum_{i=1}^R p_i \ln p_i $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;And Gini-Simpson like:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ 1 - \lambda = 1 - \sum_{i=1}^R p_i^2 $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;So the Hill numbers should be:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ H1 = exp(H') $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ H2 = 1 / 1 - \lambda $$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;With &lt;code&gt;Phyloseq&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;H1 &amp;lt;- 1 / estimate_richness(globp, measures=&amp;quot;Simpson&amp;quot;)&#xA;H2 &amp;lt;- exp(estimate_richness(globp, measures=&amp;quot;Shannon&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2022-01-28T11:38:24.847" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18444" PostTypeId="1" AcceptedAnswerId="18445" CreationDate="2022-01-28T15:07:35.630" Score="-4" ViewCount="255" Body="&lt;p&gt;I am trying to making a fasta doc using GATK tools but I don't know what does this error say&lt;/p&gt;&#xA;&lt;p&gt;Server2:/data2/RNASeq/Angel/gatk&lt;span class=&quot;math-container&quot;&gt;$ ./gatk CreateSequenceDictionary -R reference.fa&#xA;    Using GATK jar /data2/RNASeq/Angel/gatk/gatk-package-4.2.4.1-local.jar&#xA;    Running:&#xA;        java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /data2/RNASeq/Angel/gatk/gatk-package-4.2.4.1-local.jar CreateSequenceDictionary -R reference.fa&#xA;    INFO    2022-01-28 15:04:19 CreateSequenceDictionary    Output dictionary will be written in reference.dict&#xA;    15:04:19.401 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/data2/RNASeq/Angel/gatk/gatk-package-4.2.4.1-local.jar!/com/intel/gkl/native/libgkl_compression.so&#xA;    [Fri Jan 28 15:04:19 GMT 2022] CreateSequenceDictionary --REFERENCE reference.fa --TRUNCATE_NAMES_AT_WHITESPACE true --NUM_SEQUENCES 2147483647 --VERBOSITY INFO --QUIET false --VALIDATION_STRINGENCY STRICT --COMPRESSION_LEVEL 2 --MAX_RECORDS_IN_RAM 500000 --CREATE_INDEX false --CREATE_MD5_FILE false --GA4GH_CLIENT_SECRETS client_secrets.json --help false --version false --showHidden false --USE_JDK_DEFLATER false --USE_JDK_INFLATER false&#xA;    Jan 28, 2022 3:04:19 PM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine&#xA;    INFO: Failed to detect whether we are running on Google Compute Engine.&#xA;    [Fri Jan 28 15:04:19 GMT 2022] Executing as Server2 on Linux 4.15.0-166-generic amd64; OpenJDK 64-Bit Server VM 11.0.13+8-Ubuntu-0ubuntu1.18.04; Deflater: Intel; Inflater: Intel; Provider GCS is available; Picard version: Version:4.2.4.1&#xA;    [Fri Jan 28 15:04:19 GMT 2022] picard.sam.CreateSequenceDictionary done. Elapsed time: 0.00 minutes.&#xA;    Runtime.totalMemory()=2155872256&#xA;    To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp&#xA;    picard.PicardException: /data2/RNASeq/Angel/gatk/reference.dict already exists.  Delete this file and try again, or specify a different output file.&#xA;        at picard.sam.CreateSequenceDictionary.doWork(CreateSequenceDictionary.java:220)&#xA;        at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:308)&#xA;        at org.broadinstitute.hellbender.cmdline.PicardCommandLineProgramExecutor.instanceMain(PicardCommandLineProgramExecutor.java:37)&#xA;        at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)&#xA;        at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)&#xA;        at org.broadinstitute.hellbender.Main.main(Main.java:289)&#xA;    Server2:/data2/RNASeq/Angel/gatk$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2022-01-28T15:53:40.340" LastActivityDate="2022-01-28T15:53:40.340" Title="Getting error in running GATK" Tags="|gatk|picard|wes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18445" PostTypeId="2" ParentId="18444" CreationDate="2022-01-28T15:50:24.507" Score="2" Body="&lt;pre&gt;&lt;code&gt;picard.PicardException: /data2/RNASeq/Angel/gatk/reference.dict already exists.  Delete this file and try again, or specify a different output file.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I think that requires no further explanation.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2022-01-28T15:50:24.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18446" PostTypeId="2" ParentId="18430" CreationDate="2022-01-28T17:15:42.767" Score="5" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Go to the NCBI nuccore database, at &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore&lt;/a&gt;&#xA;Copy and paste the search term:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;(bacterial virus[Organism]) AND (3000:110000[sequence length]) NOT bacteria[Organism]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;You'll get 74583 results.&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Click on the link &lt;code&gt;send to&lt;/code&gt; and click the checkbox called &lt;code&gt;file&lt;/code&gt;.&#xA;It will then download.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Check the downloaded file is in &lt;code&gt;fasta format&lt;/code&gt; with &amp;quot;&amp;gt;&amp;quot; to start the ID.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If fasta format is absent, return to the webpage and where it states &lt;code&gt;summary&lt;/code&gt; switch that to &lt;code&gt;fasta text&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I can't remember if point 4 is needed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Sequence length, I am guessing that all phages are between 3000 and 105000 in length, this is to screen out partial sequences and to minimise  bacteria creeping in to the search. I don't quite understand how a phage can be 105 000bp in length, but you can twiddle these parameters to refine the search.&lt;/li&gt;&#xA;&lt;li&gt;@acvill informs us that giant phages greater than &amp;gt;200 kb have been annoated. Thus a separate search is likely required targeting this genome size.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-02T19:02:20.133" LastActivityDate="2022-02-02T19:02:20.133" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18447" PostTypeId="2" ParentId="18441" CreationDate="2022-01-28T17:17:00.810" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Comment migrated as answer as it inadvertently may have been an answer.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If the sole records in a PDB file that are of interest are &lt;code&gt;EXPDTA&lt;/code&gt;,&lt;code&gt;CRYST1&lt;/code&gt; ORIGX1``,&lt;code&gt;ORIGX2&lt;/code&gt;, &lt;code&gt;ORIGX3&lt;/code&gt;,&lt;code&gt;SCALE1&lt;/code&gt;,&lt;code&gt;SCALE2&lt;/code&gt;,&lt;code&gt;SCALE3&lt;/code&gt;, the in the &lt;a href=&quot;http://wwpdb.org/ftp/pdb-ftp-sites&quot; rel=&quot;nofollow noreferrer&quot;&gt;FTP site&lt;/a&gt; the file ftp.wwpdb.org/pub/pdb/derived_data/index/crystal.idx  contains this information in a tabular format.&lt;/p&gt;&#xA;&lt;p&gt;If an extra piece of metadata is sought, such as &lt;strong&gt;some of&lt;/strong&gt; the refinement details in &lt;code&gt;REMARK&lt;/code&gt; lines, then &lt;a href=&quot;https://www.ebi.ac.uk/pdbe/api/doc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the PDBe API can help&lt;/a&gt;. In particular, the &lt;code&gt;experiment&lt;/code&gt; entrypoint (&lt;a href=&quot;https://www.ebi.ac.uk/pdbe/api/pdb/entry/experiment/451c&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt;) is of interest.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2022-01-28T17:17:00.810" CommentCount="0" CommunityOwnedDate="2022-01-28T17:17:00.810" ContentLicense="CC BY-SA 4.0" />
  <row Id="18448" PostTypeId="1" AcceptedAnswerId="18465" CreationDate="2022-01-28T18:49:02.827" Score="1" ViewCount="48" Body="&lt;p&gt;I have an experiment with 4 conditions in it two of them are controls.  When I am correcting for batch effects (either with PCA or CCA) should I restrict my comparisons between particular groups?  So first I should do control 1 with control 2 then hopefully there will be no DEGs between them.  Next I could do Control 2 vs Condition 1 then condition 1 vs condition2 and control 2 vs condtion 2.  Sorry if this is confusing but I am trying to avoid going into the experimental details in the interest of keeping this post brief.&lt;/p&gt;&#xA;&lt;p&gt;As a sort of follow up question can anyone suggest the &amp;quot;standard&amp;quot; or best seurat method for correcting batch effects.  I've found some tutorials that recommend using CCA and others suggest PCA.&lt;/p&gt;&#xA;" OwnerUserId="2482" LastEditorUserId="104" LastEditDate="2022-02-18T19:06:59.887" LastActivityDate="2022-02-18T19:06:59.887" Title="(scRNA-seq) I have a dataset with 4 conditions (experiments) should I be anchoring aka integrating data for each comparison when looking for DEGs?" Tags="|scrnaseq|seurat|single-cell|batch-effects|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18449" PostTypeId="2" ParentId="18442" CreationDate="2022-01-28T19:15:23.490" Score="0" Body="&lt;p&gt;The sci-kit learn looks fine, for a proof of concept. Furthermore, the deep learning looks good, e.g. Adams, loss etc .. all good. Of course it at ain't RNN or CNN but its proof of concept.&lt;/p&gt;&#xA;&lt;p&gt;However, the error you encountered suggests it is about resizing the numpy array and that is better answered on Stackoverflow in the absence of an answer here. Dealing with &lt;code&gt;pandas&lt;/code&gt; is one thing, munging a &lt;code&gt;numpy array&lt;/code&gt; is pretty hardcore. Its a single line of code, but you've got to know it exactly.&lt;/p&gt;&#xA;&lt;p&gt;An example is where I d focus,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tf.reshape(x_input, (x_input.shape[0], -1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Before or whether you need to post there please check:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Where is &lt;code&gt;import tensorflow as tf&lt;/code&gt;  ?&lt;/li&gt;&#xA;&lt;li&gt;Using keras without e.g. &lt;code&gt;from tensorflow.keras import optimizers&lt;/code&gt; (well okay you can do that given point 1 but generally the keras method is specifically imported)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;If I haven't misread the code it will fall over, as soon as it gets past &lt;code&gt;sci-kit learn&lt;/code&gt;.  The advantage of placing all imports at the top of the code is that its easy to read whats going on. There is another issue, but if a big deal Stackoverflow (or someone else here) will flag it.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Just to we explain point 2, I suspect the tensor flow on question is Tensorflow 2 if this is true then  Keras is built in and thus tf.keras is fine&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-01-29T15:53:16.637" LastActivityDate="2022-01-29T15:53:16.637" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18451" PostTypeId="1" CreationDate="2022-01-29T11:00:06.410" Score="0" ViewCount="38" Body="&lt;p&gt;I am confused about the sequences one starts with to generate the PAM matrices.&lt;/p&gt;&#xA;&lt;p&gt;The sequences to start with are supposed to diverge by no more than 1% (in sequence identity) wrt a common &amp;quot;inferred&amp;quot; ancestor. - if I understood correctly.&lt;/p&gt;&#xA;&lt;p&gt;How is the common ancestor sequence inferred without having the means to do any sequence alignment to start with?&lt;/p&gt;&#xA;&lt;p&gt;Does this mean that a PAM matrix is specific for an inferred ancestor, i.e. a protein family? - I got the impression there was only one in practice.&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance,&lt;/p&gt;&#xA;&lt;p&gt;Gordon&lt;/p&gt;&#xA;" OwnerUserId="14957" LastActivityDate="2024-02-18T20:04:00.463" Title="Are there different PAM matrices for different protein families? - How is the inferred common ancestor known to start with?" Tags="|sequence-alignment|point-mutation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18452" PostTypeId="2" ParentId="18451" CreationDate="2022-01-29T15:39:18.253" Score="0" Body="&lt;p&gt;Ideally, but I wouldn't worry because the chance of meeting the criteria in practice are slim. Most importantly you need a tree which is 'independently' derived from the protein sequence family in question. The PAM matrix is absolutely dependent on this. Generating a &amp;quot;true&amp;quot; tree without reference to the sequences you are studying isn't trivial, so it's got to generated with a different data set and the question is whether that data set gives a true tree, if you're stuck the DNA sequence of is a rough approximation (not ideal but you could justify it). If you can adequately do this, I wouldn't worry about the remaining assumptions because it's simply an optimisation which can't proven.&lt;/p&gt;&#xA;&lt;p&gt;The PAM matrix generation will estimate the sequence of a common ancestor, based on a given tree. What this criteria is saying is that if you want a 'perfect matrix' the sequence reconstruction of the ancestral node needs 'x' because that is pivotal to calculating the matrix. If that's a bit out your matrix is a bit by a margin of error that isn't quantified (maybe it has been, see below)&lt;/p&gt;&#xA;&lt;p&gt;Whether you can get get something close to the 'hub' sequence within your data set is one question and whether that sequence was included in the first place is all trial and error. I would assume the criteria are ascertained has been set by simulation studies (its fairly easy to do if you've got the time), which only affects the margin of confidence within the confines of that given simulation. PAM matrix generation has been around since the dawn of time and founding principle is an independent tree, everything  else you can calculate and arm-wave, ie admit the calculation may not be so precise. In other it doesn't stop you doing it and presenting the results, that is perfectly valid, as long as simulated criteria are understood. The problem would be is if you ignore a past study and present your results 'without fault'. The biggest problem would be if you generated a tree and used the same data.to generate the matrix, because that is whatever matrix you used to generate the tree and that's a no no.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-01-29T16:04:30.317" LastActivityDate="2022-01-29T16:04:30.317" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18453" PostTypeId="1" AcceptedAnswerId="18454" CreationDate="2022-01-29T16:30:20.243" Score="2" ViewCount="245" Body="&lt;p&gt;I'm running analysis with OrthoFinder and it produces a single-copy gene tree. When I visualize the tree with iTOL, there's no bootstrap value on the branches or nodes. There's a published paper in BMC Genomics that also didn't show the bootstrap value using OrthoFinder.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bmcgenomics.biomedcentral.com/articles/10.1186/s12864-019-6065-7#Tab1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bmcgenomics.biomedcentral.com/articles/10.1186/s12864-019-6065-7#Tab1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Does anybody know why there's no bootstrap values present on the phylogeny?&lt;/li&gt;&#xA;&lt;li&gt;How can this be corrected?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="14560" LastEditorUserId="3848" LastEditDate="2022-01-30T14:57:22.680" LastActivityDate="2022-01-30T14:57:22.680" Title="No bootstrap value on single-copy gene tree created by OrthoFinder" Tags="|phylogenetics|phylogeny|orthofinder|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18454" PostTypeId="2" ParentId="18453" CreationDate="2022-01-29T19:42:30.917" Score="1" Body="&lt;p&gt;Please check the file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; `$PWD/SpeciesTree_unrooted.txt&amp;quot; `&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You should find the bootstrap support* values per branch there. If not make sure is the latest version is installed from &lt;a href=&quot;https://github.com/davidemms/OrthoFinder&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/davidemms/OrthoFinder&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The papers are: genomebiology.biomedcentral.com/articles/10.1186/… and genomebiology.biomedcentral.com/articles/10.1186/… Genome Biology 16:157 ...&#xA;It officially offers tree stats:&#xA;&amp;quot;OrthoFinder also provides comprehensive statistics for comparative genomic analyses&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;The documentation is &lt;a href=&quot;https://davidemms.github.io/orthofinder_tutorials/exploring-orthofinders-results.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The question was raised how to increase replicates by one, two or three logs. This is not really orthodox technique. The way the issue should be first addressed is the user performs replicates manually, e.g. 10 new runs, and then looks across the consistency of bootstrap values for each manual replicate. If all values of interest between manual replicates are &amp;gt;80% or &amp;lt;70% it really doesn't matter. However if a key branch ranges from 70 - 75% that is strictly unresolved and would generally be resolved using an alternative test. If the variation is &amp;gt;= 10% that would need addressing. Generally variation between replicates is very low =&amp;lt; 2%&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;note, the original answer just contained the word 'support' and has now been more correctly described as &amp;quot;bootstrap support&amp;quot;. Thanks to the OP for pointing this out in the comments below.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-01-30T14:42:26.487" LastActivityDate="2022-01-30T14:42:26.487" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18456" PostTypeId="1" CreationDate="2022-01-30T09:32:50.097" Score="1" ViewCount="125" Body="&lt;p&gt;I have calculated the dN,dS ratio for a SARS-COV-2 data set ('NC_045512.2' (Wuhan strain) and 'LC666924.1'), and results in a zero value (for most of the genes).&lt;/p&gt;&#xA;&lt;p&gt;Is it possible and what is the interpretation, for example negative selection?&lt;/p&gt;&#xA;" OwnerUserId="14960" LastEditorUserId="3848" LastEditDate="2022-02-21T18:31:42.553" LastActivityDate="2022-02-21T18:31:42.553" Title="Bio.codonalign.codonseq module (cal_dn_ds) for SARS-COV-2" Tags="|python|phylogenetics|phylogeny|sars-cov-2|covid-19|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18457" PostTypeId="2" ParentId="18456" CreationDate="2022-01-30T14:11:16.840" Score="1" Body="&lt;p&gt;Yes that is the expected result, because there are no amino acid mutations in the data sets you are using. Thus the numerator is 0 and anything divided by zero is always zero, even if every codon had a synonymous mutation.&lt;/p&gt;&#xA;&lt;p&gt;The most precise interpretation is simply there are no amino acid mutations. It doesn't matter how you reword that, all divergence is resulting from synonymous mutations.&lt;/p&gt;&#xA;&lt;p&gt;The only thing to watch out for is a rounding error, i.e. a single amino acid change is dropping off detection because dN/dS is incorrectly rounded.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;A post was raised (now deleted) about what does that say about selection. dN/dS theory states that if it is significantly &amp;gt;1 thats positive selection .. etc. However, a truer representation of the data is there is no amino acid mutation, rather than their is a significant excess of synonymous to non-synonymous mutations. A zero value makes things tricky, and is exemplified by two hypothetical data sets one where every codon had one synoymous mutation (strong negative selection) compared against one that had a single synonymous mutation (little mutation) and both data sets had zero amino acid mutations - both data sets would have a dN/dS of zero but that is not a reflection of negative selection.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To answer the point below a single mutation is not a great basis draw statistical significance and the dN/dS ratio is not a good test if either the numerator or denominator are zero.&lt;/p&gt;&#xA;&lt;p&gt;Most data is under negative selection in strict accordance with dN/dS.&lt;/p&gt;&#xA;&lt;p&gt;Neutral selection is extremely rare.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; Sometimes the statistical analysis of a data set does not say very much ... and sometimes we have to accept that as an answer.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-01T13:36:39.410" LastActivityDate="2022-02-01T13:36:39.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18459" PostTypeId="1" CreationDate="2022-01-31T13:20:02.650" Score="0" ViewCount="277" Body="&lt;p&gt;I am trying to run Mutect2 on a .bam file but I get an error by Googling can not be tackled&lt;/p&gt;&#xA;&lt;p&gt;Have you seen this error?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fi1d18@RBGO-Server2:/data2/RNASeq/Angel/gatk$ ./gatk Mutect2 -R reference.fa -I sample.bam --germline-resource af-only-gnomad.hg38.vcf.gz --panel-of-normals pon.vcf.gz -O single_sample.vcf.gz&#xA;Using GATK jar /data2/RNASeq/Angel/gatk/gatk-package-4.2.4.1-local.jar&#xA;Running:&#xA;    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /data2/RNASeq/Angel/gatk/gatk-package-4.2.4.1-local.jar Mutect2 -R reference.fa -I sample.bam --germline-resource af-only-gnomad.hg38.vcf.gz --panel-of-normals pon.vcf.gz -O single_sample.vcf.gz&#xA;13:16:43.616 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/data2/RNASeq/Angel/gatk/gatk-package-4.2.4.1-local.jar!/com/intel/gkl/native/libgkl_compression.so&#xA;Jan 31, 2022 1:16:43 PM shaded.cloud_nio.com.google.auth.oauth2.ComputeEngineCredentials runningOnComputeEngine&#xA;INFO: Failed to detect whether we are running on Google Compute Engine.&#xA;13:16:43.733 INFO  Mutect2 - ------------------------------------------------------------&#xA;13:16:43.734 INFO  Mutect2 - The Genome Analysis Toolkit (GATK) v4.2.4.1&#xA;13:16:43.734 INFO  Mutect2 - For support and documentation go to https://software.broadinstitute.org/gatk/&#xA;13:16:43.734 INFO  Mutect2 - Executing as fi1d18@RBGO-Server2 on Linux v4.15.0-166-generic amd64&#xA;13:16:43.734 INFO  Mutect2 - Java runtime: OpenJDK 64-Bit Server VM v11.0.13+8-Ubuntu-0ubuntu1.18.04&#xA;13:16:43.734 INFO  Mutect2 - Start Date/Time: 31 January 2022 at 13:16:43 GMT&#xA;13:16:43.734 INFO  Mutect2 - ------------------------------------------------------------&#xA;13:16:43.734 INFO  Mutect2 - ------------------------------------------------------------&#xA;13:16:43.735 INFO  Mutect2 - HTSJDK Version: 2.24.1&#xA;13:16:43.735 INFO  Mutect2 - Picard Version: 2.25.4&#xA;13:16:43.735 INFO  Mutect2 - Built for Spark Version: 2.4.5&#xA;13:16:43.735 INFO  Mutect2 - HTSJDK Defaults.COMPRESSION_LEVEL : 2&#xA;13:16:43.735 INFO  Mutect2 - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false&#xA;13:16:43.735 INFO  Mutect2 - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true&#xA;13:16:43.735 INFO  Mutect2 - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false&#xA;13:16:43.735 INFO  Mutect2 - Deflater: IntelDeflater&#xA;13:16:43.735 INFO  Mutect2 - Inflater: IntelInflater&#xA;13:16:43.735 INFO  Mutect2 - GCS max retries/reopens: 20&#xA;13:16:43.735 INFO  Mutect2 - Requester pays: disabled&#xA;13:16:43.735 INFO  Mutect2 - Initializing engine&#xA;13:16:44.015 INFO  Mutect2 - Shutting down engine&#xA;[31 January 2022 at 13:16:44 GMT] org.broadinstitute.hellbender.tools.walkers.mutect.Mutect2 done. Elapsed time: 0.01 minutes.&#xA;Runtime.totalMemory()=2155872256&#xA;***********************************************************************&#xA;&#xA;A USER ERROR has occurred: Cannot read file:///data2/RNASeq/Angel/gatk/pon.vcf.gz because no suitable codecs found&#xA;&#xA;***********************************************************************&#xA;Set the system property GATK_STACKTRACE_ON_USER_EXCEPTION (--java-options '-DGATK_STACKTRACE_ON_USER_EXCEPTION=true') to print the stack trace.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also get this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fi1d18@RBGO-Server2:/data2/RNASeq/Angel/gatk$ file pon.vcf.gz&#xA;pon.vcf.gz: HTML document, ASCII text, with very long lines&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="57" LastEditDate="2022-01-31T18:42:37.253" LastActivityDate="2022-01-31T18:43:47.647" Title="Error in running Mutect2" Tags="|gatk|mutations|wes|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18460" PostTypeId="2" ParentId="18459" CreationDate="2022-01-31T13:57:38.133" Score="2" Body="&lt;p&gt;Your input file &lt;code&gt;pon.vcf.gz&lt;/code&gt; should be a gzip file, but from your &lt;code&gt;file&lt;/code&gt; command it looks like it is an HTML file instead. Perhaps you downloaded the VCF file incorrectly?&lt;/p&gt;&#xA;&lt;p&gt;Here is what I get on a bona fide gzip file in Ubuntu:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$ echo hi | gzip -c &amp;gt; hi.gz&#xA;$ file hi.gz&#xA;hi.gz: gzip compressed data, from Unix, original size modulo 2^32 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="492" LastActivityDate="2022-01-31T13:57:38.133" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18461" PostTypeId="2" ParentId="18459" CreationDate="2022-01-31T15:55:35.780" Score="1" Body="&lt;p&gt;Downloading that vcf file from this link fixes the problem (but generates a new one)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;wget https://storage.googleapis.com/gatk-best-practices/somatic-hg38/1000g_pon.hg38.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="57" LastEditDate="2022-01-31T18:43:47.647" LastActivityDate="2022-01-31T18:43:47.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18462" PostTypeId="1" AcceptedAnswerId="18468" CreationDate="2022-01-31T16:31:27.897" Score="0" ViewCount="46" Body="&lt;p&gt;NOTE: I posted the same question on &lt;a href=&quot;https://www.biostars.org/p/9507222/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt; but so far no replies there so I figured I would ask here, too.&lt;/p&gt;&#xA;&lt;p&gt;So, I was running a pipeline which I developed some time ago and a very weird behaviour was found, which had quite an impact over our analyses.&lt;/p&gt;&#xA;&lt;p&gt;Indeed, the scater's runTSNE() function was subjected to a change in the parameters names and, in particular, the old use_dimred parameter now has a new name, which is dimred.&lt;/p&gt;&#xA;&lt;p&gt;Interestingly, I was running the pipeline using the runTSNE() function and passing the dimensionality reduction through use_dimred which is apparently deprecated and obtaining very weird results without being aware of that since no warning and/or error was returned.&lt;/p&gt;&#xA;&lt;p&gt;A quick example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# load some data&#xA;library(&amp;quot;scRNAseq&amp;quot;)&#xA;library(&amp;quot;scater&amp;quot;)&#xA;sce.416b &amp;lt;- LunSpikeInData(which=&amp;quot;416b&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;check the data and notice no dimensionality reduction is available:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; sce.416b&#xA;class: SingleCellExperiment &#xA;dim: 46604 192 &#xA;metadata(0):&#xA;assays(1): counts&#xA;rownames(46604): ENSMUSG00000102693 ENSMUSG00000064842 ...&#xA;  ENSMUSG00000095742 CBFB-MYH11-mcherry&#xA;rowData names(1): Length&#xA;colnames(192): SLX-9555.N701_S502.C89V9ANXX.s_1.r_1&#xA;  SLX-9555.N701_S503.C89V9ANXX.s_1.r_1 ...&#xA;  SLX-11312.N712_S508.H5H5YBBXX.s_8.r_1&#xA;  SLX-11312.N712_S517.H5H5YBBXX.s_8.r_1&#xA;colData names(9): Source Name cell line ... spike-in addition block&#xA;reducedDimNames(0):            &amp;lt;-- HERE IT IS EMPTY&#xA;mainExpName: endogenous&#xA;altExpNames(2): ERCC SIRV&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;at this point I simply run the runTSNE() function using a non-existent dimred:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# set seed&#xA;set.seed(001)&#xA;# add logcounts&#xA;sce.416b &amp;lt;- logNormCounts(sce.416b)&#xA;# add random colours to cells&#xA;colLabels(sce.416b) &amp;lt;- rep(c(1,2,3,4), 48)&#xA;# runTSNE, first try&#xA;sce.416b &amp;lt;- runTSNE(sce.416b, use_dimred=&amp;quot;made_up_dimred&amp;quot;, perplexity=30)&#xA;# and plot&#xA;dev.new(); plotTSNE(sce.416b, colour_by=&amp;quot;label&amp;quot;)&#xA;# runTSNE, second try&#xA;sce.416b &amp;lt;- runTSNE(sce.416b, use_dimred=&amp;quot;another_made_up_dimred&amp;quot;, perplexity=30)&#xA;# and plot&#xA;dev.new(); plotTSNE(sce.416b, colour_by=&amp;quot;label&amp;quot;)&#xA;# runTSNE, third try&#xA;sce.416b &amp;lt;- runTSNE(sce.416b, use_dimred=&amp;quot;a_third_made_up_dimred&amp;quot;, perplexity=30)&#xA;# and plot&#xA;dev.new(); plotTSNE(sce.416b, colour_by=&amp;quot;label&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;it, unfortunately, works. Indeed, I now have three TSNE available that I did plot and that are slightly different with each other.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; sce.416b&#xA;class: SingleCellExperiment &#xA;dim: 46604 192 &#xA;metadata(0):&#xA;assays(2): counts logcounts&#xA;rownames(46604): ENSMUSG00000102693 ENSMUSG00000064842 ...&#xA;  ENSMUSG00000095742 CBFB-MYH11-mcherry&#xA;rowData names(1): Length&#xA;colnames(192): SLX-9555.N701_S502.C89V9ANXX.s_1.r_1&#xA;  SLX-9555.N701_S503.C89V9ANXX.s_1.r_1 ...&#xA;  SLX-11312.N712_S508.H5H5YBBXX.s_8.r_1&#xA;  SLX-11312.N712_S517.H5H5YBBXX.s_8.r_1&#xA;colData names(10): Source Name cell line ... block sizeFactor&#xA;reducedDimNames(1): TSNE      &amp;lt;-- THERE YOU GO: A NICE TSNE COMPUTED OVER A NON-EXISTENT DIMRED&#xA;mainExpName: endogenous&#xA;altExpNames(2): ERCC SIRV&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What is even more interesting is that if I run the function with the newly renamed parameter, i.e. dimred, I do get an error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; sce.416b &amp;lt;- runTSNE(sce.416b, dimred=&amp;quot;made_up_dimred&amp;quot;, perplexity=30)&#xA;Error in value[[3L]](cond) : &#xA;  invalid subscript 'type' in 'reducedDim(&amp;lt;SingleCellExperiment&amp;gt;, type=&amp;quot;character&amp;quot;, ...)':&#xA;  'made_up_dimred' not in 'reducedDimNames(&amp;lt;SingleCellExperiment&amp;gt;)'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But then, if you have an old pipeline which used the runTSNE function with the use_dimred and you're now running an updated version of scater, I have no idea what you may get.&lt;/p&gt;&#xA;&lt;p&gt;So, as per my question...is this a bug? Has anyone else noticed this weird behaviour? Am I wrong?&lt;/p&gt;&#xA;" OwnerUserId="3119" LastActivityDate="2022-02-01T13:23:04.230" Title="Is use_dimred deprecated or is this a runTSNE() bug?" Tags="|single-cell|tsne|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18464" PostTypeId="1" CreationDate="2022-01-31T18:04:37.350" Score="1" ViewCount="93" Body="&lt;p&gt;I am new to HHpred/ analyzing proteins so bare with me. I have been given an uncharacterized protein, whose FASTA sequence I have obtained from Uniprot. I am looking to do the following by using HHpred.&lt;/p&gt;&#xA;&lt;p&gt;So far, I have done the following:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;obtained FASTA sequence and inputted it into HHpred.&lt;/li&gt;&#xA;&lt;li&gt;Loaded the FASTA sequence on to HHpred.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;What I am looking to do on HHpred now:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Identify the different domains/motifs of this uncharacterized protein&lt;/li&gt;&#xA;&lt;li&gt;Compare full length protein to two separate domains&lt;/li&gt;&#xA;&lt;li&gt;Determine if the predictions has improved by comparing the full length proteins to the separate domains.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;All comments help, I am very desperate for any tips.&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="14954" LastActivityDate="2022-01-31T18:04:37.350" Title="How to identify different protein domains using HHpred" Tags="|proteins|protein-structure|structural-biology|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18465" PostTypeId="2" ParentId="18448" CreationDate="2022-01-31T18:29:58.187" Score="1" Body="&lt;p&gt;Batch correction/data integration is a tricky subject for scRNA-seq, see a few recent papers for benchmarks: &lt;a href=&quot;https://www.nature.com/articles/s41592-021-01336-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41592-021-01336-8&lt;/a&gt; and &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1850-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1850-9&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Batch correction methods may not even impact differential expression. For example, if you correct the PCs, via a popular method like harmony, then this will impact the t-SNE/UMAP plots but will not impact the actual expression values in the counts matrix used to detect DEGs.&lt;/p&gt;&#xA;&lt;p&gt;For detecting DEGs, the most common methods are simple stats tests like Wilcoxan Rank test that largely ignore batch effects. However, these are likely not the best approaches. I suggest this recent paper (&lt;a href=&quot;https://www.nature.com/articles/s41467-021-25960-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41467-021-25960-2&lt;/a&gt;). Here they state that the best methods for detecting DEGs consider biological replicates, like is done with bulk RNA-seq. By considering replicates, this is effectively like batch correction&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2022-01-31T18:29:58.187" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18467" PostTypeId="2" ParentId="18456" CreationDate="2022-02-01T10:08:53.777" Score="0" Body="&lt;p&gt;So if that had a little mutation, Does that mean it's a neutral selection?&#xA;And if the sequences (codons and amino) are exactly the same? We can say something about the interpretation?&lt;/p&gt;&#xA;&lt;p&gt;See examples below-&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;ORF8&lt;/strong&gt;&#xA;Seq1:&#xA;MKFLVFLGIITTVAAFHQECSLQSCTQHQPYVVDDPCPIHFYSKWYIRVGARKSAPLIELCVDEAGSKSPIQYIDIGNYTVSCLPFTINCQEPKLGSLVVRCSFYEDFLEYHDVRVVLDFI&#xA;Seq2:&#xA;MKFLVFLGIITTVAAFHQECSLQSCTQHQPYVVDDPCPIHFYSKWYIRVGARKSAPLIELCVDEAGSKSPIQYIDIGNYTVSCLPFTINCQEPKLGSLVVRCSFYEDFLEYHDVRVVL--I&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;ORF10&lt;/strong&gt;&#xA;Seq1&#xA;MGYINVFAFPFTIYSLLLCRMNSRNYIAQVDVVNFNLT&#xA;Seq2&#xA;MGYINVFAFPFTIYSLLLCRMNSRNYIAQVDVVNFNLT&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;E&lt;/strong&gt;&#xA;(The same)&#xA;Seq1&#xA;MYSFVSEETGTLIVNSVLLFLAFVVFLLVTLAILTALRLCAYCCNIVNVSLVKPSFYVYSRVKNLNSSRVPDLLV&#xA;Seq2&#xA;MYSFVSEETGTLIVNSVLLFLAFVVFLLVTLAILTALRLCAYCCNIVNVSLVKPSFYVYSRVKNLNSSRVPDLLV&lt;/p&gt;&#xA;" OwnerUserId="14970" LastActivityDate="2022-02-01T10:08:53.777" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18468" PostTypeId="2" ParentId="18462" CreationDate="2022-02-01T13:23:04.230" Score="1" Body="&lt;p&gt;I believe it would be easier to answer the question by dividing it into related but more specific questions. Having said that, even if the answers below would be satisfactory, I would still post this on the package's Github repo, the author being informed about this can help other users of this package.&lt;/p&gt;&#xA;&lt;p&gt;Question 1: Why does &lt;code&gt;runTSNE()&lt;/code&gt; work when I use the deprecated &lt;code&gt;use_dimred&lt;/code&gt; argument?&lt;/p&gt;&#xA;&lt;p&gt;Let's create function and call it:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; my_func &amp;lt;- function(dimred) {&#xA;+   if(missing(dimred)) {&#xA;+     print(&amp;quot;dimred argument is not provided so something else is being done...&amp;quot;)&#xA;+   } else {&#xA;+     print(dimred)&#xA;+   }&#xA;+ }&#xA;&amp;gt; my_func(42)&#xA;[1] 42&#xA;&amp;gt; my_func(dimred = 42)&#xA;[1] 42&#xA;&amp;gt; my_func()&#xA;[1] &amp;quot;dimred argument is not provided so something else is being done...&amp;quot;&#xA;&amp;gt; my_func(use_dimred = 42)&#xA;Error in my_func(use_dimred = 42) : unused argument (use_dimred = 42)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As expected, we get an error if we call a function with an argument that does not belong to our function: &lt;code&gt;use_dimread&lt;/code&gt;. If we define our function with the special argument &lt;code&gt;...&lt;/code&gt; however, it works:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; my_func &amp;lt;- function(dimred, ...) {&#xA;+   if(missing(dimred)) {&#xA;+     print(&amp;quot;dimred argument is not provided so something else is being done...&amp;quot;)&#xA;+   } else {&#xA;+     print(dimred)&#xA;+   }&#xA;+ }&#xA;&amp;gt; my_func(42)&#xA;[1] 42&#xA;&amp;gt; my_func(dimred = 42)&#xA;[1] 42&#xA;&amp;gt; my_func()&#xA;[1] &amp;quot;dimred argument is not provided so something else is being done...&amp;quot;&#xA;&amp;gt; my_func(use_dimred = 42)&#xA;[1] &amp;quot;dimred argument is not provided so something else is being done...&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Basically when you call a function, the function you are calling can make calls to other functions and the &lt;code&gt;...&lt;/code&gt; is basically allowing the user to feed arguments for these let's say &amp;quot;secondary&amp;quot; functions. Apparently &lt;code&gt;R&lt;/code&gt; does not care if what you feed as an argument to a function is used or not if the function is defined with the special &lt;code&gt;...&lt;/code&gt; argument.&lt;/p&gt;&#xA;&lt;p&gt;Question 2: Why is &lt;code&gt;runTSNE()&lt;/code&gt; working if the supplied &lt;code&gt;use_dimred&lt;/code&gt; argument was not used (as shown above)?&lt;/p&gt;&#xA;&lt;p&gt;If you take a look at the &lt;a href=&quot;https://github.com/Alanocallaghan/scater/blob/RELEASE_3_12/R/runTSNE.R&quot; rel=&quot;nofollow noreferrer&quot;&gt;source code&lt;/a&gt;, &lt;code&gt;calculateTSNE()&lt;/code&gt;, which &lt;code&gt;runTSNE()&lt;/code&gt; runs under the hood, is using PCA if the &lt;code&gt;dimred&lt;/code&gt; argument is not provided, its default is NULL anyway. It is been a while since I have used &lt;code&gt;Scater&lt;/code&gt; but I think this would mean that the tSNE is being computed on (some number of) PCs.&lt;/p&gt;&#xA;&lt;p&gt;And as a conclusion, I don't think we can really call this a bug but a rather unexpected behavior because of use of the special &lt;code&gt;...&lt;/code&gt; argument in the function definition.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2022-02-01T13:23:04.230" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18469" PostTypeId="1" AcceptedAnswerId="18470" CreationDate="2022-02-01T15:07:30.490" Score="-1" ViewCount="77" Body="&lt;p&gt;I want to select a 10x single cell kit&lt;/p&gt;&#xA;&lt;p&gt;What does &lt;code&gt;2x   50     75     100    150     250&lt;/code&gt; mean in paired end sequencing?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2022-02-02T23:46:21.873" Title="Some definitions about RNA-seq" Tags="|rna-seq|scrnaseq|10x-genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18470" PostTypeId="2" ParentId="18469" CreationDate="2022-02-01T18:27:52.053" Score="2" Body="&lt;p&gt;I am pretty sure that is read length. Both reads in a pair (2x) can be 50, 75, ... 250 bases long. And I am even more sure you will find this information on the webpage of the provider and if not, you should email them.&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2022-02-02T23:46:21.873" LastActivityDate="2022-02-02T23:46:21.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18471" PostTypeId="1" AcceptedAnswerId="18479" CreationDate="2022-02-01T20:04:21.433" Score="1" ViewCount="41" Body="&lt;p&gt;I am reading &lt;a href=&quot;https://www.nature.com/articles/mp2011154&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; regarding &lt;em&gt;de novo&lt;/em&gt; CNV analysis in Schizophrenia:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Accordingly, such CNVs must frequently occur as recurrent de novo mutations. In a sample of 662 schizophrenia &lt;strong&gt;proband–parent&lt;/strong&gt; trios, we found that rare de novo CNV mutations were significantly more frequent in cases (5.1% all cases, 5.5% family history negative) compared with 2.2% among 2623 controls, confirming the involvement of de novo CNVs in the pathogenesis of schizophrenia.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I assumed that &lt;strong&gt;proband&lt;/strong&gt; stands for the sample being tested (child of the parents), however, I couldn't find an explicit definition for it. Is it just the sample being tested, or does it imply additional details?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2022-02-02T22:50:53.080" Title="Definition of &quot;proband&quot; in CNV analysis" Tags="|cnv|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18472" PostTypeId="2" ParentId="18415" CreationDate="2022-02-01T22:19:35.393" Score="1" Body="&lt;p&gt;Take a look at &lt;strong&gt;GeneWalk&lt;/strong&gt;, &lt;a href=&quot;https://churchman.med.harvard.edu/genewalk&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://churchman.med.harvard.edu/genewalk&lt;/a&gt;. It's a very interesting solution to this rather difficult problem. It is well documented and packaged. But It's open source software -- not a service. You'll have to run it on your own system.&lt;/p&gt;&#xA;" OwnerUserId="10172" LastActivityDate="2022-02-01T22:19:35.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18473" PostTypeId="1" CreationDate="2022-02-02T02:37:57.520" Score="5" ViewCount="459" Body="&lt;p&gt;I have a large protein PDB file (~3500 residues) and a couple of PDB files containing glycan structures I sourced from GLYCAM.&lt;/p&gt;&#xA;&lt;p&gt;How can I attach the N and O-glycans onto each protein residue with software? In PyMol, I am stuck at loading up the protein PDB file and glycan file together, but I don't know how to put the two elements together. Are there any commands I can use? or tutorial resources I can take a look at? I'm open to trying other platforms.&lt;/p&gt;&#xA;&lt;p&gt;I've tried glycan modelling webtools (GLYCAM, CHARMM-GUI) but they can't seem to process my large glycoprotein - leading to errors or extended waiting times which also end in failure.&lt;/p&gt;&#xA;" OwnerUserId="14917" LastEditorUserId="14917" LastEditDate="2022-02-02T03:42:07.813" LastActivityDate="2022-03-05T12:20:12.137" Title="How do I build a glycosylated protein from PDB files manually?" Tags="|protein-structure|pymol|modelling|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18474" PostTypeId="1" CreationDate="2022-02-02T06:20:57.577" Score="0" ViewCount="53" Body="&lt;p&gt;How does one go about comparing expression levels of one gene across multiple microarray datasets pulled from different experiments/papers in mice? In my example, I've downloaded the expression levels for my gene of interest from different microarray datasets for different cell populations of interest (specifically healthy hematopoietic cells). How would I pool the datasets together?&lt;/p&gt;&#xA;&lt;p&gt;Say I have common cell types from multiple datasets such as B-cells, would I be able to just combine them into one data frame and visualize them as one or would I need to normalize some aspect of that?&lt;/p&gt;&#xA;&lt;p&gt;I'm really confused because I've had different people tell me different things.&lt;/p&gt;&#xA;" OwnerUserId="14978" LastEditorUserId="14978" LastEditDate="2022-02-03T00:57:21.343" LastActivityDate="2022-07-04T13:00:20.047" Title="How to compare across Microarray datasets for a gene in Mice?" Tags="|gene-expression|microarray|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18475" PostTypeId="1" CreationDate="2022-02-02T14:01:22.933" Score="0" ViewCount="101" Body="&lt;p&gt;I am trying to align and merge different samples from NCBI.&#xA;I end up having correlation problem with these sample. The picture below shows an heatmap of the &lt;em&gt;R²&lt;/em&gt; by doing a linear regression between 2 samples' count matrices.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PqEZq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PqEZq.png&quot; alt=&quot;picture&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Samples &lt;strong&gt;A&lt;/strong&gt;, &lt;strong&gt;B&lt;/strong&gt; and &lt;strong&gt;C&lt;/strong&gt; are from the same project and sample &lt;strong&gt;D&lt;/strong&gt; from another. A, B and C are sequenced using the &lt;em&gt;10X Genomics for Single Cell V(D)J + 5' Gene Expression&lt;/em&gt;, and &lt;em&gt;Chromium Next GEM Single Cell 3ʹ v3.1&lt;/em&gt; for sample D.&lt;/p&gt;&#xA;&lt;p&gt;All samples have been recovered from NCBI using &lt;code&gt;fastq-dump&lt;/code&gt; — internaly threaded to be more efficient — and aligned using &lt;code&gt;cellranger&lt;/code&gt; with the default human genome given by 10x.&lt;/p&gt;&#xA;&lt;p&gt;QC shows some GC content small shifts along side some huge duplication level — even for scRNA-seq.&lt;/p&gt;&#xA;&lt;p&gt;Am I missing something important when it comes to merge datasets?&lt;/p&gt;&#xA;&lt;p&gt;Thank you very much&lt;/p&gt;&#xA;" OwnerUserId="14979" LastEditorUserId="104" LastEditDate="2022-02-18T19:06:21.617" LastActivityDate="2022-02-18T19:06:21.617" Title="How to solve correlation problems between different samples in scRNA-seq?" Tags="|rna-seq|scrnaseq|single-cell|phylogenetics|cellranger|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18476" PostTypeId="1" AcceptedAnswerId="18528" CreationDate="2022-02-02T17:16:27.027" Score="10" ViewCount="265" Body="&lt;p&gt;I often use &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5411777/&quot; rel=&quot;noreferrer&quot;&gt;metaSPAdes&lt;/a&gt; to assemble short reads from human microbiomes. My simplified understanding of short-read de Bruijn graph assemblers is that they fail where ambiguous paths cannot be resolved. While it can be said that these points of failure may be due to certain &lt;strong&gt;structural&lt;/strong&gt; features of the underlying sequences -- long repetitive sequences or highly conserved sequences found at multiple loci -- I'm interested in the &lt;strong&gt;functional&lt;/strong&gt; classification of sequences that are proximal to nodes on assembly graphs. Put another way, what kinds of genes are poorly represented in metagenomes because they are poorly assembled? Naively, I expect mobile elements like transposons and integrons to be in such a list of genes, since they are often flanked by repeated sequences and can be present in many copies across multiple genomic contexts, though I want to take an empirical approach.&lt;/p&gt;&#xA;&lt;p&gt;I've been dabbling with &lt;a href=&quot;http://rrwick.github.io/Bandage/&quot; rel=&quot;noreferrer&quot;&gt;Bandage&lt;/a&gt;, which allows you navigate assembly graphs in a GUI, though I'm having trouble extracting features of the underlying network that Bandage builds from the input (.fastg or .gfa). I'd like to know if there are any tools or workflows that would allow me to, given an assembly graph in &lt;a href=&quot;http://gfa-spec.github.io/GFA-spec/GFA1.html&quot; rel=&quot;noreferrer&quot;&gt;GFA format&lt;/a&gt; ...&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;filter nodes by degree,&lt;/li&gt;&#xA;&lt;li&gt;extract sequences proximal to high-degree nodes, and&lt;/li&gt;&#xA;&lt;li&gt;filter pulled sequences based on sequencing depth, so as to preclude analyzing node-adjacent sequences in the case where the assembler failed due to low depth.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I'm open to the possibility that I'm taking the wrong approach to this problem, so feel free to answer with a frame challenge.&lt;/p&gt;&#xA;" OwnerUserId="3967" LastActivityDate="2022-03-28T18:53:55.063" Title="Extract sequence context of high-degree nodes in assembly graphs" Tags="|assembly|metagenome|graphs|de-bruijn-graphs|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18477" PostTypeId="2" ParentId="18475" CreationDate="2022-02-02T19:12:55.767" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;The picture below shows an heatmap of the R² by doing a linear&#xA;regression between 2 samples' count matrices.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In addition to doing a heatmap, I'd do a conventional UMAP/t-SNE plot and see how much seperation of the samples there are across all the different clusters/cell types.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Am I missing something important when it comes to merge datasets?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Certainly there are 'technology batch effects' when comparing 3' and 5' data from the same tissue type. You can try to correct for these when merging the dataset by using a batch correction/integration tool (see reviews &lt;a href=&quot;https://www.nature.com/articles/s41592-021-01336-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt; and &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1850-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;), but will likely be imperfect.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;QC shows some GC content small shifts along side some huge duplication&#xA;level — even for scRNA-seq.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If you're using a tool like fastQC then note that high duplication levels are expected for (single cell) RNAseq from highly expressed genes.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2022-02-02T19:12:55.767" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18478" PostTypeId="2" ParentId="2923" CreationDate="2022-02-02T21:47:27.733" Score="1" Body="&lt;p&gt;I haven't used canu for several years but looking back at my labbook I ran it with two additional flags:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;-minThreads=8 -maxThreads=8&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;My memory on the subject is a bit hazy but  think I had to play around with these settings to get it to work.  I also set the flag:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;-maxMemory=6&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3100" LastActivityDate="2022-02-02T21:47:27.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18479" PostTypeId="2" ParentId="18471" CreationDate="2022-02-02T22:50:53.080" Score="1" Body="&lt;p&gt;&lt;em&gt;answer from @maximilian-press, converted from comments:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.genome.gov/genetics-glossary/Proband&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.genome.gov/genetics-glossary/Proband&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.merriam-webster.com/dictionary/proband&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.merriam-webster.com/dictionary/proband&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A &lt;strong&gt;proband&lt;/strong&gt; is an individual affected with a disorder who is the first subject in a study (as of a genetic character in a family lineage) who brings a genetic disorder to the attention of the medical community.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-02-02T22:50:53.080" CommentCount="0" CommunityOwnedDate="2022-02-02T22:50:53.080" ContentLicense="CC BY-SA 4.0" />
  <row Id="18480" PostTypeId="2" ParentId="18473" CreationDate="2022-02-02T22:52:21.947" Score="1" Body="&lt;p&gt;&lt;em&gt;answer from @coot, converted from comments (please edit this answer to improve it if you can):&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Manual model building is a common thing in structural biology. Usually, Coot is used.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-02-02T22:52:21.947" CommentCount="0" CommunityOwnedDate="2022-02-02T22:52:21.947" ContentLicense="CC BY-SA 4.0" />
  <row Id="18482" PostTypeId="1" CreationDate="2022-02-03T13:20:37.840" Score="2" ViewCount="66" Body="&lt;p&gt;I saw in the SAM format specifications that the SEQ field (10th column) can be a &amp;quot;*&amp;quot; if the sequence is not stored, instead of being the sequence of the mapped read. Under what circumstances is this expected to occur? I don't understand why it's allowed in the spec.&lt;/p&gt;&#xA;&lt;p&gt;For a concrete example, I have pacbio data mapped to a reference sequence, and some of the records in the SAM file have a sequence of *, and I don't know why or if that means anything. I don't understand why they exist, since the data is ostensibly present in the FASTQ file I use as input to the read mapper, and I don't understand what purpose they could have, since it seems to me that these records couldn't be interpreted by variant calling software. At best it would seem they can be used to infer coverage, but it's coverage by sequences of an uncertain nature?&lt;/p&gt;&#xA;" OwnerUserId="14982" LastActivityDate="2022-02-03T13:20:37.840" Title="Under what circumstances can the SEQ field in a SAM file be a *?" Tags="|sam|pacbio|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18483" PostTypeId="1" AcceptedAnswerId="18498" CreationDate="2022-02-03T13:30:58.533" Score="0" ViewCount="1562" Body="&lt;p&gt;I have called SNVs with Mutect2, now in filter column of a file called filtered vcf I have a lot of things like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; length(unique(filtered.vcf$FILTER)) [1] 409 &amp;gt; &#xA;&#xA;&amp;gt; unique(filtered.vcf $FILTER)&#xA;  [1] &amp;quot;germline&amp;quot;                                                                             &#xA;  [2] &amp;quot;PASS&amp;quot;                                                                                 &#xA;  [3] &amp;quot;clustered_events&amp;quot;                                                                     &#xA;  [4] &amp;quot;clustered_events;panel_of_normals&amp;quot;                                                    &#xA;  [5] &amp;quot;weak_evidence&amp;quot;                                                                        &#xA;  [6] &amp;quot;haplotype&amp;quot;                                                                            &#xA;  [7] &amp;quot;base_qual;haplotype&amp;quot;                                                                  &#xA;  [8] &amp;quot;clustered_events;germline;haplotype&amp;quot;                                                  &#xA;  [9] &amp;quot;base_qual;clustered_events;germline;haplotype&amp;quot;                                        &#xA; [10] &amp;quot;clustered_events;haplotype&amp;quot;                                                           &#xA; [11] &amp;quot;clustered_events;haplotype;weak_evidence&amp;quot;                                             &#xA; [12] &amp;quot;base_qual;clustered_events;haplotype&amp;quot;                                                 &#xA; [13] &amp;quot;haplotype;map_qual&amp;quot;                                                                   &#xA; [14] &amp;quot;germline;panel_of_normals&amp;quot;                                                            &#xA; [15] &amp;quot;base_qual&amp;quot;                                                                            &#xA; [16] &amp;quot;germline;haplotype&amp;quot;                                                                   &#xA; [17] &amp;quot;germline;panel_of_normals;position&amp;quot;                                                   &#xA; [18] &amp;quot;clustered_events;orientation&amp;quot;                                                         &#xA; [19] &amp;quot;clustered_events;orientation;strand_bias&amp;quot;                                             &#xA; [20] &amp;quot;base_qual;haplotype;map_qual&amp;quot;                                                         &#xA; [21] &amp;quot;clustered_events;fragment;haplotype&amp;quot;                                                  &#xA; [22] &amp;quot;base_qual;clustered_events&amp;quot;                                                           &#xA; [23] &amp;quot;germline;haplotype;panel_of_normals&amp;quot;                                                  &#xA; [24] &amp;quot;clustered_events;germline;haplotype;panel_of_normals&amp;quot;                                 &#xA; [25] &amp;quot;fragment&amp;quot;                                                                             &#xA; [26] &amp;quot;clustered_events;haplotype;map_qual&amp;quot;                                                  &#xA; [27] &amp;quot;clustered_events;germline&amp;quot;                                                            &#xA; [28] &amp;quot;clustered_events;germline;panel_of_normals&amp;quot;                                           &#xA; [29] &amp;quot;germline;weak_evidence&amp;quot;                                                               &#xA; [30] &amp;quot;orientation&amp;quot;       &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;    &#xA;I tought that the &lt;code&gt;filtered.vcf&lt;/code&gt; Mutect2 output file contain only the filtered events, but the values I see suggest otherwise. How should I understand the output?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="57" LastEditDate="2022-02-05T13:32:32.187" LastActivityDate="2022-02-05T13:33:33.677" Title="What are values of FILTER column of vcf files produced by Mutect2" Tags="|vcf|filtering|snv|wes|point-mutation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18484" PostTypeId="1" AcceptedAnswerId="18485" CreationDate="2022-02-03T15:37:24.020" Score="0" ViewCount="3428" Body="&lt;p&gt;Generally, I use &lt;code&gt;samtools&lt;/code&gt; to &lt;code&gt;sort&lt;/code&gt; and &lt;code&gt;index&lt;/code&gt; BAM files. Samtools works also with BGZF compressed SAM but not with plain ones.&lt;/p&gt;&#xA;&lt;p&gt;I have terabytes of plain SAMs at the moment and I would love to avoid compression/conversion. I have all the data in several external HD and I am not allowed to delete the original SAM files. I would need much more space than I have now to also store the BAM files.&lt;/p&gt;&#xA;&lt;p&gt;So, I was wondering if there is a way to avoid it. Is there some tool/trick to sort and index uncompressed SAM files, without converting them to BAM or compressing them?&lt;/p&gt;&#xA;" OwnerUserId="14985" LastEditorUserId="73" LastEditDate="2022-02-04T19:10:18.510" LastActivityDate="2022-02-04T19:10:18.510" Title="How to Sort and Index a SAM file without converting it to BAM?" Tags="|samtools|sam|indexing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18485" PostTypeId="2" ParentId="18484" CreationDate="2022-02-03T15:43:10.813" Score="7" Body="&lt;p&gt;Not only will you save disk space by converting to BAM, but BAM files are faster to manipulate than SAM. Source: &lt;a href=&quot;https://davetang.org/wiki/tiki-index.php?page=SAMTools#Converting_a_SAM_file_to_a_BAM_file&quot; rel=&quot;noreferrer&quot;&gt;Dave Tang's SAMTools wiki&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;sort&lt;/code&gt; supports uncompressed SAM format from a file or &lt;code&gt;stdin&lt;/code&gt;, though &lt;code&gt;index&lt;/code&gt; requires BGZIP-compressed SAM or BAM. I don't think you can get around this.&lt;/p&gt;&#xA;&lt;p&gt;If you're dead-set on reading from and writing to uncompressed SAM format, you can use pipes to avoid writing BAM files. Specify the output of &lt;code&gt;view&lt;/code&gt; to be an uncompressed bam, then pipe it to your next command, then pipe it to &lt;code&gt;view&lt;/code&gt; again to output your sorted SAM file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -u in.sam | samtools sort | samtools view -h &amp;gt; sort.sam&#xA;samtools view -u sort.sam | samtools index&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3967" LastEditorUserId="3967" LastEditDate="2022-02-03T16:00:55.447" LastActivityDate="2022-02-03T16:00:55.447" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18486" PostTypeId="2" ParentId="17933" CreationDate="2022-02-03T18:17:37.833" Score="2" Body="&lt;p&gt;The issue is with the navbar page, downgrading shiny to 1.6 or in my case 1.4 fixed it for me, the detailed link to find to the solution is here: &lt;a href=&quot;https://github.com/rstudio/shiny/issues/3556&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/rstudio/shiny/issues/3556&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Cheers.&lt;/p&gt;&#xA;" OwnerUserId="14987" LastActivityDate="2022-02-03T18:17:37.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18487" PostTypeId="1" CreationDate="2022-02-04T00:07:18.893" Score="6" ViewCount="51" Body="&lt;p&gt;I want to display E.coli BW25113 (GenBank: CP009273.1) strain in UCSC browser. This strain is not listed in &lt;a href=&quot;http://microbes.ucsc.edu/&quot; rel=&quot;noreferrer&quot;&gt;http://microbes.ucsc.edu/&lt;/a&gt; browser. How can I display E.coli BW25113 assembly in the browser?&lt;/p&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2022-12-09T01:42:45.753" Title="How to display novel genome assemblies or uncommon genome assemblies using the UCSC Genome Browser?" Tags="|assembly|ucsc|bacteria|genome-browser|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18488" PostTypeId="2" ParentId="18487" CreationDate="2022-02-04T00:14:42.240" Score="4" Body="&lt;p&gt;I found two methods:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You want to visit this page for instructions for novel assemblies: &lt;a href=&quot;http://genomewiki.ucsc.edu/index.php/Assembly_Hubs&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genomewiki.ucsc.edu/index.php/Assembly_Hubs&lt;/a&gt; .&lt;/li&gt;&#xA;&lt;li&gt;However, before doing that I highly recommend that you check Genome Assembly Hub first:&#xA;&lt;a href=&quot;https://genome-test.gi.ucsc.edu/gbdb/hubs/genbank/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genome-test.gi.ucsc.edu/gbdb/hubs/genbank/&lt;/a&gt; . There is a good chance that your genome or strain is listed there. Here is an example of how you can do that. Example specifically shows how you can display E.coli BW25113 strain in the browser.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Instructions:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Go to NCBI Nucleotide: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nucleotide&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nucleotide&lt;/a&gt; .&lt;/li&gt;&#xA;&lt;li&gt;Enter your strain name: &lt;code&gt;E.coli BW25113 (GenBank: CP009273.1)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Find Biosample ID on the page, located under DBLINK: &lt;code&gt;BioSample: SAMN03013572&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Go to GenBank assembly hubs: &lt;a href=&quot;https://genome-test.gi.ucsc.edu/gbdb/hubs/genbank/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genome-test.gi.ucsc.edu/gbdb/hubs/genbank/&lt;/a&gt; .&lt;/li&gt;&#xA;&lt;li&gt;Select bacteria. This will take you to &amp;quot;Bacterial assembly hub&amp;quot;&#xA;at &lt;a href=&quot;https://genome-test.gi.ucsc.edu/gbdb/hubs/genbank/bacteria/bacteria.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genome-test.gi.ucsc.edu/gbdb/hubs/genbank/bacteria/bacteria.html&lt;/a&gt; .&lt;/li&gt;&#xA;&lt;li&gt;Look for &lt;em&gt;E. coli&lt;/em&gt; hubs. At the time I looked #15-18 contain &lt;em&gt;E. coli&lt;/em&gt; strains.&lt;/li&gt;&#xA;&lt;li&gt;Click on each hub and search for &lt;code&gt;SAMN03013572&lt;/code&gt; using browser search function (&lt;kbd&gt;Command&lt;/kbd&gt;+&lt;kbd&gt;F&lt;/kbd&gt; on macOS). I found my genome in hub #16.&lt;/li&gt;&#xA;&lt;li&gt;Click on &amp;quot;common name&amp;quot; in the column-3. This will take you to&#xA;&lt;a href=&quot;https://genome-test.gi.ucsc.edu/cgi-bin/hgGateway?hubUrl=http://genome-test.gi.ucsc.edu/gbdb/hubs/genbank/bacteria/16/hub.txt&amp;amp;genome=GCA_000750555.1_ASM75055v1&amp;amp;position=lastDbPos&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genome-test.gi.ucsc.edu/cgi-bin/hgGateway?hubUrl=http://genome-test.gi.ucsc.edu/gbdb/hubs/genbank/bacteria/16/hub.txt&amp;amp;genome=GCA_000750555.1_ASM75055v1&amp;amp;position=lastDbPos&lt;/a&gt; , where &lt;code&gt;FindPosition&lt;/code&gt; window will be populated with your genome Assembly.&lt;/li&gt;&#xA;&lt;li&gt;Click on &lt;kbd&gt;Go&lt;/kbd&gt;. This should take you to the UCSC Genome Browser with your genome assembly.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14180" LastEditorUserId="6251" LastEditDate="2022-02-04T13:49:50.310" LastActivityDate="2022-02-04T13:49:50.310" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18489" PostTypeId="1" CreationDate="2022-02-04T00:46:21.777" Score="1" ViewCount="38" Body="&lt;p&gt;When we have a function which takes values ​​in [a,b] and which returns values ​​in [c,d], what transformation can we do so that the function takes values ​​in [0,1] and returns values ​​in [0,1] without the curve changing in both cases?&lt;/p&gt;&#xA;" OwnerUserId="14989" LastActivityDate="2022-02-07T04:39:48.050" Title="How to rescale a function?" Tags="|sequence-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18490" PostTypeId="1" AcceptedAnswerId="18493" CreationDate="2022-02-04T01:24:53.293" Score="4" ViewCount="48" Body="&lt;p&gt;I know that there are thousands of PDB files stored in Protein Data Banks.&lt;/p&gt;&#xA;&lt;p&gt;Are all these files complete in terms of the information they store?&lt;/p&gt;&#xA;&lt;p&gt;If YES, do the individual files get updated from time to time?&lt;/p&gt;&#xA;&lt;p&gt;If YES, why/when do they get updated?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="12686" LastEditDate="2022-02-04T02:33:37.637" LastActivityDate="2022-02-04T10:48:54.690" Title="Are the information stored in the PBD files in the protein data bank complete or incomplete?" Tags="|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18491" PostTypeId="1" CreationDate="2022-02-04T06:57:27.170" Score="3" ViewCount="100" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/DSSP_(hydrogen_bond_estimation_algorithm)&quot; rel=&quot;nofollow noreferrer&quot;&gt;DSSP - Wikipedia&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The above article says that DSSP is a hydrogen bond estimation algorithm. Then the article also says that this algorithm is &amp;quot;assigns secondary structure to proteins&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;If both are true, how does hydrogen bond estimation helps us to assign secondary structures of proteins?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastActivityDate="2022-02-04T09:26:54.873" Title="What is DSSP algorithm for?" Tags="|protein-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18492" PostTypeId="2" ParentId="18491" CreationDate="2022-02-04T09:07:56.417" Score="3" Body="&lt;p&gt;&lt;strong&gt;&amp;quot;Secondary structure is formally defined by the pattern of hydrogen bonds between the amino hydrogen and carboxyl oxygen atoms in the peptide backbone&amp;quot;&lt;/strong&gt; (&lt;a href=&quot;https://en.wikipedia.org/wiki/Protein_secondary_structure&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;To be more specific: secondary structures are formed explicitly because of hydrogen bonds. So when DSSP says that it estimates hydrogen bonds as a proxy for secondary structure classification, this is likely why.&lt;/p&gt;&#xA;" OwnerUserId="11856" LastEditorUserId="11856" LastEditDate="2022-02-04T09:26:54.873" LastActivityDate="2022-02-04T09:26:54.873" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18493" PostTypeId="2" ParentId="18490" CreationDate="2022-02-04T10:48:54.690" Score="5" Body="&lt;p&gt;A PDB file is the last step in a series of operations.&#xA;For a crystal structure in very simple terms, it consists of the coordinates of the atoms placed to best fit the density map, which in turn (with these atoms due to the phase problem) is a conversion from reciprocal space which is the result of transforming the quantified dots from the pretty diffraction pattern picked up from the detector during the experiment.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Are all these files complete in terms of the information they store?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Not at all, a PDB file is not complete. It nearly complete when you count density in there.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If YES, do the individual files get updated from time to time?&#xA;If YES, why/when do they get updated?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Only, minor format upgrade, eg. &lt;a href=&quot;https://www.rcsb.org/versions/1UBQ&quot; rel=&quot;noreferrer&quot;&gt;https://www.rcsb.org/versions/1UBQ&lt;/a&gt;.&#xA;It is however very common to polish structures.&#xA;There is a tool called PDB-Redo that is commonly called in many pipelines for example to &amp;quot;upgrade&amp;quot; older files.&#xA;Some folk open the files in Coot and let it do its thing.&#xA;For most molecular mechanics experiments it is common to energy minimise the structure (freely, restrained by original position or restrained to density map). It is uncommon in several docking tutorials simply to keep it simple.&#xA;However, for most uses, if an aromatic ring is not quite planar or a bond is longer than most by a fraction of an Ångström, nothing happens.&lt;/p&gt;&#xA;&lt;p&gt;Also worth remembering many structures are incorrect and not just imperfect: there may be actually an isopeptide bond in the core of the protein (many did go unnoticed) or an isoaspartate conversion happened or the loop is actually split due to an Asp-Pro pair autocatalysis, waters get mistaken for ligands or sidechains. If these are polished, they may get flagged as problematic, but they may be made worse.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2022-02-04T10:48:54.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18494" PostTypeId="2" ParentId="18474" CreationDate="2022-02-04T11:06:57.247" Score="1" Body="&lt;p&gt;If you are after expression of tissue-specific, but otherwise healthy wild-type mice, &lt;a href=&quot;https://bgee.org/?page=gene&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bgee&lt;/a&gt; is really a great resource for you. They pull a lot of data, curate it and sort it in a single database that easy to browse. They also have lots of tools for comparative genomics (if you would be wondering about expression of an ortholog in a different species).&lt;/p&gt;&#xA;&lt;p&gt;If you are just after the methodology of pulling expression data together from multiple sources be because you are interested in pulling data from specific papers, you can check &lt;a href=&quot;https://academic.oup.com/nar/article/49/D1/D831/5920517&quot; rel=&quot;nofollow noreferrer&quot;&gt;their paper&lt;/a&gt; about the database. The pipeline is available on &lt;a href=&quot;https://github.com/BgeeDB/bgee_pipeline&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt; too.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2022-02-04T11:06:57.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18495" PostTypeId="2" ParentId="18430" CreationDate="2022-02-04T14:00:16.330" Score="2" Body="&lt;p&gt;How about trying this?&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Go to &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nucleotide&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nucleotide&lt;/a&gt;. Then type the following command in the search box. It resulted in 23971 items.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;code&gt;(bacterial virus[Organism]) AND (complete genome[ti]) NOT shotgun[ti] NOT plasmid[ti] NOT bacteria[Organism]&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2022-02-04T14:00:16.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18498" PostTypeId="2" ParentId="18483" CreationDate="2022-02-05T13:17:35.727" Score="5" Body="&lt;p&gt;You haven't told us what commands you ran, so I am assuming you first ran &lt;code&gt;Mutect2&lt;/code&gt; and then &lt;code&gt;FilterMutectCalls&lt;/code&gt;. If so, the &lt;code&gt;FilterMutectCalls&lt;/code&gt; does not remove any variants from its input, it will simply flag any that fail its builtin filters. Any variants that did not fail a filter are marked as PASS and the rest have a filter added.&lt;/p&gt;&#xA;&lt;p&gt;Now, whether you want to filter these out or not is up to you. You need to look at the VCF header where each of the filters is defined (admittedly, not very clearly) and make a judgment call on whether you consider this filter to be a deal breaker or not.&lt;/p&gt;&#xA;&lt;p&gt;For instance, the &lt;code&gt;germline&lt;/code&gt; flag means:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##FILTER=&amp;lt;ID=germline,Description=&amp;quot;Evidence indicates this site is germline, not somatic&amp;quot;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So that indicates that gatk believes this is more likely to be a germline variant than a somatic one, that it is more likely to be a constituent variant present in the sample's germline cells than a variant that is only found in the sample's tumor cells.&lt;/p&gt;&#xA;&lt;p&gt;Another clear case is the &lt;code&gt;base_qual&lt;/code&gt; filter which means:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##FILTER=&amp;lt;ID=base_qual,Description=&amp;quot;alt median base quality&amp;quot;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When a variant is flagged with this filter, it means that the median fastq base quality of the reads supporting the variant was significantly lower than the median fastq base quality of the reads supporting the reference alele at the same position, making it likely that the variant is a false positive.&lt;/p&gt;&#xA;&lt;p&gt;You need to read the VCF headers and any gatk documentation you can find (warning: these filters are not very well documented at all, in my experience), understand what the filters are and then decide what variants &lt;em&gt;you&lt;/em&gt; consider real based on what you know about your sample, your experimental design and the question you are trying to answer.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2022-02-05T13:33:33.677" LastActivityDate="2022-02-05T13:33:33.677" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18499" PostTypeId="1" CreationDate="2022-02-05T15:01:06.847" Score="3" ViewCount="26" Body="&lt;p&gt;Is there any public available mapping between Online Mendelian Inheritance in Man (OMIM) diseases and Unified Medical Language System (UMLS) concepts?&lt;/p&gt;&#xA;&lt;p&gt;OMIM has mapping for genes to entrez or ensembl, but has no mapping for diseases? Does anything exists that would allow me to map OMIM diseases to UMLS?&lt;/p&gt;&#xA;" OwnerUserId="15002" LastEditorUserId="57" LastEditDate="2022-02-05T15:41:31.390" LastActivityDate="2022-02-05T15:41:31.390" Title="OMIM diseases to UMLS mapping" Tags="|public-databases|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18500" PostTypeId="1" AcceptedAnswerId="18502" CreationDate="2022-02-05T19:53:33.237" Score="2" ViewCount="254" Body="&lt;p&gt;Hi I am trying to use this python script (running with bash script) to do my analysis. I am getting the following error:&lt;/p&gt;&#xA;&lt;p&gt;File &amp;quot;my_python_script.py&amp;quot;, line 33&#xA;logging.info(&amp;quot;Reading input2 file&amp;quot;)&#xA;^&#xA;SyntaxError: invalid syntax&lt;/p&gt;&#xA;&lt;p&gt;My Python script is as folllows and albeit I omitted the indentation and my full apologies therein, please keep in mind I'm not the coder in this instance.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import itertools&#xA;import argparse&#xA;import logging&#xA;import sys&#xA;import numpy as np&#xA;import scipy&#xA;import pandas as pd&#xA;&#xA;def parse_commandline():&#xA;&amp;quot;&amp;quot;&amp;quot;Parse command line parameters.&#xA;Return type: Object&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;parser = argparse.ArgumentParser(description=&amp;quot;Association of variants&amp;quot;)&#xA;parser.add_argument(&amp;quot;--input1&amp;quot;, metavar=&amp;quot;FILE1.tsv&amp;quot;, help=&amp;quot;gene data file&amp;quot;, required=True)&#xA;parser.add_argument(&amp;quot;--input2&amp;quot;, metavar=&amp;quot;FILE2.tsv&amp;quot;, help=&amp;quot;mutation data file&amp;quot;, required=True)&#xA;parser.add_argument(&amp;quot;--output&amp;quot;, metavar=&amp;quot;FILE.tsv&amp;quot;, help=&amp;quot;Output file name&amp;quot;, required=True)&#xA;if len(sys.argv) == 1:&#xA;    parser.print_help()&#xA;    sys.exit(0)&#xA;return parser.parse_args()&#xA;&#xA;def main():&#xA;params = parse_commandline()&#xA;&#xA;#load genedata&#xA;logging.info(&amp;quot;Reading input1 file&amp;quot;)&#xA;try:&#xA;gene_data = pd.read_csv(params.input1, sep=&amp;quot;\t&amp;quot;, index_col =&amp;quot;Model&amp;quot;)&#xA;sys.exit(0)&#xA;    &#xA;#load mutation data&#xA;logging.info(&amp;quot;Reading input2 file&amp;quot;)&#xA;try:&#xA;mutation_data = pd.read_csv(params.input2, sep=&amp;quot;\t&amp;quot;, &#xA;index_col=&amp;quot;Mutation&amp;quot;)&#xA;sys.exit(0)&#xA;&#xA;##All mutations and gene in data files&#xA;all_mutations = list(mutation_data.index)&#xA;all_genes = list(gene_data.columns)&#xA;cell line Models in mutation_data file&#xA;models_mutations = list(mutation_data.columns)&#xA;&#xA;## models in gene_data file&#xA;models_genes = list(gene_data.index)&#xA;intersection_set = set.intersection(set(models_genes), &#xA;set(models_mutations))&#xA;models_common = list(intersection_set)&#xA;unique_series_gene = gene_data.index.isin(models_common)&#xA;end_gene_table = gene_data[unique_series_gene]&#xA;transposed_mutation_data = mutation_data.transpose()&#xA;unique_series_mutation = &#xA;transposed_mutation_data.index.isin(models_common)&#xA;end_mutation_table = transposed_mutation_data[unique_series_gene]&#xA;&#xA;##calcuate p-values&#xA;all_p_values = []&#xA;genes = []&#xA;mutations = []&#xA;for gene, mutation in itertools.product(all_genes, all_mutations):&#xA;gene_table_to_test = np.array(end_gene_table[gene])&#xA;mutation_table_to_test = np.array(end_mutation_table[mutation])    &#xA;p_value = scipy.stats.wilcoxon(gene_table_to_test, &#xA;mutation_table_to_test)[1]&#xA;genes.append(gene)&#xA;mutations.append(mutation)&#xA;all_p_values.append(p_value)&#xA;&#xA;## calculating adjusted_pvalues&#xA;adjusted_pvalues = p_adjust_bh(all_p_values)&#xA;d = {'Gene': genes, 'Mutation': mutations, 'pvalue': &#xA;all_p_values,'p_adj':adjusted_pvalues}&#xA;df = pd.DataFrame(data=d)&#xA;&#xA;# Print output file&#xA;logging.info(&amp;quot;Writing into file&amp;quot;)&#xA;df.to_csv(params.output, sep=&amp;quot;\t&amp;quot;)&#xA;&#xA;if __name__ == &amp;quot;__main__&amp;quot;:&#xA;main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My Bash script&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/sh&#xA;&#xA;### This script will run python script&#xA;&#xA;python3 my_python_script.py --input1 Gene.tsv --input2 &#xA;Mutation.tsv --output tmp&#xA;diff tmp adjusted_pvalues.tsv&#xA;rm tmp&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4845" LastEditorUserId="3848" LastEditDate="2022-02-06T14:18:12.660" LastActivityDate="2022-02-06T14:18:12.660" Title="Python argparse error" Tags="|python|bash|" AnswerCount="1" CommentCount="4" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18501" PostTypeId="2" ParentId="18489" CreationDate="2022-02-05T23:41:10.617" Score="2" Body="&lt;p&gt;Assuming you want a linear transformation (which is what &amp;quot;without the curve changing&amp;quot; sounds like to me), to rescale from range &lt;strong&gt;c..d&lt;/strong&gt; to range &lt;strong&gt;0..1&lt;/strong&gt; subtract the minimum then divide by the difference, i.e.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$val_t = (val - c) / (d-c)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The inverse of this can be applied to work the other way, i.e.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$inp_t = val \times (b-a) + a$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;For a function &lt;span class=&quot;math-container&quot;&gt;$f(x)$&lt;/span&gt;, with ranges as you have stated, the full transformation of input and output would be:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$g(x_t) = \frac{f(x_t \times (b-a) + a) - c}{d-c}$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-02-07T04:39:48.050" LastActivityDate="2022-02-07T04:39:48.050" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18502" PostTypeId="2" ParentId="18500" CreationDate="2022-02-06T05:06:56.237" Score="1" Body="&lt;p&gt;It is certainly possible the 'error' is the input file 2.&lt;/p&gt;&#xA;&lt;p&gt;However, both &lt;code&gt;try:&lt;/code&gt; routines are weird and if this is the real code it would fall over. Something isn't quite right however, which I'll touch on at the end.&lt;/p&gt;&#xA;&lt;p&gt;What the code is supposed to say is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;try:&#xA;    load stuff&#xA;except:&#xA;    raise Exception ('Input file 2 is has deviation from csv format and will not load')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;raise Exception&lt;/code&gt; isn't needed, the following would be a more harmonised approach:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;try:&#xA;    load stuff&#xA;except:&#xA;    print (&amp;quot;Input file 2 is has deviation from csv format and will not load')&#xA;finally:&#xA;    sys.exit('Abandon ship!')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The idea is that if the CSV file didn't load, it would stop the code and tell you approximately where to start looking. You might then ask, can it actually sees the file and then peer into the loaded CSV. If it loads the code continues.&lt;/p&gt;&#xA;&lt;p&gt;You could just stay:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;try:&#xA;    load stuff&#xA;except:&#xA;    sys.exit('Oh the file did not to load')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This isn't cool, but who am I (who sticks Exceptions in except statements), but it will work and it could be that is the actual code.&lt;/p&gt;&#xA;&lt;p&gt;What the code actually says is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;try:&#xA;    load stuff&#xA;    sys.exit()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;try: &#xA;    load stuff&#xA;sys.exit()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Same outcome the code stop because sys.exit stops everything ... nothing is left. What the code should really say is not just perform a try: except: but perform a 'getter' and 'setter' on the input file to check the CSV format, this is because your the user of the code not the developer and the coder has to put safe guards in because stuff happens.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;sys.exit&lt;/code&gt; is really used in development, for example in debugging.&lt;/p&gt;&#xA;&lt;p&gt;However, somehow you got past the first &lt;code&gt;try:, sys.exit&lt;/code&gt; and I don't know how you did it, because that should not have been possible.  I would carefully check the original code.&lt;/p&gt;&#xA;&lt;p&gt;Just to point out as @ChrisR mentioned, if this was your actual python code it would fall over because you've omitted all the identation, which in python is essential. Its not like shell where indentitation matters not a jot.&lt;/p&gt;&#xA;&lt;p&gt;Finally, as @supertech has mentioned, there are numerous line breaks in the code and whether you are in shell or python this can cause problems. I think the line break(s) are within an extra layer of identation, which Python can read through (I never understood how).&lt;/p&gt;&#xA;&lt;p&gt;Personally, I'd go back to the coder and point out the try: sys.exit() and just point out any beginners webpage on &lt;code&gt;try: except:&lt;/code&gt;, this is not advanced coding. If anything sys.exit() risks weird behaviour for a user particularly as no distinguishing information is provided.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-06T05:14:06.133" LastActivityDate="2022-02-06T05:14:06.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18505" PostTypeId="1" AcceptedAnswerId="18506" CreationDate="2022-02-06T19:25:21.627" Score="2" ViewCount="514" Body="&lt;p&gt;I have received Illumina sequencing reads for 100 samples. I have 8 &lt;code&gt;R1.fastq.gz&lt;/code&gt; and 8 &lt;code&gt;R2.fastq.gz&lt;/code&gt; files for each sample in each subfolder. I want to run an &lt;code&gt;MD5&lt;/code&gt; check for all the &lt;code&gt;fastq&lt;/code&gt; files in each subfolder.&lt;/p&gt;&#xA;&lt;p&gt;My folder structure looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/mydata/sequencing/clean&#xA;/mydata/sequencing/clean/Sample1&#xA;/mydata/sequencing/clean/Sample2&#xA;......&#xA;/mydata/sequencing/clean/Sample100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am using the following code being on the &lt;code&gt;/mydata/sequencing/clean&lt;/code&gt; directory.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;find . -type f -exec md5sum-lite {} \;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This code is printing the md5 result on Terminal.&lt;/p&gt;&#xA;&lt;p&gt;How do I get these results printed to a &lt;code&gt;.txt&lt;/code&gt; file? Besides, is there a more elegant way to do this MD5 check?&lt;/p&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="12238" LastActivityDate="2022-02-06T19:42:10.923" Title="How to run MD5 check for multiple fastq files in different subdirectories?" Tags="|genome|sequence-analysis|illumina|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18506" PostTypeId="2" ParentId="18505" CreationDate="2022-02-06T19:32:07.320" Score="2" Body="&lt;p&gt;Small modification,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;find . -type f -exec md5sum-lite {} \; &amp;gt;&amp;gt; output.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you use &lt;code&gt;&amp;gt;&lt;/code&gt; the output will almost certainly contain a single record.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-06T19:42:10.923" LastActivityDate="2022-02-06T19:42:10.923" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18508" PostTypeId="1" CreationDate="2022-02-07T05:34:06.810" Score="1" ViewCount="51" Body="&lt;p&gt;How do I test the gene expression differences between multiple groups?&lt;/p&gt;&#xA;&lt;p&gt;I have normalized TCGA RSEM data, this is the dataset&#xA;&lt;strong&gt;Human__TCGA_LAML__UNC__RNAseq__HiSeq_RNA__01_28_2016__BI__Gene__Firehose_RSEM_log2.cct&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I have different groups of data which were defined in terms of blast percentage as well as karyotype. Here I dont have control sample as such but I do have normal karyotype. Now since these are all already normalized data what would be ideal test to find the differences.&lt;/p&gt;&#xA;&lt;p&gt;What methods can I use if I have to compare between these different sub types of data? So far I have always used raw counts and used DESeq2 to perform differential and other downstream analysis how in this case should I go ahead?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm trying to find the important or significant genes that defines a particular subtypes of cancer in this AML.&lt;/p&gt;&#xA;&lt;p&gt;So far I have tried clustering and PCA on it I do see some clusters and patterns. Now I want to take those cluster for downstream analysis such as pathways plus we want to validate some of the key genes that might come out.&lt;/p&gt;&#xA;&lt;p&gt;So far I have done as such I get raw counts which i use with deseq2 to run differential analysis between multiple groups or subypes.&lt;/p&gt;&#xA;&lt;p&gt;How to do the same here when I don't have raw counts to start?&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="73" LastEditDate="2022-02-10T23:26:04.403" LastActivityDate="2022-02-10T23:26:04.403" Title="TGCA LAML rsem data" Tags="|rna-seq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18509" PostTypeId="1" CreationDate="2022-02-07T09:53:33.447" Score="3" ViewCount="54" Body="&lt;p&gt;I have a question related to the evaluation of variant pathogenicity in the context of human genetics. There are many predictors available, however, is there a specific predictor that takes into account the 3D protein structure as well?&lt;/p&gt;&#xA;&lt;p&gt;Imagine there is a mutational hotspot, i.e. a sequence of amino acids of a protein region that are pathogenic (assessed by functional studies, for example). And then, let's say several hundred residues far from this region there is a variant whose pathogenicity status is unclear. Now, what can happen here is that this variant, although far from the mutational hotspot (considering the protein secondary structure) can actually be quite close to the mutational hotspot in the 3D space. I mean, this variant may actually form a part of a functional site, together with residues from the mutational hotspot. This could indicate that this single variant is pathogenic as well, although there is no direct evidence.&lt;/p&gt;&#xA;&lt;p&gt;And so, is there is a predictor that takes 3D structure into account?&lt;/p&gt;&#xA;&lt;p&gt;In other words, I'd like to ask a question: &lt;em&gt;Are there any pathogenic variants (defined by clinvar) within a sphere with a radius of 100A centered on a particular residue of interest?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Or, are there, for example, any glycine residues within such a sphere?&lt;/p&gt;&#xA;&lt;p&gt;It's not really a prediction, it would be just a search algorithm, taking 3 parameters:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;protein 3D shape&lt;/li&gt;&#xA;&lt;li&gt;search sphere [center residue; radius]&lt;/li&gt;&#xA;&lt;li&gt;needle - the element I'm searching for, such as AA residue type, for example&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="14993" LastEditorUserId="14993" LastEditDate="2022-02-07T16:10:23.900" LastActivityDate="2022-02-07T16:10:23.900" Title="pathogeniticy prediction tool" Tags="|genetics|variants|pathogenicity|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18510" PostTypeId="2" ParentId="18509" CreationDate="2022-02-07T10:43:50.620" Score="3" Body="&lt;h2&gt;Venus&lt;/h2&gt;&#xA;&lt;p&gt;I am the developer of &lt;a href=&quot;https://venus.cmd.ox.ac.uk/venus&quot; rel=&quot;nofollow noreferrer&quot;&gt;venus.cmd.ox.ac.uk&lt;/a&gt;, which is a site that given a mutation makes the variant (and gives the ∆∆G) and shows what residues are nearby and whether these appear in clinVar or gnomAD (and scores them too) or have some annotation such as disulfide or post-translational modifications. The aim is to allow the user to see and judge for themselves what is going on as there are multiple possible mechanisms which may make a variant different (and then saving and sharing the results via &lt;a href=&quot;https://michelanglo.sgc.ox.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Michelanglo&lt;/a&gt;).  In other words it is not a predictor but a data aggregator of soughts.&#xA;It shows, in addition to the structure in the NGL viewport, a table where the nearby residues are listed (and if clicked focuses on in the protein)&lt;/p&gt;&#xA;&lt;h3&gt;Limitations&lt;/h3&gt;&#xA;&lt;p&gt;An issue according to non-users is that it does not give a statistical relevance score for the features, but I really did not want that as these can be completely off and easy to manipulate and just encourage users to blindly believe the calculations.&lt;/p&gt;&#xA;&lt;p&gt;An issue according to me, is that it is based on empirical data: if nothing is known to bind or where it binds, it cannot say much, unlike &lt;a href=&quot;http://mutpred.mutdb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MutPred2&lt;/a&gt;, a linear sequence deep-learning tool that assigns a prediction of whether there's a binding site etc. without prior empirical evident.&lt;/p&gt;&#xA;&lt;p&gt;Very very few users provide a PDB file and this aspect is limited. Venus is that it is centred around canonical Uniprot numbering, so providing PDB files with differing numbering will not work. Furthermore, the app is coded to &amp;quot;give up easily&amp;quot; on a structure and pick the next option, so does not try to parameterise unusual ligands in a provided structure (I notice now a job failure due to the ACE C-terminal cap, for example).&lt;/p&gt;&#xA;&lt;h2&gt;Other options&lt;/h2&gt;&#xA;&lt;p&gt;There are other tools that show the variant in a structural context, such as &lt;a href=&quot;http://missense3d.bc.ic.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Missense3D&lt;/a&gt; (worth a gander), but do not show other variants or annotations.&lt;/p&gt;&#xA;&lt;p&gt;However, for a specific query, such as the closest proline to the residue of interest in a structure with no relevant Uniprot, it may be best to simply use a viewer, such as &lt;a href=&quot;https://molstar.org/viewer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mol*&lt;/a&gt; or a local one, such PyMOL.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2022-02-07T14:01:07.457" LastActivityDate="2022-02-07T14:01:07.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18514" PostTypeId="1" CreationDate="2022-02-07T16:59:37.247" Score="3" ViewCount="76" Body="&lt;p&gt;I'm assuming that this is such a common question that I don't have to do my own imputation...&lt;/p&gt;&#xA;&lt;p&gt;I have a SNP from a paper that is associated with a specific trait... Actually I have 8 of them:&#xA;&lt;a href=&quot;https://www.ebi.ac.uk/gwas/publications/33307546&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/gwas/publications/33307546&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, when I search my SNP data from 23andMe (v3 NOT v5), only one of them is in the set of SNPs used by that chip, namely rs2109069.&lt;/p&gt;&#xA;&lt;p&gt;I'm wondering how I find the closest SNP in my v3 data for the remaining 7 SNPs missing in that data.&lt;/p&gt;&#xA;&lt;p&gt;Is there a quick way to find a 'tag' SNP in v3 for a given 'random' SNP?&lt;/p&gt;&#xA;&lt;p&gt;I'm aware that a tag SNP will only be as good (on average) as the LD between it and the given SNP.&lt;/p&gt;&#xA;" OwnerUserId="8875" LastActivityDate="2022-02-07T18:43:19.307" Title="Given a 'random' rsID, how can I find the closest (LD) SNP in a given set of SNPs?" Tags="|imputation|snp-chip|" AnswerCount="0" CommentCount="16" ContentLicense="CC BY-SA 4.0" />
  <row Id="18516" PostTypeId="1" CreationDate="2022-02-07T20:40:16.907" Score="0" ViewCount="81" Body="&lt;p&gt;What are some possible reasons why some trends observed in Microarray expression levels is not observed in RNA-seq. Example the difference between 2 cell types for a gene of interest show major differences when microarray is used with one cell type clearly having higher expression but rna-seq for the same cell types shows equal expression levels for the gene of interest?&lt;/p&gt;&#xA;&lt;p&gt;What are possible explanations for the discrepancy between such results ? Shouldn't they have relatively identical trends otherwise gene expression in one of them is unreliable?&lt;/p&gt;&#xA;" OwnerUserId="14978" LastActivityDate="2022-02-08T21:07:35.023" Title="What could explain the difference in RNA-seq and Microarray expression levels?" Tags="|rna-seq|gene-expression|microarray|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18517" PostTypeId="1" AcceptedAnswerId="18534" CreationDate="2022-02-07T21:10:03.970" Score="3" ViewCount="51" Body="&lt;p&gt;When I look up high-throughout (ChIP) evidence for binding of the ferric uptake regulator Fur to sites in the genome of &lt;em&gt;E. coli&lt;/em&gt;, &lt;a href=&quot;http://regulondb.ccg.unam.mx/highthroughputdatasets?term=HTRI00000827&quot; rel=&quot;nofollow noreferrer&quot;&gt;RegulonDB lists&lt;/a&gt; a whopping 4214 targets. (To repeat this search, go to the &lt;a href=&quot;https://regulondb.ccg.unam.mx/central_panel_menu/downloads_menu.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;downloads page&lt;/a&gt;, click on &amp;quot;High-throughput datasets&amp;quot;,  enter Fur in the &lt;em&gt;right-most&lt;/em&gt; search box, and click on the fourth link, for &lt;code&gt;HTRI00000827&lt;/code&gt;.) Some of these have the same gene (alternate promoters?), but I think over 2,000 unique target genes are given. When I click through to the supporting citation, &lt;a href=&quot;https://www.nature.com/articles/ncomms5910&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seo et al.&lt;/a&gt; say &amp;quot;Collectively, a total of 98 Fur-binding sites were newly identified in this study[.]&amp;quot; Clearly I'm missing something here. Why does RegulonDB list so many more target genes than the original paper? What's the evidence, if not ChIP? Thanks!&lt;/p&gt;&#xA;&lt;p&gt;Edit: I've tried to reach out to RegulonDB using the contact form on their website, but I am unable to pass their captcha test! &lt;code&gt;I am uncertain what the problem could be. My protocols are completely up to date.&lt;/code&gt; I will update the question if I manage to contact them.&lt;/p&gt;&#xA;&lt;p&gt;Edit: the paper describing incorporation of ChIP data into RegulonDB is &lt;a href=&quot;https://bmcbiol.biomedcentral.com/articles/10.1186/s12915-018-0555-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. It is obviously well-informed, including senior authors from labs behind multiple ChIP datasets. Tables 4 and 5 describe the quantity of new interactions added and are closer to the original reference than to what I am seeing from the website.&lt;/p&gt;&#xA;" OwnerUserId="1708" OwnerDisplayName="eric_kernfeld" LastEditorUserId="1708" LastEditDate="2022-02-08T16:33:20.043" LastActivityDate="2022-03-07T21:18:29.980" Title="Where do I find the primary evidence for regulatory relationships included in RegulonDB?" Tags="|chip-seq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18518" PostTypeId="2" ParentId="4604" CreationDate="2022-02-07T22:37:46.463" Score="1" Body="&lt;p&gt;You're actually not correct about the functional form of the logit. It's nonlinear, but you've written a linear function.&lt;/p&gt;&#xA;&lt;p&gt;That matters here because once you understand the functional form of logistic regression, you can write it in a way that relates it to your specific generative model for your data. That can tell you what features you ought to use if your model assumptions are correct. You can find the details in PRML section 4.2 (&lt;a href=&quot;https://www.microsoft.com/en-us/research/uploads/prod/2006/01/Bishop-Pattern-Recognition-and-Machine-Learning-2006.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;pdf&lt;/a&gt;). The essence is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If your class labels are generated from a mixture of two classes with prior probabilities &lt;span class=&quot;math-container&quot;&gt;$\pi_1$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\pi_2$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;If your covariates are generated from exponential-family distributions with densities proportional to &lt;span class=&quot;math-container&quot;&gt;$\exp(\beta_1^T\Phi(x))$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$\exp(\beta_2^T\Phi(x))$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;then the posterior probability that an observation &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is in class 1 is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\frac{\pi_1\exp(\beta_1^T\Phi(x)) }{\pi_1\exp(\beta_1^T\Phi(x)) + \pi_2\exp(\beta_2^T\Phi(x))$}$$&lt;/span&gt;. If you plug in known class labels and mangle this for a while with algebra, you end up with a logistic regression using features &lt;span class=&quot;math-container&quot;&gt;$\Phi(x)$&lt;/span&gt;. The intercept term of the logistic regression turns into a function of &lt;span class=&quot;math-container&quot;&gt;$\pi_1/\pi_2$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If your data are Gaussian, then &lt;span class=&quot;math-container&quot;&gt;$\Phi(x)$&lt;/span&gt; would generally include &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$x^2$&lt;/span&gt; and all the cross-terms if &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is multivariate. If there are special assumptions about equal covariance between classes, then the squared terms and cross terms may not be needed.&lt;/p&gt;&#xA;" OwnerUserId="1708" LastActivityDate="2022-02-07T22:37:46.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18519" PostTypeId="2" ParentId="18516" CreationDate="2022-02-08T01:58:51.040" Score="1" Body="&lt;p&gt;That is a good question. I have never done any Microarray experiments or data analysis myself. But I am very familiar with RNA-seq library workflows and data analysis. Library prep methods for RNA-seq involves many steps including use of several enzymes, which introduce certain level of bias. For example, reverse transcriptase used for making cDNA may copy certain mRNA sequences better than others. Other steps that involve enzymes: second strand synthesis, end-repair, ligation, and PCR amplification. Non-enzymatic steps could also introduce biases. I imagine Microarray and probe designs have also significant biases. There could be additional reasons I have not mentioned above, such as improper data analysis. If global gene expression levels are significantly different, I would suspect that it could be experimental reasons e.g. bad lab practices, unreliable reagents etc. I hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="14180" LastEditorUserId="14180" LastEditDate="2022-02-08T02:09:48.800" LastActivityDate="2022-02-08T02:09:48.800" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18521" PostTypeId="1" CreationDate="2022-02-08T13:52:20.937" Score="3" ViewCount="540" Body="&lt;p&gt;My whole (professional) life I am downloading sequences from SRA and ENA databases. Long time ago, I used prefetch from sra-tools, but later I switched to simply using wget from ftp servers of the two respective servers (used to work for both SRA and ENA). But it seems that SRA ftp server does not have the same data in there anymore and I kind of worry about reproducibility of my old pipelines - so what happened to sra ftp server?&lt;/p&gt;&#xA;&lt;p&gt;The paths that used to work were formatted as follows&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ftp://ftp.ncbi.nlm.nih.gov/sra/sra-instant/reads/ByRun/sra/${ACCESION::3}/${ACCESION::6}/&amp;quot;$ACCESION&amp;quot;/&amp;quot;$ACCESION&amp;quot;.sra&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but it seems &lt;code&gt;sra-instant&lt;/code&gt; directory is not accessible anymore. Did the files move? Or were the access been just restricted?&lt;/p&gt;&#xA;&lt;p&gt;The alternative workaround is to dl the files using &lt;code&gt;prefetch&lt;/code&gt; instead of wgetting. Prefetch is part of &lt;a href=&quot;https://hpc.nih.gov/apps/sratoolkit.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sra-tools&lt;/a&gt; and available on &lt;a href=&quot;https://anaconda.org/bioconda/sra-tools&quot; rel=&quot;nofollow noreferrer&quot;&gt;conda&lt;/a&gt; and it's not such a bad solution. I guess I am mostly curious what happened to the ftp server and if anyone knows what is coming next, is ENA going to restrict the access too some time soon?&lt;/p&gt;&#xA;" OwnerUserId="57" LastEditorUserId="57" LastEditDate="2022-02-08T17:12:36.810" LastActivityDate="2022-02-08T21:11:03.597" Title="What happened to sra ftp server?" Tags="|public-databases|data-download|sratoolkit|sra|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18522" PostTypeId="1" AcceptedAnswerId="18523" CreationDate="2022-02-08T15:10:57.527" Score="2" ViewCount="42" Body="&lt;p&gt;Is it possible to infer protein-coding gene evolution age by species divergence time?&lt;/p&gt;&#xA;&lt;p&gt;For example, species A diverged 300 million years (MYA) ago from its common ancestor with species C, whilst species B diverged from C 10 MYA ago.&lt;/p&gt;&#xA;&lt;p&gt;Assume they share a protein with a similar sequence, i.e. they are  assumed to be homologous. Can I say this homologous gene in species A has an older evolution age than the homologous gene in species B?&lt;/p&gt;&#xA;" OwnerUserId="14169" LastEditorUserId="3848" LastEditDate="2022-02-08T17:52:17.810" LastActivityDate="2022-02-08T17:59:26.903" Title="Protein-coding gene evolution age and species divergence time: are they correlated?" Tags="|phylogenetics|proteins|phylogeny|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18523" PostTypeId="2" ParentId="18522" CreationDate="2022-02-08T15:24:01.583" Score="3" Body="&lt;p&gt;Yes this is exactly true, proteins are ideal for deep molecular dating. DNA can suffer from compositional bias. The thing you need is a reliable tree, with strong bootstrap support. You will then calibrate it with your externally established dates.&lt;/p&gt;&#xA;&lt;p&gt;Programs such as Beast allow you to do the two simultaneously. Generally, whatever approach you take you will be required or else requested to use Beast. It is however impressive.&lt;/p&gt;&#xA;&lt;p&gt;What are you dating? Ten million years is not 'deep', however 300 million is very deep indeed for animals (Metazoa), especially so for vertebrates.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To answer your question in general terms molecular dating tends to estimate earlier divergence against the fossil record, or against the perceived view of divergence. Molecular phylogenetics argue this is because the fossil record is not sufficiently well sampled, ie the earliest fossil is missing. Palaeontology historically had a different viewpoint.&lt;/p&gt;&#xA;&lt;p&gt;The plethora of molecular dating methods has continued, an impressive Beast based approach was recently published and I'm not clear how these changes/ improve the dating estimate. My personal view is that generally each new technique improves the accuracy of molecular dating.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Answering the second part of your question, reconstructing the ancestral sequence at a given node is formal analysis and can be performed using ML, Bayes or parsimoney. Thus it attempts to estimate the ancestral sequence and you can then compare the mutations against extant taxa. It is perhaps better to say it establishes a robust hypothesis regarding ancestral mutations.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-08T17:59:26.903" LastActivityDate="2022-02-08T17:59:26.903" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18524" PostTypeId="2" ParentId="18521" CreationDate="2022-02-08T17:39:47.763" Score="3" Body="&lt;p&gt;SRA moved all of the data to the cloud per &lt;a href=&quot;https://ncbiinsights.ncbi.nlm.nih.gov/2020/02/24/sra-cloud/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ncbiinsights.ncbi.nlm.nih.gov/2020/02/24/sra-cloud/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can still download data for free using &lt;code&gt;sratoolkit&lt;/code&gt; as described in the official documentation here: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra/docs/sradownload/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/sra/docs/sradownload/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can download the data using sratoolkit without having to pay cloud providers. There are some user-pays data on the cloud but for FASTQ files you don't have to pay.&lt;/p&gt;&#xA;&lt;p&gt;For documentation on how to download data from the cloud, see &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra/docs/sra-aws-download/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/sra/docs/sra-aws-download/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3577" LastEditorUserId="73" LastEditDate="2022-02-08T21:11:03.597" LastActivityDate="2022-02-08T21:11:03.597" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18525" PostTypeId="2" ParentId="18521" CreationDate="2022-02-08T19:12:07.810" Score="2" Body="&lt;p&gt;NCBI appeared at least in part to shift onto Google Cloud buckets, one previous OP raised a question about access via GCP (I think the question was closed). Fundamentally, traditional FTP is not sustainable against data growtch. GCP is accessed via gsutil. It would be good if you could contact NCBI and post their official response here. They will response when their database provision is in a state of flux.&lt;/p&gt;&#xA;&lt;p&gt;sratoolkit appears to be a front-end to GCPs gsutil. Just to point out, it would be very unusual for NCBI to charge the user for e.g. direct gsutil download, because usually the provider will charge e.g. NCBI on a per download basis. This will likely work out cheaper overall for NCBI.&lt;/p&gt;&#xA;&lt;p&gt;Just to formalise my bit, if SRA are hosting on a GCP bucket, Google will not permit ftp access which is at least part the reason behind your question.&lt;/p&gt;&#xA;&lt;p&gt;They do have their own ftp equivalent called &lt;code&gt;gsutil&lt;/code&gt; which will permit direct download if you know the bucket name. &lt;code&gt;gsutil&lt;/code&gt; is part of the ClouD SDK package and can be downloaded via conda/anaconda &lt;code&gt;conda install gcloud-sdk&lt;/code&gt;. This will give you command line access if you know the bucket names. There is a past post on this BTW.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="73" LastEditDate="2022-02-08T21:10:08.963" LastActivityDate="2022-02-08T21:10:08.963" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18526" PostTypeId="2" ParentId="2361" CreationDate="2022-02-08T20:35:11.677" Score="1" Body="&lt;p&gt;New methods have been published in the time since this question was asked. Now, the relative abundance of organisms across different taxonomic levels can be estimated directly from metagenomic shotgun sequencing data without the need to identify and pull rRNA genes. Here are a few examples of popular tools:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Similar to the approach described in the question, &lt;a href=&quot;https://github.com/motu-tool/mOTUs&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;mOTUs&lt;/strong&gt;&lt;/a&gt; employs a database of single-copy phylogenetic marker genes to create taxonomic profiles from short-read metagenomic data.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/DerrickWood/kraken2&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Kraken&lt;/strong&gt;&lt;/a&gt; associates short genomic substrings (&lt;em&gt;k&lt;/em&gt;-mers) with the lowest common ancestor in its database. The output can then be processed with &lt;a href=&quot;https://github.com/jenniferlu717/Bracken/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Bracken&lt;/strong&gt;&lt;/a&gt; to get estimates of relative abundance at any taxonomic level, and the Bracken output table can then be passed to a tool like &lt;a href=&quot;https://github.com/acvill/bracken_plot&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;bracken_plot&lt;/strong&gt;&lt;/a&gt; to visualize differences in relative abundance between samples.&lt;/li&gt;&#xA;&lt;li&gt;Like mOTUs, &lt;a href=&quot;https://huttenhower.sph.harvard.edu/metaphlan/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;MetaPhlAn&lt;/strong&gt;&lt;/a&gt; uses a large marker gene database to assign reads to bacterial, archaeal, viral, and eukaryotic clades. This allows for relative abundance estimation in terms of number of cells. &lt;a href=&quot;https://elifesciences.org/articles/65088&quot; rel=&quot;nofollow noreferrer&quot;&gt;If their paper is to be believed&lt;/a&gt;, MetaPhlAn3 outperforms both mOTUs2 and Kraken2/Bracken in terms of abundance profile accuracy.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In addition to more accurate species detection, MetaPhlAn3 also quantified taxonomic abundance profiles more accurately compared to MetaPhlAn 2, mOTUs2, and Bracken based on Bray-Curtis similarity in most datasets ... While it was slightly outperformed by mOTUs2 (only in high-recall mode) on the synthetic mouse gut dataset, even in this case, correlation-based measures (Pearson Correlation Coefficient between estimated and expected relative abundances) found MetaPhlAn3 to be more accurate (r = 0.73) than the other considered profilers (MetaPhlAn2 r = 0.63, mOTUs2 precision r = 0.60, mOTUs2 recall r = 0.71, Bracken r = 0.43).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Although it is now dated, &lt;a href=&quot;http://ccb.jhu.edu/software/choosing-a-metagenomics-classifier/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;How to Choose Your Metagenomics Classification Tool&lt;/em&gt;&lt;/a&gt; from the Salzberg lab lists more options and links the associated publications, including &lt;a href=&quot;https://github.com/DaehwanKimLab/centrifuge&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Centrifuge&lt;/strong&gt;&lt;/a&gt; and &lt;a href=&quot;https://github.com/fbreitwieser/pavian&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Pavian&lt;/strong&gt;&lt;/a&gt;, the latter of which is geared specifically toward visualization of metagenomic classification data.&lt;/p&gt;&#xA;" OwnerUserId="3967" LastActivityDate="2022-02-08T20:35:11.677" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18527" PostTypeId="2" ParentId="18516" CreationDate="2022-02-08T21:07:35.023" Score="0" Body="&lt;p&gt;It depends on the microarray and the cDNA sequencing sample prep.&lt;/p&gt;&#xA;&lt;p&gt;One difference that I know about is that microarrays can be designed to target specific isoforms (e.g. spanning a isoform-specific splice boundary), whereas the shotgun sequencing approach of short-read cDNA sequencing makes this difficult to achieve (although in many cases isoform counting can be done to some degree by observing differences in coverage across transcripts).&lt;/p&gt;&#xA;&lt;p&gt;However, microarrays are very heavily dependent on a good annotation. They can only detect existing known sequences, so it's possible that multiple isoforms are covered by a sequence that was previously believed to be isoform-unique, or that a sequence picked up by shotgun cDNA sequencing is not detected at all by a microarray.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-02-08T21:07:35.023" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18528" PostTypeId="2" ParentId="18476" CreationDate="2022-02-08T21:29:34.880" Score="4" Body="&lt;p&gt;I don't have a recommendation of specific tools, but we might be able to use the GFA file itself to get at this with just bash commands.&lt;/p&gt;&#xA;&lt;p&gt;For these examples I use the linked GFA &lt;a href=&quot;https://towardsdatascience.com/visualising-assembly-graphs-fb631f46bbd1&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;, which is a tutorial with more useful info.&lt;/p&gt;&#xA;&lt;p&gt;GFA segment (node) lines start with &lt;code&gt;S&lt;/code&gt;, link lines start with &lt;code&gt;L&lt;/code&gt; (spec &lt;a href=&quot;http://gfa-spec.github.io/GFA-spec/&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;S   3438628 CACAATCTAAGAAATCTGAACTACCTGAAACAGGTGGAGAAGAATCAACAAACAACGGCATGTTGTTCGGCGGATTATTTAGCATTTTAG...&#xA;L   3461020 +   3461022 +   55M&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Links connect the numeric IDs of segments to each other.&lt;/p&gt;&#xA;&lt;h2&gt;Find high-degree nodes&lt;/h2&gt;&#xA;&lt;p&gt;We can just count the occurrences of node IDs in links, which should give us the degree.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;egrep &amp;quot;\bL\b&amp;quot; | cut -f4 assembly_graph_with_scaffolds.gfa  | sort | uniq -c | sort | tail&#xA;&#xA;# prints:&#xA;      &#xA;      5 3461076&#xA;      5 3461246&#xA;      5 3461286&#xA;      5 591392&#xA;      5 606626&#xA;      5 634272&#xA;      5 721568&#xA;      5 766622&#xA;      5 822586&#xA;      6 3455660&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So we can see that there are a lot of links of segment &lt;code&gt;3455660&lt;/code&gt;. We can confirm this in the GFA:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep 3455660 assembly_graph_with_scaffolds.gfa&#xA;# prints:&#xA;S       3455660 CTGATAAACGGCGGGAACAGAACCAACACTACGCGTTGCTCCGATCTCAACAACTTCTAAG   KC:i:2936&#xA;L       3455660 +       56698   -       55M&#xA;L       3455660 +       3455666 +       55M&#xA;L       3455660 +       3456290 -       55M&#xA;L       3455660 -       3455652 -       55M&#xA;L       3455660 -       3456398 -       55M&#xA;P       NODE_6_length_300278_cov_129.624456_1   3448698+,3456292-,3448662+,3448670+,3451896-,3460544-,3460388+,3434104-,3434124-,56698+,3455660-,3455652-,3460374+,3460806-,3460824-,3460806-,3460824-,3460806-,3460530-,3449022-,3454210+,3459744-,3459742-    *&#xA;P       NODE_8_length_282610_cov_136.723533_1   3460440-,3434818+,3276488+,3461076-,3461070-,3461062-,3461054-,3461046-,3461038-,3460426-,3461020+,3461022+,3461020+,3460516-,3455644+,3455652+,3455660+,3456290-,3448662+,3460546-,3275518-,3387950+,3456304+,3367628+,523826-,3418048-,3456388+,3460376-,3327686+,3455382-,3460586+,655086-,3275436+,3460670+,3278936+,3460218+,3460346+,3460498+,3459664+,3460496+,3459664+,3459200+,3459424-,3458356+,3455426-,3455420-     *&#xA;P       NODE_39_length_23557_cov_173.180283_1   3456398+,3455660+,3455666+,3418048-,3273044-,3274776+,3433484-,3433478-,3433470-,3433462-,3456812-,3460134+,3455826-,3319248+,3454700+,3421874+,3453038+,3459348-,3453046-,3234810-,3460596+,3460980+,3460654+  *&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Extract sequences proximal to a node&lt;/h2&gt;&#xA;&lt;p&gt;Having extracted the high-degree node, we can use the output of &lt;code&gt;grep&lt;/code&gt; above to find links to other sequences:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep 3455660 assembly_graph_with_scaffolds.gfa | egrep &amp;quot;\bL&amp;quot;&#xA;# prints:&#xA;L   3455660 +   56698   -   55M&#xA;L   3455660 +   3455666 +   55M&#xA;L   3455660 +   3456290 -   55M&#xA;L   3455660 -   3455652 -   55M&#xA;L   3455660 -   3456398 -   55M&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We can then go lookup &lt;code&gt;S&lt;/code&gt; lines for those segments:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;egrep &amp;quot;\bS\W56698&amp;quot; assembly_graph_with_scaffolds.gfa&#xA;# prints:&#xA;S   56698   GCATGTCGAGAAAACACCTTAGAAGTTGTTGAGATCGGAGCAACGCGTAGTGTTGGTTCTGTTCCCGCCGTT    KC:i:2067&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Coverage filtering&lt;/h2&gt;&#xA;&lt;p&gt;Read count (&lt;code&gt;RC&lt;/code&gt; tag) is an optional field for GFA, which does not exist in the linked file.&lt;/p&gt;&#xA;&lt;p&gt;However, we do have k-mer count (&lt;code&gt;KC&lt;/code&gt;), which you could use. Presumably high k-mer count would correlate well to coverage. I don't have a suggestion of a nifty tool that filters on this, I'd use a custom script.&lt;/p&gt;&#xA;&lt;h1&gt;Summary&lt;/h1&gt;&#xA;&lt;p&gt;Probably there exists some command line tool that does this stuff, but I don't know about it. It has to all exist inside assemblers, but it might be pretty hard to pull out.&lt;/p&gt;&#xA;&lt;p&gt;If I had a little more time, I might write a simple tool that uses knowledge of the GFA structure to perform these kinds of data slicing operations, it seems pretty useful. It really shouldn't be hard to do this stuff!&lt;/p&gt;&#xA;&lt;p&gt;A related approach, that I believe has a somewhat different focus, can be found &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2020.05.21.109587v2.full.pdf&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2022-02-08T21:35:41.433" LastActivityDate="2022-02-08T21:35:41.433" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18529" PostTypeId="1" CreationDate="2022-02-09T00:05:03.563" Score="4" ViewCount="97" Body="&lt;p&gt;I have generated a ggplot for 8 single-cell libraries, with the purpose of visualizing the tSNE facet plot by sample, colored by cell type -- with percentages. The best I could get to is this -&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2vUiy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2vUiy.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;however, it looks too crowded, and I also want the cell types to be labeled for each cluster, which makes it look even worse.&lt;br /&gt;&#xA;How can I improve the data visualization here?&lt;/p&gt;&#xA;&lt;p&gt;my code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;p6 &amp;lt;- ggplot(tsnePlot, aes(x=tSNE_1, y=tSNE_2, color=RefinedCellTypes)) + &#xA;  scale_color_manual(values=refinedCellTypeColors) +&#xA;  geom_point(data = tsnePlotAll, mapping = aes(x=tSNE_1, y=tSNE_2), size = 1, alpha = 1, color = &amp;quot;grey75&amp;quot;) +&#xA;  geom_point(size = 1, alpha = 1) +&#xA;  geom_text(data = tsneClusterLabelsRefined, mapping = aes(x = tSNE_1, y = tSNE_2, label = CellType),color=&amp;quot;black&amp;quot;,size=0.75,hjust=0) +&#xA;  geom_label(data = tsneCellTypeFractions, mapping = aes(x = tSNE_1, y = tSNE_2, label = scales::percent(Freq,accuracy = 0.1)),color=&amp;quot;black&amp;quot;,fill = &amp;quot;white&amp;quot;, alpha = 0.5, label.size = 0) +&#xA;  facet_wrap(~Sample) +&#xA;  theme_bw(base_size = 12) +&#xA;  theme(legend.position=&amp;quot;left&amp;quot;,&#xA;        axis.text=element_text(size=12, color = &amp;quot;black&amp;quot;),&#xA;        axis.title=element_text(size=10))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there any way I can control the number of cols/rows for the figure?&lt;/p&gt;&#xA;" OwnerUserId="14823" LastEditorUserId="73" LastEditDate="2022-02-09T21:26:30.937" LastActivityDate="2022-02-09T21:26:30.937" Title="Arrange ggplot Figure for scRNA-seq data" Tags="|scrnaseq|single-cell|visualization|ggplot2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18530" PostTypeId="1" CreationDate="2022-02-09T07:51:40.963" Score="1" ViewCount="17" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; My understanding was that if I did scATAC-seq and I have some clusters of cell groups, the only way I can label it is by correlating those groups with scRNA-seq data.&lt;/p&gt;&#xA;&lt;p&gt;My lab ordered some scATAC-seq data and the vendor did some complimentary analysis and provided cell composition plots indicating which portion of the cells of type X, Y, Z, so on.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; How is this possible to identity cell composition identifying given cell types? Alternatively is it possible they just used some default RNA-seq data from 10x genomics?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Definitions&lt;/p&gt;&#xA;&lt;p&gt;scATAC-seq stands for &amp;quot;Single-cell sequencing assay for transposase-accessible chromatin&amp;quot;  This attempts to consider genome-wide regulatory landscapes in single cells. scATAC-seq data can be sparse and noisy presenting a challenge for analysis.&lt;/p&gt;&#xA;" OwnerUserId="2519" LastEditorUserId="3848" LastEditDate="2022-03-26T14:54:08.393" LastActivityDate="2022-03-26T14:54:08.393" Title="Is it possible to do cell composition for scATAC-seq data without scRNA-seq data?" Tags="|rna-seq|single-cell|atac-seq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18531" PostTypeId="1" CreationDate="2022-02-09T14:56:31.983" Score="0" ViewCount="40" Body="&lt;p&gt;Suppose there is cohort with many samples to assay. In phase I of the project, subjects with a certain disease incidence and healthy controls were assayed. Now, in phase II of the project, we are going to assay samples with another disease and analyze them with healthy controls from phase I of the project. From past experience, we know that we have to assay healthy controls from phase I (whose bio samples are still available for assay) to avoid batch effects between phase I and phase II of the project. However, I don't know how much duplicate samples I should include and how the duplicates should be distributed.&lt;/p&gt;&#xA;&lt;p&gt;My question is certainly related to experiment design, and perhaps there have been extensive discussions on this kind of design, but I don't know the exact term of this kind of experiment design such that I couldn't find anything relevant to my case in the literature. Can anyone point me to references related to this kind of experiment design?&lt;/p&gt;&#xA;" OwnerUserId="10056" LastEditorUserId="10056" LastEditDate="2022-02-09T16:01:29.497" LastActivityDate="2024-03-01T01:09:36.237" Title="References request: deciding what duplicate samples to include to avoid batch effects with previous studies" Tags="|batch-effects|design|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18532" PostTypeId="2" ParentId="18529" CreationDate="2022-02-09T14:57:05.360" Score="1" Body="&lt;p&gt;Try&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;p6 + geom_point(alpha = 1/100)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And variations of 1/10 to 1/1000, this will give a better idea of point density.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-09T18:06:32.737" LastActivityDate="2022-02-09T18:06:32.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18533" PostTypeId="2" ParentId="18529" CreationDate="2022-02-09T18:40:17.703" Score="3" Body="&lt;p&gt;On top of M__'s suggestion to reduce the point alpha, here are a few other things you can implement to make your figures cleaner:&lt;/p&gt;&#xA;&lt;p&gt;Remove grid lines from the panel backgrounds&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;theme(panel.grid.major = element_blank(),&#xA;      panel.grid.minor = element_blank())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I think a figure legend like the one you have now is a better approach than labeling every cell type in every facet. However, if you still want to go that route, I suggest implementing repulsive labels with the &lt;a href=&quot;https://cran.r-project.org/web/packages/ggrepel/vignettes/ggrepel.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ggrepel package&lt;/a&gt;. &lt;code&gt;geom_label_repel()&lt;/code&gt; can also be applied to your percentages so they are not overlapping.&lt;/p&gt;&#xA;" OwnerUserId="3967" LastActivityDate="2022-02-09T18:40:17.703" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18534" PostTypeId="2" ParentId="18517" CreationDate="2022-02-09T20:23:20.587" Score="2" Body="&lt;p&gt;I reached out to the maintainers, and these results are derived from &lt;em&gt;re-analysis of the raw reads obtained from GEO.&lt;/em&gt; This is described in the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6324031/&quot; rel=&quot;nofollow noreferrer&quot;&gt;regulonDB 10.5 paper&lt;/a&gt; sections &amp;quot;HT dataset processing&amp;quot; and &amp;quot;Reconstructing regulatory interactions&amp;quot;. The processing pipeline, described in fig S1, is fairly typical -- finding motif matches in MACS2 peaks. The cutoff used was simply lower than the source paper, yielding more hits.&lt;/p&gt;&#xA;&lt;p&gt;Edit: This pipeline description also explains why the tables you download contain duplicates (same regulator, same target). Duplicates can arise from different binding sites upstream of the same gene.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bUY1a.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bUY1a.png&quot; alt=&quot;A bioinformatics pipeline for processing RNA-seq and ChIP data from GEO includes peak calling (MACS2), binding-site search (RSAT), and RNA differential expression (FDR-corrected t-test).&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1708" LastEditorUserId="1708" LastEditDate="2022-03-07T21:18:29.980" LastActivityDate="2022-03-07T21:18:29.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18535" PostTypeId="2" ParentId="18531" CreationDate="2022-02-09T21:21:11.517" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;at least six biological replicates should be used, rising to at least 12 when it is important to identify SDE genes for all fold changes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;See this paper for an explanation of why six replicates is good for differential expression analysis:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://dx.doi.org/10.1261/rna.053959.115&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://dx.doi.org/10.1261/rna.053959.115&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;When looking at conditions, it's necessary to think about what exactly needs to be compared (ideally prior to doing any experiments or sequencing, because that reduces the cost and frustration of re-doing things).&lt;/p&gt;&#xA;&lt;p&gt;I recommend you consult with a statistician to find out the appropriate mixture of samples for your experiment, because there can be subtle differences between studies that mean a particular design does not work.&lt;/p&gt;&#xA;&lt;p&gt;One example from our own investigations is that we were doing cDNA sequencing of cell lines with and without mitochondrial genomes, and it was only after doing two years worth of sequencing and looking at the data that we realised we'd never done any sequencing runs that only involved cell lines without mitochondrial DNA - our rationale for doing that was to reduce batch effects. This meant we couldn't perfectly distinguish between barcode spillover and a very small proportion of mitochondrial DNA in our &amp;quot;mitochondria-free&amp;quot; samples.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-02-09T21:21:11.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18536" PostTypeId="1" AcceptedAnswerId="18537" CreationDate="2022-02-10T00:43:32.300" Score="2" ViewCount="1521" Body="&lt;p&gt;Using &lt;code&gt;Nextflow&lt;/code&gt;, I need to submit &lt;code&gt;x * y&lt;/code&gt; jobs where &lt;code&gt;x&lt;/code&gt; is the number of input &lt;code&gt;.bam&lt;/code&gt; files and &lt;code&gt;y&lt;/code&gt; is the number of genome intervals (e.g., &lt;code&gt;&amp;lt;chrom&amp;gt;:&amp;lt;start&amp;gt;-&amp;lt;end&amp;gt;&lt;/code&gt;). i.e., for every &lt;code&gt;.bam&lt;/code&gt; file, submit a single job for every defined interval.&lt;/p&gt;&#xA;&lt;p&gt;I have tried a few variations with no luck (see below).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Ultimate question:&lt;/strong&gt; For each &lt;code&gt;.bam&lt;/code&gt; file, how do I submit a single job for every defined genome interval using &lt;code&gt;NextFlow&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;h2&gt;Attempts&lt;/h2&gt;&#xA;&lt;p&gt;&lt;strong&gt;Attempt 1 (submit both as individual &lt;code&gt;Channels&lt;/code&gt;):&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Only submits one &lt;code&gt;.bam&lt;/code&gt; for all intervals.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sample_bams = Channel.fromPath(sample_input_path + &amp;quot;*.bam&amp;quot;)&#xA;intervals = Channel.from(['1:10000-20000', '5:55555-77777'])&#xA;PROC(sample_bams, intervals)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Attempt 2 (submit as tuple of mapped &lt;code&gt;Channel&lt;/code&gt;s):&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Submits only 10 jobs and fails because the interval &lt;code&gt;Channel&lt;/code&gt; is passed as &lt;code&gt;DataflowBroadcast around DataflowStream[?]&lt;/code&gt; object. The &lt;code&gt;PROC&lt;/code&gt; was changed to receive a tuple rather than individual arguments.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sample_interval_tuples = Channel.fromPath(sample_input_path + &amp;quot;*.bam&amp;quot;)&#xA;                         .map { sample_file -&amp;gt; tuple(sample_file, align_to_ref, DRF_jar, Channel.from(['1:10000-20000', '5:55555-77777'])) }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Attempt 3 (submit as tuple of mapped &lt;code&gt;.bam&lt;/code&gt;s to non-&lt;code&gt;Channel&lt;/code&gt; of intervals)&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;Submits 10 jobs and fails because the intervals are passed as a single list. The &lt;code&gt;PROC&lt;/code&gt; was changed to receive a tuple rather than individual arguments.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sample_interval_tuples = Channel.fromPath(sample_input_path + &amp;quot;*.bam&amp;quot;)&#xA;                         .map { sample_file -&amp;gt; tuple(sample_file, align_to_ref, DRF_jar, ['1:10000-20000', '5:55555-77777']) }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Really appreciate your help!&lt;/p&gt;&#xA;&lt;p&gt;related query: for DSL v2, we wouldn't include from combined_inputs in the input, yeah?&lt;/p&gt;&#xA;" OwnerUserId="136" LastEditorUserId="73" LastEditDate="2022-02-10T23:12:59.463" LastActivityDate="2022-02-10T23:13:31.693" Title="Nextflow: how to create many-to-many tuple for process" Tags="|nextflow|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18537" PostTypeId="2" ParentId="18536" CreationDate="2022-02-10T01:57:46.470" Score="4" Body="&lt;p&gt;The &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#combine&quot; rel=&quot;nofollow noreferrer&quot;&gt;combine&lt;/a&gt; operator can be used to produce the Cartesian product:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sample_bams = Channel.fromPath( './path/to/bams/*.bam' )&#xA;intervals = Channel.of( '1:10000-20000', '5:55555-77777' )&#xA;&#xA;sample_bams&#xA;    .combine(intervals)&#xA;    .set { combined_inputs }&#xA;&#xA;&#xA;process test {&#xA;&#xA;    echo true&#xA;&#xA;    input:&#xA;    tuple path(bam), val(interval) from combined_inputs&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    echo -n &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${interval} $&lt;/span&gt;{bam}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Regarding the related query: Correct. The from and into 'bind' declarations can be omitted&lt;/p&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="73" LastEditDate="2022-02-10T23:13:31.693" LastActivityDate="2022-02-10T23:13:31.693" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18538" PostTypeId="1" AcceptedAnswerId="18626" CreationDate="2022-02-10T04:36:51.217" Score="3" ViewCount="3544" Body="&lt;p&gt;We're preparing to analyze thousands of &lt;code&gt;.bam&lt;/code&gt; files beginning with re-alignment, sorting, etc.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Complicated question:&lt;/strong&gt; Has anyone investigated the optimal thread and memory settings to sort hundreds to thousands of &lt;code&gt;.bam&lt;/code&gt; files using &lt;code&gt;samtools sort&lt;/code&gt; on a cluster?&lt;/p&gt;&#xA;&lt;p&gt;By 'optimal', I'm looking to sort as many samples as possible in the shortest time frame. I'm mostly interested in optimizing &lt;code&gt;samtools sort&lt;/code&gt; because the other steps (e.g., alignment) are much more obvious to fine-tune.&lt;/p&gt;&#xA;&lt;p&gt;For a single sample, throwing large amounts of memory + threads is trivial, but that isn't scalable for thousands of samples. There is also likely a &lt;a href=&quot;https://en.wikipedia.org/wiki/Diminishing_returns&quot; rel=&quot;nofollow noreferrer&quot;&gt;diminishing return&lt;/a&gt; at a certain point, even for a single sample.&lt;/p&gt;&#xA;&lt;p&gt;Ideally, I'd like to see plots identifying the 'sweet spot' for threads + memory for a single sample (e.g., x == threads, y == memory, z == time), and then plots maximizing throughput for more samples. I would expect the data to follow a -log function, where we could identify the optimal settings at the &lt;a href=&quot;https://en.wikipedia.org/wiki/Knee_of_a_curve#:%7E:text=In%20mathematics%2C%20a%20knee%20of,or%20in%20the%20other%20direction.&quot; rel=&quot;nofollow noreferrer&quot;&gt;knee in the curve&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There are obviously many other factors (e.g., cluster size, administrative scheduler settings, etc.), but I suspect findings on a given cluster will generalize to other clusters.&lt;/p&gt;&#xA;&lt;p&gt;Really appreciate any feedback.&lt;/p&gt;&#xA;" OwnerUserId="136" LastEditorUserId="73" LastEditDate="2024-01-04T21:25:34.440" LastActivityDate="2024-01-04T21:25:34.440" Title="Samtools sort: most efficient memory and thread settings for many samples on a cluster" Tags="|sequence-alignment|sequence-analysis|samtools|benchmarking|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18539" PostTypeId="1" CreationDate="2022-02-10T05:02:14.710" Score="2" ViewCount="413" Body="&lt;p&gt;Related to my other question (&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/18538/samtools-sort-most-efficient-memory-and-thread-settings-for-many-samples-on-a-c&quot;&gt;Samtools sort: most efficient memory and thread settings for many samples on a cluster&lt;/a&gt;), we need to optimize &lt;code&gt;samtools sort&lt;/code&gt; as we prepare to analyze thousands of &lt;code&gt;.bam&lt;/code&gt; files, beginning with re-alignment, sorting, etc.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Ultimate question&lt;/strong&gt;: what is the fastest way to sort a single sample after aligning the sample across many separate cluster jobs? i.e., after splitting &lt;code&gt;.fastq&lt;/code&gt;s for a single sample and aligning as distinct jobs. I'm looking for recent and clear time/thread/memory comparisons.&lt;/p&gt;&#xA;&lt;p&gt;Unless I'm missing something, &lt;code&gt;samtools merge&lt;/code&gt; was designed exactly for the purpose of merging pre-sorted bams. An excerpt from the manpage (&lt;a href=&quot;http://www.htslib.org/doc/samtools-merge.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.htslib.org/doc/samtools-merge.html&lt;/a&gt;): &amp;quot;Merge multiple sorted alignment files, producing a single sorted output file that contains all the input records and maintains the existing sort order.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Is it fastest to:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;samtools sort&lt;/code&gt; each 'mini-bam' and then &lt;code&gt;samtools merge&lt;/code&gt;?&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;samtools cat&lt;/code&gt; the 'mini-bams' together and then &lt;code&gt;samtools sort&lt;/code&gt;?&lt;/li&gt;&#xA;&lt;li&gt;Another option I haven't considered?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; I have seen the following similar questions on other forums, but they are outdated (~8 years ago) and do not answer this question directly:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.biostars.org/p/9864/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Best Way To Merge A Many Thousand Small Bam Files Into One Big Bam File?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://seqanswers.com/forums/showthread.php?t=47110&quot; rel=&quot;nofollow noreferrer&quot;&gt;Merging BAM files before sorting&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="136" LastEditorUserId="73" LastEditDate="2022-02-11T04:44:18.180" LastActivityDate="2022-02-11T05:00:00.417" Title="Samtools sort: most efficient approach to sort a single sample after aligning split .fastq files" Tags="|sequence-alignment|sequence-analysis|samtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18540" PostTypeId="2" ParentId="18536" CreationDate="2022-02-10T10:38:07.617" Score="2" Body="&lt;p&gt;I think this cheat sheet might help:&#xA;&lt;a href=&quot;https://github.com/danrlu/Nextflow_cheatsheet/blob/main/nextflow_cheatsheet.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/danrlu/Nextflow_cheatsheet/blob/main/nextflow_cheatsheet.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2486" LastActivityDate="2022-02-10T10:38:07.617" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18542" PostTypeId="1" CreationDate="2022-02-10T16:24:45.500" Score="3" ViewCount="142" Body="&lt;p&gt;I was playing around, looking at some AlphaFold predictions for certain proteins and I realized that sometimes the sequences of aminoacid look very different for the same UniProt id. For example, 15C8 from PDB has 2 chains:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;DIVLTQSPAIMSASLGERVTMTCTASSSVSSSNLHWYQQKPGSSPKLWIYSTSNLA&#xA;SGVPARFSGSGSGTSYSLTISSMEAEDAATYYCHQYHRSPYTFGGGTKLEIKRADA&#xA;APTVSIFPPSSEQLTSGGASVVCFLNNFYPKDINVKWKIDGSERQNGVLNSWTDQD&#xA;SKDSTYSMSSTLTLTKDEYERHNSYTCEATHKTSTSPIVKSFNRN&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;EVQLQQSGAELVKPGASVKLSCTASGFNIKDTYMHWVKQKPEQGLEWIAQIDPANG&#xA;NTKYDPKFQGKATITADTSSNTAYLHLSSLTSEDSAVYYCAADPPYYGHGDYWGQG&#xA;TTLTVSSAKTTPPSVYPLAPGSAAQTNSMVTLGCLVKGYFPEPVTVTWNSGSLSSG&#xA;VHTFPAVLQSDLYTLSSSVTVPSSTWPSETVTCNVAHPASSTKVDKKIV&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;At the same time, when looking at the UniProt database (the corresponding id is P01869), there's a list of multiple structures and their 3D shapes (here: &lt;a href=&quot;https://www.uniprot.org/uniprot/P01869&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uniprot/P01869&lt;/a&gt;). However, looking at the AlphaFold entry, I see that the amino acid sequence looks different, eg. for the case above it's:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;AKTTPPSVYPLAPGSAAQTNSMVTLGCLVKGYFPEPVTVTWNSGSLSSGVHTFPAV&#xA;LQSDLYTLSSSVTVPSSPRPSETVTCNVAHPASSTKVDKKIVPRDCGCKPCICTVP&#xA;EVSSVFIFPPKPKDVLTITLTPKVTCVVVDISKDDPEVQFSWFVDDVEVHTAQTQP&#xA;REEQFNSTFRSVSELPIMHQDWLNGKEFKCRVNSAAFPAPIEKTISKTKGRPKAPQ&#xA;VYTIPPPKEQMAKDKVSLTCMITDFFPEDITVEWQWNGQPAENYKNTQPIMNTNGS&#xA;YFVYSKLNVQKSNWEAGNTFTCSVLHEGLHNHHTEKSLSHSPGLQLDETCAEAQDG&#xA;ELDGLWTTITIFISLFLLSVCYSAAVTLFKVKWIFSSVVELKQTLVPEYKNMIGQA&#xA;P&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My question (as someone who's just getting started with proteins) would be: what is the relationship between all these entries corresponding to the same UniProt Id?&lt;/p&gt;&#xA;" OwnerUserId="12036" LastEditorUserId="1586" LastEditDate="2022-07-11T17:12:33.733" LastActivityDate="2022-08-10T18:00:20.050" Title="Relationship between entries for same UniProt Id" Tags="|proteins|protein-structure|pdb|uniprot|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18543" PostTypeId="2" ParentId="18542" CreationDate="2022-02-10T16:46:22.713" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://www.rcsb.org/structure/15C8&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB structure 15C8&lt;/a&gt; corresponds to the protein product of the mouse &lt;em&gt;Igkc&lt;/em&gt; gene, with the UniProt accession &lt;a href=&quot;https://www.uniprot.org/uniprot/P01837&quot; rel=&quot;nofollow noreferrer&quot;&gt;P01837&lt;/a&gt; as you can see in the linked page:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rfTlX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rfTlX.png&quot; alt=&quot;screenshot of PDB entry 15C8&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Your &lt;a href=&quot;https://www.uniprot.org/uniprot/P01869&quot; rel=&quot;nofollow noreferrer&quot;&gt;P01869&lt;/a&gt; UniProt accession is for the protein product of the mouse &lt;em&gt;Ighg1&lt;/em&gt; gene instead, so why would you expect these to be the same sequence? They are different proteins coded for by different genes.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2022-02-10T16:46:22.713" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18544" PostTypeId="1" CreationDate="2022-02-10T18:34:26.277" Score="4" ViewCount="93" Body="&lt;p&gt;I have recently acquired some 16S metagenomics data, and was wondering if anyone can speak of the potential limitations, challenges as well as advantages to conducting a network-based study on metagenomics, such as what is done w.r.t co-occurrence networks.&lt;/p&gt;&#xA;&lt;p&gt;I thought it would be interesting to deviate from the &amp;quot;classic&amp;quot; study of taxonomic abundancies present in different timepoints / conditions. However, it seems as though for such studies they are usually dealing with a relatively large number of samples - please find paper below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://animalmicrobiome.biomedcentral.com/articles/10.1186/s42523-020-00054-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://animalmicrobiome.biomedcentral.com/articles/10.1186/s42523-020-00054-4&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Could such analyses be done with much smaller sample sizes such as those generated via mouse models? What are the potential limitations of such an approach, and how could they be overcome?&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="3848" LastEditDate="2023-06-10T23:41:40.257" LastActivityDate="2023-06-10T23:41:40.257" Title="Co-occurrence networks in Metagenomics studies" Tags="|phylogenetics|phylogeny|metagenome|16rrna|microbiome|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="18545" PostTypeId="1" AcceptedAnswerId="18546" CreationDate="2022-02-10T19:26:30.550" Score="2" ViewCount="26" Body="&lt;p&gt;I have downloaded two datasets, and I am trying to remove the genes they do not share in common.  Is there a biocmanager package with a function that can compare their gene ids, match them and reorder them?&lt;/p&gt;&#xA;" OwnerUserId="2482" LastEditorUserId="298" LastEditDate="2022-02-11T09:04:45.343" LastActivityDate="2022-02-11T09:04:45.343" Title="Best way to match genes in two RNA-seq count matrices from two geo accessions" Tags="|rna-seq|bioconductor|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18546" PostTypeId="2" ParentId="18545" CreationDate="2022-02-10T23:07:06.367" Score="3" Body="&lt;p&gt;Assuming the datasets are arranged in matrices with one gene per row, this can be done by intersecting row names:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;commonNames &amp;lt;- sort(intersect(rownames(data1.mat), rownames(data2.mat)));&#xA;filtered.data1.mat &amp;lt;- data1.mat[commonNames,];&#xA;filtered.data2.mat &amp;lt;- data2.mat[commonNames,];&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It would be helpful if you could provide more information about your datasets. There are many other ways to do this, and the way that works best for you will depend on your particular situation.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-02-10T23:07:06.367" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18549" PostTypeId="1" AcceptedAnswerId="18554" CreationDate="2022-02-11T00:48:35.140" Score="5" ViewCount="2405" Body="&lt;p&gt;I have a workflow that needs to:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Generate &lt;code&gt;.fastq&lt;/code&gt; files from &lt;code&gt;.bams&lt;/code&gt; (while preserving the &lt;code&gt;@RG&lt;/code&gt; group from the original bam)&lt;/li&gt;&#xA;&lt;li&gt;Split the &lt;code&gt;.fastqs&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Align&lt;/li&gt;&#xA;&lt;li&gt;etc.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The only way I could think to preserve the &lt;code&gt;@RG&lt;/code&gt; group was to print it to a file during step 1 and &lt;code&gt;emit&lt;/code&gt; the file. I can't figure out how to synchronize the emitted &lt;code&gt;@RG&lt;/code&gt; file with the &lt;code&gt;.fastqs&lt;/code&gt;, however. I've tried several variations without luck.&lt;/p&gt;&#xA;&lt;p&gt;Questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How do I synchronize the emitted &lt;code&gt;@RG&lt;/code&gt; file (&lt;code&gt;sample_RG_file&lt;/code&gt;) with the split &lt;code&gt;.fastqs&lt;/code&gt;?&lt;/li&gt;&#xA;&lt;li&gt;Is there a better way?&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Bonus question&lt;/strong&gt; (may be worth a separate post): does Nextflow not stage files until the actual .bash script starts? I tried to operate on the &lt;code&gt;sample_RG_file&lt;/code&gt; within the &lt;code&gt;script&lt;/code&gt; using &lt;code&gt;Groovy&lt;/code&gt;, but got an error that the file didn't exist. (The code was working before I found I needed to synchronize &lt;code&gt;sample_RG_file&lt;/code&gt; with the&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Other variations:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The code was 'working' before I found I needed to synchronize &lt;code&gt;sample_RG_file&lt;/code&gt; with the split &lt;code&gt;.fastqs&lt;/code&gt;. I was originally passing &lt;code&gt;GENERATE_FASTQS_PROC.out.sample_RG&lt;/code&gt; as a separate parameter to &lt;code&gt;ALIGN_FASTQ_BWA_PROC&lt;/code&gt;, but it was always passing the same (first) file.&lt;/li&gt;&#xA;&lt;li&gt;I also tried &lt;code&gt;GENERATE_FASTQS_PROC.out.sample_RG.flatten()&lt;/code&gt; in the &lt;code&gt;.map&lt;/code&gt; operator, but that had the same error.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Current error:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The code below fails with the following error:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error executing process &amp;gt; 'REALIGN_SAMPLES_WF:ALIGN_FASTQ_BWA_PROC (1)'&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Caused by:&#xA;Not a valid path value type: groovyx.gpars.dataflow.DataflowBroadcast (DataflowBroadcast around DataflowStream[?])&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;h2&gt;Semi-pseudo code&lt;/h2&gt;&#xA;&lt;p&gt;&lt;strong&gt;Workflow&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow my_WF {&#xA;    take:&#xA;        sample_input_file_ch&#xA;        original_ref&#xA;&#xA;    main:&#xA;        GENERATE_FASTQS_PROC(sample_input_file_ch, original_ref)&#xA;        &#xA;        reads_per_job = 1_000_000&#xA;        fq_split_reads_and_RG_ch = GENERATE_FASTQS_PROC.out.fastqs&#xA;                        .splitFastq(by: reads_per_job, pe: true, file: true)&#xA;                        .map {&#xA;                            fq1, fq2 -&amp;gt;&#xA;                                tuple( fq1, fq2, GENERATE_FASTQS_PROC.out.sample_RG )&#xA;                        }&#xA;                        .view()&#xA;&#xA;        ALIGN_FASTQ_BWA_PROC(fq_split_reads_and_RG_ch, ...)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;GENERATE_FASTQS_PROC&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process GENERATE_FASTQS_PROC {&#xA;&#xA;    input:&#xA;        path(sample_input_file)&#xA;        val(original_ref)&#xA;&#xA;    output:&#xA;        tuple path(&amp;quot;**/*_R1.fastq&amp;quot;), path(&amp;quot;**/*_R2.fastq&amp;quot;), emit: fastqs&#xA;        path(&amp;quot;**/*_RG.txt&amp;quot;), emit: sample_RG&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    generate_fastqs.sh &lt;span class=&quot;math-container&quot;&gt;$sample_input_file $&lt;/span&gt;original_ref &lt;span class=&quot;math-container&quot;&gt;$task.cpus $&lt;/span&gt;mem_per_thread&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;ALIGN_FASTQ_BWA_PROC&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process ALIGN_FASTQ_BWA_PROC {&#xA;&#xA;    /*&#xA;     * Publish results. 'mode: copy' will copy the files into the publishDir&#xA;     * rather than only making links.&#xA;     */&#xA;    // publishDir(&amp;quot;${params.results_dir}/01-REALIGN_SAMPLES&amp;quot;, mode: 'copy')&#xA;&#xA;    label 'REALIGN_SAMPLES'&#xA;&#xA;    input:&#xA;        tuple path(fq1), path(fq2), path(sample_RG_file)&#xA;        val(align_to_ref)&#xA;        val(align_to_ref_tag)&#xA;        val(original_ref)&#xA;        val(output_format)&#xA;&#xA;&#xA;    output:&#xA;        path '*.{bam,cram}', emit: final_alignment&#xA;&#xA;    script:&#xA;&#xA;    /*&#xA;     * Tried to read the RG file using Groovy, but&#xA;     * got error that the file didn't exist.&#xA;     *&#xA;     * Does Nextflow not stage files until the actual .bash script starts?&#xA;     */&#xA;//    println &amp;quot;sample RG file: $sample_RG_file&amp;quot;&#xA;//    file_lines = sample_RG_file.readLines()&#xA;//    sample_name = file_lines[0]&#xA;//    sample_RG = file_lines[1]&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    lines=()&#xA;    while read line&#xA;    do&#xA;        lines+=(&amp;quot;\\\$line&amp;quot;)&#xA;    done &amp;lt; $sample_RG_file&#xA;&#xA;    sample_name=&amp;quot;\\\${lines[0]}&amp;quot;&#xA;    sample_RG=&amp;quot;\\\${lines[1]}&amp;quot;&#xA;&#xA;    bash realign_bwa-split.sh &lt;span class=&quot;math-container&quot;&gt;$fq1 $&lt;/span&gt;fq2 \\\$sample_name \&amp;quot;\\\$sample_RG\&amp;quot; &lt;span class=&quot;math-container&quot;&gt;${align_to_ref} $&lt;/span&gt;{output_format} $task.cpus&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h1&gt;Update&lt;/h1&gt;&#xA;&lt;p&gt;Per @Steve's suggestion, I'm trying to have &lt;code&gt;GENERATE_FASTQS_PROC&lt;/code&gt; create the split &lt;code&gt;.fastq&lt;/code&gt; files using &lt;code&gt;Unix&lt;/code&gt; &lt;code&gt;split&lt;/code&gt;, but I'm still struggling with various concepts.&lt;/p&gt;&#xA;&lt;p&gt;Supposed I have this emitted from my process:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[sample_A_RG.txt, [s1_R1.split_00.fq, s1_R2.split_00.fq, ..., s1_R1.split_XX.fq, s1_R2.split_XX.fq]]`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In my mind, I want to pull out &lt;code&gt;s1_RG.txt&lt;/code&gt; file from the &lt;code&gt;Channel&lt;/code&gt; and extract the &lt;code&gt;@RG&lt;/code&gt;. Then I want to call:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[&amp;lt;all_fastqs_for_sample_A&amp;gt;]&#xA;    .map{ &#xA;        // pull out split ID numbers &#xA;    }&#xA;    .groupTuple()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I believe this would group &lt;code&gt;_R1&lt;/code&gt; and &lt;code&gt;_R2&lt;/code&gt; for each split set, and then call&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[[00.fq1, 00.fq2], ..., [XX.fq1, XX.fq2]]&#xA;    .combine(sample_RG_var)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to get &lt;code&gt;[[@RG, 00.fq1, 00.fq2], ..., [@RG, XX.fq1, XX.fq2]]&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Seems like it would be so much easier if I could use &lt;code&gt;Channel&lt;/code&gt; factories and operators (e.g., &lt;code&gt;.fromFilePairs().combine( rg )&lt;/code&gt;) in the &lt;code&gt;output:&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Getting files into a &lt;code&gt;process&lt;/code&gt; is relatively straightforward, but getting them out in a reasonable manner is less so.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="136" LastEditorUserId="136" LastEditDate="2022-02-12T05:24:12.830" LastActivityDate="2022-02-16T15:02:17.650" Title="Nextflow (DSL v2): how to best synchronize multiple outputs from a single process" Tags="|nextflow|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18550" PostTypeId="2" ParentId="18539" CreationDate="2022-02-11T05:00:00.417" Score="1" Body="&lt;p&gt;As part of its operations, &lt;code&gt;samtools sort&lt;/code&gt; will split the input up into multiple temporary BAM files (which are each individually sorted) if the reads can't fit entirely within memory. Those files are merged for the final output. From the manpage:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The  sorted output is written to standard output by default, or to the specified file (out.bam) when -o is used. This command will also create temporary files tmpprefix.%d.bam as needed when the entire alignment data cannot fit into  memory (as controlled via the -m option).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Given this, I would expect that a concatenated, unsorted BAM will take more time to process than merging sorted BAM files, because in addition to creating the concatenated BAM file, the splitting would need to be done again (assuming the reads don't fit in memory) prior to the final merge carried out by &lt;code&gt;samtools sort&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I appreciate that this is not a quantified answer; it's just my guess based on what I have observed to happen when doing my own analyses.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-02-11T05:00:00.417" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18551" PostTypeId="1" AcceptedAnswerId="18590" CreationDate="2022-02-11T09:07:36.857" Score="0" ViewCount="276" Body="&lt;p&gt;The 23andMe version 3 data (v3) is now quite 'old' and new data is being collected on the updated v5 Illumina SNP-Chip (&lt;a href=&quot;https://www.mygenefood.com/blog/understanding-23andme-chip-versions-why-v3-v4-v5-and-others-can-differ/#how_do_the_various_chips_differ&quot; rel=&quot;nofollow noreferrer&quot;&gt;details here&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Is there an online service for imputing 23andMe v3 files to v4 and v5?&lt;/p&gt;&#xA;&lt;p&gt;Seems this would be quite handy!&lt;/p&gt;&#xA;" OwnerUserId="8875" LastEditorUserId="8875" LastEditDate="2022-02-14T10:50:56.677" LastActivityDate="2022-02-15T10:54:28.157" Title="Imputing 23andMe 3v data to v4 and v5?" Tags="|vcf|imputation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18552" PostTypeId="1" AcceptedAnswerId="18633" CreationDate="2022-02-11T12:17:34.103" Score="2" ViewCount="1692" Body="&lt;p&gt;I am now trying to find intersect genes of two files: one is .bigbed file (very large, 117GB), and the other is a .bed file.&lt;/p&gt;&#xA;&lt;p&gt;At first, I tried to convert bigbed file to bed file and then applied bedtools. I didn't run the bigbedtobed program on the server because I am not allowed to install the bigbedtobed program on our server. Thus I planned to convert the .bb file on my own computer and then upload the converted .bed file to the server, and then apply bedtools. Firstly I tried to convert some smaller .bb data sets (&amp;lt; 10 GB) to .bed and it worked slowly but well on my computer. However, when I tried the 117GB data there was an error:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Invalid argument, Error reading 9188986873 bytes.&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;My code is :&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;bigBedToBed JASPAR2022_hg19.bb hg1 9.bed&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Now the bug is solved, and I got many other good suggestions. Below you can find many methods to work with the bigbed file, directly working with it or converting it to bed file.&lt;/p&gt;&#xA;" OwnerUserId="12391" LastEditorUserId="12391" LastEditDate="2022-02-21T09:47:08.587" LastActivityDate="2022-02-21T09:47:08.587" Title="Methods to work directly with bigbed file?" Tags="|bedtools|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18553" PostTypeId="1" CreationDate="2022-02-11T13:55:46.770" Score="0" ViewCount="90" Body="&lt;p&gt;It is my first time using Chimera. I have uploaded a PDB file on to Chimera and opened it up on the software. I need to change the threshold of the EM map so that I can analyze what the EM map would look like with a threshold too high and too low.&lt;/p&gt;&#xA;&lt;p&gt;Please let me know if you have any suggestions.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="14954" LastActivityDate="2022-02-11T13:55:46.770" Title="Chimera- How to go about changing the threshold of the EM map?" Tags="|proteins|protein-structure|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18554" PostTypeId="2" ParentId="18549" CreationDate="2022-02-11T16:54:55.763" Score="4" Body="&lt;p&gt;Nextflow channels guarantee that items are delivered in the same order as they are sent. So if a process declares two (or more) output channels, the items that are emitted should already be synchronized. This is true unless of course one of the channels sets the &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#optional-output&quot; rel=&quot;nofollow noreferrer&quot;&gt;optional output&lt;/a&gt; attribute.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How do I synchronize the emitted &lt;code&gt;@RG&lt;/code&gt; file (&lt;code&gt;sample_RG_file&lt;/code&gt;) with the&#xA;split &lt;code&gt;.fastqs&lt;/code&gt;?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Since the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#splitfastq&quot; rel=&quot;nofollow noreferrer&quot;&gt;splitFastq&lt;/a&gt; operator &lt;em&gt;can&lt;/em&gt; create &lt;em&gt;n&lt;/em&gt; outputs for each input, you need some way to &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#combine&quot; rel=&quot;nofollow noreferrer&quot;&gt;combine&lt;/a&gt; the @RG files using some key. This key, for example, could be the basename of the sample input file. This, I think, would work nicely given the input file should just have a '.bam' extension. Another option, would be to supply some other value in the input declaration and just use that in your output declaration. If we went with the former, your output declaration might look like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;output:&#xA;tuple val(sample_input_file.baseName), path(&amp;quot;sample_R1.fastq&amp;quot;), path(&amp;quot;sample_R2.fastq&amp;quot;), \&#xA;    emit: fastqs&#xA;tuple val(sample_input_file.baseName), path(&amp;quot;sample_RG.txt&amp;quot;), \&#xA;    emit: sample_RG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then, the body of your workflow might look like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow {&#xA;&#xA;    reads = GENERATE_FASTQS_PROC(sample_input_file_ch, original_ref)&#xA;        &#xA;    reads.fastqs&#xA;        .splitFastq( by: params.reads_per_job, pe: true, file: true )&#xA;        .combine( reads.sample_RG, by: 0 )&#xA;        .map { sample, fq1, fq2, rg_txt -&amp;gt; tuple(fq1, fq2, rg_txt) }&#xA;        .set { fq_split_reads_and_RG_ch }&#xA;&#xA;    ...&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You'll notice that I removed your fuzzy glob patterns from your output declaration above. Your workflow definition suggests that the &lt;code&gt;generate_fastqs.sh&lt;/code&gt; script produces a single pair of FASTQ files and a single readgroup file for a given set of inputs. However, as you know, glob patterns can match one or more output files. When declaring output files, it's always best to name these guys explicitly. If you do use a glob pattern and for some reason your process was to produce two files that match a given pattern, you'll instead get back a tuple, which would break the pipeline.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Is there a better way?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Since you need the @RG tags with the split FASTQ files, I don't think there is. However, I generally avoid using the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#splitting-operators&quot; rel=&quot;nofollow noreferrer&quot;&gt;splitting&lt;/a&gt; operators on large files. My preference is always to pass &amp;quot;work&amp;quot; off (including IO) to a separate process. A separate process can also provide increased flexibility if/when requirements change.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Does Nextflow not stage files until the actual .bash script starts? I tried to operate&#xA;on the sample_RG_file within the script using Groovy, but got an error&#xA;that the file didn't exist.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Nextflow evaluates the code in the script definition before returning the script string to be executed. This is why it's possible to implement a &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#conditional-scripts&quot; rel=&quot;nofollow noreferrer&quot;&gt;conditional script&lt;/a&gt;. Basically, you want to avoid all IO operations here. The script string to be executed has not even been decided at this point.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm not sure how you plan to operate per sample. IIUC, &lt;code&gt;[&amp;lt;all_fastqs_for_sample_A&amp;gt;]&lt;/code&gt; is basically a Channel of lists, where each list contains all split FASTQ files for a given sample. Using &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#map&quot; rel=&quot;nofollow noreferrer&quot;&gt;map&lt;/a&gt; here to pull out the split ID numbers followed by the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#grouptuple&quot; rel=&quot;nofollow noreferrer&quot;&gt;groupTuple&lt;/a&gt; operator might not give the expected results. You might also run into the problems with calling &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#grouptuple&quot; rel=&quot;nofollow noreferrer&quot;&gt;groupTuple&lt;/a&gt;, since, without a &lt;code&gt;size&lt;/code&gt; attribute, it will basically wait for all outputs before emitting the grouped result:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;You should always specify the number of expected elements in each&#xA;tuple using the &lt;code&gt;size&lt;/code&gt; attribute to allow the &lt;code&gt;groupTuple&lt;/code&gt; operator to&#xA;stream the collected values as soon as possible. However there are use&#xA;cases in which each tuple has a different size depending on the&#xA;grouping key. In this case use the built-in function &lt;code&gt;groupKey&lt;/code&gt; that&#xA;allows you to create a special grouping key object to which it’s&#xA;possible to associate the group size for a given key.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Part of the problem is that we don't know what the &lt;code&gt;generate_fastqs.sh&lt;/code&gt; script does and how the other scripts work. If you are aligning using BWA MEM and have &lt;a href=&quot;http://www.htslib.org/doc/samtools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools&lt;/a&gt; available in your $PATH, it might even be easier to produce interleaved output (since BWA MEM can read interleaved FASTQs) using the &lt;a href=&quot;http://www.htslib.org/doc/samtools-fasta.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools fastq&lt;/a&gt; command. This would save some work having to group each list of FASTQs into pairs, ensuring they are ordered as expected. This would be reasonably involved.&lt;/p&gt;&#xA;&lt;p&gt;Not sure what BAMs you need to process, or if you're looking for a more generic solution, but I thought I'd throw in an alternative solution you might find works well for you or that you may not have yet considered. Since BAMs often contain multiple readgroups, I find that if I split by readgroup (using &lt;a href=&quot;http://www.htslib.org/doc/samtools-split.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools split&lt;/a&gt;, &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#flatten&quot; rel=&quot;nofollow noreferrer&quot;&gt;flatten&lt;/a&gt; the output Channel, then &lt;a href=&quot;http://www.htslib.org/doc/samtools-collate.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools collate&lt;/a&gt; followed by &lt;a href=&quot;http://www.htslib.org/doc/samtools-fasta.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools fastq&lt;/a&gt;, I can achieve somewhat uniform mapping walltimes (assuming the input BAMs are similar) without needing to chunk the reads and then merge the alignments back together. If your input BAMs are not similar at all, then you will likely need to chunk to generate uniform alignment jobs.&lt;/p&gt;&#xA;&lt;p&gt;Below is an example, that shows how to create a groupKey, followed by the transpose, join/combine and groupTuple pattern. There are some caveats and there is some room for optimization:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;nextflow.enable.dsl=2&#xA;&#xA;params.original_ref = './ref.fasta'&#xA;params.reads_per_job = 10000&#xA;&#xA;&#xA;process samtools_view_header {&#xA;&#xA;    tag { &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_name}:$&lt;/span&gt;{bam.name}&amp;quot; }&#xA;&#xA;    cpus 1&#xA;    memory 1.GB&#xA;&#xA;    input:&#xA;    tuple val(sample_name), path(bam)&#xA;&#xA;    output:&#xA;    tuple val(sample_name), path(&amp;quot;${bam.baseName}.header.txt&amp;quot;)&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    samtools view \\&#xA;        -H \\&#xA;        -o &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${bam.baseName}.header.txt&quot; \\&#xA;        &quot;$&lt;/span&gt;{bam}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process samtools_name_sort {&#xA;&#xA;    tag { &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_name}:$&lt;/span&gt;{bam.name}&amp;quot; }&#xA;&#xA;    cpus 4&#xA;    memory 8.GB&#xA;&#xA;    input:&#xA;    tuple val(sample_name), path(bam)&#xA;&#xA;    output:&#xA;    tuple val(sample_name), path(&amp;quot;${bam.baseName}.nsorted.bam&amp;quot;)&#xA;&#xA;    script:&#xA;    def avail_mem = task.memory ? task.memory.toGiga().intdiv(task.cpus) : 0&#xA;    def mem_per_thread = avail_mem ? &amp;quot;-m ${avail_mem}G&amp;quot; : ''&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    samtools sort \\&#xA;        -@ &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${task.cpus - 1}&quot; \\&#xA;        $&lt;/span&gt;{mem_per_thread} \\&#xA;        -n \\&#xA;        -o &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${bam.baseName}.nsorted.bam&quot; \\&#xA;        -T &quot;$&lt;/span&gt;{bam.baseName}.nsorted&amp;quot; \\&#xA;        &amp;quot;${bam}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process samtools_fastq {&#xA;&#xA;    tag { &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_name}:$&lt;/span&gt;{bam.name}&amp;quot; }&#xA;&#xA;    cpus 1&#xA;    memory 2.GB&#xA;&#xA;    input:&#xA;    tuple val(sample_name), path(bam)&#xA;&#xA;    output:&#xA;    tuple val(sample_name), path(&amp;quot;${bam.baseName}.fastq&amp;quot;)&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    samtools fastq \\&#xA;        -O \\&#xA;        -T RG,BC \\&#xA;        -0 /dev/null \\&#xA;        &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${bam}&quot; \\&#xA;        &amp;gt; \\&#xA;        &quot;$&lt;/span&gt;{bam.baseName}.fastq&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process split_fastq {&#xA;&#xA;    tag { &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_name}:$&lt;/span&gt;{fastq.name}&amp;quot; }&#xA;&#xA;    cpus 1&#xA;    memory 1.GB&#xA;&#xA;    input:&#xA;    tuple val(sample_name), path(fastq)&#xA;&#xA;    output:&#xA;    tuple val(sample_name), path(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${fastq.baseName}.$&lt;/span&gt;{/[0-9]/*5}.fastq&amp;quot;)&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    split \\&#xA;        --suffix-length=5 \\&#xA;        --additional-suffix=&amp;quot;.fastq&amp;quot; \\&#xA;        -d \\&#xA;        -l &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${params.reads_per_job*4}&quot; \\&#xA;        &quot;$&lt;/span&gt;{fastq}&amp;quot; \\&#xA;        &amp;quot;${fastq.baseName}.&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process bwa_index {&#xA;&#xA;    tag { fasta.name }&#xA;&#xA;    cpus 1&#xA;    memory 12.GB&#xA;&#xA;    input:&#xA;    path fasta&#xA;&#xA;    output:&#xA;    tuple val(fasta.name), path(&amp;quot;${fasta}.{amb,ann,bwt,pac,sa}&amp;quot;)&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    bwa index &amp;quot;${fasta}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process bwa_mem {&#xA;&#xA;    tag { &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_name}:$&lt;/span&gt;{fastq.name}&amp;quot; }&#xA;&#xA;    cpus 8&#xA;    memory 12.GB&#xA;&#xA;    input:&#xA;    tuple val(sample_name), path(fastq), path(header)&#xA;    tuple val(idxbase), path(bwa_index)&#xA;&#xA;    output:&#xA;    tuple val(sample_name), path(&amp;quot;${fastq.baseName}.bam&amp;quot;)&#xA;&#xA;    script:&#xA;    def task_cpus = task.cpus &amp;gt; 1 ? task.cpus - 1 : task.cpus&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    bwa mem \\&#xA;        -p \\&#xA;        -t &lt;span class=&quot;math-container&quot;&gt;${task_cpus} \\&#xA;        -C \\&#xA;        -H &amp;lt;(grep &quot;^@RG&quot; &quot;$&lt;/span&gt;{header}&amp;quot;) \\&#xA;        &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${idxbase}&quot; \\&#xA;        &quot;$&lt;/span&gt;{fastq}&amp;quot; |&#xA;    samtools view \\&#xA;        -1 \\&#xA;        -o &amp;quot;${fastq.baseName}.bam&amp;quot; \\&#xA;        -&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process samtools_coordinate_sort {&#xA;&#xA;    tag { &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_name}:$&lt;/span&gt;{bam.name}&amp;quot; }&#xA;&#xA;    cpus 4&#xA;    memory 8.GB&#xA;&#xA;    input:&#xA;    tuple val(sample_name), path(bam)&#xA;&#xA;    output:&#xA;    tuple val(sample_name), path(&amp;quot;${bam.baseName}.csorted.bam&amp;quot;)&#xA;&#xA;    script:&#xA;    def avail_mem = task.memory ? task.memory.toGiga().intdiv(task.cpus) : 0&#xA;    def mem_per_thread = avail_mem ? &amp;quot;-m ${avail_mem}G&amp;quot; : ''&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    samtools sort \\&#xA;        -@ &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${task.cpus - 1}&quot; \\&#xA;        $&lt;/span&gt;{mem_per_thread} \\&#xA;        -o &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${bam.baseName}.csorted.bam&quot; \\&#xA;        -T &quot;$&lt;/span&gt;{bam.baseName}.csorted&amp;quot; \\&#xA;        &amp;quot;${bam}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process samtools_merge {&#xA;&#xA;    tag { sample_name }&#xA;&#xA;    cpus 1&#xA;    memory 4.GB&#xA;&#xA;    input:&#xA;    tuple val(sample_name), path(bams)&#xA;&#xA;    output:&#xA;    tuple val(sample_name), path(&amp;quot;${sample_name}.bam{,.bai}&amp;quot;)&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    samtools merge \\&#xA;        -c \\&#xA;        -p \\&#xA;        &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_name}.bam&quot; \\&#xA;        $&lt;/span&gt;{bams}&#xA;    samtools index \\&#xA;        &amp;quot;${sample_name}.bam&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;workflow {&#xA;&#xA;    original_ref = file( params.original_ref )&#xA;    bwa_index( original_ref )&#xA;&#xA;    Channel.fromPath('./data/*.bam')&#xA;        | map { tuple( it.baseName, it ) }&#xA;        | set { sample_bam_files }&#xA;&#xA;    samtools_view_header( sample_bam_files )&#xA;&#xA;    sample_bam_files&#xA;        | samtools_name_sort&#xA;        | samtools_fastq&#xA;        | split_fastq&#xA;        | map { sample_name, fastq_files -&amp;gt;&#xA;            tuple( groupKey(sample_name, fastq_files.size()), fastq_files )&#xA;        }&#xA;        | transpose()&#xA;        | combine( samtools_view_header.out, by: 0 )&#xA;        | set { realignment_inputs }&#xA;&#xA;    bwa_mem( realignment_inputs, bwa_index.out )&#xA;        | samtools_coordinate_sort&#xA;        | groupTuple()&#xA;        | map { sample_name, bam_files -&amp;gt;&#xA;            tuple( sample_name.toString(), bam_files )&#xA;        }&#xA;        | samtools_merge&#xA;        | view()&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There are some caveats however, in no particular order:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;It unravels your shell scripts. I think this is a better solution though because it gives you control over each of the components.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;No CRAM support. But this should be easy to add if needed.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A single process to read a BAM header is wasteful, especially if the input BAM is large. More time will be spent localizing the file than actually doing work.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Replace &lt;code&gt;samtools sort -n&lt;/code&gt; with &lt;code&gt;samtools collate&lt;/code&gt;. There's really no need to name sort. You may prefer to pipe the output of &lt;code&gt;samtools collate&lt;/code&gt; into &lt;code&gt;samtools fastq&lt;/code&gt; for avoid writing an extra file. This might be useful if you need to operate on an extremely large number of large BAMs.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The 'samtools_fastq' step should write gzipped output with low compression. This means replacing coreutils &lt;code&gt;split&lt;/code&gt; with something else. A solution using Biopython could be useful here to read and write gzipped output in a single pass.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The last 'map' command in the workflow is unnecessary. I just thought I'd show that calling &lt;code&gt;.toString()&lt;/code&gt; on the GroupKey gives you back an actual String object.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Also, it's not necessary to pass around a tuple of 'sample name' and paths as process inputs and outputs. There's no harm in doing this though. When you are more confident with Nextflow Channels, refactoring these processes should be easy.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;My &lt;code&gt;samtools sort&lt;/code&gt; processes ensure that they do not oversubscribe the nodes that they land on. They use approximately 320% - 380% CPU, depending on the size of the BAM. The resource requests for the other processes are not optimized at all.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;With newer versions of samtools, BAM indexing can be done at the same time as merging, with the &lt;code&gt;--write-index&lt;/code&gt; option.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2022-02-15T06:34:44.010" LastActivityDate="2022-02-15T06:34:44.010" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="18555" PostTypeId="1" CreationDate="2022-02-11T17:18:54.607" Score="3" ViewCount="105" Body="&lt;p&gt;Hi everyone (mostly who are interested in structural bioinformatics).&#xA;I face the problem that I cannot plot the electron density of a specific part of my protein. For the whole protein it is not a problem:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.PDB import *&#xA;import nglview as nv&#xA;import ipywidgets&#xA;&#xA;&#xA;pdbl = PDBList()&#xA;path = pdbl.retrieve_pdb_file('1FAT')&#xA;parser = MMCIFParser()&#xA;struct = parser.get_structure(&amp;quot;1fat&amp;quot;, path)&#xA;view = nv.show_biopython(struct)&#xA;view&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But when I tried to pick up residues from 10 till 109 (&lt;code&gt;res_list = Selection.unfold_entities(struct, &amp;quot;R&amp;quot;)[10:109]&lt;/code&gt;) and plot I can't do that, because for &lt;code&gt;nglview&lt;/code&gt; I need to have a special Object. As I'm a newbie, I'd like to address this question to specialists.&lt;/p&gt;&#xA;&lt;p&gt;At the moment I'm trying to visualise model (structure), not electron density. But the next step is trying to find a package to visualize electron density also. I've just started becoming familiar with some open packages.&lt;/p&gt;&#xA;&lt;p&gt;Thank You in Advance!&lt;/p&gt;&#xA;" OwnerUserId="3829" LastEditorUserId="73" LastEditDate="2022-02-13T21:48:02.853" LastActivityDate="2022-02-13T21:48:02.853" Title="How to visualize electron density of a specific part of the protein residue?" Tags="|protein-structure|3d-structure|structural-biology|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18556" PostTypeId="2" ParentId="18538" CreationDate="2022-02-11T20:02:38.620" Score="8" Body="&lt;p&gt;I usually use &lt;code&gt;samtools sort -@4 -m4g&lt;/code&gt;. Samtools will be faster with more threads and more RAM, but I &lt;em&gt;feel&lt;/em&gt; the gain is not significant. I haven't done a proper evaluation, but I have run samtools sort with different settings from time to time on clusters when I have to consider speed-resource tradeoffs.&lt;/p&gt;&#xA;&lt;p&gt;When you use samtools, make sure you compile it with &lt;a href=&quot;https://github.com/ebiggers/libdeflate&quot; rel=&quot;noreferrer&quot;&gt;libdeflate&lt;/a&gt;, which is about twice as fast as zlib on compression. I don't know if the bioconda version built with libdeflate. You may consider the precompiled binary &lt;a href=&quot;https://zenodo.org/record/5731013&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Samtools sort, especially with libdeflate, is fairly fast these days. Performance metrics in the other answer are outdated.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2022-02-11T20:08:39.273" LastActivityDate="2022-02-11T20:08:39.273" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18558" PostTypeId="2" ParentId="18544" CreationDate="2022-02-11T21:53:40.087" Score="2" Body="&lt;p&gt;To answer your question more formally. I strongly believe you are treating the mice with something that will shift its microbial flora. Clearly the experiment is not described.&lt;/p&gt;&#xA;&lt;p&gt;Thus what I simply feel is you essential have a discrete time series comprising before, during and after treatment. 'Treatment' could be anything drug, vaccine, antibiotic, diet who knows.&lt;/p&gt;&#xA;&lt;p&gt;I think that is a very different time-series to a one where classic correlation statistics will be informative, I'm just guessing in the absence of a hypothesis, thus I could be wrong.&lt;/p&gt;&#xA;&lt;p&gt;I also kinda suspect lactobacillus are strong players.&lt;/p&gt;&#xA;&lt;p&gt;Anyway the stat you are looking at is a special case of the general correlation coefficient in much the same way Spearman's classic statistic is. What I can say is that the statistic is not a specialist approach for large sample sizes, which I believe is your question.&lt;/p&gt;&#xA;&lt;p&gt;'New fangled cool', might be an approach of data mining and that might be okay if you've loads of data you don't know how to analyse, like it goes this way, that way. Ultimately, statistics are used to assess an exact hypothesis. If e.g. lacks power (very common) or the assumptions are violated then look deeper. I do agree that in population biology generating a hypothesis is often helped by data mining, but what I think is you already have an exact hypothesis and I'm far from sure this would nail it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-11T22:10:52.980" LastActivityDate="2022-02-11T22:10:52.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18559" PostTypeId="1" CreationDate="2022-02-11T22:36:47.710" Score="0" ViewCount="64" Body="&lt;p&gt;I read these two questions on duplicated VCF variants: &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/7105/1771&quot;&gt;1&lt;/a&gt;, &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/7126/1771&quot;&gt;2&lt;/a&gt;.&#xA;However, it is not entirely clear to me when duplicated variants can arise (excluding merging artifacts). I want to understand if duplicated variants should be always considered artifacts, or if there are specific use cases where they are needed.&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2022-02-12T01:33:02.693" LastActivityDate="2022-02-12T01:33:02.693" Title="When can we have duplicated variants in a VCF file, excluding artifacts?" Tags="|vcf|variant-calling|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18560" PostTypeId="1" AcceptedAnswerId="18567" CreationDate="2022-02-11T23:51:02.113" Score="2" ViewCount="56" Body="&lt;p&gt;I am initiating a project reproducing MEGA style trees using R to find the best DNA protein models under AIC/BIC. I have read the R AICs and BIC documentation but I'm struggling to implement this.&lt;/p&gt;&#xA;&lt;p&gt;Could I implement AICs and BICs without having to specify the number of sequences in the fasta file?&lt;/p&gt;&#xA;&lt;p&gt;My initial code that works the same as MEGA for aligning and building a phylogenetic tree where I want to view AICs and BIC (as per reference picture):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(seqinr)&#xA;library(adegenet)&#xA;library(ape)&#xA;library(ggtree)&#xA;library(DECIPHER)&#xA;library(Biostrings)&#xA;library(viridis)&#xA;library(ggplot2)&#xA;library(msa)&#xA;&#xA;mySequenceFile &amp;lt;- system.file (&amp;quot;sequences&amp;quot;, &amp;quot;allSequences.fasta&amp;quot;, package =&amp;quot;msa&amp;quot;)&#xA;mySequence &amp;lt;- readDNAStringSet(mySequenceFile)&#xA;&#xA;mySequence&#xA;&#xA;myAlignment &amp;lt;- msa(mySequence, &amp;quot;Muscle&amp;quot;)&#xA;&#xA;myAlignment&#xA;&#xA;alignment &amp;lt;- msaConvert(myAlignment, &amp;quot;seqinr::alignment&amp;quot;)&#xA;distMatrix &amp;lt;- dist.alignment(alignnment, &amp;quot;similarity&amp;quot;)&#xA;clustering &amp;lt;- hclust(distMatrix)&#xA;&#xA;plot(clustering)&#xA;&#xA;dendrogram &amp;lt;- as.dendrogram(clustering)&#xA;phylotree &amp;lt;- as.phylo(clustering)&#xA;&#xA;plot(phylotree, type=&amp;quot;radial&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Please ignore some libraries as I previously ran into some &amp;quot;-&amp;quot; deletion issues when I was writing the code&lt;/p&gt;&#xA;" OwnerUserId="14516" LastEditorUserId="73" LastEditDate="2022-02-13T21:51:48.633" LastActivityDate="2022-02-13T21:53:30.087" Title="Displaying BIC and AICs from MSA nucleotide multisequence fasta file" Tags="|r|fasta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18561" PostTypeId="1" AcceptedAnswerId="18576" CreationDate="2022-02-12T02:54:54.247" Score="1" ViewCount="124" Body="&lt;p&gt;I have two RNA-seq datasets.  One was sequenced at an average read count of 1.5 million per cell the other at 43K average reads per cell.  For the first I also have the meta data from reads alligned and unaliagnable for each cell, but I may have to do some digging for the other dataset.  I am looking for the presence of a few very informative genes being expressed at similar levels in the second dataset to confirmed examples of these cells in the first.  The first dataset is much smaller.&lt;/p&gt;&#xA;&lt;p&gt;My question is this: Are there normalization methods which I could use given the data I have to allow me to accurately say these cells are in the 90th or 80th percentile of a a particular gene being expressed?&lt;/p&gt;&#xA;" OwnerUserId="2482" LastActivityDate="2022-02-13T22:14:35.890" Title="Normalization methods for single cell RNA sequencing that take read count into account" Tags="|rna-seq|normalization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18562" PostTypeId="2" ParentId="18476" CreationDate="2022-02-12T03:52:18.133" Score="3" Body="&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: I've written a short blog post describing my use of nodeSeqs.sh to investigate the functions of sequences at the nodes of short-read assembly graphs for a set of human oral microbiome samples.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://albertvill.com/posts/nodeseqs_oral/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://albertvill.com/posts/nodeseqs_oral/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.merriam-webster.com/dictionary/TL%3BDR&quot; rel=&quot;nofollow noreferrer&quot;&gt;TL;DR&lt;/a&gt; -- transposases and recombinases are enriched, as expected.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Based on &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/18528/3967&quot;&gt;Maximilian's excellent answer&lt;/a&gt;, I have created a script to address my own question.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/acvill/nodeSeqs&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/acvill/nodeSeqs&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here are the details on how it works:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Link lines (beginning with &lt;code&gt;L&lt;/code&gt;) are extracted from the GFA file, and segment occurrences are counted from both the &lt;code&gt;From&lt;/code&gt; and &lt;code&gt;To&lt;/code&gt; fields. To ensure that the final sequences capture the overlapping (linked) portions of segments, links are processed as segment-orientation pairs. Segment IDs with at least the specified minimum degree (&lt;code&gt;-d&lt;/code&gt;) are written to &lt;code&gt;temp_XXXXX/segments.txt&lt;/code&gt;, and the complementary segment lines (beginning with &lt;code&gt;S&lt;/code&gt;) are pulled. &lt;em&gt;k&lt;/em&gt;-mer counts (&lt;code&gt;KC&lt;/code&gt; tag) are used to compute coverage for each segment using the approximation from &lt;a href=&quot;https://dx.doi.org/10.7717%2Fpeerj.2681&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gonnella &amp;amp; Kurtz 2016&lt;/a&gt;; rewritten:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hjwHC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hjwHC.png&quot; alt=&quot;coverage estimation equation&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;em&gt;K&lt;/em&gt;&lt;sub&gt;&lt;em&gt;S&lt;/em&gt;&lt;/sub&gt; is the &lt;em&gt;k&lt;/em&gt;-mer count for a segment, &lt;em&gt;L&lt;/em&gt;&lt;sub&gt;&lt;em&gt;S&lt;/em&gt;&lt;/sub&gt; is the length of a segment, and &lt;em&gt;k&lt;/em&gt; is the &lt;em&gt;k&lt;/em&gt;-mer size used in de Bruijn graph construction (&lt;code&gt;-k&lt;/code&gt;). For (meta)SPAdes, graphs are built iteratively using an increasing &lt;em&gt;k&lt;/em&gt;-mer size, so &lt;em&gt;k&lt;/em&gt; is equal to the largest &lt;em&gt;k&lt;/em&gt;-mer size used. Sequences are extracted corresponding to segments whose coverage is at least the specified minimum coverage (&lt;code&gt;-m&lt;/code&gt;). Sequences longer than the specified context (&lt;code&gt;-c&lt;/code&gt;) are truncated to the context length with respect to the link orientation. Including the &lt;code&gt;-t&lt;/code&gt; flag will automatically remove the temporary folder when the program is finished, which holds intermediate files and &lt;code&gt;log.txt&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The code is indeed &amp;quot;shell stuff&amp;quot;, though hopefully it is not too &amp;quot;kludgey&amp;quot;. ;-)&lt;/p&gt;&#xA;" OwnerUserId="3967" LastEditorUserId="3967" LastEditDate="2022-03-28T18:53:55.063" LastActivityDate="2022-03-28T18:53:55.063" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18563" PostTypeId="1" AcceptedAnswerId="18575" CreationDate="2022-02-12T07:38:34.020" Score="3" ViewCount="39" Body="&lt;p&gt;I want to download a curated dataframe of the full archive of UniProt.&lt;/p&gt;&#xA;&lt;p&gt;Any format as long as it is a table (&lt;code&gt;.csv&lt;/code&gt;, &lt;code&gt;.tsv&lt;/code&gt;, &lt;code&gt;SQL&lt;/code&gt; etc.).&lt;/p&gt;&#xA;&lt;p&gt;Solution 1: Download the .fasta and do the formating myself -&amp;gt; very slow + information incomplete.&lt;/p&gt;&#xA;&lt;p&gt;Solution 2: Use API like &lt;a href=&quot;https://github.com/cokelaer/bioservices&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioServices&lt;/a&gt; -&amp;gt; still very slow but more information.&lt;/p&gt;&#xA;&lt;p&gt;Do we have an already-curated dataframe containing as complete as possible the entry + metadata of UniProt that can be downloaded right away?&lt;/p&gt;&#xA;" OwnerUserId="14169" LastActivityDate="2022-02-13T22:03:09.900" Title="Where can we download a curated dataframe of the full archive of UniProt?" Tags="|uniprot|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18565" PostTypeId="2" ParentId="18552" CreationDate="2022-02-13T00:06:33.913" Score="1" Body="&lt;p&gt;No matter how you approach this problem 110+ GB is big:&lt;/p&gt;&#xA;&lt;p&gt;Your options are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Break up the file and perform the intersection on a chunk by chunk basis&lt;/li&gt;&#xA;&lt;li&gt;Ask your system administrator to install the program&lt;/li&gt;&#xA;&lt;li&gt;Load it into Docker, your systems administrator will understand this. Installing docker is a single line of code (which I forget). Docker is a container which protects the system from the program and no systems administrator in this day and age would refuse a request for Docker installation.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Your.systrm administrator might be concerned about the file sizes involved, but I think a better dialogue with them is the right way forward here.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-02-13T00:06:33.913" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="18566" PostTypeId="2" ParentId="18549" CreationDate="2022-02-13T01:24:35.137" Score="2" Body="&lt;h1&gt;Update 3&lt;/h1&gt;&#xA;&lt;p&gt;While the code in &lt;strong&gt;Update 2&lt;/strong&gt; works, @Steve pointed out that my approach may not be ideal in cloud environments (see comments). He also provided a &lt;code&gt;Nextflow&lt;/code&gt; solution that works. I'm still not a fan of how &lt;code&gt;Nextflow&lt;/code&gt; &lt;code&gt;output:&lt;/code&gt; &lt;code&gt;Channels&lt;/code&gt; work, but that may be from my inexperience.&lt;/p&gt;&#xA;&lt;h1&gt;Update 2&lt;/h1&gt;&#xA;&lt;p&gt;Pretty sure I got it this time. &lt;strong&gt;The following code works&lt;/strong&gt;. I'm getting all of the &lt;code&gt;tuples&lt;/code&gt; I was expecting.&lt;/p&gt;&#xA;&lt;p&gt;Here's the updated code for my &lt;code&gt;workflow&lt;/code&gt; to create the tuples from each line in the &lt;code&gt;sample_tuple_file&lt;/code&gt; I created from a &lt;code&gt;bash&lt;/code&gt; &lt;code&gt;for&lt;/code&gt; loop. I spent quite a while trying to get a combination of &lt;code&gt;.splitText()&lt;/code&gt; and &lt;code&gt;.splitCSV()&lt;/code&gt; to work, but for some reason, you cannot chain as follows: &lt;code&gt;it.splitText().splitCSV()&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;     GENERATE_FASTQS_PROC.out.sample_tuple_file&#xA;         .map {&#xA;            it.splitText()&#xA;         }&#xA;         .flatten()&#xA;         .map {&#xA;             toks = it.trim().split(',')&#xA;             sample_name = toks[0]&#xA;             fq = toks[1]&#xA;             rg = toks[2]&#xA;             tuple(sample_name, fq, rg)&#xA;             }&#xA;         .view()&#xA;        .set { sample_fq_tuples_ch }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h1&gt;Update 1&lt;/h1&gt;&#xA;&lt;p&gt;I thought I had it, but I'm actually only getting ONE of the tuples from my &lt;code&gt;GENERATE_FASTQS_PROC.out.sample_tuple_file.map { }&lt;/code&gt; statement (the last one). And I have no idea why...&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h1&gt;Original posted answer&lt;/h1&gt;&#xA;&lt;p&gt;Despite @Steve's exceptional help and patience with me, I have really battled to figure out how to work with the &lt;code&gt;Nextflow&lt;/code&gt; &lt;code&gt;output:&lt;/code&gt; &lt;code&gt;Channels&lt;/code&gt;, but simply could not figure out any solution using them.&lt;/p&gt;&#xA;&lt;p&gt;I finally figured out a super simple method that is much easier than dealing with &lt;code&gt;Nextflow&lt;/code&gt; &lt;code&gt;Channels&lt;/code&gt;, which I share here in case it will help others. I personally will probably use this method from now on, unless someone can convince me that's a bad idea. In my view, using my approach gives me complete control over the way outputs and inputs are handled, and I don't have to worry whether &lt;code&gt;Nextflow&lt;/code&gt; is doing what I hope it's doing.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;generate_fastqs.sh&lt;/strong&gt;&#xA;&lt;code&gt;generate_fastqs.sh&lt;/code&gt; takes a single &lt;code&gt;.bam&lt;/code&gt; -&amp;gt; sorts by name -&amp;gt; converts to .fastq (now w/ &lt;code&gt;samtools fastq&lt;/code&gt;, thanks to @Steve), and splits them with &lt;code&gt;Unix&lt;/code&gt; &lt;code&gt;split&lt;/code&gt;. Then, at the end of the script, I create a single file with all of the &lt;code&gt;tuples&lt;/code&gt; that I want to pass to my next &lt;code&gt;process&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here are the key lines for these steps (including exit codes to ensure &lt;code&gt;Nextflow&lt;/code&gt; receives error notifications). The last step is the key (for me, anyway):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;&#xA;#####################&#xA;# Sort .bam by name #&#xA;#####################&#xA;if ! time samtools sort -@ &lt;span class=&quot;math-container&quot;&gt;$n_threads -n -m $&lt;/span&gt;{mem_per_thread}G &lt;span class=&quot;math-container&quot;&gt;$sample_input &amp;gt; $&lt;/span&gt;sample_sorted_by_name; then&#xA;    echo &amp;quot;ERROR: sort failed&amp;quot;&#xA;    exit 1&#xA;fi&#xA;&#xA;...&#xA;&#xA;#############################&#xA;# Create interleaved .fastq #&#xA;#############################&#xA;if ! time samtools fastq -o &lt;span class=&quot;math-container&quot;&gt;$fq -s /dev/null -0 /dev/null --reference $&lt;/span&gt;original_ref $sample_sorted_by_name; then&#xA;    echo &amp;quot;ERROR: bamtofastq failed&amp;quot;&#xA;    exit 1&#xA;fi&#xA;&#xA;...&#xA;&#xA;################&#xA;# Split fastqs #&#xA;################&#xA;if ! time split --additional-suffix=&amp;quot;.fastq&amp;quot; -dl &lt;span class=&quot;math-container&quot;&gt;$n_lines $&lt;/span&gt;fq &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${split_dir}/$&lt;/span&gt;{sample_name}.interleaved_R1_R2.split.&amp;quot;; then&#xA;    echo &amp;quot;ERROR: split failed&amp;quot;&#xA;    exit 1&#xA;fi&#xA;&#xA;...&#xA;&#xA;###########################################################&#xA;# Store sample name, .fastq, and RG to file for alignment #&#xA;###########################################################&#xA;RG_file=&amp;quot;${sample_name}.tuples.txt&amp;quot;&#xA;&#xA;# Create empty file&#xA;&amp;gt; &lt;span class=&quot;math-container&quot;&gt;$RG_file&#xA;for fq_file in $&lt;/span&gt;PWD/&lt;span class=&quot;math-container&quot;&gt;${split_dir}/*&#xA;do&#xA;echo &quot;$&lt;/span&gt;{sample_name},&lt;span class=&quot;math-container&quot;&gt;$fq_file,$&lt;/span&gt;RG&amp;quot; &amp;gt;&amp;gt; $RG_file    # &amp;lt;-- Creating future tuples in file, here (comma separated).&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;GENERATE_FASTQS_PROC&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In &lt;code&gt;GENERATE_FASTQS_PROC&lt;/code&gt;, &lt;code&gt;emit&lt;/code&gt; one simple file--the tuple file I created myself with a simple &lt;code&gt;bash&lt;/code&gt; loop (so easy and feels so good):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    emit:&#xA;        path(&amp;quot;*.tuples.txt&amp;quot;), emit: sample_tuple_file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Workflow&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In my &lt;code&gt;workflow&lt;/code&gt;, I call &lt;code&gt;GENERATE_FASTQS_PROC&lt;/code&gt;, access the file from the &lt;code&gt;Channel&lt;/code&gt;, read each line, and create a &lt;code&gt;Nextflow&lt;/code&gt; tuple for each line (again, the lines I so easily created in a &lt;code&gt;bash&lt;/code&gt; &lt;code&gt;for&lt;/code&gt; loop).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    /*&#xA;     * Generate fastq files from the original .(cr|b)am file and split into&#xA;     * mini .fastq files with `n_reads_per_run` in each mini .fastq&#xA;     */&#xA;    GENERATE_FASTQS_PROC(sample_input_file_ch, n_reads_per_run, original_ref)&#xA;&#xA;&#xA;    /*&#xA;     * Get sample name, fastq, and read group (RG) from the tuple file using Nextflow&#xA;     */&#xA;&#xA;     GENERATE_FASTQS_PROC.out.sample_tuple_file&#xA;        .map{&#xA;            sample_tuple_file -&amp;gt;&#xA;                sample_tuple_file.withReader {          // Open file&#xA;                    it.eachLine {                       // Loop over each line in file&#xA;                        line -&amp;gt;&#xA;                            toks = line.split(',')      // Split by comma&#xA;                            sample_name = toks[0]       // Get each item&#xA;                            fq = toks[1]&#xA;                            rg = toks[2]&#xA;&#xA;                            tuple(sample_name, fq, rg)  // Create the simple tuple that has eluded me for two days!&#xA;                    }&#xA;                }&#xA;        }&#xA;        .set { sample_fq_tuples_ch }                    // Assign it to a new Channel name and move along!&#xA;&#xA;&#xA;    /*&#xA;     * Align fastq file pairs&#xA;     */&#xA;&#xA;    ALIGN_FASTQ_BWA_PROC(sample_fq_tuples_ch, align_to_ref,&#xA;                             align_to_ref_tag, original_ref, output_format)&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="136" LastEditorUserId="136" LastEditDate="2022-02-16T15:02:17.650" LastActivityDate="2022-02-16T15:02:17.650" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="18567" PostTypeId="2" ParentId="18560" CreationDate="2022-02-13T02:44:15.323" Score="1" Body="&lt;p&gt;Having looked carefully at the question, I understand the 'gap in the market' but all this code will do is generate a provisional tree. I would suggest looking at protein models first because you need models to perform an AIC/BIC. So far you haven't generated used or described a protein model and without that there's nothing to compare via AIC.&lt;/p&gt;&#xA;&lt;p&gt;To answer the direct question you should seek to avoid removing the sample size from the AIC and base the work on the equation below. If you have large samples sizes n can be removed (which is what you are wanting).&lt;/p&gt;&#xA;&lt;p&gt;The work you should read is jModeltest which is here, &lt;a href=&quot;https://github.com/ddarriba/jmodeltest2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ddarriba/jmodeltest2&lt;/a&gt;.&#xA;This originated from Modeltest by David Pasado.&lt;/p&gt;&#xA;&lt;p&gt;Their code on GitHub will tell you exactly how to implement AIC and BIC. I recommend ditching R in this instance. If done via subprocessing (below) its not hard.&lt;/p&gt;&#xA;&lt;p&gt;Their work concerns nucleotides and never really hit the mainstream, but was used by some.&lt;/p&gt;&#xA;&lt;p&gt;The key criticism is like your approach that the tree generated was based on neighbor-joining. The tree will influence the matrix so there is a danger that the ML or Bayesian phylogeny resulting from the model will be biased by neighbour-joining. Secondly, it was not computationally expensive to do this entirely by ML or Bayesian approaches because it's a 4*4 matrix.&lt;/p&gt;&#xA;&lt;p&gt;Protein phylogeny is very different, obviously the matrix is bigger 20*20 the other issue is that it's a 'static matrix'. There are approaches where they &amp;quot;combine&amp;quot; matrices but I don't know the algorithm. Thus we know exactly how the protein matrices are generated, &amp;quot;combined&amp;quot; needs a clearer explanation and how this is obtained could be a good application for AIC. Thus could provide useful clarity. There is in addition.the potential to generate new matrices and assess them, but the concept of introducing meaningful dynamism into the matrix does have potential and this is absent (except for cryptic combined options). AIC would then assess whether this is helpful. It might not be hugely helpful in traditional static matrices, but would compare them against further hypotheses of mutation and innovative approaches. If used deployed correctly could be cool because it allows you to experiment and ask the question what precisely does this protein matrix mean for my data?&lt;/p&gt;&#xA;&lt;p&gt;More generally stretching the utility of AIC type metrics is very interesting in 'bleeding edge' machine learning (admittedly its a bit futuristic).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I suspect (could be wrong) jModelTest is coded in Java, an alternative would be to subprocess via Python using relevant compiled code. This was how the original Modeltest was written, essentially driven through the leading maximum likelihood package of the time known as PAUP. Python does implement AIC/BIC via the statsmodel library, &lt;code&gt;scikit-learn&lt;/code&gt; (machine learning) or use the equation directly.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VWL7w.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VWL7w.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;k= number of independent variables to build model&#xA;L= maximum likelihood estimate of model&#xA;.. for small sample sizes (n)&lt;/p&gt;&#xA;&lt;p&gt;R is not known for subprocessing and I am not aware of any ML/Bayes libraries for phylogeny.&lt;/p&gt;&#xA;&lt;p&gt;In summary, this is a good project and it is worth investing the time because it has a clear, meaningful output and potential to extend into more innovative experimental approaches. I for one would use established  code (i.e. written by someone else) that implemented AIC/BIC for proteins matrices and would be fairly straight forward (for you) to write. How you maximise its use in deploying it to help understand protein matrices needs creative thinking (remember its parameters verse likelihood).&lt;/p&gt;&#xA;&lt;p&gt;Good luck.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="73" LastEditDate="2022-02-13T21:53:30.087" LastActivityDate="2022-02-13T21:53:30.087" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18569" PostTypeId="2" ParentId="18555" CreationDate="2022-02-13T09:13:40.320" Score="2" Body="&lt;p&gt;NGLview runs off the JavaScript libray NGL, which is documented as a series of very helpful examples: &lt;a href=&quot;http://nglviewer.org/ngl/gallery/&quot; rel=&quot;nofollow noreferrer&quot;&gt;in the gallery&lt;/a&gt;. The commands are not one-to-one (slightly frustratingly), but it is helpful as you can figure it out easily with a few &lt;code&gt;dir&lt;/code&gt; and &lt;code&gt;help&lt;/code&gt; commands. In this case whatever example of density being loaded it will reveals that it's a different component —this is the same everywhere as density files do not contain atomic coordinates. Unfortunately, there's not a direct way to generate a new &lt;code&gt;ComponentViewer&lt;/code&gt; from different sources to an &lt;code&gt;NGLWidget&lt;/code&gt; and you cannot merge two of them: the various &lt;code&gt;.show_*&lt;/code&gt; functions return an &lt;code&gt;NGLWidget&lt;/code&gt; but aren't part of it.&lt;/p&gt;&#xA;&lt;p&gt;First, I am not sure why you are going through a convoluted route for loading a PDB code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pdb_code = '1fat'&#xA;&#xA;import nglview as nv&#xA;&#xA;view: nv.NGLWidget = nv.show_pdbid(pdb_code) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, on loading the component a representation is added to it.&#xA;These can hide the representation you then add and things will not make sense, until this detail is recalled! I fall in that pit nearly every time.&#xA;But we can change it with update command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# I don't like wonky sheets&#xA;view.update_cartoon(smoothSheets=True, color='salmon', component=0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now let's add the density map the circuitous way (no &lt;code&gt;.show_url&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import requests, io&#xA;response: requests.Response = requests.get('https://www.ebi.ac.uk/pdbe/coordinates/files/1fat.ccp4')&#xA;response.raise_for_status()&#xA;fh = io.BytesIO(response.content)&#xA;comp: nv.component.ComponentViewer = view.add_component(fh, ext='ccp4') &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now we can add a surface representation, but first let's clear the default.&#xA;For the arguments I had a gander at the example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;view.clear_representations(component=1)&#xA;view.add_representation('surface', &#xA;                        color='grey', &#xA;                        wireframe=True, &#xA;                        isolevel=1.5,&#xA;                        component=1)&#xA;view.center('100-110:A')&#xA;view&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2022-02-13T09:13:40.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18570" PostTypeId="1" AcceptedAnswerId="18571" CreationDate="2022-02-13T16:23:58.410" Score="2" ViewCount="442" Body="&lt;p&gt;I am trying to generate contrast matrix for different conditions using &lt;strong&gt;for&lt;/strong&gt; loop.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cluster_len &amp;lt;- 3&#xA;req_contrasts &amp;lt;- c()&#xA;&#xA;for (m in 1:cluster_len) {&#xA;  for (n in 1:cluster_len) {&#xA;    if (n &amp;gt; m) {&#xA;      req_contrasts &amp;lt;- c(req_contrasts, (paste0(paste0(&amp;quot;Group&amp;quot;, n), &amp;quot;-&amp;quot;, paste0(&amp;quot;Group&amp;quot;, m))))&#xA;  &#xA;    }&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;req_contrasts returns&#xA;&lt;code&gt;&amp;quot;Group2-Group1&amp;quot; &amp;quot;Group3-Group1&amp;quot; &amp;quot;Group3-Group2&amp;quot;&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The contrast that needs to be passed is&#xA;&lt;code&gt;&amp;quot;Group2-Group1&amp;quot;,&amp;quot;Group3-Group1&amp;quot;,&amp;quot;Group3-Group2&amp;quot;&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have tried the following code to combine each string and form a single vector&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# To separate each condition by a double quote and comma&#xA;temp_1 &amp;lt;- paste0(req_contrasts, collapse='&amp;quot;,&amp;quot;', sep=&amp;quot;&amp;quot;)&#xA;temp_1&#xA;[1] &amp;quot;Group2-Group1\&amp;quot;,\&amp;quot;Group3-Group1\&amp;quot;,\&amp;quot;Group3-Group2&amp;quot;&#xA;# To remove the escape character&#xA;temp_2 &amp;lt;- gsub(&amp;quot;\\&amp;quot;, &amp;quot;&amp;quot;, temp_1, fixed = TRUE)&#xA;temp_2&#xA;[1] &amp;quot;Group2-Group1\&amp;quot;,\&amp;quot;Group3-Group1\&amp;quot;,\&amp;quot;Group3-Group2&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This does not work&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;contrast.matrix &amp;lt;- makeContrasts(temp_2, levels=design_mat)&#xA;Error in parse(text = ej) : &amp;lt;text&amp;gt;:1:14: unexpected string constant&#xA;1: Group2-Group1&amp;quot;,&amp;quot;&#xA;^&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Limma throws the following error in presence of escape character. I am assuming it is because it considers that the string is yet to end.&lt;/p&gt;&#xA;&lt;p&gt;Following input works&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;contrast.matrix_2 &amp;lt;- makeContrasts(&amp;quot;Group2-Group1&amp;quot;, &amp;quot;Group3-Group1&amp;quot;, &amp;quot;Group3-Group2&amp;quot;,                 levels=design_mat)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In my analysis the number of clusters to be analysed is 16. Hence, the contrast conditions cannot be generated manually.&lt;/p&gt;&#xA;" OwnerUserId="2247" LastActivityDate="2022-03-12T18:27:29.667" Title="Generating contrast matrix for limma in loop" Tags="|limma|loop|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18571" PostTypeId="2" ParentId="18570" CreationDate="2022-02-13T18:19:45.197" Score="1" Body="&lt;p&gt;I wrote this snippet a while back to generate all pairwise contrasts:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&#xA;make_all_contrasts &amp;lt;- function (group, delim=&amp;quot;_vs_&amp;quot;){&#xA;  &#xA;  suppressMessages(require(limma))&#xA;  &#xA;  #/ ensure that group levels are unique&#xA;  group &amp;lt;- sort(unique(as.character(group)))&#xA;  &#xA;  #/ make all combinations&#xA;  cb   &amp;lt;- combn(group, 2, FUN = function(x){paste0(x[1], &amp;quot;-&amp;quot;, x[2])})&#xA;&#xA;  #/ make contrasts&#xA;  contrasts&amp;lt;- limma::makeContrasts(contrasts=cb, levels=group)&#xA;  colnames(contrasts) &amp;lt;- gsub(&amp;quot;-&amp;quot;, delim, colnames(contrasts))&#xA;  &#xA;  return(contrasts)&#xA;}&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Assuming three contrasts this will create a limma-like contrast matrix with all possible pairs:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;make_all_contrasts(group=c(&amp;quot;groupA&amp;quot;, &amp;quot;groupB&amp;quot;, &amp;quot;groupC&amp;quot;), delim=&amp;quot;_vs_&amp;quot;)&#xA;&#xA;        Contrasts&#xA;Levels   groupA_vs_groupB groupA_vs_groupC groupB_vs_groupC&#xA;  groupA                1                1                0&#xA;  groupB               -1                0                1&#xA;  groupC                0               -1               -1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2022-03-12T18:27:29.667" LastActivityDate="2022-03-12T18:27:29.667" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18572" PostTypeId="1" CreationDate="2022-02-13T19:54:12.017" Score="2" ViewCount="245" Body="&lt;p&gt;I have a bulk RNA-seq dataset made up of control and treatment conditions for a range of cell lines. This dataset was generated in two batches, such that the cell lines are split between batches but both the treatment and control for each cell line are within the same batch. As the cell lines are very different, I'm looking to do DE analysis for each cell line and then compare these DE responses between cell lines.&lt;/p&gt;&#xA;&lt;p&gt;I'm wondering whether batch effect correction is useful here since I'm only ever comparing DE values and not the raw counts themselves. Also, the baseline expression values for the cells lines are so different that they already constitute a strong biological batch effect that is perfectly correlated with the technical batch effect, which batch correction wouldn't be able to account for.&lt;/p&gt;&#xA;" OwnerUserId="15051" LastActivityDate="2022-07-14T00:29:43.757" Title="Comparing differential expression across samples - is batch effect correction needed?" Tags="|rna-seq|statistics|differential-expression|batch-effects|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18573" PostTypeId="2" ParentId="18572" CreationDate="2022-02-13T21:38:40.610" Score="0" Body="&lt;p&gt;If your treatment and control are prepared in different batches, it's impossible to distinguish between treatment-specific effects, and batch-specific effects. DESeq2 will complain when you attempt to carry out such correction in the statistical model for this reason.&lt;/p&gt;&#xA;&lt;p&gt;Baseline differences in expression are a large reason for carrying out batch correction in the first place; it makes differential expression analyses &lt;em&gt;more&lt;/em&gt; powerful, not less. I'd recommend consulting with a statistician (prior to carrying out an experiment / sequencing) regarding the best way to split samples into different batches so that batch effects can be discovered and accounted for. As you mention, there are multiple effects that should be considered (e.g. cell lines, sequencing run).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-02-13T21:38:40.610" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18574" PostTypeId="2" ParentId="17799" CreationDate="2022-02-13T21:40:35.603" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @supertech, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;This can be frustrating I understand. In my experience it's easier to read the paper and install the modern versions of the tools compatible with your computer (paper was published in 2014). Even more convenient you could open a free account at &lt;a href=&quot;https://galaxyproject.org/support/account-quotas/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://galaxyproject.org/support/account-quotas/&lt;/a&gt; (gives 250GB free space), and do your analysis there. Tophat aligner can be very slow. STAR is better from mapping accuracy to speed; I would recommend that you use that if possible. If you are doing  analysis with large genomes your laptop computer is unlikely to have necessary RAM to run the aligners.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Answer from @m, converted from comment:*&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Why not then simply use Bioconductor microRNA, which is under active/recent development?&lt;/p&gt;&#xA;&lt;p&gt;Looks like STAR has been used for microRNA mapping for ENCODE:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.encodeproject.org/microrna/microrna-seq/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.encodeproject.org/microrna/microrna-seq/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;From the ENCODE pipeline developer (@user3274289):&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;The ENCODE microRNA pipeline uses STAR for mapping and quantification and should be runnable on mac (you might need one on the beefier side though to have enough memory but i have run some workflows on a new macbook pro). We do not have drosophila STAR index available so you need to generate that using the workflow that is provided. See instructions on github.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-03-17T20:06:15.763" LastActivityDate="2022-03-17T20:06:15.763" CommentCount="0" CommunityOwnedDate="2022-02-13T21:40:35.603" ContentLicense="CC BY-SA 4.0" />
  <row Id="18575" PostTypeId="2" ParentId="18563" CreationDate="2022-02-13T22:03:09.900" Score="2" Body="&lt;p&gt;&lt;em&gt;Answer from @matteo-ferla, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;There are many tables in the downloads page such tables enumerating each protein in a species proteome with basic stats, but not all the data. The uniprot records are very structured so would convert poorly to a table (even filling cells with lists/arrays and dictionaries/objects)… so a fuller table would be a mess.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-02-13T22:03:09.900" CommentCount="0" CommunityOwnedDate="2022-02-13T22:03:09.900" ContentLicense="CC BY-SA 4.0" />
  <row Id="18576" PostTypeId="2" ParentId="18561" CreationDate="2022-02-13T22:14:35.890" Score="4" Body="&lt;p&gt;The default normalisation methods of DESeq2 have a quantile-like normalisation process (based on &lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#why-un-normalized-counts&quot; rel=&quot;nofollow noreferrer&quot;&gt;raw read counts&lt;/a&gt;), and this default process should be able to handle datasets with different read depths.&lt;/p&gt;&#xA;&lt;p&gt;If that's not good enough, DESeq2 allows you to provide alternative information (e.g. cell counts) to bypass the usual normalisation process using the &lt;code&gt;sizeFactors&lt;/code&gt; function:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The sizeFactors vector assigns to each column of the count matrix a value, the size factor, such that count values in the columns can be brought to a common scale by dividing by the corresponding size factor (as performed by counts(dds, normalized=TRUE)). See DESeq for a description of the use of size factors. If gene-specific normalization is desired for each sample, use normalizationFactors.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;More details here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.rdocumentation.org/packages/DESeq2/versions/1.12.3/topics/sizeFactors&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rdocumentation.org/packages/DESeq2/versions/1.12.3/topics/sizeFactors&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-02-13T22:14:35.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18577" PostTypeId="1" AcceptedAnswerId="18588" CreationDate="2022-02-14T10:53:01.690" Score="2" ViewCount="343" Body="&lt;p&gt;I've been working with a bacterial genome assembly. The initial stats look pretty good (3 main pieces, depth ~50x [20x - 100x] if I calculated it right). I'd expect this to be a virtually complete genome apart from some low-complexity regions.&lt;/p&gt;&#xA;&lt;p&gt;However, when I use CheckM to estimate completeness / redundancy, I get ~99% completion, &amp;lt;2% redundancy. (out of 579 expected single-copy marker genes, 563 were present once, and 11 were present twice). I am especially concerned about the missing genes.&lt;/p&gt;&#xA;&lt;p&gt;I can think of several possibilities:&#xA;a) Limits of CheckM. This software uses a sample of 82 genomes to select marker genes for this phylum: however, there are no available genomes in the same genus as the one I'm working with. It might be possible that this is natural variation.&lt;/p&gt;&#xA;&lt;p&gt;b) The &amp;quot;single-copy&amp;quot; genes present twice might be due to misassembly. But while the sample was not axenic, the organism of interest was majoritary, and long reads were used as well, so this shouldn't be an issue?&lt;/p&gt;&#xA;&lt;p&gt;c) The genes were somehow missed by sequencing. (but the depth is pretty good?)&lt;/p&gt;&#xA;&lt;p&gt;I don't know how likely a) is, so it's difficult for me to assess whether what I have is a normal result, or something that requires additional sequencing in order to make sense. I would appreciate your thoughts!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2022-02-15T15:54:57.210" Title="Is this a complete bacterial genome? (CheckM)" Tags="|genome|quality-control|genome-sequencing|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18578" PostTypeId="1" CreationDate="2022-02-14T11:48:11.423" Score="1" ViewCount="488" Body="&lt;p&gt;I want to find and compare the results from STAR and BWA- MEM mapped. I have 150bp paired end reads in sorted.bam files in each case and i want to find in each case uniquely mapped reads, number of reads mapped to multiple loci and number of unmappped reads.&lt;/p&gt;&#xA;&lt;p&gt;Especially for uniquely mapped reads i tried different solutions from biostars and this &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/508/obtaining-uniquely-mapped-reads-from-bwa-mem-alignment&quot;&gt;Obtaining uniquely mapped reads from BWA mem alignment&lt;/a&gt; but i dont know what to trust,&lt;/p&gt;&#xA;&lt;p&gt;Could you please suggest the best way to find mapping results from bam paired end files ?&lt;/p&gt;&#xA;" OwnerUserId="14902" LastActivityDate="2022-02-20T03:14:18.773" Title="Compare mapped reads from BWA -MEM and STAR from .bam files" Tags="|linux|bwa|star|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18579" PostTypeId="1" AcceptedAnswerId="18587" CreationDate="2022-02-14T11:49:25.580" Score="3" ViewCount="62" Body="&lt;p&gt;I have about 1,000 SNP-Chip data (samples) that I'd like to impute over (for the purpose of having more rsIDs to match against GWAS data).&lt;/p&gt;&#xA;&lt;p&gt;However, I don't know the ancestry of each sample / the ancestry hasn't been recorded in a reliable way.&lt;/p&gt;&#xA;&lt;p&gt;Is there a 'quick and dirty' method to decide which imputation panel to use based on the genotype data itself? e.g.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;sample 1 'looks' British in England and Scotland (GBR),&lt;/li&gt;&#xA;&lt;li&gt;sample 2 'looks' Colombian in Medellin, Colombia (CLM),&lt;/li&gt;&#xA;&lt;li&gt;sample 3 'looks' Bengali in Bangladesh (BEB), etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Once I know which group each sample matches most closely I can then perform imputation with the appropriate imputation panel.&lt;/p&gt;&#xA;" OwnerUserId="8875" LastEditorUserId="3694" LastEditDate="2022-02-14T12:34:16.143" LastActivityDate="2022-02-14T21:58:33.140" Title="Chosing an imputation panel for SNP-Chip data?" Tags="|imputation|1000genomes|snp-chip|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18580" PostTypeId="1" CreationDate="2022-02-14T11:57:41.623" Score="2" ViewCount="1219" Body="&lt;p&gt;Primer-BLAST is a tool for designing target-specific primers, available through the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/tools/primer-blast/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Primer-BLAST web interface&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Is it possible to run the tool locally instead of accessing it via the internet?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Primer-BLAST obviously uses NCBI Blast which we all know about, but it is a pipeline running through &lt;a href=&quot;https://primer3.ut.ee&quot; rel=&quot;nofollow noreferrer&quot;&gt;Primer3&lt;/a&gt;. This is an extremely popular and reliable primer design software, but only does single stranded DNA/RNA. There is a prettier version, Primer3plus &lt;a href=&quot;https://www.bioinformatics.nl/cgi-bin/primer3plus/primer3plus.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15054" LastEditorUserId="3848" LastEditDate="2022-03-16T15:38:44.853" LastActivityDate="2024-01-07T20:07:30.830" Title="Can Primer-BLAST be run locally?" Tags="|primer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18581" PostTypeId="2" ParentId="18580" CreationDate="2022-02-14T14:25:49.050" Score="0" Body="&lt;p&gt;To strictly answer the question its not possible because it is a pipeline based on Primer3 which is not written by NCBI, therefore they would not be in a position to distribute it. Nor would Primer3 be able to distribute Blast because that is NCBI. It is simply a web-server.&lt;/p&gt;&#xA;&lt;p&gt;NCBI and Primer3 would need to come to a joint agreement, which they must have done to present the webserver (at least it would be polite). You can of course replicate the pipeline yourself.&lt;/p&gt;&#xA;&lt;p&gt;Brief details are here,&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/tools/primer-blast/primerinfo.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/tools/primer-blast/primerinfo.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;It can be done via Docker. This exact pipeline hasn't been distributed via Docker but Blast is available as a Docker image.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;New Answer&lt;/strong&gt;&#xA;Just to follow this up, &lt;a href=&quot;https://hub.docker.com/r/csicunam/primers4clades&quot; rel=&quot;nofollow noreferrer&quot;&gt;primers4clades&lt;/a&gt; is a Docker image aimed at very similar objectives to Primer-Blast. I have not used it, but its worth considering if standalone is essential, e.g. for ease of building a pipeline.&lt;/p&gt;&#xA;&lt;p&gt;Here's the blurb&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Primers4clades provides a fully automated pipeline for the design of PCR primers for cross-species amplification of novel sequences from metagenomic DNA or from uncharacterized organisms belonging to user-specified phylogenetic lineages. It implements an extended CODEHOP strategy based on both DNA and protein multiple alignments of coding genes and evaluates thermodynamic properties of the oligonucleotide pairs, as well as the phylogenetic information content of predicted amplicons, computed from the branch support values of maximum likelihood phylogenies.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't know the user, but Docker is a safe running environment that is isolated from the system. There's no harm in checking.&lt;/p&gt;&#xA;&lt;p&gt;Finally both Blast and Primer3 are available via &lt;code&gt;conda&lt;/code&gt;, but not Primer-Blast. In theory, I believe Primer-Blast could be distributed via &lt;code&gt;conda&lt;/code&gt; under the rules of &lt;code&gt;conda&lt;/code&gt;. In fact, I think, but could be wrong, that anyone could rebuild this pipeline and post it back to &lt;code&gt;conda&lt;/code&gt;. @merv will know exactly whether this is 'permitable' and is very likely to correct me if I'm wrong.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-08-15T16:15:06.450" LastActivityDate="2022-08-15T16:15:06.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18582" PostTypeId="2" ParentId="18579" CreationDate="2022-02-14T14:34:17.960" Score="1" Body="&lt;p&gt;If I were you, I would use &lt;a href=&quot;https://vcru.wisc.edu/simonlab/bioinformatics/programs/admixture/admixture-manual.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;ADMIXTURE&lt;/a&gt; in supervised mode &lt;code&gt;--supervised&lt;/code&gt; and use some of the 1000 genomes populations as reference populations. This is a fast and accurate way to obtain broad-scale ancestry proportion estimates for each of your individuals.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2022-02-14T14:34:17.960" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18585" PostTypeId="1" AcceptedAnswerId="18586" CreationDate="2022-02-14T20:18:45.263" Score="0" ViewCount="414" Body="&lt;p&gt;I tried to install &lt;code&gt;vcf2maf&lt;/code&gt; following this tutorial from GitHub: &lt;a href=&quot;https://github.com/mskcc/vcf2maf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mskcc/vcf2maf&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I get a repeat error when I try to test out vcf2maf using the code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl vcf2maf.pl --input-vcf tests/test.vcf --output-maf tests/test.vep.maf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The error message looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;'''STATUS: Running VEP and writing to: tests/test.vep.vcf&#xA;ERROR: Cannot find VEP script under: /Users/user/miniconda3/bin'''&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Basically I followed every steps provided on the tutorial.&#xA;Does anyone could help me find out what is going wrong?&lt;/p&gt;&#xA;&lt;p&gt;I am working on macOS Monterey 12.2.1 with an Intel Core i5 processor.&lt;/p&gt;&#xA;" OwnerUserId="14102" LastEditorUserId="3848" LastEditDate="2022-02-14T20:49:09.290" LastActivityDate="2022-06-09T02:06:30.207" Title="Error in installing vcf2maf" Tags="|vcf|maf|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18586" PostTypeId="2" ParentId="18585" CreationDate="2022-02-14T20:31:07.340" Score="1" Body="&lt;p&gt;Install it via anaconda/conda either,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c bioconda vcf2maf&#xA;conda install -c bioconda/label/cf201901 vcf2maf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would be surprised if that installation fails. A possible issue is Perl version and system version.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-02-14T20:31:07.340" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18587" PostTypeId="2" ParentId="18579" CreationDate="2022-02-14T21:58:33.140" Score="1" Body="&lt;p&gt;If you're using an hmm-based method such as for example impute2 or later, then the imputation method will perform the haplotype matching for you. No need to cut down the reference panel. Try to get a big reference panel such as 1000 genomes phase 3 or the haplotype reference consortium (disclaimer: I am an author on both those papers).&lt;/p&gt;&#xA;&lt;p&gt;But do be careful about relative power: a sample with haplotypes that are better represented in the ref panel than another sample will have better-imputed genotypes than the other sample.&lt;/p&gt;&#xA;" OwnerUserId="492" LastActivityDate="2022-02-14T21:58:33.140" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18588" PostTypeId="2" ParentId="18577" CreationDate="2022-02-14T23:14:11.070" Score="2" Body="&lt;p&gt;See this table from the CheckM table to put this number in context. In general, your numbers sound excellent, even compared to isolate genomes.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JLtsr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JLtsr.png&quot; alt=&quot;checkm table (unfortunately image only)&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'd recommend for context taking some finished microbial genomes (E. coli K12 etc) and running CheckM. I bet you'd be surprised what you find.&lt;/p&gt;&#xA;&lt;p&gt;I think that you can probably not worry about (c) as a possibility. If you are very worried about (b), you could map reads against your assembly, filter out the ones that don't map well, and assemble those to see what you find. Long reads suggest that your genome is in good shape. You can of course do manual curation of your genome using tools like &lt;a href=&quot;https://anvio.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Anvi'o&lt;/a&gt;, or by scaffolding/gap-filling your assembly with long reads or an orthogonal strategy like optical mapping or Hi-C.&lt;/p&gt;&#xA;&lt;p&gt;I think that the influence of (a) is significant. Overall it sounds like you have a reasonable genome.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-02-14T23:14:11.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18589" PostTypeId="1" CreationDate="2022-02-15T06:55:27.577" Score="0" ViewCount="140" Body="&lt;p&gt;I know that before performing docking with &lt;a href=&quot;https://autodock-vina.readthedocs.io/en/latest/docking_python.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;AutoDock Vina&lt;/a&gt;, we have to compute &amp;quot;vina maps&amp;quot; like below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;v.set_ligand_from_file('1iep_ligand.pdbqt')&#xA;v.compute_vina_maps(center=[15.190, 53.903, 16.917], box_size=[20, 20, 20])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but I do not quite understand what those maps exactly are.&#xA;I found from the manual that&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;...and then compute the affinity maps for each ligand atom types accroding to the Vina forcefield.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;but again, what are affinity maps and Vina forcefield. I checked the &lt;a href=&quot;https://github.com/ccsb-scripps/AutoDock-Vina/blob/ea68a7abd8d90d2f8ab296521c0d07ee828f9d46/src/lib/vina.cpp#:%7E:text=Vina%3A%3Acompute_vina_maps&quot; rel=&quot;nofollow noreferrer&quot;&gt;source code&lt;/a&gt; to see that it creates some kind of a grid which will be then used for scoring. What kind of information does this grid data type contains? I would greatly appreciate any help.&lt;/p&gt;&#xA;" OwnerUserId="14875" LastActivityDate="2022-02-15T15:25:42.200" Title="Affinity maps and forcefields" Tags="|docking|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18590" PostTypeId="2" ParentId="18551" CreationDate="2022-02-15T10:47:24.473" Score="2" Body="&lt;p&gt;The different V numbers are just different number of SNPs as far as I'm aware. So there's not going to be a service which just imputes the variants to make e.g. V3 into V5.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://imputationserver.sph.umich.edu/index.html#!&quot; rel=&quot;nofollow noreferrer&quot;&gt;Michigan Imputation Server&lt;/a&gt; uses the Haplotype Reference Consortium to impute genotypes and allows input in 23andme format. You'll end up with many more SNPs than in the V5 chip, but filtering out the other ones should be easy.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="3694" LastEditDate="2022-02-15T10:54:28.157" LastActivityDate="2022-02-15T10:54:28.157" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18591" PostTypeId="1" CreationDate="2022-02-15T14:10:12.963" Score="3" ViewCount="833" Body="&lt;p&gt;I implemented Hirschberg's algorithm in python and used the wiki example to verify correct implementation given the scoring parameters and sequences:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Let X = AGTACGCA&#xA;    Y = TATGC&#xA;Del(X) = -2&#xA;Ins(Y) = -2&#xA;Sub(x,y) = -1 for mismatch +2 for match&#xA;&#xA;This aligns to: &#xA;&#xA;AGTACGCA&#xA;--TATGC-&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which matches the wiki output. I also printed the recursion and it matched the wiki recursive partitioning too. Then I tried it on a harder problem and got a terrible result. It was a mitochondrial Neanderthal DNA sample of about 500 nucleotides and a complete human DNA mitochondrial sample of 16504 nucleotides. The output of trying it on BlastN is here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi#alnHdr_Query_40035&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blast.ncbi.nlm.nih.gov/Blast.cgi#alnHdr_Query_40035&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;My results look something like this where the top is neanderthal and the bottom is the full human genome. It appears though it just matches each character as soon as it finds on.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;---C-CA-----A--------------------G-----T----AT---T--G--A----C-TC--------A--C-C-C-AT--CA-----A--C-----A-----A-C-C-----G-C-CA-T----GT-----ATT--T-C------------------G----TAC----A-T-T-AC---------T----------G------------------C--------CA--------------G-----C--CAC--C-A-T---------GA-AT-AT---------T-----------------------G-T-------ACAG---T-A--C-CAT-------AA--------------------T----------TA-C----TT----------------G-----A--C--T---A-C---C----------T--------------------------------------------------------G----T---A---A---------------T---A-C---ATA-----AA--AACC------------------------T--A---A--T--CC-----A---CA-T-CA---A-----------C---C-C-C--C-CCCC-----C-------------CC-A---T--------GCT-T-A---------CA-A-GCAAGCA-C--------AG------CA------A--TCA--AC---C--------------T--------------T-CA----A---C--T--G--T--C-A---------------------T-A----C----A----T---C-AA-------CTA--C-A-ACT--CC--A---------AA----G----A-C-ACC-C--T-------T-A-C--A---C--------CC-----A---------------C------T----A-----G-------------GA-T---A------TC-A----ACA-AA------C--C-----T------A-C----C----CAC-C-----CT-----------TG--A-------C---A----G-T-A-C---A----T-A---G------CA-CA----T-------A-AA------G---------T-----C--A-------T-----T--T--A---CC-----G-------T-------A--C-ATA------G--CA-C---A-----T-T-----A---TA------G---TC---A--AA----T--------C--C--------C-----T-------T-----C-T-----C--G------CCC---------C----------C-A--T-----------G--------GAT-G-----A-----C--------C--------------C------C-------------C---------C-------T----C----A------G------A---T----A-G-----------GG---------------------G--T--------------C----C----C-T--T--G---A---------------------------------------------------------------------------------------------------------------&#xA;GATCACAGGTCTATCACCCTATTAACCACTCACGGGAGCTCTCCATGCATTTGGTATTTTCGTCTGGGGGGTATGCACGCGATAGCATTGCGAGACGCTGGAGCCGGAGCACCCTATGTCGCAGTATCTGTCTTTGATTCCTGCCTCATCCTATTATTTATCGCACCTACGTTCAATATTACAGGCGAACATACTTACTAAAGTGTGTTAATTAATTAATGCTTGTAGGACATAATAATAACAATTGAATGTCTGCACAGCCACTTTCCACACAGACATCATAACAAAAAATTTCCACCAAACCCCCCCTCCCCCGCTTCTGGCCACAGCACTTAAACACATCTCTGCCAAACCCCAAAAACAAAGAACCCTAACACCAGCCTAACCAGATTTCAAATTTTATCTTTTGGCGGTATGCACTTTTAACAGTCACCCCCCAACTAACACATTATTTTCCCCTCCCACTCCCATACTACTAATCTCATCAATACAACCCCCGCCCATCCTACCCAGCACACACACACCGCTGCTAACCCCATACCCCGAACCAACCAAACCCCAAAGACACCCCCCACAGTTTATGTAGCTTACCTCCTCAAAGCAATACACTGAAAATGTTTAGACGGGCTCACATCACCCCATAAACAAATAGGTTTGGTCCTAGCCTTTCTATTAGCTCTTAGTAAGATTACACATGCAAGCATCCCCGTTCCAGTGAGTTCACCCTCTAAATCACCACGATCAAAAGGAACAAGCATCAAGCACGCAGCAATGCAGCTCAAAACGCTTAGCCTAGCCACACCCCCACGGGAAACAGCAGTGATTAACCTTTAGCAATAAACGAAAGTTTAACTAAGCTATACTAACCCCAGGGTTGGTCAATTTCGTGCCAGCCACCGCGGTCACACGATTAACCCAAGTCAATAGAAGCCGGCGTAAAGAGTGTTTTAGATCACCCCCTCCCCAATAAAGCTAAAACTCACCTGAGTTGTAAAAAACTCCAGTTGACACAAAATAGACTACGAAAGTGGCTTTAACATATCTGAACACACAATAGCTAAGACCCAAACTGGGATTAGATACCCCACTATGCTTAGCCCTAAACCTCAACAGTTAAATCAACAAAACTGCTCGCCAGAACACTACGAGCCACAGCTTAAAACTCAAAGGACCTGGCGGTGCTTCATATCCCTCTAGAGGAGCCTGTTCTGTAATCGATAAACCCCGATCAACCTCACCACCTCTTGCTCAGCCTATATACCGCCATCTTCAGCAAACCCTGATGAAGGCTACAAAGTAAGCGCAAGTACCCACGTAAAGACGTTAGGTCAAGGTGTAGCCCATGAGGTGGCAAGAAATGGGCTACATTTTCTACCCCAGAAAACTACGATAGCCCTTATGAAACTTAAGGGTCGAAGGTGGATTTAGCAGTAAACTAAGAGTAGAGTGCTTAGTTGAACAGGGCCCTGAAGCGCGTACACACCGCCCGTCACCCTCCTCAAGTATACTTCAAAGGACATTTAACTAAAACCCCTACGCATTTATATAGAGGAGACAAGTCGTAACATGGTAAGTGTACTGGAAAGTGCACTTGGACGAACCAGAGTGTAGCTTAACACAAAGCACCCAACTTACACTTAGGAGATTTCAACTTAACTTGACCGCTCTGAGCTAAACCTAGCCCCAAAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The neanderthal sample matches at the end of the human mitochondrial dna. I suspect the gap score penalty is far too costly to insert enough gaps to reach the end of the alignment but I set blast to a linear gap penalty. How do I overcome this? Here is the implementation code for my program:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;from Bio import Align&#xA;from Bio import pairwise2&#xA;insertionPenalty = -3&#xA;ambigSub = 0&#xA;match = 5&#xA;deletePenalty = -3&#xA;mismatch = -1&#xA;&#xA;#Ambiguous nucleotide meaning&#xA;ambNucleotideDict = {&amp;quot;N&amp;quot;:[&amp;quot;A&amp;quot;,&amp;quot;C&amp;quot;,&amp;quot;G&amp;quot;,&amp;quot;T&amp;quot;,&amp;quot;U&amp;quot;],&#xA;                     &amp;quot;R&amp;quot;:[&amp;quot;A&amp;quot;,&amp;quot;G&amp;quot;],&#xA;                     &amp;quot;Y&amp;quot;:[&amp;quot;T&amp;quot;,&amp;quot;C&amp;quot;],&#xA;                     &amp;quot;K&amp;quot;:[&amp;quot;G&amp;quot;,&amp;quot;T&amp;quot;],&#xA;                     &amp;quot;M&amp;quot;:[&amp;quot;A&amp;quot;,&amp;quot;C&amp;quot;],&#xA;                     &amp;quot;S&amp;quot;:[&amp;quot;G&amp;quot;,&amp;quot;C&amp;quot;],&#xA;                     &amp;quot;W&amp;quot;:[&amp;quot;A&amp;quot;,&amp;quot;T&amp;quot;],&#xA;                     &amp;quot;B&amp;quot;:[&amp;quot;C&amp;quot;,&amp;quot;G&amp;quot;,&amp;quot;T&amp;quot;],&#xA;                     &amp;quot;D&amp;quot;:[&amp;quot;A&amp;quot;,&amp;quot;G&amp;quot;,&amp;quot;T&amp;quot;],&#xA;                     &amp;quot;H&amp;quot;:[&amp;quot;A&amp;quot;,&amp;quot;C&amp;quot;,&amp;quot;T&amp;quot;],&#xA;                     &amp;quot;V&amp;quot;:[&amp;quot;A&amp;quot;,&amp;quot;C&amp;quot;,&amp;quot;G&amp;quot;]}&#xA;toPrint = False&#xA;#Scoring functions&#xA;def ins(y):&#xA;    return insertionPenalty&#xA;&#xA;&#xA;def dele(x):&#xA;    return deletePenalty&#xA;&#xA;&#xA;def sub(x, y):&#xA;    if x == y:&#xA;        return match&#xA;    elif x in ambNucleotideDict:&#xA;        if y in ambNucleotideDict[x]:&#xA;            return ambigSub&#xA;    elif y in ambNucleotideDict:&#xA;        if x in ambNucleotideDict[y]:&#xA;            return ambigSub&#xA;    else:&#xA;        return mismatch&#xA;&#xA;&#xA;&#xA;&#xA;# NWScore&#xA;def NWScore(X, Y):&#xA;    # Initialize NM Scoring Matrix. Array needs to be the length of X +1 and Y + 1&#xA;    score = [[0 for i in range(len(Y)+1)] for j in range(len(X)+1)]&#xA;    for j in range(1,len(Y)+1):&#xA;        # Insertion scoring penalty column&#xA;        score[0][j] = score[0][j-1] + ins(&amp;quot;a&amp;quot;)&#xA;    for i in range(1,len(X)+1):&#xA;&#xA;        # Delete Penalty Row&#xA;        score[i][0] = score[i-1][0] + dele(&amp;quot;a&amp;quot;)&#xA;    for i in range(1,len(X)+1):&#xA;        for j in range(1,len(Y)+1):&#xA;            # Scoring Matrix&#xA;            scoreSub = score[i-1][j - 1] + sub(X[i-1], Y[j-1])&#xA;            scoreDel = score[i-1][j] + dele(X[i-1])&#xA;            scoreIns = score[i][j - 1] + ins(Y[j-1])&#xA;            score[i][j] = max(scoreSub, scoreDel, scoreIns)&#xA;    if toPrint:&#xA;        print(X,Y)&#xA;        print(&amp;quot;Score&amp;quot;)&#xA;        for row in score:&#xA;            print(row)&#xA;    lastLine = score[-1]&#xA;    return lastLine&#xA;&#xA;&#xA;# Hirschberg&#xA;def Hirschberg(X, Y):&#xA;    Z = &amp;quot;&amp;quot;&#xA;    W = &amp;quot;&amp;quot;&#xA;    # If sequence X has no more char in the seq&#xA;    if len(X) == 0:&#xA;        for i in range(len(Y)):&#xA;            # Concatenate a gap(?) to Z and Y[i] to W&#xA;            Z = Z + '-'&#xA;            W = W + Y[i]&#xA;    # If sequence Y has no more char in the seq&#xA;    elif len(Y) == 0:&#xA;        for i in range(len(X)):&#xA;            Z = Z + X[i]&#xA;            W = W + '-'&#xA;    elif len(X) == 1 or len(Y) == 1:&#xA;        #If one seq has a single use full global alignment. For now there is a biopython implementation of global alignment&#xA;&#xA;        a = pairwise2.align.globalms(X,Y,2,-1,-2,-2)&#xA;        Z = a[0].seqA&#xA;        W = a[0].seqB&#xA;&#xA;&#xA;&#xA;    else:&#xA;        xlen = int(len(X))&#xA;        xmid = round(len(X)/2)&#xA;        ylen = int(len(Y))&#xA;        # Ok so this is the meat of the algorithm. The last few conditionals dealt with the base cases.&#xA;        # Scoreleft takes NWScore for the first half of sequence X, and Sequence Y&#xA;&#xA;&#xA;        ScoreL = NWScore(X[:xmid], Y)&#xA;        # I imagine this is to start from the right side.&#xA;        ScoreR = NWScore(X[xmid:][::-1], Y[::-1])[::-1]&#xA;&#xA;        ymid = numpy.argmax([x+y for (x,y) in zip(ScoreL,ScoreR)])&#xA;&#xA;        # Recursion. This is divide and conquer. Creating partitions that somehow dynamically reassemble even&#xA;        # though portions of it are getting reversed?!&#xA;        Z = Z+ Hirschberg(X[:xmid],Y[:ymid])[0]+Hirschberg(X[xmid:xlen],Y[ymid:ylen])[0]&#xA;        W = W+Hirschberg(X[:xmid],Y[:ymid])[1] +  Hirschberg(X[xmid:xlen],Y[ymid:ylen])[1]&#xA;&#xA;    return Z,W&#xA;seq1 = SeqIO.read(&amp;quot;seq1.fasta&amp;quot;,&amp;quot;fasta&amp;quot;)&#xA;seq2 = SeqIO.read(&amp;quot;seq2.fasta&amp;quot;,&amp;quot;fasta&amp;quot;)&#xA;&#xA;Z,W = Hirschberg(seq1.seq,seq2.seq)&#xA;print(Z)&#xA;print(W)&#xA;score = 0&#xA;for i in range(len(Z)):&#xA;    if Z[i] == '-' or W[i] == '-':&#xA;        score+= insertionPenalty&#xA;    elif Z[i]  == W[i]:&#xA;        score+=match&#xA;    else:&#xA;        score+=mismatch&#xA;&#xA;print(&amp;quot;Score: &amp;quot;,score)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15059" LastEditorUserId="3848" LastEditDate="2022-03-19T23:01:02.320" LastActivityDate="2022-03-19T23:01:02.320" Title="Hirschberg's Alignment Algorithm Implementation. Works on the wiki example but not on large sequences of dissimilar size" Tags="|sequence-alignment|phylogenetics|phylogeny|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18592" PostTypeId="2" ParentId="18591" CreationDate="2022-02-15T15:01:49.853" Score="1" Body="&lt;p&gt;&lt;strong&gt;Comparisons&lt;/strong&gt; First of all I would simply say pipe it through &lt;code&gt;Muscle&lt;/code&gt; and then compare the results.&lt;/p&gt;&#xA;&lt;p&gt;I forget the algorithm advances in the 'new' (now quite old) generation of aligners, but it is worth reading up on the latest Clustal omega, possibly MAFFT and how they view &lt;code&gt;Muscle&lt;/code&gt;. I do remember is 2018 Clustal Omega upgrade eulogising the historic innovation of Muscle around accuracy and describing how this was implemented. If that works for you, you simply replicate it in your code.&lt;/p&gt;&#xA;&lt;p&gt;The modern development in aligning algorithms is how they compare in speed versus number of sequences, but accuracy is always measured.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Amino acids -&amp;gt; nucleotides&lt;/strong&gt; The reason I don't follow the 'accuracy' issues is because its easier to align via amino acids and then switch back to codons. If you are sticking with this gap, penalty system using amino acids is a good approach and then simply back-translating, but if its a UTR then that will not work.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Brute-forcing&lt;/strong&gt; the gap penalties is certainly an option (which is the advantages of coding it). Proving why it has worked might be a bit tricky, but its doable. You could simply fire parameters onto the data and use regex (&lt;code&gt;import re&lt;/code&gt;) or straight character matching &lt;code&gt;==&lt;/code&gt; to assess the improvements. Regex is better to assess improvements, whilst &lt;code&gt;==&lt;/code&gt; approach is much faster. It depends on the parameter space (I think regex). Moving to an coded 'optical analysis' is an easy modification and could be useful.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The implication from the comment is there's suboptimal code ...&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The raw default output to Biopython's pairwise should be used to debug this. Then parameters changed stepwise to understand the anomalous output.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Skimming the code I don't spot anything obvious. What I don't understand is why the code has been duplicated&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;from Bio import Align&#xA;from Bio import pairwise2&#xA;insertionPenalty = -3&#xA;ambigSub = 0&#xA;match = 5&#xA;deletePenalty = -3&#xA;mismatch = -1&#xA;&#xA;#Ambiguous nucleotide meaning&#xA;ambNucleotideDict = {&amp;quot;N&amp;quot;:[&amp;quot;A&amp;quot;,&amp;quot;C&amp;quot;,&amp;quot;G&amp;quot;,&amp;quot;T&amp;quot;,&amp;quot;U&amp;quot;],&#xA;                     &amp;quot;R&amp;quot;:[&amp;quot;A&amp;quot;,&amp;quot;G&amp;quot;],&#xA;                     &amp;quot;Y&amp;quot;:[&amp;quot;T&amp;quot;,&amp;quot;C&amp;quot;],&#xA;                     &amp;quot;K&amp;quot;:[&amp;quot;G&amp;quot;,&amp;quot;T&amp;quot;],&#xA;                     &amp;quot;M&amp;quot;:[&amp;quot;A&amp;quot;,&amp;quot;C&amp;quot;],&#xA;                     &amp;quot;S&amp;quot;:[&amp;quot;G&amp;quot;,&amp;quot;C&amp;quot;],&#xA;                     &amp;quot;W&amp;quot;:[&amp;quot;A&amp;quot;,&amp;quot;T&amp;quot;],&#xA;                     &amp;quot;B&amp;quot;:[&amp;quot;C&amp;quot;,&amp;quot;G&amp;quot;,&amp;quot;T&amp;quot;],&#xA;                     &amp;quot;D&amp;quot;:[&amp;quot;A&amp;quot;,&amp;quot;G&amp;quot;,&amp;quot;T&amp;quot;],&#xA;                     &amp;quot;H&amp;quot;:[&amp;quot;A&amp;quot;,&amp;quot;C&amp;quot;,&amp;quot;T&amp;quot;],&#xA;                     &amp;quot;V&amp;quot;:[&amp;quot;A&amp;quot;,&amp;quot;C&amp;quot;,&amp;quot;G&amp;quot;]}&#xA;toPrint = False&#xA;#Scoring functions&#xA;def ins(y):&#xA;    return insertionPenalty&#xA;&#xA;&#xA;def dele(x):&#xA;    return deletePenalty&#xA;&#xA;&#xA;def sub(x, y):&#xA;    if x == y:&#xA;        return match&#xA;    elif x in ambNucleotideDict:&#xA;        if y in ambNucleotideDict[x]:&#xA;            return ambigSub&#xA;    elif y in ambNucleotideDict:&#xA;        if x in ambNucleotideDict[y]:&#xA;            return ambigSub&#xA;    else:&#xA;        return mismatch&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is present twice. I think Python would complain about the duplication of functions.&#xA;The duplication of variables is fine and probably imports.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;A looked through the code a final time and yes there are bugs&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def ins(y):&#xA;    return insertionPenalty&#xA;&#xA;&#xA;def dele(x):&#xA;    return deletePenalty&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That can't be right because whatever you call the functions (which you do), each time the output is &lt;code&gt;-3&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;No matter what you input the output is always &lt;code&gt;-3&lt;/code&gt;. If this was OOP and these were class methods in some sort of method chain that  might be different. These are functions which are operating not different to calling the variable direct. Thus if you substitute&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;... + dele(&amp;quot;a&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;... + deletePenalty&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or even&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;.... + in(&amp;quot;a&amp;quot;) # I know this isn't supposed to be here but its to make a point&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The answer in all cases is &lt;code&gt;-3&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you asking for bugs, thats got to be involved.&lt;/p&gt;&#xA;&lt;p&gt;Out of interest are you using an IDE with breakpoints etc...?&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-15T22:02:16.810" LastActivityDate="2022-02-15T22:02:16.810" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18593" PostTypeId="2" ParentId="18589" CreationDate="2022-02-15T15:25:42.200" Score="1" Body="&lt;p&gt;Docking is a computational calculation in the field of &lt;strong&gt;molecular mechanisms&lt;/strong&gt;.&#xA;It can be argued that whereas it is true one can 'wing it' without knowing the basics, it is best to know them —akin to the fact that it is possible to drive a car fine without knowing the road code or how a car works: many do for sure but it is not a good idea as something will go wrong.&lt;/p&gt;&#xA;&lt;p&gt;So I would suggest having a gander at the very least at:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Molecular_modelling&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Molecular_mechanics&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Force_field_(chemistry)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Force_field_(chemistry)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Lennard-Jones_potential&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Lennard-Jones_potential&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Solvent_model&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Solvent_model&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://manual.gromacs.org/documentation/current/reference-manual/topologies/particle-type.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://manual.gromacs.org/documentation/current/reference-manual/topologies/particle-type.html#atom-types&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So a forcefield is a sum of different terms that mathematically describes the forces acting between atoms. A thing to note is that these terms may be have some basis in chemistry or some may be just fudge factors to make the data fit.&lt;/p&gt;&#xA;&lt;p&gt;Regarding the atom types, in a full atom model, each atom is represented as a particle (or node in graph theory speak) and the bonds are fixed (edges).&#xA;An element behaves different depending on its hybridisation and charge etc. Therefore, each atom is classified not as an element, but as an atom type. Say, nitrogen in a ring and amine are generally different atom types. With parameterising a small molecule an atomtype may not be a perfect fit.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2022-02-15T15:25:42.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18594" PostTypeId="2" ParentId="18577" CreationDate="2022-02-15T15:54:57.210" Score="2" Body="&lt;p&gt;Don't read into the CheckM result too much. Genes having one copy in some species may have multiple copies in other species and genes present in some species may be absent in others. This may even happen between strains, let alone between species or between genera. It is rare to see 100% checkM completeness and 0% contamination. A common standard for &amp;quot;near complete&amp;quot; assembly is &amp;gt;90% completeness and &amp;lt;5% contamination. Your assembly is far above this standard.&lt;/p&gt;&#xA;&lt;p&gt;Meanwhile, beware that checkM only evaluates single-copy genes but multi-copy genes are usually harder to assemble. There might be complex multi-copy genes in the three assembly gaps. Without getting a circular contig, it is hard to say for sure your assembly is really complete.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2022-02-15T15:54:57.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18595" PostTypeId="1" CreationDate="2022-02-15T17:25:09.117" Score="2" ViewCount="1837" Body="&lt;p&gt;I want to identify &lt;code&gt;MDSCs&lt;/code&gt; cells which don't not express &lt;code&gt;HLA-DRs&lt;/code&gt; (HLA-DR negative) but express &lt;code&gt;CD14&lt;/code&gt;, &lt;code&gt;CD33&lt;/code&gt;, and &lt;code&gt;ITGAM&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;For this, I firstly plotted a feature plot to locate &lt;code&gt;HLA-DR&lt;/code&gt; by&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cd.features &amp;lt;- list(grep(pattern = '^HLA-DR', x = rownames(x = pbmc_small), value = TRUE))&#xA;&#xA;pbmc_small &amp;lt;- AddModuleScore(object = pbmc_small, features = cd.features, ctrl = 5, name = 'CDFeatures')&#xA;&#xA;&#xA;FeaturePlot(object = pbmc_small, features = 'CDFeatures1')&amp;amp; &#xA;+     scale_colour_gradientn(colours = rev(brewer.pal(n = 11, name = &amp;quot;RdBu&amp;quot;)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which returned&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Lg704.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Lg704.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For &lt;code&gt;CD14&lt;/code&gt;, &lt;code&gt;CD33&lt;/code&gt;, and &lt;code&gt;ITGAM&lt;/code&gt; I done&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; cd.features=list(c(&amp;quot;CD14&amp;quot;, &amp;quot;CD33&amp;quot;,  &amp;quot;ITGAM&amp;quot;))&#xA;&amp;gt; pbmc_small &amp;lt;- AddModuleScore(object = pbmc_small, features = cd.features, ctrl = 5, name = 'CDFeatures')&#xA;&amp;gt; FeaturePlot(object = pbmc_small, features = 'CDFeatures1')&amp;amp; &#xA;+     scale_colour_gradientn(colours = rev(brewer.pal(n = 11, name = &amp;quot;RdBu&amp;quot;)))&#xA;&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which returned&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZweUy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZweUy.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;By looking at these two plots I really can not say which part of plot is HLA-DR negative but expressing &lt;code&gt;CD14&lt;/code&gt;, &lt;code&gt;CD33&lt;/code&gt;, and &lt;code&gt;ITGAM&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Do you know a way to do this?&lt;/p&gt;&#xA;&lt;p&gt;Plotting options don't help because looking at the below plots you can not say which cells are HLA negative and CD14 positive&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/98EZD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/98EZD.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sA5YD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sA5YD.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2022-02-16T11:08:28.917" LastActivityDate="2023-12-13T09:11:12.080" Title="Featureplot by co-expression of some genes" Tags="|r|scrnaseq|seurat|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18596" PostTypeId="1" AcceptedAnswerId="18868" CreationDate="2022-02-16T08:59:41.837" Score="4" ViewCount="156" Body="&lt;p&gt;I am trying to understand the derivations from &lt;a href=&quot;https://www.nature.com/articles/ng.3211&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sullivan et al. (2015)&lt;/a&gt; in the &lt;a href=&quot;https://static-content.springer.com/esm/art%3A10.1038%2Fng.3211/MediaObjects/41588_2015_BFng3211_MOESM39_ESM.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Supplementary Material&lt;/a&gt;. There, it is mentioned in the first page that the least squares estimate of the j-th SNP effect, considering the polygenicity linear model &lt;span class=&quot;math-container&quot;&gt;$φ=Xβ + ε$&lt;/span&gt;, is &lt;span class=&quot;math-container&quot;&gt;$\hat{β_j} := X^T_j φ/N$&lt;/span&gt; . Normally, the least squares solution of &lt;span class=&quot;math-container&quot;&gt;$\hat{\beta}$&lt;/span&gt; is &lt;span class=&quot;math-container&quot;&gt;$(X^TX)^{-1}X^T\phi$&lt;/span&gt;. By comparing the two equations with each other, this means that, in this case, &lt;span class=&quot;math-container&quot;&gt;$X^TX=N I_{M\times M}$&lt;/span&gt;. However, LD is assumed, therefore &lt;span class=&quot;math-container&quot;&gt;$E(X_{ij}X_{ik})\neq 0$&lt;/span&gt;, so the covariance matrix cannot be diagonal. Could someone explain to me how the derivation for the beta estimate was performed, please? What am I missing? I checked that the same thing is mentioned in more than one sources such as &lt;a href=&quot;https://static-content.springer.com/esm/art%3A10.1038%2Fng.3404/MediaObjects/41588_2015_BFng3404_MOESM34_ESM.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15062" LastEditorUserId="15062" LastEditDate="2022-04-14T10:12:22.890" LastActivityDate="2022-04-14T10:12:22.890" Title="LD Score Regression Derivation hard to follow" Tags="|statistics|gwas|phylogenetics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18597" PostTypeId="2" ParentId="18591" CreationDate="2022-02-16T14:26:29.817" Score="0" Body="&lt;p&gt;Large sequences of dissimilar size? Sounds like you want a local alignment algorithm. Hirschberg is a global alignment algorithm, which will often perform unexpectedly on large sequences of dissimilar size.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2022-02-16T14:26:29.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18598" PostTypeId="1" AcceptedAnswerId="18603" CreationDate="2022-02-16T17:56:03.037" Score="2" ViewCount="5193" Body="&lt;p&gt;I know that there is a lot of methods to do so, but most of them need to have a reference file. (For example, the GATK force me to provide a reference.)&#xA;However, I think the BAM file I am working with already embedded the reference information in it. I tried to use the Samtool (mpileup) to do that, but it seems that the vcf file I get has different formats compared with the VCF documentations.&#xA;Could anyone give me some advice?&lt;/p&gt;&#xA;&lt;p&gt;Also, could anyone provide a method to validate the vcf file without using a reference file?&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot!&#xA;The following is what I got when running:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -H&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;@HD VN:1.5  GO:none SO:coordinate&#xA;@SQ SN:chr1 LN:248956422&#xA;@SQ SN:chr2 LN:242193529&#xA;@SQ SN:chr3 LN:198295559&#xA;@SQ SN:chr4 LN:190214555&#xA;@SQ SN:chr5 LN:181538259&#xA;@SQ SN:chr6 LN:170805979&#xA;@SQ SN:chr7 LN:159345973&#xA;@SQ SN:chr8 LN:145138636&#xA;@SQ SN:chr9 LN:138394717&#xA;@SQ SN:chr10    LN:133797422&#xA;@SQ SN:chr11    LN:135086622&#xA;@SQ SN:chr12    LN:133275309&#xA;@SQ SN:chr13    LN:114364328&#xA;@SQ SN:chr14    LN:107043718&#xA;@SQ SN:chr15    LN:101991189&#xA;@SQ SN:chr16    LN:90338345&#xA;@SQ SN:chr17    LN:83257441&#xA;@SQ SN:chr18    LN:80373285&#xA;@SQ SN:chr19    LN:58617616&#xA;@SQ SN:chr20    LN:64444167&#xA;@SQ SN:chr21    LN:46709983&#xA;@SQ SN:chr22    LN:50818468&#xA;@SQ SN:chrX LN:156040895&#xA;@SQ SN:chrY LN:57227415&#xA;@SQ SN:chrM LN:16569&#xA;@SQ SN:chr1_KI270706v1_random   LN:175055&#xA;@SQ SN:chr1_KI270707v1_random   LN:32032&#xA;@SQ SN:chr1_KI270708v1_random   LN:127682&#xA;@SQ SN:chr1_KI270709v1_random   LN:66860&#xA;@SQ SN:chr1_KI270710v1_random   LN:40176&#xA;@SQ SN:chr1_KI270711v1_random   LN:42210&#xA;@SQ SN:chr1_KI270712v1_random   LN:176043&#xA;@SQ SN:chr1_KI270713v1_random   LN:40745&#xA;@SQ SN:chr1_KI270714v1_random   LN:41717&#xA;@SQ SN:chr2_KI270715v1_random   LN:161471&#xA;@SQ SN:chr2_KI270716v1_random   LN:153799&#xA;@SQ SN:chr3_GL000221v1_random   LN:155397&#xA;@SQ SN:chr4_GL000008v2_random   LN:209709&#xA;@SQ SN:chr5_GL000208v1_random   LN:92689&#xA;@SQ SN:chr9_KI270717v1_random   LN:40062&#xA;@SQ SN:chr9_KI270718v1_random   LN:38054&#xA;@SQ SN:chr9_KI270719v1_random   LN:176845&#xA;@SQ SN:chr9_KI270720v1_random   LN:39050&#xA;@SQ SN:chr11_KI270721v1_random  LN:100316&#xA;@SQ SN:chr14_GL000009v2_random  LN:201709&#xA;@SQ SN:chr14_GL000225v1_random  LN:211173&#xA;@SQ SN:chr14_KI270722v1_random  LN:194050&#xA;@SQ SN:chr14_GL000194v1_random  LN:191469&#xA;@SQ SN:chr14_KI270723v1_random  LN:38115&#xA;@SQ SN:chr14_KI270724v1_random  LN:39555&#xA;@SQ SN:chr14_KI270725v1_random  LN:172810&#xA;@SQ SN:chr14_KI270726v1_random  LN:43739&#xA;@SQ SN:chr15_KI270727v1_random  LN:448248&#xA;@SQ SN:chr16_KI270728v1_random  LN:1872759&#xA;@SQ SN:chr17_GL000205v2_random  LN:185591&#xA;@SQ SN:chr17_KI270729v1_random  LN:280839&#xA;@SQ SN:chr17_KI270730v1_random  LN:112551&#xA;@SQ SN:chr22_KI270731v1_random  LN:150754&#xA;@SQ SN:chr22_KI270732v1_random  LN:41543&#xA;@SQ SN:chr22_KI270733v1_random  LN:179772&#xA;@SQ SN:chr22_KI270734v1_random  LN:165050&#xA;@SQ SN:chr22_KI270735v1_random  LN:42811&#xA;@SQ SN:chr22_KI270736v1_random  LN:181920&#xA;@SQ SN:chr22_KI270737v1_random  LN:103838&#xA;@SQ SN:chr22_KI270738v1_random  LN:99375&#xA;@SQ SN:chr22_KI270739v1_random  LN:73985&#xA;@SQ SN:chrY_KI270740v1_random   LN:37240&#xA;@SQ SN:chrUn_KI270302v1 LN:2274&#xA;@SQ SN:chrUn_KI270304v1 LN:2165&#xA;@SQ SN:chrUn_KI270303v1 LN:1942&#xA;@SQ SN:chrUn_KI270305v1 LN:1472&#xA;@SQ SN:chrUn_KI270322v1 LN:21476&#xA;@SQ SN:chrUn_KI270320v1 LN:4416&#xA;@SQ SN:chrUn_KI270310v1 LN:1201&#xA;@SQ SN:chrUn_KI270316v1 LN:1444&#xA;@SQ SN:chrUn_KI270315v1 LN:2276&#xA;@SQ SN:chrUn_KI270312v1 LN:998&#xA;@SQ SN:chrUn_KI270311v1 LN:12399&#xA;@SQ SN:chrUn_KI270317v1 LN:37690&#xA;@SQ SN:chrUn_KI270412v1 LN:1179&#xA;@SQ SN:chrUn_KI270411v1 LN:2646&#xA;@SQ SN:chrUn_KI270414v1 LN:2489&#xA;@SQ SN:chrUn_KI270419v1 LN:1029&#xA;@SQ SN:chrUn_KI270418v1 LN:2145&#xA;@SQ SN:chrUn_KI270420v1 LN:2321&#xA;@SQ SN:chrUn_KI270424v1 LN:2140&#xA;@SQ SN:chrUn_KI270417v1 LN:2043&#xA;@SQ SN:chrUn_KI270422v1 LN:1445&#xA;@SQ SN:chrUn_KI270423v1 LN:981&#xA;@SQ SN:chrUn_KI270425v1 LN:1884&#xA;@SQ SN:chrUn_KI270429v1 LN:1361&#xA;@SQ SN:chrUn_KI270442v1 LN:392061&#xA;@SQ SN:chrUn_KI270466v1 LN:1233&#xA;@SQ SN:chrUn_KI270465v1 LN:1774&#xA;@SQ SN:chrUn_KI270467v1 LN:3920&#xA;@SQ SN:chrUn_KI270435v1 LN:92983&#xA;@SQ SN:chrUn_KI270438v1 LN:112505&#xA;@SQ SN:chrUn_KI270468v1 LN:4055&#xA;@SQ SN:chrUn_KI270510v1 LN:2415&#xA;@SQ SN:chrUn_KI270509v1 LN:2318&#xA;@SQ SN:chrUn_KI270518v1 LN:2186&#xA;@SQ SN:chrUn_KI270508v1 LN:1951&#xA;@SQ SN:chrUn_KI270516v1 LN:1300&#xA;@SQ SN:chrUn_KI270512v1 LN:22689&#xA;@SQ SN:chrUn_KI270519v1 LN:138126&#xA;@SQ SN:chrUn_KI270522v1 LN:5674&#xA;@SQ SN:chrUn_KI270511v1 LN:8127&#xA;@SQ SN:chrUn_KI270515v1 LN:6361&#xA;@SQ SN:chrUn_KI270507v1 LN:5353&#xA;@SQ SN:chrUn_KI270517v1 LN:3253&#xA;@SQ SN:chrUn_KI270529v1 LN:1899&#xA;@SQ SN:chrUn_KI270528v1 LN:2983&#xA;@SQ SN:chrUn_KI270530v1 LN:2168&#xA;@SQ SN:chrUn_KI270539v1 LN:993&#xA;@SQ SN:chrUn_KI270538v1 LN:91309&#xA;@SQ SN:chrUn_KI270544v1 LN:1202&#xA;@SQ SN:chrUn_KI270548v1 LN:1599&#xA;@SQ SN:chrUn_KI270583v1 LN:1400&#xA;@SQ SN:chrUn_KI270587v1 LN:2969&#xA;@SQ SN:chrUn_KI270580v1 LN:1553&#xA;@SQ SN:chrUn_KI270581v1 LN:7046&#xA;@SQ SN:chrUn_KI270579v1 LN:31033&#xA;@SQ SN:chrUn_KI270589v1 LN:44474&#xA;@SQ SN:chrUn_KI270590v1 LN:4685&#xA;@SQ SN:chrUn_KI270584v1 LN:4513&#xA;@SQ SN:chrUn_KI270582v1 LN:6504&#xA;@SQ SN:chrUn_KI270588v1 LN:6158&#xA;@SQ SN:chrUn_KI270593v1 LN:3041&#xA;@SQ SN:chrUn_KI270591v1 LN:5796&#xA;@SQ SN:chrUn_KI270330v1 LN:1652&#xA;@SQ SN:chrUn_KI270329v1 LN:1040&#xA;@SQ SN:chrUn_KI270334v1 LN:1368&#xA;@SQ SN:chrUn_KI270333v1 LN:2699&#xA;@SQ SN:chrUn_KI270335v1 LN:1048&#xA;@SQ SN:chrUn_KI270338v1 LN:1428&#xA;@SQ SN:chrUn_KI270340v1 LN:1428&#xA;@SQ SN:chrUn_KI270336v1 LN:1026&#xA;@SQ SN:chrUn_KI270337v1 LN:1121&#xA;@SQ SN:chrUn_KI270363v1 LN:1803&#xA;@SQ SN:chrUn_KI270364v1 LN:2855&#xA;@SQ SN:chrUn_KI270362v1 LN:3530&#xA;@SQ SN:chrUn_KI270366v1 LN:8320&#xA;@SQ SN:chrUn_KI270378v1 LN:1048&#xA;@SQ SN:chrUn_KI270379v1 LN:1045&#xA;@SQ SN:chrUn_KI270389v1 LN:1298&#xA;@SQ SN:chrUn_KI270390v1 LN:2387&#xA;@SQ SN:chrUn_KI270387v1 LN:1537&#xA;@SQ SN:chrUn_KI270395v1 LN:1143&#xA;@SQ SN:chrUn_KI270396v1 LN:1880&#xA;@SQ SN:chrUn_KI270388v1 LN:1216&#xA;@SQ SN:chrUn_KI270394v1 LN:970&#xA;@SQ SN:chrUn_KI270386v1 LN:1788&#xA;@SQ SN:chrUn_KI270391v1 LN:1484&#xA;@SQ SN:chrUn_KI270383v1 LN:1750&#xA;@SQ SN:chrUn_KI270393v1 LN:1308&#xA;@SQ SN:chrUn_KI270384v1 LN:1658&#xA;@SQ SN:chrUn_KI270392v1 LN:971&#xA;@SQ SN:chrUn_KI270381v1 LN:1930&#xA;@SQ SN:chrUn_KI270385v1 LN:990&#xA;@SQ SN:chrUn_KI270382v1 LN:4215&#xA;@SQ SN:chrUn_KI270376v1 LN:1136&#xA;@SQ SN:chrUn_KI270374v1 LN:2656&#xA;@SQ SN:chrUn_KI270372v1 LN:1650&#xA;@SQ SN:chrUn_KI270373v1 LN:1451&#xA;@SQ SN:chrUn_KI270375v1 LN:2378&#xA;@SQ SN:chrUn_KI270371v1 LN:2805&#xA;@SQ SN:chrUn_KI270448v1 LN:7992&#xA;@SQ SN:chrUn_KI270521v1 LN:7642&#xA;@SQ SN:chrUn_GL000195v1 LN:182896&#xA;@SQ SN:chrUn_GL000219v1 LN:179198&#xA;@SQ SN:chrUn_GL000220v1 LN:161802&#xA;@SQ SN:chrUn_GL000224v1 LN:179693&#xA;@SQ SN:chrUn_KI270741v1 LN:157432&#xA;@SQ SN:chrUn_GL000226v1 LN:15008&#xA;@SQ SN:chrUn_GL000213v1 LN:164239&#xA;@SQ SN:chrUn_KI270743v1 LN:210658&#xA;@SQ SN:chrUn_KI270744v1 LN:168472&#xA;@SQ SN:chrUn_KI270745v1 LN:41891&#xA;@SQ SN:chrUn_KI270746v1 LN:66486&#xA;@SQ SN:chrUn_KI270747v1 LN:198735&#xA;@SQ SN:chrUn_KI270748v1 LN:93321&#xA;@SQ SN:chrUn_KI270749v1 LN:158759&#xA;@SQ SN:chrUn_KI270750v1 LN:148850&#xA;@SQ SN:chrUn_KI270751v1 LN:150742&#xA;@SQ SN:chrUn_KI270752v1 LN:27745&#xA;@SQ SN:chrUn_KI270753v1 LN:62944&#xA;@SQ SN:chrUn_KI270754v1 LN:40191&#xA;@SQ SN:chrUn_KI270755v1 LN:36723&#xA;@SQ SN:chrUn_KI270756v1 LN:79590&#xA;@SQ SN:chrUn_KI270757v1 LN:71251&#xA;@SQ SN:chrUn_GL000214v1 LN:137718&#xA;@SQ SN:chrUn_KI270742v1 LN:186739&#xA;@SQ SN:chrUn_GL000216v2 LN:176608&#xA;@SQ SN:chrUn_GL000218v1 LN:161147&#xA;@SQ SN:chrEBV   LN:171823&#xA;@RG ID:Wit_Exon9_DEL_FKDN202385493-1A_H5HHJDSXY_L1  CN:Novogene LB:Wit_Exon9_DEL    PL:illuminPU:Wit_Exon9_DEL_FKDN202385493-1A_H5HHJDSXY_L1    SM:Wit_Exon9_DEL&#xA;@PG ID:bwa  PN:bwa  CL:bwa mem -t 8 -M -R @RG\tID:Wit_Exon9_DEL_FKDN202385493-1A_H5HHJDSXY_L1\tSM:Wit_Exon9_DEL\tLB:Wit_Exon9_DEL\tPU:Wit_Exon9_DEL_FKDN202385493-1A_H5HHJDSXY_L1\tPL:illumina\tCN:Novogene /PUBLIC/source/HW/Disease/database/genome/human/hg38/bwa_index_whole/hg38.fa /TJPROJ6/GB_HUMAN/PJ_AI/cancer/auto-WES-Somatic/1000/X202SC2005/X202SC20050050-Z01/F002/QC/Wit_Exon9_DEL/Wit_Exon9_DEL_FKDN202385493-1A_H5HHJDSXY_L1_1.clean.fq.gz /TJPROJ6/GB_HUMAN/PJ_AI/cancer/auto-WES-Somatic/1000/X202SC2005/X202SC20050050-Z01/F002/QC/Wit_Exon9_DEL/Wit_Exon9_DEL_FKDN202385493-1A_H5HHJDSXY_L1_2.clean.fq.gz  VN:0.7.17-r1188&#xA;@PG ID:sambamba CL:view -S -f bam /dev/stdin    PP:bwa  VN:1.0&#xA;@PG ID:MarkDuplicates   VN:2.18.9-SNAPSHOT  CL:MarkDuplicates INPUT=[/TJPROJ6/GB_HUMAN/PJ_AI/service/cromwell/cromwell-executions/ToSomatic/fddd223f-25fd-4533-ae86-14e106c58ec5/call-Mapping/shard-0/Mapping/14f30bea-add3-4ead-b7e1-fcb202ae18f5/call-picardMarkdup/inputs/-714370009/Wit_Exon9_DEL.merge.bam] OUTPUT=/TJPROJ6/GB_HUMAN/PJ_AI/cancer/auto-WES-Somatic/1000/X202SC2005/X202SC20050050-Z01/F002/Mapping/Wit_Exon9_DEL/Wit_Exon9_DEL.final.bam METRICS_FILE=metric.txt REMOVE_DUPLICATES=false ASSUME_SORTED=true OPTICAL_DUPLICATE_PIXEL_DISTANCE=2500 TMP_DIR=[.] VALIDATION_STRINGENCY=SILENT MAX_RECORDS_IN_RAM=4000000 CREATE_INDEX=true    MAX_SEQUENCES_FOR_DISK_READ_ENDS_MAP=50000 MAX_FILE_HANDLES_FOR_READ_ENDS_MAP=8000 SORTING_COLLECTION_SIZE_RATIO=0.25 TAG_DUPLICATE_SET_MEMBERS=false REMOVE_SEQUENCING_DUPLICATES=false TAGGING_POLICY=DontTag CLEAR_DT=true ADD_PG_TAG_TO_READS=true DUPLICATE_SCORING_STRATEGY=SUM_OF_BASE_QUALITIES PROGRAM_RECORD_ID=MarkDuplicates PROGRAM_GROUP_NAME=MarkDuplicates READ_NAME_REGEX=&amp;lt;optimized capture of last three ':' separated fields as numeric values&amp;gt; MAX_OPTICAL_DUPLICATE_SET_SIZE=300000 VERBOSITY=INFO QUIET=false COMPRESSION_LEVEL=5 CREATE_MD5_FILE=false GA4GH_CLIENT_SECRETS=client_secrets.json USE_JDK_DEFLATER=false USE_JDK_INFLATER=false   PN:MarkDuplicates&#xA;@PG ID:samtools PN:samtools PP:sambamba VN:1.10 CL:samtools view -H /Users/XXXX/Desktop/data/02.Bam/XXXX.final.bam&#xA;@PG ID:samtools.1   PN:samtools PP:MarkDuplicates   VN:1.10 CL:samtools view -H /Users/XXXX/Desktop/data/02.Bam/XXXX.final.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output file I get when using Samtools looks like: &lt;a href=&quot;https://i.stack.imgur.com/8dIfA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8dIfA.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14102" LastEditorUserId="3694" LastEditDate="2022-02-22T13:34:52.073" LastActivityDate="2022-02-22T13:34:52.073" Title="How to convert a BAM file into a VCF file" Tags="|sam|vcf|gatk|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18599" PostTypeId="1" CreationDate="2022-02-16T20:48:43.377" Score="0" ViewCount="66" Body="&lt;p&gt;I want to use cloud computing services to analyze my oncological RNA-seq data.&lt;/p&gt;&#xA;&lt;p&gt;I came across &lt;strong&gt;Galaxy&lt;/strong&gt; and &lt;strong&gt;Cancer Genomics Cloud&lt;/strong&gt; as two cloud computing platforms which are used by the research community.&lt;/p&gt;&#xA;&lt;p&gt;Can someone give me the pros and cons of using either &lt;strong&gt;Galaxy&lt;/strong&gt; or &lt;strong&gt;Cancer Genomics Cloud&lt;/strong&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15071" LastEditorUserId="3694" LastEditDate="2022-02-17T15:12:12.050" LastActivityDate="2024-03-08T22:05:00.560" Title="Galaxy vs Cancer Genomics Cloud" Tags="|cloud|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18600" PostTypeId="2" ParentId="18599" CreationDate="2022-02-16T21:53:39.117" Score="0" Body="&lt;p&gt;OP update reveals the data is RNA-seq transcript abundance, that is very different from identifying novel driver mutations. So the stuff below isn't relevant.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Just to point out the Broad Institute has been using Google Cloud Platform (GCP) for years and they must be amongst the leaders in processing next gen sequencing data on cancer oncogenesis. They do have specific GCP pipelines available via Google.&lt;/p&gt;&#xA;&lt;p&gt;I've been tracking their work for sometime because of potential 'cross-over'. It is kinda impressive, albeit there are a few things they (excessively) over complicate.&lt;/p&gt;&#xA;&lt;p&gt;Just to throw it out there, and it depends what you mean by 'data', if I was processing 'driver mutations' in NGS there would be no question I'd use GCP under the Broad Institutes pipeline (they've changed its name, but you can just email them). So just to labor (labour) the point it depends whatever you mean by 'data'&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-17T14:09:11.833" LastActivityDate="2022-02-17T14:09:11.833" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18601" PostTypeId="2" ParentId="18598" CreationDate="2022-02-16T23:42:45.510" Score="-1" Body="&lt;p&gt;Recently, I also had a similar question and got my answer from Biostars. See my post and the answer I received: &lt;a href=&quot;https://www.biostars.org/p/9508767/#9508880&quot; rel=&quot;nofollow noreferrer&quot;&gt;Germline variant calling from BAM without reference&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Basically, I was also wondering if it is possible to call SNVs/indels from &lt;strong&gt;aligned&lt;/strong&gt; BAM files without reference because reference sequence was already used to produce such files. To this end, it appears some BAM files contain useful (but optional) read-level tags such as &lt;code&gt;MD&lt;/code&gt; which, if present, would allow one to even reconstruct the entire reference sequence. For more details, please visit the link above.&lt;/p&gt;&#xA;" OwnerUserId="12502" LastActivityDate="2022-02-16T23:42:45.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18602" PostTypeId="1" CreationDate="2022-02-17T11:12:33.907" Score="2" ViewCount="230" Body="&lt;p&gt;For my thesis, I'm working on side effect prediction in a machine learning framework and I am working on data available on SIDER (&lt;a href=&quot;http://sideeffects.embl.de/download/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://sideeffects.embl.de/download/&lt;/a&gt;). If you open the README file, the first description you'll see is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;meddra_all_se.tsv.gz&lt;br /&gt;&#xA;1 &amp;amp; 2: STITCH compound ids (flat/stereo, see above)&lt;br /&gt;&#xA;3: UMLS concept id as it was found on the label&lt;br /&gt;&#xA;4: MedDRA concept type (LLT = lowest level term, PT = preferred term; in a few cases the term is &amp;gt;neither LLT nor PT)&lt;br /&gt;&#xA;5: UMLS concept id for MedDRA term&lt;br /&gt;&#xA;6: side effect name&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I can not really understand what is the difference between flat and stereo compound ID; is it a chemical descriptor? Is it a notation issue? Moreover, I thought that the two IDS for the same compound could yield to similar results on PubChem but they don't. For example, in the dataset the STITCH ids for the first compound are CID100000085 and CID000010917; on PubChem the first is an aromatic compound and the second is not, they are widely different.&lt;br /&gt;&#xA;Thanks to everyone who will take the time to help me.&lt;/p&gt;&#xA;" OwnerUserId="15075" LastActivityDate="2024-01-08T15:09:48.253" Title="What is the difference between flat and stereo stitch id?" Tags="|database|public-databases|computational-biochemistry|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18603" PostTypeId="2" ParentId="18598" CreationDate="2022-02-17T12:00:48.757" Score="4" Body="&lt;p&gt;To go from &lt;code&gt;.bam&lt;/code&gt; to &lt;code&gt;.vcf&lt;/code&gt;, it is fairly easy using &lt;code&gt;bcftools call&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools mpileup -Ob -o &amp;lt;study.bcf&amp;gt; -f &amp;lt;ref.fa&amp;gt; &amp;lt;sample1.bam&amp;gt; &#xA;bcftools call -vmO z -o &amp;lt;study.vcf.gz&amp;gt; &amp;lt;study.bcf&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Taken from &lt;a href=&quot;http://www.htslib.org/workflow/wgs-call.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;htslib website&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2022-02-17T12:00:48.757" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18604" PostTypeId="1" AcceptedAnswerId="18606" CreationDate="2022-02-17T19:58:39.040" Score="3" ViewCount="840" Body="&lt;p&gt;I saw in the extensive Seurat documentation for Dimplot (dimensional reduction plot), &lt;a href=&quot;https://satijalab.org/seurat/reference/dimplot&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, you can plot a gene by specifying it with group.by = &amp;quot;gene&amp;quot; but this does not work in practice.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; p &amp;lt;- DimPlot(DG.combined, reduction = &amp;quot;umap&amp;quot;, group.by = &amp;quot;Fos&amp;quot;)&#xA;Error: Cannot find 'Fos' in this Seurat object&#xA;&amp;gt; p &amp;lt;- DimPlot(DG.combined, reduction = &amp;quot;umap&amp;quot;, group.by = &amp;quot;Inhba&amp;quot;)&#xA;Error: Cannot find 'Inhba' in this Seurat object&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Am I mistaken?  Do I need to add a metadata column containing expression of the gene?  How can I access individual cells gene expression data in seurat so I can create a metadata label based on binned expression levels?  Also how would I tell seurat to use a color gradient to label them?&lt;/p&gt;&#xA;&lt;p&gt;References&#xA;&lt;a href=&quot;https://satijalab.org/seurat/reference/dimplot&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/reference/dimplot&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2482" LastEditorUserId="3848" LastEditDate="2022-02-17T21:06:07.997" LastActivityDate="2022-02-17T21:09:27.950" Title="Plotting a gene in Seurat" Tags="|r|rna-seq|scrnaseq|seurat|visualization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18605" PostTypeId="2" ParentId="18604" CreationDate="2022-02-17T21:04:33.537" Score="2" Body="&lt;p&gt;I am not totally sure what you are trying to do, but I think you want FeaturePlot, not DimPlot&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2022-02-17T21:04:33.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18606" PostTypeId="2" ParentId="18604" CreationDate="2022-02-17T21:09:27.950" Score="2" Body="&lt;p&gt;As @swbarnes2 has suggested, you can plot gene expression over the top of a dimensional reduction using the &lt;code&gt;FeaturePlot&lt;/code&gt; function, using the &lt;code&gt;features&lt;/code&gt; parameter for gene names:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;p &amp;lt;- FeaturePlot(DG.combined, reduction = &amp;quot;umap&amp;quot;, features = c(&amp;quot;Fos&amp;quot;, &amp;quot;Inhba&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;See the 3k tutorial here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://satijalab.org/seurat/articles/pbmc3k_tutorial.html#cb49&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/articles/pbmc3k_tutorial.html#cb49&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-02-17T21:09:27.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18607" PostTypeId="1" AcceptedAnswerId="18608" CreationDate="2022-02-17T21:27:32.143" Score="3" ViewCount="95" Body="&lt;p&gt;I am using tblastn to search a polypeptide seq against transcripts.  One of those transcripts contains the residues for that exact polypeptide once translated, yet BLAST only gets 88% of it rather than it being full length.&lt;/p&gt;&#xA;&lt;p&gt;Here's the peptide:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Syntaxin&#xA;MTKDRLAALKAAQSDDDDNDDVAVTVDSSGFMEEFFEQVDEIREMIDKIASNVDEVKKKHSAILSAPQTDD&#xA;KMKEELEELMSEIKKNANKVRAKLKVIEQNIEQEEHTNKSSADLRIRKTQHATLSRKFVEVMNDYNACQID&#xA;YRERCKGRIKRQLAITGKTTTNEELEDMIESGNPAIFTQGIIMETQQANETLADIEARHNDIMKLETSIRD&#xA;LHDMFMDMAMLVESQGEMIDRIEYNVEQAVDYIETAKMDTKKAVKYQSKARRKKIMILVCLAILIIILVGV&#xA;IGGTLG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the &lt;a href=&quot;https://gist.github.com/jorvis/b1543e3284d80f6e7226505fcb4f5bbb&quot; rel=&quot;nofollow noreferrer&quot;&gt;transcript&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here's the first-frame translation of the transcript which contains the exact polypeptide above:&lt;/p&gt;&#xA;&lt;p&gt;frame +1&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; - 669&#xA;PATRNITIVSIYDYH*NNDLKCVFTE*MEILNRIWILLNSHSP*AQRLHAI**MTKDRLAALKAAQSDDDD&#xA;NDDVAVTVDSSGFMEEFFEQVDEIREMIDKIASNVDEVKKKHSAILSAPQTDDKMKEELEELMSEIKKNAN&#xA;KVRAKLKVIEQNIEQEEHTNKSSADLRIRKTQHATLSRKFVEVMNDYNACQIDYRERCKGRIKRQLAITGK&#xA;TTTNEELEDMIESGNPAIFTQGIIMETQQANETLADIEARHNDIMKLETSIRDLHDMFMDMAMLVESQGEM&#xA;IDRIEYNVEQAVDYIETAKMDTKKAVKYQSKARRKKIMILVCLAILIIILVGVIGGTLG***PSSLPLVYS&#xA;LIQLFYAACTTAASNIWTHHNRNSYYYNSNSNSNSNCYSNYFSNYYSNRKHYYYYHYYYYYYYYYYYYYYY&#xA;YYYYYYDYYYYYYYYYYYYYYYYYCYYYYYYYYYHFHYYYHHYYYHYYYYHYYYHYHHHHHYHYHYHYHHH&#xA;HHYHYHYHYHYHYYYYYYYYYYYYYYYYYYYYYYCYYYYYYTFSLVISLYGSSEM*MRLLVLSLYTFYGLF&#xA;YPLQQKGSFFF*SLQQTGTGFYNFFSYCQLHNLSIPSPSYFF*EWVFSFLKKHLIEFFCFLILLKFVLSFS&#xA;PIISLIISVCRKKISRIFYTIYLKMS**NHYLLF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Yet when I do a local tblastn (using the modern blast+) or use NCBI's pairwise blast tool it doesn't return a full-length match.&lt;/p&gt;&#xA;&lt;p&gt;Why does it leave off the last 12% of the sequence match?&lt;/p&gt;&#xA;&lt;p&gt;Here's the alignment:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/144nk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/144nk.png&quot; alt=&quot;screenshot of the alignment&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="659" LastEditorUserId="298" LastEditDate="2022-02-17T21:35:40.210" LastActivityDate="2022-02-17T21:51:49.527" Title="tblastn not providing full alignment even on exact-matching sequences" Tags="|blast|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18608" PostTypeId="2" ParentId="18607" CreationDate="2022-02-17T21:45:57.677" Score="2" Body="&lt;p&gt;That's because by default, blast masks &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&amp;amp;PAGE_TYPE=BlastDocs&amp;amp;DOC_TYPE=FAQ#LCR&quot; rel=&quot;nofollow noreferrer&quot;&gt;low complexity regions&lt;/a&gt;. With the online &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PROGRAM=tblastn&amp;amp;PAGE_TYPE=BlastSearch&amp;amp;BLAST_SPEC=blast2seq&amp;amp;LINK_LOC=blasttab&amp;amp;LAST_PAGE=blastn&amp;amp;BLAST_INIT=blast2seq&quot; rel=&quot;nofollow noreferrer&quot;&gt;blast2seqs tool&lt;/a&gt;, you can disable this by clicking on the &amp;quot;Algorithm options&amp;quot; button:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/W2hIB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W2hIB.png&quot; alt=&quot;Screenshot of the NCBI blast2seqs page showing the relevant option&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And then deselect the &amp;quot;Low complexity regions&amp;quot; filter:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7wKnW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7wKnW.png&quot; alt=&quot;Screenshot of the NCBI blast2seqs page showing the relevant option&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This will let you align the entire sequence as expected:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Alignment statistics for match #1&#xA;Score   Expect  Method  Identities  Positives   Gaps&#xA;Frame&#xA;588 bits(1516)  0.0 Compositional matrix adjust.    290/290(100%)   290/290(100%)   0/290(0%)&#xA;+1&#xA;Query  1     MTKDRLAALKAAQSDDDDNDDVAVTVDSSGFMEEFFEQVDEIREMIDKIASNVDEVKKKH  60&#xA;             MTKDRLAALKAAQSDDDDNDDVAVTVDSSGFMEEFFEQVDEIREMIDKIASNVDEVKKKH&#xA;Sbjct  154   MTKDRLAALKAAQSDDDDNDDVAVTVDSSGFMEEFFEQVDEIREMIDKIASNVDEVKKKH  333&#xA;&#xA;Query  61    SAILSAPQTDDKMKEELEELMSEIKKNANKVRAKLKVIEQNIEQEEHTNKSSADLRIRKT  120&#xA;             SAILSAPQTDDKMKEELEELMSEIKKNANKVRAKLKVIEQNIEQEEHTNKSSADLRIRKT&#xA;Sbjct  334   SAILSAPQTDDKMKEELEELMSEIKKNANKVRAKLKVIEQNIEQEEHTNKSSADLRIRKT  513&#xA;&#xA;Query  121   QHATLSRKFVEVMNDYNACQIDYRERCKGRIKRQLAITGKTTTNEELEDMIESGNPAIFT  180&#xA;             QHATLSRKFVEVMNDYNACQIDYRERCKGRIKRQLAITGKTTTNEELEDMIESGNPAIFT&#xA;Sbjct  514   QHATLSRKFVEVMNDYNACQIDYRERCKGRIKRQLAITGKTTTNEELEDMIESGNPAIFT  693&#xA;&#xA;Query  181   QGIIMETQQANETLADIEARHNDIMKLETSIRDLHDMFMDMAMLVESQGEMIDRIEYNVE  240&#xA;             QGIIMETQQANETLADIEARHNDIMKLETSIRDLHDMFMDMAMLVESQGEMIDRIEYNVE&#xA;Sbjct  694   QGIIMETQQANETLADIEARHNDIMKLETSIRDLHDMFMDMAMLVESQGEMIDRIEYNVE  873&#xA;&#xA;Query  241   QAVDYIETAKMDTKKAVKYQSKARRKKIMILVCLAILIIILVGVIGGTLG  290&#xA;             QAVDYIETAKMDTKKAVKYQSKARRKKIMILVCLAILIIILVGVIGGTLG&#xA;Sbjct  874   QAVDYIETAKMDTKKAVKYQSKARRKKIMILVCLAILIIILVGVIGGTLG  1023&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the part of your transcript sequence that was masked:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;acgctaggctgaccctcctccctccctctagtctactccttaatacaattgttctacgccgcgtgcacc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am guessing it is the repeated &lt;code&gt;ccct&lt;/code&gt; nucleotides that resulted in the sequence being masked, but I am not sure.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2022-02-17T21:51:49.527" LastActivityDate="2022-02-17T21:51:49.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18609" PostTypeId="1" AcceptedAnswerId="18610" CreationDate="2022-02-18T03:35:20.537" Score="6" ViewCount="1215" Body="&lt;p&gt;I'm trying to test variations of CPU and memory for a specific process (&lt;code&gt;samtools sort&lt;/code&gt;) to measure computational efficiency (measured only by time). To do this, I'd like to set the CPU and memory for a &lt;code&gt;Nextflow&lt;/code&gt; &lt;code&gt;process&lt;/code&gt; dynamically.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Ultimate question:&lt;/strong&gt; How can I dynamically set CPU and memory for many instances of a single &lt;code&gt;Nextflow&lt;/code&gt; process definition?&lt;/p&gt;&#xA;&lt;h1&gt;Attempt 1&lt;/h1&gt;&#xA;&lt;p&gt;The most obvious solution was to submit distinct &lt;code&gt;Nextflow&lt;/code&gt; workflows for each combination of CPU and memory by specifying the CPU and memory on the command line (i.e., when calling &lt;code&gt;nextflow run ...&lt;/code&gt;), where my &lt;code&gt;Nextflow&lt;/code&gt; workflow has global parameters &lt;code&gt;params.samtools_total_threads&lt;/code&gt; and &lt;code&gt;params.samtools_mem_per_thread&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This didn't work perfectly because there appeared to be 'collisions' between parent &lt;code&gt;Nextflow&lt;/code&gt; instances trying to lock the history file. I got the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Can't lock file: /path/to/project/.nextflow/history -- Nextflow needs to run in a file system that supports file locks&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I could try delaying submissions, but I suspect that will still result in collisions.&lt;/p&gt;&#xA;&lt;p&gt;Here's the &lt;code&gt;SLURM&lt;/code&gt; script that I'm using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TOTAL_THREADS=&lt;span class=&quot;math-container&quot;&gt;$1&#xA;MEM_PER_THREAD=$&lt;/span&gt;2&#xA;&#xA;nextflow run 02-SAMTOOLS_SORT.nf \&#xA;    -with-report &amp;quot;reports/report.&lt;span class=&quot;math-container&quot;&gt;${TOTAL_THREADS}_threads.$&lt;/span&gt;{MEM_PER_THREAD}_GB_per_thread.html&amp;quot; \&#xA;    -with-trace &amp;quot;traces/trace.&lt;span class=&quot;math-container&quot;&gt;${TOTAL_THREADS}_threads.$&lt;/span&gt;{MEM_PER_THREAD}_GB_per_thread.txt&amp;quot; \&#xA;    --samtools_total_threads &lt;span class=&quot;math-container&quot;&gt;$TOTAL_THREADS \&#xA;    --samtools_mem_per_thread $&lt;/span&gt;MEM_PER_THREAD&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My workflow had the following global parameters:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/*&#xA; * Make this pipeline a nextflow 2 implementation&#xA; */&#xA;nextflow.enable.dsl=2&#xA;&#xA;...&#xA;&#xA;params.samtools_total_threads = '4'&#xA;params.samtools_mem_per_thread = '8'&#xA;&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h1&gt;Attempt 2&lt;/h1&gt;&#xA;&lt;p&gt;After Attempt 1 failed, I figured the most 'Nextflow-like way' would be to generate tuples for each combination of CPU and memory that I want to test. Unfortunately, it doesn't appear that it's possible to set CPU and memory directives using parameters passed into the &lt;code&gt;process&lt;/code&gt;, but only with global variables.&lt;/p&gt;&#xA;&lt;p&gt;I get the following &lt;code&gt;No such variable&lt;/code&gt; error with the below code (note: most of this code is courtesy of @Steve):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;N E X T F L O W  ~  version 21.04.3&#xA;Launching `02-SAMTOOLS_SORT.nf` [desperate_aryabhata] - revision: fb9e79055a&#xA;No such variable: total_threads&#xA;&#xA; -- Check script '02-SAMTOOLS_SORT.nf' at line: 33 or see '.nextflow.log' file for more details&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Process code&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process samtools_coordinate_sort_proc {&#xA;&#xA;    tag { &amp;quot;${bam.baseName}&amp;quot; }&#xA;&#xA;    executor='slurm'&#xA;    queue = 'normal'&#xA;    cpus = total_threads&#xA;    memory = total_mem.GB&#xA;    clusterOptions = &amp;quot;--time 5:00:00 --account &amp;lt;account&amp;gt;&amp;quot;&#xA;&#xA;    /* delete files upon completion (I think) */&#xA;    cleanup = true&#xA;&#xA;    input:&#xA;    path(bam)&#xA;    val(total_threads)&#xA;    val(total_mem)&#xA;&#xA;    output:&#xA;    path(&amp;quot;${bam.baseName}.csorted.bam&amp;quot;)&#xA;&#xA;    script:&#xA;&#xA;    def additional_threads = task.cpus - 1&#xA;&#xA;    def avail_mem = task.memory ? task.memory.toGiga().intdiv(task.cpus) : 0&#xA;    println &amp;quot;avail_mem: &lt;span class=&quot;math-container&quot;&gt;$avail_mem&quot;&#xA;    def mem_per_thread = avail_mem ? &quot;$&lt;/span&gt;{avail_mem}G&amp;quot; : ''&#xA;    println &amp;quot;mem_per_thread: $mem_per_thread&amp;quot;&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    samtools sort \\&#xA;        -@ &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${additional_threads}&quot; \\&#xA;    -m $&lt;/span&gt;{mem_per_thread} \\&#xA;        -o &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${bam.baseName}.csorted.bam&quot; \\&#xA;    -T &quot;$&lt;/span&gt;{bam.baseName}.csorted&amp;quot; \\&#xA;        --write-index \\&#xA;        &amp;quot;${bam}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h1&gt;Possible attempt 3&lt;/h1&gt;&#xA;&lt;p&gt;Of course, it would be &lt;em&gt;super&lt;/em&gt; easy to do this with a couple of &lt;code&gt;bash&lt;/code&gt; scripts, but I'm trying to do it in &lt;code&gt;Nextflow&lt;/code&gt; for two reasons:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;To learn &lt;code&gt;Nextflow&lt;/code&gt; better&lt;/li&gt;&#xA;&lt;li&gt;Because the traces provided by &lt;code&gt;Nextflow&lt;/code&gt; are super convenient (though I could get this information from &lt;code&gt;SLURM&lt;/code&gt;, too).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="136" LastEditorUserId="136" LastEditDate="2022-02-18T14:20:53.297" LastActivityDate="2022-02-18T14:20:53.297" Title="Nextflow: dynamically setting resource directives for a process" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18610" PostTypeId="2" ParentId="18609" CreationDate="2022-02-18T05:17:49.667" Score="6" Body="&lt;p&gt;It's possible to specify &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#dynamic-computing-resources&quot; rel=&quot;noreferrer&quot;&gt;dynamic computing resources&lt;/a&gt; using a closure. For example, this sets the &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#cpus&quot; rel=&quot;noreferrer&quot;&gt;cpus&lt;/a&gt; and &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#memory&quot; rel=&quot;noreferrer&quot;&gt;memory&lt;/a&gt; directives using the values supplied via the input declaration:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;nextflow.enable.dsl=2&#xA;&#xA;process test_samtools {&#xA;&#xA;    echo true&#xA;&#xA;    cpus { cpus }&#xA;    memory { memory }&#xA;&#xA;    input:&#xA;    tuple path(bam), val(cpus), val(memory)&#xA;&#xA;    script:&#xA;    def additional_threads = task.cpus - 1&#xA;&#xA;    def avail_mem = task.memory ? task.memory.toGiga().intdiv(task.cpus) : 0&#xA;    def mem_per_thread = avail_mem ? &amp;quot;-m ${avail_mem}G&amp;quot; : ''&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    echo samtools sort \\&#xA;        -@ &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${additional_threads}&quot; \\&#xA;        $&lt;/span&gt;{mem_per_thread} \\&#xA;        -o &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${bam.baseName}.csorted.bam&quot; \\&#xA;        -T &quot;$&lt;/span&gt;{bam.baseName}.csorted&amp;quot; \\&#xA;        --write-index \\&#xA;        &amp;quot;${bam}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;workflow {&#xA;&#xA;    cpu_ch = Channel.of( 1, 2, 3, 4, 5)&#xA;    memory_ch = Channel.of( 1.GB, 2.GB, 4.GB )&#xA;&#xA;    Channel.fromPath( './data/*.bam' )&#xA;        | combine( cpu_ch )&#xA;        | combine( memory_ch )&#xA;        | test_samtools&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2022-02-18T05:17:49.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18613" PostTypeId="1" CreationDate="2022-02-18T09:17:58.673" Score="0" ViewCount="209" Body="&lt;p&gt;how can i plot the relation between rmsd and free energy like the image i attached as i need to find the most stable conformation structure for my cycle peptide after finishing REMD simulation&#xA;&lt;a href=&quot;https://i.stack.imgur.com/0495U.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0495U.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15078" LastActivityDate="2024-01-22T23:01:35.147" Title="plot the relation between RMSD and free energy" Tags="|proteins|gromacs|peptide-shaker|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18615" PostTypeId="1" AcceptedAnswerId="18623" CreationDate="2022-02-18T14:09:28.350" Score="2" ViewCount="144" Body="&lt;p&gt;The UCI gene expression cancer RNA-Seq ​dataset (&lt;a href=&quot;https://archive.ics.uci.edu/ml/datasets/gene+expression+cancer+RNA-Seq&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://archive.ics.uci.edu/ml/datasets/gene+expression+cancer+RNA-Seq&lt;/a&gt;) consists of 5 types of cancers (BRCA, KIRC, LUAD, COAD, PRAD) and gene expression levels of 20,531 genes for 801 samples. The dataset looks like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oy67i.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oy67i.png&quot; alt=&quot;gene expression UCI dataset&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;How to query in The Cancer Genome Atlas (TCGA) to create an updated version of this dataset?&lt;/p&gt;&#xA;&lt;p&gt;I've done the following &lt;a href=&quot;https://portal.gdc.cancer.gov/&amp;amp;searchTableTab=files&quot; rel=&quot;nofollow noreferrer&quot;&gt;query&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7zPXN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7zPXN.png&quot; alt=&quot;same query in visual form&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is the given query and the UCI dataset equivalent? If not, how to query properly?&lt;/p&gt;&#xA;&lt;p&gt;P.S.: I'm a CS student; do not have any biology background.&lt;/p&gt;&#xA;" OwnerUserId="15079" LastEditorUserId="15079" LastEditDate="2022-02-19T17:17:52.230" LastActivityDate="2022-02-19T17:17:52.230" Title="Querying The Cancer Genome Atlas (TCGA) for gene expression" Tags="|genome|database|gene-expression|cancer|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18617" PostTypeId="2" ParentId="16203" CreationDate="2022-02-18T16:33:30.263" Score="1" Body="&lt;p&gt;You have to click on the dropdown menu and select &amp;quot;Use transcriptome from history&amp;quot;, which will allow you to specify your own file.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YysIr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YysIr.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I suspect the default will be built-in human transcriptome.&lt;/p&gt;&#xA;&lt;p&gt;Solution is not tested, so let us know if it actually works.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2022-02-18T16:33:30.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18619" PostTypeId="1" AcceptedAnswerId="18620" CreationDate="2022-02-18T19:41:06.147" Score="4" ViewCount="231" Body="&lt;p&gt;I am looking for a tool that performs a sequence similarity algorithm for proteins. More specifically, I am looking for something that would be usable in Python (or anything else usable in the command line actually) which has a similar output to the FASTA tool, i.e:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hgDzZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hgDzZ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For my use case, the 'Source' column is not super relevant, so it does not matter if it's not included in the output. What's most important is the DB:ID and the match statistics.&lt;/p&gt;&#xA;&lt;p&gt;Have you used anything similar? Do you have any suggestions regarding such a tool?&#xA;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="12036" LastActivityDate="2022-02-18T19:50:36.147" Title="Programmatic way of accessing the FASTA sequence similarity tool (or similar) in Python" Tags="|sequence-alignment|fasta|python|proteins|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18620" PostTypeId="2" ParentId="18619" CreationDate="2022-02-18T19:50:36.147" Score="2" Body="&lt;p&gt;Apparently there is a &lt;a href=&quot;https://www.ebi.ac.uk/seqdb/confluence/display/JDSAT/FASTA+Help+and+Documentation#FASTAHelpandDocumentation-RESTAPI&quot; rel=&quot;nofollow noreferrer&quot;&gt;REST API for FASTA&lt;/a&gt;. Looks like there are few dependencies but other than that should be straightforward to use.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2022-02-18T19:50:36.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18621" PostTypeId="2" ParentId="18299" CreationDate="2022-02-18T22:24:41.660" Score="3" Body="&lt;pre&gt;&lt;code&gt;from Bio.Seq import Seq&#xA;coding_dna = Seq(&amp;quot;AAYGANAGYCARAGYAAR&amp;quot;)&#xA;print(coding_dna.translate())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This gives&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;NXSQSK&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So .translate() will read a degenerate code and provide the unambiguous amino acid, except where it is unable to do so e.g. GAN because this is a split between asp and glu. Thus GAR or GAY is fine, but GAN gives X&lt;/p&gt;&#xA;&lt;p&gt;You would then use &lt;code&gt;itertools&lt;/code&gt; by mapping (&lt;code&gt;map&lt;/code&gt;) against the triplet code (&lt;code&gt;import Bio.Data.CodonTable&lt;/code&gt;) to obtain all permutations. Doesn't work for X admittedly and stop codons a special case (3 possibilities).&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Bio.Data.CodonTable.list_possible_proteins&lt;/code&gt; would solve the &lt;code&gt;X&lt;/code&gt; problem, but stop codons remain a special case.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I seriously did like the Stackoverflow link in the comments, with the double map funtion ... very slick.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-19T03:17:43.647" LastActivityDate="2022-02-19T03:17:43.647" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18622" PostTypeId="1" CreationDate="2022-02-19T00:30:17.683" Score="3" ViewCount="1125" Body="&lt;p&gt;I would like to reproduce the results on figure 1 from &lt;a href=&quot;https://www.nature.com/articles/nmeth.4184&quot; rel=&quot;nofollow noreferrer&quot;&gt;THIS&lt;/a&gt; publication.&#xA;I downloaded the data and explore it with h5ls and h5dump &lt;a href=&quot;https://bioinformatics.cvr.ac.uk/exploring-the-fast5-format/&quot; rel=&quot;nofollow noreferrer&quot;&gt;based on HDF5&lt;/a&gt;.&#xA;I would like to understand:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;h5ls -r &amp;lt;_fast5file_&amp;gt; | grep Dataset&lt;/code&gt;:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;/Analyses/Basecall_2D_000/BaseCalled_2D/Alignment Dataset {6925}&#xA;/Analyses/Basecall_2D_000/BaseCalled_2D/Fastq Dataset {SCALAR}&#xA;/Analyses/Basecall_2D_000/BaseCalled_complement/Events Dataset {5576}&#xA;/Analyses/Basecall_2D_000/BaseCalled_complement/Fastq Dataset {SCALAR}&#xA;/Analyses/Basecall_2D_000/BaseCalled_complement/Model Dataset {4096}&#xA;/Analyses/Basecall_2D_000/BaseCalled_template/Events Dataset {5793}&#xA;/Analyses/Basecall_2D_000/BaseCalled_template/Fastq Dataset {SCALAR}&#xA;/Analyses/Basecall_2D_000/BaseCalled_template/Model Dataset {4096}&#xA;/Analyses/Basecall_2D_000/HairpinAlign/Alignment Dataset {5107}&#xA;/Analyses/Basecall_2D_000/Log Dataset {SCALAR}&#xA;/Analyses/Calibration_Strand_000/Log Dataset {SCALAR}&#xA;/Analyses/EventDetection_000/Reads/Read_0/Events Dataset {11465}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alignment? Then &lt;code&gt;complement/Fastq&lt;/code&gt; is not the complement of &lt;code&gt;template/Fastq&lt;/code&gt; (analyzed with h5dump). What is it? Also, events have sometimes 4&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Some other datasets that were acquired somewhere else have the raw data and fastq but there is no associated information about kmer and mean(current)/std/length/etc... How do I generate events from raw data?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I went through all documentations and tutorials that I could found in the web but couldn't find anything simple and complete. It feels as if it was a secret.&lt;/p&gt;&#xA;&lt;p&gt;Example &lt;em&gt;fast5file&lt;/em&gt; would be &lt;code&gt;ecoli_er2925.MSssI.timp.100215.fast5/pass/kelvin_100215_ecolisssI_2515_1_ch195_file0_strand.fast5&lt;/code&gt; from &lt;a href=&quot;http://www.ebi.ac.uk/ena/data/view/PRJEB13021&quot; rel=&quot;nofollow noreferrer&quot;&gt;PRJEB13021&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2970" LastEditorUserId="73" LastEditDate="2022-02-19T21:08:12.113" LastActivityDate="2022-02-19T21:08:12.113" Title="Understanding fast5 format" Tags="|nanopore|fast5|hdf5|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18623" PostTypeId="2" ParentId="18615" CreationDate="2022-02-19T08:02:45.877" Score="2" Body="&lt;p&gt;To start with, your query above is selecting for breast cancer samples (TCGA-BRCA) only. The original dataset seems to contain 4 other cancer types.&lt;/p&gt;&#xA;&lt;p&gt;The link you provided has a link to &lt;a href=&quot;https://www.synapse.org/#!Synapse:syn4301332&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.synapse.org/#!Synapse:syn4301332&lt;/a&gt; and by looking at filenames of the 14 files there, I can tell that there are different types of data: miRNA expression, gene expression, methylation, ... And your query is limited to gene expression only. I am not even sure if you can use just a single query to fetch all of the 14 data files, you might need to use multiple queries.&lt;/p&gt;&#xA;&lt;p&gt;If you are familiar with R, I would recommend &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/TCGAbiolinks.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;TCGABiolinks&lt;/a&gt;, it will abstract away a lot of otherwise annoying manual steps. Apparently you can also use Python retrieve TCGA data:&lt;a href=&quot;https://docs.gdc.cancer.gov/API/Users_Guide/Python_Examples/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.gdc.cancer.gov/API/Users_Guide/Python_Examples/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2022-02-19T08:02:45.877" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18624" PostTypeId="2" ParentId="18622" CreationDate="2022-02-19T14:41:44.340" Score="2" Body="&lt;p&gt;FAST5 is a proprietary format developed by Oxford Nanopore Technologies, and there's not much great documentation online. All FAST5 files will have the &lt;code&gt;Raw/&lt;/code&gt; field, which contains the original measured &lt;code&gt;Signal&lt;/code&gt;. Additional &lt;code&gt;Analyses/&lt;/code&gt; fields can be added by tools such as basecallers which convert signal-&amp;gt;ACGT (&lt;a href=&quot;https://denbi-nanopore-training-course.readthedocs.io/en/latest/basecalling/basecalling.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Guppy&lt;/a&gt;) or signal-level aligners (&lt;a href=&quot;https://github.com/nanoporetech/tombo&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tombo&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;In particular, &lt;code&gt;/Analyses/Basecall_2D_000/&lt;/code&gt; contains the results of 2D basecalling, added by Guppy when run with the &lt;code&gt;--post-out&lt;/code&gt; flag. I think Tombo adds the event-level annotations you mention (k-mer mean/std/length). In 2D basecalling, both the physical forward and reverse strands for the same stretch of DNA are measured and basecalled. The basecalls will contain errors, so these will not be exact reverse complements of one another.&lt;/p&gt;&#xA;&lt;p&gt;Hope this answers your questions, I have a slightly more in-depth explanation on my blog: &lt;a href=&quot;https://timd.one/blog/genomics/file_formats.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://timd.one/blog/genomics/file_formats.php&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6332" LastActivityDate="2022-02-19T14:41:44.340" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18625" PostTypeId="2" ParentId="18622" CreationDate="2022-02-19T16:26:24.143" Score="1" Body="&lt;p&gt;I'll give a different perspective. Oxford Nanopore FAST5 is NGS within a &lt;code&gt;HDF5&lt;/code&gt; database. A clear description is &lt;a href=&quot;https://bioinformatics.cvr.ac.uk/exploring-the-fast5-format/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;HDF5 is a format used for rapid I/O of large data sets.&lt;/p&gt;&#xA;&lt;p&gt;Oxford Nano appear to have written a bit boilerplate around HDF5 and called it something that sounds like 'fasta'.&lt;/p&gt;&#xA;&lt;p&gt;A cool part of HDF5 (not FAST5) is  &lt;code&gt;HDF5 View&lt;/code&gt;, which can be downloaded &lt;a href=&quot;https://www.hdfgroup.org/downloads/hdfview/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Its cool because you can visualise the file structure of HDF5 db via a GUI. I very much suspect if you use HDF5 View on your FAST5 database you will be able to navigate (see not change) the entire file structure.&lt;/p&gt;&#xA;&lt;p&gt;To answer your question directly &lt;code&gt;h5ls&lt;/code&gt; is listing the HDF5 database. To construct a HDF5 db you assign the data under a series of 'keys' which forms the directory structure. However, you perhaps might not need Oxford tools, because  Python pandas have direct series of methods to connect, construct and manipulate HDF5.&lt;/p&gt;&#xA;&lt;p&gt;I raised the HDF5 in a question then answered here on Bioinfo SE &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/8724/hdf5-and-biosql-solutions&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Again the reason for HDF5's importance is unlike an SQL connection, data transfer and disconnect, HDF5 shifts data very quickly.&lt;/p&gt;&#xA;&lt;p&gt;I do question how Oxford Nanopore can lay claim to FAST5 because HDF5 is open source and always has been. If they want to lay claim to some boilerplate, be my guest - I write the stuff all the time ;-)&lt;/p&gt;&#xA;&lt;p&gt;HDF5 however is a serious project with major development time and appears to still be under active development.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-19T16:51:13.640" LastActivityDate="2022-02-19T16:51:13.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18626" PostTypeId="2" ParentId="18538" CreationDate="2022-02-19T19:45:05.990" Score="6" Body="&lt;h1&gt;Update 2&lt;/h1&gt;&#xA;&lt;p&gt;Ok, for what I expect will be my final update for this answer, I compared the following:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;samtools 1.14&lt;/code&gt; + &lt;code&gt;zlib&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;samtools 1.15&lt;/code&gt; + &lt;code&gt;libdeflate&lt;/code&gt; (this is a different version, but shouldn't have a major effect. They released a new version just before I asked cluster admins to compile against &lt;code&gt;libdeflate&lt;/code&gt; and didn't notice until later.)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;sambamba v0.8.2&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;: This time I used the following settings:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Only two samples instead of three&lt;/li&gt;&#xA;&lt;li&gt;Max &lt;code&gt;Nextflow&lt;/code&gt; queue size of 30 to avoid too many threads reading from the same two files.&lt;/li&gt;&#xA;&lt;li&gt;I tested CPU requests from 1 to 7 (step size 1) and then from 9 to 17 (step size 2).&lt;/li&gt;&#xA;&lt;li&gt;For jobs where I allotted few CPUs and low memory, I provided a 200% buffer to prevent the jobs from failing with &lt;code&gt;OUT OF MEMORY&lt;/code&gt; errors. i.e., I gave the &lt;code&gt;SLURM&lt;/code&gt; job 2x the amount of memory than I told the tools they could use. &lt;code&gt;sambamba&lt;/code&gt; was the worst offender for exceeding memory in these conditions, but I gave them all 200% buffer to keep comparisons equal.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Seems pretty clear to me that &lt;code&gt;sambamba&lt;/code&gt; is the fastest and uses requested CPU resources most efficiently (i.e., is able to use all of the CPUs requested up until ~8 or 9 CPUs). For reference, &lt;strong&gt;the difference between &lt;code&gt;sambamba&lt;/code&gt; and &lt;code&gt;samtools + libdeflate&lt;/code&gt; with 9 CPUs is ~10 minutes on average.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;We will certainly be using &lt;code&gt;sambamba&lt;/code&gt; going forward, probably with ~9 CPUs and ~9GB per thread since I had to provide a 200% buffer in lower CPU/Mem jobs to avoid job failure, anyway.&lt;/p&gt;&#xA;&lt;p&gt;Again, all code and results are available on my GitHub repository (&lt;a href=&quot;https://github.com/mebbert/Samtools_sort_optimization_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;Samtools sort optimization test&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;h2&gt;(Angle 1) Comparison between &lt;code&gt;samtools + zlib&lt;/code&gt;, &lt;code&gt;samtools + libdeflate&lt;/code&gt; (Lsamtools), and &lt;code&gt;sambamba&lt;/code&gt;&lt;/h2&gt;&#xA;&lt;p&gt;Here, I'm showing the general trend that &lt;code&gt;real time&lt;/code&gt; decreases with increased CPUs for all three tools, but Mem Per CPU doesn't have a huge effect. &lt;code&gt;sambamba&lt;/code&gt; out performs &lt;code&gt;samtools + zlib&lt;/code&gt; regardless of settings. &lt;code&gt;samtools + libdeflate&lt;/code&gt; out performs &lt;code&gt;samtools + zlib&lt;/code&gt; until ~11 CPUs, where they perform the same (plot 3 below explains why).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Interesting finding:&lt;/strong&gt; &lt;code&gt;samtools + libdeflate&lt;/code&gt; performs better than &lt;code&gt;sambamba&lt;/code&gt; with a single thread, but &lt;code&gt;sambamba&lt;/code&gt; takes over from there as CPUs increase (again, plot #3 explains why).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/87BoN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/87BoN.png&quot; alt=&quot;samtools, Lsamtools, and sambamba&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h2&gt;(Angle 2) Comparison between &lt;code&gt;samtools + zlib&lt;/code&gt;, &lt;code&gt;samtools + libdeflate&lt;/code&gt; (Lsamtools), and &lt;code&gt;sambamba&lt;/code&gt;&lt;/h2&gt;&#xA;&lt;p&gt;Here, I'm showing that &lt;code&gt;samtools + zlib&lt;/code&gt; and &lt;code&gt;sambamba&lt;/code&gt; performance flattens out at ~11 CPUs (again, plot 3 below explans why).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/B8WfV.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/B8WfV.png&quot; alt=&quot;samtools, Lsamtools, and sambamba&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Requested CPUs vs. CPU utilization for &lt;code&gt;samtools + zlib&lt;/code&gt;, &lt;code&gt;samtools + libdeflate&lt;/code&gt; (Lsamtools), and &lt;code&gt;sambamba&lt;/code&gt;&lt;/h2&gt;&#xA;&lt;p&gt;Here, I'm showing how well each tool is able to utilize the CPUs allotted to it. It was fascinating to see that &lt;code&gt;samtools + libdeflate&lt;/code&gt; flattens out very early in it's CPU utilization (i.e., ~3 CPUS) and struggles to cross the 400% CPU utilization with even &amp;gt;10 CPUs. This is why &lt;code&gt;samtools + libdeflate&lt;/code&gt; performance flattens out quickly.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;samtools + zlib&lt;/code&gt; does a much better job utilizing allotted CPUs, but still lags behind &lt;code&gt;sambamba&lt;/code&gt; throughout (and is slower, anyway).&#xA;&lt;a href=&quot;https://i.stack.imgur.com/eiSsn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eiSsn.png&quot; alt=&quot;CPU usage by requested CPUs&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h1&gt;Update 1&lt;/h1&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jTNaG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jTNaG.png&quot; alt=&quot;Realtime vs. CPUs and mem per thread (samtools sort)&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;My first plots (below) were showing &lt;code&gt;duration&lt;/code&gt; on the &lt;code&gt;z-axis&lt;/code&gt;, which is the time from job &lt;em&gt;submission&lt;/em&gt; to &lt;em&gt;completion&lt;/em&gt; rather than job &lt;em&gt;start&lt;/em&gt; to &lt;em&gt;completion&lt;/em&gt;. That's why the larger memory requests were much longer in the original plots. &lt;code&gt;Duration&lt;/code&gt; is still a useful measure, but I'm now including &lt;code&gt;real time&lt;/code&gt;, which is from &lt;em&gt;start&lt;/em&gt; to &lt;em&gt;completion&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Judging solely by &lt;code&gt;real time&lt;/code&gt; (above), there is a clear gain between 5 &amp;amp; 7 CPUs (~10 minutes, on average when compared to the same mem-per-thread).&lt;/p&gt;&#xA;&lt;p&gt;More data coming, including &lt;code&gt;samtools&lt;/code&gt; compiled against &lt;code&gt;libdeflate&lt;/code&gt; and &lt;code&gt;sambamba&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h1&gt;Original post&lt;/h1&gt;&#xA;&lt;p&gt;I made time to semi-formally look at this. I'm glad I did because the results surprised me. I don't answer all of my questions directly, but these results are good enough to get me going in the right direction.&lt;/p&gt;&#xA;&lt;p&gt;My code and results are in my &lt;code&gt;GitHub&lt;/code&gt; repository (&lt;a href=&quot;https://github.com/mebbert/Samtools_sort_optimization_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;Samtools_sort_optimization_test&lt;/a&gt;) where you will also find an interactive plot you can download to see the results more clearly (see &lt;code&gt;SAMTOOLS_SORT/samtools_sort_CPU_and_memory_comparison.tar.gz&lt;/code&gt; in the repository). It's hard to truly appreciate the 3D &lt;code&gt;plotly&lt;/code&gt; plot until you start rotating it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;NOTES:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Even though this is from only three samples, I think my results are accurate. I welcome any feedback in case there's a bug.&lt;/li&gt;&#xA;&lt;li&gt;In my experience, &lt;code&gt;samtools&lt;/code&gt; often exceeds its allotted memory (at least on a &lt;code&gt;SLURM&lt;/code&gt; scheduler). i.e., the job gets killed by &lt;code&gt;SLURM&lt;/code&gt;. To avoid this, I requested an additional 20% of memory to provide a buffer for &lt;code&gt;samtools&lt;/code&gt;. I don't know if this discrepancy is because of &lt;code&gt;SLURM&lt;/code&gt; or &lt;code&gt;samtools&lt;/code&gt;, though.&lt;/li&gt;&#xA;&lt;li&gt;These results are using &lt;code&gt;samtools&lt;/code&gt; compiled against &lt;code&gt;zlib&lt;/code&gt;. We are preparing a version of &lt;code&gt;samtools&lt;/code&gt; compiled against &lt;code&gt;libdeflate&lt;/code&gt;, per @user172818's suggestion. I will add those results once completed. I may also compare against &lt;code&gt;sambamba&lt;/code&gt;, per @acvill's suggestion.&lt;/li&gt;&#xA;&lt;li&gt;These results are related to my other question about &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/18539/samtools-sort-most-efficient-approach-to-sort-a-single-sample-after-aligning-sp&quot;&gt;the best sorting/merging method&lt;/a&gt;. I will post those results in that question and provide a brief summary here.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h1&gt;tl;dr (too long, didn't read)&lt;/h1&gt;&#xA;&lt;h2&gt;Primary conclusions based on the plots below&lt;/h2&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Optimal CPUs&lt;/strong&gt; (results as expected): There's a significant gain with some additional CPUs, but it levels off around 7 CPUs (total, not 'additional'; i.e., &lt;code&gt;samtools sort -@ 6&lt;/code&gt;). I thought we might still see &lt;em&gt;some&lt;/em&gt; nice gains above 7, but this is probably what most would expect (e.g., similar to what @user172818 suggested).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Optimal memory per thread&lt;/strong&gt; (&lt;s&gt;results NOT as expected&lt;/s&gt;): &lt;s&gt;There is very little to gain with more than 1GB of memory (per thread); that surprised me. There are &lt;em&gt;some&lt;/em&gt; performance gains up until ~7GB per thread (~2-4 minutes faster). What really surprised me, however, is that performance dramatically &lt;strong&gt;decreases&lt;/strong&gt; as memory per thread goes &amp;gt;7GB&lt;/s&gt;. The updated plot (above) shows there is some gain with &amp;gt;1GB mem per thread. When running lots of samples at once (hundreds to 1000), mem per thread seems to play a much larger role (data coming).&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Effect from sorting status&lt;/strong&gt; (results as expected): Largely in an attempt to do this as quickly as possible (laziness?), I used three samples, where one was from an aligned and unsorted &lt;code&gt;.bam&lt;/code&gt; file, and the other two were from &lt;code&gt;.bam&lt;/code&gt; files that had been aligned and sorted, but I then collated them (&lt;code&gt;samtools collate&lt;/code&gt;), rather than re-aligning them. Unsurprisingly, the collated samples (not the average use case) sorted slightly faster than the&#xA;unsorted sample (average use case).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h2&gt;Initial opinion&lt;/h2&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;&amp;quot;Small&amp;quot; sample set&lt;/strong&gt;: For a small sample set, I would probably use 7 total CPUs and 7GB per thread.&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;&amp;quot;Large&amp;quot; sample set&lt;/strong&gt;: I will probably use 7 CPUs (maybe 5) with only 1GB memory per thread. That will allow the scheduler to keep as many jobs going as possible.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h2&gt;&lt;code&gt;samtools sort&lt;/code&gt;: duration vs. CPUs and mem per thread&lt;/h2&gt;&#xA;&lt;p&gt;As mentioned above, an interactive version of this plot is available on my &lt;code&gt;GitHub&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Angle 1&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/o79Ek.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/o79Ek.png&quot; alt=&quot;Duration vs. CPUs and mem per thread (samtools sort)&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Angle 2&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/UGNSu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UGNSu.png&quot; alt=&quot;Duration vs. CPUs and mem per thread (samtools sort)&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h1&gt;Additional details&lt;/h1&gt;&#xA;&lt;p&gt;Analyses were run on &lt;a href=&quot;https://docs.ccs.uky.edu/display/HPC/MCC+system+overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;University of Kentucky's MCC Cluster&lt;/a&gt;, which uses &lt;code&gt;AMD 7702P&lt;/code&gt; processors, using &lt;code&gt;samtools v.1.14&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="136" LastEditorUserId="136" LastEditDate="2022-02-27T01:12:53.420" LastActivityDate="2022-02-27T01:12:53.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18627" PostTypeId="2" ParentId="18552" CreationDate="2022-02-19T20:04:56.240" Score="1" Body="&lt;p&gt;The question I asked a couple of days ago has been solved. Here I want to make a summary of this and hope it can help people who also encounter such problems later.&lt;/p&gt;&#xA;&lt;p&gt;First, about the way to process the bigbed file. I didn't find a very good way to work directly with bigbed file at that time, so I still firstly converted the bigbed file to the bed file using &lt;code&gt;bigBedTobed&lt;/code&gt; program and then apply bedtools. The &lt;code&gt;bigBedTobed&lt;/code&gt; program can be downloaded through this link: &lt;a href=&quot;https://hgdownload.soe.ucsc.edu/admin/exe/&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC programs&lt;/a&gt;, and it also contains many useful programs from  UCSC genome browser. I finally converted my data (117 GB bigbed file) on the server, and it took ~2.5h to convert it to bed file. After conversion, the converted bed file is about 550 GB. And it took ~4h to intersect with a 4 Mb bed file using bedtools.&lt;/p&gt;&#xA;&lt;p&gt;Second, the bug that I encountered when I tried to convert the bigbed file on my own laptop. The code I used is &lt;code&gt;bigBedToBed JASPAR2022_hg19.bb hg1 9.bed&lt;/code&gt; and it has error &lt;code&gt;Invalid argument, Error reading 9188986873 bytes.&lt;/code&gt; I reported this error to the UCSC genome browser group and they said it is a MacOS specific bug, reaching a default data limit. They have solved it in their development environment but it still needs some time to reach the public site. The current solution for this is to use a URL to the file instead of a local file allows the program to read appropriately sized pieces of data at a time&lt;code&gt;(-udcDir= in bigBedToBed)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Finally, this is my first time asking questions here and I really appreciate those good suggestions from @M__ and @ColinD.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Renew: Methods from @malcook can work directly with the bigbed file! For a very large bigbed file, it is indeed a more efficient way.&lt;/p&gt;&#xA;" OwnerUserId="12391" LastEditorUserId="12391" LastEditDate="2022-02-21T09:25:23.787" LastActivityDate="2022-02-21T09:25:23.787" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18628" PostTypeId="2" ParentId="18578" CreationDate="2022-02-20T03:03:23.467" Score="2" Body="&lt;p&gt;Bam files contain a MAPQ field that informs on whether an alignment is unique or not. With STAR, a MAPQ value of 255 means the read is mapped uniquely and a score lower than 255 means not unique.&lt;/p&gt;&#xA;&lt;p&gt;You can use samtools on a bam file to count or output the uniquely or multimapped reads by filtering for the MAPQ as described here: &lt;a href=&quot;https://groups.google.com/forum/#!topic/rna-star/ipHc9FrgaPU&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://groups.google.com/forum/#!topic/rna-star/ipHc9FrgaPU&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For example, count uniquely mapped reads:&#xA;&lt;code&gt;samtools view -c -q 255 myBamFile.bam&lt;/code&gt;&lt;br /&gt;&#xA;Output uniquely mapped reads:&#xA;&lt;code&gt;samtools view -q 255 myBamFile.bam&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;There are differences between STAR and BWA MAPQ output though. see here for details: &lt;a href=&quot;https://sequencing.qcfail.com/articles/mapq-values-are-really-useful-but-their-implementation-is-a-mess/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sequencing.qcfail.com/articles/mapq-values-are-really-useful-but-their-implementation-is-a-mess/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15084" LastEditorUserId="15084" LastEditDate="2022-02-20T03:14:18.773" LastActivityDate="2022-02-20T03:14:18.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18630" PostTypeId="1" AcceptedAnswerId="18632" CreationDate="2022-02-20T10:09:04.273" Score="2" ViewCount="72" Body="&lt;p&gt;I have a few questions about these three methods.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;We commonly use 16s rRNA to identify species and construct a phylogenetic tree. If there's a new isolate from a new and undiscovered ecosystem, should we confirm this new isolate identity with ANI taxonomy/ genome to genome distance calculation (GGDC)?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If that isolate has a high similarity with a certain species in the 16s rRNA analysis result, should we proceed to analyze further with ANI/GGDC? Or just let it be because we have a robust 16s rRNA analysis? How can we be sure with our 16s rRNA BLAST result?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;When should we use 16s rRNA rather than ANI/GGDC and vice versa?&lt;/li&gt;&#xA;&lt;li&gt;Is it always necessary to confirm 16s rRNA analysis with ANI/GGDC? (Assuming there's a lot of closely-related genome data available in databases)&lt;/li&gt;&#xA;&lt;li&gt;Should we confirm the ANI result with GGDC and vice versa?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="14560" LastEditorUserId="3848" LastEditDate="2022-02-20T17:10:33.933" LastActivityDate="2022-02-20T17:10:33.933" Title="Questions about microbial 16s rRNA phylogenetic, ANI Taxonomy, and GGDC" Tags="|phylogenetics|phylogeny|bacteria|16rrna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18631" PostTypeId="1" CreationDate="2022-02-20T11:16:17.920" Score="0" ViewCount="42" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9509833/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am trying to learn and understand the correct order of data processing steps for microarrays.&lt;/p&gt;&#xA;&lt;p&gt;I have data which already was analyzed by a researcher using Partek (with an ANOVA model with batch effects). I want to test the results versus a specific gene set in GSEA. However, as I was informed, the genes were not filtered for low expression before ANOVA. So I want to filter genes out with low expression as is recommended for GSEA. However, I wonder whether it is legitimate to filter after DE was already done.&lt;/p&gt;&#xA;&lt;p&gt;In what order should one apply flooring (setting all values below a threshold to that threshold's value), normalization, and batch correction?&lt;/p&gt;&#xA;" OwnerUserId="6317" LastEditorUserId="73" LastEditDate="2022-02-22T20:39:42.617" LastActivityDate="2022-02-23T21:25:36.360" Title="What is the correct order of flooring-normalization-batch correction for microarrays?" Tags="|normalization|microarray|batch-effects|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18632" PostTypeId="2" ParentId="18630" CreationDate="2022-02-20T15:11:46.413" Score="4" Body="&lt;p&gt;I do understand that you are working with  is an isolate and thus what occurs in metagenomics you feel is over doing it, particularly if 16S is e.g. &amp;gt;99% similarity via Blast.&lt;/p&gt;&#xA;&lt;p&gt;Before reading further you might  check my answer history to understand the angle I am approaching this question from. I would see select given methods as the 'gold-standard' other methods are compared against, in addition to single versus multi-gene trees. There is good theoretical criteria for believing this. For example, bacteria are often subject to heavy %GC bias and this makes other methods error prone, because it introduces compositional bias (below).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&#xA;Blast alone - thats a 'no' (theory section).  The problem with any one approach is that errors can arise from many different sources stemming from both biology and methodology. Thus a combination of approaches is required. Therefore your maximum approach is a minimum standard.&lt;/p&gt;&#xA;&lt;p&gt;I personally would go one step further than your question: full maximum likelihood and Bayesian phylogenetics with bootstraps in addition to Blast (minor), ANI and GGDC.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;History&lt;/strong&gt;&#xA;From tradition 16S is the major standard, but even then multi-gene phylogenies were required. Thus genome to genome distance calculation (GGDC) with ANI is essential at a minimum. 16S  is back in fashion because its easy to perform a metagenomics study of species population dynamics.&lt;/p&gt;&#xA;&lt;p&gt;Species delineation has always been problematic in bacteria and the rule of thumb was 5% divergence defined a species boundary.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Advice&lt;/strong&gt;&#xA;I would use a multi-locus appproach under a full phylogenetics analysis, where a species boundary is defined by &amp;gt;5% divergence and bootstraps &amp;gt;80% with Bayes P values &amp;gt; 0.95.&lt;/p&gt;&#xA;&lt;p&gt;Recombination is essentially ignored in taxonomy providing there isn't over reliance on a single gene tree, but using BLAST, ANI, GGDC as a method data-mining would justify a select sample size. The classification is then nailed with a phylogenetics analysis.&lt;/p&gt;&#xA;&lt;p&gt;One of your methods (probably ANI) might be based machine learning/deep learning (its not GGDC) and the accuracy of misassigning a species under this method is high. Historically it was really high (80% accuracy), this might have improved to 90%. Its not like image recognition where the error rate is beyond human perception.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Theory&lt;/strong&gt; A fundamental issue is that recombination in bacteria exists on a spectrum from clonality to panmixia. This makes single gene taxonomy risky, thus at theoretical&#xA;level would not believe 16S was a singularly criterion, but it never was. Your bacteria is unknown (to us) so we'd have to assume some level of recombination.&lt;/p&gt;&#xA;&lt;p&gt;Compositional bias is well known for taxonomic misassignment. It is readily demonstrated in simulation. Thus if two unrelated bacteria demonstrate e.g. 75% GC, they are more likely to artifactually grouped together on e.g. a distance criterion.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-20T17:03:49.580" LastActivityDate="2022-02-20T17:03:49.580" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18633" PostTypeId="2" ParentId="18552" CreationDate="2022-02-20T20:30:49.180" Score="1" Body="&lt;p&gt;You asked&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Does anyone know any tools that can intersect two bigbed files&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;...but it appears you really wanted to subset a single very large bigbed file based on the contents of a much smaller bed file.&lt;/p&gt;&#xA;&lt;p&gt;I disagree with your summation &amp;quot;there is not a very efficient way to work directly with the bigbed file&amp;quot;.  I see two options.&lt;/p&gt;&#xA;&lt;p&gt;The first is to repeatedly call &lt;code&gt;bigBedTobed&lt;/code&gt; once for each row in your .bed file.  If you have multiple cores, this can be parallelized using &lt;a href=&quot;https://www.gnu.org/software/parallel/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GNU parallel&lt;/a&gt;.  The following does that, emitting each overlapping result with additional column being the index into your .bed file from which the result stems.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat your.bed | parallel --lb -j --colsep '\t' bigBedToBed -chrom={1} -start={2} -end={3}  your.bb  /dev/stdout '|' sed -e 's/$/\\t${PARALLEL_SEQ}/'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Depending upon your application, there is a second approach which uses R/BioConductor's &lt;code&gt;rtracklayer::import.bb&lt;/code&gt; which can be parallelized across (chunks of) your .bed file.&lt;/p&gt;&#xA;&lt;p&gt;I considered these approaches in what I expect is an almost identical situation:&lt;/p&gt;&#xA;&lt;p&gt;My bigbed holds genome-wide transcription factor binding site predictions from &lt;a href=&quot;https://jaspar.genereg.net/genome-tracks/#ucsc_tracks&quot; rel=&quot;nofollow noreferrer&quot;&gt;JASPAR Genome Browser tracks&lt;/a&gt; as produced by &lt;a href=&quot;https://github.com/wassermanlab&quot; rel=&quot;nofollow noreferrer&quot;&gt;wassermanlab / JASPAR-UCSC-tracks&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My bed file holds the loci of ~23K putative enhancers as determined by an analysis of regions of accessible chromatin from ATAC-Seq and various ChIP marks.&lt;/p&gt;&#xA;&lt;p&gt;In my case I wished to produce a sparse Matrix with one row for each putative enhancer, and one column for each JASPAR motif (which will form the basis of downstream enrichment analyses).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bbf&amp;lt;-BigBedFile('path/to/your.bb')&#xA;gr&amp;lt;-import.bed('path/to/your.bed')&#xA;## I found I had to harmonize seqlevels (e.g. chromosome identifiers) to make this work.  YMMV&#xA;gr&amp;lt;-renameSeqlevels(gr,paste0('chr',gsub('^(K.*)\\.(\\d+)$','Un_\\1v\\2',seqlevels(gr))))&#xA;mm&amp;lt;-motifMatrix(gr,bbf)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which results in a sparse Matrix, viz:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; rownames(mm)&amp;lt;-as.character(gr) # optional.&#xA;&amp;gt; mm[1:5,1:5]&#xA;5 x 5 sparse Matrix of class &amp;quot;dgCMatrix&amp;quot;&#xA;                 MA0002.2 MA0003.4 MA0004.1 MA0006.1 MA0007.3&#xA;chr1:11156-12180       14        .        2       28        2&#xA;chr1:17836-20023       25        .       27       63        2&#xA;chr1:22495-24367       22        1       22      137        2&#xA;chr1:27533-28038        2        2        4        7        .&#xA;chr1:36549-37079        6        .        2       41        .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This depends upon the following definition:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;motifMatrix&amp;lt;-function(gr,bbf,motifID=c('name','TFName')[[1]],mc.jobs=100) {&#xA;  ## produce a sparse Matrix with one row for each region in&#xA;  ## GenomicRanges gr, and one column for each JASPAR motif indicated&#xA;  ## as overlapping &amp;lt;gr&amp;gt; accorinding to BigBedFile &amp;lt;bbf&amp;gt;.  Default to&#xA;  ## using the 'name' column as the motif identifier, but allow for&#xA;  ## possibly using another, such as 'TFName' (as used by 2022 version&#xA;  ## of&#xA;  ## https://jaspar.genereg.net/genome-tracks/#ucsc_tracks). Parallelize&#xA;  ## into mc.jobs queries, defaulting to 100, which was found suitable&#xA;  ## for ~25000 loci in &amp;lt;gr&amp;gt; against JASPAR matches to zebrafish&#xA;  ## danRer11 genome running on a few dozen cores.  In general,&#xA;  ## knowing how &amp;quot;big&amp;quot; each parallel job *should* be depends on&#xA;  ## available RAM and cores, and number of loci in &amp;lt;gr&amp;gt;.&#xA;  ##&#xA;  ## AUTHOR: malcolm_cook@stowers.org&#xA;  ##&#xA;  ## NB: you must install a version of `rtracklayer` which addresses&#xA;  ## issue [import\.bb returning incorrect ranges &amp;amp; results as&#xA;  ## compared with kent tools&#xA;  ## bigBedToBed](https://github.com/lawremi/rtracklayer/issues/59)&#xA;  ## (such as by `install_github('lawremi/rtracklayer')`)&#xA;  si&amp;lt;-parallel::splitIndices(length(gr),mc.jobs)&#xA;  mm&amp;lt;-mclapply(si,function(i) {&#xA;    gri&amp;lt;-gr[i]&#xA;    m&amp;lt;-import.bb(bbf,selection=BigBedSelection(gri, colnames = c(motifID)))&#xA;    fo&amp;lt;-findOverlaps(gri,m)&#xA;    xt&amp;lt;-&#xA;      xtabs(~.,data=list(locus=i[queryHits(fo)],motif=mcols(m[subjectHits(fo)])[[motifID]]),sparse=TRUE) &#xA;    xt&#xA;  })&#xA;  mm&amp;lt;-&#xA;    ## rBind.fill(mm) ## NOT!  It does NOT preserve dimnames and&#xA;    ## returns matrix when given Matrix.  Issue reported:&#xA;    ## [[https://github.com/cvarrichio/Matrix.utils/issues/5][rBind.fill&#xA;    ## on a list of Matrix arguably should return Matrix but returns&#xA;    ## matrix #5]]&#xA;    Reduce(rBind.fill,mm) # which does preserve dimnames and return Matrix&#xA;  mm&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;NB: If you choose this approach, you MUST install a version of &lt;code&gt;rtracklayer&lt;/code&gt; which addresses issue the &lt;a href=&quot;https://github.com/lawremi/rtracklayer/issues/59&quot; rel=&quot;nofollow noreferrer&quot;&gt;import.bb returning incorrect ranges &amp;amp; results as compared with kent tools bigBedToBed&lt;/a&gt; (such as by &lt;code&gt;install_github('lawremi/rtracklayer')&lt;/code&gt;)&lt;/p&gt;&#xA;" OwnerUserId="1692" LastEditorUserId="1692" LastEditDate="2022-02-20T20:36:13.047" LastActivityDate="2022-02-20T20:36:13.047" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18634" PostTypeId="2" ParentId="18552" CreationDate="2022-02-21T09:27:47.607" Score="1" Body="&lt;p&gt;My 2p, a little late and maybe not too useful:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I didn't run the bigbedtobed program on the server because I am not allowed to install the bigbedtobed program on our server.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Presumably, you cannot install in &lt;code&gt;/usr/local/bin&lt;/code&gt; or some other root directory because you don't have admin rights. This is the common setup on shared servers. However, many programs can be installed in the user's home directory without admin rights. Even better, install conda and mamba first (no need to be admin) and use that to install other programs.&lt;/p&gt;&#xA;&lt;p&gt;The problem with docker is that some system administrators are concerned about security and refuse to install it (and docker does need admin rights). Whether such concerns are founded, I don't know...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I tried to convert bigbed file to bed file and then applied bedtools, but I found it was very slow and couldn't make it.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;bedtools intersect supports the &lt;code&gt;-sorted&lt;/code&gt; options that should speed things up quite a bit - Have you applied it? Your huge bigbed files should be already sorted after you convert to bed.&lt;/p&gt;&#xA;" OwnerUserId="339" LastActivityDate="2022-02-21T09:27:47.607" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18635" PostTypeId="2" ParentId="2031" CreationDate="2022-02-21T16:48:59.333" Score="0" Body="&lt;p&gt;From the comments @MollePasscantando successfully ported DNAsp onto a Mac (Air) via Wine, but was not sure how. This is good news because Wine is very commonly used for porting Windows exes. The website states,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Oracle VirtualBox (free open-source). Parallels, CrossOver, VMware&#xA;Fusion (paid-versions; trial period)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;From the website it was update in February 26, 2019 and its current Beta Version is 6.12.03.&lt;/p&gt;&#xA;&lt;p&gt;There was a major &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/29029172/&quot; rel=&quot;nofollow noreferrer&quot;&gt;published&lt;/a&gt; update to the package in Mol Biol Evol in 2017 (version 6) for large data sets (NGS).&lt;/p&gt;&#xA;&lt;p&gt;Thus the question is still relevant because they still have not written an OSX specific version and this will limit its performance for the data sets it is now targeted for.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-02-21T16:48:59.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18637" PostTypeId="1" CreationDate="2022-02-22T06:47:35.180" Score="2" ViewCount="109" Body="&lt;p&gt;Quick question, is there a place where one can download BED files for the binding motifs of certain transcription factors. In my case, I'm looking for a BED file for the genomic regions (enhancers) where TET2 binds to genome-wide. I'm looking to examine the DNA methylation levels in those regions.&lt;/p&gt;&#xA;&lt;p&gt;Thank You!&lt;/p&gt;&#xA;" OwnerUserId="14978" LastActivityDate="2022-02-24T19:45:54.637" Title="Where to get BED files for regions that contain binding site motifs for specific transcription factors?" Tags="|phylogenetics|bed|motifs|transcription-factors|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18639" PostTypeId="1" CreationDate="2022-02-22T09:35:26.407" Score="3" ViewCount="442" Body="&lt;p&gt;When I am reading papers that compares bulk RNA sequencing and single-cell RNA sequencing, we often see papers describe bulk RNA seq measures the average cell expression.&lt;/p&gt;&#xA;&lt;p&gt;For example, in this paper &lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fgene.2019.00317/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;Single-Cell RNA-Seq Technologies and Related Computational Data Analysis&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;bulk RNA-seq mainly reflects the averaged gene expression across thousands of cells&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And this paper &lt;a href=&quot;https://genomemedicine.biomedcentral.com/articles/10.1186/s13073-017-0467-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;A practical guide to single-cell RNA-sequencing for biomedical research and clinical applications&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;the averaging that occurs in pooling large numbers of cells does not allow detailed assessment of the fundamental biological unit—the cell—or the individual nuclei that package the genome&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And this paper &lt;a href=&quot;https://link.springer.com/content/pdf/10.1007/s40484-016-0089-7.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Differential expression analyses for single-cell RNA-Seq: old questions on new data&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Standard RNA-Seq experiments need millions of cells for sequencing [2,3], and therefore can only get averaged measurements of gene expressions of the cells sequenced.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;My question is, when we perform the bulk RNA seq and calculates parameters like TPM, we don't actually divide the total number of mRNA transcripts with the number of cells (I think sometimes we don't even have an accurate number on the number of cells that we used in bulk RNA seq), we simply normalise it with the transcript length and count.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, if we &lt;strong&gt;DID NOT&lt;/strong&gt; divide the number of transcripts with the total cell number, how are we measuring the average of expression, but not the total expression?&lt;/p&gt;&#xA;&lt;p&gt;Or in other words, what is the &amp;quot;average expression&amp;quot; referring to? Average to what?&lt;/p&gt;&#xA;" OwnerUserId="14961" LastEditorUserId="14961" LastEditDate="2022-02-26T09:25:25.900" LastActivityDate="2022-02-26T09:25:25.900" Title="Why is bulk RNA sequencing reflecting AVERAGE expression but not TOTAL expression of all cells?" Tags="|rna-seq|scrnaseq|transcriptome|rna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18640" PostTypeId="2" ParentId="11072" CreationDate="2022-02-22T11:15:15.867" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @terdon, converted from comment (please improve if possible):&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Have you checked on &lt;a href=&quot;https://wormbase.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;WormBase&lt;/a&gt;? That's the main repository for c. elegans data. I don't know that they have connectome data, but if they exist, that's where they will be stored.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;[see &lt;a href=&quot;https://bioinformatics.stackexchange.com/help/privileges/comment&quot;&gt;here&lt;/a&gt; for more information on the purposes of comments in the StackExchange network]&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-02-22T11:15:15.867" CommentCount="0" CommunityOwnedDate="2022-02-22T11:15:15.867" ContentLicense="CC BY-SA 4.0" />
  <row Id="18642" PostTypeId="1" AcceptedAnswerId="18686" CreationDate="2022-02-22T13:36:14.680" Score="3" ViewCount="423" Body="&lt;p&gt;I would like to download a number of SARS-CoV-2 fastas from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sars-cov-2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI/Genbank&lt;/a&gt; using a web endpoint. I looked on their website but it's confusing.&lt;/p&gt;&#xA;&lt;p&gt;I have the accession numbers, but can't figure out how to use the NCBI/Genbank Entrez API.&lt;/p&gt;&#xA;&lt;p&gt;This is how I would do it using cov-spectrum: &lt;a href=&quot;https://lapis.cov-spectrum.org/open/v1/sample/fasta?genbankAccession=MZ039713,MZ039682,OV522182&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://lapis.cov-spectrum.org/open/v1/sample/fasta?genbankAccession=MZ039713,MZ039682,OV522182&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is there something similar for NCBI/Genbank?&lt;/p&gt;&#xA;&lt;p&gt;There must be a way, I just can't figure it out. The whole entrez thing is overwhelming to me, all I want is a simple API, like covSpectrum offers, for example. I scanned this but didn't help: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sars-cov-2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/sars-cov-2/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;E.g. how could I download all these SARS-CoV-2 nucleotide sequences with accession numbers: &lt;code&gt;MZ039713&lt;/code&gt;,&lt;code&gt;MZ039682&lt;/code&gt; and &lt;code&gt;OV522182&lt;/code&gt; using a URL?&lt;/p&gt;&#xA;&lt;p&gt;Some command line tools are advertised, but I don't want to have to use a CLI tool. Just a simple URL, does that not exist? CLI tools: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets/docs/v1/quickstarts/command-line-tools/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/datasets/docs/v1/quickstarts/command-line-tools/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I also don't want to do any clicking through a web interface, just a URL so that other web apps can download it behind the scenes.&lt;/p&gt;&#xA;" OwnerUserId="12272" LastEditorUserId="12272" LastEditDate="2022-02-22T14:15:15.057" LastActivityDate="2023-03-18T08:45:20.397" Title="How to programatically download SARS-CoV-2 fasta from NCBI/Genbank via API?" Tags="|fasta|phylogenetics|sars-cov-2|covid-19|genbank|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18643" PostTypeId="2" ParentId="18642" CreationDate="2022-02-22T14:13:47.893" Score="1" Body="&lt;p&gt;There are three ways to do this,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If you click on this link &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/labs/virus/vssi/#/virus?SeqType_s=Nucleotide&amp;amp;VirusLineage_ss=SARS-CoV-2,%20taxid:2697049&amp;amp;ids=MZ039713%20MZ039682%20OV522182%20&quot; rel=&quot;nofollow noreferrer&quot;&gt;Your NCBI examples&lt;/a&gt;. Its done.&lt;/li&gt;&#xA;&lt;li&gt;You would then just click &amp;quot;download&amp;quot;&lt;/li&gt;&#xA;&lt;li&gt;Click &amp;quot;next&amp;quot;, click &amp;quot;next&amp;quot;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The sequence then downloads.&lt;/p&gt;&#xA;&lt;p&gt;Essentially all I did was in the website click on the left hand navigation bar and clicked &amp;quot;accession number&amp;quot; and type in &lt;code&gt;MZ039713 MZ039682 OV522182&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The second way is to go to &lt;a href=&quot;https://ncbi.nlm.nih.gov/nuccore&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ncbi.nlm.nih.gov/nuccore&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Type type in &lt;code&gt;MZ039713 MZ039682 OV522182&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Click on the &amp;quot;Summary&amp;quot; and click on &amp;quot;Fasta&amp;quot;&lt;/li&gt;&#xA;&lt;li&gt;Click on &amp;quot;send to&amp;quot; and click on &amp;quot;File&amp;quot;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I'm not certain is point 3 immediately above is required.&lt;/p&gt;&#xA;&lt;p&gt;The final approach is to use efetch to automate the download, but that requires downloading the NCBI's efetch suite, which is then flipped through on a bash loop.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;efetch -db nuccore -id MZ039682 -format fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The link for looking to automate is:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/labs/virus/vssi/#/virus?SeqType_s=Nucleotide&amp;amp;VirusLineage_ss=SARS-CoV-2,%20taxid:2697049&amp;amp;ids=MZ039713%20MZ039682%20OV522182%20&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/labs/virus/vssi/#/virus?SeqType_s=Nucleotide&amp;amp;VirusLineage_ss=SARS-CoV-2,%20taxid:2697049&amp;amp;ids=MZ039713%20MZ039682%20OV522182%20&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Just to highlight&#xA;&lt;strong&gt;MZ039713&lt;/strong&gt;%20&lt;strong&gt;MZ039682&lt;/strong&gt;%20&lt;strong&gt;OV522182%20&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The API is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/home/develop/api/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, but I don't use them.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-22T14:45:28.263" LastActivityDate="2022-02-22T14:45:28.263" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18644" PostTypeId="1" AcceptedAnswerId="18652" CreationDate="2022-02-22T16:15:12.397" Score="4" ViewCount="573" Body="&lt;h1&gt;Update&lt;/h1&gt;&#xA;&lt;p&gt;The issue was that &lt;code&gt;bwa&lt;/code&gt; was running out of memory and failing, but that error wasn't floating to the top (see @Steve's answer, below). I was getting an error from &lt;code&gt;samtools&lt;/code&gt;. I should have been able to figure that out, but hopefully this post saves someone else the trouble.&lt;/p&gt;&#xA;&lt;h1&gt;Original post&lt;/h1&gt;&#xA;&lt;p&gt;I'm experiencing a strange intermittent parsing error when piping &lt;code&gt;bwa mem&lt;/code&gt; to &lt;code&gt;samtools view&lt;/code&gt;. It only seems to happen when the command runs within a &lt;code&gt;Nextflow&lt;/code&gt; pipeline (though it seems unlikely that &lt;code&gt;Nextflow&lt;/code&gt; has anything to do with it). If I &lt;code&gt;cd&lt;/code&gt; to the directory and run &lt;code&gt;bash .command.sh&lt;/code&gt;, it runs to completion without issue.&lt;/p&gt;&#xA;&lt;h1&gt;Nextflow process&lt;/h1&gt;&#xA;&lt;p&gt;Here's the &lt;code&gt;Nextflow&lt;/code&gt; process I'm running (code credit to @Steve, though modified to handle &lt;code&gt;.bam&lt;/code&gt; or &lt;code&gt;.cram&lt;/code&gt;). The portion that's running in this case&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process bwa_mem_proc {&#xA;&#xA;    tag { &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_name}:$&lt;/span&gt;{fastq.name}&amp;quot; }&#xA;&#xA;    label 'bwa_mem_proc'&#xA;&#xA;    input:&#xA;    tuple val(sample_name), path(fastq), path(header)&#xA;&#xA;    output:&#xA;    tuple val(sample_name), path(&amp;quot;${fastq.baseName}.unsorted.mini.bam&amp;quot;)&#xA;&#xA;    script:&#xA;    def task_cpus = task.cpus &amp;gt; 1 ? task.cpus - 1 : task.cpus&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    if [ &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$params.output_format&quot; = &quot;bam&quot; ]; then&#xA;    bwa mem \\&#xA;        -p \\&#xA;        -t $&lt;/span&gt;{task_cpus} \\&#xA;            -M \\&#xA;            -C \\&#xA;            -H &amp;lt;(grep &amp;quot;^@RG&amp;quot; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${header}&quot;) \\&#xA;        &quot;$&lt;/span&gt;{params.align_to_ref}&amp;quot; \\&#xA;            &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${fastq}&quot; |&#xA;    samtools view \\&#xA;        -1 \\&#xA;        -o &quot;$&lt;/span&gt;{fastq.baseName}.unsorted.mini.bam&amp;quot; \\&#xA;            -&#xA;    else&#xA;        bwa mem \\&#xA;            -p \\&#xA;            -t &lt;span class=&quot;math-container&quot;&gt;${task_cpus} \\&#xA;        -M \\&#xA;        -C \\&#xA;        -H &amp;lt;(grep &quot;^@RG&quot; &quot;$&lt;/span&gt;{header}&amp;quot;) \\&#xA;            &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${params.align_to_ref}&quot; \\&#xA;        &quot;$&lt;/span&gt;{fastq}&amp;quot; |&#xA;        samtools view \\&#xA;            -C \\&#xA;            -T params.align_to_ref \\&#xA;            -o &amp;quot;${fastq.baseName}.unsorted.mini.bam&amp;quot; \\&#xA;            -&#xA;    fi&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h1&gt;Exact error&lt;/h1&gt;&#xA;&lt;p&gt;Here's an example of the intermittent errors I'm getting. I would have assumed that some read (e.g., read # &lt;code&gt;2_102_429&lt;/code&gt; in the &lt;code&gt;.fastq&lt;/code&gt;) is malformed, but as I mentioned, running &lt;code&gt;bash .command.sh&lt;/code&gt; in the working directory for the failed run finishes without issue.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Command error:&#xA;  [M::mem_pestat] skip orientation FF&#xA;  [M::mem_process_seqs] Processed 700000 reads in 109.002 CPU sec, 16.411 real sec&#xA;  [M::process] 0 single-end sequences; 700000 paired-end sequences&#xA;  [M::process] read 700000 sequences (70000000 bp)...&#xA;  [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (24, 301945, 4, 10)&#xA;  [M::mem_pestat] analyzing insert size distribution for orientation FF...&#xA;  [M::mem_pestat] (25, 50, 75) percentile: (118, 180, 399)&#xA;  [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 961)&#xA;  [M::mem_pestat] mean and std.dev: (179.38, 124.36)&#xA;  [M::mem_pestat] low and high boundaries for proper pairs: (1, 1242)&#xA;  [M::mem_pestat] analyzing insert size distribution for orientation FR...&#xA;  [M::mem_pestat] (25, 50, 75) percentile: (247, 319, 420)&#xA;  [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 766)&#xA;  [M::mem_pestat] mean and std.dev: (342.90, 127.88)&#xA;  [M::mem_pestat] low and high boundaries for proper pairs: (1, 939)&#xA;  [M::mem_pestat] skip orientation RF as there are not enough pairs&#xA;  [M::mem_pestat] analyzing insert size distribution for orientation RR...&#xA;  [M::mem_pestat] (25, 50, 75) percentile: (326, 764, 1452)&#xA;  [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 3704)&#xA;  [M::mem_pestat] mean and std.dev: (893.30, 530.98)&#xA;  [M::mem_pestat] low and high boundaries for proper pairs: (1, 4830)&#xA;  [M::mem_pestat] skip orientation FF&#xA;  [M::mem_pestat] skip orientation RR&#xA;  [M::mem_process_seqs] Processed 700000 reads in 97.823 CPU sec, 14.292 real sec&#xA;  [M::process] 0 single-end sequences; 700000 paired-end sequences&#xA;  [M::process] read 700000 sequences (70000000 bp)...&#xA;  [M::mem_pestat] # candidate unique pairs for (FF, FR, RF, RR): (14, 196658, 9, 15)&#xA;  [M::mem_pestat] analyzing insert size distribution for orientation FF...&#xA;  [M::mem_pestat] (25, 50, 75) percentile: (206, 422, 1021)&#xA;  [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 2651)&#xA;  [M::mem_pestat] mean and std.dev: (506.92, 458.91)&#xA;  [M::mem_pestat] low and high boundaries for proper pairs: (1, 3466)&#xA;  [M::mem_pestat] analyzing insert size distribution for orientation FR...&#xA;  [M::mem_pestat] (25, 50, 75) percentile: (248, 322, 429)&#xA;  [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 791)&#xA;  [M::mem_pestat] mean and std.dev: (345.17, 129.22)&#xA;  [M::mem_pestat] low and high boundaries for proper pairs: (1, 972)&#xA;  [M::mem_pestat] skip orientation RF as there are not enough pairs&#xA;  [M::mem_pestat] analyzing insert size distribution for orientation RR...&#xA;  [M::mem_pestat] (25, 50, 75) percentile: (485, 777, 1315)&#xA;  [M::mem_pestat] low and high boundaries for computing mean and std.dev: (1, 2975)&#xA;  [M::mem_pestat] mean and std.dev: (961.60, 569.27)&#xA;  [M::mem_pestat] low and high boundaries for proper pairs: (1, 3805)&#xA;  [M::mem_pestat] skip orientation FF&#xA;  [M::mem_pestat] skip orientation RR&#xA;  [M::mem_process_seqs] Processed 700000 reads in 275.897 CPU sec, 39.910 real sec&#xA;  [M::process] 0 single-end sequences; 700000 paired-end sequences&#xA;  [M::process] read 700000 sequences (70000000 bp)...&#xA;  [W::sam_read1_sam] Parse error at line 2102429&#xA;  samtools view: error reading file &amp;quot;-&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A previous sample complained about an &lt;code&gt;@SQ&lt;/code&gt; header line missing the &lt;code&gt;LN:&lt;/code&gt; tag. It's like the pipe gets interrupted and lines get truncated, or something.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  [E::sam_hrecs_update_hashes] Header includes @SQ line &amp;quot;KI270330&amp;quot; with no LN: tag&#xA;  samtools view: failed to add PG line to the header&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Anyone seen this before?&lt;/p&gt;&#xA;" OwnerUserId="136" LastEditorUserId="136" LastEditDate="2022-02-23T16:28:19.367" LastActivityDate="2022-02-23T16:28:19.367" Title="BWA mem | samtools view: Intermittent parsing error" Tags="|sequence-alignment|samtools|bwa|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18645" PostTypeId="2" ParentId="8593" CreationDate="2022-02-22T16:18:42.350" Score="2" Body="&lt;p&gt;&lt;strong&gt;high-perf-bio.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I present my &lt;a href=&quot;https://github.com/PlatonB/high-perf-bio&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;high-perf-bio&lt;/em&gt;&lt;/a&gt; project, which, for the most part, solves the problem of fast extraction of anything values from VCF, BED and unformatted data.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Preparation.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;1/ Run the automatic downloading of the toolkit and installation of the dependencies.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;wget https://github.com/PlatonB/high-perf-bio/archive/refs/heads/master.zip &amp;amp;&amp;amp; unzip -o master.zip &amp;amp;&amp;amp; rm master.zip &amp;amp;&amp;amp; cd high-perf-bio-master &amp;amp;&amp;amp; bash __install_3rd_party.sh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;2/ Reboot.&lt;/p&gt;&#xA;&lt;p&gt;3/ &lt;code&gt;cd path/to/high-perf-bio-master&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Create DB with rsIDs.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;python create.py -S path/to/gzipped_tables_dir -i rsIDs_field_path&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Search rsIDs set in DB.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;python annotate.py -S path/to/annotated_gzipped_tables_dir -D db_name -T path/to/trg_dir #-c rsIDs_column_num -f rsIDs_field_path&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;P.S.1. Detailed help for each &lt;em&gt;high-perf-bio&lt;/em&gt; component: &lt;code&gt;python tool_name.py -h&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;P.S.2. For VCF and BED you can specify the minimum of arguments.&lt;/p&gt;&#xA;&lt;p&gt;P.S.3. Readme in English is in the early stages of writing. If you have any questions, write to &lt;a href=&quot;https://github.com/PlatonB/high-perf-bio/issues&quot; rel=&quot;nofollow noreferrer&quot;&gt;Issues&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15096" LastEditorUserId="15096" LastEditDate="2023-03-02T15:01:47.590" LastActivityDate="2023-03-02T15:01:47.590" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18646" PostTypeId="1" CreationDate="2022-02-22T18:13:55.637" Score="2" ViewCount="60" Body="&lt;p&gt;Is there any CA-based online tool/web-server available for secondary structure assignment of proteins except DSSP or STRIDE?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://swift.cmbi.umcn.nl/gv/dssp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;DSSP&lt;/a&gt; is a database of secondary structure assignments (and much more) for all protein entries in the Protein Data Bank (PDB)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://webclu.bio.wzw.tum.de/cgi-bin/stride/stridecgi.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;STRIDE&lt;/a&gt; Protein secondary structure assignment with stride, Basic assignment, Visual assignment, Contact map, Ramachandran Plot&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="3848" LastEditDate="2022-03-22T17:40:55.180" LastActivityDate="2022-03-22T17:40:55.180" Title="Secondary structure assignment tool" Tags="|protein-structure|webservice|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18647" PostTypeId="2" ParentId="10633" CreationDate="2022-02-22T18:40:35.357" Score="0" Body="&lt;p&gt;I should have answered correctly ages back, because I do know it.&lt;/p&gt;&#xA;&lt;p&gt;Cytochrome b is a massively used target for arthropods - these are shrimps. However, the nomenclature changed:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Traditionally nomenclature &lt;code&gt;cytb&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;'New' nomenclature &lt;code&gt;COB&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Wait you might ask, arn't you confusing that with COI, II, III?  No because that nomenclature changed too:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Traditional nomenclature &lt;code&gt;COI&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;'New' nomenclature &lt;code&gt;COX1&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;'New' was a long time ago though.&lt;/p&gt;&#xA;&lt;p&gt;Thus the gene you are looking for is MZ571562, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/MZ571562.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; ... AND COB [gene] &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-02-22T18:40:35.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18648" PostTypeId="2" ParentId="18126" CreationDate="2022-02-22T19:06:24.107" Score="0" Body="&lt;p&gt;My guess by analyzing the definition is that CNA abundance is a proxy for the percentage of (tumor) cells carrying the alteration.&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2022-02-22T19:06:24.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18649" PostTypeId="1" CreationDate="2022-02-22T19:30:42.497" Score="4" ViewCount="46" Body="&lt;p&gt;I have a matrix, sample by exon, containing a copy number value for each pair (sample, exon). I would like to generate a second matrix, sample by gene, where the copy number of the exons is consolidated in a single value per gene.&lt;/p&gt;&#xA;&lt;p&gt;Due to the nature of copy number alteration, I am not sure if this is meaningful, or feasible (e.g., how can it be summarized, in the case a gene is interested by 2 alterations with different copy numbers?).&lt;/p&gt;&#xA;&lt;p&gt;Is there a common approach that is used in this case?&lt;/p&gt;&#xA;&lt;h4&gt;Edit&lt;/h4&gt;&#xA;&lt;p&gt;A possible example is the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;         gene1_exon1  gene1_exon2  gene1_exon3 ... geneN_exonM&#xA;sample1            2            1            1               0&#xA;sample2            2            2            1               0&#xA;...&#xA;sampleJ            3            2            2               2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A possible summarized matrix (with majority voting - not sure if this would be the best approach):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;         gene1 ... geneN&#xA;sample1      1         0&#xA;sample2      2         0&#xA;...&#xA;sampleJ      2         2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I believe this is more an issue related to the biological interpretation than the actual implementation (i.e., I am already implementing a function in &lt;code&gt;pandas&lt;/code&gt;). My concern is on the interpretation, since my prototype select as gene copy number the majority copy number from its exons. However, this has some caveats (e.g., what if you have a GAIN on an exon, and a LOSS on a second exon, for the same gene - will you call GAIN or LOSS?). The big picture is that I want to compare two samples CNAs at the gene level, but I could not find this task discussed online.&lt;/p&gt;&#xA;&lt;p&gt;The main objective of this summarization is to compute gene level concordance between samples. Exon-level or base-level concordance is computed in parallel, in order to have different CNA granularities.&lt;/p&gt;&#xA;&lt;p&gt;However, one might not use the majority voting proposed and, e.g., call a CNA on a gene if the alteration is found in at least p% of exons (e.g., 30% of exons) - this is just an example, I am not suggesting to use this approach.&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="73" LastEditDate="2022-02-23T08:58:06.430" LastActivityDate="2022-02-23T08:58:06.430" Title="How to summarize multiple exon copy numbers into copy number of the corresponding gene" Tags="|gene|cnv|exon|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18650" PostTypeId="2" ParentId="18631" CreationDate="2022-02-22T20:53:00.603" Score="1" Body="&lt;p&gt;The most important thing to do first is batch correction, and it looks like that was done in this instance.&lt;/p&gt;&#xA;&lt;p&gt;The ordering in this case seems reasonable. Normalisation is usually applied to a full dataset (it depends on the assumed model that was used for normalisation, and I'm not familiar with how Partek does things).&lt;/p&gt;&#xA;&lt;p&gt;Filtering after DE should be fine. I'm at least aware that low-count filtering with DESeq2 [cDNASeq rather than microarray] shouldn't influence differential expression results, but it can speed up the calculations slightly:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;While it is not necessary to pre-filter low count genes before running the DESeq2 functions, there are two reasons which make pre-filtering useful: by removing rows in which there are very few reads, we reduce the memory size of the dds data object, and we increase the speed of the transformation and testing functions within DESeq2.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#pre-filtering&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#pre-filtering&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-02-23T21:25:36.360" LastActivityDate="2022-02-23T21:25:36.360" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18652" PostTypeId="2" ParentId="18644" CreationDate="2022-02-23T01:39:25.750" Score="2" Body="&lt;p&gt;I haven't seen that particular error before, but I'd first make sure your pipeline errors aren't being masked by your shell. Consider adding the following to your &lt;code&gt;nextflow.config&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process {&#xA;&#xA;  shell = [ '/bin/bash', '-euo', 'pipefail' ]&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;IIRC, the default shell (as provided by Nextflow) does not include the pipefail option for &lt;a href=&quot;https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#Pipelines&quot; rel=&quot;nofollow noreferrer&quot;&gt;pipelines&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The only other thing I can think of is to make sure your reference FASTA (and BWA index files) are localized in the workDir. You might find the intermittent (filesystem?) errors maybe go away even if you are staging using symlinks.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params.ref_fasta = './bwa_index/ref.fasta'&#xA;&#xA;align_to_ref = Channel&#xA;    .fromPath( [ params.ref_fasta, &amp;quot;${params.ref_fasta}.*&amp;quot; ] )&#xA;    .collect()&#xA;&#xA;&#xA;process bwa_mem_proc {&#xA;&#xA;    tag { &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_name}:$&lt;/span&gt;{fastq.name}&amp;quot; }&#xA;&#xA;    label 'bwa_mem_proc'&#xA;&#xA;    input:&#xA;    tuple val(sample_name), path(fastq), path(header)&#xA;    path align_to_ref&#xA;&#xA;    output:&#xA;    tuple val(sample_name), path(&amp;quot;${fastq.baseName}.unsorted.mini.bam&amp;quot;)&#xA;&#xA;    script:&#xA;    def task_cpus = task.cpus &amp;gt; 1 ? task.cpus - 1 : task.cpus&#xA;&#xA;    if( params.output_format == &amp;quot;bam&amp;quot; ) {&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        bwa mem \\&#xA;            -p \\&#xA;            -t &lt;span class=&quot;math-container&quot;&gt;${task_cpus} \\&#xA;            -M \\&#xA;            -C \\&#xA;            -H &amp;lt;(grep &quot;^@RG&quot; &quot;$&lt;/span&gt;{header}&amp;quot;) \\&#xA;            &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${align_to_ref.first()}&quot; \\&#xA;            &quot;$&lt;/span&gt;{fastq}&amp;quot; |&#xA;        samtools view \\&#xA;            -1 \\&#xA;            -o &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${fastq.baseName}.unsorted.mini.bam&quot; \\&#xA;            -&#xA;        &quot;&quot;&quot;&#xA;    } else {&#xA;        &quot;&quot;&quot;&#xA;        bwa mem \\&#xA;            -p \\&#xA;            -t ${task_cpus} \\&#xA;            -M \\&#xA;            -C \\&#xA;            -H &amp;lt;(grep &quot;^@RG&quot; &quot;${header}&quot;) \\&#xA;            &quot;${align_to_ref.first()}&quot; \\&#xA;            &quot;${fastq}&quot; |&#xA;        samtools view \\&#xA;            -C \\&#xA;            -T &quot;${align_to_ref.first()}&quot; \\&#xA;            -o &quot;$&lt;/span&gt;{fastq.baseName}.unsorted.mini.bam&amp;quot; \\&#xA;            -&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that in the first case (where &lt;code&gt;params.output_format == &amp;quot;bam&amp;quot;&lt;/code&gt;), the reference FASTA file itself is not required by BWA MEM. Only the BWA index files are needed here.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2022-02-23T01:39:25.750" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18654" PostTypeId="2" ParentId="15798" CreationDate="2022-02-23T09:33:29.450" Score="1" Body="&lt;p&gt;The problem is the code itself.&lt;/p&gt;&#xA;&lt;p&gt;I suspect there are R (argenine) and Y (tyrosine) in the sequence data. These are common ambiguities but also represent amino acids. The distance estimate has not accommodated these nucleotide ambiguities but is doing a getter setter for amino acids.&lt;/p&gt;&#xA;&lt;p&gt;It would be easier to perform the same calculation in Mega X than debugging the code.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-02-23T09:33:29.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18655" PostTypeId="2" ParentId="1" CreationDate="2022-02-23T16:01:18.060" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/divonlan/genozip&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genozip&lt;/a&gt; is lossless compression software that has been optimized for genomic data. If &lt;a href=&quot;https://genozip.readthedocs.io/benchmarks.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the benchmarks&lt;/a&gt; are to be believed, Illumina FASTQ files compressed with Genozip are nearly 5× smaller than FASTQ files compressed with gzip. Equally impressive gains over gzip are seen for FASTA, VCF, and GFF3 files. See the publication in &lt;em&gt;Bioinformatics&lt;/em&gt; for more details:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://academic.oup.com/bioinformatics/article/37/16/2225/6135077&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Genozip: a universal extensible genomic data compressor&lt;/strong&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3967" LastActivityDate="2022-02-23T16:01:18.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18656" PostTypeId="1" AcceptedAnswerId="18657" CreationDate="2022-02-23T23:24:21.757" Score="3" ViewCount="1540" Body="&lt;p&gt;&lt;strong&gt;Goal:&lt;/strong&gt; To keep only rows of a GRanges object if their start coordinate is within 1 Mb of the start coordinate of a gene of interest.&lt;/p&gt;&#xA;&lt;p&gt;I am aware of some functions that come close:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;GenomicRanges::nearestKNeighbors()&lt;/code&gt; (and related &lt;code&gt;distanceToNearest()&lt;/code&gt;, &lt;code&gt;nearest()&lt;/code&gt;,&#xA;&lt;code&gt;precede()&lt;/code&gt;, &lt;code&gt;follow()&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;GenomicRanges::subsetByOverlaps()&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;GenomicRanges::window()&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;And I am aware of the UNIX command-line 'Bedtools' tools: &lt;code&gt;window&lt;/code&gt;, &lt;code&gt;closest&lt;/code&gt;, and &lt;code&gt;intersect&lt;/code&gt;, however I wish to accomplish this task using R only and the above functions have not been doing exactly as I would like.&lt;/p&gt;&#xA;&lt;p&gt;Here is an example GRanges dataset:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; gr &amp;lt;- GRanges(&#xA;        seqnames = Rle(c(&amp;quot;chr1&amp;quot;, &amp;quot;chr2&amp;quot;, &amp;quot;chr1&amp;quot;, &amp;quot;chr3&amp;quot;), c(1, 3, 2, 4)),&#xA;        ranges = IRanges(start = seq(300000, 2000000, length=10), end = seq(700000, 6000000, length=10), names = head(letters, 10)),&#xA;        strand = Rle(strand(c(&amp;quot;-&amp;quot;, &amp;quot;+&amp;quot;, &amp;quot;*&amp;quot;, &amp;quot;+&amp;quot;, &amp;quot;-&amp;quot;)), c(1, 2, 2, 3, 2)),&#xA;        score = 1:10, GC = seq(1, 0, length=10), GeneSymbol = c(paste(&amp;quot;gene&amp;quot;, seq(1:10), sep=&amp;quot;&amp;quot;)))&#xA;&#xA;&amp;gt; gr&#xA;GRanges object with 3 ranges and 3 metadata columns:&#xA;    seqnames          ranges strand |     score        GC  GeneSymbol&#xA;       &amp;lt;Rle&amp;gt;       &amp;lt;IRanges&amp;gt;  &amp;lt;Rle&amp;gt; | &amp;lt;integer&amp;gt; &amp;lt;numeric&amp;gt; &amp;lt;character&amp;gt;&#xA;  a     chr1   300000-700000      - |         1  1.000000       gene1&#xA;  b     chr2  488888-1288888      + |         2  0.888889       gene2&#xA;  c     chr2  677777-1877777      + |         3  0.777778       gene3&#xA;  d     chr2  866666-2466666      * |         4  0.666667       gene4&#xA;  e     chr1 1055555-3055555      * |         5  0.555556       gene5&#xA;  f     chr1 1244444-3644444      + |         6  0.444444       gene6&#xA;  g     chr3 1433333-4233333      + |         7  0.333333       gene7&#xA;  h     chr3 1622222-4822222      + |         8  0.222222       gene8&#xA;  i     chr3 1811111-5411111      - |         9  0.111111       gene9&#xA;  j     chr3 2000000-6000000      - |        10  0.000000      gene10&#xA;  -------&#xA;  seqinfo: 3 sequences from an unspecified genome; no seqlengths&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Ideally, by specifying the name (e.g., &lt;code&gt;&amp;quot;gene3&amp;quot;&lt;/code&gt;) or start coordinate (e.g., &lt;code&gt;chr2&lt;/code&gt; &lt;em&gt;AND&lt;/em&gt; &lt;code&gt;677777&lt;/code&gt;) of the gene of interest AND a window size (e.g., &lt;code&gt;1000000&lt;/code&gt; or &lt;code&gt;1 Mb&lt;/code&gt; upstream &lt;em&gt;AND&lt;/em&gt; downstream) to a function, I hope to get an output like below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GRanges object with 3 ranges and 3 metadata columns:&#xA;    seqnames          ranges strand |     score        GC  GeneSymbol&#xA;       &amp;lt;Rle&amp;gt;       &amp;lt;IRanges&amp;gt;  &amp;lt;Rle&amp;gt; | &amp;lt;integer&amp;gt; &amp;lt;numeric&amp;gt; &amp;lt;character&amp;gt;&#xA;  b     chr2  488888-1288888      + |         2  0.888889       gene2&#xA;  c     chr2  677777-1877777      + |         3  0.777778       gene3&#xA;  d     chr2  866666-2466666      * |         4  0.666667       gene4&#xA;  -------&#xA;  seqinfo: 3 sequences from an unspecified genome; no seqlengths&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Notice that only three rows from the original GRanges object remain in this example output because the start coordinates for all other genes were either (i) on different chromosomes or (ii) were &lt;em&gt;more than 1 Mb away&lt;/em&gt; from the start coordinate of gene3.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I have tried various methods to isolate the rows within 1 Mb of my gene of interest with no luck:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; gr1 &amp;lt;- gr[grep(&amp;quot;gene3&amp;quot;, gr$GeneSymbol),]&#xA;&amp;gt; grOut &amp;lt;- subsetByOverlaps(gr, gr1, maxgap = 1000000, invert=T)&#xA;&#xA;&amp;gt; grOut &amp;lt;- gr[mcols(gr)$GeneSymbol == &amp;quot;gene3&amp;quot; &amp;amp; start(gr) &amp;lt; 677777-1000000 &amp;amp; end(gr) &amp;gt; 1877777+1000000]&#xA;&#xA;&amp;gt; grOut &amp;lt;- window(gr, GRanges(&amp;quot;chr2&amp;quot;, IRanges(start=677777-1000000, end=1877777+1000000)))&#xA;Error: 'start' must be a single number or NA&#xA;&#xA;&amp;gt; grOut &amp;lt;- gr[IRanges(start=677777, end=1877777)]&#xA;Error: subscript contains out-of-bounds ranges&#xA;&#xA;&amp;gt; grOut &amp;lt;- nearest(gr, GRanges(&amp;quot;chr2&amp;quot;, IRanges(start=677777-1000000, end=1877777+1000000)), select=&amp;quot;all&amp;quot;, ignore.strand=T)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The last attempt recorded above was likely the closest I've gotten to my desired output by storing the index of rows in the GRanges object falling within the range specified, but it presents this index in the form of a SortedByQueryHits object, not a GRanges object as I require. It seems simple enough a task, but I haven't solved it yet; any thoughts or solutions?&lt;/p&gt;&#xA;" OwnerUserId="9007" LastEditorUserId="9007" LastEditDate="2022-02-24T01:37:03.677" LastActivityDate="2022-03-04T18:31:54.143" Title="How to subset a GRanges object based on a specific genomic window of interest?" Tags="|r|filtering|genomicranges|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18657" PostTypeId="2" ParentId="18656" CreationDate="2022-02-24T01:30:12.287" Score="4" Body="&lt;p&gt;Specifying the range of interest (+/- 1 Mb of start coordinate) as a new GRanges object and then subsetting the original GRanges object based on the overlap of genes with the query works:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(&amp;quot;GenomicRanges&amp;quot;)&#xA;&#xA;&amp;gt; chrPos &amp;lt;- as.character(seqnames(gr))[gr&lt;span class=&quot;math-container&quot;&gt;$GeneSymbol==&quot;gene3&quot;]&#xA;&amp;gt; startPos &amp;lt;- start(ranges(gr))[gr$&lt;/span&gt;GeneSymbol==&amp;quot;gene3&amp;quot;]&#xA;&amp;gt; range &amp;lt;- 1000000&#xA;&#xA;&amp;gt; q &amp;lt;- GRanges(seqnames=chrPos,&#xA;               ranges=IRanges(start=startPos-range, end=startPos+range))&#xA;&amp;gt; grOut &amp;lt;- subsetByOverlaps(gr, q)&#xA;&amp;gt; grOut&#xA;GRanges object with 3 ranges and 3 metadata columns:&#xA;    seqnames         ranges strand |     score        GC  GeneSymbol&#xA;       &amp;lt;Rle&amp;gt;      &amp;lt;IRanges&amp;gt;  &amp;lt;Rle&amp;gt; | &amp;lt;integer&amp;gt; &amp;lt;numeric&amp;gt; &amp;lt;character&amp;gt;&#xA;  b     chr2 488888-1288888      + |         2  0.888889       gene2&#xA;  c     chr2 677777-1877777      + |         3  0.777778       gene3&#xA;  d     chr2 866666-2466666      * |         4  0.666667       gene4&#xA;  -------&#xA;  seqinfo: 3 sequences from an unspecified genome; no seqlengths&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Credit to &lt;a href=&quot;https://www.biostars.org/p/351377/#351383&quot; rel=&quot;nofollow noreferrer&quot;&gt;@zx8754&lt;/a&gt; for inspiring this solution.&lt;/p&gt;&#xA;" OwnerUserId="9007" LastEditorUserId="9007" LastEditDate="2022-03-04T18:31:54.143" LastActivityDate="2022-03-04T18:31:54.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18658" PostTypeId="2" ParentId="14915" CreationDate="2022-02-24T11:23:35.720" Score="2" Body="&lt;p&gt;I wouldn't use Python for this, myself. Instead you can use the FastaToTbl and TblToFasta scripts &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2651/298&quot;&gt;I have posted before&lt;/a&gt;, and pipe to standard *nix utilities:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FastaToTbl file.fa | sort -u | TblToFasta &amp;gt; file.uniq.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or, if you don't have GNU &lt;code&gt;sort&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FastaToTbl file.fa | sort | uniq | TblToFasta &amp;gt; file.uniq.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alternatively, you can also do something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk -v RS='&amp;gt;' -v ORS='&amp;gt;' '++a[$0]==1' file.fa | sed 's/&amp;gt;$//'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For example, given this input file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat file.fa&#xA;&amp;gt;seq1&#xA;ACTTCGCAGAGGCTTCGGAGAGA&#xA;ACTTCGCAGAGGCTTCGGAGAGA&#xA;ACTTCGCAGAGGCTTCGGAGAGA&#xA;&amp;gt;seq2&#xA;ATGGCGCGCTTAGGAGCGCTAGGACT&#xA;&amp;gt;seq2&#xA;ATGGCGCGCTTAGGAGCGCTAGGACT&#xA;&amp;gt;seq1&#xA;ACTTCGCAGAGGCTTCGGAGAGA&#xA;ACTTCGCAGAGGCTTCGGAGAGA&#xA;ACTTCGCAGAGGCTTCGGAGAGA&#xA;&amp;gt;seq1&#xA;ACTTCGCAGAGGCTTCGGAGAGA&#xA;ACTTCGCAGAGGCTTCGGAGAGA&#xA;ACTTCGCAGAGGCTTCGGAGAGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ awk -v RS='&amp;gt;' -v ORS='&amp;gt;' '++a[$0]==1' file.fa | sed 's/&amp;gt;$//'&#xA;&amp;gt;seq1&#xA;ACTTCGCAGAGGCTTCGGAGAGA&#xA;ACTTCGCAGAGGCTTCGGAGAGA&#xA;ACTTCGCAGAGGCTTCGGAGAGA&#xA;&amp;gt;seq2&#xA;ATGGCGCGCTTAGGAGCGCTAGGACT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Both of the above approaches have the advantage of not needing to store the entire file in memory.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2022-02-24T12:09:38.957" LastActivityDate="2022-02-24T12:09:38.957" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18659" PostTypeId="2" ParentId="18637" CreationDate="2022-02-24T19:45:54.637" Score="1" Body="&lt;p&gt;The standard answer would be to search &lt;a href=&quot;https://jaspar.genereg.net/search?q=tet2&amp;amp;collection=CORE&amp;amp;tax_group=vertebrates&amp;amp;tax_id=all&amp;amp;type=all&amp;amp;class=all&amp;amp;family=all&amp;amp;version=all&quot; rel=&quot;nofollow noreferrer&quot;&gt;JASPAR&lt;/a&gt;, to get a curated dataset. However, there does not appear to be anything coming up for TET2 in the database.&lt;/p&gt;&#xA;&lt;p&gt;If your TET2 gene is what you want specifically, you can find &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6442383/&quot; rel=&quot;nofollow noreferrer&quot;&gt;papers&lt;/a&gt; that perform ChIP-seq and infer binding regions. I believe that &lt;a href=&quot;https://genome.cshlp.org/content/suppl/2019/03/15/gr.239277.118.DC1/Supplemental_Table_S3.xlsx&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Supp Table 3&lt;/a&gt; contains information equivalent to a BED file for TET2.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-02-24T19:45:54.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18660" PostTypeId="1" CreationDate="2022-02-24T19:46:28.583" Score="0" ViewCount="125" Body="&lt;p&gt;I have two populations of mice, say RED and GREEN; there are 5 pairs treatment/control in each.&lt;/p&gt;&#xA;&lt;p&gt;The table is the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gene_name  p-value(treatVSctrl) RED1_ctrl RED1_treat RED2_ctrl RED2_treat ... GRN1_ctrl GRN1_treat etc... &#xA;gene1      1.3                  4         10         5         10             6.2       9.75&#xA;gene2      2.8                  3         6          6         12             3         6  &#xA;gene3      4.4                  5         12         4         8              7         10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I need to compare two groups to have an idea of the expression of different genes after treatment.&#xA;I also need to find a difference between RED and GREEN group in terms of genes expression&lt;/p&gt;&#xA;&lt;p&gt;The thing is I cannot compare mice one by one just measuring the fold change, I need to consider p-value. What is the most correct way to perform this analysis?&lt;/p&gt;&#xA;" OwnerUserId="6470" LastActivityDate="2022-02-24T20:45:37.867" Title="What is the best way to compare treated/control samples?" Tags="|differential-expression|p-values|" AnswerCount="1" CommentCount="6" ClosedDate="2022-03-08T00:09:53.493" ContentLicense="CC BY-SA 4.0" />
  <row Id="18661" PostTypeId="2" ParentId="18660" CreationDate="2022-02-24T20:37:09.033" Score="3" Body="&lt;p&gt;I don't really understand the table, particularly the 'p-value' in the header. More generally I don't really understand the experiment nor the data.&lt;/p&gt;&#xA;&lt;p&gt;As a general rule the Wilcoxon would be a reasonable guess of the test using raw data. This is non-parametric version of the T-test without the rigid assumptions of parametrics. Control mice go in one column and subjects in the other. Scipy.stats has the test, its certainly in R (which I don't really use).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print (scipy.stats.wilcoxon(x = [1,2,3,4,5], y=[1,3,5,6,7], zero_method='wilcox', correction=False, alternative='two-sided', mode='auto', *, axis=0, nan_policy='propagate'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There you go, filled out with some example data.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-24T20:45:37.867" LastActivityDate="2022-02-24T20:45:37.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18662" PostTypeId="2" ParentId="18639" CreationDate="2022-02-24T22:09:25.807" Score="2" Body="&lt;p&gt;What these statements refer to is that bulk RNA-seq will overlook the different transcriptional cell types, hence the value of single cell RNA-seq in unraveling heterogeneous cell populations/tissues.&lt;/p&gt;&#xA;&lt;p&gt;I think &amp;quot;total&amp;quot; is also avoided because &lt;a href=&quot;http://core-genomics.blogspot.com/2012/03/what-is-total-rna.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;total RNA&lt;/a&gt; has another meaning related to capturing all the RNA transcripts in a cell (e.g. including non polyA transcripts), which is still not the case for most RNA-seq experiments.&lt;/p&gt;&#xA;&lt;p&gt;Also note that RNA-seq experiments generally (without spikins) are only &lt;em&gt;relative&lt;/em&gt; and not giving &lt;em&gt;absolute&lt;/em&gt; quantifications of the amount of RNA in a cell or groups of cells.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2022-02-24T22:09:25.807" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="18663" PostTypeId="1" CreationDate="2022-02-25T00:30:56.340" Score="1" ViewCount="632" Body="&lt;p&gt;How to do variants selection in some corner cases using GATK and JEXL expressions?&lt;/p&gt;&#xA;&lt;p&gt;I am following the guidelines given in this links for variant selection for some specific cases are not as much detailed.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360036430432-SelectVariants&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360036430432-SelectVariants&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035891011-JEXL-filtering-expressions&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360035891011-JEXL-filtering-expressions&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I just had to do some filtering of my own data and tested several methods (to see what the outcome would be). I am sharing those here for others and myself, if it is ever needed again.&lt;/p&gt;&#xA;" OwnerUserId="1099" LastEditorUserId="73" LastEditDate="2022-02-26T03:48:52.637" LastActivityDate="2022-02-28T14:23:10.657" Title="GATK variant filtration using &quot;SelectVariants&quot; and use of JEXL queries" Tags="|variant-calling|gatk|variant-filtration|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18664" PostTypeId="2" ParentId="18663" CreationDate="2022-02-25T00:30:56.340" Score="1" Body="&lt;p&gt;I am following the guidelines given in this links for variant selection for some specific cases that are not as much detailed in the GATK site.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360036430432-SelectVariants&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360036430432-SelectVariants&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035891011-JEXL-filtering-expressions&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360035891011-JEXL-filtering-expressions&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I just had to do some filtering of my own data and tested several methods (to see what the outcome would be). I am sharing those here for others and myself, if it is ever needed again.&lt;/p&gt;&#xA;&lt;h2&gt;&lt;strong&gt;Tutorials .... !!!&lt;/strong&gt;&lt;/h2&gt;&#xA;&lt;p&gt;&lt;strong&gt;- method to call the sites that are all homozygous Variants (1/1, 2/2 genotype)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar -T SelectVariants -R lyrata_genome.fa -V MY.phased_variants.Final.vcf -o My.filtered_01.vcf -select 'vc.getHomVarCount() == 6'&#xA;# outcome: no sites will have ambigous (./.) and/or homRef (0/0). All other homVar are allowed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;- in the above script if we want the site where at least 4 samples are homozygous ref or variants&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar -T SelectVariants -R lyrata_genome.fa -V MY.phased_variants.Final.vcf -o My.filtered_01.vcf -select 'vc.getHomVarCount() &amp;gt;= 4'&#xA;# outcome: atleast 4 samples are homVar, rest can be any genotype (./., 0/0, 0/1, etc.)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;- at least 1 sample is homVar (1/1, 2/2 or 3/3)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar -T SelectVariants -R lyrata_genome.fa -V MY.phased_variants.Final.vcf -o My.filtered_01.vcf -select 'vc.getHomVarCount() != 0'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;- all the samples are homRef (0/0 genotypes)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar -T SelectVariants -R lyrata_genome.fa -V MY.phased_variants.Final.vcf -o My.filtered_01.vcf -select 'vc.getHomRefCount() == 6'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Note: 6 chr = 3 samples (when samples are diploid)&lt;/h2&gt;&#xA;&lt;p&gt;&lt;strong&gt;- sites that are all HomVar or nocalls (./.)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar -T SelectVariants -R lyrata_genome.fa -V MY.phased_variants.Final.vcf -o My.filtered_01.vcf -select 'vc.getHomVarCount() == vc.getCalledChrCount()/2'&#xA;# outcome: no 0/0, 0/1&#xA;# Note: samples with GT = ./. are not counted as called chr&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Useful after samples are sub-sampled from multisample vcfs&lt;/h2&gt;&#xA;&lt;p&gt;&lt;strong&gt;- sites where no genotypes were called for all the samples in the selected vcf&lt;/strong&gt;&#xA;&lt;strong&gt;-  or say, select vcf sites that are not called in any sample (i.e don't have any genotypes, only contain ./.)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar -T SelectVariants -R lyrata_genome.fa -V MY.phased_variants.Final.vcf -o My.filtered_01.vcf -select 'vc.getCalledChrCount() == 0'&#xA;# outcome = vcf sites (lines) that are all no calls (GT = ./.) on all samples are selected&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;- samples are either homRef (0/0) or no call (./.)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar -T SelectVariants -R lyrata_genome.fa -V MY.phased_variants.Final.vcf -o My.filtered_01.vcf -select 'vc.getHomRefCount() == vc.getCalledChrCount()/2'&#xA;# outcome: vcf sites where all the samples are GT = ./. (no call) or GT = 0/0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;- site where 6 chromosomes (exact) or 3 samples were called for diploid genome&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar /home/everestial007/GenomeAnalysisTK-3.7/GenomeAnalysisTK.jar -T SelectVariants -R lyrata_genome.fa -V MY.phased_variants.Final.vcf -o My.filtered_01.vcf -select 'vc.getCalledChrCount() == 6'&#xA;# Note of Caution: 6 chr = 3 samples (when samples are diploid)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;- site where at least 3 samples (diploid) are called&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar -T SelectVariants -R lyrata_genome.fa -V MY.phased_variants.Final.vcf -o My.filtered_01.vcf -select 'vc.getCalledChrCount() &amp;gt;=6'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;- if I have 6-sample vcf&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar -T SelectVariants -R lyrata_genome.fa -V MY.phased_variants.Final.vcf -o My.filtered_01.vcf -select 'vc.getCalledChrCount() ==12'&#xA;# 12 haploid chr = 6 samples (diploid)&#xA;# outcome: sites that were called in all samples, there will be no GT = ./.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;- select vcf sites where no sample has hetVar (no GT = 0/1, 0/2), only GT = ./., 0/0, 1/1, 2/2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar -T SelectVariants -R lyrata_genome.fa -V MY.phased_variants.Final.vcf -o My.filtered_01.vcf -select 'vc.getHetCount() == 0'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;- select vcf sites where no sample has hetVar (no Gt = 0/1, 0/2), and at least one sample is HomVar (GT = 1/1)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar -T SelectVariants -R lyrata_genome.fa -V MY.phased_variants.Final.vcf -o My.filtered_01.vcf -select 'vc.getHetCount() == 0 &amp;amp;&amp;amp; vc.getHomVarCount() &amp;gt;= 1'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;- site where at least 1 sample is homVar (GT = 1/1 and/or 2/2 etc.)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar -T SelectVariants -R lyrata_genome.fa -V MY.phased_variants.Final.vcf -o My.filtered_01.vcf -select 'vc.getHomVarCount() &amp;gt;= 1'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;- where a site is only homRef for a particular sample and is only SNP&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar SelectVariants -R lyrata_genome.fa -V phased.MySpF1.vcf -sn ms01e -select-type SNP -select 'vc.getGenotype(&amp;quot;ms01e&amp;quot;).isHomRef()' -O test_variants.ms01e.Chr_2_3.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;- where a site is only homVar for a particular sample and is only SNP&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar SelectVariants -R lyrata_genome.fa -V phased.MySpF1.vcf -sn ms01e -select-type SNP -select 'vc.getGenotype(&amp;quot;ms01e&amp;quot;).isHomVar()' -O test_variants.ms01e.Chr_2_3.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;- where a site is not homRef for a particular sample and is only SNP. This will also select empty sites&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar SelectVariants -R lyrata_genome.fa -V phased.MySpF1.vcf -sn ms01e -select-type SNP -select '! vc.getGenotype(&amp;quot;ms01e&amp;quot;).isHomRef()' -O test_variants.ms01e.Chr_2_3.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;NOTE: following genotype call methods won't work:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# to get only heterozygous sites&#xA;-select 'vc.getGenotype(&amp;quot;ms01e&amp;quot;).isHetVar() &#xA;&#xA;# to get any variant sites&#xA;-select 'vc.getGenotype(&amp;quot;ms01e&amp;quot;).isVar() &#xA;&#xA;# won't work if getGenotype method is combined&#xA;-select 'vc.getGenotype(&amp;quot;ms01e&amp;quot;).isHomVar() || vc.getGenotype(&amp;quot;ms01e&amp;quot;.isHomRef()'  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;So, I figured out the following methods where count helps with selecting particular site&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;- where a site is a heterozygous variant site for a particular sample&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar SelectVariants -R lyrata_genome.fa -V phased.MySpF1.vcf -sn ms01e -select 'vc.getHetCount() == 1'  -O test_variants.ms01e.vcf&#xA;&#xA;NOTE: 'vc.getGenotype(&amp;quot;ms01e&amp;quot;).isHetVar()' wont work so generated the above expression.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;- where a site is a variant site (either homVar or hetVar) for a particular sample and is only SNP&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar GenomeAnalysisTK.jar SelectVariants -R lyrata_genome.fa -V phased.MySpF1.vcf -sn ms01e -select-type SNP -select 'vc.getHetCount() == 1 || vc.getHomVarCount() == 1'  -O test_variants.ms01e.vcf&#xA;&#xA;NOTE: 'vc.getGenotype(&amp;quot;NA12878&amp;quot;).isHomRef()' || vc.getGenotype(&amp;quot;NA12878&amp;quot;).isHomVar()' wont work together, so generated this above expression.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Other updates to follow.&#xA;Thanks,&lt;/p&gt;&#xA;" OwnerUserId="1099" LastEditorUserId="1099" LastEditDate="2022-02-28T14:23:10.657" LastActivityDate="2022-02-28T14:23:10.657" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18665" PostTypeId="1" CreationDate="2022-02-25T11:22:59.990" Score="1" ViewCount="26" Body="&lt;p&gt;I have to manually curate some tRNA sequences in order to have their original sequence without modifications. I am getting these sequences from &lt;a href=&quot;http://trna.bioinf.uni-leipzig.de/DataOutput/Welcome&quot; rel=&quot;nofollow noreferrer&quot;&gt;tRNAdb&lt;/a&gt;. In these sequences, I found often &amp;quot;&lt;strong&gt;_&lt;/strong&gt;&amp;quot; symbols.&#xA;They provide this &lt;a href=&quot;http://trna.bioinf.uni-leipzig.de/DataOutput/images/mod.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;list&lt;/a&gt; of symbols and &amp;quot;_&amp;quot; corresponds to &lt;em&gt;insertion&lt;/em&gt;. My question is the insertion of what? Are they referring to a reference sequence? In the &lt;a href=&quot;http://trna.bioinf.uni-leipzig.de/DataOutput/images/mod.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;list&lt;/a&gt; they say that in the footnotes they provide more info. However, there are no footnotes.&#xA;Have you ever had experience with tRNAdb?&lt;/p&gt;&#xA;" OwnerUserId="9142" LastActivityDate="2022-03-09T12:55:42.223" Title="tRNAdb insertion symbols. To what they correspond?" Tags="|trna|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18667" PostTypeId="2" ParentId="18665" CreationDate="2022-02-25T15:43:46.820" Score="3" Body="&lt;p&gt;I have BLASTed one of the tRNA sequences. Lactobacillus acidophilus, Ile, that has insertion sites.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;GGGCCTATAGCTCAGCTGGTTTAGAGC_CACGCCTGATAAGCGTG_GTCGATGGTTCAAGTCCATTTAGGCCCA&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It looks like insertion refers to sites that exist in reference DNA but missing from tRNA sequence. But I am not a tRNA expert, it's my interpretation. See the image.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/eaCZR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eaCZR.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2022-02-25T15:43:46.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18668" PostTypeId="2" ParentId="2649" CreationDate="2022-02-25T18:15:19.853" Score="1" Body="&lt;p&gt;This is an old post I have noticed and there are many offered solutions. Since it’s a frequently asked question, I thought it’s worth for me to mention that there is an overlooked tool set which contains a stand-alone program called &lt;code&gt;faToTab&lt;/code&gt; in addition to many other useful bioinformatics tools.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;faToTab inputFile.fasta outFileFasta_tab.txt&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;It’s a gold-chest in my opinion. Here are the links to utilities folder and details:&#xA;&lt;a href=&quot;http://hgdownload.soe.ucsc.edu/downloads.html#source_downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;Description and Download instructions&lt;/a&gt; -&#xA;&lt;a href=&quot;http://hgdownload.soe.ucsc.edu/admin/exe/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Binaries by machine type&lt;/a&gt; -&#xA;&lt;a href=&quot;https://github.com/ucscGenomeBrowser/kent&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link to the github page&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Anaconda installation is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c bioconda ucsc-fatotab&#xA;conda install -c bioconda/label/cf201901 ucsc-fatotab&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14180" LastEditorUserId="3848" LastEditDate="2022-02-25T18:23:14.580" LastActivityDate="2022-02-25T18:23:14.580" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18669" PostTypeId="2" ParentId="18137" CreationDate="2022-02-25T18:55:53.113" Score="0" Body="&lt;p&gt;Remember not to omit the 'clarification' that in addition to Pango lineages, GISAID clades, Nextstrain clades there is also the WHO nomenclature. We could designate the &lt;code&gt;Bioinformatics StackExchange monophyly&lt;/code&gt;, surely there is room for one more nomenclature?&lt;/p&gt;&#xA;&lt;p&gt;Here's the WHO's response copied and uploaded from &lt;a href=&quot;https://www.who.int/en/activities/tracking-SARS-CoV-2-variants/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ovWqZ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ovWqZ.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Just  to mention NCBI has adopted the Pango lineage nomenclature available on all of its sequences available via &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/labs/virus/vssi/#/virus?SeqType_s=Nucleotide&amp;amp;VirusLineage_ss=Severe%20acute%20respiratory%20syndrome%20coronavirus%202%20(SARS-CoV-2),%20taxid:2697049&amp;amp;CollectionDate_dr=1950-01-01T00:00:00Z%20TO%20NOW&amp;amp;CreateDate_dt=1950-01-01T00:00:00Z%20TO%20NOW&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I thought that Omicron was Pango lineage B12, or perhaps B.1.2? Oh I see it has been reclassified following further sequence data.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-25T19:29:07.387" LastActivityDate="2022-02-25T19:29:07.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18670" PostTypeId="2" ParentId="16348" CreationDate="2022-02-25T19:36:28.183" Score="0" Body="&lt;p&gt;The NCBI link to the metadata database is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/labs/virus/vssi/#/virus?SeqType_s=Nucleotide&amp;amp;VirusLineage_ss=Severe%20acute%20respiratory%20syndrome%20coronavirus%202%20(SARS-CoV-2),%20taxid:2697049&amp;amp;CollectionDate_dr=1950-01-01T00:00:00Z%20TO%20NOW&amp;amp;CreateDate_dt=1950-01-01T00:00:00Z%20TO%20NOW&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If I get a moment I'll talk you through it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-02-25T19:36:28.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18671" PostTypeId="2" ParentId="2649" CreationDate="2022-02-25T19:38:34.227" Score="1" Body="&lt;p&gt;In python I would do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#...Suppose you have header information and &#xA;#...sequences stored in lists&#xA;&#xA;header_info1 = [elements]&#xA;header_info2 = [elements]&#xA;sequences = [sequences]&#xA;index = enumerate(sequences)&#xA;table = open(pathtoyourfile.tsv,'w+')&#xA;for h1,h2,s,i in zip(header_info1,header_info2,sequences,index):&#xA;    table.write(f&amp;quot;&amp;gt;{h1}\t{h2}\t{s}\n&amp;quot;)&#xA;    if i+1==len(sequences):&#xA;       table.write(f&amp;quot;&amp;gt;{h1}\t{h2}\t{s}&amp;quot;)&#xA;table.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So basically I use f strings and I iterate over these three vectors that are of the same length. At the end of the iteration, I remove new line(\n) since it won't be needed anymore since you have not to write anything further.&lt;/p&gt;&#xA;" OwnerUserId="9142" LastEditorUserId="9142" LastEditDate="2022-02-25T20:39:14.490" LastActivityDate="2022-02-25T20:39:14.490" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18672" PostTypeId="1" CreationDate="2022-02-26T11:46:15.360" Score="1" ViewCount="404" Body="&lt;p&gt;What is the command in PyMOL for listing all the amino acids in a specific protein, say, 1a62.pdb?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastActivityDate="2022-02-28T01:40:05.180" Title="How can I list all amino acids in a protein using PyMOL?" Tags="|pdb|pymol|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18673" PostTypeId="2" ParentId="18672" CreationDate="2022-02-26T13:24:04.450" Score="2" Body="&lt;p&gt;At the command, change the current directory to your target directory:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cd my-directory-path-containing-my-files&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Load the file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;load 1a62.pdb&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then you can either print to screen or save to a file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print(cmd.get_fastastr('all'))&#xA;&#xA;save 1a62.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14180" LastEditorUserId="14180" LastEditDate="2022-02-28T01:40:05.180" LastActivityDate="2022-02-28T01:40:05.180" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18674" PostTypeId="1" CreationDate="2022-02-26T14:53:59.147" Score="3" ViewCount="633" Body="&lt;h2&gt;Bug fixed&lt;/h2&gt;&#xA;&lt;p&gt;This issue appears to have been fixed in latest &lt;code&gt;Nextflow&lt;/code&gt;: &lt;a href=&quot;https://github.com/nextflow-io/nextflow/issues/2693&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nextflow-io/nextflow/issues/2693&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I've been using &lt;code&gt;Nextflow&lt;/code&gt; for various workflows on a cluster using the &lt;code&gt;SLURM&lt;/code&gt; scheduler. I've noticed that the 'parent' job does not shut down when the &lt;code&gt;Nextflow&lt;/code&gt; workflow finishes successfully. I can cancel the job, but then &lt;code&gt;Nextflow&lt;/code&gt; doesn't get a chance to clean things up and also doesn't write out the &lt;code&gt;trace&lt;/code&gt;, &lt;code&gt;report&lt;/code&gt;, etc. The parent job does end properly if the workflow ends with an error.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;: Why doesn't the parent &lt;code&gt;Nextflow&lt;/code&gt; job end upon successful completion?&lt;/p&gt;&#xA;&lt;p&gt;I saw &lt;a href=&quot;https://github.com/nextflow-io/nextflow/issues/968&quot; rel=&quot;nofollow noreferrer&quot;&gt;a post on the &lt;code&gt;Nextflow&lt;/code&gt; GitHub&lt;/a&gt; where the original poster describes various acrobatics to get the parent job to exit cleanly, but this is something that &lt;code&gt;Nextflow&lt;/code&gt; should handle.&lt;/p&gt;&#xA;&lt;p&gt;Here's my &lt;code&gt;SLURM&lt;/code&gt; submission script that I submit using &lt;code&gt;sbatch run_workflow.slurm&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;run_workflow.slurm&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;#SBATCH --time=60:15:00                             # Time limit for the job (REQUIRED).&#xA;#SBATCH --job-name=NF_Parent_Identify_D&amp;amp;C_regions   # Job name&#xA;#SBATCH --ntasks=1                                  # Number of cores for the job. Same as SBATCH&#xA;#SBATCH --mem=5G                                    # Total memory requested&#xA;#SBATCH --partition=&amp;lt;partition&amp;gt;                     # Partition/queue to run the job in. (REQUIRED&#xA;#SBATCH -e slurm-%j.err                             # Error file for this job.&#xA;#SBATCH -o slurm-%j.out                             # Output file for this job.&#xA;#SBATCH -A &amp;lt;account&amp;gt;                                # Project allocation account name (REQUIRED)&#xA;&#xA;module load ccs/java/jdk1.8.0_202&#xA;nextflow run identify_dark_and_camouflaged_regions.nf \&#xA;    -with-report \&#xA;    -with-trace \&#xA;    -with-dag idcr-workflow.png \&#xA;    -with-timeline \&#xA;    -resume&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h1&gt;Update 1&lt;/h1&gt;&#xA;&lt;p&gt;I'm adding example &lt;code&gt;Nextflow&lt;/code&gt; output from a different workflow (I don't have the &lt;code&gt;SLURM&lt;/code&gt; output from the one I mention above) where the workflow has completed, but the parent job is still running. In this case, there were errors, but I added &lt;code&gt;errorStrategy 'ignore'&lt;/code&gt; to all of the processes listed:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;executor &amp;gt;  slurm (876)&#xA;[06/c5672b] process &amp;gt; samtools_csort_proc (A-CUHS... [100%] 304 of 304, faile...&#xA;[45/26a046] process &amp;gt; Lsamtools_csort_proc (A-CUH... [100%] 304 of 304, faile...&#xA;[b5/0f2d7c] process &amp;gt; sambamba_csort_proc (A-CUHS... [100%] 304 of 304, faile...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, regardless of whether the workflow completed &lt;code&gt;successfully&lt;/code&gt;, &lt;code&gt;Nextflow&lt;/code&gt; has nothing left to do, but it's still running.&lt;/p&gt;&#xA;&lt;h1&gt;Update 2 (adding partial &lt;code&gt;.nextflow.log&lt;/code&gt; file)&lt;/h1&gt;&#xA;&lt;p&gt;Per @Steve's point, I checked the &lt;code&gt;.nextflow.log&lt;/code&gt; file and am including part of the file, here. In this case, I don't think &lt;code&gt;Nextflow&lt;/code&gt; has any tasks related to &lt;code&gt;publishDir&lt;/code&gt; because I'm not using that in this workflow. The job that's still running is from the &lt;code&gt;Nextflow&lt;/code&gt; workflow in my GitHub repository for my answer posted in my related question &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/18538/samtools-sort-most-efficient-memory-and-thread-settings-for-many-samples-on-a-c&quot;&gt;Samtools sort: most efficient memory and thread settings for many samples on a cluster&lt;/a&gt;. Whether it's reproducible, I haven't checked. Not sure how common this issue is.&lt;/p&gt;&#xA;&lt;p&gt;In either case, my interpretation of the &lt;code&gt;.nextflow.log&lt;/code&gt; file below is that &lt;code&gt;Nextflow&lt;/code&gt; thinks it's hung, too. I just don't know why, or how to exit cleanly.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Partial view of &lt;code&gt;.nextflow.log&lt;/code&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;&#xA;Feb-28 05:29:04.607 [Task monitor] DEBUG n.processor.TaskPollingMonitor - !! executor slurm &amp;gt; tasks to be completed: 1 -- submitted tasks are shown below&#xA;~&amp;gt; TaskHandler[jobId: 662274; id: 876; name: sambamba_csort_proc (A-CUHS-CU003128-BL-COL-49696BL1.unsorted;CPUs:2;MEM_PER_THREAD:100GB); status: RUNNING; exit: -; error: -; workDir: /path/to/Samtools_sort_optimization_test/SORT_COMPARISON/work/b5/0f2d7c5d7f81a6e62dedb08bf6694a started: 1646038174334; exited: -; ]&#xA;Feb-28 05:30:54.613 [Task monitor] DEBUG n.processor.TaskPollingMonitor - Task completed &amp;gt; TaskHandler[jobId: 662274; id: 876; name: sambamba_csort_proc (A-CUHS-CU003128-BL-COL-49696BL1.unsorted;CPUs:2;MEM_PER_THREAD:100GB); status: COMPLETED; exit: 0; error: -; workDir: /path/to/Samtools_sort_optimization_test/SORT_COMPARISON/work/b5/0f2d7c5d7f81a6e62dedb08bf6694a started: 1646038174334; exited: 2022-02-28T10:30:53.516015Z; ]&#xA;Feb-28 05:34:04.641 [Task monitor] DEBUG n.processor.TaskPollingMonitor - No more task to compute -- Execution may be stalled&#xA;Feb-28 05:39:04.647 [Task monitor] DEBUG n.processor.TaskPollingMonitor - No more task to compute -- Execution may be stalled&#xA;Feb-28 05:44:04.653 [Task monitor] DEBUG n.processor.TaskPollingMonitor - No more task to compute -- Execution may be stalled&#xA;Feb-28 05:49:04.658 [Task monitor] DEBUG n.processor.TaskPollingMonitor - No more task to compute -- Execution may be stalled&#xA;Feb-28 05:54:04.662 [Task monitor] DEBUG n.processor.TaskPollingMonitor - No more task to compute -- Execution may be stalled&#xA;&#xA;&amp;lt; above line repeats hundreds of times &amp;gt;&#xA;&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="136" LastEditorUserId="136" LastEditDate="2022-05-16T08:57:42.217" LastActivityDate="2022-05-16T08:57:42.217" Title="Nextflow scheduler job doesn't end upon successful workflow completion" Tags="|nextflow|slurm|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18675" PostTypeId="2" ParentId="16376" CreationDate="2022-02-26T20:06:17.740" Score="0" Body="&lt;p&gt;I don't know Theta but Tajima D is a population genetics test that assumes constant sample size. When the number of segregating sites (non-conservative sites in an alignment) significantly differs from pi (Nei's p-distance between a pair of sequence) one possibility is selection given a constant population size.&lt;/p&gt;&#xA;&lt;p&gt;What Tajima found was cool: under neutrality this relationship conforms to a beta distribution. Just to explain that again so the nucleotide differences between every pair of sequences in an alignment (pi) against the total number of possible mutations conforms to a beta distribution for a constant population size under neutrality.&lt;/p&gt;&#xA;&lt;p&gt;Its a cool idea - like really cool - but the violations - any level of 'extinction' will skew the distribution, population movement and importantly so could a bias in sampling.&lt;/p&gt;&#xA;&lt;p&gt;So what you get is a pair-wise matrix at 5% critical probability. If your data entirely conformed to Tajima's D it would be an amazing result, but that is not the result you are looking for. So if Tajima D gave a particular pair &amp;lt;0.05  AND it was supported by other tests THEN (sounds like code) it provides a basis to consider selection as an explanation.&lt;/p&gt;&#xA;&lt;p&gt;Again I don't know Theta but if both tests were congruent for putative selection I'd say that was reasonable. However, Tajima D alone is not a cool result, its cool theory however but not cool in practice. I hope that explains it?&lt;/p&gt;&#xA;&lt;p&gt;Note: did any one ever follow this up in trees? Cool if they did because there will be a relationship between the balance of a tree (e.g. Bayesian 'Skyline' plots) and Tajima D.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-26T21:46:51.807" LastActivityDate="2022-02-26T21:46:51.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18677" PostTypeId="2" ParentId="18674" CreationDate="2022-02-27T14:00:35.317" Score="2" Body="&lt;p&gt;How do you know your workflow actually finishes successfully? One way to check the workflow termination status is to use the &lt;a href=&quot;https://www.nextflow.io/docs/latest/metadata.html#completion-handler&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;workflow.onComplete&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;https://www.nextflow.io/docs/latest/metadata.html#error-handler&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;workflow.onError&lt;/code&gt;&lt;/a&gt; event handlers. Consider resuming your workflow with something like the following appended to your script:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow.onComplete {&#xA;&#xA;  log.info &amp;quot;Workflow completed at: &lt;span class=&quot;math-container&quot;&gt;${workflow.complete}&quot;&#xA;  log.info &quot;Time taken: $&lt;/span&gt;{workflow.duration}&amp;quot;&#xA;  log.info &amp;quot;Execution status: ${workflow.success ? 'success' : 'failed'}&amp;quot;&#xA;}&#xA;&#xA;workflow.onError {&#xA;&#xA;  log.info &amp;quot;Execution halted: ${workflow.errorMessage}&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that the workflow event handlers may also be defined &lt;a href=&quot;https://www.nextflow.io/docs/latest/metadata.html#decoupling-metadata&quot; rel=&quot;nofollow noreferrer&quot;&gt;inside&lt;/a&gt; your &lt;code&gt;nextflow.config&lt;/code&gt; if preferred.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2022-02-27T14:00:35.317" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="18678" PostTypeId="1" AcceptedAnswerId="18679" CreationDate="2022-02-27T20:41:39.597" Score="2" ViewCount="322" Body="&lt;p&gt;I have downloaded two files:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;File1&lt;/strong&gt;.csv from:&#xA;&lt;a href=&quot;https://depmap.org/portal/download/data_slicer/download?file_path=20220227%2F3c335b7d-c18e-4788-b2e6-512d71f0a339%2Fexport.csv&amp;amp;name=Metabolomics.csv&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://depmap.org/portal/download/data_slicer/download?file_path=20220227%2F3c335b7d-c18e-4788-b2e6-512d71f0a339%2Fexport.csv&amp;amp;name=Metabolomics.csv&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;File 1&lt;/strong&gt; looks like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;,2-aminoadipate,3-phosphoglycerate,alpha-glycerophosphate&#xA;ACH-000698,6.1127272,6.034197900000001,5.8968963&#xA;ACH-000489,5.5774126,5.7270453,5.1114679,6.07325&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;File2&lt;/strong&gt;.csv from:&#xA;enter link description here&#xA;&lt;a href=&quot;https://cog.sanger.ac.uk/cmp/download/model_list_20220205.csv&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cog.sanger.ac.uk/cmp/download/model_list_20220205.csv&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;File2 looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; model_id,model_name,synonyms,model_type&#xA; SIDM01774,PK-59,,Cell Line,Adherent&#xA; SIDM00192,SNU-1033,,Cell Line,Adherent&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to replace ACH-* ids with SIDM* ids in File1.csv. The ACH-* id's for SIDM* ids are given in the &amp;quot;BROAD_ID&amp;quot; (col.no =40) column in the File2.csv&lt;/p&gt;&#xA;&lt;p&gt;Desired output&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;,2-aminoadipate,3-phosphoglycerate,alpha-glycerophosphate&#xA;SIDM00523,6.1127272,6.034197900000001,5.8968963&#xA;SIDM00835,5.5774126,5.7270453,5.1114679,6.07325&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to do this with python script. I tried some initial steps. Please forgive any grave mistake, as I am a beginner in Python&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; import pandas as pd&#xA;&#xA; df = pd.read_csv(&amp;quot;File1.csv&amp;quot;, index_col=0).T&#xA;&#xA; print(df)&#xA;&#xA; models = pd.read_csv(&amp;quot;File2.csv&amp;quot;, index_col=&amp;quot;model_id&amp;quot;, usecols=[&amp;quot;model_id&amp;quot;, &amp;quot;BROAD_ID&amp;quot;]).rename(columns={&amp;quot;model_id&amp;quot;: &amp;quot;Model&amp;quot;})&#xA;&#xA; df = df.join(models).rename_axis(index=&amp;quot;Model Name&amp;quot;).reset_index()&#xA;&#xA; print(df)&#xA;&#xA; df2 = df.set_index(&amp;quot;Model&amp;quot;).drop(columns=&amp;quot;Model Name&amp;quot;)&#xA;&#xA; df2.to_csv(&amp;quot;Meta.tsv&amp;quot;, sep=&amp;quot;\t&amp;quot;)&#xA;&#xA; df2.index = df2.index + &amp;quot;.Demeter&amp;quot;&#xA; df2.T.rename_axis(index=&amp;quot;Gene&amp;quot;).to_csv(&amp;quot;Meta_2.tsv&amp;quot;, sep=&amp;quot;\t&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4783" LastActivityDate="2022-03-02T18:08:01.473" Title="Replacing column 1 in file1 with matching ID's in file 2" Tags="|python|data-download|data-preprocessing|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18679" PostTypeId="2" ParentId="18678" CreationDate="2022-02-27T23:19:21.717" Score="4" Body="&lt;p&gt;I was not sure about the exact output (columns) you desired. The following code should inner merge both data frames. It writes a copy of 'model_list' column as a first column. You could remove any unwanted columns. I am sure there are more elegant ways of doing it. Hope it helps. It's totally different approach than yours.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;metabolics = pd.read_csv(&amp;quot;Metabolomics.csv&amp;quot;, sep=&amp;quot;,&amp;quot;, header=0) &#xA;model_list = pd.read_csv(&amp;quot;model_list_20220205.csv&amp;quot;, sep=&amp;quot;,&amp;quot;, header=0) &#xA;metabolics=metabolics.rename(columns={'Unnamed: 0':'ACH_number'}) #rename the unnamed column.&#xA;merged_df = pd.merge(metabolics, model_list, left_on='ACH_number', right_on='BROAD_ID', how='inner') #join two files&#xA;&#xA;#insert model_id column at the beginning&#xA;#You will have dupicae 'model_id' column&#xA;merged_df.insert(0, &amp;quot;model_id&amp;quot;, merged_df['model_id'], allow_duplicates=True) &#xA;merged_df.to_csv('results.csv', sep=&amp;quot;,&amp;quot;, index=False)&#xA;print(merged_df.shape)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2022-02-27T23:19:21.717" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="18680" PostTypeId="2" ParentId="18678" CreationDate="2022-02-28T04:10:05.423" Score="2" Body="&lt;p&gt;The basic strategy of &lt;code&gt;merge&lt;/code&gt; 'left' (which I think is default) is the correct thing to do. However, I would be concerned by lack of a key between the two data sets and basically the merging will take place on the row order. Within &lt;code&gt;merge&lt;/code&gt; is &lt;code&gt;on=&lt;/code&gt; then the shared &lt;code&gt;key&lt;/code&gt;, in the code by @supertech there is left-on='ACH_number', I dunno because &lt;code&gt;ACH_number&lt;/code&gt; does not appear to be shared. The code needs testing.&lt;/p&gt;&#xA;&lt;p&gt;If you assign an index it will attempt to merge on the index (which I think will therefore fail). What I think you want is the row order 1,2,3,4 in each dataframe to be the key that is merged. Therefore not specifying &lt;code&gt;key&lt;/code&gt; might be the right thing to do, what can happen is it will then use the row order as the left-key. I cannot remember.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;insert&lt;/code&gt; is correct to reorder the columns to get the desired output.&lt;/p&gt;&#xA;&lt;p&gt;Finally you want,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;del merged_df['column_name']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;for each column you don't want. There is a way to loop through this, but the above keeps things simple. You then do the standard csv dump.&lt;/p&gt;&#xA;&lt;p&gt;Note, see comments on &lt;code&gt;drop&lt;/code&gt; by @GOATNine, which strongly preferred.&lt;/p&gt;&#xA;&lt;p&gt;Setting and unsetting indexes is key to manipulating pandas but it is within a specific contex.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; the command &lt;code&gt;merge&lt;/code&gt; and &lt;code&gt;insert&lt;/code&gt; are correct with the addition of &lt;code&gt;del&lt;/code&gt; however, the code needs testing regarding merge &lt;code&gt;left-on&lt;/code&gt; and assigning the index. It is much easier to print the output and visualise it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-28T17:01:29.700" LastActivityDate="2022-02-28T17:01:29.700" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="18681" PostTypeId="1" CreationDate="2022-02-28T12:30:47.320" Score="3" ViewCount="40" Body="&lt;p&gt;I have 3 SNPs that are in LD in 'several' populations here:&#xA;&lt;a href=&quot;https://www.ebi.ac.uk/gwas/variants/rs13078854#ld_panel_label&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/gwas/variants/rs13078854#ld_panel_label&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It's a pain to look at all (30) populations one by one to manually check if the 3 SNPs are in LD in a given population or not.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to ask this question more systematically? e.g. in which common imputation populations are a given set of SNPs in LD (above a certain threshold)?&lt;/p&gt;&#xA;" OwnerUserId="8875" LastActivityDate="2022-02-28T14:15:34.183" Title="SNPs in LD in which populations?" Tags="|snp|imputation|ld|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18683" PostTypeId="2" ParentId="18681" CreationDate="2022-02-28T13:22:42.317" Score="2" Body="&lt;p&gt;There is a risk of a type 2 error.&lt;/p&gt;&#xA;&lt;p&gt;My advice would be to use &lt;code&gt;Structure&lt;/code&gt; if it is still available given the situation. Its not great and it is old and its a GUI and I don't know whether it has been updated (I'm not exactly selling this am I?), but it will do the job of navigating through a type 2 error.&lt;/p&gt;&#xA;&lt;p&gt;What you do is provide the number of populations &lt;em&gt;a priori&lt;/em&gt; and it will assign a Bayesian estimate of HW equilibrium for that number of populations. You manually input 1,2,3,4,.... 30 run and wait. The point at which the joint marginal plateaus is the minimum number of populations which do not exhibit LD (thus if two of your populations exhibit LD it collapse them into one). Thus if your answer is '20' it give some confidence it is a real result. If it is like 10 (optimistically) the conclusion would be in pretty reasonable shape.&lt;/p&gt;&#xA;&lt;p&gt;There are other ways of looking at this, but that would be my starting point.&lt;/p&gt;&#xA;&lt;p&gt;The classic calculation is LDHat, here &lt;a href=&quot;https://github.com/auton1/LDhat&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/auton1/LDhat&lt;/a&gt;, but the last update was 2015 and in this particular circumstance I'm not sure.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-28T14:15:34.183" LastActivityDate="2022-02-28T14:15:34.183" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18684" PostTypeId="2" ParentId="15162" CreationDate="2022-02-28T14:42:33.053" Score="1" Body="&lt;p&gt;Thanks for raising the issue, it is certainly topical. The question is whether the data set they unit tested is reliable.&lt;/p&gt;&#xA;&lt;p&gt;I think the issue they are trying get to is mosaic artefacts in the sequencing process and assembly thereon. This would be particularly true in a situation of &lt;em&gt;de novo&lt;/em&gt; assembly which is what the authors are targeting. There are some slick methodologies for virus sequencing, which have been around for a while now, so in any case a 'gold-standard' would be outdated at present.&lt;/p&gt;&#xA;&lt;p&gt;I do have to agree with you that the language particularly for the journal certainly could have phrased better. The data set was 2014 and genome circularisation (its a virus small genomes) to remove the otherwise high SNP error rate would have been around in 2017 when this was published.&lt;/p&gt;&#xA;&lt;p&gt;I fully understand why the authors choose this language. A professional editor, not affiliated with the journal (without up to date technical insight), would likely recommend the same language.&lt;/p&gt;&#xA;&lt;p&gt;I guess the question would be whether verification is achieved on other virus data sets. There is no shortage of them at present and its certainly an interesting tool.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-02-28T15:09:13.157" LastActivityDate="2022-02-28T15:09:13.157" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18686" PostTypeId="2" ParentId="18642" CreationDate="2022-02-28T18:12:21.523" Score="2" Body="&lt;p&gt;You can get the data from the API of the European Nucleotide Archive (ENA) which is synched with NCBI/GenBank:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ebi.ac.uk/ena/browser/api/fasta/MZ039713,MZ039682,OV522182&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/ena/browser/api/fasta/MZ039713,MZ039682,OV522182&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10114" LastActivityDate="2022-02-28T18:12:21.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18687" PostTypeId="1" AcceptedAnswerId="18688" CreationDate="2022-02-28T18:52:47.427" Score="4" ViewCount="404" Body="&lt;p&gt;When I tried to use samtools to split a bam file based on different chromosomes, I used this command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view input.bam -b chr21 | chr21.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, I get error messages like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-bash: chr21.bam: command not found&#xA;[W::hts_idx_load3] The index file is older than the data file: input.bam.bai&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How could I solve this problem? Thanks in advance~~~&lt;/p&gt;&#xA;&lt;p&gt;The version I am using is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Program: samtools (Tools for alignments in the SAM format)&#xA;Version: 1.10 (using htslib 1.10.2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14102" LastActivityDate="2022-02-28T19:05:58.300" Title="samtools view command not found error" Tags="|sam|samtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18688" PostTypeId="2" ParentId="18687" CreationDate="2022-02-28T19:05:58.300" Score="5" Body="&lt;p&gt;Try re-indexing your bam file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools index input.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or update the timestamp via &lt;code&gt;touch index.bam.bai&lt;/code&gt; but I think re-indexing is safer. You might want to save any original index file first, e.g. via &lt;code&gt;mv input.bam.bai old_input.bam.bai&lt;/code&gt; or similar&lt;/p&gt;&#xA;&lt;p&gt;Also I think you meant &lt;code&gt;&amp;gt; chr21.bam&lt;/code&gt; instead of piping via &lt;code&gt;| chr21.bam&lt;/code&gt; if you want to save the output file&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2022-02-28T19:05:58.300" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18689" PostTypeId="1" CreationDate="2022-02-28T19:43:23.047" Score="1" ViewCount="469" Body="&lt;p&gt;When I trying to use bcftools mpileup to convert bam to vcf files,&#xA;I try to use:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools mpileup -Ob -o resources-broad-hg38-v0-Homo_sapiens_assembly38.fasta.fai input.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, I got this error message:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[E::fai_build3_core] Failed to open the file resources-broad-hg38-v0-Homo_sapiens_assembly38.fasta.fai&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or, I would get another error message when I use another ref file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[E::fai_build_core] Format error, unexpected &amp;quot;c&amp;quot; at line 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does anyone know how to solve these problems? Thanks in advance&lt;/p&gt;&#xA;&lt;p&gt;Also, could anyone recommend some human gene Reference? The one I am using now is from GATK pipeline (but I don't know if that is good enough).&lt;/p&gt;&#xA;&lt;p&gt;The version I am using is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Program: bcftools (Tools for variant calling and manipulating VCFs and BCFs)&#xA;License: GNU GPLv3+, due to use of the GNU Scientific Library&#xA;Version: 1.10.2 (using htslib 1.10.2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14102" LastEditorUserId="73" LastEditDate="2022-02-28T21:56:22.603" LastActivityDate="2022-02-28T22:05:06.423" Title="bcftools mpileup &quot;Failed to open the file&quot; questions" Tags="|bcftools|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18690" PostTypeId="1" AcceptedAnswerId="18693" CreationDate="2022-02-28T19:46:38.163" Score="5" ViewCount="413" Body="&lt;p&gt;I am currently implementing some metrics I could use for comparing two conformations of the same protein in Python.&lt;/p&gt;&#xA;&lt;p&gt;For example, I know I could use the RMSD of all protein ATOMS in BioPandas using this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;biopandas.pdb.PandasPdb.rmsd(conformation1.df['ATOM'], conformation2.df['ATOM'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Are there some out-of-the-box tools in Python for measuring the RMSD between the side chains of the two conformations? When trying to do this manually, the main issue is how to identify side chain atoms, which is not clear for me. I know how to get all atoms from a residue using BioPandas:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;aminoacids = list(PandasPdb().read_pdb(my_pdb_filename).df['ATOM'].groupby(['chain_id', 'residue_number', 'residue_name']))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now &lt;code&gt;aminoacids[i]&lt;/code&gt; is a dataframe containing all the atoms of the i-th aminoacid, like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9RnK2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9RnK2.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;From here, it's not clear how to separate back bone atoms from side chain atoms. It's clear that the one with atom_number 'CA' will be the alpha-Carbon, but I am not sure how to identify the others. But once the separation is done, the side chain RMSD could be measure for example using &lt;code&gt;Bio.SVDSuperimposer.SVDSuperimposer&lt;/code&gt; which works directly on the array of 3D coordinates of the atoms.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="12036" LastEditorUserId="10049" LastEditDate="2022-03-01T12:01:30.973" LastActivityDate="2022-03-01T12:44:55.907" Title="Identify side chain atoms in BioPandas dataframe" Tags="|python|proteins|protein-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18691" PostTypeId="2" ParentId="18689" CreationDate="2022-02-28T22:05:06.423" Score="2" Body="&lt;p&gt;Regarding a human genome reference, have a look at CHM13/HG002 from the Telomere-to-Telomere consortium:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/marbl/CHM13#assembly-releases&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/marbl/CHM13#assembly-releases&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The benefit of the T2T assembly is that it is fully-assembled across all repetitive regions, which improves the mapping accuracy. The pangenome consortium is working on additional T2T assemblies from other people, trying to capture as much human [structural] variation as possible.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://humanpangenome.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://humanpangenome.org/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The v2.0 assembly, adding chrY from HG002, has only just been released. From the looks of it, it is not yet annotated, but there's gene annotation for all other chromosomes via the v1.1 release:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/marbl/CHM13#downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/marbl/CHM13#downloads&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-02-28T22:05:06.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18692" PostTypeId="1" CreationDate="2022-03-01T12:17:36.667" Score="-1" ViewCount="862" Body="&lt;p&gt;I have a heatmap&lt;/p&gt;&#xA;&lt;p&gt;I see a block of genes for a group of samples&lt;/p&gt;&#xA;&lt;p&gt;How I can select a part of heatmap?&lt;/p&gt;&#xA;&lt;p&gt;I mean is there any way to export a list of genes from rows of a heatmap in R?&lt;a href=&quot;https://i.stack.imgur.com/yTNPp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yTNPp.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have tried which returns nothing&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; library(ComplexHeatmap)&#xA;&#xA;&amp;gt; ht = Heatmap(heat1) &#xA;&amp;gt; ht[1:35,1:5]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2022-03-01T12:44:30.023" LastActivityDate="2022-03-01T15:05:09.247" Title="Selecting a part of heatmap" Tags="|r|heatmap|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18693" PostTypeId="2" ParentId="18690" CreationDate="2022-03-01T12:18:35.537" Score="3" Body="&lt;p&gt;As you suggest one way of solving your problem would be by selecting all atoms that don't have backbone atoms names. In a pdb file I believe backbone atoms would be named 'CA', 'HA', 'N', 'HN' or 'H', 'C' and 'O'. Beware of the N-terminal (where the hydrogens would be named either 'H1', 'H2' &amp;amp; 'H3'; or 'HN1', 'HN2' and 'HN3') and the C-terminal (the other oxygen would be named 'OXT').&lt;/p&gt;&#xA;&lt;p&gt;One way of doing it is by using the handy selections from pandas dataframes. It's probably a more elegant way than making a list of dataframes.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from biopandas.pdb import PandasPdb&#xA;&#xA;bpdf_1 = PandasPdb().fetch_pdb('1t48').df['ATOM']&#xA;bpdf_2 = PandasPdb().fetch_pdb('1t49').df['ATOM']&#xA;&#xA;# Function returns a list of boolean. True when it's not a backbone atom.&#xA;sel = lambda df: df['atom_name'].str.contains('[^(CA|HA|N|C|O|HN|H)]$')&#xA;&#xA;# The number of atom is not the same, so I only align the first 200&#xA;sidechain_rmsd = PandasPdb.rmsd(bpdf_1[sel][:200], bpdf_2[sel][:200])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The regex says something like &amp;quot;(not (CA or HA or …)) end of line&amp;quot;. The end of line is important, otherwise any atom containing N, C, O or H in their name would be excluded (so most of them).&lt;/p&gt;&#xA;&lt;p&gt;Note that with this method it's still possible to use the PandasPdb selectors. For example, to only consider heavy atoms (no hydrogen):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;sidechain_rmsd = PandasPdb.rmsd(bpdf_1[sel][:200],&#xA;                                bpdf_2[sel][:200],&#xA;                                s='heavy')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Another way to do it would be:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;sidechain_rmsd = PandasPdb.rmsd(bpdf_1[:200],&#xA;                                bpdf_2[:200],&#xA;                                s='main chain',&#xA;                                invert=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You don't need any selection in this case, &lt;strong&gt;BUT&lt;/strong&gt; this is a bad idea if you have hydrogens, as it will also align them (including the backbone ones).&lt;/p&gt;&#xA;" OwnerUserId="12054" LastEditorUserId="12054" LastEditDate="2022-03-01T12:44:55.907" LastActivityDate="2022-03-01T12:44:55.907" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18694" PostTypeId="2" ParentId="18692" CreationDate="2022-03-01T15:05:09.247" Score="3" Body="&lt;p&gt;You're using ComplexHeatmap, which is pretty easy to subset. Your subset operation creates a subset object of the heatmap, not of the data in the heatmap.&lt;/p&gt;&#xA;&lt;p&gt;In case you don't want to run hclust outside of the CH, you can use &lt;code&gt;row_order&lt;/code&gt; and &lt;code&gt;column_order&lt;/code&gt; to re-order your data in the CH display order.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(ComplexHeatmap)&#xA;&#xA;ht = Heatmap(heat1)&#xA;r_o &amp;lt;- row_order(ht)&#xA;c_o &amp;lt;- column_order(ht)&#xA;heat1_reordered &amp;lt;- heat1[r_o, c_o]&#xA;heat1_reordered[1:35,1:5]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;CH does say that clustering is oly &amp;quot;fixed&amp;quot; on &lt;code&gt;draw&lt;/code&gt;ing, so you may want to run this before picking the row and column orders:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pdf(NULL)&#xA;ht &amp;lt;- draw(ht)&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastActivityDate="2022-03-01T15:05:09.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18695" PostTypeId="1" CreationDate="2022-03-01T23:45:39.460" Score="2" ViewCount="185" Body="&lt;p&gt;I want to assess ploidy estimation (i.e., diploid vs tetraploid) in a set of samples for which I have both results from the variant caller (i.e., SNVs) and from the copy number caller.&lt;/p&gt;&#xA;&lt;p&gt;What are the possible approaches that can be used in this case? I know that in some in cases it might be possible to use VAF to assess ploidy estimation, but I haven't found exactly how this (possibly simple) approach is defined.&lt;/p&gt;&#xA;&lt;p&gt;Note: I already have a caller that I am testing. My question is more related to exploratory data analysis or manual inspection of the caller's results.&lt;/p&gt;&#xA;&lt;p&gt;VAF distribution examples might be similar to what is done in &lt;a href=&quot;https://caravagnalab.github.io/CNAqc/articles/a7_example_MSeq.html#peak-detection&quot; rel=&quot;nofollow noreferrer&quot;&gt;CNAqc&lt;/a&gt; (image below extracted from their online tutorial):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/nEjvZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nEjvZ.png&quot; alt=&quot;VAF distribution&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2022-03-03T18:39:15.890" LastActivityDate="2022-03-03T18:39:15.890" Title="Assess ploidy estimation through SNV and CNV results" Tags="|vcf|snp|variant-calling|cnv|allele-frequency|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18698" PostTypeId="2" ParentId="16087" CreationDate="2022-03-02T21:39:51.300" Score="1" Body="&lt;p&gt;You'll want to use the 10x provided bamtofastq tool to preserve the indices properly: &lt;a href=&quot;https://github.com/10XGenomics/bamtofastq&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/10XGenomics/bamtofastq&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="74" LastActivityDate="2022-03-02T21:39:51.300" CommentCount="0" CommunityOwnedDate="2022-03-02T21:39:51.300" ContentLicense="CC BY-SA 4.0" />
  <row Id="18699" PostTypeId="1" AcceptedAnswerId="18700" CreationDate="2022-03-02T23:39:34.160" Score="0" ViewCount="302" Body="&lt;p&gt;(VAF = &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4850126/#__sec8title&quot; rel=&quot;nofollow noreferrer&quot;&gt;variant allele frequency&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;This question follows from &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/18695/1771&quot;&gt;this one&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;What is the expected range of values for VAF of mutations with heterozygous genotype (i.e., 0/1 or 0|1)?&#xA;Is it possible to have 0% and 100% (respectively having allele-specific depth of 0 in ALT or REF, with non-zero depth in the other allele)?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="2108" LastEditDate="2022-03-03T04:29:25.013" LastActivityDate="2022-03-03T04:32:35.063" Title="Range of values for VAF in heterozygous genotype" Tags="|vcf|variant-calling|allele-frequency|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18700" PostTypeId="2" ParentId="18699" CreationDate="2022-03-03T04:32:35.063" Score="1" Body="&lt;p&gt;From the link that I edited into your question (realized that the answer was &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4850126/#__sec8title&quot; rel=&quot;nofollow noreferrer&quot;&gt;there&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;VAF is the percentage of sequence reads observed matching a specific DNA variant divided by the overall coverage at that locus. Because NGS provides a near random sample, VAF is thus a surrogate measure of the proportion of DNA molecules in the original specimen carrying the variant. For constitutional genetic testing, VAF is a measure of diploid zygosity: heterozygous loci should be near 50% VAF, homozygous loci should be near 100%, and reference loci should be near 0%. Deviations from these three expected values should be considered suspicious as potential errors due to incorrect base calls or alignment. For somatic testing, contamination from normal cells and tumor heterogeneity combine to cause unpredictable VAFs. In some clinical scenarios, such as testing a patient for therapeutic resistance mutations, the desired sensitivity dictates that variants with low VAF be included in downstream analysis. This all means that in order to maintain an acceptable level of sensitivity, variant filtering for somatic variant must be highly permissive from a technical validity perspective.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is assuming diploid humans. For other ploidies the expected proportions of heterozygosity will match the possible heterozygous genotypes (e.g. 33%/67% for triploid heterozygotes, 25%/75% for tetraploid, etc.).&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-03-03T04:32:35.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18701" PostTypeId="2" ParentId="18257" CreationDate="2022-03-03T04:55:50.797" Score="2" Body="&lt;p&gt;The simple answer is that Cox PH is a flavor of &lt;a href=&quot;https://towardsdatascience.com/understanding-multiple-regression-249b16bde83e&quot; rel=&quot;nofollow noreferrer&quot;&gt;multiple regressio&lt;/a&gt;n. Multiple regression does indeed try to subtract off the estimated effect of each covariate and look for associations of all other independent variables. Whether this works well in practice is anyone's guess. For a somewhat philosophical high-level discussion see &lt;a href=&quot;https://www.edge.org/response-detail/25387&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;One notable issue is that &amp;quot;significant&amp;quot; is &lt;a href=&quot;https://retractionwatch.com/2019/03/21/time-to-say-goodbye-to-statistically-significant-and-embrace-uncertainty-say-statisticians/#:%7E:text=In%20an%20editorial%20in%20a,it%20and%20don%27t%20use&quot; rel=&quot;nofollow noreferrer&quot;&gt;not necessarily a great way to think about statistics&lt;/a&gt;, according to professional statistical societies. So, like, if a p-value goes up or down a little bit that's kind of expected when you adjust for other variables, it's not really notable. For one thing, it's closely related to questions of &lt;a href=&quot;https://machinelearningmastery.com/statistical-power-and-power-analysis-in-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;power&lt;/a&gt;, driven largely by your sample size.&lt;/p&gt;&#xA;&lt;p&gt;In simple terms, I would urge you to put all of your data in the model, find some set of independent variables and covariates that you think makes sense &lt;em&gt;before you start analyzing the data&lt;/em&gt;, and then build your model accordingly. I don't think that anyone objects to e.g. plotting raw data or model residuals separately in males and females to demonstrate some point, but most associations that are worth caring about will be obvious in a combined dataset.&lt;/p&gt;&#xA;&lt;p&gt;You &lt;em&gt;can&lt;/em&gt; try to explicitly &lt;a href=&quot;http://www.utstat.toronto.edu/%7Ebrunner/oldclass/312s19/lectures/312s19PropHazRegWithR2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;test for an interaction effect in the model&lt;/a&gt;. This can be interesting if there's some reason to expect it to be the case, or if there is very compelling evidence that there exists some interaction, but it has the negative of adding to model complexity and further reducing power.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-03-03T04:55:50.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18702" PostTypeId="1" CreationDate="2022-03-03T09:07:49.883" Score="1" ViewCount="25" Body="&lt;p&gt;I have a conceptual question.&lt;/p&gt;&#xA;&lt;p&gt;I have a diploid, outcrossing plant genome assembly of ~1.1 Gb size. The original assembly is generated from PacBio reads. After genome annotation with the &lt;code&gt;Mercator&lt;/code&gt; software, the original assembly returned a gene count of &lt;code&gt;59,615&lt;/code&gt;. Then, the diploid genome assembly was phased with Hifiasm+Hi-C data. There, after running the annotation on two haplotype assemblies, the gene count was &lt;code&gt;75,936&lt;/code&gt; for &lt;code&gt;haplotype 1&lt;/code&gt; and &lt;code&gt;67,233&lt;/code&gt; for &lt;code&gt;haplotype 2&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Then, we sent the pacbio assembly together with the haplotypes to Dovetail genomics for scaffolding. After the scaffolding with HiRise and annotation with Mercator, I got the following gene count for the three assemblies:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Scaffolded_assembly- 75,874 &#xA;Scaffolded_haplotype_1- 67,243 &#xA;Scaffolded_haplotype_2- 86,465&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can anyone please shed some light on why there is such an improvement in gene count? Also, the gene count for haplotypes seems to be the opposite after scaffolding. Do you have any ideas? I do not know how many of those are high confidence or low confidence gene or splice variants.&lt;/p&gt;&#xA;&lt;p&gt;The BUSCO score was similar in all the cases.&lt;/p&gt;&#xA;&lt;p&gt;It would be nice if you can give some feedback or any paper explaining such phenomena. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="12238" LastEditorUserId="3694" LastEditDate="2022-03-03T11:56:10.057" LastActivityDate="2022-03-03T11:56:10.057" Title="Why does gene count increase drastically after scaffolding with Hi-C data?" Tags="|genome|sequence-annotation|pacbio|haplotypes|hi-c|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18703" PostTypeId="1" CreationDate="2022-03-03T11:09:07.490" Score="0" ViewCount="391" Body="&lt;p&gt;I have some marmoset snRNA reads that I want to align with the reference transcriptome using cellranger. The primary assembly for marmoset is available here, which is broken down into 22 parts. However, &lt;code&gt;cellranger mkref&lt;/code&gt; only accepts one &lt;code&gt;.fa&lt;/code&gt; file to generate the transcriptome.&lt;/p&gt;&#xA;&lt;p&gt;I tried concatentaing all the extracted &lt;code&gt;.fa&lt;/code&gt; files into one (After removing the first line which is a comment), but I get the following error while trying to make a reference transcriptome using cellranger:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;Creating new reference folder at /data/users/abhishek/ensembl/common_marmoset_ref/combined/common_marmoset&#xA; ...done&#xA; &#xA; Writing genome FASTA file into reference folder...&#xA; ...done&#xA; &#xA; Indexing genome FASTA file...&#xA; Traceback (most recent call last):&#xA;   File &amp;quot;/data/public-data/cellranger-6.1.1/bin/rna/mkref&amp;quot;, line 137, in &amp;lt;module&amp;gt;&#xA;     main()&#xA;   File &amp;quot;/data/public-data/cellranger-6.1.1/bin/rna/mkref&amp;quot;, line 128, in main&#xA;     reference_builder.build_gex_reference()&#xA;   File &amp;quot;/data/public-data/cellranger-6.1.1/lib/python/cellranger/reference.py&amp;quot;, line 630, in build_gex_reference&#xA;     self.process_fasta()&#xA;   File &amp;quot;/data/public-data/cellranger-6.1.1/lib/python/cellranger/reference.py&amp;quot;, line 529, in process_fasta&#xA;     subprocess.check_call([&amp;quot;samtools&amp;quot;, &amp;quot;faidx&amp;quot;, self.fasta_path])&#xA;   File &amp;quot;/data/public-data/cellranger-6.1.1/external/anaconda/lib/python3.8/subprocess.py&amp;quot;, line 364, in check_call&#xA;     raise CalledProcessError(retcode, cmd)&#xA; subprocess.CalledProcessError: Command '['samtools', 'faidx', '/data/users/abhishek/ensembl/common_marmoset_ref/combined/common_marmoset/fasta/genome.fa']' returned non-zero exit     status 1.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There are also files which contain only one or two characters. I suspect that these aren't being handled properly with the direct concatenation of all the files. Some examples are below:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;==&amp;gt; Callithrix_jacchus.mCalJac1.pat.X.dna.primary_assembly.1.fa &amp;lt;==&#xA;CCCATTATCCCTGCTTCTGCTAATCTTTCAAAATCTCACCACCATAGTC&#xA;&#xA;==&amp;gt; Callithrix_jacchus.mCalJac1.pat.X.dna.primary_assembly.20.fa &amp;lt;==&#xA;TTTTT&#xA;&#xA;==&amp;gt; Callithrix_jacchus.mCalJac1.pat.X.dna.primary_assembly.21.fa &amp;lt;==&#xA;TT&#xA;&#xA;==&amp;gt; Callithrix_jacchus.mCalJac1.pat.X.dna.primary_assembly.22.fa &amp;lt;==&#xA;AAAATTGTTTCTTGTTT&#xA;&#xA;==&amp;gt; Callithrix_jacchus.mCalJac1.pat.X.dna.primary_assembly.2.fa &amp;lt;==&#xA;C&#xA;&#xA;==&amp;gt; Callithrix_jacchus.mCalJac1.pat.X.dna.primary_assembly.3.fa &amp;lt;==&#xA;TTAGGGTTAGGGTTAGGGTTAGGGTTAGGGTTAGGG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How do I combine all these &lt;code&gt;.fa&lt;/code&gt; files to generate the reference transcriptome for downstream analysis?&lt;/p&gt;&#xA;" OwnerUserId="15143" LastActivityDate="2022-03-03T11:09:07.490" Title="How to combine multiple .fasta files of primary assembly from Ensembl into one for sequence alignment?" Tags="|transcriptome|reference-genome|ensembl|10x-genomics|cellranger|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18704" PostTypeId="1" AcceptedAnswerId="18713" CreationDate="2022-03-03T14:50:13.783" Score="0" ViewCount="30" Body="&lt;p&gt;I'm working with the CCLE dataset and I'm trying to subset the data to just 3 cell lines of interest but not sure how to it.&lt;/p&gt;&#xA;&lt;p&gt;Currently, I have been subsetting one by one but is there an easier way?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ccle_panc &amp;lt;- ccle_dataset$data[ , ccle_dataset$colAnnotation$SAMPLE_NAME==&amp;quot;KP4&amp;quot;, &amp;quot;&amp;quot;, &amp;quot;&amp;quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6827" LastActivityDate="2022-03-04T11:16:54.463" Title="subsetting more than one ccle sample" Tags="|r|rna-seq|subset|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18706" PostTypeId="1" AcceptedAnswerId="18711" CreationDate="2022-03-03T16:44:30.813" Score="1" ViewCount="1513" Body="&lt;p&gt;I have a bed file that contains a list of regions and I would like to extract only the exon regions from a GTF/GFF file that are contained in the given regions.&lt;/p&gt;&#xA;&lt;p&gt;Is there a simple command or tool that I can use for that?&lt;/p&gt;&#xA;" OwnerUserId="2594" LastActivityDate="2022-03-03T22:59:15.857" Title="Extract only exon regions from GFF/GTF file with input bed regions" Tags="|bed|gtf|exon|gff|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18707" PostTypeId="2" ParentId="18257" CreationDate="2022-03-03T17:06:46.930" Score="1" Body="&lt;p&gt;&amp;quot;Under the hood&amp;quot; a Cox model doesn't stratify, consistent with your intuition. It iteratively solves a &lt;a href=&quot;https://stats.stackexchange.com/a/560234/28500&quot;&gt;partial-likelihood vector score function&lt;/a&gt; as a function of regression-coefficient values.&lt;/p&gt;&#xA;&lt;p&gt;It tries to minimize the sum, over all event times, of the difference between each covariate value for the case having the event and a risk-weighted average of the values for all at risk at that event time. The &amp;quot;risk-weighting&amp;quot; is a function of all current regression-coefficient estimates within an iteration. Iteration continues until regression-coefficient estimates are effectively constant. This nicely handles predictors of all types you might use in a multiple-regression model, including interaction terms, while also allowing for individuals that drop out of the study before experiencing the event.&lt;/p&gt;&#xA;&lt;p&gt;There thus is nothing to be gained by doing separate models for different groups, as Maximilian Press also says in another answer. Stratified models are often used if the proportional hazards assumption doesn't hold, but those model all the data together and often are structured to provide the same regression-coefficient estimates for all strata while allowing for different baseline survival curves among strata.&lt;/p&gt;&#xA;&lt;p&gt;I also recommend including as many predictors as possible that you might expect to be associated with outcome in your model, but with one caveat. You risk overfitting if you have fewer than 10-20 &lt;em&gt;events&lt;/em&gt; per unpenalized predictor in your model (including levels beyond the first of categorical predictors and any interaction or nonlinear terms). So you might have to make some careful choices or use a penalized approach like ridge regression.&lt;/p&gt;&#xA;&lt;p&gt;For further information, there are nearly 900 answered questions about Cox models on the &lt;a href=&quot;https://stats.stackexchange.com/questions/tagged/cox-model?tab=Active&quot;&gt;Cross Validated site&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="10035" LastActivityDate="2022-03-03T17:06:46.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18709" PostTypeId="2" ParentId="18642" CreationDate="2022-03-03T18:18:16.070" Score="2" Body="&lt;p&gt;You can try the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets/docs/v1/reference-docs/rest-api/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Datasets API&lt;/a&gt;. We have an endpoint that would allow you to download virus sequences by &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets/docs/v1/reference-docs/rest-api/#get-/virus/accession/-accessions-/genome/download&quot; rel=&quot;nofollow noreferrer&quot;&gt;accession&lt;/a&gt;  The only thing to be aware (based on your list of accessions) is that our API needs the accession version to return results. In the case of your examples, they would need to be MZ039713.1, MZ039682.1 and OV522182.1.&#xA;I hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="15139" LastActivityDate="2022-03-03T18:18:16.070" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18710" PostTypeId="1" CreationDate="2022-03-03T18:27:50.473" Score="1" ViewCount="58" Body="&lt;p&gt;everyone!&lt;/p&gt;&#xA;&lt;p&gt;I'm working with haplotype inference algorithms in pedigree genotype data, and I would like to know if there is a tool or script to calculate switch errors on phased pedigree data (Plink formatted)?&lt;/p&gt;&#xA;&lt;p&gt;I already know that VCFtools and other scripts do exist, but they only work with VCF format.&lt;/p&gt;&#xA;&lt;p&gt;Note: I tried to convert my phased pedigree data to VCF but I lost phase, so, this approach does not seem to be helpful, that's why I'm looking for a specific tool to detect switches on phased pedigree data.&lt;/p&gt;&#xA;&lt;p&gt;A switch errors ocurr when heterozygous positions are misassigned relative to the previous heterozygous position.&lt;/p&gt;&#xA;&lt;p&gt;See visual example (block 2), True haplotypes up, inferred haplotypes bottom:&#xA;&lt;a href=&quot;https://www.researchgate.net/publication/310766263/figure/fig13/AS:431688409980932@1479933945501/Illustration-of-switch-errors-and-block-breakpoints-In-haplotype-block-2-there-are-two.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.researchgate.net/publication/310766263/figure/fig13/AS:431688409980932@1479933945501/Illustration-of-switch-errors-and-block-breakpoints-In-haplotype-block-2-there-are-two.png&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In my case, I got the phased pedigree plink file as follows:&lt;/p&gt;&#xA;&lt;p&gt;Infered Haplotype, individual 1:&#xA;1|2 2|1 1|2 1|2 1|2&lt;/p&gt;&#xA;&lt;p&gt;Real Haplotype, individual 1:&#xA;1|2 1|2 2|1 1|2 1|2&lt;/p&gt;&#xA;&lt;p&gt;We can see 2 switches, between position 1-2 and 3-4.&lt;/p&gt;&#xA;&lt;p&gt;Thank you very much in advance!&#xA;Regards!&lt;/p&gt;&#xA;" OwnerUserId="12647" LastEditorUserId="12647" LastEditDate="2022-03-08T18:17:37.877" LastActivityDate="2022-03-08T18:17:37.877" Title="How to calculate switch errors on plink formatted phased genotype data?" Tags="|errors|haplotypes|phasing|pedigree|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18711" PostTypeId="2" ParentId="18706" CreationDate="2022-03-03T20:36:30.463" Score="5" Body="&lt;p&gt;You could use &lt;a href=&quot;https://github.com/bedops/bedops&quot; rel=&quot;noreferrer&quot;&gt;BEDOPS&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ gtf2bed &amp;lt; annotations.gtf | grep -wF exon &amp;gt; exons.bed&#xA;$ sort-bed myRegions.unsorted.bed &amp;gt; myRegions.bed&#xA;$ bedops --element-of 100% exons.bed myRegions.bed &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or as a one-liner, which is even faster:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ bedops --element-of 100% &amp;lt;(gtf2bed &amp;lt; annotations.gtf | grep -wF exon) &amp;lt;(sort-bed myRegions.unsorted.bed) &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;--element-of 100%&lt;/code&gt; argument requires that the exon fall entirely within a region in the &lt;code&gt;myRegions&lt;/code&gt; set. This is the most stringent requirement. The most relaxed requirement would be one base of overlap: &lt;code&gt;--element-of 1&lt;/code&gt;. You can adjust to either extreme, or to values in between, depending on your criteria.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2022-03-03T20:36:30.463" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18712" PostTypeId="2" ParentId="18706" CreationDate="2022-03-03T22:59:15.857" Score="2" Body="&lt;p&gt;Bedtools &lt;code&gt;intersect&lt;/code&gt; will do this:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;By far, the most common question asked of two sets of genomic features is whether or not any of the features in the two sets “overlap” with one another. This is known as feature intersection. bedtools intersect allows one to screen for overlaps between two sets of genomic features. Moreover, it allows one to have fine control as to how the intersections are reported. bedtools intersect works with both BED/GFF/VCF and BAM files as input.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/intersect.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedtools.readthedocs.io/en/latest/content/tools/intersect.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-03-03T22:59:15.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18713" PostTypeId="2" ParentId="18704" CreationDate="2022-03-03T23:03:54.887" Score="0" Body="&lt;p&gt;Assuming you're trying to write R code, array references can use boolean values, and &lt;code&gt;in&lt;/code&gt; is useful when you want to select from more than one value:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ccle_panc &amp;lt;- ccle_dataset&lt;span class=&quot;math-container&quot;&gt;$data[ , ccle_dataset$&lt;/span&gt;colAnnotation$SAMPLE_NAME %in% c(&amp;quot;KP4&amp;quot;, &amp;quot;DC1&amp;quot;, &amp;quot;DC2&amp;quot;)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can find more information about vector and array subsetting with boolean and numerical values here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://datacarpentry.org/R-ecology-lesson/01-intro-to-r.html#Subsetting_vectors&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://datacarpentry.org/R-ecology-lesson/01-intro-to-r.html#Subsetting_vectors&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-03-03T23:03:54.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18714" PostTypeId="1" AcceptedAnswerId="18718" CreationDate="2022-03-04T06:25:23.500" Score="2" ViewCount="237" Body="&lt;p&gt;I want to map ~500,000 UniProt IDs to their representative UniRef90 IDs.&lt;/p&gt;&#xA;&lt;p&gt;What is the most efficient way of doing this?&lt;/p&gt;&#xA;&lt;p&gt;Do we have a table with the mapping?&lt;/p&gt;&#xA;" OwnerUserId="14169" LastActivityDate="2022-03-05T10:24:27.867" Title="Efficient way of mapping UniProt IDs to representative UniRef90 IDs?" Tags="|uniprot|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18717" PostTypeId="2" ParentId="18704" CreationDate="2022-03-04T11:16:54.463" Score="0" Body="&lt;p&gt;Assuming that you have a dataframe/tibble situation; you can also use the tidyverse approach and the comment from @gringer:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ccle_dataset$data %&amp;gt;%&#xA;filter(SAMPLE_NAME %in% c(&amp;quot;KP4&amp;quot;, &amp;quot;DC1&amp;quot;, &amp;quot;DC2&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11884" LastActivityDate="2022-03-04T11:16:54.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18718" PostTypeId="2" ParentId="18714" CreationDate="2022-03-04T12:13:03.623" Score="2" Body="&lt;p&gt;You can do this directly on UniProt: &lt;a href=&quot;https://www.uniprot.org/uploadlists/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uploadlists/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Just paste or upload your list of UniProt IDs, and select &amp;quot;UniProtKB AC/ID&amp;quot; in the &amp;quot;From&amp;quot; field and &amp;quot;UniParc&amp;quot; in the &amp;quot;To&amp;quot; field&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Wu1o0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Wu1o0.png&quot; alt=&quot;screenshot of the relevant Uniprot.org page&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I've also written a script, pasted below, that can do this with some useful options:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;$ uniprot_map.pl -h&#xA;&#xA;uniprot_map.pl will connect to UniProt and map IDs from the input file to ehatever ID was requested.&#xA;&#xA;USAGE:&#xA;       uniprot_map.pl [options] &amp;lt;ID_LIST&amp;gt;&#xA;&#xA;OPTIONS:&#xA;      -h : Print this help and exit.&#xA;      -f : From ID type. This is the type of ID found in the input file.&#xA;      -F : Format. See http://www.uniprot.org/faq/28#batch_retrieval_of_entries for details. &#xA;&#xA;      -l : When a requested ID maps to many output IDs, print only the first (probably the best) match.&#xA;      -L : List all recognised ID types.&#xA;      -u : Print unreviewed (trEMBL) prots as well. By default, if an ID maps to SWISSPROT and trEMBL IDs &#xA;           only the SWISSPROTs will be printed, use this flag to print trEMBL as well.&#xA;      -t : To ID. This is the ID type that the input IDs will be mapped to.&#xA;      -v : Verbose. Print progress messages to STDERR.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you save the script as &lt;code&gt;uniprot_map.pl&lt;/code&gt; in a directory in your PATH and make it executable, then you can use a file like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat file&#xA;P31946&#xA;P62258&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And feed it to the script with the appropriate options to get the mapping:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;$ uniprot_map.pl -f ID -t UniParc file&#xA;P31946  UPI000013C714&#xA;P62258  UPI0000021A46&#xA;2 of 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The script:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-perl prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/perl&#xA;use strict;&#xA;use warnings;&#xA;use LWP::UserAgent;&#xA;use Getopt::Std;&#xA;my %opts;&#xA;getopts('Lsvuhf:F:t:l', \%opts);&#xA;my &lt;span class=&quot;math-container&quot;&gt;$base = 'http://www.uniprot.org';&#xA;my $&lt;/span&gt;verbose=&lt;span class=&quot;math-container&quot;&gt;$opts{v}||undef;&#xA;my $&lt;/span&gt;tool = 'mapping';&#xA;my (@NAMES,@names);&#xA;my &lt;span class=&quot;math-container&quot;&gt;$print_only_first=$&lt;/span&gt;opts{l}||undef;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$allow_trembl=$&lt;/span&gt;opts{u}||undef;&#xA;list_ids() if &lt;span class=&quot;math-container&quot;&gt;$opts{L};&#xA;&amp;amp;usage() if $&lt;/span&gt;opts{h};&#xA;&#xA;&amp;amp;usage() unless $ARGV[0];&#xA;&#xA;####################################&#xA;# Collect the IDs we want to match #&#xA;####################################&#xA;foreach my &lt;span class=&quot;math-container&quot;&gt;$ar(@ARGV){&#xA;if (-e $ar){&#xA;open(A, $ar);&#xA;while(&amp;lt;A&amp;gt;){&#xA;    chomp;&#xA;    push @names,$_;&#xA;}&#xA;}&#xA;else{push @names,$&lt;/span&gt;ar};&#xA;}&#xA;&#xA;######################################&#xA;# What are we matching FROM and TO ? #&#xA;######################################&#xA;my &lt;span class=&quot;math-container&quot;&gt;$from=$&lt;/span&gt;opts{f}||'ID';&#xA;my &lt;span class=&quot;math-container&quot;&gt;$to = $&lt;/span&gt;opts{t}||'ACC';&#xA;&#xA;if(&lt;span class=&quot;math-container&quot;&gt;$opts{t}){&#xA;$to=translate_terms($to);&#xA;}&#xA;if($&lt;/span&gt;opts{f}){&#xA;    &lt;span class=&quot;math-container&quot;&gt;$from=translate_terms($&lt;/span&gt;from);&#xA;}&#xA;#################&#xA;# Output format #&#xA;#################&#xA;my &lt;span class=&quot;math-container&quot;&gt;$format = $&lt;/span&gt;opts{F}||'tab';&#xA;#my @query=&#xA;&#xA;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$agent = LWP::UserAgent-&amp;gt;new;&#xA;push @{$&lt;/span&gt;agent-&amp;gt;requests_redirectable}, 'POST';&#xA;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$params= {&#xA;'from' =&amp;gt; $from,&#xA;'to' =&amp;gt; $to,&#xA;'format' =&amp;gt; $&lt;/span&gt;format,&#xA;    'query' =&amp;gt; &amp;quot;@names&amp;quot;&#xA;};&#xA;&#xA;print STDERR &amp;quot;Requesting...\t&amp;quot; if &lt;span class=&quot;math-container&quot;&gt;$verbose;&#xA;my $&lt;/span&gt;try_again=1;&#xA;my &lt;span class=&quot;math-container&quot;&gt;$response;&#xA;while($&lt;/span&gt;try_again==1){&#xA;    &lt;span class=&quot;math-container&quot;&gt;$response = get_response($&lt;/span&gt;params);&#xA;    if(defined(&lt;span class=&quot;math-container&quot;&gt;$response-&amp;gt;is_success)){$try_again=0}&#xA;elsif($&lt;/span&gt;verbose){print STDERR &amp;quot;Failed, retrying...\n&amp;quot;; }&#xA;}&#xA;&#xA;if( &lt;span class=&quot;math-container&quot;&gt;$print_only_first) &#xA;{&#xA;if($response-&amp;gt;is_success){&#xA;my %k;&#xA;my @a=split(/\n/,$response-&amp;gt;content);&#xA;foreach my $line(@a){&#xA;    next if $line=~/From/;&#xA;    my @b=split(/\t/,$line);&#xA;    next if(defined($k{$b[0]}));&#xA;    $k{$b[0]}=$b[1];&#xA;}&#xA;map{print STDOUT &quot;$_\t$k{$_}\n&quot;}keys(%k);&#xA;}&#xA;else{&#xA;die 'Failed, got ' . $response-&amp;gt;status_line . ' for ' . $response-&amp;gt;request-&amp;gt;uri . &quot;\n&quot;;&#xA;}&#xA;}&#xA;else{&#xA;if($response-&amp;gt;is_success){ &#xA;my %map;&#xA;my @a=split(/\n/,$response-&amp;gt;content);&#xA;#############################&#xA;    # remove &quot;From   To&quot;   line #&#xA;    #############################&#xA;shift(@a) if $a[0] =~/^From/i;&#xA;my %kk;&#xA;map{&#xA;    ########################################################&#xA;        # UniProt sometimes returns the SAME line twice    #&#xA;        ########################################################&#xA;    $kk{$_}++;&#xA;    #print STDOUT &quot;$_\n&quot; unless $kk{$_}&amp;gt;1; &#xA;    next if $kk{$_}&amp;gt;1; &#xA;    my @b=split(/\t/);&#xA;    push @{$map{$b[0]}},$&lt;/span&gt;b[1];&#xA;    }@a;&#xA;&#xA;    foreach my &lt;span class=&quot;math-container&quot;&gt;$query (keys(%map)) {&#xA;    my %sw;&#xA;    foreach my $target (@{$map{$query}}) {&#xA;    #############################################&#xA;            # Does this prot have SWISSPROT IDs?        #&#xA;            #############################################&#xA;    if ($to eq 'ID')&#xA;    {&#xA;        my @b=split(/_/,$target);&#xA;        $sw{$target}++ if length($b[0])!=6;&#xA;    }&#xA;    }&#xA;    foreach my $target (@{$map{$query}}) {&#xA;    ################################################&#xA;    # If a protein has a least one SWISSPROT ID    #&#xA;    # skip printing its trEMBL ones unless -u      #&#xA;    ################################################&#xA;    if ($to eq 'ID' and not $allow_trembl and scalar(keys(%sw)) &amp;gt; 0) {&#xA;        print &quot;$query\t$target\n&quot; if defined($sw{$target});&#xA;    }&#xA;    else {&#xA;        print &quot;$query\t$&lt;/span&gt;target\n&amp;quot;;&#xA;        }&#xA;        }&#xA;    }&#xA;    &#xA;    ## Count how many were actually found&#xA;    my %found;&#xA;    foreach my &lt;span class=&quot;math-container&quot;&gt;$resp (@a){&#xA;    #print STDERR &quot;R:$resp\t&quot;;&#xA;    foreach my $id (@names){&#xA;    #print STDERR &quot;I:$id\n&quot;;&#xA;    $found{$id}++ if $resp=~/^$&lt;/span&gt;id\t/;&#xA;        }&#xA;        &#xA;    }&#xA;    print STDERR scalar(keys(%found)) . &amp;quot; of &amp;quot; . scalar @names . &amp;quot;\n&amp;quot;;&#xA;&#xA;    }&#xA;    else{&#xA;        die 'Failed, got ' . &lt;span class=&quot;math-container&quot;&gt;$response-&amp;gt;status_line . &#xA;    ' for ' . $&lt;/span&gt;response-&amp;gt;request-&amp;gt;uri . &amp;quot;\n&amp;quot;;&#xA;    }&#xA;}&#xA;################################################&#xA;# This will translate &lt;span class=&quot;math-container&quot;&gt;$to and $&lt;/span&gt;from values     #&#xA;# between the names used in uniprot flat files #&#xA;# and those used by the uniprot database       #&#xA;################################################&#xA;sub translate_terms{&#xA;    my &lt;span class=&quot;math-container&quot;&gt;$id=shift();&#xA;my %hash=(&#xA;&quot;AC&quot; =&amp;gt; &quot;ACC&quot;, &quot;ID&quot; =&amp;gt; &quot;ID&quot;, &quot;UniParc&quot; =&amp;gt; &quot;UPARC&quot;, &quot;UniRef50&quot; =&amp;gt; &quot;NF50&quot;, &quot;UniRef90&quot; =&amp;gt; &quot;NF90&quot;, &quot;UniRef100&quot; =&amp;gt; &quot;NF100&quot;, &quot;EMBL/GenBank/DDBJ&quot; =&amp;gt; &quot;EMBL_ID&quot;, &quot;EMBL/GenBank/DDBJ CDS&quot; =&amp;gt; &quot;EMBLD&quot;, &quot;PIR&quot; =&amp;gt; &quot;PIR&quot;, &quot;UniGene&quot; =&amp;gt; &quot;UNIGENE_ID&quot;, &quot;GI number*&quot; =&amp;gt; &quot;P_GI&quot;, &quot;IPI&quot; =&amp;gt; &quot;P_IPI&quot;, &quot;RefSeq&quot; =&amp;gt; &quot;P_REFSEQ_AC&quot;, &quot;PDB&quot; =&amp;gt; &quot;PDB_ID&quot;, &quot;DisProt&quot; =&amp;gt; &quot;DISPROT_ID&quot;, &quot;HSSP&quot; =&amp;gt; &quot;HSSP_ID&quot;, &quot;DIP&quot; =&amp;gt; &quot;DIP_ID&quot;, &quot;MINT&quot; =&amp;gt; &quot;MINT_ID&quot;, &quot;MEROPS&quot; =&amp;gt; &quot;MEROPS_ID&quot;, &quot;PeroxiBase&quot; =&amp;gt; &quot;PEROXIBASE_ID&quot;, &quot;PptaseDB&quot; =&amp;gt; &quot;PPTASEDB_ID&quot;, &quot;REBASE&quot; =&amp;gt; &quot;REBASE_ID&quot;, &quot;TCDB&quot; =&amp;gt; &quot;TCDB_ID&quot;, &quot;Aarhus/Ghent-2DPAGE&quot; =&amp;gt; &quot;AARHUS_GHENT_2DPAGE_ID&quot;, &quot;ECO2DBASE&quot; =&amp;gt; &quot;ECO2DBASE_ID&quot;, &quot;World-2DPAGE&quot; =&amp;gt; &quot;WORLD_2DPAGE_ID&quot;, &quot;Ensembl&quot; =&amp;gt; &quot;ENSEMBL_ID&quot;, &quot;Ensembl Protein&quot; =&amp;gt; &quot;ENSEMBL_PRO_ID&quot;, &quot;Ensembl  Trscrpt&quot; =&amp;gt; &quot;ENSEMBL_TRS_ID&quot;, &quot;Ensembl  Genomes&quot; =&amp;gt; &quot;ENSEMBLGENOME_ID&quot;, &quot;Ensembl    Genomes&quot; =&amp;gt; &quot;Protein&quot;, &quot;Ensembl Genomes&quot; =&amp;gt; &quot;Transcript&quot;, &quot;GeneID&quot; =&amp;gt; &quot;P_ENTREZGENEID&quot;, &quot;GenomeReviews&quot; =&amp;gt; &quot;GENOMEREVIEWS_ID&quot;, &quot;KEGG&quot; =&amp;gt; &quot;KEGG_ID&quot;, &quot;TIGR&quot; =&amp;gt; &quot;TIGR_ID&quot;, &quot;UCSC&quot; =&amp;gt; &quot;UCSC_ID&quot;, &quot;VectorBase&quot; =&amp;gt; &quot;VECTORBASE_ID&quot;, &quot;AGD&quot; =&amp;gt; &quot;AGD_ID&quot;, &quot;ArachnoServer&quot; =&amp;gt; &quot;ARACHNOSERVER_ID&quot;, &quot;CGD&quot; =&amp;gt; &quot;CGD&quot;, &quot;ConoServer&quot; =&amp;gt; &quot;CONOSERVER_ID&quot;, &quot;CYGD&quot; =&amp;gt; &quot;CYGD_ID&quot;, &quot;dictyBase&quot; =&amp;gt; &quot;DICTYBASE_ID&quot;, &quot;EchoBASE&quot; =&amp;gt; &quot;ECHOBASE_ID&quot;, &quot;EcoGene&quot; =&amp;gt; &quot;ECOGENE_ID&quot;, &quot;euHCVdb&quot; =&amp;gt; &quot;EUHCVDB_ID&quot;, &quot;EuPathDB&quot; =&amp;gt; &quot;EUPATHDB_ID&quot;, &quot;FlyBase&quot; =&amp;gt; &quot;FLYBASE_ID&quot;, &quot;GeneCards&quot; =&amp;gt; &quot;GENECARDS_ID&quot;, &quot;GeneDB_Spombe&quot; =&amp;gt; &quot;GENEDB_SPOMBE_ID&quot;, &quot;GeneFarm&quot; =&amp;gt; &quot;GENEFARM_ID&quot;, &quot;GenoList&quot; =&amp;gt; &quot;GENOLIST_ID&quot;, &quot;H-InvDB&quot; =&amp;gt; &quot;H_INVDB_ID&quot;, &quot;HGNC&quot; =&amp;gt; &quot;HGNC_ID&quot;, &quot;HPA&quot; =&amp;gt; &quot;HPA_ID&quot;, &quot;LegioList&quot; =&amp;gt; &quot;LEGIOLIST_ID&quot;, &quot;Leproma&quot; =&amp;gt; &quot;LEPROMA_ID&quot;, &quot;MaizeGDB&quot; =&amp;gt; &quot;MAIZEGDB_ID&quot;, &quot;MIM&quot; =&amp;gt; &quot;MIM_ID&quot;, &quot;MGI&quot; =&amp;gt; &quot;MGI_ID&quot;, &quot;NMPDR&quot; =&amp;gt; &quot;NMPDR&quot;, &quot;Orphanet&quot; =&amp;gt; &quot;ORPHANET_ID&quot;, &quot;PharmGKB&quot; =&amp;gt; &quot;PHARMGKB_ID&quot;, &quot;PseudoCAP&quot; =&amp;gt; &quot;PSEUDOCAP_ID&quot;, &quot;RGD&quot; =&amp;gt; &quot;RGD_ID&quot;, &quot;SGD&quot; =&amp;gt; &quot;SGD_ID&quot;, &quot;TAIR&quot; =&amp;gt; &quot;TAIR_ID&quot;, &quot;TubercuList&quot; =&amp;gt; &quot;TUBERCULIST_ID&quot;, &quot;WormBase&quot; =&amp;gt; &quot;WORMBASE_ID&quot;, &quot;WormBase Trscrpt&quot; =&amp;gt; &quot;WORMBASE_TRS_ID&quot;, &quot;WormBase Protein&quot; =&amp;gt; &quot;WORMBASE_PRO_ID&quot;, &quot;Xenbase&quot; =&amp;gt; &quot;XENBASE_ID&quot;, &quot;ZFIN&quot; =&amp;gt; &quot;ZFIN_ID&quot;, &quot;eggNOG&quot; =&amp;gt; &quot;EGGNOG_ID&quot;, &quot;HOGENOM&quot; =&amp;gt; &quot;HOGENOM_ID&quot;, &quot;HOVERGEN&quot; =&amp;gt; &quot;HOVERGEN_ID&quot;, &quot;OMA&quot; =&amp;gt; &quot;OMA_ID&quot;, &quot;OrthoDB&quot; =&amp;gt; &quot;ORTHODB_ID&quot;, &quot;ProtClustDB&quot; =&amp;gt; &quot;PROTCLUSTDB_ID&quot;, &quot;BioCyc&quot; =&amp;gt; &quot;BIOCYC_ID&quot;, &quot;Reactome&quot; =&amp;gt; &quot;REACTOME_ID&quot;, &quot;CleanEx&quot; =&amp;gt; &quot;CLEANEX_ID&quot;, &quot;GermOnline&quot; =&amp;gt; &quot;GERMONLINE_ID&quot;, &quot;DrugBank&quot; =&amp;gt; &quot;DRUGBANK_ID&quot;, &quot;NextBio&quot; =&amp;gt; &quot;NEXTBIO_ID&quot;&#xA;);&#xA;defined($&lt;/span&gt;hash{&lt;span class=&quot;math-container&quot;&gt;$id}) ? &#xA;return($&lt;/span&gt;hash{&lt;span class=&quot;math-container&quot;&gt;$id}) : &#xA;return($&lt;/span&gt;id) ;&#xA;}&#xA;&#xA;sub get_response{&#xA;    my &lt;span class=&quot;math-container&quot;&gt;$params=shift;&#xA;my $&lt;/span&gt;response = &lt;span class=&quot;math-container&quot;&gt;$agent-&amp;gt;post(&quot;$&lt;/span&gt;base/&lt;span class=&quot;math-container&quot;&gt;$tool/&quot;, $&lt;/span&gt;params);&#xA;    while (my &lt;span class=&quot;math-container&quot;&gt;$wait = $&lt;/span&gt;response-&amp;gt;header('Retry-After')) {&#xA;    print STDERR &amp;quot;Waiting (&lt;span class=&quot;math-container&quot;&gt;$wait)...\n&quot;;&#xA;sleep $&lt;/span&gt;wait;&#xA;    &lt;span class=&quot;math-container&quot;&gt;$response = $&lt;/span&gt;agent-&amp;gt;get(&lt;span class=&quot;math-container&quot;&gt;$response-&amp;gt;base);&#xA;}&#xA;return($&lt;/span&gt;response);&#xA;&#xA;}&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;sub usage{&#xA;    &lt;span class=&quot;math-container&quot;&gt;$0=~s/.+?([^\/]+)$&lt;/span&gt;/$1/;&#xA;&#xA;&#xA;    print STDERR &amp;lt;&amp;lt;EndOfHelp;&#xA;&#xA;$0 will connect to UniProt and map IDs from the input file to ehatever ID was requested.&#xA;&#xA;USAGE:&#xA;       $0 [options] &amp;lt;ID_LIST&amp;gt;&#xA;&#xA;OPTIONS:&#xA;      -h : Print this help and exit.&#xA;      -f : From ID type. This is the type of ID found in the input file.&#xA;      -F : Format. See http://www.uniprot.org/faq/28#batch_retrieval_of_entries for details. &#xA;&#xA;      -l : When a requested ID maps to many output IDs, print only the first (probably the best) match.&#xA;      -L : List all recognised ID types.&#xA;      -u : Print unreviewed (trEMBL) prots as well. By default, if an ID maps to SWISSPROT and trEMBL IDs &#xA;           only the SWISSPROTs will be printed, use this flag to print trEMBL as well.&#xA;      -t : To ID. This is the ID type that the input IDs will be mapped to.&#xA;      -v : Verbose. Print progress messages to STDERR.&#xA;EndOfHelp&#xA;&#xA;    exit(0);&#xA;&#xA;}&#xA;sub list_ids{&#xA;    open(HELP, &amp;quot;| less&amp;quot;) ;&#xA;    print HELP &amp;lt;&amp;lt;EndOfHelp;&#xA;&#xA;Database            Abbreviation&#xA;===================== UniProt =============================&#xA;UniProtKB AC/ID :   ACC+ID&#xA;UniProtKB AC    :   ACC&#xA;UniProtKB ID    :   ID&#xA;UniParc         :   UPARC&#xA;UniRef50    :   NF50&#xA;UniRef90    :   NF90&#xA;UniRef100   :   NF100&#xA;&#xA;=============== Other sequence databases ==================&#xA;EMBL/GenBank/DDBJ:  EMBL_ID&#xA;EMBL/GenBank/DDBJ CDS:  EMBLD&#xA;PIR     :   PIR&#xA;UniGene     :   UNIGENE_ID&#xA;GI number*  :   P_GI&#xA;IPI     :   P_IPI&#xA;RefSeq      :   P_REFSEQ_AC&#xA;HGNC name       :       GENENAME&#xA;================== 3D structure databases =================&#xA;PDB     :   PDB_ID&#xA;DisProt     :   DISPROT_ID&#xA;HSSP        :   HSSP_ID&#xA;&#xA;============ Protein-protein interaction databases ========&#xA;DIP     :   DIP_ID&#xA;MINT        :   MINT_ID&#xA;&#xA;============ Protein family/group databases ===============&#xA;MEROPS      :   MEROPS_ID&#xA;PeroxiBase  :   PEROXIBASE_ID&#xA;PptaseDB    :   PPTASEDB_ID&#xA;REBASE      :   REBASE_ID&#xA;TCDB        :   TCDB_ID&#xA;&#xA;================== 2D gel databases =======================&#xA;Aarhus/Ghent-2DPAGE:  AARHUS_GHENT_2DPAGE_ID&#xA;ECO2DBASE   :   ECO2DBASE_ID&#xA;World-2DPAGE    :   WORLD_2DPAGE_ID&#xA;&#xA;============= Genome annotation databases =================&#xA;Ensembl         :   ENSEMBL_ID&#xA;Ensembl Protein :   ENSEMBL_PRO_ID&#xA;Ensembl Trscrpt :   ENSEMBL_TRS_ID&#xA;Ensembl Genomes :   ENSEMBLGENOME_ID&#xA;Ensembl Genomes :   Protein&#xA;Ensembl Genomes :   Transcript&#xA;GeneID          :   P_ENTREZGENEID&#xA;GenomeReviews   :   GENOMEREVIEWS_ID&#xA;KEGG        :   KEGG_ID &#xA;TIGR        :   TIGR_ID &#xA;UCSC        :   UCSC_ID &#xA;VectorBase  :   VECTORBASE_ID&#xA;&#xA;============ Organism-specific gene databases =============&#xA;AGD     :   AGD_ID  &#xA;ArachnoServer   :   ARACHNOSERVER_ID&#xA;CGD     :   CGD &#xA;ConoServer  :   CONOSERVER_ID&#xA;CYGD        :   CYGD_ID &#xA;dictyBase   :   DICTYBASE_ID&#xA;EchoBASE    :   ECHOBASE_ID&#xA;EcoGene     :   ECOGENE_ID&#xA;euHCVdb     :   EUHCVDB_ID&#xA;EuPathDB    :   EUPATHDB_ID&#xA;FlyBase     :   FLYBASE_ID&#xA;GeneCards   :   GENECARDS_ID&#xA;GeneDB_Spombe   :   GENEDB_SPOMBE_ID&#xA;GeneFarm    :   GENEFARM_ID&#xA;GenoList    :   GENOLIST_ID&#xA;H-InvDB     :   H_INVDB_ID&#xA;HGNC        :   HGNC_ID &#xA;HPA     :   HPA_ID  &#xA;LegioList   :   LEGIOLIST_ID&#xA;Leproma     :   LEPROMA_ID&#xA;MaizeGDB    :   MAIZEGDB_ID&#xA;MIM     :   MIM_ID  &#xA;MGI     :   MGI_ID  &#xA;NMPDR       :   NMPDR   &#xA;Orphanet    :   ORPHANET_ID&#xA;PharmGKB    :   PHARMGKB_ID&#xA;PseudoCAP   :   PSEUDOCAP_ID&#xA;RGD     :   RGD_ID  &#xA;SGD     :   SGD_ID  &#xA;TAIR        :   TAIR_ID &#xA;TubercuList :   TUBERCULIST_ID&#xA;WormBase    :   WORMBASE_ID&#xA;WormBase Trscrpt:   WORMBASE_TRS_ID&#xA;WormBase Protein:   WORMBASE_PRO_ID&#xA;Xenbase     :   XENBASE_ID&#xA;ZFIN        :   ZFIN_ID&#xA;&#xA;================= Phylogenomic databases ==================&#xA;eggNOG      :   EGGNOG_ID&#xA;HOGENOM     :   HOGENOM_ID&#xA;HOVERGEN    :   HOVERGEN_ID&#xA;OMA     :   OMA_ID&#xA;OrthoDB     :   ORTHODB_ID&#xA;ProtClustDB :   PROTCLUSTDB_ID&#xA;&#xA;=================== Enzyme and pathway ====================&#xA;BioCyc      :   BIOCYC_ID&#xA;Reactome    :   REACTOME_ID&#xA;&#xA;================= Gene expression databases ===============&#xA;CleanEx     :   CLEANEX_ID&#xA;GermOnline  :   GERMONLINE_ID&#xA;&#xA;========================== Other ==========================&#xA;DrugBank    :   DRUGBANK_ID&#xA;NextBio     :   NEXTBIO_ID&#xA;&#xA;EndOfHelp&#xA;close(HELP);&#xA;    exit(0);&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you have too many IDs to do in a single batch, you could either simply split the file into smaller ones and run the script on each file in a loop or modify the script to do it.&lt;/p&gt;&#xA;&lt;p&gt;Split the input:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;$ wc -l names&#xA;$ wc -l names&#xA;500000 names&#xA;## split into 500 files of 1K lines each&#xA;$ split -l 1000 names chunk&#xA;## iterate over each chunk of names&#xA;$ for i in chunk*; do uniprot_map.pl -f ID -t UniParc &amp;quot;$i&amp;quot; &amp;gt;&amp;gt; mapped; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2022-03-05T10:24:27.867" LastActivityDate="2022-03-05T10:24:27.867" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18719" PostTypeId="1" CreationDate="2022-03-04T14:26:53.760" Score="2" ViewCount="35" Body="&lt;p&gt;The data below is coming from a &lt;strong&gt;MaxQuant&lt;/strong&gt; post-translational protein modification (PTM) site file.&lt;/p&gt;&#xA;&lt;p&gt;Why does it contain duplicate sites (yellow) and proteins (cyan)?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BujcG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BujcG.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;Source data&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://ftp.ebi.ac.uk/pride-archive/2020/12/PXD020517/MaxQuant_BA39_SOLUBLE.zip&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ftp.ebi.ac.uk/pride-archive/2020/12/PXD020517/MaxQuant_BA39_SOLUBLE.zip&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="5187" LastEditorUserId="5187" LastEditDate="2022-07-13T19:24:06.780" LastActivityDate="2022-12-11T17:43:46.647" Title="Why does this modification site file contain duplicate sites and multiple related proteins?" Tags="|proteins|mass-spectrometry|maxquant|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18720" PostTypeId="1" CreationDate="2022-03-04T15:05:27.473" Score="1" ViewCount="20" Body="&lt;p&gt;I have a problem finding a methodology to compare proteomic profiles against another. The experiment is the following:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Experiment 1. Ten pdx tumour models with their proteomic profiles were done in one Research Institute with DIA-MS. All of them are the same tumour from different patients. Differential analysis was done with Limma.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Experiment 2. Same tumour, but this time entirely on humans (not pdx) and also controls were included (SWATH-MS in another research institute). Differential analysis was done by t-test.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The biological question is which of the 10 pdx models resembles better with the human one.&lt;/p&gt;&#xA;&lt;p&gt;I tried different ways of normalizing together the raw data but there is a huge difference in the levels of abundance between both experiments. Also, the pdx ones are really similar between them.&lt;/p&gt;&#xA;&lt;p&gt;In the end, I did a one vs rest contrast on the pdx conditions to get the unique markers for each model and use them as genesets (positive and negative separated) to try to see if the upregulated and downregulated genes (tumour vs control on experiment 2) are some kind of enriched (pre-ranked GSEA with rank as -log10 p-value*log2FC).&lt;/p&gt;&#xA;&lt;p&gt;Do you think this can be a solution? Is there a more simple way to do this assessment?&lt;/p&gt;&#xA;&lt;p&gt;Thank you!!&lt;/p&gt;&#xA;" OwnerUserId="8886" LastActivityDate="2022-03-04T15:05:27.473" Title="method to compare the similarity between proteomic profiles" Tags="|proteomics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18721" PostTypeId="2" ParentId="81" CreationDate="2022-03-04T15:29:42.903" Score="0" Body="&lt;p&gt;Go to &lt;a href=&quot;https://www.niehs.nih.gov/research/resources/software/biostatistics/art/index.cfm&quot; rel=&quot;nofollow noreferrer&quot;&gt;ART&lt;/a&gt; website. Download your package, binary or source:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/137Um.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/137Um.png&quot; alt=&quot;screenshot&quot; /&gt;&lt;/a&gt;&#xA;Uncompress the .gz file, open the folder &amp;quot;art_bin_MountRainier&amp;quot;, then &amp;quot;examples&amp;quot;. Open the text file &amp;quot;run_test_examples_illumina.sh&amp;quot;. This file contains numerous example of samples codes. That will give you an idea on how you can use the software. Here is an example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# 4) simulation of paired-end reads of 100bp with the mean fragment size 500 and standard deviation 10&#xA;#   using the built-in seperated quality profiles for A, C, G, and T &#xA;$art -ss HS20 -i ./testSeq.fa -o ./paired_end_sep -l 100 -f 10 -p -m 500 -s 10 -sp -sam&#xA;#convert both aln files to a bed file&#xA;../aln2bed.pl paired_end_sep.bed paired_end_sep1.aln paired_end_sep2.aln&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2022-03-04T15:29:42.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18722" PostTypeId="2" ParentId="18656" CreationDate="2022-03-04T16:44:37.770" Score="0" Body="&lt;p&gt;If preferred, this can be achieved with base R functions:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# First store the chromosome and start position of your gene of interest in variables&#xA;&amp;gt; chrPos &amp;lt;- as.character(seqnames(gr))[gr&lt;span class=&quot;math-container&quot;&gt;$GeneSymbol==&quot;gene3&quot;]&#xA;&amp;gt; startPos &amp;lt;- start(ranges(gr))[gr$&lt;/span&gt;GeneSymbol==&amp;quot;gene3&amp;quot;]&#xA;&#xA;# Then add a column to the GRanges object with distances from the start position of each&#xA;# gene in the GRanges object to the start position of your gene of interest&#xA;&amp;gt; gr$distance &amp;lt;- start(ranges(gr))-startPos&#xA;&#xA;# Finally subset the GRanges object based on your desired window of interest (must be&#xA;# symmetrical —— e.g., +/- 1 Mb —— for the following to work)&#xA;&amp;gt; grOut &amp;lt;- gr[seqnames(gr) %in% chrPos &amp;amp; (abs(gr$distance) &amp;lt;= 1000000)]&#xA;&amp;gt; grOut&#xA;GRanges object with 3 ranges and 4 metadata columns:&#xA;    seqnames         ranges strand |     score        GC  GeneSymbol  distance&#xA;       &amp;lt;Rle&amp;gt;      &amp;lt;IRanges&amp;gt;  &amp;lt;Rle&amp;gt; | &amp;lt;integer&amp;gt; &amp;lt;numeric&amp;gt; &amp;lt;character&amp;gt; &amp;lt;integer&amp;gt;&#xA;  b     chr2 488888-1288888      + |         2  0.888889       gene2   -188889&#xA;  c     chr2 677777-1877777      + |         3  0.777778       gene3         0&#xA;  d     chr2 866666-2466666      * |         4  0.666667       gene4    188889&#xA;  -------&#xA;  seqinfo: 3 sequences from an unspecified genome; no seqlengths&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This strategy provides the added benefit of adding a 'distance-from-gene' (&lt;code&gt;gr$distance&lt;/code&gt;) column to the dataset which can be useful for visualizations.&lt;/p&gt;&#xA;" OwnerUserId="9007" LastEditorUserId="9007" LastEditDate="2022-03-04T18:25:12.067" LastActivityDate="2022-03-04T18:25:12.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18723" PostTypeId="2" ParentId="18473" CreationDate="2022-03-05T12:20:12.137" Score="2" Body="&lt;p&gt;You can use &lt;a href=&quot;https://www.cgl.ucsf.edu/chimerax/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ChimeraX&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Open your glycans and your protein pdb files as different models, then move the models of your glycans until they are near the residues you want (in the Toolbar: &lt;a href=&quot;https://www.cgl.ucsf.edu/chimerax/docs/user/tools/mousemodes.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Right Mouse&lt;/a&gt; -&amp;gt; Move Model).&lt;/li&gt;&#xA;&lt;li&gt;Then use &lt;a href=&quot;https://www.rbvi.ucsf.edu/chimerax/docs/user/commands/combine.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;combine&lt;/code&gt;&lt;/a&gt; to combine one of your glycan model with the protein&lt;/li&gt;&#xA;&lt;li&gt;Finally Use Tools -&amp;gt; Structure Editing -&amp;gt; Build Structure and its &lt;a href=&quot;https://www.cgl.ucsf.edu/chimerax/docs/user/tools/buildstructure.html#bonds&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adjust Bonds&lt;/a&gt; menu to add covalent bonds. You can also add or delete atoms this way.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This is a straightforward but complicated and tedious way of doing this, and it will give you unideal structures. There should be an equivalent way to proceed using PyMol. For a more automatized way I can only recommend having a look at &lt;a href=&quot;https://www.pyrosetta.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyRosetta&lt;/a&gt;. There is a &lt;a href=&quot;https://nbviewer.org/github/RosettaCommons/PyRosetta.notebooks/blob/master/notebooks/13.02-Glycan-Modeling-and-Design.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jupyter Notebook tutorial about adding glycosylation&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There may be other tools out there for doing this, like &lt;a href=&quot;https://software.acellera.com/docs/stable/moleculekit/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MoleculeKit&lt;/a&gt;, but I'm less aware of them. A nasty solution would be to cut your protein in smaller pieces, glycosylate each part with CHARMM-GUI, then reassemble them.&lt;/p&gt;&#xA;" OwnerUserId="12054" LastActivityDate="2022-03-05T12:20:12.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18724" PostTypeId="2" ParentId="14161" CreationDate="2022-03-05T15:16:05.943" Score="1" Body="&lt;p&gt;I would recommend a specific tool for viral meta genomics rather than using applying a general tool into a specific technical challenge. Here are the ones discussed on Bioinfo SE for &lt;em&gt;de novo&lt;/em&gt; metagenomic assembly tested/being applied to viruses. It is not an exclusive list and needs further discussion.&lt;/p&gt;&#xA;&lt;p&gt;SAVAGE&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5411778/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5411778/&lt;/a&gt;&#xA;Baaijens et al. De novo assembly of viral quasispecies using overlap graphs Genome Research 2017 May; 27: 835–848.&lt;/p&gt;&#xA;&lt;p&gt;S-aligner&#xA;&lt;a href=&quot;https://www.biorxiv.org/content/biorxiv/early/2021/02/02/2021.02.02.429443.full.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/biorxiv/early/2021/02/02/2021.02.02.429443.full.pdf&lt;/a&gt;&#xA;Juanjo Bermúdez s-aligner: a greedy algorithm for non-greedy de novo genome assembly 2021 bioRχiv February: 1-7&lt;/p&gt;&#xA;&lt;p&gt;Rey&#xA;&lt;a href=&quot;http://denovoassembler.sourceforge.net&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://denovoassembler.sourceforge.net&lt;/a&gt;&#xA;Sébastien Boisvert et al Genome Biology. 2012 13:R122, December&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-03-05T15:16:05.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18725" PostTypeId="1" AcceptedAnswerId="18727" CreationDate="2022-03-06T19:59:10.183" Score="0" ViewCount="32" Body="&lt;p&gt;I have 6000 predicted protein structures.&lt;/p&gt;&#xA;&lt;p&gt;I am comparing secondary structures predicted by an algorithm with the correct secondary structure assignments.&lt;/p&gt;&#xA;&lt;p&gt;I need to show which positions in a predicted protein chain have the most differences as to the true protein chain.&lt;/p&gt;&#xA;&lt;p&gt;How can I do that?&lt;/p&gt;&#xA;&lt;p&gt;How to represent a summary of disagreement between two secondary structure predictors given predictions for many proteins?&lt;/p&gt;&#xA;&lt;p&gt;I already prepared confusion matrices. I was looking for something graphical like chart or histogram, etc..&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="12686" LastEditDate="2022-03-07T09:52:31.453" LastActivityDate="2022-03-07T10:20:06.323" Title="How to represent a summary of disagreement between two secondary structure predictors given predictions for many proteins?" Tags="|protein-structure|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18727" PostTypeId="2" ParentId="18725" CreationDate="2022-03-07T09:38:20.567" Score="3" Body="&lt;p&gt;(This comment got too long)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;positions in a predicted protein chain&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You do mean &lt;em&gt;position&lt;/em&gt; in SS sequence, not &lt;em&gt;coordinates&lt;/em&gt; in tertiary, yes?&#xA;From the sounds of it you are talking about SS —not tertiary— and want the analysis to be independent of tertiary.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, you are comparing for each protein two 1D arrays of sequential elements containing between 3 and 6 states, one for each residue. Technically, you could add another state, a chain break, and have two colossally long 1D arrays. This very much becomes a &lt;strong&gt;statistics&lt;/strong&gt; classification topic.&lt;/p&gt;&#xA;&lt;p&gt;The properties of these arrays are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The values are sequential.&lt;/li&gt;&#xA;&lt;li&gt;Neighbouring elements influence each other as a span of residues makes a SS element. Thus a correct classification will propagate. This will mean that all metrics will be hugely inflated.&lt;/li&gt;&#xA;&lt;li&gt;Assumption: the divergence is likely on the ends of a span. This could be tested and plotted!&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The simplest thing would be collapse the states into a binary loop vs non loop and classify each element of the two arrays and do a &lt;strong&gt;typical confusion matrix&lt;/strong&gt; and its metrics (see &lt;a href=&quot;https://en.wikipedia.org/wiki/Confusion_matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;). This would have the issue that neighbours influence each other.&lt;/p&gt;&#xA;&lt;p&gt;For a figure, given N states, one could simply plot a &lt;em&gt;N&lt;/em&gt; × &lt;em&gt;N&lt;/em&gt; heatmap of the true states vs. predicted states. This would show a hot diagonal of the correct allocations, but will show which states are more prone to misallocation. The frequency of the states may need normalising... but this would still be affected by the length of the SS spans. Therefore, a more sophisticated &lt;em&gt;model&lt;/em&gt; may be needed for better normalisation.&lt;/p&gt;&#xA;&lt;p&gt;For more advanced classifications, more advanced algorithms would be needed, but it still remains a stats problem. It might be worth asking the question again under that discussion framework or look up a relevant multi-state classifier.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2022-03-07T10:20:06.323" LastActivityDate="2022-03-07T10:20:06.323" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="18728" PostTypeId="1" CreationDate="2022-03-07T14:21:03.800" Score="2" ViewCount="114" Body="&lt;p&gt;I ran my 23andMe data through the &lt;a href=&quot;https://imputation.sanger.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sanger Imputation Service&lt;/a&gt; that uses &lt;a href=&quot;https://alkesgroup.broadinstitute.org/Eagle/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Eagle&lt;/a&gt; v2.4 for phasing, &lt;a href=&quot;https://github.com/VertebrateResequencing/pbwt&quot; rel=&quot;nofollow noreferrer&quot;&gt;PBWT&lt;/a&gt; v3.1 for imputation.&lt;/p&gt;&#xA;&lt;p&gt;However, some aspects of the results are very confusing to me. Perhaps see #3 (the most confusing) first.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Point 1&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;There are 62,623 rsIDs for chromosome 3 in my 23andMe file and 2,821,894 SNPs (actually, unique rsIDs) after imputation. However, only 60,144 of the original 62,623 rsIDs are in that ~2.8 million. Where did the other 2,479 go? Here are some examples of missing rsIDs:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/rs1516332&quot; rel=&quot;nofollow noreferrer&quot;&gt;rs1516332&lt;/a&gt;,&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/rs12374025&quot; rel=&quot;nofollow noreferrer&quot;&gt;rs12374025&lt;/a&gt;,&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/rs1170695&quot; rel=&quot;nofollow noreferrer&quot;&gt;rs1170695&lt;/a&gt;,&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/rs2731343&quot; rel=&quot;nofollow noreferrer&quot;&gt;rs2731343&lt;/a&gt; and&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/rs63749864&quot; rel=&quot;nofollow noreferrer&quot;&gt;rs63749864&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT Point 1&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Looking at &lt;code&gt;rs1516332&lt;/code&gt;, the position is 3:105793, which does have a result in the file, but it doesn't have an rsID:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;3    105793    .    C    T    .    PASS    RefPanelAF=0.17878;AN=2;AC=0;INFO=1    GT:ADS:DS:GP    0|0:0,0:0:1,0,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Looking at &lt;code&gt;rs63749864&lt;/code&gt; (3:13564048) it's missing by rsID and position.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT Point 1 (again)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;As described here: &lt;a href=&quot;https://github.com/richarddurbin/pbwt/issues/51&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/richarddurbin/pbwt/issues/51&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;... of the 2,479 'missing' SNPs, 1,806 can be 'found' [by position] leaving 673 'anchors' [or rsIDs called by 23andMe] missing from chromosome 3.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Point 2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Some imputed rsIDs appear more than once in the results. How should I interpret that? e.g.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;3   64071   rs116059446 G   T   .   PASS    RefPanelAF=0.000369572;AN=2;AC=0;INFO=1 GT:ADS:DS:GP    0|0:0,0:0:1,0,0&#xA;3   64071   rs116059446 G   A   .   PASS    RefPanelAF=9.2393e-05;AN=2;AC=0;INFO=1  GT:ADS:DS:GP    0|0:0,0:0:1,0,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT Point 2&lt;/strong&gt;&#xA;From the answer below, this is a variant with more than two alleles: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/rs116059446&quot; rel=&quot;nofollow noreferrer&quot;&gt;rs116059446&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Seems the format of this VCF file only allows one alt per line, hence the two alts are split over two lines.&lt;/p&gt;&#xA;&lt;p&gt;See my question here: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/18752/is-it-valid-vcf-not-to-squash-positions-with-more-than-one-alt-allele&quot;&gt;Is it valid VCF not to &amp;#39;squash&amp;#39; positions with more than one ALT allele?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And the answer from Durbo here: &lt;a href=&quot;https://github.com/richarddurbin/pbwt/issues/52#issuecomment-1062951105&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/richarddurbin/pbwt/issues/52#issuecomment-1062951105&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the above case, the genotype is easy to interpret (REF/REF in both lines).&lt;/p&gt;&#xA;&lt;p&gt;However, I'm now finding examples that I just can't interpret:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;3   255395  rs331869    C   G   .   PASS    RefPanelAF=0.385864;AN=2;AC=1;INFO=1    GT:ADS:DS:GP    1|0:1,0:1:0,1,0&#xA;3   255395  rs331869    C   T   .   PASS    RefPanelAF=0.980998;AN=2;AC=2;INFO=1    GT:ADS:DS:GP    1|1:1,1:2:0,0,1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;Is my genotype CG or TT at this position?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Point 3&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Sometimes a rsID appears twice but at different locations:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;3   4942430 rs71634747  G   A   .   PASS    RefPanelAF=0.445673;AN=2;AC=1;INFO=1    GT:ADS:DS:GP    1|0:1,0:1:0,1,0&#xA;3   4942432 rs71634747  C   G   .   PASS    RefPanelAF=0.248907;AN=2;AC=0;INFO=1    GT:ADS:DS:GP    0|0:0,0:0:1,0,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Unless I'm really confused, this just seems wrong. If it wasn't for this weirdness, I'd seek concrete answers for #1 and #2, but it could be that they are both just a symptom of this potential error?&lt;/p&gt;&#xA;" OwnerUserId="8875" LastEditorUserId="8875" LastEditDate="2022-03-10T13:46:36.383" LastActivityDate="2022-06-10T10:56:11.587" Title="Confusing result from Sanger Imputation Service (Eagle v2.4 for phasing, PBWT v3.1 for imputation)" Tags="|vcf|snp|imputation|pbwt|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18729" PostTypeId="2" ParentId="8634" CreationDate="2022-03-07T14:49:17.370" Score="1" Body="&lt;p&gt;Run align-info  to see the references used.&#xA;Run prefetch &amp;lt;/path/to/sra/file&amp;gt; to download missed references.&lt;/p&gt;&#xA;" OwnerUserId="8875" LastActivityDate="2022-03-07T14:49:17.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18730" PostTypeId="1" CreationDate="2022-03-07T14:56:17.680" Score="0" ViewCount="81" Body="&lt;p&gt;How can I convert protein distance maps and sequences to pdb files?&lt;/p&gt;&#xA;&lt;p&gt;For the same problem, in the case of a model predicting the structure of a protein, such as AlphaFold, how do we convert the generated output tensor to a PDB file and visualize it?&lt;/p&gt;&#xA;" OwnerUserId="12688" LastActivityDate="2022-03-07T14:56:17.680" Title="How can I convert protein distance maps and sequences to a pdb files?" Tags="|proteins|pdb|machine-learning|deep-learning|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18731" PostTypeId="1" AcceptedAnswerId="18793" CreationDate="2022-03-07T17:39:03.377" Score="-1" ViewCount="56" Body="&lt;p&gt;I have some problems with geosphere package which extends/applies the Beast Bayesian MCMC package for calculating phylogenetic trees. Please, could anyone give any hints about at least one of them?&lt;/p&gt;&#xA;&lt;p&gt;Firstly, I cannot reproduce &lt;a href=&quot;https://github.com/BEAST2-Dev/beast-geo/releases/download/v1.2.0/phylogeography_s.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;the tutorial&lt;/a&gt;. I followed the instructions step-by-step but my results are too different (Fig 1) from specified in it. What could I do wrong?&#xA;Secondly, it is oft occurred that my mean and standard deviation increase too high (Fig 2). I am aware that this is bad but what is the right way not to allow such situations? Just put an upper border? In this case, what is the reasonable border?&#xA;Thirdly, why is a site model of the geodata partition set as the first option in a list of  models for amino acids or nucleotides substitution, &amp;amp; I cannot pick any other (Fig 3 &amp;amp; 4)? What do precision and fast checkbox mean? Is there any way to get under the hood of model to vary more parameters?&#xA;Finally, are there any documentation, guides, tutorials &amp;amp;c. about geosphere besides the aforementioned tutorial? I did not find any on its GitHub repository.&lt;/p&gt;&#xA;&lt;p&gt;Fig 1&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CNmJM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CNmJM.png&quot; alt=&quot;Fig 1&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Fig 2&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/EKgEl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/EKgEl.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Fig 3&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qiK0y.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qiK0y.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Fig 4&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/noenM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/noenM.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;My trees:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1gUwc.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1gUwc.jpg&quot; alt=&quot;trees in DensiTree&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bS4Uh.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bS4Uh.jpg&quot; alt=&quot;Consensus tree in Google Earth&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15161" LastEditorUserId="15161" LastEditDate="2022-03-07T20:01:28.947" LastActivityDate="2022-03-16T14:26:57.853" Title="How to use GEO_SPHERE?" Tags="|phylogenetics|phylogeny|virus|beast|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18732" PostTypeId="1" CreationDate="2022-03-07T17:39:35.293" Score="1" ViewCount="128" Body="&lt;p&gt;I am interested in differential response to a drug in the &amp;quot;RED&amp;quot; group versus the &amp;quot;GRN&amp;quot; group. I mean, I need to compare differences in DEG expression between two populations (RED vs GRN)&lt;/p&gt;&#xA;&lt;p&gt;I have 3 pairs of control vs treated samples. 3 pairs for each population.&lt;/p&gt;&#xA;&lt;p&gt;Here is an example of results for a gene named &amp;quot;gen1&amp;quot;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Gname   RED_ctl1  RED_drg1  RED_ctl2  RED_drg2  RED_ctl3  RED_drg3&#xA;gen1           4        10         5        10         6         9&#xA;Gname   GRN_ctl1  GRN_drg1  GRN_ctl2  GRN_drg2  GRN_ctl3  GRN_drg3&#xA;gen1           4         9         5        10         4         9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In the end, I expect to plot a boxplot that would show me (or not) the difference between RED and GRN groups in terms of DEG expression. I expect to find a group (either RED or GRN) with DEGs that respond to drugs much more (or less) in terms of expression.&lt;/p&gt;&#xA;&lt;p&gt;Since this is my first time doing this analysis, I would like to know what is the classical way of analyzing that kind of data? What are the steps?&lt;/p&gt;&#xA;&lt;p&gt;A related question was presented &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/18660/what-is-the-best-way-to-compare-treated-control-samples&quot;&gt;here&lt;/a&gt; on Bioinformatics SE&lt;/p&gt;&#xA;" OwnerUserId="6470" LastEditorUserId="73" LastEditDate="2022-03-08T00:07:32.227" LastActivityDate="2022-03-08T04:24:51.963" Title="What is the best way to compare differences in DEG expression between two populations?" Tags="|differential-expression|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18733" PostTypeId="2" ParentId="18732" CreationDate="2022-03-07T18:17:50.083" Score="2" Body="&lt;p&gt;A t-test or eyeballing a box plot might be okay for something very informal, but that's not the statistically right way to do it.&lt;/p&gt;&#xA;&lt;p&gt;If this is based on RNASeq data, the answer is clear.  You need DESeq2 or EdgeR.  Read their tutorials, and pick one to use.&lt;/p&gt;&#xA;&lt;p&gt;If this is microarray, limma is probably best.&lt;/p&gt;&#xA;&lt;p&gt;If it's something else, you'll need to read some papers to see what people use.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2022-03-07T18:17:50.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18734" PostTypeId="2" ParentId="18732" CreationDate="2022-03-07T18:47:58.230" Score="1" Body="&lt;p&gt;It doesn't matter what data you are dealing with classical statistics can handle this data and non-parametric are very versatile, but there is one notable limitation.&lt;/p&gt;&#xA;&lt;p&gt;If this is strictly paired data (which what I suspect):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GRN 1 | GRN 2 | GRN 3&#xA;RED 1 | RED 2 | RED 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This would be a two-way ANOVA and is a very powerful test it is fulfils the criteria (below). Interpreting the output needs a bit of background reading.&lt;/p&gt;&#xA;&lt;p&gt;However, if it is simply GRN vs RED then it is one-way ANOVA (or T-test) and Kruskal-Wallis test (non-parametric). In addition, if you are comparing&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;RED1&#xA;RED2&#xA;RED3&#xA;GRN1&#xA;GRN2&#xA;GRN3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thus 6 data sets a T-test doesn't work and its one-way ANOVA or KW test.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;ANOVA&lt;/em&gt; The Fmax test is needed for ANOVA analysis to assess the homogeneity of variance.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tQLp6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tQLp6.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Additional ANOVA criteria&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The test works best when the number of samples drawn from each population is the same (check in this case). This is not an essential requirement but if the sampling between groups is heavily that's not good.&lt;/li&gt;&#xA;&lt;li&gt;The underlying populations are normally distributed (same is true of the T-test). If Fmax says 'yes' skip this bit.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I honestly don't know if Fmax is available in Python, it is definitely not &lt;code&gt;numpy.fmax&lt;/code&gt;. It is available in R:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;hartleyTest(x, ...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where &lt;em&gt;x&lt;/em&gt; is the vector. You can do it manually.&lt;/p&gt;&#xA;&lt;p&gt;For 1-way ANOVA&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.stats import f_oneway&#xA;&#xA;f_oneway(GRN123, RED123) &#xA;f_oneway(GRN1,GRN2,GRN3,RED1,RED2,RED3) # This makes it like the second example &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The data is within a list e.g. &lt;code&gt;GRN1 = [1,2,3,1,1,14,1]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Easy.&lt;/p&gt;&#xA;&lt;p&gt;Two-way ANOVA is complicated&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import statsmodels.api as sm&#xA;from statsmodels.formula.api import old&#xA;&#xA;model = ols('treatment ~ C(RED1) + C(GRN1) ... : C(GRN1)', data=df).fit() # see note&#xA;sm.stats.anova_lm(model, typ=2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You should be able to circumvent the ols - because its complicated. It is however useful, particularly is Fmax fails.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.statology.org/two-way-anova-python/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here for details.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You might substitute your one-way ANOVA for a T-test in this instance, but it depends how you structure you comparison.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;If Fmax fails&lt;/em&gt; and you were wanting to perform a one-way ANOVA you shift to the non-parametric Kruskal-Wallis test. In python,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from scipy import stats&#xA;&#xA;stats.kruskal(GRN123, RED123)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Work around if its paired data ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;stats.kruskal (GRN1, RED1)&#xA;stats.kruskal (GRN2, RED2)&#xA;....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Its a bit cheeky but you can cover yourself with Bonferoni correction and strictly Wilcoxan is better here.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;So its easy.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;HOWEVER&lt;/strong&gt; if your data is paired and fails an Fmax test, strictly you go to the Friedman test, this is the non-parametric version of the 2-way ANOVA (by Milton Friedman). Now for some reason no-one ever does this, but  I don't remember what the reason why. Thus your options are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;in that situation you could just use the ols regression model fitted into the ANOVA - but you need to think hard about his model because it needs to present nicely paired data to ANOVA.&lt;/li&gt;&#xA;&lt;li&gt;You could of course do the Friedman test and its up to the reviewer to understand the theoretical criticism of this (I don't remember it and generally the quantitive standards in molecular peer-review isn't great)&lt;/li&gt;&#xA;&lt;li&gt;You could use multiple KW&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Alternative&lt;/strong&gt; the 'correct' way to get ANOVA working when Fmax says 'no' is transformation. This gets complicated however and is one step before a custom model.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Final points&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Exception&lt;/strong&gt; If this is microarray data - there is an issue about standardisation and that is complicated (comparing different microarray slides against experimental variation).&lt;/li&gt;&#xA;&lt;li&gt;What algorithms such as DSeq2 do will likely do is apply a fixed transformation to the data, but I don't know about standardisation. &lt;strong&gt;Non-parametric statistics will circumvent the requirement for transformation&lt;/strong&gt; ... their whole point is you don't transform and must not transform - must be raw data.&lt;/li&gt;&#xA;&lt;li&gt;Again non-parametrics become difficult in replicating 2-way ANOVA ... there are several approaches here and of course there is nothing to stop you using it and leave to someone else explain the limitation.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; the Wilcoxon is the non-parametric equivalent of the T-test, which I think we covered in a previous post (these are mice right?).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;What would I do? Personally, if a non-parametric fits I use it every time ... its simple, everyone understand it and a significant result is not in dispute. The reality is you are probably into a skewed distribution. Furthermore, no-one gets worried about a few KW tests (type 2 error) and you can always deploy bonferoni's correction.&lt;/p&gt;&#xA;&lt;p&gt;I should understand the limitation of Friedman test and I might be working under a false assumption. If a simple transformation results in good Fmax I'd use ANOVA (if raw data isn't homogeneous). Beyond its statistical modelling and that's complicated.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-03-08T04:24:51.963" LastActivityDate="2022-03-08T04:24:51.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18735" PostTypeId="2" ParentId="18731" CreationDate="2022-03-07T19:26:31.803" Score="0" Body="&lt;p&gt;This may not be the exact answer but its a pretty good guess and suggests an absence of &lt;code&gt;getters&lt;/code&gt; and &lt;code&gt;setters&lt;/code&gt; for this particular issue. Thus I accept the error does not look informative.&lt;/p&gt;&#xA;&lt;p&gt;I know 100% what is wrong with the MCMC tree model you performed on HBV, hepatitis B virus.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;ESS&lt;/code&gt; are mostly &amp;lt;&amp;lt;200, that is where they are in red. The MCMC needs to run for longer or a simpler model (probably both). For example, the &lt;code&gt;prior&lt;/code&gt; is 4 which is way too low.&lt;/p&gt;&#xA;&lt;p&gt;In particular &lt;code&gt;sprecision.geo&lt;/code&gt; is &lt;code&gt;-&lt;/code&gt; and that does not look good for the calculation you are performing because it suggests it (the &lt;code&gt;geo&lt;/code&gt; object is not present and looks remarkably like the error you are receiving). Simply this will be written in Java and that value it will see as an object. Whether this is singly due to insufficient runtime, I don't know, but the ESS values are a pretty good guess at diagnosing the situation.&lt;/p&gt;&#xA;&lt;p&gt;What might be happening is that if the values in red are too low and insufficient to calculate &lt;code&gt;sprecision.geo&lt;/code&gt; hence its &lt;code&gt;-&lt;/code&gt; and the absence of this value is preventing the &lt;code&gt;diffusionModel.s:geo&lt;/code&gt; from being calculated.&lt;/p&gt;&#xA;&lt;p&gt;Do consider Beast is a hugely impressive but complicated population dynamics Bayesian simulation which is further complicated because you are stacking a further level of parameters onto it.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I have responded to the comments below here:&lt;/p&gt;&#xA;&lt;p&gt;HBV is known to be a difficult data set in phylogeny, it has a very weird genome and if further iterations are resulting in errors with the basic MCMC something is definitely not right with the core calculation. 'Too much MCMC' isn't a valid error in a Bayesian calculation. I would simply switch data set and try again. The ESS is simply too low for any further analysis and that's why its in red.&lt;/p&gt;&#xA;&lt;p&gt;One of the photos provided doesn't seem to stack up.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-03-07T20:27:53.763" LastActivityDate="2022-03-07T20:27:53.763" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18736" PostTypeId="1" AcceptedAnswerId="18737" CreationDate="2022-03-07T21:44:10.990" Score="1" ViewCount="793" Body="&lt;p&gt;I am trying to use a commonly recommended package named &lt;a href=&quot;https://github.com/mskcc/vcf2maf&quot; rel=&quot;nofollow noreferrer&quot;&gt;vcf2maf&lt;/a&gt;to convert vcf file to maf file and is reliant on Ensembl's Variant Effect Predictor (VEP) &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-0974-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. However, if I follow the guidelines shown on the github, I would get:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl vcf2maf.pl --ref-fasta hg38.fa --input-vcf chr21.vcf --output-maf chr21.maf&#xA;dyld[33981]: Library not loaded: @rpath/libcrypto.1.0.0.dylib&#xA;  Referenced from: /usr/local/anaconda3/bin/samtools&#xA;  Reason: tried: '/usr/local/anaconda3/bin/../lib/libcrypto.1.0.0.dylib' (no such file), '/usr/local/anaconda3/bin/../lib/libcrypto.1.0.0.dylib' (no such file), '/usr/local/lib/libcrypto.1.0.0.dylib' (no such file), '/usr/lib/libcrypto.1.0.0.dylib' (no such file)&#xA;dyld[33982]: Library not loaded: @rpath/libcrypto.1.0.0.dylib&#xA;  Referenced from: /usr/local/anaconda3/bin/samtools&#xA;  Reason: tried: '/usr/local/anaconda3/bin/../lib/libcrypto.1.0.0.dylib' (no such file), '/usr/local/anaconda3/bin/../lib/libcrypto.1.0.0.dylib' (no such file), '/usr/local/lib/libcrypto.1.0.0.dylib' (no such file), '/usr/lib/libcrypto.1.0.0.dylib' (no such file)&#xA;dyld[33983]: Library not loaded: @rpath/libcrypto.1.0.0.dylib&#xA;  Referenced from: /usr/local/anaconda3/bin/samtools&#xA;  Reason: tried: '/usr/local/anaconda3/bin/../lib/libcrypto.1.0.0.dylib' (no such file), '/usr/local/anaconda3/bin/../lib/libcrypto.1.0.0.dylib' (no such file), '/usr/local/lib/libcrypto.1.0.0.dylib' (no such file), '/usr/lib/libcrypto.1.0.0.dylib' (no such file)&#xA;dyld[33984]: Library not loaded: @rpath/libcrypto.1.0.0.dylib&#xA;  Referenced from: /usr/local/anaconda3/bin/samtools&#xA;  Reason: tried: '/usr/local/anaconda3/bin/../lib/libcrypto.1.0.0.dylib' (no such file), '/usr/local/anaconda3/bin/../lib/libcrypto.1.0.0.dylib' (no such file), '/usr/local/lib/libcrypto.1.0.0.dylib' (no such file), '/usr/lib/libcrypto.1.0.0.dylib' (no such file)&#xA;ERROR: You're either using an outdated samtools, or --ref-fasta is not the same genome build as your --input-vcf. at vcf2maf.pl line 410.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I think the reference is the correct one, and I guess the error might be because of the VEP installation (as I came across a lot of bugs when following their &lt;a href=&quot;https://gist.github.com/ckandoth/61c65ba96b011f286220fa4832ad2bc0&quot; rel=&quot;nofollow noreferrer&quot;&gt;gist&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;As a result, I tried to use the &lt;a href=&quot;https://anaconda.org/bioconda/vcf2maf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conda installation&lt;/a&gt; and use the codes:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c bioconda vcf2maf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, after the package was successfully installed, I could not call it in the command line.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(base) user-MacBook-Pro:~ scottxu$ conda install -c bioconda vcf2maf&#xA;Collecting package metadata (current_repodata.json): done&#xA;Solving environment: done&#xA;&#xA;# All requested packages already installed.&#xA;&#xA;(base) user-MacBook-Pro:~ user$ vcf2maf&#xA;-bash: vcf2maf: command not found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or, if I want to use the Perl script:&#xA;I will get:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(base) user-MacBook-Pro:~ user$ perl vcf2maf.pl &#xA;Can't open Perl script &amp;quot;vcf2maf.pl&amp;quot;: No such file or directory&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Could anyone give me suggestions on how to install and use vcf2maf correctly? Thank you very much in advance~~&#xA;I am working on macOS Monterey 12.2.1 with an Intel Core i5 processor.&lt;/p&gt;&#xA;" OwnerUserId="14102" LastEditorUserId="3848" LastEditDate="2022-03-07T22:39:01.083" LastActivityDate="2022-03-07T22:39:01.083" Title="Problems when using vcf2maf" Tags="|vcf|maf|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18737" PostTypeId="2" ParentId="18736" CreationDate="2022-03-07T21:53:29.867" Score="1" Body="&lt;p&gt;Install it via Anaconda/conda.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c bioconda vcf2maf &#xA;conda install -c bioconda/label/cf201901 vcf2maf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I personally like Anaconda-navigator for OSX but you would likely go for 'command line version' in part because it will place it in your home directory. Its here &lt;a href=&quot;https://www.anaconda.com/products/individual&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.anaconda.com/products/individual&lt;/a&gt; and just use the either the above and then its done. The conda environments can be considered once vcf2maf is working.&lt;/p&gt;&#xA;&lt;p&gt;It is never cool to run Perl on the system command line.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Once installed ensure the conda activated&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda activate &amp;lt;env name&amp;gt; # if its base I don't think the &amp;lt;&amp;gt; is needed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then simply type&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;vcf2maf.pl&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; Usage:&#xA;&amp;gt;      perl maf2vcf.pl --help&#xA;&amp;gt;      perl maf2vcf.pl --input-maf test.maf --output-dir vcfs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Its a Perl script hence it will not recognise maf2vcf without the suffix &lt;code&gt;pl&lt;/code&gt;. Anaconda doesn't need &lt;code&gt;perl &amp;lt;perlscript&amp;gt;&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Oh just to mention I just &lt;code&gt;conda install vcf2maf&lt;/code&gt; without the channel but I think I've got bioconda on my repo list.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I've just uninstalled it. There appears to be an error with the help docs. The help docs (probably PODs) may not have been uploaded and in any case it will run without them. What conda gives you is the full set of dependencies for the script (makes uninstallation a headache) -  however you don't need to worry hence I would just use it &lt;code&gt;maf2vcf.pl --input-maf test.maf --output-dir vcfs&lt;/code&gt; that should work.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note of explanation: why Anaconda works&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I better explain, the script run by itself is complaining it can't see the dependencies it requires. It doesn't just need them, it needs to see them. Anaconda/conda takes care of all that by downloading the dependencies and ensuring they are all visible. The downside is uninstalling everything afterwards, but that's my problem not the OPs.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-03-07T22:32:55.917" LastActivityDate="2022-03-07T22:32:55.917" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18738" PostTypeId="1" AcceptedAnswerId="18761" CreationDate="2022-03-07T22:40:10.807" Score="2" ViewCount="141" Body="&lt;p&gt;[Note: this is for bedtools v2.30]&lt;/p&gt;&#xA;&lt;p&gt;I want to count the number of RNA-Seq reads that fully cover a given splice junction. For that, I thought of defining a BED feature around the junction, then using &lt;code&gt;bedtools coverage&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here is a simplified situation:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1-BASED POS 1|      10|       20|       30|&#xA;CHROMOSOME   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;BED FILE              **                   &#xA;BAM READ FWD &amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;                    &#xA;BAM READ REV                  &amp;lt;&amp;lt;&amp;lt;          &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We can generate the corresponding example files:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;echo -e &amp;quot;I\t9\t11\tmy_position\t.\t+&amp;quot; &amp;gt; single_place.bed&#xA;&#xA;echo -e &amp;quot;@HD\tVN:1.4\tSO:coordinate&#xA;@SQ\tSN:I\tLN:30&#xA;first_read\t99\tI\t1\t255\t10M\t=\t20\t20\tAATTCCGGAA\tDDDDDDDDDD\tNH:i:1\tHI:i:1\tAS:i:127\tnM:i:0&#xA;first_mate\t147\tI\t20\t255\t3M\t=\t1\t20\tATC\tDDD\tNH:i:1\tHI:i:1\tAS:i:127\tnM:i:0&amp;quot; | samtools view -b &amp;gt; fwd_only.bam&#xA;samtools index fwd_only.bam&#xA;&#xA;&#xA;cat single_place.bed&#xA;I       9       11      my_position     .       +&#xA;samtools view fwd_only.bam&#xA;first_read      99      I       1       255     10M     =       20      20      AATTCCGGAA      DDDDDDDDDD      NH:i:1  HI:i:1  AS:i:127        nM:i:0&#xA;first_mate      147     I       20      255     3M      =       1       20      ATC     DDD     NH:i:1  HI:i:1  AS:i:127        nM:i:0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The per-base count (with the &lt;code&gt;-d&lt;/code&gt; option) gives the expected result:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;coverageBed -d -a single_place.bed -b fwd_only.bam&#xA;I       9       11      my_position     .       +       1       1&#xA;I       9       11      my_position     .       +       2       0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And using a filter &lt;code&gt;-f 1.0&lt;/code&gt; to only keep reads that cover the entire feature indeed removes our read:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;coverageBed -a single_place.bed -b fwd_only.bam&#xA;I       9       11      my_position     .       +       1       1       2       0.5000000&#xA;coverageBed -f 1.0 -a single_place.bed -b fwd_only.bam&#xA;I       9       11      my_position     .       +       0       0       2       0.0000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, in practice, I have a lot of spliced reads, so I wish to use the &lt;code&gt;-split&lt;/code&gt; option. In that case, the filtering doesn't work anymore:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;coverageBed -split -f 1.0 -a single_place.bed -b fwd_only.bam&#xA;I       9       11      my_position     .       +       1       1       2       0.5000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;even though the per-base count is still unchanged:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;coverageBed -d -split -a single_place.bed -b fwd_only.bam&#xA;I       9       11      my_position     .       +       1       1&#xA;I       9       11      my_position     .       +       2       0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Why does &lt;code&gt;-split&lt;/code&gt; interfere with &lt;code&gt;-f 1.0&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;h3&gt;With spliced reads&lt;/h3&gt;&#xA;&lt;p&gt;Note, in practice I'm more interested in cases where the reads can be spliced, such as this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1-BASED POS 1|      10|       20|       30|&#xA;CHROMOSOME   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;BED FILE              **                    &#xA;BAM READ FWD &amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;__&amp;gt;&amp;gt;                &#xA;BAM READ REV                  &amp;lt;&amp;lt;&amp;lt;          &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where &lt;code&gt;_&lt;/code&gt; is a spliced read. The corresponding BAM can be generated with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;echo -e &amp;quot;@HD\tVN:1.4\tSO:coordinate&#xA;@SQ\tSN:I\tLN:30&#xA;first_read\t99\tI\t1\t255\t10M2N2M\t=\t20\t20\tAAAAAAAAAACC\tDDDDDDDDDDDD\tNH:i:1\tHI:i:1\tAS:i:127\tnM:i:0&#xA;first_mate\t147\tI\t20\t255\t3M\t=\t1\t20\tATC\tDDD\tNH:i:1\tHI:i:1\tAS:i:127\tnM:i:0&amp;quot; | samtools view -b &amp;gt; fwd_only.bam&#xA;samtools index fwd_only.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In that situation, it seems I need to use &lt;code&gt;-split&lt;/code&gt; and &lt;code&gt;-f 1.0&lt;/code&gt;. I thought of filtering &lt;em&gt;a posteriori&lt;/em&gt; with e.g. &lt;code&gt;awk '$10&amp;gt;0.5&lt;/code&gt;, but that doesn't seem to work either: when not using &lt;code&gt;-d&lt;/code&gt;, the output of &lt;code&gt;coverageBed&lt;/code&gt; does not separate the reads that cover the feature partially vs fully.&lt;/p&gt;&#xA;" OwnerUserId="9497" LastEditorUserId="9497" LastEditDate="2022-03-10T00:17:36.360" LastActivityDate="2022-03-10T03:03:37.427" Title="Bedtools count split reads spanning whole feature" Tags="|bedtools|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18740" PostTypeId="2" ParentId="17812" CreationDate="2022-03-07T23:54:11.833" Score="1" Body="&lt;p&gt;It is likely the sequence data in this instance, the more sequences the more chance an issue with the alignment where variations in the gap penalty are observed.&lt;/p&gt;&#xA;&lt;p&gt;MAFFT outperforms Clustal Omega to some extent over significantly larger alignments &amp;gt;50 000 sequences from published benchmark studies (FAMSA paper), but this is not relevant here.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-03-07T23:54:11.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18741" PostTypeId="2" ParentId="18728" CreationDate="2022-03-08T00:01:14.717" Score="3" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Don't know. These could be retired or inconsistent SNPs (e.g. shifted locations, or otherwise found to be different from expected). It might be that the imputation suggests that the SNP should be something other than what is observed. I can't see anything obvious from the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbSNP&lt;/a&gt; information that I've looked at.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;a) In this specific case for rs116059446, that's a SNP that has more than two variants. One way of normalising VCF files is to rearrange them so that each line only refer to a single polymorphism from the reference sequence, so if there are two non-reference variants, the variant will get two lines in the VCF file.&lt;/p&gt;&#xA;&lt;p&gt;b) For &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/rs331869&quot; rel=&quot;nofollow noreferrer&quot;&gt;rs331869&lt;/a&gt;, this is another SNP with more than two variants recorded in dbSNP. Given that you're digging through the weeds of a huge dataset, the way of thinking about this needs to change from &amp;quot;Is this common&amp;quot; to &amp;quot;Is this possible&amp;quot;. Problems are much more likely to exist with SNPs that aren't biallelic, because it indicates that something funky is going on with genetics or testing. I'll propose three possibilities for what could be happening here, in what I consider to be descending order of likelihood:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;you have one non-reference allele on one of the sister chromosome copies, and another non-reference allele on another copy&lt;/li&gt;&#xA;&lt;li&gt;you have a somatic mutation (i.e. a mutation that happened during your lifetime), and some of your cells have one variant, and others have a different variant.&lt;/li&gt;&#xA;&lt;li&gt;the probes are tagging different regions in your genome from what is expected. Probes should be location-unique, but that's impossible to guarantee for everyone, because genetic variation is a thing that happens.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In this specific case for &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/rs71634747&quot; rel=&quot;nofollow noreferrer&quot;&gt;rs71634747&lt;/a&gt;, it looks like that SNP has been retired, and has merged with &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/rs724135&quot; rel=&quot;nofollow noreferrer&quot;&gt;rs724135&lt;/a&gt;. It's possible that this is a multi-nucleotide variant where variants are only observed in tandem. This is an intronic variant, so seeing a 3-nucleotide variation is a little unexpected, but given that you're picking out exceptions from a very large dataset, not completely unexpected.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I think the main take-home from this is that you shouldn't ever expect results to be a perfect representation of reality. It can be useful to look at raw data to help understand what's going on, but there's too much weird stuff going on with biology, chemistry and physics for anything to be 100% accurate.&lt;/p&gt;&#xA;&lt;p&gt;If you did exactly the same test at a later date, some of your 23andMe results will be different. If you did imputation at a later date, with a different program version, or different imputation reference dataset, your results are likely to be different.&lt;/p&gt;&#xA;&lt;p&gt;Accuracy can be substantially improved by using different technologies to attempt to discover the same thing, and only concentrating on consistently-reported differences. You could try an Infinium Omni2.5 SNPchip, for example, or whole-genome sequencing. Unfortunately, this will lead to another problem: what should be done with the inconsistent results? The answer to &lt;em&gt;that&lt;/em&gt; is situation dependent, and is one of the hardest challenges of bioinformatics.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-03-10T03:32:09.807" LastActivityDate="2022-03-10T03:32:09.807" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18743" PostTypeId="1" CreationDate="2022-03-08T22:18:11.180" Score="1" ViewCount="301" Body="&lt;p&gt;I'm trying to adapt an existing workflow to make it more robust to failures. Below is a simplified example of the workflow.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow main {&#xA;   File filelist=&amp;quot;list_of_input_files.txt&amp;quot;&#xA;   Array[String] files=read_lines(filelist)&#xA;   scatter(file in files) {&#xA;      call task1 { input: in=file }&#xA;      if(defined(task1.out)) { call task2 { input: in=task1.out } }&#xA;      if(defined(task2.out)) { call task2 { input: in=task2.out }&#xA;   }&#xA;}&#xA;&#xA;task task1 {&#xA;   File in&#xA;   String out1 = basename(in,'.txt') + '_1.txt'&#xA;   command { if(...) { ./script1.sh &lt;span class=&quot;math-container&quot;&gt;${in} &amp;gt; $&lt;/span&gt;{out1} } }&#xA;   output { File? out = out1 }&#xA;   runtime { ... }&#xA;}&#xA;&#xA;task task2 {&#xA;   File in&#xA;   String out2 = basename(in,'.txt') + '_2.txt'&#xA;   command { if(...) {./script2.sh &lt;span class=&quot;math-container&quot;&gt;${in} &amp;gt; $&lt;/span&gt;{out2} } }&#xA;   output { File? out = out2 }&#xA;   runtime { ... }&#xA;}&#xA;&#xA;task task3 {&#xA;   File in&#xA;   String out3 = basename(in,'.txt') + '_3.txt'&#xA;   command { if(...) {./script3.sh &lt;span class=&quot;math-container&quot;&gt;${in} &amp;gt; $&lt;/span&gt;{out3} } }&#xA;   output { File? out = out3 }&#xA;   runtime {...}&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;with my list_of_input_files.txt looking like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;firstfile.txt&#xA;secondfile.txt&#xA;thirdfile.txt&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Obviously this is oversimplified, but I am trying to catch failed blocks by making the output of each task optional (using &lt;code&gt;File?&lt;/code&gt;) and then having an &lt;code&gt;if(defined(taskX.out))&lt;/code&gt; condition to call the next task. My reasoning is that if something fails (represented by the &lt;code&gt;if(...)&lt;/code&gt; within the command block of each task), then the &lt;code&gt;out&lt;/code&gt; file will never be created in that task and so won't exist as input for the next task and thus &lt;code&gt;if(defined(taskX.out))&lt;/code&gt; will resolve as false and skip the next task (and therefore all the subsequent tasks).&lt;/p&gt;&#xA;&lt;p&gt;With current testing, the first file in the list satisfies the &lt;code&gt;if(...)&lt;/code&gt; conditions of all the tasks and succeeds in giving me the final output file &lt;code&gt;firstfile_1_2_3.txt&lt;/code&gt;. However, I am getting a &lt;code&gt;Required file output 'secondfile_1.txt' does not exist'&lt;/code&gt; error message because it fails the &lt;code&gt;if(...)&lt;/code&gt; condition in &lt;code&gt;task1&lt;/code&gt; and so the output file is never created.&lt;/p&gt;&#xA;&lt;p&gt;From what I've read in the &lt;a href=&quot;https://github.com/openwdl/wdl/blob/main/versions/development/SPEC.md#declarations&quot; rel=&quot;nofollow noreferrer&quot;&gt;WDL v1.1 spec&lt;/a&gt;, optional &lt;code&gt;File&lt;/code&gt; outputs are allowed and &lt;a href=&quot;https://github.com/openwdl/wdl/blob/main/versions/development/SPEC.md#file-directory-and-optional-outputs&quot; rel=&quot;nofollow noreferrer&quot;&gt;are undefined if the files they point to don't exist&lt;/a&gt;. However, after much confused googling, I've realised it's probably because I'm using &lt;a href=&quot;https://github.com/broadinstitute/cromwell&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cromwell&lt;/a&gt;, which is stuck with WDL v1.0.&lt;/p&gt;&#xA;&lt;p&gt;My question is whether this should be working in WDL v1.0 and, if not, is my logic correct and should this (psuedo-)code work with WDL v1.1? Is there a way to achieve what I'm trying to do using pure WDL v1.0?&lt;/p&gt;&#xA;" OwnerUserId="15166" LastActivityDate="2022-05-05T09:45:47.247" Title="Handling failed blocks within scattered WDL workflow" Tags="|wdl|workflow-management|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18744" PostTypeId="2" ParentId="18738" CreationDate="2022-03-08T22:45:25.240" Score="1" Body="&lt;p&gt;Not quite sure if this is the issue you're having, but I've encountered a similar situation with coverage estimation when trying to map nanopore cDNA reads to the genome. I have a full workflow for mapping oriented long reads &lt;a href=&quot;https://dx.doi.org/10.17504/protocols.io.b54iq8ue&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, but here's a summary of the split issue:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The default options for BEDTools treat sequence deletions (which happen frequently in nanopore reads) as a drop in coverage, which can make exon hunting and coverage calculation more difficult. I submitted &lt;a href=&quot;https://github.com/arq5x/bedtools2/pull/868&quot; rel=&quot;nofollow noreferrer&quot;&gt;a pull request&lt;/a&gt; to add an additional ignoreD parameter to the command line to allow cDNA reads with split coverage across introns to ignore deletions when considering coverage; this request has now been incorporated into the main BEDtools repository (as of v2.30.0).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Depending on how your mapping is set up, adding &lt;code&gt;-ignoreD&lt;/code&gt;to your command line may allow spliced reads to be properly counted for coverage with no deletion gaps:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ bedtools genomecov -bga -strand '+' -split -ignoreD -ibam mapped_BC01.sam&#xA;chrX    0       7384244 0&#xA;chrX    7384244 7384413 1&#xA;chrX    7384413 171031299       0&#xA;&#xA;$ bedtools genomecov -bga -strand '+' -split -ignoreD -ibam mapped_BC01_exon.sam&#xA;chr11   0       62551499        0&#xA;chr11   62551499        62551595        1&#xA;chr11   62551595        62552139        0&#xA;chr11   62552139        62553212        1&#xA;chr11   62553212        122082543       0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-03-10T03:03:37.427" LastActivityDate="2022-03-10T03:03:37.427" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18745" PostTypeId="1" AcceptedAnswerId="18766" CreationDate="2022-03-08T23:33:26.323" Score="1" ViewCount="952" Body="&lt;p&gt;I have a set of BAM files where the read group tags have some (default?) values, i.e.:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@RG ID:RG0  LB:LB0  PU:PU0  SM:SM0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This creates issues in my downstream analyses, where multiple BAM files with the same SM tag are used.&lt;/p&gt;&#xA;&lt;p&gt;Samtools provides &lt;a href=&quot;http://www.htslib.org/doc/samtools-addreplacerg.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;a command&lt;/a&gt; to replace the read group tag. However, I am not sure if there are possible side-effects that I should be aware of, and I might need to remap the BAM file after this replacement. I want to replace &lt;code&gt;ID&lt;/code&gt; and &lt;code&gt;SM&lt;/code&gt; with the sample id, which is unique for each BAM file.&lt;/p&gt;&#xA;&lt;p&gt;Do I need to remap and/or run additional steps, or replacing the RG tags should be sufficient to update the BAM files in a consistent way?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2022-03-14T03:40:46.427" Title="Potential side effects of replacing read group tags in BAM file" Tags="|sam|samtools|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18746" PostTypeId="1" CreationDate="2022-03-09T00:50:41.107" Score="0" ViewCount="46" Body="&lt;p&gt;I am trying to use the following function in R:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;DBR1 &amp;lt;- readDNAStringSet(file = 'DBR1.fasta')&#xA;DBR1.string &amp;lt;- as.character(DBR1)&#xA;RepeatMatchOutput &amp;lt;- motif_detection(seqName = DBR1, chrs = '', start.position = NA,&#xA;                                     end.position = NA, motif = 'GAAGAA', nr.mismatch = 0,&#xA;                                     reverse.comp = F, print.status =F)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The error message is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error in as.vector(x) : no method for coercing this S4 class to a&#xA;vector&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I must be missing something.&lt;/p&gt;&#xA;&lt;p&gt;-- output of sessionInfo():&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;R version 4.1.2 (2021-11-01&#xA;Platform: x86_64-w64-mingw32/x64 (64-bit)&#xA;Running under: Windows 10 x64 (build 19042)&#xA;&#xA;Matrix products: default&#xA;&#xA;locale:&#xA;[1] LC_COLLATE=English_United Kingdom.1252  LC_CTYPE=English_United Kingdom.1252   &#xA;[3] LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C                           &#xA;[5] LC_TIME=English_United Kingdom.1252    &#xA;&#xA;attached base packages:&#xA;[1] stats4    stats     graphics  grDevices utils     datasets  methods   base     &#xA;&#xA;other attached packages:&#xA; [1] reshape2_1.4.4                    ggplot2_3.3.5                     gquad_2.1-1                      &#xA; [4] ape_5.6-2                         seqinr_4.2-8                      BiocManager_1.30.16              &#xA; [7] stringr_1.4.0                     STRAH_1.0                         BSgenome.Hsapiens.UCSC.hg19_1.4.3&#xA;[10] BSgenome_1.62.0                   rtracklayer_1.54.0                GenomicRanges_1.46.1             &#xA;[13] Biostrings_2.62.0                 GenomeInfoDb_1.30.1               XVector_0.34.0                   &#xA;[16] IRanges_2.28.0                    S4Vectors_0.32.3                  BiocGenerics_0.40.0              &#xA;&#xA;loaded via a namespace (and not attached):&#xA; [1] SummarizedExperiment_1.24.0 lattice_0.20-45             colorspace_2.0-3            vctrs_0.3.8                &#xA; [5] yaml_2.3.5                  utf8_1.2.2                  XML_3.99-0.8                rlang_1.0.2                &#xA; [9] pillar_1.7.0                withr_2.5.0                 glue_1.6.2                  BiocParallel_1.28.3        &#xA;[13] plyr_1.8.6                  matrixStats_0.61.0          GenomeInfoDbData_1.2.7      lifecycle_1.0.1            &#xA;[17] zlibbioc_1.40.0             MatrixGenerics_1.6.0        munsell_0.5.0               gtable_0.3.0               &#xA;[21] restfulr_0.0.13             Biobase_2.54.0              parallel_4.1.2              fansi_1.0.2                &#xA;[25] Rcpp_1.0.8                  scales_1.1.1                DelayedArray_0.20.0         Rsamtools_2.10.0           &#xA;[29] rjson_0.2.21                stringi_1.7.6               BiocIO_1.4.0                grid_4.1.2                 &#xA;[33] ade4_1.7-18                 cli_3.2.0                   tools_4.1.2                 bitops_1.0-7               &#xA;[37] magrittr_2.0.2              RCurl_1.98-1.6              tibble_3.1.6                crayon_1.5.0               &#xA;[41] pkgconfig_2.0.3             MASS_7.3-54                 ellipsis_0.3.2              Matrix_1.3-4               &#xA;[45] R6_2.5.1                    GenomicAlignments_1.30.0    nlme_3.1-153                compiler_4.1.2 &#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15167" LastEditorUserId="3694" LastEditDate="2022-11-27T09:12:48.143" LastActivityDate="2022-11-27T09:12:48.143" Title="Error in as.character.default(new(&quot;BSgenome&quot;, pkgname = &quot;BSgenome.Hsapiens.UCSC.hg19&quot;, : no method for coercing this S4 class to a vector" Tags="|r|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18747" PostTypeId="1" AcceptedAnswerId="18759" CreationDate="2022-03-09T00:59:41.300" Score="3" ViewCount="63" Body="&lt;p&gt;&lt;em&gt;de novo&lt;/em&gt; metagenomics on viral NGS data is a hot-topic. On this site alone at least 4 specific algorithms have been used to identify multi-strain/multi-species for a given data set, however these do not necessarily involve short-read data, and are listed below.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;&#xA;What approach is recommended for traditional Illumina multiple infection data both for:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;for intraspecific (intra-species)  data; and&lt;/li&gt;&#xA;&lt;li&gt;for different species co-infecting the patient, but within the same virus family?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Considerations:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Benchmarking using a common standard (&lt;a href=&quot;https://contignant.com/benchmarks&quot; rel=&quot;nofollow noreferrer&quot;&gt;for example&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Error rates, not miscalling a SNP but producing mosaic artefacts (an artificially mixed genome)&lt;/li&gt;&#xA;&lt;li&gt;Suitability of short-read data for intra-species metagenomics.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Programs discussed on Bioinformatics SE:&lt;/p&gt;&#xA;&lt;p&gt;SAVAGE &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5411778/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5411778/&lt;/a&gt; Baaijens et al. De novo assembly of viral quasispecies using overlap graphs Genome Research 2017 May; 27: 835–848.&lt;/p&gt;&#xA;&lt;p&gt;S-aligner (@juanjo75es)&#xA;&lt;a href=&quot;https://www.biorxiv.org/content/biorxiv/early/2021/02/02/2021.02.02.429443.full.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/biorxiv/early/2021/02/02/2021.02.02.429443.full.pdf&lt;/a&gt; Juanjo Bermúdez s-aligner: a greedy algorithm for non-greedy de novo genome assembly 2021 bioRχiv February: 1-7&lt;/p&gt;&#xA;&lt;p&gt;Rey &lt;a href=&quot;http://denovoassembler.sourceforge.net&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://denovoassembler.sourceforge.net&lt;/a&gt; Sébastien Boisvert et al Genome Biology. 2012 13:R122, December&lt;/p&gt;&#xA;&lt;p&gt;@LRJoshi &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/7376/viral-genome-assembly-using-broad-viral-ngs-pipeline&quot;&gt;here&lt;/a&gt; used the Broad Viral NGS pipeline for &lt;em&gt;de novo&lt;/em&gt; assembly.&#xA;&lt;a href=&quot;https://viral-ngs.readthedocs.io/en/latest/assembly.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://viral-ngs.readthedocs.io/en/latest/assembly.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;These are very small genomes &amp;lt; 30 kb. Quasi-species is a very real phenomenon (loads of mutations), however mixed infections are also well-known. In a mixed infection, producing one genome which is a mix of two genetically distinct viruses is a key concern.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-03-09T05:03:58.143" LastActivityDate="2022-03-09T19:39:12.613" Title="Metagenomics pipeline recommendations for short-read data" Tags="|ngs|covid-19|sars-cov-2|metagenome|virus|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18748" PostTypeId="1" AcceptedAnswerId="21676" CreationDate="2022-03-09T06:23:23.133" Score="2" ViewCount="86" Body="&lt;p&gt;I am learning GWAS study and PRS for predicting disease. I was aligning my sequencing data with GRCH38 reference panel. However, in the PRS analysis step, I found that most of the summary statistics are developed in GRCH37. It was a bit inconvenient for me because I have to convert vcf/snp file into GRCH37 which may end up losing some snps. I was checking this database &lt;a href=&quot;https://www.ebi.ac.uk/gwas/downloads/summary-statistics&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;.&#xA;Is there any database of summary statistics in GRCH38?&lt;/p&gt;&#xA;" OwnerUserId="10296" LastActivityDate="2023-10-11T15:34:49.173" Title="Summary statistics version problem" Tags="|gwas|polygenic-risk-score|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18750" PostTypeId="2" ParentId="18748" CreationDate="2022-03-09T08:46:34.177" Score="2" Body="&lt;p&gt;You might want to check out &lt;a href=&quot;https://github.com/hakyimlab/summary-gwas-imputation&quot; rel=&quot;nofollow noreferrer&quot;&gt;this tool&lt;/a&gt; which contains some software to &lt;a href=&quot;https://genome.sph.umich.edu/wiki/LiftOver&quot; rel=&quot;nofollow noreferrer&quot;&gt;liftover&lt;/a&gt; your coordinates from one build to another.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2022-03-09T08:46:34.177" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18751" PostTypeId="1" AcceptedAnswerId="18754" CreationDate="2022-03-09T11:45:39.827" Score="2" ViewCount="92" Body="&lt;p&gt;I am working on developing a tool to classify covid variants. So I am trying to download SARS-CoV-2 sequence data along with lineage annotation from &lt;a href=&quot;https://www.covid19dataportal.org/sequences&quot; rel=&quot;nofollow noreferrer&quot;&gt;Covid-19 Data Portal&lt;/a&gt; (I could not find a better source that provides both sequence and lineage annotated data.)&lt;/p&gt;&#xA;&lt;p&gt;Since it is a very large data (given 4,050,433 samples), my idea is to first download the metadata (&lt;code&gt;.tsv&lt;/code&gt; file) with accession ids and respective lineages from the portal, filter them to keep only the variants I am interested in, and pass the accession_ids as a text file that I would like to download to &lt;a href=&quot;https://www.covid19dataportal.org/bulk-downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;CDP File downloader&lt;/a&gt; as suggested in the portal.&lt;/p&gt;&#xA;&lt;p&gt;However, downloading metadata &lt;code&gt;.tsv&lt;/code&gt; file from the portal is taking forever (download speed is in bytes sometimes). I have tried downloading directly as a file, using &lt;code&gt;wget&lt;/code&gt; and also using &lt;code&gt;curl&lt;/code&gt;. I am not sure if there is a way to download it quicker since it is losing connection in between and restarting the download. Just as a side note, the sequence data is downloading pretty fast at 4+Mbps.&lt;/p&gt;&#xA;&lt;p&gt;I hope my question is clear. Any suggestions/tips on how can I make this happen is very much appreciated. Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="14819" LastEditorUserId="3848" LastEditDate="2022-03-09T15:22:35.773" LastActivityDate="2022-03-09T15:52:29.503" Title="Taking too long to download metadata file (estimated size 30MB) from Covid-19 Data Portal" Tags="|covid-19|data-download|api|curl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18752" PostTypeId="1" CreationDate="2022-03-09T12:31:38.500" Score="0" ViewCount="37" Body="&lt;p&gt;I'm seeing output from &lt;a href=&quot;https://github.com/VertebrateResequencing/pbwt&quot; rel=&quot;nofollow noreferrer&quot;&gt;PBWT&lt;/a&gt; that looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;3   112841  rs78923776  C   G   .   PASS    RefPanelAF=0.000153988;AN=2;AC=0;INFO=1 GT:ADS:DS:GP    0|0:0,0:0:1,0,0&#xA;3   112841  rs78923776  C   T   .   PASS    RefPanelAF=0.078534;AN=2;AC=1;INFO=1    GT:ADS:DS:GP    0|1:0,1:1:0,1,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The first line says that my genome is &lt;code&gt;CC&lt;/code&gt; at this position, but the second line (for the second alt allele) says that my genome is &lt;code&gt;CT&lt;/code&gt; at this position. OK, the first line couldn't say this, so it calls me as REF/REF, but this call has to be interpreted in the context of the second call, and can't be taken at face value.&lt;/p&gt;&#xA;&lt;p&gt;That's why I wonder if this is valid VCF or if it's actually breaking the specification.&lt;/p&gt;&#xA;&lt;p&gt;The line should be 'squashed' down to the following (I think):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;3   112841  rs78923776  C   G,T .   PASS    RefPanelAF=0.000153988,0.078534;AN=3;AC=0,1;INFO=1  GT:ADS:DS:GP    0|2:0,0:0:1,0,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8875" LastActivityDate="2022-03-09T17:47:06.633" Title="Is it valid VCF not to 'squash' positions with more than one ALT allele?" Tags="|vcf|file-formats|pbwt|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18753" PostTypeId="2" ParentId="18665" CreationDate="2022-03-09T12:55:42.223" Score="0" Body="&lt;p&gt;I post the workaround that I eventually found for this problem as it could serve as a reference for the exploration of &lt;a href=&quot;http://trna.bioinf.uni-leipzig.de/DataOutput/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tRNAdb&lt;/a&gt; bioinformatic resource.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/JSDkF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JSDkF.png&quot; alt=&quot;Just a screen shot for tdbR0000042&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you look at the picture, &lt;a href=&quot;http://trna.bioinf.uni-leipzig.de/DataOutput/&quot; rel=&quot;nofollow noreferrer&quot;&gt;tRNAdb&lt;/a&gt; provides comments about the insertion symbols. Therefore, for the specified position, nucleotides are defined. The original tRNA sequences can be reconstructed from this information.&#xA;Hope that this problem could be of help to the community!&lt;/p&gt;&#xA;" OwnerUserId="9142" LastActivityDate="2022-03-09T12:55:42.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18754" PostTypeId="2" ParentId="18751" CreationDate="2022-03-09T13:41:33.180" Score="0" Body="&lt;p&gt;30MB isn't a lot of data at all, given the issue is the EBI servers and not your institutes provision - you would have noticed this before on other projects if it was an institute issue. It therefore suggests someone else is hogging the EBI capacity and there is no provision to bypass this.&lt;/p&gt;&#xA;&lt;p&gt;The best solution is contact them directly&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;ecovid19@ebi.ac.uk&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;... and state the download size and download speed.&lt;/p&gt;&#xA;&lt;p&gt;There could be a bot sucking up the database (that is what it sounds like), which would be easy to write given FTP is permitted. They will have the ability to block an IP and NCBI have done this before albeit a long time ago.&lt;/p&gt;&#xA;&lt;p&gt;Finally filling the user experience form would help EBI &lt;a href=&quot;https://docs.google.com/forms/d/e/1FAIpQLSeF8YcPi6xxCNQQ6KJFVEHBMBAcX-mDNiWFXXoN1a5wWc-0Zw/viewform&quot; rel=&quot;nofollow noreferrer&quot;&gt;which is here&lt;/a&gt;. However the issue you described suggests escalation. NCBI are very responsive and I would be surprised if EBI did not respond quickly.&lt;/p&gt;&#xA;&lt;p&gt;Good luck with the project, sounds interesting.&lt;/p&gt;&#xA;&lt;p&gt;Generic response below if its just traditional over-demand for the service.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I suspect that the beautiful database EBI has provided (and it is amazing) has not anticipated the level of demand for the database size. They are far from being alone.&lt;/p&gt;&#xA;&lt;p&gt;Modern web architectures, such as Spotify, dedicate enormous effort to matching demand with service provision, notably using Kubernetes. When the server is overloaded they will simply duplicate their core database onto additional machines, which they will be buying in from AWS/GCP. It can even be done dynamically without intervention. NCBI are certainly heading in that direction.&lt;/p&gt;&#xA;&lt;p&gt;If the EBI wrote this under Kubernetes (it could be a monolith however), the limiting factor is the budget in computer resource allocation (again you can increase Kubernetes provision dynamically). However, if its a monolith this will not be easy and again ultimately someone has pay, i.e. EBI.&lt;/p&gt;&#xA;&lt;p&gt;Regardless, the fact you can use FTP means EBI are hosting their own service, GCP doesn't permit this and AWS is almost certainly similar. This would mean they have to physically provided additional in-house machines. I visited EBI some time ago and at that time the then director went to some lengths describing how focused EBI were on matching their in-house storage capacity to exponential rise in sequence data. Sudden popularity of a webserver is not trivial issue if it hasn't been anticipated.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Alternatives&lt;/strong&gt;&#xA;Using the NCBI service COVID service is the alternative if there is an urgent need to get the project moving. You may also wish to use GISAID &lt;a href=&quot;https://www.gisaid.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you want to use the NCBI service, I provided the detailed instructions here: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/18642/how-to-programatically-download-sars-cov-2-fasta-from-ncbi-genbank-via-api/18643#18643&quot;&gt;How to programatically download SARS-CoV-2 fasta from NCBI/Genbank via API?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is however important to understand the possible underlying causes from EBI. I do recall the then EBI director stating their biggest issue was a user simply switching to Genbank, which from their point of view isn't cool given the amount effort going into service provision.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-03-09T15:52:29.503" LastActivityDate="2022-03-09T15:52:29.503" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18755" PostTypeId="1" CreationDate="2022-03-09T16:19:13.010" Score="1" ViewCount="89" Body="&lt;p&gt;I have raw read counts of miRNAs in two conditions like below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(long1)&#xA;             miRNA Read count Condition&#xA;1:  hsa-miR-539-3p         33    living&#xA;2: hsa-miR-199b-5p        102    living&#xA;3: hsa-miR-146a-3p         27    living&#xA;4:   hsa-let-7a-3p        186    living&#xA;5:    hsa-miR-1275         54    living&#xA;6:  hsa-miR-29a-3p       6464    living&#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For each individual miRNAs, I want to show the difference of raw read counts across conditions by&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;p=ggplot(data = long1, aes(x=long1&lt;span class=&quot;math-container&quot;&gt;$miRNA, y=long1$&lt;/span&gt;`Read count`)) + geom_boxplot(aes(fill=Condition))&#xA;p &amp;lt;- p + guides(fill=guide_legend(title=&amp;quot;Condition&amp;quot;))&#xA;p &amp;lt;- p + xlab(&amp;quot;miRNAs&amp;quot;) + ylab(&amp;quot;Raw mapped reads&amp;quot;) + ggtitle(&amp;quot;s vs living with cancer&amp;quot;)&#xA;p + facet_wrap( ~ miRNA, scales=&amp;quot;free&amp;quot;)+theme_bw() add_pval(p, pairs = list(c(&amp;quot;s&amp;quot;, &amp;quot;living&amp;quot;)), test='wilcox.test')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which returns this&lt;/p&gt;&#xA;&lt;p&gt;[![enter image description here][1]][1]&lt;/p&gt;&#xA;&lt;p&gt;This plot is not good&lt;/p&gt;&#xA;&lt;p&gt;How I can a plot like below in which miRNAs are on Y axis and two colours shows the conditions like this&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0iSov.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0iSov.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2022-03-29T10:24:29.877" LastActivityDate="2022-03-29T10:24:29.877" Title="A two color bar plot in R" Tags="|r|rna-seq|ggplot2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18756" PostTypeId="2" ParentId="18755" CreationDate="2022-03-09T17:37:18.703" Score="4" Body="&lt;p&gt;Using Google well is one of the best skillsets you can possibly learn in programming-allied fields. The style of plot you're looking for is a &amp;quot;grouped bar plot&amp;quot;. I googled &amp;quot;r grouped bar plot&amp;quot; and found a nice link: &lt;a href=&quot;https://www.r-graph-gallery.com/48-grouped-barplot-with-ggplot2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.r-graph-gallery.com/48-grouped-barplot-with-ggplot2.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The key is to use &lt;code&gt;position=dodge&lt;/code&gt; alongside &lt;code&gt;color=Condition, x=&amp;quot;Read count&amp;quot;, y=miRNA&lt;/code&gt; )not sure ow you'd quote the name &lt;code&gt;&amp;quot;Read count&amp;quot;&lt;/code&gt;, might be better off renaming that column to &lt;code&gt;readcount&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="650" LastEditorUserId="650" LastEditDate="2022-03-17T21:36:00.867" LastActivityDate="2022-03-17T21:36:00.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18757" PostTypeId="1" AcceptedAnswerId="19244" CreationDate="2022-03-09T17:40:15.043" Score="1" ViewCount="109" Body="&lt;p&gt;I have &lt;a href=&quot;http://cmpg.unibe.ch/software/arlequin35/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Arlequin&lt;/a&gt; output files (*.arp) from &lt;a href=&quot;http://cmpg.unibe.ch/software/fastsimcoal27/&quot; rel=&quot;nofollow noreferrer&quot;&gt;fastSimcoal2&lt;/a&gt; that I'm trying to convert to genepop files (to read into &lt;a href=&quot;https://cran.r-project.org/web/packages/adegenet/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;adegenet&lt;/a&gt;). The Arlequin files are using the &amp;quot;DNA&amp;quot; marker (50 bp long), and each includes ~2,500 samples.&lt;/p&gt;&#xA;&lt;p&gt;When trying to use the program &lt;a href=&quot;http://www.cmpg.unibe.ch/software/PGDSpider/#Download_and_Installation_Instructions&quot; rel=&quot;nofollow noreferrer&quot;&gt;PGDspider&lt;/a&gt; to convert these (either the GUI or the CLI; v2.1.1.5), things proceed smoothly until the 1,000th allele is reached, where the program generates an error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;INFO  07-03 11:18:10,630 (GenepopWriter.java:parseAlignedSampleData:596)  -Allele &amp;quot;TGGCATCAGCTTCTTCTAAGCAGGCCCGCTTTGTCTAGCGACAGATCTCG&amp;quot; converted to &amp;quot;999&amp;quot;&#xA;INFO  07-03 11:18:10,630 (GenepopWriter.java:parseAlignedSampleData:596)  -Allele &amp;quot;ATCTTGTGGTCATGGAGCACCCAGAGCCAAACTCCTAAAGAAGACCCCGC&amp;quot; converted to &amp;quot;1000&amp;quot;&#xA;ERROR 07-03 11:18:10,630 (GenepopWriter.java:parseAlignedSampleData:611)  -data &amp;quot;1000&amp;quot; in ind &amp;quot;2_375&amp;quot; (population: Sample 2) is too long!!!&#xA;INFO  07-03 11:18:10,631 (GenepopWriter.java:parseAlignedSampleData:618)  -In GENEPOP, alleles cannot be coded with more than 3 digits.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there a way around this error? I've processed RADseq data with many 1,000s of alleles using adegenet before, so I know these types of files can be written into genpop--I just don't know how to specify this to PGDspider. There's a field in the .spid file that I might be able to use to fix this (see below), but I don't know what to populate that field with, and there aren't any suggestions from the (limited) documentation.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Specify the locus/locus combination you want to write to the GENEPOP file:&#xA;GENEPOP_WRITER_LOCUS_COMBINATION_QUESTION=&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alternatively, if there are suggestions for running this conversion without using PGDspider, I'm all ears.&lt;/p&gt;&#xA;&lt;p&gt;Thanks for your help!&lt;/p&gt;&#xA;" OwnerUserId="14041" LastEditorUserId="14041" LastEditDate="2022-03-15T14:34:30.713" LastActivityDate="2022-06-15T18:33:26.740" Title="Convert Arlequin (&quot;DNA&quot; markers) to Genepop (PGDspider or otherwise)" Tags="|linux|phylogenetics|format-conversion|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18758" PostTypeId="2" ParentId="18752" CreationDate="2022-03-09T17:47:06.633" Score="2" Body="&lt;p&gt;It is a valid multi-allelic entry, and most variant callers produce such entries. However, comparison between two such gets more complicated. With biallelic entries, 2 CHR-POS-REF-ALT entries either match or they don't. With multi-allelic entries, they can match completely, partially or not at all, which means you also need to be able to specify how much of a match is a partial match etc. This is a hindrance in annotation processes that annotate using the CHR-POS-REF-ALT fields.&lt;/p&gt;&#xA;&lt;p&gt;This also introduces cases where an indel can become a multi-allelic entry with a nearby SNV like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1    100    GA    G&#xA;chr1    101    A    AA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;can now be represented as&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1    100    GA    G,AA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which can get confusing. Operations are always &lt;em&gt;more predictable&lt;/em&gt; when VCF has been decomposed and left-aligned.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2022-03-09T17:47:06.633" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="18759" PostTypeId="2" ParentId="18747" CreationDate="2022-03-09T19:39:12.613" Score="1" Body="&lt;p&gt;Under my experience, the best ones in completeness and N50 are s-aligner, Megahit and rnaSPAdes, but I admit I didn't test the other ones for different reasons. Broad indeed seems to use internally SPAdes and Trinity. I compared the results from these three with reported results in benchmark sets from the other assemblers and the results were better for these three.&lt;/p&gt;&#xA;&lt;p&gt;From these three I get the best results with s-aligner but it has some issues: it actually requires some practice using it and it is slightly slower for this kind of data (&amp;lt;30kbp genomes). In the other hand, it usually has lower requirements in memory than other assemblers except maybe Megahit.&lt;/p&gt;&#xA;&lt;p&gt;Here there are some comparative results (NGA50) using public benchmark sets:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/GNVJn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GNVJn.png&quot; alt=&quot;Small virus assembly benchmark&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/eMs0D.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eMs0D.png&quot; alt=&quot;Large virus assembly benchmark&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Obviously, I am kinda biased as I am the author of s-aligner but the data doesn't lie.&lt;/p&gt;&#xA;&lt;p&gt;Getting the best results also may not be the first requirement in some cases, like if you want it fast and with a fast learning curve, and in this case Megahit I think is usually quite good. But in these cases, you also can design a script that will automate s-aligner for your specific needs. You can contact me if you need help.&lt;/p&gt;&#xA;&lt;p&gt;BTW, s-aligner also gets better results with other kinds of data like bacterial genomes, transcriptomes, or small eukaryotes but always at the cost of requiring more running time or more computational resources (running it in AWS, for example).&lt;/p&gt;&#xA;" OwnerUserId="5190" LastActivityDate="2022-03-09T19:39:12.613" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18760" PostTypeId="1" AcceptedAnswerId="18769" CreationDate="2022-03-09T20:30:09.820" Score="6" ViewCount="242" Body="&lt;p&gt;I am using the &lt;code&gt;internal_coords&lt;/code&gt; module from BioPython in order to compare dihedral angles of two different conformations of the same protein. The conformations are processed by me in order to make sure that their residues are aligned and they always contain exactly the same atoms, so that in the end I expect to have the same number of dihedral angles in the two structures.&lt;/p&gt;&#xA;&lt;p&gt;However, despite the match in the residues and atoms, I obtain a different number of dihedral angles for the two conformations.&lt;/p&gt;&#xA;&lt;p&gt;You can find a reproducible code sample &lt;a href=&quot;https://pastebin.com/PASnUhTa&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.PDB import *&#xA; &#xA;def extract_torsion_angles(structure):&#xA;    ic_chain = internal_coords.IC_Chain(structure)&#xA;    ic_chain.atom_to_internal_coordinates()&#xA; &#xA;    d = ic_chain.dihedra&#xA; &#xA;    &amp;quot;&amp;quot;&amp;quot; &#xA;    Update the keys of a dihedra dictionary to make searching easier.&#xA; &#xA;    The keys of the dihedra dictionary (as obtained from &#xA;    extract_torsion_angles()) are 4-tuples where each element is&#xA;    an AtomKey containing 5 fields (respos, icode, resname, atm,&#xA;    altloc, occ). We use the residue position, residue name and &#xA;    atom name in order to create a new representation for each&#xA;    AtomKey. The final dihedra dictionary will have 4-tuple keys&#xA;    where each element of the tuple is one such string.&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA; &#xA;    new_d = {}&#xA;    for key, value in sorted(d.items(), key=lambda x: x[0]):&#xA;        new_key = []&#xA;        for atom_key in key:&#xA;            # Keep: residue position + resiude name + atom name.&#xA;            atom_key_str = (atom_key.akl[0] + '_' + &#xA;                atom_key.akl[2] + '_' + atom_key.akl[3])&#xA; &#xA;            new_key.append(atom_key_str)&#xA; &#xA;        new_d[tuple(new_key)] = value&#xA; &#xA;    return new_d&#xA; &#xA; &#xA;f1 = 'protein1.pdb'&#xA;f2 = 'protein2.pdb'&#xA; &#xA;parser = PDBParser(PERMISSIVE=1, QUIET=0)&#xA;s1 = parser.get_structure('', f1)&#xA;s2 = parser.get_structure('', f2)&#xA; &#xA;c1 = list(s1.get_chains())&#xA;c2 = list(s2.get_chains())&#xA; &#xA;print('Chains: ', len(c1), ' ', len(c2))  &#xA; &#xA;r1 = list(c1[0].get_residues())&#xA;r2 = list(c2[0].get_residues())&#xA; &#xA;print('Residues: ', len(r1), ' ', len(r2))&#xA; &#xA;for i in range(len(r1)):&#xA;    resname1 = r1[i].get_resname()&#xA;    resname2 = r2[i].get_resname()&#xA; &#xA;    if resname1 != resname2:&#xA;        print('Residues not matching')&#xA;        break&#xA; &#xA;    atoms1 = list(r1[i].get_atoms())&#xA;    atoms2 = list(r2[i].get_atoms())&#xA; &#xA;    atom_names1 = sorted([atom.get_id() for atom in atoms1])&#xA;    atom_names2 = sorted([atom.get_id() for atom in atoms2])&#xA; &#xA;    if atom_names1 != atom_names2:&#xA;        print('Different atoms!')&#xA; &#xA;d1 = extract_torsion_angles(s1)&#xA;d2 = extract_torsion_angles(s2)&#xA; &#xA;print('Torsion angles: ', len(d1), ' ', len(d2))&#xA; &#xA;diff_keys = set(d2.keys()) - set(d1.keys())&#xA;print(diff_keys)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These are the two PDB files &lt;a href=&quot;https://pastebin.com/FG3hLXDW&quot; rel=&quot;nofollow noreferrer&quot;&gt;protein1.pdb&lt;/a&gt; and &lt;a href=&quot;https://pastebin.com/diFjgE3d&quot; rel=&quot;nofollow noreferrer&quot;&gt;protein2.pdb&lt;/a&gt; used.&lt;/p&gt;&#xA;&lt;p&gt;As you can see, the number of chains, the number of residues, the residue names and the atom names in each residue are the same, yet there are 4 extra dihedral angles in the second protein.&lt;/p&gt;&#xA;&lt;p&gt;My question is, am I missing something from my comparison? Is it unreasonable to expect the same dihedral angles for these two structures? (And if yes, why?). Is this the expected behaviour of the &lt;code&gt;atom_to_internal_coordinates()&lt;/code&gt; function?&lt;/p&gt;&#xA;&lt;p&gt;And extra: do you have any recommendations for other tools that can easily compute all dihedral angles (including the ones in side chains) for a protein?&lt;/p&gt;&#xA;" OwnerUserId="12036" LastEditorUserId="10049" LastEditDate="2023-11-16T19:33:05.327" LastActivityDate="2023-11-18T21:32:42.120" Title="BioPython internal_coords module returns different dihedral angles for the (seemingly) same protein structure" Tags="|python|proteins|protein-structure|pdb|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18761" PostTypeId="2" ParentId="18738" CreationDate="2022-03-10T00:16:43.427" Score="1" Body="&lt;p&gt;It appears this is an issue that has been raised on Github, and that I had unfortunately missed.&lt;/p&gt;&#xA;&lt;p&gt;This issue happens in &lt;code&gt;bedtools coverage&lt;/code&gt; (my examples above), but also in &lt;code&gt;bedtools intersect&lt;/code&gt;. So, I believe this is actually the same issue as &lt;a href=&quot;https://github.com/arq5x/bedtools2/issues/928&quot; rel=&quot;nofollow noreferrer&quot;&gt;#928&lt;/a&gt;. Indeed, switching to bedtools v2.27 solves my issue described above (with v2.30).&lt;/p&gt;&#xA;&lt;p&gt;Note that this particular issue likely appeared as a consequence of solving issues #773, #750, #673.&lt;/p&gt;&#xA;&lt;p&gt;My workaround is to use &lt;code&gt;bedtools intersect&lt;/code&gt; INVERTING the bed and bam, so that I just filter in the reads that match the feature, then count the resulting number of rows.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;intersectBed -split -bed -wb -F 1.0 \&#xA;             -a fwd_only.bam -b single_place.bed | \&#xA;  cut -f16 | \&#xA;  sort | \&#xA;  uniq -c&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9497" LastActivityDate="2022-03-10T00:16:43.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18762" PostTypeId="1" CreationDate="2022-03-10T16:09:37.927" Score="6" ViewCount="1216" Body="&lt;p&gt;I have always thought that &lt;code&gt;1/0&lt;/code&gt; and &lt;code&gt;0/1&lt;/code&gt; in VCF genotype fields are equivalent. And yet, GATK uses both. For example, these are two variants called in the same sample and the same run of GATK 4.1.4.0:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr7    117120317   .   ATTCATTGTTTTGAAAGAAAGATGGAAGAATGAACTGAAG    A   748.97  .   AC=1;AF=0.5;AN=2;DP=64;ExcessHet=3.0103;FS=0;MLEAC=1;MLEAF=0.5;MQ=60;QD=11.89;SOR=7.223 GT:AD:DP:GQ:PL:SB   1/0:0,36:63:99:2294,1042,933:0,0,0,36&#xA;chr7    117120306   .   GACGCGCT    G   516.6   .   AC=1;AF=0.5;AN=2;BaseQRankSum=0.826;DP=70;ExcessHet=3.0103;FS=0;MLEAC=1;MLEAF=0.5;MQ=60;MQRankSum=0;QD=8.2;ReadPosRankSum=-5.786;SOR=0.976  GT:AD:DP:GQ:PL:SB   0/1:37,26:63:99:524,0,1203:0,20,0,26&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And here are the same lines redacted for clarity:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr7    117120317   .   AT..AG  A   [...]   GT:AD:DP:GQ:PL:SB   1/0:0,36:63:99:2294,1042,933:0,0,0,36&#xA;chr7    117120306   .   GA..CT  G   [...]   GT:AD:DP:GQ:PL:SB   0/1:37,26:63:99:524,0,1203:0,20,0,26&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see, one is genotyped as &lt;code&gt;1/0&lt;/code&gt; and the other as &lt;code&gt;0/1&lt;/code&gt;. &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The VCF specs&lt;/a&gt; say:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;GT (String): Genotype, encoded as allele values separated by either of / or |. The allele values are 0 for the&#xA;reference allele (what is in the REF field), 1 for the first allele listed in ALT, 2 for the second allele list in&#xA;ALT and so on. For diploid calls examples could be 0/1, 1 | 0, or 1/2, etc. Haploid calls, e.g. on Y, male&#xA;non-pseudoautosomal X, or mitochondrion, are indicated by having only one allele value. A triploid call might&#xA;look like 0/0/1. If a call cannot be made for a sample at a given locus, ‘.’ must be specified for each missing&#xA;allele in the GT field (for example ‘./.’ for a diploid genotype and ‘.’ for haploid genotype). The meanings of&#xA;the separators are as follows (see the PS field below for more details on incorporating phasing information into&#xA;the genotypes):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;/ : genotype unphased&lt;/li&gt;&#xA;&lt;li&gt;| : genotype phased&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This suggests that although &lt;code&gt;1|0&lt;/code&gt; is not the same as &lt;code&gt;0|1&lt;/code&gt;, the &lt;em&gt;unphased&lt;/em&gt; &lt;code&gt;0/1&lt;/code&gt; and &lt;code&gt;1/0&lt;/code&gt; genotypes are completely equivalent. Which makes perfect sense given that the &lt;code&gt;N/N&lt;/code&gt; genotypes are unphased, so &lt;code&gt;n/N&lt;/code&gt; must be equivalent to &lt;code&gt;N/n&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So, my questions are: why does gatk produce some lines with &lt;code&gt;0/1&lt;/code&gt; and others with &lt;code&gt;1/0&lt;/code&gt;? Is it a bug? Am I missing something? I can't imagine a developer writing code to pick either &lt;code&gt;0/1&lt;/code&gt; or &lt;code&gt;1/0&lt;/code&gt; randomly, so presumably they indicate some difference in what gatk sees about a variant. So are &lt;code&gt;0/1&lt;/code&gt; and &lt;code&gt;1/0&lt;/code&gt; not 100% equivalent somehow?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;sub&gt; I have seen &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/14356/298&quot;&gt;How is the GT field in a VCF file defined?&lt;/a&gt; which suggests the two are equivalent, but this seems to be contradicted by GATK's own output which is why I am asking this again to get a clear and preferably referenced answer.&lt;/sub&gt;&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2022-03-10T22:17:58.787" LastActivityDate="2022-03-23T16:17:18.633" Title="Why does GATK produce both 0/1 and 1/0 genotypes in the same file? Are the two not equivalent?" Tags="|vcf|gatk|genotyping|phasing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18763" PostTypeId="1" CreationDate="2022-03-10T21:06:06.480" Score="3" ViewCount="484" Body="&lt;p&gt;Recent gnomAD versions include a &amp;quot;filtering allele frequency&amp;quot; which tells you when a variant can be safely adjudged not to be disease-causing. Unfortunately, I'm having trouble making sense of the definition, which is&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Technically, this is the highest disease-specific maximum credible population AF for which the observed AC is not compatible with pathogenicity. More practically, If the filter allele frequency of a variant is above the maximum credible population AF for a condition of interest, then that variant should be filtered (ie not considered a candidate causative variant). See &lt;a href=&quot;http://cardiodb.org/allelefrequencyapp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cardiodb.org/allelefrequencyapp/&lt;/a&gt; and Whiffin et al. 2017 for additional information.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I interpret this statement as &amp;quot;If&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;gnomAD_WG_AF_{population} &amp;gt; gnomAD_WG_FAF95_{population},&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;then the variant should be filtered.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;(Here &amp;quot;FAF95&amp;quot; stands for &amp;quot;filtering allele frequency at 95% confidence&amp;quot;.) However, the above inequality seems to hold for every variant in gnomAD, so I'm clearly missing something. On the other hand, perhaps the statement &amp;quot;maximum credible population AF for a condition of interest&amp;quot; means &amp;quot;proportion of people in the population who have the condition,&amp;quot; but in that case the use of the term AF doesn't make sense. Does anyone have a better understanding of this FAF95 number than I do?&lt;/p&gt;&#xA;" OwnerUserId="15178" LastEditorUserId="15178" LastEditDate="2022-03-10T22:23:17.150" LastActivityDate="2023-08-07T16:58:14.523" Title="What is the meaning of &quot;Filtering allele frequency&quot; in the gnomAD database?" Tags="|phylogenetics|allele-frequency|variant-filtration|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18765" PostTypeId="2" ParentId="18745" CreationDate="2022-03-11T10:14:10.200" Score="3" Body="&lt;p&gt;I assume you are using GATK software tools for downstream analysis. If that's the case, then you will most likely encounter issues due to missing information on read groups. I believe that GATK can't differentiate between read groups if they are all similar.&lt;/p&gt;&#xA;&lt;p&gt;In all cases, you can find more information about &lt;a href=&quot;https://samtools.github.io/hts-specs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAM/BAM and related specifications&lt;/a&gt; on the &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;1.3 The header section&lt;/a&gt; as they describe what is mandatory.&lt;/p&gt;&#xA;&lt;p&gt;So, if you have access to FastQ files, you can add them directly during the alignment process with &lt;code&gt;bwa mem&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://bio-bwa.sourceforge.net/bwa.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;bwa mem&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;-R STR    Complete read group header line. ’\t’ can be used in STR and will be converted to a TAB in the output SAM. The read group ID will&#xA;be attached to every read in the output. An example is&#xA;’@RG\tID:foo\tSM:bar’. [null]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;An example command line would be:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$bwa mem \&#xA;-M \                 // Mark shorter split hits as secondary (for Picard compatibility).&#xA;-t &amp;quot;$cpus&amp;quot; \         // number of threads&#xA;&amp;quot;$genome&amp;quot; \          // genome reference&#xA;&amp;quot;$fastq_r1&amp;quot; \        // R1&#xA;&amp;quot;$fastq_r2&amp;quot; \        // provide this line if you have R2&#xA;-R &amp;quot;@RG\tID:1\tLB:lib1\tPL:illumina\tSM:$sampleName\tPU:runBarcode&amp;quot; \&#xA;&amp;gt; &amp;quot;$sampleName&amp;quot;.sam  // Output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/bwa-mem2/bwa-mem2&quot; rel=&quot;nofollow noreferrer&quot;&gt;bwa2 mem&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Bwa-mem2 is the next version of the bwa-mem algorithm in bwa. It&#xA;produces alignment identical to bwa and is ~1.3-3.1x faster depending&#xA;on the use-case, dataset and the running machine.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The command line is exactly similar to the command line above, except that you would be using &lt;code&gt;bwa2&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://www.htslib.org/doc/samtools-addreplacerg.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Samtools&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;As you have mentioned, you can completely overwrite (default mode) the read groups already existing by doing:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;samtools addreplacerg \&#xA;-r &amp;quot;@RG\tID:1\tLB:lib1\tPL:illumina\tSM:$sampleName\tPU:runBarcode&amp;quot; \&#xA;-m overwrite_all \         // mode to replace all&#xA;-@ 5 \                     // number of cpus&#xA;-O b \                     // output bam&#xA;-o &amp;quot;$sampleName&amp;quot;_RG.bam \&#xA;&amp;quot;$sampleName&amp;quot;.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360037226472-AddOrReplaceReadGroups-Picard-&quot; rel=&quot;nofollow noreferrer&quot;&gt;Picard AddOrReplaceReadGroups&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;java -jar $picard AddOrReplaceReadGroups \     // Also available in GATK tools&#xA;I=&amp;quot;$sampleName&amp;quot;.bam \&#xA;OUTPUT=&amp;quot;$sampleName&amp;quot;_RG.bam \&#xA;RGID=1 \&#xA;RGLB=library \&#xA;RGPL=illumina \&#xA;RGPU=runBarcode \&#xA;RGSM=${sampleName} \&#xA;SORT_ORDER=coordinate&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2022-03-11T10:14:10.200" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18766" PostTypeId="2" ParentId="18745" CreationDate="2022-03-11T12:27:49.440" Score="3" Body="&lt;p&gt;You do not need to remap the files, replacing the read group information with &lt;code&gt;samtools&lt;/code&gt; is sufficient to deal with this. When you update your pipeline, have it use the sample name to construct the read group information. That way you won't run into this problem again.&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2022-03-11T12:27:49.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18767" PostTypeId="1" AcceptedAnswerId="18768" CreationDate="2022-03-11T13:14:23.003" Score="3" ViewCount="167" Body="&lt;p&gt;After sequencing (Illumina) of some DNA, I generated a sam file through alignment of a fastq file (using Bowtie2). Instead of using variant calling programs, I want to know the specific variances, compared to the reference genome, in each specific read.&lt;/p&gt;&#xA;&lt;p&gt;For example, I'm only interested in C to A mutations. I would like to include reads that contain MORE than 5 C to A mutations. Afterwards I would like to analyze the specific sequence context of that mutation and for doing that I need to know the 3 nucleotides before and after the C to A mutation, which gives a context of NNNCNNN.&lt;/p&gt;&#xA;&lt;p&gt;I looked at samtools mpileup or bcftools, but I can't find the exact thing that I want to analyze. Any ideas?&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="15183" LastActivityDate="2022-03-13T12:07:05.563" Title="Detect mutation context in a read of a sam file" Tags="|sam|samtools|variant-calling|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18768" PostTypeId="2" ParentId="18767" CreationDate="2022-03-11T13:53:28.953" Score="3" Body="&lt;p&gt;That kind of custom fiddling with reads and variants is very cumbersome, non-standard and also error-prone. Do a standard variant calling pipeline and then filter for the mutations that you want. Then extract the variant position (so the coordinates) and get the variant context from the reference genome. Using individual reads for this is not advisable as only a properly called variant is reliable. The power of NGS is the validation of an observation by redundancy (=many reads covering a variant). With individual reads your variant can simply be noise, there is hardly a way to tell. Alignment will confidently map the read to the &amp;quot;context&amp;quot; as you call it, so in my opinion there is not really an advantage of calling the &amp;quot;context&amp;quot; directly from individual reads. You could better extract every confident C&amp;gt;A mutation, then get the three bases up/downstream and then make some kind of position frequency matrix to create a motif.&lt;/p&gt;&#xA;&lt;p&gt;Example, just for illustration, imagine these were for variants (A in the middle) with three bases up/downstream:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CTAATTG&#xA;CTCATAG&#xA;CTTATAG&#xA;CCTAGTG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The motif would then be this, you can easily see whether there is some conservation in the &amp;quot;context&amp;quot; of your variant.ad&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WaU9C.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WaU9C.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The variant calling you could do with an end-to-end pipeline such as &lt;a href=&quot;https://nf-co.re/sarek&quot; rel=&quot;nofollow noreferrer&quot;&gt;nf-core sarek&lt;/a&gt;, this will eventually give you a VCF file(s) with the variant coordinates. From there simply filter for the variants you want, get the coordinate, add three bases up/downstream to start/end and then use something like &lt;code&gt;bedtools getfasta&lt;/code&gt; to get the sequences from the reference genome.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorUserId="298" LastEditDate="2022-03-12T13:38:05.060" LastActivityDate="2022-03-12T13:38:05.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18769" PostTypeId="2" ParentId="18760" CreationDate="2022-03-11T15:07:00.693" Score="7" Body="&lt;p&gt;The problem got solved after posting on the official BioPython repo.&lt;/p&gt;&#xA;&lt;p&gt;Because of the processing (where some residues were removed in order to align the proteins), some consecutive residues were too far away from each other and thus the bond length was too large. The solution was to increase the maximum peptide bond allowed, as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;internal_coords.IC_Chain.MaxPeptideBond = 5.0&#xA;ic_chain = internal_coords.IC_Chain(structure, verbose=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12036" LastActivityDate="2022-03-11T15:07:00.693" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18770" PostTypeId="2" ParentId="18767" CreationDate="2022-03-12T13:37:30.597" Score="1" Body="&lt;p&gt;If you want to look at a few, specific reads manually, you can do this on &lt;a href=&quot;https://varsome.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VarSome&lt;/a&gt;. We have recently released a new feature that allows you to enter single NGS or Sanger reads and get the list of variants detected in that read. For example, &lt;a href=&quot;https://varsome.com/single-read/hg38/AGTCCRAGTTGTAAATGGTACACTCGGCGTAAGCCTGAAAAGATAAAATCAAAGATGTAAAGGTGAGCACAGTCTAAGTTCTCTCTGAAGTGTCAATGGGAATGCAGATTGGATTAAATAAATGCTGCCCAAGTGCATACTCAAAGAGGC&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pawMJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pawMJ.png&quot; alt=&quot;screenshot of VarSome showing an aligned read&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this example, there was one variant identified when comparing to the human hg38 genome assembly:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://varsome.com/variant/hg38/CYP24A1(NM_000782.5):c.1186C%3ET&quot; rel=&quot;nofollow noreferrer&quot;&gt;CYP24A1(NM_000782.5):c.1186C&amp;gt;T&lt;/a&gt; which is at chr20:54158136 G⇒A (note that the CYP24A1 gene is on the - strand which is why the C to T variant in HGVS notation is a G to A in the genome). You can then click on the variant link(s) to investigate the variant's effect and annotation.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Obviously, this isn't practical for many reads, but it can be handy if you just want to look at a few manually. More importantly, NGS is very error prone (which is why we sequence at high depth), and you should absolutely not assume that just because a variant is present in a single read it was also actually present in the source DNA you sequenced.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Disclaimer: I work for the company that builds VarSome.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2022-03-13T12:07:05.563" LastActivityDate="2022-03-13T12:07:05.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18771" PostTypeId="1" CreationDate="2022-03-13T14:01:39.117" Score="1" ViewCount="134" Body="&lt;p&gt;I'm trying to do pan-genome analysis using BPGA and I face this error:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nyyXc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nyyXc.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aIn26.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aIn26.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kijEl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kijEl.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This error didn't show up in the other data set where all the .gbk is downloaded from NCBI. This .gbk file is our own data and has not been published in NCBI yet. There's of course protein data in it, there's this CDS qualifier &amp;quot;translation&amp;quot;, but somehow BPGA didn't detect this.&lt;/p&gt;&#xA;&lt;p&gt;The difference between our data and NCBI data is that most of the NCBI .gbk files are annotated with PGAP. On the other hand, our .gbk is annotated with Prokka (have tried it in DFAST and KBase). NCBI .gbk files that are annotated with PGAP have this CDS qualifier &amp;quot;protein_id&amp;quot; in them. Besides that, I don't think there's any difference.&lt;/p&gt;&#xA;&lt;p&gt;Does the BPGA script counts the protein number based on the &amp;quot;protein_id&amp;quot; qualifier? How should I solve this? I can't annotate the .fasta file with PGAP because it's so resource-intensive. I believe that, if Prokka is installed locally, there's this &lt;a href=&quot;https://github.com/tseemann/prokka/issues/121#issuecomment-113965257&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;--compliant&amp;quot;&lt;/a&gt; option that can integrate the &amp;quot;protein_id&amp;quot; qualifier into the output .gbk file. I have tried to install Prokka locally, but it didn't run well too, I encountered another &lt;a href=&quot;https://github.com/bioconda/bioconda-recipes/issues/10277&quot; rel=&quot;nofollow noreferrer&quot;&gt;error&lt;/a&gt; for that and all the solutions didn't work for me.&lt;/p&gt;&#xA;&lt;p&gt;Any opinion on this issue? Thanks for reading my question.&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&#xA;I tried to insert one dummy &amp;quot;protein_id&amp;quot; on my Prokka-annotated .gbk files and it run perfectly without errors.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wR2XP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wR2XP.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But when I tried to insert dummy protein_id on my DFAST-annotated .gbk files, it won't run and faced another error. What is this &amp;quot;protein_id&amp;quot; really defined by the script?&lt;/p&gt;&#xA;" OwnerUserId="14560" LastEditorUserId="14560" LastEditDate="2022-03-17T05:03:18.360" LastActivityDate="2022-05-30T07:40:30.520" Title="BPGA error: there was no protein data in file" Tags="|perl|sequence-annotation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18772" PostTypeId="1" AcceptedAnswerId="18773" CreationDate="2022-03-13T21:37:21.673" Score="2" ViewCount="282" Body="&lt;p&gt;I need a statistically significant &amp;quot;decision&amp;quot; that DEGs from group A are different (or not) from group B (after drug treatment).&#xA;For that, I have a mean fold change delta, calculated as the mean of 5 different treated samples minus the mean of the same, but untreated samples.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;        A.delta    B.delta&#xA;gen1    4.9182385  4.5472662&#xA;gen2    3.6354761  3.4474844&#xA;gen3    3.7592630  3.2838886&#xA;gen4    3.9623264  2.8189166&#xA;gen5    3.4742171  3.1231050&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In my vision, there is not much difference in fold change delta between A and B for the &lt;code&gt;gen1&lt;/code&gt; for example (4.91 is not very far from 4.54). Same for &lt;code&gt;gen2&lt;/code&gt;. While for &lt;code&gt;gen4&lt;/code&gt; there might be something interesting.&#xA;What kind of plot may be the most useful for that kind of comparison?&lt;/p&gt;&#xA;" OwnerUserId="6470" LastActivityDate="2022-03-14T03:18:42.937" Title="How to compare differential expression of same genes between two groups?" Tags="|differential-expression|" AnswerCount="1" CommentCount="2" ClosedDate="2022-03-14T09:56:54.917" ContentLicense="CC BY-SA 4.0" />
  <row Id="18773" PostTypeId="2" ParentId="18772" CreationDate="2022-03-14T01:54:08.297" Score="0" Body="&lt;p&gt;This data is genuinely paired data and you have stated values are a mean. Thus the raw data can be analysed in your previous questions on the same theme described here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/18660/what-is-the-best-way-to-compare-treated-control-samples&quot;&gt;What is the best way to compare treated/control samples?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/18732/what-is-the-best-way-to-compare-differences-in-deg-expression-between-two-popula&quot;&gt;What is the best way to compare differences in DEG expression between two populations?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If this is 'ANOVAerable' (its not a word, but if ANOVA is appropriate) than a two-way ANOVA would provide all the answers (however see below regarding the trend in the data). This analysis will perform a 'A' versus 'B' and each 'gen' compared to the other, as well as the interaction between A/B and 'gen'. You would need the raw data, you can't do anything with means (unless you are assuming the distribution). The distribution within each 'gen' is important (if its normal - conforms to the normal distribution, you're in business).&lt;/p&gt;&#xA;&lt;p&gt;There is nothing further to add and I do feel we need to draw this series of questions to a close.&lt;/p&gt;&#xA;&lt;p&gt;A regression model could be explored because there is a clear trend in both data sets, but I'm uncertain it will be significant. Within ANOVA however this might trap the outlier, but that is more complicated (you standardise against the trend thus 'gen1' is comparable to 'gen5' and therein 'gen4' can be identified as an outlier). This was touched on in the answer to the previous question (ols and python)&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-03-14T03:18:42.937" LastActivityDate="2022-03-14T03:18:42.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18774" PostTypeId="1" AcceptedAnswerId="19074" CreationDate="2022-03-14T04:43:34.647" Score="3" ViewCount="459" Body="&lt;p&gt;I am trying to run an R script as a task in WDL in a docker image. Since this is my first time working with these, I am stuck in something very simple (and in the initial stage) and am hoping somebody here can point me in the right direction.&#xA;Here is my dockerfile.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FROM rocker/r-base:latest&#xA;RUN Rscript ...r-docker/install_packages.R&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here is my WDL script&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;version 1.0&#xA;&#xA;workflow Rpipe {&#xA;   input {&#xA;     String name&#xA;   }&#xA;  call PrintNameR{&#xA;        input:&#xA;        name = name&#xA;}&#xA;  output {&#xA;     File outfile = PrintNameR.output_R&#xA;  }&#xA;}&#xA;&#xA;task PrintNameR {&#xA;      input {&#xA;           String name&#xA;}&#xA;&#xA;command&amp;lt;&amp;lt;&amp;lt;&#xA;    R --no-save --args name &amp;lt;&amp;lt;Rscript&#xA;    args &amp;lt;- commandArgs(trailingOnly = TRUE)&#xA;    for (i in args){&#xA;    print(i)}&#xA;    Rscript&#xA;  &amp;gt;&amp;gt;&amp;gt;&#xA;&#xA;runtime {&#xA;            docker: &amp;quot;firsttrydocker&amp;quot;&#xA;       }&#xA;output {&#xA;     File output_R = stdout()&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I validated, generated an input json file - edited it to include the string I want &amp;quot;Smith&amp;quot;, and ran it.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar ../cromwell/womtool-77.jar validate rtry1.wdl &#xA;java -jar ..cromwell/womtool-77.jar inputs rtry1.wdl &amp;gt; rtry1.json &#xA;java -jar ..cromwell/cromwell-77.jar run rtry1.wdl --inputs rtry1.json&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get an output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; args &amp;lt;- commandArgs(trailingOnly = TRUE)&#xA;&amp;gt; for (i in args){&#xA;+ print(i)}&#xA;[1] &amp;quot;name&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get &amp;quot;name&amp;quot;, I wanted to get &amp;quot;Smith&amp;quot; because that's what my input is.&lt;/p&gt;&#xA;&lt;p&gt;What am I doing wrong? Clearly, it seems that I am not passing the WDL task input to the WDL task command -in the Rscript. What is the proper way to do it?&lt;/p&gt;&#xA;" OwnerUserId="15197" LastActivityDate="2022-05-17T02:01:34.363" Title="Wrapping R scripts in WDL and run in docker" Tags="|r|wdl|docker|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18775" PostTypeId="1" CreationDate="2022-03-14T19:05:17.567" Score="1" ViewCount="137" Body="&lt;p&gt;I am new to RNA sequencing analysis. I have RNA samples for bacteria for different treatment. I did contaminant filtration, remove adapters and checked fastQC report: attached below for one sample (fwd and rev) &lt;a href=&quot;https://i.stack.imgur.com/SlasJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SlasJ.png&quot; alt=&quot;fwd&quot; /&gt;&lt;/a&gt;. &lt;a href=&quot;https://i.stack.imgur.com/IjGT8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IjGT8.png&quot; alt=&quot;reverse&quot; /&gt;&lt;/a&gt; After mapping with the reference genome using bbsplit but the mapping is only 0.62%.&lt;/p&gt;&#xA;&lt;p&gt;Any help/suggestions highly appreciated.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="15204" LastActivityDate="2022-08-30T00:06:45.523" Title="Low mapping reads for RNA sequence" Tags="|rna-seq|read-mapping|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18776" PostTypeId="2" ParentId="18774" CreationDate="2022-03-14T20:05:20.967" Score="4" Body="&lt;p&gt;I figured I had to pass ~{sep=&amp;quot; &amp;quot; name} as an argument&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;command&amp;lt;&amp;lt;&amp;lt;&#xA;R --no-save --args ~{sep=&amp;quot; &amp;quot; name} &amp;lt;&amp;lt;Rscript&#xA;args &amp;lt;- commandArgs(trailingOnly = TRUE)&#xA;for (i in args){&#xA;print(i)}&#xA;Rscript&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="15197" LastActivityDate="2022-03-14T20:05:20.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18777" PostTypeId="2" ParentId="13156" CreationDate="2022-03-14T20:40:20.363" Score="1" Body="&lt;p&gt;I’ve seen cases like this. For me, there were plasmids at end that didn’t get aligned.&lt;/p&gt;&#xA;" OwnerUserId="15205" LastActivityDate="2022-03-14T20:40:20.363" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18778" PostTypeId="1" CreationDate="2022-03-15T02:10:07.697" Score="1" ViewCount="41" Body="&lt;p&gt;I am looking for tidy tabular dataset that contains known bacteria and fungi down to the species level. Basically a table that includes genus and species names and other higher-level information. For example, whether it is gram-positive or negative, or anaerobic etc. A database with API would work too.&lt;/p&gt;&#xA;&lt;p&gt;I found &lt;a href=&quot;https://bacteria.ensembl.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bacteria.ensembl.org/&lt;/a&gt;, but this is not so easy to use/automate.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2022-03-15T08:26:06.510" Title="Database to download with information of bacteria?" Tags="|python|database|public-dataset|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18779" PostTypeId="2" ParentId="18778" CreationDate="2022-03-15T08:26:06.510" Score="1" Body="&lt;p&gt;You can try Entrez in Biopython. Although, there is hassle with additional information.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez&#xA;&#xA;Entrez.email = &amp;quot;your.letterbox@any.com&amp;quot;&#xA;Entrez.api_key = &amp;quot;your_key&amp;quot;&#xA;&#xA;TaxID = Entrez.read(Entrez.esearch(db=&amp;quot;taxonomy&amp;quot;, term=&amp;quot;Hepatovirus A&amp;quot;))[&amp;quot;IdList&amp;quot;][0]&#xA;Entrez.read(Entrez.efetch(db=&amp;quot;taxonomy&amp;quot;, id=TaxID))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15161" LastActivityDate="2022-03-15T08:26:06.510" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18780" PostTypeId="1" CreationDate="2022-03-15T08:34:44.287" Score="2" ViewCount="174" Body="&lt;p&gt;Im trying to install the missing modules of Circos without success.&lt;/p&gt;&#xA;&lt;p&gt;The error code is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cpan[1]&amp;gt; install Config::General&#xA;Reading '/home/asg28/.cpan/Metadata'&#xA;  Database was generated on Mon, 14 Mar 2022 03:29:03 GMT&#xA;Running install for module 'Config::General'&#xA;Fetching with HTTP::Tiny:&#xA;http://www.cpan.org/authors/id/T/TL/TLINDEN/Config-General-2.63.tar.gz&#xA;Fetching with HTTP::Tiny:&#xA;http://www.cpan.org/authors/id/T/TL/TLINDEN/CHECKSUMS&#xA;Checksum for /home/asg28/.cpan/sources/authors/id/T/TL/TLINDEN/Config-General-2.63.tar.gz ok&#xA;Scanning cache /home/asg28/.cpan/build for sizes&#xA;............................................................................DONE&#xA;'YAML' not installed, will not store persistent state&#xA;Configuring T/TL/TLINDEN/Config-General-2.63.tar.gz with Makefile.PL&#xA;Checking if your kit is complete...&#xA;Looks good&#xA;Generating a Unix-style Makefile&#xA;Writing Makefile for Config::General&#xA;Writing MYMETA.yml and MYMETA.json&#xA;  TLINDEN/Config-General-2.63.tar.gz&#xA;  /usr/bin/perl Makefile.PL INSTALLDIRS=site -- OK&#xA;Running make for T/TL/TLINDEN/Config-General-2.63.tar.gz&#xA;Can't exec &amp;quot;make&amp;quot;: No such file or directory at /usr/share/perl/5.30/CPAN/Distribution.pm line 2225.&#xA;  TLINDEN/Config-General-2.63.tar.gz&#xA;  make -- NOT OK&#xA;  No such file or directory&#xA;Failed during this command:&#xA; TLINDEN/Config-General-2.63.tar.gz           : make NO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm running Ubuntu in Windows and already tried in another computer&lt;/p&gt;&#xA;" OwnerUserId="15207" LastEditorUserId="3848" LastEditDate="2022-03-15T09:37:39.060" LastActivityDate="2022-04-14T11:07:15.707" Title="Installing modules in perl for Circos" Tags="|perl|molecular-genetics|circos|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18781" PostTypeId="2" ParentId="18780" CreationDate="2022-03-15T10:06:11.370" Score="1" Body="&lt;p&gt;The simplest solution is to install conda/Anaconda&lt;/p&gt;&#xA;&lt;p&gt;Then type any one of the following. Start with the top command and if it is unavailable go to the next commend down - only needs work once:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c bioconda circos &#xA;conda install -c bioconda/label/broken circos &#xA;conda install -c bioconda/label/cf201901 circos &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is a preferred solution because the &lt;code&gt;make&lt;/code&gt; will already be configured along with any other dependency and in addition you can run it directly on Windows without an intervening simulator as described here: &lt;a href=&quot;https://docs.anaconda.com/anaconda/install/windows/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.anaconda.com/anaconda/install/windows/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You could install &lt;code&gt;brew&lt;/code&gt; but this would be a bit messy,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/bin/bash -c &amp;quot;$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;because you would be using a perl environment within a simulator.&lt;/p&gt;&#xA;&lt;p&gt;The alternative is to do what @user324810 and install &lt;code&gt;make&lt;/code&gt; explicitly and is described here,&#xA;&lt;a href=&quot;https://stackoverflow.com/questions/23293589/perl-module-install-error-cpan&quot;&gt;https://stackoverflow.com/questions/23293589/perl-module-install-error-cpan&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The error is easy enough but there will be future dependencies which will not be installed and a once an for all solution is a better option and diagnosing and installing each dependency in turn.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-03-15T10:06:11.370" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="18784" PostTypeId="1" CreationDate="2022-03-15T15:22:11.990" Score="3" ViewCount="79" Body="&lt;p&gt;When using &lt;a href=&quot;http://homer.ucsd.edu/homer/ngs/quantification.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;annotatePeaks.pl script&lt;/a&gt; from Homer software to create histograms, the output is normalized per bp per peak (on top of normalization to 10 million tags).&lt;/p&gt;&#xA;&lt;p&gt;What does it mean to normalize per bp per peak?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://homer.ucsd.edu/homer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hypergeometric Optimization of Motif EnRichment&lt;/a&gt; (HOMER v4.11, 10-24-2019), is used for &lt;em&gt;de novo&lt;/em&gt; motif discovery and NGS analysis and written in Perl and C++.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KzP0f.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KzP0f.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Crossposted from &lt;a href=&quot;https://www.biostars.org/p/9514514/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, as there is no reply on biostars.&lt;/p&gt;&#xA;" OwnerUserId="6317" LastEditorUserId="3848" LastEditDate="2022-03-19T14:54:10.003" LastActivityDate="2022-03-19T14:54:10.003" Title="A question on Homer normalization" Tags="|ngs|normalization|homer|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18786" PostTypeId="1" AcceptedAnswerId="18791" CreationDate="2022-03-15T20:04:23.490" Score="2" ViewCount="544" Body="&lt;p&gt;I'm remarkably still struggling with some basic &lt;code&gt;Nextflow&lt;/code&gt; concepts. I'm simply trying to collect all pairs of &lt;code&gt;.cram&lt;/code&gt; and &lt;code&gt;.crai&lt;/code&gt; files from a directory using &lt;code&gt;Channel.fromFilePairs(...)&lt;/code&gt;, but I get no output when I try to &lt;code&gt;view()&lt;/code&gt; the &lt;code&gt;Channel&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt; Why does my &lt;code&gt;Channel.fromFilePairs(...)&lt;/code&gt; command not return file pairs, but I get a list of files when using &lt;code&gt;Channel.fromPath(...)&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;As far as I can tell, I'm following the same pattern from the following examples:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#fromfilepairs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nextflow fromFilePairs&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/nextflow-io/patterns/blob/master/docs/process-per-file-pairs.adoc&quot; rel=&quot;nofollow noreferrer&quot;&gt;.bam/.bai example&lt;/a&gt; (except without the extra code)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h3&gt;Code&lt;/h3&gt;&#xA;&lt;pre&gt;&lt;code&gt;nextflow.enable.dsl=2&#xA;&#xA;workflow {&#xA;&#xA;    /*&#xA;     * Channel.fromPath returns the expected files and prints them to&#xA;     * terminal.&#xA;     */&#xA;    Channel.fromPath('/path/to/adsp/cram/snd10000/A-CUHS-CU000*.cram')&#xA;        | view()&#xA; &#xA;    /*&#xA;     * Channel.fromFilePairs does not print anything to terminal.&#xA;     */&#xA;    Channel.fromFilePairs('/path/to/adsp/cram/snd10000/A-CUHS-CU000*.{cram,crai}')&#xA;        | view()&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3&gt;Output (note, no file tuples/pairs)&lt;/h3&gt;&#xA;&lt;pre&gt;&lt;code&gt;N E X T F L O W  ~  version 21.04.3&#xA;Launching `pairs.nf` [pensive_plateau] - revision: f917a5c41b&#xA;/path/to/adsp/cram/snd10000/A-CUHS-CU000563-BL-COL-47451BL1_vcpa1.1.cram.crai&#xA;/path/to/adsp/cram/snd10000/A-CUHS-CU000387-BL-COL-38200BL2_vcpa1.1.cram.crai&#xA;/path/to/adsp/cram/snd10000/A-CUHS-CU000018-BL-COL-46250BL1_vcpa1.1.cram.crai&#xA;/path/to/adsp/cram/snd10000/A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1.cram&#xA;/path/to/adsp/cram/snd10000/A-CUHS-CU000563-BL-COL-47451BL1_vcpa1.1.cram&#xA;/path/to/adsp/cram/snd10000/A-CUHS-CU000387-BL-COL-38200BL2_vcpa1.1.cram&#xA;/path/to/adsp/cram/snd10000/A-CUHS-CU000194-BL-COL-38470BL1_vcpa1.1.cram&#xA;/path/to/adsp/cram/snd10000/A-CUHS-CU000194-BL-COL-38470BL1_vcpa1.1.cram.crai&#xA;/path/to/adsp/cram/snd10000/A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1.cram.crai&#xA;/path/to/adsp/cram/snd10000/A-CUHS-CU000018-BL-COL-46250BL1_vcpa1.1.cram&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="136" LastActivityDate="2022-03-16T05:45:54.960" Title="Nextflow: cannot get file pairs from Channel.fromFilePairs" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18787" PostTypeId="1" AcceptedAnswerId="18788" CreationDate="2022-03-15T23:11:13.913" Score="4" ViewCount="337" Body="&lt;p&gt;I want to get a set of exon regions for each protein coding gene.&lt;/p&gt;&#xA;&lt;p&gt;I extracted a set of relevant information (chromosome, start, end, gene ID, gene name, gene type, exon number and exon ID) from a GTF using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '$3 == &amp;quot;exon&amp;quot;' annotation.gtf \&#xA;  | gtf2bed \&#xA;  | cut -f1-4,10 \&#xA;  | awk 'match($0, /gene_name \&amp;quot;[a-zA-Z0-9\_\-.\/]*\&amp;quot;/, gene_name) &#xA;    &amp;amp;&amp;amp; match($0, /gene_type \&amp;quot;[^&amp;quot;]*\&amp;quot;/, gene_type) &#xA;    &amp;amp;&amp;amp; match($0, /exon_number [0-9]*/, exon_number) &#xA;    &amp;amp;&amp;amp; match($0, /exon_id \&amp;quot;[^&amp;quot;]*\&amp;quot;/, exon_id) { &#xA;      print $1,$2,$3,$4,gene_name[0],gene_type[0],exon_number[0],exon_id[0];&#xA;    }' \&#xA;  | tr -d '&amp;quot;' \&#xA;  | tr ' ' $'\t' \&#xA;  | cut -f1-4,6,8,10,12 \&#xA;  | grep 'protein_coding' \&#xA;  &amp;gt; annotation.exons.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, during a sanity check, I noticed that for some genes there are some exons with overlapping coordinates, same gene information and exon number, but different exon ID, e.g.:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr2    9423650 9424052 ENSG00000119203.14      CPSF3   protein_coding  1       ENSE00001829522.1&#xA;chr2    9423653 9423823 ENSG00000119203.14      CPSF3   protein_coding  1       ENSE00001830696.2&#xA;chr2    9423883 9424960 ENSG00000119203.14      CPSF3   protein_coding  1       ENSE00001932298.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I did not directly work with exons in the past, but I am aware of alternative splicing.&#xA;My understanding was that exons can be skipped during alternative splicing, but I assumed that exon number was unique for bases, within the same gene (on the same strand).&lt;/p&gt;&#xA;&lt;p&gt;In the example above, the first 2 listed exons have the same exon number, overlapping (but not exactly the same) bases, and different exon ID. Is this due to some additional information that I can retrieve from the GTF, i.e., from:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr2    9423650 9424052 ENSG00000119203.14      .       +       HAVANA  exon    .       gene_id &amp;quot;ENSG00000119203.14&amp;quot;; transcript_id &amp;quot;ENST00000475482.5&amp;quot;; gene_type &amp;quot;protein_coding&amp;quot;; gene_name &amp;quot;CPSF3&amp;quot;; transcript_type &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;CPSF3-203&amp;quot;; exon_number 1; exon_id &amp;quot;ENSE00001829522.1&amp;quot;; level 2; protein_id &amp;quot;ENSP00000419744.1&amp;quot;; transcript_support_level &amp;quot;4&amp;quot;; hgnc_id &amp;quot;HGNC:2326&amp;quot;; tag &amp;quot;mRNA_end_NF&amp;quot;; tag &amp;quot;cds_end_NF&amp;quot;; havana_gene &amp;quot;OTTHUMG00000090415.5&amp;quot;; havana_transcript &amp;quot;OTTHUMT00000353331.3&amp;quot;;&#xA;chr2    9423653 9423823 ENSG00000119203.14      .       +       HAVANA  exon    .       gene_id &amp;quot;ENSG00000119203.14&amp;quot;; transcript_id &amp;quot;ENST00000238112.8&amp;quot;; gene_type &amp;quot;protein_coding&amp;quot;; gene_name &amp;quot;CPSF3&amp;quot;; transcript_type &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;CPSF3-201&amp;quot;; exon_number 1; exon_id &amp;quot;ENSE00001830696.2&amp;quot;; level 2; protein_id &amp;quot;ENSP00000238112.3&amp;quot;; transcript_support_level &amp;quot;1&amp;quot;; hgnc_id &amp;quot;HGNC:2326&amp;quot;; tag &amp;quot;basic&amp;quot;; tag &amp;quot;MANE_Select&amp;quot;; tag &amp;quot;appris_principal_1&amp;quot;; tag &amp;quot;CCDS&amp;quot;; ccdsid &amp;quot;CCDS1664.1&amp;quot;; havana_gene &amp;quot;OTTHUMG00000090415.5&amp;quot;; havana_transcript &amp;quot;OTTHUMT00000206843.2&amp;quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Is this because the different exons belong to different transcripts?&#xA;However, shouldn't different exons still be non-overlapping one against each other?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2023-02-19T12:35:06.737" Title="On the same strand, for the same gene, can exons be overlapping?" Tags="|proteins|sequence-annotation|gtf|rna-splicing|exon|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18788" PostTypeId="2" ParentId="18787" CreationDate="2022-03-16T00:08:33.377" Score="5" Body="&lt;p&gt;I'm not quite sure what you're asking, but pretty much anything can happen in terms of exons and transcripts:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Two different genes encoded on different strands can share the same genomic region as an exon&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Two different genes encoded on &lt;em&gt;the same strand&lt;/em&gt; can share the same genomic region as an exon (including identical exon boundaries)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A gene can have multiple transcript isoforms that share the same genomic region on the same frame&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;A gene can have multiple isoforms that share the same genomic region in different frames&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Genes can have variable regions which are modified during somatic development (e.g. T-cell and B-cell receptors)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Gene transcripts can be spliced together in different orders in different isoforms&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Exons from one transcript isoform can overlap with exons from another isoform (but not necessarily cover identical regions)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Stop codons can occasionally be &amp;quot;read-through&amp;quot;, resulting in a longer protein/peptide than the expected translation&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;As an example of different isoforms within the same gene exhibiting different translation mechanisms, here are graphical representations of annotated transcript isoforms for Gapdh on the mouse GRCm38 assembly, created in &lt;a href=&quot;https://jbrowse.org/jb2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;JBrowse&lt;/a&gt;. I wouldn't put complete trust in any of these (especially not the ones that have no defined intronic regions), but it's a useful demonstration of the types of things that can happen. Compare, Gapdh-207, where an untranslated region starts halfway through an exon, and Gapdh-209, where the translated protein starts halfway through an untranslated exon in Gapdh-207:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NHFDE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NHFDE.png&quot; alt=&quot;Gapdh isoforms, showing overlapping exons&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If there is a situation where it seems like exons &lt;em&gt;from the same isoform&lt;/em&gt; have overlapping regions, it might be worth looking elsewhere in the genome to see if there's a transcribed gene copy somewhere. That, at least, doesn't seem to be something I'd expect to happen through the normal translation process.&lt;/p&gt;&#xA;&lt;p&gt;Also, don't expect annotations to be complete. Even on something as [presumably] well-studied as the mouse mitochondrial genome, we can identify clear transcript start/stop points from long polyA cDNA sequences that are completely absent in the genome annotations. This is most obvious for the completely unannotated reverse strand (e.g. transcript starting at around 16,200bp), but also happens on the forward strand as well (e.g. coverage steps within mt-Nd5):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/srLJ4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/srLJ4.png&quot; alt=&quot;GRCm38 mitochondrial genome, with annotation and mapped transcript coverage plots&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-02-19T12:35:06.737" LastActivityDate="2023-02-19T12:35:06.737" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18790" PostTypeId="1" CreationDate="2022-03-16T05:23:46.557" Score="0" ViewCount="246" Body="&lt;p&gt;I am working with some GWAS summary statistics. My ultimate goal is to extract variants to use as instrument variables for Mendelian Randomization analysis, which requires me to find SNPs that are independently associated with the exposure variable of interest.&lt;/p&gt;&#xA;&lt;p&gt;My current workflow involves extracting SNPs from summary statistics with P-value &amp;lt; 5E-08, then performing LD Clumping with a stringent R^2 cutoff (R^2&amp;lt;0.05) within a locus. Is this sufficient to ensure that my variants are independently associated, or should I also plan on running conditional analysis(e.g. GCTA-COJO) in addition to/in lieu of clumping for my variants?&lt;/p&gt;&#xA;" OwnerUserId="15212" LastActivityDate="2022-03-17T02:20:46.047" Title="LD Clumping vs. Conditional Analysis for independent associations" Tags="|gwas|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18791" PostTypeId="2" ParentId="18786" CreationDate="2022-03-16T05:45:54.960" Score="3" Body="&lt;p&gt;&lt;strong&gt;TLDR&lt;/strong&gt;, use:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;nextflow.enable.dsl=2&#xA;&#xA;workflow {&#xA;&#xA;    Channel.fromFilePairs('/path/to/adsp/cram/snd10000/A-CUHS-CU000*.cram{,.crai}')&#xA;        | view()&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;N E X T F L O W  ~  version 20.10.0&#xA;Launching `xscript.nf` [special_caravaggio] - revision: a4e53657a0&#xA;[A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1, [/path/to/adsp/cram/snd10000/A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1.cram, /path/to/adsp/cram/snd10000/A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1.cram.crai]]&#xA;[A-CUHS-CU000194-BL-COL-38470BL1_vcpa1.1, [/path/to/adsp/cram/snd10000/A-CUHS-CU000194-BL-COL-38470BL1_vcpa1.1.cram, /path/to/adsp/cram/snd10000/A-CUHS-CU000194-BL-COL-38470BL1_vcpa1.1.cram.crai]]&#xA;[A-CUHS-CU000018-BL-COL-46250BL1_vcpa1.1, [/path/to/adsp/cram/snd10000/A-CUHS-CU000018-BL-COL-46250BL1_vcpa1.1.cram, /path/to/adsp/cram/snd10000/A-CUHS-CU000018-BL-COL-46250BL1_vcpa1.1.cram.crai]]&#xA;[A-CUHS-CU000387-BL-COL-38200BL2_vcpa1.1, [/path/to/adsp/cram/snd10000/A-CUHS-CU000387-BL-COL-38200BL2_vcpa1.1.cram, /path/to/adsp/cram/snd10000/A-CUHS-CU000387-BL-COL-38200BL2_vcpa1.1.cram.crai]]&#xA;[A-CUHS-CU000563-BL-COL-47451BL1_vcpa1.1, [/path/to/adsp/cram/snd10000/A-CUHS-CU000563-BL-COL-47451BL1_vcpa1.1.cram, /path/to/adsp/cram/snd10000/A-CUHS-CU000563-BL-COL-47451BL1_vcpa1.1.cram.crai]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;When using the &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#fromfilepairs&quot; rel=&quot;nofollow noreferrer&quot;&gt;fromFilePairs&lt;/a&gt; factory method, the trick is to ensure that the pattern you provide can return a &lt;em&gt;common&lt;/em&gt; prefix in order for the files to be grouped as expected. An easy way to check to see if this is working as expected is to directly call the &lt;a href=&quot;https://github.com/nextflow-io/nextflow/blob/v21.10.6/modules/nextflow/src/main/groovy/nextflow/Channel.groovy#L523-L585&quot; rel=&quot;nofollow noreferrer&quot;&gt;readPrefix&lt;/a&gt; helper function. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;test_cram = file('/path/to/adsp/cram/snd10000/A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1.cram')&#xA;test_crai = file('/path/to/adsp/cram/snd10000/A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1.cram.crai')&#xA;&#xA;pattern = '/path/to/adsp/cram/snd10000/A-CUHS-CU000*.{cram,crai}'&#xA;&#xA;println(&amp;quot;My CRAM prefix is: &lt;span class=&quot;math-container&quot;&gt;${Channel.readPrefix(test_cram, pattern)}&quot;)&#xA;println(&quot;My CRAI prefix is: $&lt;/span&gt;{Channel.readPrefix(test_crai, pattern)}&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can see that these results do &lt;em&gt;not&lt;/em&gt; produce a common prefix:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;N E X T F L O W  ~  version 20.10.0&#xA;Launching `script.nf` [lonely_jang] - revision: 428df7e55e&#xA;My CRAM prefix is: A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1&#xA;My CRAI prefix is: A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1.cram&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But if you change your pattern definition to:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pattern = '/path/to/adsp/cram/snd10000/A-CUHS-CU000*.cram{,.crai}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We can check that it returns a common prefix:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;N E X T F L O W  ~  version 20.10.0&#xA;Launching `script.nf` [maniac_volta] - revision: a8acf7b03f&#xA;My CRAM prefix is: A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1&#xA;My CRAI prefix is: A-CUHS-CU000169-BL-COL-57413BL1_vcpa1.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2022-03-16T05:45:54.960" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18793" PostTypeId="2" ParentId="18731" CreationDate="2022-03-16T14:26:57.853" Score="1" Body="&lt;p&gt;As it is turned out from the author's words:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;This is a normal result for the tutorial.&lt;/li&gt;&#xA;&lt;li&gt;It is recommended to border the deviation of geomodel to 3 powers.&lt;/li&gt;&#xA;&lt;li&gt;There is set up a homogenous diffusion model but BEAUTi does not show this correctly. Precision is defy τ as bt^-1 where t – a branch length, b – precision. It can be an arbitrary number that the quotient of it divided by the potential tree height would much lower than 1. Otherwise, the next approximation has some deterioration:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UZFQ7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UZFQ7.png&quot; alt=&quot;The probability density approximation&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Threshold is an angle (in degrees 0...180) below which there is no contribution to the probability density. The fast check box turns up approximation of arccos of angles close to zero (cos(x) &amp;gt; 0.9). The error is at the most 1e-10. More hidden parameters can be found through XML-editors in prepared files.&lt;/p&gt;&#xA;&lt;p&gt;By the way, the author, Remco Bouckaert, has released new version recently where we can specify regions for clades explicitly in BEAUTi.&lt;/p&gt;&#xA;&lt;p&gt;I hope that this information will be useful for anyone like me who is stuck in GEO_SPHERE.&lt;/p&gt;&#xA;" OwnerUserId="15161" LastActivityDate="2022-03-16T14:26:57.853" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18794" PostTypeId="2" ParentId="16153" CreationDate="2022-03-16T14:29:00.173" Score="0" Body="&lt;p&gt;PopArt have a mailing list popartusers+subscribe@googlegroups.com&lt;/p&gt;&#xA;&lt;p&gt;If you go &lt;a href=&quot;http://popart.otago.ac.nz/contact.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; you can subscribe to it. This sounds like an issue with Mac compatibility. OS 10.15.6 isn't a version I understand and suspect there's a typo. Currently I believe it is OSX 12.1, thus OS10.12.1&#xA;PopArt was written for OSX10.6 and upwards. HOWEVER, there can be issues with major system upgrades so Monterey OSX12, I think was a major revision. PopArt is no longer supported so it would be sensitive to changes in Mac OS.&lt;/p&gt;&#xA;&lt;p&gt;My suggestion is to try this on a Windows machine, if you get the same error the suspicion is there has been a small change to the input format.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-03-16T14:29:00.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18795" PostTypeId="1" CreationDate="2022-03-16T18:58:55.473" Score="3" ViewCount="245" Body="&lt;p&gt;I am using a function called Funcotator (&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035889931-Funcotator-Information-and-Tutorial#1.2&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) in GATK4. The code I am trying to run is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gatk Funcotator \&#xA;--variant chr21.vcf \&#xA;--reference hg38.fa.fai \&#xA;--ref-version hg38 \&#xA;--data-sources-path funcotator_dataSources.v1.2.20180329 \&#xA;--output chr21.funcotated.maf \&#xA;--output-file-format MAF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, it gives me an error message like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java.lang.IllegalArgumentException: File is not a supported reference file type: /Users/data/Bam/hg38.fa.fai&#xA;at htsjdk.samtools.reference.ReferenceSequenceFileFactory.lambda&lt;span class=&quot;math-container&quot;&gt;$getFastaExtension$&lt;/span&gt;0(ReferenceSequenceFileFactory.java:253)&#xA;at java.util.Optional.orElseGet(Optional.java:267)&#xA;at htsjdk.samtools.reference.ReferenceSequenceFileFactory.getFastaExtension(ReferenceSequenceFileFactory.java:253)&#xA;at htsjdk.samtools.reference.ReferenceSequenceFileFactory.getDefaultDictionaryForReferenceSequence(ReferenceSequenceFileFactory.java:223)&#xA;at org.broadinstitute.hellbender.utils.fasta.CachingIndexedFastaSequenceFile.checkFastaPath(CachingIndexedFastaSequenceFile.java:180)&#xA;at org.broadinstitute.hellbender.utils.fasta.CachingIndexedFastaSequenceFile.&amp;lt;init&amp;gt;(CachingIndexedFastaSequenceFile.java:143)&#xA;at org.broadinstitute.hellbender.utils.fasta.CachingIndexedFastaSequenceFile.&amp;lt;init&amp;gt;(CachingIndexedFastaSequenceFile.java:125)&#xA;at org.broadinstitute.hellbender.utils.fasta.CachingIndexedFastaSequenceFile.&amp;lt;init&amp;gt;(CachingIndexedFastaSequenceFile.java:110)&#xA;at org.broadinstitute.hellbender.engine.ReferenceFileSource.&amp;lt;init&amp;gt;(ReferenceFileSource.java:35)&#xA;at org.broadinstitute.hellbender.engine.ReferenceDataSource.of(ReferenceDataSource.java:27)&#xA;at org.broadinstitute.hellbender.engine.GATKTool.initializeReference(GATKTool.java:439)&#xA;at org.broadinstitute.hellbender.engine.GATKTool.onStartup(GATKTool.java:722)&#xA;at org.broadinstitute.hellbender.engine.VariantWalker.onStartup(VariantWalker.java:45)&#xA;at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:138)&#xA;at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:192)&#xA;at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:211)&#xA;at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)&#xA;at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)&#xA;at org.broadinstitute.hellbender.Main.main(Main.java:289)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does anyone know how to solve this problem? I tried to use the reference file downloaded from the GATK cloud, but it still doesn't work and has the same error message.&#xA;Also, I wonder if the reference files must be the version provided by the GATK team? The one I am using now is not from the GATK team, and as they force me to provide &lt;code&gt;--data-sources-path&lt;/code&gt;, I just copied the one they used on their post. Could that cause any problem in the further analysis?&#xA;Thanks a lot ~~~&lt;/p&gt;&#xA;&lt;p&gt;Here is an update:&#xA;I first tried &lt;code&gt;hg38.fa&lt;/code&gt; as my reference and got this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A USER ERROR has occurred: Fasta dict file file:///Users/data/Bam/hg38.dict for reference file:///Users/data/Bam/hg38.fa does not exist. Please see http://gatkforums.broadinstitute.org/discussion/1601/how-can-i-prepare-a-fasta-file-to-use-as-reference for help creating it.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So I generated the &lt;code&gt;hg38.fa.fai&lt;/code&gt;, then I got the error message shown above.&#xA;Then, I tried the reference file downloaded from the GATK cloud, &lt;code&gt;resources-broad-hg38-v0-Homo_sapiens_assembly38.fasta.fai&lt;/code&gt;, but the same error appeared again.&lt;/p&gt;&#xA;&lt;p&gt;Another update:&#xA;I tried to use the &lt;code&gt;hg38.dict&lt;/code&gt; as the reference and used the same code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    gatk Funcotator \&#xA;    --variant chr21.vcf \&#xA;    --reference hg38.dict \  &#xA;    --ref-version hg38 \&#xA;    --data-sources-path funcotator_dataSources.v1.2.20180329 \&#xA;    --output chr21.funcotated.maf \&#xA;    --output-file-format MAF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, I got the same error message again:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java.lang.IllegalArgumentException: File is not a supported reference file type: /Users/data/hg38.dict&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14102" LastEditorUserId="14102" LastEditDate="2022-03-17T15:25:29.510" LastActivityDate="2023-02-15T08:41:19.913" Title="Funcotator reference file error in GATK4" Tags="|gatk|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18797" PostTypeId="2" ParentId="2361" CreationDate="2022-03-16T22:11:15.387" Score="0" Body="&lt;p&gt;This is maybe a very simplified approach but have you thought about simply blasting them against the rRNA/ITS database of NCBI/ SILVA with ie. diamond?&lt;/p&gt;&#xA;&lt;p&gt;First download the database files from either of the database:&lt;/p&gt;&#xA;&lt;p&gt;SILVA: &lt;a href=&quot;https://www.arb-silva.de/no_cache/download/archive/current/Exports/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.arb-silva.de/no_cache/download/archive/current/Exports/&lt;/a&gt;&#xA;NCBI: &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/blast/db/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nlm.nih.gov/blast/db/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Second: Use DIAMOND to align fastas of interest (in your case these will be the 16S rRNA Sequences. These will give you hits per fasta which you can save into tsv or csv format and parse in excel.&lt;/p&gt;&#xA;&lt;p&gt;To get the abundances, take your fastas of interest and align them back to your raw files (paired reads). You can use BOWTIE + SAMTOOLs for this.&lt;/p&gt;&#xA;&lt;p&gt;Concatenate the results from the DIAMOND output with the abundances and voila, you should get a nice table with taxa and their abundances. If you have more samples and you would like to compare them, ie. compare the taxa composition. I would strongly suggest that you do some further normalisation ie. normalise the abundances with sequencing depths.&lt;/p&gt;&#xA;&lt;p&gt;Maybe it's a long way around it but in theory, it should do the job.&lt;/p&gt;&#xA;" OwnerUserId="14212" LastActivityDate="2022-03-16T22:11:15.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18799" PostTypeId="2" ParentId="18790" CreationDate="2022-03-16T23:05:05.233" Score="1" Body="&lt;p&gt;LD Clumping with &lt;span class=&quot;math-container&quot;&gt;$r^2$&lt;/span&gt; filtering seems like a good start, but bear in mind that population structure has a significant effect on association statistics.&lt;/p&gt;&#xA;&lt;p&gt;I recommend carrying out separate PCAs (or some other dimensional reduction) on your variable groups (e.g. risk/non-risk, or high/medium/low) to make sure that their genomic population structure is similar. Alternatively / equivalently, carry out a whole-group reduction, and confirm that the coordinate centres for each variable group are similar.&lt;/p&gt;&#xA;&lt;p&gt;If the population structures are &lt;em&gt;not&lt;/em&gt; similar (which is very likely), then it's important to [also] exclude structure-specific SNPs from the analysis, as these will confound the results of your analysis. Use a clustering method to identify people with similar genetic structure, and treat the different clusters as different trait groupings for identifying those SNPs.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-03-16T23:05:05.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18800" PostTypeId="2" ParentId="11192" CreationDate="2022-03-17T01:00:41.417" Score="0" Body="&lt;p&gt;I have just uploaded a Python tool to convert fasta to fastq: &lt;a href=&quot;https://github.com/Mostafa-MR/A_tool_to_convert_fasta_to_fastq&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Mostafa-MR/A_tool_to_convert_fasta_to_fastq&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Please note this is presented directly from a Jupyter notebook, i.e. it is a ipynb.&lt;/p&gt;&#xA;" OwnerUserId="15219" LastEditorUserId="3848" LastEditDate="2022-03-17T01:06:24.740" LastActivityDate="2022-03-17T01:06:24.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18801" PostTypeId="1" CreationDate="2022-03-17T10:59:51.273" Score="2" ViewCount="170" Body="&lt;h3&gt;EDIT&lt;/h3&gt;&#xA;&lt;p&gt;By thinking over the issue, I realized it was mixing two linked but separate questions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Q1. How to handle the situation when there are divergences between a GenBank file and an actual or theoretical molecule:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Should the difference be indicated in the feature table? If so, with which feature key? And if not, how should it be indicated or what should be done?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Further, when indicating a divergence using a feature key, which should be considered the reference: the file or the molecule?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Q2. How to keep track of changes between versions of a GenBank file when corrections or updates are made?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I have thus re-titled the question accordingly, and by making further researches into the issues I found what seems to me the appropriate solutions (see &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/18978/15221&quot;&gt;my answer&lt;/a&gt; below).&lt;/p&gt;&#xA;&lt;h3&gt;ORIGINAL QUESTION&lt;/h3&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; I am maintaining a database of sequence files describing a collection of bacterial strains and plasmids that we have in our lab.&lt;/p&gt;&#xA;&lt;p&gt;I am confronted to two sorts of issues:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;some sequence files contain errors (see comments below for examples of errors)&lt;/li&gt;&#xA;&lt;li&gt;other sequence files describe theoretical sequences of molecules that themselves contain variations (mutations, insertion or deletions) that occurred during the molecular cloning process.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Objective&lt;/strong&gt; I want to annotate the sequences to mark those errors and variations using the GenBank format. However, from the feature keys of the GenBank format described on the &lt;a href=&quot;https://www.insdc.org/documents/feature_table.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;INSDC website&lt;/a&gt;, it is not clear to me which feature key should be used in each case, essentially this is either 'variation' or 'misc_description'.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt; To better explain the confusion, here below are what seems to me the most relevant feature keys, and my comment of the problem I have with them:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Feature key           variation&#xA;Definition            a related strain contains stable mutations from the same&#xA;                      gene (e.g., RFLPs, polymorphisms, etc.) which differ&#xA;                      from the presented sequence at this location (and&#xA;                      possibly others);&#xA;...&#xA;Comment               used to describe alleles, RFLP's,and other naturally &#xA;                      occurring mutations and  polymorphisms;&#xA;                  &amp;gt;&amp;gt;&amp;gt; variability arising as a result of genetic &#xA;                      manipulation (e.g. site directed mutagenesis) should &#xA;                      be described with the misc_difference feature;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;According to the line marked &lt;code&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/code&gt; above, the feature key &lt;code&gt;misc_difference&lt;/code&gt; should be used...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Feature Key           misc_difference&#xA;Definition            feature sequence is different from that presented in the&#xA;                      entry and cannot be described by any other difference&#xA;                      key (old_sequence, variation, or modified_base);&#xA;...&#xA;Comment               the misc_difference feature key should be used to&#xA;                      describe variability that arises as a result of &#xA;                      genetic manipulation (e.g. site directed mutagenesis);&#xA;                  &amp;gt;&amp;gt;&amp;gt; use /replace=&amp;quot;&amp;quot; to annotate deletion, e.g.&#xA;                      misc_difference 412..433&#xA;                                      /replace=&amp;quot;&amp;quot;  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But now, according to the example marked &lt;code&gt;&amp;gt;&amp;gt;&amp;gt;&lt;/code&gt; above, it means the variation (here a deletion) is NOT in the considered entry...&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; So how are variations (mutation, insertion, deletion, errors) correctly described in Genbank given that are actually present in an entry ?&lt;/p&gt;&#xA;" OwnerUserId="15221" LastEditorUserId="15221" LastEditDate="2022-04-25T08:30:10.537" LastActivityDate="2022-05-02T08:58:27.963" Title="How to properly annotate sequence variants and errors in a GenBank file format and how to keep track of successive versions of a GenBank file" Tags="|file-formats|genbank|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18802" PostTypeId="2" ParentId="18801" CreationDate="2022-03-17T14:40:09.783" Score="1" Body="&lt;p&gt;Your core question is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;(mutations, insertion or deletions) that occurred during the molecular cloning process.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I asked NCBI directly and their response is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For natural genetic variation use 'Variation' feature;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For 'artificial' genetic variation (lab-produced by genetic modification) then use 'misc_difference'.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In summary in all scenarios its singly the 'misc_differences' that are used (see Conclusion).&lt;/p&gt;&#xA;&lt;p&gt;So in my scenario where I want to screen out targeted mutagenesis its the 'misc_difference', i.e. the second option of the two you present. This is also true in your second scenario because it is changes possibly unintentional, possibly random mutagenesis / or just unintentional errors during cloning. Finally, this is also true for SOME of the 'errors' you are reporting in comments:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;for the first i, ii and iii that appears to be just correcting stuff - administration, just cross-check with those submitting.&lt;/li&gt;&#xA;&lt;li&gt;for 'variations' the second i and ii in comments, that falls into the 'misc_difference', because a form of mutagenesis, albeit probably unintentional.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&#xA;The 'variation' feature description isn't clear given the description you reported against NCBI's official Helpdesk response. Its a fair question, its a clear answer, but doesn't explain why the description is confusing.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;On 'errors' of mis-identification, e.g. correctly identified e.g. via local Blast or Blat, e.g. sequenced the wrong target, I would use an internal marker, e.g. &lt;code&gt;returned to contributor for comment&lt;/code&gt;. Pass it back to the contributor and ask for their input and depending what they say simply mark the record in the 'Comments section' e.g. &lt;code&gt;originally defined as XX&lt;/code&gt;. The contributor could have made all sorts of different errors, e.g. annotation - copied and pasted the wrong stuff - experimental anything and you need to give them a chance to respond. Uploading these records on an external database, i.e. Genbank, needs a joint agreement with all parties concerned because it is certainly valid information for someone.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Note, the official blurb &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genbank/genomesubmit_annotation/#disrupted_genes&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, doesn't appear informative:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Disrupted genes and gene fragments&lt;/p&gt;&#xA;&lt;p&gt;Sometimes a genome will have adjacent or nearby genes that seem to be&#xA;only part of a protein. In many cases these indicate a possible&#xA;problem with the sequence and/or annotation. A related issue is the&#xA;presence of internal stop codons in the conceptual translation of a&#xA;CDS that looks like it should be a real CDS. These problems may be due&#xA;to a variety of reasons, including &lt;strong&gt;mutations&lt;/strong&gt; or sequencing or assembly&#xA;artifacts. They can be annotated in a number of ways:&lt;/p&gt;&#xA;&lt;p&gt;Annotate the gene with /pseudo to indicate that there is a problem&#xA;with the gene. Note that this qualifier does NOT mean that the gene is&#xA;a pseudogene. (see point 2, below, if it is known that the gene IS a&#xA;pseudogene) If multiple gene fragments were present initially, then&#xA;add a single gene feature which covers all of the potential coding&#xA;regions and add the pseudo qualifier. If known, a note qualifier may&#xA;be added indicating why this gene is disrupted, for example:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt; 1        200     gene&#xA;                         gene    phoA&#xA;                         gene_desc     alkaline phosphatase&#xA;                         locus_tag     OBB_0001&#xA;                         pseudo&#xA;                         note    nonfunctional due to frameshift&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-03-17T22:27:23.693" LastActivityDate="2022-03-17T22:27:23.693" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18803" PostTypeId="1" CreationDate="2022-03-17T17:00:23.367" Score="0" ViewCount="328" Body="&lt;p&gt;I am conducting phylogeny analyses using BEAST 2 as software tool. It is said that it can be accelerated by BEAGLE-library. I have a personal laptop with Windows &amp;amp; a cluster with CentOS.&lt;/p&gt;&#xA;&lt;p&gt;I installed both BEAST 2 &amp;amp; BEAGLE onto them. This gives me a boost by 3.6 times. It was said too that using OpenCL with BEAGLE gives me more speed-up. Nonetheless, my calculations takes from twelve hours to 3 days so I got really interested in this. Installing it on Windows were very simple. When I ran &lt;code&gt;beast -beagle_info&lt;/code&gt;, I saw:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;                        BEAST v2.6.6, 2002-2021&#xA;             Bayesian Evolutionary Analysis Sampling Trees&#xA;                       Designed and developed by&#xA; Remco Bouckaert, Alexei J. Drummond, Andrew Rambaut &amp;amp; Marc A. Suchard&#xA;&#xA;                   Centre for Computational Evolution&#xA;                         University of Auckland&#xA;                       r.bouckaert@auckland.ac.nz&#xA;                        alexei@cs.auckland.ac.nz&#xA;&#xA;                   Institute of Evolutionary Biology&#xA;                        University of Edinburgh&#xA;                           a.rambaut@ed.ac.uk&#xA;&#xA;                    David Geffen School of Medicine&#xA;                 University of California, Los Angeles&#xA;                           msuchard@ucla.edu&#xA;&#xA;                      Downloads, Help &amp;amp; Resources:&#xA;                           http://beast2.org/&#xA;&#xA;  Source code distributed under the GNU Lesser General Public License:&#xA;                   http://github.com/CompEvol/beast2&#xA;&#xA;                           BEAST developers:&#xA;   Alex Alekseyenko, Trevor Bedford, Erik Bloomquist, Joseph Heled,&#xA; Sebastian Hoehna, Denise Kuehnert, Philippe Lemey, Wai Lok Sibon Li,&#xA;Gerton Lunter, Sidney Markowitz, Vladimir Minin, Michael Defoin Platel,&#xA;          Oliver Pybus, Tim Vaughan, Chieh-Hsi Wu, Walter Xie&#xA;&#xA;                               Thanks to:&#xA;          Roald Forsberg, Beth Shapiro and Korbinian Strimmer&#xA;&#xA;&#xA;--- BEAGLE RESOURCES ---&#xA;&#xA;0 : CPU&#xA;    Flags: PRECISION_SINGLE PRECISION_DOUBLE COMPUTATION_SYNCH EIGEN_REAL EIGEN_COMPLEX SCALING_MANUAL SCALING_AUTO SCALING_ALWAYS SCALERS_RAW SCALERS_LOG VECTOR_SSE VECTOR_NONE THREADING_NONE PROCESSOR_CPU FRAMEWORK_CPU&#xA;&#xA;1 : NVIDIA GeForce 940MX&#xA;    Global memory (MB): 2048&#xA;    Clock speed (Ghz): 1.19&#xA;    Number of cores: 384&#xA;    Flags: PRECISION_SINGLE PRECISION_DOUBLE COMPUTATION_SYNCH COMPUTATION_ASYNCH EIGEN_REAL EIGEN_COMPLEX SCALING_MANUAL SCALING_AUTO SCALING_ALWAYS SCALERS_RAW SCALERS_LOG VECTOR_NONE THREADING_NONE PROCESSOR_GPU FRAMEWORK_CUDA&#xA;&#xA;2 : Intel(R) HD Graphics 620 (OpenCL 2.1 )&#xA;    Global memory (MB): 3219&#xA;    Clock speed (Ghz): 1.00&#xA;    Number of compute units: 24&#xA;    Flags: PRECISION_SINGLE PRECISION_DOUBLE COMPUTATION_SYNCH COMPUTATION_ASYNCH EIGEN_REAL EIGEN_COMPLEX SCALING_MANUAL SCALING_AUTO SCALING_ALWAYS SCALERS_RAW SCALERS_LOG VECTOR_NONE THREADING_NONE PROCESSOR_GPU FRAMEWORK_OPENCL&#xA;&#xA;3 : Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz (OpenCL 2.1 (Build 10))&#xA;    Global memory (MB): 8067&#xA;    Clock speed (Ghz): 2.50&#xA;    Number of compute units: 4&#xA;    Flags: PRECISION_SINGLE PRECISION_DOUBLE COMPUTATION_SYNCH COMPUTATION_ASYNCH EIGEN_REAL EIGEN_COMPLEX SCALING_MANUAL SCALING_AUTO SCALING_ALWAYS SCALERS_RAW SCALERS_LOG VECTOR_NONE THREADING_NONE PROCESSOR_CPU FRAMEWORK_OPENCL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, it still uses only resourse #0, even through forced &lt;code&gt;beast -beagle_order 3 -threads 1 filename.xml&lt;/code&gt;. Although, in comparison to Linux case, it was more successful.&lt;/p&gt;&#xA;&lt;p&gt;I have downloaded intel_sdk_for_opencl_applications_2020.3.494.tar.gz from &lt;a href=&quot;http://software.intel.com/en-us/vcsource/tools/opencl-sdk&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Then I unzipped it &amp;amp; run &lt;strong&gt;install.sh&lt;/strong&gt;. Installation was successful. After that, I have decided to go to &lt;strong&gt;build&lt;/strong&gt; folder in &lt;strong&gt;beagle-lib&lt;/strong&gt; &amp;amp; run &lt;code&gt;cmake -DCMAKE_INSTALL_PREFIX:PATH=$HOME ..&lt;/code&gt; in order to go on to run &lt;code&gt;make install&lt;/code&gt; but I get the next message:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-- JAVA_HOME=&#xA;-- JNI_INCLUDE_DIRS=/usr/lib/jvm/java/include;/usr/lib/jvm/java/include/linux;/usr/lib/jvm/java/include&#xA;-- JNI_LIBRARIES=/usr/lib/jvm/java/lib/libjawt.so;/usr/lib/jvm/java/lib/server/libjvm.so&#xA;-- Not using libtools for plugins&#xA;-- Could NOT find OpenCL (missing: OpenCL_LIBRARY OpenCL_INCLUDE_DIR)&#xA;CUDA_TOOLKIT_ROOT_DIR not found or specified&#xA;-- Could NOT find CUDA (missing: CUDA_TOOLKIT_ROOT_DIR CUDA_NVCC_EXECUTABLE CUDA_INCLUDE_DIRS CUDA_CUDART_LIBRARY)&#xA;-- Configuring done&#xA;-- Generating done&#xA;-- Build files have been written to: /export/home/rinkman/beagle-lib/build&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It was solved through manual specification of directories &lt;code&gt;cmake -DOpenCL_INCLUDE_DIR=~/intel/system_studio_2020/opencl/SDK/include/ -DOpenCL_LIBRARY=~/intel/system_studio_2020/opencl/SDK/lib64/libOpenCL.so.1.2 -DCMAKE_INSTALL_PREFIX:PATH=$HOME ..&lt;/code&gt;. The compiling was successful after that. &lt;code&gt;make test&lt;/code&gt; was executed without any problems. I detected the file &lt;strong&gt;libhmsbeagle-opencl.so.40.0.0&lt;/strong&gt; in &lt;strong&gt;~/lib folder&lt;/strong&gt;. But &lt;code&gt;beast -beafle_info&lt;/code&gt; did not show OpenCL. As there are older versions of BEAST &amp;amp; BEAGLE on the root, I tried to change very roughly startup script to run only my local BEAGLE but nothing changed:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/sh&#xA;&#xA;if [ -z &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$BEAST&quot; ]; then&#xA;    ## resolve links - $&lt;/span&gt;0 may be a link to application&#xA;    PRG=&amp;quot;$0&amp;quot;&#xA;&#xA;    # need this for relative symlinks&#xA;    while [ -h &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$PRG&quot; ] ; do&#xA;        ls=`ls -ld &quot;$&lt;/span&gt;PRG&amp;quot;`&#xA;        link=`expr &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$ls&quot; : '.*-&amp;gt; \(.*\)$&lt;/span&gt;'`&#xA;        if expr &amp;quot;$link&amp;quot; : '/.*' &amp;gt; /dev/null; then&#xA;        PRG=&amp;quot;$link&amp;quot;&#xA;        else&#xA;        PRG=&amp;quot;`dirname &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$PRG&quot;`/$&lt;/span&gt;link&amp;quot;&#xA;        fi&#xA;    done&#xA;&#xA;    # make it fully qualified&#xA;    saveddir=`pwd`&#xA;    BEAST0=`dirname &amp;quot;$PRG&amp;quot;`/..&#xA;    BEAST=`cd &amp;quot;$BEAST0&amp;quot; &amp;amp;&amp;amp; pwd`&#xA;    cd &amp;quot;$saveddir&amp;quot;&#xA;fi&#xA;&#xA;BEAST_LIB=&amp;quot;$BEAST/lib&amp;quot;&#xA;&#xA;if [ -z &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$JAVA_HOME&quot; ]; then&#xA;  JAVA=java&#xA;else&#xA;  JAVA=&quot;$&lt;/span&gt;JAVA_HOME&amp;quot;/bin/java&#xA;fi&#xA;&#xA;&#xA;# use BEAGLE_LIB if the BEAGLE library is not in a standard location&#xA;if [ -n &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$BEAGLE_LIB&quot; ]; then&#xA;    if [ -n &quot;$&lt;/span&gt;BEAST_EXTRA_LIBS&amp;quot; ]; then&#xA;        BEAST_EXTRA_LIBS=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$BEAST_EXTRA_LIBS:$&lt;/span&gt;BEAGLE_LIB&amp;quot;&#xA;    else&#xA;        BEAST_EXTRA_LIBS=&amp;quot;$BEAGLE_LIB&amp;quot;&#xA;    fi&#xA;fi&#xA;&#xA;# Explicitly add /usr/local/lib to library search path to ensure &#xA;# beast continues to find beagle when installed here.  (This is&#xA;# necessary due to known problems with certain JREs.)&#xA;if [ -z &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$LD_LIBRARY_PATH&quot; ]; then&#xA;    export LD_LIBRARY_PATH=/usr/local/lib&#xA;else&#xA;    export LD_LIBRARY_PATH=&quot;$&lt;/span&gt;LD_LIBRARY_PATH&amp;quot;:/usr/local/lib&#xA;fi&#xA;&#xA;BEAST_EXTRA_LIBS=&amp;quot;$HOME/lib/&amp;quot;&#xA;&#xA;# use BEAST_EXTRA_LIBS variable to load BEAGLE and other libraries from non-default locations &#xA;# this assumes that the library path contains all these libraries (or are set through LD_LIBRARY_PATH)&#xA;#if [ -n &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$BEAST_EXTRA_LIBS&quot; ]; then &#xA;#  &quot;$&lt;/span&gt;JAVA&amp;quot; -Dlauncher.wait.for.exit=true -Xms256m -Xmx1g -Djava.library.path=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$BEAST_EXTRA_LIBS&quot; -Duser.language=en -cp &quot;$&lt;/span&gt;BEAST_LIB/launcher.jar&amp;quot; beast.app.beastapp.BeastLauncher &lt;span class=&quot;math-container&quot;&gt;$*&#xA;#else   &#xA;#  &quot;$&lt;/span&gt;JAVA&amp;quot; -Dlauncher.wait.for.exit=true -Xms256m -Xmx1g -Duser.language=en -cp &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$BEAST_LIB/launcher.jar&quot; beast.app.beastapp.BeastLauncher $&lt;/span&gt;*&#xA;#fi&#xA;&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$JAVA&quot; -Dlauncher.wait.for.exit=true -Xms256m -Xmx1g -Djava.library.path=&quot;$&lt;/span&gt;BEAST_EXTRA_LIBS&amp;quot; -Duser.language=en -cp &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$BEAST_LIB/launcher.jar&quot; beast.app.beastapp.BeastLauncher $&lt;/span&gt;*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Please, could anyone help with this?&lt;/p&gt;&#xA;&lt;p&gt;P.S. I am a noviciate in unix-systems.&lt;/p&gt;&#xA;" OwnerUserId="15161" LastActivityDate="2022-04-28T17:03:10.290" Title="BEAGLE do not see and/or not use OpenCL" Tags="|phylogenetics|linux|beast|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18804" PostTypeId="1" AcceptedAnswerId="18807" CreationDate="2022-03-18T05:32:46.850" Score="2" ViewCount="114" Body="&lt;p&gt;I am thinking of reconstructing an multiple sequence alignment (MSA) of protein sequences from  a position-Specific Scoring Matrices (PSSM).&lt;/p&gt;&#xA;&lt;p&gt;Is it possible? I suppose co-evolution information is lost in PSSM so maybe it is difficult?&lt;/p&gt;&#xA;" OwnerUserId="14169" LastEditorUserId="3848" LastEditDate="2022-03-18T13:10:37.307" LastActivityDate="2022-03-18T13:10:37.307" Title="Is it possible to reconstruct an MSA with PSSM?" Tags="|phylogenetics|proteins|phylogeny|pssm|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18807" PostTypeId="2" ParentId="18804" CreationDate="2022-03-18T10:18:47.470" Score="1" Body="&lt;p&gt;In short I'm afraid it isn't possible because you lose the linkage per Haplotype which is the core information and vital for any population genetics or phylogenetic analysis.&lt;/p&gt;&#xA;&lt;p&gt;Position-specific scoring matrix e.g. A .60 C .20 G .20 T 0&#xA;I tried to produce a position scoring matrix, but it didn't work out, anyway the allele frequency for each e.g. amino acid are given for each alignment position.&lt;/p&gt;&#xA;&lt;p&gt;You could produce a consensus sequence which, albeit be hypothetical, it might be of some use to understand the generic behaviour of all the sequences in your sample.&lt;/p&gt;&#xA;&lt;p&gt;Here's a cartoon describing linkage and its importance,&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xanov.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xanov.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For a multiple sequence alignment of course AB and ab are amino acid residues within the same locus. PSSM could be used to construct a matrix to analyses an established alignment, but that's a fairly complicated Phylogenetics calculation (and a separate question).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-03-18T10:18:47.470" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18808" PostTypeId="2" ParentId="17883" CreationDate="2022-03-18T13:44:41.893" Score="0" Body="&lt;p&gt;Try&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FROM ubuntu:latest&#xA;MAINTAINER Conda Development Team &amp;lt;conda@continuum.io&amp;gt;&#xA;&#xA;RUN apt-get -qq update &amp;amp;&amp;amp; apt-get -qq -y install curl bzip2 \&#xA;    &amp;amp;&amp;amp; curl -sSL https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh -o /tmp/miniconda.sh \&#xA;    &amp;amp;&amp;amp; bash /tmp/miniconda.sh -bfp /usr/local \&#xA;    &amp;amp;&amp;amp; rm -rf /tmp/miniconda.sh \&#xA;    &amp;amp;&amp;amp; conda install -y python=3 \&#xA;    &amp;amp;&amp;amp; conda update conda \&#xA;    &amp;amp;&amp;amp; apt-get -qq -y remove curl bzip2 \&#xA;    &amp;amp;&amp;amp; apt-get -qq -y autoremove \&#xA;    &amp;amp;&amp;amp; apt-get autoclean \&#xA;    &amp;amp;&amp;amp; rm -rf /var/lib/apt/lists/* /var/log/dpkg.log \&#xA;    &amp;amp;&amp;amp; conda clean --all --yes&#xA;RUN conda install -y -c conda-forge mamba&#xA;RUN mamba create -q -y -c conda-forge -c bioconda -n snakemake snakemake snakemake-minimal --only-deps&#xA;ENV PATH /opt/conda/envs/snakemake/bin:${PATH}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &amp;amp;&amp;amp; approach is better as it will minimise the container size (which is going to be big in any case). Then look at an alpine version once its working.&lt;/p&gt;&#xA;&lt;p&gt;To be honest, I'm not sure about not activating conda base and personally I'd look to correct that.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker pull continuumio/miniconda3&#xA;docker run -i -t continuumio/miniconda3 /bin/bash&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Just for a chuckle, a cool approach is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker pull frolvlad/alpine-miniconda3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is super small.&lt;/p&gt;&#xA;&lt;p&gt;The docker file being&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FROM frolvlad/alpine-glibc:alpine-3.14&#xA;&#xA;ARG CONDA_VERSION=&amp;quot;4.7.12.1&amp;quot;&#xA;ARG CONDA_MD5=&amp;quot;81c773ff87af5cfac79ab862942ab6b3&amp;quot;&#xA;ARG CONDA_DIR=&amp;quot;/opt/conda&amp;quot;&#xA;&#xA;ENV PATH=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$CONDA_DIR/bin:$&lt;/span&gt;PATH&amp;quot;&#xA;ENV PYTHONDONTWRITEBYTECODE=1&#xA;&#xA;# Install conda&#xA;RUN echo &amp;quot;**** install dev packages ****&amp;quot; &amp;amp;&amp;amp; \&#xA;    apk add --no-cache --virtual .build-dependencies bash ca-certificates wget &amp;amp;&amp;amp; \&#xA;    \&#xA;    echo &amp;quot;**** get Miniconda ****&amp;quot; &amp;amp;&amp;amp; \&#xA;    mkdir -p &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$CONDA_DIR&quot; &amp;amp;&amp;amp; \&#xA;    wget &quot;http://repo.continuum.io/miniconda/Miniconda3-$&lt;/span&gt;{CONDA_VERSION}-Linux-x86_64.sh&amp;quot; -O miniconda.sh &amp;amp;&amp;amp; \&#xA;    echo &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$CONDA_MD5  miniconda.sh&quot; | md5sum -c &amp;amp;&amp;amp; \&#xA;    \&#xA;    echo &quot;**** install Miniconda ****&quot; &amp;amp;&amp;amp; \&#xA;    bash miniconda.sh -f -b -p &quot;$&lt;/span&gt;CONDA_DIR&amp;quot; &amp;amp;&amp;amp; \&#xA;    echo &amp;quot;export PATH=&lt;span class=&quot;math-container&quot;&gt;$CONDA_DIR/bin:\\\$PATH&quot; &amp;gt; /etc/profile.d/conda.sh &amp;amp;&amp;amp; \&#xA;    \&#xA;    echo &quot;**** setup Miniconda ****&quot; &amp;amp;&amp;amp; \&#xA;    conda update --all --yes &amp;amp;&amp;amp; \&#xA;    conda config --set auto_update_conda False &amp;amp;&amp;amp; \&#xA;    \&#xA;    echo &quot;**** cleanup ****&quot; &amp;amp;&amp;amp; \&#xA;    apk del --purge .build-dependencies &amp;amp;&amp;amp; \&#xA;    rm -f miniconda.sh &amp;amp;&amp;amp; \&#xA;    conda clean --all --force-pkgs-dirs --yes &amp;amp;&amp;amp; \&#xA;    find &quot;$&lt;/span&gt;CONDA_DIR&amp;quot; -follow -type f \( -iname '*.a' -o -iname '*.pyc' -o -iname '*.js.map' \) -delete &amp;amp;&amp;amp; \&#xA;    \&#xA;    echo &amp;quot;**** finalize ****&amp;quot; &amp;amp;&amp;amp; \&#xA;    mkdir -p &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$CONDA_DIR/locks&quot; &amp;amp;&amp;amp; \&#xA;    chmod 777 &quot;$&lt;/span&gt;CONDA_DIR/locks&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Whoever frolvlad is they certainly know their stuff, its a highly optimised image. This will absolutely nail a light-weight conda file. How well conda will work with Snakemake  ... it is  well worth trying because the alpine image is 5meg (Ubuntu is 100+ Megs) and I forget how big conda is. I need to do this more for sure, because almost certainly what they have done is streamlined conda to the absolute essentials. Get that working your looping the loop.&lt;/p&gt;&#xA;&lt;p&gt;Why is this potentially super cool? The problem with alpine is its so minimalist getting anything to work is not easy. &lt;code&gt;frolvlad&lt;/code&gt; in theory solved this bit via conda, because conda is completely self-contained once its running. Thus its doesn't need any system stuff at all, hence alpine and conda in THEORY would be a fantastic solution.&lt;/p&gt;&#xA;&lt;p&gt;My honest opinion is you are sort of trying to reinvent the wheel  (which sometimes that's a good thing). However, I think you'd find the other stuff just works because someone has sat down with yaml and tested everything endlessly.&lt;/p&gt;&#xA;&lt;p&gt;The problem with your file is you've got to clean it without additional layers being dumped onto it and that's why loads of &amp;amp;&amp;amp; because that enables a clean without a layer collapsing onto it blocking the clean. The docker file you create has loads of unnecessary stuff, which you can't delete. I guess your priority is to get it working.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-03-18T14:53:45.460" LastActivityDate="2022-03-18T14:53:45.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18809" PostTypeId="1" CreationDate="2022-03-19T19:35:18.587" Score="2" ViewCount="220" Body="&lt;p&gt;I'm trying to assemble some transcriptomes using Trinity, but am having issues getting Trinity to finish. I've been trying to get this to work for weeks, but am hitting a wall and would very much appreciate any and all help.&lt;/p&gt;&#xA;&lt;p&gt;I have 2 treatment conditions, with 3 replicates per condition, and they were sequenced on a NovaSeq (using 2x150 base reads). I originally tried assembling all the samples together, but even after running for 8+ days it did not finish.&lt;/p&gt;&#xA;&lt;p&gt;I am now trying to run just 1 sample from each condition, but even after 4 days Trinity will not finish. It seems like it is stuck at Butterfly, and I'm unsure how to proceed. Some more detail and things I've tried:&lt;/p&gt;&#xA;&lt;p&gt;These are microbial isolates grown in axenic culture, not metagenomic samples. The microbes are protists though (not bacteria), so their transcriptomes are more complex than prokaryotes, but not as complex as higher eukaryotes like humans/mice.&lt;/p&gt;&#xA;&lt;p&gt;I do not need to have high sensitivity for lowly expressed transcripts, so I tried setting the &amp;quot;min_kmer_cov&amp;quot; higher. First I set it to 2, which did not help. Then I tried 10 (seemed extreme but I was desperate, but even that has not solved it).&lt;/p&gt;&#xA;&lt;p&gt;-I am running these on a cluster, and admittedly I am not knowledgeable at all about how to optimally request resources, so I am wondering if that is part of the issue. Our cluster uses slurm, and my resource requests right now are&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#SBATCH --mem 100000&#xA;#SBATCH -n 24&#xA;#SBATCH -N 6&amp;quot;.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;-I tried running just one replicate from one of my conditions. This &lt;strong&gt;did&lt;/strong&gt; finish (after ~42 hours), so it seems like the issue is not an inherent problem with the transcriptomes/the data itself.&lt;/p&gt;&#xA;&lt;p&gt;Thank you very much!&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;By efficiently constructing and analyzing sets of de Bruijn graphs, Trinity fully reconstructs a large fraction of transcripts, including alternatively spliced isoforms and transcripts from recently duplicated genes. Compared with other de novo transcriptome assemblers, Trinity recovers more full-length transcripts across a broad range of expression levels, with a sensitivity similar to methods that rely on genome alignments. This approach provides a unified solution for transcriptome reconstruction in any sample, especially in the absence of a reference genome.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Trinity&#xA;Trinity is a de novo assembler that fully reconstructs a large fraction of transcripts, including alternatively spliced isoforms and transcripts from recently duplicated genes using de Bruijn graphs. Trinity recover  full-length transcripts across a broad range of expression levels, with a sensitivity similar to methods that rely on genome alignments.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://evomics.org/wp-content/uploads/2012/01/Grabherr_etal_2011.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Grabherr, MG et al. (2011) Full-length transcriptome assembly from RNA-Seq data without a reference genome. Nature Biotechnology. 2011 May 15;29(7):644-52.&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15233" LastEditorUserId="73" LastEditDate="2022-03-20T01:11:21.963" LastActivityDate="2022-03-20T20:35:00.227" Title="Trinity de novo assembly not completing" Tags="|rna-seq|trinity|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18810" PostTypeId="2" ParentId="18809" CreationDate="2022-03-20T00:30:21.013" Score="1" Body="&lt;p&gt;I'm not familiar with the use of Trinity for microbial assembly. There may be some assumptions for animal assembly that are broken in protists, although I can't think of any reason why that would slow the butterfly step down.&lt;/p&gt;&#xA;&lt;p&gt;It's possible that your dataset is simply too large. One thing you could try is reducing the dataset size (after pooling all samples together). Trinity can do a kmer-based normalization, more details here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/trinityrnaseq/trinityrnaseq/wiki/Trinity-Insilico-Normalization&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/trinityrnaseq/trinityrnaseq/wiki/Trinity-Insilico-Normalization&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Large RNA-Seq data sets, such as those exceeding 300M pairs, are best suited for in silico normalization prior to running Trinity.&lt;/p&gt;&#xA;&lt;p&gt;Note, all recent versions of Trinity will perform in silico normalization by default. You can turn it off with &lt;code&gt;Trinity --no_normalize_reads&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If for any reason, you'd like to normalize data separately, you can do so as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; %  $TRINITY_HOME/util/insilico_read_normalization.pl &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If that normalization is still failing to reduce the dataset down to a reasonable level, it suggests that the dataset is probably more complex than expected. This could be caused by contamination with other microbes, and may be improved by first filtering out contaminant reads.&lt;/p&gt;&#xA;&lt;p&gt;In any case, if you're having difficulty with getting Trinity to work, the best approach is to contact the developers by logging an issue on the Trinity github:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/trinityrnaseq/trinityrnaseq/issues&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/trinityrnaseq/trinityrnaseq/issues&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You could try using an alternative short-read genomic assembler like SPAdes. I see that SPAdes also has an RNA mode and a metaviral mode, so it might already be a bit more closely tuned to smaller transcriptomes:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://cab.spbu.ru/software/spades/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cab.spbu.ru/software/spades/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/ablab/spades/blob/spades_3.15.4/README.md#metapv&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ablab/spades/blob/spades_3.15.4/README.md#metapv&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-03-20T20:35:00.227" LastActivityDate="2022-03-20T20:35:00.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18811" PostTypeId="2" ParentId="17839" CreationDate="2022-03-20T13:36:38.303" Score="0" Body="&lt;p&gt;The key words in UPGMA are 'pairwise' and 'mean'.&lt;/p&gt;&#xA;&lt;p&gt;In phylogeny the format is a bit unusual because &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt; here is the MRCA (most recent common ancestor) for &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt;. In other words this represents a three taxa tree. I agree in UPGMA this works, because of its use of the mean (which is also its criticism - below).&lt;/p&gt;&#xA;&lt;p&gt;Under UPGMA - in your equation &lt;span class=&quot;math-container&quot;&gt;$d(C) = \frac{d(A,B)}{N}$&lt;/span&gt; ...where &lt;span class=&quot;math-container&quot;&gt;$N=2$&lt;/span&gt;, i.e. mean.&lt;/p&gt;&#xA;&lt;p&gt;Thus under UPGMA&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$d(C,D) = d(A,D) - \frac{d(A,B)}{2}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;There is no need to include &lt;span class=&quot;math-container&quot;&gt;$d(B,D)$&lt;/span&gt; because &lt;span class=&quot;math-container&quot;&gt;$d(B,D) = d(A,D)$&lt;/span&gt;, its the mean that was resolved in establishing &lt;span class=&quot;math-container&quot;&gt;$C$&lt;/span&gt;. If you want to assign &lt;span class=&quot;math-container&quot;&gt;$E = D U A$&lt;/span&gt; that easy too.&lt;/p&gt;&#xA;&lt;p&gt;Your equation doesn't work because |A| and |B| are both 0 under UPGMA, i.e. is pairwise so d(A,A) is always zero. Thus under your equation &lt;span class=&quot;math-container&quot;&gt;$d(C,D) = 0$&lt;/span&gt; which simply means &lt;span class=&quot;math-container&quot;&gt;$D$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$B$&lt;/span&gt; are identical and results in a matrix of 0.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;UPGMA isn't really used in general in biological systems, because the use of the mean rarely makes it a good assumption. I 100% agree has been used in genomics in the modern era, absolutely true, its use is not encouraged because in results in artifactual clusters due its inability to accommodate unequal variation in the underlying genetic process. It could be used in for example in agricultural systems because these conform to uniform distributions, e.g. the distances between apple trees in an orchard. In the natural environment species clump and don't show uniform distributions. UPGMA was used in phylogenetic but the assumption of a global clock very rarely held, so it was discontinued.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3694" LastEditDate="2023-08-13T12:48:16.393" LastActivityDate="2023-08-13T12:48:16.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18812" PostTypeId="1" CreationDate="2022-03-21T06:14:01.657" Score="0" ViewCount="85" Body="&lt;p&gt;I need to know whether my &lt;code&gt;pan-genome&lt;/code&gt; is &lt;code&gt;open or closed&lt;/code&gt;. For that, I need to calculate the &lt;code&gt;Heaps Alpha value&lt;/code&gt; from my &lt;a href=&quot;https://sanger-pathogens.github.io/Roary/&quot; rel=&quot;nofollow noreferrer&quot;&gt;roary pipeline&lt;/a&gt; results. Please help me to do the same.&lt;/p&gt;&#xA;&lt;p&gt;Many thanks in advance.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Directly quoting from the link provide:&lt;/p&gt;&#xA;&lt;p&gt;Roary By Andrew Page based on version 3.11.2 (22-Jan-2018)&lt;/p&gt;&#xA;&lt;p&gt;Roary is a high speed stand alone pan genome pipeline, which takes annotated assemblies in GFF3 format (produced by Prokka (Seemann, 2014)) and calculates the pan genome. Using a standard desktop PC, it can analyse datasets with thousands of samples, something which is computationally infeasible with existing methods, without compromising the quality of the results.&lt;/p&gt;&#xA;" OwnerUserId="4712" LastEditorUserId="3848" LastEditDate="2022-03-21T12:34:25.903" LastActivityDate="2022-03-21T12:34:25.903" Title="Help me to calculate Heaps Alpha value from the roary pangenome pipeline result?" Tags="|sequence-alignment|gene|genome|phylogeny|genomics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18813" PostTypeId="1" CreationDate="2022-03-21T12:40:15.497" Score="2" ViewCount="151" Body="&lt;p&gt;I am performing a bash loop one liner:.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for k in */Assembly/*/*.fastq; do minimap2 -ax map-ont assembly.fasta &lt;span class=&quot;math-container&quot;&gt;$k &amp;gt; mapping.sam; echo $&lt;/span&gt;k; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The file hierarchy is as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;\Drain_4 &#xA;    \Assembly&#xA;        \A4_assembly&#xA;           assembly.fasta&#xA;           A4.fastq&#xA;        \D3_assembly&#xA;        \I8_assembly&#xA;        \U7_assembly&#xA;\Drain_5&#xA;\Drain_7&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Where &lt;code&gt;\&lt;/code&gt; denotes a directory. This is the error i am getting:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[ERROR] failed to open file '.fasta': No such file or directory A4.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Both input files are in the same directory.&lt;/p&gt;&#xA;&lt;p&gt;The fastq is my current directory. I'm guessing i will need to provide the complete pathname for both my input files?&lt;/p&gt;&#xA;&lt;p&gt;I should mention that in all folders the assembly.fasta file is named the same but the fastq file is different (has a different prefix depending on which folder it is in.&lt;/p&gt;&#xA;" OwnerUserId="15239" LastEditorUserId="15239" LastEditDate="2022-03-22T09:39:47.590" LastActivityDate="2022-03-24T05:31:28.103" Title="Mapping to assemblies in subdirectories" Tags="|bash|linux|loop|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18815" PostTypeId="1" CreationDate="2022-03-21T16:20:38.893" Score="0" ViewCount="67" Body="&lt;p&gt;I'd like to perform a search for possibly unidentified splicing variants of a specific protein in &lt;em&gt;A. Thaliana&lt;/em&gt;. I do not have my own RNA-seq data. It has been suggested to me to use this workflow:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Download RNA-seq data from SRA with &lt;code&gt;fastq-dump&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Construct an index of reference sequences with &lt;code&gt;bowtie2-build&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Map the reads from RNA-seq data to the reference sequences with &lt;code&gt;bowtie2&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Remove non-specific mappings with &lt;code&gt;samtools view -q 10&lt;/code&gt;. Sort the mappings with &lt;code&gt;samtools sort&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Browse the aligned reads with &lt;code&gt;tablet&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I have been given example data of another organism (not &lt;em&gt;A. Thaliana&lt;/em&gt;) and have successfully completed these steps. However, I don't know how to apply that in my research. Specifically, I'd like to know:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there other/better way to do do this?&lt;/li&gt;&#xA;&lt;li&gt;Can I narrow the search/alignment to a specifc gene?&lt;/li&gt;&#xA;&lt;li&gt;Where can I download &lt;em&gt;A. Thaliana&lt;/em&gt;'s reference genome in FASTA format?&lt;/li&gt;&#xA;&lt;li&gt;Which SRA runs should I choose for my analysis?&lt;/li&gt;&#xA;&lt;li&gt;How to interpret the results to get the information on the different splicing variants?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Linking a similar question I asked before: &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/18205/14629&quot;&gt;How to find an unidentified splicing variant of a protein?&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14629" LastEditorUserId="14629" LastEditDate="2022-03-21T16:32:19.723" LastActivityDate="2022-03-21T18:41:43.467" Title="Workflow for identification of new splicing variants from RNA-seq data?" Tags="|rna-seq|rna-splicing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18816" PostTypeId="2" ParentId="18813" CreationDate="2022-03-21T16:42:32.163" Score="0" Body="&lt;p&gt;The solution is quite simple you issue the one liner either from within &lt;code&gt;A4_assembly&lt;/code&gt; directory or else &lt;code&gt;NC_D2&lt;/code&gt; (see below). It will then find &lt;code&gt;assembly.fasta&lt;/code&gt; and the error will disappear. That's because the current working directory is assigned to the directory in which the one liner is issued. Directory A4_assembly is where &lt;code&gt;assembly.fasta&lt;/code&gt; is situated, thus is will work.&lt;/p&gt;&#xA;&lt;p&gt;If will fail to find the &lt;code&gt;fastq&lt;/code&gt; file thus you will need to rewrite the loop, simple,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for k in *.fastq; do minimap2 -ax map-ont assembly.fasta &lt;span class=&quot;math-container&quot;&gt;$k &amp;gt; mapping.sam; echo $&lt;/span&gt;k; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... will work, executed from within &lt;code&gt;Assembly_4&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The one liner suggests you are executing the command from the directory above the &lt;code&gt;Drain_4&lt;/code&gt; directory.&#xA;If this is how you intended the one liner to operate I don't think its the solution to the one liner, because there is a risk you will encounter logical errors (not bugs) ..&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;*/Assembly/*/*.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you did execute this commend from above Drain_4 it will looking all over the place here in every Drain and every assembly directory therein. If you executed this anywhere else it will fail to find *.fastq because the start of the wildcards is the current working directory in a bash for relative paths.&lt;/p&gt;&#xA;&lt;p&gt;However, when you dump the file it goes into one one place:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; &amp;gt; mapping.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So there is a risk are performing loads and loads of minimaps but they are simply overwriting each of other, i.e. they are replacing each other.&lt;/p&gt;&#xA;&lt;p&gt;The other issue is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;NC_D2/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This directory is not represented in your directory file.&lt;/p&gt;&#xA;&lt;p&gt;Fundamentally the problem is that &lt;code&gt;assembly.fasta&lt;/code&gt; is always in one location which appears to be in the directory above the directory structure you described, viz. &lt;code&gt;*/Assembly/*/*.fastq&lt;/code&gt; ... according to the one liner, but in the directory structure its in &lt;code&gt;A4_assembly&lt;/code&gt; it is also likely to be in &lt;code&gt;\D3_assembly&lt;/code&gt; amongst loads of others and other Drains (master directories).&lt;/p&gt;&#xA;&lt;p&gt;Thus even if it 'finds' the reference genome (if that is what it is?), it will see that for everything ... and I mean everything.&lt;/p&gt;&#xA;&lt;p&gt;There are dynamic solutions to what I think the code using bash is looking to achieve because you can issue regex from within the do ... done, or simply do it as a script.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I suggest to simply execute the first script I wrote in every Assembly directory then everything will simply work and there's no risk of logical errors.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-03-21T19:27:25.327" LastActivityDate="2022-03-21T19:27:25.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18817" PostTypeId="2" ParentId="18815" CreationDate="2022-03-21T18:41:43.467" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Is there other/better way to do do this?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There is almost always a better way for pretty much anything but in this case I would stick with what your colleagues are using/suggesting. Once you have a worked-out case, you can always improve if needed. Other than that what is suggested to you seems reasonable to me (I have done sth similar for a gene family of some 100 genes using a workflow similar to what was suggested to you)&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Can I narrow the search/alignment to a specifc gene?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Technically yes, instead of the whole genome, you can use a subset of the genome. This will present some problems, for example since the aligner will not be able to find native &amp;quot;targets&amp;quot; for most of the reads, some reads will map to your &amp;quot;new reference&amp;quot; (genome subset) in a &amp;quot;False-positive&amp;quot; manner. You will need to play with the settings. I would advise against this.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Where can I download A. Thaliana's reference genome in FASTA format?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Genbank, EMBL, ... There might be a specific repo for your organism like there is Flybase for Drosophila species and Xenbase for Xenopus species.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Which SRA runs should I choose for my analysis?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;RNA-seq experiments conducted on your species of interest and ideally on a tissue where your genes of interest are highly expressed. If there is none, you could use a similar organism but this would come with some problems. The most ideal would be RNA-seq with long-read sequencing (like Pacbio).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;How to interpret the results to get the information on the different&#xA;splicing variants?&#xA;When you visualize the aligned reads and compare these with existing gene annotation model, it will immediately be obvious if there are novel isoforms, could be due to &amp;quot;new&amp;quot; or not-yet-known exons/splice signals.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2022-03-21T18:41:43.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18818" PostTypeId="2" ParentId="18813" CreationDate="2022-03-21T21:21:11.300" Score="2" Body="&lt;p&gt;The most likely issue is that the &lt;code&gt;assembly.fasta&lt;/code&gt; file is not found in the current directory. Taking @m's comments into consideration, a minimal rewriting of your script to fix this would be as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for k in */Assembly/*/*.fastq; do minimap2 -ax map-ont &lt;span class=&quot;math-container&quot;&gt;$(dirname $&lt;/span&gt;{k})/assembly.fasta &lt;span class=&quot;math-container&quot;&gt;$k &amp;gt; $&lt;/span&gt;(dirname &lt;span class=&quot;math-container&quot;&gt;${k})/mapping.sam; echo $&lt;/span&gt;k; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I'd recommend at least adding additional quote and brace protection, just in case file names contained spaces:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for k in */Assembly/*/*.fastq;&#xA;  fDir = &lt;span class=&quot;math-container&quot;&gt;$(dirname $&lt;/span&gt;{k});&#xA;  do minimap2 -ax map-ont &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${fDir}/assembly.fasta&quot; &quot;$&lt;/span&gt;{k}&amp;quot; &amp;gt; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${fDir}/mapping.sam&quot;;&#xA;  echo $&lt;/span&gt;{k};&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;edit: fixed up variable capitalisation&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-03-24T05:31:28.103" LastActivityDate="2022-03-24T05:31:28.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18819" PostTypeId="1" CreationDate="2022-03-21T21:27:47.807" Score="1" ViewCount="37" Body="&lt;p&gt;I am working with non-model plant RNA samples which we have been deep sequenced and analysed using STAR aligner under default parameters.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Aim&lt;/strong&gt; We would like to conduct SNP discovery of these samples.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Objective&lt;/strong&gt; Our ultimate goal with this genotypic data is to search for variants (both SNPs and indels) that fall within genes from a cohort of gene families, which may be responsible for phenotypic variation that we have assessed among our sample plants.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; The RNA alignments use a reference genome of the species I am dealing with, and following &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035531192-RNAseq-short-variant-discovery-SNPs-Indels-&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATK best practices&lt;/a&gt; and produced a VCF file for each of my samples.&lt;/p&gt;&#xA;&lt;p&gt;There are three things that give me pause,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Am I correct in merging the .VCF files after calling SNPs, rather than merging .BAM files and then calling SNPs?&lt;/li&gt;&#xA;&lt;li&gt;What filters are required for MAF, read-depth, base call quality, missing data, snp proximity to each other and proximity to indels, etc.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Should I produce plots to make these decisions?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Lastly how do I deal with specific allele expression skewing the results, my assumption so far is that other studies seem to have accounted for this by merely using a low bar for the MAF filter (such as &amp;gt;= 0.1), thus ensuring that as many true SNPs as possible are captured.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Any insight into any of these questions, or a point in the direction of where I might find it, would be much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="15242" LastEditorUserId="3848" LastEditDate="2022-03-21T21:43:55.940" LastActivityDate="2022-03-21T21:43:55.940" Title="RNAseq SNP discovery: deciding upon filters and dealing with allele expression bias" Tags="|rna-seq|snp|transcriptome|filtering|genotyping|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18820" PostTypeId="1" AcceptedAnswerId="18828" CreationDate="2022-03-22T08:38:54.040" Score="0" ViewCount="45" Body="&lt;p&gt;We are working on an algorithm to assign proteins only on the basis of alpha-carbons.&lt;/p&gt;&#xA;&lt;p&gt;My teacher told me to draw a diagram to represent hydrogen bonds only on the basis of alpha-carbons.&lt;/p&gt;&#xA;&lt;p&gt;I showed him the following picture:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bHeUA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bHeUA.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;He said this picture doesn't count, as this diagram shows all the atoms.&lt;/p&gt;&#xA;&lt;p&gt;I couldn't figure out how to do this without showing oxygen, non-alpha-carbon, and nitrogen atoms.&lt;/p&gt;&#xA;&lt;p&gt;How can I draw a diagram of hydrogen bonds only on the basis of C-alpha backbone?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="73" LastEditDate="2022-03-22T22:59:11.090" LastActivityDate="2022-03-23T14:05:09.963" Title="How can I draw a diagram of hydrogen bonds only on the basis of C-alpha backbone?" Tags="|protein-structure|homework|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18821" PostTypeId="1" AcceptedAnswerId="18826" CreationDate="2022-03-22T12:54:40.223" Score="3" ViewCount="448" Body="&lt;p&gt;I'm looking to make a two, two range colour scheme heatmap where each range corresponds to a separate triangle in the heatmap square.&lt;/p&gt;&#xA;&lt;p&gt;I am looking for a relevant guide for this heatmap. Any recommended package/library or guides? Thanks!&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0qE8O.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0qE8O.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14560" LastEditorUserId="3848" LastEditDate="2022-03-22T13:42:25.793" LastActivityDate="2022-03-22T17:17:22.953" Title="Constructing Two Triangle Heatmaps in One Square" Tags="|r|python|heatmap|complexheatmap|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18822" PostTypeId="1" AcceptedAnswerId="18824" CreationDate="2022-03-22T13:52:27.347" Score="3" ViewCount="107" Body="&lt;p&gt;I want to edit a tree that I got from BEAST2 treeannotator in nexus-format.&lt;/p&gt;&#xA;&lt;p&gt;Usually I use the module Phylo from Biopython for such work but &lt;code&gt;Phylo.read(r&amp;quot;filename.tree&amp;quot;, &amp;quot;nexus&amp;quot;)&lt;/code&gt; gave me the next exception:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;---------------------------------------------------------------------------&#xA;NexusError                                Traceback (most recent call last)&#xA;Input In [29], in &amp;lt;cell line: 1&amp;gt;()&#xA;----&amp;gt; 1 Phylo.read(r&amp;quot;filename.tree&amp;quot;, &amp;quot;nexus&amp;quot;)&#xA;&#xA;File ~\miniconda3\lib\site-packages\Bio\Phylo\_io.py:60, in read(file, format, **kwargs)&#xA;     58 try:&#xA;     59     tree_gen = parse(file, format, **kwargs)&#xA;---&amp;gt; 60     tree = next(tree_gen)&#xA;     61 except StopIteration:&#xA;     62     raise ValueError(&amp;quot;There are no trees in this file.&amp;quot;) from None&#xA;&#xA;File ~\miniconda3\lib\site-packages\Bio\Phylo\_io.py:49, in parse(file, format, **kwargs)&#xA;     34 &amp;quot;&amp;quot;&amp;quot;Parse a file iteratively, and yield each of the trees it contains.&#xA;     35 &#xA;     36 If a file only contains one tree, this still returns an iterable object that&#xA;   (...)&#xA;     46 &#xA;     47 &amp;quot;&amp;quot;&amp;quot;&#xA;     48 with File.as_handle(file) as fp:&#xA;---&amp;gt; 49     yield from getattr(supported_formats[format], &amp;quot;parse&amp;quot;)(fp, **kwargs)&#xA;&#xA;File ~\miniconda3\lib\site-packages\Bio\Phylo\NexusIO.py:40, in parse(handle)&#xA;     32 def parse(handle):&#xA;     33     &amp;quot;&amp;quot;&amp;quot;Parse the trees in a Nexus file.&#xA;     34 &#xA;     35     Uses the old Nexus.Trees parser to extract the trees, converts them back to&#xA;   (...)&#xA;     38     eventually change Nexus to use the new NewickIO parser directly.)&#xA;     39     &amp;quot;&amp;quot;&amp;quot;&#xA;---&amp;gt; 40     nex = Nexus.Nexus(handle)&#xA;     42     # NB: Once Nexus.Trees is modified to use Tree.Newick objects, do this:&#xA;     43     # return iter(nex.trees)&#xA;     44     # Until then, convert the Nexus.Trees.Tree object hierarchy:&#xA;     45     def node2clade(nxtree, node):&#xA;&#xA;File ~\miniconda3\lib\site-packages\Bio\Nexus\Nexus.py:668, in Nexus.__init__(self, input)&#xA;    665 self.options[&amp;quot;gapmode&amp;quot;] = &amp;quot;missing&amp;quot;&#xA;    667 if input:&#xA;--&amp;gt; 668     self.read(input)&#xA;    669 else:&#xA;    670     self.read(DEFAULTNEXUS)&#xA;&#xA;File ~\miniconda3\lib\site-packages\Bio\Nexus\Nexus.py:718, in Nexus.read(self, input)&#xA;    716     break&#xA;    717 if title in KNOWN_NEXUS_BLOCKS:&#xA;--&amp;gt; 718     self._parse_nexus_block(title, contents)&#xA;    719 else:&#xA;    720     self._unknown_nexus_block(title, contents)&#xA;&#xA;File ~\miniconda3\lib\site-packages\Bio\Nexus\Nexus.py:759, in Nexus._parse_nexus_block(self, title, contents)&#xA;    757 for line in block.commandlines:&#xA;    758     try:&#xA;--&amp;gt; 759         getattr(self, &amp;quot;_&amp;quot; + line.command)(line.options)&#xA;    760     except AttributeError:&#xA;    761         raise NexusError(&amp;quot;Unknown command: %s &amp;quot; % line.command) from None&#xA;&#xA;File ~\miniconda3\lib\site-packages\Bio\Nexus\Nexus.py:1144, in Nexus._translate(self, options)&#xA;   1142         break&#xA;   1143     elif c != &amp;quot;,&amp;quot;:&#xA;-&amp;gt; 1144         raise NexusError(&amp;quot;Missing ',' in line %s.&amp;quot; % options)&#xA;   1145 except NexusError:&#xA;   1146     raise&#xA;&#xA;NexusError: Missing ',' in line 1 AB298157.1_2015_-7.9133750332192605_114.8086828279248, 2 AB298158.1_2007_-8.41698974207…&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Using &lt;code&gt;Nexus.read(Nexus(), input=r&amp;quot;filename.tree&amp;quot;)&lt;/code&gt; gave the same result. Please could anyone help with this? I cannot understand the reason of this error because &lt;a href=&quot;https://drive.google.com/file/d/1Ex_dbdA6IdkRBU33qLuo0sOXty2k3JHu/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;nexus file&lt;/a&gt; looks correct.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Nexus format&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;A NEXUS tree file is a rigid but very useful format which can be used to store a phylogenetic tree, along with associated metadata, for example, the alignment.&lt;/p&gt;&#xA;&lt;p&gt;It will always comprise a:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;TAXA block&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The TAXA block contains information about taxa.&#xA;... can comprise a,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;DATA block&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The DATA block contains the data matrix (e.g. sequence alignment).&#xA;... in this context importantly will have a,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;TREES block&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The TREES block contains trees described via the format, e.g. ((A,B),C);:&lt;/p&gt;&#xA;&lt;p&gt;Each block will open with a BEGIN statement and terminate with an END statement. Again its versatility to store specific information within a block is valuable property of the format.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;NB!&lt;/strong&gt; I encountered some problems with my following work. So I would like to warn readers about them.&#xA;It is allowed in the &lt;strong&gt;newick&lt;/strong&gt; format to surround the label with &lt;em&gt;quotes&lt;/em&gt;, &amp;amp; some programmes or scripts add them to those names that have ambiguous characters. But it can lead to exceptions during the following phylogenetic analysis, for instance, in &lt;strong&gt;BEAST&lt;/strong&gt;. I wish you would be careful with this.&lt;/p&gt;&#xA;" OwnerUserId="15161" LastEditorUserId="15161" LastEditDate="2022-04-15T21:52:02.947" LastActivityDate="2022-04-15T21:52:02.947" Title="Missing ',' in line when Biopython reads a nexus tree" Tags="|phylogenetics|python|phylogeny|beast|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18824" PostTypeId="2" ParentId="18822" CreationDate="2022-03-22T14:31:31.763" Score="1" Body="&lt;p&gt;That's a complicated output.&lt;/p&gt;&#xA;&lt;p&gt;Two solutions, the easiest is simply to use Andrew Rambaut's &lt;code&gt;figtree&lt;/code&gt; &lt;a href=&quot;http://tree.bio.ed.ac.uk/software/figtree/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://tree.bio.ed.ac.uk/software/figtree/&lt;/a&gt; . Figtree is a very, very good tree drawing program and it will work; it works on Mac, Windows and Linux.&lt;/p&gt;&#xA;&lt;p&gt;You can then export the file from Figure - from memory in numerous formats, e.g. phylip format, and import it into Biopython if that is where it is needed. If you are parsing Beast output, Beast has A LOT of internal tools and you might be attempting to replace those via Biopython. I would avoid it. Also there is the package for processing Beast output (which I forget), I think its called Tracer.&lt;/p&gt;&#xA;&lt;p&gt;With Biopython try parse, it might help if there are multiple trees:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Phylo&#xA;trees = Phylo.parse(&amp;quot;examples.nex&amp;quot;, &amp;quot;nexus&amp;quot;)&#xA;for tree in trees:&#xA;    print(tree.name)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would avoid the 'r'. I agree it makes no difference, but really you'd need to be using unusual file names for 'r' to be needed. Its possible you are using it to get around \s, i.e. spaces in the file name. You could just think about simply not including \ and space type characters in file names. In regex it is used for certain but that's because its 'inescapable' (no pun intended, but I thought it was funny).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Just to wrap up and clarify, I recommended:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;To open the tree in Figtree;&lt;/li&gt;&#xA;&lt;li&gt;Find the export menu and output as either 'newick' or preferably 'nexus' or both;&lt;/li&gt;&#xA;&lt;li&gt;Re-run Biopython importing as 'nexus' or 'newick' (if nexus fails).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The OP reported ...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I have just compared them (nexus-&amp;gt;FigTree-&amp;gt;nexus).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Good so it opened up in FigTree and exported, this was expected.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Fig Tree compose all this actually in newick &amp;amp; stash it in nexus block decor. While in the previous case newick tree contains IDs, &amp;amp; there is a seclude block with ID-name pairs&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I missed this bit, &amp;quot;newick stashed in a nexus block&amp;quot;. That is potentially really cool because I suspect that is within the TREES block. If that's correct then that is nexus format. This should now open Biopython.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Why the bug?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;One of the issues with Nexus is there are two variants regarding how the TAXA block relates to the TREES block. In one format they recode the taxa - I definitely understand why. Recoding is possibly (very likely) be being used in Beast to increase algorithm efficiency - it makes no sense slushing a load of taxa names around a convergence algorithm. Writing all that stuff out is not cool either.&lt;/p&gt;&#xA;&lt;p&gt;However, when you are looking at a tree that's not what you want at all - efficiency is an irrelevance and readability is important. Thus recoding the taxa labels is definitely not wanted. This is a possible explanation for the read error.&lt;/p&gt;&#xA;&lt;p&gt;Whatever the reason for the bug I am certain Biopython has a work around because in phylip alignment format this is certainly true. There is a custom module that handles the specific issues the format generates.&lt;/p&gt;&#xA;&lt;p&gt;It was fun but time to move on.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="73" LastEditDate="2022-03-22T23:04:01.123" LastActivityDate="2022-03-22T23:04:01.123" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18826" PostTypeId="2" ParentId="18821" CreationDate="2022-03-22T14:51:44.250" Score="1" Body="&lt;p&gt;This solution doesn't reproduce the plot, but it would do if the numerical ranges between the triangles do not overlap.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;import seaborn as sns&#xA;&#xA;xx = plt.get_cmap('rocket').copy()&#xA;xx.set_under('none')&#xA;yy = plt.get_cmap('mako').copy()&#xA;yy.set_over('none')&#xA;axis = sns.heatmap(dataframe, vmin=-1, cmap=xx, cbar_kws={'pad': -0.04})&#xA;sns.heatmap(dataframe, vmin=0, vmax=1, cmap=yy, ax=axis)&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My colour schemes are not great but they are distinct, you can search for more here:&#xA;&lt;a href=&quot;https://seaborn.pydata.org/tutorial/color_palettes.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://seaborn.pydata.org/tutorial/color_palettes.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The idea is this would address a situation where you compare the &lt;em&gt;r&lt;/em&gt;-value OR a &lt;em&gt;p&lt;/em&gt;-value against another set of values where are &amp;gt; 1. That's what the &lt;code&gt;vmin&lt;/code&gt; and &lt;code&gt;vmax&lt;/code&gt; are doing.&lt;/p&gt;&#xA;&lt;p&gt;This is a ONE np array or dataframe solution. In other words you will need to combine your data sets. The solution you want is a two dataframes in one heatmap square. You can put two triangles side-by-side, i.e. one dataframe per heatmap, but I don't know who to place two separate dataframes in one heatmap.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-03-22T17:17:22.953" LastActivityDate="2022-03-22T17:17:22.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18827" PostTypeId="1" AcceptedAnswerId="18831" CreationDate="2022-03-23T00:08:27.130" Score="2" ViewCount="814" Body="&lt;p&gt;Within the example below a WDL array of strings, &lt;code&gt;test&lt;/code&gt; is defined as &lt;code&gt;&amp;quot;a&amp;quot;,&amp;quot;b&amp;quot;,&amp;quot;c&amp;quot;,&amp;quot;d&amp;quot;&lt;/code&gt; the value &lt;code&gt;total&lt;/code&gt; is a variable that signifies the length of the WDL array. Using the bash for loop, which will iterate from 1 up to length of the array, I seek to print out the values at each indicy in the array and output it to another array. For example when the loop is on 1 the value at indicy 1 within the array will be printed out in this case &lt;code&gt;&amp;quot;a&amp;quot;&lt;/code&gt; and stored in the array out, if the loop is on 2 then the value &lt;code&gt;&amp;quot;b&amp;quot;&lt;/code&gt; will be printed out and stored in the array out, etc. I tried the below example, however, I get an error when running. Can anyone help me with this problem?&lt;/p&gt;&#xA;&lt;p&gt;How I tried to do this in WDL:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;task A {&#xA;    input {&#xA;        Array[String] test = [&amp;quot;a&amp;quot;,&amp;quot;b&amp;quot;,&amp;quot;c&amp;quot;,&amp;quot;d&amp;quot;]&#xA;        Int total = length(test)&#xA;    }&#xA;&#xA;    command {&#xA;       for (( c = 1; c &amp;lt;= &lt;span class=&quot;math-container&quot;&gt;${total}; c++ ))&#xA;       do &#xA;           echo $&lt;/span&gt;{test[$c]}&#xA;       done&#xA;    }&#xA;    output {&#xA;        Array[String] out = read_lines(stdout())&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15246" LastEditorUserId="15246" LastEditDate="2022-03-23T10:55:50.463" LastActivityDate="2022-03-24T10:08:44.837" Title="How do I use bash for loops to loop through a WDL Array?" Tags="|bash|wdl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18828" PostTypeId="2" ParentId="18820" CreationDate="2022-03-23T14:05:09.963" Score="2" Body="&lt;p&gt;There are three options:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;it's a badly phrased Ramachandran plot question&lt;/li&gt;&#xA;&lt;li&gt;it's a question aimed at the describing the properties of different SS&lt;/li&gt;&#xA;&lt;li&gt;it's a trick question&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;φ &amp;amp; ψ can be used to classify SS (cf. Ramachandran plot), these are however proper dihedrals, so require 4 atoms, which you lack.&lt;/p&gt;&#xA;&lt;p&gt;The sequential Cαs in a strand are increasingly distant by 3–4Å, while in a helix there's a periodicity every 3–4 residues. The images you were given are likely to remind you that between two strands in a sheet the Cαs are ~5Å away from the other strand, but the order is different —the indices will increase together in parallel strands, but not in anti-parallel strands.&lt;/p&gt;&#xA;&lt;p&gt;A C-alpha trace has the issue that the residues are often not known (&lt;code&gt;UNK&lt;/code&gt; is technically represented as homoalanine, but is an unknown) and the resolution is &amp;gt; 4Å. Any SS allocation will be highly inaccurate making it a trick question.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2022-03-23T14:05:09.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18829" PostTypeId="2" ParentId="18762" CreationDate="2022-03-23T16:17:18.633" Score="2" Body="&lt;p&gt;&lt;strong&gt;Convention&lt;/strong&gt;&#xA;The convention is it should be in ascending ordering, i.e. 0/1. The question is, why the convention broken?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Phasing&lt;/strong&gt; I think, at a guess, the answer is it depends what reference genome you are using. I would suggest that this result would not occur if the reference genome is mum (or dad) and the alleles are occurring in the child. The output supports this idea because it is marked with '/', i.e. they are not phased, thus the reference genome is neither mum nor dad.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Example provided&lt;/strong&gt; The thing I would like to point out is in the example given &lt;code&gt;117120306&lt;/code&gt; and &lt;code&gt;117120317&lt;/code&gt; are not randomly scattered across the genome but appear to be 11 bp apart. For a human genome those are very strongly linked.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Linkage hypothesis&lt;/strong&gt; Thus I would suggest the answer is linkage in this example, given the reference genome isn't a parent. The mutations are occurring on different strands therefore better represented as 0/1 and then 1/0, i.e. one strand comprises 'A' allele and the opposite strand is the 'G' allele.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Why is unphased linkage important&lt;/strong&gt; In this scenario the interpretation could be hugely important because it would be stating that each allele has come from different parents. This would be more important than maintaining convention. In an inherited disorder there is a risk the child has a dominant phenotype, when each parent could have been heterozygous. Clearly it depends whether the mutation is forcing an amino acid mutation, at 11bp it is certainly possible, i.e. mutation in the first codon upstream and mutation in the second codon downstream.&lt;/p&gt;&#xA;&lt;p&gt;More generally in any population analysis, linkage is an enormously important issue and central to any calculation. The point is simply that most analysis infers independent segregation of each allele and in this example that remains a good assumption because the allele (I am suggesting) are on opposite strands. If they were on the same strand it would create a bias away from panmixia.&lt;/p&gt;&#xA;&lt;p&gt;The hypothesis is formed on this example alone and if the alleles were 10 megabp apart it would be impossible to conclude they were linked, therefore the convention of 0/1 should hold. That's my guess, its not a bad guess.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-03-23T16:17:18.633" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18831" PostTypeId="2" ParentId="18827" CreationDate="2022-03-24T10:08:44.837" Score="3" Body="&lt;p&gt;I suspect the issue here is that you are trying to use a bash variable inside of a WDL placeholder. The &lt;code&gt;${}&lt;/code&gt; are interpreted by WDL as places where some value should should be inserted into the command. So when you say &lt;code&gt;echo ${test[$c]}&lt;/code&gt;, &lt;code&gt;test[$c]&lt;/code&gt; is seen a WDL expression. This isn't a valid WDL expression, causing a parsing error.&lt;/p&gt;&#xA;&lt;p&gt;The choice to use &lt;code&gt;${}&lt;/code&gt; as placeholder syntax in WDL was a rather inconvenient decision by the WDL team, because it clashes with the bash syntax. There is an alternative syntax available for the command section in WDL 1.0, which is preferred for this very reason. This alternative uses &lt;code&gt;~{}&lt;/code&gt; as placeholder syntax, instead of &lt;code&gt;${}&lt;/code&gt;, and wraps the command in &lt;code&gt;&amp;lt;&amp;lt;&amp;lt; &amp;gt;&amp;gt;&amp;gt;&lt;/code&gt; (referred to as the &amp;quot;heredoc-like syntax&amp;quot;), rather than the usual &lt;code&gt;{ }&lt;/code&gt;. In this heredoc-like syntax the &lt;code&gt;${}&lt;/code&gt; isn't seen as a WDL placeholder and can, thus, be used in bash commands like normal. You can read about it in the &lt;a href=&quot;https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#command-section&quot; rel=&quot;nofollow noreferrer&quot;&gt;WDL SPEC&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Using this syntax your WDL would look something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;version 1.0&#xA;&#xA;workflow test {&#xA;    call A&#xA;}&#xA;&#xA;task A {&#xA;    input {&#xA;        Array[String] test = [&amp;quot;a&amp;quot;,&amp;quot;b&amp;quot;,&amp;quot;c&amp;quot;,&amp;quot;d&amp;quot;]&#xA;        Int total = length(test)&#xA;    }&#xA;&#xA;    command &amp;lt;&amp;lt;&amp;lt;&#xA;       ARRAY=(~{sep=&amp;quot; &amp;quot; test}) # Load array into bash variable&#xA;       for (( c = 0; c &amp;lt; ~{total}; c++ )) # bash array are 0-indexed ;)&#xA;       do&#xA;           echo &lt;span class=&quot;math-container&quot;&gt;${ARRAY[$&lt;/span&gt;c]}&#xA;       done&#xA;    &amp;gt;&amp;gt;&amp;gt;&#xA;&#xA;    output {&#xA;        Array[String] out = read_lines(stdout())&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that if the values in your array are more complex and might include characters with special meaning in bash (like spaces), it would probably be better to wrap the values in single quotes like so:&lt;br /&gt;&#xA;&lt;code&gt;ARRAY=('~{sep=&amp;quot;' '&amp;quot; test}')&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;And just in case you weren't aware, bash for-loops can also loop over arrays directly:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;command &amp;lt;&amp;lt;&amp;lt;&#xA;       for c in ~{sep=&amp;quot; &amp;quot; test}&#xA;       do&#xA;           echo $c&#xA;       done&#xA;&amp;gt;&amp;gt;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6955" LastActivityDate="2022-03-24T10:08:44.837" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18832" PostTypeId="1" CreationDate="2022-03-24T11:38:56.433" Score="0" ViewCount="51" Body="&lt;p&gt;Where I can find a database or tool to give me the &lt;code&gt;target genes of PIWI (piRNAs)&lt;/code&gt; in &lt;code&gt;human&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;I found one but works for worm like &lt;code&gt;piRTarBase&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;There are a lot for miRNAs around but not for piRNAs&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2022-03-24T13:29:53.957" LastActivityDate="2022-03-28T16:51:21.357" Title="Target genes for piRNA" Tags="|rna-seq|gene|genome|non-coding|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18833" PostTypeId="1" AcceptedAnswerId="18843" CreationDate="2022-03-24T16:12:47.957" Score="4" ViewCount="245" Body="&lt;p&gt;BEAST2 is currently being used for tree reconstruction prior phylogeographic analysis. The sample size and loci are described below.&lt;/p&gt;&#xA;&lt;p&gt;I thought that BEAST/BEAST2 does not allow negative lengths of branches on prior distribution level so I was surprised when I have looked at my consensus tree using &lt;strong&gt;FigTree&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ur9MA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ur9MA.png&quot; alt=&quot;A part of one of my trees&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; What could be the reason of this and how to wrestle with this phenomenon?&lt;/p&gt;&#xA;&lt;p&gt;I am analysing about 1500 taxa of HAV (subtype: IA, region: VP1) under the next parameters:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;GTR+Г,&lt;/li&gt;&#xA;&lt;li&gt;Strict Clock,&lt;/li&gt;&#xA;&lt;li&gt;Coalescent constant population.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The tree was generated by &lt;strong&gt;TreeAnnotator&lt;/strong&gt; through&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;~/beast/bin/treeannotator -heights median -burnin 20 -limit 0.5 VP1_test_bt_ExpPop.trees VP1_test_bt_ExpPop.tree&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15161" LastEditorUserId="3848" LastEditDate="2022-03-25T17:43:55.963" LastActivityDate="2022-03-28T10:57:48.880" Title="What can be the reason of getting negative branches lengths after BEAST analysis?" Tags="|phylogenetics|phylogeny|beast|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18834" PostTypeId="2" ParentId="18803" CreationDate="2022-03-24T16:45:49.403" Score="1" Body="&lt;p&gt;I better explain this style of parameter rich MCMC takes ages and the issues are how speed this up.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Having given this some thought my suggestion is to install &lt;code&gt;miniconda&lt;/code&gt; or &lt;code&gt;anaconda&lt;/code&gt; and then install the port to OpenCL, then install beast and beagle.&lt;/p&gt;&#xA;&lt;p&gt;Anaconda is here: &lt;a href=&quot;https://www.anaconda.com/products/individual&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.anaconda.com/products/individual&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then install openCL via conda:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; conda create -n beasty python=3.9&#xA; conda activate beasty&#xA; conda install -c conda-forge pyopencl &#xA; conda install -c bioconda beagle&#xA; conda install -c bioconda beast2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My idea was you'd access OpenCL via python described &lt;a href=&quot;https://documen.tician.de/pyopencl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. This wouldn't work ... its complicated.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Traditionally OpenCL and Beagle didn't work ... on an Intel Mac&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;2 : Intel(R) HD Graphics 620 (OpenCL 2.1 )&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However, if this is your chip set - I wouldn't give up. In any other scenario  I wouldn't pursue this - personally, below.&lt;/p&gt;&#xA;&lt;p&gt;One alternative is NVIDA e.g. Cloud computing used via CUDA and Beagle. CUDA will easily install and Beagle stacks straight on top.&lt;/p&gt;&#xA;&lt;p&gt;Any heavy duty MCMC is very amenable to traditional CPU parallelisation - if you think at it - it would be and thats why GPU - CUDA will work. Personally, looking at your resources I would parallelise under you cluster using a MINIMUM of 8-cores per calculation outside CUDA/OpenCL. If that has not already been done, that is the solution in context in my opinion. Unless you're a Google developer few will have access to a NVIDA cluster in any case sufficient CPU parallelisation should be able to achieve the runtime close to that of a good GPU (its gonna be a lot of CPUs though).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&#xA;Honestly, my personal advise is that phylogenetics sump clusters no matter what the calculation is being used and that has always been the case. The best solution is to max out on available cores. When eukaryote genome guys complain about 1000 hour runtimes (obviously thats the total CPU parallelised), thats a routine calculation in phylogenetics. Buying in NVIDA doesn't necessarily solve this, because whilst MCMC is massively accelerated for maximum likelihood isn't accelerated to that extent by comparison.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Extra information&lt;/strong&gt; In case the OP is not aware, the enormous advantage of Beast2 is the ability to resume an MCMC at the point it ended (or some checkpoint therein), you don't need to run the whole thing again. You should seek to exploit that, so a calculation started on a laptop can be finished on a cluster. I can't state sufficiently how powerful that is, it stands to reason because the random number stream is declared and every sample thereafter. Keeping all calculations is standard practice in any case, but especially with a Beast2 run and don't every discard it (hard disk is cheap right?). You might find you have to return to it and its a whole lot better than needing to do the whole thing again.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Its time to move on.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-03-29T16:56:11.007" LastActivityDate="2022-03-29T16:56:11.007" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="18835" PostTypeId="2" ParentId="18833" CreationDate="2022-03-25T11:50:49.330" Score="0" Body="&lt;p&gt;I am avoiding directly answering the question because in summary, I feel, this given issue is akin to project supervision and is simply too much time.&lt;/p&gt;&#xA;&lt;p&gt;One issue raised however is something called 'burnin'.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Burnin is foundational to MCMC&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In the comments I stated the burnin is too low. The burnin used ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;~/beast/bin/treeannotator -heights median -burnin 20 ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The OP believes the burnin represents a percentage, i.e. 20%.&lt;/p&gt;&#xA;&lt;p&gt;Citing the documentation on TreeAnnotator at &lt;a href=&quot;https://beast.community/treeannotator&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://beast.community/treeannotator&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;BurnIn (as trees)&#xA;In this case you just specify the actual number of&#xA;trees you wish to discard rather than the number of states.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In this given case, considering this is 1500 taxa, I would estimate the actual burnin  used in comparison to an initial assessment of completion time (technically called 'convergence') to be 0.4% maximum, i.e. well over 0.5 log smaller than the burnin believed to be used.&lt;/p&gt;&#xA;&lt;p&gt;It is easy to assess empirically e.g. comparing &lt;code&gt;wc infile&lt;/code&gt; and &lt;code&gt;wc outfile&lt;/code&gt;, where infile and outfield are MCMC treefiles.&lt;/p&gt;&#xA;&lt;p&gt;A standard burnin is 10% of the total MCMC. The threshold would be independently assessed and if inadequate a higher burnin would be used, of which 20% is reasonable.&lt;/p&gt;&#xA;&lt;p&gt;To explain the documentation, in an MCMC only a sample of the trees are written to disk (often 1 in 1000, but it can be anything) and the burnin is 10% of the total (unsampled).&lt;/p&gt;&#xA;&lt;p&gt;Thus factoring the sampling is important in calculating the precise burnin in any MCMC output analysis. Otherwise the burnin that was intended, isn't the burnin used in practice.&lt;/p&gt;&#xA;&lt;p&gt;Perhaps you can now see my point?  Not the specific technical matter above, but the issue concerning complexity and time commitment.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To address the matter concerning Beast1 vs Beast2 raised as a comment below, i.e. the proposal the original MCMC was performed in Beast2 and Beast2 is exempt. The issue under discussion is TreeAnnotator.&lt;/p&gt;&#xA;&lt;p&gt;The TreeAnnotator section of the Beast2 documentation is here &lt;a href=&quot;https://www.beast2.org/treeannotator/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.beast2.org/treeannotator/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;TREEANNOTATOR ....&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;BurnIn&#xA;This option allows you to select the amount of burn-in, i.e., the&#xA;number of samples that will be discarded at the start of the run, so&#xA;that you are only analysing the part of the trace that is in&#xA;equilibrium.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Again the Beast2 documentation is clear, 'the number of samples', not the percentage or proportion. Furthermore note it states 'samples', not 'states', i.e. the number of trees resulting from the sample frequency that was assigned.&lt;/p&gt;&#xA;&lt;p&gt;A burnin can be performed prior to TreeAnnotator analysis and that is good practice, albeit not essential. This might be where the report output the OP cites was obtained. Furthermore, if this was done here upstream, then no burnin is required for TreeAnnotator. Moreover if a correct upstream burnin was performed, burnin should either be; avoided for TreeAnnotator, or else the total combined burnin calculated and correctly reported. The final output ain't just a tree, there's a lot of important analysis/ 'meta data'.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;  I agree with the OP's second comment below this answer using information from the comment below the question: a value of -0.31 can't be ignored, i.e. its a notable value for any branch length and very large in the given context.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Its definitely time to move on&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;OP upvoted, good luck with the project and all the best.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-03-25T20:57:18.157" LastActivityDate="2022-03-25T20:57:18.157" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="18836" PostTypeId="1" CreationDate="2022-03-26T19:41:15.057" Score="2" ViewCount="38" Body="&lt;p&gt;I need beginner-level study materials for coarse-grained modeling of &lt;strong&gt;proteins&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone supply me with entry level resources besides the Wikipedia article?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Background&lt;/p&gt;&#xA;&lt;p&gt;Coarse grain protein modelling could be very broadly defined as an approximation of all-atom protein models and find current application in folding studies but are early approaches extend back to 1975.&lt;/p&gt;&#xA;&lt;p&gt;Citing a recent review where comments in brackets are for elucidation and are not part of the citation.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Native structures (proteins) are not completely fixed, but they change when&#xA;proteins perform their biological function, interact with other&#xA;biomacromolecules, or undergo unfolding–folding transitions (for example conformational changes).&#xA;Computational modeling of these processes is crucial for ... (examples truncated). Classical atom-level&#xA;molecular modeling can address many of the these tasks, but its&#xA;practical applications are still limited by its algorithmic efficiency&#xA;and the available computing power.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Kmiecik et al &amp;quot;Coarse-Grained Protein Models and Their Applications&amp;quot;  Chem. Rev. 2016, 116, 14, 7898–7936&#xA;&lt;a href=&quot;https://doi.org/10.1021/acs.chemrev.6b00163&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1021/acs.chemrev.6b00163&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="3848" LastEditDate="2022-03-26T20:16:33.283" LastActivityDate="2022-03-26T20:16:33.283" Title="What is coarse-grained modeling of proteins?" Tags="|proteins|protein-structure|modelling|protein-protein-interaction|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18838" PostTypeId="2" ParentId="15455" CreationDate="2022-03-27T19:34:45.647" Score="1" Body="&lt;p&gt;&lt;strong&gt;Brief answer&lt;/strong&gt; Firstly this is not a final solution. The simple answer is to use TreeNode.add_feature() in ETE3 which would enable NHX format to be inserted into the tree. There are two possible ways to implement the solution, but I'm not aware of a 'reader'.  I'll return to this later under the section 'Question (again)'.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;  I assume that the OP made a typo with 'witg ETE3' and probably means 'with ETE3'?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Interest&lt;/strong&gt; However as an approach to molecular phylogeny it is interesting theory and has clear application. Not in a simple tree, but when the volume of data overwhelms optical analysis. I am not aware of this being deployed in phylogeny - not done a literature search though, but would readily automate processing of complex trees.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Theory&lt;/strong&gt; The OP uses the term 'edge' rather than branch, so this is referring to depth first search (DFS) type style calculations. In Phylogenetics there are 'branches' and 'nodes' and these are distinct, in this line of theory, I think, 'edges' can be nodes ... possibly, it is possible an 'edge' connects two nodes, in which an 'edge' is synonymous with a 'branch'.&lt;/p&gt;&#xA;&lt;p&gt;The difficult I have with the information is that edge 0 is not defined. In Phylogenetics edge 0 would probably be the root. It is possibly the 'missing branch' in the trifurcation at the base of the tree. In this line of theory however 0 may simply be a null category, because an edge is not necessarily the same thing as a branch.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Data structure&lt;/strong&gt; The information present is paired. thus ... salamander is 7,0 ... frog is 6,0 ... turtle is 4, 2... We can work this out because the data structure is 7,6,5,4,3,2,1 for odd numbers.&lt;/p&gt;&#xA;&lt;p&gt;7-&amp;gt;0, 6-&amp;gt;0, 5-&amp;gt;2 and so on ...&lt;/p&gt;&#xA;&lt;p&gt;Moreover, 7 is at the top of the tree whilst, 1 is at the bottom and the 'edge' is an arbitrary designation. Phylogenetics wouldn't label nodes in this format, labelling would usually be with respect to the base of the tree (root). If I look at this from my world-view there's no 0-&amp;gt; ... but 5 the ancestor of 7 and 6 goes to turtles.&lt;/p&gt;&#xA;&lt;p&gt;Thus far the information isn't the DFS, its presented as the input, for example to calculate DFS, and can be used calculate all sorts of things in this style of graph theory.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question (again)&lt;/strong&gt; Back to the question, I am not aware of DFS style theory being used in trees, therefore I would not be aware of an interpreter that would read it. If the OP was just wanting to stash the data into a tree via  NHX it could be stacked onto the root lineage. There isn't a root lineage in this tree so I've just added a root at '1:0.01', i.e. a nominal distance to provide the OP with a place to stash the data there. This will read as a tree and the additional 1:0.01 doesn't affect the integrity of the tree - especially in this given phylogeny where the location of the root is not in dispute.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Idea 1&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;As described above the Edge data gets stored in the 1:0.01 inserted as an additional bracket at the end of the tree.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;((mammal:0.14,turtle:0.02,(rayfinfish:0.25,(frog:0.01,salamander:0.01)50:0.12)95:0.09)1:0.01);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Idea 2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is pairwise data and NXH format could insert the pairwise data for each edge (via TreeNode.add_feature()) for each 'edge' thus salamander would have 7-0 or 7,0 inserted using NHX. That is absolutely technically where the data belongs, but how this read later is more challenging.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;If I get chance&lt;/strong&gt; I'll update the question later with specific NHX output and possibly code to achieve it.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;OP upvoted. Its a good question with interesting application.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-03-28T00:47:25.197" LastActivityDate="2022-03-28T00:47:25.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18839" PostTypeId="2" ParentId="18832" CreationDate="2022-03-27T21:35:17.943" Score="1" Body="&lt;h2&gt;Update&lt;/h2&gt;&#xA;&lt;p&gt;It does indeed appear that while piRBase etc. have human piRNAs, they do not have target information.&lt;/p&gt;&#xA;&lt;p&gt;Looking at &lt;a href=&quot;https://www.biostars.org/p/235278/&quot; rel=&quot;nofollow noreferrer&quot;&gt;related questions&lt;/a&gt;, recommendations are made to map piRNA sequences to the genome to find the targets. One tool recommended is &lt;a href=&quot;https://github.com/bowhan/piPipes&quot; rel=&quot;nofollow noreferrer&quot;&gt;PiPipes&lt;/a&gt;, which might assume read input but otherwise might give some ideas.&lt;/p&gt;&#xA;&lt;h2&gt;Original answer&lt;/h2&gt;&#xA;&lt;p&gt;piRBase &lt;a href=&quot;http://bigdata.ibp.ac.cn/piRBase/target.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;has this&lt;/a&gt;. See &amp;quot;target mRNA&amp;quot; field.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2022-03-28T16:51:21.357" LastActivityDate="2022-03-28T16:51:21.357" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18840" PostTypeId="1" CreationDate="2022-03-28T02:06:07.497" Score="3" ViewCount="531" Body="&lt;p&gt;I'm working towards developing some visualization method for multiple protein sequences and subsequences using PYTHON. I want to  show the alignment of many fragment sequences to the original sequences as well as the position of fragments and the amino acids. The amino acids are to be colored by some physico-chemical properties, something similar but not exactly  as this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OEt4b.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OEt4b.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can somebody suggest some Python libraries suitable for this type of work? The output can be graphics file like png or interactive graphics or html, doesn't matter really for me&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="6649" LastActivityDate="2022-03-30T20:08:51.527" Title="Suggestion for python graphics library to plot aligned protein sequences" Tags="|python|visualization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18841" PostTypeId="1" CreationDate="2022-03-28T07:20:33.243" Score="1" ViewCount="88" Body="&lt;p&gt;I am an undergraduate novice, from CS major, intending to run a virtual-screening run using Autodock Vina. A few questions in this regard:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;How to choose the best charge model for the ligand?&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://pubs.acs.org/doi/suppl/10.1021/acs.jcim.1c00203/suppl_file/ci1c00203_si_001.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;By default&lt;/a&gt; (p. 2), Vina installs Gasteiger-Marsili (GM) partial charges prior to docking. However, GM, being a conformationally-independent approach, is a poor choice when &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4667495/&quot; rel=&quot;nofollow noreferrer&quot;&gt;compared&lt;/a&gt; to other conformationally-dependent approaches.&lt;/li&gt;&#xA;&lt;li&gt;In that case, can I choose &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4667495/&quot; rel=&quot;nofollow noreferrer&quot;&gt;B3LYP/6-311G/AIM&lt;/a&gt; which is the best-performing EEM-parametrization-set, yet published?&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What are the upsides and downsides?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How to choose the best charge model for the protein?&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Once again, Vina installs Gasteiger-Marsili (GM) partial charges prior to docking and there &lt;a href=&quot;https://jcheminf.biomedcentral.com/articles/10.1186/1758-2946-1-15&quot; rel=&quot;nofollow noreferrer&quot;&gt;exist&lt;/a&gt; better models.&lt;/li&gt;&#xA;&lt;li&gt;Is there any better open-source alternative than &lt;a href=&quot;http://openmopac.net/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MOPAC-PM7&lt;/a&gt;?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Vina removes non-polar hydrogens and lone pairs from the ligand as well as protein.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is there any upside to this operation other than &lt;a href=&quot;https://www.scripps.edu/sanner/python/adt/Tutorial/AutotorsTutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;decreasing computational complexity&lt;/a&gt;?&lt;/li&gt;&#xA;&lt;li&gt;If time etc. is not a factor, are docking outputs going to be affected to any significant degree in absence of this modification?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="15189" LastActivityDate="2022-03-28T07:20:33.243" Title="Ligand and protein preparation in Vina docking" Tags="|docking|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18842" PostTypeId="1" CreationDate="2022-03-28T09:25:18.047" Score="3" ViewCount="75" Body="&lt;p&gt;I have some doubts when i'm running LRT&lt;/p&gt;&#xA;&lt;p&gt;This is my condition data-frame small subset&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; dput(head(metadata,18))&#xA;structure(list(FAB = structure(c(1L, 1L, 1L, 1L, 6L, 2L, 6L, &#xA;4L, 3L, 5L, 7L, 3L, 2L, 7L, 6L, 2L, 6L, 2L), .Label = c(&amp;quot;C&amp;quot;, &#xA;&amp;quot;M0&amp;quot;, &amp;quot;M1&amp;quot;, &amp;quot;M2&amp;quot;, &amp;quot;M3&amp;quot;, &amp;quot;M4&amp;quot;, &amp;quot;M5&amp;quot;), class = &amp;quot;factor&amp;quot;)), row.names = c(&amp;quot;H1&amp;quot;, &#xA;&amp;quot;H2&amp;quot;, &amp;quot;H3&amp;quot;, &amp;quot;H4&amp;quot;, &amp;quot;TCGA-AB-2856&amp;quot;, &amp;quot;TCGA-AB-2849&amp;quot;, &amp;quot;TCGA-AB-2971&amp;quot;, &#xA;&amp;quot;TCGA-AB-2930&amp;quot;, &amp;quot;TCGA-AB-2891&amp;quot;, &amp;quot;TCGA-AB-2872&amp;quot;, &amp;quot;TCGA-AB-2851&amp;quot;, &#xA;&amp;quot;TCGA-AB-3011&amp;quot;, &amp;quot;TCGA-AB-2949&amp;quot;, &amp;quot;TCGA-AB-2981&amp;quot;, &amp;quot;TCGA-AB-2965&amp;quot;, &#xA;&amp;quot;TCGA-AB-2822&amp;quot;, &amp;quot;TCGA-AB-2828&amp;quot;, &amp;quot;TCGA-AB-2959&amp;quot;), class = &amp;quot;data.frame&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now I m running LRT on this.  In pairwise comparison condition I can set the reference level as such&#xA;&lt;code&gt;coldata$condition&amp;lt;- relevel(coldata$condition, ref = &amp;quot;H&amp;quot;) where  &lt;/code&gt;H` is my control set with whom I compare all other factors.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Now if I have to do the same with LRT is it correct to set the reference level?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Second doubt&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;If i have to run LRT using different reference level how to do that?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeq(dds, reduced=~condition, test=&amp;quot;LRT&amp;quot;)&#xA;dds_lrt &amp;lt;- DESeq(dds, test=&amp;quot;LRT&amp;quot;, reduced = ~ 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="285" LastActivityDate="2022-03-28T09:25:18.047" Title="LRT test in deseq2 on multiple condition" Tags="|r|deseq2|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18843" PostTypeId="2" ParentId="18833" CreationDate="2022-03-28T10:57:48.880" Score="2" Body="&lt;p&gt;I found the answer on &lt;a href=&quot;https://www.researchgate.net/post/BEAST-Real-long-branches-causing-negative-branch-lengths&quot; rel=&quot;nofollow noreferrer&quot;&gt;ResearchGate&lt;/a&gt; from Santiago Sanchez-Ramirez &amp;amp; let me quote it here:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Try using the option &lt;code&gt;-heights ca&lt;/code&gt; when you run TreeAnnotator. This stands for &amp;quot;common ancestor trees&amp;quot; and aims at summarizing clade ages across all posterior trees and not only the values for subset of trees that have that clade. If that subset of trees is low in frequency the average node height might end up being older than the direct ancestor.&#xA;&lt;a href=&quot;http://beast.community/faq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; is the explanation for negative branches:&#xA;&amp;quot;MCC trees produced by TreeAnnotator can have a descendent node that is older than its direct ancestor (a negative branch length). This may seem like an error but is actually the correct behaviour. The MCC tree is, by default, generated with average node heights across all trees in the sample which contain that clade. The negative branch lengths result when a clade is at low frequency and tends not to occur in those trees that have the MCC tree's ancestral clade (or vice versa). This means the average heights are for the adjacent nodes are derived from different sets of trees and may not have any direct ancestor-descendent relationship.&amp;quot;&#xA;In this paper you will find more information about the &lt;code&gt;ca&lt;/code&gt; option: &lt;a href=&quot;https://www.researchgate.net/publication/257460818_Looking_for_trees_in_the_forest_Summary_tree_from_posterior_samples&quot; rel=&quot;nofollow noreferrer&quot;&gt;Looking for trees in the forest: Summary tree from posterior samples&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="15161" LastActivityDate="2022-03-28T10:57:48.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18844" PostTypeId="1" AcceptedAnswerId="18860" CreationDate="2022-03-28T13:51:41.440" Score="0" ViewCount="77" Body="&lt;p&gt;Within the &lt;a href=&quot;https://en.wikipedia.org/wiki/Genome-wide_association_study&quot; rel=&quot;nofollow noreferrer&quot;&gt;GWAS wikipedia page&lt;/a&gt;, you can view the following plot:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7jbps.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7jbps.png&quot; alt=&quot;gwas&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Illustration of a simulated genotype by phenotype regression for a&#xA;single SNP. Each dot represents an individual. A GWAS of a continuous&#xA;trait essentially consists of repeating this analysis at each SNP.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is a regression plot showing on the y-axis the phenotype value for a continuous trait. Is it possible to create a similar plot with a binary trait - I can't think what this would look like, or if there is an equivalent plot I could create.&lt;/p&gt;&#xA;" OwnerUserId="10083" LastActivityDate="2022-03-31T03:47:28.903" Title="Regression plot of a continuous trait - is there a binary equivalent?" Tags="|snp|gwas|logistic-regression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18845" PostTypeId="1" AcceptedAnswerId="18847" CreationDate="2022-03-28T16:28:20.383" Score="2" ViewCount="2902" Body="&lt;p&gt;I am using Plotly dash for creating a scatter plot. The legends I am getting in the final figure are randomly placed. I want to sort labels alphabetically from A to Z with &lt;code&gt;category_orders&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fig = px.box(&#xA;        df,&#xA;        x=selected_x,&#xA;        y=selected_y,&#xA;        points='all',&#xA;        hover_data=hover_data,&#xA;        color=colour_by,&#xA;        width=800,&#xA;        height=600,&#xA;        labels={&#xA;            selected_y: &amp;quot;{} {}&amp;quot;.format(selected_y_gene, selected_y),&#xA;            selected_x: &amp;quot;{} {}&amp;quot;.format(selected_x_gene, selected_x),&#xA;        },&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4845" LastActivityDate="2022-03-28T18:21:30.470" Title="How to sort legends alphabetically in plotly dash with category_orders function" Tags="|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18846" PostTypeId="1" CreationDate="2022-03-28T16:37:48.327" Score="0" ViewCount="53" Body="&lt;p&gt;I am a beginner in this field. I would like to know which is the best way to get all the mutations from a certain sample. At this point, I am considering using GATK (&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) and Maftools (&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/maftools/inst/doc/maftools.html#1_Introduction&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). I don't know which method is better, or are there any other better tools you would recommend?&lt;/p&gt;&#xA;" OwnerUserId="14102" LastEditorUserId="3694" LastEditDate="2022-03-30T09:36:38.367" LastActivityDate="2022-03-30T09:36:38.367" Title="What is the best tools to find all the somatic mutations?" Tags="|phylogenetics|variant-calling|gatk|maftools|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18847" PostTypeId="2" ParentId="18845" CreationDate="2022-03-28T16:58:36.310" Score="2" Body="&lt;p&gt;Before you construct the figure could you try sorting the values in the dataframe in the order you require? Thus set the True/False flag according to what you want.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;orderedDF = df.sort_values(&#xA;  by=col_label, &#xA;  ascending=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;then add the following into the plot denoting the labels:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sort=False&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thus added below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fig = px.box(&#xA;    orderedDF,&#xA;    x=selected_x,&#xA;    y=selected_y,&#xA;    points='all',&#xA;    hover_data=hover_data,&#xA;    color=colour_by,&#xA;    width=800,&#xA;    height=600,&#xA;    labels={&#xA;        selected_y: &amp;quot;{} {}&amp;quot;.format(selected_y_gene, selected_y),&#xA;        selected_x: &amp;quot;{} {}&amp;quot;.format(selected_x_gene, selected_x),&#xA;# I think it should go here&#xA;        sort=False&#xA;    }&#xA;    )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The gist is you are wanting to stop the automated re-ordering and force the dataframe order onto it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-03-28T18:21:30.470" LastActivityDate="2022-03-28T18:21:30.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18848" PostTypeId="1" CreationDate="2022-03-29T00:17:11.977" Score="2" ViewCount="86" Body="&lt;p&gt;I'm comparing gene expression among 2 different datasets (&lt;em&gt;in vivo&lt;/em&gt; and &lt;em&gt;in vitro&lt;/em&gt;) I have made a heatmap showing the correlation for each and then plotted the data frame to create a scatterplot. Now I want to just pull out KP4 from each data set and do the same scatterplot but then pull put genes that aren't on the linear line. How do I do that?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;###how to compare expression across both datasets??&#xA;# Plot the correlation heatmap of intersect genes between two datasets&#xA;dat_human.df_num &amp;lt;- apply(dat_human.df,2,as.numeric)&#xA;rownames(dat_human.df_num) &amp;lt;- rownames(dat_human.df)&#xA;colnames(dat_human.df_num) &amp;lt;- c(&amp;quot;KP4_invitro&amp;quot;, &amp;quot;L3.3_invitro&amp;quot;, &amp;quot;T3M4_invitro&amp;quot;)&#xA;colnames(countmatrix_clean_cl) &amp;lt;- c(&amp;quot;KP4_invivo&amp;quot;, &amp;quot;T3M4_invivo&amp;quot;, &amp;quot;L3.3_invivo&amp;quot;)&#xA;intersect.genes &amp;lt;- intersect(rownames(countmatrix_clean_cl), rownames(dat_human.df_num))&#xA;length(intersect.genes) # 18786&#xA;#test &amp;lt;- data.frame(cbind(countmatrix_clean_cl[intersect.genes,], dat_human.df_num[intersect.genes,]))&#xA;combined_data &amp;lt;- data.frame(cbind(countmatrix_clean_cl[intersect.genes,], dat_human.df_num[intersect.genes,]))&#xA;head(combined_data)&#xA;&#xA;melted_corr_mat &amp;lt;- melt(cor(combined_data))&#xA;head(melted_corr_mat)&#xA;&#xA;ggplot(data=melted_corr_mat, aes(x=Var1, y=Var2, fill=value)) + &#xA;  geom_tile(color=&amp;quot;white&amp;quot;) + &#xA;  scale_fill_gradient2(low=&amp;quot;blue&amp;quot;, high=&amp;quot;red&amp;quot;, mid=&amp;quot;white&amp;quot;, midpoint = mean(melted_corr_mat$value), space=&amp;quot;Lab&amp;quot;) + &#xA;  geom_text(aes(Var2, Var1, label = round(value, digits=2)),&#xA;            color = &amp;quot;black&amp;quot;, size = 4) + &#xA;  theme_minimal() + &#xA;  labs(x=&amp;quot;&amp;quot;,y=&amp;quot;&amp;quot;) + &#xA;  theme(axis.text.x=element_text(angle=45, vjust=1, hjust=1)) + &#xA;  coord_fixed()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;### scatterplot for each cell line in each condition (in-vivo vs. in-vitro)&#xA;head(combined_data)&#xA;         KP4_invivo T3M4_invivo L3.3_invivo KP4_invitro L3.3_invitro T3M4_invitro&#xA;TSPAN6       24.084      12.762      24.877      31.043       69.536       30.780&#xA;TNMD          0.000       0.000       0.000       0.000        0.000        0.000&#xA;DPM1         56.015     100.402      60.142     101.408      149.830      214.116&#xA;SCYL3         3.470       5.474       8.940       6.889        9.599       10.494&#xA;C1orf112      7.910      15.051      12.932      19.363       28.695       38.230&#xA;FGR           0.042       0.776       0.128       0.010        0.392        0.361&#xA;&#xA;plot(combined_data)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0pqIP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0pqIP.png&quot; alt=&quot;in-vivo vs. in-vitro&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6827" LastActivityDate="2022-03-29T00:17:11.977" Title="Pulling out genes in a scatterplot" Tags="|r|rna-seq|ggplot2|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18849" PostTypeId="1" AcceptedAnswerId="18893" CreationDate="2022-03-29T11:32:38.613" Score="5" ViewCount="244" Body="&lt;p&gt;I have been successfully using BioPython's functionality for extracting protein torsion angles using the &lt;code&gt;IC_Chain&lt;/code&gt; module. However, I am currently looking to use the inverse functionality as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.PDB import PDBParser, internal_coords&#xA;&#xA;# Read PDB entry.&#xA;parser = PDBParser(PERMISSIVE=1, QUIET=1)&#xA;structure = parser.get_structure('', 'example.pdb')&#xA;&#xA;# Generate protein internal coordinates (angles).&#xA;ic_chain_bound = internal_coords.IC_Chain(structure)&#xA;ic_chain_bound.atom_to_internal_coordinates()&#xA;&#xA;# Generate back 3D coordinates from angles.&#xA;ic_chain_bound.internal_to_atom_coordinates()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but this fails with the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &amp;quot;test_biopython_internal_coords.py&amp;quot;, line 97, in &amp;lt;module&amp;gt;&#xA;    main(args)&#xA;  File &amp;quot;test_biopython_internal_coords.py&amp;quot;, line 89, in main&#xA;    ic_chain_bound.internal_to_atom_coordinates()&#xA;  File &amp;quot;/software/anaconda/installation/envs/alphafold/lib/python3.7/site-packages/Bio/PDB/internal_coords.py&amp;quot;, line 661, in internal_to_atom_coordinates&#xA;    verbose=verbose, start=start, fin=fin&#xA;  File &amp;quot;software/anaconda/installation/envs/alphafold/lib/python3.7/site-packages/Bio/PDB/internal_coords.py&amp;quot;, line 415, in assemble_residues&#xA;    Dict[AtomKey, numpy.array], ric.assemble(verbose=verbose)&#xA;  File &amp;quot;/software/anaconda/installation/envs/alphafold/lib/python3.7/site-packages/Bio/PDB/internal_coords.py&amp;quot;, line 1732, in assemble&#xA;    atomCoords = self.get_startpos()&#xA;  File &amp;quot;/software/anaconda/installation/envs/alphafold/lib/python3.7/site-packages/Bio/PDB/internal_coords.py&amp;quot;, line 1632, in get_startpos&#xA;    sp = self.residue.parent.internal_coord.initNCaC.get(self.rbase, None)&#xA;AttributeError: 'NoneType' object has no attribute 'initNCaC'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is this not the intended use? Could someone help me figure out why this is failing?&lt;/p&gt;&#xA;&lt;p&gt;I'm using &lt;a href=&quot;https://pastebin.com/m3P8f8rM&quot; rel=&quot;nofollow noreferrer&quot;&gt;this PDB file&lt;/a&gt; in my example, but the code previously failed for all other PDB files I tried.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="12036" LastEditorUserId="10049" LastEditDate="2023-01-24T11:50:08.353" LastActivityDate="2023-01-24T11:50:08.353" Title="Help using BioPython's IC_Chain functionality for getting protein 3D coordinates from torsion angles" Tags="|python|proteins|protein-structure|biopython|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18850" PostTypeId="1" CreationDate="2022-03-29T17:05:20.047" Score="3" ViewCount="481" Body="&lt;p&gt;&lt;em&gt;[this question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9516029/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioStars&lt;/a&gt;]&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;In relevance to my research, I've been looking for proteomics data (control vs. diabetic) and I found a dataset in the article &amp;quot;Diabetes causes marked inhibition of mitochondrial&#xA;metabolism in pancreatic β-cells&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I am actually looking for the data that was used for creating figure 1. I checked the PRIDE dataset shared in the paper (&lt;a href=&quot;https://www.ebi.ac.uk/pride/archive/projects/PXD012979&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/pride/archive/projects/PXD012979&lt;/a&gt;) and I could find the raw files (RAW file, proprietary format) but I was not sure where to look for the processed data (does the .mzid file contain precessed data) and how to generate figure 1 (added below).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rAujW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rAujW.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I checked a few workflows for processing the raw data (e.g. Progenesis and Knime- OpenMS). But the experimental conditions are missing in the PRIDE link.&#xA;I am not sure how to get started with this. Could someone please give some suggestions?&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&#xA;I understand from the answer posted below, one could use graph capture to digitize the values of relative abundance and create the heatmap. I am actually interested mainly in the enzymes involved in the glycolytic pathway Fig. 1 c,d of the paper (image shown above). Unfortunately, I couldn't find the relative abundance reported for proteins involved in this pathway.&lt;/p&gt;&#xA;" OwnerUserId="3186" LastEditorUserId="3186" LastEditDate="2022-04-03T15:17:47.437" LastActivityDate="2022-04-06T17:33:35.580" Title="Processing proteomics data" Tags="|proteins|proteomics|mass-spectrometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18851" PostTypeId="2" ParentId="17592" CreationDate="2022-03-29T19:00:39.037" Score="0" Body="&lt;p&gt;It sounds like it to me too. I have never directly accessed NCBIs stuff via Google buckets and I don't know what permissions they are setting for direct access.&lt;/p&gt;&#xA;&lt;p&gt;So ... I briefly checked it out&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gsutil ls gs://sra-pub-crun-4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and 'yes' its possible but ...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;BadRequestException: 400 Bucket is a requester pays bucket but no user project provided.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;'Requester pay' luckily I wasn't logged in :-)&lt;/p&gt;&#xA;&lt;p&gt;What the message implies given access to your (my) GCP account and they'll bill you (me). I don't think it will be much, particularly for a 'list' request, however I ain't gonna find out for a data set I don't need. I do doubt I'm going to get billed for 'list' request, however I've never paid 'list' request I prefer to keep it that way. In contrast, NCBI's front-end is free and thats what they want, because they could of course set the bucket so they pay.&lt;/p&gt;&#xA;&lt;p&gt;So in answer the question, if you wanna pay go ahead ... dial up a GCP account link it to pay.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Here's my first answer&lt;/p&gt;&#xA;&lt;p&gt;Please note I'm not really certain what information the OP seeks here, e.g. gs:// locations type stuff. &lt;em&gt;If the question is where's the SRS database on every gs:// path ...&lt;/em&gt; I don't know.&lt;/p&gt;&#xA;&lt;p&gt;I think the OP wants an API ... Google Cloud certainly use APIs by the bucket load (no pun intended but I thought it was funny). I don't use them because they are excessively verbose and I prefer SDK and gcloud tools, of which &lt;code&gt;gsutil&lt;/code&gt; is one.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;It also sounds like fastq-dump and fasterq-dump can be configured with Google Cloud credentials to download that for me.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If I have understood correctly and that is all you want then sure&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gsutil cp gs://sra-pub-crun-4/SRR1929796/SRR1929796.1 .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;will do it, if permitted (or in hindsight prepared to pay) and there will be an API version of this.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Installation via Google's &lt;code&gt;Cloud SDK&lt;/code&gt;, via  &lt;a href=&quot;https://cloud.google.com/storage/docs/gsutil_install#install&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cloud.google.com/storage/docs/gsutil_install#install&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-03-29T22:37:05.290" LastActivityDate="2022-03-29T22:37:05.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18852" PostTypeId="2" ParentId="18849" CreationDate="2022-03-29T21:11:39.903" Score="2" Body="&lt;p&gt;not able to grasp the question, tried to figure out a way to show the internal_coordinates:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.PDB import PDBParser, PICIO, PDBIO&#xA;&#xA;&#xA;# Read PDB entry.&#xA;parser = PDBParser(PERMISSIVE=1, QUIET=1)&#xA;structure = parser.get_structure('', 'example.pdb')&#xA;&#xA;print(dir(structure)) # 'atom_to_internal_coordinates' and&#xA;                      # 'internal_to_atom_coordinates' seems&#xA;                      # to be structure methods&#xA;&#xA;&#xA;# Generate protein internal coordinates (angles).&#xA;&#xA;structure.atom_to_internal_coordinates(verbose = True)&#xA;&#xA;PICIO.write_PIC(structure[0]['A'], 'internal')&#xA;&#xA;&#xA;# Generate back 3D coordinates from angles.&#xA;&#xA;&#xA;pippo = PICIO.read_PIC('internal')&#xA;&#xA;pippo.internal_to_atom_coordinates(verbose = True)&#xA;&#xA;print(pippo)&#xA;&#xA;&#xA;io = PDBIO()&#xA;&#xA;io.set_structure(pippo)&#xA;&#xA;io.save('atom_coord.pdb',  preserve_atom_numbering=True) &#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;compare &lt;code&gt;example.pdb&lt;/code&gt; to &lt;code&gt;internal&lt;/code&gt; and &lt;code&gt;atom_coord.pdb&lt;/code&gt;;&lt;/p&gt;&#xA;&lt;p&gt;have a look at &lt;code&gt;internal&lt;/code&gt;  file contents if you comment out&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;structure.atom_to_internal_coordinates(verbose = True)&lt;/code&gt; (around line 17 of&lt;/p&gt;&#xA;&lt;p&gt;code, # Generate protein internal coordinates (angles). block-code )&lt;/p&gt;&#xA;" OwnerUserId="10049" LastActivityDate="2022-03-29T21:11:39.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18854" PostTypeId="1" AcceptedAnswerId="18855" CreationDate="2022-03-30T04:08:23.640" Score="3" ViewCount="370" Body="&lt;p&gt;I am currently doing an analysis requiring a multiple-input script. I have a directory filled with 900+ samples.&lt;/p&gt;&#xA;&lt;p&gt;Each sample is comprised of two-files in a fastq format. This is an example of a sample:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MIP_REV_BC_01_01A_S1_R1_001.fastq.gz &#xA;MIP_REV_BC_01_01A_S1_R2_001.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The script has a Read1 &amp;amp; Read2 variable, and I need to make it so the R1 file is inputted for Read1 and the R2 file is inputted for Read2 for every sample. To be clear, the script is supposed to be executed on every sample separately.&lt;/p&gt;&#xA;&lt;p&gt;This was an attempt:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with open(filename) as file:&#xA;    lines=file.readlines()&#xA;    for line in lines:&#xA;        parts[]=line.split(&amp;quot;_&amp;quot;)&#xA;        if parts[7]=&amp;quot;R1&amp;quot;:&#xA;            r1[].append(line)&#xA;        else if parts[7]=&amp;quot;R2&amp;quot;:&#xA;             r2[].append(line)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We were bogged down by syntax errors, but either way, this script doesn't seem to do what we wanted it to. However, it captures the idea of what we're trying to achieve.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; I want to input a directory into the function so it automatically runs the script on the samples in the directory.&lt;/p&gt;&#xA;&lt;p&gt;Any idea what I should do?&lt;/p&gt;&#xA;" OwnerUserId="14657" LastEditorUserId="3848" LastEditDate="2022-03-30T09:12:22.913" LastActivityDate="2022-03-31T19:53:43.530" Title="Multiple file Python script" Tags="|python|fastq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18855" PostTypeId="2" ParentId="18854" CreationDate="2022-03-30T08:13:53.527" Score="2" Body="&lt;p&gt;Edited with explanation below. I don't really understand the objective, because I initially thought you were editing the content of fastaq file, but you seem to be editing a file which contains fastaq filenames. Is that correct? I know the syntax but I don't know really what the goal is.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; I'm using the pathlib which has a glob function, do consider &lt;code&gt;rglob&lt;/code&gt; under pathlib.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;import pathlib as Path&#xA;&#xA;# pathlib is super, good to habit to use it&#xA;mypath = Path('/Users/fewbugs/workingd')&#xA;allfilenames = mypath.glob(&amp;quot;*.fastaq.gz&amp;quot;)&#xA;r1, r2 = [], []&#xA;x = re.compile('_')&#xA;for filename in allfilenames:&#xA;    # the 'r' is super important &#xA;    with open(filename, 'r') as file:&#xA;    # there is no need for a readlines() &#xA;        for line in file:&#xA;    # split via re, it is better to compile (faster) &#xA;        parts=x.split(line)&#xA;    # you must use '=='. REMEMBER lists start at 0 not 1&#xA;           if parts[6]==&amp;quot;R1&amp;quot;:&#xA;    # if you want to append to a list, declare the list first &#xA;            r1.append(line)&#xA;    # there is no such thing as 'else if' REMEMBER lists start at 0 not 1&#xA;            elif parts[6]==&amp;quot;R2&amp;quot;:&#xA;                 r2.append(line)&#xA;    # when testing good idea to trap the stuff you're missing &#xA;            else:&#xA;                 print (&amp;quot;This one escaped %s&amp;quot; % (line))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Please be CAREFUL with my spacing, everything needs to be multiples of 4. I've not checked the code.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Forget this bit I thought you were processing the content of the fastaq file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;outfile1 = Path(mypath.parent, 'outfile1')&#xA;outfile2 = Path(mypath.parent, 'outfile2')&#xA;&#xA;with open (outfile1, 'w') as f:&#xA;    for item1 in r1:&#xA;        f.write(&amp;quot;%s\n&amp;quot; % item1)&#xA;with open (outfile2, 'w') as f2:&#xA;    for item2 in r2:&#xA;        f2.write(&amp;quot;%s\n&amp;quot; % item2)&#xA; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h2&gt;A simpler way?&lt;/h2&gt;&#xA;&lt;p&gt;... it looks like you're just processing filenames...&lt;/p&gt;&#xA;&lt;p&gt;Might I suggest just globing no need for all this stuff.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mypath = Path('/Users/fewbugs/workingd')&#xA;r1filenames = mypath.glob(&amp;quot;*_R1_*.fastaq.gz&amp;quot;)&#xA;r2filenames = mypath.glob(&amp;quot;*_R2_*.fastaq.gz&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If that is the aim, thats it! Just be careful about the &lt;code&gt;glob&lt;/code&gt; versus &lt;code&gt;rglob&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Idea&lt;/strong&gt; If you want two paired lists this might be better done in a nested comprehension, e.g. a counter could be inserted at the central glob (*).&lt;/p&gt;&#xA;&lt;p&gt;That may very well be what you're after. i.e.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;R1 = ['stuff_R1_001.fastaq.gz', 'stuff_R1_002.fastaq.gz' 'stuff_R1_003.fastaq.gz']&#xA;R2 = ['stuff_R2_001.fastaq.gz', 'stuff_R2_002.fastaq.gz' 'stuff_R2_003.fastaq.gz']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-03-31T19:53:43.530" LastActivityDate="2022-03-31T19:53:43.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18858" PostTypeId="2" ParentId="18840" CreationDate="2022-03-30T20:08:51.527" Score="3" Body="&lt;p&gt;You might have a look at the biotite package, and more specifically to its &lt;code&gt;biotite.sequence.graphics&lt;/code&gt; module: &lt;a href=&quot;https://www.biotite-python.org/apidoc/biotite.sequence.graphics.html#module-biotite.sequence.graphics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biotite-python.org/apidoc/biotite.sequence.graphics.html#module-biotite.sequence.graphics&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Disclaimer: I haven't used these functionalities myself, apart from using one of their colour schemes for DNA.&lt;/p&gt;&#xA;" OwnerUserId="292" LastActivityDate="2022-03-30T20:08:51.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18859" PostTypeId="1" CreationDate="2022-03-30T22:05:13.310" Score="2" ViewCount="88" Body="&lt;p&gt;I have a table of fold change shifts for two different groups. Some of them are upregulated, others are downregulated. I wonder if it is ok to subset only positive values and to boxplot them to show only upregulated genes and vice versa?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;     Gene   REDmean   GRNmean&#xA;1    gen1   4.918239  4.547266 &#xA;2    gen2   3.635476  3.447484 &#xA;3    gen3   -3.759263 -3.283889 &#xA;4    gen4   3.962326  2.818917 &#xA;5    gen5   -3.474217 -3.123105 &#xA;6    gen6   3.653160  2.907355&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6470" LastActivityDate="2022-03-30T22:05:13.310" Title="How can I separate upregulated genes from downregulated on a boxplot?" Tags="|differential-expression|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="18860" PostTypeId="2" ParentId="18844" CreationDate="2022-03-31T03:47:28.903" Score="1" Body="&lt;p&gt;I have not really seen this type of plot used when a binary outcome is being analyzed. I think in part this is due to the fact that the plot will be rather boring and difficult to interpret as there are only a limited set of possible values (4 if the response and independent variable are both binary), and without introducing some level of random noise, all of the data observations with the same X-Y value pair would overlap.&lt;/p&gt;&#xA;&lt;p&gt;The situation is a little better when the independent variable is continuous: then you can show more of a distribution. In the figure below, pulled from Figure 2 of &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/20641144/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; explaining how to visualize the discrimination abilities of a logistic regression model, the authors have plotted the risk score produced by the logistic regression model for each patient (x-axis) with the patient's actual outcome on the y-axis. Note that some random noise has been introduced to each observation's y-value so that they do not cover up each other. This plot shows that the risk score is able to somewhat separate the two outcomes.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3DGGC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3DGGC.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you are dealing with a categorical independent variable like the figure included in your question, I think it is better to visualize the proportion of the outcome for each independent variable category like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HOfGP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HOfGP.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here the outcome is whether the borrower defaulted on their loan (1 = yes, 0 = no) and the independent variable is whether they are a student (yes/no). For the y-axis, I have calculated the proportion of loan recipients in each group that defaulted. This could easily be done with GWAS data such as genotype vs whether the patient died of heart disease.&lt;/p&gt;&#xA;&lt;p&gt;I also suggest you check out &lt;a href=&quot;https://stats.stackexchange.com/questions/147721/which-is-the-best-visualization-for-contingency-tables&quot;&gt;this answer&lt;/a&gt; on how to visualize a contingency table, as the data you are describing (assuming the independent variable stays categorical) is a contingency table.&lt;/p&gt;&#xA;" OwnerUserId="6674" LastActivityDate="2022-03-31T03:47:28.903" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18862" PostTypeId="1" CreationDate="2022-03-31T08:00:22.523" Score="0" ViewCount="127" Body="&lt;p&gt;I am using minknow on ubuntu. After saving the template (settings), where is that file locally stored in the system?&#xA;Also what is its format/extension?&lt;/p&gt;&#xA;" OwnerUserId="15279" LastActivityDate="2022-03-31T08:00:22.523" Title="Locate the saved template of minknow" Tags="|ngs|sequence-analysis|linux|genome-sequencing|minknow|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18863" PostTypeId="1" AcceptedAnswerId="18865" CreationDate="2022-03-31T10:08:05.490" Score="2" ViewCount="64" Body="&lt;p&gt;I am doing an RNAseq experiment with affected (n=6) and control groups (n=6) in cow. However it turned out that 4 of my control samples have very low quality so I have to discard them.&#xA;My problem is that there is no way to repeat the sampling and sequencing.&#xA;Are there any solutions to perform DE,GSEA with these conditions (somehow generating bootstrapped samples from control groups or something similar)?&#xA;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="12203" LastActivityDate="2022-03-31T13:23:42.650" Title="RNAseq small sample size in control group" Tags="|rna-seq|differential-expression|gsea|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18865" PostTypeId="2" ParentId="18863" CreationDate="2022-03-31T13:23:42.650" Score="3" Body="&lt;p&gt;There is no obstacle in performing a DE analysis and all downstream on a 6 vs 2 comparison. The only thing to keep in mind is that the representation of the control is not particularily good, but you can technically perform the exact same analysis as if it was 6 vs 6, e.g. with DESeq2, edgeR or limma-voom/trend.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2022-03-31T13:23:42.650" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18866" PostTypeId="1" AcceptedAnswerId="18883" CreationDate="2022-03-31T20:10:42.297" Score="2" ViewCount="786" Body="&lt;p&gt;I'm following the instructions for integration:&#xA;&lt;a href=&quot;https://satijalab.org/seurat/articles/integration_introduction.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/articles/integration_introduction.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And it's taking a while to run:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;immune.anchors &amp;lt;- FindIntegrationAnchors(object.list = ifnb.list, normalization.method = &amp;quot;SCT&amp;quot;,&#xA;    anchor.features = features)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm integrating 6 datasets.&lt;/p&gt;&#xA;&lt;p&gt;Is this function designed well for large amount of data?&lt;/p&gt;&#xA;" OwnerUserId="6827" LastActivityDate="2022-04-05T17:32:56.390" Title="Seurat Integration" Tags="|seurat|single-cell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18867" PostTypeId="2" ParentId="18775" CreationDate="2022-04-01T23:05:45.727" Score="1" Body="&lt;p&gt;In addition to swbarnes2 comment, I'd suggest a more systematic approach.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ccb.jhu.edu/software/kraken2/index.shtml?t=manual&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kraken2&lt;/a&gt; is a tool for metagenomic analysis of reads, though it can be hard to set up.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1841-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;mash screen&lt;/a&gt; is another method for metagenomic analysis, much easier to set up but noisier.&lt;/p&gt;&#xA;&lt;p&gt;If you are dealing with contamination, treating your data as metagenomic might help.&lt;/p&gt;&#xA;&lt;p&gt;Also, as always, ensure that your workflow and reference assembly are correct!!!&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-04-01T23:05:45.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18868" PostTypeId="2" ParentId="18596" CreationDate="2022-04-02T15:21:39.470" Score="2" Body="&lt;p&gt;Not entirely clear on what you're asking but the beta estimates in LDSC come from GWAS summary statistics. As I understand it, they aren't &amp;quot;derived&amp;quot; so much as just set equivalent to what they linear model defines them as, which is their additive effects on the phenotype as implied by the model. So for each SNP j, it's effect size (beta) is whatever difference one copy of the allele makes on the phenotype (phi), divided by N, the number of SNPs.&lt;/p&gt;&#xA;" OwnerUserId="15286" LastActivityDate="2022-04-02T15:21:39.470" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18869" PostTypeId="1" AcceptedAnswerId="18870" CreationDate="2022-04-02T18:29:01.267" Score="2" ViewCount="465" Body="&lt;p&gt;I want to use data from the following dataset:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE84371&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE84371&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The data has been TPM normalized, which is not ideal for clustering but I have to work with what I have. TPM is a bad normalization method and it should not be used for these analyses because its laden with a lot of assumptions.&lt;/p&gt;&#xA;&lt;p&gt;Presumably it has already been scaled, &lt;em&gt;&lt;strong&gt;can I simply skip these steps and then run FindVariableFeatures() followed by the rest of the pipeline from the clustering tutorial or do the methods used assume Seurat specific preprocessing and using TPM will screw them up?&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately in this case TPM is not reversable.  &lt;a href=&quot;https://btep.ccr.cancer.gov/question/faq/what-is-the-difference-between-rpkm-fpkm-and-tpm/#:%7E:text=Here%27s%20how%20you%20calculate%20TPM,divide%20this%20number%20by%201%2C000%2C000&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://btep.ccr.cancer.gov/question/faq/what-is-the-difference-between-rpkm-fpkm-and-tpm/#:~:text=Here's%20how%20you%20calculate%20TPM,divide%20this%20number%20by%201%2C000%2C000&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;TPM is calculated as a function of the kilobases of the gene in the reference genome they used and the reads per gene across all samples (aka all cells).  They have not specified which reference genome was used in the main article, and they do not supply the read counts in the metadata or the main file.  So in this case the TPM is irreversible.&lt;/p&gt;&#xA;&lt;p&gt;I think I can get raw reads from SRA and do alignment myself. It's annoying that raw reads were not provided but as someone who's lab entirely does computational analysis on open datasets I can tell you that this is not the most egregious thing I've ever seen. As I said before it would be easier for me to get the fastq files and do my own alignment which is still a ton of work the main authors could have prevented by simply posting the read counts.&lt;/p&gt;&#xA;&lt;p&gt;Clustering Tutorial:&#xA;&lt;a href=&quot;https://satijalab.org/seurat/articles/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/articles/pbmc3k_tutorial.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;TPM means transcripts per million. It is obtained by dividing the read counts by the length of each gene in kilobases, resulting in reads per kilobase (RPK). All the RPK values in a sample are counted and then divided by 1,000,000.&lt;/p&gt;&#xA;" OwnerUserId="2482" LastEditorUserId="73" LastEditDate="2022-05-06T01:25:23.737" LastActivityDate="2022-05-06T01:38:06.420" Title="How will Seurat handle pre-normalized and pre-scaled data?" Tags="|rna-seq|scrnaseq|seurat|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18870" PostTypeId="2" ParentId="18869" CreationDate="2022-04-02T20:06:52.650" Score="1" Body="&lt;p&gt;I don't do transcriptome analysis, it ain't my thing, however I do understand statistical analysis as well as the underlying issue regarding the public availability of molecular data ... I agree with the OP its not ideal. However, yes the OP can continue with 'clustering', personally I definitely prefer it if the transformation was removed, or analysed from the raw data.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; The authors of the data set are obliged to provide the raw data, presuming its published. Why? This is because whilst the OP can perform the analysis - it is very likely the OP will be asked to repeat the analysis on the untransformed data (again note this ain't my area, so 'very likely' might be 'possibly' in this field). At this point the original authors have to declare their untransformed data, if this was a formal request to the OP's analysis.&lt;/p&gt;&#xA;&lt;p&gt;Please read the caveats and discussion.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Firstly could the OP reverse the normalisation? Its just a transformation applied to all the data which in their analysis resulted in a normal distribution, I don't know TPM but if this was log-transform you would supply 'un-log' it, square-root then square it, reciprocal do another reciprocal will 'de-transformation'. Can TPM be undone? I agree it ain't ideal, from several perspectives but it gets things moving quickly.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Ask the authors for the raw data - this is what I would do. They are obliged to provide it if the data is published. Virtually all journals have this policy in place, however I agree that ASM remain an outlier in this respect (who are significant publisher). Any other journal will return to the authors and point out their 'check-boxes' when they submitted the manuscript. It is etiquette to go to the authors first and provide sufficient chance for them to respond, prior step 3 (below) - giving them the absolute maximum benefit of the doubt.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What you do if the authors don't respond? The OP would then contact the journal explaining: a) why you need the data, i.e. the analysis performed; b) explain the analytical significance of the raw data, i.e. arguments the OP has raise 'not ideal' and general concerns raised here. I do have experience in this regard and the authors will oblige if asked by the journal.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Searaut's cluster analysis certainly used tSNE ... I believe it uses dimensionality reduction such principle component analysis (PCA) - I don't know which they use - and then tSNE analysis (which I think you are calling 'clustering*'). This is a very robust 'clustering'* (please see footnote), there is no question this is as good as it gets. If that is singly your goal I agree that dimensionality reduction calculations don't need transformations, transformations are needed for parametric statistics and other areas of model fitting.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;The problems with transformation&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I agree the authors should not have done this because:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Transformations are not perfect, if its wrong - you're stuck.&lt;/li&gt;&#xA;&lt;li&gt;Not only do you not need to transform the data, its better you don't - if untransformed data results in nice 'clusters' thats the preferred result. Dimensionality reduction + tSNE gives great results thats well known.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Corroborating the OP's concern&lt;/strong&gt;&#xA;Dimensionality reduction calculations do not need a normal distribution because its sectioning up the observable variance to maximise the classification of the data. My opinion - in statical analysis - is if you need to transform, THEN perform dimensionality reduction (PCA-stuff) THEN tSNE, that data has been heavily manipulated. It doesn't mean to say the conclusions are not valid but its been put through a lot and would STRONGLY SUGGEST THE RAW DATA HAS NOT GIVEN GOOD RESULTS. Thus, personally I would be cautious, why should be OP be put under unnecessary caution?&lt;/p&gt;&#xA;&lt;p&gt;It is important to note this is not my field and the field will have its own standards. If transformation is routine for all statical calculations (except non-parametric where it may not be valid) then authors of the data will argue they are correct and no further data needs to be released. Personally, I really doubt it, but we'll see what others say.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Footnote&lt;/strong&gt; * clustering is a statistical method usually resulting in a tree, it is generally avoided because there are so many different clustering methods giving different results and no agreement on which is the 'optimal' method. I think when the OP say 'clustering' they refer to clusters following a tSNE analysis, which will be a bivariate plot.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;From the comments&lt;/strong&gt; Failure to declare a reference genome is a formal omission that has to be rectified and the journal is obliged to do this. Thus the OP would simply contact the journal, they may tell the OP to contact the authors and when the authors respond the response is passed back to the journal. If its an online journal you can simply leave a comment in the 'comments' section: the authors will answer. In this instance officiating via the journal is preferred to ensure the information is verified by all parties - I presume the OP does not know the authors. From the comments I get the gist the OP is working via supervision, and would therefore just notify  their supervisor who should officiate on OP's behalf (they might then delegate it back to the OP - but at least the supervisor is formally aware of this).&lt;/p&gt;&#xA;&lt;p&gt;In context, stuff happens and there is formal process to ensure parity is achieved its simply patience in working through it.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Down vote&lt;/strong&gt; In response to the downvote ... unpopularity happens in data set transparency its part of the process ... I was (at least in part) attributed to changing journal group policy on data availability, stated from within the journal. Maybe they joking - but they certainly all knew about what was going on. The specific issue was a prolonged dialogue over access to a specific data set. It does demonstrate that most journals are concerned if an author does not cooperate with data transparency, in fact the journal in question explicitly said this and they were not joking.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-04-03T16:02:52.323" LastActivityDate="2022-04-03T16:02:52.323" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18871" PostTypeId="1" CreationDate="2022-04-03T03:26:35.043" Score="3" ViewCount="32" Body="&lt;p&gt;I'm looking at DNA methylation (DNAm) data such as TCGA (e.g., BRCA, KIRP, KIRC, etc.). Currently trying to build use my model to predict DNAm age on test sets, but many of the data sets  are missing key probe values used in my clock/model.&#xA;I have inspected approximately 30 GEO data series and the following TCGA cancer samples: KIRP, KIRC, LUAD, LUSC, BRCA, THCA. They are each missing key probes I am using.&#xA;Is there a common way to impute the missing values in R?&lt;/p&gt;&#xA;" OwnerUserId="15288" LastActivityDate="2022-04-03T03:26:35.043" Title="Is there a generally accepted method used to impute missing DNA methylation data (probe beta-values)?" Tags="|r|illumina|methylation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18872" PostTypeId="2" ParentId="18850" CreationDate="2022-04-03T13:33:38.377" Score="1" Body="&lt;p&gt;You asked two questions&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is it possible to find from the dataset (i.e does the raw file contain information on the sample whether it is a diabetic sample or a control sample?).&lt;/li&gt;&#xA;&lt;li&gt;Should the diabetic and control dataset be normalized separately?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Before I continue further I'd just like to point out the graph in the Supplementary information of the Nature research article of the paper in question. This graphic has not been edited in anyway.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/7a8p1.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7a8p1.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I will not comment further but it would be very unusual for an analyst to produce a y-axis as done here. Just to labour/labor the point this is the appendix of a Nature research article (not 'even' a Nature letter).&lt;/p&gt;&#xA;&lt;p&gt;The answer to the first question is you can't and will need to contact the corresponding author. There is a possible work around. The reason you need to contact the authors is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The authors have not labeled the x-axis in any heatmap figure. This is not standard procedure and in this analysis you cannot link the heatmap values for that given mouse to the data base - because the authors have not declared which mouse is which. In addition, one cannot link the RNA-seq analysis to the proteomics data and they are paired samples, because only 3 diabetic mice are presented for the RNA-seq data but 4 mice for the proteomics data, therefore again it is not possible to link the two data sets (I understand this last point is not the precise question, but is important in the 'conclusion' below).&lt;/li&gt;&#xA;&lt;li&gt;The link you provided had 12 samples - not 8 as in the paper.  It is very important to understand why there is a different number of samples (where one 'sample' is one mouse) in the proteomics data base compared with the apparent lower number samples presented in the paper.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;To reiterate, only the authorship knows this information therefore they need to be contacted. The corresponding author is different from the data base curator who has been previously contacted (with no response). The corresponding author has to uphold the terms of publication as described at the end of the paper.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The workaround is to 'reverse engineer' the heatmap to its original values and is described in this article &lt;a href=&quot;https://www.r-bloggers.com/2020/01/how-to-reverse-engineer-a-heat-map-into-its-underlying-values/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.r-bloggers.com/2020/01/how-to-reverse-engineer-a-heat-map-into-its-underlying-values/&lt;/a&gt; . This is only for R.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, you still cannot link these values to the publicly available data sets, therefore until x-axis sampling is declared it is not a singular solution.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Normalisation, I would like to point out in the appendix the authors transformed the RNA-seq data (log2) but did not do this for the proteomics data. For example aquaporins were upregulated 94 fold for proteomics data, but 4.61 fold using log2 RNA-seq data. The reason they did this is not clear and understanding this is quite important to the analysis. Why present an untransformed ratio in one analysis, but for the same experiment transform the values for precisely paired data?&lt;/p&gt;&#xA;&lt;p&gt;The Appendix describes this information in the Supplementary Information in the following table.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/o9c12.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/o9c12.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&#xA;This is a Nature paper and the authors are therefore under obligation, as per terms of publishing, to firstly supply all information on request and any information present in the published article must be explained in the Supplement Information. Those terms are stated at the end of the paper. It is clear the absence of labelling the x-axis in the heatmap in anyway (even if it is done in the legend), or the inability to link pairwise RNA-seq to proteomics data (which is very important), or the discrepancy in the samples on the database verses the samples in the paper, is not explained in Supplementary Information (Appendix).&lt;/p&gt;&#xA;&lt;p&gt;Under these technical circumstances the authors are obliged to supply the missing information on request. You may require a chain of request escalation within your organisation because they may not respond to the initial requests, but you don't know unless you try. However, at some level in the organisation, e.g. Director, the terms of publication would be upheld by authorship in question.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;First answer&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;But the experimental conditions are missing in the PRIDE link&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The experimental conditions of the heatmap are as follows: each box is a separate animal and the heat map colors (colours) are a log2 representation of Figure A (below). In other words to understand the heat map using Figure A is the starting point. You can obtain the values from this analysis easily using a graph capture package using Figure A (not Figure B). Its a long while since I've used graph capture, so I forget the packages. If you feel there are anomalies you can then contact the authors for the precise values.&lt;/p&gt;&#xA;&lt;p&gt;log2 is a bit weird, natural log would have been preferred:&lt;/p&gt;&#xA;&lt;p&gt;FIGURE A&#xA;&amp;quot;Abundance of the indicated proteins involved in lipid synthesis, measured by proteomics, in islets isolated from control (black, Ctrl, n=4) and 2-week diabetic βV59M (white, Diab, n=4) mice. Each data point indicates a separate mouse. Mean±S.E.M. **p&amp;lt;0.01, ***p&amp;lt;0.001. HMGCS1, 3-Hydroxy-3-methylglutaryl-CoA synthase 1; HMGCR, Hydroxy-3-methylglutaryl- CoA reductase; DHCR7, 7-dehydrocholesterol reductase; ACLY, ATP citrate lyase; GPD1, glycerol phosphate dehydrogenase; ACC1, acetyl-CoA carboxylase 1; AACS, acetoacetyl- CoA synthetase; FASN, Fatty acid synthase.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;FIGURE B&#xA;&amp;quot;Heat maps of mRNA and protein expression of the indicated lipid metabolism genes in islets isolated from control and 2-week diabetic bV59M mice. Each box corresponds to a different animal. Colour indicates log2 fold-change.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://static-content.springer.com/esm/art%3A10.1038%2Fs41467-019-10189-x/MediaObjects/41467_2019_10189_MOESM1_ESM.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://static-content.springer.com/esm/art%3A10.1038%2Fs41467-019-10189-x/MediaObjects/41467_2019_10189_MOESM1_ESM.pdf&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ejXRL.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ejXRL.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Data availability&#xA;RNA data&#xA;&lt;a href=&quot;https://www.ebi.ac.uk/ena/browser/view/PRJEB31793?show=reads&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/ena/browser/view/PRJEB31793?show=reads&lt;/a&gt;&#xA;Checks out ok.&lt;/p&gt;&#xA;&lt;p&gt;Proteomics&#xA;I checked one link only ...&#xA;&lt;a href=&quot;http://proteomecentral.proteomexchange.org/cgi/GetDataset?ID=PXD012979&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://proteomecentral.proteomexchange.org/cgi/GetDataset?ID=PXD012979&lt;/a&gt;&#xA;I agree something doesn't look right here just in the way it is reported, because the paper should highly polished and they did miss the link out. This could be word limit issues (Data availability might be included). The curator is listed as roman.fischer@ndm.ox.ac.uk and that would be the person to contact.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-04-06T17:33:35.580" LastActivityDate="2022-04-06T17:33:35.580" CommentCount="14" ContentLicense="CC BY-SA 4.0" />
  <row Id="18873" PostTypeId="1" CreationDate="2022-04-03T16:24:24.910" Score="4" ViewCount="144" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9517437/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have multiple vcf of CASES and CONTROLS variations annotated by VEP, SNPEff and SnpSift.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;First pair vcf -&amp;gt; only variations| CASES and CONTROLS&lt;/li&gt;&#xA;&lt;li&gt;Second pair vcf -&amp;gt; variations + SnpEff | CASES and CONTROLS&lt;/li&gt;&#xA;&lt;li&gt;Third pair vcf-&amp;gt; variations + SnpEff+VEP+SnpSIFT| CASES and CONTROLS | at so on&#xA;The data has non-overlapping samples. I tried to merge corresponding vcf of CASE and CONTROLS into one file with bcftools merge.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The code is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; bgzip -c file.vcf &amp;gt; file.vcf.gz  &#xA; tabix -p vcf file.vcf.gz&#xA; bcftools merge -o merged.vcf.gz -Oz file.vcf.gz file_2.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Unfortunatelly in files with SnpSift annotations i got this error:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error at chr1:20724637: wrong number of fields in dbNSFP_ada_score?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Please anybody knows how to solved this error or what does it mean?&lt;/p&gt;&#xA;&lt;p&gt;grep header:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;INFO=&amp;lt;ID=dbNSFP_ada_score,Number=A,Type=String,Description=&amp;quot;Field 'ada_score' from dbNSFP&amp;quot;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;-G on chr1:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CHROM POS ID REF ALT QUAL FILTER INFO&#xA;chr1 20724637 rs1280663306;rs1445195248 TGGGAGGGAGGGAGAGAGGT TGGGAGGGAGAGAGGT,TGGGAGGGAGGGAGGGAGGT,TGGGAGGGAGGGAGAGAGGG,TGGGGGGGAGGGAGAGAGGT,TGGGTGGGAGGGAGAGAGGT,TGGGAGAGAGGGAGAGAGGT 2.42927 . ANN=TGGGGGGGAGGGAGAGAGGT|splice_region_variant&amp;amp;intron_variant|LOW|SH2D5|SH2D5...SNV|1||||||||||||||||||||||||||||||||||||||||||||;dbNSFP_ada_score=1.43020658327008E-4,0.00157457299236331,5.16496450143413E-5;dbNSFP_rf_score=0.016,0.054,0.0;dbNSFP_PHRED=5.632,0.077,6.365,7.459,2.716&#xA;chr1 20724637 rs1280663306;rs1445195248 TGGGAGGGAGGGAGAGAGGT TGGGAGGGAGAGAGGT,TGGGAGGGAGGGAGGGAGGT,TGGGAGGGAGGGAGAGAGGG,TGGGGGGGAGGGAGAGAGGT,TGGGTGGGAGGGAGAGAGGT,TGGGAGAGAGGGAGAGAGGT 2.42927 . ANN=TGGGGGGGAGGGAGAGAGGT|splice_region_variant&amp;amp;intron_variant|LOW|SH2D5|SH2D5|transcript|NM_001103161.2|protein_coding|5/9|...||||||||||||||||||||||||||;dbNSFP_ada_score=1.43020658327008E-4,0.00157457299236331,5.16496450143413E-5;dbNSFP_rf_score=0.016,0.054,0.0;dbNSFP_PHRED=5.632,0.077,6.365,7.459,2.716&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;dbNSFP_ada_score&lt;/code&gt; is an ensemble prediction score, as calculated by &lt;a href=&quot;https://dx.doi.org/10.1002%2Fhumu.22932&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbNSFP&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15290" LastEditorUserId="73" LastEditDate="2022-04-05T00:29:40.623" LastActivityDate="2022-04-05T00:29:40.623" Title="Error occurrence after merging files with bcftools: wrong number of fields?" Tags="|vcf|bcftools|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18875" PostTypeId="1" CreationDate="2022-04-04T09:23:23.317" Score="1" ViewCount="224" Body="&lt;p&gt;I am trying to do a synteny analysis using MCScanX. As per requirement I have done all by all blast between two sets of protein from two closely related species. And also prepared the gff file. The file formats are as follows:&lt;/p&gt;&#xA;&lt;p&gt;cyp_dan.blast&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;NP_001347641.1  XP_042591522.1  78.31   166     32      1       1       166     1       162     5e-91   267&#xA;NP_001347641.1  XP_042577882.1  60.24   166     62      2       1       166     1       162     4e-63   196&#xA;.&#xA;.&#xA;XP_704272.2     XP_018953536.1  31.03   87      59      1       275     360     90      176     8e-07   50.4&#xA;XP_704272.2     NP_001005958.2  33.33   63      38      1       273     331     38      100     1e-06   48.9&#xA;XP_704272.2     XP_018969808.1  31.33   83      56      1       275     356     91      173     2e-06   49.3```&#xA;&#xA;cyp_dan.gff&#xA;NC_056572.1     gene-LOC1221465.1       2966    7093&#xA;NC_056572.1     id-LOC122146575.1       2966    3166&#xA;NC_056572.1     id-LOC122146575.1       3246    3357&#xA;.&#xA;.&#xA;NC_002333.2     gene-trnP;Dbxre.1       16527   16596&#xA;NC_002333.2     rna-trnP;Parent.1       16527   16596&#xA;NC_002333.2     exon-trnP-1;Par.1       16527   16596&#xA;&#xA;cmdline: /home/software/MCScanX/MCScanX cyp_dan&#xA;Output:&#xA;############### Parameters ###############&#xA;# MATCH_SCORE: 50&#xA;# MATCH_SIZE: 5&#xA;# GAP_PENALTY: -1&#xA;# OVERLAP_WINDOW: 5&#xA;# E_VALUE: 1e-05&#xA;# MAX GAPS: 25&#xA;############### Statistics ###############&#xA;# Number of collinear genes: 0, Percentage: 0.00&#xA;# Number of all genes: 108556&#xA;##########################################&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here, both of the species are from same family since there is no chance to have &amp;quot;zero collinear gene&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="12579" LastEditorUserId="3694" LastEditDate="2022-04-04T09:43:33.067" LastActivityDate="2022-04-04T13:40:02.740" Title="Number of collinear genes are zero while running MCScanX" Tags="|blastp|gff|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18876" PostTypeId="1" CreationDate="2022-04-04T10:13:45.640" Score="2" ViewCount="94" Body="&lt;p&gt;I've read about different proteomics tools that can do de novo assembly of MS/MS Ion Trap proteomics signatures for de novo antibody sequencing. Given the way CDR3s of antibodies are created, together with Somatic Hypermutation, can generate a huge diversity in the CDR regions of the heavy and light chains, the reference-based methods can fall short of determining the full sequence of an unknown antibody without a set of reference sequences including those with its CDRs.&lt;/p&gt;&#xA;&lt;p&gt;Are there any other tools that can determine antibody sequences from MS/MS proteomics raw signals?&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="180" LastEditDate="2022-04-08T13:43:25.183" LastActivityDate="2022-04-11T08:40:22.147" Title="De novo antibody sequencing fromMS/MS Ion Trap proteomics raw signal" Tags="|machine-learning|proteomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18877" PostTypeId="2" ParentId="18876" CreationDate="2022-04-04T10:19:26.350" Score="2" Body="&lt;p&gt;A few notes on data generation first:&lt;/p&gt;&#xA;&lt;p&gt;There are currently two broad approaches for the generation of bottom-up or &amp;quot;shotgun&amp;quot; MS proteomics: data-dependent acquisition (DDA) and data-independent acquisition (DIA).1 In tandem MS (MS/MS), the DDA approach only puts forward certain peptides generated during the first cycle of MS for fragmentation during the second cycle, while with the DIA approach, all peptides generated during the first MS cycle can be fragmented in the second round.&lt;/p&gt;&#xA;&lt;p&gt;The recently-introduced Orbitrap Fusion mass spectrometry permits various types of MS2 acquisition methods. There are four main MS2 strategies:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;HCD-OT (higher-energy-collisional-dissociation with Orbitrap detection),&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;HCD-IT (HCD with ion trap, IT),&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;CID-IT (collision-induced-dissociation with IT) and&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;CID-OT on Orbitrap Fusion&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Reading from the biorxiv preprint of Stitch:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2022.03.07.483237v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/10.1101/2022.03.07.483237v1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;With the aid of specialized software packages like DiPS, Supernovo, or ALPS, full and accurate sequences of the heavy and light chains can be reconstructed with the MS/MS spectra of the digested peptides.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Before running Stitch or similar tools, a peptide profiling tool needs to be executed on the .raw or .mgf data alone.&lt;/p&gt;&#xA;&lt;p&gt;STITCH: &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2022.03.07.483237v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/10.1101/2022.03.07.483237v2&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This currently requires the output of analysing .raw data via PEAKS, which is not free unfortunately. Authors currently looking into using other free de novo sequencing software for input into Stitch. The software also allows for FASTA format, which can then be executed on the output of PepNovo, Novor or similar.&lt;/p&gt;&#xA;&lt;p&gt;DiPS: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5461544/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5461544/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;ALPS: &lt;a href=&quot;https://www.nature.com/articles/srep31730&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/srep31730&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;The results show that ALPS was able to assemble three complete monoclonal antibody sequences of length 216–441 AA, at 100% coverage and 96.64–100% accuracy.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;DeepNovo: &lt;a href=&quot;https://www.pnas.org/doi/10.1073/pnas.1705691114&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.pnas.org/doi/10.1073/pnas.1705691114&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;This field has been actively studied over the past 20 y, and many de novo sequencing tools have been proposed, such as PepNovo, PEAKS, NovoHMM, MSNovo, pNovo, UniNovo, and Novor among others (6–19). The recent &amp;quot;gold rush&amp;quot; into mAbs has undoubtedly elevated the application of de novo sequencing to a new horizon (20–23)&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;DeepNovo considerably outperformed state of the art methods, achieving 7.7–22.9% higher accuracy at the amino acid level and 38.1–64.0% higher accuracy at the peptide level&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Available for download, requires activation via sending email to author, which then points to the company behind PEAKS, which will give you a 15 day evaluation license.&lt;/p&gt;&#xA;&lt;p&gt;NovoHMM: the download page leads to a missing URL&lt;/p&gt;&#xA;&lt;p&gt;DeepResolution2: &lt;a href=&quot;https://www.sciencedirect.com/science/article/abs/pii/S0039914022002119&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sciencedirect.com/science/article/abs/pii/S0039914022002119&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Capable of analyzing GC-MS data rather than MS/MS Ion Trap&lt;/p&gt;&#xA;&lt;p&gt;DePS: &lt;a href=&quot;https://arxiv.org/abs/2203.08820&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/2203.08820&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;DeepNovoV2: &lt;a href=&quot;https://github.com/volpato30/DeepNovoV2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/volpato30/DeepNovoV2&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;DPST: &lt;a href=&quot;https://arxiv.org/abs/2203.13132&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/2203.13132&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Link does not work &lt;a href=&quot;https://github.com/Yan98/DPST&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Yan98/DPST&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;PepNovo: &lt;a href=&quot;http://proteomics.ucsd.edu/software-tools/531-2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://proteomics.ucsd.edu/software-tools/531-2/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The new version is from 2012, includes models for CID and LTQ, but not HCD models.&lt;/p&gt;&#xA;&lt;p&gt;pNovo: &lt;a href=&quot;http://pfind.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://pfind.org/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Includes models for CID, HCD, ETD and HCD+ETD.&lt;/p&gt;&#xA;&lt;p&gt;Novor: &lt;a href=&quot;https://www.rapidnovor.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rapidnovor.com/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Filling in a form, someone will contact you. Doesn't seem to be free.&lt;/p&gt;&#xA;&lt;p&gt;METHODOLOGY used so far:&lt;/p&gt;&#xA;&lt;p&gt;1 - ThermoRawFileParser: Convert .raw data to .gcf&lt;/p&gt;&#xA;&lt;p&gt;2 - PepNovo: produce list of peptides, one-liner to convert to .fasta&lt;/p&gt;&#xA;&lt;p&gt;3 - Stitch: analyse given a list of V(D)J+C references from templates and the .fasta file from above.&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="180" LastEditDate="2022-04-11T08:40:22.147" LastActivityDate="2022-04-11T08:40:22.147" CommentCount="0" CommunityOwnedDate="2022-04-08T07:46:25.143" ContentLicense="CC BY-SA 4.0" />
  <row Id="18878" PostTypeId="2" ParentId="18875" CreationDate="2022-04-04T13:40:02.740" Score="1" Body="&lt;p&gt;I'm not familiar with the MCScanX tool, but I see two potential issues that are worth investigating.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The &lt;code&gt;cyp_dan.gff&lt;/code&gt; file does not look like GFF or any of its common variants (GFF3, GTF, etc.). Again, I'm unfamiliar with MCScanX, so I don't know precisely how its wants the input data formatted. But if it wants a GFF file, I would double-check to make sure that what you have matches what the tools expects/requires.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The gene/protein IDs in the &lt;code&gt;cyp_dan.gff&lt;/code&gt; file do not seem to match the query or subject IDs in the &lt;code&gt;cyp_dan.blast&lt;/code&gt; file. No tool is going to be able to find collinear adjacent genes if the IDs in the BLAST results don't match the IDs in the gene annotations.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="96" LastActivityDate="2022-04-04T13:40:02.740" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18881" PostTypeId="1" CreationDate="2022-04-05T14:45:10.837" Score="3" ViewCount="959" Body="&lt;p&gt;I have a large fasta file with RNA sequences. I need to add a &lt;code&gt;;&lt;/code&gt; at the end of each header line (lines beginning with &lt;code&gt;&amp;gt;&lt;/code&gt;) in the file.&lt;/p&gt;&#xA;&lt;p&gt;For example, I have:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Bacteria;Cyanobacteriales;Phormidiaceae;Lyngbya PCC-7419;Lyngbya sp. PCC 8106&#xA;GGAGAGUUUGAUCCUGGCUCAGGAUGAACGCUGGCGGUCUGCUUAACACAUGCAAGUCGAACGGACUCUUCGGAGUUAGU&#xA;&amp;gt;Bacteria;Cyanobacteriales;Nostocaceae;Nodularia PCC-9350;Nodularia spumigena CCY9414&#xA;GAGAGUUUGAUCCUGGCUCAGGAUGAACGCUGGCGGUAUGCUUAACACAUGCAAGUCGAACGGUCUCUUCGGAGAUAGUG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And, my desired output is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Bacteria;Cyanobacteriales;Phormidiaceae;Lyngbya PCC-7419;Lyngbya sp. PCC 8106;&#xA;GGAGAGUUUGAUCCUGGCUCAGGAUGAACGCUGGCGGUCUGCUUAACACAUGCAAGUCGAACGGACUCUUCGGAGUUAGU&#xA;&amp;gt;Bacteria;Cyanobacteriales;Nostocaceae;Nodularia PCC-9350;Nodularia spumigena CCY9414;&#xA;GAGAGUUUGAUCCUGGCUCAGGAUGAACGCUGGCGGUAUGCUUAACACAUGCAAGUCGAACGGUCUCUUCGGAGAUAGUG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15297" LastEditorUserId="73" LastEditDate="2023-07-09T20:28:47.873" LastActivityDate="2023-07-10T16:39:27.927" Title="How can I add a &quot;;&quot; at the end of each fasta header line in a file?" Tags="|phylogenetics|fasta|phylogeny|bash|bacteria|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18882" PostTypeId="2" ParentId="18881" CreationDate="2022-04-05T17:00:46.050" Score="6" Body="&lt;p&gt;... using the legendary Perl pie&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -p -i -e 's/^(&amp;gt;.*)/$1;/' mybacteria.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Input&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Bacteria;Cyanobacteriales;Phormidiaceae;Lyngbya PCC-7419;Lyngbya sp. PCC 8106&#xA;GGAGAGUUUGAUCCUGGCUCAGGAUGAACGCUGGCGGUCUGCUUAACACAUGCAAGUCGAACGGACUCUUCGGAGUUAGU&#xA;&amp;gt;Bacteria;Cyanobacteriales;Nostocaceae;Nodularia PCC-9350;Nodularia spumigena CCY9414&#xA;GAGAGUUUGAUCCUGGCUCAGGAUGAACGCUGGCGGUAUGCUUAACACAUGCAAGUCGAACGGUCUCUUCGGAGAUAGUG&#xA;A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Bacteria;Cyanobacteriales;Phormidiaceae;Lyngbya PCC-7419;Lyngbya sp. PCC 8106;&#xA;GGAGAGUUUGAUCCUGGCUCAGGAUGAACGCUGGCGGUCUGCUUAACACAUGCAAGUCGAACGGACUCUUCGGAGUUAGU&#xA;&amp;gt;Bacteria;Cyanobacteriales;Nostocaceae;Nodularia PCC-9350;Nodularia spumigena CCY9414;&#xA;GAGAGUUUGAUCCUGGCUCAGGAUGAACGCUGGCGGUAUGCUUAACACAUGCAAGUCGAACGGUCUCUUCGGAGAUAGUG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Details&lt;/strong&gt; I copy pasted your first code into a file &lt;code&gt;mybacteria.fa&lt;/code&gt; then ran the Perl pie script. Perl pie is very quick, there alternative regexs (regular expressions) that would be slightly more efficient, but Perl pie is fast regardless. In fact you don't need to use regex in this example and would be faster via a &lt;code&gt;if $line == '&amp;gt;';&lt;/code&gt; declaration (certainly true for Pythonic equivalents &lt;code&gt;if line == '&amp;gt;'&lt;/code&gt;) but it doesn't offer the ease and flexibility, which is combined with speed, that Perl pie does. The code will work in Linux (any flavour/flavor) or Mac OSX Darwin (i.e. Terminal app)&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="298" LastEditDate="2022-04-06T08:40:21.867" LastActivityDate="2022-04-06T08:40:21.867" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18883" PostTypeId="2" ParentId="18866" CreationDate="2022-04-05T17:32:56.390" Score="1" Body="&lt;p&gt;Are you using the same data from the integration tutorial or using your own data? How many cells/genes are in your dataset?&lt;/p&gt;&#xA;&lt;p&gt;Seurat's default integration method (CCA) is known to be runtime/memory intensive. It can handle large datasets but may require lots of CPU cores/memory. I have found that the amount of RAM also seems to increase as more CPU cores are used. That said, I have been able to run the integration tutorial with the included dataset on an 8GB laptop.&lt;/p&gt;&#xA;&lt;p&gt;The easiest solution is to simply use more resources when integrating: increase the amount of RAM and consider using multiple cores (there is a separate Seurat tutorial on parallelization). If you are encountering runtime/memory issues and you cannot scale up your resources, I would suggest looking at alternative integration methods:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Seurat includes another integration approach, &lt;a href=&quot;https://satijalab.org/seurat/articles/integration_rpca.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;RPCA&lt;/a&gt; which is supposed to be less memory intensive at the cost of being more conservative with integration&lt;/li&gt;&#xA;&lt;li&gt;There are several other integration methods that provide similar results. I have used &lt;a href=&quot;https://portals.broadinstitute.org/harmony/articles/quickstart.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Harmony&lt;/a&gt; and found that it requires significantly less resources than the Seurat integration approaches (both CCA and RPCA). A quick google search of single cell RNA-seq integration methods will turn up other popular methods with benchmarks to compare and contrasts different approaches.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="6674" LastActivityDate="2022-04-05T17:32:56.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18884" PostTypeId="2" ParentId="18881" CreationDate="2022-04-05T17:38:56.023" Score="3" Body="&lt;p&gt;I recommend to use Biopython as a clearer way.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;records = []&#xA;for record in SeqIO.parse(&amp;quot;file.fas&amp;quot;, &amp;quot;fasta&amp;quot;):&#xA;    record.description += &amp;quot;;&amp;quot;&#xA;    records.append(record)&#xA;SeqIO.write(records, &amp;quot;file.fas&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Before:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;OK087550.1 Procyon lotor isolate Plo1 cytochrome b gene, partial cds; mitochondrial&#xA;TCAAACATCTCAGCATGATGAAATTTCGGCTCCCTCCTCGGAATTTGTTTGCTTCTACAGATCGCAACAG&#xA;GTTTATTCTTAGCCATGCACTATACACCAGATACAACCACAGCTTTCTCATCAGTGACCCACATTTGCCG&#xA;AGATGTAAATTATGGCTGAATTATCCGATATATACACGCTAACGGAGCTTCTATATTCTTTATATGCCTA&#xA;TTCTTACACGTAGGACGAGGCTTATACTATGGCTCCTATACATTCTCTGAAACATGAAATATTGGTGTTA&#xA;TTTTACTACTCACAGTAATAGCAACCGCTTTCATAGGTTACGTCCTCCCATGAGGACAAATATCATTCTG&#xA;AGGAGCAACAGTAATTACAAATCTCCTGTCAGCTATCCCCTATATCGGATCTAACCTTGTAGAATGAATT&#xA;TGAGGAGGGTTTTCAGTAGACAAAGCCACCCTAACACGATTCTTCGCATTCCACTTCATTCTACCATTTA&#xA;TTATCACAACGCTAGCAATAATTCACCTGCTATTCCTACACGAAACAGGATCCAATAATCCTGCTGGAAT&#xA;TACATCAGAATCTGACAAAATTCCATTTCACCCATACTACACCATTAAAGACATTCTGGGAATCCTATTC&#xA;CTTATTTTTATACTTATAGGTTTAGTGCTATTTACGCCAGACCTACTAGGAGACCCAGATAACTACACAC&#xA;CCGCCAACCCCTTAAGCACCCCACCTCACATTAAACCTGAATGATATTTTCTATTCGCCTACGCAATTCT&#xA;ACGTTCCATTCCCAACAAACTAGGAGGAGTCCTAGCCCTAGTCCTCTCCATCTTAATCCTAATCATCATC&#xA;CCACTCCTACATACCTCAAAACAACGAAGCATAATATTTCGGCCACTTAGCCAATGTTCATTCTGATTCC&#xA;TAGTAGCAGACCTCCTCGTCCTAACATGAATTGGAGGTCAACCAGTAGAATATCCCTTCATCATCATCGG&#xA;CCAACTAGCCTCCATCTTCTACTTTATAATCCTCCTGGTCCT&#xA;&#xA;&amp;gt;OK087549.1 Procyon lotor isolate Plo17 cytochrome b gene, partial cds; mitochondrial&#xA;TCAAACATCTCAGCATGATGAAATTTCGGCTCCCTCCTCGGAATTTGTTTGCTTCTACAGATCGCAACAG&#xA;GTTTATTCTTAGCCATGCACTATACACCAGATACAACCACAGCTTTCTCATCAGTGACCCACATTTGCCG&#xA;AGATGTAAATTATGGCTGAATTATCCGATATATACACGCTAACGGAGCTTCTATATTCTTTATATGCCTA&#xA;TTCTTACACGTAGGACGAGGCTTATACTATGGCTCCTATACATTCTCTGAAACATGAAATATTGGTGTTA&#xA;TTTTACTACTCACAGTAATAGCAACCGCTTTCATAGGTTACGTCCTCCCATGAGGACAAATATCATTCTG&#xA;AGGAGCAACAGTAATTACAAATCTCCTGTCAGCTATCCCCTATATCGGATCTAACCTTGTAGAATGAATT&#xA;TGAGGAGGGTTTTCAGTAGACAAAGCCACCCTAACACGATTCTTCGCATTCCACTTCATTCTACCATTTA&#xA;TTATCACAACGCTAGCAATAATTCACCTGCTATTCCTACACGAAACAGGATCCAATAATCCTGCTGGAAT&#xA;TACATCAGAATCTGACAAAATTCCATTTCACCCATACTACACCATTAAAGACATTCTGGGAATCCTATTC&#xA;CTTATTTTTATACTTATAGGTTTAGTGCTATTTACGCCAGACCTACTAGGAGACCCAGATAACTACACAC&#xA;CCGCCAACCCCTTAAGCACCCCACCTCACATTAAACCTGAATGATATTTTCTATTCGCCTACGCAATTCT&#xA;ACGTTCCATTCCCAACAAACTAGGAGGAGTCCTAGCCCTAGTCCTCTCCATCTTAATCCTAATCATCATC&#xA;CCACTCCTACATACCTCAAAACAACGAAGCATAATATTTCGGCCACTTAGCCAATGTTCATTCTGATTCC&#xA;TAGTAGCAGACCTCCTCGTCCTAACATGAATTGGAGGTCAACCAGTAGAATATCCCTTCATCATCATCGG&#xA;CCAACTAGCCTCCATCTTCTACTTTATAATCCTCCTGGTCCT&#xA;&#xA;&amp;gt;OK087548.1 Procyon lotor isolate Plo11 cytochrome b gene, partial cds; mitochondrial&#xA;TCAAACATCTCAGCATGATGAAATTTCGGCTCCCTCCTTGGAATTTGTTTGCTTCTACAGATCGCAACAG&#xA;GTTTATTCTTAGCCATGCACTATACACCAGATACAACCACAGCTTTCTCATCAGTGACCCACATTTGCCG&#xA;AGATGTAAATTATGGCTGAATTATCCGATATATACACGCTAACGGAGCTTCTATATTCTTTATATGCCTA&#xA;TTCTTACACGTAGGACGAGGCTTATACTATGGCTCCTATACATTCTCTGAAACATGAAATATTGGTGTTA&#xA;TTTTACTACTCACAGTAATAGCAACCGCTTTCATAGGTTACGTCCTCCCATGAGGACAAATATCATTCTG&#xA;AGGAGCAACAGTAATTACAAATCTCCTGTCAGCTATCCCCTATATCGGATCTAACCTTGTAGAATGAATT&#xA;TGAGGAGGGTTTTCAGTAGACAAAGCCACCCTAACACGATTCTTCGCATTCCACTTCATTCTACCATTTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;OK087550.1 Procyon lotor isolate Plo1 cytochrome b gene, partial cds; mitochondrial;&#xA;TCAAACATCTCAGCATGATGAAATTTCGGCTCCCTCCTCGGAATTTGTTTGCTTCTACAG&#xA;ATCGCAACAGGTTTATTCTTAGCCATGCACTATACACCAGATACAACCACAGCTTTCTCA&#xA;TCAGTGACCCACATTTGCCGAGATGTAAATTATGGCTGAATTATCCGATATATACACGCT&#xA;AACGGAGCTTCTATATTCTTTATATGCCTATTCTTACACGTAGGACGAGGCTTATACTAT&#xA;GGCTCCTATACATTCTCTGAAACATGAAATATTGGTGTTATTTTACTACTCACAGTAATA&#xA;GCAACCGCTTTCATAGGTTACGTCCTCCCATGAGGACAAATATCATTCTGAGGAGCAACA&#xA;GTAATTACAAATCTCCTGTCAGCTATCCCCTATATCGGATCTAACCTTGTAGAATGAATT&#xA;TGAGGAGGGTTTTCAGTAGACAAAGCCACCCTAACACGATTCTTCGCATTCCACTTCATT&#xA;CTACCATTTATTATCACAACGCTAGCAATAATTCACCTGCTATTCCTACACGAAACAGGA&#xA;TCCAATAATCCTGCTGGAATTACATCAGAATCTGACAAAATTCCATTTCACCCATACTAC&#xA;ACCATTAAAGACATTCTGGGAATCCTATTCCTTATTTTTATACTTATAGGTTTAGTGCTA&#xA;TTTACGCCAGACCTACTAGGAGACCCAGATAACTACACACCCGCCAACCCCTTAAGCACC&#xA;CCACCTCACATTAAACCTGAATGATATTTTCTATTCGCCTACGCAATTCTACGTTCCATT&#xA;CCCAACAAACTAGGAGGAGTCCTAGCCCTAGTCCTCTCCATCTTAATCCTAATCATCATC&#xA;CCACTCCTACATACCTCAAAACAACGAAGCATAATATTTCGGCCACTTAGCCAATGTTCA&#xA;TTCTGATTCCTAGTAGCAGACCTCCTCGTCCTAACATGAATTGGAGGTCAACCAGTAGAA&#xA;TATCCCTTCATCATCATCGGCCAACTAGCCTCCATCTTCTACTTTATAATCCTCCTGGTC&#xA;CT&#xA;&amp;gt;OK087549.1 Procyon lotor isolate Plo17 cytochrome b gene, partial cds; mitochondrial;&#xA;TCAAACATCTCAGCATGATGAAATTTCGGCTCCCTCCTCGGAATTTGTTTGCTTCTACAG&#xA;ATCGCAACAGGTTTATTCTTAGCCATGCACTATACACCAGATACAACCACAGCTTTCTCA&#xA;TCAGTGACCCACATTTGCCGAGATGTAAATTATGGCTGAATTATCCGATATATACACGCT&#xA;AACGGAGCTTCTATATTCTTTATATGCCTATTCTTACACGTAGGACGAGGCTTATACTAT&#xA;GGCTCCTATACATTCTCTGAAACATGAAATATTGGTGTTATTTTACTACTCACAGTAATA&#xA;GCAACCGCTTTCATAGGTTACGTCCTCCCATGAGGACAAATATCATTCTGAGGAGCAACA&#xA;GTAATTACAAATCTCCTGTCAGCTATCCCCTATATCGGATCTAACCTTGTAGAATGAATT&#xA;TGAGGAGGGTTTTCAGTAGACAAAGCCACCCTAACACGATTCTTCGCATTCCACTTCATT&#xA;CTACCATTTATTATCACAACGCTAGCAATAATTCACCTGCTATTCCTACACGAAACAGGA&#xA;TCCAATAATCCTGCTGGAATTACATCAGAATCTGACAAAATTCCATTTCACCCATACTAC&#xA;ACCATTAAAGACATTCTGGGAATCCTATTCCTTATTTTTATACTTATAGGTTTAGTGCTA&#xA;TTTACGCCAGACCTACTAGGAGACCCAGATAACTACACACCCGCCAACCCCTTAAGCACC&#xA;CCACCTCACATTAAACCTGAATGATATTTTCTATTCGCCTACGCAATTCTACGTTCCATT&#xA;CCCAACAAACTAGGAGGAGTCCTAGCCCTAGTCCTCTCCATCTTAATCCTAATCATCATC&#xA;CCACTCCTACATACCTCAAAACAACGAAGCATAATATTTCGGCCACTTAGCCAATGTTCA&#xA;TTCTGATTCCTAGTAGCAGACCTCCTCGTCCTAACATGAATTGGAGGTCAACCAGTAGAA&#xA;TATCCCTTCATCATCATCGGCCAACTAGCCTCCATCTTCTACTTTATAATCCTCCTGGTC&#xA;CT&#xA;&amp;gt;OK087548.1 Procyon lotor isolate Plo11 cytochrome b gene, partial cds; mitochondrial;&#xA;TCAAACATCTCAGCATGATGAAATTTCGGCTCCCTCCTTGGAATTTGTTTGCTTCTACAG&#xA;ATCGCAACAGGTTTATTCTTAGCCATGCACTATACACCAGATACAACCACAGCTTTCTCA&#xA;TCAGTGACCCACATTTGCCGAGATGTAAATTATGGCTGAATTATCCGATATATACACGCT&#xA;AACGGAGCTTCTATATTCTTTATATGCCTATTCTTACACGTAGGACGAGGCTTATACTAT&#xA;GGCTCCTATACATTCTCTGAAACATGAAATATTGGTGTTATTTTACTACTCACAGTAATA&#xA;GCAACCGCTTTCATAGGTTACGTCCTCCCATGAGGACAAATATCATTCTGAGGAGCAACA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15161" LastActivityDate="2022-04-05T17:38:56.023" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="18885" PostTypeId="1" CreationDate="2022-04-05T21:14:30.147" Score="2" ViewCount="17" Body="&lt;p&gt;I am having difficulties with an alignment on &lt;a href=&quot;https://toolkit.tuebingen.mpg.de/tools/hhpred&quot; rel=&quot;nofollow noreferrer&quot;&gt;HHpred&lt;/a&gt; and was wondering if anyone had any answer to the following:&lt;/p&gt;&#xA;&lt;p&gt;I am trying to align my protein (PDB 2VU9, a Botulinum Neurotoxin Type a) with one of the top hits generated from Vast.&lt;/p&gt;&#xA;&lt;p&gt;I know that both my query protein and the other protein both have Glutamic acid as their active site. But, my protein has its active site on the 234th amino acid, whereas the other protein has the active site on its 230th amino acid.&lt;/p&gt;&#xA;&lt;p&gt;So my question is:&#xA;What would be the reasoning for which the top matches on Vast do not align at the active site with my protein, despite sharing the same amino acid at its active site?&lt;/p&gt;&#xA;&lt;p&gt;Any response helps!&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Abstract&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;HHpred is a fast server for remote protein homology detection and structure prediction and is the first to implement pairwise comparison of profile hidden Markov models (HMMs). It allows to search a wide choice of databases, such as the PDB, SCOP, Pfam, SMART, COGs and CDD.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Söding et al. Nucleic Acids Res. 2005 Jul 1; 33(Web Server issue): W244–W248.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1160169/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1160169/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14954" LastEditorUserId="3848" LastEditDate="2022-04-05T21:53:30.210" LastActivityDate="2022-04-05T21:53:30.210" Title="Active sites of the query protein does not align with the top hit using HHpred" Tags="|public-databases|pdb|uniprot|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18886" PostTypeId="2" ParentId="18881" CreationDate="2022-04-06T02:39:53.477" Score="7" Body="&lt;p&gt;One way, using &lt;a href=&quot;/questions/tagged/sed&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;sed&amp;#39;&quot; rel=&quot;tag&quot;&gt;sed&lt;/a&gt; and a regex &lt;a href=&quot;https://www.gnu.org/software/sed/manual/html_node/Addresses.html&quot; rel=&quot;noreferrer&quot;&gt;address&lt;/a&gt; to select only the header lines and apply a substitution to append a semicolon to the end of each line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sed '/^&amp;gt;/s/$/;/' in.fa &amp;gt; out.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or, to edit the file in place:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sed -i '/^&amp;gt;/s/$/;/' in.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Or one way using &lt;a href=&quot;/questions/tagged/awk&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; rel=&quot;tag&quot;&gt;awk&lt;/a&gt; and a ternary construct to print the line followed by a semicolon or an empty string, depending on whether or not the line is a header line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '{ print $0 (/^&amp;gt;/ ? &amp;quot;;&amp;quot; : &amp;quot;&amp;quot;) }' in.fa &amp;gt; out.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alternatively, you may prefer:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '/^&amp;gt;/ { print $0 &amp;quot;;&amp;quot;; next } { print $0 }' in.fa &amp;gt; out.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which is the same as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '/^&amp;gt;/ { print $0 &amp;quot;;&amp;quot;; next }1' in.fa &amp;gt; out.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;If you have &lt;a href=&quot;/questions/tagged/perl&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;perl&amp;#39;&quot; rel=&quot;tag&quot;&gt;perl&lt;/a&gt; installed, all you need is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -pe 's/$/;/ if /^&amp;gt;/' in.fa &amp;gt; out.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2022-04-06T11:19:42.127" LastActivityDate="2022-04-06T11:19:42.127" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18890" PostTypeId="1" AcceptedAnswerId="18891" CreationDate="2022-04-06T11:52:50.270" Score="2" ViewCount="294" Body="&lt;p&gt;I want to replicate the synergy drug response curve with the Python library given in the following paper.&#xA;Could you kindly advice on:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Suitable graphics libraries for Python&lt;/li&gt;&#xA;&lt;li&gt;Code or pseudo code&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Apologies because I don't currently have a test code.&lt;/p&gt;&#xA;&lt;p&gt;I would really appreciate if anyone could guide me in this.&#xA;Thank you in advance.&lt;/p&gt;&#xA;&lt;p&gt;Fig 1a&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/s41586-022-04437-2/figures/1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41586-022-04437-2/figures/1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OTRxc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OTRxc.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4845" LastEditorUserId="3848" LastEditDate="2022-04-06T13:04:31.337" LastActivityDate="2022-04-06T14:34:58.360" Title="How to create drug synergy plot?" Tags="|python|drugs|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="18891" PostTypeId="2" ParentId="18890" CreationDate="2022-04-06T12:47:51.410" Score="1" Body="&lt;p&gt;To get you started. Complex editing in matplotlib isn't intuitive.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;from numpy import linspace, sin&#xA;# you don't need math its just to get the value of pi for this data&#xA;import math &#xA;import matplotlib.pyplot as plt&#xA;&#xA;&#xA;xaxis = linspace(1,1.75*math.pi,10)&#xA;drug2 = linspace(1,1.45*math.pi,10)&#xA;&#xA;drug1 = sin(xaxis)&#xA;drug2b = sin(drug2)&#xA;plt.plot(xaxis, drug1, xaxis, drug2b)&#xA;plt.legend(['Drug 1', 'Drug 2'])&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MHoue.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MHoue.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The authors look to have curve fitted and for that you would use &lt;code&gt;import scipy.optimize as opt&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-04-06T14:34:58.360" LastActivityDate="2022-04-06T14:34:58.360" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18893" PostTypeId="2" ParentId="18849" CreationDate="2022-04-06T16:54:14.933" Score="2" Body="&lt;p&gt;It turns out that in order to generate the internal coordinates for a protein structure, one shouldn't manually create an &lt;code&gt;IC_Chain&lt;/code&gt; object, but rather just use the &lt;code&gt;atom_to_internal_coordinates()&lt;/code&gt; method of the structure and the &lt;code&gt;internal_to_atom_coordinates()&lt;/code&gt; one for the inverse functionality.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;structure.atom_to_internal_coordinates()&#xA;&#xA;# Now you have access to the bond angles and lengths&#xA;# of the first chain of structure as follows:&#xA;chain = list(structure.get_chains())[0]&#xA;ic_chain = chain.internal_coord&#xA;&#xA;# Dictionary mapping 3 AtomKeys to a Hedron &#xA;# (with len12, len23, angle fields):&#xA;# ic_chain.hedra&#xA;&#xA;# Dictionary mapping 4 AtomKeys to a Dihedraon&#xA;# (with angle field for torsion angles):&#xA;# ic_chain.dihedra&#xA;&#xA;structure.internal_to_atom_coordinates()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12036" LastActivityDate="2022-04-06T16:54:14.933" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18894" PostTypeId="1" CreationDate="2022-04-07T03:32:58.873" Score="2" ViewCount="56" Body="&lt;p&gt;I have a peptide X of length 50aa. Alphafold2 predicted the structure which takes a helix form and with high average  pLDDT score (&amp;gt;90).&lt;/p&gt;&#xA;&lt;p&gt;But during actual experiment (e.g. crystallography, spectroscopy), peptide X is determined to have a &lt;em&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Random_coil&quot; rel=&quot;nofollow noreferrer&quot;&gt;random coil&lt;/a&gt;&lt;/em&gt; structure.&lt;/p&gt;&#xA;&lt;p&gt;My question is how can I resolve the difference between AF2 prediction and crystallography?&#xA;To what extend we can use AF2 peptide structure prediction for downstream analysis (e.g. docking, etc)?&lt;/p&gt;&#xA;" OwnerUserId="6199" LastActivityDate="2022-04-08T08:30:33.743" Title="How reliable is Alphafold2 for folding random-coil peptide" Tags="|protein-structure|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18895" PostTypeId="2" ParentId="18894" CreationDate="2022-04-07T07:46:49.877" Score="2" Body="&lt;h2&gt;Possibility 1: technical error&lt;/h2&gt;&#xA;&lt;p&gt;AlphaFold2 uses MSA to infer residue co-evolution (due to epistasis) and thus proximity. A MSA can become contaminated with incorrect homologues, especially shorter protein. &lt;em&gt;Generally&lt;/em&gt; these will have low scores and be filtered out, but if they don't they will come in like a wrecking ball.&lt;/p&gt;&#xA;&lt;h2&gt;Possibility 2: conformational switching&lt;/h2&gt;&#xA;&lt;p&gt;Even though on circular dichroism a random coil and helix could not be more opposites, several proteins adopt both conformations and switch. Amyloids are a classic example of this, switching between a beta sheet and a random coil. Synuclein is another example.&lt;/p&gt;&#xA;&lt;p&gt;A technically problematic (vide infra) alternative option is that the helical conformation is only found when the protein is bound to another.&lt;/p&gt;&#xA;&lt;h2&gt;Solution: AlphaFold2 testing&lt;/h2&gt;&#xA;&lt;p&gt;Whereas EBI-AF2 provides a single model, the AF2 algorithm can provide multiple possible solutions, which in many cases are actually bona fide alternate conformations. On a protein I tried, the phosphorylations in PhosphoSitePlus (high-throughput) explained wonderfully a second model.&#xA;The model with the best score is not the &amp;quot;most normal&amp;quot; conformer: not only such a statement makes no sense, but is the one that best obeys the evolutionary signals.&#xA;I would suggest looking at &lt;a href=&quot;https://github.com/sokrypton/ColabFold&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/sokrypton/ColabFold&lt;/a&gt;&#xA;Set the target number of decoys to more than 5 and if all are helices then it's a technical issue. In particular the MSA data would need to be dissected if one really cared. But if nothing comes up... it would be a low reward area of investigation as one would need to discount other less and less likely possibilities.&lt;/p&gt;&#xA;&lt;p&gt;If the decoys &lt;em&gt;do&lt;/em&gt; show alternative folds, i.e. coils, then a few things spring to mind that could be tested &lt;em&gt;in silico&lt;/em&gt; (applicable only to some cases):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;different pH sidechain protonation&lt;/li&gt;&#xA;&lt;li&gt;ion binding&lt;/li&gt;&#xA;&lt;li&gt;phosphorylations&lt;/li&gt;&#xA;&lt;li&gt;binding to nucleic acids&lt;/li&gt;&#xA;&lt;li&gt;binding to known protein candidates —not worth exploring if there's no very likely binding partners. If different paralogues bind differ targets it's tricky&lt;/li&gt;&#xA;&lt;li&gt;binding to membranes&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Looking in PyMOL at the location of the high pLDDT scores and after running APBS (Adv. Poisson-Boltzmann Solver) within PyMOL may reveal something in the helical AlphaFold2 model.&#xA;Once the conditions which explain the helical model (e.g. bond to DNA as a heterotetramer in the presence of zinc at pH 10 has a very low ∆G), those conditions could be tested in vitro.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2022-04-08T08:30:33.743" LastActivityDate="2022-04-08T08:30:33.743" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18896" PostTypeId="2" ParentId="3169" CreationDate="2022-04-07T08:57:40.390" Score="1" Body="&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;record = NCBIXML.read(result_handle_xml)&#xA;query_len = record.query_length&#xA;for alignment in record.alignments:&#xA;    _ = alignment.hsps.__next__()&#xA;    query_cover = (_.query_end - _.query_start + 1) / query_len * 100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15161" LastActivityDate="2022-04-07T08:57:40.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18899" PostTypeId="1" AcceptedAnswerId="18907" CreationDate="2022-04-08T11:45:27.687" Score="2" ViewCount="341" Body="&lt;p&gt;For protein multiple sequence alignment (MSA), I am using MUSCLE (v5.1). The default setting without any specifications gives a 'bad' output even though my proteins are very similar (UniProt sequence and the corresponding PDB SEQRES sequences). 'Bad' is defined as unnecessary gaps are introduced into the sequences.&lt;/p&gt;&#xA;&lt;p&gt;However when the super5 algorithm is used the MSA output is correct, which is described here &lt;a href=&quot;https://drive5.com/muscle5/manual/cmd_super5.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://drive5.com/muscle5/manual/cmd_super5.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In addition, when I increase the gap penalty this issue can be solved but as far as I know, super4 v5.1 doesn't give you the opportunity to adjust gap penalty.&lt;/p&gt;&#xA;&lt;p&gt;Documentation describes super5 quite similarly than the default algorithm, it &amp;quot;just&amp;quot; reduces runtime for long sequences.&lt;/p&gt;&#xA;&lt;p&gt;Can someone please give me a hint what am I missing and/or why super5 outperforms default in terms of accuracy in such &amp;quot;easy&amp;quot; cases?&lt;/p&gt;&#xA;" OwnerUserId="12203" LastEditorUserId="12203" LastEditDate="2022-04-11T05:34:16.500" LastActivityDate="2022-04-11T05:34:16.500" Title="Sequence alignment with MUSCLE" Tags="|sequence-alignment|multiple-sequence-alignment|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18900" PostTypeId="1" CreationDate="2022-04-08T15:26:52.720" Score="0" ViewCount="184" Body="&lt;p&gt;I need to find homologs of certain proteins by BLAST search or similar. But I want filter (select) homologs from organisms that are classified as thermophilic or hyperthermophilic. I am wondering if there are suggestions for me how approach to this. Thank you.&lt;/p&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2022-04-11T10:13:53.030" Title="How to identify [hyper]thermophilic proteins by BLAST search?" Tags="|proteins|blast|sequence-homology|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18901" PostTypeId="2" ParentId="18900" CreationDate="2022-04-08T15:31:26.033" Score="0" Body="&lt;p&gt;I recommend use the local BLAST search. You can form the request for desired taxa in Entrez or through NCBI website &amp;amp; download fasta-file onto your disk.&lt;/p&gt;&#xA;" OwnerUserId="15161" LastActivityDate="2022-04-08T15:31:26.033" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18902" PostTypeId="1" CreationDate="2022-04-08T17:11:21.613" Score="0" ViewCount="3190" Body="&lt;p&gt;I made this wrapper for DimPlot:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;highlight_gene_expression &amp;lt;- function(seurat, gene_counts, colors = NULL, cells = NULL){&#xA;  Idents(seurat) &amp;lt;- 'cell_type'&#xA;  p &amp;lt;- DimPlot(&#xA;    seurat,&#xA;    raster = FALSE,&#xA;    shuffle = TRUE,&#xA;    reduction = &amp;quot;umap&amp;quot;,&#xA;    pt.size = 0.01,&#xA;    cells = cells,&#xA;    cols = colors,&#xA;  )&#xA;  p &amp;lt;- p + guides(color = guide_legend(override.aes = list(size = 4), ncol = 1))&#xA;&#xA;  if (!is.null(cells)) { gene_counts &amp;lt;- gene_counts[cells,] }&#xA;  expression_level &amp;lt;- {gene_counts |&amp;gt; rowSums()} / {gene_counts |&amp;gt; rowSums() |&amp;gt; max()}&#xA;  p[[1]]&lt;span class=&quot;math-container&quot;&gt;$layers[[1]]$&lt;/span&gt;aes_params$alpha &amp;lt;- expression_level&#xA;  p&#xA;}&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It takes a Seurat object and a dataframe of gene expression levels.  It returns a UMAP with the transparency (alpha) of each point determined by the gene expression level:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;highlight_gene_expression(&#xA;  seurat, # a seurat object&#xA;  trgd_counts, # A dataframe of gene expression levels.  Not important to understand for this question.&#xA;  cells = trgd_cells, # A vector of cell barcodes&#xA;  colors = colors # c(&amp;quot;Erythrocytes&amp;quot;=&amp;quot;#556b2f&amp;quot;, &amp;quot;HSPCs&amp;quot;=&amp;quot;#483d8b&amp;quot;, &amp;quot;Proliferating T Cells&amp;quot;=&amp;quot;#3cb371&amp;quot;, ...)&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4ja5F.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4ja5F.png&quot; alt=&quot;umap with identity colors&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Taking it one step further, I highlight the desired cells instead of just coloring them:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;highlight_gene_expression &amp;lt;- function(seurat, gene_counts, cells = rownames(seurat@meta.data), colors = NULL){&#xA;  Idents(seurat) &amp;lt;- 'cell_type'&#xA;  p &amp;lt;- DimPlot(&#xA;    seurat,&#xA;    raster = FALSE,&#xA;    shuffle = TRUE,&#xA;    reduction = &amp;quot;umap&amp;quot;,&#xA;    cols.highlight = colors,&#xA;    pt.size = 0.01,&#xA;    sizes.highlight = 0.01,&#xA;    cells.highlight = seurat@meta.data[cells,] %&amp;gt;% split(.$cell_type) |&amp;gt; lapply(rownames),&#xA;  )&#xA;  p &amp;lt;- p + guides(color = guide_legend(override.aes = list(size = 4), ncol = 1))&#xA;&#xA;  expression_level &amp;lt;- {gene_counts |&amp;gt; rowSums()} / {gene_counts |&amp;gt; rowSums() |&amp;gt; max()}&#xA;  p[[1]]&lt;span class=&quot;math-container&quot;&gt;$layers[[1]]$&lt;/span&gt;aes_params$alpha &amp;lt;- expression_level&#xA;  p&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This produces a similar plot, but with unhighlighted points in gray.  I think it looks quite nice:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# Same params as before&#xA;highlight_gene_expression(seurat, trgd_counts, cells = trgd_cells, colors = colors)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zQQ01.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zQQ01.png&quot; alt=&quot;highlighted umap with sequential colors&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The problem is that the colors are in the wrong places.  Dimplot accepts my color vector, but it applies it sequentially when used for highlighting.  This means colors are no longer tied to a specific cluster as they were in the previous example.  A cluster will have a different color now depending on which ones are actually highlighted.&lt;/p&gt;&#xA;&lt;p&gt;This is not a bug.  Rather, it is expected behavior if you look at the &lt;a href=&quot;https://www.rdocumentation.org/packages/Seurat/versions/4.1.0/topics/DimPlot&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation for DimPlot&lt;/a&gt;.  The &lt;em&gt;cols&lt;/em&gt; param corresponds to an identity class.  &lt;em&gt;cols.highlight&lt;/em&gt; does not.  But I want it to.  How can I make &lt;em&gt;cols.highlight&lt;/em&gt; explicitly define colors for each cluster?&lt;/p&gt;&#xA;" OwnerUserId="15309" LastActivityDate="2022-04-08T17:11:21.613" Title="DimPlot - How to highlight cells with identity colors?" Tags="|r|rna-seq|seurat|umap|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18904" PostTypeId="1" CreationDate="2022-04-09T06:05:16.520" Score="2" ViewCount="56" Body="&lt;p&gt;What are the best open source tools to analyse copy number variation using Illumina SNP array data...preferably R based tools, but open to other languages as well.&lt;/p&gt;&#xA;&lt;p&gt;I'm working with leukemia sample and I have Illumina scan data ..in IDAT/GTC file format and i wish to use tools that will utilise the BAF and LRR metrics.&lt;/p&gt;&#xA;&lt;p&gt;Microarray platform used- Illumina infinium 850K cytoSNP (beadchip array)&lt;/p&gt;&#xA;" OwnerUserId="15317" LastActivityDate="2022-04-09T06:05:16.520" Title="open source tools for CNV analysis from illumina SNP array" Tags="|r|microarray|illumina|cnv|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18905" PostTypeId="1" CreationDate="2022-04-09T17:47:04.867" Score="2" ViewCount="45" Body="&lt;p&gt;I have attached a screenshot from terminal, which is the result of me attempting to run Flexaid on my Mac.&lt;/p&gt;&#xA;&lt;p&gt;I was wondering if anyone knows why this isn't running?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&#xA;&lt;a href=&quot;https://i.stack.imgur.com/TsBrM.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TsBrM.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14954" LastActivityDate="2023-09-03T05:02:10.233" Title="Running into an error when loading Flexaid on PyMol 3.7" Tags="|proteins|docking|pymol|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18906" PostTypeId="2" ParentId="17593" CreationDate="2022-04-09T22:10:55.887" Score="2" Body="&lt;p&gt;Note to OP that your tag of @user172818 was not successful in your last comment. That's probably why they didn't respond.&lt;/p&gt;&#xA;&lt;p&gt;Because no one has gone there so far, I will simply link to &lt;a href=&quot;https://github.com/chhylp123/hifiasm&quot; rel=&quot;nofollow noreferrer&quot;&gt;HiFiAsm&lt;/a&gt;. See linked Nature Methods paper. This tool accepts HiFi reads, which are now standard, and gives you a GFA that you can &lt;a href=&quot;https://www.biostars.org/p/169516/&quot; rel=&quot;nofollow noreferrer&quot;&gt;quite easily convert to FA&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Polishing with Illumina would actively harm the assembly, as it would remove real heterozygosity or add errors where Illumina reads map ambiguously.&lt;/p&gt;&#xA;&lt;p&gt;Run &amp;quot;out of the box&amp;quot;t with default options on HiFi it will probably give you much better results than VGP, haplotype-resolved.&lt;/p&gt;&#xA;&lt;p&gt;If you also add Hi-C PE reads with no additional options, it will give you something much much better than VGP.&lt;/p&gt;&#xA;&lt;p&gt;It's great. See the docs for practical info on running it.&lt;/p&gt;&#xA;&lt;p&gt;There is no reason to follow VGP workflows, as they are very very outdated, as pointed out in comments.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-04-09T22:10:55.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18907" PostTypeId="2" ParentId="18899" CreationDate="2022-04-10T11:14:07.207" Score="1" Body="&lt;p&gt;&lt;strong&gt;HMM&lt;/strong&gt; Super5 is a hidden Markov model (HMM) where  gap penalties and the substitution matrix are the parameters at the centre of the model. That is the answer, thats why you can't change it - it does it itself and thats its purpose. Thus one algorithm (super5) is HMM for the gap-penalty and the other one I suspect isn't.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Explanation&lt;/strong&gt; To explain that further HMM uses observed outcomes to estimate/optimise the hidden parameters, the gap penalty being one of the two key hidden parameters. Thus the algorithm is exploring a parameter landscape through reiteration generating multi-multiple alignments (if that makes sense) using different gap penalties and considering the error between the alignments (highlighted below). The search strategy is key to the success of the output. Muscle 5 describes 'introducing variations (perturbations) into HMM parameters', I assume this is singularly with reference to super5. The parameter landscape might be quite small because it doesn't seem to be a directional search strategy in any way (there's no memory in it).&lt;/p&gt;&#xA;&lt;p&gt;The optimisation algorithm, i.e. what is being observed is a metric called the 'pair-wise error (PWE) and pair-wise dispersion (PWD)', so what appears to be happening is the algorithm is attempting to minimise the number of gaps in an alignment and this would be compatible with your results.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Speed&lt;/strong&gt; Super5 is breaking the alignment up into smaller alignment and solving each 'centroid' before bringing together the 'centroids'. This is an obvious strategy particularly in the era of multiple processors because it overcomes RAM bottlenecks, we are all doing that in any algorithmic context.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Overall&lt;/strong&gt; The algorithm was not absolutely clear where PWE and the feedback loop is being deployed whether it was in the generation of centroids or bringing together the centroids and its relationship to the guide tree. There was a large reliance on BLOSSUM62 - which didn't quite make sense in the given context because the background was clear about the different substitution matrices and the impact on gap penalties.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; Static gap penalty parameterisation is antithesis of what super5 is about, that is why you cannot use them. The speed is clear its using parallel processing and subalignments - there are no surprises there, thats a common strategy and easy to code these days.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Final points&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I am clear the ms has not been accepted and is available here &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2021.06.20.449169v1.full#ref-10&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/10.1101/2021.06.20.449169v1.full#ref-10&lt;/a&gt; . I don't want to comment further on the manuscript for several reasons, which I prefer to avoid explaining.&lt;/li&gt;&#xA;&lt;li&gt;It is better to wait until the paper is published because it will likely be different.&lt;/li&gt;&#xA;&lt;li&gt;The algorithm itself will unquestionably be subject to simulation studies by other authors.&lt;/li&gt;&#xA;&lt;li&gt;Simulation studies were absent in the BioRXIV manuscript.&lt;/li&gt;&#xA;&lt;li&gt;Muscle historically has been at the forefront of methods to increase sensitivity. If you read the Clustal Omega publication they describe the implementation of historical Muscle sensitivity improvements, in other words they must have been approved over the course of time.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Other authors are likely to investigate whether simulation conditions will result in super5 causing a 'false peak' scenario. That is the Achilles heal in any HMM.&lt;/p&gt;&#xA;&lt;p&gt;There are certainly situations where super5 is useful in automation, that is without question. In some pipelines there is no option of checking the alignments within the pipeline because there are just to many and it stops the pipeline (human eyes the opposite of a data pipeline).&lt;/p&gt;&#xA;&lt;p&gt;In practice optical annotation is used to overcome limitations post alignment, i.e. to resolve 'errors' in gap penalties and this is desirable because it prevents a phenomena of 'circular mathematics' which is a caution in any alignment method. The author does discuss this phenomena without calling this directly 'circular mathematics'. Any alignment algorithm can erroneously impose itself on the alignment and the metric of 'error' is extremely important in this regard. Optical analysis breaks circularisation. There is a very, very obvious solution to all this without using HMM at one extreme or 'optical analysis' at the other extreme, but one day at a time.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Example of HMM&lt;/strong&gt; 'Circularisation' is the Achilles heal of my explanations, because I explain stuff, then reexplain it which more likely increases confusion and intersperse everything with waffle. However, I do follow a HMM style algorithm because next time I explain this it will be more honed and punchier. The hidden parameter being 'clarity' and the observed phenomena being the frequency of blank expressions, which is sought to minimise on each iteration. I do hope my search of the parameter landscape is a bit more directional than that used in Super5.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;parameters looking at the documentation&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-perturb SEED&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It is important to do the same alignment using different random number streams. A given seed should result in the same gap pattern. If there are different gap patterns between different 'SEEDS', there's something suboptimal in the HMM and that is not good at all. If you think the Muscle 5 is implicit perfection, you don't need to check it - but its a trust thing. Its good the author made it explicit, i.e. you can set it.&lt;/p&gt;&#xA;&lt;p&gt;The other parameters are:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-perm PERM&#xA;-consiters N&#xA;-refineiters N&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The second most important parameter in my view is &lt;code&gt;-perm&lt;/code&gt; - this is permutations on the UPGMA guide tree.  If different &lt;code&gt;-perm&lt;/code&gt; values give different alignments thats not good. It must work reasonably well because the author will have checked it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-04-10T21:09:36.950" LastActivityDate="2022-04-10T21:09:36.950" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18908" PostTypeId="2" ParentId="12934" CreationDate="2022-04-10T16:56:02.450" Score="1" Body="&lt;p&gt;I have recently open sourced a Java NIO SPI for S3 that allows direct reads from S3 using Java applications that use the NIO model. This allows applications like GATK and some parts of Picard to understand &lt;code&gt;s3://&lt;/code&gt; URIs.&lt;/p&gt;&#xA;&lt;p&gt;It is used as a drop in so as long as the jar is on your classpath or in your JRE &lt;code&gt;ext&lt;/code&gt; folder then Java will delegate nio operations to this lib when it &amp;quot;sees&amp;quot; an &lt;code&gt;s3://&lt;/code&gt; URI. It will use the credentials and configuration of your current profile so no additional config is require and no recompilation of an application is needed.&lt;/p&gt;&#xA;&lt;p&gt;The github repo is &lt;a href=&quot;https://github.com/awslabs/aws-java-nio-spi-for-s3&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/awslabs/aws-java-nio-spi-for-s3&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Disclosure: I am the author of the library, employed by the AWS Health AI Genomics Team, and have financial interests in Amazon&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="8985" LastActivityDate="2022-04-10T16:56:02.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18909" PostTypeId="2" ParentId="18900" CreationDate="2022-04-10T19:37:53.347" Score="3" Body="&lt;p&gt;It seems like the main issue is finding a list of species to filter your blast results on. I will therefore address this list of species issue.&lt;/p&gt;&#xA;&lt;p&gt;There are some resources such as Microbe Directory that contain organismal characteristics similar to what you describe. Their &lt;a href=&quot;https://microbe.directory/&quot; rel=&quot;nofollow noreferrer&quot;&gt;main site&lt;/a&gt; appears nonfunctional but their &lt;a href=&quot;https://github.com/microbe-directory/microbe-directory/tree/master/data&quot; rel=&quot;nofollow noreferrer&quot;&gt;github still works&lt;/a&gt; and appears to have the full database. This database includes an &lt;code&gt;extreme_environment&lt;/code&gt; field, which appears to be what you are going for.&lt;/p&gt;&#xA;&lt;p&gt;I feel like PATRIC used to have something similar but I can't locate it now.&lt;/p&gt;&#xA;&lt;p&gt;For more ideas you might look at the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5883067/#SF1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Microbe Directory paper&lt;/a&gt; to see if they cite other similar databases.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-04-10T19:37:53.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18911" PostTypeId="2" ParentId="18905" CreationDate="2022-04-11T00:31:13.503" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @maximilian-press, converted from comment. Please improve where possible:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;It seems that many of the files are not being found, including both run executables and your pdb files. That's what all the &lt;code&gt;No such file or directory&lt;/code&gt; stuff is about.&lt;/p&gt;&#xA;&lt;p&gt;I would suggest confirming that you have completely installed PyMol and any of the relevant dependencies such as &lt;code&gt;Process_ligand&lt;/code&gt; that are not being found. &lt;a href=&quot;http://biophys.umontreal.ca/nrg/ewExternalFiles/NRGsuite_Guide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is one resource&lt;/a&gt; that might be relevant for that tool. Is &lt;code&gt;run_flexaid.sh&lt;/code&gt; a script that you have written or a third-party tool? I can't find it by google. Can you show us how you ran it?&lt;/p&gt;&#xA;&lt;p&gt;I am guessing that &lt;code&gt;sh ./run_flexaid.sh&lt;/code&gt; is the terminal command that you are using to run the script, but I still don't know where the script comes from. It's possible that running within the script is not using your environment properly somehow. What happens if you try to run &lt;code&gt;Process_ligand&lt;/code&gt; on its own (if that's possible?). Have you confirmed that your input pdb files are in the right directory? Basically, it's hard to know what's going on without seeing the script and knowing where your input files live on your computer.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Note: Comments are &lt;strong&gt;temporary&lt;/strong&gt; &amp;quot;Post-It&amp;quot; notes left on a question or answer. They should be used for clarification or constructive criticism of questions or answers, not for answering (no matter how minor). More details on this StackExchange policy &lt;a href=&quot;https://bioinformatics.stackexchange.com/help/privileges/comment&quot;&gt;here&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-04-11T00:31:13.503" CommentCount="0" CommunityOwnedDate="2022-04-11T00:31:13.503" ContentLicense="CC BY-SA 4.0" />
  <row Id="18912" PostTypeId="1" AcceptedAnswerId="19061" CreationDate="2022-04-11T03:41:58.460" Score="1" ViewCount="148" Body="&lt;p&gt;I am training a classifier to identify a cell type in a particular state of activity using scRNA-seq.  There is a large variation in the sequencing depth (reads average per cell) of the testing data (data I will be classifying not training or validating the classifier with) and my supervisor suggested binarizing the genes.  I was wondering if there are other options I should consider.  I was wondering if there are particular normalization methods for ML classifiers which would be best for this task.&lt;/p&gt;&#xA;" OwnerUserId="2482" LastEditorUserId="2482" LastEditDate="2022-04-12T02:21:11.527" LastActivityDate="2022-05-13T20:27:23.440" Title="Normalization methods to combine scRNA-seq experiments with different sequencing depths" Tags="|scrnaseq|transcriptome|normalization|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18913" PostTypeId="2" ParentId="18900" CreationDate="2022-04-11T10:13:53.030" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Warning: Hearsay&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As the others have said, filtering by a list of taxonomy ids is the&#xA;only one. However, there is one extra consideration if this is for wet&#xA;lab applications:  expression yields even when codon optimised may be&#xA;affected by phylum choice.&lt;/p&gt;&#xA;&lt;p&gt;Archaeal protein encoded by codon-optimised genes for &lt;em&gt;E. coli&lt;/em&gt; generally perform less well than the equivalent bacterial thermophilic protein. I have not read a study on it, but I have heard its tale over and over and had issues myself. GC-content and the amino acid composition do correlate, but I have heard of good results in &lt;em&gt;E. coli&lt;/em&gt; with &lt;em&gt;Geobacillus&lt;/em&gt; (high-GC thermophilic firmicute) as with &lt;em&gt;Bacillus&lt;/em&gt; (high GC mesophilic firmicute) after all. I don't know if its isoelectric points close to pH 7 or what.&#xA;But mid GC-content bacterial thermophiles, such as &lt;em&gt;Aquifex&lt;/em&gt; or some clades of &lt;em&gt;Betaproteobacteria&lt;/em&gt;, may be a better call than &lt;em&gt;Archaea&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So as you need to filter your blast search by a series of phyla, you may want to be selective if for expression in a different host...&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2022-04-11T10:13:53.030" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="18914" PostTypeId="1" AcceptedAnswerId="19262" CreationDate="2022-04-11T11:38:35.560" Score="3" ViewCount="1448" Body="&lt;p&gt;I am trying to plot one scatter plot for each group, where each group has continous values in x-axis.&#xA;&lt;a href=&quot;https://drive.google.com/file/d/12wCp_rMuwzjJPmtPAYP3JmuUgwjVe-nY/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reference Data&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Gives continuous x-axis but all groups are combined.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(data_for_plotting, &#xA;aes(x = log10(Count), y = log10(Nonsilent.Mutation.Rate), color=factor(Clusters))) + &#xA;geom_point(alpha = 1, size = 1) + theme_bw()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ffQse.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ffQse.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Gives grouped x-axis but the variation in x-axis is lost.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(data_for_plotting, &#xA;aes(x = factor(Clusters), y = log10(Nonsilent.Mutation.Rate), color=factor(Clusters))) + &#xA;geom_point(alpha = 1, size = 1) + theme_bw()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TmOJL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TmOJL.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Does achieve the grouping but the gives multiple plots.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(data_for_plotting, &#xA;aes(x = log10(Count),y = log10(Nonsilent.Mutation.Rate),color=factor(Clusters))) +&#xA;facet_wrap(~ factor(Clusters), shrink = TRUE) +&#xA;geom_point(alpha = 1, size = 1) + theme_bw()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nWr6C.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nWr6C.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want x-axis to be grouped based on Cluster where each cluster has continous x-axis values.&lt;/p&gt;&#xA;" OwnerUserId="2247" LastActivityDate="2022-11-14T21:00:40.083" Title="Grouping x-axis in scatter plot" Tags="|r|visualization|ggplot2|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18915" PostTypeId="1" CreationDate="2022-04-11T12:00:04.563" Score="0" ViewCount="41" Body="&lt;p&gt;This is a computational neuroscience project. We are asked to plot the Morris-Lescar model (question 3 below).&lt;/p&gt;&#xA;&lt;p&gt;The model should be plotted with scilab or R.&lt;/p&gt;&#xA;&lt;p&gt;however, I don't know where to start? For the moment, the problem is more in writing the code to represent the equations and systems of equations than in actually solving them.&lt;/p&gt;&#xA;&lt;p&gt;Any clues as to how to start writing our code would be a huge start!&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/toQtT.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/toQtT.jpg&quot; alt=&quot;Morris-Lecar P1&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/itO9R.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/itO9R.jpg&quot; alt=&quot;Morris-Lecar P2&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In concrete terms, we don't know where to start with my working group, even for question 3.a..&lt;/p&gt;&#xA;&lt;p&gt;We know that nullclines are associated with &lt;span class=&quot;math-container&quot;&gt;$dV/dt = 0$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$dW/dt = 0$&lt;/span&gt;, which gives the following entries:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$W = \frac{I-g{Ca}M{\infty}(V)(V-V_{Ca})-g_L(V-V_L)}{g_K(V-VK)}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$W = W{\infty}(V)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;But here for example, no idea how to write it on scilab, or how to display them graphically by varying I.&lt;/p&gt;&#xA;&lt;p&gt;I hope the application is now clearer.&lt;/p&gt;&#xA;&lt;p&gt;Appendix: scilab code.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;E_L = -50;&#xA;E_Ca = 100;&#xA;E_K = -70;&#xA;V1 = -1.2;&#xA;V2 = 18;&#xA;V3 = 2;&#xA;V4 = 30;&#xA;g_L = 2;&#xA;g_Ca = 4.4;&#xA;g_K = 8.0;&#xA;C = 20;&#xA;gamma = 0.04;&#xA;&#xA;minf(v)=.5(1+tanh((v-va)/vb))&#xA;&#xA;winf(v)=.5(1+tanh((v-vc)/vd))&#xA;&#xA;tauw(v)=1/cosh((v-vc)/(2*vd))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15320" LastEditorUserId="73" LastEditDate="2022-04-12T01:37:33.873" LastActivityDate="2022-04-21T16:49:19.927" Title="Problem of writing scilab code on a project - Morris-Lecar model" Tags="|r|homework|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18916" PostTypeId="1" CreationDate="2022-04-11T12:16:55.903" Score="3" ViewCount="27" Body="&lt;p&gt;I mapped my fastq files from 4SU-seq experiment to human genome hg19 with bowtie 2 and got this results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    67716784 reads; of these:&#xA;  67716784 (100.00%) were paired; of these:&#xA;    29941143 (44.22%) aligned concordantly 0 times&#xA;    33244334 (49.09%) aligned concordantly exactly 1 time&#xA;    4531307 (6.69%) aligned concordantly &amp;gt;1 times&#xA;    ----&#xA;    29941143 pairs aligned concordantly 0 times; of these:&#xA;      9287621 (31.02%) aligned discordantly 1 time&#xA;    ----&#xA;    20653522 pairs aligned 0 times concordantly or discordantly; of these:&#xA;      41307044 mates make up the pairs; of these:&#xA;        32859156 (79.55%) aligned 0 times&#xA;        3142531 (7.61%) aligned exactly 1 time&#xA;        5305357 (12.84%) aligned &amp;gt;1 times&#xA;75.74% overall alignment rate&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also ran a second mapping with --no-discordant and --no-mixed. After that I read the bam files into R like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;x1 &amp;lt;- BamFile(paste0(bam.path, &amp;quot;14751.OnlyConcordantReads.sorted.bam&amp;quot;), asMates = T)&#xA;x2 &amp;lt;- BamFile(paste0(bam.path, &amp;quot;14751.AllReads.sorted.bam&amp;quot;), asMates = T)&#xA;x1 &amp;lt;- readGAlignmentPairs(x1)&#xA;x2 &amp;lt;- readGAlignmentPairs(x2)&#xA;x1_gr &amp;lt;- granges(x1, use.names=T)&#xA;x2_gr &amp;lt;- granges(x2,use.names=T, on.discordant.seqnames=&amp;quot;drop&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The parameter on.discordant.seqnames is necessary, otherwise an error occurs.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;z &amp;lt;- GRanges(&amp;quot;chr1&amp;quot;,IRanges(236958581,237067281))&#xA;xcov &amp;lt;- coverage(x1_gr)&#xA;xnum &amp;lt;- as.numeric(xcov&lt;span class=&quot;math-container&quot;&gt;$chr1[ranges(z)])&#xA;plot(xnum)&#xA;ycov &amp;lt;- coverage(x2_gr)&#xA;ynum &amp;lt;- as.numeric(ycov$&lt;/span&gt;chr1[ranges(z)])&#xA;plot(ynum)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get very unusual coverage values (all summed up to 154 - see image) when I use on.discordant.seqnames=&amp;quot;drop&amp;quot;. Why is that?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BigoS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BigoS.png&quot; alt=&quot;normal&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PcULt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PcULt.png&quot; alt=&quot;on.discordant.seqnames=&amp;quot;drop&amp;quot;&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In addition, the coverage does not match my bed files.&lt;/p&gt;&#xA;" OwnerUserId="15321" LastActivityDate="2022-04-11T12:16:55.903" Title="GRanges coverage error with discordant reads" Tags="|bioconductor|bowtie2|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18918" PostTypeId="1" AcceptedAnswerId="19612" CreationDate="2022-04-11T19:45:26.163" Score="6" ViewCount="158" Body="&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; I submitted a &lt;a href=&quot;https://github.com/Bioconductor/Biostrings/pull/77&quot; rel=&quot;nofollow noreferrer&quot;&gt;pull request&lt;/a&gt; to the Biostrings repo. The functionality I describe in my question and answer can now be implemented with &lt;code&gt;nucleotideSubstitutionMatrix(symmetric = FALSE)&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I am using the &lt;code&gt;pairwiseAlignment&lt;/code&gt; function from the &lt;code&gt;Biostrings&lt;/code&gt; package to calculate the distance between a consensus sequence and a set of other sequences. Some example code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# R v4.2.0&#xA;library(Biostrings)   # v2.63.3&#xA;library(BiocGenerics) # v0.41.2&#xA;library(tidyverse)    # v1.3.1&#xA;&#xA;# consensus sequence&#xA;con &amp;lt;- DNAStringSet(c(&amp;quot;GTTGTGATTTGCTTTCRAATTAGTATCTTTGAACCATTGAAAACAAC&amp;quot;))&#xA;&#xA;# subject sequences&#xA;seq &amp;lt;- DNAStringSet(c(&amp;quot;ATTGTGATTTGCTTTCAAATTAGTATCTTTAAACCATTGAAAACAGC&amp;quot;,&#xA;                      &amp;quot;GTTGTGATTTGCTTTCAAATTAGTATCTTTGAACCATTGAAAACAAC&amp;quot;,&#xA;                      &amp;quot;GTTGTGATTTGCTTTCAAATTAGTATCTTTGAACCATTGAAAACAAC&amp;quot;,&#xA;                      &amp;quot;GTTGTGATTTGCTTTCAAATTAGTATCTTTGAACCATTGAAAACAAC&amp;quot;,&#xA;                      &amp;quot;GTTGTGATTTGCTTTCAAATTAGTATCTTTGAACCATTGAAAACAAC&amp;quot;))&#xA;&#xA;# substitution matrix&#xA;sub &amp;lt;- nucleotideSubstitutionMatrix(match = 0, mismatch = 1)&#xA;&#xA;# get alignments&#xA;sapply(X = 1:length(seq),&#xA;       FUN = function(i) {&#xA;         pairwiseAlignment(pattern = con,&#xA;                           subject = seq[i],&#xA;                           substitutionMatrix = sub) %&amp;gt;%&#xA;         score&#xA;      }&#xA;    ) %&amp;gt;% as_tibble %&amp;gt;% mutate(sequence = as.character(seq)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The resulting tibble looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# A tibble: 5 × 2&#xA;  value sequence                                       &#xA;  &amp;lt;dbl&amp;gt; &amp;lt;chr&amp;gt;                                          &#xA;1  -3.5 ATTGTGATTTGCTTTCAAATTAGTATCTTTAAACCATTGAAAACAGC&#xA;2  -0.5 GTTGTGATTTGCTTTCAAATTAGTATCTTTGAACCATTGAAAACAAC&#xA;3  -0.5 GTTGTGATTTGCTTTCAAATTAGTATCTTTGAACCATTGAAAACAAC&#xA;4  -0.5 GTTGTGATTTGCTTTCAAATTAGTATCTTTGAACCATTGAAAACAAC&#xA;5  -0.5 GTTGTGATTTGCTTTCAAATTAGTATCTTTGAACCATTGAAAACAAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Something isn't quite right. Take a look at the alignment of &lt;code&gt;con&lt;/code&gt; against &lt;code&gt;seq[2]&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;con:    GTTGTGATTTGCTTTCRAATTAGTATCTTTGAACCATTGAAAACAAC&#xA;        |||||||||||||||| ||||||||||||||||||||||||||||||&#xA;seq[2]: GTTGTGATTTGCTTTCAAATTAGTATCTTTGAACCATTGAAAACAAC &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that the only difference is at the position in the consensus sequence where there is &lt;a href=&quot;https://en.wikipedia.org/wiki/Nucleic_acid_notation#IUPAC_notation&quot; rel=&quot;nofollow noreferrer&quot;&gt;ambiguous IUPAC encoding&lt;/a&gt;. Because &lt;strong&gt;R&lt;/strong&gt; can represent &lt;strong&gt;A&lt;/strong&gt; or &lt;strong&gt;G&lt;/strong&gt;, the score for &lt;code&gt;seq[2]&lt;/code&gt; should be 0 (a perfect match). If the ambiguous base was instead in &lt;code&gt;seq[2]&lt;/code&gt; and &lt;code&gt;con&lt;/code&gt; was unambiguous, then the score penalty would be appropriate. The discrepancy here is due to the symmetry of the substitution matrix &lt;code&gt;sub&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;           A          C          G          T          M          R          W          S          Y          K&#xA;A  0.0000000 -1.0000000 -1.0000000 -1.0000000 -0.5000000 -0.5000000 -0.5000000 -1.0000000 -1.0000000 -1.0000000&#xA;C -1.0000000  0.0000000 -1.0000000 -1.0000000 -0.5000000 -1.0000000 -1.0000000 -0.5000000 -0.5000000 -1.0000000&#xA;G -1.0000000 -1.0000000  0.0000000 -1.0000000 -1.0000000 -0.5000000 -1.0000000 -0.5000000 -1.0000000 -0.5000000&#xA;T -1.0000000 -1.0000000 -1.0000000  0.0000000 -1.0000000 -1.0000000 -0.5000000 -1.0000000 -0.5000000 -0.5000000&#xA;M -0.5000000 -0.5000000 -1.0000000 -1.0000000 -0.5000000 -0.7500000 -0.7500000 -0.7500000 -0.7500000 -1.0000000&#xA;R -0.5000000 -1.0000000 -0.5000000 -1.0000000 -0.7500000 -0.5000000 -0.7500000 -0.7500000 -1.0000000 -0.7500000&#xA;W -0.5000000 -1.0000000 -1.0000000 -0.5000000 -0.7500000 -0.7500000 -0.5000000 -1.0000000 -0.7500000 -0.7500000&#xA;S -1.0000000 -0.5000000 -0.5000000 -1.0000000 -0.7500000 -0.7500000 -1.0000000 -0.5000000 -0.7500000 -0.7500000&#xA;Y -1.0000000 -0.5000000 -1.0000000 -0.5000000 -0.7500000 -1.0000000 -0.7500000 -0.7500000 -0.5000000 -0.7500000&#xA;K -1.0000000 -1.0000000 -0.5000000 -0.5000000 -1.0000000 -0.7500000 -0.7500000 -0.7500000 -0.7500000 -0.5000000&#xA;V -0.6666667 -0.6666667 -0.6666667 -1.0000000 -0.6666667 -0.6666667 -0.8333333 -0.6666667 -0.8333333 -0.8333333&#xA;H -0.6666667 -0.6666667 -1.0000000 -0.6666667 -0.6666667 -0.8333333 -0.6666667 -0.8333333 -0.6666667 -0.8333333&#xA;D -0.6666667 -1.0000000 -0.6666667 -0.6666667 -0.8333333 -0.6666667 -0.6666667 -0.8333333 -0.8333333 -0.6666667&#xA;B -1.0000000 -0.6666667 -0.6666667 -0.6666667 -0.8333333 -0.8333333 -0.8333333 -0.6666667 -0.6666667 -0.6666667&#xA;N -0.7500000 -0.7500000 -0.7500000 -0.7500000 -0.7500000 -0.7500000 -0.7500000 -0.7500000 -0.7500000 -0.7500000&#xA;           V          H          D          B     N&#xA;A -0.6666667 -0.6666667 -0.6666667 -1.0000000 -0.75&#xA;C -0.6666667 -0.6666667 -1.0000000 -0.6666667 -0.75&#xA;G -0.6666667 -1.0000000 -0.6666667 -0.6666667 -0.75&#xA;T -1.0000000 -0.6666667 -0.6666667 -0.6666667 -0.75&#xA;M -0.6666667 -0.6666667 -0.8333333 -0.8333333 -0.75&#xA;R -0.6666667 -0.8333333 -0.6666667 -0.8333333 -0.75&#xA;W -0.8333333 -0.6666667 -0.6666667 -0.8333333 -0.75&#xA;S -0.6666667 -0.8333333 -0.8333333 -0.6666667 -0.75&#xA;Y -0.8333333 -0.6666667 -0.8333333 -0.6666667 -0.75&#xA;K -0.8333333 -0.8333333 -0.6666667 -0.6666667 -0.75&#xA;V -0.6666667 -0.7777778 -0.7777778 -0.7777778 -0.75&#xA;H -0.7777778 -0.6666667 -0.7777778 -0.7777778 -0.75&#xA;D -0.7777778 -0.7777778 -0.6666667 -0.7777778 -0.75&#xA;B -0.7777778 -0.7777778 -0.7777778 -0.6666667 -0.75&#xA;N -0.7500000 -0.7500000 -0.7500000 -0.7500000 -0.75&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/Bioconductor/Biostrings/issues/3#issuecomment-1095143260&quot; rel=&quot;nofollow noreferrer&quot;&gt;As part of a discussion on Github&lt;/a&gt;, I provided the code to manually reassign elements of &lt;code&gt;sub&lt;/code&gt; to create an asymmetric substitution matrix:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sub[&amp;quot;N&amp;quot;, &amp;quot;A&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;N&amp;quot;, &amp;quot;T&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;N&amp;quot;, &amp;quot;G&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;N&amp;quot;, &amp;quot;C&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;M&amp;quot;, &amp;quot;A&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;M&amp;quot;, &amp;quot;C&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;R&amp;quot;, &amp;quot;A&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;R&amp;quot;, &amp;quot;G&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;W&amp;quot;, &amp;quot;A&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;W&amp;quot;, &amp;quot;T&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;S&amp;quot;, &amp;quot;C&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;S&amp;quot;, &amp;quot;G&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;Y&amp;quot;, &amp;quot;C&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;Y&amp;quot;, &amp;quot;T&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;K&amp;quot;, &amp;quot;G&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;K&amp;quot;, &amp;quot;T&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;V&amp;quot;, &amp;quot;A&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;V&amp;quot;, &amp;quot;C&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;V&amp;quot;, &amp;quot;G&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;H&amp;quot;, &amp;quot;A&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;H&amp;quot;, &amp;quot;C&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;H&amp;quot;, &amp;quot;T&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;D&amp;quot;, &amp;quot;A&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;D&amp;quot;, &amp;quot;G&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;D&amp;quot;, &amp;quot;T&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;B&amp;quot;, &amp;quot;C&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;B&amp;quot;, &amp;quot;G&amp;quot;] &amp;lt;- 0&#xA;sub[&amp;quot;B&amp;quot;, &amp;quot;T&amp;quot;] &amp;lt;- 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This covers all cases where the pattern (&lt;code&gt;con&lt;/code&gt;) contains ambiguous bases and the subjects (&lt;code&gt;seq&lt;/code&gt;) are unambiguous, changing the matrix so the subjects are not penalized. However, these edits do not account for cases where both the pattern and the subject have ambiguous encodings at the same position; &lt;em&gt;e.g.&lt;/em&gt; &lt;code&gt;[&amp;quot;M&amp;quot;,&amp;quot;V&amp;quot;]&lt;/code&gt; and &lt;code&gt;[&amp;quot;V&amp;quot;,&amp;quot;M&amp;quot;]&lt;/code&gt; have the same value associated with them, even though &lt;strong&gt;V&lt;/strong&gt; (&lt;code&gt;A || C || G&lt;/code&gt;) is a superset of &lt;strong&gt;M&lt;/strong&gt; (&lt;code&gt;A || C&lt;/code&gt;). There are also more nuanced cases where the set of bases represented by a pair of ambiguous encodings overlap incompletely in both directions, as with &lt;strong&gt;H&lt;/strong&gt; (&lt;code&gt;A || C || T&lt;/code&gt;) and &lt;strong&gt;D&lt;/strong&gt; (&lt;code&gt;A || G || T&lt;/code&gt;). Given enough time, I could probably figure out the appropriate scores for each case to complete the asymmetric substitution matrix, but I'm surely reinventing the wheel. &lt;strong&gt;Can someone supply, as a standalone table or as part of a codebase, an asymmetric nucleotide substitution matrix that satisfies the criteria outlined herein?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="3967" LastEditorUserId="3967" LastEditDate="2022-11-30T16:10:05.603" LastActivityDate="2023-05-21T21:19:57.427" Title="Where to find asymmetric nucleotide substitution matrix with IUPAC encodings?" Tags="|r|sequence-alignment|bioconductor|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18919" PostTypeId="1" CreationDate="2022-04-11T20:44:29.343" Score="2" ViewCount="234" Body="&lt;p&gt;I'm working with PTX samples (human and mouse), they're aligned to both transcriptomes and so gene names contain hg38- and mm10- at the beginning of each gene. I have eliminated teh prefixes because I want to integrate this dataset with a dataset that does not contain prefixes. I have annotated each cell type accordingly and have subsetted out the mouse cells.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;human &amp;lt;- grep(pattern = &amp;quot;^hg38-&amp;quot;, x = rownames(x = scData), value = TRUE)&#xA;mouse &amp;lt;- grep(pattern = &amp;quot;^mm10-&amp;quot;, x = rownames(x = scData), value = TRUE)&#xA;&#xA;mouse_1 &amp;lt;- substring(mouse,6)&#xA;&#xA;human_1 &amp;lt;- substring(human,6)&#xA;&#xA;human_only &amp;lt;- subset(PTX_no_prefix, features = human_1)&#xA;&#xA;mouse_only &amp;lt;- subset(PTX_no_prefix, features = mouse_1)&#xA;&#xA;Idents(mouse_only) &amp;lt;- &amp;quot;cell_ann&amp;quot;&#xA;TME_PTX &amp;lt;- subset(mouse_only, idents = c(&amp;quot;Fibroblast_mouse&amp;quot;, &amp;quot;Myeloid_mouse&amp;quot;, &amp;quot;Endothelial_mouse&amp;quot;, &amp;quot;Pericytes_mouse&amp;quot;, &amp;quot;Neutrophil_mouse&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm now trying to merge two datasets together but receive an error.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;combined_het &amp;lt;- merge(TME_het_PTX, y = skin_only_ann)&#xA;&#xA;scData &amp;lt;- combined_het&#xA;&#xA;# remove cells with &amp;lt;200 RNA molecules, or &amp;gt;6000 molecules, or &amp;gt;30% mito&#xA;scData &amp;lt;- subset(scData, subset = nFeature_RNA &amp;gt; 200 &amp;amp; nFeature_RNA &amp;lt; 6000 &amp;amp; percent.mt_mouse &amp;lt; 30)&#xA;&#xA;#scData &amp;lt;- NormalizeData(scData, normalization.method = &amp;quot;LogNormalize&amp;quot;, scale.factor = 10000)&#xA;#scData &amp;lt;- FindVariableFeatures(scData, selection.method = &amp;quot;vst&amp;quot;, nfeatures = 2000)&#xA;&#xA;all.genes &amp;lt;- rownames(scData)&#xA;GLOBALS_MAXSIZE &amp;lt;- 4000&#xA;options(future.globals.maxSize = GLOBALS_MAXSIZE * 1024^2)&#xA;scData &amp;lt;- SCTransform(scData, vars.to.regress = &amp;quot;nCount_RNA&amp;quot;, verbose = FALSE)&#xA;#scData &amp;lt;- ScaleData(scData, features = VariableFeatures(object = scData), vars.to.regress = c(&amp;quot;nCount_RNA&amp;quot;))&#xA;scData &amp;lt;- RunPCA(scData, features = VariableFeatures(object = scData))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;combined_het &amp;lt;- merge(TME_PTX, y = skin_only_ann)&#xA;Error in intI(i, n = x@Dim[1], dn[[1]], give.dn = FALSE) : &#xA;  invalid character indexing&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I checked the gene names from each dataset to make sure they're in the same format and they are.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head(rownames(skin_only_ann))&#xA;[1] &amp;quot;Xkr4&amp;quot;    &amp;quot;Gm37180&amp;quot; &amp;quot;Gm37363&amp;quot; &amp;quot;Gm37686&amp;quot; &amp;quot;Gm1992&amp;quot;  &amp;quot;Gm37329&amp;quot;&#xA;head(rownames(TME_PTX))&#xA;[1] &amp;quot;Xkr4&amp;quot;    &amp;quot;Gm37180&amp;quot; &amp;quot;Gm37363&amp;quot; &amp;quot;Gm37686&amp;quot; &amp;quot;Gm38148&amp;quot; &amp;quot;Gm19938&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6827" LastEditorUserId="3694" LastEditDate="2022-04-12T10:57:03.697" LastActivityDate="2022-04-12T10:57:03.697" Title="Subsetting out prefix from genes and merging" Tags="|r|scrnaseq|seurat|merge|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18921" PostTypeId="2" ParentId="18912" CreationDate="2022-04-12T01:51:53.813" Score="0" Body="&lt;p&gt;I don't think you need to complicate the idea of normalisation by introducing machine learning classifiers as a necessary component. Normalisation is common when comparing different datasets for all differential analysis.&lt;/p&gt;&#xA;&lt;p&gt;Given that you have single cell data, have a look at integration techniques in the Seurat workflows. These include normalisation as a component of integration:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://satijalab.org/seurat/articles/integration_rpca.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/articles/integration_rpca.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-04-12T06:03:54.223" LastActivityDate="2022-04-12T06:03:54.223" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18926" PostTypeId="1" AcceptedAnswerId="18927" CreationDate="2022-04-13T18:22:37.133" Score="1" ViewCount="528" Body="&lt;p&gt;&lt;strong&gt;Background:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;We performed NGS using cells collected from mice in a xenotransplantation study.&lt;/p&gt;&#xA;&lt;p&gt;As such, the FASTQ files contain reads of DNA from both mice and human cells.&lt;/p&gt;&#xA;&lt;p&gt;I expect ~30% of reads are contaminated by mice cells.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is it wrong to directly align using the indexed human reference with bowtie2?&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to use two indexes at once with bowtie2? The goal here would be to get the % of reads that map to each genome.&lt;/p&gt;&#xA;" OwnerUserId="7056" LastActivityDate="2022-04-27T07:30:44.697" Title="Bowtie2 mapping with multiple indexes" Tags="|sequence-alignment|bowtie2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18927" PostTypeId="2" ParentId="18926" CreationDate="2022-04-13T18:37:11.680" Score="3" Body="&lt;p&gt;I am not aware of a method using two indices in &lt;code&gt;bowtie2&lt;/code&gt; but here is a simple workaround:&lt;/p&gt;&#xA;&lt;p&gt;Get human reference genome as fasta and suffix all fasta names with &lt;code&gt;_human&lt;/code&gt;. Do the same with the mouse genome using &lt;code&gt;_mouse&lt;/code&gt;. &lt;code&gt;cat&lt;/code&gt; both together and build an index. Then you can later track back whether the alignment was done to human or mouse.&lt;/p&gt;&#xA;&lt;p&gt;Edit: Be aware though that for those parts of the genomes that have high sequence similarity you will get multimapping reads with MAPQ of 0 (or at least very low).&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2022-04-27T07:30:44.697" LastActivityDate="2022-04-27T07:30:44.697" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18928" PostTypeId="1" CreationDate="2022-04-14T14:22:20.847" Score="2" ViewCount="1118" Body="&lt;p&gt;This is a follow up to my question posted here&#xA;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/18850/processing-proteomics-data&quot;&gt;Processing proteomics data&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In relevance to my research, I've been looking for proteomics data (control vs. diabetic) and I found a dataset in the article &amp;quot;Diabetes causes marked inhibition of mitochondrial metabolism in pancreatic β-cells&amp;quot; (figures from here: &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/31171772/&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;I would like to know how to generate the following heat map from the normalized abundances reported.&lt;/p&gt;&#xA;&lt;p&gt;Fold change heatmap:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ETP3m.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ETP3m.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Abundance:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bgGte.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bgGte.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Figure description:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Heat maps of relative mRNA and protein levels of the indicated genes&#xA;in islets isolated from control and 2-week diabetic βV59M mice. Each&#xA;box corresponds to a different animal. Colour indicates log2&#xA;fold-change. Heat map: TCA cycle. Red, proteins upregulated in&#xA;diabetes; blue, proteins downregulated in diabetes. Grey, no change or&#xA;not detected. PDK1, pyruvate dehydrogenase kinase 1. Abundance:&#xA;Abundance of the indicated proteins, quantified by mass spectrometry,&#xA;in islets isolated from control (black, Ctrl, n = 4) and 2-week&#xA;diabetic βV59M (white, Diab, n = 4) mice.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I am not able to understand how the log2 fold-change is computed from the abundance.&#xA;For instance, if we consider &lt;code&gt;PDK1&lt;/code&gt; (first panel) the abundance measure of control is ~50000 and diabetic is ~200000. Heat map says log2 fold-change (in the figure description added above). What is the fold-change that is referred to? From what is shown, I think this is not diabetic/control fold change; we see two panels (i.e one for control (4 samples) and the other for diabetic (4 samples)) and not a single panel. Since the legend is from (-2 to 2) I think these values are not log2 (abundance) as well.&lt;/p&gt;&#xA;&lt;p&gt;Could someone please clarify how the heat map has been generated from the abundance measures? Basically, I want to know how to generate the first heatmap figure from abundance plots.&lt;/p&gt;&#xA;" OwnerUserId="3186" LastActivityDate="2023-02-03T04:23:16.163" Title="Heat map of protein expression from normalized abundance" Tags="|proteins|proteomics|heatmap|mass-spectrometry|meta-analysis|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="18929" PostTypeId="1" CreationDate="2022-04-14T17:26:10.373" Score="3" ViewCount="134" Body="&lt;p&gt;This is a follow up to my previous question &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/18850/processing-proteomics-data&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In relevance to my research, I've been looking for proteomics data (control vs. diabetic) and I found a dataset in the article &amp;quot;Diabetes causes marked inhibition of mitochondrial metabolism in pancreatic β-cells&amp;quot; (figures from here: &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/31171772/&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;).&#xA;I want to process RAW files from this proteomics experiment. I'm trying to set up the analysis in MaxQuant and I am looking for some help in defining some parameters.&lt;/p&gt;&#xA;&lt;p&gt;Sample:&#xA;PRIDE dataset (&lt;a href=&quot;https://www.ebi.ac.uk/pride/archive/projects/PXD012979&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/pride/archive/projects/PXD012979&lt;/a&gt;). I want to analyse&#xA;8 sample files (please find the file labels and appropriate condition labels below).&lt;/p&gt;&#xA;&lt;p&gt;Instrument used to generate the data:&#xA;Orbitrap Fusion Lumos&lt;/p&gt;&#xA;&lt;p&gt;Method:&#xA;Label free quantification&lt;/p&gt;&#xA;&lt;p&gt;Modifications:&lt;/p&gt;&#xA;&lt;p&gt;Groups/condition:&#xA;Diabetic vs. Control&lt;/p&gt;&#xA;&lt;p&gt;Diabetic mice&#xA;FL0408_MSQ911_MariaRohm_554_F30&lt;br /&gt;&#xA;FL0408_MSQ911_MariaRohm_545_F11_544_F33&lt;br /&gt;&#xA;FL0408_MSQ911_MariaRohm_554_M3&lt;br /&gt;&#xA;FL0408_MSQ911_MariaRohm_582_M1&lt;/p&gt;&#xA;&lt;p&gt;Control mice&#xA;FL0408_MSQ911_MariaRohm_544_F13&lt;br /&gt;&#xA;FL0408_MSQ911_MariaRohm_544_F31&lt;br /&gt;&#xA;FL0408_MSQ911_MariaRohm_582_M13&#xA;FL0408_MSQ911_MariaRohm_544_M1&lt;/p&gt;&#xA;&lt;p&gt;Modifications:&#xA;Fixed:&#xA;Oxidation(M)&#xA;Deamidation NQ&lt;/p&gt;&#xA;&lt;p&gt;Variable:&#xA;Carbamidomethyl(C)&lt;/p&gt;&#xA;&lt;p&gt;My objective is to get the values of the normalized abundance (kindly find the figure below from the article associated with this dataset, for 4 control vs. 4 diabetic samples) from MaxQuant analysis.&lt;/p&gt;&#xA;&lt;p&gt;Abundance (shown in the paper):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/eLkI8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eLkI8.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;MaxQuant setup:&#xA;I'm not sure what has to be filled in the fields circled in red. Could someone please have a look?&#xA;Should the fraction be set to 1? I am not sure what has to be filled for the dataset in the link that I shared above.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/GPHj3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GPHj3.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Label min. ratio coun -- should this be 1? I am not sure what this means.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/kFOCF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kFOCF.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Should the following fields be checked?&#xA;&lt;a href=&quot;https://i.stack.imgur.com/99sT9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/99sT9.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3186" LastActivityDate="2022-04-16T17:26:27.450" Title="Setting up MaxQuant parameters for processing proteomics data" Tags="|proteins|proteomics|meta-analysis|maxquant|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18931" PostTypeId="1" AcceptedAnswerId="18935" CreationDate="2022-04-15T13:58:20.680" Score="4" ViewCount="592" Body="&lt;p&gt;PacBio has the concept of IPD (InterPulse Duration) which is the time between two detected consecutive sequences in the raw signal. I have been trying to extract this value in the Nanopore, but I cannot find them in the .fast5 files. But this info must be there:&lt;/p&gt;&#xA;&lt;p&gt;This is from the paper &lt;a href=&quot;https://www.nature.com/articles/s41587-021-01108-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nanopore sequencing technology, bioinformatics, and applications&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;''Each current segment contains multiple measurements, and the corresponding mean, variance, and duration of the current measurements together make up the ‘event’ data.'' Directly quoted from the paper, which suggests we should have the information about the duration in the raw signal.&lt;/p&gt;&#xA;&lt;p&gt;How can I get the duration, I appreciate it if can anybody help?&lt;/p&gt;&#xA;&lt;p&gt;Some info I read was confusing to me (e.g. &lt;a href=&quot;http://simpsonlab.github.io/2015/04/08/eventalign/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://simpsonlab.github.io/2015/04/08/eventalign/&lt;/a&gt; which is nevertheless a great source for understanding what is going on), I knew single measurements' speeds are controlled, however, I didn't know the block stride is fixed and it made me so confused at first.&lt;/p&gt;&#xA;&lt;p&gt;What I found related to my own question:&#xA;I guess I was not completely correct in the terminology I was using, which might cause confusion. In nanopore sequencing, &lt;em&gt;translocation duration&lt;/em&gt; refers to the time a fragment of DNA/RNA passes through the nanopore.&#xA;Also for better visualization, I made this plot after running &lt;em&gt;albacore&lt;/em&gt; basecaller:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/N394i.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/N394i.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;as @gringer has explained, the &lt;em&gt;block stride&lt;/em&gt; value is fixed (=5).&#xA;The figure shows a small portion of albacore Event Table:&#xA;The x-axis corresponds to the relative time from the start of measurements and the y-axis shows the current level.&#xA;Each black dot shows a single measurement from an ONT device; the dots are segmented by a set of block strides (=5) used by albacore to call an event. The red lines show the average of the measurements in one segment. The shaded area in light gray represents the event that resulted in calling a base (one to one). Dark gray areas show events that resulted in&#xA;calling two bases (split events). The white area corresponds to events with no called bases (skipped events). The boxes below the x-axis correspond to bases called in shaded areas and the bold base is the base that is called for that event.&lt;/p&gt;&#xA;" OwnerUserId="15341" LastEditorUserId="15341" LastEditDate="2022-05-18T13:54:46.463" LastActivityDate="2022-05-18T13:54:46.463" Title="How can I get the duration of the events in Nanopore?" Tags="|nanopore|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18932" PostTypeId="1" CreationDate="2022-04-15T15:58:10.223" Score="0" ViewCount="38" Body="&lt;p&gt;Cross-posted &lt;a href=&quot;https://www.biostars.org/p/9519307/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would like to convert MASCOT mztab file to XML format. I have been looking for converters and I came across PRIDE Converter but this is no longer available for download from here &lt;a href=&quot;https://github.com/PRIDE-Toolsuite/pride-converter-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/PRIDE-Toolsuite/pride-converter-2&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Could someone please suggest alternate converters, if available?&lt;/p&gt;&#xA;" OwnerUserId="3186" LastActivityDate="2022-04-16T09:56:36.587" Title="Convert Mascot mztab file format to XML" Tags="|proteomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18933" PostTypeId="2" ParentId="18276" CreationDate="2022-04-15T20:26:30.883" Score="3" Body="&lt;p&gt;NMF of a gene expression matrix into &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; components works to decompose the expression matrix into an activity matrix (&lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt;) and latent factor matrix (&lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$W$&lt;/span&gt; describes the &lt;span class=&quot;math-container&quot;&gt;$k$&lt;/span&gt; latent factors (often called components) by providing a score for how much each gene contributes to each component. &lt;span class=&quot;math-container&quot;&gt;$H$&lt;/span&gt; describes how much each component contributes to the expression of each cell.&lt;/p&gt;&#xA;&lt;p&gt;To extract the activity and component matrices from your NMF model fit using &lt;code&gt;RcppML&lt;/code&gt;, use the following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# cells is a Seurat object&#xA;model &amp;lt;- RcppML::nmf(cells@assays&lt;span class=&quot;math-container&quot;&gt;$RNA@data, k = 5, verbose = F, seed = 1234)&#xA;W &amp;lt;- model$&lt;/span&gt;w # Components Matrix&#xA;H &amp;lt;- model$h # Activity Matrix&#xA;&#xA;# Name genes/components for latent factor matrix&#xA;colnames(W) &amp;lt;- paste0(&amp;quot;component&amp;quot;, 1:k)&#xA;rownames(W) &amp;lt;- rownames(cells) # Gene names&#xA;&#xA;# Name components/cells for activity matrix&#xA;colnames(H) &amp;lt;- colnames(cells) # Cell IDs&#xA;row names(H) &amp;lt;- paste0(&amp;quot;component&amp;quot;, 1:k)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(W)&#xA;              component1   component2   component3 component4   component5&#xA;MIR1302-2HG 0.000000e+00 0.000000e+00 0.000000e+00          0 0.000000e+00&#xA;FAM138A     0.000000e+00 0.000000e+00 0.000000e+00          0 0.000000e+00&#xA;OR4F5       0.000000e+00 0.000000e+00 0.000000e+00          0 0.000000e+00&#xA;AL627309.1  2.927886e-07 9.248074e-07 7.961105e-07          0 2.802758e-06&#xA;AL627309.3  3.486284e-08 0.000000e+00 1.916799e-07          0 7.938089e-08&#xA;AL627309.2  0.000000e+00 0.000000e+00 0.000000e+00          0 0.000000e+00&#xA;&#xA;&amp;gt; head(H)&#xA;                 cell_1       cell_2       cell_3       cell_4       cell_5&#xA;component1 0.0001370465 0.0002195795 0.0001628898 0.0002239623 0.0001484157&#xA;component2 0.0003604048 0.0002239471 0.0005688147 0.0005242678 0.0004821531&#xA;component3 0.0003948121 0.0001485325 0.0002885584 0.0002242272 0.0002865587&#xA;component4 0.0001391838 0.0002418495 0.0001361317 0.0001382889 0.0001558255&#xA;component5 0.0000000000 0.0000000000 0.0000000000 0.0000000000 0.0000000000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;People will often use the top &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; genes to describe a component. The following code gets the top &lt;span class=&quot;math-container&quot;&gt;$N$&lt;/span&gt; genes for each component and stores them in a list&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# This code assumes the above code block has been already run&#xA;require(dplyr) #used for %&amp;gt;% piping&#xA;N &amp;lt;- 100&#xA;top_genes &amp;lt;- list()&#xA;for (i in 1:dim(W)[2]) {&#xA;  name &amp;lt;- colnames(W)[i]&#xA;  top_genes[[name]] &amp;lt;- sort(W[, i], decreasing = T) %&amp;gt;% head(N) %&amp;gt;% names()&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6674" LastEditorUserId="6674" LastEditDate="2022-04-15T20:34:36.113" LastActivityDate="2022-04-15T20:34:36.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18934" PostTypeId="1" AcceptedAnswerId="19055" CreationDate="2022-04-15T22:37:44.473" Score="4" ViewCount="77" Body="&lt;p&gt;I'm part of a nanopore sequencing experiment that will sequence several generations of viruses. The intent is to perform directed evolution by putting selective pressure on these viruses and tracking the various mutations that occur. See &lt;a href=&quot;https://www.cell.com/cell/fulltext/S0092-8674(19)30622-1?_returnURL=https%3A%2F%2Flinkinghub.elsevier.com%2Fretrieve%2Fpii%2FS0092867419306221%3Fshowall%3Dtrue&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for the paper this methodology is based on.&lt;/p&gt;&#xA;&lt;p&gt;I've been working on a software package that can compile consensus sequences from longread nanopore sequencing (see &lt;a href=&quot;https://github.com/JGEnglishLab/longread_umi_python&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). However, there's a wrinkle with generating phylogenetic trees from this type of data. It appears that most tree building software ignores indels. This makes sense, as gaps could easily indicate a lack of coverage from illumina sequencing methods or similar issues rather than betray the existence of an insertion/deletion. However, the nature of our sequencing method is such that a gaps really should be evaluated as mutational changes that we want to keep track of in a phylogenetic tree.&lt;/p&gt;&#xA;&lt;p&gt;We've explored using NextStrain to generate our trees and visuals, but the tree building algorithms they employ too suffer from this gap assumption (see the issue I raised &lt;a href=&quot;https://github.com/nextstrain/auspice/issues/1500&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). These include IQTree, RAxML and PhyML. I have not had much luck finding an algorithm or software that treats gaps as indels when building a phylogenetic tree. Does anyone know of such an algorithm or package I could employ? I figure I could take a stab at writing my own algorithm, but consider that a Plan B resort. No sense re-inventing the wheel and all that.&lt;/p&gt;&#xA;" OwnerUserId="15344" LastActivityDate="2022-05-11T14:19:58.827" Title="Tree Building Algorithm that treats gaps as deletions" Tags="|phylogenetics|nanopore|long-reads|indel|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18935" PostTypeId="2" ParentId="18931" CreationDate="2022-04-16T04:16:51.063" Score="4" Body="&lt;p&gt;Unfortunately ONT no longer produce event data as default in their FAST5 output with the most recent basecallers, and they've been quite vague about how to generate it.&lt;/p&gt;&#xA;&lt;p&gt;The easiest way to fix that is to use an older basecaller. If you tell Guppy v5.1.15 to output FAST5 files, it should include an event table that includes read start time and move information. This might also work with new Guppy, but ONT has told us that the event tables have gone away.&lt;/p&gt;&#xA;&lt;p&gt;Processing the move tables to get information about signals per base is a bit complicated. Just using the metadata provided in the FAST5 files it's actually impossible, but you can get close enough to match what the ONT basecallers were trained on (which is probably close enough). To get closer than that, you need to process the raw signal yourself (e.g. see &lt;a href=&quot;https://twitter.com/gringene_bio/status/1460819769477660681&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, you might be able to get that information out by following the tomboy documentation, which produces output that demonstrates signal-level changes:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/nanoporetech/tombo&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nanoporetech/tombo&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;ONT have had various methods for basecalling over the years, and these methods change how the signal is chunked and processed.&lt;/p&gt;&#xA;&lt;p&gt;When nanopore sequencing was first commercially available, base change points &lt;em&gt;were&lt;/em&gt; resolved  down to a single-sample level (as described in &lt;a href=&quot;http://simpsonlab.github.io/2015/04/08/eventalign/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Jared Simpson's post&lt;/a&gt; which you linked). Or, at least, as accurately as they could at the time. Even with that approach, events were occasionally misclassified such that two consecutive events could represent the same base, or two consecutive events could involve a change of more than one base. To account for this, ONT created the &lt;em&gt;move table&lt;/em&gt;, which described how many bases were shifted between one base and the next base. Initially ONT were using pentamers (i.e. 5 base sequences) for the output of their calling models (this had a physical reason, because 5 bases was about as many as would contribute to the instantaneous current), so the move table could have values of 0 [indicating two adjacent events described the same base], 1, 2, 3, 4, or 5. Although 5 was the maximum, because it was a pentamer model, what it actually meant was &amp;quot;there has been a large skip; we have no idea how large.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, sample-level signal parsing is no longer the case, which makes precise event alignment tricky. Most recently, individual signal samples are aggregated into what are called &lt;em&gt;strides&lt;/em&gt;, which describe the smallest length of time that the basecaller accepts as representing a single base transition.&lt;/p&gt;&#xA;&lt;p&gt;For this explanation, I'm going to assume that the directory location in the FAST5 file starts from &lt;code&gt;/baseLoc/&lt;/code&gt;, with the FASTQ sequence found in &lt;code&gt;/baseLoc/Fastq&lt;/code&gt;. This explanation may become quickly out of date, as ONT change their data and methods frequently.&lt;/p&gt;&#xA;&lt;p&gt;Raw signal information is found at &lt;code&gt;/baseLoc/../Raw/Signal&lt;/code&gt;. The stride length information is found in the &lt;code&gt;block_stride&lt;/code&gt; attribute of &lt;code&gt;/baseLoc/Summary/basecall_1d_template&lt;/code&gt;. This indicates the number of signal samples allocated for each stride. The move table is found at &lt;code&gt;/baseLoc/Move&lt;/code&gt;, and represents the number of base shifts in each stride block (usually either 0 or 1). &lt;em&gt;&lt;strong&gt;It sounds like that move table is the statistic you're interested in&lt;/strong&gt;&lt;/em&gt;; multiply the number of 0s before a 1 by the number of samples per stride (plus 1 for the base transition), and that's the base duration in samples, which should be divided by the sampling rate to get the number of seconds per base. Sampling rate can be found in the &lt;code&gt;sampling_rate&lt;/code&gt; attribute of &lt;code&gt;/???/channel_id&lt;/code&gt; (sorry, don't have a precise location, because I usually just assume 4 kHz... which will probably be changing shortly, from the sound of it).&lt;/p&gt;&#xA;&lt;p&gt;Some of the signal at the start of the read may have been clipped, and it's necessary to take this into account in order to properly align the move table to the signal. The signal offset is found in the attribute &lt;code&gt;first_sample_template&lt;/code&gt; for &lt;code&gt;/baseLoc/../../Segmentation_000/Summary/segmentation&lt;/code&gt;, and represents the number of &lt;em&gt;samples&lt;/em&gt; (not strides) that should be clipped off the start of the raw signal. There's one more thing that may help for aligning bases to signal, and that's the &lt;em&gt;base&lt;/em&gt; offset relative to the move table. The basecaller works in chunks of multiple bases (kmers), so the base corresponding to the move table is actually somewhere in the middle of the called/modeled kmer. ONT doesn't provide any insight into this (it's possible they don't know), so I just try some offsets between 1-5 bases and pick the one that looks the best.&lt;/p&gt;&#xA;&lt;p&gt;I have a FAST5 annotation R script that may help a little bit more in understanding how the alignment works, see &lt;a href=&quot;https://gitlab.com/gringer/bioinfscripts/-/blob/master/read_annotator.r&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, specifically the code starting from the comment &lt;code&gt;## Load data from called fast5 file&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Bear in mind that &amp;quot;number of samples per base&amp;quot; varies in a non-smooth fashion with quite a large dynamic range within an individual read sequence, and could be anything from 1 to over 50 (although &lt;a href=&quot;https://twitter.com/gringene_bio/status/1204260580510453760&quot; rel=&quot;nofollow noreferrer&quot;&gt;I stop believing&lt;/a&gt; the basecaller if it reports anything over about 20). It doesn't make much sense calculating it as an aggregate statistic for each sequence, except possibly for detecting really poor basecalling (which is more easily picked up by looking at Q scores in the FASTQ file).&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-04-17T22:03:22.103" LastActivityDate="2022-04-17T22:03:22.103" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18936" PostTypeId="1" CreationDate="2022-04-16T09:53:40.020" Score="0" ViewCount="46" Body="&lt;p&gt;Over the past few months I have been retrospectively analyzing patient files for my master's thesis in medicine. As the median overall survival of my cohort is low, I have decided that I want to analyze the 2-years survival of my population. However, I have not been able to find any literature on how to perform this analysis on SPSS.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone here have any experience analyzing the median survival over a specific period of time (i.e., 1-year, 2-years or 5-years) using IBM SPSS?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;&lt;p&gt;Leon&lt;/p&gt;&#xA;" OwnerUserId="15347" LastActivityDate="2022-05-16T16:03:25.277" Title="IBM SPSS- 2-years survival of cohort" Tags="|statistics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18937" PostTypeId="2" ParentId="18932" CreationDate="2022-04-16T09:56:36.587" Score="1" Body="&lt;p&gt;There are a number of converters to be found at &lt;a href=&quot;http://www.pastelbioscience.co.uk/resources/databases.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.pastelbioscience.co.uk/resources/databases.html&lt;/a&gt; , although not sure that the one you are looking for is among them. May also be better to ask this question on twitter. There is a large #proteomics community that are very helpful and usually come up with an answer :-)&lt;/p&gt;&#xA;" OwnerUserId="15348" LastActivityDate="2022-04-16T09:56:36.587" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18938" PostTypeId="1" CreationDate="2022-04-16T10:18:16.927" Score="4" ViewCount="503" Body="&lt;p&gt;I’m currently using QCTOOL v2 to process imputed .bgen files from UK Biobank, however they seem to be processing very slowly. Is this normal?&lt;/p&gt;&#xA;&lt;p&gt;My command is pretty basic; I’m filtering out a list of SNPs and samples:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/path_to/qctool \&#xA;-g /path_to/ukbXXXX_c22_b0_v3.bgen \&#xA;-s /path_to/ukbXXXX_c22_b0_v3.sample \&#xA;-og /output_path/ukb_c22_filt.bgen \&#xA;-os /output_path/ukb_c22_filt.sample \&#xA;-excl-rsids /path_to/snps_rem_c22.txt \&#xA;-excl-samples /path_to/samples_to_rem.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It is currently processing SNPs at a rate of &lt;strong&gt;1.2/s&lt;/strong&gt; (e.g. 71169/?,57205.8s,1.2/s).&lt;/p&gt;&#xA;&lt;p&gt;The computational facility I'm using should not limit the speed of an operation.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Have I made any mistakes in my qctool query?&lt;/li&gt;&#xA;&lt;li&gt;Or any other ideas on why it's running so slowly?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Alternative suggestions on how to process these files would be appreciated for example, although I would prefer to use QCTOOL perhaps I have to use PLINK.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;From the website, &lt;a href=&quot;https://www.well.ox.ac.uk/%7Egav/qctool_v2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.well.ox.ac.uk/~gav/qctool_v2/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;QCTOOL is a command-line utility program for manipulation and quality control of gwas datasets and other genome-wide data&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="15349" LastEditorUserId="3848" LastEditDate="2022-04-16T10:37:39.157" LastActivityDate="2022-11-03T12:00:20.093" Title="Using QCTOOL v2 to process UK Biobank .bgen files - why so slow?" Tags="|plink|qc|ld|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18939" PostTypeId="2" ParentId="18936" CreationDate="2022-04-16T10:28:53.403" Score="1" Body="&lt;p&gt;What you want is Cox regression (proportional hazard) models to perform &lt;strong&gt;survival analysis&lt;/strong&gt;. It is years since I have used SPSS and would encourage consideration for the use of R (Python is a bit more complicated). There is a commercial go-between which is easier to use than R but more computationally efficient than SPSS, but I cannot remember its name at all.&lt;/p&gt;&#xA;&lt;p&gt;There is a weblink for using Cox regression in SPSS:  &lt;a href=&quot;https://www.scalestatistics.com/cox-regression.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.scalestatistics.com/cox-regression.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have no idea whether the information is correct, but it looks simple enough to point and click.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&#xA;A  proportional hazard model is one of numerous survival analysis which model unit risk, e.g. mortality, over time considering the risk e.g. mortality of the baseline population (covariates). Generally a hazard model will comprise a hazard function of the risk of events per unit time against the baseline covariates, such as age, smoking that sort of thing. Thus you wouldn't want to consider the mortality risk of a 25 year old to comparable to that of a 90 year old patient for a given factor, because there's loads of confounders.&lt;/p&gt;&#xA;&lt;p&gt;At a minimum SPSS could be slow and this is a regression analysis, but you will find out soon enough.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;It appears there's a forward prediction being sought here. A simple regression may be able to capture this, but the gist of the question appears to be a precise temporal forward prediction. Precise forward prediction are complicated and whilst there are definitely R packages that do this, at a precision level of forecast e.g. 2 vs 5 year this definitely ain't SPSS, a long way from its capabilities. The OP will need to learn R to do this study, it isn't trivial, or approximate from a hazard style regression. For this style of calculation significant epidemiological and clinical information is required. For example, these are often used in cancer studies, but precisely what the background epi/clinical is will be important as to whether a given R library is relevant.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; The answer to the question is SPSS can't do this and learning R is the way forward, because the OP will need to use an established library and this only place I'm aware that such libraries exist. The only alternative is to approximate using classical survival analysis.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-04-16T15:18:31.677" LastActivityDate="2022-04-16T15:18:31.677" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18940" PostTypeId="1" CreationDate="2022-04-16T13:48:19.737" Score="2" ViewCount="268" Body="&lt;p&gt;Currently merged two Seurat objects together and then ran Harmony for batch correction. Now I want to subset out a cell type of interest do I re run harmony or just do the standard PCA?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;scData &amp;lt;- scData %&amp;gt;% &#xA;    RunUMAP(reduction = &amp;quot;harmony&amp;quot;, dims = 1:14) %&amp;gt;% &#xA;    FindNeighbors(reduction = &amp;quot;harmony&amp;quot;, dims = 1:14) %&amp;gt;% &#xA;    FindClusters(resolution = 0.2) %&amp;gt;% &#xA;    identity()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;numPC = 14&#xA;scData &amp;lt;- FindNeighbors(scData, dims = 1:numPC)&#xA;scData &amp;lt;- FindClusters(scData, resolution = 0.2, dims = 1:numPC)&#xA;scData &amp;lt;- RunUMAP(scData, dims = 1:numPC)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Both options give me different results. Not sure what the correct way to it is.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;From the Broad Institute website:&#xA;&lt;a href=&quot;https://portals.broadinstitute.org/harmony/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://portals.broadinstitute.org/harmony/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Harmony is a general-purpose R package with an efficient algorithm for integrating multiple data sets. It is especially useful for large single-cell datasets such as single-cell RNA-seq.&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="6827" LastEditorUserId="3848" LastEditDate="2022-04-16T14:25:57.520" LastActivityDate="2022-04-16T14:25:57.520" Title="Subsetting with harmony" Tags="|r|seurat|single-cell|batch-effects|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18941" PostTypeId="2" ParentId="14667" CreationDate="2022-04-16T14:39:35.190" Score="1" Body="&lt;p&gt;I would suggest searching with terms such as &amp;quot;spatial transcriptomic&amp;quot; or &amp;quot;spatial proteomics&amp;quot;. You could also search for platform-specific data such as &amp;quot;CyCIF&amp;quot;, &amp;quot;CODEX&amp;quot;, &amp;quot;MIBI&amp;quot;, &amp;quot;slideseq&amp;quot;, &amp;quot;visium&amp;quot; etc..&lt;/p&gt;&#xA;&lt;p&gt;Here is a website that has curated some spatial data (&lt;a href=&quot;http://www.spatialomics.org/SpatialDB/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.spatialomics.org/SpatialDB/&lt;/a&gt;). We also recently released about &lt;a href=&quot;https://aacrjournals.org/cancerdiscovery/article/doi/10.1158/2159-8290.CD-21-1357/694189/The-spatial-landscape-of-progression-and&quot; rel=&quot;nofollow noreferrer&quot;&gt;3TB of spatial imaging data&lt;/a&gt; that might be helpful. Unlike transcriptomic data that is often submitted in GEO (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/&lt;/a&gt;) there is still no one place where all of the spatial data is deposited and so you will find them only as supplemental data in published manuscripts. Hope that helps.&lt;/p&gt;&#xA;" OwnerUserId="15353" LastActivityDate="2022-04-16T14:39:35.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18942" PostTypeId="1" AcceptedAnswerId="18944" CreationDate="2022-04-16T21:15:53.170" Score="0" ViewCount="129" Body="&lt;p&gt;I have a very old graph as a .jpeg picture that I would like to convert into a dataset to be able to operate with it using R/python.&#xA;I have no idea if it is even possible. &lt;a href=&quot;https://i.stack.imgur.com/lCgCk.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lCgCk.jpg&quot; alt=&quot;old_graph&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6470" LastActivityDate="2022-04-17T10:27:14.930" Title="Is there a software that allows to convert a jpeg graph into dataset?" Tags="|software-recommendation|graphs|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18944" PostTypeId="2" ParentId="18942" CreationDate="2022-04-17T10:27:14.930" Score="-2" Body="&lt;p&gt;Looks like &lt;strong&gt;Graph Grabber&lt;/strong&gt; can do the job.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.quintessa.org/software/downloads-and-demos/graph-grabber-2.0.2#download&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.quintessa.org/software/downloads-and-demos/graph-grabber-2.0.2#download&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6470" LastActivityDate="2022-04-17T10:27:14.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18945" PostTypeId="1" AcceptedAnswerId="18947" CreationDate="2022-04-18T10:36:23.857" Score="4" ViewCount="11760" Body="&lt;p&gt;I am trying to plot a box plot with &lt;a href=&quot;https://seaborn.pydata.org/generated/seaborn.boxplot.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;seaborn&lt;/a&gt; with the following code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plot = sns.boxplot(&#xA;    y='Min PPI distance',&#xA;    x='Synergy_percent',&#xA;    color='white',&#xA;    orient=&amp;quot;h&amp;quot;,&#xA;    data=synergy_df&#xA;)&#xA;plot.set(&#xA;    xlabel='Synergy (%)',&#xA;    ylabel='Min. distance',&#xA;);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The data &lt;code&gt;synergy_df&lt;/code&gt; looks like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    Combination Min PPI distance    Synergy_percent&#xA;0   1003- Camptothecin &amp;amp; 1011- Navitoclax   2.0 22.2&#xA;1   1003- Camptothecin &amp;amp; 1017- Olaparib 2.0 6.4&#xA;2   1003- Camptothecin &amp;amp; 1022- AZD7762  2.0 40.4&#xA;3   1003- Camptothecin &amp;amp; 1025- SB216763 3.0 2.1&#xA;4   1003- Camptothecin &amp;amp; 1032- Afatinib 3.0 2.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After running the above code I am getting the error&#xA;&lt;code&gt;TypeError: Horizontal orientation requires numeric `x` variable.&lt;/code&gt;&#xA;I want to add that Synergy_percent column has many zero values and about 1500 rows; I haven't tried it with a smaller number of rows.&lt;/p&gt;&#xA;&lt;p&gt;I think the reason for the error is the wide range in the synergy_percent values. The range is between 0 to 86.7. If I use xlim here what range should I use to divide the x-axis in 4 quarters (0-25-50-100)?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; Seaborn is a Python library for more complex graph plotting than the commonly used &lt;code&gt;matplotlib&lt;/code&gt; and is often used for boxplots and heatmaps usually using a &lt;code&gt;pandas&lt;/code&gt; dataframe. The official documents are described &lt;a href=&quot;https://seaborn.pydata.org/generated/seaborn.boxplot.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4783" LastEditorUserId="1157" LastEditDate="2022-07-11T16:58:15.547" LastActivityDate="2022-07-11T16:58:15.547" Title="Error in seaborn plot &quot; Horizontal orientation requires numeric `x` variable&quot;" Tags="|visualization|python|graphs|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18946" PostTypeId="2" ParentId="18945" CreationDate="2022-04-18T12:55:04.120" Score="1" Body="&lt;p&gt;I'm not familiar with seaborn, but I'd guess from the error that there's at least one text value inside the &lt;code&gt;Synergy_percent&lt;/code&gt; field that is encouraging the rest of the values to text. Either that, or spaces in the input file are messing up the parsing of the columns.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-04-18T12:55:04.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18947" PostTypeId="2" ParentId="18945" CreationDate="2022-04-18T14:03:44.603" Score="3" Body="&lt;p&gt;Very quickly because I'm not around.&lt;/p&gt;&#xA;&lt;p&gt;I think I know what's going on, but don't have enough time to figure the df. The key thing is &lt;code&gt;seaborn&lt;/code&gt; needs to know where columns are &lt;code&gt;x&lt;/code&gt; and which are &lt;code&gt;y&lt;/code&gt; ... &lt;em&gt;within the data frame&lt;/em&gt; . This information appears missing and the OP used &lt;code&gt;y='title', x='title&lt;/code&gt; without reference to the dataframe. I'm pretty certain thats the bug here.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import seaborn as sns&#xA;import matplotlib.pyplot as plt&#xA;&#xA;plot = sns.boxplot(&#xA;    color='white',&#xA;    y=synergy_df[&amp;quot;Synergy_percent&amp;quot;], &#xA;    x=synergy_df[&amp;quot;Min PPI distance&amp;quot;] &#xA;);&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alternative, which seems more what the OP wants ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import seaborn as sns&#xA;import matplotlib.pyplot as plt&#xA;&#xA;plot = sns.boxplot(&#xA;    color='white',&#xA;    # use your orientation command here&#xA;    x=synergy_df[&amp;quot;Synergy_percent&amp;quot;], &#xA;    y=synergy_df[&amp;quot;Min PPI distance&amp;quot;] &#xA;);&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Blurb&lt;/strong&gt; Normally the x-axis is the discrete boundary whilst the y-axis is continuous. In you example the x-axis feels like the continuous variable and you are producing a rotated boxplot - thats perfectly fine and seabourn will do that however its  good to get it working first before rotating the axes.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Final&lt;/strong&gt; The orientation of the dataframe might possibly be an issue, not sure unless its tried.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;If there's a formatting issue the only trick I know is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; plot = sis.boxplot(&#xA; .... # same old&#xA; dodge=True,&#xA; );&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I don't know beyond, e.g. imposing percentile ranges onto a boxplot.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-04-18T14:33:04.663" LastActivityDate="2022-04-18T14:33:04.663" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18948" PostTypeId="1" CreationDate="2022-04-18T14:23:57.883" Score="0" ViewCount="184" Body="&lt;p&gt;I want to do pathway enrichment analysis in python using &lt;code&gt;scipy.stats.hypergeom.sf function&lt;/code&gt; to select pathways that are significantly enriched in at least one tissue.&lt;/p&gt;&#xA;&lt;p&gt;my data:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Enrichment  CANCER_TYPE PATHWAY&#xA;11.6    Breast Carcinoma    Cell cycle/DNA replication&#xA;2   Breast Carcinoma    Protein stability and degradation/DNA replication&#xA;2.8 Breast Carcinoma    EGFR signaling/DNA replication&#xA;4   Breast Carcinoma    p53 pathway/DNA replication&#xA;0   Breast Carcinoma    PI3K/MTOR signaling/DNA replication&#xA;1.2 Pancreatic Carcinoma    Mitosis/RTK signaling&#xA;3.3 Pancreatic Carcinoma    EGFR signaling/RTK signaling&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4845" LastActivityDate="2022-04-18T14:23:57.883" Title="How to do enrichment analysis with scipy.stats.hypergeom.sf function" Tags="|python|statistics|scipy|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18949" PostTypeId="1" CreationDate="2022-04-18T23:38:06.010" Score="2" ViewCount="604" Body="&lt;p&gt;In Nextflow I have a process that creates a channel that holds several files in a tuple. I'm trying to call only one of those files as input for a subsequent process.&lt;/p&gt;&#xA;&lt;p&gt;So basically if I run &lt;code&gt;channel_name.view()&lt;/code&gt; the channel shows &lt;code&gt;[/home/nextflow/work/e4/bc47bceb4569cffc01bd76e49245e9/RAxML_bestTree.tree, /home/nextflow/work/e4/bc47bceb4569cffc01bd76e49245e9/RAxML_bipartitions.tree, /home/nextflow/work/e4/bc47bceb4569cffc01bd76e49245e9/RAxML_bipartitionsBranchLabels.tree, /home/nextflow/work/e4/bc47bceb4569cffc01bd76e49245e9/RAxML_bootstrap.tree, /home/nextflow/work/e4/bc47bceb4569cffc01bd76e49245e9/RAxML_info.tree] &lt;/code&gt;. I need just one file so I'm using the full name in the &lt;code&gt;input&lt;/code&gt; section of the process.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;input:&#xA;file 'RAxML_bipartitionsBranchLabels.tree' from channel_name&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That does not find the file. On the other hand, if I use &lt;code&gt;.flatten()&lt;/code&gt;, like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;input:&#xA;file 'RAxML_bipartitionsBranchLabels.tree' from raxml_out_ch.flatten()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It works but reads some files before reaching to the one that matches and so the process results in error even after completing the task for this file properly and gives several outputs (for each file it looks into), some of them empty and one with the result.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[7c/2971f1] Submitted process &amp;gt; tree_visualisation (4)&#xA;[8d/dd129b] Submitted process &amp;gt; tree_visualisation (1)&#xA;[cb/0f91f5] Submitted process &amp;gt; tree_visualisation (5)&#xA;[6c/fb23c8] Submitted process &amp;gt; tree_visualisation (3)&#xA;ERROR ~ Error executing process &amp;gt; 'tree_visualisation (5)'&#xA;&#xA;Caused by:&#xA;  Process `tree_visualisation (5)` terminated with an error exit status (1)&#xA;&#xA;Command executed:&#xA;&#xA;  Rscript tree.R RAxML_bipartitionsBranchLabels.tree phylogram.png&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm looking for a way to filter this input so that the process only runs for this file and doesn't try with the others.&lt;/p&gt;&#xA;&lt;p&gt;How can I do that?&lt;/p&gt;&#xA;" OwnerUserId="1363" LastEditorUserId="1363" LastEditDate="2022-04-19T15:49:04.127" LastActivityDate="2022-04-20T02:46:47.447" Title="How can I call one file from a channel that's composed of several files in Nextflow?" Tags="|nextflow|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18950" PostTypeId="1" AcceptedAnswerId="18973" CreationDate="2022-04-19T00:35:17.083" Score="3" ViewCount="62" Body="&lt;p&gt;How would one parse the SMILES string&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; BrC[2]:C[3]:C(:CH:CH:CH:@2):CH:CH:CH:CH:@3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I rely on tools like &lt;code&gt;rdkit&lt;/code&gt; and &lt;code&gt;OpenBabel&lt;/code&gt; to parse SMILES, but both tools aren't able to parse this string.&lt;/p&gt;&#xA;&lt;p&gt;More specifically, this SMILES string comes from the supplementary information of a &lt;a href=&quot;https://pubs.acs.org/doi/10.1021/ci700096r&quot; rel=&quot;nofollow noreferrer&quot;&gt;2007 paper on solubility prediction&lt;/a&gt;. After doing some searching, it seems that this paper was received prior to the 2007 standardization of OpenSMILES (this might be why canonical parsers like &lt;code&gt;rdkit&lt;/code&gt; have trouble with it).&lt;/p&gt;&#xA;&lt;p&gt;After searching a bit more, I found an &lt;a href=&quot;https://pubchem.ncbi.nlm.nih.gov//edit3/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;older PubChem sketcher&lt;/a&gt; that was able to convert the SMILES in question into a more recognizable format&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; (BrC1=C(C(=C(C2=C1C(=C(C(=C2[H])[H])[H])[H])[H])[H])[H])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(parsing this was successful via &lt;code&gt;rdkit&lt;/code&gt;, see molecule image below) but still need a way to convert similarly-formatted SMILES strings into the OpenSMILES standard/canonical format en mass.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/j4TC9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/j4TC9.png&quot; alt=&quot;image for BrC1=C(C(=C(C2=C1C(=C(C(=C2[H])[H])[H])[H])[H])[H])[H]&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15359" LastEditorUserId="15359" LastEditDate="2022-04-19T01:58:00.293" LastActivityDate="2022-04-27T13:21:35.187" Title="Parsing pre-2007 SMILES string" Tags="|perl|computational-biochemistry|rdkit|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="18951" PostTypeId="1" CreationDate="2022-04-19T06:22:08.263" Score="1" ViewCount="59" Body="&lt;p&gt;I want to analyze the RMSD of my trajectory files using MdTraj.&lt;/p&gt;&#xA;&lt;p&gt;I used two different codes for this purpose:&lt;/p&gt;&#xA;&lt;p&gt;the first code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rep = 1&#xA;traj_name = &amp;quot;step7_%d.dcd&amp;quot; %rep&#xA;rmsd_total = []&#xA;&#xA;while (os.path.isfile(traj_name)):&#xA;    t = md.load(traj_name, top = 'step5_input.pdb')&#xA;    top = t.topology &#xA;    protein_sel = top.select('backbone')&#xA;    protein_traj = t.atom_slice(protein_sel)&#xA;    rmsd = md.rmsd(protein_traj, protein_traj, 0)    &#xA;    rmsd_without_zero = [i for i in rmsd if i != 0.0]&#xA;    rmsd_total.extend(rmsd_without_zero)&#xA;&#xA;    rep+=1&#xA;    traj_name = &amp;quot;step7_%d.dcd&amp;quot; %rep&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and the second code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;traj_1 = md.load('step7_1.dcd', top='step5_input.pdb')&#xA;traj_2 = md.load('step7_2.dcd', top='step5_input.pdb')&#xA;traj_3 = md.load('step7_3.dcd', top='step5_input.pdb')&#xA;traj_4 = md.load('step7_4.dcd', top='step5_input.pdb')&#xA;traj_5 = md.load('step7_5.dcd', top='step5_input.pdb')&#xA;traj_6 = md.load('step7_6.dcd', top='step5_input.pdb')&#xA;traj_7 = md.load('step7_7.dcd', top='step5_input.pdb')&#xA;traj_8 = md.load('step7_8.dcd', top='step5_input.pdb')&#xA;traj_9 = md.load('step7_9.dcd', top='step5_input.pdb')&#xA;traj_10 = md.load('step7_10.dcd', top='step5_input.pdb')&#xA;traj_11 = md.load('step7_11.dcd', top='step5_input.pdb')&#xA;traj_12 = md.load('step7_12.dcd', top='step5_input.pdb')&#xA;traj_13 = md.load('step7_13.dcd', top='step5_input.pdb')&#xA;traj_14 = md.load('step7_14.dcd', top='step5_input.pdb')&#xA;traj_15 = md.load('step7_15.dcd', top='step5_input.pdb')&#xA;traj_16 = md.load('step7_16.dcd', top='step5_input.pdb')&#xA;traj_17 = md.load('step7_17.dcd', top='step5_input.pdb')&#xA;traj_18 = md.load('step7_18.dcd', top = 'step5_input.pdb')&#xA;traj_19 = md.load('step7_19.dcd', top = 'step5_input.pdb')&#xA;traj_20 = md.load('step7_20.dcd', top = 'step5_input.pdb')&#xA;&#xA;combine = traj_1 + traj_2 + traj_3 + traj_4 + traj_5 + traj_6 + traj_7 + traj_8 + traj_9 + traj_10 + traj_11 + traj_12 + traj_13 + traj_14 + traj_15 + traj_16 + traj_17 + traj_18 + traj_19 + traj_20&#xA;&#xA;protein_sel = combine.topology.select('backbone')&#xA;protein_traj = combine.atom_slice(protein_sel)&#xA;rmsd = md.rmsd(protein_traj, protein_traj, 0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I believe these two codes should result in the same RMSD. However, I get two different RMSD diagrams out of these two codes?&lt;/p&gt;&#xA;&lt;p&gt;Can anyone help me what is the problems with my first code? (since I think the second code is more accurate while is more naive)&lt;/p&gt;&#xA;" OwnerUserId="15362" LastActivityDate="2023-09-11T11:06:55.800" Title="different results of RMSD for the same trajectory file" Tags="|python|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18952" PostTypeId="2" ParentId="18951" CreationDate="2022-04-19T08:46:52.277" Score="1" Body="&lt;p&gt;It's a reference issue. The second term.&lt;/p&gt;&#xA;&lt;p&gt;Your first block makes a list of numpy arrays which are the RMSD against frame  0 of each of the stacks, while in your second you make a concatenation of frames, so your reference frame 0 is common.&lt;/p&gt;&#xA;&lt;p&gt;The first code block could be smartened by splitting (abstracting) the file valid name enumeration and the reading/analysis. Namely, generate a list of valid filenames and &lt;em&gt;then&lt;/em&gt; analyse. This separation would allow you recycle more easily the code: in the seocnd code block you have the file list problem again, which is instead resolved by a copypaste job —very inflexible. A one liner list comprehension (or, less so, a &lt;code&gt;functools.reduce&lt;/code&gt;) is a nice option.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2022-04-19T08:46:52.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18953" PostTypeId="1" CreationDate="2022-04-19T17:59:57.563" Score="2" ViewCount="11" Body="&lt;p&gt;i am new to modelling of cellular networks, and I have two basic questions, regarding the result interpration.&lt;/p&gt;&#xA;&lt;p&gt;I am using the COBRAToolbox to do a flux balance analysis of a whole genome scale model of a bacteria. To get things started i replicated the results published in a paper, I have even been in correspondence with the author of the paper which happens to be the person who created the model i am using. So I am 100% sure that all the boundary conditions exchange fluxes and so on are as in the publication. I was able to reproduce this results and they match the experimental data.&lt;/p&gt;&#xA;&lt;p&gt;The published results specifically mention the flux of a metabolite X of interest and compare the reaction rate of X to literature values and to laboratory experiments.&lt;/p&gt;&#xA;&lt;p&gt;I did a FVA of X given all the boundaries, the soultion space is between 0 - 15 [mmol/g_dw/h], and the result when doing the FBA is aorund 7 [mmol/g_dw/h]?&#xA;(I tried out multiple solvers, showing the same variation)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My questions:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I understood the FVA as a analysis tool which shows me the possible solution space for a given flux , given a system with all limitations and boundaries.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;So, how is the solver deciding which of the possible values are displayd, which heuristics are used?&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;If the scattering for the possible solution space is so huge, what is the quality measure for this results?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;&lt;p&gt;Best Regards&lt;/p&gt;&#xA;" OwnerUserId="15363" LastActivityDate="2022-04-19T17:59:57.563" Title="Flux balance Analysis, Result interpretation, solver Solver Heuristics" Tags="|genome|modelling|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18954" PostTypeId="2" ParentId="7094" CreationDate="2022-04-19T19:21:53.153" Score="3" Body="&lt;h1&gt;UCSC Utilities&lt;/h1&gt;&#xA;&lt;p&gt;Such output results when using the UCSC utilities &lt;code&gt;gtfToGenePred&lt;/code&gt; and &lt;code&gt;genePredToBed&lt;/code&gt; in series. Builds for macOS (x86_64) and Linux are available &lt;a href=&quot;http://hgdownload.soe.ucsc.edu/admin/exe/&quot; rel=&quot;nofollow noreferrer&quot;&gt;directly from UCSC&lt;/a&gt;. Alternatively, Conda builds are available through the Bioconda channel, under the package names &lt;code&gt;ucsc-gtftogenepred&lt;/code&gt; and &lt;code&gt;ucsc-genepredtobed&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Example: &lt;em&gt;Homo sapiens&lt;/em&gt; (Ensembl Release 106)&lt;/h2&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;## (optional install through Conda)&#xA;#mamba install -c conda-forge -c bioconda ucsc-gtftogenepred ucsc-genepredtobed&#xA;&#xA;## download GTF&#xA;wget ftp://ftp.ensembl.org/pub/release-106/gtf/homo_sapiens/Homo_sapiens.GRCh38.106.gtf.gz&#xA;&#xA;gzip -cd Homo_sapiens.GRCh38.106.gtf.gz |\&#xA;  gtfToGenePred /dev/stdin /dev/stdout |\&#xA;  genePredToBed /dev/stdin /dev/stdout |\&#xA;  head&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Output BED&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;1   1211339 1214153 ENST00000379236 0   -   1211554 1214127 0   7   286,129,197,67,102,123,171, 0,364,602,1298,1652,2323,2643,&#xA;1   1211339 1214138 ENST00000497869 0   -   1214138 1214138 0   5   493,197,67,794,156, 0,602,1298,1652,2643,&#xA;1   1212018 1213498 ENST00000453580 0   -   1213498 1213498 0   4   120,67,102,104, 0,619,973,1376,&#xA;1   1203507 1206571 ENST00000328596 0   -   1203590 1206571 0   4   453,88,123,187, 0,891,1862,2877,&#xA;1   1203507 1206592 ENST00000379268 0   -   1203843 1206571 0   5   461,203,88,123,208, 0,526,891,1862,2877,&#xA;1   1203843 1205680 ENST00000486728 0   -   1203843 1205463 0   4   125,203,88,311, 0,190,555,1526,&#xA;1   1203843 1206571 ENST00000379265 0   -   1203843 1206571 0   5   125,182,88,123,187, 0,211,555,1526,2541,&#xA;1   1471764 1497848 ENST00000673477 0   +   1471884 1495817 0   16  325,77,102,60,70,166,70,156,57,126,125,52,71,168,109,2364,  0,5509,6879,7284,9102,10373,10780,13251,14017,14345,14779,16098,17439,18492,18798,23720,&#xA;1   1478025 1497848 ENST00000472194 0   +   1497848 1497848 0   14  720,60,70,166,70,156,57,126,125,52,71,168,109,2364, 0,1023,2841,4112,4519,6990,7756,8084,8518,9837,11178,12231,12537,17459,&#xA;1   1479048 1482662 ENST00000378736 0   +   1482662 1482662 0   4   60,70,166,118,  0,1818,3089,3496,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="900" LastActivityDate="2022-04-19T19:21:53.153" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18955" PostTypeId="1" CreationDate="2022-04-19T20:01:38.040" Score="1" ViewCount="51" Body="&lt;p&gt;I would like to check transposable elements through a genome with a sliding window approach but I am not sure about which window and step size to use.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to optimise the window size and step size when doing sliding window based operations? I mean how to choose which sizes to pick?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="12203" LastEditorUserId="73" LastEditDate="2022-04-20T01:55:06.017" LastActivityDate="2022-04-20T01:55:06.017" Title="Sliding window optimisation" Tags="|transposable-elements|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18956" PostTypeId="1" CreationDate="2022-04-19T20:47:17.293" Score="3" ViewCount="180" Body="&lt;p&gt;I'd like to extract bacterial sequences from a metagenome where ~99% of contigs are from a host insect. My current protocol minimaps every 16S sequences present in the sample, then blasts all those sequences to create a library of full genomes that I minimap against. I'm using the full metagenome as my reference file. I wasn't sure if that makes a difference, but figured I'd mention it. I ran the following command to retrieve 16S sequences that are present in the sample:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 All_reads.fastq 16S_reference_library.fasta &amp;gt; 16S_map.paf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Minimap2 retrieves every entry in 16S_reference_library.fasta. It's highly improbable some of those sequences are present, or that 25,000 species are present at all. Are there any parameters I could adjust in Minimap2 that might help with this?&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, I'm trying to implement Krakenuniq and Centrifuge, but I'm concerned that, because I don't have a reference genome for the host insect, they'll have a very high false positive rate.&lt;/p&gt;&#xA;" OwnerUserId="15365" LastActivityDate="2022-04-22T21:50:52.683" Title="Minimap2 detects too many 16S sequences in metagenome" Tags="|metagenome|minimap2|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18957" PostTypeId="2" ParentId="18956" CreationDate="2022-04-20T01:51:03.400" Score="3" Body="&lt;p&gt;16S is a very conserved sequence, which is why it's used for targeted phylogenetic analysis; it makes it easy to amplify and analyse. Unfortunately that conservation is an issue with minimap2, which is built around the idea that matching scattered subsequences within a sequence is good enough for identifying matches to the entire sequence. With a highly-conserved sequence, there's a higher chance that subsequences will be identical, meaning that diverged sequences will appear to be the identical from the perspective of the mapper.&lt;/p&gt;&#xA;&lt;p&gt;Minimap2 can be tweaked to make it less sensitive and more specific, but you'd be better off using a different mapping tool, especially one designed for phylogenetic analysis and conserved sequences. I recommend &lt;a href=&quot;https://github.com/DerrickWood/kraken2/wiki/Manual#installation&quot; rel=&quot;nofollow noreferrer&quot;&gt;kraken2&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;pre-built 16S indexes for kraken2 can be found here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://benlangmead.github.io/aws-indexes/k2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://benlangmead.github.io/aws-indexes/k2&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you really want to map specifically to your own generated metagenome, then you could try &lt;a href=&quot;https://gitlab.com/mcfrith/last&quot; rel=&quot;nofollow noreferrer&quot;&gt;LAST&lt;/a&gt; (with results filtered through &lt;code&gt;last-map-probs&lt;/code&gt;), which I have found to be a little bit better in dealing with specificity issues.&lt;/p&gt;&#xA;&lt;p&gt;Aside: I notice that you've stated &amp;quot;bacterial sequences from a &lt;em&gt;metagenome&lt;/em&gt;&amp;quot;. If these are, in fact, metagenomic reads (and not targeted 16S reads), then you would be better off using one of the larger metagenome bacterial databases  (e.g. plusPF or plusPF-8, depending on computer memory) instead of concentrating only on 16S-matching reads. This will provide much better resolution; 16S only really works down to the family level.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-04-20T01:51:03.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18958" PostTypeId="2" ParentId="18949" CreationDate="2022-04-20T02:46:47.447" Score="2" Body="&lt;p&gt;If, in your input definition, you specify a string to represent your filename and provide &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#multiple-input-files&quot; rel=&quot;nofollow noreferrer&quot;&gt;multiple file inputs&lt;/a&gt; as a collection, a numerical suffix is appended to each filename:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;When a target file name is defined in the input parameter and a&#xA;collection of files is received by the process, the file name will be&#xA;appended by a numerical suffix representing its ordinal position in&#xA;the list.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;When you use the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#flatten&quot; rel=&quot;nofollow noreferrer&quot;&gt;flatten&lt;/a&gt; operator, the collection is flattened such that each entry becomes a separate entry in the output channel. So for each of the five 'tree' files, five tasks are spawned: one for each 'tree' file. These files are each localized using the filename specified: 'RAxML_bipartitionsBranchLabels.tree`. This usually isn't what you want to do.&lt;/p&gt;&#xA;&lt;p&gt;What you're looking for, I think, is some way to select the file called 'RAxML_bipartitionsBranchLabels.tree' from the initial collection of files. There are many ways to do this. For example if you wanted to flatten the output channel, it's possible to select the file you want using the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#first&quot; rel=&quot;nofollow noreferrer&quot;&gt;first&lt;/a&gt; operator and a closure:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bipartitionsBranchLabels = raxml_out_ch.flatten().first {&#xA;    it.name == 'RAxML_bipartitionsBranchLabels.tree'&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However you should only seldom need to do this. If your pipeline requires just one of these files in a downstream process, my suggestion would be to alter the upstream output declaration to include a separate output channel for this purpose:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process RAxML {&#xA;&#xA;    output:&#xA;    path 'RAxML_*.tree' into raxml_out_ch&#xA;    path 'RAxML_bipartitionsBranchLabels.tree' into raxml_bipartitionsBranchLabels&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    touch RAxML_bestTree.tree&#xA;    touch RAxML_bipartitions.tree&#xA;    touch RAxML_bipartitionsBranchLabels.tree&#xA;    touch RAxML_bootstrap.tree&#xA;    touch RAxML_info.tree&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2022-04-20T02:46:47.447" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18959" PostTypeId="1" AcceptedAnswerId="18960" CreationDate="2022-04-20T11:30:56.147" Score="3" ViewCount="71" Body="&lt;p&gt;I caught the next exception during my phylogeographical analysis in &lt;strong&gt;BEAST 2&lt;/strong&gt; with &lt;em&gt;GEO_SPHERE&lt;/em&gt;. What could be the reason? &amp;amp; how to evade this in the future?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;      ...&#xA;      856000000     -3662.2647      5969.5577     -9631.8225 42m16s/Msamples&#xA;      857000000     -3522.3540      6105.9849     -9628.3389 42m5s/Msamples&#xA;      858000000     -3621.7922      6022.9359     -9644.7282 41m54s/Msamples&#xA;      859000000     -3463.1050      6107.9766     -9571.0817 41m43s/Msamples&#xA;      860000000     -3470.6562      6160.7143     -9631.3706 41m35s/Msamples&#xA;File error: VP1_test.xml.state (Remote I/O error)&#xA;java.lang.RuntimeException: An error was encounted. Terminating BEAST&#xA;        at beast.app.util.ErrorLogHandler.publish(Unknown Source)&#xA;        at java.logging/java.util.logging.Logger.log(Logger.java:979)&#xA;        at java.logging/java.util.logging.Logger.doLog(Logger.java:1006)&#xA;        at java.logging/java.util.logging.Logger.log(Logger.java:1029)&#xA;        at java.logging/java.util.logging.Logger.severe(Logger.java:1776)&#xA;        at beast.app.beastapp.BeastMain.&amp;lt;init&amp;gt;(Unknown Source)&#xA;        at beast.app.beastapp.BeastMain.main(Unknown Source)&#xA;        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)&#xA;        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)&#xA;        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)&#xA;        at java.base/java.lang.reflect.Method.invoke(Method.java:566)&#xA;        at beast.app.beastapp.BeastLauncher.run(Unknown Source)&#xA;        at beast.app.beastapp.BeastLauncher.main(Unknown Source)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Unfortunately, googling this error with &lt;strong&gt;&amp;quot;BEAST&amp;quot;&lt;/strong&gt; gave me nothing. If without it, there are a lot of irrelevant problems.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have just found out that this error happened simultaneously onto all my nodes (different processors, RAM, &amp;amp; common hards &amp;amp; SSDs). All logs of BEAST 2 analyses have it. What is stranger, some of the analyses pressed on after this &amp;amp; someones were broken. To the present I got more interested not in the broken ones but could this exception spoil those which continued their work?&lt;/p&gt;&#xA;" OwnerUserId="15161" LastEditorUserId="3848" LastEditDate="2023-08-11T16:10:27.283" LastActivityDate="2023-08-11T16:10:27.283" Title="What is &amp; how to solve File error: my.xml.state (Remote I/O error)?" Tags="|phylogenetics|phylogeny|beast|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18960" PostTypeId="2" ParentId="18959" CreationDate="2022-04-20T22:39:58.977" Score="2" Body="&lt;p&gt;You can restart this at the place you left 'cause is Beast2.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  860000000     -3470.6562      6160.7143     -9631.3706 41m35s/Msamples&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, this is almost 1 billion replications and should be sufficient for convergence. The fact it ends on a round number might infer a algorithm limit is in place. If the Beast2 was restarted from this point and it abruptly ended this would be a very likely reason.&lt;/p&gt;&#xA;&lt;p&gt;The alternative is an error external to the algorithm such as a RAM error - it has been running a long time.&lt;/p&gt;&#xA;&lt;p&gt;Whatever the reason I would simply perform ESS statistics and convergence diagnostics on what you have now. There's no real reason to continue if it has converged for 10e6 replications maximum and this is much, much bigger than that.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The terminated processes have not spoiled the work again you can simply input the terminated chains into the restart for Beast2. This is because Beast2 uses &lt;code&gt;checkpoints&lt;/code&gt;. Brief information on how to resume from the last checkpoint is highlighted here &lt;a href=&quot;https://www.beast2.org/2014/04/07/checkpointing-tricks.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.beast2.org/2014/04/07/checkpointing-tricks.html&lt;/a&gt;. Convergence checks are important.&lt;/p&gt;&#xA;&lt;p&gt;The situation is only is only catastrophic for Beast1 where you would have needed to start from the beginning.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-04-21T13:47:05.407" LastActivityDate="2022-04-21T13:47:05.407" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18961" PostTypeId="1" AcceptedAnswerId="21368" CreationDate="2022-04-21T16:31:45.357" Score="3" ViewCount="560" Body="&lt;p&gt;I want to use sequence alignments found by &lt;a href=&quot;https://search.foldseek.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Foldseek&lt;/a&gt; in &lt;a href=&quot;https://pross.weizmann.ac.il/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PROSS&lt;/a&gt;. I know there are other ways to get alignments (also tried with results of HHPred), but I had in mind that Foldseek may find remote homologs that would still be structurally relevant.&lt;/p&gt;&#xA;&lt;p&gt;Currently Foldseek only gives alignment results in m8 format, however PROSS only accepts FASTA alignment files.&lt;/p&gt;&#xA;&lt;p&gt;I tried looking into BioPython but it seems it doesn't open m8 files, only m9. I also tried importing the alignment in Clustal X but it can't read it.&lt;/p&gt;&#xA;&lt;p&gt;I could probably use Foldseek CURL API to get the alignment as json and make a script to write the results in FASTA format, but this seems like a heavy handed solution.&lt;/p&gt;&#xA;&lt;p&gt;What would be other tools I could try to convert m8 to FASTA?&lt;/p&gt;&#xA;" OwnerUserId="12054" LastActivityDate="2023-07-28T20:28:31.797" Title="How to convert Foldseek .m8 alignment files to FASTA format?" Tags="|sequence-alignment|protein-structure|format-conversion|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18963" PostTypeId="1" CreationDate="2022-04-21T17:46:24.433" Score="1" ViewCount="128" Body="&lt;p&gt;I am new to working with gene expression data sets and am wondering what is the most standard or best way to impute missing values in a gene expression data? I got mine from the GEO database and the accession number is GSE70947. This is a breast cancer data set with a total of 296 samples (148 normal/148 cancer) and there are 62976 features (genes).&lt;/p&gt;&#xA;&lt;p&gt;The original data is only log2 normalized by the experimenter.&lt;/p&gt;&#xA;&lt;p&gt;When I first queried the data from GEO database using GEOquery from Bioconductor, the data set had NAs in it and a lot of them -- I found that there are 729494 of them in the data set.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3enT2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3enT2.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sum(is.na(gene.expr))&#xA;[1] 729494&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, I am looking online to see what are the most common imputation methods for gene expression data, but there are so many (local, global, hybrid, knowledge assisted methods) and I don't really know which is more suitable for my data set. I am just wondering what is the best way to impute the NAs. Should I just impute the NAs to 0s?&lt;/p&gt;&#xA;" OwnerUserId="14721" LastEditorUserId="14721" LastEditDate="2022-04-23T18:20:23.813" LastActivityDate="2022-04-23T18:20:23.813" Title="Missing value imputation method for gene expression data" Tags="|r|bioconductor|gene-expression|imputation|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="18965" PostTypeId="1" CreationDate="2022-04-22T05:04:24.757" Score="2" ViewCount="647" Body="&lt;p&gt;What is the difference between the I-TASSER, phyre2, and SWISS-model in the 3D tertiary structure?&lt;/p&gt;&#xA;&lt;p&gt;How do they get the results? When I did a prediction, I got a similar result for the highest template in I-TASSER and Swiss, while it was different with Phyre2. What does that mean?&lt;/p&gt;&#xA;" OwnerUserId="15378" LastEditorUserId="3694" LastEditDate="2022-04-22T08:49:28.537" LastActivityDate="2022-04-22T09:30:59.667" Title="What is the different between the I-TASSER, phyre2, SWISS-model in the 3D tertiary structure?" Tags="|sequence-alignment|proteins|protein-structure|sequence-analysis|3d-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18966" PostTypeId="1" AcceptedAnswerId="18969" CreationDate="2022-04-22T07:56:40.597" Score="1" ViewCount="76" Body="&lt;p&gt;I received (from manufacturer) several .bam files and I used four callers (samtools, freebayes, haplotypecaller, deepvariant) to find some sequence variants. In obtained .vcf files, I took a closer look to some calls. I found interesting, homozygous one rs477033 (C/G Ref/Alt) with flag 'COMMON=0' and very low MAF. I also checked the .bam file (in IGV) and this rs itself in &lt;a href=&quot;https://www.ensembl.org/Homo_sapiens/Variation/Population?db=core;r=1:165889893-165890893;v=rs477033;vdb=variation;vf=248830&quot; rel=&quot;nofollow noreferrer&quot;&gt;ensembl&lt;/a&gt; database. It looks like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Iy364.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Iy364.png&quot; alt=&quot;igv&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The problem is: in this case ref and alt are switched. The G is reference (ancestral) allele and C is altered. Nevertheless in the .bam file it's interpreted in opposite way. Of course it's false positive variant - there's no mutation there at all. All variant callers returned it as high quality variant. My. bam files are full of them.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What went wrong in this case? Problem with mapping?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;How can I fix this thing, to get only novel and rare true positive variants?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="2869" LastEditorUserId="3694" LastEditDate="2022-04-22T08:49:01.303" LastActivityDate="2022-04-22T14:20:08.253" Title="Reference variant detected as altered one in bam file" Tags="|sam|snp|ngs|variant-calling|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18967" PostTypeId="2" ParentId="18965" CreationDate="2022-04-22T09:30:59.667" Score="2" Body="&lt;p&gt;This is a common Q.&lt;/p&gt;&#xA;&lt;p&gt;Briefly, you have some techniques here that get used separately or combined:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ab initio: assemble the protein from scratch, often via a library fragmented peptides (3-mer, 6-mers, 9-mers) based on real structures.&lt;/li&gt;&#xA;&lt;li&gt;Template threading: given a protein structure alter it to make it like a give sequence. SWISS-MODEL, Rosetta's SimpleThreadingMover, ModBase, Phyre template threading etc. This is why some of these keep ligands and deal well with multimers. For missing loops in the template these are filled by ab initio methods.&lt;/li&gt;&#xA;&lt;li&gt;Hybridisation: give a bunch of models mix and match them for the best energy. I-Tasser, RosettaCM (which uses SimpleThreadingMover), Phyre (which uses Phyre template threading)&lt;/li&gt;&#xA;&lt;li&gt;Ab initio with contraints, such as evolutionary covariance. EVFold, AlphaFold2, trRosetta. Often mixed with template based approaches.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Wikipedia has more, but papers are very helpful despite the car-salesmanship.&lt;/p&gt;&#xA;&lt;p&gt;One thing to note is that protein can adopt different conformations. The AlphaFold2/ColabFold notebooks give multiple models: these may be legit alt conformations, e.g. inactive vs active etc etc&lt;/p&gt;&#xA;&lt;p&gt;Here is a SwissModel vs AF2:&#xA;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/17854/different-structures-of-same-protein-from-two-softwares/17857#17857&quot;&gt;Different structures of same protein from two softwares&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Other helpful As:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/16169/how-to-make-3d-model-of-a-protein-that-not-exist-in-pdb/16170#16170&quot;&gt;How to make 3d model of a protein that not exist in PDB?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/159/phyre2-vs-itasser-completely-different-models-generated&quot;&gt;Phyre2 vs ITasser, completely different models generated&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/18894/how-reliable-is-alphafold2-for-folding-random-coil-peptide&quot;&gt;How reliable is Alphafold2 for folding random-coil peptide&lt;/a&gt; —discussion of technical details going wrong... Personally, I had Phyre2 made an alignment mistake which altered the protein&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2022-04-22T09:30:59.667" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18969" PostTypeId="2" ParentId="18966" CreationDate="2022-04-22T14:20:08.253" Score="3" Body="&lt;p&gt;The ancestral allele is not always the reference allele. Ref and alt aren't switched, ref is just rare in at least the population you're looking at (it appears to be rare in humans as a whole). So, the C is indeed reference and G is indeed altered, it's just that people homozygous for the G are the overwhelming majority. This is evidenced by the rs477033 variant being named NC_000001.11:g.165890393C&amp;gt;G. So, in short, nothing went wrong, all tools functioned correctly.&lt;/p&gt;&#xA;&lt;p&gt;You'll notice a bunch of these over time, either make a database of them (or find one, someone may have made one) and filter them out or see if maybe the 1000 genomes reference genome has some of these modified (or maybe the T2T genome that was recently published).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2022-04-22T14:20:08.253" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18970" PostTypeId="1" CreationDate="2022-04-22T15:33:20.677" Score="2" ViewCount="74" Body="&lt;p&gt;I have &lt;a href=&quot;https://drive.google.com/file/d/1-5z_KG_fuSr88GIRoOiEDJRz_2h0cg_D/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;a nexus tree&lt;/a&gt; (1332 taxa) with a lot of additional data. When I tried to read it through &lt;code&gt;tree = Phylo.read(treepath, &amp;quot;nexus&amp;quot;)&lt;/code&gt;, my kernel got eternally loaded.&#xA;If I abort the process, I get the following message:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;---------------------------------------------------------------------------&#xA;KeyboardInterrupt                         Traceback (most recent call last)&#xA;Input In [95], in &amp;lt;cell line: 1&amp;gt;()&#xA;----&amp;gt; 1 tree = Phylo.read(r&amp;quot;D:\Downloads_hard_disk\Job\NAs_Hepatitis_A\Results\experiments\test_right\test_right_ExpPop_3\test_right_ExpPop_3.biopy.tree&amp;quot;, &amp;quot;nexus&amp;quot;)&#xA;&#xA;File ~\miniconda3\lib\site-packages\Bio\Phylo\_io.py:60, in read(file, format, **kwargs)&#xA;     58 try:&#xA;     59     tree_gen = parse(file, format, **kwargs)&#xA;---&amp;gt; 60     tree = next(tree_gen)&#xA;     61 except StopIteration:&#xA;     62     raise ValueError(&amp;quot;There are no trees in this file.&amp;quot;) from None&#xA;&#xA;File ~\miniconda3\lib\site-packages\Bio\Phylo\_io.py:49, in parse(file, format, **kwargs)&#xA;     34 &amp;quot;&amp;quot;&amp;quot;Parse a file iteratively, and yield each of the trees it contains.&#xA;     35 &#xA;     36 If a file only contains one tree, this still returns an iterable object that&#xA;   (...)&#xA;     46 &#xA;     47 &amp;quot;&amp;quot;&amp;quot;&#xA;     48 with File.as_handle(file) as fp:&#xA;---&amp;gt; 49     yield from getattr(supported_formats[format], &amp;quot;parse&amp;quot;)(fp, **kwargs)&#xA;&#xA;File ~\miniconda3\lib\site-packages\Bio\Phylo\NexusIO.py:40, in parse(handle)&#xA;     32 def parse(handle):&#xA;     33     &amp;quot;&amp;quot;&amp;quot;Parse the trees in a Nexus file.&#xA;     34 &#xA;     35     Uses the old Nexus.Trees parser to extract the trees, converts them back to&#xA;   (...)&#xA;     38     eventually change Nexus to use the new NewickIO parser directly.)&#xA;     39     &amp;quot;&amp;quot;&amp;quot;&#xA;---&amp;gt; 40     nex = Nexus.Nexus(handle)&#xA;     42     # NB: Once Nexus.Trees is modified to use Tree.Newick objects, do this:&#xA;     43     # return iter(nex.trees)&#xA;     44     # Until then, convert the Nexus.Trees.Tree object hierarchy:&#xA;     45     def node2clade(nxtree, node):&#xA;&#xA;File ~\miniconda3\lib\site-packages\Bio\Nexus\Nexus.py:668, in Nexus.__init__(self, input)&#xA;    665 self.options[&amp;quot;gapmode&amp;quot;] = &amp;quot;missing&amp;quot;&#xA;    667 if input:&#xA;--&amp;gt; 668     self.read(input)&#xA;    669 else:&#xA;    670     self.read(DEFAULTNEXUS)&#xA;&#xA;File ~\miniconda3\lib\site-packages\Bio\Nexus\Nexus.py:718, in Nexus.read(self, input)&#xA;    716     break&#xA;    717 if title in KNOWN_NEXUS_BLOCKS:&#xA;--&amp;gt; 718     self._parse_nexus_block(title, contents)&#xA;    719 else:&#xA;    720     self._unknown_nexus_block(title, contents)&#xA;&#xA;File ~\miniconda3\lib\site-packages\Bio\Nexus\Nexus.py:759, in Nexus._parse_nexus_block(self, title, contents)&#xA;    757 for line in block.commandlines:&#xA;    758     try:&#xA;--&amp;gt; 759         getattr(self, &amp;quot;_&amp;quot; + line.command)(line.options)&#xA;    760     except AttributeError:&#xA;    761         raise NexusError(&amp;quot;Unknown command: %s &amp;quot; % line.command) from None&#xA;&#xA;File ~\miniconda3\lib\site-packages\Bio\Nexus\Nexus.py:1181, in Nexus._tree(self, options)&#xA;   1179     elif special == &amp;quot;W&amp;quot;:&#xA;   1180         weight = float(value)&#xA;-&amp;gt; 1181 tree = Tree(name=name, weight=weight, rooted=rooted, tree=opts.rest().strip())&#xA;   1182 # if there's an active translation table, translate&#xA;   1183 if self.translate:&#xA;&#xA;File ~\miniconda3\lib\site-packages\Bio\Nexus\Trees.py:82, in Tree.__init__(self, tree, weight, rooted, name, data, values_are_support, max_support)&#xA;     80 # there's discrepancy whether newick allows semicolons et the end&#xA;     81 tree = tree.rstrip(&amp;quot;;&amp;quot;)&#xA;---&amp;gt; 82 subtree_info, base_info = self._parse(tree)&#xA;     83 root.data = self._add_nodedata(root.data, [[], base_info])&#xA;     84 self._add_subtree(parent_id=root.id, tree=subtree_info)&#xA;&#xA;File ~\miniconda3\lib\site-packages\Bio\Nexus\Trees.py:123, in Tree._parse(self, tree)&#xA;    121 elif not incomment and tree[p] == &amp;quot;)&amp;quot;:&#xA;    122     plevel -= 1&#xA;--&amp;gt; 123 elif tree[p:].startswith(NODECOMMENT_START):&#xA;    124     incomment = True&#xA;    125 elif incomment and tree[p] == NODECOMMENT_END:&#xA;&#xA;KeyboardInterrupt: &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am interested what is the specific reason of this: either the ineffective algorithm or a bug in &lt;strong&gt;Biopython&lt;/strong&gt;? Is there a way to read the tree as &lt;code&gt;Bio.Phylo.Newick.Tree&lt;/code&gt; object? Or should I completely use another package? In the last case, could you recommend effective one?&lt;/p&gt;&#xA;" OwnerUserId="15161" LastActivityDate="2022-04-25T20:43:52.823" Title="Biopython reads my tree eternally long" Tags="|phylogenetics|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18971" PostTypeId="2" ParentId="18956" CreationDate="2022-04-22T21:50:52.683" Score="2" Body="&lt;p&gt;You are taking reads as the reference. Given the similarity of 16S, you will find hits for most 16S in other species. You have to filter out poor hits. This is the faster way if you don't have many contigs.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, switch the order of reference and reads. But in this case, you need to use a high k-mer occurrence cut off as most of your k-mers will occur &amp;gt;10,000 times. Something like:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;minimap2 -c -f30000 -N100 ref-16S.fa reads.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will be fairly slow as there are too many hits in the database but it will be easier to process the output.&lt;/p&gt;&#xA;&lt;p&gt;You can try both and see what works for you.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2022-04-22T21:50:52.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18972" PostTypeId="1" AcceptedAnswerId="18974" CreationDate="2022-04-23T12:29:28.937" Score="1" ViewCount="77" Body="&lt;p&gt;I am a newbie in bioinformatics. In PDB, while searching for proteins, I sometimes get protein with ligand, such as PDB ID 1J1Z. In cases I get pure protein structure, in some cases I get protein-with ligand.&lt;/p&gt;&#xA;&lt;p&gt;I am having hard time understanding why I can have ligand in PDB structure. I am PDB is supposed to give me pure protein 3D structure. Then how can we have ligand in protein structure??&lt;/p&gt;&#xA;&lt;p&gt;And sorry if my question is stupid.&lt;/p&gt;&#xA;" OwnerUserId="15380" LastActivityDate="2023-01-12T12:49:15.087" Title="Protein-Ligand in PDB" Tags="|protein-structure|pdb|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18973" PostTypeId="2" ParentId="18950" CreationDate="2022-04-23T16:22:44.117" Score="1" Body="&lt;p&gt;I couldn't find a way to directly convert these SMILES strings to a more canonical format. However, thanks to &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/6322/matteo-ferla&quot;&gt;@Matteo Ferla&lt;/a&gt;, I was able to get around the problem entirely by looking up the chemical names  and resolving the canonical SMILES programmatically.&lt;/p&gt;&#xA;&lt;p&gt;More specifically, I used the CAS no column given in the supplementary information of the referenced paper to pull the SMILES via &lt;a href=&quot;https://cactus.nci.nih.gov/chemical/structure&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chemical Identifier Resolver&lt;/a&gt;. See &lt;a href=&quot;https://stackoverflow.com/a/54932071/12215814&quot;&gt;this answer&lt;/a&gt; for an explanation on how to do so.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: Some of the CAS nos couldn't be resolved, so I manually converted them using the PubChem sketcher tool mentioned in the question. There are quite a few (~160) of such molecules, so if anyone needs the full parsed dataset but doesn't want to spend the time manually parsing those molecules, let me know.&lt;/p&gt;&#xA;" OwnerUserId="15359" LastEditorUserId="15359" LastEditDate="2022-04-27T13:21:35.187" LastActivityDate="2022-04-27T13:21:35.187" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18974" PostTypeId="2" ParentId="18972" CreationDate="2022-04-23T19:33:23.733" Score="2" Body="&lt;p&gt;In some protein crystallography studies, the protein crystals are grown with the substrate/ligand/inhibitor added, thus the resulting structure contains both the ligand and the protein atom coordinates. This enables studying the protein ligand complex to understand binding, specificity, mechanism of catalysis...&lt;/p&gt;&#xA;&lt;p&gt;On the other hand in some experiments protein crystals are grown without the ligand, thus the resulting structure contains coordinates just for the protein atoms.&lt;/p&gt;&#xA;&lt;p&gt;On the third hand, the lucky one, your protein, overexpressed in its natural host [sometimes even in different one], co-purify with its own natural ligand, so the resulting structure contains coordinates for both the protein atoms and the sought for ligand.&lt;/p&gt;&#xA;&lt;p&gt;To understand ligand binding the protein structure without the ligand should be compared to the structure of the protein-ligand complex.&lt;/p&gt;&#xA;&lt;p&gt;Many published papers report both types of crystals.&lt;/p&gt;&#xA;&lt;p&gt;For example the crystal structure of pectin methylesterase in complex with hexasaccharide VI: &lt;a href=&quot;https://www.rcsb.org/structure/2NTP&quot; rel=&quot;nofollow noreferrer&quot;&gt;2NTP&lt;/a&gt; is from the publication &lt;a href=&quot;https://doi.org/10.1038/sj.emboj.7601816&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1038/sj.emboj.7601816&lt;/a&gt; which is the source of six other structures. These structures include the enzyme with different substrates, as well as the active site mutants (D178A) all of which were prepared to understand the molecular basis of substrate specificity. The conclusion after studying all of the mentioned structures is nicely summed in the sentence from the abstract:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The preferential binding of methylated sugar residues upstream of the catalytic site, and demethylated residues downstream, drives the enzyme along the pectin molecule and accounts for the sequential pattern of demethylation produced by both bacterial and plant PMEs.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="15383" LastEditorUserId="10049" LastEditDate="2023-01-12T12:49:15.087" LastActivityDate="2023-01-12T12:49:15.087" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18975" PostTypeId="1" AcceptedAnswerId="18977" CreationDate="2022-04-23T20:11:43.133" Score="0" ViewCount="210" Body="&lt;p&gt;The following is from &lt;em&gt;Bioinformatics Algorithms&lt;/em&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Fortunately, we can derive contigs from the de Bruijn graph. A path in a graph is called non-branching if in(v) = out(v) = 1 for each intermediate node v of this path, i.e., for each node except possibly the starting and ending node of a path. A maximal non-branching path is a non-branching path that cannot be extended into a longer non-branching path. We are interested in these paths because the strings of nucleotides that they spell out must be present in any assembly with a given k-mer composition. For this reason, contigs correspond to strings spelled by maximal non-branching paths in the de Bruijn graph. For example, the de Bruijn graph below, constructed for the 3-mer composition of TAATGCCATGGGATGTT, has nine maximal non-branching paths that spell out the contigs TAAT, TGTT, TGCCAT, ATG, ATG, ATG, TGG, GGG, and GGAT. In practice, biologists have no choice but to break genomes into contigs, even in the case of perfect coverage (like in the figure below), since repeats prevent them from being able to infer a unique Eulerian path.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/q06hd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/q06hd.png&quot; alt=&quot;breaking a de Bruijn graph into contigs&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But this quote doesn't make sense to me:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We are interested in these paths because the strings of nucleotides that they spell out &lt;strong&gt;must be present in any assembly with a given k-mer composition&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Is it not possible that we simply have a missing read that would have otherwise branched this path? How do we know that a maximal non-branching path &lt;em&gt;must&lt;/em&gt; be a contig?&lt;/p&gt;&#xA;" OwnerUserId="14160" LastActivityDate="2022-04-29T14:37:33.953" Title="Why must a maximal non-branching path be a contig?" Tags="|sequence-alignment|ngs|assembly|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18977" PostTypeId="2" ParentId="18975" CreationDate="2022-04-24T16:41:09.950" Score="1" Body="&lt;p&gt;The definition of contig is derived from assembly algorithms. A contig is not necessarily present in the genome. PS: i.e. &amp;quot;contig&amp;quot; is algorithmically defined, not biologically defined.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is it not possible that we simply have a missing read that would have otherwise branched this path?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, it is possible. Then you are likely to get a misassembled contig.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2022-04-29T14:37:33.953" LastActivityDate="2022-04-29T14:37:33.953" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18978" PostTypeId="2" ParentId="18801" CreationDate="2022-04-24T18:26:09.317" Score="1" Body="&lt;h3&gt;Overview&lt;/h3&gt;&#xA;&lt;p&gt;To properly address the questions, each possible case must be listed:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Case 1&lt;/strong&gt;: divergence between sequences (sequence in the file vs sequence of the molecule); two cases:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;a&lt;/strong&gt;- (wrong) sequence in the file vs (correct) sequence in the molecule&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;b&lt;/strong&gt;- (&amp;quot;correct&amp;quot;, theoretical) sequence in the file vs (actual, mutated) sequence in the molecule&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Case 2&lt;/strong&gt;: divergence between a feature annotation in the file vs the correct (&amp;quot;true&amp;quot; or reference) biological feature; three cases:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;a&lt;/strong&gt;- wrong location of the feature&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;b&lt;/strong&gt;- wrong feature name&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;c&lt;/strong&gt;- variant sequence of the feature&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Short answer&lt;/h3&gt;&#xA;&lt;p&gt;&lt;strong&gt;Case 1a&lt;/strong&gt;: The sequence in the GenBank file must be corrected and the &lt;em&gt;version number&lt;/em&gt; of the file must be incremented (see &lt;strong&gt;Q2&lt;/strong&gt;).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Case 1b&lt;/strong&gt;: A new GenBank file consistent with the actual molecule must be created and the &lt;em&gt;misc_difference&lt;/em&gt; feature key must be used to annotate the variations where relevant.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Cases 2a and 2b&lt;/strong&gt;: Feature information in the GenBank file must be corrected and the &lt;em&gt;Modification Date&lt;/em&gt; of the file must be updated (see &lt;strong&gt;Q2&lt;/strong&gt;).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Case 2c&lt;/strong&gt;: The &lt;em&gt;misc_difference&lt;/em&gt; feature must be used to annotate the variation and the &lt;em&gt;Modification Date&lt;/em&gt; of the file must be updated if necessary (see &lt;strong&gt;Q2&lt;/strong&gt;).&lt;/p&gt;&#xA;&lt;h3&gt;Long answer&lt;/h3&gt;&#xA;&lt;h4&gt;Q1. How to handle the situation when there are divergences between a GenBank file and an actual or theoretical molecule?&lt;/h4&gt;&#xA;&lt;h5&gt;The &lt;em&gt;misc_difference&lt;/em&gt; feature key&lt;/h5&gt;&#xA;&lt;p&gt;First, let's analyse the situation with the &lt;em&gt;misc_difference&lt;/em&gt; feature key.&#xA;According to the &lt;a href=&quot;https://www.insdc.org/documents/feature_table.html#7.2&quot; rel=&quot;nofollow noreferrer&quot;&gt;INSDC documentation&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Feature Key           misc_difference&#xA;&#xA;&#xA;Definition            feature sequence is different from that presented &#xA;                      in the entry and cannot be described by any other &#xA;                      difference key (old_sequence, variation, or modified_base);&#xA;&#xA;Optional qualifiers   /allele=&amp;quot;text&amp;quot;&#xA;                      /clone=&amp;quot;text&amp;quot;&#xA;                      /db_xref=&amp;quot;&amp;lt;database&amp;gt;:&amp;lt;identifier&amp;gt;&amp;quot;&#xA;                      /experiment=&amp;quot;[CATEGORY:]text&amp;quot;&#xA;                      /gene=&amp;quot;text&amp;quot;&#xA;                      /gene_synonym=&amp;quot;text&amp;quot;&#xA;                      /inference=&amp;quot;[CATEGORY:]TYPE[(same species)][:EVIDENCE_BASIS]&amp;quot;&#xA;                      /locus_tag=&amp;quot;text&amp;quot; (single token)&#xA;                      /map=&amp;quot;text&amp;quot; &#xA;                      /note=&amp;quot;text&amp;quot;&#xA;                      /old_locus_tag=&amp;quot;text&amp;quot; (single token)&#xA;                      /phenotype=&amp;quot;text&amp;quot;&#xA;                      /replace=&amp;quot;text&amp;quot; &#xA;                      /standard_name=&amp;quot;text&amp;quot;&#xA;&#xA;Comment               the misc_difference feature key should be used to &#xA;                      describe variability that arises as a result of &#xA;                      genetic manipulation (e.g. site directed mutagenesis);&#xA;                      use /replace=&amp;quot;&amp;quot; to annotate deletion, e.g. &#xA;                      misc_difference 412..433&#xA;                                      /replace=&amp;quot;&amp;quot;  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From the Definition above, there is still some ambiguity as to what situation is described. Indeed, &amp;quot;&lt;em&gt;feature sequence is different from that presented in the entry&lt;/em&gt;&amp;quot; can describe both of the following cases:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;the sequence in the file and/or in the molecule is not the one expected for the feature annotated&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;the sequence in the molecule is not the one represented in the file&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;em&gt;Note: In fact, as we will see below, there should be no divergence between the sequence in the file and the sequence of the molecule.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;As for the Comment that says that &amp;quot;&lt;em&gt;the misc_difference feature key should be used to describe variability that arises as a result of genetic manipulation (e.g. site directed mutagenesis)&lt;/em&gt;&amp;quot;, it does not really clarify which of the two cases above is the one intended. Further, the example given in the Comment, which at first sight seems more enlightening, still carries some ambiguity. It says that to annotate a deletion, one should use:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;misc_difference 412..433&#xA;                /replace=&amp;quot;&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;All in all, it implies that the deletion is in the molecule (vs in the represented sequence), that it arose as a result of genetic manipulation and is therefore indicated in the file as explained above.&lt;/p&gt;&#xA;&lt;p&gt;However, it is very counter-intuitive, and not very practical when searching sequences of variants among sequence files, to have a sequence file that is supposed to describe a molecule and in which the sequence is actually different from the sequence encountered in the molecule it is supposed to describe.&lt;/p&gt;&#xA;&lt;p&gt;And indeed, this view goes against common observed recent practice where the &lt;em&gt;misc_difference&lt;/em&gt; feature key is used to indicate how to go from the sequence represented in the sequence file to another version:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Example 1&lt;/strong&gt;: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/D90147.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/D90147.1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;     misc_difference 308..310&#xA;                     /gene=&amp;quot;mucA&amp;quot;&#xA;                     /note=&amp;quot;conflict: deletion&amp;quot;&#xA;                     /citation=[1]&#xA;                     /replace=&amp;quot;&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The indicated deletion is not in the sequence but is a way to go from the sequence to a previously reported CDS sequence for the annotated gene.&lt;/p&gt;&#xA;&lt;p&gt;Translation of the CDS sequence in the GenBank file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MKVDIFESSGASRVHSIPFYLQRISAGFPSPAQGYEKQELNLHEYCVRHPSATYFLRVSGSSMEDGRIHDGDVLVVDRSLTASHGSIVVACIHNEFTVKRLLLRPRPCLMPMNKDFPVYYIDPDNESVEIWGVVTHSLIEHPVCLR*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Previously reported CDS (see: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/M13388;&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/M13388;&lt;/a&gt;&#xA;hyphen/dash in the sequence added by me to mark the difference)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MKVDIFESSGAS-VHSIPFYLQRISAGFPSPAQGYEKQELNLHEYCVRHPSATYFLRVSGSSMEDGRIHDGDVLVVDRSLTASHGSIVVACIHNEFTVKRLLLRPRPCLMPMNKDFPVYYIDPDNESVEIWGVVTHSLIEHPVCLR*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Ref.: &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/1902211/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pubmed.ncbi.nlm.nih.gov/1902211/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Example 2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In each of the GenBank files below, the &lt;em&gt;misc_difference&lt;/em&gt; feature key is used to indicate &lt;em&gt;how to go from the (mutated) sequence in the GenBank file to the sequence of the &amp;quot;wild type&amp;quot;&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Escherichia coli&lt;/em&gt; &lt;em&gt;uidA&lt;/em&gt; gene for beta-glucuronidase mutant E413G, clone M2&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/AM180563.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/AM180563.1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; misc_difference 1238&#xA;               /gene=&amp;quot;uidA&amp;quot;&#xA;               /experiment=&amp;quot;experimental evidence, no additional details&#xA;               recorded&amp;quot;&#xA;               /note=&amp;quot;site directed mutagenesis&amp;quot;&#xA;               /compare=S69414.1&#xA;               /allele=&amp;quot;wild type&amp;quot;&#xA;               /phenotype=&amp;quot;Active GUS&amp;quot;&#xA;               /replace=&amp;quot;a&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Escherichia coli&lt;/em&gt; &lt;em&gt;uidA&lt;/em&gt; gene for beta-glucuronidase mutant C464R, clone M4&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/AM180564.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/AM180564.1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;misc_difference 1390&#xA;                /gene=&amp;quot;uidA&amp;quot;&#xA;                /experiment=&amp;quot;experimental evidence, no additional details&#xA;                recorded&amp;quot;&#xA;                /note=&amp;quot;site directed mutagenesis&amp;quot;&#xA;                /compare=S69414.1&#xA;                /allele=&amp;quot;wild type&amp;quot;&#xA;                /phenotype=&amp;quot;Active GUS&amp;quot;&#xA;                /replace=&amp;quot;t&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;See also:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Escherichia coli&lt;/em&gt; &lt;em&gt;uidA&lt;/em&gt; gene for beta-glucuronidase mutant Y468C, clone M6&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/AM180565.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/AM180565.1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;Escherichia coli&lt;/em&gt; &lt;em&gt;uidA&lt;/em&gt; gene for beta-glucuronidase mutant W549R, clone M12&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/AM180566.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/AM180566.1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;strong&gt;Example 3&lt;/strong&gt;: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/KC999035.4&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/KC999035.4&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; misc_difference 137414..137415&#xA;&#xA;                 /note=&amp;quot;TT compared with single T in original ISCfr1&#xA;                 sequence that would result in two partial transposase&#xA;                 proteins. Confirmed as correct in raw sequence data.&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here again the &lt;em&gt;misc_difference&lt;/em&gt; feature key is used to describe &lt;em&gt;how to pass from the represented sequence to a previous sequence version of the considered gene&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So the &lt;em&gt;misc_difference&lt;/em&gt; feature key is appropriate when the sequence in the sequence file &lt;em&gt;does&lt;/em&gt; match the sequence in the molecule but does &lt;em&gt;&lt;strong&gt;not&lt;/strong&gt; (any more)&lt;/em&gt; match &lt;em&gt;another reference sequence&lt;/em&gt; (possibly referred to by another gene-related feature) because variations in the sequence have occurred due to some genetic manipulation or, as in Example 3, due to discrepancy with previous sequencing results.&lt;/p&gt;&#xA;&lt;p&gt;Thus, the &lt;em&gt;misc_difference&lt;/em&gt; feature key is appropriate in &lt;strong&gt;Cases 1b and 2c&lt;/strong&gt;, and the &lt;em&gt;/replace&lt;/em&gt; qualifier should be used to indicate how to get from the represented sequence to the other reference sequence for the designated feature (in Example 3 above, the &lt;em&gt;/note&lt;/em&gt; qualifier is used instead to describe the difference).&lt;/p&gt;&#xA;&lt;h5&gt;Other cases&lt;/h5&gt;&#xA;&lt;p&gt;There are three other cases left:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Case 1a&lt;/strong&gt;: the features in the GenBank file are consistent with the molecule, but the sequence in the file is wrong&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Cases 2a and 2b&lt;/strong&gt;: the sequence in the GenBank file is consistent with the molecule, but information about some features is wrong&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Those three cases can all be classified as a GenBank file containing errors (in contrast with &lt;strong&gt;case 1b&lt;/strong&gt; where the &amp;quot;error&amp;quot; is in the obtained molecule and with &lt;strong&gt;case 2c&lt;/strong&gt; where the information in the GenBank file is merely incomplete). A GenBank file containing errors is just not appropriate, and the solution here is simply to correct the GenBank file to accurately describe the corresponding molecule.&lt;/p&gt;&#xA;&lt;p&gt;All that remains is to answer to the second question about how to track changes...&lt;/p&gt;&#xA;&lt;h4&gt;Q2. How to keep track of changes between versions of a GenBank file when corrections or updates are made?&lt;/h4&gt;&#xA;&lt;h5&gt;Versioning and time-stamping GenBank files&lt;/h5&gt;&#xA;&lt;p&gt;GenBank files have two fields that can be used to track their history: the &lt;strong&gt;Modification Date&lt;/strong&gt; field and the &lt;strong&gt;VERSION&lt;/strong&gt; field.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;strong&gt;VERSION&lt;/strong&gt; field uses an &lt;em&gt;accession.version&lt;/em&gt; format, where the &lt;em&gt;accession number&lt;/em&gt; is fixed and the &lt;em&gt;version number&lt;/em&gt; is incremented only when the sequence data is modified.&lt;/p&gt;&#xA;&lt;p&gt;See:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genbank/samplerecord/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/genbank/samplerecord/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genbank/samplerecord/#ModificationDateB&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/genbank/samplerecord/#ModificationDateB&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genbank/samplerecord/#VersionB&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/genbank/samplerecord/#VersionB&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;em&gt;Note that only modifications in the sequence result in a change in the version number. Changes in annotations only affect the timestamp (Modification Date) of the file.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;See:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genbank/sequenceids/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/genbank/sequenceids/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;With this information we can now describe how to properly keep track of the successive versions of a GenBank file in each of the cases described in &lt;strong&gt;Q1&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;h5&gt;Proper way to keep track of modifications in GenBank files&lt;/h5&gt;&#xA;&lt;p&gt;&lt;strong&gt;Case 1a&lt;/strong&gt;: (wrong) sequence in the file vs (correct) sequence in the molecule&lt;/p&gt;&#xA;&lt;p&gt;The sequence in the file must be corrected and the version number of the file must be incremented by one.&lt;/p&gt;&#xA;&lt;p&gt;Moreover the respective file versions (the former and the newer) must refer to each other in the COMMENT field of their respective header.&lt;/p&gt;&#xA;&lt;p&gt;See:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genbank/release/current/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/genbank/release/current/&lt;/a&gt; --&amp;gt; 3.4.2  Entry Organization&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Case 1b&lt;/strong&gt;: (&amp;quot;correct&amp;quot;, theoretical) sequence in the file vs (actual, mutated) sequence in the molecule&lt;/p&gt;&#xA;&lt;p&gt;A new GenBank file and new molecule name must be created to accurately describe the variant molecule. The &lt;em&gt;misc_difference&lt;/em&gt; feature key along with the &lt;em&gt;/replace&lt;/em&gt; qualifier must be used as described above where it is deemed appropriate.&lt;/p&gt;&#xA;&lt;p&gt;The gene-related features affected should keep the same &lt;em&gt;/gene&lt;/em&gt; qualifier, but an &lt;em&gt;/allele&lt;/em&gt; qualifier might be used to indicate the presence of a difference and the &lt;em&gt;/note&lt;/em&gt; and &lt;em&gt;/phenotype&lt;/em&gt; qualifiers can be used to indicate the reference sequence to which the allele should be compared to and any possible phenotypic consequence of the variation.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;em&gt;misc_difference&lt;/em&gt; feature key can be given the same gene-related qualifier information as the gene where the difference occurs (&lt;em&gt;/gene&lt;/em&gt;, &lt;em&gt;/allele&lt;/em&gt;, &lt;em&gt;/note&lt;/em&gt;...)&lt;/p&gt;&#xA;&lt;p&gt;In the special case of variants unintentionally obtained during genetic construction or genetic manipulation, an altered name for the molecule and GenBank file might be used to reflect the fact that the corresponding sequence is different from the theoretically expected sequence, and the &lt;em&gt;misc_difference&lt;/em&gt; feature key should be used to indicate any difference that occurred during the construction compared to the initially targetted sequence.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Cases 2a, 2b or 2c&lt;/strong&gt;: wrong or incomplete feature data&lt;/p&gt;&#xA;&lt;p&gt;The feature annotation in the file must be corrected and the &lt;em&gt;Modification Date&lt;/em&gt; field must be updated according to the correction date.&lt;/p&gt;&#xA;&lt;h3&gt;Additional references&lt;/h3&gt;&#xA;&lt;p&gt;&lt;strong&gt;Sequence revision history&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genbank/sequencerevisionhistory/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/genbank/sequencerevisionhistory/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Examples of revision history of some GenBank files&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/U46667?report=girevhist&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/U46667?report=girevhist&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NM_029716?report=girevhist&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/NM_029716?report=girevhist&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NM_033116?report=girevhist&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/NM_033116?report=girevhist&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h3&gt;Added for convenience&lt;/h3&gt;&#xA;&lt;p&gt;Here below is reproduced the description and usage of some &lt;em&gt;feature keys&lt;/em&gt; and &lt;em&gt;qualifiers&lt;/em&gt; that might be relevant to, or linked with, the question of &lt;em&gt;how to describe and annotate sequence variants&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I grouped those definitions there as it might help one to distinguish the different types of variations and divergences and to understand and chose the appropriate term to annotate sequence variants.&lt;/p&gt;&#xA;&lt;h4&gt;Selected feature keys&lt;/h4&gt;&#xA;&lt;p&gt;&lt;strong&gt;gene&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Feature Key           gene&#xA;&#xA;&#xA;Definition            region of biological interest identified as a gene &#xA;                      and for which a name has been assigned;&#xA;&#xA;Optional qualifiers   /allele=&amp;quot;text&amp;quot;&#xA;                      /db_xref=&amp;quot;&amp;lt;database&amp;gt;:&amp;lt;identifier&amp;gt;&amp;quot;&#xA;                      /experiment=&amp;quot;[CATEGORY:]text&amp;quot;&#xA;                      /function=&amp;quot;text&amp;quot;&#xA;                      /gene=&amp;quot;text&amp;quot;&#xA;                      /gene_synonym=&amp;quot;text&amp;quot;&#xA;                      /inference=&amp;quot;[CATEGORY:]TYPE[(same species)][:EVIDENCE_BASIS]&amp;quot;&#xA;                      /locus_tag=&amp;quot;text&amp;quot; (single token)&#xA;                      /map=&amp;quot;text&amp;quot;&#xA;                      /note=&amp;quot;text&amp;quot;&#xA;                      /old_locus_tag=&amp;quot;text&amp;quot; (single token)&#xA;                      /operon=&amp;quot;text&amp;quot;&#xA;                      /product=&amp;quot;text&amp;quot;&#xA;                      /pseudo&#xA;                      /pseudogene=&amp;quot;TYPE&amp;quot;&#xA;                      /phenotype=&amp;quot;text&amp;quot;&#xA;                      /standard_name=&amp;quot;text&amp;quot;&#xA;                      /trans_splicing&#xA;&#xA;&#xA;Comment               the gene feature describes the interval of DNA that &#xA;                      corresponds to a genetic trait or phenotype; the feature is,&#xA;                      by definition, not strictly bound to it's positions at the &#xA;                      ends;  it is meant to represent a region where the gene is &#xA;                      located.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;misc_difference&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Feature Key           misc_difference&#xA;&#xA;&#xA;Definition            feature sequence is different from that presented &#xA;                      in the entry and cannot be described by any other &#xA;                      difference key (old_sequence, variation, or modified_base);&#xA;&#xA;Optional qualifiers   /allele=&amp;quot;text&amp;quot;&#xA;                      /clone=&amp;quot;text&amp;quot;&#xA;                      /db_xref=&amp;quot;&amp;lt;database&amp;gt;:&amp;lt;identifier&amp;gt;&amp;quot;&#xA;                      /experiment=&amp;quot;[CATEGORY:]text&amp;quot;&#xA;                      /gene=&amp;quot;text&amp;quot;&#xA;                      /gene_synonym=&amp;quot;text&amp;quot;&#xA;                      /inference=&amp;quot;[CATEGORY:]TYPE[(same species)][:EVIDENCE_BASIS]&amp;quot;&#xA;                      /locus_tag=&amp;quot;text&amp;quot; (single token)&#xA;                      /map=&amp;quot;text&amp;quot; &#xA;                      /note=&amp;quot;text&amp;quot;&#xA;                      /old_locus_tag=&amp;quot;text&amp;quot; (single token)&#xA;                      /phenotype=&amp;quot;text&amp;quot;&#xA;                      /replace=&amp;quot;text&amp;quot; &#xA;                      /standard_name=&amp;quot;text&amp;quot;&#xA;&#xA;Comment               the misc_difference feature key should be used to &#xA;                      describe variability that arises as a result of &#xA;                      genetic manipulation (e.g. site directed mutagenesis);&#xA;                      use /replace=&amp;quot;&amp;quot; to annotate deletion, e.g. &#xA;                      misc_difference 412..433&#xA;                                      /replace=&amp;quot;&amp;quot;  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;modified_base&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Feature Key           modified_base&#xA;&#xA;&#xA;Definition            the indicated nucleotide is a modified nucleotide and&#xA;                      should be substituted for by the indicated molecule&#xA;                      (given in the mod_base qualifier value)&#xA; &#xA;Mandatory qualifiers  /mod_base=&amp;lt;modified_base&amp;gt;&#xA;&#xA;Optional qualifiers   /allele=&amp;quot;text&amp;quot;&#xA;                      /db_xref=&amp;quot;&amp;lt;database&amp;gt;:&amp;lt;identifier&amp;gt;&amp;quot;&#xA;                      /experiment=&amp;quot;[CATEGORY:]text&amp;quot;&#xA;                      /frequency=&amp;quot;text&amp;quot;&#xA;                      /gene=&amp;quot;text&amp;quot;&#xA;                      /gene_synonym=&amp;quot;text&amp;quot;&#xA;                      /inference=&amp;quot;[CATEGORY:]TYPE[(same species)][:EVIDENCE_BASIS]&amp;quot;&#xA;                      /locus_tag=&amp;quot;text&amp;quot; (single token)&#xA;                      /map=&amp;quot;text&amp;quot;&#xA;                      /note=&amp;quot;text&amp;quot;&#xA;                      /old_locus_tag=&amp;quot;text&amp;quot; (single token)&#xA;&#xA;Comment               value is limited to the restricted vocabulary for &#xA;                      modified base abbreviations;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;old_sequence&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Feature Key           old_sequence&#xA;&#xA;&#xA;Definition            the presented sequence revises a previous version of the&#xA;                      sequence at this location;&#xA;&#xA;Mandatory qualifiers  /citation=[number]&#xA;                      Or&#xA;                      /compare=[accession-number.sequence-version]&#xA;&#xA;Optional qualifiers   /allele=&amp;quot;text&amp;quot;&#xA;                      /db_xref=&amp;quot;&amp;lt;database&amp;gt;:&amp;lt;identifier&amp;gt;&amp;quot;&#xA;                      /experiment=&amp;quot;[CATEGORY:]text&amp;quot;&#xA;                      /gene=&amp;quot;text&amp;quot;&#xA;                      /gene_synonym=&amp;quot;text&amp;quot;&#xA;                      /inference=&amp;quot;[CATEGORY:]TYPE[(same species)][:EVIDENCE_BASIS]&amp;quot;&#xA;                      /locus_tag=&amp;quot;text&amp;quot; (single token)&#xA;                      /map=&amp;quot;text&amp;quot;&#xA;                      /note=&amp;quot;text&amp;quot;&#xA;                      /old_locus_tag=&amp;quot;text&amp;quot; (single token)&#xA;                      /replace=&amp;quot;text&amp;quot;&#xA;&#xA;Comment               /replace=&amp;quot;&amp;quot; is used to annotate deletion, e.g. &#xA;                      old_sequence 12..15&#xA;                      /replace=&amp;quot;&amp;quot; &#xA;                      NOTE: This feature key is not valid in entries/records&#xA;                      created from 15-Oct-2007.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;variation&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Feature Key           variation&#xA;&#xA;Definition            a related strain contains stable mutations from the same&#xA;                      gene (e.g., RFLPs, polymorphisms, etc.) which differ&#xA;                      from the presented sequence at this location (and&#xA;                      possibly others);&#xA;&#xA;Optional qualifiers   /allele=&amp;quot;text&amp;quot;&#xA;                      /db_xref=&amp;quot;&amp;lt;database&amp;gt;:&amp;lt;identifier&amp;gt;&amp;quot;&#xA;                      /experiment=&amp;quot;[CATEGORY:]text&amp;quot;&#xA;                      /frequency=&amp;quot;text&amp;quot;&#xA;                      /gene=&amp;quot;text&amp;quot;&#xA;                      /gene_synonym=&amp;quot;text&amp;quot;&#xA;                      /inference=&amp;quot;[CATEGORY:]TYPE[(same species)][:EVIDENCE_BASIS]&amp;quot;&#xA;                      /locus_tag=&amp;quot;text&amp;quot; (single token)&#xA;                      /map=&amp;quot;text&amp;quot;&#xA;                      /note=&amp;quot;text&amp;quot;&#xA;                      /old_locus_tag=&amp;quot;text&amp;quot; (single token)&#xA;                      /phenotype=&amp;quot;text&amp;quot;&#xA;                      /product=&amp;quot;text&amp;quot;&#xA;                      /replace=&amp;quot;text&amp;quot;&#xA;                      /standard_name=&amp;quot;text&amp;quot;&#xA;&#xA;Comment               used to describe alleles, RFLP's,and other naturally&#xA;                      occurring mutations and  polymorphisms; variability arising&#xA;                      as a result of genetic manipulation (e.g. site directed&#xA;                      mutagenesis) should be described with the misc_difference&#xA;                      feature; use /replace=&amp;quot;&amp;quot; to annotate deletion, e.g. &#xA;                      variation   4..5&#xA;                                  /replace=&amp;quot;&amp;quot;  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h4&gt;Selected qualifiers&lt;/h4&gt;&#xA;&lt;p&gt;&lt;strong&gt;/allele&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  Qualifier       /allele=&#xA;  Definition      name of the allele for the given gene &#xA;  Value format    &amp;quot;text&amp;quot;&#xA;  Example         /allele=&amp;quot;adh1-1&amp;quot;&#xA;  Comment         all gene-related features (exon, CDS etc) for a given &#xA;                  gene should share the same /allele qualifier value; &#xA;                  the /allele qualifier value must, by definition, be &#xA;                  different from the /gene qualifier value; when used with &#xA;                  the variation feature key, the allele qualifier value &#xA;                  should be that of the variant.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;/clone&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  Qualifier       /clone=&#xA;  Definition      clone from which the sequence was obtained&#xA;  Value format    &amp;quot;text&amp;quot;&#xA;  Example         /clone=&amp;quot;lambda-hIL7.3&amp;quot;&#xA;  Comment         not more than one clone should be specified for a given source &#xA;                  feature;  to indicate that the sequence was obtained from&#xA;                  multiple clones, multiple source features should be given.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;/compare&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  Qualifier       /compare=&#xA;  Definition      Reference details of an existing public INSD entry &#xA;                  to which a comparison is made&#xA;  Value format    [accession-number.sequence-version]&#xA;  Example         /compare=AJ634337.1&#xA;  Comment         not to be used for new entries from 14-MAY-2021; This qualifier&#xA;                  may be used on the following features: misc_difference, unsure,&#xA;                  old_sequence and variation.&#xA;                  The feature &amp;quot;old_sequence&amp;quot; must have either a&#xA;                  /citation or a /compare qualifier. Multiple /compare&#xA;                  qualifiers with different contents are allowed within a &#xA;                  single feature. &#xA;                  This qualifier is not intended for large-scale annotation &#xA;                  of variations, such as SNPs.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;/function&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  Qualifier       /function=&#xA;  Definition      function attributed to a sequence&#xA;  Value format    &amp;quot;text&amp;quot;&#xA;  Example         function=&amp;quot;essential for recognition of cofactor&amp;quot;&#xA;  Comment         /function is used when the gene name and/or product name do not &#xA;                  convey the function attributable to a sequence.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;/gene&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  Qualifier       /gene=&#xA;  Definition      symbol of the gene corresponding to a sequence region&#xA;  Value format    &amp;quot;text&amp;quot;&#xA;  Example         /gene=&amp;quot;ilvE&amp;quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;/locus_tag&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  Qualifier       /locus_tag=&#xA;  Definition      a submitter-supplied, systematic, stable identifier for a gene&#xA;                  and its associated features, used for tracking purposes&#xA;  Value Format    &amp;quot;text&amp;quot;(single token) &#xA;                  but not &amp;quot;&amp;lt;1-5 letters&amp;gt;&amp;lt;5-9 digit integer&amp;gt;[.&amp;lt;integer&amp;gt;]&amp;quot;&#xA;  Example         /locus_tag=&amp;quot;ABC_0022&amp;quot; &#xA;                  /locus_tag=&amp;quot;A1C_00001&amp;quot;&#xA;  Comment         /locus_tag can be used with any feature that /gene can be used&#xA;                  with; identical /locus_tag values may be used within an&#xA;                  entry/record, but only if the identical /locus_tag values are &#xA;                  associated with the same gene; in all other circumstances the&#xA;                  /locus_tag value must be unique within that entry/record.&#xA;                  Multiple /locus_tag values are not allowed within one feature for&#xA;                  entries created after 15-OCT-2004. &#xA;                  If a /locus_tag needs to be re-assigned the /old_locus_tag&#xA;                  qualifier should be used to store the old value. The /locus_tag&#xA;                  value should not be in a format which resembles INSD accession&#xA;                  numbers, accession.version, or /protein_id identifiers.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;/map&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  Qualifier       /map=&#xA;  Definition      genomic map position of feature&#xA;  Value format    &amp;quot;text&amp;quot;&#xA;  Example         /map=&amp;quot;8q12-q13&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;/note&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  Qualifier       /note=&#xA;  Definition      any comment or additional information&#xA;  Value format    &amp;quot;text&amp;quot;&#xA;  Example         /note=&amp;quot;This qualifier is equivalent to a comment.&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;/operon&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  Qualifier       /operon=&#xA;  Definition      name of the group of contiguous genes transcribed into a &#xA;                  single transcript to which that feature belongs.&#xA;  Value format    &amp;quot;text&amp;quot;&#xA;  Example         /operon=&amp;quot;lac&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;/phenotype&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  Qualifier       /phenotype=&#xA;  Definition      phenotype conferred by the feature, where phenotype is defined as&#xA;                  a physical, biochemical or behavioural characteristic or set of &#xA;                  characteristics&#xA;  Value format    &amp;quot;text&amp;quot;&#xA;  Example         /phenotype=&amp;quot;erythromycin resistance&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;/product&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  Qualifier       /product=&#xA;  Definition      name of the product associated with the feature, e.g. the mRNA of&#xA;                  an mRNA feature, the polypeptide of a CDS, the mature peptide of&#xA;                  a mat_peptide, etc.&#xA;  Value format    &amp;quot;text&amp;quot;&#xA;  Example         /product=&amp;quot;trypsinogen&amp;quot; (when qualifier appears in CDS feature)&#xA;                  /product=&amp;quot;trypsin&amp;quot; (when qualifier appears in mat_peptide&#xA;                  feature)&#xA;                  /product=&amp;quot;XYZ neural-specific transcript&amp;quot; (when qualifier appears&#xA;                  in mRNA feature)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;/pseudo&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  Qualifier       /pseudo&#xA;  Definition      indicates that this feature is a non-functional version of the&#xA;                  element named by the feature key&#xA;  Value format    none&#xA;  Example         /pseudo&#xA;  Comment         The qualifier /pseudo should be used to describe non-functional &#xA;                  genes that are not formally described as pseudogenes, e.g. CDS &#xA;                  has no translation due to other reasons than pseudogenisation&#xA;                  events.&#xA;                  Other reasons may include sequencing or assembly errors.&#xA;                  In order to annotate pseudogenes the qualifier /pseudogene= must&#xA;                  be used indicating the TYPE which can be taken from the INSDC&#xA;                  controlled vocabulary for pseudogenes.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;/pseudogene&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  Qualifier       /pseudogene=&#xA;  Definition      indicates that this feature is a pseudogene of the element&#xA;                  named by the feature key&#xA;  Value format    &amp;quot;TYPE&amp;quot;&#xA;                  where TYPE is one of the following:&#xA;                  processed, unprocessed, unitary, allelic, unknown&#xA;&#xA;  Example         /pseudogene=&amp;quot;processed&amp;quot;&#xA;                  /pseudogene=&amp;quot;unprocessed&amp;quot;&#xA;                  /pseudogene=&amp;quot;unitary&amp;quot;&#xA;                  /pseudogene=&amp;quot;allelic&amp;quot;&#xA;                  /pseudogene=&amp;quot;unknown&amp;quot;&#xA;&#xA;  Comment         TYPE is a term taken from the INSDC controlled vocabulary for&#xA;                  pseudogenes (http://www.insdc.org/documents/pseudogene-&#xA;                  qualifier-vocabulary):&#xA;&#xA;                  processed: the pseudogene has arisen by reverse transcription&#xA;                  of a mRNA into cDNA, followed by reintegration into the genome.&#xA;                  Therefore, it has lost any intron/exon structure, and it might&#xA;                  have a pseudo-polyA-tail.&#xA;&#xA;                  unprocessed: the pseudogene has arisen from a copy of the&#xA;                  parent gene by duplication followed by accumulation of random&#xA;                  mutations.&#xA;                  The changes, compared to their functional homolog, include&#xA;                  insertions, deletions, premature stop codons, frameshifts and a&#xA;                  higher proportion of non-synonymous versus synonymous&#xA;                  substitutions.&#xA;                  &#xA;                  unitary: the pseudogene has no parent. It is the original gene,&#xA;                  which is functional is some species but disrupted in some way&#xA;                  (indels, mutation, recombination) in another species or strain.&#xA;&#xA;                  allelic: a (unitary) pseudogene that is stable in the&#xA;                  population but importantly it has a functional alternative&#xA;                  allele also in the population. i.e., one strain may have the&#xA;                  gene, another strain may have the pseudogene.&#xA;                  MHC haplotypes have allelic pseudogenes.&#xA;&#xA;                  unknown: the submitter does not know the method of&#xA;                  pseudogenisation.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;/standard_name&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  Qualifier       /standard_name=&#xA;  Definition      accepted standard name for this feature&#xA;  Value format    &amp;quot;text&amp;quot;&#xA;  Example         /standard_name=&amp;quot;dotted&amp;quot;&#xA;  Comment         use /standard_name to give full gene name, but use /gene to&#xA;                  give gene symbol (in the above example /gene=&amp;quot;Dt&amp;quot;).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15221" LastEditorUserId="15221" LastEditDate="2022-05-02T08:58:27.963" LastActivityDate="2022-05-02T08:58:27.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18980" PostTypeId="1" CreationDate="2022-04-24T21:49:30.887" Score="0" ViewCount="27" Body="&lt;p&gt;The paper &lt;a href=&quot;https://www.nature.com/articles/s41467-018-03424-4#Sec19&quot; rel=&quot;nofollow noreferrer&quot;&gt;A comprehensive evaluation of module detection methods for gene expression data&lt;/a&gt; says that: &amp;quot;We first want to provide an overview of the characteristics and performance of current module detection methods to guide the biologist in their choice&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;My problem is that I have no gold standards for my gene dataset and I would like to know if I can use the paper results to choose a module detection method without the need of external validation measures and how can I justificate this generalization? I am not finding the justificative about how can I use this guiding.&lt;/p&gt;&#xA;" OwnerUserId="9680" LastActivityDate="2022-04-24T21:49:30.887" Title="Can I use a benchmark to choose a clustering or module detection method for a gene dataset?" Tags="|gene-expression|clustering|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18982" PostTypeId="1" AcceptedAnswerId="18983" CreationDate="2022-04-25T10:49:00.020" Score="2" ViewCount="119" Body="&lt;p&gt;I have two very similar genomes in my bowtie2 database (nuc similarity &amp;gt; 90 %). When mapping with bowtie2 against this database, most of the reads that are mapped to one of these genomes have a MAPQ score of 0. Only at the positions that are different between the two genomes I get higher MAPQ scores.&lt;/p&gt;&#xA;&lt;p&gt;I also created a database with just one of the genomes, inspecting the MAPQ score of a single read. This single read gets a MAPQ score of 24 when the database contains only one genome, but drops to zero when two similar genomes are in the database.&lt;/p&gt;&#xA;&lt;p&gt;Is this the expected behavior?&#xA;My understanding of the MAPQ score computation is, that the alignment score of a read at a specific location is only compared to the score at other positions within the same reference, not across the whole database.&#xA;If the alignment scores for a read are then pretty close to each other, it is assumed that the read does not map uniquely, which is reflected in a low MAPQ score.&lt;/p&gt;&#xA;&lt;p&gt;So my question is, whether the alignment score of a read is compared to other alignments of this read within the same reference OR against alignments with the whole database index?&#xA;Changing the database, would then result in different MAPQ scores.&lt;/p&gt;&#xA;" OwnerUserId="15386" LastActivityDate="2022-04-26T02:16:38.077" Title="Is the Bowtie2 MAPQ score dependent on the database index?" Tags="|bowtie2|mapq|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18983" PostTypeId="2" ParentId="18982" CreationDate="2022-04-25T14:36:16.353" Score="1" Body="&lt;p&gt;Yes, this is expected. I am not sure what you mean by &amp;quot;reference&amp;quot; vs &amp;quot;database&amp;quot;, but what you do is to index a fasta file or a collection of fasta files. That index you create by that is what &lt;code&gt;bowtie2&lt;/code&gt; alignes against. It does not know whether the index consisted of one, ten or a thousand fasta files, it just maps against the entire index and then check whether it can uniquely (or not) place the reads. If you combined two genomes into one index and the genomes are very similar then indeed many reads will map equally well to them and that manifests as MAPQ 0 (or low).&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2022-04-25T14:36:16.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18984" PostTypeId="2" ParentId="18970" CreationDate="2022-04-25T18:09:57.027" Score="1" Body="&lt;p&gt;The reason is the RAM has been exceeded for Biopython's capabilities and you are encountering a 'RAM-bottleneck'. I've listed the options from 1 or 3 and recommend option 3c.&lt;/p&gt;&#xA;&lt;p&gt;The options are use:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;machine with much greater RAM&lt;/li&gt;&#xA;&lt;li&gt;acceleration coding techniques in Python (such parallelisation)&lt;/li&gt;&#xA;&lt;li&gt;Beast tools written in Java&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I assume this calculation is being performed on your laptop because in theory &amp;lt;1500 isn't much, but agreed there are ~1500 taxa.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Option 1&lt;/strong&gt; I personally don't think you are very far into the bottle-neck. Certainly a Cloud computer (e.g. GCP) could easily perform this calculation and maybe your the headnode of your local cluster. Usually the headnode has the most RAM.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Option 2&lt;/strong&gt; Code acceleration is worth keeping in mind because this achieves &amp;gt;&amp;gt; 1 log performance. Parallisation will solve this using an optimal chunk-size. However, large amounts of code can be needed. Code optimisation - Python is very sensitive to this because it uses C-libraries - for example, use &lt;code&gt;map&lt;/code&gt; and &lt;code&gt;reduce&lt;/code&gt; and avoid ANY regex in your Python code because IMO its a key weakness of Python. Pandas iterrows is famously slow. If the Biopython library is the problem code efficiency isn't an option (unless you want to rewrite Biopython). Label compression or header recoding techniques work, I'm not sure whether this has been done in Beast output already(?).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Option 3&lt;/strong&gt; The tools for dealing with this which are written and optimised to achieve rapid output are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/beast-dev/tracer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tracer&lt;/a&gt; - does lots of analysis on MCMC output. Its really geared towards assessing convergence, but it can do a lot of stuff.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.beast2.org/treeannotator/&quot; rel=&quot;nofollow noreferrer&quot;&gt;TreeAnnotator&lt;/a&gt; - makes a summary tree from the MCMC output you have&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;&lt;strong&gt;&lt;a href=&quot;https://github.com/alexeid/TreeStat2&quot; rel=&quot;nofollow noreferrer&quot;&gt;TreeStat2&lt;/a&gt;&lt;/strong&gt; &amp;lt;- I think this is what you are looking for.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-04-25T20:43:52.823" LastActivityDate="2022-04-25T20:43:52.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18985" PostTypeId="2" ParentId="18982" CreationDate="2022-04-26T02:16:38.077" Score="0" Body="&lt;p&gt;Yes, this is expected.&lt;/p&gt;&#xA;&lt;p&gt;If a read maps equally well to two different locations in the reference database, then the minimum probability that read maps to another location is 0.5. MAPQ scores are ten times the negative &lt;code&gt;log10&lt;/code&gt; of the &lt;a href=&quot;https://www.acgt.me/blog/2014/12/16/understanding-mapq-scores-in-sam-files-does-37-42&quot; rel=&quot;nofollow noreferrer&quot;&gt;probability of a mismatch&lt;/a&gt;, so a mismatch probably of 0.5 would lead to a &lt;em&gt;maximum&lt;/em&gt; MAPQ score of &lt;span class=&quot;math-container&quot;&gt;$-10*log_{10}(0.5) = 3$&lt;/span&gt;. If there's any doubt about the mapping (e.g. a mismatch in the sequenced read), or the read could map to more than two locations, then that probability will drop further. I would not be surprised if these MAPQ scores, where the location is impossible to precisely determine, were rounded down to 0.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-04-26T02:16:38.077" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18986" PostTypeId="1" CreationDate="2022-04-26T16:07:22.377" Score="2" ViewCount="24" Body="&lt;p&gt;I'm looking to retrieve Expression Atlas baseline expression results in JSON format, e.g. the results on &lt;a href=&quot;https://www.ebi.ac.uk/gxa/genes/ensg00000204301?bs=%7B%22homo%20sapiens%22%3A%5B%22ORGANISM_PART%22%5D%7D&amp;amp;ds=%7B%22kingdom%22%3A%5B%22animals%22%5D%7D#baseline&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Several years ago I did a similar thing but looking at differential expression results, and was able to access the json format for those &lt;a href=&quot;https://www.ebi.ac.uk/gxa/json/search/differential_results?geneQuery=%255B%257B%2522value%2522%253A%2522ENSG00000204301%2522%257D%255D&amp;amp;conditionQuery=&amp;amp;species=homo%20sapiens&quot; rel=&quot;nofollow noreferrer&quot;&gt;results here&lt;/a&gt;. However, I can't figure out how to access the baseline expression result equivalent.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know how to access the baseline expression results in JSON format, or know how I could find out how to do this?&lt;/p&gt;&#xA;" OwnerUserId="15349" LastActivityDate="2022-04-26T16:07:22.377" Title="How to retrieve Expression Atlas baseline expression results in JSON format?" Tags="|gene-expression|api|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="18987" PostTypeId="1" AcceptedAnswerId="19001" CreationDate="2022-04-26T18:47:43.750" Score="1" ViewCount="142" Body="&lt;p&gt;I'm getting the following error in a DESeq2 analysis and I can't seem to figure out the issue:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Error in `assays&amp;lt;-`(`*tmp*`, withDimnames = withDimnames, ..., value = `*vtmp*`) :  please use 'assay(x, withDimnames=FALSE)) &amp;lt;- value' or 'assays(x, withDimnames=FALSE)) &amp;lt;- value' when the dimnames on the supplied assay(s) are not identical to the dimnames on RangedSummarizedExperiment object 'x'&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is my code:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(DESeq2)&#xA;library(edgeR)&#xA;&#xA;url &amp;lt;- &amp;quot;https://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE63310&amp;amp;format=file&amp;quot;&#xA;utils::download.file(url, destfile=&amp;quot;GSE63310_RAW.tar&amp;quot;, mode=&amp;quot;wb&amp;quot;) &#xA;utils::untar(&amp;quot;GSE63310_RAW.tar&amp;quot;, exdir = &amp;quot;.&amp;quot;)&#xA;files &amp;lt;- c(&amp;quot;GSM1545535_10_6_5_11.txt&amp;quot;, &amp;quot;GSM1545536_9_6_5_11.txt&amp;quot;, &amp;quot;GSM1545538_purep53.txt&amp;quot;,&#xA;           &amp;quot;GSM1545539_JMS8-2.txt&amp;quot;, &amp;quot;GSM1545540_JMS8-3.txt&amp;quot;, &amp;quot;GSM1545541_JMS8-4.txt&amp;quot;,&#xA;           &amp;quot;GSM1545542_JMS8-5.txt&amp;quot;, &amp;quot;GSM1545544_JMS9-P7c.txt&amp;quot;, &amp;quot;GSM1545545_JMS9-P8c.txt&amp;quot;)&#xA;for(i in paste(files, &amp;quot;.gz&amp;quot;, sep=&amp;quot;&amp;quot;))&#xA;  R.utils::gunzip(i, overwrite=TRUE)&#xA;&#xA;files &amp;lt;- c(&amp;quot;GSM1545535_10_6_5_11.txt&amp;quot;, &amp;quot;GSM1545536_9_6_5_11.txt&amp;quot;, &#xA;           &amp;quot;GSM1545538_purep53.txt&amp;quot;, &amp;quot;GSM1545539_JMS8-2.txt&amp;quot;, &#xA;           &amp;quot;GSM1545540_JMS8-3.txt&amp;quot;, &amp;quot;GSM1545541_JMS8-4.txt&amp;quot;, &#xA;           &amp;quot;GSM1545542_JMS8-5.txt&amp;quot;, &amp;quot;GSM1545544_JMS9-P7c.txt&amp;quot;, &#xA;           &amp;quot;GSM1545545_JMS9-P8c.txt&amp;quot;)&#xA;&#xA;x &amp;lt;- readDGE(files, columns=c(1,3))&#xA;&#xA;samplenames &amp;lt;- substring(colnames(x), 12, nchar(colnames(x)))&#xA;&#xA;colnames(x) &amp;lt;- samplenames&#xA;group &amp;lt;- as.factor(c(&amp;quot;LP&amp;quot;, &amp;quot;ML&amp;quot;, &amp;quot;Basal&amp;quot;, &amp;quot;Basal&amp;quot;, &amp;quot;ML&amp;quot;, &amp;quot;LP&amp;quot;, &#xA;                     &amp;quot;Basal&amp;quot;, &amp;quot;ML&amp;quot;, &amp;quot;LP&amp;quot;))&#xA;x&lt;span class=&quot;math-container&quot;&gt;$samples$&lt;/span&gt;group &amp;lt;- group&#xA;&#xA;#DESeq2 Analysis&#xA;phenodata &amp;lt;- new(&amp;quot;AnnotatedDataFrame&amp;quot;, data = data.frame(celltype = x&lt;span class=&quot;math-container&quot;&gt;$samples$&lt;/span&gt;group))&#xA;counts &amp;lt;- x$counts&#xA;ids &amp;lt;- c(1,3,4,6,7,9)&#xA;&#xA;tmpcounts = counts[, ids]&#xA;tmpph = as(phenodata[ids, ], &amp;quot;data.frame&amp;quot;)&#xA;rownames(tmpph) &amp;lt;- colnames(tmpcounts)&#xA;&#xA;dds = DESeqDataSetFromMatrix(countData = tmpcounts, colData = tmpph, design = ~ tmpph$celltype)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here are my package versions&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; packageVersion(&amp;quot;DESeq2&amp;quot;)&#xA;[1] ‘1.30.1’&#xA;&#xA;&amp;gt; packageVersion(&amp;quot;SummarizedExperiment&amp;quot;)&#xA;[1] ‘1.20.0’&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4876" LastEditorUserId="3694" LastEditDate="2022-04-28T08:09:59.517" LastActivityDate="2022-04-28T11:02:43.880" Title="Error in DESeq2 analysis with dimnames" Tags="|rna-seq|deseq2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18988" PostTypeId="1" AcceptedAnswerId="18994" CreationDate="2022-04-26T21:18:57.893" Score="2" ViewCount="943" Body="&lt;p&gt;A standard approach for scRNA-Seq is to partition the single cells into individual clusters, then use a Wilcoxon test to find markers that characterize each cluster (or other statistical methods that consider single cells as replicates).&lt;/p&gt;&#xA;&lt;p&gt;Recently, there has been a push towards using pseudobulk approaches for &amp;quot;differential state&amp;quot; (DS) analysis, which avoid the pseudoreplication bias of considering single cells as replicates. See e.g. &lt;a href=&quot;https://www.nature.com/articles/s41467-020-19894-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Crowell (2020)&lt;/a&gt;, &lt;a href=&quot;https://www.nature.com/articles/s41467-021-21038-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zimmerman (2021)&lt;/a&gt;, &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/37/19/3243/6273181&quot; rel=&quot;nofollow noreferrer&quot;&gt;Thurman (2021)&lt;/a&gt;, &lt;a href=&quot;https://www.nature.com/articles/s41467-021-25960-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Squair (2021)&lt;/a&gt;, or the &lt;a href=&quot;http://bioconductor.org/books/3.14/OSCA.multisample/multi-sample-comparisons.html#testing-for-between-label-differences&quot; rel=&quot;nofollow noreferrer&quot;&gt;OSCA book&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, all the above papers seem to use pseudobulk methods for DS of a single cluster between two conditions (e.g. drug treatment), or occasionally to compare a specific pair of clusters. I haven't seen any example of using pseudobulk to find markers distinguishing clusters (comparisons of one cluster against all others).&lt;/p&gt;&#xA;&lt;p&gt;I can think of a few reasons, for example:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;these pseudobulk methods can be more computationally intensive,&lt;/li&gt;&#xA;&lt;li&gt;perhaps the Wilcoxon test is good enough to simply find markers,&lt;/li&gt;&#xA;&lt;li&gt;pseudobulk methods can only work when each cluster is represented in several samples, which is hard to guarantee for every single cluster in the dataset.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;On the other hand it seems to me a pseudobulk approach should yield more relevant markers (genes that are robustly enriched across biological replicates). So, am I missing something? Is there a statistical reason not to use pseudobulk approaches for finding cluster-specific markers?&lt;/p&gt;&#xA;" OwnerUserId="9497" LastActivityDate="2022-12-08T22:32:15.720" Title="scRNA-Seq pseudobulk for marker detection" Tags="|scrnaseq|differential-expression|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18989" PostTypeId="2" ParentId="14042" CreationDate="2022-04-26T23:42:00.987" Score="1" Body="&lt;p&gt;@swbarnes2 code is one correct way to find marker genes that differ between samples. You could also change the &lt;code&gt;Idents&lt;/code&gt; slot and then just use &lt;code&gt;FindAllMarkers&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;It's not clear what you mean by accurate in your comment. If you look for marker genes between samples (orig.ident) without clustering, Seurat will use expression data from all the cells attributed to each sample to find sample-specific markers. This approach can be right or wrong depending on the question you are asking.&lt;/p&gt;&#xA;&lt;p&gt;For example, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5878932/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; asked which genes were unique to each patient's tumor (see Figure 2D). To recreate their analysis, you would restrict your Seurat object to only include tumor cells (removing other cell types like immune cells and fibroblasts) and then perform &lt;code&gt;FindMarkers&lt;/code&gt; on sample origin. This answers which genes are specifically expressed on each patient's tumor cells, averaged over the different tumor cell subpopulations (in HNSC there are basal, differentiated, stressed, and pEMT subpopulations). This analysis could be a good idea if you are trying to find a sample-specific marker that is expressed on the majority of tumor cells in a sample. The downside to this approach is that it averages across tumor cell subpopulations. If there is a rare tumor subpopulation (say a set of cancer stem cells) that do not express the same markers as the majority of cells, the markers Seurat finds may not be expressed on this rare population.&lt;/p&gt;&#xA;&lt;p&gt;It doesn't make sense to look for sample specific markers when your Seurat object includes several different types of cells (epithelial cells, immune cells, stromal cells etc). In other words, you should probably restrict your analysis to some major cell type before looking for marker genes. If you don't, your results will be analogous to a bulk RNA-Seq differential expression, and will be significantly influenced by the cell type composition between samples which defeats the purpose of doing single cell RNA-sequencing. In this case, one could consider the results inaccurate, as it will be difficult to attribute marker genes to one cell type.&lt;/p&gt;&#xA;&lt;p&gt;Another way to interpret your question about accuracy is whether calling marker genes between samples provides accurate statistical inference (are the marker genes identified truly up/down in specific patients). If you are using single cell data to compare expression between conditions (normal vs tumor, control vs treated cells), it is advised to not use Seurat FindMarkers or similar tools when you have multiple samples. The problem is that FindMarkers treats each cell like its own sample, even though all cells from a sample are similar and thus correlated. This violates the IID assumption used by the Wilcox test and other methods used by FindMarkers and results in really low p-values and inflated false positive rates. In this scenario, recent studies suggest you should use a pseudo-bulk approach to prevent a high number of false positives. &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/muscat.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;muscat&lt;/a&gt; is one method that performs pseudo-bulk analysis, and its paper explains pseudo-bulk analysis in more detail.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt; If you are going to look for sample-specific (or other metadata-based) marker genes, it's probably a good idea to first restrict the analysis to a major cell type. If you are using single cell data to perform differential expression analysis comparing conditions, use pseudo-bulk approaches.&lt;/p&gt;&#xA;" OwnerUserId="6674" LastActivityDate="2022-04-26T23:42:00.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18991" PostTypeId="1" CreationDate="2022-04-27T04:26:16.170" Score="1" ViewCount="114" Body="&lt;p&gt;Could someone kindly recommend a tool or R package that can identify synonymous sites in a multiple sequence alignment?&lt;/p&gt;&#xA;&lt;p&gt;I wish to select those taxa for tree reconstruction and other downstream analyses within subset of the alignment.&lt;/p&gt;&#xA;&lt;p&gt;To respond to the questions below: yes, I am interested in identifying synonymous sites within a multiple sequence alignment (DNA) of ~70 virus samples with limited genetic diversity. My goal is to subset the multiple sequence alignment including synonymous sites only, because we believe that there are parallel non-synonymous mutations occurring that do not provide information on shared evolutionary history.&lt;/p&gt;&#xA;&lt;p&gt;Thank you @M_ for the really helpful information below. In Mega, the S tool identifies singleton sites rather than synonymous sites in a multiple sequence alignment. I'm wondering if there is another tool or a PAML option to subset the synonymous sites?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance and best!&lt;/p&gt;&#xA;" OwnerUserId="15401" LastEditorUserId="15401" LastEditDate="2022-04-27T17:41:44.930" LastActivityDate="2024-01-18T03:10:06.193" Title="Select synonymous sites from a multiple sequence alignment" Tags="|sequence-alignment|phylogenetics|phylogeny|synonymous-mutations|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18992" PostTypeId="1" CreationDate="2022-04-27T09:34:36.163" Score="3" ViewCount="80" Body="&lt;p&gt;If I'm looking for a specific SNP in my SNP-Chip data and it isn't there, are there any tools that let me quickly impute that SNP from surrounding SNPs rather than running a lengthy 'whole chromosome' imputation job?&lt;/p&gt;&#xA;&lt;p&gt;If so, roughly how many upstream and downstream SNPs are generally required to predict the haploblock 'accurately'?&lt;/p&gt;&#xA;" OwnerUserId="8875" LastEditorUserId="3694" LastEditDate="2022-04-27T10:20:26.000" LastActivityDate="2022-04-27T16:09:31.337" Title="Imputing small region of the genome" Tags="|snp|imputation|haplotypes|tools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18993" PostTypeId="2" ParentId="18992" CreationDate="2022-04-27T09:59:19.607" Score="3" Body="&lt;p&gt;I would take something like 1.5Mb either side of the SNP of interest (so a 3Mb chunk), subset that out from the target and reference panel and then perform imputation on that chunk alone. I think that size will give you more than enough LD, &lt;a href=&quot;https://www.mdpi.com/2073-4425/8/5/133/htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;since LD decays by about 500kB in most human populations by then.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://odelaneau.github.io/shapeit4/#documentation&quot; rel=&quot;nofollow noreferrer&quot;&gt;shapeit4 has an option to just impute a single region&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;shapeit4 \&#xA;    --input unphased.bcf \&#xA;    --map chr20.b37.gmap.gz \&#xA;    --region chr21:1231213-1413143 \ ## change this bit&#xA;    --reference reference.bcf \&#xA;    --output phased.bcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="3694" LastEditDate="2022-04-27T16:09:31.337" LastActivityDate="2022-04-27T16:09:31.337" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="18994" PostTypeId="2" ParentId="18988" CreationDate="2022-04-27T10:24:09.963" Score="1" Body="&lt;p&gt;The only reason I see for not using pseudobulks is lack of biological replication. Even though single-cell sequencing is common these days and standardized it is still expensive. If you have several conditions plus biological replicates and need many cells, maybe due to heterogeneity, the bill can &lt;strong&gt;easily&lt;/strong&gt; reach the 5-digit realm of money. Biological replication is always desired but sometimes simply not possible. We recently did a study in which we were interested in organ heterogeneity in different conditions and both mouse work and library prep was so extensive that even hash tagging the mice we had to pool was simply not possible. So we were bound to using each cell as a replicate. I prefer limma-trend for this, not the Wilcox test. It is not computationally very demanding, neither is pseudobulks. But even if it was, I would happily wait 30min rather than 10sec to get the best possible rather than the fastest results. Pseudobulks conceptually have the advantage that you basically are forced to have biological replication (because only then they make sense) so findings are intrinsically more robust. Theoretically, you can of course have biological replicates and still treat each cell as replicate, and then blocking or adding the biological replication information into the DE testing design, but the statistical rigor of methods used for pseudobulks (DESeq2, edgeR and other) is for me the reason to always use pseudobulks if I can.&lt;/p&gt;&#xA;&lt;p&gt;I personally use pseudobulks also for markers. After all marker detection comes downstream of DE testing by aggregating the results into lists that contain unique or enriched genes per cluster. I see no reason to use multiple testing strategies.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2022-04-28T15:00:45.903" LastActivityDate="2022-04-28T15:00:45.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18995" PostTypeId="1" CreationDate="2022-04-27T11:15:46.870" Score="2" ViewCount="304" Body="&lt;p&gt;I have an RNA-seq dataset and I am using &lt;code&gt;DEseq2&lt;/code&gt; to find differentially expressed genes between the two groups.  I used pre-filtering to remove any genes that have no counts or only one count across the samples.&#xA;furthermore, I want to:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;remove genes in low counts by using a base mean threshold.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;remove genes that have low counts compared to the rest of the genes.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is there a common threshold used for the baseMean or a way to work out what this threshold should be?&lt;/p&gt;&#xA;" OwnerUserId="15404" LastEditorDisplayName="user3051" LastEditDate="2022-04-27T12:37:44.973" LastActivityDate="2022-04-27T12:37:44.973" Title="baseMean threshold" Tags="|r|rna-seq|deseq2|gene-expression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18996" PostTypeId="2" ParentId="18991" CreationDate="2022-04-27T12:23:15.430" Score="0" Body="&lt;p&gt;I proposed Syn-SCAN (below). This was used for HIV and as you are looking at viruses seems a good fit.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;There's a simpler way&lt;/strong&gt; ... just make a invariable amino acid alignment, i.e. discard all amino acid variation. MEGA will do that, or use an algorithm. You then simply check the remaining nucleotide sequences for identity and discard the identity (lots of algorithms do this).&lt;/p&gt;&#xA;&lt;p&gt;What you are left with is, is what you seek, PURE SYNONYMOUS mutations.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;discard taxa with amino acid variation&lt;/li&gt;&#xA;&lt;li&gt;retain taxa nucleotides with variation.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;However, just discarding amino acid variation and retaining everything else will either by identity or synonymous. To be honest, its a virus so the amount of sequence identity between taxa is likely to be very little for protein loci.&lt;/p&gt;&#xA;&lt;p&gt;Its really that simple. I honestly wouldn't over complicate it.&lt;/p&gt;&#xA;&lt;p&gt;Sorry for the singleton vs. synonymous error, I haven't looked at MEGA in ages. Its a bit bizarre it discarded the only thing it was actually used for.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The short answer to the question is the 3rd codon position is the traditional approximation for synonymous mutation. A small proportion of 1st codon position sites are neutral and a small proportion of 3rd codon position are non-synonymous.&lt;/p&gt;&#xA;&lt;p&gt;I agree with @terdon about and @Community regarding additional information sought, because synonymous mutations are heavily subject to compositional bias and that is a big subject. The key issue is the whether the taxa/species in the alignments are known for compositional bias and the genetic distance between different members of the alignment.&lt;/p&gt;&#xA;&lt;p&gt;The traditional package used for synonymous mutations is/was &lt;strong&gt;MEGA&lt;/strong&gt;, &lt;a href=&quot;https://academic.oup.com/mbe/article/38/7/3022/6248099&quot; rel=&quot;nofollow noreferrer&quot;&gt;currently version 11&lt;/a&gt;. Albeit, recent 'upgrades' seem to remove the synonymous distance matrix calculation. HOWEVER, they may have transferred this capability onto its what appears to be upgraded GUI alignment viewer. What I think is if you:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;load the alignment into the GUI;&lt;/li&gt;&#xA;&lt;li&gt;click alignment (big button far left of menu);&lt;/li&gt;&#xA;&lt;li&gt;you will now see the alignment;&lt;/li&gt;&#xA;&lt;li&gt;in the middle of the menu bar are 4 buttons 'C', 'V', 'Pi' and 'S'&lt;/li&gt;&#xA;&lt;li&gt;I think 'S' = synonymous&lt;/li&gt;&#xA;&lt;li&gt;All synonymous sites in the alignment are now highlighted and there is likely a method of extracting this information.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;C=constant, V=variable Pi=Nei approximation of diversity (population genetics)&lt;/p&gt;&#xA;&lt;p&gt;If this is correct then those are the subset sought, please note however you will need to select/deselect the taxa manually and reassess the synonymous sites after each 'point and click' iteration (the S button will remain active I suspect between each selection/deselection). What you appear to want is a synonymous tree and this function I believe was removed from MEGA.&lt;/p&gt;&#xA;&lt;p&gt;MEGA is rarely used except for teaching because its functionality is available elsewhere that permit a given algorithm to be integrated into a data pipeline, which MEGA would not permit (might have recently changed). Hence my recollection of MEGA is a little vague.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The package used for synonymous work is &lt;a href=&quot;http://abacus.gene.ucl.ac.uk/software/paml.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PAML&lt;/a&gt; and remains the undisputed standard without question in the context of formal selection analysis, which is heavily dependent on a estimate of synonymous mutations. What you refer to as 'synonymous mutation' isn't the same as the phylogenetic definition, i.e. your definition depends on the other members of the alignment, whereas this isn't an issue in the phylogenetic definition (because the tree circumvents this, i.e. delineates subsets) except if compositional bias occurs ... huge amounts of work is/has been dedicated to compositional bias.&lt;/p&gt;&#xA;&lt;p&gt;Final point is the widely used phylogenetic library in R called &lt;code&gt;ape&lt;/code&gt; will not perform a synonymous mutation calculation - I think, however phangorn library in R might do, but is more likely within the context of selection analysis. Thus to answer the question directly, I don't think R will do this calculation.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;It transpires S= singleton (sorry) ....&lt;/p&gt;&#xA;&lt;p&gt;Have a look at Syn-SCAN. The link is here &lt;a href=&quot;https://hivdb.stanford.edu/pages/synscan.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hivdb.stanford.edu/pages/synscan.html&lt;/a&gt; . I have never used it and the download link is on the website. The good thing is its written in Perl so it would be easy to modify and check (if you can read/write Perl).&lt;/p&gt;&#xA;&lt;p&gt;PAML: you can't detach the synonymous mutation calculation from the non-synonymous mutation calculation, without delving into the C-code (which I wouldn't recommend).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-04-28T01:34:18.787" LastActivityDate="2022-04-28T01:34:18.787" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="18997" PostTypeId="2" ParentId="18995" CreationDate="2022-04-27T12:27:20.797" Score="1" Body="&lt;p&gt;Adding my answer from the &lt;a href=&quot;https://support.bioconductor.org/p/9143776/#9143785&quot; rel=&quot;nofollow noreferrer&quot;&gt;crosspost&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;I would not use the baseMean for any filtering as it is (at least to me) hard to deconvolute. You do not know why the baseMean is low, either because there is no difference between groups and the gene is just lowly-expressed (and/or short), or it is moderately expressed in one but off in the other. The baseMean could be the same in these two scenarios. If you filter I would do it on the counts. So you could say that all or a fraction of samples of at least one group must have 10 or more counts. That will ensure that you remove genes that have many low counts or zeros across the groups rather than nested by group, the latter would be a good DE candidate so it should not be removed. Or you do that automated, e.g. using the edgeR function filterByExpr.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2022-04-27T12:27:20.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="18999" PostTypeId="1" CreationDate="2022-04-28T03:23:32.577" Score="0" ViewCount="80" Body="&lt;p&gt;Forgive the basic question here, but I'm super novice at this ...&lt;/p&gt;&#xA;&lt;p&gt;I have a series of paired-end RNA fastq files (e.g. &lt;code&gt;SRR10720226_1.fastq.gz&lt;/code&gt; and &lt;code&gt;SRR10720226_2.fastq.gz&lt;/code&gt;).  I'm trying to figure out what software I need to use to translate those into proteins.&lt;/p&gt;&#xA;&lt;p&gt;I was trying to use &lt;code&gt;biopython&lt;/code&gt; to do this using the &lt;code&gt;translate&lt;/code&gt; method.  The problem is &lt;code&gt;biopython&lt;/code&gt; doesn't seem to consider both files together, only one &lt;code&gt;fastq&lt;/code&gt; at a time.&lt;/p&gt;&#xA;&lt;p&gt;Which tool should I use to work with the reads together?  Is there anything else I should be doing besides just translating to protein?&lt;/p&gt;&#xA;" OwnerUserId="15408" LastActivityDate="2022-04-28T03:49:10.857" Title="Pipeline for paired end RNA sequence data to proteins" Tags="|python|fastq|rna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19000" PostTypeId="2" ParentId="18999" CreationDate="2022-04-28T03:49:10.857" Score="1" Body="&lt;p&gt;This is a much more complicated question than it might seem.&lt;/p&gt;&#xA;&lt;p&gt;First, you need to understand how RNA-Seq works, and what your data really is. Your &amp;quot;paired-end files&amp;quot; contain reads, which will contain fragments of transcripts.&lt;/p&gt;&#xA;&lt;p&gt;Since those are only fragments of transcripts, you don't know what frame they are in, so you don't have a single possible translation. And each read would only give you a small part of the protein.&lt;/p&gt;&#xA;&lt;p&gt;Since the data is for humans, your best course of action would be to first, align the reads to the annotated human genome (or transcriptome). Then, count the number of reads for each gene (or transcript), and decide on a condition to call a gene (or transcript) present in the sample (typically by enrichment between two sets of samples, in your case a threshold might be good enough). Finally, for the genes (or transcripts) present in the sample, find out what protein (or isoform) they correspond to, and if needed retrieve the sequence.&lt;/p&gt;&#xA;&lt;p&gt;To be clear, you will have to learn a lot more to get that done. For the first step, easier tools might include &lt;a href=&quot;https://pachterlab.github.io/kallisto/about&quot; rel=&quot;nofollow noreferrer&quot;&gt;kallisto&lt;/a&gt; or &lt;a href=&quot;https://salmon.readthedocs.io/en/latest/salmon.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;salmon&lt;/a&gt;. You would probably want to use biological replicates; SRP237926 seems to have several samples, reading the paper would be necessary to determine whether they are useful.&lt;/p&gt;&#xA;" OwnerUserId="9497" LastActivityDate="2022-04-28T03:49:10.857" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="19001" PostTypeId="2" ParentId="18987" CreationDate="2022-04-28T11:02:43.880" Score="1" Body="&lt;p&gt;What works is:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;dds = DESeqDataSetFromMatrix(countData = data.frame(tmpcounts), colData = tmpph, design = ~celltype)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Mind the &lt;code&gt;data.frame()&lt;/code&gt; around the count data object (which was a matrix) and removal of &lt;code&gt;tmpph$&lt;/code&gt; for the design because design should reference one or more columns of the colData. In newer versions of DESeq2 that by the way works without the &lt;code&gt;data.frame()&lt;/code&gt; but I am not sure why. You might want to dig that out yourself, looking at the source code of how exactly the package (SummarizedExperiment) feeds data as an &lt;code&gt;assay&lt;/code&gt; into a SummarizedExperiments object.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2022-04-28T11:02:43.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19003" PostTypeId="1" AcceptedAnswerId="19443" CreationDate="2022-04-28T15:01:38.900" Score="0" ViewCount="778" Body="&lt;p&gt;I have downloaded a predicted structure from AlphaFold as a pdb file (&lt;a href=&quot;https://alphafold.com/entry/O75376&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://alphafold.com/entry/O75376&lt;/a&gt;) and loaded it into Pymol (2.3.0). There is quite a large portion of the structure that was modelled with very low confidence (pLDDT &amp;lt; 50). Is there a way to select these regions in the structure so that I can assign a different representation to them? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="10386" LastActivityDate="2022-07-21T22:36:25.177" Title="Pymol: select low confidence regions from AlphaFold pdb file" Tags="|protein-structure|pymol|structural-biology|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19005" PostTypeId="1" CreationDate="2022-04-29T04:31:07.103" Score="0" ViewCount="39" Body="&lt;p&gt;I have a PDB file in which the indexes of atoms are repeated (or better say refreshed) every 99999.&lt;/p&gt;&#xA;&lt;p&gt;I am using MDtraj software to analyze this pdf file, so these repeating indexes cause some problems.&lt;/p&gt;&#xA;&lt;p&gt;Does everyone encounter this problem and how can I fix it?&lt;/p&gt;&#xA;" OwnerUserId="15362" LastEditorUserId="650" LastEditDate="2022-04-29T15:23:58.807" LastActivityDate="2023-01-24T17:08:21.023" Title="Repeating indexes of atoms in PDB file" Tags="|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19006" PostTypeId="2" ParentId="19005" CreationDate="2022-04-29T09:43:05.910" Score="1" Body="&lt;p&gt;Inconsistencies like randomly repeated atom indices, maxed out indices at 99999, integer overflowing indices to the negatives (yup, there's a program that has this bug), bad spacing, jumbled order etc. in a PDB are tolerated differently in different programs. Some will carp at missing atoms or non-standard atom names, which are not formatting errors.&lt;/p&gt;&#xA;&lt;p&gt;In terms of atom indices, these are used in &lt;code&gt;CONECT&lt;/code&gt; entries, but several programs bond by proximity or by known topology —&lt;code&gt;CONECT&lt;/code&gt; entries are absent for canonical amino acids. Repeated residue indices and chain letter repeats cause issues with &lt;code&gt;LINK&lt;/code&gt; and often with peptide chain sequence.&lt;/p&gt;&#xA;&lt;p&gt;However, in a multimodel PDB the numbers in each model are expected to be reset at every MODEL so if the atom indices reset after the end of a model this is 100% expected.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2022-04-29T09:43:05.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19009" PostTypeId="1" CreationDate="2022-04-29T17:45:51.970" Score="2" ViewCount="560" Body="&lt;p&gt;I am trying to access the normalized expression in both edgeR and DESeq2, yet the results are different. Does anyone know why?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;How to get normalized expression using edgeR:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&#xA;y &amp;lt;- DGEList(...)&#xA;y &amp;lt;- calcNormFactors(y, method = &amp;quot;TMM&amp;quot;)&#xA;v &amp;lt;- voom(y, design, plot=FALSE)&#xA;&#xA;# normalized expression&#xA;head(v$E)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;How to get normalized expression using DESeq2:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&#xA;dds &amp;lt;- DESeqDataSetFromMatrix(...)&#xA;dds &amp;lt;- estimateSizeFactors(dds)&#xA;&#xA;# normalized expression&#xA;normalized_counts &amp;lt;- counts(dds, normalized = TRUE)&#xA;head(normalized_counts)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Anyone know why the results are different? Aren't they both doing a version of log(cpm) under the hood? Or am I not comparing the right results from DESeq2?&lt;/p&gt;&#xA;" OwnerUserId="4876" LastEditorDisplayName="user3051" LastEditDate="2022-04-29T19:51:48.673" LastActivityDate="2022-04-29T19:53:54.547" Title="What is the difference between Normalized Expression in EdgeR vs DESeq2?" Tags="|rna-seq|deseq2|normalization|limma|edger|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19010" PostTypeId="1" AcceptedAnswerId="19012" CreationDate="2022-04-29T17:51:17.593" Score="0" ViewCount="277" Body="&lt;p&gt;I would like to get chromosome arms' coordinates for genome reference &lt;code&gt;hs37d5&lt;/code&gt; (&lt;a href=&quot;ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/phase2_reference_assembly_sequence/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp-trace.ncbi.nih.gov/1000genomes/ftp/technical/reference/phase2_reference_assembly_sequence/&lt;/a&gt;), but I could not find them either online, or the directory listed above.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to extract such coordinates from one of the available files (e.g., from the fasta)?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2022-04-29T22:43:47.067" LastActivityDate="2022-04-29T22:43:47.067" Title="Extract chromosome arms' coordinates for specific reference" Tags="|reference-genome|chromosomes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19011" PostTypeId="2" ParentId="19009" CreationDate="2022-04-29T19:48:00.940" Score="2" Body="&lt;p&gt;First, and most obviously, two independent methods will never fully agree.&lt;/p&gt;&#xA;&lt;p&gt;Second, while the DESeq2-related code in fact gets the normalized counts based on the size factors the edgeR/voom code extract the voom-transformed counts, but that is not the normalized counts itself. I would need to go back to the limma docs and paper to check what exactly the &lt;code&gt;E&lt;/code&gt; slot stores, but I think (please edit for a definite answer) it's normalized counts after the voom transformation. In contrast, the edgeR normalized counts in the sense of what woul be most comparable to the DESeq2 normalized counts would be obtained with the &lt;code&gt;edgeR::cpm()&lt;/code&gt; function. Check the manual. At no point of either the limma or edgeR manual it says that &lt;code&gt;v$E&lt;/code&gt; should be used for anything outside of the linear modelling. It clearly recommends to use &lt;code&gt;cpm()&lt;/code&gt;. Limma and edgeR are packages that have hundreds of forum questions and blog entries, many functions do exist and the packages change over time, so indeed it can be confusing what to do, but therefore I strongly recommend to consult the current manual at Bioconductor as the sole reliable reference on how to get values from the objects and how to run your analysis.&lt;/p&gt;&#xA;&lt;p&gt;By the way, don't be surprised that the DESeq2 and edgeR normalized counts will be on different scales. DESeq2 gets the counts by dividing the raw counts by the size factors while edgeR first calculates naive per-million counts and then corrects these with the TMM factors calculated by &lt;code&gt;calcNormFactors&lt;/code&gt;, so the magnitude of the counts can be quite different, simply by how it is calculated.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastEditorDisplayName="user3051" LastEditDate="2022-04-29T19:53:54.547" LastActivityDate="2022-04-29T19:53:54.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19012" PostTypeId="2" ParentId="19010" CreationDate="2022-04-29T22:43:33.820" Score="1" Body="&lt;p&gt;The chromosome sizes can be retrieved with (to use for sanity check):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; ./faSize -veryDetailed -tab hs37d5.fa.gz&#xA;1   249250621   23970000    225280621   225280621   0&#xA;2   243199373   4994855 238204518   238204518   0&#xA;3   198022430   3225295 194797135   194797135   0&#xA;4   191154276   3492600 187661676   187661676   0&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(first column is chromosome ID, second column is chromosome size).&lt;/p&gt;&#xA;&lt;p&gt;Afterwards, the centromere coordinates are retrieved through (in this case I used hg19 since I'm working with &lt;code&gt;hs37d5&lt;/code&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; curl -s &amp;quot;http://hgdownload.cse.ucsc.edu/goldenPath/hg19/database/cytoBand.txt.gz&amp;quot; \&#xA;  | gunzip -c | grep acen | sed 's/chr//g' | sort -n &amp;gt; hg19.centromeres&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The coordinates for each chromosome arm can be computed from the chromosome size and centromere coordinates.&lt;/p&gt;&#xA;&lt;p&gt;N.B. As per &lt;a href=&quot;http://genome.ucsc.edu/FAQ/FAQtracks#tracks1&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC documentation&lt;/a&gt;, cytoband's start is 0-based, while end is 1-based.&#xA;As a sanity check you might want to compare the end of the last cytoband for each chromosome, with the chromosome size (they should be the same, since end is 1-based).&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2022-04-29T22:43:33.820" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19013" PostTypeId="1" CreationDate="2022-04-30T01:28:59.893" Score="1" ViewCount="32" Body="&lt;p&gt;I've currently run pathway analysis on a set of genes. I'm now trying to figure out how to see which genes are driving each pathway.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cleaned_ductal_markers &amp;lt;- tumor_markers&#xA;markers15  = cleaned_ductal_markers %&amp;gt;% filter(cluster==&amp;quot;PSEUDOPAPILLARY&amp;quot;, avg_logFC&amp;gt;0.5, p_val_adj&amp;lt;0.01)&#xA;markers15&#xA;&#xA;#head(genes_map)&#xA;markers_entrez = genes_map[which(genes_map&lt;span class=&quot;math-container&quot;&gt;$SYMBOL %in% markers15$&lt;/span&gt;gene),&amp;quot;ENTREZID&amp;quot;]&#xA;kk &amp;lt;- enrichKEGG(gene         = markers_entrez,&#xA;                universe      = genes_map[, &amp;quot;ENTREZID&amp;quot;],&#xA;                 organism     = 'hsa',&#xA;                 pvalueCutoff = 0.05)&#xA;#head(kk)&#xA;dotplot(kk, showCategory=10)&#xA;options(repr.plot.width=7, repr.plot.height=5)&#xA;emapplot(kk, showCategory=12, font.size =16) + set_graph_style(family = &amp;quot;Arial Narrow&amp;quot;, face = &amp;quot;plain&amp;quot;,  text_size = 15, text_colour = &amp;quot;black&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm now given a list of pathways but now I'm trying to see which genes are driving those pathways.&lt;/p&gt;&#xA;&lt;p&gt;How do I see the genes?&lt;/p&gt;&#xA;" OwnerUserId="6827" LastEditorUserId="6827" LastEditDate="2022-05-02T14:35:59.573" LastActivityDate="2022-05-02T14:35:59.573" Title="What genes are driving a certain pathway" Tags="|single-cell|pathway|go-enrichment|kegg|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19014" PostTypeId="1" CreationDate="2022-05-01T06:52:26.263" Score="2" ViewCount="109" Body="&lt;p&gt;I built Krakenuniq on an HPC, and I've been trying to run it on my data using the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;krakenuniq --db DBDIR --threads 1 All_reads.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This definitely isn't powerful enough to get any reasonable output, but this is just a test run. However, I got the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;classify: unable to mmap ./DBDIR/database.kdb: Invalid argument&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I haven't been able to find anything about this error online. DBDIR is 1TB, which seemed huge, so I was wondering if there might have been a problem with the build, but I saw no issues in the log.&lt;/p&gt;&#xA;" OwnerUserId="15345" LastEditorUserId="15345" LastEditDate="2022-05-01T07:28:13.167" LastActivityDate="2022-07-20T17:15:35.370" Title="Krakenuniq mmap error" Tags="|metagenome|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="19015" PostTypeId="1" CreationDate="2022-05-01T09:05:16.663" Score="3" ViewCount="58" Body="&lt;p&gt;But I have some weeks fighting with this issue about phylogenetic tree building to use in a phyloseq object in order to calculate beta-diversity metrics that takes into account tree distance branches metrics.&lt;/p&gt;&#xA;&lt;p&gt;I have one tree for Archea and another tree for bacteria (both unrooted), created from &lt;a href=&quot;http://www.iqtree.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;IQTREE&lt;/a&gt; from the protein alignments obtained by GTDB-Tk software. I tried to use the software &lt;a href=&quot;https://github.com/computations/root_digger&quot; rel=&quot;nofollow noreferrer&quot;&gt;Root-Digger&lt;/a&gt;, but &lt;a href=&quot;https://github.com/computations/root_digger/issues/9&quot; rel=&quot;nofollow noreferrer&quot;&gt;only works with nucleotide alignments and not protein alignments&lt;/a&gt;. Do you know another software capable (working with protein alignments) to infer more objectively the root rather than define arbitrarily a root through ape R package?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Root-digger is described as &amp;quot;a non-reversible Markov model&amp;quot;, which would be assumed to be a directional mutation matrix, thus T-&amp;gt;A would have a different probability to A-&amp;gt;T using Bayesian approach to branch-length estimation&lt;/p&gt;&#xA;&lt;p&gt;IQ-TREE is an ultra-rapid method for non-parametric bootstrapping which can be essential for extremely large phylogenies.&lt;/p&gt;&#xA;" OwnerUserId="15406" LastEditorUserId="3848" LastEditDate="2022-05-02T15:47:49.130" LastActivityDate="2022-05-02T16:08:08.973" Title="Phylogenetic tree rooting in shotgun metagenomics" Tags="|phylogenetics|phylogeny|metagenome|scaffold|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19016" PostTypeId="1" AcceptedAnswerId="19027" CreationDate="2022-05-01T15:59:48.047" Score="0" ViewCount="565" Body="&lt;p&gt;I have a cluster that is set up with slurm and I seek to use Cromwell to dispatch jobs to the slurm scheduler. I followed the description explained on &lt;a href=&quot;https://cromwell.readthedocs.io/en/stable/backends/SLURM/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cromwells Documentation Website&lt;/a&gt;. And created a configuration file called &lt;code&gt;cromwellconfig.conf&lt;/code&gt;, however when I run it using the command &lt;code&gt;java -Dconfig.file=/lccs/home/logio9/Configure_Cromwell/cromwellconfig.conf \ -jar cromwell-77.jar run test-workflow.wdl&lt;/code&gt; I get an error parsing the config file. I am confused as to where the error is, bellow is the contents of the configuration file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;include required(classpath(&amp;quot;application&amp;quot;))&#xA;&#xA;backend {&#xA;  default = SLURM&#xA;  providers {&#xA;    SLURM {&#xA;      actor-factory = &amp;quot;cromwell.backend.impl.sfs.config.ConfigBackendLifecycleActorFactory&amp;quot;&#xA;      config {&#xA;        runtime-attributes = &amp;quot;&amp;quot;&amp;quot;&#xA;        Int runtime_minutes = 600&#xA;        Int cpus = 2&#xA;        Int requested_memory_mb_per_core = 8000&#xA;        String queue = &amp;quot;short&amp;quot;&#xA;        String jobname = &amp;quot;testwork&amp;quot;&#xA;        String cwd = &amp;quot;/lccs/home/logio9/Configure_Cromwell&amp;quot;&#xA;        String err = &amp;quot;errors.output&amp;quot;&#xA;        String out = &amp;quot;ouput.out&amp;quot;&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        &#xA;      &#xA;        submit = &amp;quot;&amp;quot;&amp;quot;&#xA;            sbatch -J &lt;span class=&quot;math-container&quot;&gt;${jobname} -D $&lt;/span&gt;{cwd} -o &lt;span class=&quot;math-container&quot;&gt;${out} -e $&lt;/span&gt;{err} -t &lt;span class=&quot;math-container&quot;&gt;${runtime_minutes} -p $&lt;/span&gt;{queue} \&#xA;            &lt;span class=&quot;math-container&quot;&gt;${&quot;-c &quot; + cpus} \&#xA;        --mem-per-cpu $&lt;/span&gt;{requested_memory_mb_per_core} \&#xA;            --wrap &amp;quot;/bin/bash &lt;span class=&quot;math-container&quot;&gt;${script}&quot;&#xA;    &quot;&quot;&quot;  &#xA;    kill = &quot;scancel $&lt;/span&gt;{job_id}&amp;quot;&#xA;        check-alive = &amp;quot;squeue -j ${job_id}&amp;quot;&#xA;        job-id-regex = &amp;quot;Submitted batch job (\\d+).*&amp;quot;&#xA;      }&#xA;    }&#xA;  }&#xA;}&#xA;  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the test wdl script I made, it copies a very large directory into another directory.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;version 1.0&#xA;&#xA;workflow Greetings {&#xA;    call SayHello {&#xA;    }&#xA;}&#xA;&#xA;task SayHello {&#xA;    command &amp;lt;&amp;lt;&amp;lt;&#xA;        cp -R /lccs/home/logio9/Configure_Cromwell/tf1 /lccs/home/logio9/Configure_Cromwell/tf2&#xA;&#xA;    &amp;gt;&amp;gt;&amp;gt;&#xA;    output {&#xA;        Array[String] out = read_lines(stdout())&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If anyone has had experience with this, please do share a bit on how exactly this configuration works.&lt;/p&gt;&#xA;" OwnerUserId="15246" LastActivityDate="2022-05-03T09:38:33.527" Title="How to configure Cromwell to work with slurm?" Tags="|wdl|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19018" PostTypeId="1" CreationDate="2022-05-02T08:21:25.247" Score="1" ViewCount="133" Body="&lt;p&gt;For some weeks I'm fighting with an issue about phylogenetic tree building to use in a phyloseq object in order to calculate beta-diversity metrics that takes into account phylogenetic distance.&lt;/p&gt;&#xA;&lt;p&gt;I’m trying to create a phylogenetic tree from the representative genomes of &lt;a href=&quot;https://progenomes.embl.de/data/repGenomes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;proGenomes v.2.1 database&lt;/a&gt;. However, I do not have a representative genome for each taxa id. In contrast, I have a multifasta files for more than 100 contigs for each taxa ID (file freeze12.contigs.representatives.fasta.gz).&lt;/p&gt;&#xA;&lt;p&gt;For each taxaID (the first numbers in each header of the multifasta file is NCBI taxon ID), I would like to bin the contigs into the biggest scaffold possible, and then perform an alignment (through MUSCLE as examples) followed by RaXML-NG to construct a Newick tree.&lt;/p&gt;&#xA;&lt;p&gt;However, I cannot find any software for binning contigs with only the contigs as inputs. &lt;a href=&quot;https://bitbucket.org/berkeleylab/metabat/src/master/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MetaBAT&lt;/a&gt; needs bam files from shotgun read mappings to the contigs in question (not present in the proGenomes database). I found another software (&lt;a href=&quot;https://github.com/metageni/Scaffold_builder&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scaffold_builder&lt;/a&gt;), but it needs a representative reference genome. And in microbiology I have more than 100 complete genomes for each NCBI tax id...&lt;/p&gt;&#xA;&lt;p&gt;For this reason, I ask here an open question if exists some software that can perform of contigs taking into account only a multifasta file of contigs?&lt;/p&gt;&#xA;" OwnerUserId="15406" LastEditorUserId="2108" LastEditDate="2022-05-06T22:59:15.160" LastActivityDate="2022-05-26T07:05:55.727" Title="Phylogenetic tree building from proGenomes database for shotgun metagenomics" Tags="|phylogenetics|sequence-alignment|phylogeny|scaffold|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19019" PostTypeId="1" AcceptedAnswerId="19041" CreationDate="2022-05-02T11:29:28.847" Score="2" ViewCount="87" Body="&lt;p&gt;I am new in ATAC analysis.&lt;/p&gt;&#xA;&lt;p&gt;I have a problem about how to transfer ATAC peak to gene symbol.&lt;/p&gt;&#xA;&lt;p&gt;Here is the ATAC-seq: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM3271041&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM3271041&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;GSM3271041_ATAC_sciCAR_A549_peak.txt.gz&lt;/code&gt; file contains five columns : &lt;code&gt;(id, peak, chr, start, end)&lt;/code&gt;. What I am doing is to use &lt;code&gt;bedtools&lt;/code&gt; to transfer locations in chromosome to get gene symbols.&lt;/p&gt;&#xA;&lt;p&gt;Here is an example from &lt;code&gt;GSM3271041_ATAC_sciCAR_A549_peak.txt.gz&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;id,peak,chr,start,end&#xA;9,1-68591-69091,1,68591,69091&#xA;10,1-91105-91605,1,91105,91605&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I use bedtools to get the gene symbol of &lt;code&gt;id=9&lt;/code&gt; , that is &lt;code&gt;OR4F5&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I want to use these ''symbol-transfered'' ATAC-seq to integrate with RNA-seq which has gene symbols alreadly.&lt;/p&gt;&#xA;&lt;p&gt;Am I right to do so ?&lt;/p&gt;&#xA;" OwnerUserId="15425" LastEditorUserId="15425" LastEditDate="2022-05-03T02:47:32.597" LastActivityDate="2022-05-06T15:47:58.470" Title="ATAC peak anatations" Tags="|atac-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19020" PostTypeId="2" ParentId="19015" CreationDate="2022-05-02T15:43:17.940" Score="2" Body="&lt;p&gt;The answer really is that a root is determined using external evidence, rather than from the internal branches.&lt;/p&gt;&#xA;&lt;p&gt;An algorithm such as root digger can be used to estimate the root placement based on a branch that for example might minimise the anomalous branch lengths in all other taxa. Excessive comparative branch-lengths could indicate an error in rooting the tree and this can be the basis of the estimate. However, it is an estimate.&lt;/p&gt;&#xA;&lt;p&gt;R libraries such as &lt;code&gt;ape&lt;/code&gt; are more likely to simply mid-point root the tree, which isn't really cool, but its easy to calculate.&lt;/p&gt;&#xA;&lt;p&gt;The three approaches for rooting are either:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Using a known outgroup to the taxa of interest, which unequivocally predates their divergence.&lt;/li&gt;&#xA;&lt;li&gt;External information such the evolution of key morphological features of the organism.&lt;/li&gt;&#xA;&lt;li&gt;Published studies.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;em&gt;Point 1&lt;/em&gt; This can be used singly to find the root - a &amp;quot;root-finding tree&amp;quot; - and the tree of interest presented by constructing a second tree without this root in the alignment. For example the Archaea predate bacteria, one tree determines where the Archaea attach to taxa of interest. If the outgroup is very distinct this could cause artefacts.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Point 2&lt;/em&gt; Finding an outgroup much close to the taxa of interest using external information, for example gram-negative versus gram-positive bacteria, e.g. if all the bacteria in your analysis were one or the other and again do the two tree strategy (point 1).&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Point 3&lt;/em&gt; Using large kingdom wide phylogenies from published studies to determine where the most ancestral branch in your taxa.&lt;/p&gt;&#xA;&lt;p&gt;This is not an exhaustive list about how to use external information to determine the location of the root, but it is very widely recognised.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Directional versus reversible matrices&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Directional matrix (e.g. most options in Beast) need to know where the root is, or they will 'guess it': ideally and this information can be feed into the algorithm when it is started. This is important if you are doing molecular dating.&lt;/p&gt;&#xA;&lt;p&gt;In fact, you can do this even with reversible matrices (e.g. RAxML) but it makes no difference to the answer.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Manual re-rooting&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you have identified the root and the phylogeny you have does not represent that root correctly you can &amp;quot;re-root&amp;quot; the tree using the outgroup identified from point 1 to 3. A program such FigTree are cool for re-rooting. Re-rooting is perfectly cool providing you present the methodology that identifies the root taxa or taxon (singular).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-05-02T16:08:08.973" LastActivityDate="2022-05-02T16:08:08.973" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19022" PostTypeId="1" CreationDate="2022-05-02T17:54:46.823" Score="0" ViewCount="736" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://github.com/jokergoo/ComplexHeatmap/issues/913&quot; rel=&quot;nofollow noreferrer&quot;&gt;github&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm trying to run the function HeatmapAnnotation (a function in the ComplexHeatmap package). When I run it, I get the error &amp;quot;Error in col2rgb(colors, alpha = TRUE) : invalid RGB specification&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;col2rgb() is not among the libraries I've loaded and it doesn't seem to be a method contained in HeatmapAnnotation.&lt;/p&gt;&#xA;&lt;p&gt;Here's the code block for the section:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df &amp;lt;- data.frame(labid= combmetadata&lt;span class=&quot;math-container&quot;&gt;$Sample_Name,&#xA;             Diagnosis= combmetadata$&lt;/span&gt;Diagnosis,&#xA;             Tissue = combmetadata&lt;span class=&quot;math-container&quot;&gt;$Tissue,&#xA;             Tumor.Normal = combmetadata$&lt;/span&gt;T.N,&#xA;             study = combmetadata$batch)&#xA;&#xA; labid &amp;lt;- colnames(bValsselect)&#xA; df &amp;lt;- left_join(as.data.frame(labid),df,by=&amp;quot;labid&amp;quot;)&#xA; df &amp;lt;- df[,c(2,4,5)]&#xA; df&lt;span class=&quot;math-container&quot;&gt;$study[df$&lt;/span&gt;study == &amp;quot;1&amp;quot; | df$study == &amp;quot;2&amp;quot;] &amp;lt;- &amp;quot;cgid&amp;quot;&#xA;&#xA; df&lt;span class=&quot;math-container&quot;&gt;$Diagnosis &amp;lt;- factor(df$&lt;/span&gt;Diagnosis, levels = c(&amp;quot;Uterine endometrial&amp;quot;,&amp;quot;Ovarian endometrioid&amp;quot;,&#xA;                                            &amp;quot;Ovarian mucinous&amp;quot;,&#xA;                                            &amp;quot;Colorectal mucinous&amp;quot;,&#xA;                                            &amp;quot;Pancreas mucinous&amp;quot;,&#xA;                                            &amp;quot;Stomach mucinous&amp;quot;))&#xA;&#xA; ha_column = HeatmapAnnotation(df = df, &#xA;                          col = list(Diagnosis = c(&amp;quot;Uterine endometrial&amp;quot; = &amp;quot;#DDCC77F&amp;quot;,&#xA;                                          &amp;quot;Ovarian endometrioid&amp;quot; = &amp;quot;#0F7554&amp;quot;,&#xA;                                          &amp;quot;Ovarian mucinous&amp;quot; = &amp;quot;#44AA99&amp;quot;,&#xA;                                          &amp;quot;Colorectal mucinous&amp;quot; = &amp;quot;#882255&amp;quot;,&#xA;                                          &amp;quot;Pancreas mucinous&amp;quot; = &amp;quot;#AA4499&amp;quot;,&#xA;                                          &amp;quot;Stomach mucinous&amp;quot; = &amp;quot;#D695D0&amp;quot;),&#xA;                            Tumor.Normal = c(&amp;quot;N&amp;quot; = &amp;quot;grey&amp;quot;,&#xA;                                             &amp;quot;T&amp;quot; = &amp;quot;black&amp;quot;),&#xA;                            study = c(&amp;quot;cgid&amp;quot; = &amp;quot;#332288&amp;quot;,&#xA;                                      &amp;quot;tcga&amp;quot; = &amp;quot;#BD9AF7&amp;quot;)))&#xA;&#xA;hegt &amp;lt;- Heatmap(bValsselect,&#xA;            column_split=4,&#xA;            name = &amp;quot;Beta Value&amp;quot;,&#xA;            col = colorRamp2(c(0,0.25,0.5),c(&amp;quot;#00FFCC&amp;quot;,&amp;quot;#FFFFFF&amp;quot;,&amp;quot;#0099FF&amp;quot;)),&#xA;            show_row_names = FALSE, &#xA;            top_annotation = ha_column,&#xA;            show_row_dend = F)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The code runs fine up until it reaches the end of the HeatmapAnnotation() call&lt;/p&gt;&#xA;&lt;p&gt;I've tried to load col2rgb() and override the alpha setting. Can anyone give me a clue about what's happening and any suggestions on how to correct the problem?  I'm running R 4.2. Thanks&lt;/p&gt;&#xA;" OwnerUserId="2621" LastEditorUserId="73" LastEditDate="2022-05-04T02:00:46.710" LastActivityDate="2022-05-04T02:01:11.253" Title="&quot;Error in col2rgb(colors, alpha = TRUE) : invalid RGB specification&quot; when running HeatmapAnnotation() (from ComplexHeatmap)" Tags="|r|bioconductor|heatmap|complexheatmap|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19023" PostTypeId="2" ParentId="19022" CreationDate="2022-05-02T19:04:29.350" Score="1" Body="&lt;p&gt;Looking at the &lt;a href=&quot;https://www.rdocumentation.org/packages/ComplexHeatmap/versions/1.10.2/topics/SingleAnnotation&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;, it seems that &lt;code&gt;HeatmapAnnotation&lt;/code&gt; expects a vector, whereas you are giving it a list. It is not clear on what acceptable color types are. It might even work with a list, but I'd suggest playing around with what you are supplying as your &lt;code&gt;col&lt;/code&gt; parameter.&lt;/p&gt;&#xA;&lt;p&gt;Note that &lt;code&gt;HeatmapAnnotation&lt;/code&gt; has to do color mapping of some kind, the &lt;a href=&quot;https://github.com/jokergoo/ComplexHeatmap/blob/master/R/HeatmapAnnotation-class.R&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt; is a little dense, but it does look like it calls out to &lt;code&gt;SingleAnnotation&lt;/code&gt;, &lt;a href=&quot;https://github.com/jokergoo/ComplexHeatmap/blob/master/R/SingleAnnotation-class.R#L495-L507&quot; rel=&quot;nofollow noreferrer&quot;&gt;which in turn runs &lt;code&gt;ColorRamp2&lt;/code&gt;&lt;/a&gt;, which does use &lt;code&gt;col2rgb&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;It also looks like &lt;code&gt;col2rgb&lt;/code&gt; is called in &lt;code&gt;ColorRamp2&lt;/code&gt;, which you use in the &lt;code&gt;Heatmap&lt;/code&gt; call. How sure are you that your script isn't making it that far? It would help to see error output, possibly.&lt;/p&gt;&#xA;&lt;p&gt;That said, when I take your isolated &lt;code&gt;ColorRamp2&lt;/code&gt; and run it, it goes ok. Possibly pull out the &lt;code&gt;ColorRamp2&lt;/code&gt; call and run it yourself in your environment to make sure nothing weird is happening.&lt;/p&gt;&#xA;&lt;p&gt;Try moving the assignment of the colors outside the &lt;code&gt;Heatmap&lt;/code&gt; call, occasionally supplying function calls as parameters like that causes issues.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="73" LastEditDate="2022-05-04T02:01:11.253" LastActivityDate="2022-05-04T02:01:11.253" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19024" PostTypeId="2" ParentId="19022" CreationDate="2022-05-02T19:33:27.167" Score="0" Body="&lt;p&gt;The alphanumeric value for the first color assignment contains 7 alphanumeric characters instead of 6. Correcting this typo fixed the problem.&lt;/p&gt;&#xA;" OwnerUserId="2621" LastActivityDate="2022-05-02T19:33:27.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19025" PostTypeId="1" CreationDate="2022-05-02T21:54:58.123" Score="4" ViewCount="1173" Body="&lt;p&gt;I have several hundred scRNA-seq count matrices, each from a different sample. For my other dataset containg a few dozen samples, I simply merged everything together into one Seurat object, but that won’t work here as far as I can tell. When I try to merge them in the typical way, I get the error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in .cbind2Csp(x, y) : &#xA;  Cholmod error 'problem too large' at file ../Core/cholmod_sparse.c, line 89&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This basically means I ran out of RAM.  I even ran it on the cluster with 1 TB of RAM, and I got the same error.  This is unsurprising, as memory requirements increase exponentially in relation to matrix size.  It seems that this happens when the &lt;em&gt;merge()&lt;/em&gt; function attempts to convert all the sparse matrices to dense matrices.  Here is my code:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;seurat_object_list &amp;lt;- &amp;quot;path/to/matrices&amp;quot; |&amp;gt; list.dirs(recursive = FALSE) |&amp;gt; lapply(\(matrix_folder) {&#xA;    matrix_folder |&amp;gt; Seurat::Read10X() |&amp;gt; Seurat::CreateSeuratObject()&#xA;})&#xA;seurat &amp;lt;- merge(seurat_object_list[[1]], seurat_object_list[2 : length(seurat_object_list)])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The Seurat object list is created with no issue, but it always fails on the merge.  I tested this code with a smaller subset of the matrices, and it works just fine.  It seems that I just have too many cells to process in this manner.&lt;/p&gt;&#xA;&lt;p&gt;At this point, It looks like I have a few options.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I could hack together some way to force the matrices into one Seurat object.  But I don't know if that would even be useful, since I would likely keep bumping my head on the same memory contraints whenever I try to do anything with the combined object, such as running &lt;em&gt;SCTransform()&lt;/em&gt; or &lt;em&gt;FindMarkers()&lt;/em&gt;.&lt;/li&gt;&#xA;&lt;li&gt;I could divide the samples into groups that are small enough to merge, and process them in the normal way.  But this imposes limitations on the types of comparisons/analyses I could perform.&lt;/li&gt;&#xA;&lt;li&gt;I could keep the matrices separate and analyze them iteratively.  This would solve the memory problems, but I haven't found any info online about anyone analyzing count matrices this way.  I imagine that I would lose a lot of out-of-the-box functionality that comes with the standard single-cell R libraries.  For example, I'm not sure how I would normalize all the gene counts against each other as separate objects.  This option seems like it might require a lot of re-inventing the wheel and a lot of code.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I like option 3 the best, but I'm not sure where to begin.  I am still pretty new to bioninformatics, so I would love to hear some input from those who are more experienced.  How do you deal with memory constraints when dealing with a large number of cells/samples?  Are there any standard ways to deal with this problem?  Is there some tutorial/doc online that I failed to find?  Any advice you could offer is greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="15309" LastActivityDate="2022-08-15T16:45:41.150" Title="How to manage memory contraints when analyzing a large number of gene count matrices? I keep running out of RAM with my current pipeline" Tags="|r|scrnaseq|seurat|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19027" PostTypeId="2" ParentId="19016" CreationDate="2022-05-03T09:38:33.527" Score="1" Body="&lt;p&gt;Cromwell already defines the &lt;code&gt;cwd&lt;/code&gt;, &lt;code&gt;err&lt;/code&gt; and &lt;code&gt;out&lt;/code&gt; runtime attributes by itself (it also defines a &lt;code&gt;job_name&lt;/code&gt; and &lt;code&gt;script&lt;/code&gt;). So when you define these variables in your config it causes a name conflict, since they already exist.&lt;/p&gt;&#xA;&lt;p&gt;Cromwell will make a directory inside of the &amp;quot;cromwell-executions&amp;quot; directory (created when you run cromwell) for each job it runs. It will write the script, various stdout and stderr files,  and the rc file to this directory. The intend is also for the job to run in that directory. Hence cromwell provides these runtime attributes to point to the appropriate locations within this directory so they can be used in your custom submission command.&lt;/p&gt;&#xA;&lt;p&gt;You can try just removing these variables in the runtime-attributes section, while keeping them in the submit section.&lt;/p&gt;&#xA;" OwnerUserId="6955" LastActivityDate="2022-05-03T09:38:33.527" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19028" PostTypeId="1" CreationDate="2022-05-03T17:02:16.060" Score="1" ViewCount="38" Body="&lt;p&gt;recently I'm researching on how to detect CNV from Illumina microarray data. I found that Illumina has Array Analysis CLI software for this task, and I need to train the CN model myself, since the official &lt;code&gt;.dat&lt;/code&gt; file is no where to be found.&lt;/p&gt;&#xA;&lt;p&gt;However, this requires another file: the manifest file (&lt;code&gt;.bpm&lt;/code&gt;) must be accompanied by a &amp;quot;mask file&amp;quot; (&lt;code&gt;.msk&lt;/code&gt;), which I cannot find online.&lt;/p&gt;&#xA;&lt;p&gt;Could you please help me on this issue please? And could you suggest some CNV detector for Illumina microarray? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="4134" LastActivityDate="2022-05-03T17:02:16.060" Title="Creating msk file for Illumina Array Analysis CLI Copy number train" Tags="|microarray|illumina|cnv|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19029" PostTypeId="1" CreationDate="2022-05-04T09:55:51.220" Score="1" ViewCount="30" Body="&lt;p&gt;I'm sitting on a mountain of microarray (.idat files) data, and have been tasked to analyze it. I chose the MoChA pipeline, developed by the Broad Institute. It runs through Cromwell, of which I know quite little as it is written in Java and I think it's mostly meant to be applied in HPC environments.&lt;/p&gt;&#xA;&lt;p&gt;Anyway, my research group does not currently have access to any fancy clusters so I'm running it locally. I could get most of the data through after creating a swap file of size 256GB, but for the remaining sets I would need terabytes of RAM... all because the Cromwell engine doesn't limit its resource usage.&lt;/p&gt;&#xA;&lt;p&gt;Now, there's an option in the Cromwell config file &amp;quot;concurrent-jobs-limit&amp;quot;, that would solve this resource problem, but when I attempt to use it, I get errors.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Cromwell version used: 78&lt;/li&gt;&#xA;&lt;li&gt;Exact command used: &lt;a href=&quot;https://pastebin.com/70ebMVE5&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/70ebMVE5&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Entire program log: &lt;a href=&quot;https://pastebin.com/hQ8U3f86&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/hQ8U3f86&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Configuration file contents (ctrl+F &amp;quot;concurrent-job&amp;quot;): &lt;a href=&quot;https://pastebin.com/veDikew7&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/veDikew7&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I tried adding the missing key (line 379) but it merely changes the error to &lt;a href=&quot;https://pastebin.com/dxDHrKdk&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/dxDHrKdk&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I suspect the override of &amp;quot;LocalExample&amp;quot; (line 341) somehow removes invisible default settings... and right about there I'm out of my depth.&lt;/p&gt;&#xA;&lt;p&gt;I have asked in the Cromwell slack channel already, no responses in two days. I asked in the Broad Institute forums and they said they don't support Cromwell. Sincerely hoping someone here knows enough about Cromwell to help. I'd really really appreciate it. :]&lt;/p&gt;&#xA;&lt;p&gt;Joel&lt;/p&gt;&#xA;" OwnerUserId="15433" LastActivityDate="2022-05-04T09:55:51.220" Title="Cromwell - controlling core/mem usage, and CallCaching, while running locally?" Tags="|microarray|illumina|wdl|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19031" PostTypeId="1" AcceptedAnswerId="19779" CreationDate="2022-05-04T17:33:55.633" Score="4" ViewCount="137" Body="&lt;p&gt;As we know, the amount of RNA reads per cell obtained from 10X scRNA experiment vary between cells. I wonder if this is effect of technical issues or does the number of RNA reads per cell obtained from the 10X scRNA experiment depend on the amount of mRNA in that cell? For the latter to be the case, we would have to read almost all the mRNA fragments in a cell, which I don't think is likely.&#xA;Thank you for your replies in advance!&lt;/p&gt;&#xA;" OwnerUserId="14460" LastEditorUserId="104" LastEditDate="2023-01-31T18:48:08.530" LastActivityDate="2023-01-31T18:48:08.530" Title="Does the number of RNA reads per cell obtained from the 10X scRNA experiment depend on amount of mRNA in given cell?" Tags="|ngs|scrnaseq|seurat|single-cell|10x|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19032" PostTypeId="1" AcceptedAnswerId="19036" CreationDate="2022-05-04T20:15:14.497" Score="1" ViewCount="66" Body="&lt;p&gt;I have got a fold-change dataset and I simply would like to plot something like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DbhJG.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DbhJG.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The problem is that I have only numbers in my columns and when I use a boxplot() function I can plot only boxes but not dots, jitter, etc.&lt;/p&gt;&#xA;&lt;p&gt;I use this code to build a df and then proceed to a boxplot&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A &amp;lt;- c(-1, 25, 3, 4, 50)&#xA;B &amp;lt;- c(23, -41, 32, 68, 26)&#xA;C &amp;lt;- c(63, 71, 22, -8, 2)&#xA;D &amp;lt;- c(16, -49, 2, 56, 6)&#xA;df &amp;lt;- data.frame(A, B, C, D)&#xA;print (df)&#xA;boxplot(df) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I plot something that looks like the above picture? Thanks&lt;/p&gt;&#xA;" OwnerUserId="6470" LastActivityDate="2023-01-10T14:08:40.707" Title="How to build advanced boxplots from datasets which columns contain only numbers?" Tags="|r|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19034" PostTypeId="2" ParentId="18743" CreationDate="2022-05-05T09:45:47.247" Score="1" Body="&lt;p&gt;Optional outputs should work in WDL 1.0. Here is an example that runs without issue through cromwell (version 74).&lt;/p&gt;&#xA;&lt;p&gt;Note that for the second call in the scatter the input file needs to be converted to from an optional to an non optional value. Since this only gets run if this value is defined, this conversion can safely be done using the &lt;code&gt;select_first&lt;/code&gt; function (which returns the first defined value in a given list of optionals).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;version 1.0&#xA;&#xA;workflow test {&#xA;    input {&#xA;        File filelist = &amp;quot;files.txt&amp;quot;&#xA;    }&#xA;    Array[String] files = read_lines(filelist)&#xA;&#xA;    scatter (file in  files) {&#xA;         call blah as b1 {input: in=file}&#xA;         if (defined(b1.out)) {&#xA;             call blah as b2 {input: in=select_first([b1.out])}&#xA;         }&#xA;    }&#xA;}&#xA;&#xA;&#xA;task blah {&#xA;    input {&#xA;        File in&#xA;    }&#xA;    String outPath = basename(in,'.txt') + '_blah.txt'&#xA;&#xA;    command {&#xA;        if $(grep -q &amp;quot;yay&amp;quot; ~{in})&#xA;          then&#xA;             echo yay &amp;gt; ~{outPath}&#xA;        fi&#xA;    }&#xA;&#xA;    output {&#xA;        File? out = outPath&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;files.txt looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;test1.txt&#xA;test2.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With test1.txt containing &lt;code&gt;yay&lt;/code&gt; and test2.txt containing &lt;code&gt;nay&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6955" LastActivityDate="2022-05-05T09:45:47.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19035" PostTypeId="1" CreationDate="2022-05-05T13:08:43.200" Score="1" ViewCount="41" Body="&lt;p&gt;I need to color different categories of the proteins in the interaction network automatically. Is there any way to color each category with a different color in cytoscape? or R?&lt;/p&gt;&#xA;&lt;p&gt;My input is 2 column interaction protein ID. I have different categories of proteins in CSV format separately only with protein ID.&lt;/p&gt;&#xA;" OwnerUserId="14023" LastEditorUserId="14023" LastEditDate="2022-05-06T06:26:50.653" LastActivityDate="2022-05-06T06:26:50.653" Title="Cytoscape for coloring the nodes in the network" Tags="|r|proteins|protein-protein-interaction|cytoscape|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19036" PostTypeId="2" ParentId="19032" CreationDate="2022-05-05T13:21:24.303" Score="2" Body="&lt;p&gt;&lt;strike&gt;To coerce your data into a form interpretable by &lt;code&gt;ggplot2&lt;/code&gt;, you can use the &lt;code&gt;gather&lt;/code&gt; function from &lt;code&gt;tidyr&lt;/code&gt;.&lt;/strike&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;gather&lt;/code&gt; has been superseded by &lt;code&gt;pivot_longer&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse)&#xA;df &amp;lt;- tibble(A = c(-1, 25, 3, 4, 50),&#xA;             B = c(23, -41, 32, 68, 26),&#xA;             C = c(63, 71, 22, -8, 2),&#xA;             D = c(16, -49, 2, 56, 6))&#xA;&#xA;tidyr::pivot_longer(data = df, cols = c(A,B,C,D)) |&amp;gt;&#xA;  ggplot(mapping = aes(x = name, y = value)) +&#xA;  geom_boxplot() +&#xA;  geom_dotplot(binaxis = &amp;quot;y&amp;quot;, stackdir = &amp;quot;center&amp;quot;) +&#xA;  theme_classic()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/knu0J.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/knu0J.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3967" LastEditorUserId="3967" LastEditDate="2023-01-10T14:08:40.707" LastActivityDate="2023-01-10T14:08:40.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19037" PostTypeId="2" ParentId="17809" CreationDate="2022-05-05T23:11:23.993" Score="1" Body="&lt;pre&gt;&lt;code&gt;learn = cnn_learner(data, &#xA;                    models.resnet50, &#xA;                    metrics=error_rate)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;resnet50 is one of many pre-trained 2D CNN computer vision models which use transfer learning. If you've classifying a common image such as a given mammal you're sorted because they'll be loads of pictures of that mammal all over resnet50. However, 'a cell' I dunno.&lt;/p&gt;&#xA;&lt;p&gt;When the computer vision challenges were in full swing between the tech giants each put their trained models onto publication release and are easily accessible e.g. via keras, fastai etc... I'd have to look them all up and each has different architecture, they are serious architecture like 50 layer NN. The competition was eventually stopped because computer vision had a lower error rate than human vision, at which point its pointless judging between DL algorithms because we can't say what it is how can we judge an algorithm?&lt;/p&gt;&#xA;&lt;p&gt;Transfer learning places CNN at significant advantage over RNN.&lt;/p&gt;&#xA;&lt;p&gt;The other possibility is google images&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pip install google_images_download&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I don't think its on conda. Then see if you get lucky with the following&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from google_images_download import google_images_download&#xA;response = google_images_download.googleimagesdownload()&#xA;arguments = {&amp;quot;keywords&amp;quot;:&amp;quot;cells,organelles, membranes&amp;quot;,&amp;quot;limit&amp;quot;:60,&amp;quot;print_urls&amp;quot;:False}&#xA;paths = response.download(arguments)&#xA;print(paths)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Please note I don't do 2D CNN,i.e. images.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-05-06T02:25:35.670" LastActivityDate="2022-05-06T02:25:35.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19038" PostTypeId="2" ParentId="18869" CreationDate="2022-05-06T01:38:06.420" Score="0" Body="&lt;p&gt;I notice that the paper points to the Broad website for data accession; have you looked there?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://singlecell.broadinstitute.org/single_cell/study/SCP1/-single-nucleus-rna-seq-of-cell-diversity-in-the-adult-mouse-hippocampus-snuc-seq&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://singlecell.broadinstitute.org/single_cell/study/SCP1/-single-nucleus-rna-seq-of-cell-diversity-in-the-adult-mouse-hippocampus-snuc-seq&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For specifics on the normalisation method, you could look at their associated github repository:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/yinqingl/nucseq_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/yinqingl/nucseq_analysis&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Given that the paper was published in 2016, they are TPM counts, and you have access to the raw reads, my recommendation would be to regenerate gene count tables from those raw reads. I expect it will be more time consuming, less accurate, and require more effort to track down the gene count table than to create one from scratch. You can use Salmon Alevin to do this fairly quickly:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://salmon.readthedocs.io/en/latest/alevin.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://salmon.readthedocs.io/en/latest/alevin.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;[bearing in mind that there may be some footwork required to work out how to get Salmon to interpret the barcoding method that was used]&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-05-06T01:38:06.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19039" PostTypeId="1" AcceptedAnswerId="19040" CreationDate="2022-05-06T09:07:07.077" Score="3" ViewCount="79" Body="&lt;p&gt;According to &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genbank/samplerecord/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;, accession codes should consist from a combination of uppercase letters following a combination of digits. If this is a &lt;em&gt;RefSeq&lt;/em&gt;, it can have a prefix as a combination of uppercase letters with underscore.&lt;/p&gt;&#xA;&lt;p&gt;But this accession code &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/6MWN_B&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;6MWN_B&lt;/strong&gt;&lt;/a&gt; does not consist with all aforementioned. What is it? I could not find an answer via Google.&lt;/p&gt;&#xA;" OwnerUserId="15161" LastActivityDate="2022-05-06T13:23:39.557" Title="What does this accession NCBI code mean: 6MWN_B?" Tags="|phylogenetics|genbank|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19040" PostTypeId="2" ParentId="19039" CreationDate="2022-05-06T13:23:39.557" Score="3" Body="&lt;p&gt;In general, an &lt;em&gt;accession&lt;/em&gt; is a value that points to (provides &lt;em&gt;access&lt;/em&gt; to) a single database entry. Different databases (GenBank, RefSeq, Ensembl, SwissProt, and so on) have different conventions for accessions.&lt;/p&gt;&#xA;&lt;p&gt;The accession in question points to an RNA sequence associated with a &lt;a href=&quot;https://www.rcsb.org/sequence/6MWN&quot; rel=&quot;nofollow noreferrer&quot;&gt;Protein Data Bank (PDB) record&lt;/a&gt;: &amp;quot;Crystal structure of hepatitis A virus IRES domain V in complex with Fab HAVx&amp;quot;. In the PDB record, you can click the &amp;quot;sequence&amp;quot; tab and select chain &amp;quot;B&amp;quot; from the drop-down, and you will see an RNA sequence (ACGU) matching the DNA sequence (ACGT) in the NCBI record to which you linked.&lt;/p&gt;&#xA;&lt;p&gt;I'm not sure what the rules or conventions for PDB accessions are, but they are clearly different from RefSeq accessions.&lt;/p&gt;&#xA;" OwnerUserId="96" LastActivityDate="2022-05-06T13:23:39.557" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19041" PostTypeId="2" ParentId="19019" CreationDate="2022-05-06T15:47:58.470" Score="3" Body="&lt;p&gt;ATAC-seq data is fundamentally different from RNA-seq data because they measure different things.&#xA;RNA-seq data is typically mapped from raw sequencing reads to annotated gene transcripts.&#xA;This has a nice 1:1 correspondence because RNA is transcribed from genes.&lt;/p&gt;&#xA;&lt;p&gt;ATAC-seq data, on the other hand, is used to measure accessible regions of the genome.&#xA;But accessible regions don't come from genes the same way that RNA comes from genes.&#xA;Accessible regions are scattered throughout the genome and most lie in intergenic regions where there is no annotation for a gene.&#xA;Because accessible regions don't have the same biological connection to genes that RNA transcripts do, you need to be more precise in how you want to pair up accessible regions and genes.&lt;/p&gt;&#xA;&lt;p&gt;Are you looking for &lt;em&gt;cis&lt;/em&gt;-regulatory elements (CREs) that regulate nearby genes?&#xA;Those will often be found within accessible regions, so you'd need additional data that connects an individual regulatory element to that gene.&lt;/p&gt;&#xA;&lt;p&gt;Are you looking to create a catalogue of accessible regions within a certain cell type? Then you might want to look at &lt;a href=&quot;https://www.encodeproject.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENCODE's&lt;/a&gt; candidate CREs database to see how your ATAC-seq peaks compares.&#xA;From there, you'll be able to find additional information about those peaks, such as whether other functional genomics data identifies these regions as active regulatory elements, which gene(s) these elements regulate, and more.&lt;/p&gt;&#xA;&lt;p&gt;There are different options depending on what exactly you're trying to do, but there is no one way to transfer ATAC-seq peaks to gene IDs because there's no biological question there.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2022-05-06T15:47:58.470" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19042" PostTypeId="2" ParentId="19031" CreationDate="2022-05-06T18:01:17.337" Score="1" Body="&lt;p&gt;The number of reads per cell for any given experiment is largely technical. It depends on sequencing depth. 10X recommend 20K reads per cell for the v3 chemistry (and 50K reads per cell for v2)&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2022-05-06T18:01:17.337" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19043" PostTypeId="2" ParentId="19018" CreationDate="2022-05-06T22:56:22.627" Score="1" Body="&lt;p&gt;I would not recommend the approach that you describe. I am not sure that I completely understand it. For example, metagenomic binning by e.g. MetaBAT is not a scaffolding operation, which returns a single sequence. Rather, they are methods to separate a pool of contigs into groups with similar sequence characteristics indicating they come from the same genome.&lt;/p&gt;&#xA;&lt;p&gt;It sounds like you already have a group of contigs from each genome, you just want them to all be part of a single sequence in a FASTA file, as would be suggested by your interest in the reference-guided scaffolding approach scaffold_builder.&lt;/p&gt;&#xA;&lt;p&gt;I would propose that you instead use a tool like &lt;a href=&quot;https://ecogenomics.github.io/GTDBTk/commands/classify.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GTDBtk&lt;/a&gt; to place each of your genomes (each represented by a multifasta of contigs, which is the expected input) into an existing taxonomic tree, and then use their placements in that tree as a model of the taxonomy of your genomes, rather than trying to infer it de novo. I assure you that it is a high quality phylogeny that no one will criticize, and it avoids the possibility of introducing errors into your phylogeny through problematic scaffolding operations.&lt;/p&gt;&#xA;&lt;p&gt;I would suggest reading &lt;a href=&quot;https://ecogenomics.github.io/GTDBTk/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GTDBtk documentation&lt;/a&gt; for more information.&lt;/p&gt;&#xA;&lt;h2&gt;A narrower answer to the original question&lt;/h2&gt;&#xA;&lt;p&gt;I realized that there is an answer to the question of binning contigs without any additional information. I do not recommend it as I think it would give you positively wrong answers because you are starting with your genomes already binned, and it would not provide you with scaffolds, which I think is what you actually want. But there is a way to perform this task.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2745766/&quot; rel=&quot;nofollow noreferrer&quot;&gt;You can compute tetranucleotide frequencies from contigs and use them to cluster arbitrary sets of contigs into genomes or other groupings based on similarities.&lt;/a&gt; This has a significant rate of error, but it does more or less work. It's an older approach.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2022-05-07T04:09:47.257" LastActivityDate="2022-05-07T04:09:47.257" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19046" PostTypeId="1" AcceptedAnswerId="19049" CreationDate="2022-05-09T18:33:35.873" Score="0" ViewCount="640" Body="&lt;p&gt;I am looking for a simple table with all ICD9 codes (For example: Diagnosis Code 414.0 -&amp;gt; Coronary atherosclerosis). I can only find weird PDF or other kinds of text documents. Or web forms, where I have to post a certain code to get the description.&lt;/p&gt;&#xA;&lt;h2&gt;Here is what I have found so far.&lt;/h2&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.icd9data.com/2015/Volume1/default.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.icd9data.com/2015/Volume1/default.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.aapc.com/codes/icd9-codes-vol3-range&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.aapc.com/codes/icd9-codes-vol3-range&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.cms.gov/Medicare/Coding/ICD9ProviderDiagnosticCodes/codes&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cms.gov/Medicare/Coding/ICD9ProviderDiagnosticCodes/codes&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www2.gov.bc.ca/gov/content/health/practitioner-professional-resources/msp/physicians/diagnostic-code-descriptions-icd-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www2.gov.bc.ca/gov/content/health/practitioner-professional-resources/msp/physicians/diagnostic-code-descriptions-icd-9&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Publications/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.cdc.gov/pub/Health_Statistics/NCHS/Publications/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2022-05-10T20:23:58.587" LastActivityDate="2023-05-28T06:35:05.787" Title="Where to download a table with ICD-9-CM codes?" Tags="|icd-codes|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19048" PostTypeId="2" ParentId="11609" CreationDate="2022-05-09T20:03:15.310" Score="1" Body="&lt;p&gt;I would say, 120 markers is enough for phylogenetic analysis.&#xA;see &lt;a href=&quot;https://gtdb.ecogenomic.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gtdb.ecogenomic.org/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6690" LastActivityDate="2022-05-09T20:03:15.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19049" PostTypeId="2" ParentId="19046" CreationDate="2022-05-10T02:44:28.103" Score="1" Body="&lt;p&gt;The zip files found within &lt;a href=&quot;https://www.cms.gov/Medicare/Coding/ICD9ProviderDiagnosticCodes/codes&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cms.gov/Medicare/Coding/ICD9ProviderDiagnosticCodes/codes&lt;/a&gt; contain text files and Excel spreadsheets that you can open with Excel, R or Python depending on how you want to use this data.&lt;/p&gt;&#xA;" OwnerUserId="4645" LastActivityDate="2022-05-10T02:44:28.103" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19050" PostTypeId="1" CreationDate="2022-05-10T18:20:31.430" Score="0" ViewCount="176" Body="&lt;p&gt;I’m wondering if folks have recommendations for tools/scripts to extract SNP sites from a multiple sequence alignment of consensus bacterial genomes? Specifically, I am interested in a multiple sequence alignment containing variant (SNP) positions only, to be used as input for phylogeny-building programs including RAxML and BEAST.&lt;/p&gt;&#xA;&lt;p&gt;I was previously using &lt;a href=&quot;https://github.com/sanger-pathogens/snp-sites&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/sanger-pathogens/snp-sites&lt;/a&gt;, but this is not being maintained and I’m running into errors that it seems many others are having (with regards to sequences being different lengths, although I have confirmed they are the same length).&lt;/p&gt;&#xA;&lt;p&gt;Any recommendations would be great, thank you!&lt;/p&gt;&#xA;" OwnerUserId="15401" LastEditorUserId="15401" LastEditDate="2022-05-10T19:44:17.327" LastActivityDate="2022-05-10T19:44:17.327" Title="Extract SNPs from multiple sequence alignment" Tags="|sequence-alignment|phylogenetics|variants|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19052" PostTypeId="1" AcceptedAnswerId="19567" CreationDate="2022-05-10T19:17:03.660" Score="2" ViewCount="580" Body="&lt;p&gt;I am interested in the differences between AnnotationDbi/org.Mm.eg.db and biomaRt/Mus.musculus. They yield the same results as seen in the code below.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(edgeR)&#xA;library(biomaRt)&#xA;library(Mus.musculus)&#xA;library(AnnotationDbi)&#xA;library(org.Mm.eg.db)&#xA;&#xA;url &amp;lt;- &amp;quot;https://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE63310&amp;amp;format=file&amp;quot;&#xA;utils::download.file(url, destfile=&amp;quot;GSE63310_RAW.tar&amp;quot;, mode=&amp;quot;wb&amp;quot;) &#xA;utils::untar(&amp;quot;GSE63310_RAW.tar&amp;quot;, exdir = &amp;quot;.&amp;quot;)&#xA;files &amp;lt;- c(&amp;quot;GSM1545535_10_6_5_11.txt&amp;quot;, &amp;quot;GSM1545536_9_6_5_11.txt&amp;quot;, &amp;quot;GSM1545538_purep53.txt&amp;quot;,&#xA;           &amp;quot;GSM1545539_JMS8-2.txt&amp;quot;, &amp;quot;GSM1545540_JMS8-3.txt&amp;quot;, &amp;quot;GSM1545541_JMS8-4.txt&amp;quot;,&#xA;           &amp;quot;GSM1545542_JMS8-5.txt&amp;quot;, &amp;quot;GSM1545544_JMS9-P7c.txt&amp;quot;, &amp;quot;GSM1545545_JMS9-P8c.txt&amp;quot;)&#xA;for(i in paste(files, &amp;quot;.gz&amp;quot;, sep=&amp;quot;&amp;quot;))&#xA;  R.utils::gunzip(i, overwrite=TRUE)&#xA;&#xA;files &amp;lt;- c(&amp;quot;GSM1545535_10_6_5_11.txt&amp;quot;, &amp;quot;GSM1545536_9_6_5_11.txt&amp;quot;, &#xA;           &amp;quot;GSM1545538_purep53.txt&amp;quot;, &amp;quot;GSM1545539_JMS8-2.txt&amp;quot;, &#xA;           &amp;quot;GSM1545540_JMS8-3.txt&amp;quot;, &amp;quot;GSM1545541_JMS8-4.txt&amp;quot;, &#xA;           &amp;quot;GSM1545542_JMS8-5.txt&amp;quot;, &amp;quot;GSM1545544_JMS9-P7c.txt&amp;quot;, &#xA;           &amp;quot;GSM1545545_JMS9-P8c.txt&amp;quot;)&#xA;&#xA;x &amp;lt;- readDGE(files, columns=c(1,3))&#xA;geneid &amp;lt;- rownames(x$counts)&#xA;&#xA;genes &amp;lt;- AnnotationDbi::select(org.Mm.eg.db, keys=geneid, columns=&amp;quot;SYMBOL&amp;quot;, keytype=&amp;quot;ENTREZID&amp;quot;)&#xA;dim(genes)&#xA;genes2 &amp;lt;- biomaRt::select(Mus.musculus, keys = geneid, columns = c(&amp;quot;SYMBOL&amp;quot;), keytype = &amp;quot;ENTREZID&amp;quot;)&#xA;dim(genes2)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Do they use different databases? Is one preferred or recommended over the other?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="4876" LastActivityDate="2022-08-18T20:26:46.260" Title="What is the difference between AnnotationDbi/org.Mm.eg.db and biomaRt/Mus.musculus for converting to gene symbols?" Tags="|r|differential-expression|edger|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19054" PostTypeId="1" CreationDate="2022-05-10T21:48:46.760" Score="1" ViewCount="165" Body="&lt;p&gt;I have 3 biological replicates for KO condition. What I have done so far&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Peak calling&lt;/li&gt;&#xA;&lt;li&gt;The removed blacklisted region&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Now Im bit confused about the peak merging stage. Here Im trying to use the bedops tool.&lt;/p&gt;&#xA;&lt;p&gt;So what is the best option in my case when I want to make a combined set of peaks that is present in all the three replicates what option do i have to use?&lt;/p&gt;&#xA;&lt;p&gt;Which option to use from &lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/set-operations.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedops&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; Everything (-u, –everything) or `Merge (-m, –merge)`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="285" LastActivityDate="2022-05-10T21:48:46.760" Title="chip seq biological replicate merge after peak-calling" Tags="|bed|peak-calling|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19055" PostTypeId="2" ParentId="18934" CreationDate="2022-05-11T14:19:58.827" Score="1" Body="&lt;p&gt;I need to do a more thorough testing to determine just how effective this answer may be, but I think I've put together a workflow that is close enough to an appropriate answer that I'd post it here.&lt;/p&gt;&#xA;&lt;p&gt;In my research I came across &lt;a href=&quot;https://almob.biomedcentral.com/articles/10.1186/s13015-018-0130-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;gappy&lt;/a&gt;, (GitHub &lt;a href=&quot;https://github.com/alexdonath/gappy2&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), a piece of software that identifies potentially relevant gaps in a phylogenetic analysis and returns a PHYLIP-formatted file that can be run through most tree-building programs. Gappy accounts for indels, but not substitutions, so the output would need to be combined with another analysis to generate the tree I want.&lt;/p&gt;&#xA;&lt;p&gt;Fortunately, IQTree allows for a partitioned analysis of mixed data (see &lt;a href=&quot;http://www.iqtree.org/doc/Advanced-Tutorial&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), so by feeding the original alignment AND the gappy output through IQTree you should be able to generate a tree that accounts for both substitutions and indels.&lt;/p&gt;&#xA;&lt;p&gt;Again, more thorough testing is required, but I tested this possibility using an MSA example taken from &lt;a href=&quot;https://www.ebi.ac.uk/seqdb/confluence/display/JDSAT/Multiple+Sequence+Alignment+Tool+Output+Examples&quot; rel=&quot;nofollow noreferrer&quot;&gt;this website&lt;/a&gt;. Saving the FASTA-formatted example as &lt;code&gt;alignment.fasta&lt;/code&gt;, and running that file through gappy2, you get an output file called &lt;code&gt;GAPPY2_alignment.fasta_2-inf.phy&lt;/code&gt;. Thus if you create a NEXUS-formatted file like the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#nexus&#xA;begin sets;&#xA;    charset part1 = alignment.fasta: *;&#xA;    charset part2 = GAPPY2_alignment.fasta_2-inf.phy: *;&#xA;end;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and run it through IQTree using the command &lt;code&gt;iqtree -p example.nex&lt;/code&gt;, you get the following tree:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;+---ENA|CAA23748|CAA23748.1&#xA;|&#xA;+-------------ENA|CAA24095|CAA24095.1&#xA;|&#xA;|              +-------------------------------------------ENA|BAA20512|BAA20512.1&#xA;+--------------|&#xA;               +-----------------------ENA|CAA28435|CAA28435.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, disclaimer: this is the exact same tree you get when you run &lt;code&gt;alignment.fasta&lt;/code&gt; through IQTree, hence why more thorough testing is required. The sample size for this example is quite small, and it's therefore not surprising that accounting for indels does not change the outcome in this scenario. However, I think this serves as a solid baseline workflow, where future updates will be more tweaks than complete overhauls, if that makes sense.&lt;/p&gt;&#xA;&lt;p&gt;Let me know if you see any glaring flaws in my logic, but I think I'm going to close this question. I may provide edits to this answer in the future if significant tweaks/overhauls wind up being done.&lt;/p&gt;&#xA;" OwnerUserId="15344" LastActivityDate="2022-05-11T14:19:58.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19056" PostTypeId="1" CreationDate="2022-05-11T16:44:45.213" Score="0" ViewCount="646" Body="&lt;p&gt;I want to perform a violin plot from a Seurat object and I rather have vertical lines. How can I stop this and rather have a standard Violin plot?&lt;/p&gt;&#xA;&lt;p&gt;code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Seurat::VlnPlot(object, features=c(&amp;quot;LTA&amp;quot;, &amp;quot;LTB&amp;quot;), group.by = &amp;quot;group&amp;quot;,&#xA;    pt.size = 0) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kWXF7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kWXF7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3968" LastEditorUserId="3968" LastEditDate="2022-05-11T17:01:16.747" LastActivityDate="2022-05-11T17:59:37.187" Title="Violin plots appear as vertical lines" Tags="|seurat|ggplot2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19057" PostTypeId="2" ParentId="19056" CreationDate="2022-05-11T17:59:37.187" Score="3" Body="&lt;p&gt;I believe the reason is simply LTA and LTB not being expressed (or their expression not being detected due to technical limitations). If you would set the &lt;code&gt;pt.size&lt;/code&gt; argument to something else than 0, you would see a big blob of dots at point 0.0 (depending on the number of cells/data points).&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2022-05-11T17:59:37.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19060" PostTypeId="1" CreationDate="2022-05-13T09:36:54.717" Score="1" ViewCount="44" Body="&lt;p&gt;I am trying to learn how to do association analysis of methylation array data with numerical trait like body weight, eGFR.&#xA;As I have seen some of the publications, methylation data association with eGFR has been showed through regression analysis but I am not sure how to run regression for so many CpG probes in R.&lt;/p&gt;&#xA;&lt;p&gt;Can you have look on this below publication and could you help with R code for regression analysis?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/s41467-017-01297-7.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41467-017-01297-7.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I do have methylation beta value from RnBeads but I am not sure how to read methylation data (rows/column-CpG/SampleID) and then how to read phenotype data file and finally how to make single sheet to run regression analysis?&lt;/p&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="3694" LastEditDate="2022-05-13T15:23:33.527" LastActivityDate="2022-05-15T05:45:30.180" Title="Association analysis for methylation beta value and phenotypes in R" Tags="|r|bioconductor|statistics|epigenetics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19061" PostTypeId="2" ParentId="18912" CreationDate="2022-05-13T20:27:23.440" Score="1" Body="&lt;p&gt;Coincidentally, a &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2022.05.06.490859v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;recent preprint out of Lior Pacther's lab&lt;/a&gt; covers some of these details around scRNA-seq depth normalization.&lt;/p&gt;&#xA;&lt;p&gt;They find that a &amp;quot;PFlog1pPF&amp;quot; normalization method is able to stabilize variance across genes while still reducing the impact of total cell depth on that variance. If I've read this preprint correctly, &amp;quot;PFlog1pPF&amp;quot; is essentially a transformation of the form &lt;span class=&quot;math-container&quot;&gt;$\frac{1}{N_c} \log \left(1 + \frac{k_g}{N_c} \right)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$N_c$&lt;/span&gt; is the number of reads for cell &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$k_g$&lt;/span&gt; is the number of reads for feature &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This preprint is mostly dedicated towards differential expression and identifying marker genes in certain cell populations, but that may still help you in your application.&lt;/p&gt;&#xA;&lt;p&gt;Using a transformation like this (or others that are discussed in this preprint and its references) gives you more granularity and sensitivity for highly expressed genes than what binarizing the data will give you.&#xA;And it has additional benefits like stabilized variance, which can cause problems in some downstream applications if you don't keep it under control.&#xA;But starting with a binarized input like your supervisor suggested might be a simple first attempt to see if your idea has some potential in the first place.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2022-05-13T20:27:23.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19062" PostTypeId="1" CreationDate="2022-05-13T21:09:02.930" Score="1" ViewCount="71" Body="&lt;p&gt;I have duplicate drug efficacy experiments performed six months apart. I would like to know what is the right way to perform statistical analysis to understand fold changes after treatment with a drug.&lt;/p&gt;&#xA;&lt;p&gt;Say I have&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ctrl1       tube1       ctrl2       tube2&#xA;7,732637722 10,71639307 5,454318175 8,052899428&#xA;6,69630905  8,762916793 7,441582994 9,855106201&#xA;10,35624396 11,73716043 8,976524001 11,0975511&#xA;4,29150392  4,848632578 5,901132154 7,918630661&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and the same experiment but 6 months after (numbers are slightly different)&lt;/p&gt;&#xA;&lt;p&gt;Could I take the mean between samples divided by two can work out? If not - why?&lt;/p&gt;&#xA;&lt;p&gt;I would like to have statistically significant values for futher analysis.&#xA;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="15486" LastEditorUserId="3848" LastEditDate="2022-05-14T16:40:52.920" LastActivityDate="2024-02-03T19:00:40.093" Title="What is the best way to perform batch ajustment between two experiments?" Tags="|statistics|batch-effects|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19063" PostTypeId="1" CreationDate="2022-05-14T10:53:37.937" Score="2" ViewCount="92" Body="&lt;p&gt;I'm an absolute beginner trying to solve this question &amp;quot;Align the trimmed and untrimmed reads using QuasR and plot alignment statistics, did the trimming improve alignments?&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;I did trim the reads using the preprocessReads() function, now I want to align the output but the qAlign() only takes a samples file and genome file. however, my trimmed reads are two separate files how can I align them.&lt;/p&gt;&#xA;&lt;p&gt;I asked google but I didn't find the answer&lt;/p&gt;&#xA;" OwnerUserId="15487" LastEditorUserId="73" LastEditDate="2022-05-15T03:11:28.417" LastActivityDate="2023-10-07T08:09:21.143" Title="how to create &quot;sample file&quot; for the qAlign() function after trimming the reads in R" Tags="|r|rna-seq|sequence-alignment|homework|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19065" PostTypeId="1" CreationDate="2022-05-14T13:48:30.603" Score="1" ViewCount="42" Body="&lt;p&gt;I am searching for an easy-to-use GUI management tool to handle my WDL workflows, I found &lt;a href=&quot;https://cloud-pipeline.com/release_notes/v.0.16/v.0.16_-_Release_notes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;EPAM Cloud Pipelines&lt;/a&gt;. Unfortunately, it uses cloud services such as AWS, GCP, etc. but I wanted to use my own backend instead, thus, I attempted to see if it would be possible to configure it to use my backend, however, there seems to be little to no instructions on how to install it. If anyone has any experience using this, can you please provide a description? Or, if you use or know of another GUI tool similar to EPAM Cloud Pipelines that can help manage and build WDL workflows, please share some information on that aswell.&lt;/p&gt;&#xA;" OwnerUserId="15246" LastEditorUserId="15246" LastEditDate="2022-05-15T09:31:41.950" LastActivityDate="2022-05-15T09:31:41.950" Title="How to use EPAM Cloud Pipelines?" Tags="|wdl|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19066" PostTypeId="2" ParentId="19062" CreationDate="2022-05-14T16:37:49.737" Score="0" Body="&lt;p&gt;The answer is simply you use both sets of values in a one-way paired ANOVA. Note that certain criteria need to be in place for ANOVA, notably the Fmax test.&lt;/p&gt;&#xA;&lt;p&gt;Looking at the data 2-way ANOVA might be needed and that is more complicated, or else 2 one-way ANOVAs: I don't understand 'ctl2' and 'tube2' data. Is this a single or multiple drugs?&lt;/p&gt;&#xA;&lt;p&gt;Wilcoxon signed-rank test is the first choice test because there are no prior assumptions required, which replaces both the T-test and the 1-way ANOVA.&lt;/p&gt;&#xA;&lt;p&gt;Whatever you do, taking the mean and performing a T-test is not a good idea. You asked why ... because the variation between the values is critical to the significance of the test and is simply demonstrated:&lt;/p&gt;&#xA;&lt;p&gt;Tube 1 value&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tube1 time X = 11&#xA;tube1 time y = 9&#xA;tube 1 mean = 10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Tube 2 value&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tube 2 time x = 19&#xA;tube 2 time y = 1&#xA;tube 2 mean = 10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thus both would appear to have no change - they are both 10 - and yet the variation between them means they observe very different behaviours. Thats why.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-05-14T16:47:35.557" LastActivityDate="2022-05-14T16:47:35.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19067" PostTypeId="2" ParentId="19063" CreationDate="2022-05-14T16:53:20.457" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/QuasR/inst/doc/QuasR.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;The QuasR vignette&lt;/a&gt; tells you how to run it. The samples file should contain a list of your samples and fastq files.&lt;/p&gt;&#xA;" OwnerUserId="15489" LastActivityDate="2022-05-14T16:53:20.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19068" PostTypeId="1" CreationDate="2022-05-14T21:04:11.560" Score="2" ViewCount="507" Body="&lt;p&gt;Given two separate wdl arrays, my command section will do some modifications to each array and place the output into two separate bash arrays. Finally, I want to output the two separate bash arrays in the output section of the wdl task to be used in other tasks within the workflow.  The below code is an example of what I would like to do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;task AddA {&#xA;    input {&#xA;        Array[String] wdllistX = [ &amp;quot;True&amp;quot;, &amp;quot;False&amp;quot;]&#xA;        Array[String] wdllistY = [ &amp;quot;Good&amp;quot;, &amp;quot;Food&amp;quot;]&#xA;    }&#xA;&#xA;    command &amp;lt;&amp;lt;&amp;lt;&#xA;        ###We add the letter 'a' at the end of each string in each wdl list and finally get the &#xA;      bash arrays:&#xA;&#xA;        bashX = (&amp;quot;Truea&amp;quot; Falsea&amp;quot;)&#xA;        bashY = (&amp;quot;Gooda&amp;quot; &amp;quot;Fooda&amp;quot;)&#xA;    &amp;gt;&amp;gt;&amp;gt;&#xA;    output {&#xA;        Array[String] finX = bashX&#xA;        Array[String] finY = bashY&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above example produces an error. It seems that I might have to convert the bash arrays back into wdl arrays in order to declare them as outputs in the output section, however, I am not sure how to do this. May someone help me with this? Moreover, a general explanation of how you can capture the output of specific things from the command section into the output section would be helpful aswell.&lt;/p&gt;&#xA;" OwnerUserId="15246" LastActivityDate="2023-08-23T21:00:17.907" Title="How can you put a bash array into a wdl variable?" Tags="|wdl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19069" PostTypeId="2" ParentId="19060" CreationDate="2022-05-15T05:45:30.180" Score="-1" Body="&lt;p&gt;&lt;em&gt;Answer from @user4939328, converted from comment. Please improve this answer if you can:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Have a look at this method:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://academic.oup.com/bioinformatics/article/34/15/2657/4939328&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://academic.oup.com/bioinformatics/article/34/15/2657/4939328&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;[Wondering why this was converted from a comment? See the StackExchange rules about comments &lt;a href=&quot;https://bioinformatics.stackexchange.com/help/privileges/comment&quot;&gt;here&lt;/a&gt;]&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-05-15T05:45:30.180" CommentCount="0" CommunityOwnedDate="2022-05-15T05:45:30.180" ContentLicense="CC BY-SA 4.0" />
  <row Id="19070" PostTypeId="1" CreationDate="2022-05-15T17:02:03.653" Score="1" ViewCount="20" Body="&lt;p&gt;I am interested in getting the drug indication data for all drugs in the database using sqlite.&lt;/p&gt;&#xA;&lt;p&gt;I used following step to get this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;import sqlite3&#xA;con = sqlite3.connect(&amp;quot;chembl_30.db&amp;quot;)&#xA;df = pd.read_sql_query(&amp;quot;SELECT * FROM drug_indication&amp;quot;, con)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With this I am getting a table with columns efo_id and efo_term for all disease.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;drugind_id  record_id   molregno    max_phase_for_ind   mesh_id mesh_heading    efo_id  efo_term&#xA;0   22606   1390869 675774  2   D045743 Scleroderma, Diffuse    EFO:0000404 diffuse scleroderma&#xA;1   22607   1390869 675774  4   D001172 Arthritis, Rheumatoid   EFO:0000685 rheumatoid arthritis&#xA;2   22608   1390869 675774  2   D008180 Lupus Erythematosus, Systemic   EFO:0002690 systemic lupus erythematosus&#xA;3   22609   1343491 675576  3   D009203 Myocardial Infarction   EFO:0000612 myocardial infarction&#xA;4   22610   1344515 438435  4   D011471 Prostatic Neoplasms EFO:0001663 prostate carcinoma&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there any way I can I filter EFO's based on cancer, so that I get drug indication data (max_phase) for all chembl ids used in cancer?&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;some important links&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ebi.ac.uk/chembl/db_schema&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/chembl/db_schema&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ebi.ac.uk/chembl/compound_report_card/CHEMBL941/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/chembl/compound_report_card/CHEMBL941/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="4783" LastActivityDate="2022-05-15T17:02:03.653" Title="Drug indication data for only cancer phenotype from Chembl" Tags="|cancer|drugs|cheminformatics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19071" PostTypeId="1" CreationDate="2022-05-16T16:03:37.753" Score="1" ViewCount="392" Body="&lt;p&gt;My friend has a VUS (Polymicrogyria with or without VEDS) mutation that was found in her whole-exome sequencing with respect to the phenotype given that time. At present, her doctors have more information on the complexity and are looking for more explainable mutation present in herself, if any. We asked the company and they're willing to give the raw data.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Which file format should we opt for?&lt;/strong&gt; They've given the option of BAM &amp;amp; VCF.&lt;/p&gt;&#xA;" OwnerUserId="15497" LastEditorUserId="3694" LastEditDate="2022-05-16T16:54:47.070" LastActivityDate="2022-10-13T18:03:20.347" Title="VCF or BAM file for raw data of gene test?" Tags="|sam|vcf|gene|mutations|genetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19072" PostTypeId="2" ParentId="19071" CreationDate="2022-05-16T17:22:36.817" Score="1" Body="&lt;p&gt;First of all - I would advise you against concluding anything out of your own analysis of the data, you need to consult someone with appropriate training!&lt;/p&gt;&#xA;&lt;p&gt;Can you get both? The &lt;code&gt;.bam&lt;/code&gt; file is a format that stores information about sequencing reads (usually mapped, but not always). &lt;a href=&quot;https://www.metagenomics.wiki/tools/samtools/converting-bam-to-fastq&quot; rel=&quot;nofollow noreferrer&quot;&gt;You can even extract&lt;/a&gt; the original sequencing information from bam files. This is practically the rawest information you can have. The &lt;code&gt;.vcf&lt;/code&gt; file is a variant file that gives you some information about inferred variants, but there are plenty of bits of information that will be missing. The basic information regarding sequencing depth and quality of variant calling is still included and therefore most likely sufficient for your physician to see any other variants potentially relevant to your friend's condition.&lt;/p&gt;&#xA;&lt;p&gt;I will repeat once again - don't conclude anything on your own. It takes years to gain expertise in analysing genemics data and there is very high risk of misinterpretations if not analysed with care.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2022-05-16T17:22:36.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19073" PostTypeId="2" ParentId="19068" CreationDate="2022-05-16T21:33:12.217" Score="4" Body="&lt;p&gt;When a WDL task is run, the execution engine takes the command block and evaluates it as a template, rendering any wdl expressions it contains into the final script that is executed, &lt;strong&gt;BEFORE&lt;/strong&gt; it is actually executed.&lt;/p&gt;&#xA;&lt;p&gt;The actual execution environment is generally isolated from the execution engine or from other tasks. Information is passed between tasks by reading from or passing around file based outputs and not from accessing the bash environment directly, which the engine does not have access to.&lt;/p&gt;&#xA;&lt;p&gt;So, in order to achieve what you are trying to do, you first need to get the values into a file format, and then you need to read those values using an &lt;a href=&quot;https://github.com/openwdl/wdl/blob/main/versions/1.1/SPEC.md#standard-library&quot; rel=&quot;nofollow noreferrer&quot;&gt;engine function provided&lt;/a&gt; by &lt;code&gt;WDL&lt;/code&gt;. Generally, all outputs need to be in a file if you want the engine to be able to interact with them. Please note that &lt;code&gt;stdout&lt;/code&gt; and &lt;code&gt;stderr&lt;/code&gt; are considered &amp;quot;files&amp;quot; and are accessible with &lt;code&gt;stdout()&lt;/code&gt; and &lt;code&gt;stderr()&lt;/code&gt; engine functions.&lt;/p&gt;&#xA;&lt;h2&gt;Option 1: Construct json files and use read_json&lt;/h2&gt;&#xA;&lt;pre&gt;&lt;code&gt;task AddA {&#xA;    input {&#xA;        Array[String] wdllistX = [ &amp;quot;True&amp;quot;, &amp;quot;False&amp;quot;]&#xA;        Array[String] wdllistY = [ &amp;quot;Good&amp;quot;, &amp;quot;Food&amp;quot;]&#xA;    }&#xA;&#xA;    command &amp;lt;&amp;lt;&amp;lt;&#xA;        ###We add the letter 'a' at the end of each string in each wdl list and finally get the &#xA;      bash arrays:&#xA;        echo '[&amp;quot;Truea&amp;quot;,&amp;quot;Falsea&amp;quot;]' &amp;gt; x.json&#xA;        echo '[&amp;quot;Gooda&amp;quot;,&amp;quot;Fooda&amp;quot;]' &amp;gt; y.json&#xA;    &amp;gt;&amp;gt;&amp;gt;&#xA;    output {&#xA;        Array[String] finX = read_json(&amp;quot;x.json&amp;quot;)&#xA;        Array[String] finY = read_json(&amp;quot;y.json&amp;quot;)&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Option 2: use read_lines&lt;/h2&gt;&#xA;&lt;pre&gt;&lt;code&gt;task AddA {&#xA;    input {&#xA;        Array[String] wdllistX = [ &amp;quot;True&amp;quot;, &amp;quot;False&amp;quot;]&#xA;        Array[String] wdllistY = [ &amp;quot;Good&amp;quot;, &amp;quot;Food&amp;quot;]&#xA;    }&#xA;&#xA;    command &amp;lt;&amp;lt;&amp;lt;&#xA;        ###We add the letter 'a' at the end of each string in each wdl list and finally get the &#xA;      bash arrays:&#xA;&#xA;        echo &amp;quot;Truea&amp;quot; &amp;gt;&amp;gt; x.txt&#xA;        echo &amp;quot;Falsea&amp;quot; &amp;gt;&amp;gt; x.txt&#xA;        echo &amp;quot;Gooda&amp;quot; &amp;gt;&amp;gt; y.txt&#xA;        echo &amp;quot;Fooda&amp;quot; &amp;gt;&amp;gt; y.txt&#xA;    &amp;gt;&amp;gt;&amp;gt;&#xA;    output {&#xA;        Array[String] finX = read_lines(&amp;quot;x.txt&amp;quot;)&#xA;        Array[String] finY = read_lines(&amp;quot;y.txt&amp;quot;)&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7253" LastEditorUserId="650" LastEditDate="2023-08-23T21:00:17.907" LastActivityDate="2023-08-23T21:00:17.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19074" PostTypeId="2" ParentId="18774" CreationDate="2022-05-17T02:01:34.363" Score="2" Body="&lt;p&gt;In order to reference a WDL input within a command block you need to use an &lt;a href=&quot;https://github.com/openwdl/wdl/blob/main/versions/1.1/SPEC.md#expression-placeholders-and-string-interpolation&quot; rel=&quot;nofollow noreferrer&quot;&gt;expression placeholder&lt;/a&gt;. Expression placeholders have the form &lt;code&gt;~{*wdl expression*}&lt;/code&gt;, where the inner part can be an valid WDL expression (ie reference to an input declaration)&lt;/p&gt;&#xA;&lt;p&gt;When an engine sees a &lt;code&gt;String&lt;/code&gt; or a &lt;code&gt;command&lt;/code&gt; block, it first looks to see if there are any of these expression placeholders within it. If there are, it will evaluate the inner expression, and replace the entire placeholder &lt;code&gt;~{...}&lt;/code&gt; with the resulting value.&lt;/p&gt;&#xA;&lt;p&gt;You can see how this can be applied below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;String first_name = &amp;quot;John&amp;quot;&#xA;String last_name = &amp;quot;Smith&amp;quot;&#xA;Int age = 45&#xA;&#xA;&#xA;String full_name = &amp;quot;~{first_name} ~{last_name} # equals &amp;quot;John Smith&amp;quot;&#xA;String name_and_age = &amp;quot;John is ~{age}&amp;quot; # equals &amp;quot;John is 45&#xA;&#xA;# Any valid WDL expression can go within an expression placeholder&#xA;String complex = &amp;quot;~{first_name + last_name + age}&amp;quot; # equals &amp;quot;JohnSmith45&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A key thing to note is that WDL command templating and expression evaluation is completely separate from the actual execution of the command. An execution engine will evaluate the command template, swap the expression placeholders with their evaluated values, and  &lt;em&gt;THEN&lt;/em&gt;, execute the script.&lt;/p&gt;&#xA;&lt;p&gt;This is relevant for your use case since you are using heredoc to directly define the rscript you are runnning. If we briefly look at a python example, you can see what I mean:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;task hello_world {&#xA;  input {&#xA;   String name = &amp;quot;john&amp;quot;&#xA;  }&#xA;&#xA;  command &amp;lt;&amp;lt;&amp;lt;&#xA;   python &amp;lt;&amp;lt;CODE&#xA;     # this is valid, because once the placeholder evaluates&#xA;     # this command becomes name = &amp;quot;john&amp;quot;, which is valid python syntax&#xA;     # We did not have to pass ~{name} as an arg to the python command&#xA;     name = &amp;quot;~{name}&amp;quot;&#xA;     print(name)&#xA;   CODE&#xA;  &amp;gt;&amp;gt;&amp;gt;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Now getting to your problem specifically. You are trying to reference the &lt;code&gt;name&lt;/code&gt; input value without using any sort of expression placeholder. To the execution engine, the &lt;code&gt;name&lt;/code&gt; value looks like any other string. To swap the &lt;code&gt;name&lt;/code&gt; in the command for the value passed into the task, you simply need to wrap the variable name in an expression placeholder: &lt;code&gt;~{name}&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;task PrintNameR {&#xA;  input {&#xA;    String name&#xA;  }&#xA;&#xA;  command&amp;lt;&amp;lt;&amp;lt;&#xA;    # I would wrap this in quotes to prevent any side effects of &#xA;    # shell expansion&#xA;    R --no-save --args '~{name}' &amp;lt;&amp;lt;Rscript&#xA;    args &amp;lt;- commandArgs(trailingOnly = TRUE)&#xA;    for (i in args){&#xA;    print(i)}&#xA;    Rscript&#xA;  &amp;gt;&amp;gt;&amp;gt;&#xA;&#xA;  runtime {&#xA;            docker: &amp;quot;firsttrydocker&amp;quot;&#xA;  }&#xA;  &#xA;  output {&#xA;     File output_R = stdout()&#xA;  }&#xA;}&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7253" LastActivityDate="2022-05-17T02:01:34.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19076" PostTypeId="1" AcceptedAnswerId="19077" CreationDate="2022-05-18T03:51:06.693" Score="0" ViewCount="103" Body="&lt;p&gt;I have a list named dict_eg1 whose values are lists.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dict_eg1 ={&#xA;    &amp;quot;A&amp;quot;:[1,2,3],&#xA;    &amp;quot;B&amp;quot;:[11,22,33],}&#xA;val1=dict_eg1[&amp;quot;A&amp;quot;]&#xA;val1.append (100)#modify the list by using append.&#xA;print (dict_eg1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;{'A': [1, 2, 3, 100], 'B': [11, 22, 33]}&lt;/p&gt;&#xA;&lt;p&gt;In this case, dict_eg1 changed.&lt;/p&gt;&#xA;&lt;p&gt;However,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dict_eg1 ={&#xA;    &amp;quot;A&amp;quot;:[1,2,3],&#xA;    &amp;quot;B&amp;quot;:[11,22,33],}&#xA;val1=dict_eg1[&amp;quot;A&amp;quot;]&#xA;val1 = [1, 2, 3, 100]#modify the list without using append.&#xA;dict_eg1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;{'A': [1, 2, 3], 'B': [11, 22, 33]}&#xA;In this case, dict_eg1 did not change.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone tell me why these two results were different??&lt;/p&gt;&#xA;" OwnerUserId="14429" LastActivityDate="2022-05-18T05:58:21.460" Title="Python; Question about the append (modifying lists): why modifying a list with append changes others?" Tags="|python|" AnswerCount="1" CommentCount="2" ClosedDate="2022-05-18T18:38:33.170" ContentLicense="CC BY-SA 4.0" />
  <row Id="19077" PostTypeId="2" ParentId="19076" CreationDate="2022-05-18T05:51:41.927" Score="1" Body="&lt;p&gt;Welcome to Bioinformatics.&lt;/p&gt;&#xA;&lt;p&gt;First of all, it should not be asked in this community since it is purely a question about python programming. I suggest that next time, please head to &lt;a href=&quot;https://stackoverflow.com/&quot;&gt;StackOverflow&lt;/a&gt; as you will likely receive more help from there.&lt;/p&gt;&#xA;&lt;p&gt;You also look new to programming. This is more of a logical question. Let's break it up to understand what is happening in your first example:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;# you define your dictionary containing two keys along with their values.&#xA;dict_eg1 = {&#xA;             &amp;quot;A&amp;quot;:[1,2,3],    # Key 'A' with a value as a list [1,2,3]&#xA;             &amp;quot;B&amp;quot;:[11,22,33]  # Key 'B' with a value as a list [11,22,33]&#xA;}&#xA;&#xA;# you now define a new variable 'val1' which is&#xA;# the value of key 'A' and that is the list [1,2,3]&#xA;val1=dict_eg1[&amp;quot;A&amp;quot;]&#xA;&#xA;# as you have mentioned, doing the below command&#xA;# will append the value to the list of key 'A'&#xA;# in other words, you can also write it like so&#xA;# dict_eg1[&amp;quot;A&amp;quot;].append(100)&#xA;# that's why when you make a change to 'val1'&#xA;# the value of the key 'A' of 'dict_eg1' will change&#xA;val1.append (100)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As for the second example (I will skip the same steps):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;# ...&#xA;&#xA;# you store the value of key 'A' in 'val1'&#xA;val1=dict_eg1[&amp;quot;A&amp;quot;]&#xA;&#xA;# however, in the below command, you actually&#xA;# !redefine! the value of 'val1' to a new list&#xA;# that is totally independent of your 'dict_eg1'&#xA;# In this case, 'dict_eg1' is not affected by&#xA;# any changes made to 'val1'&#xA;val1 = [1, 2, 3, 100]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Just to sum it up, when you use &lt;code&gt;val1=dict_eg1[&amp;quot;A&amp;quot;]&lt;/code&gt;, imagine that you link the value of your key &lt;code&gt;A&lt;/code&gt; to &lt;code&gt;val1&lt;/code&gt; and any changes made to &lt;code&gt;val1&lt;/code&gt; will affect &lt;code&gt;dict_eg1[&amp;quot;A&amp;quot;]&lt;/code&gt;. However, &lt;em&gt;&lt;strong&gt;redefining&lt;/strong&gt;&lt;/em&gt; &lt;code&gt;val1&lt;/code&gt; will disconnect that link.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="4413" LastEditDate="2022-05-18T05:58:21.460" LastActivityDate="2022-05-18T05:58:21.460" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19079" PostTypeId="1" CreationDate="2022-05-18T22:03:51.850" Score="0" ViewCount="75" Body="&lt;p&gt;I have different treatments (infected and control ones). When using DESeq2, I have different results in two ways of comparison.&#xA;first way:&lt;/p&gt;&#xA;&lt;p&gt;factor: treatment, factor level1: Moribund Infected, factor level2: 5week Infected, factor level3: Moribund Control, factor level4: 5week Control.&#xA;I have 6 result files (comparisons among factor levels)&lt;/p&gt;&#xA;&lt;p&gt;second way: (I compared just 2 groups to each other)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;factor: treatment, factor level1: Moribund Infected, factor level2: 5week Infected.&#xA;factor: treatment, factor level1: Moribund Infected, factor level2: Moribund Control.&#xA;factor: treatment, factor level1: Moribund Infected, factor level2:the  5week Control.&#xA;factor: treatment, factor level1: 5week Infected, factor level2: Moribund Control.&#xA;factor: treatment, factor level1: 5week Infected, factor level2: 5week Control.&#xA;factor: treatment, factor level 1: Moribund Control, factor level2: 5week Control.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Finally, the results were different in these two ways of comparison. I have more significantly expressed genes in most comparisons of first way and even I have few comparisons in the second way that have more significantly expressed genes.&lt;/p&gt;&#xA;&lt;p&gt;Why and which way is trustable?&lt;/p&gt;&#xA;" OwnerUserId="15509" LastEditorUserId="3694" LastEditDate="2022-05-19T08:06:39.533" LastActivityDate="2022-05-19T08:06:39.533" Title="Different results of DESeq2 in different ways of comparison" Tags="|deseq2|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19080" PostTypeId="1" AcceptedAnswerId="19082" CreationDate="2022-05-19T09:06:30.113" Score="2" ViewCount="74" Body="&lt;p&gt;I am using &lt;code&gt;ggtree&lt;/code&gt; to draw a phylogenetic tree.&lt;/p&gt;&#xA;&lt;p&gt;Some taxa IDs have a 0 at the beginning.&lt;/p&gt;&#xA;&lt;p&gt;However, when I plot the tiplabel the leading zero is automatically dropped. How do I retain the leading zero?&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;&lt;p&gt;The code and tree file and metadata file are attached. I would appreciate any help.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;setwd('G:\\Experiments\\Paper\\Torula\\06_iqtree')   &#xA;dir()&#xA;&#xA;# Load required packages&#xA;library(xlsx)&#xA;library(ggtree)&#xA;library(treeio)&#xA;library(ggplot2)&#xA;&#xA;# Read tree&#xA;tre &amp;lt;- read.iqtree(&amp;quot;concatenate_iqtree2.treefile&amp;quot;)&#xA;tre_tbl &amp;lt;- fortify(tre)&#xA;# Read associated information table&#xA;metadf &amp;lt;- read.xlsx('G:\\Experiments\\Paper\\Torula\\00_data\\Torula_taxa.xlsx', sheetIndex = 1)&#xA;&#xA;# Plot tree&#xA;ggtree(tre) %&amp;lt;+% metadf + &#xA;    # our own taxa darkblue&#xA;    # type strain bold&#xA;    geom_tiplab(aes(subset = (New == 1 &amp;amp; Type == 1), &#xA;                    label = paste0('bolditalic(', Genus, ')~bolditalic(', Epithet, ')~bold(', Collection,')~bold(', Number,')')), &#xA;                color = 'darkblue', parse = T) + &#xA;    geom_tiplab(aes(subset = (New == 0 &amp;amp; Type == 1), &#xA;                    label = paste0('bolditalic(', Genus, ')~bolditalic(', Epithet, ')~bold(', Collection,')~bold(', Number,')')), &#xA;                color = 'black', parse = T) + &#xA;    geom_tiplab(aes(subset = (New == 0 &amp;amp; Type == 0), &#xA;                    label = paste0('italic(', Genus, ')~italic(', Epithet, ')', '~', Collection,'~', Number)), &#xA;                color = 'black', parse = T) + &#xA;    &#xA;    geom_text(aes(label= UFboot), vjust = 0, hjust = 1, size = 3) +&#xA;    xlim(NA,0.5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Yafxt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Yafxt.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;Some taxa in the figure &lt;a href=&quot;https://i.stack.imgur.com/D0K42.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D0K42.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8757" LastEditorUserId="3848" LastEditDate="2022-05-19T12:12:46.493" LastActivityDate="2022-05-20T15:48:08.410" Title="ggtree：How to keep leading zero ‘0’ in the tiplabel？" Tags="|phylogenetics|phylogeny|ggplot2|ggtree|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19081" PostTypeId="1" CreationDate="2022-05-19T09:20:05.683" Score="1" ViewCount="119" Body="&lt;p&gt;Typically, a p-value of &lt;span class=&quot;math-container&quot;&gt;$p&amp;lt;=5*10^{-8}$&lt;/span&gt; is used for &lt;strong&gt;genome-wide&lt;/strong&gt; association testing, as there are roughly &lt;span class=&quot;math-container&quot;&gt;$1/p$&lt;/span&gt; independent LD blocks in the human genome, so it correct for multiple testing across the genome.&lt;/p&gt;&#xA;&lt;p&gt;HLA assocation tests, &lt;a href=&quot;https://www.nature.com/articles/s41588-021-00935-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;such as in this study&lt;/a&gt;, only look at a fraction of the genome, something like chr6:28–34 Mb, which is approximately 0.2% of the genome used to conduct a true 'genome-wide' association study. However, they still use the same genome-wide p-value threshold of &lt;span class=&quot;math-container&quot;&gt;$p&amp;lt;=5*10^{-8}$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Is this not too conservative, since there is a much reduced multiple testing burden? Should the threshold not be something like &lt;span class=&quot;math-container&quot;&gt;$p/0.002$&lt;/span&gt;?&lt;/p&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="3694" LastEditDate="2022-05-19T11:27:26.267" LastActivityDate="2022-05-22T03:01:51.780" Title="Why is the p-value significance threshold for HLA association tests $5*10^{-8}$?" Tags="|statistics|gwas|p-values|" AnswerCount="2" CommentCount="0" ClosedDate="2022-05-29T20:07:39.790" ContentLicense="CC BY-SA 4.0" />
  <row Id="19082" PostTypeId="2" ParentId="19080" CreationDate="2022-05-19T12:07:48.680" Score="2" Body="&lt;p&gt;I like both the methods and the study. &lt;code&gt;ggplot2&lt;/code&gt; is R's standout package/library and the library &lt;code&gt;ggtree&lt;/code&gt; looks great.&lt;/p&gt;&#xA;&lt;p&gt;Anyway, I would know how to solve this in Python or Perl and I'll use the same logic for R. My suspicion is R is automatically type casting your metadata and therefore has no option but to remove the zero.&lt;/p&gt;&#xA;&lt;p&gt;Simply convert your integer range into a string, viz.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; Number2 = as.character(Number)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or simply &lt;em&gt;in situ&lt;/em&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;as.character(Number)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This doesn't make any difference to your output, who cares whether its and integer or string and forces R into retaining the zero, '0'.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Note below that tried and didn't work. A string should be immutable.&#xA;The suspicion is the the '0' is being removed at an earlier point in the code, i.e. as soon as it type casts it - zero disappears.&lt;/p&gt;&#xA;&lt;p&gt;What I would do some print outs of the variable between its import and writing to disk. What I suspect is the conversion to string needs to occur earlier.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Second idea&lt;/strong&gt;&#xA;The other possibility is to wrap the metadata in double/single quotes in the Excel file (or flat file). This way R cannot force it into an integer when it is imported and must retain it as a string. I'd be amazed if R could wriggle out of that.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-05-20T15:48:08.410" LastActivityDate="2022-05-20T15:48:08.410" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19083" PostTypeId="2" ParentId="19081" CreationDate="2022-05-19T12:31:02.833" Score="1" Body="&lt;p&gt;Its a very good question and it is also a good study. What you are saying must be correct.&lt;/p&gt;&#xA;&lt;p&gt;I thought it was possible they used the correct critical probability in the original submission and a reviewer requested it was changed. However I checked their medRxiv upload &lt;a href=&quot;https://www.medrxiv.org/content/10.1101/2020.07.16.20155606v1.full.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and the authors clearly use this threshold prior their formal submission.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We performed stepwise conditional analysis 479 to identify additional&#xA;independent signals by adjusting for the most significant amino acid&#xA;480 position in each step until none met the significance threshold (&#xA;P = 5 10^-8)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Thus I don't know.&lt;/p&gt;&#xA;&lt;p&gt;If this paper is central to your area of interest I would use the corrected critical probability threshold and re-assess and extend their conclusions because 5 decimal places is a huge a difference.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-05-19T12:31:02.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19084" PostTypeId="1" CreationDate="2022-05-19T15:48:32.537" Score="1" ViewCount="1003" Body="&lt;p&gt;I'm new to biopython &amp;amp; I'm trying to extract residues from specific chain of each file from a list of 316 pdb files. Eg: &lt;a href=&quot;https://www.rcsb.org/structure/1d8t&quot; rel=&quot;nofollow noreferrer&quot;&gt;1d8t&lt;/a&gt; has four chains A, B, C &amp;amp; D of which my interest is to extract just the residues of C chain. As the chain of interest differs for different files, I'm not knowing how to extract specific chain residue from each file and save it separately.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import *&#xA;from Bio.PDB.PDBParser import PDBParser&#xA;from Bio.PDB.Polypeptide import PPBuilder&#xA;structure = PDBParser().get_structure('1d8t', '1d8t.pdb')    &#xA;&#xA;&#xA;model = structure[0]&#xA;chain = model['C']&#xA;&#xA;for i in chain.get_residues():&#xA;    print(i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With the above code I was able to extract the residues of C chains not sure how to proceed for multiple files in python.&lt;/p&gt;&#xA;" OwnerUserId="15511" LastActivityDate="2022-05-27T18:57:11.970" Title="How to extract residues from specific chain of interest from multiple pdb files?" Tags="|python|pdb|parsing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19086" PostTypeId="2" ParentId="19081" CreationDate="2022-05-20T04:28:23.313" Score="1" Body="&lt;p&gt;As a direct answer to your question, I would argue that the authors, in choosing this threshold, have demonstrated a lack of understanding of their process of analysis. But that answer is not a full answer, and it's a problematic answer because [almost] everyone who does population genetics does similar things. &lt;strong&gt;This p-value is not appropriate, not merely because it is at a genome-wide significance level, but because &lt;em&gt;no&lt;/em&gt; threshold is appropriate.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;To explain my perspective more, thresholding to identify &amp;quot;statistically-significant&amp;quot; results, in general, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5187603/#__p13&quot; rel=&quot;nofollow noreferrer&quot;&gt;is not a good approach&lt;/a&gt;. Regardless of what threshold is used, p-values only indicate the confidence in a statistic's value (as it relates to fitting a particular statistical model), not importance or the size of the value.&lt;/p&gt;&#xA;&lt;p&gt;The authors of this paper, as with many other authors who have written similar papers, make an obvious statistical error in the legend of figure 3, where they state: &amp;quot;One-field classical allele HLA-B*57 (P = 9.84 × 10−138) (a) and amino acid position 97 in HLA-B (Pomnibus = 1.86 × 10−184) (b) showed the strongest association signal.&amp;quot; This is an error, because they have only reported here the p-values, and p-values have no relevance to the strength or magnitude of a statistic.&lt;/p&gt;&#xA;&lt;p&gt;Given this, it doesn't make sense to rank by p-value, and it especially doesn't make sense to display p-values on a graph and highlight p-value peaks as indicating association peaks. Those regions are &lt;em&gt;confidence&lt;/em&gt; peaks.&lt;/p&gt;&#xA;&lt;p&gt;In an attempt to provide some constructive light around this, I will answer the obvious follow-up questions are &amp;quot;How &lt;em&gt;should&lt;/em&gt; p-values be used?&amp;quot; and &amp;quot;If p-values shouldn't be displayed or ranked, then what should?&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;My preference is to use p-values as a loose filter, to exclude obviously incorrect results. In the case of a GWAS, I might, for example, choose to exclude any statistic that has a p-value above 0.1, leaving the remainder as &lt;em&gt;potentially&lt;/em&gt; correct. It could possibly be argued that this is still thresholding. It is, in a sense, but importantly I am not saying that all values that pass the threshold are good, just that the ones that are excluded are all bad; I allow for some rubbish to end up in the good pile. I liken it to using a computer to find the things that look like diamonds, so that I don't waste my time inspecting dirt.&lt;/p&gt;&#xA;&lt;p&gt;After that filtering is done, the p-values should be ignored, and the descriptive statistic used to rank, determine significance, association, or whatever the goal of the analysis is. For GWAS, a commonly-used value is beta, which can be positive or negative depending on the direction of association, and relates to the variant difference between groups.&lt;/p&gt;&#xA;&lt;p&gt;If a list needs to be cut down, then this descriptive statistic should be used for thresholding, depending on the analysis. For example, it might be appropriate to display the 100 most extreme beta values, or threshold to exclude any beta values that have a magnitude less than 0.01, or display a manhattan-like plot with beta on the &lt;span class=&quot;math-container&quot;&gt;$y$&lt;/span&gt; axis, and chromosome position on the &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; axis.&lt;/p&gt;&#xA;&lt;p&gt;There shouldn't be set rules, because every situation is different - there's a human element to this. What I'm saying is that it should be the job of the researcher, not the computer, to identify which data are significant. Make the data set small enough that it can be processed by a person, then let them do the rest. I think Hannah Fry explains this quite well in &lt;a href=&quot;https://www.youtube.com/watch?v=Rzhpf1Ai7Z4&amp;amp;t=1944s&quot; rel=&quot;nofollow noreferrer&quot;&gt;this talk&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Now this is the stuff the algorithms are amazing at: looking for tiny, tiny clues in seemingly completely disconnected data sets.... Just incredibly, incredibly sensitive these algorithms, right? Just absolutely amazing what they can do. And yet remember that sensitivity does not make a perfect algorithm. That's only one half of the equation.... [Humans] are terrible at sensitivity, right? We are really, really bad. We miss things all the time. But, specificity, being specific, that's like our superpower.... And rather than choosing between human and machine, which is kind of the rhetoric that we get so much, why don't we exploit each other's strengths and just create much more of a partnership?.... Because the algorithm never gets tired, so let it trawl through all of that data and just highlight a few key areas of concern. And then the human never misdiagnoses. So they can come in, and just sweep up and have the final say.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-05-22T03:01:51.780" LastActivityDate="2022-05-22T03:01:51.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19087" PostTypeId="1" CreationDate="2022-05-20T12:30:50.167" Score="2" ViewCount="368" Body="&lt;p&gt;I generated BAM files (sorted by coordinates) by aligning human RNA reads against the human reference genome using &lt;code&gt;BWA MEM&lt;/code&gt; and &lt;code&gt;TopHat2&lt;/code&gt;. I would like to know how I can retain only reads that mapped exclusively to a single site in the reference genome with one or zero mismatches for downstream analysis.&lt;/p&gt;&#xA;&lt;p&gt;For retaining reads mapped to only one site I tried the following two commands, using &lt;code&gt;grep&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h file1.sorted.bam | grep -v -e 'XA:Z:' -e 'SA:Z:' |&#xA;samtools view -b &amp;gt; file1.filtered_unqiue.sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or filtering by &lt;code&gt;mapping quality&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -q 10 -b file1.sorted.bam &amp;gt; file1.filtered_unique.sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then I filtered &lt;code&gt;file1.filtered_unique.sorted.bam&lt;/code&gt; of either two commands above for mismatches with&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -e '[NM]&amp;lt;=0' -O BAM -o&#xA;file1.filtered_unique_mismatches.sorted.bam   &#xA;file1.filtered_unique.sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am quite unsure about the correctness of the results.&lt;/p&gt;&#xA;&lt;p&gt;I also came across a one-liner combining both efforts, where the BAM is first intersected with the coordinates of interest and then filtered for mismatches:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h file1.sorted.bam chr:start-end | egrep '(^@|(NM:i:(0|1)))' &amp;gt; file1.filtered_unique_mismatches.sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, here I think this would also include multiple mapped reads since I only look for reads mapped in a specified region?&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have an advice what the best solution is?&lt;/p&gt;&#xA;" OwnerUserId="15514" LastEditorUserId="3694" LastEditDate="2022-05-20T15:20:06.593" LastActivityDate="2022-05-20T15:20:06.593" Title="How to extract reads that map exclusively to a single site with 1 or zero mismatches from BAM files" Tags="|rna-seq|sam|samtools|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19088" PostTypeId="1" CreationDate="2022-05-20T14:58:13.120" Score="0" ViewCount="196" Body="&lt;p&gt;I have snakemake script doing fastqc with a wrapper but it errors out saying&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Command 'set -euo pipefail;  /home/mi/ppp/miniconda3/envs/meta/bin/python3.9 /workflow/.snakemake/scripts/tmpsl695gpr.wrapper.py' returned non-zero exit status 1.&#xA;  File &amp;quot;/workflow/rules/fastqc.smk&amp;quot;, line 32, in __rule_fastq1&#xA;  File &amp;quot;/home/mi/ppp/miniconda3/envs/meta/lib/python3.9/concurrent/futures/thread.py&amp;quot;, line 58, in run&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;my rule looks like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule fastq1:&#xA;    input: &#xA;        &amp;quot;../resources/samples/{sample}_1.fq.gz&amp;quot;&#xA;    output:&#xA;        html = &amp;quot;../results/html/{sample}_1.fq.html&amp;quot;,&#xA;        zip = &amp;quot;../results/zip/{sample}_1.fq.zip&amp;quot;&#xA;    conda: &#xA;        &amp;quot;../envs/meta.yaml&amp;quot;&#xA;    log: &#xA;        &amp;quot;../results/logs/fastq_raw/fastq_raw_{sample}.log&amp;quot;&#xA;    threads: &#xA;        2&#xA;    wrapper:&#xA;        &amp;quot;v1.5.0/bio/fastqc&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have a feeling this used to work before and my partner says it works fine on his machine but I can't find the issue.&lt;/p&gt;&#xA;&lt;p&gt;I'm also doing this on a clean conda/mamba install and env.&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&#xA;Snakemake version is 7.7.0&#xA;I &amp;quot;solved&amp;quot; this by installing fastqc with conda... I mean am I stupid? Is it a requirement for a wrapper to also have the package installed? I don't think this was mentioned on the Snakemake website&lt;/p&gt;&#xA;&lt;p&gt;(Im not sure wht mr communuty bot is complaining about the problem is the error message and me not being able to run the code..)&lt;/p&gt;&#xA;&lt;p&gt;Thanks for reading.&lt;/p&gt;&#xA;" OwnerUserId="4171" LastEditorUserId="4171" LastEditDate="2022-05-21T07:36:31.123" LastActivityDate="2022-05-21T07:36:31.123" Title="set -euo pipefail in Snakemake wrapper" Tags="|snakemake|conda|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19089" PostTypeId="1" CreationDate="2022-05-20T15:37:36.387" Score="1" ViewCount="120" Body="&lt;p&gt;I have integrated top ten TCR clones into Seurat object something like below&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QB0Dg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QB0Dg.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You are seeing some clones are expressed in more than one cells&lt;/p&gt;&#xA;&lt;p&gt;So in Seurat I have something like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(pbmc@meta.data$highlight)&#xA;[1] NA                              &amp;quot;CAGVDSNYQLIW_CASSPVAGGSYNEQFF&amp;quot; NA                             &#xA;[4] NA                              NA                              NA           &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Based on the number of cells in which each clone is expressed, I want to rank clones and says how percentage of cells expressed each clone these figures below&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9xmRp.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9xmRp.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;Can you help me with this please?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2022-05-20T15:37:36.387" Title="Different colors in UMAP" Tags="|scrnaseq|seurat|10x-genomics|umap|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19090" PostTypeId="1" CreationDate="2022-05-20T15:46:43.340" Score="1" ViewCount="497" Body="&lt;p&gt;I am reshaping my dataframe as follows.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cOU0g.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cOU0g.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want to set the &lt;code&gt;Wavelength column&lt;/code&gt; as the column names with the code as follows.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dcast(unique(df1, by=cols), ID + Condition ~ Wavelength, value.var = 'mean')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But all the values &lt;code&gt;became 1s&lt;/code&gt; as follows.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yDRCI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yDRCI.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would appreciate any help.&lt;/p&gt;&#xA;&lt;p&gt;Yanpeng&lt;/p&gt;&#xA;" OwnerUserId="8757" LastActivityDate="2022-06-19T18:07:20.773" Title="Why all values become 1 after dcast?" Tags="|r|statistics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19091" PostTypeId="2" ParentId="19090" CreationDate="2022-05-20T17:28:45.040" Score="1" Body="&lt;p&gt;That comment is right, it's a general programming question, and text would be much easier for people to help with rather than images.&lt;/p&gt;&#xA;&lt;p&gt;But, since I ran into this not long ago...&lt;/p&gt;&#xA;&lt;p&gt;Take a look at the message shown in blue just above the output table.  My guess is you have some repeated ID/Condition/Wavelength combinations and it's defaulting to reporting the number of matching rows in the long-format data frame.  You'll need to either make sure you have all unique combinations, or explicitly define what you expect it to do with multiple values going into a single output row/column position (like, median/mean/max/min/whatever).  The unique() call won't do it since varying mean values would still make for unique input rows.&lt;/p&gt;&#xA;&lt;p&gt;For example compare this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; reshape2::dcast(data.frame(&#xA;+     Wavelength=c(664, 666, 662),&#xA;+     Condition=&amp;quot;late&amp;quot;,&#xA;+     ID=c(6221, 6222, 6223),&#xA;+     mean=c(106817, 110357, 112998)),&#xA;+   ID + Condition ~ Wavelength, value.var = &amp;quot;mean&amp;quot;)&#xA;    ID Condition    662    664    666&#xA;1 6221      late     NA 106817     NA&#xA;2 6222      late     NA     NA 110357&#xA;3 6223      late 112998     NA     NA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;with this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; reshape2::dcast(data.frame(&#xA;+     Wavelength=c(664, 664, 666, 662),&#xA;+     Condition=&amp;quot;late&amp;quot;,&#xA;+     ID=c(6221, 6221, 6222, 6223),&#xA;+     mean=c(106800, 106817, 110357, 112998)),&#xA;+   ID + Condition ~ Wavelength, value.var = &amp;quot;mean&amp;quot;)&#xA;Aggregation function missing: defaulting to length&#xA;    ID Condition 662 664 666&#xA;1 6221      late   0   2   0&#xA;2 6222      late   0   0   1&#xA;3 6223      late   1   0   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2022-05-20T17:28:45.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19092" PostTypeId="1" AcceptedAnswerId="19093" CreationDate="2022-05-20T19:50:20.360" Score="1" ViewCount="35" Body="&lt;p&gt;I am running the following Python code with &lt;code&gt;python requests&lt;/code&gt; in my Jupiter notebook to get cancer oncology id's from EBI-EMBL.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;cancer_all = requests.get('https://www.ebi.ac.uk/ols/api/ontologies/efo/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FMONDO_0000376/children')&#xA;obi = cancer_all.json()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above code is for one link. However I want to get results for 30 such links together.&lt;/p&gt;&#xA;&lt;p&gt;Other examples of links:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;https://www.ebi.ac.uk/ols/api/ontologies/efo/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FMONDO_0000621/children&#xA;https://www.ebi.ac.uk/ols/api/ontologies/efo/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FMONDO_0000637/children&#xA;https://www.ebi.ac.uk/ols/api/ontologies/efo/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FMONDO_0000653/children&#xA;https://www.ebi.ac.uk/ols/api/ontologies/efo/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FMONDO_0002402/children&#xA;https://www.ebi.ac.uk/ols/api/ontologies/efo/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FMONDO_0002516/children&#xA;https://www.ebi.ac.uk/ols/api/ontologies/efo/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FMONDO_0002813/children&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there any way I can run all these api in one go?&lt;/li&gt;&#xA;&lt;li&gt;Could I fetch the results (in json format) of these links in a single download. Thus there would be one json file for all the links?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4783" LastEditorUserId="3848" LastEditDate="2022-05-21T02:53:17.443" LastActivityDate="2022-05-21T02:53:17.443" Title="Running a list of apis via Python's request module" Tags="|api|python|jupyter|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19093" PostTypeId="2" ParentId="19092" CreationDate="2022-05-20T23:57:06.777" Score="1" Body="&lt;pre&gt;&lt;code&gt;import requests&#xA;import json #??&#xA;&#xA;apiString = &amp;quot;https://www.ebi.ac.uk/ols/api/ontologies/efo/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FMONDO_0000621/children\nhttps://www.ebi.ac.uk/ols/api/ontologies/efo/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FMONDO_0000637/children\nhttps://www.ebi.ac.uk/ols/api/ontologies/efo/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FMONDO_0000653/children\nhttps://www.ebi.ac.uk/ols/api/ontologies/efo/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FMONDO_0002402/children\nhttps://www.ebi.ac.uk/ols/api/ontologies/efo/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FMONDO_0002516/children\nhttps://www.ebi.ac.uk/ols/api/ontologies/efo/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FMONDO_0002813/children&amp;quot;&#xA;&#xA;def requestIt():&#xA;    cancer_all = requests.get('https://www.ebi.ac.uk/ols/api/ontologies/efo/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FMONDO_0000376/children')&#xA;    return cancer_all.json()&#xA;&#xA;if __name__ == '__main__':&#xA;callList = split('\n')&#xA;obi = [requestIt(x) for x in callList]&#xA;&#xA;# e.g. &#xA;print (obi[0]&#xA;&#xA;# an alternative approach to list comprehension &#xA;# computationally faster but will not make much difference here&#xA;obi = list(map(requestIt, callList)    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; If you want to parallelise it, which might be what you meant by 'all at once', you instantiate &lt;code&gt;Pool()&lt;/code&gt; from &lt;code&gt;multiprocessor&lt;/code&gt; via &lt;code&gt;map&lt;/code&gt; and use the same comprehension but this time the function &lt;code&gt;requestIt&lt;/code&gt; will spread this over your available CPUs. Thats a little bit more complicated - namely getting the result back.&lt;/p&gt;&#xA;&lt;p&gt;If the apis are in a file they will load into a list nicely:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with open ('apis.txt', 'r') as fin:&#xA;    callList = fin.readlines()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;However&lt;/strong&gt; you want to combine the json into a single json ... you &lt;code&gt;import json&lt;/code&gt; then use &lt;code&gt;json.loads&lt;/code&gt; to convert the json into a dictionary, combine the dictionaries and then use &lt;code&gt;json.dumps&lt;/code&gt; to convert it back to json. The code for that is a separate question altogether and the comprehension might be better expressed as a dictionary comprehension (not sure).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-05-21T02:46:28.647" LastActivityDate="2022-05-21T02:46:28.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19094" PostTypeId="1" CreationDate="2022-05-21T04:04:12.353" Score="0" ViewCount="16" Body="&lt;p&gt;I found that with this string-finding algorithm, this algo can determine if a string(person) holds something(if he has sick, already got vaccinated, and et cetera.).&lt;/p&gt;&#xA;&lt;p&gt;I am trying to simulate an epidemic spread with simulation parameters and i can't find a simple mathematical method or algorithm that can help me to understand how i will do it with the simulation process.&lt;/p&gt;&#xA;&lt;p&gt;ps: its my first time to ask a question here so dont judge if i cant follow the format..&lt;/p&gt;&#xA;" OwnerUserId="15517" LastActivityDate="2022-05-21T04:04:12.353" Title="Can I use Aho-Corasick for epidemic spread simulation?" Tags="|algorithms|simulated-data|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19097" PostTypeId="1" CreationDate="2022-05-21T20:06:15.660" Score="0" ViewCount="52" Body="&lt;p&gt;I have a set of 4 .bam files containing the exome of an individual, around 400 MB each. I used samtools to generate a 2.4 GB .sam file out of one of the .bam files, and I found it contains lines with 101 base sequences but seemingly no RNAME or POS information on them. The first columns of the first lines in the read like this (I won't include here the SEQ, QUAL and optional fields):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;210713_A00776_0149_AHCN5JDSX2:1:1101:10004:18818    77  *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10004:18818    141 *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10004:32033    77  *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10004:32033    141 *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10022:12712    77  *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10022:12712    141 *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10022:21261    77  *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10022:21261    141 *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10022:34882    77  *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10022:34882    141 *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10022:07764    77  *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10022:07764    141 *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10031:11569    77  *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10031:11569    141 *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10031:26537    77  *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10031:26537    141 *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10041:21010    77  *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10041:21010    141 *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10041:27085    77  *   0   0   *   *   0   0&#xA;210713_A00776_0149_AHCN5JDSX2:1:1101:10041:27085    141 *   0   0   *   *   0   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Most fields are nullified. As far as I can understand, subsequent lines with the 77 and 141 flags form corresponding pairs of bases and the QNAME field first element is an individual identifier, as it won't change through the head and tail of the file. Same goes for the second element (1). The last three elements change and might hold the information I need, but I'm not sure how to read them.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What I want to do is to map bases to SNPs (or a nucleotide identifier like NC_000006.11).&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have no real experience with bioinformatics besides enriching a file from MyHeritage (which contains around 609,000 base pairs with a corresponding rSNP number and chromossome) and crossed it with a reference human genome database I downloaded from NBCI. As it's a personal project, I &lt;em&gt;somewhat&lt;/em&gt; validated the final result by correctly spotting a few known mutations/polymorphisms for that individual tested through usual lab tests, like the one output by the code I've wrote:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; RS1799945 (Chromosome 6 ): CG (NC_000006.11:g.26091179C&amp;gt;G, snv), https://www.ncbi.nlm.nih.gov/snp/rs1799945&#xA;          Reference Allele: C&#xA;                 Frequency:  9.6% (0.09631744626441) (~ 1:10)&#xA;              Loci (Genes): HFE  (homeostatic iron regulator)&#xA;                          : LOC108783645  (HFE antisense RNA)&#xA;        Detected Mutations: c.187C&amp;gt;G&#xA;             Variation URL: https://www.ncbi.nlm.nih.gov/clinvar/variation/10/&#xA;                  Diseases: not provided (conflicting-interpretations-of-pathogenicity), None (other), Hereditary hemochromatosis (pathogenic), Cystic fibrosis (risk-factor), Alzheimer disease (pathogenic), Familial porphyria cutanea tarda (None), Hemochromatosis type 1 (None), Microvascular complications of diabetes 7 (None), Transferrin serum leve, Microvascular complications of diabetes 7 (risk-factor), Hemochromatosis type 1 (pathogenic), Hemochromatosis type 2 (pathogenic)&#xA;                    PubMed: https://www.ncbi.nlm.nih.gov/pubmed/8696333, https://www.ncbi.nlm.nih.gov/pubmed/8896550, https://www.ncbi.nlm.nih.gov/pubmed/8943161, https://www.ncbi.nlm.nih.gov/pubmed/9106528, https://www.ncbi.nlm.nih.gov/pubmed/9326341, https://www.ncbi.nlm.nih.gov/pubmed/9462220, https://www.ncbi.nlm.nih.gov/pubmed/10673304, https://www.ncbi.nlm.nih.gov/pubmed/11358905, https://www.ncbi.nlm.nih.gov/pubmed/11423500, https://www.ncbi.nlm.nih.gov/pubmed/11812557, https://www.ncbi.nlm.nih.gov/pubmed/11874997, https://www.ncbi.nlm.nih.gov/pubmed/11904676, https://www.ncbi.nlm.nih.gov/pubmed/12429850, https://www.ncbi.nlm.nih.gov/pubmed/14673107, https://www.ncbi.nlm.nih.gov/pubmed/15347835, https://www.ncbi.nlm.nih.gov/pubmed/15858186, https://www.ncbi.nlm.nih.gov/pubmed/18194558, https://www.ncbi.nlm.nih.gov/pubmed/18199861, https://www.ncbi.nlm.nih.gov/pubmed/18566337, https://www.ncbi.nlm.nih.gov/pubmed/18603647, https://www.ncbi.nlm.nih.gov/pubmed/18795173, https://www.ncbi.nlm.nih.gov/pubmed/19084217, https://www.ncbi.nlm.nih.gov/pubmed/19159930, https://www.ncbi.nlm.nih.gov/pubmed/19165391, https://www.ncbi.nlm.nih.gov/pubmed/19237423, https://www.ncbi.nlm.nih.gov/pubmed/19401444, https://www.ncbi.nlm.nih.gov/pubmed/19554541, https://www.ncbi.nlm.nih.gov/pubmed/19862010, https://www.ncbi.nlm.nih.gov/pubmed/19879291, https://www.ncbi.nlm.nih.gov/pubmed/19884647, https://www.ncbi.nlm.nih.gov/pubmed/20110814, https://www.ncbi.nlm.nih.gov/pubmed/20301613, https://www.ncbi.nlm.nih.gov/pubmed/20435227, https://www.ncbi.nlm.nih.gov/pubmed/20556870, https://www.ncbi.nlm.nih.gov/pubmed/20659343, https://www.ncbi.nlm.nih.gov/pubmed/20846924, https://www.ncbi.nlm.nih.gov/pubmed/20876667, https://www.ncbi.nlm.nih.gov/pubmed/21138591, https://www.ncbi.nlm.nih.gov/pubmed/21149283, https://www.ncbi.nlm.nih.gov/pubmed/21208937, https://www.ncbi.nlm.nih.gov/pubmed/21540342, https://www.ncbi.nlm.nih.gov/pubmed/21553046, https://www.ncbi.nlm.nih.gov/pubmed/21679129, https://www.ncbi.nlm.nih.gov/pubmed/21909115, https://www.ncbi.nlm.nih.gov/pubmed/21978626, https://www.ncbi.nlm.nih.gov/pubmed/22074419, https://www.ncbi.nlm.nih.gov/pubmed/22100073, https://www.ncbi.nlm.nih.gov/pubmed/22125427, https://www.ncbi.nlm.nih.gov/pubmed/22232660, https://www.ncbi.nlm.nih.gov/pubmed/22408404, https://www.ncbi.nlm.nih.gov/pubmed/22611049, https://www.ncbi.nlm.nih.gov/pubmed/22735619, https://www.ncbi.nlm.nih.gov/pubmed/22815867, https://www.ncbi.nlm.nih.gov/pubmed/23389292, https://www.ncbi.nlm.nih.gov/pubmed/23446634, https://www.ncbi.nlm.nih.gov/pubmed/23792061, https://www.ncbi.nlm.nih.gov/pubmed/23935582, https://www.ncbi.nlm.nih.gov/pubmed/23996192, https://www.ncbi.nlm.nih.gov/pubmed/24033266, https://www.ncbi.nlm.nih.gov/pubmed/24121126, https://www.ncbi.nlm.nih.gov/pubmed/24398642, https://www.ncbi.nlm.nih.gov/pubmed/24755831, https://www.ncbi.nlm.nih.gov/pubmed/24798971, https://www.ncbi.nlm.nih.gov/pubmed/24825461, https://www.ncbi.nlm.nih.gov/pubmed/25071582, https://www.ncbi.nlm.nih.gov/pubmed/25112650, https://www.ncbi.nlm.nih.gov/pubmed/25287020, https://www.ncbi.nlm.nih.gov/pubmed/25741868, https://www.ncbi.nlm.nih.gov/pubmed/26024779, https://www.ncbi.nlm.nih.gov/pubmed/26025379, https://www.ncbi.nlm.nih.gov/pubmed/26159428, https://www.ncbi.nlm.nih.gov/pubmed/26365338, https://www.ncbi.nlm.nih.gov/pubmed/26416403, https://www.ncbi.nlm.nih.gov/pubmed/26597663, https://www.ncbi.nlm.nih.gov/pubmed/26695521, https://www.ncbi.nlm.nih.gov/pubmed/26716680, https://www.ncbi.nlm.nih.gov/pubmed/26833210, https://www.ncbi.nlm.nih.gov/pubmed/27221532, https://www.ncbi.nlm.nih.gov/pubmed/27255824, https://www.ncbi.nlm.nih.gov/pubmed/27280446, https://www.ncbi.nlm.nih.gov/pubmed/27317329, https://www.ncbi.nlm.nih.gov/pubmed/27437086, https://www.ncbi.nlm.nih.gov/pubmed/27547017, https://www.ncbi.nlm.nih.gov/pubmed/27584680, https://www.ncbi.nlm.nih.gov/pubmed/27657935, https://www.ncbi.nlm.nih.gov/pubmed/27661980, https://www.ncbi.nlm.nih.gov/pubmed/28151915, https://www.ncbi.nlm.nih.gov/pubmed/28684612, https://www.ncbi.nlm.nih.gov/pubmed/28820869, https://www.ncbi.nlm.nih.gov/pubmed/29167213, https://www.ncbi.nlm.nih.gov/pubmed/29201641, https://www.ncbi.nlm.nih.gov/pubmed/30409984, https://www.ncbi.nlm.nih.gov/pubmed/30536387, https://www.ncbi.nlm.nih.gov/pubmed/30798813, https://www.ncbi.nlm.nih.gov/pubmed/31220083, https://www.ncbi.nlm.nih.gov/pubmed/31226389, https://www.ncbi.nlm.nih.gov/pubmed/31469255, https://www.ncbi.nlm.nih.gov/pubmed/31970519&#xA;             Polymorphisms: c.118=, c.187=, c.118C&amp;gt;G, c.187C&amp;gt;G, g.8671=, g.8671C&amp;gt;G, g.26090951=, g.26091179=, g.26090951C&amp;gt;G, g.26091179C&amp;gt;G, p.His40=, p.His63=, p.His40Asp, p.His63Asp, n.84=, n.281=, n.309=, n.84G&amp;gt;C, n.281C&amp;gt;G, n.309C&amp;gt;G&#xA;     Clinical Significance: RCV000000026.18, RCV000000027.5, RCV000175607.6, RCV000394716.6, RCV000763144.1, RCV000844708.1, RCV000991133.1&#xA;                      HGVS:&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.reddit.com/r/bioinformatics/comments/uuuyqx/bam_file_and_no_rname_or_pos_information/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reddit&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="15521" LastEditorUserId="15521" LastEditDate="2022-05-22T19:06:23.180" LastActivityDate="2022-05-22T19:06:23.180" Title="BAM files with no RNAME and POS, how to map contents to SNPs?" Tags="|sam|snp|samtools|identifiers|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19098" PostTypeId="1" CreationDate="2022-05-21T23:23:50.397" Score="1" ViewCount="119" Body="&lt;p&gt;I'm interested in estimating global FST among several populations of a tetraploid plant species. I'm looking for an R package that can do this, and can preferably also produce an upper and lower bound for a 95% confidence interval for FST. Does anyone know of a package that can be used for this purpose? I'll post a brief overview of my understanding of some of the options, in case I am not understanding something 100% correctly:&lt;/p&gt;&#xA;&lt;p&gt;The StAMPP package seems really useful for this sort of thing with tetraploid or mixed populations, but unfortunatley it calculates pairwise FST and not a global measure. I'm aware that adegenet can calculate global FST for tetraploid populations, but it doesn't produce the confidence interval, and hierfstat can only be used with haploid or diploid populations. I've also came across the assigner package &lt;a href=&quot;https://thierrygosselin.github.io/assigner/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://thierrygosselin.github.io/assigner/index.html&lt;/a&gt;, but am not sure if this is valid for tetraploid populations.&lt;/p&gt;&#xA;" OwnerUserId="15522" LastActivityDate="2022-05-21T23:23:50.397" Title="Is there an R package to calculate global FST, with confidence intervals, for tetraploid populations" Tags="|r|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19099" PostTypeId="2" ParentId="19097" CreationDate="2022-05-22T02:55:24.730" Score="2" Body="&lt;p&gt;These lines (with no REF or POS) are unmapped reads. If those lines are the entirety of the BAM file, then it is simply another file format to store the raw sequenced reads (as an alternative to FASTQ files). It's somewhat common for BAM files to include &lt;em&gt;all&lt;/em&gt; reads, with mapped reads sorted by chromosome and position at the start, and unmapped reads appearing at the end of the file. This means that only the BAM file needs to be transferred and stored, rather than a BAM file and a separate 'unmapped' file (which can change size/content depending on the reference genome and mapper used).&lt;/p&gt;&#xA;&lt;p&gt;FWIW, the lines of a BAM file can be viewed using &lt;code&gt;samtools view&lt;/code&gt;; there's no need to convert to SAM first:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h input.bam | less&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you only want to look at the unmapped reads, then you can filter for inclusion using the read flag 4:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -f 0x04 -h input.bam | less&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you want to &lt;em&gt;exclude&lt;/em&gt; unmapped reads, you can filter that flag for exclusion (capital &lt;code&gt;F&lt;/code&gt; instead of lower-case &lt;code&gt;f&lt;/code&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -F 0x04 -h input.bam | less&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-05-22T02:55:24.730" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19100" PostTypeId="1" AcceptedAnswerId="19101" CreationDate="2022-05-22T17:45:28.440" Score="0" ViewCount="53" Body="&lt;p&gt;What does it mean by &amp;quot;topology&amp;quot; in the case of secondary and tertiary structures of proteins?&lt;/p&gt;&#xA;&lt;p&gt;N.B. I am not talking about DNA/RNA/genomics. I am talking about protein folding, as it relates to protein modeling and simulation.&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="73" LastEditDate="2022-05-23T03:00:59.467" LastActivityDate="2022-05-23T10:24:18.947" Title="What does it mean by &quot;topology&quot; in the case of secondary and tertiary structures of proteins?" Tags="|proteins|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19101" PostTypeId="2" ParentId="19100" CreationDate="2022-05-22T18:24:31.760" Score="2" Body="&lt;p&gt;The word topology gets used in two different contexts.&lt;/p&gt;&#xA;&lt;h2&gt;Common case: the geometric configuration of the protein (=fold)&lt;/h2&gt;&#xA;&lt;p&gt;In common non-computational biochemistry parlance it is just the standard English meaning of topology and just geometry, in the case of protein it means protein fold:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Protein_fold_class&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Protein_fold_class&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Wiktionary (&lt;a href=&quot;https://en.wiktionary.org/wiki/topology&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wiktionary.org/wiki/topology&lt;/a&gt;) say:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The branch of mathematics dealing with those properties of a geometrical object (of arbitrary dimensionality) that are unchanged by continuous deformations (such as stretching, bending, etc., without tearing or gluing)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the of protein, unchanged by small changes in length or amino acid sequence identity —no proteases or meat-glue ligase required!&lt;/p&gt;&#xA;&lt;h2&gt;Computational (Bio)chemistry&lt;/h2&gt;&#xA;&lt;p&gt;Topology has a technical jargon definition in computational biochemistry and computational chemistry.&lt;/p&gt;&#xA;&lt;p&gt;In computational chemistry how the connections of a molecule are configured, expressed as a graph network &lt;a href=&quot;https://en.wikipedia.org/wiki/Graph_theory&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Graph_theory&lt;/a&gt;, is referred to as topology. For example isopropanol and n-propanol have same number of heavy atom (&lt;em&gt;nodes&lt;/em&gt;) and these will have the same hybridisation, but the network will have different connectivities (&lt;em&gt;vertices&lt;/em&gt;).&#xA;The topology definition generally runs off &lt;em&gt;atom types&lt;/em&gt; more so than elements, these combine the element and hybridisation and other properties specific to a certain element in a molecule —e.g. amine vs. amide nitrogen vs. tryptophan ring nitrogens are different.&#xA;The 3D embedding of said molecule is not part of its topology. Things get a bit problematic as ideal bond lengths, angles and dihedrals in dihedral space (i.e. not in cartesian space) go into the 'configuration' file (topology file) for several forcefield systems. And in others still there may be constraints or restraints specified.&lt;/p&gt;&#xA;&lt;p&gt;In computational biochemistry, this holds true for ligands. But with proteins there is less. The connectivity within each amino acids is preset in stone: where one to use the allo isomer of threonine as opposed to the threo one would need to define a non-canonical amino acid. So by topology of a protein it is intended the connectivity of the amino acids in chains, e.g. primary sequence devoid of cartesian coordinates.&#xA;The word topology appears often in the context of ligands: these require custom definitions (=topologies) required to use a ligand not parameterised in the force field used, a common source of trouble.&lt;/p&gt;&#xA;&lt;h2&gt;See also&lt;/h2&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://manual.gromacs.org/documentation/2019/reference-manual/topologies.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://manual.gromacs.org/documentation/2019/reference-manual/topologies.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://mdtraj.org/1.9.4/api/generated/mdtraj.Topology.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://mdtraj.org/1.9.4/api/generated/mdtraj.Topology.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2022-05-23T10:24:18.947" LastActivityDate="2022-05-23T10:24:18.947" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19102" PostTypeId="2" ParentId="19097" CreationDate="2022-05-22T18:46:18.710" Score="0" Body="&lt;p&gt;This is an unmapped .bam file.  Someone just converted the fastq to bam format, but no one has aligned these to anything.  So you need to find an aligner program and a genome and align them.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2022-05-22T18:46:18.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19103" PostTypeId="1" CreationDate="2022-05-23T08:37:50.050" Score="1" ViewCount="107" Body="&lt;p&gt;Seurat V3 has a function &lt;code&gt;CreateGeneActivityMatrix&lt;/code&gt;, which can convert ATAC peaks to gene-level's matrix.&lt;/p&gt;&#xA;&lt;p&gt;Here is the prototype:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CreateGeneActivityMatrix(&#xA;  peak.matrix,&#xA;  annotation.file,&#xA;  seq.levels = c(1:22, &amp;quot;X&amp;quot;, &amp;quot;Y&amp;quot;),&#xA;  include.body = TRUE,&#xA;  upstream = 2000,&#xA;  downstream = 0,&#xA;  verbose = TRUE&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here are my questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I want to know what is the format of the first parameter &lt;code&gt;peak.matrix&lt;/code&gt; ? For example, what are the row names of this matrix? (Is there a strict rule of the row names?)&lt;/li&gt;&#xA;&lt;li&gt;The parameter &lt;code&gt;seq.levels&lt;/code&gt;. Besides 1...22,X,Y , Can I add other chromosome ? like &lt;code&gt;hs37d5&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="15425" LastEditorUserId="15425" LastEditDate="2022-05-23T11:16:08.873" LastActivityDate="2022-05-23T11:16:08.873" Title="Questions about CreateGeneActivityMatrix function in Seurat" Tags="|rna-seq|seurat|atac-seq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19104" PostTypeId="1" CreationDate="2022-05-23T11:42:46.980" Score="1" ViewCount="27" Body="&lt;p&gt;I'm a computer science person currently working on a project which uses deep learning to identify 5mC sites of Nipponbare Rice. Within the project, there is an option to view what the model 'discovers' through the generated motifs. Here are the motifs generated by most of the models&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UOhwS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UOhwS.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/nsvvF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nsvvF.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/74HHI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/74HHI.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;with the most common motif of being ACGGC. I've also read &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8809349/?fbclid=IwAR3h6riHhb22Z1mvtxduA7kkzPvmcmzbIK7zXFc4aN-OqOMsatP6BmulN5I&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; which states that&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Alignment of the 5mC methylation sites indicated the conserved dinucleotide CG in the center with less conserved flanking bases.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The 5mC motif was similar to the methylation motif 5′-CmCGG-3′ of NgoAXIV in Neisseria gonorrhoeae.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Am I correct in assuming that the model has 'discovered' something that has been in the literature, specifically the CG motif? Also, how do I make sense of the second statement in terms of the connection of the identified motif in rice and in a bacteria?&lt;/p&gt;&#xA;" OwnerUserId="15528" LastActivityDate="2022-05-23T11:42:46.980" Title="How do I intrepret the generated motifs of a dataset of 5-methylcytosine (5mC) sequences?" Tags="|methylation|motifs|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19105" PostTypeId="1" AcceptedAnswerId="19109" CreationDate="2022-05-23T14:02:13.040" Score="3" ViewCount="1120" Body="&lt;p&gt;I am looking for a Python package that performs pairwise structural alignment of protein structures (i.e., PDB files) and returns a sequence alignment. PyMOL is able to do this through the GUI, for example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mtLMF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mtLMF.png&quot; alt=&quot;PyMOL GUI showing two protein structures and their sequences&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For two protein PDBs, one can be aligned to the other with the &amp;quot;align&amp;quot; menu or &lt;code&gt;align&lt;/code&gt; command:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/weICw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/weICw.png&quot; alt=&quot;PyMOL GUI showing the two protein structures superimposed and their sequences aligned position by position (and with gaps)&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;With it, the sequences appear aligned in the sequence viewer according to the structural alignment, and an &amp;quot;alignment&amp;quot; object is created, which can be saved with the command:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;save A_aligned_with_B.aln, aln_structure_1_to_structure_0&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;which produces the &lt;code&gt;A_aligned_with_B.aln&lt;/code&gt; alignment file in Clustal format (nevermind the ?s, which correspond to special HIS resnames):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CLUSTAL&#xA;&#xA;structure_0  YIYQDSISLPWKVLLVMLLALITLATTLSNAFVIATVYRTRKL?TPANYLIASLAVTDLLVSI&#xA;structure_1  EEQGNKL?WAALLILMVIIPTIG-----GNTLVILAVSLEKKLQYATNYFLMSLAVADLLVGL&#xA;                                         .*..**..*....**....**...****.****..&#xA;&#xA;structure_0  LVMPISTMYTVT-GRWTLGQVVCDFWLSSDITCCTASIL?LCVIALDRYWAITDAVEYSAKRT&#xA;structure_1  FVMPIALLTIMFEAMWPLPLVLCPAWLFLDVLFSTASIM?LCAISVDRYIAIKKPIQANQYNS&#xA;             .****......  ..*.*..*.*..**..*....****.***.*..***.**........ ..&#xA;&#xA;structure_0  PKRAAVMIALVWVFSISISLPP-FFWRQAKAEEEVSE-CVVNTD?IL-YTVYSTVGAFYFPTL&#xA;structure_1  RATAFIKITVVWLISIGIAIPVPIKGIETDVDNPNNITCVLTKERFGDFMLFGSLAAFFTPLA&#xA;             ...*...*..**..**.*..*  .              **....    ........**..*..&#xA;&#xA;structure_0  LLIALYGRIYVEARSRILKQTPNRTGKLLEKKKLMAAR--ERKATKTLGIILGAFIVCWLPFF&#xA;structure_1  IMIVTYFLTI?ALQKKAYLVKNKPPQRLTIGKKSVQTISNEQRASKVLGIVFFLFLLMWCPFF&#xA;             ..*..*....                              * .*.*.***....*...*.***&#xA;&#xA;structure_0  IISLVMPICKDACW--F?LAIFDFFTWLGYLNSLINPIIYTMSNEDFKQAF?KLIRFK--   &#xA;structure_1  ITNITLVLCDS-CNQTTLQMLLEIFVWIGYVSSGVNPLVYTLFNKTFRDAFGRYITCNYR   &#xA;             *.......*   *   .  .....*.*.**..*..**..**..*..*..**         &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I need a Python package that is easily installable/distributable that accepts two PDBs or protein structures and returns the structural alignment of the sequences somehow. All within a Python script, and not creating intermediate files, e.g. for the alignment.&lt;/p&gt;&#xA;&lt;p&gt;There are plenty of Python tools that allow structural superimposition and RMSD calculation (e.g., Biopython), but I haven't found any that are able to return a sequence alignment from the structural alignment.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Possible answer, thanks to the answer and comments from &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/19109/15529&quot;&gt;Matteo Ferla&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pymol2&#xA;with pymol2.PyMOL() as pymol:&#xA;    pymol.cmd.load('foo.pdb', 'foo')&#xA;    pymol.cmd.load('bar.pdb', 'bar')&#xA;    pymol.cmd.align('foo', 'bar', object='aln')&#xA;    aln = pymol2.cmd2.pymol.exporting.get_alnstr('aln', _self=pymol.cmd) # I don't know if this line could be cleaner&#xA;    # aln is itself a string&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15529" LastEditorUserId="15529" LastEditDate="2022-06-27T14:52:18.250" LastActivityDate="2022-06-27T14:52:18.250" Title="Perform protein structure-based sequence alignment in Python" Tags="|sequence-alignment|proteins|protein-structure|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="19106" PostTypeId="1" CreationDate="2022-05-23T14:37:51.033" Score="0" ViewCount="17" Body="&lt;p&gt;I am trying to build a model for variant calling,but I am a bit confused how to pre-process the reads that has insertion and deletion and align them with reference?&lt;/p&gt;&#xA;&lt;p&gt;Should I add something to the location of insertion or deletion to reference or should I just delete from read?&lt;/p&gt;&#xA;" OwnerUserId="15531" LastActivityDate="2022-05-23T14:37:51.033" Title="How deep variant and others preprocessed insertion and deletion for extracting candidate variant?" Tags="|variant-calling|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19107" PostTypeId="1" CreationDate="2022-05-23T15:11:10.700" Score="2" ViewCount="32" Body="&lt;p&gt;I am planning to dock a protein to &lt;a href=&quot;https://www.selleckchem.com/screening/Flavonoid-Compound-Library.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Selleck Chem's Flavonoid Library&lt;/a&gt;. I have designed two workflows using Autodock Vina and Schrodinger as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Schrodinger&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use their proprietary tools for protein preparation followed by grid preparation under default settings.&lt;/li&gt;&#xA;&lt;li&gt;Run &lt;code&gt;LigPrep&lt;/code&gt; on the SelleckChem library to generate one ligand (at most) per compound under default settings.&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I executed another run (let's call it 'special run'; see below) generating eight ligands (at most) per compound under default settings.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;Run Glide in SP mode under default settings + add a reward for H-bonds.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Autodock Vina&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Run every module of UCSF Chimera's &lt;a href=&quot;https://www.cgl.ucsf.edu/chimera/docs/ContributedSoftware/dockprep/dockprep.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dock Prep&lt;/a&gt; on the Protein under default settings.&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Except use Reduce in place of AddH - &lt;a href=&quot;https://www.cgl.ucsf.edu/chimera/docs/ContributedSoftware/addh/addh.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;suggested&lt;/a&gt; by Chimera as a better alternative.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;Calculate grid-center to be the centroid of the default ligand and use the same grid size as used in Schrodinger workflow.&lt;/li&gt;&#xA;&lt;li&gt;Run &lt;code&gt;prepare_receptor4.py&lt;/code&gt; but in non-default settings to only remove lone pairs and non-standard residues.&#xA;&lt;ul&gt;&#xA;&lt;li&gt;We do not remove waters or add charges, as done under default settings; they have been already executed by Chimera.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;Download 3D files of each compound in the Selleckchem library from Pubchem in &lt;code&gt;sdf&lt;/code&gt; format.&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Use Obabel to convert them to &lt;code&gt;pdb&lt;/code&gt; format.&lt;/li&gt;&#xA;&lt;li&gt;Run &lt;code&gt;prepare_ligand4.py&lt;/code&gt; under default settings to get output in &lt;code&gt;pdbqt&lt;/code&gt; format.&lt;/li&gt;&#xA;&lt;li&gt;Use Obabel to replace Gasteigger charges with &lt;code&gt;eem2015ba&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Use Obabel to minimize energy using &lt;code&gt;MMFF94&lt;/code&gt; forcefield under default settings.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;Run Vina 1.2 to dock at &lt;code&gt;exhaustiveness&lt;/code&gt; of &lt;code&gt;32&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The results from the two workflows exhibit &lt;strong&gt;vast disparity&lt;/strong&gt; with the correlation coefficient of the rank being about &lt;code&gt;0.2&lt;/code&gt;. For example, the best result of Schrodinger is ranked 150 by Vina (the library has about 200 molecules in total) and so on. If I take the best 20 results from Vina and Schrodinger each, there is a single common compound.&lt;/p&gt;&#xA;&lt;p&gt;In my 'special run', I saw that Schrodinger assigns vastly different overall docking scores to various conformations of the same ligand. This might answer the above disparity unless I am doing something acutely wrong. &lt;em&gt;But&lt;/em&gt; that raises the question about how to choose the initial conformation of the ligand if I am constrained to use open-source software and starting conformations influence the final result so severely?&lt;/p&gt;&#xA;" OwnerUserId="15189" LastEditorUserId="15189" LastEditDate="2022-05-26T11:57:14.103" LastActivityDate="2022-05-26T11:57:14.103" Title="How to explain disparities in Flavonoid Docking results between Vina and Schrodinger?" Tags="|docking|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19109" PostTypeId="2" ParentId="19105" CreationDate="2022-05-24T09:11:47.763" Score="2" Body="&lt;p&gt;PyMOL in itself can be used as python package.&#xA;The GUI app has its own Python, but in a conda environment you can install it as a package.&#xA;Any PyMOL app command can be used as a cmd command. In PyMOL Wiki you see both. They follow a similar pattern and actually have more functionality.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pymol2&#xA;with pymol2.PYMOL() as pymol:&#xA;    pymol.cmd.load('foo.pdb', 'foo')&#xA;    pymol.cmd.load('bar.pdb', 'bar')&#xA;    pymol.cmd.align('foo', 'bar')&#xA;    pymol.cmd.save('foobar.aln')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For more discussion of how the alignment output object is used see &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/14919/assessing-pymol-sequence-alignment-object/14936#14936&quot;&gt;Assessing PyMol sequence alignment object&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2022-05-24T09:11:47.763" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="19110" PostTypeId="1" AcceptedAnswerId="19113" CreationDate="2022-05-24T13:56:51.650" Score="0" ViewCount="70" Body="&lt;p&gt;I have a question about syndip dataset : &lt;a href=&quot;https://github.com/lh3/CHM-eval&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/lh3/CHM-eval&lt;/a&gt; .&#xA;I'm struggling to find the syndip vcf.&lt;/p&gt;&#xA;&lt;p&gt;In the release ( &lt;a href=&quot;https://github.com/lh3/CHM-eval/releases&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/lh3/CHM-eval/releases&lt;/a&gt; ), we have a file named : &lt;code&gt;rep2.37.broad.hc.raw.vcf.gz&lt;/code&gt;, that i don't know what it is. And we have a file named &lt;code&gt;CHM-evalkit-20180222.tar&lt;/code&gt; which contains &lt;code&gt;full.37m.vcf&lt;/code&gt; and other files (bed, eval ...).&#xA;So i did my search and according to this file: they mentioned that full.37m.vcf is the truth dataset. ( &lt;a href=&quot;https://www.biorxiv.org/content/biorxiv/suppl/2018/10/29/456103.DC1/456103-1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/biorxiv/suppl/2018/10/29/456103.DC1/456103-1.pdf&lt;/a&gt; Page 16).&lt;/p&gt;&#xA;&lt;p&gt;The problem is that the file &lt;code&gt;rep2.37.broad.hc.raw.vcf.gz&lt;/code&gt; contain variants with &lt;code&gt;MQ, DP, GQ&lt;/code&gt; ... that I need to extract. But the &lt;code&gt;full.37m.vcf&lt;/code&gt; doesn't contain this information.. ( just &lt;code&gt;Chrom pos ref alt and QUAL&lt;/code&gt;.)&lt;/p&gt;&#xA;&lt;p&gt;So I tried to intersect &lt;code&gt;rep2.37.broad.hc.raw.vcf.gz&lt;/code&gt; with &lt;code&gt;full.37m.vcf&lt;/code&gt; and take the variant that present in two files, with the DP MQ GQ in &lt;code&gt;rep2.37.broad.hc.raw.vcf.gz&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Is that okay? Since I don't know what is &lt;code&gt;rep2.37.broad.hc.raw.vcf.gz&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;And I also noticed that the QUAL in the &lt;code&gt;full.37m.vcf&lt;/code&gt; is always 30. Is it normal?&lt;/p&gt;&#xA;" OwnerUserId="15536" LastEditorUserId="3694" LastEditDate="2022-05-24T15:46:30.087" LastActivityDate="2022-05-25T03:25:29.337" Title="syndip dataset for benchmark variant" Tags="|vcf|public-dataset|variants|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19112" PostTypeId="1" AcceptedAnswerId="19114" CreationDate="2022-05-24T21:05:34.143" Score="3" ViewCount="236" Body="&lt;p&gt;I have a process that emits BAM files with a key containing the sample name:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process source{&#xA;  output:&#xA;    tuple val(sample), path(&amp;quot;sample.bam&amp;quot;) into ch&#xA;&#xA;  [...]&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;1st question: I have several files per &lt;code&gt;sample&lt;/code&gt;, let's assume 2 for now. I want to merge them together with &lt;a href=&quot;http://www.htslib.org/doc/samtools-merge.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;samtools merge&lt;/code&gt;&lt;/a&gt;. I can regroup that channel with &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#grouptuple&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;groupTuple()&lt;/code&gt;&lt;/a&gt;, but how can I recover them in the next process? Considering they all have the same name:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grouped_ch = ch.groupTuple(by: 0, size: 2)&#xA;&#xA;process merge{&#xA;  input:&#xA;    grouped_ch&#xA;&#xA;  shell:&#xA;  '''&#xA;  samtool merge sample.bam sample.bam &amp;gt; !{sample}.bam&#xA;  '''&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there perhaps a way in the &lt;code&gt;input:&lt;/code&gt; block to say that I expect a list, and name the list components?&lt;/p&gt;&#xA;&lt;p&gt;2nd question: In practice, I have a variable number of &amp;quot;sample.bam&amp;quot; files for each &lt;code&gt;sample&lt;/code&gt; value, does that change the way to define the &lt;code&gt;input:&lt;/code&gt;? It also appears I will need to use the built-in &lt;code&gt;groupKey()&lt;/code&gt;, is there documentation available? I have read &lt;a href=&quot;https://github.com/nextflow-io/nextflow/issues/796#issuecomment-407108667&quot; rel=&quot;nofollow noreferrer&quot;&gt;this Github issue&lt;/a&gt;, but it would help if a general description was available.&lt;/p&gt;&#xA;" OwnerUserId="9497" LastActivityDate="2022-05-25T06:16:47.527" Title="recover files after groupTuple" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19113" PostTypeId="2" ParentId="19110" CreationDate="2022-05-25T03:25:29.337" Score="1" Body="&lt;p&gt;If you want the truth VCF, it is &lt;code&gt;full.37m.vcf&lt;/code&gt;. &lt;code&gt;rep2.37.broad.hc.raw.vcf.gz&lt;/code&gt; is an example call set by GATK.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2022-05-25T03:25:29.337" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19114" PostTypeId="2" ParentId="19112" CreationDate="2022-05-25T06:16:47.527" Score="3" Body="&lt;p&gt;My preference is to avoid working with files with the same name as much as possible. However, sometimes this might be unavoidable. Fortunately, file name collisions can be avoided when working with &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#multiple-input-files&quot; rel=&quot;nofollow noreferrer&quot;&gt;multiple input files&lt;/a&gt; by using the &lt;code&gt;*&lt;/code&gt; and &lt;code&gt;?&lt;/code&gt; wildcards:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;When a target file name is defined in the input parameter and a collection of files is received by the process, the file name will be appended by a numerical suffix representing its ordinal position in the list.&lt;/p&gt;&#xA;&lt;p&gt;The target input file name can contain the * and ? wildcards, that can be used to control the name of staged files.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Note that there is also the &lt;code&gt;stageAs&lt;/code&gt; path option (see &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#input-of-type-path&quot; rel=&quot;nofollow noreferrer&quot;&gt;input of type 'path'&lt;/a&gt;), which can also take a name pattern (using the above wildcards) to let you use a variable in your script. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process samtools_merge {&#xA;&#xA;    conda 'samtools=1.15.1'&#xA;    debug true&#xA;    stageInMode 'rellink'&#xA;&#xA;    input:&#xA;    tuple val(sample), path(bam_files, stageAs: 'mysample*.bam')&#xA;&#xA;    output:&#xA;    tuple val(sample), path(&amp;quot;${sample}.bam&amp;quot;)&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    samtools merge &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample}.bam&quot; $&lt;/span&gt;{bam_files}&#xA;    ls -nl&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;workflow {&#xA;&#xA;    Channel.fromPath( ['./path/to/files/*.bam', './path/to/more_files/*.bam'] ) \&#xA;        | map { bam -&amp;gt; tuple( bam.baseName, bam ) } \&#xA;        | groupTuple(by: 0, size: 2) \&#xA;        | samtools_merge&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;N E X T F L O W  ~  version 22.04.0&#xA;Launching `script.nf` [elated_montalcini] DSL2 - revision: 1427d64a1d&#xA;executor &amp;gt;  local (3)&#xA;[83/25654c] process &amp;gt; samtools_merge (3) [100%] 3 of 3 ✔&#xA;total 184&#xA;lrwxrwxrwx 1 1000 985     34 May 25 12:18 mysample1.bam -&amp;gt; ../../../path/to/files/sample1.bam&#xA;lrwxrwxrwx 1 1000 985     39 May 25 12:18 mysample2.bam -&amp;gt; ../../../path/to/more_files/sample1.bam&#xA;-rw-r--r-- 1 1000 985 187911 May 25 12:18 sample1.bam&#xA;&#xA;total 128&#xA;lrwxrwxrwx 1 1000 985     34 May 25 12:18 mysample1.bam -&amp;gt; ../../../path/to/files/sample2.bam&#xA;lrwxrwxrwx 1 1000 985     39 May 25 12:18 mysample2.bam -&amp;gt; ../../../path/to/more_files/sample2.bam&#xA;-rw-r--r-- 1 1000 985 129947 May 25 12:18 sample2.bam&#xA;&#xA;total 196&#xA;lrwxrwxrwx 1 1000 985     34 May 25 12:18 mysample1.bam -&amp;gt; ../../../path/to/files/sample3.bam&#xA;lrwxrwxrwx 1 1000 985     39 May 25 12:18 mysample2.bam -&amp;gt; ../../../path/to/more_files/sample3.bam&#xA;-rw-r--r-- 1 1000 985 197225 May 25 12:18 sample3.bam&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If each group contains a variable number of files, you can keep the same input declaration (as above) but you will want use a special &lt;a href=&quot;https://github.com/nextflow-io/nextflow/blob/v22.04.3/modules/nextflow/src/main/groovy/nextflow/extension/GroupKey.groovy&quot; rel=&quot;nofollow noreferrer&quot;&gt;groupKey&lt;/a&gt; object so that the collected values can be streamed as soon as possible. It's not well documented, but here's a simple example that sorts the input BAM files prior to merging. Once all of the 'samtools_sort' jobs are finished for a particular sample, that sample can proceed to merging without us having to wait for all the other 'samtools_sort' jobs to be finished (which would otherwise be the case if the 'size' parameter had not been specified):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process samtools_sort {&#xA;&#xA;    tag { bam.baseName }&#xA;&#xA;    conda 'samtools=1.15.1'&#xA;&#xA;    input:&#xA;    tuple val(sample), path(bam)&#xA;&#xA;    output:&#xA;    tuple val(sample), path(&amp;quot;${bam.baseName}.sorted.bam&amp;quot;)&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    samtools sort -o &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${bam.baseName}.sorted.bam&quot; &quot;$&lt;/span&gt;{bam}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process samtools_merge {&#xA;&#xA;    tag { sample }&#xA;&#xA;    conda 'samtools=1.15.1'&#xA;&#xA;    input:&#xA;    tuple val(sample), path(bam_files, stageAs: 'mysample*.bam')&#xA;&#xA;    output:&#xA;    tuple val(sample), path(&amp;quot;${sample}.bam&amp;quot;)&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    samtools merge &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample}.bam&quot; $&lt;/span&gt;{bam_files}&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow {&#xA;&#xA;    Channel.fromPath( params.bam_files ) \&#xA;        | map { bam -&amp;gt; tuple( bam.baseName, bam ) } \&#xA;        | groupTuple() \&#xA;        | map { sample, files -&amp;gt; tuple( groupKey(sample, files.size()), files ) } \&#xA;        | transpose() \&#xA;        | set { bam_files }&#xA;&#xA;    samtools_sort( bam_files ) \&#xA;        | groupTuple() \&#xA;        | samtools_merge \&#xA;        | view()&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ nextflow run script.nf -ansi-log false --bam_files './path/to/*/*.bam'&#xA;N E X T F L O W  ~  version 22.04.0&#xA;Launching `script.nf` [cheesy_swanson] DSL2 - revision: da1a82a059&#xA;[04/643129] Submitted process &amp;gt; samtools_sort (sample3)&#xA;[1a/93a939] Submitted process &amp;gt; samtools_sort (sample1)&#xA;[93/fde0a0] Submitted process &amp;gt; samtools_sort (sample3)&#xA;[db/208bac] Submitted process &amp;gt; samtools_sort (sample3)&#xA;[8f/f2a231] Submitted process &amp;gt; samtools_sort (sample2)&#xA;[f3/8f0f9f] Submitted process &amp;gt; samtools_sort (sample2)&#xA;[bb/cca23e] Submitted process &amp;gt; samtools_merge (sample1)&#xA;[sample1, /home/steve/testing/work/bb/cca23e016a641b201b7e42f9db74ef/sample1.bam]&#xA;[ff/bd3a2a] Submitted process &amp;gt; samtools_merge (sample2)&#xA;[sample2, /home/steve/testing/work/ff/bd3a2ac83a0691d2b80b04ef97b755/sample2.bam]&#xA;[94/a4f8eb] Submitted process &amp;gt; samtools_merge (sample3)&#xA;[sample3, /home/steve/testing/work/94/a4f8eb483602e9335b28f5d40058c8/sample3.bam]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that the &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#input-of-generic-values&quot; rel=&quot;nofollow noreferrer&quot;&gt;val&lt;/a&gt; specified as input to 'samtools_sort' is an object of class &lt;code&gt;nextflow.extension.GroupKey&lt;/code&gt;. In the example above, the input to 'samtools_merge' is also a groupKey - but it doesn't need to be. If you need a string value for whatever reason, you can just call '.toString()' to get it back:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    ...&#xA;&#xA;    samtools_sort( bam_files ) \&#xA;        | groupTuple() \&#xA;        | map { sample_key, files -&amp;gt; tuple( sample_key.toString(), files ) } \&#xA;        | samtools_merge \&#xA;        | view()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2022-05-25T06:16:47.527" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19115" PostTypeId="1" CreationDate="2022-05-25T07:58:09.853" Score="1" ViewCount="88" Body="&lt;p&gt;I would like to know what would be the best way to generate a full-length consensus sequence?&lt;/p&gt;&#xA;&lt;p&gt;I want to obtain a single representative sequence from a relatively gappy multiple sequence alignment of peptides.&lt;/p&gt;&#xA;&lt;p&gt;See example below.&lt;/p&gt;&#xA;&lt;p&gt;So far I have attempted &lt;code&gt;Consensus.py&lt;/code&gt; from &lt;a href=&quot;https://github.com/jrjhealey/bioinfo-tools&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/jrjhealey/bioinfo-tools&lt;/a&gt; but it just gives me gaps, even with a threshold close to zero.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;HX5_33766:7:1222:1022:72139&#xA;---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------SEEL---SLSLYYRKGRGPLF-------SLSKVTSVDSIGVAYLAFKDKVYFNVT--------------------------------------------&#xA;&amp;gt;HX5_33766:7:1211:6319:29841&#xA;----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------CENEKGCIITSPSKDETMKVQDNSIIINCDGY------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1222:1022:72139.2&#xA;---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------SEEL---SLSLYYRKGRGPLF-------SLSKVTSVDSIGVAYLAFKDKVYFNVT--------------------------------------------&#xA;&amp;gt;HX5_33766:7:1219:12723:20928.1&#xA;----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------T-ISTSYKDIQVNGGELILIHQNPGGFCAY--------------&#xA;&amp;gt;HX5_33766:7:1119:11069:37612&#xA;----------------------------------------------------------------------------------------------------------------------------------------VLLGAAIIQGLGLLLC--F----------------------TYI----------------------CLHFYASQ-V----------------------------------------------------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1219:12723:20928&#xA;----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------T-ISTSYKDIQVNGGELILIHQNPGGFCAY--------------&#xA;&amp;gt;HX5_33766:7:1211:6319:29841.2&#xA;----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------CENEKGCIITSPSKDETMKVQDNSIIINCDGY------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1219:12723:20928.2&#xA;----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------T-ISTSYKDIQVNGGELILIHQNPGGFCAY--------------&#xA;&amp;gt;HX5_33766:7:1107:2564:39757&#xA;-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------VFSLKGYFSEEL---SLSLYYRKGRGPLF-------SLSKVTSVDSIGVAYLAFK----------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1211:6319:29841.1&#xA;----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------CENEKGCIITSPSKDETMKVQDNSIIINCDGY------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1212:30371:5686&#xA;----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------CENEKGCIITSPSKDETMKVQDNSIXLGCN--------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1119:11069:37612.2&#xA;----------------------------------------------------------------------------------------------------------------------------------------VLLGAAIIQGLGLLLC--F----------------------TYI----------------------CLHFYASQ-V----------------------------------------------------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1212:30371:5686.2&#xA;----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------CENEKGCIITSPSKDETMKVQDNSIXLGCN--------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1221:10013:25165&#xA;----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------CENEKGCIITSPSKDETMK-------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1221:10013:25165.2&#xA;----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------CENEKGCIITSPSKDETMK-------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1106:31832:24286&#xA;----------------------------------------------------------------------SVYLRL--IVFP-PSPPLWRRG---QSP------SGLLPFATV-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1117:3691:48881&#xA;-------------------------------------------------------------------------------VFP-PSPPLWRRG---QSP------SGLLPFATV-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1221:9364:63314&#xA;-------------------------------GKFRLVTKKFPRLPAEAAVLKLRECVLLFQLPLISAPGTSVYLR---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1221:10013:25165.1&#xA;----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------CENEKGCIITSPSKDETMK-------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1221:9364:63314.2&#xA;-------------------------------GKFRLVTKKFPRLPAEAAVLKLRECVLLFQLPLISAPGTSVYLR---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1212:30371:5686.1&#xA;----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------CENEKGCIITSPSKDETMKVQDNSIXLGCN--------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1221:9364:63314.1&#xA;-------------------------------GKFRLVTKKFPRLPAEAAVLKLRECVLLFQLPLISAPGTSVYLR---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1119:11069:37612.1&#xA;----------------------------------------------------------------------------------------------------------------------------------------VLLGAAIIQGLGLLLC--F----------------------TYI----------------------CLHFYASQ-V----------------------------------------------------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1107:2564:39757.1&#xA;-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------VFSLKGYFSEEL---SLSLYYRKGRGPLF-------SLSKVTSVDSIGVAYLAFK----------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1106:31832:24286.1&#xA;----------------------------------------------------------------------SVYLRL--IVFP-PSPPLWRRG---QSP------SGLLPFATV-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1117:3691:48881.1&#xA;-------------------------------------------------------------------------------VFP-PSPPLWRRG---QSP------SGLLPFATV-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1222:1022:72139.1&#xA;---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------SEEL---SLSLYYRKGRGPLF-------SLSKVTSVDSIGVAYLAFKDKVYFNVT--------------------------------------------&#xA;&amp;gt;HX5_33766:7:1107:2564:39757.2&#xA;-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------VFSLKGYFSEEL---SLSLYYRKGRGPLF-------SLSKVTSVDSIGVAYLAFK----------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1106:31832:24286.2&#xA;----------------------------------------------------------------------SVYLRL--IVFP-PSPPLWRRG---QSP------SGLLPFATV-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1117:3691:48881.2&#xA;-------------------------------------------------------------------------------VFP-PSPPLWRRG---QSP------SGLLPFATV-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1121:4320:48213&#xA;MTVISQSLKSELECHKNNSLLAQ-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1121:4320:48213.1&#xA;MTVISQSLKSELECHKNNSLLAQ-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1121:4320:48213.2&#xA;MTVISQSLKSELECHKNNSLLAQ-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1214:16701:50287&#xA;----------------------------------------------------------------------------------------------------------------------------------------------R--KQR--EXPKIEQLSAFTRSRSPESAFSNNHSVTYFHTLWGD------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------&#xA;&amp;gt;HX5_33766:7:1220:24545:3085&#xA;------------------------------------------------------------------------------------------------------------------------------------------------------------------RSRSPESAFSNNHSVTYFHTLWGD-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------R&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="180" LastEditorUserId="3848" LastEditDate="2022-05-25T12:22:58.257" LastActivityDate="2022-05-26T02:54:40.567" Title="Generate consensus protein sequence from relatively gappy alignment?" Tags="|fasta|phylogenetics|proteins|multiple-sequence-alignment|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19117" PostTypeId="1" AcceptedAnswerId="19126" CreationDate="2022-05-25T12:03:03.857" Score="2" ViewCount="75" Body="&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; My genome tree doesn't agree with my gene trees and I get the feeling that my genome tree might be wrong, possibly due to long branch attraction, but I don't know how to check/fix it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&#xA;I have a set of genes of interest recovered from genomes and metagenomes. I also have some bins built from the metagenomes of interest.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Previous work&lt;/strong&gt; I've built multiple gene trees, which tend to agree with each other, as well as with other phylogenetic methods (16S rRNA, initial genome trees where available, conserved orientation of the genes of interest). So at this point I though I had a pretty good idea of how my genes / species of interest are linked to the species tree.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Concern&lt;/strong&gt;&#xA;I've assembled the metagenomes and recovered bins. I've refined bins of interest with &lt;a href=&quot;https://anvio.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;anvio&lt;/a&gt;, based on sequence composition / differential coverage, and kept only the bins which appeared quite high quality (&amp;gt;70% completion*, &amp;lt;10 - often &amp;lt;5% - redundancy). Thus contamination in the metagenome assemblies (MAGs) is unlikely. The contamination would have to happen multiple times in the same way to different samples from different environments, at the same coverage, to get the same effect.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;with the exception of a bin of particular interest which I accepted with 62%&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I have de-replicated these bins, added some known genomes, and tried to build a species tree with Orthofinder (with the proteomes). There is a problem occurs with a clade of 3 bins I will call 'X'.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/T9Mo1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/T9Mo1.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Trees&lt;/strong&gt; Based on the gene-of-interest phylogenies, X looks like probably a sister-group to the rest of known genomes from order 'A'. Which is very interesting.&#xA;With the 16S rRNA, it's not quite as clear (there are no 16S rRNA sequences in the bins); however, some 16S rRNA amplified from one of the original samples fit the bill as 'sister-clade to basically everything else in group A'). The first (100 or so) BLAST hits for this 16S sequence are all from group 'A', still.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Tree conflict&lt;/strong&gt; However, in the proteome tree, X as well as Y (Y being a sequenced genome that I suspected was related to X based on 16S data) end up in order 'B' instead of 'A'. GTDBtk also labels X as belonging to clade 'B', not 'A'.&lt;/p&gt;&#xA;&lt;p&gt;Now,I know that genes and genomes don't always have the same evolutionary trajectories. But multiple sets of data lead me to a different conclusion than the proteome tree, so I have to at least question whether the proteome tree is correct; what's likely to be the problem (I suspect &lt;a href=&quot;https://en.wikipedia.org/wiki/Long_branch_attraction&quot; rel=&quot;nofollow noreferrer&quot;&gt;long-branch attraction&lt;/a&gt;), and, if it is long-branch attraction (or anything else) how can I uncover the correct signal?&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your time!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastEditorUserId="3848" LastEditDate="2023-08-09T23:29:46.070" LastActivityDate="2023-08-09T23:29:46.070" Title="How can I improve or otherwise investigate an unreliable genome tree?" Tags="|phylogenetics|phylogeny|assembly|metagenome|bacteria|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19118" PostTypeId="2" ParentId="19115" CreationDate="2022-05-25T12:17:12.680" Score="0" Body="&lt;pre&gt;&lt;code&gt;from Bio import AlignIO&#xA;from pathlib import Path&#xA;from Bio.Align import AlignInfo&#xA;&#xA;path = '/my/path/'&#xA;test1 = 'testset.fa'&#xA;mypath = Path(path,test1)&#xA;&#xA;myalign = AlignIO.read(mypath, &amp;quot;fasta&amp;quot;)&#xA;summary = AlignInfo.SummaryInfo(myalign)&#xA;consensus = summary.dumb_consensus()&#xA;print (consensus)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will give a single consensus and 'X' low prevalence ambiguity. Easy to check whether its what is required because the code is ready to go.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-05-25T12:24:28.217" LastActivityDate="2022-05-25T12:24:28.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19120" PostTypeId="2" ParentId="8759" CreationDate="2022-05-26T02:24:34.043" Score="0" Body="&lt;p&gt;Please use this python program to mask elements -&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dna = input(&amp;quot;Please enter the DNA sequence or chromosome whose repetitive elements need to be masked:&amp;quot;)&#xA;repele = [&amp;quot;aaaa&amp;quot;, &amp;quot;tttt&amp;quot;, &amp;quot;cccc&amp;quot;, &amp;quot;gggg&amp;quot;]&#xA;counter=0&#xA;str1=str(dna)&#xA;for i in repele:&#xA;    str1 = str1.replace(repele[counter], &amp;quot;x&amp;quot;)&#xA;    counter=counter+1&#xA;print(str1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15539" LastActivityDate="2022-05-26T02:24:34.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19121" PostTypeId="2" ParentId="19117" CreationDate="2022-05-26T02:45:37.487" Score="1" Body="&lt;p&gt;It seems like you've got the right approach, in trying out a few different genes and looking for consistency. As you say, different genes can have different selective pressures, which influences how much they change in relation to the whole genome background.&lt;/p&gt;&#xA;&lt;p&gt;Certainly don't put too much trust in a single gene, and especially not 16S: it has a low mutation rate, can jump around and be repeated in the genome, and is an occasional target for horizontal gene transfer.&lt;/p&gt;&#xA;&lt;p&gt;The answer may be that there &lt;em&gt;isn't&lt;/em&gt; any single correct phylogeny. A primary assumption of phylogenetics is that evolutionary progression has a tree-like structure (with roots and tips), but that model is broken by horizontal gene transfer and plasmid sharing. It may be the case that you need to break the genome up into fragments, each with its own inherited history.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-05-26T02:45:37.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19122" PostTypeId="2" ParentId="19115" CreationDate="2022-05-26T02:54:40.567" Score="1" Body="&lt;p&gt;Given the alignment you've presented, there are obvious groups of sequence similarity, but I'm seeing a lot of different peptide sequences in this (e.g. &lt;code&gt;CENEKGCIITSPSKDETMKVQDNSIXLGCN&lt;/code&gt; and &lt;code&gt;GKFRLVTKKFPRLPAEAAVLKLRECVLLFQLPLISAPGTSVYLR&lt;/code&gt; as two obviously different peptides). Have you done a pre-filter for similarity to make sure you're looking at the same proteins for alignment?&lt;/p&gt;&#xA;&lt;p&gt;Multiple alignment will not work properly with peptides that are very different from each other. You need to group proteins based on similarity first (e.g. with &lt;a href=&quot;https://doi.org/10.1038%2Fs41467-018-04964-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Linclust&lt;/a&gt;), then carry out multiple alignments within the groups.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-05-26T02:54:40.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19123" PostTypeId="1" AcceptedAnswerId="19125" CreationDate="2022-05-26T06:36:01.263" Score="0" ViewCount="100" Body="&lt;p&gt;I am analyzing some RNA sequencing data from a collaborator where the effect of some ligands on the transcriptome is being looked at. I am using DESeq2 for my analysis. I am looking to compare each of the ligand treatments vs the DMSO. I have attached the MDS plot with this post. There seems to be a lot of variation between the 3 replicates across all the treatments. Based on this plot, is it worth it to use this data to run a differential expression?&lt;/p&gt;&#xA;&lt;p&gt;It is also at this link below.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ZXhk5.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://i.stack.imgur.com/ZXhk5.jpg&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4307" LastActivityDate="2022-05-26T10:35:17.103" Title="Is this RNA seq data good based on the MDS plot" Tags="|rna-seq|pca|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19124" PostTypeId="2" ParentId="19018" CreationDate="2022-05-26T07:05:55.727" Score="0" Body="&lt;p&gt;Thank you another time @Maximilian Press. Yes, GTDB-Tk it's a robust software but although the merging of archaea and bacteria taxa is still a current research topic, I have succeed with another tool that permits me obtaining an unique tree (including archea and bacteria).&lt;/p&gt;&#xA;&lt;p&gt;If I work with GTDB-Tk I've obtained one tree for archaea and other for bacteria. Then to use it in phyloseq I must work with two independent phyloseq objects which is nice also but my purpose of building a tree from shotgun is only to permits me to calculate diversities based on phylogeny.&lt;/p&gt;&#xA;&lt;p&gt;As alternative that works it was the following:&#xA;to construct a phylogenetic tree from Kaiju I used the 40 marker proteins that is based on the proGenomes database used by kaiju (&lt;a href=&quot;https://progenomes.embl.de/data/proGenomes2.1_markerGenes.tar.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://progenomes.embl.de/data/proGenomes2.1_markerGenes.tar.gz&lt;/a&gt;) and then I've used the ETE-Toolkit(&lt;a href=&quot;http://etetoolkit.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://etetoolkit.org/&lt;/a&gt;) software to construct the tree in Newick format that I can integrate into a phyloseq object.&lt;/p&gt;&#xA;&lt;p&gt;I post here if someone can help my strategy. It has been no easy (unless for a dummie as me) to construct a phylogenetic tree from Kaiju's taxonomic classifier.&lt;/p&gt;&#xA;" OwnerUserId="15406" LastActivityDate="2022-05-26T07:05:55.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19125" PostTypeId="2" ParentId="19123" CreationDate="2022-05-26T10:35:17.103" Score="2" Body="&lt;p&gt;You definitely see separation by treatment in PC1 and PC2, so that is an indication that the treatments are doing something. DMSO and TCDD cluster close, so these are probably similar. Still, you see a variation along PC1 and it seems that there is within-group separation in all groups that could indicate a batch effect. Check whether the samples per group that are more leftish have been processed as one batch, if so one could include that information into the design, see DESeq2 vignette on batch effects. Alternatively, check tools such as RUVseq or SVA to detect unknown batch effects to correct for this within-grup separation.&lt;/p&gt;&#xA;" OwnerDisplayName="user3051" LastActivityDate="2022-05-26T10:35:17.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19126" PostTypeId="2" ParentId="19117" CreationDate="2022-05-26T14:35:48.960" Score="2" Body="&lt;p&gt;The difficulty in answering the question is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The absence of &amp;quot;Y&amp;quot; in the &amp;quot;Gene of interest&amp;quot; tree&lt;/li&gt;&#xA;&lt;li&gt;The sampling bias in the number of taxa mostly between the &amp;quot;genome tree&amp;quot; and the &amp;quot;gene of interest&amp;quot; and &amp;quot;16S&amp;quot; tree&lt;/li&gt;&#xA;&lt;li&gt;The &amp;quot;genome&amp;quot; tree is unrooted so its difficult to compare it with the &amp;quot;16S&amp;quot; tree, thus its possible that if &amp;quot;16S&amp;quot; was unrooted it falls within monophyly &amp;quot;B&amp;quot;&lt;/li&gt;&#xA;&lt;li&gt;The trees don't have any branch or statistical support&lt;/li&gt;&#xA;&lt;li&gt;We don't really know the tree building method (I haven't looked it up)&lt;/li&gt;&#xA;&lt;li&gt;The bacteria might have significant recombination (we don't know the bacteria).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The answer is there is a lot of room for different tree artefacts here, however it could be a biologically real result. I agree &amp;quot;the tree&amp;quot; is simply a tool which uses a series of assumptions and obtaining confidence in a tree requires a lot of analysis. Vast amounts of computational time are dedicated to obtaining a robust tree.&lt;/p&gt;&#xA;&lt;p&gt;The principle step is to compare like for like:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;same number of taxa and&lt;/li&gt;&#xA;&lt;li&gt;same rooting position (or unrooted).&lt;/li&gt;&#xA;&lt;li&gt;second step is to obtain statistical robustness for the observation.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;There is a formal process for identifying conflicting trees and it needs code - I have written stuff that does this and maybe I should release it. Essentially, you would data chunk gene by gene to produce a genome of gene trees and use e.g. ETE3 or similar to plot the tree conflicts. It ain't rocket science. Thus the amount of user time is trivial. The idea is that once the &amp;quot;problem trees&amp;quot; are identified, e.g. rapid evolution.&lt;/p&gt;&#xA;&lt;p&gt;The one observation I would say is that the &amp;quot;Y&amp;quot; branch looks a little bit weird and this could indicate part of the data is has resulted from different evolutionary rate and that is the basis for long-branch attraction.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Scenario 1&lt;/strong&gt; It could be that the 16S and &amp;quot;genome&amp;quot; tree are part of the same tree (monophyly B), they are just rooted differently. Thus the conflict is just a small number of genes and this could lead to ...&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Scenario 2&lt;/strong&gt; Keep in mind bacteria do undergo 'horizontal transmission' and if 'gene of interest' is under strong selective pressure recombination is entirely possible.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-05-26T14:54:22.567" LastActivityDate="2022-05-26T14:54:22.567" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19128" PostTypeId="1" AcceptedAnswerId="19130" CreationDate="2022-05-26T18:36:16.640" Score="2" ViewCount="74" Body="&lt;p&gt;I have a  data frame &lt;code&gt;df&lt;/code&gt;. The below table only represent top 8 rows of the data frame. There are multiple combination of &lt;code&gt;LIBRARY_NAME&lt;/code&gt; and &lt;code&gt;ANCHOR_NAME&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BARCODE ANCHOR_NAME     LIBRARY_NAME    ANCHOR_CONC Synergy HSA     Bliss&#xA;14482   5-Fluorouracil  Cisplatin       2.5         FALSE   FALSE   FALSE&#xA;                                        10          FALSE   FALSE   FALSE&#xA;                        Docetaxel       2.5         FALSE   TRUE    TRUE&#xA;                                        10          FALSE   FALSE   FALSE&#xA;        AZD7762         Cisplatin       0.0625      TRUE    TRUE    TRUE&#xA;                                        0.25        FALSE   TRUE    TRUE&#xA;                        Docetaxel       0.0625      FALSE   FALSE   FALSE&#xA;                                        0.25        FALSE   TRUE    FALSE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to create a new column 'new_synergy 'with pandas which is based on column &lt;code&gt;ANCHOR_CONC&lt;/code&gt; and &lt;code&gt;Synergy&lt;/code&gt;. There are two ANCHOR_CONC for each anchor_name and library_name combination. If for any concentration of ANCHOR_CONC the synergy is &lt;code&gt;TRUE&lt;/code&gt;, for other concentration also synergy becomes &lt;code&gt;TRUE&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Eg: &lt;code&gt;new_synergy&lt;/code&gt;: Condition for TRUE (for AZD7762 and Cisplatin), IF at 0.0625 &lt;code&gt;Synergy&lt;/code&gt; is TRUE, it will be TRUE for 0.25 also.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Expected data_frame&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;new_df&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BARCODE ANCHOR_NAME     LIBRARY_NAME    ANCHOR_CONC Synergy HSA     Bliss  new_synergy&#xA;14482   5-Fluorouracil  Cisplatin       2.5         FALSE   FALSE   FALSE    FALSE&#xA;                                        10          FALSE   FALSE   FALSE    FALSE&#xA;                        Docetaxel       2.5         FALSE   TRUE    TRUE     FALSE  &#xA;                                        10          FALSE   FALSE   FALSE    FALSE&#xA;        AZD7762         Cisplatin       0.0625      TRUE    TRUE    TRUE     TRUE&#xA;                                        0.25        FALSE   TRUE    TRUE     TRUE&#xA;                        Docetaxel       0.0625      FALSE   FALSE   FALSE    FALSE&#xA;                                        0.25        FALSE   TRUE    FALSE    FALSE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried the following code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;import numpy as np&#xA;dd = data[[&amp;quot;BARCODE&amp;quot;, &amp;quot;LIBRARY_NAME&amp;quot;, &amp;quot;ANCHOR_NAME&amp;quot;, &amp;quot;ANCHOR_CONC&amp;quot;, &amp;quot;Synergy&amp;quot;, &amp;quot;Bliss&amp;quot;, &amp;quot;HSA&amp;quot;]]&#xA;df = pd.pivot_table(data=dd,index=[&amp;quot;BARCODE&amp;quot;, &amp;quot;ANCHOR_NAME&amp;quot;, &amp;quot;LIBRARY_NAME&amp;quot;, &amp;quot;ANCHOR_CONC&amp;quot;])&#xA;## setting conditions as pivot table step converted true to 1 and false to 0. &#xA;conditions = [(dd['Synergy'] == 1.0), (dd['Synergy'] == 0.0)]&#xA;values = ['True', 'False']&#xA;dd['new_synergy'] = np.select(conditions, values)&#xA;## However it is giving me the same results in both new_synergy and synergy column. I have no idea how to fit `ANCHOR_CONC` condition in this step.&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3766" LastEditorUserId="3766" LastEditDate="2022-05-26T19:11:25.343" LastActivityDate="2022-05-30T10:12:15.197" Title="To create a new column with Pandas on the basis of two types of column values" Tags="|python|pandas|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19130" PostTypeId="2" ParentId="19128" CreationDate="2022-05-27T06:28:19.190" Score="3" Body="&lt;p&gt;You can use &lt;a href=&quot;https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.groupby.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;groupby()&lt;/a&gt; as following :&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;dd[&amp;quot;new_synergy&amp;quot;] = dd[[&amp;quot;ANCHOR_NAME&amp;quot;, &amp;quot;LIBRARY_NAME&amp;quot;, &amp;quot;Synergy&amp;quot;]].groupby(by=[&amp;quot;ANCHOR_NAME&amp;quot;, &amp;quot;LIBRARY_NAME&amp;quot;]).any()&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Explenation : you want to group your dataframe by both &lt;code&gt;ANCHOR_NAME&lt;/code&gt; and &lt;code&gt;LIBRARY_NAME&lt;/code&gt;, and return True if any of the &lt;code&gt;synergy&lt;/code&gt;value is true, which is what &lt;code&gt;any()&lt;/code&gt; does.&lt;/p&gt;&#xA;&lt;p&gt;Note : you can check for all methods available for &lt;code&gt;GroupBy&lt;/code&gt; objects &lt;a href=&quot;https://pandas.pydata.org/docs/reference/groupby.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15546" LastActivityDate="2022-05-27T06:28:19.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19131" PostTypeId="1" CreationDate="2022-05-27T15:04:30.833" Score="1" ViewCount="52" Body="&lt;p&gt;I am using GATK HaploTypeCaller for SNP calling. I have 300 samples. As far as I understand that if a sample is missing a SNP it should be denoted by ./., however, I am getting a lot of values like 0/0:0,0:0:0:0,0,0.&lt;/p&gt;&#xA;&lt;p&gt;As a result it is considered as homozygous and downstream processing detecting no missing data at all, like when I am using vcftools. This is weird as my sequencing coverage is only 1.0X, so I should have a lot of missing data per SNP. What is going wrong here? Is it okay if I convert all the &lt;code&gt;0/0:0,0:0:0:0,0,0 values to ./.:.,.:.:.:.,.,.&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The commands I used:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gatk HaplotypeCaller &#xA;    -I P1_q10_sorted_rg_dedup.bam &#xA;    -R Zm-B73-REFERENCE-NAM-5.0.fa &#xA;    -ERC GVCF &#xA;    -O P1.g.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;gatk GenomicsDBImport \&#xA;    -V P1.g.vcf &#xA;    --genomicsdb-workspace-path /fs/project/PMIU0157/Mahmood_Hasan/Second/vcf_files/database/mop1/mop1_db_chr1 &#xA;    --intervals chr1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;gatk GenotypeGVCFs &#xA;    --max-alternate-alleles 300 &#xA;    -R Zm-B73-REFERENCE-NAM-5.0.fa  &#xA;    -V gendb://mop1_db_chr1 &#xA;    -O mop1_variants_chr1.vcf &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;gatk SelectVariants &#xA;    -R Zm-B73-REFERENCE-NAM-5.0.fa &#xA;    -V mop1_variants_chr1.vcf &#xA;    --select-type-to-include SNP &#xA;    -O raw_mop1_SNPs_chr1.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;gatk VariantFiltration \&#xA;-R /fs/project/PMIU0157/Mahmood_Hasan/B73_ref_genomeV5/Zm-B73-REFERENCE-NAM-5.0.fa \&#xA;-V raw_wt_SNPs_chr6.vcf \&#xA;-O filtered_wt_SNPs_chr6.vcf \&#xA;--filter-name &amp;quot;QD2&amp;quot; \&#xA;--filter-expression &amp;quot;QD &amp;lt; 2.0&amp;quot; \&#xA;--filter-name &amp;quot;FS60&amp;quot; \&#xA;--filter-expression &amp;quot;FS &amp;gt; 60.0&amp;quot; \&#xA;--filter-name &amp;quot;MQ40&amp;quot; \&#xA;--filter-expression &amp;quot;MQ &amp;lt; 20.0&amp;quot; \&#xA;--filter-name &amp;quot;MQRankSum-12.5&amp;quot; \&#xA;--filter-expression &amp;quot;MQRankSum &amp;lt; -12.5&amp;quot; \&#xA;--filter-name &amp;quot;ReadPosRankSum-8&amp;quot; \&#xA;--filter-expression &amp;quot;ReadPosRankSum &amp;lt; -8.0&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;#The output looks like the following&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1    321     .       T       A       526.30  PASS    AC=10;AF=0.030;AN=336;DP=42;ExcessHet=0.0000;FS=0.000;InbreedingCoeff=0.3471;MLEAC=43;MLEAF=0.128;MQ=35.97;QD=25.36;SOR=0.892   GT:AD:DP:GQ:PGT:PID:PL:PS&#xA;       0/0:1,0:1:3:.:.:0,3,36  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:1,0:1:3:.:.:0,3,26  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:1,0:1:3:.:.:0,3,12  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:1,0:1:3:.:.:0,3,33  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:1,0:1:3:.:.:0,3,12  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:2,0:2:6:.:.:0,6,49  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:1,0:1:3:.:.:0,3,14  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:2,0:2:6:.:.:0,6,39  1|1:0,2:2:6:1|1:321_T_A:90,6,0:321      0/0:2,0:2:6:.:.:0,6,40  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:1,0:1:3:.:.:0,3,16  0/0:0,0:0:0:.:.:0,0,0   0/0:2,0:2:6:.:.:0,6,45  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:1,0:1:3:.:.:0,3,25  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:1,0:1:3:.:.:0,3,24  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:1,0:1:3:.:.:0,3,42  0/0:2,0:2:6:.:.:0,6,41  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:1,0:1:3:.:.:0,3,40  0/0:0,0:0:0:.:.:0,0,0&#xA;   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:1,0:1:3:.:.:0,3,15  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:1,0:1:3:.:.:0,3,11  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   1|1:0,2:2:6:1|1:321_T_A:90,6,0:321      0/0:0,0:0:0:.:.:0,0,0   0/0:2,0:2:6:.:.:0,6,49  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:1,0:1:3:.:.:0,3,22  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   1|1:0,2:2:6:1|1:321_T_A:90,6,0:321      0/0:1,0:1:3:.:.:0,3,20  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   1|1:0,2:2:6:1|1:321_T_A:90,6,0:321      0/0:0,0:0:0:.:.:0,0,0   1|1:0,1:1:3:1|1:321_T_A:45,3,0:321      0/0:0,0:0:0:.:.:0,0,0   0/0:1,0:1:3:.:.:0,3,14  0/0:0,0:0:0:.:.:0,0,0   0/0:1,0:1:3:.:.:0,3,26  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0&#xA;   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:1,0:1:3:.:.:0,3,39  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:2,0:2:6:.:.:0,6,49  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:1,0:1:3:.:.:0,3,16  0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0   0/0:0,0:0:0:.:.:0,0,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15547" LastEditorUserId="3694" LastEditDate="2022-05-29T10:50:37.777" LastActivityDate="2022-05-29T10:50:37.777" Title="Weird data in VCF file" Tags="|vcf|snp|gatk|haplotypes|vcftools|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19132" PostTypeId="1" CreationDate="2022-05-27T16:05:33.847" Score="2" ViewCount="293" Body="&lt;p&gt;Is there a good CLI tool that can diff two multiline fasta files, ignoring order, and comparing sequences with identical names with each other pairwise.&lt;/p&gt;&#xA;&lt;p&gt;Bonus points if lower/upper case is tolerated (as in, it's considered identical) and if also indels can be tolerated, maybe through a flag?&lt;/p&gt;&#xA;&lt;p&gt;I could script this in Python myself, but I was wondering whether there was a custom tool for this purpose already out there. Maybe something more performant than a Biopython script.&lt;/p&gt;&#xA;&lt;p&gt;If there's no custom tool, I'd also be happy with a short script that uses a number of tools, e.g. &lt;code&gt;seqkit&lt;/code&gt; &lt;code&gt;samtools&lt;/code&gt; &lt;code&gt;diff&lt;/code&gt; etc.&lt;/p&gt;&#xA;" OwnerUserId="12272" LastActivityDate="2022-05-30T20:03:47.857" Title="Checking that two multiline FASTAs are identical (allowing for different order and lower/upper case)" Tags="|fasta|samtools|seqkit|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19133" PostTypeId="2" ParentId="19084" CreationDate="2022-05-27T18:57:11.970" Score="1" Body="&lt;p&gt;Create a function and run it for each pdb file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.PDB import PDBParser&#xA;&#xA;def print_residues(name, pdb_file, chain_key, out_file):&#xA;    parser = PDBParser()&#xA;    structure = parser.get_structure(name, pdb_file)&#xA;    model = structure[0]&#xA;    chain = model[chain_key]&#xA;    with open(out_file, 'w') as fp:&#xA;        for i in chain.get_residues():&#xA;            print(i, file=fp)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then run the function for all pdb files.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print_residues('1d8t', '1d8t.pdb', 'C', '1d8t.txt')&#xA;# ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15548" LastActivityDate="2022-05-27T18:57:11.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19136" PostTypeId="1" AcceptedAnswerId="19137" CreationDate="2022-05-29T15:07:18.833" Score="0" ViewCount="63" Body="&lt;p&gt;As we all know computer science, electrical engineering, etc. knowhows can be used in developing software and electronics/electrical products that can be sold to end-users (B2C).&lt;/p&gt;&#xA;&lt;p&gt;Can bioinformatics be used to develop such products other than drug discovery?&lt;/p&gt;&#xA;&lt;p&gt;Or, is it just about scientific/academic research meant for B2B clients?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="1157" LastEditDate="2023-09-07T20:23:09.537" LastActivityDate="2023-09-07T20:23:09.537" Title="Can Bioinformatics be used in developing a marketable product?" Tags="|untagged|" AnswerCount="1" CommentCount="1" ClosedDate="2022-05-29T20:12:54.013" ContentLicense="CC BY-SA 4.0" />
  <row Id="19137" PostTypeId="2" ParentId="19136" CreationDate="2022-05-29T15:40:20.127" Score="1" Body="&lt;p&gt;It is pivotal to vaccine research at multiple levels:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Deep theoretical vaccine design, which takes years to get to market (my stuff's gone into that and still stuck in advanced clinical trials).&lt;/li&gt;&#xA;&lt;li&gt;Looking for variants that might have evolved immunoevasion against current vaccines involving SARS-CoV-2 (why do we need so many boosters?) and influenza.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Bioinformatic patents are tricky.&lt;/p&gt;&#xA;&lt;p&gt;The reality is both points are not systematically automated in 'point and click' user friendliness. Its definitely of commercial value at an industrial/institutional level. However most work is submitted to the public domain and code freely available or held under a commercial firewall.&lt;/p&gt;&#xA;&lt;p&gt;Generally, I think you underestimate how much bioinformatic work takes place in modern clinical practice and when fighting a virus how it just keeps evolving around all the obstacles thrown at it.&lt;/p&gt;&#xA;&lt;p&gt;More generally, a lot of bioinformatics is used to direct wet-lab research because bioinformatics gets to the answers quicker.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-05-29T15:54:13.787" LastActivityDate="2022-05-29T15:54:13.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19138" PostTypeId="1" CreationDate="2022-05-29T16:32:07.967" Score="1" ViewCount="46" Body="&lt;p&gt;I want to explore some potential reactive peptides and seems a common step to conduct some form of docking analysis. I have a question about the theory and a practical question.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Practical question:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;An interesting secondary step seems to be to calculate the binding energy with something like &lt;a href=&quot;https://wenmr.science.uu.nl/prodigy/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PRODIGY&lt;/a&gt;, although I'm struggling to understand the .pdb files to run the webserver.&lt;/p&gt;&#xA;&lt;p&gt;Once I have created a .pdb on either HPEPDOCK or CABS-dock, how do I find the naming convention for the peptides ( Ligand ID ) within the .pdb file? So that I can run in Prodigy-Ligand.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Theory question:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm struggling to understand what to take-away from the docking analysis.&lt;/p&gt;&#xA;&lt;p&gt;I've been using HPEPDOCK ( &amp;amp; CABS-dock, although the former is significantly quicker and seems a good starting place ).&lt;/p&gt;&#xA;&lt;p&gt;I've been trying to read some papers and see how they use docking analysis. I get the feeling, particularly for peptide-protein ( known receptor ), it is simply a visual inspection to see if it makes sense. Adding the abstract docking score to double check it is low enough ( &amp;lt; -100 ).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/krluo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/krluo.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm struggling to understand further use - most papers seem to simply state they did such analysis and ~ it looked okay, or ~ chose the good ones.&lt;/p&gt;&#xA;&lt;p&gt;If anyone can recommend some literature that might help me understand, that would be great!&lt;/p&gt;&#xA;" OwnerUserId="15454" LastActivityDate="2022-05-29T16:32:07.967" Title="Understanding protein-peptide docking analysis" Tags="|docking|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19140" PostTypeId="1" AcceptedAnswerId="19141" CreationDate="2022-05-29T21:11:45.007" Score="1" ViewCount="162" Body="&lt;p&gt;Specifically, the NISTv4.2.1 HG002 dataset &lt;a href=&quot;https://ftp-trace.ncbi.nlm.nih.gov/giab/ftp/release/AshkenazimTrio/HG002_NA24385_son/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (&lt;a href=&quot;https://www.nist.gov/programs-projects/genome-bottle&quot; rel=&quot;nofollow noreferrer&quot;&gt;more info on Genome in a Bottle&lt;/a&gt;). According to their README, the HG002 v4.2.1 VCF should be fully phased.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For HG001, HG002, and HG005, we have phased the v4.2.1 benchmark vcfs by transferring phasing information from a phased assembly-based trio-hifiasm dipcall vcf to the v4.2.1 vcf&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I ask because there appear to be FAR more heterozygous variants on the second haplotype than the first (&lt;code&gt;0|1&lt;/code&gt; &amp;gt;&amp;gt; &lt;code&gt;1|0&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tail -n 500000 HG002_GRCh38_1_22_v4.2.1_benchmark.vcf | \&#xA;        cut -f 10 | cut -f1 -d: | sort | uniq -c&#xA; 302960 0/1&#xA;   1104 1/0&#xA; 190042 1/1&#xA;   2037 1/2&#xA;   3857 2/1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is this simply an artifact of how these truth sets were produced? For example, if the GRCh38 (haploid) FASTA was meant to be haplotype 1, then it makes sense that it's nearly identical to HG002's haplotype 1, and haplotype 2 has more differences. Is that the case, or are only certain variants phased (such as those that pass a minimum confidence threshold, or are in a high-quality BED region)?&lt;/p&gt;&#xA;" OwnerUserId="6332" LastActivityDate="2022-05-30T20:10:55.260" Title="Are the GIAB reference genomes (HG00N) fully phased?" Tags="|vcf|phasing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19141" PostTypeId="2" ParentId="19140" CreationDate="2022-05-30T00:44:20.463" Score="2" Body="&lt;p&gt;As the two alleles in a genotype are separated by &lt;code&gt;/&lt;/code&gt; but not &lt;code&gt;|&lt;/code&gt;, the main v4.2.1 VCF is not phased. The &lt;a href=&quot;https://ftp-trace.ncbi.nlm.nih.gov/giab/ftp/release/AshkenazimTrio/HG002_NA24385_son/NISTv4.2.1/README_v4.2.1.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;README&lt;/a&gt; says the phased VCFs are available in the &lt;a href=&quot;https://ftp-trace.ncbi.nlm.nih.gov/giab/ftp/release/AshkenazimTrio/HG002_NA24385_son/NISTv4.2.1/GRCh38/SupplementaryFiles/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SupplementaryFiles&lt;/a&gt; directory. It contains a few versions of VCFs with v4.2.1 genotype calls but with phasing from additional tools such as whatshap and hifiasm. I don't know which version is the most accurate, though.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2022-05-30T20:10:55.260" LastActivityDate="2022-05-30T20:10:55.260" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19142" PostTypeId="2" ParentId="4736" CreationDate="2022-05-30T04:21:06.030" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @devon-ryan, converted from comments:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;You'll need BAM or bigWig files for each of the tracks. If you can't find any you'll have to make them yourself from public fastq files. The fastq files can be obtained from ENA, SRA, or any of the other standard databases.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-05-30T04:21:06.030" CommentCount="0" CommunityOwnedDate="2022-05-30T04:21:06.030" ContentLicense="CC BY-SA 4.0" />
  <row Id="19143" PostTypeId="2" ParentId="19132" CreationDate="2022-05-30T04:43:08.367" Score="2" Body="&lt;p&gt;There are a lot of ambiguities in the question, but I'll provide one approach by means of a fasta/fastq sorting program (see &lt;a href=&quot;https://gitlab.com/gringer/bioinfscripts/-/blob/master/fastx-sort.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), using process substitution to avoid intermediary files. This will only work for small-ish fasta files, which can fit into computer memory:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;diff -q &amp;lt;(fastx-sort.pl a.fasta) &amp;lt;(fastx-sort.pl b.fasta)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that this will compare the entire file. If there are any sequences in one file but not the other, then the result will be that the files are different.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;-q&lt;/code&gt; option for diff will just output whether or not the two files differ; you didn't specify in your question whether you wanted any more detail. This option can be removed to show differences, or replaced with &lt;code&gt;-u&lt;/code&gt; to add context. Context and/or line changes are not necessarily going to be helpful for multi-line fasta sequences, because the compared files may have a different order from the input, and the context lines may not include the name of the sequence. This could be fixed by changing the files to single-line fasta format (e.g. by piping through &lt;code&gt;fasta_formatter&lt;/code&gt; from the &lt;a href=&quot;http://hannonlab.cshl.edu/fastx_toolkit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FASTX-Toolkit&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;diff -u &amp;lt;(fastx-sort.pl a.fasta | fasta_formatter) &amp;lt;(fastx-sort.pl b.fasta | fasta_formatter)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that this approach will output the entire sequence for any different sequences, together with adjacent context (including the sequence name). There are ways round that as well... but it would be far better if you could make your question more specific so that we don't need to explore the universe when answering.&lt;/p&gt;&#xA;&lt;p&gt;If you want to ignore case, convert everything to upper (or lower) case:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;diff -q &amp;lt;(fastx-sort.pl a.fasta | perl -pe 'tr/a-z/A-Z/') &amp;lt;(fastx-sort.pl b.fasta | perl -pe 'tr/a-z/A-Z/')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-05-30T12:46:30.683" LastActivityDate="2022-05-30T12:46:30.683" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19144" PostTypeId="1" AcceptedAnswerId="19171" CreationDate="2022-05-30T06:27:57.877" Score="1" ViewCount="713" Body="&lt;p&gt;I would like to know how to interpret the output of the formula although i been thorough loads of literature but I'm not confident yet.&lt;/p&gt;&#xA;&lt;p&gt;So this is my full model I'm running&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData=rpkm_ordered, colData=coldata, design= ~ FAB + Sex + Age + TMB + WBC + BM_percentage)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So if I have to interpret the output of this how it would be ?&lt;/p&gt;&#xA;&lt;p&gt;What I have read is this&lt;/p&gt;&#xA;&lt;p&gt;Here in the above formula we are trying or controlling the &lt;code&gt;FAB&lt;/code&gt; variable while we look for variation that is coming from other variables?&lt;/p&gt;&#xA;&lt;p&gt;Now my confusion is when I made this &lt;code&gt;dds&lt;/code&gt; object the &lt;code&gt;FAB&lt;/code&gt; was the variable which I fixed while I look for variation coming from other variable.&lt;/p&gt;&#xA;&lt;p&gt;Next thing is I would like to run a reduced model.&lt;/p&gt;&#xA;&lt;p&gt;I tried this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;test.Age &amp;lt;- as_tibble(results(DESeq(dds, test=&amp;quot;LRT&amp;quot;, reduced= ~Sex + Age )))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now my question is since I'm using the same &lt;code&gt;dds&lt;/code&gt; object here where the &lt;code&gt;FAB&lt;/code&gt; is the variable i fixed and  with new formula which is&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;reduced= ~Sex + Age&lt;/code&gt; this so how would i interpret this model?&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion or help would be really appreciated&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2022-06-02T17:54:20.540" Title="deseq2 full and reduced model interpretation" Tags="|r|deseq2|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19145" PostTypeId="2" ParentId="18771" CreationDate="2022-05-30T07:40:30.520" Score="1" Body="&lt;p&gt;@raysteven,&lt;/p&gt;&#xA;&lt;p&gt;If you follow the documnetation of BPGA at &lt;a href=&quot;https://sourceforge.net/p/bpgatool/wiki/Home/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sourceforge.net/p/bpgatool/wiki/Home/&lt;/a&gt;, you can find the example of &lt;strong&gt;SUPPORTED INPUTS&lt;/strong&gt; for all types of input files including gbk or gbff.&#xA;Unfortunately, Prokka generated files are not supported due to different or missing tags than NCBI.&lt;/p&gt;&#xA;&lt;p&gt;One simple solution you may try is to edit your genBank file to include the tags such as &lt;strong&gt;/protein_id=&amp;quot;Any_unique_protein_id_inside_quotes&amp;quot;&lt;/strong&gt; and everything else shown in the gbk example.&lt;/p&gt;&#xA;&lt;p&gt;I hope this helps.&lt;/p&gt;&#xA;" OwnerUserId="740" LastActivityDate="2022-05-30T07:40:30.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19146" PostTypeId="1" CreationDate="2022-05-30T09:01:27.003" Score="3" ViewCount="1115" Body="&lt;p&gt;FASTA files can be very big and unwieldy, especially if lines are at most 80 characters, one can't speed up browsing them by using &lt;code&gt;less&lt;/code&gt; with &lt;code&gt;-S&lt;/code&gt; to have one sequence every two lines.&lt;/p&gt;&#xA;&lt;p&gt;How can I extract just the strain names (or sequence names, i.e. the string on the line starting with &lt;code&gt;&amp;gt;&lt;/code&gt;) into a list? I'd like to use &lt;code&gt;seqkit&lt;/code&gt; since I'm already familiar with it and know some of the flags and idioms it uses.&lt;/p&gt;&#xA;&lt;p&gt;I googled the question but couldn't find an answer using seqkit - the examples all use &lt;code&gt;awk&lt;/code&gt;, &lt;code&gt;grep&lt;/code&gt;, &lt;code&gt;sed&lt;/code&gt; or similar, which are alright for copy/pasting but not very memorable.&lt;/p&gt;&#xA;" OwnerUserId="12272" LastEditorUserId="298" LastEditDate="2022-05-31T14:47:23.640" LastActivityDate="2022-05-31T19:04:43.280" Title="How to get strain names/ids contained in a multi FASTA file using seqkit?" Tags="|fasta|identifiers|seqkit|multi-fasta|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19147" PostTypeId="2" ParentId="19146" CreationDate="2022-05-30T09:04:31.710" Score="6" Body="&lt;p&gt;The nice thing about &lt;code&gt;seqkit&lt;/code&gt; is that the answer is really simple and memorable:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;seqkit seq -n sequences.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will output a text file containing one sequence name per line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MPX/PT0001/2022&#xA;MPX/PT0002/2022&#xA;MPX/PT0003/2022&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12272" LastActivityDate="2022-05-30T09:04:31.710" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19148" PostTypeId="5" CreationDate="2022-05-30T09:38:31.547" Score="0" Body="&lt;p&gt;Multi-FASTA Files are usually of the following form:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;ID1&#xA;ACGTAGTDNNNAAC&#xA;&amp;gt;ID2&#xA;AAGTAGTDNNNAAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12272" LastEditorUserId="12272" LastEditDate="2022-06-16T19:18:58.500" LastActivityDate="2022-06-16T19:18:58.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19149" PostTypeId="4" CreationDate="2022-05-30T09:38:31.547" Score="0" Body="Concerning fasta files with multiple sequences in one file" OwnerUserId="12272" LastEditorUserId="12272" LastEditDate="2022-06-16T09:08:28.410" LastActivityDate="2022-06-16T09:08:28.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19150" PostTypeId="2" ParentId="19128" CreationDate="2022-05-30T10:12:15.197" Score="3" Body="&lt;p&gt;To elaborate on &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/19130/292&quot;&gt;this answer&lt;/a&gt;, here is a full working example where I assume that &lt;code&gt;&amp;quot;BARCODE&amp;quot;&lt;/code&gt;, &lt;code&gt;&amp;quot;ANCHOR_NAME&amp;quot;&lt;/code&gt; and &lt;code&gt;&amp;quot;LIBRARY_NAME&amp;quot;&lt;/code&gt; are used as &lt;code&gt;MultiIndex&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;p&gt;Creating the test data:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;from io import StringIO&#xA;import pandas as pd&#xA;data_str = &amp;quot;&amp;quot;&amp;quot;BARCODE\tANCHOR_NAME\tLIBRARY_NAME\tANCHOR_CONC\tSynergy\tHSA\tBliss&#xA;14482\t5-Fluorouracil\tCisplatin\t2.5\tFALSE\tFALSE\tFALSE&#xA;14482\t5-Fluorouracil\tCisplatin\t10\tFALSE\tFALSE\tFALSE&#xA;14482\t5-Fluorouracil\tDocetaxel\t2.5\tFALSE\tTRUE\tTRUE&#xA;14482\t5-Fluorouracil\tDocetaxel\t10\tFALSE\tFALSE\tFALSE&#xA;14482\tAZD7762\tCisplatin\t0.0625\tTRUE\tTRUE\tTRUE&#xA;14482\tAZD7762\tCisplatin\t0.25\tFALSE\tTRUE\tTRUE&#xA;14482\tAZD7762\tDocetaxel\t0.0625\tFALSE\tFALSE\tFALSE&#xA;14482\tAZD7762\tDocetaxel\t0.25\tFALSE\tTRUE\tFALSE&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;# StringIO is to simulate an import from a file&#xA;data = pd.read_table(StringIO(data_str), sep=&amp;quot;\t&amp;quot;, index_col=[0, 1, 2])&#xA;# reset_index is here just for displaying purposes&#xA;# (MultiIndex does not seem well supported in (pandas?) markdown export)&#xA;print(data.reset_index().to_markdown(index=False))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;BARCODE&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;ANCHOR_NAME&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;LIBRARY_NAME&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;ANCHOR_CONC&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Synergy&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;HSA&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Bliss&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;5-Fluorouracil&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Cisplatin&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;2.5&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;5-Fluorouracil&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Cisplatin&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;10&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;5-Fluorouracil&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Docetaxel&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;2.5&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;5-Fluorouracil&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Docetaxel&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;10&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;AZD7762&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Cisplatin&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0.0625&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;AZD7762&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Cisplatin&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0.25&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;AZD7762&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Docetaxel&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0.0625&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;AZD7762&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Docetaxel&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0.25&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;Creating a &lt;code&gt;DataFrame&lt;/code&gt; with the new column:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;new_synergy = data.reset_index().groupby(&#xA;    [&amp;quot;BARCODE&amp;quot;, &amp;quot;ANCHOR_NAME&amp;quot;, &amp;quot;LIBRARY_NAME&amp;quot;])[[&amp;quot;Synergy&amp;quot;]].any()&#xA;# Rename the resulting &amp;quot;Synergy&amp;quot; column&#xA;new_synergy.columns = [&amp;quot;new_synergy&amp;quot;]&#xA;print(new_synergy.reset_index().to_markdown(index=False))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;BARCODE&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;ANCHOR_NAME&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;LIBRARY_NAME&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;new_synergy&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;5-Fluorouracil&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Cisplatin&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;5-Fluorouracil&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Docetaxel&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;AZD7762&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Cisplatin&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;AZD7762&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Docetaxel&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;Merging the two &lt;code&gt;DataFrame&lt;/code&gt;s based on their indices:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;new_data = data.merge(new_synergy, left_index=True, right_index=True)&#xA;print(new_data.reset_index().to_markdown(index=False))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;BARCODE&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;ANCHOR_NAME&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;LIBRARY_NAME&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;ANCHOR_CONC&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Synergy&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;HSA&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Bliss&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;new_synergy&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;5-Fluorouracil&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Cisplatin&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;2.5&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;5-Fluorouracil&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Cisplatin&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;10&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;5-Fluorouracil&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Docetaxel&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;2.5&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;5-Fluorouracil&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Docetaxel&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;10&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;AZD7762&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Cisplatin&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0.0625&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;AZD7762&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Cisplatin&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0.25&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;AZD7762&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Docetaxel&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0.0625&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;14482&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;AZD7762&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Docetaxel&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0.25&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;True&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;False&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;" OwnerUserId="292" LastActivityDate="2022-05-30T10:12:15.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19151" PostTypeId="1" CreationDate="2022-05-30T13:51:59.257" Score="2" ViewCount="54" Body="&lt;p&gt;The publicly available reference panel for GWAS analysis is the &lt;a href=&quot;http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/&quot; rel=&quot;nofollow noreferrer&quot;&gt;1000 genomes reference panel&lt;/a&gt;. The reference panel consists of several populations. However, I am searching for a population-specific publicly available reference panel. I want to perform some analysis like Linkage disequilibrium, inheritance, etc.&#xA;Now there are some imputation reference panels for the specific populations such as UK10k (European), CAAPA (African), etc.&lt;/p&gt;&#xA;&lt;p&gt;However, to access this panel we need to go through a lot of agreements and processes. Can anyone give me a list of publicly available population-specific reference panel, preferably, South Asian, European, Eastern Asian, African and Hispanic?&lt;/p&gt;&#xA;" OwnerUserId="10296" LastEditorUserId="10296" LastEditDate="2022-05-30T17:48:56.280" LastActivityDate="2022-05-30T17:48:56.280" Title="Publicly available population specific reference panels" Tags="|gwas|imputation|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19153" PostTypeId="2" ParentId="19146" CreationDate="2022-05-30T17:49:16.827" Score="4" Body="&lt;p&gt;You can trivially do this with standard *nix tools. Here are a few options:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sed -n 's/^&amp;gt;//p' file.fa &#xA;awk 'sub(/^&amp;gt;/,&amp;quot;&amp;quot;)' file.fa&#xA;grep '^&amp;gt;' file.fa | tr -d '&amp;gt;'&#xA;perl -ne 'print if s/^&amp;gt;//' file.fa &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The advantage of these is that they will work on any POSIX-compliant system and most other *nix flavors, even if they're not POSIX such as Linux, out of the box without needing to install any external tools.&lt;/p&gt;&#xA;&lt;p&gt;Another thing to take into account is how fast a command is, especially when dealing with large files. I ran a few tests using the solutions presented here on the human hg38 genome assembly which can be downloaded &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/genomes/all/GCA/000/001/405/GCA_000001405.15_GRCh38/GO_TO_CURRENT_VERSION/GCA_000001405.29_GRCh38.p14_genomic.fna.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and &lt;code&gt;seqkit&lt;/code&gt; version &lt;code&gt;2.2.0&lt;/code&gt; which was downloaded from &lt;a href=&quot;https://github.com/shenwei356/seqkit/releases/download/v2.2.0/seqkit_linux_amd64.tar.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The file was stored locally on my laptop's SSD drive, and the laptop itself is:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;$ inxi &#xA;CPU: quad core Intel Core i7-6820HQ (-MT MCP-)&#xA;speed/min/max: 2896/800/3600 MHz Kernel: 5.17.5-arch1-1 x86_64&#xA;Up: 25d 10h 27m Mem: 15035.9/31958.1 MiB (47.0%)&#xA;Storage: 476.94 GiB (103.1% used) Procs: 452 Shell: Bash inxi: 3.3.15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To get more accurate results, I used a little function I have which will run the command you give it a specified number of times and then report the average &lt;em&gt;real&lt;/em&gt; time across all runs. Here's the code if you want to try it:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;timethis(){&#xA;    max=$1;&#xA;    shift;&#xA;&#xA;    for com&#xA;    do&#xA;        printf '\nCOMMAND: %s\n' &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$com&quot;&#xA;    c=0;&#xA;    while [[ $&lt;/span&gt;c -lt &lt;span class=&quot;math-container&quot;&gt;$max ]]&#xA;    do&#xA;      let c++;&#xA;      ( time -p eval &quot;$&lt;/span&gt;com&amp;quot; ) 2&amp;gt;&amp;amp;1 | grep -oP 'real.*?\K[\d\.]+'&#xA;        done | &#xA;        awk -vm=&lt;span class=&quot;math-container&quot;&gt;$max '{k+=$&lt;/span&gt;1}END{print (k/m)}';&#xA;    done&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the results. These are the average &amp;quot;real&amp;quot; (i.e. wall clock) times taken across 10 runs:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ timethis 10 \&#xA;  &quot;rg '^&amp;gt;' GCA_000001405.29_GRCh38.p14_genomic.fna | tr -d '&amp;gt;' &amp;gt;/dev/null&quot; \&#xA;  &quot;grep '^&amp;gt;' GCA_000001405.29_GRCh38.p14_genomic.fna | tr -d '&amp;gt;' &amp;gt;/dev/null&quot; \&#xA;  &quot;sed -n 's/^&amp;gt;//p' GCA_000001405.29_GRCh38.p14_genomic.fna &amp;gt;/dev/null&quot; \&#xA;  &quot;perl -ne 'print if s/^&amp;gt;//' GCA_000001405.29_GRCh38.p14_genomic.fna &amp;gt;/dev/null&quot; \&#xA;  &quot;./seqkit seq -n GCA_000001405.29_GRCh38.p14_genomic.fna &amp;gt;/dev/null&quot; \&#xA;  &quot;awk 'sub(/^&amp;gt;/,\&quot;\&quot;)' GCA_000001405.29_GRCh38.p14_genomic.fna &amp;gt;/dev/null&quot; \&#xA;  &quot;perl -nle 'push @res, $&lt;/span&gt;_  while s/^&amp;gt;//; END {print  join \&amp;quot;\n\&amp;quot;,@res}' GCA_000001405.29_GRCh38.p14_genomic.fna&amp;quot;&#xA;&#xA;COMMAND: rg '^&amp;gt;' GCA_000001405.29_GRCh38.p14_genomic.fna | tr -d '&amp;gt;' &amp;gt;/dev/null&#xA;0.474&#xA;&#xA;COMMAND: grep '^&amp;gt;' GCA_000001405.29_GRCh38.p14_genomic.fna | tr -d '&amp;gt;' &amp;gt;/dev/null &#xA;0.583&#xA;&#xA;COMMAND: sed -n 's/^&amp;gt;//p' GCA_000001405.29_GRCh38.p14_genomic.fna &amp;gt;/dev/null &#xA;5.62&#xA;&#xA;COMMAND: perl -ne 'print if s/^&amp;gt;//' GCA_000001405.29_GRCh38.p14_genomic.fna  &amp;gt;/dev/null &#xA;6.408&#xA;&#xA;COMMAND: ./seqkit seq -n GCA_000001405.29_GRCh38.p14_genomic.fna &amp;gt;/dev/null &#xA;4.326&#xA;&#xA;COMMAND: awk 'sub(/^&amp;gt;/,&amp;quot;&amp;quot;)' GCA_000001405.29_GRCh38.p14_genomic.fna &amp;gt;/dev/null &#xA;13.518&#xA;&#xA;COMMAND: perl -nle 'push @res, 3  while s/^&amp;gt;//; END {print  join &amp;quot;\n&amp;quot;,@res}' GCA_000001405.29_GRCh38.p14_genomic.fna &#xA;8.455&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see above, the &lt;code&gt;grep&lt;/code&gt; or &lt;code&gt;rg&lt;/code&gt; and &lt;code&gt;tr&lt;/code&gt; approaches only took around half a second on average, so that is clearly much faster than any of the other choices. However, &lt;code&gt;seqkit&lt;/code&gt; is pretty good since it only took around 4 seconds, faster than any of the other ad hoc solutions.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2022-05-31T19:04:43.280" LastActivityDate="2022-05-31T19:04:43.280" CommentCount="15" ContentLicense="CC BY-SA 4.0" />
  <row Id="19154" PostTypeId="2" ParentId="19146" CreationDate="2022-05-30T19:35:52.140" Score="1" Body="&lt;p&gt;Or a bit more verbose ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -nle 'push @res, $_  while s/^&amp;gt;//; END {print  join &amp;quot;\n&amp;quot;,@res}' myfile.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="298" LastEditDate="2022-05-30T19:38:56.090" LastActivityDate="2022-05-30T19:38:56.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19155" PostTypeId="2" ParentId="19132" CreationDate="2022-05-30T20:03:47.857" Score="3" Body="&lt;p&gt;Try &lt;a href=&quot;https://github.com/samtools/samtools/blob/develop/misc/md5fa.c&quot; rel=&quot;nofollow noreferrer&quot;&gt;md5fa&lt;/a&gt; from samtools. It computes two MD5s with and without ordering, respectively. Alternatively, there is &lt;a href=&quot;https://bioinf.shenwei.me/seqkit/usage/#sum&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqkit sum&lt;/a&gt;, which is perhaps more powerful.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2022-05-30T20:03:47.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19157" PostTypeId="1" CreationDate="2022-05-31T09:31:21.367" Score="1" ViewCount="20" Body="&lt;p&gt;I'm investigating two different groups of short nucleotide sequences. The behaviour of a certain DNA-binding protein is likely influenced by common differences between these two groups leading high or low activity. Some positions are hypothesized to be more important than others. All sequences have the same length and are very similar.&lt;/p&gt;&#xA;&lt;p&gt;I'm trying to find out how well changes at certain sequence positions explain the between-group difference and which positions are of the highest importance for this relationship. By looking at the sequence logo for both groups I can easily identify crucial positions. However, I'm at a loss how to actually score the effect each position has and how well it correlates to membership in either group. I thought about calculating Cramers V at each position but that doesn't seem to be very useful.&lt;/p&gt;&#xA;&lt;p&gt;Can someone here point me in the right direction?&lt;/p&gt;&#xA;" OwnerUserId="15558" LastEditorUserId="15558" LastEditDate="2022-05-31T09:33:17.007" LastActivityDate="2022-05-31T09:33:17.007" Title="Calculating the effect of differences between two groups of nucleotide sequences at every position" Tags="|statistics|phylogenetics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19159" PostTypeId="1" AcceptedAnswerId="20467" CreationDate="2022-06-01T11:56:28.550" Score="2" ViewCount="231" Body="&lt;p&gt;I'm looking for a tool to call somatic single nucleotide variations on reads from an Oxford Nanopore Technologies (ONT) run.&#xA;I have both tumor and normal samples and have already aligned them to a reference genome.&lt;/p&gt;&#xA;&lt;p&gt;Popular tools like &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/5358911630107-Mutect2&quot; rel=&quot;nofollow noreferrer&quot;&gt;mutect2&lt;/a&gt; have been developed for short reads, and according to &lt;a href=&quot;https://link.springer.com/chapter/10.1007/978-3-030-91836-1_3&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; paper it is not recommended to use tools specifically designed for short reads with long reads.&lt;/p&gt;&#xA;&lt;p&gt;I know of tools that work with ONT reads like &lt;a href=&quot;https://github.com/HKU-BAL/Clair3&quot; rel=&quot;nofollow noreferrer&quot;&gt;clair3&lt;/a&gt; which is used in the epi2melabs workflow for SNPs &lt;a href=&quot;https://github.com/epi2me-labs/wf-human-snp&quot; rel=&quot;nofollow noreferrer&quot;&gt;wf-human-snp&lt;/a&gt;,  but clair3 takes only one sample at a time, so seems for germline calling only.&lt;/p&gt;&#xA;&lt;p&gt;Another approach would be to simply call the SNVs in both tumor and normal samples independently with &lt;a href=&quot;https://github.com/HKU-BAL/Clair3&quot; rel=&quot;nofollow noreferrer&quot;&gt;clair3&lt;/a&gt;, and then subtract normal SNVs from tumor SNVs, but according to what I have read in a few papers and online resources like &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035890491-Somatic-calling-is-NOT-simply-a-difference-between-two-callsets&quot; rel=&quot;nofollow noreferrer&quot;&gt;this from gatk&lt;/a&gt;, this is a &amp;quot;naive approach&amp;quot;, and a somatic caller should use both normal and tumor samples together.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you know of any tool that takes both tumor and normal long read samples to detect somatic variants (similar to &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/5358911630107-Mutect2&quot; rel=&quot;nofollow noreferrer&quot;&gt;mutect2&lt;/a&gt; but for long reads)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Have you used tools for short reads like &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/5358911630107-Mutect2&quot; rel=&quot;nofollow noreferrer&quot;&gt;mutect2&lt;/a&gt; with long reads?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Have you attempted the &amp;quot;naive approach&amp;quot; of simply subtracting SNVs?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Thanks,&#xA;Walter.&lt;/p&gt;&#xA;" OwnerUserId="15562" LastEditorUserId="298" LastEditDate="2022-06-01T12:01:05.610" LastActivityDate="2023-01-30T03:08:19.223" Title="somatic SNV tool for ONT samples" Tags="|variant-calling|nanopore|long-reads|somatic|snv|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19160" PostTypeId="1" AcceptedAnswerId="19252" CreationDate="2022-06-01T14:33:09.000" Score="3" ViewCount="89" Body="&lt;p&gt;I´m working with a dataset of 566 genomes to analyze a pangenome. So I was working with PANWEB to create this pan core genome curve, however, there is too much sequence to work with this webserver. Well, specifically I´m looking for this kind of plot:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/N0idM.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/N0idM.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, I just want to ask for some ideas of programs to do that with a big dataset and with a command line.&lt;/p&gt;&#xA;&lt;p&gt;Here is the reference: &lt;a href=&quot;https://www.researchgate.net/figure/Pan-genome-and-core-genome-Graph-representing-the-pan-genome-blue-and-core-genome_fig1_317390138&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.researchgate.net/figure/Pan-genome-and-core-genome-Graph-representing-the-pan-genome-blue-and-core-genome_fig1_317390138&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10557" LastActivityDate="2022-06-16T11:15:43.470" Title="How to make pan-core genome curve through command line on linux?" Tags="|genome|metagenome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19163" PostTypeId="1" CreationDate="2022-06-01T21:17:10.440" Score="3" ViewCount="213" Body="&lt;p&gt;I'm trying to run &lt;a href=&quot;https://dalexander.github.io/admixture/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ADMIXTURE&lt;/a&gt; on some files I've generated using the &lt;a href=&quot;https://catchenlab.life.illinois.edu/stacks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stacks&lt;/a&gt; pipeline. ADMIXTURE accepts .bed and .ped files, but when I try to pass the .ped file generated by Stacks to ADMIXTURE (v1.3.0), I get a &amp;quot;PLINK Input file error&amp;quot;. I've pasted the head of my (tab separated) .ped file below.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;head populations.plink.ped -n 1&lt;/p&gt;&#xA;&lt;p&gt;garden  QUAC_G_SH_Q1485 0       0       0       0       0       0       T       T       C       C       G       G       T       T       C       C       C       C       G       G       T&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Alternatively, I could convert my .ped file to .bed using plink, and try to pass this to ADMIXTURE--but Stacks didn't provide me with a map file to do this, so I don't think that will work.&lt;/p&gt;&#xA;&lt;p&gt;If anyone knows what needs to change with my .ped file (remove sample name? population designation?), I'd appreciate any tips. This is my first time running most of these programs.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="14041" LastActivityDate="2024-02-28T23:39:20.580" Title="Proper formatting of .ped files, in order to run ADMIXTURE" Tags="|plink|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19164" PostTypeId="1" AcceptedAnswerId="19251" CreationDate="2022-06-01T21:34:44.923" Score="2" ViewCount="176" Body="&lt;p&gt;I need to generate a list of unique file names without their extensions or using the shell script command &amp;quot;find&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Say I have a list such as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;file2022_col.R1.twx.gz&#xA;file2022_col.R2.twx.gz&#xA;file2022_col.bwa.bam&#xA;file2332_col.R1.twx.gz&#xA;file2332_col.bwa.bam&#xA;file3442_col.bwa.bam.bai&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want a list of unique file names such as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;file2022_col&#xA;file2332_col&#xA;file3442_col&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm trying to use &lt;code&gt;find -type f -name&lt;/code&gt; but I'm not sure how to construct the option to restrict the find command.&lt;/p&gt;&#xA;" OwnerUserId="2621" LastEditorUserId="650" LastEditDate="2022-06-02T18:16:32.403" LastActivityDate="2022-06-16T11:08:40.833" Title="How do I find unique file names without their extensions in shell script?" Tags="|linux|shell|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19165" PostTypeId="2" ParentId="19164" CreationDate="2022-06-01T22:32:37.123" Score="1" Body="&lt;pre&gt;&lt;code&gt;ls | perl -p -n -e  's/(.*)(\..*){1,3}&lt;span class=&quot;math-container&quot;&gt;$/$&lt;/span&gt;1/' | uniq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;find . -name &amp;quot;file*&amp;quot; | perl -p -n -e  's/(.*)(\..*){1,3}&lt;span class=&quot;math-container&quot;&gt;$/$&lt;/span&gt;1/' | uniq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;find&lt;/code&gt; &lt;em&gt;per se&lt;/em&gt; is a really powerful command, but its a while since I've used it full functionality.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-06-01T22:37:50.433" LastActivityDate="2022-06-01T22:37:50.433" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19166" PostTypeId="1" CreationDate="2022-06-01T23:14:40.330" Score="1" ViewCount="68" Body="&lt;p&gt;I wonder which form of a single-cell RNA sequencing abbreviation is correct:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;scRNA-seq&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;scRNA-Seq&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In the same single-cell sequencing Wikipedia article, both forms appear.&lt;br /&gt;&#xA;There is also inconsistency among papers I've seen - some authors prefer the first form, others the second one.&lt;/p&gt;&#xA;&lt;p&gt;If there's any reason behind the different form, I'd like to hear about it.&lt;/p&gt;&#xA;" OwnerUserId="9651" LastEditorUserId="3694" LastEditDate="2022-06-02T10:20:24.403" LastActivityDate="2022-06-02T13:07:23.393" Title="Single-cell RNA sequencing proper abbreviation" Tags="|scrnaseq|terminology|nomenclature|" AnswerCount="0" CommentCount="0" ClosedDate="2022-06-03T11:24:24.713" ContentLicense="CC BY-SA 4.0" />
  <row Id="19167" PostTypeId="1" AcceptedAnswerId="19173" CreationDate="2022-06-02T10:20:00.733" Score="1" ViewCount="333" Body="&lt;p&gt;I have run a GWAS on a binary trait - &amp;quot;Are you following a carnivore diet?&amp;quot; - so the cases are individuals who follow this diet, and the controls are individuals who do not follow this diet. I have a genome-wide significant SNP, with a negative beta in reference to Allele 2. Does this imply that individuals with Allele 2 are less likely to follow this diet?&lt;/p&gt;&#xA;" OwnerUserId="10083" LastActivityDate="2022-06-07T23:13:03.903" Title="How to interpret a negative beta within a binary GWAS?" Tags="|gwas|genetics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19168" PostTypeId="1" CreationDate="2022-06-02T11:57:39.937" Score="1" ViewCount="617" Body="&lt;p&gt;I am trying to install Biovia DS visualiser2021 on ubuntu 22.04. I used the procedure from the following website&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://askubuntu.com/questions/1175772/cant-install-discovery-studio-visualizer&quot;&gt;https://askubuntu.com/questions/1175772/cant-install-discovery-studio-visualizer&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;When I run the installer &lt;code&gt;./install_DSClient.sh&lt;/code&gt; I got the following error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Calling LicensePack Installer &#xA;&#xA;Verifying archive integrity... All good.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;Uncompressing License Pack and extracting files, please wait..&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;./install_lp.sh: 73: Syntax error: redirection unexpected&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Copying 1 files to ./share/license/data/tempdata.dat Please wait.&lt;/code&gt;..&#xA;&lt;code&gt;Copying 1 files to /home/user/BIOVIA/DiscoveryStudio2021/Logs/dsclientinstall.log Please wait..&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Completed installing  DiscoveryStudio2021 at /home/user/BIOVIA/DiscoveryStudio2021&#xA;&#xA;Check log file [ /home/user/BIOVIA/DiscoveryStudio2021/Logs/dsclientinstall.log ] for more information. Return code: 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How to solve this error?&lt;/p&gt;&#xA;" OwnerUserId="15571" LastActivityDate="2022-06-02T11:57:39.937" Title="Installation of Biovia DS visualiser on ubuntu" Tags="|protein-structure|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19170" PostTypeId="1" CreationDate="2022-06-02T14:21:21.993" Score="1" ViewCount="174" Body="&lt;p&gt;I am learning to use snakemake. I am following the tutorial (&lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/tutorial/tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://snakemake.readthedocs.io/en/stable/tutorial/tutorial.html&lt;/a&gt;). In the section called &amp;quot;Basics&amp;quot;, steps 1  to 4 worked well, processing one sample at a time, from mapping reads until indexing alignments.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;snakemake --cores 2 sorted_reads/{A,B,C}.bam.bai&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, when I add the rule for joint variant calling (step 5):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;snakemake --cores 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;WorkflowError:&#xA;Target rules may not contain wildcards. Please specify concrete files or a rule without wildcards at the command line, or have a rule without wildcards at the very top of your workflow (e.g. the typical &amp;quot;rule all&amp;quot; which just collects all results you want to generate in the end).&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It looks like I need to do this &amp;quot;rule all&amp;quot; step, any idea how to set this up?&lt;/p&gt;&#xA;&lt;p&gt;This is how the snakemake file looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SAMPLES = [&amp;quot;A&amp;quot;, &amp;quot;B&amp;quot;, &amp;quot;C&amp;quot;]&#xA;&#xA;rule bwa_map:&#xA;    input:&#xA;        &amp;quot;data/genome.fa&amp;quot;,&#xA;        &amp;quot;data/samples/{sample}.fastq&amp;quot;&#xA;    output:&#xA;        &amp;quot;mapped_reads/{sample}.bam&amp;quot;&#xA;    shell:&#xA;        &amp;quot;bwa mem {input} | samtools view -Sb - &amp;gt; {output}&amp;quot;&#xA;&#xA;rule samtools_sort:&#xA;    input:&#xA;        &amp;quot;mapped_reads/{sample}.bam&amp;quot;&#xA;    output:&#xA;        &amp;quot;sorted_reads/{sample}.bam&amp;quot;&#xA;    shell:&#xA;        &amp;quot;samtools sort -T sorted_reads/{wildcards.sample} &amp;quot;&#xA;        &amp;quot;-O bam {input} &amp;gt; {output}&amp;quot;&#xA;&#xA;rule samtools_index:&#xA;    input:&#xA;        &amp;quot;sorted_reads/{sample}.bam&amp;quot;&#xA;    output:&#xA;        &amp;quot;sorted_reads/{sample}.bam.bai&amp;quot;&#xA;    shell:&#xA;        &amp;quot;samtools index {input}&amp;quot;&#xA;&#xA;rule bcftools_call:&#xA;    input:&#xA;        fa=&amp;quot;data/genome.fa&amp;quot;,&#xA;        bam=expand(&amp;quot;sorted_reads/{sample}.bam&amp;quot;, sample=SAMPLES),&#xA;        bai=expand(&amp;quot;sorted_reads/{sample}.bam.bai&amp;quot;, sample=SAMPLES)&#xA;    output:&#xA;        &amp;quot;calls/all.vcf&amp;quot;&#xA;    shell:&#xA;        &amp;quot;bcftools mpileup -f {input.fa} {input.bam} | &amp;quot;&#xA;        &amp;quot;bcftools call -mv - &amp;gt; {output}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10271" LastActivityDate="2022-06-02T18:14:48.407" Title="snakemake tutorial stuck in basics-step-5 (variant calling)" Tags="|variant-calling|snakemake|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19171" PostTypeId="2" ParentId="19144" CreationDate="2022-06-02T17:54:20.540" Score="2" Body="&lt;p&gt;By fitting the full model &lt;code&gt;~FAB + Sex + Age + TMB + WBC + BM_percentage&lt;/code&gt; you can estimate the effect of FAB (or any of the other variables) on RNA expression after adjusting for differences caused by Sex, Age, TMB, WBC, and BM percent. See &lt;a href=&quot;https://stats.stackexchange.com/questions/17336/how-exactly-does-one-control-for-other-variables&quot;&gt;this post&lt;/a&gt; for exactly what it means to adjust for other variables&lt;/p&gt;&#xA;&lt;p&gt;When you fit a reduced model (in your case &lt;code&gt;~Sex + Age&lt;/code&gt;) and then conduct a likelihood ratio test (LRT), you are comparing the fit of the full model (&lt;code&gt;~FAB + Sex + Age + TMB + WBC + BM_percentage&lt;/code&gt;) to the fit of the reduced model (&lt;code&gt;~Sex + Age&lt;/code&gt;). If the fit is substantially better, the likelihood of the full model will be larger than the reduced model and the p-value will be significant. This means that adding the extra variables that are in the full model (FAB, TMB, WBC, BM_percentage) improved the fit of the model to the data because together these variables have some type of relationship with RNA expression even after adjusting for the variables in the reduced model.&lt;/p&gt;&#xA;&lt;p&gt;The challenge with using a LRT is that it doesn't tell you which of the variables added in the full model are responsible for the improved fit - it just indicates that the full model is better.  This type of test is useful when you are trying to make broad statements about several variables having a relationship with RNA expression. From the DESeq2 vignette:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The LRT is therefore useful for testing multiple terms at once, for example testing 3 or more levels of a factor at once, or all interactions between two variables. The LRT for count data is conceptually similar to an analysis of variance (ANOVA) calculation in linear regression, except that in the case of the Negative Binomial GLM, we use an analysis of deviance (ANODEV), where the deviance captures the difference in likelihood between a full and a reduced model.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Definitely read the DESeq2 vignette for more details. In your case if the LR test is significant, it means that FAB, TMB, WBC, and BM percent combined are associated with RNA expression.&lt;/p&gt;&#xA;&lt;p&gt;Most people are interested in specific comparisons when performing differential expression analyses with DESeq2. These comparisons are usually like do males have higher gene expression than females (comparing sex differences), do patients with higher TMB have greater expression than patients with lower TMB (comparing effect of TMB on RNA expression). If you want to do a specific comparison, you will likely want to use the Wald test, which calculates p-values for individual coefficients (male vs female coefficient, TMB coefficient etc). The Wald test determines whether a coefficient is equal to 0 or some other null hypothesis (see &lt;code&gt;lfcThreshold&lt;/code&gt; parameter).&lt;/p&gt;&#xA;&lt;p&gt;The Wald statistic and accompanying p-values are reported by default when using &lt;code&gt;results()&lt;/code&gt; and not specifying &lt;code&gt;test=&amp;quot;LRT&amp;quot;&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;There are some additional differences between Wald tests vs LRT, including accuracy differences when operating on small sample sizes and computational efficiency issues when operating with large sample sizes. &lt;a href=&quot;https://stats.stackexchange.com/questions/193643/likelihood-ratio-vs-wald-test&quot;&gt;Discussion on Wald vs LRT&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt; Use LRT to test if several variables or factor levels are associated with RNA expression. Use the Wald test to determine if specific comparisons are significant.&lt;/p&gt;&#xA;" OwnerUserId="6674" LastActivityDate="2022-06-02T17:54:20.540" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="19172" PostTypeId="2" ParentId="19170" CreationDate="2022-06-02T18:14:48.407" Score="2" Body="&lt;p&gt;Read &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#targets-and-aggregation&quot; rel=&quot;nofollow noreferrer&quot;&gt;the snakemake documentation&lt;/a&gt;. &lt;code&gt;all&lt;/code&gt; is a pseudo-rule that accepts a set of one or more final output files as input and does nothing with them.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2022-06-02T18:14:48.407" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19173" PostTypeId="2" ParentId="19167" CreationDate="2022-06-03T03:17:45.347" Score="1" Body="&lt;p&gt;Negative beta means that the trait under test is negatively correlated with the allele. What this actually means in relation to your specific question will depend on how the trait is coded and how the allele is coded.&lt;/p&gt;&#xA;&lt;p&gt;It might, for example, be the case that &amp;quot;yes&amp;quot; is coded as 0 and &amp;quot;no&amp;quot; as 1, in which case a &amp;quot;no&amp;quot; answer would be associated with a high allele count for allele 2. You will need to check the algorithm to find that out, or (better) do your own manual / independent checking of alleles to discover the direction of association.&lt;/p&gt;&#xA;&lt;p&gt;Take care when interpreting the results of these association tests, especially when deviating from statements beyond what the results actually say. You should also pay attention to is meaningless, whereas a beta value of -3.43 means there is a strong correlation between the trait and allele.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-06-07T23:13:03.903" LastActivityDate="2022-06-07T23:13:03.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19174" PostTypeId="1" CreationDate="2022-06-03T11:02:08.173" Score="2" ViewCount="94" Body="&lt;p&gt;I downloaded the following reference proteomes:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ftp.ebi.ac.uk/pub/databases/reference_proteomes/QfO/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ebi.ac.uk/pub/databases/reference_proteomes/QfO/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;'Drosophila':'UP000000803_7227.fasta',&#xA;'Arabidopsis':'UP000006548_3702.fasta',&#xA;'C.elegans':'UP000001940_6239.fasta',&#xA;'Zebrafish':'UP000000437_7955.fasta',&#xA;'human':'UP000005640_9606.fasta',&#xA;'mouse':'UP000000589_10090.fasta',&#xA;'yeast':'UP000002311_559292.fasta'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, I would like to have the corresponding coding sequences (CDS). My temporary solution is to download CDS files from ENSEMBL FTP server, translate those and match with the ref proteome. However, this only covers 50-90% of protein sequences.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="6463" LastActivityDate="2022-06-07T11:40:31.950" Title="How to get coding sequences (CDS) of UniProt reference proteomes?" Tags="|data-download|sequence|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19175" PostTypeId="1" AcceptedAnswerId="19178" CreationDate="2022-06-03T13:27:24.527" Score="2" ViewCount="223" Body="&lt;p&gt;Where can I find databases for breast cancer mutations (SNPs &amp;amp; indels) phenotype, with chromosome, position, ref, alt, gene, clinical significance.&lt;/p&gt;&#xA;&lt;p&gt;So far, I have only found the Clinvar database.&lt;/p&gt;&#xA;&lt;p&gt;I need to extract data for breast cancer, in order to train a machine learning model that will be able to predict the pathogenicity of variants (benign, pathogenic etc).&lt;/p&gt;&#xA;&lt;p&gt;So I did the extraction of data from clinvar vcf file (chromosome, ref, alt, clinical significance), but the clinvar data for breast cancer is very small.&lt;/p&gt;&#xA;" OwnerUserId="15536" LastEditorUserId="3694" LastEditDate="2022-06-05T09:10:00.740" LastActivityDate="2022-06-05T09:10:00.740" Title="Where to find database for breast cancer phenotype?" Tags="|database|cancer|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19176" PostTypeId="2" ParentId="19164" CreationDate="2022-06-03T17:43:55.383" Score="1" Body="&lt;p&gt;Use &lt;a href=&quot;https://www.gnu.org/software/findutils/manual/html_mono/find.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;find&lt;/code&gt;&lt;/a&gt;, remove all extensions with &lt;code&gt;perl&lt;/code&gt;, and make unique with &lt;a href=&quot;https://www.gnu.org/software/coreutils/manual/html_node/sort-invocation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;sort&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;find . -type f -printf '%P\n' | perl -pe 's{[.].*}{}' | sort -u&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The Perl one-liner uses these command line flags:&lt;br /&gt;&#xA;&lt;code&gt;-e&lt;/code&gt; : Tells Perl to look for code in-line, instead of in a file.&lt;br /&gt;&#xA;&lt;code&gt;-p&lt;/code&gt; : Loop over the input one line at a time, assigning it to &lt;code&gt;$_&lt;/code&gt; by default. Add &lt;code&gt;print $_&lt;/code&gt; after each loop iteration.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;s{[.].*}{}&lt;/code&gt; : replace &lt;code&gt;[.]&lt;/code&gt; (a literal dot), followed by &lt;code&gt;.*&lt;/code&gt; (any character repeated 0 or more times) with en empty string.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;SEE ALSO:&lt;/strong&gt;&lt;br /&gt;&#xA;&lt;a href=&quot;https://perldoc.perl.org/perlrun.html#Command-Switches&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;perldoc perlrun&lt;/code&gt;: how to execute the Perl interpreter: command line switches&lt;/a&gt;&lt;br /&gt;&#xA;&lt;a href=&quot;https://perldoc.perl.org/perlre.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;perldoc perlre&lt;/code&gt;: Perl regular expressions (regexes)&lt;/a&gt;&lt;br /&gt;&#xA;&lt;a href=&quot;https://perldoc.perl.org/perlre.html#Regular-Expressions&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;perldoc perlre&lt;/code&gt;: Perl regular expressions (regexes): Quantifiers; Character Classes and other Special Escapes; Assertions; Capture groups&lt;/a&gt;&lt;br /&gt;&#xA;&lt;a href=&quot;https://perldoc.perl.org/perlrequick&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;perldoc perlrequick&lt;/code&gt;: Perl regular expressions quick start&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6251" LastActivityDate="2022-06-03T17:43:55.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19177" PostTypeId="1" AcceptedAnswerId="19189" CreationDate="2022-06-03T21:10:03.283" Score="2" ViewCount="628" Body="&lt;p&gt;I'm currently working with a seurat object and I'd like to calculate the expression values per gene for all cells within a particular cluster.&lt;/p&gt;&#xA;&lt;p&gt;I've gone ahead and subsetted the cluster of interest.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tumor &amp;lt;- subset(x = data, idents = c(&amp;quot;Tumor&amp;quot;), invert = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How would I calculate the expression values per gene per cell?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FindAllMarkers&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Finds markers (differentially expressed genes) for each of the identity classes in a dataset per ident.&lt;/p&gt;&#xA;" OwnerUserId="6827" LastActivityDate="2022-06-06T22:25:35.370" Title="how to calculate the expression values per gene for all cells" Tags="|r|scrnaseq|seurat|gene-expression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19178" PostTypeId="2" ParentId="19175" CreationDate="2022-06-04T06:10:29.663" Score="1" Body="&lt;p&gt;Do you mean this (a product of the BRCA Challenge project):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://brcaexchange.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://brcaexchange.org/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/S08iv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/S08iv.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15519" LastEditorUserId="15519" LastEditDate="2022-06-04T06:16:14.947" LastActivityDate="2022-06-04T06:16:14.947" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="19179" PostTypeId="1" CreationDate="2022-06-04T06:20:08.997" Score="1" ViewCount="48" Body="&lt;p&gt;I'm working on a problem where the elements of multiple arrays need to be aligned with each other. While doing some research on algorithms that could possibly be applicable to this task, I stumbled on sequence alignment. I believe this is very similar to what I'm trying to achieve.&lt;/p&gt;&#xA;&lt;p&gt;In short, I need to have equal elements aligned with each other across multiple arrays. However, the difference with my problem compared to sequence alignment seems to be that the latter doesn't tolerate gaps. On the other hand, gaps would be perfectly acceptable for me. So, for example, given input:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;[A, ,B,A, ]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;[ ,C, ,D,A]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;[B, ,C,B,A]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;A desired output would be:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;[A,B, , , ,A]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;[ , ,C, ,D,A]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;[ ,B,C,B, ,A]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;But there cannot be any errors. That is, every column must contain elements of the same value.&lt;/p&gt;&#xA;&lt;p&gt;Would it be possible to achieve the desired result with sequence alignment algorithms?&lt;/p&gt;&#xA;" OwnerUserId="15577" LastActivityDate="2022-06-04T06:20:08.997" Title="Multiple sequence alignment with gaps" Tags="|sequence-alignment|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19180" PostTypeId="1" CreationDate="2022-06-04T19:51:06.353" Score="2" ViewCount="34" Body="&lt;p&gt;I have an RNAseq mouse retina gene expression matrix(24659,49300) to cluster effectively. Here is a sample(7,4):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gene            cell1   cell2   cell3&#xA;KITL            0       0       1&#xA;TMTC3           3       0       0&#xA;CEP290          1       3       0&#xA;4930430F08RIK   2       1       2&#xA;1700017N19RIK   0       0       0&#xA;MGAT4C          0       0       4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there any chance to apply a weighting schema or embedding to increase ARI, NMI or DBI? Dataset is Macosko, 2015: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE63472&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE63472&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want to try a new approach to cluster rnase data. Question is effectiveness to use weighting schema or embedding. Is it worth to try? May be someone tried it before.&lt;/p&gt;&#xA;" OwnerUserId="1315" LastEditorUserId="3694" LastEditDate="2022-06-10T09:38:04.170" LastActivityDate="2022-06-10T09:38:04.170" Title="Is it possible to cluster a sc-RNAseq expression matrix with weighting or embedding?" Tags="|scrnaseq|clustering|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19181" PostTypeId="1" CreationDate="2022-06-06T07:24:10.187" Score="1" ViewCount="102" Body="&lt;p&gt;I am using dataset &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE85241&quot; rel=&quot;nofollow noreferrer&quot;&gt;GSE85241&lt;/a&gt;, but I can't find the read counts of the dataset. It only provides with &lt;code&gt;RPKM&lt;/code&gt; values.&lt;/p&gt;&#xA;&lt;p&gt;How can I find the read counts of a dataset?&lt;/p&gt;&#xA;" OwnerUserId="15425" LastActivityDate="2023-07-01T13:08:51.087" Title="How can I get data of single cell RNA sequence with raw count?" Tags="|rna-seq|single-cell|phylogenetics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19183" PostTypeId="2" ParentId="19181" CreationDate="2022-06-06T09:19:24.290" Score="1" Body="&lt;p&gt;If the authors did not provide the raw counts, and I don't think it is compulsory, unfortunately you will need to download the raw data, align and count.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2022-06-06T09:19:24.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19184" PostTypeId="1" CreationDate="2022-06-06T10:24:05.280" Score="1" ViewCount="133" Body="&lt;p&gt;I have a very large bam file and I want to filter it to keep only a handfull of positions as defined in a bed file. Can I do this with sambamba or do i need another tool?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="2186" LastActivityDate="2022-11-03T18:01:15.177" Title="Filter bam with sambamba" Tags="|sam|sambamba|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19185" PostTypeId="2" ParentId="18938" CreationDate="2022-06-06T10:52:51.283" Score="1" Body="&lt;p&gt;I contacted the QCTOOL team directly and received this response:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The design of BGEN means that when you subset samples the data has to&#xA;be recompressed - this is essentially what makes this slow.  (By&#xA;contrast you can subset SNPs very quickly without recompression using&#xA;bgenix &lt;a href=&quot;https://code.enkre.net/bgen.&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://code.enkre.net/bgen.&lt;/a&gt;)  It is therefore definitely worth&#xA;considering not subsetting samples but using a sample&#xA;inclusion/exclusion list instead, if your analysis software supports&#xA;that.&lt;/p&gt;&#xA;&lt;p&gt;If you have to subset and want to use QCTOOL - some things to try are:&#xA;I typically use the options -bgen-compression zstd -bgen-bits 8 now,&#xA;c.f. &lt;a href=&quot;https://doi.org/10.1101/308296&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1101/308296&lt;/a&gt; this is faster but first check&#xA;your downstream software supports zstd compression. Use a map/reduce&#xA;type pipeline (i.e. chunk data for re-encoding) - this can be&#xA;implemented using bgenix and cat-bgen.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I assume you’re working with the latest version?&lt;/p&gt;&#xA;&lt;p&gt;Have you tried first stripping the SNPs, then stripping the samples?&lt;/p&gt;&#xA;&lt;p&gt;Ever since working with the imputed data from the UKB I was never&#xA;successful in chopping pieces of bgen files using qctool in a timely&#xA;manner. If my memory serves me well, qctool manages to strip SNPs&#xA;fairly quickly, but is super slow to remove samples. Try PLINK 1.9 or&#xA;2.0.&lt;/p&gt;&#xA;&lt;p&gt;As for “The computational facility I'm using should not limit the speed of an operation.”,&#xA;depending on how disk data moves in/out of slave nodes and how busy&#xA;the cluster is, I did saw processes to become I/O starved on large&#xA;clusters.&lt;/p&gt;&#xA;&lt;p&gt;Also, consider that most tools that perform association testing can&#xA;utilize SNPs  and samples lists. BOLT, SAIGE, regenie, SNPtest… so&#xA;there might not be a need to pre-filter. Put “NA” as phenotypes you’d&#xA;like to be “removed” from the testing.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the end I did not pre-filter SNPs or samples - I set samples to NA within the phenotype file, and used a SNP inclusion list in the second stage of SAIGE with the flag &lt;code&gt;idstoIncludeFile&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="10083" LastActivityDate="2022-06-06T10:52:51.283" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19186" PostTypeId="2" ParentId="19184" CreationDate="2022-06-06T15:00:13.360" Score="0" Body="&lt;p&gt;You need &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;bedtools&lt;/code&gt;&lt;/a&gt;, specifically &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/intersect.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools intersect&lt;/a&gt; command, not &lt;code&gt;sambamba&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bedtools intersect -abam A.bam -b B.bed &amp;gt; A_in_N.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6251" LastActivityDate="2022-06-06T15:00:13.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19187" PostTypeId="2" ParentId="19184" CreationDate="2022-06-06T15:25:31.003" Score="1" Body="&lt;p&gt;I think it is possible:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sambamba view --regions regions.bed --with-header --format=bam reads.bam  &amp;gt; filtered.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2186" LastEditorUserId="2186" LastEditDate="2022-06-06T16:08:42.727" LastActivityDate="2022-06-06T16:08:42.727" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19188" PostTypeId="1" AcceptedAnswerId="19196" CreationDate="2022-06-06T18:40:29.280" Score="3" ViewCount="507" Body="&lt;p&gt;I have a VCF with small variant calls against HG38 and I would like to determine which of those calls are present in the gnomAD database.&lt;/p&gt;&#xA;&lt;p&gt;Is there an existing tool that can do this? Should I be looking at variant annotation tools? or is this something that I would look at BCFtools for?&lt;/p&gt;&#xA;" OwnerUserId="15264" LastActivityDate="2023-01-07T00:51:58.393" Title="Compare my VCF to gnomAD variants" Tags="|vcf|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19189" PostTypeId="2" ParentId="19177" CreationDate="2022-06-06T22:25:35.370" Score="2" Body="&lt;p&gt;No Seurat function is needed. In the seurat object &amp;quot;cells&amp;quot;, you can access the raw counts or normalized counts using &lt;code&gt;cells@assays$RNA@counts&lt;/code&gt; (raw counts) or &lt;code&gt;cells@assays$SCT@data&lt;/code&gt; (SCT normalized data)&lt;/p&gt;&#xA;" OwnerUserId="6827" LastActivityDate="2022-06-06T22:25:35.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19190" PostTypeId="2" ParentId="19188" CreationDate="2022-06-06T23:02:37.107" Score="1" Body="&lt;p&gt;If you have sufficient available disk space, you could download the files from the &lt;a href=&quot;https://gnomad.broadinstitute.org/downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;gnomad downloads page&lt;/a&gt; and use a &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/intersect.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools intersect&lt;/a&gt; command to find the overlapping variants.&lt;/p&gt;&#xA;&lt;p&gt;However, the recommended platforms for gnomAD are to use &lt;a href=&quot;https://github.com/broadinstitute/gnomad_methods&quot; rel=&quot;nofollow noreferrer&quot;&gt;hail utilities&lt;/a&gt; developed by them and others, due to the constraints of working with ~1TB of data.&lt;/p&gt;&#xA;&lt;p&gt;I am not an expert but potentially someone else knows more about using the site hail data.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-06-06T23:02:37.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19193" PostTypeId="2" ParentId="17733" CreationDate="2022-06-07T10:41:01.247" Score="1" Body="&lt;p&gt;Ok so this might not provide a solution, but here's a couple things someone might find useful;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;First, Biopython does not automatically contain DSSP, which you have to download as mentioned in the comments below (from here swift.cmbi.umcn.nl/gv/dssp)&lt;/li&gt;&#xA;&lt;li&gt;Secondly, DSSP within Biopython, for whatever reason, accepts the path to the file, not the file itself. I see you have that correctly (&lt;code&gt;dssp = DSSP(model, &amp;quot;16pk.pdb&amp;quot;)&lt;/code&gt;), but just pointing it ou&lt;/li&gt;&#xA;&lt;li&gt;Third, DSSP originally had &lt;code&gt;dssp&lt;/code&gt; executable, but the newer version has &lt;code&gt;mkdssp&lt;/code&gt; executable, so you might need to do &lt;code&gt;dssp = DSSP(model, &amp;quot;16pk.pdb&amp;quot;, dssp=&amp;quot;mkdssp&amp;quot;)&lt;/code&gt;, so it knows what to associate it with. This was the problem I had, because without specifying the &lt;code&gt;dssp&lt;/code&gt;, it is not able to launch DSSP.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="15589" LastActivityDate="2022-06-07T10:41:01.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19194" PostTypeId="2" ParentId="19174" CreationDate="2022-06-07T11:40:31.950" Score="1" Body="&lt;p&gt;I went through diverse README files and found the answer:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Coding DNA Sequence FASTA files (*_DNA.fasta)&#xA;=============================================&#xA;&#xA;These files contain the coding DNA sequences (CDS) for the protein sequences&#xA;where it was possible.&#xA;The format is as in the following example (UP000005640_9606_DNA.fasta):&#xA;&#xA;&amp;gt;sp|A0A183|ENSP00000411070&#xA;ATGTCACAGCAGAAGCAGCAATCTTGGAAGCCTCCAAATGTTCCCAAATGCTCCCCTCCC&#xA;CAAAGATCAAACCCCTGCCTAGCTCCCTACTCGACTCCTTGTGGTGCTCCCCATTCAGAA&#xA;GGTTGTCATTCCAGTTCCCAAAGGCCTGAGGTTCAGAAGCCTAGGAGGGCTCGTCAAAAG&#xA;CTGCGCTGCCTAAGTAGGGGCACAACCTACCACTGCAAAGAGGAAGAGTGTGAAGGCGAC&#xA;TGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, EBI itself provides the corresponding coding sequences, which they name &lt;strong&gt;_DNA.fasta&lt;/strong&gt;. I misinterpreted the file-name as genomic (unspliced) DNA.&lt;/p&gt;&#xA;" OwnerUserId="6463" LastActivityDate="2022-06-07T11:40:31.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19195" PostTypeId="1" CreationDate="2022-06-07T14:56:13.143" Score="1" ViewCount="117" Body="&lt;p&gt;I need to use partitioned DNA input (concatenated multiple genes) and fixed topology (species tree from ASTRAL) to get branch lengths in MrBayes. I have input NEXUS like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#NEXUS&#xA;BEGIN TAXA;&#xA;  DIMENSIONS NTAX = 318;&#xA;  TAXLABELS&#xA;    Achillea_nobilis_SAMN11585351&#xA;    Anthemis_arvensis_SAMN11585352&#xA;    ...&#xA;    Haplopappus_sp_SAMN11585375&#xA;  ;&#xA;END&#xA;BEGIN TREES;&#xA;  TRANSLATE&#xA;    1       Achillea_nobilis_SAMN11585351,&#xA;    2       Anthemis_arvensis_SAMN11585352,&#xA;    ...&#xA;    318     Haplopappus_sp_SAMN11585375&#xA;  ;&#xA;  TREE treeshrink_exons_good_nobrl = [&amp;amp;R] (1,((2,3...);&#xA;END;&#xA;BEGIN DATA;&#xA;dimensions ntax=318 nchar=169024;&#xA;format datatype=dna missing=? gap=-;&#xA;MATRIX&#xA;Achillea_nobilis_SAMN11585351               TTATGCAA...&#xA;Anthemis_arvensis_SAMN11585352              TTATGCAA...&#xA;...&#xA;Haplopappus_sp_SAMN11585375                 TTATGCAA...&#xA;;&#xA;END;&#xA;BEGIN MRBAYES;&#xA;  charset At1g09700 = 1-213;&#xA;  charset At1g10740 = 214-456;&#xA;  ...&#xA;  charset At5g66280 = 168530-168757;&#xA;  charset At5g66680 = 168758-169024;&#xA;  outgroup Achillea_nobilis_SAMN11585351;&#xA;  partition by_gene=467:At1g09700,At1g10740,...,At5g66280,At5g66680;&#xA;  set partition=by_gene;&#xA;  prset topologypr=fixed(treeshrink_exons_good_nobrl);&#xA;  prset applyto=(all) ratepr=variable;&#xA;  lset applyto=(all) nst=mixed rates=gamma ngammacat=10;&#xA;  unlink tratio=(all) revmat=(all) omega=(all) statefreq=(all) shape=(all) pinvar=(all) correlation=(all) ratemultiplier=(all);&#xA;  mcmcp ngen=10000000 nruns=4 nchains=4 samplefreq=5000;&#xA;  ss;&#xA;  sump relburnin=yes burninfrac=0.25 nruns=4;&#xA;  sumt relburnin=yes burninfrac=0.25 nruns=4;&#xA;END;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm not 100% sure about the &lt;code&gt;MRBAYES&lt;/code&gt; block, but generally I think it's correct. When I launch it (MrBayes 3.2.7a) I get an error which I'm unable to resolve:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ mb sp_treeshrink_exons_good.mb.nex&#xA;...&#xA;   Executing file &amp;quot;sp_treeshrink_exons_good.mb.nex&amp;quot;&#xA;   UNIX line termination&#xA;   Longest line length = 169069&#xA;   Parsing file&#xA;   Expecting NEXUS formatted file&#xA;   Reading taxa block&#xA;      Allocated taxon set&#xA;      Defining new set of 318 taxa&#xA;   Exiting taxa block&#xA;   Reading trees block&#xA;      Successfully read tree 'treeshrink_exons_good_nobrl' # NOTE&#xA;   Exiting trees block&#xA;   Deleting user trees&#xA;   Deleting previously defined taxa&#xA;   Reading data block&#xA;      Allocated taxon set&#xA;      Allocated matrix&#xA;      Defining new matrix with 318 taxa and 169024 characters&#xA;      Data is Dna&#xA;      Missing data coded as ?&#xA;      Gaps coded as -&#xA;      Taxon   1 -&amp;gt; Achillea_nobilis_SAMN11585351&#xA;      Taxon   2 -&amp;gt; Anthemis_arvensis_SAMN11585352&#xA;...&#xA;      Taxon 318 -&amp;gt; Haplopappus_sp_SAMN11585375&#xA;      Successfully read matrix&#xA;      Setting default partition (does not divide up characters)&#xA;      Setting model defaults&#xA;      Seed (for generating default start values) = 1654610114&#xA;      Setting output file names to &amp;quot;sp_treeshrink_exons_good.mb.nex.run&amp;lt;i&amp;gt;.&amp;lt;p|t&amp;gt;&amp;quot;&#xA;   Exiting data block&#xA;   Reading mrbayes block&#xA;      Defining charset called 'At1g09700'&#xA;      Defining charset called 'At1g10740'&#xA;...&#xA;      Defining charset called 'At5g66280'&#xA;      Defining charset called 'At5g66680'&#xA;      Defining partition called 'by_gene'&#xA;      Setting by_gene as the partition, dividing characters into 467 parts.&#xA;      Setting model defaults&#xA;      Seed (for generating default start values) = 1901076251&#xA;      Setting Topologypr to Fixed for partition 1&#xA;      Setting Topologypr to Fixed for partition 2&#xA;...&#xA;      Setting Topologypr to Fixed for partition 466&#xA;      Setting Topologypr to Fixed for partition 467&#xA;      Could not find tree 'treeshrink_exons_good_nobrl' # NOTE&#xA;      Could not set fixed topology from user tree 'treeshrink_exons_good_nobrl' # NOTE&#xA;      Error when setting parameter &amp;quot;Topologypr&amp;quot; (2)&#xA;      The error occurred when reading char. 26-52 on line 1445&#xA;         in the file 'sp_treeshrink_exons_good.mb.nex'&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So it correctly reads tree &lt;code&gt;treeshrink_exons_good_nobrl&lt;/code&gt; and after while it complains it couldn't find tree &lt;code&gt;treeshrink_exons_good_nobrl&lt;/code&gt;...? I can't find where this weird error comes from.&lt;/p&gt;&#xA;" OwnerUserId="15590" LastEditorUserId="17320" LastEditDate="2023-10-27T21:33:48.220" LastActivityDate="2024-03-25T22:04:30.520" Title="Problem with setting fixed topology for MrBayes" Tags="|phylogenetics|phylogeny|mrbayes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19196" PostTypeId="2" ParentId="19188" CreationDate="2022-06-07T15:53:27.080" Score="3" Body="&lt;p&gt;You can use &lt;a href=&quot;https://github.com/brentp/vcfanno&quot; rel=&quot;nofollow noreferrer&quot;&gt;vcfanno&lt;/a&gt; to annotate your vcf with the gnomad vcf file. There's an example config file for annotating with gnomad (along with other databases such as dbSNP) &lt;a href=&quot;https://github.com/brentp/vcfanno/blob/master/example/gnomad.conf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9999" LastActivityDate="2022-06-07T15:53:27.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19197" PostTypeId="1" CreationDate="2022-06-08T01:24:55.103" Score="2" ViewCount="78" Body="&lt;p&gt;have a good R and statistical analysis background (also with machine learning). in addition, I'm a fresh biotechnology grad. I would like to try to replicate some Rna-seq analysis with R papers (with their provided data). Any SHORT (beginner-friendly) papers to recommend?&lt;/p&gt;&#xA;" OwnerUserId="15487" LastActivityDate="2022-06-20T13:43:35.567" Title="beginner RNA-seq Replicate papers" Tags="|r|rna-seq|bioconductor|rna|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19198" PostTypeId="1" AcceptedAnswerId="19209" CreationDate="2022-06-08T14:23:00.290" Score="1" ViewCount="60" Body="&lt;p&gt;I have a couple of prokaryotic genes that I'm trying to build phylogenetic trees from.&lt;/p&gt;&#xA;&lt;p&gt;I inputted them into IQTree, which, as part of the workflow, runs through multiple substitution models until it finds the one that seems to fit the data best.&lt;/p&gt;&#xA;&lt;p&gt;My issue: quite a few of the genes ended up with eukaryotic_specific substitution models (such as Q.plant or Q.insect). Is this likely to bias my data? Should I re-run the analysis and remove the substitution models that are meant for different groups?&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your time!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastEditorUserId="3848" LastEditDate="2023-08-07T23:57:20.173" LastActivityDate="2023-08-07T23:57:20.173" Title="Should I expect / use a prokaryote-specific substitution model when building a phylogenetic tree?" Tags="|phylogenetics|phylogeny|bacteria|substitution-model|matrix|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19199" PostTypeId="1" CreationDate="2022-06-08T15:12:28.303" Score="3" ViewCount="238" Body="&lt;p&gt;I would like to download all the ITS1 and ITS2 genes from NCBI in a fasta file. And, I'd like to download even the related taxonomy of each sequence.&lt;/p&gt;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;&lt;p&gt;Marco&lt;/p&gt;&#xA;" OwnerUserId="15596" LastActivityDate="2022-06-15T01:42:23.110" Title="How can I download from NCBI all the ITS genes and the related taxonomy?" Tags="|phylogenetics|taxonomy|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19200" PostTypeId="1" AcceptedAnswerId="21129" CreationDate="2022-06-08T16:17:00.903" Score="3" ViewCount="235" Body="&lt;p&gt;I'm trying to find unique chains or proteins within their PDB files. Many proteins have multiple chains, but very often they are identical (say, the PDB file consists of 4 homodimers, for example, so there is one unique chain represented 8 times total). I am processing certain structural features of the chains, so processing two identical chains is redundant for my pruposes.&lt;/p&gt;&#xA;&lt;p&gt;I am using Biopython and the way I'm currently identifying unique chains is looking through each PDB file, checking the sequences of the chains and comparing them to one another using alignments (specifically &lt;code&gt;pairwise2.align.globalxx(seq1, seq2)&lt;/code&gt;. I consider a chain non-unique if it shares &amp;gt;90% sequence similarity to another chain. (Most chains have 100+ residues.)&lt;/p&gt;&#xA;&lt;p&gt;My question is, is this threshold appropriate? Should I go lower, maybe 80%, or the opposite, higher? For now, it seems to work fine, but I'll be processing a lot of PDB files and want to be on the safer side. If I accidentally process identical chains, that's not much of an issue, but it adds unnecessary time to the processing. Is there maybe a &lt;strong&gt;source/article&lt;/strong&gt; to help with what sequence similarity I should be using? I've found multiple articles, but they usually talk about homology (i.e., common ancestry of proteins), but I'm only doing this within the protein itself and its subunits (chains), so this is of no use for me.&lt;/p&gt;&#xA;" OwnerUserId="15589" LastActivityDate="2023-07-15T15:47:23.660" Title="Identification of unique chains in proteins by sequence similarity" Tags="|sequence-alignment|proteins|python|sequence|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19201" PostTypeId="2" ParentId="19199" CreationDate="2022-06-08T16:26:53.653" Score="4" Body="&lt;p&gt;I recommend to use &lt;strong&gt;Biopython&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio import Entrez&#xA;&#xA;Entrez.email = &amp;quot;your@somemail.com&amp;quot;&#xA;Entrez.api_key = &amp;quot;YOUR_API_KEY&amp;quot;&#xA;&#xA;search_result = Entrez.read(Entrez.esearch(db=&amp;quot;nucleotide&amp;quot;, term=&amp;quot;ITS1[Gene Name]&amp;quot;))&#xA;with Entrez.efetch(&#xA;    db=&amp;quot;nucleotide&amp;quot;, id=search_result[&amp;quot;IdList&amp;quot;], rettype=&amp;quot;fasta&amp;quot;, retmode=&amp;quot;text&amp;quot;&#xA;) as handle, open(filepath, &amp;quot;w&amp;quot;) as file:&#xA;    file.write(handle.read())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15161" LastActivityDate="2022-06-08T16:26:53.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19202" PostTypeId="1" CreationDate="2022-06-08T16:52:22.367" Score="0" ViewCount="95" Body="&lt;p&gt;From 3 gene lists (lung, nasal, gut), how to make this gene overlap network plot?&lt;/p&gt;&#xA;&lt;p&gt;This image is figure 4a from: &lt;a href=&quot;https://www.nature.com/articles/s41591-020-01227-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41591-020-01227-z&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gRYmz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gRYmz.png&quot; alt=&quot;gene overlap of 3 gene lists&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15597" LastActivityDate="2022-06-10T18:29:59.357" Title="How to make a network overlap of gene lists across 3 conditions like this image?" Tags="|visualization|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19203" PostTypeId="2" ParentId="19202" CreationDate="2022-06-09T01:59:04.423" Score="0" Body="&lt;p&gt;You could use &lt;a href=&quot;https://www.omnigroup.com/omnigraffle&quot; rel=&quot;nofollow noreferrer&quot;&gt;OmniGraffle&lt;/a&gt; to make this figure.&lt;/p&gt;&#xA;&lt;p&gt;Objects in OmniGraffle have so-called magnets to which you can attach lines, which can join two objects.&lt;/p&gt;&#xA;&lt;p&gt;Once you have objects joined, you could group them and position them as desired.&lt;/p&gt;&#xA;&lt;p&gt;As you drag an object, any attached lines will also change their position as needed. You can drag groupings of objects (e.g., conditions or other categories of objects), which does the same thing.&lt;/p&gt;&#xA;&lt;p&gt;You can export an illustration to PDF or SVG format from OmniGraffle, which will let you do any further work in Adobe Illustrator or other vector illustration tools, or share the illustration with colleagues over the web, for instance.&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2022-06-09T01:59:04.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19204" PostTypeId="2" ParentId="19202" CreationDate="2022-06-09T02:04:44.007" Score="0" Body="&lt;p&gt;This question is a bit difficult to fathom, but essentially I think the answer you are look for is called network theory. Its quite simple, but alas not fashionable. You'll find an introduction &lt;a href=&quot;https://trenton3983.github.io/files/projects/2020-05-21_intro_to_network_analysis_in_python/2020-05-21_intro_to_network_analysis_in_python.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and try and avoid 'directed graphs' because that gets complicated.&lt;/p&gt;&#xA;&lt;p&gt;The alternative way is to use Google fusion tables, which incorporates network analysis, which is really easy to use and investigate. Its very interactive. You'll find more information here,&#xA;&lt;a href=&quot;https://support.google.com/fusiontables/answer/2566732?hl=en&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.google.com/fusiontables/answer/2566732?hl=en&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thats it and nothing more, undirected network theory is pretty simple. Its does not tell you very much (in my opinion), but it makes pretty diagrams.&lt;/p&gt;&#xA;&lt;p&gt;Please note there is an analysis called 'Graph theory' and is not needed here. Whilst this has some similarity to 'network theory' it is distinct and the two shouldn't be confused.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-06-09T03:55:45.497" LastActivityDate="2022-06-09T03:55:45.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19206" PostTypeId="2" ParentId="19195" CreationDate="2022-06-09T02:34:09.210" Score="0" Body="&lt;p&gt;The answer is pretty simple. What you are requesting is fixing the tree prior at a tree you are specifying. The getter/setter is complaining it cannot fit the data to the specified tree because of 26 bp at 26-52 on line 1445 of the nexus file.&lt;/p&gt;&#xA;&lt;p&gt;The first thing needed is to post output to:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sed -n '1445p' ./sp_treeshrink_exons_good.mb.nex&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What I suspect you will find are non-nucleotide characters associated with &amp;quot;missing data&amp;quot; for example &lt;code&gt;xxxxx&lt;/code&gt;, or &lt;code&gt;0000&lt;/code&gt; what the nexus file wants is &lt;code&gt;?????&lt;/code&gt; for missing data. It is possible part of the degenerate nucleotide code is not being observed (long while since I used MB), e.g. &lt;code&gt;RYB&lt;/code&gt; I think its unlikely.&lt;/p&gt;&#xA;&lt;p&gt;The second approach, if the output of the &lt;code&gt;sed&lt;/code&gt; command is pretty normal is to remove the prior setting the tree, i.e. the tree you are supplying is violating e.g. Jukes-Cantor correction.&lt;/p&gt;&#xA;&lt;p&gt;Finally and a related issue is there's an alignment error, i.e. the alignment at this region is out of frame for these residues and Jukes-Cantor is violated so it refuses to proceed. This would be really unusual however, because I've used MB where there are clear pockets of saturation and it hasn't complained.&lt;/p&gt;&#xA;&lt;p&gt;Personally, I'm impressed by the level of foresight in MB under written into this getter/setter (its checking your input). This algorithm is rarely used but is clearly well written.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Okay there is no line 1445. What I recommend is placing the alignment in either &lt;code&gt;Sequtron&lt;/code&gt; or &lt;code&gt;Clustal X&lt;/code&gt; and scrolling down at the position 26-52 and if not successful 'optical analysis' of the alignment.&lt;/p&gt;&#xA;&lt;p&gt;Personally, I would use Biopython AlignIO for this task and &lt;code&gt;Counter&lt;/code&gt; from the &lt;code&gt;Collections&lt;/code&gt; library to interrogate the bug. However, thats coding stuff and optical analysis does the same thing. Essentially, you loop through each sequence and &lt;code&gt;Counter&lt;/code&gt; will make a frequency count of everything in each sequence. You then say if you are not &lt;code&gt;AGCT-&lt;/code&gt; tell me what taxa this is. If you code in Python thats a good follow-up question (separate question).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-06-09T14:31:24.867" LastActivityDate="2022-06-09T14:31:24.867" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19207" PostTypeId="2" ParentId="19197" CreationDate="2022-06-09T04:25:30.840" Score="0" Body="&lt;p&gt;I am a beginner too.  I will try the galaxy (training):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;https://training.galaxyproject.org/training-material/topics/transcriptomics/tutorials/ref-based/tutorial.html&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="15519" LastActivityDate="2022-06-09T04:25:30.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19208" PostTypeId="1" CreationDate="2022-06-09T09:14:36.783" Score="1" ViewCount="8" Body="&lt;p&gt;Has anyone encountered this same installation issue of iCTNet2?&lt;/p&gt;&#xA;&lt;p&gt;I encountered this issue for installation of iCTNet2 via Cytoscape App Manager and similarly via installation from website download/install page.&lt;/p&gt;&#xA;&lt;p&gt;I tried both installations via Cytoscape and via website takes a long time, when both is still installing even 4 hours later, I cancelled the installation. Does it really take that long? Or I should just wait for more than 4hrs to install this?&lt;/p&gt;&#xA;&lt;p&gt;Any recommendations please? Thank you.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/W5lHY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W5lHY.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15603" LastActivityDate="2022-06-09T09:14:36.783" Title="Issue with iCTNet2 installation" Tags="|software-installation|cytoscape|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19209" PostTypeId="2" ParentId="19198" CreationDate="2022-06-09T14:54:37.260" Score="3" Body="&lt;p&gt;IQTree is a fashionable method for speeding up bootstrapping and used on ultra large datasets. What IQTree might be doing is rigidly fixing a model and parameterisation is not estimate via the ML. This might be how it achieves its performance.&lt;/p&gt;&#xA;&lt;p&gt;In context, IQTree can use an establised mixed model, which is one approach to protein phylogeny. As a personal note, I don't favour it and come back to the reason later. It MIGHT also generate its own mixed model. There are not 700 established (published) amino acid matrices plus established mixed models, combined there are no more than 50.&lt;/p&gt;&#xA;&lt;p&gt;There are around 30 protein mutation matrices, Dayhoff models (I honestly don't who Dayhoff is), one for insects which might be for mtDNA ... mtArt, one for mtDNA, mtREV I think and so on. Please check carefully the number of models it is evaluating.&lt;/p&gt;&#xA;&lt;p&gt;Its easy to see how 4 or 5 combinations (or possibly permutations) of 30 become 700 models if it is generating its own mixed model.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, what it does (mixed model or not) BIC and AIC do is given a tree estimate (prior) what's model is the best probabilistic fit. In your case Q.insect is cropping up, whether this is mixed model or not ... dunno.&lt;/p&gt;&#xA;&lt;p&gt;What I don't know is how it calculates the mixed model, is it creating a new singular matrix by averaging (which is probably closer to the algorithm), or is it sequentially passing the data through first one model, then another etc ... Personally, I avoid mixed models for precisely this reason, I dunno what the actual matrix is, but accept it can provide strongest AIC/BIC.&lt;/p&gt;&#xA;&lt;p&gt;It suggests either:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;BIC/AIC results in an outcome that isn't biologically realistic .... &lt;strong&gt;OR&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;there has been some weird parallel/convergent evolution - this happens in parasites for certain&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If you are using a single matrix not mixed model and Q.insect is cropping up - &lt;strong&gt;I would be concerned&lt;/strong&gt;. If it is part of a mixed model, I wouldn't worry.&lt;/p&gt;&#xA;&lt;p&gt;Is it checking 30 models (plus published mixed models) or 700 matrices??&lt;/p&gt;&#xA;&lt;p&gt;If the final point was correct (biologically real result), you check it via blast - simply exclude bacteria and see if you get a plant of insect sequence as the top hit.&lt;/p&gt;&#xA;&lt;p&gt;Ultimately, the uncertainty is the reason I avoid mixed models.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To answer the question, I'm not aware of a bacterial amino acid matrix. HOWEVER, you can select a mixed model. It might just be screening around 50 models via AIC/BIC and within that Q.insect is the most probable.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&#xA;The alternative is to look for support using a general amino acid matrix (for example JTT, LG or JTTDCMut).&lt;/p&gt;&#xA;&lt;p&gt;Personally, I would be very cautious about proceeding with a matrix generated by insect genomes outside a mixed model for bacteria and in the absence of any biological support. FOR EXAMPLE, if these were insect symbionts - that makes sense.&lt;/p&gt;&#xA;&lt;p&gt;Which ever approach is used a Bayesian approach is often required.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-06-09T18:48:21.980" LastActivityDate="2022-06-09T18:48:21.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19211" PostTypeId="1" CreationDate="2022-06-10T09:53:06.550" Score="0" ViewCount="31" Body="&lt;p&gt;I have a fasta file containing reads generated by PacBio HiFi whole genome sequencing of a feces sample from mouse.&lt;/p&gt;&#xA;&lt;p&gt;I would like to use this dataset to generate an assembled circularized genome for an Anaeroplasma organism known to be in this sample.&lt;/p&gt;&#xA;&lt;p&gt;I am familiar with Linux and R. I would very much appreciate your advice on the needed steps and the required tools or R packages.&lt;/p&gt;&#xA;" OwnerUserId="15609" LastEditorUserId="2108" LastEditDate="2022-06-14T20:59:06.410" LastActivityDate="2022-06-14T22:24:53.123" Title="Assemble Anaeroplasma species genome from metagenomic PacBio data" Tags="|assembly|linux|wgs|pacbio|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19212" PostTypeId="2" ParentId="18728" CreationDate="2022-06-10T10:56:11.587" Score="0" Body="&lt;p&gt;The answer is to relax and throw away the 0.001% of stuff that doesn't make sense, lower your standards and just get on with finishing the job you set out to do without actually caring about consistency, logic, meaning or formats that are actually defined or interpretable.&lt;/p&gt;&#xA;&lt;p&gt;Nobody cares about what these SNPs mean, how to interpret them programmatically, why there are version issues in the output of the imputation server or why it throws away ground truth in the output, leaving the user to struggle with all these issues. I mean... if nobody cares, why should I?&lt;/p&gt;&#xA;&lt;p&gt;I've written up as detailed a description of the problems as I can here:&#xA;&lt;a href=&quot;https://github.com/richarddurbin/pbwt/issues/52&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/richarddurbin/pbwt/issues/52&lt;/a&gt; and here: &lt;a href=&quot;https://github.com/richarddurbin/pbwt/issues/51&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/richarddurbin/pbwt/issues/51&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and written a script to throw away all the junk here:&#xA;&lt;a href=&quot;https://github.com/Geromics/covcheck/blob/wip/report-v2/Research/debug_imputation_results.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Geromics/covcheck/blob/wip/report-v2/Research/debug_imputation_results.py&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8875" LastActivityDate="2022-06-10T10:56:11.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19214" PostTypeId="2" ParentId="17962" CreationDate="2022-06-10T14:47:42.680" Score="3" Body="&lt;p&gt;I'm the author of &lt;a href=&quot;https://robert-haas.github.io/gravis-docs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gravis&lt;/a&gt;, an open-source package for interactive graph visualization in Python. It allows to draw directed graphs from NetworkX and the resulting visualization can be modified by user interaction (e.g. node positions, edge curvature and various other visual properties).&lt;/p&gt;&#xA;&lt;p&gt;Here's a minimal example based on your code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import gravis as gv&#xA;import networkx as nx&#xA;&#xA;g = nx.DiGraph()&#xA;g.add_edges_from([(1, 2), (1, 3), (1, 4), (2, 3)])&#xA;gv.d3(g)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output: NetworkX DiGraph visualized with gravis inside a Jupyter notebook:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/8lTu9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8lTu9.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15610" LastEditorUserId="3694" LastEditDate="2022-06-10T16:09:47.780" LastActivityDate="2022-06-10T16:09:47.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19215" PostTypeId="2" ParentId="19202" CreationDate="2022-06-10T18:29:59.357" Score="1" Body="&lt;p&gt;I'm the author of &lt;a href=&quot;https://robert-haas.github.io/gravis-docs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gravis&lt;/a&gt;, an open source package for interactive graph visualization in Python.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It can visualize directed and undirected graphs (=having edges with or without arrows) as well as multigraphs (=having more than one edge between a pair of nodes).&lt;/li&gt;&#xA;&lt;li&gt;It allows to control visual properties such as node sizes, colors, shapes or edge widths and color with data.&lt;/li&gt;&#xA;&lt;li&gt;It recognizes graph objects from various network analysis packages such as NetworkX, igraph, graph-tool and others.&lt;/li&gt;&#xA;&lt;li&gt;The output is based on web technologies (HTML/JS/CSS), so it can be viewed in a browser or Jupyter notebook. There is also support for exporting static images (JPG, PNG, SVG).&lt;/li&gt;&#xA;&lt;li&gt;It comes with many examples on the documentation site.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Here's a quick attempt to reproduce a part of the figure you've provided:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import gravis as gv&#xA;import networkx as nx&#xA;&#xA;s1 = 40&#xA;c1 = 'white'&#xA;&#xA;s2 = 20&#xA;c2 = '#ffd703'&#xA;&#xA;s3 = 20&#xA;c3 = 'orange'&#xA;&#xA;s4 = 20&#xA;c4 = 'red'&#xA;&#xA;g = nx.Graph()&#xA;g.add_node('Lung', color=c1, size=s1, border_size=1)&#xA;g.add_node('Nasal', color=c1, size=s1, border_size=1)&#xA;g.add_node('Gut', color=c1, size=s1, border_size=1)&#xA;&#xA;g.add_node('MDFI', color=c2, size=s2)&#xA;g.add_node('LOXL1', color=c2, size=s2)&#xA;g.add_node('CNBP', color=c2, size=s2)&#xA;&#xA;g.add_node('C3', color=c3, size=s3)&#xA;g.add_node('CA2', color=c3, size=s3)&#xA;g.add_node('SEPP1', color=c3, size=s3)&#xA;&#xA;g.add_node('TFF3', color=c4, size=s4)&#xA;g.add_node('PTMA', color=c4, size=s4)&#xA;&#xA;g.add_edge('Lung', 'MDFI')&#xA;g.add_edge('Nasal', 'LOXL1')&#xA;g.add_edge('Gut', 'CNBP')&#xA;&#xA;g.add_edge('Lung', 'C3')&#xA;g.add_edge('Nasal', 'C3')&#xA;&#xA;g.add_edge('Lung', 'CA2')&#xA;g.add_edge('Gut', 'CA2')&#xA;&#xA;g.add_edge('Nasal', 'SEPP1')&#xA;g.add_edge('Gut', 'SEPP1')&#xA;&#xA;g.add_edge('Lung', 'TFF3')&#xA;g.add_edge('Nasal', 'TFF3')&#xA;g.add_edge('Gut', 'TFF3')&#xA;&#xA;g.add_edge('Lung', 'PTMA')&#xA;g.add_edge('Nasal', 'PTMA')&#xA;g.add_edge('Gut', 'PTMA')&#xA;&#xA;gv.d3(g)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Resulting output in a web browser:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/PlQqj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PlQqj.png&quot; alt=&quot;gravis example&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I've moved the nodes manually a bit so that the layout resembles the provided image. Once everything looks as intended, a static image can be exported with a single button click on the right-hand side.&lt;/p&gt;&#xA;&lt;p&gt;There are many other ways to produce static graph visualizations, e.g. programmatically with other Python libraries mentioned before, or interactively in standalone tools such as Gephi, Cytoscape or Tulip.&lt;/p&gt;&#xA;" OwnerUserId="15610" LastActivityDate="2022-06-10T18:29:59.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19216" PostTypeId="2" ParentId="5012" CreationDate="2022-06-10T18:50:15.030" Score="2" Body="&lt;p&gt;I'm the author of &lt;a href=&quot;https://robert-haas.github.io/gravis-docs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gravis&lt;/a&gt;, an open source package for interactive network visualization in Python. It supports many ways to influence visual properties with data, such as node size, color or location. It also allows to control edge curvature. Therefore it's possible to place all nodes along a line (or another form) and have edges with a chosen degree of curvature between them.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://robert-haas.github.io/gravis-docs/code/examples/advanced_use.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Example 1&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/f47HU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/f47HU.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://robert-haas.github.io/gravis-docs/code/examples/science/mathematics/integer_divisors.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Example 2&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/KqdtJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KqdtJ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Edit: Python code that reproduces your image with NetworkX and gravis&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import gravis as gv&#xA;import networkx as nx&#xA;&#xA;g = nx.Graph()&#xA;g.add_node(1, x=0, y=0)&#xA;g.add_node(2, x=50, y=0)&#xA;g.add_node(3, x=100, y=0)&#xA;g.add_node(4, x=150, y=0)&#xA;g.add_edge(1, 2)&#xA;g.add_edge(1, 3)&#xA;g.add_edge(3, 4)&#xA;g.graph['node_color'] = 'blue'&#xA;g.graph['node_size'] = 15&#xA;gv.d3(g, edge_curvature=1.5, show_node_label=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Result inside a Jupyter notebook (optional, just for convenience):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/CGWmI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CGWmI.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15610" LastEditorUserId="15610" LastEditDate="2022-06-11T13:05:24.270" LastActivityDate="2022-06-11T13:05:24.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19217" PostTypeId="1" CreationDate="2022-06-11T18:26:45.653" Score="0" ViewCount="25" Body="&lt;p&gt;Continuing from the title, in my context, important means that changing the set of AAs (either or one more many) changes the domain and/or fold and/or function and/or family etc.&lt;/p&gt;&#xA;&lt;p&gt;I  noticed that there are two types of methods Protbert (protein language models, sequence-based methods) and/or Alphafold (structure (tertiary) based methods); one needs both methods to &amp;quot;reallY&amp;quot; identify the essential set of AAs in a given protein?&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="15612" LastActivityDate="2022-06-11T18:26:45.653" Title="How to identify most important Amino Acid residues in a given protein?" Tags="|sequence-alignment|protein-structure|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19220" PostTypeId="1" AcceptedAnswerId="19221" CreationDate="2022-06-12T21:07:35.383" Score="1" ViewCount="353" Body="&lt;p&gt;Is ligand a type of protein or vice versa?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastActivityDate="2022-06-13T08:55:56.667" Title="What is the basic difference between a protein and a ligand?" Tags="|proteins|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19221" PostTypeId="2" ParentId="19220" CreationDate="2022-06-13T08:55:56.667" Score="1" Body="&lt;p&gt;A ligand is any molecule that can bind to a receptor. A receptor is always a protein. A ligand can be a protein, but also something else, be it a combination of protein with carbohydrates and-or fats, or just molecules such as cyclic AMP, any chemical compound, structures expressed on the cell surface, pretty much any structure you can think of.&lt;/p&gt;&#xA;" OwnerUserId="15618" LastActivityDate="2022-06-13T08:55:56.667" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19222" PostTypeId="1" CreationDate="2022-06-13T05:54:31.143" Score="0" ViewCount="27" Body="&lt;p&gt;Could someone please explain the output file for HapFlk?&lt;/p&gt;&#xA;&lt;p&gt;My input file was a ped file which has three different populations. Two with same number of snps by imputation and third one is an outgroup population which has the same number of snps but some of them are marked as 0 since it's missing and unimputed.&lt;/p&gt;&#xA;&lt;p&gt;I ran HapFlk with only the --file and --outgroup option so the output would be Flk not the LD based HapFlk and outgroup would be leftout after building the tree&lt;/p&gt;&#xA;&lt;p&gt;When I checked the flk result after running,&#xA;the columns were rs, chr, pos, pzero, flk, pvalue which I could understand but some of the positions have&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;nan for pvalue&lt;/li&gt;&#xA;&lt;li&gt;nan for flk and pvalue&lt;/li&gt;&#xA;&lt;li&gt;1.0 for pzero and nan for flk and pvalue&lt;/li&gt;&#xA;&lt;li&gt;0.5 for pzero and 0.0 for flk and 1.0 for pvalue&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I'm not sure how to interpret these positions cause the positions aren't missing in input ped file.&lt;/p&gt;&#xA;&lt;p&gt;Please help me&lt;/p&gt;&#xA;" OwnerUserId="15621" OwnerDisplayName="Bionoob" LastActivityDate="2022-06-13T14:17:02.310" Title="Could someone please help me understand the outputs for HapFlk" Tags="|genetics|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19223" PostTypeId="1" CreationDate="2022-06-13T20:09:39.987" Score="0" ViewCount="57" Body="&lt;p&gt;I have a list of genes (n = 120) that are involved in breast cancer. And I have a list of differentially expressed genes (n = 70) that are present in the breast cancer gene list. My question is: Are differentially expressed genes significantly more likely to be breast cancer related vs non-differentially expressed genes?&#xA;which statistical test will be suitable to address this question and how I can visualize its results in &lt;code&gt;R&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6504" LastActivityDate="2022-08-14T19:15:13.763" Title="statistical test and visualization of genes present in an experiment" Tags="|r|rna-seq|statistics|visualization|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19224" PostTypeId="1" CreationDate="2022-06-14T08:40:00.233" Score="1" ViewCount="17" Body="&lt;p&gt;I'm trying to run &lt;a href=&quot;https://genome-blog.soe.ucsc.edu/blog/2017/03/17/how-portable-is-your-track-hub-use-hubcheck-to-find-out/&quot; rel=&quot;nofollow noreferrer&quot;&gt;hubCheck utility&lt;/a&gt;* from inside Kubernetes Pod but for some reason I get this Network error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# I ran hubCheck hubCheck from inside the pod&#xA;$ ./hubCheck -noTracks -verbose=2 ftp://ftp.ebi.ac.uk/pub/databases/Rfam/12.0/genome_browser_hub/hub.txt&#xA;### kent source version 431 ###&#xA;Found 2 problems:&#xA;Couldn't open ftp://ftp.ebi.ac.uk/pub/databases/Rfam/12.0/genome_browser_hub/hub.txt&#xA;TCP non-blocking connect() to ftp.ebi.ac.uk IP 193.62.193.138 timed-out in select() after 10000 milliseconds - Cancelling!&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, when I run &lt;code&gt;hubCheck&lt;/code&gt; locally it works fine.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ ./hubCheck -noTracks -verbose=2 ftp://ftp.ebi.ac.uk/pub/databases/Rfam/12.0/genome_browser_hub/hub.txt&#xA;### kent source version 430 ###&#xA;1 tracks in sacCer3&#xA;1 tracks in mm10&#xA;1 tracks in ce10&#xA;1 tracks in galGal4&#xA;1 tracks in ci2&#xA;1 tracks in danRer7&#xA;1 tracks in dm6&#xA;1 tracks in hg38&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The first thing I checked is if the pod has access to the internet, which was the case as you can see below, I can use curl to download &lt;code&gt;hubCheck&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# curl -o tools/hubCheck http://hgdownload.soe.ucsc.edu/admin/exe/linux.x86_64/hubCheck&#xA;  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current&#xA;                                 Dload  Upload   Total   Spent    Left  Speed&#xA;100 33.2M  100 33.2M    0     0  10.0M      0  0:00:03  0:00:03 --:--:-- 10.0M&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I thought this may be caused by a firewall limiting the access to external URL, but I don't think it's the case because other URLs like the one below is working fine both outside and inside K8s:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ ./hubCheck -noTracks -verbose=2 http://younglab.wi.mit.edu/mESC_ChIAPET_hub/hub.txt&#xA;### kent source version 431 ###&#xA;4 tracks in mm9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any help on this would be greatly appreciated!&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;hub Check is a database which facilitates online UCSC Genome Browser display&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="128" LastEditorUserId="3848" LastEditDate="2022-06-14T12:37:10.067" LastActivityDate="2022-06-14T12:37:10.067" Title="Running hubCheck utility from inside K8s's Pod" Tags="|networks|ucsc|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19225" PostTypeId="1" AcceptedAnswerId="19226" CreationDate="2022-06-14T10:12:41.917" Score="1" ViewCount="96" Body="&lt;p&gt;I'd like to perform the annotation (1000genomes, COSMIC etc.) of my variants using SnpSift and SnpEff, however so far all I get are vcf files for separate chromosomes:&#xA;&lt;a href=&quot;http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'd like to annotate my variants and then filter out common SNP's. What in your opinion is best annotation source and where can I get the file?&lt;/p&gt;&#xA;" OwnerUserId="2869" LastEditorUserId="3694" LastEditDate="2022-06-14T14:03:42.480" LastActivityDate="2022-06-15T07:46:47.063" Title="How can I get latest .vcf files with annotation data?" Tags="|vcf|sequence-annotation|ngs|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19226" PostTypeId="2" ParentId="19225" CreationDate="2022-06-14T14:55:16.727" Score="2" Body="&lt;p&gt;Do you mean &lt;code&gt;Somatic Sequence Variants&lt;/code&gt;?,&lt;/p&gt;&#xA;&lt;p&gt;if so, Table 1 inside the following full-text link:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5707196/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5707196/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And additional information from my search--&amp;gt;&#xA;Here's&#xA;&lt;a href=&quot;https://docs.google.com/document/d/1hVtgMYAtvDoUcigKx8OLDmongDRzDQAd/edit?usp=sharing&amp;amp;ouid=102408774767674945852&amp;amp;rtpof=true&amp;amp;sd=true&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.google.com/document/d/1hVtgMYAtvDoUcigKx8OLDmongDRzDQAd/edit?usp=sharing&amp;amp;ouid=102408774767674945852&amp;amp;rtpof=true&amp;amp;sd=true&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Zry8h.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Zry8h.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15519" LastEditorUserId="650" LastEditDate="2022-06-14T20:38:00.200" LastActivityDate="2022-06-14T20:38:00.200" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19227" PostTypeId="1" AcceptedAnswerId="19229" CreationDate="2022-06-14T15:15:41.400" Score="0" ViewCount="358" Body="&lt;p&gt;I have a bam file that contains two @RG lines:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@RG     ID:Foo.Bar_L1.Foo   PU:Bar_L1.Foo SM:Foo        PL:Illumina     LB:Library.fa&#xA;@RG     ID:Foo.Bar_L2.Foo   PU:Bar_L2.Foo SM:Foo        PL:Illumina     LB:Library.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I wish to replace it with one @RG line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@RG     ID:Foo SM:Foo        PL:Illumina     LB:Library.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What is the best way to do this?&lt;/p&gt;&#xA;&lt;p&gt;samtools addreplacerg looks useful, but it looks like I can only affect one @RG per call, and I have no desire to write out the whole bam file twice&lt;/p&gt;&#xA;" OwnerUserId="9222" LastActivityDate="2022-06-14T16:10:26.050" Title="Can I use samtools addreplacerg to replace multiple RG entries at the same time?" Tags="|sam|samtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19228" PostTypeId="1" CreationDate="2022-06-14T15:56:13.023" Score="0" ViewCount="111" Body="&lt;p&gt;I have a file containing multiple protein amino acid sequences. I am interested in visually presenting the location of amino acid motifs PQG, QQG and AQG in the sequence in the form of a heatmap in R. Every location of the motifs should be represented as a differently coloured barcode. Example sequence follows,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;cat archae.fasta&#xA;&amp;gt;A0A075FKY9&#xA;MRRITLPKGIEISGKSPFGTSSGVYEVVISISENKPSDASIASNSFNSLWKDNFHLRVSG&#xA;DHFSQVLGSGSNPIPDNVFDLGSVWIVIQDQFSPVHTSFQFNISPSNQPASKPEEKPRRV&#xA;ETSTVKRTITKPGRLGPPGGKGPRGPSGYPGIKGDKGGRGPTGDKGDKGDKGIPGPQGEK&#xA;GKTGQAGDKGDKGITGPPGEKGPRGSTGPPGDKGDKGVQGKIGDKGLTGTTGPVGDKGTQ&#xA;GPQGPPGGKGLTGIPGPQGDKGTKGPLGPVGERGPTGKPGESGLQGPQGIQGAQGERGHS&#xA;GPPGPQGEKGLIGDKGEIGERGTRGPPGPPGEKGSQGGMSDEGKRLIKELLELLASKNII&#xA;STEEQIKLTSYLY&#xA;&amp;gt;A0A075FMB0&#xA;MPGDKGTSGIRGVPGDKGDKGPQGPPGDKGLTGPVGVPGEKGPTGTSGVQGEKGIQGTPG&#xA;PLGEKGVTGPAGDKGPIGPPGPAGSKGITGPAGPLGDKGTIGPPGPLGDKGSKGPEGPTG&#xA;DKGPQGPQGPAGAKGLTGVPGPQGEKGEKGPLGPIGEKGTTGATGPPGDKGPQGPQGTQG&#xA;ERGTTGPSGEKGPQGPQGIQGPQGERGPTGPIGSIGETGPIGGQGPVGPAGPRGPPGPPG&#xA;EKGPSGGMSSEQKAIFKELLEILTTKEIISTEEQIKLMSYLY&#xA;&amp;gt;A0A075G2F8&#xA;MPKGIEISGKSPFGTSSGVYEVKISVSQSKPSNDSITNNSFNSLWKDNFHLRVSEGYFSE&#xA;ILGSDSNPIPDNVFDLGSVWIVIQDQFSPVHTSFEFKISQDSKSIPTPKEKPRRVDTSSV&#xA;KKVRTIPTRQGPTGSKGIPGERGYPGTKGEKGGKGPTGDKGDKGDKGVPGPQGEKGKTGS&#xA;TGDKGDKGITGSPGEKGPRGPTGPPGDKGDKGVQGKIGDKGLTGTTGPIGDKGPQGPQGP&#xA;PGDKGLTGIPGPQGDKGGRGTIGPSGERGPTGRPGESGLQGPQGIQGPQGERGHSGPPGP&#xA;QGEKGIIGDKGEIGERGPRGPPGPPGEKGSQGGMSDDGKILFKELLQLLASKNIISTEEQ&#xA;IKLTSYLY&#xA;&amp;gt;A0A075GLR7&#xA;MPKGIEISGKSPFGTSSGVYEVRISISQDKPSDDGITNNSFNSLWKDNFHLRVSEGHFSE&#xA;ILGTDSNPIPADVFGLGSIWIIIQDQFSPVHTSFELHISQDNKLLSVPQGKPKPVNATKV&#xA;QKVRAQPGRPGPQGEKGGKGPAGYPGTRGEKGGRGITGDKGDKGDKGIPGPQGEKGIKGP&#xA;AGDKGDKGITGPVGPSGPKGSTGPPGDKGDKGVQGKSGDKGLTGVTGPLGDKGTQGPQGP&#xA;PGPKGLTGVPGPQGDKGMRGSIGPAGERGPTGKPGDPGLPGPQGIQGPQGERGHTGPQGP&#xA;PGEKGLVGDKGEIGDRGPRGPPGPPGEKGSQGGMSEESKRLVKELLELLASKNIISTEEQ&#xA;IKLTSYLY&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As an example, please see a two-colour heatmap. I need something similar but with three colours corresponding to each of the three motifs.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/B3AVX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/B3AVX.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I looked at the heatmaply package but the biggest problem is that my matrix is not numeric. Is there way to make heatmap using text strings in R?&lt;/p&gt;&#xA;&lt;p&gt;Motif is defined as any three character string that matches the general pattern either PQG, QQG and AQG in the uploaded sequence. y-axis is the fasta code appended after &amp;gt; (A0A075FKY9 for example). x-axis would depend on how the data is converted into a matrix.&lt;/p&gt;&#xA;" OwnerUserId="15625" LastEditorUserId="15625" LastEditDate="2022-06-14T16:06:11.040" LastActivityDate="2024-03-06T01:01:30.133" Title="Heatmap of motifs present in protein sequences" Tags="|r|heatmap|motifs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19229" PostTypeId="2" ParentId="19227" CreationDate="2022-06-14T16:10:26.050" Score="2" Body="&lt;p&gt;Not sure about samtools, but &lt;a href=&quot;http://broadinstitute.github.io/picard/command-line-overview.html#AddOrReplaceReadGroups&quot; rel=&quot;nofollow noreferrer&quot;&gt;picard AddOrReplaceReadGroups&lt;/a&gt; can clear existing read groups and write a new one.&lt;/p&gt;&#xA;" OwnerUserId="15489" LastActivityDate="2022-06-14T16:10:26.050" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19230" PostTypeId="1" CreationDate="2022-06-14T16:55:04.473" Score="2" ViewCount="360" Body="&lt;p&gt;I've downloaded the RefSeq transcript sequences (FASTA format) for the GRC38 patch 14 build of the human genome and would now like to extract the gene symbols (only) in order from the headers. I'm confident I can do this using &lt;code&gt;grep&lt;/code&gt; and regular expressions--the trick will be coming up with an appropriate pattern. (The gene symbol &lt;em&gt;appears&lt;/em&gt; to be surrounded by the last pair of parentheses in a header, but I'm not sure this is absolutely true.) However, I'm wondering if there is a more elegant way to do this--e.g., via the NCBI EDirect utilities or &lt;code&gt;SeqKit&lt;/code&gt;--that I'm overlooking. (I have a list of the accession.version numbers.) Any thoughts on this would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="15627" LastActivityDate="2022-07-21T12:00:35.123" Title="Elegant Way to Extract Gene Symbols from FASTA Headers" Tags="|fasta|phylogenetics|grep|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19231" PostTypeId="2" ParentId="19228" CreationDate="2022-06-14T19:37:07.270" Score="0" Body="&lt;p&gt;Firstly, the problem with your data is that it ain't aligned. Without that your motif positions are not relative from one protein to another. Basically you need to align it and there's no wriggle room.&lt;/p&gt;&#xA;&lt;p&gt;Secondly, its easy you simply parse the motifs you want and their location within the string - after it is aligned. In Python you would use &lt;code&gt;finditer&lt;/code&gt; within a comprehension. This will go into a dataframe and subject to a heatmap package, I'm not a 'heatmap' person but it would be easy to graph where the x axis is the position where each motif is found and a qualitative score given for each type of motif (1-3 here). I don't know the equivalent in &lt;code&gt;R&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There is an issue whether an indel will remove a motif and you would need to check for that by running the same command against an unaligned protein and seeing if the frequency matches.&lt;/p&gt;&#xA;&lt;p&gt;My advice is to ask a new question with some code. I dunno whether someone has coded this already, - its easy to do. Personally, I don't think there's sufficient requirement, but I could be wrong.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-06-14T20:49:15.193" LastActivityDate="2022-06-14T20:49:15.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19232" PostTypeId="2" ParentId="19211" CreationDate="2022-06-14T20:57:16.067" Score="0" Body="&lt;p&gt;I would suggest the following workflow, understanding you to have metagenomic HiFi sequencing that you &lt;strong&gt;know&lt;/strong&gt; to include your species of interest at high abundance:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;metagenomic assembly of HiFi with &lt;a href=&quot;https://www.nature.com/articles/s41592-020-00971-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;metaFlyE&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Annotate your resulting long contigs (&amp;gt;500Kbp) using &lt;a href=&quot;https://scholar.google.com/citations?view_op=view_citation&amp;amp;hl=en&amp;amp;user=A87iE7wAAAAJ&amp;amp;citation_for_view=A87iE7wAAAAJ:Tiz5es2fbqcC&quot; rel=&quot;nofollow noreferrer&quot;&gt;GTDB-tk&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Look for your Anaeroplasma among long contigs&lt;/li&gt;&#xA;&lt;li&gt;Assess whether your genome is closed.&lt;/li&gt;&#xA;&lt;li&gt;If the genome is closed, you are done.&lt;/li&gt;&#xA;&lt;li&gt;If the genome is not closed, you need to close it. Assess its completeness using e.g. &lt;a href=&quot;https://ecogenomics.github.io/CheckM/&quot; rel=&quot;nofollow noreferrer&quot;&gt;checkM&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Map your HiFi reads to the genome, see if you can close the remaining gap with e.g. &lt;a href=&quot;https://www.pacb.com/publications/gapfinisher-a-reliable-gap-filling-pipeline-for-sspace-longread-scaffolder-output/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SSpace-LongRead&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I'd suggest using &lt;a href=&quot;https://www.nature.com/articles/s41587-021-01130-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; as an example for further workflow details.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-06-14T20:57:16.067" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19233" PostTypeId="2" ParentId="19211" CreationDate="2022-06-14T22:24:53.123" Score="1" Body="&lt;p&gt;Try &lt;a href=&quot;https://github.com/xfengnefx/hifiasm-meta&quot; rel=&quot;nofollow noreferrer&quot;&gt;hifiasm-meta&lt;/a&gt; (&lt;a href=&quot;https://www.nature.com/articles/s41592-022-01478-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper here&lt;/a&gt;). Whether it works depends on the abundance and the complexity. You may also try HiCanu.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2022-06-14T22:24:53.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19234" PostTypeId="2" ParentId="19199" CreationDate="2022-06-15T01:42:23.110" Score="3" Body="&lt;p&gt;For batch operations, it would be better to acquire a &lt;a href=&quot;https://ncbiinsights.ncbi.nlm.nih.gov/2017/11/02/new-api-keys-for-the-e-utilities/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI API key&lt;/a&gt; and combine &lt;a href=&quot;https://www.gnu.org/software/parallel/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GNU parallel&lt;/a&gt; and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez Direct&lt;/a&gt; .&#xA;If you already have the accession numbers of ITS1 and ITS2 genes, put the accession numbers in a text file like &amp;quot;accession-number.txt&amp;quot;. Otherwise search the ITS genes you want to download and then get the accession number:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db nuccore -query 'you query term[Gene Name]' |&#xA;  efetch -format acc &amp;gt;accession-number.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To download all the sequences in fasta format.:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;parallel -a accession-number.txt -j8 'efetch -db nuccore -id {} -format fasta &amp;gt;{}.fa'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here &lt;code&gt;-j8&lt;/code&gt; is used to control the number of jobs run in parallel, it should be less than the limit of 10.&lt;/p&gt;&#xA;&lt;p&gt;To get related taxonomy (for eg, taxonomy rank, scientific name, taxonomy id, genus and species) of each sequence:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;body='BEGIN { OFS = &amp;quot;\t&amp;quot; } { print &amp;quot;{}&amp;quot;, &lt;span class=&quot;math-container&quot;&gt;$0}'&#xA;paralle -a accession-number.txt -j8 &quot;elink -db nuccore -id {} -target taxonomy -name nuccore_taxonomy | &#xA;  esummary |&#xA;  xtract -pattern DocumentSummary -element Rank ScientificName TaxId Genus Species |&#xA;  awk '$&lt;/span&gt;body'&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15583" LastActivityDate="2022-06-15T01:42:23.110" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19236" PostTypeId="2" ParentId="19225" CreationDate="2022-06-15T07:46:47.063" Score="1" Body="&lt;p&gt;According to your answer I tried to get the files which might be useful regarding annotation:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ih2NE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ih2NE.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2869" LastActivityDate="2022-06-15T07:46:47.063" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19237" PostTypeId="1" AcceptedAnswerId="19238" CreationDate="2022-06-15T13:51:45.097" Score="2" ViewCount="206" Body="&lt;p&gt;I want to convert an hg38 &lt;a href=&quot;http://ftp.ensembl.org/pub/release-106/gtf/homo_sapiens/Homo_sapiens.GRCh38.106.chr.gtf.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;GTF file&lt;/a&gt; to bed format using &lt;code&gt;gtf2bed&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gtf2bed Homo_sapiens.GRCh38.106.gtf &amp;gt; Homo_sapiens.GRCh38.106.gtf.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It outputs a bed file but it only contains the coordinates for the whole genes and has lost the locations of the exons, UTRs, etc.&#xA;While I am running &lt;code&gt;gtf2bed&lt;/code&gt;, I get thousands of warning like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;WARNING:root:None   21  33276625    33276753    +   three_prime_utr&#xA;WARNING:root:WARNING End is after gene end (33279918 &amp;gt; 0)&#xA;WARNING:root:WARNING Start is after gene end&#xA;WARNING:root:None   21  33279752    33279918    +   three_prime_utr&#xA;WARNING:root:WARNING End is after gene end (33283212 &amp;gt; 0)&#xA;WARNING:root:WARNING Start is after gene end&#xA;WARNING:root:None   21  33283094    33283212    +   three_prime_utr&#xA;WARNING:root:WARNING End is after gene end (37101343 &amp;gt; 0)&#xA;WARNING:root:WARNING Start is after gene end&#xA;WARNING:root:None   21  37100814    37101343    +   transcript&#xA;WARNING:root:WARNING End is after gene end (37101343 &amp;gt; 0)&#xA;WARNING:root:WARNING Start is after gene end&#xA;WARNING:root:None   21  37100814    37101343    +   exon&#xA;WARNING:root:WARNING End is after gene end (37237744 &amp;gt; 0)&#xA;WARNING:root:WARNING Start is after gene end&#xA;WARNING:root:None   21  37221419    37237744    +   transcript&#xA;WARNING:root:WARNING End is after gene end (37221597 &amp;gt; 0)&#xA;WARNING:root:WARNING Start is after gene end&#xA;WARNING:root:None   21  37221419    37221597    +   exon&#xA;WARNING:root:WARNING End is after gene end (37237744 &amp;gt; 0)&#xA;WARNING:root:WARNING Start is after gene end&#xA;WARNING:root:None   21  37237404    37237744    +   exon&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I create a bed file that also contains the coordinates of the other genomic features?&lt;/p&gt;&#xA;" OwnerUserId="10386" LastEditorUserId="776" LastEditDate="2022-06-15T23:40:20.233" LastActivityDate="2022-06-15T23:40:20.233" Title="gtf2bed only retains gene information" Tags="|bed|gtf|gffutils|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19238" PostTypeId="2" ParentId="19237" CreationDate="2022-06-15T14:07:20.397" Score="3" Body="&lt;p&gt;I cannot reproduce this. Be sure to use the latest version of &lt;a href=&quot;https://bedops.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BEDOPS&lt;/a&gt; and to use the correct command:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;&#xA;wget http://ftp.ensembl.org/pub/release-106/gtf/homo_sapiens/Homo_sapiens.GRCh38.106.chr.gtf.gz&#xA;&#xA;gtf2bed &amp;lt; &amp;lt;(zcat Homo_sapiens.GRCh38.106.chr.gtf.gz) &amp;gt; Homo_sapiens.GRCh38.106.chr.bed&#xA;&#xA;$ head Homo_sapiens.GRCh38.106.chr.bed&#xA;1       1211339 1214153 ENSG00000186827 .       -       ensembl_havana  gene    .       gene_id &amp;quot;ENSG00000186827&amp;quot;; gene_version &amp;quot;11&amp;quot;; gene_name &amp;quot;TNFRSF4&amp;quot;; gene_source &amp;quot;ensembl_havana&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;;&#xA;1       1211339 1214153 ENSG00000186827 .       -       ensembl_havana  transcript      .       gene_id &amp;quot;ENSG00000186827&amp;quot;; gene_version &amp;quot;11&amp;quot;; transcript_id &amp;quot;ENST00000379236&amp;quot;; transcript_version &amp;quot;4&amp;quot;; gene_name &amp;quot;TNFRSF4&amp;quot;; gene_source &amp;quot;ensembl_havana&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;TNFRSF4-201&amp;quot;; transcript_source &amp;quot;ensembl_havana&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;; tag &amp;quot;CCDS&amp;quot;; ccds_id &amp;quot;CCDS11&amp;quot;; tag &amp;quot;basic&amp;quot;; transcript_support_level &amp;quot;1 (assigned to previous version 3)&amp;quot;;&#xA;1       1213982 1214153 ENSG00000186827 .       -       ensembl_havana  exon    .       gene_id &amp;quot;ENSG00000186827&amp;quot;; gene_version &amp;quot;11&amp;quot;; transcript_id &amp;quot;ENST00000379236&amp;quot;; transcript_version &amp;quot;4&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;TNFRSF4&amp;quot;; gene_source &amp;quot;ensembl_havana&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;TNFRSF4-201&amp;quot;; transcript_source &amp;quot;ensembl_havana&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;; tag &amp;quot;CCDS&amp;quot;; ccds_id &amp;quot;CCDS11&amp;quot;; exon_id &amp;quot;ENSE00001832731&amp;quot;; exon_version &amp;quot;2&amp;quot;; tag &amp;quot;basic&amp;quot;; transcript_support_level &amp;quot;1 (assigned to previous version 3)&amp;quot;;&#xA;1       1213982 1214127 ENSG00000186827 .       -       ensembl_havana  CDS     0       gene_id &amp;quot;ENSG00000186827&amp;quot;; gene_version &amp;quot;11&amp;quot;; transcript_id &amp;quot;ENST00000379236&amp;quot;; transcript_version &amp;quot;4&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;TNFRSF4&amp;quot;; gene_source &amp;quot;ensembl_havana&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;TNFRSF4-201&amp;quot;; transcript_source &amp;quot;ensembl_havana&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;; tag &amp;quot;CCDS&amp;quot;; ccds_id &amp;quot;CCDS11&amp;quot;; protein_id &amp;quot;ENSP00000368538&amp;quot;; protein_version &amp;quot;3&amp;quot;; tag &amp;quot;basic&amp;quot;; transcript_support_level &amp;quot;1 (assigned to previous version 3)&amp;quot;;&#xA;1       1214124 1214127 ENSG00000186827 .       -       ensembl_havana  start_codon     0       gene_id &amp;quot;ENSG00000186827&amp;quot;; gene_version &amp;quot;11&amp;quot;; transcript_id &amp;quot;ENST00000379236&amp;quot;; transcript_version &amp;quot;4&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;TNFRSF4&amp;quot;; gene_source &amp;quot;ensembl_havana&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;TNFRSF4-201&amp;quot;; transcript_source &amp;quot;ensembl_havana&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;; tag &amp;quot;CCDS&amp;quot;; ccds_id &amp;quot;CCDS11&amp;quot;; tag &amp;quot;basic&amp;quot;; transcript_support_level &amp;quot;1 (assigned to previous version 3)&amp;quot;;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Finishes without any of the mentioned warnings.&lt;/p&gt;&#xA;" OwnerUserId="15631" LastActivityDate="2022-06-15T14:07:20.397" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19241" PostTypeId="2" ParentId="19230" CreationDate="2022-06-15T14:51:32.567" Score="0" Body="&lt;p&gt;There are probably better ways of doing this with Python or another tool, but here's a quick method that works in bash (tested on Linux) assuming that the Gene Symbols are contained in parentheses, i.e. &amp;quot;(Gene Symbol)&amp;quot;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pat='\(([^)]+)\)'&#xA;while read line; do &#xA;    if [[ &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$line&quot; =~ $&lt;/span&gt;pat ]]; then &#xA;        echo ${BASH_REMATCH[1]}; &#xA;    fi; &#xA;done &amp;lt; human.1.rna.fna&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9999" LastEditorUserId="9999" LastEditDate="2022-06-15T15:54:03.723" LastActivityDate="2022-06-15T15:54:03.723" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19242" PostTypeId="1" AcceptedAnswerId="19360" CreationDate="2022-06-15T16:19:00.823" Score="2" ViewCount="237" Body="&lt;p&gt;I want to convert a genetic matrix (rows are samples, columns are alleles) into a STRUCTURE file (output .str) within R. Is there a preferred way to achieve this? I've found &lt;a href=&quot;https://rdrr.io/cran/graph4lg/man/genind_to_structure.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;genind_to_structure&lt;/a&gt; in the &lt;a href=&quot;https://rdrr.io/cran/graph4lg/&quot; rel=&quot;nofollow noreferrer&quot;&gt;graph4lg&lt;/a&gt; R package, but I wanted to see if there are other approaches.&lt;/p&gt;&#xA;&lt;p&gt;An example of what I mean by a &amp;quot;genetic matrix&amp;quot;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Build a matrix of alleles&#xA;temp &amp;lt;- lapply(1:30, function(i) sample(1:9, 4, replace = TRUE))&#xA;temp &amp;lt;- sapply(temp, paste, collapse=&amp;quot;&amp;quot;)&#xA;temp &amp;lt;- matrix(temp, nrow = 10, dimnames = list(paste(&amp;quot;ind&amp;quot;,1:10), paste(&amp;quot;loc&amp;quot;,1:3)))&#xA;temp&#xA;&#xA;# Convert matrix to genind format (for adegenet package)&#xA;test &amp;lt;- df2genind(temp, ploidy = 4, sep = &amp;quot;&amp;quot;)&#xA;test&#xA;&#xA;# How to convert to STRUCTURE?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If anyone has input, please chime in! Thanks&lt;/p&gt;&#xA;" OwnerUserId="14041" LastEditorUserId="73" LastEditDate="2022-06-16T22:15:34.787" LastActivityDate="2022-07-06T19:28:52.927" Title="Converting genetic matrix to STRUCTURE" Tags="|r|file-formats|phylogenetics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19243" PostTypeId="1" CreationDate="2022-06-15T16:49:50.337" Score="2" ViewCount="36" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9526743/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am using the Bioconductor package &amp;quot;DeconRNASeq&amp;quot; to perform tissue deconvolution. Let's say I run the following code (this is from the manual):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BiocManager::install(c(&amp;quot;DeconRNASeq&amp;quot;))&#xA;library(DeconRNASeq)&#xA;&#xA;data(multi_tissue)&#xA;datasets &amp;lt;- x.data[,2:11]&#xA;signatures &amp;lt;- x.signature.filtered.optimal[,2:6]&#xA;proportions &amp;lt;- fraction&#xA;&#xA;res &amp;lt;- DeconRNASeq(datasets, signatures, proportions, checksig=FALSE, known.prop = TRUE, use.scale = TRUE, fig = TRUE). &#xA;res.out &amp;lt;- as.data.frame(res[[&amp;quot;out.all&amp;quot;]])&#xA;str(res.out)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And receive the following results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;'data.frame':   10 obs. of  5 variables:&#xA; &lt;span class=&quot;math-container&quot;&gt;$ brain : num  0.0468 0.0718 0.0685 0.0424 0.6471 ...&#xA; $&lt;/span&gt; muscle: num  0 0.1172 0.6081 0.0489 0.1344 ...&#xA; &lt;span class=&quot;math-container&quot;&gt;$ lung  : num  0.0939 0.0829 0.1265 0.6147 0.0709 ...&#xA; $&lt;/span&gt; liver : num  0.0947 0.6187 0.1355 0.1897 0.0812 ...&#xA; $ heart : num  0.7645 0.1093 0.0615 0.1043 0.0665 ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is it possible for me to obtain the subset of genes that were successfully deconvolved? For instance, the first sample had 0.0468 proportion from &amp;quot;brain&amp;quot;. Is it possible for me to see which genes specifically were composing this 0.0468 proportion?&lt;/p&gt;&#xA;&lt;p&gt;Note: This R package does not appear to have a Q&amp;amp;A forum. If there is a recommended place to post this question other than Bioinformatics StackExchange, I would be grateful to hear suggestions.&lt;/p&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="15636" LastEditorUserId="73" LastEditDate="2022-06-16T22:13:30.797" LastActivityDate="2022-06-16T22:13:30.797" Title="DeconRNASeq: Extract gene names from returned mixing proportions" Tags="|r|rna-seq|bioconductor|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19244" PostTypeId="2" ParentId="18757" CreationDate="2022-06-15T18:33:26.740" Score="2" Body="&lt;p&gt;Ultimately, I achieved this using the R package &lt;a href=&quot;https://github.com/EricArcher/strataG&quot; rel=&quot;nofollow noreferrer&quot;&gt;strataG&lt;/a&gt;. The steps taken (in R) are shown below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;arp.object &amp;lt;- arlequinRead(arp.path)&#xA;gtype.object &amp;lt;- arp2gtypes(arp.object)&#xA;genind.object &amp;lt;- gtypes2genind(gtype.object)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I found that fastSimcoal2 shows an odd behavior when simulating DNA markers, in which sequences are concatenated together (regardless of marker blocks specified; relevant post &lt;a href=&quot;https://groups.google.com/g/fastsimcoal/c/cVT2jU7sqxA&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). Running the fastSimcoal2 simulations from strataG addressed this bug (and is more straightforward/reproducible, anyways).&lt;/p&gt;&#xA;" OwnerUserId="14041" LastActivityDate="2022-06-15T18:33:26.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19246" PostTypeId="1" AcceptedAnswerId="19257" CreationDate="2022-06-16T03:27:33.577" Score="3" ViewCount="282" Body="&lt;p&gt;TPM and FPKM of RNA-Seq data form GDC TCGA calculated based STAR were retrieved, respectively. The correlation between a specific gene, e.g. HIF1A, and other genes were calculated based on TPM and FPKM, respectively. And the significant genes were filtered based on P value &amp;lt; 0.05. Here is the intersection of these two significant gene set shown in venn diagram. IMO, the two gene sets should not be obiviously different. Ang advice for this discrepance would be appreciated!&#xA;&lt;a href=&quot;https://i.stack.imgur.com/hPvRN.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hPvRN.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15640" LastActivityDate="2022-06-18T08:03:39.230" Title="Different results of spearman correlation between TPM and FPKM" Tags="|rna-seq|correlation|fpkm|tpm|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19247" PostTypeId="1" AcceptedAnswerId="19294" CreationDate="2022-06-16T06:44:28.617" Score="2" ViewCount="36" Body="&lt;p&gt;I have a couple of metagenomic bins which are okay in quality, but often missing rRNAs (16S, 23S...). I assume this is due to high population variability, combined with the high conservation of rRNAs, and reads of at most 300 bp. My binning software of choice is vamb.&lt;/p&gt;&#xA;&lt;p&gt;Could this be fixed? Once a metagenomic bin is built, is there a way to see which 16S (or multiple 16S possible sequences) are likely to be associated with it? I considered using tools which explore the assembly graph, either manually (Bandage) or automatically (spacegraphcats), but I'm not sure if this is the right approach. After all, if the rRNA genes were linked in the assembly graph to genes that now clearly belong to a bin, they would have been added to that bin...It feels like this is a problem others may have encountered before, so I was wondering if anyone has an idea : )&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2022-06-21T20:06:03.050" Title="rRNA genes missing from metagenomic bins; is there a way to recruit them?" Tags="|rna|metagenome|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19248" PostTypeId="1" CreationDate="2022-06-16T08:03:25.870" Score="2" ViewCount="55" Body="&lt;p&gt;I have multiple data frames in R in the following format:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Dataframe1&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Var1    Var2    cor p   fdr&#xA;Bacteroides_galacturonicus  Lactobacillus_rogosae   0.967374392060265   0   0&#xA;Bacteroides_galacturonicus  Lachnospira_pectinoschiza   0.526779994497013   0   0&#xA;Lactobacillus_rogosae   Lachnospira_pectinoschiza   0.29419393866076    0   0&#xA;Bacteroides_galacturonicus  Bacteroides_salyersiae  0.374581964109831   0.548   0.829267117726658&#xA;Lactobacillus_rogosae   Bacteroides_salyersiae  0.45736458243079    0.002   0.03622024291498&#xA;Lachnospira_pectinoschiza   Bacteroides_salyersiae  0.566636776534  0.21    0.56910941475827&#xA;Bacteroides_galacturonicus  Holdemanella_biformis   0.342443352 0.094   0.394447279549719&#xA;Lactobacillus_rogosae   Holdemanella_biformis   0.15592768883205    0.174   0.527328455284553&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;Dataframe2&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Var1    Var2    cor p   fdr&#xA;Lactobacillus_rogosae   Bacteroides_galacturonicus  0.467374392060265   0   0&#xA;Lachnospira_pectinoschiza   Bacteroides_galacturonicus  0.426779994497013   0   0&#xA;Lachnospira_pectinoschiza   Lactobacillus_rogosae   0.69419393866076    0   0&#xA;Bacteroides_galacturonicus  Bacteroides_salyersiae  0.074581964109831   0.548   0.829267117726658&#xA;Lactobacillus_rogosae   Bacteroides_salyersiae  0.320636458243079   0.002   0.03622024291498&#xA;Lachnospira_pectinoschiza   Bacteroides_salyersiae  0.132857736776534   0.21    0.56910941475827&#xA;Bacteroides_galacturonicus  Holdemanella_biformis   0.19490130543352    0.094   0.394447279549719&#xA;Lactobacillus_rogosae   Holdemanella_biformis   0.14472768883205    0.174   0.527328455284553&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These are two correlation matrices that I want to bind through &lt;code&gt;rbind()&lt;/code&gt;. The problem is that the exact correlation is represented in two ways in those two dataframes.&lt;/p&gt;&#xA;&lt;p&gt;A bacteria is found in &lt;code&gt;Var1&lt;/code&gt; of &lt;strong&gt;Dataframe:1&lt;/strong&gt; while its pair is found in &lt;code&gt;Var2&lt;/code&gt; of &lt;strong&gt;Dataframe:2&lt;/strong&gt; (&lt;strong&gt;see first correlation&lt;/strong&gt;).&lt;/p&gt;&#xA;&lt;p&gt;I want to make a column so that each pair of correlations can be represented in the same format. e.g.:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Var1:Var2   cor p   fdr&#xA;Lactobacillus_rogosae:Bacteroides_galacturonicus    0.967374392060265   0   0 &#xA;Lactobacillus_rogosae:Bacteroides_galacturonicus    0.467374392060265   0   0 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can someone help me achieve this, please?&lt;/p&gt;&#xA;" OwnerUserId="7427" LastEditorUserId="3694" LastEditDate="2022-06-16T11:13:32.903" LastActivityDate="2023-07-11T14:04:25.477" Title="Generate a single column from two different columns in the same format" Tags="|r|correlation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19249" PostTypeId="1" CreationDate="2022-06-16T08:13:15.877" Score="2" ViewCount="373" Body="&lt;p&gt;I used bwa mem to align the DNA with the reference genome. If there are split reads (chimeric reads, come from two different parts of the genome), will it be split into two lines rather than one line?&lt;/p&gt;&#xA;" OwnerUserId="15643" LastEditorUserId="15643" LastEditDate="2022-06-19T05:38:18.160" LastActivityDate="2022-06-19T05:38:18.160" Title="How does split reads look like in sam files?" Tags="|sequence-alignment|bwa|sam|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19250" PostTypeId="2" ParentId="19248" CreationDate="2022-06-16T08:13:59.340" Score="1" Body="&lt;p&gt;I think this should be fairly straightforward:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# working on Dataframe2&#xA;Dataframe2&lt;span class=&quot;math-container&quot;&gt;$tmp &amp;lt;- Dataframe2$&lt;/span&gt;Var2   # Create a temporary column of Var2&#xA;Dataframe2&lt;span class=&quot;math-container&quot;&gt;$Var2 &amp;lt;- Dataframe2$&lt;/span&gt;Var1  # Replace Var2 values by Var1&#xA;Dataframe2&lt;span class=&quot;math-container&quot;&gt;$Var1 &amp;lt;- Dataframe2$&lt;/span&gt;tmp   # Re-attribute tmp values to Var1&#xA;Dataframe2$tmp &amp;lt;- NULL              # delete tmp column&#xA;&#xA;# Binding the two dataframes&#xA;df &amp;lt;- rbind(Dataframe1, Dataframe2)&#xA;&#xA;# Concatenating the two columns&#xA;df&lt;span class=&quot;math-container&quot;&gt;$Var1 &amp;lt;- paste(df$&lt;/span&gt;Var1, df&lt;span class=&quot;math-container&quot;&gt;$Var2, sep = &quot;:&quot;)&#xA;df$&lt;/span&gt;Var2 &amp;lt;- NULL                     # delete Var2 column&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;PS: I haven't tested this code so let me know if it works for you.&lt;/p&gt;&#xA;&lt;p&gt;EDIT1:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;InvertedRows &amp;lt;- Dataframe1[!(Dataframe1$Var1 %in% Dataframe2$Var2),]&#xA;InvertedRows$tmp &amp;lt;- InvertedRows$Var2   # Create a temporary column of Var2&#xA;InvertedRows$Var2 &amp;lt;- InvertedRows$Var1  # Replace Var2 values by Var1&#xA;InvertedRows$Var1 &amp;lt;- InvertedRows$tmp   # Re-attribute tmp values to Var1&#xA;InvertedRows$tmp &amp;lt;- NULL              # delete tmp column&#xA;Dataframe2&amp;lt;- rbind(InvertedRows, Dataframe1[Dataframe1$Var1 %in% Dataframe2$Var2,])&#xA;df &amp;lt;- rbind(Dataframe1, Dataframe2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4413" LastEditorUserId="4413" LastEditDate="2022-06-16T10:01:34.460" LastActivityDate="2022-06-16T10:01:34.460" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19251" PostTypeId="2" ParentId="19164" CreationDate="2022-06-16T11:08:40.833" Score="2" Body="&lt;p&gt;It is simpler to use &lt;code&gt;cut&lt;/code&gt; command to remove filename extensions:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ls -1 | cut -d '.' -f1 | sort | uniq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15583" LastActivityDate="2022-06-16T11:08:40.833" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19252" PostTypeId="2" ParentId="19160" CreationDate="2022-06-16T11:15:43.470" Score="3" Body="&lt;p&gt;The command &lt;code&gt;create_pan_genome_plots&lt;/code&gt; from &lt;a href=&quot;http://sanger-pathogens.github.io/Roary/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Roary&lt;/a&gt; can create a plot like this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/L3X7S.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/L3X7S.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15583" LastActivityDate="2022-06-16T11:15:43.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19253" PostTypeId="2" ParentId="19249" CreationDate="2022-06-16T13:01:54.593" Score="4" Body="&lt;p&gt;I'm assuming you mean supplementary/chimeric alignment? The &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;SAM Format Specification&lt;/a&gt; has a really detailed explanation as well as its &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMtags.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Optional Fields Specification&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In general though, you will have an additional line in the SAM file to represent a chimeric alignment yes. For a single read, one line will represent the primary alignment; this is arbitrarily chosen. The other line will represent the split/supplementary alignment and have its supplementary alignment flag set. If there are multiple chimeric alignments, each one will have its own line.&lt;/p&gt;&#xA;&lt;p&gt;You check if a primary alignment has any supplementary alignments associated to it by checking the SA tag which will list the location(s) of any supplementary alignment(s).&lt;/p&gt;&#xA;" OwnerUserId="9999" LastActivityDate="2022-06-16T13:01:54.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19254" PostTypeId="1" AcceptedAnswerId="19256" CreationDate="2022-06-16T13:17:19.410" Score="1" ViewCount="135" Body="&lt;p&gt;I understand GSEA/GSVA can take microarray-like expression matrix such as the output of voom() or vst(). However, I have a question on how we can also use svaseq() variables, correct the output from voom() or vst(), and apply GSVA on it.&lt;/p&gt;&#xA;&lt;p&gt;I assume this is possible since I have read papers showing PCA after correction by SVA (such as this &lt;a href=&quot;https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0061088&quot; rel=&quot;nofollow noreferrer&quot;&gt;one&lt;/a&gt; and this &lt;a href=&quot;https://www.pnas.org/doi/10.1073/pnas.1814397115&quot; rel=&quot;nofollow noreferrer&quot;&gt;one&lt;/a&gt; in their supplementary figure), but my statistical and coding knowledge does not allow me to recreate what they do. Is there a known pipeline that can do this&lt;/p&gt;&#xA;&lt;p&gt;Edit: to clarify, I would like to generate a normalised matrix (e.g. rlog, or out put of voom from limma, or vst from DESeq2) that is adjusted for batch effect. ComBat, for example, can provide what I need. However, since I am doing my DESeq2 DGE analysis with svaseq(), I want to adjust it with svaseq variables. I am wondering if there are pipelines that can use svaseq vairables (or any varaibles) and adjust the value in the normalised expression matrix.&lt;/p&gt;&#xA;" OwnerUserId="15648" LastEditorUserId="15648" LastEditDate="2022-06-16T16:33:51.500" LastActivityDate="2022-06-16T18:07:58.933" Title="Can I Incorporate svaseq() into GSEA/GSVA analysis?" Tags="|r|rna-seq|batch-effects|gsea|sva|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19255" PostTypeId="1" AcceptedAnswerId="19269" CreationDate="2022-06-16T14:25:30.237" Score="4" ViewCount="100" Body="&lt;p&gt;My question is about nanopore sequencing and specifically about the current that is measured by the device in each measurement. The question is:&#xA;In each measurement in nanopore sequencing, the change in the current between two sides of the membrane at the sensing region is recorded by the sequencer. Is it a single nucleotide or a k-mer or the difference between the first and last base in the kmer that contributes to one single measurement?&#xA;I am citing two papers here that I looked at:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.nature.com/articles/s41587-021-01108-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nanopore sequencing technology, bioinformatics and applications&lt;/a&gt;&#xA;(Nature 2021)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.nature.com/articles/nbt.3423&quot; rel=&quot;nofollow noreferrer&quot;&gt;Three decades of nanopore sequencing (Nature 2016)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Based on my finding in these papers, I realized that all the nucleotide molecules in the k-mer in the sensing region contribute to the measurement, and not only one base, yet I am not sure if it is a good assumption.&lt;/p&gt;&#xA;&lt;p&gt;I know some models like HMM and NN sequential models can computationally infer the individual bases (for example some basecallers). But my question is more about what is &lt;em&gt;physically&lt;/em&gt; measured?&#xA;In the second cited paper, it is stated that, in more recent nanopore designs, DNA strand can be fixed such that one individual nucleotide is positioned in the sensing region. Still, the diameter of the sensing region is not exactly one base.&#xA;Moreover, since the current &lt;em&gt;change&lt;/em&gt; is recorded not the current itself, the contributor could be the &lt;em&gt;difference&lt;/em&gt; between the base which just passed from the sensing region and the one base that just entered?&lt;/p&gt;&#xA;&lt;p&gt;So I restate my question; what is the contributing factor in single measurements of nanopore sequencing?&#xA;In case it is the k-mer that is measured, how could we find k? Is it based on the properties of the device and the kit information?&lt;/p&gt;&#xA;" OwnerUserId="15341" LastEditorUserId="15341" LastEditDate="2022-06-16T14:31:38.477" LastActivityDate="2022-06-18T07:59:17.717" Title="What is the contributing factor in single measurements of nanopore sequencing? One base, a k-mer or difference between the base left and entered?" Tags="|nanopore|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19256" PostTypeId="2" ParentId="19254" CreationDate="2022-06-16T18:07:58.933" Score="1" Body="&lt;p&gt;Use &lt;code&gt;removeBatchEffects&lt;/code&gt; from &lt;code&gt;limma&lt;/code&gt;. The input counts should be on log scale, so &lt;code&gt;vst&lt;/code&gt; and &lt;code&gt;rlog&lt;/code&gt; are ok. The &lt;code&gt;covariate&lt;/code&gt; argument can take the surrogate matrix from svaseq.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://rdrr.io/bioc/limma/man/removeBatchEffect.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rdrr.io/bioc/limma/man/removeBatchEffect.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15650" LastActivityDate="2022-06-16T18:07:58.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19257" PostTypeId="2" ParentId="19246" CreationDate="2022-06-16T21:59:43.403" Score="2" Body="&lt;p&gt;This shouldn't be surprising that you see different correlations between gene expression data when expressed in different units.&#xA;To see why, let's look at how these units are defined.&lt;/p&gt;&#xA;&lt;p&gt;Let's denote the absolute number of fragments we sequence from an RNA-seq measurement as &lt;span class=&quot;math-container&quot;&gt;$X_{gi}$&lt;/span&gt; for gene &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; in sample &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$l_g$&lt;/span&gt; as the length of gene &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; in kilobases.&#xA;Then our different expression values will be:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;f_{gi} = \frac{ X_{gi} \cdot 10^6 }{ l_g \cdot \sum_h X_{hi} } \text{ in FPKM}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;t_{gi} = \frac{ X_{gi} \cdot 10^6 }{ l_g \cdot \sum_h \frac{ X_{hi} }{ l_h } } \text{ in TPM}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;You convert from FPKM to TPM units like so:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;t_{gi} = 10^6 \cdot \frac{ f_{gi} }{ \sum_h f_{hi} }&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;As you can see, &lt;em&gt;within&lt;/em&gt; any single sample, &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;, this last denominator is fixed.&#xA;But it is not fixed &lt;em&gt;across&lt;/em&gt; samples (i.e. &lt;span class=&quot;math-container&quot;&gt;$\sum_h f_{hi} \ne \sum_h f_{hj}$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$i \ne j$&lt;/span&gt;).&#xA;This means that the ordering for a single gene's expression across samples is not preserved when you switch between units (i.e. if &lt;span class=&quot;math-container&quot;&gt;$f_{gi} &amp;gt; f_{gj}$&lt;/span&gt;, it is &lt;strong&gt;not&lt;/strong&gt; guaranteed that &lt;span class=&quot;math-container&quot;&gt;$t_{gi} &amp;gt; t_{gj}$&lt;/span&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Because Spearman correlation is based on the order of each gene across samples in your dataset, and the order of genes in FPKM will be somewhat different from the order of genes in TPM, you will find different correlations between HIF1A and other genes when using the different units.&lt;/p&gt;&#xA;&lt;p&gt;Here's a simplified, concrete example.&#xA;Imagine that your transcriptome is made of up 3 genes (A, B, and C) with lengths 1 kb, 2 kb, and 3 kb, respectively.&#xA;And imagine you're doing RNA-seq in 4 samples.&#xA;If your absolute count matrix is this&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;X = 10^6 \cdot \begin{bmatrix}&#xA;  1 &amp;amp; 2 &amp;amp; 9 &amp;amp; 7 \\&#xA;  2 &amp;amp; 2 &amp;amp; 6 &amp;amp; 5 \\&#xA;  3 &amp;amp; 2 &amp;amp; 2 &amp;amp; 5&#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then the expression matrix in FPKM is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;F = \begin{bmatrix}&#xA;  166\,666.7 &amp;amp; 333\,333.3 &amp;amp; 529\,411.76 &amp;amp; 411\,764 \\&#xA;  166\,666.7 &amp;amp; 166\,666.7 &amp;amp; 176\,470.59 &amp;amp; 147\,058.82 \\&#xA;  166\,666.7 &amp;amp; 111\,111.1 &amp;amp; 39\,215.69 &amp;amp; 98\,039.22&#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;and in TPM its&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$&#xA;T = \begin{bmatrix}&#xA;  333\,333.3 &amp;amp; 545\,454.5 &amp;amp; 710\,526.32 &amp;amp; 626\,865.7 \\&#xA;  333\,333.3 &amp;amp; 272\,727.3 &amp;amp; 236\,842.11 &amp;amp; 223\,880.6 \\&#xA;  333\,333.3 &amp;amp; 181\,818.2 &amp;amp; 52\,631.58 &amp;amp; 149\,253.7&#xA;\end{bmatrix}&#xA;$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you calculate the Spearman correlation for genes B and C against A, then you get these values.&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;Gene&lt;/th&gt;&#xA;&lt;th&gt;Correlation with A (FPKM)&lt;/th&gt;&#xA;&lt;th&gt;Correlation with A (TPM)&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;B&lt;/td&gt;&#xA;&lt;td&gt;0.316&lt;/td&gt;&#xA;&lt;td&gt;-0.8&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;C&lt;/td&gt;&#xA;&lt;td&gt;-1&lt;/td&gt;&#xA;&lt;td&gt;-1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;You can see that gene C is anti-correlated with A, but B actually switches signs and goes from moderately correlated in FPKM to strongly anti-correlated in TPM.&lt;/p&gt;&#xA;&lt;p&gt;So your observation that the same comparison on the different units gives pretty different results is expected.&#xA;If you made a scatter plot with Spearman's &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; from TPM along the x-axis and Spearman's &lt;span class=&quot;math-container&quot;&gt;$\rho$&lt;/span&gt; from FPKM along the y-axis for all the genes that had a correlation p-value &amp;lt; 0.05, you'll probably have a lot of genes that also flip signs like I showed with this toy example.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/22872506/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper by Wagner &lt;em&gt;et al.&lt;/em&gt;&lt;/a&gt; is probably a good paper to review. It can give some context about RPKM and how to think about count-based sequencing data. Correlation-based analyses on data like this are typically used as a first-pass approach to get an idea about how things are behaving. But it's not very precise, as you've experienced. If you're doing exploratory work, I'd be inclined to pick TPM units, look at the results you have, then try to come up with a differential analysis question that can be handled in a more statistically robust way.&lt;/p&gt;&#xA;" OwnerUserId="240" LastEditorUserId="73" LastEditDate="2022-06-18T08:03:39.230" LastActivityDate="2022-06-18T08:03:39.230" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19260" PostTypeId="1" CreationDate="2022-06-17T12:44:15.483" Score="3" ViewCount="249" Body="&lt;p&gt;I have an annotation file. I would like to remove redundancy, as shown in the example (in the real file, I have a lot of these redundant cases).&lt;/p&gt;&#xA;&lt;p&gt;I would like to consider only one of the following genes (the longest could be a good choice). In the case below, the 2 genes have several common exons. As you can see here in the picture, there are two identified genes in the same locus (one identified through Augustus, the other one through GeneMark. They have the same start codon, and the same initial codons, but different final codons and stop codons. Hence, I'd like to retain only one of the two genes:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7RryN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7RryN.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;How can I do that? I have already tried with bedtools, agat_sp_keep_longest_isoform.pl but with no result.&lt;/p&gt;&#xA;&lt;p&gt;Thanks for the support!&lt;/p&gt;&#xA;" OwnerUserId="15596" LastEditorUserId="73" LastEditDate="2022-06-21T09:41:02.693" LastActivityDate="2022-06-21T09:41:02.693" Title="How to remove redundancy from a gtf file?" Tags="|sequence-annotation|bedtools|gtf|merge|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19261" PostTypeId="1" CreationDate="2022-06-17T14:49:17.947" Score="3" ViewCount="90" Body="&lt;p&gt;I am looking for an algorithm to find the &amp;quot;best&amp;quot; alignment between two sequences of integers similar to how one aligns nucleic acids or amino acids for homology comparisons. For example, the best alignment for the two sequences below is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;                10 06 13 91 22 16 06&#xA;                |  |  |  |  |  |  |&#xA;       01 01 02 10 04 11 92 22 17 05 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or an example with gaps:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;                10 _  _  91 22 16 06 22 12&#xA;                |  |  |  |  |  |  |&#xA;          11 02 10 04 11 92 22 17 05 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The alignment would be based on the distance between each number similar to aligning amino acid sequences where one would prefer to align a hydrophobic amino acid with another hydrophobic amino acid because they are more &amp;quot;similar&amp;quot;. I hope this explanation makes sense.&lt;/p&gt;&#xA;&lt;p&gt;Another way to think about this problem is if you took the 20 amino acids and replaced them with a number from 1-20 instead of their single letter abbreviation and then tried to align 2 sequenced based on how close the numbers where, how would someone align them? In this example the alignment is based on how close two integers are from one another whereas when aligning amino acids the alignment is based on the chemical similarity of the amino acids, ie leucine is very similar to isoleucine whereas valine is very dissimilar to tryptophan.&lt;/p&gt;&#xA;&lt;p&gt;I've been experimenting with using the biopython pairwise library and trying to utilize it for my problem. There is a similar approach to what I am looking for here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/a/45512865/1255817&quot;&gt;https://stackoverflow.com/a/45512865/1255817&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This solution solves my problem, but only for the single character digits [0-9] and my numbers are more than one character.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know of an approach (algorithm, package, library) to align sequences of integers? Ideally, I am looking for something in Python with a minimal working example.&lt;/p&gt;&#xA;" OwnerUserId="15656" LastEditorUserId="3848" LastEditDate="2023-02-04T18:33:00.037" LastActivityDate="2023-02-04T18:33:00.037" Title="How to align integer (non-DNA/protein) sequences?" Tags="|phylogenetics|sequence-alignment|phylogeny|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19262" PostTypeId="2" ParentId="18914" CreationDate="2022-06-17T17:22:22.730" Score="1" Body="&lt;p&gt;You have two different types of values you want to display on the x-axis: &lt;code&gt;Cluster&lt;/code&gt; (a categorical variable) and &lt;code&gt;Count&lt;/code&gt; (a continuous variable).&#xA;As Ram RS suggested, the easiest way to accomplish this is by taking your first plot (which uses a continuous x-axis scale) and facet along your discrete variable.&#xA;This gets you what you want, and the rest of the work is adjusting the theme to make it look how you want.&lt;/p&gt;&#xA;&lt;p&gt;I'm guessing based on your data that you're trying to create something that looks like this figure from &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/23945592/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alexandrov &lt;em&gt;et al.&lt;/em&gt;, Nature, 2013&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HcYWE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HcYWE.png&quot; alt=&quot;Mutation distribution plot from Alexandrov et al., Nature, 2013&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can accomplish this by customizing the facet and the theme.&#xA;Here's an example.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gg &amp;lt;- (&#xA;  ggplot(&#xA;    data = data_for_plotting, &#xA;    mapping = aes(&#xA;      x = log10(Count),&#xA;      y = log10(Nonsilent.Mutation.Rate),&#xA;      color = factor(Clusters)&#xA;    )&#xA;  )&#xA;  + geom_point(alpha = 1, size = 1)&#xA;  + facet_wrap(&#xA;    ~ Cluster,&#xA;    # this ensures that all facets are shown on the same row instead of wrapping&#xA;    nrow = 1&#xA;  )&#xA;  + theme_bw()&#xA;  + theme(&#xA;    # this removes the spacing between facets&#xA;    panel.spacing = unit(0, &amp;quot;lines&amp;quot;),&#xA;    # this removes the border around facets&#xA;    panel.border = element_blank()&#xA;  )&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There are more parts of the theme you can adjust to your liking, like the facet header (e.g. &lt;code&gt;strip.text&lt;/code&gt;).&#xA;See &lt;a href=&quot;https://ggplot2.tidyverse.org/reference/theme.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ggplot2's reference documentation&lt;/a&gt; for details.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2022-06-17T17:22:22.730" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19263" PostTypeId="1" CreationDate="2022-06-17T17:44:26.220" Score="1" ViewCount="2388" Body="&lt;p&gt;I have a bunch of folders containing barcodes.tsv.gz, features.tsv.gz, and matrix.mtx.gz from the Cellranger Count output for a single-cell dataset that was sent to me from another lab. I need the .h5 version of these count matrices, but I do not have access to the original Cellranger Count output which contains the generated .h5 files.&lt;/p&gt;&#xA;&lt;p&gt;Is there an R or Python function that can reliably convert these files into the proper hdf5 format? I haven't found anything online yet.&lt;/p&gt;&#xA;&lt;p&gt;(x-post: &lt;a href=&quot;https://www.reddit.com/r/bioinformatics/comments/veku5r/how_to_convert_10x_matrix_tsv_files_to_hdf5_format/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.reddit.com/r/bioinformatics/comments/veku5r/how_to_convert_10x_matrix_tsv_files_to_hdf5_format/&lt;/a&gt;)&lt;/p&gt;&#xA;" OwnerUserId="15309" LastActivityDate="2024-03-09T07:04:50.647" Title="How to convert 10x matrix.mtx.gz files to hdf5 format?" Tags="|single-cell|cellranger|hdf5|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19264" PostTypeId="2" ParentId="5296" CreationDate="2022-06-17T17:58:28.467" Score="1" Body="&lt;p&gt;Looks like a couple options have become available:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Minnow&lt;/strong&gt;: &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/35/14/i136/5529127?login=true&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; &lt;a href=&quot;https://github.com/COMBINE-lab/minnow&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;scReadSim&lt;/strong&gt;: &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2022.05.29.493924v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;preprint&lt;/a&gt; &lt;a href=&quot;https://github.com/JSB-UCLA/scReadSim&quot; rel=&quot;nofollow noreferrer&quot;&gt;code&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="900" LastEditorUserId="900" LastEditDate="2022-06-17T18:04:18.287" LastActivityDate="2022-06-17T18:04:18.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19265" PostTypeId="2" ParentId="19263" CreationDate="2022-06-17T18:43:15.243" Score="0" Body="&lt;p&gt;Answered on biostars (&lt;a href=&quot;https://www.biostars.org/p/9527636/#9527638&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/9527636/#9527638&lt;/a&gt;) as well as on reddit: (&lt;a href=&quot;https://www.reddit.com/r/bioinformatics/comments/veku5r/how_to_convert_10x_matrix_tsv_files_to_hdf5_format/icqsfwn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.reddit.com/r/bioinformatics/comments/veku5r/how_to_convert_10x_matrix_tsv_files_to_hdf5_format/icqsfwn/&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;The biostars answer is better as it needs just one package (DropletUtils) as opposed to the 2-3 required by the Seurat-allied packages suggested on reddit. Plus, Seurat can take forever to load a dataset.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2022-06-17T18:43:15.243" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19266" PostTypeId="2" ParentId="19261" CreationDate="2022-06-17T22:28:54.540" Score="3" Body="&lt;p&gt;To parse data into &lt;code&gt;pairwise2&lt;/code&gt; so you can follow the SO post &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/19261/how-to-align-integer-non-dna-protein-sequences&quot;&gt;here&lt;/a&gt; fairly easy by simply converting your output values into a higher base.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from numpy import base_repr&#xA;base36 = []&#xA;for count in range(1,20):&#xA;    base36.append( base_repr(count, 36))&#xA;myseq36i = str(''.join(base36))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;Output&lt;/em&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;123456789ABCDEFGHIJ&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This represents 1 to 20 as single digits (of course there's 36 single digit integers) and then implement the pairwise2 solution. The advantage of this solution is you're not stuck on 20 amino acids. &lt;code&gt;str&lt;/code&gt; may not be needed.&lt;/p&gt;&#xA;&lt;p&gt;The custom matrix described in that post you would supply can be based around the exact base 10 differences under the scoring matrix - to avoid subtracting in the higher base.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Please do upvote the answer on SO as it is relevant to your problem. The first solution they presented is cool and am sure there's broader application.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-06-18T01:03:25.100" LastActivityDate="2022-06-18T01:03:25.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19267" PostTypeId="2" ParentId="19263" CreationDate="2022-06-17T22:42:42.497" Score="0" Body="&lt;p&gt;Conversion into HDF5 from Python3 is straight forward, if this is a simple key: value dictionary format then it &lt;em&gt;should&lt;/em&gt; be easy to mimic the CellRanger format. If it is a more complex data structure that would be tricky.  The key to HDF5 is 'the HDF5 key' which you need to find (you would call it the directory)&lt;/p&gt;&#xA;&lt;p&gt;To find the 'key' one approach is do is download &lt;a href=&quot;https://www.hdfgroup.org/downloads/hdfview/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.hdfgroup.org/downloads/hdfview/&lt;/a&gt; and then forward a screenshot of the CellRanger HDF5 file here. The directories are what HDF5 call 'keys' and you need these to write to and call from HDF5. There is a CLI for this operation.&lt;/p&gt;&#xA;&lt;p&gt;An example of the data you are writing to would be needed.If per chance the data is in a Python dataframe you're in business, because these easily IO to HDF5&lt;/p&gt;&#xA;&lt;p&gt;All you do is put the data into a simple dictionary and assign the dictionary to a key which is written to HDF5. Parsing the data from e.g. an AlignIO/SeqIO object to the dictionary is more complicated. When the HDF5 is read, the key has to be supplied then the dictionary is extracted and assembled back into an object.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-06-18T02:52:06.687" LastActivityDate="2022-06-18T02:52:06.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19268" PostTypeId="2" ParentId="19197" CreationDate="2022-06-18T03:53:13.163" Score="0" Body="&lt;p&gt;Here's &lt;a href=&quot;https://drive.google.com/file/d/1Lwh6ofxi_bb92mB0HoRrMJObE1MmtcMi/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://drive.google.com/file/d/1Lwh6ofxi_bb92mB0HoRrMJObE1MmtcMi/view?usp=sharing&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is from the book of &lt;code&gt;Methods of mathematical oncology&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/ncbi/sra-tools&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ncbi/sra-tools&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;Suzuki, T., Poignard, C., Chaplain, M., &amp;amp; Quaranta, V. (2021). Methods of mathematical oncology : Fusion of mathematics and biology, osaka, japan, october 26-28, 2020 (1st ed. 2021 ed.). Singapore: Springer Nature Singapore : Imprint: Springer.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15657" LastEditorUserId="15657" LastEditDate="2022-06-18T03:57:49.483" LastActivityDate="2022-06-18T03:57:49.483" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19269" PostTypeId="2" ParentId="19255" CreationDate="2022-06-18T07:53:57.830" Score="3" Body="&lt;p&gt;The sequencing channels measure electric current, i.e. the flow of charged particles through the nanopore. The circuits are not measuring the bases directly; they're measuring the way the flow of ions is changed when different things are introduced into the pores.&lt;/p&gt;&#xA;&lt;p&gt;Older R9 pores had a length that could fit about five bases in the pore at any one time, the newer R10 pores have a double-size pore that fits about ten bases. Any of these bases can potentially influence the flow of current, and that degree of influence is increased at the narrower pinch points of the Nanopore. Inconsistent unwinding of double-stranded DNA (or hybrid DNA/RNA) can also affect the current profile over time, so there may also be some effects on the current that transfer from where the unwinding occurs, 20 or so bases prior to the pore introduction.&lt;/p&gt;&#xA;&lt;p&gt;The 5/6 kmer models are (or were, with R9) a reasonable sequencing model, because that's the number of bases that would fit in the pore, so those are the bases that have the greatest influence on the current.&lt;/p&gt;&#xA;&lt;p&gt;I'll explain using a bigger physical model of hydroelectricity:&lt;/p&gt;&#xA;&lt;p&gt;The mental model that I have in my head of a flow cell is a dam with a lake / reservoir, and holes in the dam that slowly drain the lake, spinning turbines that produce electricity (i.e. current). The interesting bits in the lake are the things going through the holes that aren’t water, but the water flow is the only thing that can be measured: there are no cameras attached to the holes, only current meters attached to the turbines.&lt;/p&gt;&#xA;&lt;p&gt;It just so happens that those non-water things have a strange enough shape that they interrupt the water flow in large and predictable ways, which means that observing the water flow can be used as a good proxy for observing the things themselves.&lt;/p&gt;&#xA;&lt;p&gt;If the holes get blocked, nothing can move through the holes, so there is no current flow: sequencing cannot happen.&lt;/p&gt;&#xA;&lt;p&gt;Eventually that water reservoir will get depleted. If the reservoir is fully drained [ionic balance of the carrier ions in the flow cell is equalised], then it doesn’t matter how clean the holes are; the dam will produce no more electricity [sequencing cannot happen].&lt;/p&gt;&#xA;&lt;p&gt;As an aside, this model helps me to understand why keeping the pores occupied with long DNA prolongs the sequencing life of the flow cell. The flow of carrier ions is reduced when a pore is actively sequencing, so the carrier ion reservoir drains slower, and can keep sequencing for longer.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-06-18T07:59:17.717" LastActivityDate="2022-06-18T07:59:17.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19270" PostTypeId="1" AcceptedAnswerId="19272" CreationDate="2022-06-18T15:38:41.057" Score="2" ViewCount="46" Body="&lt;p&gt;I am trying to do somatic mutation detection via the open source software, UNCeqR&lt;/p&gt;&#xA;&lt;p&gt;I got following error was detected on mac z shell during installation.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BEGIN failed--compilation aborted at test.pl line 11.&#xA;not ok 1&#xA;make: *** [test_dynamic] Error 2&#xA;  CALLAHAN/Math-CDF-0.1.tar.gz&#xA;  /usr/bin/make test -- NOT OK&#xA;//hint// to see the cpan-testers results for installing this module, try:&#xA;  reports CALLAHAN/Math-CDF-0.1.tar.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to ask for any advice to proceed.&lt;/p&gt;&#xA;" OwnerUserId="15657" LastEditorUserId="2108" LastEditDate="2022-06-18T16:05:57.300" LastActivityDate="2022-06-20T06:41:51.907" Title="the following errors are found during the installation of an open source software, UNCeqR on z shell" Tags="|software-installation|errors|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19271" PostTypeId="1" CreationDate="2022-06-19T08:01:16.483" Score="3" ViewCount="1335" Body="&lt;p&gt;I have sam and bam files for the chimeric reads, which come from two different parts of the genome (For example, the first half of the read from part of Chromosome 1 and the second half of the read from part of Chromosome 3). I have removed the low mapping quality reads (MAPQ &amp;lt;= 30) and reads with high mismatches. This might result in some unpaired reads which need to be removed before further analysis. May I ask whether there is any way to remove the unpaired reads in sam/bam files? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="15643" LastEditorUserId="15643" LastEditDate="2022-06-19T14:35:49.430" LastActivityDate="2022-06-20T13:43:49.460" Title="How to remove the unpaired reads in sam/bam files?" Tags="|sequence-alignment|samtools|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19272" PostTypeId="2" ParentId="19270" CreationDate="2022-06-20T05:11:38.603" Score="2" Body="&lt;p&gt;It is a Perl compilation error:&lt;/p&gt;&#xA;&lt;p&gt;(Perl libraries) Math::CDF-0.1 -- Generate probabilities and quantiles from several&#xA;statistical probability functions&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;conda install -c bioconda perl-math-cdf&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;cpan INSTALL Math::CDF&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This module provides cumulative probabilities [P(X &amp;lt;= x)] and&#xA;quantiles [given p, returns x such that P(X &amp;lt;= x) = p] from several&#xA;statistical probability distributions.&lt;/p&gt;&#xA;&lt;p&gt;This package provides a perl interface to DCDFLIB&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://metacpan.org/release/CALLAHAN/Math-CDF-0.1/source/README&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://metacpan.org/release/CALLAHAN/Math-CDF-0.1/source/README&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;make&lt;/code&gt; and &lt;code&gt;make test&lt;/code&gt; of Perl libraries Math::CDF-0.1 in the installation are not okay&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;https://drive.google.com/file/d/1Bd8TklsUPMqDRDn7zImGEXI9Nk84vkfU/view?usp=sharing&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="15519" LastEditorUserId="15519" LastEditDate="2022-06-20T06:41:51.907" LastActivityDate="2022-06-20T06:41:51.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19273" PostTypeId="2" ParentId="19197" CreationDate="2022-06-20T13:43:35.567" Score="2" Body="&lt;p&gt;I like &lt;a href=&quot;https://doi.org/10.1093/bioinformatics/btv425&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gierlinksi, &lt;em&gt;et al.&lt;/em&gt;, Bioinformatics, 2015&lt;/a&gt; for this type of problem.&lt;/p&gt;&#xA;&lt;p&gt;I like this paper because the methods are descriptive and clear, they have lots of figures that you can reproduce along the way, they spend a lot of time discussing &amp;quot;normalization&amp;quot; and quality control, and the data is easy to download.&lt;/p&gt;&#xA;&lt;p&gt;Even better, the authors spent a lot of time making sure all the samples were split across lanes and flowcells in a consistent way so that both biological and technical differences could be robustly identified.&#xA;They also compared a few different methods published at the time for performing differential expression analysis, so you can see how you can process the same data with different statistical models.&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, they don't provide the explicit code they run for this paper, so you can't look at their code as a reference to guide you.&#xA;But the methods are descriptive enough that I don't think there should be too much difficulty.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2022-06-20T13:43:35.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19274" PostTypeId="2" ParentId="19271" CreationDate="2022-06-20T13:43:49.460" Score="1" Body="&lt;p&gt;The &lt;code&gt;-f&lt;/code&gt; option of &lt;code&gt;samtools view&lt;/code&gt; is for &lt;a href=&quot;http://www.htslib.org/doc/1.11/samtools-flags.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;flags&lt;/a&gt; and can be used to filter reads in bam/sam file matching certain criteria such as properly paired reads (&lt;code&gt;0x2&lt;/code&gt;) :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -f 0x2 -b in.bam &amp;gt; out.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6273" LastActivityDate="2022-06-20T13:43:49.460" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19276" PostTypeId="1" CreationDate="2022-06-20T18:15:30.183" Score="2" ViewCount="226" Body="&lt;p&gt;&lt;em&gt;this question has been asked [and answered] on &lt;a href=&quot;https://stackoverflow.com/questions/72690191/bwa-mem-and-sambamba-read-group-line-error&quot;&gt;Stack Overflow&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is a two-part question:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;help interpreting an error;&lt;/li&gt;&#xA;&lt;li&gt;help with coding.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I'm trying to run &lt;code&gt;bwa-mem&lt;/code&gt; and &lt;code&gt;sambamba&lt;/code&gt; to align raw reads to a reference genome and to sort by position. These are the commands I'm using:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;bwa mem  \&#xA;  -K 100000000 -v 3 -t 6 -Y \&#xA;  -R '\@S200031047L1C001R002\S*[1-2]' \&#xA;  /path/to/reference/GCF_009858895.2_ASM985889v3_genomic.fna \&#xA;  /path/to/raw-fastq/'S\S[^_]*_L01_[0-9]+-[0-9]+'_1.fq.gz \&#xA;  /path/to/raw-fastq/'S\S[^_]*_L01_[0-9]+-[0-9]+'_2.fq.gz | \&#xA;/path/to/genomics/sambamba-0.8.2 view -S -f bam \&#xA;  /dev/stdin | \&#xA;/path/to/genomics/sambamba-0.8.2 sort  \&#xA;  /dev/stdin \&#xA;  --out host_removal/${SAMPLE}/${SAMPLE}.hybrid.sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the error message I'm getting: &lt;code&gt;[E::bwa_set_rg] the read group line is not started with @RG&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My sequences were generated with an MGI sequencer and the readgroups are identified like this &lt;code&gt;@S200031047L1C001R0020000243/1&lt;/code&gt;, i.e., they don't begin with an @RG. How can I specify to sambamba that my readgroups start with @S and not @RG?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;The commands written above are a published pipeline I'm modifying for my own research. However, among several changes, I'm not confident on how to define sample id as such stated in the last line of the code: &lt;code&gt;--out host_removal/${SAMPLE}/${SAMPLE}.hybrid.sorted.bam&lt;/code&gt; (I'm referring to ${SAMPLE}). Any insights?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Thank you very much!&lt;/p&gt;&#xA;" OwnerUserId="15665" LastEditorUserId="73" LastEditDate="2022-06-21T04:03:34.060" LastActivityDate="2022-06-21T09:41:47.143" Title="BWA-mem and sambamba read group line error" Tags="|sequence-alignment|ngs|genomics|bwa|sambamba|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19277" PostTypeId="2" ParentId="19276" CreationDate="2022-06-20T18:38:59.847" Score="3" Body="&lt;p&gt;Read groups must start with RG, that is a hardcoded requirement of the SAM format, nothing you can do about that.&lt;/p&gt;&#xA;&lt;p&gt;See &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://samtools.github.io/hts-specs/SAMv1.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15666" LastEditorUserId="15666" LastEditDate="2022-06-20T18:40:20.017" LastActivityDate="2022-06-20T18:40:20.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19278" PostTypeId="1" AcceptedAnswerId="19280" CreationDate="2022-06-20T22:36:19.200" Score="3" ViewCount="242" Body="&lt;p&gt;This &lt;a href=&quot;https://f1000research.com/articles/9-1444&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; talked about various deseq2 designs etc. One of the designs I would like to use is explained as this:&lt;/p&gt;&#xA;&lt;p&gt;Control versus treatment average&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;makeContrasts((treatmentI+treatmentII+treatmentIII)/3-treatmentCTL,&#xA;  levels=colnames(design))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm not sure how to incorporate this in deseq2 run which step?&lt;/p&gt;&#xA;&lt;p&gt;Is it after running this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeq(dds)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can I call the &lt;code&gt;makeContrasts&lt;/code&gt; function?&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="73" LastEditDate="2022-06-21T04:32:13.917" LastActivityDate="2022-06-21T09:39:12.213" Title="deseq2 makeContrasts function" Tags="|deseq2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19279" PostTypeId="2" ParentId="19261" CreationDate="2022-06-20T23:42:40.217" Score="4" Body="&lt;p&gt;The tricky part about your request is the size of the alphabet.  If you are happy with alphabets of up to about 250 &amp;quot;characters&amp;quot;, then the programs in the FASTA package (github.com/wrpearson/fasta36) can be modified to do what you need.  The are designed to work with unsigned char strings, and could work with matrices of up to 250 x 250 or so.&lt;/p&gt;&#xA;&lt;p&gt;There is a non-vectorized version of Smith-Waterman that can calculate optimal local alignments with alphabets extended to 250 characters (with the appropriate scoring matrix).&lt;/p&gt;&#xA;&lt;p&gt;However, I suspect that pairwise could do the same thing (with modifications), again using an expanded scoring matrix.  The affine-gap Smith-Waterman algorithm is pretty simple, the trick is to get your multi-digit &amp;quot;characters&amp;quot; into a single byte (or unsigned char).&lt;/p&gt;&#xA;" OwnerUserId="15667" LastActivityDate="2022-06-20T23:42:40.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19280" PostTypeId="2" ParentId="19278" CreationDate="2022-06-21T08:14:00.880" Score="3" Body="&lt;p&gt;The &lt;code&gt;makeContrasts&lt;/code&gt; function is from edgeR so it does not help here. If you want to compare one group vs the average of several others here is a suggestion of how this could be done. It uses a design without intercept so all groups are easily accessable as coefficients in &lt;code&gt;resultsNames(dds)&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;suppressMessages(library(DESeq2))&#xA;&#xA;#/ Example with four groups&#xA;set.seed(1)&#xA;dds &amp;lt;- makeExampleDESeqDataSet(m=12)&#xA;dds$condition &amp;lt;- factor(as.character(rep(LETTERS[1:4], each=3)))&#xA;&#xA;#/ Use a design without intercept so all four groups are accessable as coefficients&#xA;design(dds) &amp;lt;- ~0+condition&#xA;dds &amp;lt;- DESeq(dds)&#xA;#&amp;gt; estimating size factors&#xA;#&amp;gt; estimating dispersions&#xA;#&amp;gt; gene-wise dispersion estimates&#xA;#&amp;gt; mean-dispersion relationship&#xA;#&amp;gt; final dispersion estimates&#xA;#&amp;gt; fitting model and testing&#xA;&#xA;#/ results using listValues to get A vs the average ov B,C,D&#xA;res &amp;lt;- results(dds, contrast=list(c(&amp;quot;conditionA&amp;quot;), c(&amp;quot;conditionB&amp;quot;, &amp;quot;conditionC&amp;quot;, &amp;quot;conditionD&amp;quot;)), listValues=c(1, -1/4))&#xA;&#xA;Created on 2022-06-21 by the reprex package (v2.0.1)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Determining whether or not it is appropriate to do this in a particular situation is up to you. You have to know what question you want to answer for your work. I cannot help with that.&lt;/p&gt;&#xA;" OwnerUserId="15668" LastEditorUserId="73" LastEditDate="2022-06-21T09:39:12.213" LastActivityDate="2022-06-21T09:39:12.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19283" PostTypeId="2" ParentId="19276" CreationDate="2022-06-21T09:41:47.143" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;My sequences were generated with an MGI sequencer and the readgroups are identified like this &lt;code&gt;@S200031047L1C001R0020000243/1&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Those are &lt;a href=&quot;https://en.wikipedia.org/wiki/DNA_nanoball_sequencing#Read_Names&quot; rel=&quot;nofollow noreferrer&quot;&gt;MGI read &lt;strong&gt;names&lt;/strong&gt;&lt;/a&gt;, not read groups. (And the &lt;code&gt;@&lt;/code&gt; is the FASTQ syntax that introduces a read name in a FASTQ file, not part of the read name or qname itself.)&lt;/p&gt;&#xA;&lt;p&gt;Read &lt;strong&gt;groups&lt;/strong&gt; more or less correspond to the sample that was sequenced. If a sample was sequenced across multiple sequencing machine runs and the resulting data merged into one file, there might be several read groups corresponding to the individual runs all pertaining to the same sample — though with increasing data volumes from individual sequencing runs this is perhaps less common than it used to be.&lt;/p&gt;&#xA;&lt;p&gt;You get to make up the ids yourself, usually corresponding to what the sample is known as in your workflows. So your &lt;code&gt;-R&lt;/code&gt; option should be something like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-R '@RG\tID:S123\tSM:S123\tDS:Sample 123\tPL:DNBSEQ'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(BWA's &lt;code&gt;-R&lt;/code&gt; syntax is unfortunately not very easy to use. See &lt;a href=&quot;https://www.htslib.org/doc/samtools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;addreplacerg&lt;/a&gt; for a similar &lt;code&gt;-r&lt;/code&gt; option that is rather more user-friendly.)&lt;/p&gt;&#xA;&lt;p&gt;The name you use for &lt;code&gt;${SAMPLE}&lt;/code&gt; would usually be the same as the value of the &lt;code&gt;SM:&lt;/code&gt; field within that &lt;code&gt;-R&lt;/code&gt; option — the identifier that is used in your LIMS and workflows to identify that particular sample.&lt;/p&gt;&#xA;" OwnerUserId="134" LastActivityDate="2022-06-21T09:41:47.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19284" PostTypeId="2" ParentId="8781" CreationDate="2022-06-21T10:06:32.687" Score="1" Body="&lt;p&gt;It's very unlikely that &amp;quot;a RPKM analysis&amp;quot; is the right answer. Assuming you'd like to do differential expression, using tools like DESeq or EdgeR on the count table are likely to be a better thing to do.&lt;/p&gt;&#xA;&lt;p&gt;For reasons why RPKM is not a good approach, have a read of &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/69/29&quot;&gt;this answer&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-06-21T10:06:32.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19285" PostTypeId="2" ParentId="8972" CreationDate="2022-06-21T10:18:05.493" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @devon-ryan and @terdon, converted from comments:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Are you sure it's not just a made up chromosome name that's useful in examples since people won't blindly copy/paste the command and have it work?&lt;/p&gt;&#xA;&lt;p&gt;It's almost certainly just a place holder, as Devon suggests, but we could give a more definitive answer if you could link us to the places you've seen this used. If you only see it in relation to freebayes then yes the author is using it as “foo”.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;[please update / edit to improve this answer if you can]&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-06-21T10:18:05.493" CommentCount="0" CommunityOwnedDate="2022-06-21T10:18:05.493" ContentLicense="CC BY-SA 4.0" />
  <row Id="19286" PostTypeId="2" ParentId="9151" CreationDate="2022-06-21T10:29:44.570" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @benn, converted from comments:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can find a step by step description in the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/limma/inst/doc/usersguide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;limma user's guide&lt;/a&gt;. Just download the CEL files, and follow the user guide.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;[please edit to improve this answer, if possible]&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-06-21T10:29:44.570" CommentCount="0" CommunityOwnedDate="2022-06-21T10:29:44.570" ContentLicense="CC BY-SA 4.0" />
  <row Id="19287" PostTypeId="2" ParentId="9294" CreationDate="2022-06-21T10:36:55.190" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @stupidwolf, @benn, @llrs, converted from comments:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;The error complains about the 1686 rownames, while you have 1684 features. I think something is wrong with the naming of your rows.&lt;/p&gt;&#xA;&lt;p&gt;This may be a bug that you should report to the package maintainer. I cannot really tell without more information about your eset. Looking at this line &amp;quot;Cell type(s): 'fibroblast' (1 total)&amp;quot;, is it possible you only have one group under fibroblast? It will not make sense to run a random effect for only 1 group.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;[please edit to improve this answer, if possible]&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-06-21T10:36:55.190" CommentCount="0" CommunityOwnedDate="2022-06-21T10:36:55.190" ContentLicense="CC BY-SA 4.0" />
  <row Id="19288" PostTypeId="2" ParentId="19230" CreationDate="2022-06-21T10:37:16.067" Score="1" Body="&lt;p&gt;We can't be sure without seeing your file, but if what you really want is the text between the last parentheses on lines that start with &lt;code&gt;&amp;gt;&lt;/code&gt;, then you can do this with GNU &lt;code&gt;grep&lt;/code&gt; (the default on Linux) with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -oP '^&amp;gt;.*\(\K[^)]+' file.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;-o&lt;/code&gt; means &amp;quot;print only the matching portion of the line&amp;quot; and the &lt;code&gt;-P&lt;/code&gt; activates Perl Compatible Regular Expressions (PCREs). The regex looks for a &lt;code&gt;&amp;gt;&lt;/code&gt; at the beginning of the line (&lt;code&gt;^&amp;gt;&lt;/code&gt;), then as many characters as possible until the last parenthesis (&lt;code&gt;.*\(&lt;/code&gt;), and then forgets everything matched up to this point with the &lt;code&gt;\K&lt;/code&gt;. This means that the &lt;code&gt;-o&lt;/code&gt; will only see things &lt;em&gt;after&lt;/em&gt; the &lt;code&gt;\K&lt;/code&gt;, a PCRE feature. So finally, we look for the longest stretch of one or more non&lt;code&gt;-)&lt;/code&gt; characters.&lt;/p&gt;&#xA;&lt;p&gt;If you don't have GNU &lt;code&gt;grep&lt;/code&gt;, you can do the same basic thing with &lt;code&gt;perl&lt;/code&gt; or &lt;code&gt;sed&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sed -n 's/^&amp;gt;.*(\([^)]*\).*/\1/p' file.fa&#xA;perl -lne '/^&amp;gt;.*\(([^)]+)/ &amp;amp;&amp;amp; print $1' file.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2022-06-21T10:37:16.067" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19289" PostTypeId="2" ParentId="10441" CreationDate="2022-06-21T10:49:43.897" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @devon-ryan, converted from comments:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;They described how they did it in that paper, there's just not a tool for it. It's not part of anything. They describe the algorithm, maybe DGCA implemented it, maybe not, but they don't mention using it. WGCNA was obviously used to define the modules, though.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-06-21T10:49:43.897" CommentCount="0" CommunityOwnedDate="2022-06-21T10:49:43.897" ContentLicense="CC BY-SA 4.0" />
  <row Id="19290" PostTypeId="1" AcceptedAnswerId="19297" CreationDate="2022-06-21T13:13:20.170" Score="1" ViewCount="104" Body="&lt;p&gt;Here I have two designs:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Design 1&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; dds &amp;lt;- DESeqDataSetFromMatrix(countData=rpkm_ordered,&#xA;           colData=coldata, design= ~ Sex + Age + TMB + WBC + BM_percentage + FAB) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To see the results I did this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;resultsNames(dds)&#xA; [1] &amp;quot;Intercept&amp;quot;          &amp;quot;Sex_Male_vs_Female&amp;quot; &amp;quot;Age&amp;quot;                &amp;quot;TMB&amp;quot;                &amp;quot;WBC&amp;quot;                &amp;quot;BM_percentage&amp;quot;      &amp;quot;FAB_M1_vs_M0&amp;quot;      &#xA; [8] &amp;quot;FAB_M2_vs_M0&amp;quot;       &amp;quot;FAB_M3_vs_M0&amp;quot;       &amp;quot;FAB_M4_vs_M0&amp;quot;       &amp;quot;FAB_M5_vs_M0&amp;quot;  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I extracted only the Age from the first design&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Design 2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I again used the design using &lt;code&gt;Age&lt;/code&gt; variable :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData=rpkm_ordered,&#xA;          colData=coldata, design= ~ Age)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As an example I take this gene this output from the first design&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  gene            baseMean log2FoldChange  lfcSE  stat   pvalue       padj&#xA;  &amp;lt;chr&amp;gt;              &amp;lt;dbl&amp;gt;          &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;    &amp;lt;dbl&amp;gt;      &amp;lt;dbl&amp;gt;&#xA;1 ENSG00000196188     97.4         -0.742  0.113 -6.58 4.57e-11 0.00000125&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This from the second design:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  gene            baseMean log2FoldChange  lfcSE  stat   pvalue       padj&#xA;  &amp;lt;chr&amp;gt;              &amp;lt;dbl&amp;gt;          &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;    &amp;lt;dbl&amp;gt;      &amp;lt;dbl&amp;gt;&#xA;  ENSG00000196188     97.4         -0.757  0.119 -6.34 2.28e-10   3.43e-07&#xA;        &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Another example this gene from the first design; the result is not significant:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;           gene  baseMean log2FoldChange  lfcSE    stat pvalue   padj&#xA;ENSG00000173826  74.54467     0.01283834  0.589  0.0218  0.983  0.993&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The same gene from &lt;strong&gt;Design 2&lt;/strong&gt; &lt;em&gt;is&lt;/em&gt; significant:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;           gene  baseMean log2FoldChange  lfcSE    stat   pvalue     padj&#xA;ENSG00000173826 74.54467       -2.004743  0.190   -10.6 4.76e-26 1.35e-21&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is this drastic outcome is due to the maths behind it or am I doing something wrong?&lt;/p&gt;&#xA;&lt;p&gt;So would my interpretation be as such when i consider Age as my only variable I can consider &lt;code&gt;ENSG00000173826&lt;/code&gt;significant ? But it is not significant when my main effect which i want to test is FAB which is my design 1.&lt;/p&gt;&#xA;&lt;p&gt;I would like to know how to interpret and report the same.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion or help would be really really appreciated&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="73" LastEditDate="2022-06-23T04:12:53.837" LastActivityDate="2022-06-25T03:37:25.517" Title="deseq2 model design : Different gene output" Tags="|deseq2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19291" PostTypeId="1" AcceptedAnswerId="19292" CreationDate="2022-06-21T13:48:24.787" Score="4" ViewCount="546" Body="&lt;p&gt;I have a set 189 taste receptor protein sequences (not aligned) in a fasta file. I like to get the phylogenetic tree in newick format. I was using earlier &lt;a href=&quot;https://ngphylogeny.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ngphylogeny.fr/&lt;/a&gt;. Unfortunately now it is not working due to some bugs. Necessarily, I just need newick information for the input fasta file.&lt;br /&gt;&#xA;Any help would be highly appreciated.&lt;/p&gt;&#xA;&lt;p&gt;Here is one newick example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;((raccoon:19.19959,bear:6.80041):0.84600,&#xA;  ((sea_lion:11.99700, seal:12.00300):7.52973,  &#xA;     ((monkey:100.85930,cat:47.14069):20.59201,&#xA;       weasel:18.87953):2.09460):3.87382,&#xA;  dog:25.46154);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15670" OwnerDisplayName="SK SARIF HASSAN" LastEditorUserId="73" LastEditDate="2022-06-23T04:28:30.707" LastActivityDate="2022-06-23T04:28:30.707" Title="Construct phylogeny from a Fasta file" Tags="|phylogenetics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19292" PostTypeId="2" ParentId="19291" CreationDate="2022-06-21T15:39:38.153" Score="5" Body="&lt;p&gt;You will first need to align your sequences, and then feed the alignment to a program that can generate trees. I am very much out of date in this field and I haven't done anything like that in at least ten years, but back when I used to do it, my workflow was:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Align the sequences using &lt;a href=&quot;https://mafft.cbrc.jp/alignment/software/&quot; rel=&quot;noreferrer&quot;&gt;MAFFT&lt;/a&gt; or &lt;a href=&quot;https://tcoffee.crg.eu/&quot; rel=&quot;noreferrer&quot;&gt;t-coffee&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Open the alignment in the &lt;a href=&quot;https://www.jalview.org/&quot; rel=&quot;noreferrer&quot;&gt;Jalview alignment editor/viewer&lt;/a&gt; which has options to generate the tree:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UD3s3.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UD3s3.png&quot; alt=&quot;screenshot of jalview showing the &amp;quot;calculate tree&amp;quot; option&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;then just save the tree in Newick format:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1ZJBn.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1ZJBn.png&quot; alt=&quot;screenshot of jalview showing the file -&amp;gt; save as Newick format option&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The screenshots were taken using the default alignment provided by JalView itself, and this is the tree I got when I saved as Newick:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat tree &#xA;((FER1_MESCR:24.484375,(((O80429_MAIZE:23.699999999999996,FER1_MAIZE:17.300000000000004):21.240625,(FER2_ARATH:13.067708333333336,((Q93Z60_ARATH:8.0,FER1_ARATH:8.0):8.401785714285715,(FER_BRANA:14.796875,FER3_RAPSA:8.203125):1.5982142857142847):0.9322916666666643):18.259375):9.75390625,((Q7XA98_TRIPR:24.8515625,FER1_PEA:27.1484375):6.643229166666671,((Q93XJ9_SOLTU:10.125,FER1_SOLLC:8.875):5.61363636363636,(FER_CAPAN:5.0,FER_CAPAA:0.0):19.38636363636364):20.48177083333333):8.97265625):9.52734375):20.2578125,FER1_SPIOL:20.2578125);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2022-06-21T15:39:38.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19293" PostTypeId="2" ParentId="19291" CreationDate="2022-06-21T15:53:59.817" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oXHWv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oXHWv.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here's the result from &lt;a href=&quot;http://tree.bio.ed.ac.uk/software/figtree/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FigTree&lt;/a&gt;. I thought there was a bug in the newick file, but it was fine.&lt;/p&gt;&#xA;&lt;p&gt;I strongly recommend the answer by @terdon because there are other issues here, for example &lt;code&gt;111.16&lt;/code&gt; value for the root and the &lt;code&gt;100.86&lt;/code&gt; which occurs twice isn't cool.&lt;/p&gt;&#xA;&lt;p&gt;I should have placed &lt;code&gt;monkey&lt;/code&gt; as the outgroup, anyway just please ensure you do this for the report.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="2108" LastEditDate="2022-06-21T18:25:52.133" LastActivityDate="2022-06-21T18:25:52.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19294" PostTypeId="2" ParentId="19247" CreationDate="2022-06-21T20:06:03.050" Score="2" Body="&lt;p&gt;I considered mentioning this but considered it implausible. However, apparently people are actually &lt;a href=&quot;https://academic.oup.com/bioinformatics/advance-article-abstract/doi/10.1093/bioinformatics/btac398/6609767&quot; rel=&quot;nofollow noreferrer&quot;&gt;able to use PE shotgun data to link rRNA genes to MAGs&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This is something that is quite trivial with linked-read or Hi-C data, but I am assuming that you don't have these data (and don't have the desire to go back and generate them). However you presumably have PE shotgun data that you used to generate your MAGs.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-06-21T20:06:03.050" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19295" PostTypeId="1" CreationDate="2022-06-22T09:28:40.943" Score="1" ViewCount="3560" Body="&lt;p&gt;I'm new to SQL. I have a table with a column of &lt;strong&gt;numeric&lt;/strong&gt;  type that include NULL values. I built the table structure in postgresql, when I try to import my table I get the following error:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;ERROR: invalid input syntax for type numeric: NULL&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I tried both&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;UPDATE MY_TABLE  SET MY_COLUMN = CAST(NULL AS NUMERIC)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;and setting manually the default value of the column as NULL, but none of them are working.&#xA;Is there another way to handle this problem?&lt;/p&gt;&#xA;" OwnerUserId="15673" LastActivityDate="2022-06-22T09:28:40.943" Title="postgreSQL error: &quot;ERROR: invalid input syntax for type numeric: NULL&quot;" Tags="|database|" AnswerCount="0" CommentCount="3" ClosedDate="2022-06-22T16:50:24.193" ContentLicense="CC BY-SA 4.0" />
  <row Id="19296" PostTypeId="1" AcceptedAnswerId="19301" CreationDate="2022-06-22T22:12:07.373" Score="3" ViewCount="214" Body="&lt;p&gt;I am trying to make a prediction application using the sequence of proteins.&lt;/p&gt;&#xA;&lt;p&gt;If we think about the 6COU protein, I'm running the code below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pdb = PDBParser().get_structure(&amp;quot;6c0u&amp;quot;, &amp;quot;6c0u.pdb&amp;quot;)&#xA;&#xA;for residue in pdb.get_residues():&#xA;    print(residue) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Some of the output looks like below when looking at residues. What is the function of the heteroatoms here, and follow up question;&lt;/p&gt;&#xA;&lt;p&gt;When reading a paper on Binding Site Detection of Proteins, there is exactly one sentence in the paper.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;&amp;quot;We first clean the input structure by removing all heteroatoms and solvent molecules from the protein structure using the Biopython47 library.&amp;quot;&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;What is the meaning of this sentence if you are working on protein sequences, heteroatoms are not already there, what does it mean to delete them?&lt;/p&gt;&#xA;&lt;p&gt;If you want to check the paper, &lt;a href=&quot;https://doi.org/10.1021/acs.jcim.1c00799&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0Cr40.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0Cr40.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15525" LastActivityDate="2022-12-12T11:42:27.427" Title="What is the function of the heteroatoms" Tags="|proteins|python|protein-structure|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19297" PostTypeId="2" ParentId="19290" CreationDate="2022-06-23T04:24:27.627" Score="2" Body="&lt;p&gt;It's not surprising to me that the adjusted fold change statistic changes in different ways for different genes when covariates are added or removed. That's sort of the point of using covariates, as long as it makes sense to include them (which is an experimental decision, not a mathematical or statistical decision).&lt;/p&gt;&#xA;&lt;p&gt;One thing I'd recommend checking is to make sure the results you're looking at are what you &lt;em&gt;think&lt;/em&gt; you are looking at. You say that you're interested in testing &lt;code&gt;FAB&lt;/code&gt;, but that is not included in the second (Age-only) design, which means that it won't be possible to extract any &lt;code&gt;FAB&lt;/code&gt; comparisons from the results. It looks like your first design might be looking at &lt;code&gt;Sex&lt;/code&gt; (based on the resultsNames ordering) or one of the &lt;code&gt;FAB&lt;/code&gt; comparisons (based on reverse ordering of variables in the model), whereas the second might be looking at &lt;code&gt;Age&lt;/code&gt;. This should be explicitly stated when displaying the results, but you haven't included that information in your output.&lt;/p&gt;&#xA;&lt;p&gt;If you want &lt;code&gt;FAB&lt;/code&gt; comparisons in the results, that variable needs to be in the specified model, e.g. &amp;quot;design = ~ Age + FAB&amp;quot;. The Design 2 model that you have in your question only includes &lt;code&gt;Age&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;A good way to do this without getting the wrong contrast is to fully specify the variables when fetching results, as explained in the &lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#multi-factor-designs&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2 vignette&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In any case, the &lt;code&gt;contrast&lt;/code&gt; argument of the function results takes a character vector of length three: the name of the variable, the name of the factor level for the numerator of the log2 ratio, and the name of the factor level for the denominator. The contrast argument can also take other forms, as described in the help page for results and below&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;res.age &amp;lt;- results(dds, name=&amp;quot;Age&amp;quot;)&#xA;res.FAB.M3.M0 &amp;lt;- results(dds, contrast=c(&amp;quot;FAB&amp;quot;,&amp;quot;M3&amp;quot;,&amp;quot;M0&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-06-25T03:37:25.517" LastActivityDate="2022-06-25T03:37:25.517" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="19298" PostTypeId="1" AcceptedAnswerId="19299" CreationDate="2022-06-23T06:34:42.357" Score="2" ViewCount="205" Body="&lt;p&gt;I have a BAM file (DNA, shallow whole genome sequencing at ~1X) where I want to filter reads (using sambamba) to keep only those which have a template length &amp;gt; 20 and mapping quality &amp;gt; 20, discarding unmapped reads.&lt;/p&gt;&#xA;&lt;p&gt;The filter I have used is this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sambamba view -h \&#xA;   -t 5 \&#xA;   -F &amp;quot;paired and template_length &amp;gt; 0 and mapping_quality &amp;gt; 20 and not (unmapped or mate_is_unmapped)&amp;quot; \&#xA;   -f bam 142-ready.bam \&#xA;   -o 142-filter.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, inspection of the reads in IGV led me to think that perhaps reads on the minus strand are being somehow filtered out.&lt;/p&gt;&#xA;&lt;p&gt;Pre filter:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/N2Ynp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/N2Ynp.png&quot; alt=&quot;Unfiltered view in IGV with mate pairs highlighted&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Post filter:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/peGxL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/peGxL.png&quot; alt=&quot;Filtered view in IGV, showing only one mate pair&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The only way I don't get such an aggressive filtering is by setting only &lt;code&gt;not (unmapped or mate_is_unmapped)&lt;/code&gt;. But that, obviously, defeats the purpose as it doesn't include what I need.&lt;/p&gt;&#xA;&lt;p&gt;I can't, however, see what exactly is wrong with the filter. What is the correct way to filter for template length while not accidentally discarding reads on the minus strand?&lt;/p&gt;&#xA;" OwnerUserId="6814" LastEditorUserId="6814" LastEditDate="2022-06-23T12:15:41.363" LastActivityDate="2022-06-23T12:15:41.363" Title="Filtering paired-end reads with sambamba: avoid discarding reads on the minus strand" Tags="|sam|filtering|reads|sambamba|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19299" PostTypeId="2" ParentId="19298" CreationDate="2022-06-23T11:52:05.837" Score="1" Body="&lt;p&gt;The reason is that &lt;code&gt;sambamba&lt;/code&gt; does not use the absolute value of the template length, so, if the read maps so the minus strand, the template length is actually negative. Setting a template length &amp;gt; 0 means that all reads mapping to the minus strand are removed.&lt;/p&gt;&#xA;&lt;p&gt;In fact, you can't still filter for negative template lengths. See &lt;a href=&quot;https://github.com/biod/sambamba/pull/479&quot; rel=&quot;nofollow noreferrer&quot;&gt;this GitHub PR&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6814" LastActivityDate="2022-06-23T11:52:05.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19300" PostTypeId="1" CreationDate="2022-06-23T13:37:28.820" Score="2" ViewCount="630" Body="&lt;p&gt;I am trying to understand the sanger sequencing ABI/AB1 file format better, and extract base calls from given signal intensities over time.&lt;/p&gt;&#xA;&lt;p&gt;As I understand, reading in a raw AB1/ABI file into python, I can access the channel corrected values for the different bases in ['DATA9'] to ['DATA12'], and understand which bases belong to a specific channel through ['FWO_1'] i.e either ordered GATC or ATGC etc.&lt;/p&gt;&#xA;&lt;p&gt;However, the file I am looking at has identified 412 bases as the output from SnapGene, yet has 4950 data records for each channel in the raw ['DATA9'] to ['DATA12']. How do you convert those 4950 data records into the bases correctly? Is there some form of normalisation - e.g every 10 raw records gives data for 1 base position, or peak calling signal detection algorithms. And if so, do you take the average over 10 records, or the highest peak in the 10 records from the channels and the channel with the highest peak/average is the correct base? Do you start at the beginning or do you have some form of cutoff? Does it make sense to convert this figure into a quality score similar to NGS, and do you use the highest peaks in the 10 record interval or the average?&lt;/p&gt;&#xA;&lt;p&gt;I've tried using scipy's signal.find_peaks() function (python) which can get me various peaks with a minimum height and a minimum distance to other peaks, currently using a minimum peak height of 150 and distance to other peaks of 5, but still getting around 20-30 more peaks than the bases identified.&lt;/p&gt;&#xA;&lt;p&gt;Could anyone suggest any logic, methods, algorithms or resources that could help with translating the raw signal intensities for the 4 channels into a sequence of accurate base calls?&lt;/p&gt;&#xA;&lt;p&gt;I know software exists to identify the bases, but that's not what I am looking for. I intend to do this from scratch to understand the workings better so I can write some custom processing algorithms.&lt;/p&gt;&#xA;&lt;p&gt;I hope this question makes sense.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="3608" LastActivityDate="2023-05-30T21:02:30.347" Title="Extracting base sequences from ABI/AB1 sanger sequencing chromatogram" Tags="|python|ngs|genomics|genetics|base-calling|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19301" PostTypeId="2" ParentId="19296" CreationDate="2022-06-23T14:09:02.853" Score="6" Body="&lt;p&gt;The text you cited mentions &amp;quot;input structure&amp;quot;, so they work with structures, not just &amp;quot;protein sequences&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Heteroatoms = atoms marked as &lt;code&gt;HETATM&lt;/code&gt; in the PDB and mmCIF formats.&#xA;In these formats each atom is listed as either &lt;code&gt;ATOM&lt;/code&gt; or &lt;code&gt;HETATM&lt;/code&gt;. These are not particularly meaningful categories, but they are there, so they are easy to use. It's a common practice to remove heteroatoms with &lt;code&gt;grep -v ^HETATM&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Note that in the files from the Protein Data Bank not only solvent and ligands are marked as HETATM, but also &amp;quot;non-standard&amp;quot; amino-acids in a protein. An amino acid is &lt;a href=&quot;https://proteopedia.org/wiki/index.php/Standard_Residues&quot; rel=&quot;nofollow noreferrer&quot;&gt;considered standard&lt;/a&gt; if it can be genetically encoded. So in the files from the PDB selenocysteine is ATOM and selenomethionine is HETATM. Some programs don't follow these rules and output all protein residues as ATOM.&lt;/p&gt;&#xA;&lt;p&gt;Removing heteroatoms works well for the vast majority of structures that have no modified residues. But in some cases it will remove also some protein residues, which may not intended.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;: if you are wondering &lt;strong&gt;why&lt;/strong&gt; there are ATOM and HETATM records, I have never came across the rationale. The &lt;a href=&quot;https://www.wwpdb.org/documentation/file-format&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB format&#xA;specification&lt;/a&gt; has had both types of records since 1976.&#xA;Different versions of the format spec use different wording,&#xA;but the meaning is always similar:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ATOM records give atomic coordinates for &amp;quot;standard&amp;quot; groups.&lt;/li&gt;&#xA;&lt;li&gt;HETATM records give atomic coordinates for &amp;quot;non-standard&amp;quot; groups.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;(Although standard residues that not in a polymer are also HETATM.)&lt;/p&gt;&#xA;&lt;p&gt;In the mmCIF format the same record is preserved in &lt;a href=&quot;https://mmcif.wwpdb.org/dictionaries/mmcif_pdbx_v50.dic/Items/_atom_site.group_PDB.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;_atom_site.group_PDB&lt;/a&gt; with the description:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This data item is provided for compatibility with the original Protein Data Bank format, and only for that purpose.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Most of the structures have only standard residues in protein/DNA/RNA polymers, so removing HETATM records removes only ligands and solvent.&lt;/p&gt;&#xA;" OwnerUserId="266" LastEditorUserId="266" LastEditDate="2022-12-12T11:42:27.427" LastActivityDate="2022-12-12T11:42:27.427" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19302" PostTypeId="1" CreationDate="2022-06-23T14:19:35.857" Score="0" ViewCount="23" Body="&lt;p&gt;I want to check virulence factor of bacteria protein sequence by using Virulence Factor Database and it shows No hit in VFDB BLAST search output. Will someone please explain what is meant by it?&lt;/p&gt;&#xA;" OwnerUserId="15679" LastActivityDate="2022-06-23T14:19:35.857" Title="VFDB search output" Tags="|pathogenicity|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19303" PostTypeId="1" AcceptedAnswerId="19321" CreationDate="2022-06-24T09:11:34.480" Score="1" ViewCount="73" Body="&lt;p&gt;This is regarding the single factor design For example if I have Age or other continuous numerical variable how to provide that into the design formula.&lt;/p&gt;&#xA;&lt;p&gt;For this &lt;a href=&quot;https://support.bioconductor.org/p/130612/#9145075&quot; rel=&quot;nofollow noreferrer&quot;&gt;post&lt;/a&gt; do i need to 'You could dichotomise your continuous variables into meaningful groups' or it can go without it?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;My simple design &#xA;&#xA;&#xA;dds &amp;lt;- DESeqDataSetFromMatrix(countData=rpkm_ordered, colData=coldata, design= ~ Age)&#xA;&#xA;dds &amp;lt;- estimateSizeFactors(dds)&#xA;&#xA;dds &amp;lt;- estimateDispersions(dds)&#xA;&#xA;dds  &amp;lt;- DESeq(dds, parallel = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here in case of metadata/coldata I m giving a single numerical value which is Blast percentage. In case of sample I have like 5 sub-types from M0 to  M5.&lt;/p&gt;&#xA;&lt;p&gt;Now for the interpretation part How do I interpret the result?&lt;/p&gt;&#xA;&lt;p&gt;Would it be as such&lt;/p&gt;&#xA;&lt;p&gt;the expression differences between the my subtypes due to 'Age'&lt;/p&gt;&#xA;&lt;p&gt;I'm bit confused since in my coldata I'm not providing any information regarding my subtypes. So when I run this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;resultsNames(dds)  I get this &#xA;&#xA;&#xA;[1] &amp;quot;Intercept&amp;quot;     &amp;quot;Age&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So if I would like to know if there is a difference between which subtype due to this Age variable how do i know or my question is violating the statistical design and rationale here?&lt;/p&gt;&#xA;&lt;p&gt;To know the differences in subtype I have do which is proving the &lt;code&gt;FAB&lt;/code&gt; which are basically my sub-types information where I have tested pairwise.&lt;/p&gt;&#xA;&lt;p&gt;I would like to know if I give any numerical variable to my design how do I interpret output?&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion or help would be really appreciated.&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2022-06-28T12:34:26.077" Title="deseq2 single factor design output interpretation" Tags="|deseq2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19305" PostTypeId="2" ParentId="8724" CreationDate="2022-06-24T23:36:53.427" Score="1" Body="&lt;p&gt;Just to conclude on the solution to this ancestral question. HDF5 is very useful, but there are better solutions for this specific question if it is focused on pandas dataframes. If you've lots of data the pythonic way to do this is pandas and therefore storing as a pandas dataframe makes sense.&lt;/p&gt;&#xA;&lt;p&gt;For long term storage, which was the original question, the storage of choice for pandas is &lt;code&gt;parquet&lt;/code&gt;. Described &lt;a href=&quot;https://pandas.pydata.org/pandas-docs/version/0.21/io.html#io-parquet&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Apache Parquet provides a partitioned binary columnar serialization&#xA;for data frames. It is designed to make reading and writing data&#xA;frames efficient, and to make sharing data across data analysis&#xA;languages easy. Parquet can use a variety of compression techniques to&#xA;shrink the file size as much as possible while still maintaining good&#xA;read performance.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In the following example, two methods &lt;code&gt;parquet&lt;/code&gt; methods are used for a pandas dataframe proteinDF which is first written and then imported.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from fastparquet import write&#xA;write('outfile.parq', proteinDF)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;pd.read_parquet('proteindata.parquet', engine='fastparquet')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above approach appears to have received most current developmental attention. However,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pyarrow as pa&#xA;import pyarrow.parquet as pq&#xA;arrow_table = pa.Table.from_pandas(proteinDF)&#xA;pq.write_table(arrow_table, 'myoutpath', use_dictionary=False,&#xA;                   compression='SNAPPY')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;pd.read_parquet('example_pa.parquet', engine='pyarrow')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above approach is compatible with &lt;code&gt;numba&lt;/code&gt;. 'Multithreading' is further described &lt;a href=&quot;https://wesmckinney.com/blog/python-parquet-multithreading/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The differences between the methods appear slight. Parallelisation is a big deal and depends on the problem.&lt;/p&gt;&#xA;&lt;p&gt;A general discussion on the methods is &lt;a href=&quot;https://blog.datasyndrome.com/python-and-parquet-performance-e71da65269ce&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;HDF5 vs parquet&lt;/strong&gt;&#xA;The backdrop for &lt;code&gt;parquet&lt;/code&gt; is the method is used for batch I/O not record by record writing to disk and that is this question and for that is pretty fast (at a guess bit better than &lt;code&gt;pickle&lt;/code&gt;, but a lot more compression).  &lt;code&gt;HDF5&lt;/code&gt; is perfect for rapid record by record writing and that is what it is designed for, so 'benchmarks' comparing I/O speed are not realistic unless they compare batch vs. record by record timings, where &lt;code&gt;HDF5&lt;/code&gt; excels. &lt;code&gt;parquet&lt;/code&gt; does achieve significantly better data compression and is considered a long term storage solution. In my hands &lt;code&gt;HDF5&lt;/code&gt; for fasta writing results in around half the size of the flat file (not much).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-06-26T13:32:16.943" LastActivityDate="2022-06-26T13:32:16.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19306" PostTypeId="1" AcceptedAnswerId="19307" CreationDate="2022-06-25T17:49:26.557" Score="2" ViewCount="3247" Body="&lt;p&gt;I am trying to create a histogram plot with &lt;code&gt;matplotlib&lt;/code&gt;. I am using &lt;code&gt;bmi&lt;/code&gt; column from four different data frames (d1, d2, d2, d4).&lt;/p&gt;&#xA;&lt;p&gt;The issue is that these 4 datasets have different sample sizes. So there is a disparity between the heights of the bar.&lt;/p&gt;&#xA;&lt;p&gt;Is there any additional option in the matplotlib to normalize the heights of the bar?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;&#xA;def bmiplot(graphWidth, graphHeight):&#xA;    f = plt.figure(figsize=(graphWidth/110.0, graphHeight/150.0), dpi=100)&#xA;    axes = f.add_subplot(111)&#xA;&#xA;    d1 = dm_group1&#xA;    d2 = dm_group2&#xA;    d3 = dm_group3&#xA;    d4 = dm_group4&#xA;&#xA;    plt.hist([d1.bmi.values.flatten(), d2.bmi.values.flatten(), d3.bmi.values.flatten(), d4.bmi.values.flatten()], &#xA;    label=['bmi_dm1', 'bmi_dm2', 'bmi_dm3', 'bmi_dm4'])&#xA;    plt.legend()&#xA;    plt.rc('xtick')&#xA;    plt.rc('ytick')&#xA;    &#xA;&#xA;graphWidth = 700&#xA;graphHeight = 800&#xA;bmiplot(graphWidth, graphHeight)&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4845" LastActivityDate="2022-06-25T18:35:30.207" Title="How to normalise the histogram height in matplotlib" Tags="|python|graphs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19307" PostTypeId="2" ParentId="19306" CreationDate="2022-06-25T18:35:30.207" Score="1" Body="&lt;p&gt;One approach is &lt;code&gt;density&lt;/code&gt;, traditionally the code would be ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;kwargs = dict(histtype='stepfilled', alpha=0.4, normed=True, bins=50)&#xA;&#xA;plt.hist(d1, **kwargs)&#xA;plt.hist(d2, **kwargs)&#xA;plt.hist(d3, **kwargs);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;alpha&lt;/code&gt; is transparency allowing overlapping distributions.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; &lt;code&gt;normed&lt;/code&gt; is deprecated, the blurb &lt;a href=&quot;https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.hist.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; describes &lt;code&gt;density&lt;/code&gt; now replacing it.&lt;/p&gt;&#xA;&lt;p&gt;The blurb is here:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;matplotlib.pyplot.hist(x, bins=None, range=None, density=False,&#xA;weights=None, ... **kwargs)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The definition of &lt;code&gt;density&lt;/code&gt; is here&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;densitybool, default: False If True, draw and return a probability&#xA;density: each bin will display the bin's raw count divided by the&#xA;total number of counts and the bin width (density = counts /&#xA;(sum(counts) * np.diff(bins))), so that the area under the histogram&#xA;integrates to 1 (np.sum(density * np.diff(bins)) == 1). If stacked is&#xA;also True, the sum of the histograms is normalized to 1.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;A different approach is relative frequency, via the &lt;code&gt;weights&lt;/code&gt; command.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; plt.hist(mydata, weights= *)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;*, relative counts per data type / total counts over all data types&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-06-25T18:35:30.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19308" PostTypeId="1" CreationDate="2022-06-26T13:31:03.077" Score="1" ViewCount="377" Body="&lt;p&gt;I am able to convert .bw files to .fq(fastq) manually in Shell, but I would like to automate the process coz I have hundreds of .bw file that I need to convert. Till now, I could think of following script, but unable to solve the problem. The .bw files are in H3K27ac folder.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import subprocess&#xA;&#xA;path_bw = &amp;quot;~/Documents/H3K27ac&amp;quot;&#xA;import os&#xA;&#xA;files = os.path.expanduser(path_bw)&#xA;for file in files:&#xA;    if file.endswith('.gz'):&#xA;        files.remove(file) &#xA;files = [x[:-3] if x.endswith('.bw') else x for x in files]&#xA;&#xA;&#xA;# for .bw files&#xA;&#xA;for file in files:&#xA;    bigwigtowig = &amp;quot;bigWigToWig &amp;quot; + path_bw + file + &amp;quot;.bw &amp;quot; + path_bw + file +&amp;quot;.wig&amp;quot;&#xA;    subprocess.call(bigwigtowig, shell=True)&#xA;&#xA;    wigtobed = &amp;quot;wig2bed &amp;lt; &amp;quot; + path_bw + file + &amp;quot;.wig &amp;gt; &amp;quot; + path_bw + file + &amp;quot;.bed&amp;quot;&#xA;    subprocess.call(wigtobed, shell=True)&#xA;&#xA;    getfasta = &amp;quot;bedtools getfasta -fi &amp;quot; + path_bw + &amp;quot;hg19.fa -bed &amp;quot; + path_bw + file + &amp;quot;.bed -fo &amp;quot; + path_bw + file + &amp;quot;.fa&amp;quot;&#xA;    subprocess.call(getfasta, shell=True)&#xA;&#xA;    perl = &amp;quot;perl &amp;quot; + path_bw + &amp;quot;fasta_to_fastq.pl&amp;quot; + path_bw + file + &amp;quot;.fa &amp;gt; &amp;quot; + path_bw + file + &amp;quot;.fq&amp;quot;&#xA;    #os.system(perl)&#xA;    subprocess.run(perl, shell=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Manually, I used the following to convert the .bw files:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;# bigWigToWig file1.bw fileout.wig&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;# wig2bed &amp;lt;fileout.wig&amp;gt; fileout.bed&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;# bedtools getfasta -fi hg19.fa -bed fileout.bed -fo fileout.fa&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;# perl fasta_to_fastq.pl fileout.fa &amp;gt; file1.fq&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;All the above packages were preinstalled via shell and works perfectly. hg19 is the genom that i am using. fasta_to_fastq.pl is openly available in github and helps to convert .fa to .fq.&lt;/p&gt;&#xA;&lt;p&gt;MAin thing i require is to convert all the files from .bw to .fq and this is the only way in my research that i could find. I need all the names to be the same as output.  For example file1.bw shoud be file1.fq , file2.bw --&amp;gt; file2.fq and so on.&lt;/p&gt;&#xA;&lt;p&gt;Do you have any ideas if this is possible or are there easier ways to convert it? It would be awesome to know your view point.&lt;/p&gt;&#xA;" OwnerUserId="15687" LastActivityDate="2022-06-26T16:35:12.513" Title="Converting .bw files to .fq (fastq)" Tags="|python|fasta|gene|fastq|hg19|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19309" PostTypeId="2" ParentId="19308" CreationDate="2022-06-26T13:59:37.797" Score="4" Body="&lt;p&gt;What you are doing makes &lt;strong&gt;no&lt;/strong&gt; sense at all. A bigwig file contains intervals and then indicates the coverage of that interval. That coverage again comes from the alignments covering that region. There is no direct connection between the interval and the fastq file other than that the fastq file is the basis for the coverage calculation. Converting the interval to a fasta and then forcing it into a fastq file is technically possible, but has absolutely no practical meaning. You cannot recover sequencing reads from a bigwig, that information is lost once you convert your alignments (usually a BAM file) into a bigwig. You should open a new question describing the problem that this is actually based on.&lt;/p&gt;&#xA;" OwnerUserId="15688" LastActivityDate="2022-06-26T13:59:37.797" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19310" PostTypeId="2" ParentId="19308" CreationDate="2022-06-26T16:06:47.543" Score="1" Body="&lt;p&gt;The code is verbose and I ain't at all sure some of it will work, at least in two places.&lt;/p&gt;&#xA;&lt;p&gt;The first point is &lt;code&gt;os.path.expanduser&lt;/code&gt; ... admittedly I prefer &lt;code&gt;pathlib&lt;/code&gt;, however that is definitely not how I understand that command. My understanding of &lt;code&gt;expanduser&lt;/code&gt; is if &lt;code&gt;~&lt;/code&gt; is being used. You definitely want to look at &lt;code&gt;globb&lt;/code&gt;-ing for the opening part of the code.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from pathlib import Path&#xA;import os&#xA;&#xA;mypath = Path(os.path.expanduser('~/Documents/H3K27ac'))&#xA;files = mypath.glob('*.bw') # or use ('*' + '.bw') &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm pretty certain that is what you want and its a lot easier than lines 1 to 10 in your code. The &lt;code&gt;files&lt;/code&gt; list will then go straight into the loop.&lt;/p&gt;&#xA;&lt;p&gt;Also, please check&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;files = [x[:-3] if x.endswith('.bw') else x for x in files]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Comprehensions are never logical in Python but convention is always to place the &lt;code&gt;if&lt;/code&gt; statement after the list of interest.&lt;/p&gt;&#xA;&lt;p&gt;Thus the following is much closer:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;files = [x for x in files if not x.endswith('.bw') else x[:-3]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; not tested, so this may throw a bug, but its better. However, using &lt;code&gt;glob&lt;/code&gt; bing you don't need this statement.&lt;/p&gt;&#xA;&lt;p&gt;The other issue is the sub.process call ... Normally they are written:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;os.chdir(path_bw)&#xA;file1 = 'file1.bw' &#xA;file2 = 'fileout.wig'&#xA;subprocess.call([&amp;quot;bigWigToWig&amp;quot;,  file1, file2], shell=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The way you've written is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Really hard to read&lt;/li&gt;&#xA;&lt;li&gt;I don't think it would work because of the absence of the &lt;code&gt;[]&lt;/code&gt;, certainly if you are using &lt;code&gt;popen&lt;/code&gt; it will not work&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Also check the following variant:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;subprocess.call([&amp;quot;bigWigToWig&amp;quot;,  file1, file2], text=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I know its seems a bit weird, but it very likely work (don't ask why)&lt;/p&gt;&#xA;&lt;p&gt;I agree &lt;code&gt;subprocess.call&lt;/code&gt; is a good idea in a Python pipeline. Issuing a hard path within &lt;code&gt;subprocess call&lt;/code&gt; I'm not a fan hence using &lt;code&gt;os&lt;/code&gt; to get Python to move into the &lt;code&gt;cwd&lt;/code&gt; is a better approach.&lt;/p&gt;&#xA;&lt;p&gt;A further point is the way you've used 'file' throughout your pipeline line, hmmmm... okay, but its difficult to read. Its not generally how a pipeline is written, but just saying.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;nextflow&lt;/code&gt; is an increasingly popular pipeline language and you may consider learning that rather than going down a Python route (Python is very good for data science), of course this particularly question could be very easily solved in &lt;code&gt;bash&lt;/code&gt; (but for some reason it ain't trendy).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-06-26T16:35:12.513" LastActivityDate="2022-06-26T16:35:12.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19311" PostTypeId="1" CreationDate="2022-06-26T20:32:52.670" Score="2" ViewCount="324" Body="&lt;p&gt;I am converting the fast5 file to fastq by using guppy basecaller after that by using kraken2 classified the sequence.&lt;/p&gt;&#xA;&lt;p&gt;Now I am trying to generate ASV file.&lt;/p&gt;&#xA;&lt;p&gt;Is this possible to generate ASV file from kraken output?&lt;/p&gt;&#xA;&lt;p&gt;Or is this any other way to generate ASV file from nanopore sequencing?&lt;/p&gt;&#xA;&lt;p&gt;I am new to nanopore sequencing data. I need help or suggestions to generate ASV file.&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="9854" LastEditorUserId="650" LastEditDate="2022-06-27T17:45:44.090" LastActivityDate="2023-08-15T07:04:50.580" Title="How can I generate ASV file from nanopore sequencing data?" Tags="|ngs|nanopore|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19312" PostTypeId="2" ParentId="17471" CreationDate="2022-06-27T10:05:16.760" Score="2" Body="&lt;p&gt;A nucleotide sequence may be a part of a genome assembly. If so, just click the assembly link from the right panel and you'll be redirected to the assembly page. &lt;a href=&quot;https://i.stack.imgur.com/jvtKd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jvtKd.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;In this assembly page, it says the assembly only contains one chromosome. &lt;a href=&quot;https://i.stack.imgur.com/tIyXT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tIyXT.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;Assembly method is also available for some assemblies. &lt;a href=&quot;https://i.stack.imgur.com/lotZz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lotZz.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15583" LastActivityDate="2022-06-27T10:05:16.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19313" PostTypeId="1" CreationDate="2022-06-27T11:20:52.230" Score="1" ViewCount="94" Body="&lt;p&gt;I want to read a PDB file and display/render the atoms using a graphics engine.&lt;/p&gt;&#xA;&lt;p&gt;If I consider each residue in the protein as a solid sphere, what would be the radius of the sphere?&lt;/p&gt;&#xA;&lt;p&gt;How can I calculate it? Or, is there any database for that?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="12686" LastEditDate="2022-06-27T12:36:10.727" LastActivityDate="2022-06-27T12:36:10.727" Title="How can I calculate the radius of a residue?" Tags="|protein-structure|pdb|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="19314" PostTypeId="2" ParentId="2543" CreationDate="2022-06-27T12:56:36.657" Score="0" Body="&lt;p&gt;If you are comfortable with Linux command line, try Entrez Direct and GNU parallel.&lt;/p&gt;&#xA;&lt;p&gt;First, I recommend acquiring a &lt;a href=&quot;https://ncbiinsights.ncbi.nlm.nih.gov/2017/11/02/new-api-keys-for-the-e-utilities/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI API key&lt;/a&gt; to increase the maximum requests per second.&lt;/p&gt;&#xA;&lt;p&gt;After that, if you already have the accession numbers that you want to download, just create a tab delimited file (eg arg.tsv) contains 3 columns (no header), including accession number, sequence start and sequence stop.&lt;/p&gt;&#xA;&lt;p&gt;To fetch the desired sequence region and save it in separate file in parallel:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;parallel -a arg.tsv -C '\t' -j8 &amp;quot;efetch -db nuccore -id {1} -seq_start {2} -seq_stop {3} -format fasta &amp;gt;{1}.fa&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or you can use Biopython &lt;code&gt;from Bio import Entrez&lt;/code&gt;. But I have not tried yet.&lt;/p&gt;&#xA;" OwnerUserId="15583" LastActivityDate="2022-06-27T12:56:36.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19315" PostTypeId="2" ParentId="16293" CreationDate="2022-06-27T13:08:02.697" Score="2" Body="&lt;p&gt;If your Windows 10 build number is 17063 or later, you can use &lt;code&gt;tar&lt;/code&gt; command to compress file in Windows PowerShell:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tar -vczf output.fq.gz input.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;-z&lt;/code&gt; flag means to compress archive with gzip.&lt;/p&gt;&#xA;" OwnerUserId="15583" LastActivityDate="2022-06-27T13:08:02.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19316" PostTypeId="2" ParentId="14903" CreationDate="2022-06-27T13:33:30.960" Score="0" Body="&lt;p&gt;A fasta file is just in plain text format. &lt;a href=&quot;https://pandoc.org/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pandoc&lt;/a&gt; can convert &lt;code&gt;.docx&lt;/code&gt; format to plain text format:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pandoc -i input.docx -t plain &amp;gt;output.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15583" LastActivityDate="2022-06-27T13:33:30.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19317" PostTypeId="1" AcceptedAnswerId="19318" CreationDate="2022-06-27T16:47:38.940" Score="1" ViewCount="64" Body="&lt;p&gt;The 33rd atom information in &lt;em&gt;&lt;strong&gt;3nir.pdb&lt;/strong&gt;&lt;/em&gt; is written as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;ATOM     33  N  ATHR A   2       4.935 -11.632  15.046  0.74  2.95           N&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;How should I interpret &lt;code&gt;N  A&lt;/code&gt; in the above line?&lt;/p&gt;&#xA;&lt;p&gt;I mean, what does it mean? Is it Sodium or Nitrogen?&lt;/p&gt;&#xA;&lt;p&gt;If it is Nitrogen, what does &lt;code&gt;A&lt;/code&gt; mean then?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="12686" LastEditDate="2022-06-27T16:57:21.760" LastActivityDate="2022-06-27T18:28:05.323" Title="What do these characters mean in a PDB file?" Tags="|pdb|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19318" PostTypeId="2" ParentId="19317" CreationDate="2022-06-27T18:17:36.800" Score="2" Body="&lt;p&gt;from &lt;a href=&quot;https://www.wwpdb.org/documentation/file-format&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.wwpdb.org/documentation/file-format&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5tbgL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5tbgL.png&quot; alt=&quot;https://ftp.wwpdb.org/pub/pdb/doc/format_descriptions/Format_v33_A4.pdf&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Protein Data Bank Contents Guide:&#xA;Atomic Coordinate Entry Format Description&#xA;Version 3.30&#xA;Document Published by the wwPDB&#xA;This format complies with the PDB Exchange Dictionary (PDBx)&#xA;&lt;a href=&quot;http://mmcif.pdb.org/dictionaries/mmcif_pdbx.dic/Index/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://mmcif.pdb.org/dictionaries/mmcif_pdbx.dic/Index/index.html&lt;/a&gt;.&#xA;I believe PDB format is kind of outdated, but not sure..&lt;/p&gt;&#xA;&lt;p&gt;At page 177 it says about altLoc:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;AltLoc is the place holder to indicate alternate conformation. The alternate conformation can be&#xA;in the entire polymer chain, or several residues or partial residue (several atoms within one&#xA;residue). If an atom is provided in more than one position, then a non-blank alternate location&#xA;indicator must be used for each of the atomic positions. Within a residue, all atoms that are&#xA;associated with each other in a given conformation are assigned the same alternate position&#xA;indicator. There are two ways of representing alternate conformation- either at atom level or at&#xA;residue level (see examples).&lt;/li&gt;&#xA;&lt;li&gt;For atoms that are in alternate sites indicated by the alternate site indicator, sorting of atoms in&#xA;the ATOM/HETATM list uses the following general rules:&#xA;• In the simple case that involves a few atoms or a few residues with alternate sites, the&#xA;coordinates occur one after the other in the entry.&#xA;• In the case of a large heterogen groups which are disordered, the atoms for each&#xA;conformer are listed together.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;your question could be more useful if you post the entire description for residue 2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATOM     33  N  ATHR A   2       4.935 -11.632  15.046  0.74  2.95           N&#xA;ATOM     34  N  BTHR A   2       5.078 -11.406  15.180  0.31  2.78           N  &#xA;ATOM     35  CA ATHR A   2       5.757 -11.521  13.850  0.81  3.02           C  &#xA;ATOM     36  CA BTHR A   2       5.773 -11.153  13.921  0.20  2.67           C  &#xA;ATOM     37  C  ATHR A   2       7.070 -10.839  14.210  0.74  2.82           C  &#xA;ATOM     38  C  BTHR A   2       7.155 -10.559  14.193  0.29  1.80           C  &#xA;ATOM     39  O  ATHR A   2       7.152  -9.941  15.050  0.80  3.31           O  &#xA;ATOM     40  O  BTHR A   2       7.214  -9.641  15.012  0.25  2.41           O  &#xA;ATOM     41  CB ATHR A   2       4.976 -10.693  12.813  0.87  5.53           C  &#xA;ATOM     42  CB BTHR A   2       4.896 -10.354  12.941  0.25 12.07           C  &#xA;ATOM     43  OG1ATHR A   2       4.611  -9.432  13.388  1.00  6.88           O  &#xA;ATOM     44  OG1BTHR A   2       3.743 -11.083  12.501  0.25  9.57           O  &#xA;ATOM     45  CG2ATHR A   2       3.858 -11.584  12.293  0.75 10.03           C  &#xA;ATOM     46  CG2BTHR A   2       5.683  -9.885  11.726  0.27  5.90           C  &#xA;ATOM     47  H  ATHR A   2       4.547 -10.814  15.527  0.75  3.44           H  &#xA;ATOM     48  H  BTHR A   2       5.510 -10.211  15.754  0.25  2.90           H  &#xA;ATOM     49  HA ATHR A   2       5.962 -12.339  13.548  0.75  3.32           H  &#xA;ATOM     50  HA BTHR A   2       4.036  -9.929  13.477  0.25  2.86           H  &#xA;ATOM     51  HB ATHR A   2       5.648 -10.589  11.938  0.75  5.43           H  &#xA;ATOM     52  HB BTHR A   2       4.644  -9.326  13.574  0.25  5.67           H  &#xA;ATOM     53  HG1ATHR A   2       5.030  -9.344  14.216  0.75  8.74           H  &#xA;ATOM     54  HG1BTHR A   2       3.236 -11.198  13.399  0.25 10.21           H  &#xA;ATOM     55 HG21ATHR A   2       4.096 -12.441  11.924  0.75 10.92           H  &#xA;ATOM     56 HG21BTHR A   2       6.542  -9.278  12.024  0.25  9.66           H  &#xA;ATOM     57 HG22ATHR A   2       3.222 -10.974  11.650  0.75 10.92           H  &#xA;ATOM     58 HG22BTHR A   2       5.039  -9.142  11.179  0.25  9.66           H  &#xA;ATOM     59 HG23ATHR A   2       3.163 -11.738  13.200  0.75 10.92           H  &#xA;ATOM     60 HG23BTHR A   2       5.904 -10.639  11.169  0.25  9.66           H  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2022-06-27T18:28:05.323" LastActivityDate="2022-06-27T18:28:05.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19319" PostTypeId="1" AcceptedAnswerId="19323" CreationDate="2022-06-27T20:18:04.857" Score="3" ViewCount="72" Body="&lt;p&gt;I am trying to parse PDB files.&lt;/p&gt;&#xA;&lt;p&gt;Say, a PDB file has the following data:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATOM     33  N  ATHR A   2       4.935 -11.632  15.046  0.74  2.95           N&#xA;ATOM     34  N  BTHR A   2       5.078 -11.406  15.180  0.31  2.78           N  &#xA;ATOM     35  CA ATHR A   2       5.757 -11.521  13.850  0.81  3.02           C  &#xA;ATOM     36  CA BTHR A   2       5.773 -11.153  13.921  0.20  2.67           C  &#xA;ATOM     37  C  ATHR A   2       7.070 -10.839  14.210  0.74  2.82           C  &#xA;ATOM     38  C  BTHR A   2       7.155 -10.559  14.193  0.29  1.80           C  &#xA;ATOM     39  O  ATHR A   2       7.152  -9.941  15.050  0.80  3.31           O  &#xA;ATOM     40  O  BTHR A   2       7.214  -9.641  15.012  0.25  2.41           O  &#xA;ATOM     41  CB ATHR A   2       4.976 -10.693  12.813  0.87  5.53           C  &#xA;ATOM     42  CB BTHR A   2       4.896 -10.354  12.941  0.25 12.07           C  &#xA;ATOM     43  OG1ATHR A   2       4.611  -9.432  13.388  1.00  6.88           O  &#xA;ATOM     44  OG1BTHR A   2       3.743 -11.083  12.501  0.25  9.57           O  &#xA;ATOM     45  CG2ATHR A   2       3.858 -11.584  12.293  0.75 10.03           C  &#xA;ATOM     46  CG2BTHR A   2       5.683  -9.885  11.726  0.27  5.90           C  &#xA;ATOM     47  H  ATHR A   2       4.547 -10.814  15.527  0.75  3.44           H  &#xA;ATOM     48  H  BTHR A   2       5.510 -10.211  15.754  0.25  2.90           H  &#xA;ATOM     49  HA ATHR A   2       5.962 -12.339  13.548  0.75  3.32           H  &#xA;ATOM     50  HA BTHR A   2       4.036  -9.929  13.477  0.25  2.86           H  &#xA;ATOM     51  HB ATHR A   2       5.648 -10.589  11.938  0.75  5.43           H  &#xA;ATOM     52  HB BTHR A   2       4.644  -9.326  13.574  0.25  5.67           H  &#xA;ATOM     53  HG1ATHR A   2       5.030  -9.344  14.216  0.75  8.74           H  &#xA;ATOM     54  HG1BTHR A   2       3.236 -11.198  13.399  0.25 10.21           H  &#xA;ATOM     55 HG21ATHR A   2       4.096 -12.441  11.924  0.75 10.92           H  &#xA;ATOM     56 HG21BTHR A   2       6.542  -9.278  12.024  0.25  9.66           H  &#xA;ATOM     57 HG22ATHR A   2       3.222 -10.974  11.650  0.75 10.92           H  &#xA;ATOM     58 HG22BTHR A   2       5.039  -9.142  11.179  0.25  9.66           H  &#xA;ATOM     59 HG23ATHR A   2       3.163 -11.738  13.200  0.75 10.92           H  &#xA;ATOM     60 HG23BTHR A   2       5.904 -10.639  11.169  0.25  9.66           H  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We see that there are many alternative atoms in the 2nd residue.&lt;/p&gt;&#xA;&lt;p&gt;How should I choose the atoms?&lt;/p&gt;&#xA;&lt;p&gt;Should I just randomly choose one atom from the alternatives? How can my parser learn to differentiate between those two atoms (say, 33 and 34)? Coz, from the parsing point of view, there is no indication that they are related.&lt;/p&gt;&#xA;&lt;p&gt;How can I parse alternative atom information in a PDB file?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastActivityDate="2022-07-17T08:24:47.010" Title="How can I parse alternative atom information?" Tags="|proteins|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19320" PostTypeId="2" ParentId="16293" CreationDate="2022-06-28T10:45:42.803" Score="-1" Body="&lt;p&gt;Bioinformatics is 99% done in &lt;a href=&quot;https://en.wikipedia.org/wiki/Unix-like&quot; rel=&quot;nofollow noreferrer&quot;&gt;Unix-like&lt;/a&gt; environments, and trying to come up with Windows solutions is unnecessary effort. The Windows Subsystem Linux (WSL2) provides you with a native Linux distribution within Windows that easily allow you to do any Unix-ish thing such as running gzip. It is straight-forward to install, see its online documentation.&lt;/p&gt;&#xA;" OwnerUserId="15694" LastEditorUserId="298" LastEditDate="2023-09-26T08:07:27.913" LastActivityDate="2023-09-26T08:07:27.913" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19321" PostTypeId="2" ParentId="19303" CreationDate="2022-06-28T12:34:26.077" Score="2" Body="&lt;p&gt;Your design is &lt;code&gt;~ Age&lt;/code&gt; so you can only test how Age affects each gene and it is assumed that the effect is linear. If you would like to test the effect of sub type on gene expression you'll have to introduce it to the design e.g. &lt;code&gt;~ Age + Sub.Type&lt;/code&gt; (or whatever the column name is). This will not test how the effect of Age is different between between the subtypes, if you would like to test this you will have to include the interaction term in the design formula: &lt;code&gt;~ Age + Sub.Type + Age:Sub.Type&lt;/code&gt;, this design have a lot of parameters and you will need a lot of samples to have a reliable test.&lt;/p&gt;&#xA;&lt;p&gt;Hope it helped&lt;/p&gt;&#xA;" OwnerUserId="1540" LastActivityDate="2022-06-28T12:34:26.077" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="19322" PostTypeId="1" AcceptedAnswerId="19327" CreationDate="2022-06-28T13:32:34.530" Score="2" ViewCount="148" Body="&lt;p&gt;Currently algorithmically developing a calculation for &amp;quot;co-dependent&amp;quot; mutation (quotation appropriate). The algorithm uses chunked biopython alignment objects which are parallelized.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Aim&lt;/strong&gt;&#xA;One optization is caching a database within parallelization, but I ain't sure whether the parallelization duplicates the cache.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Method&lt;/strong&gt;&#xA;Each alignment chunk (biopython object) is passed onto a separate CPU and refers back to the original database to obtain its metadata in the final stage of the calculation (dragging metadata around this algorithm makes no sense).&lt;/p&gt;&#xA;&lt;p&gt;The idea is to cache the database to prevent pulling it in from the harddrive:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from multiprocessor import Pool&#xA;from myOOPlibrary import mystuff    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Facade design, the parallelised index to the alignment chunks enter into the myOOPlibrary:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from cachetools import cached, TTLCache&#xA;&#xA;cache = TTLCache(maxsize=1, ttl=500)&#xA;&#xA;Class mystuff (stuff2):&#xA;... # init omitted&#xA;&#xA;    def mymethod(self, ..):&#xA;    # to be updated &#xA;    &#xA;Class stuff2 ():&#xA;... # init omitted&#xA;&#xA;    @cached(cache)&#xA;    def getDB(self, ...):&#xA;.... #  pull it in (harddrive) and parse to a dictionary&#xA;    return metadataDB&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&#xA;I'm wondering if caching is the right approach in parallelization, specifically whether this will end up with multiplying the database held in RAM...&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Alternatives&lt;/strong&gt;&#xA;The alternativees to caching are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;IO to JSON (not cool)&lt;/li&gt;&#xA;&lt;li&gt;IO via pickle (binary IO): a JSON will readily&lt;code&gt;pickle&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The possible advantage of the alternativees are the database is written once and accessed by all cores for certain.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Underlying question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;When caching is the database  cached for each core? If so the RAM overhead becomes an issue because the database is duplicated &lt;code&gt;total CPUs - 1&lt;/code&gt; times&lt;/li&gt;&#xA;&lt;li&gt;Alternatively is the cache shared between CPUs centrally and each CPU refers to this location?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-06-28T22:15:30.240" LastActivityDate="2022-07-02T19:46:31.090" Title="Caching and parallelization" Tags="|python|covid-19|algorithms|mutations|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="19323" PostTypeId="2" ParentId="19319" CreationDate="2022-06-28T13:42:44.787" Score="4" Body="&lt;p&gt;Generally alternate location are used in structures with partial occupancy.&lt;/p&gt;&#xA;&lt;p&gt;In &lt;a href=&quot;https://www.wwpdb.org/documentation/file-format-content/format33/sect9.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.wwpdb.org/documentation/file-format-content/format33/sect9.html&lt;/a&gt; you will see that the fourth numerical column is occupancy. So you could take the higher occupancy version.&#xA;The tool PDBfixer can do it for you: github.com/openmm/pdbfixer&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Do not assume the sum of occupancy is 1 —it should but errors do creep up especially for close alternate location that overlap.&lt;/li&gt;&#xA;&lt;li&gt;The anisotropic and isotropic B-factors will be lower than were the structure refined with a single conformation (like normal atoms).&lt;/li&gt;&#xA;&lt;li&gt;Same letters in different residues should not be assumed to be paired up. Models as seen for NMR structures are a solution for that.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Lastly... Your structure has protons, which is unusual (exceptions exist). It is fine if it is a neutron diffraction structure or NMR. If you are not expecting protons (X-ray structure), skip it out of safety.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2022-07-17T08:24:47.010" LastActivityDate="2022-07-17T08:24:47.010" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19324" PostTypeId="1" CreationDate="2022-06-28T18:39:11.203" Score="4" ViewCount="41" Body="&lt;p&gt;I only have the accession codes of several proteins from the MGnify database (&lt;a href=&quot;https://www.ebi.ac.uk/metagenomics/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/metagenomics/&lt;/a&gt;). I would like to retrieve the full amino acid sequence data from the database, but I have not been able to find a way to do this using the web server or available API.&lt;/p&gt;&#xA;&lt;p&gt;The website provides sequence search with the HMMER tool (&lt;a href=&quot;https://www.ebi.ac.uk/metagenomics/sequence-search/search/phmmer&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/metagenomics/sequence-search/search/phmmer&lt;/a&gt;). However, I need the sequence data to search with HMMER.&lt;/p&gt;&#xA;&lt;p&gt;How can I get the protein sequence only with the accession code?&lt;/p&gt;&#xA;&lt;p&gt;P.S: An example of the accession code is MGYP001242958860.&lt;/p&gt;&#xA;" OwnerUserId="15568" LastEditorUserId="15568" LastEditDate="2022-06-28T18:47:42.757" LastActivityDate="2022-06-28T18:47:42.757" Title="Retrieve protein sequence from Mgnify given only accession code" Tags="|data-download|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19325" PostTypeId="2" ParentId="5642" CreationDate="2022-06-28T20:06:27.213" Score="1" Body="&lt;p&gt;I wrote a function to do that a while ago, maybe it helps. It only works with dot-bracket notation, and it turns it into ordered pairs which then you can easily plot.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pandas as pd&#xA;import seaborn as sns&#xA;&#xA;&#xA;def dot_bracket_to_pairs(ss_string):&#xA;    '''&#xA;    Takes dot and bracket string, returns dataframe&#xA;    with paired bases.&#xA;    If any invalid characters are in the structure, it&#xA;    will interpret them as dots, as it only reads parentheses.&#xA;    '''&#xA;    index_list = []&#xA;    pairs = {}&#xA;&#xA;    for index, char in enumerate(ss_string):&#xA;        if char == '(':&#xA;            index_list.append(index)&#xA;        if char == ')':&#xA;            try:&#xA;                # pair to last item in the list in dictionary&#xA;                pairs[index_list.pop()] = index&#xA;            except IndexError:&#xA;                print(f'Invalid structure, found extra \')\' in position {index}')&#xA;&#xA;    if len(index_list) != 0:&#xA;        for item in index_list:&#xA;            print(f'Invalid structure, found extra in \'(\' in position {item}')&#xA;&#xA;    df_pairs = pd.DataFrame(pairs.items(), columns=['Y', 'X'])&#xA;&#xA;    return df_pairs&#xA;&#xA;db_string = '((((((((((..(((((((.......)))))))......).((((((.......))))))..)))))))))'&#xA;df = dot_bracket_to_pairs(db_string)&#xA;&#xA;sns.scatterplot(data=df, x=&amp;quot;X&amp;quot;, y=&amp;quot;Y&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15654" LastEditorUserId="15654" LastEditDate="2022-06-28T20:07:38.610" LastActivityDate="2022-06-28T20:07:38.610" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19326" PostTypeId="2" ParentId="5642" CreationDate="2022-06-28T20:59:05.883" Score="1" Body="&lt;p&gt;One way simply is to use VARNA, here is a link ... &lt;a href=&quot;https://varna.lri.fr&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://varna.lri.fr&lt;/a&gt;. This is well known for its power to view secondary structure (e.g. its in Jalview). It is generally considered the best viewer app for secondary structure.&lt;/p&gt;&#xA;&lt;p&gt;However, the solution by @nquinoneso whilst simple, is preferable. The output is here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MC2ii.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MC2ii.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The y-axis is the 3' and the x-axis is the 5', the dots represent hairpins. Whilst its simple, the power of this approach is the automation by direct access to the (pandas) dataframe.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;I do recommend upvoting the post.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Complex secondary structure would not work because it will confuse the 5' stems with the immediate 3' stems which don't hybridise and  a more advanced solution using nucleotide code would be needed. Thats the bit for further development, using the index both for the dot-bracket and the nucleotide sequence to confirm the hairpin. VARNA does perform this calculation (because it identifies canonical mutations)... but you can't access the underlying array/dataframe which is certainly a limitation.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-06-28T23:41:48.237" LastActivityDate="2022-06-28T23:41:48.237" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19327" PostTypeId="2" ParentId="19322" CreationDate="2022-06-29T02:45:50.883" Score="1" Body="&lt;p&gt;&lt;em&gt;&lt;strong&gt;Summarising&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;@Jesse's comment I think is the right answer, open the database outside parallelisation. What I didn't thoroughly describe (and what was essentially requested) is the architecture, the alignments entering into an entire OOP library and the whole library is parallelised, i.e. every new alignment entering library is on its own CPU. The answer is obvious: instantiate the OOP library with the entire database and thats a key purpose of the design.&lt;/li&gt;&#xA;&lt;li&gt;The technical answer is &lt;code&gt;joblib&lt;/code&gt;. This is built to cache across parallelised archetechture as the &lt;code&gt;joblib&lt;/code&gt; team point out &lt;strong&gt;&lt;a href=&quot;https://stackoverflow.com/questions/25033631/multiple-processes-sharing-a-single-joblib-cache&quot;&gt;here&lt;/a&gt;&lt;/strong&gt;. The criticism is that there's parallelisation within parallelisation. The discussion* is as much in the comments as in the response.&lt;/li&gt;&#xA;&lt;li&gt;@user172818 response is correct too, the algorithm would benefit from C. Its a lot of code to rewrite in &lt;code&gt;C&lt;/code&gt;, but sidestepping via Cython is a future development (its a hybrid language connecting C and Python).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;*, &lt;a href=&quot;https://stackoverflow.com/questions/25033631/multiple-processes-sharing-a-single-joblib-cache&quot;&gt;https://stackoverflow.com/questions/25033631/multiple-processes-sharing-a-single-joblib-cache&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;There was a comment hinting this is Stack Overflow, that's difficult. Ultimately the technical answer (which is Stack Overflow) isn't the correct answer as far as I understand OOP.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Feedback&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Just to wrap this, point 1. worked fine, thank you. It means a database sits on each processor in parallelisation, but is imported  once. In truth, it worked nicely and provided cleaner code. Re-writing to instantiate once would affect other stuff so would be more costly than a cost-saving.&lt;/p&gt;&#xA;&lt;p&gt;Python can be very fast (e.g. numpy), but also really slow (hence it has critiques), extensively parallelising helps bridge the gap.&#xA;Python parallelisation has critiques which I strongly, strongly believe are unfounded: I parallelise loads via &lt;code&gt;multiprocessor&lt;/code&gt; library no problems.&lt;/p&gt;&#xA;&lt;p&gt;Caching here, i.e. parallelising across a parallelisation could stretch things too far.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Last point&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Point 3, re-writing in Cython/&lt;code&gt;C&lt;/code&gt;, is a greater priority than the testing required for parallelised caching: can't parallelise all data and that code is better as &lt;code&gt;C&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-07-02T19:46:31.090" LastActivityDate="2022-07-02T19:46:31.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19328" PostTypeId="1" CreationDate="2022-06-29T04:35:30.877" Score="2" ViewCount="64" Body="&lt;p&gt;I have about 300 90-bp sequences which I would like to map to a reference genome to make it longer to a 300bp sequence, wherein the 90bp is at the middle.&lt;/p&gt;&#xA;&lt;p&gt;Anyone knows what bioinformatics tool I can use for this? Thank you!&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Actual scenario:&lt;/em&gt;&#xA;The SNP sequences are from a SNP panel data (90bp sequence, SNP is at the middle). We would like to extend the sequence to 300bp, retaining the SNP at the middle so we can make a sequence capture panel of the SNP.&lt;/p&gt;&#xA;" OwnerUserId="15697" LastActivityDate="2022-06-29T06:04:41.580" Title="What tool I can use to map short-reads sequences to reference genome and get specific mapped size" Tags="|sequence-alignment|snp|read-mapping|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19329" PostTypeId="2" ParentId="19328" CreationDate="2022-06-29T06:04:41.580" Score="3" Body="&lt;p&gt;If you have a SNP panel then you have the coordinates on the genome. You can use tools such as &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/slop.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools slop&lt;/a&gt; to expand the coordinates to the desired length and then extract those regions from the target genome using &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/getfasta.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools getfasta&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1540" LastActivityDate="2022-06-29T06:04:41.580" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19330" PostTypeId="1" CreationDate="2022-06-29T07:46:25.357" Score="2" ViewCount="63" Body="&lt;p&gt;I am doing a gene enrichment of different clusters obtained by WGCNA, but there are some of them in which I don't get enriched processes, is there any way to find processes to these modules, for example to associate them to orthologues of other organisms. If there are different ways, could you send me links on how to do it?&lt;/p&gt;&#xA;" OwnerUserId="15698" LastActivityDate="2022-06-29T16:24:56.127" Title="Find enriched processes in modules obtained by WGCNA" Tags="|r|rna-seq|wgcna|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19331" PostTypeId="1" CreationDate="2022-06-29T08:56:29.283" Score="6" ViewCount="981" Body="&lt;p&gt;I want to perform a multi-sequence alignment on 12 samples that clustered based on cgMLST. Ultimate goal is to find out whether they differ by the presence of a few genes.&lt;/p&gt;&#xA;&lt;p&gt;I have performed multi-sequence alignment in the past using MAFFT, the main difference being that, back then, I was aligning protein sequences, and now I want to align multiple &lt;strong&gt;denovo assembled genomes, with multiple contigs each&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I first had a look at &lt;a href=&quot;https://mafft.cbrc.jp/alignment/software/&quot; rel=&quot;noreferrer&quot;&gt;MAFFT&lt;/a&gt;, the program that I used in the past for protein multi-sequence alignment, though I noticed that it considers each contig in a denovo assembly as independent sequence, which is not what I want&lt;/p&gt;&#xA;&lt;p&gt;Then I had a look at &lt;a href=&quot;https://darlinglab.org/mauve/user-guide/aligning.html&quot; rel=&quot;noreferrer&quot;&gt;Mauve&lt;/a&gt;, but was discouraged by installation trouble-shooting. Also I read on their webpage:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;When an individual file contains several sequence entries, they will&#xA;be concatenated and the whole concatenated sequence will be aligned to&#xA;the sequences in the other files. This behavior allows&#xA;multi-chromosomal genomes to be aligned by including all chromosomes&#xA;in a single sequence file. Similarly, incomplete genomes that consist&#xA;of several sequence contigs can be aligned, though beware that&#xA;incorrectly ordered contigs will appear as genome rearrangements in&#xA;the Mauve viewer.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So I guess concatenating all contigs in a single fasta file is not an option, as artefacts (genome rearrangements) might appear&lt;/p&gt;&#xA;&lt;p&gt;It turns out that MAFFT has also an experimental feature to multi-align &lt;a href=&quot;https://mafft.cbrc.jp/alignment/server/index-rawreads.html&quot; rel=&quot;noreferrer&quot;&gt;fastq&lt;/a&gt; reads, which sounds attractive to me. Therefore I would like to ask if anyone can recommend this new/ experimental feature. Also if this option is available on the command-line tool, as the mafft help is quite limited (it does not specify the format of input and output files):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  MAFFT v7.505 (2022/Apr/10)&#xA;  https://mafft.cbrc.jp/alignment/software/&#xA;  MBE 30:772-780 (2013), NAR 30:3059-3066 (2002)&#xA;------------------------------------------------------------------------------&#xA;High speed:&#xA;  % mafft in &amp;gt; out&#xA;  % mafft --retree 1 in &amp;gt; out (fast)&#xA;&#xA;High accuracy (for &amp;lt;~200 sequences x &amp;lt;~2,000 aa/nt):&#xA;  % mafft --maxiterate 1000 --localpair  in &amp;gt; out (% linsi in &amp;gt; out is also ok)&#xA;  % mafft --maxiterate 1000 --genafpair  in &amp;gt; out (% einsi in &amp;gt; out)&#xA;  % mafft --maxiterate 1000 --globalpair in &amp;gt; out (% ginsi in &amp;gt; out)&#xA;&#xA;If unsure which option to use:&#xA;  % mafft --auto in &amp;gt; out&#xA;&#xA;--op # :         Gap opening penalty, default: 1.53&#xA;--ep # :         Offset (works like gap extension penalty), default: 0.0&#xA;--maxiterate # : Maximum number of iterative refinement, default: 0&#xA;--clustalout :   Output: clustal format, default: fasta&#xA;--reorder :      Outorder: aligned, default: input order&#xA;--quiet :        Do not report progress&#xA;--thread # :     Number of threads (if unsure, --thread -1)&#xA;--dash :         Add structural information (Rozewicki et al, submitted)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any other recommendation on another strategy to align multiple denovo assembled genomes with multiple contigs each are welcome&lt;/p&gt;&#xA;" OwnerUserId="3452" LastActivityDate="2022-11-17T09:49:53.257" Title="multi-sequence alignment of samples with multiple contigs each" Tags="|sequence-alignment|fasta|sam|multiple-sequence-alignment|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19332" PostTypeId="2" ParentId="19331" CreationDate="2022-06-29T09:29:08.593" Score="1" Body="&lt;p&gt;It won't work on more than 3 genomes at a time, but SynMap3D from the Comparative Genomics webserver (&lt;a href=&quot;https://genomevolution.org/coge/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genomevolution.org/coge/&lt;/a&gt;) should do the trick. I'm mentioning it here because the advantage, for me (in contrast to, say, Mauve), was user-friendliness.&lt;/p&gt;&#xA;&lt;p&gt;Small note: Be mindful of what you name your uploaded samples. You can set your sequence not to be publicly available, but if you give it a new name, as opposed to a species name already in the database, that will be visible.&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2022-06-29T09:29:08.593" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19333" PostTypeId="1" AcceptedAnswerId="19347" CreationDate="2022-06-29T09:39:32.543" Score="2" ViewCount="41" Body="&lt;p&gt;NCBI has several labels for assembly completeness - Complete, Scaffold, Chromosome and Contig. Complete would be a circularized genome (or linear, rarely)&lt;/p&gt;&#xA;&lt;p&gt;For a Complete genome it's fairly straightforward - if not present, a gene can safely assumed to be missing (or possibly present on plasmids, if those aren't available in the database).&lt;/p&gt;&#xA;&lt;p&gt;But what about the others? With Contig-level genomes, I'd feel uncertain about making any statements (especially since some of these projects are split into thousands of contigs). But what about Scaffold, or Chromosome? Are they expected to be complete in terms of genes, but just missing repetitive sequences?&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2022-06-30T19:40:06.570" Title="How good does the assembly of an NCBI prokaryotic genome have to be in order to argue gene loss?" Tags="|gene|assembly|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19334" PostTypeId="1" AcceptedAnswerId="19409" CreationDate="2022-06-29T11:36:32.097" Score="3" ViewCount="59" Body="&lt;p&gt;I am developing a bioinformatics framework only for learning purposes. I want to calculate --&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;dihedral angles&lt;/li&gt;&#xA;&lt;li&gt;detect H, E, and C components&lt;/li&gt;&#xA;&lt;li&gt;detect Hydrogen bonds&lt;/li&gt;&#xA;&lt;li&gt;etc.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Therefore, I want a PDB file that represents a protein with a small chain but is complex enough for me to experiment with.&lt;/p&gt;&#xA;&lt;p&gt;Can you suggest such a PDB file?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="96" LastEditDate="2022-06-29T13:14:35.310" LastActivityDate="2022-07-17T14:14:18.823" Title="Example of a small but moderately complex PDB file?" Tags="|pdb|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19335" PostTypeId="1" AcceptedAnswerId="19341" CreationDate="2022-06-29T12:02:40.680" Score="4" ViewCount="175" Body="&lt;p&gt;I am developing an educational Bioinformatics framework, I need to know&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Which atoms are absolutely not found in any PDB files?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;in the following list -&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Atomic_radii_of_the_elements_(data_page)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Atomic radii of the elements (data page)&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="12686" LastActivityDate="2022-06-30T19:42:22.480" Title="Which atoms are not found in protein PDB files?" Tags="|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19337" PostTypeId="1" CreationDate="2022-06-29T13:13:50.427" Score="2" ViewCount="71" Body="&lt;p&gt;Are there any programs (web server or installable) out there to calculate segment overlap score (SOV) for protein secondary structures?&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="12203" LastActivityDate="2022-06-29T13:13:50.427" Title="Program for SOV calculation" Tags="|proteins|secondary-structure|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19338" PostTypeId="2" ParentId="19331" CreationDate="2022-06-29T13:18:38.407" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/mummer4/mummer&quot; rel=&quot;nofollow noreferrer&quot;&gt;MUMmer4&lt;/a&gt; is a versatile alignment tool for DNA and protein sequences. It supports one reference genome and up to 32 query genomes. MUMmer4 will align every contig in each genome to the reference genome.&lt;/p&gt;&#xA;" OwnerUserId="15583" LastEditorUserId="15583" LastEditDate="2022-06-29T13:25:03.590" LastActivityDate="2022-06-29T13:25:03.590" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19339" PostTypeId="1" AcceptedAnswerId="19344" CreationDate="2022-06-29T14:44:23.660" Score="2" ViewCount="134" Body="&lt;p&gt;I am using&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BiocManager::install(&amp;quot;DiffBind&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to install the package &amp;quot;DiffBind&amp;quot; on CentOS_7,&#xA;I got the error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gcc -I&amp;quot;/opt/common/CentOS_7/R/R-4.0.5/lib64/R/include&amp;quot; -DNDEBUG -D_FILE_OFFSET_BITS=64 -I'/lila/home/lid/R/x86_64-pc-linux-gnu-library/4.0/Rhtslib/include' -I'/admin/opt/common/CentOS_7/R/R-4.0.5/lib64/R/library/Rcpp/include' -I/usr/local/include   -fpic  -g -O2  -c mergeOne.c -o mergeOne.o&#xA;mergeOne.c: In function ‘mungeTarget’:&#xA;mergeOne.c:341:3: error: ‘for’ loop initial declarations are only allowed in C99 mode&#xA;   for (int i=l;i&amp;lt;l+r;i++) {&#xA;   ^&#xA;mergeOne.c:341:3: note: use option -std=c99 or -std=gnu99 to compile your code&#xA;make: *** [mergeOne.o] Error 1&#xA;ERROR: compilation failed for package ‘DiffBind’&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can anyone tell me what should I do?&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="9698" LastEditorUserId="73" LastEditDate="2022-06-30T04:14:17.437" LastActivityDate="2022-06-30T15:22:39.543" Title="BiocManager::install(&quot;DiffBind&quot;) mergeOne.c:341:3: note: use option -std=c99 or -std=gnu99 to compile your code" Tags="|r|bioconductor|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19340" PostTypeId="2" ParentId="19330" CreationDate="2022-06-29T16:24:56.127" Score="2" Body="&lt;p&gt;Well, not all modules will have significant enrichment in gene sets such as GO terms or KEGG pathways. In my experience, the smaller modules are often not significantly enriched. They could be the result of some technical variation, they could contain genes localized to a particular genomic region (do you also calculate enrichment in genomic regions?) or they could be a particular gene family. WGCNA modules often are enriched in known pathways and processes, but that does not mean that all will be.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2022-06-29T16:24:56.127" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19341" PostTypeId="2" ParentId="19335" CreationDate="2022-06-29T17:28:41.850" Score="4" Body="&lt;h2&gt;PDB file readability&lt;/h2&gt;&#xA;&lt;p&gt;It is a question of implementation.&#xA;For example:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from rdkit import Chem&#xA;&#xA;mol = Chem.MolFromSmiles('[Og]')&#xA;pdbblock = Chem.MolToPDBBlock(mol)&#xA;&#xA;import pymol2&#xA;&#xA;with pymol2.PyMOL() as pymol:&#xA;    pymol.cmd.read_pdbstr(pdbblock, 'Oganesson')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;shows that both RDKit and PyMol can deal with oganesson.&#xA;However, this is just handling the atoms in coordinate space. But as soon as we touch it in molecular mechanics way things blow up:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from rdkit.Chem import AllChem&#xA;&#xA;AllChem.EmbedMolecule(mol)  # UFFTYPER: Unrecognized atom type: Og (0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But generally most molecular mechanics tools struggle with coordinated iron or similar. Take the mangled case of &lt;a href=&quot;https://www.rcsb.org/ligand/SIR&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/ligand/SIR&lt;/a&gt; which ought to have been planar...&lt;/p&gt;&#xA;&lt;h2&gt;PDB DB&lt;/h2&gt;&#xA;&lt;p&gt;In the PDB polymeric units and ligands are called chemical components.&#xA;They have a 3 letter code. The list is easily parsable.&#xA;The problem is that some lists are incomplete: &lt;a href=&quot;https://ftp.ebi.ac.uk/pub/databases/pdb/data/component-models/complete/chem_comp_model.cif.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ebi.ac.uk/pub/databases/pdb/data/component-models/complete/chem_comp_model.cif.gz&lt;/a&gt;&#xA;when read&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from pdbecif.mmcif_tools import MMCIF2Dict&#xA;models: dict = MMCIF2Dict().parse('chem_comp_model.cif')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;has a large number of entries missing because of &amp;lt;reasons&amp;gt;.&lt;/p&gt;&#xA;&lt;p&gt;So using a more crude approach and simply querying component by component:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;with open('chem_comp.list') as fh:  # this is from somewhere else in the ftp&#xA;    chem_resns = *map(str.strip, fh),&#xA;    &#xA;import requests&#xA;import re&#xA;elements = set()&#xA;for i in range(len(chem_resns)//100):&#xA;    response = requests.post('https://www.ebi.ac.uk/pdbe/api/pdb/compound/summary/', ','.join(list(mia)[100*i:100*(i+1)]))&#xA;    response.raise_for_status()&#xA;    for resn in response.json().values():&#xA;        for entry in resn:&#xA;            if entry['formula'] is None: # the unknown ligand/ions etc.&#xA;                continue&#xA;            elements.update(re.findall('[^\d ]+', entry['formula']))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Gives the following elements that are present:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{'Ag',&#xA; 'Al',&#xA; 'Am',&#xA; 'Ar',&#xA; 'As',&#xA; 'Au',&#xA; 'B',&#xA; 'Ba',&#xA; 'Be',&#xA; 'Bi',&#xA; 'Br',&#xA; 'C',&#xA; 'Ca',&#xA; 'Cd',&#xA; 'Ce',&#xA; 'Cf',&#xA; 'Cl',&#xA; 'Cm',&#xA; 'Co',&#xA; 'Cr',&#xA; 'Cs',&#xA; 'Cu',&#xA; 'D',&#xA; 'Dy',&#xA; 'Er',&#xA; 'Eu',&#xA; 'F',&#xA; 'Fe',&#xA; 'Ga',&#xA; 'Gd',&#xA; 'H',&#xA; 'Hf',&#xA; 'Hg',&#xA; 'Ho',&#xA; 'I',&#xA; 'In',&#xA; 'Ir',&#xA; 'K',&#xA; 'Kr',&#xA; 'La',&#xA; 'Li',&#xA; 'Lu',&#xA; 'Mg',&#xA; 'Mn',&#xA; 'Mo',&#xA; 'N',&#xA; 'Na',&#xA; 'Nd',&#xA; 'Ni',&#xA; 'O',&#xA; 'Os',&#xA; 'P',&#xA; 'Pb',&#xA; 'Pd',&#xA; 'Pr',&#xA; 'Pt',&#xA; 'Pu',&#xA; 'Rb',&#xA; 'Re',&#xA; 'Rh',&#xA; 'Ru',&#xA; 'S',&#xA; 'Sb',&#xA; 'Sc',&#xA; 'Se',&#xA; 'Si',&#xA; 'Sm',&#xA; 'Sn',&#xA; 'Sr',&#xA; 'Ta',&#xA; 'Tb',&#xA; 'Te',&#xA; 'Th',&#xA; 'Ti',&#xA; 'Tl',&#xA; 'U',&#xA; 'V',&#xA; 'W',&#xA; 'X',&#xA; 'Xe',&#xA; 'Y',&#xA; 'Yb',&#xA; 'Zn',&#xA; 'Zr'}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;While those that are absent are:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;pt = Chem.GetPeriodicTable()&#xA;full = set(map(pt.GetElementSymbol, range(1, 118+1)))&#xA;missing = full - elements&#xA;for m in sorted(missing, key=pt.GetAtomicNumber):&#xA;    print(pt.GetAtomicNumber(m), m)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;2 He&#xA;10 Ne&#xA;32 Ge&#xA;41 Nb&#xA;43 Tc&#xA;61 Pm&#xA;69 Tm&#xA;84 Po&#xA;85 At&#xA;86 Rn&#xA;87 Fr&#xA;88 Ra&#xA;89 Ac&#xA;91 Pa&#xA;93 Np&#xA;97 Bk&#xA;99 Es&#xA;100 Fm&#xA;101 Md&#xA;102 No&#xA;103 Lr&#xA;104 Rf&#xA;105 Db&#xA;106 Sg&#xA;107 Bh&#xA;108 Hs&#xA;109 Mt&#xA;110 Ds&#xA;111 Rg&#xA;112 Cn&#xA;113 Nh&#xA;114 Fl&#xA;115 Mc&#xA;116 Lv&#xA;117 Ts&#xA;118 Og&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="-1" LastEditDate="2022-06-30T19:42:22.480" LastActivityDate="2022-06-30T19:42:22.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19343" PostTypeId="2" ParentId="19334" CreationDate="2022-06-30T04:19:26.177" Score="2" Body="&lt;p&gt;Insulin, perhaps?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.rcsb.org/structure/1trz&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/structure/1trz&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;From &lt;a href=&quot;https://pdb101.rcsb.org/motm/14&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Insulin is a perfect molecule for exploring protein structure. It is small enough that you can display all of the atoms and still have a picture that is not too confusing. Human insulin is pictured here, using entry 1trz. This structure, like many other insulin structures, includes a hexameric complex of the hormone, and only one monomer (composed of one A-chain and one B-chain) is shown here. In the structure, you can see many of the key features that stabilize protein structure. Carbon-rich amino acids, like leucine and isoleucine, cluster in the middle of the molecule, forming a hydrophobic core, and the surface is covered with charged amino acids like arginine and glutamate that interact favorably with the surrounding water. Also notice the three disulfide bridges between cysteine amino acids, which stabilize this tiny protein.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-06-30T04:19:26.177" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19344" PostTypeId="2" ParentId="19339" CreationDate="2022-06-30T10:52:28.477" Score="1" Body="&lt;p&gt;You can set a specific compiler in &lt;code&gt;~/.R/Makevars&lt;/code&gt; file, create the folder and write it down in Makevers:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CC = gcc -std=c99&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CC = gcc -std=gnu99&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4282" LastActivityDate="2022-06-30T10:52:28.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19345" PostTypeId="2" ParentId="19339" CreationDate="2022-06-30T13:55:23.667" Score="3" Body="&lt;p&gt;The previous answer is correct, in that the compiler wants to be able to use &lt;code&gt;C99&lt;/code&gt; to compile your library.&lt;/p&gt;&#xA;&lt;p&gt;However, the solution of putting &lt;code&gt;CC = gcc -std=c99&lt;/code&gt; into your &lt;code&gt;~/.R/Makevars&lt;/code&gt; is a little bit blunt, since &lt;strong&gt;everything&lt;/strong&gt; that you now compile not directly from source is going to use the &lt;code&gt;-std=c99&lt;/code&gt; flag. This is a recipe for some confusing and hard to track down error messages. It's a bit like installing everything in your base conda environment and waiting for everything to break.&lt;/p&gt;&#xA;&lt;p&gt;It's a safer option to use &lt;code&gt;withr&lt;/code&gt;, which allows you to specify compiler flags for individual installations, keeping everything nice and neat. It works a bit like the &lt;code&gt;with&lt;/code&gt; (or &lt;code&gt;within&lt;/code&gt;) function in R which constructs an environment to execute some command in.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(withr)&#xA;&#xA;with_makevars(&#xA;    c(PKG_CFLAGS = &amp;quot;-std=c99&amp;quot;), &#xA;    BiocManager::install(&amp;quot;DiffBind&amp;quot;), &#xA;    assignment = &amp;quot;+=&amp;quot;&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It can also be performed directly with &lt;code&gt;BiocManager&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BiocManager::install(&#xA;    &amp;quot;DiffBind&amp;quot;, &#xA;    configure.vars = c(&amp;quot;CC&amp;quot; = &amp;quot;gcc&amp;quot;, &amp;quot;-std&amp;quot; = &amp;quot;c99&amp;quot;)&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="3694" LastEditDate="2022-06-30T15:22:39.543" LastActivityDate="2022-06-30T15:22:39.543" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19346" PostTypeId="1" CreationDate="2022-06-30T15:15:58.917" Score="2" ViewCount="69" Body="&lt;p&gt;I'm trying to run a simple task to pull multiple files via rsync with WDL. The task should be quite simple but how to pull the multiple files subsequently is giving me issues. Below is a snippet of my task and how it's being called:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;task MultiFiles {&#xA;  input {&#xA;    String wallclock&#xA;    String memory&#xA;    Int tmpspace_gb&#xA;    Int cpu&#xA;    Array[String] remote_files #files we want to pull&#xA;    Array[File] local_files&#xA;    String data_host &#xA;    Boolean? checksum_flag = false&#xA;&#xA;  }&#xA;    #Optionality for WDL&#xA;    String checksum = if checksum_flag == true then &amp;quot;--checksum&amp;quot; else &amp;quot;&amp;quot;&#xA;&#xA;  output {&#xA;    Array[File] output_files = &amp;quot;${local_files}&amp;quot;&#xA;  }&#xA;&#xA;  command &amp;lt;&amp;lt;&amp;lt;&#xA;&#xA;    set -e&#xA;&#xA;    #rsync command&#xA;    rsync -avP&#xA;      --log-file=rsync.log \&#xA;      ~{checksum} \&#xA;      &amp;quot;~{data_host}&amp;quot;:&amp;quot;~{remote_files}&amp;quot; \&#xA;      &amp;quot;~{local_files}&amp;quot;&#xA;&#xA;  &amp;gt;&amp;gt;&amp;gt;&#xA;  runtime {&#xA;    cpu: 4&#xA;    memory: &amp;quot;2GB&amp;quot;&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Call&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    call MultiFiles {&#xA;      input:&#xA;        #Task Resources&#xA;        remote_files=remote_files,&#xA;        data_host=data_host,&#xA;        local_files=local_files&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15427" LastEditorUserId="15427" LastEditDate="2022-06-30T15:16:23.340" LastActivityDate="2022-06-30T19:58:20.020" Title="Pulling Multiple Files via rsync with WDL" Tags="|wdl|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19347" PostTypeId="2" ParentId="19333" CreationDate="2022-06-30T19:40:06.570" Score="2" Body="&lt;p&gt;I would suggest that in the absence of &amp;quot;Complete&amp;quot; no very confident assertion can be made about gene presence/absence in a genome assembly. You need to actually  have the chromosome sequence to make strong statements.&lt;/p&gt;&#xA;&lt;p&gt;I think that most people (me included) would reluctantly accept that genomes with very good BUSCO or CheckM numbers (depending on your organism's taxonomy) are probably more or less accurate in terms of presence/absence. For metagenomics, I would extend that to the &lt;a href=&quot;https://www.nature.com/articles/nbt.3893&quot; rel=&quot;nofollow noreferrer&quot;&gt;MIMAG&lt;/a&gt; criteria, which are pretty stringent.&lt;/p&gt;&#xA;&lt;p&gt;These criteria are of course independent of the NCBI categories, which are pretty coarse-grained.&lt;/p&gt;&#xA;&lt;p&gt;That said, I think that you can make stronger arguments about gene presence/absence from the sequencing reads from which the genome assembly is derived. Find all the reads for your genome, map them to the genome assembly and also to the closest related ortholog of the gene in question. How well is the gene covered? How do the alignments look? Harder for metagenomics, but there are still probably tricks one could use.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-06-30T19:40:06.570" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19349" PostTypeId="2" ParentId="15645" CreationDate="2022-07-01T07:15:26.797" Score="1" Body="&lt;p&gt;You were trapped by arXiv versioning convention that shows you the &lt;strong&gt;latest&lt;/strong&gt; version of the preprint as default. It seems that this experiment was removed from the final publication.&lt;/p&gt;&#xA;&lt;p&gt;If you have a look at the v1 (&lt;a href=&quot;https://arxiv.org/abs/1610.02415v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1610.02415v1&lt;/a&gt;) instead of the latest version (v3), you find the explanation:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;As a simple example, we first attempt to maximize the water-octanol&#xA;partition coefficient &lt;span class=&quot;math-container&quot;&gt;$logP$&lt;/span&gt;, as estimated by RDkit. [43] &lt;span class=&quot;math-container&quot;&gt;$logP$&lt;/span&gt; is an&#xA;important element in characterizing the druglikeness of a molecule,&#xA;and is of interest in drug design.  To ensure that the resulting&#xA;molecules to be easy to synthesize in practice, we also incorporate&#xA;the synthetic accessibility [44] (SA) score into our objective. Our&#xA;initial experiments, optimizing only the &lt;span class=&quot;math-container&quot;&gt;$logP$&lt;/span&gt; and SA scores, produced&#xA;novel molecules, but ones having unrealistically large rings of carbon&#xA;atoms. To avoid this problem, we added a penalty for having carbon&#xA;rings of size larger than 6 to our objective. Thus our preferred&#xA;objective is, for a given molecule m, given by:&#xA;&lt;span class=&quot;math-container&quot;&gt;$J(m) = logP(m) − SA(m) − ring-penalty(m)$&lt;/span&gt;,          (1)&lt;/p&gt;&#xA;&lt;p&gt;where the scores &lt;span class=&quot;math-container&quot;&gt;$logP(m)$&lt;/span&gt;, &lt;span class=&quot;math-container&quot;&gt;$SA(m)$&lt;/span&gt;, and &lt;span class=&quot;math-container&quot;&gt;$ring-penalty(m)$&lt;/span&gt; are normalized to&#xA;have zero mean.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="14884" LastEditorUserId="3694" LastEditDate="2022-07-01T11:51:40.963" LastActivityDate="2022-07-01T11:51:40.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19350" PostTypeId="1" AcceptedAnswerId="19352" CreationDate="2022-07-01T13:49:46.600" Score="2" ViewCount="112" Body="&lt;p&gt;I'm a wet-lab biologist desperately trying to get to grips with some Bioinformatics.&lt;/p&gt;&#xA;&lt;p&gt;I have some shotgun Illumina paired-end data from a gut microbiome project I'm working on. I'm considering how to go about generating contigs from reads. I've been recommended to use SPAdes which I'm reading up on at the moment.&lt;/p&gt;&#xA;&lt;p&gt;Originally I was thinking of using FLASh however I was told it might not be suitable for metagenomic data.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My question simply is, what makes certain programs suitable for merging paired-end reads from a genome but not a metagenome?&lt;/strong&gt; I find myself a little spoilt for choice (eg. FLASh/SPAdes/BBmap... )  with bioinformatics tools I'm always concerned I'm picking the wrong one.&lt;/p&gt;&#xA;&lt;p&gt;Should I ideally be running my data through a couple of tools and comparing them?&lt;/p&gt;&#xA;" OwnerUserId="15713" LastActivityDate="2022-07-02T20:00:12.580" Title="Generating contigs from illumina Paired End Reads (Metagenome)" Tags="|metagenome|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19351" PostTypeId="1" CreationDate="2022-07-02T14:32:49.113" Score="2" ViewCount="431" Body="&lt;p&gt;I am trying to reproduce the results of the R tutorial of the WGCNA package.&lt;/p&gt;&#xA;&lt;p&gt;In section I number 5, when generating the heatmap it is quite similar to the one provided by the pdf but the colours of the heatmap differ from the document. Is it possible to change it?&lt;/p&gt;&#xA;&lt;p&gt;I attach the image together with the code used.&lt;/p&gt;&#xA;&lt;p&gt;[&lt;img src=&quot;https://i.stack.imgur.com/hufTa.png&quot; alt=&quot;image obtained&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Image Obtained&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zQR2m.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zQR2m.png&quot; alt=&quot;image of tutorial&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Image of tutorial&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;nSelect = 400&#xA;set.seed(10);&#xA;select = sample(nGenes, size = nSelect);&#xA;selectTOM = dissTOM[select, select];&#xA;selectTree = hclust(as.dist(selectTOM), method = &amp;quot;average&amp;quot;)&#xA;selectColors = moduleColors[select];&#xA;sizeGrWindow(9,9)&#xA;plotDiss = selectTOM^7;&#xA;diag(plotDiss) = NA;&#xA;TOMplot(plotDiss, selectTree, selectColors, main = &amp;quot;Network heatmap plot, selected genes&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15698" LastEditorUserId="650" LastEditDate="2022-12-01T18:56:38.097" LastActivityDate="2023-08-28T20:04:25.533" Title="Tutorial for the WGCNA: changes in heatmap colours" Tags="|rna-seq|wgcna|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19352" PostTypeId="2" ParentId="19350" CreationDate="2022-07-02T20:00:12.580" Score="2" Body="&lt;p&gt;NB: &amp;quot;merging PE reads&amp;quot; (such as e.g. FLASH does) is not the same as &amp;quot;sequence assembly from PE reads&amp;quot; (what MetaSPAdes does). The latter generates contigs, the former generates (some) merged shotgun library inserts. I think you want sequence assembly, i.e. a tool like MetaSPAdes.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Don't be tricked by the loose language of &amp;quot;merging&amp;quot; that is sometimes applied to the sequence assembly problem. It is much more complicated than that.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you want to learn more about sequence assembly, I suggest you read the &lt;a href=&quot;https://en.wikipedia.org/wiki/Sequence_assembly&quot; rel=&quot;nofollow noreferrer&quot;&gt;sequence assembly&lt;/a&gt; wikipedia page.&lt;/p&gt;&#xA;&lt;p&gt;I would probably recommend &lt;a href=&quot;https://github.com/voutcn/megahit&quot; rel=&quot;nofollow noreferrer&quot;&gt;MegaHit&lt;/a&gt; over metaSPAdes, because it requires much less memory / runtime, and lower frequency of errors.&lt;/p&gt;&#xA;&lt;p&gt;But generally speaking, both programs are building different flavors of &lt;a href=&quot;https://www.cs.jhu.edu/%7Elangmea/resources/lecture_notes/assembly_dbg.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;de Bruijn Graphs&lt;/a&gt; from metagenomic reads to generate contigs/scaffolds. This is the accepted method by this point for Illumina metagenomic data, because it is the method that is capable of handling the amount of small pieces of data in a tractable fashion. Alternatives such as &lt;a href=&quot;https://www.cs.jhu.edu/%7Elangmea/resources/lecture_notes/assembly_olc.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;overlap-layout-consensus graphs&lt;/a&gt; or &lt;a href=&quot;https://bioinformaticshome.com/tools/wga/descriptions/SGA.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;string graphs&lt;/a&gt; just don't work as well for the metagenomic problem.&lt;/p&gt;&#xA;&lt;p&gt;You are probably looking for methods that talk about complicated, computationally complex assembly algorithms and graphical methods for summarizing lots of short reads. You are not looking for simple read processing/alignment tools like BBMap/BBTools or FLASH. &lt;strong&gt;To reiterate, simple read merging tools will not give you assembled contigs.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-07-02T20:00:12.580" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="19353" PostTypeId="2" ParentId="19331" CreationDate="2022-07-03T05:28:07.153" Score="2" Body="&lt;p&gt;If you have a reference genome (or are willing to designate one of your de novo assemblies a reference), you may find &lt;a href=&quot;http://quast.sourceforge.net/quast.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;QUAST&lt;/a&gt; helpful.&lt;/p&gt;&#xA;&lt;p&gt;QUAST will perform an alignment of all genomes against the reference genome and use that as the basis for comparisons. I believe that &lt;a href=&quot;https://github.com/ablab/quast/blob/master/quast_libs/minimap2/minimap2.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;under the hood it is running minimap2&lt;/a&gt; for this purpose.&lt;/p&gt;&#xA;&lt;p&gt;It includes a large-scale alignment viewer (&amp;quot;&lt;a href=&quot;http://quast.sourceforge.net/quast.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Icarus&lt;/a&gt;&amp;quot;) to investigate possible differences within contigs, computes high-level statistics on genome completeness/etc, and has other handy features.&lt;/p&gt;&#xA;&lt;p&gt;You can see the &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/34/13/i142/5045727?login=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;most recent QUAST paper&lt;/a&gt; or the &lt;a href=&quot;http://quast.sourceforge.net/docs/manual.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt; for more information.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2022-07-04T20:07:50.577" LastActivityDate="2022-07-04T20:07:50.577" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19356" PostTypeId="1" AcceptedAnswerId="19357" CreationDate="2022-07-04T11:26:02.380" Score="4" ViewCount="106" Body="&lt;p&gt;I am trying to download the &lt;code&gt;_protein.faa.gz&lt;/code&gt; files for genomes given their accession numbers through Python. Ideally, I would like to do this without third party libraries. Essentially what I have is a list of just the GCA or GCF accessions. The issue with the ftp site is that it includes the project name along with the accession, which I do not have ahead of time. It would be perfect if I could run downloads from within Python that matched a pattern something like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/145/295/GCF_000145295.1_*/*_protein.faa.gz&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is it at all possible to do this? Or is it possible to run something similar &lt;em&gt;via&lt;/em&gt; efetch in Biopython?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="15719" LastActivityDate="2022-09-20T15:18:42.843" Title="Downloading genomic protein files from accessions in Python" Tags="|python|phylogenetics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19357" PostTypeId="2" ParentId="19356" CreationDate="2022-07-04T15:33:40.647" Score="1" Body="&lt;p&gt;Concept code only&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez&#xA;&#xA;Entrez.email = &amp;quot;m@M__&amp;quot;&#xA;handle = Entrez.efetch(db=&amp;quot;protein&amp;quot;, id=&amp;quot;GCF_000145295&amp;quot;, &amp;quot;GCF_000145294&amp;quot;, &amp;quot;GCF_000145293&amp;quot;, retmode=&amp;quot;text&amp;quot;,rettype=&amp;quot;gb&amp;quot;) # check the quotes&#xA;records = Entrez.parse(handle)&#xA;handle.close()&#xA;for record in records:&#xA;    print (record)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can write a loop, flatten it and then insert it to the &lt;code&gt;id=&lt;/code&gt; tag.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mylist = ['GCF_000145295', 'GCF_000145294', 'GCF_000145293']&#xA;flattened = ','.join(mylist)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-07-04T21:31:14.373" LastActivityDate="2022-07-04T21:31:14.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19358" PostTypeId="2" ParentId="13872" CreationDate="2022-07-06T04:11:22.860" Score="0" Body="&lt;p&gt;You can do this in one line using &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI's EntrezDirect e-utils&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;esearch -db sra -query PRJNA506829 | efetch -format runinfo&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The output will be in csv, but you can format it to tab using &lt;a href=&quot;https://pypi.org/project/csvkit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;csvkit's&lt;/a&gt; &lt;code&gt;csvformat -T&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3381" LastActivityDate="2022-07-06T04:11:22.860" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19359" PostTypeId="1" CreationDate="2022-07-06T19:28:38.637" Score="2" ViewCount="222" Body="&lt;p&gt;I have been attempting to extract particular clusters, given by hierarchical clustering outputs from the &lt;code&gt;pheatmap&lt;/code&gt; function (in R).&lt;/p&gt;&#xA;&lt;p&gt;Please find the &lt;code&gt;dput&lt;/code&gt; output of the dataset in question below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(`72v24.logFC` = c(-1.47511280138092, -1.10386310802134, &#xA;-1.14476608274903, -1.63563574551239, -0.895377907265352, -1.35216488439027, &#xA;-2.04949381135359, -1.0344565728341, -0.967008696515716, -1.11233305998901, &#xA;-2.02663681865178, -1.74756562789736, -1.02076838668592, -1.27981009782201, &#xA;-1.13181804725663, -1.19537930856639, -2.23476188503608, -1.67500131867632, &#xA;-0.950059974858215, -2.3124598902615, -1.93757457440581, -3.24602437386808, &#xA;-0.839869104053795, -1.40329424193006, -1.63484798229567, -0.912148120310492, &#xA;-1.33467209346367, -1.27031027753064, -0.925498608058671, -0.802956638081314, &#xA;-0.85457796397113, -1.0591348681193, -0.825278459132589, -0.671570119524199, &#xA;-1.8439768319437, -0.947033608766851, -1.63849733567277, -1.40584263305038, &#xA;-1.1317958487115, -1.40724695083764, -1.21842141154588, -2.29248800050516, &#xA;-1.41699787526721, -0.879218050664484, -2.25064962332145, 1.4685855647974, &#xA;-2.02553848518381, 1.32808748053321, -2.46062484389627, -2.45830217362141, &#xA;-0.880536960021976, -0.689060222340137, -0.863262774594, -1.64961312646758, &#xA;1.10572346456398, -0.891760767302047, -1.22716047163938, -0.680697741299578, &#xA;-1.24551344420046, -0.891918251648935, -1.64722787848732, -0.832845262108387, &#xA;-0.937129577660045, -0.891684122691785, 1.96452911491088, -1.59420664438654, &#xA;-0.921295293242016, -1.8304480291305, -0.632859040349798, 1.69982819847455, &#xA;-0.982668795391973, -1.61816842714933, -1.23602172818196, -2.15356245206668, &#xA;-1.67961932563564, 1.15478114880933, 1.32665444151868, -1.04590896017973, &#xA;-2.29770026795487, -1.27498150009975, -1.36310217581123, -1.09169550738617, &#xA;1.5349413586602, -1.66842984591719, -0.760122936706606, -1.10054393155051, &#xA;-1.00344072779106, -0.993259123487476, -1.43136824963825, -1.07214189069438, &#xA;-0.990813303020613, -1.51324812740189, -1.23465012540084, -1.00098231964124, &#xA;-1.92195764952845, -1.00139179559519, -0.940118366042393, -2.35969198434574, &#xA;-0.574626090826994, -0.884870428965888, -0.87601460107501, 1.61336157755029, &#xA;-0.777670642457121, -0.7312214522498, -0.812926777785119, -1.13443421373561, &#xA;-1.56498420350444, -0.69126972981765, -1.44351026281231, -1.21962275860236, &#xA;-2.21155555407497, -1.02015250027788, -0.604620179871691, -1.53013572266942, &#xA;-0.790050747397837, -1.17950736026735, -0.781075150161975, -0.888554144538286, &#xA;3.59172865862026, -0.8665071699621, -2.97701792242157, -1.44424941630738, &#xA;-1.50528882788757, -0.772465957539335, -1.57664185213987, -1.02371443439054, &#xA;-0.931807409493221, 1.77728969821973, -1.05455101137715, -1.09765136526264, &#xA;-0.881436416213335, -0.985605985458853, -1.31644334991692, 2.26917914828263, &#xA;-1.43844640540614, -1.75106843701475, -1.22483991752173, -1.65032906954133, &#xA;-0.686319914242903, 2.45473055753775, -1.15594379837188, 1.46397122502826, &#xA;-0.91260428248088, -0.726540148411507, -1.18227838019011, -0.940722690371935, &#xA;-1.28074043595918, -0.629033607437977, 1.01053893402133, -0.735590704487842, &#xA;-2.0620695625823, -0.692251638781327, -0.717818968339447, -0.564023101160995, &#xA;1.51541391124355, -1.89444914881727, -0.737931396419173, -1.17763728235316, &#xA;-0.887605568116903, -2.8392997898765, -1.48018352596355, 1.60698486226092, &#xA;-1.96804216410227, -0.881437183709851, 3.08730764755417, -0.600282670155725, &#xA;-0.905719394102218, -1.05553935946213, -1.13508709782814, -0.613381276596874, &#xA;-0.642647139688194, -0.861929974132332, -1.30652597213552, -1.73647794867498, &#xA;-0.619909931049252, -0.880145154869759, -0.60084263914673, -1.16825573255754, &#xA;-0.942919502394457, -1.3709930239358, -0.80595477526712, -0.628115199442632, &#xA;-1.11502998682131, -1.01274335471793, -1.16595378001606, -3.36098944183559, &#xA;-1.19617345920841, -2.10811718229285, 3.33752122936, -1.67696701892834, &#xA;-1.90639199489612, -0.574737209544065, -1.66853541929234, -0.595093777485911, &#xA;-0.7964813653225, -0.791102813817634, -0.601987407016077, -1.52559881980829, &#xA;1.15480324851026, -0.830167728660421), `72v0.logFC` = c(-5.10937036648522, &#xA;-8.41670756910048, -8.18671783299995, -2.79415692883273, -3.62090016917187, &#xA;-8.37523504570539, -3.79100494829738, -7.20659380156151, -5.46135634576091, &#xA;3.43493215839382, -1.67650075643956, -2.99391647274475, -5.77357941089076, &#xA;-5.22675557853878, 3.26436094268851, -5.98693257837598, -5.85864442290848, &#xA;-2.39995052889286, -5.19221784614079, -4.72634343497625, -7.46407809296449, &#xA;-3.04588366694082, -3.85014181238163, 4.44756662720351, -4.42031795110716, &#xA;-2.58478993868624, -5.46121416816988, -2.09364835701952, -5.42397023596482, &#xA;-5.26192471384563, -4.48979100280513, -5.14171942073048, -5.5267515390142, &#xA;3.88762033289334, -5.88489034628499, -2.85490179670763, -5.31586479048071, &#xA;-2.64877712735579, -6.24766069294439, -5.35826818217055, -4.97204642950896, &#xA;-2.82989963758335, -4.75016577988789, -3.51684070536132, -3.20931950649022, &#xA;-3.62394584217955, -2.81449967053025, -5.68308664428684, -3.11500402887324, &#xA;-4.08965188145047, -5.43176189260205, -4.30737780091769, 2.52666133143524, &#xA;-3.88213803962076, -5.517506523237, -3.27502948978619, -5.37724717693793, &#xA;2.77090845859761, -4.22554364178978, -5.81944792042417, -2.23700806354102, &#xA;-1.81804660477942, -3.45613077773548, -4.88444075082709, -5.48877199958173, &#xA;-4.90077849970695, -3.79304730634383, -0.953597409660203, -5.573920008042, &#xA;-1.83250444732307, -1.73248469438834, -2.35460087734472, -4.43029236446579, &#xA;-3.37654864236982, -3.63410171518379, -2.78276966423079, -2.01664452557169, &#xA;-3.54834187597395, -2.04395822264753, -2.15693095907293, -4.04759665569609, &#xA;-4.87928960250415, 1.1358557257212, -4.89657506999093, -2.42698466771808, &#xA;-4.55703295523896, -2.53230009478764, -1.74571568381108, 1.93875826054123, &#xA;-2.29178704384166, -1.88631492984573, -4.17811684330152, -4.55155213716622, &#xA;1.93093495783903, -3.66429213603731, 2.73817157445051, -3.96977406502995, &#xA;-4.18487497756956, 2.17512996773018, -4.41168757499049, -2.96267804428878, &#xA;-4.60683474159913, -3.0573109426718, -1.59500391193586, -4.65882130908176, &#xA;-2.68903554919404, -4.44676809434802, 1.60275224879078, -3.83760729167497, &#xA;-3.09907199540008, -2.37905633686461, -4.29650784478684, -1.88615561094275, &#xA;-4.44026540480975, -2.01996858091239, -2.46842498723876, -4.24053886261736, &#xA;-4.32561109033911, -2.96097578889787, -3.76708395833968, -2.45229520135946, &#xA;-2.33856203921674, -4.23241140779399, -3.48627475173498, 2.25888018977625, &#xA;-3.85271202182208, -4.61317704601435, -1.95186634348091, 1.3205477461731, &#xA;-2.51561846705872, -1.7031812569795, -4.44717736673796, -3.88478215160905, &#xA;-3.27547652595564, -2.81690169904308, -4.83981492357284, -2.4486942125238, &#xA;-4.31360736969916, -4.22122930700826, -3.49045260391997, -2.87862947524065, &#xA;-4.14268228866617, -2.62252967420329, -4.23902905919912, -4.14777621540367, &#xA;-2.86056694414781, -1.70668584273452, 1.3624660569909, -2.59153842674144, &#xA;-4.12393693326418, -2.12959469550212, 1.46923996462969, -1.98633421258236, &#xA;3.22176343046404, -3.52110060349997, -2.69900299957347, -4.20126962458439, &#xA;-4.68116989520068, 1.57592717464553, -3.93999603065316, -2.79446235029201, &#xA;1.53128139579193, 2.65531461899762, -1.33692502254506, -1.40309746138702, &#xA;7.1503230589606, -3.03232696595877, -3.99799016693196, 1.36106448869886, &#xA;2.48321747034381, -1.81760324077147, -2.93792828398095, -4.24318581706171, &#xA;-5.02935968459404, -2.8067837982494, -3.18182441332852, -3.17383228514224, &#xA;-3.75210155230786, -3.87059449392306, -3.38923922246314, -3.8492188722049, &#xA;-4.54207754798088, -3.65594980956956, -4.083692310451, 1.30846845334021, &#xA;-2.14067263149232, 0.87594188820449, -2.36549598411797, -1.6026649015159, &#xA;3.06122694909985, -2.80125769402892, -1.21061533381979, 2.45621751938726, &#xA;-1.29479960958983, -1.32215076834374, -3.40665434275434, -3.87581533996246, &#xA;2.25817917663687, -4.47531796954113, -3.0627832955887), `24v0.logFC` = c(-5.63038870666508, &#xA;-6.44381761773614, -9.71855520117086, 2.83617729007977, -3.39220401105853, &#xA;-9.03206578896647, 4.72316302034493, 2.69600384046815, -3.70005627557764, &#xA;1.57004091545159, -2.69837465613229, -1.96398920017231, -6.78881951127235, &#xA;2.88395305274553, 2.2815237548239, 4.26299665984366, 1.80759581466942, &#xA;-6.13159865127454, 2.31984691801986, 2.42017005000243, 1.90522291464817, &#xA;-6.08799743507621, 1.93712248733409, -2.85900752558212, -2.53314737897312, &#xA;-5.39825301043452, -2.67723981805717, 1.99371094854688, -6.30018858053764, &#xA;-5.51269257077614, 1.22241656195939, 2.21258569930309, -3.34360626488873, &#xA;2.84622416535537, 1.62066860166637, -4.78372642162402, -5.70058663575352, &#xA;-5.38484766744183, -5.09622306588101, 1.4720608195385, 2.81999896316429, &#xA;-5.00082275569591, -4.22688753149231, 3.13575564914666, -3.65108053066376, &#xA;-4.96357882349085, 1.41296031244116, 2.02777799934303, -2.69828900301196, &#xA;3.3280993785528, -4.91524305899842, 1.33559456145784, -3.20704073441948, &#xA;3.4589509748052, -4.76220190139647, -5.41080867153898, -2.52368474702804, &#xA;2.35227625126478, -1.66843249760588, -5.98409816921998, 1.97783827997787, &#xA;-5.05713668502354, -5.66050145489872, -3.23578154397873, -5.47590590732768, &#xA;1.79478248294125, 1.53715323981994, 1.17924158724962, -3.38135082340585, &#xA;-6.0533528170201, 3.64133939415158, -5.18911327526881, 1.12836935335145, &#xA;-5.50962223997537, -5.13018380364927, 2.43347308480167, 3.09821960202745, &#xA;1.08371028206663, 1.15073045150599, 1.66618928654645, -4.57926216582183, &#xA;-4.39710926752864, 2.29767598663356, 1.11369483753166, 1.90655785334723, &#xA;-4.57402537033236, -1.64795888694238, -4.90903507231585, -4.69225607476014, &#xA;-3.78658812613927, 2.71158710397443, 2.0014280958994, 1.42976609883019, &#xA;1.69469179645011, 2.02542067939079, 1.29620189704961, -2.11262483065018, &#xA;-4.45917113273047, -5.00209626672923, -2.22313144605634, 2.28015100958449, &#xA;2.99018492359164, -4.55487718731015, -4.4548605761304, -2.74564764773372, &#xA;1.50453692418261, -1.80916071906589, -3.78464149018, 5.44303553967597, &#xA;1.63876298449708, -2.55262897048338, -4.52516489294599, 1.86420921266273, &#xA;1.92394285952202, -1.87942876800687, -4.72261410600152, -2.57805004195914, &#xA;-4.47016439150427, -2.79823793519691, 1.49664827810044, -5.12377787039451, &#xA;-1.64302437544027, -2.32977998370269, -1.54481646746192, 1.51575792656483, &#xA;-1.66914425228449, -5.67499677983564, -2.41648326328024, 1.4178331810756, &#xA;1.34477707145884, 1.6168542644666, 1.51101526992765, 2.85584943327062, &#xA;-4.19149110174371, 1.69988651757158, 2.42852927065545, 1.66468253437414, &#xA;3.21304236161755, 2.52923647014318, -3.32575501787469, 1.32217968706286, &#xA;-4.3404208961354, 1.60065358035513, 1.11559852847827, 2.24622871386089, &#xA;-2.07936338453629, -4.096641542765, -1.81072034410725, -1.71317603678557, &#xA;3.48120770906848, -3.87801910383696, -4.08020770103266, 1.11478378387699, &#xA;-3.46079055745071, 1.7322630648363, 1.54861321913459, 1.96850013686628, &#xA;2.50282552968501, -1.81077760463368, 2.02487013804019, 1.95047564703855, &#xA;-2.99930567848563, -1.79193606152703, -2.54724389366544, -4.15112678166129, &#xA;1.13069494609896, -1.43241741260756, 2.36383913233529, 1.65786717131242, &#xA;-2.29507370488367, 1.5786351022915, -3.31042999789919, 2.582031624397, &#xA;-3.62438687496853, -4.20056844288066, 1.59787844454997, -4.39077226372811, &#xA;-4.50758458577604, -3.5310616440773, 1.34960913912344, 1.56097908933761, &#xA;1.80910781056029, -4.1248594098749, 1.5318048796277, -3.97753680837585, &#xA;-2.80501778688795, -3.20129716412254, -1.98305552020422, -4.25045728757901, &#xA;-4.08742040090535, -4.31190449886363, 1.39895836419766, 1.67102855271528, &#xA;-5.13520809951127, -4.31701572736756, 0.832778953035371, 2.30899115735356, &#xA;-3.18620970041199, -3.80052092618969, 1.59131320240714)), row.names = c(&amp;quot;ENSMUSG00000097417.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000084950.7&amp;quot;, &amp;quot;ENSMUSG00000021824.12&amp;quot;, &amp;quot;ENSMUSG00000085505.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000018381.15&amp;quot;, &amp;quot;ENSMUSG00000098716.1&amp;quot;, &amp;quot;ENSMUSG00000092134.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000097272.1&amp;quot;, &amp;quot;ENSMUSG00000039206.13&amp;quot;, &amp;quot;ENSMUSG00000097860.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000047880.14&amp;quot;, &amp;quot;ENSMUSG00000087700.1&amp;quot;, &amp;quot;ENSMUSG00000094127.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000101162.2&amp;quot;, &amp;quot;ENSMUSG00000079179.9&amp;quot;, &amp;quot;ENSMUSG00000107317.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000102451.1&amp;quot;, &amp;quot;ENSMUSG00000098609.1&amp;quot;, &amp;quot;ENSMUSG00000109669.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000086836.1&amp;quot;, &amp;quot;ENSMUSG00000092545.7&amp;quot;, &amp;quot;ENSMUSG00000106093.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000090625.1&amp;quot;, &amp;quot;ENSMUSG00000009585.17&amp;quot;, &amp;quot;ENSMUSG00000047988.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000027950.7&amp;quot;, &amp;quot;ENSMUSG00000086430.1&amp;quot;, &amp;quot;ENSMUSG00000105941.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000004071.12&amp;quot;, &amp;quot;ENSMUSG00000104208.1&amp;quot;, &amp;quot;ENSMUSG00000022223.8&amp;quot;, &#xA;&amp;quot;ENSMUSG00000031145.15&amp;quot;, &amp;quot;ENSMUSG00000047369.15&amp;quot;, &amp;quot;ENSMUSG00000097347.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000086680.2&amp;quot;, &amp;quot;ENSMUSG00000097660.1&amp;quot;, &amp;quot;ENSMUSG00000002109.14&amp;quot;, &#xA;&amp;quot;ENSMUSG00000108524.1&amp;quot;, &amp;quot;ENSMUSG00000067786.16&amp;quot;, &amp;quot;ENSMUSG00000021485.13&amp;quot;, &#xA;&amp;quot;ENSMUSG00000097768.1&amp;quot;, &amp;quot;ENSMUSG00000085437.8&amp;quot;, &amp;quot;ENSMUSG00000086220.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000075576.12&amp;quot;, &amp;quot;ENSMUSG00000102299.1&amp;quot;, &amp;quot;ENSMUSG00000079293.11&amp;quot;, &#xA;&amp;quot;ENSMUSG00000090952.2&amp;quot;, &amp;quot;ENSMUSG00000040723.14&amp;quot;, &amp;quot;ENSMUSG00000027329.9&amp;quot;, &#xA;&amp;quot;ENSMUSG00000106005.1&amp;quot;, &amp;quot;ENSMUSG00000085425.1&amp;quot;, &amp;quot;ENSMUSG00000046027.17&amp;quot;, &#xA;&amp;quot;ENSMUSG00000006517.5&amp;quot;, &amp;quot;ENSMUSG00000004221.16&amp;quot;, &amp;quot;ENSMUSG00000049775.16&amp;quot;, &#xA;&amp;quot;ENSMUSG00000109881.1&amp;quot;, &amp;quot;ENSMUSG00000023988.8&amp;quot;, &amp;quot;ENSMUSG00000092371.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000087611.1&amp;quot;, &amp;quot;ENSMUSG00000102349.1&amp;quot;, &amp;quot;ENSMUSG00000092103.2&amp;quot;, &#xA;&amp;quot;ENSMUSG00000089636.1&amp;quot;, &amp;quot;ENSMUSG00000086360.1&amp;quot;, &amp;quot;ENSMUSG00000036138.16&amp;quot;, &#xA;&amp;quot;ENSMUSG00000045382.6&amp;quot;, &amp;quot;ENSMUSG00000090081.1&amp;quot;, &amp;quot;ENSMUSG00000107999.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000041774.14&amp;quot;, &amp;quot;ENSMUSG00000030720.16&amp;quot;, &amp;quot;ENSMUSG00000027398.13&amp;quot;, &#xA;&amp;quot;ENSMUSG00000090246.1&amp;quot;, &amp;quot;ENSMUSG00000104823.1&amp;quot;, &amp;quot;ENSMUSG00000032528.4&amp;quot;, &#xA;&amp;quot;ENSMUSG00000100783.1&amp;quot;, &amp;quot;ENSMUSG00000082848.1&amp;quot;, &amp;quot;ENSMUSG00000021250.13&amp;quot;, &#xA;&amp;quot;ENSMUSG00000009292.17&amp;quot;, &amp;quot;ENSMUSG00000109491.1&amp;quot;, &amp;quot;ENSMUSG00000090252.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000087443.1&amp;quot;, &amp;quot;ENSMUSG00000110148.1&amp;quot;, &amp;quot;ENSMUSG00000085754.7&amp;quot;, &#xA;&amp;quot;ENSMUSG00000064339.1&amp;quot;, &amp;quot;ENSMUSG00000091160.1&amp;quot;, &amp;quot;ENSMUSG00000108332.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000108321.1&amp;quot;, &amp;quot;ENSMUSG00000048721.2&amp;quot;, &amp;quot;ENSMUSG00000032363.15&amp;quot;, &#xA;&amp;quot;ENSMUSG00000003868.14&amp;quot;, &amp;quot;ENSMUSG00000107075.1&amp;quot;, &amp;quot;ENSMUSG00000102961.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000096957.1&amp;quot;, &amp;quot;ENSMUSG00000093006.2&amp;quot;, &amp;quot;ENSMUSG00000109754.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000027774.16&amp;quot;, &amp;quot;ENSMUSG00000090016.1&amp;quot;, &amp;quot;ENSMUSG00000012126.16&amp;quot;, &#xA;&amp;quot;ENSMUSG00000074136.9&amp;quot;, &amp;quot;ENSMUSG00000089875.1&amp;quot;, &amp;quot;ENSMUSG00000092190.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000074213.2&amp;quot;, &amp;quot;ENSMUSG00000086564.9&amp;quot;, &amp;quot;ENSMUSG00000055725.11&amp;quot;, &#xA;&amp;quot;ENSMUSG00000028451.12&amp;quot;, &amp;quot;ENSMUSG00000078786.9&amp;quot;, &amp;quot;ENSMUSG00000092655.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000098482.1&amp;quot;, &amp;quot;ENSMUSG00000087442.1&amp;quot;, &amp;quot;ENSMUSG00000108574.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000087669.1&amp;quot;, &amp;quot;ENSMUSG00000053442.4&amp;quot;, &amp;quot;ENSMUSG00000101674.6&amp;quot;, &#xA;&amp;quot;ENSMUSG00000078653.4&amp;quot;, &amp;quot;ENSMUSG00000108353.1&amp;quot;, &amp;quot;ENSMUSG00000097748.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000050010.8&amp;quot;, &amp;quot;ENSMUSG00000029623.14&amp;quot;, &amp;quot;ENSMUSG00000097060.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000050627.13&amp;quot;, &amp;quot;ENSMUSG00000092386.1&amp;quot;, &amp;quot;ENSMUSG00000085471.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000087607.1&amp;quot;, &amp;quot;ENSMUSG00000061607.14&amp;quot;, &amp;quot;ENSMUSG00000026729.9&amp;quot;, &#xA;&amp;quot;ENSMUSG00000097854.1&amp;quot;, &amp;quot;ENSMUSG00000064330.9&amp;quot;, &amp;quot;ENSMUSG00000039753.16&amp;quot;, &#xA;&amp;quot;ENSMUSG00000027706.8&amp;quot;, &amp;quot;ENSMUSG00000029360.3&amp;quot;, &amp;quot;ENSMUSG00000103864.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000085208.3&amp;quot;, &amp;quot;ENSMUSG00000073628.5&amp;quot;, &amp;quot;ENSMUSG00000102776.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000032487.8&amp;quot;, &amp;quot;ENSMUSG00000086998.1&amp;quot;, &amp;quot;ENSMUSG00000006728.12&amp;quot;, &#xA;&amp;quot;ENSMUSG00000021273.9&amp;quot;, &amp;quot;ENSMUSG00000062591.5&amp;quot;, &amp;quot;ENSMUSG00000040936.14&amp;quot;, &#xA;&amp;quot;ENSMUSG00000002985.16&amp;quot;, &amp;quot;ENSMUSG00000046145.7&amp;quot;, &amp;quot;ENSMUSG00000038274.11&amp;quot;, &#xA;&amp;quot;ENSMUSG00000087366.7&amp;quot;, &amp;quot;ENSMUSG00000039452.5&amp;quot;, &amp;quot;ENSMUSG00000087660.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000087206.1&amp;quot;, &amp;quot;ENSMUSG00000079355.3&amp;quot;, &amp;quot;ENSMUSG00000086693.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000030579.10&amp;quot;, &amp;quot;ENSMUSG00000084878.1&amp;quot;, &amp;quot;ENSMUSG00000108235.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000097077.1&amp;quot;, &amp;quot;ENSMUSG00000101438.1&amp;quot;, &amp;quot;ENSMUSG00000087367.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000064351.1&amp;quot;, &amp;quot;ENSMUSG00000086312.1&amp;quot;, &amp;quot;ENSMUSG00000097221.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000107090.1&amp;quot;, &amp;quot;ENSMUSG00000037982.16&amp;quot;, &amp;quot;ENSMUSG00000095280.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000017837.12&amp;quot;, &amp;quot;ENSMUSG00000029422.14&amp;quot;, &amp;quot;ENSMUSG00000102564.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000026749.11&amp;quot;, &amp;quot;ENSMUSG00000067288.12&amp;quot;, &amp;quot;ENSMUSG00000025353.4&amp;quot;, &#xA;&amp;quot;ENSMUSG00000109251.1&amp;quot;, &amp;quot;ENSMUSG00000086600.8&amp;quot;, &amp;quot;ENSMUSG00000052214.9&amp;quot;, &#xA;&amp;quot;ENSMUSG00000090330.2&amp;quot;, &amp;quot;ENSMUSG00000090009.1&amp;quot;, &amp;quot;ENSMUSG00000059791.14&amp;quot;, &#xA;&amp;quot;ENSMUSG00000101695.1&amp;quot;, &amp;quot;ENSMUSG00000103698.1&amp;quot;, &amp;quot;ENSMUSG00000097472.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000084128.10&amp;quot;, &amp;quot;ENSMUSG00000026156.8&amp;quot;, &amp;quot;ENSMUSG00000075297.10&amp;quot;, &#xA;&amp;quot;ENSMUSG00000055692.18&amp;quot;, &amp;quot;ENSMUSG00000038502.16&amp;quot;, &amp;quot;ENSMUSG00000015942.9&amp;quot;, &#xA;&amp;quot;ENSMUSG00000106478.1&amp;quot;, &amp;quot;ENSMUSG00000039307.16&amp;quot;, &amp;quot;ENSMUSG00000054556.6&amp;quot;, &#xA;&amp;quot;ENSMUSG00000045576.16&amp;quot;, &amp;quot;ENSMUSG00000037936.15&amp;quot;, &amp;quot;ENSMUSG00000087486.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000082180.1&amp;quot;, &amp;quot;ENSMUSG00000014470.4&amp;quot;, &amp;quot;ENSMUSG00000056078.5&amp;quot;, &#xA;&amp;quot;ENSMUSG00000049928.15&amp;quot;, &amp;quot;ENSMUSG00000013997.10&amp;quot;, &amp;quot;ENSMUSG00000105198.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000086714.1&amp;quot;, &amp;quot;ENSMUSG00000109274.1&amp;quot;, &amp;quot;ENSMUSG00000100548.1&amp;quot;, &#xA;&amp;quot;ENSMUSG00000053706.2&amp;quot;, &amp;quot;ENSMUSG00000109854.1&amp;quot;, &amp;quot;ENSMUSG00000060802.8&amp;quot;, &#xA;&amp;quot;ENSMUSG00000093598.1&amp;quot;), class = &amp;quot;data.frame&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Please note, I have checked out many posts regarding this, most notably: &lt;a href=&quot;https://www.biostars.org/p/287512/#287518&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/287512/#287518&lt;/a&gt;, which highlights how to perform this procedure. Regardless, there are some unanswered questions.&lt;/p&gt;&#xA;&lt;p&gt;The problem is the following: After running the code suggested in the post, I do indeed obtain x number of clusters, depending on the height I decide to cut the tree at. However, they are in an incomprehensible order, making it extremely difficult to extract the clusters as they are observed in the heatmap. Please note, the dataframe I have uploaded is only a snippet of the true data - and is called logFC_mat, in the code below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pheatmap(logFC_mat, &#xA;         show_rownames=F, cluster_cols=F, cluster_rows=T, scale=&amp;quot;row&amp;quot;,&#xA;         cex=1, clustering_distance_rows=&amp;quot;euclidean&amp;quot;, cex=1,&#xA;         clustering_distance_cols=&amp;quot;euclidean&amp;quot;, &#xA;         clustering_method=&amp;quot;complete&amp;quot;, border_color=FALSE) -&amp;gt; heatof&#xA;&#xA;#make clusters based on h=1.5&#xA;sort(cutree(heatof$tree_row, h=1.5)) -&amp;gt; hi_clusters&#xA;&#xA;#check if clusters are in same order as in heatmap&#xA;hi_clusters[hi_clusters==1] %&amp;gt;% names() -&amp;gt; clust1&#xA;hi_clusters[hi_clusters==2] %&amp;gt;% names() -&amp;gt; clust2&#xA;&#xA;length(clust1) #56&#xA;length(clust2) #78&#xA;all(clust1 %in% &#xA;      rownames(logFC_mat[heatof&lt;span class=&quot;math-container&quot;&gt;$tree_row$&lt;/span&gt;order[1:56],])) #TRUE&#xA;&#xA;all(clust2 %in%&#xA;      rownames(logFC_mat[heatof&lt;span class=&quot;math-container&quot;&gt;$tree_row$&lt;/span&gt;order[57:134],])) #FALSE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, from visual inspection of the heatmap, it becomes clear that cluster 2 is in fact the last 78 genes rather than the middle cluster. The problem is, why is it labeled as cluster 2 then !? In addition, this is a snippet of the data. The real data contains in fact thousands of rows, and so such visual deductions can simply be impossible.&lt;/p&gt;&#xA;&lt;p&gt;Now for the question: How to extract the clusters, such that they are in the SAME ORDER, as what is seen in the heatmap (Top to bottom)?&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="4001" LastEditDate="2022-07-08T13:48:25.610" LastActivityDate="2022-07-08T13:48:25.610" Title="Extracting clusters, hierarchical clustering - bulk RNASeq" Tags="|r|rna-seq|clustering|unsupervised-learning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19360" PostTypeId="2" ParentId="19242" CreationDate="2022-07-06T19:28:52.927" Score="1" Body="&lt;p&gt;I ultimately found &lt;a href=&quot;https://github.com/lvclark/R_genetics_conv/blob/master/genind2structure.R&quot; rel=&quot;nofollow noreferrer&quot;&gt;this function&lt;/a&gt;, which I used for the purpose of converting a genind to a STRUCTURE file. The graph4lg &lt;a href=&quot;https://rdrr.io/cran/graph4lg/man/genind_to_structure.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;genind_2_structure&lt;/a&gt; command would regularly error on me (present working directory couldn't be passed as the &lt;code&gt;output&lt;/code&gt; argument).&lt;/p&gt;&#xA;" OwnerUserId="14041" LastActivityDate="2022-07-06T19:28:52.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19361" PostTypeId="1" CreationDate="2022-07-07T08:40:38.843" Score="1" ViewCount="360" Body="&lt;p&gt;I am currently using VEP for variant annotation.&lt;/p&gt;&#xA;&lt;p&gt;I am facing an error as below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[E::fai_build_core] Different line length in sequence 'Pn9'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I understand there is an issue with the difference in line length of Pn9 in the  fasta file. However, the sequences of fasta file all have different line length. I don't get why there is an error specifically on Pn9.&lt;/p&gt;&#xA;&lt;p&gt;I have tried using both snpEff and Annovar but doesn't work. Any thoughts on this would be really appreciated.&lt;/p&gt;&#xA;&lt;p&gt;I am attaching the length of sequence here :&#xA;&lt;img src=&quot;https://i.stack.imgur.com/6xMuH.jpg&quot; alt=&quot;length per chromosome&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="15730" LastActivityDate="2022-07-08T12:49:02.990" Title="Different line length in fasta file" Tags="|fasta|sequence-annotation|samtools|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19362" PostTypeId="2" ParentId="19359" CreationDate="2022-07-07T10:21:11.580" Score="1" Body="&lt;p&gt;I would recommend not to use &lt;code&gt;h&lt;/code&gt; to cut the tree but to use &lt;code&gt;k&lt;/code&gt;, as &lt;code&gt;k&lt;/code&gt; instructs the number of clusters to obtain, and then leaves it to the function to find the correct height for that. From there on you can easily extract the genes per cluster, for example with this code obtaining the first three clusters:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&#xA;heatof &amp;lt;- &#xA;pheatmap(logFC_mat, &#xA;         show_rownames=F, cluster_cols=F, cluster_rows=T, scale=&amp;quot;row&amp;quot;,&#xA;         cex=1, clustering_distance_rows=&amp;quot;euclidean&amp;quot;, cex=1,&#xA;         clustering_distance_cols=&amp;quot;euclidean&amp;quot;, &#xA;         clustering_method=&amp;quot;complete&amp;quot;, border_color=FALSE)&#xA;&#xA;#/ extract the top three clusters&#xA;k &amp;lt;- 3&#xA;sort(cutree(heatof$tree_row, k=3)) -&amp;gt; hi_clusters&#xA;&#xA;genes_per_cluster &amp;lt;- do.call(rbind, lapply(unique(hi_clusters), function(x){&#xA;  &#xA;  data.frame(cluster=x, gene=names(hi_clusters[hi_clusters %in% x]))&#xA;             &#xA;  &#xA;}))&#xA;&#xA;genes_per_cluster&#xA;#&amp;gt;     cluster                  gene&#xA;#&amp;gt; 1         1  ENSMUSG00000097417.1&#xA;#&amp;gt; 2         1  ENSMUSG00000084950.7&#xA;#&amp;gt; 3         1 ENSMUSG00000021824.12&#xA;#&amp;gt; 4         1 ENSMUSG00000018381.15&#xA;#&amp;gt; 5         1  ENSMUSG00000098716.1&#xA;#&amp;gt; 6         1 ENSMUSG00000039206.13&#xA;#&amp;gt; 7         1 ENSMUSG00000047880.14&#xA;#&amp;gt; 8         1  ENSMUSG00000094127.1&#xA;#&amp;gt; 9         1  ENSMUSG00000098609.1&#xA;#&amp;gt; 10        1  ENSMUSG00000106093.1&#xA;#&amp;gt; 11        1 ENSMUSG00000009585.17&#xA;#&amp;gt; 12        1  ENSMUSG00000027950.7&#xA;#&amp;gt; 13        1 ENSMUSG00000004071.12&#xA;#&amp;gt; 14        1  ENSMUSG00000104208.1&#xA;#&amp;gt; 15        1 ENSMUSG00000047369.15&#xA;#&amp;gt; 16        1  ENSMUSG00000097660.1&#xA;#&amp;gt; 17        1 ENSMUSG00000002109.14&#xA;#&amp;gt; 18        1  ENSMUSG00000108524.1&#xA;#&amp;gt; 19        1 ENSMUSG00000067786.16&#xA;#&amp;gt; 20        1  ENSMUSG00000085437.8&#xA;#&amp;gt; 21        1  ENSMUSG00000086220.1&#xA;#&amp;gt; 22        1  ENSMUSG00000102299.1&#xA;#&amp;gt; 23        1 ENSMUSG00000079293.11&#xA;#&amp;gt; 24        1  ENSMUSG00000085425.1&#xA;#&amp;gt; 25        1  ENSMUSG00000006517.5&#xA;#&amp;gt; 26        1 ENSMUSG00000049775.16&#xA;#&amp;gt; 27        1  ENSMUSG00000109881.1&#xA;#&amp;gt; 28        1  ENSMUSG00000102349.1&#xA;#&amp;gt; 29        1  ENSMUSG00000089636.1&#xA;#&amp;gt; 30        1  ENSMUSG00000086360.1&#xA;#&amp;gt; 31        1 ENSMUSG00000036138.16&#xA;#&amp;gt; 32        1  ENSMUSG00000045382.6&#xA;#&amp;gt; 33        1 ENSMUSG00000030720.16&#xA;#&amp;gt; 34        1 ENSMUSG00000027398.13&#xA;#&amp;gt; 35        1  ENSMUSG00000104823.1&#xA;#&amp;gt; 36        1  ENSMUSG00000100783.1&#xA;#&amp;gt; 37        1  ENSMUSG00000082848.1&#xA;#&amp;gt; 38        1  ENSMUSG00000110148.1&#xA;#&amp;gt; 39        1  ENSMUSG00000085754.7&#xA;#&amp;gt; 40        1  ENSMUSG00000108321.1&#xA;#&amp;gt; 41        1 ENSMUSG00000032363.15&#xA;#&amp;gt; 42        1 ENSMUSG00000003868.14&#xA;#&amp;gt; 43        1  ENSMUSG00000107075.1&#xA;#&amp;gt; 44        1  ENSMUSG00000074136.9&#xA;#&amp;gt; 45        1  ENSMUSG00000089875.1&#xA;#&amp;gt; 46        1 ENSMUSG00000055725.11&#xA;#&amp;gt; 47        1 ENSMUSG00000028451.12&#xA;#&amp;gt; 48        1  ENSMUSG00000078786.9&#xA;#&amp;gt; 49        1  ENSMUSG00000087442.1&#xA;#&amp;gt; 50        1  ENSMUSG00000053442.4&#xA;#&amp;gt; 51        1  ENSMUSG00000101674.6&#xA;#&amp;gt; 52        1  ENSMUSG00000097748.1&#xA;#&amp;gt; 53        1  ENSMUSG00000050010.8&#xA;#&amp;gt; 54        1 ENSMUSG00000029623.14&#xA;#&amp;gt; 55        1  ENSMUSG00000097060.1&#xA;#&amp;gt; 56        1 ENSMUSG00000050627.13&#xA;#&amp;gt; 57        1  ENSMUSG00000085471.1&#xA;#&amp;gt; 58        1 ENSMUSG00000039753.16&#xA;#&amp;gt; 59        1  ENSMUSG00000027706.8&#xA;#&amp;gt; 60        1  ENSMUSG00000032487.8&#xA;#&amp;gt; 61        1 ENSMUSG00000002985.16&#xA;#&amp;gt; 62        1 ENSMUSG00000038274.11&#xA;#&amp;gt; 63        1  ENSMUSG00000087206.1&#xA;#&amp;gt; 64        1  ENSMUSG00000079355.3&#xA;#&amp;gt; 65        1  ENSMUSG00000086693.1&#xA;#&amp;gt; 66        1 ENSMUSG00000030579.10&#xA;#&amp;gt; 67        1  ENSMUSG00000108235.1&#xA;#&amp;gt; 68        1  ENSMUSG00000097077.1&#xA;#&amp;gt; 69        1  ENSMUSG00000087367.1&#xA;#&amp;gt; 70        1 ENSMUSG00000037982.16&#xA;#&amp;gt; 71        1 ENSMUSG00000029422.14&#xA;#&amp;gt; 72        1 ENSMUSG00000026749.11&#xA;#&amp;gt; 73        1 ENSMUSG00000067288.12&#xA;#&amp;gt; 74        1  ENSMUSG00000090330.2&#xA;#&amp;gt; 75        1 ENSMUSG00000059791.14&#xA;#&amp;gt; 76        1  ENSMUSG00000103698.1&#xA;#&amp;gt; 77        1  ENSMUSG00000097472.1&#xA;#&amp;gt; 78        1  ENSMUSG00000026156.8&#xA;#&amp;gt; 79        1 ENSMUSG00000075297.10&#xA;#&amp;gt; 80        1 ENSMUSG00000055692.18&#xA;#&amp;gt; 81        1 ENSMUSG00000039307.16&#xA;#&amp;gt; 82        1 ENSMUSG00000045576.16&#xA;#&amp;gt; 83        1  ENSMUSG00000087486.1&#xA;#&amp;gt; 84        1  ENSMUSG00000014470.4&#xA;#&amp;gt; 85        1  ENSMUSG00000056078.5&#xA;#&amp;gt; 86        1 ENSMUSG00000049928.15&#xA;#&amp;gt; 87        1  ENSMUSG00000086714.1&#xA;#&amp;gt; 88        1  ENSMUSG00000109274.1&#xA;#&amp;gt; 89        1  ENSMUSG00000109854.1&#xA;#&amp;gt; 90        1  ENSMUSG00000060802.8&#xA;#&amp;gt; 91        2  ENSMUSG00000085505.1&#xA;#&amp;gt; 92        2  ENSMUSG00000092134.1&#xA;#&amp;gt; 93        2  ENSMUSG00000097272.1&#xA;#&amp;gt; 94        2  ENSMUSG00000087700.1&#xA;#&amp;gt; 95        2  ENSMUSG00000101162.2&#xA;#&amp;gt; 96        2  ENSMUSG00000107317.1&#xA;#&amp;gt; 97        2  ENSMUSG00000102451.1&#xA;#&amp;gt; 98        2  ENSMUSG00000109669.1&#xA;#&amp;gt; 99        2  ENSMUSG00000086836.1&#xA;#&amp;gt; 100       2  ENSMUSG00000092545.7&#xA;#&amp;gt; 101       2  ENSMUSG00000090625.1&#xA;#&amp;gt; 102       2  ENSMUSG00000047988.1&#xA;#&amp;gt; 103       2  ENSMUSG00000086430.1&#xA;#&amp;gt; 104       2  ENSMUSG00000105941.1&#xA;#&amp;gt; 105       2  ENSMUSG00000022223.8&#xA;#&amp;gt; 106       2 ENSMUSG00000031145.15&#xA;#&amp;gt; 107       2  ENSMUSG00000086680.2&#xA;#&amp;gt; 108       2 ENSMUSG00000021485.13&#xA;#&amp;gt; 109       2  ENSMUSG00000097768.1&#xA;#&amp;gt; 110       2 ENSMUSG00000075576.12&#xA;#&amp;gt; 111       2  ENSMUSG00000090952.2&#xA;#&amp;gt; 112       2 ENSMUSG00000040723.14&#xA;#&amp;gt; 113       2  ENSMUSG00000027329.9&#xA;#&amp;gt; 114       2  ENSMUSG00000106005.1&#xA;#&amp;gt; 115       2 ENSMUSG00000046027.17&#xA;#&amp;gt; 116       2 ENSMUSG00000004221.16&#xA;#&amp;gt; 117       2  ENSMUSG00000023988.8&#xA;#&amp;gt; 118       2  ENSMUSG00000087611.1&#xA;#&amp;gt; 119       2  ENSMUSG00000092103.2&#xA;#&amp;gt; 120       2  ENSMUSG00000090081.1&#xA;#&amp;gt; 121       2  ENSMUSG00000107999.1&#xA;#&amp;gt; 122       2 ENSMUSG00000041774.14&#xA;#&amp;gt; 123       2  ENSMUSG00000090246.1&#xA;#&amp;gt; 124       2  ENSMUSG00000032528.4&#xA;#&amp;gt; 125       2 ENSMUSG00000021250.13&#xA;#&amp;gt; 126       2 ENSMUSG00000009292.17&#xA;#&amp;gt; 127       2  ENSMUSG00000109491.1&#xA;#&amp;gt; 128       2  ENSMUSG00000090252.1&#xA;#&amp;gt; 129       2  ENSMUSG00000087443.1&#xA;#&amp;gt; 130       2  ENSMUSG00000064339.1&#xA;#&amp;gt; 131       2  ENSMUSG00000091160.1&#xA;#&amp;gt; 132       2  ENSMUSG00000108332.1&#xA;#&amp;gt; 133       2  ENSMUSG00000048721.2&#xA;#&amp;gt; 134       2  ENSMUSG00000102961.1&#xA;#&amp;gt; 135       2  ENSMUSG00000096957.1&#xA;#&amp;gt; 136       2  ENSMUSG00000093006.2&#xA;#&amp;gt; 137       2 ENSMUSG00000027774.16&#xA;#&amp;gt; 138       2 ENSMUSG00000012126.16&#xA;#&amp;gt; 139       2  ENSMUSG00000092190.1&#xA;#&amp;gt; 140       2  ENSMUSG00000074213.2&#xA;#&amp;gt; 141       2  ENSMUSG00000086564.9&#xA;#&amp;gt; 142       2  ENSMUSG00000092655.1&#xA;#&amp;gt; 143       2  ENSMUSG00000098482.1&#xA;#&amp;gt; 144       2  ENSMUSG00000108574.1&#xA;#&amp;gt; 145       2  ENSMUSG00000087669.1&#xA;#&amp;gt; 146       2  ENSMUSG00000078653.4&#xA;#&amp;gt; 147       2  ENSMUSG00000108353.1&#xA;#&amp;gt; 148       2  ENSMUSG00000092386.1&#xA;#&amp;gt; 149       2  ENSMUSG00000087607.1&#xA;#&amp;gt; 150       2 ENSMUSG00000061607.14&#xA;#&amp;gt; 151       2  ENSMUSG00000026729.9&#xA;#&amp;gt; 152       2  ENSMUSG00000064330.9&#xA;#&amp;gt; 153       2  ENSMUSG00000103864.1&#xA;#&amp;gt; 154       2  ENSMUSG00000085208.3&#xA;#&amp;gt; 155       2  ENSMUSG00000073628.5&#xA;#&amp;gt; 156       2  ENSMUSG00000102776.1&#xA;#&amp;gt; 157       2  ENSMUSG00000086998.1&#xA;#&amp;gt; 158       2 ENSMUSG00000006728.12&#xA;#&amp;gt; 159       2  ENSMUSG00000021273.9&#xA;#&amp;gt; 160       2  ENSMUSG00000062591.5&#xA;#&amp;gt; 161       2 ENSMUSG00000040936.14&#xA;#&amp;gt; 162       2  ENSMUSG00000046145.7&#xA;#&amp;gt; 163       2  ENSMUSG00000087366.7&#xA;#&amp;gt; 164       2  ENSMUSG00000039452.5&#xA;#&amp;gt; 165       2  ENSMUSG00000087660.1&#xA;#&amp;gt; 166       2  ENSMUSG00000084878.1&#xA;#&amp;gt; 167       2  ENSMUSG00000101438.1&#xA;#&amp;gt; 168       2  ENSMUSG00000064351.1&#xA;#&amp;gt; 169       2  ENSMUSG00000086312.1&#xA;#&amp;gt; 170       2  ENSMUSG00000097221.1&#xA;#&amp;gt; 171       2  ENSMUSG00000107090.1&#xA;#&amp;gt; 172       2  ENSMUSG00000095280.1&#xA;#&amp;gt; 173       2 ENSMUSG00000017837.12&#xA;#&amp;gt; 174       2  ENSMUSG00000109251.1&#xA;#&amp;gt; 175       2  ENSMUSG00000086600.8&#xA;#&amp;gt; 176       2  ENSMUSG00000090009.1&#xA;#&amp;gt; 177       2  ENSMUSG00000101695.1&#xA;#&amp;gt; 178       2 ENSMUSG00000084128.10&#xA;#&amp;gt; 179       2 ENSMUSG00000038502.16&#xA;#&amp;gt; 180       2  ENSMUSG00000015942.9&#xA;#&amp;gt; 181       2  ENSMUSG00000106478.1&#xA;#&amp;gt; 182       2  ENSMUSG00000054556.6&#xA;#&amp;gt; 183       2  ENSMUSG00000082180.1&#xA;#&amp;gt; 184       2 ENSMUSG00000013997.10&#xA;#&amp;gt; 185       2  ENSMUSG00000100548.1&#xA;#&amp;gt; 186       2  ENSMUSG00000053706.2&#xA;#&amp;gt; 187       2  ENSMUSG00000093598.1&#xA;#&amp;gt; 188       3  ENSMUSG00000097860.1&#xA;#&amp;gt; 189       3  ENSMUSG00000079179.9&#xA;#&amp;gt; 190       3  ENSMUSG00000097347.1&#xA;#&amp;gt; 191       3  ENSMUSG00000092371.1&#xA;#&amp;gt; 192       3  ENSMUSG00000109754.1&#xA;#&amp;gt; 193       3  ENSMUSG00000090016.1&#xA;#&amp;gt; 194       3  ENSMUSG00000097854.1&#xA;#&amp;gt; 195       3  ENSMUSG00000029360.3&#xA;#&amp;gt; 196       3  ENSMUSG00000102564.1&#xA;#&amp;gt; 197       3  ENSMUSG00000025353.4&#xA;#&amp;gt; 198       3  ENSMUSG00000052214.9&#xA;#&amp;gt; 199       3 ENSMUSG00000037936.15&#xA;#&amp;gt; 200       3  ENSMUSG00000105198.1&#xA;&#xA;Created on 2022-07-07 by the reprex package (v2.0.1)&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15731" LastActivityDate="2022-07-07T10:21:11.580" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19363" PostTypeId="1" CreationDate="2022-07-07T11:50:14.277" Score="2" ViewCount="187" Body="&lt;p&gt;I can't find an answer to this in previous questions, so hoping someone can help me now.&lt;/p&gt;&#xA;&lt;p&gt;We have nanopore sequenced a PCR product, and I've filtered our reads to +/- 10 bp of the expected product length (~4,400bp), leaving ~55,000 reads.&lt;/p&gt;&#xA;&lt;p&gt;We now want to work out how many of the reads correspond to each of 6 different potential arrangements. The section we're interested in is in the middle of the 4,400 bp PCR product, and each different potential arrangement is ~1,500 bp. We have reference sequences for each of the 1,500 bp arrangements. Basically, we want to know what portion of our starting cells had each of the different arrangements of genes in this operon (e.g arrangement 1=30%, arrangement 2=15%, etc.).&lt;/p&gt;&#xA;&lt;p&gt;This seems like it should be a really easy thing to do using read mapping to count alignments for each reference separately, but I just can't get my head around it. No matter what options I've tried with minimap2 and samtools, every read maps every time on every different arrangement. I think the problem is that the genes in our reference sequences &lt;em&gt;are&lt;/em&gt; all present in every read, just in different arrangements. So we need to &lt;strong&gt;only&lt;/strong&gt; count alignments which align to the whole 1,500 bp reference sequence, not partial hits or hits with huge gaps in them.&lt;/p&gt;&#xA;&lt;p&gt;I have tried the minimap2 options -r, -G and --secondary=no to restrict the output to single alignments with no large gaps, and combined that with samtools view -m 1000 to output only alignments longer than 1,000 bp (this is far stricter than it needs to be to only output alignments to any of the different potential arrangements). But still, every time, every read is mapping to every reference, usually multiple times.&lt;/p&gt;&#xA;&lt;p&gt;What am I doing wrong?! Am I totally over-complicating this, or doing something obviously stupid? Almost feel like using blastn might be better at this point...&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR: how do I use minimap2 and/or samtools (or any other tool better suited to this) to count how many of our ~4,400 bp reads contain &lt;em&gt;only full-length, almost exact matches&lt;/em&gt; to our 1,500 bp reference sequence, excluding any partial hits?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;My latest attempt (broken into individual steps, though I'd usually pipe the samtools steps together, also not bothered with samtools sort at this point):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 -ax map-ont -r10 -N5 --secondary=no --no-long-join reference.fasta reads.fastq &amp;gt; reads.minimap.sam&#xA;&#xA;samtools view -m 1000 -b -o reads.minimap.bam reads.minimap.sam&#xA;&#xA;samtools -c reads.minimap.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15732" LastActivityDate="2022-07-08T06:44:03.883" Title="Get number of reads with a single, (almost) exact match to the full length of a reference sequence" Tags="|sam|samtools|nanopore|read-mapping|minimap2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19364" PostTypeId="1" CreationDate="2022-07-07T13:48:51.627" Score="1" ViewCount="508" Body="&lt;p&gt;I have the following quiz question, but the Pattern1 for both &lt;code&gt;(ACC|ATA)&lt;/code&gt; and &lt;code&gt;(CGA|ACT)&lt;/code&gt; are unique (just grep for &lt;code&gt;ACC&lt;/code&gt; and &lt;code&gt;CGA&lt;/code&gt;), which means it might be difficult to find the starting pattern for the DNA genome string in the following case. Do I need something like &lt;a href=&quot;https://sci-hub.ee/https://ieeexplore.ieee.org/document/8247239&quot; rel=&quot;nofollow noreferrer&quot;&gt;GapReduce algorithm&lt;/a&gt; ?&lt;/p&gt;&#xA;&lt;p&gt;Quiz : There is a single (linear) string with the following (3,1)-mer composition. Find it.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(ACC|ATA) (ACT|ATT) (ATA|TGA) (ATT|TGA) (CAC|GAT) (CCG|TAC) (CGA|ACT)&#xA;&#xA;(CTG|AGC) (CTG|TTC) (GAA|CTT) (GAT|CTG) (GAT|CTG) (TAC|GAT) (TCT|AAG)&#xA;&#xA;(TGA|GCT) (TGA|TCT) (TTC|GAA)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QT9yQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QT9yQ.png&quot; alt=&quot;k-d mer&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15733" LastEditorUserId="73" LastEditDate="2022-07-08T06:46:24.270" LastActivityDate="2023-08-03T10:02:25.310" Title="DNA genome string reconstruction from k-mer" Tags="|assembly|algorithms|k-mer|homework|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19365" PostTypeId="1" CreationDate="2022-07-07T14:11:47.080" Score="1" ViewCount="79" Body="&lt;p&gt;I want to have a jitter violin coloured by sample ids but I get this weird plot&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; str(long)&#xA;Classes ‘data.table’ and 'data.frame':  156 obs. of  3 variables:&#xA; &lt;span class=&quot;math-container&quot;&gt;$ gene    : chr  &quot;gene2&quot; &quot;gene3&quot; &quot;gene4&quot; &quot;gene1&quot; ...&#xA; $&lt;/span&gt; variable: chr  &amp;quot;1016&amp;quot; &amp;quot;1016&amp;quot; &amp;quot;1016&amp;quot; &amp;quot;1016&amp;quot; ...&#xA; $ value   : num  5.82 8.41 11.22 7.21 5.61 ...&#xA; - attr(*, &amp;quot;.internal.selfref&amp;quot;)=&amp;lt;externalptr&amp;gt; &#xA;&amp;gt;&#xA;&#xA;&amp;gt; p &amp;lt;- ggplot(long, aes(x=gene, y=value,colour=variable))&#xA;&amp;gt; p + geom_violin(width = 1.3) + geom_quasirandom(alpha = 0.2, width = 0.2)+ theme_classic()&#xA;There were 21 warnings (use warnings() to see them)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cMycr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cMycr.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I can not figure out where I am doing something wrong that I get this instead of getting a proper violin plot by dots coloured with sample IDs&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="650" LastEditDate="2022-07-07T19:00:10.253" LastActivityDate="2022-07-08T12:55:28.453" Title="Jitter violin coloured by sample id" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19366" PostTypeId="2" ParentId="10977" CreationDate="2022-07-07T14:35:50.960" Score="1" Body="&lt;p&gt;Please, try this code to get all modules in datasets:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import statsmodels.api as sm&#xA;import inspect&#xA;&#xA;def get_list_of_modules_in_datasets():&#xA;  # get list of strings for the objects in datasets&#xA;  list_str_in_datasets = sm.datasets.__all__&#xA;  print(list_str_in_datasets)&#xA;  # get list of objects in datasets&#xA;  list_objs_in_datasets = [eval('sm.datasets.' + dsstr) for dsstr in list_str_in_datasets]&#xA;  print(list_objs_in_datasets)&#xA;  # get list of modules in datasets&#xA;  list_modules_in_datasets = []&#xA;  for dso in list_objs_in_datasets:&#xA;    if (inspect.ismodule(dso)):&#xA;      list_modules_in_datasets.append(dso)&#xA;  print(list_modules_in_datasets)&#xA;  print(&amp;quot;Number of Modules in datasets = %d&amp;quot; %(len(list_modules_in_datasets)))&#xA;  return (list_modules_in_datasets)&#xA;&#xA;if __name__ == '__main__':&#xA;&#xA;  list_modules_in_datasets = get_list_of_modules_in_datasets()&#xA;&#xA;  # print data for all dataframes from the datasets&#xA;  for ds in list_modules_in_datasets:&#xA;    df = ds.load_pandas().data&#xA;    print(list(df.columns))&#xA;    print(df.shape)&#xA;    print(df)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15734" LastEditorUserId="15734" LastEditDate="2022-07-07T14:42:48.887" LastActivityDate="2022-07-07T14:42:48.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19367" PostTypeId="2" ParentId="103" CreationDate="2022-07-07T14:57:14.770" Score="0" Body="&lt;p&gt;&lt;strong&gt;Origin of name&lt;/strong&gt;&#xA;Bootstrapping is a term of derision in many ways, prior widespread use in phylogenetics. The response here is also the story that I recall the name meaning 'pulling yourself up by your bootstraps'. If I remember the story of it origin correctly, it means its impossible to 'pull-yourself-up' with you bootstraps because there's nothing external to leverage.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Historic usage&lt;/strong&gt; Thus traditionally, it was a last resort approach in statistics, or else a crude data mining approach, because a lot statistics at that time worked by comparing a result against a theoretical reference, i.e. critical probability. So for example, T-test, Chi-squared, ANOVA etc ... a theoretical null distribution is used or else created against which the observation is compared. The idea of not using any theoretical external distribution was not deemed cool in the days of critical probability. Bootstrapping does weird things to the distribution that few really understand (spent time investigating).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Method&lt;/strong&gt; Technically bootstrapping is simply resampling with replacement. So if you've a data set of 100 nucleotides replicates of a resampled data set are constructed each of 100 nucleotides by randomly selecting 100 nucleotides (with replacement) from the original data set.  Performing tree estimate calculations over multiple replicates permits a robustness estimate of each 'node' in the tree (hypothetical common ancestor).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;History in phylogenetics&lt;/strong&gt;&#xA;With onset of phylogenetics however bootstrapping became a statistical norm for several different reasons, which are quite complex.  For example, attempting to perform critical probability across ever branch of a tree could run into statistical problems, what is a 'null distribution' in evolutionary biology (that question has since been reasonably well answered), in the days of the rows about the parsimony method - bootstrapping was well under radar. Hence bootstrapping is now mainstream, with some caveats.&lt;/p&gt;&#xA;&lt;p&gt;I assume bootstrapping was originally supported by simulation studies, however it so established I don't even know it theoretical history.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Jack-knifing&lt;/strong&gt;&#xA;'Jack-knifing' is named because it was deemed a rough and ready tool (a 'pen-knife') to assess a data set and a forerunner of the bootstrap. Its so old now I don't even remember the resampling strategy it uses, chopping the data set in half rings a bell - I honestly forget.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Today&lt;/strong&gt;&#xA;These days there is no real reason to know the method, other than using it and the thresholds commonly accepted. Very little statistical development is based around bootstrapping. The underlying tenant is that each site is independent of the others and that is a widespread assumption across phylogenetics. That assumptions has been assessed (obviously) but the solutions were not implemented. Continually reassessing this assumption remains very interesting and could be productive longterm, but it doesn't negate the basic approach to treat each alignment position as an independent event to every other alignment position because its a baseline used throughout molecular phylogenetics.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-07-07T15:46:00.673" LastActivityDate="2022-07-07T15:46:00.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19368" PostTypeId="1" CreationDate="2022-07-07T20:38:59.287" Score="1" ViewCount="66" Body="&lt;p&gt;I've been working with the fastStructure program and am on the step of analyzing model complexity using the provided chooseK.py script. I have been running into the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &amp;quot;chooseK.py&amp;quot;, line 109, in &amp;lt;module&amp;gt;&#xA;    bestKs = parse_varQs(files)&#xA;  File &amp;quot;chooseK.py&amp;quot;, line 54, in parse_varQs&#xA;    Q = Q/utils.insum(Q,[1])&#xA;AttributeError: module 'vars' has no attribute 'insum'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have run into 'AttributeError' messages in the past and have attempted the solution of renaming files in my working directory that may be shadowing the real module (similar to: &lt;a href=&quot;https://stackoverflow.com/questions/36530726/using-numpy-module-object-has-no-attribute-array&quot;&gt;https://stackoverflow.com/questions/36530726/using-numpy-module-object-has-no-attribute-array&lt;/a&gt;), but this hasn't fixed anything. Reinstalling Numpy/Scipy/fastStructure has also been a dead end.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestions are much appreciated! Happy to provide more info as needed.&lt;/p&gt;&#xA;&lt;p&gt;-A&lt;/p&gt;&#xA;" OwnerUserId="15735" LastActivityDate="2024-03-29T06:07:15.963" Title="AttributeError [in fastStructure's chooseK.py]: module 'vars' has no attribute 'insum'" Tags="|python|scipy|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19369" PostTypeId="1" AcceptedAnswerId="19370" CreationDate="2022-07-08T01:46:40.020" Score="3" ViewCount="1590" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://stackoverflow.com/questions/72893148/r-applying-different-colors-to-a-phylogenetic-tree-branch&quot;&gt;StackOverflow&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have a phylogenetic tree made with a Newick format&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;((a:1,b:1):2, (c:1, d:1):3):1;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output will be&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AfB3P.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AfB3P.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have drawn this with &lt;a href=&quot;http://etetoolkit.org/treeview/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://etetoolkit.org/treeview/&lt;/a&gt;, but output with R will be same.&lt;/p&gt;&#xA;&lt;p&gt;What I want to do is apply colors to a certain branch.&lt;/p&gt;&#xA;&lt;p&gt;That would be&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/x4NXL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/x4NXL.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For example,&#xA;&lt;a href=&quot;https://i.stack.imgur.com/KII40.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KII40.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since the Newick input doesn't include any information other than the branch (c,d) has length 3,&#xA;is there any way I can include multiple color information on the branch?&lt;/p&gt;&#xA;&lt;p&gt;I usually use ggtree for making trees, but other suggestions can be helpful.&lt;/p&gt;&#xA;&lt;p&gt;The main script for this kind of tree building is like this, but I can't figure out how this works.&lt;/p&gt;&#xA;&lt;p&gt;(Full script Github link: &lt;a href=&quot;https://github.com/seongyeol-park/Human_Lineage_Tracing/blob/main/Rscripts/HSC_commitment.R&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/seongyeol-park/Human_Lineage_Tracing/blob/main/Rscripts/HSC_commitment.R&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ggtree(tree) %&amp;lt;+% m_lng_dt + theme_tree2()+&#xA;  coord_cartesian(xlim = c(-1,35))+&#xA;  geom_text2(aes(subset = is.na(Source_class2) == F, x=x-n_pointmt+3, y = y+0.25, label = Source_class2), size=2.5, color=&amp;quot;red&amp;quot;)+&#xA;  geom_segment2(aes(subset = is.na(VAF1) == F, x=x-n_pointmt, xend = x -n_pointmt +1, y = y, yend = y, color = VAF1))+&#xA;  geom_text2(aes(subset = is.na(VAF1) == F, x=x-n_pointmt, y = y, label = VAF1), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF2) == F, x=x-n_pointmt+1, xend = x -n_pointmt +2, y = y, yend = y, color = VAF2))+&#xA;  geom_text2(aes(subset = is.na(VAF2) == F, x=x-n_pointmt+1, y = y, label = VAF2), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF3) == F, x=x-n_pointmt+2, xend = x -n_pointmt +3, y = y, yend = y, color = VAF3))+&#xA;  geom_text2(aes(subset = is.na(VAF3) == F, x=x-n_pointmt+2, y = y, label = VAF3), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF4) == F, x=x-n_pointmt+3, xend = x -n_pointmt +4, y = y, yend = y, color = VAF4))+&#xA;  geom_text2(aes(subset = is.na(VAF4) == F, x=x-n_pointmt+3, y = y, label = VAF4), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF5) == F, x=x-n_pointmt+4, xend = x -n_pointmt +5, y = y, yend = y, color = VAF5))+&#xA;  geom_text2(aes(subset = is.na(VAF5) == F, x=x-n_pointmt+4, y = y, label = VAF5), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF6) == F, x=x-n_pointmt+5, xend = x -n_pointmt +6, y = y, yend = y, color = VAF6))+&#xA;  geom_text2(aes(subset = is.na(VAF6) == F, x=x-n_pointmt+5, y = y, label = VAF6), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF7) == F, x=x-n_pointmt+6, xend = x -n_pointmt +7, y = y, yend = y, color = VAF7))+&#xA;  geom_text2(aes(subset = is.na(VAF7) == F, x=x-n_pointmt+6, y = y, label = VAF7), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF8) == F, x=x-n_pointmt+7, xend = x -n_pointmt +8, y = y, yend = y, color = VAF8))+&#xA;  geom_text2(aes(subset = is.na(VAF8) == F, x=x-n_pointmt+7, y = y, label = VAF8), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF9) == F, x=x-n_pointmt+8, xend = x -n_pointmt +9, y = y, yend = y, color = VAF9))+&#xA;  geom_text2(aes(subset = is.na(VAF9) == F, x=x-n_pointmt+8, y = y, label = VAF9), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF10) == F, x=x-n_pointmt+9, xend = x -n_pointmt +10, y = y, yend = y, color = VAF10))+&#xA;  geom_text2(aes(subset = is.na(VAF10) == F, x=x-n_pointmt+9, y = y, label = VAF10), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF11) == F, x=x-n_pointmt+10, xend = x -n_pointmt +11, y = y, yend = y, color = VAF11))+&#xA;  geom_text2(aes(subset = is.na(VAF11) == F, x=x-n_pointmt+10, y = y, label = VAF11), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF12) == F, x=x-n_pointmt+11, xend = x -n_pointmt +12, y = y, yend = y, color = VAF12))+&#xA;  geom_text2(aes(subset = is.na(VAF12) == F, x=x-n_pointmt+11, y = y, label = VAF12), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF13) == F, x=x-n_pointmt+12, xend = x -n_pointmt +13, y = y, yend = y, color = VAF13))+&#xA;  geom_text2(aes(subset = is.na(VAF13) == F, x=x-n_pointmt+12, y = y, label = VAF13), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF14) == F, x=x-n_pointmt+13, xend = x -n_pointmt +14, y = y, yend = y, color = VAF14))+&#xA;  geom_text2(aes(subset = is.na(VAF14) == F, x=x-n_pointmt+13, y = y, label = VAF14), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF15) == F, x=x-n_pointmt+14, xend = x -n_pointmt +15, y = y, yend = y, color = VAF15))+&#xA;  geom_text2(aes(subset = is.na(VAF15) == F, x=x-n_pointmt+14, y = y, label = VAF15), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF16) == F, x=x-n_pointmt+15, xend = x -n_pointmt +16, y = y, yend = y, color = VAF16))+&#xA;  geom_text2(aes(subset = is.na(VAF16) == F, x=x-n_pointmt+15, y = y, label = VAF16), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF17) == F, x=x-n_pointmt+16, xend = x -n_pointmt +17, y = y, yend = y, color = VAF17))+&#xA;  geom_text2(aes(subset = is.na(VAF17) == F, x=x-n_pointmt+16, y = y, label = VAF17), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF18) == F, x=x-n_pointmt+17, xend = x -n_pointmt +18, y = y, yend = y, color = VAF18))+&#xA;  geom_text2(aes(subset = is.na(VAF18) == F, x=x-n_pointmt+17, y = y, label = VAF18), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF19) == F, x=x-n_pointmt+18, xend = x -n_pointmt +19, y = y, yend = y, color = VAF19))+&#xA;  geom_text2(aes(subset = is.na(VAF19) == F, x=x-n_pointmt+18, y = y, label = VAF19), size=2.5)+&#xA;  geom_segment2(aes(subset = is.na(VAF20) == F, x=x-n_pointmt+19, xend = x -n_pointmt +20, y = y, yend = y, color = VAF20))+&#xA;  geom_text2(aes(subset = is.na(VAF20) == F, x=x-n_pointmt+19, y = y, label = VAF20), size=2.5)+&#xA;  scale_color_gradientn(colors = my_palette, breaks = col_breaks)+&#xA;  theme(axis.text = element_text(size=15, angle=45, hjust=1), plot.title = element_text(size=20), plot.margin= unit(c(0.2,0,0.2,0.2),&amp;quot;cm&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15736" LastEditorUserId="3848" LastEditDate="2022-07-10T12:15:37.703" LastActivityDate="2022-07-12T02:29:58.513" Title="R ggtree apply colors to branches with multiple colors" Tags="|r|phylogenetics|phylogeny|ggtree|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19370" PostTypeId="2" ParentId="19369" CreationDate="2022-07-08T03:40:35.510" Score="1" Body="&lt;p&gt;The simple solution is &lt;a href=&quot;http://tree.bio.ed.ac.uk/software/figtree/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Figtree&lt;/a&gt; a GUI that easily colours trees on a 'point and click' basis. In linux this can be driven via CLI, which is useful.&lt;/p&gt;&#xA;&lt;p&gt;The easy coding solution is to use extended Newick format (NHX) to colour the tree.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(((a:1,b:1[&amp;amp;NHX:C=2.2.2]):2[&amp;amp;NHX:C=2.2.2], (c:1[&amp;amp;NHX:C=100.100.100], d:1[&amp;amp;NHX:C=100.100.100]):3[&amp;amp;NHX:C=10.10.10]):1);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GGrX2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GGrX2.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The next thing is a viewer that accepts NHX format and here I used 'Icytree' &lt;a href=&quot;https://icytree.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The alternative format a tree viewer might take is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(((a:1,b:1[&amp;amp;&amp;amp;NHX:C=2.2.2]):2[&amp;amp;&amp;amp;NHX:C=2.2.2], (c:1[&amp;amp;&amp;amp;NHX:C=100.100.100], d:1[&amp;amp;&amp;amp;NHX:C=100.100.100]):3[&amp;amp;&amp;amp;NHX:C=10.10.10]):1);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I noted you'd used ETE3 toolkit website for your tree. Just to mention ETE3 has two levels,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the underlying raw Python library&lt;/li&gt;&#xA;&lt;li&gt;the website GUI&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The underlying raw Python is cool, except it was written to v3.6 because you combine it with your own code, the website is alright, but I think there's better.&lt;/p&gt;&#xA;&lt;p&gt;Using NHX allows the branch width to be set as well:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(((a:1,b:1[&amp;amp;NHX:C=2.2.2:W=3]):2[&amp;amp;NHX:C=2.2.2:W=3], (c:1[&amp;amp;NHX:C=100.100.100:W=3], d:1[&amp;amp;NHX:C=100.100.100:W=3]):3[&amp;amp;NHX:C=10.10.10:W=3]):1);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Sadly you can't set the font (which you obviously can in &lt;code&gt;ggtree&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;NHX normally used for auxiliary information, such as species name, or a particular phenotype.&lt;/p&gt;&#xA;&lt;p&gt;In summary, I do prefer NHX if you are coding over &lt;code&gt;R&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; For tree colouring you have to set the use NHX format BTW in the options, a tree browser will not do it automatically.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The basic &lt;code&gt;ggtree&lt;/code&gt; first step is to make the tree ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(&amp;quot;ggtree&amp;quot;)&#xA;tree &amp;lt;- read.tree(text = &amp;quot;(((a:1,b:1):2, (c:1, d:1):3):1);&amp;quot;) # my bracketing might be wrong&#xA;p&amp;lt;-ggtree(tree)&#xA;p+geom_tiplab()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then use &lt;code&gt;ggplot2&lt;/code&gt; style functions to overlay the presentation, I'm assuming you do understand &lt;code&gt;ggplot2&lt;/code&gt;. What the code you showed was doing was individually changing every font and the position of every branch in the tree. This is definitely not cool code and I wouldn't consider it further, i.e. its not even worth bothering with, that ain't how graphs nor trees are supposed to be represented.&lt;/p&gt;&#xA;&lt;p&gt;In any case, overall &lt;code&gt;ggplot2&lt;/code&gt; style code is always fiddly and to be honest I gave up using it, but you increment/or layer the changes you want as the code you supplied describes. Honestly I don't want to get into the fiddly incremental layered changes that &lt;code&gt;ggplot2&lt;/code&gt; graphs require.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;ETE3&lt;/code&gt; is good and does do coloured trees, which are described here, &lt;a href=&quot;http://etetoolkit.org/docs/latest/tutorial/tutorial_drawing.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://etetoolkit.org/docs/latest/tutorial/tutorial_drawing.html&lt;/a&gt;. The only hassle it was written for Python 3.6 - that long ago, but you can access it via &lt;code&gt;conda&lt;/code&gt; and set the Python version.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;ETE4&lt;/code&gt; will be very cool but its badly delayed in its release date, they did get funding for this so the delay is not cool, maybe never who knows. Getting it up to Python 3.10 would be useful at the very least.&lt;/p&gt;&#xA;&lt;p&gt;My personal take is ETE3 is useful for automating the tree output so you can manually assess the iterations, its quite critical for e.g. testing and debugging. Whether you are turning that into publication quality - which is universally how graphs are handled - I personally find excessive. The difference is that complex graphs represent loads of information which is not always immediately obvious to a casual observer, therefore gamma (transparency), plot type loads of stuff are relevant. A tree is a tree and 'dressing up a tree' makes not a jot of difference for an experienced observer. If someone wasn't quite clear about a tree, they'd just ask you for the tree file and file it through 'Figtree'.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-07-09T14:05:12.643" LastActivityDate="2022-07-09T14:05:12.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19371" PostTypeId="2" ParentId="19368" CreationDate="2022-07-08T04:05:50.523" Score="0" Body="&lt;p&gt;Could you try loading &lt;code&gt;fastStructure&lt;/code&gt; via conda?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda create --name faststruc python=3.9 # there can be issues with 3.10&#xA;conda activate faststruc&#xA;conda install -c bioconda faststructure &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It will then ask you whether you want to install &lt;code&gt;numpy&lt;/code&gt; and &lt;code&gt;scipy&lt;/code&gt; automatically. Furthermore all dependencies are installed and connectivity will have already been tested. You can easily delete everything afterwards by deleting the environment &lt;code&gt;faststruc&lt;/code&gt; so keeps everything clean.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The code in question is here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://programtalk.com/python-examples/vars.utils.insum/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://programtalk.com/python-examples/vars.utils.insum/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Its a &lt;code&gt;numpy&lt;/code&gt; calculation, what they are doing is navigating Python by leveraging the power of &lt;code&gt;numpy&lt;/code&gt;, which is very fast.&lt;/p&gt;&#xA;&lt;p&gt;Code ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bestKs = []&#xA;for file in files:&#xA;    handle = open(file,'r')&#xA;    Q = np.array([map(float,line.strip().split()) for line in handle])&#xA;    Q = Q/utils.insum(Q,[1])&#xA;    handle.close()&#xA;    N = Q.shape[0]&#xA;    C = np.cuemsum(np.sort(Q.sum(0))[::-1])&#xA;    bestKs.append(np.sum(C&amp;lt;N-1)+1)&#xA;&#xA;return bestKs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The code looks good, they know their stuff.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-07-08T04:13:45.433" LastActivityDate="2022-07-08T04:13:45.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19372" PostTypeId="2" ParentId="19361" CreationDate="2022-07-08T06:34:44.367" Score="1" Body="&lt;p&gt;This could be a bug in the fasta file reader component of fai_build_core. The fasta file format is very relaxed, and allows some odd constructions, for example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;cb3c6a5f-17dd-4cd6-a9ca-02124d2a32de&#xA;AAGAAATTGACCTCCGTAACCCCAAAGGAGACAAAAAGAATGCTGGCTTTACTCCC&#xA;CAGGAGCGCCAGGTTTTGGGGAACTGCCTACAAGCTGTACCATTGGCTGACAGCTT&#xA;TTAGGGCATCTACCCCCAACACTGCTTACACTTACACTTTCTGGACTTACATG&#xA;ATGAATGCCCGCTCTAAGAATGTTGGTTGGCGCCTTGATTACTTTTTGCTTTCC&#xA;CACTCTCTTTTTACCTGCATTGTGACAGCAAGATCCGGTCAAGGCTCTTG&#xA;GCAGTACACTGTCCCATCACCGCTACCTGTGATACCCCTCCCTGCAGTAGCTT&#xA;CCTGCCTGGGAGATGGCTCTCTCTGCAGAAGTCTGGTGTTTTAGCC&#xA;TTCAGGTGTTTGGTTTGTATGTGCTCCCTCATTTAAACATTAAACCAAACTT&#xA;CTGGTTTCCTTTAGACAATCCAAGAGAAATAAAAGGCCCTAC&#xA;TTTCAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA&#xA;AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A few programs will complain about fasta files that look like this, because it usually means that data has been corrupted. These programs only accept fasta files that have sequences with identical line lengths for all but the last line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;cb3c6a5f-17dd-4cd6-a9ca-02124d2a32de&#xA;AAGAAATTGACCTCCGTAACCCCAAAGGAGACAAAAAGAATGCTGGCTTTACTCCC&#xA;CAGGAGCGCCAGGTTTTGGGGAACTGCCTACAAGCTGTACCATTGGCTGACAGCTT&#xA;TTAGGGCATCTACCCCCAACACTGCTTACACTTACACTTTCTGGACTTACATGATG&#xA;AATGCCCGCTCTAAGAATGTTGGTTGGCGCCTTGATTACTTTTTGCTTTCCCACTC&#xA;TCTTTTTACCTGCATTGTGACAGCAAGATCCGGTCAAGGCTCTTGGCAGTACACTG&#xA;TCCCATCACCGCTACCTGTGATACCCCTCCCTGCAGTAGCTTCCTGCCTGGGAGAT&#xA;GGCTCTCTCTGCAGAAGTCTGGTGTTTTAGCCTTCAGGTGTTTGGTTTGTATGTGC&#xA;TCCCTCATTTAAACATTAAACCAAACTTCTGGTTTCCTTTAGACAATCCAAGAGAA&#xA;ATAAAAGGCCCTACTTTCAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA&#xA;AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;One way to work around this input error problem is to get rid of &lt;em&gt;all&lt;/em&gt; line breaks in the sequence of a fasta file, but then you run the risk of programs not properly being able to read all the characters in a line. A better approach is to feed the fasta file through a less restrictive data cleaning program and convert the lines back to all have a consistent length.&lt;/p&gt;&#xA;&lt;p&gt;There's a good chance this is &lt;em&gt;not&lt;/em&gt; the problem you are experiencing, but without more information about your input data, all we have to go on are the error messages and the sequence lengths [not line lengths] you have provided.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-07-08T12:49:02.990" LastActivityDate="2022-07-08T12:49:02.990" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19373" PostTypeId="2" ParentId="19363" CreationDate="2022-07-08T06:44:03.883" Score="1" Body="&lt;p&gt;Because minimap2 is a minimiser-based mapper, it's less useful for determining mapping accuracy to single-base precision.&lt;/p&gt;&#xA;&lt;p&gt;Where precision matters, you can use LAST, bearing in mind that it calculates &lt;em&gt;local&lt;/em&gt; alignments, and can report multiple hits per query/reference pair. The usual approach I use for LAST mapping is as follows:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;lastdb -uNEAR -R01 reference.fa reference.fa&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;last-train -Q 1 reference.fa &amp;lt;(cat reads.fastq | head -n 40000) &amp;gt; train.mat&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;lastal -p train.mat reference.fa reads.fastq &amp;gt; reads_vs_ref.maf&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;That final MAF file can be inspected for mapping. The default output produced from LAST is not very human-readable, so I've written &lt;a href=&quot;https://gitlab.com/gringer/bioinfscripts/-/blob/master/maf2csv.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;a perl script&lt;/a&gt; that converts MAF results into a CSV format with reference-relative statistics that I have found useful:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;maf2csv.pl reads_vs_ref.maf &amp;gt; reads_vs_ref.csv&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;This can be inspected in a spreadsheet viewer (e.g. LibreOffice Calc, or Excel) to look at the mapped locations, mapped fraction, identities, etc..&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, the MAF results can be converted to SAM/BAM format using &lt;code&gt;maf-convert&lt;/code&gt; (bearing in mind the previously-mentioned caveats about multiple hits):&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;maf-convert sam reads_vs_ref.maf | samtools view -b -T reference.fa | samtools sort &amp;gt; reads_vs_ref.bam&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-07-08T06:44:03.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19374" PostTypeId="2" ParentId="19365" CreationDate="2022-07-08T06:51:03.497" Score="1" Body="&lt;p&gt;What happens if you group by gene?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; long |&amp;gt;&#xA;  ggplot() +&#xA;  aes(x=gene, y=value, colour=variable, group=gene) +&#xA;  geom_violin(width = 1.3) +&#xA;  geom_quasirandom(alpha = 0.2, width = 0.2) +&#xA;  theme_classic()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;@llrs raised an insightful point in their comment. There's a good chance that the &amp;quot;weird plot&amp;quot; that you see is a side effect of trying to display individual &amp;quot;grouped-by-variable&amp;quot; boxplots that have too little data. Without knowing what you would like the boxplot to look like, it's hard to create something to improve the presentation.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-07-08T12:55:28.453" LastActivityDate="2022-07-08T12:55:28.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19377" PostTypeId="1" AcceptedAnswerId="19378" CreationDate="2022-07-11T13:38:04.040" Score="1" ViewCount="307" Body="&lt;p&gt;I want to have a plot violin plot around quasirandom dots - like below&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9EXRz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9EXRz.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is my data&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; str(long1)&#xA;Classes ‘data.table’ and 'data.frame':  116 obs. of  4 variables:&#xA; &lt;span class=&quot;math-container&quot;&gt;$ gene    : chr  &quot;WT1&quot; &quot;KRT7&quot; &quot;MUC16&quot; &quot;EPCAM&quot; ...&#xA; $&lt;/span&gt; variable: chr  &amp;quot;1016&amp;quot; &amp;quot;1016&amp;quot; &amp;quot;1016&amp;quot; &amp;quot;1016&amp;quot; ...&#xA; &lt;span class=&quot;math-container&quot;&gt;$ value   : num  5.82 8.41 11.22 7.21 5.4 ...&#xA; $&lt;/span&gt; group   : chr  &amp;quot;Tumour&amp;quot; &amp;quot;Tumour&amp;quot; &amp;quot;Tumour&amp;quot; &amp;quot;Tumour&amp;quot; ...&#xA; - attr(*, &amp;quot;.internal.selfref&amp;quot;)=&amp;lt;externalptr&amp;gt; &#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;By this code I get plot which does not have any violin&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(ggplot2)&#xA;library(ggpubr)&#xA;library(ggbeeswarm)&#xA;p &amp;lt;- ggplot(long1, aes(x=group, y=value,fill=group))&#xA;p +geom_quasirandom(method = 'quasirandom',alpha = 0.2, width = 0.2,aes(colour = variable))+ labs(title=&amp;quot;Supplementary Figure 6B&amp;quot;, y = &amp;quot;Expression(logcounts)&amp;quot;)+theme_classic()+&#xA;  theme(legend.position=&amp;quot;right&amp;quot;) +&#xA;  theme(legend.title=element_blank())+facet_wrap(~gene, scales = &amp;quot;free&amp;quot;)+theme(strip.background = element_blank(),strip.text.x = element_text(size = 12,face=&amp;quot;italic&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jzkyP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jzkyP.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I also tried adding geom_violin like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;p &amp;lt;- ggplot(long1, aes(x=group, y=value,fill=group))&#xA;p +geom_violin()+geom_quasirandom(method = 'quasirandom',alpha = 0.2, width = 0.2,aes(colour = variable))+ labs(title=&amp;quot;Supplementary Figure 6B&amp;quot;, y = &amp;quot;Expression(logcounts)&amp;quot;)+theme_classic()+&#xA;  theme(legend.position=&amp;quot;right&amp;quot;) +&#xA;  theme(legend.title=element_blank())+facet_wrap(~gene, scales = &amp;quot;free&amp;quot;)+theme(strip.background = element_blank(),strip.text.x = element_text(size = 12,face=&amp;quot;italic&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But gives a weird plot not similar to what I want&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kVc4e.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kVc4e.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Any help please&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;EDITED&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thanks to @haci&lt;/p&gt;&#xA;&lt;p&gt;By this code I get this plot but as you see the point color inside the violin is not really similar to what I want&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion to get a similar color scheme for dots inside the violins&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;p &amp;lt;- ggplot(long1, aes(x=group, y=value,fill=group))+guides(fill = &amp;quot;none&amp;quot;)&#xA;p +geom_violin(alpha = 0.1)+scale_fill_manual(values=c(&amp;quot;#fcf4e0&amp;quot;, &amp;quot;#fcf4e0&amp;quot;))+geom_quasirandom(method = 'quasirandom',alpha = 0.2, width = 0.2,size=2,stroke=0.5,aes(colour = variable))+scale_color_hue(l = 5, c = 150)+ labs(title=&amp;quot;Supplementary Figure 6B&amp;quot;, y = &amp;quot;Expression(logcounts)&amp;quot;)+theme_classic()+&#xA;  theme(legend.position=&amp;quot;right&amp;quot;,legend.text=element_text(size=12)) +&#xA;  theme(legend.title=element_blank(),legend.text.align = 0,text = element_text(family = &amp;quot;Arial&amp;quot;),&#xA;        axis.title.x = element_text(size = 0.00001),&#xA;        axis.title.y = element_text(size = 17))+theme(axis.text = element_text(&#xA;          color=&amp;quot;black&amp;quot;, &#xA;          size=12, &#xA;          face=1)&#xA;        )+facet_wrap(~gene, scales = &amp;quot;free&amp;quot;)+theme(strip.background = element_blank(),strip.text.x = element_text(size = 17,face=&amp;quot;italic&amp;quot;))+ theme(axis.line = element_line(colour = 'black', size = 0.8))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/siY8V.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/siY8V.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="57" LastEditDate="2022-07-19T23:49:09.697" LastActivityDate="2022-07-19T23:49:09.697" Title="How to make a violin plot around quasirandom dots" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19378" PostTypeId="2" ParentId="19377" CreationDate="2022-07-11T17:03:26.477" Score="3" Body="&lt;p&gt;You are almost there, if you set the &lt;code&gt;alpha&lt;/code&gt; parameter within your &lt;code&gt;geom_violin()&lt;/code&gt; call (to sth like 0.1) you should be able to get a plot similar to the example plot.&lt;/p&gt;&#xA;&lt;p&gt;The fact that you only have two data points for the &lt;code&gt;Stroma&lt;/code&gt; group and these two points having similar values, you will end up getting a horizontal-line like violin for this group. Moreover, you do not have that many data points in general anyway so you might consider not using violins at all, especially if the outliers in &lt;code&gt;Tumour&lt;/code&gt; group is coming from the same (maybe somewhat problematic?) sample, that might need to be dropped from the analysis. The reader should be able to grasp the essence of the data with our without violins (violins could help making the outliers obvious though).&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2022-07-11T17:03:26.477" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19379" PostTypeId="2" ParentId="12868" CreationDate="2022-07-12T00:45:19.410" Score="0" Body="&lt;p&gt;You can do it &lt;a href=&quot;https://kb.10xgenomics.com/hc/en-us/articles/360022448251-How-to-filter-the-BAM-file-produced-by-10x-pipelines-with-a-list-of-barcodes-&quot; rel=&quot;nofollow noreferrer&quot;&gt;with samtools&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Save the header lines&#xA;samtools view -H $BAM_FILE &amp;gt; SAM_header&#xA;&#xA;# Filter alignments using filter.txt. Use LC_ALL=C to set C locale instead of UTF-8&#xA;samtools view $BAM_FILE | LC_ALL=C grep -F -f filter.txt &amp;gt; filtered_SAM_body&#xA;&#xA;# Combine header and body&#xA;cat SAM_header filtered_SAM_body &amp;gt; filtered.sam&#xA;&#xA;# Convert filtered.sam to BAM format&#xA;samtools view -b filtered.sam &amp;gt; filtered.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And 10x Genomics has developed &lt;a href=&quot;https://github.com/10XGenomics/subset-bam&quot; rel=&quot;nofollow noreferrer&quot;&gt;subset-bam&lt;/a&gt; a tool that can split a BAM file by barcode.&lt;/p&gt;&#xA;" OwnerUserId="3575" LastActivityDate="2022-07-12T00:45:19.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19380" PostTypeId="2" ParentId="19369" CreationDate="2022-07-12T02:29:58.513" Score="1" Body="&lt;p&gt;See &lt;a href=&quot;https://yulab-smu.top/treedata-book/faq.html#change-colors-or-line-types-of-arbitrarily-selected-branches&quot; rel=&quot;nofollow noreferrer&quot;&gt;change colors or line types of arbitrarily selected branches&lt;/a&gt; in &lt;code&gt;ggtree&lt;/code&gt; documentation for details. Here's my R code for applying a color to a specific branch in &lt;code&gt;ggtree&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;suppressPackageStartupMessages(library(ggtree))&#xA;# read tree data&#xA;tree &amp;lt;- read.newick('tree.nwk')&#xA;# create a dataframe of the default color (black) for branches&#xA;branchColor &amp;lt;- data.frame(node = 1:Nnode2(tree), colour = 'black')&#xA;# set a color for the specifc branch&#xA;branchColor[7, 2] &amp;lt;- 'blue'&#xA;branchColor[3, 2] &amp;lt;- 'orange'&#xA;branchColor[6, 2] &amp;lt;- 'purple'&#xA;branchColor[c(1, 2), 2] &amp;lt;- 'red'&#xA;# plot tree&#xA;ggtree(tree, size = 1) %&amp;lt;+%&#xA;  branchColor +&#xA;  aes(colour = I(colour)) +&#xA;  geom_tippoint(size = 2, color = 'black') +&#xA;  geom_tiplab(size = 5, color = 'black') +&#xA;  geom_treescale(x = 0, y = 0, width = 0.5, fontsize = 5, linesize = 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The tree looks like this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/12LJS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/12LJS.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15583" LastActivityDate="2022-07-12T02:29:58.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19383" PostTypeId="1" CreationDate="2022-07-12T09:21:39.443" Score="2" ViewCount="96" Body="&lt;p&gt;We know that in next-generation sequencing (NGS), the unique molecular identifier (UMI) can reduce or eliminate sequencing or PCR errors and result in very high accurate data. Therefore, UMI is widely used in rare mutation detection. A typical application is cancer early detection or minimal residual disease (MRD) detection that try to find the very rare DNA from cancer cells in the blood.&lt;/p&gt;&#xA;&lt;p&gt;However, we also know that the cell-free DNA in the blood is very low, typically 5 ng in 1 mL blood. And it usually takes about 10 mL blood to get about 50 ng DNA to get sequenced with UMI. Limited DNA input leads to many false negatives.&lt;/p&gt;&#xA;&lt;p&gt;So I came up with an idea: &lt;strong&gt;What if I got a sample with plenty of DNA? Can I detect rare mutations using UMI in this sample?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The UMI library preparation protocols I have met all use a small amount of DNA input, say 50ng. &lt;strong&gt;If I have 1μg DNA, do I have to cut it to 50ng to prepare the UMI library?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="6776" LastActivityDate="2022-09-05T05:18:51.447" Title="Can we do NGS library preparation using UMI with large amount of DNA input?" Tags="|ngs|mutations|library|umi|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19384" PostTypeId="2" ParentId="12868" CreationDate="2022-07-12T09:34:09.520" Score="0" Body="&lt;p&gt;If you have access to a lot of cores:&lt;/p&gt;&#xA;&lt;p&gt;10x has &lt;a href=&quot;https://github.com/10XGenomics/subset-bam&quot; rel=&quot;nofollow noreferrer&quot;&gt;subset-bam&lt;/a&gt; a pretty fast way to pull out all the cells with a specific CB/BX if you only need a small number.&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2022-07-12T09:34:09.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19385" PostTypeId="2" ParentId="19383" CreationDate="2022-07-12T13:55:34.460" Score="1" Body="&lt;p&gt;The question is&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Do I add 20-fold more DNA than the kit states?&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Its not my area - &lt;em&gt;this is wet-lab&lt;/em&gt; - but there will be a tolerance however 20-fold is excessive (&amp;gt;1 log), i.e. no it doesn't help and could cause (expensive) downstream problems. The people to contact are the support for the given kit, there are unlikely to support your idea.&lt;/p&gt;&#xA;&lt;p&gt;The answer isn't wet-lab but almost certainly is bioinformatics and detecting low frequency somatic driver mutations. The Broad Institute worked on this problem for years, its nothing new, i.e. identifying driver mutations in highly mixed sample comprising healthy somatic tissue and cancer cells.&lt;/p&gt;&#xA;&lt;p&gt;I've tracked their work and have a reasonable understanding of what they do and why: oncology isn't my field. Your in-house bioinformatician should know this stuff.&lt;/p&gt;&#xA;&lt;p&gt;One suggest is using &lt;a href=&quot;https://software.broadinstitute.org/cancer/cga/mutect&quot; rel=&quot;nofollow noreferrer&quot;&gt;MuTect&lt;/a&gt; from the Broad Institute, I assume there have been many custom solutions since then. Their famous product is GATK for VCF, I don't think thats the solution here.&lt;/p&gt;&#xA;&lt;p&gt;Again cancer isn't my area BTW this is just common knowledge within bioinformatics, i.e. knowing what someone else is doing and why is part of the knowledge base.&lt;/p&gt;&#xA;&lt;p&gt;Please do try to retain a bioinformatics focus for future questions.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-07-12T14:35:11.420" LastActivityDate="2022-07-12T14:35:11.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19386" PostTypeId="1" CreationDate="2022-07-12T20:32:43.477" Score="1" ViewCount="17" Body="&lt;p&gt;We are getting sequence data from microbiota samples as part of a FDA submission identifying changes in Micobiota from skin samples over time and compared with different treatments.  the data will be first presented in FASTA formate then analyzed for OTUs and differences.  The submission has to follow eCTD format.&lt;/p&gt;&#xA;" OwnerUserId="15762" LastActivityDate="2022-07-12T20:32:43.477" Title="Can FASTA data be submitted directly into eCTD submission to the FDA?" Tags="|fasta|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19388" PostTypeId="1" CreationDate="2022-07-13T15:34:18.583" Score="1" ViewCount="39" Body="&lt;p&gt;I'm trying to find if the peptides in column A are present within the longer peptides of column B.&lt;/p&gt;&#xA;&lt;p&gt;Pairwise alignment tools take the data from both columns as one string (i.e., TIIDYTNNHYTNNHLEII... and PFYMNITGYTDAYNTDIIDYIIDYTNNHLEIITLFN...) which leads to a number of peptides from column A left unaligned even though the peptide is present within one of the longer peptides in column B.&lt;/p&gt;&#xA;&lt;p&gt;I would like to find a method that works for detecting such peptides.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/i621I.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/i621I.jpg&quot; alt=&quot;columns depicted here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15767" LastEditorUserId="2108" LastEditDate="2022-07-13T17:12:55.313" LastActivityDate="2022-07-13T17:14:56.063" Title="Problem with sequence alignment" Tags="|proteins|sequence-alignment|sequence|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19389" PostTypeId="2" ParentId="19388" CreationDate="2022-07-13T17:14:56.063" Score="3" Body="&lt;p&gt;I don't think that you want an &lt;em&gt;alignment&lt;/em&gt; so much as you want to &lt;em&gt;test for the presence of an exact substring&lt;/em&gt; (column A) &lt;em&gt;within larger strings&lt;/em&gt; (column B).&lt;/p&gt;&#xA;&lt;p&gt;There are simple excel utilities that do this, for example the &lt;code&gt;SEARCH()&lt;/code&gt; &lt;a href=&quot;https://exceljet.net/formula/cell-contains-specific-text&quot; rel=&quot;nofollow noreferrer&quot;&gt;function&lt;/a&gt;. There is no need to use alignments.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-07-13T17:14:56.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19390" PostTypeId="2" ParentId="18719" CreationDate="2022-07-13T19:23:40.803" Score="1" Body="&lt;p&gt;According to a proteomics researcher at the NIH:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;proteins&lt;/code&gt; column contains uniprot IDs.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;position within proteins&lt;/code&gt; column are the starting position of each ID.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;position&lt;/code&gt; is the &lt;em&gt;leading&lt;/em&gt; protein&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="5187" LastActivityDate="2022-07-13T19:23:40.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19391" PostTypeId="2" ParentId="8853" CreationDate="2022-07-14T00:22:48.193" Score="2" Body="&lt;p&gt;The problem is that you are transposing the vector of GA values with &lt;code&gt;t(ano$GA)&lt;/code&gt;. Why would you do that? It produces a row matrix that is inappropriate for input to &lt;code&gt;model.matrix&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;All you need is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;design &amp;lt;- model.matrix(~ano$GA)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and everything will work fine.&lt;/p&gt;&#xA;" OwnerUserId="9042" LastEditorUserId="9042" LastEditDate="2022-07-14T03:16:12.347" LastActivityDate="2022-07-14T03:16:12.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19392" PostTypeId="2" ParentId="18572" CreationDate="2022-07-14T00:29:43.757" Score="2" Body="&lt;p&gt;There is no need for batch correction because all your comparisons are within batch. As long as you are making comparisons for each line, and treatment and control for the same line are in the same batch, then any batch effects are already accounted. for.&lt;/p&gt;&#xA;" OwnerUserId="9042" LastActivityDate="2022-07-14T00:29:43.757" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19393" PostTypeId="1" CreationDate="2022-07-14T02:23:48.087" Score="1" ViewCount="105" Body="&lt;p&gt;I have two matrices, one for individuals before treatment and one for the same individuals after treatment. Both matrices are raw read counts of RNA expression.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;        Treated_1  Treated_2  Treated_3  Treated_4  Treated_5&#xA;RNA_1   105        283        64         155        51&#xA;RNA_2   359        27         47         348        84&#xA;RNA_3   99         10         89         345        77&#xA;RNA_4   48         100        77         74         83&#xA;&#xA;        Untreated_1  Untreated_2  Untreated_3  Untreated_4  Untreated_5&#xA;RNA_1   100          200          50           130          33&#xA;RNA_2   200          10           30           300          75&#xA;RNA_3   90           10           60           320          60&#xA;RNA_4   66           50           33           60           55&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I tell which RNA's are significantly differentially expressed in R using &lt;code&gt;edgeR&lt;/code&gt; or &lt;code&gt;DESeq2&lt;/code&gt;? I'm thinking first normalize using TMM then compare the mean of the normalized values for treated vs. untreated?&lt;/p&gt;&#xA;" OwnerUserId="15769" LastEditorUserId="15769" LastEditDate="2022-07-14T02:48:46.857" LastActivityDate="2022-07-14T08:49:07.320" Title="Basic RNA Differential Expression in R" Tags="|r|deseq2|differential-expression|rna|edger|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19394" PostTypeId="2" ParentId="19393" CreationDate="2022-07-14T04:40:14.650" Score="3" Body="&lt;p&gt;See here for a Differential Expression guide which discusses carrying out differential expression using DESeq2:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It starts with a standard &amp;quot;quick start&amp;quot; workflow, and goes into quite a lot of detail explaining all the different things that can be done:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData = cts,&#xA;                              colData = coldata,&#xA;                              design= ~ batch + condition)&#xA;dds &amp;lt;- DESeq(dds)&#xA;resultsNames(dds) # lists the coefficients&#xA;res &amp;lt;- results(dds, name=&amp;quot;condition_trt_vs_untrt&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-07-14T04:40:14.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19395" PostTypeId="2" ParentId="19393" CreationDate="2022-07-14T07:55:07.853" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;I'm thinking first normalize using TMM then compare the mean of the normalized values for treated vs. untreated?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;That is a bit too simple. You need to analyse your experiment as a paired comparison. You need to combine the treated and untreated count matrices into one and form factors for individual and treatment:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Counts &amp;lt;- cbind(Counts.treated, Counts.untreated)&#xA;Individual &amp;lt;- gl(5,1,length=10)&#xA;Treatment &amp;lt;- gl(2,5,length=10,labels=c(&amp;quot;Treated&amp;quot;,&amp;quot;Untreated&amp;quot;))&#xA;Treatment &amp;lt;- relevel(Treatment, ref=&amp;quot;Untreated&amp;quot;)&#xA;design &amp;lt;- model.matrix(~Individual+Treatment)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then follow a standard workflow:&#xA;&lt;a href=&quot;https://bioconductor.org/packages/release/workflows/vignettes/RnaSeqGeneEdgeRQL/inst/doc/edgeRQL.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/workflows/vignettes/RnaSeqGeneEdgeRQL/inst/doc/edgeRQL.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(edgeR)&#xA;y &amp;lt;- DGEList(counts=Counts)&#xA;y &amp;lt;- calcNormFactors(y)&#xA;y &amp;lt;- estimateDisp(y,design)&#xA;fit &amp;lt;- glmQLFit(y,design)&#xA;test &amp;lt;- glmQLFTest(fit)&#xA;topTags(test)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9042" LastEditorUserId="9042" LastEditDate="2022-07-14T08:49:07.320" LastActivityDate="2022-07-14T08:49:07.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19396" PostTypeId="1" CreationDate="2022-07-14T13:20:46.023" Score="2" ViewCount="302" Body="&lt;p&gt;I have ~500,000 rsids - but want which chromosome each SNP resides on. How can I do this?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MarkerName A1 A2 &#xA;rs2326918 a g &#xA;rs7929618 c g &#xA;rs66941928 t c &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want the chromosome number associated with each MarkerName. So the output would be like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MarkerName A1 A2 Chr&#xA;rs2326918 a g     6 &#xA;rs7929618 c g     6&#xA;rs66941928 t c    6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can i do this?&lt;/p&gt;&#xA;" OwnerUserId="609" LastEditorUserId="609" LastEditDate="2022-07-14T13:25:02.547" LastActivityDate="2023-08-19T12:08:05.997" Title="Retrieve chromosome number with just RSID?" Tags="|bioconductor|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19397" PostTypeId="1" AcceptedAnswerId="19399" CreationDate="2022-07-15T08:04:25.577" Score="1" ViewCount="180" Body="&lt;p&gt;I would like to convert the fastq files from this dataset: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE98679&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE98679&lt;/a&gt; to .bam files.  The group did not provide the scripts they used to to their alignment, I know they already posted their reads but I need the .bam files for my pipeline.  Is it possible to do this without knowing the cell barcodes?  If so with what tools?&lt;/p&gt;&#xA;" OwnerUserId="2482" LastActivityDate="2022-07-15T15:18:13.110" Title="Aligning scRNA-seq fastq to .bam without cell barcodes" Tags="|sequence-alignment|scrnaseq|sam|fastq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19399" PostTypeId="2" ParentId="19397" CreationDate="2022-07-15T11:04:39.303" Score="1" Body="&lt;p&gt;The linked dataset is based on the Smart-seq2 technology. This is a plate-based assay in which individual cells are first sorted into individual wells of a microwell plate via FACS. The cells are then individually lysed and libraries are prepared. Since this happens per well one assigns a single sequencing barcode to each well/cell, hence no further cellular barcoding as in droplet-based methods such as 10x Chromatium is required. Long story short, each fastq file that one obtains from the sequencing represents one individual cell. Hence, one can use any standard RNA-seq alignment or quantification technique such as &lt;code&gt;STAR&lt;/code&gt; or &lt;code&gt;salmon&lt;/code&gt; to obtain alignments/quantifications. One would then need to parse the individual quantifications into a single count matrix. If one used aligners such as &lt;code&gt;STAR&lt;/code&gt; this could be done with high-performance quantifiers such as &lt;code&gt;featureCounts&lt;/code&gt;. If using pseudo- or selective aligners such as &lt;code&gt;salmon&lt;/code&gt; or &lt;code&gt;kallisto&lt;/code&gt; I would use &lt;code&gt;tximport&lt;/code&gt; to aggregate the per-transcript quantifications to the gene level while importing into R to get a gene-level count matrix. As each file is a single cell there is no need for more complicated solutions such as &lt;code&gt;CellRanger&lt;/code&gt; which do not apply here.&lt;/p&gt;&#xA;" OwnerUserId="15777" LastEditorUserId="-1" LastEditDate="2022-07-15T15:18:13.110" LastActivityDate="2022-07-15T15:18:13.110" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19400" PostTypeId="1" CreationDate="2022-07-15T13:35:18.850" Score="1" ViewCount="1123" Body="&lt;p&gt;I have several protein sequences with an asterisk in the middle of the sequence, and I'd like to remove those sequences.&lt;/p&gt;&#xA;&lt;p&gt;In the example below, I have three sequences:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NXLBh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NXLBh.png&quot; alt=&quot;Enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;From this example, I would like to obtain only the following output:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nuMbc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nuMbc.png&quot; alt=&quot;Enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15596" LastEditorUserId="3848" LastEditDate="2022-12-03T13:43:53.843" LastActivityDate="2022-12-03T13:43:53.843" Title="How can I remove FASTA sequences with an asterisk inside the sequence?" Tags="|fasta|phylogenetics|proteins|phylogeny|sequence|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19401" PostTypeId="2" ParentId="19400" CreationDate="2022-07-15T14:06:11.463" Score="4" Body="&lt;p&gt;Try &lt;code&gt;seqkit grep&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqkit grep --by-seq --invert-match --pattern '*' seq.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;--by-seq&lt;/code&gt; search by seq&lt;/p&gt;&#xA;&lt;p&gt;It will output sequences without an asterisk.&lt;/p&gt;&#xA;" OwnerUserId="15583" LastEditorUserId="15583" LastEditDate="2022-07-15T14:32:06.223" LastActivityDate="2022-07-15T14:32:06.223" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19402" PostTypeId="2" ParentId="19400" CreationDate="2022-07-15T15:18:01.320" Score="4" Body="&lt;p&gt;It’s  good you brought up the question.&lt;/p&gt;&#xA;&lt;h3&gt;&lt;code&gt;*&lt;/code&gt; are stop codons&lt;/h3&gt;&#xA;&lt;p&gt;Don't simply parse out the &lt;code&gt;*&lt;/code&gt; symbol. If done incorrectly, it could constitute &lt;strong&gt;'tampering'&lt;/strong&gt; and that’s a &lt;em&gt;red line&lt;/em&gt; in biology.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;*&lt;/code&gt; means a stop codon in the middle of a protein sequence. There are quite a number of reasons why there is an &lt;code&gt;*&lt;/code&gt;, i.e., a stop codon in the sequence, for example, in no particular order:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the open reading frame is out of frame which is part of the natural molecular biology, i.e., there can be multiple reading frames in a gene&lt;/li&gt;&#xA;&lt;li&gt;the open reading frame is out of frame due to a sequencing error and poor annotation (which is common);&lt;/li&gt;&#xA;&lt;li&gt;the wrong reading frame was used (very, very common);&lt;/li&gt;&#xA;&lt;li&gt;the locus isn't translated, e.g., it’s an intron or UTR (untranslated region)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The list goes on, e.g., it could be expression regulation ...&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If it is not an intron/UTR the way you might 'remove' &lt;code&gt;*&lt;/code&gt; is by changing the reading frame, e.g., try using the second codon position (then the third) as the opening reading frame: if the &lt;code&gt;*&lt;/code&gt; 'disappear' it was very likely just a reading frame issue. How to do that is a separate question. The resulting amino acid sequence will of course be very different.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There's a second possibility if the reading frame is correct, which is to treat each locus between the &lt;code&gt;*&lt;/code&gt; as a separate protein. That way the &lt;code&gt;*&lt;/code&gt; is 'removed' because their a multiple proteins within the locus. In that case, a &lt;code&gt;\n&lt;/code&gt; would replace the &lt;code&gt;*&lt;/code&gt; (Mac and Linux), i.e., carriage return.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="1775" LastEditDate="2022-07-16T13:56:58.370" LastActivityDate="2022-07-16T13:56:58.370" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19403" PostTypeId="1" CreationDate="2022-07-16T05:56:00.577" Score="1" ViewCount="42" Body="&lt;p&gt;I have a 3.7 GB whole genome data of a eukaryote, for which genome assembly, gene prediction, and annotation steps have to be performed. In some time I would also need to analyze the transcriptome data of the organism. Can someone please let me know what are the system requirements for carrying out the aforementioned NGS data analysis? Thank you!&lt;/p&gt;&#xA;" OwnerUserId="15783" LastActivityDate="2022-07-16T05:56:00.577" Title="System specifications for NGS data analysis" Tags="|genome|ngs|transcriptome|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19404" PostTypeId="1" CreationDate="2022-07-16T09:16:42.333" Score="1" ViewCount="36" Body="&lt;p&gt;I'm looking for some good molecular docking software (preferably, one that comes with a Python API). I obviously found Vina and that's been quite helpful but I was wondering if there's anything out there that's more accurate and reliable ? Since there's so much software and publications out there, I thought someone here might be aware of some things that may be overlooked.&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance !&lt;/p&gt;&#xA;" OwnerUserId="12431" LastActivityDate="2022-07-16T09:16:42.333" Title="Good molecular docking software?" Tags="|docking|cheminformatics|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="19405" PostTypeId="2" ParentId="11115" CreationDate="2022-07-16T11:23:30.180" Score="1" Body="&lt;p&gt;Here's a python script based on the &lt;a href=&quot;https://github.com/chapmanb/bcbb/tree/master/gff&quot; rel=&quot;nofollow noreferrer&quot;&gt;bcbio-gff&lt;/a&gt; (reading gff3) and &lt;a href=&quot;https://biopython.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;biopython&lt;/a&gt; (writing genbank): &lt;a href=&quot;https://github.com/chapmanb/bcbb/blob/master/gff/Scripts/gff/gff_to_genbank.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/chapmanb/bcbb/blob/master/gff/Scripts/gff/gff_to_genbank.py&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="123" LastActivityDate="2022-07-16T11:23:30.180" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19406" PostTypeId="2" ParentId="19383" CreationDate="2022-07-16T11:49:21.567" Score="1" Body="&lt;p&gt;Whole genome sequencing uses between 50 ng and 1 μg of high-quality DNA (A260/A280: 1.7–2.0) for PCR-amplified or non-amplified libraries if your focus is precision oncology as described in the following open access review in &lt;a href=&quot;https://drive.google.com/file/d/1sRrhn_oCpf1cxnFshPl0-7XGjtZAkUfm/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Seminars in Cancer Biology paper by Meggendorfer &lt;em&gt;et al&lt;/em&gt; 2022&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15710" LastEditorUserId="3848" LastEditDate="2022-07-16T14:04:07.813" LastActivityDate="2022-07-16T14:04:07.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19408" PostTypeId="2" ParentId="19400" CreationDate="2022-07-16T19:38:17.097" Score="1" Body="&lt;p&gt;You could use Biopython for this. It makes FASTA parsing really easy.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;'''&#xA;Python version: 3.10.0&#xA;Biopython version: 1.79&#xA;'''&#xA;from Bio import SeqIO&#xA;for seq_record in SeqIO.parse(&amp;quot;input.fasta&amp;quot;, &amp;quot;fasta&amp;quot;):&#xA;    if &amp;quot;*&amp;quot; not in (seq_record.seq):&#xA;        print(&amp;quot;&amp;gt;&amp;quot;,seq_record.id)&#xA;        print(seq_record.seq)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2022-07-16T19:38:17.097" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19409" PostTypeId="2" ParentId="19334" CreationDate="2022-07-17T14:14:18.823" Score="3" Body="&lt;p&gt;Although Insulin is a great, well studied example of a small structure it does have the slight downside of not having any sheet structure in the monomer. There is a small cross-chain sheet between residues 24-26 in both the B and D chains (from &lt;a href=&quot;https://www.ebi.ac.uk/pdbe/entry/pdb/1zni/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/pdbe/entry/pdb/1zni/&lt;/a&gt; ).&lt;/p&gt;&#xA;&lt;p&gt;Another possibility might be crambin (&lt;a href=&quot;http://www.cathdb.info/version/latest/domain/1ejgA00&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cathdb.info/version/latest/domain/1ejgA00&lt;/a&gt;) which does have a small antiparallel sheet, although it is tiny at only 2 residues per strand.&lt;/p&gt;&#xA;&lt;p&gt;A more general approach might be to use something like the top2018 structure set from the Richardson lab (&lt;a href=&quot;https://zenodo.org/record/5777651#.YtQB59LMJH4&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://zenodo.org/record/5777651#.YtQB59LMJH4&lt;/a&gt;) . I used the top500 back in the day, and they are a high-quality set of structures.&lt;/p&gt;&#xA;" OwnerUserId="745" LastActivityDate="2022-07-17T14:14:18.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19410" PostTypeId="1" AcceptedAnswerId="19416" CreationDate="2022-07-17T17:18:08.423" Score="1" ViewCount="73" Body="&lt;p&gt;I am a computational scientist with background in computer science and engineering however very new to the field of bioinformatics and computational genomics. Looking at papers in nature, PLOS, Cell, PNAS etc. within the topic of computational genomics, I see a very similar set of very sophisticated graphical results, ranging from clustered/sorted heat map representation of gene expressions to log2 fold changes, volcano plots, circular graph network visualizations etc. Coming from an engineering background I would often generate customized tools for visualizing results myself as part of each paper, albeit not at the sophistication level of results I see in bioinformatics. But here, it seems there is a set of tools standardized where many are using them to display their results. Is there a summary of these result generation/visualization tools along with the references to their implementation/usage (opensource I assume)? Something like that could be very helpful to newcomers like myself. Thanks in advance.&lt;/p&gt;&#xA;&lt;p&gt;Below is an example /elaboration based to guide the question further into specifics:&lt;/p&gt;&#xA;&lt;p&gt;Consider the example paper:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1016/j.cell.2018.04.018&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1016/j.cell.2018.04.018&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This paper contains a principle set of results/visualizations I see almost universally in many nature/cell computational genomics papers. I am wondering if there exists an &lt;strong&gt;integrated library&lt;/strong&gt; either in python, R (or Matlab for example when they create toolboxes and integrate a commonly used subset of tools in an area) that everyone taps into for generating these type of results. The &amp;quot;principal set&amp;quot; that I am referring to include:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Genome browser track snippets: Figure 1A&lt;/li&gt;&#xA;&lt;li&gt;log2Fold x-y scatter plots : Figure 1B&lt;/li&gt;&#xA;&lt;li&gt;Hierarchically clustered/sorted gene exp heatmaps: Figure 1C,1F&lt;/li&gt;&#xA;&lt;li&gt;Venn diagrams : Figure 4F&lt;/li&gt;&#xA;&lt;li&gt;Hi-C type square ligation heat-maps: Figure 6A&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="15741" LastEditorUserId="15741" LastEditDate="2022-07-18T06:06:43.443" LastActivityDate="2022-07-18T09:38:43.143" Title="Common tools for generating results and visualizations for bioinformatics and genomics publications" Tags="|bioconductor|visualization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19411" PostTypeId="1" AcceptedAnswerId="19412" CreationDate="2022-07-17T19:39:33.467" Score="2" ViewCount="49" Body="&lt;p&gt;I have identified a conserved region in a protein I'm studying, by doing multiple sequence alignments against a dozen of its orthologs. I've then found that this region also aligns relatively well with a functional region (binding motif) in a completely unrelated protein, and I'm trying to estimate whether it's reasonable to believe the two regions might be bound by the same protein. blastp doesn't identify this match, would I then assume that this is insignificant?&lt;/p&gt;&#xA;&lt;p&gt;Let's say I have a 15 aa long peptide sequence &lt;strong&gt;X&lt;/strong&gt;, which I align against a longer 200 aa protein sequence &lt;strong&gt;Y&lt;/strong&gt;. I find some best match (gaps allowed).&lt;/p&gt;&#xA;&lt;p&gt;What statistical test could I now perform to answer the question:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Is the match I found for sequence &lt;strong&gt;X&lt;/strong&gt; in sequence &lt;strong&gt;Y&lt;/strong&gt; significantly better than what I would expect to find between &lt;strong&gt;X&lt;/strong&gt; and a random 200 aa sequence &lt;strong&gt;Z&lt;/strong&gt;?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="11919" LastEditorUserId="73" LastEditDate="2022-07-19T06:12:50.377" LastActivityDate="2022-07-19T06:12:50.377" Title="Statistical significance of pairwise protein sequence alignment" Tags="|sequence-alignment|statistics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19412" PostTypeId="2" ParentId="19411" CreationDate="2022-07-17T20:42:17.133" Score="1" Body="&lt;p&gt;Just formalise the answer, its a straight use of the Blast E-value and I wouldn't personally advise against anything beyond that (explained below) within molecular evolution.&lt;/p&gt;&#xA;&lt;p&gt;So the Google gives the definition&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The BLAST E-value is the number of expected hits of similar quality&#xA;(score) that could be found just by chance. E-value of 10 means that&#xA;up to 10 hits can be expected to be found just by chance, given the&#xA;same size of a random database.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Thus the smaller the E-value the better the hit.&lt;/p&gt;&#xA;&lt;p&gt;Using a local blast server will provide a specific query verse reference E-value.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastp -query gene.fa -subject genome.fa &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The broader question concerns the whether the Blast hit is a genuine homology and that is very complicated. This is simply because amino acid loci could have arisen by convergent evolution rather than direct descendancy which are homologues and this can become controversial.&lt;/p&gt;&#xA;&lt;p&gt;When trying to assign a function to a amino acid locus this starts becoming relevant because e.g. the selection pressures of convergence could very difference.&lt;/p&gt;&#xA;&lt;p&gt;Identifying homologues would require a phylogenetic tree which is compatible with the established phylogenetic tree, e.g. for house-keeping genes as a starting point. It gets complex quickly beyond that. Functional domains within the Blast variant (Blast-Psi) could be of use, its field specific however.&lt;/p&gt;&#xA;&lt;p&gt;Thus the simple answer is to use Blast E-value (bit-score) and it is a universally recognised metric, the reader can then make up their own mind what the value represents. You could simply simple state the definition of the E-value in the report.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-07-17T20:55:00.023" LastActivityDate="2022-07-17T20:55:00.023" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19415" PostTypeId="1" CreationDate="2022-07-18T07:01:00.987" Score="2" ViewCount="33" Body="&lt;p&gt;Integral membrane proteins are proteins that have at least one transmembrane (TM) segment.&#xA;I performed predictions of TM domains using three tools: ProtScale, ΔG predictor 1.0, TOPCONS.&lt;/p&gt;&#xA;&lt;p&gt;For example, the results in terms of aminoacid positions for the TM1 domain were as follows:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;ProtScale&lt;/strong&gt;: 37-55&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;ΔG predictor&lt;/strong&gt;: 37-55&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;TOPCONS&lt;/strong&gt;: 36-56&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The limits of the predicted TM domains can be used for drawing the topology of this protein.&lt;/p&gt;&#xA;&lt;p&gt;For this purpose, we need to have representative limits from these results.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Statistically speaking:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;which values should be chosen?&lt;/li&gt;&#xA;&lt;li&gt;should we calculate a mean : (37+37+36)/3=36.67 knowing that these values correspond to aminoacid positions and 36.67 doesn't mean anything?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;How can I use these results (37, 37, 36) to determine the limit corresponding to the 1st aminoacid of TM1 segment?&lt;/p&gt;&#xA;" OwnerUserId="15792" LastEditorUserId="15792" LastEditDate="2022-07-18T12:07:13.890" LastActivityDate="2023-12-13T01:09:31.133" Title="Determination of transmembrane domain limits using multiple prediction tools" Tags="|membrane-protein|tools|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19416" PostTypeId="2" ParentId="19410" CreationDate="2022-07-18T09:38:43.143" Score="2" Body="&lt;p&gt;Figure 1A: These genome browser snippets are either plain screenshots from applications like &lt;a href=&quot;https://software.broadinstitute.org/software/igv/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IGV&lt;/a&gt; or created from within R with software like gviz.&lt;/p&gt;&#xA;&lt;p&gt;Figure 1B: This looks like base R plotting routine. Like &lt;code&gt;plot(x,y)&lt;/code&gt; and then some coloring and legending. No particular &amp;quot;tool&amp;quot; here other than really just base R or ggplot2.&lt;/p&gt;&#xA;&lt;p&gt;Figure 1C/F: Heatmap packages include ComplexHeatmap and pheatmap. I personally use the former as it is versatile with extensive documentation.&lt;/p&gt;&#xA;&lt;p&gt;Figure 4F: Venn diagrams, just google &amp;quot;Venn&amp;quot; and &amp;quot;R&amp;quot; for several packages that do it in R. Same for other languages. There are plenty of packages. No preference.&lt;/p&gt;&#xA;&lt;p&gt;Figure 6A: These plots are often returned by specialized HiC software such as &lt;a href=&quot;https://hicexplorer.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HiCexplorer&lt;/a&gt; which is powerful with lots of documentation.&lt;/p&gt;&#xA;" OwnerUserId="15793" LastActivityDate="2022-07-18T09:38:43.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19417" PostTypeId="1" CreationDate="2022-07-18T17:17:42.697" Score="1" ViewCount="61" Body="&lt;p&gt;Very new to bioinformatics. I am trying to figure out the steps required to convert a chip-seq fastq file into a genome browser track, with the end goal of being able to access the sequence (x) vs. activity level/count (y) data associated.&lt;/p&gt;&#xA;&lt;p&gt;Any genome browser activity/count track data has the following functional format&#xA;&lt;a href=&quot;https://i.stack.imgur.com/i3fP4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/i3fP4.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$y=F(x)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here x is the sequence axis, and y is the activity/count axis. Typically the x(i) is the nucleotide associated with the precise genomic coordinate i (an integer number) depending on the genome used to align (human, mouse, etc.). What I need is a format allowing direct access to the nucleotide representation x(i), associated with the index i, as well as the count levels at y(x(i)). What is the closest data file type that I need? And how do I get to it?&lt;/p&gt;&#xA;&lt;p&gt;In other words, what are the steps for a fastq file associated with a chip-seq data to be transformed to this x(i),y(i) format, or best yet if there is an example Python or R code out there that could illustrate the steps of going from file1.fastq to something like to two files &lt;strong&gt;file1_x_sequences&lt;/strong&gt; and &lt;strong&gt;file2_y_counts&lt;/strong&gt;? I only know the first step is an alignment to a genome, but stuck on the next steps to complete the picture. Thank you.&lt;/p&gt;&#xA;&lt;p&gt;Ultimately I need a list of sequences (tags) lets say 250bp long, and a count number that represents the chip-seq enrichment for the part of the genome (the 250bp region). Further reading, indicates I need to detect/call &amp;quot;peaks&amp;quot; with MACS2/homer to generate a BAM file and ultimately a Fasta file for the sequence lists, but I don't know yet how the 'counts' come in (in what file format and where from). Any hints there would complete the picture I think. Ultimately this is training data for x=250bp sequence to y=count/chip-seq enrichment activity model.&lt;/p&gt;&#xA;" OwnerUserId="15741" LastEditorUserId="73" LastEditDate="2022-07-19T06:11:16.000" LastActivityDate="2022-07-19T06:11:16.000" Title="Converting Chip-seq fastq format ({$x_k$} list of seq segments) to genome browser {sequence $x_i$, count-number $y_i$} format" Tags="|r|python|bioconductor|chip-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19419" PostTypeId="1" CreationDate="2022-07-18T19:44:09.877" Score="1" ViewCount="60" Body="&lt;p&gt;I have total of 6 scRNA seq data samples which can be divided into group A and B. I would like to create Seuratobject A and B by combining 3 samples counts data for each group. Eventually I would like to see UMAP with colored by group A and B. I tried many ways but failed. I can see UMAP with 6 colors representing 6 samples. Could you please help me with this issue?&#xA;Please find below my code for your reference-&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A1.data &amp;lt;-Read10X(data.dir = &amp;quot;C:/Users/mad1188/Box/SequencingData/cellranger_count/-A-7-1/filtered_feature_bc_matrix&amp;quot;)&#xA;A2.data &amp;lt;-Read10X(data.dir = &amp;quot;C:/Users/mad1188/Box/Sequencing Data/cellranger_count/-A-7-2/filtered_feature_bc_matrix&amp;quot;)&#xA;A3.data &amp;lt;-Read10X(data.dir = &amp;quot;C:/Users/mad1188/Box/Sequencing Data/cellranger_count/-A-7-3/filtered_feature_bc_matrix&amp;quot;)&#xA;A1 &amp;lt;- CreateSeuratObject(counts = A1.data,&#xA;project = &amp;quot;A7&amp;quot;)&#xA;A2 &amp;lt;- CreateSeuratObject(counts = A2.data,project = &amp;quot;A7&amp;quot;)&#xA;A3 &amp;lt;- CreateSeuratObject(counts = A3.data,project = &amp;quot;A7&amp;quot;)&#xA;head(x = colnames(x = A1))&#xA;A1 &amp;lt;- RenameCells(object = A1, add.cell.id = &amp;quot;A7&amp;quot;)&#xA;head(x = colnames(x = A1))&#xA;head(x = colnames(x = A2))&#xA;A2 &amp;lt;- RenameCells(object = A2, add.cell.id = &amp;quot;A7&amp;quot;)&#xA;head(x = colnames(x = A7))&#xA;head(x = colnames(x = A3))&#xA;A3 &amp;lt;- RenameCells(object = A3, add.cell.id = &amp;quot;A7&amp;quot;)&#xA;head(x = colnames(x = A3))&#xA;A1[[&amp;quot;percent.mt&amp;quot;]] &amp;lt;- PercentageFeatureSet(object = A1, pattern = &amp;quot;^MT-&amp;quot;)&#xA;A2[[&amp;quot;percent.mt&amp;quot;]] &amp;lt;- PercentageFeatureSet(object = A2, pattern = &amp;quot;^MT-&amp;quot;)&#xA;A3[[&amp;quot;percent.mt&amp;quot;]] &amp;lt;- PercentageFeatureSet(object = A3, pattern = &amp;quot;^MT-&amp;quot;)&#xA;p1 &amp;lt;- VlnPlot(object = A1, features = c(&amp;quot;nFeature_RNA&amp;quot;, &amp;quot;nCount_RNA&amp;quot;, &amp;quot;percent.mt&amp;quot;), ncol = 3)&#xA;p2 &amp;lt;- VlnPlot(object = A2, features = c(&amp;quot;nFeature_RNA&amp;quot;, &amp;quot;nCount_RNA&amp;quot;, &amp;quot;percent.mt&amp;quot;), ncol = 3)&#xA;p3 &amp;lt;- VlnPlot(object = A3, features = c(&amp;quot;nFeature_RNA&amp;quot;, &amp;quot;nCount_RNA&amp;quot;, &amp;quot;percent.mt&amp;quot;), ncol = 3)&#xA;CombinePlots(plots = list(p1, p2, p3), ncol =1)&#xA;p4 &amp;lt;- FeatureScatter(object = A1, feature1 = &amp;quot;nCount_RNA&amp;quot;, feature2 = &amp;quot;percent.mt&amp;quot;)&#xA;p5 &amp;lt;- FeatureScatter(object = A1, feature1 = &amp;quot;nCount_RNA&amp;quot;, feature2 = &amp;quot;nFeature_RNA&amp;quot;)&#xA;p6 &amp;lt;- FeatureScatter(object = A2, feature1 = &amp;quot;nCount_RNA&amp;quot;, feature2 = &amp;quot;percent.mt&amp;quot;)&#xA;p7 &amp;lt;- FeatureScatter(object = A2, feature1 = &amp;quot;nCount_RNA&amp;quot;, feature2 = &amp;quot;nFeature_RNA&amp;quot;)&#xA;p8 &amp;lt;- FeatureScatter(object = A3, feature1 = &amp;quot;nCount_RNA&amp;quot;, feature2 = &amp;quot;percent.mt&amp;quot;)&#xA;p9 &amp;lt;- FeatureScatter(object = A3, feature1 = &amp;quot;nCount_RNA&amp;quot;, feature2 = &amp;quot;nFeature_RNA&amp;quot;)&#xA;CombinePlots(plots = list(p4, p5, p6, p7, p8, p9), ncol =2)&#xA;A1 &amp;lt;- subset(x = A1, subset = nFeature_RNA &amp;gt; 200 &amp;amp; nFeature_RNA &amp;lt; 8000 &amp;amp; percent.mt &amp;lt; 20)&#xA;A2 &amp;lt;- subset(x = A2, subset = nFeature_RNA &amp;gt; 200 &amp;amp; nFeature_RNA &amp;lt; 8000 &amp;amp; percent.mt &amp;lt; 20)&#xA;A3 &amp;lt;- subset(x = A3, subset = nFeature_RNA &amp;gt; 200 &amp;amp; nFeature_RNA &amp;lt; 8000 &amp;amp; percent.mt &amp;lt; 20)&#xA;p10 &amp;lt;- VlnPlot(object = A1, features = c(&amp;quot;nFeature_RNA&amp;quot;, &amp;quot;nCount_RNA&amp;quot;, &amp;quot;percent.mt&amp;quot;), ncol = 3)&#xA;p11 &amp;lt;- VlnPlot(object = A2, features = c(&amp;quot;nFeature_RNA&amp;quot;, &amp;quot;nCount_RNA&amp;quot;, &amp;quot;percent.mt&amp;quot;), ncol = 3)&#xA;p12 &amp;lt;- VlnPlot(object = A3, features = c(&amp;quot;nFeature_RNA&amp;quot;, &amp;quot;nCount_RNA&amp;quot;, &amp;quot;percent.mt&amp;quot;), ncol = 3)&#xA;CombinePlots(plots = list(p10, p11, p12), ncol =1)&#xA;p13 &amp;lt;- FeatureScatter(object = A1, feature1 = &amp;quot;nCount_RNA&amp;quot;, feature2 = &amp;quot;percent.mt&amp;quot;)&#xA;p14 &amp;lt;- FeatureScatter(object = A1, feature1 = &amp;quot;nCount_RNA&amp;quot;, feature2 = &amp;quot;nFeature_RNA&amp;quot;)&#xA;p15 &amp;lt;- FeatureScatter(object = A2, feature1 = &amp;quot;nCount_RNA&amp;quot;, feature2 = &amp;quot;percent.mt&amp;quot;)&#xA;p16 &amp;lt;- FeatureScatter(object = A2, feature1 = &amp;quot;nCount_RNA&amp;quot;, feature2 = &amp;quot;nFeature_RNA&amp;quot;)&#xA;p17 &amp;lt;- FeatureScatter(object = A3, feature1 = &amp;quot;nCount_RNA&amp;quot;, feature2 = &amp;quot;percent.mt&amp;quot;)&#xA;p18 &amp;lt;- FeatureScatter(object = A3, feature1 = &amp;quot;nCount_RNA&amp;quot;, feature2 = &amp;quot;nFeature_RNA&amp;quot;)&#xA;CombinePlots(plots = list(p13, p14, p15, p16, p17, p18), ncol =2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15795" LastEditorUserId="3848" LastEditDate="2022-07-18T20:02:04.363" LastActivityDate="2022-07-18T20:02:04.363" Title="How to create Seuratobject using 3 samples" Tags="|seurat|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19420" PostTypeId="1" AcceptedAnswerId="19421" CreationDate="2022-07-18T20:51:44.177" Score="1" ViewCount="240" Body="&lt;p&gt;I have used the empirical RUVg method (from RUVseq) to estimate the unwanted variation of my dataset (consisting of several public datasets analysed together, with controls and case samples in different datasets.&lt;/p&gt;&#xA;&lt;p&gt;Afterwards, I used EdgeR to perform my DE analysis with 2 RUV covariates:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;designRUV &amp;lt;- model.matrix(~0 + condition4 + W_1 + W_2, data = pData(set2))&#xA;&#xA;y &amp;lt;- DGEList(counts=counts(set2), group=pheno$condition4)&#xA;y &amp;lt;- calcNormFactors(y, method=&amp;quot;upperquartile&amp;quot;)&#xA;y &amp;lt;- estimateGLMCommonDisp(y, designRUV)&#xA;y &amp;lt;- estimateGLMTagwiseDisp(y, designRUV)&#xA;fit &amp;lt;- glmFit(y, designRUV)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I got my results after making all the contrast that I needed but I would like to plot the expression of some genes of interest. One plot per gene with all the conditions in &lt;code&gt;condition4&lt;/code&gt; has 9 categories (expression level or norm counts on the Y axis).&lt;/p&gt;&#xA;&lt;p&gt;My doubt is which counts to use. I have the normalised ones from RUVg, but it states they should only use them for exploratory analysis (maybe this is the case??). These counts as I understand are the residuals of the model, so not really sure what to do.&lt;/p&gt;&#xA;&lt;p&gt;Another option is to use the normalised counts produced by EdgeR, but these ones doesn't account for the unwanted variation (W_1, W_2 covariates) (right???)&lt;/p&gt;&#xA;&lt;p&gt;Any advice or nice solution?? :=)&#xA;Thanks in advance!!&lt;/p&gt;&#xA;" OwnerUserId="8886" LastActivityDate="2022-07-19T18:19:41.650" Title="plotting gene expression after EdgeR DE analysis using RUVg (RUVseq) covariates" Tags="|rna-seq|normalization|edger|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19421" PostTypeId="2" ParentId="19420" CreationDate="2022-07-18T22:34:44.660" Score="2" Body="&lt;p&gt;You could correct your logCPMs (&lt;code&gt;edgeR::cpm(y, log=TRUE)&lt;/code&gt;) with the covariates from RUVseq using limma and then use these for plotting:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;set2 &amp;lt;- RUVg(...)&#xA;&#xA;designRUV &amp;lt;- model.matrix(~0 + condition4 + W_1 + W_2, data = pData(set2))&#xA;&#xA;y &amp;lt;- DGEList(counts=counts(set2), group=pheno$condition4)&#xA;y &amp;lt;- calcNormFactors(y, method=&amp;quot;upperquartile&amp;quot;)&#xA;y &amp;lt;- estimateGLMCommonDisp(y, designRUV)&#xA;y &amp;lt;- estimateGLMTagwiseDisp(y, designRUV)&#xA;fit &amp;lt;- glmFit(y, designRUV)&#xA;&#xA;# correction with limma&#xA;logcpm &amp;lt;- cpm(y, log=TRUE)&#xA;design &amp;lt;- model.matrix(~0+condition4, pData(set2))&#xA;corrected &amp;lt;- removeBatchEffect(logcpm, design=design, covariates=as.matrix(pData(set2)[,c(W_1,&amp;quot;W_2&amp;quot;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15793" LastEditorUserId="8886" LastEditDate="2022-07-19T18:19:41.650" LastActivityDate="2022-07-19T18:19:41.650" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19422" PostTypeId="2" ParentId="19417" CreationDate="2022-07-18T22:39:43.433" Score="5" Body="&lt;p&gt;I have no idea about what you are actually trying to achieve, but getting these browser tracks is simple. First you align your fastq files to a reference genome, e.g. with &lt;code&gt;bowtie2&lt;/code&gt;, this returns a SAM file. Sort the SAM file, convert to BAM and sort + index it with &lt;code&gt;samtools&lt;/code&gt;. Then use something like &lt;code&gt;bamCoverage&lt;/code&gt; from the deeptools suite to get your browser tracks.&lt;/p&gt;&#xA;&lt;p&gt;A word of caution. Since you seem to be new to OMICS and bioinformatics in general: NGS data, especially applications that are as noisy as ChIP-seq have lots of pitfalls. Be sure to go through the literature to find references for what you are trying to develop, at least the basic idea and discuss with experienced folks whether it makes sense. It sounds like you want counts of the peak summits, is that correct? In any case, be sure not to reinvent the wheel and to do a meaningful analysis. Discussion with experienced peers is key, that may save you from pitfalls and rookie mistakes. This entire choice of words &amp;quot;sequence axis&amp;quot;, &amp;quot;activity level&amp;quot; is very uncommon, I have never heard of this before despite working on NGS data for years, for what it's worth.&lt;/p&gt;&#xA;" OwnerUserId="15793" LastActivityDate="2022-07-18T22:39:43.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19425" PostTypeId="1" AcceptedAnswerId="19429" CreationDate="2022-07-19T14:22:17.543" Score="1" ViewCount="87" Body="&lt;p&gt;A BigWig file for something like Chip-seq, or ATAC data has the following format for each line i:&lt;/p&gt;&#xA;&lt;p&gt;chrom_i - start_i - end_i - value_i&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/buiri.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/buiri.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I need to convert for each line the (start_i -&amp;gt; end_i) reference to the actual DNA sequence 'ATATCCTAG...' so that at the end I have a list of DNA sequences like 'ATATCCTAG...' and the Chip-seq activity value associated with that sequence segment. What are the ways to do this? For example, what tool(s) can I use to generate the (sequence, value) pairs from browser tracks for the Chip-seq or ATAC etc. And in fact, going from BigWig browser track files for Chip-seq the most direct way to produce this kind of (seq,value) data? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="15741" LastActivityDate="2022-07-20T01:33:55.447" Title="How to extract DNA sequence from browser track input files (BigWig, bed etc.) files" Tags="|r|python|chip-seq|atac-seq|macs2|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19426" PostTypeId="2" ParentId="19425" CreationDate="2022-07-19T14:48:04.013" Score="1" Body="&lt;p&gt;Bedtools can take a list of coordinates and a fasta and return a multifasta of the specified regions.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2022-07-19T14:48:04.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19427" PostTypeId="1" CreationDate="2022-07-19T17:20:58.537" Score="2" ViewCount="522" Body="&lt;p&gt;I am new to bioinformatics and have been assigned the task of discovering which isoforms of a certain protein-coding gene are present in the RNA-seq data that I have. There are several isoforms of this gene listed on Entrez &amp;amp; Ensembl and I want to know which of these isoforms are expressed in the RNA-seq data that I have been given.&#xA;I am not sure how to begin to tackle this issue so If anyone has any ideas or advice, it would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="15801" LastEditorUserId="-1" LastEditDate="2022-07-19T19:52:40.833" LastActivityDate="2022-07-28T09:24:20.240" Title="How to identify which isoforms of a gene are actually expressed in my data?" Tags="|rna-seq|ensembl|entrez|isoform|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19428" PostTypeId="2" ParentId="19427" CreationDate="2022-07-19T17:57:43.920" Score="4" Body="&lt;p&gt;Tools like &lt;a href=&quot;https://github.com/pachterlab/kallisto&quot; rel=&quot;nofollow noreferrer&quot;&gt;kallisto&lt;/a&gt; or &lt;a href=&quot;https://github.com/COMBINE-lab/salmon&quot; rel=&quot;nofollow noreferrer&quot;&gt;salmon&lt;/a&gt; can perform a process call pseudo-alignment that matches reads from RNA-seq data to transcript isoforms present in a reference genome.&#xA;You can then look at the output of these tools to see which isoforms for a given gene appear to be present in your RNA-seq data.&#xA;These tools answer the question &amp;quot;how abundant is each transcript in my data?&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;But if you want to answer a question like &amp;quot;are there differences in which transcripts are present between experimental conditions?&amp;quot;, I'd look for tutorials on differential expression analysis, differential splicing analysis, or differential transcript usage.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2022-07-19T17:57:43.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19429" PostTypeId="2" ParentId="19425" CreationDate="2022-07-20T01:33:55.447" Score="1" Body="&lt;p&gt;[Here is the link.][1]. Instructions are very clear at the website&lt;br /&gt;&#xA;[1]: https://bedtools.readthedocs.io/en/latest/content/tools/getfasta.html&lt;/p&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2022-07-20T01:33:55.447" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19430" PostTypeId="1" CreationDate="2022-07-20T06:07:19.783" Score="2" ViewCount="170" Body="&lt;p&gt;I am attempting to identify DNA sequences/regions associated with enhancers in my NGS sequencing data. How can I identify (from peak files/counts for chip-seq, atac etc.) genomics regions/peaks associated with enhancers? I know proximity to genes and possibly histone (i.e. h3k27ac) activity near known genes is possibly an indicator, but is this the only way and how is the computational process formalized/captured? What are the specific set of tools involved? i.e. Peak annotations etc.? Thank you.&lt;/p&gt;&#xA;&lt;p&gt;As I understand it in my limited bio knowledge, enhancers are at varying distances from genes and sometimes they are multiples (super-enhancers). Are all enhancers already &amp;quot;known/described&amp;quot; (akin to genes)? What is/was the process of finding them (human manual procedure)? It's that more manual/heuristic process (without ML) I am looking for so we can consider formalizing and automating it.&lt;/p&gt;&#xA;&lt;p&gt;Based on a very insightful answer: I think some input/clarity on the following bit would make the answer(s) even more valuable:&lt;/p&gt;&#xA;&lt;p&gt;Possible list of steps towards this include following tools/functions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Bowtie (to align fastq to a genome, mouse, human etc.)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Homer/makeTagDirectory (Sep files, associate chr/coords with tags/sequence reads)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Homer/findPeaks (identify and list genomic regions/sequences with count values above a threshold)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Homer/pos2bed (convert peaks to bed file, genome browser compatible form)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Homer/annotatePeaks.pl (Runs on *bed file - Don't know much about this, but intuitively associate a label with peaks found in 4?)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Question: Is there a step 6? Where do we go after annotatedPeaks for example to get to enhancers?&lt;/p&gt;&#xA;" OwnerUserId="15741" LastEditorUserId="15741" LastEditDate="2022-07-22T17:06:50.067" LastActivityDate="2022-07-22T17:06:50.067" Title="How to identify genomic regions / peaks associated with enhancers (TF binding sites)? Is there a tool or a formal recipe?" Tags="|r|rna-seq|chip-seq|machine-learning|atac-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19431" PostTypeId="1" CreationDate="2022-07-20T12:05:10.287" Score="0" ViewCount="31" Body="&lt;p&gt;Almost every food product has nutritional values labelling. Is there a digital format, one that would be standardised for use in such a case?&lt;/p&gt;&#xA;&lt;p&gt;For example I want to create a QR code which is going to contain nutritional facts of a product but I want some other person/applications to be able to read this information without having to adjust their parsing code to get the information required. For this I would need to follow some standardised rules to write down this information, so that everyone can easily parse it, thus I would need some rules that describe: &amp;quot;This should be a .json format, it should have this and that written like this.&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Reference to what I mean about nutritional facts:&#xA;&lt;a href=&quot;https://www.fao.org/3/y2770e/y2770e06.htm#TopOfPage&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.fao.org/3/y2770e/y2770e06.htm#TopOfPage&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have tried to search for it and I couldn't find anything. Also I could definitely create my own format if none exists but if there is already one - I bet it's best to use it.&lt;/p&gt;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-javascript prettyprint-override&quot;&gt;&lt;code&gt;{&#xA;  &amp;quot;nutrients&amp;quot;: {&#xA;    &amp;quot;Protein&amp;quot;: 50,&#xA;    &amp;quot;Vitamin A&amp;quot;: 800,&#xA;    &amp;quot;Vitamin D&amp;quot;: 5,&#xA;    &amp;quot;Vitamin C&amp;quot;: 60,&#xA;    &amp;quot;Thiamin&amp;quot;: 1.4&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jpWgDm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jpWgDm.png&quot; alt=&quot;Nutrients QR Code Example&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you read plain text of this QR code - you will get the same &lt;code&gt;.json&lt;/code&gt; text.&lt;/p&gt;&#xA;&lt;p&gt;Consider this situation:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Similar QR code is printed on milk, fish, meat and other products in a supermarket.&lt;/li&gt;&#xA;&lt;li&gt;Users can use some kind of application to scan these QR codes.&lt;/li&gt;&#xA;&lt;li&gt;Each user might use a different application.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In this situation each application would read it and need to parse it to make use of it.&lt;/p&gt;&#xA;&lt;p&gt;If product companies that package milk, fish, meat, etc. each use a different format to store this information - then each application has to adjust for every possible format the companies provide on their products.&lt;/p&gt;&#xA;&lt;p&gt;If there would be a standardised format that every company used - then each application would only need to write code to parse it once or even use a predefined parser that everyone shares.&lt;/p&gt;&#xA;&lt;p&gt;I am looking for such a format and if the standard exists at all.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The reason I am writing this in Bioinformatics is because it's related to computer science and people related to biology are more likely to have encountered this in their experience.&lt;/p&gt;&#xA;" OwnerUserId="15804" LastEditorUserId="15804" LastEditDate="2022-07-22T08:03:34.257" LastActivityDate="2022-07-22T08:03:34.257" Title="Does standardised format to write nutrients of a product exist?" Tags="|file-formats|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19432" PostTypeId="2" ParentId="19415" CreationDate="2022-07-20T22:31:42.777" Score="0" Body="&lt;p&gt;The variation you are seeing are minor differences in estimates in the transmembrane domain, these are usually modelled by hydrophobicity and are accurate predictions. If you look at the secondary/tertiary structure you might see alpha helices which will 'burrow' into the membrane. The region in this particular question is a small domain however, but the position is 'about right'.&lt;/p&gt;&#xA;&lt;p&gt;Thus question is whether 1 amino acid at 5' and 3' site is inside or outside the membrane - I think this is overthinking. Edge-effects are common in any estimate and 1 amino isn't bad variation at all. To double check have a look at the Kyte-Doolittle index on the Expasy website (which is one of the first), and &lt;a href=&quot;https://services.healthtech.dtu.dk/service.php?TMHMM-2.0&quot; rel=&quot;nofollow noreferrer&quot;&gt;TMHMM&lt;/a&gt;. Finally, you need to check the crystal structure if available.&lt;/p&gt;&#xA;&lt;p&gt;What does puzzle me is the apparent absence of a 3' GPI anchor, but I dunno the biological system so ain't for me to comment.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-07-20T22:47:44.670" LastActivityDate="2022-07-20T22:47:44.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19433" PostTypeId="2" ParentId="19311" CreationDate="2022-07-21T05:03:44.030" Score="1" Body="&lt;p&gt;I am not sure that kraken/kraken2 does anything similar to this.&lt;/p&gt;&#xA;&lt;p&gt;Usually to get OTUs/ESVs I think that you want to use a tool that clusters your sequences into groupings that represent the same sequence, allowing for technical error.&lt;/p&gt;&#xA;&lt;p&gt;Examples might include &lt;a href=&quot;https://benjjneb.github.io/dada2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dada2&lt;/a&gt; or &lt;a href=&quot;https://academic.oup.com/gigascience/article/9/12/giaa135/6011256&quot; rel=&quot;nofollow noreferrer&quot;&gt;dadasnake&lt;/a&gt;, or &lt;a href=&quot;https://forum.qiime2.org/t/can-we-process-nanopore-data-using-qiime2/16156/4&quot; rel=&quot;nofollow noreferrer&quot;&gt;qiime&lt;/a&gt;. This is after the technical steps for adapter/trimming/denoising steps.&lt;/p&gt;&#xA;&lt;p&gt;kraken2 is more for classifying into known taxonomic categories based on a reference sequence, which is a related but distinct task.&lt;/p&gt;&#xA;&lt;p&gt;I don't think that I understand what the ESV file format is supposed to look like. &lt;a href=&quot;https://cmap.readthedocs.io/en/latest/catalog/datasets/ESV.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;This resource&lt;/a&gt; seems to have some info on processing data into ESVs, but it is not for nanopore. Possibly it is still useful for the final step of creating that file format. A &lt;a href=&quot;https://osf.io/g7xsj/&quot; rel=&quot;nofollow noreferrer&quot;&gt;linked repository&lt;/a&gt; has a ton of one-off scripts that may be helpful but are not very well documented. For example, here is a script that they use to generate TSV files which &lt;em&gt;may&lt;/em&gt; be the &amp;quot;ASV&amp;quot;/&amp;quot;ESV&amp;quot; format:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;source activate qiime2-2018.8&#xA;&#xA;for item in `ls */*table.qza`&#xA;    do&#xA;    name=`echo $item | cut -d\/ -f1`&#xA;    qiime tools export --input-path &lt;span class=&quot;math-container&quot;&gt;$item --output-path EXPORTS-GTDB-tax/$&lt;/span&gt;name&#xA;    mv EXPORTS-GTDB-tax/&lt;span class=&quot;math-container&quot;&gt;$name/feature-table.biom EXPORTS-GTDB-tax/$&lt;/span&gt;name.biom&#xA;    rmdir EXPORTS-GTDB-tax/$name&#xA;done&#xA;&#xA;qiime tools export --input-path 11_ANT-28-5-SPA_seqs_classified_against_GTDB_r86.1_807F-1050R_primer_sliced/classification.qza --output-path EXPORTS-GTDB-tax/&#xA;&#xA;sed -i '1c#OTUID    taxonomy    confidence' EXPORTS-GTDB-tax/taxonomy.tsv&#xA;&#xA;#Tutorial here: https://forum.qiime2.org/t/exporting-and-modifying-biom-tables-e-g-adding-taxonomy-annotations/3630&#xA;for item in `ls EXPORTS-GTDB-tax/*biom`&#xA;    do&#xA;    filestem=`basename $item .biom`&#xA;    biom add-metadata -i &lt;span class=&quot;math-container&quot;&gt;$item -o EXPORTS-GTDB-tax/$&lt;/span&gt;filestem.with-tax.biom --observation-metadata-fp EXPORTS-GTDB-tax/taxonomy.tsv --sc-separated taxonomy&#xA;done&#xA;&#xA;for item in `ls EXPORTS-GTDB-tax/*.with-tax.biom`&#xA;    do&#xA;    filestem=`basename $item .biom`&#xA;    biom convert -i &lt;span class=&quot;math-container&quot;&gt;$item -o EXPORTS-GTDB-tax/$&lt;/span&gt;filestem.tsv --to-tsv --header-key taxonomy&#xA;done&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-07-21T05:03:44.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19435" PostTypeId="1" AcceptedAnswerId="19448" CreationDate="2022-07-21T06:17:51.200" Score="1" ViewCount="161" Body="&lt;p&gt;I have a phylogenetic tree.&lt;/p&gt;&#xA;&lt;p&gt;Each branch ends have samples (s1, s2, ~ s16).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/dYIsT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dYIsT.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What I want to do is, I want to make PCA analysis plot for each sample.&lt;/p&gt;&#xA;&lt;p&gt;First I thought each sample has each lineage (expressed in image as (1,1,1,1)~(2,2,2,2)).&lt;/p&gt;&#xA;&lt;p&gt;So with that coordinates, I can do PCA analysis.&lt;/p&gt;&#xA;&lt;p&gt;But on the second thought, the difference between (1,1,1,1) and (1,1,1,2) is much less than that of  (1,1,1,1) and (2,1,1,1), but in the analysis, the difference between them will be same.&lt;/p&gt;&#xA;&lt;p&gt;I am not sure if this kind of approach is suitable for making a PCA plot with a phylogenetic tree.&lt;/p&gt;&#xA;&lt;p&gt;How can I make this work?&lt;/p&gt;&#xA;" OwnerUserId="15736" LastActivityDate="2022-07-23T13:54:36.427" Title="PCA analysis of samples in a phylogenetic tree" Tags="|phylogenetics|pca|tsne|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19436" PostTypeId="2" ParentId="19003" CreationDate="2022-07-21T07:38:49.637" Score="-1" Body="&lt;p&gt;use:&#xA;Spectrum b, Red_white_blue&lt;/p&gt;&#xA;" OwnerUserId="15807" LastActivityDate="2022-07-21T07:38:49.637" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19437" PostTypeId="1" AcceptedAnswerId="19439" CreationDate="2022-07-21T11:01:04.203" Score="2" ViewCount="190" Body="&lt;p&gt;My problem is when using Snakemake scatter-gather feature the documentation is basic and i modified my code according to mentioned in &lt;a href=&quot;https://stackoverflow.com/q/64898553/19589226&quot;&gt;this link&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule fastq_fasta:&#xA;    input:rules.trimmomatic.output.out_file&#xA;    output:&amp;quot;data/trimmed/{sample}.fasta&amp;quot;&#xA;    shell:&amp;quot;sed -n '1~4s/^@/&amp;gt;/p;2~4p' {input} &amp;gt; {output}&amp;quot;&#xA;&#xA;rule split:&#xA;    input:&#xA;        &amp;quot;data/trimmed/{sample}.fasta&amp;quot;&#xA;    params:&#xA;        scatter_count=config[&amp;quot;scatter_count&amp;quot;],&#xA;        scatter_item = lambda wildcards: wildcards.scatteritem&#xA;    output:&#xA;        temp(scatter.split(&amp;quot;data/trimmed/{{sample}}_{scatteritem}.fasta&amp;quot;))&#xA;    script:&#xA;        &amp;quot;scripts/split_files.py&amp;quot;&#xA;        &#xA;rule process:&#xA;    input:&amp;quot;data/trimmed/{sample}_{scatteritem}.fasta&amp;quot;&#xA;    output:&amp;quot;data/processed/{sample}_{scatteritem}.csv&amp;quot;&#xA;    script:&#xA;        &amp;quot;scripts/process.py&amp;quot;&#xA;&#xA;rule gather:&#xA;    input:&#xA;        gather.split(&amp;quot;data/processed/{{sample}}_{scatteritem}.csv&amp;quot;)&#xA;    output:&#xA;        &amp;quot;data/processed/{sample}.csv&amp;quot;&#xA;    shell:&#xA;        &amp;quot;cat {input} &amp;gt; {output}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I added wildcard option but, i got &lt;code&gt;AmbiguousRuleException: Rules fastq_to_fasta(which is previous rule) and split are ambiguous for the file data/trimmed/Ornek_411-of-81-of-81-of-81-of-81-of-81-of-81-of-81-of-81-of-8.fasta&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I tried lots of things but either rules are not calling or take AmbiguousRuleException. What am i missing, can someone help?&lt;/p&gt;&#xA;" OwnerUserId="15809" LastEditorUserId="92" LastEditDate="2022-07-23T06:31:15.580" LastActivityDate="2022-07-23T06:31:15.580" Title="Snakemake scatter-gather with wildcard AmbiguousRuleException" Tags="|python|snakemake|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19438" PostTypeId="1" CreationDate="2022-07-21T15:01:01.897" Score="1" ViewCount="36" Body="&lt;p&gt;I have around 4000 orthologous proteins for each of 5 species, I'd like to align them and to concatenate alignments. Which tools can I use? And how?&lt;/p&gt;&#xA;&lt;p&gt;Marco&lt;/p&gt;&#xA;" OwnerUserId="15596" LastActivityDate="2022-07-21T16:21:10.373" Title="Multifasta protein alignment" Tags="|proteins|sequence-alignment|multi-fasta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19439" PostTypeId="2" ParentId="19437" CreationDate="2022-07-21T15:02:54.840" Score="1" Body="&lt;p&gt;It's not a problem with the scatter-gather feature per-se; Snakemake is just confusing itself with what text should match what wildcards with these filenames.  You could fix this by defining a &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#wildcards&quot; rel=&quot;nofollow noreferrer&quot;&gt;wildcard constraint&lt;/a&gt; for your samples, and/or changing your filename patterns to clarify what should match what.  The &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#defining-scatter-gather-processes&quot; rel=&quot;nofollow noreferrer&quot;&gt;docs on scatter-gather&lt;/a&gt; show an example with different directory names and file suffixes that prevent this from happening.  Probably the simplest fix would be to follow that idea and use different directory names for the split parts, like &lt;code&gt;trimmed-split&lt;/code&gt; and &lt;code&gt;processed-split&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;A hint of what's happening is the repeating &lt;code&gt;...-of-81-...&lt;/code&gt; thing, and the mention of ambiguity between rules.  (See also the &lt;code&gt;--debug-dag&lt;/code&gt; option.)  If you have a sample named Ornek_411 and you want to make data/processed/Ornek_411.csv, Snakemake will look for data/processed/Ornek_411_1-of-N.csv and so on, and try to figure out how to make those.  But a filename like that could be the output of either the process rule (&lt;code&gt;sample=Ornek_411&lt;/code&gt; and &lt;code&gt;scatteritem=1-of-N&lt;/code&gt;) or the gather rule itself again (&lt;code&gt;sample=Ornek_411_1-of-N&lt;/code&gt;).  Obviously you want the former and not the latter, but it doesn't know that automatically.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2022-07-21T15:02:54.840" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19440" PostTypeId="2" ParentId="19438" CreationDate="2022-07-21T16:21:10.373" Score="1" Body="&lt;p&gt;4000 isn't much. You'll need to start with an aligner. For aligners the common options are in no particular order:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Clustal Omega&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://mafft.cbrc.jp/alignment/software/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MAFFT&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;FAMSA&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.drive5.com/muscle/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Muscle v5&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The current one that 'outperforms' the others strictly on a combination of two published benchmarks comprising 'accuracy' and 'speed' is MAFFT. FAMSA isn't relevant here unless you've a really small computer, like an old laptop, it excels at speed.&lt;/p&gt;&#xA;&lt;p&gt;However, the situation could have changed recently with the  &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2021.06.20.449169v1.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioRXIV&lt;/a&gt; release of Muscle v5. Please note, I have not used it. Here's the Abstract:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;On Balifam, ensembles generated by MUSCLE are shown to align an&#xA;average of 59% of columns correctly, 13% better than Clustal-omega&#xA;(52% correct) and 26% better than MAFFT (47% correct). The ensemble&#xA;bootstrap is applied to a previously published tree of RNA viruses ...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;On this benchmark they are suggest Clustal Omega is better than MAFFT. Its not been the run of play that is certain MAFFT rose to dominance in recent history. Des Higgins Clustal W and X was historically the aligner of choice by a long shot, then Muscle, then MAFFT ... now its context specific.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Muscle v5&lt;/em&gt; Its difficult to really get to grips with a BioRXIV ms as an algorithm unless you're a specialist in aligner construction. Please note, the 'RNA viruses' is complicated. As an algorithm this author has kudos and has extended that kudos beyond Muscle. The BioRXIV would have been subject to peer-review, but I am not aware Muscle v5 has been accepted for publication.&lt;/p&gt;&#xA;&lt;p&gt;Historically, Muscle made early advances in accuracy which were duplicated by Clustal Omega at a much later date (2017?).&lt;/p&gt;&#xA;&lt;p&gt;Essentially, here your choice is between MAFFT and Muscle v5 for this data set (context specific).&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;concatenate&lt;/code&gt; this is less clear but it appears you are performing 4000 alignments with 5 taxa in each? You'll need a data pipeline.&lt;/p&gt;&#xA;&lt;p&gt;Trimming and concatenation are easily performed in BioPython using &lt;code&gt;BioAlignIO&lt;/code&gt; but you would need coding because that can't be done manually. Computationally, &lt;code&gt;BioAlignIO&lt;/code&gt; objects are annoyingly expensive - they are &lt;code&gt;SeqIO&lt;/code&gt; objectives inside a &lt;code&gt;MultipleAlignment&lt;/code&gt; object. However in your case it matters not a jot and would the recommended route all things considered (unless your coding expertise is Java).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-07-21T16:21:10.373" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19441" PostTypeId="1" CreationDate="2022-07-21T19:31:38.437" Score="2" ViewCount="73" Body="&lt;p&gt;I have a complete genome sequence of a Sars-CoV-2 variant, but I am only interested in the Spike sequence?&lt;/p&gt;&#xA;&lt;p&gt;I should note that I am not a biologist (nor bioinformatics).&lt;/p&gt;&#xA;&lt;p&gt;The complete genome looks like the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;AAACAAACCAACCAACTTTCGATCTCTTGTAGATCTGTTCTCTAAACGAACTTTAAAATCTGTGTGGCTG&#xA;TCACTCGGCTGCATGCTTAGTGCACTCACGCAGTATAATTAATAACTAATTACTGTCGTTGACAGGACAC&#xA;GAGTAACTCGTCT...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Its length is 30230.&lt;/p&gt;&#xA;&lt;p&gt;My question is, is there a way to know where exactly the spike sequence lies among this complete genome sequence, and if possible, are there already existing tools to extract it programmatically (in Python).&#xA;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="15813" LastEditorUserId="3848" LastEditDate="2022-07-21T19:40:27.310" LastActivityDate="2023-12-14T01:07:05.827" Title="Is there a way to extract spike sequence from a Sars-CoV-2 complete genome (preferably in Python)?" Tags="|genome|covid-19|sars-cov-2|python|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19442" PostTypeId="2" ParentId="19441" CreationDate="2022-07-21T20:05:53.267" Score="0" Body="&lt;p&gt;Spike Wuhan is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/MN996527.1?from=21530&amp;amp;to=25351&amp;amp;report=fasta&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Using Python then use the BioPython &lt;code&gt;Bio.pairwise2&lt;/code&gt; program &lt;a href=&quot;https://biopython.org/docs/1.75/api/Bio.pairwise2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Yours sequence is the reference, the shorter Wuhan query is the query.&#xA;The genomic positions &lt;code&gt;position1&lt;/code&gt; and &lt;code&gt;position2&lt;/code&gt; will then be defined via the output of &lt;code&gt;pairwise2&lt;/code&gt;, then cut it by using the genomic position -1 as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;spike = completegenome[int(position1)-1:int(position2)-1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;-1&lt;/code&gt; is needed because lists start at 0 rather than 1.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-07-21T22:14:01.120" LastActivityDate="2022-07-21T22:14:01.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19443" PostTypeId="2" ParentId="19003" CreationDate="2022-07-21T22:36:25.177" Score="2" Body="&lt;p&gt;&lt;em&gt;Answer from @matteo-ferla, converted from &lt;a href=&quot;https://bioinformatics.stackexchange.com/help/privileges/comment&quot;&gt;comment&lt;/a&gt;:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;They are stored as b-factors. This is well described in the &lt;a href=&quot;https://pymolwiki.org/index.php/Selection_Algebra&quot; rel=&quot;nofollow noreferrer&quot;&gt;PyMol wiki&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;[please edit to improve this answer, if possible]&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-07-21T22:36:25.177" CommentCount="0" CommunityOwnedDate="2022-07-21T22:36:25.177" ContentLicense="CC BY-SA 4.0" />
  <row Id="19444" PostTypeId="1" CreationDate="2022-07-22T07:13:37.877" Score="1" ViewCount="275" Body="&lt;p&gt;How can I do to extract a single frame as a pdb file from a trajectory traj.xtc file, by using MDAnalysis?&lt;/p&gt;&#xA;&lt;p&gt;I've got a universe&#xA;u=mda.Universe('confout.gro','traj.xtc')&lt;/p&gt;&#xA;&lt;p&gt;Then I selected a residue by doing select_atoms, and I would like to extract a single frame as a pdb file&lt;/p&gt;&#xA;" OwnerUserId="15814" LastActivityDate="2022-12-28T18:58:07.987" Title="Extract single frame with MDAnalysis" Tags="|python|pdb|gromacs|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19445" PostTypeId="2" ParentId="19430" CreationDate="2022-07-22T08:52:47.517" Score="2" Body="&lt;p&gt;If you have reads from chip-seq or atac you can use tools like &lt;a href=&quot;http://homer.ucsd.edu/homer/ngs/peaks.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;homer&lt;/a&gt; or &lt;a href=&quot;https://hbctraining.github.io/Intro-to-ChIPseq/lessons/05_peak_calling_macs.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;macs&lt;/a&gt; to identify peaks in your data. Once you have the peaks, these programs allow you to do motif discovery to annotate your peaks with e.g. what transcription factor binding sites they contain.&lt;/p&gt;&#xA;" OwnerUserId="10318" LastActivityDate="2022-07-22T08:52:47.517" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19446" PostTypeId="1" CreationDate="2022-07-22T19:10:21.077" Score="2" ViewCount="465" Body="&lt;p&gt;Is there a MATLAB, or else Python (not R) tool for visualizing sequence Logos from probability weight matrices (PWMs)?&lt;/p&gt;&#xA;&lt;p&gt;This seems like a basic and simple tool to implement, yet just above the threshold in the space of tedious to write my own from scratch - I was surprised I couldn't find a basic function like a&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;visualizePWM(pwm_matrix) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;that would take a pwm_matrix that is a (4 by m) matrix and spit out the visualization of an m-length sequence motif/logo like:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Uf9QO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Uf9QO.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I see all kinds of over-complicated bloated tools that I thought I ask in case I am missing something.&lt;/p&gt;&#xA;" OwnerUserId="15741" LastEditorUserId="15741" LastEditDate="2022-07-24T02:46:56.233" LastActivityDate="2022-07-24T23:21:58.127" Title="PWM (4xM matrices) to Sequence Logo Visualizations in MATLAB or Python" Tags="|rna-seq|python|visualization|motifs|matlab|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19447" PostTypeId="1" AcceptedAnswerId="19450" CreationDate="2022-07-22T23:17:15.407" Score="2" ViewCount="831" Body="&lt;p&gt;I have been using the &lt;code&gt;-q&lt;/code&gt; option of &lt;code&gt;samtools view&lt;/code&gt; to filter out reads whose mapping quality (MAPQ) scores are below a given threshold when mapping reads to a reference assembly with either &lt;code&gt;bwa mem&lt;/code&gt; or &lt;code&gt;minimap2&lt;/code&gt;. However, using &lt;code&gt;samtools idxstats&lt;/code&gt; to count total mapped reads and unmapped reads indicates that these reads with lower MAPQ scores are actually being discarded altogether, since the number of unmapped reads reported (last row) is always zero when the BAM file is created with a &lt;code&gt;-q&lt;/code&gt; argument, even though fewer reads are being mapped to the assembly.&lt;/p&gt;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 -ax map-ont assembly.fasta reads.fastq | samtools view -b | samtools sort --reference assembly.fasta &amp;gt; nanopore_mapped_no_q_thresh.bam&#xA;samtools idxstats nanopore_mapped_no_q_thresh.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;gives&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;contig_1      4751250 9897    0&#xA;contig_2      37697   528     0&#xA;contig_3      9131    2366    0&#xA;contig_4      98997   701     0&#xA;contig_5      4539    365     0&#xA;contig_6      3933    3164    0&#xA;*       0       0       142&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;while&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 -ax map-ont assembly.fasta reads.fastq | samtools view -b -q 2 | samtools sort --reference assembly.fasta &amp;gt; nanopore_mapped_q2.bam&#xA;samtools idxstats nanopore_mapped_q2.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;gives&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;contig_1      4751250 9716    0&#xA;contig_2      37697   528     0&#xA;contig_3      9131    1415    0&#xA;contig_4      98997   700     0&#xA;contig_5      4539    365     0&#xA;contig_6      3933    3161    0&#xA;*       0       0       0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there an alternative way to filter mapped reads by their MAPQ scores so that the filtered reads are still retained in the BAM file?&lt;/p&gt;&#xA;" OwnerUserId="15819" LastActivityDate="2022-07-23T20:47:04.253" Title="how to retain reads with low mapping quality (MAPQ) scores when using samtools view -q" Tags="|sam|samtools|genomics|read-mapping|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19448" PostTypeId="2" ParentId="19435" CreationDate="2022-07-23T10:15:39.597" Score="1" Body="&lt;p&gt;There appear two basic issues here:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The PCA would represent arbitrary lineage designation not a genetic quantification.&lt;/li&gt;&#xA;&lt;li&gt;The PCA will weight the first, second ... fourth position equally, i.e. it is not hierarchical. This is very important for the analysis.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;em&gt;Summary&lt;/em&gt; The hierarchical -&amp;gt; unweighted PCA will cause analytical problems and should not be used, given it is ultimately arbitrary. If there was a tree specialist reviewing your work it would not be accepted.&lt;/p&gt;&#xA;&lt;p&gt;The reason is that position one in the hierarchical cluster defines the entire data set - it cuts it in half, whilst position 4 defines a small a very small subtree. PCA will not see the hierarchy it will see position 1 as of equal worth to position 4. Therefore PCA will start associating arbitrary designated position 4 e.g. branch 2, with each other. Thus 1,1,1,2 is a long way from 2,1,1,2 but PCA will miss that it will think the shared '2' means their closer than they really are and find it of equal distance to e.g. 1,2,1,2 - which or course it isn't.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Fixing it&lt;/em&gt; You ask how to fix it&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You can go back to the original data and use that to create a PCA based on SNPs ... this has been done for other genetic data types, but cannot done reinterpreting a tree. Using this approach there will be a problem with back mutation, but its a parsimony analysis and you would simply state your assumption - ultimately the influence of back mutation concerns the genetic distance between all taxa in the analysis and if thats not big, the error will be small.&lt;/li&gt;&#xA;&lt;li&gt;Alternatively if you had a phylogram you could construct a genetic matrix and use genetic distance instead. Thats a separate question because its quite complicated.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;tSNE is often used to refine a multivariate analysis and I think thats what you getting at, PCA followed by tSNE is valid. A tree isn't any regular multivariate analysis, its a tightly defined approximation within clear mathematical boundaries measuring/reconstructing evolutionary divergence. A tree specialist will see this, whilst a general multivariate stats specialist will miss it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-07-23T13:54:36.427" LastActivityDate="2022-07-23T13:54:36.427" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19450" PostTypeId="2" ParentId="19447" CreationDate="2022-07-23T20:47:04.253" Score="2" Body="&lt;p&gt;Looking at the &lt;a href=&quot;http://www.htslib.org/doc/samtools-view.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools view docs&lt;/a&gt;, I believe that you need to also set the &lt;code&gt;-p&lt;/code&gt; flag if you want to retain &amp;quot;filtered&amp;quot; reads in the file and simply have them be unmapped:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The -p, option sets the UNMAP flag on filtered alignments then writes them to the output file.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-07-23T20:47:04.253" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19451" PostTypeId="1" CreationDate="2022-07-24T03:49:51.950" Score="2" ViewCount="53" Body="&lt;p&gt;When we use &lt;a href=&quot;https://antismash.secondarymetabolites.org/#!/start&quot; rel=&quot;nofollow noreferrer&quot;&gt;antismash&lt;/a&gt; command with the genome sequence file as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;antismash GCA_002095535.1_ASM209553v1_genomic.gbff&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;it screens the genome for all the possible biosynthetic gene clusters (BGCs).&lt;/p&gt;&#xA;&lt;p&gt;Is there a way the tool can be directed to screen for only one specific type of BGC (for example, terpene)?&lt;/p&gt;&#xA;" OwnerUserId="15823" LastEditorUserId="3848" LastEditDate="2022-07-24T04:03:44.577" LastActivityDate="2022-07-24T04:03:44.577" Title="Is there a way antiSMASH can be used to screen a large number of genomes just for one type of biosynthetic gene cluster?" Tags="|genome|gene|bash|sequence|data-mining|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19452" PostTypeId="2" ParentId="19446" CreationDate="2022-07-24T07:11:50.990" Score="1" Body="&lt;p&gt;&lt;em&gt;answer from @M__, converted from &lt;a href=&quot;https://bioinformatics.stackexchange.com/help/privileges/comment&quot;&gt;comments&lt;/a&gt;:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;This has been answered on stackoverflow using matplotlib for seq logo plot:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/42615527/sequence-logos-in-matplotlib-aligning-xticks&quot;&gt;https://stackoverflow.com/questions/42615527/sequence-logos-in-matplotlib-aligning-xticks&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;IMO its only useful for presentation, pretty pictures, thus there's no need for someone to write a library (which is what you're asking). If I'm using a mutation matrix (which I do), I want the raw numbers and significance values I don't want pretty graphs getting in the way.&lt;/p&gt;&#xA;&lt;p&gt;Conceptually you would move from 2D to 1D, without subplots (which is doable), i.e. its a matrix. My view on code availability is not universal. I've had requests for separate CLIs (useful for someone clearly) ... which just never happened.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-07-24T23:21:58.127" LastActivityDate="2022-07-24T23:21:58.127" CommentCount="1" CommunityOwnedDate="2022-07-24T07:11:50.990" ContentLicense="CC BY-SA 4.0" />
  <row Id="19453" PostTypeId="2" ParentId="19446" CreationDate="2022-07-24T22:17:23.680" Score="1" Body="&lt;p&gt;After much searching and trial and error, located possibly the optimal tool for my question:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;seqlogo&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/betteridiot/seqlogo&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/betteridiot/seqlogo&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Surprisingly not only seqlogo is an elegant and minimal solution in Python (and R) I have discovered it has a relative implementation in MATLAB! It installed and worked the first try - beautiful output with all the output options I needed - rock solid.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Python&lt;/strong&gt; one liner (not counting the example pwm input generation)&lt;/p&gt;&#xA;&lt;p&gt;random_ppm = np.random.dirichlet(np.ones(4), size=16)&lt;/p&gt;&#xA;&lt;p&gt;ppm = seqlogo.Ppm(random_ppm)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;seqlogo.seqlogo(ppm  ,format='png',filename='out.png')&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Produces the following ('out.png')&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hrIZJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hrIZJ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And here is &lt;strong&gt;MATLAB&lt;/strong&gt; Port:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.mathworks.com/help/bioinfo/ref/seqlogo.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.mathworks.com/help/bioinfo/ref/seqlogo.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Weeh9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Weeh9.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15741" LastEditorUserId="15741" LastEditDate="2022-07-24T23:10:17.103" LastActivityDate="2022-07-24T23:10:17.103" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19454" PostTypeId="1" AcceptedAnswerId="19455" CreationDate="2022-07-25T09:19:40.473" Score="2" ViewCount="692" Body="&lt;pre&gt;&lt;code&gt;file_name = &amp;quot;consen1.txt&amp;quot;&#xA;    with open(file_name) as infile:&#xA;        new_dict = {}&#xA;        for line in infile:&#xA;            if line.startswith(&amp;quot;&amp;gt;&amp;quot;):&#xA;                key = line.strip('\n')&#xA;            else:&#xA;                line = ''.join(x.strip('\n') for x in infile)&#xA;                new_dict[key] = line&#xA;    print(new_dict)  &#xA;    for key,values in new_dict.items():&#xA;        gc = values.count('G') + values.count('C')&#xA;        content = (gc/len(values)) *100&#xA;        new_dict[key] = content&#xA;    print(new_dict)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am reading from a text file, and everything works fine but for some reason it is skipping the line of sequence from the text file. Here's how the text file looks like,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Rosalind_4478&#xA;GCTACATCAACGCCGTGCATGGCGTGTAAAGACCCGCATTTTACAGGCAGTGTATACGAC&#xA;CGGTAACACGATTTAGCATTTACTGAGCCCCTGCGTTATTGGAATATGAGGGAACGTTGC&#xA;TTATGTGCAAGACATTCCACCTCATCCATAGATACTCTCGCGGCAGATTCTGTGACGGAT&#xA;CGGATCGCTTCAAAGCACGGTACGGTATTAGACGAATGGGTTAAAACTCTCTACAAGCAC&#xA;TCGTACAACTTATGCACACCAACTTCAGCGTATAACATGAGAGCCCGGAAAGGCATATTC&#xA;CTTATCTTAAGGTAGGGAATACGAAAACCCGCGTCGTCCGCCCACCTCGACGGTACGGAA&#xA;TGTTTGAGGGGCATCCGGGTCTGGCAGGAACTGTGTGGACTGGTGCGTCATCAGCTGGGC&#xA;TGATTGCGCGAGCCAGAATCGCATGCGCTTCGAATTTTAGGAGAGCGTGCCGACTCACGT&#xA;ATATCATTGACATGGTGAGGGTAGCAGAATCTACATTGCAGCCTATGACGCTACACGGCC&#xA;GTATAATCAGTGTTAGTGCAGTATTGTGGCATTTCCGTTTCGCATGGTTAACTGAGGCCA&#xA;AGGTTTGTGCCGGGGGACCGGATGGCTACCTGGAAACTAGGATCTGGCCGTAGTGGTTCT&#xA;CGCGAACCCTAGTTTCAAGCAAACGCCAGCCGGAGACCGAAGTTAAACGCATCACTACTA&#xA;CTCACGAGTCTTGGACCAAAACAGAAAGTCGGATGCACCATGTACGAGCTGACGTTGTCT&#xA;TTATGTCCAGTTGGATCTGGACAGCACATCCGAGATTTTGAATTAAAACTTCTTTTTGTG&#xA;TGATGCTTGGAACACCCGCCATACAACAAATAATAGCGCCTTAAACAGCCTACGGCACAG&#xA;TA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Expected output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; {'&amp;gt;Rosalind_4478': 'GCTACATCAACGCCGTGCATGGCGTGTAAAGACCCGCATTTTACAGGCAGTGTATACGACCGGTAACACGATTTAGCATTTACTGAGCCCCTGCGTTATTGGAATATGAGGGAACGTTGCTTATGTGCAAGACATTCCACCTCATCCATAGATACTCTCGCGGCAGATTCTGTGACGGATCGGATCGCTTCAAAGCACGGTACGGTATTAGACGAATGGGTTAAAACTCTCTACAAGCACTCGTACAACTTATGCACACCAACTTCAGCGTATAACATGAGAGCCCGGAAAGGCATATTCCTTATCTTAAGGTAGGGAATACGAAAACCCGCGTCGTCCGCCCACCTCGACGGTACGGAATGTTTGAGGGGCATCCGGGTCTGGCAGGAACTGTGTGGACTGGTGCGTCATCAGCTGGGCTGATTGCGCGAGCCAGAATCGCATGCGCTTCGAATTTTAGGAGAGCGTGCCGACTCACGTATATCATTGACATGGTGAGGGTAGCAGAATCTACATTGCAGCCTATGACGCTACACGGCCGTATAATCAGTGTTAGTGCAGTATTGTGGCATTTCCGTTTCGCATGGTTAACTGAGGCCAAGGTTTGTGCCGGGGGACCGGATGGCTACCTGGAAACTAGGATCTGGCCGTAGTGGTTCTCGCGAACCCTAGTTTCAAGCAAACGCCAGCCGGAGACCGAAGTTAAACGCATCACTACTACTCACGAGTCTTGGACCAAAACAGAAAGTCGGATGCACCATGTACGAGCTGACGTTGTCTTTATGTCCAGTTGGATCTGGACAGCACATCCGAGATTTTGAATTAAAACTTCTTTTTGTGTGATGCTTGGAACACCCGCCATACAACAAATAATAGCGCCTTAAACAGCCTACGGCACAGTA'}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Real Output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{'&amp;gt;Rosalind_4478': 'CGGTAACACGATTTAGCATTTACTGAGCCCCTGCGTTATTGGAATATGAGGGAACGTTGCTTATGTGCAAGACATTCCACCTCATCCATAGATACTCTCGCGGCAGATTCTGTGACGGATCGGATCGCTTCAAAGCACGGTACGGTATTAGACGAATGGGTTAAAACTCTCTACAAGCACTCGTACAACTTATGCACACCAACTTCAGCGTATAACATGAGAGCCCGGAAAGGCATATTCCTTATCTTAAGGTAGGGAATACGAAAACCCGCGTCGTCCGCCCACCTCGACGGTACGGAATGTTTGAGGGGCATCCGGGTCTGGCAGGAACTGTGTGGACTGGTGCGTCATCAGCTGGGCTGATTGCGCGAGCCAGAATCGCATGCGCTTCGAATTTTAGGAGAGCGTGCCGACTCACGTATATCATTGACATGGTGAGGGTAGCAGAATCTACATTGCAGCCTATGACGCTACACGGCCGTATAATCAGTGTTAGTGCAGTATTGTGGCATTTCCGTTTCGCATGGTTAACTGAGGCCAAGGTTTGTGCCGGGGGACCGGATGGCTACCTGGAAACTAGGATCTGGCCGTAGTGGTTCTCGCGAACCCTAGTTTCAAGCAAACGCCAGCCGGAGACCGAAGTTAAACGCATCACTACTACTCACGAGTCTTGGACCAAAACAGAAAGTCGGATGCACCATGTACGAGCTGACGTTGTCTTTATGTCCAGTTGGATCTGGACAGCACATCCGAGATTTTGAATTAAAACTTCTTTTTGTGTGATGCTTGGAACACCCGCCATACAACAAATAATAGCGCCTTAAACAGCCTACGGCACAGTA'}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I do realize that I can do this kind of computation easily with modules like biopython, But I wanted to try doing this from scratch.&#xA;Thanks in advance for the help.&lt;/p&gt;&#xA;" OwnerUserId="9772" LastActivityDate="2022-07-28T15:45:05.587" Title="Reading lines from a fasta format file" Tags="|python|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19455" PostTypeId="2" ParentId="19454" CreationDate="2022-07-25T09:44:16.570" Score="1" Body="&lt;p&gt;Your approach is a bit risky code ... Anyway not tested but I think this will work:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with open(file_name) as infile:&#xA;    new_dict = {}&#xA;    for l in infile:&#xA;        line = l.rstrip('\n')&#xA;        if line.startswith(&amp;quot;&amp;gt;&amp;quot;):&#xA;            k = line.lstrip('&amp;gt;')&#xA;            new_dict[k] = [] # must initiate&#xA;            # new_dict = {k:[]} is possible alternative initiation &#xA;        else:&#xA;            new_dict[key].append(line)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I not sure without testing which dictionary initiation will work. People do use &lt;code&gt;defaultdict&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from collections import defaultdict&#xA;new = defaultdict(def_value)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This circumvents the initiation issue to a large extent (if the dictionary key ain't correctly initiated it will throw a value error, or some sort of bug). However it isn't actually needed and I find it unnecessary.&lt;/p&gt;&#xA;&lt;p&gt;What I am certain about is the following should be avoided because your code depends on the exact line by line order between the fasta header and the code you are associating it with. You likely break that association with the code below,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;line = ''.join(x.strip('\n') for x in infile) # you are calling the entire infile not the line&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;BTW&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from collections import Count ()&#xA;new = Count()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is a very good approach for the next part of your script, i.e. counting %GC&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-07-25T09:50:09.260" LastActivityDate="2022-07-25T09:50:09.260" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19456" PostTypeId="1" CreationDate="2022-07-25T10:40:29.977" Score="2" ViewCount="440" Body="&lt;p&gt;I am trying to adjust my RNA data using &lt;code&gt;ComBat-Seq&lt;/code&gt;  (from &lt;code&gt;sva&lt;/code&gt; R package) since I realised that there are 3 batches that I need to adjust for:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Place (2 levels: place 1, place 2)&lt;/li&gt;&#xA;&lt;li&gt;Library Preparation Date (16 levels - different dates)&lt;/li&gt;&#xA;&lt;li&gt;Type of tube (2 levels: A, B)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I have 960 samples and around 62000 genes.&lt;/p&gt;&#xA;&lt;p&gt;In my biological matrix, I have: Age, Sex, Group (cases,controls..) and WBC counts.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;biol_mat = model.matrix(~Age + as.factor(Sex) + as.factor(Group) + LYMPH + MONO + NEUT, data=phenotype)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In the tutorial of &lt;code&gt;Combat-Seq&lt;/code&gt; appears how to adjust by 1 variable but it doesn't tell you how to adjust by more than 1.&lt;/p&gt;&#xA;&lt;p&gt;I have seen a lot of posts using &lt;code&gt;combat&lt;/code&gt; that the only way is to adjust by 1 variable and then, with those results, adjust again by the 2nd variable and so on.&lt;/p&gt;&#xA;&lt;p&gt;That would be:&lt;/p&gt;&#xA;&lt;h3&gt;Adjust by library prep.&lt;/h3&gt;&#xA;&lt;pre&gt;&lt;code&gt;raw.cts_adjustedLibPrep &amp;lt;- ComBat_seq(counts = raw_cts_matrix, batch=batch_libraryprep, group=NULL, covar_mod = biol_mat)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3&gt;Adjust by library prep + type of tube.&lt;/h3&gt;&#xA;&lt;pre&gt;&lt;code&gt;raw.cts_adjusted_LibPrep_TypeTube &amp;lt;- ComBat_seq(counts = raw.cts_adjustedLibPrep, batch=batch_type_tube, group=NULL, covar_mod = biol_mat)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3&gt;Adjust by library prep + type of tube + place&lt;/h3&gt;&#xA;&lt;pre&gt;&lt;code&gt; raw.cts_adjusted_LibPrep_TypeTube_Place &amp;lt;- ComBat_seq(counts = raw.cts_adjusted_LibPrep_TypeTube, batch=batch_place, group=NULL, covar_mod = biol_mat)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For the first adjustment (library prep) it takes around 15min, but for the second... it has been running for more than 2 days.. I stopped it and launch it again, changing the adjustment but I am not sure if it will work..&lt;/p&gt;&#xA;&lt;p&gt;Does anybody have an idea about how to fix the problem?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;&lt;p&gt;Question already asked &lt;a href=&quot;https://www.biostars.org/p/9532142/#9532171&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15844" LastEditorUserId="15844" LastEditDate="2022-07-25T13:50:44.057" LastActivityDate="2022-07-25T13:50:44.057" Title="How to adjust by multiple variables using ComBat-Seq?" Tags="|r|rna-seq|bioconductor|batch-effects|combat|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19457" PostTypeId="2" ParentId="19396" CreationDate="2022-07-25T10:49:30.223" Score="2" Body="&lt;p&gt;Download dbsnp VCF file from &lt;a href=&quot;https://ftp.ncbi.nih.gov/snp/organisms/human_9606/VCF/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nih.gov/snp/organisms/human_9606/VCF/&lt;/a&gt;. This contains rs IDs and chromosome info (among other info). Can match your rs IDs to the rs IDs in dbsnp VCF, and extract chromosome from there.&lt;/p&gt;&#xA;&lt;p&gt;See &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/variation/docs/human_variation_vcf/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/variation/docs/human_variation_vcf/&lt;/a&gt; for more information on the dbsnp files.&lt;/p&gt;&#xA;" OwnerUserId="10318" LastActivityDate="2022-07-25T10:49:30.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19458" PostTypeId="1" CreationDate="2022-07-25T16:35:57.090" Score="2" ViewCount="258" Body="&lt;p&gt;I'm running a DEG analysis with Deseq2 by specifing the following design that includes 12 samples from 6 pts in 4 different conditions:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;coldata &amp;lt;- data.frame(&amp;quot;sample&amp;quot;=c(&#xA;  &amp;quot;T58&amp;quot;, &amp;quot;T86&amp;quot;, &amp;quot;T87&amp;quot;,&#xA;  &amp;quot;D58&amp;quot;, &amp;quot;D86&amp;quot;, &amp;quot;D87&amp;quot;,&#xA;  &amp;quot;C62&amp;quot;, &amp;quot;C80&amp;quot;, &amp;quot;C84&amp;quot;,&#xA;  &amp;quot;C62&amp;quot;, &amp;quot;C80&amp;quot;, &amp;quot;C84&amp;quot;&#xA;))&#xA;coldata$Pts &amp;lt;- c(&#xA;  &amp;quot;m58&amp;quot;, &amp;quot;m86&amp;quot;, &amp;quot;m87&amp;quot;,&#xA;  &amp;quot;m58&amp;quot;, &amp;quot;m86&amp;quot;, &amp;quot;m87&amp;quot;,&#xA;  &amp;quot;c62&amp;quot;, &amp;quot;c80&amp;quot;, &amp;quot;c84&amp;quot;,&#xA;  &amp;quot;c62&amp;quot;, &amp;quot;c80&amp;quot;, &amp;quot;c84&amp;quot;&#xA;)&#xA;coldata$class &amp;lt;- c(&#xA;  &amp;quot;Treated&amp;quot;, &amp;quot;Treated&amp;quot;, &amp;quot;Treated&amp;quot;,&#xA;  &amp;quot;Distal&amp;quot;, &amp;quot;Distal&amp;quot;, &amp;quot;Distal&amp;quot;,&#xA;  &amp;quot;CLEFT&amp;quot;, &amp;quot;CLEFT&amp;quot;, &amp;quot;CLEFT&amp;quot;,&#xA;  &amp;quot;CRIGHT&amp;quot;, &amp;quot;CRIGHT&amp;quot;, &amp;quot;CRIGHT&amp;quot;&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would model the differences among the conditions by considering that the same individuals are nested in different conditions&lt;/p&gt;&#xA;&lt;p&gt;I tried the following code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ddsMat &amp;lt;- DESeqDataSetFromMatrix(&#xA;  countData = as.matrix(matrix),&#xA;  colData = coldata,&#xA;  design = ~ class+Pts&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but this produces the famous error&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;the model matrix is not full rank...&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If I reduce the previous dataset to&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;coldata2 &amp;lt;- as.data.frame(c(&#xA;  &amp;quot;T58&amp;quot;, &amp;quot;T86&amp;quot;, &amp;quot;T87&amp;quot;,&#xA;  &amp;quot;D58&amp;quot;, &amp;quot;D86&amp;quot;, &amp;quot;D87&amp;quot;&#xA;))&#xA;coldata2$Pts &amp;lt;- c(&#xA;  &amp;quot;m58&amp;quot;, &amp;quot;m86&amp;quot;, &amp;quot;m87&amp;quot;,&#xA;  &amp;quot;m58&amp;quot;, &amp;quot;m86&amp;quot;, &amp;quot;m87&amp;quot;&#xA;)&#xA;coldata2$class &amp;lt;- c(&#xA;  &amp;quot;Treated&amp;quot;, &amp;quot;Treated&amp;quot;, &amp;quot;Treated&amp;quot;,&#xA;  &amp;quot;Distal&amp;quot;, &amp;quot;Distal&amp;quot;, &amp;quot;Distal&amp;quot;&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;then everything goes well and I'm able to create a &lt;code&gt;DESeq&lt;/code&gt; object with the following design:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ddsMat &amp;lt;- DESeqDataSetFromMatrix(&#xA;  countData = as.matrix(matrix[,1:6]),&#xA;  colData = coldata2,&#xA;  design = ~ class+Pts&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Please can you provide to me some suggestion on how to solve the problem?&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="15846" LastEditorUserId="15931" LastEditDate="2022-08-19T15:16:05.343" LastActivityDate="2022-08-19T15:16:05.343" Title="nested samples in design DEseq2" Tags="|rna-seq|deseq2|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19459" PostTypeId="2" ParentId="19458" CreationDate="2022-07-25T18:40:44.237" Score="0" Body="&lt;p&gt;Your original was,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ddsMat &amp;lt;- DESeqDataSetFromMatrix(countData = as.matrix(matrix), # colData = coldata, design = ~ class+Pts)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What happens when you use&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ddsMat &amp;lt;- DESeqDataSetFromMatrix(countData = as.matrix(matrix), colData = coldata, design = ~ class+Pts)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thus &lt;code&gt;#&lt;/code&gt; is removed. The suspicion is &lt;code&gt;#&lt;/code&gt; is causing the code to fall over. &lt;code&gt;#&lt;/code&gt;  is used for comments in &lt;code&gt;R&lt;/code&gt; so the presence of this within a line of code will cause the following code &lt;code&gt;colData = coldata, design = ~ class+Pts&lt;/code&gt; to not be interpreted because it can't be seen.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-07-25T18:47:31.870" LastActivityDate="2022-07-25T18:47:31.870" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19460" PostTypeId="1" CreationDate="2022-07-25T20:57:30.557" Score="0" ViewCount="71" Body="&lt;ol&gt;&#xA;&lt;li&gt;I got the data of stomach cancer from TCGA, I found some different&#xA;expression genes between cancer and normal sample.&lt;/li&gt;&#xA;&lt;li&gt;I put aside my normal sample, and divide my cancer sample into&#xA;train(80%) and test(20%) groups.&lt;/li&gt;&#xA;&lt;li&gt;I did Unicox (with survival package), lasso (with glmnet package)&#xA;and multicox ( with survival package) on train group, to find out&#xA;which genes are more related to survival.&lt;/li&gt;&#xA;&lt;li&gt;I use coefficients of each genes (that I got from multicox), to&#xA;built my model and calculate the risk score. Like that: (expression&#xA;of gene1×coefficient gene1 + expression of gene2×coefficient gene2&#xA;+...).&lt;/li&gt;&#xA;&lt;li&gt;I divided my sample in to high risk and low risk groups, to compare&#xA;survival between them, using Kaplan-Meier.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In training group, Kaplan-Meier shows significant difference between high and low risk group. However when I use my model on my test group, it is not significant. Why this problem occurs? I assumed my model is overfit of my train group, but how can I fix it? Why does this happen even when I use lasso?&lt;/p&gt;&#xA;" OwnerUserId="15851" LastEditorUserId="73" LastEditDate="2022-07-26T21:28:06.717" LastActivityDate="2023-12-18T23:07:15.860" Title="My model doesn't work in test dataset" Tags="|r|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19461" PostTypeId="1" CreationDate="2022-07-25T21:35:17.000" Score="1" ViewCount="20" Body="&lt;p&gt;I'm trying to detect convergent evolution in different samples of the same species.&#xA;I'm just trying to see if past 20 generations the nucleotide frequency of snps under pressure converges somewhat for all samples.&#xA;What i want now is to validate my pipeline using a simulation.&#xA;I would like to simulate the evolution of populations and see if the pipeline i built would detect the snps from the reads.&#xA;I'm using pool-seq and illumina with reads of around 100base pairs.&#xA;Can you recommend me some software and maybe point me to the tutorial?&lt;/p&gt;&#xA;" OwnerUserId="9373" LastActivityDate="2022-07-25T21:35:17.000" Title="Simulation software recommendation to confirm pipeline validity" Tags="|simulated-data|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19462" PostTypeId="1" CreationDate="2022-07-25T22:47:15.373" Score="1" ViewCount="335" Body="&lt;p&gt;I’d like to perform the following, but I am having difficulty.&lt;/p&gt;&#xA;&lt;p&gt;I want to convert amino acid sequences in multifasta files into matrices (or vectors) with Euclidean distance and Pearson correlation coefficient.&lt;/p&gt;&#xA;&lt;p&gt;If you know a solution, please let me know. Thanking you in advance.&lt;/p&gt;&#xA;&lt;h3&gt;File1&lt;/h3&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;BDP27474.1 |surface glycoprotein [Severe acute respiratory syndrome coronavirus 2]&#xA;MFVFLVLLPL&#xA;&amp;gt;BDP27485.1 |surface glycoprotein [Severe acute respiratory syndrome coronavirus 2]&#xA;MFVFLVLLPL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3&gt;File2&lt;/h3&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;BDD77411.1 |surface glycoprotein [Severe acute respiratory syndrome coronavirus 2]&#xA;MFVFLVLLPL&#xA;&amp;gt;BDD77515.1 |surface glycoprotein [Severe acute respiratory syndrome coronavirus 2]&#xA;MFVFFVLLPL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3&gt;Code&lt;/h3&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse)&#xA;library(ape)&#xA;library(Biostrings)&#xA;&#xA;naseq1 &amp;lt;- readAAStringSet(&amp;quot;File1.fasta&amp;quot;)&#xA;naseq2 &amp;lt;- readAAStringSet(&amp;quot;File2.fasta&amp;quot;)&#xA;&#xA;dist.aa(naseq1)&#xA;dist.aa(naseq2)&#xA;&#xA;##Error in numeric(n * (n - 1)/2) : invalid 'length' argument&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h3&gt;Add info.&lt;/h3&gt;&#xA;&lt;p&gt;iOS 10.15, Rstudio cloud&lt;/p&gt;&#xA;" OwnerUserId="15848" LastActivityDate="2022-07-25T22:47:15.373" Title="How to calculate distance matrix from fasta?" Tags="|r|fasta|statistics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19463" PostTypeId="2" ParentId="17717" CreationDate="2022-07-26T04:51:26.013" Score="-1" Body="&lt;p&gt;well I came up with the solution. I think it is a little unconventional.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    rule gatk_CombineGVCFs:                                                 &#xA;    &#xA;        input:&#xA;            vcf_dummy = expand(&amp;quot;gvcf/{sample}.g.vcf.gz&amp;quot;, sample = SAMPLES), # a dummy vcf to connect this rule to gatk_HaplotypeCaller&#xA;            fa = &amp;quot;data/genome.fa&amp;quot;,&#xA;    &#xA;        output:&#xA;            gvcf = &amp;quot;gvcf_combined/all.g.vcf.gz&amp;quot;,&#xA;            #index = directory(&amp;quot;gvcf_index&amp;quot;)                                 #index generated without seperate output&#xA;        params:&#xA;            list = &amp;quot;gvcfs.list&amp;quot;&#xA;            log: &amp;quot;logs/CombineGVCFs.log&amp;quot;&#xA;        shell:&#xA;            &amp;quot;ls gvcf/*.gz &amp;gt; gvcfs.list &amp;amp;&amp;amp; gatk --java-options -Xmx7g CombineGVCFs -R {input.fa} --variant {params.list} -O {output.gvcf} &amp;quot;   &#xA;             #Two bash comands running sequentially to make a gvcf.list as during my testing the combineGVCFs tool takes lists as input. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15776" LastEditorUserId="15776" LastEditDate="2022-07-26T05:01:30.233" LastActivityDate="2022-07-26T05:01:30.233" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19465" PostTypeId="1" CreationDate="2022-07-26T07:47:06.617" Score="1" ViewCount="290" Body="&lt;p&gt;I have mapped reads to the yeast (R64-1-1) genome with STAR like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;STAR --runThreadN 44 --runMode alignReads --genomeDir /home/user/Documents/scripts/pyseqtools/index/STAR/R64-1-1 --readFilesIn /home/user/Documents/analyses/TT-Seq/user/trim/WT_3_R1_001_val_1.fq.gz /home/user/Documents/analyses/TT-Seq/user/trim/WT_3_R2_001_val_2.fq.gz --readFilesCommand zcat --quantMode TranscriptomeSAM GeneCounts --twopassMode Basic --outSAMunmapped None --outSAMattrRGline ID:WT_3 PU:WT_3 SM:WT_3 LB:unknown PL:illumina --outSAMtype BAM Unsorted --outTmpDir /home/user/Documents/analyses/TT-Seq/user/tmp --outFileNamePrefix /home/user/Documents/analyses/TT-Seq/user/bam/R64-1-1/WT_3/WT_3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I wanted to count the reads using &lt;code&gt;htseq-count&lt;/code&gt; as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;htseq-count --quiet --format bam --stranded yes /home/user/Documents/analyses/TT-Seq/user/bam/R64-1-1/WT_3/WT_3Aligned.out.bam /home/user/Documents/scripts/pyseqtools/gtf/R64-1-1/Saccharomyces_cerevisiae.R64-1-1.106.gtf &amp;gt; /home/user/Documents/analyses/TT-Seq/user/htseq-count/WT_3.count.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The problem is that most reads map to &lt;code&gt;no_feature&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;__no_feature    4462786&#xA;__ambiguous 12644&#xA;__too_low_aQual 0&#xA;__not_aligned   4907&#xA;__alignment_not_unique  12529444&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have check whether the chromosome names in the BAM file and the GTF file matched (which they seem to do):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BAM:&#xA;@HD VN:1.4&#xA;@SQ SN:I    LN:230218&#xA;@SQ SN:II   LN:813184&#xA;@SQ SN:III  LN:316620&#xA;@SQ SN:IV   LN:1531933&#xA;@SQ SN:V    LN:576874&#xA;@SQ SN:VI   LN:270161&#xA;@SQ SN:VII  LN:1090940&#xA;@SQ SN:VIII LN:562643&#xA;@SQ SN:IX   LN:439888&#xA;@SQ SN:X    LN:745751&#xA;@SQ SN:XI   LN:666816&#xA;@SQ SN:XII  LN:1078177&#xA;@SQ SN:XIII LN:924431&#xA;@SQ SN:XIV  LN:784333&#xA;@SQ SN:XV   LN:1091291&#xA;@SQ SN:XVI  LN:948066&#xA;@SQ SN:Mito LN:85779&#xA;&#xA;GTF: &#xA;IV  sgd gene    8683    9756    .   -   .   gene_id &amp;quot;YDL246C&amp;quot;; gene_name &amp;quot;SOR2&amp;quot;; gene_source &amp;quot;sgd&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;;&#xA;IV  sgd transcript  8683    9756    .   -   .   gene_id &amp;quot;YDL246C&amp;quot;; transcript_id &amp;quot;YDL246C_mRNA&amp;quot;; gene_name &amp;quot;SOR2&amp;quot;; gene_source &amp;quot;sgd&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;SOR2&amp;quot;; transcript_source &amp;quot;sgd&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;;&#xA;IV  sgd exon    8683    9756    .   -   .   gene_id &amp;quot;YDL246C&amp;quot;; transcript_id &amp;quot;YDL246C_mRNA&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;SOR2&amp;quot;; gene_source &amp;quot;sgd&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;SOR2&amp;quot;; transcript_source &amp;quot;sgd&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;; exon_id &amp;quot;YDL246C_mRNA-E1&amp;quot;;&#xA;IV  sgd CDS 8686    9756    .   -   0   gene_id &amp;quot;YDL246C&amp;quot;; transcript_id &amp;quot;YDL246C_mRNA&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;SOR2&amp;quot;; gene_source &amp;quot;sgd&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;SOR2&amp;quot;; transcript_source &amp;quot;sgd&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;; protein_id &amp;quot;YDL246C&amp;quot;;&#xA;IV  sgd start_codon 9754    9756    .   -   0   gene_id &amp;quot;YDL246C&amp;quot;; transcript_id &amp;quot;YDL246C_mRNA&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;SOR2&amp;quot;; gene_source &amp;quot;sgd&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;SOR2&amp;quot;; transcript_source &amp;quot;sgd&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;;&#xA;IV  sgd stop_codon  8683    8685    .   -   0   gene_id &amp;quot;YDL246C&amp;quot;; transcript_id &amp;quot;YDL246C_mRNA&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;SOR2&amp;quot;; gene_source &amp;quot;sgd&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;SOR2&amp;quot;; transcript_source &amp;quot;sgd&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;;&#xA;IV  sgd gene    17577   18566   .   -   .   gene_id &amp;quot;YDL243C&amp;quot;; gene_name &amp;quot;AAD4&amp;quot;; gene_source &amp;quot;sgd&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;;&#xA;IV  sgd transcript  17577   18566   .   -   .   gene_id &amp;quot;YDL243C&amp;quot;; transcript_id &amp;quot;YDL243C_mRNA&amp;quot;; gene_name &amp;quot;AAD4&amp;quot;; gene_source &amp;quot;sgd&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;AAD4&amp;quot;; transcript_source &amp;quot;sgd&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;;&#xA;IV  sgd exon    17577   18566   .   -   .   gene_id &amp;quot;YDL243C&amp;quot;; transcript_id &amp;quot;YDL243C_mRNA&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;AAD4&amp;quot;; gene_source &amp;quot;sgd&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;AAD4&amp;quot;; transcript_source &amp;quot;sgd&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;; exon_id &amp;quot;YDL243C_mRNA-E1&amp;quot;;&#xA;IV  sgd CDS 17580   18566   .   -   0   gene_id &amp;quot;YDL243C&amp;quot;; transcript_id &amp;quot;YDL243C_mRNA&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;AAD4&amp;quot;; gene_source &amp;quot;sgd&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;AAD4&amp;quot;; transcript_source &amp;quot;sgd&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;; protein_id &amp;quot;YDL243C&amp;quot;;&#xA;IV  sgd start_codon 18564   18566   .   -   0   gene_id &amp;quot;YDL243C&amp;quot;; transcript_id &amp;quot;YDL243C_mRNA&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;AAD4&amp;quot;; gene_source &amp;quot;sgd&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;AAD4&amp;quot;; transcript_source &amp;quot;sgd&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;;&#xA;IV  sgd stop_codon  17577   17579   .   -   0   gene_id &amp;quot;YDL243C&amp;quot;; transcript_id &amp;quot;YDL243C_mRNA&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;AAD4&amp;quot;; gene_source &amp;quot;sgd&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;AAD4&amp;quot;; transcript_source &amp;quot;sgd&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;;&#xA;IV  sgd gene    1248154 1249821 .   -   .   gene_id &amp;quot;YDR387C&amp;quot;; gene_name &amp;quot;CIN10&amp;quot;; gene_source &amp;quot;sgd&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;;&#xA;IV  sgd transcript  1248154 1249821 .   -   .   gene_id &amp;quot;YDR387C&amp;quot;; transcript_id &amp;quot;YDR387C_mRNA&amp;quot;; gene_name &amp;quot;CIN10&amp;quot;; gene_source &amp;quot;sgd&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;CIN10&amp;quot;; transcript_source &amp;quot;sgd&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;;&#xA;IV  sgd exon    1248154 1249821 .   -   .   gene_id &amp;quot;YDR387C&amp;quot;; transcript_id &amp;quot;YDR387C_mRNA&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;CIN10&amp;quot;; gene_source &amp;quot;sgd&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;CIN10&amp;quot;; transcript_source &amp;quot;sgd&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;; exon_id &amp;quot;YDR387C_mRNA-E1&amp;quot;;&#xA;IV  sgd CDS 1248157 1249821 .   -   0   gene_id &amp;quot;YDR387C&amp;quot;; transcript_id &amp;quot;YDR387C_mRNA&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;CIN10&amp;quot;; gene_source &amp;quot;sgd&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;CIN10&amp;quot;; transcript_source &amp;quot;sgd&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;; protein_id &amp;quot;YDR387C&amp;quot;;&#xA;IV  sgd start_codon 1249819 1249821 .   -   0   gene_id &amp;quot;YDR387C&amp;quot;; transcript_id &amp;quot;YDR387C_mRNA&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;CIN10&amp;quot;; gene_source &amp;quot;sgd&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;CIN10&amp;quot;; transcript_source &amp;quot;sgd&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;;&#xA;IV  sgd stop_codon  1248154 1248156 .   -   0   gene_id &amp;quot;YDR387C&amp;quot;; transcript_id &amp;quot;YDR387C_mRNA&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;CIN10&amp;quot;; gene_source &amp;quot;sgd&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_name &amp;quot;CIN10&amp;quot;; transcript_source &amp;quot;sgd&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;;&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Edit: the output of &lt;code&gt;ReadsPerGene.out.tab&lt;/code&gt; is (thanks&#xA;swbarnes2) :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;N_unmapped  73359652    73359652    73359652&#xA;N_multimapping  15789900    15789900    15789900&#xA;N_noFeature 192619  5872967 380420&#xA;N_ambiguous 648956  13479   430527&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Would anyone know why most reads map to &lt;code&gt;no_feature&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="10386" LastEditorUserId="10386" LastEditDate="2022-07-29T11:48:05.380" LastActivityDate="2022-07-29T11:48:05.380" Title="Most reads no_feature with htseq-count" Tags="|sequence-alignment|star|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19466" PostTypeId="1" AcceptedAnswerId="19475" CreationDate="2022-07-26T10:00:14.517" Score="1" ViewCount="98" Body="&lt;p&gt;I'm using the &lt;a href=&quot;http://www.bioconductor.org/packages/release/bioc/html/flowCore.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;flowCore&lt;/a&gt; packages to analyze cytometry data, and I am trying to understand all the parameters stored in the FCS file.&#xA;I've figured out most of the elements, but despite a thorough search I haven't been able to find documentation on some of the parameters.&lt;/p&gt;&#xA;&lt;p&gt;For each channel, there are 9 associated parameters. Here's P1 and its associated values from one of my files :&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;$P1N&lt;/code&gt;    FSC-A&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$P1R&lt;/code&gt;    262144&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$P1B&lt;/code&gt;    32&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$P1E&lt;/code&gt;    0,0&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$P1V&lt;/code&gt;    225&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$P1G&lt;/code&gt;    1&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$P1DISPLAY&lt;/code&gt;  LIN&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$P1BS&lt;/code&gt;   0&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$P1MS&lt;/code&gt;   0&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I've figured out the following from various sources (notably the FCS data standards, given &lt;a href=&quot;https://isac-net.org/page/Data-Standards&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) or from the fields themselves:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;$PnN&lt;/code&gt; Short name for parameter n.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$PnR&lt;/code&gt; Range for parameter number n.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$PnB&lt;/code&gt; Number of bits reserved for parameter number n.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$PnE&lt;/code&gt; Amplification type for parameter n.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$PnV&lt;/code&gt; Detector voltage for parameter n.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$PnG&lt;/code&gt; Amplifier gain used for acquisition of parameter n.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$PnDISPLAY&lt;/code&gt;  the transformation to apply when displaying the data (e.g. LIN, LOG)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, these 2 don't appear in the file standard description, and flowCore does not have documentation on the parameters either.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;$PnBS&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$PnMS&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I've seen &lt;code&gt;$PnBS&lt;/code&gt; set at -1, 0, 48, 57, 54, 52 or 25 for various channels in my data, while &lt;code&gt;$PnMS&lt;/code&gt; always seems to be at 0.&lt;/p&gt;&#xA;&lt;p&gt;Does someone know what it is, or where to look for a definition?&#xA;If that helps, my FCS files were generated through BD FACSDiva Software Version 9.0.1.&lt;/p&gt;&#xA;" OwnerUserId="8575" LastActivityDate="2022-07-28T09:33:56.327" Title="What do &quot;PnBS&quot; and &quot;PnMS&quot; mean in FCS flowCore parameters?" Tags="|r|flow-cytometry|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19467" PostTypeId="2" ParentId="19458" CreationDate="2022-07-26T12:47:42.553" Score="2" Body="&lt;p&gt;It is actually simple. The first two conditions are nested with the Pt's starting with m and the other two conditions are nested with those PT's starting with c. That means you cannot run this design. The only solution is to subset the data into two chunks and analyze separately.&lt;/p&gt;&#xA;" OwnerUserId="15854" LastActivityDate="2022-07-26T12:47:42.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19468" PostTypeId="1" AcceptedAnswerId="19470" CreationDate="2022-07-26T14:10:20.243" Score="1" ViewCount="88" Body="&lt;p&gt;I'm working with whole genome metagenomic samples from human skin, and I'm using MEGAHIT for assembly and MetaQuast for evaluation. However, MetaQuast requires a list of reference genomes for the analysis (if I understood correctly). When I perform the evaluation without the references, it uses BLASTN for aligning contigs to SILVA 16S rRNA database. Since I'm working with whole genome sequences, this is not an optimal choice. As expected, MetaQuast is identifying only two species in my contigs file, although I have manually checked through BLASTN that there are contigs that match other species' genomes. And this evaluation makes my assembly looks terrible. Since I'm working with samples from human skin, it wouldn't be viable to download a database of reference genomes and perform MetaQuast using a local alignment. So, what would be the best approach to perform the evaluation correctly in my case?&lt;/p&gt;&#xA;&lt;p&gt;Looking forward to discussing it!&lt;/p&gt;&#xA;" OwnerUserId="8872" LastActivityDate="2022-07-26T20:29:31.980" Title="MetaQuast for assembling samples from complex communities" Tags="|assembly|metagenome|wgs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19469" PostTypeId="2" ParentId="19460" CreationDate="2022-07-26T15:17:07.250" Score="0" Body="&lt;p&gt;There is no overtraining here, it isn't deep learning and there appears no parameterisation. If parameterisation was occurring on the training set that can and does cause over-tightening.&lt;/p&gt;&#xA;&lt;p&gt;Essentially, you haven't included a data split for parameterisation, i.e. there is no parameterisation. The theory of a parameterisation is it must be distinct from training (and obviously) testing. It allows parameter tuned without overtightening. Lasso regression will require parameterisation.&lt;/p&gt;&#xA;&lt;p&gt;I don't know about the survival regressions for machine learning, but lasso/ridge regression are standard approaches.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;So just to explain, what you do is have a split for parameters. Thus instead of 80:20 split it could be 70:20:10 split where the 10% is used for parameter optimisation. Bad code risks leakage from the parameter split and that can result in over-tightening.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-07-26T20:01:37.923" LastActivityDate="2022-07-26T20:01:37.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19470" PostTypeId="2" ParentId="19468" CreationDate="2022-07-26T20:29:31.980" Score="1" Body="&lt;p&gt;I would suggest first using a binning approach such as &lt;a href=&quot;https://bitbucket.org/berkeleylab/metabat&quot; rel=&quot;nofollow noreferrer&quot;&gt;MetaBAT2&lt;/a&gt; (or an equivalent tool, there are many) to separate your metagenomic assembly into bins roughly corresponding to genomes. I would then run &lt;a href=&quot;https://ecogenomics.github.io/CheckM/#:%7E:text=CheckM%20provides%20a%20set%20of,copy%20within%20a%20phylogenetic%20lineage.&quot; rel=&quot;nofollow noreferrer&quot;&gt;CheckM&lt;/a&gt; to assess the bins' quality/completeness/purity.&lt;/p&gt;&#xA;&lt;p&gt;If you still want to use quast, you could then run metaquast with a reference genome appropriate to each bin independently. I would suggest &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/36/6/1925/5626182&quot; rel=&quot;nofollow noreferrer&quot;&gt;GTDB-tk&lt;/a&gt; to perform a classification and identify such an appropriate reference genome.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-07-26T20:29:31.980" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19471" PostTypeId="2" ParentId="19458" CreationDate="2022-07-26T22:11:24.543" Score="1" Body="&lt;p&gt;The &lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2 documentation&lt;/a&gt; explains this error as follows:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;While most experimental designs run easily using design formula, some design formulas can cause problems and result in the DESeq function returning an error with the text: “the model matrix is not full rank, so the model cannot be fit as specified.” There are two main reasons for this problem: either one or more columns in the model matrix are linear combinations of other columns, or there are levels of factors or combinations of levels of multiple factors which are missing samples.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In your case, I think the error is popping up because there are too many defined values in the &amp;quot;Pts&amp;quot; variable. DESeq2 doesn't require you to separately define replicates as a variable; it assumes that different data columns are taken from different biological replicates.&lt;/p&gt;&#xA;&lt;p&gt;Getting the models right is tricky, and needs a deeper discussion about what the goal of analysis should be. My guess is that the following should work, but only if you're not interested in comparing expression results from different patients:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;coldata$Pts &amp;lt;- c(&#xA;  &amp;quot;m&amp;quot;, &amp;quot;m&amp;quot;, &amp;quot;m&amp;quot;,&#xA;  &amp;quot;m&amp;quot;, &amp;quot;m&amp;quot;, &amp;quot;m&amp;quot;,&#xA;  &amp;quot;c&amp;quot;, &amp;quot;c&amp;quot;, &amp;quot;c&amp;quot;,&#xA;  &amp;quot;c&amp;quot;, &amp;quot;c&amp;quot;, &amp;quot;c&amp;quot;&#xA;)&#xA;coldata$class &amp;lt;- c(&#xA;  &amp;quot;Treated&amp;quot;, &amp;quot;Treated&amp;quot;, &amp;quot;Treated&amp;quot;,&#xA;  &amp;quot;Distal&amp;quot;,  &amp;quot;Distal&amp;quot;,  &amp;quot;Distal&amp;quot;,&#xA;  &amp;quot;CLEFT&amp;quot;,   &amp;quot;CLEFT&amp;quot;,   &amp;quot;CLEFT&amp;quot;,&#xA;  &amp;quot;CRIGHT&amp;quot;,  &amp;quot;CRIGHT&amp;quot;,  &amp;quot;CRIGHT&amp;quot;&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This may not be what you want to do; other ways to deal with this are explained in the DESeq2 documentation on the Bioconductor website:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#model-matrix-not-full-rank&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#model-matrix-not-full-rank&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-07-26T22:11:24.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19472" PostTypeId="2" ParentId="14625" CreationDate="2022-07-27T20:03:23.030" Score="1" Body="&lt;p&gt;The code is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;&#xA;pos = [2, 5, 10, 11, 14]&#xA;newseq = []&#xA;prot = 'M----KFLAFLCLLGFANAQ-------------D-GKCG----TLSNKSPS--------------------K'&#xA;count = 0&#xA;for n,s in enumerate(prot):    &#xA;    if str(s) == '-':&#xA;        count += 1&#xA;        newseq.append('-')&#xA;    else:&#xA;        if (n + 1 - count) in pos:&#xA;            s = re.sub(r'[A-Za-z]', 'Z', s)&#xA;        newseq.append(s)&#xA;print(''.join(newseq))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;M----ZFLZFLCLZZFAZAQ-------------D-GKCG----TLSNKSPS--------------------K&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Precisely as requested, apologies for the delay.&lt;/p&gt;&#xA;&lt;p&gt;Happy to explain why it works - its just a couple of Python tricks and a method. It must be run under Python 3 (unlikely to work in Python 2) and probably &amp;gt;3.6.&lt;/p&gt;&#xA;&lt;p&gt;BTW I normally avoid &lt;code&gt;re&lt;/code&gt; for complicated reasons but its easy, hence its here.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;As @gaspanic has pointed out &lt;code&gt;re&lt;/code&gt; is not only unnecessary it will slow the code (&lt;code&gt;re&lt;/code&gt; is very inefficient). It is in fact one of the 'legendary inefficiencies' of Python. Thus removing it ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# variable declarations omitted&#xA;newseq = list()&#xA;count = 0&#xA;for n,s in enumerate(prot):    &#xA;    if str(s) == '-':&#xA;        count += 1&#xA;        newseq.append('-')&#xA;    else:&#xA;        if (n + 1 - count) in pos:&#xA;            s = 'Z'&#xA;        newseq.append(s)&#xA;print(''.join(newseq))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Same output.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-07-28T02:16:22.137" LastActivityDate="2022-07-28T02:16:22.137" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19473" PostTypeId="2" ParentId="14625" CreationDate="2022-07-28T00:55:26.400" Score="1" Body="&lt;p&gt;Another alternative, without the need for &lt;code&gt;re&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seq = 'M----KFLAFLCLLGFANAQ-------------D-GKCG----TLSNKSPS--------------------K'&#xA;pos = [2, 5, 10, 11, 14]&#xA;&#xA;new_seq = []&#xA;count = 0&#xA;for s in seq:&#xA;    if s == &amp;quot;-&amp;quot;:&#xA;        new_seq.append(s)&#xA;    else:&#xA;        count += 1&#xA;        if count in pos:&#xA;            new_seq.append('Z')&#xA;        else:&#xA;            new_seq.append(s)&#xA;&#xA;print(''.join(new_seq))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Result:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;M----ZFLZFLCLZZFAZAQ-------------D-GKCG----TLSNKSPS--------------------K&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11919" LastEditorUserId="11919" LastEditDate="2022-07-28T01:21:56.273" LastActivityDate="2022-07-28T01:21:56.273" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="19474" PostTypeId="2" ParentId="19427" CreationDate="2022-07-28T09:24:20.240" Score="3" Body="&lt;p&gt;If you do as @james-hawley rightly suggests, keep in mind that pseudo-alignment could detect  a lot of transcripts with low TPM that are actually false positives, as reported &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2021.05.05.442755v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (it's a paper focused on scRNA-seq, but many statements are also valid for bulk). So, you may want to try to remove false positive transcripts, that will likely result with very low read count and TPM. Salmon's decoy function could help here.&lt;/p&gt;&#xA;&lt;p&gt;As an alternative to pseudo-alignment, you could run &lt;a href=&quot;https://github.com/gpertea/stringtie&quot; rel=&quot;nofollow noreferrer&quot;&gt;StringTie&lt;/a&gt; on your BAM file to detect transcript isoforms, using a reasonable threshold of coverage over splice junctions.&lt;/p&gt;&#xA;" OwnerUserId="2355" LastActivityDate="2022-07-28T09:24:20.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19475" PostTypeId="2" ParentId="19466" CreationDate="2022-07-28T09:33:56.327" Score="0" Body="&lt;p&gt;I found my answer in &lt;a href=&quot;https://lists.purdue.edu/pipermail/cytometry/2012-June/043612.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;an old mailing list&lt;/a&gt;. Credit to Josef Spidlen who authored the mail.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Short answer&lt;/em&gt;: both parameters are proprietary keywords from the BD FACSDiva software for storing scaling values.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Long answer&lt;/em&gt; from the mailing list:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;PnBS&lt;/strong&gt; Indicates the automatically set Biexponential scale value for Instrument  Parameter (-1 indicates an invalid scale value). When&#xA;importing an FCS  file, if the FCS file has valid Biexponential&#xA;scales, they should be  applied to the data. If the FCS file doesn't&#xA;have valid biexponential  scales then the scales are calculated from&#xA;the entire data. If  biexponential scales exist, then if AUTOBS is&#xA;set, the PnBS values are  used and applied to the data. If AUTOBS is&#xA;not set, the PnMS values are  used and applied to the data. The&#xA;information represented by this keyword becomes part of a Tube's&#xA;attributes, and does not appear in the Tube's list of keywords. It can&#xA;be viewed in the Biexponential inspector' when the auto scaling option&#xA;is selected.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;PnMS&lt;/strong&gt; Indicates the manually set Biexponential scale value for Instrument  Parameter (-1 indicates an invalid scale value). When&#xA;importing an FCS  file, if the FCS file has valid Biexponential&#xA;scales, they should be  applied to the data. If the FCS file doesn't&#xA;have valid biexponential  scales then the scales are calculated from&#xA;the entire data. If  biexponential scales exist, then if AUTOBS is&#xA;set, the PnBS values are  used and applied to the data. If AUTOBS is&#xA;not set, the PnMS values are  used and applied to the data. The&#xA;information represented by this keyword becomes part of a Tube's&#xA;attributes, and does not appear in the Tube's list of keywords. It can&#xA;be viewed in the Biexponential inspector' when the manual scaling&#xA;option  is selected.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="8575" LastActivityDate="2022-07-28T09:33:56.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19476" PostTypeId="1" CreationDate="2022-07-28T13:17:37.153" Score="2" ViewCount="19" Body="&lt;p&gt;I am looking for a easy way to calculate the intervals of flanking introns based on a transcript annotation file. I have chosen a very complicated method:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;bedtools subtract -s -a genes.bed -b onlyExons.bed &amp;gt; introns_hg38.bed&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then I calculated the start and end positions of the transcripts from transcript.gtf annotation file and searched for adjacent intron intervals in R. Unfortunately I don't find all the flanking introns. Perhaps this is due to the 3'UTR or 5'UTR or other sequences that are not introns or exons. Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="15321" LastActivityDate="2022-07-28T13:17:37.153" Title="Calculate interval of flanking introns" Tags="|r|phylogenetics|bedtools|exon|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19477" PostTypeId="1" CreationDate="2022-07-28T14:42:43.200" Score="1" ViewCount="39" Body="&lt;p&gt;I am wondering what is the significance of finding a particular protein specific to a disease or control group? when we detect 1000s of proteins in a proteomics experiment, how can one be sure that the specific proteins found in say, a disease group is really absent in control group than by random chance?&lt;/p&gt;&#xA;" OwnerUserId="15094" LastEditorUserId="650" LastEditDate="2022-07-28T15:15:32.503" LastActivityDate="2022-07-28T15:15:32.503" Title="Gene/protein expression specific to a group in omics" Tags="|rna-seq|genomics|proteomics|mass-spectrometry|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19478" PostTypeId="2" ParentId="19477" CreationDate="2022-07-28T14:53:21.247" Score="2" Body="&lt;p&gt;You can never be 100% sure that a difference is real and not a statistical fluke. This is the entire concept underlying Type I errors - these are errors where the null hypothesis (usually, that there is no difference) is incorrectly rejected (a difference is called significant, when in fact none truly exists). Especially when performing many hypothesis tests, it's possible that a variable with truly identical distributions between two groups appears to have very different distributions simply due to sampling variability.&lt;/p&gt;&#xA;&lt;p&gt;You can reduce the chance of making such an error by reducing the level of alpha used to determine significance, and by performing appropriate multiple hypothesis correction. With stringent significance thresholds and conservative correction, you can make it extremely unlikely that a &amp;quot;significant&amp;quot; difference could be due to random chance, but it's not usually possible to conduct an experiment with a Type I error rate of 0%.&lt;/p&gt;&#xA;" OwnerUserId="1930" LastActivityDate="2022-07-28T14:53:21.247" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19479" PostTypeId="2" ParentId="19454" CreationDate="2022-07-28T15:45:05.587" Score="2" Body="&lt;p&gt;needed to edit accepted code to make it works:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;file_name = &amp;quot;consen1.txt&amp;quot;&#xA;&#xA;with open(file_name) as infile:&#xA;    new_dict = {}&#xA;    for l in infile:&#xA;        line = l.rstrip('\n')&#xA;        if line.startswith(&amp;quot;&amp;gt;&amp;quot;):&#xA;            k = line.lstrip('&amp;gt;')&#xA;            new_dict[k] = str() # must initiate like string !!!!!!!!!!!!!&#xA;            # new_dict = {k:[]} #is possible alternative initiation &#xA;        else:&#xA;            new_dict[k] += line&#xA;            &#xA;print(new_dict)&#xA;&#xA;for key,values in new_dict.items():&#xA;    # print(key, values)&#xA;    gc = values.count('G') + values.count('C')&#xA;    content = (gc/len(values)) *100&#xA;    new_dict[key] = content&#xA;    &#xA;print(new_dict)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{'Rosalind_4478': 'GCTACATCAACGCCGTGCATGGCGTGTAAAGACCCGCATTTTACAGGCAGTGTATACGACCGGTAACACGATTTAGCATTTACTGAGCCCCTGCGTTATTGGAATATGAGGGAACGTTGCTTATGTGCAAGACATTCCACCTCATCCATAGATACTCTCGCGGCAGATTCTGTGACGGATCGGATCGCTTCAAAGCACGGTACGGTATTAGACGAATGGGTTAAAACTCTCTACAAGCACTCGTACAACTTATGCACACCAACTTCAGCGTATAACATGAGAGCCCGGAAAGGCATATTCCTTATCTTAAGGTAGGGAATACGAAAACCCGCGTCGTCCGCCCACCTCGACGGTACGGAATGTTTGAGGGGCATCCGGGTCTGGCAGGAACTGTGTGGACTGGTGCGTCATCAGCTGGGCTGATTGCGCGAGCCAGAATCGCATGCGCTTCGAATTTTAGGAGAGCGTGCCGACTCACGTATATCATTGACATGGTGAGGGTAGCAGAATCTACATTGCAGCCTATGACGCTACACGGCCGTATAATCAGTGTTAGTGCAGTATTGTGGCATTTCCGTTTCGCATGGTTAACTGAGGCCAAGGTTTGTGCCGGGGGACCGGATGGCTACCTGGAAACTAGGATCTGGCCGTAGTGGTTCTCGCGAACCCTAGTTTCAAGCAAACGCCAGCCGGAGACCGAAGTTAAACGCATCACTACTACTCACGAGTCTTGGACCAAAACAGAAAGTCGGATGCACCATGTACGAGCTGACGTTGTCTTTATGTCCAGTTGGATCTGGACAGCACATCCGAGATTTTGAATTAAAACTTCTTTTTGTGTGATGCTTGGAACACCCGCCATACAACAAATAATAGCGCCTTAAACAGCCTACGGCACAGTA'}&#xA;{'Rosalind_4478': 49.55654101995565}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;works using this snippet too. but feel like is not orthodox python:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;file_name = &amp;quot;consen1.txt&amp;quot;&#xA;&#xA;&#xA;with open(file_name) as infile:&#xA;        new_dict = {}&#xA;        for line in infile:&#xA;            if line.startswith(&amp;quot;&amp;gt;&amp;quot;):&#xA;                key = line.strip('\n')&#xA;                &#xA;                print(key)&#xA;                &#xA;                line_2 = None&#xA;                &#xA;            else:&#xA;                &#xA;                if not line_2:&#xA;                &#xA;                    line_2 = ''.join(line.strip('\n'))&#xA;                &#xA;                else:&#xA;                    &#xA;                    line_2 += ''.join(line.strip('\n'))&#xA;                &#xA;                # print(line_2)&#xA;                &#xA;                new_dict[key] = line_2&#xA;                &#xA;                &#xA;print(new_dict)  &#xA;&#xA;for key,values in new_dict.items():&#xA;    gc = values.count('G') + values.count('C')&#xA;    content = (gc/len(values)) *100&#xA;    new_dict[key] = content&#xA;    &#xA;print(new_dict)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Rosalind_4478&#xA;{'&amp;gt;Rosalind_4478': 'GCTACATCAACGCCGTGCATGGCGTGTAAAGACCCGCATTTTACAGGCAGTGTATACGACCGGTAACACGATTTAGCATTTACTGAGCCCCTGCGTTATTGGAATATGAGGGAACGTTGCTTATGTGCAAGACATTCCACCTCATCCATAGATACTCTCGCGGCAGATTCTGTGACGGATCGGATCGCTTCAAAGCACGGTACGGTATTAGACGAATGGGTTAAAACTCTCTACAAGCACTCGTACAACTTATGCACACCAACTTCAGCGTATAACATGAGAGCCCGGAAAGGCATATTCCTTATCTTAAGGTAGGGAATACGAAAACCCGCGTCGTCCGCCCACCTCGACGGTACGGAATGTTTGAGGGGCATCCGGGTCTGGCAGGAACTGTGTGGACTGGTGCGTCATCAGCTGGGCTGATTGCGCGAGCCAGAATCGCATGCGCTTCGAATTTTAGGAGAGCGTGCCGACTCACGTATATCATTGACATGGTGAGGGTAGCAGAATCTACATTGCAGCCTATGACGCTACACGGCCGTATAATCAGTGTTAGTGCAGTATTGTGGCATTTCCGTTTCGCATGGTTAACTGAGGCCAAGGTTTGTGCCGGGGGACCGGATGGCTACCTGGAAACTAGGATCTGGCCGTAGTGGTTCTCGCGAACCCTAGTTTCAAGCAAACGCCAGCCGGAGACCGAAGTTAAACGCATCACTACTACTCACGAGTCTTGGACCAAAACAGAAAGTCGGATGCACCATGTACGAGCTGACGTTGTCTTTATGTCCAGTTGGATCTGGACAGCACATCCGAGATTTTGAATTAAAACTTCTTTTTGTGTGATGCTTGGAACACCCGCCATACAACAAATAATAGCGCCTTAAACAGCCTACGGCACAGTA'}&#xA;{'&amp;gt;Rosalind_4478': 49.55654101995565}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;be sure to check if they are both working with your input&lt;/p&gt;&#xA;" OwnerUserId="10049" LastActivityDate="2022-07-28T15:45:05.587" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19480" PostTypeId="1" CreationDate="2022-07-29T09:12:19.747" Score="3" ViewCount="69" Body="&lt;p&gt;I have the following protein structural variance list:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mutations = ['F594_D600dup', 'D586_E596dup', 'D593_F594insSPEDNEYFYVD', 'E598_F612dup', 'E598_Y599insSGSSDNEYFYVDFREYE', 'E598_Y599insVAYVDFREYE', 'E604_F605insSPRGGNEYFYVDFREYEYDLKWE', 'F594_R595insGTGSSDNEYFYVDF', 'F612_G613insQGFYVDFREYEYDLKWEFPRENLEF', 'L601_K602insNVDFREYEYDL', 'M578_E598dup', 'N587_D600dup', 'R595_E596insDYVDFR', 'R595_L601dup', 'S584_D600dup', 'S584_F605dup', 'S585_F594dup', 'Y597_E598insAGSSDNEYFYVDFREY', 'Y597_E598insDEYFYVDFREY', 'Y597_K602dup', 'Y599_D600insEYEYEYEY', 'Y599_D600insPAPQIMSTSTLISENMNIA', 'Y599_E604dup']) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and I have this protein sequence:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seq=&amp;quot;MPALARDGGQLPLLVVFSAMIFGTITNQDLPVIKCVLINHKNNDSSVGKSSSYPMVSESPEDLGCALRPQSSGTVYEAAAVEVDVSASITLQVLVDAPGNISCLWVFKHSSLNCQPHFDLQNRGVVSMVILKMTETQAGEYLLFIQSEATNYTILFTVSIRNTLLYTLRRPYFRKMENQDALVCISESVPEPIVEWVLCDSQGESCKEESPAVVKKEEKVLHELFGTDIRCCARNELGRECTRLFTIDLNQTPQTTLPQLFLKVGEPLWIRCKAVHVNHGFGLTWELENKALEEGNYFEMSTYSTNRTMIRILFAFVSSVARNDTGYYTCSSSKHPSQSALVTIVEKGFINATNSSEDYEIDQYEEFCFSVRFKAYPQIRCTWTFSRKSFPCEQKGLDNGYSISKFCNHKHQPGEYIFHAENDDAQFTKMFTLNIRRKPQVLAEASASQASCFSDGYPLPSWTWKKCSDKSPNCTEEITEGVWNRKANRKVFGQWVSSSTLNMSEAIKGFLVKCCAYNSLGTSCETILLNSPGPFPFIQDNISFYATIGVCLLFIVVLTLLICHKYKKQFRYESQLQMVQVTGSSDNEYFYVDFREYEYDLKWEFPRENLEFGKVLGSGAFGKVMNATAYGISKTGVSIQVAVKMLKEKADSSEREALMSELKMMTQLGSHENIVNLLGACTLSGPIYLIFEYCCYGDLLNYLRSKREKFHRTWTEIFKEHNFSFYPTFQSHPNSSMPGSREVQIHPDSDQISGLHGNSFHSEDEIEYENQKRLEEEEDLNVLTFEDLLCFAYQVAKGMEFLEFKSCVHRDLAARNVLVTHGKVVKICDFGLARDIMSDSNYVVRGNARLPVKWMAPESLFEGIYTIKSDVWSYGILLWEIFSLGVNPYPGIPVDANFYKLIQNGFKMDQPFYATEEIYIIMQSCWAFDSRKRPSFPNLTSFLGCQLADAEEAMYQNVDGRVSECPHTYQNRRPFSREMDLGLLSPQAQVEDS&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would be happy to work with &lt;a href=&quot;https://github.com/biocommons/hgvs&quot; rel=&quot;nofollow noreferrer&quot;&gt;hgvs&lt;/a&gt; in order to project the mutations on the given sequence. However I am not sure how to feed &lt;code&gt;hgvs&lt;/code&gt; with an inline sequence (not via a reference sequence)&lt;/p&gt;&#xA;&lt;p&gt;First, we need to convert all single letters to a three-letter format. So for example, &lt;code&gt;E27del&lt;/code&gt; will be converted to &lt;code&gt;Glu27del&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;str = &amp;quot;Test:p.Glu27del&amp;quot;&#xA;hp = hgvs.parser.Parser()&#xA;var_str = hp.parse_hgvs_variant(str)&#xA;print(var_str)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I am not sure how to project the &lt;a href=&quot;https://hgvs.readthedocs.io/en/stable/examples/using-hgvs.html#projecting-variants-between-sequences&quot; rel=&quot;nofollow noreferrer&quot;&gt;mutation&lt;/a&gt; if the sequence is simply a string. In the example from their paper they use hg19 as a reference and I want to inject a short sequence instead:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import hgvs.variantmapper&#xA;vm = hgvs.assemblymapper.AssemblyMapper(&#xA;    hdp, assembly_name='GRCh37', alt_aln_method='splign')&#xA;var_g = vm.c_to_g(var_c1)&#xA;var_g&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="734" LastEditorUserId="734" LastEditDate="2022-07-30T20:55:13.380" LastActivityDate="2022-07-30T20:55:13.380" Title="How to use hgvs to project a variant list on a given protein sequence string?" Tags="|python|mutations|structural-variation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19481" PostTypeId="1" CreationDate="2022-07-29T20:55:31.307" Score="1" ViewCount="192" Body="&lt;p&gt;I'm exploring WGCNA for bulkRNA sequencing analysis with human subjects.  I have healthy, myocarditis, and heart failure patients (which can be further broken into ischemic or nonischemic)&lt;/p&gt;&#xA;&lt;p&gt;I have been spending quite a bit of time tinkering with the parameters in blockwisemodules() to identify modules/MEs, and I'm only able to assess the quality of this step with a rudimentary intuition.&lt;/p&gt;&#xA;&lt;p&gt;For example, if I use&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;net = blockwiseModules(datExpr, power = 10,maxBlockSize = 20000,TOMType = &amp;quot;unsigned&amp;quot;, minModuleSize = 20,reassignThreshold = 50, mergeCutHeight = 0.35,numericLabels = TRUE, pamStage = TRUE, pamRespectsDendro = TRUE)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I get what is so far the &amp;quot;best&amp;quot; module detection - many modules (although small), and about 60-70% of genes seem to fall into a module. Image below.&lt;/p&gt;&#xA;&lt;p&gt;I find myself at a bit of a loss as to how to judge the quality of this step.  The module detection looks extremely messy compared with the tutorial data, which i wasn't surprised at really, but can anyone provide guidance here?&lt;/p&gt;&#xA;&lt;p&gt;I can provide other images of what I get from this step that looks like much poorer detection of modules if it would help.&lt;/p&gt;&#xA;&lt;p&gt;Last related question, I have 19 healthy, and about 41 patient samples, I wasn't sure if the best approach to module detection is to put all samples together, then look at what modules are associated with disease states which i could convert into numeric for that step, or would it be better as i have started - to detect modules in healthy, or patient groups in independent datasets?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bsiSp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bsiSp.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15850" LastActivityDate="2022-07-29T20:55:31.307" Title="How to assess quality of WGCNA module identification in blockwisemodules()" Tags="|rna-seq|wgcna|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19482" PostTypeId="1" CreationDate="2022-07-30T12:08:07.303" Score="1" ViewCount="19" Body="&lt;p&gt;I am trying to design a plasmid for our knock in experiment and wanted to clarify some technical issues.&lt;/p&gt;&#xA;&lt;p&gt;I wanted to know exactly which components must be included in the plasmid.&lt;/p&gt;&#xA;&lt;p&gt;I know that I have to include the 5' homology arm , insertion gene sequence, marker sequence and 3' homology arm.&#xA;But, I would like to know for certain the following details:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Should I take the whole sequence of the gene or do I need to remove the introns/UTR regions or any other unnecessary part of the sequence?&lt;/li&gt;&#xA;&lt;li&gt;Which sequence should I choose for gene insertion , gene sequence or mRNA transcript sequence?&lt;/li&gt;&#xA;&lt;li&gt;When designing plasmids, should I insert extra promoters and/or terminators with the insertion genes?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Additionally, can you recommend an online tool to design HDR Plasmid?&#xA;I have already tried using Alt R HDR design tool and Benchling but our target genome is not available there.&lt;/p&gt;&#xA;" OwnerUserId="15874" LastActivityDate="2022-07-30T12:08:07.303" Title="Which components must be included in the HDR plasmid?" Tags="|gene|genetics|computational-biochemistry|design|crispr|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19483" PostTypeId="1" CreationDate="2022-07-30T16:56:58.180" Score="3" ViewCount="339" Body="&lt;p&gt;I am using the below code to get the bar plot. I want to get the bar plot with error bars displayed.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import seaborn as sns&#xA;&#xA;def plot_coefficients(target, coeffs_df, n=10):&#xA;    if coeffs_df.shape[0]==0:&#xA;        return None # nothing to plot here!&#xA;    data = coeffs_df.copy()&#xA;    data = data.sort_values(('Coefficient','mean'), ascending=False).head(n)&#xA;    g = sns.barplot(&#xA;        data=data,&#xA;        y='Gene',&#xA;        x=('Coefficient','mean'),&#xA;        hue='Feature Type',&#xA;        hue_order=[&#xA;            h for h in [&#xA;                'Genomic Variant',&#xA;                'Gene Expression',&#xA;                'Proteomics',&#xA;                'Methylation',&#xA;            ] &#xA;        ],&#xA;        dodge=False&#xA;    )&#xA;    g.set(&#xA;        xlim=(0,1),&#xA;        xlabel='Importance [0,1]',&#xA;        ylabel='',&#xA;        title=f&amp;quot;Top {target} predictor features&amp;quot;&#xA;    )&#xA;    return g&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Coefficient here is a score generated from predictive modelling that ranges from 0 to 1. 2. data here is entire data and I created a mean column. Ideally I wanted to calculate it in &lt;a href=&quot;https://seaborn.pydata.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;seaborn&lt;/a&gt; plot rather than creating a mean column separately before.&lt;/p&gt;&#xA;" OwnerUserId="4783" LastEditorUserId="57" LastEditDate="2022-08-01T12:55:53.860" LastActivityDate="2022-08-01T14:54:20.490" Title="How To display bar graph with error bars?" Tags="|python|visualization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19484" PostTypeId="1" CreationDate="2022-07-31T08:42:16.067" Score="1" ViewCount="135" Body="&lt;p&gt;I'm using snpEff to annotate genetic variants. However, the annotation often returns more than one annotation (see below for an example):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;NZ_JAKVDJ010000001.1    119361  .   T   C   23.1481 .   DP=22;VDB=0.124119;SGB=-0.690438;RPB=0.369955;MQB=0.0631559;MQSB=0.000399184;BQB=1;MQ0F=0;AC=1;AN=1;DP4=0,5,12,5;MQ=5;ANN=C|synonymous_variant|LOW|oprB|gene-MMZ02_RS00620|transcript|TRANSCRIPT_gene-MMZ02_RS00620|protein_coding|1/1|c.444A&amp;gt;G|p.Leu148Leu|444/1359|444/1359|148/452||WARNING_TRANSCRIPT_NO_START_CODON,C|upstream_gene_variant|MODIFIER|MMZ02_RS00610|gene-MMZ02_RS00610|transcript|TRANSCRIPT_gene-MMZ02_RS00610|protein_coding||c.-3612A&amp;gt;G|||||3612|,C|upstream_gene_variant|MODIFIER|MMZ02_RS00615|gene-MMZ02_RS00615|transcript|TRANSCRIPT_gene-MMZ02_RS00615|protein_coding||c.-1069A&amp;gt;G|||||1069|,C|downstream_gene_variant|MODIFIER|MMZ02_RS00625|gene-MMZ02_RS00625|transcript|TRANSCRIPT_gene-MMZ02_RS00625|protein_coding||c.*785A&amp;gt;G|||||785|,C|downstream_gene_variant|MODIFIER|MMZ02_RS00630|gene-MMZ02_RS00630|transcript|TRANSCRIPT_gene-MMZ02_RS00630|protein_coding||c.*1063A&amp;gt;G|||||1063|,C|downstream_gene_variant|MODIFIER|MMZ02_RS00635|gene-MMZ02_RS00635|transcript|TRANSCRIPT_gene-MMZ02_RS00635|protein_coding||c.*2025A&amp;gt;G|||||2025|,C|downstream_gene_variant|MODIFIER|MMZ02_RS00640|gene-MMZ02_RS00640|transcript|TRANSCRIPT_gene-MMZ02_RS00640|protein_coding||c.*2876A&amp;gt;G|||||2876|,C|downstream_gene_variant|MODIFIER|MMZ02_RS00645|gene-MMZ02_RS00645|transcript|TRANSCRIPT_gene-MMZ02_RS00645|protein_coding||c.*3698A&amp;gt;G|||||3698|,C|intragenic_variant|MODIFIER|gene-MMZ02_RS01495|null|gene_variant|null|||n.119361T&amp;gt;C||||||    GT:PL   1:50,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, now I wonder if the first annotation always refers to the observed variant (whose position is listed at the start of the line with the annotations an other quality parameters. Thus, if I got the annotation from the example above, can I always trust that the variant I'm looking at occurred in the gene oprB? Or will snpEff sometimes also give annotations where only the 2nd, 3rd, etc. annotation is directly related to the varying DNA position? If so, how can I know with certainty in which gene one variant appears?&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot for your help in advance!&lt;/p&gt;&#xA;" OwnerUserId="15878" LastEditorUserId="15878" LastEditDate="2022-07-31T09:08:12.537" LastActivityDate="2022-08-01T06:36:41.290" Title="snpEff annotation and variant in a gene" Tags="|gene|sequence-annotation|snpeff|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19485" PostTypeId="1" AcceptedAnswerId="19487" CreationDate="2022-07-31T08:59:36.990" Score="2" ViewCount="167" Body="&lt;p&gt;I annotated a bunch of genomes with snpeff. And although the original reads' qualities and the matches with the reference genomes were quite high, I see the above mentioned warning in almost every variant in my vcf file. Thus, I wonder how this could happen. Since I'm using bacterial genomes with a manually built database (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly/?term=Kiel%20pseudomonas%20aeruginosa&quot; rel=&quot;nofollow noreferrer&quot;&gt;from these reference genomes&lt;/a&gt;) and there are quite some genes (especially in gene clusters) that have the alternative start codon GTG, I suspect this causing the warning. Any idea how to test if this is really the reason for the warning?&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot in advance!&lt;/p&gt;&#xA;&lt;p&gt;Edit:&#xA;As requested, &lt;a href=&quot;https://drive.google.com/file/d/1ebJVeaVgLTfPW1rN_MZnl2FPH6thWKcu/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is the html snpEff report.&#xA;And I've also uploaded an example vcf file after annotation &lt;a href=&quot;https://drive.google.com/file/d/1NzAkngXs8VUog_Jhyl0jrq1egtgNnG8J/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15878" LastEditorUserId="15878" LastEditDate="2022-07-31T09:36:08.737" LastActivityDate="2022-10-26T07:58:15.803" Title="Bacterial genomes and snpeff warnings 'WARNING_TRANSCRIPT_NO_START_CODON'" Tags="|genome|bacteria|codon|snpeff|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19487" PostTypeId="2" ParentId="19485" CreationDate="2022-08-01T06:37:41.280" Score="2" Body="&lt;p&gt;Bacteria have a different translation table from eukaryote nuclear DNA; it may be that SnpEff is expecting human genes in the genome. Have you followed the build steps outlined in the documentation?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://pcingola.github.io/SnpEff/se_build_db/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pcingola.github.io/SnpEff/se_build_db/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If your genome, or a chromosome, uses non-standard codon tables you must update snpEff.config accordingly. A typical case is when you use mitochondrial DNA. Then you specify that chromosome 'MT' uses codon. Invertebrate_Mitochondrial codon table. Another common case is when you are adding a bacterial genome, then you specify that the codon table is Bacterial_and_Plant_Plastid.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-08-01T06:37:41.280" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19488" PostTypeId="2" ParentId="19483" CreationDate="2022-08-01T14:41:23.057" Score="1" Body="&lt;p&gt;I did put forward a comment, it was removed I think the idea is its a full post or zero. The problem proving a negative is tricky.&lt;/p&gt;&#xA;&lt;p&gt;I don't think its doable within the exact system specified, i.e. a Seaborn bar chart. Again I could be wrong.&lt;/p&gt;&#xA;&lt;p&gt;If the OP went to a Seaborn dot line formation - that would work &lt;code&gt;sns.barplot&lt;/code&gt; is changed to &lt;code&gt;sns.pointplot&lt;/code&gt; but then all the nice colouring disappears. Standard deviation  is one error range if all the values are present: but I think the OP will need to request it (&lt;code&gt;.sd()&lt;/code&gt; ?? can't remember).&lt;/p&gt;&#xA;&lt;p&gt;In &lt;code&gt;matplotlib&lt;/code&gt; there is an error bar method where the coder specifies the error bars and that is &lt;code&gt;plt.errorbar&lt;/code&gt;. An example from the official docs is &lt;a href=&quot;https://matplotlib.org/stable/gallery/lines_bars_and_markers/errorbar_limits_simple.html#sphx-glr-gallery-lines-bars-and-markers-errorbar-limits-simple-py&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Thus call a bar chart and then add the error bars onto it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; The reasons I don't think its doable in Seaborn:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I'm pretty certain Seaborn has no independent errorbar method unlike matplotlib&lt;/li&gt;&#xA;&lt;li&gt;Seaborn can do error bars given the full data set, but those are created automatically for a given method. Maybe there is a &amp;quot;sns.barplot-errorbar&amp;quot; method but I don't know it.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The Seaborn approach does carry the advantage of not having to work out the error bar independently.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-08-01T14:54:20.490" LastActivityDate="2022-08-01T14:54:20.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19490" PostTypeId="1" CreationDate="2022-08-01T17:52:13.293" Score="1" ViewCount="23" Body="&lt;p&gt;Reading from &lt;a href=&quot;https://cnvkit.readthedocs.io/en/stable/sex.html#reference-sex-chromosome-ploidy&quot; rel=&quot;nofollow noreferrer&quot;&gt;CNVkit documentation&lt;/a&gt;, &lt;a href=&quot;https://www.mathworks.com/help/bioinfo/ug/detecting-dna-copy-number-alteration-in-array-based-cgh-data.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mathworks documentation&lt;/a&gt;, &lt;a href=&quot;https://bionanogenomics.com/wp-content/uploads/2018/04/30210-Introduction-to-Copy-Number-Analysis.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bionano documentation&lt;/a&gt;, for CNV calling, it seems that the common practice for array is to compute the linear copy ratio by normalizing against the expected depth for the reference ploidy, i.e.,:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$CR = \frac{D_{OBS}}{P_{REF}}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$D_{OBS}$&lt;/span&gt; is the observed depth, and &lt;span class=&quot;math-container&quot;&gt;$P_{REF}$&lt;/span&gt; is expected depth for the reference ploidy for a specific segment (i.e., expected depth for 2 copies for females, and males outside sex chromosomes or in PAR regions).&lt;/p&gt;&#xA;&lt;p&gt;Is that correct? Are there alternative approaches used in array copy number calling?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2022-08-01T18:14:01.807" LastActivityDate="2022-08-01T18:14:01.807" Title="Handling of sex chromosomes in male when computing CNV copy ratio" Tags="|microarray|copy-number|chromosomes|array-cgh|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19492" PostTypeId="2" ParentId="19485" CreationDate="2022-08-02T04:34:47.310" Score="1" Body="&lt;p&gt;According to your report, you are using human HG18 as the reference genome version.  Therefore, the report will be wrong along with the wrong codons warnings.  It is unfruitful to try with a human database with bacteria.&lt;/p&gt;&#xA;" OwnerUserId="15884" LastActivityDate="2022-08-02T04:34:47.310" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19493" PostTypeId="1" AcceptedAnswerId="19513" CreationDate="2022-08-02T10:45:48.747" Score="2" ViewCount="159" Body="&lt;p&gt;I have BAM files from an RNA-Seq experiment and for all genes (or a subset) I want to get the number of reads in regions around the TSS (e.g. 2kbp) and the TES (e.g. 2 kbp) and calculate the ratio between them. I was thinking of first creating these regions for each gene and store them in a BED file and then use &lt;code&gt;bedtools intersect&lt;/code&gt; with the BAM files to get the reads in these regions. My questions is how would I make such a BED file that contains only the specified regions downstream of the TSS and specified regions upstream of the TSS?&lt;/p&gt;&#xA;" OwnerUserId="10386" LastActivityDate="2022-08-05T11:05:28.227" Title="Count reads at specific gene features" Tags="|sam|ngs|bedtools|bed|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19494" PostTypeId="1" CreationDate="2022-08-02T11:33:00.890" Score="3" ViewCount="254" Body="&lt;p&gt;Is there a function that can map genomic position (hg19) back from a protein position? I can have name of a particular transcript, and exon number.&lt;/p&gt;&#xA;&lt;p&gt;For example, I have KRAS gene for which I would like to know the exact genomic position of G12, and I expect to have an output saying:&lt;/p&gt;&#xA;&lt;p&gt;chr12, from 25398283 to 25398285.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/zwdzwd/transvar&quot; rel=&quot;nofollow noreferrer&quot;&gt;transvar&lt;/a&gt; could be an excellent solution if the current built was working (broken links for fasta annotations). The web search is working but not allowing files heavier than 5kB which is my case.&lt;/p&gt;&#xA;" OwnerUserId="2876" LastEditorUserId="73" LastEditDate="2022-08-05T13:34:12.997" LastActivityDate="2023-09-01T20:05:35.697" Title="Find a genomic coordinates for a protein aminoacid position" Tags="|sequence-annotation|codon|genomicranges|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19495" PostTypeId="1" CreationDate="2022-08-02T12:59:59.017" Score="2" ViewCount="97" Body="&lt;p&gt;I have the following file in tab separated values (TSV).&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;ID&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Taxonomy&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;76980f6d906c6baaef3f96559fc7ba1b&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;d__Eukaryota; p__Cercozoa; c__Vampyrellidae; o__Vampyrellidae; f__Vampyrellidae; g__uncultured&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;4f808d3adec894395304661391037369&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;d__Eukaryota; p__Ochrophyta; c__Xanthophyceae; o__uncultured; f__uncultured; g__uncultured&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;777584465097568f5b30bc9e2b6584c1&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;d__Eukaryota; p__Labyrinthulomycetes; c__uncultured; o__uncultured; f__uncultured; g__uncultured&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;5a99dafabe78847d8f5fed0d37f8672a&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;d__Eukaryota; p__Labyrinthulomycetes; c__Labyrinthulomycetes; o__Labyrinthulomycetes; f__Thraustochytriaceae; g__uncultured&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;I would like to modify the &amp;quot;g__uncultured&amp;quot; with the last known rank possibly using either shell or R scripts.&lt;/p&gt;&#xA;&lt;p&gt;The new table would then read:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;ID&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Taxonomy&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;76980f6d906c6baaef3f96559fc7ba1b&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;d__Eukaryota; p__Cercozoa; c__Vampyrellidae; o__Vampyrellidae; f__Vampyrellidae; g__f__Vampyrellidae&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;4f808d3adec894395304661391037369&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;d__Eukaryota; p__Ochrophyta; c__Xanthophyceae; o__uncultured; f__uncultured; g__c__Xanthophyceae&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;777584465097568f5b30bc9e2b6584c1&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;d__Eukaryota; p__Labyrinthulomycetes; c__uncultured; o__uncultured; f__uncultured; g__p__Labyrinthulomycetes&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;5a99dafabe78847d8f5fed0d37f8672a&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;d__Eukaryota; p__Labyrinthulomycetes; c__Labyrinthulomycetes; o__Labyrinthulomycetes; f__Thraustochytriaceae; g__f__Thraustochytriaceae&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;" OwnerUserId="15596" LastEditorUserId="3848" LastEditDate="2022-08-02T21:18:24.130" LastActivityDate="2022-08-04T11:50:07.333" Title="Parsing text within a taxonomy.csv file" Tags="|r|phylogenetics|taxonomy|parsing|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19496" PostTypeId="1" CreationDate="2022-08-02T13:24:03.377" Score="0" ViewCount="1115" Body="&lt;p&gt;Which of the following RNA strings could translate into the amino acid string PRTEIN? (Select all that apply.)&lt;/p&gt;&#xA;&lt;p&gt;I am not sure if the following mass of amino acid is relevant to the question.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gMKna.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gMKna.png&quot; alt=&quot;mass&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CCAAGUACAGAGAUUAAC&#xA;&#xA;CCAAGAACAGAUAUCAAU&#xA;&#xA;CCACGUACUGAAAUUAAC&#xA;&#xA;CCGAGGACCGAAAUCAAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15733" LastEditorUserId="3848" LastEditDate="2023-07-04T23:49:49.237" LastActivityDate="2023-07-04T23:49:49.237" Title="RNA strings and amino acid string" Tags="|python|phylogenetics|phylogeny|rna|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19497" PostTypeId="2" ParentId="19496" CreationDate="2022-08-02T13:39:45.837" Score="3" Body="&lt;p&gt;I don't quite get the question but here's the answer and the code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;from Bio.Seq import Seq&#xA;&#xA;for dna in ['CCAAGUACAGAGAUUAAC','CCAAGAACAGAUAUCAAU', 'CCACGUACUGAAAUUAAC','CCGAGGACCGAAAUCAAC']:&#xA;    print(Seq(dna).translate())&#xA;    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;PSTEIN PRTDIN PRTEIN PRTEIN&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-08-02T13:39:45.837" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="19498" PostTypeId="1" AcceptedAnswerId="19499" CreationDate="2022-08-02T14:05:14.530" Score="1" ViewCount="301" Body="&lt;p&gt;what is the opposite of &lt;a href=&quot;https://biopython.org/docs/1.75/api/Bio.Seq.html#Bio.Seq.Seq.translate&quot; rel=&quot;nofollow noreferrer&quot;&gt;.translate()&lt;/a&gt; function calls ?&lt;/p&gt;&#xA;&lt;p&gt;I mean let's say I am given an amino acid string &lt;code&gt;CYCLIC&lt;/code&gt;, how do I obtain &lt;strong&gt;all the possible combinations of DNA strings&lt;/strong&gt; before the translation process ?&lt;/p&gt;&#xA;&lt;p&gt;Note: I was told that &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4135748/&quot; rel=&quot;nofollow noreferrer&quot;&gt;degeneracy&lt;/a&gt; has an important role in this particular aspect, but I am not entirely sure about the degeneracy calculation in the following screenshot highlighted in yellow color.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/R0aG5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/R0aG5.png&quot; alt=&quot;degeneracy&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The degenerate code is as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;table = {&#xA;        'ATA':'I', 'ATC':'I', 'ATT':'I', 'ATG':'M',&#xA;        'ACA':'T', 'ACC':'T', 'ACG':'T', 'ACT':'T',&#xA;        'AAC':'N', 'AAT':'N', 'AAA':'K', 'AAG':'K',&#xA;        'AGC':'S', 'AGT':'S', 'AGA':'R', 'AGG':'R',                 &#xA;        'CTA':'L', 'CTC':'L', 'CTG':'L', 'CTT':'L',&#xA;        'CCA':'P', 'CCC':'P', 'CCG':'P', 'CCT':'P',&#xA;        'CAC':'H', 'CAT':'H', 'CAA':'Q', 'CAG':'Q',&#xA;        'CGA':'R', 'CGC':'R', 'CGG':'R', 'CGT':'R',&#xA;        'GTA':'V', 'GTC':'V', 'GTG':'V', 'GTT':'V',&#xA;        'GCA':'A', 'GCC':'A', 'GCG':'A', 'GCT':'A',&#xA;        'GAC':'D', 'GAT':'D', 'GAA':'E', 'GAG':'E',&#xA;        'GGA':'G', 'GGC':'G', 'GGG':'G', 'GGT':'G',&#xA;        'TCA':'S', 'TCC':'S', 'TCG':'S', 'TCT':'S',&#xA;        'TTC':'F', 'TTT':'F', 'TTA':'L', 'TTG':'L',&#xA;        'TAC':'Y', 'TAT':'Y', 'TAA':'_', 'TAG':'_',&#xA;        'TGC':'C', 'TGT':'C', 'TGA':'_', 'TGG':'W',&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15733" LastEditorUserId="3848" LastEditDate="2023-07-17T19:26:08.750" LastActivityDate="2023-07-17T19:26:08.750" Title="reverse translation from amino acid string to DNA strings" Tags="|python|phylogenetics|phylogeny|sequence-analysis|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19499" PostTypeId="2" ParentId="19498" CreationDate="2022-08-02T15:01:19.477" Score="3" Body="&lt;p&gt;Given the table above the solution for CYCLE is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from itertools import product&#xA;&#xA;prot = 'CYCLE'&#xA;degenDict = dict()&#xA;for v,k in table.items():&#xA;    if k in degenDict:&#xA;        degenDict[k].append(v)&#xA;    else:&#xA;        degenDict[k]=[]&#xA;        degenDict[k].append(v)&#xA;nucs = [degenDict[resid] for resid in prot]&#xA;for degenNuc in product(*nucs):&#xA;    print(''.join(degenNuc))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;TGCTACTGCCTAGAA&#xA;TGCTACTGCCTAGAG&#xA;TGCTACTGCCTCGAA&#xA;TGCTACTGCCTCGAG&#xA;TGCTACTGCCTGGAA&#xA;TGCTACTGCCTGGAG&#xA;TGCTACTGCCTTGAA&#xA;TGCTACTGCCTTGAG&#xA;TGCTACTGCTTAGAA&#xA;TGCTACTGCTTAGAG&#xA;TGCTACTGCTTGGAA&#xA;TGCTACTGCTTGGAG&#xA;TGCTACTGTCTAGAA&#xA;TGCTACTGTCTAGAG&#xA;TGCTACTGTCTCGAA&#xA;TGCTACTGTCTCGAG&#xA;TGCTACTGTCTGGAA&#xA;TGCTACTGTCTGGAG&#xA;TGCTACTGTCTTGAA&#xA;TGCTACTGTCTTGAG&#xA;TGCTACTGTTTAGAA&#xA;TGCTACTGTTTAGAG&#xA;TGCTACTGTTTGGAA&#xA;TGCTACTGTTTGGAG&#xA;TGCTATTGCCTAGAA&#xA;TGCTATTGCCTAGAG&#xA;TGCTATTGCCTCGAA&#xA;TGCTATTGCCTCGAG&#xA;TGCTATTGCCTGGAA&#xA;TGCTATTGCCTGGAG&#xA;TGCTATTGCCTTGAA&#xA;TGCTATTGCCTTGAG&#xA;TGCTATTGCTTAGAA&#xA;TGCTATTGCTTAGAG&#xA;TGCTATTGCTTGGAA&#xA;TGCTATTGCTTGGAG&#xA;TGCTATTGTCTAGAA&#xA;TGCTATTGTCTAGAG&#xA;TGCTATTGTCTCGAA&#xA;TGCTATTGTCTCGAG&#xA;TGCTATTGTCTGGAA&#xA;TGCTATTGTCTGGAG&#xA;TGCTATTGTCTTGAA&#xA;TGCTATTGTCTTGAG&#xA;TGCTATTGTTTAGAA&#xA;TGCTATTGTTTAGAG&#xA;TGCTATTGTTTGGAA&#xA;TGCTATTGTTTGGAG&#xA;TGTTACTGCCTAGAA&#xA;TGTTACTGCCTAGAG&#xA;TGTTACTGCCTCGAA&#xA;TGTTACTGCCTCGAG&#xA;TGTTACTGCCTGGAA&#xA;TGTTACTGCCTGGAG&#xA;TGTTACTGCCTTGAA&#xA;TGTTACTGCCTTGAG&#xA;TGTTACTGCTTAGAA&#xA;TGTTACTGCTTAGAG&#xA;TGTTACTGCTTGGAA&#xA;TGTTACTGCTTGGAG&#xA;TGTTACTGTCTAGAA&#xA;TGTTACTGTCTAGAG&#xA;TGTTACTGTCTCGAA&#xA;TGTTACTGTCTCGAG&#xA;TGTTACTGTCTGGAA&#xA;TGTTACTGTCTGGAG&#xA;TGTTACTGTCTTGAA&#xA;TGTTACTGTCTTGAG&#xA;TGTTACTGTTTAGAA&#xA;TGTTACTGTTTAGAG&#xA;TGTTACTGTTTGGAA&#xA;TGTTACTGTTTGGAG&#xA;TGTTATTGCCTAGAA&#xA;TGTTATTGCCTAGAG&#xA;TGTTATTGCCTCGAA&#xA;TGTTATTGCCTCGAG&#xA;TGTTATTGCCTGGAA&#xA;TGTTATTGCCTGGAG&#xA;TGTTATTGCCTTGAA&#xA;TGTTATTGCCTTGAG&#xA;TGTTATTGCTTAGAA&#xA;TGTTATTGCTTAGAG&#xA;TGTTATTGCTTGGAA&#xA;TGTTATTGCTTGGAG&#xA;TGTTATTGTCTAGAA&#xA;TGTTATTGTCTAGAG&#xA;TGTTATTGTCTCGAA&#xA;TGTTATTGTCTCGAG&#xA;TGTTATTGTCTGGAA&#xA;TGTTATTGTCTGGAG&#xA;TGTTATTGTCTTGAA&#xA;TGTTATTGTCTTGAG&#xA;TGTTATTGTTTAGAA&#xA;TGTTATTGTTTAGAG&#xA;TGTTATTGTTTGGAA&#xA;TGTTATTGTTTGGAG&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-08-02T15:01:19.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19500" PostTypeId="1" CreationDate="2022-08-02T15:47:49.733" Score="1" ViewCount="153" Body="&lt;p&gt;I was doing a simple quiz which I encountered the following two questions related to mass spectrum matching with cyclic peptide and linear peptide.&lt;/p&gt;&#xA;&lt;p&gt;I have done some preliminary google search, but I am not sure how to use either  &lt;a href=&quot;https://codereview.stackexchange.com/questions/193859/find-a-cyclic-peptide-with-a-given-non-ideal-spectrum&quot;&gt;https://codereview.stackexchange.com/questions/193859/find-a-cyclic-peptide-with-a-given-non-ideal-spectrum&lt;/a&gt; or &lt;a href=&quot;https://github.com/rochoa85/pepfun&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/rochoa85/pepfun&lt;/a&gt; in this case.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Which of the following cyclic peptides could have generated the&#xA;theoretical spectrum 0 71 101 113 131 184 202 214 232 285 303 315 345&#xA;416? (Select all that apply.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;TAIM&#xA;&#xA;TMIA&#xA;&#xA;MAIT&#xA;&#xA;MTAI&#xA;&#xA;TMLA&#xA;&#xA;TLAM&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Which of the following linear peptides is consistent with Spectrum =&#xA;{0 71 99 101 103 128 129 199 200 204 227 230 231 298 303 328 330 332&#xA;333}? (Select all that apply.)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;AVQ&#xA;&#xA;TCQ&#xA;&#xA;TVQ&#xA;&#xA;QCV&#xA;&#xA;TCE&#xA;&#xA;ETC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15733" LastEditorUserId="3694" LastEditDate="2022-08-02T15:49:46.203" LastActivityDate="2022-08-02T16:52:59.303" Title="mass spectrum matching with cyclic peptide and linear peptide" Tags="|proteins|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19502" PostTypeId="2" ParentId="19500" CreationDate="2022-08-02T16:52:59.303" Score="3" Body="&lt;p&gt;You are way overthinking this.&lt;/p&gt;&#xA;&lt;p&gt;As this is homework I won't give any code, but I will throw back some questions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What is the definition of m/z?&lt;/li&gt;&#xA;&lt;li&gt;Were the only tool available Wikipedia, could you simply read the infobox of each residue?&lt;/li&gt;&#xA;&lt;li&gt;If so, what values would you need and what elements do you need to subtract?&lt;/li&gt;&#xA;&lt;li&gt;Does residue order matter for linear and backbone cyclised peptides? (Let's ignore isoaspartate etc.)&lt;/li&gt;&#xA;&lt;li&gt;Have you been introduced to RDKit in your course? E.g. what would this give you?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from rdkit import Chem&#xA;from rdkit.Chem import Descriptors&#xA;mol: Chem.Mol = Chem.MolFromFASTA('ELVIS')&#xA;print( Descriptors.ExactMolWt(mol) )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2022-08-02T16:52:59.303" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="19503" PostTypeId="2" ParentId="19495" CreationDate="2022-08-02T18:58:13.983" Score="1" Body="&lt;p&gt;The &lt;code&gt;pandas&lt;/code&gt; solution is here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;import pandas as pd&#xA;from pathlib import Path&#xA;&#xA;path = '/Whatever'&#xA;infile = 'myfile.csv'&#xA;outfile = 'outfile.csv'&#xA;pd.set_option('display.max_colwidth', None)&#xA;df = pd.read_csv(Path(path,infile), sep='\t')&#xA;taxaChange = {'76980f6d906c6baaef3f96559fc7ba1b':'g__f__Vampyrellidae', '4f808d3adec894395304661391037369':'g__c__Xanthophyceae'}&#xA;df2 = df.set_index('ID')&#xA;df3 = df2&#xA;for x,y in taxaChange.items():&#xA;    df3.loc[x, 'Taxonomy'] = df2.loc[x, 'Taxonomy'].replace('g__uncultured', y)&#xA;print (df3)&#xA;&#xA;df.to_csv(Path(path,outfile), sep='\t')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ID                                       Taxonomy                                                                                                                                             &#xA;76980f6d906c6baaef3f96559fc7ba1b         d__Eukaryota; p__Cercozoa; c__Vampyrellidae; o__Vampyrellidae; f__Vampyrellidae; g__f__Vampyrellidae&#xA;4f808d3adec894395304661391037369         d__Eukaryota; p__Ochrophyta; c__Xanthophyceae; o__uncultured; f__uncultured; g__c__Xanthophyceae&#xA;777584465097568f5b30bc9e2b6584c1         d__Eukaryota; p__Labyrinthulomycetes; c__uncultured; o__uncultured; f__uncultured; g__uncultured&#xA;5a99dafabe78847d8f5fed0d37f8672a         d__Eukaryota; p__Labyrinthulomycetes; c__Labyrinthulomycetes; o__Labyrinthulomycetes; f__Thraustochytriaceae; g__uncultured&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Both &lt;code&gt;g__f__Vampyrellidae&lt;/code&gt; and &lt;code&gt;g__c__Xanthophyceae&lt;/code&gt; are there, i.e. the code was tested and worked fine.&lt;/p&gt;&#xA;&lt;p&gt;The point for the &lt;code&gt;pandas&lt;/code&gt; approach is you've a natural table format so it makes sense to take it straight to a dataframe (table -&amp;gt; dataframe). regex-ing using &lt;code&gt;\t&lt;/code&gt; and &lt;code&gt;\n&lt;/code&gt; is of course the alternative and you need to formulate the table as a hash (Perl) or dictionary (Python).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt; The &lt;code&gt;df&lt;/code&gt; and &lt;code&gt;df2&lt;/code&gt; can reduced a single dataframe by updating the &lt;code&gt;df&lt;/code&gt; (&lt;code&gt;inplace&lt;/code&gt; was omitted and will work for an index), rather than creating a new &lt;code&gt;df&lt;/code&gt;. However, the &lt;code&gt;df3&lt;/code&gt; is needed (need to copy the dataframe) because 'inplace' didn't work. &lt;code&gt;inplace&lt;/code&gt; is a command to force the dataframe to update. The &lt;code&gt;set_option&lt;/code&gt; is just so I can print the display without truncation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Finally&lt;/em&gt;&#xA;You substitute the infile and outfile to correct path and name values, save it as a &lt;code&gt;.py&lt;/code&gt; file and just &lt;code&gt;python3 myscript.py&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-08-04T11:50:07.333" LastActivityDate="2022-08-04T11:50:07.333" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19504" PostTypeId="1" CreationDate="2022-08-03T08:42:41.477" Score="1" ViewCount="44" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/9533382/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;From hearsay I know that de Bruijn graphs of large genomes (e.g. human) are usually constructed with k = 51, or that k = 51 is at least a good initial choice.&lt;/p&gt;&#xA;&lt;p&gt;I however am unable to find any source for this, does anyone know where it is coming from?&lt;/p&gt;&#xA;" OwnerUserId="9255" LastEditorUserId="73" LastEditDate="2022-08-03T12:10:35.460" LastActivityDate="2022-08-03T12:10:35.460" Title="Where does the choice of k = 51 for de Bruijn graphs of large genomes come from?" Tags="|de-bruijn-graphs|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19505" PostTypeId="1" AcceptedAnswerId="19538" CreationDate="2022-08-03T18:08:37.700" Score="2" ViewCount="62" Body="&lt;p&gt;The design matrix i have is this I would like to know how to use they way its done in deseq2 where we can use the contrast function to make particular comparison&lt;/p&gt;&#xA;&lt;p&gt;The code im running to make differential testing is this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tcgaGlm = lmFit(as.matrix(tcgaExpr), design = tcgaDesign) &#xA;tcgaGlm = eBayes(tcgaGlm)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now I will show the part of the data frame and the question Im trying to ask&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head(tcgaDesign)&#xA;             Offset CEBPA DNMT3A FAM5C FLT3_ITD FLT3_TKD IDH1 IDH2 KIT KRAS NPM1 NRAS PHF6 PTPN11 RAD21 RUNX1 SMC1A SMC3 STAG2 TET2 TP53 U2AF1 WT1&#xA;TCGA-AB-2803      1     0      0     0        0        0    0    0   0    0    0    0    0      0     0     0     0    0     0    0    0     0   0&#xA;TCGA-AB-2805      1     0      0     0        0        0    0    1   0    0    0    0    0      0     0     1     0    0     0    0    0     0   0&#xA;TCGA-AB-2806      1     0      0     0        0        0    0    0   0    0    0    0    0      0     0     0     0    0     0    0    0     0   0&#xA;TCGA-AB-2807      1     0      0     0        0        0    0    1   0    0    0    0    0      0     0     1     0    0     0    0    0     0   0&#xA;TCGA-AB-2808      1     1      0     0        0        0    0    0   0    0    0    1    0      0     0     0     0    0     0    0    0     0   0&#xA;TCGA-AB-2810      1     0      0     0        0        0    0    1   0    0    1    0    0      0     0     0     0    0     0    0    0     0   0&#xA;               complex  minus5_5q   minus7q  plus8_8q     plus21    t_15_17     t_8_21 inv16_t16_16&#xA;TCGA-AB-2803 0.0000000 0.00000000 0.0000000 0.0000000 0.00000000 1.00000000 0.00000000   0.00000000&#xA;TCGA-AB-2805 0.0000000 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.00000000   0.00000000&#xA;TCGA-AB-2806 0.0000000 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 1.00000000   0.00000000&#xA;TCGA-AB-2807 0.1410256 0.08974359 0.1217949 0.1153846 1.00000000 0.09615385 0.04487179   0.04487179&#xA;TCGA-AB-2808 0.0000000 0.00000000 0.0000000 0.0000000 0.00000000 0.00000000 0.00000000   0.00000000&#xA;TCGA-AB-2810 0.1410256 0.08974359 0.1217949 0.1153846 0.05063291 0.09615385 0.04487179   0.04487179&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here if I have to make a comparison between CEBPA vs DNMT3A mutation in samples which carries CEBPA mutation and other which carries DNMT3A what or how should I do it.&lt;/p&gt;&#xA;&lt;p&gt;To begin with I followed this &lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/limma/inst/doc/usersguide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt; I m still not sure once you run the model do i need to form a design matrix which comprises of sample which contain mutation in CEBPA and DNMT3A ? or I can go ahead with this experimental design&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dput(tcgaDesign)&#xA;structure(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, &#xA;1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, &#xA;1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, &#xA;1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, &#xA;1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, &#xA;1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, &#xA;1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, &#xA;1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, &#xA;1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, &#xA;0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, &#xA;0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 1, 0, 0, 1, 0, 1, 0, 0, &#xA;0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, &#xA;1, 0, 1, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, &#xA;1, 0, 0, 1, 0, 0, 0, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, &#xA;1, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, &#xA;0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, &#xA;1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, &#xA;0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, &#xA;0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, &#xA;1, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, &#xA;1, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, &#xA;0, 1, 1, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 0, 0, 0, &#xA;0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, &#xA;1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, &#xA;0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, &#xA;0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, &#xA;1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, &#xA;1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, &#xA;0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0, 1, 1, 1, &#xA;0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, &#xA;1, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, &#xA;0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 1, &#xA;0, 1, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, &#xA;1, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 1, 1, 1, 0, &#xA;1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, &#xA;1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 1, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, &#xA;0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, &#xA;0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, &#xA;0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, &#xA;0, 1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, &#xA;1, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, &#xA;0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, &#xA;0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, &#xA;0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, &#xA;0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0.141025641025641, &#xA;0, 0.141025641025641, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, &#xA;0, 0, 0, 0.141025641025641, 0, 0, 0, 0.141025641025641, 0, 0, &#xA;0, 1, 0, 0, 0, 0.141025641025641, 0.141025641025641, 0.141025641025641, &#xA;0, 0, 0, 0, 0.141025641025641, 0, 0, 0.141025641025641, 1, 0, &#xA;1, 0, 0, 1, 0, 0, 0, 0.141025641025641, 0, 0, 1, 0, 0, 0, 1, &#xA;0, 0, 0, 0.141025641025641, 0, 0, 0, 0, 0, 1, 0, 0, 0.141025641025641, &#xA;0, 0, 1, 0.141025641025641, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0, 0, &#xA;1, 0, 0, 0, 1, 0.141025641025641, 1, 0, 0, 1, 0, 0, 0, 0, 0, &#xA;1, 0, 0.141025641025641, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 1, &#xA;0.141025641025641, 0.141025641025641, 0, 1, 0, 1, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0.0897435897435897, 0, 0.0897435897435897, 0, 0, &#xA;1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0897435897435897, &#xA;0, 0, 0, 0.0897435897435897, 0, 0, 0, 1, 0, 0, 0, 0.0897435897435897, &#xA;0.0897435897435897, 0.0897435897435897, 0, 0, 0, 0, 0.0897435897435897, &#xA;0, 0, 0.0897435897435897, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0.0897435897435897, &#xA;0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0.0897435897435897, 0, 0, 0, 0, &#xA;0, 1, 0, 0, 0.0897435897435897, 0, 0, 0, 0.0897435897435897, &#xA;0, 0, 0, 0, 0, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0.0897435897435897, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0897435897435897, 0, 0, 0, &#xA;1, 0, 0, 0, 1, 0, 1, 0, 1, 0.0897435897435897, 0.0897435897435897, &#xA;0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.121794871794872, 0, 0.121794871794872, &#xA;0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.121794871794872, &#xA;0, 0, 0, 0.121794871794872, 0, 0, 0, 1, 0, 0, 0, 0.121794871794872, &#xA;0.121794871794872, 0.121794871794872, 0, 0, 1, 0, 0.121794871794872, &#xA;0, 0, 0.121794871794872, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0.121794871794872, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0.121794871794872, 0, 0, 0, 1, &#xA;0, 1, 0, 1, 0.121794871794872, 0, 0, 0, 0.121794871794872, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.121794871794872, &#xA;1, 0, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0.121794871794872, 0, 0, 0, &#xA;1, 0, 0, 0, 0, 0, 1, 0, 0, 0.121794871794872, 0.121794871794872, &#xA;0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0.115384615384615, 0, 0.115384615384615, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 0, 0.115384615384615, &#xA;0, 0, 0, 0.115384615384615, 0, 0, 0, 1, 0, 0, 0, 0.115384615384615, &#xA;0.115384615384615, 0.115384615384615, 0, 0, 0, 0, 0.115384615384615, &#xA;0, 0, 0.115384615384615, 0, 0, 0, 0, 0, 0, 1, 0, 1, 0.115384615384615, &#xA;0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0.115384615384615, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0.115384615384615, 0, 0, 0, 0.115384615384615, 0, &#xA;0, 1, 0, 0, 1, 0, 1, 0, 0, 0, 0, 1, 1, 0, 0, 0.115384615384615, &#xA;0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0.115384615384615, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0.115384615384615, 0.115384615384615, &#xA;0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0.0506329113924051, &#xA;0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0506329113924051, &#xA;0, 0, 0, 0.0506329113924051, 0, 0, 0, 1, 0, 0, 0, 0.0506329113924051, &#xA;0.0506329113924051, 0.0506329113924051, 0, 0, 0, 0, 0.0506329113924051, &#xA;0, 0, 0.0506329113924051, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, &#xA;1, 0, 0, 0, 0, 0, 0, 0, 0.0506329113924051, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0.0506329113924051, 0, 0, 0, 0.0506329113924051, 0, 0, &#xA;0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0.0506329113924051, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0506329113924051, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 1, 0, 0, 0, 0.0506329113924051, 0.0506329113924051, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0.0961538461538462, 0, 0.0961538461538462, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0.0961538461538462, &#xA;0, 0, 0, 0.0961538461538462, 0, 0, 0, 0, 0, 1, 1, 0.0961538461538462, &#xA;0.0961538461538462, 0.0961538461538462, 0, 0, 0, 0, 0.0961538461538462, &#xA;0, 0, 0.0961538461538462, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0961538461538462, &#xA;0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0.0961538461538462, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0.0961538461538462, 0, 0, 0, 0.0961538461538462, &#xA;0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0961538461538462, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0961538461538462, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0961538461538462, 0.0961538461538462, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 1, 1, 0, &#xA;1, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 1, 0.0448717948717949, 0, 0.0448717948717949, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0.0448717948717949, &#xA;0, 0, 0, 0.0448717948717949, 0, 0, 0, 0, 0, 0, 0, 0.0448717948717949, &#xA;0.0448717948717949, 0.0448717948717949, 0, 0, 0, 0, 0.0448717948717949, &#xA;0, 0, 0.0448717948717949, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0.0448717948717949, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0.0448717948717949, 0, 0, 0, 0, &#xA;0, 0, 1, 0, 0.0448717948717949, 0, 0, 0, 0.0448717948717949, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0448717948717949, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0448717948717949, 0, 0, 0, &#xA;0, 0, 1, 0, 0, 0, 0, 0, 0, 0.0448717948717949, 0.0448717948717949, &#xA;0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0448717948717949, 0, 0.0448717948717949, &#xA;0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0448717948717949, &#xA;0, 0, 0, 0.0448717948717949, 0, 0, 0, 0, 0, 0, 0, 0.0448717948717949, &#xA;0.0448717948717949, 0.0448717948717949, 0, 1, 0, 0, 0.0448717948717949, &#xA;0, 0, 0.0448717948717949, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0448717948717949, &#xA;0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0.0448717948717949, 0, 0, 1, 0, &#xA;0, 0, 0, 0, 0.0448717948717949, 1, 0, 0, 0.0448717948717949, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0.0448717948717949, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0.0448717948717949, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 1, 0, 0.0448717948717949, 0.0448717948717949, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, &#xA;0, 0, 0, 0, 0, 0, 0, 0, 0), .Dim = c(173L, 31L), .Dimnames = list(&#xA;    c(&amp;quot;TCGA-AB-2803&amp;quot;, &amp;quot;TCGA-AB-2805&amp;quot;, &amp;quot;TCGA-AB-2806&amp;quot;, &amp;quot;TCGA-AB-2807&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2808&amp;quot;, &amp;quot;TCGA-AB-2810&amp;quot;, &amp;quot;TCGA-AB-2811&amp;quot;, &amp;quot;TCGA-AB-2812&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2813&amp;quot;, &amp;quot;TCGA-AB-2814&amp;quot;, &amp;quot;TCGA-AB-2815&amp;quot;, &amp;quot;TCGA-AB-2816&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2817&amp;quot;, &amp;quot;TCGA-AB-2818&amp;quot;, &amp;quot;TCGA-AB-2819&amp;quot;, &amp;quot;TCGA-AB-2820&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2821&amp;quot;, &amp;quot;TCGA-AB-2822&amp;quot;, &amp;quot;TCGA-AB-2823&amp;quot;, &amp;quot;TCGA-AB-2824&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2825&amp;quot;, &amp;quot;TCGA-AB-2826&amp;quot;, &amp;quot;TCGA-AB-2828&amp;quot;, &amp;quot;TCGA-AB-2830&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2832&amp;quot;, &amp;quot;TCGA-AB-2833&amp;quot;, &amp;quot;TCGA-AB-2834&amp;quot;, &amp;quot;TCGA-AB-2835&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2836&amp;quot;, &amp;quot;TCGA-AB-2837&amp;quot;, &amp;quot;TCGA-AB-2838&amp;quot;, &amp;quot;TCGA-AB-2839&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2840&amp;quot;, &amp;quot;TCGA-AB-2841&amp;quot;, &amp;quot;TCGA-AB-2842&amp;quot;, &amp;quot;TCGA-AB-2843&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2844&amp;quot;, &amp;quot;TCGA-AB-2845&amp;quot;, &amp;quot;TCGA-AB-2846&amp;quot;, &amp;quot;TCGA-AB-2847&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2848&amp;quot;, &amp;quot;TCGA-AB-2849&amp;quot;, &amp;quot;TCGA-AB-2851&amp;quot;, &amp;quot;TCGA-AB-2853&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2854&amp;quot;, &amp;quot;TCGA-AB-2855&amp;quot;, &amp;quot;TCGA-AB-2856&amp;quot;, &amp;quot;TCGA-AB-2857&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2858&amp;quot;, &amp;quot;TCGA-AB-2859&amp;quot;, &amp;quot;TCGA-AB-2860&amp;quot;, &amp;quot;TCGA-AB-2861&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2862&amp;quot;, &amp;quot;TCGA-AB-2863&amp;quot;, &amp;quot;TCGA-AB-2865&amp;quot;, &amp;quot;TCGA-AB-2866&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2867&amp;quot;, &amp;quot;TCGA-AB-2868&amp;quot;, &amp;quot;TCGA-AB-2869&amp;quot;, &amp;quot;TCGA-AB-2870&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2871&amp;quot;, &amp;quot;TCGA-AB-2872&amp;quot;, &amp;quot;TCGA-AB-2873&amp;quot;, &amp;quot;TCGA-AB-2874&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2875&amp;quot;, &amp;quot;TCGA-AB-2877&amp;quot;, &amp;quot;TCGA-AB-2879&amp;quot;, &amp;quot;TCGA-AB-2880&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2881&amp;quot;, &amp;quot;TCGA-AB-2882&amp;quot;, &amp;quot;TCGA-AB-2884&amp;quot;, &amp;quot;TCGA-AB-2885&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2886&amp;quot;, &amp;quot;TCGA-AB-2887&amp;quot;, &amp;quot;TCGA-AB-2888&amp;quot;, &amp;quot;TCGA-AB-2889&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2890&amp;quot;, &amp;quot;TCGA-AB-2891&amp;quot;, &amp;quot;TCGA-AB-2895&amp;quot;, &amp;quot;TCGA-AB-2896&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2897&amp;quot;, &amp;quot;TCGA-AB-2898&amp;quot;, &amp;quot;TCGA-AB-2899&amp;quot;, &amp;quot;TCGA-AB-2900&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2901&amp;quot;, &amp;quot;TCGA-AB-2903&amp;quot;, &amp;quot;TCGA-AB-2904&amp;quot;, &amp;quot;TCGA-AB-2908&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2909&amp;quot;, &amp;quot;TCGA-AB-2910&amp;quot;, &amp;quot;TCGA-AB-2911&amp;quot;, &amp;quot;TCGA-AB-2912&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2913&amp;quot;, &amp;quot;TCGA-AB-2914&amp;quot;, &amp;quot;TCGA-AB-2915&amp;quot;, &amp;quot;TCGA-AB-2916&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2917&amp;quot;, &amp;quot;TCGA-AB-2918&amp;quot;, &amp;quot;TCGA-AB-2919&amp;quot;, &amp;quot;TCGA-AB-2920&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2921&amp;quot;, &amp;quot;TCGA-AB-2924&amp;quot;, &amp;quot;TCGA-AB-2925&amp;quot;, &amp;quot;TCGA-AB-2927&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2928&amp;quot;, &amp;quot;TCGA-AB-2929&amp;quot;, &amp;quot;TCGA-AB-2930&amp;quot;, &amp;quot;TCGA-AB-2931&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2932&amp;quot;, &amp;quot;TCGA-AB-2933&amp;quot;, &amp;quot;TCGA-AB-2934&amp;quot;, &amp;quot;TCGA-AB-2935&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2936&amp;quot;, &amp;quot;TCGA-AB-2937&amp;quot;, &amp;quot;TCGA-AB-2938&amp;quot;, &amp;quot;TCGA-AB-2939&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2940&amp;quot;, &amp;quot;TCGA-AB-2941&amp;quot;, &amp;quot;TCGA-AB-2942&amp;quot;, &amp;quot;TCGA-AB-2943&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2944&amp;quot;, &amp;quot;TCGA-AB-2946&amp;quot;, &amp;quot;TCGA-AB-2948&amp;quot;, &amp;quot;TCGA-AB-2949&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2950&amp;quot;, &amp;quot;TCGA-AB-2952&amp;quot;, &amp;quot;TCGA-AB-2954&amp;quot;, &amp;quot;TCGA-AB-2955&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2956&amp;quot;, &amp;quot;TCGA-AB-2959&amp;quot;, &amp;quot;TCGA-AB-2963&amp;quot;, &amp;quot;TCGA-AB-2964&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2965&amp;quot;, &amp;quot;TCGA-AB-2966&amp;quot;, &amp;quot;TCGA-AB-2967&amp;quot;, &amp;quot;TCGA-AB-2969&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2970&amp;quot;, &amp;quot;TCGA-AB-2971&amp;quot;, &amp;quot;TCGA-AB-2972&amp;quot;, &amp;quot;TCGA-AB-2973&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2975&amp;quot;, &amp;quot;TCGA-AB-2976&amp;quot;, &amp;quot;TCGA-AB-2977&amp;quot;, &amp;quot;TCGA-AB-2978&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2979&amp;quot;, &amp;quot;TCGA-AB-2980&amp;quot;, &amp;quot;TCGA-AB-2981&amp;quot;, &amp;quot;TCGA-AB-2982&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2983&amp;quot;, &amp;quot;TCGA-AB-2984&amp;quot;, &amp;quot;TCGA-AB-2985&amp;quot;, &amp;quot;TCGA-AB-2986&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2987&amp;quot;, &amp;quot;TCGA-AB-2988&amp;quot;, &amp;quot;TCGA-AB-2990&amp;quot;, &amp;quot;TCGA-AB-2991&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2992&amp;quot;, &amp;quot;TCGA-AB-2993&amp;quot;, &amp;quot;TCGA-AB-2994&amp;quot;, &amp;quot;TCGA-AB-2995&amp;quot;, &#xA;    &amp;quot;TCGA-AB-2996&amp;quot;, &amp;quot;TCGA-AB-2998&amp;quot;, &amp;quot;TCGA-AB-2999&amp;quot;, &amp;quot;TCGA-AB-3000&amp;quot;, &#xA;    &amp;quot;TCGA-AB-3001&amp;quot;, &amp;quot;TCGA-AB-3002&amp;quot;, &amp;quot;TCGA-AB-3005&amp;quot;, &amp;quot;TCGA-AB-3006&amp;quot;, &#xA;    &amp;quot;TCGA-AB-3007&amp;quot;, &amp;quot;TCGA-AB-3008&amp;quot;, &amp;quot;TCGA-AB-3009&amp;quot;, &amp;quot;TCGA-AB-3011&amp;quot;, &#xA;    &amp;quot;TCGA-AB-3012&amp;quot;), c(&amp;quot;Offset&amp;quot;, &amp;quot;CEBPA&amp;quot;, &amp;quot;DNMT3A&amp;quot;, &amp;quot;FAM5C&amp;quot;, &#xA;    &amp;quot;FLT3_ITD&amp;quot;, &amp;quot;FLT3_TKD&amp;quot;, &amp;quot;IDH1&amp;quot;, &amp;quot;IDH2&amp;quot;, &amp;quot;KIT&amp;quot;, &amp;quot;KRAS&amp;quot;, &amp;quot;NPM1&amp;quot;, &#xA;    &amp;quot;NRAS&amp;quot;, &amp;quot;PHF6&amp;quot;, &amp;quot;PTPN11&amp;quot;, &amp;quot;RAD21&amp;quot;, &amp;quot;RUNX1&amp;quot;, &amp;quot;SMC1A&amp;quot;, &amp;quot;SMC3&amp;quot;, &#xA;    &amp;quot;STAG2&amp;quot;, &amp;quot;TET2&amp;quot;, &amp;quot;TP53&amp;quot;, &amp;quot;U2AF1&amp;quot;, &amp;quot;WT1&amp;quot;, &amp;quot;complex&amp;quot;, &amp;quot;minus5_5q&amp;quot;, &#xA;    &amp;quot;minus7q&amp;quot;, &amp;quot;plus8_8q&amp;quot;, &amp;quot;plus21&amp;quot;, &amp;quot;t_15_17&amp;quot;, &amp;quot;t_8_21&amp;quot;, &amp;quot;inv16_t16_16&amp;quot;&#xA;    )))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="285" LastActivityDate="2022-08-13T14:38:22.993" Title="How to use the contrast function in ebayes for making different comparisons" Tags="|rna-seq|limma|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19506" PostTypeId="1" AcceptedAnswerId="19508" CreationDate="2022-08-03T19:33:25.690" Score="1" ViewCount="57" Body="&lt;p&gt;I discussed a question with @gaspanic &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/14625/python-biopython-replace-amino-acid-residue-on-msa-with-z-from-a-list-of-una&quot;&gt;Python/Biopython - Replace amino acid residue on MSA with &amp;quot;z&amp;quot; from a list of unaligned positions&lt;/a&gt;&#xA;. The issue emerged was speeding up lists in Python. The solutions we discussed were&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Raw code&lt;/li&gt;&#xA;&lt;li&gt;Predefining a list size&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;deque&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;1. @gaspanic&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;my code&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;2. @gaspanic&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;my code&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;3. @gaspanic&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;my code&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;3.45-3.48&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;7.17-7.2&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;4.38-4.85&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;7.09-7.23&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;3.44-3.53&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;7.21-7.39&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;Run timings are seconds (3 replicated) above with @gaspanic's code being faster.&lt;/p&gt;&#xA;&lt;p&gt;What improvements would speed this up?&lt;/p&gt;&#xA;&lt;p&gt;Here I extended the data set&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from itertools import repeat&#xA;def iterit(x):&#xA;    seq = 'M----KFLAFLCLLGFANAQ-------------D-GKCG----TLSNKSPS--------------------K'&#xA;    return ''.join(list(repeat(seq, x)))&#xA;&#xA;if __name__ == '__main__':&#xA;    seqE3 = iterit(2800) # length is 201600&#xA;    seqE4 = iterit(28000) # length is 2016000&#xA;    seqE5 = iterit(280000) # length is 20160000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;@gaspanic code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for seqbit in [seqE3,seqE4,seqE5]:&#xA;    seq = seqbit&#xA;    pos = [2, 5, 10, 11, 14]    &#xA;    new_seq = []&#xA;    count = 0&#xA;    for s in seq:&#xA;        if s == &amp;quot;-&amp;quot;:&#xA;            new_seq.append(s)&#xA;        else:&#xA;            count += 1&#xA;            if count in pos:&#xA;                new_seq.append('Z')&#xA;            else:&#xA;                new_seq.append(s)&#xA;   # print(''.join(new_seq))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# same variables and loop structure&#xA;newseq = list()&#xA;count = 0&#xA;for n,s in enumerate(prot):    &#xA;    if str(s) == '-':&#xA;        count += 1&#xA;        s = '-'&#xA;    else:&#xA;        if (n + 1 - count) in pos:&#xA;            s = 'Z'&#xA;    newseq.append(s)&#xA;# print(''.join(newseq))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Predefining lists code (additional counter needed for @gaspanic's code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# same variables and loop structure&#xA;new_seq = [] * len(seq)&#xA;count = 0&#xA;for n,s in enumerate(prot):    &#xA;    if str(s) == '-':&#xA;        count += 1&#xA;        s = '-'&#xA;    else:&#xA;        if (n + 1 - count) in pos:&#xA;            s = 'Z'&#xA;    newseq[n] = s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Deque code simply any of the above substituting&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;newseq = list() &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from collections import deque &#xA;newseq = deque()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Suggestions for efficiency improvements welcome.&lt;/p&gt;&#xA;&lt;p&gt;A solution in &lt;code&gt;Cython&lt;/code&gt; code would be cool. &lt;code&gt;PyPI&lt;/code&gt; is not sought here.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="73" LastEditDate="2022-08-04T06:17:32.120" LastActivityDate="2022-08-05T05:29:05.230" Title="Improving list speed: glycosylation example" Tags="|python|phylogenetics|phylogeny|algorithms|benchmarking|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19507" PostTypeId="1" AcceptedAnswerId="19514" CreationDate="2022-08-03T21:28:07.893" Score="1" ViewCount="71" Body="&lt;p&gt;I need a tool or a function I can use in my code (R, or Python) that I can cross reference a MOTIF against known MOTIFS, a function that will take as input a MOTIF (a probability weight matrix, PWM which is a 4 by m matrix for an m-length motif) and spit out the closest -n- known motif(s). Is there such a function/tool or something close to it? Thanks much.&lt;/p&gt;&#xA;" OwnerUserId="15741" LastActivityDate="2022-08-05T15:46:30.670" Title="Tool for cross referencing a newly found MOTIF to database of known MOTIFS" Tags="|r|rna-seq|python|motifs|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19508" PostTypeId="2" ParentId="19506" CreationDate="2022-08-03T21:34:59.983" Score="2" Body="&lt;p&gt;I'd start by saying that Python is inherently slow and this feels like a case of perhaps unnecessary optimisation. With that in mind here are my suggested (untested) optimisations of @gaspanic code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pos = {2, 5, 10, 11, 14} # make pos a set for O(1) member testing, and don't define inside the loop&#xA;for seq in (seqE3,seqE4,seqE5): # Define seq here rather than inside the loop, and use an immutable tuple instead of a mutable list  &#xA;    new_seq = []&#xA;    count = 0&#xA;    for s in seq:&#xA;        if s == &amp;quot;-&amp;quot;:&#xA;            new_seq.append(s)&#xA;        else:&#xA;            count += 1&#xA;            new_seq.append('Z' if count in pos else s) # use ternary operator&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastActivityDate="2022-08-03T21:34:59.983" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19509" PostTypeId="2" ParentId="19507" CreationDate="2022-08-04T06:51:56.020" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @m, converted from comment:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Hi @ZebraFish Motifs package Biopython&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://biopython.org/docs/1.76/api/Bio.motifs.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biopython.org/docs/1.76/api/Bio.motifs.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-08-04T06:51:56.020" CommentCount="0" CommunityOwnedDate="2022-08-04T06:51:56.020" ContentLicense="CC BY-SA 4.0" />
  <row Id="19510" PostTypeId="2" ParentId="11512" CreationDate="2022-08-04T09:26:38.473" Score="1" Body="&lt;p&gt;The logic I would us is that if they come from the same genome, coverage peaks will go in multiplications of 2 - 1n, 2n, 4n, ..., while if it's a contamination, the coverage will go wild. So, I would recommend to look at the coverage vs GC to decide where it comes from.&lt;/p&gt;&#xA;&lt;p&gt;You could either assemble the genome and use something like &lt;a href=&quot;https://blobtools.readme.io/docs&quot; rel=&quot;nofollow noreferrer&quot;&gt;Blobtools&lt;/a&gt; to check coverage, GC and putative origin of sequences (and you should do that anyway).&lt;/p&gt;&#xA;&lt;p&gt;Perhaps a bit simpler would be to look at it using k-mers, but that assumes you have reads with low error rate (In your case PacBio CCS or HiFi). If you do, you can use &lt;a href=&quot;https://kat.readthedocs.io/en/latest/walkthrough.html#detecting-gc-bias&quot; rel=&quot;nofollow noreferrer&quot;&gt;KAT&lt;/a&gt; that can do GC vs coverage plot - they intended it to look at GC bias in sequencing, but the vary same plot should show you if the three GC peaks have the same coverage or not.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2022-08-04T09:26:38.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19511" PostTypeId="1" CreationDate="2022-08-04T07:54:57.500" Score="1" ViewCount="96" Body="&lt;p&gt;I have a bunch of files in FASTQ format and need to calculate the Shannon diversity index from all of them.&lt;/p&gt;&#xA;&lt;p&gt;Is there a simple way to do that with R, some kind of step-by-step recipe?&lt;/p&gt;&#xA;&lt;p&gt;My problem is not so much finding a function for calculating the index but how to do that on FASTQ files.&lt;/p&gt;&#xA;&lt;p&gt;Grateful for every hint, Thank you!&lt;/p&gt;&#xA;" OwnerUserId="15901" OwnerDisplayName="vonjd" LastEditorUserId="73" LastEditDate="2022-08-05T06:12:21.837" LastActivityDate="2022-08-05T06:12:21.837" Title="Simple template for calculating Shannon diversity index from FASTQ files with R" Tags="|sequence-analysis|microbiome|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19512" PostTypeId="2" ParentId="19506" CreationDate="2022-08-05T02:28:03.897" Score="1" Body="&lt;p&gt;Huge thanks to @Chris_Rands, this solution increases runtime by ~ 0.5 seconds uniformly across all coded solutions, so this code has a runtime of just over 3 seconds runtime on the benchmark machine. I incorporated my solution with @Chris_Rands solution and the final runtime I got was:&lt;/p&gt;&#xA;&lt;p&gt;@gaspanic&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;1.7 - 1.8 seconds&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This being twice the fastest speed in the question and four times the speed of my raw code.&lt;/p&gt;&#xA;&lt;p&gt;My code&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;4.4 - 4.5 seconds&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However a unique combination is needed:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;tuple&lt;/code&gt; (Chris Rands)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;set&lt;/code&gt; - important (Chris Rands)&lt;/li&gt;&#xA;&lt;li&gt;an &lt;code&gt;iterator&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;a &lt;code&gt;list comprehension&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In particular 3. and 4. are needed in combination. This is explained below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def iterit(x):&#xA;    seq = 'M----KFLAFLCLLGFANAQ-------------D-GKCG----TLSNKSPS--------------------K'&#xA;    return tuple(''.join(list(repeat(seq, x))))&#xA;&#xA;def mIterit(seqE3,seqE4,seqE5):&#xA;    pos = {2, 5, 10, 11, 14}&#xA;    for seqbit in (seqE3,seqE4,seqE5):&#xA;        seq = seqbit&#xA;        count = 0&#xA;        # loc = 0&#xA;        for s in seq:&#xA;            if s == &amp;quot;-&amp;quot;:&#xA;                yield '-'&#xA;            else:&#xA;                count += 1&#xA;                if count in pos:&#xA;                    yield 'Z'&#xA;                else:&#xA;                    yield s&#xA;&#xA;f __name__ == '__main__':&#xA;    &#xA;    seqE3 = iterit(2800) # length is 201600&#xA;    seqE4 = iterit(28000) # length is 2016000&#xA;    seqE5 = iterit(280000) # length is 20160000&#xA;    &#xA;    listComp = [i for i in mIterit(seqE3,seqE4,seqE5)]&#xA;    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The list comprehension (variable &lt;code&gt;listComp&lt;/code&gt;) is the very last line and is crucial to the solution otherwise there is little increase in speed to @Chris_Rand's code. This alternative is to replace the last line with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;newseq = list()&#xA;for i in mIterit(seqE3,seqE4,seqE5):&#xA;    newseq.append(i)   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A simple:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in mIterit(seqE3,seqE4,seqE5):&#xA;    pass&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is the same speed as the list comprehension, but its of no value because the output isn't saved to anything.&lt;/p&gt;&#xA;&lt;p&gt;I was impressed by the interaction between the &lt;code&gt;iterator&lt;/code&gt; and &lt;code&gt;list comprehension&lt;/code&gt;. The other notable influence on speed is the &lt;code&gt;set&lt;/code&gt; described by @Chris_Rand.&lt;/p&gt;&#xA;&lt;p&gt;How does it work? &lt;code&gt;tuples&lt;/code&gt; do increase speed by removing some functional of a &lt;code&gt;list&lt;/code&gt; (immutability). Here the gain is small. Placing the &lt;code&gt;pos&lt;/code&gt; variable outside the loop speaks for itself (0.1 seconds gain). &lt;code&gt;set&lt;/code&gt; is interesting.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;iterator&lt;/code&gt; spews out a stream of results via &lt;code&gt;yield&lt;/code&gt; which is where it attains its speed. The problem is capturing that stream outside the &lt;code&gt;iterator&lt;/code&gt; is expensive. However &lt;code&gt;list comprehensions&lt;/code&gt; are faster than standard loops and in this scenerio carry little/no detectable overhead. I was impressed. Up to this point I thought &lt;code&gt;list comprehensions&lt;/code&gt; were a bit faster and a lot 'cooler' than loops. I hadn't understood there was a synergy with an &lt;code&gt;iterator&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Is it all worth it? &lt;code&gt;Python&lt;/code&gt; can easily achieve faster run times than &lt;code&gt;Java&lt;/code&gt; using Cython but I will place the discussion in the &lt;em&gt;chat&lt;/em&gt; forum next week.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-08-05T05:29:05.230" LastActivityDate="2022-08-05T05:29:05.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19513" PostTypeId="2" ParentId="19493" CreationDate="2022-08-05T11:05:28.227" Score="3" Body="&lt;p&gt;First of all, you need gene coordinates. You can retrieve them in GTF format from &lt;a href=&quot;https://www.gencodegenes.org/human/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GENCODE&lt;/a&gt; or other sources.&lt;/p&gt;&#xA;&lt;p&gt;Next, extract TSS and TES +/- 2kb windows from each gene, taking in mind that genes can be in the forward or reverse strand. Something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk -F&amp;quot;\t|\&amp;quot;&amp;quot; -vOFS=&amp;quot;\t&amp;quot; '$3==&amp;quot;gene&amp;quot; {&#xA;  TSS=($7==&amp;quot;+&amp;quot; ? $4-2000&amp;quot;\t&amp;quot;$4+2000 : $5-2000&amp;quot;\t&amp;quot;$5+2000)&#xA;  TES=($7==&amp;quot;+&amp;quot; ? $5-2000&amp;quot;\t&amp;quot;$5+2000 : $4-2000&amp;quot;\t&amp;quot;$4+2000)&#xA;  print $1,TSS,$10&amp;quot;_TSS_2kb&amp;quot;,0,$7&#xA;  print $1,TES,$10&amp;quot;_TES_2kb&amp;quot;,0,$7&#xA;}' annotation.gtf &amp;gt; tss_tes.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here I used GNU Awk from Bash, but you can write your own solution using Python, R or whatever you like.&lt;/p&gt;&#xA;&lt;p&gt;Finally, count reads falling in these intervals with your tool of choice (you mentioned bedtools, which is fine).&lt;/p&gt;&#xA;" OwnerUserId="2355" LastActivityDate="2022-08-05T11:05:28.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19514" PostTypeId="2" ParentId="19507" CreationDate="2022-08-05T15:46:30.670" Score="2" Body="&lt;p&gt;A popular tool for doing this is tomtom, part of the meme package.&lt;/p&gt;&#xA;&lt;p&gt;You can try it via interactive web form &lt;a href=&quot;https://meme-suite.org/meme/tools/tomtom&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Once installed locally it can be &lt;a href=&quot;https://meme-suite.org/meme/doc/tomtom.html?man_type=web&quot; rel=&quot;nofollow noreferrer&quot;&gt;run via command line&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Additionally, it is available in the Bioconductor package &lt;code&gt;memes&lt;/code&gt; as explained in &lt;a href=&quot;https://snystrom.github.io/memes-manual/articles/core_tomtom.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Motif Comparison using TomTom&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1692" LastActivityDate="2022-08-05T15:46:30.670" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19515" PostTypeId="1" AcceptedAnswerId="19525" CreationDate="2022-08-05T17:56:15.487" Score="1" ViewCount="90" Body="&lt;p&gt;&lt;a href=&quot;https://alphafold.com/entry/Q9Y499&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://alphafold.com/entry/Q9Y499&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Starting at line 918 of the downloadable mmCIF file, it looks like there might be coordinates in x, y,z:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATOM 1    N N   . TRP A 1 1   ? 12.549  -2.565  -21.811 1.0 96.06 ? 1   TRP A N   1 Q9Y499 UNP 1   W &#xA;ATOM 2    C CA  . TRP A 1 1   ? 12.477  -2.290  -20.365 1.0 96.06 ? 1   TRP A CA  1 Q9Y499 UNP 1   W &#xA;ATOM 3    C C   . TRP A 1 1   ? 11.221  -2.928  -19.812 1.0 96.06 ? 1   TRP A C   1 Q9Y499 UNP 1   W &#xA;ATOM 4    C CB  . TRP A 1 1   ? 12.469  -0.785  -20.103 1.0 96.06 ? 1   TRP A CB  1 Q9Y499 UNP 1   W &#xA;ATOM 5    O O   . TRP A 1 1   ? 10.206  -2.922  -20.501 1.0 96.06 ? 1   TRP A O   1 Q9Y499 UNP 1   W &#xA;ATOM 6    C CG  . TRP A 1 1   ? 13.795  -0.114  -20.270 1.0 96.06 ? 1   TRP A CG  1 Q9Y499 UNP 1   W &#xA;ATOM 7    C CD1 . TRP A 1 1   ? 14.286  0.508   -21.369 1.0 96.06 ? 1   TRP A CD1 1 Q9Y499 UNP 1   W &#xA;ATOM 8    C CD2 . TRP A 1 1   ? 14.791  0.098   -19.228 1.0 96.06 ? 1   TRP A CD2 1 Q9Y499 UNP 1   W &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am a bioinformatics noob (took an intro class in my undergrad CS degree at a Canadian university) but know some Unity game engine with which I want to build a standalone visualization tool for users with VR headsets.&lt;/p&gt;&#xA;" OwnerUserId="9650" LastEditorUserId="9650" LastEditDate="2022-08-07T21:03:21.067" LastActivityDate="2022-08-08T09:42:15.523" Title="How do I read mmCIF files from Alphafold DB as coordinates to use in replicating their visualization for Unity VR?" Tags="|proteins|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19516" PostTypeId="2" ParentId="17977" CreationDate="2022-08-06T14:12:30.873" Score="2" Body="&lt;p&gt;The Python solution is here. IMO Bash definitely isn't a good solution for this calculation.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from itertools import permutations, combinations&#xA;&#xA;one2all ={'A': ('A', 'ALA', 'alanine'),&#xA;          'R': ('R', 'ARG', 'arginine'),&#xA;          'N': ('N', 'ASN', 'asparagine'),&#xA;          'D': ('D', 'ASP', 'aspartic acid'),&#xA;          'C': ('C', 'CYS', 'cysteine'),&#xA;          'Q': ('Q', 'GLN', 'glutamine'),&#xA;          'E': ('E', 'GLU', 'glutamic acid'),&#xA;          'G': ('G', 'GLY', 'glycine'),&#xA;          'H': ('H', 'HIS', 'histidine'),&#xA;          'I': ('I', 'ILE', 'isoleucine'),&#xA;          'L': ('L', 'LEU', 'leucine'),&#xA;          'K': ('K', 'LYS', 'lysine'),&#xA;          'M': ('M', 'MET', 'methionine'),&#xA;          'F': ('F', 'PHE', 'phenylalanine'),&#xA;          'P': ('P', 'PRO', 'proline'),&#xA;          'S': ('S', 'SER', 'serine'),&#xA;          'T': ('T', 'THR', 'threonine'),&#xA;          'W': ('W', 'TRP', 'tryptophan'),&#xA;          'Y': ('Y', 'TYR', 'tyrosine'),&#xA;          'V': ('V', 'VAL', 'valine'),&#xA;          'X': ('X', 'GLX', 'glutaminx'),&#xA;          'Z': ('Z', 'GLI', 'glycine'),&#xA;          'J': ('J', 'NLE', 'norleucine'),&#xA;          'U': ('U', 'CYC', 'cysteinc')}&#xA;myprotein = [x for x in one2all.keys()]        &#xA;tetrapeps = combinations(myprotein,4)&#xA;peptide = []&#xA;count = 0&#xA;for x in list(tetrapeps):&#xA;    peptide.append(''.join(x))&#xA;for tetra in peptide:    &#xA;    permpeptides = list(permutations (tetra))&#xA;    for i in permpeptides:&#xA;        count +=1&#xA;        print (''.join(i))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get 255024 permutations so more than the estimate of 160000, however the code is correct (please see &lt;strong&gt;Validation&lt;/strong&gt; below).&lt;/p&gt;&#xA;&lt;p&gt;Truncated output&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;....&#xA;JUXV&#xA;UVXJ&#xA;UVJX&#xA;UXVJ&#xA;UXJV&#xA;UJVX&#xA;UJXV&#xA;VZJU&#xA;VZUJ&#xA;VJZU&#xA;VJUZ&#xA;VUZJ&#xA;VUJZ&#xA;ZVJU&#xA;ZVUJ&#xA;ZJVU&#xA;ZJUV&#xA;ZUVJ&#xA;ZUJV&#xA;JVZU&#xA;JVUZ&#xA;JZVU&#xA;JZUV&#xA;JUVZ&#xA;JUZV&#xA;UVZJ&#xA;UVJZ&#xA;UZVJ&#xA;UZJV&#xA;UJVZ&#xA;UJZV&#xA;XZJU&#xA;XZUJ&#xA;XJZU&#xA;XJUZ&#xA;XUZJ&#xA;XUJZ&#xA;ZXJU&#xA;ZXUJ&#xA;ZJXU&#xA;ZJUX&#xA;ZUXJ&#xA;ZUJX&#xA;JXZU&#xA;JXUZ&#xA;JZXU&#xA;JZUX&#xA;JUXZ&#xA;JUZX&#xA;UXZJ&#xA;UXJZ&#xA;UZXJ&#xA;UZJX&#xA;UJXZ&#xA;UJZX&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I cut and paste the amino dictionary from &lt;a href=&quot;http://aria.pasteur.fr/example-files/modified-residues/aria-files/AminoAcid.py/view&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; (I wasn't going to write it by hand), &lt;code&gt;AminoAcid.py&lt;/code&gt; by Pasteur.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Validation&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The theoretical number of permutations from 4 samples of 20 amino acids is&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;116280&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;However, the amino acid chart I used comprised &lt;strong&gt;24&lt;/strong&gt; amino acids (I just copied and pasted it). The theoretical number for 24 amino acids is therefore:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;255024&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This was the number observed in the output of the above code and validated the calculation.&lt;/p&gt;&#xA;&lt;p&gt;A permutations calutation can be performed online &lt;a href=&quot;https://www.calculatorsoup.com/calculators/discretemathematics/permutations.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am not sure how the expected output of 160000 was derived but suspect it was a guess.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The permutation equation is as follows, the output of this equation and the code above coincide,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$ P_{n, r}=\frac{n !}{(n-r) !}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;n = no. amino acids&lt;/p&gt;&#xA;&lt;p&gt;r = size of oligopeptide&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h4&gt;Permutation by repetition&lt;/h4&gt;&#xA;&lt;p&gt;If the permutation by repetition equation  is sought (comment below). The code is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from itertools import product&#xA;myprotein = [x for x in one2all.keys()]&#xA;print(len([p for p in product(myprotein, repeat=4)]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;for 20 amino acids this is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;160000&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;for 24 amino acids:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;331776&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-08-07T02:09:01.507" LastActivityDate="2022-08-07T02:09:01.507" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19517" PostTypeId="2" ParentId="17977" CreationDate="2022-08-06T18:09:44.563" Score="1" Body="&lt;p&gt;Here's a bash solution using brace expansion and the &lt;a href=&quot;https://biology.stackexchange.com/questions/105321&quot;&gt;single-letter codes for the 20 common amino acids&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;aalist=D,E,K,R,H,N,Q,S,T,P,G,A,V,I,L,M,C,F,Y,W&#xA;eval printf '&amp;quot;%s\n&amp;quot;' {&amp;quot;$aalist&amp;quot;}{&amp;quot;$aalist&amp;quot;}{&amp;quot;$aalist&amp;quot;}{&amp;quot;$aalist&amp;quot;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This prints 160,000 lines, one for each tetrapeptide.&lt;/p&gt;&#xA;" OwnerUserId="3967" LastActivityDate="2022-08-06T18:09:44.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19518" PostTypeId="1" AcceptedAnswerId="19519" CreationDate="2022-08-06T18:41:30.133" Score="1" ViewCount="80" Body="&lt;h1&gt;&lt;/h1&gt;&#xA;&lt;p&gt;I wrote simple Python script which has an excessive run time, notably when using large data sets. For example a data set of 1 000 000 sequences.&lt;/p&gt;&#xA;&lt;p&gt;I am seeking assistance refactoring the code and would be very grateful for any assistance.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;import re&#xA;&#xA;def move_item(old, new):&#xA;    k, v = old.popitem()&#xA;    new[k] = v&#xA;&#xA;def map_ids_to_aa_sequence_list(id_):&#xA;    id_ = id_.strip()&#xA;    for s in aa:&#xA;        if re.search(id_,s, re.I):&#xA;            return s&#xA;&#xA;file = open(r'list_of_ids', 'r').readlines()&#xA;names = [for name.strip() in file if len(name) &amp;gt; 1]&#xA;        &#xA;handle = open('sequences.fasta','r')&#xA;sequences = list(SeqIO.parse(handle, 'fasta'))&#xA;&#xA;seq_dict = {}&#xA;&#xA;for s in sequences:&#xA;    seq_dict[str(s.description)] = str(s.seq)&#xA;    &#xA;aa = []&#xA;&#xA;for k, v in seq_dict.items():&#xA;    sequence = '&amp;gt;' + k +'\n' + v + '\n'&#xA;    aa.append(sequence)&#xA;    &#xA;separated_aminoacid_sequences_set = set(map(map_ids_to_aa_sequence_list, names))&#xA;&#xA;separated_aminoacid_sequences_list = list(separated_aminoacid_sequences_set)&#xA;    &#xA;with open('separated_sequences.fasta','w') as fs:&#xA;    for s in separated_aminoacid_sequences_list:&#xA;        fs.write(str(s))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1224" LastEditorUserId="73" LastEditDate="2022-08-07T07:34:59.693" LastActivityDate="2022-09-06T08:01:20.120" Title="Refactoring a script mapping ID to sequence" Tags="|python|algorithms|benchmarking|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19519" PostTypeId="2" ParentId="19518" CreationDate="2022-08-06T23:42:58.050" Score="2" Body="&lt;p&gt;This is beyond a single answer. 'Refactoring' (code efficiency, high performance code) is a huge topic. Like its own project.&lt;/p&gt;&#xA;&lt;p&gt;The first thing I would personally absolutely target first is &lt;code&gt;re&lt;/code&gt; here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    if re.search(id_,s, re.I):&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Python is not Perl and the &lt;code&gt;re&lt;/code&gt; is EXTREMELY slow and in particular using &lt;code&gt;search&lt;/code&gt; is especially slow, most notably when uncompiled (see notes and point 3 below).&lt;/p&gt;&#xA;&lt;h3&gt;How to avoid &lt;code&gt;re.search&lt;/code&gt;&lt;/h3&gt;&#xA;&lt;p&gt;Could you revise this to a direct &lt;code&gt;C&lt;/code&gt; style operation? This is the best solution if you get it working.&#xA;For example,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Using &lt;code&gt;==&lt;/code&gt;:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;if str(id_) == str(s):&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;if the case is important (like they're different) then...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;if str(id_).lower() == str(s).lower():&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You could dump the &lt;code&gt;str()&lt;/code&gt; it ain't doing much its just bug insurance.&lt;/p&gt;&#xA;&lt;p&gt;The other approach if you want to stick with &lt;code&gt;re&lt;/code&gt; is to switch to &lt;code&gt;match&lt;/code&gt;, that will be quicker.&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Using &lt;code&gt;match&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;if re.match(id_, s, re.I):&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;match&lt;/code&gt; anchors  the search at the first character, i.e. id_[0], &lt;code&gt;search&lt;/code&gt; goes everywhere. &lt;code&gt;match&lt;/code&gt; is faster.&lt;/p&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Sticking with &lt;code&gt;search&lt;/code&gt; but combining with compile:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;id_ = id_.strip()&#xA;x = re.compile (id_) &#xA;if x.search(s, re.I): # don't use re. x replaces it&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;id_ = id_.strip()&#xA;x = re.compile (id_)&#xA;if x.match(s, re.I):&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have not put these in order of preference. My preference is for 1. but it will risk throwing logical errors (missing stuff).&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;If you are using &lt;code&gt;re&lt;/code&gt; you must compile the search term&lt;/em&gt;. Its a pain but whilst compiled &lt;code&gt;re&lt;/code&gt; is slow, uncompiled &lt;code&gt;re&lt;/code&gt; is really, really slow. Disclaimer, I'm from a Perl background and from this perspective &lt;code&gt;re&lt;/code&gt; is the pits because previously relied on the efficient regex (Perl).&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;==&lt;/code&gt; and &lt;code&gt;match&lt;/code&gt; could fall over if &lt;code&gt;&amp;gt;&lt;/code&gt; is in one variable and not in the other, but the &lt;code&gt;search&lt;/code&gt; will return the correct 'match' (i.e. not the output of 'match') at the cost of algorithm efficiency.&lt;/p&gt;&#xA;&lt;p&gt;This isn't an exhaustive analysis of &lt;code&gt;re&lt;/code&gt; but thats where I would start.&lt;/p&gt;&#xA;&lt;p&gt;Please note this is a starting point from prior experience to pick a 'big issues'. It is useful for future, particularly for questions, to place time checks across the code to identify the bottlenecks.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The thing we don't know is what the fasta headers look like.&lt;/li&gt;&#xA;&lt;li&gt;For every query in &lt;code&gt;search&lt;/code&gt; there are &lt;span class=&quot;math-container&quot;&gt;$1.10^6$&lt;/span&gt; fasta headers. So if you've &lt;span class=&quot;math-container&quot;&gt;$1.10^5$&lt;/span&gt; queries thats &lt;span class=&quot;math-container&quot;&gt;$1.10^{11}$&lt;/span&gt; searches for &lt;code&gt;search&lt;/code&gt; thats why it can be taking up a lot of time.&lt;/li&gt;&#xA;&lt;li&gt;Rewriting the code using Biopython objects throughout is good practice. &lt;code&gt;SeqIO&lt;/code&gt; in particular are cool. &lt;code&gt;AlignIO&lt;/code&gt; are algorithmically expensive however.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-08-07T03:47:08.363" LastActivityDate="2022-08-07T03:47:08.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19520" PostTypeId="5" CreationDate="2022-08-07T07:36:20.240" Score="0" Body="" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-08-07T07:36:20.240" LastActivityDate="2022-08-07T07:36:20.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19521" PostTypeId="4" CreationDate="2022-08-07T07:36:20.240" Score="0" Body="Benchmarking problems relate to problems where efficiency is key, beyond just solving a problem itself. What's the fastest or most efficient way to solve a problem?" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-08-07T07:36:20.240" LastActivityDate="2022-08-07T07:36:20.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19524" PostTypeId="2" ParentId="19494" CreationDate="2022-08-07T17:27:26.827" Score="1" Body="&lt;p&gt;You can do this manually by looking up the protein variant on &lt;a href=&quot;https://varsome.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://varsome.com&lt;/a&gt;. For your example, that would mean searching for &lt;code&gt;KRAS G12V&lt;/code&gt; (change &lt;code&gt;V&lt;/code&gt; for anything else, the idea is to give an HGVS protein notation for a variant, so any single letter amino acid code would work). That would take you to this page &lt;a href=&quot;https://varsome.com/variant/hg19/KRAS%20G12V?annotation-mode=germline&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://varsome.com/variant/hg19/KRAS%20G12V?annotation-mode=germline&lt;/a&gt;, which includes:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/P590B.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/P590B.png&quot; alt=&quot;screenshot of varsome.com&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Of course, I imagine you want to automate this. VarSome can help there too, through our API, but that is a paid service. If you have the option of paying for this, the VarSome API can be used to (among other things) look up arbitrary variants and translate between different notations, so you could easily adapt this to get the information you need.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Disclosure&lt;/strong&gt;&lt;/em&gt;: I work for the company behind VarSome.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2022-08-07T17:27:26.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19525" PostTypeId="2" ParentId="19515" CreationDate="2022-08-08T09:42:15.523" Score="3" Body="&lt;h2&gt;Don't&lt;/h2&gt;&#xA;&lt;p&gt;A protein is not only list of atomic coordinates. Bar for a proof of concept (i.e. learn to code in Unity VR), it is an endless spiral of hurt in a field where there is a lot of competition and were you to push a product you'd need to invest more manhours in publicity than in coding...&lt;/p&gt;&#xA;&lt;h2&gt;Connectivity&lt;/h2&gt;&#xA;&lt;p&gt;Here is the short of it. Chemistry is classically a graph network: nodes=atoms, edges=bonds. The connectivity of the atoms is dictated by their residue's 3-letter name, which is either&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;assumed —software know the 20 AAs (and several modifications, eg. &lt;code&gt;SEP&lt;/code&gt;, 4x2 bases and some ligands, e.g. &lt;code&gt;HOH&lt;/code&gt;— or&lt;/li&gt;&#xA;&lt;li&gt;a special entry in the PDB or mmCIF file —in PDB it's a CONECT entry.&#xA;E.g. &lt;a href=&quot;https://www.rcsb.org/ligand/CFF&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/ligand/CFF&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;AlphaFold2 does not have ligands, AlphaFill does. But scientists (not PR specialists) use PDB entries or run their own modelling. AlphaFold2 from EBI is rarely used: oligomers with ligands is where it's at. Especially if doing a drug discovery campaign (and red biotech can and does afford VR headsets).&lt;/p&gt;&#xA;&lt;h2&gt;Format&lt;/h2&gt;&#xA;&lt;p&gt;mmCIF is a deposition standard, PDB is the workhorse standard. mmTF is a cool idea that is not catching on (cf XKCD comic strip about standard proliferation...). Using PDB is easier: &lt;a href=&quot;https://www.wwpdb.org/documentation/file-format-content/format33/v3.3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.wwpdb.org/documentation/file-format-content/format33/v3.3.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Representation&lt;/h2&gt;&#xA;&lt;p&gt;Problems arise when offering representations —everyone has their favourite. See NGL gallery for an array: &lt;a href=&quot;https://nglviewer.org/ngl/gallery/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nglviewer.org/ngl/gallery/index.html&lt;/a&gt;&#xA;Surface display has additional issues —PyMOL simply and openly reuses the Advance Poisson-Boltzmann Solvation tool as it's tricky.&lt;/p&gt;&#xA;&lt;h2&gt;Dragons in the minefield&lt;/h2&gt;&#xA;&lt;p&gt;In a PDB entry not from AF2, you can have disulfides (&lt;code&gt;SSBOND&lt;/code&gt; and/or &lt;code&gt;CIZ&lt;/code&gt; entry), isopeptide bonds (&lt;code&gt;LINK&lt;/code&gt;), missing atoms, the &lt;code&gt;UNK&lt;/code&gt;, &lt;code&gt;UNX&lt;/code&gt; and &lt;code&gt;UNL&lt;/code&gt; residues, alphatraces, insertion sequences, alt. occupancy, gaps, non-standard usages of all of these etc etc. and my favourite implied proximity bonding.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2022-08-08T09:42:15.523" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="19526" PostTypeId="1" CreationDate="2022-08-09T16:04:20.187" Score="2" ViewCount="174" Body="&lt;p&gt;I am now trying to conduct batch effect correction for my data matrix, whose samples (human) have different gender and come from different sequencing platforms. I have tried &lt;code&gt;Combat&lt;/code&gt;, &lt;code&gt;Combat-seq&lt;/code&gt;, and &lt;code&gt;limma&lt;/code&gt;. However, the outcomes are bad. It seems that &lt;code&gt;Combat-seq&lt;/code&gt; is based on Negative binomial regression models, &lt;code&gt;Combat&lt;/code&gt; is based on Gaussian distribution, and &lt;code&gt;limma&lt;/code&gt; is based on the linear model. I wonder if my data violates their assumption, so I check my data and get this outcome.&lt;a href=&quot;https://i.stack.imgur.com/51MbG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/51MbG.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt; It seems that my data follows Gamma distribution. I am wondering what I could do in this situation.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The data I am working on is DNAse-seq data, which is a matrix with 16 samples. As the samples come from different platforms, I would like to correct the batch effect. I am also considering gender information as there are male and female samples. To sum up, my questions are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Do we need to consider the distribution of the data when we apply the batch correction (limma, combat, and combat-seq)?&lt;/li&gt;&#xA;&lt;li&gt;If we need to consider the distribution, which method do I need to correct the data which follows gamma distribution?&lt;/li&gt;&#xA;&lt;li&gt;Are there methods designed for read counts? (I know there are many methods designed for surat objects, but do we have others that are more general?)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thank you very much for all kinds of help.&lt;/p&gt;&#xA;" OwnerUserId="15740" LastEditorUserId="73" LastEditDate="2022-08-10T07:22:16.383" LastActivityDate="2023-09-04T11:05:25.650" Title="Problems in conducting batch effect correction" Tags="|limma|batch-effects|combat|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19527" PostTypeId="2" ParentId="19526" CreationDate="2022-08-09T22:30:17.490" Score="1" Body="&lt;p&gt;The ComBat-seq method requires raw counts which can be modelled as a negative binomial. The other methods are typically used with normalized counts transformed to log2 scale. Obviously for any method that tries to regress a factor out of a dataset you have to make sure that none of the experimental variables are confounded with the batch variables that you want to regress. More details can be given if you edit the post to say what the data are, how the experimental setup is and what the factors to regress are.&lt;/p&gt;&#xA;" OwnerUserId="15917" LastEditorUserId="73" LastEditDate="2022-08-10T09:42:05.613" LastActivityDate="2022-08-10T09:42:05.613" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19528" PostTypeId="1" CreationDate="2022-08-10T14:41:52.113" Score="3" ViewCount="186" Body="&lt;p&gt;I have about 750 vcf files and I am looking to generate a Polygenic Risk Score for each individual. I am planning to use either PLINK or LDPhred to generate a Risk Score. I have phenotype information of these individuals.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone verify if the polygenic risk score is calculated using individual vcf files or should I look into combining the vcf files?&lt;/p&gt;&#xA;" OwnerUserId="15921" LastActivityDate="2024-03-11T02:08:00.947" Title="Polygenic Risk score calculation using vcf files" Tags="|vcf|plink|polygenic-risk-score|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19529" PostTypeId="1" AcceptedAnswerId="19553" CreationDate="2022-08-10T17:19:18.973" Score="1" ViewCount="57" Body="&lt;p&gt;I'm working with ATACSeq data from multiple tissue/cell types. The data is binned in 1 megabase bins. I'd like to identify the bins that are &amp;quot;highly variable&amp;quot; across the different tissue types. I can't seem to find a tool to do this, like there exists for identifying highly variables genes in a gene expression setting (e.g Scanpy's &lt;code&gt;scanpy.pp.highly_variable_genes&lt;/code&gt;). I was wondering if I could use the gene selection tools for my data, and if there are any other suggestions?&lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2022-08-16T14:09:34.267" Title="Can I use a tool for highly variable gene selection for selecting highly variable &quot;bins&quot; from an ATACSeq dataset?" Tags="|atac-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19530" PostTypeId="2" ParentId="13013" CreationDate="2022-08-10T19:19:28.507" Score="0" Body="&lt;p&gt;I just saw this question. The solution to your problem is &lt;a href=&quot;https://github.com/satijalab/seurat/issues/2769#issuecomment-1211152502&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6504" LastActivityDate="2022-08-10T19:19:28.507" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19531" PostTypeId="1" CreationDate="2022-08-10T19:43:25.700" Score="1" ViewCount="2526" Body="&lt;p&gt;I am looking for a way to present some single-cell RNA sequencing data on expression of a certain gene. My dataset has 2 variables, cell type and condition. I'm looking to create a grouped aligned scatter plot bar graph of the expression of the gene, where the y-axis is expression levels, while X axis is the different cell types, with each cell type having 2 bars (1 for healthy, 1 for diseased). My dataset has 3 healthy and 3 diseased samples, but all of the data is integrated into a Seurat object. To first create an aligned scatter plot bar graph, what I did was generate a DotPlot for the expression of gene X in each sample, split by cell-type. Then I realized that DotPlot() prints out a data table with values including avg.exp, pct.exp and avg.scaled.exp. I used the avg.exp metric to plot the data, but I was posting this question to see if there are other ways I can access and present expression data from a Seurat object. Is the way I've presented accurate? I've posted a sample bar graph of what I'm looking for, and the main question is how I can gather the expression levels across cell types AND diseased or healthy condition.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&#xA;&lt;a href=&quot;https://i.stack.imgur.com/xqVRw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xqVRw.png&quot; alt=&quot;Example Bar Graph&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15922" LastActivityDate="2024-02-15T15:33:02.407" Title="Bar Graph of Expression Data from Seurat Object" Tags="|r|scrnaseq|seurat|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19532" PostTypeId="1" CreationDate="2022-08-10T20:46:23.037" Score="1" ViewCount="100" Body="&lt;p&gt;I have assembled the transcriptome a plant species using Trinity.&lt;/p&gt;&#xA;&lt;p&gt;Here is the command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Trinity --samples_file ./sample_file.txt --seqType fq --max_memory 80G --CPU 30 --output ./trinity_outdir&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried to get transcript abundance using the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/home/programs/build/trinityrnaseq/util/align_and_estimate_abundance.pl \&#xA;  --transcripts ./Trinity.fasta \&#xA;  --seqType fq \&#xA;  --samples_file ./sample_file.txt \&#xA;  --thread_count 6 \&#xA;  --gene_trans_map ./trinity_outdir.Trinity.fasta.gene_trans_map \&#xA;  --est_method RSEM \&#xA;  --aln_method bowtie2 \&#xA;  --prep_reference \&#xA;  --coordsort_bam \&#xA;  --output_dir /rsem_outdir/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And I keep getting the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[E::sam_hrecs_update_hashes] Duplicate entry &amp;quot;TRINITY....&amp;quot; in sam header &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;samtools view: failed to add PG line to the header&lt;/p&gt;&#xA;&lt;p&gt;I will appreciate your suggestions.&lt;/p&gt;&#xA;" OwnerUserId="15923" LastEditorUserId="650" LastEditDate="2022-08-10T22:04:16.777" LastActivityDate="2022-08-10T22:04:16.777" Title="Trinity de novo transcriptome assembly: samtools view failed to add PG line to the header" Tags="|samtools|transcriptome|trinity|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19533" PostTypeId="1" CreationDate="2022-08-12T16:24:57.647" Score="2" ViewCount="4021" Body="&lt;p&gt;I was trying to do subsetting via:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pbmc = subset(pbmc, subset = nFeature &amp;gt; 1000 &amp;amp; nCount &amp;gt; 4000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;R returns:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in FetchData.Seurat(object = object, vars = unique(x = expr.char[vars.use]), :&#xA;None of the requested variables were found:&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and then nothing, so it says cannot find requested variables and doesn't tell me what's missing...&lt;/p&gt;&#xA;&lt;p&gt;I'm very confused. Thank you so much for your help!&lt;/p&gt;&#xA;" OwnerUserId="15927" LastEditorUserId="73" LastEditDate="2022-08-13T11:09:19.223" LastActivityDate="2023-06-28T23:34:57.817" Title="Seurat: Error in FetchData.Seurat(object = object, vars = unique(x = expr.char[vars.use]), : None of the requested variables were found:" Tags="|r|seurat|subset|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19534" PostTypeId="2" ParentId="14790" CreationDate="2022-08-13T05:20:24.507" Score="1" Body="&lt;p&gt;Try this command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$AMBERHOME/bin/nab -v lr14.nab -o ./a.out, &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;then execute a.out separately using&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./a.out. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You will get the &lt;code&gt;pdb&lt;/code&gt; file of nab extension.&lt;/p&gt;&#xA;" OwnerUserId="15929" LastEditorUserId="3848" LastEditDate="2022-08-13T14:40:29.273" LastActivityDate="2022-08-13T14:40:29.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19535" PostTypeId="2" ParentId="19533" CreationDate="2022-08-13T11:08:38.133" Score="0" Body="&lt;p&gt;From the steps in the &lt;a href=&quot;https://satijalab.org/seurat/archive/v3.0/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PBMC_3k tutorial&lt;/a&gt;, it looks like you need to suffix the name with the assay as well, separated by an underscore (in this case, &lt;code&gt;RNA&lt;/code&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pbmc &amp;lt;- subset(pbmc, subset = (nFeature_RNA &amp;gt; 1000) &amp;amp; (nCount_RNA &amp;gt; 4000))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-08-13T11:08:38.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19536" PostTypeId="2" ParentId="19533" CreationDate="2022-08-13T12:07:55.640" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Error in FetchData.Seurat(object = object, vars = unique(x = expr.char[vars.use]), :&#xA;None of the requested variables were found:&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As the error is pointing out here, you should check your variable names that you have used here. It must match the variable names in the pbmc object that you want to subset.&lt;/p&gt;&#xA;" OwnerUserId="15931" LastEditorUserId="3848" LastEditDate="2022-08-13T14:34:51.083" LastActivityDate="2022-08-13T14:34:51.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19537" PostTypeId="2" ParentId="19223" CreationDate="2022-08-13T12:37:19.343" Score="1" Body="&lt;p&gt;This question can be addressed by gene set enrichment analysis. You should use Gene Ontology and KEGG Pathways to find out which genes from your gene set are enriched in GO terms and KEGG pathway terms. You can use &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/clusterProfiler.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;clusterProfiler&lt;/a&gt; package in R for this anlysis and for visualization also. Here is the link to the guide:&#xA;&lt;a href=&quot;https://yulab-smu.top/biomedical-knowledge-mining-book/enrichment-overview.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://yulab-smu.top/biomedical-knowledge-mining-book/enrichment-overview.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15931" LastEditorUserId="15931" LastEditDate="2022-08-14T19:15:13.763" LastActivityDate="2022-08-14T19:15:13.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19538" PostTypeId="2" ParentId="19505" CreationDate="2022-08-13T13:19:36.383" Score="2" Body="&lt;p&gt;A design matrix is a matrix of values of the grouping variable.&lt;/p&gt;&#xA;&lt;p&gt;ANOVA needs such a matrix to know which samples belong to which group. Since &lt;code&gt;limma&lt;/code&gt; performs an ANOVA it needs this design matrix.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You can create it using the &lt;code&gt;model.matrix()&lt;/code&gt; method.&lt;/li&gt;&#xA;&lt;li&gt;Afterwards you can tell &lt;code&gt;limma&lt;/code&gt; which groups you want to compare.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;For point 2 you define a contrast matrix defining the comparisons of interest by using the &lt;code&gt;makeContrasts()&lt;/code&gt; method.&lt;/p&gt;&#xA;&lt;p&gt;It is well explained in this &lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/limma/inst/doc/usersguide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual's&lt;/a&gt; section 9.2&lt;/p&gt;&#xA;" OwnerUserId="15931" LastEditorUserId="3848" LastEditDate="2022-08-13T14:38:22.993" LastActivityDate="2022-08-13T14:38:22.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19539" PostTypeId="2" ParentId="19531" CreationDate="2022-08-13T17:29:21.230" Score="1" Body="&lt;p&gt;In single-cell data a bargraph with error bar is not usually shown since the gene expression does not have a normal distribution due to drop-outs. A violin-plot is commonly used. In your case, first set the active.ident to cell types&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seuratObj &amp;lt;- SetIdent(seuratObj , value = &amp;quot;Cell.Types&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then call the violin plot&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;VlnPlot(seuratObj , features = &amp;quot;geneX&amp;quot;, split.by = &amp;quot;condition&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you still specifically need bargraph, you can get the output of the violin plot as a dataframe and then draw the barplot.&lt;/p&gt;&#xA;" OwnerUserId="8550" LastActivityDate="2022-08-13T17:29:21.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19540" PostTypeId="1" CreationDate="2022-08-14T01:38:00.577" Score="3" ViewCount="2249" Body="&lt;p&gt;What is the fundamental difference between STAR and Bowtie(2). Specifically, what is the difference in their final output (regardless of run-time differences, speed, memory usage etc.). Both seem to work on the same kind of input *.fastq files without discriminating what they are associated with and generate an output version of the *.fastq files. Would my downstream analysis be incorrect if I use Bowtie2 where STAR was supposed to be used, and vice versa? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="15741" LastEditorUserId="15741" LastEditDate="2022-08-14T17:48:15.427" LastActivityDate="2022-08-14T17:48:15.427" Title="STAR vs Bowtie2" Tags="|sequence-alignment|bowtie2|star|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19541" PostTypeId="2" ParentId="19540" CreationDate="2022-08-14T04:02:29.477" Score="3" Body="&lt;p&gt;The seeds are clustered together by proximity to a selected set of ‘anchor’ seeds in the 2nd phase of the STAR algorithm,  but this •clustering/stitching/scoring process• is not present in Bowtie2.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/izJSN.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/izJSN.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://drive.google.com/file/d/1l8dAS9fA3wpPiExRXGIJ4aE4gJltl_1A/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;mapping by performing a two-step process&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15519" LastActivityDate="2022-08-14T04:02:29.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19542" PostTypeId="2" ParentId="19540" CreationDate="2022-08-14T04:48:09.160" Score="6" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;What is the fundamental difference between STAR and Bowtie(2).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;STAR for mapping spliced (i.e. with introns) short RNA-seq reads against a genome. Bowtie2 for mapping short reads without splicing.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Would my downstream analysis be incorrect if I use Bowtie2 where STAR was supposed to be used (?)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes. If you use bowtie2 to map spliced short reads against a genome, STAR-based downstream analysis is likely to be wrong.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;... and vice versa?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If you map genomic reads with STAR, the result will be suboptimal. Use the right tool for the right task.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2022-08-14T04:48:09.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19544" PostTypeId="2" ParentId="19458" CreationDate="2022-08-14T20:32:22.837" Score="1" Body="&lt;p&gt;check if &lt;code&gt;Pts&lt;/code&gt; and &lt;code&gt;class&lt;/code&gt; are of &lt;code&gt;factor&lt;/code&gt; class or not by using&#xA;&lt;code&gt;is.factor(coldata$Pts)&lt;/code&gt; and &lt;code&gt;is.factor(coldata$class)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;if it is not then convert it into &lt;code&gt;factor&lt;/code&gt; by&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;coldata$Pts = factor(coldata$Pts)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;coldata$class = factor(coldata$class)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then try to design your matrix (A design matrix is a matrix of values of the grouping variable).&#xA;I think this would work as it works with&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;design &amp;lt;- model.matrix(~0+coldata$Pts+coldata$class)&lt;/code&gt; in &lt;code&gt;limma&lt;/code&gt; package.&lt;/p&gt;&#xA;&lt;p&gt;You can also check in &lt;a href=&quot;https://master.bioconductor.org/packages/release/workflows/vignettes/rnaseqGene/inst/doc/rnaseqGene.html#the-deseqdataset-object-sample-information-and-the-design-formula&quot; rel=&quot;nofollow noreferrer&quot;&gt;this RNA-seq analysis workflow&lt;/a&gt; of &lt;code&gt;DESeq2&lt;/code&gt; where they also used nested samples for their design matrix.&lt;/p&gt;&#xA;" OwnerUserId="15931" LastEditorUserId="15931" LastEditDate="2022-08-15T05:34:57.340" LastActivityDate="2022-08-15T05:34:57.340" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19545" PostTypeId="1" CreationDate="2022-08-15T09:45:44.667" Score="1" ViewCount="105" Body="&lt;p&gt;I would need some help regarding certain points in a RNASeq gene expression analysis (sorry if there are stupid questions, its my first project and i have no capable supervision). My workflow is the following:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;TMM normalization&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;obtaining CPMs&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;filtering CPMs that are under 1 in half the samples&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;obtaining log2(CPM) - after I perform this step I get Inf values, should I treat those values as NA and remove them or should I use CPM + 1 (I know that adding +1 introduces certain bias)?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;apply Median Absolute Deviation and use different thresholds to subset data&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;apply NMF clustering and find cluster specific genes and samples&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;use the sample clusters to obtain DEGs (using DESeq2) and proceed with GSEA&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Now for the questions.. Firstly, I tried to look for the best practices in scientific papers and on forums but they vary in a huge manner.&lt;/p&gt;&#xA;&lt;p&gt;In papers, such as &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/26909576/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pubmed.ncbi.nlm.nih.gov/26909576/&lt;/a&gt; the authors use log2(CPM) for NMF clustering analysis, while looking online at other suggestions, they all suggest using variance stabilizing transformation or z-scores etc. Now, what I'm trying to do is find subtype specific genes and significant DE genes based on those subtypes.&lt;/p&gt;&#xA;&lt;p&gt;From what I understand, VST transformation is ideal but why do all the papers that I've found only use TPM/CPM for subsequent analysis such as NMF clustering (other papers only mention gene expression or I missed that)?&lt;/p&gt;&#xA;&lt;p&gt;I've tried both ways, with log2(CPM) and with VST and I've found that VST clusters contain more genes and tend to be of equal size (both genes and samples clusters) while log2(CPM) clusters returns a cluster 3 times bigger than the second (both for genes and samples). Both VST and log2(CPM) clusters were selected by the best cophenetic coef and also samples had silhouette widths above 0.6.&lt;/p&gt;&#xA;&lt;p&gt;Now, what should be the best approach considering that I'm expected to write a decent paper with this dataset.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;&lt;p&gt;PS: all the above are done in R&lt;/p&gt;&#xA;" OwnerUserId="15944" LastActivityDate="2022-08-15T09:45:44.667" Title="RNASeq NMF clustering using what kind of expression unit?" Tags="|r|rna-seq|nmf|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19546" PostTypeId="2" ParentId="13740" CreationDate="2022-08-15T16:36:48.157" Score="0" Body="&lt;p&gt;Not a direct solution to your problem because it does not use SCE. But if you (and others who ended on this post) are facing the RAM bottleneck when analysing single-cell data, then &lt;strong&gt;Scarf&lt;/strong&gt; might be useful.&lt;/p&gt;&#xA;&lt;p&gt;Here is the Github link:&#xA;&lt;a href=&quot;https://github.com/parashardhapola/scarf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/parashardhapola/scarf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Documentation:&#xA;&lt;a href=&quot;https://scarf.readthedocs.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://scarf.readthedocs.io/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Paper:&#xA;&lt;a href=&quot;https://www.nature.com/articles/s41467-022-32097-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41467-022-32097-3&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15945" LastActivityDate="2022-08-15T16:36:48.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19547" PostTypeId="2" ParentId="19025" CreationDate="2022-08-15T16:45:11.827" Score="1" Body="&lt;p&gt;Scarf has been developed to specifically solve this issue. Scarf handles  large-scale scRNA-Seq/scATAC-Seq/CITE-Seq datasets that will otherwise not fit into memory. Unfortunately, this is not available in R yet.&lt;/p&gt;&#xA;&lt;p&gt;Here is the Github link:&#xA;&lt;a href=&quot;https://github.com/parashardhapola/scarf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/parashardhapola/scarf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Documentation:&#xA;&lt;a href=&quot;https://scarf.readthedocs.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://scarf.readthedocs.io/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Paper:&#xA;&lt;a href=&quot;https://www.nature.com/articles/s41467-022-32097-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41467-022-32097-3&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Please feel free to create an issue on Github for you use case (if you cant already find a solution in the documentation).&lt;/p&gt;&#xA;&lt;p&gt;/PD&lt;/p&gt;&#xA;" OwnerUserId="15945" LastEditorUserId="15945" LastEditDate="2022-08-15T16:45:41.150" LastActivityDate="2022-08-15T16:45:41.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19548" PostTypeId="2" ParentId="15061" CreationDate="2022-08-15T21:05:37.173" Score="2" Body="&lt;p&gt;The example given a 3 taxa tree, isn't an ideal example. What is required is a raw treefile in a common format (nexus, philip, newark). The example below uses ETE3 for a 5 taxa tree A,B,C,D,E. Its ROOTED - very important to know the direction.&lt;/p&gt;&#xA;&lt;p&gt;ETE3 is providing a means of traversing down the tree from the root, however the coder needs to interrogate the path it has taken.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Tree file imported&lt;/li&gt;&#xA;&lt;li&gt;Ladderizing ... so you know the horizontal direction of the tree (see *)&lt;/li&gt;&#xA;&lt;li&gt;Ask the ETE3 to search down the nodes ... each child is a node from the root.&lt;/li&gt;&#xA;&lt;li&gt;Parse the output and identify the clade.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The two outputs from point 3 are either it encounters a node - in this case it returns all the children below it (taxa names). The alternative is that it encounters a terminal node or a leaf, like a species name or isolate. In this case the taxa name is returned.&lt;/p&gt;&#xA;&lt;p&gt;The caveat is the terminal node (taxa label or leaf) is always bifurcation and the taxa names are split into [1] and [0] on the ETE3 object (see code) ... you have to interrogate both.&lt;/p&gt;&#xA;&lt;p&gt;Thats what the OP is looking for and this part of the code is marked &lt;strong&gt;'Part 1'&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Parsing &lt;strong&gt;Part 2&lt;/strong&gt; is more complicated, because usually the coder wants the clade a particular taxa (or leaf) belongs to. Essentially, the coder needs to ask what the leaf name is, if the target leaf name is found then the clade is then identified, i.e. the other stuff next to it.&lt;/p&gt;&#xA;&lt;p&gt;The code is below. I agree if you don't think trees, this is a rushed explanation.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from ete3 import Tree&#xA;&#xA;# Part 1&#xA;tree = Tree(intree, format=1)&#xA;tree.ladderize(direction=1)&#xA;clade1_ = tree.children[0].children[0].children[0].name&#xA;leaf1_ = tree.children[0].children[0].children[0].children[0].name &#xA;leaf2_ = tree.children[0].children[0].children[0].children[1].name&#xA;&#xA;# Part 2&#xA;if leaf1 == 'A' or leaf2 == 'A':&#xA;    CladeA = clade1_&#xA;if leaf2 == 'B' or leaf2 == 'B':&#xA;    CladeA = clade1_&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;*, ladderizing that can be important for classifying [0] and [1] in ETE3 ... its a weird one in trees 'cause it shouldn't matter&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-08-16T12:26:48.513" LastActivityDate="2022-08-16T12:26:48.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19549" PostTypeId="1" AcceptedAnswerId="19554" CreationDate="2022-08-16T11:12:37.527" Score="3" ViewCount="91" Body="&lt;p&gt;hey i have the problem with bash script. It should run interproscan REST&#xA;Im green in bash scripting. I found this script on the &lt;a href=&quot;https://gist.github.com/terrymun/889f28314643d429496881adbdd40039&quot; rel=&quot;nofollow noreferrer&quot;&gt;web&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;i=1&#xA;waitevery=30&#xA;&#xA;mkdir -p out&#xA;&#xA;for j in &lt;span class=&quot;math-container&quot;&gt;$(find `pwd` -type f -name &quot;*.fa&quot;)&#xA;do&#xA;    echo &quot;Iteration: $&lt;/span&gt;i; File: &lt;span class=&quot;math-container&quot;&gt;$j&quot;&#xA;    filename=$&lt;/span&gt;(basename &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$j&quot;)&#xA;    python3 iprscan5.py \&#xA;        --goterms \&#xA;        --pathways \&#xA;        --email=blabla@protonmail.com \&#xA;        --outfile=out/$&lt;/span&gt;{filename} \&#xA;        --outformat=tsv \&#xA;        --quiet \&#xA;        $j &amp;amp; (( i++%waitevery==0 )) &amp;amp;&amp;amp; wait&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;out:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Iteration: 1; File: /home/marcin/interproscan-5.57-90.0-64-bit/interproscan-5.57-90.0/test/xtr.fa&#xA;interpro_batch_submit.sh: 18: i++%waitevery==0: not found&#xA;Iteration: 1; File: /home/marcin/interproscan-5.57-90.0-64-bit/interproscan-5.57-90.0/test/xts.fa&#xA;interpro_batch_submit.sh: 18: i++%waitevery==0: not found&#xA;Iteration: 1; File: /home/marcin/interproscan-5.57-90.0-64-bit/interproscan-5.57-90.0/test/xtt.fa&#xA;interpro_batch_submit.sh: 18: i++%waitevery==0: not found&#xA;Iteration: 1; File: /home/marcin/interproscan-5.57-90.0-64-bit/interproscan-5.57-90.0/test/xtu.fa&#xA;interpro_batch_submit.sh: 18: i++%waitevery==0: not found&#xA;Iteration: 1; File: /home/marcin/interproscan-5.57-90.0-64-bit/interproscan-5.57-90.0/test/xtv.fa&#xA;interpro_batch_submit.sh: 18: i++%waitevery==0: not found&#xA;Iteration: 1; File: /home/marcin/interproscan-5.57-90.0-64-bit/interproscan-5.57-90.0/test/xtw.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&lt;p&gt;I think I forgot to add &lt;code&gt;--sequence&lt;/code&gt;, but something was still wrong after it.&lt;/p&gt;&#xA;" OwnerUserId="1224" LastEditorUserId="73" LastEditDate="2022-08-17T11:45:38.483" LastActivityDate="2022-08-17T21:55:08.610" Title="bash script for interproscan REST" Tags="|bash|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19551" PostTypeId="2" ParentId="19531" CreationDate="2022-08-16T12:54:47.100" Score="1" Body="&lt;p&gt;Following the answer of @&lt;em&gt;Ajay&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You can add &lt;code&gt;+ geom_boxplot()&lt;/code&gt; after the command to turn the violin plot to a box plot.&lt;/p&gt;&#xA;&lt;p&gt;As for the barplot, you can do it with these:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# the object is your seurat object&#xA;data.df &amp;lt;- object@meta.data[, c(&amp;quot;celltypes&amp;quot;, &amp;quot;condition&amp;quot;)]&#xA;data.df[&amp;quot;value&amp;quot;] &amp;lt;- object$RNA@data[&amp;quot;geneX&amp;quot;, ]&#xA;data.df &amp;lt;- data.df %&amp;gt;% mutate(group = paste(celltypes, condition, sep=&amp;quot;:&amp;quot;))&#xA;&#xA;# remove cells that do not express the gene&#xA;data.df &amp;lt;- data.df %&amp;gt;% filter(value &amp;gt; 0)&#xA;&#xA;data.df.2 &amp;lt;- data.df %&amp;gt;% group_by(group) %&amp;gt;% summarise(sd=sd(value),value=mean(value))&#xA;X &amp;lt;- str_split(data.df.2&lt;span class=&quot;math-container&quot;&gt;$'group', &quot;:&quot;, 2, simplify=TRUE)&#xA;data.df.2$&lt;/span&gt;'celltypes' &amp;lt;- X[, 1]&#xA;data.df.2$'condition' &amp;lt;- X[, 2]&#xA;&#xA;ggplot(data.df.2, aes(fill=condition, y=value, x=celltypes)) + &#xA;    geom_bar(color=&amp;quot;black&amp;quot;, position=&amp;quot;dodge&amp;quot;, stat=&amp;quot;identity&amp;quot;) +&#xA;    geom_errorbar(&#xA;        aes(ymin=value, ymax=value+sd), width=.2,&#xA;        position=position_dodge(.9)&#xA;    )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Dropout is very usual in single-cell data. If you really want to show the expression with barplot, I would suggest removing the zero values or using denoised data for better visualisation.&lt;/p&gt;&#xA;" OwnerUserId="15952" LastEditorUserId="15952" LastEditDate="2022-08-16T13:43:18.550" LastActivityDate="2022-08-16T13:43:18.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19553" PostTypeId="2" ParentId="19529" CreationDate="2022-08-16T14:04:13.520" Score="0" Body="&lt;p&gt;I don't think is a good idea to apply the gene selection method on the ATAC-seq genomic regions because of the high sparsity of ATAC-seq data.&lt;/p&gt;&#xA;&lt;p&gt;A common way is to select the most frequent regions (like the &lt;a href=&quot;https://satijalab.org/signac/reference/findtopfeatures&quot; rel=&quot;nofollow noreferrer&quot;&gt;Signac&lt;/a&gt; does), or to discard the house-keeping regions (like the &lt;a href=&quot;https://github.com/r3fang/SnapATAC/blob/master/examples/10X_brain_5k/README.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;SnapATAC&lt;/a&gt; does).&lt;/p&gt;&#xA;&lt;p&gt;And you can also use the &lt;a href=&quot;https://episcanpy.readthedocs.io/en/latest/api/episcanpy.pp.select_var_feature.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;episcanpy&lt;/a&gt;, a epigenomics version of scanpy.&lt;/p&gt;&#xA;" OwnerUserId="15952" LastEditorUserId="15952" LastEditDate="2022-08-16T14:09:34.267" LastActivityDate="2022-08-16T14:09:34.267" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19554" PostTypeId="2" ParentId="19549" CreationDate="2022-08-17T11:08:10.003" Score="1" Body="&lt;p&gt;First of all &lt;em&gt;never&lt;/em&gt; do &lt;code&gt;for foo in $(find ...)&lt;/code&gt;. That is very bad practice as it can't handle even slightly odd file names like those with a space. See &lt;a href=&quot;https://mywiki.wooledge.org/BashPitfalls#for_f_in_.24.28ls_.2A.mp3.29&quot; rel=&quot;nofollow noreferrer&quot;&gt;bash pitfall #1&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Next, you don't need &lt;code&gt;pwd&lt;/code&gt;, just run &lt;code&gt;find -type f -name &amp;quot;*.fa&amp;quot;&lt;/code&gt; which will search in the current directory by default and unlike &lt;code&gt;find `pwd`... &lt;/code&gt;, won't break if your path contains whitespace.&lt;/p&gt;&#xA;&lt;p&gt;That's just general best practice, the real issue here is that you are probably running the script like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sh /path/to/script.sh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Right? And you are using a Debian-based system like Ubuntu, right? On such systems, &lt;code&gt;/bin/sh&lt;/code&gt; is actually not bash but a very basic POSIX compatible shell called &lt;code&gt;dash&lt;/code&gt; and &lt;code&gt;dash&lt;/code&gt; doesn't understand the &lt;code&gt;(( i++%waitevery==0 ))&lt;/code&gt; syntax.&lt;/p&gt;&#xA;&lt;p&gt;Since you have a shebang line (&lt;code&gt;#!/bin/bash&lt;/code&gt;), you can simply make your script executable (&lt;code&gt;chmod a+x /path/to/script.sh&lt;/code&gt;) and then run the script directly like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/path/to/script.sh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or, alternatively, run it explicitly with &lt;code&gt;bash&lt;/code&gt; instead of &lt;code&gt;sh&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bash /path/to/script.sh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Either should work and if your file names and paths are sane, the issues I mentioned above won't be relevant. But to keep things nice and clean, I would also change the script to (note that this assumes you are using Linux and GNU &lt;code&gt;find&lt;/code&gt;):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;#!/bin/bash&#xA;i=1&#xA;waitevery=30&#xA;&#xA;mkdir -p out&#xA;&#xA;find -type f -name &amp;quot;*.fa&amp;quot; -print0 |&#xA;  while IFS= read -r -d '' j; do &#xA;    echo &amp;quot;Iteration: &lt;span class=&quot;math-container&quot;&gt;$i; File: $&lt;/span&gt;j&amp;quot;&#xA;    filename=&lt;span class=&quot;math-container&quot;&gt;$(basename &quot;$&lt;/span&gt;j&amp;quot;)&#xA;    python3 iprscan5.py \&#xA;        --goterms \&#xA;        --pathways \&#xA;        --email=&amp;quot;blabla@protonmail.com&amp;quot; \&#xA;        --outfile=out/&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$filename&quot; \&#xA;    --outformat=tsv \&#xA;    --quiet \&#xA;    &quot;$&lt;/span&gt;j&amp;quot; &amp;amp; (( i++%waitevery==0 )) &amp;amp;&amp;amp; wait&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2022-08-17T11:08:10.003" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19555" PostTypeId="2" ParentId="19549" CreationDate="2022-08-17T11:50:59.717" Score="1" Body="&lt;p&gt;I'd guess that the specific error that's coming up is because there are spaces missing from the wait line. This seems to be a problem in the source script, i.e.:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$j &amp;amp; (( i++%waitevery==0 )) &amp;amp;&amp;amp; wait&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;should probably be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$j &amp;amp; (( i++ % waitevery == 0 )) &amp;amp;&amp;amp; wait&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but for me (following @terdon's reproduction advice via &lt;code&gt;dash -c &amp;quot;i=9; ((++i%5==0)) &amp;amp;&amp;amp; echo 'worked!'&amp;quot;&lt;/code&gt;) that only changes the thing that is not found from &lt;code&gt;i++%waitevery==0&lt;/code&gt; to &lt;code&gt;i++&lt;/code&gt;, suggesting that the post-increment operator is not supported in the shell. Post-increment &lt;em&gt;is&lt;/em&gt; &lt;a href=&quot;https://linuxhint.com/bash_operator_examples/#o13&quot; rel=&quot;nofollow noreferrer&quot;&gt;supported in bash&lt;/a&gt;, suggesting that this script is being run (as @RamRS commented) in an unexpected way.&lt;/p&gt;&#xA;&lt;p&gt;Try running the script directly:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./interpro_batch_submit.sh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or explicitly using bash:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bash ./interpro_batch_submit.sh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... but see @terdon's answer for a better approach to doing this.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-08-17T21:55:08.610" LastActivityDate="2022-08-17T21:55:08.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19556" PostTypeId="1" CreationDate="2022-08-17T16:50:49.547" Score="1" ViewCount="20" Body="&lt;p&gt;I am new to whole genome sequencing. I want to compare illumina whole genome sequencing of mutants of a particular bacteria with a genome (not annotated) of the original strain to look for what the mutations are, how would I do about doing that?&lt;/p&gt;&#xA;" OwnerUserId="15856" LastEditorUserId="15856" LastEditDate="2022-08-17T17:11:49.020" LastActivityDate="2022-08-17T17:11:49.020" Title="Compare illumina sequencing output with reference genome" Tags="|genomics|wgs|bacteria|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19557" PostTypeId="1" CreationDate="2022-08-17T22:22:41.677" Score="3" ViewCount="186" Body="&lt;p&gt;As the title states, I've got two sets of unaligned amino acid sequences (~25k sequences in one, ~3k in the other). I want to find the minimum distance between a sequence in the first set and a sequence in the second, expressed in terms of percent identity (so, maximum percent identity). So far I've only been able to find R packages that will compute percent identity for two already aligned sequences, which would then allow me to do a double loop over both sets of sequences to find the shortest distance between them. But this seems woefully inefficient - is there an easier/faster way? I also tried playing around with mmseqs2 but couldn't tell if there was a way to use it for anything other than clustering.&lt;/p&gt;&#xA;&lt;p&gt;To clarify - there is a large amount of variation between the sequences in question, including in overall length (ranging from 148 residues to 872). They don't all belong to one family of homologs or anything. I'm expecting many of them to have quite low identity with each other. But the constraints of the problem I'm working on require me to put a number on that identity, or at least an upper bound.&lt;/p&gt;&#xA;&lt;p&gt;I'm not constrained to using R, it's just what I'm most familiar with. Solutions with command line tools or python would also be fine - I'm sure there are better tools out there that I'm not aware of.&lt;/p&gt;&#xA;" OwnerUserId="8937" LastEditorUserId="3848" LastEditDate="2023-06-24T15:27:40.090" LastActivityDate="2023-06-24T15:27:40.090" Title="How can I find the maximum percent identity between two sets of (unaligned) sequences?" Tags="|phylogenetics|sequence-alignment|phylogeny|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19558" PostTypeId="1" CreationDate="2022-08-18T04:08:15.387" Score="1" ViewCount="122" Body="&lt;p&gt;Just as the title says, I am working in PLINK 1.9 trying to convert the following VCF into a PLINK file (binary or not either is fine), while maintaining the SNP names from the VCF file. I am on Windows, so it would be easier for me to utilize PLINK for this. Does anyone know how to achieve this? The link below is the VCF file I am using.&lt;/p&gt;&#xA;&lt;p&gt;I tried using the --vcf command by itself, along with using the --vcf command in conjunction with the --recode command. I also tried using the --double-id command to see if that would affect anything, but it did not.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr22.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr22.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15970" LastEditorUserId="15970" LastEditDate="2022-08-18T15:56:09.273" LastActivityDate="2022-08-18T15:56:09.273" Title="Converting from VCF to PLINK while maintaining the SNP names from the VCF file" Tags="|vcf|snp|plink|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19559" PostTypeId="2" ParentId="17878" CreationDate="2022-08-18T07:08:26.897" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;So I need to combine all the datasets to only compare the genes found in all the datasets mentioned and then perform a PCA analysis on those genes only.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You can merge all your datasets for common genes in all datasets and perform &lt;code&gt;PCA&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I was told that the data in the table has to be transposed basically, so that the genes are the variables and the samples are the rows. I am just not sure how to do this in R.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Your choice of transposing data will depend on whether you want to show variability between genes or samples. However you can get results for both individuals (genes) as well as variables (samples), using the &lt;code&gt;prcomp()&lt;/code&gt;, &lt;code&gt;factoMineR&lt;/code&gt; and &lt;code&gt;factoexta&lt;/code&gt; packages in 'R'. Usually I like to use &lt;code&gt;$rotation&lt;/code&gt; values ​​to plot the variability between samples using &lt;code&gt;ggplot2&lt;/code&gt; package. You can use this &lt;a href=&quot;http://www.sthda.com/english/articles/31-principal-component-methods-in-r-practical-guide/118-principal-component-analysis-in-r-prcomp-vs-princomp/#:%7E:text=The%20function%20princomp()%20uses,preferred%20compared%20to%20princomp().&quot; rel=&quot;nofollow noreferrer&quot;&gt;PCA Tutorial&lt;/a&gt; to learn more.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I was also told there is a good chance that the noise resulting from the use of different platforms would likely cause these datasets to separate from each other based on that, making this analysis meaningless or not allowing for comparison of gene expression that is conserved across the datasets as a result of mixing the two species during growth.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is true as the datasets are from different platforms, there may be some noise or batch effects. However, you can plot the 'PCA' and decide for yourself whether it is good or not.&lt;/p&gt;&#xA;" OwnerUserId="15931" LastActivityDate="2022-08-18T07:08:26.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19560" PostTypeId="2" ParentId="17467" CreationDate="2022-08-18T07:22:50.807" Score="1" Body="&lt;p&gt;As the warning message is suggesting, you should convert the &lt;code&gt;class&lt;/code&gt; of your &lt;code&gt;treatment&lt;/code&gt; column in &lt;code&gt;coldata&lt;/code&gt; from &lt;code&gt;character&lt;/code&gt; to &lt;code&gt;factor&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Use this:&lt;br /&gt;&#xA;&lt;code&gt;coldata$treatment = factor(coldata$treatment)&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="15931" LastActivityDate="2022-08-18T07:22:50.807" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19561" PostTypeId="1" CreationDate="2022-08-18T10:27:10.493" Score="1" ViewCount="31" Body="&lt;p&gt;I am trying to assemble reads belonging to two different readlength. Is it a valid way since I am looking for common genes among the species I am assembling.&lt;/p&gt;&#xA;" OwnerUserId="15972" LastActivityDate="2022-08-18T10:27:10.493" Title="Transcriptome analysis" Tags="|r|rna-seq|genomics|rna|metagenome|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19563" PostTypeId="2" ParentId="19557" CreationDate="2022-08-18T15:49:42.187" Score="1" Body="&lt;p&gt;When I've been in 'possibly' comparable situations I've just used &lt;code&gt;local Blast&lt;/code&gt; and parsed out the top hit using Python. It may not be the most optimal package (actually its not), but its easy and works, whilst NCBI's Blast support desk is efficient. Each query in turn is tested via a loop against the whole local database.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-08-18T15:49:42.187" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19564" PostTypeId="1" CreationDate="2022-08-18T17:54:23.640" Score="2" ViewCount="123" Body="&lt;p&gt;I'm going to call variants with VarScan from a pileup files created with samtools. I realized that there are in general two major possibilities to call variants, pileup as well as mpileup. The VarScan manual says that pileup is for single-sample, mpileup for multi-sample calling. Does that mean that one should use pileup to analyze clones, and mpileup if whole populations have been sequenced? What is the difference between both, pileup and mpileup? What would be the consequences to use the &amp;quot;wrong&amp;quot; pileup?&lt;/p&gt;&#xA;&lt;p&gt;Moreover, I realized that samtools formerly also had the option pileup. However, pileup was later on deprecated. Thus, would you agree that using mpileup would do the job regardless of the sample?&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot in advance!&lt;/p&gt;&#xA;" OwnerUserId="15878" LastEditorUserId="15878" LastEditDate="2022-08-18T18:27:18.803" LastActivityDate="2022-08-18T18:27:18.803" Title="Difference between pileup and mpileup in VarScan and samtools" Tags="|samtools|genomics|variant-calling|mpileup|variant-filtration|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19566" PostTypeId="1" CreationDate="2022-08-18T19:54:31.837" Score="1" ViewCount="99" Body="&lt;p&gt;I've got a list of proteins, complete with Uniprot IDs (for some of them) and gene names. I'd like to group them into orthology groups based on data from KEGG, but I have no idea how to add a column with KEGG groups. I've tried poking around in the documentation for UniProt, but all I can find is this page on KEGG - no mention of how to link to KEGG data. I've also found these KEGG mapper tools (&lt;a href=&quot;https://www.kegg.jp/kegg/tool/conv_id.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;web portal&lt;/a&gt; and &lt;a href=&quot;https://www.genome.jp/kegg/mapper/&quot; rel=&quot;nofollow noreferrer&quot;&gt;command line&lt;/a&gt;) but those don't seem to be what I need in order to cluster my proteins into groups of KEGG orthologues.&lt;/p&gt;&#xA;&lt;p&gt;I've tried using R via the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(KEGGREST)&#xA;&#xA;kegg_ids &amp;lt;- KEGGREST::keggConv(&#xA;  target = &amp;quot;genes&amp;quot;,&#xA;  source = paste(&amp;quot;uniprot&amp;quot;, uniprot_ids, sep = ':'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But this turns up matches for only a fraction of my protein IDs... and anyway, only about half of them have those IDs - the rest are human proteins for which I only have the gene name.&lt;/p&gt;&#xA;" OwnerUserId="8937" LastEditorUserId="8937" LastEditDate="2022-08-29T20:43:52.360" LastActivityDate="2022-08-29T20:43:52.360" Title="How to annotate list of proteins with KEGG orthology groups?" Tags="|uniprot|kegg|orthologues|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19567" PostTypeId="2" ParentId="19052" CreationDate="2022-08-18T20:26:46.260" Score="2" Body="&lt;p&gt;&lt;code&gt;AnnotationDBi&lt;/code&gt; uses &lt;code&gt;AnnotationDb&lt;/code&gt; objects which is the virtual base class for all annotation packages. It contain a database connection and is meant to be the parent for a set of classes in the Bioconductor annotation packages. Examples of classes derived from the &lt;code&gt;AnnotationDb&lt;/code&gt; object include (but are not limited to): &lt;code&gt;ChipDb&lt;/code&gt;, &lt;code&gt;OrgDb&lt;/code&gt;, &lt;code&gt;GODb&lt;/code&gt;, &lt;code&gt;OrthologyDb&lt;/code&gt; and &lt;code&gt;ReactomeDb&lt;/code&gt;. The database from one of these classes is used to extract annotations. &lt;code&gt;org.Mm.eg.db&lt;/code&gt; belongs to &lt;code&gt;OrgDb&lt;/code&gt; class.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioconductor.org/packages/release/data/annotation/html/Mus.musculus.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;Mus.musculus&lt;/code&gt;&lt;/a&gt;  represents a collection of annotation packages that can be used as a single object. It combines different information about same organisms (Mus musculus) into one place. It also uses &lt;code&gt;org.Mm.eg.db&lt;/code&gt; as a dependency. So, both uses same database to extract &lt;code&gt;Gene symbols&lt;/code&gt;. That is the reason they are producing same results. There is helpful &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/manuals/AnnotationDbi/man/AnnotationDbi.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; available on Bioconductor.&lt;/p&gt;&#xA;" OwnerUserId="15931" LastActivityDate="2022-08-18T20:26:46.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19568" PostTypeId="2" ParentId="19557" CreationDate="2022-08-18T22:45:40.223" Score="1" Body="&lt;p&gt;Try &lt;a href=&quot;https://github.com/soedinglab/MMseqs2&quot; rel=&quot;nofollow noreferrer&quot;&gt;mmseqs2&lt;/a&gt;. It is largely a faster version of blastp. I haven't used mmseqs2 before but its documentation seems simple enough.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2022-08-18T22:45:40.223" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19569" PostTypeId="1" CreationDate="2022-08-19T06:36:36.317" Score="3" ViewCount="53" Body="&lt;p&gt;Can someone explain how the score is calculated for &lt;a href=&quot;https://biopython.org/docs/latest/api/Bio.Blast.Record.html#Bio.Blast.Record.Description&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bio.Blast.Record.Description&lt;/a&gt;?  For example, if &lt;code&gt;Record.Description.num_alignments&lt;/code&gt; is &amp;gt;1, is &lt;code&gt;Record.Description.score&lt;/code&gt; simply the sum of the alignment scores for the description's hit?  Can alignments overlap?  How does the score of a given alignment deal with the scores of overlapping HSPs in that alignment?&lt;/p&gt;&#xA;&lt;p&gt;I'm getting the &lt;code&gt;Blast.Record&lt;/code&gt; object via &lt;a href=&quot;https://biopython.org/docs/latest/api/Bio.Blast.NCBIXML.html#Bio.Blast.NCBIXML.parse&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBIXML.parse()&lt;/a&gt;, and would like to rely on the description data as much as possible to remove the need for code that parses the &lt;a href=&quot;https://biopython.org/docs/latest/api/Bio.Blast.Record.html#Bio.Blast.Record.Alignment&quot; rel=&quot;nofollow noreferrer&quot;&gt;Record.Alignments&lt;/a&gt; objects, but I'm a nervous including things when I don't know exactly how they were created.&lt;/p&gt;&#xA;&lt;p&gt;Cheers!&#xA;Tim&lt;/p&gt;&#xA;" OwnerUserId="15908" LastActivityDate="2024-03-21T16:31:17.643" Title="How are BLAST record scores calculated in biopython" Tags="|sequence-alignment|python|phylogenetics|blastp|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19570" PostTypeId="1" AcceptedAnswerId="19604" CreationDate="2022-08-19T17:48:32.673" Score="7" ViewCount="1434" Body="&lt;p&gt;As the title summarizes, I am trying to:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Read a PDB file (for example, &lt;a href=&quot;https://files.rcsb.org/download/1ENH.pdb&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;1enh.pdb&lt;/code&gt;&lt;/a&gt;).&lt;/li&gt;&#xA;&lt;li&gt;Extract the backbone dihedral angles (phi, psi).&lt;/li&gt;&#xA;&lt;li&gt;Modify the dihedral angles (phi, psi) arbitrarily (for example, replace them with &lt;code&gt;np.random.uniform(-np.pi, np.pi, size = (56, 2))&lt;/code&gt; in 1enh, since there are 56 pairs of dihedrals).&lt;/li&gt;&#xA;&lt;li&gt;Reconstruct the Cartesian coordinates of the modified backbone.&lt;/li&gt;&#xA;&lt;li&gt;Write a PDB file based on the modified backbone.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;From &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/18849/help-using-biopythons-ic-chain-functionality-for-getting-protein-3d-coordinates&quot;&gt;this question&lt;/a&gt; it seems the tools for what I want are available in BioPython. There is also a &lt;code&gt;set_angle()&lt;/code&gt; method. But I only have a very limited knowledge of Python and BioPython. I am stuck at Steps 3 and 4, and dubious on Step 5.&lt;/p&gt;&#xA;&lt;p&gt;This is what I have tried so far:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/python&#xA;&#xA;import numpy as np&#xA;from Bio.PDB import PDBParser, PICIO, PDBIO&#xA;&#xA;## Step 1&#xA;&#xA;parser = PDBParser()&#xA;structure = parser.get_structure(&amp;quot;&amp;quot;, &amp;quot;1enh.pdb&amp;quot;)&#xA;&#xA;## Step 2&#xA;&#xA;structure.atom_to_internal_coordinates()&#xA;chain = list(structure.get_chains())[0]&#xA;ic_chain = chain.internal_coord&#xA;dihedrals = ic_chain.dihedra&#xA;&#xA;## Step 3 - ??&#xA;&#xA;# Modifiy (phi,psi). For example, replace (phi,psi) with &#xA;newdihedrals = np.random.uniform(-np.pi, np.pi, size = (56, 2))&#xA;&#xA;## Step 4 - ??&#xA;&#xA;modified = PICIO.read_PIC('internal')&#xA;modified.internal_to_atom_coordinates()&#xA;&#xA;## Step 5&#xA;&#xA;io = PDBIO()&#xA;io.set_structure(modified)&#xA;io.save('1enh_modified.pdb', preserve_atom_numbering = True) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Many thanks in advance.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; I do not care about possible clashes if the side-chain is included in the modified PDB. In any case, writing a modified PDB file that only includes the backbone is also fine.&lt;/p&gt;&#xA;" OwnerUserId="15987" LastEditorUserId="298" LastEditDate="2023-01-24T11:51:09.423" LastActivityDate="2023-01-24T11:51:09.423" Title="Read PDB file, extract dihedral angles, modify dihedral angles, reconstruct Cartesian coordinates, and write PDB file" Tags="|python|protein-structure|pdb|3d-structure|biopython|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19571" PostTypeId="1" AcceptedAnswerId="19576" CreationDate="2022-08-19T18:14:58.603" Score="2" ViewCount="66" Body="&lt;p&gt;Is there a sequence format allowing multiple sequences to be grouped  across a flat file? The specific application is stacking separate nucleotide sequences against a given RNA secondary structure.&lt;/p&gt;&#xA;&lt;p&gt;To rephrase the same question I'm looking to stack viral nucleotide data in different groups across a single flat file. FastaQ provides an example:&lt;/p&gt;&#xA;&lt;p&gt;Traditional fastQ,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@sequence_name&#xA;AGCTGAT&#xA;+&#xA;...(quality score)&#xA;&#xA;@sequence2&#xA;AGCTGAC&#xA;+&#xA;...(quality score)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What I'm looking for is a format,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@Gp1_virus1-2-3&#xA;AGACGTAG&#xA;+&#xA;AGAGACAA&#xA;+&#xA;AGAGACAT&#xA;&#xA;@Gp2_virus1-2&#xA;AGATGTAG&#xA;+&#xA;AGATACAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In the above example there are two groups comprising 3 (group 1) and 2 (group 2)  individual sequences (viruses). Here 'group' represents different structures.&lt;/p&gt;&#xA;&lt;p&gt;Stockholme format has potential. My understanding of this format, which is basic, is it stacks a sequence, together with structural information. What is sought is stacking multiple sequences and then a structure format. Any clarification would help.&lt;/p&gt;&#xA;&lt;p&gt;There are multiple XML formats, which may be relevant.&lt;/p&gt;&#xA;&lt;p&gt;Just to mention a 'flat file' is needed to allow manual inspection. Dumping JSON is a solution, but isn't friendly for optical inspection.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Application&lt;/strong&gt;&#xA;The immediate downstream application is structure/sequence viewers. I think at least one. viewer will accept Stockholme 1.0 format. rFAM is very similar (@terdon's comment)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summarised Question&lt;/strong&gt; The question could be rephrased as to whether multiple RNA families can be stacked inside one Stockholme 1.0 or else rFAM format?&lt;/p&gt;&#xA;&lt;p&gt;Some of the members here are likely familiar with &lt;strong&gt;pFAM&lt;/strong&gt;. So could multiple protein structures be stored and retrieved inside one &lt;strong&gt;pFAM&lt;/strong&gt; file.&lt;/p&gt;&#xA;&lt;p&gt;@gringer's contribution is very welcome.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-08-22T14:59:39.160" LastActivityDate="2022-09-01T20:05:08.297" Title="Sequence format: grouping sequences within a flat file" Tags="|fastq|file-formats|format-conversion|secondary-structure|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19572" PostTypeId="1" AcceptedAnswerId="19575" CreationDate="2022-08-19T20:22:07.743" Score="2" ViewCount="133" Body="&lt;p&gt;I'm trying to decide between PacBio HiFi or Illumina sequencing platforms for sequencing the genome of aChrysina scarab. We want to identify the color pattern loci and also perform a phylogenetic analysis as we'll be sequencing several species of that genus.&lt;/p&gt;&#xA;&lt;p&gt;When considering a hybrid approach I was told by someone at a sequencing facility that adding short reads to polish the HiFi reads assembly lowers the quality of the assembly and introduces more errors that corrections. Why is this? I haven't been able to find this statement in the literature.&lt;/p&gt;&#xA;" OwnerUserId="14030" LastEditorUserId="73" LastEditDate="2022-08-20T06:00:17.723" LastActivityDate="2022-08-20T06:00:17.723" Title="What is the most accurate approach for de Novo sequencing?" Tags="|ngs|illumina|pacbio|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19573" PostTypeId="2" ParentId="19572" CreationDate="2022-08-19T20:53:51.633" Score="0" Body="&lt;p&gt;Very interesting. Personally, I think the most important thing than anything I can input is it is worth asking about the HiFi quality index being considered and whether this was compatible with your biological objective.&lt;/p&gt;&#xA;&lt;p&gt;Just to highlight that insect genomes are very different to genomics here - traditionally and even now - there has been a disproportionate focus on dipteran genomes. Okay certain Lepidoptera, namely the silk moth (Bombyx), is an advanced area of study. Beetles genomes are even more different still, agreed they holometabolous insects but still along way from dipterans. In addition, its worth pointing out beetles were late on the genome sequencing scene.&lt;/p&gt;&#xA;&lt;p&gt;I don't know the precise biological objective - but I'm going to guess it.  I'm assuming your wanting SNP variation to associate with phenotype colour variation, for the target genes (just guessing).&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;HiFi reads&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;With regarding mixing the genome sequencing approaches: what is the quality scale being used? If its lowering the chromosomal assembly (traditionally a key difficulty in insects), or weakening contig assembly then ok I could imagine. This is very different DNA. If the quality index is relating to lowering SNP accuracy I would be surprised, for others to comment, but just can't see it.&lt;/p&gt;&#xA;&lt;p&gt;I am aware that a key issue is that insect genome, specifically with reference to dipteran genomes, has caused extreme difficult assembling due to the high AT content and repeat regions, with chromosomal identification a traditiona nightmare and requiring wet lab chromosomal mapping (Diptera). I think beetles are an outlier even on this spectrum - against other insects. The advice you might have been given could be with this issue in mind, particularly if it was from another entomologist.&lt;/p&gt;&#xA;&lt;p&gt;I don't know if this genome has been sequenced, the implication of the question (title) is that is hasn't been sequenced yet and which leads to the second issue,&lt;/p&gt;&#xA;&lt;p&gt;JBS Haldane&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;God has an inordinate fondness for stars and beetles.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There are an awful lot of beetles and a lot of those genomes not sequenced yet.&lt;/p&gt;&#xA;&lt;p&gt;Personally I think you don't want accurate (really to maximise) genome assembly. In that scenario its got to be Illumina short-read, because this is a SNP study, where high accuracy calling from read depth is needed, given my guessed objective.&lt;/p&gt;&#xA;&lt;p&gt;If the genome hasn't been sequenced there could be pressure to enhance genome assembly rather than target your biological objective. This could be why there has been mixed advice, between what's good for the community and what's needed for your biological objective, this being colour variation genes - for very pretty beetles.&lt;/p&gt;&#xA;&lt;p&gt;I think, personally, its not necessarily a technical issue its either a win for others, or win for you. The only caveat is if the genes of interest are some extensive, physically linked gene family, then of course short-read may not be cool. I am not aware of that is a key feature of insect genomes.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;In insect genomes there has been misplaced excitement ... new insect genome ... hang on thats just unordered  contigs. This will happen using a short-read only approach.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-08-19T22:37:43.320" LastActivityDate="2022-08-19T22:37:43.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19574" PostTypeId="2" ParentId="5345" CreationDate="2022-08-19T21:48:28.770" Score="1" Body="&lt;p&gt;You can use  &lt;code&gt;org.At.tair.db&lt;/code&gt;: Genome wide annotation for Arabidopsis, which is an R package available on &lt;a href=&quot;https://bioconductor.org/packages/release/data/annotation/html/org.At.tair.db.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioconductor&lt;/a&gt; to extract gene symbols.&lt;/p&gt;&#xA;" OwnerUserId="15931" LastActivityDate="2022-08-19T21:48:28.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19575" PostTypeId="2" ParentId="19572" CreationDate="2022-08-19T22:44:47.463" Score="4" Body="&lt;p&gt;HiFi reads can resolve most repeats in a genome. Illumina reads from these repetitive regions are likely to get mismapped and these mapping errors may become consensus errors. Most short-read polishing tools developed in the old days (e.g. pilon) will reduce the contig consensus quality. Furthermore, when you have a non-inbred diploid genome, HiFi reads can locally phase a large portion of the genome, while Illumina reads are likely to cancel/discard most HiFi phasing information, which would lead to a worse and less informative assembly.&lt;/p&gt;&#xA;&lt;p&gt;In theory, if you use Illumina reads the right way, you can further improve HiFi contig consensus like what the telomere-to-telomere team is &lt;a href=&quot;https://github.com/arangrhie/T2T-Polish/blob/master/doc/T2T_polishing_case_study.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;doing here&lt;/a&gt;. However, this pipeline is for haploid/inbred genomes only. There are no tools for generic genomes. Developing your own is beyond the capability of most people in the field. To this end, the person at your sequencing facility is mostly right: don't polish HiFi assemblies with Illumina reads.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2022-08-19T23:24:48.483" LastActivityDate="2022-08-19T23:24:48.483" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19576" PostTypeId="2" ParentId="19571" CreationDate="2022-08-20T06:24:19.657" Score="2" Body="&lt;p&gt;An unmapped SAM file format allows for read groups. You can use &lt;code&gt;samtools import&lt;/code&gt; to create unmapped SAM files from FASTQ files:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools import -R virus-1-2 in12.fq &amp;gt; 12_reads.sam&#xA;samtools import -R virus-1-2-3 in123.fq &amp;gt; 123_reads.sam&#xA;samtools merge out.sam 12_reads.sam 123_reads.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-08-20T06:24:19.657" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19577" PostTypeId="2" ParentId="908" CreationDate="2022-08-20T06:38:35.903" Score="0" Body="&lt;p&gt;If the files already have defined read groups inside them, then doing a merge without the &lt;code&gt;-r&lt;/code&gt; option should work:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools merge merged.bam s1.sort.sam s2.sort.sam s3.sort.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If there are multiple input files that share the same read group, then by default they will have random strings appended to make the read groups unique. This can be stopped by using the &lt;code&gt;-c&lt;/code&gt; option, as mentioned in &lt;code&gt;man samtools merge&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   -c      When  several  input files contain @RG headers with the same ID, emit only&#xA;           one of them (namely, the header line from the first file we find  that  ID&#xA;           in) to the merged output file.  Combining these similar headers is usually&#xA;           the right thing to do when the files being merged originated from the same&#xA;           file.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-08-20T06:38:35.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19578" PostTypeId="1" AcceptedAnswerId="19580" CreationDate="2022-08-20T16:21:35.203" Score="1" ViewCount="41" Body="&lt;p&gt;My end goal is annotate a de-novo assembled genome. When trying to select the best method for transcriptome assembly I read Iso-seq was the preferred method. However other people suggested Nanopore as reverse transcriptase artifacts can be avoided. Is there any other way one can avoid these artifacts without the need for Nanopore?&#xA;Also, what is the recommended depth? Is using more SMRTcells worth the cost?&lt;/p&gt;&#xA;" OwnerUserId="14030" LastActivityDate="2022-08-20T18:37:37.367" Title="How can Iso-Seq reverse transcriptase artifacts be avoided?" Tags="|rna-seq|ngs|nanopore|pacbio|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19579" PostTypeId="1" CreationDate="2022-08-20T16:38:03.427" Score="3" ViewCount="32" Body="&lt;p&gt;I would like to have high depth of coverage of the transcriptome, but I need to analyze several tissues. I was suggested to pool all 5 tissue samples in same SMRT 8M cell. Will this result in a smaller coverage when compared to non-multiplex version?&lt;/p&gt;&#xA;" OwnerUserId="14030" LastActivityDate="2022-08-20T16:38:03.427" Title="Can multiplexing in Sequel II SMRTcells reduce the coverage?" Tags="|rna-seq|ngs|long-reads|pacbio|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19580" PostTypeId="2" ParentId="19578" CreationDate="2022-08-20T18:37:37.367" Score="2" Body="&lt;p&gt;It is &lt;a href=&quot;https://genome.cshlp.org/content/28/3/396&quot; rel=&quot;nofollow noreferrer&quot;&gt;possible to filter out&lt;/a&gt; iso-seq reads with RT artifacts, though I don't know how effective these informatics approaches are. With nanopore, you will get more false junctions in alignment or transcript assembly due to sequencing errors. I more often see iso-seq used for new species.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2022-08-20T18:37:37.367" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19581" PostTypeId="1" CreationDate="2022-08-21T02:35:34.840" Score="1" ViewCount="99" Body="&lt;p&gt;I am trying to make a figure in which the protein structures are colored by the distance between the calphas in the overlayed structures.&lt;/p&gt;&#xA;&lt;p&gt;I am comparing our cryoEM structure to that of a previous crystal structure and trying to show the major distance shifts in an easy to look at manner. I can get the RMSD using some online servers but it doesn't give me a good visual way of looking at the data.&lt;/p&gt;&#xA;" OwnerUserId="15994" LastEditorUserId="10049" LastEditDate="2023-07-23T04:36:45.160" LastActivityDate="2023-07-23T04:36:45.160" Title="How can I measure the distance between each pair of carbon alphas in overlayed protein structures?" Tags="|python|protein-structure|pymol|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19583" PostTypeId="1" CreationDate="2022-08-21T12:25:05.217" Score="2" ViewCount="28" Body="&lt;p&gt;Recently I've been trying to use &lt;a href=&quot;https://github.com/Edinburgh-Genome-Foundry/DnaWeaver/blob/master/README.rst&quot; rel=&quot;nofollow noreferrer&quot;&gt;DNA Weaver&lt;/a&gt; for customized gene assembly design.&#xA;Although there is a web version that allows to perform some assembly examples, I've tried to replicate them with python code.&lt;/p&gt;&#xA;&lt;p&gt;There is some documentation about it and it the developers have provided some examples within the documentation. However, I wonder if is there any other more extensive documentation with all the possible parameters / attributes for each function / objects.&lt;/p&gt;&#xA;&lt;p&gt;An example of this is the definition of a &amp;quot;fixed size overhangs&amp;quot; during Gibson Assembly design, instead of using a Tm range for this task.  It has something to do with the &amp;quot;Overhang selector&amp;quot;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;overhang_selector=dw.TmSegmentSelector(min_tm=55, max_tm=70), &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I have not found any information about what would be the code to run to replace the &amp;quot;TmSegmentSelector&amp;quot; for &amp;quot;FixedSizeSegmentSelector&amp;quot;. However there are no clear instructions about this.&lt;/p&gt;&#xA;&lt;p&gt;Other example would be how to specify a custom - genome or DNA sequence from which regions could be PCR amplified as part of the routine assembly.&lt;/p&gt;&#xA;&lt;p&gt;And so on...&lt;/p&gt;&#xA;&lt;p&gt;Does anybody could provide some light about this?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;&lt;p&gt;Jorge.&lt;/p&gt;&#xA;" OwnerUserId="15995" LastEditorUserId="15995" LastEditDate="2022-08-21T12:29:17.520" LastActivityDate="2022-08-21T12:29:17.520" Title="Is there a DNAWeaver How-to guide?" Tags="|phylogenetics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19584" PostTypeId="1" AcceptedAnswerId="19585" CreationDate="2022-08-21T13:38:50.417" Score="1" ViewCount="488" Body="&lt;p&gt;I have a data frame that I want to switch its row names from EnsembleID (GENEID) to gene symbol (SYMBOL). When I try to switch, I get this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in `.rowNamesDF&amp;lt;-`(x, value = value) : &#xA;  duplicate 'row.names' are not allowed&#xA;In addition: Warning message:&#xA; Error in `.rowNamesDF&amp;lt;-`(x, value = value) : &#xA;  duplicate 'row.names' are not allowed &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The code I'm using works for other similar data frames. Only this one is giving me trouble. There are no duplicates in the row names, I checked, so the source of this error is strange.&lt;/p&gt;&#xA;&lt;p&gt;This is the code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GetSymbols &amp;lt;- function(df) {   # Switch from geneID format to Symbol format&#xA;  &#xA;  ensembl.genes = as.vector(rownames(df))&#xA;  geneIDs = ensembldb::select(EnsDb.Hsapiens.v86, keys= ensembl.genes, keytype = &amp;quot;GENEID&amp;quot;, columns = c(&amp;quot;SYMBOL&amp;quot;,&amp;quot;GENEID&amp;quot;))&#xA;  dupli = duplicated(geneIDs&lt;span class=&quot;math-container&quot;&gt;$GENEID)&#xA;  geneIDs = geneIDs[!dupli,]&#xA;  rownames(geneIDs) = geneIDs$&lt;/span&gt;GENEID&#xA;  rownames(df) = geneIDs$SYMBOL&#xA;  return(df)&#xA;  &#xA;}&#xA;&#xA;#Calling the function:&#xA; df = GetSymbols(df)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h4&gt;A sample of the dataframe:&lt;/h4&gt;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(logFC = c(0.362414024114299, 0.270660335317552, &#xA;0.250197714648158, 0.355106754020497, 0.860436247608723, 0.331838240496886, &#xA;0.642000001979217, -0.506238094622956, 0.199130048525963, 0.226010606161764, &#xA;-0.300273572714138, 0.763447672404514, -0.282220097838676, 0.686284440644756, &#xA;0.209266959343866, 0.201216572039244, 0.467207891400503, 0.207476905368996, &#xA;0.548263917432251, 0.582095719576479, 0.167369145307915, -0.239886539805618, &#xA;-0.133088162158502, 0.432528267703316, -0.232843401197013), AveExpr = c(2.63663927914016, &#xA;6.23730774047258, 6.74130527553022, 6.51256660562955, 3.51386518575641, &#xA;0.962682352871946, 3.60110717340049, -1.07537043066235, 4.55672701732791, &#xA;3.96125159652725, 1.20473881401599, 1.72955742088622, 2.10495435034315, &#xA;2.95446094011986, 4.31205976523475, 4.85412008885978, 4.09171246258306, &#xA;3.94681319329962, 6.27193781389662, 0.936821016422818, 7.97270280545117, &#xA;2.4470079966345, 5.75991260844846, 3.84225682530045, 3.35515997418382&#xA;), t = c(5.15993300980101, 4.74644046948475, 4.67895886939685, &#xA;4.65270476519835, 4.39369295809775, 4.51083389626156, 4.20489542862464, &#xA;-4.49987927160777, 4.1096966271885, 4.03273843222089, -4.09143780518073, &#xA;4.07713970281633, -3.99157698713235, 3.97814493972727, 3.94402861982903, &#xA;3.92673590933521, 3.92436825669156, 3.90538427413924, 3.89364498564491, &#xA;3.93360332510695, 3.80233390398449, -3.76935782400013, -3.7013905266149, &#xA;3.68313224298669, -3.68114307289328), P.Value = c(4.50582660244125e-07, &#xA;3.21245260466636e-06, 4.37260426507432e-06, 4.92523952290176e-06, &#xA;1.54869545509172e-05, 9.2842399715271e-06, 3.45373717288344e-05, &#xA;9.74370585569957e-06, 5.12014322967072e-05, 7.00192255628614e-05, &#xA;5.51721193906694e-05, 5.84844087520547e-05, 8.26184479488647e-05, &#xA;8.71763749953859e-05, 9.9850615705057e-05, 0.000106924308803713, &#xA;0.000107929018751166, 0.000116314737750083, 0.000121805424073131, &#xA;0.000104060063751203, 0.000173710084683786, 0.000197140244520663, &#xA;0.000255164374909145, 0.000273300166932535, 0.000275347732934516&#xA;), adj.P.Val = c(0.00727510763230165, 0.0198807293341929, 0.0198807293341929, &#xA;0.0198807293341929, 0.0357217668827299, 0.0262203124576875, 0.06970505049172, &#xA;0.0262203124576875, 0.0858444785191522, 0.09421086799483, 0.0858444785191522, &#xA;0.0858444785191522, 0.0968123298197961, 0.0968123298197961, 0.0968123298197961, &#xA;0.0968123298197961, 0.0968123298197961, 0.098333518854239, 0.098333518854239, &#xA;0.0968123298197961, 0.130401412989609, 0.138392451653505, 0.150726778618736, &#xA;0.150726778618736, 0.150726778618736), B = c(5.59649208873277, &#xA;4.23844568248881, 3.95926511142815, 3.85011503575047, 2.70095619489428, &#xA;2.52151007226572, 2.02920447371096, 1.82077681362955, 1.73104697407381, &#xA;1.44381690648743, 1.32264026808754, 1.26349133643967, 1.17476645994761, &#xA;1.17423988785926, 1.13844103561967, 1.07813935334313, 1.06670248721183, &#xA;1.0004861785984, 0.949862540556422, 0.634079467108277, 0.620590447127368, &#xA;0.489071423128816, 0.29054426889472, 0.254833988789798, 0.246174538031329&#xA;)), row.names = c(&amp;quot;ENSG00000188938&amp;quot;, &amp;quot;ENSG00000182768&amp;quot;, &amp;quot;ENSG00000174579&amp;quot;, &#xA;&amp;quot;ENSG00000119523&amp;quot;, &amp;quot;ENSG00000271798&amp;quot;, &amp;quot;ENSG00000204923&amp;quot;, &amp;quot;ENSG00000281910&amp;quot;, &#xA;&amp;quot;ENSG00000271092&amp;quot;, &amp;quot;ENSG00000153767&amp;quot;, &amp;quot;ENSG00000116903&amp;quot;, &amp;quot;ENSG00000168101&amp;quot;, &#xA;&amp;quot;ENSG00000206834&amp;quot;, &amp;quot;ENSG00000180035&amp;quot;, &amp;quot;ENSG00000251733&amp;quot;, &amp;quot;ENSG00000058729&amp;quot;, &#xA;&amp;quot;ENSG00000132467&amp;quot;, &amp;quot;ENSG00000212464&amp;quot;, &amp;quot;ENSG00000115561&amp;quot;, &amp;quot;ENSG00000230669&amp;quot;, &#xA;&amp;quot;ENSG00000206612&amp;quot;, &amp;quot;ENSG00000275272&amp;quot;, &amp;quot;ENSG00000162139&amp;quot;, &amp;quot;ENSG00000143889&amp;quot;, &#xA;&amp;quot;ENSG00000207233&amp;quot;, &amp;quot;ENSG00000119878&amp;quot;), class = &amp;quot;data.frame&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15996" LastActivityDate="2023-02-13T18:01:24.463" Title="Error of duplicated rownames although there are no duplicates" Tags="|r|gene|ensembl|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19585" PostTypeId="2" ParentId="19584" CreationDate="2022-08-21T13:45:52.727" Score="4" Body="&lt;p&gt;I found an answer shortly after posting this question:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ensembl.genes &amp;lt;- rownames(df)&#xA;geneIDs &amp;lt;- ensembldb::select(EnsDb.Hsapiens.v86, keys= ensembl.genes, keytype = &amp;quot;GENEID&amp;quot;, columns = c(&amp;quot;SYMBOL&amp;quot;,&amp;quot;GENEID&amp;quot;))&#xA;dupli = duplicated(geneIDs&lt;span class=&quot;math-container&quot;&gt;$SYMBOL)&#xA;geneIDs = geneIDs[!dupli,]&#xA;ind = intersect(rownames(df), geneIDs$&lt;/span&gt;GENEID)&#xA;df = df[ind,]&#xA;rownames(df) = geneIDs$SYMBOL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15996" LastActivityDate="2022-08-21T13:45:52.727" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19586" PostTypeId="1" AcceptedAnswerId="19594" CreationDate="2022-08-22T15:48:51.953" Score="1" ViewCount="839" Body="&lt;p&gt;I have the results of cellranger analysis for single-nucleus RNA seq data that was done by someone else. So I do not know which parameters were used for cellranger. In the results, there are 77000 estimated cells (which is way too high) and around 800 median reads per cell. Those are filtered counts. Raw &lt;code&gt;feature_bc_matrix&lt;/code&gt; has almost 2 million cells.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4Lcsd.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4Lcsd.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;How to tune cellranger count to get lower number of cells?&lt;/p&gt;&#xA;" OwnerUserId="16022" LastEditorUserId="16022" LastEditDate="2022-08-23T13:36:01.883" LastActivityDate="2022-08-23T18:15:14.077" Title="Cellranger results have too many cells" Tags="|scrnaseq|single-cell|10x-genomics|cellranger|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19587" PostTypeId="1" CreationDate="2022-08-23T02:52:55.723" Score="2" ViewCount="455" Body="&lt;p&gt;I noticed the tutorials that Scanpy and Seurat use do not demonstrate doublet removal in their down stream analysis.&#xA;Is the dataset output of cellranger count already doublet removed or do I need to incorporate doublet removal before doing quality control etc?&lt;/p&gt;&#xA;" OwnerUserId="15919" LastEditorUserId="15919" LastEditDate="2022-08-23T05:00:28.207" LastActivityDate="2022-08-23T20:04:42.907" Title="Are the doublets in the tutorial dataset in Scanpy/ Seurat already filtered?" Tags="|scrnaseq|transcriptome|10x-genomics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19588" PostTypeId="1" CreationDate="2022-08-23T04:02:16.423" Score="1" ViewCount="74" Body="&lt;p&gt;I tried to analyze single-cell RNA sequencing data from PMID: 32849601. I downloaded fastq files. I also made a canine genome reference from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly/GCF_000002285.3/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/assembly/GCF_000002285.3/&lt;/a&gt; according to a guideline from cellranger (&lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/advanced/references#mkref-intro&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/advanced/references#mkref-intro&lt;/a&gt;). I ran cellranger count as usual and created Seurat objects using features, barcodes, and matrix files based on the Seurat tutorial without modification.&lt;/p&gt;&#xA;&lt;p&gt;I found all gene name has the suffix gene &amp;quot;-4&amp;quot; as below. [![enter image description here][1]][1].&lt;/p&gt;&#xA;&lt;p&gt;Which step should I correct to have a plain gene name without the suffix? Since I have never found this issue before, it's really interesting. Is this related to the step of making a reference genome?&#xA;[1]: &lt;a href=&quot;https://i.stack.imgur.com/oF1oi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://i.stack.imgur.com/oF1oi.png&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="8657" LastActivityDate="2022-08-23T04:02:16.423" Title="Why my Seurat object has gene name as &quot;gene-number&quot;" Tags="|seurat|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19589" PostTypeId="1" CreationDate="2022-08-23T06:14:35.780" Score="1" ViewCount="125" Body="&lt;p&gt;I’m standing up a production bioinformatics pipeline that’ll be producing standard VCF files. I’d like to store this data in an SQL database that ingest the VCFs and allow me and my team to easily query and visualize many VCFs. Because multiple people will have access to the data, roles/permissions need to be built in. I don’t want to create this system from scratch, but it needs to be both robust and intuitive. Are there any products like this available today?&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;[Edit]:&lt;/em&gt; The answers provided to &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/15248/how-to-import-large-bed-gff-vcf-paf-sam-files-into-an-sql-database&quot;&gt;this question&lt;/a&gt; are on the right track, but only provide a small slice of the functionality I'm looking for. For starters, I'm looking for a hosted/managed solution; something potentially like &lt;a href=&quot;https://www.illumina.com/products/by-type/informatics-products/basespace-sequence-hub.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Basespace&lt;/a&gt;, but more user-friendly and not quite so bloated.&lt;/p&gt;&#xA;&lt;p&gt;Additionally, the linked solutions are mostly about getting vcfs into a db, but don't make much headway into providing exploration/visualization tooling.&lt;/p&gt;&#xA;" OwnerUserId="800" LastEditorUserId="800" LastEditDate="2022-08-24T23:12:41.867" LastActivityDate="2022-09-22T04:01:57.977" Title="Production database system for storing vcf records" Tags="|vcf|database|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19590" PostTypeId="1" CreationDate="2022-08-23T10:40:52.783" Score="5" ViewCount="1395" Body="&lt;p&gt;In the literature, it specifically states that AlphaFold has &amp;quot;Has not been trained to predict structural consequences of &lt;strong&gt;point mutations&lt;/strong&gt;&amp;quot;. See : &lt;a href=&quot;https://alphafold.com/faq&quot; rel=&quot;noreferrer&quot;&gt;https://alphafold.com/faq&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;AlphaFold has not been validated for predicting the effect of mutations. In particular, AlphaFold is not expected to produce an unfolded protein structure given a sequence containing a destabilising point mutation&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What I want to understand, is why?&lt;/p&gt;&#xA;&lt;p&gt;If I change &lt;strong&gt;one amino acid&lt;/strong&gt;, and it's &lt;strong&gt;not destabilising&lt;/strong&gt;, why can't AlphaFold use its training and form a moderately confident structure? I mean, to a computer, what's the difference between a wild-type and a mutated protein? I know what you may be thinking, why don't you just &lt;strong&gt;try it&lt;/strong&gt; yourself, and see if AlphaFold works. But if I don't have the &lt;strong&gt;experimental structure&lt;/strong&gt; of the mutant, how on earth am I supposed to believe that this is even close to the theoretical real thing?&lt;/p&gt;&#xA;&lt;p&gt;Maybe I'm not understanding the &lt;strong&gt;main premise&lt;/strong&gt;, but it's something that's been bugging me for a while now. Any insight into this would be much appreciated.&lt;/p&gt;&#xA;" OwnerUserId="16026" LastEditorUserId="73" LastEditDate="2022-08-24T06:17:32.637" LastActivityDate="2022-08-25T16:22:26.553" Title="Why can't AlphaFold predict the consequences of point-mutations?" Tags="|protein-structure|mutations|modelling|deep-learning|point-mutation|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19591" PostTypeId="1" CreationDate="2022-08-23T10:52:11.773" Score="1" ViewCount="47" Body="&lt;p&gt;I am planning to add colocalization method as a post-GWAS analysis in my study which is solely based on east Asian populations. I have noticed that &lt;strong&gt;coloc&lt;/strong&gt;, &lt;strong&gt;enloc&lt;/strong&gt; and &lt;strong&gt;eCaviar&lt;/strong&gt; are being commonly used for colocalization analysis but all of which are being conducted in European GWAS or European dominant meta-analyses. Some stated that GTEx eQTL data, which might be a part of requirements in these tools, was mainly generated from European populations, and thus, it would be difficult to get the accurate results in Asian populations because of LD differences. So, if anybody who had similar experiences or who did colocalization methods in Asian GWAS is here, please let me know how to deal with this.&lt;/p&gt;&#xA;" OwnerUserId="12699" LastActivityDate="2022-08-23T10:52:11.773" Title="Applying colocalization analysis in Asian populations" Tags="|gwas|software-usage|meta-analysis|gtex|qtls|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19592" PostTypeId="2" ParentId="19570" CreationDate="2022-08-23T13:28:04.517" Score="4" Body="&lt;p&gt;I do not know if there is a function in Biopython.PDB for dihedral space transformations, but due to the lack of answers I would say no. Biopython.PDB is nice and clean, but I would use something with more molecular mechanics power... or should I say potential (:drum::laughing:).&lt;/p&gt;&#xA;&lt;p&gt;Personally, I use PyRosetta, but it is not open source (code available to academics), so others may opt for OpenForceField.&#xA;PyRosetta requires a licence (free for academia), which is different from the Rosetta proper licence.&lt;/p&gt;&#xA;&lt;p&gt;The code below uses a helper module I wrote, so I'll use the installer therein (which is shorthand for the official pip wheel, which provides different options etc., while the official shorthand is actually &lt;code&gt;pyrosettacolabsetup&lt;/code&gt;)&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;pip3 install pyrosetta-help&#xA;install_pyrosetta -u  -p &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Ligands?&lt;/h2&gt;&#xA;&lt;p&gt;First lets make sure there's nothing interesting ligand wise.&lt;/p&gt;&#xA;&lt;p&gt;NB. RCSB has a new shiny API which may be better... Not tried it: currently using European.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import requests&#xA;&#xA;def retrieve_nonpolymer_data(entry_id: str) -&amp;gt; dict:&#xA;    &amp;quot;&amp;quot;&amp;quot;cf. https://www.ebi.ac.uk/pdbe/api/doc/&amp;quot;&amp;quot;&amp;quot;&#xA;    response: requests.Response = requests.get(f' https://www.ebi.ac.uk/pdbe/api/pdb/entry/molecules/{entry_id.lower()}')&#xA;    response.raise_for_status()&#xA;    return response.json()[entry_id.lower()]&#xA;&#xA;info = retrieve_nonpolymer_data('1enh')&#xA;chem_resns = [c for entity in info for c in entity.get('chem_comp_ids', [])]&#xA;print(chem_resns)  # ['HOH']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If is were something that is not water, I made a &lt;a href=&quot;https://blog.matteoferla.com/2019/11/go-away-glycerol.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;list here&lt;/a&gt;, which is based off Peter Curran's repo for the HotSpots algorithm, which allows the elimination of worthless ligands like glycol.&lt;/p&gt;&#xA;&lt;p&gt;I mention this because the route differs if there is a ligand.&lt;/p&gt;&#xA;&lt;h2&gt;Init and read&lt;/h2&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pyrosetta&#xA;import pyrosetta_help as ph&#xA;from types import ModuleType&#xA;&#xA;logger = ph.configure_logger()&#xA;pyrosetta.init(extra_options=ph.make_option_string(no_optH=False,&#xA;                                                ex1=None,&#xA;                                                ex2=None,&#xA;                                                #mute='all',&#xA;                                                ignore_unrecognized_res=True,&#xA;                                                load_PDB_components=False,&#xA;                                                ignore_waters=True)&#xA;                               )&#xA;&#xA;# shorthand for sanity&#xA;prc: ModuleType = pyrosetta.rosetta.core&#xA;&#xA;# this route strips ligands&#xA;pose: Pyrosetta.Pose = pyrosetta.toolbox.pose_from_rcsb('1enh')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note &lt;code&gt;ignore_unrecognized_res&lt;/code&gt;, &lt;code&gt;load_PDB_components&lt;/code&gt; and &lt;code&gt;ignore_waters&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Angles&lt;/h2&gt;&#xA;&lt;p&gt;This will not happen to an non-Rosetta user, but I get tripped out by the fact that ψ &amp;amp; φ data is not stored in residue, but in the pose.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;for residue in pose.residues:  #: prc.conformation.Residue&#xA;    # residue.get_phi() # ---&amp;gt; Error!&#xA;    pass&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For the residue index in pose, Rosetta was written in Fortran so is one-indexed.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np&#xA;import numpy.typing as npt&#xA;phi:npt.NDArray[float] = np.fromiter(map(pose.phi, range(1, pose.total_residue()+1)), dtype=float)&#xA;psi:npt.NDArray[float] = np.fromiter(map(pose.psi, range(1, pose.total_residue()+1)), dtype=float)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Change φ/ψ angles&lt;/h2&gt;&#xA;&lt;p&gt;Time for some fun in dihedral space, say:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;r = 2&#xA;phi = 180&#xA;pose.set_phi(r, phi)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For modelling, I wrote some code for setting spans with the archetypical angles for different SS: &lt;a href=&quot;https://pyrosetta-help.readthedocs.io/en/latest/pyrosetta_help.common_ops.html?highlight=helix#pyrosetta_help.common_ops.ss_changes.make_sheet&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I do not recall whence the data came (maybe Wikipedia?), but it may be handy anyway.&lt;/p&gt;&#xA;&lt;h2&gt;Save&lt;/h2&gt;&#xA;&lt;p&gt;To save a PDB file:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;pose.dump_pdb('.pdb')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There is also &lt;code&gt;pose.dump_scored_pdb&lt;/code&gt;, but that is only for friends of J. Willard Gibbs.&lt;/p&gt;&#xA;&lt;h2&gt;Minimisation&lt;/h2&gt;&#xA;&lt;p&gt;It is said in the comments that energy minimisation is not required. But here is a snippet that minimises the sidechains only&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;scorefxn = pyrosetta.get_fa_scorefxn()  # ref2015 will do.&#xA;before: float = scorefxn(pose)&#xA;cycles = 5&#xA;movemap = pyrosetta.MoveMap()&#xA;movemap.set_bb(False)  # backbone&#xA;movemap.set_chi(True)  # sidechain&#xA;relax = pyrosetta.rosetta.protocols.relax.FastRelax(scorefxn, cycles)&#xA;relax.set_movemap(movemap)&#xA;relax.apply(pose)&#xA;after: float = scorefxn(pose)&#xA;print(after - before)  # Willard says &amp;quot;Remember free energy is a potential so it should be negative&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If the BB tweaks are minimal, there's also the option of scoring/minimising against the electron density etc.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="15987" LastEditDate="2022-08-23T19:10:37.830" LastActivityDate="2022-08-23T19:10:37.830" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19593" PostTypeId="2" ParentId="19590" CreationDate="2022-08-23T14:11:16.437" Score="1" Body="&lt;p&gt;This answer is from common knowledge rather than specific infield expertise. The question could be more expertly answered by other members.&lt;/p&gt;&#xA;&lt;p&gt;If you really want to pursue this type of calculation, it's possible that something like &lt;code&gt;INPS-MD&lt;/code&gt; &lt;em&gt;de novo&lt;/em&gt; -&amp;gt; protein structure stability from variants will provide an answer. So maybe compare the outputs with &lt;code&gt;AlphaFold&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Intuitively, I don't think it's cool, just my opinion. In my opinion, an experimentally derived structure is required as the basis for your calculation, where you want to consider the impact of an amino acid mutation.&lt;/p&gt;&#xA;&lt;p&gt;It's important to separate &lt;em&gt;de novo&lt;/em&gt; prediction from variant prediction. Alphafold is &lt;em&gt;de novo&lt;/em&gt; prediction. What you are looking at is structure stability from a variant (see Summary). That's my core answer, these different calculations. To explain this more clearly ...&lt;/p&gt;&#xA;&lt;p&gt;Good variant stability calculators are DUET (2014) and &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/32/16/2542/1743481?login=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;INPS-MD&lt;/a&gt; (2016), amongst others. Just to reiterate, a good starting point is an experimentally derived protein structure.&lt;/p&gt;&#xA;&lt;p&gt;The webservers are here:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;http://structure.bioc.cam.ac.uk/duet&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://structure.bioc.cam.ac.uk/duet&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://inpsmd.biocomp.unibo.it/inpsSuite&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://inpsmd.biocomp.unibo.it/inpsSuite&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I personally don't know &lt;code&gt;Alphafold&lt;/code&gt; but understand its predecessors which accommodate convolutions by leveraging deep-learning. The basis of the predecessor's calculations firstly are &lt;em&gt;de novo&lt;/em&gt; prediction and secondly can miss the impact of variants if they are not directly involved in the core-algorithm. It's about identifying which amino acids are interacting considering all available homologous amino acids within a database. The calculation can predict local interactions with extreme accuracy, but it's a prediction. It will also depend on whether there are single amino acid changes or two- or more amino acid changes (they could compensate each other, particularly for a natural variant).&lt;/p&gt;&#xA;&lt;p&gt;In contrast, DUET and INPS-MD, amongst a lot of other similar programs, consider the property shift of an individual amino acid in relation to its immediate local stereo-chemical environment measured by a shift of free-energy.&lt;/p&gt;&#xA;&lt;p&gt;Essentially, these are two sets of very different calculations.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; The problem I intuitively see is if &lt;em&gt;de novo&lt;/em&gt; prediction in any form is used, okay they are very good, but they're just predictions. Variant structural predictions are also just predictions  which are very widely respected. Stacking &lt;em&gt;de novo&lt;/em&gt; prediction onto predicting protein structure stability resulting from variants has scope for error, especially if there was some hidden non-linear effect.&lt;/p&gt;&#xA;&lt;p&gt;There could be additional issues because the &lt;em&gt;de novo&lt;/em&gt; calculations immediately prior to &lt;code&gt;Alphafold&lt;/code&gt; don't predict the precise local distances.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;code&gt;AlphaFold&lt;/code&gt; would simply respond&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;'it's not been trained'&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;They'd want a wet-lab experimental dataset with wild-type vs. variants, if these exist with sufficient abundance to prevent over-training. It would be a very skewed dataset so of limited application in my personal opinion.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="298" LastEditDate="2022-08-24T15:43:35.923" LastActivityDate="2022-08-24T15:43:35.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19594" PostTypeId="2" ParentId="19586" CreationDate="2022-08-23T18:15:14.077" Score="2" Body="&lt;p&gt;The Barcode Rank plot shows very high background in your sample.&lt;br /&gt;&#xA;The cell estimation from &lt;code&gt;Cell Ranger&lt;/code&gt; will look for a sudden drop in the number of UMIs and use this to call cells. In your case, the drop is too much to the right because the empty droplet plateau is almost completely merged with the real cells. If you have already looked at &lt;a href=&quot;https://cellbender.readthedocs.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CellBender&lt;/a&gt; your case best corresponds to the rightmost example image.&lt;/p&gt;&#xA;&lt;p&gt;With high background of free mRNA, even empty droplets will contain a significant number of mRNA molecules and become hard to distinguish from real cells.&lt;/p&gt;&#xA;&lt;p&gt;This likely is a consequence of the single nulcei preparation. To isolate the nuclei, the cells necessarily need to be broken and all the cytoplasmic mRNA is freed contributing to the background. You could try removing this background in future experiments by increasing the number of washing steps after cell lysis.&lt;/p&gt;&#xA;&lt;p&gt;Using software such as CellBender, &lt;a href=&quot;https://github.com/constantAmateur/SoupX&quot; rel=&quot;nofollow noreferrer&quot;&gt;SoupX&lt;/a&gt; or &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/DropletUtils.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DropletUtils&lt;/a&gt; may help with identifying and reducing the background. CellBender also will rerun the cell estimation step, automatically adjusting the number of called cells.&lt;/p&gt;&#xA;&lt;p&gt;In any case you can manually reduce the number of cells by filtering based on the minimum number of UMIs or genes. However, the higher the background the more arbitrary such a thresholding strategy will become.&lt;/p&gt;&#xA;" OwnerUserId="4885" LastActivityDate="2022-08-23T18:15:14.077" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19595" PostTypeId="2" ParentId="19587" CreationDate="2022-08-23T19:59:41.507" Score="2" Body="&lt;p&gt;No in general cellranger does not remove doublets. You can apply a tool like &lt;a href=&quot;https://scanpy.readthedocs.io/en/stable/generated/scanpy.external.pp.scrublet.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scrublet&lt;/a&gt; in scanpy or &lt;a href=&quot;https://github.com/chris-mcginnis-ucsf/DoubletFinder&quot; rel=&quot;nofollow noreferrer&quot;&gt;Doubletfinder&lt;/a&gt; in Seurat to infer doublets based on a mixed-transcriptional signal.&lt;/p&gt;&#xA;&lt;p&gt;I say in general because cellranger does detect certain types of doublets in two contexts:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Multi-species experiments, e.g. the so-called &lt;a href=&quot;https://kb.10xgenomics.com/hc/en-us/articles/360001074271-Does-Cell-Ranger-automatically-exclude-doublets-&quot; rel=&quot;nofollow noreferrer&quot;&gt;barnyard experiment&lt;/a&gt;, cellranger detects cross-species doublets&lt;/li&gt;&#xA;&lt;li&gt;With &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/output/cellplex-metrics&quot; rel=&quot;nofollow noreferrer&quot;&gt;CellPlex&lt;/a&gt; sample multiplexing, cellranger detects cross-tag (cross-CMO) doublets&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="104" LastEditorUserId="104" LastEditDate="2022-08-23T20:04:42.907" LastActivityDate="2022-08-23T20:04:42.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19596" PostTypeId="2" ParentId="19590" CreationDate="2022-08-24T00:29:37.477" Score="1" Body="&lt;p&gt;As marcin points out in the comments: by &amp;quot;AphaFold (AF) has not been trained to predict structural consequences of point mutations&amp;quot; it is meant &amp;quot;AF is not able to tell you whether your point mutation is destabilizing or not, and if it is destabilizing then the structure it predicts will most probably be wrong&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;It is totally possible that AF predicts the structure of a protein well when only non destabilizing mutations are introduced. As pointed out in the paragraph marcin quotes, however: &amp;quot;AlphaFold has not been validated for predicting the effect of mutations.&amp;quot; And as long as we don't know, we should be warry of any of AlphaFold predictions with point mutations. Maybe it works, maybe it doesn't, or maybe it's case dependent.&lt;/p&gt;&#xA;" OwnerUserId="12054" LastActivityDate="2022-08-24T00:29:37.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19597" PostTypeId="2" ParentId="19570" CreationDate="2022-08-24T13:46:36.817" Score="4" Body="&lt;p&gt;I answered about doing it with PyRosetta as I assumed that as BioPython PDB module is basic and the fact it was being asked made me think it was not possible.&#xA;Turns out I was wrong. So I am making a separate answer —as I do not actually believe doing it with Bio.PDB is the best solution.&lt;/p&gt;&#xA;&lt;p&gt;The provided code from above with type hints is:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np&#xA;from Bio import PDB&#xA;&#xA;from typing import TypedDict, Dict, Tuple&#xA;&#xA;parser = PDB.PDBParser()&#xA;structure: PDB.Structure.Structure = parser.get_structure(&amp;quot;&amp;quot;, &amp;quot;/Users/matteo/Downloads/1enh.pdb&amp;quot;)&#xA;structure.atom_to_internal_coordinates()&#xA;chain: PDB.Chain.Chain = list(structure.get_chains())[0]&#xA;ic_chain: PDB.internal_coords.IC_Chain = chain.internal_coord&#xA;dihedrals:  Dict[Tuple[PDB.internal_coords.AtomKey, &#xA;                       PDB.internal_coords.AtomKey,&#xA;                       PDB.internal_coords.AtomKey,&#xA;                       PDB.internal_coords.AtomKey],&#xA;                 PDB.internal_coords.Dihedron] = ic_chain.dihedra&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I got the types simply by using &lt;code&gt;type&lt;/code&gt; and the attributes of note with &lt;code&gt;dir&lt;/code&gt;. The latter helps me explore the &lt;code&gt;Bio.PDB.internal_coords.Dihedron&lt;/code&gt; telling me of the &lt;code&gt;angle&lt;/code&gt; attribute:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;d: PDB.internal_coords.Dihedron = list(dihedrals.values())[0]&#xA;d.angle: np.float64&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A change here appears it the main &lt;code&gt;PDB.Structure.Structure&lt;/code&gt; instance, i.e. it is not a disconnected copy.&lt;/p&gt;&#xA;&lt;p&gt;The class &lt;code&gt;PDB.internal_coords.AtomKey&lt;/code&gt; appears to be hard to instantiate by humans and may not work as expected —&lt;code&gt;help&lt;/code&gt; is gibberish so it is a case for &lt;code&gt;inspect.getsource&lt;/code&gt;. So iterating across the dihedrals for the Ramachandran dihedrals is the solution. Now, a φ angle is N-CA-C-N and a ψ is C-N-CA-C (in PDB atom names the carboxylic carbon is &lt;code&gt;C &lt;/code&gt;). When the &lt;code&gt;__str__&lt;/code&gt; method of one of these &lt;code&gt;PDB.internal_coords.AtomKey&lt;/code&gt; instances is called some values are given, which are available in the tuple attribute &lt;code&gt;akl&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;atom_key: PDB.internal_coords.AtomKey = list(ic_chain.dihedra.keys())[0][0]&#xA;print(atom_key)  #3_R_N&#xA;print(atom_key.akl) #('3', None, 'R', 'N', None, None)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The first entry is the residue number and is a string because of insertion codes.&#xA;The fourth entry is the atom name. So iterating across the keys for the correct sequence is the trick —I know there's bounty asking for a complete solution, but it's simple, so anyone care to fill out the blanks and get the meaningless points, please do!&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2022-08-24T13:46:36.817" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19598" PostTypeId="1" CreationDate="2022-08-24T20:23:35.573" Score="2" ViewCount="103" Body="&lt;p&gt;I am trying to reproduce the results from this paper &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/gb-2014-15-6-r88&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;Human genomic regions with exceptionally high levels of population differentiation identified from 911 whole-genome sequences&amp;quot;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Specifically I want to extract  high differentiated  ( or less) SNPs from 1k Genome data set representing high proportion of the population present.&lt;/p&gt;&#xA;&lt;p&gt;The paper mentioned using Vcffixup from Vcflib (&lt;a href=&quot;https://github.com/vcflib/vcflib&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/vcflib/vcflib&lt;/a&gt;) but there might still be other things I am missing.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Update 2 (MWE)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So this is my code snippet.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## to dowload datafile, from linux terminal, use curl or wget&#xA;curl -O https://1000genomes.s3.amazonaws.com/release/20110521/ALL.chr22.phase1_release_v3.20101123.snps_indels_svs.genotypes.vcf.gz&#xA;### unzip the file&#xA;gunzip dowloaded file&#xA;### take the first few lines from the vcf file.&#xA;### since actual file is huge, the name of sfile for testing is &#xA;### myfile.vcf&#xA;head -200 downloadedfile &amp;gt; myfile.vcf &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now the next section was done in python environment (jupyter notebook).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## using the library pysam&#xA;from pysam import VariantFile&#xA;import pandas as pd &#xA;with VariantFile(test_datafile) as vcf_reader:&#xA;for record in vcf_reader:&#xA;    sample_file = record.samples.keys() &#xA;    alleles = [record.samples[x].allele_indices for x in record.samples]&#xA;    break ## just to make sure my data is as intended&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now to the VCF tools part - vcffixup was used in the paper.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## Install vcflib&#xA;   sudo apt-get install libvcflib-tools libvcflib-dev&#xA;### export path&#xA;   export PATH=$PATH:/usr/lib/vcflib/bin &#xA;### refresh enviroment variables&#xA;   source ~/.bashrc &#xA;## run vcffixup&#xA;  vcffixup myfile.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A sample of vcffixup output is hown below:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/R7SlY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/R7SlY.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So the questions questions I have is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;how to compute derived allele frequency differences at all variants for pairs of continent and pairs of population within continent&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="16034" LastEditorUserId="16034" LastEditDate="2022-08-27T00:07:03.860" LastActivityDate="2022-08-27T00:07:03.860" Title="SNPs with high population differentiation from 1k Genome dataset" Tags="|vcf|snp|1000genomes|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19599" PostTypeId="1" CreationDate="2022-08-24T20:39:43.317" Score="1" ViewCount="46" Body="&lt;p&gt;I have run GSEA - on Gene Ontology , Biological Processes (GO and BP hereafter) - on some Mouse bulk RNA-seq data, and am currently in the process of going through the results. To do this, I have used the library &lt;code&gt;fgsea&lt;/code&gt; in the R programming environment.&lt;/p&gt;&#xA;&lt;p&gt;Prior to running the GSEA, in an attempt to refine the results (many many significant gene sets were found), I wish to exclude particular categories in the GO BP tree of gene sets - see the following link: &lt;a href=&quot;http://www.informatics.jax.org/vocab/gene_ontology/GO:0008150&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.informatics.jax.org/vocab/gene_ontology/GO:0008150&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For the purpose of this question, lets assume I would like to exclude all gene sets which fall under the two categories: &lt;em&gt;&lt;strong&gt;pigmentation&lt;/strong&gt;&lt;/em&gt; and &lt;em&gt;&lt;strong&gt;reproduction&lt;/strong&gt;&lt;/em&gt;. Is there any way in which I can obtain &lt;em&gt;all&lt;/em&gt; of the genesets under these two categories, in order to later subset the &lt;code&gt;.gmt&lt;/code&gt; file, prior to running &lt;code&gt;fgsea&lt;/code&gt; ? Essentially, what I mean is, without going one by one and taking down hundreds of gene sets by hand. Is there a resource which contains this hierarchy of GO BP genesets as some sort of downloadable data structure?&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="4001" LastActivityDate="2022-08-24T20:39:43.317" Title="GSEA. Subsetting Gene Ontology, Biological Processes, by gene set families of interest" Tags="|rna-seq|database|public-databases|gsea|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19600" PostTypeId="2" ParentId="19496" CreationDate="2022-08-24T21:18:42.353" Score="0" Body="&lt;p&gt;Here's how you can do the translation yourself, using the genetic code in form of a dictionary from Biopython:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import Bio&#xA;&#xA;# Load the genetic code (a mapping from RNA triplet to amino acid)&#xA;d = Bio.Data.CodonTable.standard_rna_table.forward_table&#xA;print('Genetic code:', d)&#xA;print()&#xA;&#xA;# Use it to translate an RNA sequence into an amino acid sequence&#xA;s = 'CCAAGUACAGAGAUUAAC'&#xA;for triplet in (s[i:i+3] for i in range(0, len(s), 3)):&#xA;    print(triplet, '-&amp;gt;', d[triplet])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Printed result:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Genetic code: {'UUU': 'F', 'UUC': 'F', 'UUA': 'L', 'UUG': 'L', 'UCU': 'S', 'UCC': 'S', 'UCA': 'S', 'UCG': 'S', 'UAU': 'Y', 'UAC': 'Y', 'UGU': 'C', 'UGC': 'C', 'UGG': 'W', 'CUU': 'L', 'CUC': 'L', 'CUA': 'L', 'CUG': 'L', 'CCU': 'P', 'CCC': 'P', 'CCA': 'P', 'CCG': 'P', 'CAU': 'H', 'CAC': 'H', 'CAA': 'Q', 'CAG': 'Q', 'CGU': 'R', 'CGC': 'R', 'CGA': 'R', 'CGG': 'R', 'AUU': 'I', 'AUC': 'I', 'AUA': 'I', 'AUG': 'M', 'ACU': 'T', 'ACC': 'T', 'ACA': 'T', 'ACG': 'T', 'AAU': 'N', 'AAC': 'N', 'AAA': 'K', 'AAG': 'K', 'AGU': 'S', 'AGC': 'S', 'AGA': 'R', 'AGG': 'R', 'GUU': 'V', 'GUC': 'V', 'GUA': 'V', 'GUG': 'V', 'GCU': 'A', 'GCC': 'A', 'GCA': 'A', 'GCG': 'A', 'GAU': 'D', 'GAC': 'D', 'GAA': 'E', 'GAG': 'E', 'GGU': 'G', 'GGC': 'G', 'GGA': 'G', 'GGG': 'G'}&#xA;&#xA;CCA -&amp;gt; P&#xA;AGU -&amp;gt; S&#xA;ACA -&amp;gt; T&#xA;GAG -&amp;gt; E&#xA;AUU -&amp;gt; I&#xA;AAC -&amp;gt; N&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that the same RNA sequence can also be translated into another sequence of amino acids by shifting the reading frame 1 or 2 positions. For example, the RNA sequence &lt;code&gt;'CCAAGUACAGAGAUUAAC'&lt;/code&gt; can be split into different triplets:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;'CCA', 'AGU', 'ACA', ...&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;'CAA', 'GUA', 'CAG', ...&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;'AAG', 'UAC', 'AGA', ...&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In biological systems, an mRNA would have a &lt;a href=&quot;https://en.wikipedia.org/wiki/Start_codon&quot; rel=&quot;nofollow noreferrer&quot;&gt;start codon&lt;/a&gt; that marks the beginning of an open reading frame, which is often the triplet &lt;code&gt;AUG&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15610" LastActivityDate="2022-08-24T21:18:42.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19601" PostTypeId="2" ParentId="19571" CreationDate="2022-08-24T21:25:06.087" Score="0" Body="&lt;p&gt;I'm awarding 'solved' to @gringer, because thats the context of the question.&lt;/p&gt;&#xA;&lt;p&gt;The best solution I could find is stacking &lt;code&gt;Stockholm (sth)&lt;/code&gt;  formats together. In this example the RNA families are associated with one structure:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# STOCKHOLM 1.0&#xA;#=GF ID    UPSK&#xA;AF035635.1/619-641             UGAGUUCUCGAUCUCUAAAAUCG&#xA;M24804.1/82-104                UGAGUUCUCUAUCUCUAAAAUCG&#xA;J04373.1/6212-6234             UAAGUUCUCGAUCUUUAAAAUCG&#xA;M24803.1/1-23                  UAAGUUCUCGAUCUCUAAAAUCG&#xA;#=GC SS_cons                   .AAA....&amp;lt;&amp;lt;&amp;lt;&amp;lt;aaa....&amp;gt;&amp;gt;&amp;gt;&amp;gt;&#xA;//&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;Biopython&lt;/code&gt; allows &lt;code&gt;sth&lt;/code&gt; format to be input.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import AlignIO&#xA;from pathlib import Path&#xA;&#xA;file = 'test.sth'&#xA;path = '/Volumes/mypath/testSth'&#xA;alignment = AlignIO.read(open(Path(path, file)), &amp;quot;stockholm&amp;quot;)&#xA;print(&amp;quot;Alignment length %i&amp;quot; % alignment.get_alignment_length())&#xA;print (alignment.column_annotations['secondary_structure'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Alignment length 23 &#xA;.AAA....&amp;lt;&amp;lt;&amp;lt;&amp;lt;aaa....&amp;gt;&amp;gt;&amp;gt;&amp;gt; # this is the RNA structure &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If we look the &lt;code&gt;alignment&lt;/code&gt; object via&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from pprint import pprint&#xA;pprint(vars(alignment)) # or via print(alignment.__dict__)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;{'_annotations': {},  '&lt;strong&gt;_per_col_annotations&lt;/strong&gt;': {&lt;strong&gt;'secondary_structure'&lt;/strong&gt;:&#xA;'.AAA....&amp;lt;&amp;lt;&amp;lt;&amp;lt;aaa....&amp;gt;&amp;gt;&amp;gt;&amp;gt;'},&lt;br /&gt;&#xA;'_records':&#xA;[SeqRecord(seq=Seq('UGAGUUCUCGAUCUCUAAAAUCG'),&#xA;id='AF035635.1/619-641', name='AF035635.1',&#xA;description='AF035635.1/619-641', dbxrefs=[]),&#xA;SeqRecord(seq=Seq('UGAGUUCUCUAUCUCUAAAAUCG'), id='M24804.1/82-104', name='M24804.1', description='M24804.1/82-104',&#xA;dbxrefs=[]),&#xA;SeqRecord(seq=Seq('UAAGUUCUCGAUCUUUAAAAUCG'), id='J04373.1/6212-6234', name='J04373.1',&#xA;description='J04373.1/6212-6234', dbxrefs=[]),&#xA;SeqRecord(seq=Seq('UAAGUUCUCGAUCUCUAAAAUCG'), id='M24803.1/1-23', name='M24803.1', description='M24803.1/1-23',&#xA;dbxrefs=[])],  'annotations': {}} {&lt;strong&gt;'secondary_structure'&lt;/strong&gt;:&#xA;'.AAA....&amp;lt;&amp;lt;&amp;lt;&amp;lt;aaa....&amp;gt;&amp;gt;&amp;gt;&amp;gt;'}&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The reason &lt;code&gt;vars(alignment)&lt;/code&gt; is required is to internally construct an alignment object in Stockholm format, here's the first part of the skeleton 'pseudo-code',&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.Align import MultipleSeqAlignment&#xA;from Bio.SeqRecord import SeqRecord&#xA;&#xA;outAlign = MultipleSeqAlignment([])&#xA;record in alignment:&#xA;    name = record.id &#xA;    myseq = record.seq&#xA;    sr = SeqRecord(Seq(name), myseq, '', '')&#xA;    outAlign.append(sr)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above code constructs a basic alignment, the bit thats missing is to add the following key/value to the &lt;code&gt;alignment&lt;/code&gt; object of &lt;code&gt;{'secondary_structure': '.AAA....&amp;lt;&amp;lt;&amp;lt;&amp;lt;aaa....&amp;gt;&amp;gt;&amp;gt;&amp;gt;'}&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Its complicated, but you get use to it.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/biopython/biopython/blob/master/Bio/AlignIO/StockholmIO.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/biopython/biopython/blob/master/Bio/AlignIO/StockholmIO.py&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-09-01T20:05:08.297" LastActivityDate="2022-09-01T20:05:08.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19602" PostTypeId="1" CreationDate="2022-08-25T00:54:26.843" Score="1" ViewCount="91" Body="&lt;p&gt;What is the difference between lower case and upper case nucleotides in a sequence? My other question is what are the dots at the end of the sequence? Some examples are shown below:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;GGgG,GGgG,GGgG,GGgG,GGGG,GGGG…&lt;/li&gt;&#xA;&lt;li&gt;GGGG,GGGG,GGGG,GGGG,GGGG,GGGG…&lt;/li&gt;&#xA;&lt;li&gt;GGGG,gggg,GGGG,GGGG,UUUU&lt;/li&gt;&#xA;&lt;li&gt;ggGg,ggGg,ggGg,GGgG,GGgG,GGgG…&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;These sequences are real experimental samples. I obtained them from QuadBase. Here is the link below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://quadbase.igib.res.in/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://quadbase.igib.res.in/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Hopefully you will provide some literature within the thread in order to understand this concept in bioinformatics.&#xA;BTW I'm a beginner in this field and hopefully this&#xA;Bioinformatics Stack Exchange site will help me find out the answers.&lt;/p&gt;&#xA;" OwnerUserId="16035" LastEditorUserId="298" LastEditDate="2022-08-25T12:21:07.817" LastActivityDate="2024-01-17T14:04:20.500" Title="Lower case vs. upper case nucleotids in sequence vs. dots at the end" Tags="|sequence-alignment|genome|ngs|database|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19603" PostTypeId="2" ParentId="19602" CreationDate="2022-08-25T02:21:12.633" Score="0" Body="&lt;ul&gt;&#xA;&lt;li&gt;GGGG,gggg,GGGG,GGGG,UUUU&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Quadbase appears to be about DNA secondary structures, albeit the above example is RNA. The website states,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;G-Quadruplexes or G4s are secondary DNA structures that are now established to have functionally crucial roles in various cellular processes.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Two guesses, firstly its all just a bit weird, secondly its a representation of hypoxanthine. Its more likely to be the second guess than the first.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The nomenclature is a bit weird: gggg with UUUU would infer non-canonical binding/bounding in &lt;a href=&quot;https://www.tbi.univie.ac.at/RNA/ViennaRNA/doc/html/rna_structure_notations.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;WUSS&lt;/a&gt; (Washington University Secondary Structure) format for a psuedoknot  (type of secondary structure). If that wasn't the interpretation it would be kinda weird to share nomenclature with a major secondary structure format (WUSS) for secondary structure notation.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The 'dots', i.e. '.' appear to represent continuation, although in dot-bracket format (secondary structure format) this represents a nucleotide that doesn't hybridise.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Its worth reading the paper for hypoxanthine representation (normally its 'I'), because 'I' will behave like a guanine residue and would then make sense if it was being represented as 'g'. Thus, the following means no hypoxanthine,&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;GGGG,GGGG,GGGG,GGGG,GGGG,GGGG…&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;My bet is its point 2, otherwise it doesn't make sense.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-08-25T13:01:34.710" LastActivityDate="2022-08-25T13:01:34.710" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19604" PostTypeId="2" ParentId="19570" CreationDate="2022-08-25T09:46:34.193" Score="5" Body="&lt;p&gt;ok, here my input pdb, &lt;code&gt;example_short.pdb&lt;/code&gt;  :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATOM      1  N   MET A   1      14.067  31.664  -1.639  1.00 80.87           N&#xA;ATOM      2  HN1 MET A   1      14.250  31.259  -0.699  1.00  0.00           H&#xA;ATOM      3  HN2 MET A   1      14.891  32.220  -1.945  1.00  0.00           H&#xA;ATOM      4  HN3 MET A   1      13.229  32.279  -1.596  1.00  0.00           H&#xA;ATOM      5  CA  MET A   1      13.824  30.564  -2.612  1.00 79.17           C&#xA;ATOM      6  HA  MET A   1      14.702  29.921  -2.671  1.00  0.00           H&#xA;ATOM      7  C   MET A   1      13.556  31.158  -3.989  1.00 78.52           C&#xA;ATOM      8  O   MET A   1      13.301  32.365  -4.085  1.00 79.40           O&#xA;ATOM      9  CB  MET A   1      12.634  29.724  -2.148  1.00 83.38           C&#xA;ATOM     10  HB1 MET A   1      12.600  29.755  -1.059  1.00  0.00           H&#xA;ATOM     11  HB2 MET A   1      11.725  30.170  -2.552  1.00  0.00           H&#xA;ATOM     12  CG  MET A   1      12.680  28.264  -2.580  1.00 85.71           C&#xA;ATOM     13  HG1 MET A   1      12.066  28.145  -3.473  1.00  0.00           H&#xA;ATOM     14  HG2 MET A   1      12.274  27.650  -1.776  1.00  0.00           H&#xA;ATOM     15  SD  MET A   1      14.354  27.697  -2.951  1.00100.24           S&#xA;ATOM     16  CE  MET A   1      15.137  27.675  -1.338  1.00 97.10           C&#xA;ATOM     17  HE1 MET A   1      15.118  28.678  -0.913  1.00  0.00           H&#xA;ATOM     18  HE2 MET A   1      14.597  26.991  -0.683  1.00  0.00           H&#xA;ATOM     19  HE3 MET A   1      16.170  27.342  -1.441  1.00  0.00           H&#xA;ATOM     20  N   GLU A   2      13.604  30.306  -5.018  1.00 75.32           N&#xA;ATOM     21  H   GLU A   2      13.628  29.291  -4.794  1.00  0.00           H&#xA;ATOM     22  CA  GLU A   2      13.626  30.684  -6.444  1.00 72.66           C&#xA;ATOM     23  HA  GLU A   2      13.508  29.732  -6.961  1.00  0.00           H&#xA;ATOM     24  C   GLU A   2      14.903  31.384  -6.843  1.00 65.44           C&#xA;ATOM     25  O   GLU A   2      15.489  31.127  -7.892  1.00 67.46           O&#xA;ATOM     26  CB  GLU A   2      12.516  31.672  -6.810  1.00 72.71           C&#xA;ATOM     27  HB1 GLU A   2      12.094  32.068  -5.886  1.00  0.00           H&#xA;ATOM     28  HB2 GLU A   2      12.960  32.486  -7.383  1.00  0.00           H&#xA;ATOM     29  CG  GLU A   2      11.397  31.081  -7.622  1.00 77.72           C&#xA;ATOM     30  HG1 GLU A   2      10.901  31.877  -8.178  1.00  0.00           H&#xA;ATOM     31  HG2 GLU A   2      11.809  30.354  -8.321  1.00  0.00           H&#xA;ATOM     32  CD  GLU A   2      10.385  30.393  -6.731  1.00 79.89           C&#xA;ATOM     33  OE1 GLU A   2       9.171  30.574  -6.987  1.00 82.69           O&#xA;ATOM     34  OE2 GLU A   2      10.824  29.702  -5.779  1.00 72.73           O&#xA;ATOM     35  N   ASN A   3      15.194  32.399  -6.043  1.00 60.12           N&#xA;ATOM     36  H   ASN A   3      14.712  32.444  -5.123  1.00  0.00           H&#xA;ATOM     37  CA  ASN A   3      16.140  33.452  -6.363  1.00 56.21           C&#xA;ATOM     38  HA  ASN A   3      16.072  33.715  -7.419  1.00  0.00           H&#xA;ATOM     39  C   ASN A   3      17.548  32.968  -6.047  1.00 50.40           C&#xA;ATOM     40  O   ASN A   3      18.513  33.666  -6.331  1.00 50.95           O&#xA;ATOM     41  CB  ASN A   3      15.821  34.667  -5.493  1.00 57.10           C&#xA;ATOM     42  HB1 ASN A   3      15.913  34.381  -4.445  1.00  0.00           H&#xA;ATOM     43  HB2 ASN A   3      16.538  35.457  -5.717  1.00  0.00           H&#xA;ATOM     44  CG  ASN A   3      14.425  35.196  -5.732  1.00 62.07           C&#xA;ATOM     45  OD1 ASN A   3      14.220  36.001  -6.641  1.00 69.12           O&#xA;ATOM     46  ND2 ASN A   3      13.464  34.748  -4.923  1.00 57.21           N&#xA;ATOM     47 1HD2 ASN A   3      12.486  35.079  -5.045  1.00  0.00           H&#xA;ATOM     48 2HD2 ASN A   3      13.693  34.068  -4.170  1.00  0.00           H&#xA;ATOM     49  N   PHE A   4      17.643  31.764  -5.492  1.00 47.14           N&#xA;ATOM     50  H   PHE A   4      16.781  31.193  -5.378  1.00  0.00           H&#xA;ATOM     51  CA  PHE A   4      18.914  31.219  -5.038  1.00 46.77           C&#xA;ATOM     52  HA  PHE A   4      19.696  31.915  -5.340  1.00  0.00           H&#xA;ATOM     53  C   PHE A   4      19.239  29.888  -5.695  1.00 48.73           C&#xA;ATOM     54  O   PHE A   4      18.490  28.919  -5.594  1.00 49.50           O&#xA;ATOM     55  CB  PHE A   4      18.922  31.095  -3.507  1.00 42.32           C&#xA;ATOM     56  HB1 PHE A   4      18.091  30.456  -3.207  1.00  0.00           H&#xA;ATOM     57  HB2 PHE A   4      19.862  30.635  -3.201  1.00  0.00           H&#xA;ATOM     58  CG  PHE A   4      18.787  32.415  -2.806  1.00 39.42           C&#xA;ATOM     59  CD1 PHE A   4      17.545  32.878  -2.408  1.00 40.76           C&#xA;ATOM     60  HD1 PHE A   4      16.673  32.232  -2.509  1.00  0.00           H&#xA;ATOM     61  CD2 PHE A   4      19.881  33.258  -2.648  1.00 35.13           C&#xA;ATOM     62  HD2 PHE A   4      20.857  32.941  -3.015  1.00  0.00           H&#xA;ATOM     63  CE1 PHE A   4      17.397  34.138  -1.889  1.00 36.49           C&#xA;ATOM     64  HE1 PHE A   4      16.400  34.520  -1.669  1.00  0.00           H&#xA;ATOM     65  CE2 PHE A   4      19.757  34.474  -2.043  1.00 32.88           C&#xA;ATOM     66  HE2 PHE A   4      20.640  35.090  -1.873  1.00  0.00           H&#xA;ATOM     67  CZ  PHE A   4      18.507  34.929  -1.643  1.00 43.95           C&#xA;ATOM     68  HZ  PHE A   4      18.401  35.892  -1.144  1.00  0.00           H&#xA;END&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;here my code :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import PDB&#xA;&#xA;from Bio.PDB import PICIO, PDBIO&#xA;&#xA;from typing import TypedDict, Dict, Tuple&#xA;&#xA;&#xA;&#xA;parser = PDB.PDBParser(PERMISSIVE=1, QUIET=1)&#xA;&#xA;structure: PDB.Structure.Structure = parser.get_structure(&amp;quot;prova&amp;quot;, &amp;quot;example_short.pdb&amp;quot;)&#xA;&#xA;structure.atom_to_internal_coordinates()&#xA;chain: PDB.Chain.Chain = list(structure.get_chains())[0]&#xA;ic_chain: PDB.internal_coords.IC_Chain = chain.internal_coord&#xA;d:  Dict[Tuple[PDB.internal_coords.AtomKey, &#xA;                       PDB.internal_coords.AtomKey,&#xA;                       PDB.internal_coords.AtomKey,&#xA;                       PDB.internal_coords.AtomKey],&#xA;                 PDB.internal_coords.Dihedron] = ic_chain.dihedra&#xA;&#xA;&#xA;&#xA;cnt = 1&#xA;for key in d:&#xA;&#xA;    &#xA;    if key[0].akl[3] == 'N':&#xA;        if key[1].akl[3] == 'CA':&#xA;            if key[2].akl[3] == 'C':&#xA;                if key[3].akl[3] == 'N':&#xA;        &#xA;                    print ('\n',cnt,' :   ',  [x.akl[3] for x in key], d[key].angle)&#xA;                    &#xA;                    d[key].angle += 45&#xA;        &#xA;                    cnt += 1&#xA;&#xA;cnt = 1&#xA;for key in d:&#xA;&#xA;    &#xA;    if key[0].akl[3] == 'N':&#xA;        if key[1].akl[3] == 'CA':&#xA;            if key[2].akl[3] == 'C':&#xA;                if key[3].akl[3] == 'N':&#xA;        &#xA;                    print ('\n',cnt,' :   ',  [x.akl[3] for x in key], d[key].angle)&#xA;        &#xA;                    cnt += 1&#xA;                    &#xA;structure.internal_to_atom_coordinates(verbose = True)&#xA;&#xA;io = PDBIO()&#xA;&#xA;io.set_structure(structure)&#xA;&#xA;io.save('atom_coord.pdb',  preserve_atom_numbering=True) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;here an image of the original vs new quadripeptid (cartoon repr, just backbone) :&lt;/p&gt;&#xA;&lt;p&gt;original is rainbowed one (blue to red), magenta is the new one&lt;/p&gt;&#xA;&lt;p&gt;saved as &lt;code&gt;atom_coord.pdb&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3OtQt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3OtQt.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I believe I changed φ (Phi) adding 45° since&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Now, a φ angle is N-CA-C-N&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is worth mentioning that changing&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; `d[key].angle += 45` to `d[key].angle += 6645`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;works too. Apparently, need confirmation from someone more knowledgeable,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;`structure.internal_to_atom_coordinates`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;can cope with more than 360° angle values.&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2022-08-25T13:06:26.193" LastActivityDate="2022-08-25T13:06:26.193" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19605" PostTypeId="1" CreationDate="2022-08-25T09:57:42.983" Score="2" ViewCount="29" Body="&lt;p&gt;A question to those who have experience with &lt;a href=&quot;https://github.com/muellan/metacache#detailed-installation-instructions&quot; rel=&quot;nofollow noreferrer&quot;&gt;MetaCache (metagenomic tool)&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;I have tried to build the MetaCache taxonomy from refseq (only bacteria), &lt;a href=&quot;https://github.com/muellan/metacache/blob/master/docs/mode_build.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;as described in the EXAMPLES here&lt;/a&gt;. However, the process turned out to be excruciatingly long, and I eventually gave up waiting after a week, reaching the progress of only 45%. I am using a 32-core server and the MetaCache seemed to make use of at least 16 cores, so I am puzzled by such slowness.&lt;/p&gt;&#xA;&lt;p&gt;Is this normal behavior? Are there ways to do this faster or pre-built (relatively recent) databases? Is there a specific MetaCache forum, where I could pose a question to the developers?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://academic.oup.com/bioinformatics/article/33/23/3740/4083578&quot; rel=&quot;nofollow noreferrer&quot;&gt;MetaCache&lt;/a&gt; is a classifier in the Kraken mould. Here's the blurb.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Metagenomic shotgun sequencing studies are becoming increasingly popular with prominent examples including the sequencing of human microbiomes and diverse environments.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;As a result, MetaCache consumes significantly less memory compared to the state-of-the-art read classifiers Kraken and CLARK while achieving highly competitive sensitivity and precision at comparable speed.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="1813" LastEditorUserId="3848" LastEditDate="2022-08-25T12:36:49.057" LastActivityDate="2022-08-25T12:36:49.057" Title="Building database for MetaCache takes extremely long" Tags="|database|software-installation|public-dataset|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19606" PostTypeId="2" ParentId="19590" CreationDate="2022-08-25T15:12:32.343" Score="3" Body="&lt;p&gt;Other have addressed why AlphaFold2 cannot really get the necessary signal from MSA (Multiple Sequence Alignment) and memory to properly model a variant, but I thought I'd add my observations/rants.&lt;/p&gt;&#xA;&lt;p&gt;I have modelled &lt;em&gt;in silico&lt;/em&gt; both engineering and clinical variants in the past, using PyRosetta, MD and other tools. For me, AlphaFold2 has really stood out but in a different way than the hype.&lt;/p&gt;&#xA;&lt;h2&gt;Variants can have complicated effects&lt;/h2&gt;&#xA;&lt;p&gt;I developed a tool Venus, &lt;a href=&quot;https://venus.sgc.ox.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://venus.sgc.ox.ac.uk/&lt;/a&gt;, which collates different pieces of information to help figure out the effect a variant was having beyond a simple ∆∆G calculation. This is because often a variant is having a variety of effects, for example a stable but catalytically dead variant can still form a complex and sequester a critical component (e.g. in &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/31353023/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ref&lt;/a&gt; POLR2E-sequestering POR2A is more pathogenic than a simple truncation). This is not only true with human clinical variants, but also with engineering variants: for one protein I have worked on, DogCatcher, lowering the isoelectric point away from neutral pH was critical, not the stabilisation of the transition state (&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/34324879/&quot; rel=&quot;nofollow noreferrer&quot;&gt;ref&lt;/a&gt;). Another example is that the entrance loops in cytochromes drive selectivity more so than the active site configuration.&lt;/p&gt;&#xA;&lt;h2&gt;The unknowns: alt conformers and complexes&lt;/h2&gt;&#xA;&lt;p&gt;I will be the first to admit that most folk want a deleterious ∆∆G value and will p-value shop for it and not catering for this is a bad idea for a publication. But in terms of mechanism, the biggest issue I face are dynamic processes involving a residue where the other active/inactive conformer is unknown or when the residue is a surface residue possible at the interface in a complex, which is known simply to exist from a coIP western. No nearby gnomAD variants or presence of post-translational sites or sequence conservation do show that something may be important but not why.&lt;/p&gt;&#xA;&lt;h2&gt;Enter AlphaFold2&lt;/h2&gt;&#xA;&lt;p&gt;AlphaFold2, via the more user-friendly ColabFold, allow the creation of wild type monomers, homo-oligomers, but also hetero-oligomers (complexes) and alternative conformations.&#xA;These days I seen a cartoon of an interaction and I model it in quintuplicate. Sometimes nothing binds, but sometimes a complex is found 5 out of 5. The weirdest part is that it reveals some novel mechanism that makes sense that has probably never been seen before, but is beyond the scope of the experiment so one has the power to just put them aside —interfaces which are stronger phosphorylated, inhibitory domains binding in the same site but in a different way, unexpected complex arrangements etc. So the true revolutionary power of AlphaFold2 is predicting complexes and alternative conformations, which can be used to predict the mechanism of a variant.&lt;/p&gt;&#xA;&lt;h2&gt;WIP&lt;/h2&gt;&#xA;&lt;p&gt;Complexes do not always resolve and there is a lot of discussion in terms of massaging the MSA —e.g. boney-fish+tetrapods only etc. And there is very little cross-linking mass-spec data available for validation and/or constraining. But hopefully things will get better.&lt;/p&gt;&#xA;&lt;h2&gt;Footnote&lt;/h2&gt;&#xA;&lt;p&gt;The last paragraph sounds like an exaggeration, so I best give an example. Here is an example for the drawer from today. A target protein (catalytic domain = pale) has a regulatory terminal which inhibits in isolation (terminus = turquoise), however, when modelled with another protein (mustard) this terminal (terminus = coral) is flipped away. Thus revealing the mechanism by which the mustard protein sequesters the regulatory helix (coral), which would otherwise (turquoise) inhibit the catalytic domain (pale cyan).  In the literature, this is a western blot. Where a variant of interest involved in this mechanism one state may be more energetically favourable by scoring the ∆G of the different &amp;quot;snapshots&amp;quot; (=static conformers).&#xA;&lt;a href=&quot;https://i.stack.imgur.com/xzf2h.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xzf2h.png&quot; alt=&quot;complex&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="6322" LastEditDate="2022-08-25T16:22:26.553" LastActivityDate="2022-08-25T16:22:26.553" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19607" PostTypeId="2" ParentId="2649" CreationDate="2022-08-26T00:38:11.723" Score="3" Body="&lt;p&gt;This could be easily done by &lt;a href=&quot;https://bioinf.shenwei.me/seqkit/usage/#fx2tab-tab2fx&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqkit fx2tab&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqkit fx2tab seq.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, seqkit will not print the &amp;quot;greater than&amp;quot; symbol (&amp;quot;&amp;gt;&amp;quot;). If you do need the symbol:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqkit fx2tab seq.fa | sed 's/^/&amp;gt;/g'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15583" LastActivityDate="2022-08-26T00:38:11.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19608" PostTypeId="1" CreationDate="2022-08-26T02:04:17.427" Score="2" ViewCount="697" Body="&lt;p&gt;Is there a way write a script in python to use &amp;quot;super&amp;quot; to align proteins from a 2D array? Each row of the array has 2 columns, and for each row, I want to &amp;quot;super&amp;quot; the two proteins (one in each column)&lt;/p&gt;&#xA;" OwnerUserId="16041" LastEditorUserId="16041" LastEditDate="2022-08-26T15:36:17.900" LastActivityDate="2022-09-02T10:04:42.003" Title="PyMol alignment script" Tags="|proteins|sequence-alignment|pymol|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19609" PostTypeId="1" CreationDate="2022-08-26T09:32:39.853" Score="1" ViewCount="62" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9536359/#9536376&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'd like to download multiple genome assemblies or proteomes using a set of BioSample IDs from NCBI.&lt;/p&gt;&#xA;&lt;p&gt;I'm able to find the assemblies belonging to the BioSample IDS in a browser (in the search field of &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/&lt;/a&gt;), but couldn't find a commandline interface solution (CLI).&lt;/p&gt;&#xA;&lt;p&gt;E.g. for BioSample &lt;code&gt;SAMN09405588&lt;/code&gt; the assembly id is &lt;code&gt;PDT000806148.1&lt;/code&gt;, and from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly/GCA_014136285.1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/assembly/GCA_014136285.1/&lt;/a&gt; I can download the proteome: &lt;code&gt;GCA_014136285.1_PDT000806148.1_protein.faa.gz&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your help!&lt;/p&gt;&#xA;" OwnerUserId="6935" LastEditorUserId="73" LastEditDate="2022-08-27T09:44:10.437" LastActivityDate="2022-08-27T09:46:32.180" Title="CLI access to genomes/proteomes by BioSample ID" Tags="|python|phylogenetics|identifiers|" AnswerCount="1" CommentCount="1" ClosedDate="2022-08-27T09:54:57.607" ContentLicense="CC BY-SA 4.0" />
  <row Id="19610" PostTypeId="1" CreationDate="2022-08-26T11:42:43.320" Score="1" ViewCount="37" Body="&lt;p&gt;I have to analyze circular dichroism data of proteins and nucleic acids for my PhD. Despite I have been able to find vast literature that helps to interpret a spectrum, most of the papers describe only the experiment as is, so telling what the methods to acquire a good set of spectra were, using JASCO technology or Synchrotron Radiation Circular Dichroism (SRCD), and the final interpretation of the plot.&lt;/p&gt;&#xA;&lt;p&gt;Basically, I come from biomedical signal analysis and processing. So I was thinking that extracting features from spectra could be kind of similar to extracting features from an ECG signal for example.&lt;/p&gt;&#xA;&lt;p&gt;Given that I don't precisely know what the experts do with cd spectra, If you have any advice could be of great value to me.&lt;/p&gt;&#xA;" OwnerUserId="9142" LastActivityDate="2022-08-26T11:42:43.320" Title="How to analyze Circular Dichroism data? Best practices, typical approaches and so on" Tags="|structural-biology|" AnswerCount="0" CommentCount="3" ClosedDate="2022-08-27T09:30:55.343" ContentLicense="CC BY-SA 4.0" />
  <row Id="19611" PostTypeId="1" CreationDate="2022-08-26T13:31:20.783" Score="1" ViewCount="18" Body="&lt;p&gt;In addition to the Wisconsin breast cancer (diagnosis) can be found in UCI machine learning library for breast cancer datasets, what websites can you find similar breast cancer datasets?Or which websites can provide free breast cancer data sets?&lt;/p&gt;&#xA;" OwnerUserId="16044" LastEditorUserId="3694" LastEditDate="2022-08-26T14:53:53.987" LastActivityDate="2022-08-26T14:53:53.987" Title="Experimental data on searching for breast cancer" Tags="|cancer|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19612" PostTypeId="2" ParentId="18918" CreationDate="2022-08-26T14:40:42.830" Score="1" Body="&lt;h2&gt;Background&lt;/h2&gt;&#xA;&lt;p&gt;I've dug into the source code for the &lt;code&gt;nucleotideSubstitutionMatrix()&lt;/code&gt; function from the &lt;code&gt;Biostrings&lt;/code&gt; package. Here is the code for the full function &lt;a href=&quot;https://github.com/Bioconductor/Biostrings/blob/4f9031a1bc0e102148060cd39aa441d812d75ce7/R/pairwiseAlignment.R#L16&quot; rel=&quot;nofollow noreferrer&quot;&gt;from GitHub&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;nucleotideSubstitutionMatrix &amp;lt;- function(match = 1, mismatch = 0, baseOnly = FALSE, type = &amp;quot;DNA&amp;quot;)&#xA;{&#xA;  &amp;quot;%safemult%&amp;quot; &amp;lt;- function(x, y) ifelse(is.infinite(x) &amp;amp; y == 0, 0, x * y)&#xA;  type &amp;lt;- match.arg(type, c(&amp;quot;DNA&amp;quot;, &amp;quot;RNA&amp;quot;))&#xA;  if (!isSingleNumber(match) || !isSingleNumber(mismatch))&#xA;    stop(&amp;quot;'match' and 'mismatch' must be non-missing numbers&amp;quot;)&#xA;  if (baseOnly)&#xA;    letters &amp;lt;- IUPAC_CODE_MAP[DNA_BASES]&#xA;  else&#xA;    letters &amp;lt;- IUPAC_CODE_MAP&#xA;  if (type == &amp;quot;RNA&amp;quot;)&#xA;    names(letters) &amp;lt;- chartr(&amp;quot;T&amp;quot;, &amp;quot;U&amp;quot;, names(letters))&#xA;  nLetters &amp;lt;- length(letters)&#xA;  splitLetters &amp;lt;- strsplit(letters,split=&amp;quot;&amp;quot;)&#xA;  submat &amp;lt;- matrix(0, nrow = nLetters, ncol = nLetters, dimnames = list(names(letters), names(letters)))&#xA;  for(i in 1:nLetters)&#xA;    for(j in i:nLetters)&#xA;      submat[i,j] &amp;lt;- submat[j,i] &amp;lt;- mean(outer(splitLetters[[i]], splitLetters[[j]], &amp;quot;==&amp;quot;))&#xA;  abs(match) * submat - abs(mismatch) %safemult% (1 - submat)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Dissecting this, we can see that the matrix is constructed in four main steps.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;1&lt;/strong&gt;. The &lt;code&gt;IUPAC_CODE_MAP&lt;/code&gt; named character vector is split into a list of character vectors, where each named element in the list gives an IUPAC character and its constituent DNA bases.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; IUPAC_CODE_MAP&#xA;     A      C      G      T      M      R      W      S      Y      K      V      H      D      B      N &#xA;   &amp;quot;A&amp;quot;    &amp;quot;C&amp;quot;    &amp;quot;G&amp;quot;    &amp;quot;T&amp;quot;   &amp;quot;AC&amp;quot;   &amp;quot;AG&amp;quot;   &amp;quot;AT&amp;quot;   &amp;quot;CG&amp;quot;   &amp;quot;CT&amp;quot;   &amp;quot;GT&amp;quot;  &amp;quot;ACG&amp;quot;  &amp;quot;ACT&amp;quot;  &amp;quot;AGT&amp;quot;  &amp;quot;CGT&amp;quot; &amp;quot;ACGT&amp;quot; &#xA;&amp;gt; splitLetters$A&#xA;[1] &amp;quot;A&amp;quot;&#xA;&amp;gt; splitLetters$V&#xA;[1] &amp;quot;A&amp;quot; &amp;quot;C&amp;quot; &amp;quot;G&amp;quot;&#xA;&amp;gt; splitLetters$N&#xA;[1] &amp;quot;A&amp;quot; &amp;quot;C&amp;quot; &amp;quot;G&amp;quot; &amp;quot;T&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;2&lt;/strong&gt;. &lt;code&gt;submat&lt;/code&gt; is initiated as an &lt;span class=&quot;math-container&quot;&gt;$n × n$&lt;/span&gt; matrix of zeros, where &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; is &lt;code&gt;length(names(IUPAC_CODE_MAP))&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; matrix(0, nrow = nLetters, ncol = nLetters, dimnames = list(names(letters), names(letters)))&#xA;  A C G T M R W S Y K V H D B N&#xA;A 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0&#xA;C 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0&#xA;G 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0&#xA;T 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0&#xA;M 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0&#xA;R 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0&#xA;W 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0&#xA;S 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0&#xA;Y 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0&#xA;K 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0&#xA;V 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0&#xA;H 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0&#xA;D 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0&#xA;B 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0&#xA;N 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;3&lt;/strong&gt;. Each element of the matrix is calculated by counting the overlapping elements in the arrays represented by the IUPAC labels given as row and column names.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; for(i in 1:nLetters)&#xA;&amp;gt;     for(j in i:nLetters)&#xA;&amp;gt;         submat[i,j] &amp;lt;- submat[j,i] &amp;lt;- mean(outer(splitLetters[[i]], splitLetters[[j]], &amp;quot;==&amp;quot;))&#xA;&amp;gt; submat&#xA;          A         C         G         T         M         R         W         S         Y         K         V         H         D&#xA;A 1.0000000 0.0000000 0.0000000 0.0000000 0.5000000 0.5000000 0.5000000 0.0000000 0.0000000 0.0000000 0.3333333 0.3333333 0.3333333&#xA;C 0.0000000 1.0000000 0.0000000 0.0000000 0.5000000 0.0000000 0.0000000 0.5000000 0.5000000 0.0000000 0.3333333 0.3333333 0.0000000&#xA;G 0.0000000 0.0000000 1.0000000 0.0000000 0.0000000 0.5000000 0.0000000 0.5000000 0.0000000 0.5000000 0.3333333 0.0000000 0.3333333&#xA;T 0.0000000 0.0000000 0.0000000 1.0000000 0.0000000 0.0000000 0.5000000 0.0000000 0.5000000 0.5000000 0.0000000 0.3333333 0.3333333&#xA;M 0.5000000 0.5000000 0.0000000 0.0000000 0.5000000 0.2500000 0.2500000 0.2500000 0.2500000 0.0000000 0.3333333 0.3333333 0.1666667&#xA;R 0.5000000 0.0000000 0.5000000 0.0000000 0.2500000 0.5000000 0.2500000 0.2500000 0.0000000 0.2500000 0.3333333 0.1666667 0.3333333&#xA;W 0.5000000 0.0000000 0.0000000 0.5000000 0.2500000 0.2500000 0.5000000 0.0000000 0.2500000 0.2500000 0.1666667 0.3333333 0.3333333&#xA;S 0.0000000 0.5000000 0.5000000 0.0000000 0.2500000 0.2500000 0.0000000 0.5000000 0.2500000 0.2500000 0.3333333 0.1666667 0.1666667&#xA;Y 0.0000000 0.5000000 0.0000000 0.5000000 0.2500000 0.0000000 0.2500000 0.2500000 0.5000000 0.2500000 0.1666667 0.3333333 0.1666667&#xA;K 0.0000000 0.0000000 0.5000000 0.5000000 0.0000000 0.2500000 0.2500000 0.2500000 0.2500000 0.5000000 0.1666667 0.1666667 0.3333333&#xA;V 0.3333333 0.3333333 0.3333333 0.0000000 0.3333333 0.3333333 0.1666667 0.3333333 0.1666667 0.1666667 0.3333333 0.2222222 0.2222222&#xA;H 0.3333333 0.3333333 0.0000000 0.3333333 0.3333333 0.1666667 0.3333333 0.1666667 0.3333333 0.1666667 0.2222222 0.3333333 0.2222222&#xA;D 0.3333333 0.0000000 0.3333333 0.3333333 0.1666667 0.3333333 0.3333333 0.1666667 0.1666667 0.3333333 0.2222222 0.2222222 0.3333333&#xA;B 0.0000000 0.3333333 0.3333333 0.3333333 0.1666667 0.1666667 0.1666667 0.3333333 0.3333333 0.3333333 0.2222222 0.2222222 0.2222222&#xA;N 0.2500000 0.2500000 0.2500000 0.2500000 0.2500000 0.2500000 0.2500000 0.2500000 0.2500000 0.2500000 0.2500000 0.2500000 0.2500000&#xA;          B    N&#xA;A 0.0000000 0.25&#xA;C 0.3333333 0.25&#xA;G 0.3333333 0.25&#xA;T 0.3333333 0.25&#xA;M 0.1666667 0.25&#xA;R 0.1666667 0.25&#xA;W 0.1666667 0.25&#xA;S 0.3333333 0.25&#xA;Y 0.3333333 0.25&#xA;K 0.3333333 0.25&#xA;V 0.2222222 0.25&#xA;H 0.2222222 0.25&#xA;D 0.2222222 0.25&#xA;B 0.3333333 0.25&#xA;N 0.2500000 0.25&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;4&lt;/strong&gt;. Finally, the base substitution matrix is converted to a scoring matrix given the user-defined values for &lt;code&gt;match&lt;/code&gt; and &lt;code&gt;mismatch&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h2&gt;Answer&lt;/h2&gt;&#xA;&lt;p&gt;Concerning step &lt;strong&gt;3&lt;/strong&gt;, the &lt;code&gt;Biostrings&lt;/code&gt; approach asks whether each element of each vector matches each element of the other vector, and then records the proportion of &lt;code&gt;TRUE&lt;/code&gt; values in the  matrix of results. Note that the code &lt;code&gt;submat[i,j] &amp;lt;- submat[j,i]&lt;/code&gt; creates a symmetric matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt;. However, even if &lt;span class=&quot;math-container&quot;&gt;$[A]_{ij}$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$[A]_{ji}$&lt;/span&gt; were generated by separate calls, the commutativity of the &lt;code&gt;mean(outer(i, j, &amp;quot;==&amp;quot;))&lt;/code&gt; computation means that &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; will always be symmetric:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; outer(splitLetters&lt;span class=&quot;math-container&quot;&gt;$K, splitLetters$&lt;/span&gt;V, &amp;quot;==&amp;quot;)&#xA;      [,1]  [,2]  [,3]&#xA;[1,] FALSE FALSE  TRUE&#xA;[2,] FALSE FALSE FALSE&#xA;&amp;gt; mean(outer(splitLetters&lt;span class=&quot;math-container&quot;&gt;$K, splitLetters$&lt;/span&gt;V, &amp;quot;==&amp;quot;))&#xA;[1] 0.1666667&#xA;&#xA;&amp;gt; outer(splitLetters&lt;span class=&quot;math-container&quot;&gt;$V, splitLetters$&lt;/span&gt;K, &amp;quot;==&amp;quot;)&#xA;      [,1]  [,2]&#xA;[1,] FALSE FALSE&#xA;[2,] FALSE FALSE&#xA;[3,]  TRUE FALSE&#xA;&amp;gt; mean(outer(splitLetters&lt;span class=&quot;math-container&quot;&gt;$V, splitLetters$&lt;/span&gt;K, &amp;quot;==&amp;quot;))&#xA;[1] 0.1666667&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Instead of checking for pairwise equality of elements (&lt;code&gt;==&lt;/code&gt;), we can check if a given element from the query is contained in the set of elements in the consensus / subject (&lt;code&gt;%in%&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; outer(splitLetters&lt;span class=&quot;math-container&quot;&gt;$K, splitLetters$&lt;/span&gt;V, &amp;quot;%in%&amp;quot;)&#xA;      [,1]  [,2]  [,3]&#xA;[1,]  TRUE  TRUE  TRUE&#xA;[2,] FALSE FALSE FALSE&#xA;&amp;gt; mean(outer(splitLetters&lt;span class=&quot;math-container&quot;&gt;$K, splitLetters$&lt;/span&gt;V, &amp;quot;%in%&amp;quot;))&#xA;[1] 0.5&#xA;&#xA;&amp;gt; outer(splitLetters&lt;span class=&quot;math-container&quot;&gt;$V, splitLetters$&lt;/span&gt;K, &amp;quot;%in%&amp;quot;)&#xA;      [,1]  [,2]&#xA;[1,] FALSE FALSE&#xA;[2,] FALSE FALSE&#xA;[3,]  TRUE  TRUE&#xA;&amp;gt; mean(outer(splitLetters&lt;span class=&quot;math-container&quot;&gt;$V, splitLetters$&lt;/span&gt;K, &amp;quot;%in%&amp;quot;))&#xA;[1] 0.3333333&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is not a commutative operation, so the resulting substitution matrix is asymmetric.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;asym_NSM &amp;lt;- function(match = 1, mismatch = 0, baseOnly = FALSE, type = &amp;quot;DNA&amp;quot;)&#xA;{&#xA;  &amp;quot;%safemult%&amp;quot; &amp;lt;- function(x, y) ifelse(is.infinite(x) &amp;amp; y == 0, 0, x * y)&#xA;  type &amp;lt;- match.arg(type, c(&amp;quot;DNA&amp;quot;, &amp;quot;RNA&amp;quot;))&#xA;  if (!isSingleNumber(match) || !isSingleNumber(mismatch))&#xA;    stop(&amp;quot;'match' and 'mismatch' must be non-missing numbers&amp;quot;)&#xA;  if (baseOnly)&#xA;    letters &amp;lt;- IUPAC_CODE_MAP[DNA_BASES]&#xA;  else&#xA;    letters &amp;lt;- IUPAC_CODE_MAP&#xA;  if (type == &amp;quot;RNA&amp;quot;)&#xA;    names(letters) &amp;lt;- chartr(&amp;quot;T&amp;quot;, &amp;quot;U&amp;quot;, names(letters))&#xA;  nLetters &amp;lt;- length(letters)&#xA;  splitLetters &amp;lt;- strsplit(letters,split=&amp;quot;&amp;quot;)&#xA;  submat &amp;lt;- matrix(0, nrow = nLetters, ncol = nLetters, dimnames = list(names(letters), names(letters)))&#xA;  for(i in 1:nLetters) {&#xA;    for(j in i:nLetters) {&#xA;      submat[i,j] &amp;lt;- mean(outer(splitLetters[[i]], splitLetters[[j]], &amp;quot;%in%&amp;quot;))&#xA;      submat[j,i] &amp;lt;- mean(outer(splitLetters[[j]], splitLetters[[i]], &amp;quot;%in%&amp;quot;))&#xA;    }&#xA;  }&#xA;  &#xA;  abs(match) * submat - abs(mismatch) %safemult% (1 - submat)&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; asym_NSM()&#xA;          A         C         G         T         M         R         W         S         Y         K         V         H         D&#xA;A 1.0000000 0.0000000 0.0000000 0.0000000 1.0000000 1.0000000 1.0000000 0.0000000 0.0000000 0.0000000 1.0000000 1.0000000 1.0000000&#xA;C 0.0000000 1.0000000 0.0000000 0.0000000 1.0000000 0.0000000 0.0000000 1.0000000 1.0000000 0.0000000 1.0000000 1.0000000 0.0000000&#xA;G 0.0000000 0.0000000 1.0000000 0.0000000 0.0000000 1.0000000 0.0000000 1.0000000 0.0000000 1.0000000 1.0000000 0.0000000 1.0000000&#xA;T 0.0000000 0.0000000 0.0000000 1.0000000 0.0000000 0.0000000 1.0000000 0.0000000 1.0000000 1.0000000 0.0000000 1.0000000 1.0000000&#xA;M 0.5000000 0.5000000 0.0000000 0.0000000 1.0000000 0.5000000 0.5000000 0.5000000 0.5000000 0.0000000 1.0000000 1.0000000 0.5000000&#xA;R 0.5000000 0.0000000 0.5000000 0.0000000 0.5000000 1.0000000 0.5000000 0.5000000 0.0000000 0.5000000 1.0000000 0.5000000 1.0000000&#xA;W 0.5000000 0.0000000 0.0000000 0.5000000 0.5000000 0.5000000 1.0000000 0.0000000 0.5000000 0.5000000 0.5000000 1.0000000 1.0000000&#xA;S 0.0000000 0.5000000 0.5000000 0.0000000 0.5000000 0.5000000 0.0000000 1.0000000 0.5000000 0.5000000 1.0000000 0.5000000 0.5000000&#xA;Y 0.0000000 0.5000000 0.0000000 0.5000000 0.5000000 0.0000000 0.5000000 0.5000000 1.0000000 0.5000000 0.5000000 1.0000000 0.5000000&#xA;K 0.0000000 0.0000000 0.5000000 0.5000000 0.0000000 0.5000000 0.5000000 0.5000000 0.5000000 1.0000000 0.5000000 0.5000000 1.0000000&#xA;V 0.3333333 0.3333333 0.3333333 0.0000000 0.6666667 0.6666667 0.3333333 0.6666667 0.3333333 0.3333333 1.0000000 0.6666667 0.6666667&#xA;H 0.3333333 0.3333333 0.0000000 0.3333333 0.6666667 0.3333333 0.6666667 0.3333333 0.6666667 0.3333333 0.6666667 1.0000000 0.6666667&#xA;D 0.3333333 0.0000000 0.3333333 0.3333333 0.3333333 0.6666667 0.6666667 0.3333333 0.3333333 0.6666667 0.6666667 0.6666667 1.0000000&#xA;B 0.0000000 0.3333333 0.3333333 0.3333333 0.3333333 0.3333333 0.3333333 0.6666667 0.6666667 0.6666667 0.6666667 0.6666667 0.6666667&#xA;N 0.2500000 0.2500000 0.2500000 0.2500000 0.5000000 0.5000000 0.5000000 0.5000000 0.5000000 0.5000000 0.7500000 0.7500000 0.7500000&#xA;          B N&#xA;A 0.0000000 1&#xA;C 1.0000000 1&#xA;G 1.0000000 1&#xA;T 1.0000000 1&#xA;M 0.5000000 1&#xA;R 0.5000000 1&#xA;W 0.5000000 1&#xA;S 1.0000000 1&#xA;Y 1.0000000 1&#xA;K 1.0000000 1&#xA;V 0.6666667 1&#xA;H 0.6666667 1&#xA;D 0.6666667 1&#xA;B 1.0000000 1&#xA;N 0.7500000 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3967" LastActivityDate="2022-08-26T14:40:42.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19613" PostTypeId="1" CreationDate="2022-08-26T14:46:42.663" Score="2" ViewCount="1444" Body="&lt;p&gt;Using Seurat's VlnPlot, how can I remove the black outline around the violin plot?&lt;/p&gt;&#xA;&lt;p&gt;For example, how can I change from the following graph with a (black) outline:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;    VlnPlot(ilc2,&#xA;        features = &amp;quot;gene&amp;quot;,&#xA;        split.by = &amp;quot;group&amp;quot;, &#xA;        cols = c(&amp;quot;#198BFD&amp;quot;, &amp;quot;#EC6464&amp;quot;),&#xA;        pt.size = 0) +&#xA;  geom_boxplot(width = 0.1) + theme(aspect.ratio = 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wBszy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wBszy.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;To a graph like the one below, which does not have an outline around the violin plot:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/va4ps.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/va4ps.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3968" LastEditorUserId="650" LastEditDate="2023-04-11T19:26:08.860" LastActivityDate="2023-09-08T22:05:25.560" Title="How to remove black outline from Seurat's VlnPlot violin plot" Tags="|seurat|ggplot2|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19614" PostTypeId="1" CreationDate="2022-08-26T15:08:37.583" Score="3" ViewCount="25" Body="&lt;p&gt;Is there really any advantage to filtering the GTF annotation for transcriptome reconstruction (or, for example, for pseudo-alignment quantification)? Are there any downsides (i.e. reasons why I should NOT do that)?&lt;/p&gt;&#xA;&lt;p&gt;I'm going to run a transcriptome reconstruction with StringTie on several RNA-seq samples, and I want to be stringent in the detection of novel isoforms.&lt;/p&gt;&#xA;&lt;p&gt;To do that, I'm thinking of filtering the reference annotation that I provide to StringTie, removing transcripts that are not well supported by strong evidences. I found that the GENCODE tag &amp;quot;transcript_support_level&amp;quot; could be what I should filter for.&lt;/p&gt;&#xA;&lt;p&gt;However, I couldn't find anybody suggesting to filter the GENCODE reference annotation by the &lt;a href=&quot;https://www.gencodegenes.org/pages/data_format.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;transcript_support_level tag&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;From the GENCODE documentation:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;transcript_support_level    {1,2,3,4,5,NA} [0,1]&#xA;transcripts are scored according to how well mRNA and EST alignments match over its full length:&#xA;1 (all splice junctions of the transcript are supported by at least one non-suspect mRNA),&#xA;2 (the best supporting mRNA is flagged as suspect or the support is from multiple ESTs),&#xA;3 (the only support is from a single EST),&#xA;4 (the best supporting EST is flagged as suspect),&#xA;5 (no single transcript supports the model structure),&#xA;NA (the transcript was not analyzed)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I ran a test on GENCODE v40, and it seems that the cumulative distribution of genes from support level 1 to NA is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SupportLevel | # genes&#xA;           1 | 19311&#xA;           2 | 23680&#xA;           3 | 28212&#xA;           4 | 29429&#xA;           5 | 31566&#xA;          NA | 61552&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So it seems that I would miss out many genes from the reference annotation if I filter by, let's say, &lt;code&gt;transcript_support_level &amp;lt;= 2&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Couldn't find any info about advantages or disadvantages... apparently most people simply use the comprehensive GENCODE annotation with StringTie or Salmon or other tools like that, but I also know that these approaches generates many false detections that needs to be filtered downstream.&lt;/p&gt;&#xA;&lt;p&gt;Hopefully I'm not the first person (over)thinking this :)&lt;/p&gt;&#xA;" OwnerUserId="2355" LastEditorUserId="73" LastEditDate="2022-08-27T09:59:43.550" LastActivityDate="2022-08-27T09:59:43.550" Title="Genome-guided transcriptome reconstruction: should I filter my reference annotation by transcript_support_level or other tags?" Tags="|rna-seq|stringtie|gencode|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19615" PostTypeId="1" CreationDate="2022-08-26T18:23:11.967" Score="3" ViewCount="185" Body="&lt;p&gt;I am trying to pick the best representative sequence from an amino acid MSA. I do not want to use the consensus, as it generates an artificial one. Here is my approach:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I calculate a Percentage Identity Matrix (PIM) from the MSA file&lt;/li&gt;&#xA;&lt;li&gt;I check the highest value in each column and rank the respective&#xA;sequences. What I mean by this, for example, the values in the first&#xA;column would be the Percentage Identity of the first seq compared&#xA;the rest of the dataset. If I pick the highest value in there, that&#xA;would give me the sequences that are highly identical to the first&#xA;seq.&lt;/li&gt;&#xA;&lt;li&gt;I do this for the every column and rank the sequences. The top&#xA;ranked sequence would yield me the best representetive sequence&#xA;within my MSA data set.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The problem with this approach is that its computationally expensive to calculate the PIM as its doing a character match, comparing each sequence's position with all of the dataset. Is there any easier way to achieve this? Im not sure, if we could achieve something similar by looking at the tree. I think we can find the lowest common ancestor by the tree but I do not think that would yield a representetive sequence.&lt;/p&gt;&#xA;" OwnerUserId="16046" LastEditorUserId="73" LastEditDate="2022-08-27T10:00:59.153" LastActivityDate="2022-09-01T14:20:17.733" Title="Select representetive sequence within MSA (protein)" Tags="|proteins|multiple-sequence-alignment|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19617" PostTypeId="1" CreationDate="2022-08-27T03:12:09.483" Score="2" ViewCount="163" Body="&lt;p&gt;I am very new here. Please help me with my codes as am trying to draw multiple sequence alignment as tree suing ggtree package in R and i have following error.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(tidyverse)&#xA;library(ggtree) &#xA;library(seqinr)&#xA;tree = read.tree(&amp;quot;get_result.nwk&amp;quot;)&#xA;msaplot(p=ggtree(tree), fasta = &amp;quot;get_results&amp;quot;, window=c(150, 175))&#xA;&#xA;Error in inds:inde : NA/NaN argument&#xA;&amp;gt; msaplot(p=ggtree(tree) + geom_tiplab(align=TRUE), fasta = &amp;quot;get_result.fasta&amp;quot;, offset = 5)&#xA;Error in inds:inde : NA/NaN argument&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My data looks like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;LN681202.1_Ralstonia_solanacearum_isolate RsBe2&#xA;TGCAAGTCGAACGGCAGCGGGGGTAGCTTGCTACCTGCCGGCGAGTGGCGAACGGGTGAGTAATACATCGGAACGTGCCCTGTAGTGGGGGATAACTAGTCGAAAGACTAGCTAATACCGCATACGACCTGAGGGTGAAAGTGGGGGACCGCAAGGCCTCATGCTATAGGAGCGGCCGATGTCTGATTAGCTAGTTGGTGGGGTAAAGGCCCACCAAGGCGACGATCAGTAGCTGGTCTGAGAGGACGATCAGCCACACTGGGACTGAGACACGGCCCAGACTCCTACGGGAGGCAGCAGTGGGGAATTTTGGACAATGGGGGCAACCCTGATCCAGCAATGCCGCGTGTGTGAAGAAGGCCTTCGGGTTGTAAAGCACTTTTGTCCGGAAAGAAATCGCTTCGGTTAATACCTGGAGTGGATGACGGTACCGGAAGAATAAGGACCGGCTAACTACGTGCCAGCAGCCGCGGTAATACGTAGGGTCCAAGCGTTAATCGGAATTACTGGGCGTAAAGCGTGCGCAGGCGGTTGTGCAAGACCGATGTGAAATCCCCGGGCTTAACCTGGGAATTGCATTGGTGACTGCACAGCTAGAGTGTGTCAGAGGGAGGTAGAATTCCACGTGTAGCAGTGAAATGCGTAGAGATGTGGAGGAATACCGATGGCGAAGGCAGCCTCCTGGGATAACACTGACGCTCATGCACGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACGCCCTAAACGATGTCAACTAGTTGTTGGGGATTCATTTCCTTAGTAACGTAGCTAACGCGTGAAGTTGA&#xA;MF179616.1_Kv-1&#xA;GGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCTGTCAAGTCGGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGAGCTACGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAACTTTCCAGAGATGGATTGGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGGTTGTGAAATGTTGGGTTAAGTCCCGCAACCGCAACCCTTATCCTTTGTTGCCAGCGGTCCGGCCGGGAACTCAAAGGAGACTGCAGTGATAAACTGGAGGAAGGTGGGGATGACGTCAAGTCATCATGGCCCTTACCCAGGGCTACACACGTGCTCAATGGCATATACAAAGAGAAGCGACCAGAGCAAGCGGACCTCATAAAGTATGTCGTAGTCGATTGGAGTCTGCAACTCGACTCCATGAAGTCGGAATCGCTAGTAATCGTAGATCAGAATGCTCGGTGAATACGTTCCCGGGCCTTGTACACACCGC&#xA;MN725749.1_SA006&#xA;GGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCTGTCAAGTCGGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGAGCTACGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAACTTTCCAGAGATGGATTGGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGGTTGTGAAATGTTGGGTTAAGTCCCGCAACCGCAACCCTTATCCTTTGTTGCCAGCGGTCCGGCCGGGAACTCAAAGGAGACTGCAGTGATAAACTGGAGGAAGGTGGGGATGACGTCAAGTCATCATGGCCCTTACCCAGGGCTACACACGTGCTCAATGGCATATACAAAGAGAAGCGACCAGAGCAAGCGGACCTCATAAAGTATGTCGTAGTCGATTGGAGTCTGCAACTCGACTCCATGAAGTCGGAATCGCTAGTAATCGTAGATCAGAATGCTCGGTGAATACGTTCCCGGGCCTTGTACACACCGC&#xA;MF183971.1_Klebsiella_sp._strain_Kv-2&#xA;GGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCTGTCAAGTCGGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGAGCTACGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAACTTTCCAGAGATGGATTGGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGGTTGTGAAATGTTGGGTTAAGTCCCGCAACCGCAACCCTTATCCTTTGTTGCCAGCGGTTAGGCCGGGAACTCAAAGGAGACTGCAGTGATAAACTGGAGGAAGGTGGGGATGACGTCAAGTCATCATGGCCCTTACCCAGGGCTACACACGTGCTCAATGGCATATACAAAGAGAAGCGACCAGAGCAAGCGGACCTCATAAAGTATGTCGTAGTCGATTGGAGTCTGCAACTCGACTCCATGAAGTCGGAATCGCTAGTAATCGTAGATCAGAATGCTCGGTGAATACGTTCCCGGGCCTTGTACACACCGC&#xA;MT507217.1_Uyi23r&#xA;GGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCTGTCAAGTCGGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGAGCTACGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAACTTTCCAGAGATGGATTGGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGGTTGTGAAATGTTGGGTTAAGTCCCGCAACCGCAACCCTTATCCTTTGTTGCCAGCGGTTAGGCCGGGAACTCAAAGGAGACTGCAGTGATAAACTGGAGGAAGGTGGGGATGACGTCAAGTCATCATGGCCCTTACCCAGGGCTACACACGTGCTCAATGGCATATACAAAGAGAAGCGACCAGAGCAAGCGGACCTCATAAAGTATGTCGTAGTCGATTGGAGTCTGCAACTCGACTCCATGAAGTCGGAATCGCTAGTAATCGTAGATCAGAATGCTCGGTGAATACGTTCCCGGGCCTTGTACACACCGC&#xA;MF183969.1_Kv-3&#xA;GGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCTGTCAAGTCGGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGAGCTACGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAACTTTCCAGAGATGGATTGGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGGTTGTGAAATGTTGGGTTAAGTCCCGCAACCGCAACCCTTATCCTTTGTTGCCAGCGGTTAGGCCGGGAACTCAAAGGAGACTGCAGTGATAAACTGGAGGAAGGTGGGGATGACGTCAAGTCATCATGGCCCTTACCCAGGGCTACACACGTGCTCAATGGCATATACAAAGAGAAGCGACCAGAGCAAGCGGACCTCATAAAGTATGTCGTAGTCGATTGGAGTCTGCAACTCGACTCCATGAAGTCGGAATCGCTAGTAATCGTAGATCAGAATGCTCGGTGAATACGTTCCCGGGCCTTGTACACACCGC&#xA;MK929064.1_NM43TS4-1&#xA;GGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCTGTCAAGTCGGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGAGCTACGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAACTTTCCAGAGATGGATTGGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGGTTGTGAAATGTTGGGTTAAGTCCCGCAACCGCAACCCTTATCCTTTGTTGCCAGCGGTTAGGCCGGGAACTCAAAGGAGACTGCAGTGATAAACTGGAGGAAGGTGGGGATGACGTCAAGTCATCATGGCCCTTACCCAGGGCTACACACGTGCTCAATGGCATATACAAAGAGAAGCGACCAGAGCAAGCGGACCTCATAAAGTATGTCGTAGTCGATTGGAGTCTGCAACTCGACTCCATGAAGTCGGAATCGCTAGTAATCGTAGATCAGAATGCTCGGTGAATACGTTCCCGGGCCTTGTACACACCGC&#xA;MF183973.1_Kv-4&#xA;GGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCTGTCAAGTCGGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGAGCTACGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAACTTTCCAGAGATGGATTGGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGGTTGTGAAATGTTGGGTTAAGTCCCGCAACCGCAACCCTTATCCTTTGTTGCCAGCGGTTAGGCCGGGAACTCAAAGGAGACTGCAGTGATAAACTGGAGGAAGGTGGGGATGACGTCAAGTCATCATGGCCCTTACCCAGGGCTACACACGTGCTCAATGGCATATACAAAGAGAAGCGACCAGAGCAAGCGGACCTCATAAAGTATGTCGTAGTCGATTGGAGTCTGCAACTCGACTCCATGAAGTCGGAATCGCTAGTAATCGTAGATCAGAATGCTCGGTGAATACGTTCCCGGGCCTTGTACACACCGC&#xA;MN082125.1_IILSFPR619&#xA;GGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCTGTCAAGTCGGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGAGCTACGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAACTTTCCAGAGATGGATTGGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGGTTGTGAAATGTTGGGTTAAGTCCCGCAACCGCAACCCTTATCCTTTGTTGCCAGCGGTTAGGCCGGGAACTCAAAGGAGACTGCAGTGATAAACTGGAGGAAGGTGGGGATGACGTCAAGTCATCATGGCCCTTACCCAGGGCTACACACGTGCTCAATGGCATATACAAAGAGAAGCGACCAGAGCAAGCGGACCTCATAAAGTATGTCGTAGTCGATTGGAGTCTGCAACTCGACTCCATGAAGTCGGAATCGCTAGTAATCGTAGATCAGAATGCTCGGTGAATACGTTCCCGGGCCTTGTACACACCGC&#xA;MF183974.1_Kv-5&#xA;GGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCTGTCAAGTCGGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGAGCTACGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAACTTTCCAGAGATGGATTGGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGGTTGTGAAATGTTGGGTTAAGTCCCGCAACCGCAACCCTTATCCTTTGTTGCCAGCGGTTAGGCCGGGAACTCAAAGGAGACTGCAGTGATAAACTGGAGGAAGGTGGGGATGACGTCAAGTCATCATGGCCCTTACCCAGGGCTACACACGTGCTCAATGGCATATACAAAGAGAAGCGACCAGAGCAAGCGGACCTCATAAAGTATGTCGTAGTCGATTGGAGTCTGCAACTCGACTCCATGAAGTCGGAATCGCTAGTAATCGTAGATCAGAATGCTCGGTGAATACGTTCCCGGGCCTTGTACACACCGC&#xA;MK671348.1_1&#xA;GGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCTGTCAAGTCGGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGAGCTACGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAACTTTCCAGAGATGGATTGGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGGTTGTGAAATGTTGGGTTAAGTCCCGCAACCGCAACCCTTATCCTTTGTTGCCAGCGGTTAGGCCGGGAACTCAAAGGAGACTGCAGTGATAAACTGGAGGAAGGTGGGGATGACGTCAAGTCATCATGGCCCTTACCCAGGGCTACACACGTGCTCAATGGCATATACAAAGAGAAGCGACCAGAGCAAGCGGACCTCATAAAGTATGTCGTAGTCGATTGGAGTCTGCAACTCGACTCCATGAAGTCGGAATCGCTAGTAATCGTAGATCAGAATGCTCGGTGAATACGTTCCCGGGCCTTGTACACACCGC&#xA;MF183970.1_Kv-6&#xA;GGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCTGTCAAGTCGGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGAGCTACGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAACTTTCCAGAGATGGATTGGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGGTTGTGAAATGTTGGGTTAAGTCCCGCAACCGCAACCCTTATCCTTTGTTGCCAGCGGTTCGGCCGGGAACTCAAAGGAGACTGCAGTGATAAACTGGAGGAAGGTGGGGATGACGTCAAGTCATCATGGCCCTTACCCAGGGCTACACACGTGCTCAATGGCATATACAAAGAGAAGCGACCAGAGCAAGCGGACCTCATAAAGTATGTCGTAGTCGATTGGAGTCTGCAACTCGACTCCATGAAGTCGGAATCGCTAGTAATCGTAGATCAGAATGCTCGGTGAATACGTTCCCGGGCCTTGTACACACCGC&#xA;MF682957.1_BT3409&#xA;GGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCTGTCAAGTCGGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGAGCTACGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAACTTTCCAGAGATGGATTGGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGGTTGTGAAATGTTGGGTTAAGTCCCGCAACCGCAACCCTTATCCTTTGTTGCCAGCGGTTCGGCCGGGAACTCAAAGGAGACTGCAGTGATAAACTGGAGGAAGGTGGGGATGACGTCAAGTCATCATGGCCCTTACCCAGGGCTACACACGTGCTCAATGGCATATACAAAGAGAAGCGACCAGAGCAAGCGGACCTCATAAAGTATGTCGTAGTCGATTGGAGTCTGCAACTCGACTCCATGAAGTCGGAATCGCTAGTAATCGTAGATCAGAATGCTCGGTGAATACGTTCCCGGGCCTTGTACACACCGC&#xA;MF183972.1_Kv-7&#xA;GGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCTGTCAAGTCGGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGAGCTACGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAACTTTCCAGAGATGGATTGGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGGTTGTGAAATGTTGGGTTAAGTCCCGCAACCGCAACCCTTATCCTTTGTTGCCAGCGGTTCGGCCGGGAACTCAAAGGAGACTGCAGTGATAAACTGGAGGAAGGTGGGGATGACGTCAAGTCATCATGGCCCTTACCCAGGGCTACACACGTGCTCAATGGCATATACAAAGAGAAGCGACCAGAGCAAGCGGACCTCATAAAGTATGTCGTAGTCGATTGGAGTCTGCAACTCGACTCCATGAAGTCGGAATCGCTAGTAATCGTAGATCAGAATGCTCGGTGAATACGTTCCCGGGCCTTGTACACACCGC&#xA;MZ475068.1_Klebsiella_variicola_EMBL-1&#xA;GGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCTGTCAAGTCGGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGAGCTACGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAACTTTCCAGAGATGGATTGGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGGTTGTGAAATGTTGGGTTAAGTCCCGCAACCGCAACCCTTATCCTTTGTTGCCAGCGGTTCGGCCGGGAACTCAAAGGAGACTGCAGTGATAAACTGGAGGAAGGTGGGGATGACGTCAAGTCATCATGGCCCTTACCCAGGGCTACACACGTGCTCAATGGCATATACAAAGAGAAGCGACCAGAGCAAGCGGACCTCATAAAGTATGTCGTAGTCGATTGGAGTCTGCAACTCGACTCCATGAAGTCGGAATCGCTAGTAATCGTAGATCAGAATGCTCGGTGAATACGTTCCCGGGCCTTGTACACACCGC&#xA;MF373424.1_KV8&#xA;GCTCTCGGGTGACGAGCGGCGGACGGGTGAGTAATGTCTGGGAAACTGCCTGATGGAGGGGGATAACCTGGAAACGGTAGCTAATACCGCATAACGTCGCAAGACCAAAGTGGGGGACCTTCGGCCTCATGCCATCAGATGTGCCCAGATGGGATTAGCTGGTAGGTGGGGTAACGGCTCCCTAGGCGACGATCCCTAGCTGGTCTGAGAGGATGACCAGCCAACTGGAACTGAGACACGGTCCAGACTCCTACGGGAGGCAGCAGTGGGGAAATTGCACAATGGGCGCAAGCCTGATGCAGCCATGCCGCGTGTGTGAAGAAGGCCTTCGGGTTGTAAAGACTTTCAGCGGGGAGGAAGGCGGTGAGGTTAATAACCTCATCGATTGACGTTACCCGCAGAAGAAGCACCGGCTAACTCCGTGCCAGCAGCCGCGGTAATACGGAGGGTGCAAGCGTTAATCGGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCGTCAAGTCGGATGTGAAATCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCTCAGGTGCGAAAGCGTGGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACGCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGATAACCGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGTTAAAACTCAAATG&#xA;KC853296.1_JM16&#xA;GCTCTCGGGTGACGAGCGGCGGACGGGTGAGTAATGTCTGGGAAACTGCCTGATGGAGGGGGATAACCTGGAAACGGTAGCTAATACCGCATAACGTCGCAAGACCAAAGTGGGGGACCTTCGGCCTCATGCCATCAGATGTGCCCAGATGGGATTAGCTGGTAGGTGGGGTAACGGCTCCCTAGGCGACGATCCCTAGCTGGTCTGAGAGGATGACCAGCCAACTGGAACTGAGACACGGTCCAGACTCCTACGGGAGGCAGCAGTGGGGAAATTGCACAATGGGCGCAAGCCTGATGCAGCCATGCCGCGTGTGTGAAGAAGGCCTTCGGGTTGTAAAGACTTTCAGCGGGGAGGAAGGCGGTGAGGTTAATAACCTTATCGATTGACGTTACCCGCAGAAGAAGCACCGGCTAACTCCGTGCCAGCAGCCGCGGTAATACGGAGGGTGCAAGCGTTAATCGGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCGTCAAGTCGGATGTGAAATCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCTCAGGTGCGAAAGCGTGGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACGCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGATAACCGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGTTAAAACTCAAATG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The tree info looks like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(((((MF183974.1_Kv-5:0.0000000001,&#xA;(MN082125.1_IILSFPR619:0.0000000001,&#xA;(MF183973.1_Kv-4:0.0000000001,&#xA;(MK929064.1_NM43TS4-1:0.0000000001,&#xA;(MF183969.1_Kv-3:0.0000000001,&#xA;(MT507217.1_Uyi23r:0.0000000001,&#xA;MF183971.1_Klebsiella_sp._strain_Kv-2:0.0000000001)0.0000000000:0.0000000001)&#xA;0.0000000000:0.0000000001)0.0000000000:0.0000000001)0.0000000000:0.0000000001)&#xA;0.0000000000:0.0000000001)0.0000000000:0.0000000001,&#xA;MK671348.1_1:0.0000000001)0.8360000000:0.0012113715,&#xA;(MZ475068.1_Klebsiella_variicola_EMBL-1:0.0000000001,&#xA;(MF183972.1_Kv-7:0.0000000001,&#xA;(MF682957.1_BT3409:0.0000000001,&#xA;MF183970.1_Kv-6:0.0000000001)&#xA;0.0000000000:0.0000000001)0.0000000000:0.0000000001)0.0000000000:0.0000000001)&#xA;0.0000000000:0.0012061821,&#xA;((MN725749.1_SA006:0.0000000001,&#xA;MF179616.1_Kv-1:0.0000000001)&#xA;0.0000000000:0.0000024449,&#xA;LN681202.1_Ralstonia_solanacearum_isolate_RsBe2:99.9997264606)&#xA;0.0000000000:0.0000002190)1.0000000000:4.9050549661,&#xA;KC853296.1_JM16:0.0012459932,&#xA;MF373424.1_KV8:0.0000028057);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16048" LastEditorUserId="650" LastEditDate="2022-08-29T20:33:38.377" LastActivityDate="2022-08-29T20:33:38.377" Title="drawing multiple sequence alignment as tree in R" Tags="|phylogenetics|phylogeny|multiple-sequence-alignment|ggtree|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19618" PostTypeId="2" ParentId="19617" CreationDate="2022-08-27T03:35:13.687" Score="1" Body="&lt;p&gt;Firstly, do check &lt;code&gt;tree&lt;/code&gt; is not an empty object, i.e. it uploads the tree file. Secondly, check the path to the alignment is correct &lt;code&gt;get_results&lt;/code&gt;. Normally an alignment file would be suffixed with the alignment format, e.g. &lt;code&gt;get_results.fa&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The immediate thing that strikes me, if all the paths are correct and the file names is the alignment format supplied isn't &lt;code&gt;fasta&lt;/code&gt; format.&lt;/p&gt;&#xA;&lt;p&gt;If you replace the contents of the file &lt;code&gt;get_results&lt;/code&gt; (not &lt;code&gt;get_results.nwk&lt;/code&gt; - because that's your tree file, with the following (wait ... see &lt;strong&gt;Note&lt;/strong&gt; and &lt;strong&gt;Note2&lt;/strong&gt; ) ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    &amp;gt;LN681202.1_Ralstonia_solanacearum_isolate_RsBe2&#xA;TGCAAGTCGAACGGCAGCGGGGGTAGCTTGCTACCTGCCGGCGAGTGGCGAACGGGTGAGTAATACATCGGAACGTGCCCTGTAGTGGGGGATAACTAGTCGAAAGACTAGCTAATACCGCATACGACCTGAGGGTGAAAGTGGGGGACCGCAAGGCCTCATGCTATAGGAGCGGCCGATGTCTGATTAGCTAGTTGGTGGGGTAAAGGCCCACCAAGGCGACGATCAGTAGCTGGTCTGAGAGGACGATCAGCCACACTGGGACTGAGACACGGCCCAGACTCCTACGGGAGGCAGCAGTGGGGAATTTTGGACAATGGGGGCAACCCTGATCCAGCAATGCCGCGTGTGTGAAGAAGGCCTTCGGGTTGTAAAGCACTTTTGTCCGGAAAGAAATCGCTTCGGTTAATACCTGGAGTGGATGACGGTACCGGAAGAATAAGGACCGGCTAACTACGTGCCAGCAGCCGCGGTAATACGTAGGGTCCAAGCGTTAATCGGAATTACTGGGCGTAAAGCGTGCGCAGGCGGTTGTGCAAGACCGATGTGAAATCCCCGGGCTTAACCTGGGAATTGCATTGGTGACTGCACAGCTAGAGTGTGTCAGAGGGAGGTAGAATTCCACGTGTAGCAGTGAAATGCGTAGAGATGTGGAGGAATACCGATGGCGAAGGCAGCCTCCTGGGATAACACTGACGCTCATGCACGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACGCCCTAAACGATGTCAACTAGTTGTTGGGGATTCATTTCCTTAGTAACGTAGCTAACGCGTGAAGTTGA&#xA;&amp;gt;MF179616.1_Kv-1&#xA;GGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCTGTCAAGTCGGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGAGCTACGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAACTTTCCAGAGATGGATTGGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGGTTGTGAAATGTTGGGTTAAGTCCCGCAACCGCAACCCTTATCCTTTGTTGCCAGCGGTCCGGCCGGGAACTCAAAGGAGACTGCAGTGATAAACTGGAGGAAGGTGGGGATGACGTCAAGTCATCATGGCCCTTACCCAGGGCTACACACGTGCTCAATGGCATATACAAAGAGAAGCGACCAGAGCAAGCGGACCTCATAAAGTATGTCGTAGTCGATTGGAGTCTGCAACTCGACTCCATGAAGTCGGAATCGCTAGTAATCGTAGATCAGAATGCTCGGTGAATACGTTCCCGGGCCTTGTACACACCGC&#xA;&amp;gt;MN725749.1_SA006&#xA;GGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCTGTCAAGTCGGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGAGCTACGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAACTTTCCAGAGATGGATTGGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGGTTGTGAAATGTTGGGTTAAGTCCCGCAACCGCAACCCTTATCCTTTGTTGCCAGCGGTCCGGCCGGGAACTCAAAGGAGACTGCAGTGATAAACTGGAGGAAGGTGGGGATGACGTCAAGTCATCATGGCCCTTACCCAGGGCTACACACGTGCTCAATGGCATATACAAAGAGAAGCGACCAGAGCAAGCGGACCTCATAAAGTATGTCGTAGTCGATTGGAGTCTGCAACTCGACTCCATGAAGTCGGAATCGCTAGTAATCGTAGATCAGAATGCTCGGTGAATACGTTCCCGGGCCTTGTACACACCGC&#xA;&amp;gt;MF183971.1_Klebsiella_sp._strain_Kv-2&#xA; GGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCTGTCAAGTCGGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGAGCTACGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAACTTTCCAGAGATGGATTGGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGGTTGTGAAATGTTGGGTTAAGTCCCGCAACCGCAACCCTTATCCTTTGTTGCCAGCGGTTAGGCCGGGAACTCAAAGGAGACTGCAGTGATAAACTGGAGGAAGGTGGGGATGACGTCAAGTCATCATGGCCCTTACCCAGGGCTACACACGTGCTCAATGGCATATACAAAGAGAAGCGACCAGAGCAAGCGGACCTCATAAAGTATGTCGTAGTCGATTGGAGTCTGCAACTCGACTCCATGAAGTCGGAATCGCTAGTAATCGTAGATCAGAATGCTCGGTGAATACGTTCCCGGGCCTTGTACACACCGC&#xA;&amp;gt;MT507217.1_Uyi23r&#xA;GGAATTACTGGGCGTAAAGCGCACGCAGGCGGTCTGTCAAGTCGGATGTGAAATCCCCGGGCTCAACCTGGGAACTGCATTCGAAACTGCAGGCTAGAGTCTTGTAGAGGGGGGTAGAATTCCAGGTGTAGCGGTGAAATGCGTAGAGATCTGGAGGAATACCGGTGGCGAAGGCGGCCCCCTGGACAAAGACTGACGCCAGGTGCGAAAGCGTGGGGAGCAAACAGGATTAGATACCCTGGTAGTCCACCTGTAAACGATGTCGATTTGGAGGTTGTGCCCTTGAGGCGTGGCTTCCGGAGCTACGTTAAATCGACCGCCTGGGGAGTACGGCCGCAAGGAAACTCAAATGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATGTGGTTTAATTCGATGAACGCGAAGAACCTTACCTGGTCTTGACATCCACAGAACTTTCCAGAGATGGATTGGTGCCTTCGGGAACTGTGAGACAGGTGCTGCATGGCTGTCGTCAGCTCGGTTGTGAAATGTTGGGTTAAGTCCCGCAACCGCAACCCTTATCCTTTGTTGCCAGCGGTTAGGCCGGGAACTCAAAGGAGACTGCAGTGATAAACTGGAGGAAGGTGGGGATGACGTCAAGTCATCATGGCCCTTACCCAGGGCTACACACGTGCTCAATGGCATATACAAAGAGAAGCGACCAGAGCAAGCGGACCTCATAAAGTATGTCGTAGTCGATTGGAGTCTGCAACTCGACTCCATGAAGTCGGAATCGCTAGTAATCGTAGATCAGAATGCTCGGTGAATACGTTCCCGGGCCTTGTACACACCGC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Also ....&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Please remove ALL space after the taxa name both in the alignment file and the treefile (I've done it for you in the alignment file)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;LN681202.1_Ralstonia_solanacearum_isolate RsBe2&lt;/code&gt; - the space is a problem between the word &lt;code&gt;isolate&lt;/code&gt; and &lt;code&gt;RsBe2&lt;/code&gt; ... please make sure in the treefile the space is replaced with a &lt;code&gt;_&lt;/code&gt; character.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The names in the treefile must exactly match the names in the alignment file. Just to confirm, the last alignment file you forwarded had more taxa (that space is still an issue BTW), however ...&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Fasta format must start the taxa name with a &lt;code&gt;&amp;gt;&lt;/code&gt; otherwise it will not recognise it.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I think thats the basic error and why its saying &lt;code&gt;null&lt;/code&gt; because it can't parse the alignment file. Getting the treefile names precisely aligned with the alignment taxa names might be an issue.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;&#xA;My fasta alignment format still looks like a dogs dinner, I can't get the fasta format on the markdown language working ....&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;fasta format there must be a carriage return between the taxa name and the sequence.&lt;/li&gt;&#xA;&lt;li&gt;the &lt;code&gt;&amp;gt;&lt;/code&gt; must not have any spaces between the start of the line (unlike my version) until the &lt;code&gt;&amp;gt;&lt;/code&gt; ... nor any spaces anywhere on the line ...&lt;/li&gt;&#xA;&lt;li&gt;Spaces within the taxa name could cause problems in &lt;code&gt;Newick&lt;/code&gt; tree format (depends what program you've used).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note2&lt;/strong&gt;&#xA;The alignment I edited and the new alignment you just supplied (presumably complete) obviously can't be simply cut and paste as I originally thought, but you get the gist. Writing code is a much easier way to do phylogenetics because there's very little manual editing and parsers do everything for you.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;In the tree file you've correctly named &lt;code&gt;LN681202.1_Ralstonia_solanacearum_isolate_RsBe2&lt;/code&gt;, but the same name must in the alignment file. Generally taxa names don't use spaces and are replaced with &lt;code&gt;_&lt;/code&gt;, in &lt;code&gt;phylip&lt;/code&gt; format for example this causes mayhem with the parser.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Finally&lt;/strong&gt;&#xA;If thats all thats wrong it would be cool, I think they'll be more to get it working correctly.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-08-27T12:03:47.653" LastActivityDate="2022-08-27T12:03:47.653" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19619" PostTypeId="2" ParentId="19609" CreationDate="2022-08-27T09:46:32.180" Score="0" Body="&lt;p&gt;&lt;em&gt;Answer from @m, converted from &lt;a href=&quot;https://bioinformatics.stackexchange.com/help/privileges/comment&quot;&gt;comment&lt;/a&gt;:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Biostars are recommending esearch ... its not the approach I'd have taken but presume will do the job. Its available via SRA, so I'd have gone for the SRA toolkit. Is SRA more appropriate in context? (I liked Biostars BeautifulSoup solution)&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;[Please update or improve this answer if you can]&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-08-27T09:46:32.180" CommentCount="0" CommunityOwnedDate="2022-08-27T09:46:32.180" ContentLicense="CC BY-SA 4.0" />
  <row Id="19620" PostTypeId="2" ParentId="19615" CreationDate="2022-08-27T11:47:05.220" Score="0" Body="&lt;p&gt;Suggestions in increasing order of relevance:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The only program that is close is &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-021-03956-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;rootdigger&lt;/a&gt;.&#xA;This finds the sequence most likely to be the root, as the name suggests. Available via &lt;a href=&quot;https://github.com/computations/root_digger&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt;, but not via conda.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;You could just write your &lt;code&gt;PIM&lt;/code&gt; approach as small program? It wouldn't be difficult.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Alternatively, you could just select the longest, shortest or median branch length and use that as a representative sequence. This wouldn't be computationally expensive at all (you'd need the correct root however and a rooted tree*).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The shortest branch length from the root is a reasonable solution. You can do it via a GUI (if its e.g. a Bayesian tree, there's one program that does this), write you own code, or else manually via optical analysis (easy to spot the shortest branch in a rooted tree).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The shortest branch-length is the closest sequence to the ancestral sequence for the in-groups (non-root). You can obtain a reconstruction of the ancestral sequence - it is computationally (very) expensive and isn't a real sequence.&lt;/p&gt;&#xA;&lt;p&gt;*, Point 3 and 4 require a phylogram rather than a dendrogram of course.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-08-27T13:49:05.303" LastActivityDate="2022-08-27T13:49:05.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19622" PostTypeId="1" CreationDate="2022-08-28T12:06:11.870" Score="1" ViewCount="71" Body="&lt;p&gt;I am looking to perform expression analysis of miRNAs with normal RNA-seq data lacking small RNA-seq data?&lt;/p&gt;&#xA;&lt;p&gt;Which path should I choose for known miRNAs and unknown new miRNAs?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Data set:&lt;/strong&gt; rna-seq data that analyzes the expression differences of a plant under stress.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Objective&lt;/strong&gt; To assess known and new miRNAs.&lt;/p&gt;&#xA;" OwnerUserId="16055" LastEditorUserId="3848" LastEditDate="2022-08-29T17:34:41.510" LastActivityDate="2022-08-29T17:34:41.510" Title="Expression analysis of miRNAs with normal RNA-seq data without small RNA-seq data" Tags="|rna-seq|ngs|rna|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19623" PostTypeId="1" CreationDate="2022-08-29T15:07:23.660" Score="1" ViewCount="42" Body="&lt;p&gt;I have a single cell dataset which I consider to be a reference , let say, for an human organ, I have identified some new clusters that correspond to cell types. I also have WGCNA modules that correspond to a identified cluster.&#xA;Given this background, is there any possibilities to validate the presence of the cell types from my reference dataset inside another single cell dataset ?&lt;/p&gt;&#xA;&lt;p&gt;I was thinking about GSEA, but all the gene in a set have the same importance.&#xA;Could I use eigen vectors from first principal component for a given module expressed in the corresponding cell type, as a weight. Then calculate inside another dataset a sort of &amp;quot;first principal component&amp;quot; using the previous eigen vectors. Then I would simply calculate the correlation between the first principal component and the calculated one to validate whether my cell type is present or not.&#xA;I was also thinking about a sort of weigthed GSEA, if anything like this ever existed.&lt;/p&gt;&#xA;" OwnerUserId="16057" LastActivityDate="2022-08-29T17:54:57.213" Title="How can to validate the presence of a certain type of cells in a single cell dataset?" Tags="|single-cell|transcriptome|wgcna|gsea|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19624" PostTypeId="2" ParentId="19622" CreationDate="2022-08-29T16:59:11.753" Score="1" Body="&lt;p&gt;You are going to have to look up the library prep protocol used, and determine if it will capture miRNA.  My guess is, in the course of most RNASeq preps, small products are removed as contamination, so you won't have any real miRNA left behind.&lt;/p&gt;&#xA;&lt;p&gt;I believe you need to use a special library prep designed for miRNA to study them.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2022-08-29T16:59:11.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19625" PostTypeId="2" ParentId="19623" CreationDate="2022-08-29T17:54:57.213" Score="1" Body="&lt;p&gt;I think the typical way to do this would be to identify a small number of marker genes whose presence and absence is specific for that cell type.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2022-08-29T17:54:57.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19626" PostTypeId="1" CreationDate="2022-08-29T23:14:15.770" Score="2" ViewCount="1893" Body="&lt;p&gt;I generated a multiple sequence alignment using mafft, but I would like to convert it to nexus or phylip format (required by a downstream software). What's the easiest way to do the conversion?&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2022-08-30T14:34:53.630" Title="How to convert fasta alignment to nexus or phylip format?" Tags="|sequence-alignment|format-conversion|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19627" PostTypeId="2" ParentId="19626" CreationDate="2022-08-29T23:14:15.770" Score="1" Body="&lt;p&gt;EMBO's bioinformatic suite &lt;a href=&quot;https://www.ebi.ac.uk/Tools/emboss/&quot; rel=&quot;nofollow noreferrer&quot;&gt;EMBOSS&lt;/a&gt; has &lt;a href=&quot;http://emboss.sourceforge.net/apps/cvs/emboss/apps/seqret.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqret&lt;/a&gt; program that can do exactly that. You can install it via conda (&lt;code&gt;conda install -c bioconda emboss&lt;/code&gt;) and then the conversion to &lt;code&gt;nexus&lt;/code&gt; is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqret -sequence input.mafft.fasta -outseq output.mafft.nex -osformat nexus&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="57" LastActivityDate="2022-08-29T23:14:15.770" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19628" PostTypeId="2" ParentId="19626" CreationDate="2022-08-30T01:42:16.613" Score="1" Body="&lt;pre&gt;&lt;code&gt;from Bio import AlignIO&#xA;from pathlib import Path&#xA;from Bio.AlignIO.PhylipIO import SequentialPhylipWriter&#xA;import re&#xA;&#xA;file = 'input.mafft.fasta'&#xA;fileoutnex = re.sub('fasta', 'nex', file)&#xA;pathway = '/mypath/'&#xA;alignment = AlignIO.read(Path(pathway, file), 'fasta')&#xA;&#xA;# write out in nexus&#xA;AlignIO.write(alignment, Path(pathway, fileoutnex), 'nexus')&#xA;&#xA;# write out in relaxed phylip&#xA;fileoutphy = re.sub('fasta', 'phy', file)&#xA;with open(Path(pathway, fileoutphy), 'w') as output_handle:        &#xA;    SequentialPhylipWriter(output_handle).write_alignment(alignment, id_width=30)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The advantage here is you can select what phylip format you want. That is a BIG advantage, traditional phylip format is terrible, particularly with taxa ids lengths. Here relaxed phylip format is being used.&lt;/p&gt;&#xA;&lt;p&gt;The other advantages,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Easily parse, change, alter the taxa names;&lt;/li&gt;&#xA;&lt;li&gt;Phylip taxa id can be ANY length (&lt;code&gt;id_width=&lt;/code&gt;);&lt;/li&gt;&#xA;&lt;li&gt;save it as a script;&lt;/li&gt;&#xA;&lt;li&gt;no need to write an outfile (ok minor);&lt;/li&gt;&#xA;&lt;li&gt;pipe the alignment straight into downstream stuff;&lt;/li&gt;&#xA;&lt;li&gt;very easy manipulate the alignment.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-08-30T14:34:53.630" LastActivityDate="2022-08-30T14:34:53.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19629" PostTypeId="1" CreationDate="2022-08-30T11:14:05.443" Score="1" ViewCount="23" Body="&lt;p&gt;I am confused about the difference between sequence-independent and sequence-order independent structural alignment. TM-align for example claims to be sequence-independent; however, I've seen it is also sequence order independent from the papers. To me, they sound like the same thing, but obviously, they aren't. Does anyone know about anything the difference?&lt;/p&gt;&#xA;" OwnerUserId="16065" LastActivityDate="2022-08-30T11:14:05.443" Title="The difference between sequence-independent and sequence-order independent structural alignment?" Tags="|sequence-alignment|structural-biology|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19630" PostTypeId="1" AcceptedAnswerId="19647" CreationDate="2022-08-30T18:35:24.517" Score="4" ViewCount="115" Body="&lt;p&gt;I'm a new Bioinformatic scientist working for a yeast genetics company.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Objective&lt;/strong&gt; To create a database of yeast genomes from NCBI and identify SNP variants.&lt;/p&gt;&#xA;&lt;p&gt;In my pipeline&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;FastQC,&lt;/li&gt;&#xA;&lt;li&gt;Trimmomatic,&lt;/li&gt;&#xA;&lt;li&gt;BWA&lt;/li&gt;&#xA;&lt;li&gt;GATK&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The method being to check the quality of the downloaded reads, trim off any unnecessary information, and to align them prior to VCF.&lt;/p&gt;&#xA;&lt;p&gt;NCBI sequence data comes from both &lt;em&gt;Illumina&lt;/em&gt; sequencing and &lt;em&gt;PacBio&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is the pipeline reasonable, for example, is BWA ok over more recent developments?&lt;/li&gt;&#xA;&lt;li&gt;Could the same pipeline be used both for Illumina and PacBio sequences?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;&#xA;I don't plan on assembling any of the reads I have as I'm just looking to comb through to find variations.&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="3694" LastEditDate="2022-09-02T08:37:52.187" LastActivityDate="2023-06-01T07:07:01.017" Title="Find SNPs in yeast genomes" Tags="|snp|variant-calling|genetics|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="19631" PostTypeId="1" CreationDate="2022-08-31T16:05:48.523" Score="1" ViewCount="77" Body="&lt;p&gt;I tried to run the below code but ended having issue. I have checked thoroughly  and I didn't find any mismatch in fasta sequence&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tree = read.tree(&amp;quot;phylotree.txt&amp;quot;)&#xA;&#xA;ggtree(tree, layout='circular') + &#xA;   geom_tiplab(offset=4, align=TRUE) + xlim(NA, 12)&#xA;msaplot(tree, fasta =&amp;quot;phylotree.fasta&amp;quot;, window=c(120, 200))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in msaplot(tree, fasta = &amp;quot;phylotree.fasta&amp;quot;, window = c(120,&#xA;200)) :    taxa name in input sequences are not match with the ones on&#xA;the tree, please check your input files...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16048" LastEditorUserId="650" LastEditDate="2022-08-31T18:16:30.383" LastActivityDate="2022-08-31T18:16:30.383" Title="difficult in runing the R code for ggtree" Tags="|r|phylogeny|multiple-sequence-alignment|ggtree|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19632" PostTypeId="1" AcceptedAnswerId="19633" CreationDate="2022-08-31T18:25:10.837" Score="3" ViewCount="78" Body="&lt;p&gt;I have done a DGE recently and have been looking at the DGE list. One of the genes is HLA-A. However, when I dug deeper I realised there are hundreds of HLA-A genes with unique ENSEMBL number (of course it's HLA...). A few other HLA-A genes also show up on the table but are not significant at all.&lt;/p&gt;&#xA;&lt;p&gt;Considering the way Salmon deals with multimapping reads, can I assume these expressing genes are different enough that they shouldn't be consider the same? Or should the expression of all HLA-A genes considered as one single gene for the purpose of identifying differentially expressed genes?&lt;/p&gt;&#xA;&lt;p&gt;I do understand that this may be more of a biological question than bioinformatics, but I also want to see if it is a practice pouring the HLA genes (or other variable genes/loci of similar nature) with different sequences together when doing DGE, and whether the statistics would be valid if I do so?&lt;/p&gt;&#xA;" OwnerUserId="15648" LastEditorUserId="15648" LastEditDate="2022-09-01T15:16:52.790" LastActivityDate="2022-09-02T04:02:41.947" Title="How should I interpret DGE results if only one HLA-A gene shows up as significant but not the others?" Tags="|r|rna-seq|differential-expression|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19633" PostTypeId="2" ParentId="19632" CreationDate="2022-08-31T21:24:00.503" Score="0" Body="&lt;p&gt;It is bioinformatics IMO because HLA is a massive area of human population genetics and its complex. As a personal opinion I think you need further supporting evidence.&lt;/p&gt;&#xA;&lt;p&gt;If I understand the scenario correctly,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;you are confident the data is correct (see below),&lt;/li&gt;&#xA;&lt;li&gt;a given HLA-A genotype is significant above other HLA-A and HLA-B genotypes&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thats publishable result, cutting-edge potential, you just need to find the given genotype and represent the association within a given disease association study. Yes your result is absolutely of interest particularly if you consider the wider output of association studies (HLA-B). It is trendy to identify a given HLA to a given association. If this is non-pathogen related they HLA exhibits very strong population skews.&lt;/p&gt;&#xA;&lt;p&gt;I have put a large caveat below, because:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;you shouldn't underestimate the complexity of this research field and&lt;/li&gt;&#xA;&lt;li&gt;I don't really understand the specifics of your study.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;If you want to ditch or include HLA it would require carefully defined biological aims. It is a specialist area of investigation IMO (below).&lt;/p&gt;&#xA;&lt;p&gt;HLA diversity and its link to population shifts in infection susceptibility is subject to repeated studies. Different populations show very distinct allelic frequencies. For example, &amp;quot;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3858009/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HLA-B*40 Allele Plays a Role in the Development of Acute Leukemia&lt;/a&gt;&amp;quot;. Distinct link between &lt;a href=&quot;https://onlinelibrary.wiley.com/doi/full/10.1111/mec.14366&quot; rel=&quot;nofollow noreferrer&quot;&gt;African population and malaria&lt;/a&gt; .. These just being a very small sample in a hugely active area of investigation. The idea is that specific genotypes have very specific outcomes in human health.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Rationale&lt;/strong&gt; The single gist is HLA this represents the cellular immune response and is considered essential for infection control, with the notable exception of pathogenic bacteria. Each genotype is considered to be uniquely functional and again is well known to exhibit population skews. It is beyond 'trendy' is huge and established field of investigation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Caveat&lt;/strong&gt; If you were looking for 'wiggle room' to avoid HLA-A diversity and allelic frequency and all the associated work it requires ... or you specifically want to make it a standout result ... then the angles that might work are ...&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;alot of examples of association focus more on HLA-B than HLA-A. This is a rule of thumb and might be disputed. So if you're going to ignore HLA-A you could perform an extensive literature review to see the extent that associations of HLA-B predominate. I think you might find that holds. So its either really cool cause its an outlier, or you can ditch it because its an outlier.&lt;/li&gt;&#xA;&lt;li&gt;If pathogenic bacteria were the focus of your investigation, then most would accept HLA is not relevant.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I'm sure there's other angles of justification, e.g. &amp;quot;I recognise HLA-A .... but this will be subject to further investigation by (putative) collaboration&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Advice&lt;/strong&gt; If I was tackling this data I would contact a &lt;em&gt;bioinformatics immunologist&lt;/em&gt;, because they will have databases and pipelines dedicated to the exact task you require. Personally, I would be reluctant to take this on without specialist assistance, at a minimum someone with current experience. Essentially, what I'm trying to say, is you will need to understand the HLA field because crossing into it isn't trivial. The situations where I could be wrong are that its simply so complex that no-one really understands it, in which case your good to go.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-08-31T22:21:47.283" LastActivityDate="2022-08-31T22:21:47.283" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19634" PostTypeId="1" CreationDate="2022-09-01T09:20:22.713" Score="2" ViewCount="914" Body="&lt;p&gt;I have a FASTA file with 97,000 entires arranged like this :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;tr|A0A075B6G3|A0A075B6G3_HUMAN Dystrophin OS=Homo sapiens OX=9606 GN=DMD PE=1 SV=1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to have an output like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;A0A075B6G3|1|training&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am aware of many one liners in &lt;code&gt;sed&lt;/code&gt;, &lt;code&gt;perl&lt;/code&gt; and &lt;code&gt;awk&lt;/code&gt; but none are working . I have something like this :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '/&amp;gt;/{$0=&amp;quot;&amp;gt;ID|1|training&amp;quot;}1'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Kindly help.&lt;/p&gt;&#xA;" OwnerUserId="7236" LastActivityDate="2022-09-04T10:27:32.450" Title="change/edit FASTA headers" Tags="|fasta|awk|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19635" PostTypeId="1" CreationDate="2022-09-01T09:32:32.133" Score="1" ViewCount="46" Body="&lt;p&gt;I conducted a differential expression analysis over several datasets, using &lt;code&gt;LIMMA&lt;/code&gt;, each one on its own. For each dataset, I have a data frame of &lt;strong&gt;all&lt;/strong&gt; the genes, with the LFC and the p-value. Those data frames are called &lt;code&gt;deg1&lt;/code&gt;, &lt;code&gt;deg2&lt;/code&gt;, &lt;code&gt;deg3&lt;/code&gt; ... and so on.&lt;/p&gt;&#xA;&lt;p&gt;I want to use the &lt;code&gt;MetaVolcanoR&lt;/code&gt; package, using the &lt;code&gt;votecount_mv&lt;/code&gt; approach since it is the only approach that I can pick the p-value and LFC thresholds that I desire.&lt;/p&gt;&#xA;&lt;p&gt;The problem is I'm not sure how to tune the parameters. There are several problems that I'm facing and I would really use some help please:&lt;/p&gt;&#xA;&lt;p&gt;How do I set the log fold change threshold if I want only &lt;code&gt;LFC &amp;gt; 0.5 and LFC &amp;lt; -0.5&lt;/code&gt;?&#xA;Is it &lt;code&gt;log fold change = 0.5&lt;/code&gt; ? just making sure.&lt;/p&gt;&#xA;&lt;p&gt;What is the &lt;code&gt;metathr&lt;/code&gt; parameter? I read about it and still don't understand what it does. I set it as 0.01 for now.&lt;/p&gt;&#xA;&lt;p&gt;And the main problem, the plot is showing much more gene number than there actually is. You can see in &lt;code&gt;Dataset6&lt;/code&gt; for example, the plot says it has over 50,000 genes while in reality, it has something like 18,000 genes. The same for Dataset8, why is this happening?&lt;/p&gt;&#xA;&lt;h4&gt;This is the plot:&lt;/h4&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/tRe1r.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tRe1r.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h4&gt;This is my code:&lt;/h4&gt;&#xA;&lt;pre&gt;&lt;code&gt;totalDEG = list(Dataset1 = deg1, Dataset2 = deg2,&#xA;                Dataset3 = deg3, Dataset4 = deg4, Dataset5 = deg5, Dataset6 = deg6, &#xA;                Dataset7 = deg7, Dataset8 = deg8 ,Dataset9 = deg9, Dataset10 = deg10,&#xA;                Dataset11 = deg11, Dataset12 = deg12, Dataset13 = deg13 , Dataset15 = deg15,&#xA;                Dataset16 = deg16, Dataset20 = deg20, Dataset21 = deg21)&#xA;&#xA;totalDEG = map(totalDEG, ~ .x %&amp;gt;% rownames_to_column(&amp;quot;symbol&amp;quot;) %&amp;gt;% `row.names&amp;lt;-`(.$symbol))&#xA;&#xA;meta_degs_vote &amp;lt;- votecount_mv(diffexp=totalDEG,&#xA;                               pcriteria='P.Value', &#xA;                               foldchangecol='logFC',&#xA;                               genenamecol='symbol',&#xA;                               geneidcol=NULL,&#xA;                               pvalue = 0.05,&#xA;                               foldchange = 0.5, &#xA;                               metathr=0.01,&#xA;                               collaps=FALSE,&#xA;                               jobname=&amp;quot;MetaVolcano&amp;quot;, &#xA;                               outputfolder=&amp;quot;.&amp;quot;,&#xA;                               draw='HTML')&#xA;&#xA;head(meta_degs_vote@metaresult, 50)&#xA;meta_degs_vote@degfreq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h4&gt;This is a small sample of the &lt;code&gt;deg13&lt;/code&gt; dataframe, just as an example of how it looks:&lt;/h4&gt;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(logFC = c(3.39094029649578, 2.91185452846888, &#xA;-4.18752593306854, 4.15866027194434, -3.46058235630518, -2.87208933878881, &#xA;-3.08172118179278, -2.7458737595494, -4.086372553458, -2.68889593085408, &#xA;-3.03835320252487, 2.61330670793883, -2.48123655627091, 1.49263378072949, &#xA;1.95754205818365, -2.70230318294949, -2.9154447157702, -3.38110311321268, &#xA;-2.35284609598761, -3.61327208834916, 2.03692346164301, -1.70931971282814, &#xA;2.27925549238084, -3.79524834465106, 1.64515339181668, -4.30959811283984, &#xA;3.57930610863801, -2.54394115096987, -1.98143797060853, 4.03178891878422&#xA;), AveExpr = c(6.4934411845873, 6.79507589173635, 3.87626322696796, &#xA;5.02990511836996, 3.37154863566726, 4.54352970704667, 2.51617183960423, &#xA;5.36760482831628, 8.74162861573995, 3.52708473144296, 1.56671205994866, &#xA;5.87684931921495, 4.44218678637856, 5.12219572880673, 7.17539095745285, &#xA;3.53733757517713, 3.35148844504943, 5.15160050823647, 4.75910049955374, &#xA;2.76067393839851, 7.59404366884022, 3.26758602409262, 6.6742932845645, &#xA;2.02865432630183, 7.35343444625846, 1.63566796485655, 6.63046957559406, &#xA;4.0296762052982, 5.52320581646106, 5.21519413005468), t = c(7.67283765970783, &#xA;7.48874311179818, -6.82705440075341, 6.17183091062419, -6.04487485029223, &#xA;-5.8487763400982, -5.92000790680834, -5.64958317461715, -5.61111489307273, &#xA;-5.58440722941211, -5.78651741270403, 5.34175048353288, -5.31088208376663, &#xA;5.08100365503679, 5.03973023175764, -5.07773225272668, -5.04632997176375, &#xA;-4.96736474738699, -4.82583305184561, -4.92525789154973, 4.78892270501915, &#xA;-4.75294466621978, 4.64502680172295, -4.88600418339975, 4.62243262623093, &#xA;-4.78866983562446, 4.56376665911121, -4.57063797766699, -4.48421852802591, &#xA;4.47493043300733), P.Value = c(8.61557824744483e-06, 1.08572492426076e-05, &#xA;2.57808593137873e-05, 6.41321288302935e-05, 7.70283126846018e-05, &#xA;0.000102673342223582, 9.24388613776644e-05, 0.000138234657017388, &#xA;0.000146500473045098, 0.000152546919410067, 0.000112607650587545, &#xA;0.000221306832937214, 0.000232172444273951, 0.000333174926637586, &#xA;0.000355778600664596, 0.000334909952712188, 0.000352058129044564, &#xA;0.000399405716020234, 0.000501862041539822, 0.000427357183393412, &#xA;0.000532900955175219, 0.000565105101377844, 0.000674574457630558, &#xA;0.000455275577064366, 0.000700192843853878, 0.000533120436605821, &#xA;0.000771595292301189, 0.000762850982694916, 0.00088083209108589, &#xA;0.000894604612049976), adj.P.Val = c(0.105689892752164, 0.105689892752164, &#xA;0.167309183326708, 0.24749466449955, 0.24749466449955, 0.24749466449955, &#xA;0.24749466449955, 0.24749466449955, 0.24749466449955, 0.24749466449955, &#xA;0.24749466449955, 0.322868951254968, 0.322868951254968, 0.364560714544159, &#xA;0.364560714544159, 0.364560714544159, 0.364560714544159, 0.388801494259897, &#xA;0.397198753468022, 0.396200809689826, 0.397198753468022, 0.397198753468022, &#xA;0.397198753468022, 0.397198753468022, 0.397198753468022, 0.397198753468022, &#xA;0.397198753468022, 0.397198753468022, 0.397198753468022, 0.397198753468022&#xA;), B = c(3.51320342275366, 3.35460378796296, 2.44903607778273, &#xA;1.62363531926393, 1.57839963166078, 1.44388223679157, 1.36898233186455, &#xA;1.24528027435772, 1.14177006290097, 1.04403303682497, 0.945243757754988, &#xA;0.859404654981856, 0.797031901979986, 0.496397382158138, 0.450148400219772, &#xA;0.418557695883232, 0.3417613688148, 0.324284286390148, 0.136478287041671, &#xA;0.0982449107580958, 0.0925437909507512, -0.0280517206307875, &#xA;-0.112494849608916, -0.129424832005619, -0.147717311976667, -0.200056933894112, &#xA;-0.234222113297706, -0.244412573722838, -0.351160223852428, -0.386270016806261&#xA;)), row.names = c(&amp;quot;FOS&amp;quot;, &amp;quot;DUSP1&amp;quot;, &amp;quot;TP53AIP1&amp;quot;, &amp;quot;FOSB&amp;quot;, &amp;quot;TFAP2C&amp;quot;, &#xA;&amp;quot;KIF26A&amp;quot;, &amp;quot;GRHL3&amp;quot;, &amp;quot;CTAGE4&amp;quot;, &amp;quot;FCGBP&amp;quot;, &amp;quot;FBXO27&amp;quot;, &amp;quot;AC087289.3&amp;quot;, &#xA;&amp;quot;SH3BP5&amp;quot;, &amp;quot;GABRE&amp;quot;, &amp;quot;ADAMTS4&amp;quot;, &amp;quot;DDHD1&amp;quot;, &amp;quot;KCNS3&amp;quot;, &amp;quot;ARHGEF16&amp;quot;, &amp;quot;SERINC2&amp;quot;, &#xA;&amp;quot;OPLAH&amp;quot;, &amp;quot;WNT7B&amp;quot;, &amp;quot;GSN&amp;quot;, &amp;quot;OVOL2&amp;quot;, &amp;quot;CD163&amp;quot;, &amp;quot;SVOPL&amp;quot;, &amp;quot;ZFP36&amp;quot;, &#xA;&amp;quot;SPESP1&amp;quot;, &amp;quot;FGD5&amp;quot;, &amp;quot;FAAH&amp;quot;, &amp;quot;ERBB3&amp;quot;, &amp;quot;PARM1&amp;quot;), class = &amp;quot;data.frame&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15996" LastEditorUserId="15996" LastEditDate="2022-09-03T10:39:33.353" LastActivityDate="2022-09-03T10:39:33.353" Title="How to tune and use the MetaVolcanoR package" Tags="|r|bioconductor|differential-expression|limma|meta-analysis|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19636" PostTypeId="2" ParentId="19634" CreationDate="2022-09-01T11:19:36.463" Score="7" Body="&lt;p&gt;One way using &lt;a href=&quot;/questions/tagged/awk&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; rel=&quot;tag&quot; aria-labelledby=&quot;awk-container&quot;&gt;awk&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN { FS=OFS=&amp;quot;|&amp;quot; } /^&amp;gt;/ { print &amp;quot;&amp;gt;&amp;quot; $2, &amp;quot;1&amp;quot;, &amp;quot;training&amp;quot;; next }1' file.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This sets the &lt;a href=&quot;https://www.gnu.org/software/gawk/manual/html_node/Field-Separators.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;field separator&lt;/em&gt; (FS)&lt;/a&gt; and the &lt;a href=&quot;https://www.gnu.org/software/gawk/manual/html_node/Output-Separators.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;output field separator&lt;/em&gt; (OFS)&lt;/a&gt;  to a pipe character. Then for each header line, we can just print the fields we want. The output fields will be delimited with pipe characters since we already set &lt;code&gt;OFS&lt;/code&gt; to a pipe character in the &lt;code&gt;BEGIN&lt;/code&gt; block. We can use &lt;a href=&quot;https://www.gnu.org/software/gawk/manual/html_node/Next-Statement.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;next&lt;/em&gt;&lt;/a&gt; here to force awk to immediately stop processing the current record and move on to the next record, which is just the next line, since, by default, awk's &lt;a href=&quot;https://www.gnu.org/software/gawk/manual/html_node/awk-split-records.html&quot; rel=&quot;noreferrer&quot;&gt;&lt;em&gt;record separator&lt;/em&gt;&lt;/a&gt; is a newline character.&lt;/p&gt;&#xA;&lt;p&gt;Since the &lt;code&gt;1&lt;/code&gt; on the end just returns true, it performs the default action, which is to print the current record stored in &lt;code&gt;$0&lt;/code&gt;. It is just the same as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN { FS=OFS=&amp;quot;|&amp;quot; } /^&amp;gt;/ { print &amp;quot;&amp;gt;&amp;quot; $2, &amp;quot;1&amp;quot;, &amp;quot;training&amp;quot;; next } { print $0 }'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which is the same as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN { FS=OFS=&amp;quot;|&amp;quot; } /^&amp;gt;/ { print &amp;quot;&amp;gt;&amp;quot; $2, &amp;quot;1&amp;quot;, &amp;quot;training&amp;quot;; next } { print }'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2022-09-01T11:19:36.463" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19637" PostTypeId="1" CreationDate="2022-09-01T11:25:03.730" Score="3" ViewCount="53" Body="&lt;p&gt;I have the results of a virtual screening experiment using docking simulations with Autodock Vina. The result is a matrix of 7 (proteins) by 28000 (ligands) with the calculated binding energies for each protein/ligand pair:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;Ligands - Proteins&lt;/th&gt;&#xA;&lt;th&gt;Protein1&lt;/th&gt;&#xA;&lt;th&gt;Protein2&lt;/th&gt;&#xA;&lt;th&gt;Protein3&lt;/th&gt;&#xA;&lt;th&gt;Protein4&lt;/th&gt;&#xA;&lt;th&gt;Protein5&lt;/th&gt;&#xA;&lt;th&gt;Protein6&lt;/th&gt;&#xA;&lt;th&gt;Protein7&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Ligand1&lt;/td&gt;&#xA;&lt;td&gt;-5.4&lt;/td&gt;&#xA;&lt;td&gt;-5.9&lt;/td&gt;&#xA;&lt;td&gt;-5.0&lt;/td&gt;&#xA;&lt;td&gt;-5.2&lt;/td&gt;&#xA;&lt;td&gt;-5.8&lt;/td&gt;&#xA;&lt;td&gt;-5.7&lt;/td&gt;&#xA;&lt;td&gt;-4.5&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Ligand2&lt;/td&gt;&#xA;&lt;td&gt;-9.5&lt;/td&gt;&#xA;&lt;td&gt;-8.6&lt;/td&gt;&#xA;&lt;td&gt;-8.9&lt;/td&gt;&#xA;&lt;td&gt;-9.0&lt;/td&gt;&#xA;&lt;td&gt;-9.2&lt;/td&gt;&#xA;&lt;td&gt;-7.1&lt;/td&gt;&#xA;&lt;td&gt;-6.4&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Ligand3&lt;/td&gt;&#xA;&lt;td&gt;-9.5&lt;/td&gt;&#xA;&lt;td&gt;-8.3&lt;/td&gt;&#xA;&lt;td&gt;-8.2&lt;/td&gt;&#xA;&lt;td&gt;-9.9&lt;/td&gt;&#xA;&lt;td&gt;nan&lt;/td&gt;&#xA;&lt;td&gt;-6.8&lt;/td&gt;&#xA;&lt;td&gt;nan&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Ligand4&lt;/td&gt;&#xA;&lt;td&gt;-9.4&lt;/td&gt;&#xA;&lt;td&gt;-7.8&lt;/td&gt;&#xA;&lt;td&gt;-8.0&lt;/td&gt;&#xA;&lt;td&gt;-6.9&lt;/td&gt;&#xA;&lt;td&gt;-9.5&lt;/td&gt;&#xA;&lt;td&gt;-1.3&lt;/td&gt;&#xA;&lt;td&gt;-7.8&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Ligand5&lt;/td&gt;&#xA;&lt;td&gt;-9.3&lt;/td&gt;&#xA;&lt;td&gt;-6.9&lt;/td&gt;&#xA;&lt;td&gt;-8.5&lt;/td&gt;&#xA;&lt;td&gt;-7.9&lt;/td&gt;&#xA;&lt;td&gt;-9.2&lt;/td&gt;&#xA;&lt;td&gt;-6.8&lt;/td&gt;&#xA;&lt;td&gt;-6.9&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;---&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Ligand28000&lt;/td&gt;&#xA;&lt;td&gt;-6.3&lt;/td&gt;&#xA;&lt;td&gt;-7.9&lt;/td&gt;&#xA;&lt;td&gt;-5.4&lt;/td&gt;&#xA;&lt;td&gt;-2.7&lt;/td&gt;&#xA;&lt;td&gt;-10.1&lt;/td&gt;&#xA;&lt;td&gt;-9.4&lt;/td&gt;&#xA;&lt;td&gt;-3.9&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;nan results are either failed dockings, or positive values.&lt;/p&gt;&#xA;&lt;p&gt;The scores are not only affected by how well a ligand docks with each protein, but also &lt;strong&gt;bigger (higher molecular weight) ligands tend to show better (more negative) scores&lt;/strong&gt;. However, this is in part an artifact and it does not imply that bigger molecules are better and smaller ones are worse.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/JgNKo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JgNKo.png&quot; alt=&quot;energy/size graph&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Typically, for a virtual screening with just one protein, you would select the top ranking ligands and that would be it (e.g. keep the top 1% and test them). However, this results in some ligands selected for most proteins because they invariably score better due to their size.&lt;/p&gt;&#xA;&lt;p&gt;In this case, I could also take into account the variability of each ligand because I have observations for different unrelated proteins (i.e. see if a particular ligand systematically binds with better scores across proteins).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is there any way to take into consideration both factors when selecting the best ligands?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I thought of calculating Z-scores 1) across rows and 2) across columns, and select the ligands for each protein that have less than -1.65 standard deviations (p=0.05) and that also fill the same criteria across proteins. However, there are only 7 observations for each ligand to calculate the Z-scores, and it does not feel correct to give it the same weight to the Z-scores calculated from the 28000 observations for each protein. Plus, in the end, protein docking effect is more important than the ligand effect.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestions would be greatly appreciated!&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://jcheminf.biomedcentral.com/articles/10.1186/s13321-019-0330-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ligand efficiency&lt;/a&gt; looks like a promising suggestion. However, upon trying (calculating ∆G/MW or ∆G/n.heavy_atoms) resulted in very small ligands with mediocre or bad energies going to the top (e.g. urea with -3.1 became a Z-score of -8.5 SDs).&lt;/p&gt;&#xA;" OwnerUserId="10254" LastEditorUserId="73" LastEditDate="2022-09-02T10:43:46.983" LastActivityDate="2022-09-02T10:43:46.983" Title="How to select the best ligands in a virtual screening matrix" Tags="|statistics|docking|drugs|cheminformatics|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="19638" PostTypeId="2" ParentId="19634" CreationDate="2022-09-01T12:39:44.120" Score="1" Body="&lt;pre&gt;&lt;code&gt;perl -p -i -e 's/&amp;gt;[a-z]+?\|([A-Z]+[^\|]+?\|).*SV=([0-9]+)/&amp;gt;&lt;span class=&quot;math-container&quot;&gt;$1$&lt;/span&gt;2\|training/g' myfilename.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Not tested yet. Just check the sequence ID starts with a capital A-Z character. Probably wise to test it without the &lt;code&gt;-i&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-09-01T13:20:23.577" LastActivityDate="2022-09-01T13:20:23.577" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19639" PostTypeId="2" ParentId="19615" CreationDate="2022-09-01T14:20:17.733" Score="2" Body="&lt;p&gt;I have two suggestions that I used in the past (both using Python/BioPython though). They are arguably the same, but obtained through different paths.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Calculate the (artificial) consensus sequence of your alignment. Then, compare each sequence in your MSA to the consensus in a pairwise alignment, keeping the score. The highest scoring &lt;strong&gt;real&lt;/strong&gt; sequence will be your representative.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import AlignIO&#xA;from Bio import pairwise2&#xA;from collections import Counter&#xA;&#xA;aln_file = &amp;quot;MSA.aln&amp;quot;&#xA;alignment = AlignIO.read(aln_file, &amp;quot;clustal&amp;quot;) # whatever format you have&#xA;&#xA;for i in range(0,alignment.get_alignment_length()):  # iterate position by position&#xA;    aa_position = []&#xA;&#xA;    # Get all residues in that position&#xA;    for record in alignment:&#xA;        aa_position.append(record.seq[i])&#xA;&#xA;     # Use the most common residue as representative&#xA;    consensus_list.append(Counter(aa).most_common()[0][0])&#xA;consensus_seq = &amp;quot;&amp;quot;.join(consensus_list)&#xA;&#xA;ref_seq = None&#xA;highest_score = 0&#xA;for record in alignment:&#xA;    # globalxx finds the best global alignment between both sequences.&#xA;    # The first x indicates that identical characters are given 1 point, 0 otherwise&#xA;    # Second x indicates no gap penalties&#xA;    pairwise_score = pairwise2.align.globalxx(record.seq,&#xA;                                              consensus_seq,&#xA;                                              score_only=True&#xA;                                              )&#xA;    if pairwise_score &amp;gt; highest_score:&#xA;        highest_score = pairwise_score&#xA;        ref_seq = record&#xA;    elif pairwise_score == highest_score:&#xA;        # If they are equal, you can prioritize other things&#xA;        pass&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;You can again obtain the consensus sequence, and then calculate the Levenshtein (edit) distance between each record and the consensus. The sequence with the lowest distance will be the most representative. The easiest way to calculate it is via installing the python-Levenshtein module with &lt;code&gt;pip install python-Levenshtein&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Levenshtein import distance as edit_distance&#xA;&#xA;# First, calculate the consensus as described above&#xA;&#xA;ref_seq = None&#xA;best_distance = 999999999999&#xA;for record in alignment:&#xA;    distance = edit_distance(record.seq, consensus_seq)&#xA;    if distance &amp;lt; best_distance:&#xA;        best_distance = distance &#xA;        ref_seq = record&#xA;    elif distance == best_distance:&#xA;        # If they are equal, you can prioritize other things&#xA;        pass&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10254" LastActivityDate="2022-09-01T14:20:17.733" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19640" PostTypeId="2" ParentId="19300" CreationDate="2022-09-01T14:26:39.717" Score="1" Body="&lt;p&gt;You can access the peak positions of Sanger Sequencing chromatograms from &lt;code&gt;[&amp;quot;PLOC1&amp;quot;]&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The position values in &lt;code&gt;[&amp;quot;PLOC1&amp;quot;]&lt;/code&gt; correspond to the index values in &lt;code&gt;[&amp;quot;DATA9&amp;quot;]&lt;/code&gt;, &lt;code&gt;[&amp;quot;DATA10&amp;quot;]&lt;/code&gt;, &lt;code&gt;[&amp;quot;DATA11&amp;quot;]&lt;/code&gt;, &lt;code&gt;[&amp;quot;DATA12&amp;quot;]&lt;/code&gt;, so you can extract the signal values of each base at the positions.&lt;/p&gt;&#xA;" OwnerUserId="16083" LastEditorUserId="650" LastEditDate="2022-09-02T19:11:05.987" LastActivityDate="2022-09-02T19:11:05.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19641" PostTypeId="2" ParentId="19632" CreationDate="2022-09-02T04:02:41.947" Score="3" Body="&lt;p&gt;You need specialized tools to study HLA genes. Generic tools won't work well. Before other analysis, you should get the correct genotype first. Try &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/36/1/33/5512361&quot; rel=&quot;nofollow noreferrer&quot;&gt;arcasHLA&lt;/a&gt;. It has been a popular HLA genotyper for RNA-seq data. When you know the genotype, I guess you may get rid of other HLA-A alleles and redo the differential analysis if you are interested in HLA-A. I haven't done this, though. Just a thought.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2022-09-02T04:02:41.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19642" PostTypeId="2" ParentId="19608" CreationDate="2022-09-02T10:04:42.003" Score="1" Body="&lt;p&gt;You can use Python &amp;quot;normally&amp;quot; and import the &lt;code&gt;pymol&lt;/code&gt; module. In the script that holds the proteins in your matrix, you can add a function that does the super alignment.&lt;/p&gt;&#xA;&lt;p&gt;To use it, in your console type &lt;code&gt;pymol -cq this_script_name.py&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You need to have pymol installed and accessed via console. Easiest way I found to manage it is to create a new environment via conda and install pymol there.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pymol&#xA;&#xA;&#xA;def super_prots(prot_1, prot_2):&#xA;    pymol.cmd.fetch(prot_1, 'prot_1')&#xA;    pymol.cmd.fetch(prot_2, 'prot_2')&#xA;&#xA;    # If you are using already downloaded pdb files, use load instead:&#xA;    # pymol.cmd.load(prot_1, 'prot_1')&#xA;    # pymol.cmd.load(prot_2, 'prot_2')&#xA;&#xA;    pymol.cmd.hide('all')&#xA;    pymol.cmd.set('ray_opaque_background', 0)&#xA;&#xA;    # Show cartoon and paint them different&#xA;    pymol.cmd.show('cartoon', 'prot_1')&#xA;    pymol.cmd.color('orange', 'prot_1')&#xA;    pymol.cmd.show('cartoon', 'prot_2')&#xA;    pymol.cmd.color('lightblue', 'prot_2')&#xA;&#xA;    # Align both proteins&#xA;    pymol.cmd.super('prot_1', 'prot_2')&#xA;&#xA;    # Improve image quality, and get png.&#xA;    pymol.cmd.set('depth_cue', 0)&#xA;    pymol.cmd.set('spec_reflect', 0)&#xA;    pymol.cmd.set('cartoon_sampling', 15)&#xA;    pymol.cmd.set('ribbon_sampling', 15)&#xA;    pymol.cmd.set('antialias', 2)&#xA;    pymol.cmd.space('cmyk')&#xA;&#xA;    # This creates high-quality images, but might take a long time to&#xA;    # process. Adjust parameters accordingly.&#xA;    pymol.cmd.png(f'{prot_1}_{prot_2}.png',&#xA;                  width=2000,&#xA;                  dpi=300,&#xA;                  ray=1&#xA;                  )&#xA;    pymol.cmd.reinitialize()&#xA;&#xA;if __name__ == '__main__':&#xA;    # These are entries from PDB. It can work with already downloaded&#xA;    # files, but you have to change the function accordingly&#xA;    protein_matrix = [['1EWL', '4W5B'],&#xA;                      ['4E2D', '3ATZ'],&#xA;                      ]&#xA;    for prot_1, prot_2 in protein_matrix:&#xA;        super_prots(prot_1, prot_2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will generate .png images of the alignment, but you can also ask it to save the alignment into .pdb files adding &lt;code&gt;pymol.cmd.save(f'{prot_1}_{prot_2}.pdb')&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The commands' documentation can be found &lt;a href=&quot;https://pymol.org/pymol-command-ref.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="10254" LastActivityDate="2022-09-02T10:04:42.003" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19643" PostTypeId="2" ParentId="13477" CreationDate="2022-09-02T11:13:03.143" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @an-ignorant-wanderer, converted from &lt;a href=&quot;https://bioinformatics.stackexchange.com/help/privileges/comment&quot;&gt;comment&lt;/a&gt;:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I ended up using the annotation file from &lt;a href=&quot;https://support.illumina.com/sequencing/sequencing_software/igenome.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.illumina.com/sequencing/sequencing_software/igenome.html&lt;/a&gt; and it worked.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;[Please update/improve this answer if possible]&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-09-02T11:13:03.143" CommentCount="0" CommunityOwnedDate="2022-09-02T11:13:03.143" ContentLicense="CC BY-SA 4.0" />
  <row Id="19644" PostTypeId="1" CreationDate="2022-09-02T12:30:22.757" Score="1" ViewCount="33" Body="&lt;p&gt;&lt;a href=&quot;http://cole-trapnell-lab.github.io/monocle-release/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Monocle&lt;/a&gt; is a comprehensive bioinformatics package which has the function to run the pseudo-time trajectory and changes in direction.&lt;/p&gt;&#xA;&lt;p&gt;I wish to replicate this analysis. Like in the figure I would like the right side to be the pseudostar.&lt;/p&gt;&#xA;&lt;p&gt;Thank you so much for your guys help&lt;img src=&quot;https://i.stack.imgur.com/d4D0a.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;" OwnerUserId="16087" LastEditorUserId="16087" LastEditDate="2022-10-30T01:59:39.107" LastActivityDate="2022-10-30T01:59:39.107" Title="how to change the pseudo time trajectory direction?" Tags="|seurat|single-cell|monocle|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19645" PostTypeId="1" CreationDate="2022-09-02T14:25:24.277" Score="2" ViewCount="42" Body="&lt;p&gt;How does one convert &amp;quot;flowJO&amp;quot; flow-cytometry cell-count data into &amp;quot;.json&amp;quot; or relational-database-management-system compatible files?&lt;/p&gt;&#xA;&lt;p&gt;The way flow-jo exports is in wide rather than long format. It also exports everything per subject/animal/lab-rat per specific timepoints rather than aggregating. I need the aggregated files to plot groups.&lt;/p&gt;&#xA;&lt;p&gt;It's also not shaped in a &amp;quot;primary-key&amp;quot;, column 1, column 2, column 3, or .json (NO_SQL) format.&#xA;Running statistical programs or databases is intractable with the export options Flowjo gives.&lt;/p&gt;&#xA;&lt;p&gt;I am looking for some way to convert it, or maybe the problem lies in flowjo. It does not support the export formats I need.&#xA;I amk open to other flow-cytometry software that offers better integration with no-sql/sql database.&lt;/p&gt;&#xA;" OwnerUserId="16088" LastEditorUserId="16088" LastEditDate="2022-09-02T14:34:52.753" LastActivityDate="2022-09-02T14:34:52.753" Title="How does one convert &quot;flowJO&quot; flow-cytometry cell-count, etc, bead, etc data into .json or relational-database-management-system compatible files?" Tags="|database|software-recommendation|format-conversion|flow-cytometry|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19646" PostTypeId="2" ParentId="13643" CreationDate="2022-09-02T16:02:45.543" Score="1" Body="&lt;p&gt;To clarify, you are seeing reds and blues on the surface? If so, these are the sidechain oxygens and nitrogens. So you need to colour the whole residue not only the &lt;code&gt;element C&lt;/code&gt; atoms. &lt;code&gt;color 0xFF7F50, resi 123&lt;/code&gt; will colour the residue in the cartoon and surface as I omitted and &lt;code&gt;element C&lt;/code&gt;. To revert &lt;code&gt;color atomic&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In PyMOL an atom has a colour, which is then used by the representations. To have a differentially coloured representation, you need to make a copy of the object. &lt;code&gt;create copied, obj_name&lt;/code&gt; and &lt;code&gt;hide cartoon&lt;/code&gt; and &lt;code&gt;show surface&lt;/code&gt; will do it.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2022-09-02T16:02:45.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19647" PostTypeId="2" ParentId="19630" CreationDate="2022-09-03T09:31:36.507" Score="3" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;for illumina reads (only)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://docs.google.com/document/d/1ZJpGTMl319v-YRrXvNbM5OGSy0jBTVfUbpawVAPqaMU/edit?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;prerequisite: download n prepare&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://docs.google.com/document/d/1p9ZsfkqykNgcX7F5fpOl64vDi46tMEL7yY3VfiK8mZE/edit?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;the fna n gff file of yeast reference&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://drive.google.com/drive/folders/1rRfMXvK3KtrppFQm-TChRDIOYR0UphKI?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;mapping_in_02.directory&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;1st step is mapping the downloaded reads to the reference genome:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;build index for the reference using the following command&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;bwa index ../01./GCF_000146045.2_R64_genomic.fna&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;index the reference genome for rapid searching and aligning&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;do the mapping using bwa mem (better with bash shell script, then there will be no need for each individual one by one)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;After mapping, you could use GATK4 for the variation calling based on the mapping result.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The steps of&#xA;variation calling by GATK4 include dealing with the mapping result ( mark duplicates and add read group ), raw variation&#xA;identification ( joint call followed by single individual variation calling ), quality recalibration and realign, second round of variation&#xA;calling and final variation filtering ( filtering is&#xA;applied to haplotyping to remove uninformative mutations. An obvious filter is to remove SNPs with identical calls in all the samples ).&lt;/p&gt;&#xA;&lt;p&gt;The final variation calling result with SNPs and InDels could be obtained and the final vcf file could be sent to SnpEff for further annotation ( an easier way is using SnpEff in Galaxy with the right reference genome version,  eg  R64-1-1.75 ).&lt;/p&gt;&#xA;&lt;p&gt;The isolates from NCBI would indicate that there are 16 chromosomes and one mitochondrial chromosome after SnpEff analysis.  In the analysis, the numbers of insertions and deletions will be obtained.&lt;/p&gt;&#xA;" OwnerUserId="15519" LastEditorUserId="15519" LastEditDate="2022-09-04T06:22:26.047" LastActivityDate="2022-09-04T06:22:26.047" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19648" PostTypeId="1" CreationDate="2022-09-03T10:23:48.433" Score="1" ViewCount="33" Body="&lt;p&gt;Do proper .fcs flow-cytometry files already have the gating hierarchy, so one may calculate cell-counts without extra work?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/whitews/FlowKit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/whitews/FlowKit/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# setup some file paths for our data&#xA;base_dir = &amp;quot;data/8_color_data_set&amp;quot;&#xA;&#xA;sample_path = os.path.join(base_dir, &amp;quot;fcs_files&amp;quot;)&#xA;wsp_path = os.path.join(base_dir, &amp;quot;8_color_ICS.wsp&amp;quot;)&#xA;session = fk.Session(sample_path)&#xA;session.import_flowjo_workspace(wsp_path)&#xA;# that's it, let's see what sample groups were loaded&#xA;session.get_sample_groups()&#xA;['default', 'All Samples', 'DEN']&#xA;# I happen to know all the &amp;quot;real&amp;quot; analysis is within the &amp;quot;DEN&amp;quot; group&#xA;sample_group = 'DEN'&#xA;# and the sample that are included in the group&#xA;session.get_group_sample_ids(sample_group)&#xA;['101_DEN084Y5_15_E03_009_clean.fcs',&#xA; '101_DEN084Y5_15_E01_008_clean.fcs',&#xA; '101_DEN084Y5_15_E05_010_clean.fcs']&#xA;# so what is the gating hierarchy for this group&#xA;print(session.get_gate_hierarchy(sample_group))&#xA;root&#xA;╰── Time&#xA;    ╰── Singlets&#xA;        ╰── aAmine-&#xA;            ╰── CD3+&#xA;                ├── CD4+&#xA;                │   ├── CD107a+&#xA;                │   ├── IFNg+&#xA;                │   ├── IL2+&#xA;                │   ╰── TNFa+&#xA;                ╰── CD8+&#xA;                    ├── CD107a+&#xA;                    ├── IFNg+&#xA;                    ├── IL2+&#xA;                    ╰── TNFa+&#xA;# looks good, let's go ahead and analyze them (using verbose mode to see each gate as it's processed)&#xA;session.analyze_samples(sample_group, verbose=True)&#xA;#### Processing gates for 3 samples (multiprocessing is enabled - 3 cpus) ####&#xA;101_DEN084Y5_15_E03_009_clean.fcs: processing gate Time&#xA;101_DEN084Y5_15_E03_009_clean.fcs: processing gate Singlets&#xA;101_DEN084Y5_15_E03_009_clean.fcs: processing gate aAmine-&#xA;101_DEN084Y5_15_E01_008_clean.fcs: processing gate Time&#xA;101_DEN084Y5_15_E01_008_clean.fcs: processing gate Singlets&#xA;101_DEN084Y5_15_E01_008_clean.fcs: processing gate aAmine-&#xA;101_DEN084Y5_15_E05_010_clean.fcs: processing gate Time&#xA;101_DEN084Y5_15_E03_009_clean.fcs: processing gate CD3+&#xA;101_DEN084Y5_15_E05_010_clean.fcs: processing gate Singlets&#xA;101_DEN084Y5_15_E05_010_clean.fcs: processing gate aAmine-&#xA;101_DEN084Y5_15_E03_009_clean.fcs: processing gate CD4+&#xA;101_DEN084Y5_15_E01_008_clean.fcs: processing gate CD3+&#xA;101_DEN084Y5_15_E01_008_clean.fcs: processing gate CD4+&#xA;101_DEN084Y5_15_E05_010_clean.fcs: processing gate CD3+&#xA;101_DEN084Y5_15_E03_009_clean.fcs: processing gate CD8+&#xA;101_DEN084Y5_15_E05_010_clean.fcs: processing gate CD4+&#xA;101_DEN084Y5_15_E01_008_clean.fcs: processing gate CD8+&#xA;101_DEN084Y5_15_E03_009_clean.fcs: processing gate CD107a+&#xA;101_DEN084Y5_15_E05_010_clean.fcs: processing gate CD8+&#xA;101_DEN084Y5_15_E01_008_clean.fcs: processing gate CD107a+&#xA;101_DEN084Y5_15_E03_009_clean.fcs: processing gate IFNg+&#xA;101_DEN084Y5_15_E05_010_clean.fcs: processing gate CD107a+&#xA;101_DEN084Y5_15_E01_008_clean.fcs: processing gate IFNg+&#xA;101_DEN084Y5_15_E03_009_clean.fcs: processing gate IL2+&#xA;101_DEN084Y5_15_E05_010_clean.fcs: processing gate IFNg+&#xA;101_DEN084Y5_15_E01_008_clean.fcs: processing gate IL2+&#xA;101_DEN084Y5_15_E03_009_clean.fcs: processing gate TNFa+&#xA;101_DEN084Y5_15_E05_010_clean.fcs: processing gate IL2+&#xA;101_DEN084Y5_15_E01_008_clean.fcs: processing gate TNFa+&#xA;101_DEN084Y5_15_E03_009_clean.fcs: processing gate CD107a+&#xA;101_DEN084Y5_15_E05_010_clean.fcs: processing gate TNFa+&#xA;101_DEN084Y5_15_E01_008_clean.fcs: processing gate CD107a+&#xA;101_DEN084Y5_15_E03_009_clean.fcs: processing gate IFNg+&#xA;101_DEN084Y5_15_E05_010_clean.fcs: processing gate CD107a+&#xA;101_DEN084Y5_15_E01_008_clean.fcs: processing gate IFNg+&#xA;101_DEN084Y5_15_E03_009_clean.fcs: processing gate IL2+&#xA;101_DEN084Y5_15_E05_010_clean.fcs: processing gate IFNg+&#xA;101_DEN084Y5_15_E01_008_clean.fcs: processing gate IL2+&#xA;101_DEN084Y5_15_E03_009_clean.fcs: processing gate TNFa+&#xA;101_DEN084Y5_15_E05_010_clean.fcs: processing gate IL2+&#xA;101_DEN084Y5_15_E01_008_clean.fcs: processing gate TNFa+&#xA;101_DEN084Y5_15_E05_010_clean.fcs: processing gate TNFa+&#xA;# and a look a the results&#xA;session.get_group_report(sample_group)&#xA;sample  gate_path   gate_name   gate_type   quadrant_parent parent  count   absolute_percent    relative_percent    level&#xA;0   101_DEN084Y5_15_E03_009_clean.fcs   (root,) Time    RectangleGate   None    None    283968  99.999648   99.999648   1&#xA;1   101_DEN084Y5_15_E03_009_clean.fcs   (root, Time)    Singlets    PolygonGate None    Time    236780  83.382341   83.382635   2&#xA;2   101_DEN084Y5_15_E03_009_clean.fcs   (root, Time, Singlets)  aAmine- PolygonGate None    Singlets    161823  56.986150   68.343188   3&#xA;3   101_DEN084Y5_15_E03_009_clean.fcs   (root, Time, Singlets, aAmine-) CD3+    PolygonGate None    aAmine- 132200  46.554377   81.694197   4&#xA;4   101_DEN084Y5_15_E03_009_clean.fcs   (root, Time, Singlets, aAmine-, CD3+)   CD4+    PolygonGate None    CD3+    81855   28.825330   61.917549   5&#xA;5   101_DEN084Y5_15_E03_009_clean.fcs   (root, Time, Singlets, aAmine-, CD3+)   CD8+    PolygonGate None    CD3+    46965   16.538777   35.525719   5&#xA;6   101_DEN084Y5_15_E03_009_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD4+) CD107a+ RectangleGate   None    CD4+    59  0.020777    0.072079    6&#xA;10  101_DEN084Y5_15_E03_009_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD8+) CD107a+ RectangleGate   None    CD8+    814 0.286651    1.733206    6&#xA;7   101_DEN084Y5_15_E03_009_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD4+) IFNg+   RectangleGate   None    CD4+    598 0.210586    0.730560    6&#xA;11  101_DEN084Y5_15_E03_009_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD8+) IFNg+   RectangleGate   None    CD8+    1224    0.431033    2.606196    6&#xA;8   101_DEN084Y5_15_E03_009_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD4+) IL2+    RectangleGate   None    CD4+    64  0.022538    0.078187    6&#xA;12  101_DEN084Y5_15_E03_009_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD8+) IL2+    RectangleGate   None    CD8+    127 0.044723    0.270414    6&#xA;9   101_DEN084Y5_15_E03_009_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD4+) TNFa+   RectangleGate   None    CD4+    288 0.101420    0.351842    6&#xA;13  101_DEN084Y5_15_E03_009_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD8+) TNFa+   RectangleGate   None    CD8+    821 0.289116    1.748110    6&#xA;0   101_DEN084Y5_15_E01_008_clean.fcs   (root,) Time    RectangleGate   None    None    290166  99.997932   99.997932   1&#xA;1   101_DEN084Y5_15_E01_008_clean.fcs   (root, Time)    Singlets    PolygonGate None    Time    239001  82.365287   82.366990   2&#xA;2   101_DEN084Y5_15_E01_008_clean.fcs   (root, Time, Singlets)  aAmine- PolygonGate None    Singlets    164655  56.743931   68.893017   3&#xA;3   101_DEN084Y5_15_E01_008_clean.fcs   (root, Time, Singlets, aAmine-) CD3+    PolygonGate None    aAmine- 133670  46.065782   81.181865   4&#xA;4   101_DEN084Y5_15_E01_008_clean.fcs   (root, Time, Singlets, aAmine-, CD3+)   CD4+    PolygonGate None    CD3+    82484   28.425899   61.707189   5&#xA;5   101_DEN084Y5_15_E01_008_clean.fcs   (root, Time, Singlets, aAmine-, CD3+)   CD8+    PolygonGate None    CD3+    47165   16.254153   35.284656   5&#xA;6   101_DEN084Y5_15_E01_008_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD4+) CD107a+ RectangleGate   None    CD4+    68  0.023434    0.082440    6&#xA;10  101_DEN084Y5_15_E01_008_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD8+) CD107a+ RectangleGate   None    CD8+    73  0.025157    0.154776    6&#xA;7   101_DEN084Y5_15_E01_008_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD4+) IFNg+   RectangleGate   None    CD4+    4   0.001378    0.004849    6&#xA;11  101_DEN084Y5_15_E01_008_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD8+) IFNg+   RectangleGate   None    CD8+    2   0.000689    0.004240    6&#xA;8   101_DEN084Y5_15_E01_008_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD4+) IL2+    RectangleGate   None    CD4+    6   0.002068    0.007274    6&#xA;12  101_DEN084Y5_15_E01_008_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD8+) IL2+    RectangleGate   None    CD8+    10  0.003446    0.021202    6&#xA;9   101_DEN084Y5_15_E01_008_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD4+) TNFa+   RectangleGate   None    CD4+    21  0.007237    0.025459    6&#xA;13  101_DEN084Y5_15_E01_008_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD8+) TNFa+   RectangleGate   None    CD8+    8   0.002757    0.016962    6&#xA;0   101_DEN084Y5_15_E05_010_clean.fcs   (root,) Time    RectangleGate   None    None    284846  99.844369   99.844369   1&#xA;1   101_DEN084Y5_15_E05_010_clean.fcs   (root, Time)    Singlets    PolygonGate None    Time    235331  82.488345   82.616923   2&#xA;2   101_DEN084Y5_15_E05_010_clean.fcs   (root, Time, Singlets)  aAmine- PolygonGate None    Singlets    160955  56.418031   68.395154   3&#xA;3   101_DEN084Y5_15_E05_010_clean.fcs   (root, Time, Singlets, aAmine-) CD3+    PolygonGate None    aAmine- 131305  46.025097   81.578702   4&#xA;4   101_DEN084Y5_15_E05_010_clean.fcs   (root, Time, Singlets, aAmine-, CD3+)   CD4+    PolygonGate None    CD3+    81135   28.439483   61.791249   5&#xA;5   101_DEN084Y5_15_E05_010_clean.fcs   (root, Time, Singlets, aAmine-, CD3+)   CD8+    PolygonGate None    CD3+    46546   16.315328   35.448764   5&#xA;6   101_DEN084Y5_15_E05_010_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD4+) CD107a+ RectangleGate   None    CD4+    70  0.024536    0.086276    6&#xA;10  101_DEN084Y5_15_E05_010_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD8+) CD107a+ RectangleGate   None    CD8+    306 0.107259    0.657414    6&#xA;7   101_DEN084Y5_15_E05_010_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD4+) IFNg+   RectangleGate   None    CD4+    9   0.003155    0.011093    6&#xA;11  101_DEN084Y5_15_E05_010_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD8+) IFNg+   RectangleGate   None    CD8+    348 0.121981    0.747647    6&#xA;8   101_DEN084Y5_15_E05_010_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD4+) IL2+    RectangleGate   None    CD4+    8   0.002804    0.009860    6&#xA;12  101_DEN084Y5_15_E05_010_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD8+) IL2+    RectangleGate   None    CD8+    65  0.022784    0.139647    6&#xA;9   101_DEN084Y5_15_E05_010_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD4+) TNFa+   RectangleGate   None    CD4+    28  0.009815    0.034510    6&#xA;13  101_DEN084Y5_15_E05_010_clean.fcs   (root, Time, Singlets, aAmine-, CD3+, CD8+) TNFa+   RectangleGate   None    CD8+    336 0.117775    0.721867    6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16088" LastEditorUserId="16088" LastEditDate="2022-09-03T10:57:51.103" LastActivityDate="2022-09-03T10:57:51.103" Title="Do proper .fcs flow-cytometry files specify the gating hierarchy, so one may calculate cell-counts without specifying it themselves?" Tags="|file-formats|data-retrieval|flow-cytometry|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19649" PostTypeId="1" CreationDate="2022-09-03T19:36:22.687" Score="4" ViewCount="110" Body="&lt;p&gt;I'm interested in doing de-novo sequencing but also phylogenetic analysis. In particular, after de-novo sequencing and annotating the genome, I need to align the CO1 gene and the nuclear 28S rRNA gene of several species. When extracting DNA and sequencing with PacBio for example, how does the assembler know what corresponds to nuclear and what to mitochondrial DNA?&lt;/p&gt;&#xA;" OwnerUserId="14030" LastActivityDate="2023-06-02T07:07:30.990" Title="How does one distinguish nuclear DNA from mitochondrial DNA when doing WGS?" Tags="|assembly|phylogenetics|mtdna|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19650" PostTypeId="1" AcceptedAnswerId="19653" CreationDate="2022-09-03T22:50:26.933" Score="3" ViewCount="60" Body="&lt;p&gt;This strategy seems to be most commonly used for plants. When crossing animals isn't possible, how can I do a similar study to identify a particular locus responsible for a polymorphic trait (specifically, the pigmentation of a beetle)?&lt;/p&gt;&#xA;" OwnerUserId="14030" LastEditorUserId="298" LastEditDate="2022-09-04T10:09:52.223" LastActivityDate="2022-09-05T21:08:27.237" Title="Is there an alternative to bulked segregant analysis for insects?" Tags="|wgs|genotyping|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19652" PostTypeId="2" ParentId="19634" CreationDate="2022-09-04T10:18:53.820" Score="1" Body="&lt;p&gt;Here's another simple perl solution:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ perl -lpe 's/\|.*(\d+)/|$1|training/ if s/&amp;gt;.+?\|/&amp;gt;/' file.fa &#xA;&amp;gt;A0A075B6G3|1|training&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that you didn't tell us where the &lt;code&gt;1&lt;/code&gt; came from, so I am taking the last numbers on the line, the &lt;code&gt;SV=1&lt;/code&gt; in your case. If you want to hard-code the &lt;code&gt;1&lt;/code&gt;, you can do so easily with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -lpe 's/\|.*/|1|training/ if s/&amp;gt;.+?\|/&amp;gt;/' file.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;-lpe&lt;/code&gt; options mean &amp;quot;&lt;strong&gt;p&lt;/strong&gt;rint each line, adding a new&lt;strong&gt;l&lt;/strong&gt;ine character to each &lt;code&gt;print&lt;/code&gt; call, after applying the script given by &lt;code&gt;-e&lt;/code&gt;&amp;quot;. So this will read the file line by line, do its thing and print the result.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;s/old/new/&lt;/code&gt; is the substitution operator and will replace &lt;code&gt;old&lt;/code&gt; with &lt;code&gt;new&lt;/code&gt;. In this case, we are replacing a &lt;code&gt;|&lt;/code&gt; character (&lt;code&gt;\|&lt;/code&gt;) and everything after it (&lt;code&gt;.*&lt;/code&gt;) until the last set of numbers on the line which we capture and save as &lt;code&gt;$1&lt;/code&gt; with &lt;code&gt;(\d+)&lt;/code&gt;. This substitution will only happen if we have already successfully substituted the leading &lt;code&gt;&amp;gt;&lt;/code&gt; and everything until the first &lt;code&gt;|&lt;/code&gt; (&lt;code&gt;&amp;gt;.+?\|&lt;/code&gt;) with just a &lt;code&gt;&amp;gt;&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you want to edit the original file in place, use the &lt;code&gt;-i&lt;/code&gt; flag (but give it a value, &lt;code&gt;.bak&lt;/code&gt; in the example below, since that ensures a backup file with the original data and that extension will be created):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -i .bak -lpe 's/\|.*(\d+)/|$1|training/ if s/&amp;gt;.+?\|/&amp;gt;/' file.fa &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2022-09-04T10:27:32.450" LastActivityDate="2022-09-04T10:27:32.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19653" PostTypeId="2" ParentId="19650" CreationDate="2022-09-04T14:35:52.337" Score="2" Body="&lt;p&gt;I'll have a go at answering this. I don't understand beetles within Insecta: I am well aware they very distinct, obviously in phenotype and ecology, but also genetically. I think they have unique compositional bias even amongst holometabolous insects.&lt;/p&gt;&#xA;&lt;p&gt;Personally I think if you've a recent genetic trait, e.g. insecticide resistance, that evolved within the last 30 years or less, you're dealing with pretty homogeneous backgrounds between traits. In this example trait, bulk segregant analysis might work.&lt;/p&gt;&#xA;&lt;p&gt;Beetles are really ancient. I suspect their families are particularly ancient, because they are just so successful and I'd guess Scarabaeidae are ancient too, i.e. the family scarab beetles. This family could easily be 50 million years old, or older. Each species therein could a few million years old. For insects thats normal and these are beetles.&lt;/p&gt;&#xA;&lt;p&gt;The concern here is over million+ years of evolution (I don't know the study size), they will comprise loads of neutral signal that would act as noise, i.e. there will be excessive third codon wobble which will mask SNPs/genes causing colour (color) variation.&lt;/p&gt;&#xA;&lt;p&gt;My approach, given I know nothing about colour (color) variation in insects other than &amp;quot;white-eyed&amp;quot; phenotype in Drosophila melanogaster,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;assess genetic complexity of colour variation in other insect Orders;&lt;/li&gt;&#xA;&lt;li&gt;get hold of as many proteins involved in colour in insects as possible;&lt;/li&gt;&#xA;&lt;li&gt;particularly get hold of colour variation proteins in beetles;&lt;/li&gt;&#xA;&lt;li&gt;sequence the genome of a representative scarab beetle (which I think your doing)&lt;/li&gt;&#xA;&lt;li&gt;perform &lt;code&gt;psi-blast&lt;/code&gt; using the hypotheses from 1., 2. and particularly 3. to assess which proteins in the scarab beetle might be responsible.&lt;/li&gt;&#xA;&lt;li&gt;perform deep sequencing on those targets, because they could be multigene families &lt;strong&gt;*&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The next step is phylogenetics to assess convergent and/or parallel evolution &lt;strong&gt;**&lt;/strong&gt;. That in my view is a separate question because its quite complicated and you'd need to write your own pipeline to automate it (you could do it by hand however).&lt;/p&gt;&#xA;&lt;p&gt;What I would be cautious about is that beetles are not any other insect so inter-Order homology needs to be treated with caution.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;*&lt;strong&gt;,&lt;/strong&gt; Just to point out, there are lots sequencing strategies its not my thing, e.g. low coverage draft genomes. PCR deep sequencing for a given gene target, without any draft genomes and isn't conserved e.g. rRNA types genes, will need small PCR fragments because of the neutral mutations which will disrupt primer specificity. If you're good at sequencing a draft genome could be cool (just guessing).&lt;/p&gt;&#xA;&lt;p&gt;**&lt;strong&gt;,&lt;/strong&gt; A phylogenetics approach would dovetail with the COI and rRNA gene sequencing (28S?) already underway, because these genes should approximate to the species phylogeny. What is sought are residues  within the colour genes (or perhaps presence/absence genes) that will cluster not according to their evolutionary descent but according to their colours (colors).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-09-05T21:08:27.237" LastActivityDate="2022-09-05T21:08:27.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19654" PostTypeId="2" ParentId="19649" CreationDate="2022-09-05T03:03:17.897" Score="1" Body="&lt;p&gt;According to this pdf,  &lt;a href=&quot;https://www.pacb.com/wp-content/uploads/Chakraborty-AGBT-2018-Mitochondrial-DNA-Sequencing-Using-the-PacBio-Sequel-System-.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;from pacb.com&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;isolation with mitoISO &lt;a href=&quot;https://www.sciencellonline.com/PS/8268.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;MITOISO&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The kit will include three steps: 1) mechanical&#xA;rupturing of tissues or cells 2) removing cellular debris and nuclei by low speed centrifugation and 3) harvesting&#xA;mitochondria by high speed centrifugation.&lt;/p&gt;&#xA;&lt;p&gt;Then, you can follow the DNA extraction with mitochondrial DNA, and then sequenced on the PacBio Sequel.&lt;/p&gt;&#xA;&lt;p&gt;If you are doing whole genome sequencing, the mitogenome will have a higher read depth according to &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-017-1927-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article stating that lower read depth of the nuclear genome separating the two different origins:  &lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;those of mitochondrial origin &amp;amp; those of nuclear origin&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="15519" LastEditorUserId="15519" LastEditDate="2022-09-05T03:28:59.487" LastActivityDate="2022-09-05T03:28:59.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19655" PostTypeId="2" ParentId="19383" CreationDate="2022-09-05T05:18:51.447" Score="-1" Body="&lt;p&gt;Do you mean the following type of technology?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/U6VGq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/U6VGq.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mTCb3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mTCb3.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15905" LastActivityDate="2022-09-05T05:18:51.447" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19656" PostTypeId="1" CreationDate="2022-09-06T20:17:54.493" Score="1" ViewCount="123" Body="&lt;p&gt;I recently prefetched 157 SRA files from an NCBI BioProject using the SRAtoolkit. I then used the toolkit to download those files in FASTA format.&lt;/p&gt;&#xA;&lt;p&gt;Each individual FASTA file looks something like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;(example for SRR5678544 file)&lt;/em&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;SRR5678544.1 HWI-ST1146_88:5:1112:6472:81473 length=194&#xA;CCATGCGGGGTATCGTATGCTTCCTTCTGCACTACCCTTTAGCTGTTCTATATGCTGCCACTCCTCAATTGGATTAGTCTCATCCTTCAATGCTATCACAAGAAGTAGAGACACAAATGCAAGAGGAGCATATAAATTACAAAACACCATCACTGAGGGCCCTAAAGCGGTTCCCACGAAAAAAAGGAGAGTAG   &#xA;&amp;gt;SRR5678544.2 HWI-ST1146_88:5:1113:13218:62635 length=194&#xA; TGGATGGTGGTTGGGAGTGGTAAGGTTGAATGAGACACGGTAACGAGTGGGGAGGTAGGGTAATGGAGGGTAAGTTGAGAGACAGGTTCGTCAGGGGACACACCACACCACACCCACACCCACACACCCACACACCACACCCACACACCACACCCACACCCACACCCCCACACACCACACCCCCACCCCACCCA  &#xA;&amp;gt;SRR5678544.3 HWI-ST1146_88:5:2206:11224:22269 length=194&#xA; GTCTCTTAACTTACCCTCCATTACCCTACCTCCCGACTCGTTACCCTGTCTCATTCAACCATACCACTCCCGACCACCATCCATCCCTCTACTTACTACAGTATGGTGAGTGGGACATGGTGGATGGTAGGGTAAGTGGCAGTGGAGTTGGATATGGGCAATTGGAGGGTAACGGTTATGGTGGACGGGGGGTG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And so on and so forth. (Ignore the &amp;quot; and ' marks I had to input those to format it right on here)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 1:&lt;/strong&gt; Can someone explain why this file has so many sequences (&amp;gt;)? Every FASTA file I have worked with has one line with &amp;quot;&amp;gt;&amp;quot; at the top and a header sentence followed by a single big paragraph of the genome sequence.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 2:&lt;/strong&gt; If I wanted to combine all the sequences in a single FASTA file to look like a &amp;quot;normal&amp;quot; FASTA file - just a single paragraph - how would I do that?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Reasoning:&lt;/strong&gt; I'm asking simply because I need to make a BLAST database out of the 157 sequences and the &amp;quot;makeblastdb&amp;quot; command only takes in a single file as the input. I was going to try and combine each individual FASTA file into single paragraphs and then make a huge FASTA file that includes all 157 sequences separated by &amp;quot;&amp;gt;&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="3848" LastEditDate="2023-08-19T21:05:50.223" LastActivityDate="2023-08-19T21:05:50.223" Title="Fixing FASTA file for Local BLAST Database" Tags="|phylogenetics|fasta|phylogeny|makeblastdb|sra|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="19657" PostTypeId="1" CreationDate="2022-09-06T21:23:06.083" Score="1" ViewCount="92" Body="&lt;p&gt;I want to create a model to predict proteins which could be associated with drug response in cancer cell lines. I have cell line proteomics data, with compound screening data they have gained for a number of these cells. My aim is to relate these proteins to drug response. I am using this exercise as an introduction in the use of ML in biology.&lt;/p&gt;&#xA;&lt;p&gt;To this end, I have created dictionaries, with selected compounds as the keys, and their respective datasets. Cells are rows, proteins are columns, and the drug response is the &lt;code&gt;y&lt;/code&gt; variable.&lt;/p&gt;&#xA;&lt;p&gt;I have approached the task as a regression task. The aim being to predict the ln(IC50) - a measure of the concentration required to induce cell death - and subsequently pull out the features (proteins) which have made the largest contribution to this prediction. As my baseline model, I have attempted linear regression. However, my model performance has consistently been very poor. A problem with this dataset is the large number of missing values (heatmap below - yellow = missing). However, even when trying a simple feature selection strategy with sklearn (first function), the explained variance is around 0.3 for the best performing compound. I have plotted the residuals in attempt to dig into this further, but the spread looks quite distributed.&lt;/p&gt;&#xA;&lt;p&gt;Any advise/suggestions would be appreciated. Or any further reading/papers relating to ML in biology which could help a beginner build and use effective models.&lt;/p&gt;&#xA;&lt;p&gt;Training and test functions shown below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    def process_training_and_test_data_for_LG(df):&#xA;        training_datasets = {}&#xA;        test_datasets = {}&#xA;        for drug in df.keys():&#xA;            df[drug].dropna(axis=1,inplace=True)&#xA;            print(&amp;quot;at &amp;quot;+drug)&#xA;            X = df[drug].drop([&amp;quot;cell_line_name&amp;quot;, &amp;quot;ln_IC50&amp;quot;,&amp;quot;putative_target&amp;quot;], axis=1)&#xA;            y = df[drug][&amp;quot;ln_IC50&amp;quot;]&#xA;            X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)&#xA;&#xA;            feature_selector = SelectKBest(score_func=f_regression, k=50)&#xA;            feature_selector.fit(X_train, y_train)&#xA;        &#xA;            X_train = feature_selector.transform(X_train)&#xA;            X_test = feature_selector.transform(X_test)&#xA;        &#xA;            #Add data to relevant dictionaries&#xA;            training_datasets[drug] = [X_train, y_train]&#xA;            test_datasets[drug] = [X_test, y_test]&#xA;        return training_datasets, test_datasets&#xA;&#xA;def train_models(training_data):&#xA;    trained_models = {}&#xA;    for drug in training_data.keys():&#xA;        #initiate model&#xA;        lm = LinearRegression()&#xA;        #extract drug specific training data&#xA;        X_train = training_data[drug][0]&#xA;        y_train = training_data[drug][1]&#xA;        #fit model&#xA;        model = lm.fit(X_train, y_train)&#xA;        #Append dict of trained models&#xA;        trained_models[drug] = model&#xA;        print(&amp;quot;Model for {} trained&amp;quot;.format(drug))&#xA;    return trained_models&#xA;&#xA;def test_models(model_dict, test_data):&#xA;    prediction_dict = {}&#xA;    for drug in model_dict.keys():&#xA;        drug_model = model_dict[drug]&#xA;        X_test = test_data[drug][0]&#xA;        predictions = drug_model.predict(X_test)&#xA;        prediction_dict[drug] = predictions&#xA;    return prediction_dict&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ON9TG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ON9TG.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mNmy6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mNmy6.png&quot; alt=&quot;Residual plot per compound&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15861" LastEditorUserId="17705" LastEditDate="2024-02-22T16:01:01.760" LastActivityDate="2024-03-17T13:39:17.793" Title="Advise on building an effect ML model for predicting important proteins for drug response" Tags="|python|machine-learning|proteomics|modelling|linear-regression|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19658" PostTypeId="2" ParentId="19657" CreationDate="2022-09-06T22:16:56.163" Score="0" Body="&lt;p&gt;The gist looks fine ..&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;pandas&lt;/code&gt; - looks fine&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;SelectKBest&lt;/code&gt; - cool&lt;/li&gt;&#xA;&lt;li&gt;transformation (obviously there's a lot of them)&lt;/li&gt;&#xA;&lt;li&gt;standard ML&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;A common ML situation is, the classifier working but the accuracy is notably lower than expected. My personal &lt;em&gt;first thing to check&lt;/em&gt; is the use of &lt;em&gt;linear regression&lt;/em&gt; as a baseline model. The scatterplots give a gist, we don't know in particular what data points belong to what classification - thats crucial. What's helpful are coloured dots to assess how they sit against the regression line.&lt;/p&gt;&#xA;&lt;p&gt;What I think has happened is there's been a PCA via &lt;code&gt;SelectKBest&lt;/code&gt;, unsupervised learning, which doesn't account for much of variance (&lt;em&gt;cf&lt;/em&gt; 0.3), i.e. this is not an accuracy score.&lt;/p&gt;&#xA;&lt;p&gt;I've three suggestions:&lt;/p&gt;&#xA;&lt;p&gt;1/ An acceptable baseline model often used is &lt;code&gt;k-nearest neighbour&lt;/code&gt; ..&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from sklearn.model_selection import train_test_split&#xA;from sklearn.neighbors import KNeighborsClassifier&#xA;clf = KNeighborsClassifier(n_neighbors=3)&#xA;clf.fit(X_train, y_train)&#xA;print(&amp;quot;Test set accuracy: {:.2f}&amp;quot;.format(clf.score(X_test, y_test)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I think this will produce much better results with an accuracy of &amp;gt;0.7&lt;/p&gt;&#xA;&lt;p&gt;2/ However,  I would check the following (please see &lt;strong&gt;caveat&lt;/strong&gt;),&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for drug in df.keys():&#xA;    df ....&#xA;    X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Just make sure that X_train, etc ... is supposed within the loop, rather than immediately outside it (four spaces to the left). Looking at your outputs they look fine.&lt;/p&gt;&#xA;&lt;p&gt;The thing to check is its not:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for drug in df.keys():&#xA;    df....&#xA;X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33, random_state=42)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above loop makes sense to me.&lt;/p&gt;&#xA;&lt;p&gt;The original loop position &lt;code&gt;train_test_split&lt;/code&gt; looks like the &lt;code&gt;key&lt;/code&gt; loop might repeatedly over-ride the &lt;code&gt;X_train, X_test, y_train, y_test&lt;/code&gt; outputs.&lt;/p&gt;&#xA;&lt;p&gt;3/ Very final point &lt;code&gt;SelectKBest&lt;/code&gt; followed by &lt;code&gt;fit&lt;/code&gt;. What about &lt;code&gt;fit_transform&lt;/code&gt; instead? If the issue is the features are resulting in low description of the variation, maybe thats key here?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Caveat&lt;/strong&gt; I do stress that its hard to comment beyond general terms because a good biological understanding is needed of the goal of the ML training. In context its not my area and I haven't read any of the  papers.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-09-07T02:28:35.597" LastActivityDate="2022-09-07T02:28:35.597" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19659" PostTypeId="1" CreationDate="2022-09-07T09:59:49.733" Score="0" ViewCount="95" Body="&lt;p&gt;I am dealing with several genes in my dataset. For each of the genes, I have built gene tree, with their best estimated model. I am intended to look into the effect of the tree from concatenated sequences (supermatrix tree), using Maximum Likelihood approach.&lt;/p&gt;&#xA;&lt;p&gt;I am considering two options to apply the model for the supermatrix tree inference.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Obtain a single best estimated model for the supermatrix gene&lt;/li&gt;&#xA;&lt;li&gt;Apply partitioned model for the supermatrix gene, based on each gene's model&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;From what I have read in most if not all of the research articles, they seemed to be using option 1, where a single model applied for the supermatrix tree inference. Since the purpose of using evolutionary model is to describe the changes in the gene and given that the evolution pattern in each gene differ, why wouldn't the respective model of each gene be considered in the tree inference?&lt;/p&gt;&#xA;&lt;p&gt;For examples,&lt;/p&gt;&#xA;&lt;p&gt;Dataset A: gene1 and gene2 are having LG and WAG model respectively as the best estimated model. But when the two genes are being concatenated, it turned out that LG is the best model for the matrix. It seemed that most if not all of the experiment opts for option 1 in this case. Theoretically, will the supermatrix tree using option 1 and option 2 in the model application still similar? Also, will the evolutionary pattern in gene2 still being represented correctly on the tree when we opt for option 1 in the supermatrix? Or it actually does not impact much on the calculation of the tree?&lt;/p&gt;&#xA;&lt;p&gt;Dataset B: gene1 and gene2 are having LG as their best estimated model. The concatenated matrix has LG as its best estimated model too. Will model partitioning has any effect in this case? Does the ML tree/site likelihood calculation affected by the length of the input sequences?&lt;/p&gt;&#xA;" OwnerUserId="5054" LastEditorUserId="5054" LastEditDate="2022-09-08T09:16:47.637" LastActivityDate="2024-01-31T11:10:56.337" Title="Single or partitioned model for supermatrix tree inference?" Tags="|phylogenetics|phylogeny|modelling|substitution-model|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="19660" PostTypeId="1" CreationDate="2022-09-07T12:32:22.660" Score="2" ViewCount="92" Body="&lt;p&gt;I'm currently analyzing snRNA-seq data and have to demultiplex them using HTODemux in Seurat.&lt;/p&gt;&#xA;&lt;p&gt;The question is : should I do my QC filters on my droplets before or after the demultiplexing ?&lt;/p&gt;&#xA;&lt;p&gt;From what I can test, the QC filters actually is going to have an effect on the assignation to each sample, as you can see in the small summary data below&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Demultiplex, then QC :&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Cutoff for 2-2 : 104 reads&#xA;Cutoff for 2-3 : 242 reads&#xA;Cutoff for 5-2 : 80 reads&#xA;Cutoff for 5-5 : 78 reads&#xA;&#xA; Doublet Negative      2-2      2-3      5-5      5-2 &#xA;     347     1754     1182      960     1278      959 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;QC, then Demultiplex :&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Cutoff for 2-2 : 109 reads&#xA;Cutoff for 2-3 : 194 reads&#xA;Cutoff for 5-2 : 83 reads&#xA;Cutoff for 5-5 : 74 reads&#xA;&#xA; Doublet Negative      2-2      2-3      5-5      5-2 &#xA;     426     1466     1066     1284     1302      936 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see, in the case of sample 2-3 we have a 25% difference in assignation, so I don't think we can completely ignore that.&lt;/p&gt;&#xA;&lt;p&gt;To be clear, I understand why there's a difference (normalisation, clusterisation differences, quantiles differences), I am looking for best practices here.&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="16104" LastActivityDate="2022-09-07T12:32:22.660" Title="snRNA-seq Demultiplexing before or after QC" Tags="|single-cell|quality-control|demultiplexing|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19661" PostTypeId="1" AcceptedAnswerId="19663" CreationDate="2022-09-07T15:06:53.927" Score="2" ViewCount="70" Body="&lt;p&gt;There is a graph in the link &lt;a href=&quot;https://odelaneau.github.io/GLIMPSE/rsquare_eur.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://odelaneau.github.io/GLIMPSE/rsquare_eur.html&lt;/a&gt;. This graph shows the imputation accuracy of certain MAF bins. Now I understand that &lt;span class=&quot;math-container&quot;&gt;$r^2$&lt;/span&gt; is the correlation between imputed genotype and true genotype. What I don't understand is how they calculate this &lt;span class=&quot;math-container&quot;&gt;$r^2$&lt;/span&gt; correlation. Can anyone suggest to me how I can define the r^2 correlation between true genotype and imputed genotype? I suppose it depends on the individual who defines his own &lt;span class=&quot;math-container&quot;&gt;$r^2$&lt;/span&gt;. So, it may differ for the different algorithms. However, different aspects or different &lt;span class=&quot;math-container&quot;&gt;$r^2$&lt;/span&gt; definitions will help me understand and represent this graph slightly differently.&lt;/p&gt;&#xA;" OwnerUserId="10296" LastEditorUserId="3694" LastEditDate="2022-09-07T18:02:54.420" LastActivityDate="2022-09-08T12:37:56.857" Title="correlation between imputed genotype and true genotype" Tags="|correlation|imputation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19662" PostTypeId="2" ParentId="19656" CreationDate="2022-09-07T16:37:27.687" Score="2" Body="&lt;p&gt;Briefly the best strategy I think is to download fastQ files and process this via an assembler. I wouldn't enter raw short read Illumina data into a local Blast database. What I'm pretty certain of is thats a bad move.&lt;/p&gt;&#xA;&lt;p&gt;Which assembler I personally think is a separate question on Bioinformatics SE. It will depend on the genome, for large eukaryote genomes BWA-mem2 seems popular, but its not my thing and there's other stuff upstream and downstream (someone might dispute that). For assembling small genomes, Spades is popular, but preferences are changing and there's also upstream QC. The other question is whether you have a reference genome to assist assembly.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The update is this a yeast genome with a reference. This is complicated because its a small genome but will comprise heterozygosity. If it was me - i.e. someone way outside the yeast community  - I'd use Spades and provide a reference genome to perform an assembly. However, I'm not sure thats at all cool, because you will will reduce the eukaryotic genome to a haplotype. This could get complicated because what you would be left with may not be biologically real, e.g. two alleles at different parts of the same chromosome on different strands may result in a &lt;em&gt;in silico&lt;/em&gt; artefact. I think its a separate question personally.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-09-07T18:12:45.083" LastActivityDate="2022-09-07T18:12:45.083" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19663" PostTypeId="2" ParentId="19661" CreationDate="2022-09-07T17:35:05.630" Score="2" Body="&lt;p&gt;This is done by downsampling. Take the 1000 genomes, set some genotypes as missing &lt;code&gt;./.&lt;/code&gt;, impute them using GLIMPSE, then measure correlation between the genotype dosages in the imputed dataset and the same sample at high coverage - in this case, the 1000 genomes sequenced to 30x coverage by the NY Genome Centre.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="492" LastEditDate="2022-09-08T12:37:56.857" LastActivityDate="2022-09-08T12:37:56.857" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19664" PostTypeId="1" AcceptedAnswerId="19665" CreationDate="2022-09-07T18:32:40.493" Score="2" ViewCount="75" Body="&lt;p&gt;I have obtained several hundred raw, unassembled yeast genomes from NCBI and I am looking for advice on how to process the genomes for downstream analysis.&lt;/p&gt;&#xA;&lt;p&gt;I have a reference genome (&lt;a href=&quot;http://sgd-archive.yeastgenome.org/sequence/S288C_reference/genome_releases/&quot; rel=&quot;nofollow noreferrer&quot;&gt;S288C&lt;/a&gt;) to use for assembly/alignment but I'm not sure what to do first.&lt;/p&gt;&#xA;&lt;p&gt;So far I have run a quality assessment (FastQC) followed by trimming (Trimmomatic). But I'm not sure if I should align or assemble next. The problem with yeast is if I assemble to a reference genome it reduces the genome to a haplotype and could miss moments of heterozygosity present. And I'm not sure how it would work to do a de novo assembly as I'm assuming it's the same problem. Same with aligning to the reference genome.&lt;/p&gt;&#xA;&lt;p&gt;Are there ways to work around this? Does the yeast research community just ignore this issue? Are there tools currently out there that take all of this into account?&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="73" LastEditDate="2022-09-09T08:58:41.087" LastActivityDate="2022-09-09T08:58:41.087" Title="What is the best way to process yeast genomes?" Tags="|sequence-alignment|genome|assembly|genotyping|haplotypes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19665" PostTypeId="2" ParentId="19664" CreationDate="2022-09-07T18:51:52.320" Score="2" Body="&lt;p&gt;What is your data type? Many assemblers can resolve heterozygosity.&lt;/p&gt;&#xA;&lt;p&gt;PacBio HiFi reads can be used with &lt;a href=&quot;https://github.com/chhylp123/hifiasm&quot; rel=&quot;nofollow noreferrer&quot;&gt;hifiasm&lt;/a&gt; to resolve both haplotypes rather trivially. This is the best case scenario and older sequencing projects are unlikely to have HiFi reads.&lt;/p&gt;&#xA;&lt;p&gt;If they are merely PacBio subreads, you can use &lt;a href=&quot;https://pb-falcon.readthedocs.io/en/latest/about.html.&quot; rel=&quot;nofollow noreferrer&quot;&gt;Falcon-Unzip&lt;/a&gt; to get heterozygous regions (though this is now quite old and I am not sure that it is recommended anymore).&lt;/p&gt;&#xA;&lt;p&gt;For Illumina, the options are worse. &lt;a href=&quot;http://platanus.bio.titech.ac.jp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Platanus&lt;/a&gt; is an assembler intended for the purpose of assembling heterozygous genomes. If nothing else, you can make the assembly and then just call variants.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note that many yeast are probably sequenced from a haploid strain.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2022-09-07T19:30:15.480" LastActivityDate="2022-09-07T19:30:15.480" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19666" PostTypeId="1" CreationDate="2022-09-08T00:59:31.310" Score="2" ViewCount="78" Body="&lt;p&gt;So I'm downloading XML files from Kegg using BioConductor and and I'm running into a problem. If we look at &lt;a href=&quot;https://www.genome.jp/pathway/hsa04066&quot; rel=&quot;nofollow noreferrer&quot;&gt;HIF-1 signaling pathway&lt;/a&gt; we see the node &lt;a href=&quot;https://www.kegg.jp/entry/hsa:1950+hsa:3479+hsa:3630&quot; rel=&quot;nofollow noreferrer&quot;&gt;growth factor (GF)&lt;/a&gt; on the left hand side, we can trace it's XML entry to this entry:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    &amp;lt;entry id=&amp;quot;17&amp;quot; name=&amp;quot;hsa:1950 hsa:3479 hsa:3630&amp;quot; type=&amp;quot;gene&amp;quot;&#xA;        link=&amp;quot;https://www.kegg.jp/dbget-bin/www_bget?hsa:1950+hsa:3479+hsa:3630&amp;quot;&amp;gt;&#xA;        &amp;lt;graphics name=&amp;quot;EGF, HOMG4, URG...&amp;quot; fgcolor=&amp;quot;#000000&amp;quot; bgcolor=&amp;quot;#BFFFBF&amp;quot;&#xA;             type=&amp;quot;rectangle&amp;quot; x=&amp;quot;87&amp;quot; y=&amp;quot;272&amp;quot; width=&amp;quot;46&amp;quot; height=&amp;quot;17&amp;quot;/&amp;gt;&#xA;    &amp;lt;/entry&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;While there is no GF on this, there are the three info card links that I can use to extract information. However, when I click on that link, there isn't a place on those info cards that says that these three genes are related to that general title of GF.&lt;/p&gt;&#xA;&lt;p&gt;Normally this isn't a problem, I would just look at the map, but I'm trying to do something at scale where I can't look at every instance of this.&lt;/p&gt;&#xA;&lt;p&gt;So my question is, how do I find the labels that Kegg uses in its map using the XML file/python package?&lt;/p&gt;&#xA;" OwnerUserId="16106" LastActivityDate="2022-09-26T17:50:32.450" Title="Naming conventions for kegg nodes with multiple genes" Tags="|python|kegg|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19667" PostTypeId="2" ParentId="19659" CreationDate="2022-09-08T01:48:54.733" Score="0" Body="&lt;p&gt;Okay, you can't concatenate genes into a singular matrix particularly if these are rRNA genes versus protein genes, except in rare situations where the tests say 'thats okay' (below). This is simply because they evolve at very different rates rRNA genes have loci which evolve very slowly and are highly conserved.&lt;/p&gt;&#xA;&lt;p&gt;The theory goes back to incongruence length difference test (ILD), which is somewhat confusingly also known as the partition homogeneity test and was then modified to become the localised ILD test.&lt;/p&gt;&#xA;&lt;p&gt;The modern version of this test, simply called 'homogeneity' is found in &lt;a href=&quot;http://www.iqtree.org/doc/Assessing-Phylogenetic-Assumptions&quot; rel=&quot;nofollow noreferrer&quot;&gt;IQ-Tree&lt;/a&gt;. Thus, the best approach is to test the assumption empirically,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;IQ-TREE provides three matched-pairs tests of symmetry (Naser-Khdour&#xA;et al., 2019) to test the two assumptions of stationarity and&#xA;homogeneity. A simple analysis:&lt;/p&gt;&#xA;&lt;p&gt;iqtree2 -s example.phy -p example.nex --symtest-only&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Just to be clear this tests whether the rate between two partitions (genes) is homogeneous. If 'yes' the supermatrix is fine, if 'no' the data needs partitioning.&lt;/p&gt;&#xA;&lt;p&gt;The theory is simply that if two genes evolve at noticeably different rates they can't be combined into a singular matrix. The theory goes back a long way is used to circumvent long-branch attraction issues. Thus, at a minimum the rRNA genes and protein genes need to be compartmentalised, i.e. have their own matrix. Between protein genes, if they are house-keeping genes, generally they'll have homogeneous rates (which you can empirically test via IQ-TREE).&lt;/p&gt;&#xA;&lt;p&gt;If someone is concatenating rRNA with protein genes without accommodating rate variation via partitioning it is simply sloppy phylogenetics and if published represents a shortcoming with the reviewing process. This is because in all certainly these loci will fail any rate homogeneity test, there's a slim off-chance its okay which you can test it empirically. Say if 18S and 28S were concatenated (or just 16S if these are bacteria) with an atp gene, that can give a different tree to 18S with atpA, atpB, atpC, atpD etc... so the ratio between rRNA to protein genes can start influencing the tree ... which is simply wrong.&lt;/p&gt;&#xA;&lt;p&gt;If someone concatenates a load of house-keeping genes thats okay because there's a good chance they're homogeneous and if even the test failed, they may not be that far out, rRNA versus proteins can be way out. This can easily be an order of magnitude out and thats difficult to ignore. When there's proteins under selection versus conserved housekeeping genes ... its very unlikely homogeneity would hold.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; As a general rule of thumb in your method of tree building compartmentalising all genes is safer route, computationally there's no difference between zero partitions and a partition for every gene. You are never going to make a mistake.&lt;/p&gt;&#xA;&lt;p&gt;Where it gets complicated is when the phylogenetic parameters, particularly in advanced models, are themselves estimated via maximum likelihood alongside the topology. The rational for this is complicated. Computationally, this can get very, very expensive even on supercomputers and hence there's a tendency to make an assumption (often without checking it) that there are fewer partitions than there actually are. Regardless, assuming away rRNA versus protein genes even in this method of tree estimation should not done and the test will almost certainly fail.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-09-08T01:55:24.667" LastActivityDate="2022-09-08T01:55:24.667" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19668" PostTypeId="1" CreationDate="2022-09-08T09:03:49.300" Score="2" ViewCount="27" Body="&lt;p&gt;I want to run the experiments &amp;quot;&lt;em&gt;Sample Application to Genotype-Tissue Expression (GTEx) Project Gene Expression Data of Postmortem Tissues&lt;/em&gt;&amp;quot; mentioned in &amp;quot;&lt;em&gt;Enter the Matrix: Factorization Uncovers Knowledge from Omics&lt;/em&gt;&amp;quot; by Stein-O'Brien et.al &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0168952518301240&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The paper metioned&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;dataset from postmortem samples in the GTEx project [42] (Figure 3).&#xA;Specifically, we select a subset of GTEx data containing 12 brain&#xA;tissues in the GTEx data for seven individuals for which we had&#xA;previously performed only NMF analysis [41] (codes are provided in the&#xA;supplemental information online).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I check with &lt;a href=&quot;https://gtexportal.org/home/datasets&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gtexportal.org/home/datasets&lt;/a&gt; and assumed that the paper used the &lt;code&gt;GTEx Analysis V6&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;But which files do I need to download? I am computer scientist I am unsure which ones contain the &lt;code&gt;brain  tissues&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Further I found&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Brain_Anterior_cingulate_cortex_BA24_Analysis_cis-eQTLs.txt.gz  1.5G&#xA;Brain_Caudate_basal_ganglia_Analysis_cis-eQTLs.txt.gz   1.9G&#xA;Brain_Cerebellar_Hemisphere_Analysis_cis-eQTLs.txt.gz   1.7G&#xA;Brain_Cerebellum_Analysis_cis-eQTLs.txt.gz  1.9G&#xA;Brain_Cortex_Analysis_cis-eQTLs.txt.gz  1.9G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I am unsure which one was used in the paper&lt;/p&gt;&#xA;" OwnerUserId="2026" LastEditorUserId="2026" LastEditDate="2022-09-08T09:20:04.417" LastActivityDate="2022-09-08T09:20:04.417" Title="What data set was used Gene Expression Data of Postmortem Tissues?" Tags="|ngs|public-databases|unsupervised-learning|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19669" PostTypeId="1" CreationDate="2022-09-08T15:45:29.067" Score="1" ViewCount="22" Body="&lt;p&gt;I was given a data set consisting of 6 samples: 3 control and 3 treated. I dont have information about how data were generated and what the outputs represent.&lt;/p&gt;&#xA;&lt;p&gt;Each sample corresponds to a folder within which there are the following files:&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;DB search psm.csv&amp;quot; &amp;quot;de novo only peptides.csv&amp;quot; &amp;quot;de novo only peptides.xml&amp;quot; &amp;quot;de novo only spectra.mgf&amp;quot; &amp;quot;filter.csv&amp;quot; &amp;quot;peptide features.csv&amp;quot;&#xA;&amp;quot;peptide.csv&amp;quot; &amp;quot;peptides_1_1_0.mzid&amp;quot; &amp;quot;peptides.pep.xml&amp;quot; &amp;quot;protein-peptides.csv&amp;quot; &amp;quot;proteins.csv&amp;quot; &amp;quot;proteins.fasta&amp;quot; &amp;quot;PSM ions.csv&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;The &amp;quot;peptide.csv&amp;quot; files have the following column names:&lt;/p&gt;&#xA;&lt;p&gt;Peptide -10lgP Mass Length ppm m/z RT 1/k0 Range Area Ctrl-2 Fraction Precursor Id Source&lt;/p&gt;&#xA;&lt;p&gt;The &amp;quot;proteins.csv&amp;quot; files have the following column names:&lt;/p&gt;&#xA;&lt;p&gt;Protein Group Protein ID Accession -10lgP Coverage (%) Coverage (%) Ctrl-1 Area Ctrl-1 #Peptides #Unique #Spec Ctrl-1 PTM Avg. Mass Description&lt;/p&gt;&#xA;&lt;p&gt;Has anybody ever worked with these type of data?&lt;/p&gt;&#xA;&lt;p&gt;I would like to do exploit these data and perform a differential analysis but i don't know how to create a SummarizedExperiment from these data.&lt;/p&gt;&#xA;&lt;p&gt;Any help would be greatly appreciated.&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="16110" LastActivityDate="2022-09-08T15:45:29.067" Title="Mass spectrometry: How to obtain expression counts for protein" Tags="|differential-expression|proteomics|mass-spectrometry|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19670" PostTypeId="1" CreationDate="2022-09-08T17:12:26.863" Score="1" ViewCount="29" Body="&lt;p&gt;I want to map TFBS (transcription factor binding sites) on to the sequences from dataset containing 8448 (210 base) sequences, but it is  not giving all the hits I need&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pickle&#xA;from pickle import load, dump&#xA;from collections import defaultdict&#xA;from Bio.Seq import Seq&#xA;&#xA;#first element is BREu = SSRCGCC, length = 7 bases, size of matrix = 4*7, max score = 3.33&#xA;# first row = A, second row = T, third row = G and forth row = C&#xA;BREu = [&#xA;    [0.0, 0.0, 0.5, 0, 0, 0, 0],&#xA;    [0.0, 0.0, 0.0, 0, 0, 0, 0],&#xA;    [0.5, 0.5, 0.5, 0, 1, 0, 0],&#xA;    [0.5, 0.5, 0.0, 1, 0, 1, 1]]&#xA;&#xA;#second element is ETS = MGGAART, length = 7 bases, size of matrix = 4*7, max score = 6&#xA;ETS = [&#xA;    [0.5, 0, 0, 1, 1, 0.5, 0],&#xA;    [0.0, 0, 0, 0, 0, 0.0, 1],&#xA;    [0.0, 1, 1, 0, 0, 0.5, 0],&#xA;    [0.5, 0, 0, 0, 0, 0.0, 0]]&#xA;&#xA;#Third element is INR = YYANWYY, length = 7 bases, size of matrix = 4*7, max score = 3.75&#xA;INR = [&#xA;    [0.0, 0.0, 1, 0.25, 0.5, 0.0, 0.0],&#xA;    [0.5, 0.5, 0, 0.25, 0.5, 0.5, 0.5],&#xA;    [0.0, 0.0, 0, 0.25, 0.0, 0.0, 0.0],&#xA;    [0.5, 0.5, 0, 0.25, 0.0, 0.5, 0.5]]&#xA;&#xA;#forth element is NFY = CCAAT, length = 5 bases, size of matrix = 4*5, max score = 5&#xA;NFY = [&#xA;    [0, 0, 1, 1, 0],&#xA;    [0, 0, 0, 0, 1],&#xA;    [0, 0, 0, 0, 0],&#xA;    [1, 1, 0, 0, 0]]&#xA;&#xA;#seventh element is SP1_NC = GGGNGGG, length = 7 bases, size of matrix = 4*7, max score = 7&#xA;#Gives some possibilities of having any bases at position 4 while still favoring G&#xA;#threshold score &amp;gt;= 6.5 : keep all degenerated motif with degenerated = 6.5 and canonical = 7&#xA;&#xA;SP1 = [&#xA;    [0, 0, 0, .5, 0, 0, 0],&#xA;    [0, 0, 0, .5, 0, 0, 0],&#xA;    [1, 1, 1, .5, 1, 1, 1],&#xA;    [0, 0, 0, 1., 0, 0, 0]]&#xA;&#xA;#sixth element is TATA = TATAWAAG, length = 8 bases, size of matrix = 4*8, max score = 7.5&#xA;#threshold score &amp;gt; 20 : keep all degenerated motif with degenerated = 20 and canonical = 23&#xA;TATA = [&#xA;    [0, 4, 0, 4, 4, 1, 1, 0],&#xA;    [4, 0, 4, 0, 4, 0, 0, 0],&#xA;    [0, 0, 0, 0, 0, 0, 0, 1],&#xA;    [0, 0, 0, 0, 0, 0, 0, 0]]&#xA;&#xA;# define matrix and thresholds as tuple&#xA;motifs = dict(&#xA;    BREu = (BREu, 5.5),&#xA;    ETS = (ETS, 6),&#xA;    INR = (INR, 3.75),&#xA;    NFY = (NFY, 5),&#xA;    SP1 = (SP1, 6.25),&#xA;    TATA = (TATA, 20))&#xA;&#xA;def get_motif_score(seq, motif):&#xA;&#xA;    motif_size = len(motif[0])&#xA;    bases_encodings = {'A': 0, 'T': 1, 'G': 2, 'C': 3}&#xA;    coded_seq = [&#xA;        bases_encodings[b] if b in 'ATGC' else None&#xA;        for b in seq]&#xA;&#xA;    return [&#xA;        sum(&#xA;            0 if coded_b is None else motif[coded_b][i]&#xA;            for i, coded_b in enumerate(coded_seq[start:start+motif_size])&#xA;        ) for start in range(len(seq) - motif_size)]&#xA;&#xA;&#xA;def get_motifs(seq):&#xA;    &amp;quot;&amp;quot;&amp;quot;find motif occurences in given sequence&#xA;    &#xA;    return list of (motif sequence, start, end, score)&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    seq_size = len(seq)&#xA;    motif_hits = {}&#xA;&#xA;    for motif_name, (motif_matrix, threshold) in motifs.items():&#xA;        motif_size = len(motif_matrix[0])&#xA;        rcseq = str(Seq(seq).reverse_complement())&#xA;        scores = get_motif_score(seq, motif_matrix)&#xA;        rc_scores = get_motif_score(rcseq, motif_matrix)&#xA;        &#xA;        motif_hits[motif_name] = [&#xA;            (seq[i:i+motif_size], i, i+motif_size, score,'+')&#xA;            for i, score in enumerate(scores)&#xA;            if score &amp;gt;= threshold&#xA;        ] + [&#xA;            (rcseq[i:i+motif_size], seq_size-(i+motif_size), (seq_size-i), score,'-')&#xA;            for i, score in enumerate(rc_scores)&#xA;            if score &amp;gt;= threshold&#xA;        ]&#xA;&#xA;    return motif_hits&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16108" LastEditorUserId="10049" LastEditDate="2022-09-10T02:01:06.380" LastActivityDate="2022-09-10T02:01:06.380" Title="I want to map TFBS on to the sequences from dataset containing 8448 (210 base) sequences. its not giving all the hits I need" Tags="|python|phylogenetics|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="19671" PostTypeId="1" AcceptedAnswerId="19674" CreationDate="2022-09-09T10:53:40.440" Score="1" ViewCount="216" Body="&lt;p&gt;From the preprint published by 1000 genome project (&lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2021.02.06.430068v1.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/10.1101/2021.02.06.430068v1.full&lt;/a&gt;) I think the 30x data is for WGS. Can anyone confirm for me if the following file is really WGS data?&#xA;&lt;a href=&quot;ftp://ftp.sra.ebi.ac.uk/vol1/run/ERR324/ERR3242643/HG03007.final.cram&quot; rel=&quot;nofollow noreferrer&quot;&gt;ftp://ftp.sra.ebi.ac.uk/vol1/run/ERR324/ERR3242643/HG03007.final.cram&lt;/a&gt;.&lt;br /&gt;&#xA;Also, Is there any more freely available sequencing data where I can find WGS data for 30x and low coverage(&amp;lt;10x) for the same sample for the South Asian population?&lt;/p&gt;&#xA;" OwnerUserId="10296" LastEditorUserId="3694" LastEditDate="2022-09-12T09:51:32.287" LastActivityDate="2022-09-12T09:51:32.287" Title="Where can I download 30x 1000 genomes cram files?" Tags="|wgs|1000genomes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19672" PostTypeId="1" CreationDate="2022-09-09T11:56:05.530" Score="1" ViewCount="41" Body="&lt;p&gt;I am working on a datasets and after some discussion with my group, we doubt that maybe one or more of our controls are different than the other controls. The motivation is to see if one or more controls have been effected differently by the solvent they were kept in.&lt;/p&gt;&#xA;&lt;p&gt;I have been suggested to use bootstrap method. If we suppose that I have a dataset with 5 controls and 3 treated samples. I want to create 5 new data frames with the information such as every new data frame skips one out of 5 controls and do resampling with replacement. I want to see how stable the controls are and by skipping a specific control, would the DEGs result change.&lt;/p&gt;&#xA;&lt;p&gt;Let us suppose that the original data frame is like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;x &amp;lt;- round(matrix(rexp(480 * 10, rate=.1), ncol=8), 0)&#xA;&#xA;rownames(x) &amp;lt;- paste(&amp;quot;gene&amp;quot;, 1:nrow(x))&#xA;&#xA;colnames(x) &amp;lt;-c(&amp;quot;control1&amp;quot;,&amp;quot;control2&amp;quot;,&amp;quot;control3&amp;quot;,&amp;quot;control4&amp;quot;,&amp;quot;control5&amp;quot;,&amp;quot;treated&amp;quot;,&amp;quot;treated&amp;quot;,&amp;quot;treated&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to create 5 new data frames (as there are 5 controls in this study) where each data frame skips one specific control and replace with with another control (which means some other control will repeat).&lt;/p&gt;&#xA;&lt;p&gt;For example one of the 5 data frame can look like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;x1 &amp;lt;- round(matrix(rexp(480 * 10, rate=.1), ncol=8), 0)&#xA;&#xA;rownames(x1) &amp;lt;- paste(&amp;quot;gene&amp;quot;, 1:nrow(x1))&#xA;&#xA;colnames(x1) &amp;lt;-c(&amp;quot;control1&amp;quot;,&amp;quot;control1.1&amp;quot;,&amp;quot;control3&amp;quot;,&amp;quot;control4&amp;quot;,&amp;quot;control5&amp;quot;,&amp;quot;treated&amp;quot;,&amp;quot;treated&amp;quot;,&amp;quot;treated&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can see that this new data frame skipped control2 with a copy of control1 called control1.1.&lt;/p&gt;&#xA;" OwnerUserId="16114" LastEditorUserId="3694" LastEditDate="2023-06-12T08:27:28.317" LastActivityDate="2023-06-12T08:27:28.317" Title="Creating multiple phenotype datasets using bootstrap method &quot;Bootstrap-samples-by-column-of-a-data-frame-in-r&quot; for DEG analysis" Tags="|r|deseq2|dataframe|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19673" PostTypeId="1" CreationDate="2022-09-09T12:14:05.087" Score="0" ViewCount="63" Body="&lt;p&gt;Do LoF/GoF transcripts count toward the RNA-seq TPM count? Or would these LoF/GoF transcripts only be detected by isoform quantification?&lt;/p&gt;&#xA;" OwnerUserId="5187" LastEditorUserId="5187" LastEditDate="2022-09-09T12:46:45.730" LastActivityDate="2022-09-10T16:41:32.403" Title="Is loss/gain of function reflected in RNA-seq transcript counts?" Tags="|rna-seq|gene-expression|rna|rna-alignment|rna-structure|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19674" PostTypeId="2" ParentId="19671" CreationDate="2022-09-09T12:16:42.277" Score="2" Body="&lt;p&gt;Yes, those are the .cram files for the 30x whole genome sequencing data for 1000 genomes.&lt;/p&gt;&#xA;&lt;p&gt;If you want 10x and 30x sequence data for the same individual, you can just downsample the 30x .cram file using &lt;code&gt;samtools view -s&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="10296" LastEditDate="2022-09-09T14:33:20.050" LastActivityDate="2022-09-09T14:33:20.050" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19675" PostTypeId="2" ParentId="18182" CreationDate="2022-09-09T16:50:01.740" Score="2" Body="&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; values are the ratio of methylated to unmethylated probe intensities.&#xA;This means that &lt;span class=&quot;math-container&quot;&gt;$\beta \in [0, 1]$&lt;/span&gt;, so it can't be normally distributed, by definition.&#xA;Its distribution may have sections that look like bell curves, but it isn't normally distributed.&#xA;You can take a look at this paper by Du &lt;em&gt;et al.&lt;/em&gt; [1] for details about the statistics of &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; values.&lt;/p&gt;&#xA;&lt;p&gt;That said, the distribution of all &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt; values from an array or bisulfite sequencing (or some other DNA methylation assay) should be bimodal with modes at 0 and 1.&#xA;Nearly all CpGs in the human genome are methylated or unmethylated.&#xA;There may be some hemi-methylated sites, but for the 450K array you mentioned, there should be few of these.&lt;/p&gt;&#xA;&lt;p&gt;It sounds like you might be looking at some residual value, or some other value from the analysis in the paper for dataset &lt;span class=&quot;math-container&quot;&gt;$b$&lt;/span&gt; other than &lt;span class=&quot;math-container&quot;&gt;$\beta$&lt;/span&gt;.&#xA;Could you be looking at &lt;span class=&quot;math-container&quot;&gt;$M$&lt;/span&gt; values, or something else?&lt;/p&gt;&#xA;&lt;p&gt;[1]: &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-11-587&quot; rel=&quot;nofollow noreferrer&quot;&gt;Du &lt;em&gt;et al.&lt;/em&gt;, BMC Bioinformatics, 2010&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2022-09-09T16:50:01.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19677" PostTypeId="2" ParentId="19673" CreationDate="2022-09-10T16:41:32.403" Score="1" Body="&lt;h1&gt;Short answer&lt;/h1&gt;&#xA;&lt;p&gt;Since the effects of any given mutation is so context-dependent (and deciding which effects matter will change between researchers), the process of &amp;quot;variant annotation&amp;quot; is typically separate from RNA-seq mapping/quantification.&lt;/p&gt;&#xA;&lt;p&gt;If you use a common RNA-seq mapping/quantification tool like STAR, HISAT2, or Kallisto, these counts will combine transcripts with and without the mutation of interest.&#xA;This is because for the purpose of mapping, mutations are no different than sequencing errors, and when you're mapping you don't actually know what is what.&lt;/p&gt;&#xA;&lt;h1&gt;Long answer&lt;/h1&gt;&#xA;&lt;p&gt;You might not be able to quantify GoF/LoF at the isoform level, either.&#xA;Again, this is because the effects of mutations are so context-dependent.&lt;/p&gt;&#xA;&lt;p&gt;If you want to know how many transcripts do or don't contain a specific mutation, you'll have to work backwards from that mutation and look at the RNA-seq reads themselves.&#xA;You can use tools like Rsamtools, htseq-count, or even IGV for this.&lt;/p&gt;&#xA;&lt;p&gt;If you want to know whether a given protein's function is affected, you have to consider all the types of mutations than can lead to that protein being affected (in your biological scenario of interest, of course).&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Some proteins may lose their binding affinity if a single amino acid is changed, in which case you can look for non-synonymous mutations in the RNA-seq reads.&lt;/li&gt;&#xA;&lt;li&gt;Some genes may undergo alternative splicing and produce different proteins because of some mutations.&#xA;You could find this with a differential isoform expression analysis.&lt;/li&gt;&#xA;&lt;li&gt;Some genes could be deleted on one allele, leading to a lower expression of that protein.&#xA;Biologically, that protein is still expressed, but having less of it makes it less functional in general.&#xA;You might be able to find that with copy number estimation or allele-specific expression analysis.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If you're looking at non-coding RNA, what mutations affect their function will be different from what you'd look for if you care about protein function.&lt;/p&gt;&#xA;&lt;p&gt;All in all, any investigation about what a GoF/LoF mutation looks like and how to go about finding it will depend on you biological scenario.&#xA;And you may not find it by just looking at the TPM counts of a gene or its isoforms.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2022-09-10T16:41:32.403" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19678" PostTypeId="1" CreationDate="2022-09-10T19:17:52.937" Score="1" ViewCount="132" Body="&lt;p&gt;I am trying to split my &lt;em&gt;.bed/&lt;/em&gt;.fa file genomic segments into arbitrary smaller overlapping intervals:&lt;/p&gt;&#xA;&lt;p&gt;Consider a typical line/row in my *.bed file as follows:&lt;/p&gt;&#xA;&lt;p&gt;chrom. &lt;span class=&quot;math-container&quot;&gt;$\ \ \ \ \ $&lt;/span&gt;  start &lt;span class=&quot;math-container&quot;&gt;$\ \ \ \ \ \ \ \ \ \ \ \ $&lt;/span&gt; end&lt;/p&gt;&#xA;&lt;p&gt;chr7 &lt;span class=&quot;math-container&quot;&gt;$\ \ \ $&lt;/span&gt;   127471196 &lt;span class=&quot;math-container&quot;&gt;$\ \ \ $&lt;/span&gt;    127472363&lt;span class=&quot;math-container&quot;&gt;$\ \ \ $&lt;/span&gt;      ...&lt;/p&gt;&#xA;&lt;p&gt;The start and end genomic positions include 1167 base pairs for this line. I need to extend this line, for example, into multiple lines where each line covers exactly 500bp like below:&lt;/p&gt;&#xA;&lt;p&gt;chr7 &lt;span class=&quot;math-container&quot;&gt;$\ \ \ $&lt;/span&gt;    127471196 &lt;span class=&quot;math-container&quot;&gt;$\ \ \ $&lt;/span&gt;    127471696 &lt;span class=&quot;math-container&quot;&gt;$\ \ \ $&lt;/span&gt;       ...&lt;/p&gt;&#xA;&lt;p&gt;chr7 &lt;span class=&quot;math-container&quot;&gt;$\ \ \ $&lt;/span&gt;    127471863  &lt;span class=&quot;math-container&quot;&gt;$\ \ \ $&lt;/span&gt;   127472363 &lt;span class=&quot;math-container&quot;&gt;$\ \ \ $&lt;/span&gt;       ...&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ws3ts.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ws3ts.png&quot; alt=&quot;![enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;While each line covers 500pb, between the two new lines they both include the same &amp;quot;range&amp;quot; of genomic positions as the original line including some overlap which is fine (and desired)&lt;/p&gt;&#xA;&lt;p&gt;Is there an existing tool (perhaps part of bedtools?) that can do something like this?&lt;/p&gt;&#xA;&lt;p&gt;I know there is the &lt;strong&gt;bedops&lt;/strong&gt; tool that does ALMOST this using the --chop option, where it takes the original segment x1:x2 and chops the original segment into specified fixed bp length intervals, however, if there is any left over towards the end is creates a new non-standard length segment (i.e. x1:x2=10bp and chop length=3, it will give you 4 segments that are length 3, 3, 3 and 1 - what I am looking for is something that adjusts for the last remainder and outputs segments of length 3,3,3 and 3 where the last length 3 segment is right adjusted, while the first three segments are left adjusted or better yet it gives the user the choice of precise redefinition of segment coordinates y1,y2,y3 and y4).  Thank you.&lt;/p&gt;&#xA;" OwnerUserId="15741" LastEditorUserId="15741" LastEditDate="2022-09-11T17:50:53.933" LastActivityDate="2022-09-19T16:58:53.990" Title="How to chop fasta / bed /peak file genomic segments into smaller fixed or custom genomic intervals" Tags="|python|fasta|sequence-alignment|chip-seq|bedtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19679" PostTypeId="1" CreationDate="2022-09-11T10:23:49.200" Score="0" ViewCount="76" Body="&lt;p&gt;Sorry for this very beginner homework question:&lt;/p&gt;&#xA;&lt;p&gt;Which of the following linear peptides is consistent with Spectrum = {0 71 99 101 103 128 129 199 200 204 227 230 231 298 303 328 330 332 333}? (Select all that apply.)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CTV&#xA;&#xA;TCE&#xA;&#xA;QCV&#xA;&#xA;AQV&#xA;&#xA;VAQ&#xA;&#xA;TCQ&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried to use the molecular weight to deduce the answer, but I failed.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;In [1]: from rdkit import Chem&#xA;   ...: from rdkit.Chem import Descriptors&#xA;&#xA;In [2]: chemical_string = 'A'&#xA;&#xA;In [3]: mol: Chem.Mol = Chem.MolFromFASTA(chemical_string)&#xA;&#xA;In [4]: print( Descriptors.ExactMolWt(mol) )&#xA;89.047678464&#xA;&#xA;In [5]: chemical_string = 'C'&#xA;&#xA;In [6]: mol: Chem.Mol = Chem.MolFromFASTA(chemical_string)&#xA;&#xA;In [7]: print( Descriptors.ExactMolWt(mol) )&#xA;121.019749464&#xA;&#xA;In [8]: chemical_string = 'E'&#xA;&#xA;In [9]: mol: Chem.Mol = Chem.MolFromFASTA(chemical_string)&#xA;&#xA;In [10]: print( Descriptors.ExactMolWt(mol) )&#xA;147.053157768&#xA;&#xA;In [11]: chemical_string = 'T'&#xA;&#xA;In [12]: mol: Chem.Mol = Chem.MolFromFASTA(chemical_string)&#xA;&#xA;In [13]: print( Descriptors.ExactMolWt(mol) )&#xA;119.058243148&#xA;&#xA;In [14]: chemical_string = 'Q'&#xA;&#xA;In [15]: mol: Chem.Mol = Chem.MolFromFASTA(chemical_string)&#xA;&#xA;In [16]: print( Descriptors.ExactMolWt(mol) )&#xA;146.06914218&#xA;&#xA;In [17]: chemical_string = 'V'&#xA;&#xA;In [18]: mol: Chem.Mol = Chem.MolFromFASTA(chemical_string)&#xA;&#xA;In [19]: print( Descriptors.ExactMolWt(mol) )&#xA;117.078978592&#xA;&#xA;In [20]: &#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15733" LastEditorUserId="73" LastEditDate="2022-09-12T18:45:14.097" LastActivityDate="2022-09-12T18:45:14.097" Title="Relationship between molecular weight and linear peptide" Tags="|sequence-alignment|homework|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="19680" PostTypeId="1" CreationDate="2022-09-12T16:17:19.330" Score="2" ViewCount="31" Body="&lt;p&gt;I have currently a following problem. I have a one sample that I did my metagenomics on (Illumina shotgun + nanopore) and have recovered some high quality MAGs. I also did a metatranscriptomics on the same sample (merging reads with PEAR, removing low quality with Trimmomatic and removing RNA with SortMeRNA). What I would like to do now is to check the expression of the genes in my MAGs with the metatranscriptomics of the sample (hope it's clear enough). I aligned metatranscriptomics reads to each MAG and retained only those reads that aligned (used BOWTIE2). Now I am thinking of doing a functional analysis to check what genes are expressed in each MAG. However, I have no idea how and where to start. Was the alignment even a good idea? How would you tackle this problem?&lt;/p&gt;&#xA;&lt;p&gt;My idea is to try TransDecoder on these aligned transcriptomics reads to get the ORFs and cluster them with CD-HIT2. Then perform a functional analysis as normal with KEGG/InterProSCAN/Blast etc. Would this be a correct way to go?&lt;/p&gt;&#xA;&lt;p&gt;Your advice will be highly appreciated.&lt;/p&gt;&#xA;&lt;p&gt;B&lt;/p&gt;&#xA;" OwnerUserId="14212" LastEditorUserId="14212" LastEditDate="2022-09-12T16:23:06.837" LastActivityDate="2022-09-12T16:23:06.837" Title="MAGs transcriptomics pipeline question" Tags="|gene-expression|transcriptome|read-mapping|metagenome|functional-annotation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19682" PostTypeId="2" ParentId="16415" CreationDate="2022-09-12T17:51:44.813" Score="1" Body="&lt;p&gt;The way I would do it is to make an assumption that the reads mapping to your contig == (abundance of the genes in the contig)/contig length.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Map raw reads to the contig containing your hit (original contig, pre-translation)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Normalise it with RPKM:&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;RPKM = NumberOfReadsThatAligned / ( ContigLength/1000 * totalNumberReads/1,000,000 )&lt;/p&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;This will give you the relative abundance of the contig which is an APPROXIMATION of the abundance of your gene.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Alternatively, to be more precise you can FragGeneScan your raw reads, extract the sequence of your putative protein and map translated raw reads to it.&lt;/p&gt;&#xA;&lt;p&gt;For your other question: Yes, HMMs are more sensitive than blast or diamond since its output is probability score that these protein contain motifs extracted from your alignments (given that there is one). For that reason you need to look at the bitscore and E-value. If the bitscore is low and/or E-value is high, you have a high chance of false-positive.&lt;/p&gt;&#xA;&lt;p&gt;Best wishes,&lt;/p&gt;&#xA;&lt;p&gt;A&lt;/p&gt;&#xA;" OwnerUserId="14212" LastEditorUserId="3848" LastEditDate="2022-09-12T20:41:40.907" LastActivityDate="2022-09-12T20:41:40.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19683" PostTypeId="1" CreationDate="2022-09-13T06:28:11.007" Score="2" ViewCount="42" Body="&lt;p&gt;I want to run experiments on the data used in &lt;em&gt;PatternMarkers &amp;amp; GWCoGAPS for novel data-driven biomarkers via whole transcriptome NMF&lt;/em&gt; &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/33/12/1892/2975325?login=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So far, the paper has  reduced the dimension to&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;ammon's horn&lt;/li&gt;&#xA;&lt;li&gt;amygdala&lt;/li&gt;&#xA;&lt;li&gt;anterior cingulate cortex&lt;/li&gt;&#xA;&lt;li&gt;caudate nucleus&lt;/li&gt;&#xA;&lt;li&gt;dorsolateral prefrontal cortex&lt;/li&gt;&#xA;&lt;li&gt;first cervical spinal cord segment&lt;/li&gt;&#xA;&lt;li&gt;frontal cortex&lt;/li&gt;&#xA;&lt;li&gt;hypothalamus&lt;/li&gt;&#xA;&lt;li&gt;putamen&lt;/li&gt;&#xA;&lt;li&gt;substantia nigra&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;on the samples/patients GTEX-[11GSP, 11O72, 11OF3, 11TUW, Q2AG, QDT8, XAB4]&lt;/p&gt;&#xA;&lt;p&gt;I assume they are using it.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Gene read count.    GTEx_Analysis_v6_RNA-seq_RNA-SeQCv1.1.8_gene_reads.gct.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A small amount of the data is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Name               Description  GTEX-111CU-1826-SM-5GZYN  GTEX-111FC-0226-SM-5N9B8  GTEX-111VG-2326-SM-5N9BK  ...&#xA;ENSG00000223972.4  DDX11L1      0                         0                         0                         ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From what I can gather, &lt;code&gt;ENSG00000223972.4  DDX11L1 &lt;/code&gt; is the &lt;code&gt;DEAD/H-box helicase 11 like 1&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Therefore, I assumed that looking for the term &lt;code&gt;first cervical spinal cord segment&lt;/code&gt; would give me the name and description.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, I am wondering if I have the correct data set.&lt;/p&gt;&#xA;" OwnerUserId="2026" LastActivityDate="2022-09-13T06:28:11.007" Title="What codes represent what genes?" Tags="|rna-seq|database|unsupervised-learning|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19684" PostTypeId="1" CreationDate="2022-09-13T10:31:44.003" Score="2" ViewCount="118" Body="&lt;p&gt;I have multiple lists on python:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;a = [&amp;quot;house&amp;quot;,&amp;quot;garden&amp;quot;, &amp;quot;living room&amp;quot;, &amp;quot;dog&amp;quot;,&amp;quot;cat&amp;quot;]&#xA;&#xA;b= [&amp;quot;cat&amp;quot;,&amp;quot;dog&amp;quot;, &amp;quot;chicken&amp;quot;]&#xA;&#xA;c=[&amp;quot;house&amp;quot;, &amp;quot;garden&amp;quot;,&amp;quot;bathroom&amp;quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'd like to create the following dataframe:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;a&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: center;&quot;&gt;b&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;c&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;house&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;0&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;garden&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;0&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;living room&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;0&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;dog&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;1&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;cat&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;1&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;Thanks for the help!&lt;/p&gt;&#xA;" OwnerUserId="15596" LastEditorUserId="3848" LastEditDate="2022-09-13T13:20:20.403" LastActivityDate="2023-10-10T07:51:13.817" Title="How to merge multiple list into a single dataframe replacing common values with 1 -python" Tags="|python|phylogenetics|merge|pandas|dataframe|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19685" PostTypeId="2" ParentId="19684" CreationDate="2022-09-13T13:19:04.133" Score="1" Body="&lt;p&gt;Here  you go, just add column 'C' and you are sorted. Just to note, you must be seeking a &lt;code&gt;pandas&lt;/code&gt; solution because &lt;code&gt;merge&lt;/code&gt; is a &lt;code&gt;pandas&lt;/code&gt; command and you're asking for a &lt;code&gt;Python&lt;/code&gt; solution.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;a = [&amp;quot;house&amp;quot;,&amp;quot;garden&amp;quot;, &amp;quot;living room&amp;quot;, &amp;quot;dog&amp;quot;,&amp;quot;cat&amp;quot;]&#xA;b= [&amp;quot;cat&amp;quot;,&amp;quot;dog&amp;quot;, &amp;quot;chicken&amp;quot;]&#xA;df = pd.DataFrame(a, columns = ['a'])&#xA;df2 = pd.DataFrame(b, columns = ['b'])&#xA;dfa = df['a'].value_counts()&#xA;dfa.columns = ['a']&#xA;dfb = df2['b'].value_counts()&#xA;dfb.columns = ['b']&#xA;dfa = dfa.to_frame()&#xA;dfb = dfb.to_frame()&#xA;df3 = dfa.join(dfb).replace(np.nan, 0).astype(int)&#xA;print (df3)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;             a  b&#xA;house        1  0&#xA;garden       1  0&#xA;living room  1  0&#xA;dog          1  1&#xA;cat          1  1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and if you want to remove the 'a' column&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print (df3.drop('a', axis=1))&#xA;&#xA;             b&#xA;house        0&#xA;garden       0&#xA;living room  0&#xA;dog          1&#xA;cat          1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt; The key command here is &lt;code&gt;value_counts&lt;/code&gt; and makes a frequency plot. Its output is a &lt;code&gt;Series&lt;/code&gt; rather than a &lt;code&gt;DataFrame&lt;/code&gt; so it needs converting.&lt;/p&gt;&#xA;&lt;p&gt;If you want to keep all inputs the commands is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df3 = pd.concat([dfa, dfb]).replace(np.nan, 0).astype(int)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Alternatively,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;a = [&amp;quot;house&amp;quot;,&amp;quot;garden&amp;quot;, &amp;quot;living room&amp;quot;, &amp;quot;dog&amp;quot;,&amp;quot;cat&amp;quot;]&#xA;b= [&amp;quot;cat&amp;quot;,&amp;quot;dog&amp;quot;, &amp;quot;chicken&amp;quot;]&#xA;df = pd.DataFrame(a).value_counts().to_frame('a')&#xA;df2 = pd.DataFrame(b).value_counts().to_frame('b')&#xA;df3 = df.join(df2).replace(np.nan, 0).astype(int).rename_axis(None)&#xA;print (df3.drop('a', axis=1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-09-13T19:24:08.713" LastActivityDate="2022-09-13T19:24:08.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19686" PostTypeId="1" CreationDate="2022-09-14T00:58:46.237" Score="2" ViewCount="340" Body="&lt;p&gt;I can understand the motive behind the BLOSUM62 matrix, this being a pairwise mutation matrix describing aggregate mutations between the 20 amino acids.&lt;/p&gt;&#xA;&lt;p&gt;However how would you calculate the average BLOSUM62 score within a hydrophobic amino acids (K, D, E) vs hydrophilic amino acids (I, L, V)?&lt;/p&gt;&#xA;&lt;p&gt;How about between those two groups?&lt;/p&gt;&#xA;" OwnerUserId="16132" LastEditorUserId="3848" LastEditDate="2022-09-14T02:10:26.157" LastActivityDate="2024-02-06T20:07:15.820" Title="How to calculate average BLOSUM62 scores?" Tags="|proteins|sequence-alignment|mutations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19688" PostTypeId="1" CreationDate="2022-09-14T08:28:33.883" Score="2" ViewCount="23" Body="&lt;p&gt;When analysing a single-cell RNAseq dataset, one question I like to ask is: &lt;strong&gt;At a single-cell level, are these two genes correlated (expressed together) or anticorrelated (only one or the other is expressed)&lt;/strong&gt;? However, I am concerned that - given the low read count per cell - it may falsely appear that two genes are anticorrelated when in fact it is just unlikely that both RNAs are detected at the same time.&lt;/p&gt;&#xA;&lt;p&gt;This problem can partially be dealt with using an imputation technique such as SAVER, to estimate the true relative expression values of undetected genes. Additionally, a statistical test (e.g. permutation) can be performed to ask whether the incidence of cells with both transcripts detected is higher or lower than would be expected, given the total number of trancripts.&lt;/p&gt;&#xA;&lt;p&gt;My concern, however, is that there may be additional &lt;strong&gt;technical aspects of the sequencing process that will exaggerate the apparent anticorrelation between genes&lt;/strong&gt;. For example, in a cell where the total number of RNA transcripts at any one time is low, a single transcript may be read multiple times (due to multiple primers binding, resulting in multiple cDNAs).&lt;/p&gt;&#xA;&lt;p&gt;In addition to this, are there any other technical or analytical aspects of ssRNAdeq likely to lead to false anti-correlations?&lt;/p&gt;&#xA;" OwnerUserId="16133" LastActivityDate="2022-09-14T08:28:33.883" Title="Avoiding false anticorrelations between individual genes per cell in single-cell RNAseq" Tags="|rna-seq|scrnaseq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19689" PostTypeId="1" CreationDate="2022-09-14T10:59:51.910" Score="0" ViewCount="59" Body="&lt;p&gt;Let us say that we have some data frames named S1,S2,S3... that I want to join independently by column names to dataframe X. This is how they look like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;S1 &amp;lt;- data.frame(&#xA;  values = c (1:5), &#xA;  numbers = c(11:15))&#xA;S1&#xA;&#xA;&#xA;S2 &amp;lt;- data.frame(&#xA;  values = c (1:5), &#xA;  numbers = c(11:15))&#xA;S2&#xA;&#xA;&#xA;S3 &amp;lt;- data.frame(&#xA;  values = c (1:5), &#xA;  numbers = c(16:20))&#xA;S3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;#Dataframe X&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;X &amp;lt;- data.frame(&#xA;  values = c (26:30), &#xA;  numbers = c(501:505))&#xA;X&#xA;&#xA;S1&#xA;S2&#xA;S3&#xA;X&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Using rbind, it is really easy to merge/join them according to column names. As below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;S1_X&amp;lt;- rbind(S1,X)&#xA;&#xA;S1_X&#xA;&#xA;&#xA;S2_X&amp;lt;- rbind(S2,X)&#xA;&#xA;S2_X&#xA;&#xA;&#xA;S3_X&amp;lt;- rbind(S3,X)&#xA;&#xA;S3_X&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now the problem is that I have 500 plus data frames (S1,S2,S3,S4,S5.....S500) that I want to join to dataframe &amp;quot;X&amp;quot; by any function. For example rbind. Is it possible to use some loop/function for that.&lt;/p&gt;&#xA;&lt;p&gt;What I have best tried is create a list (nested list of dataframes) that I can later split into individual dataframes:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;output = lapply(list(S1=S1,S2=S2,S3=S3),rbind,X)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output is what I expected. So it works The only problem is, how do I specify S1 to S500?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="16114" LastActivityDate="2022-09-21T08:49:18.753" Title="Merge/Join one data frame (X) to multiple data frames separately" Tags="|r|dataframe|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19690" PostTypeId="2" ParentId="18226" CreationDate="2022-09-14T12:46:26.713" Score="1" Body="&lt;p&gt;You need to set your Java version to Java 8.&lt;/p&gt;&#xA;&lt;p&gt;If you are using conda, run this in your conda environment:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install openjdk=8&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16136" LastActivityDate="2022-09-14T12:46:26.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19691" PostTypeId="1" AcceptedAnswerId="19700" CreationDate="2022-09-14T14:14:54.103" Score="1" ViewCount="92" Body="&lt;p&gt;I have downloaded the Gencode PolyA feature annotation GTF &lt;a href=&quot;https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_41/gencode.v41.polyAs.gtf.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The first 10 lines are like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##description: evidence-based annotation of the human genome (GRCh38), version 41 (Ensembl 107) - polyA features&#xA;##provider: GENCODE&#xA;##contact: gencode-help@ebi.ac.uk&#xA;##format: gtf&#xA;##date: 2022-05-12&#xA;chr1    HAVANA  pseudo_polyA    134957  134962  .   -   .   gene_id &amp;quot;383460&amp;quot;; transcript_id &amp;quot;383460&amp;quot;; gene_type &amp;quot;pseudo_polya&amp;quot;; gene_name &amp;quot;383460&amp;quot;; transcript_type &amp;quot;pseudo_polya&amp;quot;; transcript_name &amp;quot;383460&amp;quot;; level 2;&#xA;chr1    HAVANA  polyA_site  141474  141475  .   -   .   gene_id &amp;quot;383461&amp;quot;; transcript_id &amp;quot;383461&amp;quot;; gene_type &amp;quot;polya_site&amp;quot;; gene_name &amp;quot;383461&amp;quot;; transcript_type &amp;quot;polya_site&amp;quot;; transcript_name &amp;quot;383461&amp;quot;; level 2;&#xA;chr1    HAVANA  polyA_signal    141477  141482  .   -   .   gene_id &amp;quot;383462&amp;quot;; transcript_id &amp;quot;383462&amp;quot;; gene_type &amp;quot;polya_signal&amp;quot;; gene_name &amp;quot;383462&amp;quot;; transcript_type &amp;quot;polya_signal&amp;quot;; transcript_name &amp;quot;383462&amp;quot;; level 2;&#xA;chr1    HAVANA  pseudo_polyA    184934  184939  .   -   .   gene_id &amp;quot;383463&amp;quot;; transcript_id &amp;quot;383463&amp;quot;; gene_type &amp;quot;pseudo_polya&amp;quot;; gene_name &amp;quot;383463&amp;quot;; transcript_type &amp;quot;pseudo_polya&amp;quot;; transcript_name &amp;quot;383463&amp;quot;; level 2;&#xA;chr1    HAVANA  pseudo_polyA    258530  258535  .   -   .   gene_id &amp;quot;383464&amp;quot;; transcript_id &amp;quot;383464&amp;quot;; gene_type &amp;quot;pseudo_polya&amp;quot;; gene_name &amp;quot;383464&amp;quot;; transcript_type &amp;quot;pseudo_polya&amp;quot;; transcript_name &amp;quot;383464&amp;quot;; level 2;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In this GTF file, what does the &lt;code&gt;gene_id&lt;/code&gt; mean? In the whole genome &lt;a href=&quot;https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_41/gencode.v41.annotation.gtf.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;GTF&lt;/a&gt; (same build) the &lt;code&gt;gene_id&lt;/code&gt; is clearly the ENSEMBL gene_id:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##description: evidence-based annotation of the human genome (GRCh38), version 41 (Ensembl 107)&#xA;##provider: GENCODE&#xA;##contact: gencode-help@ebi.ac.uk&#xA;##format: gtf&#xA;##date: 2022-05-12&#xA;chr1    HAVANA  gene    11869   14409   .   +   .   gene_id &amp;quot;ENSG00000223972.5&amp;quot;; gene_type &amp;quot;transcribed_unprocessed_pseudogene&amp;quot;; gene_name &amp;quot;DDX11L1&amp;quot;; level 2; hgnc_id &amp;quot;HGNC:37102&amp;quot;; havana_gene &amp;quot;OTTHUMG00000000961.2&amp;quot;;&#xA;chr1    HAVANA  transcript  11869   14409   .   +   .   gene_id &amp;quot;ENSG00000223972.5&amp;quot;; transcript_id &amp;quot;ENST00000456328.2&amp;quot;; gene_type &amp;quot;transcribed_unprocessed_pseudogene&amp;quot;; gene_name &amp;quot;DDX11L1&amp;quot;; transcript_type &amp;quot;processed_transcript&amp;quot;; transcript_name &amp;quot;DDX11L1-202&amp;quot;; level 2; transcript_support_level &amp;quot;1&amp;quot;; hgnc_id &amp;quot;HGNC:37102&amp;quot;; tag &amp;quot;basic&amp;quot;; havana_gene &amp;quot;OTTHUMG00000000961.2&amp;quot;; havana_transcript &amp;quot;OTTHUMT00000362751.1&amp;quot;;&#xA;chr1    HAVANA  exon    11869   12227   .   +   .   gene_id &amp;quot;ENSG00000223972.5&amp;quot;; transcript_id &amp;quot;ENST00000456328.2&amp;quot;; gene_type &amp;quot;transcribed_unprocessed_pseudogene&amp;quot;; gene_name &amp;quot;DDX11L1&amp;quot;; transcript_type &amp;quot;processed_transcript&amp;quot;; transcript_name &amp;quot;DDX11L1-202&amp;quot;; exon_number 1; exon_id &amp;quot;ENSE00002234944.1&amp;quot;; level 2; transcript_support_level &amp;quot;1&amp;quot;; hgnc_id &amp;quot;HGNC:37102&amp;quot;; tag &amp;quot;basic&amp;quot;; havana_gene &amp;quot;OTTHUMG00000000961.2&amp;quot;; havana_transcript &amp;quot;OTTHUMT00000362751.1&amp;quot;;&#xA;chr1    HAVANA  exon    12613   12721   .   +   .   gene_id &amp;quot;ENSG00000223972.5&amp;quot;; transcript_id &amp;quot;ENST00000456328.2&amp;quot;; gene_type &amp;quot;transcribed_unprocessed_pseudogene&amp;quot;; gene_name &amp;quot;DDX11L1&amp;quot;; transcript_type &amp;quot;processed_transcript&amp;quot;; transcript_name &amp;quot;DDX11L1-202&amp;quot;; exon_number 2; exon_id &amp;quot;ENSE00003582793.1&amp;quot;; level 2; transcript_support_level &amp;quot;1&amp;quot;; hgnc_id &amp;quot;HGNC:37102&amp;quot;; tag &amp;quot;basic&amp;quot;; havana_gene &amp;quot;OTTHUMG00000000961.2&amp;quot;; havana_transcript &amp;quot;OTTHUMT00000362751.1&amp;quot;;&#xA;chr1    HAVANA  exon    13221   14409   .   +   .   gene_id &amp;quot;ENSG00000223972.5&amp;quot;; transcript_id &amp;quot;ENST00000456328.2&amp;quot;; gene_type &amp;quot;transcribed_unprocessed_pseudogene&amp;quot;; gene_name &amp;quot;DDX11L1&amp;quot;; transcript_type &amp;quot;processed_transcript&amp;quot;; transcript_name &amp;quot;DDX11L1-202&amp;quot;; exon_number 3; exon_id &amp;quot;ENSE00002312635.1&amp;quot;; level 2; transcript_support_level &amp;quot;1&amp;quot;; hgnc_id &amp;quot;HGNC:37102&amp;quot;; tag &amp;quot;basic&amp;quot;; havana_gene &amp;quot;OTTHUMG00000000961.2&amp;quot;; havana_transcript &amp;quot;OTTHUMT00000362751.1&amp;quot;;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What I want to do eventually is to assign the PolyA sites to ENSEMBL gene_ids.&lt;/p&gt;&#xA;" OwnerUserId="10386" LastEditorUserId="10386" LastEditDate="2022-09-15T12:47:19.270" LastActivityDate="2022-09-16T10:24:32.410" Title="Gencode PolyA feature annotation GTF gene_id not the same as Comprehensive gene annotation GTF" Tags="|sequence-annotation|ensembl|gtf|gencode|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19692" PostTypeId="2" ParentId="19686" CreationDate="2022-09-14T14:47:24.783" Score="0" Body="&lt;p&gt;I've not done this precise calculation. I would use point 3, but its complicated to implement.&lt;/p&gt;&#xA;&lt;p&gt;Just for information BLOSUM62 is a static matrix 20x20 representing relative amino acid mutations. However, I am not exactly sure how the matrix was calculated.  The algorithm looks quite simple, then a BLOSUM62 score can be obtained by comparing the observed matrix to the static matrix, this would provide a score. The paper is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC50453/pdf/pnas01096-0363.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Maximum likelihood approaches require a phylogeny.&lt;/p&gt;&#xA;&lt;p&gt;There are three possibilities I can think of, the first one wouldn't work under a maximum likelihood approach:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Remove all hydrophobic residues from the alignment, check the BLOSUM62 score, remove all hydrophilic check the score etc... again a maximum likelihood approach is not meaningful in this example, but a BLOSUM62 score might be okay;&lt;/li&gt;&#xA;&lt;li&gt;Recode the amino acids to generate 6-state amino acid data, or recode hydrophobics and in a separate data set hydrophilics and leave the rest uncoded. I've never done it but at a guess &lt;code&gt;I, L, V&lt;/code&gt; could be represented by just one amino acid, e.g. &lt;code&gt;L&lt;/code&gt;. If the observed BLOSUM62 matrix algorithm was calculated on the observed data the new amino acid e.g. &lt;code&gt;L&lt;/code&gt; would have an &lt;code&gt;average&lt;/code&gt; score.&lt;/li&gt;&#xA;&lt;li&gt;Alternatively, manipulate the BLOSUM62 matrix, however its no longer a BLOSUM62 matrix, but it is highly amenable to maximum likelihood analysis. Thus I &amp;lt;-&amp;gt; V and I &amp;lt;-&amp;gt; L is set to zero (or massively negatively weight them) check likelihood, then hydrophilics are set to zero check likelihood. Do the same with the observed versus BLOSUM62 matrix as the control.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Point 3 what a maximum likelihood approach would do is say how close a given matrix is to observed data set for a given tree under a likelihood probability, the smallest is the best. So if you've got loads of matrices its a really cool approach. There is a membrane protein matrix - which is likely heavily skewed towards hydrophilicity. Basically, I'd test all the matrices - but not recode or delete the data set in any way. The matrix that is closer to the data set using likelihood is the best representation of its hydrophilicity vs. hydrophobicity.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; Just finally mention you can simply calculate the observed amino acid matrix for that data set (via a program) and then average the amino acids. That might be easier and using parsimony its very quick but you'd need a tree. The underlying metric (or metric behind the metric) in all cases is mutations.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-09-14T15:52:10.090" LastActivityDate="2022-09-14T15:52:10.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19693" PostTypeId="1" CreationDate="2022-09-14T21:37:08.850" Score="4" ViewCount="131" Body="&lt;p&gt;I have a bam file that I want to run through GATK's SplitNCigarReads tool. Because of the way the bam file was generated, the program will often fail, with an error message stating:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;code&gt;##### ERROR MESSAGE: Bad input: Cannot split this read (might be an empty section between Ns, for example 1N1D1N): ABCDEFG&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;My question is: how do I write a bash script to:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Start GATK on the input bam file.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When (and if) the error message arises, take the bad read (e.g., ABCDEFG) and use samtools to remove it,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h input.bam | grep -v ABCDEFG | samtools view -bS -o intermediate1.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Index the new bam file&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Run the same GATK command as #1, but with the newly generated bam file&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Keep performing steps 1-4 until the GATK tool runs to completion&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I realize this may be quite complicated and/or involved, so I don't expect someone to write the entire script for me. Are there any online resources dedicated to writing a script like this?&lt;/p&gt;&#xA;&lt;p&gt;P.S. This is my first StackExchange question, please let me know if I did anything incorrectly or inefficiently.&lt;/p&gt;&#xA;" OwnerUserId="16139" LastEditorUserId="29" LastEditDate="2022-09-16T13:44:39.980" LastActivityDate="2022-09-18T15:25:23.350" Title="Write a bash script to run gatk, fix errors with input, and rerun until completion" Tags="|samtools|bash|gatk|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19694" PostTypeId="2" ParentId="19684" CreationDate="2022-09-14T21:49:42.547" Score="1" Body="&lt;p&gt;I'm aware that this asks specifically for a python solution, but I'd like to add an R answer as well, just in case someone using R has a similar problem:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; a &amp;lt;- c(&amp;quot;house&amp;quot;,&amp;quot;garden&amp;quot;, &amp;quot;living room&amp;quot;, &amp;quot;dog&amp;quot;,&amp;quot;cat&amp;quot;)&#xA;&amp;gt; b &amp;lt;- c(&amp;quot;cat&amp;quot;,&amp;quot;dog&amp;quot;, &amp;quot;chicken&amp;quot;)&#xA;&amp;gt; c &amp;lt;- c(&amp;quot;house&amp;quot;, &amp;quot;garden&amp;quot;,&amp;quot;bathroom&amp;quot;)&#xA;&#xA;&amp;gt; (result &amp;lt;- data.frame(row.names=a, a=a %in% a, b=a %in% b, c=a %in% c))&#xA;               a     b     c&#xA;house       TRUE FALSE  TRUE&#xA;garden      TRUE FALSE  TRUE&#xA;living room TRUE FALSE FALSE&#xA;dog         TRUE  TRUE FALSE&#xA;cat         TRUE  TRUE FALSE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or for strictly what you asked for, the TRUE/FALSE values can be encouraged into numbers by adding zero:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; (result &amp;lt;- data.frame(a=a, b=(a %in% b) + 0, c=(a %in% c) + 0))&#xA;            a b c&#xA;1       house 0 1&#xA;2      garden 0 1&#xA;3 living room 0 0&#xA;4         dog 1 0&#xA;5         cat 1 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-09-14T21:49:42.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19695" PostTypeId="2" ParentId="13395" CreationDate="2022-09-14T22:58:09.957" Score="1" Body="&lt;p&gt;I don't have enough reputation for a comment but the place where the option to add scoresums is listed is here &lt;a href=&quot;https://www.cog-genomics.org/plink/2.0/formats#sscore&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cog-genomics.org/plink/2.0/formats#sscore&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16140" LastActivityDate="2022-09-14T22:58:09.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19697" PostTypeId="1" CreationDate="2022-09-15T07:34:20.177" Score="0" ViewCount="47" Body="&lt;p&gt;I have a list of genes (about 2000) from lines of &lt;strong&gt;drosophila subobscura&lt;/strong&gt;, in the following format:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;LOC117900589&lt;/li&gt;&#xA;&lt;li&gt;LOC11788959&lt;/li&gt;&#xA;&lt;li&gt;LOC11790331&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I'm trying to represent this set of genes in a useful way.&lt;/p&gt;&#xA;&lt;p&gt;So, I was recommended to represent the ontology function of these genes in a graphical way.&#xA;I was recommended this tool:&#xA;&lt;a href=&quot;http://cbl-gorilla.cs.technion.ac.il/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cbl-gorilla.cs.technion.ac.il/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;One inputs a list of genes and it outputs this kind of graph:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/cQZXm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cQZXm.png&quot; alt=&quot;GOrilla example output&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The problem is that, this tool, GOrilla only accepts genes from &lt;strong&gt;drosophila melanogaster&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I have to either:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;a)&lt;/strong&gt; convert the genes id to a homologous gene in &lt;strong&gt;melanogaster&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;b)&lt;/strong&gt; find an alternative tool&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;A solution to either a) or b) would be helpful. Or an alternative solution.&lt;/p&gt;&#xA;&lt;p&gt;For a) I tried:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Ncbi's &lt;strong&gt;homologene&lt;/strong&gt;: which doesn't seem to be working for a the genes ID I tried so far (I didn't try them all)&lt;/li&gt;&#xA;&lt;li&gt;Blasting the gene sequence on melanogaster's genome and looking through the list manually to find an homologous gene.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So far these methods have been impractical for the number of genes I have.&lt;/p&gt;&#xA;" OwnerUserId="9373" LastActivityDate="2022-09-21T06:50:23.553" Title="Building a group representation of genes' ontologies" Tags="|gene|gene-ontology|ontology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19698" PostTypeId="1" CreationDate="2022-09-15T16:38:40.077" Score="3" ViewCount="555" Body="&lt;p&gt;I am trying to align a yeast strain sequenced by PacBio HiFi reads to the reference genome S288C (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/data-hub/genome/GCF_000146045.2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/data-hub/genome/GCF_000146045.2/&lt;/a&gt;) using pbmm2 but for some reason I am getting an error that I don't understand.&lt;/p&gt;&#xA;&lt;p&gt;I first indexed the reference sequence with the following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pbmm2 index S288Cseq.fa S288Cindx.mmi&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And then I attempted to align using this line of code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pbmm2 align S288Cseq.mmi 088.fastq.gz mapped088.aligned.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I keep getting this error:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;ERROR: unsupported extension on input file: S288Cindx.mmi&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I get that error no matter if I use the reference file in either mmi or the fasta format. Can someone please help and let me know what I'm doing wrong? (I downloaded the tool using bioconda: &lt;code&gt;conda install -c bioconda pbmm2&lt;/code&gt; just in case that helps)&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="3848" LastEditDate="2022-09-15T18:23:24.327" LastActivityDate="2022-09-15T18:23:24.327" Title="Aligning PacBio HiFi reads to reference genome using pbmm2" Tags="|sequence-alignment|reference-genome|pacbio|errors|bioconda|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19699" PostTypeId="1" AcceptedAnswerId="19704" CreationDate="2022-09-16T04:12:12.787" Score="2" ViewCount="108" Body="&lt;p&gt;My question is related to the output of Tombo re-squiggling algorithm on nanopore data.&#xA;Given the position/coordinate on one strand (+ or -), I want to find the coordinate of its exact location on the opposite strand.&#xA;For example, let's say I want to find data/signal on specific coordinates on both strands on Tombo path after re-squiggling. Let's say position 13936 - 14036 on genome on + and - negative strands. I wrote the following code but the sequence obtained from Tombo is not similar at all. I want to know how I can find the coordinate corresponding to a given position on the opposite strand.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# one read from the + strand&#xA;mapped_start, mapped_end, read_id, fast5_path, _, _ = reads_per_win_pos.loc[rd_id, :]&#xA;&amp;gt;&amp;gt;&amp;gt; start&#xA;13936&#xA;&amp;gt;&amp;gt;&amp;gt; end&#xA;13942&#xA;&amp;gt;&amp;gt;&amp;gt; mapped_start, mapped_end, read_id, fast5_path, _, _ = reads_per_win_pos.loc[rd_id, :]&#xA;&amp;gt;&amp;gt;&amp;gt; mapped_start&#xA;11205&#xA;&amp;gt;&amp;gt;&amp;gt; mapped_end&#xA;20084&#xA;&amp;gt;&amp;gt;&amp;gt; f = h5py.File(fast5_path, 'r')&#xA;&amp;gt;&amp;gt;&amp;gt; tombo_events = list(f['Analyses/RawGenomeCorrected_000/BaseCalled_template/Events'])&#xA;&amp;gt;&amp;gt;&amp;gt; forward_seq = ''.join([e[4].decode() for e in tombo_events])&#xA;&amp;gt;&amp;gt;&amp;gt; start_idx = start - mapped_start&#xA;&amp;gt;&amp;gt;&amp;gt; start_idx&#xA;2731&#xA;&amp;gt;&amp;gt;&amp;gt; win_seq = seq[start_idx: start_idx + 100]&#xA;&amp;gt;&amp;gt;&amp;gt; forward_win_seq = forward_seq[start_idx: start_idx + 100]&#xA;&amp;gt;&amp;gt;&amp;gt; # one other overlapping read on the other strand (-)&#xA;...&#xA;&amp;gt;&amp;gt;&amp;gt; mapped_start, mapped_end, read_id, fast5_path, _, _ = reads_per_win_neg.loc[rd_id, :]&#xA;&amp;gt;&amp;gt;&amp;gt; mapped_start&#xA;10979&#xA;&amp;gt;&amp;gt;&amp;gt; mapped_end&#xA;20091&#xA;&amp;gt;&amp;gt;&amp;gt; f = h5py.File(fast5_path, 'r')&#xA;&amp;gt;&amp;gt;&amp;gt; tombo_events = list(f['Analyses/RawGenomeCorrected_000/BaseCalled_template/Events'])&#xA;&amp;gt;&amp;gt;&amp;gt; reverse_seq = ''.join([e[4].decode() for e in tombo_events])&#xA;&amp;gt;&amp;gt;&amp;gt; start_idx = start - mapped_start&#xA;&amp;gt;&amp;gt;&amp;gt; start_idx&#xA;2957&#xA;&amp;gt;&amp;gt;&amp;gt; reverse_win_seq = reverse_seq[start_idx: start_idx + 100]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now the forward_win_seq and reverse (or even the reverse complement) of it are not similar:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; forward_win_seq&#xA;'GAGACGGAGCAGACCCATCTGCTACTGCCCTTTCTATAATAACTAAAGTTAGCTGCCCTGGACTATTCACCCCCTAGTCTCAATTTAAAAAGATCCCCAT'&#xA;&amp;gt;&amp;gt;&amp;gt; reverse_win_seq&#xA;'GGGCCAGGTGCCTGAGATCCATGTTCCATCCTACCTGCCTGACCTGCCCGGCATTGCCAACGACCTCATGTACAGTGCCGACCTGGGCCCCGGCATTGCC'&#xA;&amp;gt;&amp;gt;&amp;gt; complement = {'A': 'T', 'C': 'G', 'G': 'C', 'T': 'A'}&#xA;&amp;gt;&amp;gt;&amp;gt; com_seq = ''.join([complement[base] if base in complement.keys() else base for base in reverse_win_seq])&#xA;&amp;gt;&amp;gt;&amp;gt; com_seq[::-1]&#xA;'GGCAATGCCGGGGCCCAGGTCGGCACTGTACATGAGGTCGTTGGCAATGCCGGGCAGGTCAGGCAGGTAGGATGGAACATGGATCTCAGGCACCTGGCCC'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15341" LastActivityDate="2022-09-17T13:43:01.283" Title="Coordinate numbering between forward (+) and reverse (-) strand in Tombo" Tags="|nanopore|read-mapping|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19700" PostTypeId="2" ParentId="19691" CreationDate="2022-09-16T10:24:32.410" Score="2" Body="&lt;p&gt;According to the GENCODE ftp &lt;a href=&quot;https://ftp.ebi.ac.uk/pub/databases/gencode/_README.TXT&quot; rel=&quot;nofollow noreferrer&quot;&gt;README&lt;/a&gt; file, the value of the 'gene_id', 'transcript_id', 'gene_name' and 'transcript name' fields corresponds to a random identifier. The polyA features are not directly associated to any gene or transcript when manually annotated by Havana.&lt;/p&gt;&#xA;&lt;p&gt;There is other file in the GENCODE release that includes links between polyA features and transcripts:&#xA;&lt;a href=&quot;https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_41/gencode.v41.metadata.PolyA_feature.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_41/gencode.v41.metadata.PolyA_feature.gz&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&#xA;ENST00000485748.5       3232    3233    chr1    1496201 1496202 +       polya_site&#xA;ENST00000485748.5       3211    3216    chr1    1496180 1496185 +       polya_signal&#xA;ENST00000474481.1       3367    3368    chr1    1496201 1496202 +       polya_site&#xA;ENST00000474481.1       3346    3351    chr1    1496180 1496185 +       polya_signal&#xA;ENST00000308647.8       2114    2115    chr1    1496201 1496202 +       polya_site&#xA;ENST00000308647.8       2093    2098    chr1    1496180 1496185 +       polya_signal&#xA;&lt;/pre&gt;&#xA;&lt;pre&gt;&#xA;22. gencode.vX.metadata.PolyA_feature.gz:&#xA;  Manually annotated polyA feature overlapping the transcript 3'-end.&#xA;  1 - transcript id&#xA;  2 - transcript-based start coordinate of the polyA feature&#xA;  3 - transcript-based end coordinate of the polyA feature&#xA;  4 - polyA feature chromosome&#xA;  5 - polyA feature start coordinate&#xA;  6 - polyA feature end coordinate&#xA;  7 - polyA feature strand&#xA;  8 - polyA feature type (&quot;polyA_site&quot;, &quot;polyA_signal&quot;, &quot;pseudo_polyA&quot;)&#xA;&lt;/pre&gt;&#xA;" OwnerUserId="16146" LastActivityDate="2022-09-16T10:24:32.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19701" PostTypeId="1" CreationDate="2022-09-16T14:29:43.397" Score="1" ViewCount="38" Body="&lt;p&gt;This is a very naive question - I am trying to compare and get the common lncRNA genes and transcripts between Gencode and Refseq from their gff files.&lt;/p&gt;&#xA;&lt;p&gt;Since their gene_id, gene_names are different, is there a metric that I can use to compare to see if they share similar genes?&#xA;Or a conversion table that would convert one gene/transcript name format into another...&lt;/p&gt;&#xA;&lt;p&gt;Thank you so much.&lt;/p&gt;&#xA;&lt;p&gt;gencode.gff3&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1    HAVANA  gene    29554   31109   .   +   .   ID=ENSG00000243485.5;gene_id=ENSG00000243485.5;gene_type=lncRNA;gene_name=MIR1302-2HG;level=2;hgnc_id=HGNC:52482;tag=ncRNA_host;havana_gene=OTTHUMG00000000959.2&#xA;chr1    HAVANA  transcript  29554   31097   .   +   .   ID=ENST00000473358.1;Parent=ENSG00000243485.5;gene_id=ENSG00000243485.5;transcript_id=ENST00000473358.1;gene_type=lncRNA;gene_name=MIR1302-2HG;transcript_type=lncRNA;transcript_name=MIR1302-2HG-202;level=2;transcript_support_level=5;hgnc_id=HGNC:52482;tag=not_best_in_genome_evidence,dotter_confirmed,basic,Ensembl_canonical;havana_gene=OTTHUMG00000000959.2;havana_transcript=OTTHUMT00000002840.1&#xA;chr1    HAVANA  exon    29554   30039   .   +   .   ID=exon:ENST00000473358.1:1;Parent=ENST00000473358.1;gene_id=ENSG00000243485.5;transcript_id=ENST00000473358.1;gene_type=lncRNA;gene_name=MIR1302-2HG;transcript_type=lncRNA;transcript_name=MIR1302-2HG-202;exon_number=1;exon_id=ENSE00001947070.1;level=2;transcript_support_level=5;hgnc_id=HGNC:52482;tag=not_best_in_genome_evidence,dotter_confirmed,basic,Ensembl_canonical;havana_gene=OTTHUMG00000000959.2;havana_transcript=OTTHUMT00000002840.1&#xA;chr1    HAVANA  exon    30564   30667   .   +   .   ID=exon:ENST00000473358.1:2;Parent=ENST00000473358.1;gene_id=ENSG00000243485.5;transcript_id=ENST00000473358.1;gene_type=lncRNA;gene_name=MIR1302-2HG;transcript_type=lncRNA;transcript_name=MIR1302-2HG-202;exon_number=2;exon_id=ENSE00001922571.1;level=2;transcript_support_level=5;hgnc_id=HGNC:52482;tag=not_best_in_genome_evidence,dotter_confirmed,basic,Ensembl_canonical;havana_gene=OTTHUMG00000000959.2;havana_transcript=OTTHUMT00000002840.1&#xA;chr1    HAVANA  exon    30976   31097   .   +   .   ID=exon:ENST00000473358.1:3;Parent=ENST00000473358.1;gene_id=ENSG00000243485.5;transcript_id=ENST00000473358.1;gene_type=lncRNA;gene_name=MIR1302-2HG;transcript_type=lncRNA;transcript_name=MIR1302-2HG-202;exon_number=3;exon_id=ENSE00001827679.1;level=2;transcript_support_level=5;hgnc_id=HGNC:52482;tag=not_best_in_genome_evidence,dotter_confirmed,basic,Ensembl_canonical;havana_gene=OTTHUMG00000000959.2;havana_transcript=OTTHUMT00000002840.1&#xA;chr1    HAVANA  transcript  30267   31109   .   +   .   ID=ENST00000469289.1;Parent=ENSG00000243485.5;gene_id=ENSG00000243485.5;transcript_id=ENST00000469289.1;gene_type=lncRNA;gene_name=MIR1302-2HG;transcript_type=lncRNA;transcript_name=MIR1302-2HG-201;level=2;transcript_support_level=5;hgnc_id=HGNC:52482;tag=not_best_in_genome_evidence,basic;havana_gene=OTTHUMG00000000959.2;havana_transcript=OTTHUMT00000002841.2&#xA;chr1    HAVANA  exon    30267   30667   .   +   .   ID=exon:ENST00000469289.1:1;Parent=ENST00000469289.1;gene_id=ENSG00000243485.5;transcript_id=ENST00000469289.1;gene_type=lncRNA;gene_name=MIR1302-2HG;transcript_type=lncRNA;transcript_name=MIR1302-2HG-201;exon_number=1;exon_id=ENSE00001841699.1;level=2;transcript_support_level=5;hgnc_id=HGNC:52482;tag=not_best_in_genome_evidence,basic;havana_gene=OTTHUMG00000000959.2;havana_transcript=OTTHUMT00000002841.2&#xA;chr1    HAVANA  exon    30976   31109   .   +   .   ID=exon:ENST00000469289.1:2;Parent=ENST00000469289.1;gene_id=ENSG00000243485.5;transcript_id=ENST00000469289.1;gene_type=lncRNA;gene_name=MIR1302-2HG;transcript_type=lncRNA;transcript_name=MIR1302-2HG-201;exon_number=2;exon_id=ENSE00001890064.1;level=2;transcript_support_level=5;hgnc_id=HGNC:52482;tag=not_best_in_genome_evidence,basic;havana_gene=OTTHUMG00000000959.2;havana_transcript=OTTHUMT00000002841.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;refseq.gff&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1    RefSeq  region  1       248956422       .       +       .       ID=NC_000001.11:1..248956422;Dbxref=taxon:9606;Name=1;chromosome=1;gbkey=Src;genome=chromosome;mol_type=genomic DNA&#xA;chr1    Gnomon  lnc_RNA 29774   35418   .   +   .   ID=rna-XR_007065314.1;Parent=gene-MIR1302-2HG;Dbxref=GeneID:107985730,Genbank:XR_007065314.1,HGNC:HGNC:52482;Name=XR_007065314.1;gbkey=ncRNA;gene=MIR1302-2HG;model_evidence=Supporting evidence includes similarity to: 1 mRNA%2C 5 ESTs%2C 10 long SRA reads%2C and 96%25 coverage of the annotated genomic feature by RNAseq alignments%2C including 4 samples with support for all annotated introns;product=MIR1302-2 host gene;transcript_id=XR_007065314.1&#xA;chr1    BestRefSeq  lnc_RNA 34611   36081   .   -   .   ID=rna-NR_026818.1;Parent=gene-FAM138A;Dbxref=GeneID:645520,Genbank:NR_026818.1,HGNC:HGNC:32334;Name=NR_026818.1;gbkey=ncRNA;gene=FAM138A;product=family with sequence similarity 138 member A;transcript_id=NR_026818.1&#xA;chr1    Gnomon  lnc_RNA 95222   107727  .   +   .   ID=rna-XR_007065337.1;Parent=gene-LOC124903816;Dbxref=GeneID:124903816,Genbank:XR_007065337.1;Name=XR_007065337.1;experiment=COORDINATES: polyA evidence [ECO:0006239];gbkey=ncRNA;gene=LOC124903816;model_evidence=Supporting evidence includes similarity to: 1 EST%2C 8 long SRA reads%2C and 84%25 coverage of the annotated genomic feature by RNAseq alignments%2C including 2 samples with support for all annotated introns;product=uncharacterized LOC124903816%2C transcript variant X1;transcript_id=XR_007065337.1&#xA;chr1    Gnomon  lnc_RNA 95222   107727  .   +   .   ID=rna-XR_007065338.1;Parent=gene-LOC124903816;Dbxref=GeneID:124903816,Genbank:XR_007065338.1;Name=XR_007065338.1;experiment=COORDINATES: polyA evidence [ECO:0006239];gbkey=ncRNA;gene=LOC124903816;model_evidence=Supporting evidence includes similarity to: 1 EST%2C 8 long SRA reads%2C and 84%25 coverage of the annotated genomic feature by RNAseq alignments%2C including 2 samples with support for all annotated introns;product=uncharacterized LOC124903816%2C transcript variant X2;transcript_id=XR_007065338.1&#xA;chr1    Gnomon  lnc_RNA 120849  175246  .   -   .   ID=rna-XR_001737579.3;Parent=gene-LOC124900384;Dbxref=GeneID:124900384,Genbank:XR_001737579.3;Name=XR_001737579.3;gbkey=ncRNA;gene=LOC124900384;model_evidence=Supporting evidence includes similarity to: 5 long SRA reads%2C and 78%25 coverage of the annotated genomic feature by RNAseq alignments%2C including 34 samples with support for all annotated introns;product=uncharacterized LOC124900384%2C transcript variant X13;transcript_id=XR_001737579.3&#xA;chr1    Gnomon  lnc_RNA 146300  175246  .   -   .   ID=rna-XR_007065328.1;Parent=gene-LOC124900384;Dbxref=GeneID:124900384,Genbank:XR_007065328.1;Name=XR_007065328.1;gbkey=ncRNA;gene=LOC124900384;model_evidence=Supporting evidence includes similarity to: 4 ESTs%2C 24 long SRA reads%2C and 76%25 coverage of the annotated genomic feature by RNAseq alignments%2C including 19 samples with support for all annotated introns;product=uncharacterized LOC124900384%2C transcript variant X9;transcript_id=XR_007065328.1&#xA;chr1    Gnomon  lnc_RNA 146301  175246  .   -   .   ID=rna-XR_007065325.1;Parent=gene-LOC124900384;Dbxref=GeneID:124900384,Genbank:XR_007065325.1;Name=XR_007065325.1;gbkey=ncRNA;gene=LOC124900384;model_evidence=Supporting evidence includes similarity to: 4 ESTs%2C 25 long SRA reads%2C and 78%25 coverage of the annotated genomic feature by RNAseq alignments%2C including 14 samples with support for all annotated introns;product=uncharacterized LOC124900384%2C transcript variant X6;transcript_id=XR_007065325.1&#xA;chr1    Gnomon  lnc_RNA 146435  175246  .   -   .   ID=rna-XR_007065330.1;Parent=gene-LOC124900384;Dbxref=GeneID:124900384,Genbank:XR_007065330.1;Name=XR_007065330.1;gbkey=ncRNA;gene=LOC124900384;model_evidence=Supporting evidence includes similarity to: 1 EST%2C 25 long SRA reads%2C and 75%25 coverage of the annotated genomic feature by RNAseq alignments%2C including 58 samples with support for all annotated introns;product=uncharacterized LOC124900384%2C transcript variant X11;transcript_id=XR_007065330.1&#xA;chr1    Gnomon  lnc_RNA 146470  175246  .   -   .   ID=rna-XR_007065322.1;Parent=gene-LOC124900384;Dbxref=GeneID:124900384,Genbank:XR_007065322.1;Name=XR_007065322.1;gbkey=ncRNA;gene=LOC124900384;model_evidence=Supporting evidence includes similarity to: 4 ESTs%2C 24 long SRA reads%2C and 76%25 coverage of the annotated genomic feature by RNAseq alignments%2C including 43 samples with support for all annotated introns;product=uncharacterized LOC124900384%2C transcript variant X3;transcript_id=XR_007065322.1&#xA;chr1    Gnomon  lnc_RNA 149034  175246  .   -   .   ID=rna-XR_007065324.1;Parent=gene-LOC124900384;Dbxref=GeneID:124900384,Genbank:XR_007065324.1;Name=XR_007065324.1;gbkey=ncRNA;gene=LOC124900384;model_evidence=Supporting evidence includes similarity to: 4 ESTs%2C 25 long SRA reads%2C and 78%25 coverage of the annotated genomic feature by RNAseq alignments%2C including 17 samples with support for all annotated introns;product=uncharacterized LOC124900384%2C transcript variant X5;transcript_id=XR_007065324.1&#xA;chr1    Gnomon  lnc_RNA 149035  175246  .   -   .   ID=rna-XR_007065331.1;Parent=gene-LOC124900384;Dbxref=GeneID:124900384,Genbank:XR_007065331.1;Name=XR_007065331.1;gbkey=ncRNA;gene=LOC124900384;model_evidence=Supporting evidence includes similarity to: 1 EST%2C 24 long SRA reads%2C and 76%25 coverage of the annotated genomic feature by RNAseq alignments%2C including 38 samples with support for all annotated introns;product=uncharacterized LOC124900384%2C transcript variant X12;transcript_id=XR_007065331.1&#xA;chr1    Gnomon  lnc_RNA 149035  175246  .   -   .   ID=rna-XR_007065327.1;Parent=gene-LOC124900384;Dbxref=GeneID:124900384,Genbank:XR_007065327.1;Name=XR_007065327.1;gbkey=ncRNA;gene=LOC124900384;model_evidence=Supporting evidence includes similarity to: 4 ESTs%2C 24 long SRA reads%2C and 76%25 coverage of the annotated genomic feature by RNAseq alignments%2C including 33 samples with support for all annotated introns;product=uncharacterized LOC124900384%2C transcript variant X8;transcript_id=XR_007065327.1&#xA;chr1    Gnomon  lnc_RNA 151857  175246  .   -   .   ID=rna-XR_007065323.1;Parent=gene-LOC124900384;Dbxref=GeneID:124900384,Genbank:XR_007065323.1;Name=XR_007065323.1;gbkey=ncRNA;gene=LOC124900384;model_evidence=Supporting evidence includes similarity to: 4 ESTs%2C 25 long SRA reads%2C and 81%25 coverage of the annotated genomic feature by RNAseq alignments%2C including 5 samples with support for all annotated introns;product=uncharacterized LOC124900384%2C transcript variant X4;transcript_id=XR_007065323.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16148" LastEditorUserId="16148" LastEditDate="2022-09-16T14:31:55.313" LastActivityDate="2022-09-16T14:31:55.313" Title="Comparing common genes and transcripts between Gencode and Refseq" Tags="|gene|refseq|gencode|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19703" PostTypeId="1" CreationDate="2022-09-16T20:39:01.410" Score="2" ViewCount="122" Body="&lt;p&gt;I just ran an assembly on yeast genomes using Flye and I want to polish those assemblies with Pilon but it requires a sorted BAM file.&lt;/p&gt;&#xA;&lt;p&gt;How do I make a BAM file of the resulting assembled.fasta?&lt;/p&gt;&#xA;" OwnerUserId="16067" LastActivityDate="2022-12-05T16:54:30.013" Title="need bam file for pilon" Tags="|sam|sequence-analysis|assembly|conversion|genome-index|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19704" PostTypeId="2" ParentId="19699" CreationDate="2022-09-17T13:43:01.283" Score="2" Body="&lt;p&gt;When &lt;strong&gt;Tombo re-squiggle&lt;/strong&gt; and aligns a read it adds two attributes to it: &lt;strong&gt;mapped_start&lt;/strong&gt; and &lt;strong&gt;mapped_end&lt;/strong&gt;. Tombo also gives chromosomes and strands.&lt;/p&gt;&#xA;&lt;p&gt;Imagine you are looking for &lt;strong&gt;some value of a signal[s] in position x to y on a chromosome on both strands.&lt;/strong&gt; In this range, you probably have reads that span the positive strand (forward) and the reads that cover the negative strand (reverse).&lt;/p&gt;&#xA;&lt;p&gt;Now let a given read that has mapped to a positive strand and covers the entire [x, y], have attributes: &lt;strong&gt;pos_mapped_start&lt;/strong&gt; and &lt;strong&gt;pos_mapped_end&lt;/strong&gt;. Similarly let the reads mapped to negative strands covering [x, y] have attributes: &lt;strong&gt;neg_mapped_start&lt;/strong&gt; and &lt;strong&gt;neg_mapped_end&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For reads mapped to + strand:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;start_idx = x- pos_mapped_start&#xA;end_idx = y - pos_mapped_start + 1&#xA;required_signal= read_signal[start_idx: end_idx]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For reads mapped to - strand:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;start_idx = neg_mapped_end - y- 1&#xA;end_idx = neg_mapped_end - x&#xA;required_signal= read_signal[start_idx: end_idx]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Note that x and y are &lt;strong&gt;inclusive&lt;/strong&gt; and the length of the required signal = y - x + 1.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="15341" LastActivityDate="2022-09-17T13:43:01.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19705" PostTypeId="1" CreationDate="2022-09-18T05:46:27.427" Score="2" ViewCount="91" Body="&lt;p&gt;I am trying to do adapter trimming, alignment and sorting for a range of large scale  paired end fastq files. The code I am using is given below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;# Do Trimming, Alignment according to Config file&#xA;&#xA;#SBATCH --job-name=T_Align_single&#xA;#SBATCH --nodes=1&#xA;#SBATCH --ntasks=1&#xA;#SBATCH --cpus-per-task=32&#xA;#SBATCH --partition=batch&#xA;#SBATCH --account=markay&#xA;#SBATCH --time=72:00:00&#xA;#SBATCH --mem=16G&#xA;#SBATCH --mail-type=TIME_LIMIT_80,BEGIN,END&#xA;#SBATCH --mail-user=&amp;lt;email-redacted&amp;gt;&#xA;&#xA;module load bowtie2/2.4.1 &#xA;module load samtools/1.9&#xA;&#xA;mkdir -p Mapping/&#xA;&#xA;config=${INPUT_FILE}&#xA;&#xA;file1=&lt;span class=&quot;math-container&quot;&gt;$(cat $&lt;/span&gt;config | awk '&lt;span class=&quot;math-container&quot;&gt;$1 ~ /fastq/ {print $1}')&#xA;file2=$&lt;/span&gt;(cat &lt;span class=&quot;math-container&quot;&gt;$config | awk '$&lt;/span&gt;1 ~ /fastq/ {print &lt;span class=&quot;math-container&quot;&gt;$2}')&#xA;file3=$&lt;/span&gt;(cat &lt;span class=&quot;math-container&quot;&gt;$config | awk '$&lt;/span&gt;1 ~ /index=/ {print $1}'| sed 's/ref_index=//')&#xA;&#xA;&#xA;#adapter trimming&#xA;trimfile1=Mapping/&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${file1%.fastq}&quot;.trim.fastq&#xA;trimfile2=Mapping/&quot;$&lt;/span&gt;{file2%.fastq}&amp;quot;.trim.fastq&#xA;&#xA; java -jar /labs/markay/Aranyak/Adriana_intro_CnT/Cnt_Test/Test-analysis/fastq_processing/Trimmomatic-0.39/trimmomatic-0.39.jar  PE -phred33 &lt;span class=&quot;math-container&quot;&gt;$file1 $&lt;/span&gt;file2 \&#xA;   &lt;span class=&quot;math-container&quot;&gt;$trimfile1 Mapping/&quot;$&lt;/span&gt;{file1%.fastq}&amp;quot;.unpaired.fastq \&#xA;   &lt;span class=&quot;math-container&quot;&gt;$trimfile2 Mapping/&quot;$&lt;/span&gt;{file2%.fastq}&amp;quot;.unpaired.fastq \&#xA;   LEADING:3 TRAILING:3 SLIDINGWINDOW:4:15 \&#xA;   MINLEN:20 ILLUMINACLIP: /labs/markay/Aranyak/Adriana_intro_CnT/adapter.fa:2:30:10 -threads ${SLURM_CPUS_PER_TASK}&#xA;&#xA;# mapping using bowtie2&#xA;sortbam=Mapping/&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${file1%_R1_001.fastq}&quot;.sort.bam&#xA;sample_ID=$&lt;/span&gt;(cat &lt;span class=&quot;math-container&quot;&gt;$config | awk '$&lt;/span&gt;1 ~ /fastq/ {print &lt;span class=&quot;math-container&quot;&gt;$4}')&#xA;bowtie2 -p $&lt;/span&gt;{SLURM_CPUS_PER_TASK} --local --very-sensitive-local --no-unal \&#xA;   --no-mixed --no-discordant --phred33 -I 10 -X 700 \&#xA;   -x &lt;span class=&quot;math-container&quot;&gt;$file3 \&#xA;   -1 $&lt;/span&gt;trimfile1 -2 &lt;span class=&quot;math-container&quot;&gt;$trimfile2 --rg-id $&lt;/span&gt;sample_ID | samtools view -Sb - | samtools sort -@&lt;span class=&quot;math-container&quot;&gt;${SLURM_CPUS_PER_TASK} -o $&lt;/span&gt;sortbam&#xA;&#xA;#Index the sorting by coordinate bam file&#xA;samtools index ${sortbam}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My original fastq files before adapter trimming looked like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;R3-LiD-M-LK_S16_L002_R2_001.fastq.gz&#xA;R3-LiD-M-LK_S16_L002_R1_001.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However after adapter trimming I removed the lane names and my file looks like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;R3-LiD-M-LK_S16_R2_001.fastq.gz&#xA;R3-LiD-M-LK_S16_R1_001.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The code that worked for the original file is not generating a proper bam file at the end though I am getting a list of trimmed fastq files. The error message I am getting is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[E::hts_open_format] Failed to open file R1-LiD-H-DJ_S3.sort.rmdup.bam&#xA;samtools index: failed to open &amp;quot;R1-LiD-H-DJ_S3.sort.rmdup.bam&amp;quot;: No such file or directory&#xA;[E::hts_open_format] Failed to open file R1-LiD-H-DJ_S3.sort.bam&#xA;The file 'R1-LiD-H-DJ_S3.sort.bam' does not exist&#xA;[E::hts_open_format] Failed to open file R1-LiD-H-DJ_S3.sort.rmdup.bam&#xA;The file 'R1-LiD-H-DJ_S3.sort.rmdup.bam' does not exist&#xA;R1-LiD-H-DJ_S3 DONE.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How should I change the code to generate proper alignment and bam files?&lt;/p&gt;&#xA;" OwnerUserId="9659" LastEditorUserId="650" LastEditDate="2022-09-19T21:57:13.677" LastActivityDate="2022-09-19T21:57:13.677" Title="Adapter trimming" Tags="|ngs|genomics|rna|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19706" PostTypeId="2" ParentId="19703" CreationDate="2022-09-19T15:51:27.023" Score="2" Body="&lt;p&gt;Run a short-read mapper. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bwa index assembled.fasta&#xA;bwa mem -pt16 assembled.fasta read1.fq.gz read2.fq.gz \&#xA;  | samtools sort -m4G -@4 -o align.bam -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37" LastActivityDate="2022-09-19T15:51:27.023" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19707" PostTypeId="2" ParentId="19678" CreationDate="2022-09-19T16:58:53.990" Score="2" Body="&lt;p&gt;You can run &lt;code&gt;--chop&lt;/code&gt; with &lt;code&gt;-x&lt;/code&gt; to excise the last piece, adding the &lt;code&gt;--stagger&lt;/code&gt; to size the chopped pieces as needed, e.g.:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ echo -e 'chr1\t0\t10' | bedops --chop 3 --stagger 2 -x -&#xA;chr1    0       3&#xA;chr1    2       5&#xA;chr1    4       7&#xA;chr1    6       9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Adjust parameters to taste.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/set-operations/bedops.html#chop-w-chop&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedops.readthedocs.io/en/latest/content/reference/set-operations/bedops.html#chop-w-chop&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/set-operations/bedops.html#stagger-stagger&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedops.readthedocs.io/en/latest/content/reference/set-operations/bedops.html#stagger-stagger&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/set-operations/bedops.html#exclude-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedops.readthedocs.io/en/latest/content/reference/set-operations/bedops.html#exclude-x&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2022-09-19T16:58:53.990" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19708" PostTypeId="2" ParentId="19613" CreationDate="2022-09-19T21:50:19.270" Score="1" Body="&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;vln_df = data.frame(gene = ilc2_bs[[&amp;quot;RNA&amp;quot;]]@data[ &amp;quot;gene&amp;quot;, ],&#xA;                    group = ilc2$group)&#xA;&#xA;ggplot(vln_df, aes(x = group, &#xA;                   y = gene, fill = group)) + &#xA;  geom_violin(aes(fill = group, &#xA;                  color = gene),&#xA;              color = NA) +&#xA;  scale_fill_manual(values = c(&amp;quot;#619CFF&amp;quot;, &amp;quot;#F8766D&amp;quot;)) +&#xA;  geom_boxplot(aes(fill = group),&#xA;               width = 0.1) + theme(aspect.ratio = 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3968" LastEditorUserId="650" LastEditDate="2023-04-11T19:26:17.940" LastActivityDate="2023-04-11T19:26:17.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19709" PostTypeId="6" CreationDate="2022-09-20T01:01:43.760" Score="0" Body="&lt;h1&gt;M__&lt;/h1&gt;&#xA;&lt;h3&gt;History on Stack-exchange&lt;/h3&gt;&#xA;&lt;p&gt;Community member for 3.5+ years, with the highest voting record in Bioinformatics SE (3000+), so I've definitely upvoted you, and I'm the only member to have 2 silver badges for the review queues . More generally, 400+ answers, an equal number of question edits and 'closer' to a 'phylogenetics' silver badge (subject to interpretation).&lt;/p&gt;&#xA;&lt;p&gt;I spend more time on Bioinformatics SE than as a senior journal editor.&lt;/p&gt;&#xA;&lt;h3&gt;Tech stuff&lt;/h3&gt;&#xA;&lt;p&gt;My specialism is small genomes, notably haplotypes. I am tightly focused on most things Python, with solid experience in stats, pop gen, machine learning and particularly as mentioned phylogenetics.  I hold experience in niche areas of eukaryotes, like immunology. Right now I'm into 'high performance' Python. My trick is I can quickly spot biological interpretation through numbers.&lt;/p&gt;&#xA;&lt;h3&gt;Why mod?&lt;/h3&gt;&#xA;&lt;p&gt;Firstly, I like the community a lot and think my stats show that. Secondly, I enjoy biological code-based problem solving and how others solve those problems. Finally, I'd like it to be busier still and want to do my part.&lt;/p&gt;&#xA;&lt;p&gt;Ultimately, I enjoying spending time here and its the next logical step.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-09-20T01:01:43.760" LastActivityDate="2022-09-20T01:01:43.760" CommentCount="0" CommunityOwnedDate="2022-09-20T01:01:43.760" ContentLicense="CC BY-SA 4.0" />
  <row Id="19712" PostTypeId="5" CreationDate="2022-09-20T12:27:45.610" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/alexdobin/STAR/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github repository&lt;/a&gt; &amp;amp;&#xA;&lt;a href=&quot;https://github.com/alexdobin/STAR/blob/master/doc/STARmanual.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Manual&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2022-09-20T12:27:45.610" LastActivityDate="2022-09-20T12:27:45.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19713" PostTypeId="4" CreationDate="2022-09-20T12:27:45.610" Score="0" Body="About Spliced Transcripts Alignment to a Reference (STAR)" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2022-09-20T12:27:45.610" LastActivityDate="2022-09-20T12:27:45.610" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19715" PostTypeId="2" ParentId="19356" CreationDate="2022-09-20T15:18:42.843" Score="3" Body="&lt;p&gt;The above answer is right - here are some more links that describe what you want to do in more detail:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://eeob-biodata.github.io/BCB546X-python/06-biopython/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://eeob-biodata.github.io/BCB546X-python/06-biopython/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://medium.com/@insidedna/biopython-to-retrieving-fasta-sequences-from-ncbi-databases-20e35772a25&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://medium.com/@insidedna/biopython-to-retrieving-fasta-sequences-from-ncbi-databases-20e35772a25&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You're going to want to use the BioPython package as it has functions that do exactly what you're looking for.&lt;/p&gt;&#xA;" OwnerUserId="16067" LastActivityDate="2022-09-20T15:18:42.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19717" PostTypeId="2" ParentId="19697" CreationDate="2022-09-21T06:40:30.057" Score="1" Body="&lt;p&gt;Have you tried &lt;a href=&quot;https://david.ncifcrf.gov/summary.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;DAVID&lt;/a&gt; for functional annotation (Gene Ontology and KEGG terms)?&lt;/p&gt;&#xA;&lt;p&gt;I usually use DAVID to extract GO and KEGG terms and plot the results using R.&lt;/p&gt;&#xA;&lt;p&gt;If you also want to plot a protein-protein interaction (PPI) network for your gene list you can use STRING database to extract interactions and network and use Cytoscape for visualizing this result.&lt;/p&gt;&#xA;" OwnerUserId="15931" LastEditorUserId="15931" LastEditDate="2022-09-21T06:50:23.553" LastActivityDate="2022-09-21T06:50:23.553" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19718" PostTypeId="2" ParentId="19689" CreationDate="2022-09-21T08:49:18.753" Score="1" Body="&lt;p&gt;You can use For Loop here. Here I have created code using your algorithm, first created a list of merged dataframes and then split the list into separate dataframes.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;S1 &amp;lt;- data.frame(values = c (1:5), numbers = c(11:15))&#xA;S2 &amp;lt;- data.frame(values = c (1:5), numbers = c(11:15))&#xA;S3 &amp;lt;- data.frame(values = c (1:5), numbers = c(16:20))&#xA;#Dataframe X&#xA;X &amp;lt;- data.frame(values = c (26:30), numbers = c(501:505))&#xA;&#xA;#defining a NULL object&#xA;S=NULL&#xA;#creating a list of merged dataframes&#xA;for (i in 1:3){&#xA;S[i] = list(rbind(get(paste(&amp;quot;S&amp;quot;,i,sep='')),X))&#xA;names(S)[i] = paste(&amp;quot;S&amp;quot;,i,&amp;quot;_X&amp;quot;,sep='')}&#xA;&#xA;#extracting dataframes&#xA;for(i in 1:3){&#xA;assign(paste(&amp;quot;S&amp;quot;,i,&amp;quot;_X&amp;quot;,sep = &amp;quot;&amp;quot;),S[[i]])&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15931" LastActivityDate="2022-09-21T08:49:18.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19719" PostTypeId="1" CreationDate="2022-09-21T13:50:16.533" Score="2" ViewCount="63" Body="&lt;p&gt;I'm doing a WCGNA analysis (signed network) on microbiome 16S data.&#xA;I have transformed counts to centeres log-ratio transformed data (CLR) to address the compositional characteristics of the data and have obtained a pretty decent cluster (around 7 modules, 20 taxa each).&lt;/p&gt;&#xA;&lt;p&gt;I correlated the different modules eigengenes to my features of interest using both bicor and pearson correlation measures. Overall, the results I'm getting make sense but I'm having some questions regarding the method: The % of variance explained of each eigengen is around 30%. Is this value normal?&#xA;Is it ok that I'm using only the PC1 of the module as a summary of the module itself when variances explained of the PC1 are so low?&lt;/p&gt;&#xA;&lt;p&gt;Thx!!&lt;/p&gt;&#xA;" OwnerUserId="16170" LastEditorUserId="16170" LastEditDate="2022-09-21T13:51:08.897" LastActivityDate="2022-09-22T11:07:42.650" Title="WCGNA - Relate modules with Y features when the % of variance explained of each eigengen is low" Tags="|rna-seq|pca|wgcna|microbiome|16rrna|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19720" PostTypeId="2" ParentId="19719" CreationDate="2022-09-21T15:07:02.630" Score="1" Body="&lt;p&gt;&lt;strong&gt;Part 1&lt;/strong&gt; Just to clarify on the specific issue of using PC1 alone given the reported result ...&lt;/p&gt;&#xA;&lt;p&gt;Without knowing the underlying ecology, which is important, 30% is too low to any draw conclusions in PCA-based analysis, particularly because this could be a really complex niche, ecotope, ecoregion, etc ...&lt;/p&gt;&#xA;&lt;p&gt;Thus this specific result from the unsupervised learning shouldn't be used as the basis for any downstream analysis. This is because the remaining 70% of the variance could overturn PC1, where PC1 could be separating a minor feature of the community ecology.&lt;/p&gt;&#xA;&lt;p&gt;Thus, its informative because it states the first component is insufficient to proceed - however I stress its difficult to give a clear answer without knowing the expectedly complexity of the meta-genomic sample.&lt;/p&gt;&#xA;&lt;p&gt;I suspect its a nice result, but in broad general terms its not a good idea.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Part 2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;of each eigengen is around 30%.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If PC1-PC3 is 90%, then thats good and maybe its not so complex. Personally, I'd be delighted with that result. However, the inference is that the clustering is not clear. I would simply suggest further data mining is required.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-09-22T03:35:00.640" LastActivityDate="2022-09-22T03:35:00.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19721" PostTypeId="1" CreationDate="2022-09-21T15:16:23.330" Score="1" ViewCount="25" Body="&lt;p&gt;I have a set of .bgen files split by chromosomes, including both SNPs and INDELS (named with both rsids and chr_pos_a1_a2). I know a large portion of them are encoded on the wrong strand, and I want to recode them against a reference genome (in my case, 1KGP).&lt;/p&gt;&#xA;&lt;p&gt;How can I recode the variants in my starting files against the reference genome?&lt;/p&gt;&#xA;" OwnerUserId="12128" LastEditorUserId="12128" LastEditDate="2022-10-06T02:56:32.407" LastActivityDate="2022-10-06T02:56:32.407" Title="Recode bgen files against reference genome" Tags="|software-recommendation|human-genome|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19722" PostTypeId="1" CreationDate="2022-09-21T20:42:00.883" Score="2" ViewCount="544" Body="&lt;p&gt;So I am trying to parse through a genbank file, extract particular feature information and output that information to a csv file. The example genbank file looks like this:&lt;/p&gt;&#xA;&lt;p&gt;SBxxxxxx.LargeContigs.gbk&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;LOCUS       scaffold_31            38809 bp    DNA              UNK 01-JAN-1980&#xA;DEFINITION  scaffold_31.&#xA;ACCESSION   scaffold_31&#xA;VERSION     scaffold_31&#xA;KEYWORDS    .&#xA;SOURCE      .&#xA;      ORGANISM  .&#xA;COMMENT     ##antiSMASH-Data-START##&#xA;            Version      :: 6.1.1&#xA;            Run date     :: 2022-09-21 11:09:55&#xA;            ##antiSMASH-Data-END##&#xA;FEATURES             Location/Qualifiers&#xA;            protocluster    26198..38809&#xA;                            /aStool=&amp;quot;rule-based-clusters&amp;quot;&#xA;                            /category=&amp;quot;terpene&amp;quot;&#xA;                            /contig_edge=&amp;quot;True&amp;quot;&#xA;                            /core_location=&amp;quot;[36197:37079](-)&amp;quot;&#xA;                            /cutoff=&amp;quot;20000&amp;quot;&#xA;                            /detection_rule=&amp;quot;(Terpene_synth or Terpene_synth_C or&#xA;                            phytoene_synt or Lycopene_cycl or terpene_cyclase or NapT7&#xA;                            or fung_ggpps or fung_ggpps2 or trichodiene_synth or TRI5)&amp;quot;&#xA;                            /neighbourhood=&amp;quot;10000&amp;quot;&#xA;                            /product=&amp;quot;terpene&amp;quot;&#xA;                            /protocluster_number=&amp;quot;1&amp;quot;&#xA;                            /tool=&amp;quot;antismash&amp;quot;&#xA;&#xA;&#xA;  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now for the output file, I want to create a csv with 3 columns. One column will have the Scaffold information (ie. scaffold_31), the second column will have the category value in the protocluster feature (ie. /category = &amp;quot;terpene&amp;quot;) and the third column will have the product value in the protocluster feature (ie. /product=&amp;quot;terpene&amp;quot;)&lt;/p&gt;&#xA;&lt;p&gt;This is what I have so far for code. I know I can sort through the feature.qualifiers in the protocluster feature to get the category and product. I am not sure how to extract the scaffold information. I am completely new to parsing through gene bank files so have little knowledge in this domain. Thanks in advance for any assitance!&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;import Bio&#xA;from Bio import SeqIO&#xA;import os&#xA;&#xA;input = &amp;quot;/Path to SBxxxxxx.LargeContigs.gbk&amp;quot;&#xA;output = open(&amp;quot;output.csv&amp;quot;, &amp;quot;w&amp;quot;)&#xA;&#xA;if not os.path.exists(output):&#xA;     for record in SeqIO.parse(input, &amp;quot;genbank&amp;quot;)&#xA;          for feature in record.features:&#xA;              if feature.type == &amp;quot;protocluster&amp;quot; and &amp;quot;category&amp;quot; and &amp;quot;product&amp;quot; in feature.qualifiers:&#xA;                  outfile = feature.qualifiers[&amp;quot;category&amp;quot;][0] + &amp;quot;,&amp;quot; + feature.qualifiers[&amp;quot;product&amp;quot;][0] + &amp;quot;\n&amp;quot;&#xA;                  output.write(outfile)&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16173" LastEditorUserId="73" LastEditDate="2022-09-22T11:31:59.480" LastActivityDate="2022-09-26T03:43:17.027" Title="Parsing a genbank file and outputting specific feature information to a csv using BioPython" Tags="|python|genbank|parsing|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19723" PostTypeId="1" AcceptedAnswerId="19738" CreationDate="2022-09-21T21:04:22.540" Score="0" ViewCount="282" Body="&lt;p&gt;I have a vcf info formatted in GT:PL.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;chr22  49994037    .   G   A   345.64  PASS    AC=1;AF=0.500;AN=2;BaseQRankSum=-3.605;DP=35;ExcessHet=3.0103;FS=1.377;MLEAC=1;MLEAF=0.500;MQ=60.00;MQRankSum=0.000;QD=10.17;ReadPosRankSum=-0.035;SOR=0.392    GT:AD:DP:GQ:PL  0/1:20,14:34:99:353,0,588&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want to convert it into GT:GP. But I have to write a script for that. Is there any way to do it fast and simple?&lt;/p&gt;&#xA;&lt;p&gt;Note:&lt;/p&gt;&#xA;&lt;p&gt;GT:Genotype&lt;/p&gt;&#xA;&lt;p&gt;PL: Phred-scaled Genotype Likelihoods&lt;/p&gt;&#xA;&lt;p&gt;GP: Genotype Probabilities&lt;/p&gt;&#xA;&lt;p&gt;check the link below for all the info tags&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://samtools.github.io/hts-specs/VCFv4.2.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10296" LastEditorUserId="10296" LastEditDate="2022-09-22T21:54:44.543" LastActivityDate="2022-09-24T19:31:36.613" Title="convert GT:PL format to GT:GP format" Tags="|vcf|format-conversion|conversion|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="19725" PostTypeId="6" CreationDate="2022-09-22T03:47:37.990" Score="0" Body="&lt;h1&gt;gringer&lt;/h1&gt;&#xA;&lt;p&gt;I am a bioinformatics researcher currently working at a medical research institute in Te Whanganui-a-Tara, Aotearoa (New Zealand). I have a formal education in computer science, maths, and biology, and enjoy translating technical language and data into things that other people can understand. I am an existing Bioinformatics Stack Exchange moderator, and have been involved in this community since its proposal and beta phase five years ago.&lt;/p&gt;&#xA;&lt;p&gt;I have a daily email list set up to see recent questions, and typically encounter new questions about half a day after they're asked. My priority is in cleaning up questions (mostly incorporating comments and changes into the asked question), with occasional removal of unanswered posts that seem stale or obsolete.&lt;/p&gt;&#xA;&lt;p&gt;I try to value and support new community members, and encourage all answers, regardless of how short or insignificant they may seem.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-09-23T02:29:14.050" LastActivityDate="2022-09-23T02:29:14.050" CommentCount="0" CommunityOwnedDate="2022-09-22T03:47:37.990" ContentLicense="CC BY-SA 4.0" />
  <row Id="19727" PostTypeId="2" ParentId="19719" CreationDate="2022-09-22T11:07:42.650" Score="0" Body="&lt;p&gt;Proportion of variance explained of about 30% is indeed on the low side, but nothing out of the ordinary. I have no experience with 16S data; in gene expression data, between 30 and 50% is actually considered good. PVE byt the eigengene depends strongly on the number of samples; the more samples, the lower the PVE will be.&lt;/p&gt;&#xA;&lt;p&gt;You can try to calculate additional singular vectors and their PVE. Usually this is much lower than the eigengene (1st singular vector). So I would focus more on the biological significance - do the modules make biological sense?&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2022-09-22T11:07:42.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19728" PostTypeId="1" CreationDate="2022-09-22T14:34:58.130" Score="2" ViewCount="104" Body="&lt;p&gt;When I run following code, I get error at -&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;slingshot_df &amp;lt;- data.frame(colData(sce))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;the error is :&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error in as.vector(x) : no method for coercing this S4 class to a&#xA;vector&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(Polychrome)&#xA;library(ggbeeswarm)&#xA;library(ggthemes)&#xA;&#xA;# this define the cluster color. You can change it with different color scheme.&#xA;my_color &amp;lt;- createPalette(length(levels(sce&lt;span class=&quot;math-container&quot;&gt;$ident)), c(&quot;#010101&quot;, &quot;#ff0000&quot;), M=1000)&#xA;names(my_color) &amp;lt;- unique(as.character(sce$&lt;/span&gt;ident))&#xA;&#xA;slingshot_df &amp;lt;- data.frame(colData(sce))&#xA;&#xA;# re-order y-axis for better figure: This should be tailored with your own cluster names&#xA;# slingshot_df&lt;span class=&quot;math-container&quot;&gt;$ident = factor(slingshot_df$&lt;/span&gt;ident, levels=c(4,2,1,0,3,5,6))&#xA;&#xA;ggplot(slingshot_df, aes(x = slingPseudotime_1, y = ident, &#xA;                              colour = ident)) +&#xA;    geom_quasirandom(groupOnX = FALSE) + theme_classic() +&#xA;    xlab(&amp;quot;First Slingshot pseudotime&amp;quot;) + ylab(&amp;quot;cell type&amp;quot;) +&#xA;    ggtitle(&amp;quot;Cells ordered by Slingshot pseudotime&amp;quot;)+scale_colour_manual(values = my_color)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16179" LastEditorUserId="3848" LastEditDate="2022-09-22T14:38:00.110" LastActivityDate="2022-09-22T14:38:00.110" Title="I am trying to use slingshot for pseudotime analysis. I am getting error while saving an object as dataframe" Tags="|r|scrnaseq|seurat|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19729" PostTypeId="2" ParentId="19722" CreationDate="2022-09-22T15:34:42.377" Score="2" Body="&lt;p&gt;Depending on which field you want to pull the &amp;quot;scaffold_31&amp;quot; text from, you have a few options:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;GenBank Text&lt;/th&gt;&#xA;&lt;th&gt;BioPython object&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;&lt;code&gt;LOCUS&lt;/code&gt;&lt;/td&gt;&#xA;&lt;td&gt;&lt;code&gt;record.name&lt;/code&gt;&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;&lt;code&gt;DEFINITION&lt;/code&gt;&lt;/td&gt;&#xA;&lt;td&gt;&lt;code&gt;record.description&lt;/code&gt;&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;&lt;code&gt;ACCESSION&lt;/code&gt;&lt;/td&gt;&#xA;&lt;td&gt;&lt;code&gt;record.accessions[0]&lt;/code&gt;&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;&lt;code&gt;VERSION&lt;/code&gt;&lt;/td&gt;&#xA;&lt;td&gt;&lt;code&gt;record.version&lt;/code&gt;&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;Python's built in &lt;code&gt;dir()&lt;/code&gt; function is handy for figuring out this kind of thing.&lt;/p&gt;&#xA;&lt;p&gt;If you're working with a draft flat file (like BankIt gives you just before submitting) note that some of those are placeholders that get updated with the actual accession info when it's finalized.  So your &amp;quot;scaffold_31&amp;quot; text will only show up I think in the DEFINITION line in the end if I remember right.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2022-09-22T15:34:42.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19730" PostTypeId="1" CreationDate="2022-09-22T18:01:32.213" Score="1" ViewCount="355" Body="&lt;p&gt;I would like to visualize the secondary structure of co-folded RNA strands in Python, preferably in a Jupyter notebook.&#xA;What are the recommended tools? Can anyone provide an example?&lt;/p&gt;&#xA;" OwnerUserId="16162" LastActivityDate="2022-09-23T10:52:48.613" Title="Visualize RNA co-folded secondary structure with Python" Tags="|python|visualization|rna|secondary-structure|jupyter|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19731" PostTypeId="2" ParentId="19730" CreationDate="2022-09-22T18:27:31.393" Score="1" Body="&lt;p&gt;... . I know both aspects of this question and I'm not aware of an interactive Python package using e.g. iPython magic commands that will do this. The thing you could try is  &lt;code&gt;draw_RNA&lt;/code&gt; &lt;a href=&quot;https://github.com/DasLab/draw_rna&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/DasLab/draw_rna&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;draw_RNA&lt;/code&gt; will operate on Jupyter, but I've not tried it.&lt;/p&gt;&#xA;&lt;p&gt;A very, very good package is &lt;a href=&quot;https://varna.lri.fr&quot; rel=&quot;nofollow noreferrer&quot;&gt;VARNA&lt;/a&gt; (its Java BTW). The &lt;code&gt;VARNA&lt;/code&gt; group make further recommendations from there for complex structures. I would vouch for this package for complex structures.&lt;/p&gt;&#xA;&lt;p&gt;A good approach is to output in Stockholm 1.0 format. It then just drops and drags into &lt;code&gt;VARNA&lt;/code&gt;. &lt;code&gt;Dot-bracket&lt;/code&gt; format also works and I would assume &lt;code&gt;ct&lt;/code&gt; format works too.&lt;/p&gt;&#xA;&lt;p&gt;A good reason why in-line graph imaging isn't the best solution for complex structures, i.e. the in-line graphs in a Python debugger or Jupiter is complex RNA structures need to be interactive because you need to move them: they &amp;quot;clump&amp;quot; together and have to manually separate them.&lt;/p&gt;&#xA;&lt;p&gt;As a final note &lt;a href=&quot;http://rna.tbi.univie.ac.at/forna/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Forna&lt;/a&gt; may be of benefit,&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-09-23T10:52:48.613" LastActivityDate="2022-09-23T10:52:48.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19732" PostTypeId="1" CreationDate="2022-09-23T08:30:50.847" Score="1" ViewCount="86" Body="&lt;p&gt;I want to load multiple sample data(pbmc16 and pbmc45), and integrate by following the tutorial below&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Performing integration on datasets normalized with SCTransform&#xA;&lt;a href=&quot;https://satijalab.org/seurat/articles/integration_introduction.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/articles/integration_introduction.html&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;performing integration on datasets normalized with sctransform-1&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;What I did:&lt;/p&gt;&#xA;&lt;p&gt;1. I performed the pre-processing step (filtering) and SCtransform in each sample, and then save each data with the code&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;saveRDS(pbmc16, file = &amp;quot;../output/pbmc16_Seurat_Guided_Clustering_Tutorial.rds&amp;quot;)&#xA;saveRDS(pbmc45, file = &amp;quot;../output/pbmc45_Seurat_Guided_Clustering_Tutorial.rds&amp;quot;)&#xA;&amp;gt; pbmc16; type S4 [20542x20700](SeuratObject:: Seurat) assays; list [2]&#xA;&amp;gt; meta.data; list [20700x8] (S3:data.frame)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;2. read each sample data by using &lt;code&gt;readRDS&lt;/code&gt; function&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;pbmc16 &amp;lt;- readRDS(&amp;quot;../output/pbmc16_Seurat_Guided_Clustering_Tutorial.rds&amp;quot;)&#xA;pbmc45 &amp;lt;- readRDS(&amp;quot;../output/pbmc45_Seurat_Guided_Clustering_Tutorial.rds&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;3. After that, I want to load the data as shown in the tutorial&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;(LoadData(&amp;quot;ifnb&amp;quot;))&#xA;LoadData(pbmc16)&#xA;LoadData(pbmc45)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then I got this error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; LoadData(pbmc16) Error in match(x, table, nomatch = 0L) : 'match'&#xA;&amp;gt; requires vector arguments&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;Problem&lt;/em&gt;: I don't know how to load each data and integrate the multiple data.&lt;/p&gt;&#xA;&lt;p&gt;Could you please suggest me the recommended code or advice?&lt;/p&gt;&#xA;" OwnerUserId="16184" LastEditorUserId="3694" LastEditDate="2022-11-27T09:12:34.357" LastActivityDate="2022-11-27T09:12:34.357" Title="Cannot run `loadData()` after saving RDS file and reading RDS file using `saveRDS` &amp; `readRDS` in Seurat" Tags="|r|seurat|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19733" PostTypeId="1" CreationDate="2022-09-23T17:55:48.950" Score="1" ViewCount="71" Body="&lt;p&gt;I want to obtain a direct graph from KEGG as well as the genes present in a single term. While it is pretty easy to get list of genes/compounds that are part of a specific category, I am kinda stucked to obtain the hierarchical relationships between terms. The docs on biopython.KGML are not really helpful. This is what I have so far&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;import Bio.KEGG.REST as kegg&#xA;from Bio.KEGG.KGML import KGML_parser&#xA;from Bio.Graphics.KGML_vis import KGMLCanvas&#xA;from Bio.Graphics.ColorSpiral import ColorSpiral&#xA;from itertools import chain&#xA;&#xA;k =  kegg.kegg_list('pathway', 'pae').read()&#xA;todf = []&#xA;for x in k.split('\n'):&#xA;    if x=='':&#xA;        next&#xA;    dd = x.split(' - ')&#xA;    dd = [v.split('\t') if '\t' in v else v for v in dd]&#xA;    dd = list(chain.from_iterable(dd))[:2]&#xA;    try:&#xA;        ll= kegg.kegg_get(dd[0]).read().split('GENE')[-1]&#xA;        ll = [x.lstrip() for x in ll.split('COMPOUND')[0].split('\n')]&#xA;        ll= [x.rstrip().split('\t') for x in ll]&#xA;        for gn in ll:&#xA;            gn.extend(dd)&#xA;        todf.append(pd.DataFrame(ll))&#xA;    except IndexError:&#xA;        pass&#xA;todf = pd.concat(todf)&#xA;# remove compounds and random strings&#xA;todf= todf[todf[0].str.startswith('PA')]&#xA;todf[0] = [x.split(';')[0] if ';' in x else x for x in todf[0]]&#xA;todf[0] = todf[0].str.split('  ')&#xA;todf=todf.explode(0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and now I have the protein to term relationship but not sure how to get the hierarchy between terms.&#xA;Any idea?&lt;/p&gt;&#xA;" OwnerUserId="16187" LastActivityDate="2022-09-23T21:28:36.233" Title="KEGG retrieve hierarchy for organism in biopython" Tags="|python|kegg|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19734" PostTypeId="2" ParentId="2427" CreationDate="2022-09-23T21:24:23.113" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://msberends.github.io/AMR/reference/mo_property.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;AMR R package&lt;/a&gt; can assign gram staining based on taxonomy. See &lt;code&gt;mo_gramstain(x, language = get_AMR_locale(), ...)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16188" LastActivityDate="2022-09-23T21:24:23.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19735" PostTypeId="2" ParentId="19733" CreationDate="2022-09-23T21:28:36.233" Score="2" Body="&lt;p&gt;Ok realized it was much easier than expected.&#xA;On KEGG it is possible to DL a json file, which is intrinsically ordered. So   this parses it into a df&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd &#xA;import json&#xA;&#xA;todf = []&#xA;&#xA;kegg = json.load(open('meta/KEGG_pae00001.json'))&#xA;for main in kegg['children']:&#xA;    for broad in main['children']:&#xA;        for sub in broad['children']:&#xA;            for gn in sub.get('children', [None]):&#xA;                if gn:&#xA;                     nm = gn['name'].split(';')[0].split(' ')&#xA;                     todf.append([nm[0], nm[1], sub['name'], broad['name'], main['name']])&#xA;                else:&#xA;                     # some terms have no children&#xA;                     pass&#xA;                     #print(sub)&#xA;            &#xA;            &#xA;            &#xA;todf = pd.DataFrame(todf)&#xA;todf.columns=['Locus', 'GeneName', 'Cat1', 'Cat2', 'Cat3']&#xA;todf.to_csv('meta/kegg_parsed.csv')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16187" LastActivityDate="2022-09-23T21:28:36.233" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19736" PostTypeId="1" CreationDate="2022-09-24T15:59:59.850" Score="1" ViewCount="90" Body="&lt;p&gt;I've had my whole genome sequenced (at 30x average coverage) by a lab, and they have provided the raw files to me (BAM, FASTQ, and VCF).&lt;/p&gt;&#xA;&lt;p&gt;How can I assemble it?&lt;/p&gt;&#xA;&lt;p&gt;And does assembly provide any further insights that I can't get from the separated fragments?&lt;/p&gt;&#xA;&lt;p&gt;I'd like to assemble and annotate it, and find insights, but I don't know where to start.&lt;/p&gt;&#xA;" OwnerUserId="16190" LastActivityDate="2022-09-28T14:34:10.237" Title="How can I assemble my genome from raw files?" Tags="|assembly|human-genome|sequence-annotation|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19737" PostTypeId="2" ParentId="19722" CreationDate="2022-09-24T16:33:16.623" Score="0" Body="&lt;p&gt;I've used SARS-CoV-2 (Genbank: PA544053), because there was no Genbank entry given in the OPs question. If my example is representative (might not be) I think its about the object attributes. To understand the object I listed its attributes,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print(record[0].__dict__.keys())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;dict_keys(['_seq', 'id', 'name', 'description', 'dbxrefs',&#xA;'annotations', '_per_letter_annotations', 'features'])&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The docs and @jesse's very kind response says there's a 'accession' attribute (Biopython docs below). @Jesse did mention &lt;code&gt;dir()&lt;/code&gt; which was cool. I couldn't find &lt;code&gt;record[0].accession&lt;/code&gt; or perhaps &lt;code&gt;record[0].accessions&lt;/code&gt; and the OP might have had the same problem.&lt;/p&gt;&#xA;&lt;p&gt;In my example &lt;em&gt;there is an &lt;code&gt;'annotations'&lt;/code&gt; attribute&lt;/em&gt;  and beneath that was &lt;code&gt;'accession'&lt;/code&gt; accessed via,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print(record[0].annotations['accessions'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;['PA544053']&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There it is ... Here's the full code including the CSV package, I'm using &lt;code&gt;efetch&lt;/code&gt; so it'll just copy and paste and run.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez&#xA;from Bio import SeqIO&#xA;import csv&#xA;from pathlib import Path&#xA;&#xA;fileout = Path('/mypath/mydir', 'test.csv')&#xA;handle = Entrez.efetch(db=&amp;quot;nucleotide&amp;quot;, id=&amp;quot;PA544053&amp;quot;, rettype=&amp;quot;gb&amp;quot;,retmode=&amp;quot;text&amp;quot;) #, &#xA;record = [rec for rec in SeqIO.parse(handle,'genbank')]&#xA;fieldnames = ['Accession', 'db_xref', 'organism']&#xA;&#xA;# print(record[0].__dict__.keys())&#xA;print(record[0].annotations['accessions'])&#xA;&#xA;with open (fileout, 'w') as f:&#xA;    csvwriter = csv.writer(f, delimiter=&amp;quot;\t&amp;quot;, quotechar='&amp;quot;')&#xA;    csvwriter.writerow(fieldnames)    &#xA;    for feature in record[0].features:&#xA;        if feature.type == 'organism' or &amp;quot;mol_type&amp;quot; and &amp;quot;db_xref&amp;quot; in feature.qualifiers:&#xA;            csvwriter.writerow([''.join(record[0].annotations['accessions']), ''.join(feature.qualifiers['db_xref']), ''.join(feature.qualifiers['organism'])])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Accession      db_xref       organism&#xA;PA544053       taxon:2697049 Severe acute respiratory syndrome coronavirus 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Biopython docs&lt;/strong&gt;&#xA;A likely reason for the question is the missing attribute is described in the official docs. Using a &lt;code&gt;GenBank&lt;/code&gt; object (not &lt;code&gt;SeqIO&lt;/code&gt;) there is certainly an &lt;code&gt;accession&lt;/code&gt; attribute, &lt;a href=&quot;https://biopython.org/docs/1.75/api/Bio.GenBank.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biopython.org/docs/1.75/api/Bio.GenBank.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The official Biopython page ...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import GenBank&#xA;with open(&amp;quot;GenBank/NC_000932.gb&amp;quot;) as handle: &#xA;    record = GenBank.read(handle)    &#xA;    print(record.accession)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I think the basis of the question is to associate the accession number with the biochemical/genetic info. Note, I don't know the difference between &lt;code&gt;SeqIO&lt;/code&gt; and &lt;code&gt;GenBank&lt;/code&gt; objects.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-09-26T03:43:17.027" LastActivityDate="2022-09-26T03:43:17.027" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19738" PostTypeId="2" ParentId="19723" CreationDate="2022-09-24T19:31:36.613" Score="1" Body="&lt;p&gt;Currently you have &lt;strong&gt;phred-scaled genotype likelihoods&lt;/strong&gt; or &lt;strong&gt;PL&lt;/strong&gt; values. Genotype likelihoods give the likelihood of observing the read data (i.e. the number of reads which map to either the ref or alt allele), given a particular underlying genotype (0/0, 0/1, 1/1 at a bi-allelic locus); &lt;span class=&quot;math-container&quot;&gt;$GL = p(G|D)$&lt;/span&gt;. Phred scaling simply scales it like &lt;span class=&quot;math-container&quot;&gt;$PL = -10 * log(GL)$&lt;/span&gt;. These are usually given by a genotype caller.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Genotype probabilities&lt;/strong&gt; are slightly different in that they are posterior probabilities that the genotype is correct, given either other genotypes from a population or trio and a prior distribution for the allele frequencies. Therefore, you can't trivially go from likelihoods to probabilities without accounting for this external information and the prior. As suggested by JRodrigoF, you should use something like &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360036730531-CalculateGenotypePosteriors&quot; rel=&quot;nofollow noreferrer&quot;&gt;CalculateGenotypePosteriors&lt;/a&gt; to do this, maybe including 1000 genomes allele counts.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2022-09-24T19:31:36.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19739" PostTypeId="2" ParentId="19736" CreationDate="2022-09-24T19:43:15.520" Score="2" Body="&lt;p&gt;Given you've tagged this as 'human-genome' there is likely little value in performing genome assembly. De-novo assembly of WGS is generally for non-model organisms without a reference genome available, and would require much higher than 30X coverage (and ideally long reads). By providing the BAM and VCF, they'e already saved you some upstream work unless there are needs to re-process the data from FASTQ&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2022-09-24T19:43:15.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19740" PostTypeId="2" ParentId="19736" CreationDate="2022-09-25T11:27:58.333" Score="2" Body="&lt;p&gt;No, assembling won't tell you anything new. When working with organisms whose reference genome is known, all you care about is where &lt;em&gt;your&lt;/em&gt; sequences differ from the reference. If you know where your sequences differ, then you can infer the rest, they will be whatever is in the reference.&lt;/p&gt;&#xA;&lt;p&gt;All you want to do is identify &lt;em&gt;variants&lt;/em&gt;, those sites where your genome differs from the reference. That's what the VCF file contains. Bear in mind that the reference isn't some sort of platonic ideal of the perfect human, it's just the collected, pooled DNA of &lt;a href=&quot;https://en.wikipedia.org/wiki/Reference_genome#Human_reference_genome&quot; rel=&quot;nofollow noreferrer&quot;&gt;13 anonymous, apparently healthy individuals&lt;/a&gt;. You can expect anywhere from about 3 to 5 million differences in your own genome, the overwhelming majority of which will be completely innocuous.&lt;/p&gt;&#xA;&lt;p&gt;Since you already have your VCF file, your list of variants, that's all you need to investigate. If you really want to, you can make your assembly by taking the reference genome and then adding the variants in your VCF file, but it won't be of much practical use.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2022-09-25T11:27:58.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19742" PostTypeId="1" CreationDate="2022-09-25T18:16:30.683" Score="3" ViewCount="281" Body="&lt;p&gt;I'm trying to find the HLA-B*15:01 variant in my DNA results, prompted from this research paper: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8142661/#:%7E:text=HLA%2DB*15%3A01%20is%20strongly%20associated%20with%20asymptomatic,mechanism%20underlying%20early%20viral%20clearance&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8142661/#:~:text=HLA%2DB*15%3A01%20is%20strongly%20associated%20with%20asymptomatic,mechanism%20underlying%20early%20viral%20clearance&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I have DNA test results from Dante Labs that contain the following files&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;*.bam&amp;quot;&#xA;&amp;quot;*.bam.bai&amp;quot;&#xA;&amp;quot;*.cnv.vcf.gz&amp;quot;&#xA;&amp;quot;*.cnv.vcf.gz.tbi&amp;quot;&#xA;&amp;quot;*.filtered.indel.vcf.gz&amp;quot;&#xA;&amp;quot;*.filtered.indel.vcf.gz.tbi&amp;quot;&#xA;&amp;quot;*.filtered.snp.vcf.gz&amp;quot;&#xA;&amp;quot;*.filtered.snp.vcf.gz.tbi&amp;quot;&#xA;&amp;quot;*1_SA_L001_R1_001.fastq.gz&amp;quot;&#xA;&amp;quot;*SA_L001_R2_001.fastq.gz&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried loading this on the Genome Explorer on Sequencing.com, but it couldn't find HLA-B at all, only HLA-A and HLA-C.&lt;/p&gt;&#xA;&lt;p&gt;Then I tried loading the VCF files in IGV. I was able to filter to Chromosome 6 and HLA-B, which resolved to position&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr6:31,319,652-31,326,956&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The CNV VCF says no variables found for chr6:31,319,652-31,326,956&#xA;The filtered Indel VCF did produce some variants for this position, but I have no clue how to search deeper specifically for HLA-B*15:01. Here is a screenshot: &lt;a href=&quot;https://i.stack.imgur.com/eCnFM.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eCnFM.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The filtered SNP VCF produced &lt;em&gt;alot&lt;/em&gt; more variants for this position, but again, I don't know how to go deeper than this.&lt;/p&gt;&#xA;&lt;p&gt;I'm not really sure what to do from here?&lt;/p&gt;&#xA;&lt;p&gt;I don't have data on what the reference was, so that is unknown to me. I will reach out to Dante Labs about that, is there some sort of standard that is supposed to be used, or is it different for every genome provider?&lt;/p&gt;&#xA;&lt;p&gt;How do I actually perform the check in IGV assuming the reference did not have it and the VCF did. I also assume this VCF does not have this allele, but I frankly am lost into how to confirm it to the &amp;quot;last mile&amp;quot; through the VCF so to speak, or even just go straight to BAM&lt;/p&gt;&#xA;" OwnerUserId="16192" LastEditorUserId="73" LastEditDate="2022-09-27T03:23:21.403" LastActivityDate="2022-09-27T03:24:15.620" Title="Searching for HLA-B in DNA results" Tags="|gene|genome|igv|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19743" PostTypeId="2" ParentId="19742" CreationDate="2022-09-25T20:32:16.017" Score="1" Body="&lt;p&gt;The specific question and the easiest question is the absence of HLA-B*15:01 from your sample and that is not unexpected at all.&lt;/p&gt;&#xA;&lt;p&gt;The maximum frequency of this allele is 40% found in &amp;quot;&lt;a href=&quot;http://www.allelefrequencies.net/hla6006a.asp?page=40&amp;amp;hla_locus=B&amp;amp;hla_locus_type=Classical&amp;amp;hla_allele1=15&amp;amp;hla_allele2=&amp;amp;hla_selection=&amp;amp;hla_pop_selection=&amp;amp;hla_population=&amp;amp;hla_country=&amp;amp;hla_dataset=&amp;amp;hla_region=&amp;amp;hla_ethnic=&amp;amp;hla_study=&amp;amp;hla_sample_size=&amp;amp;hla_sample_size_pattern=equal&amp;amp;hla_sample_year=&amp;amp;hla_sample_year_pattern=equal&amp;amp;hla_level=&amp;amp;hla_level_pattern=equal&amp;amp;hla_show=&amp;amp;hla_order=order_1&amp;amp;standard=g&quot; rel=&quot;nofollow noreferrer&quot;&gt;China Beijing pop 2&lt;/a&gt;&amp;quot; , this however is really unusually high and was published by Guang Yang et al (2009, IJI 37, 205-212).&lt;/p&gt;&#xA;&lt;p&gt;Mostly this allele is present at 5% for moderate frequency of occurrence for example Europe, such as in Germany &lt;a href=&quot;http://www.allelefrequencies.net/hla6006a.asp?page=40&amp;amp;hla_locus=B&amp;amp;hla_locus_type=Classical&amp;amp;hla_allele1=15&amp;amp;hla_allele2=&amp;amp;hla_selection=&amp;amp;hla_pop_selection=&amp;amp;hla_population=&amp;amp;hla_country=&amp;amp;hla_dataset=&amp;amp;hla_region=&amp;amp;hla_ethnic=&amp;amp;hla_study=&amp;amp;hla_sample_size=&amp;amp;hla_sample_size_pattern=equal&amp;amp;hla_sample_year=&amp;amp;hla_sample_year_pattern=equal&amp;amp;hla_level=&amp;amp;hla_level_pattern=equal&amp;amp;hla_show=&amp;amp;hla_order=order_1&amp;amp;standard=g&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Moreover, in some populations the frequency is very low &amp;lt;&amp;lt;1%, e.g. India or Israel.&lt;/p&gt;&#xA;&lt;p&gt;So a sample genome at random the will not have the B*15:01.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The more complicated issue is the VCF and the reference genome. This is because the basis on whether an allele is present or absent is a comparison against the reference genome which will generate the VCF. Thus understanding the HLA-B alleles of the reference genome is really important. You need a reference genome which lacks HLA-B*15:01, therefore if your VCF also lacks the allele, its missing. However, if HLA-B*15:01 is present in the reference genome your VCF will not present the allele if its present.&lt;/p&gt;&#xA;&lt;p&gt;The probability is that the allele is missing from both the reference and sample genome.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Comments&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Easy way&lt;/strong&gt; The easiest way is search the IGV for &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/LT618832.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;LT618832.1&lt;/a&gt; (HLA-B*15:01 reference), IGV has a motif finder, here &lt;a href=&quot;https://software.broadinstitute.org/software/igv/motif_finder&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://software.broadinstitute.org/software/igv/motif_finder&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Hard way&lt;/strong&gt;&#xA;This is how I'd do it ..&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;convert the bam to fasta then&lt;/li&gt;&#xA;&lt;li&gt;run a blast against Genbank reference LT618832.1&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I'd section the bam file into the locus of interest (e.g. chromosome 6) e.g. ... &lt;code&gt;samtools view ... ChrName: ... &amp;gt; chromo6.bam&lt;/code&gt;: you'd need to cross check how to do this last bit ... then via &lt;code&gt;samtools &amp;gt;1.3&lt;/code&gt;,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools fasta chromosome6.bam &amp;gt; chromosome6.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Finally you download a standalone version of &lt;code&gt;blast&lt;/code&gt;, I assume its on &lt;code&gt;-c bioconda&lt;/code&gt; ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastn -query HLAB1501.faa -db chromosome6.fa -out myhits.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From memory I don't think you need to &lt;code&gt;makeblastdb&lt;/code&gt; for a fasta onto a fasta db.&lt;/p&gt;&#xA;&lt;p&gt;You'd need to map the SNPs in introns parsing (the coordinates are in the Genbank reference), or optically against the Genbank reference onto the &lt;code&gt;blast&lt;/code&gt; output.&lt;/p&gt;&#xA;&lt;p&gt;You may want to cross check the HLA-B*15:01 against &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/?term=HLA-B%2A15%3A01&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/?term=HLA-B%2A15%3A01&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;[but be careful about the introns]&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;motif_finder&lt;/code&gt; seems easier and accommodates potential heterozygosity.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="73" LastEditDate="2022-09-27T03:24:15.620" LastActivityDate="2022-09-27T03:24:15.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19745" PostTypeId="1" CreationDate="2022-09-26T21:50:18.943" Score="2" ViewCount="175" Body="&lt;p&gt;My question is twofold. First, I've run into an issue working with .cif files and pandas dataframes. With BioPandas, I can read a .pdb file into a dataframe and then go the opposite direction, writing a .pdb file from a dataframe. With .cif files, I can only write into a dataframe - PandasMmcif does not contain a &lt;code&gt;to_cif&lt;/code&gt; function like PandasPdb has a &lt;code&gt;to_pdb&lt;/code&gt; function. Are there any alternative methods I could use to get from .cif to pandas dataframe back to .cif?&lt;/p&gt;&#xA;&lt;p&gt;Second, the reason I am not simply converting all of my .cif files to .pdb is that I am uncertain whether I will lose data in the conversion. Is this fear unfounded? I don't know the specific differences between .pdb and .cif.&lt;/p&gt;&#xA;" OwnerUserId="16195" LastActivityDate="2022-09-26T21:50:18.943" Title="Writing to .cif from BioPandas" Tags="|python|protein-structure|structural-biology|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19746" PostTypeId="1" CreationDate="2022-09-26T23:57:02.203" Score="1" ViewCount="131" Body="&lt;p&gt;I am developing a gatk pipeline for variant calling and want to include trimmomatic as a preprocessing step. I would like the user to specify a program file of parameters to feed into trimmomatic, and a file of adapter sequences to provide to the ILLUMINACLIP parameter. My goal is to include a variable (“~{adapter_file}” below) in the program file that will expand to include the adapter filename, like so:&lt;/p&gt;&#xA;&lt;p&gt;Adapter file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;IL_Multiplex_five_prime_fwd&#xA;AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGTAGATCTCGGTGGTCGCCGTATCATT&#xA;&amp;gt;IL_Multiplex_three_prime_rev&#xA;AGATCGGAAGAGCACACGTCTGAACTCCAGTCAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Program file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ILLUMINACLIP:~{adapter_file}:2:30:10:7:keepBothReads&#xA;MINLEN:36&#xA;SLIDINGWINDOW:4:15&#xA;LEADING:3&#xA;TRAILING:3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In this way, the user can change the adaptor file without having to alter the program file each time. However, I can’t seem to get the ~{adapter_file} variable to expand inside the ~{trim_program_array} variable. Here’s a simplified version of my trimmomatic WDL file:&lt;/p&gt;&#xA;&lt;p&gt;Sample workflow:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;version 1.0&#xA; &#xA;Workflow trim {&#xA;    input {&#xA;        File fastq1 = “R1.fastq.gz”&#xA;        File fastq2 = “R2.fastq.gz”&#xA;        File program_file = “trim_program.txt”&#xA;        File adapter_file = “adapters.fa”&#xA;    }&#xA; &#xA;    Array[String] trim_program_array = read_lines(program_file)&#xA; &#xA;    call trimmomatic {&#xA;        input:&#xA;            fastq1 = fastq1,&#xA;            fastq2 = fastq2,&#xA;            trim_program_array = trim_program_array,&#xA;            adapter_file = adaptor_file&#xA;    }&#xA;}&#xA; &#xA;Task trimmomatic {&#xA;    input {&#xA;        File fastq1&#xA;        File fastq2&#xA;        File outfile1 = “trimmed_R1.fastq.gz”&#xA;        File outfile2 = “trimmed_R2.fastq.gz”&#xA;        Array[String] trim_program_array&#xA;        File adapter_file&#xA;     }&#xA; &#xA;     command {&#xA;        TrimmomaticPE \&#xA;        ~{fastq1} ~{fastq2} \&#xA;        ~{outfile1} ~{outfile2} \&#xA;        ~{sep = “ “ trim_program_array}&#xA;     }&#xA; &#xA;     output {&#xA;        File out1 = outfile1&#xA;        File out2 = outfile2&#xA;     }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My goal is for the first element of ~{sep = &amp;quot; &amp;quot; trim_program_array} (aka the first line of trim_program.txt) to evaluate to “ILLUMINACLIP:adapters.fa:2:30:10:7:keepBothReads” rather than “ILLUMINACLIP:~{adapter_file}:2:30:10:7:keepBothReads”. I know this is because ~{adapter_file} is nested inside trim_program_array[0], so it’s only expanding the “outer” variable. How can I convince WDL to expand the variable nested inside the first element of that array before executing the bash command?&lt;/p&gt;&#xA;" OwnerUserId="16196" LastEditorUserId="73" LastEditDate="2022-09-27T07:46:33.753" LastActivityDate="2022-10-04T08:34:33.483" Title="Nested variable expansion in WDL code block" Tags="|wdl|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19747" PostTypeId="2" ParentId="19742" CreationDate="2022-09-27T02:11:42.073" Score="2" Body="&lt;p&gt;Typical SNP calling wouldn't give you good results on classical HLA genes. Ignore the VCF. Try a dedicated HLA genotyper on raw reads instead. I heard &lt;a href=&quot;https://github.com/FRED-2/OptiType&quot; rel=&quot;nofollow noreferrer&quot;&gt;optitype&lt;/a&gt; is a good one for exome or whole-genome data. I haven't used it myself, though.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2022-09-27T02:11:42.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19748" PostTypeId="1" CreationDate="2022-09-27T08:38:45.507" Score="0" ViewCount="370" Body="&lt;p&gt;lets say I have the following dataframe:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Ensembl_ID          A   B   C   D   E   F   G&#xA;ENSG00000236601.2   0   0   0   1   0   0   1&#xA;ENSG00000237094.12  2   187 442 109 88  144 486&#xA;ENSG00000269732.1   0   0   0   0   0   0   0&#xA;ENSG00000284733.1   0   0   0   0   0   0   0&#xA;ENSG00000233653.3   0   0   0   0   0   0   0&#xA;ENSG00000250575.1   1   11  17  5   5   8   17&#xA;ENSG00000278757.1   0   0   2   0   1   0   7&#xA;ENSG00000230021.9   0   25  30  27  9   8   11&#xA;ENSG00000235146.2   0   0   0   0   0   0   0&#xA;ENSG00000225972.1   0   7   3016    11  5   14  5&#xA;ENSG00000225630.1   1   113 194 76  91  47  94&#xA;ENSG00000237973.1   25  1037    9767    1160    886 1321    5220&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What I would like to do is only keep the ~top 50 highly variable genes:&lt;/p&gt;&#xA;&lt;p&gt;I thought it would be like this:  &lt;code&gt;dif_genes &amp;lt;- order(apply(df, 1, var), decreasing=TRUE)[1:50]&lt;/code&gt;&#xA;but that leaves me with a vector containing variances without genes.&lt;/p&gt;&#xA;" OwnerUserId="16197" LastActivityDate="2022-09-27T10:10:20.030" Title="select highly variable genes out of dataframe" Tags="|r|rna-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19749" PostTypeId="2" ParentId="19748" CreationDate="2022-09-27T10:10:20.030" Score="3" Body="&lt;p&gt;One way is adding a new &lt;code&gt;var&lt;/code&gt; column to your original data frame and sorting the updated data frame by the newly introduced column:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df&lt;span class=&quot;math-container&quot;&gt;$var &amp;lt;- apply(df, 1, var)&#xA;df[order(df$&lt;/span&gt;var, decreasing=TRUE),] &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2022-09-27T10:10:20.030" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19750" PostTypeId="1" CreationDate="2022-09-27T13:06:22.090" Score="3" ViewCount="181" Body="&lt;p&gt;This is my first time working on using WGCNA on a microarray dataset. One of the major problems I am facing is merging close modules which is not really working well. I have quantile normalised the data before working on it. I have put my code below and link to some of the plots generated during this process. I was wondering can somebody advice on what could be improved upon in this analysis?&#xA;Link to plots: ![plots] &lt;a href=&quot;https://i.stack.imgur.com/dF6NA.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://i.stack.imgur.com/dF6NA.jpg&lt;/a&gt;&#xA;I have a total of 767 samples and 13466 probes.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;powers = c(c(1:10))&#xA;## Scale free topology since it can find hubs&#xA;sft = pickSoftThreshold(bryois_norm_keep_use, powerVector = powers, verbose = 5)&#xA;par(mfrow = c(1,2));&#xA;cex1 = 0.9;&#xA;plot(sft$fitIndices[,1], -sign(sft$fitIndices[,3])*sft$fitIndices[,2],&#xA;xlab=&amp;quot;Soft Threshold (power)&amp;quot;,ylab=&amp;quot;Scale Free Topology Model Fit,signed R^2&amp;quot;,type=&amp;quot;n&amp;quot;,&#xA;main = paste(&amp;quot;Scale independence&amp;quot;));&#xA;text(sft$fitIndices[,1], -sign(sft$fitIndices[,3])*sft$fitIndices[,2],&#xA;labels=powers,cex=cex1,col=&amp;quot;red&amp;quot;);&#xA;abline(h=0.90,col=&amp;quot;red&amp;quot;)&#xA;# Mean connectivity as a function of the soft-thresholding power&#xA;plot(sft$fitIndices[,1], sft$fitIndices[,5],&#xA;xlab=&amp;quot;Soft Threshold (power)&amp;quot;,ylab=&amp;quot;Mean Connectivity&amp;quot;, type=&amp;quot;n&amp;quot;,&#xA;main = paste(&amp;quot;Mean connectivity&amp;quot;))&#xA;text(sft$fitIndices[,1], sft$fitIndices[,5], labels=powers, cex=cex1,col=&amp;quot;red&amp;quot;)&#xA;# Turn data expression into topological overlap matrix&#xA;power=sft$powerEstimate&#xA;# Even though estimate suggests 3, trying ahigh number&#xA;# Turn adjacency into topological overlap matrix (TOM)&#xA;adjacency &amp;lt;- adjacency(bryois_norm_keep_use, power = 6)&#xA;TOMadj &amp;lt;- TOMsimilarity(adjacency,verbose = 5)&#xA;dissTOMadj &amp;lt;- 1- TOMadj&#xA;# Clustering using TOM&#xA;# Call the hierarchical clustering function&#xA;hclustGeneTree &amp;lt;- hclust(as.dist(dissTOMadj), method = &amp;quot;average&amp;quot;)&#xA;# Plot the resulting clustering tree (dendogram)&#xA;sizeGrWindow(12, 9)&#xA;plot(hclustGeneTree, xlab = &amp;quot;&amp;quot;, sub = &amp;quot;&amp;quot;,&#xA;main = &amp;quot;Gene Clustering on TOM-based disssimilarity&amp;quot;,&#xA;labels = FALSE, hang = 0.04)&#xA;# Make the modules larger, so set the minimum higher&#xA;minModuleSize &amp;lt;- 30&#xA;# Module ID using dynamic tree cut&#xA;dynamicMods &amp;lt;- cutreeDynamic(dendro = hclustGeneTree,&#xA;distM = dissTOMadj,&#xA;deepSplit = 2, pamRespectsDendro = FALSE,&#xA;minClusterSize = minModuleSize,verbose = 5)&#xA;table(dynamicMods)&#xA;# Convert numeric lables into colors&#xA;dynamicColors &amp;lt;- labels2colors(dynamicMods)&#xA;table(dynamicColors)&#xA;# Plot the dendrogram and colors underneath&#xA;sizeGrWindow(8,6)&#xA;plotDendroAndColors(hclustGeneTree, dynamicColors, &amp;quot;Dynamic Tree Cut&amp;quot;,&#xA;dendroLabels = FALSE, hang = 0.03,&#xA;addGuide = TRUE, guideHang = 0.05,&#xA;main = &amp;quot;Gene dendrogram and module colors&amp;quot;)&#xA;# Calculate eigengenes&#xA;dynamic_MEList &amp;lt;- moduleEigengenes(bryois_norm_keep_use, colors = dynamicColors)&#xA;dynamic_MEs &amp;lt;- dynamic_MEList$eigengenes&#xA;library(dynamicTreeCut)&#xA;# Calculate dissimilarity of module eigengenes&#xA;dynamic_MEDiss &amp;lt;- 1-cor(dynamic_MEs)&#xA;dynamic_METree &amp;lt;- hclust(as.dist(dynamic_MEDiss))&#xA;# Plot the hclust&#xA;sizeGrWindow(7,6)&#xA;plot(dynamic_METree, main = &amp;quot;Dynamic Clustering of module eigengenes&amp;quot;,&#xA;xlab = &amp;quot;&amp;quot;, sub = &amp;quot;&amp;quot;)&#xA;######################## MERGE SIMILAR MODULES&#xA;dynamic_MEDissThres &amp;lt;- 0.95&#xA;# Plot the cut line&#xA;#abline(h = dynamic_MEDissThres, col = &amp;quot;red&amp;quot;)&#xA;# Call an automatic merging function&#xA;merge_dynamic_MEDs &amp;lt;- mergeCloseModules(bryois_norm_keep_use, dynamicColors, cutHeight = dynamic_MEDissThres, verbose = 5)&#xA;# The Merged Colors&#xA;dynamic_mergedColors &amp;lt;- merge_dynamic_MEDs$colors&#xA;# Eigen genes of the new merged modules&#xA;mergedMEs &amp;lt;- merge_dynamic_MEDs$newMEs&#xA;mergedMEs&#xA;table(dynamic_mergedColors)&#xA;sizeGrWindow(12,9)&#xA;plotDendroAndColors(hclustGeneTree, cbind(dynamicColors, dynamic_mergedColors),&#xA;c(&amp;quot;Dynamic Tree Cut&amp;quot;, &amp;quot;Merged dynamic&amp;quot;),&#xA;dendroLabels = FALSE, hang = 0.03,&#xA;addGuide = TRUE, guideHang = 0.05)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4307" LastEditorUserId="4307" LastEditDate="2022-09-27T13:35:41.487" LastActivityDate="2022-09-27T14:07:31.997" Title="How to improve this WGCNA analysis" Tags="|wgcna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19751" PostTypeId="2" ParentId="19750" CreationDate="2022-09-27T14:07:31.997" Score="3" Body="&lt;p&gt;I took a quick look and don't see anything wrong with the module merging. You could lower the merging threshold a bit - 0.95 is very high, I would use 0.9 or 0.85. Other than that though, really nothing seems out of the ordinary.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2022-09-27T14:07:31.997" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19752" PostTypeId="1" AcceptedAnswerId="19753" CreationDate="2022-09-27T20:18:47.640" Score="0" ViewCount="267" Body="&lt;p&gt;I'm new to this business, so my question may seem as dumb. I want to make an mpileup file using a reference sequence and several (two to begin with) aligned sorted BAM files. And I want the output file to have certain easy-to-read tab-delimited formatting (which is helpful for subsequent analyses of the output file):&lt;/p&gt;&#xA;&lt;p&gt;field 1 - chromosome,&lt;br /&gt;&#xA;field 2 - position,&lt;br /&gt;&#xA;field 3 - reference base,&lt;br /&gt;&#xA;field 4 - base(s) in SAMPLE1.BAM for position in field 2,&lt;br /&gt;&#xA;field 5 - number of reads supporting bases in field 4,&lt;br /&gt;&#xA;field 6 - base(s) in SAMPLE2.BAM for position in field 2,&lt;br /&gt;&#xA;field 7 - number of reads supporting bases in field 6.&lt;/p&gt;&#xA;&lt;p&gt;Something like shown below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;field1   field2   field3   field4   field5   field6   field7&#xA;Chr1     15967    T        T,A      15,17    T,_      25,_&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where _ means no alternative (or second) base (or zero reads) for this position in the BAM file. '0' or empty space (or nothing) instead of _ is ok. It's ok if information in fields 4-7 is shown not in a single field but in several fields.&lt;/p&gt;&#xA;&lt;p&gt;Which command is suitable for such a purpose?&#xA;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="16199" LastEditorUserId="73" LastEditDate="2022-09-27T23:05:36.213" LastActivityDate="2022-09-28T12:40:17.837" Title="Make mpileup file with several BAM files" Tags="|sam|mpileup|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19753" PostTypeId="2" ParentId="19752" CreationDate="2022-09-27T23:04:29.207" Score="3" Body="&lt;p&gt;Have you tried using &lt;code&gt;samtools mpileup in.bam [in2.bam [...]]&lt;/code&gt;? Here's information about the &lt;a href=&quot;https://www.htslib.org/doc/samtools-mpileup.html#Pileup_Format&quot; rel=&quot;nofollow noreferrer&quot;&gt;output format from the manual page&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;The first three columns give the position and reference:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Chromosome name.&lt;/li&gt;&#xA;&lt;li&gt;1-based position on the chromosome.&lt;/li&gt;&#xA;&lt;li&gt;Reference base at this position (this will be “N” on all lines if -f/--fasta-ref has not been used).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;em&gt;The remaining columns show the pileup data, and are repeated for each input BAM file specified:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Number of reads covering this position.&lt;/li&gt;&#xA;&lt;li&gt;Read bases.  This encodes information on matches, mismatches, indels, strand, mapping quality, and starts and ends of reads.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This format seems nearly identical to what you've asked for, so hopefully is close enough for you to tweak your downstream analyses.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="134" LastEditDate="2022-09-28T12:40:17.837" LastActivityDate="2022-09-28T12:40:17.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19754" PostTypeId="1" CreationDate="2022-09-27T23:47:26.880" Score="1" ViewCount="79" Body="&lt;p&gt;I have some raw counts from HTSeq after aligning with hg38 human reference genome. I want to do filtering in a way that the filtered count files should have the same number of lines. The reason behind this is that I want to use DESeq2 for differential gene expression which requires all the count files to be of the same length to carry out the analysis.&lt;/p&gt;&#xA;&lt;p&gt;I have written a python program to select reads greater than 5 from the second column.&lt;/p&gt;&#xA;&lt;p&gt;My count files look like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ENSG00000000003 20&#xA;ENSG00000000005 0&#xA;ENSG00000000419 123&#xA;ENSG00000000457 35&#xA;ENSG00000000460 56&#xA;ENSG00000000938 0&#xA;ENSG00000000971 0&#xA;ENSG00000001036 164&#xA;ENSG00000001084 83&#xA;ENSG00000001167 68&#xA;ENSG00000001460 8&#xA;ENSG00000001461 13&#xA;ENSG00000001497 80&#xA;ENSG00000001561 0&#xA;ENSG00000001617 39&#xA;ENSG00000001626 0&#xA;ENSG00000001629 146&#xA;ENSG00000001630 4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Python program I have written is as follows&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import sys&#xA;&#xA;inputlist = [sys.argv[1], sys.argv[2], sys.argv[3], sys.argv[4], sys.argv[5], sys.argv[6], sys.argv[7], sys.argv[8]]  &#xA;&#xA;for i in range(len(inputlist)): &#xA;    inputfile = open(inputlist[i], 'r')         &#xA;    for line in inputfile:              &#xA;        splitline = line.strip().split('\t')            &#xA;        if int(splitline[1]) &amp;gt;= 5:      &#xA;            if not splitline[0] in genelist:    &#xA;                genelist.append(splitline[0])   &#xA;    inputfile.close()&#xA;    &#xA;outputlist = open(sys.argv[9], 'w') &#xA;for gene in genelist:&#xA;    outputlist.write(gene + '\n')&#xA;outputlist.close()&#xA;    &#xA;for i in range(len(inputlist)):&#xA;    inputfile = open(inputlist[i], 'r') &#xA;    outputfile = open(sys.argv[i+10], 'w')&#xA;        for line in inputfile:      &#xA;        splitline = line.strip().split('\t')        &#xA;        if splitline[0] in genelist:                &#xA;            outputfile.write(line)  &#xA;    inputfile.close()&#xA;    outputfile.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;python filtercounts.py 'mir99-1-L1count.txt' 'mir99-1-L2count.txt' 'mir99-2-L1count.txt' 'mir99-2-L2count.txt' 'mir99-3-L1count.txt' 'mir99-3-L2count.txt' 'genelist.txt' 'mir99-1-L1filtered.txt' 'mir99-1-L2filtered.txt' 'mir99-2-L1filtered.txt'  'mir99-2-L2filtered.txt'  'mir99-3-L1filtered.txt' 'mir99-3-L2filtered.txt'&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;This program has worked well for me in past , but now I see that although we see many such reads removed but there are many zero reads or reads less than 5 in the dataset.&lt;/p&gt;&#xA;&lt;p&gt;Some of my output looks like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ENSG00000000003 0&#xA;ENSG00000000419 0&#xA;ENSG00000000457 15&#xA;ENSG00000000460 12&#xA;ENSG00000001036 0&#xA;ENSG00000001084 0&#xA;ENSG00000001167 0&#xA;ENSG00000001460 1&#xA;ENSG00000001461 0&#xA;ENSG00000001497 1&#xA;ENSG00000001617 0&#xA;ENSG00000001629 0&#xA;ENSG00000001631 0&#xA;ENSG00000002016 2&#xA;ENSG00000002330 2&#xA;ENSG00000002549 0&#xA;ENSG00000002822 0&#xA;ENSG00000002834 0&#xA;ENSG00000002919 0&#xA;ENSG00000003056 4&#xA;ENSG00000003147 0&#xA;ENSG00000003249 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any help regarding the problematic behavior of the problem will be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="9659" LastEditorUserId="73" LastEditDate="2022-09-28T23:15:48.887" LastActivityDate="2022-10-11T08:56:47.687" Title="Filtering reads greater than 5 from HT-seq count files" Tags="|python|deseq2|genomics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19755" PostTypeId="2" ParentId="19754" CreationDate="2022-09-28T01:34:10.047" Score="1" Body="&lt;p&gt;After seeing that you want to use DESeq2, I'd recommend you do the count filtering within R, after generating the count matrix. Note that DESeq2 includes functions for loading from HTSeq-count output files:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#htseq-count-input&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#htseq-count-input&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It is not necessary to filter out low count data when processing with DESeq2, but if desired this can be done after converting to a DESeq2 object:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;While it is not necessary to pre-filter low count genes before running the DESeq2 functions, there are two reasons which make pre-filtering useful: by removing rows in which there are very few reads, we reduce the memory size of the dds data object, and we increase the speed of the transformation and testing functions within DESeq2. It can also improve visualizations, as features with no information for differential expression are not plotted.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;keep &amp;lt;- apply(counts(dds), 1, function(x){all(x &amp;gt;= 5)} # for all gene counts &amp;gt;= 5&#xA;keep &amp;lt;- apply(counts(dds), 1, function(x){any(x &amp;gt;= 5)} # for any gene counts &amp;gt;= 5&#xA;dds &amp;lt;- dds[keep,]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#pre-filtering&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#pre-filtering&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note that DESeq2 cares about replicates, and will almost always pin an adjusted p-value near 1 if any conditions have 1 or 2 replicates (three is the minimum number required to identify outlier values). There's not really any way to work around that other than including more replicates for the low-replicate conditions.&lt;/p&gt;&#xA;&lt;p&gt;Another program might give different results, but that won't get around the fundamental issue: two replicates is not sufficient to determine sample variation. Either accept that p-values cannot be used as a reliable estimate of model fit this case, or add more replicates.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-10-11T08:56:47.687" LastActivityDate="2022-10-11T08:56:47.687" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19757" PostTypeId="1" AcceptedAnswerId="19763" CreationDate="2022-09-28T13:40:16.050" Score="3" ViewCount="135" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; I want to trim leading and (likely just) trailing N's from my WES (Illumina NextSeq500) reads with &lt;a href=&quot;https://cutadapt.readthedocs.io/en/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;cutadapt&lt;/code&gt;&lt;/a&gt; (&lt;code&gt;--trim-n&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;I also want to filter for reads with &amp;gt; 40%? N's (&lt;code&gt;--max-n&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; Does &lt;code&gt;cutadapt&lt;/code&gt; first remove the trailing/leading N's before removing reads with &amp;gt;X% N's? This is because if, hypothetically, 40% of the N's are on the end of the reads, I want to keep them.&lt;/p&gt;&#xA;&lt;p&gt;Cheers and thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="16204" LastEditorUserId="3848" LastEditDate="2022-09-28T13:54:56.687" LastActivityDate="2022-09-29T14:56:42.460" Title="Does cutadapt trim trailing N's first and then use max_n to filter reads?" Tags="|quality-control|trimming|cutadapt|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19759" PostTypeId="2" ParentId="19736" CreationDate="2022-09-28T14:34:10.237" Score="2" Body="&lt;p&gt;To add the &amp;quot;do not assemble your genome&amp;quot; answers. There might be specific circumstances when you would actually like to do that. For example, if the 30x you got are HiFi long reads and have a reason to suspect there are some interesting large rearrangements in your genome (rare disease?), it might be informative to actually assemble the genome and compare it to the reference to find these rearrangements.&lt;/p&gt;&#xA;&lt;p&gt;Of course there is also plenty of possibilities in between (still assuming long reads), for example to map your sequencing data to a pan-genome graph or simply use a mapper that is aware of genomic rearrangements (like &lt;a href=&quot;https://github.com/philres/ngmlr&quot; rel=&quot;nofollow noreferrer&quot;&gt;ngmlr&lt;/a&gt;) and call the &lt;a href=&quot;https://www.nature.com/articles/s41592-018-0001-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;structural variants&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;But in the end it boils down to two aspects you did not mention in your question: What would you like to learn about your genome?&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2022-09-28T14:34:10.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19760" PostTypeId="2" ParentId="11101" CreationDate="2022-09-28T15:39:23.780" Score="2" Body="&lt;p&gt;I have come across this page today with same query. After visiting here and there, finally following approach helped to resolve this issue. I must confess that I am just a beginner to R Shiny.&lt;/p&gt;&#xA;&lt;p&gt;Lets say we have ui.R, server.R and global.R.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Inside global.R, add following code:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;glimmaMDSOutput &amp;lt;- function(outputId, width = &amp;quot;100%&amp;quot;, height = &amp;quot;400px&amp;quot;) {&#xA;        htmlwidgets::shinyWidgetOutput(outputId, 'glimmaMDS', width, height, package = 'Glimma')&#xA;}&#xA;&#xA;renderglimmaMDS &amp;lt;- function(expr, env = parent.frame(), quoted = FALSE) {&#xA;        if (!quoted) { expr &amp;lt;- substitute(expr) }&#xA;        htmlwidgets::shinyRenderWidget(expr, glimmaMDSOutput, env, quoted = TRUE)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Inside ui.R, add following:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;glimmaMDSOutput(outputId = 'MDSplot')&#xA;&#xA;**Inside server.R, add following:**&#xA;output$MDSplot &amp;lt;- renderglimmaMDS({&#xA;                Glimma::glimmaMDS(x = .., labels = .., groups = .., top = .., gene.selection = &amp;quot;pairwise&amp;quot;, launch = FALSE, html = NULL, width = 1000, height = 500);&#xA;})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16206" LastEditorUserId="3848" LastEditDate="2022-09-28T16:27:52.547" LastActivityDate="2022-09-28T16:27:52.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19761" PostTypeId="2" ParentId="7007" CreationDate="2022-09-28T16:36:27.797" Score="1" Body="&lt;p&gt;I can't comment, but here's an improvement for the top answer to list more information:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda list | awk '{if(NR&amp;gt;3) printf(&amp;quot;%s=%s\n&amp;quot;, &lt;span class=&quot;math-container&quot;&gt;$1, $&lt;/span&gt;2)}' | xargs conda info | grep 'name\|version\|license'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16207" LastEditorUserId="16208" LastEditDate="2022-10-01T04:40:59.277" LastActivityDate="2022-10-01T04:40:59.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19762" PostTypeId="1" CreationDate="2022-09-29T11:51:16.353" Score="0" ViewCount="45" Body="&lt;p&gt;I was given some Kraken output files but the file extension is .out. I know how to use .KREPORT files but not the .out file. Is there any way convert it to a KREPORT file or what do you recommend?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="16212" LastActivityDate="2022-09-29T11:51:16.353" Title="How to use a .out Kraken output format" Tags="|format-conversion|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19763" PostTypeId="2" ParentId="19757" CreationDate="2022-09-29T14:56:42.460" Score="3" Body="&lt;p&gt;Yes, it behaves just like you expect, where a read with &amp;gt;40% Ns but only at the ends will still be kept (after trimming).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://cutadapt.readthedocs.io/en/stable/guide.html#read-processing-stages&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cutadapt runs in two stages&lt;/a&gt;, with read modification first and read filtering after.  So first, &lt;code&gt;--trim-n&lt;/code&gt; removes Ns at the start and end of reads are removed, and only then is a rule like &lt;code&gt;--max-n&lt;/code&gt; applied.&lt;/p&gt;&#xA;&lt;p&gt;Here's an example where the first 5 sequences have Ns at the ends and the next 5 have Ns interspersed:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;seq01  0%&#xA;CACATCAAGAGGTCCTTGGCGACTATTAGCGCCCGGCGGACCCAATCGTA&#xA;&amp;gt;seq02 20%&#xA;NNNNNCAAGAGGTCCTTGGCGACTATTAGCGCCCGGCGGACCCAANNNNN&#xA;&amp;gt;seq03 40%&#xA;NNNNNNNNNNGGTCCTTGGCGACTATTAGCGCCCGGCGGANNNNNNNNNN&#xA;&amp;gt;seq04 60%&#xA;NNNNNNNNNNNNNNNTTGGCGACTATTAGCGCCCGNNNNNNNNNNNNNNN&#xA;&amp;gt;seq05 80%&#xA;NNNNNNNNNNNNNNNNNNNNGACTATTAGCNNNNNNNNNNNNNNNNNNNN&#xA;&amp;gt;seq06 10%&#xA;CACATCANGAGGTCCTTNGCGACTATTNGCGCCCGGCNGACCCAATCNTA&#xA;&amp;gt;seq07 20%&#xA;CANATCANGAGGNCCTTNGCGANTATTNGCGCNCGGCNGACCNAATCNTA&#xA;&amp;gt;seq08 40%&#xA;CNNATCNNGAGNNCCTNNGCGNNTATNNGCGNNCGGNNGACNNAATNNTA&#xA;&amp;gt;seq09 60%&#xA;CNNANCNNGNGNNCNTNNGNGNNTNTNNGNGNNCNGNNGNCNNANTNNNA&#xA;&amp;gt;seq10 80%&#xA;CNNNNNNNGNNNNCNNNNGNNNNTNNNNGNNNNCNNNNGNNNNANNNNNA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With just read filtering:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cutadapt --quiet --max-n 0.4 seqs.fa -o -&#xA;&amp;gt;seq01  0%&#xA;CACATCAAGAGGTCCTTGGCGACTATTAGCGCCCGGCGGACCCAATCGTA&#xA;&amp;gt;seq02 20%&#xA;NNNNNCAAGAGGTCCTTGGCGACTATTAGCGCCCGGCGGACCCAANNNNN&#xA;&amp;gt;seq03 40%&#xA;NNNNNNNNNNGGTCCTTGGCGACTATTAGCGCCCGGCGGANNNNNNNNNN&#xA;&amp;gt;seq06 10%&#xA;CACATCANGAGGTCCTTNGCGACTATTNGCGCCCGGCNGACCCAATCNTA&#xA;&amp;gt;seq07 20%&#xA;CANATCANGAGGNCCTTNGCGANTATTNGCGCNCGGCNGACCNAATCNTA&#xA;&amp;gt;seq08 40%&#xA;CNNATCNNGAGNNCCTNNGCGNNTATNNGCGNNCGGNNGACNNAATNNTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With trimming also:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cutadapt --quiet --max-n 0.4 --trim-n seqs.fa -o -&#xA;&amp;gt;seq01  0%&#xA;CACATCAAGAGGTCCTTGGCGACTATTAGCGCCCGGCGGACCCAATCGTA&#xA;&amp;gt;seq02 20%&#xA;CAAGAGGTCCTTGGCGACTATTAGCGCCCGGCGGACCCAA&#xA;&amp;gt;seq03 40%&#xA;GGTCCTTGGCGACTATTAGCGCCCGGCGGA&#xA;&amp;gt;seq04 60%&#xA;TTGGCGACTATTAGCGCCCG&#xA;&amp;gt;seq05 80%&#xA;GACTATTAGC&#xA;&amp;gt;seq06 10%&#xA;CACATCANGAGGTCCTTNGCGACTATTNGCGCCCGGCNGACCCAATCNTA&#xA;&amp;gt;seq07 20%&#xA;CANATCANGAGGNCCTTNGCGANTATTNGCGCNCGGCNGACCNAATCNTA&#xA;&amp;gt;seq08 40%&#xA;CNNATCNNGAGNNCCTNNGCGNNTATNNGCGNNCGGNNGACNNAATNNTA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2022-09-29T14:56:42.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19764" PostTypeId="1" CreationDate="2022-09-29T17:49:59.020" Score="1" ViewCount="127" Body="&lt;p&gt;Following a &lt;a href=&quot;https://rnabioco.github.io/cellar/previous/2019/docs/3_norm_viz_clustering.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;single-cell RNA-seq workshop&lt;/a&gt;, I created a Seurat object (&lt;code&gt;my_data&lt;/code&gt;), normalized the data, and then tried to identify highly variable genes using two different R packages: &lt;code&gt;Seurat&lt;/code&gt; and &lt;code&gt;M3Drop&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;variable_genes_Seurat &amp;lt;- my_data %&amp;gt;%&#xA;  FindVariableFeatures(selection.method = 'vst', nfeatures = 2000) %&amp;gt;%&#xA;  VariableFeatures()&#xA;&#xA;variable_genes_M3Drop &amp;lt;- my_data %&amp;gt;%&#xA;  GetAssayData('counts') %&amp;gt;% # unnormalized&#xA;  NBumiConvertData() %&amp;gt;%&#xA;  NBumiFitModel() %&amp;gt;%&#xA;  NBumiFeatureSelectionCombinedDrop(ntop = 2000) %&amp;gt;%&#xA;  rownames()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I compared the results and found out that the gene lists were pretty different and shared only 588 of 2000 genes:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;shared_variable_genes &amp;lt;- intersect(variable_genes_Seurat, variable_genes_M3Drop)&#xA;length(shared_variable_genes)&#xA;&amp;gt; 588&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Why are the results so different?&lt;/li&gt;&#xA;&lt;li&gt;Which feature list - &lt;code&gt;shared_variable_genes&lt;/code&gt;, &lt;code&gt;variable_genes_Seurat&lt;/code&gt; or &lt;code&gt;variable_genes_M3Drop&lt;/code&gt; - should I use for dimensionality reduction and clustering?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="16216" LastActivityDate="2022-09-29T17:49:59.020" Title="Identifying highly variable genes in scRNA-seq: Seurat vs M3Drop" Tags="|seurat|single-cell|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19765" PostTypeId="1" CreationDate="2022-09-29T18:11:32.523" Score="2" ViewCount="722" Body="&lt;p&gt;I am trying to get the bioinformatics tool &lt;code&gt;deepbgc&lt;/code&gt; up and running. Now I have created a &lt;code&gt;conda&lt;/code&gt; environment and have currently installed &lt;code&gt;deepbgc&lt;/code&gt; onto it. Now when I run the tool, the program works but it outputs the following warning message as a result:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; /EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages/sklearn/base.py:306:&#xA; UserWarning: Trying to unpickle estimator RandomForestClassifier from version 0.18.2 &#xA; when using version 0.21.3. This might lead to breaking code or invalid results. Use at your own risk.  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This led me to believe that the program is trying to utilize an older version of the &lt;code&gt;scikit-learn&lt;/code&gt; package (&lt;code&gt;version 0.18.2&lt;/code&gt; instead of the currently installed 0.21.3 version) for this &lt;code&gt;RandomForestClassifer&lt;/code&gt; estimator. This pointed me to the next logical solution of downgrading my current version of &lt;code&gt;scikit-learn&lt;/code&gt;. So, I ran a simple &lt;code&gt;pip&lt;/code&gt; install of the older package version:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pip install scikit-learn==0.18.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For some reason, the package downgrade failed and outputted the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    error: Command &amp;quot;g++ -pthread -B /EFS/tools/miniconda/envs/deepbgc2/compiler_compat -&#xA;Wl,--sysroot=/ -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -&#xA;I/EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages/numpy/core/include -&#xA;I/EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages/numpy/core/include -&#xA;I/EFS/tools/miniconda/envs/deepbgc2/include/python3.7m -c &#xA;sklearn/cluster/_dbscan_inner.cpp -o build/temp.linux-x86_64-cpython-&#xA;37/sklearn/cluster/_dbscan_inner.o -MMD -MF build/temp.linux-x86_64-cpython-&#xA;37/sklearn/cluster/_dbscan_inner.o.d&amp;quot; failed with exit status 1&#xA;&#xA;      [end of output]&#xA;  &#xA;  note: This error originates from a subprocess, and is likely not a problem with pip.&#xA;  WARNING: No metadata found in /EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages&#xA;  Rolling back uninstall of scikit-learn&#xA;  Moving to /EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages/scikit_learn-0.21.3.dist-info/&#xA;   from /EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages/~cikit_learn-0.21.3.dist-info&#xA;  Moving to /EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages/sklearn/&#xA;   from /EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages/~klearn&#xA;error: legacy-install-failure&#xA;&#xA;× Encountered error while trying to install package.&#xA;╰─&amp;gt; scikit-learn&#xA;&#xA;note: This is an issue with the package mentioned above, not `pip`.&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Currently, I have &lt;code&gt;Python&lt;/code&gt; version &lt;code&gt;3.7&lt;/code&gt; installed on the environment and version 22.1.2 of &lt;code&gt;pip&lt;/code&gt;. Any guidance appreciated, thanks!&lt;/p&gt;&#xA;" OwnerUserId="16173" LastEditorUserId="3848" LastEditDate="2022-09-30T13:19:59.660" LastActivityDate="2022-09-30T17:58:56.480" Title="pip installation within conda env: Error downgrading scikit-learn - WARNING: No metadata" Tags="|python|deeptools|conda|scipy|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19766" PostTypeId="2" ParentId="19765" CreationDate="2022-09-29T19:29:23.737" Score="1" Body="&lt;p&gt;&lt;code&gt;pip&lt;/code&gt; and &lt;code&gt;conda&lt;/code&gt; don't mix and good advice is don't ever mix them. Thus to avoid this delete your old environment, create a new &lt;code&gt;conda&lt;/code&gt; environment, or use a &lt;code&gt;conda&lt;/code&gt; environment where &lt;code&gt;pip&lt;/code&gt; has not been used ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c anaconda scikit-learn=0.18.2 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you are using &lt;code&gt;pip&lt;/code&gt; it is recommended to use it only with &lt;code&gt;virtualenv&lt;/code&gt; &lt;code&gt;venv&lt;/code&gt;. If there is a dependency issue delete the environment and start over is much easier.&lt;/p&gt;&#xA;&lt;p&gt;The rationale is for not mixing them is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;conda&lt;/code&gt; installs lots of stuff not just Python and it gets the &lt;code&gt;env&lt;/code&gt; (environment) right for this purpose&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;pip&lt;/code&gt; only installs Python within a Python environment&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The simple answer is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c bioconda deepbgc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;conda will then install the correct version of scikit-learn automatically. If doing this it important it is in its own (new) &lt;code&gt;conda&lt;/code&gt; environment however.&lt;/p&gt;&#xA;&lt;p&gt;If the above doesn't work further stuff can be done, but the 'simple answer' should work.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;@Joe_G says its hanging ... on &lt;code&gt;solving environment&lt;/code&gt; just leave it ... you will then see why,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;The following NEW packages will be INSTALLED:&#xA;&#xA;  _tflow_select      pkgs/main/osx-64::_tflow_select-2.3.0-mkl&#xA;  absl-py            pkgs/main/noarch::absl-py-0.15.0-pyhd3eb1b0_0&#xA;  appdirs            pkgs/main/noarch::appdirs-1.4.4-pyhd3eb1b0_0&#xA;  astor              pkgs/main/osx-64::astor-0.8.1-py37hecd8cb5_0&#xA;  biopython          pkgs/main/osx-64::biopython-1.76-py37h1de35cc_0&#xA;  blas               pkgs/main/osx-64::blas-1.0-mkl&#xA;  bottleneck         pkgs/main/osx-64::bottleneck-1.3.5-py37h67323c0_0&#xA;  brotli             pkgs/main/osx-64::brotli-1.0.9-hca72f7f_7&#xA;  brotli-bin         pkgs/main/osx-64::brotli-bin-1.0.9-hca72f7f_7&#xA;  c-ares             pkgs/main/osx-64::c-ares-1.18.1-hca72f7f_0&#xA;  ca-certificates    pkgs/main/osx-64::ca-certificates-2022.07.19-hecd8cb5_0&#xA;  certifi            pkgs/main/osx-64::certifi-2022.9.14-py37hecd8cb5_0&#xA;  cycler             pkgs/main/noarch::cycler-0.11.0-pyhd3eb1b0_0&#xA;  dataclasses        pkgs/main/noarch::dataclasses-0.8-pyh6d0b6a4_7&#xA;  deepbgc            bioconda/noarch::deepbgc-0.1.23-py_0&#xA;  fftw               pkgs/main/osx-64::fftw-3.3.9-h9ed2024_1&#xA;  fonttools          pkgs/main/noarch::fonttools-4.25.0-pyhd3eb1b0_0&#xA;  freetype           pkgs/main/osx-64::freetype-2.11.0-hd8bbffd_0&#xA;  gast               pkgs/main/noarch::gast-0.5.3-pyhd3eb1b0_0&#xA;  giflib             pkgs/main/osx-64::giflib-5.2.1-haf1e3a3_0&#xA;  grpcio             pkgs/main/osx-64::grpcio-1.42.0-py37ha29bfda_0&#xA;  h5py               pkgs/main/osx-64::h5py-3.7.0-py37h4a1dd59_0&#xA;  hdf5               pkgs/main/osx-64::hdf5-1.10.6-h10fe05b_1&#xA;  hmmer              bioconda/osx-64::hmmer-3.3.2-h9722bc1_2&#xA;  importlib-metadata pkgs/main/osx-64::importlib-metadata-4.11.3-py37hecd8cb5_0&#xA;  intel-openmp       pkgs/main/osx-64::intel-openmp-2021.4.0-hecd8cb5_3538&#xA;  joblib             pkgs/main/noarch::joblib-1.1.0-pyhd3eb1b0_0&#xA;  jpeg               pkgs/main/osx-64::jpeg-9e-hca72f7f_0&#xA;  keras              pkgs/main/osx-64::keras-2.2.4-0&#xA;  keras-applications pkgs/main/noarch::keras-applications-1.0.8-py_1&#xA;  keras-base         pkgs/main/osx-64::keras-base-2.2.4-py37_0&#xA;  keras-preprocessi~ pkgs/main/noarch::keras-preprocessing-1.1.2-pyhd3eb1b0_0&#xA;  kiwisolver         pkgs/main/osx-64::kiwisolver-1.4.2-py37he9d5cce_0&#xA;  lcms2              pkgs/main/osx-64::lcms2-2.12-hf1fd2bf_0&#xA;  lerc               pkgs/main/osx-64::lerc-3.0-he9d5cce_0&#xA;  libbrotlicommon    pkgs/main/osx-64::libbrotlicommon-1.0.9-hca72f7f_7&#xA;  libbrotlidec       pkgs/main/osx-64::libbrotlidec-1.0.9-hca72f7f_7&#xA;  libbrotlienc       pkgs/main/osx-64::libbrotlienc-1.0.9-hca72f7f_7&#xA;  libcxx             pkgs/main/osx-64::libcxx-14.0.6-h9765a3e_0&#xA;  libdeflate         pkgs/main/osx-64::libdeflate-1.8-h9ed2024_5&#xA;  libffi             bioconda/osx-64::libffi-3.2.1-1&#xA;  libgfortran        pkgs/main/osx-64::libgfortran-5.0.0-11_2_0_h246ff09_26&#xA;  libgfortran5       pkgs/main/osx-64::libgfortran5-11.2.0-h246ff09_26&#xA;  libpng             pkgs/main/osx-64::libpng-1.6.37-ha441bb4_0&#xA;  libprotobuf        pkgs/main/osx-64::libprotobuf-3.20.1-h8346a28_0&#xA;  libtiff            pkgs/main/osx-64::libtiff-4.4.0-h2ef1027_0&#xA;  libwebp            pkgs/main/osx-64::libwebp-1.2.2-h56c3ce4_0&#xA;  libwebp-base       pkgs/main/osx-64::libwebp-base-1.2.2-hca72f7f_0&#xA;  llvm-openmp        pkgs/main/osx-64::llvm-openmp-14.0.6-h0dcd299_0&#xA;  lz4-c              pkgs/main/osx-64::lz4-c-1.9.3-h23ab428_1&#xA;  markdown           pkgs/main/osx-64::markdown-3.3.4-py37hecd8cb5_0&#xA;  matplotlib-base    pkgs/main/osx-64::matplotlib-base-3.5.2-py37hfb0c5b7_0&#xA;  mkl                pkgs/main/osx-64::mkl-2021.4.0-hecd8cb5_637&#xA;  mkl-service        pkgs/main/osx-64::mkl-service-2.4.0-py37h9ed2024_0&#xA;  mkl_fft            pkgs/main/osx-64::mkl_fft-1.3.1-py37h4ab4a9b_0&#xA;  mkl_random         pkgs/main/osx-64::mkl_random-1.2.2-py37hb2f4e1b_0&#xA;  munkres            bioconda/noarch::munkres-1.0.7-py_1&#xA;  ncurses            pkgs/main/osx-64::ncurses-6.3-hca72f7f_3&#xA;  numexpr            pkgs/main/osx-64::numexpr-2.8.3-py37h2e5f0a9_0&#xA;  numpy              pkgs/main/osx-64::numpy-1.21.5-py37h2e5f0a9_3&#xA;  numpy-base         pkgs/main/osx-64::numpy-base-1.21.5-py37h3b1a694_3&#xA;  openssl            pkgs/main/osx-64::openssl-1.1.1q-hca72f7f_0&#xA;  packaging          pkgs/main/noarch::packaging-21.3-pyhd3eb1b0_0&#xA;  pandas             pkgs/main/osx-64::pandas-1.3.5-py37h743cdd8_0&#xA;  pillow             pkgs/main/osx-64::pillow-9.2.0-py37hde71d04_1&#xA;  pip                pkgs/main/osx-64::pip-22.2.2-py37hecd8cb5_0&#xA;  prodigal           bioconda/osx-64::prodigal-2.6.3-ha5712d3_4&#xA;  protobuf           pkgs/main/osx-64::protobuf-3.20.1-py37he9d5cce_0&#xA;  pyparsing          pkgs/main/osx-64::pyparsing-3.0.9-py37hecd8cb5_0&#xA;  python             pkgs/main/osx-64::python-3.7.7-hc70fcce_0_cpython&#xA;  python-dateutil    pkgs/main/noarch::python-dateutil-2.8.2-pyhd3eb1b0_0&#xA;  pytz               pkgs/main/osx-64::pytz-2022.1-py37hecd8cb5_0&#xA;  pyyaml             pkgs/main/osx-64::pyyaml-6.0-py37hca72f7f_1&#xA;  readline           pkgs/main/osx-64::readline-8.1.2-hca72f7f_1&#xA;  scikit-learn       pkgs/main/osx-64::scikit-learn-1.0.2-py37hae1ba45_1&#xA;  scipy              pkgs/main/osx-64::scipy-1.7.3-py37h214d14d_2&#xA;  setuptools         pkgs/main/osx-64::setuptools-63.4.1-py37hecd8cb5_0&#xA;  six                pkgs/main/noarch::six-1.16.0-pyhd3eb1b0_1&#xA;  sqlite             pkgs/main/osx-64::sqlite-3.39.3-h707629a_0&#xA;  tensorboard        pkgs/main/osx-64::tensorboard-1.14.0-py37h80053f4_0&#xA;  tensorflow         pkgs/main/osx-64::tensorflow-1.14.0-mkl_py37h085be34_0&#xA;  tensorflow-base    pkgs/main/osx-64::tensorflow-base-1.14.0-mkl_py37h5a24fda_0&#xA;  tensorflow-estima~ pkgs/main/noarch::tensorflow-estimator-1.14.0-py_0&#xA;  termcolor          pkgs/main/osx-64::termcolor-1.1.0-py37hecd8cb5_1&#xA;  threadpoolctl      pkgs/main/noarch::threadpoolctl-2.2.0-pyh0d69192_0&#xA;  tk                 pkgs/main/osx-64::tk-8.6.12-h5d9f67b_0&#xA;  typing_extensions  pkgs/main/osx-64::typing_extensions-4.3.0-py37hecd8cb5_0&#xA;  werkzeug           pkgs/main/noarch::werkzeug-2.0.3-pyhd3eb1b0_0&#xA;  wheel              pkgs/main/noarch::wheel-0.37.1-pyhd3eb1b0_0&#xA;  wrapt              pkgs/main/osx-64::wrapt-1.14.1-py37hca72f7f_0&#xA;  xz                 pkgs/main/osx-64::xz-5.2.6-hca72f7f_0&#xA;  yaml               pkgs/main/osx-64::yaml-0.2.5-haf1e3a3_0&#xA;  zipp               pkgs/main/osx-64::zipp-3.8.0-py37hecd8cb5_0&#xA;  zlib               pkgs/main/osx-64::zlib-1.2.12-h4dc903c_3&#xA;  zstd               pkgs/main/osx-64::zstd-1.5.2-hcb37349_0&#xA;&#xA;Proceed ([y]/n)? &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Its an enormous data pipeline. Everything is there, thats why it takes so long.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-09-30T17:58:56.480" LastActivityDate="2022-09-30T17:58:56.480" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="19767" PostTypeId="1" CreationDate="2022-09-30T06:41:04.367" Score="1" ViewCount="26" Body="&lt;p&gt;Can prokka be used to annotate a set of protein sequences (so bulk protein annotation), or does it require an entire genome for the annotation?&lt;/p&gt;&#xA;&lt;p&gt;I've been looking through the documentation but haven't found a clear answer to this. I tested it, and the annotation of my protein sequences worked, but I wanted to check if the program expected a genome and that I can trust the output.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="16218" LastActivityDate="2022-09-30T15:09:43.313" Title="Bulk protein annotation with Prokka" Tags="|functional-annotation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19768" PostTypeId="6" CreationDate="2022-09-30T12:39:09.003" Score="0" Body="&lt;h1&gt;terdon&lt;/h1&gt;&#xA;&lt;p&gt;I am a long time Stack Exchange user, currently a mod on Unix &amp;amp; Linux, and I also used to be a mod on Ask Ubuntu and here on Bioinformatics, where I have been an active user (I am the 3rd user by rep, 3rd most prolific voter and 4th editor)  since the very beginning and was appointed pro-tem moderator when we first launched.&lt;/p&gt;&#xA;&lt;p&gt;I believe I'd make a good moderator because:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I hope my presence here has shown that I am fair, polite and patient.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I am always open to—and indeed welcome—&lt;a href=&quot;https://data.stackexchange.com/bioinformatics/query/1649158&quot; rel=&quot;nofollow noreferrer&quot;&gt;corrections&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I believe it is very important to be welcoming and polite to new users. Even when they post horrible questions or haven't read the help center. Instead of berating them, we should educate them and explain how the site works.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As a moderator on other sites, I am familiar with the mod tools and well aware of what it actually means to be a mod (more of a janitor than a ruler).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Finally, I feel I am quite good at &lt;em&gt;moderating&lt;/em&gt;: keeping the peace and encouraging civil discussions.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2022-09-30T12:43:38.137" LastActivityDate="2022-09-30T12:43:38.137" CommentCount="0" CommunityOwnedDate="2022-09-30T12:39:09.003" ContentLicense="CC BY-SA 4.0" />
  <row Id="19770" PostTypeId="1" CreationDate="2022-09-30T13:40:24.470" Score="1" ViewCount="450" Body="&lt;p&gt;A collegue of mine recently suggested to try the louvain algorithm for clustering multiplex cytometry data. However, implementations of louvain are kind of rare in R. To my knowledge the only stand-alone implementation is included in the &lt;code&gt;igraph&lt;/code&gt; package.&#xA;So far i tried to use the adjacency matrix that the &lt;code&gt;umap&lt;/code&gt; function returns (&lt;code&gt;uwot&lt;/code&gt; package) to create a graph using &lt;code&gt;igraphs&lt;/code&gt; &lt;code&gt;graph_from_adjacency_matrix&lt;/code&gt; function. I can run the louvain algorithm on the graph, but the result is always a few thousand clusters with a hand-full if cells. changing the resolution parameter does not change anything.&lt;/p&gt;&#xA;&lt;p&gt;If i remember correctly, &lt;code&gt;Seurats&lt;/code&gt; &lt;code&gt;findClusters&lt;/code&gt; function uses &lt;code&gt;louvain&lt;/code&gt;, however i don't want to use PCA reduction before clustering, which is requiered in &lt;code&gt;Seurat&lt;/code&gt; to find clusters.&lt;/p&gt;&#xA;&lt;p&gt;Does anybody know of a different implementation of louvain or leiden algorithm that or an easier way to use &lt;code&gt;igraphs&lt;/code&gt;functions? I obviously don't have to use the matrix umap returned, but what would be a better approach to make my data &amp;quot;louvain-able&amp;quot;?&lt;/p&gt;&#xA;" OwnerUserId="16220" LastEditorUserId="73" LastEditDate="2022-10-02T00:34:31.257" LastActivityDate="2022-11-01T01:08:56.063" Title="How to get started with louvain/leiden algorithm with UMAP in R" Tags="|r|seurat|clustering|umap|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19771" PostTypeId="2" ParentId="19770" CreationDate="2022-09-30T15:05:04.780" Score="1" Body="&lt;p&gt;Both conda and PyPI have leiden clustering in Python which operates via &lt;code&gt;iGraph&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c conda-forge leidenalg&#xA;&#xA;pip install leiden-clustering&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Used via&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import leidenalg as la&#xA;import igraph as ig&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Example output&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/F8xMh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/F8xMh.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The docs are &lt;a href=&quot;https://leidenalg.readthedocs.io/en/stable/intro.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. You will not need much Python to use it.&lt;/p&gt;&#xA;&lt;p&gt;There is an entire Leiden package in R-cran &lt;a href=&quot;https://cran.r-project.org/web/packages/leiden/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Python implements Rs Leiden package here,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c conda-forge r-leiden&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-09-30T15:51:59.780" LastActivityDate="2022-09-30T15:51:59.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19772" PostTypeId="2" ParentId="19767" CreationDate="2022-09-30T15:09:43.313" Score="1" Body="&lt;p&gt;Prokka doesn't need whole genomes, I've definitely used it for smaller contigs before so I think you're fine!&lt;/p&gt;&#xA;" OwnerUserId="252" LastActivityDate="2022-09-30T15:09:43.313" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19773" PostTypeId="7" CreationDate="2022-10-01T22:14:54.363" Score="0" Body="&lt;p&gt;On Stack Exchange, we believe the core moderators should come &lt;em&gt;from&lt;/em&gt; the community, and be elected by the community itself through popular vote. We hold regular elections to determine who these community moderators will be.&lt;/p&gt;&#xA;&lt;p&gt;Community moderators are &lt;a href=&quot;https://stackoverflow.blog/2009/05/18/a-theory-of-moderation/&quot;&gt;accorded the highest level of privilege on our community&lt;/a&gt;, and should themselves be exemplars of positive behavior and leaders within the community.&lt;/p&gt;&#xA;&lt;p&gt;Our general criteria for moderators is as follows:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;patient and fair&lt;/li&gt;&#xA;&lt;li&gt;leads by example&lt;/li&gt;&#xA;&lt;li&gt;shows respect for their fellow community members in their actions and words&lt;/li&gt;&#xA;&lt;li&gt;open to some light but firm moderation to keep the community on track and resolve (hopefully) uncommon disputes and exceptions&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;em&gt;Note: moderators are picked based on the preferences of the community. While the candidate scores can help inform community members about the work of the nominees on this site, they should not be the sole indicator of a good moderator. Voters are free to choose people with low candidate scores as moderators, and a high score does not guarantee that a nominee would do well as a moderator.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Full elections have three phases and an optional fourth phase (Primary):&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Question Collection&lt;/li&gt;&#xA;&lt;li&gt;Nomination&lt;/li&gt;&#xA;&lt;li&gt;Primary&lt;/li&gt;&#xA;&lt;li&gt;Election&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Please participate in the moderator elections by voting, and perhaps even by nominating yourself to be a community moderator!&lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="73" LastEditDate="2022-10-01T22:15:12.393" LastActivityDate="2022-10-01T22:15:12.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19774" PostTypeId="1" CreationDate="2022-10-02T17:37:51.463" Score="0" ViewCount="131" Body="&lt;p&gt;I am learning about &lt;a href=&quot;https://en.wikipedia.org/wiki/Genetic_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;genetic algorithms&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Why chromosomes are represented using 8 bit binary.&lt;/p&gt;&#xA;&lt;p&gt;This arose when I apply crossover techniques to the bits.&lt;/p&gt;&#xA;&lt;p&gt;Please help me to find the answer.Thanks in advance !&lt;/p&gt;&#xA;" OwnerUserId="16228" LastEditorUserId="3848" LastEditDate="2022-10-02T23:03:17.037" LastActivityDate="2022-10-03T09:10:52.187" Title="Why are genetic algorithm chromosomes represented using 8 bit binary?" Tags="|gene|gene-expression|algorithms|genetics|chromosomes|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19775" PostTypeId="2" ParentId="19774" CreationDate="2022-10-03T09:10:52.187" Score="1" Body="&lt;p&gt;Not always. The representation of chromosomes in genetic algorithms is often dependent on the specific problem to be solved and number of variables to be found through the optimization process.&lt;/p&gt;&#xA;&lt;p&gt;Not only you can chose a 16-bit representation, but integer-based representations are also used as mentioned &lt;a href=&quot;https://hub.packtpub.com/using-genetic-algorithms-for-optimizing-your-models-tutorial/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;At the computation level, using a binary representation usually facilitates cross over operations on the bit arrays. 8 bits are compact while already give you a range of &lt;a href=&quot;https://en.wikipedia.org/wiki/Chromosome_(genetic_algorithm)#Chromosome_design&quot; rel=&quot;nofollow noreferrer&quot;&gt;0-255&lt;/a&gt; different values for a given variable, thus provides a first option to go unless you need a larger value space.&lt;/p&gt;&#xA;" OwnerUserId="475" LastActivityDate="2022-10-03T09:10:52.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19776" PostTypeId="1" CreationDate="2022-10-03T13:59:43.950" Score="0" ViewCount="19" Body="&lt;p&gt;Could someone recommend me popular and reliable tools for HLA imputation?&#xA;I'm starting my master thesis project and I will infer the HLA region of 2000 sample in a case-control study; I searched for benchmarks on PubMed but I really wanted to now if someone experienced in this kind of analysis can help me&#xA;ty&lt;/p&gt;&#xA;" OwnerUserId="16233" LastActivityDate="2022-10-03T13:59:43.950" Title="reliable and popular HLA imputation tools" Tags="|imputation|tools|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19777" PostTypeId="1" CreationDate="2022-10-03T18:24:21.363" Score="1" ViewCount="62" Body="&lt;p&gt;In an &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/9768693/&quot; rel=&quot;nofollow noreferrer&quot;&gt;old article&lt;/a&gt;, they gave a table of mutations found in a gene and its positions which points to another &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC326418/pdf/nar00364-0263.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;article&lt;/a&gt; containing the FASTA sequence.&lt;/p&gt;&#xA;&lt;p&gt;This FASTA is the precursor sequence of the gene. How can I know which transcripts those mutations belong to? And how can I obtain their HGVS format?&lt;/p&gt;&#xA;&lt;p&gt;For example, &lt;code&gt;C-7111-G&lt;/code&gt; should have an HGVS notation like &lt;code&gt;c.XXXC&amp;gt;G&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UHzyl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UHzyl.png&quot; alt=&quot;Article&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OuJsU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OuJsU.png&quot; alt=&quot;FASTA&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2022-10-03T18:24:21.363" Title="How can I get the HGVS notation of a mutation from a FASTA?" Tags="|fasta|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19778" PostTypeId="1" CreationDate="2022-10-03T18:43:08.530" Score="1" ViewCount="1029" Body="&lt;p&gt;I have come across some related issues pertaining to this tensorflow warning but none related specifically to deepbgc&lt;/p&gt;&#xA;&lt;p&gt;I have created a conda environment and have installed onto it the bioinformatics analysis tool deepbgc. Now the program runs but it does output some warning messages related to the tensorflow package. When I run the &amp;quot;deepbgc info&amp;quot; command through terminal, I get the following output messages related to the tensor flow package:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Using TensorFlow backend.&#xA;WARNING 03/10 12:55:05   From /EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:74: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.&#xA;&#xA;WARNING 03/10 12:55:05   From /EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.&#xA;&#xA;WARNING 03/10 12:55:05   From /EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.&#xA;&#xA;WARNING 03/10 12:55:06   From /EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:133: The name tf.placeholder_with_default is deprecated. Please use tf.compat.v1.placeholder_with_default instead.&#xA;&#xA;WARNING 03/10 12:55:06   From /EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.&#xA;Instructions for updating:&#xA;Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.&#xA;WARNING 03/10 12:55:06   From /EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:174: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.&#xA;&#xA;WARNING 03/10 12:55:06   From /EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:181: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.&#xA;&#xA;WARNING 03/10 12:55:06   From /EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:186: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.&#xA;&#xA;WARNING 03/10 12:55:06   From /EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:190: The name tf.global_variables is deprecated. Please use tf.compat.v1.global_variables instead.&#xA;&#xA;WARNING 03/10 12:55:06   From /EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:199: The name tf.is_variable_initialized is deprecated. Please use tf.compat.v1.is_variable_initialized instead.&#xA;&#xA;WARNING 03/10 12:55:06   From /EFS/tools/miniconda/envs/deepbgc2/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:206: The name tf.variables_initializer is deprecated. Please use tf.compat.v1.variables_initializer instead.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Could this be related to the version of tensorflow? Currently, I have version 1.15.4 installed onto the conda environment. Are these error messages significant and could they impact the data running through deepbgc? Or can they mostly be ignored? Any guidance appreciated, thanks!&lt;/p&gt;&#xA;" OwnerUserId="16173" LastActivityDate="2022-10-05T21:14:43.193" Title="Warning message related to tensorflow package when trying to run deepbgc bioinformatics tool" Tags="|python|deeptools|conda|tensorflow|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19779" PostTypeId="2" ParentId="19031" CreationDate="2022-10-03T19:35:21.423" Score="1" Body="&lt;p&gt;I am inclined to agree with you that the reads-per-cell is dependent on the cell type and cell cycle state.  In the datasets I have analyzed, I observed that immune cells tend to have far fewer reads than fibroblasts, which have fewer reads than larger epithelial cell types.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: I have also observed that macrophage populations generally have more RNA than lymphocytes. Granulocytes have even less if you even capture them in your sample.&lt;/p&gt;&#xA;" OwnerUserId="64" LastActivityDate="2022-10-03T19:35:21.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19780" PostTypeId="1" AcceptedAnswerId="19797" CreationDate="2022-10-04T07:33:32.627" Score="2" ViewCount="61" Body="&lt;p&gt;This question was addressed &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/2700/how-to-map-pdb-chains-to-uniprot-ids-using-api-services&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, including &lt;code&gt;bioservices&lt;/code&gt;, the solutions &lt;code&gt;pypdb&lt;/code&gt; and &lt;code&gt;map_pdb_to_uniprot&lt;/code&gt; do not work for now.&lt;/p&gt;&#xA;&lt;p&gt;Besides, I do not need residue level mapping as one of &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2719/14169&quot;&gt;the suggested solution&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Does the failure of known methods have anything to deal with the &lt;a href=&quot;https://twitter.com/uniprot/status/1541797066120302597?s=20&amp;amp;t=3j0E6JfW9szPtLJgX96Mlw&quot; rel=&quot;nofollow noreferrer&quot;&gt;recent update&lt;/a&gt; of the UniProt database?&lt;a href=&quot;https://twitter.com/uniprot/status/1541797066120302597?s=20&amp;amp;t=3j0E6JfW9szPtLJgX96Mlw&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Please let me know if you know a workaround.&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="14169" LastActivityDate="2022-10-07T08:29:12.580" Title="Mapping PDB ID + chain ID to UniProt ID" Tags="|pdb|uniprot|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19781" PostTypeId="2" ParentId="19746" CreationDate="2022-10-04T08:34:33.483" Score="1" Body="&lt;p&gt;Putting the placeholder inside of an input file won't work, since the contents will just be turned into a string when read with &lt;code&gt;read_lines&lt;/code&gt; and won't get treated like WDL code. Since the lines from the &lt;code&gt;program_file&lt;/code&gt; are simply getting pasted into the command, you can move the contents into the command section. In a placeholder (&lt;code&gt;~{}&lt;/code&gt;), adding a string to an optional value will result in an empty string if that optional value is not supplied (see the spec &lt;a href=&quot;https://github.com/openwdl/wdl/blob/main/versions/1.0/SPEC.md#prepending-a-string-to-an-optional-parameter&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). So you can use optional inputs to make the different lines non-mandatory:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;task trimmomatic {&#xA;    input {&#xA;        File fastq1&#xA;        File fastq2&#xA;        String outfile1 = “trimmed_R1.fastq.gz”&#xA;        String outfile2 = “trimmed_R2.fastq.gz”&#xA;        File? adapter_file&#xA;        Int? minlen&#xA;        String? slidingwindow&#xA;        Int? leading&#xA;        Int? trailing&#xA;     }&#xA; &#xA;     command {&#xA;        TrimmomaticPE \&#xA;        ~{fastq1} ~{fastq2} \&#xA;        ~{outfile1} ~{outfile2} \&#xA;        ~{&amp;quot;ILLUMINACLIP:&amp;quot; + adapter_file + &amp;quot;:2:30:10:7:keepBothReads&amp;quot;} \&#xA;        ~{&amp;quot;MINLEN:&amp;quot; + minlen} \&#xA;        ~{&amp;quot;SLIDINGWINDOW:&amp;quot; + slidingwindow} \&#xA;        ~{&amp;quot;LEADING:&amp;quot; + leading} \&#xA;        ~{&amp;quot;TRAILING:&amp;quot; + trailing}&#xA;     }&#xA; &#xA;     output {&#xA;        File out1 = outfile1&#xA;        File out2 = outfile2&#xA;     }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If, say, &lt;code&gt;adapter_file&lt;/code&gt; were to be omitted in the inputs, then the produced command should look something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TrimmomaticPE \&#xA;R1.fq R2.fq \&#xA;trimmed_R1.fastq.gz trimmed_R2.fastq.gz \&#xA; \&#xA;MINLEN:36 \&#xA;SLIDINGWINDOW:4:15 \&#xA;LEADING:3 \&#xA;TRAILING:3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Also note that the output file paths in the input section should be &lt;code&gt;String&lt;/code&gt;s. Since they don't exist yet, the execution engine will complain that it can't find them if they are &lt;code&gt;File&lt;/code&gt;s&lt;/p&gt;&#xA;&lt;p&gt;Ps. I'm not familiar with the actual trimmomatic command options, so please do check that this produces the desired command. I suspect that the &lt;code&gt;:2:30:10:7:keepBothReads&lt;/code&gt; bit could be split up into additional inputs as well.&lt;/p&gt;&#xA;" OwnerUserId="6955" LastActivityDate="2022-10-04T08:34:33.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19782" PostTypeId="1" CreationDate="2022-10-04T12:23:44.247" Score="4" ViewCount="63" Body="&lt;p&gt;Note: I've never submitted an assembly/genome to NCBI, so excuse if my perspective is flawed.&lt;/p&gt;&#xA;&lt;p&gt;I'm working with &lt;em&gt;Drosophila subobscura&lt;/em&gt;. (spring fruit fly)&lt;/p&gt;&#xA;&lt;p&gt;I see here &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/data-hub/genome/?taxon=7241&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/data-hub/genome/?taxon=7241&lt;/a&gt; that there are at least 2 assemblies of better quality than the only available genome (I checked checked N50 and number of scaffolds).&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/data-hub/genome/GCA_903684685.1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/data-hub/genome/GCA_903684685.1/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/data-hub/genome/GCA_018903505.1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/data-hub/genome/GCA_018903505.1/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, these assemblies weren't promoted to genomes.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What is required to consider them genomes?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it the scaffold to chromosome notation that is missing? Or also the annotations?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How would I promote the better assemblies into genomes?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;And also, if I wanted to annotate one of the better assemblies, how would I proceed?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9373" LastEditorUserId="3848" LastEditDate="2022-10-04T15:46:47.993" LastActivityDate="2024-02-27T02:02:20.253" Title="How to promote assemblies into genomes in NCBI?" Tags="|genome|assembly|phylogenetics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19783" PostTypeId="2" ParentId="2700" CreationDate="2022-10-04T14:01:42.167" Score="2" Body="&lt;p&gt;While not as nice as using SIFT. You can use the &lt;a href=&quot;https://sparql.uniprot.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;uniprot sparql&lt;/a&gt; endpoint as well to get mappings.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;PREFIX up: &amp;lt;http://purl.uniprot.org/core/&amp;gt;&#xA;SELECT&#xA;  ?pdbId ?chain ?pdbChain ?uniprot&#xA;WHERE&#xA;{&#xA;  # A space separated list of pairs of PDB identifiers and the chain code.&#xA;  VALUES(?pdbId ?pdbChain) { ('6VXC' 'A') ('1BG3' 'B') }&#xA;&#xA;  # Make an IRI out of the pdbId&#xA;  BIND(iri(concat('http://rdf.wwpdb.org/pdb/', ?pdbId)) AS ?pdb)&#xA;&#xA;  # Map to UniProt entries&#xA;  ?uniprot rdfs:seeAlso ?pdb .&#xA;  ?pdb up:database &amp;lt;http://purl.uniprot.org/database/PDB&amp;gt; ;&#xA;   up:chainSequenceMapping ?chainSm .&#xA;  ?chainSm up:chain ?chainsPlusRange .&#xA;&#xA;  # Extract the list of chains from the text representation.&#xA;  BIND(STRBEFORE(?chainsPlusRange, '=') AS ?chain)&#xA;&#xA;  # Filter those that match.&#xA;  FILTER(CONTAINS(?chain, ?pdbChain))&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2499" LastActivityDate="2022-10-04T14:01:42.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19785" PostTypeId="2" ParentId="19782" CreationDate="2022-10-04T22:03:55.460" Score="1" Body="&lt;p&gt;The reason is simple, insect genomes are often heavily fragmented and historically relied on wet-lab experimentation - polytene chromosome hybridisation to correcty assemble the chromosomes. Thus you probe the various contigs, prep the contigs often from salivary glands, stick them under a microscope and map the location of the probes. Hence, the reluctance to assign a 'genome' based on contigs because the chromosomes cannot be assembled. This is particularly true for a very close relative to the premier model of the Insecta &lt;em&gt;D. melanogaster&lt;/em&gt;. A lot of work will have gone into the correct chromosome annotation of this genome. Thus, you can't simply 'promote' a genome without further genomics analysis and it will be a community decision based on the analysis (i.e. reviewers).&lt;/p&gt;&#xA;&lt;p&gt;Bioinformatic assemblies often get it right. What I would do is use a reference genome assembly based on the nearest ancestor, which you'll know more than me, I'm pretty sure its not &lt;em&gt;D. melanogaster&lt;/em&gt;. This is within the context of a comparative genomic analysis between the species/subspecies associated with &lt;em&gt;D. subobscura&lt;/em&gt; and then say to NCBI 'thats a genome'. You then have to publish the analysis and submit your assembled genome to NCBI. Its not trivial, but if you 'live next door' to one of the genetic models of eukaryotes (well metazoa) the standards of annotation and assembly will be high.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-10-04T23:09:23.960" LastActivityDate="2022-10-04T23:09:23.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19786" PostTypeId="2" ParentId="19778" CreationDate="2022-10-04T22:18:13.840" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;The 'blunderbus' way to do this is as follows (not recommended in this instance :-) ),&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;    conda update --all&#xA;    conda update -n base conda&#xA;    conda update -n myenv --all&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;The refined way to do this is&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;    conda update -n myenv tensorflow # see `Best guess, notes&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In theory &lt;code&gt;conda&lt;/code&gt; was supposed to sort out the deprecation, because all the versions were supposed to be harmonious. I'm a bit puzzled (see below). What might have happened is &lt;code&gt;tensorflow&lt;/code&gt; has been upgraded (or &lt;code&gt;keras&lt;/code&gt;) against the version the developer had written it for.&lt;/p&gt;&#xA;&lt;p&gt;Here &lt;code&gt;myenv&lt;/code&gt; is the name you gave that given conda environment.&lt;/p&gt;&#xA;&lt;p&gt;In your case, I would look at point 2, because the environment is very specific to your project and ensures reproducibility.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Note&lt;/em&gt; A purist would specify the exact channel &lt;code&gt;-c&lt;/code&gt; that &lt;code&gt;tensorflow&lt;/code&gt; is found in, or whatever dependency. This actually can be very important because there can be name conflicts, obviously not for &lt;code&gt;tensorflow&lt;/code&gt; because I'm pretty sure Google have the name registered.&lt;/p&gt;&#xA;&lt;p&gt;The reason I do point 1 is the problem might a dependency of &lt;code&gt;tensorflow&lt;/code&gt; and you'd have to figure it out, but again see 'notes'.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Best guess&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this particular example,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda update -n myenv keras&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Might be the ticket, because what I think is going on is its not actually &lt;code&gt;tensorflow&lt;/code&gt; but the &lt;code&gt;keras&lt;/code&gt; sitting above it.&lt;/p&gt;&#xA;&lt;p&gt;The alternative (probably better) guess is ....&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda update -n myenv -c bioconda deepbgc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is probably what I would try first.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; I'm surprised that &lt;code&gt;conda&lt;/code&gt;/&lt;code&gt;mamba&lt;/code&gt; has not resolved this issue on installation because thats exactly what is it is supposed to do. In other words its supposed to make a perfect installation of &lt;code&gt;deepbgc&lt;/code&gt; precisely as the developer had intended it. There is a risk that updating might not be the solution ... and solution is a new &lt;code&gt;conda&lt;/code&gt;/&lt;code&gt;mamba&lt;/code&gt; installation (see note 2).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note 2&lt;/strong&gt; Watch for the &lt;code&gt;Python&lt;/code&gt; version, you are using 3.7 ... that might be an issue. If the developer had written this for &amp;gt;&lt;code&gt;3.7&lt;/code&gt; that could cause problems. Different Python versions have different methods and if one of the 'new' methods was being used that would be an issue. Ultimately your  package/pipeline is really complex and the more stuff the more chance stuff doesn't fit together. We're currently on &lt;code&gt;3.10&lt;/code&gt; and &lt;code&gt;3.9&lt;/code&gt; is really stable.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;BTW&lt;/strong&gt; &lt;code&gt;3.11&lt;/code&gt; will be a major upgrade of &lt;code&gt;Python3&lt;/code&gt; and will be the 'goto' version.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I looked at &lt;code&gt;deepbgc&lt;/code&gt; its built on &lt;code&gt;Python3.7&lt;/code&gt;, I don't know why because it was released in 2019. My advice is start again with a new install and follow the instructions exactly as on Github. The order of the build is really important (omitting pip use &lt;code&gt;conda&lt;/code&gt;/&lt;code&gt;mamba&lt;/code&gt; instead). It doesn't seem to be very &lt;code&gt;conda&lt;/code&gt; but thats how the developer did it. In theory the whole pipeline could have been placed on &lt;code&gt;conda&lt;/code&gt;. Instructions here &lt;a href=&quot;https://github.com/Merck/deepbgc&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Merck/deepbgc&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The key issue is that it is built on an old build of Python, with a lot of dependencies which have substancially changed. There is concern about ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda create -n deepbgc python=3.7 hmmer prodigal&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;because the author is not specifying the versions of hmmer or prodigal. These should have been built into the &lt;code&gt;conda&lt;/code&gt; upload.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-10-05T21:14:43.193" LastActivityDate="2022-10-05T21:14:43.193" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19787" PostTypeId="1" CreationDate="2022-10-05T09:26:08.593" Score="1" ViewCount="39" Body="&lt;p&gt;A human &amp;quot;X&amp;quot; chromosome has a centromere and two &amp;quot;identical&amp;quot; chromatids. If the chromatids are not identical, this fact is not assembled, correct?&lt;/p&gt;&#xA;&lt;p&gt;The fasta string for a chromomsome is actually representing just one of the chromatids, correct?&lt;/p&gt;&#xA;&lt;p&gt;So from my understanding, for diploids, the fasta chromosome is actually a mix of 4 chromatids: 2chromatids * 2 pairs of chromosomes into one string. Which for some heterozygous nucleotides (same percentage of each allele) will just be randomly selected from the reads as the correct one.&lt;/p&gt;&#xA;&lt;p&gt;(unless we are talking about some methods like &lt;strong&gt;10x&lt;/strong&gt; that can distinguish between chromosomes pairs)&lt;/p&gt;&#xA;&lt;p&gt;I was thinking which side of the X is read first into the fasta string, and I came to the above conclusion.&lt;/p&gt;&#xA;" OwnerUserId="9373" LastActivityDate="2022-10-05T22:45:43.950" Title="How is an X chromosome encoded into a fasta string?" Tags="|assembly|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19788" PostTypeId="1" CreationDate="2022-10-05T09:54:15.330" Score="1" ViewCount="15" Body="&lt;p&gt;I want to run the experiments in mentioned in &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0168952518301240#bib0205&quot; rel=&quot;nofollow noreferrer&quot;&gt;enter the matrix&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For the the plot in figure 3B it mentioned that&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The relative absolute value of the magnitude of each element in the&#xA;pattern matrix indicates the extent to which that sample contributes&#xA;to the corresponding source of variation. The sign of the values&#xA;indicate over- or underexpression in that factor depending on the sign&#xA;of the corresponding gene weights in the amplitude matrix. As a&#xA;result, the values can be plotted on the y axis against known&#xA;covariates on the x axis to directly interpret the relationship&#xA;between samples. When applied to GTEx, we observe one pattern&#xA;associated with cerebellum, another pattern that has large positive&#xA;values for one donor and large negative values for another donor, and&#xA;eight other patterns associated with other sources of variation&#xA;(supplemental information online).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I calculated the ICA with &lt;code&gt;head()&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;        PC1       PC2&#xA;0  1.317276 -0.682285&#xA;1  1.696110 -0.549004&#xA;2  0.037709 -0.548736&#xA;3  0.702004 -0.545255&#xA;4  1.227363 -0.574219&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but fail to see how the authors plotted their ICA graphs&lt;/p&gt;&#xA;" OwnerUserId="2026" LastActivityDate="2022-10-05T09:54:15.330" Title="What visualization was used for ICA in the tissue experiment?" Tags="|gene|unsupervised-learning|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19789" PostTypeId="1" AcceptedAnswerId="19790" CreationDate="2022-10-05T15:23:19.927" Score="1" ViewCount="34" Body="&lt;p&gt;I would like to add &lt;code&gt;n&lt;/code&gt; (e.g., 1k) bases to a PyRange object (i.e., shifting PyRanges coordinates).&#xA;How can I do that?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Input:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pyranges as pr&#xA;gr = pr.random(3)&#xA;&#xA;# gr:&#xA;# +--------------+-----------+-----------+--------------+&#xA;# | Chromosome   |     Start |       End | Strand       |&#xA;# | (category)   |   (int32) |   (int32) | (category)   |&#xA;# |--------------+-----------+-----------+--------------|&#xA;# | chr1         |  10000000 |  20000000 | -            |&#xA;# | chr2         |  20000000 |  30000000 | +            |&#xA;# | chr22        |  30000000 |  40000000 | -            |&#xA;# +--------------+-----------+-----------+--------------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Desired output:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;# magical command[gr + 1000]:&#xA;# +--------------+-----------+-----------+--------------+&#xA;# | Chromosome   |     Start |       End | Strand       |&#xA;# | (category)   |   (int32) |   (int32) | (category)   |&#xA;# |--------------+-----------+-----------+--------------|&#xA;# | chr1         |  10001000 |  20001000 | -            |&#xA;# | chr2         |  20001000 |  30001000 | +            |&#xA;# | chr22        |  30001000 |  40001000 | -            |&#xA;# +--------------+-----------+-----------+--------------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16242" LastActivityDate="2022-10-05T17:30:38.160" Title="How to add N bases to PyRange object (i.e., shifting PyRanges coordinates)" Tags="|python|pyranges|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19790" PostTypeId="2" ParentId="19789" CreationDate="2022-10-05T17:12:03.337" Score="2" Body="&lt;p&gt;You can use the &lt;code&gt;.extend()&lt;/code&gt; method. Starting with your data above, &lt;code&gt;gr.extend({&amp;quot;3&amp;quot;: -10000, &amp;quot;5&amp;quot;: 10000})&lt;/code&gt; would generate the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;+--------------+-----------+-----------+--------------+&#xA;| Chromosome   |     Start |       End | Strand       |&#xA;| (category)   |   (int32) |   (int32) | (category)   |&#xA;|--------------+-----------+-----------+--------------|&#xA;| chr1         |  10010000 |  20010000 | -            |&#xA;| chr2         |  19990000 |  29990000 | +            |&#xA;| chr22        |  30010000 |  40010000 | -            |&#xA;+--------------+-----------+-----------+--------------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I don't know a way of to apply this &amp;quot;range extension&amp;quot; in a strand specific way (to be able to generate the desired output exactly) but if this is really needed, you should be able to generate 2 &lt;code&gt;gr&lt;/code&gt; objects, one for each strand, do the manipulation and bring these together.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2022-10-05T17:30:38.160" LastActivityDate="2022-10-05T17:30:38.160" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19791" PostTypeId="2" ParentId="19787" CreationDate="2022-10-05T22:45:43.950" Score="2" Body="&lt;p&gt;So, the two chromatids you see only during meta-phase of replication. In more &amp;quot;regular&amp;quot; state of a cell, there are only the two haplotypes (in the case of a diploid organism)&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://upload.wikimedia.org/wikipedia/commons/thumb/3/39/Condensation_and_resolution_of_human_sister_chromatids_in_early_mitosis.svg/1024px-Condensation_and_resolution_of_human_sister_chromatids_in_early_mitosis.svg.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;Well, and regardless of the ploidy, we usually aim for a haploid (monoploid) representation, so it is usually exactly as you say - an arbitrary one of the two states on each heterozygous site is chosen.&lt;/p&gt;&#xA;&lt;p&gt;However, there are also approaches to phase genomes (to sort which alleles are on which haplotypes). That is possible not just with 10x, but also with long reads (to certain extend) or using &lt;a href=&quot;https://www.nature.com/articles/nbt.4277&quot; rel=&quot;nofollow noreferrer&quot;&gt;trio-binning&lt;/a&gt; - a strategy to sequence two parents and the offspring.&lt;/p&gt;&#xA;&lt;p&gt;However, even phased genomes still contain the reference (fasta) in a haploid state and usually provide the alternative haplotype as a second fasta file.&lt;/p&gt;&#xA;&lt;p&gt;For the record, these apply to all chromosomes, not just the X.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2022-10-05T22:45:43.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19792" PostTypeId="2" ParentId="18034" CreationDate="2022-10-06T06:56:58.613" Score="1" Body="&lt;p&gt;&lt;em&gt;Answer from @james-hawley, converted from &lt;a href=&quot;https://bioinformatics.stackexchange.com/help/privileges/comment&quot;&gt;comment&lt;/a&gt;:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;In any high throughput sequencing experiment, there will be lowly expressed RNA molecules that are not captured by the library prep + sequencing. So defining what &amp;quot;is expressed&amp;quot; is somewhat arbitrary from RNA-seq data, regardless of how many replicates you have and what method you use. That said, you can control some of the noise in your RNA-seq data using a variance-stabilized transformation, like @gringer suggests, and look at the distribution of genes/transcripts. Some threshold like &lt;span class=&quot;math-container&quot;&gt;$rlog \ge x$&lt;/span&gt; in at least &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; samples is usually a good starting point (for some arbitrary &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt;).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-10-06T06:56:58.613" CommentCount="0" CommunityOwnedDate="2022-10-06T06:56:58.613" ContentLicense="CC BY-SA 4.0" />
  <row Id="19793" PostTypeId="1" CreationDate="2022-10-06T10:02:39.403" Score="0" ViewCount="433" Body="&lt;p&gt;I'm new to single cell sequencing analysis. I have made different clusters using seurat. Now I would like to extract the raw gene counts based on the cluster generated by seurat.&#xA;i have provided my R code used in the analysis.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(dplyr)&#xA;library(Seurat)&#xA;library(patchwork)&#xA;ftp=&amp;quot;https://ftp.ncbi.nlm.nih.gov/geo/series/GSE201nnn/GSE201254/suppl/GSE201254_count_mat_Control-NoCrush.csv.gz&amp;quot;&#xA;counts=data.table::fread(ftp)&#xA;C_nocrush=counts %&amp;gt;%&#xA;select(contains(c(&amp;quot;V1&amp;quot;,&amp;quot;CtCReRGCS&amp;quot;)))&#xA;rownames(C_nocrush)=C_nocrush$V1&#xA;C_nocrush$V1=NULL&#xA;CNC=CreateSeuratObject(counts = C_nocrush,project = &amp;quot;He RGC Data&amp;quot;, min.cells = 3, min.features = 200)&#xA;CNC[[&amp;quot;percent.mt&amp;quot;]] = PercentageFeatureSet(CNC, pattern = &amp;quot;^MT-&amp;quot;)&#xA;VlnPlot(CNC, features = c(&amp;quot;nFeature_RNA&amp;quot;, &amp;quot;nCount_RNA&amp;quot;,&amp;quot;percent.mt&amp;quot;), ncol = 3)&#xA;plot1 &amp;lt;- FeatureScatter(CNC, feature1 = &amp;quot;nCount_RNA&amp;quot;, feature2 = &amp;quot;percent.mt&amp;quot;)&#xA;plot2 &amp;lt;- FeatureScatter(CNC, feature1 = &amp;quot;nCount_RNA&amp;quot;, feature2 = &amp;quot;nFeature_RNA&amp;quot;)&#xA;plot1+plot2&#xA;CNC = subset(CNC, subset = nFeature_RNA &amp;gt; 200 &amp;amp; nFeature_RNA &amp;lt; 2500 &amp;amp; percent.mt &amp;lt; 5)&#xA;CNC = NormalizeData(CNC, normalization.method = &amp;quot;LogNormalize&amp;quot;, scale.factor = 10000)&#xA;CNC = FindVariableFeatures(CNC, selection.method = &amp;quot;vst&amp;quot;, nfeatures = 2000)&#xA;top10 &amp;lt;- head(VariableFeatures(CNC), 10)&#xA;plot1 &amp;lt;- VariableFeaturePlot(CNC)&#xA;plot2 &amp;lt;- LabelPoints(plot = plot1, points = top10, repel = TRUE)&#xA;plot1+plot2&#xA;all.genes &amp;lt;- rownames(CNC)&#xA;CNC &amp;lt;- ScaleData(CNC, features = all.genes)&#xA;CNC &amp;lt;- RunPCA(CNC, features = VariableFeatures(object = CNC))&#xA;VizDimLoadings(CNC, dims = 1:2, reduction = &amp;quot;pca&amp;quot;)&#xA;DimPlot(CNC, reduction = &amp;quot;pca&amp;quot;)&#xA;DimHeatmap(CNC, dims = 1:15, cells = 500, balanced = TRUE)&#xA;CNC &amp;lt;- JackStraw(CNC, num.replicate = 100)&#xA;CNC &amp;lt;- ScoreJackStraw(CNC, dims = 1:20)&#xA;JackStrawPlot(CNC, dims = 1:15)&#xA;ElbowPlot(CNC)&#xA;CNC &amp;lt;- FindNeighbors(CNC, dims = 1:10)&#xA;CNC &amp;lt;- FindClusters(CNC, resolution = 0.5)&#xA;head(Idents(CNC), 5)&#xA;CNC &amp;lt;- RunUMAP(CNC, dims = 1:10)&#xA;DimPlot(CNC, reduction = &amp;quot;umap&amp;quot;)&#xA;saveRDS(CNC, file = &amp;quot;Desktop/Single_cell_RNA/CNC.rds&amp;quot;)&#xA;cluster2.markers &amp;lt;- FindMarkers(CNC, ident.1 = 2, min.pct = 0.25)&#xA;head(cluster2.markers, n = 5)&#xA;new.cluster.ids=c(&amp;quot;T5/F RGCs&amp;quot;, &amp;quot;T5/F RGCs&amp;quot;,&amp;quot;T-RGCs&amp;quot;,&amp;quot;Other RGcs and cartpt RGCs&amp;quot;,&amp;quot;T5/F RGCs&amp;quot;,&amp;quot;IP-RGCs&amp;quot;,&amp;quot;Other RGcs and cartpt RGCs&amp;quot;,&amp;quot;N-RGCs&amp;quot;,&amp;quot;Alpha-RGCs&amp;quot; )&#xA;names(new.cluster.ids)=levels(CNC)&#xA;CNC=RenameIdents(CNC,new.cluster.ids)&#xA; DimPlot(CNC,reduction = &amp;quot;umap&amp;quot;, label = T, pt.size = 0.5)+NoLegend()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Tried using this command to extract the raw counts based on the cluster but is not working for me.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;meta.data.cluster=unique(x=CNC@meta.data$active.ident)&#xA;meta.data.cluster=unique(Idents(CNC))&#xA;for(group in meta.data.cluster) {&#xA;    group.cells &amp;lt;- WhichCells(object = CNC, idents = group)&#xA;    data_to_write_out &amp;lt;- as.data.frame(GetAssayData(CNC,slot = 'counts')[,group.cells])&#xA;    write.csv(data_to_write_out, row.names = TRUE, file = &amp;quot;1_cluster_outfile.csv&amp;quot;)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16246" LastEditorUserId="16246" LastEditDate="2022-10-06T12:46:30.980" LastActivityDate="2022-10-06T12:46:30.980" Title="extracting raw gene counts based on the cluster generated by seurat" Tags="|scrnaseq|seurat|single-cell|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19794" PostTypeId="1" AcceptedAnswerId="19796" CreationDate="2022-10-06T16:00:51.060" Score="3" ViewCount="412" Body="&lt;p&gt;I am trying to search the NCBI non-redundant database for sequences similar to a few other (~40) sequences that I already have.&lt;/p&gt;&#xA;&lt;p&gt;So I've tried running blastn remotely, and looping through multiple queries. I've tried it both as a job (via slurm) and from the command line.&lt;/p&gt;&#xA;&lt;p&gt;In addition, for every query, I am trying to run 3 searches with slightly different parameters (in WebBlast, I noticed that I recover more sequences of interest this way).&lt;/p&gt;&#xA;&lt;p&gt;I am trying to dump the output data in a single file (stuff like duplicates can be sorted out later).&lt;/p&gt;&#xA;&lt;p&gt;This is my code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in /directory/*.fa;do #this is the directory containing the queries&#xA;j=${i#&amp;quot;/directory/&amp;quot;}&#xA;k=${j%&amp;quot;.fa&amp;quot;} #getting a short version of the name for the log file&#xA;echo &amp;quot;$k&amp;quot; &amp;gt;&amp;gt; &amp;quot;log.log&amp;quot;&#xA;blastn -query &amp;quot;$i&amp;quot; -db &amp;quot;nt&amp;quot; -remote -entrez_query 700:1000[slen] -perc_identity 94 -outfmt &amp;quot;6 sacc sseqid sseq evalue bitscore length&amp;quot; &amp;gt;&amp;gt; &amp;quot;results.tsv&amp;quot;&#xA;blastn -query &amp;quot;$i&amp;quot; -db &amp;quot;nt&amp;quot; -remote -entrez_query 1000:1400[slen] -perc_identity 94 -outfmt &amp;quot;6 sacc sseqid sseq evalue bitscore length&amp;quot; &amp;gt;&amp;gt; &amp;quot;results.tsv&amp;quot;&#xA;blastn -query &amp;quot;$i&amp;quot; -db &amp;quot;nt&amp;quot; -remote -perc_identity 94 -outfmt &amp;quot;6 sacc sseqid sseq evalue bitscore length&amp;quot; &amp;gt;&amp;gt; &amp;quot;results.tsv&amp;quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, it...doesn't seem to work? Either only the first search for the first query is completed, and then the code does nothing else, or nothing happens. (in a rare case, the first two searches worked) There seems to be no clear distinction between 'it works a tiny bit' and 'it doesn't work at all', which leads me to think it might be something outside the code, e.g. connection issues.&lt;/p&gt;&#xA;&lt;p&gt;Has anyone faced this before, and would you know how to fix it?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&#xA;I tried to run a single search as well. As well as adding the waiting period. Right now it doesn't work (same issue, no error message, just sort of appearing to run without results). In the link that Scot provided below, from the NCBI, it was mentioned 'We will move searches of users who submit more than 100 searches in a 24 hour period to a slower queue, or, in extreme cases, will block the requests.' I haven't run &amp;gt;100 searches yesterday (was planning to before I knew this, but that's a different issue).&lt;/p&gt;&#xA;&lt;p&gt;I think I'll limit the searches as recommended and try to run it from a different computer later on.&lt;/p&gt;&#xA;" OwnerUserId="4401" LastEditorUserId="4401" LastEditDate="2022-10-07T11:18:01.037" LastActivityDate="2022-10-08T00:12:46.483" Title="Remote blastn - what is breaking my (bash) loop?" Tags="|blast|bash|loop|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19795" PostTypeId="2" ParentId="19794" CreationDate="2022-10-07T01:28:40.670" Score="2" Body="&lt;p&gt;This is likely a throttling issue - see &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?CMD=Web&amp;amp;PAGE_TYPE=BlastDocs&amp;amp;DOC_TYPE=DeveloperInfo&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for some details, specifically the one about contacting the NCBI server more than once every 10 seconds.&lt;/p&gt;&#xA;&lt;p&gt;You may be able to submit a multiFASTA of all your queries to get around this issue, rather than individual FASTA sequences, or alternatively, add in a &lt;code&gt;sleep 10&lt;/code&gt; command between your BLAST commands and at the end of each &lt;code&gt;for&lt;/code&gt; loop iteration to throttle the requests.&lt;/p&gt;&#xA;" OwnerUserId="92" LastEditorUserId="92" LastEditDate="2022-10-07T01:38:51.927" LastActivityDate="2022-10-07T01:38:51.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19796" PostTypeId="2" ParentId="19794" CreationDate="2022-10-07T01:43:10.310" Score="2" Body="&lt;p&gt;I gave it a whirl using a random bit of SARS-CoV-2&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastn -query out.fa -db &amp;quot;nt&amp;quot; -remote  -outfmt &amp;quot;6 sacc sseqid sseq evalue bitscore length&amp;quot; &amp;gt;&amp;gt; &amp;quot;results.tsv&amp;quot;&#xA;&#xA;blastn -query out.fa -db &amp;quot;nt&amp;quot; -remote -perc_identity 94 -outfmt &amp;quot;6 sacc sseqid sseq evalue bitscore length&amp;quot; &amp;gt;&amp;gt; &amp;quot;results2.tsv&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Both gave a 500 line output - the limit of the blast specified.&lt;/p&gt;&#xA;&lt;p&gt;One possibility is to place a &lt;code&gt;sleep 10&lt;/code&gt; line between each &lt;code&gt;blastn&lt;/code&gt; command (3 additional lines).&lt;/p&gt;&#xA;&lt;p&gt;The rationale is specifications NCBI set:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Do not contact the server more often than &lt;strong&gt;once every 10 seconds&lt;/strong&gt;. Do&#xA;not poll for any single RID more often than once a minute. Use the URL&#xA;parameter email and tool, so that the NCBI can contact you if there is&#xA;a problem. Run scripts weekends or between 9 pm and 5 am Eastern time&#xA;on weekdays if more than 50 searches will be submitted.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The suggestions haven't worked, so I'd recommend trying &lt;code&gt;sleep 60&lt;/code&gt; between each &lt;code&gt;blastn&lt;/code&gt;command on the basis this might be an issue with &lt;code&gt;RID&lt;/code&gt;. If that fails the fall back is to run the job overnight US time via &lt;code&gt;cron tab&lt;/code&gt;, which I'll post the code if it is required. Thus,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Run scripts weekends or between 9 pm and 5 am Eastern time&#xA;on weekdays if more than 50 searches will be submitted.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The final thing to add is that it is a fairly long job so connectivity might be an issue. If this was the case then it might also be resolved by running the script overnight in the OPs timezone, i.e. via &lt;code&gt;cron&lt;/code&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;code&gt;sleep 60&lt;/code&gt; works but is very slow.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Agreed one option is to download the database and perform the Blast locally.&lt;/li&gt;&#xA;&lt;li&gt;Alternative use &lt;a href=&quot;https://github.com/soedinglab/MMseqs2&quot; rel=&quot;nofollow noreferrer&quot;&gt;mmseqs2&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;You could perform the Blast on the Cloud, like GCP (much faster).&lt;/li&gt;&#xA;&lt;li&gt;Personally I would look at performing over night runs when the network was not in use.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-10-08T00:12:46.483" LastActivityDate="2022-10-08T00:12:46.483" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19797" PostTypeId="2" ParentId="19780" CreationDate="2022-10-07T08:29:12.580" Score="0" Body="&lt;p&gt;Please find the solution here:&#xA;&lt;a href=&quot;https://www.biostars.org/p/9540519/#9540826&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/9540519/#9540826&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14169" LastActivityDate="2022-10-07T08:29:12.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19798" PostTypeId="1" CreationDate="2022-10-07T14:14:13.547" Score="2" ViewCount="24" Body="&lt;p&gt;I have a bam file with reads, and a set of features. The particularity is that some features are &amp;quot;split&amp;quot;, being made of several sub-features. For example, here &lt;code&gt;bed1&lt;/code&gt; and &lt;code&gt;bed2&lt;/code&gt; are made of a simple feature, &lt;code&gt;bed3&lt;/code&gt; contains a split feature:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;     0        10        20&#xA;     |---------|---------|&#xA;bam   &amp;gt;&amp;gt;  &#xA;       &amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&#xA;           &amp;gt;&amp;gt;&#xA;     |---------|---------|&#xA;bed1 ####&#xA;bed2             #####&#xA;bed3 ####        #####&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to count the reads overlapping with each (set of) feature, but importantly I want to count only once a read that intersects both features in a set:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;feature   count&#xA;   bed1       2&#xA;   bed2       1&#xA;   bed3       2   -&amp;gt; NOT 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In R, I can do that with a GRangesList where &lt;code&gt;bed1&lt;/code&gt; and &lt;code&gt;bed2&lt;/code&gt; are list elements containing 1 GRange, whereas &lt;code&gt;bed3&lt;/code&gt; is a list element containing 2 GRanges; find the reads that intersect a list element, then use &lt;code&gt;unique()&lt;/code&gt; to avoid double-counting reads that intersect two GRanges in the same list element:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;suppressPackageStartupMessages(library(GenomicAlignments))&#xA;exple_beds &amp;lt;- GRangesList(list(GRanges(&amp;quot;I&amp;quot;,&#xA;                         ranges =  IRanges(start = 0, end = 3),&#xA;                         strand = &amp;quot;+&amp;quot;,&#xA;                         names = &amp;quot;bed1-a&amp;quot;),&#xA;                 GRanges(&amp;quot;I&amp;quot;,&#xA;                         ranges = IRanges(start = 12, end = 16),&#xA;                         strand = &amp;quot;+&amp;quot;,&#xA;                         names = &amp;quot;bed2-a&amp;quot;),&#xA;                 GRanges(&amp;quot;I&amp;quot;,&#xA;                         ranges = IRanges(start = c(1,12), end = c(3,16)),&#xA;                         strand = &amp;quot;+&amp;quot;,&#xA;                         names = c(&amp;quot;bed3-a&amp;quot;, &amp;quot;bed3-b&amp;quot;)))) |&amp;gt;&#xA;  setNames(c(&amp;quot;bed1&amp;quot;,&amp;quot;bed2&amp;quot;,&amp;quot;bed3&amp;quot;))&#xA;&#xA;&#xA;&#xA;exple_bam &amp;lt;- GAlignments(seqnames = Rle(rep(&amp;quot;I&amp;quot;, 3)),&#xA;            strand = Rle(rep(factor(&amp;quot;+&amp;quot;, levels = c(&amp;quot;+&amp;quot;,&amp;quot;-&amp;quot;,&amp;quot;*&amp;quot;)),3)),&#xA;            pos = c(1L, 2L, 6L),&#xA;            cigar = c(&amp;quot;2M&amp;quot;,&amp;quot;12M&amp;quot;,&amp;quot;2M&amp;quot;))&#xA;&#xA;&#xA;sapply(exple_beds,&#xA;       \(.x) findOverlaps(.x, exple_bam) |&amp;gt;&#xA;         to() |&amp;gt;&#xA;         unique() |&amp;gt;&#xA;         length())&#xA;#&amp;gt; bed1 bed2 bed3 &#xA;#&amp;gt;    2    1    2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;sup&gt;Created on 2022-10-07 by the &lt;a href=&quot;https://reprex.tidyverse.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;reprex package&lt;/a&gt; (v2.0.1)&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;p&gt;However this gets really slow with a big BAM file.&lt;/p&gt;&#xA;&lt;p&gt;So, is there&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;either a more efficient way to do it in R,&lt;/li&gt;&#xA;&lt;li&gt;or a way to do it with bedtools and friends without creating 100,000 BED files?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="9497" LastActivityDate="2022-10-07T14:14:13.547" Title="Efficiently count reads overlapping a split feature" Tags="|bioconductor|bedtools|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19799" PostTypeId="1" AcceptedAnswerId="19801" CreationDate="2022-10-07T15:14:02.753" Score="3" ViewCount="280" Body="&lt;p&gt;I don't know if I'm in the right place but I have a technical problem to fix. I would have to align &lt;strong&gt;paired-end reads&lt;/strong&gt; from Illumina sequencing to compare a normal genome with a tumor one.&lt;/p&gt;&#xA;&lt;p&gt;When I align with &lt;strong&gt;bwa-mem (default parameters) with paired-end&lt;/strong&gt; mode I get a bigWig file where the coverage of the two samples is quite similar. When I align with &lt;strong&gt;bwa-mem (default parameters) with single-end&lt;/strong&gt; mode I get coverage holes (definitely does not align any read) in some specific regions. But this only happens in tumor genomes.&lt;/p&gt;&#xA;&lt;p&gt;Is it possible that something is really going on in that region? Or is it just a bias? And if it's a bias, why in the normal condition, in that specific region, do they align?&lt;/p&gt;&#xA;&lt;p&gt;I think I understand that by treating paired-end reads as single-end I lose the directionality of mate2 if I align mate 1. But I still can't understand why I have this alignment defect only in the tumor genome and not in the normal genome (I observe this phenomenon in all the samples analyzed, not just in one).&lt;/p&gt;&#xA;&lt;p&gt;Could someone give me some explanation or recommend any checks to do?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;spoiler&lt;/strong&gt; - these holes fall exactly in highly repetitive regions&lt;/p&gt;&#xA;" OwnerUserId="16250" LastEditorUserId="298" LastEditDate="2022-10-07T15:51:25.080" LastActivityDate="2022-10-08T14:32:40.610" Title="What can be the bias of aligning paired-reads in a single-end mode?" Tags="|sequence-alignment|sam|fastq|bwa|somatic|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19800" PostTypeId="1" AcceptedAnswerId="19807" CreationDate="2022-10-07T22:49:58.527" Score="1" ViewCount="60" Body="&lt;p&gt;I have recently been working with variants which are represented in a non-standard format. This alternative format uses a period instead of prefixing variants with a reference base. What is this format called?&lt;/p&gt;&#xA;&lt;p&gt;There is no specific output format associated with it. It's part of some calling code I've inherited. I figured that this existed elsewhere. But, I could be wrong.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Expected Format:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Insertion:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pos=4&#xA;alt=ATTT&#xA;ref=A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Deletion:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pos=4&#xA;alt=A&#xA;ref=ATTT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Alternative Format:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Insertion:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pos=5&#xA;alt=TTT&#xA;ref=.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Deletion:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pos=5&#xA;alt=.&#xA;ref=TTT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16253" LastEditorUserId="73" LastEditDate="2022-10-09T11:37:08.447" LastActivityDate="2022-10-10T11:50:43.680" Title="Name this alternative INDEL format" Tags="|variant-calling|variants|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19801" PostTypeId="2" ParentId="19799" CreationDate="2022-10-08T11:11:41.430" Score="3" Body="&lt;p&gt;In principle one should simply not align paired-end reads as if they were single end reads, not only because you are not taking advantage of the forward/reverse information contained in the reads, but also you are not taking into account the expected distance between the two pairs, both of which result in better mapping rates/accuracy.&lt;/p&gt;&#xA;&lt;p&gt;Having said that, and not recalling from the top of my head right now the relevant BWA-mem algorithmic implementation details, it is generally difficult to map (even) paired-reads to repetitive regions. Still there will be some rate of success. In tumor samples however, we could reasonably imagine that chromosomal rearrangements, INDELs, CNVs, etc. will make it even harder to confidently assign these reads to a location in the reference, since these extra variation will increase e.g. multiple mapping locations and mapping qualities for some fraction of the reads, thus resulting in regions with low or no coverage compared to non-tumor samples.&lt;/p&gt;&#xA;&lt;p&gt;Repetitive regions might exacerbate the problem and get affected even further compared to other regions.&lt;/p&gt;&#xA;" OwnerUserId="475" LastActivityDate="2022-10-08T11:11:41.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19802" PostTypeId="2" ParentId="19799" CreationDate="2022-10-08T14:32:40.610" Score="1" Body="&lt;p&gt;I tried to give me this explanation: with single-end alignment of paired-end reads, losing the directionality of the reads and also the expected fragment length, the R2 no longer maps if it maps the R1, but can map as a single reads anywhere, or don't map at all.&#xA;I thought that therefore I don't see alignment (in tumors) in repetitive regions because of the high mutation rate I have in the tumor sample. While in normal I don't have this situation.&#xA;However, aligning with paired-end reads, I see coverage in these regions again despite the high mutation rate because R1 and R2 help each other map in the best position, and so even if I have mutations I still see the alignment.&#xA;Could it be appropriate to make a variant call despite being on repetitive regions?&lt;/p&gt;&#xA;" OwnerUserId="16250" LastActivityDate="2022-10-08T14:32:40.610" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19803" PostTypeId="1" CreationDate="2022-10-08T16:25:51.330" Score="2" ViewCount="404" Body="&lt;p&gt;I recently read that original the Smith-Waterman algorithm to do a local pairwise alignment of sequences has a complexity of &lt;span class=&quot;math-container&quot;&gt;$O(m^2 \times n)$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; and &lt;span class=&quot;math-container&quot;&gt;$n$&lt;/span&gt; would be the lengths of the sequences with &lt;span class=&quot;math-container&quot;&gt;$m$&lt;/span&gt; the potentially longer one. What I am wondering is where the additional complexity arises in the algorithm compared to the Needelmann-Wunsch algorithm of global alignment, which has complexity &lt;span class=&quot;math-container&quot;&gt;$O(m \times n)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Am I correct to assume the &lt;span class=&quot;math-container&quot;&gt;$m\times n$&lt;/span&gt; for the Needelmann-Wunsch algorithm comes from building up the scoring matrix - whereas the traceback is negligible in complexity i.e. &lt;span class=&quot;math-container&quot;&gt;$O(m\times n + m)$&lt;/span&gt;? Hence we arrive at &lt;span class=&quot;math-container&quot;&gt;$O(m\times n)$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Why is there a &lt;span class=&quot;math-container&quot;&gt;$m^2$&lt;/span&gt; term for the Smith-Watermann algorithm?&lt;/p&gt;&#xA;" OwnerUserId="14957" LastActivityDate="2022-10-08T22:43:10.083" Title="Why does the Smith-Waterman algorithm have a complexity of $O(m^2 \times n)$?" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19804" PostTypeId="2" ParentId="18034" CreationDate="2022-10-08T18:58:06.617" Score="1" Body="&lt;p&gt;I don't think there is a very good way to do what you want to do.  I'd think some kind of RPM would be better than strictly raw counts; you have to account for overall library size.  Or TPM might be better.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2022-10-08T18:58:06.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19805" PostTypeId="2" ParentId="19803" CreationDate="2022-10-08T22:43:10.083" Score="2" Body="&lt;p&gt;I think your source is wrong. You are correct to assume the brunt of its complexity comes from building the &lt;span class=&quot;math-container&quot;&gt;$m *n$&lt;/span&gt; matrix. The main differences are in the calculations per cell and the traceback, but these don't make &lt;span class=&quot;math-container&quot;&gt;$m ^2$&lt;/span&gt;. Wikipedia also gives an identical &lt;span class=&quot;math-container&quot;&gt;$O(m * n)$&lt;/span&gt; as worst case for both algorithms.&lt;/p&gt;&#xA;&lt;p&gt;edit:&#xA;Aparently in &lt;a href=&quot;https://www.sciencedirect.com/science/article/abs/pii/0022283682903989?via%3Dihub&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; the algorithms were improved to go from &lt;span class=&quot;math-container&quot;&gt;$m^2n$&lt;/span&gt; to &lt;span class=&quot;math-container&quot;&gt;$m*n$&lt;/span&gt;&lt;/p&gt;&#xA;" OwnerUserId="3818" LastActivityDate="2022-10-08T22:43:10.083" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19806" PostTypeId="1" CreationDate="2022-10-09T08:58:22.417" Score="2" ViewCount="299" Body="&lt;p&gt;I am trying to plot a stacked bar graph with mean values from multiple repeats, also showing the range of the data and values labelled.&lt;/p&gt;&#xA;&lt;p&gt;My data frame is below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;structure(list(Average.DB.per.FA = c(0, 0.5, 1, 1.5, 2, 0, 0.5, &#xA;1, 1.5, 2, 0, 0.5, 1, 1.5, 2, 0, 0.5, 1, 1.5, 2, 0, 0.5, 1, 1.5, &#xA;2, 0, 0.5, 1, 1.5, 2), condition = c(&amp;quot;CDD&amp;quot;, &amp;quot;CDD&amp;quot;, &amp;quot;CDD&amp;quot;, &amp;quot;CDD&amp;quot;, &#xA;&amp;quot;CDD&amp;quot;, &amp;quot;CDD&amp;quot;, &amp;quot;CDD&amp;quot;, &amp;quot;CDD&amp;quot;, &amp;quot;CDD&amp;quot;, &amp;quot;CDD&amp;quot;, &amp;quot;CDD&amp;quot;, &amp;quot;CDD&amp;quot;, &amp;quot;CDD&amp;quot;, &#xA;&amp;quot;CDD&amp;quot;, &amp;quot;CDD&amp;quot;, &amp;quot;STD&amp;quot;, &amp;quot;STD&amp;quot;, &amp;quot;STD&amp;quot;, &amp;quot;STD&amp;quot;, &amp;quot;STD&amp;quot;, &amp;quot;STD&amp;quot;, &amp;quot;STD&amp;quot;, &#xA;&amp;quot;STD&amp;quot;, &amp;quot;STD&amp;quot;, &amp;quot;STD&amp;quot;, &amp;quot;STD&amp;quot;, &amp;quot;STD&amp;quot;, &amp;quot;STD&amp;quot;, &amp;quot;STD&amp;quot;, &amp;quot;STD&amp;quot;), percentage = c(0, &#xA;0.807218011, 0.192781989, 0, 0, 0, 0.737968015, 0.262031985, &#xA;0, 0, 0, 0.739096101, 0.260903899, 0, 0, 0, 0.466751289, 0.346156739, &#xA;0.170617491, 0.016474482, 0, 0.462997344, 0.331833715, 0.183359127, &#xA;0.021809813, 0.012053568, 0.47623097, 0.307175237, 0.182459607, &#xA;0.022080618)), row.names = c(NA, -30L), class = &amp;quot;data.frame&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried to use the following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    ggplot(long_DF, aes(fill=Average.DB.per.FA, y=percentage, x=condition)) + &#xA;  geom_bar(position=position_dodge(), stat=&amp;quot;identity&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But the graph I obtained does not have bars add up to 1:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/OAc5O.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OAc5O.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would like to have graphs like below (which I created by Prism using the same set of data but Prism does not allow me to label bars with numbers):&#xA;&lt;a href=&quot;https://i.stack.imgur.com/pjEOg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pjEOg.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is there any way to reproduce the graph by Prism using R and add values to each stacked bar?&lt;/p&gt;&#xA;&lt;p&gt;Appreciate your help and thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="16260" LastEditorUserId="3694" LastEditDate="2022-10-10T19:01:23.990" LastActivityDate="2023-07-07T22:02:05.343" Title="How to plot stacked bar chart using R showing mean with range and labelled values?" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19807" PostTypeId="2" ParentId="19800" CreationDate="2022-10-09T11:35:50.587" Score="1" Body="&lt;p&gt;The alternative deletion representation that you are describing, i.e. using &lt;code&gt;.&lt;/code&gt; to represent alternate alleles that are not present, is allowed within the definition of the VCF format (in other words, this representation can be found in a standard format). An example of this is provided on the &lt;a href=&quot;https://en.wikipedia.org/wiki/Variant_Call_Format&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia page for VCF&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM POS      ID         REF   ALT    QUAL  FILTER&#xA;20     1230237  .          T     .      47    PASS&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;While missing values are allowed for &lt;em&gt;ALT&lt;/em&gt;, they are not typically used because they lead to ambiguous variant definitions.&lt;/p&gt;&#xA;&lt;p&gt;For missing &lt;em&gt;REF&lt;/em&gt; values, the &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VCF format specification&lt;/a&gt; has the following explanation:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Each base must be one of A,C,G,T,N (case insensitive). Multiple bases are permitted. The value in the POS field refers to the position of the first base in the String. For simple insertions and deletions in which either the REF or one of the ALT alleles would otherwise be null/empty, the REF and ALT Strings must include the base before the event (which must be reflected in the POS field), unless the event&#xA;occurs at position 1 on the contig in which case it must include the base after the event....&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;To work around this ambiguity, VCF files are commonly left-normalised so that the specified sequences and variants can be precisely placed within the reference genome. Left normalisation can be carried out using &lt;a href=&quot;https://samtools.github.io/bcftools/bcftools.html#norm&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;bcftools norm&lt;/code&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;More details on left normalisation can be found here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://genome.sph.umich.edu/wiki/Variant_Normalization#Left_alignment&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genome.sph.umich.edu/wiki/Variant_Normalization#Left_alignment&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A variant is normalized if and only if it is parsimonious and left aligned.&lt;/li&gt;&#xA;&lt;li&gt;A variant is parsimonious if and only if it is represented in as few nucleotides as possible&#xA;without an allele of length 0.&lt;/li&gt;&#xA;&lt;li&gt;A variant is left aligned if and only if it is no longer possible to shift its position to the left while keeping the length of all its alleles constant.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In other words, these are &lt;em&gt;&lt;strong&gt;non-parsimonious variants&lt;/strong&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-10-10T11:50:43.680" LastActivityDate="2022-10-10T11:50:43.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19808" PostTypeId="2" ParentId="19806" CreationDate="2022-10-10T12:13:35.427" Score="1" Body="&lt;p&gt;It seems like this is missing a grouping variable in the aesthetics definition. The R help page for &lt;code&gt;position_dodge&lt;/code&gt; mentions the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Position_dodge() requires the grouping variable to be be specified in the global or ⁠geom_*⁠ layer. Unlike position_dodge(), position_dodge2() works without a grouping variable in a layer. position_dodge2() works with bars and rectangles, but is particulary useful for arranging box plots, which can have variable widths.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Bearing this in mind, setting &lt;code&gt;group=Average.DB.per.FA&lt;/code&gt; seems to give the result you want:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;long_DF %&amp;gt;%&#xA;  ggplot() +&#xA;  aes(fill=Average.DB.per.FA,&#xA;      y=percentage, x=condition, group=Average.DB.per.FA) +&#xA;  geom_col()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... although this is adding up to 3 rather than 1, as would be expected from looking at the category counts:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; long_DF %&amp;gt;% &#xA;+   group_by(condition) %&amp;gt;%&#xA;+   summarise(sumPct = sum(percentage))&#xA;# A tibble: 2 × 2&#xA;  condition sumPct&#xA;  &amp;lt;chr&amp;gt;      &amp;lt;dbl&amp;gt;&#xA;1 CDD            3&#xA;2 STD            3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NGRES.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NGRES.png&quot; alt=&quot;stacked column plot with grouping based on Average.DB.per.FA&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note: as seen above, I'm using &lt;code&gt;geom_col&lt;/code&gt;instead of &lt;code&gt;geom_bar&lt;/code&gt; with a different statistical summary method. If you're using &lt;em&gt;stat=%something%&lt;/em&gt;, there's probably a better way to approach the task. The following is from the first paragraph of the R help page for &lt;code&gt;geom_bar&lt;/code&gt;/&lt;code&gt;geom_col&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;There are two types of bar charts: geom_bar() and geom_col(). geom_bar() makes the height of the bar proportional to the number of cases in each group (or if the weight aesthetic is supplied, the sum of the weights). If you want the heights of the bars to represent values in the data, use geom_col() instead. geom_bar() uses stat_count() by default: it counts the number of cases at each x position. geom_col() uses stat_identity(): it leaves the data as is.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-10-10T12:13:35.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19809" PostTypeId="1" AcceptedAnswerId="19811" CreationDate="2022-10-10T14:39:20.363" Score="1" ViewCount="180" Body="&lt;p&gt;I am in the process of writing up a bash script that runs three bioinformatics tools and then executes a python script. This python script compiles some of the data from the informatics tools output into a single merged output file. Now I need to run this on a large number of samples at a time so I am trying to set up the script to read through the directory path to where all the output from the tools is stored.&lt;/p&gt;&#xA;&lt;p&gt;These are the directory paths for the python script input files:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;EFS/Analyses/BGC.test/Sample_Name/antismash2/sample.LargeContigs.gbk&#xA;EFS/Analyses/BGC.test/Sample_Name/Bagel4/00.OverviewGeneTables.json&#xA;EFS/Analyses/BGC.test/Sample_Name/Deepbgc/Deepbgc.bgc.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now there are multiple Sample_Name directories containing identical structures in the EFS/Analyses/BGC.test path. I would like to run the script on all of them&lt;/p&gt;&#xA;&lt;p&gt;To do this, I have written a &lt;code&gt;for&lt;/code&gt; loop within the bash script as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sample_paths=&lt;span class=&quot;math-container&quot;&gt;$(find $&lt;/span&gt;EFS/Analyses/BGC.test -mindepth 1 -maxdepth 1 -type d \ )&#xA;while read sample_paths; &#xA;do&#xA;   sample=&lt;span class=&quot;math-container&quot;&gt;$(basename $&lt;/span&gt;sample_paths)&#xA;   GenBank_Input=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$sample/antismash2/$&lt;/span&gt;sample.LargeContigs.gbk&amp;quot;;&#xA;   TsvFile_Input=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$sample/Deepbgc/Deepbgc.bgc.tsv&quot;;&#xA;   Json_Input=&quot;$&lt;/span&gt;sample/Bagel4/00.OverviewGeneTables.json&amp;quot;;&#xA;done &amp;lt;&amp;lt;&amp;lt; “$sample_paths”&#xA;&#xA;export GenBank_Input&#xA;export TsvFile_Input&#xA;export Json_Input&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The goal here being to define the sample path for every sample and then for every sample, looping through to the specific directories of each input file. I then exported the variables and used os.environ from the os package to use them in the python script as such:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    #parse gbk input file&#xA;    for record in SeqIO.parse(os.environ[&amp;quot;GenBank_Input&amp;quot;], 'genbank'):&#xA;    #parse tsv input file&#xA;    file_handle = open(os.environ[&amp;quot;TsvFile_Input&amp;quot;])&#xA;    #parse json input file&#xA;    json_data = json.load(open(os.environ[&amp;quot;Json_Input&amp;quot;]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now the error I get when I run the script is a &lt;code&gt;FileNotFoundError: [Errno 2] No such file or directory: '“”/antismash2/“”.LargeContigs.gbk'&lt;/code&gt; so the error seems pretty clear cut. The issue seems to be with the path that I defined within the while loop variables but as of right now, I have been unable to resolve. I am trying to familiarize myself more with bash scripting as I am still fairly new to it. Any guidance on how I might adjust the code to resolve this and get the script working is much appreciated, thanks!&lt;/p&gt;&#xA;" OwnerUserId="16173" LastEditorUserId="73" LastEditDate="2022-10-12T11:29:03.233" LastActivityDate="2022-10-12T11:29:03.233" Title="Running a Python Script from a bash script on multiple input files across different directory paths" Tags="|python|bash|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19811" PostTypeId="2" ParentId="19809" CreationDate="2022-10-10T15:57:37.620" Score="2" Body="&lt;p&gt;Your loop is more complicated than you need and it won't work properly anyway since you're using the same variable (&lt;code&gt;sample_paths&lt;/code&gt;) to iterate and save. Try this instead:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for dir in EFS/Analyses/BGC.test/*/; do ... done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Next, all you really need is the sample names. The rest can be inferred:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sample_names=&amp;quot;&amp;quot;&#xA;for dir in EFS/Analyses/BGC.test/*/; do&#xA;    sample_names+=&amp;quot;$(basename &amp;quot;$dir&amp;quot;) &amp;quot;&#xA;done&#xA;## remove the extra space added in the end&#xA;sample_names=$(sed 's/  *$//' &amp;lt;&amp;lt;&amp;lt; &amp;quot;$sample_names&amp;quot;)&#xA;export sample_paths&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then, in python, you would do:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;import os&#xA;&#xA;base_dir=&amp;quot;EFS/Analyses/BGC.test&amp;quot;&#xA;for sample_name in os.environ[&amp;quot;sample_names&amp;quot;].split(' '):&#xA;    gbk_file = &amp;quot;&amp;quot;&amp;quot;%s/%s/antismash2/%s.LargeContigs.gbk&amp;quot;&amp;quot;&amp;quot; % (&#xA;        base_dir, sample_name, sample_name&#xA;    )&#xA;    tsv_input = &amp;quot;&amp;quot;&amp;quot;%s/%s/Deepbgc/Deepbgc.bgc.tsv&amp;quot;&amp;quot;&amp;quot; % (base_dir, sample_name)&#xA;    json_input = &amp;quot;&amp;quot;&amp;quot;%s/%s//Bagel4/00.OverviewGeneTables.json&amp;quot;&amp;quot;&amp;quot; % (base_dir, sample_name)&#xA;    print(&amp;quot;============== Sample Name:&amp;quot;,sample_name, &amp;quot;=============&amp;quot;)&#xA;    print(&amp;quot;GenBank file:&amp;quot;, gbk_file)&#xA;    print(&amp;quot;Tsv file:&amp;quot;, tsv_input)&#xA;    print(&amp;quot;Tsv file:&amp;quot;, json_input)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I created the following test directories:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ tree&#xA;.&#xA;└── EFS&#xA;    └── Analyses&#xA;        └── BGC.test&#xA;            ├── Sample_1&#xA;            │   ├── antismash2&#xA;            │   │   └── sample.LargeContigs.gbk&#xA;            │   ├── Bagel4&#xA;            │   │   └── 00.OverviewGeneTables.json&#xA;            │   └── Deepbgc&#xA;            │       └── Deepbgc.bgc.tsv&#xA;            ├── Sample_2&#xA;            │   ├── antismash2&#xA;            │   │   └── sample.LargeContigs.gbk&#xA;            │   ├── Bagel4&#xA;            │   │   └── 00.OverviewGeneTables.json&#xA;            │   └── Deepbgc&#xA;            │       └── Deepbgc.bgc.tsv&#xA;            └── Sample_3&#xA;                ├── antismash2&#xA;                │   └── sample.LargeContigs.gbk&#xA;                ├── Bagel4&#xA;                │   └── 00.OverviewGeneTables.json&#xA;                └── Deepbgc&#xA;                    └── Deepbgc.bgc.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And running the two snippets of code above produces:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ foo.py&#xA;============== Sample Name: Sample_1 =============&#xA;GenBank file: EFS/Analyses/BGC.test/Sample_1/antismash2/Sample_1.LargeContigs.gbk&#xA;Tsv file: EFS/Analyses/BGC.test/Sample_1/Deepbgc/Deepbgc.bgc.tsv&#xA;Tsv file: EFS/Analyses/BGC.test/Sample_1//Bagel4/00.OverviewGeneTables.json&#xA;============== Sample Name: Sample_2 =============&#xA;GenBank file: EFS/Analyses/BGC.test/Sample_2/antismash2/Sample_2.LargeContigs.gbk&#xA;Tsv file: EFS/Analyses/BGC.test/Sample_2/Deepbgc/Deepbgc.bgc.tsv&#xA;Tsv file: EFS/Analyses/BGC.test/Sample_2//Bagel4/00.OverviewGeneTables.json&#xA;============== Sample Name: Sample_3 =============&#xA;GenBank file: EFS/Analyses/BGC.test/Sample_3/antismash2/Sample_3.LargeContigs.gbk&#xA;Tsv file: EFS/Analyses/BGC.test/Sample_3/Deepbgc/Deepbgc.bgc.tsv&#xA;Tsv file: EFS/Analyses/BGC.test/Sample_3//Bagel4/00.OverviewGeneTables.json&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;All this said, I don't really understand why you are using the shell at all. You can implement all of this directly in python:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;import os&#xA;&#xA;base_dir=&amp;quot;EFS/Analyses/BGC.test&amp;quot;&#xA;sample_names = os.listdir('EFS/Analyses/BGC.test/')&#xA;&#xA;for sample_name in sample_names:&#xA;    gbk_file = &amp;quot;&amp;quot;&amp;quot;%s/%s/antismash2/%s.LargeContigs.gbk&amp;quot;&amp;quot;&amp;quot; % (&#xA;        base_dir, sample_name, sample_name&#xA;    )&#xA;    tsv_input = &amp;quot;&amp;quot;&amp;quot;%s/%s/Deepbgc/Deepbgc.bgc.tsv&amp;quot;&amp;quot;&amp;quot; % (base_dir, sample_name)&#xA;    json_input = &amp;quot;&amp;quot;&amp;quot;%s/%s//Bagel4/00.OverviewGeneTables.json&amp;quot;&amp;quot;&amp;quot; % (base_dir, sample_name)&#xA;    &#xA;    # do something with these files here&#xA; &#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2022-10-10T15:57:37.620" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19812" PostTypeId="1" AcceptedAnswerId="19830" CreationDate="2022-10-11T06:13:36.627" Score="0" ViewCount="77" Body="&lt;p&gt;I need some suggestions regarding an analysis. I might be wrong as I have no stats background.&lt;/p&gt;&#xA;&lt;p&gt;I have list of exon with differential usage across samples. I can find lots of denovo mutations from both proband and siblings in those exons. Now I am trying to check was there any enrichment of proband specific denovo mutations in those exons. I can think of doing a chi square test. But I am not sure what will be expected value in chi square test. Imagine I am having 700 denovo proband mutation in 650 samples out of 1500 probands and 300 denovo mutations in 320 unaffected  siblings, out of 1200. Any suggestions? Thanks&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;Denovo&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;WT&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;Total&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Proband&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;553&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1202&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1755&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Unaffected sibs&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;505&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;813&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1318&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;&lt;strong&gt;Total&lt;/strong&gt;&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1058&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;2015&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;3073&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;" OwnerUserId="4951" LastEditorUserId="1157" LastEditDate="2023-05-09T12:23:41.777" LastActivityDate="2023-05-09T12:23:41.777" Title="Regarding chi square?" Tags="|statistics|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19813" PostTypeId="2" ParentId="19812" CreationDate="2022-10-11T08:27:13.980" Score="1" Body="&lt;p&gt;It seems reasonable, it depends how you do the Chi2.&lt;/p&gt;&#xA;&lt;p&gt;The Chi2 distribution depends on a good estimate of the expected (E). Here your observed (O) is the &amp;quot;affecteds&amp;quot; and you want to assess whether this has a suspected elevated  mutation rate, then that looks a meaningful measure.&#xA;There's two approaches&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Your E is the unaffected siblings and E is calculated by averaging this value. You could also perform the reverse calculation where E was the average for the affecteds and O is the unaffected. There will be an issue about the degrees of freedom, which would require some thought.&lt;/li&gt;&#xA;&lt;li&gt;There is an alternative way to perform Chi2, which I forget because O and E are matched (both siblings), here there is no averaging. You'd need to look it up, the numerator is the summed difference for each sibling group ... but I can't remember.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Point 2 would be a good test, but you'd need to check the test.&lt;/p&gt;&#xA;&lt;p&gt;A phylogeneticist would also use the Chi2 in a very different test. They would draw a tree and designate two rates, one for affected and unaffected. If there was elevated mutation rate in the affected it could be tested on a likelihood ratio test, i.e. the rate of evolution was increased. This can be done because the test tree would be one where the rate was the same between affecteds and unaffected progeny. The way the test works is the likelihood difference between the two trees is assessed and because the number of increased parameters (degrees of freedom) is known and log probabilities can be subtracted from each other and directly assessed (I think its degrees of freedom + 1 ... 1 additional parameter so its 2) - and therefore the difference in likelihood is tested directly against the Chi2. Its kinda neat and exploits the behaviour of log probabilities.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-10-12T02:57:17.723" LastActivityDate="2022-10-12T02:57:17.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19814" PostTypeId="1" CreationDate="2022-10-11T10:30:49.593" Score="1" ViewCount="159" Body="&lt;p&gt;I have been having some trouble with running command line BLASTn remotely. Therefore, upon recommendations, and considering I have &amp;gt;100 searches planned, I am considering just downloading the nt database (thankfully possible).&lt;/p&gt;&#xA;&lt;p&gt;My issue is that one of the settings that proved valuable with my data in WebBlast is running an Entrez query. I get more useful results for each query when instead of one search, I run three, with each of the sub-searches delimited by subject sequence length (e.g. &lt;code&gt;-entrez_query 700:1000[slen]&lt;/code&gt; to only get the results which are between 700 and 1000 nucleotides long).&lt;/p&gt;&#xA;&lt;p&gt;WebBlast and remote Blast have that, local Blast does not.&lt;/p&gt;&#xA;&lt;p&gt;Since I only really need 2 Entrez queries, I guess it would not be conceptually impossible to get my copy of nt database as FASTA, and then run a script to split it into three directories (700-1000nt, 1000-1400nt, the rest while discarding all&amp;lt;700 nt), then make three blastable databases out of it. However it does feel rather inelegant. I was wondering if anyone might have a better solution!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2023-11-14T07:06:45.883" Title="Is there an equivalent of Entrez queries for local BLAST?" Tags="|blast|bash|blastn|entrez|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19815" PostTypeId="2" ParentId="19814" CreationDate="2022-10-11T13:50:00.357" Score="1" Body="&lt;p&gt;I looked back at the question and have a more comprehensive solution.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;There is no option for &lt;code&gt;-entrez_query&lt;/code&gt; on a local blast search. The information is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK279684/table/appendices.T.options_common_to_all_blast/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;entrez_query   Restrict search with the given Entrez query. Remote searches only.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There is an option without &lt;code&gt;-outfmt&lt;/code&gt; for &lt;code&gt;length&lt;/code&gt; and that is worth investigating.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;length means Alignment length&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;h3&gt;Perl one liner solution to NCBI's many parameter problem&lt;/h3&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;a) Preprocess your fasta file into separate databases of the required length ... by turning a fasta flatfile db into one where all the subject sequences are 700 - 1000.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -0777 -nle 'while (/^(&amp;gt;.+\n^[AGCT-NRYMKWS]{700,1000})\n/gm){print $1,&amp;quot;\n&amp;quot;}' example.fa &amp;gt; example700-1000.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;b) Now make the database on the restricted length fasta flatfile db,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;makeblastdb -in example700-1000.fa -parse_seqids -blastdb_version 5  -out laura700-1000.db -title &amp;quot;Laura 700 - 1000&amp;quot; -dbtype nucl&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This db will now contain an equivalent of the &lt;code&gt;-entrez_query 700:1000[slen]&lt;/code&gt;. Obviously, you would need to make a new db for each change in the sequence length query. Note the option &lt;code&gt;-taxid_map&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;c) Finally, run the blast search on the restricted length blast db.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastn -query laura.fas -db laura700-1000.db -evalue 1e-6 -num_threads 4 -out blasthits.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h4&gt;Code testing&lt;/h4&gt;&#xA;&lt;p&gt;&lt;strong&gt;Input is test.fa&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;here&#xA;AGAGACGTAGAGAAAAAAAAAA&#xA;&amp;gt;here2&#xA;TAGAGA&#xA;&amp;gt;here3&#xA;AGAGACGTAGAGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Modified perl oneliner because sequence length is different,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -0777 -nle 'while (/^(&amp;gt;.+\n^[AGC-T]{6,13})\n/gm){print $1,&amp;quot;\n&amp;quot;}' test.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;here2&#xA;TAGAGA&#xA;&amp;gt;here3&#xA;AGAGACGTAGAGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thus the sequence that was outside the range 6 to 13 (sequence &lt;code&gt;here2&lt;/code&gt; and &lt;code&gt;here3&lt;/code&gt;) is excluded (&lt;code&gt;here&lt;/code&gt;). This seems alot easier to me that all those &lt;code&gt;blastn&lt;/code&gt; options.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; NRYMKWS refer to the degenerate nucleotide code&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-10-14T21:34:28.830" LastActivityDate="2022-10-14T21:34:28.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19817" PostTypeId="1" CreationDate="2022-10-12T03:15:13.307" Score="1" ViewCount="92" Body="&lt;p&gt;I have some  tsRNA count data in excel which I need to add selectively in blocks based on the count&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;refID   Count&#xA;1) tRNA-Ala-AGC-1   1&#xA;tRNA-Ala-AGC-1  2&#xA;tRNA-Ala-AGC-1  2&#xA;tRNA-Ala-AGC-1  1&#xA;tRNA-Ala-AGC-1  3&#xA;tRNA-Ala-AGC-1  300&#xA;tRNA-Ala-AGC-1  51&#xA;tRNA-Ala-AGC-1  86&#xA;tRNA-Ala-AGC-1  6&#xA;tRNA-Ala-AGC-1  2&#xA;tRNA-Ala-AGC-1  1&#xA;tRNA-Ala-AGC-1  5&#xA;&#xA;2)tRNA-Ala-AGC-11 1&#xA;tRNA-Ala-AGC-11 2&#xA;tRNA-Ala-AGC-11 1&#xA;tRNA-Ala-AGC-11 64&#xA;tRNA-Ala-AGC-11 1&#xA;tRNA-Ala-AGC-11 1&#xA;tRNA-Ala-AGC-11 2&#xA;tRNA-Ala-AGC-11 3&#xA;tRNA-Ala-AGC-11 1&#xA;tRNA-Ala-AGC-11 1&#xA;tRNA-Ala-AGC-11 1&#xA;&#xA;3) tRNA-Ala-AGC-3&#xA;tRNA-Ala-AGC-3&#xA;tRNA-Ala-AGC-3&#xA;tRNA-Ala-AGC-3&#xA;tRNA-Ala-AGC-3&#xA;tRNA-Ala-AGC-3&#xA;tRNA-Ala-AGC-3&#xA;tRNA-Ala-AGC-3&#xA;tRNA-Ala-AGC-3&#xA;tRNA-Ala-AGC-3&#xA;tRNA-Ala-AGC-3&#xA;tRNA-Ala-AGC-3&#xA;tRNA-Ala-AGC-3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am using sum and subtotal functions in excel to get the total counts of each of the blocks.&#xA;I have about 22,000 values and several hundred groups like this.&#xA;Is there a way to sum up all such blocks all at once based on the groups or at least a shorter way to sum things work other than autosum and subtotal?&lt;/p&gt;&#xA;&lt;p&gt;Any help will be useful.&lt;/p&gt;&#xA;" OwnerUserId="9659" LastEditorUserId="73" LastEditDate="2022-10-12T10:42:36.257" LastActivityDate="2022-10-13T23:22:56.200" Title="Summing up data effectively in excel" Tags="|sequence-analysis|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19818" PostTypeId="2" ParentId="19817" CreationDate="2022-10-12T03:35:26.037" Score="1" Body="&lt;p&gt;Suppose the header is on row 1 and starts at column 'A'. Then, for the above example, just point to a cell at say &lt;code&gt;C1&lt;/code&gt; to &lt;code&gt;C3&lt;/code&gt; and write ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;=sum(B2:B12)&#xA;=sum(B14:B25)&#xA;=sum(B27:B38)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This gives the subtotals.&lt;/p&gt;&#xA;&lt;p&gt;For the total sum point the mouse at &lt;code&gt;D1&lt;/code&gt; and write:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;=sum(B2:B38)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... except of course the real value is ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;=sum(B2:B22000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In all cases you must put the &lt;code&gt;=&lt;/code&gt; to start the cell.&lt;/p&gt;&#xA;&lt;p&gt;Its better to try out &lt;code&gt;R&lt;/code&gt;, or Python &lt;code&gt;pandas&lt;/code&gt; or just a Python list &lt;code&gt;print (sum([1,2,3,4,5])&lt;/code&gt; but Excel's okay.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The issue is its slow, its taking hours to calculate the sum of 22 000 cells - thats not a lot even for Excel. In Python that wouldn't even exceed 1 second, literally. What you are facing is &lt;strong&gt;a RAM bottleneck&lt;/strong&gt; - &lt;em&gt;there is literally no other explanation&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Two suggestions firstly, look at Windows Excel, particularly if you are using OSX. This is because historically Windows was much more RAM efficient than OSX. Now however, I don't know. Secondly, simply switch to a desktop with 4 Gig RAM or more. Literally thats all it will take.&lt;/p&gt;&#xA;&lt;p&gt;The drop back position is to do the calculation in two halves on the low RAM machine, i.e. split the data set in two. Not a cool solution but it will probably work. The final solution is to look at the stuff below the line.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The solutions are to shift to &lt;code&gt;VB&lt;/code&gt; which none of us code in these days, but its the code underneath Excel, or the alternative is to write a custom script in a different language that will do the adding such as &lt;code&gt;Python&lt;/code&gt;, &lt;code&gt;R&lt;/code&gt; ... even &lt;code&gt;bash&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-10-12T15:26:29.667" LastActivityDate="2022-10-12T15:26:29.667" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19820" PostTypeId="1" AcceptedAnswerId="19833" CreationDate="2022-10-12T09:05:59.203" Score="1" ViewCount="109" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; I am using &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;snakemake&lt;/code&gt;&lt;/a&gt; and I want to add the external (cluster) jobid to my cluster output and error log files. Below is my shell script.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;snakemake --configfiles config/config_resources.yml config/config_parameters.yml --latency-wait 60 --use-conda --printshellcmds --jobname PipeName_{name}.jobid{jobid} --jobs 300 --cluster &amp;quot;bsub -W 480 -n {threads} -M {resources.mem_gb}G -R 'rusage[mem={resources.mem_gb}G]' -o logs/{name}_{jobid}.out -e logs/{name}_{jobid}.err&amp;quot;&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;But I only get the internal one in de log name from snakemake (e.g. 1 to 5), instead of the one from the cluster.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; How to I get the external job id in there and not the local one made by snakemake? Or where in my shell script did I go wrong?&lt;/p&gt;&#xA;&lt;p&gt;Cheers and thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="16204" LastActivityDate="2022-10-13T14:51:19.907" Title="How to get the external (cluster) job id and not internal snakemake job id in cluster log files?" Tags="|snakemake|hpc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19821" PostTypeId="1" CreationDate="2022-10-12T10:05:30.323" Score="0" ViewCount="234" Body="&lt;p&gt;I have 8 RNA-seq datasets and am interested in looking at genes co-expressed with a specific gene.&lt;/p&gt;&#xA;&lt;p&gt;Among 8 RNA-seq datasets, 6 have less than 20 samples. Rather than working on each dataset individually for WGCNA, do you think I can merge all datasets and use them for WGCNA?&lt;/p&gt;&#xA;" OwnerUserId="9114" LastEditorUserId="9114" LastEditDate="2022-10-12T10:16:26.313" LastActivityDate="2022-10-15T03:38:36.037" Title="Is it possible to integrate multiple gene expression datasets and use it for WGCNA?" Tags="|r|rna-seq|wgcna|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19822" PostTypeId="2" ParentId="19817" CreationDate="2022-10-12T11:12:19.177" Score="0" Body="&lt;p&gt;It looks like what you want to do is a pivot table, which provides summary statistics for fields, grouped by specified categories. As a prerequisite, this requires an input dataset that is well formatted; more generally:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Put each observation or sample in its own row; don't leave row gaps between samples or groups. Leaving gaps makes bulk analysis more difficult and time consuming. It can still be done, it just means that the data needs to be pre-cleaned, or alternatively analysed.&lt;/li&gt;&#xA;&lt;li&gt;Put all your variables in columns - the thing that vary between samples, like ‘strain’ or ‘DNA-concentration’&lt;/li&gt;&#xA;&lt;li&gt;Have column names be explanatory, but without spaces&lt;/li&gt;&#xA;&lt;li&gt;Do not combine multiple pieces of information in one cell&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For more information, see this Data Carpentries Workshop on the recommended way to structure data in spreadsheets:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://datacarpentry.org/organization-genomics/01-tidiness/index.html#structuring-data-in-spreadsheets&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://datacarpentry.org/organization-genomics/01-tidiness/index.html#structuring-data-in-spreadsheets&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If the data is prepared in this way, then selecting the whole table and creating a pivot table should work. Unfortunately, the programs keep changing where in the menu that can be found. Currently in LibreOffice, it's &lt;code&gt;Data&lt;/code&gt; -&amp;gt; &lt;code&gt;Pivot Table&lt;/code&gt; -&amp;gt; &lt;code&gt;Insert or Edit...&lt;/code&gt;, wheareas on my version of Excel on Mac OSX, it's found at &lt;code&gt;Data&lt;/code&gt; -&amp;gt; &lt;code&gt;Summarise With Pivot Table&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This will load up a visual interface that allows you to drag and drop fields to summarise. For this data, you'd probably want &lt;em&gt;RefID&lt;/em&gt; in rows, and &lt;em&gt;sum(Count)&lt;/em&gt; as the data field:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SrROh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SrROh.png&quot; alt=&quot;Pivot table customisation dialog, showing RefID in rows, and sum(Count) as the data field&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;After setting this up and clicking &lt;code&gt;Ok&lt;/code&gt;, you should be greeted with a smaller table containing summarised variables:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/kqDkl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/kqDkl.png&quot; alt=&quot;Summary table, showing counts per RefID&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-10-13T23:22:56.200" LastActivityDate="2022-10-13T23:22:56.200" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19823" PostTypeId="2" ParentId="13722" CreationDate="2022-10-12T11:38:51.013" Score="2" Body="&lt;p&gt;With the current versions of samtools, you can also use:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools coverage sample.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The documents are &lt;a href=&quot;https://www.htslib.org/doc/samtools-coverage.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;samtools coverage – produces a histogram or table of coverage per&#xA;chromosome&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;A further example from the site,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools coverage -r chr1:1M-12M input.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#rname  startpos  endpos    numreads  covbases  coverage  meandepth  meanbaseq  meanmapq&#xA;chr1    1000000   12000000  528695    1069995   9.72723   3.50281    34.4       55.8&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The histogram option is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools coverage -A -w 32 -r chr1:1M-12M input.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... and output available on the website link.&lt;/p&gt;&#xA;" OwnerUserId="16288" LastEditorUserId="3848" LastEditDate="2022-10-12T15:30:16.433" LastActivityDate="2022-10-12T15:30:16.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19824" PostTypeId="2" ParentId="19821" CreationDate="2022-10-12T11:51:00.130" Score="1" Body="&lt;p&gt;I would try to process all data sets in a similar way, then combine them and batch-correct, e.g., using ComBat or WGCNA's own empiricalBayesLM.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2022-10-12T11:51:00.130" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19826" PostTypeId="1" CreationDate="2022-10-12T19:03:28.107" Score="2" ViewCount="819" Body="&lt;p&gt;I have two strains, &lt;strong&gt;X&lt;/strong&gt; and &lt;strong&gt;Y&lt;/strong&gt;, and I would like to compare the variants found in both. I have my own parent strain, &lt;strong&gt;Z&lt;/strong&gt;, that was sequenced and assembled to be used as the reference genome.&lt;/p&gt;&#xA;&lt;p&gt;I first aligned &lt;strong&gt;X&lt;/strong&gt; and &lt;strong&gt;Y&lt;/strong&gt; to &lt;strong&gt;Z&lt;/strong&gt; using minimap2 and then I used FreeBayes as the variant caller. Once I had the two separate VCF files I looked online to see what was the best way to filter out the unique and common variants in &lt;strong&gt;X&lt;/strong&gt; and &lt;strong&gt;Y&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I found the &lt;code&gt;bcftools isec&lt;/code&gt; tool to be my best bet so I went ahead and ran this line of code:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;./bcftools isec -p out/path/ X.vcf.gz Y.vcf.gz&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now I have three new VCF files:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;0000.vcf&lt;/code&gt; (variants unique to X)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;0001.vcf&lt;/code&gt; (variants unique to Y)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;0002.vcf&lt;/code&gt; (variants in both X and Y)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, it's still in VCF format. I need a way to parse through the VCF file to list out the SNPs with high quality with their position on the genome. I tried using &lt;code&gt;bcftools stats&lt;/code&gt; but it just lists out how many SNPs there are and not their position.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;TLDR:&lt;/strong&gt; Can someone help me find a way to list out the locations of SNPs from a VCF file in human readable format?&lt;/p&gt;&#xA;" OwnerUserId="16067" LastActivityDate="2023-11-07T11:02:10.223" Title="Compare VCF files" Tags="|vcf|snp|variant-calling|bcftools|variants|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19828" PostTypeId="2" ParentId="19826" CreationDate="2022-10-13T08:57:33.987" Score="1" Body="&lt;p&gt;If I understood you correctly you are searching for a way to represent the genomic locations in your vcf file. However, the vcf file is already a human readable file (&lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;vcf specification&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The first lines are the header, but afterwards you can find your information in a tab delimited fashion:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM POS ID REF ALT QUAL FILTER INFO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So the information on genomic positions is directly readable from the vcf file (first 2 columns)&lt;/p&gt;&#xA;" OwnerUserId="16294" LastEditorUserId="298" LastEditDate="2022-10-13T09:03:24.947" LastActivityDate="2022-10-13T09:03:24.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19829" PostTypeId="1" CreationDate="2022-10-13T09:22:01.683" Score="1" ViewCount="127" Body="&lt;p&gt;Hi my RNAseq workflow is ending up with wrongly formatted gene IDs (and separately transcript _IDs) after a Hisat2 -&amp;gt;samtools sort -&amp;gt; stringtie -&amp;gt; DEprep.py workflow.&#xA;DEprep.py outputs a gene_counts.csv which includes this...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gene:ENSRNOG00000000017         47&#xA;ENSRNOG00000000021              220&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Spelling this out &amp;quot;gene:&amp;quot; should not be in the gene_ID.  First thought of course was to just delete the &amp;quot;gene:&amp;quot; string with search and replace.&#xA;However, sometimes, the same gene appears as both indicating the error could be really damaging:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Gene_ID                   Counts &#xA;ENSRNOG00000004937        5234&#xA;gene:ENSRNOG00000004937   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Digging around I have found that the issue is present in the initial stringtie /ballgown/ GTF output prior to running DEprep.py&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1    ensembl exon    la la   gene_id &#xA; &amp;quot;gene:ENSRNOG00000046319&amp;quot; la la la&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Curiously my workflow is copied from my own local Galaxy installation workflow that uses the same steps together with the SAME fastq's the same, Hisat2 index, the same Guide.GFF, but there is no issue there.&lt;/p&gt;&#xA;&lt;p&gt;I have spent a day googling this and no success yet, lots of peep have missing ids. One other person had this symptom, but sadly no resolution.  Any suggestions gratefully received!!&#xA;Code details below and an attached folder in the comments with dummy data and r script that reproduces this error.&#xA;HiSat2:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;hisat2 --dta --add-chrname --time --summary-file YH2minihisat.txt&#xA;--rna-strandness FR  --pen-cansplice 0 --pen-noncansplice 12 --pen-canintronlen G,-8.0,1.0 --pen-noncanintronlen G,-8.0,1.0 --min-intronlen 20 --max-intronlen 500000 --threads 12 -x rn6/genome -q -1 YH2mini_R1.fastq.gz -2 YH2mini_R2.fastq.gz |  samtools sort -m20G -l 0 -T /Users/me/Documents/TEMP/b -O bam | samtools view -O bam -@ 12 -o YH2mini.bam&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Stringtie:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;stringtie -fr -p 12 -G rat6.gtf -e -b ballgown/YH2mini -f 0.15 -m 200&#xA;-a 10 -j 1 -c 2 -g 50 -M 0.95 -o ballgown/YH2mini/YH2mini.gtf YH2mini.bam&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;PrepDE:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;prepDE.py -i ballgown -g YH2mini_gene_counts.csv -t&#xA;YH2mini_transcript_counts.csv -l 100&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So peep can see I'm not lazy, I have a work around where I can run this, extra prior to prepDE.py, but seems very wrong!!&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;if(CORRECTGTF){&#xA;    &#xA;        command &amp;lt;- paste0(&amp;quot;sed 's/gene\\://g' ballgown/&amp;quot;,PREFIX,&amp;quot;/&amp;quot;,PREFIX,&amp;quot;.gtf &amp;gt; temp.txt&amp;quot;) &#xA;        system(command)&#xA;        command &amp;lt;- paste0(&amp;quot;sed 's/transcript\\://g' temp.txt &amp;gt; ballgown/&amp;quot;,PREFIX,&amp;quot;/&amp;quot;,PREFIX,&amp;quot;.gtf&amp;quot;) &#xA;        system(command)&#xA;        command &amp;lt;- paste0(&amp;quot;sed 's/gene\\://g' ballgown/&amp;quot;,PREFIX,&amp;quot;/&amp;quot;,&amp;quot;t_data.ctab &amp;gt; t_datafix.ctab.tmp&amp;quot;)&#xA;        system(command)&#xA;        command &amp;lt;- paste0(&amp;quot;sed 's/transcript\\://g ' t_datafix.ctab.tmp &amp;gt; ballgown/&amp;quot;,PREFIX,&amp;quot;/&amp;quot;,&amp;quot;t_data.ctab&amp;quot;)&#xA;        system(command)&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16227" LastEditorUserId="16227" LastEditDate="2022-10-14T10:06:14.590" LastActivityDate="2022-10-14T10:06:14.590" Title="Stringtie/DEprep.py gene/transcript IDs are wrongly formatted" Tags="|rna-seq|gtf|gff3|gff|stringtie|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19830" PostTypeId="2" ParentId="19812" CreationDate="2022-10-13T12:27:37.550" Score="1" Body="&lt;p&gt;I'm a bioinformatician, but not a statistician, and it's been a while since I did any &lt;span class=&quot;math-container&quot;&gt;$\chi^2$&lt;/span&gt; tests, so take this answer with a grain of salt.&lt;/p&gt;&#xA;&lt;p&gt;From what I remember, the most important thing is to be clear about what your observed and expected values are, and to be consistent. In other words, &lt;em&gt;&lt;strong&gt;it doesn't matter what you pick, as long as you make it clear what you've picked&lt;/strong&gt;&lt;/em&gt;. For your situation there are at least four possible &amp;quot;expected&amp;quot; values that can be used:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Average counts based on your available data (this is probably the most commonly used approach)&lt;/li&gt;&#xA;&lt;li&gt;Equal proportions based on your available data (this is less common, because it alters table totals)&lt;/li&gt;&#xA;&lt;li&gt;Unaffected sib counts (also less common, because it alters totals)&lt;/li&gt;&#xA;&lt;li&gt;Population average counts based on public data&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Unfortunately, there's no answer that will work for all situations. It's a good idea to think about which of these (or any others) makes sense in your particular case, what you're actually trying to compare, and how much you want to generalise to other populations. It's almost always the case that you know your own data better than anybody else, so are in the best position to make this decision. For example, if this is a very rare variant in the general population, you might choose to look for general population data for the variant, but if you're only interested in the impact of people who are aware of a family history, then unaffected sibs might be a better approach.&lt;/p&gt;&#xA;&lt;p&gt;In any case, the calculations for generating the expected values are similar: work out the proportion of the variant in the expected population, and apply that proportion to the total count in the observed population.&lt;/p&gt;&#xA;&lt;p&gt;In the first case, the expected counts would be something like this [note that the table totals are preserved]:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;Denovo&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;WT&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;Total&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Proband&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;*604.2&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1150.8&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1755&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Unaffected sibs&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;453.8&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;864.2&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1318&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;&lt;strong&gt;Total&lt;/strong&gt;&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1058&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;2015&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;3073&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;* calculation: row total (1755) as a proportion of full total (3073) = 0.5711, which is then multiplied by the column total (1058) = 604.2&lt;/p&gt;&#xA;&lt;p&gt;In the second case, it would be this [note that the row totals are different]:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;Denovo&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;WT&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;Total&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Proband&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;529&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1007.5&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1536.5&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Unaffected sibs&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;529&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1007.5&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1536.5&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;&lt;strong&gt;Total&lt;/strong&gt;&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1058&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;2015&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;3073&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;In the third case, it's simply comparing Proband vs Unaffected counts, normalising based on the row total:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;Denovo&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;WT&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;Total&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Proband&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;672.4&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1082.6&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1755&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Unaffected sibs&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;505&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;813&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1318&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;&lt;strong&gt;Total&lt;/strong&gt;&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1177.4&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;1895.6&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;3073&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;Note that the second row can be omitted from &lt;span class=&quot;math-container&quot;&gt;$\chi^2$&lt;/span&gt; calculations here, because it's &lt;em&gt;identical&lt;/em&gt; to the expected values, so adding that difference will not change the calculated &lt;span class=&quot;math-container&quot;&gt;$\chi^2$&lt;/span&gt; sum.&lt;/p&gt;&#xA;&lt;p&gt;[this could also be done based on the column total, if you wanted to compare Denovo vs WT, instead of Proband vs Unaffected]&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-10-13T12:27:37.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19832" PostTypeId="1" AcceptedAnswerId="19850" CreationDate="2022-10-13T13:29:04.450" Score="1" ViewCount="87" Body="&lt;p&gt;&lt;strong&gt;The data&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have a large number of reads from sequences that were generated by randomly shuffling regions of two parent sequences together. See the following image:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LgU0I.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LgU0I.png&quot; alt=&quot;1&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The regions that are shuffled are around 100-200 bp in length. The full shuffled sequence is around 1000 bp long. In general, the regions will be in the same order as the original sequence. The shuffled sequences are fragmented and sequenced into 150bp reads. One read can contain more than two borders between shuffle regions from a different parent sequence.&lt;/p&gt;&#xA;&lt;p&gt;The sequences are very similar (95%). Because of this high similarity you cannot pinpoint the border between two regions. The border will be a range between a variant that is unique to source1 and a variant that is unique to source2. That is ok because I am not interested in knowing from which source an identical region came.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would like to identify which part of the read originates from parent sequence one or two.&#xA;What would be a suitable tool to do this?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My attempts&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I tried to look at some multiple sequence alignment tools like MAFFT or Clustal but I am not interested in aligning all reads to each other. They also don't identify the shuffle regions well in my testing.&lt;/p&gt;&#xA;" OwnerUserId="16298" LastEditorUserId="73" LastEditDate="2022-10-19T11:43:45.623" LastActivityDate="2022-11-03T19:19:23.417" Title="How to identify sequence origin in DNA shuffle reads?" Tags="|sequence-alignment|genomics|multiple-sequence-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19833" PostTypeId="2" ParentId="19820" CreationDate="2022-10-13T14:51:19.907" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=o-o&quot; rel=&quot;nofollow noreferrer&quot;&gt;bsub supports some filename placeholders using a percent sign&lt;/a&gt;, like &lt;code&gt;%J&lt;/code&gt; for the LSF job ID, so you can use that:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;snakemake ... --cluster &amp;quot;bsub ... -o logs/{name}_%J.out -e logs/{name}_%J.err&amp;quot;&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;That way snakemake will format the cluster command string with its wildcards (like &lt;code&gt;name&lt;/code&gt;) as it constructs the bsub command, and then bsub will put the job ID in place in those filenames.  Since Snakemake just sees that as one big command to run for the cluster and doesn't know the details other than formatting the text, I think the two shouldn't clash.&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2022-10-13T14:51:19.907" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19834" PostTypeId="1" CreationDate="2022-10-13T15:06:09.840" Score="1" ViewCount="42" Body="&lt;p&gt;I have got an integrated seurat object of 21 Spatial samples. I want to know how many spots are in all the samples together, Is there a way I can do that? Also, Please suggest any R Packages that do good ploting with spatial data (Like dittoSeq for Single cell).&#xA;Thanks a lot,&#xA;AMK&lt;/p&gt;&#xA;" OwnerUserId="9817" LastActivityDate="2022-10-13T15:06:09.840" Title="Number of Spots in a Spatial Seurat object" Tags="|rna-seq|transcriptome|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19835" PostTypeId="1" CreationDate="2022-10-13T15:47:59.797" Score="2" ViewCount="116" Body="&lt;p&gt;We have 2 vcf (Whole Exome Sequencing (&lt;strong&gt;WES&lt;/strong&gt;) data; &lt;strong&gt;germline&lt;/strong&gt; samples) files (e.g., &lt;strong&gt;vcf_1&lt;/strong&gt; and &lt;strong&gt;vcf_2&lt;/strong&gt;). &lt;strong&gt;vcf_1&lt;/strong&gt; was generated (Ref. genome: hg38) using the &lt;strong&gt;GATK&lt;/strong&gt; pipeline for 250 children and their parents and &lt;strong&gt;vcf_2&lt;/strong&gt; was generated (Ref. genome: hg38) using the &lt;strong&gt;DRAGEN&lt;/strong&gt; pipeline for another 50 children and their parents (Samples are different). These are raw vcf files and I have not applied any quality-related false positive filters to them.&lt;/p&gt;&#xA;&lt;p&gt;Now, when we compare the sample-wise SNV counts from &lt;strong&gt;vcf_1&lt;/strong&gt; and &lt;strong&gt;vcf_2&lt;/strong&gt;, there is a huge difference. &lt;strong&gt;vcf_2&lt;/strong&gt; contains on average 4-6 times lesser number of SNVs per sample than &lt;strong&gt;vcf_1&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;N.B:&lt;/strong&gt;&#xA;I found this article (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4418901/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4418901/&lt;/a&gt;), where it has been reported that the mean numbers of single-nucleotide variants (SNVs) and small insertions/deletions (indels) detected per sample was around 98k, respectively, for WES.  For our case, in the case of GATK-generated vcf samples, this varies from 130k to 188k, and for DRAGEN-generated vcf samples, this varies from 30k to 40k.&lt;/p&gt;&#xA;&lt;p&gt;I understand both pipelines (GATK and DRAGEN) implement different methods to generate vcf files but is it common/normal to examine that much difference in SNV counts? I would appreciate it if someone can give some explanation (or provide some materials) about it.&lt;/p&gt;&#xA;" OwnerUserId="16300" LastEditorUserId="16300" LastEditDate="2022-10-14T08:52:35.813" LastActivityDate="2024-03-08T12:01:50.180" Title="Is it common to get different number of SNVs+Indels across samples from vcf files generated using GATK and DRAGEN (counts are higher for GATK)?" Tags="|vcf|ngs|gatk|snv|wes|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19837" PostTypeId="1" CreationDate="2022-10-13T19:00:11.070" Score="1" ViewCount="674" Body="&lt;p&gt;I have several RNA-seq datasets. Some of them provide RNA-seq raw counts, some provide FPKM, RPKM and some have transcripts per million (TPM) data. Non of them provide fastq files, all data is processed already. At the end I want all datasets to be normalized to TPM.&lt;/p&gt;&#xA;&lt;p&gt;I'm using this code in order to normalize raw counts to TPM: (using R)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rpkm &amp;lt;- apply(X = subset(dataset),&#xA;                MARGIN = 2,&#xA;                FUN = function(x) {&#xA;                  10^9 * x / genelength / sum(as.numeric(x))&#xA;                })&#xA;  &#xA;TPM &amp;lt;- apply(rpkm, 2, function(x) x / sum(as.numeric(x)) * 10^6) %&amp;gt;% as.data.frame()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When RPKM is provided, and no raw counts is available, I use the second line in the same code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TPM &amp;lt;- apply(rpkm, 2, function(x) x / sum(as.numeric(x)) * 10^6) %&amp;gt;% as.data.frame()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And when FPKM is provided, I use this formula to transform the data to TPM:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TPM = FPKM*X &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where &lt;code&gt;X = 1e6/[sum of all FPKM of a sample]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;NOTE: &lt;code&gt;genelength&lt;/code&gt; is obtained using the &lt;code&gt;biomart&lt;/code&gt; package in R, to get the transcript length directly out of ensemble.&lt;/p&gt;&#xA;&lt;p&gt;However, because I know the steps of TPM normalization in theory, one should firstly normalize to gene length, and then to gene depth. I'm not sure this is what the code does, feeling very skiptical about it even thought this code was given to me by a Phd student.&lt;/p&gt;&#xA;&lt;p&gt;Can you guys please help me with this question? is my code right or should I alter something?&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="16302" OwnerDisplayName="Kev" LastActivityDate="2022-10-19T17:05:18.737" Title="From RNA-seq raw counts to normalized data" Tags="|genetics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19838" PostTypeId="2" ParentId="19837" CreationDate="2022-10-13T22:14:47.343" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;However, because I know the steps of TPM normalization in theory, one&#xA;should firstly normalize to gene length, and then to gene depth.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No, you should normalize to &lt;strong&gt;transcript length&lt;/strong&gt;.  Which requires you to know the relative abundance of all the transcripts so you can correct for all their lengths.&lt;/p&gt;&#xA;&lt;p&gt;I do not think it's possible to un-normalize RPKM or TPM without knowing all the lengths involved in doing the calculations.&lt;/p&gt;&#xA;&lt;p&gt;Let me give an example: here is RSEM output for just one gene.  Note that the software thinks there are three transcripts present for this one gene:&lt;/p&gt;&#xA;&lt;p&gt;transcript_id  gene_id length  effective_length    expected_count  TPM FPKM    IsoPct&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ENST00000373020 ENSG00000000003 3768    3718.54 179.95  13.88   7.94    48.61&#xA;&#xA;ENST00000494424 ENSG00000000003 820 770.54  0.00    0.00    0.00    0.00&#xA;&#xA;ENST00000496771 ENSG00000000003 1025    975.54  11.94   3.51    2.01    12.29&#xA;&#xA;ENST00000612152 ENSG00000000003 3796    3746.54 0.00    0.00    0.00    0.00&#xA;&#xA;ENST00000614008 ENSG00000000003 900 850.54  33.11   11.17   6.39    39.10&#xA;&#xA;ENST00000373031 ENSG00000000005 1205    1155.54 0.00    0.00    0.00    0.00&#xA;&#xA;ENST00000485971 ENSG00000000005 542 492.54  0.00    0.00    0.00    0.00&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, what would you say the effective length should be for the gene level calculations, and would you get that right without having this transcript-level breakdown?&lt;/p&gt;&#xA;" OwnerUserId="2388" LastEditorUserId="2388" LastEditDate="2022-10-19T17:05:18.737" LastActivityDate="2022-10-19T17:05:18.737" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19839" PostTypeId="1" AcceptedAnswerId="19841" CreationDate="2022-10-14T03:02:46.270" Score="2" ViewCount="41" Body="&lt;p&gt;I'm training a ML model for disease prediction using protein composition as input, but overfit is present. While looking to &lt;em&gt;remove proteins and reduce multi-correlation&lt;/em&gt;, as remove composition predictors through PCA have worked but not enough, this problem appeared and managed to use Metascape (1) for it.&lt;/p&gt;&#xA;&lt;p&gt;I'm using the &lt;strong&gt;Protein Set Enrichment Analysis&lt;/strong&gt; (&lt;strong&gt;PSEA&lt;/strong&gt;, GSEA for genes) for significance of function as is a method to identify classes of genes or proteins that are over-represented in a large set of genes or proteins, and may have an association with disease phenotypes (2). So:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;A higher significance coefficient (log10pvalue) between two proteins suggest they have a similar function so one should be removed, or a lower coef shows the need to erase this quasi-independent protein from the analysis?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is this analysis good enough to reduce overfitting by remove redundant observations?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;References:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;(1) &lt;a href=&quot;https://metascape.org/gp/index.html#/main/step1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://metascape.org/gp/index.html#/main/step1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;(2) &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S2376999819300406&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sciencedirect.com/science/article/pii/S2376999819300406&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16306" LastActivityDate="2022-10-14T21:07:03.350" Title="Reduce Overfit by removing insignificant proteins through PSEA analysis" Tags="|machine-learning|proteomics|features|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19840" PostTypeId="1" CreationDate="2022-10-14T09:57:34.543" Score="0" ViewCount="294" Body="&lt;p&gt;While merging datasets from similar biological conditions, but different experimental conditions is a well-studied topic, and there are many batch correction techniques available. What I want to understand is the right way to merge scRNA seq counts data of healthy and tumor cells.&lt;/p&gt;&#xA;&lt;p&gt;Does individually log-normalising the counts, followed by a simple concatenation of the objects works, or there is a better way to do this?&lt;/p&gt;&#xA;" OwnerUserId="3183" LastActivityDate="2022-10-15T08:58:58.750" Title="Correct way of merging scRNAseq datasets of healthy and tumor cell?" Tags="|python|scrnaseq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19841" PostTypeId="2" ParentId="19839" CreationDate="2022-10-14T11:47:09.010" Score="1" Body="&lt;p&gt;Overfitting is usually resolved by adding more data and not removing data sets. The technical blurb is 'augmenting' your training data set to revolve overfitting.&lt;/p&gt;&#xA;&lt;p&gt;It is possible there is a very large multi-gene family involved distorting the classification, thus reducing that could reduce the overfit and cause the algorithm to spread the classification over a much broader group of genes. The general advice is to find an alternative genomic source of proteins and add that, not for this specific question but in reducing overfit &lt;em&gt;per se&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There are technical issues such as:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a parameter split, does the training have this? If not its strongly recommended - depending on the algorithm.&lt;/li&gt;&#xA;&lt;li&gt;algorithm, for example decision trees are known for overfitting*, whilst naive Bayes would find difficulty overfitting**.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I personally think multiple measures would resolve this. However, general advice is to look at alternative methods and training approaches before looking at slimming down a data set. I would suggest there needs to be good evidence - which would need to be formally presented. You could be 100% correct (probably are), but its not an orthodox approach.&lt;/p&gt;&#xA;&lt;p&gt;Clearly there is information here that isn't presented. What you've done is a feature selection based on genes and we don't know what that result was nor the gene group that emerged from that analysis - so it is difficult to dissect this precisely. It could be there is a compelling biological reason for concern of overfitting based on the result, but we'll not know that. What I am certain about is the orthodoxy in resolving overfitting is not leveraged here and any data scientist would be aware of this.&lt;/p&gt;&#xA;&lt;p&gt;*, this is why random forests are considered strong&lt;/p&gt;&#xA;&lt;p&gt;**, there are a lot of different approaches to ML and how they are classifying is very different, some are more prone to overfitting than others. Naive Bayesian and 'linear regression' are the least sensitive to overfitting.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Please be aware I do ML within the Python framework and its one of the few areas I'm not aware of the ease of cross-over into other platforms. This is simply because its an analysis where Python excels.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-10-14T21:07:03.350" LastActivityDate="2022-10-14T21:07:03.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19843" PostTypeId="1" CreationDate="2022-10-14T13:33:15.733" Score="1" ViewCount="151" Body="&lt;p&gt;I have a binary matrix with individuals as row names, and gene name as column name.&#xA;So, if the gene is present in an individual, we have 1, otherwise 0.&lt;/p&gt;&#xA;&lt;p&gt;I would like to cluster individuals based on the gene presence/absence pattern.&lt;/p&gt;&#xA;&lt;p&gt;Is it right to perform an MCA? Or a logistic PCA is needed?&lt;/p&gt;&#xA;&lt;p&gt;Thanks,&#xA;Marco&lt;/p&gt;&#xA;" OwnerUserId="15596" LastEditorUserId="15596" LastEditDate="2022-10-14T14:21:47.843" LastActivityDate="2022-10-14T14:21:47.843" Title="Clustering individuals by gene presence/absence" Tags="|r|statistics|clustering|pca|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19845" PostTypeId="1" AcceptedAnswerId="19852" CreationDate="2022-10-14T19:54:46.000" Score="2" ViewCount="185" Body="&lt;p&gt;I'm trying to get genome coordinate information programmatically for multiple genes.&lt;/p&gt;&#xA;&lt;p&gt;As an example, I'd like to get the annotation (list of exons and their chromosomes, starts, and ends) for this UniProt entry: &lt;a href=&quot;https://www.uniprot.org/uniprotkb/B6TU39/entry&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uniprotkb/B6TU39/entry&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There is a corresponding GenBank entry, which gives the CDS (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/EU968504&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/EU968504&lt;/a&gt;) but I can't see anywhere how to extract the genome coordinates. How can I do this?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="16316" LastEditorUserId="16316" LastEditDate="2022-10-14T19:57:31.830" LastActivityDate="2022-10-17T18:16:32.803" Title="Get genome coordinates from UniProt entry?" Tags="|uniprot|genbank|sequence-annotation|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19846" PostTypeId="2" ParentId="13177" CreationDate="2022-10-14T22:47:41.430" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;it seems cumbersome to create them to ensure execution order in my case&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It took me a while to figure out, but it did not seem cumbersome to me.&lt;/p&gt;&#xA;&lt;p&gt;You can re-use your rules, see below, and make Step 1 of S2 depend on Step 2 of S1 like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SAMPLES = ['S1', 'S2']&#xA;&#xA;&#xA;wildcard_constraints:&#xA;    sample = &amp;quot;S[12]&amp;quot;&#xA;&#xA;rule all:&#xA;  input: expand(&amp;quot;{sample}.done&amp;quot;, sample = SAMPLES)&#xA;&#xA;&#xA;&#xA;rule step1:&#xA;  input: data=&amp;quot;{sample}&amp;quot;&#xA;  output: data=&amp;quot;{sample}.step1.done&amp;quot;&#xA;  shell: &amp;quot;touch {output.data}&amp;quot;&#xA;&#xA;rule step2:&#xA;  input: data=&amp;quot;{sample}.step1.done&amp;quot;&#xA;  output: data=&amp;quot;{sample}.done&amp;quot;&#xA;  shell: &amp;quot;touch {output.data}&amp;quot;&#xA;&#xA;&#xA;# step2 of S1 has the flag file as output&#xA;&#xA;use rule step2 as step2ForS1 with:&#xA;    input: data=&amp;quot;S1.step1.done&amp;quot;&#xA;    output: data=&amp;quot;S1.done&amp;quot;, flag=touch(&amp;quot;flags/step2-s1-done&amp;quot;)&#xA;&#xA;# step1 of S2 needs the flag file as input&#xA;&#xA;use rule step1 as step1ForS2 with:&#xA;    input: data=&amp;quot;S2&amp;quot;, flag=&amp;quot;flags/step2-s1-done&amp;quot;&#xA;    output: data=&amp;quot;S2.step1.done&amp;quot;&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16314" LastEditorUserId="16314" LastEditDate="2022-10-14T22:48:52.220" LastActivityDate="2022-10-14T22:48:52.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19847" PostTypeId="2" ParentId="19821" CreationDate="2022-10-15T03:38:36.037" Score="2" Body="&lt;p&gt;Yes, if the expression datasets are from the same platform, you can merge all the datasets for common genes among them. However, if they are from different platforms then some batch correction needs to be done before analysis. You can plot a PCA to check how much variability there is among datasets from different platforms and then decide whether batch correction is needed.&lt;/p&gt;&#xA;" OwnerUserId="15931" LastActivityDate="2022-10-15T03:38:36.037" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19848" PostTypeId="2" ParentId="19840" CreationDate="2022-10-15T08:58:58.750" Score="1" Body="&lt;p&gt;I'm aware that you've tagged this as python, but unfortunately I'm most familiar with doing single cell analysis using R. Perhaps there's a python way to do something similar.&lt;/p&gt;&#xA;&lt;p&gt;Seurat has recently included a workflow for reciprocal PCA integration, which the authors claim should work well when the combined datasets are more different than usual:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;RPCA-based integration runs significantly faster, and also represents a more conservative approach where cells in different biological states are less likely to ‘align’ after integration. We therefore, recommend RPCA during integrative analysis where:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A substantial fraction of cells in one dataset have no matching type in the other&lt;/li&gt;&#xA;&lt;li&gt;Datasets originate from the same platform (i.e. multiple lanes of 10x genomics)&lt;/li&gt;&#xA;&lt;li&gt;There are a large number of datasets or cells to integrate (see INSERT LINK for more tips on integrating large datasets)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;A tutorial workflow can be found here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://satijalab.org/seurat/articles/integration_rpca.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/articles/integration_rpca.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-10-15T08:58:58.750" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19849" PostTypeId="2" ParentId="19835" CreationDate="2022-10-15T09:26:54.033" Score="0" Body="&lt;p&gt;I suspect what's happening here is an unfortunate side-effect of making variant-finding algorithms memory-efficient, so that they can work on extremely large populations. By doing the processor-intensive variant calling separately on each individual, calculations can be easily parallelised to run quickly.&lt;/p&gt;&#xA;&lt;p&gt;For the numbers you have (i.e. 300 children and their parents), there's an alternative approach for generating variant calling that uses all mapped files as input, rather than only one at a time, i.e. instead of running:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools mpileup [options] in1.bam&#xA;bcftools mpileup [options] in2.bam&#xA;bcftools mpileup [options] ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The all-at-once approach has a single mpileup call:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools mpileup [options] in1.bam in2.bam ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will call multiple samples at the same time, so that the output will have identical variant lines for all individuals. It also has the added benefit of getting more information on variation, allowing more rare variants to be identified.&lt;/p&gt;&#xA;&lt;p&gt;I'm not sure which approach is being used for DRAGEN vs GATK, but I do know that at some stage in the past the GATK standard practice was calling one sample at a time. Curiously, if this were still the case, I'd expect the results to be the other way round, as the one-at-a-time approach tends to be less sensitive.&lt;/p&gt;&#xA;&lt;p&gt;I guess it's also possible that the pipelines have different sensitivity / specificity tweaks, that could also explain the differences in variant counts.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-10-15T09:26:54.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19850" PostTypeId="2" ParentId="19832" CreationDate="2022-10-15T09:33:02.307" Score="2" Body="&lt;p&gt;This is the sort of thing that &lt;a href=&quot;https://gitlab.com/mcfrith/last&quot; rel=&quot;nofollow noreferrer&quot;&gt;LAST&lt;/a&gt; has been specifically designed to work well for:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;LAST is designed for moderately large data (e.g. genomes, DNA reads,&#xA;proteomes). It's especially good at finding rearrangements and recombinations: we believe last-split does that more rigorously than anything else.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here's my usual approach for mapping with LAST:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;lastdb -uRY4 -R01 reference_sequences.fa reference_sequences.fa&#xA;# training is probably not needed if your sub-sequences have no&#xA;# variation from the parental sequences&#xA;last-train reference_sequences.fa input_sequences.fa &amp;gt; trained.mat&#xA;lastal -P 10 -p trained.mat --split reference_sequences.fa input_sequences.fa &amp;gt; aligned.maf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;LAST is &lt;em&gt;very&lt;/em&gt; customisable. If you want to adjust the mapping or split sensitivity, have a look at the &lt;code&gt;lastal&lt;/code&gt; documentation &lt;a href=&quot;https://gitlab.com/mcfrith/last/-/blob/main/doc/lastal.rst&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, and the &lt;code&gt;last-split&lt;/code&gt; documentation &lt;a href=&quot;https://gitlab.com/mcfrith/last/-/blob/main/doc/last-split.rst&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For example, there's a mismap probability adjustment which excludes alignments with a mismap probability above a certain threshold:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# only form alignments if the mismap probability is 0.1% or less&#xA;lastal --split-m 0.001 ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-11-03T19:19:23.417" LastActivityDate="2022-11-03T19:19:23.417" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19852" PostTypeId="2" ParentId="19845" CreationDate="2022-10-16T12:21:30.777" Score="2" Body="&lt;p&gt;You don't want the EU968504, that's the mRNA not the genome sequence. You want &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/CM007647&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/CM007647&lt;/a&gt;, also linked to from the UniProt page, just under the link to EU968504:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WFaDX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WFaDX.png&quot; alt=&quot;Screenshot of the UniProt page with the link&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You also need the gene name, &lt;code&gt;ZEAMMB73_Zm00001d029604&lt;/code&gt; in this case, available further up on the page in the &amp;quot;Names &amp;amp; Taxonomy&amp;quot; section:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7NOIt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7NOIt.png&quot; alt=&quot;Screenshot of UniProt page showing the relevant section&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Next, we can download the GenBank file for the relevant chromosomal contig  using NCBI's entrez API:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;curl &amp;quot;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=nuccore&amp;amp;id=CM007647&amp;amp;rettype=gbwithparts&amp;amp;retmode=text&amp;quot; &amp;gt; CM007647&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will take a few minutes. Once that is done, you can parse the file and extract the start and end coordinates using something like this:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;from Bio import SeqIO&#xA;import sys&#xA;&#xA;gb_record=SeqIO.read(open(sys.argv[1],&amp;quot;r&amp;quot;), &amp;quot;genbank&amp;quot;)&#xA;&#xA;for feature in gb_record.features:&#xA;    if feature.type == &amp;quot;gene&amp;quot; and sys.argv[2] in feature.qualifiers['locus_tag']:&#xA;        if feature.location.strand == 1:&#xA;            strand = &amp;quot;+&amp;quot;&#xA;        elif feature.location.strand == -1:&#xA;            strand = &amp;quot;-&amp;quot;&#xA;        else:&#xA;            print(&amp;quot;Unknown strand: &amp;quot;, feature.location.strand)&#xA;            sys.exit(1)&#xA;            &#xA;        print(&amp;quot;%s\t%s\t%s\t%s\t%s\t%s\t%s\t.\t.&amp;quot; % (gb_record.id, &amp;quot;GenBank&amp;quot;,&#xA;                                                    feature.type, feature.location.start,&#xA;                                                    feature.location.end, &amp;quot;.&amp;quot;, strand))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You pass the GenBank file and gene name as arguments, and this is the result:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ foo.py CM007647  ZEAMMB73_Zm00001d029604&#xA;CM007647.1  GenBank gene    78770080    78773320    .   +   .   .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2022-10-16T13:46:04.170" LastActivityDate="2022-10-16T13:46:04.170" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19853" PostTypeId="1" CreationDate="2022-10-17T08:07:08.473" Score="2" ViewCount="33" Body="&lt;p&gt;I have a list of protein accessions:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;XP_025107904.1&#xA;PVD22034.1&#xA;AJA37836.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I obtain the definition of the protein, and the related organism?&lt;/p&gt;&#xA;&lt;p&gt;Output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;XP_025107904.1  receptor-type tyrosine-protein phosphatase kappa-like isoform X2  Pomacea canaliculata&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thanks for the help,&lt;/p&gt;&#xA;&lt;p&gt;Marco&lt;/p&gt;&#xA;" OwnerUserId="15596" LastEditorUserId="298" LastEditDate="2022-10-17T09:14:58.157" LastActivityDate="2023-07-14T20:06:00.710" Title="Retrieval protein definition from accession ID" Tags="|proteins|phylogenetics|uniprot|identifiers|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19854" PostTypeId="2" ParentId="19845" CreationDate="2022-10-17T09:44:25.730" Score="0" Body="&lt;p&gt;You could align back the peptide sequence to the genome, respecting splicing, for example using &lt;a href=&quot;https://github.com/lh3/miniprot&quot; rel=&quot;nofollow noreferrer&quot;&gt;miniprot&lt;/a&gt; from Heng Li.&lt;/p&gt;&#xA;" OwnerUserId="16326" LastActivityDate="2022-10-17T09:44:25.730" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19855" PostTypeId="1" CreationDate="2022-10-17T11:00:45.697" Score="0" ViewCount="212" Body="&lt;p&gt;I have a data frame (named as df) with single column of protein IDs and other file with two columns of protein IDs (named as df1). I need to find the df IDs present in either first or second column of df1 and extract corresponding Id from the df1.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df = data.frame(p = c(&amp;quot;ENSG00000198400&amp;quot;,&amp;quot;ENSG00000066044&amp;quot;,&amp;quot;ENSG00000188612&amp;quot;,&amp;quot;ENSG00000177606&amp;quot;,&amp;quot;ENSG00000146648&amp;quot;))&#xA;df1 = data.frame(p1 = c(&amp;quot;ENSG00000105647&amp;quot;,&amp;quot;ENSG00000110395&amp;quot;,&amp;quot;ENSG00000165119&amp;quot;,&amp;quot;ENSG00000160293&amp;quot;,&amp;quot;ENSG00000198400&amp;quot;,&amp;quot;ENSG00000198400&amp;quot;),p2 = c(&amp;quot;ENSG00000198400&amp;quot;,&amp;quot;ENSG00000198400&amp;quot;,&amp;quot;ENSG00000198400&amp;quot;,&amp;quot;ENSG00000142949&amp;quot;,&amp;quot;ENSG00000112936&amp;quot;,&amp;quot;ENSG00000162434&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Sample output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df2= data.frame(p = c(&amp;quot;ENSG00000198400&amp;quot;,&amp;quot;ENSG00000198400&amp;quot;,&amp;quot;ENSG00000198400&amp;quot;,&amp;quot;ENSG00000198400&amp;quot;,&amp;quot;ENSG00000198400&amp;quot;),p2 = c(&amp;quot;ENSG00000105647&amp;quot;,&amp;quot;ENSG00000110395&amp;quot;,&amp;quot;ENSG00000165119&amp;quot;,&amp;quot;ENSG00000112936&amp;quot;,&amp;quot;ENSG00000162434&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14023" LastEditorUserId="3694" LastEditDate="2022-10-17T20:50:34.427" LastActivityDate="2022-10-18T07:19:07.053" Title="subset from a dataframe using other dataframe in R" Tags="|r|dataframe|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19856" PostTypeId="1" CreationDate="2022-10-17T13:40:53.743" Score="1" ViewCount="647" Body="&lt;p&gt;My goal is to compare reads from two different fastq files on a Linux machine.&#xA;The following are the comparisons to perform:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How many common reads are between the two fastq files?&lt;/li&gt;&#xA;&lt;li&gt;How many reads are present in one fastq file that are not present in the other? i.e reads that are unique to each file.&lt;/li&gt;&#xA;&lt;li&gt;Where and how do the sequences differ in both files? Is there a difference in nucleotide base at the end of the sequence(3' end), the start of the sequence(5'end) or is it somewhere in the middle of the sequence? Is there a difference in length of the sequence? If so, what is the length and what is the difference? I would like to get the original sequences plus the difference between them for analysis.&lt;/li&gt;&#xA;&lt;li&gt;Does the quality score differ? Where and how do the quality of sequences differs?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Any guidance regarding how to approach this question is very appreciated.&lt;/p&gt;&#xA;&lt;p&gt;Edit: I'm working on paired-end reads from the same individual. What I want to do is to compare different trimming tools from their output.&lt;/p&gt;&#xA;" OwnerUserId="16329" LastEditorUserId="16328" LastEditDate="2022-10-18T08:18:25.980" LastActivityDate="2022-10-19T13:11:21.083" Title="Comparison of fastq files reads" Tags="|python|fastq|reads|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19857" PostTypeId="2" ParentId="19853" CreationDate="2022-10-17T17:13:12.707" Score="1" Body="&lt;p&gt;You can do this using the Entrez API. For example, using the three identifiers from your example, you can do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;curl &amp;quot;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=protein&amp;amp;id=XP_025107904.1,PVD22034.1,AJA37836.1&amp;quot; &amp;gt; file.xml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will bring an XML file with the following contents:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat file.xml &#xA;&amp;lt;?xml version=&amp;quot;1.0&amp;quot; encoding=&amp;quot;UTF-8&amp;quot; ?&amp;gt;&#xA;&amp;lt;!DOCTYPE eSummaryResult PUBLIC &amp;quot;-//NLM//DTD esummary v1 20041029//EN&amp;quot; &amp;quot;https://eutils.ncbi.nlm.nih.gov/eutils/dtd/20041029/esummary-v1.dtd&amp;quot;&amp;gt;&#xA;&amp;lt;eSummaryResult&amp;gt;&#xA;&amp;lt;DocSum&amp;gt;&#xA;    &amp;lt;Id&amp;gt;1397721498&amp;lt;/Id&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Caption&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;XP_025107904&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Title&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;receptor-type tyrosine-protein phosphatase kappa-like isoform X2 [Pomacea canaliculata]&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Extra&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;gi|1397721498|ref|XP_025107904.1|[1397721498]&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Gi&amp;quot; Type=&amp;quot;Integer&amp;quot;&amp;gt;1397721498&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;CreateDate&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;2018/06/07&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;UpdateDate&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;2018/06/07&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Flags&amp;quot; Type=&amp;quot;Integer&amp;quot;&amp;gt;512&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;TaxId&amp;quot; Type=&amp;quot;Integer&amp;quot;&amp;gt;400727&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Length&amp;quot; Type=&amp;quot;Integer&amp;quot;&amp;gt;1271&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Status&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;live&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;ReplacedBy&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Comment&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;&amp;lt;![CDATA[  ]]&amp;gt;&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;AccessionVersion&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;XP_025107904.1&amp;lt;/Item&amp;gt;&#xA;&amp;lt;/DocSum&amp;gt;&#xA;&#xA;&amp;lt;DocSum&amp;gt;&#xA;    &amp;lt;Id&amp;gt;1383748625&amp;lt;/Id&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Caption&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;PVD22034&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Title&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;hypothetical protein C0Q70_17837 [Pomacea canaliculata]&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Extra&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;gi|1383748625|gb|PVD22034.1||gnl|WGS:PZQS|C0Q70_17837[1383748625]&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Gi&amp;quot; Type=&amp;quot;Integer&amp;quot;&amp;gt;1383748625&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;CreateDate&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;2018/04/26&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;UpdateDate&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;2018/04/26&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Flags&amp;quot; Type=&amp;quot;Integer&amp;quot;&amp;gt;0&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;TaxId&amp;quot; Type=&amp;quot;Integer&amp;quot;&amp;gt;400727&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Length&amp;quot; Type=&amp;quot;Integer&amp;quot;&amp;gt;602&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Status&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;live&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;ReplacedBy&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Comment&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;&amp;lt;![CDATA[  ]]&amp;gt;&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;AccessionVersion&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;PVD22034.1&amp;lt;/Item&amp;gt;&#xA;&amp;lt;/DocSum&amp;gt;&#xA;&#xA;&amp;lt;DocSum&amp;gt;&#xA;    &amp;lt;Id&amp;gt;735659541&amp;lt;/Id&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Caption&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;AJA37836&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Title&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;fucolectin-related molecule [Littorina littorea]&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Extra&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;gi|735659541|gb|AJA37836.1|[735659541]&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Gi&amp;quot; Type=&amp;quot;Integer&amp;quot;&amp;gt;735659541&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;CreateDate&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;2014/12/25&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;UpdateDate&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;2014/12/25&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Flags&amp;quot; Type=&amp;quot;Integer&amp;quot;&amp;gt;0&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;TaxId&amp;quot; Type=&amp;quot;Integer&amp;quot;&amp;gt;31216&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Length&amp;quot; Type=&amp;quot;Integer&amp;quot;&amp;gt;722&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Status&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;live&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;ReplacedBy&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;Comment&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;&amp;lt;![CDATA[  ]]&amp;gt;&amp;lt;/Item&amp;gt;&#xA;    &amp;lt;Item Name=&amp;quot;AccessionVersion&amp;quot; Type=&amp;quot;String&amp;quot;&amp;gt;AJA37836.1&amp;lt;/Item&amp;gt;&#xA;&amp;lt;/DocSum&amp;gt;&#xA;&#xA;&amp;lt;/eSummaryResult&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can now parse this in your favorite language/parser and get the information you need. I... am not too good with XML so my attempts with &lt;code&gt;xmllint&lt;/code&gt; failed, but here's a naive awk approach:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ awk -F'[][&amp;gt;&amp;lt;]' '{ if(/Name=&amp;quot;Title&amp;quot;/){desc=$3; spc=$4}else if(/Name=&amp;quot;AccessionVersion&amp;quot;/){name=$3} if($0==&amp;quot;&amp;lt;/DocSum&amp;gt;&amp;quot;){print name,desc,spc; name=&amp;quot;&amp;quot;; desc=&amp;quot;&amp;quot;; spc=&amp;quot;&amp;quot;}}' file.xml &#xA;XP_025107904.1 receptor-type tyrosine-protein phosphatase kappa-like isoform X2  Pomacea canaliculata&#xA;PVD22034.1 hypothetical protein C0Q70_17837  Pomacea canaliculata&#xA;AJA37836.1 fucolectin-related molecule  Littorina littorea&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I stress that this is a very bad idea and you should use a proper XML parser, but it might work...&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2022-10-17T17:13:12.707" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19859" PostTypeId="1" CreationDate="2022-10-17T18:48:30.913" Score="1" ViewCount="83" Body="&lt;p&gt;I'm trying to run a snakefile for the first time with limited coding experience using salmon to index a reference genome. I'm not too sure what I'm doing wrong based on this error message.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error: Building DAG of jobs...&#xA;MissingRuleException:&#xA;No rule to produce snakefile (if you use input functions make sure that they don't raise unexpected exceptions).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule salmon_index:&#xA;        input:&#xA;                R1=&amp;quot;sequences/*R1.fastq.gz&amp;quot;,&#xA;                R2=&amp;quot;sequences/*R2.fastq.gz&amp;quot;&#xA;        output:&#xA;                Index=&amp;quot;results/salmon_index&amp;quot;&#xA;        shell:&#xA;                &amp;quot;&amp;quot;&amp;quot;&#xA;                salmon index --threads 8 -t references/genome.fa.gz -i salmon_index&#xA;                &amp;quot;&amp;quot;&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16331" LastEditorUserId="650" LastEditDate="2022-10-17T20:22:52.007" LastActivityDate="2022-10-17T20:26:04.173" Title="Snakemake MissingRuleException" Tags="|rna-seq|python|snakemake|conda|salmon|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19860" PostTypeId="1" AcceptedAnswerId="20048" CreationDate="2022-10-17T19:41:43.383" Score="5" ViewCount="119" Body="&lt;p&gt;I am fairly new to scRNAseq analysis and keep running into the same problem in the two datasets I am currently working with. We work with kidney inflammation and have two new mouse models, which we are now analysing in scRNAseq. I followed the usual Seurat workflow and ended up getting some nice UMAPs. With &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/scDblFinder/inst/doc/scDblFinder.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;FindAllMarkers&lt;/code&gt;&lt;/a&gt; I was able to identify the clusters pretty well.&lt;/p&gt;&#xA;&lt;p&gt;But once I do differential expression analysis (either with &lt;code&gt;FindMarkers&lt;/code&gt; or &lt;code&gt;pseudobulking&lt;/code&gt;) I got weird results.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; There is a highly specialized epithelial celltype in the kidney (the podocyte) that specifically expresses a host of markers (like e.g. &lt;em&gt;Nphs1&lt;/em&gt; and &lt;em&gt;Nphs2&lt;/em&gt;). They get downregulated when the kidney gets damaged (as it does in our disease models).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I do not expect to see those markers to be present or regulated in any other celltype. But when I do the &lt;strong&gt;DE&lt;/strong&gt;, those podocyte-specific marker genes pop up in almost all other clusters and are described to be highly regulated (downregulated, as I would expect in the pdoocytes).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;So in my eyes, this points to &amp;quot;doublets of podocytes&amp;quot; and other celltypes being involved here. Maybe some damaged podocytes got into the same droplet as other celltypes and thus distort the analysis.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;I then used somewhat stringent filtering via &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/scDblFinder/inst/doc/scDblFinder.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;scDblFinder&lt;/code&gt;&lt;/a&gt;* to remove a lot of doublets (~ 10 % of my dataset).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;However, I still get the same problem.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Then I analysed each batch (we have 3 in total) individually, but in each one this problem comes up.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Overall, I see 6.4% of doublets (that is 8,710 doublets in a total of 135,844 cells). I have two podocyte clusters with 1.7% doublets (that is 462 doublets in a total of 26,693 podocytes).&lt;/p&gt;&#xA;&lt;p&gt;Some help / advice would be highly appreciated as I am becoming a little frustrated with this issue.&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot!&lt;/p&gt;&#xA;&lt;p&gt;*, Bioconductor package&lt;/p&gt;&#xA;" OwnerUserId="16332" LastEditorUserId="73" LastEditDate="2022-10-20T08:55:30.900" LastActivityDate="2022-12-20T01:07:32.773" Title="Ubiquitous regulation of highly specific marker genes" Tags="|r|rna-seq|scrnaseq|seurat|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19861" PostTypeId="2" ParentId="19859" CreationDate="2022-10-17T20:26:04.173" Score="2" Body="&lt;p&gt;You're not using wildcards the way snakemake is equipped to work with them. See:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;FAQs such as &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/project_info/faq.html#how-do-i-run-my-rule-on-all-files-of-a-certain-directory&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://snakemake.readthedocs.io/en/stable/project_info/faq.html#how-do-i-run-my-rule-on-all-files-of-a-certain-directory&lt;/a&gt; and &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/project_info/faq.html#i-don-t-want-expand-to-use-the-product-of-every-wildcard-what-can-i-do&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://snakemake.readthedocs.io/en/stable/project_info/faq.html#i-don-t-want-expand-to-use-the-product-of-every-wildcard-what-can-i-do&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#wildcards&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#wildcards&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="650" LastActivityDate="2022-10-17T20:26:04.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19862" PostTypeId="2" ParentId="19855" CreationDate="2022-10-18T07:19:07.053" Score="2" Body="&lt;p&gt;you can try this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df = data.frame(p = c(&amp;quot;ENSG00000198400&amp;quot;,&amp;quot;ENSG00000066044&amp;quot;,&amp;quot;ENSG00000188612&amp;quot;,&#xA;                       &amp;quot;ENSG00000177606&amp;quot;,&amp;quot;ENSG00000146648&amp;quot;))&#xA;df1 = data.frame(p1 = c(&amp;quot;ENSG00000105647&amp;quot;,&amp;quot;ENSG00000110395&amp;quot;,&amp;quot;ENSG00000165119&amp;quot;,&#xA;                         &amp;quot;ENSG00000160293&amp;quot;,&amp;quot;ENSG00000198400&amp;quot;,&amp;quot;ENSG00000198400&amp;quot;),&#xA;                 p2 = c(&amp;quot;ENSG00000198400&amp;quot;,&amp;quot;ENSG00000198400&amp;quot;,&amp;quot;ENSG00000198400&amp;quot;,&#xA;                         &amp;quot;ENSG00000142949&amp;quot;,&amp;quot;ENSG00000112936&amp;quot;,&amp;quot;ENSG00000162434&amp;quot;))&#xA;&#xA;df2 = rbind(df1[df1&lt;span class=&quot;math-container&quot;&gt;$p1 %in% df$&lt;/span&gt;p,],df1[df1&lt;span class=&quot;math-container&quot;&gt;$p2 %in% df$&lt;/span&gt;p,])&#xA;&#xA;df2 &amp;lt;- transform(df2, p2 = ifelse(df2&lt;span class=&quot;math-container&quot;&gt;$p2%in%df$&lt;/span&gt;p, p1, p2), p1 = ifelse(df2&lt;span class=&quot;math-container&quot;&gt;$p2%in%df$&lt;/span&gt;p, p2, p1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15931" LastActivityDate="2022-10-18T07:19:07.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19864" PostTypeId="1" CreationDate="2022-10-18T12:30:33.010" Score="1" ViewCount="106" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GQbqb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GQbqb.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The above is a 2D histogram of y and z internal coordinates describing the positions of C-alpha atoms involved in hydrogen bonds between antiparallel Beta-strands.&lt;/p&gt;&#xA;&lt;p&gt;There is no color coding. It is given as it is.&lt;/p&gt;&#xA;&lt;p&gt;What can we assume about the data points if a histogram is circular?&lt;/p&gt;&#xA;&lt;p&gt;When does a histogram take a circular shape?&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;[Note: &amp;quot;&lt;a href=&quot;https://plotly.com/python/2D-Histogram/&quot; rel=&quot;nofollow noreferrer&quot;&gt;2D histogram&lt;/a&gt;&amp;quot; is a synonym for what bioinformaticians typically call heat maps, see &lt;a href=&quot;https://stackoverflow.com/a/42262754/459780&quot;&gt;here&lt;/a&gt;]&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="73" LastEditDate="2022-10-19T11:44:26.507" LastActivityDate="2022-10-19T14:50:32.307" Title="When does a 2D histogram / heatmap take a circular shape?" Tags="|protein-structure|homework|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19866" PostTypeId="1" CreationDate="2022-10-18T14:34:34.327" Score="1" ViewCount="103" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://onlinelibrary.wiley.com/doi/full/10.1002/jcc.26372&quot; rel=&quot;nofollow noreferrer&quot;&gt;High-performance transformation of protein structure representation from internal to Cartesian coordinates&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DwjRd.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DwjRd.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In Figure 3a, each residue has its own local coordinates, which are illustrated shown using different colors.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here are two more papers:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/2204.10673.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generative De Novo Protein Design with Global Context&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/pdf/2204.09410.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Generating 3D Molecules for Target Protein Binding&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;By global coordinate, I understand the coordinate system in which we view the entire protein chain.&lt;/p&gt;&#xA;&lt;p&gt;What does it &lt;strong&gt;generally&lt;/strong&gt; (not only in this particular paper) mean by a Local Coordinate in the case of protein chains?&lt;/p&gt;&#xA;&lt;p&gt;Why was it necessary to introduce this terminology when they could have just used the term &amp;quot;vector&amp;quot;?&lt;/p&gt;&#xA;&lt;p&gt;Can you supply me with an example with an explanatory diagram?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="12686" LastEditDate="2022-10-19T11:00:45.380" LastActivityDate="2023-04-11T14:56:23.830" Title="What does it mean by a Local Coordinate in the case of protein chains?" Tags="|protein-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19867" PostTypeId="1" CreationDate="2022-10-18T15:59:39.987" Score="1" ViewCount="320" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/9542032/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am trying to create a Seurat object using the package &amp;quot;Seurat&amp;quot;. When I am reading my raw files using the function &lt;code&gt;Read10X&lt;/code&gt;, I am getting this error&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;Error in dimnamesGets(x, value) : length of Dimnames[[1]] (45948) is not equal to Dim[1] (45947).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am unsure how to resolve this issue.&lt;/p&gt;&#xA;&lt;p&gt;R sessionInfo:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;R version 4.1.2 (2021-11-01)&#xA;Platform: x86_64-w64-mingw32/x64 (64-bit)&#xA;Running under: Windows 10 x64 (build 19044)&#xA;&#xA;Matrix products: default&#xA;&#xA;locale:&#xA;[1] LC_COLLATE=English_United States.1252  LC_CTYPE=English_United States.1252&#xA;[3] LC_MONETARY=English_United States.1252 LC_NUMERIC=C&#xA;[5] LC_TIME=English_United States.1252&#xA;&#xA;attached base packages:&#xA;[1] stats     graphics  grDevices utils     datasets  methods   base&#xA;&#xA;other attached packages:&#xA;[1] sp_1.5-0           SeuratObject_4.1.1 Seurat_4.1.1       Matrix_1.5-1&#xA;&#xA;loaded via a namespace (and not attached):&#xA;  [1] nlme_3.1-153          spatstat.sparse_2.1-1 matrixStats_0.62.0    RcppAnnoy_0.0.19&#xA;  [5] RColorBrewer_1.1-3    httr_1.4.4            sctransform_0.3.4     tools_4.1.2&#xA;  [9] utf8_1.2.2            R6_2.5.1              irlba_2.3.5           rpart_4.1-15&#xA; [13] KernSmooth_2.23-20    uwot_0.1.14           mgcv_1.8-38           rgeos_0.5-9&#xA; [17] DBI_1.1.3             lazyeval_0.2.2        colorspace_2.0-3      tidyselect_1.1.2&#xA; [21] gridExtra_2.3         compiler_4.1.2        progressr_0.11.0      cli_3.4.0&#xA; [25] plotly_4.10.0         scales_1.2.1          lmtest_0.9-40         spatstat.data_2.2-0&#xA; [29] ggridges_0.5.3        pbapply_1.5-0         goftest_1.2-3         stringr_1.4.1&#xA; [33] digest_0.6.29         spatstat.utils_2.3-1  pkgconfig_2.0.3       htmltools_0.5.3&#xA; [37] parallelly_1.32.1     fastmap_1.1.0         htmlwidgets_1.5.4     rlang_1.0.5&#xA; [41] rstudioapi_0.14       shiny_1.7.2           generics_0.1.3        zoo_1.8-10&#xA; [45] jsonlite_1.8.0        spatstat.random_2.2-0 ica_1.0-3             dplyr_1.0.10&#xA; [49] magrittr_2.0.3        patchwork_1.1.2       Rcpp_1.0.9            munsell_0.5.0&#xA; [53] fansi_1.0.3           abind_1.4-5           reticulate_1.26       lifecycle_1.0.2&#xA; [57] stringi_1.7.6         MASS_7.3-54           Rtsne_0.16            plyr_1.8.7&#xA; [61] grid_4.1.2            parallel_4.1.2        listenv_0.8.0         promises_1.2.0.1&#xA; [65] ggrepel_0.9.1         deldir_1.0-6          miniUI_0.1.1.1        lattice_0.20-45&#xA; [69] cowplot_1.1.1         splines_4.1.2         tensor_1.5            pillar_1.8.1&#xA; [73] igraph_1.3.4          spatstat.geom_2.4-0   future.apply_1.9.1    reshape2_1.4.4&#xA; [77] codetools_0.2-18      leiden_0.4.3          glue_1.6.2            data.table_1.14.2&#xA; [81] png_0.1-7             vctrs_0.4.1           httpuv_1.6.6          polyclip_1.10-0&#xA; [85] gtable_0.3.1          RANN_2.6.1            purrr_0.3.4           spatstat.core_2.4-4&#xA; [89] tidyr_1.2.1           scattermore_0.8       future_1.28.0         assertthat_0.2.1&#xA; [93] ggplot2_3.3.6         mime_0.12             xtable_1.8-4          later_1.3.0&#xA; [97] survival_3.2-13       viridisLite_0.4.1     tibble_3.1.8          cluster_2.1.2&#xA;[101] globals_0.16.1        fitdistrplus_1.1-8    ellipsis_0.3.2        ROCR_1.0-11&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16335" LastEditorUserId="650" LastEditDate="2022-10-19T21:06:53.793" LastActivityDate="2022-10-19T21:06:53.793" Title="Error in dimnamesGets(x, value) when trying to read data using Seurat package" Tags="|r|scrnaseq|seurat|bioconductor|single-cell|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19868" PostTypeId="1" CreationDate="2022-10-18T18:27:38.987" Score="2" ViewCount="297" Body="&lt;p&gt;I found a definition:&#xA;&amp;quot;A pattern is thus a qualitative description of a motif in terms of amino acid sequence. The concept of a profile extends this concept, allowing a quantitative description of a motif, by assigning probabilities to the occurrence of a particular amino acid at each position of a motif.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;This is example of profile in prosite&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Hits for all PROSITE (release 2022_04) motifs on sequence P98073 [UniProtKB/Swiss-Prot]:&#xA;Example of profile&#xA;PS50024   SEA   SEA domain profile :&#xA;54 - 169:   score = 32.979&#xA;QSHEARATFKITsGVTYNPNLQDKLSVDFKVLAFDLQQMIDEIFLSSNLKNEYKNSRVLQ&#xA;FEN--GSIIVVFDLFFAQWVSD--ENVKEELIQGLEANKssQLVTFHIDLNSVDILDKLT&#xA;&#xA;This is example of pattern in Prosite&#xA;PS01209   LDLRA_1   LDL-receptor class A (LDLRA) domain signature :&#xA;197 - 221:     [confidence level: (0)]  CIkadlf.CDgevNCpdgsDEDnkm..C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What is the difference in that case? So in profile I have probabilities (in definition I have &amp;quot;profile extends this concept, allowing a quantitative description of a motif, by assigning probabilities to the occurrence of a particular amino acid&amp;quot;. I don't see any probabilities in that sequence.&lt;/li&gt;&#xA;&lt;li&gt;What does it mean &amp;quot;-&amp;quot; in profiles? Maybe any aminoacid or gap?&lt;/li&gt;&#xA;&lt;li&gt;Why I have sometimes lowercase letter and uppercase letter in profile and pattern? I don't get it. It's a probability?&lt;/li&gt;&#xA;&lt;li&gt;What is more important profile or pattern?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16327" LastActivityDate="2022-11-19T08:27:03.070" Title="What is the difference between profile and pattern in Prosite?" Tags="|proteins|prosite|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19869" PostTypeId="2" ParentId="19867" CreationDate="2022-10-19T07:12:09.417" Score="1" Body="&lt;p&gt;Check the length of your files. The error clearly says that there is a difference in the length of your files. Your file length should be equal.&lt;/p&gt;&#xA;&lt;p&gt;I think the problem may be the presence of a header in one of your files. Removing the header may solve the problem.&lt;/p&gt;&#xA;" OwnerUserId="15931" LastActivityDate="2022-10-19T07:12:09.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19870" PostTypeId="2" ParentId="19867" CreationDate="2022-10-19T07:54:17.497" Score="1" Body="&lt;p&gt;There is a mismatch of 1 line between the several files, suggesting that probably either the barcodes of features file has a header line which is not standard. Remove any header lines from these files.&lt;/p&gt;&#xA;" OwnerUserId="16342" LastActivityDate="2022-10-19T07:54:17.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19871" PostTypeId="2" ParentId="19856" CreationDate="2022-10-19T13:11:21.083" Score="1" Body="&lt;p&gt;There are several things you can try. For example, Simka (&lt;a href=&quot;https://github.com/GATB/simka&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/GATB/simka&lt;/a&gt;) is tool that compares the profiles of the genomic content of fastq files and generates several types of plots e.g. heatmap.&lt;/p&gt;&#xA;&lt;p&gt;Another tool that might be useful is &lt;a href=&quot;https://multiqc.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;multifastqc&lt;/a&gt; (&lt;a href=&quot;https://multiqc.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://multiqc.info/&lt;/a&gt;), which aggregates results and has multiple options.&lt;/p&gt;&#xA;" OwnerUserId="2245" LastActivityDate="2022-10-19T13:11:21.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19872" PostTypeId="2" ParentId="19814" CreationDate="2022-10-20T05:05:56.143" Score="0" Body="&lt;p&gt;&amp;quot;Often, one needs to search ... a specific subset of sequences within an existing database. For these type of searches a convenient way to conduct them is by creating a virtual BLAST database&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;From &amp;quot;Use blastdb_aliastool to manage the BLAST databases&amp;quot; &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK569848/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/books/NBK569848/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1692" LastActivityDate="2022-10-20T05:05:56.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19873" PostTypeId="1" CreationDate="2022-10-20T05:27:14.217" Score="1" ViewCount="60" Body="&lt;p&gt;I am in need to benchmark the variant identification pipeline which uses RNA seq data alone without any matched-normal. I would like to know the reference dataset (and the pipeline on which the dataset was run) that can I use to compare the pipeline results on my dataset. The reference dataset which I use should be comparable regardless of the dataset specific to certain disease. Thanks all in advance for your responses.&lt;/p&gt;&#xA;" OwnerUserId="16346" LastActivityDate="2022-10-20T05:27:14.217" Title="Benchmarking for variant identification using RNA-seq data" Tags="|rna-seq|vcf|variant-calling|variation|benchmarking|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19874" PostTypeId="2" ParentId="933" CreationDate="2022-10-20T07:22:25.223" Score="1" Body="&lt;p&gt;I created a python package based on SQLite databases, where you can easily query all gnomAD variants for GRCh37/38. &lt;a href=&quot;https://github.com/KalinNonchev/gnomAD_DB&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/KalinNonchev/gnomAD_DB&lt;/a&gt; I have precomputed SQLite databases for gnomAD WGS for GRCh37/38 in the description of the package. Please take a look there.&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="16347" LastEditorUserId="16347" LastEditDate="2022-10-20T07:22:41.927" LastActivityDate="2022-10-20T07:22:41.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19875" PostTypeId="1" CreationDate="2022-10-20T14:27:49.237" Score="1" ViewCount="90" Body="&lt;p&gt;We are dealing with relatively large family data (&amp;gt; 1,000 trios) with genotyping array data.&#xA;We would like to see the allele effects on children separately of mothers, fathers, and children, as in a previous study (&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/34282336/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PMID: 34282336&lt;/a&gt;).&#xA;Unfortunately, this is difficult for us to replicate, so we need another method.&#xA;Is there any software that can achieve this or a similar traditional approach?&#xA;Also, can that be applied to grandparents?&lt;/p&gt;&#xA;&lt;p&gt;In my understanding of the above mentioned study is they:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;identified the parental genomes within the child genome;&lt;/li&gt;&#xA;&lt;li&gt;thereon conducted a GWAS for birth weight&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Overall, the aim of the previous study was to estimate the alleles responsible for birth weight or in our situation a comparable phenotype. It is important to note the prior study did not use publicly available algorithms.&lt;/p&gt;&#xA;&lt;p&gt;Thus, we wish to identify the maternal and paternal alleles within their progeny via public domain algorithms, and therefore do not necessarily need to exactly reproduce the previous study.&lt;/p&gt;&#xA;&lt;p&gt;We already have imputed data, and are using GCTA for usual GWAS. To achieve our goal, we need to identify the transmitted and non-transmitted alleles in parents.&lt;/p&gt;&#xA;" OwnerUserId="15346" LastEditorUserId="73" LastEditDate="2022-10-21T18:26:17.163" LastActivityDate="2022-10-21T18:26:17.163" Title="Allele effects by parental-origin estimated from genotyping array data" Tags="|genome|gwas|software-recommendation|microarray|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19876" PostTypeId="1" CreationDate="2022-10-20T15:03:31.723" Score="2" ViewCount="140" Body="&lt;p&gt;I'm  working with a vcf file, in particular I'm trying to identify a gene duplication in a sequenced genome.&lt;/p&gt;&#xA;&lt;p&gt;I studied bcftools cnv command, but for this tool is needed a vcf file containing a BAF and LRR values, that there are not present in my vcf file.&lt;/p&gt;&#xA;&lt;p&gt;And so the first question is, can I calculate them starting from read depth and allele depth? Once that I have calculate BAF and LRR, can I modify the original vcf file inserting the calculated values? To have a vcf file containing BAF and LRR values I have to ask to the geneticist to use particular pipeline in NGS instrument?&lt;/p&gt;&#xA;" OwnerUserId="16349" LastEditorUserId="298" LastEditDate="2022-10-21T12:43:21.610" LastActivityDate="2022-10-21T12:43:21.610" Title="BAF and LRR calculation and usage" Tags="|vcf|ngs|bcftools|allele-frequency|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19877" PostTypeId="1" CreationDate="2022-10-20T15:55:49.267" Score="3" ViewCount="723" Body="&lt;p&gt;I want to filter my VCF file to include only the relevant information I need but I have some questions about the results I got.&lt;/p&gt;&#xA;&lt;p&gt;This is what the first 10 lines of my VCF file looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  DATA_VARIANT&#xA;chrI    826 .   T   C   255 .   CNV=1   GT:PL:GQ:DP:BSDP:ACN    1/1:344,30,0:57:10:0,10,0,0:0,5&#xA;chrI    831 .   C   T   71  .   CNV=1   GT:PL:GQ:DP:BSDP:ACN    0/1:139,36,274:71:12:0,8,0,4:3,2&#xA;chrI    836 .   G   T   59  .   CNV=1   GT:PL:GQ:DP:BSDP:ACN    0/1:140,49,373:59:16:0,0,12,4:4,1&#xA;chrI    848 .   C   T   255 .   CNV=1   GT:PL:GQ:DP:BSDP:ACN    1/1:556,48,0:75:16:0,0,0,16:0,5&#xA;chrI    855 .   G   A   255 .   CNV=1   GT:PL:GQ:DP:BSDP:ACN    0/1:399,48,139:59:16:12,0,4,0:1,4&#xA;chrI    858 .   ATT AT  255 .   CNV=1;TYPE=INDEL    GT:PL:GQ:DP:ADP:ACN 0/1:480,48,160:85:16:4,12:1,4&#xA;chrI    876 .   T   C   255 .   CNV=1   GT:PL:GQ:DP:BSDP:ACN    1/1:865,75,0:102:25:0,25,0,0:0,5&#xA;chrI    904 .   G   A   255 .   CNV=1   GT:PL:GQ:DP:BSDP:ACN    1/1:1261,112,0:138:37:37,0,0,0:0,5&#xA;chrI    920 .   A   T   67  .   CNV=1   GT:PL:GQ:DP:BSDP:ACN    0/1:244,145,1426:67:48:41,0,0,7:4,1&#xA;chrI    925 .   T   C   255 .   CNV=1   GT:PL:GQ:DP:BSDP:ACN    1/1:1786,157,1:255:52:0,52,0,0:0,5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;QUAL:&lt;/strong&gt;&#xA;As I understand it, the QUAL is based off of Phred quality scores, which would mean that a QUAL value of anything greater than 20 would be fairly sufficient in terms of quality for that genotype. However, most of my QUAL scores are 255 and from what I've read that means there is no mapping quality available?&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What does this mean for my data? I used the same tool (NGSEP) to map and call the variants on this strain so it should have mapping quality scores.&lt;/li&gt;&#xA;&lt;li&gt;Do I need to filter out all the 255 scores? Or can they still be used?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;PL:&lt;/strong&gt; From what I've read about PL scores, they indicate the likelihood of the presence of a genotype (0/0, 0/1, 1/1). Based on the results, there should be a clear least likely genotype, &amp;quot;almost&amp;quot; likely genotype, and a most likely genotype indicated by a value of 0.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Some of my PL values do not include a 0 (i.e., the second variant call = 139,36,274). What does this mean for my data?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Any help would be great! Thank you!&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; I used NGSEP as my variant caller&lt;/p&gt;&#xA;" OwnerUserId="16067" LastActivityDate="2022-10-21T03:42:12.323" Title="PL and QUAL values on VCF file?" Tags="|vcf|filtering|quality-control|variant-filtration|phred-scores|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19879" PostTypeId="2" ParentId="19877" CreationDate="2022-10-21T03:42:12.323" Score="5" Body="&lt;p&gt;The &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.2.pdf&quot; rel=&quot;noreferrer&quot;&gt;VCF specification&lt;/a&gt; provides the definition for the QUAL field.&lt;/p&gt;&#xA;&lt;p&gt;However, QUAL values are often capped by variant callers to a given value. I have seen 100 being used and according to &lt;a href=&quot;https://www.google.com/url?q=https://cgspace.cgiar.org/bitstream/handle/10568/68332/UserManualNGSEP.pdf%3Fsequence%3D1%26isAllowed%3Dy&amp;amp;sa=U&amp;amp;ved=2ahUKEwjpiZKorvD6AhV2QfEDHfHDAhcQFnoECAIQAg&amp;amp;usg=AOvVaw36qKssVh0QwBsvE47IafmQ&quot; rel=&quot;noreferrer&quot;&gt;this documentation&lt;/a&gt; on NGSEP it seems 255 has been chosen for this caller as the maximum value. So you are all good keeping these high QUAL variant sites and filtering only those below a certain value (e.g. 20)&lt;/p&gt;&#xA;&lt;p&gt;PL is also defined in the VCF specification: phred-scaled genotype likelihoods rounded to the closest integer (and otherwise defined precisely as&#xA;the GL field).&lt;/p&gt;&#xA;&lt;p&gt;Again, different variant callers may have different implementations for the specification. &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035890451-Calculation-of-PL-and-GQ-by-HaplotypeCaller-and-GenotypeGVCFs&quot; rel=&quot;noreferrer&quot;&gt;GATK for example&lt;/a&gt; normalizes the values so that the most likely genotype has a PL of 0. But this might not be followed in other callers. It seems that NGSEP simply does not apply this normalization or does it differently. I think this is a detail you can ask directly the authors &lt;a href=&quot;https://github.com/NGSEP/NGSEPcore&quot; rel=&quot;noreferrer&quot;&gt;here&lt;/a&gt; by opening an &amp;quot;issue&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;My suggestion is that you rather use GQ (genotype quality) if you want to hard-filter your variants. This is more standard to do, as opposed to filtering using PLs. These values are also in phred-scale.&lt;/p&gt;&#xA;" OwnerUserId="475" LastActivityDate="2022-10-21T03:42:12.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19881" PostTypeId="2" ParentId="19875" CreationDate="2022-10-21T11:45:00.763" Score="1" Body="&lt;p&gt;The first haplotyping step is typically done in two stages:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Imputation of missing genotypes - can be done using the &lt;a href=&quot;https://imputationserver.sph.umich.edu/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Michigan Imputation Server&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Phasing of called genotypes to create semi-contiguous haplotypes - can be done using &lt;a href=&quot;https://odelaneau.github.io/shapeit4/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SHAPEIT&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I have &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/8563/73&quot;&gt;many thoughts&lt;/a&gt; about the best approach for GWAS (mostly around the idea of population subsampling), which could potentially also apply to what you want to do. Unfortunately, it's been a long time since I did family population genetics, and I've forgotten the things we did to deal with complex population structure. I'm pretty sure we used &lt;a href=&quot;https://www.cog-genomics.org/plink/2.0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PLINK&lt;/a&gt; for at least some of it, but I notice that PLINK has a disclaimer about close family structure, which suggests it may not work so well for trio data:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;This method does not properly adjust for small-scale family structure. As a consequence, it is usually necessary to prune close relations with e.g. --king-cutoff before using --glm for genome-wide association analysis. (Note that biobank data usually comes with a relationship-pruned sample ID list; you can use --keep on that list, instead of performing your own expensive --king-cutoff run.) If this throws out more samples than you'd like, consider using mixed model association software such as SAIGE, BOLT-LMM, GCTA, or FaST-LMM instead; or regenie's whole genome regression.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-10-21T11:45:00.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19882" PostTypeId="1" AcceptedAnswerId="19883" CreationDate="2022-10-21T15:55:52.247" Score="2" ViewCount="108" Body="&lt;p&gt;Hi I'm trying to filter a column in a PAF file with the AWK command and write it to a new file:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;awk '$12&amp;gt;=20 {print $0}' /PATH/FILE.paf &amp;gt; /OUT/PATH/FITLERED_FILE.paf&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;This command makes sure the 12th column has a value of at least 20 and then prints the entire contents of the file - with the filtered column - to a new file.&lt;/p&gt;&#xA;&lt;p&gt;This works for one file but I have 90+ I need to filter through. I tried using a for loop:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;for i in /PATH/TO/FILES/*_mapped.paf; do awk '$12&amp;gt;=20 {print $0 &amp;gt; &amp;quot;awk_{}.out&amp;quot;}' $i; done&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Which didn't work. And then I tried using a pipe command:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;cat /Users/rimo/Desktop/Pat_Assemblies/flye_mapped.txt | parallel &amp;quot;awk '{$12&amp;gt;=20 print $0}' /Users/rimo/Desktop/Pat_Assemblies/Assemblies_Flye/{} &amp;gt; awk_{}&amp;quot;&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Which also didn't work. I think the problem I am having is format-based but I would appreciate it if someone could steer me in the right direction!&lt;/p&gt;&#xA;" OwnerUserId="16067" LastActivityDate="2022-10-21T16:47:46.080" Title="Filtering file with AWK and writing output to new file" Tags="|filtering|awk|parallel|command-line|paf|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19883" PostTypeId="2" ParentId="19882" CreationDate="2022-10-21T16:30:51.057" Score="3" Body="&lt;p&gt;The &lt;code&gt;{}&lt;/code&gt; isn't in any way special to the shell or &lt;code&gt;awk&lt;/code&gt;. It is used as a replacement by certain other programs, off the top of my head I can only think of &lt;code&gt;find&lt;/code&gt; and &lt;code&gt;xargs&lt;/code&gt; where it is used by convention, but it isn't a shell thing or an awk thing. Here, you are using it inside the &lt;code&gt;awk&lt;/code&gt; command so it just breaks the awk's format because &lt;code&gt;{ }&lt;/code&gt; introduces a command block and this one is empty.&lt;/p&gt;&#xA;&lt;p&gt;Your file name is &lt;code&gt;$i&lt;/code&gt;, so if you want to make &lt;code&gt;foo_mapped.paf&lt;/code&gt; into &lt;code&gt;awk_foo_mapped.paf.out&lt;/code&gt;, you need &lt;code&gt;awk_$i.out&lt;/code&gt;. Something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for file in /PATH/TO/FILES/*_mapped.paf; do &#xA;    fileName=&amp;quot;${file##*/}&amp;quot;&#xA;    awk '$12&amp;gt;=20' &amp;quot;$file&amp;quot; &amp;gt; awk_&amp;quot;$fileName&amp;quot;.out&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note how I removed the &lt;code&gt;print $0&lt;/code&gt;. This is because the default action for awk when something evaluates to &lt;code&gt;true&lt;/code&gt; is to print the current line, so &lt;code&gt;awk '$12&amp;gt;=20 {print $0 }&lt;/code&gt; is equivalent to &lt;code&gt;awk '$12&amp;gt;=20&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You can also use your original approach of having &lt;code&gt;awk&lt;/code&gt; deal with the file name by using awk's internal &lt;code&gt;FILENAME&lt;/code&gt; variable. Like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '$12&amp;gt;=20{ print &amp;gt; &amp;quot;awk_&amp;quot;FILENAME&amp;quot;.out&amp;quot; }' /PATH/TO/FILES/*_mapped.paf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Depending on the number of files, you might need to close the file handle after writing. If you get an error about too many open files, try:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mawk '$12&amp;gt;=20{print &amp;gt; &amp;quot;awk_&amp;quot;FILENAME&amp;quot;.out&amp;quot;; close(&amp;quot;awk_&amp;quot;FILENAME&amp;quot;.out&amp;quot;)}' /PATH/TO/FILES/*_mapped.paf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2022-10-21T16:47:46.080" LastActivityDate="2022-10-21T16:47:46.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19885" PostTypeId="1" AcceptedAnswerId="19887" CreationDate="2022-10-21T20:01:49.797" Score="2" ViewCount="184" Body="&lt;p&gt;I have several fasta files that represent the proteins from different orthogroups. However, the orthogroup that each of the proteins belongs to is not indicated within the files. I am leveraging &lt;a href=&quot;https://biopython.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioPython&lt;/a&gt; and I want to add this information to the header of each sequence in the fasta files.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import os&#xA;from pathlib import Path&#xA;from Bio import SeqIO&#xA;&#xA;with open('output.fasta', 'w') as output:&#xA;    for file in os.listdir(dirname):&#xA;        seq = SeqIO.parse(file, 'fasta')&#xA;        ortog = Path(file).stem&#xA;        for record in seq:&#xA;            record.description = ortog&#xA;        SeqIO.write(record, output, 'fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, the output always appears with the last sequence duplicated. In the case of three fasta files, the output appears as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;BBC_8_672 OG0015396-PROT&#xA;MKFSSVTAITLATVATVATAKKGEHDFTTTLTLSSDGSLTTTTSTHTTHKYGKFNKTSKS&#xA;KTPNHTGTHKYGKFNKTSKSKTP&#xA;&amp;gt;CAT_6_3128 OG0020551-PROT&#xA;MEKIKEKLNSLKLESESWQEKYEELREQLKELEQSNTEKENEIKSLSAKNEQLDSEVEKL&#xA;ESQLSDTKQLAEDSNNLRSNNENYTKKNQDLEQQLEDSEAKLKEAMDKLKEADLNSEQMG&#xA;RRIVALEEERDEWEKKCEEFQSKYEEAQKELDEIANSLENL&#xA;&amp;gt;ZSA_0_3121 OG0025647-PROT&#xA;LLLQYQLHPALLPKALLLQYQLHQALLLKALLLQ*PALLLKALLLQYQLHQALLLKALLL&#xA;QYQLHPALLLKALLLQYQLHP&#xA;&amp;gt;ZSA_0_3121 OG0025647-PROT&#xA;LLLQYQLHPALLPKALLLQYQLHQALLLKALLLQ*PALLLKALLLQYQLHQALLLKALLL&#xA;QYQLHPALLLKALLLQYQLHP&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Could anyone tell me why this is happening?&lt;/p&gt;&#xA;" OwnerUserId="16352" LastEditorUserId="3848" LastEditDate="2022-10-21T20:17:19.933" LastActivityDate="2022-10-24T09:11:52.450" Title="Add file name to FASTA file header using SeqIO" Tags="|python|phylogenetics|orthologues|multi-fasta|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19887" PostTypeId="2" ParentId="19885" CreationDate="2022-10-21T20:19:05.517" Score="0" Body="&lt;p&gt;This is because you are also parsing your output file. Remember, the first rule of debugging is to print all the things! Most bugs are because some variable doesn't actually have the value you think it has, and printing them is the best way of finding the problem. I reproduced what (I am guessing) is your setup here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ ls&#xA;OG0015396-PROT.fa  OG0020551-PROT.fa  OG0025647-PROT.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then ran your script with an extra &lt;code&gt;print&lt;/code&gt; statement:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;import os&#xA;from pathlib import Path&#xA;from Bio import SeqIO&#xA;&#xA;with open('output.fasta', 'w') as output:&#xA;       for file in os.listdir(&amp;quot;.&amp;quot;):&#xA;              print(&amp;quot;FF:&amp;quot;,file)&#xA;              seq = SeqIO.parse(file, 'fasta')&#xA;              ortog = Path(file).stem&#xA;              for record in seq:&#xA;                     record.description = ortog&#xA;              SeqIO.write(record, output, 'fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which printed:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ foo.py&#xA;FF: OG0015396-PROT.fa&#xA;FF: OG0020551-PROT.fa&#xA;FF: output.fasta&#xA;FF: OG0025647-PROT.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see, since the &lt;code&gt;output.fasta&lt;/code&gt; file is created by the &lt;code&gt;open()&lt;/code&gt; call, by the time you use &lt;code&gt;listdir&lt;/code&gt;, it's already there and included. This results in duplicated sequences (not always the last one). I'm sure there are more elegant solutions, but a simple workaround is to skip the output file name:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;import os&#xA;from pathlib import Path&#xA;from Bio import SeqIO&#xA;&#xA;with open('output.fasta', 'w') as output:&#xA;       for file in os.listdir(&amp;quot;.&amp;quot;):&#xA;              if file == &amp;quot;output.fasta&amp;quot;:&#xA;                     continue&#xA;              seq = SeqIO.parse(file, 'fasta')&#xA;              ortog = Path(file).stem&#xA;              for record in seq:&#xA;                     record.description = ortog&#xA;              SeqIO.write(record, output, 'fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Of course, this whole thing is a bit overkill for such a simple task and you could just use &lt;code&gt;sed&lt;/code&gt; and the shell instead:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for file in *fa; do sed &amp;quot;1s|$| ${file//.fa}|&amp;quot; &amp;quot;$file&amp;quot;; done &amp;gt; output.fasta &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2022-10-21T20:19:05.517" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19888" PostTypeId="1" AcceptedAnswerId="19891" CreationDate="2022-10-21T20:47:07.943" Score="2" ViewCount="821" Body="&lt;p&gt;Does anyone have any experience with PAF files (minimap2 output)? I don’t fully understand what my output means.&lt;/p&gt;&#xA;&lt;p&gt;I’ve looked at the minimap2 explanation (&lt;a href=&quot;https://github.com/lh3/miniasm/blob/master/PAF.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/lh3/miniasm/blob/master/PAF.md&lt;/a&gt;) but I want to use the data here to go back and extract regions in the genome that the gene target mapped to. I just don't know what specific columns to look at to accomplish this.&lt;/p&gt;&#xA;&lt;p&gt;Here is a little snippet of a PAF file I have:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;contig_25   522460  514241  515712  +   GENE_TARGET 4988    1962    3649    258   1687  60&#xA;contig_26   413461  403718  406719  +   GENE_TARGET 4988    17      3649    1762  3633  60&#xA;contig_44   24218   16005   17476   +   GENE_TARGET 4988    1962    3649    258   1687  60&#xA;contig_52   244411  236199  237670  +   GENE_TARGET 4988    1962    3649    258   1687  60&#xA;contig_57   1043583 1025163 1026895 +   GENE_TARGET 4988    3046    4986    1376  2059  60&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is this saying that the gene target mapped to coordinates &lt;code&gt;514241 through 515712&lt;/code&gt; on &lt;code&gt;contig_25&lt;/code&gt; but only coordinates &lt;code&gt;1962 through 3649&lt;/code&gt; of the gene actually mapped to that contig? I guess I’m mostly confused on the descriptions for the coordinates and what they actually mean.&lt;/p&gt;&#xA;&lt;p&gt;I think for my use case I have to use the gene target as the reference sequence as I am trying to extract homologous regions to sequences of interest. I.e., my project involves 90+ assemblies and I would like to extract the regions in the contigs where they map to the gene target. I used minimap2 to map the assemblies to the gene target, used sort and awk to get the best/high quality matches, converted PAF to BED format, and then used seqtk subseq to extract the sequences from the respective assemblies.&lt;/p&gt;&#xA;&lt;p&gt;This is what I get when I use the gene target as a reference sequence:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GENE    4988    17      3649    +   contig_26   413461  403718  406719  1762    3633    60&#xA;...&#xA;GENE    4988    3046    4986    +   contig_57   1043583 1025163 1026895 1376    2059    60&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Would I use columns 8 and 9 as the region range to go back and extract the sequences from?&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="73" LastEditDate="2022-10-24T21:07:38.560" LastActivityDate="2022-10-31T20:44:33.623" Title="Understanding PAF format" Tags="|read-mapping|minimap2|paf|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19891" PostTypeId="2" ParentId="19888" CreationDate="2022-10-22T22:02:51.617" Score="5" Body="&lt;p&gt;As @user172818 mentioned in a comment to your question, it looks like you might have used the query as the reference; they should be switched round.&lt;/p&gt;&#xA;&lt;p&gt;A minimap2 mapping command is typically represented as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 reference.fa query.fa &amp;gt; results.paf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This follows the &amp;quot;indexed reference first&amp;quot; command line order used in the overwhelming majority of mapping tools. In the minimap2 PAF output (which you have linked), the first column represents the &lt;em&gt;query&lt;/em&gt; sequence name, and the 6th column represents the &lt;em&gt;target&lt;/em&gt; sequence name (i.e. the opposite order from the command line order, also following with established convention).&lt;/p&gt;&#xA;&lt;p&gt;Based on the desired mapping operation that you have described (i.e. &amp;quot;I want to use the data here to go back and extract regions in the genome that the gene target mapped to&amp;quot;), your minimap2 command should probably look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 genome_contigs.fa GENE_TARGET.fa &amp;gt; results.paf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will appropriately map the target region to the regions in the genome, and then columns 8 and 9 will represent &amp;quot;regions in the genome that the gene target mapped to.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Minimap2 (and other mappings programs) match query sequences to a reference, with the usual expectation being that the output will contain a single primary mapping per input query sequence (sometimes with secondary mappings to other locations). In other words, they try to find one match to any reference sequence for each query. Changing the order of mapping to target-as-reference means that the program will attempt to find one match to the target sequence(s) for each input reference sequence, which is usually not what is desired.&lt;/p&gt;&#xA;&lt;p&gt;In any case, the columns in the PAF file correspond to the closest sequence ID to the &lt;em&gt;left&lt;/em&gt; of them (which is, confusingly, ordered the opposite way round to the command line order):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ minimap2 [genome_contigs.fa] {GENE_TARGET.fa}&#xA;{GENE 4988 17   3649} + [contig_26 413461  403718  406719 ] 1762 3633 60&#xA;...&#xA;{GENE 4988 3046 4986} + [contig_57 1043583 1025163 1026895] 1376 2059 60&#xA;&#xA;$ minimap2 [GENE_TARGET.fa] {genome_contigs.fa}&#xA;{contig_26 413461  403718  406719 } + [GENE 4988 17   3649] 1762 3633 60&#xA;...&#xA;{contig_57 1043583 1025163 1026895} + [GENE 4988 3046 4986] 1376 2059 60&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-10-31T20:44:33.623" LastActivityDate="2022-10-31T20:44:33.623" CommentCount="1" CommunityOwnedDate="2022-10-22T22:02:51.617" ContentLicense="CC BY-SA 4.0" />
  <row Id="19892" PostTypeId="1" AcceptedAnswerId="19898" CreationDate="2022-10-24T17:53:00.983" Score="1" ViewCount="110" Body="&lt;p&gt;GnomAD has publicly available VCF files that have variant data gathered from over 15k individuals. For a university project, I want to randomly select 1k of these individuals and get the variant data belgoning to them only.&lt;/p&gt;&#xA;&lt;p&gt;I am very new to bioinformatics in general and upon checking VCF files and how they work, my current conclusion is that this task is not achievable through filtering the vcf file in any way since variants in these files have no direct connection to the genomes they were found in (no array of individual IDs in a variant entry that show which individuals had this variant).&lt;/p&gt;&#xA;&lt;p&gt;But this task was given to me by a professor whom I believe is competent to realize the task was impossible if it is like I thought, so is there something I am missing?&lt;/p&gt;&#xA;&lt;p&gt;How can I go about randomly selecting a 1000 individuals from this 15 000 pool, and then generating a new VCF file with the data from these selected 1000?&lt;/p&gt;&#xA;&lt;p&gt;Are there any fields that show which individuals genome samples showed a spesific variant entry in the VCF file?&lt;/p&gt;&#xA;" OwnerUserId="16379" LastEditorUserId="16379" LastEditDate="2022-10-25T12:26:01.147" LastActivityDate="2022-10-26T09:35:57.703" Title="Is picking variants belonging to a specific group of genomes from VCF file possible?" Tags="|vcf|gnomad|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19893" PostTypeId="1" AcceptedAnswerId="20087" CreationDate="2022-10-24T22:36:01.547" Score="1" ViewCount="102" Body="&lt;p&gt;I try to pass a &lt;em&gt;.fasta&lt;/em&gt; file with headers where tabs are used as separators as input file to &lt;a href=&quot;https://toolshed.g2.bx.psu.edu/repository/display_tool?repository_id=1d92ebdf7e8d466c&amp;amp;tool_config=%2Fsrv%2Ftoolshed-repos%2Fmain%2F000%2Frepo_241%2Ftools%2Fncbi_blast_plus%2Fncbi_segmasker_wrapper.xml&amp;amp;changeset_revision=5edc472ec434&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;NCBI BLAST+ segmasker&lt;/em&gt;&lt;/a&gt;:&lt;br /&gt;&#xA;&lt;code&gt;&amp;gt;A0A059VAR9    IPR022879   V-type ATP synthase regulatory subunit B/beta&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, the &lt;em&gt;segmasker&lt;/em&gt; output's headers are shortened to the first tab:&lt;br /&gt;&#xA;&lt;code&gt;&amp;gt;A0A059VAR9&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to get whole headers in the output file (without replacing all the tabs in the input file with white spaces)?&lt;/p&gt;&#xA;" OwnerUserId="9651" LastEditorUserId="9651" LastEditDate="2022-11-26T11:09:27.470" LastActivityDate="2022-11-29T01:10:10.040" Title="How to get full headers with tabs as separators with segmasker?" Tags="|fasta|phylogenetics|segmasker|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19895" PostTypeId="1" CreationDate="2022-10-25T05:53:21.373" Score="1" ViewCount="122" Body="&lt;p&gt;I have 3884 PDB files generated by &lt;a href=&quot;http://www.sbg.bio.ic.ac.uk/%7Ephyre2/PD2_ca2main/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pd2_ca2main&lt;/a&gt;. PD2 CA2main is server for reconstructing the backbone of Calpha proteins using Gaussian mixture models.&lt;/p&gt;&#xA;&lt;p&gt;All of these files have duplicate atoms or alternate coordinates.&lt;/p&gt;&#xA;&lt;p&gt;How can I clean these PDB files using Visual Molecular Dynamics (&lt;a href=&quot;https://www.ks.uiuc.edu/Training/SumSchool/materials/sources/tutorials/01-vmd-tutorial/html/node2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;VMD&lt;/a&gt;) or Scaleable Molecular Dynamics (NAMD) &lt;strong&gt;*&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Can anyone tell me what command I should use in console mode?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;*&lt;/strong&gt; note, this server may be unavailable&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="3848" LastEditDate="2022-12-17T09:46:10.373" LastActivityDate="2022-12-17T09:46:10.373" Title="How can I clean a PDB file using VMD or NAMD?" Tags="|protein-structure|pdb|protein-protein-interaction|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19896" PostTypeId="1" CreationDate="2022-10-25T08:51:55.087" Score="1" ViewCount="180" Body="&lt;p&gt;I am using SIMCA software to perform multivariate analysis and want a VIP scores plot in R as shown below. &lt;a href=&quot;https://i.stack.imgur.com/8YLC6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8YLC6.png&quot; alt=&quot;I want this plot from the VIP scores I retrieved from SIMCA&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;which package is used for this type of plot (image retrieved from Google-images)??&lt;/p&gt;&#xA;" OwnerUserId="9234" LastActivityDate="2022-10-25T08:51:55.087" Title="Create VIP plot using R from the SIMCA VIP values" Tags="|ggplot2|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19898" PostTypeId="2" ParentId="19892" CreationDate="2022-10-25T13:54:55.187" Score="0" Body="&lt;p&gt;It is. A VCF ought to be sub-settable to a specific list of samples. The only thing you have to come up with for your specific case, is a list of 1K random sample IDs within the original VCF.&lt;/p&gt;&#xA;&lt;p&gt;Geenerally speaking, knowing that you can do this and how involves getting familiar with the &lt;strong&gt;&lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VCF specification&lt;/a&gt;&lt;/strong&gt; as well as the main bioinfo tool used to manipulate and work with this format: &lt;strong&gt;&lt;a href=&quot;https://samtools.github.io/bcftools/bcftools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;BCFtools&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Let's use &lt;strong&gt;&lt;a href=&quot;https://www.internationalgenome.org/data/&quot; rel=&quot;nofollow noreferrer&quot;&gt;1000 genomes data&lt;/a&gt;&lt;/strong&gt; as an example.&lt;/p&gt;&#xA;&lt;p&gt;You can use a bit of UNIX together with BCFtools in order to get a list of 1K random IDs from individuals:&lt;/p&gt;&#xA;&lt;p&gt;(sort -R instead of shuf might work too)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools query -l ALL.chr22.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz | shuf | head -1000 &amp;gt; myRandomIDs.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then again BCFtools to subset the list of individuals from the original VCF and into a new VCF file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view --samples-file myRandomIDs.txt ALL.chr22.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.vcf.gz -o myNewVCF.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you dig a bit deeper into the BCFtools documentation, you will find many things you can do extra if you wanted, like re-calculating the allele frequencies based on this new cohort of individuals, if you wanted that too. Or subset for positions/variants, instead of individuals.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;GnomAD data&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Using GnomAD data, selecting 1K random samples is impossible since data is aggregated and instead of genotypes per individual (and individuals you can select thus) we only have the frequency of the variants across various sub-cohorts within the GnomAD project. No individual level data.&lt;/p&gt;&#xA;&lt;p&gt;The most you can do is select one of such sub-cohorts and filter for variants with a frequency &amp;gt; 0 (or a range) in such cohort. You could get variants present only and only is a specific cohort too, by using either frequency information or other info such as allele counts, etc.&lt;/p&gt;&#xA;&lt;p&gt;The example below filters for variants with a frequency above 0 in the South Asian cohort and 0 frequency in the Southern European cohort. You can have a look at the header of the VCF in order to get to know all cohorts included and the meaning of all the information contained within the INFO filed in the VCF&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools filter -i &amp;quot;AF_sas &amp;gt; 0 &amp;amp; AF_nfe_seu = 0&amp;quot; gnomad.exomes.r2.1.1.sites.21.vcf.bgz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I'm afraid that's not exactly what you wanted to do in the first place.&lt;/p&gt;&#xA;" OwnerUserId="475" LastEditorUserId="475" LastEditDate="2022-10-26T09:35:57.703" LastActivityDate="2022-10-26T09:35:57.703" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19899" PostTypeId="1" CreationDate="2022-10-25T16:01:51.703" Score="3" ViewCount="53" Body="&lt;p&gt;I am planning on performing phylogenetic analysis of Salmonella specimens using WGS data from PulseNet and GenomeTrakr, for the purpose of public health surveillance &amp;amp; to provide context for foodborne outbreaks.&lt;/p&gt;&#xA;&lt;p&gt;I have been using the Nextstrain workflow to generate and visualize trees for SARS-CoV-2 specimens, and I am planning on adapting the bacterial workflow outlined here for my bacterial isolates: &lt;a href=&quot;https://docs.nextstrain.org/en/latest/tutorials/creating-a-bacterial-pathogen-workflow.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.nextstrain.org/en/latest/tutorials/creating-a-bacterial-pathogen-workflow.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;What approach would you use to process my reads into VCF files for phylogenetic analysis? I've been referencing information from this similar post from 5 years ago: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/654/how-do-i-generate-a-variant-list-i-e-vcf-file-using-illumina-reads-from-a-hum&quot;&gt;How do I generate a variant list (i.e. VCF file) using Illumina reads from a human genome?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;How much of this information is still current/relevant and applicable to Salmonella isolates?&lt;/p&gt;&#xA;" OwnerUserId="16387" LastActivityDate="2023-03-25T04:03:00.397" Title="How do I generate a variant list using Illumina reads from a Salmonella genome?" Tags="|phylogenetics|variant-calling|read-mapping|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19900" PostTypeId="1" CreationDate="2022-10-25T17:02:22.157" Score="1" ViewCount="16" Body="&lt;p&gt;I had a few questions raised by the below diagram in relation to how ChIP-seq reads look like (&lt;a href=&quot;https://hbctraining.github.io/Intro-to-ChIPseq/lessons/05_peak_calling_macs.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;). I would be very grateful for your insight on them!&lt;/p&gt;&#xA;&lt;p&gt;From the below diagram, it seems that, in a ChIP-seq experiment, we get information about the binding site in relation to both the Watson and Crick strands. That is, from our experiment, we get reads complimentary to both strands.&lt;/p&gt;&#xA;&lt;p&gt;(Q1) I suppose this is because proteins bind to both strands of DNA, so that in our ChIP-seq experiment, we ultimately sequence reads from both the Watson and the Crick strands? Is that true?&lt;/p&gt;&#xA;&lt;p&gt;Also, my impression from the &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/gb-2008-9-9-r137&quot; rel=&quot;nofollow noreferrer&quot;&gt;MACS paper&lt;/a&gt; was that &amp;quot;ChIP-Seq tags represent only the ends of the ChIP fragments, instead of precise protein-DNA binding site&amp;quot; (Background).&lt;/p&gt;&#xA;&lt;p&gt;(Q2) Therefore, shouldn't it be that we get reads mapping to either side of the binding site on BOTH strands? That is, we get reads more towards the 3' end of the Watson strand, and reads more towards the 5' end of the Watson strand, and similarly for the Crick strand.&lt;/p&gt;&#xA;&lt;p&gt;(Q3) The diagram below shows we &lt;em&gt;only&lt;/em&gt; get reads more towards the 5' end of the strands. Why is that?&lt;/p&gt;&#xA;&lt;p&gt;(Q4) Finally, I know that MACs shifts reads by some bases towards the 3' end for each strand, in order to interpret the precise binding site location. It seems to assume what I am asking about in (Q3). However, &lt;em&gt;&lt;strong&gt;if&lt;/strong&gt;&lt;/em&gt; the answer to my (Q2) is &amp;quot;yes, we do get 4 such clusters of reads&amp;quot;, then why doesn't MACs account for that?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MjTqr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MjTqr.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16389" LastActivityDate="2022-10-25T17:02:22.157" Title="Questions about ChIP-seq reads and MACS" Tags="|chip-seq|macs2|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19901" PostTypeId="1" CreationDate="2022-10-25T18:37:15.857" Score="1" ViewCount="120" Body="&lt;p&gt;I have two new amino acid gene sequences that I  want to align to a reference genome. Is it a good idea to have a combined phylogenetic tree? what will this type phylogenetic analysis called?&lt;/p&gt;&#xA;" OwnerUserId="16390" LastEditorUserId="3848" LastEditDate="2023-08-13T00:02:35.677" LastActivityDate="2023-08-13T00:02:35.677" Title="How can I align two novel sequences to a reference genome and build a phylogenetic tree?" Tags="|python|phylogenetics|sequence-alignment|phylogeny|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19902" PostTypeId="1" CreationDate="2022-10-25T18:53:51.277" Score="1" ViewCount="297" Body="&lt;p&gt;I'm trying to create a Nextflow script that maps assemblies to reference gene sequences. This is what I have so far:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;#!/usr/bin/env nexflow&#xA;nextflow.enable.dsl = 1&#xA;&#xA;&#xA;params.assmblreads = '/Users/rimo/Desktop/Assemblies/Assemblies_Flye/flye_assembly.fasta'&#xA;params.assmblref = '/Users/rimo/Desktop/Assemblies/gene_targets/GENE.fsa'&#xA;&#xA;refgenome_file = file(params.assmblref)&#xA;fasta_files = Channel.fromPath(params.assmblreads)&#xA;                     .map { file -&amp;gt; tuple(file.baseName, file) }&#xA;&#xA;&#xA;process minimapping {&#xA;&#xA;    input:&#xA;    file genome from refgenome_file&#xA;    set val(readID), file(readFile) from fasta_files&#xA;&#xA;    output:&#xA;    set val(readID), file(&amp;quot;${readID}.paf&amp;quot;) into mapped_reads&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    ./minimap2 &lt;span class=&quot;math-container&quot;&gt;$readFile $&lt;/span&gt;genome &amp;gt; ${readID}.paf&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm running this on a macOS so I have to include the &amp;quot;./&amp;quot; before a command or it will give the error: command not found&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;But everytime I run the script with the command: &lt;code&gt;./nextflow run trials.nf&lt;/code&gt; it results in this error message:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(base) rimo@Ritas-MBP minimap2 % ./nextflow run ../attempt_out.nf&#xA;N E X T F L O W  ~  version 22.10.0&#xA;Launching `../attempt_out.nf` [nasty_baekeland] DSL1 - revision: 98d1deaebd&#xA;executor &amp;gt;  local (1)&#xA;[ae/47e3c0] process &amp;gt; minimapping (1) [  0%] 0 of 1&#xA;Error executing process &amp;gt; 'minimapping (1)'&#xA;&#xA;Caused by:&#xA;executor &amp;gt;  local (1)&#xA;[ae/47e3c0] process &amp;gt; minimapping (1) [100%] 1 of 1, failed: 1 ✘&#xA;Error executing process &amp;gt; 'minimapping (1)'&#xA;&#xA;Caused by:&#xA;  Process `minimapping (1)` terminated with an error exit status (1)&#xA;&#xA;Command executed:&#xA;&#xA;  ./minimap2 flye_assembly.fasta GENE.fsa &amp;gt; flye_assembly.paf&#xA;&#xA;Command exit status:&#xA;  1&#xA;&#xA;Command output:&#xA;  (empty)&#xA;&#xA;Command error:&#xA;  .command.sh: line 2: ./minimap2: No such file or directory&#xA;&#xA;Work dir:&#xA;  /Users/rimo/Desktop/AWS/minimap2/work/ae/47e3c0708f33b1949315fc83066b95&#xA;&#xA;Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is my first time ever writing a Nextflow script so I'm very new and confused to this new way of scripting. If someone could help me out that would be great!&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="73" LastEditDate="2022-10-26T10:52:12.480" LastActivityDate="2022-10-26T10:52:12.480" Title="Nextflow with Minimap2 not working" Tags="|read-mapping|nextflow|errors|minimap2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19903" PostTypeId="2" ParentId="19899" CreationDate="2022-10-25T18:58:48.603" Score="1" Body="&lt;p&gt;I would express caution about cross-over because for both SARS-CoV-2 and the human genome you can obtain a good assembly from a reference genome. The SARS-CoV-2 genome is so small and read depth high enough this is not needed  however. SARS-CoV-2 there are critical issues in assemblies for certain regions which have not been addressed, but for CDS there's no real difficulties here.&lt;/p&gt;&#xA;&lt;p&gt;A reference genome assembly is  not good practice for what are often environmental bacteria (bacteria &lt;em&gt;per se&lt;/em&gt;) and &lt;em&gt;de novo&lt;/em&gt; assembly is strongly recommended, even if it is to complement a reference genome assembly. Bacterial evolution is very different from either human or virus evolution, it's full of indels, missing genes, genes 'flying' around on plasmids and potential non-homologous recombination, risk of rearrangements etc ...&lt;/p&gt;&#xA;&lt;p&gt;The challenge you face is automating the assemblies and then polishing those assemblies and there has been enormous progress recently, &amp;quot;closing the gaps&amp;quot; manually is a thing of the past - ancient history. There should be no under-estimation as the volume of recent work that has been dedicated particularly towards long read data. Even pre-assembly, pre-polishing the progress shouldn't be underestimated, albeit I suspect you're familiar with that. The old ideas of QUAST are now historic. The raised ante is noticeable.&lt;/p&gt;&#xA;&lt;p&gt;Back to the original question consistently using a reference genome risks something going wrong that could cause an outlier artefact in the VCF and in my opinion its no longer cool for bacteria.&lt;/p&gt;&#xA;&lt;p&gt;The rationale is clear, mis-assembly  would cause an artefact in the tree and therefore in the epidemiological inference of the outbreak.&lt;/p&gt;&#xA;&lt;p&gt;It's worth doing specific background reading because albeit I've deliberately not mentioned a given approach except for QUAST, the rate of progress has been significant. The world has changed.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-10-26T03:11:53.030" LastActivityDate="2022-10-26T03:11:53.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19904" PostTypeId="1" AcceptedAnswerId="19911" CreationDate="2022-10-25T19:19:47.643" Score="2" ViewCount="406" Body="&lt;p&gt;I use &lt;strong&gt;bedtools&lt;/strong&gt; to convert genomic coordinates (in the form of bed files, chrX x1:x2) to genomic sequence content in the form of fasta files ('ACGT...G').&lt;/p&gt;&#xA;&lt;p&gt;The bedtools format for extracting sequence (&lt;em&gt;.fa file) from coordinates (&lt;/em&gt;.bed file) is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bedtools getfasta -fi  input_sequences.fa -bed  genomic_coordinates.bed -fo output_sequences.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Very frequently I get a &amp;quot;Segmentation fault (cored dumped)&amp;quot; during the run.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Warning: the index file is older than the FASTA file.&#xA;Segmentation fault (core dumped)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The strange thing is that the error will go away, if for example I add additional data (chromosomes) to the input fasta file (input_sequences.fa), OR remove some chromosomes?? I am having trouble finding a logical path to explain this behavior so I can work around it.&lt;/p&gt;&#xA;&lt;p&gt;There is no changing of fasta files while bedtools is working - in fact there is nothing in the primary directory other than the two files, the fasta file input_sequence.fa and the bed file genomic_coordinates.bed. There is 256GB of RAM memory on the machine, but that's not an issue, especially since the error goes away sometimes when additional chromosomes added to the fasta file.&lt;/p&gt;&#xA;&lt;p&gt;Anyone experienced issues with bedtools getfasta erratic Segmentation faults? Known workarounds? I am using the current version of bedtools v2.30.0&lt;/p&gt;&#xA;&lt;p&gt;What are some of the alternatives to &amp;quot;bedtools getfasta&amp;quot; for converting genomic coordinates to sequences?&lt;/p&gt;&#xA;&lt;p&gt;Thanks...&lt;/p&gt;&#xA;" OwnerUserId="15741" LastEditorUserId="73" LastEditDate="2022-10-27T08:49:22.567" LastActivityDate="2022-10-27T08:52:19.323" Title="converting coordinates to sequences using bedtools getfasta - [segmentation fault] errors" Tags="|bedtools|pybedtools|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19905" PostTypeId="2" ParentId="19901" CreationDate="2022-10-25T22:19:25.220" Score="2" Body="&lt;p&gt;Edit: just to point out a tree needs 4 taxa (sequences), so rather than one reference genome, 3 reference genomes. Thus for Blast you might need to produce a database (using 3 reference genomes) before Blasting. There are other Blast approaches to obtaining a large meaningful sample size ... thats a separate question&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;It called a combined alignment, or concatenated genes.&lt;/p&gt;&#xA;&lt;p&gt;The theory is simply that if two genes evolve at noticeably different rates they can't be combined into a single alignment and make a tree from them. The theory goes back a long but the modern test is in IQTREE.&lt;/p&gt;&#xA;&lt;p&gt;You take your two alignments and test them for equal rates via IQTREE&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;iqtree2 -s example.phy -p example.nex --symtest-only&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Testing for a combined alignment is also described in my past answer &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/19659/single-or-partitioned-model-for-supermatrix-tree-inference/19667#19667&quot;&gt;here&lt;/a&gt;: The previous question was can you combined an rRNA gene with a protein gene and the answer is ...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Okay, you can't concatenate genes into a singular matrix particularly if these are rRNA genes versus protein genes, except in rare situations where the tests say 'thats okay' (below). This is simply because they evolve at very different rates rRNA genes have loci which evolve very slowly and are highly conserved.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The first part of the question was mapping the gene to a reference genome. Anything will do that ... local Blast works and using that is described in my answer to this question &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/19814/is-there-an-equivalent-of-entrez-queries-for-local-blast&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;If you loads of fasta seqs you'll need to make a database...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;makeblastdb -in example700-1000.fa -parse_seqids -blastdb_version 5  -out laura700-1000.db -title &amp;quot;Laura 700 - 1000&amp;quot; -dbtype nucl&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then do this ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastn -query laura.fas -db laura700-1000.db -evalue 1e-6 -num_threads 4 -out blasthits.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However if its just reference vs. query then you can just use the above command.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The alignment is best done via a specialist alignment package Muscle Super5 and that is described in my answer &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/18899/sequence-alignment-with-muscle&quot;&gt;here&lt;/a&gt;. This is done after your obtained the homologous regions in your reference genomes.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;HMM Super5 is a hidden Markov model (HMM) where gap penalties and the substitution matrix are the parameters at the centre of the model. That is the answer, thats why you can't change it - it does it itself and thats its purpose. Thus one algorithm (super5) is HMM for the gap-penalty and the other one I suspect isn't.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-10-26T10:40:51.320" LastActivityDate="2022-10-26T10:40:51.320" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="19906" PostTypeId="1" AcceptedAnswerId="19910" CreationDate="2022-10-25T22:53:54.143" Score="3" ViewCount="39" Body="&lt;p&gt;I was working with a given GFF3 file, and I observed that some transcripts have orientation opposite to their transcripts. Here is a snippet:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1    .       gene    2189548 2194772 .       +       .       ID=c5eaf8c1-2bb2-566f-e154-6a9be797c11f;Name=FAAP20;gene_biotype=protein_coding;gene_id=34a96673-bfab-ebbf-b2b7-f9cb73163a4a;gene_name=FAAP20&#xA;chr1    .       transcript      2189548 2194772 .       -       .       ID=f47feeac-74ce-1543-53b6-afd9bbb8ab19;Name=rna-NM_182533.4;gene_biotype=protein_coding;gene_id=34a96673-bfab-ebbf-b2b7-f9cb73163a4a;gene_name=FAAP20;Parent=c5eaf8c1-2bb2-566f-e154-6a9be797c11f;transcript_biotype=protein_coding;transcript_name=rna-NM_182533.4;protein_id=NP_872339.3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How do I interpret this? I checked UCSC Browser and in each case it seems that the browser agrees with the transcript in each case, which seems like it should be canonical.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, I observed in this file that gene orientation is always &lt;code&gt;+&lt;/code&gt;. Whereas a highly curated GFF3 of the human genome represents both directions:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# my file&#xA;$ awk '$3 == &amp;quot;gene&amp;quot; {print}' filtered_potential_genes.gff | cut -f7 | sort | uniq -c&#xA;   1600 +&#xA;# highly curated annotations&#xA;$ awk '$3 == &amp;quot;gene&amp;quot; {print}' /MANE.GRCh38.v0.95.select_refseq_genomic.gff | cut -f7 | sort|  uniq -c &#xA;   9393 +&#xA;   9191 -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;According to the person who provided me the file, the existence of bidirectional transcription was indeed the motivation for this. However, it's obviously confusing to have it still be a definitive &lt;code&gt;+&lt;/code&gt; rather than &lt;code&gt;.&lt;/code&gt; or something to signify unstrandedness.&lt;/p&gt;&#xA;&lt;p&gt;It seems likely that this is due to an artifact in my file. Are there other interpretations? Is the standard simply that gene orientation is meaningless and transcript orientation is important?&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="73" LastEditDate="2022-10-26T22:07:20.383" LastActivityDate="2022-10-26T22:07:20.383" Title="What does it mean when a gene and its transcript have opposite orientations in a GFF3 file?" Tags="|sequence-annotation|gff3|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19907" PostTypeId="2" ParentId="19902" CreationDate="2022-10-26T00:49:27.853" Score="4" Body="&lt;blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;Command error:&#xA;  .command.sh: line 2: ./minimap2: No such file or directory&#xA;&#xA;Work dir:&#xA;  /Users/rimo/Desktop/AWS/minimap2/work/ae/47e3c0708f33b1949315fc83066b95&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The problem is that your command is expecting the &lt;code&gt;minimap2&lt;/code&gt; binary inside your working directory (i.e. the 'work dir', as defined above), but it could not be found. To avoid using relative (or absolute) paths, and to help keep your workflow portable, instead move your &lt;code&gt;minimap2&lt;/code&gt; binary into a directory in your &lt;code&gt;$PATH&lt;/code&gt;. Then, you should be able to call &lt;code&gt;minimap2&lt;/code&gt; without a relative (or absolute) path to it. Here's one way using the new &lt;a href=&quot;https://www.nextflow.io/docs/latest/dsl2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DSL2&lt;/a&gt;, which comes enabled by default with Nextflow 22.10.0:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params.assmblreads = '/Users/rimo/Desktop/Assemblies/Assemblies_Flye/flye_assembly.fasta'&#xA;params.assmblref = '/Users/rimo/Desktop/Assemblies/gene_targets/GENE.fsa'&#xA;&#xA;&#xA;process minimap2 {&#xA;&#xA;    input:&#xA;    tuple val(readID), path(readFile)&#xA;    path genome&#xA;&#xA;    output:&#xA;    tuple val(readID), path(&amp;quot;${readID}.paf&amp;quot;)&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    minimap2 &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${readFile}&quot; &quot;$&lt;/span&gt;{genome}&amp;quot; &amp;gt; &amp;quot;${readID}.paf&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;workflow {&#xA;&#xA;    refgenome_file = file( params.assmblref )&#xA;    fasta_files = Channel.fromPath( params.assmblreads )&#xA;                         .map { file -&amp;gt; tuple( file.baseName, file ) }&#xA;&#xA;    minimap2( fasta_files, refgenome_file )&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To make your pipeline more portable, and to avoid manually installing and configuring your own tools, another way is to use &lt;a href=&quot;https://www.nextflow.io/docs/latest/conda.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conda environments&lt;/a&gt;. Nextflow has built-in support for &lt;a href=&quot;https://docs.conda.io/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Conda&lt;/a&gt; and gives you access to all of the tools, including &lt;a href=&quot;https://anaconda.org/bioconda/minimap2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Minimap2&lt;/a&gt;, available in &lt;a href=&quot;https://bioconda.github.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioconda&lt;/a&gt;. Install &lt;a href=&quot;https://docs.conda.io/en/latest/miniconda.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;miniconda&lt;/a&gt; to get started, then add &lt;code&gt;conda.enabled = true&lt;/code&gt; to your &lt;a href=&quot;https://www.nextflow.io/docs/latest/config.html#configuration-file&quot; rel=&quot;nofollow noreferrer&quot;&gt;nextflow.config&lt;/a&gt; and the &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#conda&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;conda&lt;/code&gt;&lt;/a&gt; directive (e.g. &lt;code&gt;conda 'minimap2=2.24'&lt;/code&gt;) to your process definition for example. In the same way, you could also use &lt;a href=&quot;https://www.nextflow.io/docs/latest/container.html#docker&quot; rel=&quot;nofollow noreferrer&quot;&gt;Docker&lt;/a&gt; (or &lt;a href=&quot;https://www.nextflow.io/docs/latest/container.html#singularity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Singularity&lt;/a&gt;) containers with the &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#container&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;container&lt;/code&gt;&lt;/a&gt; directive. Note that Docker containers are also built by the Bioconda build system and these are uploaded to &lt;a href=&quot;https://quay.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;quay.io&lt;/a&gt; as part of the &lt;a href=&quot;https://quay.io/organization/biocontainers&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biocontainers project&lt;/a&gt; (e.g. &lt;a href=&quot;https://quay.io/repository/biocontainers/minimap2&quot; rel=&quot;nofollow noreferrer&quot;&gt;biocontainers/minimap2&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2022-10-26T01:45:00.793" LastActivityDate="2022-10-26T01:45:00.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19909" PostTypeId="1" CreationDate="2022-10-26T10:06:52.943" Score="3" ViewCount="183" Body="&lt;p&gt;I have a collection of fasta files, each containing three aligned sequences. I am interested in understanding the distribution of a specific sequence motif PGP, RGP and KGP in all the alignments. I was wondering if there is possible way in R to create a heatmap highlighting the location of these motifs.&lt;/p&gt;&#xA;&lt;p&gt;I have tried the code in this &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/13414/how-can-i-plot-multiple-sequence-alignment-in-r-as-a-heatmap&quot;&gt;post&lt;/a&gt; to create a heatmap but it is not working as expected. One issue is that ggmsa does not import the fasta file. The fasta file has to be copied into the installation directory for it to work. I have close to 50 aligned files, so not very efficient. The other major problem is it creates a heatmap compared to a reference sequence and not the motifs. Can't figure how to change this.&lt;/p&gt;&#xA;&lt;p&gt;Any help is much appreciated. Two fasta files containing aligned sequence are as follows,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;chain A CO1A1_HUMAN/1-1464 Collagen alpha-1(I) chain&#xA;MFSFVDLRLLLLLAATALLTHGQEEGQVEGQDEDIPPITCVQNGLRYHDRDVWKPEPCRICVCDNGKVLCDDVICDETKNCPGAEVPEGECCPVCPDGSESPTDQETTGVEGPKGDTGPRGPRGPAGPPGRDGIPGQPGLPGPPGPPGPPGPPGLGGNFAPQLSYGYDEKSTGGISVPGPMGPSGPRGLPGPPGAPGPQGFQGPPGEPGEPGASGPMGPRGPPGPPGKNGDDGEAGKPGRPGERGPPGPQGARGLPGTAGLPGMKGHRGFSGLDGAKGDAGPAGPKGEPGSPGENGAPGQMGPRGLPGERGRPGAPGPAGARGNDGATGAAGPPGPTGPAGPPGFPGAVGAKGEAGPQGPRGSEGPQGVRGEPGPPGPAGAAGPAGNPGADGQPGAKGANGAPGIAGAPGFPGARGPSGPQGPGGPPGPKGNSGEPGAPGSKGDTGAKGEPGPVGVQGPPGPAGEEGKRGARGEPGPTGLPGPPGERGGPGSRGFPGADGVAGPKGPAGERGSPGPAGPKGSPGEAGRPGEAGLPGAKGLTGSPGSPGPDGKTGPPGPAGQDGRPGPPGPPGARGQAGVMGFPGPKGAAGEPGKAGERGVPGPPGAVGPAGKDGEAGAQGPPGPAGPAGERGEQGPAGSPGFQGLPGPAGPPGEAGKPGEQGVPGDLGAPGPSGARGERGFPGERGVQGPPGPAGPRGANGAPGNDGAKGDAGAPGAPGSQGAPGLQGMPGERGAAGLPGPKGDRGDAGPKGADGSPGKDGVRGLTGPIGPPGPAGAPGDKGESGPSGPAGPTGARGAPGDRGEPGPPGPAGFAGPPGADGQPGAKGEPGDAGAKGDAGPPGPAGPAGPPGPIGNVGAPGAKGARGSAGPPGATGFPGAAGRVGPPGPSGNAGPPGPPGPAGKEGGKGPRGETGPAGRPGEVGPPGPPGPAGEKGSPGADGPAGAPGTPGPQGIAGQRGVVGLPGQRGERGFPGLPGPSGEPGKQGPSGASGERGPPGPMGPPGLAGPPGESGREGAPGAEGSPGRDGSPGAKGDRGETGPAGPPGAPGAPGAPGPVGPAGKSGDRGETGPAGPAGPVGPVGARGPAGPQGPRGDKGETGEQGDRGIKGHRGFSGLQGPPGPPGSPGEQGPSGASGPAGPRGPPGSAGAPGKDGLNGLPGPIGPPGPRGRTGDAGPVGPPGPPGPPGPPGPPSAGFDFSFLPQPPQEKAHDGGRYYRADDANVVRDRDLEVDTTLKSLSQQIENIRSPEGSRKNPARTCRDLKMCHSDWKSGEYWIDPNQGCNLDAIKVFCNMETGETCVYPTQPSVAQKNWYISKNPKDKRHVWFGESMTDGFQFEYGGQGSDPADVAIQLTFLRLMSTEASQNITYHCKNSVAYMDQQTGNLKKALLLQGSNEIEIRAEGNSRFTYSVTVDGCTSHTGAWGKTVIEYKTTKTSRLPIIDVAPLDVGAPDQEFGFDVGPVCFL&#xA;&amp;gt;chain B CO1A1_HUMAN/1-1464 Collagen alpha-1(I) chain&#xA;MFSFVDLRLLLLLAATALLTHGQEEGQVEGQDEDIPPITCVQNGLRYHDRDVWKPEPCRICVCDNGKVLCDDVICDETKNCPGAEVPEGECCPVCPDGSESPTDQETTGVEGPKGDTGPRGPRGPAGPPGRDGIPGQPGLPGPPGPPGPPGPPGLGGNFAPQLSYGYDEKSTGGISVPGPMGPSGPRGLPGPPGAPGPQGFQGPPGEPGEPGASGPMGPRGPPGPPGKNGDDGEAGKPGRPGERGPPGPQGARGLPGTAGLPGMKGHRGFSGLDGAKGDAGPAGPKGEPGSPGENGAPGQMGPRGLPGERGRPGAPGPAGARGNDGATGAAGPPGPTGPAGPPGFPGAVGAKGEAGPQGPRGSEGPQGVRGEPGPPGPAGAAGPAGNPGADGQPGAKGANGAPGIAGAPGFPGARGPSGPQGPGGPPGPKGNSGEPGAPGSKGDTGAKGEPGPVGVQGPPGPAGEEGKRGARGEPGPTGLPGPPGERGGPGSRGFPGADGVAGPKGPAGERGSPGPAGPKGSPGEAGRPGEAGLPGAKGLTGSPGSPGPDGKTGPPGPAGQDGRPGPPGPPGARGQAGVMGFPGPKGAAGEPGKAGERGVPGPPGAVGPAGKDGEAGAQGPPGPAGPAGERGEQGPAGSPGFQGLPGPAGPPGEAGKPGEQGVPGDLGAPGPSGARGERGFPGERGVQGPPGPAGPRGANGAPGNDGAKGDAGAPGAPGSQGAPGLQGMPGERGAAGLPGPKGDRGDAGPKGADGSPGKDGVRGLTGPIGPPGPAGAPGDKGESGPSGPAGPTGARGAPGDRGEPGPPGPAGFAGPPGADGQPGAKGEPGDAGAKGDAGPPGPAGPAGPPGPIGNVGAPGAKGARGSAGPPGATGFPGAAGRVGPPGPSGNAGPPGPPGPAGKEGGKGPRGETGPAGRPGEVGPPGPPGPAGEKGSPGADGPAGAPGTPGPQGIAGQRGVVGLPGQRGERGFPGLPGPSGEPGKQGPSGASGERGPPGPMGPPGLAGPPGESGREGAPGAEGSPGRDGSPGAKGDRGETGPAGPPGAPGAPGAPGPVGPAGKSGDRGETGPAGPAGPVGPVGARGPAGPQGPRGDKGETGEQGDRGIKGHRGFSGLQGPPGPPGSPGEQGPSGASGPAGPRGPPGSAGAPGKDGLNGLPGPIGPPGPRGRTGDAGPVGPPGPPGPPGPPGPPSAGFDFSFLPQPPQEKAHDGGRYYRADDANVVRDRDLEVDTTLKSLSQQIENIRSPEGSRKNPARTCRDLKMCHSDWKSGEYWIDPNQGCNLDAIKVFCNMETGETCVYPTQPSVAQKNWYISKNPKDKRHVWFGESMTDGFQFEYGGQGSDPADVAIQLTFLRLMSTEASQNITYHCKNSVAYMDQQTGNLKKALLLQGSNEIEIRAEGNSRFTYSVTVDGCTSHTGAWGKTVIEYKTTKTSRLPIIDVAPLDVGAPDQEFGFDVGPVCFL&#xA;&amp;gt;chain C CO1A2_HUMAN/1-1366 Collagen alpha-2(I) chain&#xA;----------------------------------------------------------------------------------------MLSFVDTRTLLLLAVTLCLATCQSLQEETVRKGPAGDRGPRGERGPPGPPGRDGEDGPTGPPGPPGPPGPPGLGGNFAAQYDGKGVGLGPGPMGLMGPRGPPGAAGAPGPQGFQGPAGEPGEPGQTGPAGARGPAGPPGKAGEDGHPGKPGRPGERGVVGPQGARGFPGTPGLPGFKGIRGHNGLDGLKGQPGAPGVKGEPGAPGENGTPGQTGARGLPGERGRVGAPGPAGARGSDGSVGPVGPAGPIGSAGPPGFPGAPGPKGEIGAVGNAGPAGPAGPRGEVGLPGLSGPVGPPGNPGANGLTGAKGAAGLPGVAGAPGLPGPRGIPGPVGAAGATGARGLVGEPGPAGSKGESGNKGEPGSAGPQGPPGPSGEEGKRGPNGEAGSAGPPGPPGLRGSPGSRGLPGADGRAGVMGPPGSRGASGPAGVRGPNGDAGRPGEPGLMGPRGLPGSPGNIGPAGKEGPVGLPGIDGRPGPIGPAGARGEPGNIGFPGPKGPTGDPGKNGDKGHAGLAGARGAPGPDGNNGAQGPPGPQGVQGGKGEQGPPGPPGFQGLPGPSGPAGEVGKPGERGLHGEFGLPGPAGPRGERGPPGESGAAGPTGPIGSRGPSGPPGPDGNKGEPGVVGAVGTAGPSGPSGLPGERGAAGIPGGKGEKGEPGLRGEIGNPGRDGARGAPGAVGAPGPAGATGDRGEAGAAGPAGPAGPRGSPGERGEVGPAGPNGFAGPAGAAGQPGAKGERGAKGPKGENGVVGPTGPVGAAGPAGPNGPPGPAGSRGDGGPPGMTGFPGAAGRTGPPGPSGISGPPGPPGPAGKEGLRGPRGDQGPVGRTGEVGAVGPPGFAGEKGPSGEAGTAGPPGTPGPQGLLGAPGILGLPGSRGERGLPGVAGAVGEPGPLGIAGPPGARGPPGAVGSPGVNGAPGEAGRDGNPGNDGPPGRDGQPGHKGERGYPGNIGPVGAAGAPGPHGPVGPAGKHGNRGETGPSGPVGPAGAVGPRGPSGPQGIRGDKGEPGEKGPRGLPGLKGHNGLQGLPGIAGHHGDQGAPGSVGPAGPRGPAGPSGPAGKDGRTGHPGTVGPAGIRGPQGHQGPAGPPGPPGPPGPPGVSGGGYDFGYDGDFYRADQPRSAPSLRPKDYEVDATLKSLNNQIETLLTPEGSRKNPARTCRDLRLSHPEWSSGYYWIDPNQGCTMDAIKVYCDFSTGETCIRAQPENIPAKNWYRSSKDKKHVWLGETINAGSQFEYNVEGVTSKEMATQLAFMRLLANYASQNITYHCKNSIAYMDEETGNLKKAVILQGSNDVELVAEGNSRFTYTVLVDGCSKKTNEWGKTIIEYKTNKPSRLPFLDIAPLDIGGADQEFFVDIGPVCFK&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;chain A CO4A1_HUMAN/1-1669 Collagen alpha-1(IV) chain&#xA;---------MGPRLSVWLLLLPAALLLHEEHSRAAA--KGGCAGSGC-GKCDCHGVKGQKGERGLPGLQGVIGFPGMQGPEGPQGPPGQKGDTGEPGLPGTKGTRGPPGASGYPGNPGLPGIPGQDGPPGPPGIPGCNGTKGERGPLGPPGLPGFAGNPGPPGLPGMKGDPGEILGHVPGMLLKGERGFPGIPGTPGPPGLPGLQGPVGPPGFTGPPGPPGPPGPPGEKGQMGLSFQGPKGDKGDQGVSGPPGVPGQA-------QVQEKGDFATKGEKGQKGEPGFQGMPGVGEKGEPGKPGPRGKPGKDGDKGEKGSPGFPGEPGYPGLIGRQGPQGEKGEAGPPGPPGIVIGTGPLGEKGERGYPGTPGPRGEPGPKGFPGLPGQPGPPGLPVPGQAGAPGFPGERGEKGDRGFPGTS-LP-GPSGRDGLPGPPGSPGPPGQPGYTNGIVECQPGPPGDQGPPGIPGQPGFIGEIGEKGQKGESCLICDIDGYRGPPGPQGPPGEIGFPGQPGAKGDRGLPGRDGVAGVPGPQGTPGLIGQPGAKGEPGEFYFDLRLKGDKGDPGFPGQPGMPGRAGSPGRDGHPGLPGPKGSPGSVGLKGERGPPGGVGFPGSRGDTGPPGPPGY---GPAGPIGDKGQAGFPGGPGSPGLPGPKGEPGKIVP--------------------LPGPPGAEGLPGSPGFPGPQGDRGFPGTPGRPGLPGEKGAVGQPGI-GFPGPPGPKGVDGLPGDMGPPGTPGRPGFNGLPGNPGVQGQKGEP---GVGLPGLKGLPGLPGIPGTPGEKGSIGVPGVPGEHGAIGPPGLQGIRGEPGPPGLPGSVGSPGVPGI-GPPGARGPPGGQGPPGLSGPPGIKGEKGFPGFPGLD-MPGPKGDKGAQGLPGITGQSGLPGLPGQQGAPGIPGFPGSKGEMGVMGTPGQPGSPGPVGAPGLPGEKGDHGFPGSSGPRGDPGLKGDKGDVGLPGKPGSMDKVDMGSMKGQKGDQGEKGQIGPIGEKGSRGDPGTPGVPGKDGQAGQPGQP-GPKGDPGISGTPGAPGLPGPKGSVGGMGLPGTPGEKGVPGIPGPQGSPGLPGDKGAKGEKGQAGPPGIGIPGLRGEKGDQGIAGFPGSPGEKGEKGSIGIPGMPGSPGLKGSPGSVGYPGSPGLPGEKGDKGLPGLDGIPGVKGEAGLPGTPGPTGPAGQKGEPGSDGIPGSAGEKGEPGLPGRGFPGFPGAKGDKGSKGEVGFP-GLAGSPGIPGSKGEQGFMGPPGPQGQPGLPGSPGHA-TEGPKGDRGPQGQPGLPGLPGPMGPPGLPGIDGVKGDKGNPGWPGAPGVPGPKGDPGFQGMPGIGGSPGITGSKGDMGPPGVPGFQGPKGLPGLQGIKGDQGDQGVPGAKGLPGPPGPPGPYDIIKGEPGLPGPEGPPGLKGLQGLPGPKGQQGVTGLVGIPGPPGIPGFDGAPGQKGEMGPAGPTGPRGFPGPPGPDGLPGSMGPPGTPSVDHGFLVTRHSQTIDDPQCPSGTKILYHGYSLLYVQGNERAHGQDLGTAGSCLRKFSTMPFLFCNINNVCNFASRNDYSYWLSTPEPMPMSMAPITGENIRPFISRCAVCEAPAMVMAVHSQTIQIPPCPSGWSSLWIGYSFVMHTSAGAEGSGQALASPGSCLEEFRSAPFIECHG-RGTCNYYANAYSFWLATIERSEMFKKPTPSTLKAGELRTHVSRCQVCMRRT&#xA;&amp;gt;chain B CO4A1_HUMAN/1-1669 Collagen alpha-1(IV) chain&#xA;---------MGPRLSVWLLLLPAALLLHEEHSRAAA--KGGCAGSGC-GKCDCHGVKGQKGERGLPGLQGVIGFPGMQGPEGPQGPPGQKGDTGEPGLPGTKGTRGPPGASGYPGNPGLPGIPGQDGPPGPPGIPGCNGTKGERGPLGPPGLPGFAGNPGPPGLPGMKGDPGEILGHVPGMLLKGERGFPGIPGTPGPPGLPGLQGPVGPPGFTGPPGPPGPPGPPGEKGQMGLSFQGPKGDKGDQGVSGPPGVPGQA-------QVQEKGDFATKGEKGQKGEPGFQGMPGVGEKGEPGKPGPRGKPGKDGDKGEKGSPGFPGEPGYPGLIGRQGPQGEKGEAGPPGPPGIVIGTGPLGEKGERGYPGTPGPRGEPGPKGFPGLPGQPGPPGLPVPGQAGAPGFPGERGEKGDRGFPGTS-LP-GPSGRDGLPGPPGSPGPPGQPGYTNGIVECQPGPPGDQGPPGIPGQPGFIGEIGEKGQKGESCLICDIDGYRGPPGPQGPPGEIGFPGQPGAKGDRGLPGRDGVAGVPGPQGTPGLIGQPGAKGEPGEFYFDLRLKGDKGDPGFPGQPGMPGRAGSPGRDGHPGLPGPKGSPGSVGLKGERGPPGGVGFPGSRGDTGPPGPPGY---GPAGPIGDKGQAGFPGGPGSPGLPGPKGEPGKIVP--------------------LPGPPGAEGLPGSPGFPGPQGDRGFPGTPGRPGLPGEKGAVGQPGI-GFPGPPGPKGVDGLPGDMGPPGTPGRPGFNGLPGNPGVQGQKGEP---GVGLPGLKGLPGLPGIPGTPGEKGSIGVPGVPGEHGAIGPPGLQGIRGEPGPPGLPGSVGSPGVPGI-GPPGARGPPGGQGPPGLSGPPGIKGEKGFPGFPGLD-MPGPKGDKGAQGLPGITGQSGLPGLPGQQGAPGIPGFPGSKGEMGVMGTPGQPGSPGPVGAPGLPGEKGDHGFPGSSGPRGDPGLKGDKGDVGLPGKPGSMDKVDMGSMKGQKGDQGEKGQIGPIGEKGSRGDPGTPGVPGKDGQAGQPGQP-GPKGDPGISGTPGAPGLPGPKGSVGGMGLPGTPGEKGVPGIPGPQGSPGLPGDKGAKGEKGQAGPPGIGIPGLRGEKGDQGIAGFPGSPGEKGEKGSIGIPGMPGSPGLKGSPGSVGYPGSPGLPGEKGDKGLPGLDGIPGVKGEAGLPGTPGPTGPAGQKGEPGSDGIPGSAGEKGEPGLPGRGFPGFPGAKGDKGSKGEVGFP-GLAGSPGIPGSKGEQGFMGPPGPQGQPGLPGSPGHA-TEGPKGDRGPQGQPGLPGLPGPMGPPGLPGIDGVKGDKGNPGWPGAPGVPGPKGDPGFQGMPGIGGSPGITGSKGDMGPPGVPGFQGPKGLPGLQGIKGDQGDQGVPGAKGLPGPPGPPGPYDIIKGEPGLPGPEGPPGLKGLQGLPGPKGQQGVTGLVGIPGPPGIPGFDGAPGQKGEMGPAGPTGPRGFPGPPGPDGLPGSMGPPGTPSVDHGFLVTRHSQTIDDPQCPSGTKILYHGYSLLYVQGNERAHGQDLGTAGSCLRKFSTMPFLFCNINNVCNFASRNDYSYWLSTPEPMPMSMAPITGENIRPFISRCAVCEAPAMVMAVHSQTIQIPPCPSGWSSLWIGYSFVMHTSAGAEGSGQALASPGSCLEEFRSAPFIECHG-RGTCNYYANAYSFWLATIERSEMFKKPTPSTLKAGELRTHVSRCQVCMRRT&#xA;&amp;gt;chain C CO4A2_HUMAN/1-1712 Collagen alpha-2(IV) chain&#xA;MGRDQRAVAGPALRRWLLLGTVTVGFLAQSVLAGVKKFDVPCGGRDCSGGCQCYPEKGGRGQPGPVGPQGYNGPPGLQGFPGLQGRKGDKGERGAPGVTGPKGDVGARGVSGFPGADGIPGHPGQGGPRGRPGYDGCNGTQGDSGPQGPPGSEGFTGPPGPQGPKGQKGEP-YALPKEERDRYRGEPGEPGLVGFQGPPGRPGHVGQMGPVGAPGRPGPPGPPGPKGQQGNRGLGFYGVKGEKGDVGQPGPNGIPSDTLHPIIAPTGVTFHPDQYKGEKGSEGEPGIRGISLKGEEGIMGFPGLRGYPGLSGEKGSPGQKGSRGLDGYQGPDGPRGPKGEAGDPGPPGLP--AYSPHPSLAKGARGDPGFPGAQGEPGSQGEPGDPGLPGPPGLSIGDGDQRRGLPGEMGPKGFIGDPGIPALYGGPPGPDGKRGPPGPPGLPGPPGPDGFL-FGLKGAKGRAGFPGLPGSPGARGPKGWKGDAGECRCTEGDEAIKGLPGLPGPKGFAGINGEPGRKGDRGDPGQHGLPGFPGLKGVPGNIGAPGPKGAKGDS-RTITTKGERGQPGVPGVPGMKGDDGSPGRDGLDGFPGLPGPPGD-GIKGPPGDPGYPGIPGTKGTPGEMGPPGLGLPGLKGQRGFPGDAGLPGPPGFLGPPGPAGTPGQIDCDTDVKRAVGGDRQEAIQPGCIGGPKGLPGLPGPPGPTGAKGLRGIPGFAGADGGPGPRGLPGDAGREGFPGPPGFIGPRGSKGAVGLPGPDGSPGPIGLPGPDGPPGERGLPGEVLGAQPGPRGDAGVPGQPGLKGLPGDRGPPGFRGSQGMPGMPGLKGQPGLPGPSGQPGLYGPPGLHGFPGAPGQEGPLGLPGIPGREGLPGDRGDPGDTGAPGPVGMKGLSGDRGDAGFTGEQGHPGSPGFKGIDGMPGTPGLKGDRGSPGMDGFQGMPGLKGRPGFPGSKGEAGFFGIPGLKGLAGEPGFKGSRGDPGPPGPP-PVILPGMKDIKGEKGDEGPMGLKGYLGAKGIQGMPGIPGLSGIPGLPGRPGHIKGVKGDIGVPGIPGLPGFPGVAGPPGITGFPGFIGSRGDKGAPGRAGLYGEIGATGDFGDIGDT-INLPGRPGLKGERGTTGIPGLKGFFGEKGTEGDIGFPGITGVTGVQGPPGLKGQTGFPGLTGPPGSQGELGRIGLPGGKGDDGWPGAPGLPGFPGLRGIRGLHGLPGTKGFPGSPGSDIHGDPGFPGPPGERGDPGEANTLPGPVGVPGQKGDQGAPGERGPPGSPGLQGFPGITPPSNISGAPGDKGAPGIFGLKGYRGPPGPPGSAALPGSKGDTGNPGAPGTPGTKGWAGDSGPQGRPGVFGLPGEKGPRGEQGFMGNTGPTGAVGDRGPKGPKGDPGFPGAPGTVGAPGIAGIPQKIAVQPGTVGPQGRRGPPGAPGEMGPQGPPGEPGFRGAPGKAGPQGRGGVSAVPGFRGDEGPIGHQGPIGQEGAPGRPGSPGLPGMPGR-SVSIGYLLVKHSQTDQEPMCPVGMNKLWSGYSLLYFEGQEKAHNQDLGLAGSCLARFSTMPFLYCNPGDVCYYASRNDKSYWLSTTAPLP--MMPVAEDEIKPYISRCSVCEAPAIAIAVHSQDVSIPHCPAGWRSLWIGYSFLMHTAAGDEGGGQSLVSPGSCLEDFRATPFIECNGGRGTCHYYANKYSFWLTTIPEQSFQGSPSADTLKAGLIRTHISRCQVCMKNL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15625" LastEditorUserId="298" LastEditDate="2022-10-26T11:55:19.303" LastActivityDate="2022-12-06T15:22:06.030" Title="heatmap of specific sequence motifs in aligned fasta files" Tags="|r|sequence-alignment|heatmap|motifs|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19910" PostTypeId="2" ParentId="19906" CreationDate="2022-10-26T10:41:13.493" Score="2" Body="&lt;p&gt;Looks like a poor annotation to me. I could understand a few scattered ncRNAs being arranged in the opposite direction from the gene, but all genes being oriented identically doesn't make sense.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-10-26T10:41:13.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19911" PostTypeId="2" ParentId="19904" CreationDate="2022-10-26T10:46:34.410" Score="1" Body="&lt;p&gt;The error about old index files is insightful to me. Bedtools presumably assumes that the input files are static / unchanging. If bedtools is automatically indexing the files before extracting sequences (when index files don't already exist), and the input files keep changing, that would be a good explanation for why problems are happening.&lt;/p&gt;&#xA;&lt;p&gt;Do these input fasta files have bedtools and/or samtools index files in the same directory (usually &lt;code&gt;fileName.fa.fai&lt;/code&gt;). Are you re-indexing the fasta files after changing them (e.g. via &lt;code&gt;samtools faidx fileName.fa&lt;/code&gt;)?&lt;/p&gt;&#xA;&lt;p&gt;I don't mean changing &lt;em&gt;during&lt;/em&gt; the running of bedtools, I mean changing between different bedtools runs. If you create a file &lt;code&gt;input_sequences.fa&lt;/code&gt;, then the first time it is subset, it will create an index file &lt;code&gt;input_sequences.fa.fai&lt;/code&gt;. The next time bedtools is run to subset an &lt;code&gt;input_sequences.fa&lt;/code&gt; within the same directory, it will check to see if an index file already exists; if one exists, it won't be regenerated. If the file &lt;code&gt;input_sequences.fa&lt;/code&gt; has changed in the interim, then the index file will be incorrect, and the results of subsetting will be unpredictable.&lt;/p&gt;&#xA;&lt;p&gt;This is why Bedtools reports the error: it can cause strange things to happen with output (intermittent segmentation faults, for example).&lt;/p&gt;&#xA;&lt;p&gt;In fact, bedtools is explicit about this index generation in its output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ ls&#xA;sampleTagsNew.bed  sampleTagsNew.fa&#xA;&#xA;$ bedtools getfasta -fi sampleTagsNew.fa -bed sampleTagsNew.bed &#xA;index file sampleTagsNew.fa.fai not found, generating...&#xA;&amp;gt;NewSampleTagX01:0-20&#xA;GTTGTCAAGATGCTACCGTT&#xA;&amp;gt;NewSampleTagX02:1-20&#xA;TTGTCAAGATGCTACCGTT&#xA;&#xA;$ ls&#xA;sampleTagsNew.bed  sampleTagsNew.fa  sampleTagsNew.fa.fai&#xA;&#xA;$ perl -i -pe 's/NewSampleTagX01/NewSampleTagX01_updated/' sampleTagsNew.fa&#xA;&#xA;$ bedtools getfasta -fi sampleTagsNew.fa -bed sampleTagsNew.bed &#xA;Warning: the index file is older than the FASTA file.&#xA;&amp;gt;NewSampleTagX01:0-20&#xA;updatedGTTGTCAAGATG&#xA;&amp;gt;NewSampleTagX02:1-20&#xA;TagX02GTTGTCAAGATG&#xA;&#xA;$ rm sampleTagsNew.fa.fai&#xA;&#xA;$ bedtools getfasta -fi sampleTagsNew.fa -bed sampleTagsNew.bed &#xA;index file sampleTagsNew.fa.fai not found, generating...&#xA;WARNING. chromosome (NewSampleTagX01) was not found in the FASTA file. Skipping.&#xA;&amp;gt;NewSampleTagX02:1-20&#xA;TTGTCAAGATGCTACCGTT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In the example above, I carry out the following operations:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;List the files in the directory&lt;/li&gt;&#xA;&lt;li&gt;Run &lt;code&gt;bedtools getfasta&lt;/code&gt; to subset a fasta file. Bedtools reports that an index file did not exist, so it is generating a new one.&lt;/li&gt;&#xA;&lt;li&gt;List the files in the directory. Note that there is a new file: &lt;code&gt;sampleTagsNew.fa.fai&lt;/code&gt;, which is the fasta file index.&lt;/li&gt;&#xA;&lt;li&gt;Modify the sample tags fasta file [to change the name of a sequence]&lt;/li&gt;&#xA;&lt;li&gt;Run &lt;code&gt;bedtools getfasta&lt;/code&gt; again with an identical command line. It produces the same warning you have seen, and produces unexpected output, i.e. incorrect results.&lt;/li&gt;&#xA;&lt;li&gt;Remove the fasta index file&lt;/li&gt;&#xA;&lt;li&gt;Run &lt;code&gt;bedtools getfasta&lt;/code&gt; again with an identical command line. This time the output is correct, stating that the first sequence cannot be found in the input fasta file.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-10-27T08:52:19.323" LastActivityDate="2022-10-27T08:52:19.323" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19913" PostTypeId="2" ParentId="19901" CreationDate="2022-10-26T16:22:32.973" Score="0" Body="&lt;p&gt;I would strongly recommend to find an outgroup or at least a fourth sequence. Building a phylogenetic tree with 3 sequences is doable but is tricky and error prone.&lt;/p&gt;&#xA;&lt;p&gt;If you still want to do a 3-sequence analysis, you can align the two homologous proteins to the reference genome with exonerate, spaln or miniprot, extract the reference subsequence in the alignment, do a proper multiple-sequence alignment (MSA) with muscle or MAFFT, build a tree and put the root at the middle point of the longest branch (this assumes a &lt;a href=&quot;https://en.wikipedia.org/wiki/Molecular_clock&quot; rel=&quot;nofollow noreferrer&quot;&gt;molecular clock&lt;/a&gt;). After that, you can do typical phylogenetic analysis.&lt;/p&gt;&#xA;&lt;p&gt;Whether the above works depends on the divergence of your sequences. If they are closely related and one sequence is a clear outgroup to the other two, the procedure could give you something, though having more sequences is still recommended.&lt;/p&gt;&#xA;&lt;p&gt;EDIT in response to the comments: midpoint rooting assumes an approximate molecular clock across the whole tree. Severe violation of the assumption may lead to a wrong root. This approach is not very reliable due to the clock assumption.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2022-10-27T03:57:24.797" LastActivityDate="2022-10-27T03:57:24.797" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19914" PostTypeId="1" CreationDate="2022-10-26T21:02:34.380" Score="1" ViewCount="12" Body="&lt;p&gt;I am trying to generate &lt;code&gt;seq.dat&lt;/code&gt; and &lt;code&gt;seq.txt&lt;/code&gt; files myself, so that I can use them with &lt;a href=&quot;https://zhanggroup.org/REMO/&quot; rel=&quot;nofollow noreferrer&quot;&gt;REMO&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I am using &lt;code&gt;mkseq.pl&lt;/code&gt; script in &lt;a href=&quot;https://zhanggroup.org/REMO/REMO.v2.tar.bz2&quot; rel=&quot;nofollow noreferrer&quot;&gt;REMOv2&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The following was my command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;user_name@server_name:~/spatial/reconstruction_tools/remo_v2&lt;span class=&quot;math-container&quot;&gt;$ perl mkseq.pl /home/spatial/input_pdb/pdb1a6j.pdb 1&#xA;PSIPRED finished&#xA;user_name@server_name:~/spatial/reconstruction_tools/remo_v2$&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This command generated four files:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;seq.dat&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;protein.fasta&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;psitmp27331007f0101.blast&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;psitmp27331007f0101.fasta&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;code&gt;seq.dat&lt;/code&gt; is totally empty. &lt;code&gt;psitmp27331007f0101.blast&lt;/code&gt; is also practically empty. There is no &lt;code&gt;seq.txt.&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;What am I doing incorrectly?&lt;/p&gt;&#xA;&#xA;" OwnerUserId="12686" LastEditorUserId="12686" LastEditDate="2022-10-26T21:55:39.843" LastActivityDate="2022-10-26T21:55:39.843" Title="How can I use REMO tool to generate seq.dat and seq.txt files?" Tags="|software-usage|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19915" PostTypeId="7" CreationDate="2022-10-27T11:05:42.280" Score="0" Body="&lt;ul&gt;&#xA;&lt;li&gt;Specific issues with bioinformatics&lt;/li&gt;&#xA;&lt;li&gt;Real problems or questions that you’ve encountered&lt;/li&gt;&#xA;&lt;li&gt;Error messages that you don't understand&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="73" LastEditDate="2022-10-27T11:06:13.733" LastActivityDate="2022-10-27T11:06:13.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19916" PostTypeId="7" CreationDate="2022-10-27T11:10:12.437" Score="0" Body="&lt;p&gt;&lt;b&gt;Bioinformatics Stack Exchange&lt;/b&gt; is a question and answer site for researchers, developers, students, teachers, and end users interested in bioinformatics. It's built and run &lt;i&gt;by you&lt;/i&gt; as part of the &lt;a href=&quot;https://stackexchange.com&quot;&gt;Stack Exchange&lt;/a&gt; network of Q&amp;amp;A sites. With your help, we're working together to build a library of detailed answers to common questions in the field of bioinformatics.&lt;/p&gt;&#xA;&lt;p&gt;In bioinformatics, context is important. We love people who talk about the projects behind their problems, because the best solutions are created from a good understanding of the &lt;em&gt;whole&lt;/em&gt; problem.&lt;/p&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="73" LastEditDate="2022-11-19T11:56:33.417" LastActivityDate="2022-11-19T11:56:33.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19917" PostTypeId="2" ParentId="4736" CreationDate="2022-10-27T14:31:24.587" Score="1" Body="&lt;p&gt;Like others have mentioned, these plots come from BAM, BED, and BigWig files.&#xA;Those files contain the data that you can give to tools like the &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgGateway&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC Genome Browser&lt;/a&gt;, &lt;a href=&quot;https://igv.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IGV&lt;/a&gt;,  &lt;a href=&quot;http://higlass.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HiGlass&lt;/a&gt;/&lt;a href=&quot;https://gosling.js.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gosling&lt;/a&gt;, or other bioinformatics visualization software.&lt;/p&gt;&#xA;&lt;p&gt;For the specific cell line you mention, you'll have to find the relevant RNA-seq and ChIP-seq data from some data repository, like CCLE, ENA, SRA, or GEO.&#xA;You may be to put in some work converting available files to one of these compatible formats before plotting them.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2022-10-27T14:31:24.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19919" PostTypeId="1" CreationDate="2022-10-27T19:23:39.803" Score="2" ViewCount="115" Body="&lt;p&gt;I am in the process of creating a bioinformatics pipeline and I am currently trying to setup the AWS infrastructure for which it will run.&lt;/p&gt;&#xA;&lt;p&gt;The script I am running to setup the AWS batch job looks like the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#! /bin/bash&#xA;&#xA;# Setup the AWS Batch architecture&#xA;# Copyright 2022, SolareaBio&#xA;&#xA;set -e&#xA;&#xA;&#xA;# ***** BEGIN custom lines&#xA;&#xA;container=&amp;quot;BGC_bacteria&amp;quot;&#xA;vCPUs_req=12&#xA;memory_MiB_req=40000&#xA;# ASK ABOUT CUSTOM CPU AND MEMORY SETTINGS&#xA;&#xA;# ***** END custom lines&#xA;&#xA;&#xA;&#xA;container_lc=&lt;span class=&quot;math-container&quot;&gt;$(echo $&lt;/span&gt;container | tr [A-Z] [a-z])&#xA;&#xA;subnet_useast1a=&amp;quot;subnet-42371d0a&amp;quot;&#xA;subnet_useast1b=&amp;quot;subnet-b03c54ea&amp;quot;&#xA;subnet_useast1c=&amp;quot;subnet-00ee1364&amp;quot;&#xA;&#xA;usage() {&#xA;  cat &amp;lt;&amp;lt; EOF&#xA;Usage $0 [-h]&#xA;&#xA;Builds the &lt;span class=&quot;math-container&quot;&gt;$container docker container, pushes it to the AWS ECR, and sets up the $&lt;/span&gt;container AWS Batch environment&#xA;EOF&#xA;}&#xA;&#xA;while getopts &amp;quot;h&amp;quot; option; do&#xA;  case $option in&#xA;    h)   usage;exit;;&#xA;    [?]) usage; exit;;&#xA;  esac&#xA;done&#xA;&#xA;script_dir=&lt;span class=&quot;math-container&quot;&gt;$(dirname &quot;$&lt;/span&gt;0&amp;quot;)&#xA;&#xA;repo=&amp;quot;990850896279.dkr.ecr.us-east-1.amazonaws.com&amp;quot;&#xA;&#xA;echo &amp;quot;Logging into ECR&amp;quot;&#xA;aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin ${repo}&#xA;&#xA;image_and_tag=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${container_lc}:latest&quot;&#xA;echo &quot;Building the docker container: $&lt;/span&gt;image_and_tag&amp;quot;&#xA;&#xA;# If you get errors from the docker build command, try adding --no-cache&#xA;docker build --no-cache -f &lt;span class=&quot;math-container&quot;&gt;$script_dir/$&lt;/span&gt;{container}.dockerfile -t &lt;span class=&quot;math-container&quot;&gt;$image_and_tag $&lt;/span&gt;script_dir&#xA;&#xA;echo &amp;quot;Tagging Image&amp;quot;&#xA;docker tag &lt;span class=&quot;math-container&quot;&gt;${image_and_tag} $&lt;/span&gt;{repo}/${image_and_tag}&#xA;&#xA;# Create repository in AWS ECR&#xA;repo_container_info=&lt;span class=&quot;math-container&quot;&gt;$(aws ecr describe-repositories --repository-names $&lt;/span&gt;{container_lc})&#xA;if [[ &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$repo_container_info&quot; == &quot;&quot; ]]; then&#xA;  aws ecr create-repository --repository-name $&lt;/span&gt;{container_lc}&#xA;fi&#xA;&#xA;echo &amp;quot;Pushing Image up to AWS&amp;quot;&#xA;docker push &lt;span class=&quot;math-container&quot;&gt;${repo}/$&lt;/span&gt;{image_and_tag}&#xA;&#xA;# Create compute environment&#xA;EC2_type=ON_DEMAND&#xA;ce_info=&lt;span class=&quot;math-container&quot;&gt;$(aws batch describe-compute-environments --compute-environments $&lt;/span&gt;{container}_&lt;span class=&quot;math-container&quot;&gt;${EC2_type}_CE)&#xA;ce_status=$&lt;/span&gt;(echo &lt;span class=&quot;math-container&quot;&gt;$ce_info | jq '.computeEnvironments[0] | .status // &quot;&quot;' | tr -d '&quot;')&#xA;if [[ $&lt;/span&gt;ce_status != &amp;quot;VALID&amp;quot; ]]; then&#xA;  compute_resources='{&amp;quot;type&amp;quot;:&amp;quot;EC2&amp;quot;,&amp;quot;allocationStrategy&amp;quot;:&amp;quot;BEST_FIT_PROGRESSIVE&amp;quot;,&amp;quot;minvCpus&amp;quot;:0,&amp;quot;maxvCpus&amp;quot;:10000,&amp;quot;tags&amp;quot;:{&amp;quot;Name&amp;quot;:&amp;quot;'&lt;span class=&quot;math-container&quot;&gt;${container}_Batch'&quot;},&quot;instanceTypes&quot;:[&quot;c5a&quot;,&quot;m5a&quot;,&quot;c5&quot;,&quot;m5&quot;],&quot;subnets&quot;:[&quot;'$&lt;/span&gt;subnet_useast1a'&amp;quot;,&amp;quot;'&lt;span class=&quot;math-container&quot;&gt;$subnet_useast1b'&quot;,&quot;'$&lt;/span&gt;subnet_useast1c'&amp;quot;],&amp;quot;securityGroupIds&amp;quot;:[&amp;quot;sg-ff2fd48f&amp;quot;,&amp;quot;sg-65ff5315&amp;quot;],&amp;quot;ec2KeyPair&amp;quot;:&amp;quot;SBTools&amp;quot;,&amp;quot;instanceRole&amp;quot;:&amp;quot;ecsInstanceRole&amp;quot;}'&#xA;  aws batch create-compute-environment \&#xA;    --compute-environment-name &lt;span class=&quot;math-container&quot;&gt;${container}_$&lt;/span&gt;{EC2_type}_CE \&#xA;    --type MANAGED \&#xA;    --state ENABLED \&#xA;    --compute-resources $compute_resources \&#xA;    --service-role AWSBatchServiceRole&#xA;&#xA;  printf &amp;quot;Waiting for Compute Environment to be created&amp;quot;&#xA;  while [[ &lt;span class=&quot;math-container&quot;&gt;$ce_status != &quot;VALID&quot; ]]; do&#xA;sleep 5&#xA;printf &quot;.&quot;&#xA;ce_info=$&lt;/span&gt;(aws batch describe-compute-environments --compute-environments &lt;span class=&quot;math-container&quot;&gt;${container}_$&lt;/span&gt;{EC2_type}_CE)&#xA;    ce_status=&lt;span class=&quot;math-container&quot;&gt;$(echo $&lt;/span&gt;ce_info | jq '.computeEnvironments[0] | .status // &amp;quot;&amp;quot;' | tr -d '&amp;quot;')&#xA;  done&#xA;  echo &amp;quot;&amp;quot;&#xA;fi&#xA;&#xA;# Create the job queue&#xA;jq_info=&lt;span class=&quot;math-container&quot;&gt;$(aws batch describe-job-queues --job-queues $&lt;/span&gt;{container}_&lt;span class=&quot;math-container&quot;&gt;${EC2_type}_JQ)&#xA;jq_status=$&lt;/span&gt;(echo &lt;span class=&quot;math-container&quot;&gt;$jq_info | jq '.jobQueues[0] | .status // &quot;&quot;' | tr -d '&quot;')&#xA;if [[ $&lt;/span&gt;jq_status != &amp;quot;VALID&amp;quot; ]]; then&#xA;  aws batch create-job-queue \&#xA;    --job-queue-name &lt;span class=&quot;math-container&quot;&gt;${container}_$&lt;/span&gt;{EC2_type}_JQ \&#xA;    --state ENABLED \&#xA;    --priority 5 \&#xA;    --compute-environment-order '{&amp;quot;order&amp;quot;:1,&amp;quot;computeEnvironment&amp;quot;:&amp;quot;'&lt;span class=&quot;math-container&quot;&gt;${container}_$&lt;/span&gt;{EC2_type}_CE'&amp;quot;}'&#xA;fi&#xA;&#xA;# Register job definition&#xA;job_definition=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${container}_JD&quot;&#xA;jd_info=$&lt;/span&gt;(aws batch describe-job-definitions --status ACTIVE --job-definition-name &lt;span class=&quot;math-container&quot;&gt;$job_definition)&#xA;jd_status=$&lt;/span&gt;(echo &lt;span class=&quot;math-container&quot;&gt;$jd_info | jq '.jobDefinitions[0] | .status // &quot;&quot;' | tr -d '&quot;')&#xA;if [[ $&lt;/span&gt;jd_status != &amp;quot;ACTIVE&amp;quot; ]]; then&#xA;  aws batch register-job-definition \&#xA;    --job-definition-name &lt;span class=&quot;math-container&quot;&gt;$job_definition \&#xA;    --type container \&#xA;    --container-properties '{&quot;privileged&quot;:true,&quot;image&quot;:&quot;'$repo/$image_and_tag'&quot;,&quot;vcpus&quot;:'${vCPUs_req}',&quot;memory&quot;:'$&lt;/span&gt;{memory_MiB_req}',&amp;quot;mountPoints&amp;quot;:[{&amp;quot;sourceVolume&amp;quot;:&amp;quot;SB-EFS&amp;quot;,&amp;quot;containerPath&amp;quot;:&amp;quot;/EFS&amp;quot;}],&amp;quot;volumes&amp;quot;:[{&amp;quot;host&amp;quot;:{&amp;quot;sourcePath&amp;quot;:&amp;quot;/&amp;quot;},&amp;quot;name&amp;quot;:&amp;quot;SB-EFS&amp;quot;}],&amp;quot;command&amp;quot;:[&amp;quot;Ref::container&amp;quot;,&amp;quot;Ref::run_id&amp;quot;,&amp;quot;Ref::input_file&amp;quot;]}' \&#xA;    --retry-strategy '{&amp;quot;attempts&amp;quot;: 10,&amp;quot;evaluateOnExit&amp;quot;:[{&amp;quot;onStatusReason&amp;quot; :&amp;quot;Host EC2*&amp;quot;,&amp;quot;action&amp;quot;: &amp;quot;RETRY&amp;quot;},{&amp;quot;onReason&amp;quot; : &amp;quot;*&amp;quot;,&amp;quot;action&amp;quot;: &amp;quot;EXIT&amp;quot;}]}'&#xA;fi&#xA;&#xA;# Make sure the EFS has an access point set up&#xA;efs_id=&lt;span class=&quot;math-container&quot;&gt;$(aws efs describe-file-systems | jq -r '.FileSystems[] | select (.Name==&quot;SB-EFS&quot;) | .FileSystemId')&#xA;if [[ &quot;$&lt;/span&gt;efs_id&amp;quot; == &amp;quot;&amp;quot; ]]; then&#xA;  echo &amp;quot;ERROR: SB-EFS does not exist&amp;quot;&#xA;  exit&#xA;fi&#xA;ap_id=&lt;span class=&quot;math-container&quot;&gt;$(aws efs describe-access-points | jq -r '.AccessPoints[] | select (.Name==&quot;EFS_AP&quot;) | .AccessPointId')&#xA;if [[ &quot;$&lt;/span&gt;ap_id&amp;quot; == &amp;quot;&amp;quot; ]]; then&#xA;  # 1001:1001 is user sb in group sb&#xA;  ap_id=&lt;span class=&quot;math-container&quot;&gt;$(aws efs create-access-point --file-system-id &quot;$&lt;/span&gt;efs_id&amp;quot; --posix-user '{&amp;quot;Uid&amp;quot;:1001,&amp;quot;Gid&amp;quot;:1001}' --root-directory '{&amp;quot;Path&amp;quot;:&amp;quot;/&amp;quot;}' --tags '[{&amp;quot;Key&amp;quot;:&amp;quot;Name&amp;quot;,&amp;quot;Value&amp;quot;:&amp;quot;EFS_AP&amp;quot;}]' | jq -r '.AccessPointId')&#xA;fi  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now when I run it, everything works as it should until getting to the --retry-strategy command line in the &amp;quot;register job definition&amp;quot; section of the script. The following error is subsequently thrown:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Parameter validation failed:&#xA;Unknown parameter in retryStrategy: &amp;quot;evaluateOnExit&amp;quot;, must be one of: attempts&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Not sure what the issue is with the &amp;quot;attempts&amp;quot; parameter that is set. Any potential solutions or guidance to solving this error is much appreciated. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="16173" LastEditorUserId="650" LastEditDate="2022-10-28T15:39:53.450" LastActivityDate="2024-03-24T05:05:45.667" Title="Error Registering Job Definition AWS" Tags="|bash|shell|aws|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19920" PostTypeId="1" CreationDate="2022-10-29T06:38:33.740" Score="1" ViewCount="174" Body="&lt;p&gt;I want to use &lt;a href=&quot;https://github.com/ccsb-scripps/AutoDock-GPU&quot; rel=&quot;nofollow noreferrer&quot;&gt;AutoDock GPU&lt;/a&gt;, which accepts protein.maps.fld and ligand.pdbqt files.&lt;/p&gt;&#xA;&lt;p&gt;Since I am performing high throughput virtual screening, shall I just load protein AutoDockTools and create protein.maps.fld file, which I can use in my command line run of AutoDock GPU run?&lt;/p&gt;&#xA;&lt;p&gt;I have 1 million ligands in my screening library, so can't repeat the above procedure on all ligands, or shall I just pick a ligand that contains all types of atoms a ligand can have in that ligand library and produce protein.maps.fld based upon that ligand along with the given protein?&lt;/p&gt;&#xA;" OwnerUserId="16355" LastEditorUserId="3848" LastEditDate="2022-10-29T13:06:37.510" LastActivityDate="2022-10-29T13:06:37.510" Title="Creating maps.fld file of protein for AutoDock GPU" Tags="|public-databases|docking|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19921" PostTypeId="1" AcceptedAnswerId="19944" CreationDate="2022-10-29T14:58:12.537" Score="1" ViewCount="90" Body="&lt;p&gt;I am new to using both DESeq2 and EdgeR in Bioconductor used for transforming my RNA expression data.&lt;/p&gt;&#xA;&lt;p&gt;However, I am struggling to understand their specific purpose, differences between them and underlying theory. The information is required for a thesis with an imminent deadline&lt;/p&gt;&#xA;" OwnerUserId="16412" LastEditorUserId="73" LastEditDate="2022-10-30T10:59:38.113" LastActivityDate="2022-11-02T19:52:30.997" Title="DESeq2 and EdgeR" Tags="|deseq2|edger|homework|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19922" PostTypeId="1" AcceptedAnswerId="19935" CreationDate="2022-10-29T16:33:01.167" Score="0" ViewCount="140" Body="&lt;p&gt;I have used WGCNA by integrating several datasets which are processed in a similar way. Identified 17 modules, followed by performed pathway analysis with the genes present in those modules.&lt;/p&gt;&#xA;&lt;p&gt;Among all the protein-coding genes,ncRNAs and other genes, I'm particularly interested in a specific lncRNA &lt;code&gt;MALAT1&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I found that &lt;code&gt;MALAT1&lt;/code&gt; is co-expressed with more than 800 genes in a module named &lt;code&gt;blue&lt;/code&gt;. I have few questions, can anyone please help me in this?&lt;/p&gt;&#xA;&lt;p&gt;Questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I would like to check which are the closest co-expressed genes to &lt;code&gt;MALAT1&lt;/code&gt;. How can I know that?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Once I get the closest genes to &lt;code&gt;MALAT1&lt;/code&gt;, I would like to make a network with those genes and MALAT1. How to do this network?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="9114" LastActivityDate="2022-11-01T13:52:00.387" Title="How to find closely related genes for a specific gene from WGCNA modules?" Tags="|r|rna-seq|networks|wgcna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19923" PostTypeId="1" CreationDate="2022-10-29T19:05:52.757" Score="5" ViewCount="110" Body="&lt;p&gt;I'm trying to use snakemake to replace some make workflows, but I'm finding that snakemake does not use the file modification time to trigger a rule.&lt;/p&gt;&#xA;&lt;p&gt;Here is my snakefile&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule sample:&#xA;    input:&#xA;        &amp;quot;a&amp;quot;&#xA;    output:&#xA;        &amp;quot;b&amp;quot;&#xA;    shell:&#xA;        &amp;quot;touch {output}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If I create &lt;code&gt;a&lt;/code&gt; then run snakemake, &lt;code&gt;b&lt;/code&gt; is created as expected:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ touch a&#xA;$&lt;/span&gt; snakemake -c1 b&#xA;Building DAG of jobs...&#xA;Using shell: /usr/bin/bash&#xA;Provided cores: 1 (use --cores to define parallelism)&#xA;Rules claiming more threads will be scaled down.&#xA;Job stats:&#xA;job      count    min threads    max threads&#xA;-----  -------  -------------  -------------&#xA;sample       1              1              1&#xA;total        1              1              1&#xA;&#xA;Select jobs to execute...&#xA;&#xA;[Sat Oct 29 14:36:43 2022]&#xA;rule sample:&#xA;    input: a&#xA;    output: b&#xA;    jobid: 0&#xA;    reason: Missing output files: b&#xA;    resources: tmpdir=/tmp&#xA;&#xA;[Sat Oct 29 14:36:43 2022]&#xA;Finished job 0.&#xA;1 of 1 steps (100%) done&#xA;Complete log: .snakemake/log/2022-10-29T143643.334050.snakemake.log&#xA;$ ls -l a b&#xA;-rw-r--r-- 1 ... ... 0 Oct 29 14:36 a&#xA;-rw-r--r-- 1 ... ... 0 Oct 29 14:36 b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, if I touch &lt;code&gt;a&lt;/code&gt; then ask snakemake to update &lt;code&gt;b&lt;/code&gt;, nothing happens:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ touch a&#xA;$ snakemake -c1 b&#xA;Building DAG of jobs...&#xA;Nothing to be done (all requested files are present and up to date).&#xA;Complete log: .snakemake/log/2022-10-29T144005.589943.snakemake.log&#xA;$ ls -l a b&#xA;-rw-r--r-- 1 ... ... 0 Oct 29 14:40 a&#xA;-rw-r--r-- 1 ... ... 0 Oct 29 14:36 b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Modifying the contents of &lt;code&gt;a&lt;/code&gt; triggers an update as expected:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ echo new &amp;gt; a&#xA;$&lt;/span&gt; snakemake -c1 b&#xA;Building DAG of jobs...&#xA;Using shell: /usr/bin/bash&#xA;Provided cores: 1 (use --cores to define parallelism)&#xA;Rules claiming more threads will be scaled down.&#xA;Job stats:&#xA;job      count    min threads    max threads&#xA;-----  -------  -------------  -------------&#xA;sample       1              1              1&#xA;total        1              1              1&#xA;&#xA;Select jobs to execute...&#xA;&#xA;[Sat Oct 29 14:42:13 2022]&#xA;rule sample:&#xA;    input: a&#xA;    output: b&#xA;    jobid: 0&#xA;    reason: Updated input files: a&#xA;    resources: tmpdir=/tmp&#xA;&#xA;[Sat Oct 29 14:42:13 2022]&#xA;Finished job 0.&#xA;1 of 1 steps (100%) done&#xA;Complete log: .snakemake/log/2022-10-29T144213.161985.snakemake.log&#xA;$ ls -l a b&#xA;-rw-r--r-- 1 ... ... 4 Oct 29 14:42 a&#xA;-rw-r--r-- 1 ... ... 0 Oct 29 14:42 b&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I installed snakemake-minimal (version 7.17.1) from bioconda, am using python 3.9.13 from conda-forge, on a RHEL 7 system.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ snakemake --version&#xA;7.17.1&#xA;$ python --version&#xA;Python 3.9.13&#xA;$ cat /etc/redhat-release&#xA;Red Hat Enterprise Linux Server release 7.8 (Maipo)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm clearly misunderstanding something very basic about snakemake.  Any suggestions on how to debug this, descriptions of how to have snakemake pay attention to file modification times, or just an explanation of why snakemake behaves differently from make would be much appreciated!&lt;/p&gt;&#xA;&lt;p&gt;thanks in advance,&lt;/p&gt;&#xA;" OwnerUserId="16413" LastActivityDate="2022-10-31T04:17:13.090" Title="Modification time does not trigger a snakemake rule" Tags="|snakemake|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19924" PostTypeId="1" AcceptedAnswerId="19945" CreationDate="2022-10-30T14:46:31.113" Score="2" ViewCount="36" Body="&lt;p&gt;as the title says I´m trying to run &lt;a href=&quot;https://bionanogenomics.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bionanos&lt;/a&gt; HybridScaffolding from the command line rather than using their own software interface. This is commercial software to primarily to identify genomewide cancer analysis but also has options for genome assembly.&lt;/p&gt;&#xA;&lt;p&gt;I installed Perl and all the necessary packages to run the HybridScaffolding Perl script. According to their &lt;a href=&quot;https://bionanogenomics.com/wp-content/uploads/2018/04/30205-Guidelines-for-Running-Bionano-Solve-Pipeline-on-Command-Line.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; I started the script with the correct parameters.&#xA;After 2 minutes it stops giving me the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Beginning initial NGS CMAP to BioNano CMAP alignment...&#xA;Running command: /home/user/workflow/scripts/bionano/tools/pipeline/1.0/RefAligner/1.0/RefAligner &#xA;-ref /home/user/results/bionano/fa2cmap/pac_bio_assembly_CTTAAG_0kb_0labels.cmap &#xA;-i /home/user/data/bionano/bCalAnn1_Saphyr_DLE1.cmap.gz &#xA;-o align0 -stdout -stderr -maxmem 128 -maxthreads 64 &#xA;-maxvirtmem 0 -maxmemIncrease 8 -RAmem 3 1 0.9 -M 1 3 -ScaleDelta 0.03 2 -ScaleDeltaBPP &#xA;-hashScaleDelta 2 -res 2.6 -resSD 0.7 -FP 0.2 -FN 0.02 -sf 0.10 -sd 0.0 -sr 0.01 -se &#xA;0.2 -extend 1 -outlier 0.0001 -endoutlier 0.001 -PVendoutlier -deltaX 12 -deltaY 12 &#xA;-xmapchim 12 -hashgen 5 11 2.4 1.4 0.05 3.0 1 1 1 -hash -hashdelta 27 -hashoffset 1 &#xA;-hashrange 0 -hashGrouped 6 11 -hashMultiMatch 50 2 -hashGC 300 -hashT2 1 -hashkeys 1 &#xA;-HSDrange 1.0 -insertThreads 4 -nosplit 2 -biaswt 0 -T 1e-10 -S -1000 -indel -PVres 2 &#xA;-rres 0.9 -outlierBC -xmapUnique 12 -AlignRes 2. -outlierExtend 12 24 -resEstimate -f &#xA;-mres 0.9 -thetaScale 0.5 -KL 1.5 -KF 1.5 -Kmax 4 -MinFN 0.01 -MinFP 0.16 -MinSF 0.05 &#xA;-MaxSF 0.15 -MinSD 0.0 -MaxSD 0.0 -MinSR 0.007 -MaxSR 0.015 -MaxSE 0.5 -MinRes 2.0 &#xA;-MinResSD 0.7 -hashSD 0 1 -hashSR 0 1&#xA;&#xA;ERROR: Cannot open align0.stdout: No such file or directory&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For context: The HybridScaffolding script contains several scripts which are started automatically over time. One of these is the RefAligner script, which throws the error in this case.&lt;/p&gt;&#xA;&lt;p&gt;I have a folder called align0 which lies inside the directory that the script is running, therefore I guess this is not the casual &amp;quot;The path is missing&amp;quot; issue..&lt;/p&gt;&#xA;&lt;p&gt;Inside the Perl Script 'align0.stdout' appears two times:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# error check&#xA;if (errCheckRefAligner(&amp;quot;align0.stdout&amp;quot;, &amp;quot;END of output&amp;quot;, &amp;quot;Initial alignment complete in &lt;span class=&quot;math-container&quot;&gt;$span.&quot;, &quot;ERROR: Initial alignment cannot be completed.&quot;, 0) != 1)  {&#xA;    my $errMsg = &quot;ERROR: Initial alignment cannot be completed.&quot;;&#xA;    printStatus(\*STATUS_FILE, &quot;align0&quot;, $errMsg, 1);&#xA;} else  {&#xA;    printStatus(\*STATUS_FILE, &quot;align0&quot;, &quot;Done&quot;, 0);&#xA;} # RefAligner check&#xA;errCheckRefAligner(&quot;align0.stdout&quot;, &quot;END of output&quot;, &quot;Initial alignment complete in $&lt;/span&gt;span.&amp;quot;, &amp;quot;ERROR: Initial alignment cannot be completed.&amp;quot;, 1);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Sadly I don´t have any Perl knowledge at all. Does anyone have experience with Bionanos Software and/or knowledge of Perl and could explain to me what is happening inside this part of the script?&lt;/p&gt;&#xA;" OwnerUserId="16419" LastEditorUserId="3848" LastEditDate="2022-10-30T15:43:20.467" LastActivityDate="2022-11-03T10:00:48.150" Title="Running Bionano´s HybridScaffolding from the command line [Error: align0.stdout]" Tags="|perl|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19925" PostTypeId="1" CreationDate="2022-10-30T16:57:20.227" Score="2" ViewCount="36" Body="&lt;p&gt;For example I checked motifs in Transferrin P02787  &lt;a href=&quot;https://prosite.expasy.org/cgi-bin/prosite/ScanView.cgi?scanfile=816050634835.scan.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://prosite.expasy.org/cgi-bin/prosite/ScanView.cgi?scanfile=816050634835.scan.gz&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have&#xA;hits by profiles: [2 hits (by 1 profile) on 1 sequence]&#xA;and&#xA;hits by patterns: [5 hits (by 3 distinct patterns) on 1 sequence]&lt;/p&gt;&#xA;&lt;p&gt;Can I say that I detect &lt;strong&gt;4 different motifs&lt;/strong&gt;? One motif is searched by profile searching (PS51408   TRANSFERRIN_LIKE_4   Transferrin-like domain profile two times) and three motifs are searched by pattern searching (PS00205   TRANSFERRIN_LIKE_1   Transferrin-like domain signature 1, PS00206   TRANSFERRIN_LIKE_2   Transferrin-like domain signature 2 two times, PS00207   TRANSFERRIN_LIKE_3   Transferrin-like domain signature 3 two times).&lt;/p&gt;&#xA;&lt;p&gt;All the motifs found are equal to 7? Is it right? Can I talk about the different motifs found by the profile and pattern, or are they the same in some way?&lt;/p&gt;&#xA;" OwnerUserId="16327" LastActivityDate="2022-11-02T19:43:23.747" Title="How to count number of motifs in Prosite?" Tags="|motifs|prosite|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19926" PostTypeId="2" ParentId="19919" CreationDate="2022-10-31T01:09:41.197" Score="0" Body="&lt;p&gt;You didn't say what version of the AWS CLI you're using, but I suspect it's version 1. It looks like the &lt;code&gt;evaluateOnExit&lt;/code&gt; feature was added in version 2 but was not backported. Installing AWS CLI version 2 resolves the issue:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note that &lt;a href=&quot;https://www.nextflow.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nextflow&lt;/a&gt; also provides built-in support for &lt;a href=&quot;https://www.nextflow.io/docs/latest/aws.html#aws-batch&quot; rel=&quot;nofollow noreferrer&quot;&gt;AWS Batch&lt;/a&gt;. You'd still need to configure some AWS infrastructure using Nextflow, but it should be a lot less painful than writing/maintaining your own script(s). It's definitely worth checking out if you haven't already.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2022-10-31T01:09:41.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19927" PostTypeId="2" ParentId="19923" CreationDate="2022-10-31T04:17:13.090" Score="5" Body="&lt;p&gt;This is a Snakemake feature: For small files (less than 100000 bytes), checksums are compared to determine if a job needs to run if the input file is newer than output file:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/snakemake/snakemake/pull/1568&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/snakemake/snakemake/pull/1568&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;With a size of 99999 bytes:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ dd if=/dev/zero of=a bs=99999 count=1&#xA;1+0 records in&#xA;1+0 records out&#xA;99999 bytes (100 kB, 98 KiB) copied, 0.000502722 s, 199 MB/s&#xA;$&lt;/span&gt; snakemake -c1 b&#xA;Building DAG of jobs...&#xA;Using shell: /usr/bin/bash&#xA;Provided cores: 1 (use --cores to define parallelism)&#xA;Rules claiming more threads will be scaled down.&#xA;Job stats:&#xA;job       count    min threads    max threads&#xA;------  -------  -------------  -------------&#xA;sample        1              1              1&#xA;total         1              1              1&#xA;&#xA;Select jobs to execute...&#xA;&#xA;[Mon Oct 31 14:38:00 2022]&#xA;rule sample:&#xA;    input: a&#xA;    output: b&#xA;    jobid: 0&#xA;    reason: Missing output files: b&#xA;    resources: tmpdir=/tmp&#xA;&#xA;[Mon Oct 31 14:38:00 2022]&#xA;Finished job 0.&#xA;1 of 1 steps (100%) done&#xA;Complete log: .snakemake/log/2022-10-31T143800.486966.snakemake.log&#xA;&lt;span class=&quot;math-container&quot;&gt;$ dd if=/dev/zero of=a bs=99999 count=1&#xA;1+0 records in&#xA;1+0 records out&#xA;99999 bytes (100 kB, 98 KiB) copied, 0.000180857 s, 553 MB/s&#xA;$&lt;/span&gt; snakemake -c1 b&#xA;Building DAG of jobs...&#xA;Nothing to be done (all requested files are present and up to date).&#xA;Complete log: .snakemake/log/2022-10-31T143813.285988.snakemake.log&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With a size of 100000 bytes:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ dd if=/dev/zero of=a bs=100000 count=1&#xA;1+0 records in&#xA;1+0 records out&#xA;100000 bytes (100 kB, 98 KiB) copied, 0.000173008 s, 578 MB/s&#xA;$&lt;/span&gt; snakemake -c1 b&#xA;Building DAG of jobs...&#xA;Using shell: /usr/bin/bash&#xA;Provided cores: 1 (use --cores to define parallelism)&#xA;Rules claiming more threads will be scaled down.&#xA;Job stats:&#xA;job       count    min threads    max threads&#xA;------  -------  -------------  -------------&#xA;sample        1              1              1&#xA;total         1              1              1&#xA;&#xA;Select jobs to execute...&#xA;&#xA;[Mon Oct 31 14:41:14 2022]&#xA;rule sample:&#xA;    input: a&#xA;    output: b&#xA;    jobid: 0&#xA;    reason: Missing output files: b&#xA;    resources: tmpdir=/tmp&#xA;&#xA;[Mon Oct 31 14:41:14 2022]&#xA;Finished job 0.&#xA;1 of 1 steps (100%) done&#xA;Complete log: .snakemake/log/2022-10-31T144114.211255.snakemake.log&#xA;&lt;span class=&quot;math-container&quot;&gt;$ dd if=/dev/zero of=a bs=100000 count=1&#xA;1+0 records in&#xA;1+0 records out&#xA;100000 bytes (100 kB, 98 KiB) copied, 5.35361 s, 18.7 kB/s&#xA;$&lt;/span&gt; snakemake -c1 b&#xA;Building DAG of jobs...&#xA;Using shell: /usr/bin/bash&#xA;Provided cores: 1 (use --cores to define parallelism)&#xA;Rules claiming more threads will be scaled down.&#xA;Job stats:&#xA;job       count    min threads    max threads&#xA;------  -------  -------------  -------------&#xA;sample        1              1              1&#xA;total         1              1              1&#xA;&#xA;Select jobs to execute...&#xA;&#xA;[Mon Oct 31 14:41:26 2022]&#xA;rule sample:&#xA;    input: a&#xA;    output: b&#xA;    jobid: 0&#xA;    reason: Updated input files: a&#xA;    resources: tmpdir=/tmp&#xA;&#xA;[Mon Oct 31 14:41:26 2022]&#xA;Finished job 0.&#xA;1 of 1 steps (100%) done&#xA;Complete log: .snakemake/log/2022-10-31T144125.949828.snakemake.log&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2022-10-31T04:17:13.090" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19928" PostTypeId="1" AcceptedAnswerId="19931" CreationDate="2022-10-31T07:47:28.180" Score="1" ViewCount="77" Body="&lt;p&gt;In Pandas, we have &lt;a href=&quot;https://pandas.pydata.org/pandas-docs/stable/user_guide/options.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;pandas.options.display&lt;/code&gt;&lt;/a&gt; to adjust how a DataFrame should be displayed when printed.&lt;/p&gt;&#xA;&lt;p&gt;I wonder if &lt;code&gt;PyRanges&lt;/code&gt; has a similar set of options?&lt;/p&gt;&#xA;&lt;p&gt;For example, if I have a PyRange object like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pyranges as pr&#xA;data = pr.data.ucsc_bed()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I can do &lt;code&gt;print(data)&lt;/code&gt; and get:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;+--------------+-----------+-----------+------------+------------+-------+&#xA;| Chromosome   | Start     | End       | Feature    | gene_id    | +4    |&#xA;| (category)   | (int32)   | (int32)   | (object)   | (object)   | ...   |&#xA;|--------------+-----------+-----------+------------+------------+-------|&#xA;| chr1         | 12776117  | 12788726  | gene       | AADACL3    | ...   |&#xA;| chr1         | 169075927 | 169101957 | gene       | ATP1B1     | ...   |&#xA;| chr1         | 6845383   | 7829766   | gene       | CAMTA1     | ...   |&#xA;| chr1         | 20915589  | 20945396  | gene       | CDA        | ...   |&#xA;| ...          | ...       | ...       | ...        | ...        | ...   |&#xA;| chrX         | 152661096 | 152663330 | exon       | PNMA6E     | ...   |&#xA;| chrX         | 152661096 | 152666808 | transcript | PNMA6E     | ...   |&#xA;| chrX         | 152664164 | 152664378 | exon       | PNMA6E     | ...   |&#xA;| chrX         | 152666701 | 152666808 | exon       | PNMA6E     | ...   |&#xA;+--------------+-----------+-----------+------------+------------+-------+&#xA;Stranded PyRanges object has 5,519 rows and 9 columns from 30 chromosomes.&#xA;For printing, the PyRanges was sorted on Chromosome and Strand.&#xA;4 hidden columns: transcript_id, Strand, exon_number, transcript_name&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I increase the number of columns shown?&lt;/p&gt;&#xA;&lt;p&gt;Also appreciated if you refer me to a documentation that explains other related display options.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt; For future reference, @terdon remark encouraged me to check if such an adjustment is even considered in the &lt;code&gt;PyRanges&lt;/code&gt;. I found &lt;a href=&quot;https://github.com/biocore-ntnu/pyranges/blob/b8d3f60f454511eb52e591b7d1a6b2708b2f986f/pyranges/tostring2.py#L170&quot; rel=&quot;nofollow noreferrer&quot;&gt;the line that is responsible&lt;/a&gt; for this representation, and indeed the &lt;code&gt;COLUMN&lt;/code&gt; environment variable is the only defining parameter. In conclusion, the adjustment that I am looking for is simply not implemented in &lt;code&gt;PyRanges&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16242" LastEditorUserId="16242" LastEditDate="2022-11-02T09:46:12.810" LastActivityDate="2022-11-02T09:46:12.810" Title="How to adjust #columns shown when PyRange is printed?" Tags="|pyranges|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="19930" PostTypeId="1" CreationDate="2022-10-31T13:11:46.983" Score="1" ViewCount="417" Body="&lt;p&gt;I am working on a large data-frame that has 4 columns and each column has variable rows. I am trying to find unique and identical pathways between the 4 columns (each column represents a particular day of treatment with a drug). Here below is a small example.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Pathways_1Day &amp;lt;- c(&amp;quot;blood&amp;quot;,&amp;quot;kidney&amp;quot;,&amp;quot;testis&amp;quot;,&amp;quot;No&amp;quot;,&amp;quot;bone&amp;quot;,&amp;quot;liver&amp;quot;,&amp;quot;intestine&amp;quot;,&amp;quot;lungs&amp;quot;,&amp;quot;ABC&amp;quot;,&amp;quot;pancreas&amp;quot;,&amp;quot;Yes&amp;quot;,&amp;quot;spleen&amp;quot;)&#xA;&#xA;Pathways_2Day &amp;lt;- c(&amp;quot;blood&amp;quot;,&amp;quot;kidney&amp;quot;,&amp;quot;testis&amp;quot;,&amp;quot;eyes&amp;quot;,&amp;quot;bone&amp;quot;,&amp;quot;cells&amp;quot;,&amp;quot;intestine&amp;quot;,&amp;quot;cervix&amp;quot;,&amp;quot;ABC&amp;quot;,&amp;quot;pancreas&amp;quot;,&amp;quot;None&amp;quot;)&#xA;&#xA;Pathways_3Day &amp;lt;- c(&amp;quot;blood&amp;quot;,&amp;quot;kidney&amp;quot;,&amp;quot;vessels&amp;quot;,&amp;quot;lymph&amp;quot;,&amp;quot;t-cells&amp;quot;,&amp;quot;liver&amp;quot;,&amp;quot;intestine&amp;quot;,&amp;quot;lungs&amp;quot;,&amp;quot;ABC&amp;quot;,&amp;quot;epidermis&amp;quot;,&amp;quot;None&amp;quot;)&#xA;&#xA;df&amp;lt;-data.frame(Pathways_1Day,Pathways_2Day,Pathways_3Day) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to get a summary of the no of pathways that are common between different timepoints (1, 2 and 3 days).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Important: The no of pathways is not the same for each day.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have tried this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;All_pathwayNames &amp;lt;- df%&amp;gt;%group_by_all%&amp;gt;%count&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But the desired output is not what I am trying to get.&lt;/p&gt;&#xA;&lt;p&gt;There can be different ways to address that. It will be great if I can get matching rows infront of each other across all columns.&lt;/p&gt;&#xA;&lt;p&gt;Regards&lt;/p&gt;&#xA;" OwnerUserId="16114" LastEditorUserId="298" LastEditDate="2022-10-31T16:20:12.667" LastActivityDate="2022-11-02T22:45:32.943" Title="How to identify matching and unique rows across different columns in R data-frame?" Tags="|r|dataframe|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19931" PostTypeId="2" ParentId="19928" CreationDate="2022-10-31T13:49:39.010" Score="1" Body="&lt;p&gt;When trying this on my machine, if I resize the terminal window before running &lt;code&gt;print(data)&lt;/code&gt;, I get the full output:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cRwQ8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cRwQ8.png&quot; alt=&quot;screenshot showing both truncated and full output&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can also force this, even if your terminal is actually smaller, by setting the &lt;code&gt;COLUMNS&lt;/code&gt; variable to something larger than the actual size when layching &lt;code&gt;python&lt;/code&gt; or &lt;code&gt;ipython&lt;/code&gt;. This is a special bash variable that holds the current size of the terminal. From &lt;code&gt;man bash&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   COLUMNS&#xA;          Used  by  the select compound command to determine the terminal&#xA;          width when printing selection lists.  Automatically set if  the&#xA;          checkwinsize  option is enabled or in an interactive shell upon&#xA;          receipt of a SIGWINCH.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So if you set it manually, you can get the entire data frame printed, BUT you don't really want to do that since it won't scroll, it just wraps the text. For example, try:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;COLUMNS=300 python&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That will set the size to 300, which results in this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MvZaw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MvZaw.png&quot; alt=&quot;screenshot of the wrapped text produced when setting the COLUMNS variable&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2022-10-31T13:49:39.010" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19932" PostTypeId="2" ParentId="19930" CreationDate="2022-10-31T21:59:35.563" Score="0" Body="&lt;p&gt;You're misusing data frames. Split the df into 4 vectors and use a Venn diagram generator to denerate a Venn diagram with the 4 vectors.&lt;/p&gt;&#xA;&lt;p&gt;Adapting the first example from &lt;a href=&quot;https://r-graph-gallery.com/14-venn-diagramm&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;#untested&#xA;venn.diagram(&#xA;  x = list(df[,1], df[,2], df[,3], df[,4]), &#xA;  category.names = colnames(df),&#xA;  filename = 'my_venn_diagram.png',&#xA;  output=TRUE&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastActivityDate="2022-10-31T21:59:35.563" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19933" PostTypeId="1" CreationDate="2022-11-01T08:40:08.163" Score="2" ViewCount="179" Body="&lt;p&gt;I would like to compare gene level counts (FPKM) from StringTie output. I understand FPKM is outdated but my PI prefers to use it as a reference/guide in conjunction with the normalised counts from DESeq2.&lt;/p&gt;&#xA;&lt;p&gt;I processed my samples by first running StringTie on my aligned read files (*.bam) with reference to the ensemble GTF:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;stringtie sample$_{i}.sorted.bam -p 12 -G reference.gtf --rf -o $OUT_DIR/sample_${i}/sample_${i}.gtf -v -B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I left out the &lt;code&gt;-e&lt;/code&gt; flag in order for novel transcripts to be assembled.&lt;/p&gt;&#xA;&lt;p&gt;I then merged the individual GTFs using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;stringtie --merge -G referece.gtf -o merged.gtf -p 12 -i assembly_GTF_list.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Finally I re-run StringTie using the &lt;code&gt;merged.gtf&lt;/code&gt; as reference to obtain expression values:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;stringtie sample_${i}.sorted.bam -p 12 -G merged.gtf --rf -o sample_{i}.gtf -v -e -B -A sample_{i}.gene_abund.tab&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, in the &lt;code&gt;gene_abund.tab&lt;/code&gt; of one sample, I find that the same Gene ID is quantified twice (albeit with different coordinates):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Gene ID Gene Name       Reference       Strand  Start   End     Coverage        FPKM    TPM&#xA;ENSG00000225657 LINC01716       20      +       56577563        56596126        0.000000        0.000000        0.000000&#xA;ENSG00000225657 LINC01716       20      +       56596310        56598621        0.0     0.0     0.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These are the transcripts corresponding to 'ENSG00000225657' in the reference gtf I used:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;20      havana  transcript      56577563        56596126        .       +       .       gene_id &amp;quot;ENSG00000225657&amp;quot;; transcript_id &amp;quot;ENST00000434176&amp;quot;; gene_name &amp;quot;LINC01716&amp;quot;; ref_gene_id &amp;quot;ENSG00000225657&amp;quot;;&#xA;20      havana  exon    56577563        56577612        .       +       .       gene_id &amp;quot;ENSG00000225657&amp;quot;; transcript_id &amp;quot;ENST00000434176&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;LINC01716&amp;quot;;&#xA;20      havana  exon    56577742        56577950        .       +       .       gene_id &amp;quot;ENSG00000225657&amp;quot;; transcript_id &amp;quot;ENST00000434176&amp;quot;; exon_number &amp;quot;2&amp;quot;; gene_name &amp;quot;LINC01716&amp;quot;;&#xA;20      havana  exon    56579391        56579553        .       +       .       gene_id &amp;quot;ENSG00000225657&amp;quot;; transcript_id &amp;quot;ENST00000434176&amp;quot;; exon_number &amp;quot;3&amp;quot;; gene_name &amp;quot;LINC01716&amp;quot;;&#xA;20      havana  exon    56580201        56580335        .       +       .       gene_id &amp;quot;ENSG00000225657&amp;quot;; transcript_id &amp;quot;ENST00000434176&amp;quot;; exon_number &amp;quot;4&amp;quot;; gene_name &amp;quot;LINC01716&amp;quot;;&#xA;20      havana  exon    56595063        56595130        .       +       .       gene_id &amp;quot;ENSG00000225657&amp;quot;; transcript_id &amp;quot;ENST00000434176&amp;quot;; exon_number &amp;quot;5&amp;quot;; gene_name &amp;quot;LINC01716&amp;quot;;&#xA;20      havana  exon    56596054        56596126        .       +       .       gene_id &amp;quot;ENSG00000225657&amp;quot;; transcript_id &amp;quot;ENST00000434176&amp;quot;; exon_number &amp;quot;6&amp;quot;; gene_name &amp;quot;LINC01716&amp;quot;;&#xA;20      havana  transcript      56596310        56598621        .       +       .       gene_id &amp;quot;ENSG00000225657&amp;quot;; transcript_id &amp;quot;ENST00000437350&amp;quot;; gene_name &amp;quot;LINC01716&amp;quot;; ref_gene_id &amp;quot;ENSG00000225657&amp;quot;;&#xA;20      havana  exon    56596310        56596488        .       +       .       gene_id &amp;quot;ENSG00000225657&amp;quot;; transcript_id &amp;quot;ENST00000437350&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_name &amp;quot;LINC01716&amp;quot;;&#xA;20      havana  exon    56598503        56598621        .       +       .       gene_id &amp;quot;ENSG00000225657&amp;quot;; transcript_id &amp;quot;ENST00000437350&amp;quot;; exon_number &amp;quot;2&amp;quot;; gene_name &amp;quot;LINC01716&amp;quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I assumed that gene level counts would be the aggregation of all transcript level counts. However, it looks like StringTie is reporting the transcript level counts instead. Have I missed out a step here? Or is the &lt;code&gt;gene_abund.tab&lt;/code&gt; not meant to contain gene-level counts?&lt;/p&gt;&#xA;" OwnerUserId="6646" OwnerDisplayName="nattzy" LastEditorUserId="1157" LastEditDate="2022-11-20T13:17:27.313" LastActivityDate="2022-11-20T13:17:27.313" Title="Why does gene abundance table from StringTie output the same gene ID twice?" Tags="|stringtie|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19934" PostTypeId="1" CreationDate="2022-11-01T09:56:16.337" Score="0" ViewCount="181" Body="&lt;p&gt;I am having possibly a simple question which is proving difficult.&lt;/p&gt;&#xA;&lt;p&gt;My objective is to take the reference FASTA file and flat file database with 1500 restriction enzymes with its cutting sites (mostly pattern I would say).&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I first want to see that how many times each enzyme cut the FASTA file and;&lt;/li&gt;&#xA;&lt;li&gt;second to check the number and size of fragment that each enzyme would give me, this output pregerably would be a CSV file may be.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; I tried to do pattern search using R, but its just collapsing my system. I am looking for bash command may be.&lt;/p&gt;&#xA;" OwnerUserId="16429" OwnerDisplayName="Santosh" LastEditorUserId="3848" LastEditDate="2022-11-01T14:32:39.020" LastActivityDate="2022-11-01T22:17:59.073" Title="How to find restriction enzyme frequency in whole genome and count the distance between them?" Tags="|python|r|linux|fasta|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19935" PostTypeId="2" ParentId="19922" CreationDate="2022-11-01T13:52:00.387" Score="3" Body="&lt;p&gt;The simplest way is to re-calculate (or load, if saved) the TOM matrix and find the genes with the highest TOM to MALAT1. If you have enough RAM to carry out WGCNA in a single block, you can use TOMsimilarityFromExpr, make sure you give it the same arguments as the ones you used in network construction, and select the appropriate column from the result. Then order the column and retain top genes, as many as you think is reasonable. You can also use adjacency instead of TOM, though TOM is a more natural quantity to use if you used in your network construction. Once you figure out which genes you want to retain, you can subset the TOM matrix (both rows and columns) to those genes and you have the sub-network among these genes.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2022-11-01T13:52:00.387" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19936" PostTypeId="2" ParentId="19934" CreationDate="2022-11-01T14:58:35.130" Score="2" Body="&lt;p&gt;Biopython has a module dedicated to working with &lt;a href=&quot;http://biopython.org/DIST/docs/cookbook/Restriction.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;restriction enzymes&lt;/a&gt;. Without any code examples we cannot help you much further.&lt;/p&gt;&#xA;" OwnerUserId="3818" LastActivityDate="2022-11-01T14:58:35.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19937" PostTypeId="1" AcceptedAnswerId="19940" CreationDate="2022-11-01T18:10:34.363" Score="3" ViewCount="381" Body="&lt;p&gt;I was wondering if someone can help me figure out this error message. I’m at the end of my Nextflow pipeline and I want to change the header in a FASTA file and it works when I use this command:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;sed -i &amp;quot;s/^&amp;gt;/&amp;gt;${assembly_id}_${gene_id}_/g&amp;quot; &amp;quot;${assembly_id}/${gene_id}/${assembly_id}_${gene_id}.fasta&amp;quot;&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;but it prints out much more than I actually need (i.e., “goodID_extra_extra_extra”). That’s just part of the file name (it includes what assembler was used) but I only want it to add “goodID_” to the header.&lt;/p&gt;&#xA;&lt;p&gt;I tried using this command:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;sed -i &amp;quot;s/^&amp;gt;/&amp;gt;${assembly_id%%_*}_${gene_id}_/g&amp;quot; &amp;quot;${assembly_id}/${gene_id}/${assembly_id}_${gene_id}.fasta&amp;quot;  &lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;but I get this error message:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;N E X T F L O W  ~  version 21.10.6&#xA;Launching `geneTargets.nf` [amazing_elion] - revision: b875027a52&#xA;Script compilation error&#xA;- file : /Users/rimo/Desktop/AWS/nextflow_work/PROJECTS/gene_targets/geneTargets.nf&#xA;- cause: Unexpected input: '{' @ line 94, column 32.&#xA;   process publish_gene_sequences {&#xA;                                  ^&#xA;&#xA;1 error&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can someone explain why this is happening? If this is something I can’t do without breaking the script that’s okay the header will still do it’s purpose I just wanted it to be clearer/less noise.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: this is the process block&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process publish_gene_sequences {&#xA;publishDir params.publish_dir&#xA;input:&#xA;    tuple val(assembly_id), val(gene_id), path(&amp;quot;gene.fasta&amp;quot;)&#xA;output:&#xA;    path(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${gene_id}/$&lt;/span&gt;{assembly_id}_&lt;span class=&quot;math-container&quot;&gt;${gene_id}.fasta&quot;)&#xA;script:&#xA;    &quot;&quot;&quot;&#xA;    mkdir -p $&lt;/span&gt;{gene_id}&#xA;    cp gene.fasta &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${gene_id}/$&lt;/span&gt;{assembly_id}_&lt;span class=&quot;math-container&quot;&gt;${gene_id}.fasta&quot;&#xA;    sed -i &quot;s/^&amp;gt;/&amp;gt;$&lt;/span&gt;{assembly_id}_&lt;span class=&quot;math-container&quot;&gt;${gene_id}_/g&quot; &quot;$&lt;/span&gt;{gene_id}/&lt;span class=&quot;math-container&quot;&gt;${assembly_id}_$&lt;/span&gt;{gene_id}.fasta&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="16067" LastEditDate="2022-11-01T20:33:23.283" LastActivityDate="2022-11-02T14:46:43.203" Title="Using SED command in Nextflow script" Tags="|fasta|nextflow|errors|command-line|sed|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19938" PostTypeId="1" CreationDate="2022-11-01T19:00:16.457" Score="2" ViewCount="96" Body="&lt;p&gt;I want to create a PDB file of a drug that is given in &lt;a href=&quot;https://pubs.acs.org/doi/full/10.1021/acs.jmedchem.6b01541&quot; rel=&quot;nofollow noreferrer&quot;&gt;a paper&lt;/a&gt;.&#xA;Can somebody share a tutorial or similar resources for a beginner-level user?&#xA;&lt;a href=&quot;https://i.stack.imgur.com/N4gNJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/N4gNJ.png&quot; alt=&quot;the chemical structure I want to create a PDB from &quot; /&gt;&lt;/a&gt;&#xA;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="16355" LastEditorUserId="298" LastEditDate="2022-11-01T22:38:02.553" LastActivityDate="2023-01-12T12:24:01.597" Title="Creating a PDB from a figure in paper" Tags="|protein-structure|pdb|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19939" PostTypeId="2" ParentId="19934" CreationDate="2022-11-01T22:17:59.073" Score="2" Body="&lt;p&gt;The &lt;code&gt;diffHic&lt;/code&gt; package might have solved this issue in R.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Load the genomic data&#xA;genome &amp;lt;- biostrings::readDNAStringSet(&amp;quot;your_file.fa&amp;quot;)&#xA;&#xA;# Run the in silico restriction digest &#xA;cuts &amp;lt;- cutGenome(genome, &amp;quot;CCGCGG&amp;quot;, overhang=2L) # SacII&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;the &lt;code&gt;cuts&lt;/code&gt; GRanges object contains the boundaries of each restriction fragment for given genome.&lt;/p&gt;&#xA;" OwnerUserId="7307" LastActivityDate="2022-11-01T22:17:59.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19940" PostTypeId="2" ParentId="19937" CreationDate="2022-11-02T00:01:38.293" Score="5" Body="&lt;p&gt;Inside your script's command string, &lt;em&gt;assembly_id&lt;/em&gt; is actually a &lt;a href=&quot;https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/String.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;java.lang.String&lt;/code&gt;&lt;/a&gt; so trying to manipulate it with a Bash operation (i.e. using &lt;code&gt;${assembly_id%%_*}&lt;/code&gt;) won't produce the expected result. Here's one way to return a substring up to the first underscore; note that we can use single quotes around our sed expression:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process publish_gene_sequences {&#xA;&#xA;    publishDir params.publish_dir&#xA;&#xA;    input:&#xA;    tuple val(assembly_id), val(gene_id), path(&amp;quot;gene.fasta&amp;quot;)&#xA;&#xA;    output:&#xA;    path(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${gene_id}/$&lt;/span&gt;{assembly_id}_${gene_id}.fasta&amp;quot;)&#xA;&#xA;    script:&#xA;    def assembly = assembly_id.substring(0, assembly_id.indexOf('_'))&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    mkdir &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${gene_id}&quot;&#xA;    cp gene.fasta &quot;$&lt;/span&gt;{gene_id}/&lt;span class=&quot;math-container&quot;&gt;${assembly_id}_$&lt;/span&gt;{gene_id}.fasta&amp;quot;&#xA;    sed -i 's/^&amp;gt;/&amp;gt;&lt;span class=&quot;math-container&quot;&gt;${assembly}_$&lt;/span&gt;{gene_id}_/g' &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${gene_id}/$&lt;/span&gt;{assembly_id}_${gene_id}.fasta&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note also that in the above example, copying the file can be avoided by applying the expression directly and redirecting stdout:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;script:&#xA;def assembly = assembly_id.substring(0, assembly_id.indexOf('_'))&#xA;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;mkdir &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${gene_id}&quot;&#xA;sed \\&#xA;    -e 's/^&amp;gt;/&amp;gt;$&lt;/span&gt;{assembly}_&lt;span class=&quot;math-container&quot;&gt;${gene_id}_/g' \\&#xA;    gene.fasta \\&#xA;    &amp;gt; &quot;$&lt;/span&gt;{gene_id}/&lt;span class=&quot;math-container&quot;&gt;${assembly_id}_$&lt;/span&gt;{gene_id}.fasta&amp;quot;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2022-11-02T14:46:43.203" LastActivityDate="2022-11-02T14:46:43.203" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19941" PostTypeId="2" ParentId="19930" CreationDate="2022-11-02T10:25:00.133" Score="1" Body="&lt;p&gt;When I create the dataframe using the commands you have provided, I get an error, and the data frame can't be created:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in data.frame(Pathways_1Day, Pathways_2Day, Pathways_3Day) : &#xA;  arguments imply differing number of rows: 12, 11&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What I &lt;em&gt;think&lt;/em&gt; you were trying to do is something more like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse)&#xA;tibble() %&amp;gt;% &#xA;  bind_rows(tibble(day=1, pathway = Pathways_1Day)) %&amp;gt;%&#xA;  bind_rows(tibble(day=2, pathway = Pathways_2Day)) %&amp;gt;%&#xA;  bind_rows(tibble(day=3, pathway = Pathways_3Day)) -&amp;gt; combined.tbl&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Producing a long-format data structure that looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; combined.tbl&#xA;# A tibble: 34 x 2&#xA;     day pathway  &#xA;   &amp;lt;dbl&amp;gt; &amp;lt;chr&amp;gt;    &#xA; 1     1 blood    &#xA; 2     1 kidney   &#xA; 3     1 testis   &#xA; 4     1 No       &#xA; 5     1 bone     &#xA; 6     1 liver    &#xA; 7     1 intestine&#xA; 8     1 lungs    &#xA; 9     1 ABC      &#xA;10     1 pancreas &#xA;# ... with 24 more rows&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When laid out in this way, you can do something like grouping by pathway, and then do a &lt;code&gt;pivot_wider&lt;/code&gt; to show the sets:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; combined.tbl %&amp;gt;% &#xA;     group_by(pathway) %&amp;gt;%&#xA;     arrange(pathway) %&amp;gt;%&#xA;     pivot_wider(names_from=day, values_from=1)&#xA;# A tibble: 20 x 4&#xA;# Groups:   pathway [20]&#xA;   pathway     `1`   `2`   `3`&#xA;   &amp;lt;chr&amp;gt;     &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;&#xA; 1 ABC           1     2     3&#xA; 2 No            1    NA    NA&#xA; 3 None         NA     2     3&#xA; 4 Yes           1    NA    NA&#xA; 5 blood         1     2     3&#xA; 6 bone          1     2    NA&#xA; 7 cells        NA     2    NA&#xA; 8 cervix       NA     2    NA&#xA; 9 epidermis    NA    NA     3&#xA;10 eyes         NA     2    NA&#xA;11 intestine     1     2     3&#xA;12 kidney        1     2     3&#xA;13 liver         1    NA     3&#xA;14 lungs         1    NA     3&#xA;15 lymph        NA    NA     3&#xA;16 pancreas      1     2    NA&#xA;17 spleen        1    NA    NA&#xA;18 t-cells      NA    NA     3&#xA;19 testis        1     2    NA&#xA;20 vessels      NA    NA     3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And insert a filter to select only pathways that appear in all three days (i.e. &lt;code&gt;n() == 3&lt;/code&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; combined.tbl %&amp;gt;% &#xA;     group_by(pathway) %&amp;gt;%&#xA;     filter(n() == 3) %&amp;gt;%&#xA;     arrange(pathway) %&amp;gt;%&#xA;     pivot_wider(names_from=day, values_from=1)&#xA;# A tibble: 4 x 4&#xA;# Groups:   pathway [4]&#xA;  pathway     `1`   `2`   `3`&#xA;  &amp;lt;chr&amp;gt;     &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;&#xA;1 ABC           1     2     3&#xA;2 blood         1     2     3&#xA;3 intestine     1     2     3&#xA;4 kidney        1     2     3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or at least two days (&lt;code&gt;n() &amp;gt;= 2&lt;/code&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; combined.tbl %&amp;gt;% &#xA;     group_by(pathway) %&amp;gt;%&#xA;     filter(n() &amp;gt;= 2) %&amp;gt;%&#xA;     arrange(pathway) %&amp;gt;%&#xA;     pivot_wider(names_from=day, values_from=1)&#xA;# A tibble: 10 x 4&#xA;# Groups:   pathway [10]&#xA;   pathway     `1`   `2`   `3`&#xA;   &amp;lt;chr&amp;gt;     &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;&#xA; 1 ABC           1     2     3&#xA; 2 None         NA     2     3&#xA; 3 blood         1     2     3&#xA; 4 bone          1     2    NA&#xA; 5 intestine     1     2     3&#xA; 6 kidney        1     2     3&#xA; 7 liver         1    NA     3&#xA; 8 lungs         1    NA     3&#xA; 9 pancreas      1     2    NA&#xA;10 testis        1     2    NA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="7307" LastEditDate="2022-11-02T22:45:32.943" LastActivityDate="2022-11-02T22:45:32.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19943" PostTypeId="2" ParentId="19925" CreationDate="2022-11-02T19:43:23.747" Score="2" Body="&lt;p&gt;As per PROSITE database manual:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Unlike patterns, profiles are usually not confined to small regions&#xA;with high sequence similarity. Rather they attempt to characterize a&#xA;protein family or domain over its entire length. This can lead to&#xA;specific problems not arising with PROSITE patterns. With a profile&#xA;covering conserved as well as divergent sequence regions, there is a&#xA;chance to obtain a significant similarity score even with a partially&#xA;incorrect alignment. This possibility is taken into account by our&#xA;quality evaluation procedures. &lt;strong&gt;In order to be acceptable, a profile&#xA;must not only assign high similarity scores to true motif occurrences&#xA;and low scores to false matches.&lt;/strong&gt; In addition, it should correctly&#xA;align those residues having analogous functions or structural&#xA;properties according to experimental data.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Profiles are supposed to be more sensitive and more robust than&lt;/strong&gt;&#xA;&lt;strong&gt;patterns because they provide discriminatory weights not only for the&lt;/strong&gt;&#xA;&lt;strong&gt;residues already found at a given position of a motif but also for&lt;/strong&gt;&#xA;&lt;strong&gt;those not yet found.&lt;/strong&gt; The weights for those not yet found are&#xA;extrapolated from the observed amino acid compositions using empiric&#xA;knowledge about amino acid substitutability.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Given the less biased and somewhat more balanced result provided by a profile I would say that the sequence contains two motifs that correspond to a PS51408 TRANSFERRIN_LIKE_4 Transferrin-like domain profile and provide the bitscore and e-value for the match.&lt;/p&gt;&#xA;&lt;p&gt;Best,&lt;/p&gt;&#xA;&lt;p&gt;av&lt;/p&gt;&#xA;" OwnerUserId="14212" LastActivityDate="2022-11-02T19:43:23.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19944" PostTypeId="2" ParentId="19921" CreationDate="2022-11-02T19:52:30.997" Score="2" Body="&lt;p&gt;I think your question has been answered very well here:&#xA;&lt;a href=&quot;https://www.biostars.org/p/284775/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/284775/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you want to go deeper perhaps you can look at this paper by Li et al.&#xA;&lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-022-02648-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-022-02648-4&lt;/a&gt;&#xA;where they argue that for large scale datasets (multiple conditions) both of these methods are not optimal and instead suggests using Wilcoxon rank-sum test to decrease chances of FDRs.&lt;/p&gt;&#xA;&lt;p&gt;Best,&lt;/p&gt;&#xA;&lt;p&gt;av&lt;/p&gt;&#xA;" OwnerUserId="14212" LastActivityDate="2022-11-02T19:52:30.997" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19945" PostTypeId="2" ParentId="19924" CreationDate="2022-11-03T10:00:48.150" Score="2" Body="&lt;p&gt;I was able to spot the problem. I thought the Server I was working on had at least 256 RAM. Turns out it only had 64.&#xA;I think HybridScaffolding needs at least 128 RAM. So when I ran the whole HybridScaffolding script on a stronger Server (1000 RAM) it worked just fine without any problems :)&lt;/p&gt;&#xA;&lt;p&gt;Edit: For the record, I also had to unzip the cmap.gz file because the RefAligner could not read it zipped.&lt;/p&gt;&#xA;" OwnerUserId="16419" LastActivityDate="2022-11-03T10:00:48.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19948" PostTypeId="1" CreationDate="2022-11-04T15:07:46.873" Score="1" ViewCount="296" Body="&lt;p&gt;When running CellRanger on 10x scRNA-Seq data, a BAM file is output with the aligned reads. These reads have an &lt;code&gt;xf&lt;/code&gt; tag, described &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/output/bam&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The bits of this tag are interpreted as follows:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1 The read is confidently mapped to the transcriptome&lt;/li&gt;&#xA;&lt;li&gt;2 This read's barcode, UMI, and feature combination was discarded in favor of a different feature with higher read support&lt;/li&gt;&#xA;&lt;li&gt;4 This read pair maps to a discordant pair of genes, and is not treated as a UMI count&lt;/li&gt;&#xA;&lt;li&gt;8 This read is representative for a molecule and is treated as a UMI count&lt;/li&gt;&#xA;&lt;li&gt;16 This read maps to exactly one feature, and is identical to bit 1 for transcriptomic reads. Notably, this bit is set for a Feature&#xA;Barcode read, while bit 1 is not&lt;/li&gt;&#xA;&lt;li&gt;32 This read was removed by targeted UMI filtering.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In a real dataset, I have this distribution of xf values:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ samtools view possorted_genome_bam.bam |  egrep -o &amp;quot;xf:i:[0-9]+&amp;quot; | sort | uniq -c&#xA;66263599 xf:i:0&#xA;99892363 xf:i:17&#xA;  245179 xf:i:19&#xA; 8283967 xf:i:25&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It is clear to me that &lt;code&gt;0&lt;/code&gt; are unaligned, &lt;code&gt;19 = 16+2+1&lt;/code&gt; are discarded, &lt;code&gt;25 = 16+8+1&lt;/code&gt; are kept (and used in the feature_barcode_matrix); but I do not understand what happens to the &lt;code&gt;17 = 16 +1&lt;/code&gt;. They seem to be good reads, they are aligned on genes, yet they are discarded.&lt;/p&gt;&#xA;&lt;p&gt;One possible explanation from my reading of &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/algorithms/overview#umi_counting&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paragraph&lt;/a&gt; is that the reads with &lt;code&gt;17&lt;/code&gt; are PCR duplicates, thus ignored, whereas the reads with &lt;code&gt;19&lt;/code&gt; are in a group matching different genes (several reads with the same BC/UMI but matching different genes). But this doesn't look right when exploring the BAM file manually.&lt;/p&gt;&#xA;" OwnerUserId="9497" LastActivityDate="2023-04-11T05:34:31.557" Title="In CellRanger output, what are the reads with an xf:i:17 tag?" Tags="|sam|10x-genomics|cellranger|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19949" PostTypeId="1" AcceptedAnswerId="19968" CreationDate="2022-11-04T17:42:01.557" Score="1" ViewCount="201" Body="&lt;p&gt;I am using &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/index.html#&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools&lt;/a&gt; to caculate the coverage of my targets of my WES data, to later plot. But to plot, I need to know what to plot and what it is I am seeing. I have unsuccefully tried to find what exactly the columns are in the output of the &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/coverage.html#&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtool coverage&lt;/a&gt;, and how to read them. Also, the documentation does not help me a lot, and I find it confusing. Does anyone know, or where to find this a bit more elaboratly explained?&lt;/p&gt;&#xA;&lt;p&gt;I used the following code:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;bedtools coverage -hist -abam input.bam -b input.bed &amp;gt; output &amp;gt;2 log&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Correction note&lt;/strong&gt;: the code above is actually wrong (thanks to @Alexlok) and should be:&#xA;&lt;code&gt;bedtools coverage -hist -a input.bed -b input.bam &amp;gt; output 2&amp;gt; log&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;And the output is (first line):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/M2fdP.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/M2fdP.png&quot; alt=&quot;First 5 lines&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Last 5 lines:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/008DK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/008DK.png&quot; alt=&quot;Last 5 Lines&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance&lt;/p&gt;&#xA;" OwnerUserId="16204" LastEditorUserId="16204" LastEditDate="2022-11-09T10:58:02.997" LastActivityDate="2022-11-09T10:58:02.997" Title="What are the columns in bedtools coverage output hist file?" Tags="|sequence-analysis|bedtools|coverage|exome|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19950" PostTypeId="1" AcceptedAnswerId="19965" CreationDate="2022-11-04T17:50:04.983" Score="2" ViewCount="98" Body="&lt;p&gt;I am converting a biallelic VCF into a SQL table and one of my tables will be something like:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-markdown prettyprint-override&quot;&gt;&lt;code&gt;SAMPLE | custom variant identifier | Genotype&#xA;-------|---------------------------|---------&#xA;0001   | 3456                      | ./.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My question is that for the diploid &lt;code&gt;./.&lt;/code&gt; or haploid &lt;code&gt;.&lt;/code&gt; missing alleles, whether I can just drop them to save space. My thinking is that since the dot represents missing data, the &lt;code&gt;./.&lt;/code&gt; are just placeholders in the VCF but provide no information at all.&lt;/p&gt;&#xA;&lt;p&gt;Is this correct? Should I retain the rows with &lt;code&gt;./.&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Showing the end of one line of the VCF (really long lines because of high N):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FORMAT          SampleX             SampleY&#xA;GT:AD:DP:GQ:PL  0/0:3,0:3:4:0,4,45  ./.:3,0:3:.:0,0,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I thought to also show the result after &lt;code&gt;bcftools norm -m&lt;/code&gt; to split to biallelic and queried per sample.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sample_id   chr@pos@ref@alt genotype    &#xA;sample1 chr1@10158@ACCCT@A       0/0&#xA;sample2 chr1@10158@ACCCT@A       ./. &amp;lt;- does this provide any info or can I discard?&#xA;sample3 chr1@10158@ACCCT@A       0/0&#xA;sample4 chr1@10158@ACCCT@A       ./.&#xA;sample5 chr1@10158@ACCCT@A       0/0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16168" LastEditorUserId="16168" LastEditDate="2022-11-05T16:34:14.333" LastActivityDate="2022-11-07T07:29:29.923" Title="Can I discard missing alleles?" Tags="|vcf|genomics|bcftools|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19951" PostTypeId="1" CreationDate="2022-11-04T18:01:19.923" Score="1" ViewCount="576" Body="&lt;p&gt;I have a fasta file of 3'UTR sequences of the genes. Some of the gene do not have a 3'UTR sequence and reported with &lt;code&gt;sequence unavailable&lt;/code&gt; but there is FASTA header.&lt;/p&gt;&#xA;&lt;p&gt;I need to delete or remove all the &amp;quot;sequence unavailable&amp;quot; lines along with their header, i.e. all blank fasta entries.&lt;/p&gt;&#xA;&lt;p&gt;Please suggest a command with &lt;code&gt;awk&lt;/code&gt; or &lt;code&gt;sed&lt;/code&gt; command.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5orlh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5orlh.png&quot; alt=&quot;Image of FASTA file&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16442" LastEditorUserId="3848" LastEditDate="2023-08-26T13:42:27.360" LastActivityDate="2023-08-26T13:42:27.360" Title="Removing particular line along with Fasta header in fasta file" Tags="|phylogenetics|phylogeny|fasta|awk|grep|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19952" PostTypeId="2" ParentId="19951" CreationDate="2022-11-04T18:51:03.250" Score="2" Body="&lt;p&gt;If there are no line breaks within the sequence data (the format I use) then the following will work.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -0777 -nle 'while (/^(&amp;gt;.+\n^[AGCT-]+)\n/gm){print $1,&amp;quot;\n&amp;quot;}' myfile.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Test&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;foo&#xA;AGAGTAGATAGAAGATA&#xA;&amp;gt;bar&#xA;AGAGATAGAGAGATATA&#xA;&amp;gt;baz&#xA;Sequence unavailable&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;foo&#xA;AGAGTAGATAGAAGATA&#xA;&amp;gt;bar&#xA;AGAGATAGAGAGATATA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;However&lt;/strong&gt; If there are line breaks within the sequence data, please use the following ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -0777 -nle 'while (/^(&amp;gt;.+\n[AGCT-]{10}.+\n[^&amp;gt;]+)/gm){print $1}' myfile.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then pipe to a file,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -0777 -nle 'while (/^(&amp;gt;.+\n[AGCT-]{10}.+\n[^&amp;gt;]+)/gm){print $1}' myfile.fa &amp;gt; myfile_minus_blanks.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This works due to a multi-line &lt;code&gt;m&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Test ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;foo&#xA;AGAGTAGATAGAAGATA&#xA;AGAGTAGATAGAAGATA&#xA;AGAGTAGATAGAAGATA&#xA;AGAGTAGATAGAAGATA&#xA;&amp;gt;bar&#xA;AGAGATAGAGAGATATA&#xA;AGAGATAGAGAGATATA&#xA;&amp;gt;baz&#xA;Sequence unavailable&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Out ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;foo&#xA;AGAGTAGATAGAAGATA&#xA;AGAGTAGATAGAAGATA&#xA;AGAGTAGATAGAAGATA&#xA;AGAGTAGATAGAAGATA&#xA;&amp;gt;bar&#xA;AGAGATAGAGAGATATA&#xA;AGAGATAGAGAGATATA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;&#xA;If you've mixed bases, degenerate nucleotides,&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;AGCT-&lt;/code&gt; should be replaced with &lt;code&gt;AGCTRYWSMKNBDHV&lt;/code&gt; (omitting &lt;code&gt;-&lt;/code&gt; [not needed])&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="298" LastEditDate="2022-11-05T12:58:19.177" LastActivityDate="2022-11-05T12:58:19.177" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19953" PostTypeId="2" ParentId="19951" CreationDate="2022-11-05T13:06:34.450" Score="3" Body="&lt;p&gt;Try this, if you have GNU &lt;code&gt;grep&lt;/code&gt; (the default on Linux systems):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep --no-group-separator -i -B 1 '^[ACGTN]\+$' input.fasta &amp;gt; filtered.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Explanation: find all lines that are composed of only A/C/G/T/N (case insensitive search), and show that line plus the preceding line. Don't put any breaks between matches that are not consecutive.&lt;/p&gt;&#xA;&lt;p&gt;This should work on both multi-line and single line fasta files.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-11-05T13:14:16.047" LastActivityDate="2022-11-05T13:14:16.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19954" PostTypeId="2" ParentId="19951" CreationDate="2022-11-05T13:08:12.123" Score="4" Body="&lt;p&gt;You can use &lt;code&gt;awk&lt;/code&gt; here, by setting the input (&lt;code&gt;RS&lt;/code&gt;) and output (&lt;code&gt;ORS&lt;/code&gt;) record separator, this is what defines a &amp;quot;line&amp;quot; in awk, to &lt;code&gt;&amp;gt;&lt;/code&gt;. Then, tell it to print records that do not match &lt;code&gt;Sequence unavailable&lt;/code&gt;. For instance, using this input file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat file&#xA;&amp;gt;foo&#xA;AGAGTAGATAGAAGATA&#xA;&amp;gt;bar&#xA;AGAGATAGAGAGATATA&#xA;&amp;gt;baz&#xA;Sequence unavailable&#xA;&amp;gt;bar1&#xA;AGAGATAGAGAGATATA&#xA;&amp;gt;asld&#xA;Sequence unavailable&#xA;&amp;gt;asd&#xA;ACTTC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ awk  'BEGIN{RS=ORS=&amp;quot;&amp;gt;&amp;quot;}!/Sequence unavailable/' file &#xA;&amp;gt;foo&#xA;AGAGTAGATAGAAGATA&#xA;&amp;gt;bar&#xA;AGAGATAGAGAGATATA&#xA;&amp;gt;bar1&#xA;AGAGATAGAGAGATATA&#xA;&amp;gt;asd&#xA;ACTTC&#xA;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That will add a trailing &lt;code&gt;&amp;gt;&lt;/code&gt;, so you need one more step to remove it:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ awk  'BEGIN{RS=ORS=&amp;quot;&amp;gt;&amp;quot;}!/Sequence unavailable/' file | sed 's/&amp;gt;$//'&#xA;&amp;gt;foo&#xA;AGAGTAGATAGAAGATA&#xA;&amp;gt;bar&#xA;AGAGATAGAGAGATATA&#xA;&amp;gt;bar1&#xA;AGAGATAGAGAGATATA&#xA;&amp;gt;asd&#xA;ACTTC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Alternatively, using the &lt;code&gt;FastaToTbl&lt;/code&gt; and &lt;code&gt;TblToFasta&lt;/code&gt; scripts I have posted &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2651/298&quot;&gt;here&lt;/a&gt;, you can do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ FastaToTbl file | grep -v 'Sequence unavailable' | TblToFasta &#xA;&amp;gt;foo &#xA;AGAGTAGATAGAAGATA&#xA;&amp;gt;bar &#xA;AGAGATAGAGAGATATA&#xA;&amp;gt;bar1 &#xA;AGAGATAGAGAGATATA&#xA;&amp;gt;asd &#xA;ACTTC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Both approaches have the advantage of not needing to read the entire file to memory so can be used on files of any size.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2022-11-05T13:08:12.123" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19955" PostTypeId="2" ParentId="19951" CreationDate="2022-11-05T13:43:35.993" Score="2" Body="&lt;p&gt;One way using &lt;a href=&quot;/questions/tagged/sed&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;sed&amp;#39;&quot; aria-label=&quot;show questions tagged &amp;#39;sed&amp;#39;&quot; rel=&quot;tag&quot; aria-labelledby=&quot;sed-container&quot;&gt;sed&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sed -e '/^&amp;gt;/ { N; }; /Sequence unavailable/ { d; }' file.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For each header line, append the next line of input into the pattern space. Then, if the pattern space contains &amp;quot;Sequence unavailable&amp;quot; delete the pattern space and start a new cycle.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Or using &lt;a href=&quot;/questions/tagged/awk&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; aria-label=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; rel=&quot;tag&quot; aria-labelledby=&quot;awk-container&quot;&gt;awk&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '/^&amp;gt;/ { f=$0; next } /^Sequence unavailable/ { next } f { print f; f=&amp;quot;&amp;quot; }1' file.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For each header line, use a variable to store the current record. We can use &lt;a href=&quot;https://www.gnu.org/software/gawk/manual/html_node/Next-Statement.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;next&lt;/em&gt;&lt;/a&gt; here to force awk to immediately stop processing the current record and move on to the next record, which is just the next line, since, by default, awk's &lt;a href=&quot;https://www.gnu.org/software/gawk/manual/html_node/awk-split-records.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;record separator&lt;/em&gt;&lt;/a&gt; is a newline character. Then, stop processing the current record and move on to the next record if that line begins with &amp;quot;Sequence unavailable&amp;quot;. Finally, print the header line if it exists and unset the variable that we used to store it if it does. Once we've printed the header, we can then just print the current record. We can do this by appending &lt;em&gt;1&lt;/em&gt; to our command string. The &lt;em&gt;1&lt;/em&gt; on the end just returns true, and by doing so, it performs the default action, which is to print the current record.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2022-11-06T14:22:43.730" LastActivityDate="2022-11-06T14:22:43.730" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19956" PostTypeId="1" AcceptedAnswerId="19982" CreationDate="2022-11-05T16:26:02.437" Score="1" ViewCount="179" Body="&lt;p&gt;In the &lt;code&gt;biomaRt&lt;/code&gt; package, there is a function &lt;code&gt;getBM&lt;/code&gt; which among a few things is useful for mapping between different gene representations. Currently, I'm using it as follows, to map Ensembl transcript IDs to gene names and Ensembl gene IDs.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ensemble2gene &amp;lt;- getBM(attributes=c(&amp;quot;ensembl_transcript_id&amp;quot;, &#xA;                                    &amp;quot;external_gene_name&amp;quot;,&#xA;                                    &amp;quot;ensembl_gene_id&amp;quot;),&#xA;                       values = as.list(transcripts), &#xA;                       mart = mart)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How is the following code different:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ensemble2gene &amp;lt;- getBM(attributes=c(&amp;quot;ensembl_transcript_id&amp;quot;, &#xA;                                    &amp;quot;external_gene_name&amp;quot;,&#xA;                                    &amp;quot;ensembl_gene_id&amp;quot;),&#xA;                       filters = &amp;quot;ensembl_transcript_id&amp;quot;,&#xA;                       values = as.list(transcripts), &#xA;                       mart = mart)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It's unclear to me from the description of the &lt;code&gt;filters&lt;/code&gt; argument &lt;a href=&quot;https://www.rdocumentation.org/packages/biomaRt/versions/2.28.0/topics/getBM&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. But just based on results, I get less results when I run the second one.&lt;/p&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2022-11-09T12:05:21.980" Title="What does the &quot;filters&quot; argument in &quot;getBM&quot; do?" Tags="|ensembl|biomart|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19957" PostTypeId="1" AcceptedAnswerId="19983" CreationDate="2022-11-05T16:33:44.773" Score="1" ViewCount="248" Body="&lt;p&gt;When I run the following R code to convert Ensembl transcript IDs to gene names and Ensembl gene IDs:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mart &amp;lt;- useMart(&amp;quot;ensembl&amp;quot;, dataset = &amp;quot;hsapiens_gene_ensembl&amp;quot;)&#xA;ensemble2gene &amp;lt;- getBM(attributes=c(&amp;quot;ensembl_transcript_id&amp;quot;, &#xA;                                    &amp;quot;external_gene_name&amp;quot;,&#xA;                                    &amp;quot;ensembl_gene_id&amp;quot;),&#xA;                       filters = &amp;quot;ensembl_transcript_id&amp;quot;,&#xA;                       values = as.list(transcripts), &#xA;                       mart = mart)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;some of rows returned have NA values for both the gene name and Ensembl gene ID. This is a bit odd because for example, one of the transcript IDs with missing gene IDs is ENST00000414213 but when I google it, I find that it corresponds to gene ENSG00000235143 (according to &lt;a href=&quot;http://genome-euro.ucsc.edu/cgi-bin/hgGene?hgg_gene=ENST00000431202.1&amp;amp;hgg_chrom=chr1&amp;amp;hgg_start=30858157&amp;amp;hgg_end=30860254&amp;amp;hgg_type=knownGene&amp;amp;db=hg38&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;). Why am I getting missing results in &lt;code&gt;getBM&lt;/code&gt; then? Is it because maybe the transcript is from an old version of the database? If so, is that the only reason or are there other reasons this could be happening?&lt;/p&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="298" LastEditDate="2022-11-07T09:25:04.907" LastActivityDate="2022-11-09T12:09:51.040" Title="Why do I have missing values returned from getBM when converting Ensembl transcript IDs to gene names?" Tags="|r|ensembl|biomart|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19958" PostTypeId="1" CreationDate="2022-11-05T18:08:04.040" Score="2" ViewCount="199" Body="&lt;p&gt;I am new to proteomics analysis, so apologies if my question is silly! For context, I am working with proteome samples from the postmortem human brain for a case-control study.&lt;/p&gt;&#xA;&lt;p&gt;Our lab generally receives an output file of &lt;a href=&quot;https://www.thermofisher.com/uk/en/home/industrial/mass-spectrometry/liquid-chromatography-mass-spectrometry-lc-ms/lc-ms-software/multi-omics-data-analysis/proteome-discoverer-software.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Proteome Discoverer&lt;/a&gt;&lt;strong&gt;*&lt;/strong&gt; (v2.5) from the proteomics center that we work with. From what I understand, PD perform t-tests on the normalized data, but it is a background-based t-test, so there is a weighting factor is applied that takes into account the distribution of ratios for all other proteins and/or the protein abundances themselves? As such, I am able to see an &amp;quot;Abundance Ratio Adj. P-Value&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;My problem is that my data &lt;strong&gt;does not meet the assumptions for a parametric test&lt;/strong&gt; and it has a small sample size, so the t-test is not valid (as far as I can tell, unless there is some sort of log transformation that PD does to the data before the t-test). Therefore, I would like to run a Mann-Whitney U Test. Can this be done in the PD software?&lt;/p&gt;&#xA;&lt;p&gt;If not, &lt;strong&gt;how can I do this background adjustment manually such that I can run a Mann-Whitney U Test but still take into protein abundance like in PD?&lt;/strong&gt; I do most of my stats in R in case that is relevant.&lt;/p&gt;&#xA;&lt;p&gt;Thank you so much in advance for your help!&lt;/p&gt;&#xA;&lt;p&gt;Kelly&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Package distributed by ThermoFisher&lt;/p&gt;&#xA;" OwnerUserId="16445" LastEditorUserId="3848" LastEditDate="2022-11-06T14:09:54.883" LastActivityDate="2022-11-06T14:09:54.883" Title="Non-parametric, background-based tests on proteomics data from Proteome Discoverer v2.5?" Tags="|proteins|statistics|differential-expression|proteomics|data-preprocessing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19961" PostTypeId="2" ParentId="19958" CreationDate="2022-11-06T11:05:02.633" Score="1" Body="&lt;p&gt;Small samples size and parametric tests are not mutually exclusive. Specialized software has been developed to leverage sharing information across  measurements to boost power, that is called Empirical Bayes. You can use the limma package for this sort of analysis, contrasting groups. Input is expected to be normalized and on log2-scale. It's an R package hosted at Bioconductor. Don't try a MWU, with small sample size that is massively underpowered.&lt;/p&gt;&#xA;" OwnerUserId="16448" LastActivityDate="2022-11-06T11:05:02.633" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19962" PostTypeId="1" AcceptedAnswerId="19963" CreationDate="2022-11-06T13:38:12.580" Score="1" ViewCount="149" Body="&lt;p&gt;This is my first nextflow pipeline and i am struggling with the .fromFilePairs concept of Nextflow.&lt;/p&gt;&#xA;&lt;p&gt;I want to process paired-end read files and align them against a reference genome. I use Hisat2 so the index files come in form of 8 files with the suffix .ht2. Hisat needs the file path and the filename without &amp;quot;.ht2&amp;quot;. All worked well with a single-end version of the process. The reference also has to be passed through a channel because i use a hisat docker-container and otherwise the files are not mounted.&lt;/p&gt;&#xA;&lt;p&gt;The idea is to pass read-pairs to a preprocessing step and then pass them (as tuples, like a Channel.fromPairs()) into the hisat process and then further downstream.&lt;/p&gt;&#xA;&lt;p&gt;My problem is all file-pairs are processed during the first step but only the first pair is passed further downstream. All downstream steps finish with the one pair and the rest simply stays in the output-folder of the preprocessing step.&lt;/p&gt;&#xA;&lt;p&gt;I tried to simply skip the first step to see if the &amp;quot;connection&amp;quot; between the first and second process is not working but the Channel which works for the first step does not for the hisat step. So i think there is a problem with the combination of multiple read-pairs as input but only one reference file?!&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process hisat2_paired {&#xA;  tag &amp;quot;$sample_id&amp;quot;&#xA; &#xA;  container &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${params.container_hisat}&quot;&#xA;  publishDir &quot;$&lt;/span&gt;{params.out_dir}/Nextflow_output/hisat/&amp;quot;&#xA;&#xA;  input:&#xA;  tuple val(sample_id), path(input_files)&#xA;  path( index_files )&#xA;&#xA;  output:&#xA;  path('*.sam') , emit: hisat2_sam_out_ch&#xA;  path('*.txt')&#xA;&#xA;  script:&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  INDEX=`find -L ${index_files}/ -name &amp;quot;*.1.ht2&amp;quot; | sed 's/.1.ht2//'` &#xA;  hisat2 --rna-strandness RF \&#xA;          -x \\\$INDEX \&#xA;          -1 &lt;span class=&quot;math-container&quot;&gt;${input_files[0]} \&#xA;      -2 $&lt;/span&gt;{input_files[1]} \&#xA;          -S &lt;span class=&quot;math-container&quot;&gt;${sample_id}.sam &amp;amp;&amp;gt; $&lt;/span&gt;{sample_id}.hisat_summary.txt&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow {&#xA;&#xA;Channel&#xA;    .fromFilePairs( &amp;quot;${params.reads_folder}/*_{R1,R2}*f*q.gz&amp;quot;)&#xA;    .ifEmpty { exit 1, &amp;quot;no fastq files found at given path&amp;quot; }&#xA;    .view()&#xA;    .set{ read_pair_ch }&#xA;&#xA;Channel&#xA;    .fromPath( params.hisat2_index )&#xA;    .ifEmpty { exit 1, &amp;quot;no hisat2 index files found - path was empty&amp;quot; }&#xA;    //.view()&#xA;    .set { hisat2_index_ch }&#xA;&#xA;hisat2_paired( read_pair_ch, hisat2_index_ch )&#xA;&#xA;}&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I started the workflow with 3 test file-pairs and the Output on the CLI states that only one process is started although if i look at the Channel.view() all 3 file pairs are printed.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;executor &amp;gt;  local (1)&#xA;[2f/30a4d6] process &amp;gt; hisat2_paired (MC3T3-Ko1_S2_L001) [  0%] 0 of 1&#xA;[MC3T3-Ko1_S2_L001, [/mnt/c/Path/to/MC3T3-Ko1_S2_L001_R1_001.fastq.gz, /mnt/c/Path/to/MC3T3-Ko1_S2_L001_R2_001.fastq.gz]]&#xA;[MC3T3-Ko2_S4_L001, [/mnt/c/Path/to/MC3T3-Ko2_S4_L001_R1_001.fastq.gz, /mnt/c/Path/to/MC3T3-Ko2_S4_L001_R2_001.fastq.gz]]&#xA;[MC3T3-Ko3_S6_L001, [/mnt/c/Path/to/MC3T3-Ko3_S6_L001_R1_001.fastq.gz, /mnt/c/Path/to/MC3T3-Ko3_S6_L001_R2_001.fastq.gz]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I was already thinking about memory and resources. Since I prototype on my local PC i set the queue size in the config file to 2. I know that my hisat process needs about 7Gb RAM (for Mus.musculus) per process so there should be enough resources. Also the problem was that the local executor is not even trying to start a second process.&lt;/p&gt;&#xA;" OwnerUserId="16449" LastEditorUserId="73" LastEditDate="2022-11-07T10:20:50.063" LastActivityDate="2022-11-07T10:20:50.063" Title="Only the first file Pair is processed from Channel.fromFilePairs()" Tags="|nextflow|rna-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19963" PostTypeId="2" ParentId="19962" CreationDate="2022-11-06T15:42:37.613" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;So i think there is a problem with the combination of multiple&#xA;read-pairs as input but only one reference file?!&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Correct. The problem is that the &lt;em&gt;hisat2_index_ch&lt;/em&gt; is a &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#queue-channel&quot; rel=&quot;nofollow noreferrer&quot;&gt;queue channel&lt;/a&gt;, but what you want is a &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#value-channel&quot; rel=&quot;nofollow noreferrer&quot;&gt;value channel&lt;/a&gt; so that the (file) values can be read an unlimited number of times without being consumed. Assuming &lt;em&gt;params.hisat2_index&lt;/em&gt; is intended to supply a glob pattern, you could simply call the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#collect&quot; rel=&quot;nofollow noreferrer&quot;&gt;collect&lt;/a&gt; operator to produce a list of files as the sole emission:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Channel&#xA;    .fromPath( params.hisat2_index )&#xA;    .collect()&#xA;    .set { hisat2_index_ch }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You would then need to update your process definition accordingly. Alternatively, have your workflow create the HISAT2 index files for you. This simplifies things considerably, by requiring only a single reference FASTA file as input. Note that value channels are implicitly created as output for a process whose inputs are all value channels. This means in the example below, &lt;em&gt;hisat2_index.out&lt;/em&gt; is also a value channel:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params.publish_dir = './results'&#xA;&#xA;params.container_hisat = 'quay.io/biocontainers/hisat2:2.2.1--h87f3376_4'&#xA;&#xA;params.reads = '*_R{1,2}.fastq.gz'&#xA;params.ref_fasta = 'chr1.fasta'&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;process hisat2_index {&#xA;&#xA;    container params.container_hisat&#xA;    cpus 8&#xA;&#xA;    input:&#xA;    path fasta&#xA;&#xA;    output:&#xA;    tuple val(fasta.baseName), path(&amp;quot;${fasta.baseName}.*.ht2&amp;quot;)&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    hisat2-build \\&#xA;        -p &lt;span class=&quot;math-container&quot;&gt;${task.cpus} \\&#xA;        &quot;$&lt;/span&gt;{fasta}&amp;quot; \\&#xA;        &amp;quot;${fasta.baseName}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;process hisat2_paired {&#xA;&#xA;    tag { sample_id }&#xA;&#xA;    container params.container_hisat&#xA;    publishDir &amp;quot;${params.publish_dir}/hisat&amp;quot;&#xA;&#xA;    input:&#xA;    tuple val(sample_id), path(input_files)&#xA;    tuple val(idx), path('hisat2-idx/*')&#xA;&#xA;    output:&#xA;    path('*.sam'), emit: hisat2_sam&#xA;    path('*.txt')&#xA;&#xA;    script:&#xA;    def (fq1, fq2) = input_files&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    hisat2 \\&#xA;      --rna-strandness RF \\&#xA;      -x &amp;quot;hisat2-idx/&lt;span class=&quot;math-container&quot;&gt;${idx}&quot; \\&#xA;      -1 $&lt;/span&gt;{fq1} \\&#xA;      -2 &lt;span class=&quot;math-container&quot;&gt;${fq2} \\&#xA;      -S $&lt;/span&gt;{sample_id}.sam \\&#xA;      &amp;amp;&amp;gt; ${sample_id}.hisat_summary.txt&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow {&#xA;&#xA;    read_pairs = Channel.fromFilePairs( params.reads )&#xA;    ref_fasta = file( params.ref_fasta )&#xA;&#xA;    hisat2_index( ref_fasta )&#xA;&#xA;    hisat2_paired( read_pairs, hisat2_index.out )&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2022-11-06T16:14:45.087" LastActivityDate="2022-11-06T16:14:45.087" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="19964" PostTypeId="1" CreationDate="2022-11-06T15:29:33.300" Score="0" ViewCount="88" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; I have a data set of 4665 proteins from &lt;em&gt;Phytophthora cactorum&lt;/em&gt; and want to analyze them in the &lt;a href=&quot;https://string-db.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;String database&lt;/a&gt; as other database does not have &lt;em&gt;Phytophthora cactorum&lt;/em&gt; for GO or KEGG analysis. The String database states I can upload a maximum of 2000 proteins at once.&lt;/p&gt;&#xA;&lt;p&gt;Thus I subset my data into 3 sets and upload. However, it gives/risks statistically biased analysis.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; Is there any possibility I can analyze all of my 4665 protein sequences at once?&lt;/p&gt;&#xA;&lt;p&gt;Looking forward for the help.&lt;/p&gt;&#xA;&lt;p&gt;Best regards,&lt;/p&gt;&#xA;&lt;p&gt;Bikal&lt;/p&gt;&#xA;" OwnerUserId="16456" OwnerDisplayName="Bikal" LastEditorUserId="3848" LastEditDate="2022-11-08T14:23:21.113" LastActivityDate="2022-11-10T10:27:50.613" Title="Analysis of 4665 proteins in String database" Tags="|sequence-analysis|kegg|stringdb|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="19965" PostTypeId="2" ParentId="19950" CreationDate="2022-11-06T16:11:42.897" Score="1" Body="&lt;p&gt;If you can, do not discard missing allele information.&lt;/p&gt;&#xA;&lt;p&gt;Generally you often benefit from being able to distinguish between a position that you interrogated and genotyped as REF (reference) versus one that you interrogated but could not determine the genotype (e.g due to low coverage in the sequencing or probing issues in a microarray setup) and which usually is encoded as a missing (./. if ploidy = 2) genotype. A missing genotype tell us that the genotype at this position could be REF, but could also be some other base, we simply do not know.&lt;/p&gt;&#xA;&lt;p&gt;Knowing this is critical and/or useful depending the downstream analysis. For example it is quite important and informative when merging datasets. Since formats like VCF or PLINK do not record non-variant positions, if you discarded missing genotypes prior to merging, how can you tell between genotypes that truly were REF in one dataset (fully invariant position) and some recorded missing genotypes at the same position but which you decided to discard? You will end up assuming REF status for everything not recorded and this will inevitably lead to batch effects in your combined dataset. For a better discussion see this &lt;a href=&quot;http://massgenomics.org/2014/03/gwas-sequencing-realities.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, if your relational database (your particular situation) for instance will not incorporate external databases in order to produce joint datasets, perhaps discarding missing genotypes is all fine.&lt;/p&gt;&#xA;" OwnerUserId="475" LastEditorUserId="475" LastEditDate="2022-11-07T07:29:29.923" LastActivityDate="2022-11-07T07:29:29.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19966" PostTypeId="1" CreationDate="2022-11-06T19:47:57.940" Score="1" ViewCount="57" Body="&lt;p&gt;While I am familiar with calculating information criteria for various ml models of a multiple sequence alignment, I am not aware of how to use the derived weights to create a tree that combines multiple models weighted accordingly.&lt;/p&gt;&#xA;&lt;p&gt;Thus, I have X trees, computed by multiple different models on the same alignment. Each has an associated BIC. BICs can be converted to weights. I wish to average the trees according to the BIC derived weights.&lt;/p&gt;&#xA;&lt;p&gt;I would prefer to be able to do this with R.&lt;/p&gt;&#xA;" OwnerUserId="16453" LastEditorUserId="3848" LastEditDate="2022-11-07T00:14:22.807" LastActivityDate="2024-04-02T17:01:25.193" Title="Using information criteria weights to create consensus tree" Tags="|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19967" PostTypeId="2" ParentId="19966" CreationDate="2022-11-07T00:24:24.637" Score="0" Body="&lt;p&gt;Calculating a consensus is easy, &lt;code&gt;RAxML&lt;/code&gt; has a tree consensus module, however calculating it according to weights is a rare procedure. &lt;code&gt;ETE3&lt;/code&gt; (Python) will have a consensus library and I assume in R it will be &lt;code&gt;ape&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The only way I can see this being done is to use the relative BIC as a duplicate factor of the number of trees of a given model and then perform a global tree consensus on the resulting treefile. Thus if Bayes Information Criterion top value is twice the size as the lower value the number of 'top value BIC trees' is double that of the lower value. That could be coded up to be cleaner. . Alternatively a multiplication factor could be introduced to ensure the number of duplicated trees within a scale of 100 - to give a cleaner more interpretable 'bootstrap' (but its not a bootstrap) score for each branch.&lt;/p&gt;&#xA;&lt;p&gt;Here the consensus frequency is a direct function of BIC, essentially you just have an alternative measure to the &amp;quot;non-parametric bootstraps&amp;quot;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I better finish this question. The solution to the problem is to produce separate trees for each model, provide a method of robustness, e.g. bootstrapping and then produce a single tree with all the bootstraps for each model along the branches. That is the formal approach if you're not going to simply select the model with highest probability, highest BIC in this case. This however is not the question.&lt;/p&gt;&#xA;&lt;p&gt;The problem with the solution above the line ... is the question. I was sort of being polite when I said 'rare procedure'. Weighting a consensus tree is not done in this context, weighting a locus or specific site - alignment position - can be done with good justification within the tree building method. The problems sort of multiple when it is weighted according to exact difference in BIC because the probability of a given branch being robust - which does use consensus is calculated differently. What you are doing is assigning shifts in the BIC/maximum likelihood not to the global topology but to specific incongruent branches. Thats not how it works. The BIC/maximum likelihood - whatever - is for the whole tree. The difference between models is the whole tree - every single branch change - not just a branch here and a branch there because its the score (probability ML) of a singular hypothesis. So a branch change here or there is a different hypothesis - for which you might not have generated a BIC score. The branch here and a branch there is MCMCMC consensus or bootstrap resampling consensus not precise shifts in BIC/maximum likelihood. You would need to dedicate the entire project to this specific method and even then it might be not accepted ... its not as a passing calculation.&lt;/p&gt;&#xA;&lt;p&gt;There could be biological situations where weighting a consensus tree is doable in the way described here. If a specialist reviewer saw the calculation they would refuse to permit publication because it violates the foundations of tree building theory. What you may not be aware is the long historic that has led to the current acceptance of modern methods and what you are proposing is dismantling that theory.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-11-09T13:50:23.733" LastActivityDate="2022-11-09T13:50:23.733" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19968" PostTypeId="2" ParentId="19949" CreationDate="2022-11-07T13:27:50.730" Score="1" Body="&lt;p&gt;Found it, it is an extended .bed file format, so not specific to bedtools output: &lt;a href=&quot;http://www.ensembl.org/info/website/upload/bed.html#optional&quot; rel=&quot;nofollow noreferrer&quot;&gt;source bed file output&lt;/a&gt;.&#xA;With 4 added columns to the end from bedtools -hist option:&#xA;&lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/coverage.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools -hist option&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16204" LastActivityDate="2022-11-07T13:27:50.730" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19969" PostTypeId="1" AcceptedAnswerId="19970" CreationDate="2022-11-07T14:30:09.273" Score="3" ViewCount="473" Body="&lt;p&gt;I want to filter a SNP, specifically &lt;code&gt;CHROM1:POS:630128&lt;/code&gt;. I tried to use &lt;code&gt;bcftools&lt;/code&gt; for this.&lt;/p&gt;&#xA;&lt;p&gt;I came up with the following variant of a bcftools call:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools filter -e &amp;quot;CHROM=1&amp;amp;POS=63018&amp;quot; /home/oem/Documents/GitHub/Genomics_prac_guide/trio/files/genome_file.vcf.gz &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which leads to:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error: cannot use arithmetic operators to compare strings and numbers&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and the call with unquoted parameters leads to&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools filter -e CHROM=1&amp;amp;POS=63018 /home/oem/Documents/GitHub/Genomics_prac_guide/trio/files/genome_file.vcf.gz &#xA;&#xA;sh: 1: /home/oem/Documents/GitHub/Genomics_prac_guide/trio/file/genome_file.vcf.gz: Permission denied&#xA;Failed to read from standard input: unknown file type&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Q1: Is bfctools filter the correct tool here to use?&lt;/li&gt;&#xA;&lt;li&gt;Q2: Regarding &lt;a href=&quot;https://github.com/samtools/bcftools/issues/1028#issuecomment-497651048&quot; rel=&quot;nofollow noreferrer&quot;&gt;this comment&lt;/a&gt; the first error might indicate that I have to annotate the CHROM &amp;amp; POS Tags in the header file. Is this correct?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;An example of the file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.2&#xA;##fileDate=20221104&#xA;##source=PLINKv1.90&#xA;##contig=&amp;lt;ID=1,length=248928329&amp;gt;&#xA;##contig=&amp;lt;ID=2,length=242157857&amp;gt;&#xA;##contig=&amp;lt;ID=3,length=198157342&amp;gt;&#xA;##contig=&amp;lt;ID=4,length=190106194&amp;gt;&#xA;##contig=&amp;lt;ID=5,length=181288140&amp;gt;&#xA;##contig=&amp;lt;ID=6,length=170595721&amp;gt;&#xA;##contig=&amp;lt;ID=7,length=159331484&amp;gt;&#xA;##contig=&amp;lt;ID=8,length=145075470&amp;gt;&#xA;##contig=&amp;lt;ID=9,length=138211490&amp;gt;&#xA;##contig=&amp;lt;ID=10,length=133659511&amp;gt;&#xA;##contig=&amp;lt;ID=11,length=135075227&amp;gt;&#xA;##contig=&amp;lt;ID=12,length=133261768&amp;gt;&#xA;##contig=&amp;lt;ID=13,length=114341522&amp;gt;&#xA;##contig=&amp;lt;ID=14,length=106873816&amp;gt;&#xA;##contig=&amp;lt;ID=15,length=101922277&amp;gt;&#xA;##contig=&amp;lt;ID=16,length=90172890&amp;gt;&#xA;##contig=&amp;lt;ID=17,length=83203771&amp;gt;&#xA;##contig=&amp;lt;ID=18,length=80252738&amp;gt;&#xA;##contig=&amp;lt;ID=19,length=58586567&amp;gt;&#xA;##contig=&amp;lt;ID=20,length=64323573&amp;gt;&#xA;##contig=&amp;lt;ID=21,length=46679699&amp;gt;&#xA;##contig=&amp;lt;ID=22,length=50776369&amp;gt;&#xA;##INFO=&amp;lt;ID=PR,Number=0,Type=Flag,Description=&amp;quot;Provisional reference allele, may not be based on real reference genome&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&amp;quot;Genotype&amp;quot;&amp;gt;&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  FAM001_ID001&#xA;1   69869   rs548049170 T   .   .   .   PR  GT  0/0&#xA;1   74792   rs13328684  N   .   .   .   PR  GT  ./.&#xA;1   630128  rs9283150   A   .   .   .   PR  GT  0/0&#xA;1   791532  i713426 N   .   .   .   PR  GT  ./.&#xA;1   792461  rs116587930 G   .   .   .   PR  GT  0/0&#xA;1   817341  rs3131972   G   .   .   .   PR  GT  0/0&#xA;1   818725  rs12184325  C   .   .   .   PR  GT  0/0&#xA;1   820888  rs12567639  A   .   .   .   PR  GT  0/0&#xA;1   823656  rs114525117 G   .   .   .   PR  GT  0/0&#xA;1   841166  rs12124819  G   A   .   .   PR  GT  0/1&#xA;1   858952  rs12127425  G   .   .   .   PR  GT  0/0&#xA;1   866156  rs79373928  T   .   .   .   PR  GT  0/0&#xA;1   880041  rs72888853  N   .   .   .   PR  GT  ./.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16454" LastEditorUserId="298" LastEditDate="2022-11-07T14:57:08.550" LastActivityDate="2022-11-07T14:57:08.550" Title="Filter with bcftools" Tags="|vcf|filtering|bcftools|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19970" PostTypeId="2" ParentId="19969" CreationDate="2022-11-07T14:56:26.803" Score="3" Body="&lt;p&gt;First of all, bcftools might be a bit of overkill if you want one specific variant. All you need is GNU &lt;code&gt;grep&lt;/code&gt; (the default on Linux):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zgrep -P '^(#|1\t630128\t)' file.vcf.gz &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or, with any &lt;code&gt;grep&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zgrep -E '^(#|1'$'\t''630128'$'\t'')' file.vcf.gz &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That will print all lines that start with a &lt;code&gt;#&lt;/code&gt; (the headers) or that start with &lt;code&gt;1&lt;/code&gt;, followed by a tab, then &lt;code&gt;630128&lt;/code&gt; and another tab.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, you can use &lt;code&gt;awk&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zcat file.vcf.gz |&#xA;    awk -F'\t' '/^#/ || ( $1==1 &amp;amp;&amp;amp; $2==630128)'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;All that said, yes, you can also do it with &lt;code&gt;bcftools&lt;/code&gt; and in most cases that is the right tool for the job so you might as well use it here too to get used to it. However, you need the right command and that's &lt;code&gt;view&lt;/code&gt;, not &lt;code&gt;filter&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view -r 1:630128 /home/oem/Documents/GitHub/Genomics_prac_guide/trio/files/genome_file.vcf.gz &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that the file needs to be indexed, so if you have not done so already, run&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tabix file.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Also, the reason your unquoted command failed as it did is because the &lt;code&gt;&amp;amp;&lt;/code&gt; has special meaning to the shell, it means &amp;quot;run this command in the background&amp;quot;. Therefore, when you ran &lt;code&gt;bcftools filter -e CHROM=1&amp;amp;POS=63018 file.vcf.gz&lt;/code&gt; that actually ran two separate commands:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools filter -e CHROM=1&amp;amp;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And then:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;POS=63018 file.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which means &amp;quot;set the variable &lt;code&gt;POS&lt;/code&gt; to the value &lt;code&gt;63018&lt;/code&gt; and execute the command &lt;code&gt;file.vcf.gz&lt;/code&gt;. Since the file isn't executable, you got:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sh: 1: file.vcf.gz: Permission denied&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2022-11-07T14:56:26.803" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19971" PostTypeId="1" CreationDate="2022-11-07T21:23:00.680" Score="2" ViewCount="38" Body="&lt;p&gt;Do you think it's possible to combine short read Illumina libraries (WGS) from multiple closely related eukaryotic microbial strains (e.g. libraries from a re-sequencing study, &amp;gt;99% ITS1 sequence) to boost read coverage during a de novo genome assembly run (e.g. with SPAdes)? My target genome is much larger than anticipated, and I'm wondering if increasing read coverage by an order of magnitude could improve length accuracy, contiguity, etc.&lt;/p&gt;&#xA;&lt;p&gt;Maybe differences in repetitive regions or transposable element movement prevent such an approach? Or maybe kmer frequencies are affected enough that de brujin graph construction gets messed up.&lt;/p&gt;&#xA;&lt;p&gt;There seem to be several pitfalls, but maybe benefits could outweigh the costs here. If anyone else has done this or considered it, I'd be interested in your thoughts. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="2907" LastEditorUserId="298" LastEditDate="2022-11-08T14:21:18.027" LastActivityDate="2022-11-08T14:21:18.027" Title="Using very closely related strains to increase coverage for short read, de novo assembly" Tags="|assembly|genomics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19972" PostTypeId="2" ParentId="19971" CreationDate="2022-11-08T05:07:06.277" Score="2" Body="&lt;p&gt;Given that the differences between your strains probably involve rearrangements and copy number variations, this approach seems inadvisable if your goal is a highly contiguous genome. You will make the dBG into more of a hairball if such differences exist.&lt;/p&gt;&#xA;&lt;p&gt;Illumina is quite cheap these days, I would vastly prefer simply sequencing more deeply if your issue in inadequate coverage. &lt;a href=&quot;https://support.illumina.com/downloads/sequencing_coverage_calculator.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Online tools exist&lt;/a&gt; for figuring out how to get there.&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, &lt;a href=&quot;https://www.illumina.com/Documents/products/technotes/technote_denovo_assembly_ecoli.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;you will swiftly plateau in terms of contiguity even for quite simple genomes using Illumina&lt;/a&gt;. So you can expect quickly diminishing returns.&lt;/p&gt;&#xA;&lt;p&gt;Knowing more stats on your assembly would help evaluate this. For example, if you are working with effectively 10X coverage with N50 ~ 500bp, I might be willing to look the other way while you co-assemble such a synthetic metagenome, because you will at least be able to assemble gene length contigs, which are unlikely to vary too dramatically between your strains. However, if the issue is that your assembly has an N50 ~ 10Kbp and you want it to be higher, then you should consider using other approaches to improve your assemblies, such as long reads. In a pinch, you could do something like optical maps or Hi-C, though I believe that is not preferred these days given much cheaper/better long reads. (Full disclosure: I used to work at a Hi-C company and still hold stock.)&lt;/p&gt;&#xA;&lt;p&gt;The field of metagenomics is based on the assumption that your suggested approach is ok to get a basic idea of e.g. gene complement, and if you are clever you can even &lt;a href=&quot;https://en.wikipedia.org/wiki/Binning_(metagenomics)&quot; rel=&quot;nofollow noreferrer&quot;&gt;deconvolute such mixtures into genomes&lt;/a&gt;, under the assumption that different genomes are at most distantly related. But that doesn't mean that it's a good idea to do it in silico by combining isolates.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-11-08T05:07:06.277" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19973" PostTypeId="2" ParentId="19964" CreationDate="2022-11-08T12:55:59.367" Score="1" Body="&lt;p&gt;I didn't even know STRING has the kind of analysis you are requesting, I don't see it on their webpage. In any case, the standard tool for GO or KEGG analyses, at least back when I was doing this sort of thing, is &lt;a href=&quot;https://david.ncifcrf.gov/tools.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;DAVID&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I have confirmed that it does have Phytophthora cactorum so all you need to do is upload your file there and use their tools.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2022-11-09T09:26:18.620" LastActivityDate="2022-11-09T09:26:18.620" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19974" PostTypeId="2" ParentId="19964" CreationDate="2022-11-08T13:57:45.427" Score="0" Body="&lt;p&gt;Just to confirm the OPs question&#xA;going to&#xA;&lt;a href=&quot;https://string-db.org/cgi/organisms&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://string-db.org/cgi/organisms&lt;/a&gt; and typing &amp;quot;Phytophthora cactorum&amp;quot;, then clicking 'random network' results in..&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://string-db.org/cgi/network?taskId=bbh7BVTwU0tH&amp;amp;sessionId=bA71KKVVotqM&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://string-db.org/cgi/network?taskId=bbh7BVTwU0tH&amp;amp;sessionId=bA71KKVVotqM&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I've encountered String for metabolic pathways, so I don't have direct experience with the OPs query.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;All databases are all limited by skews in underlying wet-lab work, e.g. there are some large taxonomic groups of bacteria that are complete blanks, i.e. no-one has ever worked on them. This however doesn't concern the OP because they are focussing on a single plant species (I think thats what it is, but could be wrong).&lt;/p&gt;&#xA;&lt;p&gt;The limit on uploading is understandable - these are not our computers and someone else pays for the OPs analysis.&lt;/p&gt;&#xA;&lt;p&gt;What the OP suggests is the '3 uploads' comprise a sampling bias and this too is very, very understandable regardless of what analysis is performed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Example&lt;/strong&gt; If you've loads of &lt;code&gt;COG&lt;/code&gt; 'V' type genes (disease resistance), for example, in one of the uploads, but not evenly mixed this will certainly skew the results.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Solution&lt;/strong&gt; The solution in this scenario is to randomly sample the genes amongst the 3 uploads. This is easy to do in Python via &lt;code&gt;random&lt;/code&gt; and &lt;code&gt;itertools&lt;/code&gt; libraries/packages . It would be easy to write a custom script. (separate question).&lt;/p&gt;&#xA;&lt;p&gt;Normally in a randomisation procedure there needs to be several, e.g. 10, iterations. So essential the OP would need to upload and perform the analysis 30 time comprising 3 groups of 10 replicates.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Example 2&lt;/strong&gt; Equally if the OP is honing a particularly metabolic pathway and the genes got split between the three uploads then visibility of the pathway would be partial.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Solution&lt;/strong&gt; The OP could cluster genes into one of the three uploads by membership to a given metabolic pathway. This is more difficult. This will depend on what the upstream annotation software/algorithm was because often the pathway membership is described in that annotation file. &lt;code&gt;string&lt;/code&gt; is a strong database for pathway membership.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Example 3&lt;/strong&gt; The skew is taxonomic membership, I don't this this counts here because it is a single species.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Ultimately, the preferred solution is to perform the analysis locally this way the OP will not need to break up their file. With KEGG the databases can be downloaded.&lt;/li&gt;&#xA;&lt;li&gt;The OP could contact &lt;code&gt;string&lt;/code&gt; essentially asking if they could supply an account that lifts the upload limit. &lt;code&gt;string&lt;/code&gt; would likely want a fee if they will do this at all. I honestly have no idea.&lt;/li&gt;&#xA;&lt;li&gt;There are other servers as @terdon recommends, I think BRITE is another one which might have the plant (which is what I think it is) under investigation.&lt;/li&gt;&#xA;&lt;li&gt;The OP could go through solutions 1 and 2 above. Solution 1 is likely to be 'the solution', but there needs to be a supporting script written (separate question).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;APIs&lt;/strong&gt; I don't know about &lt;code&gt;string&lt;/code&gt; but if there was an API this would enable a multiple replicate analysis to be coded without all the manual point and click that it would otherwise involve.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;My opinion of &lt;code&gt;Go&lt;/code&gt; is that it is sparsely populated, just saying.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-11-08T20:52:19.333" LastActivityDate="2022-11-08T20:52:19.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19977" PostTypeId="1" AcceptedAnswerId="19996" CreationDate="2022-11-08T22:21:49.503" Score="1" ViewCount="459" Body="&lt;p&gt;I lifted some unmerged trio files from hg19 to hg38 using &lt;a href=&quot;https://github.com/sritchie73/liftOverPlink&quot; rel=&quot;nofollow noreferrer&quot;&gt;liftOverPlink&lt;/a&gt;. It looks like that several houndred allels on each chromosome of one file shifted/switched to non-standard alleles. I recorded some of the differences. And will add them to the post. I will also add a sample of the file to this post. My goal is to merge the lifted trio files to one singe file.&lt;/p&gt;&#xA;&lt;p&gt;My command to merge is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools  merge -l merge.txt --force-samples -Ob -o file.bcf&#xA;&#xA;&#xA;The REF prefixes differ: C vs N (1,1)&#xA;Failed to merge alleles at 1:3940550 in file_1.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;First, I tried to remove those alleles in question but I realized that these are just to many to do this by hand.&lt;/p&gt;&#xA;&lt;p&gt;I used R to edit the file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;file &amp;lt;- read_delim(&#xA;  gzfile(&#xA;    str_glue(&amp;quot;{some_path}file.vcf.gz&amp;quot;)),&#xA;  delim=&amp;quot;|&amp;quot;,  quote='', escape_backslash = T, col_names = F) %&amp;gt;%&#xA;#A list of all the REF prefix differences: number equals position in chromsome 1&#xA;  #630128 N vs A, 791532, N vs A, 841166 G vs A, 937572 C vs T, 1110226 N vs A, 1486611 N vs G, 1554290 C vs G,&#xA;  #1654663 N vs T, 1671724 C vs T, 1755523 C vs T, 1774697 C vs T, 1916092 C vs T, 1949922 C vs T, 1959595 C vs n&#xA;  #1959717 C vs T, 1983048 A vs G, 1991690 A vs G, 1999315 C vs T 2096621 C vs T 2107385 A vs G, 2124676 A vs G&#xA;  #2274142 A vs C, 2284952 A vs G, 2293812 C vs T, 2303949 N vs C, 2321888 T vs N, 2351827 C vs T,2364905 N vs G&#xA;  #2364905 T vs C, 2420707 T vs C, 2431388 T vs C, 2441501 T vs C, 2461209 C vs G, 2614790 A vs G, 2620461 G vs A&#xA;  #2622185 C vs T, 2792599 A vs C, 2861494 C vs T 2864702 G vs N, 2884008 T vs C , 2894837 A vs C, 2925918 T vs C,&#xA;  #2944366 T vs C, 3097126 G vs A,, 3107701 C vs A, 3109422  C vs A, 3112423 G vs A, , 3119756 C vs T, 3146311 G vs T&#xA;  #3282557 G vs A,, 3294386 A vs G, 3294894 A vs C, 3303337 A vs G, 3320610 T vs C, 3335540 T vs C, 3344952 T vs C&#xA;  #3345118 T vs C 3355977 G vs A, 3401871 G vs A, 3443779 G vs T,, 3534441 T vs N, 3591297 A vs G,&#xA;  #3595370  A vs G, 3601923 T vs C, 3624792 G vs N, 3689917 T vs C, 3689917 G vs A, 3695328 G vs A, 3696503 G vs A&#xA;  #3762340 T vs C, 3775163 A vs C, 3870048 T vs Cl 3924314 G vs A3935847 T vs C, 3940550 G vs A&#xA;   filter(!grepl('.+630128.+|.+791532.+|.+841166.+|.+937572.+|.+1110226.+|.+1486611.+|.+1554290.+|.+1654663.+|.+1671724.+|.+1755523.+|.+1774697.+|.+1916092.+|.+1949922.+|.+1959595.+|.+1959717.+|.+1983048.+|.+1991690.+|.+1999315.+|.+2096621.+|.+2107385.+|.+2124676.+|.+2274142.+|.+2284952.+|.+2293812.+|.+2303949.+|.+2321888.+|.+2324778.+|.+2351678.+|.+2351827.+|.+2364905.+|.+2364905.+|.+2420707.+|.+2431388.+|.+2441501.+|.+2461209.+|.+2614790.+|.+2620461.+|.+2622185.+|.+2792599.+|.+2861494.+|.+2864702.+|.+2884008.+|.+2894837.+|.+2925918.+|.+2944366.+|.+3097126.+|.+3107701.+|.+3109422.+|.+3112423.+|.+3119756.+|.+3146311.+|.+3282557.+|.+3294386.+|.+3294894.+|.+3303337.+|.+3320610.+|.+3335540.+|.+3344952.+|.+3345118.+|.+3355977.+|.+3401871.+|.+3443779.+|.+3534441.+|.+3591297.+|.+3595370.+|.+3601923.+|.+3624792.+|.+3689917.+|.+3689917.+|.+3695328.+|.+3696503.+|.+3762340.+|.+3775163.+|.+3870048.+|.+3924314.+|.+3935847.+|.+3940550.+',X1)) %&amp;gt;%&#xA;  write_delim(file = str_glue(&amp;quot;{some_path}some_file_v1.vcf&amp;quot;),&#xA;                                 k                        col_names = F, quote = 'none', escape = 'none')&#xA;&#xA;#zip the file&#xA;call = str_glue(&amp;quot;bcftools view {some_path}some_file_v1.vcf -o {some_path}some_file_v1.vcf.gz&amp;quot;)&#xA;system(call)&#xA;#index the new file&#xA;call = str_glue(&amp;quot;bcftools index -f {f}.gz&amp;quot;)&#xA;system(call)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Q1: How can one fix the large amount of REF prefix differences in the chromosome files?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Q2: If one uses the fixref plugin of bcftools which is the correct reference file to use after lifting was performed?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; JRodrigoF recommended to use the &lt;code&gt;bcftools norm --check-ref s --fasta-ref GRCh38_full_analysis_set_plus_decoy_hla.fa &lt;/code&gt; to fix the issue. It seems this produces the following error on my end:&#xA;&lt;code&gt; [E::faidx_adjust_position] The sequence &amp;quot;1&amp;quot; was not found faidx_fetch_seq failed at 1:69869&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;To make sure this is not a problem created during liftover I will post my liftover procedure here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse)&#xA;&#xA;&#xA;#set working directory to data directory&#xA;trio_wd &amp;lt;- str_glue(here::here(),'/trio/SomeName/')&#xA;&#xA;#create file list for raw data&#xA;file_list &amp;lt;- str_c(trio_wd,dir(trio_wd)) %&amp;gt;% str_extract('.+txt') %&amp;gt;%&#xA;  {.[!is.na(.)]}&#xA;&#xA;&#xA;    #generate map files from 23andme text file&#xA;    for(f in file_list){&#xA;      call = str_glue('plink --23file {f} --recode --out {str_remove(f,&amp;quot;.txt&amp;quot;)}&#xA;                      ')&#xA;      system(call)&#xA;    }&#xA;    &#xA;    #create file list for map data&#xA;    file_list &amp;lt;- str_c(trio_wd,dir(trio_wd)) %&amp;gt;% str_extract('.+map') %&amp;gt;%&#xA;      {.[!is.na(.)]}&#xA;    &#xA;    for (f in file_list){&#xA;    call &amp;lt;- str_glue('python3 reference/liftOverPlink.py --map {f} --out {str_replace(f,&amp;quot;.map&amp;quot;,&amp;quot;_lifted&amp;quot;)} --chain reference/hg19ToHg38.over.chain.gz  ')&#xA;    system(call)&#xA;    &#xA;    #create valid mapping file for hg19&#xA;    call &amp;lt;- str_glue('python3 reference/rmBadLifts.py -m {str_replace(f,&amp;quot;.map&amp;quot;,&amp;quot;_lifted&amp;quot;)}.map --out {str_replace(f,&amp;quot;.map&amp;quot;,&amp;quot;_good_lifted&amp;quot;)}.map -l {str_replace(f,&amp;quot;.map&amp;quot;,&amp;quot;_bad_lifted.dat&amp;quot;)}')&#xA;    system(call)&#xA;    &#xA;    call &amp;lt;- str_glue('cut -f 2 {str_replace(f,&amp;quot;.map&amp;quot;,&amp;quot;_bad_lifted.dat&amp;quot;)} &amp;gt; {str_replace(f,&amp;quot;.map&amp;quot;,&amp;quot;_exclude_lifted.dat&amp;quot;)}')&#xA;    system(call)&#xA;    call &amp;lt;- str_glue('cut -f 4 {str_replace(f,&amp;quot;.map&amp;quot;,&amp;quot;_lifted&amp;quot;)}.bed.unlifted | sed &amp;quot;/^#/d&amp;quot; &amp;gt;&amp;gt; {str_replace(f,&amp;quot;.map&amp;quot;,&amp;quot;_exclude_lifted.dat&amp;quot;)} ')&#xA;    system(call)&#xA;    # Note: this will clobber the lifted MAP file generated by `liftOverPlink`:&#xA;    call &amp;lt;- str_glue('plink --file {str_remove(f,&amp;quot;.map&amp;quot;)} --recode --out {str_replace(f,&amp;quot;.map&amp;quot;,&amp;quot;_lifted&amp;quot;)} --exclude {str_replace(f,&amp;quot;.map&amp;quot;,&amp;quot;_exclude_lifted.dat&amp;quot;)} ')&#xA;    system(call)&#xA;    call &amp;lt;- str_glue('plink --ped {str_replace(f,&amp;quot;.map&amp;quot;,&amp;quot;_lifted.ped&amp;quot;)} --map {str_replace(f,&amp;quot;.map&amp;quot;,&amp;quot;_good_lifted&amp;quot;)}.map --recode --out {str_replace(f,&amp;quot;.map&amp;quot;,&amp;quot;_final&amp;quot;)}')&#xA;    system(call)&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT II:&lt;/strong&gt;&#xA;Sample of the file including header&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.2&#xA;##FILTER=&amp;lt;ID=PASS,Description=&amp;quot;All filters passed&amp;quot;&amp;gt;&#xA;##fileDate=20221108&#xA;##source=PLINKv1.90&#xA;##contig=&amp;lt;ID=1,length=248928329&amp;gt;&#xA;##contig=&amp;lt;ID=2,length=242157857&amp;gt;&#xA;##contig=&amp;lt;ID=3,length=198157342&amp;gt;&#xA;##contig=&amp;lt;ID=4,length=190106194&amp;gt;&#xA;##contig=&amp;lt;ID=5,length=181288140&amp;gt;&#xA;##contig=&amp;lt;ID=6,length=170595721&amp;gt;&#xA;##contig=&amp;lt;ID=7,length=159331484&amp;gt;&#xA;##contig=&amp;lt;ID=8,length=145075470&amp;gt;&#xA;##contig=&amp;lt;ID=9,length=138211490&amp;gt;&#xA;##contig=&amp;lt;ID=10,length=133659511&amp;gt;&#xA;##contig=&amp;lt;ID=11,length=135075227&amp;gt;&#xA;##contig=&amp;lt;ID=12,length=133261768&amp;gt;&#xA;##contig=&amp;lt;ID=13,length=114341522&amp;gt;&#xA;##contig=&amp;lt;ID=14,length=106873816&amp;gt;&#xA;##contig=&amp;lt;ID=15,length=101922277&amp;gt;&#xA;##contig=&amp;lt;ID=16,length=90172890&amp;gt;&#xA;##contig=&amp;lt;ID=17,length=83203771&amp;gt;&#xA;##contig=&amp;lt;ID=18,length=80252738&amp;gt;&#xA;##contig=&amp;lt;ID=19,length=58586567&amp;gt;&#xA;##contig=&amp;lt;ID=20,length=64323573&amp;gt;&#xA;##contig=&amp;lt;ID=21,length=46679699&amp;gt;&#xA;##contig=&amp;lt;ID=22,length=50776369&amp;gt;&#xA;##INFO=&amp;lt;ID=PR,Number=0,Type=Flag,Description=&amp;quot;Provisional reference allele, may not be based on real reference genome&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&amp;quot;Genotype&amp;quot;&amp;gt;&#xA;##bcftools_viewVersion=1.13+htslib-1.13+ds&#xA;##bcftools_viewCommand=view -r 1 -o /home/oem/Documents/GitHub/Genomics_prac_guide/trio/some_name/some_name//genome_some_name_v5_Full_20191220113716_final_1.vcf.gz /home/oem/Documents/GitHub/Genomics_prac_guide/trio/some_name/some_name_v5_Full_20191220113716_final.vcf.gz; Date=Tue Nov  8 13:02:01 2022&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  FAM001_ID001&#xA;1   69869   rs548049170 T   .   .   .   PR  GT  0/0&#xA;1   74792   rs13328684  N   .   .   .   PR  GT  ./.&#xA;1   792461  rs116587930 G   .   .   .   PR  GT  0/0&#xA;1   817341  rs3131972   G   .   .   .   PR  GT  0/0&#xA;1   818725  rs12184325  C   .   .   .   PR  GT  0/0&#xA;1   820888  rs12567639  A   .   .   .   PR  GT  0/0&#xA;1   823656  rs114525117 G   .   .   .   PR  GT  0/0&#xA;1   858952  rs12127425  G   .   .   .   PR  GT  0/0&#xA;1   866156  rs79373928  T   .   .   .   PR  GT  0/0&#xA;1   880041  rs72888853  N   .   .   .   PR  GT  ./.&#xA;1   889018  rs7538305   A   .   .   .   PR  GT  0/0&#xA;1   894801  rs28444699  A   .   .   .   PR  GT  0/0&#xA;1   895351  i713449 N   .   .   .   PR  GT  ./.&#xA;1   899450  rs116452738 G   .   .   .   PR  GT  0/0&#xA;1   899712  rs72631887  T   .   .   .   PR  GT  0/0&#xA;1   903285  rs28678693  T   C   .   .   PR  GT  0/1&#xA;1   905373  rs4970382   T   C   .   .   PR  GT  0/1&#xA;1   911428  rs4475691   C   .   .   .   PR  GT  0/0&#xA;1   916010  rs72631889  G   .   .   .   PR  GT  0/0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT III:&lt;/strong&gt;&#xA;Output of &lt;code&gt;grep '^&amp;gt;' reference/GRCh38_full_analysis_set_plus_decoy_hla.fa&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;chr1  AC:CM000663.2  gi:568336023  LN:248956422  rl:Chromosome  M5:6aef897c3d6ff0c78aff06ac189178dd  AS:GRCh38&#xA;&amp;gt;chr2  AC:CM000664.2  gi:568336022  LN:242193529  rl:Chromosome  M5:f98db672eb0993dcfdabafe2a882905c  AS:GRCh38&#xA;&amp;gt;chr3  AC:CM000665.2  gi:568336021  LN:198295559  rl:Chromosome  M5:76635a41ea913a405ded820447d067b0  AS:GRCh38&#xA;&amp;gt;chr4  AC:CM000666.2  gi:568336020  LN:190214555  rl:Chromosome  M5:3210fecf1eb92d5489da4346b3fddc6e  AS:GRCh38&#xA;&amp;gt;chr5  AC:CM000667.2  gi:568336019  LN:181538259  rl:Chromosome  M5:a811b3dc9fe66af729dc0dddf7fa4f13  AS:GRCh38  hm:47309185-49591369&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT IV:&lt;/strong&gt; Edited the fasta-file and the corresponding index file as mentioned in the comments. Still, the error from EDIT I sustains.&lt;/p&gt;&#xA;&lt;p&gt;First five chromosomes of the modified FASTA-file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep '^&amp;gt;' reference/GRCh38_full_analysis_set_plus_decoy_hla_modified.fa&#xA;&amp;gt;1  AC:CM000663.2  gi:568336023  LN:248956422  rl:Chromosome  M5:6aef897c3d6ff0c78aff06ac189178dd  AS:GRCh38&#xA;&amp;gt;2  AC:CM000664.2  gi:568336022  LN:242193529  rl:Chromosome  M5:f98db672eb0993dcfdabafe2a882905c  AS:GRCh38&#xA;&amp;gt;3  AC:CM000665.2  gi:568336021  LN:198295559  rl:Chromosome  M5:76635a41ea913a405ded820447d067b0  AS:GRCh38&#xA;&amp;gt;4  AC:CM000666.2  gi:568336020  LN:190214555  rl:Chromosome  M5:3210fecf1eb92d5489da4346b3fddc6e  AS:GRCh38&#xA;&amp;gt;5  AC:CM000667.2  gi:568336019  LN:181538259  rl:Chromosome  M5:a811b3dc9fe66af729dc0dddf7fa4f13  AS:GRCh38  hm:47309185-49591369&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Addjusted the &lt;code&gt;bcftools&lt;/code&gt;-call of EDIT I afterwards so the modified FASTA-file will be used&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;call = str_glue('bcftools  norm --check-ref s {f} --fasta-ref reference/GRCh38_full_analysis_set_plus_decoy_hla_modified.fa -o {out} ')&#xA;[E::faidx_adjust_position] The sequence &amp;quot;1&amp;quot; was not found&#xA;faidx_fetch_seq failed at 1:69869&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;View region in question&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.2&#xA;##FILTER=&amp;lt;ID=PASS,Description=&amp;quot;All filters passed&amp;quot;&amp;gt;&#xA;##fileDate=20221108&#xA;##source=PLINKv1.90&#xA;##contig=&amp;lt;ID=1,length=248928329&amp;gt;&#xA;##contig=&amp;lt;ID=2,length=242157857&amp;gt;&#xA;##contig=&amp;lt;ID=3,length=198157342&amp;gt;&#xA;##contig=&amp;lt;ID=4,length=190106194&amp;gt;&#xA;##contig=&amp;lt;ID=5,length=181288140&amp;gt;&#xA;##contig=&amp;lt;ID=6,length=170595721&amp;gt;&#xA;##contig=&amp;lt;ID=7,length=159331484&amp;gt;&#xA;##contig=&amp;lt;ID=8,length=145075470&amp;gt;&#xA;##contig=&amp;lt;ID=9,length=138211490&amp;gt;&#xA;##contig=&amp;lt;ID=10,length=133659511&amp;gt;&#xA;##contig=&amp;lt;ID=11,length=135075227&amp;gt;&#xA;##contig=&amp;lt;ID=12,length=133261768&amp;gt;&#xA;##contig=&amp;lt;ID=13,length=114341522&amp;gt;&#xA;##contig=&amp;lt;ID=14,length=106873816&amp;gt;&#xA;##contig=&amp;lt;ID=15,length=101922277&amp;gt;&#xA;##contig=&amp;lt;ID=16,length=90172890&amp;gt;&#xA;##contig=&amp;lt;ID=17,length=83203771&amp;gt;&#xA;##contig=&amp;lt;ID=18,length=80252738&amp;gt;&#xA;##contig=&amp;lt;ID=19,length=58586567&amp;gt;&#xA;##contig=&amp;lt;ID=20,length=64323573&amp;gt;&#xA;##contig=&amp;lt;ID=21,length=46679699&amp;gt;&#xA;##contig=&amp;lt;ID=22,length=50776369&amp;gt;&#xA;##INFO=&amp;lt;ID=PR,Number=0,Type=Flag,Description=&amp;quot;Provisional reference allele, may not be based on real reference genome&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&amp;quot;Genotype&amp;quot;&amp;gt;&#xA;##bcftools_viewVersion=1.13+htslib-1.13+ds&#xA;##bcftools_viewCommand=view -r 1 -o /home/oem/Documents/GitHub/Genomics_prac_guide/trio/Some_name/Some_name//genome_Some_name_v5_Full_20191220113716_final_1.vcf.gz /home/oem/Documents/GitHub/Genomics_prac_guide/trio/Some_name/genome_Some_name_v5_Full_20191220113716_final.vcf.gz; Date=Tue Nov  8 13:02:01 2022&#xA;##bcftools_viewCommand=view -o /home/oem/Documents/GitHub/Genomics_prac_guide/trio/Some_name/Some_name/genome_Some_name_v5_Full_20191220113716_final_1_1.vcf.gz /home/oem/Documents/GitHub/Genomics_prac_guide/trio/Some_name/Some_name/genome_Some_name_v5_Full_20191220113716_final_1_1.vcf; Date=Tue Nov  8 18:16:55 2022&#xA;##bcftools_viewCommand=view -r 1:69869 /home/oem/Documents/GitHub/Genomics_prac_guide/trio/Some_name/Some_name/genome_Some_name_v5_Full_20191220113716_final_1_1.vcf.gz; Date=Thu Nov 10 10:56:54 2022&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  FAM001_ID001&#xA;1   69869   rs548049170 T   .   .   .   PR  GT  0/0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16454" LastEditorUserId="16454" LastEditDate="2022-11-10T10:05:31.810" LastActivityDate="2022-11-11T09:31:26.577" Title="Large amount of REF prefix differences after lifting" Tags="|bcftools|liftover|merge|" AnswerCount="2" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="19978" PostTypeId="1" AcceptedAnswerId="19981" CreationDate="2022-11-09T00:36:19.403" Score="1" ViewCount="89" Body="&lt;p&gt;I have a long list of JSON data, with repeats of contents similar to followings.&lt;/p&gt;&#xA;&lt;p&gt;Due to the original JSON file is too long, I will just shared the hyperlinks here. This is a result generated from a database called RegulomeDB.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://regulomedb.org/regulome-search/?regions=chr16:28539847-28539848&amp;amp;genome=GRCh37&amp;amp;format=json&quot; rel=&quot;nofollow noreferrer&quot;&gt;Direct link to the JSON file&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would like to extract specific data (eQTLs) from &lt;code&gt;&amp;quot;method&amp;quot;: &amp;quot;eQTLs&amp;quot;&lt;/code&gt; and &lt;code&gt;&amp;quot;value&amp;quot;: &amp;quot;xxxx&amp;quot;&lt;/code&gt;, and put them into 2 columns (tab delimited) exactly like below.&#xA;Note: &lt;code&gt;&amp;quot;value&amp;quot;:&amp;quot;xxxx&amp;quot;&lt;/code&gt; is extracted right after &lt;code&gt;&amp;quot;method&amp;quot;: &amp;quot;eQTLs&amp;quot;&lt;/code&gt;is detected.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;eQTLs   firstResult, secondResult, thirdResult, ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In this example, the desired output is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;eQTLs   EIF3S8, EIF3CL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've tried using a python script but was unsuccessful.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import json&#xA;with open('file.json') as f:&#xA;    f_json = json.load(f)&#xA;    print 'f_json[0]['&amp;quot;method&amp;quot;: &amp;quot;eQTLs&amp;quot;'] + &amp;quot;\t&amp;quot; + f_json[0][&amp;quot;value&amp;quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thank you for your kind help.&lt;/p&gt;&#xA;" OwnerUserId="16470" LastEditorUserId="16470" LastEditDate="2022-11-09T09:49:47.303" LastActivityDate="2022-11-10T10:10:23.450" Title="eQTL data extraction from database JSON file" Tags="|python|bash|text-processing|data-retrieval|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19979" PostTypeId="2" ParentId="19978" CreationDate="2022-11-09T07:57:19.360" Score="0" Body="&lt;p&gt;It's a little hard to know what your goal is here based on the sample code, but I do find that your JSON snippet does not match any spec. I was able to edit it to add brackets to make the entries elements of a list:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[&#xA;  {&#xA;  &amp;quot;biosample_ontology&amp;quot;: {&#xA;    &amp;quot;@id&amp;quot;: &amp;quot;https://www.encodeproject.org/biosample-types/primary_cell_CL_0000576/&amp;quot;,&#xA;    &amp;quot;@type&amp;quot;: [&#xA;      &amp;quot;BiosampleType&amp;quot;,&#xA;      &amp;quot;Item&amp;quot;&#xA;    ],&#xA;    &amp;quot;aliases&amp;quot;: [],&#xA;    &amp;quot;cell_slims&amp;quot;: [&#xA;      &amp;quot;hematopoietic cell&amp;quot;,&#xA;      &amp;quot;monocyte&amp;quot;,&#xA;      &amp;quot;myeloid cell&amp;quot;,&#xA;      &amp;quot;leukocyte&amp;quot;&#xA;    ],&#xA;    &amp;quot;classification&amp;quot;: &amp;quot;primary cell&amp;quot;,&#xA;    &amp;quot;dbxrefs&amp;quot;: [],&#xA;    &amp;quot;developmental_slims&amp;quot;: [&#xA;      &amp;quot;mesoderm&amp;quot;&#xA;    ],&#xA;    &amp;quot;name&amp;quot;: &amp;quot;primary_cell_CL_0000576&amp;quot;,&#xA;    &amp;quot;organ_slims&amp;quot;: [&#xA;      &amp;quot;blood&amp;quot;,&#xA;      &amp;quot;bodily fluid&amp;quot;&#xA;    ],&#xA;    &amp;quot;references&amp;quot;: [],&#xA;    &amp;quot;schema_version&amp;quot;: &amp;quot;1&amp;quot;,&#xA;    &amp;quot;status&amp;quot;: &amp;quot;released&amp;quot;,&#xA;    &amp;quot;synonyms&amp;quot;: [],&#xA;    &amp;quot;system_slims&amp;quot;: [&#xA;      &amp;quot;immune system&amp;quot;&#xA;    ],&#xA;    &amp;quot;term_id&amp;quot;: &amp;quot;CL:0000576&amp;quot;,&#xA;    &amp;quot;term_name&amp;quot;: &amp;quot;monocyte&amp;quot;,&#xA;    &amp;quot;uuid&amp;quot;: &amp;quot;20685264-1187-4c5c-b4ce-dc85ea53fc11&amp;quot;&#xA;  },&#xA;  &amp;quot;chrom&amp;quot;: &amp;quot;chr11&amp;quot;,&#xA;  &amp;quot;dataset&amp;quot;: &amp;quot;https://www.encodeproject.org/annotations/ENCSR464XOM/&amp;quot;,&#xA;  &amp;quot;dataset_rel&amp;quot;: &amp;quot;/annotations/ENCSR464XOM/&amp;quot;,&#xA;  &amp;quot;documents&amp;quot;: [],&#xA;  &amp;quot;end&amp;quot;: 59936979,&#xA;  &amp;quot;file&amp;quot;: &amp;quot;ENCFF814NRI&amp;quot;,&#xA;  &amp;quot;method&amp;quot;: &amp;quot;eQTLs&amp;quot;,&#xA;  &amp;quot;start&amp;quot;: 59936978,&#xA;  &amp;quot;strand&amp;quot;: null,&#xA;  &amp;quot;targets&amp;quot;: [],&#xA;  &amp;quot;value&amp;quot;: &amp;quot;MS4A4A&amp;quot;&#xA;},&#xA;&#xA;    {&#xA;  &amp;quot;biosample_ontology&amp;quot;: {&#xA;    &amp;quot;@id&amp;quot;: &amp;quot;https://www.encodeproject.org/biosample-types/primary_cell_CL_0000576/&amp;quot;,&#xA;    &amp;quot;@type&amp;quot;: [&#xA;      &amp;quot;BiosampleType&amp;quot;,&#xA;      &amp;quot;Item&amp;quot;&#xA;    ],&#xA;    &amp;quot;aliases&amp;quot;: [],&#xA;    &amp;quot;cell_slims&amp;quot;: [&#xA;      &amp;quot;hematopoietic cell&amp;quot;,&#xA;      &amp;quot;monocyte&amp;quot;,&#xA;      &amp;quot;myeloid cell&amp;quot;,&#xA;      &amp;quot;leukocyte&amp;quot;&#xA;    ],&#xA;    &amp;quot;classification&amp;quot;: &amp;quot;primary cell&amp;quot;,&#xA;    &amp;quot;dbxrefs&amp;quot;: [],&#xA;    &amp;quot;developmental_slims&amp;quot;: [&#xA;      &amp;quot;mesoderm&amp;quot;&#xA;    ],&#xA;    &amp;quot;name&amp;quot;: &amp;quot;primary_cell_CL_0000576&amp;quot;,&#xA;    &amp;quot;organ_slims&amp;quot;: [&#xA;      &amp;quot;blood&amp;quot;,&#xA;      &amp;quot;bodily fluid&amp;quot;&#xA;    ],&#xA;    &amp;quot;references&amp;quot;: [],&#xA;    &amp;quot;schema_version&amp;quot;: &amp;quot;1&amp;quot;,&#xA;    &amp;quot;status&amp;quot;: &amp;quot;released&amp;quot;,&#xA;    &amp;quot;synonyms&amp;quot;: [],&#xA;    &amp;quot;system_slims&amp;quot;: [&#xA;      &amp;quot;immune system&amp;quot;&#xA;    ],&#xA;    &amp;quot;term_id&amp;quot;: &amp;quot;CL:0000576&amp;quot;,&#xA;    &amp;quot;term_name&amp;quot;: &amp;quot;monocyte&amp;quot;,&#xA;    &amp;quot;uuid&amp;quot;: &amp;quot;20685264-1187-4c5c-b4ce-dc85ea53fc11&amp;quot;&#xA;  },&#xA;  &amp;quot;chrom&amp;quot;: &amp;quot;chr11&amp;quot;,&#xA;  &amp;quot;dataset&amp;quot;: &amp;quot;https://www.encodeproject.org/annotations/ENCSR464XOM/&amp;quot;,&#xA;  &amp;quot;dataset_rel&amp;quot;: &amp;quot;/annotations/ENCSR464XOM/&amp;quot;,&#xA;  &amp;quot;documents&amp;quot;: [],&#xA;  &amp;quot;end&amp;quot;: 59936979,&#xA;  &amp;quot;file&amp;quot;: &amp;quot;ENCFF814NRI&amp;quot;,&#xA;  &amp;quot;method&amp;quot;: &amp;quot;eQTLs&amp;quot;,&#xA;  &amp;quot;start&amp;quot;: 59936979,&#xA;  &amp;quot;strand&amp;quot;: null,&#xA;  &amp;quot;targets&amp;quot;: [],&#xA;  &amp;quot;value&amp;quot;: &amp;quot;MS4A4B&amp;quot;&#xA;}&#xA;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I did this, I was able to slightly modify your code to print out each element's &lt;code&gt;method&lt;/code&gt; key, value pair:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import json&#xA;with open('/Users/max.press/Downloads/file.json') as f:&#xA;    f_json = json.load(f)&#xA;    for i in range(len(f_json)):&#xA;        print(f_json[i][&amp;quot;method&amp;quot;] + &amp;quot;\t&amp;quot; + f_json[i][&amp;quot;value&amp;quot;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;% python ~/Downloads/readfile.py&#xA;# prints this:&#xA;eQTLs   MS4A4A&#xA;eQTLs   MS4A4B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-11-09T07:57:19.360" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19980" PostTypeId="1" CreationDate="2022-11-09T11:06:35.150" Score="2" ViewCount="100" Body="&lt;p&gt;I am seeking statistical support for the following bacterial growth time series.&lt;/p&gt;&#xA;&lt;p&gt;In the lab, we have cell cultures growing in bioreactors. Every day we measure the concentration in &lt;em&gt;m&lt;/em&gt;mol&lt;sup&gt;-1&lt;/sup&gt; of a set of metabolites and amino acids in the media with a cell culture analyser.&lt;/p&gt;&#xA;&lt;p&gt;Some metabolites are consumed by the cells and some are produced during the time-course of the culture. Along with this, we also count the number of cells every day to calculate the cell growth dynamics.&lt;/p&gt;&#xA;&lt;p&gt;We wish to assess the relationship between the consumption or production of any of these compounds and cell growth. For example, if the consumption of a specific metabolite from the media has a significant effect on increasing the cell count and would assist media formulation.&lt;/p&gt;&#xA;&lt;p&gt;One measurement per day is taken over four consecutive days from multiple bioreactors.&lt;/p&gt;&#xA;&lt;p&gt;What kind of analysis is would be appropriate?&lt;/p&gt;&#xA;" OwnerUserId="15700" LastEditorUserId="3848" LastEditDate="2022-11-12T12:21:31.107" LastActivityDate="2022-12-12T13:09:16.097" Title="Multivariate analysis of cell culture bioprocess" Tags="|statistics|machine-learning|bacteria|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19981" PostTypeId="2" ParentId="19978" CreationDate="2022-11-09T11:41:41.097" Score="3" Body="&lt;p&gt;Here a &lt;code&gt;jq&lt;/code&gt; solution if you can use it:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;jq -r --arg f &amp;quot;eQTLs&amp;quot; '$f + &amp;quot;\t&amp;quot; + ([.&amp;quot;@graph&amp;quot;[] | select(.method == $f) | .value] | join(&amp;quot;, &amp;quot;))' file.json&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Explanations:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;-r&lt;/code&gt;: Raw string result (so result string will be without between &lt;code&gt;&amp;quot;&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;--arg name value&lt;/code&gt;: Your filter on &lt;code&gt;eQTLs&lt;/code&gt; method value&lt;/li&gt;&#xA;&lt;li&gt;Search in &lt;code&gt;@graph&lt;/code&gt; array elements with &lt;code&gt;[]&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Select only array element with &lt;code&gt;method&lt;/code&gt; attribute equals to &lt;code&gt;eQTLs&lt;/code&gt; (in &lt;code&gt;$f&lt;/code&gt; argument)&lt;/li&gt;&#xA;&lt;li&gt;The result, 2 in this case, are put in a array &lt;code&gt;[...]&lt;/code&gt; and join with string &lt;code&gt;, &lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;$f&lt;/code&gt; string, a tabulation &lt;code&gt;&amp;quot;\t&amp;quot;&lt;/code&gt; and the join result are concatenated to one string with &lt;code&gt;+&lt;/code&gt; operators&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Result:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;eQTLs   EIF3S8, EIF3CL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16471" LastEditorUserId="16471" LastEditDate="2022-11-09T20:37:40.473" LastActivityDate="2022-11-09T20:37:40.473" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19982" PostTypeId="2" ParentId="19956" CreationDate="2022-11-09T12:05:21.980" Score="0" Body="&lt;p&gt;Hi An Ignorant Wanderer,&lt;/p&gt;&#xA;&lt;p&gt;Filters allow you to narrow down the dataset to query a specific subset of features within your dataset. A description and tutorial of the filtering steps within BioMart can be found here:&#xA;&lt;a href=&quot;https://www.ensembl.org/info/data/biomart/how_to_use_biomart.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ensembl.org/info/data/biomart/how_to_use_biomart.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you remove filters, you will query the full dataset (all human genes, in the example above). However, this is not recommended for BioMart as large, genome-wide queries will overload the BioMart servers. For queries of this size, we suggest using the Ensembl REST API or FTP site.&lt;/p&gt;&#xA;" OwnerUserId="6500" LastActivityDate="2022-11-09T12:05:21.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19983" PostTypeId="2" ParentId="19957" CreationDate="2022-11-09T12:09:51.040" Score="0" Body="&lt;p&gt;Hi An Ignorant Wanderer,&lt;/p&gt;&#xA;&lt;p&gt;Yes, you're right - ENST00000414213 is a deprecated ID from Ensembl release 75.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ensembl.org/Homo_sapiens/Transcript/Idhistory?t=ENST00000414213&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ensembl.org/Homo_sapiens/Transcript/Idhistory?t=ENST00000414213&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;To convert a list of IDs from GRCh37, you will need to use the GRCh37 dataset. You can do this using the following documentation/example:&#xA;&lt;a href=&quot;https://www.ensembl.org/info/data/biomart/biomart_r_package.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ensembl.org/info/data/biomart/biomart_r_package.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6500" LastActivityDate="2022-11-09T12:09:51.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19984" PostTypeId="1" AcceptedAnswerId="20009" CreationDate="2022-11-09T14:55:24.597" Score="3" ViewCount="153" Body="&lt;p&gt;I want to compare the headers from the fasta file to file1, and if there's a match, reorganize the header and put the match first. If there's no match between fasta file and file1,  look at file2 and if there's a match, put that name first on the header.&lt;/p&gt;&#xA;&lt;p&gt;The goal is to organize the fasta file header with the similar match listed as the first entry on the header. How can I do this with this code or is there another way to write it in python or biopython?&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;BioPython&lt;/code&gt; solutions welcome.&lt;/p&gt;&#xA;&lt;p&gt;Current files look like this&lt;/p&gt;&#xA;&lt;p&gt;seqs.fasta:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;ant| bee| fly &#xA;AGCT... &#xA;&amp;gt;bird| eagle| hawk| vulture &#xA;GATCG...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;file1.txt:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mouse &#xA;hawk &#xA;cow&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;file2.txt:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;crane &#xA;fly&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output file should look like this, so that the order of the header starts with the match from file 1 or file 2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; fly| ant| bee &#xA;AGCT... &#xA;&amp;gt; hawk| bird| eagle| vulture &#xA;AGCT... &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My code is:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;fasta= open('seq.fasta', 'r')&#xA;one= open('file1.txt' , 'r')&#xA;two= open('file2.txt' , 'r')&#xA;output= open('newseq.fasta', 'w')&#xA;&#xA;for line in fasta:&#xA;    if line.startswith('&amp;gt;'): # getting just the header&#xA;        header= line.strip().split(&amp;quot;|&amp;quot;) # there are several entries on the header separated by the |&#xA;        for name in one:&#xA;           if name in header: # looking to see if file1.txt has match in fasta file&#xA;              header.insert(0,line.pop(header.index(name))) # insert the name from file1 as the first header entry&#xA;        else: # if no match in file1 look at file2&#xA;            for name in two:&#xA;               if name in header:&#xA;                    header.insert(0,line.pop(header.index(name))) # insert the name from file2 as first header entry   &#xA;    else:&#xA;output.write(line)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16191" LastEditorUserId="3848" LastEditDate="2022-11-13T16:35:25.820" LastActivityDate="2022-11-21T16:42:22.017" Title="Compare and Reorganize Fasta Headers Python" Tags="|python|fasta|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19985" PostTypeId="1" CreationDate="2022-11-09T16:13:08.433" Score="0" ViewCount="43" Body="&lt;p&gt;I want to use &lt;a href=&quot;https://faculty.washington.edu/browning/conform-gt.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;conform-gt&lt;/a&gt; to get rid of some REF prefix differences see &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/19977/large-amount-of-ref-prefix-differences-after-lifting?noredirect=1#comment29310_19977&quot;&gt;here&lt;/a&gt;. Conform-gt needs the reference FASTA file split by chromosome and in .vcf.gz format. How can I do this?&lt;/p&gt;&#xA;&lt;p&gt;I already tried &lt;code&gt;faidx&lt;/code&gt; with&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;faidx --transform bed reference/GRCh38_full_analysis_set_plus_decoy_hla.fa &amp;gt; reference/GRCh38_full_analysis_set_plus_decoy_hla.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but this produced the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &amp;quot;/home/oem/.local/bin/faidx&amp;quot;, line 8, in &amp;lt;module&amp;gt;&#xA;    sys.exit(main())&#xA;  File &amp;quot;/home/oem/.local/lib/python3.9/site-packages/pyfaidx/cli.py&amp;quot;, line 202, in main&#xA;    write_sequence(args)&#xA;  File &amp;quot;/home/oem/.local/lib/python3.9/site-packages/pyfaidx/cli.py&amp;quot;, line 51, in write_sequence&#xA;    outfile.write(transform_sequence(args, fasta, name, start, end))&#xA;  File &amp;quot;/home/oem/.local/lib/python3.9/site-packages/pyfaidx/cli.py&amp;quot;, line 121, in transform_sequence&#xA;    line_len = fasta.faidx.index[name].lenc&#xA;KeyError: 'HLA-A*01'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16454" LastEditorUserId="298" LastEditDate="2022-11-09T18:14:21.607" LastActivityDate="2022-11-09T18:14:21.607" Title="Conversion FASTA to BED fails" Tags="|format-conversion|python|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="19986" PostTypeId="1" CreationDate="2022-11-09T20:37:07.663" Score="1" ViewCount="198" Body="&lt;p&gt;I am trying to find some matching motifs on a sequence, as well as the position that the motif is located in and then output that into a fasta file. The code below shows that the motif &lt;code&gt;[L*L*L]&lt;/code&gt; is present in the sequence, when I run it returns as &amp;quot;YES&amp;quot; but I do not know where it is positioned.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;peptide1=&amp;quot;MKFSNEVVHKSMNITEDCSALTGALLKYSTDKSNMNFETLYRDAAVESPQHEVSNESGSTLKEHDYFGLSEVSSSNSSSGKQPEKCCREELNLNESATTLQLGPPAAVKPSGHADGADAHDEGAGPENPAKRPAHHMQQESLADGRKAAAEMGSFKIQRKNILEEFRAMKAQAHMTKSPKPVHTMQHNMHASFSGAQMAFGGAKNNGVKRVFSEAVGGNHIAASGVGVGVREGNDDVSRCEEMNGTEQLDLKVHLPKGMGMARMAPVSGGQNGSAWRNLSFDNMQGPLNPFFRKSLVSKMPVPDGGDSSANASNDCANRKGMVASPSVQPPPAQNQTVGWPPVKNFNKMNTPAPPASTPARACPSVQRKGASTSSSGNLVKIYMDGVPFGRKVDLKTNDSYDKLYSMLEDMFQQYISGQYCGGRSSSSGESHWVASSRKLNFLEGSEYVLIYEDHEGDSMLVGDVPWELFVNAVKRLRIMKGSEQVNLAPKNADPTKVQVAVG&amp;quot;&#xA;&#xA;if re.search(r&amp;quot;L*L*L&amp;quot;, peptide1):&#xA;    print(&amp;quot;YES&amp;quot;)&#xA;else: &#xA;    print(&amp;quot;NO&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The code that I wrote to find the position is below, but when I run it says invalid syntax. Could you please assist as I am still new in Bioinformatics, and I have no clue whether in the right track or not.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;for position in range(len(peptide1)):&#xA;    if peptide1[position:].startswith(re.search(r&amp;quot;L*L*L&amp;quot;)):&#xA;        print(position+1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Lastly once the position of these motifs has been identified, how do I highlight them such that they are visible in the output fasta file.&lt;/p&gt;&#xA;&lt;p&gt;I am searching for &lt;code&gt;LxLxL&lt;/code&gt; where &lt;code&gt;x&lt;/code&gt; can be any amino acid. I used the &lt;code&gt;*&lt;/code&gt; to indicate to python that it should use any amino acid, but I think I am incorrect. Could you please assist on how I should actually be writing that motif?&lt;/p&gt;&#xA;" OwnerUserId="16468" LastEditorUserId="298" LastEditDate="2022-11-10T13:27:24.830" LastActivityDate="2022-11-10T13:27:24.830" Title="finding matching motifs in a sequence and their position" Tags="|fasta|python|motifs|sequence|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="19987" PostTypeId="2" ParentId="19980" CreationDate="2022-11-09T23:30:51.817" Score="1" Body="&lt;p&gt;This is a machine learning application. Its clear cut. Whether you want to perform unsupervised learning is your call.&lt;/p&gt;&#xA;&lt;p&gt;Input data is for the cell growth, metabolite concentration time series would be:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;T1 ... T4 (cell growth) + T1 ... T4 (metabolite 1) etc ...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The training target is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;media type&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I would start with a simple naive Bayes model and see how it goes. If you obtain a high accuracy score you're in business, its worked. You would simply plot the result. The classification may not be linear in which case an alternative model is used.&lt;/p&gt;&#xA;&lt;p&gt;You can manually test it, by inputing the data of cell growth etc ... and seeing if it correctly predicts the media type.&lt;/p&gt;&#xA;&lt;p&gt;You then ask the ML algorithm which the metabolite/cell growth the 'media type' is influencing and it will tell you. Setting it up is complex I agree, but thats the gist and was the question.&lt;/p&gt;&#xA;&lt;p&gt;Just to finish, if you want to limit what a given media is in particular influencing, i.e. not loads of measurements but just a few inputs/variables, e.g. a metabolite at a particular time point and/or cell growth at a given time point, then ML delivers that information. Thats a key part of its strength. Thus you can then trial loads of different media - even stuff not trained - and just measure a small number of select inputs/variables rather than the entire kitchen sink - which is what the OP is doing right now.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The overall input of ML into biological problems is a bit worrying in truth and tends to be over-represented in a select number of fields, protein modelling, and zero in everything else.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-11-10T14:35:30.173" LastActivityDate="2022-11-10T14:35:30.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19988" PostTypeId="2" ParentId="19978" CreationDate="2022-11-10T10:10:23.450" Score="0" Body="&lt;p&gt;A (very) simple Python solution:&lt;/p&gt;&#xA;&lt;p&gt;The script file &lt;code&gt;readfile.py&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#! /usr/bin/python3&#xA;&#xA;import sys&#xA;import json&#xA;&#xA;filename=sys.argv[1]&#xA;filter=sys.argv[2]&#xA;&#xA;results=[]&#xA;with open(filename) as json_file:&#xA;    json_content = json.load(json_file)&#xA;    for json_graph_element in json_content[&amp;quot;@graph&amp;quot;]:&#xA;        if json_graph_element[&amp;quot;method&amp;quot;] == filter:&#xA;            results.append(json_graph_element[&amp;quot;value&amp;quot;])&#xA;print(&amp;quot;%s\t%s&amp;quot; % (filter, &amp;quot;, &amp;quot;.join(results)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Make the script executable:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;chmod +x readfile.py&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Use the script:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;./readfile.py /path/to/the/file.json  eQTLs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Result:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;eQTLs   EIF3S8, EIF3CL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16471" LastActivityDate="2022-11-10T10:10:23.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19989" PostTypeId="2" ParentId="19964" CreationDate="2022-11-10T10:27:50.613" Score="0" Body="&lt;p&gt;Hej Bikal,&lt;/p&gt;&#xA;&lt;p&gt;I am not sure why you would like to inspect all 4,000+ proteins at once.  I would expect that you have a look on a subset of &amp;quot;interesting&amp;quot; genes, eg from a differential expression experiment.&lt;/p&gt;&#xA;&lt;p&gt;Yet, since you've asked, yet it is possible to get the data for all proteins at once.&lt;/p&gt;&#xA;&lt;p&gt;You can &lt;a href=&quot;https://string-db.org/cgi/download?sessionId=bZFrd03XKud9&amp;amp;species_text=Phytophthora+cactorum&quot; rel=&quot;nofollow noreferrer&quot;&gt;download&lt;/a&gt;&#xA;the network for entire genomes&#xA;and analyze with what ever tool you seem fit.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, may I recommend you to have a look at the Cytoscape plugin &lt;a href=&quot;https://apps.cytoscape.org/apps/stringapp&quot; rel=&quot;nofollow noreferrer&quot;&gt;stringApp&lt;/a&gt;. It should help you with the download, analysis (including pathway enrichment), and visualizations. Unlike the String network in your web browser, Cytoscape can easily handle large networks.&lt;/p&gt;&#xA;&lt;p&gt;The tool's authors have a nice series of&#xA;&lt;a href=&quot;https://www.youtube.com/playlist?list=PLjnQS7gxY4Ar6VukbDHp0q0atJAFDWkRX&quot; rel=&quot;nofollow noreferrer&quot;&gt;video tutorials&lt;/a&gt; of how to use it. Hopefully that helps in your work.&lt;/p&gt;&#xA;" OwnerUserId="16386" LastActivityDate="2022-11-10T10:27:50.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19990" PostTypeId="2" ParentId="19977" CreationDate="2022-11-10T10:38:34.793" Score="0" Body="&lt;p&gt;Thanks for @JRodrigoF for pointing towards &lt;code&gt;bcftools norm --check-ref s --fasta-ref&lt;/code&gt;. This will indeed fix the initial REF prefix diff issue of my question.&lt;/p&gt;&#xA;&lt;p&gt;If anyone else stumbles over this question while facing a similar error to :&lt;code&gt;[E::faidx_adjust_position] The sequence &amp;quot;1&amp;quot; was not found faidx_fetch_seq failed at 1:69869&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Please try to re-index your FASTA-file using samtools&#xA;&lt;code&gt;samtools faidx reference_file.fa&lt;/code&gt; Most of the times this seems to solve this kind of error!&lt;/p&gt;&#xA;" OwnerUserId="16454" LastEditorUserId="16454" LastEditDate="2022-11-10T10:50:50.360" LastActivityDate="2022-11-10T10:50:50.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19991" PostTypeId="2" ParentId="15376" CreationDate="2022-11-10T11:26:18.520" Score="1" Body="&lt;p&gt;ZINC20 is a free database of commercially-available compounds for virtual screening. It contains 1.3M compounds with molecular weight less than 200.&#xA;&lt;a href=&quot;https://zinc20.docking.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://zinc20.docking.org/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Cartblanche22 is an interface to ZINC22. Here you can find more compounds than ZINC20.&#xA;&lt;a href=&quot;https://cartblanche22.docking.org/cartblanche&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cartblanche22.docking.org/cartblanche&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6717" LastActivityDate="2022-11-10T11:26:18.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19992" PostTypeId="2" ParentId="19986" CreationDate="2022-11-10T11:36:05.570" Score="2" Body="&lt;p&gt;&lt;code&gt;re.search&lt;/code&gt; method do not return a simply boolean but a useful object contains many information:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#! /usr/bin/env python3&#xA;&#xA;import re&#xA;&#xA;peptide1=&amp;quot;MKFSNEVVHKSMNITEDCSALTGALLKYSTDKSNMNFETLYRDAAVESPQHEVSNESGSTLKEHDYFGLSEVSSSNSSSGKQPEKCCREELNLNESATTLQLGPPAAVKPSGHADGADAHDEGAGPENPAKRPAHHMQQESLADGRKAAAEMGSFKIQRKNILEEFRAMKAQAHMTKSPKPVHTMQHNMHASFSGAQMAFGGAKNNGVKRVFSEAVGGNHIAASGVGVGVREGNDDVSRCEEMNGTEQLDLKVHLPKGMGMARMAPVSGGQNGSAWRNLSFDNMQGPLNPFFRKSLVSKMPVPDGGDSSANASNDCANRKGMVASPSVQPPPAQNQTVGWPPVKNFNKMNTPAPPASTPARACPSVQRKGASTSSSGNLVKIYMDGVPFGRKVDLKTNDSYDKLYSMLEDMFQQYISGQYCGGRSSSSGESHWVASSRKLNFLEGSEYVLIYEDHEGDSMLVGDVPWELFVNAVKRLRIMKGSEQVNLAPKNADPTKVQVAVG&amp;quot;&#xA;&#xA;re_search_result = re.search(r'L*L*L', peptide1)&#xA;if re_search_result:&#xA;    print(&amp;quot;YES&amp;quot;)&#xA;    motif_start_position=re_search_result.start()&#xA;    motif_end_position=re_search_result.end()&#xA;    motif_length = re_search_result.end() - re_search_result.start()&#xA;    print(&amp;quot;Motif start at position: %d&amp;quot; % motif_start_position)&#xA;    print(&amp;quot;Motif end at position  : %d&amp;quot; % motif_end_position)&#xA;    print(&amp;quot;SO length is ......... : %d&amp;quot; % motif_length)&#xA;    print(&amp;quot;The motif is : &amp;lt;%s&amp;gt;&amp;quot; % peptide1[motif_start_position:motif_end_position])&#xA;else:&#xA;    print(&amp;quot;NO&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Result is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;YES&#xA;Motif start at position: 20&#xA;Motif end at position  : 488&#xA;SO length is ......... : 468&#xA;The motif is : &amp;lt;LTGALLKYSTDKSNMNFETLYRDAAVESPQHEVSNESGSTLKEHDYFGLSEVSSSNSSSGKQPEKCCREELNLNESATTLQLGPPAAVKPSGHADGADAHDEGAGPENPAKRPAHHMQQESLADGRKAAAEMGSFKIQRKNILEEFRAMKAQAHMTKSPKPVHTMQHNMHASFSGAQMAFGGAKNNGVKRVFSEAVGGNHIAASGVGVGVREGNDDVSRCEEMNGTEQLDLKVHLPKGMGMARMAPVSGGQNGSAWRNLSFDNMQGPLNPFFRKSLVSKMPVPDGGDSSANASNDCANRKGMVASPSVQPPPAQNQTVGWPPVKNFNKMNTPAPPASTPARACPSVQRKGASTSSSGNLVKIYMDGVPFGRKVDLKTNDSYDKLYSMLEDMFQQYISGQYCGGRSSSSGESHWVASSRKLNFLEGSEYVLIYEDHEGDSMLVGDVPWELFVNAVKRLRIMKGSEQVNL&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The motif &lt;code&gt;L*L*L&lt;/code&gt; is a quite strange: &lt;code&gt;L&lt;/code&gt; zero or many times followed by &lt;code&gt;L&lt;/code&gt; zero or many times followed by followed by a &lt;code&gt;L&lt;/code&gt;. It's an equivalent of &lt;code&gt;L*L&lt;/code&gt;, equivalent of &lt;code&gt;L+&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Do you search for &lt;code&gt;L.*L.*L&lt;/code&gt;, &lt;code&gt;L[^L]*L[^L]*L&lt;/code&gt; or simply &lt;code&gt;LLL&lt;/code&gt; ?&lt;/p&gt;&#xA;&lt;p&gt;Notes:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;When you post code, use block delimited by two ``` (no by two simple `)&lt;/li&gt;&#xA;&lt;li&gt;In Python, code indentation is very important, always use spaces (4)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;If you want to loop over all motifs (like strings), use &lt;code&gt;findall&lt;/code&gt; method:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#! /usr/bin/env python3&#xA;&#xA;import re&#xA;&#xA;peptide1=&amp;quot;MKFSNEVVHKSMNITEDCSALTGALLKYSTDKSNMNFETLYRDAAVESPQHEVSNESGSTLKEHDYFGLSEVSSSNSSSGKQPEKCCREELNLNESATTLQLGPPAAVKPSGHADGADAHDEGAGPENPAKRPAHHMQQESLADGRKAAAEMGSFKIQRKNILEEFRAMKAQAHMTKSPKPVHTMQHNMHASFSGAQMAFGGAKNNGVKRVFSEAVGGNHIAASGVGVGVREGNDDVSRCEEMNGTEQLDLKVHLPKGMGMARMAPVSGGQNGSAWRNLSFDNMQGPLNPFFRKSLVSKMPVPDGGDSSANASNDCANRKGMVASPSVQPPPAQNQTVGWPPVKNFNKMNTPAPPASTPARACPSVQRKGASTSSSGNLVKIYMDGVPFGRKVDLKTNDSYDKLYSMLEDMFQQYISGQYCGGRSSSSGESHWVASSRKLNFLEGSEYVLIYEDHEGDSMLVGDVPWELFVNAVKRLRIMKGSEQVNLAPKNADPTKVQVAVG&amp;quot;&#xA;&#xA;re_findall_result = re.findall(r'L[^L]*L[^L]*L', peptide1)&#xA;for element in re_findall_result:&#xA;    print(&amp;quot;Element: &amp;lt;%s&amp;gt;&amp;quot; % element)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Result will be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Element: &amp;lt;LTGALL&amp;gt;&#xA;Element: &amp;lt;LYRDAAVESPQHEVSNESGSTLKEHDYFGL&amp;gt;&#xA;Element: &amp;lt;LNLNESATTL&amp;gt;&#xA;Element: &amp;lt;LGPPAAVKPSGHADGADAHDEGAGPENPAKRPAHHMQQESLADGRKAAAEMGSFKIQRKNIL&amp;gt;&#xA;Element: &amp;lt;LDLKVHL&amp;gt;&#xA;Element: &amp;lt;LSFDNMQGPLNPFFRKSL&amp;gt;&#xA;Element: &amp;lt;LVKIYMDGVPFGRKVDLKTNDSYDKL&amp;gt;&#xA;Element: &amp;lt;LEDMFQQYISGQYCGGRSSSSGESHWVASSRKLNFL&amp;gt;&#xA;Element: &amp;lt;LIYEDHEGDSMLVGDVPWEL&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;If you want all find objects like with &lt;code&gt;search&lt;/code&gt; method, use &lt;code&gt;finditer&lt;/code&gt; method:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#! /usr/bin/env python3&#xA;&#xA;import re&#xA;&#xA;peptide1=&amp;quot;MKFSNEVVHKSMNITEDCSALTGALLKYSTDKSNMNFETLYRDAAVESPQHEVSNESGSTLKEHDYFGLSEVSSSNSSSGKQPEKCCREELNLNESATTLQLGPPAAVKPSGHADGADAHDEGAGPENPAKRPAHHMQQESLADGRKAAAEMGSFKIQRKNILEEFRAMKAQAHMTKSPKPVHTMQHNMHASFSGAQMAFGGAKNNGVKRVFSEAVGGNHIAASGVGVGVREGNDDVSRCEEMNGTEQLDLKVHLPKGMGMARMAPVSGGQNGSAWRNLSFDNMQGPLNPFFRKSLVSKMPVPDGGDSSANASNDCANRKGMVASPSVQPPPAQNQTVGWPPVKNFNKMNTPAPPASTPARACPSVQRKGASTSSSGNLVKIYMDGVPFGRKVDLKTNDSYDKLYSMLEDMFQQYISGQYCGGRSSSSGESHWVASSRKLNFLEGSEYVLIYEDHEGDSMLVGDVPWELFVNAVKRLRIMKGSEQVNLAPKNADPTKVQVAVG&amp;quot;&#xA;&#xA;re_findall_result = re.finditer(r'L[^L]*L[^L]*L', peptide1)&#xA;for element in re_findall_result:&#xA;    print(&amp;quot;peptide1[%s:%s]=&amp;lt;%s&amp;gt;&amp;quot; % (element.start(), element.end(), peptide1[element.start():element.end()]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Result will be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;peptide1[20:26]=&amp;lt;LTGALL&amp;gt;&#xA;peptide1[39:69]=&amp;lt;LYRDAAVESPQHEVSNESGSTLKEHDYFGL&amp;gt;&#xA;peptide1[90:100]=&amp;lt;LNLNESATTL&amp;gt;&#xA;peptide1[101:163]=&amp;lt;LGPPAAVKPSGHADGADAHDEGAGPENPAKRPAHHMQQESLADGRKAAAEMGSFKIQRKNIL&amp;gt;&#xA;peptide1[248:255]=&amp;lt;LDLKVHL&amp;gt;&#xA;peptide1[278:296]=&amp;lt;LSFDNMQGPLNPFFRKSL&amp;gt;&#xA;peptide1[378:404]=&amp;lt;LVKIYMDGVPFGRKVDLKTNDSYDKL&amp;gt;&#xA;peptide1[407:443]=&amp;lt;LEDMFQQYISGQYCGGRSSSSGESHWVASSRKLNFL&amp;gt;&#xA;peptide1[449:469]=&amp;lt;LIYEDHEGDSMLVGDVPWEL&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16471" LastActivityDate="2022-11-10T11:36:05.570" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="19993" PostTypeId="1" CreationDate="2022-11-10T16:03:18.420" Score="2" ViewCount="42" Body="&lt;p&gt;I am a master's student in computational and mathematical biology, with a bachelor's degree in pure mathematics.&#xA;I am really struggling with my computational biology course, we are supposed to study problems related to evolutionary dynamics like:&#xA;1)the rock-paper-scissors game which explains the evolution of male mating strategies in side-blotched lizards&#xA;2)fixation probability of a new mutation.&#xA;Could someone link me to some books or references that might be helpful in solving such problems?&lt;/p&gt;&#xA;" OwnerUserId="16457" LastEditorUserId="3848" LastEditDate="2023-08-31T02:19:21.360" LastActivityDate="2023-08-31T02:19:21.360" Title="Is there any book or reference that explains evolutionary dynamics and fixation probability?" Tags="|phylogenetics|phylogeny|genetics|population-genetics|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19994" PostTypeId="2" ParentId="19993" CreationDate="2022-11-10T20:57:38.603" Score="2" Body="&lt;ul&gt;&#xA;&lt;li&gt;Maynard Smith, J. (1989) Evolutionary Genetics. Oxford: Oxford University Press. ISBN 0-19-850231-1&lt;/li&gt;&#xA;&lt;li&gt;Maynard Smith, J. and Harper, D. (2003) Animal Signals. Oxford University Press. ISBN 0-19-852685-7[16]&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Then perform a PubMed search for ‪&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/?term=Eyre-Walker&quot; rel=&quot;nofollow noreferrer&quot;&gt;Adam Eyre-Walker&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-11-10T20:57:38.603" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19995" PostTypeId="2" ParentId="19993" CreationDate="2022-11-11T00:20:51.387" Score="2" Body="&lt;p&gt;I'm fond of Joe Felsenstein's &lt;a href=&quot;https://evolution.genetics.washington.edu/pgbook/pgbook.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;book&lt;/a&gt; on evolutionary genetics as a relatively recent source that presents a lot of these problems at a very simple level for the newer student, and then adds complexity gradually. Page 260 seems relevant.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-11-11T00:20:51.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19996" PostTypeId="2" ParentId="19977" CreationDate="2022-11-11T09:31:26.577" Score="1" Body="&lt;p&gt;It is important that after lifting over a VCF/BCF file from one coordinate system to another, the header of the resulting file contains the chromosome/contig names as present in the coordinate system of the new reference genome. it works as a double check that the assumptions of the coordinate system (particular reference) are consistent with the actual data (VCF lines). PLINK files for example do not have a header section and this doesn't not apply.&lt;/p&gt;&#xA;&lt;p&gt;Looks like BCFTools (merge) complains whenever if finds positions that have one base ('C') as reference in one of the input files, and another base ('N') in another file. When working with VCF/BCF files, &lt;strong&gt;bcftools norm&lt;/strong&gt; can be used to fix 'N' reference position into their actual base (if that is the case) and/or fix bases listed as reference (e.g. coming from a PLINK major-allele based BED file), but which actually correspond to an alternate allele based on either GRCh37/GRCh38, ... .&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools norm --check-ref s --fasta-ref &amp;lt;reference_file&amp;gt; &amp;lt;my_VCF_file&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="475" LastActivityDate="2022-11-11T09:31:26.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="19997" PostTypeId="1" AcceptedAnswerId="19998" CreationDate="2022-11-11T13:36:34.603" Score="1" ViewCount="98" Body="&lt;p&gt;After seeing a video of a combination of CRISPR and AI, and an article of someone who made two babies immune against HIV and still healthy, I wondered about something. If such complicated thing, to edit someone's genome, so that the person isn't likely to get other diseases or complications, while having a selected function, is (almost) possible, will it be possible currently to let AI generate RNA of virus particles like from a Generative Adversarial Network?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We show that these&#xA;tools capture important structures of the data and, when applied to designing probes&#xA;for protein binding microarrays (PBMs), allow us to generate new sequences whose&#xA;properties are estimated to be superior to those found in the training data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://arxiv.org/abs/1712.06148&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://arxiv.org/abs/1712.06148&lt;/a&gt; In this research article, it is stated that the researchers were successful in training the model so that it would generate new sequences, while having a specific functioning protein. Note that this was from the time when ProGAN wasn't even invented yet, so they used a Wasserstein-GAN.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;While we found these techniques to generally be well-behaved, we&#xA;did notice some undesirable cases during our explorations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;They concluded that even one mistake in nucleotides could mess the protein up. But now in 2022, we have StyleGAN-Ada (&lt;a href=&quot;https://github.com/NVlabs/stylegan2-ada&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/NVlabs/stylegan2-ada&lt;/a&gt;), which is far better in generating stuff, we've already (almost) overcome that stuff? So if you basically train such model on RNA of virus particles (most viruses have really short genome compared to an eukaryote cell, so a supercomputer is not required for training such thing, and there are databases online: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/548/publicly-available-genome-sequence-database-for-viruses&quot;&gt;Publicly available genome sequence database for viruses?&lt;/a&gt;), such as every mutation of SARS-CoV-2, HIV, ebola and so on, it would basically be possible to let it generate something that is as contageous as SARS-CoV-2, whereas as deadly as HIV? Keep in mind that one single generated sequence that functions, would already be devastating. It doesn't need to be 100% accurate, because after all we can just experiment with it and find out whether it is working or not as supposed to be.&lt;/p&gt;&#xA;&lt;p&gt;It doesn't have to be a GAN:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We present three approaches: creating&#xA;synthetic DNA sequences using a generative adversarial network (GAN); a DNAbased variant of the activation maximization (“deep dream”) design method; and a&#xA;joint procedure which combines these two approaches together.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The RNA-sequence can be brought into a cell, and the cell will automatically create the components of the virus, which will become the virus ultimately.&lt;/p&gt;&#xA;&lt;p&gt;Anyone can recall that this is possible? I have done my searches on Google, but I haven't found any article particularly describing this concept.&lt;/p&gt;&#xA;&lt;p&gt;I hope this is on-topic, I don't know how the forums work. Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="16483" LastEditorUserId="16483" LastEditDate="2022-11-11T15:45:08.313" LastActivityDate="2022-11-11T20:36:51.723" Title="Is current AI able to generate RNA sequences of viruses?" Tags="|rna|machine-learning|virus|" AnswerCount="1" CommentCount="1" ClosedDate="2022-11-13T01:40:56.013" ContentLicense="CC BY-SA 4.0" />
  <row Id="19998" PostTypeId="2" ParentId="19997" CreationDate="2022-11-11T14:11:57.647" Score="1" Body="&lt;p&gt;Reinforced learning is an interesting technology, but they are prone to extreme results - which are not necessarily at all biological feasible. Google can use it to outperform, or very possibly outcompete, the world's best chess player, because all of the rules are known. In biology we simply do not know all the rules.&lt;/p&gt;&#xA;&lt;p&gt;To reiterate, the problem with reinforced learning in biology is not only do we not know all the rules, in fact in reality we know very few of the rules. HIV - why does it cause persistent infection? SARS-CoV-2 why doesn't the vaccine confer complete protection? Moreover, the GAN originator needs to know all of the biological &lt;em&gt;known&lt;/em&gt; rules and apply them within the GAN and there are a lot of rules. Does the originator know the conformational change of env or spike and on which amino acid that are invariant? I doubt it. That information would be pivotal to generate AI sequence for a biologically feasible virus.&lt;/p&gt;&#xA;&lt;p&gt;Even in the original Atari video game experiment &lt;strong&gt;*&lt;/strong&gt; whilst GANS outcompeted humans by &amp;gt;1000%, equally they under-performed humans considerable for around &amp;gt;20% games tested. Thus even if all the rules are known GANs can mess-up a large proportion of the time.&lt;/p&gt;&#xA;&lt;p&gt;These are simply hypotheses in an emerging technology within biology that require biological verifiable results and those proofs will be difficult, i.e. time consuming. This is not the case within video games, or board games.&lt;/p&gt;&#xA;&lt;p&gt;In the first instance, GAN output can be investigated for biological validity against strongly established bioinformatics analysis - the stuff we use to assess whether there are errors in the data. However, there needs to be a lot incentive to perform this research at this point in the technological cycle. When the authors have something concrete, we might then reinvestigate the fundamental issues of their predictions.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;*&lt;/strong&gt;, the old Atari video games were subject to GANs and the computer then compared its performance in the video game against an average human. The results were highly variable, moreover humans are highly variable - which to my knowledge was ever investigated - so it was never at all clear the &amp;quot;the average human&amp;quot; represented in these tests.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;A GAN isn't 'trained against [biological entity]' in the technical way supervised learning is trained. A GAN 'trains' against itself via a 'win/lose' quantification. Establishing a valid 'win/lose' is a pivotal part of the GAN. Thus in chess thats easy ... you win or lose the game, in biological that is not trivial.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;&#xA;To address the point below, yes GANs devise their own &amp;quot;rules&amp;quot; to maximise win/loss WITHIN the rules already in place. However, they needs 'ground rules' otherwise how could it play a game of chess if it doesn't follow the rules of chess? How could it play an Atari video game if it does not know what a &amp;quot;joystick&amp;quot; is? The problem is without knowing the ground rules (what is biologically permitted), what's the validity of the subsequent rule set, i.e. predicted sequence in this case?&lt;/p&gt;&#xA;&lt;p&gt;So a virus that can't attach to the host cellular receptor, isn't a virus because it will never be able to enter the cell.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Further note&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The OP pointed out in the notes&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;The idea of style transfer has largely only been explored in image-based tasks, which we attribute in part to the specific nature of loss functions used for style transfer.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The author is talking about a variant of transfer learning. What the author is reminding us is that overall transferring-learning is the technique which powered the success of computer vision. They are saying they have replicated that in their GANs.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Please note any further points should be separate questions&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-11-11T20:36:51.723" LastActivityDate="2022-11-11T20:36:51.723" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="19999" PostTypeId="1" CreationDate="2022-11-11T16:43:02.817" Score="2" ViewCount="90" Body="&lt;p&gt;Similar to: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/5400/how-to-convert-data-in-gmt-format-to-dataframe&quot;&gt;How to convert data in gmt format to dataframe?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have downloaded a GMT file from the GSEA database (&lt;a href=&quot;https://www.gsea-msigdb.org/gsea/msigdb/download_file.jsp?filePath=/msigdb/release/2022.1.Hs/c2.all.v2022.1.Hs.symbols.gmt&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.gsea-msigdb.org/gsea/msigdb/download_file.jsp?filePath=/msigdb/release/2022.1.Hs/c2.all.v2022.1.Hs.symbols.gmt&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;I have imported this into R using the GSA.read.gmt tool from GSA as I want to do pathway analysis using GAUSS (&lt;a href=&quot;https://github.com/diptavo/GAUSS/blob/master/example_gmt.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/diptavo/GAUSS/blob/master/example_gmt.txt&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;GAUSS needs the table to have the genes in a comma separated manner as well as there being headers in the table.&lt;/p&gt;&#xA;&lt;p&gt;I have tried using the gsub function on the third column but it doesn't work, I feel this is because a GMT file is not the same as a dataframe.&lt;/p&gt;&#xA;&lt;p&gt;Would anyone be able to help me turn this column into a comma separated one please?&lt;/p&gt;&#xA;&lt;p&gt;Many thanks&lt;/p&gt;&#xA;" OwnerUserId="9990" LastActivityDate="2022-11-12T16:08:34.623" Title="Turning GMT file into data frame so that third column is comma seperated" Tags="|r|pathway|dataframe|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20000" PostTypeId="1" AcceptedAnswerId="20001" CreationDate="2022-11-11T19:11:05.023" Score="2" ViewCount="72" Body="&lt;p&gt;I have pairwise distance data for single linkage cluster and I would like to convert it to &lt;a href=&quot;http://en.wikipedia.org/wiki/Newick_format&quot; rel=&quot;nofollow noreferrer&quot;&gt;newick format&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I was unable to find a conversion algorithm.&lt;/p&gt;&#xA;&lt;p&gt;I require newick format for visualization and annotation of a phylogeny.&lt;/p&gt;&#xA;&lt;p&gt;Here is the output.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;INFO  : ======================================&#xA;INFO  : Pairwise single linkage clustering&#xA;INFO  : ======================================&#xA;INFO  : Hierarchical Tree&#xA;INFO  : ======================================&#xA;INFO  : Node     Item 1   Item 2      Distance&#xA;INFO  :     0 :       53       52        0.001  5zxf  5z02&#xA;INFO  :    -1 :       64       60        0.002  7bj6  7bir&#xA;INFO  :    -2 :       62       -1        0.002  7biv          &#xA;INFO  :    -3 :       34       26        0.002  4wt2  4ode&#xA;INFO  :    -4 :       65       -2        0.003  7bmg          &#xA;INFO  :    -5 :       25       13        0.003  4occ  4erf&#xA;INFO  :    -6 :       29       -3        0.009  4ogt          &#xA;INFO  :    -7 :       39       38        0.010  4zyi  4zyf&#xA;INFO  :    -8 :       50       48        0.011  5oc8  5ln2&#xA;INFO  :    -9 :       54       -8        0.012  6ggn          &#xA;INFO  :   -10 :       42       41        0.015  5hmi  5hmh&#xA;INFO  :   -11 :       -4       61        0.020            7bit&#xA;INFO  :   -12 :       70       40        0.028  7qdq  5c5a&#xA;INFO  :   -13 :       47       68        0.032  5laz  7na3&#xA;INFO  :   -14 :       -9       45        0.035            5law&#xA;INFO  :   -15 :       49       59        0.038  5oai  6q9o&#xA;INFO  :   -16 :       32       -5        0.039  4qo4          &#xA;INFO  :   -17 :      -15        8        0.044            3uu1&#xA;INFO  :   -18 :       23      -16        0.047  4oas          &#xA;INFO  :   -19 :      -14       -7        0.053                    &#xA;INFO  :   -20 :       24      -18        0.057  4oba          &#xA;INFO  :   -21 :      -19        6        0.057            3lbl&#xA;INFO  :   -22 :       43      -10        0.058  5hmk          &#xA;INFO  :   -23 :       44       19        0.059  5lav  4jvr&#xA;INFO  :   -24 :      -23      -20        0.061                    &#xA;INFO  :   -25 :      -11       15        0.061            4hg7&#xA;INFO  :   -26 :       57      -24        0.062  6q9h          &#xA;INFO  :   -27 :       27       -6        0.065  4odf          &#xA;INFO  :   -28 :      -25      -17        0.067                    &#xA;INFO  :   -29 :       14      -28        0.067  4hbm          &#xA;INFO  :   -30 :      -13      -21        0.067                    &#xA;INFO  :   -31 :      -27      -29        0.067                    &#xA;INFO  :   -32 :      -26      -30        0.068                    &#xA;INFO  :   -33 :       58       21        0.068  6q9l  4mdn&#xA;INFO  :   -34 :      -31      -32        0.070                    &#xA;INFO  :   -35 :       17        5        0.071  4jv9  3lbk&#xA;INFO  :   -36 :       30      -34        0.074  4ogv          &#xA;INFO  :   -37 :      -33        2        0.077            1t4e&#xA;INFO  :   -38 :       35      -36        0.079  4zfi          &#xA;INFO  :   -39 :       20      -38        0.086  4jwr          &#xA;INFO  :   -40 :      -39      -37        0.094                    &#xA;INFO  :   -41 :        7      -40        0.106  3tj2          &#xA;INFO  :   -42 :       51      -41        0.111  5trf          &#xA;INFO  :   -43 :       69      -42        0.112  7na4          &#xA;INFO  :   -44 :       56      -43        0.113  6q96          &#xA;INFO  :   -45 :        4      -44        0.127  3jzk          &#xA;INFO  :   -46 :       33      -45        0.129  4qoc          &#xA;INFO  :   -47 :       22      -46        0.137  4mdq          &#xA;INFO  :   -48 :      -12      -47        0.151                    &#xA;INFO  :   -49 :       37      -48        0.155  4zyc          &#xA;INFO  :   -50 :       31      -49        0.156  4oq3          &#xA;INFO  :   -51 :       12      -50        0.158  4ere          &#xA;INFO  :   -52 :      -22       46        0.161            5lay&#xA;INFO  :   -53 :      -35      -51        0.182                    &#xA;INFO  :   -54 :       55      -53        0.186  6i29          &#xA;INFO  :   -55 :       10      -54        0.224  3w69          &#xA;INFO  :   -56 :       18      -55        0.238  4jve          &#xA;INFO  :   -57 :        9      -56        0.265  3vzv          &#xA;INFO  :   -58 :       28      -57        0.271  4ogn          &#xA;INFO  :   -59 :      -58        1        1.000            1rv1&#xA;INFO  :   -60 :       66      -59        1.000  7na1          &#xA;INFO  :   -61 :        0      -60        1.000                    &#xA;INFO  :   -62 :      -52      -61        1.000                    &#xA;INFO  :   -63 :       11      -62        1.000  4dij          &#xA;INFO  :   -64 :       67      -63        1.000  7na2          &#xA;INFO  :   -65 :       16      -64        1.000  4jv7          &#xA;INFO  :   -66 :       36      -65        1.000  4zgk          &#xA;INFO  :   -67 :        3      -66        1.000  2lzg          &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6717" LastEditorUserId="3848" LastEditDate="2022-11-12T12:11:54.687" LastActivityDate="2022-11-12T12:11:54.687" Title="Converting Pairwise single- linkage clustering distance data to &quot;newick&quot; format" Tags="|phylogenetics|phylogeny|ggtree|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20001" PostTypeId="2" ParentId="20000" CreationDate="2022-11-11T20:13:32.717" Score="2" Body="&lt;p&gt;The problem with the tree format presented is that it does not conform to a known standard. Whilst its possible to write a parser I don't think its a good idea.&lt;/p&gt;&#xA;&lt;p&gt;This output is hierarchical clustering, UPGMA, that sort of thing. Its not phylogenetics because this is a linkage tree.&lt;/p&gt;&#xA;&lt;p&gt;Can you access the distance matrix prior to the clustering? If so the information below the line will help.&lt;/p&gt;&#xA;&lt;p&gt;The output should provide the raw distance matrix prior to hierarchical clustering and will certainly have generated it in the calculation. BTW that particular clustering method isn't cool compared with nj (below) for complex reasons.&lt;/p&gt;&#xA;&lt;p&gt;The calculation will be:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Perform pairwise linkage analysis&lt;/li&gt;&#xA;&lt;li&gt;Perform cluster analysis on the resulting matrix&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;You need neighbor-joining to convert the distance matrix into a newick-format. There is no possibility of a direct conversion.&lt;/p&gt;&#xA;&lt;p&gt;If you are using &lt;code&gt;R&lt;/code&gt; its in the &lt;code&gt;ape&lt;/code&gt; library via&#xA;&lt;code&gt;treenk &amp;lt;- nj(M)&lt;/code&gt; with &lt;code&gt;M&lt;/code&gt; is the matrix. The treenk will be a format that can be read straight into &lt;code&gt;ggtree&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Briefly, a distance matrix requires a clustering algorithm and neighbor-joining is by far the best clustering method developed in phylogenetics.&lt;/p&gt;&#xA;&lt;p&gt;The following is a phylogenetics example ... this is different from the example you have 'cause thats about linkage.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data(mynucleotides)&#xA;tr &amp;lt;- nj(dist.dna(mynucleotides))&#xA;plot(tr)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I your case the final line will be something like &lt;code&gt;ggtree(tr)&lt;/code&gt; ... with various options.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-11-12T12:09:59.023" LastActivityDate="2022-11-12T12:09:59.023" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20002" PostTypeId="2" ParentId="19999" CreationDate="2022-11-12T16:08:34.623" Score="2" Body="&lt;p&gt;I used the unite function from tidyr but first I read the gmt file in as a data table using the read.delim function (with header set to False). Then I used unite:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyr)&#xA;&#xA;gmt_file &amp;lt;- read.delim(&amp;quot;~/path_to_file/file.gmt&amp;quot;, header = F)&#xA;gmt_file &amp;lt;- gmt_file %&amp;gt;% unite(genes,3:ncol(gmt_file), sep=&amp;quot;,&amp;quot;)&#xA;colnames(gmt_file) &amp;lt;- c(&amp;quot;pathway&amp;quot;, &amp;quot;source&amp;quot;, &amp;quot;genes&amp;quot;)&#xA;write.table.... &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9990" LastActivityDate="2022-11-12T16:08:34.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20003" PostTypeId="2" ParentId="19938" CreationDate="2022-11-13T00:15:06.313" Score="2" Body="&lt;p&gt;You can use &lt;a href=&quot;https://github.com/spacemanidol/Image2Smiles&quot; rel=&quot;nofollow noreferrer&quot;&gt;Image2Smiles&lt;/a&gt; tool. It produces smi format, then you can convert smi to pdb using &lt;a href=&quot;https://github.com/openbabel/openbabel&quot; rel=&quot;nofollow noreferrer&quot;&gt;openbabel&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you just have a few figures, I suggest using &lt;a href=&quot;https://pubchem.ncbi.nlm.nih.gov//edit3/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PubChem Sketcher&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6717" LastActivityDate="2022-11-13T00:15:06.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20004" PostTypeId="2" ParentId="19895" CreationDate="2022-11-13T00:28:09.900" Score="1" Body="&lt;p&gt;You can use &lt;a href=&quot;https://www.cgl.ucsf.edu/chimera/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chimera&lt;/a&gt; for this. The following code cleans your pdb file and write new pdb file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import chimera&#xA;from DockPrep import prep&#xA;from chimera import runCommand, openModels&#xA;&#xA;models = chimera.openModels.list(modelTypes=[chimera.Molecule])&#xA;prep(models)&#xA;runCommand(&amp;quot;write format pdb 0 molecule.pdb&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can run the code with&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; chimera --nogui file.pdb code.py&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6717" LastActivityDate="2022-11-13T00:28:09.900" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20005" PostTypeId="1" CreationDate="2022-11-13T03:36:20.293" Score="2" ViewCount="205" Body="&lt;p&gt;I am able to find matching motifs on a single sequence, as well as their position as shown below :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;demo = &amp;quot;ABCLXLYLZLABC&amp;quot;&#xA;motif=&amp;quot;L.L.L&amp;quot;&#xA;for element in re.finditer(motif,demo):&#xA;    print(element.start(),element.end,element.group())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This works and gives me the desired output which is:&#xA;&lt;code&gt; 3 8 LXLYL&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now I want to know how can I then do this same process but on a multifasta file.&#xA;As done in this example script from &lt;a href=&quot;https://www.biostars.org/p/14305/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/14305/&lt;/a&gt; .&lt;/p&gt;&#xA;&lt;p&gt;The protein sequences on the file are :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; P1&#xA;MPPRRSIVEVKVLDVQKRRVPNKHYVYIIRVTWSSGATEAIYRRYSKFFDLQMQMLDKFP&#xA;MEGGQKDPKQRIIPFLPGKILFRRSHIRDVAVKRLIPIDEYCKALIQLPPYISQCDEVLQ&#xA;FFETRPEDLNPPKEEHIGKKKSGNDPTSVDPMVLEQYVVVADYQKQESSEISLSVGQVVD&#xA;IIEKNESGWWFVSTAEEQGWVPATCLEGQDGVQDEFSLQPEEEEKYTVIYPYTARDQDEM&#xA;NLERGAVVEVVQKNLEGWWKIRYQGKEGWAPASYLKKNSGEPLPPKLGPSSPAHSGALDL&#xA;DGVSRHQNAMGREKELLNNQRDGRFEGRLVPDGDVKQRSPKMRQRPPPRRDMTIPRGLNL&#xA;&amp;gt;P2&#xA; MAEVRKFTKRLSKPGTAAELRQSVSEAVRGSVVLEKAKLVEPLDYENVITQRKTQIYSDP&#xA; LRDLLMFPMEDISISVIGRQRRTVQSTVPEDAEKRAQSLFVKECIKTYSTDWHVVNYKYE&#xA; DFSGDFRMLPCKSLRPEKIPNHVFEIDEDCEKDEDSSSLCSQKGGVIKQGWLHKANVNST&#xA; ITVTMKVFKRRYFYLTQLPDGSYILNSYKDEKNSKESKGCIYLDACIDVVQCPKMRRHAF&#xA; ELKMLDKYSHYLAAETEQEMEEWLIMLKKIIQINTDSLVQEKKDTVEAIQEEETSSQGKA&#xA; ENIMASLERSMHPELMKYGRETEQLNKLSRGDGRQNLFSFDSEVQRLDFSGIEPDVKPFE&#xA; EKCNKRFMVNCHDLTFNILGHIGDNAKGPPTNVEPFFINLALFDVKNNCKISADFHVDLN&#xA; PPSVREMLWGTSTQLSNDGNAKGFSPESLIHGIAESQLCYIKQGIFSVTNPHPEIFLVVR&#xA;&amp;gt;P3&#xA;GDDSEWLKLPVDQKCEHKLWKARLSGYEEALKIFQKIKDEKSPEWSKYLGLIKKFVTDS&#xA;NAVVQLKGLEAALVYVENAHVAGKTTGEVVSGVVSKVFNQPKAKAKELGIEICLMYVEIE&#xA;KGESVQEELLKGLDNKNPKIIVACIETLRKALSEFGSKIISLKPIIKVLPKLFESRDKAV&#xA;RDEAKLFAIEIYRWNRDAVKHTLQNINSVQLKELEEEWVKLPTGAPKPSRFLRSQQELEA&#xA;KLEQQQSAGGDAEGGGDDGDEVPQVDAYELLDAVEILSKLPKDFYDKIEAKKWQERKEAL&#xA;EAVEVLVKNPKLEAGDYADLVKALKKVVGKDTNVMLVALAAKCLTGLAVGLRKKFGQYAG&#xA;HVVPTILEKFKEKKPQVVQALQEAIDAIFLTTTLQNISEDVLAVMDNKNPTIKQQTSLFI&#xA;ARSFRHCTSSTLPKSLLKPFCAALLKHINDSAPEVRDAAFEALGTALKVVGEKSVNPFLA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The script that was used to find the motif is below, the problem now is that it does not show the position where the motif is found. Could you assist in how one could do so.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;infile=open(&amp;quot;seq.fasta&amp;quot;,'r')&#xA;out=open(&amp;quot;results.csv&amp;quot;,'w')&#xA;pattern=re.compile(r&amp;quot;(P[A-Z]{2}P)&amp;quot;)&#xA;for line in infile:&#xA;  line = line.strip(&amp;quot;\n&amp;quot;)&#xA;  if line.startswith('&amp;gt;'):&#xA;     name=line&#xA;else:      &#xA;   s = re.findall(pattern,line)&#xA;   print '%s:%s' %(name,s)&#xA;   out.write('%s:\t%s\n' %(name,s))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Their script seems to have worked because the desired output was&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;p1: &#xA;PGCP&#xA;&#xA;&amp;gt;p1: &#xA;PHCP,&#xA;PKCP&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The problem with their script is that, it does not provide the position where the sequence was found. Could you please assist on how that script can be modified? I want to try it but I want my output to have (1) Name Of Sequence, (2) The Motif Found , (3) The Position&lt;/p&gt;&#xA;" OwnerUserId="16468" LastActivityDate="2024-04-06T21:01:15.150" Title="finding motifs in fasta file with multiple sequences" Tags="|proteins|python|motifs|multi-fasta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20006" PostTypeId="2" ParentId="20005" CreationDate="2022-11-13T04:49:33.200" Score="0" Body="&lt;p&gt;This is easy just replace &lt;code&gt;findall&lt;/code&gt; with &lt;code&gt;finditer&lt;/code&gt; in the above code. The index will then appear where the match first appears and where it stops. However, you will to parse the &lt;code&gt;finditer&lt;/code&gt; object which in the code is &lt;code&gt;s&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then replace&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print '%s:%s' %(name,s)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for match_obj in s:&#xA;    # print each re.Match object&#xA;    print(match_obj)&#xA;    &#xA;    # extract each matching number&#xA;    print(match_obj.group())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That should work. When it does you simply pass the &lt;code&gt;match_obj&lt;/code&gt; and &lt;code&gt;match_obj.group()&lt;/code&gt; to a file by replacing &lt;code&gt;s&lt;/code&gt; with &lt;code&gt;match_obj, match_obj.group()&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thus replace&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  out.write('%s:\t%s\n' %(name,s))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  out.write('%s:\t%s\t%s\n' %(name,match_obj,match_obj.group()))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You might need to do an append BTW.&lt;/p&gt;&#xA;&lt;p&gt;thus replace&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;out=open(&amp;quot;results.csv&amp;quot;,'w')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;if os.path.exists(&amp;quot;results.csv&amp;quot;):&#xA;    os.unlink (&amp;quot;results.csv&amp;quot;)&#xA;out=open(&amp;quot;results.csv&amp;quot;,'a')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-11-13T16:34:42.213" LastActivityDate="2022-11-13T16:34:42.213" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20007" PostTypeId="1" CreationDate="2022-11-13T05:28:52.690" Score="2" ViewCount="60" Body="&lt;p&gt;What will be the best tool to call isoforms from long read data generated from partially degraded RNA. By mistake we processed some samples with poor quality RNA to generate long read. Now we are looking for some tool which can merged the reads for the potential degraded transcripts. I think majority of the available tools like Squanti, Talon, bamboo, flair considered independent read as full length. Do the community has any suggestions?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4951" LastActivityDate="2022-11-14T15:01:39.223" Title="Calling isoforms from long read data generated from partially degraded RNA" Tags="|rna-seq|assembly|long-reads|isoform|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20009" PostTypeId="2" ParentId="19984" CreationDate="2022-11-13T16:12:45.023" Score="1" Body="&lt;p&gt;There is nothing wrong with the concept of the code, that is exactly how I would do it. The problem is here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    else:&#xA;output.write(line)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Corrections&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;output = 'newseq.fasta'&#xA;if os.path.exists(output):&#xA;    os.unlink(output) &#xA;&#xA;for line in fasta:&#xA;    if line.startswith('&amp;gt;'): # getting just the header&#xA;        header= line.strip().split(&amp;quot;|&amp;quot;) # there are several entries on the header separated by the |&#xA;        for name in one:&#xA;           if name in header: &#xA;              line = header.insert(0,line.pop(header.index(name))) # insert the name from file1 as the first header entry&#xA;        else: # if no match in file1 look at file2&#xA;            for name in two:&#xA;               if name in header:&#xA;                    line = header.insert(0,line.pop(header.index(name))) &#xA;&#xA;# The indentation must be absolutely correct &#xA;# otherwise the code falls over. Must be one indent from&#xA;# the first for statement&#xA;    with open(output, 'a') as fout:&#xA;        fout.write(line)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I assume that &lt;code&gt;fasta&lt;/code&gt;, &lt;code&gt;one&lt;/code&gt; and &lt;code&gt;two&lt;/code&gt; are loaded successfully.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-11-13T16:19:09.203" LastActivityDate="2022-11-13T16:19:09.203" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20010" PostTypeId="1" AcceptedAnswerId="20012" CreationDate="2022-11-13T19:54:31.517" Score="2" ViewCount="258" Body="&lt;p&gt;I am having trouble trying to search for motifs in a multi fasta file. I have used two techniques one gives me the name of the sequence where the motif is found but doesn't give me the the motif and its position. The other method does not return anything. Please see the code below, any assistance will be highly appreciated.&lt;/p&gt;&#xA;&lt;p&gt;This is the protein sequence:&#xA;(Please note I have only put in the first 3 sequences, the actual file I am working with has 100 sequences)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;P1&#xA;MSDNGPQNQRNAPRITFGGPSDSTGSNQNGERSGARSKQRRPQGLPNNTASWFTALTQHG&#xA;KEDLKFPRGQGVPINTNSSPDDQIGYYRRATRRIRGGDGKMKDLSPRWYFYYLGTGPEAG&#xA;LPYGANKDGIIWVATEGALNTPKDHIGTRNPANNAAIVLQLPQGTTLPKGFYAEGSRGGS&#xA;QASSRSSSRSRNSSRNSTPGSSRGISPARMAGNGGDAALALLLLDRLNQLESKMSGKGQQ&#xA;QQGQXXTXKSAAEASKKPRQKRTATKAYNVTQAFGRRGPEQTQGNFGDQELIRQGTDYKH&#xA;WPQIAQFAPSASAFFGMSRIGMEVTPSGTWLTYTGAIKLDDKDPNFKDQVILLNKHIDAY&#xA;KTFPPTEPKKDKKKKADETQALPQRQKKQQTVTLLPAADLDDFSKQLQQSMSSADSTQA*&#xA;&#xA;&amp;gt;P2&#xA;MSDNGPQNQRNAPRITFGGPSDSTGSNQNGERSGARSKQRRPQGLPNNTASWFTALTQHG&#xA;KEDLKFPRGQGVPINTNSSPDDQIGYYRRATRRIRGGDGKMKDLSPRWYFYYLGTGPEAG&#xA;LPYGANKDGIIWVATEGALNTPKDHIGTRNPANNAAIVLQLPQGTTLPKGFYAEGSRGGS&#xA;QASSRSSSRSRNSSRNSTPGSSRGISPARMAGNGGDAALALLLLDRLNQLESKMSGKGQQ&#xA;QQGQTVTKKSAAEASKKPRQKRTATKAYNVTQAFGRRGPEQTQGNFGDQELIRQGTDYKH&#xA;WPQIAQFAPSASAFFGMSRIGMEVTPSGTWLTYTGAIKLDDKDPNFKDQVILLNKHIDAY&#xA;KTFPPTEPKKDKKKKADETQALPQRQKKQQTVTLLPAADLDDFSKQLQQSMSSADSTQA*&#xA;&#xA;&amp;gt;P3&#xA;MSDNGPQNQRNAPRITFGGPSDSTGSNQNGERSGARSKQRRPQGLPNNTASWFTALTQHG&#xA;KEDLKFPRGQGVPINTNSSPDDQIGYYRRATRRIRGGDGKMKDLSPRWYFYYLGTGPEAG&#xA;LPYGANKDGIIWVATEGALNTPKDHIGTRNPANNAAIVLQLPQGTTLPKGFYAEGSRGGS&#xA;QASSRSSSRSRNSSRNSTPGSSRGISPARMAGNGGDAALALLLLDRLNQLESKMSGKGQQ&#xA;QQGQTVTKKSAAEASKKPRQKRTATKAYNVTQAFGRRGPEQTQGNFGDQELIRQGTDYKH&#xA;WPQIAQFAPSASAFFGMSRIGMEVTPSGTWLTYTGAIKLDDKDPNFKDQVILLNKHIDAY&#xA;KTFPPTEPKKDKKKKADETQALPQRQKKQQTVTLLPAADLDDFSKQLQQSMSSADSTQA*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The first one is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;infile=open(&amp;quot;sequence.fasta&amp;quot;,'r')&#xA;out=open(&amp;quot;Result.csv&amp;quot;,'w')&#xA;pattern=re.compile(r&amp;quot;[YFWLIMVA]..[LMALVN].[AGSTCD]..[LAIVNFYMW]&amp;quot;)&#xA;for line in infile:&#xA;  line = line.strip(&amp;quot;\n&amp;quot;)&#xA;  if line.startswith('&amp;gt;'):&#xA;     name=line&#xA;else:&#xA;   s=re.finditer(pattern,line)&#xA;   print('%s:%s' %(name,s))&#xA;   out.write('%s:\t%s\n' %(name,s))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This one above returns, the output below. It gives me the name of the sequence which is P100, but it does not give me (1) The motif found (2) the position of the motif. And the Results.csv file that it generates is blank.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#output&#xA;&amp;gt;P100:&amp;lt;callable_iterator object at 0x000001D0611B7AF0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then tried a different technique:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;infile=open(&amp;quot;sequence.fasta&amp;quot;,'r')&#xA;open=open(&amp;quot;result.csv&amp;quot;,'w')&#xA;pattern=re.compile(r&amp;quot;(P[A-Z]{2}P&amp;quot;)&#xA;for line in infile:&#xA;  line=line.strip(&amp;quot;\n&amp;quot;)&#xA;  if line.startswith('&amp;gt;'):&#xA;     name=line&#xA;else:&#xA;   s = re.finditer(pattern,line)&#xA;   for match_obj in s:&#xA;       print(match_obj)&#xA;       print(match_obj.group())&#xA;  out.write('%s:\t%s\t%s\n' %(name, match_obj,match_obj.group()))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The one above does not return anything at all. the Results.csv file is also blank.&#xA;I'm still new in this python, and some of the techniques I used here I found on bioinformatics stackexchange.&lt;/p&gt;&#xA;&lt;p&gt;The output I desire, is where I can get (1) Name of sequence, (2) Motif found and (3) position of motif (where it starts and ends) as shown in the example below in the context of every sequence in the multi fasta file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;P100&#xA;motif:&#xA;position:&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any format is fine as long as  I can get something similar to the format above. Could I also be assistet with   the indention as that cofuses me.&lt;/p&gt;&#xA;" OwnerUserId="16468" LastEditorUserId="16468" LastEditDate="2022-11-14T10:12:34.377" LastActivityDate="2023-10-30T00:08:47.043" Title="motif searching in a multi-fasta file" Tags="|proteins|python|motifs|multi-fasta|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20011" PostTypeId="2" ParentId="20007" CreationDate="2022-11-14T11:53:49.947" Score="1" Body="&lt;p&gt;You could have a go using LAST. I've been developing and using a protocol for mapping cDNA reads that is based around LAST-mapping since 2017. The most recent RY model addition in LAST seems to clean up the majority of the random gene counting noise issues that we were having.&lt;/p&gt;&#xA;&lt;p&gt;See the gene counting part of the protocol here. If you're working from direct/native RNA rather than cDNA (or don't care about strand-specific mapping), then the strand correction in previous steps is not needed:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://dx.doi.org/10.17504/protocols.io.5qpvonn2bl4o/v17&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://dx.doi.org/10.17504/protocols.io.5qpvonn2bl4o/v17&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-11-14T11:53:49.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20012" PostTypeId="2" ParentId="20010" CreationDate="2022-11-14T12:40:17.207" Score="3" Body="&lt;p&gt;First of all, you can't do this this way. Fasta sequences can, and indeed yours do, span multiple lines so in order to find a pattern, you need to join those lines into one to make sure you don't miss any that are found across more than one line.&lt;/p&gt;&#xA;&lt;p&gt;Next, as a general rule, it is a bad idea to hardcode output file names. That means the user cannot control the output, and that any existing files will be overwritten, so just print to standard output and redirect to a file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;foo.py &amp;gt; out_file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You also don't want to hardcode the name of the input file since that just makes the script harder to use. Instead, read the input file name as an argument. With all this in mind, here is a working version of your script:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;import re&#xA;import sys&#xA;&#xA;def print_matches(seq_name, matches):&#xA;     if matches:&#xA;          for match in matches:&#xA;               print(name,match.start(), match.end(),match.group(),sep='\t' )&#xA;      &#xA;&#xA;pattern = re.compile(r&amp;quot;[YFWLIMVA]..[LMALVN].[AGSTCD]..[LAIVNFYMW]&amp;quot;)&#xA;seq = &amp;quot;&amp;quot;&#xA;&#xA;try:&#xA;     with open(sys.argv[1],'r') as infile:&#xA;          for line in infile:&#xA;               line = line.strip(&amp;quot;\n&amp;quot;)&#xA;               if line.startswith('&amp;gt;'):&#xA;                    name=line.lstrip('&amp;gt;')&#xA;                    if seq != &amp;quot;&amp;quot;:&#xA;                         matches = re.finditer(pattern, seq)&#xA;                         print_matches(name, matches)&#xA;                         seq = &amp;quot;&amp;quot;&#xA;               else:&#xA;                    seq += line&#xA;          else:&#xA;               matches = re.finditer(pattern, seq)&#xA;               print_matches(name, matches)&#xA;                    &#xA;except IOError as e:&#xA;     print(&amp;quot;Failed to open file %s with error %s&amp;quot; % (sys.argv[1], e))&#xA;     sys.exit(1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Running that on your example gives:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ foo.py sequence.fasta &#xA;P2  44  53  LPNNTASWF&#xA;P2  122 131 YGANKDGII&#xA;P2  209 218 MAGNGGDAA&#xA;P2  219 228 ALLLLDRLN&#xA;P2  304 313 AQFAPSASA&#xA;P2  391 400 VTLLPAADL&#xA;P2  410 419 MSSADSTQA&#xA;P3  44  53  LPNNTASWF&#xA;P3  122 131 YGANKDGII&#xA;P3  209 218 MAGNGGDAA&#xA;P3  219 228 ALLLLDRLN&#xA;P3  304 313 AQFAPSASA&#xA;P3  391 400 VTLLPAADL&#xA;P3  410 419 MSSADSTQA&#xA;P3  44  53  LPNNTASWF&#xA;P3  122 131 YGANKDGII&#xA;P3  209 218 MAGNGGDAA&#xA;P3  219 228 ALLLLDRLN&#xA;P3  304 313 AQFAPSASA&#xA;P3  391 400 VTLLPAADL&#xA;P3  410 419 MSSADSTQA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output is 4 tab-separated columns per match with the sequence name, the match's start position, its end position and the actual sequence matched.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2022-11-14T12:40:17.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20013" PostTypeId="2" ParentId="20007" CreationDate="2022-11-14T15:01:39.223" Score="2" Body="&lt;p&gt;I haven't used these long-read RNA-seq tools so I could be wrong here. Transcript reconstruction combines reads from the same transcript and should be robust to partial transcripts. In addition, Nanopore direct RNA-seq produces a lot of partial transcripts. I tend to believe many transcript reconstruction tools can deal with them.&lt;/p&gt;&#xA;&lt;p&gt;As to what tool to use, &lt;a href=&quot;https://github.com/gpertea/stringtie&quot; rel=&quot;nofollow noreferrer&quot;&gt;StringTie&lt;/a&gt; is a popular one and has been shown to be better than Flair. &lt;a href=&quot;https://github.com/ablab/IsoQuant&quot; rel=&quot;nofollow noreferrer&quot;&gt;IsoQuant&lt;/a&gt; seems promising. Its developers &lt;a href=&quot;https://www.researchsquare.com/article/rs-1571850/v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;argued&lt;/a&gt; it is more accurate than StringTie, Bambu, Talon and Flair. Squanti only works for iso-seq.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2022-11-14T15:01:39.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20017" PostTypeId="1" CreationDate="2022-11-16T15:00:42.667" Score="1" ViewCount="17" Body="&lt;p&gt;My group is interested in genetically engineering mice with label for the Slc45a2 gene, since &amp;quot;the protein is expressed in a high percentage of melanoma cell lines&amp;quot;. (from Refseq)&#xA;How can I choose with high confidence the best isoform to use, i.e., the one that is most upregulated in melanoma cell lines?&lt;/p&gt;&#xA;&lt;p&gt;Is there a curated database/ web-source of any kind that I can look it up?&lt;/p&gt;&#xA;" OwnerUserId="16054" LastEditorUserId="3694" LastEditDate="2022-11-16T16:57:39.790" LastActivityDate="2022-11-16T16:57:39.790" Title="Good source for specific Isoforms overexpression in tumor cells (specifically, melanoma)" Tags="|isoform|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20018" PostTypeId="2" ParentId="20010" CreationDate="2022-11-16T22:49:00.737" Score="1" Body="&lt;p&gt;Read the sequences with Biopython, &lt;a href=&quot;https://edinburgh-genome-foundry.github.io/DnaChisel/ref/biotools.html?highlight=reverse#dnachisel.biotools.reverse_translate&quot; rel=&quot;nofollow noreferrer&quot;&gt;reverse translate&lt;/a&gt; the sequences and the pattern peptides, construct pattern regexes, search (&lt;code&gt;find_matches()&lt;/code&gt;) for the &lt;a href=&quot;https://edinburgh-genome-foundry.github.io/DnaChisel/ref/core_classes.html?highlight=sequencepattern#dnachisel.SequencePattern&quot; rel=&quot;nofollow noreferrer&quot;&gt;patterns&lt;/a&gt; for each sequence in a loop using &lt;a href=&quot;https://edinburgh-genome-foundry.github.io/DnaChisel/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DNA Chisel&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Disclaimer: I'm the maintainer of DNA Chisel&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="208" LastActivityDate="2022-11-16T22:49:00.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20019" PostTypeId="2" ParentId="10781" CreationDate="2022-11-17T01:05:42.210" Score="2" Body="&lt;p&gt;If you prefer to have a more concise report than a per-base one, &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/genomecov.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools genomecov&lt;/a&gt; could be a better choice.&lt;/p&gt;&#xA;&lt;p&gt;Its &lt;code&gt;-bga&lt;/code&gt; option allows you to report consecutive bases that have the same coverage as a range in a single line.&lt;/p&gt;&#xA;&lt;p&gt;In your case, you could achieve this (unfortunately they only offer the strandedness option for a specified strand) by running the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bedtools genomecov -bga -ibam aln.bam -g chr1.bed -strand + &amp;gt; coverage.perposition_chr1.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output will look like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1  554304  554309  5&#xA;chr1  554309  554313  6&#xA;chr1  554313  554314  1&#xA;chr1  554315  554316  6&#xA;chr1  554316  554317  5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16501" LastActivityDate="2022-11-17T01:05:42.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20020" PostTypeId="2" ParentId="19331" CreationDate="2022-11-17T09:49:53.257" Score="2" Body="&lt;p&gt;in the end, I just loaded the &lt;code&gt;.bam&lt;/code&gt; files into &lt;a href=&quot;https://www.sanger.ac.uk/tool/artemis/&quot; rel=&quot;nofollow noreferrer&quot;&gt;artemis&lt;/a&gt; and, by inspecting the depth (heat-map), I could check which samples had the genes I was looking for:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/U2PqZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/U2PqZ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3452" LastActivityDate="2022-11-17T09:49:53.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20021" PostTypeId="1" AcceptedAnswerId="20022" CreationDate="2022-11-17T10:22:26.767" Score="2" ViewCount="35" Body="&lt;p&gt;I am able to find matching motif in my sequence, and I would like to now find overlapping motifs. Basically, after matching my motif, I want to find the 6 amino acids after it. This is the code below that I used to find the motif:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import Bio&#xA;import regex&#xA;&#xA;from Bio import SeqIO&#xA;input_file = 'sequences.fasta'&#xA;fasta_sequences = SeqIO.parse(open(input_file),'fasta')&#xA;for fasta in fasta_sequences:&#xA;     name, sequence = fasta.id, str(fasta.seq)&#xA;     result=regex.finditer(r&amp;quot;[YFWLIMVA]..[LMALVN]..[AGSTCD].[LAIVNFYMW]&amp;quot;,sequence)&#xA;     for x in result:&#xA;     print(name, x.start(), x.end(), x.group())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above code works perfectly becasue it give me the sequence id, positions and the motif. The output is below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;P1  33 41 VTLLPAADL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Right now, what I want to do is to also get the 6 amino acids after matching this motif, such that I get an output like the one below where the length will now be 15,.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;P1 33 47 VTLLPAADLLMAIID&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The code that I have tried to get the 6 amino acids after my match is below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import Bio&#xA;import regex&#xA;&#xA;from Bio import SeqIO&#xA;input_file = 'sequences.fasta'&#xA;fasta_sequences = SeqIO.parse(open(input_file),'fasta')&#xA;for fasta in fasta_sequences:&#xA;     name, sequence = fasta.id, str(fasta.seq)&#xA;     result=regex.finditer(r&amp;quot;[YFWLIMVA]..[LMALVN]..[AGSTCD].[LAIVNFYMW]&amp;quot;,sequence)&#xA;     for x in result:&#xA;     print(name, x.start(), x.end() + 6, x.group())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This the output it gives me:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#It does not extend my motif by 6 amino acids, after getting the match, it only extends the &#xA;#end position by 6. From 41 to 47.     &#xA;P1  33 47 VTLLPAADL &#xA;&#xA;#My desired output is this which include the overlapping LMAIID, the LMAIID is after the &#xA;#match.&#xA;P1   33 47 VTLLPAADLLMAIID&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also tried the code below, but it returns an error.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import Bio&#xA;import regex&#xA;&#xA;from Bio import SeqIO&#xA;input_file = 'sequences.fasta'&#xA;fasta_sequences = SeqIO.parse(open(input_file),'fasta')&#xA;for fasta in fasta_sequences:&#xA;     name, sequence = fasta.id, str(fasta.seq)&#xA;     result=regex.finditer(r&amp;quot;[YFWLIMVA]..[LMALVN]..[AGSTCD].[LAIVNFYMW]&amp;quot;,sequence)&#xA;     for x in result:&#xA;     print(name, x.start(), x.end() + 6, x.group() +6)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16468" LastActivityDate="2022-11-17T12:02:04.597" Title="finding overlapping motifs to increase length of motif" Tags="|proteins|python|motifs|multi-fasta|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20022" PostTypeId="2" ParentId="20021" CreationDate="2022-11-17T10:47:50.243" Score="1" Body="&lt;p&gt;This is how it managed to give the desired output.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import Bio&#xA;import re&#xA;&#xA;from Bio import SeqIO&#xA;input_file = 'sequences.fasta'&#xA;fasta_sequences = SeqIO.parse(open(input_file),'fasta')&#xA;for fasta in fasta_sequences:&#xA;     name, sequence = fasta.id, str(fasta.seq)&#xA;     matches=re.finditer(r&amp;quot;[YFWLIMVA]..[LMALVN].[AGSTCD]..[LAIVNFYMW]&amp;quot;,sequence)&#xA;     for x in matches:&#xA;     print(name, x.start(), x.end()+6,sequence[x.start():x.end()+6])&#xA;&#xA;#Output&#xA;P1 33 47 VTLLPAADLLMAIID&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16468" LastActivityDate="2022-11-17T10:47:50.243" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20023" PostTypeId="1" AcceptedAnswerId="20059" CreationDate="2022-11-17T11:06:07.663" Score="1" ViewCount="163" Body="&lt;p&gt;I am running a Snakemake (V7.14.0) pipeline on the cluster and provided resources in a resource config file like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;threads:&#xA;    tool1: 4&#xA;mem_gb:&#xA;    tool1: 12&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Point to it in my rules like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;resources:&#xA;        mem_gb=config['mem_gb']['tool1']&#xA;threads:&#xA;        config['threads']['tool1']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the snakemake cluster command like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;snakemake --configfiles \&#xA;    config/config_resources.yml \&#xA;    config/config_parameters.yml \&#xA;    --latency-wait 60 \&#xA;    --use-conda \&#xA;    --use-singularity \&#xA;    --printshellcmds \&#xA;    --jobname MyPipe{name}_{jobid} \&#xA;    --jobs 300 \&#xA;    --cluster \&#xA;    &amp;quot;bsub -W 200 -n {threads} -M {resources.mem_gb}G \&#xA;    -R 'rusage[mem={resources.mem_gb}G]' \&#xA;    -o run_logs/{name}_%J.out \&#xA;    -e run_logs/{name}_%J.err&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What I do not understand when I run the pipeline is that it shows this in the terminal:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;threads: 4&lt;/p&gt;&#xA;&lt;p&gt;resources: mem_mb=19442, disk_mb=19442, tmpdir=/tmp, mem_gb=12&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So next to the one I provided (mem_gb), there are: mem_mb, and disk_mb. They are clearly not the same value. In the cluster job it looks as though the MEMLIMIT is indeed the one I give, but the MEM usage is different (higher). Is it because Snakemake does not understand the mem_gb and I specifically need to use that word?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="16204" LastEditorUserId="16204" LastEditDate="2022-11-22T13:42:40.110" LastActivityDate="2022-11-22T14:53:20.653" Title="Multiple values for memory resources shown when Snakemake is run" Tags="|snakemake|hpc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20024" PostTypeId="2" ParentId="20021" CreationDate="2022-11-17T12:02:04.597" Score="0" Body="&lt;p&gt;Just add 6 extra aminoacids to your motif:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;matches=re.finditer(r&amp;quot;[YFWLIMVA]..[LMALVN].[AGSTCD]..[LAIVNFYMW]......&amp;quot;,sequence)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That's all you need, now the existing code will give you the answer you want.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2022-11-17T12:02:04.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20025" PostTypeId="1" AcceptedAnswerId="21556" CreationDate="2022-11-17T15:21:09.437" Score="2" ViewCount="353" Body="&lt;p&gt;ETE3 &lt;strong&gt;*&lt;/strong&gt; (Environment for Tree Exploration) is a major package and in particular Python package to navigate phylogenetic trees. It is the premier coding library/package for tree manipulation.&lt;/p&gt;&#xA;&lt;p&gt;ETE4.0.0 beta has been &lt;a href=&quot;https://github.com/etetoolkit/ete/releases/tag/4.0.0-beta&quot; rel=&quot;nofollow noreferrer&quot;&gt;released&lt;/a&gt; via GitHub. The download contains an old ETE manual (2009) and a README file for ETE4, thats about it. Does anyone know the location of the new ETE4 manual, or list of functionality beyond what is in the README (posted below). Reading the code is the last resort ;-)&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;README ETE4&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Read and write support for trees in Newick format&lt;/li&gt;&#xA;&lt;li&gt;Multiple functions for traversing, searching and manipulating tree topologies and node annotations&lt;/li&gt;&#xA;&lt;li&gt;Integration with NCBI Taxonomic database&lt;/li&gt;&#xA;&lt;li&gt;Integration with GTDB database&lt;/li&gt;&#xA;&lt;li&gt;Programmatic visualization framework&lt;/li&gt;&#xA;&lt;li&gt;Visualization of huge trees based on adaptive zooming&lt;/li&gt;&#xA;&lt;li&gt;Comparing trees&lt;/li&gt;&#xA;&lt;li&gt;Phylogenetic functions:&#xA;&lt;ul&gt;&#xA;&lt;li&gt;orthology detection&lt;/li&gt;&#xA;&lt;li&gt;phylogenetic distance?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Reference&lt;/strong&gt;&#xA;&lt;strong&gt;*&lt;/strong&gt; ETE 3: Reconstruction, analysis and visualization of phylogenomic data.&#xA;Jaime Huerta-Cepas, Francois Serra and Peer Bork.&#xA;Mol Biol Evol 2016; &lt;a href=&quot;https://doi:%2010.1093/molbev/msw046&quot; rel=&quot;nofollow noreferrer&quot;&gt;doi: 10.1093/molbev/msw046&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-09-16T13:15:02.733" LastActivityDate="2023-09-16T13:15:02.733" Title="ETE4 manual or improved code description" Tags="|python|phylogenetics|phylogeny|ete3|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20027" PostTypeId="1" AcceptedAnswerId="20038" CreationDate="2022-11-17T18:45:42.240" Score="2" ViewCount="3200" Body="&lt;p&gt;I have two csv formatted data frames that I want to merge. First data frame contains 9 columns and the second contains three columns. Based on column names: start, end, function I am trying to merge the two data frames. Following is what I tried:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df1&amp;lt;-read.csv(file = &amp;quot;LuC_A_TE_gff.csv&amp;quot;)&#xA;df2&amp;lt;-read.csv(file = &amp;quot;LuC_A_TE_1kb_range_ids.csv&amp;quot;)&#xA;merged.data&amp;lt;-merge(df1, df2, by.x=c(&amp;quot;start&amp;quot;,&amp;quot;end&amp;quot;,&amp;quot;function&amp;quot;), by.y=c(&amp;quot;start&amp;quot;,&amp;quot;end&amp;quot;,&amp;quot;function&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error in fix.by(by.x, x) : 'by' must specify a uniquely valid column.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;df1&#xA; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr,RM,sim,start,end,stat,strand,id,function&#xA;Chr01A,RepeatMasker,similarity,1,1122,4438,+,.,Target=rnd-4_family-4692 2 652;Class=LTR;PercDiv=4.9;PercDel=9.9;PercIns=5.4;&#xA;Chr01A,RepeatMasker,similarity,1123,2001,6668,+,.,Target=25:218931..219866_LTR 171 936;Class=LTR/Gypsy;PercDiv=6.5;PercDel=6.5;PercIns=0.7;&#xA;Chr01A,RepeatMasker,similarity,1998,2421,3164,+,.,Target=rnd-4_family-4692 102 559;Class=LTR;PercDiv=6.1;PercDel=8.0;PercIns=0.0;&#xA;Chr01A,RepeatMasker,similarity,2002,2119,1182,+,.,Target=rnd-5_family-5429 198 324;Class=DNA/Tc1-Mariner;PercDiv=5.1;PercDel=5.7;PercIns=0.0;&#xA;Chr01A,RepeatMasker,similarity,2284,3113,6265,+,.,Target=25:218931..219866_LTR 32 908;Class=LTR/Gypsy;PercDiv=4.6;PercDel=6.4;PercIns=0.7;&#xA;Chr01A,RepeatMasker,similarity,3003,3103,1238,+,.,Target=rnd-5_family-5429 159 202;Class=DNA/Tc1-Mariner;PercDiv=9.1;PercDel=5.5;PercIns=1.6;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;start,end,function&#xA;10000060,10000222,Target=rnd-6_family-1538 599 686;Class=DNA/CMC-EnSpm;PercDiv=24.8;PercDel=1.1;PercIns=3.2;&#xA;10000190,10000200,Target=33:34356884..34364107_INT-int 202 270;Class=LTR;PercDiv=20.5;PercDel=5.9;PercIns=1.6;&#xA;10000201,10000302,Target=rnd-6_family-3392 448 555;Class=DNA;PercDiv=10.9;PercDel=6.9;PercIns=0.9;&#xA;10000223,10000356,Target=rnd-6_family-1538 196 343;Class=DNA/CMC-EnSpm;PercDiv=20.3;PercDel=11.2;PercIns=0.7;&#xA;10000312,10000423,Target=fa.05_5_22233 5 115;Class=DNA;PercDiv=3.6;PercDel=0.0;PercIns=0.9;&#xA;10001256,10001332,Target=rnd-4_family-2291 200 273;Class=DNA;PercDiv=17.6;PercDel=0.0;PercIns=4.0;&#xA;10001590,10001909,Target=rnd-4_family-274 376 722;Class=DNA/CMC-EnSpm;PercDiv=12.3;PercDel=9.4;PercIns=0.9;&#xA;10001650,10001757,Target=rnd-6_family-7385 84 183;Class=DNA;PercDiv=16.3;PercDel=0.9;PercIns=9.0;&#xA;10002071,10002421,Target=rnd-3_family-11 116 430;Class=DNA/hAT-Ac;PercDiv=4.2;PercDel=0.0;PercIns=11.4;&#xA;10002933,10003171,Target=rnd-5_family-572 47 262;Class=DNA;PercDiv=17.4;PercDel=4.6;PercIns=15.7;&#xA;10003024,10003085,Target=Chr23:11562195..11564516 1491 1552;Class=RC/Helitron;PercDiv=4.8;PercDel=0.0;PercIns=0.0;&#xA;10003030,10003207,Target=rnd-4_family-825 65 227;Class=DNA;PercDiv=12.9;PercDel=7.3;PercIns=17.2;&#xA;10003132,10003142,Target=rnd-5_family-1284 903 950;Class=DNA/Kolobok;PercDiv=16.7;PercDel=7.7;PercIns=2.4;&#xA;10003143,10003941,Target=fa.24_3_1586 1 753;Class=DNA;PercDiv=17.3;PercDel=4.8;PercIns=11.2;&#xA;10003209,10003451,Target=rnd-5_family-502 491 715;Class=DNA/IS3EU;PercDiv=19.6;PercDel=7.4;PercIns=16.0;&#xA;10003271,10003496,Target=rnd-6_family-3401 557 818;Class=DNA/IS3EU;PercDiv=22.4;PercDel=19.5;PercIns=3.0;&#xA;10003414,10003567,Target=rnd-1_family-1158 5 166;Class=Unknown;PercDiv=25.1;PercDel=9.7;PercIns=4.3;&#xA;10003494,10003560,Target=rnd-4_family-825 2 63;Class=DNA;PercDiv=13.7;PercDel=4.5;PercIns=12.9;&#xA;10003631,10003854,Target=rnd-1_family-378 1 406;Class=DNA/P;PercDiv=4.2;PercDel=0.0;PercIns=0.1;&#xA;10004081,10004200,Target=fa.00_5_3944 125 245;Class=DNA;PercDiv=10.8;PercDel=0.8;PercIns=0.0;&#xA;10004147,10004282,Target=fa.13_3_23430 1409 1547;Class=DNA/hAT;PercDiv=6.7;PercDel=2.9;PercIns=0.7;&#xA;10004172,10004199,Target=fa.02_2_13330 434 524;Class=DNA;PercDiv=16.6;PercDel=3.1;PercIns=7.6;&#xA;10004200,10004427,Target=rnd-1_family-377 2 229;Class=DNA;PercDiv=11.3;PercDel=3.1;PercIns=3.1;&#xA;10004279,10004320,Target=fa.13_3_23430 882 923;Class=DNA/hAT;PercDiv=6.8;PercDel=0.0;PercIns=0.0;&#xA;10004321,10004656,Target=fa.15_1_6197 348 728;Class=DNA/Tc1-Mariner;PercDiv=13.1;PercDel=13.4;PercIns=0.0;&#xA;10004428,10004480,Target=fa.02_2_13330 525 709;Class=DNA;PercDiv=16.6;PercDel=3.1;PercIns=7.6;&#xA;10004482,10004804,Target=rnd-1_family-77 1 331;Class=SINE/tRNA-Core-RTE;PercDiv=7.5;PercDel=4.0;PercIns=1.5;&#xA;10004515,10004566,Target=rnd-6_family-2042 304 370;Class=DNA/TcMar-Tc1;PercDiv=7.7;PercDel=28.9;PercIns=0.0;&#xA;10004658,10005425,Target=rnd-5_family-3762 370 1163;Class=DNA/hAT-Ac;PercDiv=7.8;PercDel=3.8;PercIns=0.4;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16506" LastEditorUserId="73" LastEditDate="2022-11-19T10:48:38.470" LastActivityDate="2022-11-19T11:10:25.910" Title="How to resolve Error in fix.by(by.x, x) : 'by' must specify a uniquely valid column while using merge function in R" Tags="|r|merge|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20028" PostTypeId="1" AcceptedAnswerId="20029" CreationDate="2022-11-17T19:52:22.170" Score="1" ViewCount="55" Body="&lt;p&gt;I have a fasta file with multiple sequences comprising 38 sequences. The length of the sequences are around 11000 bp.&lt;/p&gt;&#xA;&lt;p&gt;How can i get changes in the genomes based in a reference genome? (aa subtitutions or deletions)&lt;/p&gt;&#xA;" OwnerUserId="12707" LastEditorUserId="3848" LastEditDate="2022-11-18T03:19:30.540" LastActivityDate="2022-11-18T18:35:44.813" Title="How to identify mutations in a viral genome" Tags="|phylogenetics|genome|phylogeny|reference-genome|mutations|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20029" PostTypeId="2" ParentId="20028" CreationDate="2022-11-18T03:18:22.520" Score="1" Body="&lt;p&gt;Python3 via the &lt;code&gt;difflib&lt;/code&gt; library/package. &lt;code&gt;AlignIO&lt;/code&gt; can be leveraged (Biopython) to construct the &lt;code&gt;align&lt;/code&gt; list. To be honest if your coding isn't strong you could just cut and paste the alignment into this code. Nucleotides not amino acids ... same idea just easier to type. It will work for aa and gaps. &amp;quot;Ref&amp;quot; (output) refers to the first alignment position only.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import difflib&#xA;&#xA;align=['AGCTAGATGATATGA', 'AGCTAGATGATATGA', 'AGCTAGGTGATATGA', 'AGCTAGATGATATGA', 'AGCTAGATGATATGA', 'AGCTAGAAGATATGA', 'AGCTAGAGGATATGA', 'AGCTAGATCATATGT'] &#xA;ref= 'AGCTAGATGATATGA'&#xA;for query in align:&#xA;    print('{} (ref) =&amp;gt; {} (query)'.format(ref,query))  &#xA;    for loc,pos in enumerate(difflib.ndiff(ref, query)):&#xA;        if pos[0]==' ': &#xA;            continue&#xA;        elif pos[0]=='-':&#xA;            print(u'Ref &amp;quot;{}&amp;quot;'.format(pos[-1]))&#xA;        elif pos[0]=='+':&#xA;            print(u'Query has &amp;quot;{}&amp;quot; at position {}'.format(pos[-1],loc))    &#xA;    print ('')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;AGCTAGATGATATGA (ref) =&amp;gt; AGCTAGATGATATGA (query)&#xA;&#xA;AGCTAGATGATATGA (ref) =&amp;gt; AGCTAGATGATATGA (query)&#xA;&#xA;AGCTAGATGATATGA (ref) =&amp;gt; AGCTAGGTGATATGA (query)&#xA;Ref &amp;quot;A&amp;quot;&#xA;Query has &amp;quot;G&amp;quot; at position 7&#xA;&#xA;AGCTAGATGATATGA (ref) =&amp;gt; AGCTAGATGATATGA (query)&#xA;&#xA;AGCTAGATGATATGA (ref) =&amp;gt; AGCTAGATGATATGA (query)&#xA;&#xA;AGCTAGATGATATGA (ref) =&amp;gt; AGCTAGAAGATATGA (query)&#xA;Ref &amp;quot;T&amp;quot;&#xA;Query has &amp;quot;A&amp;quot; at position 8&#xA;&#xA;AGCTAGATGATATGA (ref) =&amp;gt; AGCTAGAGGATATGA (query)&#xA;Ref &amp;quot;T&amp;quot;&#xA;Query has &amp;quot;G&amp;quot; at position 8&#xA;&#xA;AGCTAGATGATATGA (ref) =&amp;gt; AGCTAGATCATATGA (query)&#xA;Ref &amp;quot;G&amp;quot;&#xA;Query has &amp;quot;C&amp;quot; at position 9&#xA;Query has &amp;quot;T&amp;quot; at position 16&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Sequences must be aligned, i.e. ref sequence within the global alignment and separated. Hope that makes sense. It will work with multiple mutations between ref and query as the last example points out (two mutations per example query was just too much to type).&lt;/p&gt;&#xA;&lt;p&gt;A reproducible example in future please.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-11-18T18:35:44.813" LastActivityDate="2022-11-18T18:35:44.813" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20030" PostTypeId="1" CreationDate="2022-11-18T09:41:05.027" Score="0" ViewCount="379" Body="&lt;p&gt;I’m looking to lift over a dataset of GWAS summary stats from build 37 to 38, including converting rsid. Downstream it is to be used for co-localisation with another dataset of build 38.&lt;/p&gt;&#xA;&lt;p&gt;Would anyone be able to help with code for doing this please?&lt;/p&gt;&#xA;" OwnerUserId="16458" LastEditorUserId="475" LastEditDate="2022-11-18T12:19:55.377" LastActivityDate="2023-01-11T19:49:40.370" Title="Lifting snp rsid build 37 to 38" Tags="|gwas|identifiers|liftover|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20031" PostTypeId="1" AcceptedAnswerId="20032" CreationDate="2022-11-18T12:55:05.290" Score="1" ViewCount="46" Body="&lt;p&gt;We developed a neural network-based protein reconstruction tool to reconstruct the main chain from only CA atoms.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;we generated data from some selected PDBs from the RCSB website to train an NN model.&lt;/li&gt;&#xA;&lt;li&gt;we then use those data to train the NN model.&lt;/li&gt;&#xA;&lt;li&gt;we select some test PDBs, strip all atoms except CA atoms, and save them in files.&lt;/li&gt;&#xA;&lt;li&gt;we pass those CA-only PDBs through the NN model, obtain a reconstructed main chain, and save them in files.&lt;/li&gt;&#xA;&lt;li&gt;we compare original and reconstructed PDB files and calculate CRMSD values.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;If we obtain a CRMSD value of 0.3559 and want to improve it what options should we explore?&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Using Keras.&lt;/li&gt;&#xA;&lt;li&gt;Not CNN nor RNN. It's a straightforward MLP (4 layers in total). No feedback, no convolution.&lt;/li&gt;&#xA;&lt;li&gt;training set is actually large: 1398438 rows and 102 columns in the data file.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="73" LastEditDate="2022-11-19T12:00:43.233" LastActivityDate="2022-11-19T12:00:43.233" Title="What options should I explore to improve the output of the protein reconstruction?" Tags="|protein-structure|machine-learning|public-dataset|deep-learning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20032" PostTypeId="2" ParentId="20031" CreationDate="2022-11-18T15:11:04.333" Score="1" Body="&lt;p&gt;The first step is to assess the accuracy, recall and precision of the model then look at confusion matrix. Thereafter consider ROC AUC and if all those are fine ... it's biological. If the model is not 'accurate' where accuracy &amp;gt;0.85 (minimum) its about training the model and that is a different question.&lt;/p&gt;&#xA;&lt;p&gt;My guess (&lt;em&gt;edit: which from the comments was wrong&lt;/em&gt;) is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the sample size is insufficient for a presumed deep learning model.&lt;/li&gt;&#xA;&lt;li&gt;this will generate problems in the accuracy of the model.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The solution to low sample is called 'data augmentation', a lot of biological knowledge is needed to successfully augment a data set. In this case I suspect the level of biological understanding is strong, but the understanding of deep learning is less advanced.&lt;/p&gt;&#xA;&lt;p&gt;In summary, low accuracy is a feature of small sample sizes within a deep learning model. To give some idea deep learning needs 500 000 sample size to be reasonable.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update from the comments&lt;/strong&gt;, the data set is large:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a sample of 1398438 rows and 102 columns (data points).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This is a good sample for deep learning and low sample size is not the cause.&lt;/p&gt;&#xA;&lt;p&gt;The model is &lt;code&gt;keras&lt;/code&gt; via a 4-layered MLP, so thats fine, but not clear whether its CNN (probably) or RNN.&lt;/p&gt;&#xA;&lt;p&gt;The likely issue is under-fitting and this is diagnosed via the above statistics for example accuracy and ROC AUC and solution is to enhance the MLP. If there is over-fitting occurring thats a different question.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-11-18T18:30:57.130" LastActivityDate="2022-11-18T18:30:57.130" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20033" PostTypeId="2" ParentId="20030" CreationDate="2022-11-18T16:52:59.410" Score="1" Body="&lt;p&gt;There are multiple tools to use for this. Might I suggest the following reads: &lt;a href=&quot;https://genviz.org/module-01-intro/0001/06/02/liftoverTools/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Introduction to liftover tools&lt;/a&gt;, where they also refer to this usefull post &lt;a href=&quot;https://www.biostars.org/p/65558/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioStars post&lt;/a&gt;. Maybe also consider reading this paper for a word of caution: &lt;a href=&quot;https://academic.oup.com/bib/article/22/5/bbab069/6210068?login=true&quot; rel=&quot;nofollow noreferrer&quot;&gt;Paper&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Maybe you can also clarify you question more, if these links do not help you with what you need.&lt;/p&gt;&#xA;" OwnerUserId="16204" LastActivityDate="2022-11-18T16:52:59.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20034" PostTypeId="1" CreationDate="2022-11-18T20:46:06.303" Score="1" ViewCount="60" Body="&lt;p&gt;I start saying my background is Biology and I'm totally new to WDL, so I'm aware my code will be messy but I'm trying to learn.&lt;/p&gt;&#xA;&lt;p&gt;My problem is, essentially, to run a Docker container I built and tested for a tool on WDL (Terra platform). Now, I'm struggling a bit in understanding how to get input and generate output, and if they have to be contained in a path within the Docker. I successfully run this tool on a normal cluster, but Terra is a different story...&lt;/p&gt;&#xA;&lt;p&gt;Shortly, what the tool does is determining the k-mers distribution in a pangenome and use the haplotypes support from the graph structure to infer genotypes for samples &amp;quot;mapped&amp;quot; on the pangenome graph space (&lt;a href=&quot;https://github.com/eblerjana/pangenie&quot; rel=&quot;nofollow noreferrer&quot;&gt;more info here&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The tool requires as input a VCF file from the pangenome, a reference genome and a set of samples to genotype; all this information is specified within a .yaml file the user would normally edit once the tool is installed on a cluster to point to the correct directories where files are.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Firstly, I would like to give users the choice over what pangenome VCF file is loaded, as there are two main ways to build pangenome graphs and the genome inference is substantially affected by them.&lt;/li&gt;&#xA;&lt;li&gt;Secondly, I want also to give a choice over the type of reference used, as variants for the samples can be called both against GRCh38 as well as CHM13.&lt;/li&gt;&#xA;&lt;li&gt;Finally, I would like to give users freedom over the number of samples to genotype but also understand where this data (uncompressed .fastq files) have to be access on Terra and loaded in WDL&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The tools generates five directories after the execution is completed, I'm not sure whether I have to create them beforehand for output to be access by users. Let me know.&lt;/p&gt;&#xA;&lt;p&gt;Following, the script I wrote up to now. I have a few problems at different stages of the implementation as you can see. Please let me know if you can help and what kind of additional information you might need. Thanks in advance!&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;code version 1.0&#xA;&#xA;workflow PanGenie {&#xA;    input {&#xA;        File? PANGENOME_VCF # input vcf with variants to be genotyped&#xA;        File? REFERENCE_GENOME # reference genome used to call variants from the graph&#xA;        Array[File]? INUPT_READS_SET # actual .fastq file&#xA;        Array[String]? SAMPLEs_FASTQ_READS # reads (FASTA/FASTQ format, uncompressed) for samples' to genotype&#xA;&#xA;        Int CORES = 24 # number of cores to allocate for PanGenie execution&#xA;        Int DISK_STORAGE = 100 # storage memory for output files&#xA;    }&#xA;&#xA;    File PANGENOME_VCF ## lets the user decide whether to use a mc-CACTUS or a PGGB based vcf file&#xA;    File REFERENCE_GENOME ## lets the user chose what reference to call variants from (GRCh38 or CHM13)&#xA;    File PANGENIE = &amp;quot;/app/pangenie/build/src/PanGenie&amp;quot; ## path to executable within Docker&#xA;&#xA;    task pangenie_exe {&#xA;        input {&#xA;            File PANGENOME_VCF&#xA;            File REFERENCE_GENOME&#xA;            Array[File] INUPT_READS_SET&#xA;        }&#xA;&#xA;        command {&#xA;            snakemake --cores CORES&#xA;        }&#xA;&#xA;        output {&#xA;            Directory benchmark = &amp;quot;&amp;quot;&#xA;            Directory genotypes = &amp;quot;&amp;quot;&#xA;            Directory input_vcf = &amp;quot;&amp;quot;&#xA;            Directory pangenie = &amp;quot;&amp;quot;&#xA;            Directory pangenome = &amp;quot;&amp;quot;&#xA;        }&#xA;&#xA;        runtime {&#xA;            memory: &amp;quot;300 GB&amp;quot;&#xA;            CPU: CORES&#xA;            disks: &amp;quot;local-disk &amp;quot; + DISK_STORAGE + &amp;quot; SSD&amp;quot;&#xA;            docker: &amp;quot;quay.io/overcraft90/eblerjana_pangenie&amp;quot;&#xA;            preemptible: 1 # can be useful for tools which execute sequential steps in a pipeline generating intermediate outputs&#xA;        }&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14271" LastEditorUserId="14271" LastEditDate="2022-11-18T22:07:27.160" LastActivityDate="2022-11-18T22:07:27.160" Title="Implement a WDL workflow for a docker container I built" Tags="|wdl|workflow-management|docker|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20035" PostTypeId="1" CreationDate="2022-11-19T07:31:43.363" Score="1" ViewCount="717" Body="&lt;p&gt;I want to get the annotations in gff format for a assembled genome. But for some species there is no available annotation in gff format in NCBI but only in gbff format. And I don’t know how to convert a gbff file to a gff file. So, I am seeking your help. Can I have your suggestions here? Your guidence will be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="12579" LastActivityDate="2022-11-19T07:31:43.363" Title="conversion of a gbff file to a gff file" Tags="|sequence-annotation|gff|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20038" PostTypeId="2" ParentId="20027" CreationDate="2022-11-19T11:10:25.910" Score="1" Body="&lt;p&gt;When I loaded these files into R, I got the following result, which matches your error (except that it's been translated into German):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; df1 &amp;lt;- read.csv(&amp;quot;df1.csv&amp;quot;)&#xA;&amp;gt; df2 &amp;lt;- read.csv(&amp;quot;df2.csv&amp;quot;)&#xA;&amp;gt; merge(df1, df2, by=c(&amp;quot;start&amp;quot;,&amp;quot;end&amp;quot;,&amp;quot;function&amp;quot;))&#xA;Fehler in fix.by(by.x, x) : 'by' muss gültige Spalte(n) spezifizieren&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This error is obscure, and it took me a while to get my head around what was going on. The problem is that 'function' is a reserved word in R, so it renamed the &lt;code&gt;function&lt;/code&gt; column to add a period [&lt;code&gt;.&lt;/code&gt;] at the end of it:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; colnames(df1)&#xA;[1] &amp;quot;chr&amp;quot;       &amp;quot;RM&amp;quot;        &amp;quot;sim&amp;quot;       &amp;quot;start&amp;quot;     &amp;quot;end&amp;quot;       &amp;quot;stat&amp;quot;      &amp;quot;strand&amp;quot;   &#xA;[8] &amp;quot;id&amp;quot;        &amp;quot;function.&amp;quot;&#xA;&amp;gt; colnames(df2)&#xA;[1] &amp;quot;start&amp;quot;     &amp;quot;end&amp;quot;       &amp;quot;function.&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So R was looking for a column name called &amp;quot;function&amp;quot;, didn't find one, and got confused.&lt;/p&gt;&#xA;&lt;p&gt;I think the easiest fix to this is to use the new names that R has given the tables:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; merge(df1, df2, by=c(&amp;quot;start&amp;quot;,&amp;quot;end&amp;quot;,&amp;quot;function.&amp;quot;))&#xA;[1] start     end       function. chr       RM        sim       stat      strand    id       &#xA;&amp;lt;0 Zeilen&amp;gt; (oder row.names mit Länge 0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;[note: this has no output, because the examples you provided have no common start/end/function]&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, let R work out the common fields itself, as its default for &lt;code&gt;merge&lt;/code&gt; is to intersect the column names (which is what you want in this case):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; merge(df1, df2)&#xA;[1] start     end       function. chr       RM        sim       stat      strand   id       &#xA;&amp;lt;0 Zeilen&amp;gt; (oder row.names mit Länge 0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which, for completeness, is equivalent to this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; merge(df1, df2, by=intersect(names(df1), names(df2)))&#xA;[1] start     end       function. chr       RM        sim       stat      strand    id       &#xA;&amp;lt;0 Zeilen&amp;gt; (oder row.names mit Länge 0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-11-19T11:10:25.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20039" PostTypeId="7" CreationDate="2022-11-19T11:54:50.493" Score="0" Body="&lt;ul&gt;&#xA;                                &lt;li&gt;Anything not directly related to bioinformatics&lt;/li&gt;&#xA;&lt;li&gt;Questions that are primarily opinion-based&lt;/li&gt;&#xA;&lt;li&gt;Questions with too many possible answers&lt;/li&gt;&#xA;&lt;li&gt;Questions that would require an extremely long answer&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="-1" LastEditorUserId="73" LastEditDate="2022-11-19T11:54:50.493" LastActivityDate="2022-11-19T11:54:50.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20040" PostTypeId="1" AcceptedAnswerId="20042" CreationDate="2022-11-19T16:43:39.347" Score="0" ViewCount="99" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/9545742/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want to find the frequency of each motif-matching peptide.&#xA;The sequences are:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; Sequence_1&#xA;MPPRRSIVEVKVLDVQKRRVPNKHYVYIIRVTWSSGATEAIYRRYSKFFDLQMQMLDKFP&#xA;MEGGQKDPKQRIIPFLPGKILFRRSHIRDVAVKRLIPIDEYCKALIQLPPYISQCDEVLQ&#xA;FFETRPEDLNPPKEEHIGKKKSGNDPTSVDPMVLEQYVVVADYQKQESSEISLSVGQVVD&#xA;IIEKNESGWWFVSTAEEQGWVPATCLEGQDGVQDEFSLQPEEEEKYTVIYPYTARDQDEM&#xA;NLERGAVVEVVQKNLEGWWKIRYQGKEGWAPASYLKKNSGEPLPPKLGPSSPAHSGALDL&#xA;DGVSRHQNAMGREKELLNNQRDGRFEGRLVPDGDVKQRSPKMRQRPPPRRDMTIPRGLNL&#xA;&#xA;&amp;gt;Sequence_2&#xA;MAEVRKFTKRLSKPGTAAELRQSVSEAVRGSVVLEKAKLVEPLDYENVITQRKTQIYSDP&#xA;LRDLLMFPMEDISISVIGRQRRTVQSTVPEDAEKRAQSLFVKECIKTYSTDWHVVNYKYE&#xA;DFSGDFRMLPCKSLRPEKIPNHVFEIDEDCEKDEDSSSLCSQKGGVIKQGWLHKANVNST&#xA;ITVTMKVFKRRYFYLTQLPDGSYILNSYKDEKNSKESKGCIYLDACIDVVQCPKMRRHAF&#xA;ELKMLDKYSHYLAAETEQEMEEWLIMLKKIIQINTDSLVQEKKDTVEAIQEEETSSQGKA&#xA;ENIMASLERSMHPELMKYGRETEQLNKLSRGDGRQNLFSFDSEVQRLDFSGIEPDVKPFE&#xA;EKCNKRFMVNCHDLTFNILGHIGDNAKGPPTNVEPFFINLALFDVKNNCKISADFHVDLN&#xA;PPSVREMLWGTSTQLSNDGNAKGFSPESLIHGIAESQLCYIKQGIFSVTNPHPEIFLVVR&#xA;&#xA;&amp;gt;Sequence_3&#xA;GDDSEWLKLPVDQKCEHKLWKARLSGYEEALKIFQKIKDEKSPEWSKYLGLIKKFVTDS&#xA;NAVVQLKGLEAALVYVENAHVAGKTTGEVVSGVVSKVFNQPKAKAKELGIEICLMYVEIE&#xA;KGESVQEELLKGLDNKNPKIIVACIETLRKALSEFGSKIISLKPIIKVLPKLFESRDKAV&#xA;RDEAKLFAIEIYRWNRDAVKHTLQNINSVQLKELEEEWVKLPTGAPKPSRFLRSQQELEA&#xA;KLEQQQSAGGDAEGGGDDGDEVPQVDAYELLDAVEILSKLPKDFYDKIEAKKWQERKEAL&#xA;EAVEVLVKNPKLEAGDYADLVKALKKVVGKDTNVMLVALAAKCLTGLAVGLRKKFGQYAG&#xA;HVVPTILEKFKEKKPQVVQALQEAIDAIFLTTTLQNISEDVLAVMDNKNPTIKQQTSLFI&#xA;ARSFRHCTSSTLPKSLLKPFCAALLKHINDSAPEVRDAAFEALGTALKVVGEKSVNPFLA&#xA;&#xA;&amp;gt; Sequence_4&#xA;MPPRRSIVEVKVLDVQKRRVPNKHYVYIIRVTWSSGATEAIYRRYSKFFDLQMQMLDKFP&#xA;MEGGQKDPKQRIIPFLPGKILFRRSHIRDVAVKRLIPIDEYCKALIQLPPYISQCDEVLQ&#xA;FFETRPEDLNPPKEEHIGKKKSGNDPTSVDPMVLEQYVVVADYQKQESSEISLSVGQVVD&#xA;IIEKNESGWWFVSTAEEQGWVPATCLEGQDGVQDEFSLQPEEEEKYTVIYPYTARDQDEM&#xA;NLERGAVVEVVQKNLEGWWKIRYQGKEGWAPASYLKKNSGEPLPPKLGPSSPAHSGALDL&#xA;DGVSRHQNAMGREKELLNNQRDGRFEGRLVPDGDVKQRSPKMRQRPPPRRDMTIPRGLNL &#xA;&#xA;&#xA;&amp;gt;Sequence_5&#xA;GDDSEWLKLPVDQKCEHKLWKARLSGYEEALKIFQKIKDEKSPEWSKYLGLIKKFVTDS&#xA;NAVVQLKGLEAALVYVENAHVAGKTTGEVVSGVVSKVFNQPKAKAKELGIEICLMYVEIE&#xA;KGESVQEELLKGLDNKNPKIIVACIETLRKALSEFGSKIISLKPIIKVLPKLFESRDKAV&#xA;RDEAKLFAIEIYRWNRDAVKHTLQNINSVQLKELEEEWVKLPTGAPKPSRFLRSQQELEA&#xA;KLEQQQSAGGDAEGGGDDGDEVPQVDAYELLDAVEILSKLPKDFYDKIEAKKWQERKEAL&#xA;EAVEVLVKNPKLEAGDYADLVKALKKVVGKDTNVMLVALAAKCLTGLAVGLRKKFGQYAG&#xA;HVVPTILEKFKEKKPQVVQALQEAIDAIFLTTTLQNISEDVLAVMDNKNPTIKQQTSLFI&#xA;ARSFRHCTSSTLPKSLLKPFCAALLKHINDSAPEVRDAAFEALGTALKVVGEKSVNPFLA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The fasta file actually has more sequences, I just chose the top 5, to put on here.&lt;/p&gt;&#xA;&lt;p&gt;The code used is below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import Bio&#xA;import regex&#xA;&#xA;from Bio import SeqIO&#xA;input_file = 'Sequences.fasta'&#xA;fasta_sequences = SeqIO.parse(open(input_file),'fasta')&#xA;for fasta in fasta_sequences:&#xA;     name, sequence = fasta.id, str(fasta.seq)&#xA;     matches=regex.finditer(r&amp;quot;(P[A-Z]{2}P..)&amp;quot;,sequence)&#xA;     for m in matches:&#xA;        print(name, m.start(), m.end(), m.group())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is giving me my desired output, which is below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Sequence1 73 79 PFLPGK&#xA;Sequence1 281 287 PLPPKL&#xA;Sequence1 288 294 PSSPAH&#xA;Sequence4 73 79 PFLPGK&#xA;Sequence4 281 287 PLPPKL&#xA;Sequence4 288 294 PSSPAH&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now I want to:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate the number of sequences in that file. I know in linux environment I would have to use &lt;code&gt;grep -c &amp;quot;&amp;gt;&amp;quot; Sequences.fasta&lt;/code&gt; to get the total number of sequences in the fasta file, but how do I do it in python?.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Once I know how many sequences are in file in total, I want to count how many of those sequences have the motif-matching peptides above.&#xA;For example how many sequences have PFLPGK at positions 73 79? In the example above PFLPGK its in sequence 1 &amp;amp; 4, so the &lt;strong&gt;frequency&lt;/strong&gt; is 2, meaning that two sequences have PFLPGK. How do I find that frequency for each of the peptides using python.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Many thanks for your assistance.&lt;/p&gt;&#xA;" OwnerUserId="16468" LastEditorUserId="73" LastEditDate="2022-11-20T12:52:23.390" LastActivityDate="2022-11-22T12:09:17.830" Title="finding the frequency of a motif-matching peptide" Tags="|phylogenetics|proteins|python|motifs|homework|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20041" PostTypeId="1" CreationDate="2022-11-19T17:23:51.370" Score="2" ViewCount="56" Body="&lt;p&gt;I have a txt file summarising the result of a GWAS on an European population. Its structure is the next one:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data = data.frame(chr = c('1', '1', '1', '1', '1'),&#xA;              bp = c(740098, 787889, 952073, 993492),&#xA;              snp = c('rs12138618',  'rs4951864',  'rs3128126',  'rs4075116'),&#xA;              p = c(0.1, 0.04, 0.7, 0.9))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Imagine the first two snp listed (rs12138618, rs4951864) belong to the same LD-block (for r^2 = 0.1 or similar criteria). Where can I get this information from (table/database) or what package in R can I use to label these dependent SNPs and obtain something as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   chr    bp        snp    p    ld_block&#xA;1   1 740098 rs12138618 0.10        21&#xA;2   1 787889  rs4951864 0.04        21&#xA;3   1 952073  rs3128126 0.70        29&#xA;4   1 993492  rs4075116 0.90        34&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am using big data, so something computationally-wise is preferred.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; Obviously I am making up the numbers, but I hope my point is clear. I am also new to GWAS, so it is very likely that I can be missing/misunderstanding concepts.&lt;/p&gt;&#xA;&lt;p&gt;What if I do not have the genotype data but I know that my population is CEU? Is there any reference-genotype data that I can provide to plink instead?&lt;/p&gt;&#xA;" OwnerUserId="12360" LastEditorUserId="73" LastEditDate="2022-11-20T12:50:48.893" LastActivityDate="2022-11-20T12:50:48.893" Title="LD block labels annotation for GWAS summary data" Tags="|r|gwas|ld|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20042" PostTypeId="2" ParentId="20040" CreationDate="2022-11-19T17:42:08.523" Score="0" Body="&lt;p&gt;For the first part is easy,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;seqs = SeqIO.to_dict(SeqIO.parse(&amp;quot;myfile.fa&amp;quot;, &amp;quot;fasta&amp;quot;))&#xA;print (len(seqs))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The second part is solvable Python's amazing slicing, this is the concept ....&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;loci = {'PFLPGK':[1,10], 'PFLPGK':[50:60] }   &#xA;fasta_sequences = SeqIO.parse(open(input_file),'fasta')&#xA;mydict = {}&#xA;for key, values in loci.items():&#xA;    myrange = '-'.join(values)&#xA;    mydict = {key:myrange} # you need to initiate the dictionary without defaultdict&#xA;    for record in fasta_sequences:&#xA;        if record.seq[values[0]-1:values[1]-1] == key:&#xA;             mydict[key][myrange].append(record.id) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To print it:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;key, value in mydict.items():&#xA;    print(&amp;quot;No. of&amp;quot; + key + '=' + len(mydict[key][value]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note 1&lt;/strong&gt;, the second answer is concept only.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note 2&lt;/strong&gt; there can be an issue between the &lt;code&gt;list&lt;/code&gt; location and the sequence location ... that is &lt;strong&gt;super&lt;/strong&gt; important otherwise it will not work. I've added &lt;code&gt;-1&lt;/code&gt; from both locations of the amino acid and the location on the &lt;code&gt;list&lt;/code&gt;, &lt;em&gt;but this needs checking&lt;/em&gt;. This is because &lt;code&gt;list&lt;/code&gt;s start at &lt;code&gt;0&lt;/code&gt; whilst amino acids start at &lt;code&gt;1&lt;/code&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt; &lt;em&gt;Advice on Python&lt;/em&gt; requested by OP&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Answer to Q1 has been debugged, tested and works.&lt;/li&gt;&#xA;&lt;li&gt;Use Q2 as the basis ... just debug it (see point 4.), it will be a bit fiddly because moving from &lt;code&gt;lists&lt;/code&gt; to aa strings.&lt;/li&gt;&#xA;&lt;li&gt;Use an IDE, I use Spyder: don't get too caught up in &amp;quot;the best IDE&amp;quot; they just make life easier.&lt;/li&gt;&#xA;&lt;li&gt;Book: Mark Lutz &amp;quot;Learning Python 5th Edition&amp;quot; by O'Reilly section on &lt;code&gt;lists&lt;/code&gt; (slicing) and &lt;code&gt;dictionaries&lt;/code&gt;  &lt;strong&gt;*&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Online resources &lt;code&gt;GeekforGeeks&lt;/code&gt; on &lt;a href=&quot;https://www.geeksforgeeks.org/python-lists/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;lists&lt;/code&gt;&lt;/a&gt; (slicing) and &lt;a href=&quot;https://www.geeksforgeeks.org/python-dictionary/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;dictionaries&lt;/code&gt;&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://biopython.org/wiki/SeqIO&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioPython docs on SeqIO&lt;/a&gt; search for &lt;code&gt;record.id&lt;/code&gt; and &lt;code&gt;record.seq&lt;/code&gt; ... you need to understand the SeqIO object.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;*&lt;/strong&gt; This is physically a huge book so an electronic version is strongly preferred just for logistics. This goes beyond the full modern implementation of &lt;code&gt;OOP&lt;/code&gt; in bioinformatics (which is not very modern in truth and can become unreadable).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-11-22T12:09:17.830" LastActivityDate="2022-11-22T12:09:17.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20043" PostTypeId="1" AcceptedAnswerId="20083" CreationDate="2022-11-20T15:07:44.757" Score="1" ViewCount="55" Body="&lt;p&gt;CpGs are symmetrical, in that a CG sequence on the forward strand is hybridized to a GC --- and both dinucleotides on each opposing strand are CpGs dinucleotides which can be methylated. Conversely, CpGs can be GC on the forward strand but CG on the reverse strand.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FORWARD -&amp;gt; 5'--CG--3'  [OR]  5'--GC--3' &amp;lt;- FORWARD&#xA;REVERSE -&amp;gt; 3'--GC--5'        3'--CG--5' &amp;lt;- REVERSE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The assignment of &amp;quot;forward&amp;quot; and &amp;quot;reverse&amp;quot; strandedness is more or less arbitrary.&lt;/p&gt;&#xA;&lt;p&gt;Given the above, why does it seem like the Illumina 850k (aka EPIC) array only profiles methylation from CpGs which are CG in the forward strand, while ignoring CpGs which are GC in the forward strand? I would also love to hear if my premises are wrong.&lt;/p&gt;&#xA;&lt;p&gt;Here is some R code illustrating the issue at hand:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;suppressPackageStartupMessages({&#xA;  library(IlluminaHumanMethylationEPICanno.ilm10b4.hg19)&#xA;  library(tidyverse)})&#xA;data(IlluminaHumanMethylationEPICanno.ilm10b4.hg19)&#xA;&#xA;IlluminaHumanMethylationEPICanno.ilm10b4.hg19 %&amp;gt;%&#xA;  getAnnotation() %&amp;gt;%&#xA;  as_tibble() %&amp;gt;%&#xA;  count(forward_seq=str_extract(Forward_Sequence, &amp;quot;\\[[ATCG]{2}\\]&amp;quot;))&#xA;&#xA;# Results:&#xA;# # A tibble: 3 × 2&#xA;#   forward_seq      n&#xA;#   &amp;lt;chr&amp;gt;        &amp;lt;int&amp;gt;&#xA;# 1 [CA]          2922&#xA;# 2 [CG]        862927&#xA;# 3 [CT]            10&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6520" LastEditorUserId="2108" LastEditDate="2022-11-21T00:15:48.730" LastActivityDate="2022-11-28T08:07:38.090" Title="Why do Illumina 850k/EPIC arrays ignore CpGs which are &quot;GC&quot; in the forward strand?" Tags="|genome|database|sequence-annotation|genomics|public-databases|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20044" PostTypeId="1" AcceptedAnswerId="20047" CreationDate="2022-11-20T16:05:26.827" Score="2" ViewCount="823" Body="&lt;p&gt;Say I want to use wget to download a gff file for the X chromosome of release 5.57 of the Drosophila melanogaster genome. The NCBI page with the gff file for just this &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NC_004354.3&quot; rel=&quot;nofollow noreferrer&quot;&gt;is here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, I do not see any way to directly wget the gff from this page. What I can do is click &amp;quot;Send to&amp;quot; at the top, select File, and change the Format dropdown to GFF3, and finally click 'Create File', and the gff file downloads onto my computer.&lt;/p&gt;&#xA;&lt;p&gt;But then I lose the ability to use the -a flag to log the download details, I can't download the gff using wget from a remote server directly but instead I need to download it locally and then upload it, etc. Is there any way to use wget to download gff files from NCBI programmatically instead of having to download it manually from the website?&lt;/p&gt;&#xA;" OwnerUserId="16493" LastEditorUserId="298" LastEditDate="2022-11-21T10:22:29.473" LastActivityDate="2022-11-22T09:19:43.710" Title="How to wget a gff from NCBI database?" Tags="|phylogenetics|bash|data-download|gff3|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20045" PostTypeId="2" ParentId="4133" CreationDate="2022-11-20T16:42:30.837" Score="1" Body="&lt;p&gt;If you can add a column of RefSeq identifiers for each gene to your table, first download a whole-genome GFF file for your organism:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;wget ftp://ftp.ncbi.nlm.nih.gov/genomes/all/annotation_releases/4232/101/GCF_002127325.2_HanXRQr2.0-SUNRISE/GCF_002127325.2_HanXRQr2.0-SUNRISE_genomic.gff.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and then open the file in Python using a package like gffutils and extract all the rows which contain a RefSeq identifier located in table (the RefSeq identifier is the first column in the gff file linked in the wget command above). For all the rows you extract, which chromosome it belongs to will be stated in the Attributes column, which you can also extract using gffutils.&lt;/p&gt;&#xA;" OwnerUserId="16493" LastActivityDate="2022-11-20T16:42:30.837" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20046" PostTypeId="1" AcceptedAnswerId="20050" CreationDate="2022-11-20T20:02:39.203" Score="0" ViewCount="18" Body="&lt;p&gt;I'm reading through this paper: Baran, Y., Bercovich, A., Sebe-Pedros, A., Lubling, Y., Giladi, A., Chomsky, E., ... &amp;amp; Tanay, A. (2019). &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1812-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;MetaCell: analysis of single-cell RNA-seq data using K-nn graph partitions&lt;/a&gt;. Genome biology, 20(1), 1-19.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BMITn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BMITn.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Maybe I am not familiar with this notation but I don't understand the regularization procedure as follows:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What is happening in these max functions? What does the 0 indicate? They have some terms &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and then in the max function they write &lt;span class=&quot;math-container&quot;&gt;$max( x, 0)$&lt;/span&gt;.&lt;/li&gt;&#xA;&lt;li&gt;I assume &lt;span class=&quot;math-container&quot;&gt;$K$&lt;/span&gt; is some regularization parameter, like a cost. Is &lt;span class=&quot;math-container&quot;&gt;$s_{ij} * s_{ji}$&lt;/span&gt; the rank product?  I.e. the geometric mean?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2482" LastEditorUserId="2108" LastEditDate="2022-11-21T00:10:07.307" LastActivityDate="2022-11-21T00:10:07.307" Title="Understanding regularization step in Meta-cell pre processing" Tags="|scrnaseq|statistics|graphs|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20047" PostTypeId="2" ParentId="20044" CreationDate="2022-11-20T20:30:35.443" Score="3" Body="&lt;p&gt;my suggestion would be to go via the assembly page.&#xA;For your example link, I would&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Click the link to the BioProject: PRJNA164&lt;/li&gt;&#xA;&lt;li&gt;On the right side, under &amp;quot;Related information&amp;quot; choose the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly/GCF_000001215.4&quot; rel=&quot;nofollow noreferrer&quot;&gt;assembly&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;On the assembly page, under &amp;quot;Access the data&amp;quot; on the right side, I would choose &amp;quot;FTP directory for RefSeq assembly&amp;quot;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Once in the RefSeq folder, copy the link to the &lt;code&gt;*_genomic.gff.gz&lt;/code&gt; and use wget.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;wget https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/001/215/GCF_000001215.4_Release_6_plus_ISO1_MT/GCF_000001215.4_Release_6_plus_ISO1_MT_genomic.gff.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: The initial question requested an older versions, which can be found in RefSeq's &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/genomes/archive/old_refseq/&quot; rel=&quot;nofollow noreferrer&quot;&gt;archive&lt;/a&gt; of prior releases. For this specific Drosophila assembly chromosome X, the version 5 can be found at &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/genomes/archive/old_refseq/Drosophila_melanogaster/RELEASE_5/CHR_X/NC_004354.gff&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nlm.nih.gov/genomes/archive/old_refseq/Drosophila_melanogaster/RELEASE_5/CHR_X/NC_004354.gff&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16386" LastEditorUserId="16386" LastEditDate="2022-11-22T09:19:43.710" LastActivityDate="2022-11-22T09:19:43.710" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20048" PostTypeId="2" ParentId="19860" CreationDate="2022-11-20T20:39:49.863" Score="2" Body="&lt;p&gt;&lt;em&gt;&lt;strong&gt;An explanation in 3 parts&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Final Edit:&lt;/strong&gt;  It turns out there was contamination in the data which you can read about in the comments, without being able to see it myself I could not know this.  If you see this in an experiment and you're clustering from totally raw unfiltered data this maybe the case for you.  Nevertheless as you subcluster within cell types of particular interest to get more refined marker genes the advice I provide below may be of use.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; The note above about looking at the violin plots may be useful, but again what your downstream analysis or experiment should impact your decision making.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;TLDR:&lt;/strong&gt;  The presence of marker genes in other cell types is not surprising nor necessarily a problem.  A marker gene that seurat finds are highly expressed in a cell type relative to the average in all cells not a gene that is uniquely expressed, if you are looking for uniquely expressed genes you will probably not find them.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;1) A note on seurat&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;So most genes are expressed in most cells there is nothing wrong with your data or your analysis.  Gene expression is very messy and the term &amp;quot;marker gene&amp;quot; is a bit of a misnomer.  Note that other analysis pipelines like Monocle have different methods for determining what constitutes a marker gene.&lt;/p&gt;&#xA;&lt;p&gt;Please consider this meta-analysis of gabanergic cerebral cortex neurons (i.e. inhibitory cortical neurons, interneurons etc etc).&lt;br /&gt;&#xA;&lt;a href=&quot;https://ieeexplore.ieee.org/abstract/document/9669888&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ieeexplore.ieee.org/abstract/document/9669888&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The fact is that most cell populations express a gradient of genes and that each gene likely has multiple roles in each cell.  If your goal is to simply publish on expressed marker genes taking the top highest uniquely expressed genes which achieve significance in each cluster will be sufficient, just be sure to report ALL your settings and provide the meta data when you publish.  If you are going on to do MERFISH or stain with the results you may want to be more selective.&lt;/p&gt;&#xA;&lt;p&gt;FindaAllMarkers() uses a simple DEG analysis where it takes the average expression of each feature (the genes you selected, which I hope you did because it's not wroth looking at every gene for clustering just the top most variable) in the cluster against the average expression for these genes in all other clusters.  Note that this means you are doing A TON of statistical tests so they use the bonferroni correction which would normally be considered to conservative but in this case is probably still too lenient.  You should only be reporting on the very top 5 maybe 10 expressed genes. If your lucky one of these will be noticeably more prominent than the others.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;2) Alternatives to seurat&lt;/strong&gt;&#xA;Monocle package has a different definition which arguably is better: &lt;a href=&quot;http://cole-trapnell-lab.github.io/monocle-release/docs/#classifying-and-counting-cells&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cole-trapnell-lab.github.io/monocle-release/docs/#classifying-and-counting-cells&lt;/a&gt; though this package is less widely used.  The monocle score accounts for how widely expressed across all clusters a marker is and the more unique to a cluster a gene is the greater its score in addition to its actual expression strength.&lt;br /&gt;&#xA;There is also a package called pandora, but I know next to nothing about it.  Read up on them and decide which has features you like best.&lt;/p&gt;&#xA;&lt;p&gt;Lastly I think part of your problem is just wanting biology to be a cleaner science than it is.  The reality is that things we call cell types are simply transient states a population of cells passes through during development and various environmental perturbations, the reality is each cell is a dynamic system and you are sampling a portion of these graded changes.  Some states are more stable than others and we have taken to calling these cell types but remember you can always subcluster within a cluster and pull out more clusters which can themselves be subclustered and so on and so forth.  It calls into question the definition of a cell type even more and to be honest at this point finding cell types is somewhat of an art, the science is yet to be settled.&lt;/p&gt;&#xA;&lt;p&gt;I encourage you to do more reading on gene expression within single cells and the definition of a cell type (or if such a thing even exists).  Here is some reading to get you started.&lt;/p&gt;&#xA;&lt;p&gt;Cembrowski, Mark S., and Vilas Menon. &amp;quot;Continuous variation within cell types of the nervous system.&amp;quot; Trends in Neurosciences 41, no. 6 (2018): 337-348.&#xA;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0166223618300596&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sciencedirect.com/science/article/pii/S0166223618300596&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Muñoz-Manchado, A. B., Gonzales, C. B., Zeisel, A., Munguba, H., Bekkouche, B., Skene, N. G., ... &amp;amp; Hjerling-Leffler, J. (2018). Diversity of interneurons in the dorsal striatum revealed by single-cell RNA sequencing and PatchSeq. Cell reports, 24(8), 2179-2190.&#xA;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S2211124718311562&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sciencedirect.com/science/article/pii/S2211124718311562&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you really want to lean into this you should start digging into the concept of meta-cells, there is a package called Metacell2 which may be of use, be sure to read the documentation and methods section for Metacell (the first version) as Metacell2 won't make much sense without it:&lt;/p&gt;&#xA;&lt;p&gt;Harris, Benjamin D., Megan Crow, Stephan Fischer, and Jesse Gillis. &amp;quot;Single-cell co-expression analysis reveals that transcriptional modules are shared across cell types in the brain.&amp;quot; Cell Systems 12, no. 7 (2021): 748-756.&#xA;-this is not metacell but is a meta analysis of cell markers using metacells with some pretty surprising and counter intuitive results about marker genes&lt;/p&gt;&#xA;&lt;p&gt;Baran, Y., Bercovich, A., Sebe-Pedros, A., Lubling, Y., Giladi, A., Chomsky, E., ... &amp;amp; Tanay, A. (2019). MetaCell: analysis of single-cell RNA-seq data using K-nn graph partitions. Genome biology, 20(1), 1-19.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Ensure you read all documentation carefully and understand the stats they are using.  The methods use are a specific definition.  Also a real marker gene will be robust to multiple methods of identifying them, if they disappear based on a change in normalization or in other packages you may have your answer there.  Bear in mind if you publish your results should be reproducible and people will need to know exactly how you got your results so share your code and metadata please.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;3) What do you want to get from this analysis?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;As I mentioned earlier if your goal is to get published, simply finding some markers in seurat may suffice.  If your goal is to do further experiments based on these results consider your hypothesis carefully.&#xA;If you want more certainty pull some similar kidney scRNA seq data from GEO and compare your results to their or integrate your data with theirs seurat has a tutorial you can follow along with to do this &lt;a href=&quot;https://satijalab.org/seurat/articles/integration_introduction.html#performing-integration-on-datasets-normalized-with-sctransform-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/articles/integration_introduction.html#performing-integration-on-datasets-normalized-with-sctransform-1&lt;/a&gt; Note you can also use this pipeline to do batch correction, that may help you.&#xA;The sct transform give pretty good results.&lt;/p&gt;&#xA;&lt;p&gt;Sorry nature isn't like a biology textbook =(,  I'm not saying that to be facetious, I just want to remind you that living things are very complex.  Lean into that complexity and make some interesting discoveries or just accept it and dig out some results you can publish.  Talk with your supervisor and explain the issue.  Hope this helps!&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;4) I lied sorry this is a response in 4 parts, make sure you report exactly what you did and share your metadata and pipeline&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Did I mention your results should be reproducible?  How can one do this?  Here are some tips.  Make sure your keeping your analysis on github, so you can keep track of all the changes it goes through and revert to previous versions if you change things.  Keep all your code, its worth having a file that is full of old code you threw away cause it was messy or you didn't end up using it. Not broken code but stuff you don't think is useful.  Report your final results, share your data and make sure a monkey could follow you instructions to reproduce your results as well as being able to double check they get the same thing.  Use a conda environment to ensure software updates will not destroy your code for future lab members or people who read your future paper.  Save all file paths as variables at the top of your code in your scripts so people can easily change the file structure without having to go through your code and struggle finding which read.csv() functions they need to edit.&lt;/p&gt;&#xA;" OwnerUserId="2482" LastEditorUserId="2482" LastEditDate="2022-12-20T01:07:32.773" LastActivityDate="2022-12-20T01:07:32.773" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20049" PostTypeId="2" ParentId="20044" CreationDate="2022-11-20T21:16:50.567" Score="1" Body="&lt;p&gt;I believe I figured out a way to do this for my particular example (although in general cases Adrian's answer works). To get release 5.57 (an earlier release) of chromosome X of Drosophila melanogaster, I believe you can go to the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/data-hub/genome/GCF_000001215.4/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Datasets Genome page&lt;/a&gt; of D. melanogaster, scroll down to the Revision History and select Release 5, and at the top of the page, NCBI lets you click on a 'datasets' or 'curl' box to copy the exact command line argument needed to download a GFF (and other files) of that whole genome. Then, I believe it should be possible to open the gff in Python using a package like gffutils, and filter only for chromosome X rows, and then write those rows to a new gff file.&lt;/p&gt;&#xA;" OwnerUserId="16493" LastActivityDate="2022-11-20T21:16:50.567" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20050" PostTypeId="2" ParentId="20046" CreationDate="2022-11-20T22:22:30.313" Score="2" Body="&lt;p&gt;Without looking in too much detail:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;We may suppose that &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is a matrix with dimensions indexed as &lt;span class=&quot;math-container&quot;&gt;$(i, j)$&lt;/span&gt;. So &lt;span class=&quot;math-container&quot;&gt;$s_{ij}$&lt;/span&gt; is the cell of &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; indexed to the &lt;span class=&quot;math-container&quot;&gt;$i$&lt;/span&gt;th row and &lt;span class=&quot;math-container&quot;&gt;$j$&lt;/span&gt;th column, and &lt;span class=&quot;math-container&quot;&gt;$s_{ji}$&lt;/span&gt; is the cell vice versa. Therefore, symmetrizing &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; is a matter of ensuring that &lt;span class=&quot;math-container&quot;&gt;$s_{ij} == s_{ji}$&lt;/span&gt;, which you can achieve by multiplying the (ranked?) entries. Symmetric matrices are nice for doing certain kinds of math, and yield a handy representation of undirected graphs, which the authors appear to be using for &lt;a href=&quot;https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;kNN&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;span class=&quot;math-container&quot;&gt;$max(x, 0)$&lt;/span&gt; is just saying that you want either &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; if &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is non-negative, or &lt;span class=&quot;math-container&quot;&gt;$0$&lt;/span&gt; if it is negative. Thus, you make the whole matrix non-negative, which makes some handy matrix math possible. You can see how this works computationally by inspecting e.g. the Python &lt;code&gt;max()&lt;/code&gt; &lt;a href=&quot;https://docs.python.org/3/library/functions.html#max&quot; rel=&quot;nofollow noreferrer&quot;&gt;function&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-11-20T22:22:30.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20051" PostTypeId="1" AcceptedAnswerId="20088" CreationDate="2022-11-21T04:09:47.547" Score="0" ViewCount="401" Body="&lt;p&gt;I have tried to use UpsetR to visualize the input file which can be found &lt;a href=&quot;https://cloudstor.aarnet.edu.au/plus/s/OupXC8rF2bDFtRV&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; library(&amp;quot;UpSetR&amp;quot;)&#xA;&amp;gt; orthogroups_df&amp;lt;- read.table(&amp;quot;orthogroups.GeneCount.tsv&amp;quot;,  header=T, stringsAsFactors = F)&#xA;&amp;gt; #All species&#xA;&amp;gt; selected_species &amp;lt;- colnames(orthogroups_df)[2:(ncol(orthogroups_df) -1)] &#xA;&amp;gt; selected_species&#xA; [1] &amp;quot;Atha&amp;quot; &amp;quot;Cann&amp;quot; &amp;quot;NQLD&amp;quot; &amp;quot;Natt&amp;quot; &amp;quot;Ngla&amp;quot; &amp;quot;Nlab&amp;quot; &amp;quot;Nsyl&amp;quot; &amp;quot;Ntab&amp;quot; &amp;quot;Ntom&amp;quot; &amp;quot;Slyc&amp;quot; &amp;quot;Stub&amp;quot; &amp;quot;Vvin&amp;quot;&#xA;&amp;gt; head(orthogroups_df)&#xA;  Orthogroup Atha Cann NQLD Natt Ngla Nlab Nsyl Ntab Ntom Slyc Stub Vvin Total&#xA;1  OG0000000    0    0  965    0    0    3    0    0    0    0    0    0   968&#xA;2  OG0000001    0    1    3    0    0  448    0    0    0    0    0    0   452&#xA;3  OG0000002    0    1  313    0    0  120    1    0    1    0    0    0   436&#xA;4  OG0000003    0   93   15   21   46   16   33   63   36   25   39   26   413&#xA;5  OG0000004    1   42    2   34  109    6    8  154   11    9    4    0   380&#xA;6  OG0000005    0    2   61    1   34   44   91   70   43   20    1    0   367&#xA;&amp;gt; ncol(orthogroups_df)&#xA;[1] 14&#xA;&amp;gt; orthogroups_df[orthogroups_df &amp;gt; 0] &amp;lt;- 1&#xA;&amp;gt; upset(orthogroups_df, &#xA;+       nsets = ncol(orthogroups_df), &#xA;+       sets = rev(c(selected_species)), &#xA;+       queries = list(list(query = intersects, params = list(&amp;quot;NQLD&amp;quot;, &amp;quot;Nlab&amp;quot;, &amp;quot;Nsyl&amp;quot;), color = &amp;quot;#238c45&amp;quot;, active = T),&#xA;+                 list(query = intersects, params = list(&amp;quot;NQLD&amp;quot;, &amp;quot;Nlab&amp;quot;), color = &amp;quot;#ffd977&amp;quot;, active = T)))&#xA;&#xA;&#xA;Error in `$&amp;lt;-.data.frame`(`*tmp*`, &amp;quot;freq&amp;quot;, value = 45L) : &#xA;  replacement has 1 row, data has 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How is it possible to fix the above error?&lt;/p&gt;&#xA;" OwnerUserId="3329" LastActivityDate="2022-11-28T11:40:53.433" Title="replacement has 1 row, data has 0" Tags="|r|ggplot2|dataframe|upsetr|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20052" PostTypeId="1" AcceptedAnswerId="20055" CreationDate="2022-11-21T06:21:40.440" Score="4" ViewCount="382" Body="&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://onlinelibrary.wiley.com/doi/10.1002/jcc.23330&quot; rel=&quot;nofollow noreferrer&quot;&gt;High–quality protein backbone reconstruction from alpha carbons using Gaussian mixture models&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The above research paper is about a software tool for reconstructing a protein's main chain model only from its alpha-carbon backbone (aka, C-alpha trace).&lt;/p&gt;&#xA;&lt;p&gt;This tool provides two options: conversion (1) with energy minimization, and (2) without energy minimization.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.intechopen.com/chapters/74105&quot; rel=&quot;nofollow noreferrer&quot;&gt;Energy Minimization&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The above paper says that:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;...&lt;br /&gt;&#xA;The goal of energy Minimization is to find a set of coordinates representing the minimum energy conformation for the given structure.&lt;br /&gt;&#xA;...&lt;br /&gt;&#xA;In molecular modeling we tend to are particularly curious about minimum points on the energy surface. Minimum energy arrangements of the atoms correspond to stable states of the system; any movement off from a minimum provides a configuration with a better energy.&lt;br /&gt;&#xA;...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I haven't understood the quote from the second paper.&lt;/p&gt;&#xA;&lt;p&gt;Why do we need to know the minimum energy conformation of a protein chain? I.e., why do we want to do &lt;em&gt;energy minimization&lt;/em&gt;?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastActivityDate="2022-11-21T14:42:08.557" Title="Why do we need to find minimum energy in a protein chain?" Tags="|proteins|protein-structure|protein-protein-interaction|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20053" PostTypeId="1" CreationDate="2022-11-21T07:07:31.490" Score="0" ViewCount="33" Body="&lt;p&gt;I calculated the distance between C-alphas using Julia,&#xA;but there is no way to extract distance map data from Julia.&#xA;I don't even know the data file form, png or csv or etc.&#xA;I don't want to extract it as an image, I just want all raw data.&lt;/p&gt;&#xA;" OwnerUserId="16527" LastEditorUserId="3694" LastEditDate="2022-11-21T08:33:30.557" LastActivityDate="2022-11-23T04:53:12.517" Title="How can I extract the Protein distance map data in Julia?" Tags="|proteins|protein-protein-interaction|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20054" PostTypeId="1" AcceptedAnswerId="20066" CreationDate="2022-11-21T10:03:15.800" Score="1" ViewCount="192" Body="&lt;p&gt;I tried adding the F_MISSING tag using bcftools 1.16. When I run this command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools +fill-tags input.vcf.gz -- -t 'F_MISSING' | bcftools view -i 'INFO/F_MISSING&amp;lt;0.25' -Oz -o output.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error parsing &amp;quot;--tags F_MISSING&amp;quot;: the tag &amp;quot;F_MISSING&amp;quot; is not supported&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This command runs fine using bcftools 1.15. However, version 1.15 gives complications with other packages I use in my snakefile. Do you maybe know alternatives for how to add F_MISSING using bcftls 1.16 (if possible)?&lt;/p&gt;&#xA;&lt;p&gt;When I type &lt;code&gt;bcftools +fill-tags --version&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;p&gt;bcftools 1.9 using htslib 1.9&lt;/p&gt;&#xA;&lt;p&gt;plugin at 1.9 using htslib 1.9&lt;/p&gt;&#xA;&lt;p&gt;##SOLUTION##&lt;/p&gt;&#xA;&lt;p&gt;Indeed the issue was that I was not installing the most recent version of Conda.&#xA;I solved it by changing the .condarc file to &lt;strong&gt;solelely&lt;/strong&gt; include the following lines:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;channels:&#xA;    - conda-forge&#xA;    - bioconda&#xA;    - defaults&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The order is crucial as well.&lt;/p&gt;&#xA;" OwnerUserId="16530" LastEditorUserId="16530" LastEditDate="2022-11-23T14:15:12.590" LastActivityDate="2022-11-23T22:11:18.167" Title="Bcftools 1.16 able to add F_MISSING tag?" Tags="|bcftools|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="20055" PostTypeId="2" ParentId="20052" CreationDate="2022-11-21T14:42:08.557" Score="6" Body="&lt;p&gt;After translation, proteins fold to achieve a specific 3D structure which allows them to perform their specific functions. The structure of a protein computationally inferred through energy minimization often corresponds quite well to that specific, real-life folded 3D structure.&lt;/p&gt;&#xA;&lt;p&gt;If you are interested in the exact reason why proteins tend to fold to a conformation of minimal energy (even though this is an oversimplification of what's really happening) you would have to dive a bit deeper into the thermodynamics of proteins: &lt;a href=&quot;https://en.wikipedia.org/wiki/Protein_dynamics&quot; rel=&quot;noreferrer&quot;&gt;https://en.wikipedia.org/wiki/Protein_dynamics&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR: Energy minimization is often used to get an idea of the protein's real-life conformation (i.e. structure) through computational means.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="14627" LastActivityDate="2022-11-21T14:42:08.557" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20057" PostTypeId="1" AcceptedAnswerId="20107" CreationDate="2022-11-21T21:51:50.067" Score="1" ViewCount="21" Body="&lt;p&gt;I am reading the Online methods of the &lt;a href=&quot;https://www.nature.com/articles/nbt.4042&quot; rel=&quot;nofollow noreferrer&quot;&gt;demuxlet paper&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The genotype &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is taken from the set &lt;span class=&quot;math-container&quot;&gt;$\{0, 1, 2\}$&lt;/span&gt;, defined as &amp;quot;the true genotype of the sample corresponding to &lt;span class=&quot;math-container&quot;&gt;$c$&lt;/span&gt;-th droplet at &lt;span class=&quot;math-container&quot;&gt;$v$&lt;/span&gt;-th variant&amp;quot;.&#xA;What is the mapping between these values and the corresponding genotype?&lt;/p&gt;&#xA;&lt;p&gt;Each variant is modeled using a binomial with &lt;span class=&quot;math-container&quot;&gt;$n=2,p=\frac{g}{2}$&lt;/span&gt;, so I would assume that 1 is heterozygous, 0 is homozygous REF and 2 homozygous ALT. Is this assumption correct?&lt;/p&gt;&#xA;" OwnerUserId="1771" LastActivityDate="2022-11-30T23:39:32.950" Title="Definition of genotype in demuxlet" Tags="|single-cell|genotyping|demultiplexing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20058" PostTypeId="1" CreationDate="2022-11-22T07:18:13.080" Score="1" ViewCount="213" Body="&lt;p&gt;I was trying to format my GWAS summary statistics with the &lt;code&gt;MungeSumstats&lt;/code&gt; R package and would need to install &lt;code&gt;SNPlocs.Hsapiens.dbSNP155.GRCh37&lt;/code&gt; and &lt;code&gt;BSgenome.Hsapiens.1000genomes.hs37d5&lt;/code&gt; as reference panel. &lt;br /&gt;&#xA;I typed&#xA;&lt;code&gt;BiocManager::install(&amp;quot;SNPlocs.Hsapiens.dbSNP144.GRCh37&amp;quot;) BiocManager::install(&amp;quot;BSgenome.Hsapiens.1000genomes.hs37d5&amp;quot;)&lt;/code&gt;&#xA;but failed due to Internet connection problems. So I planned to download the files from the web browser and copy them to the R library. I was wondering where I should put the files. If they were downloaded from R, where would they be saved? &lt;br /&gt;&#xA;I would really appreciate any answers, thanks!&lt;/p&gt;&#xA;" OwnerUserId="16537" LastActivityDate="2022-11-22T19:46:54.840" Title="file path of BiocManager:install()" Tags="|r|bioconductor|reference-genome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20059" PostTypeId="2" ParentId="20023" CreationDate="2022-11-22T14:53:20.653" Score="1" Body="&lt;p&gt;You have defined &lt;code&gt;mem_gb&lt;/code&gt; but not &lt;code&gt;mem_mb&lt;/code&gt; in your resource configuration file. You can set them to same value by adding something like &lt;code&gt;mem_mb=config['mem_gb']['tool1']&lt;/code&gt;. Additional information in their &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/executing/cli.html#profiles&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2022-11-22T14:53:20.653" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20060" PostTypeId="2" ParentId="20058" CreationDate="2022-11-22T18:25:52.597" Score="3" Body="&lt;p&gt;&lt;strong&gt;First, I would try again to download via the bioconductor command. If your internet is intermittent it may just take a few tries.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;If that doesn't work, I would then strongly recommend using &lt;code&gt;BioCManager&lt;/code&gt; &lt;a href=&quot;https://www.biostars.org/p/157366/&quot; rel=&quot;nofollow noreferrer&quot;&gt;to install from a local source&lt;/a&gt; wherever you've downloaded the files. If you try to copy the files around there are a lot of ways that things can go wrong, and there are a lot of ways that it still might not work even if you put it in the right place, if you don't use the package manager as intended.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2022-11-22T19:46:54.840" LastActivityDate="2022-11-22T19:46:54.840" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20061" PostTypeId="1" CreationDate="2022-11-22T20:35:05.273" Score="1" ViewCount="33" Body="&lt;p&gt;The following are the first 14 atoms from the protein &lt;strong&gt;1CRN&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATOM      1  N   THR A   1      17.047  14.099   3.625  1.00 13.79           N  &#xA;ATOM      2  CA  THR A   1      16.967  12.784   4.338  1.00 10.80           C  &#xA;ATOM      3  C   THR A   1      15.685  12.755   5.133  1.00  9.19           C  &#xA;ATOM      4  O   THR A   1      15.268  13.825   5.594  1.00  9.85           O  &#xA;ATOM      5  CB  THR A   1      18.170  12.703   5.337  1.00 13.02           C  &#xA;ATOM      6  OG1 THR A   1      19.334  12.829   4.463  1.00 15.06           O  &#xA;ATOM      7  CG2 THR A   1      18.150  11.546   6.304  1.00 14.23           C  &#xA;&#xA;ATOM      8  N   THR A   2      15.115  11.555   5.265  1.00  7.81           N  &#xA;ATOM      9  CA  THR A   2      13.856  11.469   6.066  1.00  8.31           C  &#xA;ATOM     10  C   THR A   2      14.164  10.785   7.379  1.00  5.80           C  &#xA;ATOM     11  O   THR A   2      14.993   9.862   7.443  1.00  6.94           O  &#xA;ATOM     12  CB  THR A   2      12.732  10.711   5.261  1.00 10.32           C  &#xA;ATOM     13  OG1 THR A   2      13.308   9.439   4.926  1.00 12.81           O  &#xA;ATOM     14  CG2 THR A   2      12.484  11.442   3.895  1.00 11.90           C  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These fourteen atoms are members of two &lt;strong&gt;THR&lt;/strong&gt; residues.&lt;/p&gt;&#xA;&lt;p&gt;Can the information regarding atomic bonds be interpreted only from these fourteen lines?&lt;/p&gt;&#xA;&lt;p&gt;For example,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the first atom, N, has three bonds attached to CA.&lt;/li&gt;&#xA;&lt;li&gt;the second atom, CA, has four bonds; three are attached to N, and one is to C.&lt;/li&gt;&#xA;&lt;li&gt;the third atom, C, has four bonds; one is attached to CA, one to oxygen, and two are empty.&lt;/li&gt;&#xA;&lt;li&gt;the fourth atom, O, has two bonds; one is attached to the third carbon, and one is to the fifth atom beta-carbon.&lt;/li&gt;&#xA;&lt;li&gt;the fifth atom, beta-carbon, has four bonds; one is attached to the fourth atom, two are attached to the first branch (OG1), and one is attached to the second branch (CG2).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Or do I have to use my background knowledge of amino acids to find bonds?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note:&lt;/strong&gt; I am not talking about hydrogen bonds. I am talking about atomic bonds only in the atoms in the residue.&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="12686" LastEditDate="2022-11-22T22:43:33.073" LastActivityDate="2022-11-24T18:46:32.797" Title="Can the information regarding atomic bonds be interpreted only from atom lines in a PDB file?" Tags="|proteins|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20062" PostTypeId="2" ParentId="20053" CreationDate="2022-11-23T04:50:36.550" Score="1" Body="&lt;p&gt;for someone who have same question as mine...&#xA;this helps you to extract distance map data file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;julia&amp;gt; using DelimitedFiles&#xA;&#xA;julia&amp;gt; writedlm(&amp;quot;calphadistances.out&amp;quot;,dm.data,&amp;quot;&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16527" LastEditorUserId="3848" LastEditDate="2022-11-23T04:53:12.517" LastActivityDate="2022-11-23T04:53:12.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20063" PostTypeId="2" ParentId="20061" CreationDate="2022-11-23T10:56:16.683" Score="1" Body="&lt;p&gt;You can use topology files to extract bond information.&#xA;Here is an example at &lt;a href=&quot;https://charmm-gui.org/?doc=open_toppar&amp;amp;filename=top_all36_prot.rtf&quot; rel=&quot;nofollow noreferrer&quot;&gt;CHARMM36&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6717" LastActivityDate="2022-11-23T10:56:16.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20064" PostTypeId="2" ParentId="19860" CreationDate="2022-11-23T11:11:03.513" Score="1" Body="&lt;p&gt;You should make violin plots of the genes of interest (x-axis is your different clusters (i.e. cell-types) and y-axis is expression). Maybe they are differentially expressed but at the same time not expressed very highly.&lt;/p&gt;&#xA;&lt;p&gt;Also why does their expression matter in other cell types? Do you have strong evidence to expect that these genes would only be expressed in podocytes?&lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="383" LastEditDate="2022-11-23T12:01:49.503" LastActivityDate="2022-11-23T12:01:49.503" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20066" PostTypeId="2" ParentId="20054" CreationDate="2022-11-23T22:11:18.167" Score="0" Body="&lt;p&gt;The issue was that not installing the most recent version of Conda.&#xA;It was solved by changing the .condarc file to &lt;strong&gt;solely&lt;/strong&gt; include the following lines:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;channels:&#xA;    - conda-forge&#xA;    - bioconda&#xA;    - defaults&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The order is crucial as well.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Converted to answer from question.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2022-11-23T22:11:18.167" CommentCount="0" CommunityOwnedDate="2022-11-23T22:11:18.167" ContentLicense="CC BY-SA 4.0" />
  <row Id="20067" PostTypeId="1" CreationDate="2022-11-24T14:07:06.070" Score="2" ViewCount="29" Body="&lt;p&gt;When parsing &lt;a href=&quot;https://toolshed.g2.bx.psu.edu/repository/display_tool?repository_id=1d92ebdf7e8d466c&amp;amp;tool_config=%2Fsrv%2Ftoolshed-repos%2Fmain%2F000%2Frepo_241%2Ftools%2Fncbi_blast_plus%2Fncbi_segmasker_wrapper.xml&amp;amp;changeset_revision=5edc472ec434&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;NCBI BLAST+ segmasker&lt;/em&gt;&lt;/a&gt; output, I realized the program modifies almost every header containing these strings: &lt;code&gt;( &lt;/code&gt; or &lt;code&gt; )&lt;/code&gt; for no (known to me) reason.&lt;/p&gt;&#xA;&lt;p&gt;In the majority of the strings mentioned above, the whitespace is removed:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;input file:&lt;br /&gt;&#xA;&lt;code&gt;&amp;gt;tr|Q9UXV4|Q9UXV4_PYRAB Pyruvate formate-lyase activating enzyme ( act-1) OS=Pyrococcus abyssi (strain GE5 / Orsay) OX=272844 GN=act-1 PE=4 SV=1&lt;/code&gt;,&lt;br /&gt;&#xA;output file:&lt;br /&gt;&#xA;&lt;code&gt;&amp;gt;tr|Q9UXV4|Q9UXV4_PYRAB Pyruvate formate-lyase activating enzyme (act-1) OS=Pyrococcus abyssi (strain GE5 / Orsay) OX=272844 GN=act-1 PE=4 SV=1&lt;/code&gt;,&lt;/li&gt;&#xA;&lt;li&gt;input file:&lt;br /&gt;&#xA;&lt;code&gt;&amp;gt;tr|A0A5K1JWI6|A0A5K1JWI6_9APHY Cysteine protease ATG4 ) OS=Ganoderma boninense OX=34458 GN=Q523C3 PE=4 SV=1&lt;/code&gt;,&lt;br /&gt;&#xA;output file:&lt;br /&gt;&#xA;&lt;code&gt;&amp;gt;tr|A0A5K1JWI6|A0A5K1JWI6_9APHY Cysteine protease ATG4) OS=Ganoderma boninense OX=34458 GN=Q523C3 PE=4 SV=1&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;It isn't a significant change, but it complicates searching for headers in the output file.&lt;/p&gt;&#xA;&lt;p&gt;However, the program is not consistent - if there is a comma (&lt;code&gt;,&lt;/code&gt;) before the &lt;code&gt; )&lt;/code&gt; string, the whitespace is not removed:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;input file:&lt;br /&gt;&#xA;&lt;code&gt;&amp;gt;tr|A0A3M9YA12|A0A3M9YA12_9PEZI Myeloid lymphoid or mixed-lineage leukemia 5 (Trithorax, ) OS=Verticillium nonalfalfae OX=1051616 GN=MLL5 PE=4 SV=1&lt;/code&gt;,&lt;br /&gt;&#xA;output file:&lt;br /&gt;&#xA;&lt;code&gt;&amp;gt;tr|A0A3M9YA12|A0A3M9YA12_9PEZI Myeloid lymphoid or mixed-lineage leukemia 5 (Trithorax, ) OS=Verticillium nonalfalfae OX=1051616 GN=MLL5 PE=4 SV=1&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;What's the reason for such a behaviour?&lt;br /&gt;&#xA;Is there any option to prevent it and keep headers unchanged?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;edit.&lt;br /&gt;&#xA;The whitespace is also removed for:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;  &lt;/code&gt; → &lt;code&gt; &lt;/code&gt;,&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt; ,&lt;/code&gt; → &lt;code&gt;,&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Moreover, it removes a semicolon when it is at the end of a header:&lt;br /&gt;&#xA;&lt;code&gt;&amp;gt;ABCD/1-4 ABCD;ABCD;&lt;/code&gt; → &lt;code&gt;&amp;gt;ABCD/1-4 ABCD;ABCD&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9651" LastEditorUserId="9651" LastEditDate="2022-11-26T18:02:40.263" LastActivityDate="2022-11-26T18:02:40.263" Title="Why does segmasker modify headers?" Tags="|phylogenetics|segmasker|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20068" PostTypeId="1" AcceptedAnswerId="20072" CreationDate="2022-11-24T18:17:04.437" Score="1" ViewCount="51" Body="&lt;p&gt;Let us consider the first seven atoms of the  protein &lt;strong&gt;&lt;code&gt;1CRN.pdb&lt;/code&gt;&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATOM      1  N   THR A   1      17.047  14.099   3.625  1.00 13.79           N  &#xA;ATOM      2  CA  THR A   1      16.967  12.784   4.338  1.00 10.80           C  &#xA;ATOM      3  C   THR A   1      15.685  12.755   5.133  1.00  9.19           C  &#xA;ATOM      4  O   THR A   1      15.268  13.825   5.594  1.00  9.85           O  &#xA;ATOM      5  CB  THR A   1      18.170  12.703   5.337  1.00 13.02           C  &#xA;ATOM      6  OG1 THR A   1      19.334  12.829   4.463  1.00 15.06           O  &#xA;ATOM      7  CG2 THR A   1      18.150  11.546   6.304  1.00 14.23           C  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This represents an amino acid called &lt;em&gt;&lt;strong&gt;Threonine&lt;/strong&gt;&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The general bond diagram for Threonine is as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/etZHI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/etZHI.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, it has at least six bonds (I am actually not sure!) which are also evident from the seven &lt;code&gt;ATOM&lt;/code&gt; lines.&lt;/p&gt;&#xA;&lt;p&gt;Each atom has its radius. In general, if the distance between two atoms is less than the sum of the radii, there is a bond between these two atoms.&lt;/p&gt;&#xA;&lt;p&gt;Based on the above information, I tried to identify the bond information, and I could detect only two:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;     atom1           atom2        distance       atm1 Radi      atom2 Radi        sum radi        is bond?&#xA;         N              CA            1.50            0.65            0.70            1.35           False&#xA;         N               C            2.44            0.65            0.70            1.35           False&#xA;         N               O            2.67            0.65            0.60            1.25           False&#xA;         N              CB            2.48            0.65            0.70            1.35           False&#xA;         N             OG1            2.75            0.65            0.60            1.25           False&#xA;         N             CG2            3.86            0.65            0.60            1.25           False&#xA;        CA               N            1.50            0.70            0.65            1.35           False&#xA;        CA               C            1.51            0.70            0.70            1.40           False&#xA;        CA               O            2.36            0.70            0.60            1.30           False&#xA;        CA              CB            1.57            0.70            0.70            1.40           False&#xA;        CA             OG1            2.37            0.70            0.60            1.30           False&#xA;        CA             CG2            2.61            0.70            0.60            1.30           False&#xA;         C               N            2.44            0.70            0.65            1.35           False&#xA;         C              CA            1.51            0.70            0.70            1.40           False&#xA;         C               O            1.24            0.70            0.60            1.30            True&#xA;         C              CB            2.49            0.70            0.70            1.40           False&#xA;         C             OG1            3.71            0.70            0.60            1.30           False&#xA;         C             CG2            2.98            0.70            0.60            1.30           False&#xA;         O               N            2.67            0.60            0.65            1.25           False&#xA;         O              CA            2.36            0.60            0.70            1.30           False&#xA;         O               C            1.24            0.60            0.70            1.30            True&#xA;         O              CB            3.12            0.60            0.70            1.30           False&#xA;         O             OG1            4.34            0.60            0.60            1.20           False&#xA;         O             CG2            3.74            0.60            0.60            1.20           False&#xA;        CB               N            2.48            0.70            0.65            1.35           False&#xA;        CB              CA            1.57            0.70            0.70            1.40           False&#xA;        CB               C            2.49            0.70            0.70            1.40           False&#xA;        CB               O            3.12            0.70            0.60            1.30           False&#xA;        CB             OG1            1.46            0.70            0.60            1.30           False&#xA;        CB             CG2            1.51            0.70            0.60            1.30           False&#xA;       OG1               N            2.75            0.60            0.65            1.25           False&#xA;       OG1              CA            2.37            0.60            0.70            1.30           False&#xA;       OG1               C            3.71            0.60            0.70            1.30           False&#xA;       OG1               O            4.34            0.60            0.60            1.20           False&#xA;       OG1              CB            1.46            0.60            0.70            1.30           False&#xA;       OG1             CG2            2.54            0.60            0.60            1.20           False&#xA;       CG2               N            3.86            0.60            0.65            1.25           False&#xA;       CG2              CA            2.61            0.60            0.70            1.30           False&#xA;       CG2               C            2.98            0.60            0.70            1.30           False&#xA;       CG2               O            3.74            0.60            0.60            1.20           False&#xA;       CG2              CB            1.51            0.60            0.70            1.30           False&#xA;       CG2             OG1            2.54            0.60            0.60            1.20           False&#xA;2&#xA;Done!&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I detect all bonds?&lt;/p&gt;&#xA;&lt;p&gt;EDIT: As per pippo1980's request:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-csharp prettyprint-override&quot;&gt;&lt;code&gt;public class ADTResidue: Dictionary&amp;lt;int, ADTAtom&amp;gt;, IEquatable&amp;lt;ADTResidue&amp;gt;&#xA;{&#xA;    public string ProteinName { get; set; }&#xA;    public string ChainName { get; set; }&#xA;    public int ResidueNo { get; set; }&#xA;    public Point3d Coordinate { get; private set; }&#xA;    public string Code { get; set; }&#xA;    public string Full { get; set; }&#xA;    public Color ColorRGB { get; set; }&#xA;&#xA;    &#xA;    public ADTResidue(string code, string full, Color colorRgb)&#xA;    {&#xA;        Coordinate = new Point3d();&#xA;&#xA;        Code = code;&#xA;        Full = full;&#xA;        ColorRGB = colorRgb;&#xA;    }&#xA;&#xA;    private readonly Dictionary&amp;lt;int, CovalentBond&amp;gt; BondDictionary;&#xA;&#xA;    public List&amp;lt;CovalentBond&amp;gt; GetBonds()&#xA;    {&#xA;        return new List&amp;lt;CovalentBond&amp;gt;(BondDictionary.Values);&#xA;    }&#xA;&#xA;    public ADTResidue()&#xA;    {&#xA;        BondDictionary = new Dictionary&amp;lt;int, CovalentBond&amp;gt;();&#xA;    }&#xA;&#xA;    public void CalculateBonds()&#xA;    {&#xA;        foreach (ADTAtom atom1 in this.Values)&#xA;        {&#xA;            foreach (ADTAtom atom2 in this.Values)&#xA;            {&#xA;                if (atom2.Radius &amp;lt;= 0)&#xA;                {&#xA;                    throw new Exception(&amp;quot;zero radius in &amp;quot; + atom2.Name);&#xA;                }&#xA;&#xA;                if (atom1 != atom2)&#xA;                {&#xA;                    double distance = Point3d.Distance(atom1.Coordinate, atom2.Coordinate);&#xA;&#xA;                    bool exists = ABond.IsBondExists(atom1.Coordinate, atom2.Coordinate, atom1.Radius, atom2.Radius);&#xA;&#xA;                    if (exists)&#xA;                    {&#xA;                        int hashCode = atom1.GetHashCode(atom2);&#xA;                        if (!BondDictionary.ContainsKey(hashCode))&#xA;                        {&#xA;                            BondDictionary.Add(hashCode, new CovalentBond(atom1, atom2));&#xA;                        }&#xA;                    }&#xA;                }&#xA;            }&#xA;        }&#xA;&#xA;        int count = BondDictionary.Count;&#xA;&#xA;        Console.WriteLine(count);&#xA;    }&#xA;&#xA;    #region equality comparison&#xA;    // IEquatable&amp;lt;&amp;gt; implementation...&#xA;    #endregion&#xA;} &#xA;&#xA;public abstract class ABond: Pair&amp;lt;ADTAtom, ADTAtom&amp;gt;&#xA;{&#xA;    public ABond():base()&#xA;    {&#xA;    }&#xA;&#xA;    public ABond(ADTAtom first, ADTAtom second) : base(first, second)&#xA;    {&#xA;    }&#xA;&#xA;    public static bool IsBondExists(Point3d first, Point3d second, double radius1, double radius2)&#xA;    {&#xA;        double distance = Point3d.Distance(first, second);&#xA;&#xA;        return (distance &amp;lt;= (radius1 + radius2));&#xA;    }&#xA;}&#xA;&#xA;public class CovalentBond : ABond&#xA;{&#xA;    public CovalentBond():base()&#xA;    {&#xA;    }&#xA;&#xA;    public CovalentBond(ADTAtom first, ADTAtom second) : base(first, second)&#xA;    {&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&#xA;" OwnerUserId="12686" LastEditorUserId="12686" LastEditDate="2022-11-26T18:19:29.493" LastActivityDate="2022-11-26T18:19:29.493" Title="Why am I not being able to detect all the bonds in a residue?" Tags="|proteins|protein-structure|pdb|3d-structure|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="20069" PostTypeId="2" ParentId="20061" CreationDate="2022-11-24T18:46:32.797" Score="2" Body="&lt;p&gt;The official list of bonds in each residue in the PDB is in the &lt;a href=&quot;https://www.wwpdb.org/data/ccd&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chemical Component Dictionary&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You could also use interatomic distances to infer connectivity, but that's less reliable.&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2022-11-24T18:46:32.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20071" PostTypeId="1" CreationDate="2022-11-24T20:26:09.047" Score="0" ViewCount="96" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/9546317/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Hi, I am very new to this area, and I am taking a class about bioinformatics. For an independent project assignment, I need to do a GWAS. I am using the bash terminal. I downloaded all the fastq I need,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;trimmed the files,&lt;/li&gt;&#xA;&lt;li&gt;converted them into bam/sam&lt;/li&gt;&#xA;&lt;li&gt;then performed vcf&lt;/li&gt;&#xA;&lt;li&gt;and then bed/bim/fam etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, when I tried to perform GWAS in plink, I realized I dont have phenotype data. It supposed to have two phenotypes.&lt;/p&gt;&#xA;&lt;p&gt;There are two groups/phenotypes of fastq files, each containing 29 samples. Let's say they are group 1 and 2.&lt;/p&gt;&#xA;&lt;p&gt;For each group, I converted every fastq to &lt;code&gt;sam&lt;/code&gt; then &lt;code&gt;bam&lt;/code&gt;, then I combined 29 bam to one bam. Then I combined two bams (for the two groups) together to a vcf.gz.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Problem.&lt;/strong&gt; There is no phenotype data in the following plink files.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Questions&lt;/strong&gt;&#xA;What step has been wrong?&#xA;Or what I should do to incorporate the phenotype data?&lt;/p&gt;&#xA;&lt;p&gt;I dont have to be perfect (like the QC steps), and I cannot understand too complicated codes. I am seeking a simple output such as a Manhattan plot or alternative pipeline.&lt;/p&gt;&#xA;" OwnerUserId="16548" LastEditorUserId="73" LastEditDate="2022-11-25T23:20:35.703" LastActivityDate="2022-11-25T23:20:35.703" Title="Help sought with GWAS and vcf files, lack phenotype labels" Tags="|sam|vcf|gwas|homework|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20072" PostTypeId="2" ParentId="20068" CreationDate="2022-11-24T23:29:11.930" Score="3" Body="&lt;p&gt;Your method doesn't work for two possible reasons;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The radius information may be incorrect.&lt;/li&gt;&#xA;&lt;li&gt;Your molecule requires some minimization. This option helps to optimize distance between atoms.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;You can find some information about the minimization at the following links&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Energy_minimization&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://manual.gromacs.org/documentation/current/reference-manual/algorithms/energy-minimization.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gromacs&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="6717" LastActivityDate="2022-11-24T23:29:11.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20073" PostTypeId="1" CreationDate="2022-11-25T00:19:14.383" Score="-1" ViewCount="82" Body="&lt;p&gt;&lt;a href=&quot;https://www.10xgenomics.com/resources/datasets/20-k-human-pbm-cs-5-ht-v-2-0-2-high-6-1-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;10x Genomics: 20k Human PBMCs&lt;/a&gt; is the dataset.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cWXuH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cWXuH.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Description of the dataset:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Inputs/Libraries&#xA;Human peripheral blood mononuclear cells (PBMCs) of a healthy male donor aged 30-35 were obtained by 10x Genomics from AllCells.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Gene expression and V(D)J libraries were generated from ~33,000 cells (18,470 cells recovered) as described in the Chromium Next GEM Single Cell 5' HT Reagent Kits v2 User Guide (CG000421) using the Chromium and sequenced on an Illumina NovaSeq 6000 to a read depth of approximately 25,000 mean reads per cell for Gene Expression, 15,000 mean reads per cell for TCR Amplified, and 25,000 mean reads per cell for BCR Amplified libraries.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Paired-end, dual indexing&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Read 1: 26 cycles (16 bp barcode, 10 bp UMI) &lt;br /&gt;&#xA;i5 index: 10 cycles (sample index) &lt;br /&gt;&#xA;i7 index: 10 cycles (sample index) &lt;br /&gt;&#xA;Read 2: 90 cycles (transcript)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is the &lt;a href=&quot;https://rpubs.com/ap3340/SingleCellHumanPBMC&quot; rel=&quot;nofollow noreferrer&quot;&gt;RPubs Link&lt;/a&gt; for the Single Cell Analysis work I've performed on it.&lt;/p&gt;&#xA;&lt;p&gt;Can someone confirm that I can do both Single-Cell RNA Seq Analysis &lt;em&gt;AND&lt;/em&gt; Differential Gene Expression Analysis?&lt;/p&gt;&#xA;&lt;p&gt;I know that Expression Analysis comes from having a control and a treatment group. I honestly don't know anything about bioinformatics but I'm diving into the fire to get a solid understanding once I've coded everything.&lt;/p&gt;&#xA;&lt;p&gt;I followed &lt;a href=&quot;https://www.youtube.com/watch?v=5HBzgsz8qyk&amp;amp;ab_channel=Bioinformagician&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt; and want to follow &lt;a href=&quot;https://www.youtube.com/watch?v=OzNzO8qwwp0&amp;amp;ab_channel=Bioinformagician&quot; rel=&quot;nofollow noreferrer&quot;&gt;this other video&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Thank you guys.&lt;/p&gt;&#xA;" OwnerUserId="16549" LastEditorUserId="16549" LastEditDate="2022-11-25T23:52:01.060" LastActivityDate="2023-08-23T00:03:40.400" Title="Can I perform Expression Analysis on this 10x dataset?" Tags="|r|rna-seq|python|differential-expression|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20074" PostTypeId="2" ParentId="20073" CreationDate="2022-11-25T16:39:44.943" Score="0" Body="&lt;p&gt;Typically what this PMBC RNA-seq data set would be used for is for a control against individuals who would exhibit a pronounced cellular immune response. For example, following a COVID19 infection the PMBCs would be have lots of T-cell CD4+ and CD8+ and are essential for controlling the infection. Thus it is a really good approach to understand the reasons for different clinical outcomes, such chronic infection and severe infections.&lt;/p&gt;&#xA;&lt;p&gt;I would suspect a cancer patient would show a pronounced shift in cellular immune response, but I'm not certain. Anyway, by comparing a control data set against a COVID19 patient a better understanding of the cellular immune response can be obtained and a differential gene expression analysis would be an important inroad to assess the T-cell up-regulation. DEG would show very pronounced differences between a healthy individual and a patient.&lt;/p&gt;&#xA;&lt;p&gt;Having a hypothesis about the expected results from DEG is important, because interpreting DEG output is otherwise difficult.&lt;/p&gt;&#xA;&lt;p&gt;With regards your question, a DEG analysis IMO would be a good analysis and is easy to perform, however there only appears to be a single sample. A DEG analysis in context is control X vs patient Y. The sample you have appears to be a control sample so you would need a second sample. Performing DEG is easy, but do stress that interpreting DEG isn't easy because everything is relative. Thus what is sought usually in PMBC comparisons healthy vs patient is evidence of T-cell CD8+/CD4+ up regulation, any indication about which possibly which HLA type is active (I dunno whether RNAseq will do this - good if it can) and information on the signalling cascade. Basically as much information on the cellular immune response that can be gleaned via RNAseq without performing laborious and difficult immunological experiments requiring specialist apparatus.&lt;/p&gt;&#xA;&lt;p&gt;There are investigators here with more experience than I in eukaryotic RNAseq analysis, so there input would be helpful.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-11-25T17:23:52.987" LastActivityDate="2022-11-25T17:23:52.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20076" PostTypeId="5" CreationDate="2022-11-26T11:21:20.533" Score="0" Body="&lt;p&gt;&lt;a href=&quot;https://toolshed.g2.bx.psu.edu/repository/display_tool?repository_id=1d92ebdf7e8d466c&amp;amp;tool_config=%2Fsrv%2Ftoolshed-repos%2Fmain%2F000%2Frepo_241%2Ftools%2Fncbi_blast_plus%2Fncbi_segmasker_wrapper.xml&amp;amp;changeset_revision=5edc472ec434&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI BLAST+ segmasker&lt;/a&gt; is a part of the BLAST (Basic Local Alignment Search Tool) command line applications developed at the National Center for Biotechnology Information (NCBI).&lt;/p&gt;&#xA;&lt;p&gt;It identifies and masks out low complexity regions of a protein database (or proteins in FASTA format) by using the &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/8743706/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SEG&lt;/a&gt; algorithm.&lt;/p&gt;&#xA;" OwnerUserId="9651" LastEditorUserId="9651" LastEditDate="2022-11-26T12:17:54.440" LastActivityDate="2022-11-26T12:17:54.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20077" PostTypeId="4" CreationDate="2022-11-26T11:21:20.533" Score="0" Body="Segmasker identifies and masks out low complexity regions of a proteins by using the SEG algorithm." OwnerUserId="9651" LastEditorUserId="9651" LastEditDate="2022-11-26T12:17:45.060" LastActivityDate="2022-11-26T12:17:45.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20078" PostTypeId="5" CreationDate="2022-11-26T12:32:28.070" Score="0" Body="&lt;p&gt;In computer technology, a bug is a coding error in a computer program.&lt;/p&gt;&#xA;&lt;p&gt;Major faults in code could be also termed 'errors', but within bioinformatics this generally refers to problems with wet-lab procedure (&lt;a href=&quot;https://bioinformatics.stackexchange.com/tags/errors/info&quot;&gt;more&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="9651" LastEditorUserId="9651" LastEditDate="2022-11-26T22:28:45.840" LastActivityDate="2022-11-26T22:28:45.840" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20079" PostTypeId="4" CreationDate="2022-11-26T12:32:28.070" Score="0" Body="In computer technology, a bug is a coding error in a computer program." OwnerUserId="9651" LastEditorUserId="9651" LastEditDate="2022-11-26T22:28:30.577" LastActivityDate="2022-11-26T22:28:30.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20080" PostTypeId="1" CreationDate="2022-11-26T20:38:00.220" Score="1" ViewCount="22" Body="&lt;p&gt;During &lt;a href=&quot;https://toolshed.g2.bx.psu.edu/repository/display_tool?repository_id=1d92ebdf7e8d466c&amp;amp;tool_config=%2Fsrv%2Ftoolshed-repos%2Fmain%2F000%2Frepo_241%2Ftools%2Fncbi_blast_plus%2Fncbi_segmasker_wrapper.xml&amp;amp;changeset_revision=5edc472ec434&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI BLAST+ segmasker&lt;/a&gt; output exploration, I found out that for some sequences, the program returns the same interval twice:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;tr|A0A0D2B4V6|A0A0D2B4V6_9EURO Uncharacterized protein OS=Cladophialophora immunda OX=569365 GN=PV07_04097 PE=3 SV=1&#xA;131 - 139&#xA;131 - 139&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This seems not to be a planned behaviour, but maybe someone has an explanation.&lt;/p&gt;&#xA;&lt;p&gt;The command I ran was:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;segmasker -in &amp;lt;input_file&amp;gt; -out &amp;lt;output_file&amp;gt; -locut 1.5 -hicut 1.8 -window 15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Exemplary sequences triggering such an event:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;sequence1&#xA;MDSHNSSTLIILSLPPKTFVGLDLVSFDSSPNFQGIAKIPVGIHFLYTGTDASLSIRHGRWLNLTSTHPIQVLRWNSDGETLQPVDQNDQTARNAIAAISNGRGLVDYGALQSATSELSTRESQSHDASSLAAAAAAAAADEDRAESTDWPALTAHISSPLLTRILSPDWIVSSISSAPTDTETIPGLSHLEASNALHQLPLDLLPVNLKQTWAGGDIGRARTERARDRSWYLGQLIEQAARPQGGDKTAGAREVLGELQFCFLMVLTLANYSCLEQWKRLLSVFFTCRTALDEVEGYFVQVIRVLRLQIKHVDDVEGGLFELRDESGSAWLRTLWARFRSVVDDADKGERVSLKKEVDALEKLFEGKYGWQSERDILRRGMLELEDGERVEVTMHGVDEDEETGEYAPVIVET&#xA;&amp;gt;sequence2&#xA;VDKLFKKNAMGNNIFHEIAIQGSLMLLQTIRDNVDGQMDTYLSEINDQNESCITIVADRHRGLLAKDLIEIFVDLGADINETDNWGNTALHYSVFHRDHVLAEWLCQQPGINLNAINNDEHTPLGLAILWNIQSIKTLLESKGARFNNIEFSDSETSDDDDDDDDDGHDENDE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9651" LastEditorUserId="298" LastEditDate="2022-11-29T19:13:11.440" LastActivityDate="2022-11-29T19:13:11.440" Title="segmasker doubles found intervals" Tags="|phylogenetics|segmasker|bugs|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20081" PostTypeId="2" ParentId="19938" CreationDate="2022-11-26T22:04:56.027" Score="2" Body="&lt;p&gt;or you could taket it from :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.rcsb.org/structure/7WMV&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/structure/7WMV&lt;/a&gt; pdb file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;HETATM 4865  C13 1YI A 701     106.585 114.218 112.698  1.00 74.13           C  &#xA;HETATM 4866  C15 1YI A 701     106.050 112.633 114.547  1.00102.13           C  &#xA;HETATM 4867  C17 1YI A 701     104.024 111.140 110.766  1.00 89.07           C  &#xA;HETATM 4868  C20 1YI A 701     103.626 108.779 110.527  1.00 83.00           C  &#xA;HETATM 4869  C21 1YI A 701     104.626 108.507 111.437  1.00 84.86           C  &#xA;HETATM 4870  C22 1YI A 701     102.234 110.466 109.205  1.00 85.32           C  &#xA;HETATM 4871  C24 1YI A 701     100.911 110.073 109.846  1.00 66.83           C  &#xA;HETATM 4872  C26 1YI A 701      92.545 108.407 114.139  1.00 95.68           C  &#xA;HETATM 4873  C28 1YI A 701     100.141 109.053 109.319  1.00 63.43           C  &#xA;HETATM 4874  C11 1YI A 701     105.694 113.343 111.806  1.00 75.91           C  &#xA;HETATM 4875  C12 1YI A 701     105.840 111.929 112.354  1.00 91.76           C  &#xA;HETATM 4876  C14 1YI A 701     105.990 114.087 114.099  1.00 85.43           C  &#xA;HETATM 4877  C16 1YI A 701     105.029 110.853 111.663  1.00 99.03           C  &#xA;HETATM 4878  C18 1YI A 701     105.319 109.548 112.006  1.00100.47           C  &#xA;HETATM 4879  C19 1YI A 701     103.333 110.096 110.188  1.00 81.96           C  &#xA;HETATM 4880  C23 1YI A 701     104.801 110.649 116.200  1.00 88.38           C  &#xA;HETATM 4881  C25 1YI A 701     102.892 107.590 109.921  1.00 99.79           C  &#xA;HETATM 4882  C27 1YI A 701     100.517 110.745 110.979  1.00 88.80           C  &#xA;HETATM 4883  C29 1YI A 701      98.556 109.383 111.080  1.00 77.66           C  &#xA;HETATM 4884  C30 1YI A 701      97.245 109.006 111.756  1.00 83.68           C  &#xA;HETATM 4885  C31 1YI A 701      97.108 107.503 111.935  1.00 85.27           C  &#xA;HETATM 4886  C32 1YI A 701      99.328 110.410 111.602  1.00 90.75           C  &#xA;HETATM 4887  C33 1YI A 701      98.954 108.713 109.937  1.00 76.18           C  &#xA;HETATM 4888  C34 1YI A 701      96.107 107.127 113.039  1.00 98.03           C  &#xA;HETATM 4889  C35 1YI A 701      94.780 107.913 113.042  1.00 92.89           C  &#xA;HETATM 4890  C36 1YI A 701      91.696 107.883 115.298  1.00113.05           C  &#xA;HETATM 4891  C37 1YI A 701      92.889 109.868 114.425  1.00 87.90           C  &#xA;HETATM 4892  C38 1YI A 701      91.711 108.277 112.861  1.00 77.85           C  &#xA;HETATM 4893  N8  1YI A 701      93.804 107.673 114.094  1.00 93.06           N  &#xA;HETATM 4894  N9  1YI A 701      90.968 106.633 115.149  1.00106.00           N  &#xA;HETATM 4895  O2  1YI A 701     105.306 111.885 113.648  1.00 99.43           O  &#xA;HETATM 4896  O3  1YI A 701     106.050 113.444 110.457  1.00 80.66           O  &#xA;HETATM 4897  O4  1YI A 701     106.560 115.552 112.284  1.00 86.11           O  &#xA;HETATM 4898  O5  1YI A 701     106.634 114.878 115.050  1.00 99.48           O  &#xA;HETATM 4899  O6  1YI A 701      94.512 108.703 112.208  1.00 84.41           O  &#xA;HETATM 4900  O7  1YI A 701      91.625 108.487 116.316  1.00101.67           O  &#xA;HETATM 4901  S1  1YI A 701     105.267 112.404 116.160  1.00121.62           S  &#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;picture :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cVWOx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cVWOx.png&quot; alt=&quot;picture got with https://litemol.org/Viewer/&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;obtained with : &lt;a href=&quot;https://litemol.org/Viewer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://litemol.org/Viewer/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;OOOPSSSSSSSSSS&lt;/p&gt;&#xA;&lt;p&gt;realized that ligand in pdb its not the full ligand of your picture,&lt;/p&gt;&#xA;&lt;p&gt;need to get it from Ligand page on RCSB_PDB : &lt;a href=&quot;https://www.rcsb.org/ligand/1YI&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/ligand/1YI&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;download links highlighted in pics :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5nKtS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5nKtS.png&quot; alt=&quot;ligs&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;download &lt;code&gt;/1YI_ideal.sdf&lt;/code&gt;  convert via openbabel :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.cheminfo.org/Chemistry/Cheminformatics/FormatConverter/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cheminfo.org/Chemistry/Cheminformatics/FormatConverter/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;result&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;WARNING: damped steplength 0.0184262 to 0.00725371&#xA;WARNING: damped steplength 0.00508674 to 0.00100138&#xA;COMPND    1YI &#xA;AUTHOR    GENERATED BY OPEN BABEL 2.3.2&#xA;HETATM    1  C   LIG     1       3.537  -2.101   2.235  1.00  0.00           C  &#xA;HETATM    2  C   LIG     1       2.812  -4.346   1.266  1.00  0.00           C  &#xA;HETATM    3  C   LIG     1       2.262  -2.902  -2.712  1.00  0.00           C  &#xA;HETATM    4  C   LIG     1       0.447  -1.602  -3.663  1.00  0.00           C  &#xA;HETATM    5  C   LIG     1       0.405  -0.869  -2.474  1.00  0.00           C  &#xA;HETATM    6  C   LIG     1       1.522  -3.533  -5.065  1.00  0.00           C  &#xA;HETATM    7  C   LIG     1       0.305  -3.796  -5.950  1.00  0.00           C  &#xA;HETATM    8  C   LIG     1      -5.920  -1.833 -12.368  1.00  0.00           C  &#xA;HETATM    9  C   LIG     1      -0.581  -4.848  -5.684  1.00  0.00           C  &#xA;HETATM   10  C   LIG     1       2.908  -1.583   0.938  1.00  0.00           C  &#xA;HETATM   11  C   LIG     1       3.139  -2.484  -0.313  1.00  0.00           C  &#xA;HETATM   12  C   LIG     1       3.714  -3.615   2.273  1.00  0.00           C  &#xA;HETATM   13  C   LIG     1       2.202  -2.172  -1.496  1.00  0.00           C  &#xA;HETATM   14  C   LIG     1       1.247  -1.156  -1.414  1.00  0.00           C  &#xA;HETATM   15  C   LIG     1       1.366  -2.666  -3.802  1.00  0.00           C  &#xA;HETATM   16  C   LIG     1       1.091  -5.111   3.310  1.00  0.00           C  &#xA;HETATM   17  C   LIG     1      -0.531  -1.162  -4.707  1.00  0.00           C  &#xA;HETATM   18  C   LIG     1      -0.009  -2.939  -7.020  1.00  0.00           C  &#xA;HETATM   19  C   LIG     1      -2.131  -4.074  -7.418  1.00  0.00           C  &#xA;HETATM   20  C   LIG     1      -3.479  -4.118  -8.115  1.00  0.00           C  &#xA;HETATM   21  C   LIG     1      -3.492  -4.837  -9.473  1.00  0.00           C  &#xA;HETATM   22  C   LIG     1      -1.224  -3.051  -7.715  1.00  0.00           C  &#xA;HETATM   23  C   LIG     1      -1.778  -4.991  -6.415  1.00  0.00           C  &#xA;HETATM   24  C   LIG     1      -4.883  -4.859 -10.129  1.00  0.00           C  &#xA;HETATM   25  C   LIG     1      -5.503  -3.491 -10.484  1.00  0.00           C  &#xA;HETATM   26  C   LIG     1      -7.501  -1.753 -12.406  1.00  0.00           C  &#xA;HETATM   27  C   LIG     1      -5.427  -1.719 -13.834  1.00  0.00           C  &#xA;HETATM   28  C   LIG     1      -5.340  -0.600 -11.644  1.00  0.00           C  &#xA;HETATM   29  N   LIG     1      -5.408  -3.127 -11.821  1.00  0.00           N  &#xA;HETATM   30  N   LIG     1      -8.129  -2.821 -13.038  1.00  0.00           N  &#xA;HETATM   31  O   LIG     1       3.072  -3.897  -0.073  1.00  0.00           O  &#xA;HETATM   32  O   LIG     1       3.421  -0.244   0.755  1.00  0.00           O  &#xA;HETATM   33  O   LIG     1       2.698  -1.683   3.334  1.00  0.00           O  &#xA;HETATM   34  O   LIG     1       5.094  -3.922   1.972  1.00  0.00           O  &#xA;HETATM   35  O   LIG     1      -6.061  -2.821  -9.650  1.00  0.00           O  &#xA;HETATM   36  O   LIG     1      -8.120  -0.753 -12.076  1.00  0.00           O  &#xA;HETATM   37  S   LIG     1       1.026  -4.388   1.646  1.00  0.00           S  &#xA;HETATM   38  C   LIG     1      -9.774  -4.156 -16.922  1.00  0.00           C  &#xA;HETATM   39  C   LIG     1      -9.839  -1.879 -16.189  1.00  0.00           C  &#xA;HETATM   40  N   LIG     1      -9.383  -3.231 -15.850  1.00  0.00           N  &#xA;HETATM   41  C   LIG     1      -9.964  -3.665 -14.556  1.00  0.00           C  &#xA;HETATM   42  C   LIG     1      -9.568  -2.827 -13.310  1.00  0.00           C  &#xA;CONECT    1   10   12   33                                            &#xA;CONECT    2   31   12   37                                            &#xA;CONECT    3   15   13                                                 &#xA;CONECT    4   17   15    5                                            &#xA;CONECT    5    4   14                                                 &#xA;CONECT    6    7   15                                                 &#xA;CONECT    7    6    9   18                                            &#xA;CONECT    8   28   29   27   26                                       &#xA;CONECT    8                                                           &#xA;CONECT    9    7   23                                                 &#xA;CONECT   10   11    1   32                                            &#xA;CONECT   11   13   10   31                                            &#xA;CONECT   12    1    2   34                                            &#xA;CONECT   13    3   14   11                                            &#xA;CONECT   14    5   13                                                 &#xA;CONECT   15    6    4    3                                            &#xA;CONECT   16   37                                                      &#xA;CONECT   17    4                                                      &#xA;CONECT   18    7   22                                                 &#xA;CONECT   19   23   22   20                                            &#xA;CONECT   20   19   21                                                 &#xA;CONECT   21   20   24                                                 &#xA;CONECT   22   18   19                                                 &#xA;CONECT   23    9   19                                                 &#xA;CONECT   24   21   25                                                 &#xA;CONECT   25   35   24   29                                            &#xA;CONECT   26    8   30   36                                            &#xA;CONECT   27    8                                                      &#xA;CONECT   28    8                                                      &#xA;CONECT   29   25    8                                                 &#xA;CONECT   30   26   42                                                 &#xA;CONECT   31   11    2                                                 &#xA;CONECT   32   10                                                      &#xA;CONECT   33    1                                                      &#xA;CONECT   34   12                                                      &#xA;CONECT   35   25                                                      &#xA;CONECT   36   26                                                      &#xA;CONECT   37    2   16                                                 &#xA;CONECT   38   40                                                      &#xA;CONECT   39   40                                                      &#xA;CONECT   40   38   39   41                                            &#xA;CONECT   41   40   42                                                 &#xA;CONECT   42   41   30                                                 &#xA;MASTER        0    0    0    0    0    0    0    0   42    0   42    0&#xA;END&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;picture here :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TaPtS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TaPtS.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;more about the compound at:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://ligand-expo.rcsb.org/pyapps/ldHandler.py?formid=cc-index-search&amp;amp;target=1YI&amp;amp;operation=ccid&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB Chemical Component 1YI&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Last link doesnt work anymore :&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Ligand Expo&#xA;Ligand Expo is down from January 1-31, 2023. Requests to this service will return a 410 http error code (“Gone”). Please help us better understand which Ligand Expo features are utilized and should be integrated with RCSB.org by taking this survey.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Users can access CCD data in mmCIF format at &lt;a href=&quot;https://files.wwpdb.org/pub/pdb/refdata/chem_comp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://files.wwpdb.org/pub/pdb/refdata/chem_comp/&lt;/a&gt;. For other data download during server downtime, please contact chenghua.shao@rcsb.org.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Can use : &lt;a href=&quot;https://www.ebi.ac.uk/pdbe-srv/pdbechem/chemicalCompound/show/1YI&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/pdbe-srv/pdbechem/chemicalCompound/show/1YI&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2023-01-12T12:24:01.597" LastActivityDate="2023-01-12T12:24:01.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20082" PostTypeId="1" CreationDate="2022-11-27T05:43:42.360" Score="0" ViewCount="39" Body="&lt;p&gt;I have an Excel sheet from the CRISPR library where I have sequences of gRNAs (with 30 nucleotides) in a column and I only need to keep the first 20 nucleotides for those gRNAs and delete the rest nucleotides. The column has 1000 rows that have 30 nucleotide gRNA sequences. How can I delete extra nucleotides using R&lt;/p&gt;&#xA;" OwnerUserId="16558" LastEditorUserId="3694" LastEditDate="2022-11-27T09:12:14.690" LastActivityDate="2022-11-28T18:54:17.443" Title="Keep first 20 nucleotides of sequences using R" Tags="|r|scrnaseq|bioconductor|crispr|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20083" PostTypeId="2" ParentId="20043" CreationDate="2022-11-28T08:07:38.090" Score="1" Body="&lt;p&gt;My question was kindly answered by Biostars user &lt;a href=&quot;https://www.biostars.org/p/9545804/#9545835&quot; rel=&quot;nofollow noreferrer&quot;&gt;Papyrus&lt;/a&gt;, who pointed out that a GC in the 5' to 3' direction is a GpC dinucleotide, not a CpG dinucleotide. GpC dinucleotides are then biologically and chemically distinct from CpG dinucleotides, due to the directionality of the DNA polymer.&lt;/p&gt;&#xA;" OwnerUserId="6520" LastActivityDate="2022-11-28T08:07:38.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20084" PostTypeId="1" CreationDate="2022-11-28T08:38:31.797" Score="1" ViewCount="135" Body="&lt;p&gt;I have a bam-file of reads and reference genome. How can I located matches, mismatches and indels in three different files with Phred score in each data of the files?&lt;/p&gt;&#xA;" OwnerUserId="16563" LastActivityDate="2022-11-28T08:52:38.723" Title="Matches, mismatches and indels" Tags="|sequence-alignment|indel|phred-scores|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20085" PostTypeId="2" ParentId="20084" CreationDate="2022-11-28T08:52:38.723" Score="4" Body="&lt;p&gt;Try one of these:&lt;/p&gt;&#xA;&lt;p&gt;GATK:&#xA;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360036194592-Getting-started-with-GATK4&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360036194592-Getting-started-with-GATK4&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;freebayes:&#xA;&lt;a href=&quot;https://bioinformaticsworkbook.org/dataAnalysis/VariantCalling/freebayes-dnaseq-workflow.html#gsc.tab=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioinformaticsworkbook.org/dataAnalysis/VariantCalling/freebayes-dnaseq-workflow.html#gsc.tab=0&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="383" LastActivityDate="2022-11-28T08:52:38.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20086" PostTypeId="2" ParentId="11354" CreationDate="2022-11-28T09:03:30.357" Score="1" Body="&lt;p&gt;Sample 2 looks comparable but Sample 1 shows a drastic reduction in unique alignments.&lt;/p&gt;&#xA;&lt;p&gt;Your strandedness param is going to dictate how your alignments are done in the sense that discordant pairs will either be detected or not.&lt;/p&gt;&#xA;&lt;p&gt;I would identify the strandedness of your library and use the appropriate respective param.&lt;/p&gt;&#xA;" OwnerUserId="383" LastEditorUserId="383" LastEditDate="2022-11-29T13:08:29.237" LastActivityDate="2022-11-29T13:08:29.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20087" PostTypeId="2" ParentId="19893" CreationDate="2022-11-28T09:46:17.040" Score="1" Body="&lt;p&gt;I used a local version of &lt;code&gt;segmasker&lt;/code&gt; &lt;strong&gt;*&lt;/strong&gt;. One way to solve this is,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -p -i -e 's/ //g' bioinfo.fa&#xA;perl -p -e 's/\t/^/g' bioinfo.fa | segmasker | perl -p -e 's/^/\t/g'  &amp;gt; bioinfo2.sm&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;Edit: I changed &lt;code&gt;,&lt;/code&gt; for &lt;code&gt;^&lt;/code&gt; because the OPs code already has &lt;code&gt;,&lt;/code&gt; and will insert an extra tab (\t). Same change in the line below&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here the input file you presented in the link is called &lt;code&gt;bioinfo.fa&lt;/code&gt;. If you need the pipe explaining let me know, but it's&#xA;fairly straight forward.&lt;/p&gt;&#xA;&lt;p&gt;If you want to preserve space characters, i.e. ' ' or &amp;quot;\s&amp;quot; - because in the code above I delete them - then (not tested):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -p -e 's/\t/^/g' bioinfo.fa | perl -p -e 's/ /_/g' | segmasker | perl -p -e 's/_/ /g' | perl -p -e 's/^/\t/g'  &amp;gt; bioinfo2.sm&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The issue the OP encounter is not unique to tabs (\t) but also occurs with spaces (\s), at least with &lt;code&gt;makebasedb&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;input&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;A0A000 IPR004839   Aminotransferase, class I/classII   PF00155 41  381&#xA;GVSAIDGMNTLVRGQKLPIFSSSGQPHSELAMQIARQASVPEEEEGGDDEEGSEFAVIFGAMGITQEEANEFMEDFERTGALERSVVFMNLADDPAVERTVTPRMVLTTAEYLAFEKDYHVLVILTDMTNYCEALREIGAAREEVPGRRGYPG&#xA;&amp;gt;A0A000 IPR010961   Tetrapyrrole biosynthesis, 5-aminolevulinic acid synthase   TIGR01821   12  391&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;output&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;A0A000 IPR004839       Aminotransferase        class_I/classII        PF00155 41      381&#xA;41 - 53&#xA;&amp;gt;A0A000 IPR010961       Tetrapyrrole_biosynthesis       _5aminolevulinic_acid_synthase  TIGR01821       12      391&#xA;41 - 53&#xA;&amp;gt;A0A000 IPR015421       Pyridoxal phosphate dependent transferase        major domainG3DSA:3.40.640.10  48      288&#xA;106 - 126&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Please don't ask about solving this via an upstream &lt;code&gt;makeblastdb&lt;/code&gt; because that gets complicated. The rate limiting step, i.e. why it is ignoring tabs is because of &lt;code&gt;makeblastdb&lt;/code&gt;. Thats a consequence of the way its written ... its a developer issue (see &lt;strong&gt;blurb&lt;/strong&gt;)&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;*&lt;/strong&gt; Local &lt;code&gt;segmasker&lt;/code&gt; installation ...&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Install &lt;code&gt;conda&lt;/code&gt; via &lt;code&gt;Anaconda&lt;/code&gt; or &lt;code&gt;miniconda&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;conda create -n blast3 python=3.11&lt;/code&gt; (creating an environment ... yep 3.11 is out (wow!, if you don't understand why &lt;strong&gt;WOW!&lt;/strong&gt; ask in chat)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;conda activate blast3&lt;/code&gt; (this is the conda environment, you can call it anything)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;conda install -c bioconda blast&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;code&gt;perl | segmasker | perl&lt;/code&gt; will now work. The Python installation is just good practice.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;blurb about NCBI's blast&lt;/strong&gt; The issue is &lt;code&gt;makeblastdb&lt;/code&gt;, what happens is that long headers are a complete pain in algorithms because they use loads of RAM and you've got to carry them through reiterative loops (convergence etc ...). The way to solve it is via recoding (thats how I've done it). Essentially, you have a mini database forcing header recoding which activates at the start of the  algorithm and reassembles the original header at the end of the algorithm. The user never sees this step, but it's really important for efficiency. Thus within the reiteration the header isn't consuming RAM, because it's an internal code, beyond base 10 to squash the overhead even further. The thing is header can become a lot bigger than the sequence data and thats not cool when you've loads and loads of data, e.g. a blast database. NCBI just haven't bothered, like if you can't see it why do it attitude. What they've said is that 50 characters is the limitation of the header for &lt;code&gt;makeblastdb&lt;/code&gt;... what NCBI is looking for is a unique id at the start of the header it then truncates everything beyond that, i.e. tabs, spaces ... anything it can grab. NCBI have carried this through into &lt;code&gt;segmasker&lt;/code&gt; it's not needed, but &lt;code&gt;makeblastdb&lt;/code&gt; and &lt;code&gt;segmasker&lt;/code&gt; would normally be used together - thats why. In the OPs case this results in duplicate headers, so the file format falls over. This isn't the OPs fault.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-11-29T01:10:10.040" LastActivityDate="2022-11-29T01:10:10.040" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20088" PostTypeId="2" ParentId="20051" CreationDate="2022-11-28T11:40:53.433" Score="2" Body="&lt;p&gt;I suspect the error is caused by a 'lack' of intersection between &amp;quot;NQLD&amp;quot;, &amp;quot;Nlab&amp;quot;, and &amp;quot;Nsyl&amp;quot;? If you remove &amp;quot;Nlab&amp;quot; from the first query the command appears to complete as expected, e.g.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(UpSetR)&#xA;orthogroups_df &amp;lt;- read.table(&amp;quot;~/Desktop/orthogroups.GeneCount.tsv&amp;quot;,&#xA;                            header=T, stringsAsFactors = F)&#xA;selected_species &amp;lt;- colnames(orthogroups_df)[2:(ncol(orthogroups_df) -1)]&#xA;selected_species&#xA;#&amp;gt;  [1] &amp;quot;Athaliana_447_Araport11.protein_primaryTranscriptOnly&amp;quot;&#xA;#&amp;gt;  [2] &amp;quot;Cann&amp;quot;                                                 &#xA;#&amp;gt;  [3] &amp;quot;NQLD&amp;quot;                                                 &#xA;#&amp;gt;  [4] &amp;quot;Natt&amp;quot;                                                 &#xA;#&amp;gt;  [5] &amp;quot;Ngla&amp;quot;                                                 &#xA;#&amp;gt;  [6] &amp;quot;Nlab&amp;quot;                                                 &#xA;#&amp;gt;  [7] &amp;quot;Nsyl&amp;quot;                                                 &#xA;#&amp;gt;  [8] &amp;quot;Ntab&amp;quot;                                                 &#xA;#&amp;gt;  [9] &amp;quot;Ntom&amp;quot;                                                 &#xA;#&amp;gt; [10] &amp;quot;Slyc&amp;quot;                                                 &#xA;#&amp;gt; [11] &amp;quot;Stub&amp;quot;                                                 &#xA;#&amp;gt; [12] &amp;quot;Vvin&amp;quot;&#xA;head(orthogroups_df)&#xA;#&amp;gt;   Orthogroup Athaliana_447_Araport11.protein_primaryTranscriptOnly Cann NQLD&#xA;#&amp;gt; 1  OG0000000                                                     0    0  965&#xA;#&amp;gt; 2  OG0000001                                                     0    1    3&#xA;#&amp;gt; 3  OG0000002                                                     0    1  313&#xA;#&amp;gt; 4  OG0000003                                                     0   93   15&#xA;#&amp;gt; 5  OG0000004                                                     1   42    2&#xA;#&amp;gt; 6  OG0000005                                                     0    2   61&#xA;#&amp;gt;   Natt Ngla Nlab Nsyl Ntab Ntom Slyc Stub Vvin Total&#xA;#&amp;gt; 1    0    0    3    0    0    0    0    0    0   968&#xA;#&amp;gt; 2    0    0  448    0    0    0    0    0    0   452&#xA;#&amp;gt; 3    0    0  120    1    0    1    0    0    0   436&#xA;#&amp;gt; 4   21   46   16   33   63   36   25   39   26   413&#xA;#&amp;gt; 5   34  109    6    8  154   11    9    4    0   380&#xA;#&amp;gt; 6    1   34   44   91   70   43   20    1    0   367&#xA;ncol(orthogroups_df)&#xA;#&amp;gt; [1] 14&#xA;orthogroups_df[orthogroups_df &amp;gt; 0] &amp;lt;- 1&#xA;&#xA;upset(orthogroups_df,&#xA;      nsets = ncol(orthogroups_df),&#xA;      sets = rev(c(selected_species)),&#xA;      queries = list(list(query = intersects,&#xA;                          params = list(&amp;quot;NQLD&amp;quot;, &amp;quot;Nsyl&amp;quot;),&#xA;                          color = &amp;quot;#238c45&amp;quot;, active = T),&#xA;                     list(query = intersects,&#xA;                          params = list(&amp;quot;NQLD&amp;quot;, &amp;quot;Nlab&amp;quot;),&#xA;                          color = &amp;quot;#ffd977&amp;quot;, active = T)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/sUrHL.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;sup&gt;Created on 2022-11-28 with &lt;a href=&quot;https://reprex.tidyverse.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;reprex v2.0.2&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can see in the plot that &amp;quot;NQLD&amp;quot;, &amp;quot;Nlab&amp;quot;, and &amp;quot;Nsyl&amp;quot; don't appear to have any values in common, i.e. the 'dots' corresponding to those variables aren't joined. So, to fix the error you need to select variables that intersect in your queries.&lt;/p&gt;&#xA;" OwnerUserId="9098" LastActivityDate="2022-11-28T11:40:53.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20089" PostTypeId="1" AcceptedAnswerId="20109" CreationDate="2022-11-28T12:42:16.203" Score="3" ViewCount="1177" Body="&lt;p&gt;I have performed a Seurat PCA via &lt;a href=&quot;https://satijalab.org/seurat/reference/dimplot&quot; rel=&quot;nofollow noreferrer&quot;&gt;Dimplot&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;How do I extend the x axis?&lt;/p&gt;&#xA;&lt;p&gt;As you can see in my figure the double x axes overlap. &lt;a href=&quot;https://i.stack.imgur.com/Jf7Ym.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Jf7Ym.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16087" LastEditorUserId="3848" LastEditDate="2022-11-28T12:51:58.973" LastActivityDate="2022-12-01T02:44:12.110" Title="How to extend the x axis in Dimplot Seurat" Tags="|seurat|single-cell|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20091" PostTypeId="2" ParentId="17868" CreationDate="2022-11-28T16:25:43.773" Score="2" Body="&lt;p&gt;There is now an Open Source online tool that does the conversion, &lt;a href=&quot;https://codon2nucleotide.theo.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Codon2Nucleotide&lt;/a&gt; by Theo Sanderson.&lt;/p&gt;&#xA;" OwnerUserId="14239" LastActivityDate="2022-11-28T16:25:43.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20092" PostTypeId="1" CreationDate="2022-11-28T17:56:15.260" Score="1" ViewCount="34" Body="&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE117872&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE117872&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm teaching myself sequence analysis.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;This is the count data:&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/AT8s4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AT8s4.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;and then this is, what I believe to be, the coldata:&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/jNH25.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jNH25.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;All I want to know is if I can perform Differential Gene Expression analysis on this using just the &lt;code&gt;drug_status&lt;/code&gt; column since it seems like the second dataset already has results to it.&lt;/p&gt;&#xA;&lt;p&gt;What would be the reference if listings are either Sensitive, Resistant, or Holiday?&lt;/p&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="16549" LastEditorUserId="16549" LastEditDate="2022-11-28T18:06:30.510" LastActivityDate="2023-01-24T16:10:04.287" Title="Could someone help me understand if all sample within this are control group?" Tags="|r|sequence-analysis|gene-expression|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20093" PostTypeId="2" ParentId="20082" CreationDate="2022-11-28T18:54:17.443" Score="2" Body="&lt;p&gt;You will need to &amp;quot;read&amp;quot; your data. There are packages that can read Excel files directly but I would save the data as csv first and then use &lt;code&gt;read.csv()&lt;/code&gt;. The following would create a new column with your desired output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data_frame&lt;span class=&quot;math-container&quot;&gt;$new_col &amp;lt;- substring(data_frame$&lt;/span&gt;col_with_seqs, 1, 20)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alternatively you can just use Excel's &lt;code&gt;LEFT()&lt;/code&gt; since your data is already in Excel:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;=LEFT(A1, 20)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2022-11-28T18:54:17.443" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20094" PostTypeId="1" CreationDate="2022-11-28T22:39:00.813" Score="2" ViewCount="59" Body="&lt;p&gt;I want to perform Differential Gene Expression Analysis. I recently completed RNA Sequence Analysis using the Seraut Tutorial up to &lt;em&gt;this&lt;/em&gt; point: &lt;a href=&quot;https://satijalab.org/seurat/articles/pbmc3k_tutorial.html#finding-differentially-expressed-features-cluster-biomarkers-&quot; rel=&quot;nofollow noreferrer&quot;&gt;Finding Differentially Expressed Features (Cluster Biomarkers)&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My project can be found at &lt;em&gt;this&lt;/em&gt; &lt;a href=&quot;https://rpubs.com/ap3340/SingleCellHumanPBMC&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rpubs Link&lt;/a&gt; which contains the link to the dataset on the 10x genomics website.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My question is... is the following table derived from performing the steps found in the first link I provided?&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/sr9p0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sr9p0.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;If so, once I derive that table ( ID, Name, L2FC, p-value, etc.), does that mean I can perform Differential Gene Expression Analysis?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am a complete newbie and am teaching myself all of this. Thanks, y'all.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE:&lt;/strong&gt;&#xA;Yes, it did provide me that table. With this table, am I able to perform gene expression?&lt;/p&gt;&#xA;" OwnerUserId="16549" LastEditorUserId="16549" LastEditDate="2022-11-28T23:13:15.050" LastActivityDate="2022-11-29T02:28:41.133" Title="What are the conditions to perform Gene Expression Analysis on data you've performed RNA Sequence Analysis on?" Tags="|r|rna-seq|gene-expression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20096" PostTypeId="1" AcceptedAnswerId="20097" CreationDate="2022-11-29T00:04:16.173" Score="4" ViewCount="1084" Body="&lt;p&gt;When trying to run epi2me-labs/wf-artic, I get the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;❯ nextflow run epi2me-labs/wf-artic \&#xA; -r v0.3.18 --fastq  ~/Downloads/barcode95.fastq.gz \&#xA; --scheme_version Midnight-ONT/V3 &#xA;&#xA;N E X T F L O W  ~  version 22.10.1&#xA;Launching `https://github.com/epi2me-labs/wf-artic` [ecstatic_marconi] DSL2 - revision: a60a1e1e73 [v0.3.18]&#xA;Trace file already exists: /Users/corneliusromer/Downloads/BJGMC_ONT_Run71/output/execution/trace.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It seems that nextflow doesn't like to rerun after it failed. Is there a convenient way to force nextflow to rerun, in a similar way to &lt;code&gt;snakemake --force-rerun&lt;/code&gt;? Or do I need to manually delete the trace file every time? That works, but it's tedious.&lt;/p&gt;&#xA;" OwnerUserId="12272" LastActivityDate="2022-11-29T02:34:31.107" Title="How to solve Nextflow error: &quot;Trace file already exists&quot;?" Tags="|assembly|sars-cov-2|nextflow|workflow-management|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20097" PostTypeId="2" ParentId="20096" CreationDate="2022-11-29T02:12:10.603" Score="5" Body="&lt;p&gt;Rotation of report files (trace, execution, timeline etc) was removed recently (since version &lt;a href=&quot;https://github.com/nextflow-io/nextflow/releases/tag/v22.10.0&quot; rel=&quot;noreferrer&quot;&gt;22.10.0&lt;/a&gt; in &lt;a href=&quot;https://github.com/nextflow-io/nextflow/commit/a762ed59c1af255183da8a2d26907206964eade0&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;a762ed5&lt;/code&gt;&lt;/a&gt;). To overwrite the existing trace file you will need to add the following to your Nextflow &lt;a href=&quot;https://www.nextflow.io/docs/latest/config.html&quot; rel=&quot;noreferrer&quot;&gt;config&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;trace.overwrite = true&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;See: &lt;a href=&quot;https://github.com/nextflow-io/nextflow/issues/3317&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/nextflow-io/nextflow/issues/3317&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2022-11-29T02:34:31.107" LastActivityDate="2022-11-29T02:34:31.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20098" PostTypeId="2" ParentId="20094" CreationDate="2022-11-29T02:28:41.133" Score="4" Body="&lt;p&gt;The table pictured here &lt;em&gt;is&lt;/em&gt; the result of a differential gene expression (DGE) test. In the table, each row is a gene (labeled with the gene ID and name) and each column (besides the first two) are the results of DGE tests. Each cluster has two sub-columns: L2FC (log2 fold-change) which tells you the average relative expression changes between cells of the cluster vs. others; and p-value, which is how significant the change is.&lt;/p&gt;&#xA;&lt;p&gt;You can make this table yourself by performing DGE tests for all clusters. You've already done this yourself in your Rpubs script! To make it easier though, so you don't have to manually test each cluster, use the &lt;a href=&quot;https://www.rdocumentation.org/packages/Seurat/versions/1.4.0/topics/FindAllMarkers&quot; rel=&quot;nofollow noreferrer&quot;&gt;FindAllMarkers&lt;/a&gt; function. It will compare each cluster to all other clusters and report the L2FCs and p-values.&lt;/p&gt;&#xA;&lt;p&gt;However, I recommend using the p_val_adj (adjusted p-value) instead of the p-value. This is due to a problem called multiple testing: you are performing many, many tests (each gene in each cluster is a separate statistical test), and we expect 5% of differentially expressed genes (DEGs) to be DE by random chance. Over something like 100,000 tests, that means you'd expect 5,000 of those to be false positives (which is huge). So p-value is adjusted to account for multiple tests.&lt;/p&gt;&#xA;&lt;p&gt;Once you filter the DEG table (p-adj&amp;lt;0.05), you can do other downstream analyses to describe what has changed between the conditions. You already have a good start in your Rpub script with the violin plots. Cluster 6 is clearly distinct in terms of those genes' expression levels. You could try to figure out what's special with that cluster. Good luck!&lt;/p&gt;&#xA;" OwnerUserId="16565" LastActivityDate="2022-11-29T02:28:41.133" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20100" PostTypeId="1" CreationDate="2022-11-29T22:17:28.143" Score="3" ViewCount="50" Body="&lt;p&gt;Is there a way to submit data to ucsc genome browser for result/track display, on the fly, as a local bedgraph file with example contents like below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;browser position chr19:49302001-49304701&#xA;browser hide all&#xA;browser pack refGene encodeRegions&#xA;browser full altGraph&#xA;#   300 base wide bar graph, autoScale is on by default == graphing&#xA;#   limits will dynamically change to always show full range of data&#xA;#   in viewing window, priority = 20 positions this as the second graph&#xA;#   Note, zero-relative, half-open coordinate system in use for bedGraph format&#xA;track type=bedGraph name=&amp;quot;BedGraph Format&amp;quot; description=&amp;quot;BedGraph format&amp;quot; visibility=full color=200,100,0 altColor=0,100,200 priority=20&#xA;chr19 49302000 49302300 -1.0&#xA;chr19 49302300 49302600 -0.75&#xA;chr19 49302600 49302900 -0.50&#xA;chr19 49302900 49303200 -0.25&#xA;chr19 49303200 49303500 0.0&#xA;chr19 49303500 49303800 0.25&#xA;chr19 49303800 49304100 0.50&#xA;chr19 49304100 49304400 0.75&#xA;chr19 49304400 49304700 1.00 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Normally, you can input the data in the data box like below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PJyns.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PJyns.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and submit - and display results after another [GO] button that's pushed manually on the following web page to produce the output below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/AWl01.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AWl01.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am hoping there is a command line option for this where it takes the data input above as a local file name and generates the output that normally one can generate with a few manual clicks (including pasting of the data text into the window or upload of the data file) on the ucsc web page.&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="15741" LastEditorUserId="298" LastEditDate="2022-11-30T10:06:02.253" LastActivityDate="2022-11-30T10:06:02.253" Title="Command line based interaction with the UCSC genome browser, automated data upload and track generation" Tags="|ucsc|genome-browser|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20101" PostTypeId="1" AcceptedAnswerId="20102" CreationDate="2022-11-29T23:15:36.957" Score="2" ViewCount="103" Body="&lt;p&gt;I have a question related with the most popular frameworks for network construction: &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-9-559&quot; rel=&quot;nofollow noreferrer&quot;&gt;WGCNA&lt;/a&gt; for weighted networks and &lt;a href=&quot;https://igraph.org/r/&quot; rel=&quot;nofollow noreferrer&quot;&gt;igraph&lt;/a&gt; package for unweighted networks.&lt;/p&gt;&#xA;&lt;p&gt;What are the advantages or disadvantages of applying these approaches and when it is recommended to use one over another.&lt;/p&gt;&#xA;" OwnerUserId="15698" LastEditorUserId="3694" LastEditDate="2022-11-30T19:47:16.087" LastActivityDate="2022-11-30T19:47:16.087" Title="Coexpression network analysis" Tags="|r|networks|wgcna|coexpression|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20102" PostTypeId="2" ParentId="20101" CreationDate="2022-11-29T23:44:53.483" Score="3" Body="&lt;p&gt;&lt;code&gt;igraph&lt;/code&gt; is a serious general network theory framework for any given data science application. It permits full directionality and describes vertices (nodes), edges (branches)  and does cool stuff like De Bruijn graphs and Laplacian graphs. Understanding graph theory would be helpful and will be a steep learning curve. It's a serious data science package.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;WGCNA&lt;/code&gt; is a specific ready made RNA expression package which appears to be network theory and is undirectional, i.e. no directionality which is fine for RNA expression. It will have far less overall power but if you've a load of RNA-seq data and want 'plug and play' type stuff WGCNA is the only way. It's a correlation network, based on &lt;em&gt;r&lt;/em&gt;&lt;sup&gt;2&lt;/sup&gt; (Pearson's correlation), to assess co-expression. I would assume the weighting is geared towards identifying co-expression. &lt;em&gt;There is an error in Fig. 2 legend&lt;/em&gt; BTW Langfelder &amp;amp; Horvath (2008): the legend states r^2 what they mean is &lt;em&gt;r&lt;/em&gt;&lt;sup&gt;2&lt;/sup&gt;. I guess they messed up on their LaTeX.&lt;/p&gt;&#xA;&lt;p&gt;If this is a one of coexpression analysis &lt;code&gt;WGCNA&lt;/code&gt;. If this is a big project, graph theory is central to that effort and you're looking towards developing a data science tool set then its &lt;code&gt;igraph&lt;/code&gt;. There are other graph theory packages BTW.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;In case my answer wasn't clear &lt;code&gt;igraph&lt;/code&gt; will do weighted edges via &lt;code&gt;is_weight&lt;/code&gt;. The information is &lt;a href=&quot;https://igraph.org/r/doc/is_weighted.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, basically it will do anything.&lt;/p&gt;&#xA;&lt;p&gt;If your question is &amp;quot;what does weighting contribute to coexpression analysis&amp;quot;, that is a good question. I know graph theory, I dunno the biology of gene expression versus the maths. What &lt;code&gt;WGCNA&lt;/code&gt; is doing is statistically is simple if weighting is excluded ... my reasoning is it must be pivotal to its success.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;To address the OPs comments.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Statistical power&lt;/em&gt; In the pure sense, I was incorrect to use this term because technically 'lack of power' is the inability to reject a hypothesis when it is false. For example, parametric statistics are more powerful than non-parametric statistics for this reason.&lt;/p&gt;&#xA;&lt;p&gt;What I meant was two things:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;In graph theory you can do anything within the &amp;quot;network&amp;quot;, what is called a &amp;quot;graph&amp;quot;, e.g. find the centre of the graph, minimal distance across the &amp;quot;graph&amp;quot;, cyclisation, anything you can think of.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;1a. &lt;code&gt;WGCNA&lt;/code&gt; is fixed it only does what is described in Langfelder &amp;amp; Horvath (2008). Nothing can be done outside that without rewriting the underlying code and it would be better to use &lt;code&gt;igraph&lt;/code&gt; instead if that was the objective in my opinion.&lt;/p&gt;&#xA;&lt;p&gt;Thus &lt;code&gt;igraph&lt;/code&gt; is more &amp;quot;powerful&amp;quot; because it will do a lot more stuff.&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;My concern of &lt;code&gt;WGCNA&lt;/code&gt; is its over-reliance on &lt;em&gt;r&lt;/em&gt;&lt;sup&gt;2&lt;/sup&gt;. This is a parametric statistic and may not always be applicable but from my reading of the paper (albeit briefly) it was entirely dependent on that. To centre everything on &lt;em&gt;r&lt;/em&gt;&lt;sup&gt;2&lt;/sup&gt;, personally  I would at a minimum use &lt;em&gt;r&lt;/em&gt; in addition (Spearman's correlation). This is extremely complex data and there is a risk of over-simplification. &lt;em&gt;r&lt;/em&gt;  and &lt;em&gt;r&lt;/em&gt;&lt;sup&gt;2&lt;/sup&gt; can certainly give different results.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The answer to the question however it is not what I think but everyone else thinks and Langfelder &amp;amp; Horvath (2008) has almost 10000 citations. I am not aware that the calculation has been overturned, e.g. DSeq2 has been overturned and demonstrated to be inappropriate. So I would assume it must be capturing the overall picture, so the positives outweigh potential negatives.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-11-30T14:46:49.147" LastActivityDate="2022-11-30T14:46:49.147" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20103" PostTypeId="2" ParentId="20089" CreationDate="2022-11-30T11:33:08.760" Score="0" Body="&lt;p&gt;As a direct answer to your question, restrict the output to producing a single column:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Dimplot(..., ncol=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But this doesn't seem like a useful answer. Please provide more context to your question if you want more help than this.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-11-30T11:33:08.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20105" PostTypeId="1" CreationDate="2022-11-30T21:24:26.617" Score="2" ViewCount="493" Body="&lt;p&gt;I was wondering if somebody here has experience with installing scvi-tools on a Mac with M1 chip. I tried the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install scvi-tools -c conda-forge&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But got the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Collecting package metadata (current_repodata.json): failed&#xA;CondaSSLError: OpenSSL appears to be unavailable on this machine. OpenSSL is required to&#xA;download and install packages.&#xA;Exception: HTTPSConnectionPool(host='conda.anaconda.org', port=443): Max retries exceeded with url: /conda-forge/osx-64/current_repodata.json (Caused by SSLError(&amp;quot;Can't connect to HTTPS URL because the SSL module is not available.&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then I tried to install OpenSSL with following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c anaconda openssl &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and I got the exact same error as the conda install scvi-tools -c conda-forge command.&lt;/p&gt;&#xA;&lt;p&gt;I do not understand why collection of metadata is failing for openssl. Any inputs would be highly helpful.&#xA;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="4307" LastActivityDate="2022-11-30T21:55:59.440" Title="Struggling to install scvi-tools on Mac M1" Tags="|software-installation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20106" PostTypeId="2" ParentId="20105" CreationDate="2022-11-30T21:55:59.440" Score="2" Body="&lt;p&gt;Did you try:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pip install scvi-tools&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;?&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2022-11-30T21:55:59.440" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20107" PostTypeId="2" ParentId="20057" CreationDate="2022-11-30T23:17:19.957" Score="1" Body="&lt;p&gt;The genotypes &lt;span class=&quot;math-container&quot;&gt;$\{0,1,2\}$&lt;/span&gt; are mappings to the genotypes AA (homref), AB (heterozygous), BB (homalt) (from &lt;a href=&quot;https://static-content.springer.com/esm/art%3A10.1038%2Fnbt.4042/MediaObjects/41587_2018_BFnbt4042_MOESM6_ESM.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;supplementary&lt;/a&gt; resources). This is also mapped to the number of alterations (e.g., 0 -&amp;gt; 0 alterations -&amp;gt; homref).&lt;/p&gt;&#xA;" OwnerUserId="1771" LastEditorUserId="1771" LastEditDate="2022-11-30T23:39:32.950" LastActivityDate="2022-11-30T23:39:32.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20109" PostTypeId="2" ParentId="20089" CreationDate="2022-12-01T02:44:12.110" Score="1" Body="&lt;p&gt;Looking at the source code for Seurat's &lt;code&gt;DimPlot()&lt;/code&gt; it's based on ggplot2 graphics (&lt;a href=&quot;https://github.com/satijalab/seurat/blob/HEAD/R/visualization.R#L794&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/satijalab/seurat/blob/HEAD/R/visualization.R#L794&lt;/a&gt;), so you can manipulate the dimplot object and save it using ggplot2 commands. Using &lt;code&gt;ggsave()&lt;/code&gt; you can specify the figure dimensions; if you increase the width of the final figure it should 'fix' your label overlap issue, i.e.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;#BiocManager::install(&amp;quot;Seurat&amp;quot;)&#xA;library(Seurat)&#xA;#&amp;gt; Attaching SeuratObject&#xA;library(ggplot2) # for plotting&#xA;library(cowplot) # for the theme used by Seurat&#xA;&#xA;# example data&#xA;data(&amp;quot;pbmc_small&amp;quot;)&#xA;p &amp;lt;- DimPlot(object = pbmc_small, split.by = &amp;quot;ident&amp;quot;)&#xA;&#xA;# example dimplot&#xA;p&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/P5FmK.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# increase font size to make the labels overlap&#xA;p +&#xA;  theme_cowplot(font_size = 28) +&#xA;  theme(strip.background = element_blank())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/R0m1g.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# save the 'overlapping label' figure and specify a relatively large width to 'space out' the three facets&#xA;ggsave(&amp;quot;dimplot_test.png&amp;quot;, width = 40, height = 10, units = &amp;quot;cm&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/88ZWH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/88ZWH.png&quot; alt=&quot;final.png&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;You could also decrease the font size and keep the original dimensions to fix the problem, or alter the breaks on the x axis, e.g.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# reduce the size of the x axis text labels&#xA;p +&#xA;  theme_cowplot() +&#xA;  theme(strip.background = element_blank(),&#xA;        axis.text.x = element_text(size = 8))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/ojFaU.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# change the number of breaks&#xA;p +&#xA;  theme_cowplot() +&#xA;  theme(strip.background = element_blank()) +&#xA;  scale_x_continuous(breaks = scales::pretty_breaks(n = 2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/sEMYl.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# Specify the breaks you want to use&#xA;p +&#xA;  theme_cowplot() +&#xA;  theme(strip.background = element_blank()) +&#xA;  scale_x_continuous(breaks = c(-20, 0, 20, 40))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/Yfdwy.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;sup&gt;Created on 2022-12-01 with &lt;a href=&quot;https://reprex.tidyverse.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;reprex v2.0.2&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="9098" LastActivityDate="2022-12-01T02:44:12.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20110" PostTypeId="2" ParentId="19351" CreationDate="2022-12-01T15:14:59.720" Score="1" Body="&lt;p&gt;You can add a color palette to the plot.&lt;/p&gt;&#xA;&lt;p&gt;e.g.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;my_palette &amp;lt;- colorRampPalette(c(&amp;quot;red&amp;quot;, &amp;quot;orange&amp;quot;, &amp;quot;yellow&amp;quot;,&amp;quot;white&amp;quot;))(n = 100);&#xA;TOMplot(plotDiss, selectTree, selectColors, main =&amp;quot;Network heatmap plot, selected genes&amp;quot;, col=my_palette)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15700" LastEditorUserId="650" LastEditDate="2022-12-01T18:57:04.057" LastActivityDate="2022-12-01T18:57:04.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20111" PostTypeId="1" CreationDate="2022-12-02T01:53:48.913" Score="3" ViewCount="223" Body="&lt;p&gt;I have tried to use UpsetR to visualize the input file which can be found &lt;a href=&quot;https://cloudstor.aarnet.edu.au/plus/s/OupXC8rF2bDFtRV&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/P2RAE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/P2RAE.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(&amp;quot;UpSetR&amp;quot;)&#xA;orthogroups_df&amp;lt;- read.table(&amp;quot;orthogroups.GeneCount.tsv&amp;quot;,  header=T, stringsAsFactors = F)&#xA;&#xA;#All species&#xA;selected_species &amp;lt;- colnames(orthogroups_df)[2:(ncol(orthogroups_df) -1)] &#xA;selected_species&#xA; [1] &amp;quot;Atha&amp;quot; &amp;quot;Cann&amp;quot; &amp;quot;NQLD&amp;quot; &amp;quot;Natt&amp;quot; &amp;quot;Ngla&amp;quot; &amp;quot;Nlab&amp;quot; &amp;quot;Nsyl&amp;quot; &amp;quot;Ntab&amp;quot; &amp;quot;Ntom&amp;quot; &amp;quot;Slyc&amp;quot; &amp;quot;Stub&amp;quot; &amp;quot;Vvin&amp;quot;&#xA;&#xA;head(orthogroups_df)&#xA;  Orthogroup Atha Cann NQLD Natt Ngla Nlab Nsyl Ntab Ntom Slyc Stub Vvin Total&#xA;1  OG0000000    0    0  965    0    0    3    0    0    0    0    0    0   968&#xA;2  OG0000001    0    1    3    0    0  448    0    0    0    0    0    0   452&#xA;3  OG0000002    0    1  313    0    0  120    1    0    1    0    0    0   436&#xA;4  OG0000003    0   93   15   21   46   16   33   63   36   25   39   26   413&#xA;5  OG0000004    1   42    2   34  109    6    8  154   11    9    4    0   380&#xA;6  OG0000005    0    2   61    1   34   44   91   70   43   20    1    0   367&#xA;&#xA;ncol(orthogroups_df)&#xA;[1] 14&#xA;&#xA;orthogroups_df[orthogroups_df &amp;gt; 0] &amp;lt;- 1&#xA;# we only show intersections of interest&#xA;intersections=list(list(selected_species),&#xA;                   list(&amp;quot;Nlab&amp;quot;, &amp;quot;NQLD&amp;quot;, &amp;quot;Ngla&amp;quot;), &#xA;                   list(&amp;quot;Nlab&amp;quot;, &amp;quot;NQLD&amp;quot;, &amp;quot;Nsyl&amp;quot;), &#xA;                   list(&amp;quot;Nlab&amp;quot;, &amp;quot;Ngla&amp;quot;, &amp;quot;Nsyl&amp;quot;), &#xA;                   list(&amp;quot;NQLD&amp;quot;, &amp;quot;Nsyl&amp;quot;, &amp;quot;Ngla&amp;quot;))&#xA;upset(orthogroups_df, &#xA;      text.scale = c(1.4, 1.4, 1.4, 1.4, 1.4, 1.4),&#xA;      sets=rev(selected_species), &#xA;      nsets = ncol(orthogroups_df),&#xA;      #keep.order=T, &#xA;      #mb.ratio=c(0.5,0.5), &#xA;      #order.by='degree', &#xA;      #order.by='freq',&#xA;      intersections = intersections, &#xA;      sets.x.label=&amp;quot;Total nbr of orthogroups&amp;quot;, &#xA;      mainbar.y.label = &amp;quot;Nbr of orthogroups&amp;quot;, &#xA;      #empty.intersections = &amp;quot;on&amp;quot;, &#xA;      group.by = &amp;quot;intersections&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How is possible to make the bar bigger of the double digits values or is a way to rotate the plot?&lt;/p&gt;&#xA;" OwnerUserId="3329" LastEditorUserId="650" LastEditDate="2022-12-02T15:02:20.553" LastActivityDate="2022-12-04T19:47:55.867" Title="How to make bar bigger in UpsetR plot" Tags="|r|ggplot2|upsetr|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20115" PostTypeId="1" CreationDate="2022-12-04T09:20:49.983" Score="1" ViewCount="85" Body="&lt;p&gt;Could someone direct me to a resource (textbook chapter, lecture notes, online video, etc.) that demonstrates, in a mathematically or computationally rigorous way, how to use maximum likelihood estimation to generate a phylogenetic tree, given a set of aligned sequences?&lt;/p&gt;&#xA;&lt;p&gt;I've found several resources which describe the procedure for maximum likelihood estimation, such as:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;giving the formula and the parameters that need to be determined (which I already know);&lt;/li&gt;&#xA;&lt;li&gt;how to use it for aligned sequences (I already know this),&#xA;I am seeking a worked-out example.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Alternatively, if someone is willing to work out a small example, how would you use maximum likelihood estimation to generate a tree for the following three sequences?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;organism_1 ATCG&#xA;organism_2 ATCC&#xA;organism_3 ATTT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6205" LastEditorUserId="3848" LastEditDate="2022-12-04T16:07:57.410" LastActivityDate="2022-12-04T16:07:57.410" Title="Maximum likelihood estimation for tree construction" Tags="|sequence-alignment|phylogenetics|phylogeny|sequence-analysis|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20116" PostTypeId="2" ParentId="20115" CreationDate="2022-12-04T11:45:57.433" Score="1" Body="&lt;p&gt;@DataScienceNovice ... here's a rough explanation. Just to note a tree with three taxa doesn't really work as a tree - you need 4 taxa.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If we have &lt;strong&gt;tree 1&lt;/strong&gt; ((A,B),(C,D)) the probability of the tree is expressed as the summed log likelihood for each position 1, 2, 3 .... in the alignment. The probability is derived from a mutation matrix, thus certain mutations e.g. transitions &lt;code&gt;A&amp;lt;-&amp;gt;G&lt;/code&gt;; &lt;code&gt;C&amp;lt;-&amp;gt;T&lt;/code&gt; are more probable (high likelihood), e.g. transversions &lt;code&gt;A&amp;lt;-&amp;gt;C&lt;/code&gt;; &lt;code&gt;A&amp;lt;-&amp;gt;T&lt;/code&gt;; &lt;code&gt;G&amp;lt;-&amp;gt;C&lt;/code&gt;; &lt;code&gt;G&amp;lt;-&amp;gt;T&lt;/code&gt; are less probable in accordance with the matrix. This hypothetical matrix (most matrices) is scoring transitions as much more probable than transversions.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The 'total probability' expressed  log likelihood for each column, i.e. each alignment position is added together leveraging an interesting property of logs (see final sentence of this paragraph). That becomes the 'score' or 'likelihood' for that tree. Note in the example you provided you have 4 alignment positions, so 4 probabilities of the log-likelihood will be added. Note, the probabilities are actually multiplied but log-likelihoods added is the same thing.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;We will now rearrange the tree lets try &lt;strong&gt;tree 2&lt;/strong&gt; ((C,D),(B,G)), again the hypothetical matrix scores transition/transversions, thus essentially if this tree has higher transitions than the tree in point 1. that will provide a higher likelihood. The tree that maximises the transitions &lt;strong&gt;as sister groups&lt;/strong&gt; will maximise the likelihood. Equally maximising transversions as the most &lt;strong&gt;divergent groups&lt;/strong&gt; will maximise the likelihood of that tree.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Re-arrange the tree again &lt;strong&gt;tree 3&lt;/strong&gt; ... ((C,B),(D,A)) ... which tree has the highest transitions vs transversions? ...&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In this example Tree 2 has the highest likelihood. Thus that is the maximum likelihood. Simple?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In the example you have the outcome will depend on the matrix. Alignment position 3 comprises a transition, whilst position 4 comprises a transition and a transversion. Generally transversions are 1:2 to 1:4 less likely even though there are twice as many. So the example you gave the 'tree' (you can't really have a tree with 3 taxa) would be ((&lt;code&gt;organism2&lt;/code&gt;,&lt;code&gt;organism3&lt;/code&gt;),&lt;code&gt;organism1&lt;/code&gt;). This is against the total number of mutations because &lt;code&gt;organism2&lt;/code&gt; and &lt;code&gt;organism3&lt;/code&gt; has two mutations whilst &lt;code&gt;organism2&lt;/code&gt; and &lt;code&gt;organism1&lt;/code&gt; only have one, but the mutation types are different (also see point 3 below). In other words the occurrence of a transversion between &lt;code&gt;organism1&lt;/code&gt; against &lt;code&gt;organism2&lt;/code&gt;,&lt;code&gt;organism3&lt;/code&gt;, mean &lt;code&gt;organism2&lt;/code&gt; and &lt;code&gt;organism3&lt;/code&gt; are more likely &lt;strong&gt;sister groups&lt;/strong&gt; even though they comprise 2 mutations (transitions) against the single mutation between &lt;code&gt;organism1&lt;/code&gt; and &lt;code&gt;organism2&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;A good maximum likelihood program for phylogenetics is &lt;strong&gt;&lt;a href=&quot;https://cme.h-its.org/exelixis/web/software/raxml/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RAxML&lt;/a&gt;&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The reality is it's not that simple because the mutation matrix is a function of the tree, which is a function of the mutation matrix and in technical tree building the matrix is also calculated alongside the total probability of the tree via maximum likelihood. Just to be clear it can be calculated as a separate maximum likelihood calculation and then fixed for the tree calculation, but it's not a good idea because of non-linear interactions.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In addition, mutation rate (heterogeneity) is also calculated via maximum likelihood.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The final thing to note is back-mutations are also calculated and that refers to past posts on this site, which I will link if I find them.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Complexity&lt;/strong&gt; If the parameters are calculated via maximum likelihood simultaneous to the tree the parameter space becomes EXTREMELY large and the calculation is searching the tree landscape against the parameter landscape which becomes a multi-dimensional calculation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Reality&lt;/strong&gt; In reality much of a maximum likelihood calculation becomes about the search strategy because its impossible to traverse all of the parameter space so a heuristic estimate needs to be performed and the calculation can get stuck on lower plateaus of probability and fail to identify the tree maximum likelihood. This is the nuts and bolts of real world tree building.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; However if you fix the matrix and the rate, which a lot of people do, then points 1 to 5 describe the maximum likelihood tree.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-12-04T14:22:25.470" LastActivityDate="2022-12-04T14:22:25.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20117" PostTypeId="2" ParentId="17990" CreationDate="2022-12-04T18:37:56.807" Score="1" Body="&lt;p&gt;I came up with this idea. So, the information for the translation table of any particular organism can be found on its page on the Taxonomy Browser: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If I search a taxid into the website, eg 7227, the page for &lt;em&gt;Drosophila melanogaster&lt;/em&gt; will come up but the URL will remain unmodified. However, you can get a modified URL specific to a taxid by appending ?mode=Info&amp;amp;id=&lt;em&gt;your_taxid&lt;/em&gt; to the above URL. Eg for &lt;em&gt;D. melanogaster&lt;/em&gt; again, it has this unique URL for its Taxonomy browser page: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?mode=Info&amp;amp;id=7227&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?mode=Info&amp;amp;id=7227&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since you have a list of taxids, you can programmatically access Taxonomy browser pages by modifying the URL in the way I said. To programmatically get the translation tables of &lt;em&gt;D. melanogaster&lt;/em&gt; with this approach, use the urllib package in Python.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import urllib.request&#xA;url = &amp;quot;https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?mode=Info&amp;amp;id=7227&amp;quot;&#xA;weburl = urllib.request.urlopen(url)&#xA;data = weburl.read().decode(&amp;quot;utf-8&amp;quot;)  # decode because page returned in binary&#xA;&#xA;data_s = data.split('&amp;lt;')&#xA;&#xA;for d in data_s:&#xA;    if &amp;quot;Translation table&amp;quot; in d:&#xA;        print(d.split('&amp;gt;')[1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Translation table 1 (Standard)&#xA;Translation table 5 (Invertebrate Mitochondrial)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will work for one taxid. But you have a list of them. However, with this approach in mind, it's elementary to construct a Python function for getting the Translation table of all your taxids.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import urllib.request&#xA;&#xA;my_taxids = ['7227','9606','10090']&#xA;&#xA;def get_translation_table(taxid):&#xA;    url = &amp;quot;https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?mode=Info&amp;amp;id=&amp;quot; + taxid&#xA;    weburl = urllib.request.urlopen(url)&#xA;    data = weburl.read().decode(&amp;quot;utf-8&amp;quot;)&#xA;&#xA;    data_s = data.split('&amp;lt;')&#xA;&#xA;    tr_table = []&#xA;&#xA;    for d in data_s:&#xA;        if &amp;quot;Translation table&amp;quot; in d:&#xA;            tr_table += [d.split('&amp;gt;')[1]]&#xA;&#xA;    return tr_table&#xA;&#xA;your_tables = list(map(get_translation_table,my_taxids))&#xA;&#xA;print(your_tables)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[['Translation table 1 (Standard)', 'Translation table 5 (Invertebrate Mitochondrial)'], ['Translation table 1 (Standard)', 'Translation table 2 (Vertebrate Mitochondrial)'], ['Translation table 1 (Standard)', 'Translation table 2 (Vertebrate Mitochondrial)']]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And if you want the results nicely formatted into a pandas dataframe, just add:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To the top of the script, and&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;t = pd.DataFrame(your_tables,columns=[&amp;quot;Nuclear&amp;quot;,&amp;quot;Mitochondrial&amp;quot;])&#xA;print(t)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To the bottom. Output of this print (if you have too many taxids use the head method instead so not everything prints):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;                          Nuclear                                     Mitochondrial&#xA;0  Translation table 1 (Standard)  Translation table 5 (Invertebrate Mitochondrial)&#xA;1  Translation table 1 (Standard)    Translation table 2 (Vertebrate Mitochondrial)&#xA;2  Translation table 1 (Standard)    Translation table 2 (Vertebrate Mitochondrial)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16493" LastActivityDate="2022-12-04T18:37:56.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20118" PostTypeId="2" ParentId="20111" CreationDate="2022-12-04T19:47:55.867" Score="1" Body="&lt;p&gt;I'm not quite sure what you mean.  You can alter the % of height that the bottom part of the graph takes up, that will make all the vertical bar parts proportionately taller, but I don't think you should because you have so many groups to display in the bottom part.&lt;/p&gt;&#xA;&lt;p&gt;I think what you'd want to do is to alter the y-axis so that it stops being totally proportional, to compress the size of the tallest bar, so you can better see the difference in bar sizes between the other comparisons?&lt;/p&gt;&#xA;&lt;p&gt;I'm not sure you can do that.  Making the y axis logged with &amp;quot;scale.intersections&amp;quot; might help.&lt;/p&gt;&#xA;&lt;p&gt;You'd probably be best off making a second graph that omits that &amp;quot;common to all&amp;quot; intersection.  Once that is out of the picture, the difference between the others will be more visible.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2022-12-04T19:47:55.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20119" PostTypeId="1" CreationDate="2022-12-04T22:20:11.680" Score="3" ViewCount="73" Body="&lt;p&gt;I'm pretty new in bioinformatics.&lt;/p&gt;&#xA;&lt;p&gt;I need to download FASTA sequences of several genes. The list of genes I have assembled consists of 140 genes, so I'd rather do this through via code than download each gene manually from the NCBI database.&lt;/p&gt;&#xA;&lt;p&gt;The genes belong to 2 different organisms (bacteria) I have chosen, most of them are found in both organisms and some are specific to one or the other bacterium.&lt;/p&gt;&#xA;&lt;p&gt;I've tried running this code which I found here: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/books/NBK179288/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db nucleotide -query &amp;quot;aminoacyl-tRNA\ hydrolase&amp;quot; | efetch -format fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The gene of interest being aminoacyl-tRNA hydrolase here. But I don't know how to filter it for the specific organism, Neisseria meningitidis. Tried running this, without luck:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db nucleotide -query &amp;quot;aminoacyl-tRNA\ hydrolase&amp;quot; | efilter -organism [Neisseria meningitidis] | efetch -format fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Appreciate any help :)&lt;/p&gt;&#xA;" OwnerUserId="16594" LastEditorUserId="298" LastEditDate="2022-12-05T09:50:43.237" LastActivityDate="2022-12-05T21:31:48.227" Title="Downloading genes from NCBI in fasta format" Tags="|fasta|phylogenetics|data-download|sequence|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20120" PostTypeId="1" CreationDate="2022-12-05T03:46:06.280" Score="4" ViewCount="293" Body="&lt;p&gt;The code I ran was here, so nothing fancy.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;breseq -l 110 -o RifR_align -r Big_burk_assembly.fasta RifRNano_nanopore.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This was the output.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;NOW PROCESSING Read alignment to reference genome&#xA;[system] bowtie2-build -q &amp;quot;RifR_align/data/reference.fasta&amp;quot; &amp;quot;RifR_align/02_reference_alignment/reference&amp;quot;&#xA;    [system] bowtie2 -t --no-unal -p 1 -L 31 --ma 1 --mp 3 --np 0 --rdg 2,3 --rfg 2,3 --ignore-   quals --local -i S,1,0.25 --score-min L,1,0.9 -k 2000 --reorder -x &amp;quot;RifR_align/02_reference_alignment/reference&amp;quot; -U &amp;quot;RifR_align/01_sequence_conversion/RifRNano_nanopore.converted.fastq&amp;quot; -S &amp;quot;RifR_align/02_reference_alignment/RifRNano_nanopore.stage1.sam&amp;quot; --un &amp;quot;RifR_align/02_reference_alignment/RifRNano_nanopore.stage1.unmatched.fastq&amp;quot;&#xA;    Time loading reference: 00:00:00&#xA;    Time loading forward index: 00:00:00&#xA;    Time loading mirror index: 00:00:00&#xA;    Killed&#xA;    Use of uninitialized value &lt;span class=&quot;math-container&quot;&gt;$l in scalar chomp at /home/lts/miniconda3/envs/assemblers/bin/bowtie2 line 588, &amp;lt;BT&amp;gt; line 4626.&#xA;Use of uninitialized value $&lt;/span&gt;l in substitution (s///) at /home/lts/miniconda3/envs/assemblers/bin/bowtie2 line 589, &amp;lt;BT&amp;gt; line 4626.&#xA;Use of uninitialized value $l in print at /home/lts/miniconda3/envs/assemblers/bin/bowtie2 line 593, &amp;lt;BT&amp;gt; line 4626.&#xA;(ERR): bowtie2-align exited with value 137&#xA;!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!&amp;gt; FATAL ERROR &amp;lt;!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!&#xA;&#xA;Error running command:&#xA;[system] bowtie2 -t --no-unal -p 1 -L 31 --ma 1 --mp 3 --np 0 --rdg 2,3 --rfg 2,3 --ignore-quals --local -i S,1,0.25 --score-min L,1,0.9 -k 2000 --reorder -x &amp;quot;RifR_align/02_reference_alignment/reference&amp;quot; -U &amp;quot;RifR_align/01_sequence_conversion/RifRNano_nanopore.converted.fastq&amp;quot; -S &amp;quot;RifR_align/02_reference_alignment/RifRNano_nanopore.stage1.sam&amp;quot; --un &amp;quot;RifR_align/02_reference_alignment/RifRNano_nanopore.stage1.unmatched.fastq&amp;quot;&#xA;Result code: 35072&#xA;FILE: libbreseq/common.h   LINE: 1411&#xA;!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!&amp;gt; STACK TRACE &amp;lt;!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!&#xA;Backtrace with 6 stack frames.&#xA;breseq(+0x44cf4) [0x56102c3c3cf4]&#xA;breseq(+0x4a4f1) [0x56102c3c94f1]&#xA;breseq(+0x6d142) [0x56102c3ec142]&#xA;breseq(+0x36cc4) [0x56102c3b5cc4]&#xA;/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf3) [0x7f71ba61e0b3]&#xA;breseq(+0x40559) [0x56102c3bf559]&#xA;!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The programs seemed to have no trouble aligning the reference genome, however the Bowtie2 package is throwing error code 137 and terminating the program.&lt;/p&gt;&#xA;" OwnerUserId="16595" LastEditorUserId="3848" LastEditDate="2022-12-05T05:39:16.270" LastActivityDate="2023-09-14T06:01:05.150" Title="Breseq error (code 137). Any ideas?" Tags="|reference-genome|bowtie2|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20121" PostTypeId="2" ParentId="17797" CreationDate="2022-12-05T11:54:19.270" Score="2" Body="&lt;p&gt;The &lt;code&gt;concat&lt;/code&gt; command from &lt;a href=&quot;https://bioinf.shenwei.me/seqkit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqkit&lt;/a&gt; could concatenate sequences with same ID from multiple files.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqkit concat [file1] [file2] ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15583" LastActivityDate="2022-12-05T11:54:19.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20122" PostTypeId="1" CreationDate="2022-12-05T14:24:41.433" Score="2" ViewCount="76" Body="&lt;p&gt;I am a biologist, and I do not fully understand the tree topology of the experimental species.&#xA;I used four-taxon set (4 sequences) to identify the Four-cluster Likelihood-Mapping by using&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;iqtree2 -s ***.fa -lmap 2000 -n 0 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have obtained the svg file - triangle (attached - 1), however there were no data mentioned about T1, T2 and T3 unrooted tree based on my input 4  sequences  (Haur_2, Shad_6, Equa_oga2 and apa). I used single orthologous copy genes as the supermatrix for this data.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, I do not understand which topology of tree is reliable depends on the sequence data. I knew that my data set showed similar result topology from Iq2 tree output.&#xA;There is no species sequence data available for the species that I am looking for a reliable topology of those species.&#xA;If anyone knows how to understand the assigned sequence on the unrooted tree based on Four-cluster Likelihood-Mapping by using iqtree2? Could you kindly answer this question? I want to know the orange unrooted tree (image &lt;a href=&quot;https://i.stack.imgur.com/lv9pW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lv9pW.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;2) for my analyzed data. Thank you very much.&lt;/p&gt;&#xA;&lt;p&gt;output ***.iqtree&#xA;Quartet support of regions a1, a2, a3 (mainly for clustered analysis):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;     #quartets    a1 (% a1)        a2 (% a2)        a3 (% a3)    name&#xA;-----------------------------------------------------------------------------&#xA;1   2000     648     (32.40 ) 712     (35.60 ) 640     (32.00 )  Haur_2&#xA;2   2000     648     (32.40 ) 712     (35.60 ) 640     (32.00 )  Shad_6&#xA;3   2000     648     (32.40 ) 712     (35.60 ) 640     (32.00 )  Equa_oga2&#xA;4   2000     648     (32.40 ) 712     (35.60 ) 640     (32.00 )  apa&#xA;-----------------------------------------------------------------------------&#xA;2000     648     (32.40 ) 712     (35.60 ) 640     (32.00 )  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NlUpk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NlUpk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16596" LastEditorUserId="3848" LastEditDate="2022-12-06T13:56:21.193" LastActivityDate="2022-12-06T13:56:21.193" Title="How to make unrooted tree for Likelihood mapping result by using IQ2-tree?" Tags="|rna-seq|phylogenetics|phylogeny|transcriptome|orthologues|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20123" PostTypeId="1" CreationDate="2022-12-05T14:33:21.717" Score="3" ViewCount="149" Body="&lt;p&gt;Sorry if this question seems basic. I am trying to run the MAGeCK pipeline to analyze CRISPR knockout screen data produced by someone in my lab around 5 years ago. I was given the data as bam files and also have the sequencing sample statistics from when the samples were run including the number of total reads, aligned reads, and unaligned reads. I converted the bam files back to fastq files using&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -h -F 2048 filename.bam &amp;gt; tmp.bam&#xA;&#xA;bedtools bamtofastq -i tmp.bam -fq filename.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After converting the bam files back to fastq files, I ran the MAGeCK count function. I saw that none of the total reads matched between the MAGeCK count summary output file and the original sequencing statistics.&lt;/p&gt;&#xA;&lt;p&gt;More problematically, a few samples had significantly fewer total reads (like 50,000) shown on the MAGeCK count summary output files relative to the original sequencing statistics (~44 million). As a result, many of the sgRNAs are not represented at all in the sample (there are ~61,000 zero counts relative to the total 77441 sgRNAs) which is impacting my analysis.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone help me understand why I might be losing sgRNAs using MAGeCK? Is it a problem with my source files? I would appreciate any help!&lt;/p&gt;&#xA;" OwnerUserId="16597" LastEditorUserId="3848" LastEditDate="2022-12-05T16:27:42.617" LastActivityDate="2022-12-08T23:24:11.147" Title="CRISPR genome wide screen analysis using MAGeCK-- Loss of reads/sgRNA using MAGeCK count function?" Tags="|samtools|fastq|crispr|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20124" PostTypeId="2" ParentId="20122" CreationDate="2022-12-05T16:42:35.067" Score="1" Body="&lt;p&gt;You have provided two different trees. The second was the Arthropoda and labeled &amp;quot;image 2&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;The first example the explanation is that it is an exact polytomy. I know this because the values are equally split between the 3 possible alternatives for a 4-taxon tree. Thus there is no bifurcated solution to the data, any bifurcating combination is equally valid therefore it is simply expressed as all 4-taxa sharing the same node (hypothetical common ancestor). In other words there are 3 equally valid solutions.&lt;/p&gt;&#xA;&lt;p&gt;Thus, the tree is 4-taxa tree with 1-node. Thats the solution. A normal 4-taxon tree will have 2-nodes, this is a star-tree, i.e. one node, technically a polytomy.&lt;/p&gt;&#xA;&lt;p&gt;The example below concerns Arthropoda and demonstrates the top tree ((Chelicerta, Myriapoda),(Hexapoda, misc.)) is the correct tree. That 90.4 value demonstrates that. What this says is that neither myriapods or Chelicerta share a direct common ancestor with hexapods, moreover &amp;quot;misc&amp;quot; (whatever that is) shares a direct ancestor with the hexapods. &amp;quot;Image 2&amp;quot; is a classical bifurcating tree and represents only one solution.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;So to specifically answer the question. The tree for arthropoda is:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/cxdhk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cxdhk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and the other tree is,&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gvKM3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gvKM3.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I should have removed the scale bar in this example, because I didn't have the data, but not when you have genetic quantification of the distances.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-12-06T13:50:52.603" LastActivityDate="2022-12-06T13:50:52.603" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20126" PostTypeId="1" AcceptedAnswerId="20133" CreationDate="2022-12-05T18:31:39.493" Score="1" ViewCount="53" Body="&lt;p&gt;Are there any programs (preferably command line tools) for calculating molecular descriptors of DNA? I am looking for something like Chemopy or RDKit but specifically for DNA.&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="12203" LastEditorUserId="3848" LastEditDate="2023-09-02T13:19:26.900" LastActivityDate="2023-09-02T13:19:26.900" Title="DNA, molecular descriptors" Tags="|phylogenetics|phylogeny|rdkit|matrix|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20127" PostTypeId="1" AcceptedAnswerId="20130" CreationDate="2022-12-05T19:38:25.777" Score="2" ViewCount="109" Body="&lt;p&gt;I'd like to run dotPlotly from RStudio but I can't find any description/help for this. Is it at all possible?&#xA;Another side to this question: does anyone know any tool for dot plots for genome comparison apart from pafr (doesn't provide colour legend) and d-genies (we have problems with installation)? Is there a way to make dot plots for a minimap2 output with ggplot2?&lt;/p&gt;&#xA;&lt;p&gt;To specify:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I need help in running dotPlotly (preferably from Rstudio)&lt;/li&gt;&#xA;&lt;li&gt;I'd like to build dot plots for genome comparisons showing % of identity&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="14269" LastEditorUserId="14269" LastEditDate="2022-12-06T10:59:33.170" LastActivityDate="2022-12-06T10:59:33.170" Title="Can I run dotPlotly from Rstudio?" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20128" PostTypeId="2" ParentId="20119" CreationDate="2022-12-05T21:31:48.227" Score="2" Body="&lt;p&gt;If I put the species name in quotes, I get expected (by me) output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;% esearch -db nucleotide -query &amp;quot;aminoacyl-tRNA\ hydrolase&amp;quot; | efilter -organism &amp;quot;Neisseria meningitidis&amp;quot; | efetch -format fasta&#xA;# outputs:&#xA;&amp;gt;NZ_JWNU01000010.1 Neisseria meningitidis strain Nm440716 Scaffold10, whole genome shotgun sequence&#xA;GACCGCCTATTCGCTTGCCGCCGGCGGCCCCATCATGCAGGCAGGATTACACGCCTTCACACTCGTACCC&#xA;# [snip, tons of output]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that filtering down to only &lt;em&gt;N. meningitidis&lt;/em&gt; still leaves all of the different sequenced accessions of this species, such that you will still have a lot of different annotated copies of this gene. You will need to filter more specifically to reduce that.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-12-05T21:31:48.227" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20129" PostTypeId="1" CreationDate="2022-12-05T21:37:20.513" Score="1" ViewCount="70" Body="&lt;p&gt;I have a sequence of pdb files which I want to load in pymol window so that it gives an impression of a video. I am using cmd.load function in a loop and deleting the loaded file in the next step. But for some reason the window pauses until all iteration steps have been completed and giving nothing else in the end. I know that the files are loaded because without the “delete” line I can find all pdbs overlapped on one another. Any suggestions? Thanks and regards&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def trial(pathname, num_files):&#xA;    for i in range(num_files):&#xA;        filename = (pathname + str(i + 1)  +'.pdb')&#xA;        print(filename)&#xA;        # print(i)&#xA;        cmd.load(filename, (i+1))&#xA;        cmd.hide()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16600" LastEditorUserId="3848" LastEditDate="2022-12-05T23:23:59.733" LastActivityDate="2022-12-05T23:23:59.733" Title="Visualise multiple disappearing pdb files in Pymol" Tags="|python|pymol|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20130" PostTypeId="2" ParentId="20127" CreationDate="2022-12-05T21:57:01.223" Score="1" Body="&lt;p&gt;For the issue of just making a dotplot in ggplot2 from a PAF file, it is pretty trivial to do a simple one, though it is necessarily not the most beautiful.&lt;/p&gt;&#xA;&lt;p&gt;I grabbed 2 random E. coli genomes and mapped them:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;% minimap2 ~/Downloads/GCF_000157115.2_Escherichia_sp_3_2_53FAA_V2_genomic.fna.gz ~/Downloads/GCF_000308975.1_ASM30897v2_genomic.fna.gz &amp;gt; 2_ecolis.paf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then I opened Rstudio and ran the following (which is only slightly modified from what I was first shown by an old colleague, Zev Kronenberg):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ecoli_paf = read.table(&amp;quot;~/2_ecolis.paf&amp;quot;, sep=&amp;quot;\t&amp;quot;, fill=T)&#xA;ggplot() + &#xA;geom_segment(&#xA;  data=ecoli_paf, mapping=aes(&#xA;    x=ecoli_paf$V3, y=ecoli_paf$V8, xend=ecoli_paf$V4, yend=ecoli_paf$V9, color=ecoli_paf$V6&#xA;  )&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am far from a ggplot2 expert as you can tell, and there are a few issues here (reverse complement is shown opposite to what you'd expect, linear interpolation of mappings from mapped endpoints), but it makes a somewhat informative dotplot:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/h7WCR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/h7WCR.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am confident that with minimal fiddling you can make this look a lot better. I will try to revisit and beautify.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-12-05T21:57:01.223" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20131" PostTypeId="2" ParentId="4976" CreationDate="2022-12-06T07:40:04.940" Score="0" Body="&lt;p&gt;Use the NCBI Datasets command line tool to download genomes by taxon (NCBI Taxonomy ID, scientific or common name at any tax rank).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;datasets download genome taxon cyanobacterial \&#xA;  --assembly-source refseq --assembly-level complete_genome \&#xA;  --exclude-genomic-cds --exclude-gff3 --exclude-protein \&#xA;  --exclude-rna --exclude-seq --include-gbff \&#xA;  --filename genomes.zip&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15583" LastActivityDate="2022-12-06T07:40:04.940" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20132" PostTypeId="2" ParentId="19909" CreationDate="2022-12-06T10:51:25.680" Score="1" Body="&lt;p&gt;Here's a solution based on Jesse's comment. The steps are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;read in MSA with &lt;code&gt;Biostrings::readAAStringSet()&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;right-fill sequences so they are all the same length (assumes left alignment)&lt;/li&gt;&#xA;&lt;li&gt;search for motif substrings in each amino acid sequence with &lt;code&gt;gregexpr()&lt;/code&gt;. If a motif in the motif set is found, record position of center of motif.&lt;/li&gt;&#xA;&lt;li&gt;bind the results from each sequence to a single table&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;find_motifs &amp;lt;- function(msa, motifs) {&#xA;&#xA;  require(tibble)&#xA;  require(Biostrings)&#xA;&#xA;  # read in fasta as Biostrings object&#xA;  seqs &amp;lt;- Biostrings::readAAStringSet(filepath = msa, format = &amp;quot;fasta&amp;quot;)&#xA;  maxseqlen &amp;lt;- max(width(seqs))&#xA;&#xA;  # initiate results tibble&#xA;  results &amp;lt;- tibble::tibble(position = seq(1:maxseqlen))&#xA;&#xA;  # readAAMultipleAlignment requires equal length of sequences&#xA;  # instead, use readAAStringSet and fill &amp;quot;-&amp;quot; on right&#xA;  # assumes left-aligned MSA&#xA;  for (i in 1:length(seqs)) {&#xA;    &#xA;    seqlen &amp;lt;- nchar(seqs[i])&#xA;    &#xA;    if (seqlen &amp;lt; maxseqlen) {&#xA;      fill &amp;lt;- maxseqlen - seqlen&#xA;      seqs[[i]] &amp;lt;- paste0(as.character(seqs[[i]]),&#xA;                          paste(rep(&amp;quot;-&amp;quot;, fill), collapse = &amp;quot;&amp;quot;))&#xA;    }&#xA;&#xA;    # initiate vector of zeros to store hit positions&#xA;    positions &amp;lt;- rep(0, maxseqlen)&#xA;    &#xA;    lapply(motifs, function(motif) {&#xA;      &#xA;      # factor to add to gregexpr output to center hit&#xA;      # for motifs with an even number of characters, use left center&#xA;      shift &amp;lt;- ceiling(nchar(motif)/2)&#xA;      &#xA;      hits &amp;lt;- unlist(gregexpr(motif, seqs[[i]])) + shift&#xA;      &#xA;      lapply(hits, function(hit) {&#xA;        positions[hit] &amp;lt;&amp;lt;- 1&#xA;      }) # end apply over hits&#xA;    &#xA;    }) # end apply over motifs&#xA;    &#xA;    results &amp;lt;- dplyr::bind_cols(results, positions)&#xA;    &#xA;  } # end loop over sequences&#xA;  &#xA;  names(results) &amp;lt;- c(&amp;quot;position&amp;quot;, paste0(&amp;quot;seq&amp;quot;, seq(1:length(seqs))))&#xA;  results&#xA;  &#xA;} # end function&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Running this function on the provided fastas, we get a tables where positions in sequences matching motifs are recorded as &lt;code&gt;1&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;setwd(&amp;quot;/path/to/files&amp;quot;)&#xA;ngp &amp;lt;- c(&amp;quot;PGP&amp;quot;,&amp;quot;RGP&amp;quot;,&amp;quot;KGP&amp;quot;)&#xA;msa1 &amp;lt;- find_motifs(msa = &amp;quot;msa1.fasta&amp;quot;,&#xA;                    motifs = ngp)&#xA;msa2 &amp;lt;- find_motifs(msa = &amp;quot;msa2.fasta&amp;quot;,&#xA;                    motifs = ngp)&#xA;&#xA;dplyr::slice(msa1, 120:130)&#xA;&#xA;# A tibble: 11 × 4&#xA;   position  seq1  seq2  seq3&#xA;      &amp;lt;int&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;&#xA; 1      120     0     0     0&#xA; 2      121     0     0     0&#xA; 3      122     1     1     1&#xA; 4      123     0     0     0&#xA; 5      124     0     0     0&#xA; 6      125     1     1     0&#xA; 7      126     0     0     0&#xA; 8      127     0     0     0&#xA; 9      128     0     0     1&#xA;10      129     0     0     0&#xA;11      130     0     0     0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And here's a function to make the plots.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;heatmap_separate &amp;lt;- function(data) {&#xA;  require(ggplot2)&#xA;  require(tidyr)&#xA;  numseqs &amp;lt;- ncol(data) - 1&#xA;  numrows &amp;lt;- nrow(data)&#xA;  toplot &amp;lt;- data |&amp;gt; tidyr::pivot_longer(cols = -position,&#xA;                                        names_to = &amp;quot;sequence&amp;quot;,&#xA;                                        values_to = &amp;quot;hits&amp;quot;)&#xA;    ggplot(toplot) +&#xA;    geom_tile(mapping = aes(x = position,&#xA;                            y = sequence,&#xA;                            fill = as.factor(hits),&#xA;                            color = as.factor(hits))) +&#xA;    scale_x_continuous(limits = c(1,numrows),&#xA;                       n.breaks = 10,&#xA;                       expand = c(0,0)) +&#xA;    scale_y_discrete(expand = c(0,0),&#xA;                     limits = rev) +&#xA;    scale_fill_manual(values = c(&amp;quot;#f2f0f7&amp;quot;,&amp;quot;#54278f&amp;quot;)) +&#xA;    scale_color_manual(values = c(&amp;quot;#f2f0f7&amp;quot;,&amp;quot;#54278f&amp;quot;)) +&#xA;    geom_hline(yintercept = 0.5 + 0:numseqs,&#xA;               color = &amp;quot;black&amp;quot;,&#xA;               linewidth = 1.5) + &#xA;    theme_minimal() +&#xA;    theme(legend.position = &amp;quot;none&amp;quot;,&#xA;          axis.title.y = element_blank(),&#xA;          panel.grid = element_blank(),&#xA;          axis.ticks.x = element_line(color = &amp;quot;black&amp;quot;),&#xA;          panel.background = element_blank())&#xA;}&#xA;&#xA;heatmap_separate(msa1)&#xA;heatmap_separate(msa2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h4&gt;Plot for fasta #1&lt;/h4&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RuttW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RuttW.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h4&gt;Plot for fasta #2&lt;/h4&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FCFLC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FCFLC.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The lines representing hits are all plotted as the same color (&lt;code&gt;#54278f&lt;/code&gt;), but they appear to be different colors as an artifact of plotting lots of narrow tiles. This can be rectified by viewing the plots as svg files, though I can't display that file type here.&lt;/p&gt;&#xA;" OwnerUserId="3967" LastEditorUserId="3967" LastEditDate="2022-12-06T15:22:06.030" LastActivityDate="2022-12-06T15:22:06.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20133" PostTypeId="2" ParentId="20126" CreationDate="2022-12-06T11:11:12.253" Score="1" Body="&lt;p&gt;Just found the right solution.&#xA;iFeatureOmega does exactly what I was looking for. Sequence based and has CLI.&#xA;&lt;a href=&quot;https://ifeatureomega.erc.monash.edu/iFeature2/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ifeatureomega.erc.monash.edu/iFeature2/index.html&lt;/a&gt;&#xA;Additionally it supports protein, DNA/RNA and also small molecule inputs.&lt;/p&gt;&#xA;" OwnerUserId="12203" LastActivityDate="2022-12-06T11:11:12.253" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20134" PostTypeId="1" CreationDate="2022-12-06T22:58:44.087" Score="1" ViewCount="120" Body="&lt;p&gt;There are a few different commonly used clustering algorithms within the single-cell space, although Leiden seems to be the top choice these days. &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/FlowSOM.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;FlowSOM&lt;/a&gt; is a classic package for analyzing flow cytometry data. It has a two-step approach for clustering. First, it builds a self-organizing map (SOM) where cells are assigned to 100 grid points. Then, it uses ConsensusClusterPlus to meta-cluster the SOM codes. I have not seen other packages that take this approach. There is no publication for FlowSOM and the documentation does not explain this part. Why this two-step approach and why isn't this more common?&lt;/p&gt;&#xA;" OwnerUserId="35" LastEditorUserId="3848" LastEditDate="2022-12-07T00:32:59.757" LastActivityDate="2022-12-07T14:52:53.620" Title="FlowSOM multi-step clustering" Tags="|single-cell|clustering|flow-cytometry|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20135" PostTypeId="2" ParentId="20134" CreationDate="2022-12-07T00:20:48.483" Score="1" Body="&lt;p&gt;The two step approach will be used in any cluster algorithm (except see &amp;quot;network theory&amp;quot; below), you just may not see it. Clustering methods are very common in genomics, they are popular because they are very visual and will stack against a heatmap nicely. They have been replaced by dimensionality reduction such as PCA type analysis. For data mining PCA is a much more robust method for a number of reasons for example they overcome the problems listed as &amp;quot;The Bad&amp;quot; and &amp;quot;The Ugly&amp;quot; below. PCA type analysis is difficult to visualise, particularly when it steps into multi-dimensions and thats why developers like clustering ... easy for the developer to understand and easy for the use to visually appreciate, particularly against a heat map.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Any classical clustering&lt;/strong&gt; The first step is to build a matrix and the second is to reorganise the relationships within the matrix using a pre-defined algorithm, unweighted pairwise group method with arithmetic mean (UPGMA).&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The matrix in this case will be the grid.&lt;/li&gt;&#xA;&lt;li&gt;The second step is the clustering method. The method used ConsensusClusterPlus by &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2881355/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wilkerson and Hayes (2010)&lt;/a&gt; is, 0&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The consensus clustering (CC) method provides quantitative and visual stability evidence for estimating the number of unsupervised classes in a dataset.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;All that means is it stacks a heat map onto a clustering tree. Here's the output of the paper and the 'workhorse' is the tree presented in &amp;quot;A&amp;quot;,&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/d96jt.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/d96jt.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Like all great movies, it has 3 faces,&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The Good&lt;/strong&gt; using &lt;code&gt;k-means clustering&lt;/code&gt; and that is well respected method in supervised-learning and suspect it performs pretty well in unsupervised learning too. It's a good method.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The bad&lt;/strong&gt; There could easily be artefacts in the deeper relationships of the tree and that would need a full empirical investigation. I know that because genetic diversity is often measured by trees and loads of attention has gone into modelling the process and CCP is assuming homogeneity in the rate process - which rarely hplds in biology. Thus, there will be situations where that falls over, however, what I strongly suspect is that the k-means methods is attempting to circumvent this situation, without doing so explicitly (if that makes any sense). I would need to study the algorithm to assess this and I would not use clustering in data mining/unsupervised learning.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The Ugly&lt;/strong&gt; The basic problem with clustering and the reason it isn't used in genetic diversity &lt;strong&gt;*&lt;/strong&gt;, is because there's loads of clustering algorithms, they give different answers and there's no way to distinguish which is which. Having said all that &lt;code&gt;k-means clustering&lt;/code&gt; is very respectable.&lt;/p&gt;&#xA;&lt;p&gt;The reason it's not common is that there has never been a consensus on the cluster algorithm and modellers prefer to avoid them because they are only used for data exploration/mining and not useful for hypothesis testing. &lt;em&gt;However&lt;/em&gt; I do stress that &lt;code&gt;k-means&lt;/code&gt; supervised learning is a very good method, but that is not the same as clustering (which is unsupervised learning).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;*&lt;/strong&gt; or at least if it is used its just a method to visual a matrix and a like &lt;em&gt;Warning clustering!&lt;/em&gt; will appear somewhere in the legend.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Parameters and conclusion&lt;/strong&gt; There is a parameter issue with the Figure above is tracking manually. It demonstrates &lt;em&gt;k&lt;/em&gt; has a &lt;em&gt;HUGE&lt;/em&gt; influence on the result. In real world supervised learning parameterisation is automated, never ever would it be done manually. The technique is called parameterisation and it's kinda important (so like the user can't use a parameter that gives an output that suits them!! [I've not said that BTW]). So you'd ask an algorithm above the &lt;em&gt;k&lt;/em&gt;=&lt;em&gt;x&lt;/em&gt; algorithm to find the optimal value of &lt;em&gt;k&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;If&lt;/em&gt; &lt;code&gt;FlowSOM&lt;/code&gt; automates the value for &lt;em&gt;k&lt;/em&gt; and then tells you ... thats good and I'd be happy with that. &lt;em&gt;If&lt;/em&gt; it doesn't, you have to manually feed loads of values of &lt;em&gt;k&lt;/em&gt; and provide an independent assessment on which was the best. That is not cool and you will have to perform a laborious manual calculation.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Network theory&lt;/strong&gt; Leiden looks to be part of network theory e.g. &lt;a href=&quot;https://www.nature.com/articles/s41598-019-41695-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trag and van Eck (2019)&lt;/a&gt;. That is a significant advantage over classical clustering, because network theory is being dumped over the top, i.e. the third step, which can yield real power over the data set. Personally I follow graph theory, but both have major potential and are heavily under-utilised. Its not really fair to compare a classical method with the power of network theory. Certainly when entering graph theory what you can leverage from the graph (network) is enormous. I personally, thought the whole area was being ignored (there's lots of applications), but maybe its time will yet come into the mainstream.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To @Pallie's comments of&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;dimensionality and distance&lt;/strong&gt;. I do not disagree at all and is the reason why dimensionality reduction approaches such as PCA or multi-dimensional scaling are preferred.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;RAM overhead&lt;/strong&gt; Of course the calculation will be hard on RAM: if &lt;em&gt;k&lt;/em&gt; was fixed at 2 or 3, it will run reasonably quickly, but might not be meaning. Any &lt;em&gt;k&lt;/em&gt; parameterisation will be RAM intensive, but IMO thats the nature of big data calculation and cloud computing: you just pick a computer with loads of RAM or just fit more RAM. There will also be a risk of course of RAM bottlenecks on standard machines (where it just freezes). The key thing is whether the approaches are giving biologically significant different results and personally I would consider it empirically. These are different calculations and this is complex data.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-12-07T14:52:53.620" LastActivityDate="2022-12-07T14:52:53.620" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20137" PostTypeId="1" AcceptedAnswerId="20145" CreationDate="2022-12-07T21:03:20.650" Score="0" ViewCount="486" Body="&lt;p&gt;I am looking for a way to obtain SNP names from their coordinates (Chromosome:BP) - very much the reverse of the question answered here:&lt;/p&gt;&#xA;&lt;p&gt;(&lt;a href=&quot;https://stackoverflow.com/questions/20251612/map-snp-ids-to-genome-coordinates&quot;&gt;https://stackoverflow.com/questions/20251612/map-snp-ids-to-genome-coordinates&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Example:&lt;/em&gt; I have identified that a SNP at position 64377836 on Chromosome 17 is associated with a certain factor in my analysis and would like to find out whether it has a SNP ID and what this ID is.&lt;/p&gt;&#xA;&lt;p&gt;Manually, I can search databases for Chr17:64377836 and find rs668 - but I would like to find a way to do this automatically, preferably in R.&lt;/p&gt;&#xA;&lt;p&gt;Is this possible in R - and if so, how?&lt;/p&gt;&#xA;" OwnerUserId="16606" LastEditorUserId="16606" LastEditDate="2022-12-07T22:44:26.413" LastActivityDate="2022-12-08T14:54:17.223" Title="Identify SNP from genetic position" Tags="|r|gwas|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20138" PostTypeId="1" CreationDate="2022-12-07T21:22:18.987" Score="2" ViewCount="54" Body="&lt;p&gt;After annotating with &lt;a href=&quot;https://www.ensembl.org/vep&quot; rel=&quot;nofollow noreferrer&quot;&gt;VEP&lt;/a&gt; a VCF file, we obtain different fields. One of them is called &lt;code&gt;Codons&lt;/code&gt; which represents the affected codon in the transcript of the gene. Below is a screenshot of &lt;code&gt;Insertions&lt;/code&gt; from a sample:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cm3Xc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cm3Xc.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Variants and INDELs affect genes differently depending on the transcript we are looking at. My question is, for some notations for Insertions like in the screenshot, how can we interpret these results? Let's take the example of &lt;code&gt;-/TC&lt;/code&gt;. Is &lt;code&gt;TC&lt;/code&gt; inserted at the end of the previous codon? or the next codon of the transcript?&lt;/p&gt;&#xA;&lt;p&gt;Below are the different representations I have in mind where &lt;code&gt;|&lt;/code&gt; is the position of the indel and &lt;code&gt;^&lt;/code&gt; is the possible insertion scenario of &lt;code&gt;TC&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;                     Before insertion               after insertion&#xA;                           v|&#xA;first scenario          xxx xxx xxx                   xxx TCx xxx&#xA;&#xA;second scenario         xxx xxx xxx                   xxx xxx TCx&#xA;                            |  ^ &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My gut is telling me that this depends more on the strand I am looking at. If the strand is &lt;code&gt;+&lt;/code&gt;, it is likely to be the second scenario. Otherwise, it would be the first scenario. Am I thinking correctly? or does this notation have another meaning? Or maybe VEP is not able its position?&lt;/p&gt;&#xA;&lt;p&gt;Thanks for the help.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2022-12-07T21:22:18.987" Title="In VEP annotation, how is the codon field interpreted?" Tags="|sequence-annotation|vep|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20139" PostTypeId="1" AcceptedAnswerId="20146" CreationDate="2022-12-07T22:14:59.163" Score="0" ViewCount="190" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9547684/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have tried to use UpsetR to visualise the input file, which can be found &lt;a href=&quot;https://cloudstor.aarnet.edu.au/plus/s/OupXC8rF2bDFtRV&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. How is it possible to make UpsetR accept an order defined in &lt;code&gt;sets=c(&amp;quot;Nlab&amp;quot;, &amp;quot;NQLD&amp;quot;, &amp;quot;Ngla&amp;quot;, &amp;quot;Nsyl&amp;quot;, &amp;quot;Ntom&amp;quot;, &amp;quot;Ntab&amp;quot;, &amp;quot;Natt&amp;quot;, &amp;quot;Cann&amp;quot;, &amp;quot;Stub&amp;quot;, &amp;quot;Slyc&amp;quot;, &amp;quot;Vvin&amp;quot;, &amp;quot;Atha&amp;quot;)&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;with keep.order = T&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/UemPY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UemPY.png&quot; alt=&quot;with keep.order = T&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;with keep.order = F&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/gkcGe.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gkcGe.png&quot; alt=&quot;with keep.order = F&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; library(&amp;quot;UpSetR&amp;quot;)&#xA;&amp;gt; orthogroups_df&amp;lt;- read.table(&amp;quot;orthogroups.GeneCount.tsv&amp;quot;,  header=T, stringsAsFactors = F)&#xA;&amp;gt; #All species&#xA;&amp;gt; selected_species &amp;lt;- colnames(orthogroups_df)[2:(ncol(orthogroups_df) -1)] &#xA;&amp;gt; selected_species&#xA; [1] &amp;quot;Atha&amp;quot; &amp;quot;Cann&amp;quot; &amp;quot;NQLD&amp;quot; &amp;quot;Natt&amp;quot; &amp;quot;Ngla&amp;quot; &amp;quot;Nlab&amp;quot; &amp;quot;Nsyl&amp;quot; &amp;quot;Ntab&amp;quot; &amp;quot;Ntom&amp;quot; &amp;quot;Slyc&amp;quot; &amp;quot;Stub&amp;quot; &amp;quot;Vvin&amp;quot;&#xA;&amp;gt; head(orthogroups_df)&#xA;  Orthogroup Atha Cann NQLD Natt Ngla Nlab Nsyl Ntab Ntom Slyc Stub Vvin Total&#xA;1  OG0000000    0    0  965    0    0    3    0    0    0    0    0    0   968&#xA;2  OG0000001    0    1    3    0    0  448    0    0    0    0    0    0   452&#xA;3  OG0000002    0    1  313    0    0  120    1    0    1    0    0    0   436&#xA;4  OG0000003    0   93   15   21   46   16   33   63   36   25   39   26   413&#xA;5  OG0000004    1   42    2   34  109    6    8  154   11    9    4    0   380&#xA;6  OG0000005    0    2   61    1   34   44   91   70   43   20    1    0   367&#xA;&amp;gt; ncol(orthogroups_df)&#xA;[1] 14&#xA;&amp;gt; orthogroups_df[orthogroups_df &amp;gt; 0] &amp;lt;- 1&#xA;&amp;gt; # we only show intersections of interest ,  &#xA;&amp;gt; intersections=list(list(selected_species),&#xA;+                    list(&amp;quot;NQLD&amp;quot;, &amp;quot;Ngla&amp;quot;, &amp;quot;Natt&amp;quot;, &amp;quot;Nlab&amp;quot;, &amp;quot;Nsyl&amp;quot;, &amp;quot;Ntab&amp;quot;, &amp;quot;Ntom&amp;quot;),&#xA;+                    list(&amp;quot;Stub&amp;quot;, &amp;quot;Slyc&amp;quot;),&#xA;+                    list(&amp;quot;Atha&amp;quot;, &amp;quot;Vvin&amp;quot;),&#xA;+                    list(&amp;quot;Ntab&amp;quot;, &amp;quot;Nsyl&amp;quot;, &amp;quot;Ntom&amp;quot;),&#xA;+                    list(&amp;quot;Nlab&amp;quot;, &amp;quot;NQLD&amp;quot;, &amp;quot;Ngla&amp;quot;), &#xA;+                    list(&amp;quot;Nlab&amp;quot;, &amp;quot;NQLD&amp;quot;, &amp;quot;Nsyl&amp;quot;), &#xA;+                    list(&amp;quot;Nlab&amp;quot;, &amp;quot;Ngla&amp;quot;, &amp;quot;Nsyl&amp;quot;), &#xA;+                    list(&amp;quot;NQLD&amp;quot;, &amp;quot;Nsyl&amp;quot;, &amp;quot;Ngla&amp;quot;))&#xA;&amp;gt; upset(orthogroups_df, &#xA;+       text.scale = c(1.4),&#xA;+       sets=c(&amp;quot;Nlab&amp;quot;, &amp;quot;NQLD&amp;quot;, &amp;quot;Ngla&amp;quot;, &amp;quot;Nsyl&amp;quot;, &amp;quot;Ntom&amp;quot;, &amp;quot;Ntab&amp;quot;, &amp;quot;Natt&amp;quot;, &amp;quot;Cann&amp;quot;, &amp;quot;Stub&amp;quot;, &amp;quot;Slyc&amp;quot;, &amp;quot;Vvin&amp;quot;, &amp;quot;Atha&amp;quot;), &#xA;+       order.by = &amp;quot;freq&amp;quot;,&#xA;+       keep.order=T, &#xA;+       intersections = intersections, &#xA;+       sets.x.label=&amp;quot;Total number of orthogroups&amp;quot;, &#xA;+       mainbar.y.label = &amp;quot;Number of orthogroups&amp;quot;) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3329" LastEditorUserId="73" LastEditDate="2022-12-08T19:04:30.637" LastActivityDate="2022-12-14T04:26:49.670" Title="Manually order the set fails in UpsetR" Tags="|r|ggplot2|upsetr|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20140" PostTypeId="1" CreationDate="2022-12-08T06:53:27.380" Score="2" ViewCount="101" Body="&lt;p&gt;I'm conducting a simulation and I need to obtain polygenic risk scores (PRS) using genome wide association studies (GWAS) summary statistics.&#xA;It is known that the GWAS will give BETA and OR for quantitative and binary phenotypes respectively.&#xA;When conducting the simulation for quantitative traits I used the following code to obtain PRS from GWAS performed:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./plink1.9 --bfile binary_file1_name --linear interaction --pheno pheno_file_name --covar cov_file_name --parameters 1 --allow-no-sex&#xA;./plink1.9 --bfile binary_file2_name --score plink.assoc.linear 2 4 7 header --out prs_quantitative&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is fine, where plink.assoc.linear file contains BETA.&lt;/p&gt;&#xA;&lt;p&gt;When conducting simulation for binary traits I used the following code to obtain PRS from GWAS performed:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./plink1.9 --bfile binary_file1_name --logistic interaction --pheno pheno_file_name --covar cov_file_name --parameters 1 --allow-no-sex&#xA;./plink1.9 --bfile binary_file2_name --score plink.assoc.logistic 2 4 7 header --out prs_binary&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm not sure whether we need to transform OR to BETA before inputting the file to --score in performing this for binary traits or, simply use the original plink.assoc.logistic which has OR (instead of BETA) to go ahead.&lt;/p&gt;&#xA;&lt;p&gt;Simply, do we need to input file containing BETA values &amp;quot;only&amp;quot; for --score or either a file containing OR values or BETA values is fine?&lt;/p&gt;&#xA;" OwnerUserId="16608" LastActivityDate="2022-12-08T06:53:27.380" Title="Input for --score in PLINK 1.9 when generationg PRS using GWAS summary statistics" Tags="|gwas|plink|polygenic-risk-score|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20142" PostTypeId="1" CreationDate="2022-12-08T10:15:04.943" Score="4" ViewCount="177" Body="&lt;p&gt;I am trying to get dot-bracket-notations of single stranded RNA via the ViennaRNA python package (&lt;a href=&quot;https://pypi.org/project/ViennaRNA/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pypi.org/project/ViennaRNA/&lt;/a&gt;) at different temperatures. I have read in the docs (&lt;a href=&quot;https://www.tbi.univie.ac.at/RNA/ViennaRNA/doc/html/examples_c.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.tbi.univie.ac.at/RNA/ViennaRNA/doc/html/examples_c.html&lt;/a&gt;) that the folding prediction at different temperatures is possible with C..but nowhere it is described how to specify the temperature when using python. Does anyone know how to specify the temperature at which the RNA folding is predicted with the ViennaRNA package in python?&lt;/p&gt;&#xA;&lt;p&gt;Thanks for the help!&lt;/p&gt;&#xA;" OwnerUserId="16610" LastActivityDate="2022-12-19T19:36:46.717" Title="RNA folding at specific temperature with ViennaRNA in python" Tags="|python|rna|secondary-structure|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20144" PostTypeId="1" CreationDate="2022-12-08T12:07:24.457" Score="2" ViewCount="116" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9547754/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am using CNVkit on my data using hg38 as reference. The command that I am using is the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cnvkit.py batch sample.bam  -n control.bam -m wgs -f reference.fasta  --target-avg-size 1000 --output-dir results/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, on the &lt;a href=&quot;https://cnvkit.readthedocs.io/en/stable/nonhybrid.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;CNVkit documentation page&lt;/a&gt; is reported that&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The &lt;code&gt;batch --method wgs&lt;/code&gt; option uses the given reference genome’s sequencing-accessible regions (“access” BED) as the “targets” – these will be calculated on the fly if not provided.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Problem is, when it calculates accesibile regions from the reference, there are a lot of them listed in the log file that are not present in the final .bed output, and I cannot figure out why. Accessible regions are later divided in bins, and bins without enough coverage are discarded later in the process. These are the outputs I get on a short example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Log file:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;WGS protocol: recommend '--annotate' option (e.g. refFlat.txt) to help locate genes in output files.&#xA;NC_000001.10: Scanning for accessible regions&#xA;      Accessible region NC_000001.10:10000-177417 (size 167417)&#xA;      Accessible region NC_000001.10:227417-267719 (size 40302)&#xA;      Accessible region NC_000001.10:317719-471368 (size 153649)&#xA;      Accessible region NC_000001.10:521368-2634220 (size 2112852)&#xA;      Accessible region NC_000001.10:2684220-3845268 (size 1161048)&#xA;      Accessible region NC_000001.10:3995268-13052998 (size 9057730)&#xA;      Accessible region NC_000001.10:13102998-13219912 (size 116914)&#xA;      Accessible region NC_000001.10:13319912-13557162 (size 237250)&#xA;      Accessible region NC_000001.10:13607162-17125658 (size 3518496)&#xA;      Accessible region NC_000001.10:17175658-29878082 (size 12702424)&#xA;      Accessible region NC_000001.10:30028082-103863906 (size 73835824)&#xA;      Accessible region NC_000001.10:103913906-120697156 (size 16783250)&#xA;      Accessible region NC_000001.10:120747156-120936695 (size 189539)&#xA;      Accessible region NC_000001.10:121086695-121485434 (size 398739)&#xA;      Accessible region NC_000001.10:142535434-142731022 (size 195588)&#xA;      Accessible region NC_000001.10:142781022-142967761 (size 186739)&#xA;      Accessible region NC_000001.10:143117761-143292816 (size 175055)&#xA;      Accessible region NC_000001.10:143342816-143544525 (size 201709)&#xA;      Accessible region NC_000001.10:143644525-143771002 (size 126477)&#xA;      Accessible region NC_000001.10:143871002-144095783 (size 224781)&#xA;      Accessible region NC_000001.10:144145783-144224481 (size 78698)&#xA;      Accessible region NC_000001.10:144274481-144401744 (size 127263)&#xA;      Accessible region NC_000001.10:144451744-144622413 (size 170669)&#xA;      Accessible region NC_000001.10:144672413-144710724 (size 38311)&#xA;      Accessible region NC_000001.10:144810724-145833118 (size 1022394)&#xA;      Accessible region NC_000001.10:145883118-146164650 (size 281532)&#xA;      Accessible region NC_000001.10:146214650-146253299 (size 38649)&#xA;      Accessible region NC_000001.10:146303299-148026038 (size 1722739)&#xA;      Accessible region NC_000001.10:148176038-148361358 (size 185320)&#xA;      Accessible region NC_000001.10:148511358-148684147 (size 172789)&#xA;      Accessible region NC_000001.10:148734147-148954460 (size 220313)&#xA;      Accessible region NC_000001.10:149004460-149459645 (size 455185)&#xA;      Accessible region NC_000001.10:149509645-205922707 (size 56413062)&#xA;      Accessible region NC_000001.10:206072707-206332221 (size 259514)&#xA;      Accessible region NC_000001.10:206482221-223747846 (size 17265625)&#xA;      Accessible region NC_000001.10:223797846-235192211 (size 11394365)&#xA;      Accessible region NC_000001.10:235242211-248908210 (size 13665999)&#xA;      Accessible region NC_000001.10:249058210-249240621 (size 182411)&#xA;NT_113878.1: Scanning for accessible regions&#xA;      Accessible region NT_113878.1:0-106433 (size 106433)&#xA;NT_167207.1: Scanning for accessible regions&#xA;      Accessible region NT_167207.1:0-547496 (size 547496)&#xA;NC_000002.11: Scanning for accessible regions&#xA;      Accessible region NC_000002.11:10000-3529312 (size 3519312)&#xA;      Accessible region NC_000002.11:3579312-5018788 (size 1439476)&#xA;      Accessible region NC_000002.11:5118788-16279724 (size 11160936)&#xA;      Accessible region NC_000002.11:16329724-21153113 (size 4823389)&#xA;      Accessible region NC_000002.11:21178113-31705550 (size 10527437)&#xA;      Accessible region NC_000002.11:31705551-31725939 (size 20388)&#xA;      Accessible region NC_000002.11:31726790-31816827 (size 90037)&#xA;      Accessible region NC_000002.11:31816828-31816854 (size 26)&#xA;      Accessible region NC_000002.11:31816855-31816858 (size 3)&#xA;      Accessible region NC_000002.11:31816859-33092197 (size 1275338)&#xA;      Accessible region NC_000002.11:33093197-33141692 (size 48495)&#xA;      Accessible region NC_000002.11:33142692-87668206 (size 54525514)&#xA;      Accessible region NC_000002.11:87718206-89630436 (size 1912230)&#xA;      Accessible region NC_000002.11:89830436-90321525 (size 491089)&#xA;      Accessible region NC_000002.11:90371525-90545103 (size 173578)&#xA;      Accessible region NC_000002.11:91595103-92326171 (size 731068)&#xA;      Accessible region NC_000002.11:95326171-110109337 (size 14783166)&#xA;      Accessible region NC_000002.11:110251337-149690582 (size 39439245)&#xA;      Accessible region NC_000002.11:149790582-234003741 (size 84213159)&#xA;      Accessible region NC_000002.11:234053741-239801978 (size 5748237)&#xA;      Accessible region NC_000002.11:239831978-240784132 (size 952154)&#xA;      Accessible region NC_000002.11:240809132-243102476 (size 2293344)&#xA;      Accessible region NC_000002.11:243152476-243189373 (size 36897)&#xA;NC_000003.11: Scanning for accessible regions&#xA;      Accessible region NC_000003.11:60000-17137943 (size 17077943)&#xA;NT_113878.1: Joining over small gaps&#xA;NT_167207.1: Joining over small gaps&#xA;Wrote GCF_000001405.25_GRCh37.p13_genomic.bed with 2 regions&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, multiple accessible regions are present for sequences NC_000001.10 , NC_000002.11 and NC_000003.11, but only regions for NT_113878.1 and NT_167207.1 are reported:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;NT_113878.1     0       106433&#xA;NT_167207.1     0       547496&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Why is that so? I cannot really figure out.&lt;/p&gt;&#xA;&lt;p&gt;I am sure the missed regions have data, because I am also using another CNVs caller on the same exact data, and it shows high coverage for the same regions CNVkit is discarding. Also the coverage for control data is high in those discarder regions, so this shouldn't be the matter, but I have no ideas at this point.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&#xA;The question was answered on &lt;a href=&quot;https://www.biostars.org/p/9547754/#9547948&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;. Looking into the code, CNVkit is skipping chromosomes which don't have canonical names, like 'NCs' (so, the fact that it outputs the 'NTs' chromosomes is kind of incorrect, because they're also non-canonical and they should be skipped).&lt;/p&gt;&#xA;" OwnerUserId="16611" LastEditorUserId="16611" LastEditDate="2022-12-09T15:33:54.800" LastActivityDate="2024-01-11T15:07:00.993" Title="CNVKit does not output all the accessible regions in the targets bed file" Tags="|reference-genome|bed|wgs|cnv|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20145" PostTypeId="2" ParentId="20137" CreationDate="2022-12-08T14:54:17.223" Score="2" Body="&lt;p&gt;You can do this in R using the library &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/biomaRt.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;biomaRt&lt;/a&gt; from Bioconductor. Here an example starting from a table with 3 columns: Chr, Start, End. Positions are assumed to be in GRCh38.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(biomaRt)&#xA;snpMart = useEnsembl(biomart = &amp;quot;snps&amp;quot;, &#xA;             dataset = &amp;quot;hsapiens_snp&amp;quot;)&#xA;&#xA;# assuming your table is tab-separated, with column names&#xA;myData &amp;lt;- read.table(&amp;quot;myTable.txt&amp;quot;, sep=&amp;quot;\t&amp;quot;, header=True)&#xA;&#xA;coords &amp;lt;- apply(myData, 1, paste, collapse = &amp;quot;:&amp;quot;)&#xA;&#xA;&#xA;# submit the query&#xA;getBM(attributes = c('refsnp_id', 'chr_name', 'chrom_start', 'chrom_end', 'allele'),&#xA;  filters = c('chromosomal_region'), &#xA;  values = coords, &#xA;  mart = snpMart)  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://support.bioconductor.org/p/133105/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Source&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="475" LastActivityDate="2022-12-08T14:54:17.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20146" PostTypeId="2" ParentId="20139" CreationDate="2022-12-08T19:09:34.870" Score="2" Body="&lt;p&gt;It's possible that &lt;code&gt;upset&lt;/code&gt; is using factor levels for ordering, which is the default approach for ggplot2. If this is the case, then it can be fixed by converting the variable to a factor, and using an explicit level definition:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sets &amp;lt;- factor(sets, levels=sets)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For your specific example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;upSets &amp;lt;- c(&amp;quot;Nlab&amp;quot;, &amp;quot;NQLD&amp;quot;, &amp;quot;Ngla&amp;quot;, &amp;quot;Nsyl&amp;quot;, &amp;quot;Ntom&amp;quot;, &amp;quot;Ntab&amp;quot;, &amp;quot;Natt&amp;quot;, &amp;quot;Cann&amp;quot;, &amp;quot;Stub&amp;quot;, &amp;quot;Slyc&amp;quot;, &amp;quot;Vvin&amp;quot;, &amp;quot;Atha&amp;quot;)&#xA;upSets &amp;lt;- factor(upSets, levels=upSets)&#xA;upset(orthogroups_df, &#xA;   text.scale = c(1.4),&#xA;   sets=upSets, &#xA;   order.by = &amp;quot;freq&amp;quot;,&#xA;   keep.order=T, &#xA;   intersections = intersections, &#xA;   sets.x.label=&amp;quot;Total number of orthogroups&amp;quot;, &#xA;   mainbar.y.label = &amp;quot;Number of orthogroups&amp;quot;) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-12-14T04:26:49.670" LastActivityDate="2022-12-14T04:26:49.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20147" PostTypeId="2" ParentId="20123" CreationDate="2022-12-08T20:19:45.393" Score="1" Body="&lt;p&gt;It's hard to say without looking more closely at the data, but you may be able to find a few guides that are drastically different in your counting, and see how prevalent they are within the original BAM file (before filtering). Perhaps the &lt;code&gt;-F 2048&lt;/code&gt; being used to filter out chimeric reads is overfiltering, and causing the discrepancy? Are unaligned reads included in the BAM - if not, does this explain the discrepancy?&lt;/p&gt;&#xA;&lt;p&gt;Since the total reads doesn't match the original, I'd look into this. Are the extracted FASTQs consistent read length? Maybe some trimming was performed that removed the guides?&lt;/p&gt;&#xA;&lt;p&gt;Do you know what reference the BAM files were mapped to? Perhaps this reference did not include the guide sequences, and were mapped to the genome instead? You may be able to extract this information from the BAM like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; samtools view -H filename.bam &amp;gt; filename.header&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Ideally you'd have the original FASTQs to work with, but it sounds like those are unavailable...&lt;/p&gt;&#xA;" OwnerUserId="92" LastEditorUserId="92" LastEditDate="2022-12-08T23:24:11.147" LastActivityDate="2022-12-08T23:24:11.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20148" PostTypeId="2" ParentId="18988" CreationDate="2022-12-08T22:32:15.720" Score="1" Body="&lt;p&gt;If you want a statistical reason not to use pseudobulk to do DE for marker finding, I think it is the same reason not to use any DE method for marker finding: the &lt;strong&gt;double dipping problem&lt;/strong&gt; (see &lt;a href=&quot;https://arxiv.org/abs/2012.02936&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gao and Witten 2020&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Briefly, if you cluster a dataset based on genes (we can safely say that clustering in PCA space is &lt;em&gt;still&lt;/em&gt; based on genes), then asking which genes are differentially expressed between clusters is a statistical fallacy.&#xA;We could just as easily divide apples in 2 groups according to their shade of red and then ask which RGB channels exhibit a statistically significant difference.&lt;/p&gt;&#xA;&lt;p&gt;This means that even in the absence of real structure, you will identify very low p-values for your DE test. The OSCA book section you mention links to another &lt;a href=&quot;http://bioconductor.org/books/3.14/OSCA.advanced/marker-detection-redux.html#p-value-invalidity&quot; rel=&quot;nofollow noreferrer&quot;&gt;section&lt;/a&gt; in the advanced book that tells you the same thing with a small practical demonstration generating a 200-dimensional Gaussian and clustering it, then testing for differences in clusters.&lt;/p&gt;&#xA;&lt;p&gt;This does &lt;em&gt;not&lt;/em&gt; mean, however, that markers found this way are meaningless; it simply means that you cannot use a p-value cutoff to define whether a gene is a marker in this scenario, or say that since you have statistically significant markers two clusters are different.&lt;/p&gt;&#xA;&lt;p&gt;You can still rank these markers by fold change, difference in percentage expression, AUC, entropy, etc &lt;em&gt;as well as p-value&lt;/em&gt;. Moreover, markers identified using the traditional methods tend to make biological sense, and statistically rigorous methods are still in development (there was a beautiful talk by Anna Neufeld on an approach involving count splitting, you can read the preprint &lt;a href=&quot;https://arxiv.org/abs/2207.00554&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The test you suggest is interesting, but I don't think you would solve the double dipping problem; this being said it looks more robust and appropriate than using every single cell as a replicate.&lt;/p&gt;&#xA;" OwnerUserId="16618" LastActivityDate="2022-12-08T22:32:15.720" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20149" PostTypeId="1" CreationDate="2022-12-08T22:49:21.267" Score="1" ViewCount="91" Body="&lt;p&gt;Hi there I'm new to WDL and I need to complete a two-step workflow,&lt;/p&gt;&#xA;&lt;p&gt;Basically, what I need to do is to extract and merge two reads dataset (forward and reverse) for a sample and pass the merged, uncompressed file to a different tool for genome inference. I'm able to complete the first step with no major issues; however, when it comes to get my output to the next task I'm facing some problems. Below a copy of the workflow I wrote&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;version 1.0&#xA;&#xA;workflow step2 {&#xA;    input {&#xA;        String PANGENIE_CONTAINER = &amp;quot;overcraft90/eblerjana_pangenie:2.0.1&amp;quot;&#xA;        &#xA;        File FORWARD_FASTQ # compressed R1&#xA;        File REVERSE_FASTQ # compressed R2&#xA;        String NAME = &amp;quot;sample&amp;quot; # how to loop over samples' name in numerical order (maybe grub names' prefix)!?&#xA;&#xA;        File PANGENOME_VCF # input vcf with variants to be genotyped&#xA;        File REF_GENOME # reference for variant calling&#xA;        File FASTQ_FILE # sample's FASTQ&#xA;        String VCF_PREFIX = &amp;quot;genotype&amp;quot; # string to attach to a sample's genotype&#xA;        String EXE_PATH = &amp;quot;/app/pangenie/build/src/PanGenie&amp;quot; # path to PanGenie executable in Docker&#xA;&#xA;        Int CORES = 24 # number of cores to allocate for PanGenie execution&#xA;        Int DISK = 300 # storage memory for output files&#xA;        Int MEM = 100 # RAM memory allocated&#xA;    }&#xA;&#xA;    call reads_extraction_and_merging {&#xA;        input:&#xA;        in_container_pangenie=PANGENIE_CONTAINER,&#xA;        in_forward_fastq=FORWARD_FASTQ,&#xA;        in_reverse_fastq=REVERSE_FASTQ,&#xA;        in_label=NAME, #later can be plural&#xA;        in_cores=CORES,&#xA;        in_disk=DISK,&#xA;        in_mem=MEM&#xA;    }&#xA;&#xA;    call genome_inference {&#xA;        input:&#xA;        in_container_pangenie=PANGENIE_CONTAINER, # not sure whether Docker needs to be re-run&#xA;        in_pangenome_vcf=PANGENOME_VCF,&#xA;        in_reference_genome=REF_GENOME,&#xA;        in_executable=EXE_PATH,&#xA;        fastq_file=FASTQ_FILE,&#xA;        prefix_vcf=VCF_PREFIX,&#xA;        in_cores=CORES,&#xA;        in_disk=DISK,&#xA;        in_mem=MEM&#xA;    }&#xA;&#xA;    output {&#xA;        File sample = reads_extraction_and_merging.fastq_file&#xA;        File genotype = genome_inference.vcf_file&#xA;    }&#xA;}&#xA;&#xA;task reads_extraction_and_merging {&#xA;    input {&#xA;        String in_container_pangenie&#xA;        File in_forward_fastq&#xA;        File in_reverse_fastq&#xA;        String in_label&#xA;        Int in_cores&#xA;        Int in_disk&#xA;        Int in_mem&#xA;    }&#xA;    command &amp;lt;&amp;lt;&amp;lt;&#xA;        cat ~{in_forward_fastq} ~{in_reverse_fastq} | gzip -dc &amp;gt; ~{in_label}.fastq&#xA;        cp ~{in_label}.fastq /home/mat/cromwell-executions/step2/&#xA;    &amp;gt;&amp;gt;&amp;gt;&#xA;    output {&#xA;        File fastq_file = &amp;quot;~{in_label}.fastq&amp;quot;&#xA;    }&#xA;    runtime {&#xA;        docker: in_container_pangenie&#xA;        memory: in_mem + &amp;quot; GB&amp;quot;&#xA;        cpu: in_cores&#xA;        disks: &amp;quot;local-disk &amp;quot; + in_disk + &amp;quot; SSD&amp;quot;&#xA;    }&#xA;}&#xA;&#xA;task genome_inference {&#xA;    input {&#xA;        String in_container_pangenie&#xA;        File in_reference_genome&#xA;        File in_pangenome_vcf&#xA;        String in_executable&#xA;        String prefix_vcf&#xA;        File fastq_file&#xA;        Int in_cores&#xA;        Int in_disk&#xA;        Int in_mem&#xA;    }&#xA;    command &amp;lt;&amp;lt;&amp;lt;&#xA;        echo &amp;quot;vcf: ~{in_pangenome_vcf}&amp;quot; &amp;gt; /app/pangenie/pipelines/run-from-callset/config.yaml&#xA;        echo &amp;quot;reference: ~{in_reference_genome}&amp;quot; &amp;gt;&amp;gt; /app/pangenie/pipelines/run-from-callset/config.yaml&#xA;        echo $'reads:\n sample: ~{fastq_file}' &amp;gt;&amp;gt; /app/pangenie/pipelines/run-from-callset/config.yaml&#xA;        echo &amp;quot;pangenie: ~{in_executable}&amp;quot; &amp;gt;&amp;gt; /app/pangenie/pipelines/run-from-callset/config.yaml&#xA;        echo &amp;quot;outdir: /app/pangenie&amp;quot; &amp;gt;&amp;gt; /app/pangenie/pipelines/run-from-callset/config.yaml&#xA;        cd /app/pangenie/pipelines/run-from-callset&#xA;        snakemake --cores ~{in_cores}&#xA;    &amp;gt;&amp;gt;&amp;gt;&#xA;    output {&#xA;        File vcf_file = &amp;quot;~{prefix_vcf}.vcf&amp;quot;&#xA;    }&#xA;    runtime {&#xA;        docker: in_container_pangenie&#xA;        memory: in_mem + &amp;quot; GB&amp;quot;&#xA;        cpu: in_cores&#xA;        disks: &amp;quot;local-disk &amp;quot; + in_disk + &amp;quot; SSD&amp;quot;&#xA;        preemptible: 1 # can be useful for tools which execute sequential steps in a pipeline generating intermediate outputs&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I try to summarize what's going on. After getting my two reads dataset, merging them and unzipping the output to &lt;code&gt;~{in_label}.fastq&lt;/code&gt;; I have to get this new file and feed it to the next task, so my idea was to copy it to a directory and form there get its path in the &lt;em&gt;&lt;strong&gt;.json&lt;/strong&gt;&lt;/em&gt; I use to run the WDL with Cromwell. First of all, I believe it might not be the right way to do it and if it is the case what should I do instead? Secondly, is there anything else in the code which might be changed? For instance, I realised I cannot use &lt;code&gt;bgzip&lt;/code&gt; to speed up the unzipping step taking advantage of multiple threads; this will be useful when scaling to multiple samples.&#xA;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="14271" LastActivityDate="2022-12-08T22:49:21.267" Title="Feeding the output of one task into another in WDL" Tags="|wdl|docker|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20150" PostTypeId="2" ParentId="18487" CreationDate="2022-12-09T01:42:45.753" Score="1" Body="&lt;p&gt;I am not sure if this qualifies as an official answer. I have contacted UCSC bioinformatics team. They were were very kind and told me that they would add any track that was requested by users. After my request, they kindly added the track for E.coli BW25113. If you want your reference tracks to be added you could submit a request at this website:&#xA;&lt;a href=&quot;https://genome.ucsc.edu/assemblyRequest.html?all&quot; rel=&quot;nofollow noreferrer&quot;&gt;Geneme assembly search and request page.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then, at the upper right corner click on &amp;quot;Can't find your assembly?&amp;quot; button and submit your request. See the screenshot below from the webpage.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Yacuh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Yacuh.png&quot; alt=&quot;screenshot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2022-12-09T01:42:45.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20151" PostTypeId="1" CreationDate="2022-12-09T03:47:11.287" Score="5" ViewCount="285" Body="&lt;p&gt;So I have been trying to get into visualizing proteins in python, so after some research I ended up on a tutorial that was teaching you how to visualize a protein from the COVID-19 virus, so I went and setup anaconda, got jupyter notebook working in vscode, and downloaded the necessary files from the PDB database, and made sure they were in the same directory as my notebook but when I run the the &lt;code&gt;nglview.show_biopython(structure)&lt;/code&gt; function I get &lt;code&gt;ValueError: I/O opertation on a closed file&lt;/code&gt;. I'm stymied this is my first time using jupyter notebook so maybe there is something I'm missing, I don't know.&lt;/p&gt;&#xA;&lt;p&gt;This what the code looks like&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio.PDB import * &#xA;import nglview as nv&#xA;&#xA;&#xA;parser = PDBParser()&#xA;structure = parser.get_structure(&amp;quot;6YYT&amp;quot;, &amp;quot;6YYT.pdb&amp;quot;)&#xA;view = nv.show_biopython(structure)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The error looks like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Output exceeds the size limit. Open the full output data in a text editor&#xA;---------------------------------------------------------------------------&#xA;ValueError                                Traceback (most recent call last)&#xA;~\AppData\Local\Temp\ipykernel_8776\2743687014.py in &amp;lt;module&amp;gt;&#xA;----&amp;gt; 1 view = nv.show_biopython(structure)&#xA;&#xA;c:\Users\jerem\anaconda3\lib\site-packages\nglview\show.py in show_biopython(entity, **kwargs)&#xA;    450     '''&#xA;    451     entity = BiopythonStructure(entity)&#xA;--&amp;gt; 452     return NGLWidget(entity, **kwargs)&#xA;    453 &#xA;    454 &#xA;&#xA;c:\Users\jerem\anaconda3\lib\site-packages\nglview\widget.py in __init__(self, structure, representations, parameters, **kwargs)&#xA;    243         else:&#xA;    244             if structure is not None:&#xA;--&amp;gt; 245                 self.add_structure(structure, **kwargs)&#xA;    246 &#xA;    247         if representations:&#xA;&#xA;c:\Users\jerem\anaconda3\lib\site-packages\nglview\widget.py in add_structure(self, structure, **kwargs)&#xA;   1111         if not isinstance(structure, Structure):&#xA;   1112             raise ValueError(f'{structure} is not an instance of Structure')&#xA;-&amp;gt; 1113         self._load_data(structure, **kwargs)&#xA;   1114         self._ngl_component_ids.append(structure.id)&#xA;   1115         if self.n_components &amp;gt; 1:&#xA;...&#xA;--&amp;gt; 200         return io_str.getvalue()&#xA;    201 &#xA;    202 &#xA;&#xA;ValueError: I/O operation on closed file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I only get this error when using &lt;code&gt;nglview.show_biopython&lt;/code&gt;, when I run the &lt;code&gt;get_structure()&lt;/code&gt; function it appears to read the file just fine. I can visualize other molucles just fine, or maybe that's because I was using the ASE library instead of a file. I don't know, that's why I'm here.&lt;/p&gt;&#xA;&lt;p&gt;Update: Recently I found out that I can visualize the protein using &lt;code&gt;nglview.show_file()&lt;/code&gt; instead of using &lt;code&gt;nglview.show_biopython()&lt;/code&gt;. Even though I can visualize proteins now and technically my problem has been solved I would still like to know why the &lt;code&gt;show_biopython()&lt;/code&gt; function isn't working properly.&lt;/p&gt;&#xA;" OwnerUserId="16619" LastEditorUserId="10049" LastEditDate="2023-01-24T11:50:02.143" LastActivityDate="2023-01-24T11:50:02.143" Title="File I/O error using nglview.show_biopython(structure)" Tags="|python|structural-biology|biopython|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20152" PostTypeId="1" CreationDate="2022-12-09T10:23:25.930" Score="0" ViewCount="27" Body="&lt;p&gt;I have a partially described gene from a plant with CDS and 3'UTR described but no 5'UTR; if I know the locus and upstream and downstream regions, is there a tool I can use for predicting regulatory DNA and UTRs for a given gene?&lt;/p&gt;&#xA;&lt;p&gt;My annotation data source is the assembly of &lt;em&gt;Phaseolus vulgaris&lt;/em&gt; &lt;a href=&quot;https://ftp.ensemblgenomes.org/pub/plants/release-55/gff3/phaseolus_vulgaris/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16622" LastEditorUserId="2108" LastEditDate="2022-12-14T21:00:55.307" LastActivityDate="2022-12-14T21:00:55.307" Title="I have a partially described gene from a plant, which tool may I use for predicting regolatory and UTRs sequences from a single DNA fragment?" Tags="|sequence-annotation|functional-annotation|tools|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20153" PostTypeId="1" AcceptedAnswerId="20170" CreationDate="2022-12-09T17:39:50.300" Score="0" ViewCount="246" Body="&lt;p&gt;I would like to extract date of acquisiton from mzXML and mzML files, using Python. Is it possible e.g. with &lt;a href=&quot;https://pypi.org/project/pyteomics/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pyteomics&lt;/a&gt;? pymzml is only for mzML files. Are there other libraries that can do that? I assume of course that the date of acquisition is recored in mzXML/mzML files.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3848" LastEditDate="2022-12-09T23:24:14.403" LastActivityDate="2022-12-12T22:07:18.220" Title="How to extract date of acquisiton from mzXML and mzML files with Python?" Tags="|python|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20154" PostTypeId="2" ParentId="20153" CreationDate="2022-12-09T21:44:27.283" Score="0" Body="&lt;p&gt;&lt;code&gt;pymzml&lt;/code&gt; certainly works via &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/28/7/1052/209917&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bald &lt;em&gt;et al&lt;/em&gt; 2012&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c bioconda pymzml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Whilst this only works with &lt;code&gt;mzML&lt;/code&gt; you simply convert &lt;code&gt;mzXML&lt;/code&gt; to &lt;code&gt;pymzml&lt;/code&gt; via &lt;code&gt;ProteoWizard&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;msconvert your_mzXML_file --mzML&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/10807/how-to-convert-mzxml-to-mzml-format-mass-spectrometry-files&quot;&gt;It is described here with your previous question with @haci&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I believe you managed this via&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda create -n openms -c bioconda openms&#xA;FileConverter -in file.mzXML -out file.mzML&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I suspect the underlying question is direct parsing in &lt;code&gt;mzXML&lt;/code&gt; without the need to perform a conversion from &lt;code&gt;mzXML&lt;/code&gt; to &lt;code&gt;mzML&lt;/code&gt; or without using &lt;code&gt;Bioconductor&lt;/code&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Possibility&lt;/strong&gt; The following may be of use &lt;code&gt;massspec-toolbox&lt;/code&gt; at:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://code.google.com/p/massspec-toolbox/source/browse/#svn/trunk/mzxml&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://code.google.com/p/massspec-toolbox/source/browse/#svn/trunk/mzxml&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note, I've not tested nor read this but the repository looks genuine.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-12-09T21:44:27.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20155" PostTypeId="1" CreationDate="2022-12-09T22:13:55.740" Score="1" ViewCount="209" Body="&lt;p&gt;I'm having trouble install shapeit4 4.2.2 on my new Mint 21 installation (Ubuntu 22.04 LTS). &lt;a href=&quot;https://odelaneau.github.io/shapeit4/#installation&quot; rel=&quot;nofollow noreferrer&quot;&gt;According to the official docs&lt;/a&gt;, the requirements are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;HTSlib: A great C library for reading/writing high-throughput sequencing data.&lt;/li&gt;&#xA;&lt;li&gt;BOOST: A free peer-reviewed portable C++ source libraries. SHAPEIT4 uses two specific BOOST libraries: iostreams and program_options.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I installed &lt;a href=&quot;http://www.htslib.org/download/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HTSlib&lt;/a&gt; with &lt;code&gt;make&lt;/code&gt; and &lt;code&gt;sudo make install&lt;/code&gt;, and libboost using its calls &lt;code&gt;./bootstrap&lt;/code&gt; and &lt;code&gt;./b2&lt;/code&gt; but I still get errors when trying to run the &lt;code&gt;make&lt;/code&gt; file for shapeit4:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/usr/bin/ld: cannot find /home/user/Tools/htslib-1.11/libhts.a: No such file or directory&#xA;/usr/bin/ld: cannot find /usr/lib/x86_64-linux-gnu/libboost_iostreams.a: No such file or directory&#xA;/usr/bin/ld: cannot find /usr/lib/x86_64-linux-gnu/libboost_program_options.a: No such file or directory&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9417" LastEditorUserId="3694" LastEditDate="2022-12-13T12:32:06.847" LastActivityDate="2023-02-17T12:19:41.890" Title="How do I install shapeit4 on Ubuntu/Mint?" Tags="|software-installation|phasing|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20156" PostTypeId="2" ParentId="20155" CreationDate="2022-12-09T22:13:55.740" Score="0" Body="&lt;p&gt;The documentation appears to be missing an edit one has to do. Open the &lt;code&gt;makefile&lt;/code&gt; in the shapeit4 dir, and you will see something like (for version 4.2.2):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#COMPILER MODE C++11&#xA;CXX=g++ -std=c++11&#xA;&#xA;#HTSLIB LIBRARY [SPECIFY YOUR OWN PATHS]&#xA;HTSLIB_INC=&lt;span class=&quot;math-container&quot;&gt;$(HOME)/Tools/htslib-1.11&#xA;HTSLIB_LIB=$&lt;/span&gt;(HOME)/Tools/htslib-1.11/libhts.a&#xA;&#xA;#BOOST IOSTREAM &amp;amp; PROGRAM_OPTION LIBRARIES [SPECIFY YOUR OWN PATHS]&#xA;BOOST_INC=/usr/include&#xA;BOOST_LIB_IO=/usr/lib/x86_64-linux-gnu/libboost_iostreams.a&#xA;BOOST_LIB_PO=/usr/lib/x86_64-linux-gnu/libboost_program_options.a&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So the paths appear to be hardcoded. Set these to the paths where you installed htslib (and compile it first).&lt;/p&gt;&#xA;&lt;p&gt;For libboost, I found that its own installer doesn't place files in this expected location. So you can either change these paths to where you installed libboost, or you can use the apt manager version which will put the files in the right location by default. This is done with &lt;code&gt;sudo apt-get install libboost-all-dev&lt;/code&gt; (&lt;a href=&quot;https://stackoverflow.com/questions/24173330/cmake-is-not-able-to-find-boost-libraries&quot;&gt;found here&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;After this, just run the &lt;code&gt;make&lt;/code&gt; file again, and the output should looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;user@computer:/dir/shapeit4-4.2.2$ make&#xA;make: Nothing to be done for 'all'.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9417" LastActivityDate="2022-12-09T22:13:55.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20157" PostTypeId="1" CreationDate="2022-12-09T22:35:07.287" Score="1" ViewCount="382" Body="&lt;p&gt;I am trying to extract data from fast5-file with python 3.9.13 in Ubuntu. I have found a library &amp;quot;fast5_research&amp;quot;(This package comprises an API to HDF containers used by the research groups within Oxford Nanopore Technologies), installed on my PC and updated my software with &amp;quot;sudo apt update&amp;quot;. According to &lt;a href=&quot;https://nanoporetech.github.io/fast5_research/examples.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; I attempted to read fasta5-file with the next code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from fast5_research import Fast5&#xA;filename='/home/master/Desktop/0.fast5'&#xA;with Fast5(filename) as fh:&#xA;    raw = fh.get_read(raw=True)&#xA;    summary = fh.summary()&#xA;print('Raw is {} samples long.'.format(len(raw)))&#xA;print('Summary {}.'.format(summary))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but as a result, I have an error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;KeyError                                  Traceback (most recent call last)&#xA;/tmp/ipykernel_410530/3650860803.py in &amp;lt;module&amp;gt;&#xA;      4 from fast5_research import Fast5&#xA;      5 filename='/home/master/Desktop/0.fast5'&#xA;----&amp;gt; 6 with Fast5(filename) as fh:&#xA;      7     raw = fh.get_read(raw=True)&#xA;      8     summary = fh.summary()&#xA;&#xA;~/anaconda3/lib/python3.9/site-packages/fast5_research/fast5.py in __init__(self, fname, read)&#xA;     85 &#xA;     86         # Attach channel_meta as attributes, slightly redundant&#xA;---&amp;gt; 87         for k, v in _clean_attrs(self[self.__channel_meta_path__].attrs).items():&#xA;     88             setattr(self, k, v)&#xA;     89         # Backward compat.&#xA;&#xA;h5py/_objects.pyx in h5py._objects.with_phil.wrapper()&#xA;&#xA;h5py/_objects.pyx in h5py._objects.with_phil.wrapper()&#xA;&#xA;~/anaconda3/lib/python3.9/site-packages/h5py/_hl/group.py in __getitem__(self, name)&#xA;    175                 raise ValueError(&amp;quot;Invalid HDF5 object reference&amp;quot;)&#xA;    176         else:&#xA;--&amp;gt; 177             oid = h5o.open(self.id, self._e(name), lapl=self._lapl)&#xA;    178 &#xA;    179         otype = h5i.get_type(oid)&#xA;&#xA;h5py/_objects.pyx in h5py._objects.with_phil.wrapper()&#xA;&#xA;h5py/_objects.pyx in h5py._objects.with_phil.wrapper()&#xA;&#xA;h5py/h5o.pyx in h5py.h5o.open()&#xA;&#xA;KeyError: 'Unable to open object (component not found)'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How to fix this error?&lt;/p&gt;&#xA;" OwnerUserId="16563" LastEditorUserId="10049" LastEditDate="2022-12-10T23:10:30.330" LastActivityDate="2024-01-05T02:06:10.690" Title="Reading a Fast5-file with Python" Tags="|python|nanopore|jupyter|base-calling|fast5|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20158" PostTypeId="2" ParentId="20157" CreationDate="2022-12-09T23:18:11.237" Score="0" Body="&lt;p&gt;The code looks fine &lt;code&gt;fast5&lt;/code&gt; is simply &lt;code&gt;HDF5&lt;/code&gt; format (rapid I/O).&lt;/p&gt;&#xA;&lt;p&gt;The error,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;'Unable to open object (component not found)'&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You are not the only person &lt;a href=&quot;https://stackoverflow.com/questions/55473368/h5py-randomly-unable-to-open-object-component-not-found&quot;&gt;to have this error&lt;/a&gt; and it could be a bug in the code with &lt;code&gt;hdf5&lt;/code&gt; in Python. Just to note I use &lt;code&gt;hdf5&lt;/code&gt; without a any problem.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Problem with the installation (?), this is the best guess to resolve it but I notice from your error log .. the errors are coming from Anaconda, I assume you made a &lt;code&gt;conda&lt;/code&gt; installation, but the post implies &lt;code&gt;apt-get&lt;/code&gt;?? Just check where the package is located.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c hcc fast5-research&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then perform a conda environment creation  and activation of the same package as &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/20161/how-to-run-how-are-we-stranded-here&quot;&gt;here&lt;/a&gt;. I think you've already done that. &lt;code&gt;anaconda3&lt;/code&gt; is definitely your Python3 version ... and in that case it might be better to do the full &lt;code&gt;conda&lt;/code&gt; installation because it stops things getting mixed up.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;NOTE&lt;/strong&gt;&#xA;There is a Docker container&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker pull quay.io/biocontainers/fast5-research&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would recommend using this (it will almost certainly work), if you know how to leverage Docker (which is sort of complicated).&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;File corruption ... examine the file via the HDFView GUI, here &lt;a href=&quot;https://www.hdfgroup.org/downloads/hdfview/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.hdfgroup.org/downloads/hdfview/&lt;/a&gt; . If you can't see within the file via the viewer - I think the file has a problem. Perhaps examine via hdf5 ... wait no that gets complicated.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It can't find the file or a path error (below).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Then kindly try:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from fast5_research import Fast5&#xA;import pathlib as Path&#xA;import sys&#xA;&#xA;filename=Path('/home/master/Desktop/0.fast5')&#xA;if filename.exists():&#xA;    print ('Okay {} is fine.'.format(filename))&#xA;    filename='/home/master/Desktop/0.fast5'&#xA;else:&#xA;    pathway = Path(__file__).absolute().parent&#xA;    if not Path(pathway, '0.fast5').is_file():&#xA;        raise ValueError ('Sorry filename has not been found')&#xA;        sys.exit()&#xA;    filename = str(Path(pathway, '0.fast5'))&#xA;try:&#xA;    if Path(filename).exists():&#xA;        with Fast5(filename) as fh:&#xA;            raw = fh.get_read(raw=True)&#xA;            summary = fh.summary()&#xA;        print('Raw is {} samples long.'.format(len(raw)))&#xA;        print('Summary {}.'.format(summary))&#xA;except Exception as err:&#xA;    print (&amp;quot;The error codes are {} and {}&amp;quot;.format(err.message, err.args))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Please execute the code in the same &lt;code&gt;dir&lt;/code&gt; as &lt;code&gt;'/home/master/Desktop/'&lt;/code&gt; or maybe just &lt;code&gt;'/home/Desktop&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;In &lt;code&gt;Fasta5&lt;/code&gt; will almost certainly take a &lt;code&gt;path&lt;/code&gt; from &lt;code&gt;pathlib&lt;/code&gt; but I've erred in case not. &lt;code&gt;HDF5&lt;/code&gt; will certainly accept a &lt;code&gt;pathlib&lt;/code&gt; pathway.&lt;/p&gt;&#xA;&lt;p&gt;I've not tested it but just post back what happens.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-12-10T16:08:35.553" LastActivityDate="2022-12-10T16:08:35.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20159" PostTypeId="2" ParentId="20155" CreationDate="2022-12-09T23:29:44.217" Score="1" Body="&lt;p&gt;Manually editing makefiles is just... ugh. Save yourself the effort and use conda:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda create -n shapeit&#xA;conda install -c bioconda shapeit4 -n shapeit&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2022-12-09T23:29:44.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20160" PostTypeId="1" CreationDate="2022-12-10T10:32:13.610" Score="0" ViewCount="370" Body="&lt;p&gt;Can you guide me on how could take an input fastq file, read the length from read and then feed it into the findtail's command?&lt;/p&gt;&#xA;&lt;p&gt;I have suggested concerns about documentation and program defaults to my prof and asked him to switch to cutadapt instead of using findtail that has no documentation. But he refuses so I have no option but to proceed with what he wants.&lt;/p&gt;&#xA;&lt;p&gt;I want the user to input a RNA-seq data fastq file. Since the first line of each file contains the sequence length information, such as described:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;@SRR3003130.1 HWI-ST741:596:H5FMVBCXX:1:1101:1244:1952 length=51&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How do I pick the &lt;em&gt;length=51&lt;/em&gt;, the &lt;strong&gt;51&lt;/strong&gt; from this and feed in into the following command's &lt;strong&gt;--seqlength&lt;/strong&gt; parameter:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    ./findtail --input_file Rrp6_S38_all_R1_001_egal.fastq --seqlength 51 \&#xA;--endgap 5 --taillength 10 --identity 100 --ptype A --stype T \&#xA;--output_format fasta --output_type pr &amp;gt; pr.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The tool's command that I'm using only requires us to enter --seqlength once. So I do not think that determining the length of each and every read in the fastq file is of any use. Just simply the length from the first line of first read would suffice.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;If something is lacking from my question, please let me know how can I improve.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="16328" LastEditorUserId="73" LastEditDate="2022-12-15T19:57:43.810" LastActivityDate="2022-12-15T19:57:43.810" Title="How do I determine read length from a user-inputted fastq file and then input that information into a tool's command line?" Tags="|rna-seq|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20161" PostTypeId="1" AcceptedAnswerId="20163" CreationDate="2022-12-10T12:19:08.160" Score="1" ViewCount="205" Body="&lt;p&gt;I'm trying to run the python library called how_are_we_stranded_here. I have installed with &lt;code&gt;pip install how_are_we_stranded_here&lt;/code&gt;. I have also installed its dependencies i.e kallisto 0.44.0 and RSeQC. On how_are_we_stranded_here's pypi page, it gives the following command under the usage heading:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; check_strandedness --gtf Yeast.gtf --transcripts Yeast_cdna.fasta \&#xA;--reads_1 Sample_A_1.fq.gz --reads_2 Sample_A_2.fq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So I tried running the program in the terminal using my own files, the command was:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    check_strandedness --gtf Glycine_max.Glycine_max_v2.1.55.gtf \&#xA;--transcripts Glycine_max.Glycine_max_v2.1.cdna.all.fa \&#xA;--reads_1 SRR2079644.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But it keeps giving the error&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;./how_stranded.sh: line 1: check_strandedness: command not found&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I will be very thankful for any help on how to run it.&lt;/p&gt;&#xA;" OwnerUserId="16328" LastActivityDate="2022-12-10T15:45:32.150" Title="How to run how_are_we_stranded_here?" Tags="|rna-seq|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20162" PostTypeId="2" ParentId="20160" CreationDate="2022-12-10T12:20:57.640" Score="3" Body="&lt;p&gt;I think that you cannot rely on that information being in a fastq header. &lt;strong&gt;Note also that each read has its own header! This is not just the first line!&lt;/strong&gt; For example when I google &amp;quot;sample fastq file&amp;quot; &lt;a href=&quot;https://zenodo.org/record/3736457&quot; rel=&quot;nofollow noreferrer&quot;&gt;the first examples&lt;/a&gt; do not have it. &lt;a href=&quot;https://en.wikipedia.org/wiki/FASTQ_format&quot; rel=&quot;nofollow noreferrer&quot;&gt;Spec&lt;/a&gt; doesn't seem to say anything about it.&lt;/p&gt;&#xA;&lt;p&gt;For a one-off on a specific dataset this &lt;em&gt;might&lt;/em&gt; be adequate:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -Eo &amp;quot;length=([0-9]+)&amp;quot; test.fq | cut -d&amp;quot;=&amp;quot; -f2&#xA;# prints &amp;quot;75&amp;quot; for each occurrence, so if you trust the first line you could pass to head:&#xA;grep -Eo &amp;quot;length=([0-9]+)&amp;quot; test.fq | cut -d&amp;quot;=&amp;quot; -f2 | head -1&#xA;# or you could explicitly sort and grab the longest read len:&#xA;grep -Eo &amp;quot;length=([0-9]+)&amp;quot; test.fq | cut -d&amp;quot;=&amp;quot; -f2 | sort | tail -1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For a more general application &lt;a href=&quot;https://www.biostars.org/p/459116/&quot; rel=&quot;nofollow noreferrer&quot;&gt;I'd suggest using some existing/explicit tools for measuring read length&lt;/a&gt; like seqkit or &lt;a href=&quot;https://www.biostars.org/p/459116/#459135&quot; rel=&quot;nofollow noreferrer&quot;&gt;awk&lt;/a&gt;, rather than trusting some information that may or may not be in the header.&lt;/p&gt;&#xA;&lt;p&gt;I can't find any docs for &lt;code&gt;findtail&lt;/code&gt;, depending on what your application is I'd suggest &lt;a href=&quot;https://findtail&quot; rel=&quot;nofollow noreferrer&quot;&gt;using cutadapt instead&lt;/a&gt;, which does not require a single sequence length input.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="298" LastEditDate="2022-12-15T16:02:29.680" LastActivityDate="2022-12-15T16:02:29.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20163" PostTypeId="2" ParentId="20161" CreationDate="2022-12-10T15:32:19.067" Score="2" Body="&lt;p&gt;&lt;code&gt;pip&lt;/code&gt; should just work, anyway... Two approaches, point 1 is a last resort approach (not recommended), I definitely recommend point 2.&lt;/p&gt;&#xA;&lt;p&gt;First try,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;python3 -m pip install --upgrade how_are_we_stranded_here&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Try again does it work? Error logs?, then possibly ...&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;You need to find out where it is,&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;   pip show how_are_we_stranded_here&#xA;   pip show check_strandedness&#xA;   pip show how_stranded.sh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What you really want is the &lt;code&gt;check_strandedness&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cd ~&#xA;mkdir bin&#xA;./pathtoapp/check_stranded # make sure it works&#xA;ln -si /pathwaytoapp/how_stranded.sh /home/whatever/bin/&#xA;ln -si /pathwaytoapp/check_strandedness ~/bin/check_strandedness&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then write&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;export PATH=&amp;quot;/home/whatever/bin:$PATH&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or modify &lt;code&gt;.profile&lt;/code&gt; or &lt;code&gt;.bashrc&lt;/code&gt;, then&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;source ~/.bash_profile&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Try again ...&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;It is better to perform a conda installation&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Install either,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://docs.conda.io/en/latest/miniconda.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Miniconda&lt;/a&gt; or;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://docs.anaconda.com/anaconda/install/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Anaconda&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;    conda install -c bioconda how_are_we_stranded_here&#xA;    conda create --name stranded&#xA;    conda activate stranded&#xA;    check_strandedness -h&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-12-10T15:45:32.150" LastActivityDate="2022-12-10T15:45:32.150" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20164" PostTypeId="2" ParentId="20160" CreationDate="2022-12-10T18:54:00.000" Score="3" Body="&lt;p&gt;To get the length of the first sequence from a single-line-per-sequence fastq, the second line can be piped through &lt;code&gt;wc -c&lt;/code&gt;, and stored in a shell variable:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqLength=$( head -n 2 input_file_name.fastq | tail -n 1 | wc -c)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This variable can then be used for the program arguments using &lt;code&gt;${variable}&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqLength=$(head -n 2 input_file_name.fastq | tail -n 1 | wc -c)&#xA;./findtail --input_file input_file_name.fastq --seqlength ${seqLength} \&#xA;    --endgap 5 --taillength 10 --identity 100 --ptype A --stype T \&#xA;    --output_format fasta --output_type pr &amp;gt; pr.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If the input file is gzipped [I have no idea if findtail supports gzipped files], then the length can be determined by piping through zcat:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqLength=$( zcat input_file_name.fastq.gz | head -n 2 | tail -n 1 | wc -c)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As mentioned by many comments, note that some FASTQ files won't have the same length for each read, and it's unclear how &lt;code&gt;findtail&lt;/code&gt; will behave for such a file.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2022-12-15T19:56:36.323" LastActivityDate="2022-12-15T19:56:36.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20165" PostTypeId="1" AcceptedAnswerId="20168" CreationDate="2022-12-11T17:28:31.120" Score="3" ViewCount="120" Body="&lt;p&gt;I'm using the following script to detect strandedness of my paired end RNA-seq data.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env bash&#xA;&#xA;cd ~/Desktop/fastq&#xA;&#xA;for infile in *_1.fastq&#xA;do&#xA;   base=&lt;span class=&quot;math-container&quot;&gt;$(basename $&lt;/span&gt;{infile} _1.fastq)&#xA;&#xA;    docker run quay.io/biocontainers/guessmylt:0.2.5--py_0 GUESSmyLT \&#xA;    --reads &lt;span class=&quot;math-container&quot;&gt;${infile} $&lt;/span&gt;{base}_2.fastq&#xA;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But when I run it, it keeps giving the error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Building DAG of jobs... File path //SRR3475890_1.fastq contains double&#xA;'/'. This is likely unintended. It can also lead to inconsistent&#xA;results of the file-matching approach used by Snakemake. File path&#xA;//SRR3475890_2.fastq contains double '/'. This is likely unintended.&#xA;It can also lead to inconsistent results of the file-matching approach&#xA;used by Snakemake. MissingInputException in line 46 of&#xA;/usr/local/lib/python3.6/site-packages/GUESSmyLT/subsample: Missing&#xA;input files for rule subsample_fastq: //SRR3475890_2.fastq&#xA;//SRR3475890_1.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Why is it appending double '/' to the fastq file's path? Nowhere in my script did I mention any path.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;I apologize for the very vague title, English is not my first language, so I did not know how else to phrase it.&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="16328" LastEditorUserId="650" LastEditDate="2022-12-12T20:48:30.887" LastActivityDate="2022-12-12T20:48:30.887" Title="How to remove double '/' from file path, bash script" Tags="|rna-seq|bash|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20166" PostTypeId="2" ParentId="18719" CreationDate="2022-12-11T17:43:46.647" Score="2" Body="&lt;p&gt;There are multiple protein isoforms that the gene &lt;em&gt;may&lt;/em&gt; encode. The hyphen-number after the Uniprot ID represents the isoform —in a Uniprot you find the detail of these at the bottom.&#xA;The canonical isoform in Uniprot is the most likely one to be biologically relevant, while in NCBI and Ensembl it is the longest.&#xA;In a limited amount of case multiple isoforms are biologically relevant.&#xA;For the sake of sanity most variant annotation tools have a &lt;code&gt;canonical&lt;/code&gt; argument to use the canonical.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2022-12-11T17:43:46.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20168" PostTypeId="2" ParentId="20165" CreationDate="2022-12-12T05:45:04.677" Score="4" Body="&lt;p&gt;The issue here is that the current working directory inside the container is &lt;code&gt;/&lt;/code&gt;. You can see this with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker run --rm quay.io/biocontainers/guessmylt:0.2.5--py_0 python \&#xA;    -c 'import os; print(os.getcwd())'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When &lt;a href=&quot;https://github.com/NBISweden/GUESSmyLT&quot; rel=&quot;nofollow noreferrer&quot;&gt;GUESSmyLT&lt;/a&gt; tries to get the absolute path of it's arguments (&lt;code&gt;--reads&lt;/code&gt;, &lt;code&gt;--reference&lt;/code&gt;, &lt;code&gt;--output&lt;/code&gt;, etc), it actually appends a trailing slash to the current working directory&lt;sup&gt;&lt;a href=&quot;https://github.com/NBISweden/GUESSmyLT/blob/v1.0/GUESSmyLT/GUESSmyLT.py#L21&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;&lt;/sup&gt;, hence the double slash in the returned file names.&lt;/p&gt;&#xA;&lt;p&gt;To pass the files into the container and ensure that the results are available when it exits, you'll need a &lt;a href=&quot;https://docs.docker.com/engine/reference/run/#volume-shared-filesystems&quot; rel=&quot;nofollow noreferrer&quot;&gt;volume&lt;/a&gt; mount anyway. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker run --rm -v &amp;quot;${PWD}:/working&amp;quot; quay.io/biocontainers/guessmylt:0.2.5--py_0 GUESSmyLT \&#xA;    --reads &amp;quot;/working/${base}_1.fastq&amp;quot; &amp;quot;/working/${base}_2.fastq&amp;quot; \&#xA;    --reference &amp;quot;/working/reference.fa&amp;quot; \&#xA;    --annotation &amp;quot;/working/annotation.gtf&amp;quot; \&#xA;    --output &amp;quot;/working&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2022-12-12T05:57:59.443" LastActivityDate="2022-12-12T05:57:59.443" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20169" PostTypeId="1" CreationDate="2022-12-12T14:00:32.040" Score="1" ViewCount="144" Body="&lt;p&gt;I would like to grep everything inside the second square bracket of a file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;AB1311658.1 unnamed protein product [C.sp. balc.]&#xA;&#xA;&amp;gt;evm.model.pilon.8 [actin][D.r.]&#xA;&#xA;&amp;gt;NP_0232343.1 [fucose mutarotase] [E.l.]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Obtaining:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;C.sp. balc.&#xA;&#xA;D.r.&#xA;&#xA;E.l.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I solve it?&lt;/p&gt;&#xA;&lt;p&gt;Marco&lt;/p&gt;&#xA;" OwnerUserId="15596" LastEditorUserId="12589" LastEditDate="2022-12-12T14:11:08.630" LastActivityDate="2023-01-11T23:45:34.247" Title="Grep peculiar pattern inside a file" Tags="|fasta|genome|grep|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20170" PostTypeId="2" ParentId="20153" CreationDate="2022-12-12T14:00:42.780" Score="3" Body="&lt;p&gt;You should be able to extract anything from mzML and mzXML with Pyteomics. You will have to look at your files and figure out what you need to do, though, because you need to know what element contains the information that you want. I don't often work with mzXML and in the files that I have I do not see an acquisition date at all. With mzML it appears that the &lt;code&gt;&amp;lt;run&amp;gt;&lt;/code&gt; element has this information:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;run id=&amp;quot;small_raw&amp;quot; defaultInstrumentConfigurationRef=&amp;quot;IC1&amp;quot; startTimeStamp=&amp;quot;2005-07-20T14:44:22&amp;quot;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So I can get it like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;In [1]: from pyteomics import mzml&#xA;&#xA;In [2]: with mzml.MzML('tests/test.mzML') as f:&#xA;   ...:     print(next(f.iterfind('run', recursive=False))['startTimeStamp'])&#xA;   ...: &#xA;2005-07-20T14:44:22&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;P.S. Disclaimer: I am the maintainer of Pyteomics.&lt;/p&gt;&#xA;" OwnerUserId="12589" LastEditorUserId="12589" LastEditDate="2022-12-12T22:07:18.220" LastActivityDate="2022-12-12T22:07:18.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20171" PostTypeId="2" ParentId="20169" CreationDate="2022-12-12T15:08:05.770" Score="2" Body="&lt;p&gt;For OSX, I don't have a &lt;code&gt;-P&lt;/code&gt; option (Perl pattern matching) so the best I can do is finish the one-liner by piping into &lt;code&gt;perl&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -o '\[.*\]\[[^]]*' input.fa | perl -p -e 's/[][]/ /g'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;-o&lt;/code&gt; makes &lt;code&gt;grep&lt;/code&gt; match the substring, rather than the entire line.&lt;/p&gt;&#xA;&lt;p&gt;With GNU &lt;code&gt;grep&lt;/code&gt; (the default on Linux) and the &lt;code&gt;-P&lt;/code&gt; option, it's a single line of grep where &lt;code&gt;\K&lt;/code&gt; or &lt;code&gt;(?=pattern)&lt;/code&gt;, &lt;code&gt;(?&amp;lt;=pattern)&lt;/code&gt; are used for non-capture characters and these are placed around the &lt;code&gt;[&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -oP '(?&amp;lt;=\[).*(?=\]).*(?&amp;lt;=\[)[^]]*' input.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;?&amp;lt;=&lt;/code&gt; is a &lt;a href=&quot;https://www.regextutorial.org/positive-and-negative-lookbehind-assertions.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;positive lookbehind&amp;quot;&lt;/a&gt; and the &lt;code&gt;?=&lt;/code&gt; is a &lt;a href=&quot;https://www.regextutorial.org/positive-and-negative-lookahead-assertions.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;positive lookahead&amp;quot;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;It's a lot easier to use a &lt;code&gt;perl&lt;/code&gt; one-liner.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="298" LastEditDate="2022-12-14T14:36:21.067" LastActivityDate="2022-12-14T14:36:21.067" CommentCount="15" ContentLicense="CC BY-SA 4.0" />
  <row Id="20172" PostTypeId="2" ParentId="20169" CreationDate="2022-12-12T15:49:15.667" Score="4" Body="&lt;p&gt;One way using &lt;a href=&quot;/questions/tagged/sed&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;sed&amp;#39;&quot; aria-label=&quot;show questions tagged &amp;#39;sed&amp;#39;&quot; rel=&quot;tag&quot; aria-labelledby=&quot;sed-container&quot;&gt;sed&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sed -n '/^&amp;gt;/s/.*\[\([^]]*\)\]$/\1/p' input.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;C.sp. balc.&#xA;D.r.&#xA;E.l.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Explanation:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Turn off automatic printing of the pattern space using &lt;code&gt;-n&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Address lines starting with a header using &lt;code&gt;/^&amp;gt;/&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Perform a substitution using a capture group to select everything between the last set of brackets on these lines, with &lt;code&gt;s/.*\[\([^]]*\)\]$/\1/&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Print the pattern space with &lt;code&gt;p&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2022-12-12T15:56:15.797" LastActivityDate="2022-12-12T15:56:15.797" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20173" PostTypeId="1" CreationDate="2022-12-12T22:45:13.527" Score="1" ViewCount="123" Body="&lt;p&gt;I am writing a benchling uploader script in python to upload data from a csv file in a JSON formatted schema to the benchling API platform. My function to create the JSON schema in the script looks like the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def create_antismash_schema(dataframe):&#xA;    &amp;quot;&amp;quot;&amp;quot;Create a json object for antismash output&amp;quot;&amp;quot;&amp;quot;&#xA;    return {&#xA;        &amp;quot;assayResults&amp;quot;: [&#xA;            {&#xA;                &amp;quot;fields&amp;quot;: {&#xA;                    &amp;quot;entity&amp;quot;: {&amp;quot;value&amp;quot;: j['SBI ID']},&#xA;                    &amp;quot;sequence_id&amp;quot;: {&amp;quot;value&amp;quot;: j['sequence_id']},&#xA;                    &amp;quot;category&amp;quot;: {&amp;quot;value&amp;quot;: j['Category']},&#xA;                    &amp;quot;product&amp;quot;: {&amp;quot;value&amp;quot;: j['Product']},&#xA;                    &amp;quot;start&amp;quot;: {&amp;quot;value&amp;quot;: j['start']},&#xA;                    &amp;quot;end&amp;quot;: {&amp;quot;value&amp;quot;: j['end']},&#xA;                    &amp;quot;nucleotide_length&amp;quot;: {&amp;quot;value&amp;quot;: j['nucl_length']}&#xA;                },&#xA;                &amp;quot;schemaId&amp;quot;: &amp;quot;assaysch_Z6B7TlWG&amp;quot;,&#xA;                &amp;quot;projectId&amp;quot;: 'src_tIUAcxDH'&#xA;            } for i, j in dataframe.iterrows()&#xA;        ]&#xA;    }    &#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now the data I have in benchling for the fields matches the keys in the schema and the values match the headers in the csv file I'm pulling the data from.&lt;/p&gt;&#xA;&lt;p&gt;benchling results schema looks like:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/lLCb1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lLCb1.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;csv file:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RMEX6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RMEX6.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The error I am getting when I try to upload is the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    &#xA;Uploading Antismash data to Benchling...&#xA;  0%|                                                                                                                                                                               | 0/159 [00:00&amp;lt;?, ?it/s]{&#xA;    &amp;quot;assayResults&amp;quot;: null,&#xA;    &amp;quot;errors&amp;quot;: [&#xA;        {&#xA;            &amp;quot;fields&amp;quot;: {&#xA;                &amp;quot;nucleotide_length&amp;quot;: &amp;quot;Value must be an integer&amp;quot;&#xA;            },&#xA;            &amp;quot;global&amp;quot;: []&#xA;        },&#xA;        {&#xA;            &amp;quot;fields&amp;quot;: {&#xA;                &amp;quot;nucleotide_length&amp;quot;: &amp;quot;Value must be an integer&amp;quot;&#xA;            },&#xA;            &amp;quot;global&amp;quot;: []&#xA;        },&#xA;        {&#xA;            &amp;quot;fields&amp;quot;: {&#xA;                &amp;quot;nucleotide_length&amp;quot;: &amp;quot;Value must be an integer&amp;quot;&#xA;            },&#xA;            &amp;quot;global&amp;quot;: []&#xA;        }&#xA;    ]&#xA;}&#xA;  0%|                                                                                                                                                                               | 0/159 [00:00&amp;lt;?, ?it/s]&#xA;Failed to upload summary.&#xA;HTTP status code: 400&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The field is set to be an integer in benchling and all the values I'm pulling from in the csv file for the nucl_length column are integer values so not sure why this error is beig thrown. Any guidance here appreciated, thanks.&lt;/p&gt;&#xA;" OwnerUserId="16173" LastEditorUserId="3848" LastEditDate="2022-12-13T01:12:14.737" LastActivityDate="2022-12-13T01:41:18.973" Title="HTTP status code: 400 &quot;Value must be an integer&quot; error uploading JSON schema results to benchling" Tags="|python|dataframe|pandas|json|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20175" PostTypeId="2" ParentId="20173" CreationDate="2022-12-13T01:41:18.973" Score="1" Body="&lt;p&gt;There is a major issue about the &lt;code&gt;json&lt;/code&gt; format required, however just converting to &lt;code&gt;json&lt;/code&gt; is easy because &lt;code&gt;iterrows&lt;/code&gt; has to be &lt;code&gt;pandas&lt;/code&gt; and &lt;code&gt;pandas&lt;/code&gt; has its own output &lt;code&gt;json&lt;/code&gt; format ... For example,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dataframe.to_json('benchling.json'', orient='records', lines=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If that fails then,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;jsonOut = dataframe.to_json(orient='records')[1:-1].replace('},{', '} {')&#xA;with open('benchling.json', 'w') as fout:&#xA;    fout.write(jsonOut)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, I don't think it would work and at a minimum I think the headers of the &lt;code&gt;csv&lt;/code&gt; will need to change.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The error&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;nucleotide_length&amp;quot;: &amp;quot;Value must be an integer&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Looking at your code ....&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;nucleotide_length&amp;quot;: {&amp;quot;value&amp;quot;: j['nucl_length']}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Sure this is right?&lt;/p&gt;&#xA;&lt;p&gt;Why not try ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;nucleotide_length&amp;quot;: int(j['nucl_length'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thats what the error seems to be saying.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-12-13T01:41:18.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20176" PostTypeId="1" AcceptedAnswerId="20178" CreationDate="2022-12-13T07:58:13.297" Score="3" ViewCount="345" Body="&lt;p&gt;I have the following simple nextflow script, that takes fastq paired reads and runs them through the atria adapter trimming.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;// Declare syntax version&#xA;nextflow.enable.dsl=2&#xA;&#xA;/* &#xA; * pipeline input parameters &#xA; */&#xA;&#xA;&#xA;params.reads = &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$projectDir/data/*_{1,2}.fastq&quot;&#xA;params.transcriptome_file = &quot;$&lt;/span&gt;projectDir/data/Glycine_max.cdna.all.fasta&amp;quot;&#xA;params.gtf_file = &amp;quot;$projectDir/data/Glycine_max.gtf&amp;quot;&#xA;params.outdir = &amp;quot;results&amp;quot;&#xA;&#xA;&#xA;/*&#xA; * define the `TRIM` process that creates trimmed.reads.fastq files&#xA; * given the reads.fastq file. &#xA; */&#xA; &#xA;process TRIM {&#xA;&#xA;    tag &amp;quot;$pair_id&amp;quot;&#xA;&#xA;    input:&#xA;    tuple val(pair_id), path(reads)&#xA;&#xA;    output:&#xA;    path(pair_id)&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    atria -r &lt;span class=&quot;math-container&quot;&gt;${reads[0]} -R $&lt;/span&gt;{reads[1]} -t 1 -l 12 -q 30 -o $pair_id&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;workflow {&#xA;&#xA;    read_pairs_ch = Channel.fromFilePairs( params.reads, checkIfExists:true )&#xA;&#xA;    trim_ch=TRIM(read_pairs_ch)&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Next what I want to achieve is input the atria.trimmed.fastq files generated by the trim process into a channel so that I could use it as an input for other process. But atria trimming tool also creates two additional log files along with atria.trimmed.fastq. How do i exclude those additional files and only input the trimmed files into a channel for later use.&lt;/p&gt;&#xA;&lt;p&gt;Best wishes.&lt;/p&gt;&#xA;" OwnerUserId="16328" LastActivityDate="2022-12-13T14:22:48.330" Title="How to include only the trimmed fastq files into a nextflow channel and exclude any secondary log files created?" Tags="|nextflow|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20177" PostTypeId="2" ParentId="20151" CreationDate="2022-12-13T08:49:47.930" Score="3" Body="&lt;p&gt;I had a similar problem with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.PDB import * &#xA;import nglview as nv&#xA;&#xA;&#xA;parser = PDBParser(QUIET = True)&#xA;structure = parser.get_structure(&amp;quot;2ms2&amp;quot;, &amp;quot;2ms2.pdb&amp;quot;)&#xA;&#xA;save_pdb = PDBIO()&#xA;save_pdb.set_structure(structure)&#xA;save_pdb.save('pdb_out.pdb')&#xA;&#xA;view = nv.show_biopython(structure)&#xA;view&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;error was like in question:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;.................site-packages/nglview/adaptor.py:201, in BiopythonStructure.get_structure_string(self)&#xA;    199 io_str = StringIO()&#xA;    200 io_pdb.save(io_str)&#xA;--&amp;gt; 201 return io_str.getvalue()&#xA;&#xA;ValueError: I/O operation on closed file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I modified &lt;code&gt;site-packages/nglview/adaptor.py:201, in  BiopythonStructure.get_structure_string(self)&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def get_structure_string(self):&#xA;        from Bio.PDB import PDBIO&#xA;        from io import StringIO&#xA;        io_pdb = PDBIO()&#xA;        io_pdb.set_structure(self._entity)&#xA;        io_str = StringIO()&#xA;        io_pdb.save(io_str)&#xA;        return io_str.getvalue()&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;with :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def get_structure_string(self):&#xA;        from Bio.PDB import PDBIO&#xA;        &#xA;        import mmap&#xA;        &#xA;        io_pdb = PDBIO()&#xA;        &#xA;        io_pdb.set_structure(self._entity)&#xA;        &#xA;        mo = mmap_str()&#xA;        &#xA;        io_pdb.save(mo)&#xA;        &#xA;        return mo.read()&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and added this new class &lt;code&gt;mmap_str()&lt;/code&gt; , in same file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import mmap&#xA;import copy&#xA;&#xA;class mmap_str():&#xA;&#xA;    import mmap #added import at top&#xA;    &#xA;    instance = None&#xA;&#xA;    def __init__(self):&#xA;    &#xA;        self.mm = mmap.mmap(-1, 2)&#xA;        &#xA;        self.a = ''&#xA;        &#xA;        b = '\n'&#xA;        &#xA;        self.mm.write(b.encode(encoding = 'utf-8'))&#xA;        &#xA;        self.mm.seek(0)&#xA;        &#xA;        #print('self.mm.read().decode() ',self.mm.read().decode(encoding = 'utf-8'))&#xA;        &#xA;        self.mm.seek(0)&#xA;        &#xA;    def __new__(cls, *args, **kwargs):&#xA;        if not isinstance(cls.instance, cls):&#xA;            cls.instance = object.__new__(cls)&#xA;        return cls.instance&#xA;        &#xA;    def write(self, string):&#xA;        &#xA;        self.a = str(copy.deepcopy(self.mm.read().decode(encoding = 'utf-8'))).lstrip('\n')&#xA;        &#xA;        self.mm.seek(0)&#xA;        &#xA;        #print('a -&amp;gt; ', self.a)&#xA;        &#xA;        len_a = len(self.a)&#xA;        &#xA;        self.mm = mmap.mmap(-1, len(self.a)+len(string))&#xA;        &#xA;        #print('a :', self.a)&#xA;        &#xA;        #print('len self.mm ', len(self.mm))&#xA;        &#xA;        #print('lenght string : ', len(string))&#xA;        &#xA;        #print(bytes((self.a+string).encode()))&#xA;        &#xA;        self.mm.write(bytes((self.a+string).encode()))&#xA;        &#xA;        self.mm.seek(0)&#xA;        &#xA;        #print('written once ')&#xA;        &#xA;        #self.mm.seek(0)&#xA;        &#xA;    def read(self):&#xA;    &#xA;        self.mm.seek(0)&#xA;        &#xA;        a = self.mm.read().decode().lstrip('\n')&#xA;        &#xA;        self.mm.seek(0)&#xA;        &#xA;        return a&#xA;        &#xA;    def __enter__(self):&#xA;        &#xA;        return self&#xA; &#xA;    def __exit__(self, *args):&#xA;        &#xA;        pass&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;if I uncomment the print statements I'll get the :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;IOPub data rate exceeded.&#xA;The notebook server will temporarily stop sending output&#xA;to the client in order to avoid crashing it. &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;error , but commenting them out I get:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/siSKY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/siSKY.png&quot; alt=&quot;&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;while using the&lt;code&gt;nglview.show_file(filename)&lt;/code&gt; I get:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/olG6j.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/olG6j.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;tha's because, as could be seen looking at the &lt;code&gt;pdb_out.pdb&lt;/code&gt; file&lt;/p&gt;&#xA;&lt;p&gt;outputted by my code, &lt;code&gt;Biopytho.PDB.PDBParser.get_structure(name , filename)&lt;/code&gt; doesnt retrieve the pdb header responsible for generate full CRYSTALLOGRAPHIC SYMMETRY/or biopython can't handle it (not sure about this, help if you know better),  but just the coordinates.&lt;/p&gt;&#xA;&lt;p&gt;Still don't understand what is going on with the :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;--&amp;gt; 201 return io_str.getvalue()&#xA;&#xA;ValueError: I/O operation on closed file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;it could be something related to jupiter ipykernal ? hope somebody could shed more light into this, dont know how the framework runs, but is definitely different from a normal python interpreter. As an example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/cXRlL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cXRlL.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Same code in one of my Python virtualenv, will run forever, so it could be ipykernel dont like &lt;code&gt;StringIO()&lt;/code&gt;s or do something strange to them ?&lt;/p&gt;&#xA;&lt;p&gt;OK thanks to the hint in the answer below, I went inspecting PDBIO.py in github repo for version Biopython 1.80 and compared the save method of PDBIO : &lt;a href=&quot;https://github.com/biopython/biopython/blob/5ffdce852ba3721ca945ec383a69382507b4fa1b/Bio/PDB/PDBIO.py#L299&quot; rel=&quot;nofollow noreferrer&quot;&gt;def save(self, file, select=_select, write_end=True, preserve_atom_numbering=False):&lt;/a&gt; with the one in Biopython 1.79,&lt;/p&gt;&#xA;&lt;p&gt;see first bit:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/PCtMW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PCtMW.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and last bit:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/1hjO5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1hjO5.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;so apparently the big difference is the &lt;code&gt;with fhandle:&lt;/code&gt; block in version 1.80.&lt;/p&gt;&#xA;&lt;p&gt;So I realized that changing &lt;code&gt;adaptor.py&lt;/code&gt; with adding a subclass of &lt;code&gt;StringIO&lt;/code&gt; that looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from io import StringIO&#xA;class StringIO(StringIO):&#xA;&#xA;    def __exit__(self, *args, **kwargs):&#xA;        &#xA;        print('exiting from subclassed StringIO !!!!!')&#xA;        &#xA;        pass&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and modifying &lt;code&gt;def get_structure_string(self):&lt;/code&gt; like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def get_structure_string(self):&#xA;        from Bio.PDB import PDBIO&#xA;        #from io import StringIO&#xA;        io_pdb = PDBIO()&#xA;        io_pdb.set_structure(self._entity)&#xA;        io_str = StringIO()&#xA;        io_pdb.save(io_str)&#xA;        return io_str.getvalue()&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;was enough to get my Biopython 1.80 work in jupiter with nglview.&lt;/p&gt;&#xA;&lt;p&gt;That told I am not sure what are the pitfalls of not closing the StringIO object we use for the visualization, but apparently its what Biopython 1.79 was doing like my first answer using a mmap object was doing too (not closing the mmap_str)&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2022-12-15T07:26:19.927" LastActivityDate="2022-12-15T07:26:19.927" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20178" PostTypeId="2" ParentId="20176" CreationDate="2022-12-13T10:10:11.750" Score="2" Body="&lt;p&gt;With &lt;code&gt;output: path(pair_id)&lt;/code&gt;, you're telling Nextflow to expect a file or directory with the value of &lt;em&gt;pair_id&lt;/em&gt; in the process working directory when the command exits (successfully). &lt;a href=&quot;https://github.com/cihga39871/Atria&quot; rel=&quot;nofollow noreferrer&quot;&gt;Atria&lt;/a&gt; outputs all files to a directory specified using &lt;code&gt;-o PATH&lt;/code&gt;&lt;sup&gt;&lt;a href=&quot;https://github.com/cihga39871/Atria/blob/v3.2.1/docs/2.Atria_trimming_methods_and_usages.md#usage&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;&lt;/sup&gt;. This directory is created if it doesn't already exist. Note that the output directory defaults to the current working directory if &lt;code&gt;-o&lt;/code&gt; is not specified. If you'd like to write the results to a sub-directory, you could try:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process TRIM {&#xA;&#xA;    tag { pair_id }&#xA;&#xA;    publishDir &amp;quot;${params.outdir}/TRIM&amp;quot;, mode: 'copy'&#xA;&#xA;    cpus 1&#xA;&#xA;    input:&#xA;    tuple val(pair_id), path(reads)&#xA;&#xA;    output:&#xA;    tuple val(pair_id), path(trimmed_reads), emit: trimmed_fastqs&#xA;    path &amp;quot;${pair_id}/*.atria.log*&amp;quot;, emit: logs&#xA;&#xA;    script:&#xA;    def (r1, r2) = reads&#xA;&#xA;    trimmed_reads = reads.collect {&#xA;        &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${pair_id}/$&lt;/span&gt;{it.baseName}.atria.fastq&amp;quot;&#xA;    }&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    atria \\&#xA;        -r &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${r1}&quot; \\&#xA;        -R &quot;$&lt;/span&gt;{r2}&amp;quot; \\&#xA;        -o &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${pair_id}&quot; \\&#xA;        -t $&lt;/span&gt;{task.cpus} \\&#xA;        -l 12 \\&#xA;        -q 30&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In the above, we used the &lt;a href=&quot;https://www.nextflow.io/docs/latest/dsl2.html#process-named-output&quot; rel=&quot;nofollow noreferrer&quot;&gt;emit&lt;/a&gt; option in the process output definition to assign a name identifier to each declaration. We can use this feature to reference the desired channel in the workflow block. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow {&#xA;&#xA;    read_pairs_ch = Channel.fromFilePairs( params.reads, checkIfExists:true )&#xA;&#xA;    trim_out = TRIM(read_pairs_ch)&#xA;&#xA;    trim_out.trimmed_fastqs.view()&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ nextflow run main.nf &#xA;N E X T F L O W  ~  version 22.10.0&#xA;Launching `main.nf` [distraught_dalembert] DSL2 - revision: aea2d37092&#xA;executor &amp;gt;  local (3)&#xA;[5f/78908b] process &amp;gt; TRIM (A) [100%] 3 of 3 ✔&#xA;[B, [/path/to/work/6a/440a50e9a0725c18f9eede80549cb8/B/B_1.atria.fastq, /path/to/work/6a/440a50e9a0725c18f9eede80549cb8/B/B_2.atria.fastq]]&#xA;[A, [/path/to/work/5f/78908b96adee8b8d663d9b4eb4faba/A/A_1.atria.fastq, /path/to/work/5f/78908b96adee8b8d663d9b4eb4faba/A/A_2.atria.fastq]]&#xA;[C, [/path/to/work/b0/a3bb3024a4c05fffec555cce5097a7/C/C_1.atria.fastq, /path/to/work/b0/a3bb3024a4c05fffec555cce5097a7/C/C_2.atria.fastq]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To input the trimmed reads into a downstream process, just ensure that the input tuple matches the input set cardinality declared by the process. If the input declaration is not the same, you may need to apply one or more &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;operators&lt;/a&gt; to your channel to make it fit. For your alignment process, you might not need to make any changes. Also, if your alignment process needs to declare one or more reference files, just make sure these are &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#value-channel&quot; rel=&quot;nofollow noreferrer&quot;&gt;value channels&lt;/a&gt;. Most of the time, what you want is one &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#queue-channel&quot; rel=&quot;nofollow noreferrer&quot;&gt;queue channel&lt;/a&gt; and one or more value channels when you require multiple input channels, otherwise you might observe unusual behavior (e.g. see &lt;a href=&quot;https://stackoverflow.com/q/74537223/751863&quot;&gt;this question&lt;/a&gt;). The section on &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#process-multiple-input-channels&quot; rel=&quot;nofollow noreferrer&quot;&gt;multiple input channels&lt;/a&gt; in the docs is well worth taking the time to read in my opinion.&lt;/p&gt;&#xA;&lt;p&gt;A dummy alignment process might look like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process ALIGN {&#xA;&#xA;    tag { sample }&#xA;&#xA;    debug true&#xA;&#xA;    input:&#xA;    tuple val(sample), path(reads)&#xA;    path fasta&#xA;    path gtf&#xA;&#xA;    script:&#xA;    def (r1, r2) = reads&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    ls -g &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${r1}&quot;&#xA;    ls -g &quot;$&lt;/span&gt;{r2}&amp;quot;&#xA;    ls -g &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${fasta}&quot;&#xA;    ls -g &quot;$&lt;/span&gt;{gtf}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the updated workflow:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow {&#xA;&#xA;    read_pairs_ch = Channel.fromFilePairs( params.reads, checkIfExists:true )&#xA;&#xA;    transcriptome = file( params.transcriptome_file )&#xA;    gtf = file( params.gtf_file )&#xA;&#xA;    TRIM( read_pairs_ch )&#xA;&#xA;    ALIGN( TRIM.out.trimmed_fastqs, transcriptome, gtf )&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ nextflow run main.nf &#xA;N E X T F L O W  ~  version 22.10.0&#xA;Launching `main.nf` [extravagant_lattes] DSL2 - revision: 29b5315739&#xA;executor &amp;gt;  local (6)&#xA;[51/e4e261] process &amp;gt; TRIM (A)  [100%] 3 of 3 ✔&#xA;[cc/72c53d] process &amp;gt; ALIGN (A) [100%] 3 of 3 ✔&#xA;lrwxrwxrwx 1 users 57 Dec 14 00:21 C_1.atria.fastq -&amp;gt; ../../c1/9a85e19c601f2ab2c4af9c6fd203f0/C/C_1.atria.fastq&#xA;lrwxrwxrwx 1 users 57 Dec 14 00:21 C_2.atria.fastq -&amp;gt; ../../c1/9a85e19c601f2ab2c4af9c6fd203f0/C/C_2.atria.fastq&#xA;lrwxrwxrwx 1 users 40 Dec 14 00:21 Glycine_max.cdna.all.fasta -&amp;gt; ../../../data/Glycine_max.cdna.all.fasta&#xA;lrwxrwxrwx 1 users 29 Dec 14 00:21 Glycine_max.gtf -&amp;gt; ../../../data/Glycine_max.gtf&#xA;&#xA;lrwxrwxrwx 1 users 57 Dec 14 00:21 B_1.atria.fastq -&amp;gt; ../../3f/16747fbd2c9bc4c0dc945b24873cca/B/B_1.atria.fastq&#xA;lrwxrwxrwx 1 users 57 Dec 14 00:21 B_2.atria.fastq -&amp;gt; ../../3f/16747fbd2c9bc4c0dc945b24873cca/B/B_2.atria.fastq&#xA;lrwxrwxrwx 1 users 40 Dec 14 00:21 Glycine_max.cdna.all.fasta -&amp;gt; ../../../data/Glycine_max.cdna.all.fasta&#xA;lrwxrwxrwx 1 users 29 Dec 14 00:21 Glycine_max.gtf -&amp;gt; ../../../data/Glycine_max.gtf&#xA;&#xA;lrwxrwxrwx 1 users 57 Dec 14 00:21 A_1.atria.fastq -&amp;gt; ../../51/e4e261d019c65377d63826963b19b1/A/A_1.atria.fastq&#xA;lrwxrwxrwx 1 users 57 Dec 14 00:21 A_2.atria.fastq -&amp;gt; ../../51/e4e261d019c65377d63826963b19b1/A/A_2.atria.fastq&#xA;lrwxrwxrwx 1 users 40 Dec 14 00:21 Glycine_max.cdna.all.fasta -&amp;gt; ../../../data/Glycine_max.cdna.all.fasta&#xA;lrwxrwxrwx 1 users 29 Dec 14 00:21 Glycine_max.gtf -&amp;gt; ../../../data/Glycine_max.gtf&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2022-12-13T14:22:48.330" LastActivityDate="2022-12-13T14:22:48.330" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20179" PostTypeId="2" ParentId="20169" CreationDate="2022-12-13T11:32:04.507" Score="3" Body="&lt;p&gt;With GNU &lt;code&gt;grep&lt;/code&gt; (the default on Linux):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -oP '.*\[\K[^]]+' file&#xA;C.sp. balc.&#xA;D.r.&#xA;E.l.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;-o&lt;/code&gt; means &amp;quot;print only the matched portion of the line&amp;quot; and &lt;code&gt;-P&lt;/code&gt; enables Perl Compatible Regular Expressions which give us &lt;code&gt;\K&lt;/code&gt; for &amp;quot;ignore everything matched up to this point&amp;quot;. So we then look for the longest possible string ending in a &lt;code&gt;[&lt;/code&gt; and discard it (&lt;code&gt;.*\[\K&lt;/code&gt;), and then look for the longest possible stretch of non-&lt;code&gt;]&lt;/code&gt; characters after that.&lt;/p&gt;&#xA;&lt;p&gt;For very large files, you can speed things up a little by focusing only on the header lines:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -oP '^&amp;gt;.*\[\K[^]]+' file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2022-12-13T11:32:04.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20180" PostTypeId="2" ParentId="20151" CreationDate="2022-12-13T17:21:36.050" Score="4" Body="&lt;p&gt;I also figured out another way to fix this problem. After going back to the tutorial I was talking about I saw that it was made back in 2021. After seeing this I wonder if we were using the same verions of each package, turns out we were not. I'm not sure what version of nglview they were using, but they were using biopython 1.79 which was the latest verion back in 2021 and I was using biopython 1.80. While using biopython 1.80 I was getting the error seen above. But now that I'm using biopython 1.79 I get this&#xA;&lt;a href=&quot;https://i.stack.imgur.com/iRzUG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iRzUG.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So I guess there is something going on with biopython 1.80, so I'm going to stick with 1.79&lt;/p&gt;&#xA;" OwnerUserId="16619" LastActivityDate="2022-12-13T17:21:36.050" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20181" PostTypeId="1" CreationDate="2022-12-14T04:58:51.940" Score="0" ViewCount="84" Body="&lt;p&gt;I noticed that some gnomad SNPs have allele frequency =0 and some are just not found : No genes were found in this region. what is the difference between them : for example:Y-10010520-10010520.&lt;/p&gt;&#xA;&lt;p&gt;Is all frq=0 means that it is very rare and therefore zero? then what about AF=NA? Also, does the gnomad genome data contain information about the exomes also, or is the information in the exome data is  not included in the genome data in gnomad?&lt;/p&gt;&#xA;" OwnerUserId="16642" LastEditorUserId="3694" LastEditDate="2022-12-14T15:34:41.547" LastActivityDate="2023-05-13T16:06:15.893" Title="Allele frequency gnomad data" Tags="|genome|exome|gnomad|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20182" PostTypeId="1" AcceptedAnswerId="20183" CreationDate="2022-12-14T10:22:30.720" Score="2" ViewCount="986" Body="&lt;p&gt;The following Nextflow scripts takes the $gtf as an input and outputs a .bed file. What I want is to keep just the basename of the gtf file and add the .bed extension to it. I tried declaring a variable x and using basename from bash but it gives error.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;// Declare syntax version&#xA;nextflow.enable.dsl=2&#xA;&#xA;/* &#xA; * pipeline input parameters &#xA; */&#xA;&#xA;params.gtf_file = &amp;quot;$projectDir/data/Glycine_max.gtf&amp;quot;&#xA;&#xA;&#xA;    /*&#xA; * define the `GTF2BED` process that creates a BED file&#xA; * given the params.gtf_file as input. &#xA; */&#xA; &#xA;process GTF2BED {&#xA;&#xA;    input:&#xA;    path gtf&#xA;&#xA;    output:&#xA;    &amp;quot;${gtf}/.bed&amp;quot;&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    gtf2bed --gtf &lt;span class=&quot;math-container&quot;&gt;$gtf --bed &quot;$&lt;/span&gt;{gtf}/.bed&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Best wishes.&lt;/p&gt;&#xA;" OwnerUserId="16328" LastActivityDate="2022-12-14T11:58:07.167" Title="How to keep the basename of an input file and change its extension in Nextflow?" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20183" PostTypeId="2" ParentId="20182" CreationDate="2022-12-14T11:33:16.280" Score="3" Body="&lt;p&gt;You can use the &lt;a href=&quot;https://www.nextflow.io/docs/latest/script.html#check-file-attributes&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;getBaseName()&lt;/code&gt;&lt;/a&gt; method on &lt;code&gt;file&lt;/code&gt; objects to get the basename:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The invocation of any method name starting with the &lt;code&gt;get&lt;/code&gt; prefix can be shortcut by omitting the &lt;code&gt;get&lt;/code&gt; prefix and &lt;code&gt;()&lt;/code&gt; parentheses. Therefore, writing &lt;code&gt;myFile.getName()&lt;/code&gt; is exactly the same as &lt;code&gt;myFile.name&lt;/code&gt; and &lt;code&gt;myFile.getBaseName()&lt;/code&gt; is the same as &lt;code&gt;myFile.baseName&lt;/code&gt; and so on.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process GTF2BED {&#xA;&#xA;    input:&#xA;    path gtf&#xA;&#xA;    output:&#xA;    path &amp;quot;${gtf.baseName}.bed&amp;quot;&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    gtf2bed --gtf &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${gtf}&quot; --bed &quot;$&lt;/span&gt;{gtf.baseName}.bed&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you need to remove a double extension (.e.g. &lt;code&gt;.tar.gz&lt;/code&gt;) and don't want to remove all extensions (e.g. using the &lt;code&gt;getSimpleName()&lt;/code&gt; method), you can use the long form to supply the number of extensions to remove. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tarball = file( 'my.file.tar.gz' )&#xA;&#xA;println( tarball.getBaseName(2) )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Prints:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;my.file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2022-12-14T11:58:07.167" LastActivityDate="2022-12-14T11:58:07.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20184" PostTypeId="1" AcceptedAnswerId="20185" CreationDate="2022-12-14T12:38:00.500" Score="0" ViewCount="47" Body="&lt;p&gt;I downloaded exome data from gnomad to get the allele frequency of some SNPs that I'm analyzing. The SNPs are from external resource. Some SNPs (in my data) are not present in the exome data (in gnomad):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;No genes were found in this region&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As part of my report question for homework I need to answer this:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Are the SNPs in your data  that are not in gnomad are outside genes?&amp;quot;,&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What does it mean SNP outside genes? Does it mean that they are not in the exome data but in the genome data on gnomad?&lt;/p&gt;&#xA;&lt;p&gt;I found this in an article:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;&lt;a href=&quot;https://medlineplus.gov/genetics/understanding/genomicresearch/snp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Single-nucleotide polymorphisms (SNPs) are single base-pair substitutions that occur within and outside genes&lt;/a&gt;&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Also found this:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;SNP markers might be selected outside of exomes or the direct cause of a genetic mutation&amp;quot; does this means that it is found in the genome but not exome data in gnomad as it is outside the gene?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Basically, I can't seem to understand what is a SNP outside a gene and can I see this information in gnomad.&lt;/p&gt;&#xA;" OwnerUserId="16642" LastEditorUserId="3848" LastEditDate="2022-12-14T14:27:24.083" LastActivityDate="2022-12-14T14:42:06.707" Title="SNPs location out side the gene" Tags="|gene|snp|homework|exome|gnomad|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20185" PostTypeId="2" ParentId="20184" CreationDate="2022-12-14T14:36:08.017" Score="0" Body="&lt;p&gt;The answer is you can't see a SNP outside a gene for &lt;code&gt;gnomad&lt;/code&gt; because you stated it is &lt;em&gt;exome data&lt;/em&gt;. Thus, a minimal definition of the gene is it comprises: introns, exons and a promoter region.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Exome&lt;/em&gt; = &lt;em&gt;exon&lt;/em&gt; so you downloaded a subsection the gene across the genome.&lt;/p&gt;&#xA;&lt;p&gt;To reiterate, you don't have any external allele frequency data outside the gene. However, if your &lt;em&gt;in-house&lt;/em&gt; data set is whole genome data, i.e. for making your SNP calls then most of these SNPs will be outside the exon (thats a result of comparative selection pressure). So the answer is yes. It isn't clear what the &lt;em&gt;in-house&lt;/em&gt; data set comprises however and outside an &lt;em&gt;exon&lt;/em&gt; does not necessarily mean its outside the gene. A lot of mutations will occur within introns (which is outside the exon but still within a gene), but in addition more mutation will occur in intergenic regions, i.e. completely outside the exon + intron + promoter. In this sense the question seems to possibly lack clarity.&lt;/p&gt;&#xA;&lt;p&gt;To be honest, questions on bioinformatic practicals are often after-thoughts to the core activity, so I wouldn't worry too much about the semantics.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-12-14T14:42:06.707" LastActivityDate="2022-12-14T14:42:06.707" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20186" PostTypeId="2" ParentId="20181" CreationDate="2022-12-14T14:53:50.433" Score="1" Body="&lt;p&gt;I don't do eukaryotes, well humans anyway, but in population genetics an allele frequency of zero simply means it is present in no individuals against the reference allele. It doesn't mean the allele is heterozygous nor homozygous its simply absent. Whether that is informative or not is an interesting question. In addition, it does not necessarily mean the allele is rare, it is simply absent against the population in your analysis. The sickle cell anaemia allele is not rare, in certain parts of Africa it has high frequency, but it would be absent if all the individuals in the  population of the analysis that did not have African ancestry. In this specific example, it would likely reflect inaccurate sampling of the population in the analysis.&lt;/p&gt;&#xA;&lt;p&gt;If allele frequency was 1 it would mean complete homozygosity and identity, which is very informative.&lt;/p&gt;&#xA;&lt;p&gt;I'd simply refer to the &lt;a href=&quot;https://en.wikipedia.org/wiki/Allele_frequency&quot; rel=&quot;nofollow noreferrer&quot;&gt;wiki&lt;/a&gt; which is pretty good for this. Allele frequency is the entire basis of population genetics.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-12-14T15:52:28.697" LastActivityDate="2022-12-14T15:52:28.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20187" PostTypeId="2" ParentId="20151" CreationDate="2022-12-14T17:33:26.187" Score="3" Body="&lt;p&gt;This is an issue with Biopython 1.80, in particular introduced by &lt;a href=&quot;https://github.com/biopython/biopython/commit/be77131200821bb01254c87fb14ee1bf6c81dd4b&quot; rel=&quot;nofollow noreferrer&quot;&gt;a commit&lt;/a&gt; that automatically closes IO objects sent to &lt;code&gt;PDBIO.save&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I made &lt;a href=&quot;https://github.com/biopython/biopython/pull/4200&quot; rel=&quot;nofollow noreferrer&quot;&gt;a PR&lt;/a&gt; to change the behaviour which fixes this issue and should hopefully be in the next Biopython release. Until then, avoiding using Biopython 1.80 should work.&lt;/p&gt;&#xA;&lt;p&gt;Thanks @jeremiah-wade and @pippo1980 for digging into this.&lt;/p&gt;&#xA;" OwnerUserId="4622" LastActivityDate="2022-12-14T17:33:26.187" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20188" PostTypeId="2" ParentId="20152" CreationDate="2022-12-14T20:59:08.607" Score="1" Body="&lt;p&gt;Assuming that you know the gene name, you would be able to look it up in a &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-0883-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;transcriptome assembly&lt;/a&gt; presumably, or perhaps &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI&lt;/a&gt; has a full length transcript (search &lt;code&gt;txid3885[Organism:exp]&lt;/code&gt; with your gene name).&lt;/p&gt;&#xA;&lt;p&gt;For nearby regulatory DNA, you could consult &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSM3674728&quot; rel=&quot;nofollow noreferrer&quot;&gt;an ATAC-seq dataset&lt;/a&gt; or &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2022.06.15.496248v1.full.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;other similar&lt;/a&gt;, though I doubt that causal links of regulatory DNA to your gene will have been directly assayed.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-12-14T20:59:08.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20189" PostTypeId="1" AcceptedAnswerId="20190" CreationDate="2022-12-15T02:18:49.530" Score="4" ViewCount="293" Body="&lt;p&gt;I am utilizing &lt;code&gt;Kallisto&lt;/code&gt; in &lt;code&gt;Anaconda&lt;/code&gt;/&lt;code&gt;miniconda&lt;/code&gt; for RNA sequencing. I have successfully made an index; using said index to analyze RNA sequencing data has yielded an error of:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Error: index input file could not be opened!&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The implication is that the code used does in fact work but raises the question as to why the index file could not be opened. I have tried searching this error to see if others may have faced the same error message, but unfortunately no luck. Any recommendations and or solutions would be greatly appreciated. The code and output can be found below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;kallisto quant -i Homo_sapiens.GRCh38.cdna.all -o test -t 8 --single -l 250 -s 30 SRR8668755_1M_subsample.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;[quant] fragment length distribution is truncated gaussian with mean = 250, sd = 30&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error: index input file could not be opened!&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="16647" LastEditorUserId="73" LastEditDate="2022-12-15T19:48:30.910" LastActivityDate="2023-01-11T13:36:55.717" Title="Kallisto error: index input file could not be opened!" Tags="|rna-seq|read-mapping|kallisto|anaconda|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20190" PostTypeId="2" ParentId="20189" CreationDate="2022-12-15T15:20:29.313" Score="5" Body="&lt;p&gt;Usually errors like these occur when an input file doesn't exist or is corrupt. But you might also see these errors if you don't have permissions to access the file for whatever reason. Although unlikely, these can also occur if/when your system is out of memory.&lt;/p&gt;&#xA;&lt;p&gt;Assuming the index is corrupt, I think just retry the command. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;curl -O ftp://ftp.ensembl.org/pub/release-108/fasta/homo_sapiens/cdna/Homo_sapiens.GRCh38.cdna.all.fa.gz&#xA;&#xA;kallisto index \&#xA;    -i Homo_sapiens.GRCh38.cdna.all.release-108_k31.idx \&#xA;    Homo_sapiens.GRCh38.cdna.all.fa.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And test:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;kallisto quant \&#xA;    -i Homo_sapiens.GRCh38.cdna.all.release-108_k31.idx \&#xA;    -o test \&#xA;    -t 8 \&#xA;    --single \&#xA;    -l 250 \&#xA;    -s 30 \&#xA;    SRR8668755_1M_subsample.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To get this working using Windows/Anaconda Powershell, I just downloaded the cDNA sequences from &lt;a href=&quot;https://ftp.ensembl.org/pub/release-108/fasta/homo_sapiens/cdna/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ensembl.org/pub/release-108/fasta/homo_sapiens/cdna/&lt;/a&gt; and moved them to my working directory. I also downloaded and extracted the Windows Kallisto executable v0.46.1 from &lt;a href=&quot;https://pachterlab.github.io/kallisto/download.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pachterlab.github.io/kallisto/download.html&lt;/a&gt;. Then, with the FASTQ file in my working directory:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; .\kallisto_windows-v0.46.1\kallisto\kallisto.exe index `&#xA;    -i Homo_sapiens.GRCh38.cdna.all.release-108_k31.idx `&#xA;    .\Homo_sapiens.GRCh38.cdna.all.fa.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And tested with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;.\kallisto_windows-v0.46.1\kallisto\kallisto.exe quant `&#xA;    -i .\Homo_sapiens.GRCh38.cdna.all.release-108_k31.idx `&#xA;    -o test `&#xA;    -t 8 `&#xA;    --single `&#xA;    -l 250 `&#xA;    -s 30 `&#xA;    .\SRR8668755_1M_subsample.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2023-01-11T13:36:55.717" LastActivityDate="2023-01-11T13:36:55.717" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20191" PostTypeId="2" ParentId="20144" CreationDate="2022-12-15T17:07:32.923" Score="0" Body="&lt;p&gt;The OP received an answer on their question on Biostars:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The question was answered on &lt;a href=&quot;https://www.biostars.org/p/9547754/#9547948&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;. Looking into the code, CNVkit is skipping chromosomes which don't have canonical names, like 'NCs' (so, the fact that it outputs the 'NTs' chromosomes is kind of incorrect, because they're also non-canonical and they should be skipped).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2022-12-17T14:02:34.173" LastActivityDate="2022-12-17T14:02:34.173" CommentCount="0" CommunityOwnedDate="2022-12-17T14:02:22.500" ContentLicense="CC BY-SA 4.0" />
  <row Id="20192" PostTypeId="1" CreationDate="2022-12-15T20:15:23.913" Score="5" ViewCount="456" Body="&lt;p&gt;I have tried multiple different ways to get Ensemble's &lt;a href=&quot;https://www.ensembl.org/info/docs/tools/vep/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Variant Effect Predictor&lt;/a&gt; (&lt;code&gt;VEP&lt;/code&gt;) to properly install on my mac, but I keep getting same error.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;In /Users/username/ensembl-vep/htslib&#xA;make: Nothing to be done for `all'.&#xA; - unpacking ./Bio/tmp/biodbhts.zip to ./Bio/tmp/&#xA;./Bio/tmp/Bio-DB-HTS-2.11 - moving files to ./biodbhts&#xA; - making Bio::DB:HTS&#xA;clang-4.0: error: invalid version number in '-mmacosx-version-min=12.4'&#xA;Warning: ExtUtils::CBuilder not installed or no compiler detected&#xA;Proceeding with configuration, but compilation may fail during Build&#xA;&#xA;Created MYMETA.yml and MYMETA.json&#xA;Creating new 'Build' script for 'Bio-DB-HTS' version '2.11'&#xA;Building Bio-DB-HTS&#xA;Error: no compiler detected to compile 'lib/Bio/DB/HTS.c'.  Aborting&#xA;ERROR: Shared Bio::DB:HTS library not found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;--&amp;gt; When I check for the version of clang, I always get version 12, not sure where clang-4.0 is. I think error is specifically not correct path to current version of &lt;code&gt;C++&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16648" LastEditorUserId="3848" LastEditDate="2022-12-15T22:18:56.977" LastActivityDate="2022-12-30T18:16:38.280" Title="Error installing VEP on mac" Tags="|ensembl|software-installation|vep|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20193" PostTypeId="2" ParentId="20192" CreationDate="2022-12-15T21:38:09.107" Score="2" Body="&lt;p&gt;You can try and get Perl's ExUtils::Builder, described &lt;a href=&quot;https://metacpan.org/pod/ExtUtils::CBuilder&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; correctly installed and working, for example&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -MCPAN -e shell&#xA;force install Bundle::CPAN&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You might want to use &lt;code&gt;brew&lt;/code&gt; to make the build safe.&lt;/p&gt;&#xA;&lt;p&gt;I don't do this stuff, I use OSX, I would install ensemble's VEP via &lt;code&gt;conda&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c bioconda ensembl-vep&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You might want to (good idea) create a specific environment and obviously download &lt;code&gt;Anaconda&lt;/code&gt; or &lt;code&gt;miniconda&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda create --name vep&#xA;conda activate vep&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It will work, the output is below. I didn't proceed with the installation because of the shear amount of stuff downloaded into a given conda environment.  There's everything in there but the kitchen sink, like there is &lt;code&gt;mafft&lt;/code&gt;, &lt;code&gt;t-coffee&lt;/code&gt;, &lt;code&gt;clustalw&lt;/code&gt;, &lt;code&gt;muscle&lt;/code&gt; and &lt;code&gt;clustalo&lt;/code&gt; required in the download. I am honestly clueless because they are all alignment algorithms. Anyway I honestly wouldn't chase the MCPAN stuff, in this instance I definitely recommend &lt;code&gt;conda&lt;/code&gt;. Finally, you honestly you don't want all this extra stuff on your hard drive when you've finished just delete the entire environment.&lt;/p&gt;&#xA;&lt;p&gt;The final point I'd like to make is &lt;code&gt;muscle5&lt;/code&gt; (which is also in there) has recently significantly been modified making all other aligners obsolete on the runtime issue, but has to be run under the &lt;code&gt;super5&lt;/code&gt; mode.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## Package Plan ##&#xA;&#xA;  environment location: /opt/anaconda3/envs/XXX # environment was edited  out&#xA;&#xA;  added / updated specs:&#xA;    - ensembl-vep&#xA;&#xA;&#xA;The following packages will be downloaded:&#xA;&#xA;    package                    |            build&#xA;    ---------------------------|-----------------&#xA;    argtable2-2.13             |    h1de35cc_1001         2.7 MB  conda-forge&#xA;    clustalo-1.2.4             |       h9722bc1_5         331 KB  bioconda&#xA;    clustalw-2.1               |       hcd10b59_7         326 KB  bioconda&#xA;    ensembl-vep-108.2          | pl5321h4a94de4_0         3.1 MB  bioconda&#xA;    expat-2.4.9                |       he9d5cce_0         127 KB&#xA;    htslib-1.14                |       hc38c3fb_0         2.0 MB  bioconda&#xA;    libdeflate-1.7             |       h9ed2024_5          52 KB&#xA;    libgfortran-5.0.0          |11_3_0_hecd8cb5_28         142 KB&#xA;    libgfortran5-11.3.0        |      h9dfd629_28         1.4 MB&#xA;    libzlib-1.2.12             |       hfd90126_3          65 KB  conda-forge&#xA;    mafft-7.508                |       ha5712d3_0         8.7 MB  bioconda&#xA;    muscle-5.1                 |       hb339e23_1         298 KB  bioconda&#xA;    mysql-connector-c-6.1.11   |       hccea1a4_1         1.2 MB&#xA;    paml-4.9                   |       hb4d813b_6         1.2 MB  bioconda&#xA;    perl-algorithm-diff-1.1903 | pl5321hdfd78af_3          36 KB  bioconda&#xA;    perl-base-2.23             | pl5321hdfd78af_2          12 KB  bioconda&#xA;    perl-bio-asn1-entrezgene-1.73| pl5321hdfd78af_3          24 KB  bioconda&#xA;    perl-bio-coordinate-1.007001| pl5321hdfd78af_3          36 KB  bioconda&#xA;    perl-bio-db-hts-3.01       | pl5321hc7384eb_4         116 KB  bioconda&#xA;    perl-bio-featureio-1.6.905 | pl5321hdfd78af_4          41 KB  bioconda&#xA;    perl-bio-samtools-1.43     | pl5321h1f540d2_3         161 KB  bioconda&#xA;    perl-bio-tools-phylo-paml-1.7.3| pl5321hdfd78af_3          58 KB  bioconda&#xA;    perl-bio-tools-run-alignment-clustalw-1.7.4| pl5321hdfd78af_3          25 KB  bioconda&#xA;    perl-bio-tools-run-alignment-tcoffee-1.7.4| pl5321hdfd78af_5          25 KB  bioconda&#xA;    perl-bioperl-1.7.8         |       hdfd78af_0           8 KB  bioconda&#xA;    perl-bioperl-core-1.7.8    | pl5321h527b516_0         1.8 MB  bioconda&#xA;    perl-bioperl-run-1.007003  | pl5321hdfd78af_0         340 KB  bioconda&#xA;    perl-business-isbn-3.007   | pl5321hdfd78af_0          18 KB  bioconda&#xA;    perl-business-isbn-data-20210112.006| pl5321hdfd78af_0          21 KB  bioconda&#xA;    perl-capture-tiny-0.48     | pl5321hdfd78af_2          19 KB  bioconda&#xA;    perl-class-data-inheritable-0.09| pl5321hdfd78af_0          10 KB  bioconda&#xA;    perl-constant-1.33         | pl5321hdfd78af_2          12 KB  bioconda&#xA;    perl-data-dumper-2.183     | pl5321ha5712d3_1          38 KB  bioconda&#xA;    perl-dbd-mysql-4.046       | pl5321hcd10b59_4          94 KB  bioconda&#xA;    perl-dbi-1.643             | pl5321ha5712d3_1         602 KB  bioconda&#xA;    perl-devel-stacktrace-2.04 | pl5321hdfd78af_1          17 KB  bioconda&#xA;    perl-digest-hmac-1.04      | pl5321hdfd78af_0          10 KB  bioconda&#xA;    perl-digest-md5-2.58       | pl5321ha5712d3_1          18 KB  bioconda&#xA;    perl-encode-locale-1.05    | pl5321hdfd78af_7          13 KB  bioconda&#xA;    perl-exception-class-1.45  | pl5321hdfd78af_0          21 KB  bioconda&#xA;    perl-file-listing-6.15     | pl5321hdfd78af_0          14 KB  bioconda&#xA;    perl-file-slurp-tiny-0.004 | pl5321hdfd78af_2          10 KB  bioconda&#xA;    perl-file-sort-1.01        | pl5321hdfd78af_3          20 KB  bioconda&#xA;    perl-file-spec-3.48_01     | pl5321hdfd78af_2           6 KB  bioconda&#xA;    perl-getopt-long-2.54      | pl5321hdfd78af_0          31 KB  bioconda&#xA;    perl-html-parser-3.80      | pl5321hb3ea9af_0          67 KB  bioconda&#xA;    perl-html-tagset-3.20      | pl5321hdfd78af_4          14 KB  bioconda&#xA;    perl-http-cookies-6.10     | pl5321hdfd78af_0          22 KB  bioconda&#xA;    perl-http-daemon-6.14      | pl5321hdfd78af_0          20 KB  bioconda&#xA;    perl-http-date-6.05        | pl5321hdfd78af_0          13 KB  bioconda&#xA;    perl-http-message-6.36     | pl5321hdfd78af_0          54 KB  bioconda&#xA;    perl-http-negotiate-6.01   | pl5321hdfd78af_4          14 KB  bioconda&#xA;    perl-io-html-1.004         | pl5321hdfd78af_0          16 KB  bioconda&#xA;    perl-io-socket-ssl-2.074   | pl5321hdfd78af_0         162 KB  bioconda&#xA;    perl-io-string-1.08        | pl5321hdfd78af_4          12 KB  bioconda&#xA;    perl-io-tty-1.16           | pl5321ha5712d3_1          30 KB  bioconda&#xA;    perl-ipc-run-20200505.0    | pl5321hdfd78af_0          76 KB  bioconda&#xA;    perl-libwww-perl-6.67      | pl5321hdfd78af_0          97 KB  bioconda&#xA;    perl-libxml-perl-0.08      | pl5321hdfd78af_3          40 KB  bioconda&#xA;    perl-lwp-mediatypes-6.04   | pl5321hdfd78af_1          24 KB  bioconda&#xA;    perl-mime-base64-3.16      | pl5321ha5712d3_2          16 KB  bioconda&#xA;    perl-net-http-6.22         | pl5321hdfd78af_0          21 KB  bioconda&#xA;    perl-net-ssleay-1.92       | pl5321h5d619eb_1         299 KB  bioconda&#xA;    perl-ntlm-1.09             | pl5321hdfd78af_5          16 KB  bioconda&#xA;    perl-perlio-gzip-0.20      | pl5321h1f540d2_3          16 KB  bioconda&#xA;    perl-sereal-4.019          | pl5321hdfd78af_0          11 KB  bioconda&#xA;    perl-sereal-decoder-4.025  | pl5321ha5712d3_1         156 KB  bioconda&#xA;    perl-sereal-encoder-4.025  | pl5321ha5712d3_1         365 KB  bioconda&#xA;    perl-set-intervaltree-0.12 | pl5321h9722bc1_2          31 KB  bioconda&#xA;    perl-socket-2.027          | pl5321ha5712d3_3          29 KB  bioconda&#xA;    perl-sub-uplevel-0.2800    | pl5321ha5712d3_4          14 KB  bioconda&#xA;    perl-test-deep-1.130       | pl5321hdfd78af_0          37 KB  bioconda&#xA;    perl-test-differences-0.69 | pl5321hdfd78af_0          19 KB  bioconda&#xA;    perl-test-exception-0.43   | pl5321hdfd78af_3          15 KB  bioconda&#xA;    perl-test-harness-3.44     | pl5321hdfd78af_0          85 KB  bioconda&#xA;    perl-test-most-0.38        | pl5321hdfd78af_0          20 KB  bioconda&#xA;    perl-test-warn-0.36        | pl5321hdfd78af_2          14 KB  bioconda&#xA;    perl-text-csv-2.01         | pl5321hdfd78af_0          70 KB  bioconda&#xA;    perl-text-diff-1.45        | pl5321hdfd78af_1          23 KB  bioconda&#xA;    perl-time-local-1.30       | pl5321hdfd78af_0          12 KB  bioconda&#xA;    perl-timedate-2.33         | pl5321hdfd78af_2          31 KB  bioconda&#xA;    perl-tree-dag_node-1.32    | pl5321hdfd78af_0          43 KB  bioconda&#xA;    perl-try-tiny-0.31         | pl5321hdfd78af_1          19 KB  bioconda&#xA;    perl-uri-5.12              | pl5321hdfd78af_0          59 KB  bioconda&#xA;    perl-url-encode-0.03       | pl5321h527b516_0          12 KB  bioconda&#xA;    perl-www-robotrules-6.02   | pl5321hdfd78af_4          14 KB  bioconda&#xA;    perl-xml-dom-1.46          | pl5321hdfd78af_1          59 KB  bioconda&#xA;    perl-xml-dom-xpath-0.14    | pl5321hdfd78af_2          12 KB  bioconda&#xA;    perl-xml-parser-2.44       | pl5321h50c80e2_8         162 KB  bioconda&#xA;    perl-xml-regexp-0.04       | pl5321hdfd78af_3          10 KB  bioconda&#xA;    perl-xml-xpathengine-0.14  | pl5321hdfd78af_3          28 KB  bioconda&#xA;    t-coffee-11.00.8cbe486     |       h26a2512_0         1.1 MB  bioconda&#xA;    unzip-6.0                  |       h3b3adb9_0          71 KB&#xA;    ------------------------------------------------------------&#xA;                                           Total:        28.7 MB&#xA;&#xA;The following NEW packages will be INSTALLED:&#xA;&#xA;  argtable2          conda-forge/osx-64::argtable2-2.13-h1de35cc_1001&#xA;  blast              bioconda/osx-64::blast-2.12.0-h0370960_3&#xA;  clustalo           bioconda/osx-64::clustalo-1.2.4-h9722bc1_5&#xA;  clustalw           bioconda/osx-64::clustalw-2.1-hcd10b59_7&#xA;  ensembl-vep        bioconda/noarch::ensembl-vep-108.2-pl5321h4a94de4_0&#xA;  entrez-direct      bioconda/osx-64::entrez-direct-16.2-h193322a_1&#xA;  expat              pkgs/main/osx-64::expat-2.4.9-he9d5cce_0&#xA;  htslib             bioconda/osx-64::htslib-1.14-hc38c3fb_0&#xA;  libdeflate         pkgs/main/osx-64::libdeflate-1.7-h9ed2024_5&#xA;  libgfortran        pkgs/main/osx-64::libgfortran-5.0.0-11_3_0_hecd8cb5_28&#xA;  libgfortran5       pkgs/main/osx-64::libgfortran5-11.3.0-h9dfd629_28&#xA;  libidn2            pkgs/main/osx-64::libidn2-2.3.2-h9ed2024_0&#xA;  libunistring       pkgs/main/osx-64::libunistring-0.9.10-h9ed2024_0&#xA;  libzlib            conda-forge/osx-64::libzlib-1.2.12-hfd90126_3&#xA;  llvm-openmp        pkgs/main/osx-64::llvm-openmp-14.0.6-h0dcd299_0&#xA;  mafft              bioconda/osx-64::mafft-7.508-ha5712d3_0&#xA;  muscle             bioconda/osx-64::muscle-5.1-hb339e23_1&#xA;  mysql-connector-c  pkgs/main/osx-64::mysql-connector-c-6.1.11-hccea1a4_1&#xA;  paml               bioconda/osx-64::paml-4.9-hb4d813b_6&#xA;  pcre               pkgs/main/osx-64::pcre-8.45-h23ab428_0&#xA;  perl               pkgs/main/osx-64::perl-5.32.1-0_h435f0c2_perl5&#xA;  perl-algorithm-di~ bioconda/noarch::perl-algorithm-diff-1.1903-pl5321hdfd78af_3&#xA;  perl-archive-tar   bioconda/noarch::perl-archive-tar-2.40-pl5321hdfd78af_0&#xA;  perl-base          bioconda/noarch::perl-base-2.23-pl5321hdfd78af_2&#xA;  perl-bio-asn1-ent~ bioconda/noarch::perl-bio-asn1-entrezgene-1.73-pl5321hdfd78af_3&#xA;  perl-bio-coordina~ bioconda/noarch::perl-bio-coordinate-1.007001-pl5321hdfd78af_3&#xA;  perl-bio-db-hts    bioconda/osx-64::perl-bio-db-hts-3.01-pl5321hc7384eb_4&#xA;  perl-bio-featureio bioconda/noarch::perl-bio-featureio-1.6.905-pl5321hdfd78af_4&#xA;  perl-bio-samtools  bioconda/osx-64::perl-bio-samtools-1.43-pl5321h1f540d2_3&#xA;  perl-bio-tools-ph~ bioconda/noarch::perl-bio-tools-phylo-paml-1.7.3-pl5321hdfd78af_3&#xA;  perl-bio-tools-ru~ bioconda/noarch::perl-bio-tools-run-alignment-clustalw-1.7.4-pl5321hdfd78af_3&#xA;  perl-bio-tools-ru~ bioconda/noarch::perl-bio-tools-run-alignment-tcoffee-1.7.4-pl5321hdfd78af_5&#xA;  perl-bioperl       bioconda/noarch::perl-bioperl-1.7.8-hdfd78af_0&#xA;  perl-bioperl-core  bioconda/osx-64::perl-bioperl-core-1.7.8-pl5321h527b516_0&#xA;  perl-bioperl-run   bioconda/noarch::perl-bioperl-run-1.007003-pl5321hdfd78af_0&#xA;  perl-business-isbn bioconda/noarch::perl-business-isbn-3.007-pl5321hdfd78af_0&#xA;  perl-business-isb~ bioconda/noarch::perl-business-isbn-data-20210112.006-pl5321hdfd78af_0&#xA;  perl-capture-tiny  bioconda/noarch::perl-capture-tiny-0.48-pl5321hdfd78af_2&#xA;  perl-carp          bioconda/noarch::perl-carp-1.38-pl5321hdfd78af_4&#xA;  perl-class-data-i~ bioconda/noarch::perl-class-data-inheritable-0.09-pl5321hdfd78af_0&#xA;  perl-common-sense  bioconda/noarch::perl-common-sense-3.75-pl5321hdfd78af_0&#xA;  perl-compress-raw~ bioconda/osx-64::perl-compress-raw-bzip2-2.201-pl5321h9722bc1_0&#xA;  perl-compress-raw~ bioconda/osx-64::perl-compress-raw-zlib-2.105-pl5321h9722bc1_0&#xA;  perl-constant      bioconda/noarch::perl-constant-1.33-pl5321hdfd78af_2&#xA;  perl-data-dumper   bioconda/osx-64::perl-data-dumper-2.183-pl5321ha5712d3_1&#xA;  perl-dbd-mysql     bioconda/osx-64::perl-dbd-mysql-4.046-pl5321hcd10b59_4&#xA;  perl-dbi           bioconda/osx-64::perl-dbi-1.643-pl5321ha5712d3_1&#xA;  perl-devel-stackt~ bioconda/noarch::perl-devel-stacktrace-2.04-pl5321hdfd78af_1&#xA;  perl-digest-hmac   bioconda/noarch::perl-digest-hmac-1.04-pl5321hdfd78af_0&#xA;  perl-digest-md5    bioconda/osx-64::perl-digest-md5-2.58-pl5321ha5712d3_1&#xA;  perl-encode        bioconda/osx-64::perl-encode-3.19-pl5321ha5712d3_1&#xA;  perl-encode-locale bioconda/noarch::perl-encode-locale-1.05-pl5321hdfd78af_7&#xA;  perl-exception-cl~ bioconda/noarch::perl-exception-class-1.45-pl5321hdfd78af_0&#xA;  perl-exporter      bioconda/noarch::perl-exporter-5.72-pl5321hdfd78af_2&#xA;  perl-exporter-tiny bioconda/noarch::perl-exporter-tiny-1.002002-pl5321hdfd78af_0&#xA;  perl-extutils-mak~ conda-forge/noarch::perl-extutils-makemaker-7.64-pl5321hd8ed1ab_0&#xA;  perl-file-listing  bioconda/noarch::perl-file-listing-6.15-pl5321hdfd78af_0&#xA;  perl-file-slurp-t~ bioconda/noarch::perl-file-slurp-tiny-0.004-pl5321hdfd78af_2&#xA;  perl-file-sort     bioconda/noarch::perl-file-sort-1.01-pl5321hdfd78af_3&#xA;  perl-file-spec     bioconda/noarch::perl-file-spec-3.48_01-pl5321hdfd78af_2&#xA;  perl-getopt-long   bioconda/noarch::perl-getopt-long-2.54-pl5321hdfd78af_0&#xA;  perl-html-parser   bioconda/osx-64::perl-html-parser-3.80-pl5321hb3ea9af_0&#xA;  perl-html-tagset   bioconda/noarch::perl-html-tagset-3.20-pl5321hdfd78af_4&#xA;  perl-http-cookies  bioconda/noarch::perl-http-cookies-6.10-pl5321hdfd78af_0&#xA;  perl-http-daemon   bioconda/noarch::perl-http-daemon-6.14-pl5321hdfd78af_0&#xA;  perl-http-date     bioconda/noarch::perl-http-date-6.05-pl5321hdfd78af_0&#xA;  perl-http-message  bioconda/noarch::perl-http-message-6.36-pl5321hdfd78af_0&#xA;  perl-http-negotia~ bioconda/noarch::perl-http-negotiate-6.01-pl5321hdfd78af_4&#xA;  perl-io-compress   bioconda/osx-64::perl-io-compress-2.201-pl5321h9722bc1_0&#xA;  perl-io-html       bioconda/noarch::perl-io-html-1.004-pl5321hdfd78af_0&#xA;  perl-io-socket-ssl bioconda/noarch::perl-io-socket-ssl-2.074-pl5321hdfd78af_0&#xA;  perl-io-string     bioconda/noarch::perl-io-string-1.08-pl5321hdfd78af_4&#xA;  perl-io-tty        bioconda/osx-64::perl-io-tty-1.16-pl5321ha5712d3_1&#xA;  perl-io-zlib       bioconda/noarch::perl-io-zlib-1.11-pl5321hdfd78af_0&#xA;  perl-ipc-run       bioconda/noarch::perl-ipc-run-20200505.0-pl5321hdfd78af_0&#xA;  perl-json          bioconda/noarch::perl-json-4.10-pl5321hdfd78af_0&#xA;  perl-json-xs       bioconda/osx-64::perl-json-xs-2.34-pl5321hcd10b59_5&#xA;  perl-libwww-perl   bioconda/noarch::perl-libwww-perl-6.67-pl5321hdfd78af_0&#xA;  perl-libxml-perl   bioconda/noarch::perl-libxml-perl-0.08-pl5321hdfd78af_3&#xA;  perl-list-moreuti~ bioconda/noarch::perl-list-moreutils-0.430-pl5321hdfd78af_0&#xA;  perl-list-moreuti~ bioconda/osx-64::perl-list-moreutils-xs-0.430-pl5321ha5712d3_1&#xA;  perl-lwp-mediatyp~ bioconda/noarch::perl-lwp-mediatypes-6.04-pl5321hdfd78af_1&#xA;  perl-mime-base64   bioconda/osx-64::perl-mime-base64-3.16-pl5321ha5712d3_2&#xA;  perl-net-http      bioconda/noarch::perl-net-http-6.22-pl5321hdfd78af_0&#xA;  perl-net-ssleay    bioconda/osx-64::perl-net-ssleay-1.92-pl5321h5d619eb_1&#xA;  perl-ntlm          bioconda/noarch::perl-ntlm-1.09-pl5321hdfd78af_5&#xA;  perl-parent        bioconda/noarch::perl-parent-0.236-pl5321hdfd78af_2&#xA;  perl-pathtools     bioconda/osx-64::perl-pathtools-3.75-pl5321ha5712d3_3&#xA;  perl-perlio-gzip   bioconda/osx-64::perl-perlio-gzip-0.20-pl5321h1f540d2_3&#xA;  perl-scalar-list-~ bioconda/osx-64::perl-scalar-list-utils-1.62-pl5321ha5712d3_0&#xA;  perl-sereal        bioconda/noarch::perl-sereal-4.019-pl5321hdfd78af_0&#xA;  perl-sereal-decod~ bioconda/osx-64::perl-sereal-decoder-4.025-pl5321ha5712d3_1&#xA;  perl-sereal-encod~ bioconda/osx-64::perl-sereal-encoder-4.025-pl5321ha5712d3_1&#xA;  perl-set-interval~ bioconda/osx-64::perl-set-intervaltree-0.12-pl5321h9722bc1_2&#xA;  perl-socket        bioconda/osx-64::perl-socket-2.027-pl5321ha5712d3_3&#xA;  perl-sub-uplevel   bioconda/osx-64::perl-sub-uplevel-0.2800-pl5321ha5712d3_4&#xA;  perl-test-deep     bioconda/noarch::perl-test-deep-1.130-pl5321hdfd78af_0&#xA;  perl-test-differe~ bioconda/noarch::perl-test-differences-0.69-pl5321hdfd78af_0&#xA;  perl-test-excepti~ bioconda/noarch::perl-test-exception-0.43-pl5321hdfd78af_3&#xA;  perl-test-harness  bioconda/noarch::perl-test-harness-3.44-pl5321hdfd78af_0&#xA;  perl-test-most     bioconda/noarch::perl-test-most-0.38-pl5321hdfd78af_0&#xA;  perl-test-warn     bioconda/noarch::perl-test-warn-0.36-pl5321hdfd78af_2&#xA;  perl-text-csv      bioconda/noarch::perl-text-csv-2.01-pl5321hdfd78af_0&#xA;  perl-text-diff     bioconda/noarch::perl-text-diff-1.45-pl5321hdfd78af_1&#xA;  perl-time-local    bioconda/noarch::perl-time-local-1.30-pl5321hdfd78af_0&#xA;  perl-timedate      bioconda/noarch::perl-timedate-2.33-pl5321hdfd78af_2&#xA;  perl-tree-dag_node bioconda/noarch::perl-tree-dag_node-1.32-pl5321hdfd78af_0&#xA;  perl-try-tiny      bioconda/noarch::perl-try-tiny-0.31-pl5321hdfd78af_1&#xA;  perl-types-serial~ bioconda/noarch::perl-types-serialiser-1.01-pl5321hdfd78af_0&#xA;  perl-uri           bioconda/noarch::perl-uri-5.12-pl5321hdfd78af_0&#xA;  perl-url-encode    bioconda/osx-64::perl-url-encode-0.03-pl5321h527b516_0&#xA;  perl-www-robotrul~ bioconda/noarch::perl-www-robotrules-6.02-pl5321hdfd78af_4&#xA;  perl-xml-dom       bioconda/noarch::perl-xml-dom-1.46-pl5321hdfd78af_1&#xA;  perl-xml-dom-xpath bioconda/noarch::perl-xml-dom-xpath-0.14-pl5321hdfd78af_2&#xA;  perl-xml-parser    bioconda/osx-64::perl-xml-parser-2.44-pl5321h50c80e2_8&#xA;  perl-xml-regexp    bioconda/noarch::perl-xml-regexp-0.04-pl5321hdfd78af_3&#xA;  perl-xml-xpatheng~ bioconda/noarch::perl-xml-xpathengine-0.14-pl5321hdfd78af_3&#xA;  t-coffee           bioconda/osx-64::t-coffee-11.00.8cbe486-h26a2512_0&#xA;  unzip              pkgs/main/osx-64::unzip-6.0-h3b3adb9_0&#xA;  wget               pkgs/main/osx-64::wget-1.21.3-h6dfd666_0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="1157" LastEditDate="2022-12-30T18:16:38.280" LastActivityDate="2022-12-30T18:16:38.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20194" PostTypeId="2" ParentId="20192" CreationDate="2022-12-16T06:02:51.000" Score="1" Body="&lt;p&gt;VEP requires &lt;code&gt;gcc&lt;/code&gt; and &lt;code&gt;g++&lt;/code&gt; for installation&lt;sup&gt;&lt;a href=&quot;https://github.com/Ensembl/ensembl-vep#installation-and-requirements&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;&lt;/sup&gt;. Before running the build script, try setting the &lt;code&gt;CC&lt;/code&gt; and &lt;code&gt;CXX&lt;/code&gt; environment variables to the paths of your &lt;code&gt;gcc&lt;/code&gt; and &lt;code&gt;g++&lt;/code&gt; compilers, respectively:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;export CC=gcc&#xA;export CXX=g++&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You'll need to install these compilers if they don't already exist on your system.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2022-12-16T06:02:51.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20195" PostTypeId="2" ParentId="19860" CreationDate="2022-12-16T07:29:18.423" Score="2" Body="&lt;p&gt;The violin plots show only little expression in non-podocyte clusters, but the podocyte marker transcripts are only the most obvious manifestation of this problem. Also, small but significant amounts of endothelial and other celltype transcripts show up in most of the other clusters. Identification of the celltype works fine but these contaminating counts throw off most of the downstream analyses.&lt;/p&gt;&#xA;&lt;p&gt;By now, we did manage to find a solution for the problem, though: It turns out the problem was due to high amounts of ambient RNA contamination. I tried three different packages to remove the contaminating counts (DIEM, DecontX and SoupX) and found that DecontX (and SoupX) worked best for my dataset.&lt;/p&gt;&#xA;" OwnerUserId="16332" LastActivityDate="2022-12-16T07:29:18.423" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20196" PostTypeId="1" AcceptedAnswerId="20199" CreationDate="2022-12-16T14:17:48.100" Score="1" ViewCount="508" Body="&lt;p&gt;HG38 human reference is the collection of reference from all 22 chromosomes,x,y chromosomes, and some others. I was targeting to split the reference file into 22 chromosomes fasta files. I am following this link &lt;a href=&quot;https://crashcourse.housegordon.org/split-fasta-files.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://crashcourse.housegordon.org/split-fasta-files.html&lt;/a&gt; to split the fasta files.More specifically I want to split the fasta file into all possible (placed,unlocalized and unplaced) sequence and the merge placed and localized sequences.&#xA;However, I am unsure where to put the HLA part of the reference. Should I put it in Chromosome 6? Also, should I dump the unknown chromosomes as I am specifically interested in chromosomes 1-22?&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion on splitting the fasta file into 22 chromosomes would be nice.&lt;/p&gt;&#xA;" OwnerUserId="10296" LastEditorUserId="10296" LastEditDate="2022-12-18T20:47:47.037" LastActivityDate="2022-12-18T20:47:47.037" Title="split fast reference fast file by 22 chromosome" Tags="|fasta|reference-genome|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20197" PostTypeId="1" AcceptedAnswerId="20198" CreationDate="2022-12-16T21:15:25.510" Score="1" ViewCount="918" Body="&lt;p&gt;I have some .bed files (and .bim and .fam files) containing data for a number of different samples, and I need to convert them to .vcf.&lt;/p&gt;&#xA;&lt;p&gt;I've found &lt;a href=&quot;https://www.rdocumentation.org/packages/bedr/versions/1.0.7/topics/bed2vcf&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;bed2vcf&lt;/code&gt; which is from the &lt;code&gt;bedr&lt;/code&gt; R package&lt;/a&gt;, and also available as a stand-alone python program of the same name, but it requires a .fasta file corresponding to the reference genome used to create the .bed file. So we need to know what genome reference build was used to get the right &lt;code&gt;.fasta&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; Is there any tool or way to inspect the files I have and determine what genome reference build is required to do a correct conversion to .vcf, or is there any other method?&lt;/p&gt;&#xA;&lt;p&gt;There is a &lt;code&gt;bed2vcf&lt;/code&gt; related post &lt;a href=&quot;https://stackoverflow.com/questions/63306507/convert-bed-file-to-vcf-with-bed2vcf-function&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16652" LastEditorUserId="16652" LastEditDate="2022-12-16T21:48:32.573" LastActivityDate="2022-12-18T11:06:56.587" Title="Need to convert .bed to .vcf. Can the reference build (and necessary .fasta) be determined for a .bed file?" Tags="|ngs|fasta|vcf|bed|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20198" PostTypeId="2" ParentId="20197" CreationDate="2022-12-17T02:13:38.100" Score="2" Body="&lt;p&gt;There are two different kinds of &lt;code&gt;.bed&lt;/code&gt; files in bioinformatics; one related to storing &lt;a href=&quot;https://en.wikipedia.org/wiki/BED_(file_format)&quot; rel=&quot;nofollow noreferrer&quot;&gt;genomic coordinate data&lt;/a&gt; and one which is part of the &lt;a href=&quot;https://www.cog-genomics.org/plink/1.9/formats#bed&quot; rel=&quot;nofollow noreferrer&quot;&gt;plink binary dataset&lt;/a&gt;, so it's important to check which kind of file you are actually converting. If you have &lt;code&gt;.bed&lt;/code&gt;, &lt;code&gt;.bim&lt;/code&gt; and &lt;code&gt;.fam&lt;/code&gt; files, you could use &lt;a href=&quot;https://www.cog-genomics.org/plink/2.0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PLINK 2.0&lt;/a&gt; to export them as VCF:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plink2 --bfile [prefix] --export vcf --out &amp;lt;prefix&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The relevant options from the docs:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;...&amp;gt;&#xA;&#xA;  --bfile  &amp;lt;prefix&amp;gt; ['vzs'] : Specify .bed + .bim[.zst] + .fam prefix&#xA;&#xA;&amp;lt;...&amp;gt;&#xA;&#xA;  --out &amp;lt;prefix&amp;gt;     : Specify prefix for output files.&#xA;&#xA;&amp;lt;...&amp;gt;&#xA;&#xA;  --export &amp;lt;output format(s)...&amp;gt; [{01 | 12}] ['bgz'] ['id-delim='&amp;lt;char&amp;gt;]&#xA;           ['id-paste='&amp;lt;column set descriptor&amp;gt;] ['include-alt']&#xA;           ['omit-nonmale-y'] ['spaces'] ['vcf-dosage='&amp;lt;field&amp;gt;] ['ref-first']&#xA;           ['bits='&amp;lt;#&amp;gt;] ['sample-v2']&#xA;    Create a new fileset with all filters applied.  The following output&#xA;    formats are supported:&#xA;&#xA;...&#xA;&#xA;    * 'vcf',     : VCF (default version 4.3).  If PAR1 and PAR2 are present,&#xA;      'vcf-4.2',   they are automatically merged with chrX, with proper&#xA;      'bcf',       handling of chromosome codes and male ploidy.&#xA;      'bcf-4.2'    When the 'bgz' modifier is present, the VCF file is&#xA;                   block-gzipped.  (This always happens with BCF output.)&#xA;                   The 'id-paste' modifier controls which .psam columns are&#xA;                   used to construct sample IDs (choices are maybefid, fid,&#xA;                   iid, maybesid, and sid; default is maybefid,iid,maybesid),&#xA;                   while the 'id-delim' modifier sets the character between the&#xA;                   ID pieces (default '_').&#xA;&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2022-12-18T11:06:56.587" LastActivityDate="2022-12-18T11:06:56.587" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20199" PostTypeId="2" ParentId="20196" CreationDate="2022-12-17T07:02:08.757" Score="2" Body="&lt;p&gt;The HLA region is already a part of chromosome 6. When you split the file, it will just be included with the rest of the chromosome. Whether or not you include the remaining contigs depends on your specific objectives. Inclusion of the unplaced and unlocalized scaffolds usually helps to reduce mapping errors, for example. To split/extract &lt;strong&gt;only&lt;/strong&gt; chr1-22 from the UCSC &lt;a href=&quot;http://hgdownload.cse.ucsc.edu/goldenPath/hg38/bigZips/hg38.fa.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;hg38.fa.gz&lt;/a&gt;, we can just use &lt;a href=&quot;/questions/tagged/awk&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; aria-label=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; rel=&quot;tag&quot; aria-labelledby=&quot;awk-container&quot;&gt;awk&lt;/a&gt;. With the following in a file called &lt;code&gt;script.awk&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BEGIN {&#xA;    for(i=1;i&amp;lt;=22;i++) {&#xA;        arr[&amp;quot;chr&amp;quot; i]&#xA;    }&#xA;}&#xA;&#xA;/^&amp;gt;/ {&#xA;    f=substr($0,2)&#xA;}&#xA;&#xA;f in arr {&#xA;    print &amp;gt; f &amp;quot;.fa&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Run using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk -f script.awk &amp;lt;(zcat hg38.fa.gz)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2022-12-17T07:02:08.757" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20200" PostTypeId="2" ParentId="20120" CreationDate="2022-12-17T17:58:05.877" Score="1" Body="&lt;p&gt;[Not quite an answer but too long for a comment]&lt;/p&gt;&#xA;&lt;p&gt;I am guessing that your assembly &lt;code&gt;Big_burk_assembly.fasta&lt;/code&gt; is a rather large assembly. Assembly indexing can be very memory intensive for complex assemblies. If it is eating enough memory to kill the bowtie2 indexing process (as noted in comment by @terdon), I'd consider filtering it, for example if there are lots of very short contigs that are unlikely to be meaningful, consider removing those.&lt;/p&gt;&#xA;&lt;p&gt;Just looking at the way you've named the assembly file, it looks like it might be a somewhat ad hoc assembly, so maybe there is a meaningful way to filter it. We might be able to suggest more disciplined ways to do this if you tell us more about it.&lt;/p&gt;&#xA;&lt;p&gt;Alternately, just get more memory as suggested in comments by @terdon. This will be hard if you are running on a personal computer, but on a cluster you can usually specify this in a job parameter file. &lt;a href=&quot;https://ecs.rutgers.edu/slurm_commands.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; (section &amp;quot;submit scripts&amp;quot;) is how to set it on SLURM, for example.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2022-12-18T04:54:23.897" LastActivityDate="2022-12-18T04:54:23.897" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20201" PostTypeId="2" ParentId="3680" CreationDate="2022-12-18T05:12:46.263" Score="1" Body="&lt;p&gt;Since this is an old unanswered question where a lot of method development has since happened, going to put some answers here:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.nature.com/articles/ncomms11307&quot; rel=&quot;nofollow noreferrer&quot;&gt;This rather old tool&lt;/a&gt; also mentioned in comments (graphmap) is supposedly good for this, though it may not be relevant anymore.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7814537/&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; uses minimap (not minimap2) alignments as a heuristic for read quality, suggesting that minimap2 should be good enough.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Note that depending on how you have processed (specifically basecalled) the data, 1d2 &amp;quot;second&amp;quot; reads &lt;a href=&quot;https://gigabaseorgigabyte.wordpress.com/2020/04/18/removing-accidental-2d-1d2-reads-from-an-alignment/&quot; rel=&quot;nofollow noreferrer&quot;&gt;may have low accuracy&lt;/a&gt;. That matches your ~50% unaligned proportion. You may need to do additional work during basecalling to fully use 1d2, as discussed in &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-020-02255-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Note: I am not an expert on nanopore and I put this answer together using google. Others are welcome to edit/editorialize.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-12-18T05:12:46.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20202" PostTypeId="2" ParentId="5026" CreationDate="2022-12-18T05:35:10.600" Score="1" Body="&lt;p&gt;Based on &lt;a href=&quot;https://groups.google.com/a/soe.ucsc.edu/g/genome/c/ffskDdFf6gA&quot; rel=&quot;nofollow noreferrer&quot;&gt;this old answer&lt;/a&gt;, this is not recommended for liftOver. For between-species liftovers, it is claimed that you are likely to want a different tool such as pslMap. For a lot of rather detailed discussion of how liftOver operates in this context and how to parameterize properly, see &lt;a href=&quot;https://genome.soe.ucsc.narkive.com/OtNLdT6G/advanced-liftover-questions&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;See also &lt;a href=&quot;https://www.biostars.org/p/70648/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this old Biostars question&lt;/a&gt; addressing the issue of mapping not between versions of the same assemblies but different genomes entirely. This thread also notes that using liftOver can be dangerous. That isn't to say that you can't learn something interesting, just that you might make mistakes.&lt;/p&gt;&#xA;&lt;p&gt;pslMap apparently drops non-syntenic regions, which may lose a lot of data, but it is supposedly making rather fewer assumptions about assembly relationships.&lt;/p&gt;&#xA;&lt;p&gt;orthoMap is apparently a similar tool to pslMap, but I can't find a way to run it except for a very old and restricted web interface. There are other tools with very similar names (&amp;quot;&lt;a href=&quot;https://github.com/kullrich/orthomap&quot; rel=&quot;nofollow noreferrer&quot;&gt;orthomap&lt;/a&gt;&amp;quot;) but somewhat different uses, which may increase confusion.&lt;/p&gt;&#xA;&lt;p&gt;A counterpoint to all the foregoing is that &lt;a href=&quot;https://genviz.org/module-01-intro/0001/06/02/liftoverTools/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this rather detailed sum-up of liftover tools&lt;/a&gt; does not seem to be very concerned about the issues of using tools like liftOver between species. &lt;strong&gt;It specifically uses dog to human liftover as the example, just as you want to&lt;/strong&gt;. It's notable that the assumption is made in that page that a high-quality chain file exists, which does seem to be true for you.&lt;/p&gt;&#xA;&lt;p&gt;To go through your questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;depends, see above.&lt;/li&gt;&#xA;&lt;li&gt;For suggested parameters for running blat/chain generation/liftOver for this application, see &lt;a href=&quot;https://genome.soe.ucsc.narkive.com/OtNLdT6G/advanced-liftover-questions&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;This is derived in part from the [chain file] I believe rather than liftOver parameters (&lt;a href=&quot;http://genomewiki.ucsc.edu/index.php/LiftOver_Howto&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://genomewiki.ucsc.edu/index.php/LiftOver_Howto&lt;/a&gt;). However overall this is set by the parameters you used in generating a chain file, which probably boils down to parameters of the workflow &lt;a href=&quot;http://genomewiki.ucsc.edu/index.php/LiftOver_Howto#:%7E:text=A%20liftOver%20file%20is%20a,of%20a%20genome%20to%20another.&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. liftOver itself is just relying on the chain alignments for each region with the best score.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2022-12-19T19:58:31.240" LastActivityDate="2022-12-19T19:58:31.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20203" PostTypeId="2" ParentId="7331" CreationDate="2022-12-18T05:57:40.153" Score="1" Body="&lt;p&gt;[Old question, but possibly still relevant to answer.]&lt;/p&gt;&#xA;&lt;p&gt;This will not necessarily be rare. One or the other of the mates could span the ligation junction itself, in which case your read start and end could be entirely different chromosomes. In fact, &lt;a href=&quot;https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1008839&quot; rel=&quot;nofollow noreferrer&quot;&gt;some tools&lt;/a&gt; specifically measure the quality of Hi-C libraries by measuring the number of such ligations directly observable within reads. This will be a much larger number of reads than bin-crossing reads.&lt;/p&gt;&#xA;&lt;p&gt;The safest thing to do in all such cases (IMO) is to take the read start position as the location of interest for each mate. That way, if a junction does occur, you still have an unambiguous position which is your best guess of where each end of the ligation product actually originates. This gets around both the minor issue of the bin-crossing problem and the major issue of the ligation junction problem. There is probably some imprecision here and you throw out some information, but sorting out the complex possible situations otherwise seems like a lot of hassle for minimal gain.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-12-18T05:57:40.153" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20204" PostTypeId="1" AcceptedAnswerId="20205" CreationDate="2022-12-18T11:17:21.787" Score="3" ViewCount="111" Body="&lt;p&gt;I want to use Arlequin for AMOVA. I have my FASTA files which I want to convert into .arp files which are recognizable by arlequin. When I try to convert it using Population Genetics Data (format) Spider, or PGDSpider by &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/28/2/298/198891?login=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lisher and Excoffier (2018)&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The error message says the name of my populations is missing.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone tell me what's wrong with my FASTA files? Check the following example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;CcHr1 &#xA;ATAAAGATATTGGTACGTTGTATATATTATTGGGTGTGTGGTGTGGTATAGTTGGTACAGGATTATCATT&#xA;ACTAATTCGTCTTGAGTTAGGTACTTCTGGTGTCTTGACTGATGAGCATTTTTATAATGTAATTGTAACA&#xA;GCCCACGCTTTTGTAATGATTTTTTTTATGGTTATACCGATTATAATTGGTGGTTTTGGAAATTGAATAG&#xA;TTCCTTTATTAATTGGTGCTCCAGACATAAGGTTCCCTCGGATAAATAATATAAGATTTTGATTATTACC&#xA;TCCTGCTTTTGTACTACTGATTAGTAGAAGCCTTGTTGAGGGAGGGGCTGGGACTGGCTGAACTGTTTAT&#xA;CCTCCACTTAGGAGGGTAACAGGACATAGTGGGGCTTCTGTAGACTTGGCTATTTTTTCCTTACATCTCG&#xA;CGGGCATATCATCAATTTTGGGGGCAATTAATTTTATTACAACGATCTTCAATATGCGGTCCCCTGGGTT&#xA;AACACTCGAGCGGATAAGGCTATTTGTATGGTCTATTTTAGTAACAGTTTTTTTATTGTTACTATCACTT&#xA;CCAGTTCTTGCAGGGGCAATTACGATGCTCTTAACTGATCGTAATTTTAGTACTTCATTTTTTGATCCAG&#xA;&amp;gt;CcMath1  &#xA;ATAAAGATATTGGTACGTTATATATATTATTGGGTGTGTGGTGTGGAATAGTTGGCACAGGGTTGTCATT&#xA;ATTAATTCGTCTTGAGTTAGGTACTTCTGGTATCTTGACTGATGAGCATTTTTATAACGTAATTGTAACA&#xA;GCCCACGCTTTTGTAATAATTTTTTTTATGGTTATGCCAATTATAATTGGTGGTTTTGGAAATTGAATAG&#xA;TCCCTTTATTAATTGGTGCTCCAGACATAAGGTTCCCTCGAATAAATAATATAAGATTTTGACTATTACC&#xA;TCCTGCGTTTGTACTGCTGATTAGTAGAAGCCTTGTTGAGGGAGGGGCTGGAACTGGCTGAACTGTTTAT&#xA;CCTCCACTTAGAAGGGTAACAGGGCATAGTGGGGCTTCTGTAGACTTGGCGATCTTTTCCTTGCATCTCG&#xA;CGGGTATATCATCAATTTTGGGGGCAATTAATTTTATTACAACGATCTTCAATATGCGGTCCCCTGGGTT&#xA;AACACTCGAACGGATAAGGTTATTTGTATGGTCTATTTTAGTAACAGTTTTTTTATTGTTATTGTCACTT&#xA;CCAGTTCTTGCAGGGGCTATTACGATACTCTTAACTGATCGTAATTTTAGTACTTCGTTTTTTGACCCAG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see I have to populations but it only names it as pop1.&lt;/p&gt;&#xA;" OwnerUserId="16657" LastEditorUserId="3848" LastEditDate="2022-12-18T17:04:20.563" LastActivityDate="2022-12-18T17:04:20.563" Title="How can I resolve &quot;missing name of population&quot; message in PGDSpider?" Tags="|phylogenetics|fasta|phylogeny|populations|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20205" PostTypeId="2" ParentId="20204" CreationDate="2022-12-18T12:01:18.700" Score="2" Body="&lt;p&gt;You could try specifying the &lt;code&gt;population&lt;/code&gt; keyword in the record header. From the &lt;a href=&quot;http://www.cmpg.unibe.ch/software/PGDSpider/PGDSpider%20manual_vers%202-1-1-5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PGDSpider manual (version 2.1.1.5 (May 2018))&lt;/a&gt;, at the bottom of page 67:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Population definition in header line: String after “population:” until&#xA;the first space is parsed as population name e.g.: &lt;code&gt;&amp;gt;sample1 population:pop1 gene:HMA4&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2022-12-18T12:01:18.700" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20206" PostTypeId="2" ParentId="20204" CreationDate="2022-12-18T13:02:14.760" Score="1" Body="&lt;p&gt;It's important to understand what Arlequin is doing for MANOVA. It will compare two or more populations to assess whether there is a difference in the allele frequency between the populations. It will also assess &amp;quot;interaction&amp;quot; (interpreting that in &amp;quot;gene flow&amp;quot; needs some thinking).&lt;/p&gt;&#xA;&lt;p&gt;The basic theory is that if two or more populations cannot be distinguished under a formal parametric test there must be very strong gene flow supporting the lack of population structure between these populations.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Geographic foci&lt;/strong&gt; Population designation is usually done between geographic foci, thus population A is usually geographic location (focus) A and population B is a geographically separated region (focus). Thus the fasta header needs to know the geographic region of the sample. Thats it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Population info&lt;/strong&gt; The issue is that information is required within the fasta header. Thus population delineation is therefore essential. This is confirmed in the PGDSpider manual below the line for Arlequin: which you don't need to do because thats what the conversion does.&lt;/p&gt;&#xA;&lt;p&gt;Arlequin therefore needs to group each member of the population into a (usually geographically seperated) population to perform MANOVA. It will require this information specifying on the fasta header line. At the moment the data is saying there is a single population, it thinks all the sequences are from the same geographic location.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Procedure&lt;/strong&gt; Thus it's likely simply a case of assigning two or more broad geographic foci for your samples. Then going through the fasta file and designating each sequence into one of the geographic focus under the keyword &amp;quot;population&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Just modifying @Steve's reading of the manual&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; CcHr1 population:geoFocusA gene:X&#xA;ATAAAGATATTGGTACGTTGTATATATTATTGGGTGTGTGGTGTGGTATAGTTGGTACAGGATTATCATT&#xA;&#xA;&amp;gt;CcMath1 population:geoFocusB gene:X&#xA; ATAAAGATATTGGTACGTTATATATATTATTGGGTGTGTGGTGTGGAATAGTTGGCACAGGGTTGTCATT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then run PGDSpider&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -Xmx1024m -Xms512m -jar PGDSpider2-cli.jar -inputfile examples\example_Structure.fasta -inputformat FASTA -outputfile examples\output_Arlequin.arp -outputformat ARLEQUIN -spid examples\Structure_Arlequin.spid&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You did attain the critical anomaly in the sequence format,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;each line of a sequence should have fewer than 80 characters&lt;/em&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Your rows have 70 residues.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Genetic structure (only required for AMOVA):&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The genetic structure specifies the hierarchical genetic structure of the samples. It is possible to define groups of populations.&#xA;o start of the subsection: [[Structure]]&#xA;o name for the genetic structure (string within ” ”):  StructureName = “name”&#xA;o number of groups defined in the structure (int value): NbGroups = 5&#xA;o group definitions (list containing the names of the samples belonging to the group,&#xA;entered within braces “{ }”):&#xA;Id1 2 ACTCGGGTTCGCGCGC # the first pseudo-haplotype ACTCGGGCTCACGCGC # the second pseudo-haplotype&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;NbGroups=2&#xA;Group ={&#xA;  population1&#xA;}&#xA;Group ={&#xA;  population2&#xA;  population3&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Advantage&lt;/strong&gt; The advantage of MANOVA is you can have LOADS of populations and it will assess which have high levels of gene flow between them. Statistically it is fantastic for loads of different reasons (quite complicated reasons). The disadvantage of output of MANOVA is there will likely be  significant differences between many of the populations - possibly all of them. This is a result of the power of parametric statistics, but may not be particularly helpful for the analysis. If that is the outcome switching to &lt;em&gt;Fst&lt;/em&gt; is the next part of the analysis (and a different question). Goodluck and you can always post back if you get stuck.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-12-18T13:25:00.740" LastActivityDate="2022-12-18T13:25:00.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20207" PostTypeId="1" CreationDate="2022-12-19T08:30:21.700" Score="3" ViewCount="421" Body="&lt;p&gt;I read this paper (&lt;em&gt;Colletotrichum shisoi sp. nov., an anthracnose pathogen of Perilla frutescens in Japan: molecular phylogenetic, morphological and genomic evidence&lt;/em&gt;). We ran Orthofinder and also created an Upset plot. Unfortunately, I do not understand how they calculated the following percentages in the &lt;em&gt;&amp;quot;Conservation of proteins amongst Colletotrichum species&amp;quot;&lt;/em&gt; section:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;A total of 11,914 orthogroups with two or more proteins were&#xA;identified (Fig. 4b). Of these, 7,950 groups (74.0% proteins from C.&#xA;shisoi, 63.0% from C. higginsianum, 78.3% from C. incanum, 74.2% from&#xA;C. tofieldiae and 73.7% from C. graminicola) were conserved in all&#xA;five species (Supplementary Tables S3 and S4)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How is it possible to calculate the values?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance&lt;/p&gt;&#xA;" OwnerUserId="3329" LastActivityDate="2022-12-19T11:58:39.677" Title="How to analyse Orthofinder results" Tags="|clustering|orthologues|clusterprofiler|orthofinder|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20209" PostTypeId="1" AcceptedAnswerId="20221" CreationDate="2022-12-19T10:01:47.437" Score="1" ViewCount="44" Body="&lt;p&gt;I have mtDNA sequences of the &lt;em&gt;cox1&lt;/em&gt; gene from various populations of one species. I want to perform an AMOVA allele frequency analysis via Arlequin. This allele frequency package however requires my sequences to have the same length.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is it possible to use my sequences?&lt;/li&gt;&#xA;&lt;li&gt;Do I have to align them first maybe?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The length difference is most likely due to indels/deletions. Do I have to use specific parts of the sequence for each gene of the subunit?&lt;/p&gt;&#xA;" OwnerUserId="16657" LastEditorUserId="3848" LastEditDate="2022-12-20T17:09:10.393" LastActivityDate="2022-12-20T17:09:10.393" Title="How can I use sequences with different lengths in arlequin?" Tags="|phylogenetics|genetics|allele-frequency|populations|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20210" PostTypeId="2" ParentId="20207" CreationDate="2022-12-19T11:58:39.677" Score="4" Body="&lt;p&gt;I think they simply calculate the percentage of proteins in each species that belongs to an orthogroup conserved in the five species.&lt;/p&gt;&#xA;&lt;p&gt;There should be a tab separated text file called Orthogroups.GeneCount.tsv (at least in the Orthofinder version used in the paper). This file contains counts of the number of genes for each species in each orthogroup. You can use this file to get the total number of genes in each species associated with those 7950 orthogroups and then calculate its proportion compared to the total number of genes.&lt;/p&gt;&#xA;&lt;p&gt;Hope this helps&lt;/p&gt;&#xA;" OwnerUserId="15700" LastActivityDate="2022-12-19T11:58:39.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20211" PostTypeId="2" ParentId="20151" CreationDate="2022-12-19T12:44:09.193" Score="1" Body="&lt;p&gt;tried to understand git, I ended up with this, seems more coherent with the previous habits in the biopython project, but cant push it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zGqfC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zGqfC.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Anyone could pass it along ? [of course needs to be checked out, my tests are OK, but I am a novice].&lt;/p&gt;&#xA;" OwnerUserId="10049" LastActivityDate="2022-12-19T12:44:09.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20212" PostTypeId="1" AcceptedAnswerId="20213" CreationDate="2022-12-19T13:00:34.287" Score="2" ViewCount="76" Body="&lt;p&gt;I am performing an  AMOVA analysis via Arlequin.&lt;/p&gt;&#xA;&lt;p&gt;At this point, I have:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;collected sequences of mtDNA from the &lt;em&gt;cox&lt;/em&gt;1 gene,&lt;/li&gt;&#xA;&lt;li&gt;aligned the different genes using clustalX&lt;/li&gt;&#xA;&lt;li&gt;converted the files into .arp format.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I then altered the structure of my populations in Arlequin so that all the different populations are in one group. When I tried to run AMOVA I get the error message&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Ill-Defined genetic structure&amp;quot;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Any clues as to what I have to adjust?&lt;/p&gt;&#xA;" OwnerUserId="16657" LastEditorUserId="3848" LastEditDate="2022-12-20T17:08:56.527" LastActivityDate="2022-12-20T17:08:56.527" Title="How can I fix &quot;Ill-defined genetic structure&quot; Error message in Arlequin?" Tags="|phylogenetics|phylogeny|allele-frequency|populations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20213" PostTypeId="2" ParentId="20212" CreationDate="2022-12-19T13:35:17.553" Score="0" Body="&lt;p&gt;If one group is one population this will not work with Arlequin AMOVA. There must be more than one population (I might have misread the question however).&lt;/p&gt;&#xA;&lt;p&gt;There could several reasons for this output. The most likely is there are insufficient SNPs within the cox1 locus for MANOVA to identify a population structure with sufficient statistical robustness. What it requires are a minimum of 3 independent alleles (SNPs), really 3 genotypes. If the SNPs are linked on the same bit of DNA that doesn't count - thats just one genotype. What I suspect is happening is there is a dearth (not enough) genetic information within cox1 to perform MANOVA.&lt;/p&gt;&#xA;&lt;p&gt;There could be other explanations, but this is the first issue to address. This isn't just Arlequin it is standard theory for any allele frequency based calculations and why this is important is more complicated to explain.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Please note, it is the run-up to Christmas .. okay put Christmas on hold. To respond to the comments ...&lt;/p&gt;&#xA;&lt;p&gt;Firstly, if you hold the data set it is very easy to identify the number of genotypes. There are past posts on this herein. I am not inclined to describe alternative approaches to an algorithm I've still not yet published :-) ... however it is really easy and needs an empirical test. It is good to assess number of genotypes &lt;em&gt;per se&lt;/em&gt; Arlequin will do this analysis ... finding that within the output I dunno.&lt;/p&gt;&#xA;&lt;p&gt;Extending loci is always good and for mtDNA I would not recommend with 16S nor cox2. rRNA should be avoided in allele frequency calculations, its okay for taxonomic assignment (bacteria rather than mtDNA). cox2 is a small locus. I would recommend for mtDNA&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;cobB, formerly called cytB&lt;/li&gt;&#xA;&lt;li&gt;intergenic region of mtDNA&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;cobB is a good target, it has a lot more selection on it for within species analysis than cox1. The intergenic region is rarely used, its a good locus because of the higher rates of mutation therein.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-12-20T17:00:29.880" LastActivityDate="2022-12-20T17:00:29.880" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20214" PostTypeId="2" ParentId="20142" CreationDate="2022-12-19T19:36:46.717" Score="1" Body="&lt;p&gt;I think that the &amp;quot;Global Variables&amp;quot; section of the &lt;a href=&quot;https://www.tbi.univie.ac.at/RNA/ViennaRNA/doc/html/wrappers.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;docs&lt;/a&gt; describes this:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For the Python interface(s) SWIG places global variables of the C-library into an additional namespace cvar. For instance, changing the global temperature variable thus becomes&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;RNA.cvar.temperature = 25&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Globals are somewhat surprising to me as a way of handling this. It seems like you should be able to specify this differently. Maybe I'm missing something?&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-12-19T19:36:46.717" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20215" PostTypeId="1" AcceptedAnswerId="20223" CreationDate="2022-12-19T21:45:21.007" Score="3" ViewCount="459" Body="&lt;p&gt;I am trying to run a very large number of transposase-accessible chromatin (ATAC)-seq samples through DESeq2 to find peaks of differential chromatin accessible across my study genome.&lt;/p&gt;&#xA;&lt;p&gt;I have about 100 samples, and 1,000,000 &amp;quot;genes&amp;quot; which are really peaks.&lt;/p&gt;&#xA;&lt;p&gt;Doing this takes a significant amount of RAM; right now I am running on a 1 TB bigmem node and still getting an out-of-memory error. The code works perfectly fine for a small subset of samples.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have an alternative strategy to use DESeq2 on a very large dataset? Should I look into doing a Wilcox-rank test myself? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="16665" LastEditorUserId="73" LastEditDate="2022-12-29T18:15:45.110" LastActivityDate="2022-12-29T18:15:45.110" Title="DESeq2 for large number of samples takes too much RAM" Tags="|r|deseq2|differential-expression|edger|atac-seq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20216" PostTypeId="2" ParentId="20215" CreationDate="2022-12-20T00:22:38.917" Score="1" Body="&lt;p&gt;Absolutely, look at Wilcox-rank, not purely because of the computational overhead, but the parametric assumption that DESeq2 uses  &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-022-02648-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;are not correct&lt;/a&gt; on large data sets. This might describe your data sets too:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Abstract&lt;/strong&gt;&#xA;DESeq2 and edgeR, have unexpectedly high false discovery rates.&lt;/p&gt;&#xA;&lt;p&gt;Expanding the analysis to limma-voom, NOISeq, dearseq, and &lt;strong&gt;Wilcoxon rank-sum test&lt;/strong&gt;, we found that [false discovery rates] FDR control is often failed &lt;strong&gt;except for the Wilcoxon rank-sum test&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Particularly, the actual &lt;strong&gt;FDRs of DESeq2&lt;/strong&gt; and edgeR sometimes exceed &lt;strong&gt;20%&lt;/strong&gt; when the target FDR is 5%. Based on these results, for population-level RNA-seq studies with large sample sizes, &lt;strong&gt;we recommend the Wilcoxon rank-sum test&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Results and Discussion&lt;/strong&gt;&#xA;DESeq2 and edgeR had only an 8% overlap in the DEGs they identified&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;hr /&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="73" LastEditDate="2022-12-24T07:53:34.673" LastActivityDate="2022-12-24T07:53:34.673" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20217" PostTypeId="1" AcceptedAnswerId="20219" CreationDate="2022-12-20T08:39:32.050" Score="3" ViewCount="242" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://stackoverflow.com/questions/74852238/read-clustal-file-in-python&quot;&gt;StackOverflow&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have a multiple sequence alignment (MSA) file derived from mafft in clustal format which I want to import into Python and save into a PDF file. I need to import the file and then highlight some specific words. I've tried to simply import the pdf of the MSA but after the highlight command doesn't work.&lt;/p&gt;&#xA;&lt;p&gt;I need to print the file like this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/OKNW2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OKNW2.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Multi.txt&lt;/code&gt; :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CLUSTAL format alignment by MAFFT FFT-NS-i (v7.453)&#xA;&#xA;&#xA;Consensus       --------------------------------------acgttttcgatatttatgccat&#xA;AMP             tttatattttctcctttttatgatggaacaagtctgcgacgttttcgatatttatgccat&#xA;                                                      **********************&#xA;&#xA;Consensus       atgtgcatgttgtaaggttgaaagcaaaaatgaggggaaaaaaaatgaggtttttaataa&#xA;AMP             atgtgcatgttgtaaggttgaaagcaaaaatgaggggaaaaaaaatgaggtttttaataa&#xA;                ************************************************************&#xA;&#xA;Consensus       ctacacatttagaggtctaggaaataaaggagtattaccatggaaatgtatttccctaga&#xA;AMP             ctacacatttagaggtctaggaaataaaggagtattaccatggaaatgtaattccctaga&#xA;                ************************************************** *********&#xA;&#xA;Consensus       tatgaaatattttcgtgcagttacaacatatgtgaatgaatcaaaatatgaaaaattgaa&#xA;AMP             tatgaaatattttcgtgcagttacaacatatgtgaatgaatcaaaatatgaaaaattgaa&#xA;                ************************************************************&#xA;&#xA;Consensus       atataagagatgtaaatatttaaacaaagaaactgtggataatgtaaatgatatgcctaa&#xA;AMP             atataagagatgtaaatatttaaacaaagaaactgtggataatgtaaatgatatgcctaa&#xA;                ************************************************************&#xA;&#xA;Consensus       ttctaaaaaattacaaaatgttgtagttatgggaagaacaaactgggaaagcattccaaa&#xA;AMP             ttctaaaaaattacaaaatgttgtagttatgggaagaacaaactgggaaagcattccaaa&#xA;                ************************************************************&#xA;&#xA;Consensus       aaaatttaaacctttaagcaataggataaatgttatattgtctagaaccttaaaaaaaga&#xA;AMP             aaaatttaaacctttaagcaataggataaatgttatattgtctagaaccttaaaaaaaga&#xA;                ************************************************************&#xA;&#xA;Consensus       agattttgatgaagatgtttatatcattaacaaagttgaagatctaatagttttacttgg&#xA;AMP             agattttgatgaagatgtttatatcattaacaaagttgaagatctaatagttttacttgg&#xA;                ************************************************************&#xA;&#xA;Consensus       gaaattaaattactataaatgttttattataggaggttccgttgtttatcaagaattttt&#xA;AMP             gaaattaaattactataaatgttttattataggaggttccgttgtttatcaagaattttt&#xA;                ************************************************************&#xA;&#xA;Consensus       agaaaagaaattaataaaaaaaatatattttactagaataaatagtacatatgaatgtga&#xA;AMP             agaaaagaaattaataaaaaaaatatattttactagaataaatagtacatatgaatgtga&#xA;                ************************************************************&#xA;&#xA;Consensus       tgtattttttccagaaataaatgaaaatgagtatcaaattatttctgttagcgatgtata&#xA;AMP             tgtattttttccagaaataaatgaaaatgagtatcaaattatttctgttagcgatgtata&#xA;                ************************************************************&#xA;&#xA;Consensus       tactagtaacaatacaacattgga----------------------------------&#xA;AMP             tactagtaacaatacaacattggattttatcatttataagaaaacgaataataaaatg&#xA;                ************************                                  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I import the alignment and print in the new PDF with the right alignment of the sequences.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="8933" LastEditorUserId="73" LastEditDate="2022-12-20T19:41:10.823" LastActivityDate="2022-12-24T00:47:27.317" Title="Read Clustal file in Python" Tags="|python|sequence-alignment|phylogenetics|multiple-sequence-alignment|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20218" PostTypeId="1" CreationDate="2022-12-20T10:22:22.230" Score="3" ViewCount="440" Body="&lt;p&gt;I can create an Upset plot with the input file, which can be found &lt;a href=&quot;https://cloudstor.aarnet.edu.au/plus/s/OupXC8rF2bDFtRV&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; library(&amp;quot;UpSetR&amp;quot;)&#xA;&amp;gt; orthogroups_df&amp;lt;- read.table(&amp;quot;Orthogroups.GeneCount.tsv.txt&amp;quot;,  header=T, stringsAsFactors = F)&#xA;&amp;gt; #All species&#xA;&amp;gt; selected_species &amp;lt;- colnames(orthogroups_df)[2:(ncol(orthogroups_df) -1)] &#xA;&amp;gt; selected_species&#xA; [1] &amp;quot;Atha&amp;quot; &amp;quot;Vvin&amp;quot; &amp;quot;Slyc&amp;quot; &amp;quot;Stub&amp;quot; &amp;quot;Cann&amp;quot; &amp;quot;Natt&amp;quot; &amp;quot;Ntab&amp;quot; &amp;quot;Ntom&amp;quot; &amp;quot;Nsyl&amp;quot; &amp;quot;Ngla&amp;quot; &amp;quot;NQLD&amp;quot; &amp;quot;Nlab&amp;quot;&#xA;&amp;gt; head(orthogroups_df)&#xA;  Orthogroup Atha Vvin Slyc Stub Cann Natt Ntab Ntom Nsyl Ngla NQLD Nlab Total&#xA;1  OG0000000    0    0    0    0    0    0    0    0    0    0  965    3   968&#xA;2  OG0000001    0    0    0    0    1    0    0    0    0    0    3  448   452&#xA;3  OG0000002    0    0    0    0    1    0    0    1    1    0  313  120   436&#xA;4  OG0000003    0   26   25   39   93   21   63   36   33   46   15   16   413&#xA;5  OG0000004    1    0    9    4   42   34  154   11    8  109    2    6   380&#xA;6  OG0000005    0    0   20    1    2    1   70   43   91   34   61   44   367&#xA;&amp;gt; ncol(orthogroups_df)&#xA;[1] 14&#xA;&amp;gt; orthogroups_df[orthogroups_df &amp;gt; 0] &amp;lt;- 1&#xA;&amp;gt; upset(orthogroups_df, nsets = ncol(orthogroups_df), sets = rev(selected_species), keep.order = T, order.by = &amp;quot;freq&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I read &lt;a href=&quot;https://www.nature.com/articles/s41598-019-50076-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;, and in &lt;em&gt;Figure 2b&lt;/em&gt; and &lt;em&gt;Supplementary Table S4&lt;/em&gt;, they had the gene numbers additionally to a number of families.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Lg0AC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Lg0AC.png&quot; alt=&quot;Figur2b&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/yzXnm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yzXnm.png&quot; alt=&quot;Supplementary Table S4&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;How can the upset plot be reproduced with gene numbers or/and create such a table?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance,&lt;/p&gt;&#xA;" OwnerUserId="3329" LastActivityDate="2022-12-20T10:22:22.230" Title="Expending UpsetR plot with Orthofinder families and gene numbers" Tags="|r|ggplot2|dataframe|upsetr|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20219" PostTypeId="2" ParentId="20217" CreationDate="2022-12-20T11:10:16.550" Score="2" Body="&lt;p&gt;Ok, figured out a way, not sure its the best one,&lt;/p&gt;&#xA;&lt;p&gt;nedd to install &lt;code&gt;fpdf2&lt;/code&gt; (&lt;code&gt;pip install fpdf2&lt;/code&gt;)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from io import StringIO&#xA;&#xA;from Bio import AlignIO  # Biopython 1.80&#xA;&#xA;from fpdf import FPDF # pip install fpdf2&#xA; &#xA;alignment = AlignIO.read(&amp;quot;Multi.txt&amp;quot;, &amp;quot;clustal&amp;quot;)&#xA;&#xA;&#xA;stri = StringIO()&#xA;    &#xA;AlignIO.write(alignment, stri, 'clustal' )&#xA;&#xA;# print(stri.getvalue())&#xA;&#xA;stri_lines = [ i for i in stri.getvalue().split('\n')]&#xA;&#xA;# print(stri_lines)&#xA;&#xA;pdf = FPDF(orientation=&amp;quot;P&amp;quot;, unit=&amp;quot;mm&amp;quot;, format=&amp;quot;A4&amp;quot;)&#xA; &#xA;# Add a page&#xA;pdf.add_page()&#xA;&#xA;pdf.add_font('FreeMono', '', 'FreeMono.ttf')&#xA;&#xA;pdf.set_font(&amp;quot;FreeMono&amp;quot;, size = 8)&#xA;&#xA;for x in stri_lines:&#xA;    pdf.cell(0, 5, txt = x, border = 0,  new_x=&amp;quot;LMARGIN&amp;quot; , new_y=&amp;quot;NEXT&amp;quot;,  align = 'L', fill = False)&#xA;    &#xA;    # print(len(x))&#xA;&#xA;pdf.output(&amp;quot;out.pdf&amp;quot;) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output pdf  &lt;code&gt;out.pdf&lt;/code&gt; :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oDg7V.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oDg7V.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Not sure why the file Header is changed, think is something within Biopythion (!!! ???) you can check adding:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with open('file_output.txt', 'w') as filehandler:&#xA;    AlignIO.write(alignment, filehandler, 'clustal')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I had to place a tff FreeMono font (Mono spaced font in my script directory) see: &lt;code&gt;pdf.add_font('FreeMono', '', 'FreeMono.ttf')&lt;/code&gt; otherwise the alignement won't be printed in the correct way&#xA;&lt;a href=&quot;https://graphicdesign.stackexchange.com/questions/13148/which-fonts-have-the-same-width-for-every-character&quot;&gt;Which fonts have the same width for every character?&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Attached a png of my pdf. See that you can highlight it,&lt;/p&gt;&#xA;&lt;p&gt;using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pdf.set_fill_color(255, 255, 0)&#xA;&#xA;filling = False&#xA;for x in stri_lines:&#xA;    if 'Consensus' in x:&#xA;        filling = True&#xA;    else:&#xA;        filling = False&#xA;    pdf.cell(0, 5, txt = x, border = 0,  new_x=&amp;quot;LMARGIN&amp;quot; , new_y=&amp;quot;NEXT&amp;quot;,  align = 'L', fill = filling)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or something similar you can highlight while printing:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9Dlzn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9Dlzn.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2022-12-20T21:57:50.623" LastActivityDate="2022-12-20T21:57:50.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20220" PostTypeId="1" AcceptedAnswerId="20222" CreationDate="2022-12-20T15:58:59.327" Score="1" ViewCount="68" Body="&lt;p&gt;I have been submitting my snakemake pipeline to a node-shared high performance computer (HPC) via &lt;a href=&quot;https://hpc.llnl.gov/banks-jobs/running-jobs/lsf-user-manual#:%7E:text=The%20bsub%20command%20is%20used,it%20with%20the%20necessary%20corrections.&quot; rel=&quot;nofollow noreferrer&quot;&gt;bsub&lt;/a&gt; the following way:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bsub -W {resources.runtime_min} -n {threads} -M {resources.mem_mb} -R 'rusage[mem={resources.mem_mb}]' -o run_logs/{name}_%J.out -e run_logs/{name}_%J.err&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Works great, no problem.&lt;/p&gt;&#xA;&lt;p&gt;However, I noticed that the memory limit was &lt;code&gt;tasks*memory&lt;/code&gt; given. I am not aware of any memory issues on the HPC. The &amp;quot;issue&amp;quot; was not noticed until I asked why my memory limit was so high compared to what I gave with the &lt;code&gt;bsub&lt;/code&gt; flag &lt;code&gt;-M&lt;/code&gt; (I wanted to understand how everything worked).&lt;/p&gt;&#xA;&lt;p&gt;I was recommend to use a &lt;code&gt;-hl&lt;/code&gt; flag to hard cap the usage. It was actually an administator that recommended it, after I asked him about how it worked. They now want to implement it for other users. I found this in the &lt;a href=&quot;https://www.ibm.com/docs/en/spectrum-lsf/10.1.0?topic=options-hl&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; too, so I understand this.&lt;/p&gt;&#xA;&lt;p&gt;But I see no one else is using this and it is also not suggested when you want to use the &lt;code&gt;bsub&lt;/code&gt; outfiles to understand how much resources you need (&lt;a href=&quot;https://hpc.ncsu.edu/Documents/lsf_submit_faq.php#:%7E:text=back%20to%20top-,How%20much%20memory%20does%20my%20code%20use%3F,-Do%20a%20sample&quot; rel=&quot;nofollow noreferrer&quot;&gt;how much memory should I use&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have thoughts on this?&lt;/p&gt;&#xA;&lt;p&gt;Somehow, it makes sense to use &lt;code&gt;-hl&lt;/code&gt;, but I am hesitant since no one seems to use &lt;code&gt;-hl&lt;/code&gt; for some reason, and I feel like I do not fully understand it.&lt;/p&gt;&#xA;&lt;p&gt;I'm asking here, because I felt I kept bugging the administrator with the same questions :'-). Thank you in advance for any advice.&lt;/p&gt;&#xA;" OwnerUserId="16204" LastEditorUserId="73" LastEditDate="2022-12-21T21:50:06.600" LastActivityDate="2022-12-21T21:50:06.600" Title="bsub with -M and -hl a good idea?" Tags="|linux|hpc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20221" PostTypeId="2" ParentId="20209" CreationDate="2022-12-20T16:29:03.920" Score="0" Body="&lt;p&gt;All that is needed is to align the sequences within a formal aligner like MAFFT or muscle. Then pass that through the PDGSpider converter. Arlequin should just take it from there.&lt;/p&gt;&#xA;&lt;p&gt;Manual annotation of the alignment is normal e.g. &lt;code&gt;seqoTron&lt;/code&gt; and is traditionally regarded as the best approach rather than do this algorithmically. Thus I'm suggesting (strongly suggesting) do this prior the PGDSpider conversion. If an alignment has not been performed Arlequin should throw an error.&lt;/p&gt;&#xA;&lt;p&gt;If there are trailing gaps (indels or '-') at the beginning and end of the alignment Arlequin will/should ignore these. You can of course simply cut them out with the alignment editor (Mega X will also work for alignment editing). The important point is that Arlequin must correctly identify homologues its an allele frequency package and nothing more. If there are errors in the alignment Arlequin will believe there is population structure when it could be absent.&lt;/p&gt;&#xA;&lt;p&gt;I can't honestly believe that Arlequin will count an indel '-' as an informative site. It should just ignore those sites, because it is a very respectable package that has been around for years and years.&lt;/p&gt;&#xA;&lt;p&gt;Okay, I am now signing off for Christmas... with occasional exceptions.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-12-20T16:29:03.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20222" PostTypeId="2" ParentId="20220" CreationDate="2022-12-20T16:50:43.237" Score="1" Body="&lt;p&gt;What this question appears to be about is capping the RAM usage on a node within a shared HPC.&lt;/p&gt;&#xA;&lt;p&gt;Being 'nice' on an HPC is very honourable indeed, it is however the system administrators jurisdiction. It simply requires an email to them and they will provide the authoritative answer. Thus if there were RAM usage issues within the nodes then they will be aware of it. Limiting RAM on a job is risky because the biggest 'fall-over' in bioinformatics is a job running into a RAM bottleneck. If that happens the process just freezes and literally will take 1 log more run-time. If the machine doesn't have enough RAM for the job, thats a separate issue and easily solvable via cloud computing.&lt;/p&gt;&#xA;&lt;p&gt;The individual who made the suggestion might be running into RAM bottleneck issues on the machine. However, they should inform the HPC administrator who could then provide an assessment whether all other users should cap their RAM usage. There are alternatives - the user experiencing RAM bottlenecks can parallelise their job and this will overcome RAM issues. This can be a bit tricky on a course-grain parallelisation with a shared HPC.&lt;/p&gt;&#xA;&lt;p&gt;So in summary, I wouldn't do this personally and if I was concerned I would seek the advice of the administrator who will give the definitive response. If admin says 'do it' then do it, but it your jobs take way-way longer than they should I'd relax the constraint and inform admin.&lt;/p&gt;&#xA;&lt;p&gt;Admin could enforce this constraint without the user base being aware of what was happening if it was a major issue on the server.&lt;/p&gt;&#xA;&lt;p&gt;These days if there are HPC issues its easier to shift on a cloud computer and is not worth the arguments when everyone is trying to get their jobs done before the vacation.&lt;/p&gt;&#xA;&lt;p&gt;Merry Christmas!&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;From the comments ...&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;System admin recommended this ... okay they should now inform the rest of the user base (which is what you are implying), unless there are been a specific issue. Thats a bit different and it is polite to comply with their request;&lt;/li&gt;&#xA;&lt;li&gt;What you can do is justify to the system admin why you job requires extra RAM;&lt;/li&gt;&#xA;&lt;li&gt;Systems admin might have automatically restricted the RAM access on the account - you can ask them to lift that;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You said,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;1 log more run-time&amp;quot;?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;That is extremely important to understand. Bioinformatic jobs can use loads of RAM, if the job exceeds the RAM capacity of the node essentially the job &amp;quot;freezes&amp;quot;. What it really does is slow the time to completion of the job dramatically, so I'm saying in my experience it's about 10-fold longer to complete than normal. That is what I am calling a RAM-bottleneck.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Final point&lt;/strong&gt;&#xA;On the &lt;code&gt;bsub&lt;/code&gt; parallelisation appears to be efficiently implemented and if RAM issues are occurring parallelising the job is the best solution. This would be a work around if the systems admin wants to restrict RAM usage and as a user you are wanting quick run-time (fast completion). Not all algorithms will parallelise but most will these days.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-12-20T18:14:32.587" LastActivityDate="2022-12-20T18:14:32.587" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20223" PostTypeId="2" ParentId="20215" CreationDate="2022-12-20T19:30:19.047" Score="4" Body="&lt;p&gt;I'll preface this by saying that I don't think DESeq2 is the right tool to use for ATAC-Seq data. My own study of ATAC-Seq patterns [admittedly only a couple of runs that were our first exploration of the technique] indicates that it has a fractal expression which is difficult to properly summarise at any scale, especially at a bulk sample level. I found &lt;a href=&quot;https://twitter.com/ryanjames722/status/1545261940406116352&quot; rel=&quot;nofollow noreferrer&quot;&gt;this poster&lt;/a&gt; from a quick search on Twitter, which recommends &lt;a href=&quot;https://github.com/Zhang-lab/ATAC-seq_QC_analysis&quot; rel=&quot;nofollow noreferrer&quot;&gt;AIAP&lt;/a&gt; for ATAC-Seq analysis (but don't read too much into that; I have no experience with doing ATAC-Seq analysis). What I &lt;em&gt;do&lt;/em&gt; have a lot of experience with is DESeq2.&lt;/p&gt;&#xA;&lt;p&gt;DESeq2 has certain assumptions about the expression patterns of genes which are built into its differential expression model, and the further you get from gene-level analysis, the less appropriate DESeq2 is for data analysis (for example, &lt;a href=&quot;https://doi.org/10.1093/nar/gkz622&quot; rel=&quot;nofollow noreferrer&quot;&gt;Swish&lt;/a&gt; is a better approach for isoform level and allelic analysis).&lt;/p&gt;&#xA;&lt;p&gt;However, if you're absolutely set on using DESeq2...  the &lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#speed-up-and-parallelization-thoughts&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2 data analysis documentation&lt;/a&gt; has a few different suggestions for speeding up analysis:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The above steps should take less than 30 seconds for most analyses. For experiments with complex designs and many samples (e.g. dozens of coefficients, ~100s of samples), one may want to have faster computation than provided by the default run of DESeq. We have two recommendations:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;By using the argument fitType=&amp;quot;glmGamPoi&amp;quot;, one can leverage the faster NB GLM engine written by Constantin Ahlmann-Eltze. Note that glmGamPoi’s interface in DESeq2 requires use of test=&amp;quot;LRT&amp;quot; and specification of a reduced design.&lt;/li&gt;&#xA;&lt;li&gt;One can take advantage of parallelized computation. Parallelizing DESeq, results, and lfcShrink can be easily accomplished by loading the BiocParallel package, and then setting the following arguments: parallel=TRUE and BPPARAM=MulticoreParam(4), for example, splitting the job over 4 cores. However, some words of advice on parallelization: first, it is recommend to filter genes where all samples have low counts, to avoid sending data unnecessarily to child processes, when those genes have low power and will be independently filtered anyway; secondly, there is often diminishing returns for adding more cores due to overhead of sending data to child processes, therefore I recommend first starting with small number of additional cores. Note that obtaining results for coefficients or contrasts listed in resultsNames(dds) is fast and will not need parallelization. As an alternative to BPPARAM, one can register cores at the beginning of an analysis, and then just specify parallel=TRUE to the functions when called.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Also, applying a low count filter can help a lot.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Pre-filtering&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;While it is not necessary to pre-filter low count genes before running the DESeq2 functions, there are two reasons which make pre-filtering useful: by removing rows in which there are very few reads, &lt;em&gt;&lt;strong&gt;we reduce the memory size of the dds data object, and we increase the speed of the transformation and testing functions within DESeq2.&lt;/strong&gt;&lt;/em&gt; It can also improve visualizations, as features with no information for differential expression are not plotted.&lt;/p&gt;&#xA;&lt;p&gt;Here we perform a minimal pre-filtering to keep only rows that have at least 10 reads total. Note that more strict filtering to increase power is automatically applied via independent filtering on the mean of normalized counts within the results function.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;keep &amp;lt;- rowSums(counts(dds)) &amp;gt;= 10&#xA;dds &amp;lt;- dds[keep,]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Alternatively, a popular filter is to ensure at least X samples with a count of 10 or more, where X can be chosen as the sample size of the smallest group of samples:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;# unevaluated chunk...&#xA;keep &amp;lt;- rowSums(counts(dds) &amp;gt;= 10) &amp;gt;= X&#xA;dds &amp;lt;- dds[keep,]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2022-12-20T19:30:19.047" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20224" PostTypeId="2" ParentId="20217" CreationDate="2022-12-21T10:20:09.240" Score="3" Body="&lt;p&gt;&lt;a href=&quot;https://github.com/moshi4/pyMSAviz&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyMSAviz&lt;/a&gt; may help with this problem.&#xA;pyMSAviz is a Python tool with a convenient CLI/API implementation for visualizing MSA files.&lt;/p&gt;&#xA;&lt;p&gt;For example, you can output MSA figure in PDF format with the following command.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pymsaviz -i Multi.txt -o msa_result.pdf --format clustal \&#xA;         --wrap_length 60 --show_count --color_scheme Identity&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xTbPh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xTbPh.png&quot; alt=&quot;msa_result.png&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can also use pyMSAviz's API to configure the appearance of the figure in more detail, such as highlighting.&lt;/p&gt;&#xA;" OwnerUserId="16672" LastActivityDate="2022-12-21T10:20:09.240" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20225" PostTypeId="2" ParentId="6910" CreationDate="2022-12-21T19:09:44.800" Score="0" Body="&lt;p&gt;As others have mentioned, cost is probably one of the main issue.&lt;/p&gt;&#xA;&lt;p&gt;Other than that, PacBio sequencing also requires more input DNA material (~3-10 ug for human) while Illumina sequencing works with just 100ng. This makes PacBio sequencing more challenging when samples are rare. I'm sure people at PacBio are trying to resolve some of these concerns already though.&lt;/p&gt;&#xA;" OwnerUserId="7349" LastActivityDate="2022-12-21T19:09:44.800" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20226" PostTypeId="2" ParentId="20169" CreationDate="2022-12-22T01:34:15.903" Score="1" Body="&lt;p&gt;Some of the solutions suggested above utilize GNU &lt;code&gt;grep.&lt;/code&gt; If you are a Mac user, you can easily install GNU version of the commands into your Mac. You can do this by installing the GNU Command Line Tools. The easiest way is to use Homebrew, then &lt;code&gt;brew install coreutils.&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The default commands will still be MacOS version (unless you intentionally set otherwise), but when you want to use GNU version, you could just add a &lt;code&gt;g&lt;/code&gt; before the command. For example, &lt;code&gt;sort&lt;/code&gt; vs &lt;code&gt;gsort&lt;/code&gt;, &lt;code&gt;grep&lt;/code&gt; vs &lt;code&gt;ggrep&lt;/code&gt;. You can also check manual of the command the same way, e.g. &lt;code&gt;man gsort&lt;/code&gt;. Trust me this will make your life easier as a Mac user.&lt;/p&gt;&#xA;&lt;p&gt;Also, in recent years default MacOS Terminal shell is &lt;code&gt;zsh.&lt;/code&gt; If you desire you can switch it back to &lt;code&gt;bash&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2022-12-22T01:34:15.903" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20227" PostTypeId="1" AcceptedAnswerId="20229" CreationDate="2022-12-22T08:45:45.560" Score="2" ViewCount="512" Body="&lt;p&gt;Suppose you have a pipeline that takes reads as input from the user and trims them.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;// Declare syntax version&#xA;nextflow.enable.dsl=2&#xA;&#xA;/* &#xA; * pipeline input parameters &#xA; */&#xA;&#xA;&#xA;params.reads = &amp;quot;$projectDir/data/*_{1,2}.fastq&amp;quot;&#xA;&#xA;&#xA;/*&#xA; * define the `TRIM` process that creates trimmed.reads.fastq files&#xA; * given the reads.fastq file. &#xA; */&#xA; &#xA;process TRIM {&#xA;&#xA;    tag &amp;quot;$pair_id&amp;quot;&#xA;&#xA;    input:&#xA;    tuple val(pair_id), path(reads)&#xA;&#xA;    output:&#xA;    path(pair_id)&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    atria -r &lt;span class=&quot;math-container&quot;&gt;${reads[0]} -R $&lt;/span&gt;{reads[1]} -t 1 -l 12 -q 30 -o $pair_id&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;workflow {&#xA;&#xA;    read_pairs_ch = Channel.fromFilePairs( params.reads, checkIfExists:true )&#xA;&#xA;    trim_ch=TRIM(read_pairs_ch)&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This particular one takes paired end reads as input. What if the user wanted to trim both single end and paired end reads, how to account for single end reads as the input?&lt;/p&gt;&#xA;&lt;p&gt;Best wishes.&lt;/p&gt;&#xA;" OwnerUserId="16328" LastActivityDate="2023-04-02T01:00:22.533" Title="How does one account for both single end and paired end reads as input in a Nextflow pipeline?" Tags="|nextflow|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20229" PostTypeId="2" ParentId="20227" CreationDate="2022-12-22T14:22:38.737" Score="2" Body="&lt;p&gt;There are many ways to accomplish this. Assuming you don't need to process single-end and paired-end reads simultaneously (i.e. within the same run), one way using the &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#fromfilepairs&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;fromFilePairs&lt;/code&gt;&lt;/a&gt; factory method is to just use an additional flag to define the number of files each emitted item is expected to hold. And then adjust your process definition accordingly:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params.reads = &amp;quot;${projectDir}/data/*_{1,2}.fastq.gz&amp;quot;&#xA;params.singleEnd = false&#xA;&#xA;&#xA;process TRIM {&#xA;&#xA;    tag &amp;quot;${pair_id}&amp;quot;&#xA;&#xA;    cpus 1&#xA;&#xA;    input:&#xA;    tuple val(pair_id), path(reads)&#xA;&#xA;    output:&#xA;    tuple val(pair_id), path(&amp;quot;${pair_id}/*.atria.fastq{,.gz}&amp;quot;)&#xA;&#xA;    script:&#xA;    def single = reads instanceof Path&#xA;&#xA;    def read1 = !single ? /-r &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${reads[0]}&quot;/ : /-r &quot;$&lt;/span&gt;{reads}&amp;quot;/&#xA;    def read2 = !single ? /-R &amp;quot;${reads[1]}&amp;quot;/ : ''&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    atria \\&#xA;        -t &lt;span class=&quot;math-container&quot;&gt;${task.cpus} \\&#xA;        $&lt;/span&gt;{read1} \\&#xA;        &lt;span class=&quot;math-container&quot;&gt;${read2} \\&#xA;        -l 12 \\&#xA;        -q 30 \\&#xA;        -o &quot;$&lt;/span&gt;{pair_id}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow {&#xA;&#xA;    Channel&#xA;        .fromFilePairs( params.reads, size: params.singleEnd ? 1 : 2 )&#xA;        .ifEmpty { error &amp;quot;Could not find any reads matching pattern: ${params.reads}&amp;quot; }&#xA;        .set { reads }&#xA;&#xA;    TRIM( reads ).view()&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Although we've used an additional flag, it does let the pipeline fail immediately if it can't find the files it expects. I think this approach is more explicit and provides a better user experience:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ nextflow run main.nf --reads './data/*.fastq'&#xA;N E X T F L O W  ~  version 22.10.0&#xA;Launching `main.nf` [nasty_allen] DSL2 - revision: 2cbc311974&#xA;Could not find any reads matching pattern: ./data/*.fastq&#xA;&#xA; -- Check script 'main.nf' at line: 38 or see '.nextflow.log' file for more details&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Doh! I forgot that these are single ended reads:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ nextflow run main.nf --reads './data/*.fastq' --singleEnd&#xA;N E X T F L O W  ~  version 22.10.0&#xA;Launching `main.nf` [clever_gutenberg] DSL2 - revision: 2cbc311974&#xA;executor &amp;gt;  local (3)&#xA;[8e/f90c90] process &amp;gt; TRIM (X) [100%] 3 of 3 ✔&#xA;[Y, /path/to/work/e6/48df1485bb7de8355ed8ce0f1b53da/Y/Y.atria.fastq]&#xA;[Z, /path/to/work/eb/aca3389344ac03ff283ded0c46a7a6/Z/Z.atria.fastq]&#xA;[X, /path/to/work/8e/f90c90210103c03a3fbde697e01227/X/X.atria.fastq]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2022-12-22T14:22:38.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20230" PostTypeId="1" CreationDate="2022-12-22T18:01:02.563" Score="2" ViewCount="59" Body="&lt;p&gt;I am searching for a good reference that explains the statistical methodology used in what is variously known as Joint Genotyping, Cohort Genotyping or Multi-sample Genotyping. One with a pseudo code implementation would also be ideal.&lt;/p&gt;&#xA;&lt;p&gt;I have found several discussing the approach in very general terms and the relative benefits of the technique but nothing with actual equations or algorithms. Failing that, if there is an open source implementation that people would consider to be a reference implementation and that has excellent code documentation that would also be great.&lt;/p&gt;&#xA;" OwnerUserId="8985" LastEditorUserId="8985" LastEditDate="2022-12-27T19:59:47.213" LastActivityDate="2022-12-28T12:34:05.457" Title="Reference for the statistical methodology used in joint genotype calling" Tags="|genotyping|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20231" PostTypeId="2" ParentId="20227" CreationDate="2022-12-22T20:23:26.103" Score="2" Body="&lt;p&gt;My personal strategy is to always read data from a samplesheet which then auto-detects whether a sample is paired-end or single-end and stores this information in a so-called meta map (inspired by nf-core) which then passes this information to the modules.&lt;/p&gt;&#xA;&lt;p&gt;Example:&lt;/p&gt;&#xA;&lt;p&gt;Say you have a samplesheet that is a three-column csv file with a header:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat samplesheet.csv &#xA;sample,r1,r2&#xA;sample1,/Users/atpoint/sample1_R1.fq.gz,/Users/atpoint/sample1_R2.fq.gz&#xA;sample2,/Users/atpoint/sample2_R1.fq.gz,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Read this into Nextflow and detect whether r2 has an entry. If so it is paired-end, else it is single-end:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;# main.nf&#xA;&#xA;nextflow.enable.dsl = 2&#xA;&#xA;// Channel for the samplesheet&#xA;ch_samplesheet = Channel.fromPath(&amp;quot;/Users/atpoint/samplesheet.csv&amp;quot;)&#xA;&#xA;// Parse it line by line&#xA;ch_reads = ch_samplesheet.splitCsv(header:true).map {&#xA;&#xA;    // This is the read1 and read2 entry&#xA;    r1 = it['r1']&#xA;    r2 = it['r2']&#xA;&#xA;    // Detect wiether single-end or paired-end&#xA;    is_singleEnd = r2.toString()=='' ? true : false&#xA;    &#xA;    // The &amp;quot;meta&amp;quot; map, which is a Nextflow/Groovy map with id (the sample name) and a single_end logical entry&#xA;    meta = [id: it['sample'], single_end: is_singleEnd]&#xA;    &#xA;    // We return a nested map, the first entry is the meta map, the second one is the read(s)&#xA;    r2.toString()=='' ? [meta, [r1]] : [meta, [r1, r2]]&#xA;&#xA;}&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you &lt;code&gt;.view()&lt;/code&gt; the &lt;code&gt;ch_reads&lt;/code&gt; you get:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[[id:sample1, single_end:false], [/Users/atpoint/sample1_R1.fq.gz, /Users/atpoint/sample1_R2.fq.gz]]&#xA;[[id:sample2, single_end:true], [/Users/atpoint/sample2_R1.fq.gz]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A process that can use that could be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;process A {&#xA;&#xA;    tag &amp;quot;$meta.id&amp;quot;&#xA;&#xA;    input:&#xA;    tuple val(meta), path(reads)&#xA;&#xA;    script:&#xA;    &#xA;    // The meta map holds the single/paired information that can now easily be parsed in the module in a standardized way&#xA;    if(meta.single_end){&#xA;&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        do_single_end_things with ${reads}&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;    } else {&#xA;&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        do_paired_end_things &lt;span class=&quot;math-container&quot;&gt;${reads}[0] $&lt;/span&gt;{reads}[1] &#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        &#xA;    }&#xA;&#xA;}&#xA;&#xA;A(ch_reads)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The meta map idea (cudos nf-core) is simple, yet versatile and powerful as it allows standardized parsing of such metadata-like information (single_end, libtype, id/sample_name, any other value you can think of) and most importantly, it allows to process single-end paired end data at the same time. All the decision making is based on the meta map and therefore automated if the modules are coded appropriately as above.&lt;/p&gt;&#xA;&lt;p&gt;Hope that helps.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastEditorUserId="16676" LastEditDate="2022-12-22T20:58:29.777" LastActivityDate="2022-12-22T20:58:29.777" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20232" PostTypeId="1" CreationDate="2022-12-22T20:58:13.017" Score="0" ViewCount="173" Body="&lt;p&gt;I downloaded the following vcf file from &lt;a href=&quot;https://www.internationalgenome.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.internationalgenome.org/&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr22.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/ALL.chr22.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I wanted to first try and calculate allele frequencies, and then from there calculate allele frequencies based on populations and so on. However, I ran into an issue where while I was using plink, I would utilize the --vcf command to try and calculate frequencies. The frequencies were calculated, but the SNP names were missing. I used the following command in Git Bash on Windows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    plink --vcf ALL.chr22.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.gz --freq --out chr22datanew&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then tried to convert the file to plink binary (bed, bim, fam) to try and use that for the basis of the allele frequencies. I used the following commands:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plink --vcf ALL.chr22.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes.vcf.vcf --make-bed --out chr22data&#xA;&#xA;plink --bfile chr22data --freq --out chr22freqs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Once again, the frequencies were calculated with no issue, but the SNP names were missing. I checked the .fam file that was created, and the SNPs were there, but not in the .bim file. This may be a simple, elementary level question, but does anyone know how to fix this? I am using plink 1.9 instead of vcftools because I'm using a computer that runs on Windows OS.&lt;/p&gt;&#xA;" OwnerUserId="15970" LastEditorUserId="475" LastEditDate="2022-12-23T13:39:04.130" LastActivityDate="2022-12-23T13:39:04.130" Title="Maintaining SNP names when calculating allele frequencies from a VCF file on plink 1.9" Tags="|vcf|snp|plink|allele-frequency|rsid|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20233" PostTypeId="1" AcceptedAnswerId="20241" CreationDate="2022-12-22T22:21:05.220" Score="1" ViewCount="142" Body="&lt;p&gt;I tried umap visualization with &lt;a href=&quot;https://github.com/scverse/scanpy&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;scanpy&lt;/code&gt;&lt;/a&gt; :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sc.pp.normalize_total(adata, target_sum=1e6)&#xA;sc.pp.log1p(adata, base=2)&#xA;sc.pp.highly_variable_genes(adata, flavor='cell_ranger', n_top_genes=400)&#xA;&#xA;sc.pp.scale(adata, zero_center=True, max_value=None, copy=False, layer=None, obsm=None)&#xA;sc.pp.pca(adata, n_comps=50, use_highly_variable=True, svd_solver='arpack')&#xA;sc.pp.neighbors(adata, n_neighbors=50)&#xA;&#xA;sc.tl.umap(adata, min_dist=0.5, spread=1.0)&#xA;sc.pl.umap(adata, color='fullname', use_raw=False, save='samples_umap.pdf')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;version&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;anndata     0.7.5&#xA;scanpy      1.6.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But the cells can't separate well&#xA;&lt;img src=&quot;https://user-images.githubusercontent.com/33963919/209233854-db64fddd-4266-4f87-805b-dced45b1547f.png&quot; alt=&quot;image&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;I tried another small dataset with &lt;code&gt;scanpy&lt;/code&gt; using the same parameters as before:&#xA;&lt;img src=&quot;https://user-images.githubusercontent.com/33963919/209387824-3a5b1037-f226-49c8-9222-f54c04a62155.jpg&quot; alt=&quot;1&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;sc.tl.umap&lt;/code&gt; still failed to down dimension the data properly.&lt;/p&gt;&#xA;&lt;p&gt;Then I tried the original &lt;a href=&quot;https://umap-learn.readthedocs.io/en/latest/plotting.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;umap&lt;/code&gt; package&lt;/a&gt; using the same data set:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import umap&#xA;import umap.plot&#xA;mapper = umap.UMAP().fit(adata.X)&#xA;umap.plot.points(mapper)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now the original &lt;code&gt;umap&lt;/code&gt; package can do down dimension very well:&#xA;&lt;img src=&quot;https://user-images.githubusercontent.com/33963919/209387903-0161dfa6-0ca5-48cc-8661-465930e23fef.jpg&quot; alt=&quot;2&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;I think there may be something wrong with the &lt;code&gt;umap&lt;/code&gt; function in &lt;code&gt;scanpy&lt;/code&gt;&#xA;Can anyone please let me know the reason?&#xA;Thanks a lot.&lt;/p&gt;&#xA;" OwnerUserId="9698" LastEditorUserId="104" LastEditDate="2023-01-31T18:47:21.623" LastActivityDate="2023-01-31T18:47:21.623" Title="umap failed to cluster the cells" Tags="|scrnaseq|single-cell|umap|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20234" PostTypeId="1" AcceptedAnswerId="20262" CreationDate="2022-12-23T11:04:31.397" Score="2" ViewCount="55" Body="&lt;p&gt;I'm working with gvcf files, containing also non-variant positions. I have noticed some insertions that I'm not able to correctly interpret or handle. An example (selected samples from a huge vcf):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;NC_013991.2 1403    .   G   GAA 176,32  .   AC=4;AF=0.014;AN=286;DP=1294;ExcessHet=0.0001;FS=0;InbreedingCoeff=0.5011;MLEAC=3;MLEAF=0.01;MQ=28.09;QD=28.2;SOR=3.912 GT:AD:DP:GQ:PGT:PID:PL:PS   1|1:0,2:2:6:1|1:1403_G_GAA:90,6,0:1403  0/0:2,0:2:6:.:.:0,6,88:.    ./.:0,0:1:.:.:.:0,0,0:. ./.:0,0:0:.:.:.:0,0,0:. ./.:1,0:1:.:.:.:0,0,0:. 0/0:2,0:2:3:.:.:0,3,45:.    ./.:1,0:1:.:.:.:0,0,0:. ./.:1,0:1:.:.:.:0,0,0:. 1/1:0,3:3:9:.:.:123,9,0:.   0/0:1,0:1:3:.:.:0,3,16:.    0/0:1,0:1:3:.:.:0,3,45:.    0/0:2,0:2:6:.:.:0,6,90:.    0/0:9,0:9:21:.:.:0,21,315:.&#xA;NC_013991.2 1404    .   A   .   .   .   DP=1286 GT:AD:DP:RGQ    0/0:2:2:6   0/0:2:2:6   0/0:2:2:6   ./.:0:0:0   0/0:1:1:3   0/0:2:2:6   0/0:1:1:3   0/0:1:1:3   0/0:3:3:9   0/0:1:1:3   0/0:1:1:3   0/0:2:2:6   0/0:9:9:21&#xA;NC_013991.2 1405    .   A   .   .   .   DP=1282 GT:AD:DP:RGQ    0/0:2:2:6   0/0:2:2:6   0/0:2:2:6   ./.:0:0:0   0/0:1:1:3   0/0:2:2:6   0/0:1:1:3   0/0:1:1:3   0/0:3:3:9   0/0:1:1:3   0/0:1:1:3   0/0:2:2:6   0/0:9:9:21&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here, the reference sequence from position 1403 to 1405 is &amp;quot;GAA&amp;quot;. At site 1403, an alternative allele &amp;quot;GAA&amp;quot; is indicated, which I would interpret as an insertion of &amp;quot;AA&amp;quot;. Two individuals thus would have genotype &amp;quot;GAAAA&amp;quot; here while the rest has &amp;quot;GAA&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I've seen multiple cases like this, here with longer alternative allele (again, selected samples, not necessarily the same as before):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;NC_013991.2 23  .   T   TAATACTTATGTGTTG    66,98   .   AC=2;AF=0.009346;AN=214;DP=314;ExcessHet=0.0105;FS=0;InbreedingCoeff=0.3797;MLEAC=3;MLEAF=0.014;MQ=51.77;QD=33.49;SOR=2.303 GT:AD:DP:GQ:PL  1/1:0,2:2:6:85,6,0  0/0:3,0:3:9:0,9,135 0/0:3,0:3:9:0,9,95  0/0:1,0:1:3:0,3,45  ./.:1,1:2:.:0,0,0   0/0:3,1:4:0:0,0,102&#xA;NC_013991.2 24  .   A   .   .   .   DP=322  GT:AD:DP:RGQ    0/0:3:3:9   0/0:3:3:9   0/0:3:3:9   0/0:1:1:3   0/0:2:2:6   0/0:4:4:12&#xA;NC_013991.2 25  .   A   .   .   .   DP=339  GT:AD:DP:RGQ    0/0:3:3:9   0/0:3:3:9   0/0:3:3:9   0/0:1:1:3   0/0:3:3:9   0/0:4:4:12&#xA;NC_013991.2 26  .   T   .   .   .   DP=353  GT:AD:DP:RGQ    0/0:3:3:9   0/0:3:3:9   0/0:3:3:9   0/0:1:1:3   0/0:3:3:9   ./.:4:5:0&#xA;NC_013991.2 27  .   A   .   .   .   DP=361  GT:AD:DP:RGQ    0/0:3:3:9   0/0:3:3:9   0/0:3:3:9   0/0:1:1:3   0/0:3:3:9   0/0:5:5:15&#xA;NC_013991.2 28  .   C   .   .   .   DP=368  GT:AD:DP:RGQ    0/0:3:3:9   0/0:3:3:9   0/0:3:3:9   0/0:1:1:3   0/0:3:3:9   0/0:5:5:15&#xA;NC_013991.2 29  .   T   .   .   .   DP=386  GT:AD:DP:RGQ    0/0:3:3:9   0/0:3:3:9   0/0:3:3:9   0/0:1:1:3   0/0:3:3:9   ./.:4:5:0&#xA;NC_013991.2 30  .   T   .   .   .   DP=388  GT:AD:DP:RGQ    0/0:3:3:9   0/0:3:3:9   0/0:3:3:9   0/0:1:1:3   0/0:3:3:9   0/0:5:5:12&#xA;NC_013991.2 31  .   A   .   .   .   DP=398  GT:AD:DP:RGQ    0/0:3:3:9   0/0:3:3:9   0/0:3:3:9   0/0:1:1:3   0/0:3:3:9   0/0:5:5:12&#xA;NC_013991.2 32  .   T   .   .   .   DP=405  GT:AD:DP:RGQ    0/0:3:3:9   0/0:3:3:9   0/0:3:3:9   0/0:1:1:3   0/0:3:3:9   0/0:5:5:12&#xA;NC_013991.2 33  .   G   .   .   .   DP=419  GT:AD:DP:RGQ    0/0:3:3:9   0/0:3:3:9   0/0:3:3:9   0/0:1:1:3   0/0:3:3:9   ./.:4:5:0&#xA;NC_013991.2 34  .   T   .   .   .   DP=427  GT:AD:DP:RGQ    0/0:3:3:9   0/0:3:3:9   0/0:3:3:9   0/0:1:1:3   0/0:3:3:9   0/0:5:5:12&#xA;NC_013991.2 35  .   G   .   0,09    LowQual BaseQRankSum=-1;DP=433;ExcessHet=3;MLEAC=.;MLEAF=.;MQ=51.42;MQRankSum=1;ReadPosRankSum=0    GT:DP:RGQ   0/0:3:9 0/0:3:9 0/0:3:9 0/0:1:3 0/0:3:9 0/0:5:0&#xA;NC_013991.2 36  .   T   .   .   .   DP=441  GT:AD:DP:RGQ    0/0:3:3:9   0/0:3:3:9   0/0:3:3:9   0/0:1:1:3   0/0:3:3:9   0/0:5:5:12&#xA;NC_013991.2 37  .   T   .   .   .   DP=447  GT:AD:DP:RGQ    0/0:3:3:9   0/0:3:3:9   0/0:3:3:9   0/0:2:2:6   0/0:3:3:9   0/0:5:5:12&#xA;NC_013991.2 38  .   G   .   .   .   DP=449  GT:AD:DP:RGQ    0/0:3:3:9   0/0:3:3:9   0/0:3:3:9   0/0:2:2:6   0/0:3:3:9   ./.:4:5:0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The alternative allele &amp;quot;TAATACTTATGTGTTG&amp;quot; is exactly the reference. Of course, the sample in question is homozygous. It seems to me these are &amp;quot;false&amp;quot; insertions: they look like insertions, but aren't.&#xA;I can write a simple awk script trimming the the alternative allele, but there are cases where there are real insertions, and I don't know how to distinguish those.&lt;/p&gt;&#xA;&lt;p&gt;I've tried the solution mentioned in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/7127/16680&quot;&gt;this answer&lt;/a&gt; (double pass through bcftools norm) but it doesn't resolve these issues.&lt;/p&gt;&#xA;&lt;p&gt;I could spend a few days coding a C++ program to do this, but surely someone with better understanding of vcf, C++, etc, has already done this?&lt;/p&gt;&#xA;&lt;p&gt;Here's a case I really don't know how to interpret:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;NC_013991.2 3321    .   A   ATTTCATT    638,04  .   AC=14;AF=0.044;AN=320;DP=1873;ExcessHet=0;FS=0;InbreedingCoeff=0.5521;MLEAC=10;MLEAF=0.031;MQ=29.5;QD=34.42;SOR=1.739   GT:AD:DP:GQ:PGT:PID:PL:PS   1|1:0,2:2:6:1|1:3321_A_ATTTCATT:90,6,0:3321 1|1:0,4:4:12:1|1:3321_A_ATTTCATT:180,12,0:3321  1|1:0,2:2:6:1|1:3321_A_ATTTCATT:90,6,0:3321 0/0:2,0:2:3:.:.:0,3,45:.    0/0:1,0:1:3:.:.:0,3,45:.    0/0:1,0:1:3:.:.:0,3,45:.    0/0:1,0:1:3:.:.:0,3,16:.    0/0:1,0:1:3:.:.:0,3,25:.    0/0:8,0:8:21:.:.:0,21,315:. 0/0:33,0:33:96:.:.:0,96,1440:.  0/0:15,0:15:39:.:.:0,39,585:.&#xA;NC_013991.2 3322    .   T   .   .   .   DP=1870 GT:AD:DP:RGQ    0/0:2:2:6   0/0:4:4:12  0/0:2:2:6   0/0:2:2:3   0/0:1:1:3   0/0:1:1:3   0/0:1:1:3   0/0:1:1:3   0/0:8:8:21  0/0:33:33:96    0/0:14:14:36&#xA;NC_013991.2 3323    .   T   .   .   .   DP=1859 GT:AD:DP:RGQ    0/0:2:2:6   0/0:4:4:12  0/0:2:2:6   ./.:1:2:0   0/0:1:1:3   0/0:1:1:3   0/0:1:1:3   0/0:1:1:3   0/0:8:8:21  0/0:34:34:99    0/0:14:14:36&#xA;NC_013991.2 3324    .   T   .   .   .   DP=1849 GT:AD:DP:RGQ    0/0:2:2:6   0/0:4:4:12  0/0:2:2:6   0/0:1:1:3   0/0:1:1:3   0/0:1:1:3   0/0:1:1:3   0/0:1:1:3   0/0:8:8:21  0/0:34:34:99    0/0:14:14:33&#xA;NC_013991.2 3325    .   TGG T   639,64  .   AC=14;AF=0.044;AN=318;DP=1843;ExcessHet=0;FS=0;InbreedingCoeff=0.5416;MLEAC=11;MLEAF=0.035;MQ=29.53;QD=28.53;SOR=1.739  GT:AD:DP:GQ:PGT:PID:PL:PS   1|1:0,2:2:6:1|1:3321_A_ATTTCATT:90,6,0:3321 1|1:0,4:4:12:1|1:3321_A_ATTTCATT:180,12,0:3321  1|1:0,2:2:6:1|1:3321_A_ATTTCATT:90,6,0:3321 0/0:1,0:1:3:.:.:0,3,45:.    0/0:1,0:1:3:.:.:0,3,45:.    0/0:1,0:1:3:.:.:0,3,45:.    0/0:1,0:1:3:.:.:0,3,45:.    0/0:1,0:1:3:.:.:0,3,25:.    0/0:8,0:8:21:.:.:0,21,315:. 0/0:34,0:34:99:.:.:0,99,1485:.  0/0:14,0:14:33:.:.:0,33,495:.&#xA;NC_013991.2 3326    .   G   T   1294,06 .   AC=15;AF=0.055;AN=274;BaseQRankSum=0;DP=1833;ExcessHet=0;FS=2.451;InbreedingCoeff=0.5364;MLEAC=17;MLEAF=0.062;MQ=56.66;MQRankSum=0;QD=16.38;ReadPosRankSum=0;SOR=0.828  GT:AD:DP:GQ:PGT:PID:PL:PS   ./.:0,2:2:.:.:.:0,0,0:. ./.:0,4:4:.:.:.:0,0,0:. ./.:0,2:2:.:.:.:0,0,0:. ./.:0,1:1:.:.:.:0,0,0:. ./.:0,1:1:.:.:.:0,0,0:. ./.:0,1:1:.:.:.:0,0,0:. ./.:0,1:1:.:.:.:0,0,0:. 1|1:0,1:1:3:1|1:3309_G_A:45,3,0:3309    1/1:0,4:4:12:.:.:145,12,0:. 0/0:34,0:34:99:.:.:0,99,1485:.  0/0:14,0:14:33:.:.:0,33,495:.&#xA;NC_013991.2 3327    .   G   .   0,02    LowQual DP=1832;ExcessHet=3;MLEAC=.;MLEAF=.;MQ=29   GT:DP:RGQ   0/0:2:0 0/0:4:0 0/0:2:0 0/0:1:0 0/0:1:0 0/0:1:0 0/0:1:0 0/0:1:3 0/0:8:18    0/0:34:99   0/0:14:33&#xA;NC_013991.2 3328    .   G   A   650,66  .   AC=15;AF=0.048;AN=312;BaseQRankSum=0;DP=1826;ExcessHet=0;FS=3.31;InbreedingCoeff=0.5371;MLEAC=12;MLEAF=0.038;MQ=34.34;MQRankSum=0;QD=27.23;ReadPosRankSum=0;SOR=0.324   GT:AD:DP:GQ:PGT:PID:PL:PS   1|1:0,2:2:6:1|1:3321_A_ATTTCATT:90,6,0:3321 1|1:0,4:4:12:1|1:3321_A_ATTTCATT:180,12,0:3321  1|1:0,2:2:6:1|1:3321_A_ATTTCATT:90,6,0:3321 ./.:0,1:1:.:.:.:0,0,0:. ./.:0,1:1:.:.:.:0,0,0:. ./.:0,1:1:.:.:.:0,0,0:. 0/0:1,0:1:3:.:.:0,3,45:.    0/0:1,0:1:3:.:.:0,3,25:.    0/0:8,0:8:18:.:.:0,18,270:. 0/0:34,0:34:99:.:.:0,99,1485:.  0/0:14,0:14:33:.:.:0,33,495:.&#xA;NC_013991.2 3329    .   G   .   18,49   LowQual BaseQRankSum=-1;DP=1818;ExcessHet=3;MLEAC=.;MLEAF=.;MQ=57.63;MQRankSum=0;ReadPosRankSum=1   GT:DP:RGQ   0/0:2:6 0/0:4:12    0/0:2:6 0/0:1:3 0/0:1:3 0/0:1:3 0/0:1:3 0/0:1:3 0/0:7:18    0/0:35:99   0/0:13:33&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It seems like only the &amp;quot;CATT&amp;quot; part of the alternative allele is really inserted...&lt;/p&gt;&#xA;&lt;p&gt;The mapping was done either with bwa aln or bwa mem, and we've observed this behaviour both when using GATK4 and bcftools for genotyping.&lt;/p&gt;&#xA;" OwnerUserId="16680" LastEditorUserId="16680" LastEditDate="2022-12-23T15:16:16.970" LastActivityDate="2022-12-27T12:03:44.033" Title="VCF: false insertions?" Tags="|vcf|snp|indel|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20235" PostTypeId="2" ParentId="20232" CreationDate="2022-12-23T12:39:46.547" Score="1" Body="&lt;p&gt;The standard and most common way to name SNPs is by using &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/docs/RefSNP_about/&quot; rel=&quot;nofollow noreferrer&quot;&gt;rsIDs&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The reason your output from PLINK does not include SNP names is because these are not included in the original VCF you are working with. There are at least two possible options in order to fix this.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Annotate first the source VCF with rsIDs from dbSNP. You can do this using bcftools and a recent (or latest) human rsID compilation in VCF format. E.g.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Downloading rsIDs&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;wget ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606_b151_GRCh37p13/VCF/All_20180423.vcf.gz&#xA;wget ftp://ftp.ncbi.nih.gov/snp/organisms/human_9606_b151_GRCh37p13/VCF/All_20180423.vcf.gz.tbi&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Subsetting chr22 from annotations&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This makes the annotation of the chr22 VCF faster&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view --threads 10 --regions 22 All_20180423.vcf.gz -Oz -o 22_20180423.vcf.gz&#xA;bcftools index 22_20180423.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;VCF Annotation&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools annotate --threads 10 -a 22_20180423.vcf.gz -c ID -o ALL.chr22.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes_annotated.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;PLINK frequency estimations&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Now you can proceed as you had done already&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plink --vcf ALL.chr22.phase3_shapeit2_mvncall_integrated_v5b.20130502.genotypes_annotated.vcf.gz --make-bed --out chr22data&#xA;plink --bfile chr22data --freq --out chr22freqs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;An alternative option is to build your own 'SNP names' based on chromosome and cordinate (e.g. 22_16255382) using plink commands.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="475" LastActivityDate="2022-12-23T12:39:46.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20236" PostTypeId="1" AcceptedAnswerId="20239" CreationDate="2022-12-23T12:56:31.833" Score="2" ViewCount="261" Body="&lt;p&gt;I'm using bowtie2 to build a small index. After building the index, I want to pass it to the bowtie2 alignment process. The issue is bowtie2-build outputs multiple index files with the .bt2 extension. When I try to provide these index files as an input to the Alignmnet process, it gives the following error:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error executing process &amp;gt; 'SAM_FOR_STRAND (1)'&lt;/p&gt;&#xA;&lt;p&gt;Caused by:   Process &lt;code&gt;SAM_FOR_STRAND (1)&lt;/code&gt; terminated with an error&#xA;exit status (255)&lt;/p&gt;&#xA;&lt;p&gt;Command executed:&lt;/p&gt;&#xA;&lt;p&gt;bowtie2 -t -x index.1.bt2 index.2.bt2 index.3.bt2 index.4.bt2&#xA;index.rev.1.bt2 index.rev.2.bt2 -1 SRR5655560_1.atria.fastq -2&#xA;SRR5655560_2.atria.fastq -S &amp;quot;SRR5655560.sam&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Command exit status:   255&lt;/p&gt;&#xA;&lt;p&gt;Command output:   (empty)&lt;/p&gt;&#xA;&lt;p&gt;Command error:   (ERR): &amp;quot;index.1.bt2&amp;quot; does not exist or is not a&#xA;Bowtie 2 index   Exiting now ...&lt;/p&gt;&#xA;&lt;p&gt;executor &amp;gt;  local (4) [71/c095e4] process &amp;gt; INDEX              [100%]&#xA;1 of 1 ✔ [1f/35a1e6] process &amp;gt; GTF2BED            [100%] 1 of 1 ✔&#xA;[fa/b1e933] process &amp;gt; TRIM (SRR5655560)  [100%] 1 of 1 ✔ [d6/d47ead]&#xA;process &amp;gt; SAM_FOR_STRAND (1) [100%] 1 of 1, failed: 1 ✘ Error&#xA;executing process &amp;gt; 'SAM_FOR_STRAND (1)'&lt;/p&gt;&#xA;&lt;p&gt;Caused by:   Process &lt;code&gt;SAM_FOR_STRAND (1)&lt;/code&gt; terminated with an error&#xA;exit status (255)&lt;/p&gt;&#xA;&lt;p&gt;Command executed:&lt;/p&gt;&#xA;&lt;p&gt;bowtie2 -t -x index.1.bt2 index.2.bt2 index.3.bt2 index.4.bt2&#xA;index.rev.1.bt2 index.rev.2.bt2 -1 SRR5655560_1.atria.fastq -2&#xA;SRR5655560_2.atria.fastq -S &amp;quot;SRR5655560.sam&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The following is simplifies version of the pipeline:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;// Declare syntax version&#xA;nextflow.enable.dsl=2&#xA;&#xA;/* &#xA; * pipeline input parameters &#xA; */&#xA;&#xA;&#xA;params.reads = &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$projectDir/data/*_{1,2}.fastq&quot;&#xA;params.ref_genome = &quot;$&lt;/span&gt;projectDir/data/Glycine_max.Glycine_max_v2.1.dna.toplevel.fa&amp;quot;&#xA;params.gtf_file = &amp;quot;$projectDir/data/Glycine_max.Glycine_max_v2.1.55.gtf&amp;quot;&#xA;params.outdir = &amp;quot;results&amp;quot;&#xA;&#xA;&#xA;&#xA;log.info &amp;quot;&amp;quot;&amp;quot;\&#xA;&#xA;        R N A S E Q - N F   P I P E L I N E    &#xA;        ===================================&#xA;        reads            : &lt;span class=&quot;math-container&quot;&gt;${params.reads}&#xA;        reference genome : $&lt;/span&gt;{params.ref_genome}&#xA;        gtf              : &lt;span class=&quot;math-container&quot;&gt;${params.gtf_file}&#xA;        outdir           : $&lt;/span&gt;{params.outdir}&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        .stripIndent()&#xA;&#xA;/*&#xA; * define the `index` process that creates a binary index&#xA; * given the reference genome file&#xA; */&#xA;&#xA;process INDEX {&#xA;&#xA;  input:&#xA;  path ref_genome&#xA;&#xA;  output:&#xA;  path &amp;quot;index.*&amp;quot;&#xA;&#xA;  script:&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  bowtie2-build $ref_genome index&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;/*&#xA; * define the `GTF2BED` process that creates a BED file&#xA; * given the params.gtf_file as input. &#xA; */&#xA; &#xA;process GTF2BED {&#xA;&#xA;    input:&#xA;    path gtf&#xA;&#xA;    output:&#xA;    path &amp;quot;${gtf.baseName}.bed&amp;quot;&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    gtf2bed --gtf &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${gtf}&quot; --bed &quot;$&lt;/span&gt;{gtf.baseName}.bed&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;&#xA;/*&#xA; * define the `TRIM` process that creates trimmed fastq files&#xA; * given the reads fastq file. &#xA; */&#xA; &#xA;process TRIM {&#xA;&#xA;    tag { pair_id }&#xA;&#xA;    input:&#xA;    tuple val(pair_id), path(reads)&#xA;&#xA;    output:&#xA;    tuple val(pair_id), path(trimmed_reads), emit: trimmed_fastqs&#xA;    path &amp;quot;${pair_id}/*.atria.log*&amp;quot;, emit: logs&#xA;&#xA;    script:&#xA;    def (r1, r2) = reads&#xA;&#xA;    trimmed_reads = reads.collect {&#xA;        &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${pair_id}/$&lt;/span&gt;{it.baseName}.atria.fastq&amp;quot;&#xA;    }&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    atria \\&#xA;        -r &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${r1}&quot; \\&#xA;        -R &quot;$&lt;/span&gt;{r2}&amp;quot; \\&#xA;        -o &amp;quot;${pair_id}&amp;quot; \\&#xA;        -t 1 \\&#xA;        -l 12 \\&#xA;        -q 30&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;/*&#xA; * define the `SAM_FOR_STRAND` process that creates a SAM file&#xA; * to be used by `STRAND_CHECK` process to determine &#xA; * the strandedness of each fastq file pair. &#xA; */&#xA; &#xA;process SAM_FOR_STRAND {&#xA;&#xA;    input:&#xA;    path index&#xA;    tuple val(pair_id), path(reads)&#xA;&#xA;    output:&#xA;    path &amp;quot;${pair_id}.sam&amp;quot;&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    bowtie2 -x &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${index.baseName}&quot; -1 $&lt;/span&gt;{reads[0]} -2 &lt;span class=&quot;math-container&quot;&gt;${reads[1]} -S $&lt;/span&gt;{pair_id}.sam&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;&#xA;workflow {&#xA;&#xA;    index_ch = INDEX(params.ref_genome)&#xA;&#xA;    gtf2bed_ch=GTF2BED(params.gtf_file)&#xA;&#xA;    read_pairs_ch = Channel.fromFilePairs(params.reads, checkIfExists:true)&#xA;&#xA;    trim_out = TRIM(read_pairs_ch)&#xA;&#xA;    sam_for_strand_ch = SAM_FOR_STRAND(index_ch, TRIM.out.trimmed_fastqs)&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've tried passing the &lt;em&gt;index&lt;/em&gt; argument directly to the script in the SAM_FOR_STRAND Alignmnet process, it gave the  &amp;quot;index.1.bt2&amp;quot; does not exist error. Then I tried giving the basename of the index files (as shown in the below). It still does not work, even though the index files are being created properly.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; /*&#xA;     * define the `SAM_FOR_STRAND` process that creates a SAM file&#xA;     * to be used by `STRAND_CHECK` process to determine &#xA;     * the strandedness of each fastq file pair. &#xA;     */&#xA;     &#xA;    process SAM_FOR_STRAND {&#xA;    &#xA;        input:&#xA;        path index&#xA;        tuple val(pair_id), path(reads)&#xA;    &#xA;        output:&#xA;        path &amp;quot;${pair_id}.sam&amp;quot;&#xA;    &#xA;        script:&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        bowtie2 -x &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${index.baseName}&quot; -1 $&lt;/span&gt;{reads[0]} -2 &lt;span class=&quot;math-container&quot;&gt;${reads[1]} -S $&lt;/span&gt;{pair_id}.sam&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Best wishes.&lt;/p&gt;&#xA;" OwnerUserId="16328" LastActivityDate="2022-12-23T14:28:26.750" Title="Alignment process in Nextflow not picking up the index files from the index process" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20238" PostTypeId="1" AcceptedAnswerId="20240" CreationDate="2022-12-23T14:01:07.690" Score="3" ViewCount="419" Body="&lt;p&gt;I have created three Nextflow processes: One checks the strandedness of the fastq file and outputs it to a .txt file. Second one executes a python script with different commands depending upon the condition. And the third one aligns them.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/*&#xA; * the `CHECK_STRANDEDNESS` process takes SAM file &#xA; * and a BED file&#xA; * and outputs a a txt file containing information about strandedness. &#xA; */&#xA; &#xA;process CHECK_STRANDEDNESS {&#xA;&#xA;    input:&#xA;    path &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${pair_id}.sam&quot;&#xA;    path &quot;$&lt;/span&gt;{gtf.baseName}.bed&amp;quot;&#xA;&#xA;    output:&#xA;    path &amp;quot;${pair_id}_strand_check.txt&amp;quot;&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    infer_experiment.py -i &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${pair_id}.sam&quot; -r &quot;$&lt;/span&gt;{gtf.baseName}.bed&amp;quot; &amp;gt; &amp;quot;${pair_id}_strand_check.txt&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;&#xA;/*&#xA; * the `FINDTAIL` process takes &amp;quot;${pair_id}_strand_check.txt&amp;quot; file  &#xA; * and executes commands according to the reads strandedness. &#xA; * The output generated is FASTA files.  &#xA; */&#xA; &#xA;process FINDTAIL {&#xA;&#xA;    input:&#xA;    path &amp;quot;${pair_id}_strand_check.txt&amp;quot;&#xA;    path tuple val(pair_id), path(reads)&#xA;&#xA;    output:&#xA;    &#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    #!/usr/bin/env python3&#xA;&#xA;    import subprocess&#xA;    import pandas as pd&#xA;    import os&#xA;&#xA;&#xA;    result = pd.read_csv(&amp;quot;${pair_id}_strand_check.txt&amp;quot;, sep=&amp;quot;\r\n&amp;quot;, header=None, engine='python')&#xA;&#xA;    failed = float(result.iloc[1,0].replace('Fraction of reads failed to determine: ', ''))&#xA;    fwd = float(result.iloc[2,0].replace('Fraction of reads explained by &amp;quot;1++,1--,2+-,2-+&amp;quot;: ', ''))&#xA;    rev = float(result.iloc[3,0].replace('Fraction of reads explained by &amp;quot;1+-,1-+,2++,2--&amp;quot;: ', ''))&#xA;    fwd_percent = fwd/(fwd+rev)&#xA;    rev_percent = rev/(fwd+rev)&#xA;&#xA;    if float(result.iloc[1,0].replace('Fraction of reads failed to determine: ', '')) &amp;gt; 0.50:&#xA;        cmd1 = &amp;quot;findtail_v1.01 --input_file &lt;span class=&quot;math-container&quot;&gt;${reads[0]} --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type sl --d &amp;gt; tail_$&lt;/span&gt;{pair_id}_1.fasta&#xA;        cmd2 = &amp;quot;findtail_v1.01 --input_file &lt;span class=&quot;math-container&quot;&gt;${reads[1]}  --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type sl --d &amp;gt; tail_$&lt;/span&gt;{pair_id}_2.fasta&#xA;        print(cmd1)&#xA;        subprocess.call(cmd1, shell=True)&#xA;        print(cmd2)&#xA;        subprocess.call(cmd2, shell=True)&#xA;&#xA;        if fwd_percent &amp;gt; 0.9:&#xA;            #Over 90% of reads explained by &amp;quot;1++,1--,2+-,2-+&#xA;            #Data is likely FR/fr-secondstrand&#xA;            cmd1 = &amp;quot;findtail_v1.01 --input_file &lt;span class=&quot;math-container&quot;&gt;${reads[0]} --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type pr --d &amp;gt; tail_$&lt;/span&gt;{pair_id}_1.fasta&#xA;            cmd2 = &amp;quot;findtail_v1.01 --input_file &lt;span class=&quot;math-container&quot;&gt;${reads[1]} --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type pr --d &amp;gt; tail_$&lt;/span&gt;{pair_id}_2.fasta&#xA;            print(cmd1)&#xA;            subprocess.call(cmd1, shell=True)&#xA;            print(cmd2)&#xA;            subprocess.call(cmd2, shell=True)&#xA;&#xA;        elif rev_percent &amp;gt; 0.9:&#xA;            # Over 90% of reads explained by &amp;quot;1+-,1-+,2++,2--&#xA;            # Data is likely RF/fr-firststrand&#xA;            cmd1 = &amp;quot;findtail_v1.01 --input_file &lt;span class=&quot;math-container&quot;&gt;${reads[0]} --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type sl --d &amp;gt; tail_$&lt;/span&gt;{pair_id}_1.fasta&#xA;            cmd2 = &amp;quot;findtail_v1.01 --input_file &lt;span class=&quot;math-container&quot;&gt;${reads[1]} --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type sl --d &amp;gt; tail_$&lt;/span&gt;{pair_id}_2.fasta&#xA;            print(cmd1)&#xA;            subprocess.call(cmd1, shell=True)&#xA;            print(cmd2)&#xA;            subprocess.call(cmd2, shell=True)&#xA;&#xA;        elif max(fwd_percent, rev_percent) &amp;lt; 0.6:&#xA;            #Under 60% of reads explained by one direction&#xA;            #Data is likely unstranded&#xA;            cmd1 = &amp;quot;findtail_v1.01 --input_file &lt;span class=&quot;math-container&quot;&gt;${reads[0]} --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type sl --d &amp;gt; tail_$&lt;/span&gt;{pair_id}_1.sl.fasta&#xA;            cmd2 = &amp;quot;findtail_v1.01 --input_file &lt;span class=&quot;math-container&quot;&gt;${reads[0]} --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type pr --d &amp;gt; tail_$&lt;/span&gt;{pair_id}_1.pr.fasta&#xA;            cmd3 = &amp;quot;findtail_v1.01 --input_file &lt;span class=&quot;math-container&quot;&gt;${reads[1]} --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type sl --d &amp;gt; tail_$&lt;/span&gt;{pair_id}_2.sl.fasta&#xA;            cmd4 = &amp;quot;findtail_v1.01 --input_file &lt;span class=&quot;math-container&quot;&gt;${reads[1]} --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type pr --d &amp;gt; tail_$&lt;/span&gt;{pair_id}_2.pr.fasta&#xA;            print(cmd1)&#xA;            subprocess.call(cmd1, shell=True)&#xA;            print(cmd2)&#xA;            subprocess.call(cmd2, shell=True)&#xA;            print(cmd3)&#xA;            subprocess.call(cmd3, shell=True)&#xA;            print(cmd4)&#xA;            subprocess.call(cmd4, shell=True)&#xA;&#xA;        else:&#xA;            #if strand couldnt be detected&#xA;            # we assume it is first-stranded&#xA;            # since most illumina reads are&#xA;            cmd1 = &amp;quot;findtail_v1.01 --input_file &lt;span class=&quot;math-container&quot;&gt;${reads[0]} --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type sl --d &amp;gt; tail_$&lt;/span&gt;{pair_id}_1.fasta&#xA;            cmd2 = &amp;quot;findtail_v1.01 --input_file &lt;span class=&quot;math-container&quot;&gt;${reads[1]} --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type sl --d &amp;gt; tail_$&lt;/span&gt;{pair_id}_2.fasta&#xA;            print(cmd1)&#xA;            subprocess.call(cmd1, shell=True)&#xA;            print(cmd2)&#xA;            subprocess.call(cmd2, shell=True)&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;/*&#xA; * define the `ALIGN` process that outputs unmapped files in FASTA format&#xA; * It will take the FASTA files creates by the `FINDTAIL` process &#xA; * and try to align them INDIVIDUALLY.&#xA; */&#xA; &#xA;process ALIGN {&#xA;&#xA;    input:&#xA;    path index&#xA;    path &amp;quot;tail_${pair_id}*.fasta&amp;quot;&#xA;&#xA;    output:&#xA;    path &amp;quot;unmapped_tail_${pair_id}*.fasta&amp;quot;&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    bowtie2 -x &lt;span class=&quot;math-container&quot;&gt;$index -f &quot;tail_$&lt;/span&gt;{pair_id}*.fasta&amp;quot; --un &amp;quot;unmapped_tail_${pair_id}*.fasta&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After creating a &lt;em&gt;&amp;quot;${pair_id}_strand_check.txt&amp;quot;&lt;/em&gt; file, I input it to the FINDTAIL process that finds reads with polyadenylated ends. It takes fastq files individually, so I have to run it individually for each file.&lt;/p&gt;&#xA;&lt;p&gt;If fastq file pair is first stranded or second stranded, I executed findtail twice,once for R1 and once for R2, which creates two fasta files. But if it is unstranded, I have to run findtail four times, twice for each file. Thus, depending on the paired-end reads' strandedness, I have different number of inputs. &lt;strong&gt;How do I define the output of this process?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the next process I align these files individually to get unmapped reads.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;This issue has been eating my brain away for 5 days. So please let me know if something is unclear.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Best wishes!&lt;/p&gt;&#xA;" OwnerUserId="16328" LastEditorUserId="16328" LastEditDate="2022-12-25T12:11:25.830" LastActivityDate="2022-12-25T12:11:25.830" Title="How to handle output from a python script that gives different outputs depending upon the condition - Nextflow" Tags="|nextflow|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20239" PostTypeId="2" ParentId="20236" CreationDate="2022-12-23T14:28:26.750" Score="1" Body="&lt;p&gt;When we run Bowtie2, we just need to make sure we supply the index basename using &lt;code&gt;-x&lt;/code&gt;. This is the index filename prefix minus the &lt;code&gt;.X.bt2&lt;/code&gt; extension. A safe assumption is that the first file in the collection of output files (sorted in lexicographical order) has the &lt;code&gt;.1.bt2&lt;/code&gt; suffix. We can then just strip this off to get the required prefix. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process SAM_FOR_STRAND {&#xA;&#xA;    input:&#xA;    path index_files, stageAs: &amp;quot;bt2_index/*&amp;quot;&#xA;    tuple val(pair_id), path(reads)&#xA;&#xA;    output:&#xA;    path &amp;quot;${pair_id}.sam&amp;quot;&#xA;&#xA;    script:&#xA;    def idx = index_files[0].getBaseName(2)    &#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    bowtie2 \\&#xA;        -x &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${idx}&quot; \\&#xA;        -1 &quot;$&lt;/span&gt;{reads[0]}&amp;quot; \\ &#xA;        -2 &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${reads[1]}&quot; \\ &#xA;        -S &quot;$&lt;/span&gt;{pair_id}.sam&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2022-12-23T14:28:26.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20240" PostTypeId="2" ParentId="20238" CreationDate="2022-12-23T16:25:42.950" Score="4" Body="&lt;p&gt;You should be able to just use a glob pattern that accounts for all possible output files, regardless of the specified conditions. As long as a single file matches the glob pattern when the process completes (successfully), Nextflow won't complain. I think this should suffice, but please let me know in the comments if it doesn't:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;output:&#xA;tuple val(pair_id), path(&amp;quot;tail_${pair_id}_{1,2}{,.pr,.sl}.fasta&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2022-12-23T16:25:42.950" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20241" PostTypeId="2" ParentId="20233" CreationDate="2022-12-23T17:03:02.753" Score="2" Body="&lt;p&gt;Without more code and data, this can't be debugged. But first question would be does the data pass basic QC (e.g. look at cellranger web summaries or equivalent). If yes, you could trying different normalisation (code not shown) and/or no scaling. Try &lt;code&gt;sc.tl.umap()&lt;/code&gt; with default params. Plot the PCA, try t-SNE. Plot the basic QC metrics following the introductory scanpy tutorial. That is how I would start debugging&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2022-12-23T17:03:02.753" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20242" PostTypeId="2" ParentId="20234" CreationDate="2022-12-23T22:42:42.000" Score="0" Body="&lt;p&gt;INDELs tend to occur tandemly. Your observation makes them more likely to be real.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2022-12-23T22:42:42.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20243" PostTypeId="2" ParentId="20230" CreationDate="2022-12-23T23:05:34.913" Score="2" Body="&lt;p&gt;To the best of my knowledge, the first joint calling model was published by &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/20980557/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Le &amp;amp; Durbin (2011)&lt;/a&gt; and refined in &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/21903627/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Li (2012)&lt;/a&gt;. They come with detailed derivation. Both samtools and older GATK are using this model. GATK now uses a different model which only works with high-coverage data but is faster to compute. There are other alternatives (e.g. in &lt;a href=&quot;https://arxiv.org/abs/1207.3907&quot; rel=&quot;nofollow noreferrer&quot;&gt;freebayes&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2022-12-23T23:05:34.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20244" PostTypeId="2" ParentId="20217" CreationDate="2022-12-24T00:33:59.087" Score="2" Body="&lt;p&gt;adding, a second answer because the OP request about&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I need to import the file and then highlight some specific word&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;was making me uneasy. I kept using the &lt;code&gt;Pyfpdf2&lt;/code&gt; library and made use of the Biopython &lt;code&gt;SeqRecord.letter_annotations&lt;/code&gt; attribute to track Nucleotides index composing the words found by RegEx. Tried to use the &lt;code&gt;page.search_for()&lt;/code&gt; of the &lt;code&gt;PyMuPDF&lt;/code&gt; library directly on the output pdf of my previous answer but was not getting word accross line breaks [if anybody know better, please help].&lt;/p&gt;&#xA;&lt;p&gt;Here my code, it's a two half problem, first find the word (in my example is '&lt;em&gt;atat&lt;/em&gt;'), then write everything to pdf, I stole it from &lt;a href=&quot;https://github.com/biopython/biopython/blob/8d9d20d09c6ee2a672c8477e890f0ab4e9e48825/Bio/AlignIO/ClustalIO.py#L22&quot; rel=&quot;nofollow noreferrer&quot;&gt;from /master/Bio/AlignIO/ClustalIO.py/   def write_alignment(self, alignment): from class ClustalWriter(SequentialAlignmentWriter)&lt;/a&gt; with some modding [I believe but not absolutely sure that ...A cell by definition is uniform in font and color... ]:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;&#xA;from Bio import AlignIO  # Biopython 1.80&#xA;&#xA;from fpdf import FPDF # pip install fpdf2&#xA; &#xA;&#xA;alignment = AlignIO.read(&amp;quot;Multi.txt&amp;quot;, &amp;quot;clustal&amp;quot;)&#xA;&#xA;&#xA;word_to_find = 'atat'&#xA;&#xA;for rec in alignment:&#xA;    &#xA;    rec.letter_annotations['highlight'] = 'N'*len(rec.seq)&#xA;&#xA;# for rec in alignment:    &#xA;#     print('rec.id : ', rec.id, 'highlight : ', rec.letter_annotations['highlight'])&#xA;    &#xA;    &#xA;for rec in alignment:&#xA;    &#xA;    # print(str(rec.seq))&#xA;    &#xA;    found = [m.start() for m in re.finditer( '(?='+re.escape(word_to_find)+')' , re.escape(str(rec.seq.replace('-', 'Z'))))]&#xA;    &#xA;    # print(rec.id , ' found : ', found)&#xA;    &#xA;    for i in found :&#xA;        &#xA;            a = list(rec.letter_annotations['highlight'])&#xA;    &#xA;            a[i:i+len(word_to_find)] = 'Y'*len(word_to_find)&#xA;            &#xA;            rec.letter_annotations['highlight'] = ''.join(i for i in a)&#xA;            &#xA;            # print(rec.letter_annotations['highlight'])&#xA;            &#xA;            &#xA;&#xA;max_length = alignment.get_alignment_length()&#xA;&#xA;&#xA;# stolen from /master/Bio/AlignIO/ClustalIO.py/   def write_alignment(self, alignment): from class ClustalWriter(SequentialAlignmentWriter):&#xA;def write_alignment(alignment):&#xA;    &#xA;        x = 2&#xA;        &#xA;        y = 2&#xA;&#xA;        pdf = FPDF(orientation=&amp;quot;P&amp;quot;, unit=&amp;quot;mm&amp;quot;, format=&amp;quot;A4&amp;quot;)&#xA;         &#xA;        # Add a page&#xA;        pdf.add_page()&#xA;&#xA;        pdf.add_font('FreeMono', '', 'FreeMono.ttf')&#xA;&#xA;        pdf.set_font(&amp;quot;FreeMono&amp;quot;, size = 12)&#xA;        &#xA;        pdf.set_fill_color(255, 255, 0)&#xA;&#xA;    &#xA;        &amp;quot;&amp;quot;&amp;quot;Use this to write (another) single alignment to an open file.&amp;quot;&amp;quot;&amp;quot;&#xA;        if len(alignment) == 0:&#xA;            raise ValueError(&amp;quot;Must have at least one sequence&amp;quot;)&#xA;        if alignment.get_alignment_length() == 0:&#xA;            # This doubles as a check for an alignment object&#xA;            raise ValueError(&amp;quot;Non-empty sequences are required&amp;quot;)&#xA;&#xA;        # Old versions of the parser in Bio.Clustalw used a ._version property&#xA;        try:&#xA;            version = str(alignment._version)&#xA;        except AttributeError:&#xA;            version = &amp;quot;&amp;quot;&#xA;        if not version:&#xA;            version = &amp;quot;1.81&amp;quot;&#xA;        if version.startswith(&amp;quot;2.&amp;quot;):&#xA;            # e.g. 2.0.x&#xA;            output = f&amp;quot;CLUSTAL {version} multiple sequence alignment&amp;quot;&#xA;        else:&#xA;            # e.g. 1.81 or 1.83&#xA;            output = f&amp;quot;CLUSTAL X ({version}) multiple sequence alignment&amp;quot;&#xA;            &#xA;        pdf.cell(0, y, txt = output , border = 0,  new_x='LMARGIN'  , new_y=&amp;quot;NEXT&amp;quot; ,  align = 'L', fill = False)&#xA;        pdf.cell(0, 4*y, txt = '' , border = 0,  new_x='LMARGIN'  , new_y=&amp;quot;NEXT&amp;quot; ,  align = 'L', fill = False)&#xA;        &#xA;&#xA;        cur_char = 0&#xA;        # # max_length = len(alignment[0])&#xA;        max_length = alignment.get_alignment_length()&#xA;&#xA;&#xA;        if max_length &amp;lt;= 0:&#xA;            raise ValueError(&amp;quot;Non-empty sequences are required&amp;quot;)&#xA;&#xA;        if &amp;quot;clustal_consensus&amp;quot; in alignment.column_annotations:&#xA;            star_info = alignment.column_annotations[&amp;quot;clustal_consensus&amp;quot;]&#xA;        else:&#xA;            try:&#xA;                # This was originally stored by Bio.Clustalw as ._star_info&#xA;                star_info = alignment._star_info&#xA;            except AttributeError:&#xA;                star_info = None&#xA;&#xA;        # keep displaying sequences until we reach the end&#xA;        &#xA;        rec_cnt = 0&#xA;        &#xA;        while cur_char != max_length:&#xA;            # calculate the number of sequences to show, which will&#xA;            # be less if we are at the end of the sequence&#xA;            if (cur_char + 50) &amp;gt; max_length:&#xA;                show_num = max_length - cur_char&#xA;            else:&#xA;                show_num = 50&#xA;&#xA;            # go through all of the records and print out the sequences&#xA;            # when we output, we do a nice 80 column output, although this&#xA;            # may result in truncation of the ids.&#xA;            for record in alignment:&#xA;                &#xA;                rec_cnt += 1 &#xA;                &#xA;                # Make sure we don't get any spaces in the record&#xA;                # identifier when output in the file by replacing&#xA;                # them with underscores:&#xA;                line = record.id[0:30].replace(&amp;quot; &amp;quot;, &amp;quot;_&amp;quot;).ljust(50)&#xA;                &#xA;                pdf.cell(50, y, txt = line, border = 0,  new_x='RIGHT' , align = 'L', fill = False)&#xA;&#xA;                &#xA;                line_seq = list(str(record.seq[cur_char : (cur_char + show_num)]))&#xA;                &#xA;                line_seq_highlight = list(str(record.letter_annotations['highlight'][cur_char : (cur_char + show_num)]))&#xA;                &#xA;                &#xA;                for i in range(len(line_seq)):&#xA;                    &#xA;                    if line_seq_highlight[i] == 'N' :&#xA;                        &#xA;                        filling = False&#xA;                    &#xA;                    if line_seq_highlight[i] == 'Y' :&#xA;                        &#xA;                        filling = True&#xA;                        &#xA;                    else:&#xA;                        &#xA;                        filling = False&#xA;                        &#xA;                    pdf.cell(x, y, txt = line_seq[i], border = 0,  new_x= 'RIGHT' ,  align = 'C', fill = filling)&#xA;                    &#xA;                pdf.cell(x, 2*y, txt = ' ' , border = 0,  new_x=&amp;quot;LMARGIN&amp;quot; , new_y=&amp;quot;NEXT&amp;quot; ,  align = 'L', fill = False)&#xA;                &#xA;                # now we need to print out the star info, if we've got it&#xA;                &#xA;                if star_info and rec_cnt == len(alignment):&#xA;                    &#xA;                    pdf.cell(x, 2*y, txt = ' ' , border = 0,  new_x=&amp;quot;LMARGIN&amp;quot; , new_y=&amp;quot;NEXT&amp;quot; ,  align = 'L', fill = False)&#xA;                    &#xA;                    rec_cnt = 0&#xA;                    &#xA;                    # print(star_info)&#xA;                    &#xA;                    line = (' ' * 50)&#xA;                    &#xA;                    pdf.cell(50, y, txt = line, border = 0,  new_x='RIGHT' , align = 'L', fill = False)&#xA;                    &#xA;                    star_info_seq = list(str(star_info[cur_char : (cur_char + show_num)]))&#xA;&#xA;                    for i in range(len(star_info_seq)):&#xA;                        &#xA;                        pdf.cell(x, y, txt = star_info_seq[i], border = 0,  new_x= 'RIGHT' ,  align = 'C', fill = False)&#xA;&#xA;&#xA;                    &#xA;                pdf.cell(x, 2*y, txt = ' ' , border = 0,  new_x=&amp;quot;LMARGIN&amp;quot; , new_y=&amp;quot;NEXT&amp;quot; ,  align = 'L', fill = False)&#xA;                &#xA;            &#xA;                &#xA;            cur_char += show_num  &#xA;            &#xA;            pdf.cell(x, 3*y, txt = ' ' , border = 0,  new_x=&amp;quot;LMARGIN&amp;quot; , new_y=&amp;quot;NEXT&amp;quot; ,  align = 'L', fill = False)&#xA;&#xA;&#xA;        &#xA;        pdf.output(&amp;quot;out.pdf&amp;quot;) &#xA;&#xA;&#xA;write_alignment(alignment)&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tested it, with the &lt;code&gt;Multi.txt&lt;/code&gt; Clustal alignment from above, output is&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;out.pdf&lt;/code&gt; :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/h6sMB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/h6sMB.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;.&lt;/p&gt;&#xA;&lt;p&gt;One last note:&lt;/p&gt;&#xA;&lt;p&gt;needed to replace &lt;code&gt;-&lt;/code&gt; with &lt;code&gt;Z&lt;/code&gt; in the Consensus string to be able to get reliable results from the RegEX line : &lt;code&gt;found = [m.start() for m in re.finditer( '(?='+re.escape(word_to_find)+')' , re.escape(str(rec.seq.replace('-', 'Z'))))]&lt;/code&gt; , not sure what is going on here, got the snippet from a SO post.&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2022-12-24T00:47:27.317" LastActivityDate="2022-12-24T00:47:27.317" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20247" PostTypeId="1" AcceptedAnswerId="20249" CreationDate="2022-12-24T11:46:22.773" Score="2" ViewCount="710" Body="&lt;p&gt;I have a FASTA file that has reads like this:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;gt;SRR5655563.745 745 length=126 (type=T,start=2,end=12,length=11,identity=81.8182%) CttttatttgttGTGTGTGAGGTTTGATTTGATGGGAAAATATCTTGAATCTGCGGCGAGGTTGGAAGAGCTATCGCGGATTGTGTCATCTGCTGCGAAGCCCAATAGGTCAAAGGGAACGCTACC&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;As you can see, there is information about the start &amp;quot;&lt;code&gt;start=2&lt;/code&gt;&amp;quot; and end &amp;quot;&lt;code&gt;end=12&lt;/code&gt;&amp;quot; of a sequence within the header. I would like to slice the sequence like [start:end] and keep the rest of it. e.g&lt;/p&gt;&#xA;&lt;p&gt;The part I would like to trim:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Ctttggtttcctttt&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And after trimming, I would like to keep the rest of the read:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;GTGTGTGAGGTTTGATTTGATGGGAAAATATCTTGAATCTGCGGCGAGGTTGGAAGAGCTATCGCGGATTGTGTCATCTGCTGCGAAGCCCAATAGGTCAAAGGGAACGCTACC&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So the new FASTA file will have reads like this:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;gt;SRR5655563.745 745 length=126 (type=T,start=2,end=12,length=11,identity=81.8182%)&#xA;GTGTGTGAGGTTTGATTTGATGGGAAAATATCTTGAATCTGCGGCGAGGTTGGAAGAGCTATCGCGGATTGTGTCATCTGCTGCGAAGCCCAATAGGTCAAAGGGAACGCTACC&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I have written a python script that takes a FASTA file as an input and outputs a trimmed FASTA file, like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tailtrim.py --fasta input.fasta --output output.tail.trimmed.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The code does trim the reads where I want it to.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;import argparse&#xA;import re&#xA;&#xA;def main():&#xA;    parser = argparse.ArgumentParser()&#xA;&#xA;    parser.add_argument(&amp;quot;--fasta&amp;quot;, type=str, required=True, help=&amp;quot;input fasta file&amp;quot;)&#xA;    parser.add_argument(&amp;quot;--output&amp;quot;, type=str, required=True, help=&amp;quot;output tail.trimmed.fasta file&amp;quot;)&#xA;&#xA;    args = parser.parse_args()&#xA;&#xA;    out_fasta = open(args.tail.trimmed.fasta, 'w')&#xA;&#xA;    for line in open(args.fasta):&#xA;        header = line.rstrip()&#xA;        read = next(args.fasta).rstrip()&#xA;&#xA;    #Extract header into a list&#xA;        header_lst = []&#xA;        header_lst= header.split(&amp;quot; &amp;quot;)&#xA;    &#xA;        tmp_start_lst = []&#xA;        tmp_end_lst = []&#xA;        tmp_lst = []&#xA;&#xA;        for i in header_lst:&#xA;            if i.startswith(&amp;quot;(&amp;quot;) and i.endswith(&amp;quot;)&amp;quot;):&#xA;                tmp_lst.append(i.split(&amp;quot;,&amp;quot;))&#xA;        #print(tmp_lst)&#xA;&#xA;    # Extract the start and end numbers in a list&#xA;        for i in tmp_lst:&#xA;            for str in i: &#xA;                if str.startswith(&amp;quot;start=&amp;quot;):&#xA;                    tmp_start_lst.append(str)&#xA;                if str.startswith(&amp;quot;end=&amp;quot;):&#xA;                    tmp_end_lst.append(str)&#xA;&#xA;        pattern = re.compile(r'\d+$')&#xA;&#xA;        start_lst = []&#xA;        for str in tmp_start_lst:&#xA;            match = re.search(pattern, str)&#xA;            start_lst.append(int(match.group()))&#xA;&#xA;        end_lst = []&#xA;        for str in tmp_end_lst:&#xA;            match = re.search(pattern, str)&#xA;            end_lst.append(int(match.group()))&#xA;&#xA;        #Extract reads into a list to manipulate i.e. trim&#xA;        read_lst = []&#xA;        read_lst.append(read)&#xA;&#xA;        new_lst = []&#xA;        &#xA;        for str in read_lst:&#xA;            for num_start in start_lst:&#xA;                for num_end in end_lst:&#xA;                    s = str.replace(str[(num_start-1):num_end],&amp;quot;&amp;quot;)&#xA;                    new_lst.append(s)&#xA;        &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I do get the trimmed reads, but what I'm having trouble figuring out is how to write these trimmed reads with their corresponding headers to a new FASTA file.&lt;/p&gt;&#xA;&lt;p&gt;Also if there is any way that I could improve my code or employ better strategy to approach the problem, I would love to know.&lt;/p&gt;&#xA;&lt;p&gt;Best wishes.&lt;/p&gt;&#xA;&lt;p&gt;Edit: I tried running the solution by the super helpful user @Steve.&lt;/p&gt;&#xA;&lt;p&gt;The command I ran:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;python3 tailTrimming.py --fasta tailtrim.fasta --output testtrim.trim.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The command line error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &amp;quot;/home/x/Desktop/fastq/tailTrimming.py&amp;quot;, line 61, in &amp;lt;module&amp;gt;&#xA;    main()&#xA;  File &amp;quot;/home/x/Desktop/fastq/tailTrimming.py&amp;quot;, line 58, in main&#xA;    SeqIO.write(trim_reads(fasta), out, &amp;quot;fasta&amp;quot;)&#xA;  File &amp;quot;/home/x/.local/lib/python3.10/site-packages/Bio/SeqIO/__init__.py&amp;quot;, line 478, in write&#xA;    count = writer_class(fp).write_file(sequences)&#xA;  File &amp;quot;/home/x/.local/lib/python3.10/site-packages/Bio/SeqIO/Interfaces.py&amp;quot;, line 209, in write_file&#xA;    count = self.write_records(records)&#xA;  File &amp;quot;/home/x/.local/lib/python3.10/site-packages/Bio/SeqIO/Interfaces.py&amp;quot;, line 193, in write_records&#xA;    for record in records:&#xA;  File &amp;quot;/home/x/Desktop/fastq/tailTrimming.py&amp;quot;, line 20, in trim_reads&#xA;    result = match.groupdict()&#xA;AttributeError: 'NoneType' object has no attribute 'groupdict'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the whole input file I supplied:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;SRR5655563.9676 9676 length=126 (type=T,start=1,end=11,length=11,identity=81.8182%)&#xA;tttctttctttAGTTCCAATAGTCTTGACCACCTTTTATCAACAGAAGCTTGGCCTTGTCCAGTTATATATTGTATTGTACCCTCAGATTTCTCGCACAAT&#xA;&amp;gt;SRR5655563.10087 10087 length=126 (type=T,start=2,end=16,length=15,identity=80%)&#xA;GtttgtttcattttttCCAAGGAAGAAATTCGACCTTCCAGCGATTTGTGGAGGTCATTCATCTGTTGAGTAAAGGATTGGTGCGCGACCTTGTAATCCTTGTGGAGGGTGTAG&#xA;&amp;gt;SRR5655563.13902 13902 length=126 (type=T,start=2,end=16,length=15,identity=80%)&#xA;CtttttcttcttgtttGATGTAAAAAATCTTCGGTATCTAAAACGTATGAAACCTTAGATGAACTGCAAGCTATCTTACACTCTAGACCATTGATAATTT&#xA;&amp;gt;SRR5655563.18489 18489 length=126 (type=T,start=2,end=12,length=11,identity=81.8182%)&#xA;CtttttcttcttCTTGGAAGTTTTGGATTCTGCAGGGACATTTTCTTTCTTTTCTCCATCTCCATCAGTCTCAACACCTGTCTTATCCTGAGCACCTTGAGACTCATCTTGACCATCTTTACTTT&#xA;&amp;gt;SRR5655563.20782 20782 length=126 (type=T,start=2,end=15,length=14,identity=71.4286%)&#xA;CttcattttagttttCAACTCCTCTACCAACTGGGATGGTGCATTCAACTGTAGAACTGTTTCACGAATCTCCTTGAGAACACTGAAGTCTCCCTCAGTTAACTTCTTTTTCAAATTATTGA&#xA;&amp;gt;SRR5655563.23972 23972 length=126 (type=T,start=2,end=29,length=28,identity=75%)&#xA;CttttttgttgttctcttctttcttctttGCATGGCTTGATGATGACTGATCCGATTGATTATTTCTTGATAGAATATCAATGTCATCACCAATTGTTGAGT&#xA;&amp;gt;SRR5655563.31739 31739 length=126 (type=T,start=1,end=10,length=10,identity=80%)&#xA;ttttcttcttCTTGCATGAAAGTGGGTTTGGACCCTTAGCTCTTTTTCTCTTGAACTGAGGTTT&#xA;&amp;gt;SRR5655563.34244 34244 length=126 (type=T,start=2,end=14,length=13,identity=76.9231%)&#xA;CtttcctttattttCTTGGAGGTTAGCCAGGATATTGGGATCAAAGCATGTAGCAAAACACACCTTGCTATTTCTTGCTCGTACAGATACATGGGATAGGA&#xA;&amp;gt;SRR5655563.38334 38334 length=126 (type=T,start=2,end=12,length=11,identity=81.8182%)&#xA;CtttattgttttCTCATCACCCATATTATATTTCTTAAGGGCAGATTCCAACTCTTCCATTGTGATGTACCCGCTCTTATCATTGTCAAAATATTCAAAGGCTTTATATAGGCGATCCTCTCTTTC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And this is the whole output I have gotten:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;SRR5655563.9676 9676 length=126 (type=T,start=1,end=11,length=11,identity=81.8182%)&#xA;AGTTCCAATAGTCTTGACCACCTTTTATCAACAGAAGCTTGGCCTTGTCCAGTTATATAT&#xA;TGTATTGTACCCTCAGATTTCTCGCACAAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Just one read, that's all. Whereas I would like to keep all the sequences because all of them have this &lt;code&gt;start&lt;/code&gt;, &lt;code&gt;end&lt;/code&gt; attribute. The sequence being on two lines in the output is not something I did. It is like this in the output file. So it's no longer FASTA file, right?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance for any help!&lt;/p&gt;&#xA;" OwnerUserId="16328" LastEditorUserId="16328" LastEditDate="2022-12-28T04:59:29.040" LastActivityDate="2023-02-10T21:19:35.993" Title="How to manually trim FASTA file sequences with the information provided in the header and store it into a new FASTA file? - Python" Tags="|python|fasta|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20248" PostTypeId="2" ParentId="20230" CreationDate="2022-12-24T12:38:43.137" Score="2" Body="&lt;p&gt;Here is a recent example of the derivation of the genotype model from &lt;a href=&quot;https://www.nature.com/articles/s41587-021-00861-3#Sec11&quot; rel=&quot;nofollow noreferrer&quot;&gt;Octopus genotype calling paper&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;All samples have known ploidy and copy number, so the likelihood function of reads &lt;span class=&quot;math-container&quot;&gt;$R$&lt;/span&gt; given genotype &lt;span class=&quot;math-container&quot;&gt;$g$&lt;/span&gt; is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p(R|g) = \mathop {\prod }\limits_{n = 1}^{|R|} \frac{1}{{|g|}}\mathop {\sum }\limits_{i = 1}^{|g|} p(r_n|g_i)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where &lt;span class=&quot;math-container&quot;&gt;$|g|$&lt;/span&gt; is the ploidy and &lt;span class=&quot;math-container&quot;&gt;$|R|$&lt;/span&gt; is the number of reads. The joint genotype posterior for &lt;span class=&quot;math-container&quot;&gt;$S$&lt;/span&gt; samples is therefore given by:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$p({\bf{g}}|{{R}},{\cal{M}}_{{g}}) \propto p({\bf{g}}|{\cal{M}}_g)\mathop {\prod }\limits_{s = 1}^S p({{R}}_s|{\bf{g}}_s)$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where the genotype prior model, &lt;span class=&quot;math-container&quot;&gt;$\cal{M}$&lt;/span&gt;, is either the uniform or HWE-coalescent prior. Unfortunately, the number of genotype combinations g grows exponentially in the number of samples S, so we cannot evaluate the full posterior distribution in general. Therefore, other than for trivial cases, we first approximate the sample marginal genotype posterior distribution under the HWE model (without mutations) &lt;span class=&quot;math-container&quot;&gt;$(|,\cal{M})$&lt;/span&gt; and use these marginal probabilities to select K genotype combinations &lt;span class=&quot;math-container&quot;&gt;$1,…,$&lt;/span&gt; (K is user-defined) to evaluate under the full joint genotype model. The approximate posterior marginals are computed with expectation maximization.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The source code can be found &lt;a href=&quot;https://github.com/luntergroup/octopus/blob/develop/src/core/callers/population_caller.cpp&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="3694" LastEditDate="2022-12-28T12:34:05.457" LastActivityDate="2022-12-28T12:34:05.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20249" PostTypeId="2" ParentId="20247" CreationDate="2022-12-24T14:12:46.383" Score="3" Body="&lt;p&gt;Here's one way using &lt;a href=&quot;https://biopython.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biopython&lt;/a&gt; and the &lt;a href=&quot;https://biopython.org/wiki/SeqIO&quot; rel=&quot;nofollow noreferrer&quot;&gt;SeqIO&lt;/a&gt; interface to read and write &lt;a href=&quot;https://biopython.org/wiki/SeqRecord&quot; rel=&quot;nofollow noreferrer&quot;&gt;SeqRecord&lt;/a&gt; objects. Using a generator, we can produce (yield) trimmed sequence records that can then be written to the output handle. This avoids storing an entire list of trimmed sequences in memory and is usually the way to go, assuming you're dealing with large files containing thousands of records:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import argparse&#xA;import re&#xA;&#xA;from Bio import SeqIO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;def trim_reads(fileobj):&#xA;&#xA;    pattern = (&#xA;        r'type=(?P&amp;lt;type&amp;gt;\w+),'&#xA;        r'start=(?P&amp;lt;start&amp;gt;\d+),'&#xA;        r'end=(?P&amp;lt;end&amp;gt;\d+),'&#xA;        r'length=(?P&amp;lt;length&amp;gt;\d+),'&#xA;        r'identity=(?P&amp;lt;identity&amp;gt;\d+(?:\.\d+)?)%'&#xA;    )&#xA;&#xA;    for seq_record in SeqIO.parse(fileobj, &amp;quot;fasta&amp;quot;):&#xA;        match = re.search(pattern, seq_record.description)&#xA;        result = match.groupdict()&#xA;&#xA;        start = int(result['start']) - 1&#xA;        end = int(result['end'])&#xA;&#xA;        head = seq_record.seq[:start]&#xA;        tail = seq_record.seq[end:]&#xA;&#xA;        seq_record.seq = head + tail&#xA;&#xA;        yield seq_record&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;def get_argument_parser():&#xA;&#xA;    parser = argparse.ArgumentParser()&#xA;&#xA;    parser.add_argument(&#xA;        &amp;quot;--fasta&amp;quot;,&#xA;         type=str,&#xA;         required=True,&#xA;         help=&amp;quot;input fasta file&amp;quot;,&#xA;    )&#xA;    parser.add_argument(&#xA;        &amp;quot;--output&amp;quot;,&#xA;        type=str,&#xA;        required=True,&#xA;        help=&amp;quot;output tail.trimmed.fasta file&amp;quot;,&#xA;    )&#xA;&#xA;    return parser&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;def main():&#xA;&#xA;    parser = get_argument_parser()&#xA;    args = parser.parse_args()&#xA;&#xA;    with open(args.fasta) as fasta, open(args.output, &amp;quot;w&amp;quot;) as out:&#xA;        SeqIO.write(trim_reads(fasta), out, &amp;quot;fasta-2line&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;if __name__ == &amp;quot;__main__&amp;quot;:&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2022-12-28T14:33:02.990" LastActivityDate="2022-12-28T14:33:02.990" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="20250" PostTypeId="1" CreationDate="2022-12-24T20:54:12.353" Score="1" ViewCount="69" Body="&lt;p&gt;I am trying to extract the hydrogen atoms attached to the Ca carbon in N no. of proteins. I would be needing the coordinates to calculate its centroid. How should I do it in python ?&lt;/p&gt;&#xA;" OwnerUserId="16691" LastEditorUserId="10049" LastEditDate="2022-12-27T11:49:19.993" LastActivityDate="2022-12-28T18:53:57.207" Title="How can I get the coordinates of Hydrogen atoms attached to alpha-carbon?" Tags="|python|proteins|protein-structure|pdb|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20251" PostTypeId="1" CreationDate="2022-12-25T11:39:33.647" Score="0" ViewCount="68" Body="&lt;p&gt;I am tasked with designing primers for a &lt;a href=&quot;https://www.gabi-kat.de/db/showseq.php?line=165A12&quot; rel=&quot;nofollow noreferrer&quot;&gt;particular mutant&lt;/a&gt; (Target Gene Locus: At1g28490 i.e. SYP61) of Arabidopsis, obtained from Gabi-Kat:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The NCBI reference sequence for the genome &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NM_102617.4?report=fasta&quot; rel=&quot;nofollow noreferrer&quot;&gt;goes&lt;/a&gt;:&#xA;GGCTTTATAAGATTCTCCAGTAGCCCAAAAATAAACTTTCAGCGAATCTCAAAGCCAACCGCAGGTGATGCGTTTTGCTATGTAAGAGAGACGAATCTCACGCCGGAATCTCTTTTACTTCACCGTCGTTTGTTCTCCGTCGCGTCACCTTTCGATTAATTCCTTCACAGGGTAGCAAAATTTTGTTGATTTCGGG&lt;span class=&quot;math-container&quot;&gt;$\color{red}{\textrm{[}}$&lt;/span&gt;ATGTCTTCAGCTCAAGATCCATTCTACATTGTTAAAGAAGAGATTCAGGATTCTATTGATAAGTTGCAATCTACATTCCACAAATGGG&lt;strong&gt;AGCG&lt;/strong&gt;T&lt;strong&gt;ATCTCTCCCGACATGG&lt;/strong&gt;G&lt;strong&gt;TGATCAAGCACATGTGGCGAAGGAGCTTGTTGCTACTTGTGGAAGCATTGAGTGGCAGGT&lt;/strong&gt;A͟G͟A͟T͟G͟A͟G͟C͟T͟G͟G͟A͟A͟A͟A͟A͟G͟C͟G͟A͟T͟T͟A͟C͟T͟G͟T͟T͟G͟C͟A͟G͟C͟A͟A͟A͟A͟G͟A͟T͟C͟C͟T͟T͟C͟ATGGTATGGCATTGATGAAGCTGAGCTTGAAAAACGGAGGAGATGGACTAGTAATGCTAGGACACAGGTGCGGAATGTGAAGTCTGGTGTTTTAGCTGGAAAGGTTAGTTCTGGAGCAGGTCATGCAAGTGAAGTGCGTAGAGAACTAATGAGAATGCCAAACTCGGGTGAAGCAAGTAGATATGATCAGTATGGAGGGAGAGACGATGATGGTTTTGTACAGTCAGAATCAGATAGGCAAATGCTGTTGATAAAGCAACAAGACGAAGAATTGGATGAGCTTAGTAAAAGTGTACAGAGAATTGGAGGAGTGGGGCTCACTATACACGACGAACTCGTTGCACAGGAGAGAATAATAGATGAATTGGATACGGAGATGGACAGCACAAAGAACCGTCTTGAATTTGTTCAGAAAAAAGTGGGAATGGTAATGAAGAAAGCTGGAGCAAAAGGTCAAATGATGATGATATGTTTCTTGCTCGTCTTGTTCATCATCCTATTCGTTCTTGTCTTCTTGACCTAA&lt;span class=&quot;math-container&quot;&gt;$\color{red}{\textrm{]}}$&lt;/span&gt;AGAACAGAGCCGCGGGATCTTATAGATGGGAGTTTAATGTCTTACATTATCATGTTTGTGGTGAAACAAATGGAATATTTACATAAGGTTGTCTGTTTTTTCTTCTCCTATCCAATCCCACATATAATTTTGAACTTTGGCTGTGTGATCTCTAGAGAACCTGAAATTAAATACTTTCATAGCTTGATCCCATGGCTAAGTTCCT&lt;/li&gt;&#xA;&lt;li&gt;Gabi-Kat &lt;a href=&quot;https://www.gabi-kat.de/db/showseq.php?line=165A12&quot; rel=&quot;nofollow noreferrer&quot;&gt;provides&lt;/a&gt; the following sequence from the mutant as a confirmatory match with the genome locus:&#xA;&lt;strong&gt;AGCG&lt;/strong&gt;G&lt;strong&gt;ATCTCTCCCGACATGG&lt;/strong&gt;C&lt;strong&gt;TGATCAAGCACATGTGGCGAAGGAGCTTGTTGCTACTTGTGGAAGCATTGAGTGGCAGGT&lt;/strong&gt;C͟C͟T͟C͟C͟T͟A͟T͟T͟C͟A͟C͟C͟C͟A͟C͟G͟G͟T͟T͟C͟T͟A͟T͟T͟A͟A͟T͟C͟C͟C͟C͟T͟C͟T͟T͟T͟A͟T͟A͟T͟A͟T͟A͟A͟&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I have &lt;strong&gt;bold-faced&lt;/strong&gt; the common region to both sequences. What explains the &lt;em&gt;complete mismatch&lt;/em&gt; of the trailing part (u͟n͟d͟e͟r͟l͟i͟n͟e͟d͟) of the Gabi-Kat sequence with the ref genome?&lt;/p&gt;&#xA;&lt;p&gt;It is &lt;strong&gt;not&lt;/strong&gt; the case that the mutation happened there; we are in the coding region (demarcated by red square brackets in the NCBI sequence) while Gabi-Kat's mutation is in &lt;a href=&quot;https://www.gabi-kat.de/faq.html#faq9&quot; rel=&quot;nofollow noreferrer&quot;&gt;an untranslated region&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Update 1:&lt;/h2&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://plants.ensembl.org/Arabidopsis_thaliana/Tools/Blast/Results?tl=5YJb4Lf7RtlEb7qc-21959324-2406285229&quot; rel=&quot;nofollow noreferrer&quot;&gt;Running&lt;/a&gt; Ensembl Blast on the GabiKat seq returns a 92.9% identicality with SYP61. Their sequence for the gene (taken from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NC_003070.9?report=fasta&quot; rel=&quot;nofollow noreferrer&quot;&gt;TAIR10.1&lt;/a&gt;) goes:&lt;/p&gt;&#xA;&lt;p&gt;AGCGGATCTCTCCCGACATGGCTGATCAAGCACATGTGGCGAAGGAGCTTGTTGCTACTTGTGGAAGCATTGAGTGGCAGGTA͟C͟T͟C͟G͟T͟A͟T͟T͟T͟A͟G͟T͟C͟A͟C͟G͟G͟T͟T͟C͟T͟A͟T͟T͟A͟A͟T͟C͟T͟C͟G͟T͟C͟T͟T͟T͟A͟T͟A͟T͟A͟T͟A͟A͟&lt;/p&gt;&#xA;&lt;p&gt;Note the difference in the trailing part from NCBI ref; this almost coincides with the Gabi-Kat seq!&lt;/p&gt;&#xA;&lt;p&gt;So, is the NCBI ref-genome and &lt;a href=&quot;https://www.arabidopsis.org/servlets/TairObject?type=sequence&amp;amp;id=2002978274&quot; rel=&quot;nofollow noreferrer&quot;&gt;the data for the particular TAIR genome&lt;/a&gt; inaccurate? Maybe, TAIR updated their whole genome set but forgot to update data for individual genes?&lt;/p&gt;&#xA;&lt;h2&gt;Update 2:&lt;/h2&gt;&#xA;&lt;p&gt;My proposed explanation for the anomaly makes no sense. TAIR 7 (2004) has the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NC_003070.5?report=fasta&quot; rel=&quot;nofollow noreferrer&quot;&gt;same sequence&lt;/a&gt;. It looks like I am missing out on something more fundamental.&lt;/p&gt;&#xA;" OwnerUserId="15189" LastEditorUserId="15189" LastEditDate="2022-12-26T10:33:36.080" LastActivityDate="2024-01-20T16:00:25.043" Title="Mismatch between mutant gene and reference gene outside the site of mutation" Tags="|gene|genomics|blastn|blastp|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20252" PostTypeId="2" ParentId="20251" CreationDate="2022-12-25T18:17:44.710" Score="0" Body="&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt; The evidence suggests the region you are looking at &lt;strong&gt;Gabi-kat&lt;/strong&gt; appears either 27045 downstream or 192250 upstream of syntaxin SNARE (SYP61), but is definitely not within the gene. It is chance homology (e.g. a pseudo-gene) that appears to occur within syntaxin (SYP61). The intergenic region could be much closer to the SYP61 gene if the same reference genome was used for both analyses Genbank LR782542.1 versus Genbank NC_003070.9 (different reference genomes in my analysis). Hence, the University is probably correct.&lt;/p&gt;&#xA;&lt;p&gt;What I suggest is the genome of &lt;em&gt;A. thaliana&lt;/em&gt; is downloaded and Blast analysis is performed locally. This way differences in the reference genome are removed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;The analysis is as follows&lt;/em&gt;:&lt;/p&gt;&#xA;&lt;p&gt;The target protein in question is involved in vesicle trafficking (&lt;a href=&quot;https://www.ebi.ac.uk/interpro/entry/InterPro/IPR000727/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) called SNARE, comprising coiled-coil domains and are highly conserved, for example show 100% conservation between &lt;em&gt;Arabidopsis thaliana&lt;/em&gt;, &lt;em&gt;Arabidopsis suecica&lt;/em&gt; and   &lt;em&gt;Camelina sativa&lt;/em&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;MSSAQDPFYIVKEEIQDSIDKLQSTFHKWERISPDMGDQAHVAKELVATCGSIEWQVDELEKAITVAAKDP&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If Gabi-kat was a protein its sequence would be ...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;RISPDMADQAHVAKELVATCGSIEWQVLLFTHGSINPLFIY&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Here's a blast of the whole regions of the Gabi-kat. The region in green is the SNARE proteins ...&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uBRgA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uBRgA.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So the issue is what is the terminal 3' protein and Blast that, i.e. the final bit that matches no known plant and the locus you flagged as being anomalous ..&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6WqUm.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6WqUm.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is basically zero result via Blast, the protein with low homology are lower metazoans (animals) and a few bacteria ... a long way from plants. In other words this isn't a known protein of natural origin at least on NCBI.&lt;/p&gt;&#xA;&lt;p&gt;The locus is literally like no protein on earth.&lt;/p&gt;&#xA;&lt;p&gt;However ... Blasting the nucleotide ...&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Reference &lt;em&gt;Arabidopsis thaliana&lt;/em&gt; = syntaxin (SNARE; SYP61)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Gabi-Kat &lt;em&gt;Arabidopsis thaliana&lt;/em&gt; top two Blast hits are ...&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Chromosome 1 at 10039082 - 10039208, or 9819787 to 981991 or&lt;/li&gt;&#xA;&lt;li&gt;Both value    1e-42&lt;/li&gt;&#xA;&lt;li&gt;Both dentity is 93%&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So Gabi-Kat is a real part of the &lt;em&gt;Arabidopsis thaliana&lt;/em&gt;, &lt;strong&gt;Gabi-kat is a non-coding region&lt;/strong&gt;. &lt;em&gt;Thats the major conclusion&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The question where is the location of SYP61? And that is&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Chromosome 1 at 10012037 to 10020311&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Described here from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene?Db=gene&amp;amp;Cmd=DetailsSearch&amp;amp;Term=839749&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI's full report&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/u46cD.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/u46cD.gif&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thus the mutant is 27045bp downstream or 192250 bp upstream &lt;strong&gt;IF&lt;/strong&gt; the same reference genome is used for the blasting (Genome LR782542.1) and the Genbank report NC_003070.9. They could easily be much closer to each other is the same reference genome was used for both sets of analysis.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To answer the comments&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I [the OP] do not follow your [my] blast analysis&lt;/li&gt;&#xA;&lt;li&gt;My [the OP] blasts gives ... ['hits within the gene']&lt;/li&gt;&#xA;&lt;li&gt;etc ....&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;It appears to me there is a psuedo-gene phenomenon occurring which is making you think a mutant is located within a gene. However, if you disagree thats perfectly okay and the original version of this answer I had assumed the Gaba-kat team had got this wrong (I always from the basis of supporting the OP).&lt;/p&gt;&#xA;&lt;p&gt;NCBI was the database repository in the remote analysis, but there's an issue with multiple references ... so to reiterate I suggest you perform this analysis as a local blast to remove multiple reference phenomena.&lt;/p&gt;&#xA;&lt;p&gt;To conclude I personally don't think there is a basis to believe the University responsible for Gaba-kat has got this wrong &lt;a href=&quot;https://www.gabi-kat.de&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.gabi-kat.de&lt;/a&gt;. However, if you feel otherwise then I would recommend contacting them with your analysis and any supportive analysis for example from this thread or elsewhere. Please note, I don't have a connection with this university.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-12-26T14:05:28.910" LastActivityDate="2022-12-26T14:05:28.910" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20253" PostTypeId="1" AcceptedAnswerId="20259" CreationDate="2022-12-26T07:53:11.723" Score="2" ViewCount="62" Body="&lt;p&gt;I was wondering does the gnomad genome data that is downloaded in the vcf format on variants contain information on what is the nearest gene and is the genomic location available exome/intron?&lt;/p&gt;&#xA;&lt;p&gt;if yes then what are the columns that contain this data? Is it the INFO column?&lt;/p&gt;&#xA;&lt;p&gt;I have a list of SNPs  with chromosome position and the reference and alternative allele information and for each, I would like to know what is he nearest gene and the location:exome/intron.&lt;/p&gt;&#xA;&lt;p&gt;example data&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QcvkZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QcvkZ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16697" LastEditorUserId="3694" LastEditDate="2022-12-26T14:06:37.027" LastActivityDate="2022-12-27T08:07:47.337" Title="Availability of information on genes in Gnomad VCF data" Tags="|vcf|genome|gene|exon|gnomad|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20254" PostTypeId="1" AcceptedAnswerId="20255" CreationDate="2022-12-26T14:10:05.163" Score="4" ViewCount="348" Body="&lt;p&gt;I have a process called &lt;code&gt;FINDTAIL&lt;/code&gt; that generates different number of files depending on the input data. Its either 2 files or four files i.e.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;code&gt;1. read_1{,.pr,.sl}.fasta and read_2{,.pr,.sl}.fasta&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;code&gt;2. read_1.pr.fasta, read_1.sl.fasta, read_2.pr.fasta and read_2.sl.fasta&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;/*&#xA; * the `FINDTAIL` process takes &amp;quot;${pair_id}_strand_check.txt&amp;quot; file  &#xA; * and executes commands according to the reads strandedness. &#xA; * The output generated is FASTA files.  &#xA; */&#xA; &#xA;process FINDTAIL {&#xA;&#xA;    tag { pair_id }&#xA;&#xA;    debug true&#xA;&#xA;    input:&#xA;    path strand_check_file&#xA;    tuple val(pair_id), path(reads)&#xA;&#xA;    output:&#xA;    tuple val(pair_id), path(&amp;quot;tail_${pair_id}_{1,2}{,.pr,.sl}.fasta&amp;quot;)&#xA;    &#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;#!/usr/bin/env python3&#xA;&#xA;import subprocess&#xA;import pandas as pd&#xA;import os&#xA;&#xA;&#xA;result = pd.read_csv(&amp;quot;${strand_check_file}&amp;quot;, sep=&amp;quot;\\r\\n&amp;quot;, header=None, engine='python')&#xA;&#xA;failed = float(result.iloc[1,0].replace('Fraction of reads failed to determine: ', ''))&#xA;fwd = float(result.iloc[2,0].replace('Fraction of reads explained by &amp;quot;1++,1--,2+-,2-+&amp;quot;: ', ''))&#xA;rev = float(result.iloc[3,0].replace('Fraction of reads explained by &amp;quot;1+-,1-+,2++,2--&amp;quot;: ', ''))&#xA;fwd_percent = fwd/(fwd+rev)&#xA;rev_percent = rev/(fwd+rev)&#xA;&#xA;if float(result.iloc[1,0].replace('Fraction of reads failed to determine: ', '')) &amp;gt; 0.50:&#xA;    cmd1 = &amp;quot;findtail_v1.01 --input_file &amp;quot; + &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${reads[0]}&quot; + &quot; --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type sl --d &amp;gt; &quot; + &quot;tail_$&lt;/span&gt;{pair_id}_1.sl.fasta&amp;quot;&#xA;    cmd2 = &amp;quot;findtail_v1.01 --input_file &amp;quot; + &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${reads[1]}&quot; + &quot; --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type sl --d &amp;gt; &quot; + &quot;tail_$&lt;/span&gt;{pair_id}_2.sl.fasta&amp;quot;&#xA;    print(cmd1)&#xA;    subprocess.call(cmd1, shell=True)&#xA;    print(cmd2)&#xA;    subprocess.call(cmd2, shell=True)&#xA;&#xA;if fwd_percent &amp;gt; 0.9:&#xA;    #Over 90% of reads explained by &amp;quot;1++,1--,2+-,2-+&#xA;    #Data is likely FR/fr-secondstrand&#xA;    cmd1 = &amp;quot;findtail_v1.01 --input_file &amp;quot; + &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${reads[0]}&quot; + &quot; --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type pr --d &amp;gt; &quot; + &quot;tail_$&lt;/span&gt;{pair_id}_1.pr.fasta&amp;quot;&#xA;    cmd2 = &amp;quot;findtail_v1.01 --input_file &amp;quot; + &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${reads[1]}&quot; + &quot; --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type pr --d &amp;gt; &quot; + &quot;tail_$&lt;/span&gt;{pair_id}_2.pr.fasta&amp;quot;&#xA;    print(cmd1)&#xA;    subprocess.call(cmd1, shell=True)&#xA;    print(cmd2)&#xA;    subprocess.call(cmd2, shell=True)&#xA;&#xA;&#xA;elif rev_percent &amp;gt; 0.9:&#xA;    # Over 90% of reads explained by &amp;quot;1+-,1-+,2++,2--&#xA;    # Data is likely RF/fr-firststrand&#xA;    cmd1 = &amp;quot;findtail_v1.01 --input_file &amp;quot; + &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${reads[0]}&quot; + &quot; --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type sl --d &amp;gt; &quot; + &quot;tail_$&lt;/span&gt;{pair_id}_1.sl.fasta&amp;quot;&#xA;    cmd2 = &amp;quot;findtail_v1.01 --input_file &amp;quot; + &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${reads[1]}&quot; + &quot; --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type sl --d &amp;gt; &quot; + &quot;tail_$&lt;/span&gt;{pair_id}_2.sl.fasta&amp;quot;&#xA;    print(cmd1)&#xA;    subprocess.call(cmd1, shell=True)&#xA;    print(cmd2)&#xA;    subprocess.call(cmd2, shell=True)&#xA;&#xA;elif max(fwd_percent, rev_percent) &amp;lt; 0.6:&#xA;    #Under 60% of reads explained by one direction&#xA;    #Data is likely unstranded&#xA;    cmd1 = &amp;quot;findtail_v1.01 --input_file &amp;quot; + &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${reads[0]}&quot; + &quot; --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type sl --d &amp;gt; &quot; + &quot;tail_$&lt;/span&gt;{pair_id}_1.sl.fasta&amp;quot;&#xA;    cmd2 = &amp;quot;findtail_v1.01 --input_file &amp;quot; + &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${reads[0]}&quot; + &quot; --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type pr --d &amp;gt; &quot; + &quot;tail_$&lt;/span&gt;{pair_id}_1.pr.fasta&amp;quot;&#xA;    cmd3 = &amp;quot;findtail_v1.01 --input_file &amp;quot; + &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${reads[1]}&quot; + &quot; --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type sl --d &amp;gt; &quot; + &quot;tail_$&lt;/span&gt;{pair_id}_2.sl.fasta&amp;quot;&#xA;    cmd4 = &amp;quot;findtail_v1.01 --input_file &amp;quot; + &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${reads[1]}&quot; + &quot; --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type pr --d &amp;gt; &quot; + &quot;tail_$&lt;/span&gt;{pair_id}_2.pr.fasta&amp;quot;&#xA;    print(cmd1)&#xA;    subprocess.call(cmd1, shell=True)&#xA;    print(cmd2)&#xA;    subprocess.call(cmd2, shell=True)&#xA;    print(cmd3)&#xA;    subprocess.call(cmd3, shell=True)&#xA;    print(cmd4)&#xA;    subprocess.call(cmd4, shell=True)&#xA;&#xA;else:&#xA;    #if strand couldnt be detected&#xA;    # we assume it is first-stranded&#xA;    # since most illumina reads are&#xA;    cmd1 = &amp;quot;findtail_v1.01 --input_file &amp;quot; + &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${reads[0]}&quot; + &quot; --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type sl --d &amp;gt; &quot; + &quot;tail_$&lt;/span&gt;{pair_id}_1.sl.fasta&amp;quot;&#xA;    cmd2 = &amp;quot;findtail_v1.01 --input_file &amp;quot; + &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${reads[1]}&quot; + &quot; --seqlength 500 --endgap 2 --taillength 10 --identity 10 --ptype A --stype T --output_format fasta --output_type sl --d &amp;gt; &quot; + &quot;tail_$&lt;/span&gt;{pair_id}_2.sl.fasta&amp;quot;&#xA;    print(cmd1)&#xA;    subprocess.call(cmd1, shell=True)&#xA;    print(cmd2)&#xA;    subprocess.call(cmd2, shell=True)&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to pass these &lt;code&gt;fasta file&lt;/code&gt; outputs of the &lt;code&gt;FINDTAIL&lt;/code&gt; process into an &lt;code&gt;ALIGN&lt;/code&gt; process that runs on each file separately.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;ALIGN&lt;/code&gt; process:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/*&#xA; * define the `ALIGN` process that outputs unmapped files in FASTA format&#xA; * It will take the FASTA files creates by the `FINDTAIL` process &#xA; * and try to align them INDIVIDUALLY.&#xA; */&#xA;&#xA;fasta_file_ch = find_tail_ch.flatten()&#xA; &#xA;process ALIGN {&#xA;&#xA;    tag { pair_id }&#xA;&#xA;    input:&#xA;    path index_files, stageAs: &amp;quot;bt2_index/*&amp;quot;&#xA;    tuple val(pair_id)&#xA;    file(fasta_file) from fasta_file_ch&#xA;&#xA;    output:&#xA;    tuple val(pair_id), path(&amp;quot;unmapped_tail_${pair_id}_{1,2}{,.pr,.sl}.fasta&amp;quot;)&#xA;&#xA;    script:&#xA;    def idx = index_files[0].getBaseName(2)    &#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    bowtie2 -x &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${idx}&quot; -f &quot;$&lt;/span&gt;{fasta_file}&amp;quot; --un &amp;quot;unmapped_tail_&lt;span class=&quot;math-container&quot;&gt;${pair_id}/$&lt;/span&gt;{it.baseName}.fasta&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have tried using the &lt;code&gt;flatten()&lt;/code&gt; function to format my &lt;code&gt;find_tail_ch&lt;/code&gt; but it still does not work.&lt;/p&gt;&#xA;&lt;p&gt;Best wishes~&lt;/p&gt;&#xA;" OwnerUserId="16328" LastActivityDate="2022-12-27T02:56:06.523" Title="How to run a nextflow process for each file generated by another process separately - Input tuple does not match input set cardinality error" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20255" PostTypeId="2" ParentId="20254" CreationDate="2022-12-26T14:39:42.750" Score="4" Body="&lt;p&gt;If the output channel produces a tuple where the first element is a simple value and the second element is a collection of files, like in this case, you can simply use the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#transpose&quot; rel=&quot;nofollow noreferrer&quot;&gt;transpose&lt;/a&gt; operator to produce a transposition where the first element in each resulting tuple is the &lt;em&gt;pair_id&lt;/em&gt; and the second is a FASTA file. You'll then need to update your process definition to receive this tuple, for example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process ALIGN {&#xA;&#xA;    tag { pair_id }&#xA;&#xA;    input:&#xA;    path index_files, stageAs: &amp;quot;bt2_index/*&amp;quot;&#xA;    tuple val(pair_id), path(fasta_file)&#xA;&#xA;    output:&#xA;    tuple val(pair_id), path(&amp;quot;${fasta_file.baseName}.unmapped.fasta&amp;quot;)&#xA;&#xA;    script:&#xA;    def idx = index_files[0].getBaseName(2)    &#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    bowtie2 \\&#xA;        -x &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${idx}&quot; \\&#xA;        -f &quot;$&lt;/span&gt;{fasta_file}&amp;quot; \\&#xA;        --un &amp;quot;${fasta_file.baseName}.unmapped.fasta&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Your workflow block might look like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow {&#xA;&#xA;    bt2_index = ...&#xA;&#xA;    ...&#xA;&#xA;    align_input_ch = find_tail_ch.transpose()&#xA;&#xA;    ALIGN( bt2_index, align_input_ch )&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2022-12-27T02:56:06.523" LastActivityDate="2022-12-27T02:56:06.523" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20256" PostTypeId="2" ParentId="20250" CreationDate="2022-12-26T17:25:59.563" Score="2" Body="&lt;p&gt;OK, tried to figure out a way using Biopython library, I believe there are tens of faster ways, having &lt;code&gt;1lkk_pdb_test.pdb&lt;/code&gt; as input :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATOM      1  N   LEU A 122      49.672   7.391  11.364  1.00 20.16           N  &#xA;ANISOU    1  N   LEU A 122     2561   2767   2050    670   -700     42       N  &#xA;ATOM      2  CA  LEU A 122      50.030   7.561  12.734  1.00 20.25           C  &#xA;ANISOU    2  CA  LEU A 122     2020   3335   2058   -226   -571    629       C  &#xA;ATOM      3  C   LEU A 122      48.886   7.480  13.726  1.00 16.62           C  &#xA;ANISOU    3  C   LEU A 122     1863   2277   1944   -333   -710    706       C  &#xA;ATOM      4  O   LEU A 122      47.816   8.070  13.517  1.00 15.23           O  &#xA;ANISOU    4  O   LEU A 122     2132   1556   1887   -122   -845    184       O  &#xA;ATOM      5  CB  LEU A 122      51.116   8.488  13.074  1.00 21.98           C  &#xA;ANISOU    5  CB  LEU A 122     1769   3688   2587   -148   -358    491       C  &#xA;ATOM      6  CG  LEU A 122      51.381   8.978  14.477  1.00 22.50           C  &#xA;ANISOU    6  CG  LEU A 122     2191   3459   2586   -263   -523    565       C  &#xA;ATOM      7  CD1 LEU A 122      52.229   8.017  15.285  1.00 23.90           C  &#xA;ANISOU    7  CD1 LEU A 122     2795   3452   2500   -216   -104    138       C  &#xA;ATOM      8  CD2 LEU A 122      51.959  10.377  14.502  1.00 26.42           C  &#xA;ANISOU    8  CD2 LEU A 122     3125   3322   3222    375   -323    398       C  &#xA;ATOM      9  H1  LEU A 122      48.995   6.786  11.300  1.00 30.24           H  &#xA;ANISOU    9  H1  LEU A 122     3829   3829   3829      0      0      0       H  &#xA;ATOM     10  H2  LEU A 122      49.399   8.188  11.020  1.00 30.24           H  &#xA;ANISOU   10  H2  LEU A 122     3829   3829   3829      0      0      0       H  &#xA;ATOM     11  H3  LEU A 122      50.392   7.090  10.897  1.00 30.24           H  &#xA;ANISOU   11  H3  LEU A 122     3829   3829   3829      0      0      0       H  &#xA;ATOM     12  HA  LEU A 122      50.481   6.686  12.910  1.00 24.30           H  &#xA;ANISOU   12  HA  LEU A 122     3077   3077   3077      0      0      0       H  &#xA;ATOM     13  HB2 LEU A 122      51.955   8.066  12.760  1.00 26.37           H  &#xA;ANISOU   13  HB2 LEU A 122     3339   3339   3339      0      0      0       H  &#xA;ATOM     14  HB3 LEU A 122      50.982   9.295  12.515  1.00 26.37           H  &#xA;ANISOU   14  HB3 LEU A 122     3339   3339   3339      0      0      0       H  &#xA;ATOM     15  HG  LEU A 122      50.493   9.028  14.933  1.00 27.00           H  &#xA;ANISOU   15  HG  LEU A 122     3419   3419   3419      0      0      0       H  &#xA;ATOM     16 HD11 LEU A 122      51.771   7.153  15.351  1.00 35.85           H  &#xA;ANISOU   16 HD11 LEU A 122     4540   4540   4540      0      0      0       H  &#xA;ATOM     17 HD12 LEU A 122      53.095   7.895  14.844  1.00 35.85           H  &#xA;ANISOU   17 HD12 LEU A 122     4540   4540   4540      0      0      0       H  &#xA;ATOM     18 HD13 LEU A 122      52.368   8.382  16.184  1.00 35.85           H  &#xA;ANISOU   18 HD13 LEU A 122     4540   4540   4540      0      0      0       H  &#xA;ATOM     19 HD21 LEU A 122      52.806  10.390  14.010  1.00 39.63           H  &#xA;ANISOU   19 HD21 LEU A 122     5019   5019   5019      0      0      0       H  &#xA;ATOM     20 HD22 LEU A 122      51.327  10.999  14.083  1.00 39.63           H  &#xA;ANISOU   20 HD22 LEU A 122     5019   5019   5019      0      0      0       H  &#xA;ATOM     21 HD23 LEU A 122      52.116  10.649  15.431  1.00 39.63           H  &#xA;ANISOU   21 HD23 LEU A 122     5019   5019   5019      0      0      0       H  &#xA;ATOM     22  N   GLU A 123      49.124   6.750  14.812  1.00 14.09           N  &#xA;ANISOU   22  N   GLU A 123     1166   1937   2055   -326   -424    468       N  &#xA;ATOM     23  CA  GLU A 123      48.215   6.514  15.890  1.00 12.65           C  &#xA;ANISOU   23  CA  GLU A 123     1143   1584   1903   -273   -331    -29       C  &#xA;ATOM     24  C   GLU A 123      46.790   6.330  15.399  1.00 10.88           C  &#xA;ANISOU   24  C   GLU A 123     1138   1238   1605   -155   -376    -53       C  &#xA;ATOM     25  O   GLU A 123      45.875   7.059  15.809  1.00 10.93           O  &#xA;ANISOU   25  O   GLU A 123     1300   1048   1652     37   -173   -144       O  &#xA;ATOM     26  CB  GLU A 123      48.256   7.537  17.028  1.00 14.54           C  &#xA;ANISOU   26  CB  GLU A 123     1615   1778   1930   -299   -450   -280       C  &#xA;ATOM     27  CG  GLU A 123      47.696   6.998  18.341  1.00 19.07           C  &#xA;ANISOU   27  CG  GLU A 123     2450   2721   1808   -148   -132   -221       C  &#xA;ATOM     28  CD  GLU A 123      47.620   7.948  19.490  1.00 21.48           C  &#xA;ANISOU   28  CD  GLU A 123     3199   2805   1858    113    106      6       C  &#xA;ATOM     29  OE1 GLU A 123      47.461   9.187  19.266  1.00 25.80           O  &#xA;ANISOU   29  OE1 GLU A 123     4100   2736   2608   1002   -257     -4       O  &#xA;ATOM     30  OE2 GLU A 123      47.491   7.497  20.672  1.00 22.45           O  &#xA;ANISOU   30  OE2 GLU A 123     3024   3373   1819    545    690    367       O  &#xA;ATOM     31  H   GLU A 123      49.919   6.375  14.873  1.00 16.91           H  &#xA;ANISOU   31  H   GLU A 123     2141   2141   2141      0      0      0       H  &#xA;ATOM     32  HA  GLU A 123      48.484   5.640  16.294  1.00 15.18           H  &#xA;ANISOU   32  HA  GLU A 123     1922   1922   1922      0      0      0       H  &#xA;ATOM     33  HB2 GLU A 123      49.194   7.821  17.171  1.00 17.45           H  &#xA;ANISOU   33  HB2 GLU A 123     2210   2210   2210      0      0      0       H  &#xA;ATOM     34  HB3 GLU A 123      47.737   8.336  16.761  1.00 17.45           H  &#xA;ANISOU   34  HB3 GLU A 123     2210   2210   2210      0      0      0       H  &#xA;ATOM     35  HG2 GLU A 123      46.784   6.654  18.167  1.00 22.88           H  &#xA;ANISOU   35  HG2 GLU A 123     2897   2897   2897      0      0      0       H  &#xA;ATOM     36  HG3 GLU A 123      48.252   6.227  18.618  1.00 22.88           H  &#xA;ANISOU   36  HG3 GLU A 123     2897   2897   2897      0      0      0       H  &#xA;ATOM     37  N   PRO A 124      46.576   5.340  14.522  1.00  9.86           N  &#xA;ANISOU   37  N   PRO A 124      893   1140   1575     13   -285      8       N  &#xA;ATOM     38  CA  PRO A 124      45.182   5.138  14.041  1.00  8.89           C  &#xA;ANISOU   38  CA  PRO A 124      875   1151   1229     -6   -227     93       C  &#xA;ATOM     39  C   PRO A 124      44.244   4.845  15.208  1.00  8.67           C  &#xA;ANISOU   39  C   PRO A 124      858   1071   1245    -81   -189    -53       C  &#xA;ATOM     40  O   PRO A 124      44.593   4.143  16.165  1.00  8.98           O  &#xA;ANISOU   40  O   PRO A 124     1088   1065   1134    -47   -410    127       O  &#xA;ATOM     41  CB  PRO A 124      45.309   3.946  13.121  1.00  9.41           C  &#xA;ANISOU   41  CB  PRO A 124     1048   1085   1312     99     -1    -37       C  &#xA;ATOM     42  CG  PRO A 124      46.555   3.221  13.556  1.00  9.55           C  &#xA;ANISOU   42  CG  PRO A 124     1054   1161   1282     62   -243      6       C  &#xA;ATOM     43  CD  PRO A 124      47.489   4.326  13.988  1.00 10.06           C  &#xA;ANISOU   43  CD  PRO A 124      911   1179   1591    122   -319   -125       C  &#xA;ATOM     44  HA  PRO A 124      44.868   5.939  13.533  1.00 10.67           H  &#xA;ANISOU   44  HA  PRO A 124     1351   1351   1351      0      0      0       H  &#xA;ATOM     45  HB2 PRO A 124      44.517   3.357  13.204  1.00 11.29           H  &#xA;ANISOU   45  HB2 PRO A 124     1429   1429   1429      0      0      0       H  &#xA;ATOM     46  HB3 PRO A 124      45.388   4.241  12.180  1.00 11.29           H  &#xA;ANISOU   46  HB3 PRO A 124     1429   1429   1429      0      0      0       H  &#xA;ATOM     47  HG2 PRO A 124      46.363   2.605  14.307  1.00 11.46           H  &#xA;ANISOU   47  HG2 PRO A 124     1451   1451   1451      0      0      0       H  &#xA;ATOM     48  HG3 PRO A 124      46.943   2.702  12.807  1.00 11.46           H  &#xA;ANISOU   48  HG3 PRO A 124     1451   1451   1451      0      0      0       H  &#xA;ATOM     49  HD2 PRO A 124      48.119   4.010  14.683  1.00 12.07           H  &#xA;ANISOU   49  HD2 PRO A 124     1528   1528   1528      0      0      0       H  &#xA;ATOM     50  HD3 PRO A 124      48.005   4.678  13.220  1.00 12.07           H  &#xA;ANISOU   50  HD3 PRO A 124     1528   1528   1528      0      0      0       H  &#xA;END                                                                             &#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;my code is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import Bio&#xA;&#xA;from Bio.PDB import PDBParser&#xA;&#xA;&#xA;print('imported Biopython module vesion : ' , Bio.__version__)&#xA;&#xA;&#xA;parser = PDBParser (QUIET = True)&#xA;&#xA;structure = parser.get_structure('test',  '1lkk_pdb_test.pdb')&#xA;&#xA;&#xA;atoms = structure.get_atoms()&#xA;    &#xA;&#xA;coordz = []&#xA;&#xA;for i in [i for i in atoms]:&#xA;    &#xA;    if i.get_id() == 'HA' :&#xA;     &#xA;        print(i.get_parent(), i.get_parent().id, i, i.coord)&#xA;        &#xA;        coordz.append(i.coord)        &#xA;&#xA;print('coordz list : ')&#xA;&#xA;for i in coordz :&#xA;    print(i , type(i))&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;imported Biopython module vesion :  1.80&#xA;imported Biopython module vesion :  1.80&#xA;&amp;lt;Residue LEU het=  resseq=122 icode= &amp;gt; (' ', 122, ' ') &amp;lt;Atom HA&amp;gt; [50.481  6.686 12.91 ]&#xA;&amp;lt;Residue GLU het=  resseq=123 icode= &amp;gt; (' ', 123, ' ') &amp;lt;Atom HA&amp;gt; [48.484  5.64  16.294]&#xA;&amp;lt;Residue PRO het=  resseq=124 icode= &amp;gt; (' ', 124, ' ') &amp;lt;Atom HA&amp;gt; [44.868  5.939 13.533]&#xA;coordz list : &#xA;[50.481  6.686 12.91 ] &amp;lt;class 'numpy.ndarray'&amp;gt;&#xA;[48.484  5.64  16.294] &amp;lt;class 'numpy.ndarray'&amp;gt;&#xA;[44.868  5.939 13.533] &amp;lt;class 'numpy.ndarray'&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;as per the reason &lt;code&gt;HA&lt;/code&gt; represent Cα hydrogen, please refer to _&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Westbrook, John D et al. “The chemical component dictionary: complete descriptions of constituent molecules&#xA;in experimentally determined 3D macromolecules in the Protein Data Bank.”&#xA;Bioinformatics (Oxford, England) vol. 31,8 (2015): 1274-8. doi:10.1093/bioinformatics/btu789&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://httpswww.ncbi.nlm.nih.govpmcarticlesPMC4393513pdfbtu789.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;httpswww.ncbi.nlm.nih.govpmcarticlesPMC4393513pdfbtu789.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.wwpdb.org/data/ccd&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chemical Component Dictionary&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;as explamples pages for Leucine links to :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ebi.ac.uk/pdbe-srv/pdbechem/chemicalCompound/show/LEU&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/pdbe-srv/pdbechem/chemicalCompound/show/LEU&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and &lt;a href=&quot;http://ligand-expo.rcsb.org/reports/L/LEU/LEU_D3L3.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ligand-expo.rcsb.org/reports/L/LEU/LEU_D3L3.gif&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;lot of trouble to find a diagram:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/6e28A.gif&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/6e28A.gif&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Just loop over all your proteins in a similar way and store the coords as you need/wish, remember coordinates are retrieved as numpy arrays.&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2022-12-26T17:51:00.200" LastActivityDate="2022-12-26T17:51:00.200" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20257" PostTypeId="1" AcceptedAnswerId="20738" CreationDate="2022-12-27T00:57:58.083" Score="2" ViewCount="362" Body="&lt;p&gt;This is really a programming question but &lt;code&gt;ggtree&lt;/code&gt; might be too obscure for this question to be posted on StackOverFlow as a general R related question.&lt;/p&gt;&#xA;&lt;p&gt;I am having trouble extrapolating this this &lt;a href=&quot;https://guangchuangyu.github.io/treedata-workshop/r-ggtree-data.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial&lt;/a&gt; and the &lt;code&gt;%&amp;lt;+%&lt;/code&gt; operator in &lt;code&gt;ggtree&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I have a Newick file and a table that has a number for every tip and node in the Newick file. I want to produce a final graphic that shows a tree where the numbers from the table are shown next to respective tips and nodes.&lt;/p&gt;&#xA;&lt;p&gt;What I want to do is close to the tutorial, I want to take a tree object and use a data-frame where one column is node-names and the second column holds numerical information per node. I want to attach the information from the data-frame to the tree object.&lt;/p&gt;&#xA;&lt;p&gt;The data-frame looks something like this:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Node / tips&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: center;&quot;&gt;Count&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;A&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;12&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;B&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;14&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;So, I want to ultimately display the number 12 next to node A and number 14 next to node B while keeping the names intact. Now I have the code working up to the following point:&lt;/p&gt;&#xA;&lt;p&gt;(I am really only just working my way through learning R's grammar of graphics and R in general, so assume I am doing something wrong.)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(ggplot2)&#xA;library(ggtree)&#xA;&#xA;x &amp;lt;- read.tree(&amp;quot;tree.txt&amp;quot;)&#xA;info &amp;lt;- read.csv(&amp;quot;counts.csv&amp;quot;,sep = '\t',header=TRUE)&#xA;&#xA;p &amp;lt;- ggtree(x) %&amp;lt;+% info + xlim(-.1, 5)&#xA;&#xA;p2 &amp;lt;- p + geom_tiplab(offset = 0.2) + geom_tippoint(aes(caption = Increase)) + theme_tree2()+geom_tiplab(align=TRUE, linesize=.5)+ xlim(0,25)&#xA;&#xA;ggsave(&amp;quot;test_2.pdf&amp;quot;, width = 60, height = 80, units = &amp;quot;cm&amp;quot;, limitsize = FALSE)&#xA;&#xA;print(p2)&#xA;&#xA;?geom_tippoint::aes() # is there a way to list all the option inside geom_tippoint::aes()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above code overlays the &lt;code&gt;Newick_label&lt;/code&gt; column with the node names instead of attaching the value in the second column.&lt;/p&gt;&#xA;&lt;p&gt;But specifically, I am at a loss as to how I can list the options available within &lt;code&gt;geom_tippoint(aes())&lt;/code&gt;. I can use &lt;code&gt;?geom_tippoint&lt;/code&gt; to access the documentation of &lt;code&gt;geom_tippoint&lt;/code&gt; but I think my task needs to manipulate options within the &lt;code&gt;aes()&lt;/code&gt; function and the the geom_tippoint() documentation does not list all the &lt;code&gt;aes()&lt;/code&gt; options available. And am stuck trying different keywords for &lt;code&gt;geom_tippoint(aes())&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Can that be done in &lt;code&gt;ggtree&lt;/code&gt; or is this specific application of &lt;code&gt;ggtree&lt;/code&gt; obscure enough that there is no way to do this and produce a legible graphic? (eg issue: the graphic getting mushy overlays). I have hundred of (not uniform) genes trees so splitting data for each layer (node and tips) might be not the best way.&lt;/p&gt;&#xA;&lt;p&gt;The files if anyone wants to try: &lt;a href=&quot;https://drive.google.com/file/d/1z7A0Pq50Zh5zqqSNQ8Yc_k9p5MnWX7Lq/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;counts&lt;/a&gt; and &lt;a href=&quot;https://drive.google.com/file/d/1xGs2zwnS7_43AlH2M9zKU2AKtS0hI4rk/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;tree&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Clarification on the labels for nodes and tips, I want to label both if possible. The data contains information for both.&lt;/p&gt;&#xA;&lt;p&gt;Is annotating tips AND nodes out of the ordinary? Tips and nodes are technically different but I have quantitative data at both levels and am looking for the option to annotate at both levels.&lt;/p&gt;&#xA;" OwnerUserId="16699" LastEditorUserId="73" LastEditDate="2022-12-28T08:44:50.847" LastActivityDate="2023-03-17T21:52:22.600" Title="Using R `ggtree`, attach extra data next to the nodes on an illustrated tree/newick file" Tags="|r|phylogenetics|ggtree|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20258" PostTypeId="1" AcceptedAnswerId="20292" CreationDate="2022-12-27T03:21:34.200" Score="3" ViewCount="442" Body="&lt;p&gt;I'm trying to do a specific molecular docking using Autodock Vina in PyRx. So, one way to do that is to mark all of the interacting residues and make sure the grid box encompasses all of that interacting residues.&lt;/p&gt;&#xA;&lt;p&gt;So, my protein already has a co-crystalized ligand attached to it. However, I'm not sure what method to use.&lt;/p&gt;&#xA;&lt;p&gt;After searching online, I found out that I should identify the interacting residues of that ligand. Then, I have opened this protein-ligand complex in 2 applications and 1 web server.&lt;/p&gt;&#xA;&lt;p&gt;Here are the information of the protein-ligand complex:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The protein PDB ID is: 5TL8 (Naegleria fowleri CYP51-posaconazole complex)&lt;/li&gt;&#xA;&lt;li&gt;The ligand is Posaconazole&lt;/li&gt;&#xA;&lt;li&gt;The cofactor and other small molecules: HEME, 1,2-ETHANEDIOL, calcium ion&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I tried opening the complex in Discovery Studio&lt;a href=&quot;https://i.stack.imgur.com/JS4LY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JS4LY.png&quot; alt=&quot;Discovery Studio&quot; /&gt;&lt;/a&gt;&#xA;Ligplus&lt;a href=&quot;https://i.stack.imgur.com/Nis5n.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Nis5n.png&quot; alt=&quot;Ligplus&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and Protein Plus webserver (this server has more interacting residues). All of them have different residues.&lt;/p&gt;&#xA;&lt;p&gt;Which one should I follow? Should we included the HEME or do we just look at the Posaconazole itself?&lt;/p&gt;&#xA;" OwnerUserId="16679" LastEditorUserId="16679" LastEditDate="2023-01-01T02:58:35.230" LastActivityDate="2023-01-01T03:19:39.407" Title="What is the correct method of identifying the interacting residues for specific molecular docking?" Tags="|protein-structure|pdb|docking|structural-biology|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20259" PostTypeId="2" ParentId="20253" CreationDate="2022-12-27T08:07:47.337" Score="4" Body="&lt;p&gt;This question has gone through some iterations and commentary, so I've decided to just answer it, though it's more of a &amp;quot;homework&amp;quot; question. What you need to do is to look at the VCF file, the main thing you want is already in there.&lt;/p&gt;&#xA;&lt;p&gt;Go to the VCF of interest. I chose the &lt;a href=&quot;https://storage.googleapis.com/gcp-public-data--gnomad/release/3.1/vcf/genomes/gnomad.genomes.v3.1.sites.chrM.vcf.bgz&quot; rel=&quot;nofollow noreferrer&quot;&gt;mtDNA&lt;/a&gt; because it's smaller and more convenient. You can &lt;code&gt;zless&lt;/code&gt; into the bgz file to see what it looks like, but if you want uncompressed you can run this (head -1000 just to be a tractable example):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gzip -dc gnomad.genomes.v3.1.sites.chrM.vcf.bgz | grep -v intergenic | head -1000 &amp;gt; mt.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I &lt;code&gt;grep -v intergenic&lt;/code&gt; because one of the first things that pops up is that most of the first variants in the file are specifically annotated as &lt;code&gt;intergenic_variant&lt;/code&gt;, so that answers part of your question. For those variants you can just run a &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/closest.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools closest&lt;/a&gt; against the right GFF3 version, if you think the nearest gene will be informative.&lt;/p&gt;&#xA;&lt;p&gt;For variants in genes, what you want is already in the VCF. See the entry for this variant:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chrM    1492    .       A       C       .       npg     filters=npg;variant_colla&#xA;psed=A1492C;vep=C|non_coding_transcript_exon_variant|MODIFIER|MT-RNR1|ENSG0000021&#xA;1459|Transcript|ENST00000389680|Mt_rRNA|1/1||ENST00000389680.2:n.845A&amp;gt;C||845|||||&#xA;1||1|SNV||HGNC|HGNC:7470|YES||||||||1||||||||||||;base_qual_hist=0|0|0|0|0|0|0|0|&#xA;0|0;position_hist=0|0|0|0|0|0|0|0|0|0;strand_bias_hist=0|0|0|0|0|0|0|0|0|0;weak_e&#xA;vidence_hist=0|0|0|0|0|0|0|0|0|0;contamination_hist=0|0|0|0|0|0|0|0|0|0;heteropla&#xA;smy_below_min_het_threshold_hist=2|0|0|0|0|0|0|0|0|0;excluded_AC=2;AN=56432;AC_ho&#xA;m=0;AC_het=0;hl_hist=0|0|0|0|0|0|0|0|0|0;dp_mean=2976.53;mq_mean=.;tlod_mean=.;AF&#xA;_hom=0.00000;AF_het=0.00000;max_hl=0.00000;hap_AN=2680|1537|868|603|34|282|91|147&#xA;83|701|934|3144|2732|663|2977|4724|5672|126|1|1298|366|7|393|3079|6037|1234|819|5&#xA;46|12|89;hap_AC_het=0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0;hap&#xA;_AC_hom=0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0;hap_AF_hom=0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00|0.00000e+00;hap_AF_het=0.00000e+00|0.00000e+00|0.00 ...&#xA;# this is of course a very long line!!! it goes on but all the interesting stuff already happened.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can see that this variant lies within the &lt;em&gt;RNR1&lt;/em&gt; rRNA gene of the mitochondrial genome (&lt;code&gt;MT-RNR1&lt;/code&gt; in the file). Granted it mangles it a little bit by describing a variant in a non-protein-coding gene as a &lt;code&gt;non_coding_transcript_exon_variant&lt;/code&gt;, but that's the kind of thing that you can work around.&lt;/p&gt;&#xA;&lt;p&gt;Indeed, if you look at the header of the file, you can see that quite extensive annotations are included (where available):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##INFO=&amp;lt;ID=vep,Number=.,Type=String,Description=&amp;quot;Consequence annotations from Ensembl VEP; note that the SINGLE_EXON flag and END_TRUNC filters have been removed from the LOFTEE annotations to avoid misinterpretation in context of the mitochondrial genome. Format: Allele|Consequence|IMPACT|SYMBOL|Gene|Feature_type|Feature|BIOTYPE|EXON|INTRON|HGVSc|HGVSp|cDNA_position|CDS_position|Protein_position|Amino_acids|Codons|ALLELE_NUM|DISTANCE|STRAND|VARIANT_CLASS|MINIMISED|SYMBOL_SOURCE|HGNC_ID|CANONICAL|TSL|APPRIS|CCDS|ENSP|SWISSPROT|TREMBL|UNIPARC|GENE_PHENO|SIFT|PolyPhen|DOMAINS|HGVS_OFFSET|MOTIF_NAME|MOTIF_POS|HIGH_INF_POS|MOTIF_SCORE_CHANGE|LoF|LoF_filter|LoF_flags|LoF_info&amp;quot;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In conclusion, &lt;strong&gt;look at your data&lt;/strong&gt;!&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2022-12-27T08:07:47.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20260" PostTypeId="2" ParentId="20258" CreationDate="2022-12-27T08:56:50.807" Score="3" Body="&lt;p&gt;not an answer, just more inputs, but &lt;a href=&quot;https://proteins.plus/5tl8#poseview&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://proteins.plus/5tl8#poseview&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;gives me this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QzBxw.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QzBxw.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;references:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://academic.oup.com/nar/article/48/W1/W48/5820880&quot; rel=&quot;nofollow noreferrer&quot;&gt;ProteinsPlus: interactive analysis of protein–ligand binding interfaces &lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://academic.oup.com/bioinformatics/article/22/14/1710/226866&quot; rel=&quot;nofollow noreferrer&quot;&gt;Molecular complexes at a glance: automated generation of two-dimensional complex diagrams &lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://pubs.acs.org/doi/10.1021/ml100164p&quot; rel=&quot;nofollow noreferrer&quot;&gt;Drawing the PDB: Protein−Ligand Complexes in Two Dimensions&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;from this last paper :&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Hydrogen bonds are implemented mainly following the measures published by Desiraju and Steiner in 2001.14 The optimal distance between two atoms connected by a hydrogen bond is set to 1.9 Å with a tolerance of 0.5 Å. Additional to this measure, the acceptor−hydrogen−donor angle must not fall below 120°. Hydrogen atoms that are bound to a noncarbon atom are treated as hydrogen donor candidates. Potential acceptor atoms are either nitrogen, oxygen, or sulfur atoms provided that they are uncharged or negatively charged and that their surface is accessible.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Metal interactions are calculated between metal atoms embedded in the receptor and metal acceptor atoms, which are identical to the hydrogen bond acceptor atoms. Their geometry is based on the calculated coordination geometry of the metal.15 Each coordination point that is not occupied by a receptor atom is checked for close ligand atoms, and the maximal distance deviation is set to 0.8 Å. If no geometry can be calculated, a sphere with a radius of 2 Å is placed around the metal. In this case, all atoms lying on the sphere, again with a tolerance of 0.8 Å, are regarded as potential interaction partners.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In contrast to the formerly mentioned interactions, hydrophobic contacts are estimated based on the distance between two hydrophobic atoms only. They are visualized not by a dashed interaction line but by drawing the label of the contacting residue and a spline segment denoting the hydrophobic part of the ligand. Because many atoms typically form a hydrophobic subpocket, this representation reflects the interaction geometry better. A prerequisite for a hydrophobic contact is that at least three hydrophobic ligand atoms lie in the range of the currently examined receptor residue. Hydrophobic atoms are in this context carbon atoms with accessible surface and halogens. The maximum distance is set to the sum of the van der Waals radii of atoms in question and a tolerance of 0.8 Å.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDITED :&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Adding more  on a side note I came accross this paper, I am completely naive about the topic but the title :&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;“Observing Noncovalent Interactions in Experimental Electron Density for Macromolecular Systems: A Novel Perspective for Protein-Ligand Interaction Research.” Journal of chemical information and modeling vol. 62,7 (2022): 1734-1743. doi:10.1021/acs.jcim.1c01406&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;is kind of intriguing, I am not an expert and cannot vauch for its strictness but to me sounds like a good why to validate an interaction like seeing it even if in abstract terms:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;... Non-covalent interactions (NCI) can be observed by pinpointing the ED saddle point, i.e. (3,−1) critical points and further quantified by measuring ED deviation from a homogeneous electron distribution using density and its first derivative (s = [1/(2(3π2)1/3)]|∇ρ|/ρ4/3)....&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Please feel free to add to this, its not my field.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2022.01.24.468575v2.full&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/10.1101/2022.01.24.468575v2.full&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2022-12-28T18:55:53.953" LastActivityDate="2022-12-28T18:55:53.953" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20262" PostTypeId="2" ParentId="20234" CreationDate="2022-12-27T12:03:44.033" Score="2" Body="&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/Microsatellite&quot; rel=&quot;nofollow noreferrer&quot;&gt;Microsatellites&lt;/a&gt;, &lt;a href=&quot;https://en.wikipedia.org/wiki/Minisatellite&quot; rel=&quot;nofollow noreferrer&quot;&gt;minisatellites&lt;/a&gt; and other type of repetitive variation are a known feature of genomes.&lt;/p&gt;&#xA;&lt;p&gt;I agree with previous comments in the sense that what you are observing here might as well be real and not just an artefact caused by real repetitive variation in your genome of study, plus read mapping together with variant calling issues at the site.&lt;/p&gt;&#xA;&lt;p&gt;The latter might be the case, but before getting rid of these alleles I suggest that you inspect the sites and regions around these alleles more carefully: GATK HaplotypeCaller has an option (-- output-bam or similar) that writes a BAM file with the reads as they were used for variant calling after the tool itself performs local re-alignment of reads around the variants. I would use this tool together with the original BAM file in order to inspect the region visually and try to understand it better. For this you can use samtools tview or &lt;a href=&quot;https://software.broadinstitute.org/software/igv/home&quot; rel=&quot;nofollow noreferrer&quot;&gt;IGV&lt;/a&gt;. The length of your reads becomes somewhat relevant here, but even if you are sequencing around 100 bp, given the small size of these observed alleles you should be able to draw some more solid insights about the variants/errors and how to handle them.&lt;/p&gt;&#xA;" OwnerUserId="475" LastActivityDate="2022-12-27T12:03:44.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20263" PostTypeId="2" ParentId="20258" CreationDate="2022-12-27T12:22:10.883" Score="4" Body="&lt;p&gt;Not an answer either. But a series of caveats.&lt;/p&gt;&#xA;&lt;p&gt;First, &lt;code&gt;HEM&lt;/code&gt; parameterisation. For molecular thermodynamics tools you will need to generate a topology file for this bad-boi. The iron has 2 donor bonds and 4 single bonds, one of which with the coordinating cysteine and the other with an oxygen, which will attach the substrate, in this case a steroid —let's ignore the radical state of the activated bound oxygen, but do keep in mind that the partial charges and atom type auto-assigned will be wrong. A cheat would be constraining said molecular oxygen and using a porphyrin ring with no iron: not reality, but would make the experiment work, is as physically questionable as any iron-bound topology you'll get and the oxygen is the key player here.&lt;/p&gt;&#xA;&lt;p&gt;Second, cytochromes act on greasy substrates, there are few if any polar interactions generally, so constraining to polymer residues may not be that useful. Specificity is controlled by the gating and entranceway (think of trucks reversing into a loading bay), which is annoying the gating loops are floppy loops that are generally not crystallised, so tethered MD from a bound to unbound state (undocking) is sometimes done.&#xA;There's a lot of literature on P450 and P411 cytochromes, but often it's not as clear cut and simple as say a hydrolase.&lt;/p&gt;&#xA;&lt;p&gt;However, if your substrate you want placed is similar to one present you may be interested in a tool of mine &lt;a href=&quot;https://github.com/matteoferla/Fragmenstein&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fragmenstein&lt;/a&gt;, which can be used to convert a compound to a similar one (stripping the haem and adding it later: I had a Rosetta params file for &lt;code&gt;HEM&lt;/code&gt; somewhere but not sure where it is nowadays)&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2022-12-27T12:22:10.883" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20264" PostTypeId="1" CreationDate="2022-12-28T04:36:10.813" Score="-2" ViewCount="376" Body="&lt;p&gt;I have two data frames. One data frame with one column and second data frame with two columns. I need to merge first data frame with either column of the second data frame and returns the values. After merging the values,the order of the first column should be same as the input. Please find the sample input below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;first_column = c(&amp;quot;ENSG00000165588&amp;quot;,&amp;quot;ENSG00000213551&amp;quot;)&#xA;df1 = data.frame(first_column)&#xA;first_column = c(&amp;quot;ENSG00000142192&amp;quot;, &amp;quot;ENSG00000140575&amp;quot;, &amp;quot;ENSG00000165588&amp;quot;, &amp;quot;ENSG00000165588&amp;quot;, &amp;quot;ENSG00000213551&amp;quot;, &amp;quot;ENSG00000213551&amp;quot;,&amp;quot;ENSG00000197153&amp;quot;)&#xA;second_column = c(&amp;quot;ENSG00000165588&amp;quot;, &amp;quot;ENSG00000165588&amp;quot;, &amp;quot;ENSG00000186908&amp;quot;, &amp;quot;ENSG00000135446&amp;quot;, &amp;quot;ENSG00000273983&amp;quot;, &amp;quot;ENSG00000274267&amp;quot;,&amp;quot;ENSG00000213551&amp;quot;)&#xA;df2 = data.frame(first_column,second_column)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Please find the output below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;first_column = c(&amp;quot;ENSG00000165588&amp;quot;,&amp;quot;ENSG00000165588&amp;quot;,&amp;quot;ENSG00000165588&amp;quot;,&amp;quot;ENSG00000165588&amp;quot;,&amp;quot;ENSG00000213551&amp;quot;,&amp;quot;ENSG00000213551&amp;quot;,&amp;quot;ENSG00000213551&amp;quot;)&#xA;second_column = c(&amp;quot;ENSG00000142192&amp;quot;,&amp;quot;ENSG00000140575&amp;quot;,&amp;quot;ENSG00000186908&amp;quot;,&amp;quot;ENSG00000135446&amp;quot;,&amp;quot;ENSG00000273983&amp;quot;,&amp;quot;ENSG00000274267&amp;quot;,&amp;quot;ENSG00000197153&amp;quot;)&#xA;output = data.frame(first_column,second_column)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14023" LastEditorUserId="3848" LastEditDate="2023-01-02T03:35:35.590" LastActivityDate="2023-01-24T15:46:13.270" Title="Merge data frame on either of the 2 columns in R or python" Tags="|r|python|dataframe|pandas|merge|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20265" PostTypeId="2" ParentId="20250" CreationDate="2022-12-28T18:53:57.207" Score="1" Body="&lt;p&gt;I would import the PDB into the molecular dynamics package GROMACS and use the built-in utilities to calculate the centroid (&lt;a href=&quot;https://manual.gromacs.org/current/onlinehelp/gmx-rms.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://manual.gromacs.org/current/onlinehelp/gmx-rms.html&lt;/a&gt;).  This is much simpler than trying to write your own utility for this.  Other pre-written calculations to perform on your PDB files:&#xA;&lt;a href=&quot;https://manual.gromacs.org/current/user-guide/cmdline.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://manual.gromacs.org/current/user-guide/cmdline.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16705" LastActivityDate="2022-12-28T18:53:57.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20266" PostTypeId="2" ParentId="19444" CreationDate="2022-12-28T18:58:07.987" Score="1" Body="&lt;p&gt;Check out gmx_trjconv - this command is in GROMACS and accepts several options where you can specify the specific single frame that you want to output/write - out to the new pdb/gro/xtc file.  &lt;a href=&quot;https://manual.gromacs.org/current/onlinehelp/gmx-trjconv.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://manual.gromacs.org/current/onlinehelp/gmx-trjconv.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16705" LastActivityDate="2022-12-28T18:58:07.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20267" PostTypeId="2" ParentId="18613" CreationDate="2022-12-28T19:01:21.697" Score="0" Body="&lt;p&gt;You would need to calculate the free energy along a trajectory and the rmsd along that trajectory and then cross-correlate/eliminate variables.  There may be a utility which does this but I am not ware of it.&#xA;&lt;a href=&quot;https://manual.gromacs.org/current/user-guide/cmdline.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://manual.gromacs.org/current/user-guide/cmdline.html&lt;/a&gt;&#xA;There are two utilities of interest for doing just that.  See gmx_energy and gmx-rmsd.  Then you can plot one against the other by elimination of the time/space variables. Check out other calculations you can do with GROMACS:&#xA;&lt;a href=&quot;https://manual.gromacs.org/current/user-guide/cmdline.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://manual.gromacs.org/current/user-guide/cmdline.html&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16705" LastActivityDate="2022-12-28T19:01:21.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20268" PostTypeId="2" ParentId="14619" CreationDate="2022-12-28T23:38:38.270" Score="0" Body="&lt;p&gt;Anecdotally, I have been told that it varies from person to person, also might be affected by disease or hygiene in a particular environment. Here is an article dealing with that problem:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://journals.plos.org/plosone/article/comments?id=10.1371/journal.pone.0076096&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://journals.plos.org/plosone/article/comments?id=10.1371/journal.pone.0076096&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Not sure if they have references useful for you in their paper, but they describe a way of enriching microbial or human DNA. It might be helpful.&lt;/p&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2022-12-28T23:38:38.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20269" PostTypeId="2" ParentId="20264" CreationDate="2022-12-29T01:18:40.367" Score="0" Body="&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt; The OP in hindsight is requesting an index merge. The first solution, presented here as &lt;strong&gt;Column merge solution&lt;/strong&gt; was rejected by the OP.&lt;/p&gt;&#xA;&lt;p&gt;From the comments below this post the order requested by the OP was:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# 0 ENSG00000165588 ENSG00000142192 &#xA;# 1 ENSG00000165588 ENSG00000140575 &#xA;# 2 ENSG00000165588 ENSG00000186908 &#xA;# 3 ENSG00000165588 ENSG00000135446 &#xA;# 4 ENSG00000213551 ENSG00000273983 &#xA;# 5 ENSG00000213551 ENSG00000274267 &#xA;# 6 ENSG00000213551 ENSG00000197153,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Column merge solution&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The first operation is a concatenation and inner merge &lt;strong&gt;*&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;li&gt;The second operation are two outer merges, the second requiring a 'left-hand' merge &lt;strong&gt;*&lt;/strong&gt;.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The code is here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;df1 = pd.DataFrame({'A':[&amp;quot;ENSG00000165588&amp;quot;,&amp;quot;ENSG00000213551&amp;quot;]})&#xA;df2i = pd.DataFrame({'A':[&amp;quot;ENSG00000142192&amp;quot;, &amp;quot;ENSG00000140575&amp;quot;, &amp;quot;ENSG00000165588&amp;quot;,\&#xA;                          &amp;quot;ENSG00000165588&amp;quot;, &amp;quot;ENSG00000213551&amp;quot;, &amp;quot;ENSG00000213551&amp;quot;, &amp;quot;ENSG00000197153&amp;quot;]})&#xA;df2ii = pd.DataFrame({'A':[&amp;quot;ENSG00000165588&amp;quot;, &amp;quot;ENSG00000165588&amp;quot;, &amp;quot;ENSG00000186908&amp;quot;, \&#xA;                            &amp;quot;ENSG00000135446&amp;quot;, &amp;quot;ENSG00000273983&amp;quot;, &amp;quot;ENSG00000274267&amp;quot;,&amp;quot;ENSG00000213551&amp;quot;]})&#xA;    &#xA;# Column 'A'&#xA;df3in = pd.concat([df2i, df2ii])&#xA;df4inner = pd.merge(df3in,df1, on='A', how='inner')&#xA;&#xA;# Column 'A+'&#xA;df3out = pd.merge(df2i, df2ii, on='A', how='outer')&#xA;df4outer = pd.merge(df3in,df1, on='A', how='outer', indicator = True).query('_merge==&amp;quot;left_only&amp;quot;')&#xA;df5 = df4outer.drop('_merge', axis=1).reset_index(drop=True).rename(columns={'A':'A+'})   &#xA;df6 = df4inner.join(df5)&#xA;&#xA;print (df6)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;                 A               A+&#xA;0  ENSG00000165588  ENSG00000142192&#xA;1  ENSG00000165588  ENSG00000140575&#xA;2  ENSG00000165588  ENSG00000197153&#xA;3  ENSG00000165588  ENSG00000186908&#xA;4  ENSG00000213551  ENSG00000135446&#xA;5  ENSG00000213551  ENSG00000273983&#xA;6  ENSG00000213551  ENSG00000274267&#xA;                      &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This approach was rejected (comments) because of the position of &lt;code&gt;ENSG00000197153&lt;/code&gt; at row 2, Column A+ in relation to &lt;code&gt;ENSG00000165588&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Index merge solution&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;def indexmerge(dfa, dfb, df1):&#xA;    df3out = dfa.reset_index().merge(dfb, how=&amp;quot;left&amp;quot;).set_index('index')&#xA;    df4outer = pd.merge(df3out,df1, on='A', how='outer', indicator = True).query('_merge==&amp;quot;left_only&amp;quot;')&#xA;    return df4outer&#xA;&#xA;if __name__ == '__main__':&#xA;&#xA;    df1 = pd.DataFrame({'A':[&amp;quot;ENSG00000165588&amp;quot;,&amp;quot;ENSG00000213551&amp;quot;]})&#xA;    df2i = pd.DataFrame({'A':[&amp;quot;ENSG00000142192&amp;quot;, &amp;quot;ENSG00000140575&amp;quot;, &amp;quot;ENSG00000165588&amp;quot;,\&#xA;                              &amp;quot;ENSG00000165588&amp;quot;, &amp;quot;ENSG00000213551&amp;quot;, &amp;quot;ENSG00000213551&amp;quot;, &amp;quot;ENSG00000197153&amp;quot;]})&#xA;    df2ii = pd.DataFrame({'A':[&amp;quot;ENSG00000165588&amp;quot;, &amp;quot;ENSG00000165588&amp;quot;, &amp;quot;ENSG00000186908&amp;quot;, \&#xA;                                &amp;quot;ENSG00000135446&amp;quot;, &amp;quot;ENSG00000273983&amp;quot;, &amp;quot;ENSG00000274267&amp;quot;,&amp;quot;ENSG00000213551&amp;quot;]})&#xA;    df3in = pd.concat([df2i, df2ii])&#xA;    df4inner = pd.merge(df3in,df1, on='A', how='inner')&#xA;&#xA;    df4outer = indexmerge(df2i,df2ii,df1)&#xA;    df4outer2 = indexmerge(df2ii,df2i,df1)&#xA;    df4outer2.rename(columns={'A':'_A'},inplace=True)&#xA;    df6 = pd.concat([df4outer,df4outer2],axis=1)&#xA;    df6['A+'] = df6['A'].astype(str).replace('nan','') + df6['_A'].astype(str).replace('nan','')&#xA;    df7 = df6.sort_index().reset_index()&#xA;    df8 = df4inner.join(df7['A+'])&#xA;    print (df8)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;                 A               A+&#xA;0  ENSG00000165588  ENSG00000142192&#xA;1  ENSG00000165588  ENSG00000140575&#xA;2  ENSG00000165588  ENSG00000186908&#xA;3  ENSG00000165588  ENSG00000135446&#xA;4  ENSG00000213551  ENSG00000273983&#xA;5  ENSG00000213551  ENSG00000274267&#xA;6  ENSG00000213551  ENSG00000197153&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This solution is identical to the order requested in the comments and is now also presented at the beginning of this post. The solution used here pivots around line 1 of &lt;code&gt;indexmerge()&lt;/code&gt; and thus is an index merge operation. The remaining code is common &lt;code&gt;pandas&lt;/code&gt; manipulation supporting this operation.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; the new position of &lt;code&gt;ENSG00000197153&lt;/code&gt; row 6 column A+. The index merge retains &lt;code&gt;ENSG00000197153&lt;/code&gt; at the base of the column and therefore in association, i.e. same row, as &lt;code&gt;ENSG00000213551&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;*&lt;/strong&gt; technical &lt;code&gt;pandas&lt;/code&gt; terminology&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-01T17:08:24.433" LastActivityDate="2023-01-01T17:08:24.433" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20272" PostTypeId="1" AcceptedAnswerId="20278" CreationDate="2022-12-29T15:48:45.887" Score="2" ViewCount="231" Body="&lt;p&gt;I want to see if there is any GC coverage bias in some paired-end Illumina human exome data. &lt;strong&gt;Update&lt;/strong&gt; This is for a variant calling project. I used the &lt;code&gt;CollectMultipleMetrics&lt;/code&gt; from &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360037594031-CollectMultipleMetrics-Picard-&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATK4 toolkit&lt;/a&gt; with &lt;code&gt;-PROGRAM&lt;/code&gt; &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360036801531-CollectGcBiasMetrics-Picard-&quot; rel=&quot;nofollow noreferrer&quot;&gt;CollectGcBiasMetrics&lt;/a&gt;. An example of the command I used is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gatk CollectMultipleMetrics -I input.bam -O prefix_dir_name -R input.ref_genome -PROGRAM CollectGcBiasMetrics -PROGRAM CollectAlignmentSummaryMetrics -PROGRAM CollectInsertSizeMetrics -PROGRAM QualityScoreDistribution -PROGRAM MeanQualityByCycle -PROGRAM CollectBaseDistributionByCycle&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt; This is my result for the QC coverage bias with one sample highlighted:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/WgnZn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WgnZn.png&quot; alt=&quot;GC coverage bias plot&quot; /&gt;&lt;/a&gt;&#xA;There seems to be a large bias. I would expect the peak to be around 50% for human. And those weird peaks around 90% are also puzzling to me. On the other hand, I am working with exome data so it might be logical to have a bias towards the higher values of GC% because you have reads on GC-rich exons?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt; I have another plot with the bins produced by the above mentioned tool (.gc_bias.pdf)&#xA;&lt;a href=&quot;https://i.stack.imgur.com/tVCcb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tVCcb.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt; first part of useful file (gc_bias.detail_metrics) which shows the reads per bin, as suggested by @Maximilian Press. This I will plot.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## htsjdk.samtools.metrics.StringHeader&#xA;# CollectMultipleMetrics --INPUT WESP_run/results/aligned/RM8398_MF_S8_MD.sorted.bam --OUTPUT WESP_run/results/aligned/post_alignment_QC/multiple_metrics/RM8398_MF_S8 --PROGRAM CollectAlignmentSummaryMetrics --P&#xA;ROGRAM CollectBaseDistributionByCycle --PROGRAM CollectInsertSizeMetrics --PROGRAM MeanQualityByCycle --PROGRAM QualityScoreDistribution --PROGRAM CollectGcBiasMetrics --REFERENCE_SEQUENCE /mnt/scratch_dir/deute&#xA;koe/projects/HP/WESPipe/workflow/resources/reference_genomes/hg38/alt_plus/GCA_000001405.15_GRCh38_full_plus_hs38d1_analysis_set.fna --ASSUME_SORTED true --STOP_AFTER 0 --METRIC_ACCUMULATION_LEVEL ALL_READS --IN&#xA;CLUDE_UNPAIRED false --VERBOSITY INFO --QUIET false --VALIDATION_STRINGENCY STRICT --COMPRESSION_LEVEL 2 --MAX_RECORDS_IN_RAM 500000 --CREATE_INDEX false --CREATE_MD5_FILE false --GA4GH_CLIENT_SECRETS client_sec&#xA;rets.json --help false --version false --showHidden false --USE_JDK_DEFLATER false --USE_JDK_INFLATER false&#xA;## htsjdk.samtools.metrics.StringHeader&#xA;# Started on: Sat Dec 03 14:25:15 CET 2022&#xA;&#xA;## METRICS CLASS        picard.analysis.GcBiasDetailMetrics&#xA;ACCUMULATION_LEVEL      READS_USED      GC      WINDOWS READ_STARTS     MEAN_BASE_QUALITY       NORMALIZED_COVERAGE     ERROR_BAR_WIDTH SAMPLE  LIBRARY READ_GROUP&#xA;All Reads       ALL     0       139942  1326    19      0.413544        0.011357                        &#xA;All Reads       ALL     1       102412  61      17      0.025996        0.003328                        &#xA;All Reads       ALL     2       118957  74      16      0.02715 0.003156                        &#xA;All Reads       ALL     3       152181  929     20      0.266429        0.008741                        &#xA;All Reads       ALL     4       167732  248     16      0.06453 0.004098                        &#xA;All Reads       ALL     5       173319  232     16      0.058421        0.003836                        &#xA;All Reads       ALL     6       196365  304     15      0.067567        0.003875                        &#xA;All Reads       ALL     7       211217  343     17      0.070875        0.003827                        &#xA;All Reads       ALL     8       241125  441     18      0.079822        0.003801                        &#xA;All Reads       ALL     9       270272  1068    18      0.172463        0.005277     &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have read a &lt;a href=&quot;https://hereditasjournal.biomedcentral.com/articles/10.1186/s41065-021-00171-3#Sec14&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; that uses the CollectGcBiasMetrics on exome data and they magically produce a plot without bias. I say magically because they don't explain how they use CollectGcBiasMetrics.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt; I added the target coverage plot.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/UNqTb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UNqTb.png&quot; alt=&quot;Target region coverage&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt; Am I missing something? Should I, for instance, use my target regions in instead of the reference genome when using &lt;code&gt;CollectGcBiasMetrics&lt;/code&gt;? I will try it, but I am not sure this is the correct thing to do... It is hard to find any workflow for exome data with gatk and particularly &lt;code&gt;CollectGcBiasMetrics&lt;/code&gt;, which specifies to use the reference genome.&lt;/p&gt;&#xA;&lt;p&gt;From the comment @Maximilian Press&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Do you have on-instrument diagnostics? Did you run FastQC on the raw reads? What do the other picard metrics look like? Did you manually inspect your alignments (strongly recommended)? Do you actually have coverage of the exome?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I will see if I can get some instrument diagnostics. I did look at the alignments, and as far as I can tell I don't see anything weird? Just that it is very deeply sequenced. I do see some intronic regions, with some reads, but only a few reads compared to the many on the targets, which I think is a consequence of the (too) deep sequencing. The fastqc of the raw reads didn't look troubling, and it did not give me any warnings. The picard metrics I thought did not look out of the ordinary. There is good target coverage. Yes, it is sequenced quite deep. Also, the lab tech did tell me they over-cycled because they wanted to be sure to have all the regions/targets.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt; So this bias might be due to too deep sequencing? And another important question, can I still reliably use this data? If so, I guess I can perform some corrections like suggested &lt;a href=&quot;https://support.illumina.com/content/dam/illumina-support/help/Illumina_DRAGEN_Bio_IT_Platform_v3_7_1000000141465/Content/SW/Informatics/Dragen/BiasCorrections_fDG_dtSW.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Any help/suggestions would be very welcome. Thank you in advance.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;" OwnerUserId="16204" LastEditorUserId="16204" LastEditDate="2022-12-30T17:10:44.033" LastActivityDate="2022-12-30T22:50:09.297" Title="What should GC coverage bias plot of exome data look like?" Tags="|gatk|illumina|human-genome|coverage|exome|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20273" PostTypeId="1" CreationDate="2022-12-29T16:18:24.253" Score="1" ViewCount="1014" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9549650/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt; and &lt;a href=&quot;https://support.bioconductor.org/p/9148604/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioconductor Support&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;My aim is to perform gene clustering based on RNA seq datasets (raw expression values) from NCBI database. After clustering I realized that it is hard for me to get the biological understanding of the clusters. So, I thought it will be better if I start with data such as expression change in different conditions with respect to control (log fold change) for my later interpretation.&lt;/p&gt;&#xA;&lt;p&gt;I need to calculate log2 fold change values for lot of different experimental conditions when compared to their corresponding controls. Just to mention, I am not going to use these for differential expression analysis but for some other downstream analysis like clustering and stuff. Traditionally in my field, counts are normalized by TPM method and then fold change values are calculated by log2(TPM_exp+1)-log2(TPM_control+1) [using 1 or 0.5 as pseudo counts for log transformation]. In my case, I realized that TPM is not a good way to normalize the data as I have few samples with lot of reads mapping to only one or two genes [RNA composition bias]. DESeq2 median of ratios nomalization seems to take care of that issue. So, I prefer using DESeq2 normalization. But I cannot use DESeq2 for getting log2 fold change values because I don't have replicates for some of the experimental conditions and DESeq2 needs replicates to estimate log2 fold change values. So, I want to manually calculate log2 fold change values from DESeq2 normalized counts.&lt;/p&gt;&#xA;&lt;p&gt;So, I am using log2(DESeq2norm_exp+0.5)-log2(DESeq2norm_control+0.5) for calculating log2 fold change values. I am not sure whether it is a good idea or the choice of pseudo-count here is very critical. Any comments or help is really appreciated.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;[This question was originally posted on StackOverflow, and someone commented suggesting other communities that would be more appropriate, in particular, here and Bioconductor Support]&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="16709" LastEditorUserId="73" LastEditDate="2023-01-01T07:44:09.053" LastActivityDate="2023-01-01T07:44:09.053" Title="Manually calculating log2 fold change values from DESeq2 normalized counts" Tags="|deseq2|differential-expression|fold-change|" AnswerCount="0" CommentCount="3" ClosedDate="2023-11-16T04:32:43.987" ContentLicense="CC BY-SA 4.0" />
  <row Id="20274" PostTypeId="1" AcceptedAnswerId="20329" CreationDate="2022-12-29T19:20:43.073" Score="2" ViewCount="112" Body="&lt;p&gt;Consider the following short indel polymorphism rs59679400 on chr7.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ rsidx search dbSNP/dbSNP_GRCh38.{vcf.gz,rsidx} rs59679400 | cut -f 1-7&#xA;[rsidx] running version 0.3&#xA;NC_000007.14    12510038        rs59679400      GAAAA   G,GA,GAA,GAAA,GAAAAA    .       .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In the Phase III (2015) 1000 Genomes VCFs (GRCh37), I see the following record for the variant.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ rsidx search 1000GenomesPhaseIII/ALL.chr7.phase3_shapeit2_mvncall_integrated_v5a.20130502.genotypes.{vcf.gz,rsidx} rs59679400 | cut -f 1-7&#xA;[rsidx] running version 0.3&#xA;7       12549664        rs59679400      GA      G       100     PASS&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That's easy enough to interpret, although it looks like only two of the several possible alleles at that locus are represented.&lt;/p&gt;&#xA;&lt;p&gt;When I look at the VCFs from &lt;a href=&quot;https://doi.org/10.1016/j.cell.2022.08.004&quot; rel=&quot;nofollow noreferrer&quot;&gt;the latest NYGC update to the 1000 Genomes data&lt;/a&gt;, I see the following. (&lt;em&gt;I'll note that it's really inconvenient that RS numbers are no longer included for any variants.&lt;/em&gt;)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ tabix 1000GenomesNYGC/1kGP_high_coverage_Illumina.chr7.filtered.SNV_INDEL_SV_phased_panel.vcf.gz chr7:12510038-12510038 | cut -f 1-7&#xA;chr7    12474780        HGSV_106040     G       &amp;lt;DUP&amp;gt;   .       .&#xA;chr7    12510038        7:12510038:GA:G GA      G       .       .&#xA;chr7    12510038        7:12510039:GAA:G        GAA     G       .       .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Ignoring the first record, which is a larger structural variant, we see two records at the position in question. &lt;strong&gt;My initial question is why they've used two records to encode variation for a single indel variant?&lt;/strong&gt; As shown in the dbSNP record in the first snippet, VCF allows one to encode an arbitrary number of alternate alleles in a single record.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My second—and more important—question is how one is to interpret these records and resolve the sequence for any particular individual genotyped at the locus?&lt;/strong&gt; Selecting an arbitrary sample &lt;code&gt;HG00103&lt;/code&gt;, I get the following allele calls for the two records? What am I to make of this?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; from pysam import VariantFile&#xA;&amp;gt;&amp;gt;&amp;gt; vcf = VariantFile(&amp;quot;1000GenomesNYGC/1kGP_high_coverage_Illumina.chr7.filtered.SNV_INDEL_SV_phased_panel.vcf.gz&amp;quot;)&#xA;&amp;gt;&amp;gt;&amp;gt; records = list(vcf.fetch(&amp;quot;chr7&amp;quot;, 12510037, 12510038))&#xA;&amp;gt;&amp;gt;&amp;gt; len(records)&#xA;3&#xA;&amp;gt;&amp;gt;&amp;gt; records[1].samples[&amp;quot;HG00103&amp;quot;].alleles&#xA;('G', 'G')&#xA;&amp;gt;&amp;gt;&amp;gt; records[2].samples[&amp;quot;HG00103&amp;quot;].alleles&#xA;('GAA', 'GAA')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="96" LastActivityDate="2023-01-06T20:00:12.420" Title="Interpreting short indel calls in 1000 Genomes Project VCFs" Tags="|vcf|1000genomes|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20275" PostTypeId="1" AcceptedAnswerId="20299" CreationDate="2022-12-29T19:44:39.280" Score="1" ViewCount="121" Body="&lt;p&gt;Some of my samples have empty clusters. I want to keep the same color for each cluster in different plots.&lt;/p&gt;&#xA;&lt;p&gt;How do I get &lt;code&gt;sc.pl.umap&lt;/code&gt; to keep the same colors for each cluster, even when a cluster is empty?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from matplotlib.pyplot import rc_context&#xA;with rc_context({'figure.figsize': (3, 3)}):&#xA;    fig, (ax1, ax2) = plt.subplots(2, 2, figsize=(6,6))&#xA;    ax1[0] = sc.pl.umap(adata[adata.obs['sample'] == adata.obs['sample'].cat.categories[0]], &#xA;                 color = 'leiden_r1', size=20,&#xA;                 legend_loc = 'on data', ax=ax1[0],&#xA;                 palette = mycolormap_26,&#xA;                 frameon=True, legend_fontsize = 8,title = adata.obs['sample'].cat.categories[0], show=False)&#xA;    ax1[0].set_xlabel('')&#xA;    ax1[0].set(xlim=xlim, ylim=ylim)&#xA;    ax1[1] = sc.pl.umap(adata[adata.obs['sample'] == adata.obs['sample'].cat.categories[1]], &#xA;                 color = 'leiden_r1', size=20,&#xA;                 legend_loc = 'on data', ax=ax1[1],&#xA;                 palette = mycolormap_26,&#xA;                 frameon=True, legend_fontsize = 8,title = adata.obs['sample'].cat.categories[1], show=False)&#xA;    ax1[1].set_xlabel('')&#xA;    ax1[1].set_ylabel('')&#xA;    ax1[1].set(xlim=xlim, ylim=ylim)&#xA;    ax2[0] = sc.pl.umap(adata[adata.obs['sample'] == adata.obs['sample'].cat.categories[2]], &#xA;                 color = 'leiden_r1', size=20,&#xA;                 legend_loc = 'on data', ax=ax2[0],&#xA;                 palette = mycolormap_26,&#xA;                 frameon=True, legend_fontsize = 8,title = adata.obs['sample'].cat.categories[2], show=False)&#xA;    ax2[0].set(xlim=xlim, ylim=ylim)&#xA;    #ax2[0].set_xlabel('')&#xA;    ax2[1] = sc.pl.umap(adata[adata.obs['sample'] == adata.obs['sample'].cat.categories[3]], &#xA;                 color = 'leiden_r1', size=20,&#xA;                 legend_loc = 'on data', ax=ax2[1],&#xA;                 palette = mycolormap_26,&#xA;                 frameon=True, legend_fontsize = 8,title = adata.obs['sample'].cat.categories[3], show=False)&#xA;    ax2[1].set(xlim=xlim, ylim=ylim)&#xA;    #ax2[1].set_xlabel('')&#xA;    ax2[1].set_ylabel('')&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://user-images.githubusercontent.com/33963919/209980418-3ffb8db2-bcdc-4baf-8177-7c0d332394c6.png&quot; alt=&quot;image&quot; /&gt;&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="9698" LastEditorUserId="73" LastEditDate="2023-01-01T07:32:03.447" LastActivityDate="2023-01-02T15:36:06.080" Title="How to preserve colour assignments in matplotlib when a category is empty?" Tags="|single-cell|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20277" PostTypeId="2" ParentId="20274" CreationDate="2022-12-30T02:10:10.690" Score="3" Body="&lt;p&gt;Older GATK was using the so-called variant model where overlapping variants were combined during calling. More recent GATK is using the allelic model where one non-reference allele is evaluated at a time.&lt;/p&gt;&#xA;&lt;p&gt;With the allelic model, a reference allele doesn't mean the sample possesses a reference allele; it means the sample doesn't have the alternate allele. For this particular example, records[1] means a homozygous -A deletion. records[2] means the sample doesn't have the -AA deletion. The two records are consistent with each other.&lt;/p&gt;&#xA;&lt;p&gt;It is still possible to have conflictive records (for example, homozygous -A deletion and homozygous -AA deletion at the same location). At least this used to be a problem with GATK. Not sure if/how much this has been improved recently.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2022-12-30T02:10:10.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20278" PostTypeId="2" ParentId="20272" CreationDate="2022-12-30T03:40:19.663" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Am I missing something? Should I, for instance, use my target regions&#xA;instead of the reference genome?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes you should do this. Why do you think it's wrong doing that? It should be the first thing to check. And that GC bias might be effected by multiple factors. Most prominent one would be PCR over amplification. Most PCR polymerases give horrible GC bias when library is over cycled. There are many articles published on this.&lt;/p&gt;&#xA;&lt;p&gt;I also suggest that you look at the number of reads in each bin. Peaks around 90% GC should not puzzle you so much. There is chance that you have only few reads in these bins out of millions of reads.&lt;/p&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2022-12-30T03:40:19.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20280" PostTypeId="1" AcceptedAnswerId="20282" CreationDate="2022-12-30T14:11:23.090" Score="3" ViewCount="39" Body="&lt;p&gt;So I have ran SAMOVA and made calculations regarding haplotype diversity and nucleotide diversity of several populations. I have also made raster data regarding geographical data like slope, ruggedness etc. Is there a program/package that allows me to combine these to types of data into a map, so I can study if there is a correlation between these indexes and the diversity?&lt;/p&gt;&#xA;" OwnerUserId="16657" LastActivityDate="2022-12-30T19:23:02.563" Title="Is there a way I can combine geological data with population genetics data?" Tags="|phylogenetics|geo|populations|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20281" PostTypeId="1" AcceptedAnswerId="20283" CreationDate="2022-12-30T14:38:54.600" Score="4" ViewCount="211" Body="&lt;p&gt;This is a two part question.&#xA;I am searching for a motif, and in that search I wanted to also find the total number of sequences in my FASTA file, but the code I wrote is not yielding that please see the attempt below. The second part is when I have found the matching motifs, as well as the frequencies of the matches how do i go about to exclude the reference sequence matches and only get matches and frequncies from sequence2-sequence5.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; Ref&#xA;MPPRRSIVEVKVLDVQKRRVPNKHYVYIIRVTWSSGATEAIYRRYSKFFDLQMQMLDKFP&#xA;MEGGQKDPKQRIIPFLPGKILFRRSHIRDVAVKRLIPIDEYCKALIQLPPYISQCDEVLQ&#xA;FFETRPEDLNPPKEEHIGKKKSGNDPTSVDPMVLEQYVVVADYQKQESSEISLSVGQVVD&#xA;IIEKNESGWWFVSTAEEQGWVPATCLEGQDGVQDEFSLQPEEEEKYTVIYPYTARDQDEM&#xA;NLERGAVVEVVQKNLEGWWKIRYQGKEGWAPASYLKKNSGEPLPPKLGPSSPAHSGALDL&#xA;DGVSRHQNAMGREKELLNNQRDGRFEGRLVPDGDVKQRSPKMRQRPPPRRDMTIPRGLNL&#xA;&#xA;&amp;gt;Sequence1&#xA;MAEVRKFTKRLSKPGTAAELRQSVSEAVRGSVVLEKAKLVEPLDYENVITQRKTQIYSDP&#xA;LRDLLMFPMEDISISVIGRQRRTVQSTVPEDAEKRAQSLFVKECIKTYSTDWHVVNYKYE&#xA;DFSGDFRMLPCKSLRPEKIPNHVFEIDEDCEKDEDSSSLCSQKGGVIKQGWLHKANVNST&#xA;ITVTMKVFKRRYFYLTQLPDGSYILNSYKDEKNSKESKGCIYLDACIDVVQCPKMRRHAF&#xA;ELKMLDKYSHYLAAETEQEMEEWLIMLKKIIQINTDSLVQEKKDTVEAIQEEETSSQGKA&#xA;ENIMASLERSMHPELMKYGRETEQLNKLSRGDGRQNLFSFDSEVQRLDFSGIEPDVKPFE&#xA;EKCNKRFMVNCHDLTFNILGHIGDNAKGPPTNVEPFFINLALFDVKNNCKISADFHVDLN&#xA;PPSVREMLWGTSTQLSNDGNAKGFSPESLIHGIAESQLCYIKQGIFSVTNPHPEIFLVVR&#xA;&#xA;&amp;gt;Sequence2&#xA;GDDSEWLKLPVDQKCEHKLWKARLSGYEEALKIFQKIKDEKSPEWSKYLGLIKKFVTDS&#xA;NAVVQLKGLEAALVYVENAHVAGKTTGEVVSGVVSKVFNQPKAKAKELGIEICLMYVEIE&#xA;KGESVQEELLKGLDNKNPKIIVACIETLRKALSEFGSKIISLKPIIKVLPKLFESRDKAV&#xA;RDEAKLFAIEIYRWNRDAVKHTLQNINSVQLKELEEEWVKLPTGAPKPSRFLRSQQELEA&#xA;KLEQQQSAGGDAEGGGDDGDEVPQVDAYELLDAVEILSKLPKDFYDKIEAKKWQERKEAL&#xA;EAVEVLVKNPKLEAGDYADLVKALKKVVGKDTNVMLVALAAKCLTGLAVGLRKKFGQYAG&#xA;HVVPTILEKFKEKKPQVVQALQEAIDAIFLTTTLQNISEDVLAVMDNKNPTIKQQTSLFI&#xA;ARSFRHCTSSTLPKSLLKPFCAALLKHINDSAPEVRDAAFEALGTALKVVGEKSVNPFLA&#xA;&#xA;&amp;gt; Sequence3&#xA;MPPRRSIVEVKVLDVQKRRVPNKHYVYIIRVTWSSGATEAIYRRYSKFFDLQMQMLDKFP&#xA;MEGGQKDPKQRIIPFLPGKILFRRSHIRDVAVKRLIPIDEYCKALIQLPPYISQCDEVLQ&#xA;FFETRPEDLNPPKEEHIGKKKSGNDPTSVDPMVLEQYVVVADYQKQESSEISLSVGQVVD&#xA;IIEKNESGWWFVSTAEEQGWVPATCLEGQDGVQDEFSLQPEEEEKYTVIYPYTARDQDEM&#xA;NLERGAVVEVVQKNLEGWWKIRYQGKEGWAPASYLKKNSGEPLPPKLGPSSPAHSGALDL&#xA;DGVSRHQNAMGREKELLNNQRDGRFEGRLVPDGDVKQRSPKMRQRPPPRRDMTIPRGLNL&#xA;&#xA;&amp;gt;Sequence5&#xA;GDDSEWLKLPVDQKCEHKLWKARLSGYEEALKIFQKIKDEKSPEWSKYLGLIKKFVTDS&#xA;NAVVQLKGLEAALVYVENAHVAGKTTGEVVSGVVSKVFNQPKAKAKELGIEICLMYVEIE&#xA;KGESVQEELLKGLDNKNPKIIVACIETLRKALSEFGSKIISLKPIIKVLPKLFESRDKAV&#xA;RDEAKLFAIEIYRWNRDAVKHTLQNINSVQLKELEEEWVKLPTGAPKPSRFLRSQQELEA&#xA;KLEQQQSAGGDAEGGGDDGDEVPQVDAYELLDAVEILSKLPKDFYDKIEAKKWQERKEAL&#xA;EAVEVLVKNPKLEAGDYADLVKALKKVVGKDTNVMLVALAAKCLTGLAVGLRKKFGQYAG&#xA;HVVPTILEKFKEKKPQVVQALQEAIDAIFLTTTLQNISEDVLAVMDNKNPTIKQQTSLFI&#xA;ARSFRHCTSSTLPKSLLKPFCAALLKHINDSAPEVRDAAFEALGTALKVVGEKSVNPFLA&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The code to search the peptides is below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;import collections&#xA;m_list =[]&#xA;from Bio import SeqIO&#xA;input_file = 'Sequences2.fasta'&#xA;num_sequences = 0&#xA;fasta_sequences = SeqIO.parse(open(input_file),'fasta')&#xA;for fasta in fasta_sequences:&#xA;    num_sequence=+1&#xA;    name, sequence = fasta.id, str(fasta.seq)&#xA;    matches=re.finditer(r&amp;quot;P[A-Z]{2}P..&amp;quot;,sequence)&#xA;    for m in matches:&#xA;        match_id = f'{m.start()}_{m.end()}_{m.group()}'&#xA;        m_list.append(match_id)&#xA;        print(name, m.start(), m.end(), m.group())        &#xA;print(&amp;quot;total sequences&amp;quot;,num_sequences)&#xA;print(&amp;quot;frequency&amp;quot;,collections.Counter(m_list))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It yields the following output below, which I am happy with but now it doesn't give me the total number of sequences:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Ref 73 79 PFLPGK&#xA;Ref 281 287 PLPPKL&#xA;Ref 288 294 PSSPAH&#xA;Sequence3 73 79 PFLPGK&#xA;Sequence3 281 287 PLPPKL&#xA;Sequence3 288 294 PSSPAH&#xA;total sequences 0&#xA;frequency Counter({'73_79_PFLPGK': 2, '281_287_PLPPKL': 2, '288_294_PSSPAH': 2})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;strong&gt;total sequences&lt;/strong&gt; should be &lt;strong&gt;5&lt;/strong&gt;. I used the code below to find out. I do not know where I went wrong since it is not giving me that on the more compact code above.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sequence = open('Sequences2.fasta')&#xA;x = 0&#xA;for line in sequence:&#xA;    if line.startswith('&amp;gt;'):&#xA;        x += 1&#xA;sequence.close()&#xA;print(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The second part of my question is how do I modify my code such that, the matches and frequencies of the reference sequence(Ref) are excluded. The idea would be to get an output such as the one below. I do not even know whether that is possible (A disclaimer this second part is a question on a tutorial that I don't know how to do. I am still learning)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Sequence3 73 79 PFLPGK&#xA;Sequence3 281 287 PLPPKL&#xA;Sequence3 288 294 PSSPAH&#xA;total sequences 5  #this would be the total number of sequences in file including the refrence&#xA;frequency Counter({'73_79_PFLPGK': 1, '281_287_PLPPKL': 1, '288_294_PSSPAH': 1}) #frequency of matches, excluding the reference&#xA;   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16468" LastEditorUserId="16468" LastEditDate="2022-12-30T14:50:42.807" LastActivityDate="2022-12-30T16:03:10.617" Title="Searching motifs in sequence and their frequencies" Tags="|python|proteins|sequence-analysis|motifs|multi-fasta|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20282" PostTypeId="2" ParentId="20280" CreationDate="2022-12-30T14:58:55.097" Score="1" Body="&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt; I looked into modern use of Mantel's test and it will not accept categorical variables. I now remember the null hypothesis of Mantel's ... the association is random between the matrices, i.e. no genetic structure. Therefore &lt;em&gt;p &amp;lt; 0.05&lt;/em&gt; excluding degrees of freedom (so in practice its smaller).&lt;/p&gt;&#xA;&lt;p&gt;The options are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;em&gt;Fst&lt;/em&gt; implementing that is easy but interpreting is complicated because it's about what constitutes genetic structure&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;this can be done in Arlequin &lt;a href=&quot;https://www.youtube.com/watch?v=U2fj3pfPUlE&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.youtube.com/watch?v=U2fj3pfPUlE&lt;/a&gt; as a tutorial&lt;/li&gt;&#xA;&lt;li&gt;Fstat is the more &lt;a href=&quot;https://www2.unil.ch/popgen/softwares/fstat.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;famous package&lt;/a&gt;, but notably old&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;General linear modelling;&lt;/li&gt;&#xA;&lt;li&gt;Machine learning for example random forest, SVM etc ...&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I would use points 1 and point 3, 2 is old school now. I would use point 1 to datamine the ML.&lt;/p&gt;&#xA;&lt;p&gt;The key to Fst is that 0.2 comprises population structure for complicated reasons. Implementing ML is not trivial but is the trendiest analysis going. It is not simply implementation its checking and verifying the associations and avoiding overtraining, which would be the major criticism.&lt;/p&gt;&#xA;&lt;p&gt;The ML would permits the genetic structure to be predicted across broad geographic regions and those predictions are tested and used to reformulate the model. However, you'd need significant expertise in ML (and coding).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The technical word for this is &lt;strong&gt;allopatry&lt;/strong&gt; and if that doesn't mean much, please do read on the subject because that is central to the theory behind the calculations you are performing. There is a HUGE bank of theory on the subject and is the central alternative to Darwinian selection, yep it's that big. So this is not a passing calculation but a fundamental calculation at the heart of the model of speciation of the organisms within your study. What you want to show is your species does not demonstrate allopatry but niche selection. Ultimately this may not be precisely decipherable (depends on the geographic distances).&lt;/p&gt;&#xA;&lt;p&gt;Very briefly the analysis to perform now is the &lt;strong&gt;Mantel's test&lt;/strong&gt;. This should be within the Arlequin package. Mantel's test will assess if there is a 'cline' in your data, i.e. an association with gene frequency and geographic distance - in an allele frequency package (genetic distance of point mutation otherwise).&lt;/p&gt;&#xA;&lt;p&gt;Geological data versus geographic distance ... Mantel's test will cover both calculations, however I think that a simple numerical code will be needed for categorical data. I'd need to check that carefully, because I get that wrong the calculation with be wrong. Basically you need to tell the matrix correlation test that this categorical data being tested.&lt;/p&gt;&#xA;&lt;p&gt;The next level of analysis in allele frequency calculations involves plotting the pairwise (not global) &lt;em&gt;Fst&lt;/em&gt; against against geographic distance and geological type - if the association in Mantel's test is not linear. If the association is linear the Mantel's will be positive, i.e. null hypothesis will be accept, &lt;em&gt;p&lt;/em&gt; &amp;gt; 0.05.&lt;/p&gt;&#xA;&lt;p&gt;I'll check the location of the calculations in various packages later (not necessarily today). I need to check the categorical variable for Mantel's - I've never done that calculation so am not sure.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What you are looking for is Mantel's to reject geographic distance and accept geological data type. That would be a decisive and very publishable result.&lt;/strong&gt;  &lt;em&gt;Fst&lt;/em&gt; will be the fall-back to shift between the hypotheses.&lt;/p&gt;&#xA;&lt;p&gt;If you have loads of SNPs there are phylogenetic tree solutions based on modelling point mutation. The next layer of biophylo-geographic analysis involves packages within the Beast framework. I'm sure that would work here because the number of mutations would be low, also this is really just geographic distance rather than niche. If you've loads of SNPs this would be a good analysis to perform however, but for intraspecifies mtDNA particularly involving &lt;em&gt;cox&lt;/em&gt;1 I personally wouldn't think it would be informative. It is perfectly doable though.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;A positive Mantel's test is a rare result, but generally allopatry is very common so it is a VERY strong null hypothesis for the data. What I am trying to say is you can't just reject allopatry without testing it. ... well you can but not if you want to publish in e.g. Molecular Ecology.&lt;/p&gt;&#xA;&lt;p&gt;Good luck with the study and again I'll update the post later. It looks really good and is more than interesting.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-12-30T19:23:02.563" LastActivityDate="2022-12-30T19:23:02.563" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20283" PostTypeId="2" ParentId="20281" CreationDate="2022-12-30T15:18:24.723" Score="3" Body="&lt;p&gt;The first part appears to be a simple bug if there is no copy/paste error in the code:&lt;/p&gt;&#xA;&lt;p&gt;Original code,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;num_sequence=0&#xA;fasta_sequences = SeqIO.parse(open(input_file),'fasta')&#xA;for fasta in fasta_sequences:&#xA;    num_sequence=+1 # &amp;lt; its here&#xA;print (num_sequence)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The code I'd recommend is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;num_sequence=0&#xA;fasta_sequences = SeqIO.parse(input_file,'fasta')&#xA;for fasta in fasta_sequences:&#xA;    num_sequence += 1 # corrected&#xA;print (num_sequence)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I will look at the second part later.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-12-30T15:18:24.723" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20284" PostTypeId="2" ParentId="20281" CreationDate="2022-12-30T16:03:10.617" Score="4" Body="&lt;p&gt;for the second part re-arrange the &lt;code&gt;for m in matches&lt;/code&gt; block to :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for m in matches:&#xA;        &#xA;        match_id = f'{m.start()}_{m.end()}_{m.group()}'&#xA;        &#xA;&#xA;        print(name, m.start(), m.end(), m.group())  &#xA;        &#xA;        if num_sequences &amp;gt; 1:       #### added if clause so that appends only after first sequence to m_list&#xA;        &#xA;                m_list.append(match_id)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10049" LastActivityDate="2022-12-30T16:03:10.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20285" PostTypeId="2" ParentId="20272" CreationDate="2022-12-30T18:35:40.143" Score="1" Body="&lt;p&gt;Just to formalise my comments.&lt;/p&gt;&#xA;&lt;p&gt;@Supertech's is the right answer.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; The one thing that is unusual is that the bias peaks at around 75%. The only time I've seen strong GC% bias is for particular groups of bacteria. 75% is about the maximum bias that can be achieved in a &lt;em&gt;protein coding gene&lt;/em&gt; and these &amp;quot;extreme bacteria&amp;quot; hit that limit.&lt;/p&gt;&#xA;&lt;p&gt;I better try and explain that a bit better. The primary selection pressure on a gene is the amino acid composition and the GC% is a second issue, i.e. selection pressure, for some organisms (e.g. certain bacteria, insects ..). Thus amino acids are not mutating, but the triplet codon degeneracy permits a bias in the GC% without alternating a given amino acids. In other words this GC% bias is occurring only for &amp;quot;neutral&amp;quot; mutations (clearly there must be a selection pressure, so they are not really neutral).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary and personal thoughts&lt;/strong&gt; In summary the GC% limit is around 75% for a protein coding gene and beyond that it requires the amino acids to be changed. The peak in @Dandelion's data on that limit is weird because GC% artefacts from &lt;em&gt;Taq&lt;/em&gt; errors are not subject to selection pressure: there should not be a peak.&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Contamination&amp;quot; I would avoid as an explanation because in my experience these groups of bacteria are unlikely to invade a lab environment. Sample mix up is a possible explanation. However, I would be cautious about assigning blame because in the chain of command within the wet-lab it could have occurred anywhere.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Official response&lt;/strong&gt; To reiterate the formal and official answer would be @supertech's response and the 75% GC peak remains a mystery.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Please note&lt;/strong&gt; The second plot has a distribution around GC content around 40% ... There appears a discrepancy between plots. Neither output appears compatible with the GC% of humans, but 40% GC with a reasonable proportion of that data on the 50% mark is better the 75% peak plot. Personally I would investigate the empirical basis for the differences.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2022-12-30T22:50:09.297" LastActivityDate="2022-12-30T22:50:09.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20286" PostTypeId="1" CreationDate="2022-12-31T06:28:24.727" Score="2" ViewCount="147" Body="&lt;p&gt;I have plotted a heat-map with 235 rows and 570 columns which is below: how do we make clear names of row and columns on this graph using R?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9s74d.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9s74d.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14059" LastEditorUserId="3694" LastEditDate="2022-12-31T11:50:59.937" LastActivityDate="2023-01-05T19:49:36.570" Title="Display row and column names in R" Tags="|rna-seq|ggplot2|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20287" PostTypeId="1" AcceptedAnswerId="20290" CreationDate="2022-12-31T08:19:45.093" Score="4" ViewCount="409" Body="&lt;p&gt;I'm trying to build a Dockerfile. It has a tool called Atria that utilizes Julia. Since Ubuntu 22.04 does not have Julia package, I had to resort to installing it. The following is the Dockerfile:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FROM ubuntu:22.04&#xA;&#xA;# Install tzdata&#xA;RUN apt-get update &amp;amp;&amp;amp;\&#xA;    DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get -y install tzdata&#xA;&#xA;# Install Python, Bowtie2 and Java&#xA;RUN apt-get install -y python3.10 python3-pip \ &#xA;    openjdk-8-jdk  \ &#xA;    bowtie2 \ &#xA;    wget&#xA;&#xA;RUN  apt-get install --yes --no-install-recommends \&#xA;     zlib1g-dev \&#xA;     libbz2-dev \&#xA;     liblzma-dev  &#xA;&#xA;# Install RSeQC&#xA;RUN pip3 install RSeQC&#xA;&#xA;# Install biopython=1.80&#xA;RUN pip3 install biopython&#xA;&#xA;#Install Julia&#xA;RUN wget https://julialang-s3.julialang.org/bin/linux/x64/1.8/julia-1.8.1-linux-x86_64.tar.gz &amp;amp;&amp;amp; \&#xA;    tar zxvf julia-1.8.1-linux-x86_64.tar.gz &amp;amp;&amp;amp; \&#xA;    mv julia-1.8.1/bin/julia /usr/local/bin/julia &amp;amp;&amp;amp; \&#xA;    chmod +x /usr/local/bin/julia&#xA;&#xA;RUN /sbin/ldconfig -v&#xA;&#xA;# Install Atria&#xA;RUN wget https://github.com/cihga39871/Atria/releases/download/v3.1.2/atria-3.1.2-linux.tar.gz &amp;amp;&amp;amp; \&#xA;    tar -zxf atria-3.1.2-linux.tar.gz &amp;amp;&amp;amp; \&#xA;    mv atria-3.1.2/bin/atria /usr/local/bin/atria &amp;amp;&amp;amp; \&#xA;    chmod +x /usr/local/bin/atria&#xA;&#xA;#Atria dependencies&#xA;RUN  apt-get install pigz pbzip2&#xA;&#xA;# Install findtail&#xA; RUN wget https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/findtail/findtail_v1.01 &amp;amp;&amp;amp; \&#xA;     mv findtail_v1.01 /usr/local/bin/findtail_v1.01 &amp;amp;&amp;amp; \&#xA;     chmod +x /usr/local/bin/findtail_v1.01 &#xA;# Cleanup &#xA;RUN apt clean&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What I have tried so far is :&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Julia v1.6.7 and v1.8.4&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;RUN /sbin/ldconfig -v&lt;/code&gt; as suggested &lt;a href=&quot;https://itsfoss.com/solve-open-shared-object-file-quick-tip/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;How to fix it? Should I try to install Julia using PPA?&lt;/p&gt;&#xA;&lt;p&gt;edit- The error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;root@b0f4b17a7c18:/# atria&#xA;/usr/local/bin/julia: error while loading shared libraries: libjulia.so.1: cannot open shared object file: No such file or directory&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It is recognizing the atria command tho.&lt;/p&gt;&#xA;" OwnerUserId="16328" LastActivityDate="2022-12-31T16:45:54.957" Title="ERR: error while loading shared libraries: libjulia.so.1: cannot open shared object file [Julia] - Ubuntu 22.04" Tags="|docker|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20288" PostTypeId="2" ParentId="16240" CreationDate="2022-12-31T09:11:26.517" Score="1" Body="&lt;p&gt;I think &lt;code&gt;SEQAN_HAS_ZLIB&lt;/code&gt; flag not works correctly as it supposed to be.&#xA;So you can check the seqan library to see if &lt;code&gt;SEQAN_HAS_ZLIB&lt;/code&gt; works correctly, and seek the reason of extension error.&lt;/p&gt;&#xA;&lt;p&gt;Seqan library is a mature library which using template to define general-purpose classes and structs.&lt;/p&gt;&#xA;&lt;p&gt;1.The reason for extension error.&lt;/p&gt;&#xA;&lt;p&gt;Step into the seqan library:&lt;/p&gt;&#xA;&lt;p&gt;When you call BamfileIn: &lt;code&gt;BamFileIn bamFileIn(toCString(bamFileName));&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;An object of BamFileIn struct is created, and &lt;code&gt;bamFileName&lt;/code&gt; file is opened by the struct construction function.&lt;/p&gt;&#xA;&lt;p&gt;And when this object is created, some extra information is wrapped with the object to check the type of file.&lt;/p&gt;&#xA;&lt;p&gt;This information include suffix categories. In bam_file.h header file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;template &amp;lt;typename TDirection, typename TSpec&amp;gt;&#xA;struct FileFormat&amp;lt;FormattedFile&amp;lt;Bam, TDirection, TSpec&amp;gt; &amp;gt;&#xA;{&#xA;#if SEQAN_HAS_ZLIB&#xA;    typedef TagSelector&amp;lt;&#xA;                TagList&amp;lt;Bam,&#xA;                TagList&amp;lt;Sam&#xA;                &amp;gt; &amp;gt;&#xA;            &amp;gt; Type;&#xA;#else&#xA;    typedef Sam Type;&#xA;#endif&#xA;};&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see, if &lt;code&gt;SEQAN_HAS_ZLIB&lt;/code&gt; is setted, &amp;quot;Bam&amp;quot; file will be added to TagSelector which will be wrapped into the BamFileIn struct and redefined by &lt;code&gt;typedef&lt;/code&gt; as &lt;code&gt;format&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Then _open() function is called to open Bam file, and extension is checked in guess_format.h&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;template &amp;lt;typename TFilename, typename TTagList&amp;gt;&#xA;inline bool guessFormatFromFilename(TFilename const &amp;amp;fname, TagSelector&amp;lt;TTagList&amp;gt; &amp;amp;format)&#xA;{&#xA;    typedef typename TTagList::Type TFormat;&#xA;&#xA;    if (value(format) == -1 || isEqual(format, TFormat()))&#xA;    {&#xA;        // if tagId is set to -1 (auto-detect) or the current format (TFormatTag) then test for TFormatTag format&#xA;        if (guessFormatFromFilename(fname, TFormat()))&#xA;        {&#xA;            assign(format, TFormat());&#xA;            return true;&#xA;        }&#xA;    }&#xA;    return guessFormatFromFilename(fname, static_cast&amp;lt;typename TagSelector&amp;lt;TTagList&amp;gt;::Base &amp;amp;&amp;gt;(format));&#xA;}&#xA;&#xA;template &amp;lt;typename TFilename, typename TFormat_&amp;gt;&#xA;inline bool guessFormatFromFilename(TFilename const &amp;amp; fileName, Tag&amp;lt;TFormat_&amp;gt; /*formatTag*/)&#xA;{&#xA;    typedef typename Value&amp;lt;TFilename&amp;gt;::Type                                     TValue;&#xA;    typedef ModifiedString&amp;lt;TFilename const, ModView&amp;lt;FunctorLowcase&amp;lt;TValue&amp;gt; &amp;gt; &amp;gt;    TLowcase;&#xA;    typedef Tag&amp;lt;TFormat_&amp;gt;                                                       TFormat;&#xA;&#xA;    TLowcase lowcaseFileName(fileName);&#xA;    for (unsigned i = 0; i &amp;lt; sizeof(FileExtensions&amp;lt;TFormat&amp;gt;::VALUE) / sizeof(char*); ++i)&#xA;        if (endsWith(lowcaseFileName, lowerString(FileExtensions&amp;lt;TFormat&amp;gt;::VALUE[i])))&#xA;            return true;&#xA;&#xA;    return false;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above code is two instance of  &lt;code&gt;guessFormatFromFilename&lt;/code&gt; function.&#xA;It's pretty complex here, but if &lt;code&gt;SEQAN_HAS_ZLIB&lt;/code&gt; is set correctly, &lt;code&gt;TagSelector&lt;/code&gt; will include bam and sam, and the first &lt;code&gt;guessFormatFromFilename&lt;/code&gt; above will return true in if condition, and the error will not happend. But if &lt;code&gt;SEQAN_HAS_ZLIB&lt;/code&gt; is not working, second formt &lt;code&gt;guessFormatFromFilename&lt;/code&gt; function will be called and only &amp;quot;sam&amp;quot; suffix is exist in &lt;code&gt;TFormat&lt;/code&gt;, and it will be exhibited by &lt;code&gt;FileExtensions&amp;lt;TFormat&amp;gt;::VALUE[i]&lt;/code&gt;. You can using std::cerr to print the suffix on your screen to see if there is only &amp;quot;sam&amp;quot; suffix is exist in &lt;code&gt;TFormat&lt;/code&gt;.&#xA;Althrough you add -DSEAN_HAS_ZLIB, I think you could check the code in library to see if it really works fine.&lt;/p&gt;&#xA;&lt;p&gt;If &lt;code&gt;SEQAN_HAS_ZLIB&lt;/code&gt; is working, you could check other variables or functiones related with open bam process.These function are in &lt;code&gt;bam_file.h&lt;/code&gt;, &lt;code&gt;formatted_file.h&lt;/code&gt;, &lt;code&gt;fundamental_tag.h&lt;/code&gt; and so on.&lt;/p&gt;&#xA;" OwnerUserId="16704" LastEditorUserId="16704" LastEditDate="2023-01-01T08:45:26.457" LastActivityDate="2023-01-01T08:45:26.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20289" PostTypeId="2" ParentId="20286" CreationDate="2022-12-31T11:01:34.710" Score="4" Body="&lt;p&gt;You don't, simple as that, it is too many in both columns and rows to show them all. You could shrink size, but then it becomes unreadable. You can highlight individual genes (rows) that are important to support the message you want to send with that experiment, with packages such as &lt;a href=&quot;https://jokergoo.github.io/ComplexHeatmap-reference/book/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;ComplexHeatmap&lt;/a&gt;, see the &lt;a href=&quot;https://jokergoo.github.io/ComplexHeatmap-reference/book/heatmap-annotations.html#mark-annotation&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mark annotation&lt;/a&gt; section.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, you can use something like hierarchical clustering to form groups, then make functional annotations (such as REACTOME terms) and then annotate eaqch cluster with selected terms, again to support the message you want to send. Go through the ComplexHeatmap manual, lots of examples with code there to get inspiration from.&lt;/p&gt;&#xA;&lt;p&gt;As for the columns, you could use &lt;a href=&quot;https://jokergoo.github.io/ComplexHeatmap-reference/book/heatmap-annotations.html#block-annotation&quot; rel=&quot;nofollow noreferrer&quot;&gt;block annotations&lt;/a&gt; to color-highlight columns bhelonging to the same experimental group, or split the heatmap by these groups to put some visual separation/guidance.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2022-12-31T11:01:34.710" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20290" PostTypeId="2" ParentId="20287" CreationDate="2022-12-31T14:13:34.443" Score="2" Body="&lt;p&gt;You just need to make sure you include the required library files when you install Atria. Since you're not building any software (i.e. using &lt;code&gt;make&lt;/code&gt;), the usual way is to simply move the directory under &lt;code&gt;/opt&lt;/code&gt; and add the executable(s) to your &lt;code&gt;PATH&lt;/code&gt; using the &lt;a href=&quot;https://docs.docker.com/engine/reference/builder/#env&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;ENV&lt;/code&gt;&lt;/a&gt; keyword. Note that Julia comes bundled with Atria, so all you need is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FROM ubuntu:22.04&#xA;&#xA;RUN apt-get update &amp;amp;&amp;amp; apt-get install -y \&#xA;    bowtie2 \&#xA;    openjdk-8-jdk \&#xA;    pbzip2 \&#xA;    pigz \&#xA;    python3.10 \&#xA;    python3-pip \&#xA;    python3-biopython \&#xA;    wget \&#xA;    &amp;amp;&amp;amp; apt-get clean \&#xA;    &amp;amp;&amp;amp; rm -rf /var/lib/apt/lists/*&#xA;&#xA;# Install RSeQC&#xA;RUN pip3 install --no-cache-dir RSeQC&#xA;&#xA;# Install Atria&#xA;RUN wget -q https://github.com/cihga39871/Atria/releases/download/v3.1.2/atria-3.1.2-linux.tar.gz \&#xA;    &amp;amp;&amp;amp; tar xf atria-3.1.2-linux.tar.gz \&#xA;    &amp;amp;&amp;amp; mv atria-3.1.2 /opt/atria&#xA;&#xA;# Install findtail&#xA;RUN wget -q https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/findtail/findtail_v1.01 \&#xA;    &amp;amp;&amp;amp; chmod +x findtail_v1.01 \&#xA;    &amp;amp;&amp;amp; mv findtail_v1.01 /usr/local/bin/findtail &#xA;&#xA;# Add the Atria executable to the PATH&#xA;ENV PATH=&amp;quot;/opt/atria/bin:${PATH}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2022-12-31T14:13:34.443" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20291" PostTypeId="2" ParentId="20287" CreationDate="2022-12-31T16:45:54.957" Score="1" Body="&lt;p&gt;Just to note &lt;code&gt;RUN apt clean&lt;/code&gt;  ... is much better performed using &lt;code&gt;&amp;amp;&amp;amp; apt clean&lt;/code&gt;, like in @Steve's approach, recommended for complicated reasons.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Suggestion&lt;/strong&gt;: what about using the official Julia Docker as the base rather than create it from scratch? There isn't a Ubuntu Julia but there is a Debian base.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker run -it --rm Julia&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker pull julia&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or via a Dockerfile,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FROM Julia:buster&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FROM Julia:latest&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2022-12-31T16:45:54.957" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20292" PostTypeId="2" ParentId="20258" CreationDate="2023-01-01T03:19:39.407" Score="3" Body="&lt;p&gt;I think I've found the answer to my own question.&lt;/p&gt;&#xA;&lt;p&gt;So, I tried using this PRANK web server for predicting the binding site of my protein. The results is suprisingly very similar to the one from the Protein Plus web server. So, I think it's better to use the one from Protein Plus.&lt;/p&gt;&#xA;&lt;p&gt;You can see the results on the link below.&lt;/p&gt;&#xA;&lt;p&gt;References:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://prankweb.cz/viewer?id=5tl8&amp;amp;database=v3-conservation-hmm&amp;amp;created=2022-01-27T07:50:59&amp;amp;lastChange=2022-01-27T07:50:59%20%20&amp;amp;structureName=structure.pdb&amp;amp;predictionName=5TL8&amp;amp;predictedStructure=undefined&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://prankweb.cz/viewer?id=5tl8&amp;amp;database=v3-conservation-hmm&amp;amp;created=2022-01-27T07:50:59&amp;amp;lastChange=2022-01-27T07:50:59%20%20&amp;amp;structureName=structure.pdb&amp;amp;predictionName=5TL8&amp;amp;predictedStructure=undefined&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://proteins.plus/5tl8&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://proteins.plus/5tl8&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16679" LastActivityDate="2023-01-01T03:19:39.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20293" PostTypeId="1" CreationDate="2023-01-01T17:53:35.637" Score="1" ViewCount="21" Body="&lt;p&gt;I have a 1vfl macro molecule with 3000 atoms in it. When i open it in DC accelrys visualizer. It has much bonds. but when i open the pdb file it just have 8 connections. You can download the pdb here(I'm linking because the pdb is much) : &lt;a href=&quot;https://www.rcsb.org/structure/1VFL&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/structure/1VFL&lt;/a&gt;&lt;br&gt;&#xA;I'm using this pdb for my program so i need to know all the connections between atoms.&lt;/p&gt;&#xA;&lt;p&gt;Any help would be good; I'm a complete beginner at using PDB.&lt;/p&gt;&#xA;" OwnerUserId="16720" LastEditorUserId="73" LastEditDate="2023-01-02T19:46:23.517" LastActivityDate="2023-01-02T19:46:23.517" Title="CONECT doesnt include all the connections the 1vfl macro molecule have in a pdb file" Tags="|pdb|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20294" PostTypeId="2" ParentId="20293" CreationDate="2023-01-01T19:06:38.763" Score="5" Body="&lt;p&gt;The &lt;code&gt;CONECT&lt;/code&gt; entries specify the bonding of the ligands (&lt;code&gt;HETATM&lt;/code&gt;), not your polymeric residues (&lt;code&gt;ATOM&lt;/code&gt;). By residue I mean a collection of atoms that forms a unit, not only your standard amino acid definition.&lt;/p&gt;&#xA;&lt;p&gt;You can export in PyMOL all connections if you wanted, but the connectivity of the polymeric residues is standardised and does not need to be specified as they ought to obey the definitions (the 'chemical component' dataset), for example &lt;a href=&quot;https://www.rcsb.org/ligand/ILE&quot; rel=&quot;noreferrer&quot;&gt;https://www.rcsb.org/ligand/ILE&lt;/a&gt;.&#xA;If I were to have a ligand, &lt;code&gt;LIG&lt;/code&gt;, it would be &lt;code&gt;HETATM&lt;/code&gt; entered and with &lt;code&gt;CONECT&lt;/code&gt; entries as most likely it is not the RCSB's &lt;a href=&quot;https://www.rcsb.org/ligand/LIG&quot; rel=&quot;noreferrer&quot;&gt;https://www.rcsb.org/ligand/LIG&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Crosslinks between certain residues ought to be entered via the &lt;code&gt;LINK&lt;/code&gt; record, not &lt;code&gt;CONECT&lt;/code&gt; so they will not feature.&lt;/p&gt;&#xA;&lt;p&gt;Some programs connect atoms by proximity regardless of stated connectivity —spiderwebs in PyMOL.&lt;/p&gt;&#xA;&lt;p&gt;An odd corner case are non-standard amino acids, such as phosphorylations (e.g. &lt;a href=&quot;https://www.rcsb.org/ligand/SEP&quot; rel=&quot;noreferrer&quot;&gt;https://www.rcsb.org/ligand/SEP&lt;/a&gt;) which are &lt;code&gt;ATOM&lt;/code&gt; not &lt;code&gt;HETATM&lt;/code&gt;, which are not meant to have &lt;code&gt;CONECT&lt;/code&gt; entries, but some were deposited with them.&#xA;Water (&lt;code&gt;HOH&lt;/code&gt;) (from neutron diffraction structures) does not need connections.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2023-01-01T19:06:38.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20295" PostTypeId="1" AcceptedAnswerId="20296" CreationDate="2023-01-01T22:33:45.550" Score="1" ViewCount="765" Body="&lt;p&gt;I want to get a ranked gene list for GSEA analysis from single cell data, which needs a group differential expression statistic for ranking.&lt;/p&gt;&#xA;&lt;p&gt;Which is the best method to compare the differently expressed genes between 2 clusters &lt;code&gt;scanpy.tl.rank_genes_groups&lt;/code&gt; (&lt;a href=&quot;https://scanpy.readthedocs.io/en/stable/generated/scanpy.tl.rank_genes_groups.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://scanpy.readthedocs.io/en/stable/generated/scanpy.tl.rank_genes_groups.html&lt;/a&gt;) or &lt;code&gt;MAST.zlm&lt;/code&gt; (&lt;a href=&quot;https://rdrr.io/bioc/MAST/man/zlm.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rdrr.io/bioc/MAST/man/zlm.html&lt;/a&gt;) or &lt;code&gt;FindMarkers&lt;/code&gt; function of &lt;code&gt;Seurat&lt;/code&gt; (&lt;a href=&quot;https://satijalab.org/seurat/reference/findmarkers&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/reference/findmarkers&lt;/a&gt;)?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="9698" LastEditorUserId="73" LastEditDate="2023-01-03T00:58:54.743" LastActivityDate="2023-01-03T00:58:54.743" Title="Best method to compare differentially expressed genes between 2 single cell clusters for GSEA" Tags="|single-cell|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20296" PostTypeId="2" ParentId="20295" CreationDate="2023-01-02T03:29:31.477" Score="1" Body="&lt;p&gt;I wish to focus my contribution singly on &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-017-1382-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;ScanPy&lt;/a&gt; vs. Seurat FindMarkers.&lt;/p&gt;&#xA;&lt;p&gt;ScanPy's claim is it is essentially a speeded up version of Seurat FindMarkers with better performance (discussed below) written in Python.&lt;/p&gt;&#xA;&lt;p&gt;What they are doing are essentially datamining the expression signal using multivariate statistics (PCA) focused through tSNE. There is no controversy or dispute with this approach at all (please see &lt;strong&gt;Caveat&lt;/strong&gt; I shouldn't use the term &amp;quot;datamining&amp;quot; in context - bad habit).&lt;/p&gt;&#xA;&lt;p&gt;What ScanPy is doing is using Graph theory &lt;code&gt;Louvian groups&lt;/code&gt; to extract the differential signal, presumably from the tSNE focused PCA. Cool.&lt;/p&gt;&#xA;&lt;p&gt;If you've already got to grips with ScanPy then leveraging it as a data mining approach - to me - looks sensible. The statistics will not be hugely powerful, that doesn't mean they are bad, it approximately means they are solid.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What is ScanPy doing?&lt;/strong&gt;&#xA;The speeding up is via parallelisation, which is very easy in Python. The implication is Seurat is not parallelised, which would amaze me. The authors talk about some form of 'chunking' (they say 'partially loading the object') via HDF5. This is officially what they say&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;ANNDATA is similar to R’s EXPRESSIONSET [26], but supports sparse data and allows HDF5-based backing of ANNDATA objects on disk, a format independent of platform, framework, and language. This allows operating on an ANNDATA object without fully loading it into memory&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I am confused by this, because &lt;code&gt;HDF5&lt;/code&gt; is a rapid I/O format - its not a datachunking format which his what the authors imply. Yes it facilitates datachunking but simply via rapid I/O. &lt;code&gt;HDF5&lt;/code&gt; is of course independent Python, but why discuss that at length (we all know that) when the critical information is how the partially loaded object is achieved. There's a lot of strategies to achieve the last bit - none of which are discussed.&lt;/p&gt;&#xA;&lt;p&gt;The overall levels of high performance Python  (outside ScanPy) - because the reality is Python's core code is slow - are three steps:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Parallelisation and code optimisation.&lt;/li&gt;&#xA;&lt;li&gt;Data chunking - using algorithm strategies to partial load data e.g. Hadoop strategies.&lt;/li&gt;&#xA;&lt;li&gt;Compiling via &lt;code&gt;JIT&lt;/code&gt;, &lt;code&gt;numba&lt;/code&gt; and static typing via  &lt;code&gt;Cython&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The authors have done point 1, however again &lt;code&gt;HDF5&lt;/code&gt; doesn't facilitate point 2 above, all it does is enable faster read/write. &lt;code&gt;HDF5&lt;/code&gt; is not actually the fastest I/O either both &lt;code&gt;Pacquet&lt;/code&gt; (sp?) and &lt;code&gt;feather&lt;/code&gt; are faster but normally for &lt;code&gt;pandas&lt;/code&gt; dataframes. &lt;code&gt;HDF5&lt;/code&gt; is actually just as fast as &lt;code&gt;pickle&lt;/code&gt; under many tests (!).&lt;/p&gt;&#xA;&lt;p&gt;So the paper is a bit weird in that it is sparsely described - but this does often happen admittedly. The partial loading stuff via &lt;code&gt;HDF5&lt;/code&gt; is weird, because they are not describing how that is done.&lt;/p&gt;&#xA;&lt;p&gt;The other bit that is weird is the ML and ANNDATA class. They state,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;ANNDATA is similar to R’s EXPRESSIONSET [26]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I rarely use &lt;code&gt;R&lt;/code&gt; so to me that isn't clear. The other weird thing is the authors talk about ML, they use an ML library, but give no further details nor why it is used. What I would assume is either: that they use it to assist with &lt;code&gt;pseudotemporal ordering&lt;/code&gt; analysis &lt;strong&gt;OR&lt;/strong&gt; they use it for unsupervised learning (PCA, tSNE). The latter is more likely. &lt;code&gt;Louvian groups&lt;/code&gt; are from the &lt;code&gt;igraph&lt;/code&gt; package they use&lt;/p&gt;&#xA;&lt;p&gt;So in my personal view, despite the paper being a bit (to very) obscure I see no reason why ScanPy cannot replace Seurat purely in terms of the datamining being performed. Python is just as good at PCA and tSNE etc ...  The use of Louvain groups is cool to trap that information. The key benefit of ScanPy is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;your already familiar with it.&lt;/li&gt;&#xA;&lt;li&gt;faster - it definitely could be accelerated further.&lt;/li&gt;&#xA;&lt;li&gt;good for Python&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The drawbacks I can see are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;RNA analysis is dominated by &lt;code&gt;R&lt;/code&gt; and you'll likely pick up a &lt;code&gt;R&lt;/code&gt; focused reviewer (which could be fun).&lt;/li&gt;&#xA;&lt;li&gt;As &lt;em&gt;per&lt;/em&gt; your previous post there appear  shortcomings with ScanPy - but like the development team can't think of everything first time round.&lt;/li&gt;&#xA;&lt;li&gt;ScanPy2 is not released.&lt;/li&gt;&#xA;&lt;li&gt;its not generating a critical (test) probability to distinguish between expression levels just like Seurat - there are real advantages to this however.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Caveat&lt;/strong&gt;  I keep saying &amp;quot;data-mining&amp;quot; and I shouldn't. If the reviewers are perfectly happy with the above analysis alone, that's cool. This is because what I repeatedly call &amp;quot;data-mining&amp;quot; are other investigators final results, particularly in RNA expression.  If your aim is to publish a big RNA-seq data set and identify the core features of e.g. a cellular immune response - good approach. I don't generate data, mine is purely analytics and algorithms, so my perspective (and reviewing standards) are different.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Just to state my interest in DEGs (or DGE). I use unsupervised learning, PCA, tSNE to generate a hypothesis and get a feel for the data. Then however, I do customised full-scale ML (supervised learning) to interrogate the data. Finally, I use DEG to assess which way around the ML result is pointing (under- or over-represented). I will not let the DEG override the ML (supervised learning) analysis - which I see as much stronger analysis. My context is different however and I'm consolidating the result is via extensive custom databases, i.e. very different.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-02T04:11:37.570" LastActivityDate="2023-01-02T04:11:37.570" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20297" PostTypeId="1" AcceptedAnswerId="20300" CreationDate="2023-01-02T10:52:12.680" Score="1" ViewCount="57" Body="&lt;p&gt;I have some problems with ClinCNV(&lt;a href=&quot;https://github.com/imgag/ClinCNV&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/imgag/ClinCNV&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;p&gt;I made a BED file from a BAM, then i made a GC count with ngs-bits (link: &lt;a href=&quot;https://github.com/imgag/ngs-bits&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/imgag/ngs-bits&lt;/a&gt; )&#xA;The main idea of redacting a BED file is making GC content, but i think i need to remove all other columns. Here an example:&lt;/p&gt;&#xA;&lt;p&gt;My BED:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1    10041   10086   NB501410:34:H23TGAFXY:1:11102:23474:15441/1     0       +       0.4889 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In description says:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1    12171   12245   0.4595 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried to run script with my BED file and there is an issue:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; Rscript ClinCNV/clinCNV.R --normal myfolder/myfile.cov --out myfolder/ --bed myfolder/myfileGCannotated.bed --folderWithScript ClinCNV/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and there is an output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;We run script located in folder /root/myfolder . Please, specify ABSOLUTE paths, relative paths do not work for every machine. If everything crashes, please, check the correctness of this path first.&amp;quot;&#xA;&#xA;&#xA;[1] &amp;quot;START cluster allocation.&amp;quot;&#xA;&#xA;[1] &amp;quot;Cluster allocated.&amp;quot;&#xA;&#xA;[1] &amp;quot;END cluster allocation.&amp;quot;&#xA;&#xA;[1] &amp;quot;We are started with reading the coverage files and bed files 2023-01-02 13:49:43&amp;quot;&#xA;&#xA;Error in fread(fileName, header = F, stringsAsFactors = F, sep = &amp;quot;\t&amp;quot;) :&#xA;  File 'myfolder/myfileGCannotated.bed' does not exist or is non-readable. getwd()=='/root/ClinCNV'&#xA;Calls: ReadFileFast -&amp;gt; as.data.frame -&amp;gt; fread&#xA;Execution halted&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now I started the script with absolute paths and it works but I have a new problem:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in setnames(localDf &amp;lt;- as.data.frame(fread(fileName, skip = 1, header = F,  :&#xA;  Can't assign 4 names to a 7 column data.table&#xA;Calls: ReadFileFast -&amp;gt; setnames&#xA;Execution halted&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16725" LastEditorUserId="73" LastEditDate="2023-01-02T19:39:42.620" LastActivityDate="2023-01-02T19:39:42.620" Title="Problems with BED format for ClinCNV" Tags="|ngs|bash|bed|coverage|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20299" PostTypeId="2" ParentId="20275" CreationDate="2023-01-02T14:23:18.287" Score="1" Body="&lt;p&gt;My recommendation is to report the issue directly on the &lt;code&gt;ScanPy&lt;/code&gt; Github page here: &lt;a href=&quot;https://github.com/scverse/scanpy/issues&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/scverse/scanpy/issues&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is a shortcoming with the testing of the algorithm, without any question. Replicates with experimental modification are standard procedure in this line of analysis. For example, one &lt;em&gt;in vitro&lt;/em&gt; cell culture does not receive the drug therapy, i.e. a control, and the rest do, which results in one group with a complete absence of a transcript profile. The authors simply have not described this possibility in any of their documentation and therefore may not have originally considered it.&lt;/p&gt;&#xA;&lt;p&gt;What I suspect has happened is the authorship only tested extensive and complete data - like really big data sets. &amp;quot;BigData&amp;quot; is what &lt;code&gt;ScanPy&lt;/code&gt; is targeting because they look to have adapted to large data in Wolf &lt;em&gt;et al&lt;/em&gt; Genome Biology 2018 paper, notably by specifying how quick their algorithm performs with increasing data set size.&lt;/p&gt;&#xA;&lt;p&gt;What I suspect has happened is that smaller data sets, with experimental replication and modification, were not tested so these issues simply get missed. It's easy to overlook, the authors can't think of everything.&lt;/p&gt;&#xA;&lt;p&gt;The authors of &lt;code&gt;ScanPy&lt;/code&gt; are responding to issues arising unevenly. So it will be hit and miss whether they respond. Similar issues should have been reported to them before because its a common wet-lab experimental design. However, I've not checked their full intray.&lt;/p&gt;&#xA;&lt;p&gt;They should provide the bit of code that will prevent the tedium described above, but then again they may not. If they do, please do write back. The whole &lt;code&gt;ScanPy&lt;/code&gt; project is interesting.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-02T15:36:06.080" LastActivityDate="2023-01-02T15:36:06.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20300" PostTypeId="2" ParentId="20297" CreationDate="2023-01-02T15:41:59.457" Score="1" Body="&lt;p&gt;The error message is telling you:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;We run script located in folder /root/myfolder . Please, specify ABSOLUTE paths, relative paths do not work for every machine. If everything crashes, please, check the correctness of this path first.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So it sounds like you simply need to use absolute instead of relative paths. Something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Rscript /home/yourUser/ClinCNV/clinCNV.R \&#xA;    --normal /home/yourUser/myfolder/myfile.cov \&#xA;    --out /home/yourUser/myfolder/ \&#xA;    --bed /home/yourUser/myfolder/myfileGCannotated.bed \&#xA;    --folderWithScript /home/yourUser/ClinCNV/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Also, don't run things as root unless absolutely necessary. if anything goes wrong or if any of your scripts or tools has a bug, you could end up breaking your machine if you run as root.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-01-02T15:41:59.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20301" PostTypeId="1" CreationDate="2023-01-02T16:21:47.743" Score="1" ViewCount="800" Body="&lt;p&gt;I have some PDB IDs and for every structure, I need all its ligands, so I want to automate the process.&lt;/p&gt;&#xA;&lt;p&gt;The ligands are different for every chain:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat ./7rdx_N_1N7.smi &#xA;C1[C@@]2(C)[C@@H](C[C@@H](C1)O)[C][C]([C]1[C@H]3[C@]([C@H](C[C]21)O)([C@H](CC3)[C]([C])[C][C])C)O   1N7&#xA;$ cat ./7rdx_U_1N7.smi &#xA;C1[C@@]2(C)[C@@H](C[C@@H](C1)O)C[C@H]([C@H]1[C@H]3[C@]([C@H](C[C@H]21)O)([C@H](CC3)[C@H](C)CCC(=O)NC[C][C][N+]([C])([C])[C])C)O 1N7&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, I cannot just download the ligands by its name (e.g. &lt;code&gt;https://files.rcsb.org/ligands/download/ZN_ideal.sdf&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;I've tried to find the structures in &lt;em&gt;.pdb&lt;/em&gt; file for a given structure, but I see only their names:&lt;br /&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/bgwRX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bgwRX.png&quot; alt=&quot;pdb_file_part&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Also, as you could see above, I'll convert the &lt;em&gt;.sdf&lt;/em&gt; / &lt;em&gt;.mol2&lt;/em&gt; files to &lt;em&gt;SMARTS&lt;/em&gt; which seems to be a simpler representation - maybe this information'll make the task easier.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I want to make a script for the process in &lt;em&gt;Python&lt;/em&gt;, and the only way to access the &lt;em&gt;RSCB PDB&lt;/em&gt; with a library I've found was using &lt;code&gt;biopython&lt;/code&gt; but I found there an option to download just a &lt;em&gt;.pdb&lt;/em&gt; file (&lt;code&gt;Bio.PDB.PDBList.retrieve_pdb_file&lt;/code&gt;), not ligands.&lt;/p&gt;&#xA;&lt;p&gt;I've also tried to make my own script, but still I don't know how to access ligands files automatically for every structure. The &lt;em&gt;.pdb&lt;/em&gt; files I can access simply by &lt;code&gt;ftp&lt;/code&gt; (e.g. &lt;code&gt;https://ftp.wwpdb.org/pub/pdb/data/biounit/PDB/divided/rd/&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="9651" LastEditorUserId="73" LastEditDate="2023-01-03T19:10:29.087" LastActivityDate="2023-03-15T21:26:13.740" Title="How to download ligands for PDB structure" Tags="|python|pdb|data-download|" AnswerCount="4" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="20302" PostTypeId="1" CreationDate="2023-01-02T16:50:49.573" Score="2" ViewCount="40" Body="&lt;p&gt;I've just used &lt;em&gt;Open Babel&lt;/em&gt; to convert &lt;em&gt;.sdf&lt;/em&gt; and &lt;em&gt;.mol2&lt;/em&gt; files to &lt;em&gt;SMARTS&lt;/em&gt;, and I'm a bit surprised the output can be different for the same structure. It's confusing me that one structure, one piece of information can be different depending on the file format in which it was saved.&lt;/p&gt;&#xA;&lt;p&gt;I converted a ligand &lt;em&gt;1N7&lt;/em&gt; in &lt;a href=&quot;https://i.stack.imgur.com/16iaJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;two available formats (&lt;em&gt;.sdf&lt;/em&gt; and &lt;em&gt;.mol2&lt;/em&gt;)&lt;/a&gt;  of the &lt;em&gt;7RDX&lt;/em&gt; structure from &lt;em&gt;RSCB PDB&lt;/em&gt; for chain &lt;em&gt;N&lt;/em&gt;, and also for chain &lt;em&gt;U&lt;/em&gt;.&#xA;After the conversion, the &lt;em&gt;SMARTS&lt;/em&gt; strings for the ligand in both chains are not the same for both input files types (i.e. the difference between the same ligand, from the same chain, just in two different formats available to download from RSCB PDB):&lt;/p&gt;&#xA;&lt;h1&gt;example no. 1:&lt;/h1&gt;&#xA;&lt;p&gt;[additional &lt;code&gt;@H&lt;/code&gt; added in places, some square brackets added]&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat ./7rdx_N_1N7_from_mol2.smi&#xA;&#xA;C1[C@@]2(C)[C@@H](C[C@@H](C1)O)[C][C]([C]1[C@H]3[C@]([C@H](C[C]21)O)([C@H](CC3)[C]([C])[C][C])C)O   1N7&#xA;&#xA;$ cat ./7rdx_N_1N7_from_sdf.smi&#xA;&#xA;C1[C@@]2(C)[C@@H](C[C@@H](C1)O)C[C@H]([C@H]1[C@H]3[C@]([C@H](C[C@H]21)O)([C@H](CC3)[C@H](C)CC)C)O   1N7&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h1&gt;example no. 2:&lt;/h1&gt;&#xA;&lt;p&gt;[some square brackets added]&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat ./7rdx_U_1N7_from_mol2.smi&#xA;&#xA;C1[C@@]2(C)[C@@H](C[C@@H](C1)O)C[C@H]([C@H]1[C@H]3[C@]([C@H](C[C@H]21)O)([C@H](CC3)[C@H](C)CCC(=O)NC[C][C][N+]([C])([C])[C])C)O 1N7&#xA;&#xA;$ cat ./7rdx_U_1N7_from_sdf.smi&#xA;&#xA;C1[C@@]2(C)[C@@H](C[C@@H](C1)O)C[C@H]([C@H]1[C@H]3[C@]([C@H](C[C@H]21)O)([C@H](CC3)[C@H](C)CCC(=O)NCCC[N+](C)(C)C)C)O   1N7&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In both examples, the structures look quite similar to each other, but the difference is a little worrisome, because there is no chance to check every output by hand.&lt;/p&gt;&#xA;&lt;p&gt;Is it a normal behaviour during the conversion?&lt;br /&gt;&#xA;Is it related to a different notation in the input files?&lt;br /&gt;&#xA;Which input format is better?&lt;/p&gt;&#xA;" OwnerUserId="9651" LastEditorUserId="73" LastEditDate="2023-01-03T19:09:20.287" LastActivityDate="2023-01-05T15:33:57.257" Title="Why are conversion output SMARTS strings are not the same for different file types of the same structure?" Tags="|pdb|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20303" PostTypeId="1" CreationDate="2023-01-02T17:34:57.093" Score="2" ViewCount="641" Body="&lt;p&gt;In the beginning of my pipeline, I just fed the paired reads (2 files) into fastp, with the default options, and assumed it would do a good job preparing the reads for the next step:  alignment&lt;/p&gt;&#xA;&lt;p&gt;But I now checked the files in FastQC and it doesn't show much difference from before the fastp processing: Some extra yellow and red signals appear. I think the end result is even worse.&lt;/p&gt;&#xA;&lt;p&gt;fastqc doesn't analyse paired reads files together. So it might be wrong regarding the paired quality, taken as a whole.&lt;/p&gt;&#xA;&lt;p&gt;Anything else I should use together with FastQC, to verify the output quality?&lt;/p&gt;&#xA;&lt;p&gt;I'm aligning a bunch of poolseqs and then comparing them to find consistency in the change of snp frequencies&lt;/p&gt;&#xA;&lt;p&gt;As requested here is one of the examples that made me ask this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;Before fastp. only one problem noted by fastqc&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8Fl4c.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8Fl4c.png&quot; alt=&quot;before fastp. only one problem noted by fastqc&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;After fastp. overall quality seems to have decreased&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/LPxDf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LPxDf.png&quot; alt=&quot;after fastp. overall quality seems to have decreased&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The raw data is a pool seq from illumina.&lt;/p&gt;&#xA;" OwnerUserId="9373" LastEditorUserId="9373" LastEditDate="2023-01-30T17:43:12.687" LastActivityDate="2023-01-30T17:43:12.687" Title="If fastp output is not a good measure of FASTQ correctness, what is?" Tags="|fastq|reads|fastqc|read-correction|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="20304" PostTypeId="1" CreationDate="2023-01-02T21:21:46.180" Score="2" ViewCount="104" Body="&lt;p&gt;There is no &lt;code&gt;filtered_feature_bc_matrix&lt;/code&gt; in 10X Genomics' &lt;code&gt;cellranger multi&lt;/code&gt; pipeline output when multiplexing analysis was performed.&#xA;There are indeed &lt;code&gt;sample_filtered_feature_bc_matrix&lt;/code&gt; for each sample after demultiplexing. But there are still some No tag-assigned cells that are not included in any sample, and I have to filter those cells before analysis.&#xA;Is there a package that can filter the &lt;code&gt;raw_feature_bc_matrix&lt;/code&gt; of &lt;code&gt;cellranger multi&lt;/code&gt; pipeline output?&#xA;Is this package &lt;code&gt;scCB2&lt;/code&gt;(&lt;a href=&quot;https://github.com/zijianni/scCB2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/zijianni/scCB2&lt;/a&gt;) suitable for this purpose?&lt;/p&gt;&#xA;&lt;p&gt;I tried scCB2 with the following parameters, I am not sure whether these parameters will produce similar output with cellranger’s filter method.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(scCB2)&#xA;library(SummarizedExperiment)&#xA;&#xA;for (i in data_file) {&#xA;    print(i)&#xA;    tmp =  QuickCB2(dir = i, FDR_threshold = 0.01, lower = 100, Ncores = 12)&#xA;    DropletUtils::write10xCounts(paste0(&amp;quot;./data/filtered/&amp;quot;, basename(i)), tmp, version = &amp;quot;3&amp;quot;)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I got 83836 cells after filtering, but the &lt;code&gt;tag_calls_per_cell.csv&lt;/code&gt; file of the &lt;code&gt;multiplexing_analysis&lt;/code&gt; only contains 19427 cells.&#xA;It seems I have to use a more stringent filter.&lt;/p&gt;&#xA;&lt;p&gt;Is there a python package that can do a similar thing?&#xA;Thanks&lt;/p&gt;&#xA;" OwnerUserId="9698" LastEditorUserId="9698" LastEditDate="2023-01-03T14:41:24.840" LastActivityDate="2023-01-03T14:41:24.840" Title="Is there a package that can filter the `raw_feature_bc_matrix` of 10X's `cellranger multi` pipeline output?" Tags="|single-cell|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20305" PostTypeId="1" CreationDate="2023-01-03T12:08:59.687" Score="3" ViewCount="128" Body="&lt;p&gt;I come from this question of gene exression analysis: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/4818/should-pca-be-standardized-for-gene-expression&quot;&gt;Should PCA be standardized for gene expression?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;My experiment is based in 151 x 51 (individuals/samples x genes), in which, patients are subjected to 3 possible groups (~ balanced groups), so we have 2 measurements of gene expression: one in the baseline, and one after the intervention. The gene expression is actually summarised in an scalar (number) called fold-change. Fold-change is calculated through the &lt;em&gt;&lt;strong&gt;Ct&lt;/strong&gt;&lt;/em&gt; (&lt;em&gt;cycle threshold&lt;/em&gt;), which is the number of the cycle in which you are able to detect a certain amount of the target gen. This value or cycle is normalised using Ct values of 3 reference genes which are supposed to remain stable, and each patient is measured using oneself baseline value, in the same run, to remove run-to-run variation. As you can see I am working with RT-qPCR and relative quantification values.&lt;/p&gt;&#xA;&lt;p&gt;As you can see, &amp;quot;normalization&amp;quot; or &amp;quot;standardization&amp;quot; is performed, not in the strict meaning of mathematical foundation, but to normalize quantitation of targets in the experiments. I was wondering if, taking into account that normalization is done, when I try to perform PCA to see how these genes behave, if they form cluster, if the intervention received has influence of the expression, patterns...&lt;/p&gt;&#xA;&lt;p&gt;Reading about, I found quite divergent opinions about scaling or standardizing the results. The conflict, as far as I can see in my own experiment, lies in all my variables are genes, measured with the same high-throughput analyzer, different batches, but normalized with reference genes, and their own baseline values.&lt;/p&gt;&#xA;&lt;p&gt;On one hand I found quite homogeneous working with scaled values, but on the other hand, I am working with values that seem to be working on the same scale, and have been through several filters to normalize results. Not sure if executing more mathematical operations could modify in the wrong way. &lt;strong&gt;What is your opinion about it?&lt;/strong&gt; (all is about using the covariance matrix or correlation matrix based PCA, as far as I understand)&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, the dataset contain missing values, so I am working with the missMDA package to impute them. &lt;strong&gt;In case I scaled the data, what should I do first: scale or impute the missing values?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The steps to calculate fold-change:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uclJ8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uclJ8.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update 1:&lt;/strong&gt; About the technology used, is a OpenArray plate (analyzed with QuantStudio™ 12K Flex Real-Tim) which contains subarrays with customised target genes.It is a 3,072-reaction high-throughput real-time PCR. The reaction are individually performed (as far as I know, getting &lt;em&gt;Ct&lt;/em&gt; for each sample and assay). About the 2nd point, it is 2x time series study (baseline and post-intervention)- But if you see the equations to calculate the fold-change, I am going to obtain a single value of gene expression combining the 2 &lt;em&gt;Ct&lt;/em&gt; or measurements for each patient (as I told: baseline and post-intervention).&lt;/p&gt;&#xA;&lt;p&gt;About the &amp;quot;genes aim&amp;quot; you can see that my interest is purely human genes, after an intervention (inflammation-related genes).&lt;/p&gt;&#xA;&lt;p&gt;About the pippeting issue. As you know is never discarded, but the internal controls or reference genes (as I am adressing them) goal is to normalize results and remove potential quantification errors. However, as you said the RNA concentration (A260) and purity were calculated spectrophotometrically from peripheral blood samples origin&lt;/p&gt;&#xA;&lt;p&gt;As far as I know, the terms correlation and covariance related to PCA, are meant to discrimanate the standardization process (in correlation matrix) and not standardized matrix (scale function in R). Not sure if I understood this is what you imply?&lt;/p&gt;&#xA;&lt;p&gt;My guess is nowadays everything is RNAseq and microarrays, whose mechanisms, and therefore, statistical preprocesses are different from RT-qPCR. But there are papers of PCA working wiht data coming from RT-qPCR. Shouldn't it work though?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Covariance and correlation: &lt;em&gt;&amp;quot;Covariance analysis of PCA: PCA is dimensionality reduction based on maximising multi-dimensional variance so that would be unusual, but correlation matrix of PCA is understandable. Normally covariance analysis is performed directly on standardised data.&amp;quot;&lt;/em&gt;&#xA;Let me insist on this topic a little bit more.I am not sure if I understood correctly what you stated, but the information I've reviewed seems to say otherwise.&#xA;As far as I was able to find, &lt;a href=&quot;https://aedin.github.io/PCAworkshop/articles/b_PCA.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://aedin.github.io/PCAworkshop/articles/b_PCA.html&lt;/a&gt; : &amp;quot;&lt;em&gt;PCA was computed as a singular value decomposition (SVD) of a column centered, scaled matrix. This was PCA of the correlation matrix. If the matrix is centered but not scaled, it is PCA of the covariance matrix.&lt;/em&gt;&amp;quot; I am not sure if I understood correctly what you stated, but the information I've reviewed seems to say otherwise&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;&lt;em&gt;The correlation matrix is the standardized version of the covariance matrix. Analysing the correlation matrix is a useful default method because it takes the standardized form of the matrix; therefore, if variables have been measured using different scales this will not affect the analysis. Often you will want to analyse variables that use different measurement scales. Analysing the correlation matrix ensures that differences in measurement scales are accounted for. In addition, even variables measured using the same scale can have very different variances and this too creates problems for principal component analysis. Using the correlation matrix eliminates this problem also. There are statistical reasons for preferring to analyse the covariance matrix (the reason being that correlation coefficients are insensitive to variations in the dispersion of data whereas covariance is and so produces better-defined factor structures (Tinsley &amp;amp; Tinsley, 1987)) and generally the results will differ from analysis on the correlation matrix. However, the covariance matrix should be analysed only when your variables are commensurable.&lt;/em&gt;&amp;quot;&#xA;Conclusion: shouldn't it be the covariance matrix the one without standardized values, because variables share same scale?&lt;/p&gt;&#xA;&lt;p&gt;Missing values: this is an issue, at least with several functions in R (prcomp(), princomp())which don't work with NAs in the matrix.&lt;/p&gt;&#xA;&lt;p&gt;About this statement: &amp;quot;The key question, which is standardisation and I would say it is essential for RT-qPCR derived data at a minimum to at least investigate consistency of signal against a control. That minimum is what the second equation describes (why it works is complicated).&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;I don't really understand what you mean standardize at a minimum. The endogenous control genes we are using are individualized for every sample. Let say we are investigating the fold-change in the individual &lt;strong&gt;i&lt;/strong&gt; and the variable &lt;strong&gt;k&lt;/strong&gt;. We are using endogenous controls in the &lt;strong&gt;i&lt;/strong&gt; individual to &amp;quot;standardize&amp;quot; values. The&lt;/p&gt;&#xA;" OwnerUserId="16535" LastEditorUserId="16535" LastEditDate="2023-01-09T10:27:31.693" LastActivityDate="2024-02-03T12:02:25.360" Title="Should be RT-qPCR values standardized before PCA analysis?" Tags="|r|gene-expression|pca|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20306" PostTypeId="2" ParentId="20305" CreationDate="2023-01-03T15:55:00.957" Score="0" Body="&lt;p&gt;This is as good an answer I can give based the information. The issue here is that the experiment is derived via RT-qPCR and thats not the same as RNAseq type approach that the normal basis for questions on this site. Generally, we sort of assume RNAseq for gene expression where all the gene expression within a sample are automatically standardised against each other. Thus PCA can be performed directly on that data type. I will summarise this at the end because it's the key point I'm trying to make.&lt;/p&gt;&#xA;&lt;p&gt;In your experiment there are 51 separate reactions per patient (or is 51 the total time series e.g. for one gene??) and it's not clear whether this constitutes a time series (but there must be some time-series aspect to the study: I assume the 'baseline' is at least one point in a time series). That is a lot of opportunity for variation within &lt;strong&gt;one&lt;/strong&gt; patient sample, even if it is the same time point. Maybe the tech doing the RT-qPCR makes a pipetting error for 1 in 10 samples - the results could collapse. The other issue is whether the RT-qPCR is internally standardised, i.e. are you assessing two amplicons per PCR and the same control amplicon is used for each different gene under investigation? This traps the 1 in 10 hypothetical error rate because that is the first point of standardisation. The question is whether a further standardisation is required beyond that, i.e. per patient - thats tricky but basically its difficult to avoid because of variation between different samples from the same patient (again you must be doing some sort of time series). So for example the RNA extraction could have different amounts of blood etc ... However, it needs some thought because here you are standardisation a standardised result against a backdrop of an unstated hypothesis (comments).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; The single point I want to make is that large scale RT-qPCR is quite different from the standard RNA-seq approach on this site from a statistical perspective, i.e. it appear to be 51 reactions per sample whereas normally the approach is 1 reaction per sample which generates all the expression data across thousands of genes, thus PCA is a fantastic analysis for this data-type. This difference creates a layer of statistical complexity and I what I would do is carefully consider my hypothesis and formulate an analytical plan in accordance with the aims.&lt;/p&gt;&#xA;&lt;p&gt;The question is whether a simple standardisation could be performed and then shift straight into PCA? Yes. Is that sufficient? I dunno. I think you might be avoiding time-series analysis by standardisation and then subtraction of the baseline which is then placed into PCA. At a minimum I would perform two PCA analyses one for baseline and one for post-treatment.&lt;/p&gt;&#xA;&lt;p&gt;Generally, a complex standardisation (like I'm inferring) is not cool if the next stage of the analysis is PCA.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Final thoughts&lt;/strong&gt; Although PCA is considered the primary analysis here. I think it takes serious thinking given all the available information to formulate an analytical plan from my experience in this type of data, but as a rough guide (given I dunno the experimental aims) a complex standardisation and then I personally would consider covariance thereafter for a time series rather than PCA as general approach here. &lt;em&gt;However&lt;/em&gt;, two PCAs before and after  using a simple standardisation and compare presence/absence, that could work and very clear results would be accepted.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; Just to note that &amp;quot;correlation matrix based PCA&amp;quot; is basically PCA, it's just trying to sort out the groups. However &amp;quot;covariance&amp;quot; is a different analysis to PCA and statistically and the aims are very different. Also just to mention, but I assume you know this, &amp;quot;correlation&amp;quot; is different from &amp;quot;correlation matrix based PCA&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Missing values&lt;/strong&gt;&#xA;Missing values for PCA isn't an issue - you just miss them out. Missing values in a time series can have lots of different solutions, the key is justification.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Covariance analysis of PCA: PCA is dimensionality reduction based on maximising multi-dimensional variance so that would be unusual, but correlation matrix of PCA  is understandable. Normally covariance analysis is performed directly on standardised data.&lt;/p&gt;&#xA;&lt;p&gt;Overall, I am now clearer one part of the calculation: this is essentially an &lt;em&gt;ad hoc&lt;/em&gt; statistic to assess fold-change of the raw &lt;em&gt;Ct&lt;/em&gt; via a single /minimal PCAs following one or more standardisations.&lt;/p&gt;&#xA;&lt;p&gt;What you are measuring is the optimised multi-dimensional variance of the fold-change of raw &lt;em&gt;Ct&lt;/em&gt; after at least one or else numerous standardisations. So this is not the actual fold-change of &lt;em&gt;Ct&lt;/em&gt;, but variance of the fold-change derived via Eigen values.&lt;/p&gt;&#xA;&lt;p&gt;If you are clear about that and what this represents and the reader is clear about this then it would be up to the reviewers.&lt;/p&gt;&#xA;&lt;p&gt;The key question, which is standardisation and I would say it is essential for RT-qPCR derived data at a minimum to at least investigate consistency of signal against a control. That &lt;em&gt;minimum&lt;/em&gt; is what the second equation describes (why it works is complicated).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-04T16:08:23.347" LastActivityDate="2023-01-04T16:08:23.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20307" PostTypeId="1" CreationDate="2023-01-03T16:44:08.530" Score="3" ViewCount="222" Body="&lt;p&gt;I used &lt;code&gt;muon&lt;/code&gt;(&lt;a href=&quot;https://github.com/scverse/muon&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/scverse/muon&lt;/a&gt;) for single cell Multimodal omics analysis.&#xA;How to concatenate 2 &lt;code&gt;mdata&lt;/code&gt; from different samples? How can I combine more than 2 samples for analysis together?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import anndata as ad&#xA;import muon as mu&#xA;import os&#xA;&#xA;mdata = mu.read_10x_h5(os.path.join(samples[0], &amp;quot;filtered_feature_bc_matrix.h5&amp;quot;))&#xA;mdata.var_names_make_unique()&#xA;mdata2 = mu.read_10x_h5(os.path.join(samples[1], &amp;quot;filtered_feature_bc_matrix.h5&amp;quot;))&#xA;mdata2.var_names_make_unique()&#xA;&#xA;mdata3 = mdata.concatenate(mdata2, batch_key='sample_id')&#xA;---------------------------------------------------------------------------&#xA;AttributeError                            Traceback (most recent call last)&#xA;Cell In[25], line 1&#xA;----&amp;gt; 1 mdata3 = mdata.concatenate(mdata2, batch_key='sample_id')&#xA;      2 mdata3&#xA;&#xA;AttributeError: 'MuData' object has no attribute 'concatenate'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;mdata3 = ad.AnnData.concatenate(mdata, mdata2)&#xA;mdata3&#xA;---------------------------------------------------------------------------&#xA;TypeError                                 Traceback (most recent call last)&#xA;Cell In[26], line 1&#xA;----&amp;gt; 1 mdata3 = ad.AnnData.concatenate(mdata, mdata2)&#xA;      2 mdata3&#xA;&#xA;File ~/miniconda3/lib/python3.8/site-packages/anndata/_core/anndata.py:1770, in AnnData.concatenate(self, join, batch_key, batch_categories, uns_merge, index_unique, fill_value, *adatas)&#xA;   1767     adatas = adatas[0]  # backwards compatibility&#xA;   1768 all_adatas = (self,) + tuple(adatas)&#xA;-&amp;gt; 1770 out = concat(&#xA;   1771     all_adatas,&#xA;   1772     axis=0,&#xA;   1773     join=join,&#xA;   1774     label=batch_key,&#xA;   1775     keys=batch_categories,&#xA;   1776     uns_merge=uns_merge,&#xA;   1777     fill_value=fill_value,&#xA;   1778     index_unique=index_unique,&#xA;   1779     pairwise=False,&#xA;   1780 )&#xA;   1782 # Backwards compat (some of this could be more efficient)&#xA;   1783 # obs used to always be an outer join&#xA;   1784 out.obs = concat(&#xA;   1785     [AnnData(sparse.csr_matrix(a.shape), obs=a.obs) for a in all_adatas],&#xA;   1786     axis=0,&#xA;   (...)&#xA;   1790     index_unique=index_unique,&#xA;   1791 ).obs&#xA;&#xA;File ~/miniconda3/lib/python3.8/site-packages/anndata/_core/merge.py:873, in concat(adatas, axis, join, merge, uns_merge, label, keys, index_unique, fill_value, pairwise)&#xA;    870 X = concat_Xs(adatas, reindexers, axis=axis, fill_value=fill_value)&#xA;    872 if join == &amp;quot;inner&amp;quot;:&#xA;--&amp;gt; 873     layers = inner_concat_aligned_mapping(&#xA;    874         [a.layers for a in adatas], axis=axis, reindexers=reindexers&#xA;    875     )&#xA;    876     concat_mapping = inner_concat_aligned_mapping(&#xA;    877         [getattr(a, f&amp;quot;{dim}m&amp;quot;) for a in adatas], index=concat_indices&#xA;    878     )&#xA;    879     if pairwise:&#xA;&#xA;File ~/miniconda3/lib/python3.8/site-packages/anndata/_core/merge.py:459, in inner_concat_aligned_mapping(mappings, reindexers, index, axis)&#xA;    456 def inner_concat_aligned_mapping(mappings, reindexers=None, index=None, axis=0):&#xA;    457     result = {}&#xA;--&amp;gt; 459     for k in intersect_keys(mappings):&#xA;    460         els = [m[k] for m in mappings]&#xA;    461         if reindexers is None:&#xA;&#xA;File ~/miniconda3/lib/python3.8/site-packages/anndata/_core/merge.py:68, in intersect_keys(ds)&#xA;     67 def intersect_keys(ds: Collection) -&amp;gt; OrderedSet:&#xA;---&amp;gt; 68     return reduce(and_, map(OrderedSet, ds))&#xA;&#xA;File ~/miniconda3/lib/python3.8/site-packages/anndata/_core/merge.py:32, in OrderedSet.__init__(self, vals)&#xA;     31 def __init__(self, vals=()):&#xA;---&amp;gt; 32     self.dict = OrderedDict(zip(vals, repeat(None)))&#xA;&#xA;TypeError: 'NoneType' object is not iterable&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="9698" LastEditorUserId="298" LastEditDate="2023-01-03T20:36:35.703" LastActivityDate="2023-01-03T20:36:35.703" Title="How to concatenate 2 `mdata` of `muon` package?" Tags="|python|single-cell|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20308" PostTypeId="1" CreationDate="2023-01-03T23:10:36.043" Score="1" ViewCount="121" Body="&lt;p&gt;I have a matrix that looks something like this:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;chr&lt;/th&gt;&#xA;&lt;th&gt;pos&lt;/th&gt;&#xA;&lt;th&gt;het&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chrI&lt;/td&gt;&#xA;&lt;td&gt;27&lt;/td&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chrI&lt;/td&gt;&#xA;&lt;td&gt;55&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chrI&lt;/td&gt;&#xA;&lt;td&gt;27&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chrI&lt;/td&gt;&#xA;&lt;td&gt;55&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;It's essentially a VCF file that I have converted into a TSV. I want to be able to take all the values in the &lt;code&gt;het&lt;/code&gt; column and apply this function:&#xA;&lt;code&gt;het_df['het'] = het_df['het'].transform(lambda x: -1*(x-1))&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;That line is from a python script and it works really well in transforming that column. However, as I am making plots from this data I want to be able to do this in R. When I went to convert it to an R script I started with this function:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;convert_df &amp;lt;- lapply(het_df[,3], FUN=function(x) formula=(-1*(x-1)))&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;As I saw it in this issue: &lt;a href=&quot;https://stackoverflow.com/questions/7009333/how-to-use-lapply-with-a-formula&quot;&gt;https://stackoverflow.com/questions/7009333/how-to-use-lapply-with-a-formula&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, I keep getting error messages such as this one:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in x - 1 : non-numeric argument to binary operator&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which is odd because when I run &lt;code&gt;het_df[,3]&lt;/code&gt; it gives me the correct output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   [1] &amp;quot;2&amp;quot; &amp;quot;0&amp;quot; &amp;quot;1&amp;quot; &amp;quot;1&amp;quot; &amp;quot;1&amp;quot; &amp;quot;1&amp;quot; &amp;quot;1&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I think I'm just a little lost and I don't know what to do to run this function...everything worked fine in my python script (I was able to generate plots with matplotlib) but I like manipulating plots more in R than python so I wanted to be able to convert the script. If anyone is able to see what I'm clearly doing wrong please let me know! Thank you!&lt;/p&gt;&#xA;&lt;p&gt;Here is the python code I've been using in case this helps (I'm transforming the last column so I always get a 1 or a 0 for heterozygosity/homosygosity and then using a rolling window to obtain the variability in the data series and plotting them):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;import matplotlib.pyplot as plt&#xA;import numpy as np&#xA;import seaborn as sns&#xA;&#xA;# read in file&#xA;het_df = pd.read_csv(&amp;quot;het_file.tsv&amp;quot;, sep=&amp;quot;\t&amp;quot;, header=None).set_axis(['chr', 'pos', 'het'], axis=1, copy=False)&#xA;&#xA;# getting rid of homozygous alt allele&#xA;# always get 1 for heterozygosity&#xA;het_df['het'] = het_df['het'].transform(lambda x: -1*(x-1))&#xA;&#xA;windows = [25, 50, 100, 200] # 100 bp, get rid of a lot of noise&#xA;steps = [5]&#xA;&#xA;for window in windows:&#xA;    for step in steps:&#xA;        het_ratio = {}&#xA;        for chr_het_df in het_df.groupby('chr'):&#xA;            # rolling over the window&#xA;            # center = have the count in the center of the window&#xA;            # 50bp window count in the front or back of window or count in the center&#xA;            # ratio of heterozygosity&#xA;            het_ratio[chr_het_df[0]] = chr_het_df[1].set_index('pos').rolling(window=window, step=step, center=True).mean()&#xA;        fig, ax = plt.subplots(len(het_ratio), 1, sharex=True)&#xA;        for index in range(len(het_ratio)):&#xA;            chromosome = list(het_ratio.keys())[index] &#xA;            sns.scatterplot(het_ratio[chromosome], ax=ax[index], s=4)&#xA;            ax[index].set_ylim(bottom=0, top=1)&#xA;            ax[index].legend([], [], frameon=False)&#xA;            ax[index].set(xlabel=&amp;quot;Chromosome position&amp;quot;, ylabel=chromosome)&#xA;            ax[index].set_yticks([0, 0.5, 1])&#xA;            ax[index].set_yticklabels([0, 0.5, 1], size = 6)&#xA;        fig.subplots_adjust(top=0.98, bottom=0.05, left=0.05, right=0.98)&#xA;        fig.set_size_inches(20, 15)&#xA;        plt.savefig(&amp;quot;het_plot&amp;quot; + str(window) + &amp;quot;_step&amp;quot; + str(step) + &amp;quot;.png&amp;quot;, dpi=150)&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="3694" LastEditDate="2023-01-04T11:24:58.060" LastActivityDate="2023-01-05T20:02:45.980" Title="How to apply a mathematical function to a matrix in R" Tags="|r|python|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20309" PostTypeId="1" CreationDate="2023-01-03T23:52:36.797" Score="2" ViewCount="75" Body="&lt;p&gt;I have an input file that is a URL. Cromwell (v84) fails to pull it when I'm not using &amp;quot;Local&amp;quot; as a backend.  The only difference between a working case and failed case is the name of the backend in the cromwell config.&lt;/p&gt;&#xA;&lt;p&gt;For example,&lt;/p&gt;&#xA;&lt;p&gt;this works&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;default = &amp;quot;Local&amp;quot;&#xA;  providers&#xA;  {&#xA;    Local&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And this fails&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;default = &amp;quot;MyLocal&amp;quot;&#xA;  providers&#xA;  {&#xA;    MyLocal&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Command to reproduce error&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;export _JAVA_OPTIONS=&amp;quot;--add-opens=java.base/sun.security.util=ALL-UNNAMED&amp;quot;&#xA;&#xA;java -Dconfig.file=cromwell_docker.conf \&#xA;-Dbackend.providers.MyLocal.config.dockerRoot=&lt;span class=&quot;math-container&quot;&gt;$(pwd)/cromwell-executions \&#xA;-Dbackend.providers.MyLocal.config.root=$&lt;/span&gt;(pwd)/cromwell-executions \&#xA;-jar ~/cromwell/cromwell-84.jar run fq_count.wdl -i fq_count.json&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Error:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java.lang.IllegalArgumentException: Could not build the path &amp;quot;https://portal.nersc.gov/cfs/m342/jaws/test_data/sample.fastq&amp;quot;. It may refer to a filesystem not supported by this instance of Cromwell. Supported filesystems are: MacOSXFileSystem. Failures: &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;WDL file&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;version 1.0&#xA;workflow fq_count {&#xA;    input {&#xA;        File fastq_file&#xA;    }&#xA;    call count_reads {&#xA;      input: fastq = fastq_file&#xA;    }&#xA;}&#xA;&#xA;task count_reads {&#xA;  input {&#xA;    File fastq&#xA;  }&#xA;&#xA;  command &amp;lt;&amp;lt;&amp;lt;&#xA;    ls ~{fastq}&#xA;  &amp;gt;&amp;gt;&amp;gt;&#xA;&#xA;  runtime {&#xA;    docker: &amp;quot;ubuntu:latest&amp;quot;&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;json&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;  &amp;quot;fq_count.fastq_file&amp;quot;: &amp;quot;https://portal.nersc.gov/cfs/m342/jaws/test_data/sample.fastq&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Cromwell conf&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;include required(classpath(&amp;quot;application&amp;quot;))&#xA;&#xA;webservice&#xA;{&#xA;  port = 50110&#xA;}&#xA;&#xA;workflow-options&#xA;{&#xA;  workflow-log-dir: &amp;quot;cromwell-workflow-logs&amp;quot;&#xA;  workflow-log-temporary: false&#xA;}&#xA;&#xA;call-caching&#xA;{&#xA;  enabled = true&#xA;  invalidate-bad-cache-result = true&#xA;}&#xA;&#xA;backend&#xA;{&#xA;  default = &amp;quot;MyLocal&amp;quot;&#xA;&#xA;  providers&#xA;  {&#xA;&#xA;    MyLocal&#xA;    {&#xA;      actor-factory = &amp;quot;cromwell.backend.impl.sfs.config.ConfigBackendLifecycleActorFactory&amp;quot;&#xA;&#xA;      config&#xA;      {&#xA;&#xA;        concurrent-job-limit = 5&#xA;        run-in-background = true&#xA;&#xA;        # The list of possible runtime custom attributes.&#xA;        runtime-attributes = &amp;quot;&amp;quot;&amp;quot;&#xA;          String? docker&#xA;          String? docker_user&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;        # Submit string when there is no &amp;quot;docker&amp;quot; runtime attribute.&#xA;        submit = &amp;quot;/usr/bin/env bash ${script}&amp;quot;&#xA;&#xA;        # Submit string when there is a &amp;quot;docker&amp;quot; runtime attribute.&#xA;        submit-docker = &amp;quot;docker run --volume=&lt;span class=&quot;math-container&quot;&gt;${cwd}:$&lt;/span&gt;{docker_cwd} &lt;span class=&quot;math-container&quot;&gt;${docker} /usr/bin/env bash $&lt;/span&gt;{script}&amp;quot;&#xA;&#xA;        root = &amp;quot;/cromwell-executions&amp;quot;&#xA;        dockerRoot = &amp;quot;/cromwell-executions&amp;quot;&#xA;&#xA;        filesystems&#xA;        {&#xA;          local&#xA;          {&#xA;            localization: [ &amp;quot;soft-link&amp;quot;, &amp;quot;hard-link&amp;quot;, &amp;quot;copy&amp;quot; ]&#xA;            http { enabled: true }&#xA;&#xA;            caching {&#xA;                duplication-strategy: [ &amp;quot;soft-link&amp;quot;, &amp;quot;hard-link&amp;quot;, &amp;quot;copy&amp;quot; ]&#xA;                hashing-strategy: &amp;quot;xxh64&amp;quot;&#xA;            }&#xA;          }&#xA;        }&#xA;&#xA;        default-runtime-attributes&#xA;        {&#xA;          failOnStderr: false&#xA;          continueOnReturnCode: 0&#xA;        }&#xA;      }&#xA;    }&#xA;  }&#xA;}&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16656" LastEditorUserId="16656" LastEditDate="2023-01-13T03:56:55.663" LastActivityDate="2023-01-13T03:56:55.663" Title="cromwell fails to download url input file" Tags="|wdl|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20310" PostTypeId="2" ParentId="20308" CreationDate="2023-01-04T09:58:49.233" Score="1" Body="&lt;p&gt;If the numbers are in quotes it means that it has read them as text. A numeric vector will be:&lt;/p&gt;&#xA;&lt;p&gt;[1] 2 0 1 1 1 1 1&lt;/p&gt;&#xA;&lt;p&gt;To replace that column with the new value you can use this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;het_df[,3] &amp;lt;- sapply(as.numeric(het_df[,3]), FUN=function(x){-1*(x-1)})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I used &lt;code&gt;het_df[, 3] &amp;lt;- &lt;/code&gt; to replace the content of the column.&#xA;I changed &lt;code&gt;lapply&lt;/code&gt; to &lt;code&gt;sapply&lt;/code&gt;, to get a vector and not a list.&#xA;I forced the input to be numeric via &lt;code&gt;as.numeric&lt;/code&gt;.&lt;br /&gt;&#xA;The syntax of the functions was wrong, to create a function in R you need &lt;code&gt;function(x){#code}&lt;/code&gt;. The linked question is for a function that has an argument called formula, which is not the case here, because you want to define your own (anonymous) function.&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2023-01-04T09:58:49.233" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20311" PostTypeId="1" CreationDate="2023-01-04T13:09:08.757" Score="2" ViewCount="447" Body="&lt;p&gt;I've been able to download specific data types from the TCGA portal (&lt;a href=&quot;https://portal.gdc.cancer.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://portal.gdc.cancer.gov/&lt;/a&gt;) using the Manifest files and the GDC Tool.&lt;/p&gt;&#xA;&lt;p&gt;But it's not clear to me how to download the metadata (e.g. cancer type, stage, etc.) with the links to the UUIDs of a specific data type, e.g. the UUID of a methylation data file to the case UUID:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://portal.gdc.cancer.gov/files/14dc6f1f-21d0-4b05-9ad5-1d52f5020b12&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://portal.gdc.cancer.gov/files/14dc6f1f-21d0-4b05-9ad5-1d52f5020b12&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="180" LastEditorUserId="73" LastEditDate="2023-01-05T20:06:54.470" LastActivityDate="2023-01-12T12:55:22.470" Title="how to download all metadata for a single TCGA dataset and link to data file UUIDs?" Tags="|public-databases|cancer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20312" PostTypeId="1" AcceptedAnswerId="20313" CreationDate="2023-01-04T16:35:46.433" Score="1" ViewCount="192" Body="&lt;p&gt;In single-cell RNA sequencing analysis, I am wondering how many percents explained variance by the first 50 principal components of PCA analysis is suitable for downstream &lt;code&gt;KNN&lt;/code&gt; and &lt;code&gt;umap&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;If there are lots of samples (&amp;gt;40) and cells (&amp;gt;100,000) to analyze together, there are a lot of &lt;code&gt;highly_variable_genes&lt;/code&gt; (&amp;gt;4000) can be found&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import scanpy as sc&#xA;sc.pp.highly_variable_genes(adata, flavor='cell_ranger', n_top_genes=None)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The explained variance by the first 50 principal components of PCA analysis &lt;code&gt;sc.pp.pca(adata, n_comps=50, use_highly_variable=True, svd_solver='arpack')&lt;/code&gt; is very low (&amp;lt;12%), increase the number of principal components does not help a lot.&#xA;I assume that the low explained variance of PCs will make &lt;code&gt;KNN&lt;/code&gt; and &lt;code&gt;umap&lt;/code&gt; difficult to separate the cells well. Am I correct?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sc.pp.neighbors(adata, n_neighbors=15)&#xA;sc.tl.umap(adata, min_dist=0.3, spread=2.0)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I can reduce the number of &lt;code&gt;highly_variable_genes&lt;/code&gt; to get more explained variance by the first 50 principal components, but &lt;code&gt;KNN&lt;/code&gt; and &lt;code&gt;umap&lt;/code&gt; still can not separate the cells well.&lt;/p&gt;&#xA;&lt;p&gt;I checked the first 5 PCs when I reduce the number of highly_variable_genes, and it doesn't look very high:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; np.cumsum(adata.uns['pca']['variance_ratio'])&#xA;&#xA;array([0.08419679, 0.12635055, 0.13610176, 0.14490107, 0.15326007])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any suggestions would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="9698" LastEditorUserId="73" LastEditDate="2023-01-05T20:10:56.440" LastActivityDate="2023-01-05T20:10:56.440" Title="How many percents explained variance by the first 50 principal components of PCA analysis is suitable for downstream `KNN` and `umap`?" Tags="|single-cell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20313" PostTypeId="2" ParentId="20312" CreationDate="2023-01-04T17:34:42.287" Score="1" Body="&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt; To answer the question, you're looking for 80% variance .... from the comments the actual command is &lt;code&gt;np.cumsum&lt;/code&gt; which changes things a lot.&lt;/p&gt;&#xA;&lt;p&gt;Previous response ...&#xA;Firstly it's still good work. It wouldn't be sensible to proceed with &lt;code&gt;KNN&lt;/code&gt; if 12% explained 50 PCs, now there are 15% in 5 PCs.&lt;/p&gt;&#xA;&lt;p&gt;You wrote,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;0.08419679, 0.12635055, 0.13610176, 0.14490107, 0.15326007&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;PC1 ~ 0.085, PC2 ~ 0.068&lt;/p&gt;&#xA;&lt;p&gt;I think that you've cross checked the analysis and that is a good sign of good analysis.&lt;/p&gt;&#xA;&lt;p&gt;The only easy possibility is to completely tighten down on all &amp;quot;variable genes&amp;quot; parameter. The total variance for the analysis think 50% total variance rather than an idealistic 80%.&lt;/p&gt;&#xA;&lt;p&gt;The emergency measures ...&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Simply use say 10 samples&lt;/li&gt;&#xA;&lt;li&gt;Reducing the total number of genes, e.g. removing more variable genes looks good, anything else&lt;/li&gt;&#xA;&lt;li&gt;Check the input internally to make sure there is no formatting bug&lt;/li&gt;&#xA;&lt;li&gt;Check via Seraut&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The thing I would cross check is reducing the sample size a lot ... is there a stray sample(s) here that is throwing everything out? That would be thing I'd really look into. What happens to the total variance when say 20 samples, or 10 samples are used?&lt;/p&gt;&#xA;&lt;p&gt;Just to finalise, are there any natural population divisions in the sample that could be used? If this results in a huge jump in variance that would be an answer. The only thing that is unusual is that there are a lot of samples for this type of study (from my perspective).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-04T23:20:13.580" LastActivityDate="2023-01-04T23:20:13.580" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20314" PostTypeId="1" CreationDate="2023-01-05T08:48:48.200" Score="2" ViewCount="71" Body="&lt;p&gt;I am a student in a Cancer lab. Working with sanger is new to me. While analyzing a report we found an insertion that has not been reported in any databases so far, we were working on checking if the insertion is pathogenic or not but when we reanalyzed the sanger annotation through variant reporter of the insertion it was giving it one position ahead and we analyzed that with the VEP as well that is were the confusion started.&lt;/p&gt;&#xA;&lt;p&gt;The sequencer showed annotation of an insertion not between the bases but a step/position next to it:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;c.5532-5533insGAA&lt;/strong&gt; is the insertion position came after position 32 and before 33 but the sequencer annotates it like &lt;strong&gt;c.5533-5534insGAA&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;We are working on human genome hg38.&lt;/p&gt;&#xA;&lt;p&gt;Should I consider it a machine error or should I take that position for the result? Or is there a nomenclature error I am making here?&lt;/p&gt;&#xA;&lt;p&gt;And&#xA;the annotation is in this form [c.33-34insAUU]+[=] and I don't know what the +[=] represents here.&lt;/p&gt;&#xA;" OwnerDisplayName="user16732" LastEditorDisplayName="user16732" LastEditDate="2023-01-06T13:34:50.790" LastActivityDate="2024-01-16T03:07:35.890" Title="Sanger sequencing annotation error" Tags="|ngs|sequence-annotation|assembly|mutations|sequence|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="20315" PostTypeId="2" ParentId="20274" CreationDate="2023-01-05T11:00:52.823" Score="1" Body="&lt;p&gt;Often multi-allelic sites would be split into several lines (one per alternate allele) to make filtering easier for specific types of alternate alleles (e.g. filter out MNV vs SNV).&lt;/p&gt;&#xA;&lt;p&gt;VCF-level, one thing you can do to identify the actual genotype over the locus easier is to join the split alleles into a single multi-allelic record. You can easily do this using bcftools:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view --samples HG00103 --regions chr7:12510038-12510038 1kGP_high_coverage_Illumina.chr7.filtered.SNV_INDEL_SV_phased_panel.vcf.gz | bcftools norm -m + | bcftools view -H&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From here is clear that starting on position chr7:12510038, instead of &amp;quot;GAA&amp;quot;, you have homozygous &amp;quot;GA&amp;quot; (1|1), therefore a hmozygous &amp;quot;A&amp;quot; deletion, which could be either the first or second &amp;quot;A&amp;quot; at positions 12510039 and 12510040.&lt;/p&gt;&#xA;" OwnerUserId="475" LastActivityDate="2023-01-05T11:00:52.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20317" PostTypeId="1" AcceptedAnswerId="20328" CreationDate="2023-01-05T12:40:35.370" Score="2" ViewCount="836" Body="&lt;p&gt;I have a Seurat object and plotted the Dimplot for UMAP visualization for 2 variables, as shown in the image below.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2gj8q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2gj8q.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, the problem is that I want the group by variables such as &lt;code&gt;Non-responder and Responder&lt;/code&gt; and &lt;code&gt;anti-CLTA4, anti-CLTA4+PD1, anti-PD1&lt;/code&gt; on the top of the UMAP plot and not on the right side. My desired output would look like the following.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/imshU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/imshU.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The timely help is highly appreciated, many thanks.&lt;/p&gt;&#xA;" OwnerUserId="16733" LastActivityDate="2023-01-06T10:55:17.423" Title="How to change the position of groups colors in DimPlot of Seurat" Tags="|scrnaseq|seurat|ggplot2|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20318" PostTypeId="1" AcceptedAnswerId="20320" CreationDate="2023-01-05T12:50:42.550" Score="2" ViewCount="256" Body="&lt;p&gt;Considering that the p-value of a chi statistic(df=1) is uniform, the expected genomic inflation factor is λ=1. I show its calculation in R with the next chunk of code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chi = qchisq(df$pval,df=1,lower.tail=FALSE)&#xA;factor = median(chi)/qchisq(p=0.5,df=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My question comes here. Imagine that I compute λ on several (e.g. &amp;gt;30 independent GWAS) studies. My null hypothesis is that all of them lack true-polygenic signals (i.e. uniform p-value distribution). What test can I do to reject H0? What is the distribution that I can assume for the inflation factors?&lt;/p&gt;&#xA;" OwnerUserId="12360" LastActivityDate="2023-01-05T16:14:10.617" Title="Genomic inflation factor distribution (lambda)" Tags="|statistics|gwas|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20319" PostTypeId="2" ParentId="20302" CreationDate="2023-01-05T15:26:05.150" Score="1" Body="&lt;p&gt;to me your 2 strings seem different, no N in first molecule.&lt;/p&gt;&#xA;&lt;p&gt;Here pictures, I used a PDB version of the original mmCIF file&lt;/p&gt;&#xA;&lt;p&gt;but dindt use the RCSB PDB available one, imported .cif into&lt;/p&gt;&#xA;&lt;p&gt;Biopython and parsed the resulting Struct object [so there could be&#xA;mistakes].&lt;/p&gt;&#xA;&lt;p&gt;so chain N --&amp;gt; A and chain U --&amp;gt; E&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wVXug.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wVXug.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Q1FrI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Q1FrI.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;as per in my comment:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Mmmhhh be sure that each of the 3 modeled ligands correspond to the full one, could be that the density is poor and pdb shows only what is really ((??) there and not what they soaked the crystal with, actually 1N7 is CHAPSO I believe a detergent&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;... I could be wrong though, this mmCIF format is driving me crazy&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2023-01-05T15:33:57.257" LastActivityDate="2023-01-05T15:33:57.257" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20320" PostTypeId="2" ParentId="20318" CreationDate="2023-01-05T16:14:10.617" Score="2" Body="&lt;p&gt;The distribution as I suspected seems very much like a normal distribution. However if someone knows to explain the reason (limit central theorem?) I will give select their answer.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    # Do simulation    &#xA;    factor_calc = function(pvals){&#xA;      chi = qchisq(pvals,df=1,lower.tail=FALSE)&#xA;      return(median(chi)/qchisq(p=0.5,df=1))}&#xA;    pvals_studies = sapply(1:10000, function(i){runif(10000)})&#xA;    factor_distribution = apply(pvals_studies, 2, function(pvals){factor_calc(pvals)})&#xA;&#xA;    # Plot resulting distribution&#xA;    hist(factor_distribution)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5PmPD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5PmPD.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12360" LastActivityDate="2023-01-05T16:14:10.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20322" PostTypeId="2" ParentId="20301" CreationDate="2023-01-05T18:04:11.050" Score="1" Body="&lt;p&gt;Ok, cannot try the API approach, no resources during vacation. Got the download and parse/split approach, added conversion to SMILES by &lt;strong&gt;Openbabel&lt;/strong&gt;. I believe it is exactly the same question of : &lt;a href=&quot;https://stackoverflow.com/questions/61390035/how-to-save-each-ligand-from-a-pdb-file-separately-with-bio-pdb&quot;&gt;https://stackoverflow.com/questions/61390035/how-to-save-each-ligand-from-a-pdb-file-separately-with-bio-pdb&lt;/a&gt; from where I stole the juicy bit of the code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## stealing from https://stackoverflow.com/questions/61390035/how-to-save-each-ligand-from-a-pdb-file-separately-with-bio-pdb&#xA;&#xA;from Bio import __version__&#xA;&#xA;&#xA;print('BIOPYTHON Version : ' , __version__)&#xA;&#xA;from Bio.PDB import MMCIFParser, PDBIO, Select, PDBList&#xA;&#xA;import subprocess&#xA;&#xA;filename = '7rdx.cif'&#xA;&#xA;&#xA;pdblist = PDBList()&#xA;&#xA;pdblist.retrieve_pdb_file(filename.upper().split('.')[0] , pdir = '.')&#xA;&#xA;&#xA;pars = MMCIFParser(QUIET = True)&#xA;&#xA;struct = pars.get_structure('7rdx',  filename)&#xA;&#xA;&#xA;print(struct, type(struct))&#xA;&#xA;&#xA;def is_het(residue):&#xA;    res = residue.id[0]&#xA;    &#xA;    if res != &amp;quot; &amp;quot; and res != &amp;quot;W&amp;quot;:&#xA;        &#xA;        return True #1&#xA;&#xA;    else:&#xA;    &#xA;        return False #0&#xA; &#xA;&#xA;class ResidueSelect(Select):&#xA;    def __init__(self, chain, residue):&#xA;        self.chain = chain&#xA;        self.residue = residue&#xA;&#xA;    def accept_chain(self, chain):&#xA;    &#xA;        if chain.id == self.chain.id:&#xA;            &#xA;            return True #1&#xA;        &#xA;        else:&#xA;            &#xA;            return False #0&#xA;        &#xA;        &#xA;    def accept_residue(self, residue):&#xA;        &amp;quot;&amp;quot;&amp;quot; Recognition of heteroatoms - Remove water molecules &amp;quot;&amp;quot;&amp;quot;&#xA;        &#xA;        if residue == self.residue and is_het(residue):&#xA;            &#xA;            return True #1&#xA;        &#xA;        else:&#xA;            &#xA;            return False #0&#xA;&#xA;&#xA;for model in struct :&#xA;    &#xA;    &#xA;    dizio = model.child_dict&#xA;    &#xA;    print('\n'+ str(dizio))&#xA;    &#xA;    &#xA;&#xA;for model in struct:&#xA;    &#xA;    i = 1&#xA;    &#xA;    print(model, model.id)&#xA;    &#xA;    for chain in model:&#xA;        &#xA;        print(chain, chain.id)&#xA;        &#xA;        for res in chain:&#xA;            &#xA;            # print('\n'+'res : ', res, 'res.resname : ', res.resname)&#xA;            &#xA;            if res.id[0] != &amp;quot; &amp;quot; :&#xA;                &#xA;                &#xA;                print('\n'+'res : ', res, res.id, res.resname)&#xA;                &#xA;                io = PDBIO()&#xA;                &#xA;                io.set_structure(struct)&#xA;                &#xA;                io.save(f&amp;quot;lig_{chain.id}_{res.resname}_{i}.pdb&amp;quot;, ResidueSelect(chain, res))&#xA;                &#xA;                try:&#xA;                    subprocess.run([&amp;quot;openbabel.obabel&amp;quot;, &amp;quot;-ipdb&amp;quot;, f&amp;quot;lig_{chain.id}_{res.resname}_{i}.pdb&amp;quot;, &amp;quot;-osmi&amp;quot; ,&amp;quot;-O&amp;quot;,&#xA;                                    &#xA;                                    f&amp;quot;lig_{chain.id}_{res.resname}_{i}.smi&amp;quot;])&#xA;                    &#xA;&#xA;                    &#xA;                except Exception as excp:&#xA;                    &#xA;                    print('\n ', excp, '\n')&#xA;                    &#xA;                    &#xA;                &#xA;                i += 1&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Problem is Biopython parses &lt;strong&gt;mmCIF&lt;/strong&gt;s files; that are now the official standard for RCSB PDB &lt;a href=&quot;https://www.rcsb.org/news/feature/5c767ef5ea7d0653b99c876f&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Submission of PDBx/mmCIF format files for crystallographic depositions to the PDB will be mandatory from July 1st 2019 onward&lt;/strong&gt;. PDB format files will no longer be accepted for deposition of structures solved by MX techniques.&lt;/a&gt; A nice review about the long history of the 'new' format :[ &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0022283622001796?via%3Dihub&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDBx/mmCIF Ecosystem: Foundational Semantic Tools for Structural Biology&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately the &lt;code&gt;Bio.PDB.MMCIFParser module&lt;/code&gt; return a a structure that is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;&amp;lt;Structure id=7rdx&amp;gt; &amp;lt;class 'Bio.PDB.Structure.Structure'&amp;gt;&lt;/code&gt; so all the ligands are related to PDB chains classification and not PDBx entity.&lt;/p&gt;&#xA;&lt;p&gt;I could be wrong here, I am a novice in both Biopython and Structural Bioinformatics but this is what I've noticed, please feel free to elaborate on this and add a solution using the RCSB APIes ... before curiosity kills the cat&lt;/p&gt;&#xA;&lt;p&gt;PS:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;NEEDs to be tested with sample containing water&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2023-01-05T18:26:24.320" LastActivityDate="2023-01-05T18:26:24.320" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20323" PostTypeId="2" ParentId="20286" CreationDate="2023-01-05T19:44:04.667" Score="2" Body="&lt;p&gt;&lt;em&gt;[note: This is a generic answer; the post was not specific enough for me to be able to include any more useful detail]&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;A heatmap of 235 rows and 570 columns is within the realm of what should be explorable / visible with a bit of tweaking. I have needed to do this with large heatmap plots - for example, single cell data with 500 genes and 2000 cells, where I care about the genes but not the cell IDs. My usual approach is to make the dimensions of the output image larger, and also reduce the text size.&lt;/p&gt;&#xA;&lt;p&gt;The method of doing that depends a lot on how the plot was generated initially. For example, if using a base R package:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pdf(&amp;quot;output_file_name.pdf&amp;quot;, width=57, height=23.5); # about 10 squares per inch&#xA;## &amp;lt;plot generation function&amp;gt;&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If using ggplot:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse);&#xA;input_data %&amp;gt;%&#xA;  ggplot() +&#xA;  aes(...) +&#xA;  # &amp;lt;plot generation function&amp;gt;&#xA;ggsave(&amp;quot;output_file_name.pdf&amp;quot;, width=57, height=23.5);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Changing the function used to display the heatmap can sometimes help as well (e.g. &lt;code&gt;pheatmap&lt;/code&gt; instead of &lt;code&gt;heatmap&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Within the functions, text size can often be reduced to help with this as well (e.g. &lt;code&gt;cex = 0.2&lt;/code&gt;). There is a limit to how much this can be done, as most PDF viewing programs have a zoom limit - I think Adobe Reader's limit is 6400% (i.e. 64 times magnification) - as well as maximum page dimension limits.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-01-05T19:49:36.570" LastActivityDate="2023-01-05T19:49:36.570" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20324" PostTypeId="2" ParentId="20308" CreationDate="2023-01-05T20:02:45.980" Score="0" Body="&lt;p&gt;Doing mathematical operations on a matrix will only work if &lt;em&gt;all&lt;/em&gt; the matrix elements are numerical. In your case, you have mixed character and numerical variables, and it looks like a data frame or tibble would be more appropriate (calling it &lt;code&gt;het_df&lt;/code&gt; is a strong indicator that this is actually what you want to do, as well as the &lt;code&gt;pandas&lt;/code&gt; structures in the python code). You can load a TSV file into a tibble in R using the &lt;code&gt;read_tsv&lt;/code&gt; function from &lt;code&gt;tidyverse&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse);&#xA;het_df &amp;lt;- read_tsv(&amp;quot;input_file_name.tsv&amp;quot;);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After this, numerical mathematical operations can be carried out column-by-column, because the columns are treated as separate structures internally:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;het_df$het &amp;lt;- -1 * (het_df$het - 1); # approach 1 [base R]&#xA;het_df &amp;lt;- mutate(het_df, het = -1 * (het - 1); # approach 2 [tidyverse]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-01-05T20:02:45.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20325" PostTypeId="1" AcceptedAnswerId="20326" CreationDate="2023-01-05T21:56:06.170" Score="3" ViewCount="216" Body="&lt;p&gt;I have 3 experiment files (&lt;code&gt;File1.tsv, File2.tsv, File3.tsv&lt;/code&gt;) from three experiments and 1 library file (&lt;code&gt;library_file.tsv&lt;/code&gt;). I want to merge the files by creating a python iterator because the number of experiment files will increase with every additional experiment and the library file will be always the same. All the experiment files are in the same directory.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;File 1&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;id  gene    Exp_1&#xA;ID000000001 EGFR    0.127&#xA;ID000000002 ADAD1   0.421&#xA;ID000000003 ADAD1   0.301&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;File2&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;id  gene    Exp2&#xA;ID000000001 EGFR    0.303&#xA;ID000000002 ADAD1   0.236&#xA;ID000000003 ADAD1   0.071&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;File3&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;id  gene    Exp_3&#xA;ID000000001 EGFR    0.847&#xA;ID000000002 ADAD1   0.992&#xA;ID000000003 ADAD1   0.912&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;library_file&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;id  value1  value2  &#xA;ID000000001 Single  Guides_Single&#xA;ID000000002 Single  Guides_Single&#xA;ID000000003 Single  Guides_Single&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;Expected output&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;id  gene    Exp_1   Exp2    Exp_3   value1  value2&#xA;ID000000001 EGFR    0.127   0.303   0.847   Single  Guides_Single&#xA;ID000000002 ADAD1   0.421   0.236   0.992   Single  Guides_Single&#xA;ID000000003 ADAD1   0.301   0.071   0.912   Single  Guides_Single&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4845" LastEditorUserId="3848" LastEditDate="2023-01-05T23:35:02.080" LastActivityDate="2023-01-05T23:52:47.563" Title="Merge the dataframes with python iterator" Tags="|python|pandas|merge|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20326" PostTypeId="2" ParentId="20325" CreationDate="2023-01-05T23:25:42.590" Score="2" Body="&lt;p&gt;Its a simple join operation.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;df1 = pd.DataFrame({'id':['ID000000001','ID000000002','ID000000003'], 'gene1':['EGFR', 'ADAD1','ADAD1'],'score1':[0.127, 0.421, 0.301]}).set_index('id')&#xA;df2 = pd.DataFrame({'id':['ID000000001','ID000000002','ID000000003'], 'gene2':['EGFR', 'ADAD1','ADAD1'], 'score2':[0.303, 0.236, 0.071]}).set_index('id')&#xA;df3 = pd.DataFrame({'id':['ID000000001','ID000000002','ID000000003'], 'gene3':['EGFR', 'ADAD1','ADAD1'], 'score3':[0.847,0.992,0.912]}).set_index('id')&#xA;df4 = pd.DataFrame({'id':['ID000000001','ID000000002','ID000000003'], 'v1':['Single', 'Single', 'Single'], 'v2':['Guide', 'Guide', 'Guide']}).set_index('id')&#xA;&#xA;df = df1.join([df2,df3,df4])&#xA;df.drop(['gene2', 'gene3'],axis=1,inplace=True)&#xA;print (df)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;             gene1  score1  score2  score3      v1     v2&#xA;id                                                       &#xA;ID000000001   EGFR   0.127   0.303   0.847  Single  Guide&#xA;ID000000002  ADAD1   0.421   0.236   0.992  Single  Guide&#xA;ID000000003  ADAD1   0.301   0.071   0.912  Single  Guide&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;'Iterator' just &lt;code&gt;pickle&lt;/code&gt; via&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;path='/home/desktop/df.pkl'&#xA;df.to_pickle(path)&#xA;# Reload&#xA;df = pd.read_pickle(path)&#xA;# Add another data set&#xA;df = df.join(newdf)&#xA;df.drop(['gene4'],axis=1,inplace=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... then pickle it again&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-05T23:52:47.563" LastActivityDate="2023-01-05T23:52:47.563" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20327" PostTypeId="2" ParentId="20314" CreationDate="2023-01-06T07:46:54.153" Score="0" Body="&lt;p&gt;Just to give my thoughts:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The missing information is the actual sequence&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is this a conflict in the change in position of the insert?&lt;/li&gt;&#xA;&lt;li&gt;Could it simply be a consequence of down stream numbering, either a logical error (coding stuff), or lab artefact.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My bet is its point 2, but really we don't know except for the OP because we don't know the sequence.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Sources of error, Point 2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Sanger sequencing was traditionally known for &amp;quot;frame-shift&amp;quot; errors if there were mono-mers, e.g. AAAA -&amp;gt; AAAAA, it can mess up over the number of bases in a mono-mer.&lt;/li&gt;&#xA;&lt;li&gt;PCR amplification can also cause slippage against monomers, same thing as the above point - but it's real.&lt;/li&gt;&#xA;&lt;li&gt;1 vs 0 ... this is because in a Pythonic list or any C/Perl-based array numbering starts at 0 (not 1) - hence @terdon's comments.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If upstream mono-mers are present that might be the source of the confusion here and given the amount of sequence data thats very likely.&lt;/p&gt;&#xA;&lt;p&gt;What I'd do:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Just Blast the new sequence and see what emerges: if it is genuinely novel - they'll be zero 100% identity hits.&lt;/li&gt;&#xA;&lt;li&gt;VCFs ... I think this is VarSome's stuff (forget).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If there are Blast hits it will identify whether there is any upstream lab artefacts.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&#xA;If this was any other situation in human genetics, or even mammalian genetics I'd be excited. However, in this case - I think its a routine result. Firstly, the chance of finding a new &amp;quot;driver&amp;quot; mutation are slim. Secondly, I've looked at this chromosome in a cancer patient (only once) and it was nuts (and thats an under statement) - I've a fairly good memory for numerical patterns, and I'd never seen a pathogenic species behaving like that. I think this insert is just a consequence of the entire mutational chaos that are cancer cells.&lt;/p&gt;&#xA;&lt;p&gt;Just to make it clear: I don't do cancer but if you want an authoritative assessment and think you've something major I'd contact (or your PI) the Broad Institute via Geraldine A. Van der Auwera who also has an account on this site via @Geraldine_VdAuwera. Sanger sequencing is not what they do, but they'd know at the drop of a hat what the significance and if they don't respond - thats your answer ;-). Alternatively, VarSome would know, albeit they're commercial enterprise offering freeware.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-06T07:56:16.997" LastActivityDate="2023-01-06T07:56:16.997" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20328" PostTypeId="2" ParentId="20317" CreationDate="2023-01-06T10:55:17.423" Score="3" Body="&lt;p&gt;DimPlot returns a ggplot object, so ggplot functions can be applied to it. You can change the legend position like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;DimPlot(data) + theme(legend.position = &amp;quot;top&amp;quot;)&#xA;# or&#xA;DimPlot(data) + theme(legend.position = c(.1, .9))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15489" LastActivityDate="2023-01-06T10:55:17.423" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20329" PostTypeId="2" ParentId="20274" CreationDate="2023-01-06T20:00:12.420" Score="2" Body="&lt;p&gt;this is Marta from the NYGC. Thanks for reaching out with your questions.&lt;/p&gt;&#xA;&lt;p&gt;The full SNV/INDEL callset that we released (listed as &amp;quot;SNV/INDEL VCFs&amp;quot; in the key resource table in &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0092867422009916?via%3Dihub#sec4&quot; rel=&quot;nofollow noreferrer&quot;&gt;our paper&lt;/a&gt;) includes multiallelic representation of all multiallelic sites (i.e. multiple ALT alleles are listed together in column 5 separated by commas) and the record you showed as an example above is listed there in the following way:&lt;/p&gt;&#xA;&lt;p&gt;chr7    12510038    .   GAAA    GAA,GA,G&lt;/p&gt;&#xA;&lt;p&gt;The VCF file you were looking at is the phased panel that we generated and released alongside the full unphased callset. The phasing software tools that we used in the paper (SHAPEIT2, SHAPEIT4, Eagle2) as well as most other phasing tools do not support multiallelic sites and require that such sites are split into separate lines and represented as biallelic for the purposes of phasing. As you noted, this is not ideal and can lead to difficulties in interpretation (as others explained above) but it's the best we can do with the existing approaches. We mention this in the Methods section of the paper as well as in the &lt;a href=&quot;http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000G_2504_high_coverage/working/20220422_3202_phased_SNV_INDEL_SV/README_1kGP_phased_panel_110722.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;README&lt;/a&gt; that's included with the phased panel on the IGSR ftp (all linked in the paper). Please let us know if you have any further questions related to the high-coverage 1kGP resources. Thanks!&lt;/p&gt;&#xA;" OwnerUserId="16739" LastActivityDate="2023-01-06T20:00:12.420" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20330" PostTypeId="2" ParentId="20301" CreationDate="2023-01-07T00:30:39.627" Score="1" Body="&lt;p&gt;adding a different way to solve the question , still without using RCSB PDB database, but using &lt;code&gt;Biopython.PDB.MMCIF2Dict&lt;/code&gt; module to start to get a feeling on how info are stored in .cif format files. Ligands are selected amongs the ones in the structure for their Mw.&lt;/p&gt;&#xA;&lt;p&gt;Ligands are then saved using the same approach of first answer or a new one that takes advantage of &lt;code&gt;Biopython.PDB.Dice&lt;/code&gt; module to extract and save portion of PDB. Again Openbabel is used to translate PDBs to SMILES.&lt;/p&gt;&#xA;&lt;p&gt;Code :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;# -*- coding: utf-8 -*-&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;Created on Thu Jan  5 11:43:07 2023&#xA;&#xA;@author: bob&#xA;&#xA;https://bioinformatics.stackexchange.com/questions/20301/how-to-download-ligands-for-pdb-structure?noredirect=1#comment29972_20301&#xA;&#xA;&#xA;&#xA;&#xA;NEEDs to be tested with sample containing water&#xA;&#xA;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;## stealing from https://stackoverflow.com/questions/61390035/how-to-save-each-ligand-from-a-pdb-file-separately-with-bio-pdb&#xA;&#xA;from Bio import __version__&#xA;&#xA;&#xA;print('BIOPYTHON Version : ' , __version__)&#xA;&#xA;from Bio.PDB import (&#xA;                    MMCIFParser, &#xA;                    PDBIO, &#xA;                    Select, &#xA;                    PDBList, &#xA;                    MMCIF2Dict,&#xA;                    Dice&#xA;                    )&#xA;&#xA;import subprocess&#xA;&#xA;filename = '7rdx.cif'&#xA;&#xA;# filename = '7RDX.cif'  # this will  work , but biopython saves in lower case &#xA;&#xA;&#xA;# pdblist = PDBList()&#xA;&#xA;# pdblist.retrieve_pdb_file(filename.split('.')[0] , pdir = '.')&#xA;&#xA;&#xA;pars = MMCIFParser(QUIET = True)&#xA;&#xA;struct = pars.get_structure('7rdx',  filename)&#xA;&#xA;&#xA;# print(struct, type(struct))&#xA;&#xA;&#xA;mmcifdict = MMCIF2Dict.MMCIF2Dict(filename)&#xA;&#xA;&#xA;# print(mmcifdict)&#xA;&#xA;# for i in mmcifdict:&#xA;    &#xA;#     print('\n', i ,'   ---&amp;gt; ', mmcifdict[i])&#xA;    &#xA;print('##############################################################')&#xA;    &#xA;# print(mmcifdict['_entity'])&#xA;&#xA; &#xA;print(mmcifdict['_pdbx_nonpoly_scheme.entity_id'])&#xA;print(mmcifdict['_pdbx_nonpoly_scheme.asym_id'])&#xA;print(mmcifdict['_pdbx_nonpoly_scheme.entity_id'])&#xA;print(mmcifdict['_pdbx_nonpoly_scheme.mon_id'])&#xA;&#xA;ligands_map_mmcif = list(zip(mmcifdict['_pdbx_nonpoly_scheme.entity_id'],&#xA;                             &#xA;                             mmcifdict['_pdbx_nonpoly_scheme.asym_id'],&#xA;                             &#xA;                             mmcifdict['_pdbx_nonpoly_scheme.entity_id'],&#xA;                             &#xA;                             mmcifdict['_pdbx_nonpoly_scheme.mon_id']))&#xA;&#xA;&#xA;print('#########################################')&#xA;&#xA;print(ligands_map_mmcif)&#xA;&#xA;print(mmcifdict['_pdbx_nonpoly_scheme.entity_id'])&#xA;print(mmcifdict['_pdbx_nonpoly_scheme.pdb_seq_num'])&#xA;print(mmcifdict['_pdbx_nonpoly_scheme.pdb_strand_id'])&#xA;print(mmcifdict['_pdbx_nonpoly_scheme.pdb_mon_id'])&#xA;&#xA;ligands_map_pdb = list(zip(mmcifdict['_pdbx_nonpoly_scheme.entity_id'],&#xA;                           &#xA;                           mmcifdict['_pdbx_nonpoly_scheme.pdb_seq_num'],&#xA;                           &#xA;                           mmcifdict['_pdbx_nonpoly_scheme.pdb_strand_id'],&#xA;                           &#xA;                           mmcifdict['_pdbx_nonpoly_scheme.pdb_mon_id']))&#xA;&#xA;print('#########################################')&#xA;&#xA;print(ligands_map_pdb)&#xA;&#xA;&#xA;print('WWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWWw')&#xA;&#xA;print(mmcifdict['_entity.id'])&#xA;print(mmcifdict['_entity.formula_weight'])&#xA;&#xA;entity_Mw = dict(zip(mmcifdict['_entity.id'],&#xA;                     &#xA;                     mmcifdict['_entity.formula_weight']))&#xA;&#xA;print(entity_Mw)&#xA;&#xA;print('-------------------------------------------------------------')&#xA;&#xA;&#xA;ligands_map_pdb_red = [i for i in ligands_map_pdb if float(entity_Mw[i[0]]) &amp;gt;= 200 ]&#xA;&#xA;print(ligands_map_pdb_red)&#xA;&#xA;&#xA;&#xA;&#xA;&#xA;#########################################################   first approach &#xA;&#xA;&#xA;# class ResidueSelect(Select):&#xA;#     def __init__(self, chain, residue):&#xA;#         self.chain = chain&#xA;#         self.residue = residue&#xA;&#xA;#     def accept_chain(self, chain):&#xA;    &#xA;#       if chain.id == self.chain.id:&#xA;            &#xA;#           return True #1&#xA;        &#xA;#       else:&#xA;            &#xA;#           return False #0&#xA;        &#xA;        &#xA;#     def accept_residue(self, residue):&#xA;#         &amp;quot;&amp;quot;&amp;quot; Recognition of heteroatoms - Remove water molecules &amp;quot;&amp;quot;&amp;quot;&#xA;        &#xA;#         if residue == self.residue :&#xA;            &#xA;            &#xA;#             return True #1&#xA;        &#xA;#         else:&#xA;            &#xA;#             return False #0&#xA;&#xA;&#xA;&#xA;&#xA;# for model in struct:&#xA;    &#xA;#     print(model)&#xA;    &#xA;#     for chain in model:&#xA;        &#xA;#         if chain.id == 'A':&#xA;            &#xA;#             for res in chain:&#xA;                &#xA;#                 print(res, res.id, res.get_resname(), type(res))&#xA;&#xA;# i = 1    &#xA;&#xA;# for res in ligands_map_pdb_red :&#xA;    &#xA;#     print('\n'+'res : ', res[3], res[0], res[1], type(res[1]), res[2])&#xA;    &#xA;#     print(struct[0][res[2]] , type(struct[0][res[2]]) )&#xA;    &#xA;#     print(struct[0][res[2]][('H_'+res[3], int(res[1]), ' ')] , type(struct[0][res[2]][('H_'+res[3], int(res[1]), ' ')]) )&#xA;    &#xA;                &#xA;#     io = PDBIO()&#xA;                    &#xA;#     io.set_structure(struct)&#xA;                    &#xA;#     io.save(f&amp;quot;lig_{res[3]}_{res[2]}_{str(i).zfill(5)}.pdb&amp;quot;, ResidueSelect(struct[0][res[2]],&#xA;                                                                          &#xA;#                                                     struct[0][res[2]][('H_'+res[3], int(res[1]), ' ')]))&#xA;                    &#xA;#     try:&#xA;#         subprocess.run([&amp;quot;openbabel.obabel&amp;quot;, &amp;quot;-ipdb&amp;quot;, f&amp;quot;lig_{res[3]}_{res[2]}_{str(i).zfill(5)}.pdb&amp;quot;, &amp;quot;-osmi&amp;quot; ,&amp;quot;-O&amp;quot;,&#xA;                                        &#xA;#         f&amp;quot;lig_{res[3]}_{res[2]}_{str(i).zfill(5)}.smi&amp;quot;])&#xA;                        &#xA;    &#xA;                        &#xA;#     except Exception as excp:&#xA;                        &#xA;#         print('\n ', excp, '\n')&#xA;                        &#xA;                        &#xA;                    &#xA;#     i += 1&#xA;    &#xA;########################################################   second approach &#xA;&#xA;&#xA;import warnings&#xA;&#xA;from Bio import BiopythonWarning&#xA;&#xA;&#xA;def accept_residue_mod(self, residue):&#xA;        &amp;quot;&amp;quot;&amp;quot;Verify if a residue sequence is between the start and end sequence.&amp;quot;&amp;quot;&amp;quot;&#xA;        # residue - between start and end&#xA;        hetatm_flag, resseq, icode = residue.get_id()&#xA;        # if hetatm_flag != &amp;quot; &amp;quot;:&#xA;        #     # skip HETATMS&#xA;        #     return 0&#xA;        if icode != &amp;quot; &amp;quot;:&#xA;            warnings.warn(&#xA;                f&amp;quot;WARNING: Icode {icode} at position {resseq}&amp;quot;, BiopythonWarning&#xA;            )&#xA;        if self.start &amp;lt;= resseq &amp;lt;= self.end:&#xA;            return 1&#xA;        return 0&#xA;&#xA;Dice.ChainSelector.accept_residue = accept_residue_mod&#xA;&#xA;&#xA;i = 1&#xA;&#xA;for res in ligands_map_pdb_red :&#xA;    &#xA;    print(struct, res[2], int(res[1]), int(res[1]))&#xA;&#xA;    Dice.extract(struct, res[2], int(res[1]), int(res[1]), f&amp;quot;lig_{res[3]}_{res[2]}_{str(i).zfill(5)}.pdb&amp;quot;)&#xA;    &#xA;    &#xA;    try:&#xA;        subprocess.run([&amp;quot;openbabel.obabel&amp;quot;, &amp;quot;-ipdb&amp;quot;, f&amp;quot;lig_{res[3]}_{res[2]}_{str(i).zfill(5)}.pdb&amp;quot;, &amp;quot;-osmi&amp;quot; ,&amp;quot;-O&amp;quot;,&#xA;                                           &#xA;        f&amp;quot;lig_{res[3]}_{res[2]}_{str(i).zfill(5)}.smi&amp;quot;])&#xA;                            &#xA;        &#xA;    except Exception as excp:&#xA;                            &#xA;        print('\n ', excp, '\n')&#xA;                            &#xA;                            &#xA;                        &#xA;    i += 1&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2023-01-08T15:20:45.277" LastActivityDate="2023-01-08T15:20:45.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20331" PostTypeId="2" ParentId="19188" CreationDate="2023-01-07T00:51:58.393" Score="2" Body="&lt;p&gt;I haven't checked how fast this is but you could try using &lt;a href=&quot;https://hail.is&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hail&lt;/a&gt; (disclaimer: I'm a maintainer). You'll need to install the GCS connector first to read from Google Cloud Storage:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;curl https://broad.io/install-gcs-connector | python3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import hail as hl&#xA;&#xA;mt = hl.import_vcf('your.vcf')&#xA;mt = hl.split_multi_hts(mt)  # use the same biallelic representation that gnomAD uses&#xA;mt.write('data.mt')  # store in Hail's native format for faster queries&#xA;mt = hl.read_matrix_table('data.mt')&#xA;sites = hl.read_table(&#xA;    'gs://gcp-public-data--gnomad/release/3.1.2/ht/genomes/gnomad.genomes.v3.1.2.sites.ht'&#xA;)&#xA;mt = mt.anti_join_rows(sites)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That'll filter down to just those variants (defined by chromosome, position, and alleles) which are not present in gnomAD v3.&lt;/p&gt;&#xA;" OwnerUserId="1087" LastActivityDate="2023-01-07T00:51:58.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20332" PostTypeId="2" ParentId="10731" CreationDate="2023-01-07T01:01:10.520" Score="1" Body="&lt;p&gt;You might try &lt;code&gt;ldscsim&lt;/code&gt; which was developed by Nik Baya for this purpose.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://hail.is/docs/0.2/experimental/ldscsim.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hail.is/docs/0.2/experimental/ldscsim.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import hail as hl&#xA;&#xA;mt = hl.import_vcf('path/to/your/data.vcf')&#xA;mt.write('data.mt') # store in Hail's native format for speed&#xA;mt = hl.read_matrix_table('data.mt')&#xA;mt = hl.experimental.ldscsim(&#xA;    mt, mt.GT, h2=0.5&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;ldscsim&lt;/code&gt; can generate phenotypes under a number of different models.&lt;/p&gt;&#xA;&lt;p&gt;For the GWAS tutorial, AFAIK, we just simulated betas for each site, then computed the phenotype from that. If you ask on &lt;a href=&quot;https://discuss.hail.is&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://discuss.hail.is&lt;/a&gt; and tag Tim Poterba he can probably give you the precise answer.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def logistic(x):&#xA;    return 1 / (1 + hl.exp(-x))&#xA;&#xA;mt = mt.annotate_rows(beta = hl.rand_norm(0, 1))&#xA;mt = mt.annotate_cols(&#xA;    is_case = 0.5 &amp;lt; logistic(hl.agg.sum(beta * mt.GT.n_alt_alleles()))&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1087" LastActivityDate="2023-01-07T01:01:10.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20333" PostTypeId="2" ParentId="20301" CreationDate="2023-01-08T13:05:33.160" Score="1" Body="&lt;p&gt;OK, seems ligands and PDBs are not really a trending topic here.... (or my posts are really poor).&lt;/p&gt;&#xA;&lt;p&gt;Nevertheless the question is literally eating my brain so I went into a second direction: &amp;quot;the API&amp;quot; already skimmed here &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/18076/help-me-in-understanding-the-pdb-file&quot;&gt;Help me in understanding the PDB file&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;So inspecting and copying directly from the PDB entry &lt;a href=&quot;https://www.rcsb.org/structure/7RDX&quot; rel=&quot;nofollow noreferrer&quot;&gt;7RDX&lt;/a&gt; webpage the download value: &lt;a href=&quot;https://models.rcsb.org/v1/7rdx/ligand?auth_seq_id=1005&amp;amp;label_asym_id=M&amp;amp;encoding=sdf&amp;amp;filename=7rdx_M_1N7.sdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://models.rcsb.org/v1/7rdx/ligand?auth_seq_id=1005&amp;amp;label_asym_id=M&amp;amp;encoding=sdf&amp;amp;filename=7rdx_M_1N7.sdf&lt;/a&gt;I got this bit of code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;&#xA;&#xA;def main():   &#xA;    risposta = requests.get(f&amp;quot;https://models.rcsb.org/v1/7rdx/ligand?auth_seq_id=1005&amp;amp;label_asym_id=M&amp;amp;encoding=sdf&amp;quot;,                      &#xA;                      allow_redirects=True)   &#xA;    open(filename, 'wb').write(risposta.content)&#xA;    print(risposta.status_code)&#xA;&#xA;if __name__ == '__main__':  &#xA;   filename = '7rdx_M_1N7.sdf'   &#xA;   main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That is equivalent to feeding this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;  &amp;quot;atom_site&amp;quot;: [&#xA;    {&#xA;      &amp;quot;auth_seq_id&amp;quot;: 1005,&#xA;      &amp;quot;label_asym_id&amp;quot;: &amp;quot;M&amp;quot;&#xA;    }&#xA;  ]&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to the 'https://models.rcsb.org/#/General/ligand-post' test webpage Request body.&lt;/p&gt;&#xA;&lt;p&gt;now need to figure out how to get the &lt;code&gt;auth_seq_id&lt;/code&gt; and the &lt;code&gt;label_asym_id &lt;/code&gt; parameters for all the&#xA;'non_polymer_entity_ids' of my targed PDB 7RDX from the &lt;a href=&quot;https://data.rcsb.org/rest/v1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://data.rcsb.org/rest/v1/&lt;/a&gt; DATA API.&lt;/p&gt;&#xA;&lt;p&gt;Went till here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;import json&#xA;&#xA;def main(entry_id: str):&#xA;    risposta = requests.get(f&amp;quot;https://data.rcsb.org/rest/v1/core/entry/{entry_id}/&amp;quot;)   &#xA;    parsed = json.loads(risposta.content.decode())    &#xA;    print(risposta.status_code)    &#xA;    print((parsed))    &#xA;    print(parsed[&amp;quot;rcsb_entry_container_identifiers&amp;quot;][&amp;quot;non_polymer_entity_ids&amp;quot;])    &#xA;    print('\n', type(parsed))&#xA;    &#xA;    parsed_lig = parsed[&amp;quot;rcsb_entry_container_identifiers&amp;quot;][&amp;quot;non_polymer_entity_ids&amp;quot;]    &#xA;    print('\n\n',parsed_lig, type(parsed_lig))    &#xA;    &#xA;&#xA;if __name__ == '__main__':         &#xA;   main('7rdx')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; ['7', '8', '9', '10', '11'] &amp;lt;class 'list'&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The list values are the ids of the 'non_polymer_entity_ids' of my targed PDB 7RDX.&lt;/p&gt;&#xA;&lt;p&gt;Right now I am trying to recover the the &lt;code&gt;auth_seq_id&lt;/code&gt; and the &lt;code&gt;label_asym_id &lt;/code&gt; for every instance of the 'non_polymer_entity_ids' inside my mmCIF/PDBx using the DATA API.&lt;/p&gt;&#xA;&lt;p&gt;Not sure this is the fastest and right way using the RCSB PDB API to accomplish the OP question (download all ligands of the model structure (not the ligands models)) from the PDB database. Please feel free to add more or point me towards the best solution.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDITED:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;It took a while, I am sure this is not the right way to make good use of all the databases, there should be a way to navigate the API schemas better and faster, and to get and manage the intermediate values needed to get the two keys for the download from ModelServer &amp;quot;Post /v1/{id}/ligand Coordinates of the first group satisfying the given criteria&amp;quot;, anyway:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;import json&#xA;&#xA;parsed_lig_dict = {}&#xA;&#xA;def main(entry_id: str):    &#xA;    risposta = requests.get(f&amp;quot;https://data.rcsb.org/rest/v1/core/entry/{entry_id}/&amp;quot;)   &#xA;    parsed = json.loads(risposta.content.decode())    &#xA;    # print(risposta.status_code)    &#xA;    # print((parsed))    &#xA;    # print(parsed[&amp;quot;rcsb_entry_container_identifiers&amp;quot;][&amp;quot;non_polymer_entity_ids&amp;quot;])&#xA;    # print('\n', type(parsed))&#xA;    &#xA;    parsed_lig = parsed[&amp;quot;rcsb_entry_container_identifiers&amp;quot;][&amp;quot;non_polymer_entity_ids&amp;quot;]    &#xA;    # print('\n\n',parsed_lig, type(parsed_lig))&#xA;    &#xA;    # for i in parsed:        &#xA;    #     print('\n', i  , parsed[i])&#xA;                  &#xA;    for lig in parsed_lig:    &#xA;        risposta = requests.get(f&amp;quot;https://data.rcsb.org/rest/v1/core/nonpolymer_entity/{entry_id}/{lig}&amp;quot;)    &#xA;        parsed = json.loads(risposta.content.decode())    &#xA;        # print('\n',risposta.status_code, risposta.content)        &#xA;        &#xA;        # print(parsed[&amp;quot;rcsb_nonpolymer_entity_container_identifiers&amp;quot;][&amp;quot;asym_ids&amp;quot;], &#xA;              # parsed[&amp;quot;pdbx_entity_nonpoly&amp;quot;][&amp;quot;comp_id&amp;quot;])&#xA;                &#xA;        parsed_lig_dict[lig] = [parsed[&amp;quot;pdbx_entity_nonpoly&amp;quot;][&amp;quot;comp_id&amp;quot;] , &#xA;                                dict.fromkeys(parsed[&amp;quot;rcsb_nonpolymer_entity_container_identifiers&amp;quot;][&amp;quot;asym_ids&amp;quot;])]&#xA;    &#xA;    for lig in parsed_lig_dict:        &#xA;        for chain in parsed_lig_dict[lig][1]:    &#xA;            risposta = requests.get(f&amp;quot;https://data.rcsb.org/rest/v1/core/nonpolymer_entity_instance/{entry_id}/{chain}&amp;quot;)        &#xA;            # print(risposta.status_code)&#xA;        &#xA;            parsed = json.loads(risposta.content.decode())        &#xA;            # print('\n',risposta.status_code, risposta.content)            &#xA;            # print(chain , '--&amp;gt; ',parsed[&amp;quot;rcsb_nonpolymer_entity_instance_container_identifiers&amp;quot;][&amp;quot;auth_seq_id&amp;quot;])&#xA;            &#xA;            parsed_lig_dict[lig][1][chain] = parsed[&amp;quot;rcsb_nonpolymer_entity_instance_container_identifiers&amp;quot;][&amp;quot;auth_seq_id&amp;quot;]        &#xA;               &#xA;    &#xA;    # print(parsed_lig_dict)&#xA;    &#xA;    # for i in parsed_lig_dict:&#xA;    #     print(parsed_lig_dict[i])&#xA;    &#xA;    # for i in parsed_lig_dict:&#xA;    #     for y in parsed_lig_dict[i][1]:&#xA;    #         print(i,parsed_lig_dict[i][0], y , parsed_lig_dict[i][1][y])&#xA;                        &#xA;    cnt = 1     &#xA;    for i in parsed_lig_dict:        &#xA;        for y in parsed_lig_dict[i][1]:&#xA;            seq_id = parsed_lig_dict[i][1][y]            &#xA;            chain = y            &#xA;            comp_id = parsed_lig_dict[i][0]            &#xA;            print(seq_id , y)&#xA;&#xA;            risposta = requests.get(f&amp;quot;https://models.rcsb.org/v1/7rdx/ligand?auth_seq_id={seq_id}&amp;amp;label_asym_id={chain}&amp;amp;encoding=sdf&amp;quot; ,                              &#xA;                              allow_redirects=True)             &#xA;            print(risposta.status_code)&#xA;                        &#xA;            filename = f&amp;quot;lig_{comp_id}_{chain}_{seq_id}_{cnt}.sdf&amp;quot;            &#xA;            open(filename, 'wb').write(risposta.content)            &#xA;            cnt += 1&#xA;&#xA;    &#xA;if __name__ == '__main__':         &#xA;   main('7rdx')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This approach is very slow on my machine, the other two answers that parse the downloaded full mmCIF file via Biopython Structure Object are faster even adding the translation &lt;code&gt;.sdf&lt;/code&gt; files to SMILES via Openbabel. I think this is because I translate the DATA API JSON formatted schema into a dictionary to extract the data I need, there should be another way to make use of that data. This way doesn't need installed Biopython though.&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="3446" LastEditDate="2023-03-01T14:26:12.993" LastActivityDate="2023-03-01T14:26:12.993" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20334" PostTypeId="1" CreationDate="2023-01-09T06:08:47.143" Score="2" ViewCount="165" Body="&lt;p&gt;I am using Mauve app to align two whole genomes. I've aligned these genomes (gbk files) with the progressive Mauve aligner, and I got sp.sslist, sp.xmfa, sp.guide_tree,sp.backbone file, sp.bbcols etc file. I'm trying to generate a circos plot from MAUVE output. However, The output file format is different from the format required for CIRCOS. Here is the&#xA;output of sp.backbone:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seq0_leftend    seq0_rightend   seq1_leftend    seq1_rightend&#xA;-14446737   -14446957   89204   89424&#xA;-14446680   -14446734   89445   89499&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but the CIRCOS input format is different,  as shown here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr - hs1 1 0 249250621 chr1&#xA;chr - hs2 2 0 243199373 chr2&#xA;chr - hs3 3 0 198022430 chr3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to draw synteny in a similar way as this example figure, which containg contigs of two chromosomes. Kindly help; I am new to python and all. A python script would be a great help for me. Thanks&#xA;&lt;a href=&quot;https://i.stack.imgur.com/yqBdY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yqBdY.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16745" LastEditorUserId="73" LastEditDate="2023-01-09T19:34:59.823" LastActivityDate="2023-01-09T19:34:59.823" Title="Drawing synteny using CIRCOS from progressive MAUVE alignment" Tags="|python|circos|java|mauve|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20335" PostTypeId="1" CreationDate="2023-01-09T06:12:15.937" Score="2" ViewCount="104" Body="&lt;p&gt;I am reading this paper called Birth and death in the Terminal Complement pathway, doi : &lt;a href=&quot;https://doi.org/10.1101/2022.04.21.489004&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1101/2022.04.21.489004&lt;/a&gt; where they have performed CLANS analysis.&#xA;I understand that CLANS performs blast and visually represents the results as clusters.&lt;/p&gt;&#xA;&lt;p&gt;What does these clusters mean and is there meaning to the distance between them?&lt;/p&gt;&#xA;&lt;p&gt;And in the perspective of the paper, what does the CLANS analysis tell the researchers?&lt;/p&gt;&#xA;&lt;p&gt;Thank you, I am still a student, sorry if this is a simple question. And I have attached a screenshot from the supplementary materials of the article&lt;a href=&quot;https://i.stack.imgur.com/LiqWJ.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LiqWJ.jpg&quot; alt=&quot;&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="9089" LastEditorUserId="9089" LastEditDate="2023-01-10T05:48:43.617" LastActivityDate="2023-01-12T12:51:52.453" Title="What do you use the CLANS program for?" Tags="|phylogeny|blast|clustering|sequence-homology|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20336" PostTypeId="1" AcceptedAnswerId="20340" CreationDate="2023-01-09T12:48:28.390" Score="2" ViewCount="203" Body="&lt;p&gt;I have what at first sight appears to be a high-quality MAG (~10 pieces, high completion%) that I built from a hybrid assembly (Illumina + Nanopore data) from a cyanobacterium.&lt;/p&gt;&#xA;&lt;p&gt;Workflow:&#xA;Quality control (BBDuk) &amp;gt; Hybrid assembly (Unicycler) &amp;gt; co-binning multiple different samples (vamb) &amp;gt; Quality control (CheckM, anvi'o)&lt;/p&gt;&#xA;&lt;p&gt;*The Nanopore data is pretty old, and therefore I'd thought Unicycler might be a safer bet rather than long-read-first assemblers, due to the higher error rate&lt;/p&gt;&#xA;&lt;p&gt;The same strain also has 1-2 very fragmented genomes (100s of pieces) available in NCBI at scaffold level. The reads themselves are not available, so I can't just add them to the assembly.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to use the available NCBI genome(s) to try to check/polish mine? (I've checked out &lt;a href=&quot;https://bioinformatics.stackexchange.com/q/97/4401&quot;&gt;Improve a reference genome with sequencing data&lt;/a&gt; but it's already a few years old so I imagine different tools might be applicable)&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your time!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastEditorUserId="4401" LastEditDate="2023-01-10T12:19:24.930" LastActivityDate="2023-01-16T13:03:14.997" Title="Improving prokaryotic assembly with other contig/scaffold-level data?" Tags="|genome|assembly|nanopore|illumina|" AnswerCount="2" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="20337" PostTypeId="1" AcceptedAnswerId="20339" CreationDate="2023-01-09T19:38:17.830" Score="1" ViewCount="40" Body="&lt;p&gt;I used &lt;a href=&quot;https://github.com/rlabduke/reduce&quot; rel=&quot;nofollow noreferrer&quot;&gt;a tool called reduce&lt;/a&gt; to add Hydrogen bonds information to the PDB file.&lt;/p&gt;&#xA;&lt;p&gt;I used the command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ ./reduce -NOFLIP 2GB1.pdb &amp;gt; 2GB1_with_H.pdb&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output file is supplied as follows:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://textdoc.co/ZiuA0tY2elnoh5VK&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://textdoc.co/ZiuA0tY2elnoh5VK&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I haven't understood the output.&lt;/p&gt;&#xA;&lt;p&gt;What special change was made by the tool to the &lt;code&gt;2GB1.pdb&lt;/code&gt; file?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastActivityDate="2023-01-10T12:45:55.980" Title="What is changed in this PDB file?" Tags="|proteins|pdb|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20338" PostTypeId="2" ParentId="20303" CreationDate="2023-01-10T09:42:16.683" Score="3" Body="&lt;p&gt;On a regular basis, using FastQC is quite enough as to assess the quality of FastQ-formatted data. It gives you plenty of details as can be seen in the screenshots you shared.&lt;/p&gt;&#xA;&lt;p&gt;As others have commented  or suggested, there is  nothing wrong with having more 'yellow' warnings after processing the FASTQ files using fastp. At lest not as long as what you observe in the data looks fine and the warning has some explanation.&lt;/p&gt;&#xA;&lt;p&gt;The sequence length distribution warning is very often present post-processing since at the begging all your reads are of the same length but this is not the case any more after you trim the adapters and the low quality bases at the end of the read for example. You have to go to the plots specific to that section and make sure all looks fine. It should be the case.&lt;/p&gt;&#xA;&lt;p&gt;The GC sequence content warning. It depends a lot what organism you are sequencing, which regions of the genomes, etc. Importantly, what is the expected GC content of those regions. So you can compare to these observed numbers. Once again the warning can be easily triggered after the processing, since for a start Illumina sequencing has known GC bias&lt;/p&gt;&#xA;&lt;p&gt;Finally, I think actually your reads look better  after the processing with fastp. You are showing us the per base sequence quality and before, there are many reads with bases on the yellow and even the red sections of the quality scale. But this is not the case any more after the processing !&lt;/p&gt;&#xA;" OwnerUserId="475" LastEditorUserId="475" LastEditDate="2023-01-11T09:48:54.027" LastActivityDate="2023-01-11T09:48:54.027" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="20339" PostTypeId="2" ParentId="20337" CreationDate="2023-01-10T12:45:55.980" Score="5" Body="&lt;p&gt;So the 'right' answer here would be for you to use a &lt;code&gt;diff&lt;/code&gt; tool of some kind (WinDiff, or just the diff command in a Unix-like environment). That would show you the exact added lines. (Although, trying this myself does not produce very clear results...)&lt;/p&gt;&#xA;&lt;p&gt;However, I know from running &lt;code&gt;reduce&lt;/code&gt; on PDB files that it's added a few things:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The &amp;quot;USER MOD&amp;quot; lines, there's one on the first line and a large block just before the ATOM records&lt;/li&gt;&#xA;&lt;li&gt;The hydrogen atoms, of course, needed for correct H-bond calculations&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;As some commenters have pointed out, this should really be in the paper or other documentation for the tool.&lt;/p&gt;&#xA;" OwnerUserId="745" LastActivityDate="2023-01-10T12:45:55.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20340" PostTypeId="2" ParentId="20336" CreationDate="2023-01-10T18:19:19.223" Score="2" Body="&lt;p&gt;If your goal is to compare to the NCBI version I'd suggest using &lt;a href=&quot;https://github.com/ablab/quast&quot; rel=&quot;nofollow noreferrer&quot;&gt;QUAST&lt;/a&gt; to compare the two assemblies, supplying the NCBI assembly as the reference genome (&lt;code&gt;-r&lt;/code&gt; option I think). This will not guarantee anything, but you would expect the two assemblies to be fairly syntenic and to have good overlap of sequences. The tool includes visualization, but you could also use an approach like &lt;a href=&quot;https://academic.oup.com/bioinformatics/article-abstract/26/20/2620/194655&quot; rel=&quot;nofollow noreferrer&quot;&gt;circoletto&lt;/a&gt; or &lt;a href=&quot;http://blog.dnanexus.com/2018-01-11-dot-an-interactive-dot-plot-viewer-for-comparative-genomics#&quot; rel=&quot;nofollow noreferrer&quot;&gt;dot plots&lt;/a&gt; for something a little more intuitive/simpler.&lt;/p&gt;&#xA;&lt;p&gt;I'd expect your genome to be quite a bit higher quality even as a metagenome-derived assembly due to nanopore contiguity, I'm not sure that using the NCBI reference to improve yours is a great idea. I'd instead suggest following @M__'s suggestion to scaffold with your nanopore reads (you will have to map them onto your full metagenome first, though approach may differ based on what the scaffolder wants). I'm not up on the current methods here but &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-021-04451-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; is a recent tool. If you have 10 pieces I'd expect to be able to close at least some of that using nanopore (even old).&lt;/p&gt;&#xA;&lt;p&gt;If you do choose to use the reference to improve, the tool that people seem to be using now is &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-022-02823-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;RagTag&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-01-10T18:19:19.223" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20341" PostTypeId="1" CreationDate="2023-01-10T23:11:34.423" Score="3" ViewCount="287" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://stackoverflow.com/questions/75076433/run-nextflow-pipeline-multiple-time-depending-on-length-of-a-list&quot;&gt;Stack Overflow&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have a Nextflow pipeline that runs a BUSCO search on input data, returns the output and then creates a report based on the output. One of my parameters is the lineage dataset. I would like to have my pipeline run multiple times (preferably at the same time) based on how long my list of lineage datasets is. For example, this is my nextflow.config file. The goal is that my pipeline would run twice in this case. If there are n lineages specified I would like it to run n times.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params { &#xA;    inputFilePath           = &amp;quot;$PWD/input_data/test_data.fasta&amp;quot;&#xA;    lineage_datasets        = [&#xA;        &amp;quot;fungi_odb10&amp;quot;,&#xA;        &amp;quot;microsporidia_odb10&amp;quot;&#xA;        ]&#xA;    augustus_species        = &amp;quot;fumigatus&amp;quot;&#xA;    mode                    = &amp;quot;geno&amp;quot;&#xA;    &#xA;    threads                 = 16&#xA;    help                    = false&#xA;    outdir {&#xA;        main                = &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$PWD/results&quot;&#xA;    intermediate        = &quot;$&lt;/span&gt;PWD/busco_outputs&amp;quot;&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The first part of my busco process looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  publishDir params.outdir.intermediate, mode: 'copy'&#xA;&#xA;  input:&#xA;    path inputFile&#xA;    string lineage_dataset&#xA;  &#xA;  output:&#xA;    path 'hap1/*hap1.txt'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The idea is that each of the summary.txt files will be collected inside of my outdir.intermediate folder which is where my createReport process will target. For context, this is my createReport process:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process createReport {&#xA;&#xA;  publishDir params.outdir.main, mode: 'copy'&#xA;&#xA;  input: &#xA;    path 'busco_outputs/*'&#xA;&#xA;  output:&#xA;    path 'report.html'&#xA;&#xA;  script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    source &amp;quot;$launchDir/venv/bin/activate&amp;quot;&#xA;    report.py &amp;gt; report.html&#xA;    &amp;quot;&amp;quot;&amp;quot; &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have tried the following in my workflow process but it does not seem to work:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow {&#xA;  runBuscoChannel = Channel.fromPath(params.inputFilePath).cross(Channel.of(params.lineage_datasets)) &#xA;  | mix &#xA;  | collect &#xA;  | set { outputs }&#xA;&#xA;  createReport(outputs)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I try to run the pipeline it does not run properly&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have any ideas of how I could change my file to allow for this? Thanks very much!&lt;/p&gt;&#xA;" OwnerUserId="16749" LastEditorUserId="73" LastEditDate="2023-01-12T18:22:16.273" LastActivityDate="2023-03-07T09:32:53.053" Title="Run nextflow pipeline multiple times depending on length of a list" Tags="|bash|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20342" PostTypeId="1" AcceptedAnswerId="20347" CreationDate="2023-01-10T20:15:30.540" Score="3" ViewCount="558" Body="&lt;p&gt;I would be highly appreciated to have a script able to perform the following operation:&lt;/p&gt;&#xA;&lt;p&gt;Only extract scaffold_1 lines from file1.vcf in which POS number (column 2) falls within column 2 and column 3 numbers of scaffold_1 lines from listfile.txt (inclusive). Similarly, only extract scaffold_2 lines from file1.vcf in which POS number falls within column 2 and column 3 numbers of scaffold_2 lines from listfile.txt. In reality the vcf I have continues until scaffold_12 and has 17,000 lines.&lt;/p&gt;&#xA;&lt;p&gt;file1.vcf&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM      POS ID  REF ALT QUAL    FILTER  INFO    FORMAT          COUNTS&#xA;scaffold_1  160 .   C   A   40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_1  210 .   AC  A   40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_1  340 .   A   C   40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_1  470 .   T   C   40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_1  580 .   A   AC  40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_1  630 .   CA  C   40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_2  40  .   C   A   40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_2  100 .   AC  A   40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_2  370 .   A   C   40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_2  600 .   T   C   40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_2  800 .   A   AC  40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_2      850     .       C       CTA     40      PASS    .       GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_2  1120    .   CA  C   40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;listfile.txt&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;scaffold_1  50  100&#xA;scaffold_1  200 350&#xA;scaffold_2  350 450&#xA;scaffold_2  750 850&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The desired output file is the following:&lt;/p&gt;&#xA;&lt;p&gt;desired_file.vcf&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM      POS ID  REF ALT QUAL    FILTER  INFO    FORMAT          COUNTS&#xA;scaffold_1  210 .   AC  A   40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_1  340 .   A   C   40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_2  370 .   A   C   40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_2  800 .   A   AC  40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_2      850     .       C       CTA     40      PASS    .       GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To do this job I found the below awk code from another similar post but they don't differentiate scaffold_1 from scaffold_2, and I don't how to add this extra variable properly. Thank you.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'NR==FNR{rng++;start[rng]=$2;end[rng]=$3;next} {for (i=1;i&amp;lt;=rng;i++) if (($2&amp;gt;=start[i])&amp;amp;&amp;amp;($2&amp;lt;=end[i])) {print; next}}' listfile.txt file1.vcf &amp;gt; output.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;listfile.txt awk '{ print &amp;quot;$2 &amp;gt;= &amp;quot; $2 &amp;quot; &amp;amp;&amp;amp; $2 &amp;lt;= &amp;quot; $3 }' | awk -f - file1.vcf &amp;gt; output.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried to modify the last awk code like this, but it didn't work:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;listfile.txt awk '{ print &amp;quot;$1 == &amp;quot; $1 &amp;quot; &amp;amp;&amp;amp; $2 &amp;gt;= &amp;quot; $2 &amp;quot; &amp;amp;&amp;amp; $2 &amp;lt;= &amp;quot; $3 }' | awk -f - file1.vcf &amp;gt; output.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16757" OwnerDisplayName="rseg" LastEditorUserId="3694" LastEditDate="2023-01-11T11:53:58.303" LastActivityDate="2023-01-11T15:02:26.347" Title="Extract lines from a file based on a range of coordinates listed on a second file" Tags="|vcf|awk|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20343" PostTypeId="2" ParentId="20342" CreationDate="2023-01-11T07:08:40.970" Score="7" Body="&lt;p&gt;If you have a VCF file with an intact header, you can use &lt;a href=&quot;/questions/tagged/bcftools&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;bcftools&amp;#39;&quot; aria-label=&quot;show questions tagged &amp;#39;bcftools&amp;#39;&quot; rel=&quot;tag&quot; aria-labelledby=&quot;bcftools-container&quot;&gt;bcftools&lt;/a&gt; for this. All you need to do first is to bgzip and index your VCF:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ bgzip file.vcf&#xA;$ bcftools index file.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then, assuming your list of regions are actually 1-based tab-delimited, use &lt;a href=&quot;https://samtools.github.io/bcftools/bcftools.html#view&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;bcftools view&lt;/code&gt;&lt;/a&gt; and the &lt;code&gt;--regions-file&lt;/code&gt; option to restrict the search to the regions listed. Use also the &lt;code&gt;--no-header&lt;/code&gt; option to suppress the header in the output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view -HR regions.txt file.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From the &lt;a href=&quot;https://samtools.github.io/bcftools/bcftools.html#common_options&quot; rel=&quot;noreferrer&quot;&gt;common options&lt;/a&gt; documentation:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;-R, --regions-file FILE&lt;/p&gt;&#xA;&lt;p&gt;Regions can be specified either on command line or in a VCF, BED, or tab-delimited file (the default). The&#xA;columns of the tab-delimited file can contain either positions&#xA;(two-column format) or intervals (three-column format): CHROM, POS,&#xA;and, optionally, END, where positions are 1-based and inclusive. The&#xA;columns of the tab-delimited BED file are also CHROM, POS and END&#xA;(trailing columns are ignored), but coordinates are 0-based,&#xA;half-open. To indicate that a file be treated as BED rather than the&#xA;1-based tab-delimited file, the file must have the &amp;quot;.bed&amp;quot; or &amp;quot;.bed.gz&amp;quot;&#xA;suffix (case-insensitive). Uncompressed files are stored in memory,&#xA;while bgzip-compressed and tabix-indexed region files are streamed.&#xA;Note that sequence names must match exactly, &amp;quot;chr20&amp;quot; is not the same&#xA;as &amp;quot;20&amp;quot;. Also note that chromosome ordering in FILE will be respected,&#xA;the VCF will be processed in the order in which chromosomes first&#xA;appear in FILE. However, within chromosomes, the VCF will always be&#xA;processed in ascending genomic coordinate order no matter what order&#xA;they appear in FILE. Note that overlapping regions in FILE can result&#xA;in duplicated out of order positions in the output. This option&#xA;requires indexed VCF/BCF files. Note that -R cannot be used in&#xA;combination with -r.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-01-11T07:08:40.970" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20345" PostTypeId="1" AcceptedAnswerId="20348" CreationDate="2023-01-11T13:09:42.577" Score="0" ViewCount="55" Body="&lt;p&gt;I have some problems with ClinCNV:&lt;/p&gt;&#xA;&lt;p&gt;My input: &lt;code&gt;Rscript /root/ClinCNV/clinCNV.R --normal /root/samples/final.cov --out /root/samples/ -bed /root/samples/fileGC.bed --folderWithScript /root/ClinCNV/&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;And there is an output: &lt;code&gt;/root/ClinCNV/clinCNV.R: error: Error in getopt(spec = spec, opt = args) : short flag &amp;quot;b&amp;quot; is invalid&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I've already ask a question about ClinCNV and the main idea was to run script with absolute pathways.&lt;/p&gt;&#xA;" OwnerUserId="16725" LastActivityDate="2023-01-11T15:48:59.120" Title="Problem with ClinCNV script" Tags="|bed|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20347" PostTypeId="2" ParentId="20342" CreationDate="2023-01-11T14:54:08.437" Score="4" Body="&lt;p&gt;Using GNU awk for arrays of arrays:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ cat tst.awk&#xA;NR==FNR {&#xA;    ranges[$1][$2 FS $3]&#xA;    next&#xA;}&#xA;FNR == 1 {&#xA;    print&#xA;    next&#xA;}&#xA;$&lt;/span&gt;1 in ranges {&#xA;    for ( range in ranges[&lt;span class=&quot;math-container&quot;&gt;$1] ) {&#xA;        split(range,r)&#xA;        if ( (r[1] &amp;lt;= $2) &amp;amp;&amp;amp; ($&lt;/span&gt;2 &amp;lt;= r[2]) ) {&#xA;            print&#xA;            next&#xA;        }&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ awk -f tst.awk listfile.txt file1.vcf&#xA;#CHROM      POS ID  REF ALT QUAL    FILTER  INFO    FORMAT          COUNTS&#xA;scaffold_1  210 .   AC  A   40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_1  340 .   A   C   40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_2  370 .   A   C   40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_2  800 .   A   AC  40  PASS    .   GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;scaffold_2      850     .       C       CTA     40      PASS    .       GT:DP:AD:RO:QR:AO:QA    0:1:1,2:1:40:2:40&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Using any awk it'd be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ cat tst.awk&#xA;NR==FNR {&#xA;    ranges[$1] = ranges[$1] $2 FS $3 RS&#xA;    next&#xA;}&#xA;FNR == 1 {&#xA;    print&#xA;    next&#xA;}&#xA;$&lt;/span&gt;1 in ranges {&#xA;    n = split(ranges[&lt;span class=&quot;math-container&quot;&gt;$1],list,RS)&#xA;    for ( i=1; i&amp;lt;n; i++ ) {&#xA;        range = list[i]&#xA;        split(range,r)&#xA;        if ( (r[1] &amp;lt;= $2) &amp;amp;&amp;amp; ($&lt;/span&gt;2 &amp;lt;= r[2]) ) {&#xA;            print&#xA;            next&#xA;        }&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16755" LastEditorUserId="16755" LastEditDate="2023-01-11T15:02:26.347" LastActivityDate="2023-01-11T15:02:26.347" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20348" PostTypeId="2" ParentId="20345" CreationDate="2023-01-11T15:48:59.120" Score="3" Body="&lt;p&gt;You are using &lt;code&gt;-bed&lt;/code&gt;. Multi-letter flags are usually set using &lt;code&gt;--&lt;/code&gt; and not &lt;code&gt;-&lt;/code&gt;. I don't know this particular program but since all other long options use &lt;code&gt;--&lt;/code&gt;, and the only option with a &lt;code&gt;b&lt;/code&gt; is your &lt;code&gt;-bed&lt;/code&gt;, I suspect you just want &lt;code&gt;--bed&lt;/code&gt; instead:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Rscript /root/ClinCNV/clinCNV.R --normal /root/samples/final.cov \&#xA;   --out /root/samples/ \&#xA;   --bed /root/samples/fileGC.bed \&#xA;   --folderWithScript /root/ClinCNV/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;This is confirmed by the tool's &lt;a href=&quot;https://github.com/imgag/ClinCNV&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;, actually:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;--bed - .bed file with coordinates of on-target regions, GC-annotated. Gene anotations in the last column can be used in output.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So yes, you need &lt;code&gt;--bed&lt;/code&gt; and not &lt;code&gt;-bed&lt;/code&gt;. Because you used &lt;code&gt;-&lt;/code&gt;, the tool reads that as a single character flag (&lt;code&gt;-b&lt;/code&gt;) with the value &lt;code&gt;ed&lt;/code&gt; and doesn't know what to do with it.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;On an unrelated note, you seem to be running these things as root which is a very, very bad idea. One bug in the program or one misconfiguration and you can mess up your entire system. I strongly urge you to never run commands as root unless absolutely necessary.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-01-11T15:48:59.120" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20349" PostTypeId="2" ParentId="20030" CreationDate="2023-01-11T19:49:40.370" Score="0" Body="&lt;p&gt;I have recently worked on a couple of &lt;a href=&quot;https://github.com/freeseek/score&quot; rel=&quot;nofollow noreferrer&quot;&gt;tools&lt;/a&gt; to do exactly this. They are +munge to convert summary statistics into files following the &lt;a href=&quot;https://github.com/MRCIEU/gwas-vcf-specification&quot; rel=&quot;nofollow noreferrer&quot;&gt;GWAS-VCF specification&lt;/a&gt; and +liftover to convert a VCF from one genome build to another. This will automatically take care of REF⇆ALT switches by flipping the effect size and it will correctly handle indels, if there are any, in your summary statistics file&lt;/p&gt;&#xA;" OwnerUserId="9030" LastActivityDate="2023-01-11T19:49:40.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20350" PostTypeId="2" ParentId="14937" CreationDate="2023-01-11T19:57:23.703" Score="2" Body="&lt;p&gt;I would advise against using dbSNP rsIDs to perform a liftover as they get updated over time. I recently wrote a software to liftover summary statistics and the &lt;a href=&quot;https://github.com/freeseek/score&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; includes instructions that show you how to easily generate educational attainment summary statistics for GRCh38&lt;/p&gt;&#xA;" OwnerUserId="9030" LastActivityDate="2023-01-11T19:57:23.703" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20353" PostTypeId="2" ParentId="20335" CreationDate="2023-01-12T12:51:52.453" Score="1" Body="&lt;p&gt;CLANS (Clustering Locus-specific ANnotationS) is a bioinformatics tool that uses BLAST to compare a set of nucleotide or protein sequences to a reference database and visualizes the results as clusters, where each cluster represents a group of similar sequences, the distance between the clusters represents the degree of similarity between the sequences in each cluster. In the context of the paper you are reading, the CLANS analysis likely helped the researchers to identify functional domains within the Terminal Complement pathway, and the clusters formed by the CLANS analysis could represent different functional domains or sub-systems within the pathway, and the distance between the clusters could indicate the degree of similarity or dissimilarity between these domains.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2023-01-12T12:51:52.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20354" PostTypeId="2" ParentId="20311" CreationDate="2023-01-12T12:55:22.470" Score="2" Body="&lt;p&gt;To download metadata associated with a specific data file, such as the cancer type and stage, you can use the GDC API (Application Programming Interface). The API allows you to query the GDC database and retrieve information in a structured format, such as JSON or XML.&lt;/p&gt;&#xA;&lt;p&gt;To download the metadata for a specific data file, you will need the file's unique identifier, also known as its UUID. In your example, the UUID of the methylation data file is &amp;quot;14dc6f1f-21d0-4b05-9ad5-1d52f5020b12&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;To use the GDC API, you can construct a query using the GDC API endpoint and the file's UUID. For example, to retrieve the metadata for the methylation data file with the UUID &amp;quot;14dc6f1f-21d0-4b05-9ad5-1d52f5020b12&amp;quot;, you could use the following query:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://api.gdc.cancer.gov/files/14dc6f1f-21d0-4b05-9ad5-1d52f5020b12?expand=metadata&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://api.gdc.cancer.gov/files/14dc6f1f-21d0-4b05-9ad5-1d52f5020b12?expand=metadata&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This query uses the &amp;quot;files&amp;quot; endpoint of the GDC API and specifies the file's UUID. The &amp;quot;expand=metadata&amp;quot; parameter tells the API to return the metadata associated with the file. The response will be in json format:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{&amp;quot;data&amp;quot;: {&amp;quot;data_format&amp;quot;: &amp;quot;TXT&amp;quot;, &amp;quot;access&amp;quot;: &amp;quot;open&amp;quot;, &amp;quot;file_name&amp;quot;: &amp;quot;5b6353ee-af39-4bb4-a801-1454dfbc338b.methylation_array.sesame.level3betas.txt&amp;quot;, &amp;quot;submitter_id&amp;quot;: &amp;quot;d67719e6-4149-4492-a265-1f5d5689b904&amp;quot;, &amp;quot;data_category&amp;quot;: &amp;quot;DNA Methylation&amp;quot;, &amp;quot;acl&amp;quot;: [&amp;quot;open&amp;quot;], &amp;quot;type&amp;quot;: &amp;quot;methylation_beta_value&amp;quot;, &amp;quot;platform&amp;quot;: &amp;quot;Illumina Human Methylation 450&amp;quot;, &amp;quot;file_size&amp;quot;: 13168876, &amp;quot;created_datetime&amp;quot;: &amp;quot;2022-02-15T17:17:23.357435-06:00&amp;quot;, &amp;quot;md5sum&amp;quot;: &amp;quot;ab0c90db97a3e12634ef3f54f3ec835d&amp;quot;, &amp;quot;updated_datetime&amp;quot;: &amp;quot;2022-02-16T19:13:44.087132-06:00&amp;quot;, &amp;quot;file_id&amp;quot;: &amp;quot;14dc6f1f-21d0-4b05-9ad5-1d52f5020b12&amp;quot;, &amp;quot;data_type&amp;quot;: &amp;quot;Methylation Beta Value&amp;quot;, &amp;quot;state&amp;quot;: &amp;quot;released&amp;quot;, &amp;quot;experimental_strategy&amp;quot;: &amp;quot;Methylation Array&amp;quot;, &amp;quot;version&amp;quot;: &amp;quot;1&amp;quot;, &amp;quot;data_release&amp;quot;: &amp;quot;32.0 - 36.0&amp;quot;}, &amp;quot;warnings&amp;quot;: {&amp;quot;expand&amp;quot;: &amp;quot;unrecognized values: [metadata]&amp;quot;}}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3061" LastActivityDate="2023-01-12T12:55:22.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20355" PostTypeId="2" ParentId="18441" CreationDate="2023-01-12T12:57:22.293" Score="2" Body="&lt;p&gt;running the query builder at RCS PDB against id with space group (so crystal info) I get:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/S3rGv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/S3rGv.png&quot; alt=&quot;172456 hits&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;more about RCSB PDB web services here: &lt;a href=&quot;https://www.rcsb.org/docs/programmatic-access/web-services-overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/docs/programmatic-access/web-services-overview&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and the search API here: &lt;a href=&quot;https://search.rcsb.org/index.html#search-api&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://search.rcsb.org/index.html#search-api&lt;/a&gt;  think you should query for entries where &lt;code&gt;symmetry.pdbx_full_space_group_name_H_M&lt;/code&gt; exists (&lt;a href=&quot;https://search.rcsb.org/structure-search-attributes.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://search.rcsb.org/structure-search-attributes.html&lt;/a&gt; and ask to have returned entry.id (a list of in this case)&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2023-01-12T13:03:11.937" LastActivityDate="2023-01-12T13:03:11.937" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20357" PostTypeId="1" CreationDate="2023-01-12T21:01:34.327" Score="2" ViewCount="24" Body="&lt;p&gt;I can download information from ChEBI if I know the corresponding ChEBI ID:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from libchebipy import ChebiEntity&#xA;chebi_entity = ChebiEntity(&amp;quot;15903&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And then look at all the synonyms that the compound has:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chebi_entity.get_names()&#xA;&amp;gt; [{'_Name__name': 'beta-D-Glucose', '_Name__typ': 'SYNONYM', '_Name__source': 'KEGG COMPOUND', '_Name__adapted': False, '_Name__language': 'en'},&#xA; {'_Name__name': 'beta-D-glucose', '_Name__typ': 'NAME', '_Name__source': 'UniProt', '_Name__adapted': False, '_Name__language': 'en'},&#xA; {'_Name__name': 'beta-D-glucopyranose', '_Name__typ': 'IUPAC NAME', '_Name__source': 'IUPAC', '_Name__adapted': False, '_Name__language': 'en'},&#xA; {'_Name__name': 'BETA-D-GLUCOSE', '_Name__typ': 'SYNONYM', '_Name__source': 'PDBeChem', '_Name__adapted': False, '_Name__language': 'en'},&#xA; {'_Name__name': 'WURCS=2.0/1,1,0/[a2122h-1b_1-5]/1/', '_Name__typ': 'SYNONYM', '_Name__source': 'GlyTouCan', '_Name__adapted': False, '_Name__language': 'en'}]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there a way to get the ChEBI ID with one of  the synonyms?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;beta-D-glucose -&amp;gt; 15903&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there any other python tool or web-api that can achieve this?&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2023-01-12T22:00:39.213" LastActivityDate="2023-01-14T12:56:40.460" Title="Search ChebiEntity search by common or synonym name?" Tags="|embl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20358" PostTypeId="2" ParentId="20357" CreationDate="2023-01-13T00:50:58.920" Score="1" Body="&lt;p&gt;How very interesting ...&lt;/p&gt;&#xA;&lt;p&gt;Yes its easy just download the flat file database here:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ftp.ebi.ac.uk/pub/databases/chebi/Flat_file_tab_delimited/names.tsv.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ebi.ac.uk/pub/databases/chebi/Flat_file_tab_delimited/names.tsv.gz&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;WARNING&lt;/strong&gt;: it's a 5.5M file ... don't click the link unless you really want to download a file this size.&lt;/p&gt;&#xA;&lt;p&gt;You can simply load it straight into a &lt;code&gt;pandas&lt;/code&gt; dataframe from &lt;code&gt;read_csv()&lt;/code&gt; and then interrogation it via &lt;code&gt;loc&lt;/code&gt;. You could place the header &lt;code&gt;NAME&lt;/code&gt; as the index. There are other way in, you can shift the dataframe to a dictionary and interrogate that way, or place it straight it SQL via e.g. &lt;code&gt;sqlite3&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here I just loaded it onto Excel and found the entry you were looking for ...&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nmGKa.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nmGKa.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;....&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ogolh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ogolh.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-14T12:56:40.460" LastActivityDate="2023-01-14T12:56:40.460" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20359" PostTypeId="1" AcceptedAnswerId="20363" CreationDate="2023-01-13T06:46:32.250" Score="1" ViewCount="57" Body="&lt;p&gt;I used prefetch to get the Pacbio reads of chicken from the SRA database. I want to align these reads against a reference genome, but not all the reads. I am only interested in a particular region on Chromosome Z. Is there a way that I can subset the downloaded SRA file?&lt;/p&gt;&#xA;&lt;p&gt;I am trying to use blast, but don't know where to start. How should I prepare the SRA file from prefetch for the blast? I have currently started fasterq-dump for the downloaded SRA file.&lt;/p&gt;&#xA;&lt;p&gt;Or should I subset after aligning the entire SRA reads data with the reference genome, and in that case what tools do I use?&lt;/p&gt;&#xA;" OwnerUserId="9089" LastEditorUserId="3446" LastEditDate="2023-03-01T14:25:18.907" LastActivityDate="2023-03-01T14:25:18.907" Title="How to subset an SRA file for a single chromosome?" Tags="|assembly|reference-genome|pacbio|sratoolkit|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20361" PostTypeId="1" CreationDate="2023-01-13T07:59:23.990" Score="1" ViewCount="710" Body="&lt;p&gt;I have 5609 chromosome positions with both ref and alt alleles. I want to retrieve rsIDs for the same. I tried &lt;code&gt;get_rsid_from_position()&lt;/code&gt; of the &lt;code&gt;MRutils&lt;/code&gt; package in R, but after some time while fetching data from the internet, the cache memory of my system got full, and it froze the system.&#xA;Any other way to resolve this issue?&lt;/p&gt;&#xA;&lt;p&gt;I have 200 GB allotted for the cache memory.&lt;/p&gt;&#xA;" OwnerUserId="16764" LastEditorUserId="3694" LastEditDate="2023-01-13T13:33:17.230" LastActivityDate="2023-01-13T13:33:17.230" Title="Retrieve rsIDs from chromosome positions using MRutils::get_rsid_from_position" Tags="|r|snp|rsid|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20362" PostTypeId="2" ParentId="20361" CreationDate="2023-01-13T10:06:49.087" Score="3" Body="&lt;p&gt;An alternative possibility in R is using the library biomaRt from Bioconductor. Here an example starting from a table with 3 columns: Chr, Start, End. Organism is human and positions are assumed to be based on the reference version GRCh38.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(biomaRt)&#xA;snpMart = useEnsembl(biomart = &amp;quot;snps&amp;quot;, &#xA;         dataset = &amp;quot;hsapiens_snp&amp;quot;)&#xA;&#xA;# assuming your table is tab-separated, with column names&#xA;myData &amp;lt;- read.table(&amp;quot;myTable.txt&amp;quot;, sep=&amp;quot;\t&amp;quot;, header=True)&#xA;&#xA;coords &amp;lt;- apply(myData, 1, paste, collapse = &amp;quot;:&amp;quot;)&#xA;&#xA;&#xA;# submit the query&#xA;getBM(attributes = c('refsnp_id', 'chr_name', 'chrom_start', 'chrom_end', 'allele'),&#xA;filters = c('chromosomal_region'), &#xA;values = coords, &#xA;mart = snpMart)  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://support.bioconductor.org/p/133105/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Source&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="475" LastActivityDate="2023-01-13T10:06:49.087" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20363" PostTypeId="2" ParentId="20359" CreationDate="2023-01-13T10:39:44.550" Score="3" Body="&lt;p&gt;After dumping out the reads using &lt;code&gt;fasterq-dump&lt;/code&gt;, you'll need to align them first and then extract those that map to your region of interest. I think &lt;a href=&quot;https://github.com/lh3/minimap2&quot; rel=&quot;nofollow noreferrer&quot;&gt;minimap2&lt;/a&gt; is an excellent choice. If you have HiFi reads, you can just use something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 -ax map-hifi ref.fa pacbio-ccs.fq.gz &amp;gt; aln.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then, using &lt;a href=&quot;http://www.htslib.org/doc/samtools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools&lt;/a&gt;, coordinate-&lt;a href=&quot;http://www.htslib.org/doc/samtools-sort.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sort&lt;/a&gt; and &lt;a href=&quot;http://www.htslib.org/doc/samtools-index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;index&lt;/a&gt; the sorted alignments:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools sort -o aln.sorted.bam aln.sam&#xA;samtools index aln.sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With an indexed BAM, you can then &lt;a href=&quot;http://www.htslib.org/doc/samtools-view.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;view&lt;/a&gt; the alignments that overlap your region of interest:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view aln.sorted.bam chrZ:1-10000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you instead write these to a file, you can then convert them using samtools  &lt;a href=&quot;http://www.htslib.org/doc/samtools-fasta.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;fasta&lt;/a&gt;. I think something like the following should suffice:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -bo chrZ_1_10000.bam aln.sorted.bam chrZ:1-10000&#xA;samtools fasta -0 chrZ_1_10000.fa -o /dev/null chrZ_1_10000.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You could then upload the FASTA file to the NCBI &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;Web BLAST&lt;/a&gt; server.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2023-01-13T11:54:22.083" LastActivityDate="2023-01-13T11:54:22.083" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20364" PostTypeId="1" CreationDate="2023-01-13T12:14:28.153" Score="2" ViewCount="65" Body="&lt;p&gt;I am currently looking into some of the published methods for deconvolution of spatial transcriptomics data where each spot does not have single-cell resolution. These methods all rely on cell type annotated scRNA-seq data to build a reference of expected expression for each cell population. So in essence, because ground truth is more or less impossible to obtain for complete cell typing of a spatially intact tissue section, another type of data (scRNA) is used instead. This lets you infer something about the spatial organization of the tissue despite no spatial information being present in the training data, but in the end no ground truth is available for validating the results.&lt;/p&gt;&#xA;&lt;p&gt;I was trying to think of an analogous machine learning problem, biological or not, where something similar is done to deal with unattainable ground truch. Are there any obvious/good examples of something like this? Maybe some inspiration can be drawn for how such results are validated.&lt;/p&gt;&#xA;" OwnerUserId="16570" LastActivityDate="2023-01-13T15:29:12.267" Title="Examples of machine learning approaches to validate results where ground truth is lacking?" Tags="|rna-seq|scrnaseq|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20365" PostTypeId="2" ParentId="20364" CreationDate="2023-01-13T12:43:00.667" Score="1" Body="&lt;p&gt;If the problem concerns spatial clustering of cells then the ML algorithm is without question &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.cluster.DBSCAN.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DBSCAN&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The problem with deep learning and CNN (convolution) not &amp;quot;deconvolution&amp;quot;, i.e. the MBE paper cited, is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The model is an &lt;em&gt;ad hoc&lt;/em&gt; construction, which can be also a strength;&lt;/li&gt;&#xA;&lt;li&gt;Under-training, is resolved by adding more layers;&lt;/li&gt;&#xA;&lt;li&gt;Overtraining, can only be assessed post-analysis, in ML its algorithm specific;&lt;/li&gt;&#xA;&lt;li&gt;CNN approaches require large datasets for training of 2 million+.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The issue with ML is its performance tapers at around the 1 or 2 million data set size mark, whereas deep learning &lt;em&gt;can&lt;/em&gt; (stress &lt;em&gt;can&lt;/em&gt;) continue to use massive data sets to enhance performance (thats its strength).&lt;/p&gt;&#xA;&lt;p&gt;Whether, DBSCAN has received publications in bioinformatics - I'm not sure. However, what is present in the public domain isn't always the cutting edge work: a lesson I've very recently learnt. DBSCAN is very fashionable for spatial problems at present and in context personally would rank it above CNN.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;CNN vs ML&lt;/strong&gt; The thing is most of us - unless you're working for Google - don't have access to data sets in 2 million+ range. This can be circumvented by &lt;em&gt;transfer learning&lt;/em&gt; but it is not sufficiently developed outside Google and a few major players in computer vision. Has Google performed CNN on one million+ cell images - I doubt it. If the problem is ecological species identification via remote cameras - Googles CNN image libraries would be just the ticket (lots of dogs, cats and bear photos) but for microscopy studies I seriously doubt it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&#xA;What I am 100% certain on is DBSCAN is VERY fashionable right now and certain ML algorithms repeatedly outperform deep learning in certain situations. What I am trying to say is ML is not necessarily just a low data size version of deep learning: in some instances it surpasses it and I think DBSCAN has that potential. The broader challenges of deep learning hinge around 'transfer learning'. If biology manages to resolve that &lt;em&gt;then&lt;/em&gt; deep learning will start surpassing machine learning, but firstly it's about sharing, secondly in e.g. RNN this is not technically trivial.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;What you can do is pose a detailed problem in a later question and I could explain how training would be used ... generally in complex analytical problems it is difficult to assess precisely how a solution would be implemented.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-13T15:29:12.267" LastActivityDate="2023-01-13T15:29:12.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20366" PostTypeId="1" CreationDate="2023-01-13T18:57:31.097" Score="3" ViewCount="30" Body="&lt;p&gt;Cromwell can download files that are URLs (i.e. URLs in the inputs.json).  However, the permissions of these files, once they are downloaded, are very strict (&lt;code&gt;rw------- bmac bmac&lt;/code&gt;).  I was wondering if there was a way to control the file permissions of said files; I need them accessible to other users.&lt;/p&gt;&#xA;&lt;p&gt;Cromwell uses a library called &lt;code&gt;akka.http&lt;/code&gt; to download url-files.  As stated in the&lt;br /&gt;&#xA;&lt;a href=&quot;https://cromwell.readthedocs.io/en/stable/Configuring/&quot; rel=&quot;nofollow noreferrer&quot;&gt;cromwell configuration docs&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Cromwell uses akka-http to serve requests. For more advanced&#xA;configuration settings, refer to the akka-http documentation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If I run cromwell with the given files&#xA;&lt;code&gt;java -jar ~/cromwell/cromwell-84.jar run fq_count.wdl -i fq_count.url.json &lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I see the permissions for the url-input file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ls -l cromwell-executions/fq_count/2a65649a-999e-44ef-bec8-89953e0e3f1f/call-count_reads/inputs/429920399/12673489687364390176&#xA;-rw-------  2 bmac  bmac&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Question:  Is there a way to control the file permissions, i.e. through akka.http or other?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;WDL&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;version 1.0&#xA;workflow fq_count {&#xA;    input {&#xA;        File fastq_file&#xA;    }&#xA;    call count_reads {&#xA;      input: fastq = fastq_file&#xA;    }&#xA;}&#xA;&#xA;task count_reads {&#xA;  input {&#xA;    File fastq&#xA;    String memory = &amp;quot;5G&amp;quot;&#xA;    String time = &amp;quot;00:10:00&amp;quot;&#xA;  }&#xA;&#xA;  command &amp;lt;&amp;lt;&amp;lt;&#xA;    set -euo pipefail&#xA;    fastq='~{fastq}'&#xA;    extension=&amp;quot;${fastq##*.}&amp;quot;&#xA;&#xA;    if [ &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${extension}&quot; = 'gz' ]; then&#xA;  cat_cmd='zcat'&#xA;else&#xA;  cat_cmd='cat'&#xA;fi&#xA;# using awk instead of bc as bc isn't in debian slim images&#xA;&quot;$&lt;/span&gt;{cat_cmd}&amp;quot; &amp;quot;~{fastq}&amp;quot; | wc -l | awk '{ print int($1/4) }' &amp;gt; num_reads.txt&#xA;  &amp;gt;&amp;gt;&amp;gt;&#xA;&#xA;  output {&#xA;    File num_reads = &amp;quot;num_reads.txt&amp;quot;&#xA;  }&#xA;&#xA;  runtime {&#xA;    cpu: 1&#xA;    memory: memory&#xA;    time: time&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Input json&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{       &#xA;  &amp;quot;fq_count.fastq_file&amp;quot;: https://portal.nersc.gov/cfs/m342/jaws/test_data/sample.fastq&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16656" LastActivityDate="2023-01-13T18:57:31.097" Title="cromwell files downloaded from a URL have too strict permissions" Tags="|wdl|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20367" PostTypeId="2" ParentId="20336" CreationDate="2023-01-14T23:12:47.037" Score="2" Body="&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&#xA;What I am saying in this post is there approximates to 3 &amp;quot;generations&amp;quot; of &lt;em&gt;in silico&lt;/em&gt; gap-closers/polishes:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;GapFiller - amongst other &amp;quot;early&amp;quot;  generations (2012);&lt;/li&gt;&#xA;&lt;li&gt;medaka/racon/Nanopolish/Pilon (2014 - 2018 inclusively)&lt;/li&gt;&#xA;&lt;li&gt;Nextpolish/PEPPER/Apollo/Homopolish/NeuralPolish (2020 onwards).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The trendy approach is combining 2. and 3. I can't guarantee it will work for cyanobacteria gap-closing, but the shear volume of algorithms dedicated to this purpose, infers they must be delivering results.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&#xA;Whats the issue with a reference genome? The key thing about &lt;em&gt;many&lt;/em&gt;  bacteria is their ability to &amp;quot;dump&amp;quot; genes and then regain them via plasmid associated recombination. This can occur extensively within a single species. This makes reference associated assembly essentially impossible in many bacteria - &lt;em&gt;particularly against the official NCBI reference genome&lt;/em&gt;. Hence, the official &lt;em&gt;NCBI reference gene&lt;/em&gt;, is not like a human genome reference - which is a gold standard. With due to respect to those who have dedicated time to presenting the &lt;em&gt;official bacterial reference genomes&lt;/em&gt; are not very useful.&lt;/p&gt;&#xA;&lt;p&gt;This BioStars post &lt;a href=&quot;https://www.biostars.org/p/462167/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; demonstrates that even &lt;em&gt;Mycobacteria tuberculosis&lt;/em&gt; would not assemble via a reference genome and TB in context is one of the most conserved genomes in bacteria, the SNP behaviour is famously limited.&lt;/p&gt;&#xA;&lt;p&gt;If attempting a reference genome it is worth considering the following:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Any amount of genetic divergence can result &lt;em&gt;genetic island&lt;/em&gt; phenomena and non-contiguous genomes that will make assembly difficult, therefore a complete genome as close to the target genome as possible is a good reference.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For example, simply Blasting sections of you genome to look for a consistent reference that is close to yours. A custom database of complete genomes for your species my be useful, but just sending bits of your target onto the NCBI website will be useful. The idea is to get lots of bits of the genome matching the closest strain.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;QUAST was going to be my suggestion too.&lt;/li&gt;&#xA;&lt;li&gt;Mapping via e.g. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5802927/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mummer4&lt;/a&gt; against the reference genome as the &amp;quot;scaffold&amp;quot;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Key question:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The same strain also has 1-2 very fragmented genomes (100s of pieces) available in NCBI at scaffold level. The reads themselves are not available, so I can't just add them to the assembly.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I'm not sure what &amp;quot;not available&amp;quot; means.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Key suggestion: Polishing&lt;/strong&gt;&#xA;The other approach are recent &lt;em&gt;de novo&lt;/em&gt; assemblers and &lt;em&gt;in silico&lt;/em&gt; &amp;quot;polishing&amp;quot;. What I'm recommending is to first try &amp;quot;polishing&amp;quot;. This doesn't exclude subsequent reference genome assemblies - &lt;em&gt;not at all&lt;/em&gt; - but it's a way to improve the &lt;em&gt;de novo&lt;/em&gt; assemblies and if there are still issues then consider a reference genome.&lt;/p&gt;&#xA;&lt;p&gt;Firstly I would read someone who is &lt;em&gt;really&lt;/em&gt; up to date, e.g.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/s41598-021-00178-w&quot; rel=&quot;nofollow noreferrer&quot;&gt;Comparative evaluation of Nanopore polishing tools for microbial genome assembly and polishing strategies for downstream analysis&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;They are &lt;a href=&quot;https://github.com/nanoporetech/medaka&quot; rel=&quot;nofollow noreferrer&quot;&gt;medaka&lt;/a&gt; fans and are using this as part of a combination approach. medaka was part of the established stable of &amp;quot;polishers&amp;quot;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/1471-2105-13-S14-S8&quot; rel=&quot;nofollow noreferrer&quot;&gt;GapFiller&lt;/a&gt; - I don't think this is useful.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5411768/&quot; rel=&quot;nofollow noreferrer&quot;&gt;racon&lt;/a&gt; is specifically for long reads&lt;/li&gt;&#xA;&lt;li&gt;Nanopolish &lt;a href=&quot;https://www.nature.com/articles/nmeth.3444&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The authors prefer medaka then goes on to describe&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://academic.oup.com/bioinformatics/article/36/7/2253/5645175&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nextpolish&lt;/a&gt;, &lt;a href=&quot;https://www.nature.com/articles/s41592-021-01299-w&quot; rel=&quot;nofollow noreferrer&quot;&gt;PEPPER&lt;/a&gt;, Apollo, Homopolish, and NeuralPolish&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Much more recent from 2020 ... thats a lot of polishing tools. Note, PEPPER and Apollo are often used for algorithms names - make sure its 2020 onwards, with PEPPER being 2021. They suggest medaka x PEPPER.&lt;/p&gt;&#xA;&lt;p&gt;I really think this combination polishing approach with an established (medaka) and one the very latest polishing algorithms, such as PEPPER (2021) is really worth trying. If it doesn't work for the specific purpose it will certainly improve the contig quality. However, given the shear volume of work dedicated to this area then there has to be good reason.&lt;/p&gt;&#xA;&lt;p&gt;By the same token that&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Money doesnt give happiness, but sure gives a better standard of misery.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Then ...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Polishing might not give gap closure, but sure gives a better standard of quality to the same old bunch of miserable contigs&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;de novo&lt;/em&gt; Assembly&lt;/strong&gt;&#xA;Maybe &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/32/14/2103/1742895&quot; rel=&quot;nofollow noreferrer&quot;&gt;Miniasm&lt;/a&gt;? I'd look at &amp;quot;polishing&amp;quot; the &lt;em&gt;de novo&lt;/em&gt; assembly you have.&lt;/p&gt;&#xA;&lt;p&gt;RagTag as a SPAdes alternative is interesting @MaximilianPress&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-16T13:03:14.997" LastActivityDate="2023-01-16T13:03:14.997" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20368" PostTypeId="1" AcceptedAnswerId="20369" CreationDate="2023-01-15T10:41:51.840" Score="2" ViewCount="457" Body="&lt;p&gt;I have a data frame (df) which has correlations calculated for different genes with respect to different ID combinations. I want to get separate histogram plot based on  the gene name (separate plot for separate gene). Also after plotting I want to know which genes have more deviations in their correlation values (Like more deviated from the median in each plot).&lt;/p&gt;&#xA;&lt;p&gt;Example:&#xA;&lt;code&gt;df&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Gene    ID_1    ID_2    Correlation&#xA;TNF 1175077258  1175075956  0.626115&#xA;TNF 1175077261  1175075956  0.542499&#xA;TNF 1175077261  1175077258  0.50113&#xA;TNF 1175077263  1175075956  0.431587&#xA;TNF 1175077263  1175077258  0.734858&#xA;TNF 1175077263  1175077261  0.622206&#xA;CCL2    952399885   952399878   0.632703&#xA;CCL2    952399894   952399878   0.622244&#xA;CCL2    952399894   952399885   0.697313&#xA;CCL2    952399923   952399878   0.687826&#xA;CCL2    952399923   952399885   0.612089&#xA;CCL2    952399923   952399894   0.603834&#xA;IL10    1068768000  1068767927  0.118955&#xA;IL10    1068768147  1068767927  0.32038&#xA;IL10    1068768147  1068768000  0.430287&#xA;IL10    1068768409  1068767927  0.335264&#xA;IL10    1068768409  1068768000  0.406426&#xA;IL10    1068768409  1068768147  0.546452&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried the below code which only works when combining all genes together.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;import seaborn as sns&#xA;plt.rcParams[&amp;quot;figure.figsize&amp;quot;] = (8,6)&#xA;&#xA;g1 = sns.histplot(df.Correlation)&#xA;g1.set(&#xA;    xlim=(-1,1),&#xA;    title=&amp;quot;correlation&amp;quot;&#xA;)&#xA;g1.axvline(0.0, c='black', linestyle=&amp;quot;--&amp;quot;)&#xA;g1.axvline(df.Correlation.median(), c='red', linestyle=&amp;quot;--&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The result/plot I am expecting is this (I have created this example plot in excel)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vghLg.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vghLg.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4845" LastActivityDate="2023-01-15T12:59:08.977" Title="How to get separate histograms plots on the basis of the column value?How to detect which plot has most deviation?" Tags="|python|visualization|correlation|pandas|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20369" PostTypeId="2" ParentId="20368" CreationDate="2023-01-15T12:59:08.977" Score="2" Body="&lt;p&gt;You could use the &lt;a href=&quot;https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.groupby.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;groupby()&lt;/a&gt; pandas function to group the dataframe by gene name. And then just loop through each group to plot the histogram of correlation values. I think you can use the mean of the ratio (deviation / median) to compare deviation for different genes:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;&#xA;for name, group in df.groupby('Gene'):&#xA;&#xA;    median = group['Correlation'].median()&#xA;    deviation = abs(group['Correlation'] - median)&#xA;    ratio = deviation / median&#xA;    ratio_mean = ratio.mean()&#xA;&#xA;    id_1 = group['ID_1'].astype(str)&#xA;    id_2 = group['ID_2'].astype(str)&#xA;&#xA;    group['Combination'] = id_1 + '/' + id_2&#xA;&#xA;    plt.figure(figsize=(8, 6))&#xA;    plt.bar(group['Combination'], group['Correlation'])&#xA;&#xA;    plt.xlabel('Combination', fontsize=12, fontweight='bold')&#xA;    plt.ylabel('Correlation', fontsize=12, fontweight='bold')&#xA;&#xA;    plt.xticks(rotation=45, ha=&amp;quot;right&amp;quot;) &#xA;    plt.title(f'{name} (deviation ratio: {ratio_mean.round(3)})')&#xA;    plt.tight_layout()&#xA;&#xA;    plt.gca().spines['top'].set_visible(False)&#xA;    plt.gca().spines['right'].set_visible(False)&#xA;    &#xA;    plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FRqki.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FRqki.png&quot; alt=&quot;CCL2&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QAY8t.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QAY8t.png&quot; alt=&quot;IL10&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zhzld.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zhzld.png&quot; alt=&quot;TNF&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-01-15T12:59:08.977" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20370" PostTypeId="1" CreationDate="2023-01-15T23:06:20.147" Score="3" ViewCount="284" Body="&lt;p&gt;I'm new to dealing with big and compressed data so, I have some questions regarding that&lt;/p&gt;&#xA;&lt;p&gt;I have 1000 files, each file has the extension (vep.txt.gz) I want to read these files, then filter them. The size of these files is very large. I tried to read these files in Python and R but I am facing a memory error issue.&#xA;I need to read these files and extract some rows from each file for example I have two columns. In the first column, I want the rows with low values in the impact column and the rows with likely pathogenic values from the clinSIG column. Also, I have to add a new column to put the files' names as IDs for each sample.&lt;/p&gt;&#xA;&lt;p&gt;So, how can I read a list of big (vep.txt.gz) files in R or Python, then filter the content of the files based on two columns (impact = low) and (clin_SIG = Likely pathogenic) ?&lt;/p&gt;&#xA;&lt;p&gt;Should I decompress the files or just read them?&lt;/p&gt;&#xA;&lt;p&gt;The following is an example of the files but each file is compressed and all the files have the same header (you can find the original file, which is tab separated, [here][1]):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## ENSEMBL VARIANT EFFECT PREDICTOR v108.0&#xA;## Output produced at 2017-03-21 14:51:27&#xA;## Connected to homo_sapiens_core_108_38 on ensembldb.ensembl.org&#xA;## Using cache in /homes/user/.vep/homo_sapiens/108_GRCh38&#xA;## Using API version 108, DB version 108&#xA;## polyphen version 2.2.2&#xA;## sift version sift5.2.2&#xA;## COSMIC version 78&#xA;## ESP version 20141103&#xA;## gencode version GENCODE 25&#xA;## genebuild version 2014-07&#xA;## HGMD-PUBLIC version 20162&#xA;## regbuild version 16&#xA;## assembly version GRCh38.p7&#xA;## ClinVar version 201610&#xA;## dbSNP version 147&#xA;## Column descriptions:&#xA;## Uploaded_variation : Identifier of uploaded variant&#xA;## Location : Location of variant in standard coordinate format (chr:start or chr:start-end)&#xA;## Allele : The variant allele used to calculate the consequence&#xA;## Gene : Stable ID of affected gene&#xA;## Feature : Stable ID of feature&#xA;## Feature_type : Type of feature - Transcript, RegulatoryFeature or MotifFeature&#xA;## Consequence : Consequence type&#xA;## cDNA_position : Relative position of base pair in cDNA sequence&#xA;## CDS_position : Relative position of base pair in coding sequence&#xA;## Protein_position : Relative position of amino acid in protein&#xA;## Amino_acids : Reference and variant amino acids&#xA;## Codons : Reference and variant codon sequence&#xA;## Existing_variation : Identifier(s) of co-located known variants&#xA;## Extra column keys:&#xA;## IMPACT : Subjective impact classification of consequence type&#xA;## DISTANCE : Shortest distance from variant to transcript&#xA;## STRAND : Strand of the feature (1/-1)&#xA;## FLAGS : Transcript quality flags&#xA;&#xA;#Uploaded_variation  Location   Allele  Gene             Feature          Feature_type  Consequence                            cDNA_position  CDS_position  Protein_position  Amino_acids  Codons   Existing_variation  IMPACT    DISTANCE  STRAND  CLINSIG &#xA;11_224088_C/A        11:224088  A       ENSG00000142082  ENST00000525319  Transcript    missense_variant                       742            716           239               S/I          aGc/aTc  -                   LOW  -         -1      likely pathogenic  &#xA;11_224088_C/A        11:224088  A       ENSG00000142082  ENST00000534381  Transcript    downstream_gene_variant                -              -             -                 -            -        -                   MODIFIER  1674      -1      -&#xA;11_224088_C/A        11:224088  A       ENSG00000142082  ENST00000529055  Transcript    downstream_gene_variant                -              -             -                 -            -        -                   HIGH  134       -1      -&#xA;11_224585_G/A        11:224585  A       ENSG00000142082  ENST00000529937  Transcript    intron_variant,NMD_transcript_variant  -              -             -                 -            -        -                   MODIFIER  -         -1      -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From the previous file, the expected output will be one row which is the first row because the impact is low &amp;amp; CLINSIG is likely pathogenic so i want to find all rows with this condition for 1000 files&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ID  #Uploaded_variation  Location   Allele  Gene    Feature Feature_type  Consequence   cDNA_position   CDS_position    Protein_position  Amino_acids  Codons   Existing_variation  IMPACT    DISTANCE  STRAND  CLINSIG &#xA;M1457.vep   11_224088_C/A   11:224088   A   ENSG00000142082 ENST00000525319 Transcript  missense_variant    742 716 239 S/I aGc/aTc -   LOW  -  -1  likely pathogenic&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="16772" LastEditorUserId="16772" LastEditDate="2023-01-18T11:36:54.997" LastActivityDate="2023-01-26T12:46:18.530" Title="dealing with a list of VEP files" Tags="|r|python|filtering|vep|" AnswerCount="2" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="20371" PostTypeId="2" ParentId="20370" CreationDate="2023-01-16T01:06:45.570" Score="3" Body="&lt;p&gt;For these requirements, it might be possible to simply use &lt;a href=&quot;https://asia.ensembl.org/info/docs/tools/vep/script/vep_filter.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;filter_vep&lt;/a&gt; included in the VEP package for this. Note that multiple &lt;code&gt;--filter&lt;/code&gt; flags are ANDed together (i.e. all filters must pass for a line to be printed). For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in *.vep.txt.gz; do&#xA;    filter_vep \&#xA;        -i &amp;quot;${i}&amp;quot; \&#xA;        -o &amp;quot;${i%.vep.txt.gz}.filtered.vep.txt&amp;quot; \&#xA;        --filter &amp;quot;IMPACT is LOW&amp;quot; \&#xA;        --filter &amp;quot;CLIN_SIG matches likely_pathogenic&amp;quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you then wanted to merge the output and include the filename, you could just use &lt;a href=&quot;/questions/tagged/awk&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; aria-label=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; rel=&quot;tag&quot; aria-labelledby=&quot;awk-container&quot;&gt;awk&lt;/a&gt;. With the following in a file called &lt;code&gt;merge_filtered_vep_txt.awk&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BEGIN {&#xA;    FS=OFS=&amp;quot;\t&amp;quot;&#xA;    f=1&#xA;}&#xA;&#xA;/^##/ {&#xA;    next&#xA;}&#xA;&#xA;sub(/^#/, &amp;quot;&amp;quot;) &amp;amp;&amp;amp; f {&#xA;    print &amp;quot;ID&amp;quot;, $0&#xA;    f=0&#xA;    next&#xA;}&#xA;&#xA;{&#xA;    print FILENAME, $0&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Run using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk -f merge_filtered_vep_txt.awk /path/to/*.filtered.vep.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Python can also open gzip-compressed files for reading (and writing), using the &lt;a href=&quot;https://docs.python.org/3/library/gzip.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;gzip&lt;/a&gt; module in the standard library. These can be parsed line-by-line to avoid memory errors. Here's one way using the &lt;a href=&quot;https://docs.python.org/3/library/csv.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;csv&lt;/a&gt; module (also in the standard library):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import argparse&#xA;import contextlib&#xA;import csv&#xA;import gzip&#xA;import pathlib&#xA;import sys&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;def get_argument_parser():&#xA;&#xA;    parser = argparse.ArgumentParser(add_help=False)&#xA;&#xA;    group = parser.add_argument_group('input options')&#xA;    group.add_argument(&#xA;        &amp;quot;-i&amp;quot;,&#xA;        &amp;quot;--input&amp;quot;,&#xA;        dest='glob_pattern',&#xA;        type=str,&#xA;        default=&amp;quot;**/*.vep.txt.gz&amp;quot;,&#xA;        metavar=&amp;quot;STR&amp;quot;,&#xA;        help=&amp;quot;The relative glob pattern (default: '%(default)s')&amp;quot;,&#xA;    )&#xA;    group.add_argument(&#xA;        &amp;quot;-d&amp;quot;,&#xA;        &amp;quot;--dir&amp;quot;,&#xA;        dest='glob_path',&#xA;        type=pathlib.Path,&#xA;        metavar=&amp;quot;DIR&amp;quot;,&#xA;        default='.',&#xA;        help=&amp;quot;Glob files in DIR (default: '%(default)s')&amp;quot;,&#xA;    )&#xA;&#xA;    group = parser.add_argument_group('output options')&#xA;    group.add_argument(&#xA;        &amp;quot;-o&amp;quot;,&#xA;        &amp;quot;--out&amp;quot;,&#xA;        type=pathlib.Path,&#xA;        metavar=&amp;quot;FILE&amp;quot;,&#xA;        default='-',&#xA;        help=&amp;quot;Write the filtered output to FILE (default: stdout)&amp;quot;,&#xA;    )&#xA;    group.add_argument(&#xA;        &amp;quot;-f&amp;quot;,&#xA;        &amp;quot;--force&amp;quot;,&#xA;        action='store_true',&#xA;        help=&amp;quot;Overwrite the output file if it exists&amp;quot;,&#xA;    )&#xA;&#xA;    group = parser.add_argument_group('additional options')&#xA;    group.add_argument(&#xA;        &amp;quot;-h&amp;quot;,&#xA;        &amp;quot;--help&amp;quot;,&#xA;        action=&amp;quot;help&amp;quot;,&#xA;        help=&amp;quot;Show this help message and exit&amp;quot;,&#xA;    )&#xA;&#xA;    return parser&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;def main():&#xA;&#xA;    parser = get_argument_parser()&#xA;    args = parser.parse_args()&#xA;&#xA;    with contextlib.ExitStack() as stack:&#xA;        if args.out.name != '-':&#xA;            out = stack.enter_context(open(args.out, 'wt' if args.force else 'xt'))&#xA;        else:&#xA;            out = sys.stdout&#xA;&#xA;        # Get a list of files from the directory specified by args.glob_path&#xA;&#xA;        file_list = sorted(args.glob_path.glob(args.glob_pattern))&#xA;&#xA;        # Build up a unique list of header fields from each input file&#xA;&#xA;        header_cols = ['ID']&#xA;        for gz_file in file_list:&#xA;            with gzip.open(gz_file, 'rt') as infile:&#xA;                for line in infile:&#xA;                    if not line.startswith('##'):&#xA;                        break&#xA;&#xA;                assert line.startswith('#')&#xA;&#xA;                for field in line[1:].rstrip().split('\t'):&#xA;                    if field not in header_cols:&#xA;                        header_cols.append(field)&#xA;&#xA;        # Create a csv.DictWriter() and write the header fields&#xA;&#xA;        writer = csv.DictWriter(out, fieldnames=header_cols, delimiter='\t')&#xA;        writer.writeheader()&#xA;&#xA;        # Now re-open each file in the arguments list for reading&#xA;&#xA;        for gz_file in file_list:&#xA;            with gzip.open(gz_file, 'rt') as infile:&#xA;&#xA;                # Skip the info header lines&#xA;                line = next(infile)&#xA;                while line.startswith('##'):&#xA;                    line = next(infile)&#xA;&#xA;                # Get the column headers&#xA;                fieldnames = line.rstrip().removeprefix('#').split('\t')&#xA;&#xA;                # Parse each input file and store the desired rows in memory&#xA;                reader = csv.DictReader(infile, fieldnames=fieldnames, delimiter='\t')&#xA;                rows = []&#xA;                for row in reader:&#xA;&#xA;                    impact = row.get('IMPACT')&#xA;                    clinsig = row.get('CLINSIG')&#xA;&#xA;                    if impact == 'LOW' and clinsig == 'likely pathogenic':&#xA;                        rows.append({'ID': gz_file, **row})&#xA;&#xA;                # Write the list of rows to the output file&#xA;                writer.writerows(rows)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;if __name__ == '__main__':&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With the above in a file called &lt;code&gt;filter_vep_txt.py&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;usage: filter_vep_txt.py [-i STR] [-d DIR] [-o FILE] [-f] [-h]&#xA;&#xA;input options:&#xA;  -i STR, --input STR  The relative glob pattern (default: '**/*.vep.txt.gz')&#xA;  -d DIR, --dir DIR    Glob files in DIR (default: '.')&#xA;&#xA;output options:&#xA;  -o FILE, --out FILE  Write the filtered output to FILE (default: stdout)&#xA;  -f, --force          Overwrite the output file if it exists&#xA;&#xA;additional options:&#xA;  -h, --help           Show this help message and exit&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2023-01-26T12:46:18.530" LastActivityDate="2023-01-26T12:46:18.530" CommentCount="21" ContentLicense="CC BY-SA 4.0" />
  <row Id="20372" PostTypeId="1" CreationDate="2023-01-16T12:31:35.870" Score="1" ViewCount="126" Body="&lt;p&gt;I was able to get the exons of a particular gene as a fasta file from the NCBI genome viewer. As for introns I only found a bed file that showed where the intron starts and ends, along with its direction. What tool do I use to get the exact sequence of these 11 introns using this bed file?&#xA;And is there a more direct way?&#xA;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="9089" LastActivityDate="2023-01-16T13:17:28.990" Title="How to get a file with the introns of a gene?" Tags="|gene|reference-genome|bed|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20373" PostTypeId="2" ParentId="20372" CreationDate="2023-01-16T13:17:28.990" Score="1" Body="&lt;p&gt;First I edited the Chromosome column of the bed file from ncbi, because it contained the accession number instead of the chromosome. Then I used the getfasta tool of the bedtools utility.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;bedtools getfasta -fi input_genome_file.fa -fo output.fa -bed bed_file&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The output contained the introns in the format I wanted.&#xA;If anybody had a better way, feel free to post.&lt;/p&gt;&#xA;" OwnerUserId="9089" LastActivityDate="2023-01-16T13:17:28.990" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20374" PostTypeId="1" AcceptedAnswerId="20377" CreationDate="2023-01-16T19:51:58.967" Score="2" ViewCount="76" Body="&lt;p&gt;Subset of &lt;code&gt;df&lt;/code&gt; before &lt;code&gt;groupby&lt;/code&gt; looks like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Diabetes_Group  DM-1    DM-2    corr&#xA;Group1  56  56  1&#xA;Group1  56  58  0.611&#xA;Group1  56  61  0.448&#xA;Group1  56  63  0.3587&#xA;Group1  58  56  0.611&#xA;Group1  58  58  1&#xA;Group1  58  61  0.60113&#xA;Group1  58  63  0.534858&#xA;Group1  61  56  0.448&#xA;Group1  61  58  0.60113&#xA;Group1  61  61  1&#xA;Group1  61  63  0.622206&#xA;Group1  63  56  0.3587&#xA;Group1  63  58  0.534858&#xA;Group1  63  61  0.622206&#xA;Group1  63  63  1&#xA;Group2  78  78  1&#xA;Group2  78  85  0.622703&#xA;Group2  78  94  0.622244&#xA;Group2  78  23  0.687826&#xA;Group2  85  78  0.622703&#xA;Group2  85  85  1&#xA;Group2  85  94  0.697313&#xA;Group2  85  23  0.612089&#xA;Group2  94  78  0.622244&#xA;Group2  94  85  0.697313&#xA;Group2  94  94  1&#xA;Group2  94  23  0.603834&#xA;Group2  23  78  0.687826&#xA;Group2  23  85  0.612089&#xA;Group2  23  94  0.603834&#xA;Group2  23  23  1&#xA;Group3  27  27  1&#xA;Group3  27  80  0.118955&#xA;Group3  27  147 0.32038&#xA;Group3  27  9   0.335264&#xA;Group3  80  27  0.118955&#xA;Group3  80  80  1&#xA;Group3  80  147 0.430287&#xA;Group3  80  9   0.406426&#xA;Group3  147 27  0.32038&#xA;Group3  147 80  0.430287&#xA;Group3  147 147 1&#xA;Group3  147 9   0.546452&#xA;Group3  9   27  0.335264&#xA;Group3  9   80  0.406426&#xA;Group3  9   147 0.546452&#xA;Group3  9   9   1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What statistical test can be used to do comparative analysis between the different ID's (&lt;code&gt;DM1 or DM2&lt;/code&gt;) to check which ID pair has weaker/stronger correlation within each diabetes_group?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there any statistical test to categorise the data into three categories:&lt;/p&gt;&#xA;&lt;p&gt;(a) &lt;code&gt;pairs of DM1 AND DM2&lt;/code&gt; which are highly correlated than others.&lt;/p&gt;&#xA;&lt;p&gt;(b) pairs that are not highly correlated&lt;/p&gt;&#xA;&lt;p&gt;(c) For the category 1 pairs (highly correlated), is there any particular ID that outperforms than other ID's among different groups?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="4783" LastEditorUserId="4783" LastEditDate="2023-01-17T07:57:57.747" LastActivityDate="2023-01-17T17:13:20.150" Title="How to statistically test the strength and weakness of a correlated pair?" Tags="|python|statistics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20375" PostTypeId="1" AcceptedAnswerId="20376" CreationDate="2023-01-16T20:37:03.097" Score="2" ViewCount="37" Body="&lt;p&gt;How to find specific types of assemblies for specific species using entrez tools?&lt;/p&gt;&#xA;&lt;p&gt;Task: Trying to specifically find transcriptomes and associated cDNA data for a list of speices.&lt;/p&gt;&#xA;&lt;p&gt;I can use this oneliner &lt;code&gt;esearch -db assembly -query &amp;quot;{species name }&amp;quot; | esummary&lt;/code&gt; to get a list of assemblies and associated data available for any given species.&lt;/p&gt;&#xA;&lt;p&gt;I can use &lt;code&gt;esearch -db assembly -query &amp;quot;{species name }&amp;quot; | xtract -pattern ENTREZ_DIRECT -element Count&lt;/code&gt; to find any specific info nested within the XML tree.&lt;/p&gt;&#xA;&lt;p&gt;Now I am not sure how to find if the assembly is a transcriptome (which is what I am looking for). I have tried to &lt;code&gt;CRTL +f&lt;/code&gt; terms like &lt;code&gt;rna&lt;/code&gt;, &lt;code&gt;peptide&lt;/code&gt;,&lt;code&gt;amino acids&lt;/code&gt; to no avail and I guess there is a chance I am missing something while looking it over manually.&lt;/p&gt;&#xA;&lt;p&gt;For example here is the output for &lt;a href=&quot;https://www.pastery.net/atxtjn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;zea mays&amp;quot;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have any clues? Or should I be using a different db or different tool?&lt;/p&gt;&#xA;" OwnerUserId="16699" LastActivityDate="2023-02-15T23:02:00.347" Title="How to find specific types of assemblies for specific species using entrez tools?" Tags="|assembly|transcriptome|entrez|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20376" PostTypeId="2" ParentId="20375" CreationDate="2023-01-16T20:54:27.110" Score="0" Body="&lt;p&gt;I strongly recommend switching to SRA &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/sra&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Whether the following would work ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db sra -query &amp;quot;{species name }&amp;quot; | esummary&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm not at all sure because SRA has its own toolkit. Maybe NCBI &lt;code&gt;e-utilities&lt;/code&gt; &lt;code&gt;e-summary&lt;/code&gt; &lt;code&gt;efetch&lt;/code&gt; would work on the SRA database. However, the weblink certainly works for transcriptomics.&lt;/p&gt;&#xA;&lt;p&gt;NCBI has shifted some of its database to be housed off-site onto GCP and AWS and I have a feeling these may not be accessible to &lt;code&gt;e-utilities&lt;/code&gt;, but I could be wrong.&lt;/p&gt;&#xA;&lt;p&gt;Just to note, what I envisage the question meaning is a complete transcriptomic study for a given species (or given sample) under given experimental conditions.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Comments. I better clarify, SRA will be raw Illumina output, it is not assembled into exome cDNA. &lt;code&gt;e-utils&lt;/code&gt; is basically for assembled cDNA, it might give you the information - I really don't think it will download it though. So to answer your questions, yes the data will be there on SRA, whether this is the format you want is a different question.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-16T22:21:19.213" LastActivityDate="2023-01-16T22:21:19.213" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20377" PostTypeId="2" ParentId="20374" CreationDate="2023-01-16T22:39:02.247" Score="2" Body="&lt;p&gt;This comprehensive analytical outline for the stats and all the stats questions could be captured inside 3 analyses. Implementing this will obviously take a bit of time, but there should not be any hidden caveats (transformations etc ...).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 3&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The statistical test you want is K-means clustering based on the correlation score where the number of predefined groups is 3. The labels ... you will need to combine the group with the&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Label          Value&#xA;Group1-56-56   1&#xA;... &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is an absolute solution to the data and really cool use of K-means.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 3c&lt;/strong&gt;&#xA;An easy solution is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;perform a new K-means with e.g. 10 groups and find the group with highest correlation values.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;A better solution is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;hierarchical clustering would identify specific groups and for this sub-question is a good analysis. The input is the correlation matrix, the output is a tree. HOWEVER, interpreting this could be difficult if you're not trained in reading trees.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;A 2-way ANOVA would be a really good solution to this data for Q2. The test would be the columns were DM1 DM2 and the rows Group1, Group2, Group3. Interpreting it would be fun because you'd also have to grapple with a concept called &amp;quot;interaction&amp;quot; - which could be important here. However, the ability to address a whole raft of questions within &lt;em&gt;one&lt;/em&gt; analysis is what 2-way ANOVA would permit. There is a slight issue of unequal group sizes 1, 2 and 3, because it would be better if Group 1,2 and 3 etc... were equal in size. However, the raw data looks to be very amenable to ANOVA. You must not use the correlation index in this test - just the raw values.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;K-means clustering is in &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.cluster.vq.kmeans.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy&lt;/a&gt; and &lt;code&gt;scikit-learn&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Hierarchical clustering is also in &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/cluster.hierarchy.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;scipy&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For ANOVA&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import statsmodels.api as sm&#xA;from statsmodels.formula.api import ols&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There will be easy approaches to 2-way ANOVA e.g. SPSS and I think Excel (needs a plug in). I suspect that Hierarchical clustering will also be in a nice point and click package like SPSS (I don't use them, so I don't precisely know.)&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-17T17:13:20.150" LastActivityDate="2023-01-17T17:13:20.150" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20378" PostTypeId="1" AcceptedAnswerId="20380" CreationDate="2023-01-16T23:42:31.753" Score="2" ViewCount="67" Body="&lt;p&gt;I'm trying to use Velvet Optimizer for a De Novo Assembly; I set my hash-lengths to be between 55 and 69&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/projects/path/to/perlscripts/VelvetOptimiser-2.2.6/VelvetOptimiser.pl -s 55 -e 69 -f '-shortPaired -fastq.gz -separate trial2.trimmed.paired.R1.fastq.gz trial2.trimmed.paired.R2.fastq.gz' -f '-short -fastq.gz trial2.trimmed.unpaired.R1.fastq.gz' -f '-short -fastq.gz trial2.trimmed.unpaired.R2.fastq.gz' -x 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I check the output file, it returns me with&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Starting to check input parameters.&#xA;    Start hash value too high.  New start hash value is 31.&#xA;    End hash value not in workable range.  New end hash value is 31&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any idea where I could be going wrong in my command?&lt;/p&gt;&#xA;" OwnerUserId="14657" LastActivityDate="2023-01-17T02:49:52.767" Title="Velvet Optimizer automatically changes to hash-length 31" Tags="|assembly|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20379" PostTypeId="2" ParentId="17984" CreationDate="2023-01-17T00:36:54.427" Score="0" Body="&lt;p&gt;my attempt working as of the 17th January 2023 using RCSB PDB Search API:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import json&#xA;&#xA;# import urllib.parse&#xA;&#xA;import requests&#xA;&#xA;import sys&#xA;&#xA;# data = {&amp;quot;query&amp;quot;:{&amp;quot;type&amp;quot;:&amp;quot;terminal&amp;quot;,&amp;quot;service&amp;quot;:&amp;quot;text&amp;quot;,&amp;quot;parameters&amp;quot;:{&amp;quot;attribute&amp;quot;:&amp;quot;rcsb_entry_container_identifiers.entry_id&amp;quot;,&#xA;            # &amp;quot;operator&amp;quot;:&amp;quot;exists&amp;quot;,&amp;quot;negation&amp;quot;:False}},&amp;quot;return_type&amp;quot;:&amp;quot;entry&amp;quot;}&#xA;&#xA;data = {&amp;quot;query&amp;quot;:{&amp;quot;type&amp;quot;:&amp;quot;terminal&amp;quot;,&amp;quot;service&amp;quot;:&amp;quot;text&amp;quot;,&amp;quot;parameters&amp;quot;:{&amp;quot;attribute&amp;quot;:&amp;quot;rcsb_entry_container_identifiers.entry_id&amp;quot;,&#xA;             &amp;quot;operator&amp;quot;:&amp;quot;exists&amp;quot;,&amp;quot;negation&amp;quot;:False}},&amp;quot;request_options&amp;quot;:{&amp;quot;return_all_hits&amp;quot;:True},&amp;quot;return_type&amp;quot;:&amp;quot;entry&amp;quot;}&#xA;&#xA;data = json.dumps(data)&#xA;&#xA;&#xA;headers = {'content-type': 'application/json'}&#xA;url = 'https://search.rcsb.org/rcsbsearch/v2/query?json='&#xA;&#xA;params = {}&#xA;&#xA;risposta = requests.post(url, params=params, data = data, headers=headers)&#xA;&#xA;# print(risposta.status_code,'\n', risposta.content.decode())&#xA;&#xA;rispo = json.loads(risposta.content.decode())&#xA;&#xA;# print(rispo[&amp;quot;result_set&amp;quot;], type(rispo[&amp;quot;result_set&amp;quot;]), len(rispo[&amp;quot;result_set&amp;quot;]))&#xA;&#xA;&#xA;id_list = []&#xA;for i in rispo[&amp;quot;result_set&amp;quot;] :&#xA;    &#xA;    id_list.append(i['identifier'])&#xA;&#xA;&#xA;print(id_list, type(id_list) , 'lenght : ', len(id_list), 'size bytes  : ' ,sys.getsizeof(id_list))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;you get a huge list :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;.............................&#xA;'2V0H', '2V0G', '2V0F', '2V0E', '2V0D', '370D', '2ULL', '2UKD', '2UGI', '2UDP', '2UCZ', '2UBP', '2UAG', '2TYS', '369D', '368D', '367D', '366D', '2TUN', '2TSS', '2TSR', '2TSY', '365D', '2TRT', '2TRS', '2TS1', '2TRY', '2TRX', '364D', '2TSC', '2TSB', '2TSA', '363D', '2TRC', '2TRA', '2TRM', '2TRH'] &#xA;&#xA;&amp;lt;class 'list'&amp;gt; lenght :  200069 size bytes  :  1671784&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Using the requests GET endpoints :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;&#xA;import sys&#xA;&#xA;&#xA;url = 'https://search.rcsb.org/rcsbsearch/v2/query?json=%7B%22query%22%3A+%7B%22type%22%3A+%22terminal%22%2C+%22service%22%3A+%22text%22%2C+%22parameters%22%3A+%7B%22attribute%22%3A+%22rcsb_entry_container_identifiers.entry_id%22%2C+%22operator%22%3A+%22exists%22%2C+%22negation%22%3A+false%7D%7D%2C+%22request_options%22%3A+%7B%22return_all_hits%22%3A+true%7D%2C+%22return_type%22%3A+%22entry%22%7D'&#xA;&#xA;&#xA;risposta = requests.get(url)&#xA;&#xA;&#xA;print('+++++++++++++++++++++++++++++')&#xA;&#xA;rispo = risposta.json()&#xA;&#xA;id_list = []&#xA;for i in rispo[&amp;quot;result_set&amp;quot;] :&#xA;    &#xA;    id_list.append(i['identifier'])&#xA;&#xA;print(id_list, type(id_list) , 'lenght : ', len(id_list), 'size bytes  : ' ,sys.getsizeof(id_list))&#xA;&#xA;print('--------------------------------------')&#xA;&#xA;print(risposta.status_code)&#xA;&#xA;print('\n\n',risposta.url)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;getting as of the 19th January 2023 --&amp;gt; 200375 entries&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2023-01-19T15:17:13.560" LastActivityDate="2023-01-19T15:17:13.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20380" PostTypeId="2" ParentId="20378" CreationDate="2023-01-17T02:49:52.767" Score="4" Body="&lt;p&gt;If a value for &lt;code&gt;MAXKMERLENGTH&lt;/code&gt; was not set when compiling &lt;code&gt;velveth&lt;/code&gt;, VelvetOptimizer just uses the default &lt;code&gt;$maxhash&lt;/code&gt; value of 31&lt;sup&gt;&lt;a href=&quot;https://github.com/tseemann/VelvetOptimiser/blob/2.2.6/VelvetOptimiser.pl#L110-L112&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;&lt;/sup&gt;. The hash lengths that you specify on the command line must be less than or equal to this value to run the tool successfully. The solution is to re-compile &lt;code&gt;velveth&lt;/code&gt; with a larger &lt;code&gt;MAXKMERLENGTH&lt;/code&gt;, for example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;make 'MAXKMERLENGTH=69'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alternatively, re-install using the &lt;a href=&quot;/questions/tagged/bioconda&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;bioconda&amp;#39;&quot; aria-label=&quot;show questions tagged &amp;#39;bioconda&amp;#39;&quot; rel=&quot;tag&quot; aria-labelledby=&quot;bioconda-container&quot;&gt;bioconda&lt;/a&gt; package manager:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c bioconda perl-velvetoptimiser&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This package includes &lt;a href=&quot;https://anaconda.org/bioconda/velvet&quot; rel=&quot;nofollow noreferrer&quot;&gt;velvet&lt;/a&gt;, which is built using a much higher maximum k-mer length and is usually sufficient for most purposes: &lt;code&gt;MAXKMERLENGTH=191&lt;/code&gt;&lt;sup&gt;&lt;a href=&quot;https://github.com/bioconda/bioconda-recipes/blob/master/recipes/velvet/build.sh&quot; rel=&quot;nofollow noreferrer&quot;&gt;2&lt;/a&gt;&lt;/sup&gt;.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-01-17T02:49:52.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20381" PostTypeId="1" CreationDate="2023-01-17T07:58:27.710" Score="2" ViewCount="83" Body="&lt;p&gt;I tried the below code to get the background_gradient plot for each diabetes_group. It is giving me only plot for the last group in the plot. How to get background_gradiant for all items (&lt;code&gt;x&lt;/code&gt;) in &lt;code&gt;diabetes_group&lt;/code&gt; with item name on the top ?&lt;/p&gt;&#xA;&lt;p&gt;Subset of &lt;code&gt;df&lt;/code&gt; before &lt;code&gt;groupby&lt;/code&gt; looks like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Diabetes_Group  DM-1    DM-2    corr&#xA;Group1  56  56  1&#xA;Group1  56  58  0.611&#xA;Group1  56  61  0.448&#xA;Group1  56  63  0.3587&#xA;Group1  58  56  0.611&#xA;Group1  58  58  1&#xA;Group1  58  61  0.60113&#xA;Group1  58  63  0.534858&#xA;Group1  61  56  0.448&#xA;Group1  61  58  0.60113&#xA;Group1  61  61  1&#xA;Group1  61  63  0.622206&#xA;Group1  63  56  0.3587&#xA;Group1  63  58  0.534858&#xA;Group1  63  61  0.622206&#xA;Group1  63  63  1&#xA;Group2  78  78  1&#xA;Group2  78  85  0.622703&#xA;Group2  78  94  0.622244&#xA;Group2  78  23  0.687826&#xA;Group2  85  78  0.622703&#xA;Group2  85  85  1&#xA;Group2  85  94  0.697313&#xA;Group2  85  23  0.612089&#xA;Group2  94  78  0.622244&#xA;Group2  94  85  0.697313&#xA;Group2  94  94  1&#xA;Group2  94  23  0.603834&#xA;Group2  23  78  0.687826&#xA;Group2  23  85  0.612089&#xA;Group2  23  94  0.603834&#xA;Group2  23  23  1&#xA;Group3  27  27  1&#xA;Group3  27  80  0.118955&#xA;Group3  27  147 0.32038&#xA;Group3  27  9   0.335264&#xA;Group3  80  27  0.118955&#xA;Group3  80  80  1&#xA;Group3  80  147 0.430287&#xA;Group3  80  9   0.406426&#xA;Group3  147 27  0.32038&#xA;Group3  147 80  0.430287&#xA;Group3  147 147 1&#xA;Group3  147 9   0.546452&#xA;Group3  9   27  0.335264&#xA;Group3  9   80  0.406426&#xA;Group3  9   147 0.546452&#xA;Group3  9   9   1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;My code&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;corr_matrix = []&#xA;&#xA;for x, group in df.groupby('diabetes_group'):&#xA;    matrix = group[[&amp;quot;diabetes_group&amp;quot;, 'DM-1', 'DM-2', 'corr']]&#xA;    matrix_pivot = pd.pivot_table(matrix, values = 'corr', index=['DM-1'], columns=['DM-2'])&#xA;    d = matrix_pivot.style.background_gradient(cmap='Set3')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4783" LastActivityDate="2023-01-20T14:35:37.787" Title="To get background_gradient plot for all data frames in the for loop" Tags="|python|visualization|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20382" PostTypeId="2" ParentId="20381" CreationDate="2023-01-17T16:45:25.550" Score="1" Body="&lt;pre&gt;&lt;code&gt;corr_matrix = []&#xA;&#xA;for x, group in df.groupby('diabetes_group'):&#xA;    matrix = group[[&amp;quot;diabetes_group&amp;quot;, 'DM-1', 'DM-2', 'corr']]&#xA;    matrix_pivot = pd.pivot_table(matrix, values = 'corr', index=['DM-1'], columns=['DM-2'])&#xA;    d = matrix_pivot.style.background_gradient(cmap='Set3').set_caption(x)&#xA;    display(d)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4783" LastEditorUserId="4783" LastEditDate="2023-01-17T20:27:51.773" LastActivityDate="2023-01-17T20:27:51.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20383" PostTypeId="1" CreationDate="2023-01-17T19:06:12.267" Score="2" ViewCount="339" Body="&lt;p&gt;I saw in the literature a lot genome wide scans showing &lt;span class=&quot;math-container&quot;&gt;$F_{ST}$&lt;/span&gt; values in a 'z-transformed' format which seems to be bounded between 0 to infinity.&lt;/p&gt;&#xA;&lt;p&gt;Examples are found easily searching for 'genome-wide scan &lt;span class=&quot;math-container&quot;&gt;$F_{ST}$&lt;/span&gt;' and here is a paper where I could find such z-transformed &lt;span class=&quot;math-container&quot;&gt;$F_{ST}$&lt;/span&gt;:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.researchgate.net/publication/290186025_Identification_of_genomic_variants_putatively_targeted_by_selection_during_dog_domestication&quot; rel=&quot;nofollow noreferrer&quot;&gt;Identification of genomic variants putatively targeted by selection during dog domestication&lt;/a&gt; (see figure 1)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.researchgate.net/publication/353800474_Genomic_divergence_during_feralization_reveals_both_conserved_and_distinct_mechanisms_of_parallel_weediness_evolution/figures?lo=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Genomic divergence during feralization reveals both conserved and distinct mechanisms of parallel weediness evolution&lt;/a&gt; (see figure 3)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;How are the z-transformed &lt;span class=&quot;math-container&quot;&gt;$F_{ST}$&lt;/span&gt; values calculated?&lt;/p&gt;&#xA;&lt;p&gt;The only reference I could find so far is this: &lt;a href=&quot;https://www.biostars.org/p/336980/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Calculate &lt;span class=&quot;math-container&quot;&gt;$F_{ST}$&lt;/span&gt;to draw Manhattan plot&lt;/a&gt;. And this &lt;a href=&quot;https://bmcgenomics.biomedcentral.com/articles/10.1186/s12864-020-06933-z#Sec9&quot; rel=&quot;nofollow noreferrer&quot;&gt;Population analysis of the Korean native duck using whole-genome sequencing data&lt;/a&gt; where they just take the (&lt;span class=&quot;math-container&quot;&gt;$F_{ST}$&lt;/span&gt; - mean &lt;span class=&quot;math-container&quot;&gt;$F_{ST}$&lt;/span&gt;)/sd_&lt;span class=&quot;math-container&quot;&gt;$F_{ST}$&lt;/span&gt;. But this is giving a lot of negative values, so should this be also surrounded by an absolute value?&lt;/p&gt;&#xA;&lt;p&gt;Then is it possible to use the z-scores to get -log(p-values)?&lt;/p&gt;&#xA;" OwnerUserId="6669" LastEditorUserId="73" LastEditDate="2023-01-18T20:35:16.613" LastActivityDate="2023-01-18T20:35:16.613" Title="How to z-transform Fst and -log(p) values for genome wide selection scan?" Tags="|statistics|gwas|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20385" PostTypeId="1" AcceptedAnswerId="20386" CreationDate="2023-01-17T21:26:03.190" Score="5" ViewCount="498" Body="&lt;p&gt;I have a lot of FASTA files that I recently scaffolded with RagTag. The output of RagTag includes a directory for each sample and the corresponding FASTA file written as &amp;quot;ragtag.scaffold.fasta&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;I would like to go through each file and rename it like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Original Path with File name:&#xA;directoryName_ID/ragtag.scaffold.fasta&#xA;&#xA;Changed File name:&#xA;directoryName_ID/ID_ragtag.scaffold.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see, the directory name includes the ID number I would like to use to rename the fasta file with (along with an underscore to space it out). Is there a way to loop through all of the directories, using that last number after the &amp;quot;_&amp;quot; in the directory name and to use that to rename my fasta file?&lt;/p&gt;&#xA;&lt;p&gt;Here is what I have tried so far:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for filename in ./directory*/ragtag.scaffold.fasta; do &#xA;    mv filename ${./directory%*_*}_ragtag.scaffold.fasta; &#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It doesn't work at all but I know I need to use mv to rename the filename. However, when it comes to referencing the directory name the file is located in I get lost. I also have a .txt file that contains a list of IDs if that might be easier to reference. I'm still kinda new to the command line so if anyone could help that would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="298" LastEditDate="2023-01-18T12:13:31.217" LastActivityDate="2023-01-21T04:19:41.587" Title="How to rename a file based on a directory name?" Tags="|fasta|bash|command-line|ragtag|for-loop|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20386" PostTypeId="2" ParentId="20385" CreationDate="2023-01-17T21:45:54.757" Score="3" Body="&lt;p&gt;Just execute the following immediately above the directories of interest, (&lt;strong&gt;post edited&lt;/strong&gt; to include OPs &amp;quot;constant term&amp;quot;)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for d in *; do &#xA;  mv &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$d&quot;/ragtag.scaffold.fasta &#xA; &quot;$&lt;/span&gt;d/${d/000mergedCluster_/}_ragtag.scaffold.fasta&amp;quot;;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The key thing I assumed is that the directory name comprises a &amp;quot;constant term&amp;quot; and is suffixed by a &amp;quot;variable term&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Just to note the &lt;code&gt;$d&lt;/code&gt; here is directory not file name - because that is constant in each directory.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Comments...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$d/$&lt;/span&gt;{d/000mergedCluster_/}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The key part of the code is above.&lt;/p&gt;&#xA;&lt;p&gt;What its doing is saying from the variable &lt;code&gt;$d&lt;/code&gt; delete the term &lt;code&gt;000mergedCluster_&lt;/code&gt; (or whatever the &amp;quot;constant term&amp;quot; is)&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="298" LastEditDate="2023-01-18T12:14:58.090" LastActivityDate="2023-01-18T12:14:58.090" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="20387" PostTypeId="1" AcceptedAnswerId="20389" CreationDate="2023-01-17T21:51:37.940" Score="2" ViewCount="65" Body="&lt;p&gt;Background: I have a list of species that I want to put through homology inference. The goal of homology inference is to investigate the evolution of a trait on a species tree. I want to use the quantitative aspect of the data like CNV and recreate nodal CNV using cafe or count. And use gene trees of the homology to see if I can discern divergence patterns across gene families of interest.&lt;/p&gt;&#xA;&lt;p&gt;The issue is some species only have RNA-seq generated transcriptions (peptides and cDNA) while others have genomes. The issue of missing introns aside, how can I circumvent this gap?&lt;/p&gt;&#xA;&lt;p&gt;Should I:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Just use the cDNA data with genomic data? compare nucleic acid to nucleic acid. I need to eventually make gene trees as well and as such MSA is important.The list of species is huge and I anticipate seg-faults during MSA if not during homology inference itself.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use emboss or something else to convert the nucleic acid data to peptides.And then compare peptides to peptides? Which makes the homology inference → MSA (hopefully pal2nal and gene tree inference later) a little easier. I have never done translation before so I do not know the pitfalls per se.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My list of species is green plants.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone share their experience if you have done something like this before?&lt;/p&gt;&#xA;" OwnerUserId="16699" LastEditorUserId="3848" LastEditDate="2023-04-30T18:25:29.760" LastActivityDate="2023-04-30T18:25:29.760" Title="Is it possible to do homology inference across species using different kinds of NGS data?" Tags="|rna-seq|phylogenetics|phylogeny|sequence-homology|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20389" PostTypeId="2" ParentId="20387" CreationDate="2023-01-17T22:15:59.097" Score="4" Body="&lt;p&gt;Personally, I would use &lt;code&gt;blastx&lt;/code&gt; against a local protein database. This converts query cDNA-&amp;gt;amino acids and therefore combines points 1 and 2 without need to perform a translation in advance.&lt;/p&gt;&#xA;&lt;p&gt;Genetic distance and saturation might become an issue here depending whether the search is outside the family and especially Order&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;MAFFT vs Muscle Super5. I would try &amp;quot;Muscle5&amp;quot; or rather the latest Muscle under the &amp;quot;Super5&amp;quot; algorithm. It outperforms everything.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-17T22:37:20.767" LastActivityDate="2023-01-17T22:37:20.767" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20390" PostTypeId="2" ParentId="20385" CreationDate="2023-01-18T00:06:52.680" Score="3" Body="&lt;p&gt;Assuming the output is only a list of directories and each directory includes a file called &amp;quot;ragtag.scaffold.fasta&amp;quot;, you could use &lt;a href=&quot;https://www.gnu.org/software/bash/manual/html_node/Brace-Expansion.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;brace expansion&lt;/a&gt; to generate the SOURCE and DEST strings:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in directoryName_* ; do&#xA;    echo mv &amp;quot;${i}&amp;quot;/{,&amp;quot;${i#directoryName_}&amp;quot;_}ragtag.scaffold.fasta&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But the above will only work using Bash and Zsh, and maybe some other shells. Without Bash/Zsh, just use:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fasta=&amp;quot;ragtag.scaffold.fasta&amp;quot;&#xA;for i in directoryName_* ; do&#xA;    echo mv &amp;quot;${i}/${fasta}&amp;quot; &amp;quot;${i}/${i#directoryName_}_${fasta}&amp;quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Remove the &lt;code&gt;echo&lt;/code&gt; to perform the rename.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-01-18T00:06:52.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20391" PostTypeId="2" ParentId="20370" CreationDate="2023-01-18T11:21:15.047" Score="1" Body="&lt;p&gt;You don't need R or Python for this, it is a very straightforward text parsing problem and you can easily do it in an awk one-liner:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ awk 'BEGIN{FS=OFS=&amp;quot;\t&amp;quot;} &#xA;      /^[^#]/ &amp;amp;&amp;amp; $14==&amp;quot;LOW&amp;quot; &amp;amp;&amp;amp; $17=&amp;quot;likely pathogenic&amp;quot; ' file &#xA;11_224088_C/A   11:224088   A   ENSG00000142082 ENST00000525319 Transcript  missense_variant    742 716 239 S/I aGc/aTc -   LOW -   -1  likely pathogenic   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Explanation&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;BEGIN{FS=OFS=&amp;quot;\t&amp;quot;}&lt;/code&gt;: the &lt;code&gt;BEGIN{}&lt;/code&gt; block is run before processing the file. Here we are setting the field separator (&lt;code&gt;FS&lt;/code&gt;) and output field separator (&lt;code&gt;OFS&lt;/code&gt;) to a tab so that awk will split each input line on tab characters.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;/^[^#]/ &amp;amp;&amp;amp; $14==&amp;quot;LOW&amp;quot; &amp;amp;&amp;amp; $17=&amp;quot;likely pathogenic&amp;quot;&lt;/code&gt;: this will evaluate to true if the current line does not start with a &lt;code&gt;#&lt;/code&gt;, and the 14th field is &lt;code&gt;LOW&lt;/code&gt; and the 17th field is &lt;code&gt;likely pathogenic&lt;/code&gt;. In awk, the default action when something evaluates to true is to print the current line, so this results in the expected output.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Now, if you also want to add the file name as a field, you can do this to have it as the last field:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ awk 'BEGIN{FS=OFS=&amp;quot;\t&amp;quot;} /^[^#]/ &amp;amp;&amp;amp; $14==&amp;quot;LOW&amp;quot; &amp;amp;&amp;amp; $17=&amp;quot;likely pathogenic&amp;quot;{$(NF+1)=FILENAME; print} ' file &#xA;11_224088_C/A   11:224088   A   ENSG00000142082 ENST00000525319 Transcript  missense_variant    742 716 239 S/I aGc/aTc -   LOW -   -1  likely pathogenic       yourfilename&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Next, if your files are compressed and you want to do this to all of them, you could simply do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for file in *vep.txt.gz; do&#xA;  zcat &amp;quot;$file&amp;quot; | awk -v fname=&amp;quot;${file%.vep.txt.gz}&amp;quot; '&#xA;                       BEGIN{FS=OFS=&amp;quot;\t&amp;quot;} &#xA;                       /^[^#]/ &amp;amp;&amp;amp; $14==&amp;quot;LOW&amp;quot; &amp;amp;&amp;amp; $17=&amp;quot;likely pathogenic&amp;quot;{&#xA;                          $(NF+1)=FILENAME; &#xA;                          print&#xA;                       }'&#xA;done &amp;gt; allfiles.filtered&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-01-18T11:21:15.047" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="20392" PostTypeId="2" ParentId="20383" CreationDate="2023-01-18T12:18:20.373" Score="3" Body="&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/Standard_score&quot; rel=&quot;nofollow noreferrer&quot;&gt;standard score&lt;/a&gt; can be estimated by using the sample mean and sample standard deviation as estimates of the population values. Just as you wrote it in the question.&lt;/p&gt;&#xA;&lt;p&gt;The z-score is given by&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$ z = \frac{x - \bar{x}}{s}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;where:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; is the sample observation&lt;br /&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$\bar{x}$&lt;/span&gt; is the mean of the sample,&lt;br /&gt;&#xA;&lt;span class=&quot;math-container&quot;&gt;$s$&lt;/span&gt; is the standard deviation of the sample.&lt;/p&gt;&#xA;&lt;p&gt;Negative values mean that particular observation is &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; standard deviations below the mean value, and negative &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;-scores are very much expected. The process of standardization is a simple linear transformation and thus does not remove or change any degree of skewness in the original data. I guess your data is skew from the begging to values below the mean.&lt;/p&gt;&#xA;&lt;p&gt;Yes you can compute the exact &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;-values from the &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt;-scores and then apply a negative log transform. You can use a &lt;span class=&quot;math-container&quot;&gt;$z$&lt;/span&gt; pre-computed table for this. Check &lt;a href=&quot;https://www.youtube.com/watch?v=zTABmVSAtT0&quot; rel=&quot;nofollow noreferrer&quot;&gt;this video&lt;/a&gt; for an example.&lt;/p&gt;&#xA;" OwnerUserId="475" LastActivityDate="2023-01-18T12:18:20.373" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20393" PostTypeId="2" ParentId="20383" CreationDate="2023-01-18T14:07:04.417" Score="1" Body="&lt;p&gt;@JRodrigoF is the correct answer. I'll attempt to explain why the z-score is being used, because it is not immediately obvious.&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$F_{ST}$&lt;/span&gt; is not trivial for anyone to understand &lt;em&gt;per se&lt;/em&gt;. It is extremely useful for assessing migration - but what is the actual index that Ron Fisher devised?&lt;/p&gt;&#xA;&lt;p&gt;Simulations suggest that &lt;span class=&quot;math-container&quot;&gt;$F_{ST}$&lt;/span&gt; = 0.2 is threshold above which which strong population structure is considered. Thus &lt;span class=&quot;math-container&quot;&gt;$F_{ST}$&lt;/span&gt; = 0 panmixia and &lt;span class=&quot;math-container&quot;&gt;$F_{ST}$&lt;/span&gt; = 1 is zero migration. How to interpret everything in-between is difficult, except for the 0.2 threshold if the theoretical assumptions in the threshold calculation apply to real-world data. Thus the Z-score is an interesting concept.&lt;/p&gt;&#xA;&lt;p&gt;In other words, the 0.2 threshold simulation suggests &lt;span class=&quot;math-container&quot;&gt;$F_{ST}$&lt;/span&gt; is a non-linear index, which is very interesting in understanding it. Non-linear indices are a bit hairy because we all think they are linear, so 0.1 to 0.3 is one increment of migration, 0.3 to 0.5 another increment of migration. This is not true according to the simulation studies.&lt;/p&gt;&#xA;&lt;p&gt;Z-scores get around this difficult by simply standardising the combined &lt;span class=&quot;math-container&quot;&gt;$F_{ST}$&lt;/span&gt; and thats why they are using it.&lt;/p&gt;&#xA;&lt;p&gt;It becomes particularly interesting if &lt;span class=&quot;math-container&quot;&gt;$F_{ST}$&lt;/span&gt; is calculated per allele because then &amp;quot;fitness traits&amp;quot; can then be tracked. The allele doesn't necessarily need to be the actual fitness trait because of molecular &amp;quot;hitchhiking&amp;quot; (old John Maynard Smith terminology). Thus &lt;span class=&quot;math-container&quot;&gt;$F_{ST}$&lt;/span&gt; Z-score within a population within a genome is the beginning of a datamining process. So example, the domestication of dogs why it becomes such a big deal, particularly if the comparison is a dog breed that is closest to the wolf genome. The actually traits selected for in dogs can now be identified. It would also operate between genomes, to identify outliers in dog dispersal, but the behavioural (behavioral) modifications that make dogs dogs is very interesting.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The question is are super small probabilities generated to make log meaningful? I'm not sure in this instance probabilities well beyond very highly, highly significant are generated. If they are then a log transformation is useful.&lt;/p&gt;&#xA;&lt;p&gt;For background the log-scale is used for transforming very small probabilities rather than use &lt;em&gt;e&lt;/em&gt; nomenclature. It makes them more understandable for comparative purposes and allows you to do cool things like add and subtract them instead of multiplying and dividing. The smaller the probability value the higher the transformed value ... so in this case the higher the value the more interesting the result.&lt;/p&gt;&#xA;&lt;p&gt;However, if the field simply does not use log transformation then it's not cool, because no-one will be able to comparatively interpret the transformed values. In phylogenetics it's done all the time, because the probabilities are really small, but anyone can immediately understand the values - because everyone uses this system. A key part of stats is clarity, not necessarily for those outside the field, but certainly for those within the field.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-18T15:27:10.270" LastActivityDate="2023-01-18T15:27:10.270" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20394" PostTypeId="1" CreationDate="2023-01-18T15:35:45.473" Score="2" ViewCount="124" Body="&lt;p&gt;Hi I ran pyScenic's ctx via the command in command prompt:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pyscenic ctx /home/vboxuser/pyScenic/adj.csv \&#xA;    /home/vboxuser/pyScenic/mm10__refseq-r80__10kb_up_and_down_tss.mc9nr.genes_vs_motifs.rankings.feather \&#xA;    --annotations_fname /home/vboxuser/pyScenic/motifs-v9-nr.mgi-m0.001-o0.0.tbl \&#xA;    --expression_mtx_fname /home/vboxuser/pyScenic/csk_filtered_scenic.loom \&#xA;    --mode dask_multiprocessing \&#xA;    --output reg.csv \&#xA;    --mask_dropouts \&#xA;    --num_workers 20&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It gives me a bunch of warnings and then an error, as shown below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;2023-01-17 09:54:52,135 - pyscenic.cli.pyscenic - INFO - Creating modules.&#xA;&#xA;2023-01-17 09:54:52,165 - pyscenic.cli.pyscenic - INFO - Loading expression matrix.&#xA;&#xA;2023-01-17 09:54:55,458 - pyscenic.utils - INFO - Calculating Pearson correlations.&#xA;&#xA;2023-01-17 09:54:55,482 - pyscenic.utils - WARNING - Note on correlation calculation: the default behaviour for calculating the correlations has changed after pySCENIC verion 0.9.16. Previously, the default was to calculate the correlation between a TF and target gene using only cells with non-zero expression values (mask_dropouts=True). The current default is now to use all cells to match the behavior of the R verision of SCENIC. The original settings can be retained by setting 'rho_mask_dropouts=True' in the modules_from_adjacencies function, or '--mask_dropouts' from the CLI.&#xA;    Dropout masking is currently set to [True].&#xA;&#xA;2023-01-17 09:54:56,273 - pyscenic.utils - INFO - Creating modules.&#xA;/home/vboxuser/mambaforge/lib/python3.10/site-packages/pyscenic/utils.py:244: FutureWarning: Not prepending group keys to the result index of transform-like apply. In the future, the group keys will be included in the index, regardless of whether the applied function returns a like-indexed object.&#xA;To preserve the previous behavior, use&#xA;&#xA;    &amp;gt;&amp;gt;&amp;gt; .groupby(..., group_keys=False)&#xA;&#xA;To adopt the future behavior and silence this warning, use &#xA;&#xA;    &amp;gt;&amp;gt;&amp;gt; .groupby(..., group_keys=True)&#xA;  df = adjacencies.groupby(by=COLUMN_NAME_TARGET).apply(&#xA;/home/vboxuser/mambaforge/lib/python3.10/site-packages/pyscenic/utils.py:244: FutureWarning: Not prepending group keys to the result index of transform-like apply. In the future, the group keys will be included in the index, regardless of whether the applied function returns a like-indexed object.&#xA;To preserve the previous behavior, use&#xA;&#xA;    &amp;gt;&amp;gt;&amp;gt; .groupby(..., group_keys=False)&#xA;&#xA;To adopt the future behavior and silence this warning, use &#xA;&#xA;    &amp;gt;&amp;gt;&amp;gt; .groupby(..., group_keys=True)&#xA;  df = adjacencies.groupby(by=COLUMN_NAME_TARGET).apply(&#xA;/home/vboxuser/mambaforge/lib/python3.10/site-packages/pyscenic/utils.py:244: FutureWarning: Not prepending group keys to the result index of transform-like apply. In the future, the group keys will be included in the index, regardless of whether the applied function returns a like-indexed object.&#xA;To preserve the previous behavior, use&#xA;&#xA;    &amp;gt;&amp;gt;&amp;gt; .groupby(..., group_keys=False)&#xA;&#xA;To adopt the future behavior and silence this warning, use &#xA;&#xA;    &amp;gt;&amp;gt;&amp;gt; .groupby(..., group_keys=True)&#xA;  df = adjacencies.groupby(by=COLUMN_NAME_TARGET).apply(&#xA;&#xA;2023-01-17 09:54:56,350 - pyscenic.cli.pyscenic - ERROR - Not a single module loaded&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried uninstalling and reinstalling pyScenic, but it didn't change anything. It shouldn't be pyScenic overall either that is wrong, since I ran pyScenic grnboost command and that worked fine.&lt;/p&gt;&#xA;&lt;p&gt;It seems like it might be specific to the pyScenic commands. Has anyone run into the same issues?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="16791" LastActivityDate="2023-01-18T15:35:45.473" Title="pyScenic CLI for ctx is giving error: Not a single module loaded" Tags="|python|scrnaseq|single-cell|transcriptome|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20395" PostTypeId="1" CreationDate="2023-01-18T18:49:06.197" Score="1" ViewCount="58" Body="&lt;p&gt;I want to predict the folding of a chimeric antigen receptor sequence. These sequences are completely made out of small parts of different proteins. Do you know if AlphaFold can be used for this?&lt;/p&gt;&#xA;" OwnerUserId="15343" LastActivityDate="2023-01-24T19:34:57.140" Title="How can I predict the folding of a synthetic protein?" Tags="|protein-structure|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20396" PostTypeId="1" AcceptedAnswerId="20397" CreationDate="2023-01-18T21:07:47.650" Score="4" ViewCount="139" Body="&lt;p&gt;&lt;code&gt;df&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gene_name   guide_1 guide_1_new Correlation&#xA;MMP-1   A   A   1&#xA;MMP-1   A   B   0.426115&#xA;MMP-1   A   C   0.522499&#xA;MMP-1   A   D   0.431587&#xA;MMP-1   B   A   0.426115&#xA;MMP-1   B   B   1&#xA;MMP-1   B   C   0.60113&#xA;MMP-1   B   D   0.534858&#xA;MMP-1   C   A   0.522499&#xA;MMP-1   C   B   0.60113&#xA;MMP-1   C   C   1&#xA;MMP-1   C   D   0.622206&#xA;MMP-1   D   A   0.431587&#xA;MMP-1   D   B   0.534858&#xA;MMP-1   D   C   0.622206&#xA;MMP-1   D   D   1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I used four guides in an experiment (data = &lt;code&gt;df&lt;/code&gt;). I calculated correlation between them based on the their log fold changes. I want to check which guide is outperforming or underperforming in comparison to other three guides in my experiment. I tried Kruskal and Wallis test to do this analysis with the script given below. However, I got an error &lt;code&gt;ValueError: Samples must be one-dimensional&lt;/code&gt;. Also is this the right test to do this analysis?&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;My script&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from scipy import stats&#xA;&#xA;x = df[['guide_1']]&#xA;y = df[['guide_1_new']]&#xA;z = df[['Correlation']]&#xA;stats.kruskal(x, y, z)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3766" LastActivityDate="2023-01-19T02:08:56.290" Title="How to fix the Error in Kruskal Wallis rank sum test for comparing the guides in python? Also to check whether using the right test for my data?" Tags="|statistics|pandas|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20397" PostTypeId="2" ParentId="20396" CreationDate="2023-01-18T22:56:15.793" Score="3" Body="&lt;p&gt;Reformatting the data for the KW and doing the test is easy.&lt;/p&gt;&#xA;&lt;p&gt;Using &amp;quot;guide_1&amp;quot; alone and pivot() to get A = [1, 0.426115 ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df2 = df.pivot(columns='guide_1', values='Correlation')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will now present A, B, C, D etc ... into columns...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;x = df2[['A']].dropna() &#xA;y = df2[['B']].dropna() &#xA;z = df2[['C']].dropna() &#xA;a = df2[['D']].dropna() &#xA;&#xA;stats.kruskal(x, y, z, a)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That will work ... whether that is a good test is a complicated question because this is log transformed data. Transformation is used in parametric tests. Removing the log transformation is strongly recommended because KW works with raw data.&lt;/p&gt;&#xA;&lt;p&gt;You can switch the calculation to &lt;code&gt;guide_1_new&lt;/code&gt; instead. It will give a different answer ... but I don't know the experiment. The original guide looks cool.&lt;/p&gt;&#xA;&lt;p&gt;Looking at the data - the test should work because I believe KW is 'unpaired' (you need to check) and looking at the value the distribution looks pretty similar between A, B, C and D. However, just because the KW says 'yes' doesn't mean thats a robust answer (log data). If the untransformed data says 'yes' ... thats a robust answer for KW. Just check the 'unpaired' stuff - thats important for the result.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Personal note, I originally used a third df but was not needed,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df3 = pd.concat([df2[col].sort_values().reset_index(drop=True) for col in df2], axis=1, ignore_index=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-19T02:08:56.290" LastActivityDate="2023-01-19T02:08:56.290" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20398" PostTypeId="2" ParentId="20385" CreationDate="2023-01-19T04:16:59.123" Score="1" Body="&lt;p&gt;If you want to use Perl:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-pl prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/perl&#xA;use 5.030;&#xA;use warnings;&#xA;&#xA;use File::Find::Rule qw( );&#xA;&#xA;for my &lt;span class=&quot;math-container&quot;&gt;$file (File::Find::Rule-&amp;gt;name(&quot;ragtag.scaffold.fasta&quot;)-&amp;gt;in(&quot;.&quot;)){&#xA;  my $newfile = $file;&#xA;  $newfile =~ s{(.+_(\d+)/)(.+.fasta)}{$1$2_$&lt;/span&gt;3};&#xA;&#xA;  if (-e &lt;span class=&quot;math-container&quot;&gt;$newfile) {&#xA;    warn &quot;can't rename $file to $newfile: $newfile exists\n&quot;;&#xA;  }elsif (rename $&lt;/span&gt;file, &lt;span class=&quot;math-container&quot;&gt;$newfile) {&#xA;    # do nothing&#xA;  }else{&#xA;    warn &quot;rename $file to $newfile failed: $&lt;/span&gt;!\n&amp;quot;;&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14180" LastEditorUserId="14180" LastEditDate="2023-01-19T04:35:12.333" LastActivityDate="2023-01-19T04:35:12.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20400" PostTypeId="2" ParentId="20381" CreationDate="2023-01-20T14:16:36.867" Score="1" Body="&lt;p&gt;Sorry I am really late answering, the &lt;code&gt;set_caption&lt;/code&gt; was cool.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; What we are talking about here is the &lt;code&gt;style&lt;/code&gt; Class in &lt;code&gt;pandas&lt;/code&gt; of which &lt;code&gt;set_background&lt;/code&gt; is one method. Usually, it is for outputting a &lt;code&gt;pandas&lt;/code&gt; dataframe as a webpage. If I was front-end developer it would be cool. In this example it is used to colour the different bits of the dataframe, presumably to make it easier to read a lot of data.&lt;/p&gt;&#xA;&lt;p&gt;The thing I would have done is dump to Excel. This will preserve the colouration. The bits contained within &lt;code&gt;#&lt;/code&gt; are my codes.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Pathlib import Path&#xA;import pandas as pd&#xA;&#xA;##########&#xA;path = Path('/path/todir/')&#xA;##########&#xA;for x, group in df.groupby('diabetes_group'):&#xA;    matrix = group[[&amp;quot;diabetes_group&amp;quot;, 'DM-1', 'DM-2', 'corr']]&#xA;    matrix_pivot = pd.pivot_table(matrix, values = 'corr', index=['DM-1'], columns=['DM-2'])&#xA;    d = matrix_pivot.style.background_gradient(cmap='Set3').set_caption(x)&#xA;&#xA;    ##########&#xA;    pathout = Path(path, x + '.xlsx')&#xA;    d.to_excel(pathout)  &#xA;    ##########&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What was happening originally was the 'loop' was overriding the diabetes &lt;code&gt;Group&lt;/code&gt; in previous loop, hence appeared to be disappearing. What will happen now is an excel file will be produced separately for each group and the file name labeled as &lt;code&gt;Group1.xlsx&lt;/code&gt;, &lt;code&gt;Group2.xlsx&lt;/code&gt; etc ...&lt;/p&gt;&#xA;&lt;p&gt;Here I dumped my test output to HTML simply via &lt;code&gt;to_html&lt;/code&gt; and then to pdf.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zdPyp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zdPyp.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt;&#xA;The variable &lt;code&gt;corr_matrix = []&lt;/code&gt; does not take part in this code. It may be important to other bits of code. I'm not sure &lt;code&gt;matrix&lt;/code&gt; variable does much, but I've not checked.&lt;/p&gt;&#xA;&lt;p&gt;The issue with IPython &lt;code&gt;display&lt;/code&gt; is the output is not saved, so the code needs to be run each time the analyst wants to examine the correlation coefficients between &lt;code&gt;DM-1&lt;/code&gt; and &lt;code&gt;DM-2&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;My personal dataframe was&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df = pd.DataFrame({'diabetes_group':['Group1','Group1','Group2','Group2', 'Group3','Group3'],'DM-1':[56,56,56,56,56,56], 'DM-2':[56,58,61,63,56,58], 'corr':[0.2,0.3,0.5,0.3,0.2,0.1]})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-20T14:35:37.787" LastActivityDate="2023-01-20T14:35:37.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20401" PostTypeId="1" CreationDate="2023-01-20T17:29:27.757" Score="0" ViewCount="102" Body="&lt;p&gt;I have the peptide sequences and fasta files separately. I first aligned the fasta files using &lt;code&gt;msa&lt;/code&gt; package. After that I'm trying to highlight the peptide sequences in the multiple sequence alignment output. I couldn't find a way to do that. Any suggestions on how to do this would be very helpful.&lt;/p&gt;&#xA;&lt;p&gt;My R script so far:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# load required packages&#xA;library(seqinr)&#xA;library(msa)&#xA;library(dplyr)&#xA;&#xA;# read the file containing the list of peptide sequences&#xA;peptides &amp;lt;- read.table(&amp;quot;./peptides/AB_hydrolase-1_domain-containing_protein.txt&amp;quot;, header=F, stringsAsFactors=F)&#xA;&#xA;# read the file containing the list of fasta files&#xA;#hemoSeq &amp;lt;- readAAStringSet(system.file(&amp;quot;examples/HemoglobinAA.fasta&amp;quot;,package=&amp;quot;msa&amp;quot;))&#xA;fasta_file &amp;lt;- &amp;quot;./fasta/AB_hydrolase-1_domain-containing_protein.txt&amp;quot;&#xA;fasta_data &amp;lt;- readAAStringSet(fasta_file)&#xA;&#xA;# multiple sequence alignment with ClustalW algorithm&#xA;msa &amp;lt;- msa(fasta_data, method = &amp;quot;ClustalW&amp;quot;)&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14290" LastEditorUserId="650" LastEditDate="2023-01-20T23:25:33.127" LastActivityDate="2023-01-20T23:25:33.127" Title="How to highlight the specific peptide sequences after performing multiple sequence alignment for the fasta file?" Tags="|r|bioconductor|multiple-sequence-alignment|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20402" PostTypeId="1" AcceptedAnswerId="20418" CreationDate="2023-01-20T20:40:12.030" Score="1" ViewCount="192" Body="&lt;p&gt;I have a directory of FASTA files, some of which are empty (placeholders that were output from a pipeline) and I would like to delete the empty files using a for loop on the command line.&lt;/p&gt;&#xA;&lt;p&gt;From what I have seen so far I can use the &lt;code&gt;wc -l [file]&lt;/code&gt; approach to see &amp;quot;if there are lines present, delete the file&amp;quot; but I want to make sure the syntax is right before deleting important files. I was thinking it might be easier for the command to look for the &amp;quot; &amp;gt; &amp;quot; symbol found at the header in FASTA files and if it's not present delete the file.&lt;/p&gt;&#xA;&lt;p&gt;Here is what I have so far:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for file in *.fasta; do &amp;quot;if wc -l &amp;lt; 1 rm file&amp;quot;; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for file in *.fasta; do &amp;quot;grep &amp;gt; | rm file&amp;quot;; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16067" LastActivityDate="2023-01-24T16:09:48.353" Title="How to remove empty files from a directory?" Tags="|fasta|bash|grep|command-line|" AnswerCount="4" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20403" PostTypeId="2" ParentId="20402" CreationDate="2023-01-20T20:44:31.390" Score="3" Body="&lt;p&gt;Always Google before asking on forums. I googled &amp;quot;delete empty files linux&amp;quot; and this was the second result: &lt;a href=&quot;https://www.baeldung.com/linux/delete-empty-files-dirs&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.baeldung.com/linux/delete-empty-files-dirs&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Print and delete empty files just in the current directory&#xA;$ find . -maxdepth 1 -type f -empty -print -delete&#xA;# Print and delete empty files in the current directory and all sub-directories&#xA;$ find . -type f -empty -print -delete&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-01-20T20:44:31.390" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="20404" PostTypeId="2" ParentId="20402" CreationDate="2023-01-20T22:28:03.020" Score="1" Body="&lt;p&gt;@Ram RS has identified the solution here via &lt;code&gt;find&lt;/code&gt;, if that does not work then the following will work,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;find . -maxdepth 1 -type f -empty | xargs rm &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit 2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The solution is obvious ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;find . -type l -delete&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The code under &lt;strong&gt;Edit&lt;/strong&gt; is is better if the OP knows in advance &lt;em&gt;precisely&lt;/em&gt; how many links the symbolic link file has, basically its always one. However, we don't know, thus &lt;code&gt;-type l&lt;/code&gt; will delete every symbolic link.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To address @Steve's point of inadvertent deletion in case of a null return ... The code was checked prior release and this didn't occur (and also now rechecked).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="298" LastEditDate="2023-01-23T14:53:57.047" LastActivityDate="2023-01-23T14:53:57.047" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="20405" PostTypeId="2" ParentId="20401" CreationDate="2023-01-20T22:38:53.223" Score="1" Body="&lt;p&gt;Checkout an MSA visualisation and analysis package that you can load the alignment into. It may not be the newest tool, but I've used Jalview (&lt;a href=&quot;https://www.jalview.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.jalview.org/&lt;/a&gt;) in the past&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2023-01-20T22:38:53.223" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20406" PostTypeId="2" ParentId="20385" CreationDate="2023-01-21T04:19:41.587" Score="1" Body="&lt;p&gt;Perl one-liner:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;ls */filename | perl -lane 'print &amp;quot;mv $1/$3 $1/$2_$3&amp;quot; if /(\S+_([^\/_\s]+))\/(\S+)/'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37" LastActivityDate="2023-01-21T04:19:41.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20407" PostTypeId="2" ParentId="20402" CreationDate="2023-01-21T11:24:35.630" Score="2" Body="&lt;p&gt;As already stated, one way is to use the GNU &lt;code&gt;find&lt;/code&gt; (the default on Linux) command. This will find and print all empty files in the current directory:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;find . -maxdepth 1 -type f -empty -print&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Append the &lt;code&gt;-delete&lt;/code&gt; action to the command above when you're ready to remove them. If you need to find (and then delete) empty files recursively, simply remove the &lt;code&gt;-maxdepth 1&lt;/code&gt; option.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;In a comment, you mention that these are actually &amp;quot;alias&amp;quot; files which are the results of a &lt;a href=&quot;https://www.nextflow.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nextflow&lt;/a&gt; pipeline. Unless the default &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#publishdir&quot; rel=&quot;nofollow noreferrer&quot;&gt;publishDir&lt;/a&gt; mode has been changed, Nextflow will publish output files to the publishDir using symbolic links to the files in the working directory (i.e. &lt;code&gt;./work&lt;/code&gt;). I suspect what you are trying to do is to find and remove the links to these empty files, but not the files themselves. In this case, I think what you want is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;find . -type l -exec test ! -s '{}' \; -print&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Again, append the &lt;code&gt;-delete&lt;/code&gt; action to the command above to actually remove the symlinks. If you instead need to delete files from Nextflow's working directory, use &lt;code&gt;nextflow clean&lt;/code&gt; in conjunction with &lt;code&gt;nextflow log&lt;/code&gt; to safely clean up the failed runs.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="298" LastEditDate="2023-01-23T14:57:27.600" LastActivityDate="2023-01-23T14:57:27.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20408" PostTypeId="1" CreationDate="2023-01-21T18:51:49.777" Score="3" ViewCount="1669" Body="&lt;p&gt;&lt;strong&gt;Update:&lt;/strong&gt;&#xA;Biopython document says that &amp;quot;Bio.SeqIO.FastaIO.FastaWriter&amp;quot; class is obsolete. Now my question becomes how to I use &lt;code&gt;Bio.SeqIO.write()&lt;/code&gt; fucntion to achive the same thing I asked.&lt;/p&gt;&#xA;&lt;p&gt;I use the following code to extract specific FASTA records from a file based on a list of FASTA IDs given in another file. The script works just fine. However, I want to able to write the results using &amp;quot;Bio.SeqIO.FastaIO.FastaWriter&amp;quot; class into a file instead of Python's print or write function. How can I do that in my last if statement?&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;from Bio import SeqIO&#xA;#import pprint&#xA;import textwrap&#xA;&#xA;my_id_file     = open('sample_files2/fasta_id_records.txt','r')&#xA;my_fasta_file  = open('sample_files2/fasta_file.fasta','r')&#xA;&#xA;my_dictionary = {} # fasta IDs are keys, value can be anything.&#xA;for line in my_id_file:&#xA;    my_dictionary[line[:-1]] = 'value'&#xA;#pprint.pprint(my_dictionary)&#xA;&#xA;for seq_record in SeqIO.parse(my_fasta_file, &amp;quot;fasta&amp;quot;):&#xA;    if seq_record.id in my_dictionary:&#xA;        sequence = str(seq_record.seq)&#xA;        fasta_record = textwrap.fill(sequence, width=60)&#xA;        print(f&amp;quot;&amp;gt;{seq_record.id}\n{fasta_record}&amp;quot;)&#xA;&#xA;my_fasta_file.close()&#xA;my_id_file.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14180" LastEditorUserId="298" LastEditDate="2023-01-24T10:42:09.767" LastActivityDate="2023-01-24T10:42:09.767" Title="How to write FASTA records using &quot;Bio.SeqIO.write()&quot;" Tags="|python|fasta|biopython|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20409" PostTypeId="2" ParentId="20408" CreationDate="2023-01-21T19:20:38.793" Score="3" Body="&lt;p&gt;I figured it out:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;from Bio import SeqIO&#xA;#import pprint&#xA;import textwrap&#xA;&#xA;my_id_file     = open('sample_files2/fasta_id_records.txt','r')&#xA;my_fasta_file  = open('sample_files2/fasta_file.fasta','r')&#xA;result_file    = open('sample_files2/results.fasta', &amp;quot;w&amp;quot;)&#xA;&#xA;my_dictionary = {} # fasta IDs are keys, value can be anything.&#xA;for line in my_id_file:&#xA;    my_dictionary[line[:-1]] = 'value'&#xA;&#xA;#pprint.pprint(my_dictionary)&#xA;for seq_record in SeqIO.parse(my_fasta_file, &amp;quot;fasta&amp;quot;):&#xA;    if seq_record.id in my_dictionary:      &#xA;        #use this line to write standard fasta record using SeqIO.write&#xA;        SeqIO.write(seq_record, result_file, &amp;quot;fasta&amp;quot;)&#xA;&#xA;my_fasta_file.close()&#xA;my_id_file.close()&#xA;result_file.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2023-01-21T19:20:38.793" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20410" PostTypeId="2" ParentId="20408" CreationDate="2023-01-22T12:21:28.767" Score="2" Body="&lt;p&gt;You could instead pass in a generator that yields &lt;a href=&quot;https://biopython.org/wiki/SeqRecord&quot; rel=&quot;nofollow noreferrer&quot;&gt;SeqRecords&lt;/a&gt;. This avoids storing the entire list of  sequences in memory and is usually the way to go assuming that you need to handle large files containing thousands of records. This example use a generator expression, which is a bit like a list comprehension, but instead of creating a list, it creates a generator:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import argparse&#xA;import contextlib&#xA;import pathlib&#xA;import sys&#xA;&#xA;from Bio import SeqIO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;def get_argument_parser():&#xA;&#xA;    parser = argparse.ArgumentParser()&#xA;&#xA;    parser.add_argument(&#xA;        &amp;quot;fasta_file&amp;quot;,&#xA;        type=pathlib.Path,&#xA;        metavar='FILE',&#xA;        help=&amp;quot;Extract sequence records from FILE&amp;quot;,&#xA;    )&#xA;    parser.add_argument(&#xA;        &amp;quot;ids_file&amp;quot;,&#xA;        type=pathlib.Path,&#xA;        metavar='FILE',&#xA;        help=&amp;quot;Sequence record IDs/names to extract, one per line&amp;quot;,&#xA;    )&#xA;    parser.add_argument(&#xA;        &amp;quot;-o&amp;quot;,&#xA;        &amp;quot;--output&amp;quot;,&#xA;        type=pathlib.Path,&#xA;        metavar='FILE',&#xA;        default=&amp;quot;-&amp;quot;,&#xA;        help=&amp;quot;Write extracted sequences to FILE (default: stdout)&amp;quot;,&#xA;    )&#xA;    parser.add_argument(&#xA;        &amp;quot;-f&amp;quot;,&#xA;        &amp;quot;--force&amp;quot;,&#xA;        action='store_true',&#xA;        help=&amp;quot;Overwrite the output file if it exists&amp;quot;,&#xA;    )&#xA;&#xA;    return parser&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;def main():&#xA;&#xA;    parser = get_argument_parser()&#xA;    args = parser.parse_args()&#xA;&#xA;    with contextlib.ExitStack() as stack:&#xA;&#xA;        ids_file = stack.enter_context(open(args.ids_file))&#xA;        fasta_file = stack.enter_context(open(args.fasta_file))&#xA;&#xA;        if args.output != '-':&#xA;            out = stack.enter_context(open(args.output, 'w' if args.force else 'x'))&#xA;        else:&#xA;            out = sys.stdout&#xA;&#xA;        record_ids = {line.rstrip('\n') for line in ids_file}&#xA;&#xA;        fasta = SeqIO.parse(fasta_file, &amp;quot;fasta&amp;quot;)&#xA;        records = (rec for rec in fasta if rec.id in record_ids)&#xA;&#xA;        SeqIO.write(records, out, &amp;quot;fasta&amp;quot;)&#xA;&#xA;if __name__ == &amp;quot;__main__&amp;quot;:&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10049" LastEditDate="2023-01-22T13:02:31.890" LastActivityDate="2023-01-22T13:02:31.890" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20411" PostTypeId="2" ParentId="20408" CreationDate="2023-01-22T12:53:39.803" Score="2" Body="&lt;p&gt;my attempt at changing the way Biopython formats fasta file sequence width,&#xA;read into code to figure out standard behaviour:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO, __version__&#xA;#import pprint&#xA;# import textwrap&#xA;&#xA;print('Biopython version : ' , __version__)&#xA;&#xA;# my_id_file     = open('sample_files2/fasta_id_records.txt','r')&#xA;# my_fasta_file  = open('sample_files2/fasta_file.fasta','r')&#xA;#result_file    = open('sample_files2/results.fasta', &amp;quot;w&amp;quot;)&#xA;### CHANGED TO:&#xA;    &#xA;my_id_file     = open('fasta_id_records.txt','r')&#xA;my_fasta_file  = open('fasta_file.fasta','r')&#xA;result_file    = open('results.fasta', &amp;quot;w&amp;quot;)&#xA;&#xA;&#xA;my_dictionary = {} # fasta IDs are keys, value can be anything.&#xA;for line in my_id_file:&#xA;    my_dictionary[line[:-1]] = 'value'&#xA;&#xA;#expected lenght of sequences in my fasta&#xA;length = 10&#xA;&#xA;#pprint.pprint(my_dictionary)&#xA;for seq_record in SeqIO.parse(my_fasta_file, &amp;quot;fasta&amp;quot;):&#xA;    if seq_record.id in my_dictionary:      &#xA;        &#xA;        &#xA;        ########################################################################################&#xA;        #use this line to write standard fasta record using SeqIO.write&#xA;        &#xA;        # SeqIO.write(seq_record, result_file, &amp;quot;fasta-2line&amp;quot;) # writes fasta in 2 line format&#xA;        &#xA;        # SeqIO.write(seq_record, result_file, &amp;quot;fasta&amp;quot;) # writes fasta with width of 60 as standard biopython format&#xA;        &#xA;        ### NOT POSSIBLE !!!!!!!!!!!!&#xA;        # seq_record.seq = textwrap.fill(seq_record.seq , width=10) ### --&amp;gt; AttributeError: 'Seq' object has no attribute 'expandtabs'&#xA;        &#xA;        # print(seq_record.seq, type(seq_record.seq), dir(seq_record.seq))&#xA;        &#xA;        sequence = str(seq_record.seq)&#xA;        &#xA;        result_file.write('&amp;gt;'+seq_record.id+'\n')&#xA;        &#xA;        while len(sequence) &amp;gt; 0:&#xA;            result_file.write(sequence[:length]+'\n') &#xA;            sequence = sequence[length:]&#xA;        result_file.write('\n')&#xA;        &#xA;        &#xA;        ### if you try to add EOL inside seq_record.seq of course you get &#xA;        ### TypeError: SeqRecord (id=seq1) has an invalid sequence.&#xA;        # SeqIO.write(seq_record, result_file, &amp;quot;fasta&amp;quot;) # writes fasta with sequence width as 60   !!!! TypeError: SeqRecord (id=seq1) has an invalid sequence.&#xA;&#xA;my_fasta_file.close()&#xA;my_id_file.close()&#xA;result_file.close()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;input files :&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;fasta_id_records.txt&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seq1&#xA;seq2&#xA;seq3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt; fasta_file.fasta&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;seq0&#xA;AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA&#xA;&amp;gt;seq1&#xA;TTTT&#xA;TTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTT&#xA;TTTTTTTTTTTTTTT&#xA;&amp;gt;seq3&#xA;CCCCCCCCCCCCCCCCCGGGGGGGGGGGGGGGGGGGGGGGGGGGGCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCGGGGGGGG&#xA;GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG&#xA;GGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGGG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output file:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;v &lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;seq1&#xA;TTTTTTTTTT&#xA;TTTTTTTTTT&#xA;TTTTTTTTTT&#xA;TTTTTTTTTT&#xA;TTTTTTTTTT&#xA;TTTTTTTTTT&#xA;TTTTTTTTTT&#xA;TTTTTTTTTT&#xA;TTT&#xA;&#xA;&amp;gt;seq3&#xA;CCCCCCCCCC&#xA;CCCCCCCGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGCCCCC&#xA;CCCCCCCCCC&#xA;CCCCCCCCCC&#xA;CCCCCCCCCC&#xA;CCCCCCCCCG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;GGGGGGGGGG&#xA;G&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2023-01-22T13:01:45.073" LastActivityDate="2023-01-22T13:01:45.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20412" PostTypeId="1" AcceptedAnswerId="20413" CreationDate="2023-01-22T16:34:09.910" Score="1" ViewCount="58" Body="&lt;p&gt;I am using the below script to get the histogram plot for all values together. However, I want a single plot but different colours for different groups (A, B, C)&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;data file&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Group_name  code-1    code-2    Expression&#xA;GroupA  6  6  1.21&#xA;GroupA  6  8  0.611&#xA;GroupA  6  1  0.448&#xA;GroupA  6  3  0.3587&#xA;GroupB  8  6  0.611&#xA;GroupB  8  8  2.11&#xA;GroupB  8  1  0.60113&#xA;GroupB  8  3  0.534858&#xA;GroupC  1  6  0.448&#xA;GroupC  1  8  0.60113&#xA;GroupC  1  1  2.2&#xA;GroupC  1  3  0.622206&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;Script&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;import seaborn as sns&#xA;plt.rcParams[&amp;quot;figure.figsize&amp;quot;] = (12,8)&#xA;&#xA;g1 = sns.histplot(df.Expression)&#xA;g1.set(&#xA;    xlim=(-2,2),&#xA;    title=&amp;quot;Expression&amp;quot;&#xA;)&#xA;g1.axvline(0.0, c='black', linestyle=&amp;quot;--&amp;quot;)&#xA;g1.axvline(df.Expression.median(), c='red', linestyle=&amp;quot;--&amp;quot;)&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10684" LastActivityDate="2023-01-22T18:07:17.613" Title="How to get single histogram plot for all groups in different colours?" Tags="|python|visualization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20413" PostTypeId="2" ParentId="20412" CreationDate="2023-01-22T18:07:17.613" Score="1" Body="&lt;p&gt;Just change your &lt;code&gt;g1 = sns.histplot(df.Expression)&lt;/code&gt;&#xA;to &lt;code&gt;g1 = sns.histplot(data=df, x=&amp;quot;Expression&amp;quot;, hue=&amp;quot;Group_name&amp;quot;)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here's the full code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;import seaborn as sns&#xA;import pandas as pd&#xA;&#xA;plt.rcParams[&amp;quot;figure.figsize&amp;quot;] = (12,8)&#xA;&#xA;df = pd.DataFrame({&#xA;    &amp;quot;Group_name&amp;quot;: [&amp;quot;GroupA&amp;quot;, &amp;quot;GroupA&amp;quot;, &amp;quot;GroupA&amp;quot;, &amp;quot;GroupA&amp;quot;, &amp;quot;GroupB&amp;quot;, &amp;quot;GroupB&amp;quot;, &amp;quot;GroupB&amp;quot;, &amp;quot;GroupB&amp;quot;, &amp;quot;GroupC&amp;quot;, &amp;quot;GroupC&amp;quot;, &amp;quot;GroupC&amp;quot;, &amp;quot;GroupC&amp;quot;],&#xA;    &amp;quot;code-1&amp;quot;: [6, 6, 6, 6, 8, 8, 8, 8, 1, 1, 1, 1],&#xA;    &amp;quot;code-2&amp;quot;: [6, 8, 1, 3, 6, 8, 1, 3, 6, 8, 1, 3],&#xA;    &amp;quot;Expression&amp;quot;: [1.21, 0.611, 0.448, 0.3587, 0.611, 2.11, 0.60113, 0.534858, 0.448, 0.60113, 2.2, 0.622206]&#xA;})&#xA; &#xA;g1 = sns.histplot(data=df, x=&amp;quot;Expression&amp;quot;, hue=&amp;quot;Group_name&amp;quot;)&#xA;g1.set(&#xA;    xlim=(-2,2),&#xA;    title=&amp;quot;Expression&amp;quot;&#xA;)&#xA;g1.axvline(0.0, c='black', linestyle=&amp;quot;--&amp;quot;)&#xA;g1.axvline(df.Expression.median(), c='red', linestyle=&amp;quot;--&amp;quot;)&#xA;&#xA;g1.figure.savefig(&amp;quot;hist.png&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;'''&lt;a href=&quot;https://i.stack.imgur.com/1snf8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1snf8.png&quot; alt=&quot;New output&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16575" LastActivityDate="2023-01-22T18:07:17.613" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20414" PostTypeId="1" CreationDate="2023-01-22T23:16:25.763" Score="0" ViewCount="159" Body="&lt;p&gt;I want to manually fold and pair two sequences (U4 and U6). And I want to make niceish figures.&lt;/p&gt;&#xA;&lt;p&gt;I haven't found a single program that does that.&lt;/p&gt;&#xA;&lt;p&gt;thanks&lt;/p&gt;&#xA;" OwnerUserId="16811" LastActivityDate="2023-01-23T04:14:53.973" Title="Is there a tool to visualize RNA secondary structure?" Tags="|visualization|rna|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20415" PostTypeId="2" ParentId="20414" CreationDate="2023-01-22T23:58:26.600" Score="1" Body="&lt;p&gt;The RNA viewer that is particularly good is &lt;a href=&quot;http://varna.lri.fr&quot; rel=&quot;nofollow noreferrer&quot;&gt;VA RNA&lt;/a&gt;. This stands for &amp;quot;Visualisation Applet for RNA&amp;quot;. Its a java app and run it via,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar /Route2Progam/vaRNA/VARNAv3-93-API.jar&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Where &lt;code&gt;Route2Progam&lt;/code&gt; is the path to where the jar files is stored, all other info in the run code is literal.&lt;/p&gt;&#xA;&lt;p&gt;Very nice pictures, point and clicky, drag structures into different shapes and easy to colour ... so for a GUI its great.&lt;/p&gt;&#xA;&lt;p&gt;They recommend &lt;a href=&quot;http://wilab.inha.ac.kr/pseudoviewer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PsuedoViewer&lt;/a&gt; for psuedoknots, but if that doesn't mean much ... it doesn't matter.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-23T04:14:53.973" LastActivityDate="2023-01-23T04:14:53.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20416" PostTypeId="2" ParentId="20414" CreationDate="2023-01-23T03:33:00.400" Score="1" Body="&lt;p&gt;You could try mxfold2.&#xA;&lt;a href=&quot;https://github.com/mxfold/mxfold2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mxfold/mxfold2&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;% mxfold2 predict test.fa&#xA;&amp;gt;DS4440&#xA;GGAUGGAUGUCUGAGCGGUUGAAAGAGUCGGUCUUGAAAACCGAAGUAUUGAUAGGAAUACCGGGGGUUCGAAUCCCUCUCCAUCCG&#xA;(((((((........(((((..((((.....))))...)))))...................(((((.......)))))))))))). (24.8)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2023-01-23T03:33:00.400" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20418" PostTypeId="2" ParentId="20402" CreationDate="2023-01-23T15:00:59.940" Score="2" Body="&lt;p&gt;This isn't the most efficient approach, using &lt;code&gt;find&lt;/code&gt; will be easier and faster as long as your &lt;code&gt;find&lt;/code&gt; supports the &lt;code&gt;-empty&lt;/code&gt; option, but since this is what you asked for, here is how to do it based on the number of lines:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;for file in *.fasta; do &#xA; [[ $(wc -l &amp;lt; &amp;quot;$file&amp;quot;) -eq 0 ]] &amp;amp;&amp;amp; echo rm -- &amp;quot;$file&amp;quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you want to recursively descend into subdirectories, use:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;shopt -s globstar&#xA;for file in  *.fasta */**.fasta; do.&#xA;  [[ $(wc -l &amp;lt; &amp;quot;$file&amp;quot;) -eq 0 ]] &amp;amp;&amp;amp; echo rm -- &amp;quot;$file&amp;quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If that prints the right files, remove the &lt;code&gt;echo&lt;/code&gt; to actually delete them.&lt;/p&gt;&#xA;&lt;p&gt;Note that this will also delete files that are not empty but don't have a newline character. These are not valid &amp;quot;text files&amp;quot;, but strictly speaking they aren't empty, they just have no &lt;em&gt;lines&lt;/em&gt; as lines are defined by the &lt;code&gt;\n&lt;/code&gt; character. For example, this file would be deleted:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;printf 'aaa' &amp;gt; file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2023-01-24T16:09:48.353" LastActivityDate="2023-01-24T16:09:48.353" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20419" PostTypeId="1" CreationDate="2023-01-23T15:43:01.987" Score="0" ViewCount="109" Body="&lt;p&gt;I have sent two sets (two batches in matter of sending for sequencing) of different samples (plasma) to small RNA-seq to Qiagen company&lt;/p&gt;&#xA;&lt;p&gt;This is how my meta data look&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(meta)&#xA;           batch group    sex&#xA;30678.009 batch2     C   Male&#xA;30678.010 batch2     C Female&#xA;30941.001 batch1     C Female&#xA;30941.002 batch1     C   Male&#xA;30941.003 batch1     C   Male&#xA;30941.004 batch1     C   Male&#xA;&amp;gt; tail(meta)&#xA;           batch group    sex&#xA;30941.016 batch1    LT Female&#xA;30941.017 batch1    LT   Male&#xA;30941.019 batch1    LT   Male&#xA;30941.022 batch1    LT Female&#xA;30941.023 batch1    LT   Male&#xA;30941.024 batch1    LT Female&#xA;&amp;gt;   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to extract differentially expressed RNAs between my two groups without batch effect&lt;/p&gt;&#xA;&lt;p&gt;I am doing this&lt;/p&gt;&#xA;&lt;p&gt;Is this correct please?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData = count,&#xA;                              colData = meta,&#xA;                              design= ~ batch + group)&#xA;&#xA;dds &amp;lt;- DESeq(dds, test=&amp;quot;LRT&amp;quot;, reduced=~batch)&#xA;&#xA;res &amp;lt;- results(dds)&#xA;resultsNames(dds) &#xA;&#xA;[1] &amp;quot;Intercept&amp;quot;              &amp;quot;batch_batch2_vs_batch1&amp;quot; &amp;quot;group_LT_vs_C&amp;quot;         &#xA;&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2023-01-25T15:28:13.603" LastActivityDate="2023-01-25T15:28:13.603" Title="Batch correction in differential expression analysis" Tags="|r|deseq2|differential-expression|edger|batch-effects|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20421" PostTypeId="2" ParentId="20419" CreationDate="2023-01-23T19:37:53.713" Score="2" Body="&lt;p&gt;If both batches contain replicates of both groups (LT and C) then the presented code is what you should do, exactly pointed out in the vignette for the LRT: &lt;a href=&quot;http://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#likelihood-ratio-test&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://bioconductor.org/packages/release/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#likelihood-ratio-test&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-01-23T19:37:53.713" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20422" PostTypeId="1" CreationDate="2023-01-24T04:27:27.013" Score="2" ViewCount="56" Body="&lt;p&gt;The cell subpopulation that I am interested in only accounts for around 1.2% of the total cells. I have previous FACS experiments that sort out the subpopulation from the samples (using markers CD166+ CD49fhi CD104- Lin-, see images). May I ask if there are any ways to identify this cell population from single-cell RNA-seq data of the tumor epithelial samples using the marker information?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Eq7Id.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Eq7Id.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15643" LastActivityDate="2023-01-24T09:08:15.107" Title="How to identify a low proportion cell subpopulation in the single-cell RNA-seq data?" Tags="|scrnaseq|single-cell|clustering|cancer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20424" PostTypeId="2" ParentId="20422" CreationDate="2023-01-24T09:08:15.107" Score="2" Body="&lt;p&gt;Run the usual steps including clustering and visualization via something like UMAP and then color the UMAP by these four markers. That assumes that the surface marker separation you see in flow holds true on a transcriptional level which is not necessarily the case. For example, hematopoietic surface-defined populations such as the Granulocyte-Macrophage Progenitor (GMP) are well-separated in flow but on transcriptional single-cell cell level span many clusters.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, check the tool &lt;code&gt;UCell&lt;/code&gt; which can calculate a score per cell based on a combination of positive and negative markers. You can then color the UMAP by this to see whether a single cluster stands out or whether this celltype contributes to many clusters ir is not even present at all. Include some marker combinations that you know are oresent and some nonsense combination that must be absent to have positive and negative controls. That will give you a feel whether that score for your cells is reliable.&lt;/p&gt;&#xA;&lt;p&gt;This all assumes that the cells are even in the dataset. Depending on cell size and frequency (underrepresentation) they might even have dropped out during library prep.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-01-24T09:08:15.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20426" PostTypeId="1" AcceptedAnswerId="20446" CreationDate="2023-01-24T14:31:38.963" Score="3" ViewCount="80" Body="&lt;p&gt;Based on the ENSEMBL &lt;a href=&quot;https://ftp.ensembl.org/pub/release-108/gtf/homo_sapiens/Homo_sapiens.GRCh38.108.gtf.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;GTF&lt;/a&gt; file, I counted the number of exons per gene as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '$3 == &amp;quot;exon&amp;quot; {print}' Homo_sapiens.GRCh38.108.gtf | gffread -F --keep-exon-attrs --table &amp;quot;gene_id&amp;quot;,&amp;quot;transcript_id&amp;quot;,@numexons | sort -k1,1 | datamash -sg1 max 3 &amp;gt; gene_lists/exons_per_gene.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;An extract of the output is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ENSG00000034677 11&#xA;ENSG00000034693 12&#xA;ENSG00000034713 4&#xA;ENSG00000034971 4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, when I check the ENSG00000034713 ENSEMBL &lt;a href=&quot;https://www.ensembl.org/Homo_sapiens/Gene/Sequence?db=core;g=ENSG00000034713;r=16:75566375-75577881&quot; rel=&quot;nofollow noreferrer&quot;&gt;site&lt;/a&gt;, I count 5 exons. Why is this different?&lt;/p&gt;&#xA;" OwnerUserId="10386" LastActivityDate="2023-01-25T19:39:16.657" Title="Exon number between GTF file and ENSEMBL site does not match" Tags="|ensembl|gtf|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20427" PostTypeId="1" CreationDate="2023-01-24T14:57:20.760" Score="1" ViewCount="16" Body="&lt;p&gt;I have an edge list object weighted by a correlation value. I would like to know the changes in the node's connectivity on a continuous scale.&lt;/p&gt;&#xA;&lt;p&gt;In other words, how does the node connectivity differ from the edge weight scale (-1) to the edge weight (+1)? I would like to know how to do this or whether any R package or statistical model could solve the problems etc.&lt;/p&gt;&#xA;&lt;p&gt;I would like to know whether node connectivity increases or decreases in relation to the correlation weight. I.e. how it looks at the correlation between node degree and edge weight. But I don't know how to do it.&lt;/p&gt;&#xA;&lt;p&gt;A example edge list object is created in the code below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data &amp;lt;- structure(list(from = c(5L, 5L, 5L, 1L, 1L, 1L, 1L, 4L, 4L, 4L, &#xA;                                2L, 2L, 2L, 2L, 3L, 3L, 3L, 3L), &#xA;                       to = c(1L, 2L, 3L, 5L, 4L, 2L, 3L, 1L, 2L, 3L, 5L, 1L, 4L, 3L, 5L, 1L, 4L, 2L), &#xA;                   weight = c(runif(18, -1,1))), &#xA;                  row.names = c(NA, -18L), &#xA;                  class = &amp;quot;data.frame&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I appreciate your help!!&#xA;Best,&#xA;Amare&lt;/p&gt;&#xA;" OwnerUserId="9160" LastEditorUserId="73" LastEditDate="2023-01-25T19:44:38.360" LastActivityDate="2023-01-25T19:44:38.360" Title="analysing node connectivity difference in a continuous scale (edge weighted by correlations)" Tags="|r|graphs|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20428" PostTypeId="2" ParentId="9354" CreationDate="2023-01-24T15:15:05.500" Score="3" Body="&lt;p&gt;Currently in MiXCR SERIES 4 you can use &lt;code&gt;--dont-split-files &lt;/code&gt; parameter here.&#xA;Please check our new documentation portal:&#xA;&lt;a href=&quot;https://docs.milaboratories.com/mixcr/reference/mixcr-export/#clonotype-tables&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.milaboratories.com/mixcr/reference/mixcr-export/#clonotype-tables&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Also now we support presets that allow to perform analysis for a lot of standard protocols using a one line command.&lt;/p&gt;&#xA;" OwnerUserId="16818" LastActivityDate="2023-01-24T15:15:05.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20429" PostTypeId="2" ParentId="8862" CreationDate="2023-01-24T15:33:28.783" Score="2" Body="&lt;p&gt;I would also suggest using &lt;a href=&quot;https://www.gnu.org/software/parallel/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GNU parallel&lt;/a&gt; as described in our guides :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://docs.milaboratories.com/mixcr/guides/generic-multiplex-bcr/#one-command-solution&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.milaboratories.com/mixcr/guides/generic-multiplex-bcr/#one-command-solution&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Briefly:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ls /Documents/Data/Input/*_R1* | \&#xA;    parallel -j 4 \&#xA;    'mixcr analyze &#xA;    …&#xA;    {} \&#xA;    {=s:R1:R2:=} \             &#xA;{=s:.*/:/Documents/Data/result/:;s:_R.*::=}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That will take all fastq pairs from /Documents/Data/Input and put the result output files into /Documents/Data/result folder.&lt;/p&gt;&#xA;" OwnerUserId="16818" LastEditorUserId="16820" LastEditDate="2023-01-26T16:27:20.327" LastActivityDate="2023-01-26T16:27:20.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20430" PostTypeId="2" ParentId="20264" CreationDate="2023-01-24T15:35:36.250" Score="1" Body="&lt;p&gt;I think this question is more about subsetting than merging. You can try this solution in R:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;first_column = c(&amp;quot;ENSG00000165588&amp;quot;,&amp;quot;ENSG00000213551&amp;quot;)&#xA;df1 = data.frame(first_column)&#xA;first_column = c(&amp;quot;ENSG00000142192&amp;quot;, &amp;quot;ENSG00000140575&amp;quot;, &amp;quot;ENSG00000165588&amp;quot;, &amp;quot;ENSG00000165588&amp;quot;,&#xA;                 &amp;quot;ENSG00000213551&amp;quot;, &amp;quot;ENSG00000213551&amp;quot;,&amp;quot;ENSG00000197153&amp;quot;)&#xA;second_column = c(&amp;quot;ENSG00000165588&amp;quot;, &amp;quot;ENSG00000165588&amp;quot;, &amp;quot;ENSG00000186908&amp;quot;, &amp;quot;ENSG00000135446&amp;quot;,&#xA;                  &amp;quot;ENSG00000273983&amp;quot;, &amp;quot;ENSG00000274267&amp;quot;,&amp;quot;ENSG00000213551&amp;quot;)&#xA;df2 = data.frame(first_column,second_column)&#xA;&#xA;output = rbind(df2[df2&lt;span class=&quot;math-container&quot;&gt;$first_column %in% df1$&lt;/span&gt;first_column,],&#xA;               df2[df2&lt;span class=&quot;math-container&quot;&gt;$second_column %in% df1$&lt;/span&gt;first_column,])&#xA;&#xA;output &amp;lt;- transform(output, second_column = ifelse(output&lt;span class=&quot;math-container&quot;&gt;$second_column%in%df1$&lt;/span&gt;first_column,&#xA;                                                   first_column, second_column),&#xA;                            first_column = ifelse(output&lt;span class=&quot;math-container&quot;&gt;$second_column%in%df1$&lt;/span&gt;first_column,&#xA;                                                  second_column, first_column))&#xA;output &amp;lt;- output[ order(row.names(output)), ]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15931" LastEditorUserId="15931" LastEditDate="2023-01-24T15:46:13.270" LastActivityDate="2023-01-24T15:46:13.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20432" PostTypeId="2" ParentId="17483" CreationDate="2023-01-24T16:16:04.313" Score="2" Body="&lt;p&gt;According to our experience 3’ sequencing does not produce enough data to recover TCR and BCR repertoire. But we have some experience recovering repertoires from non enriched 5’ 10x RNA-Seq GEX data.&lt;/p&gt;&#xA;&lt;p&gt;With MiXCR we have specific presets that allow analyse 10x data in one single line command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mixcr analyze 10x-5gex-full-length \&#xA; --species hsa \&#xA; sample_R1.fastq.gz \&#xA; sample_R2.fastq.gz \&#xA; sample_result&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Also feel free to check out our documentation portal:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://docs.milaboratories.com/mixcr/reference/overview-built-in-presets/#10x&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.milaboratories.com/mixcr/reference/overview-built-in-presets/#10x&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16818" LastEditorUserId="16820" LastEditDate="2023-01-26T16:27:11.983" LastActivityDate="2023-01-26T16:27:11.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20433" PostTypeId="2" ParentId="9177" CreationDate="2023-01-24T16:22:24.970" Score="1" Body="&lt;p&gt;That can be easily done with MiXCR software.&#xA;If you first analyze the raw data using mixcr, which can be easily done with &lt;code&gt;mixcr analyze&lt;/code&gt; command depending on your data structure:&lt;/p&gt;&#xA;&lt;p&gt;e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mixcr analyze generic-tcr-amplicon \&#xA;--species mmu \&#xA;--rna \&#xA;--rigid-left-alignment-boundary \&#xA;--floating-right-alignment-boundary C \&#xA;  input_R1.fastq.gz \&#xA;  input_R2.fastq.gz \&#xA;  result&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;More on that here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://docs.milaboratories.com/mixcr/reference/mixcr-analyze/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.milaboratories.com/mixcr/reference/mixcr-analyze/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;After that you will have .clns file for every sample.&lt;/p&gt;&#xA;&lt;p&gt;Then, to merge all samples you can use &lt;code&gt;mixcr exportClonesOverlap&lt;/code&gt; and all .clns files as an input, e.g.:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mixcr exportClonesOverlap \&#xA;  --criteria &amp;quot;CDR3|AA|V|J&amp;quot; \&#xA;  clonesets/*.clns&#xA;  overlapTable.tsv &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;--criteria&lt;/code&gt; allows you to specify the criteria required for clonotypes to be treated as identical. &lt;code&gt;&amp;quot;CDR3|AA|V|J&amp;quot;&lt;/code&gt; means that two clonotypes have to share cdr3 amino acid sequence, v gene and j gene.&lt;/p&gt;&#xA;&lt;p&gt;overlapTable.tsv is exactly what you need.&lt;/p&gt;&#xA;&lt;p&gt;There are other parameters you can tune (like export only productive clonotypes etc.), more on that here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://docs.milaboratories.com/mixcr/reference/mixcr-exportClonesOverlap/#example&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.milaboratories.com/mixcr/reference/mixcr-exportClonesOverlap/#example&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16818" LastEditorUserId="16818" LastEditDate="2023-01-25T21:45:14.377" LastActivityDate="2023-01-25T21:45:14.377" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20434" PostTypeId="2" ParentId="20395" CreationDate="2023-01-24T19:34:57.140" Score="1" Body="&lt;p&gt;I'd consult references about AlphaFold. For example, wikipedia has a &amp;quot;&lt;a href=&quot;https://en.wikipedia.org/wiki/AlphaFold&quot; rel=&quot;nofollow noreferrer&quot;&gt;Limitations&lt;/a&gt;&amp;quot; page on the tool, though the page doesn't mention anything about synthetic proteins.&lt;/p&gt;&#xA;&lt;p&gt;The hesitation that I would have is that the tool relies to some extent on coevolutionary models of amino acids to infer interactions at both training and prediction steps, so throwing a synthetic protein at it may confuse it. See e.g. slide 10 &lt;a href=&quot;https://predictioncenter.org/casp14/doc/presentations/2020_12_01_TS_predictor_AlphaFold2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In an &lt;a href=&quot;https://github.com/deepmind/alphafold/issues/569&quot; rel=&quot;nofollow noreferrer&quot;&gt;interview&lt;/a&gt;, they acknowledge this rather directly:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Now, if you have a protein which is completely new, which does not have an evolutionary history—a synthetic protein, which has just been designed— then our system would not be as accurate as it would be for existing proteins.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I am not totally clear on whether this is due to learning a latent representation of constraints on similar proteins, or whether the tool attempts to directly find homologs. The &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8371605/&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt; methods suggest to me that it is the former, but I'm not sure. They describe training on only chunks of a big MSA. If that's the case, I'd guess that they still can use information about the natural chunks of your synthetic protein, and aren't expecting to be able to do a full-length protein alignment.&lt;/p&gt;&#xA;&lt;p&gt;So I think that you can expect the prediction to be &lt;em&gt;relatively&lt;/em&gt; low quality. It is still probably much better than any alternative way of predicting structure!&lt;/p&gt;&#xA;&lt;p&gt;As always, not a bad idea to &lt;a href=&quot;https://github.com/deepmind/alphafold&quot; rel=&quot;nofollow noreferrer&quot;&gt;post an issue on github&lt;/a&gt; asking this question.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-01-24T19:34:57.140" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20435" PostTypeId="1" AcceptedAnswerId="20436" CreationDate="2023-01-24T20:19:47.010" Score="2" ViewCount="62" Body="&lt;p&gt;The input file looks like this, and the complete file can be found &lt;a href=&quot;https://cloudstor.aarnet.edu.au/plus/s/oSWlTOXTIF1KWKJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;OG0000008 NbL07g11380.1 NbL19g07810.1 NbL19g09170.1 NbL19g19070.1 NbQ01g01670.1 NbQ01g03330.1 NbQ01g04070.1 NbQ01g04670.1 NbQ01g05120.1 NbQ01g05870.1 NbQ01g06940.1 NbQ01g07580.1 NbQ01g08860.1 NbQ01g10050.1 NbQ01g10360.1 NbQ01g14200.1 NbQ01g14790.1 NbQ01g16080.1 NbQ01g17760.1 NbQ01g19270.1 NbQ01g19310.1 NbQ01g19390.1 NbQ01g21260.1 NbQ01g21330.1 NbQ01g21740.1 NbQ01g21910.1 NbQ01g23100.1 NbQ01g24620.1 NbQ01g25340.1 NbQ01g26060.1 NbQ01g26320.1 NbQ02g00750.1 NbQ02g03100.1 NbQ02g03420.1 NbQ02g03610.1 NbQ02g03680.1 NbQ02g05120.1 NbQ02g07460.1 NbQ02g08170.1 NbQ02g08330.1 NbQ02g09220.1 NbQ02g09400.1 NbQ02g10620.1 NbQ02g11310.1 NbQ02g14330.1 NbQ02g14460.1 NbQ02g14520.1 NbQ02g15320.1 NbQ02g17090.1 NbQ02g17130.1 NbQ02g20290.1 NbQ02g23070.1 NbQ02g23420.1 NbQ02g24450.1 NbQ02g24480.1 NbQ02g26700.1 NbQ03g00830.1 NbQ03g01970.1 NbQ03g04460.1 NbQ03g06900.1 NbQ03g09530.1 NbQ03g10620.1 NbQ03g12760.1 NbQ03g13450.1 NbQ03g15540.1 NbQ03g15640.1 NbQ03g17180.1 NbQ03g20740.1 NbQ03g21510.1 NbQ03g24670.1 NbQ04g01350.1 NbQ04g01720.1 NbQ04g08420.1 NbQ04g09090.1 NbQ04g10450.1 NbQ04g11470.1 NbQ04g12120.1 NbQ04g14130.1 NbQ04g15440.1 NbQ04g15860.1 NbQ04g16450.1 NbQ04g16620.1 NbQ04g17760.1 NbQ04g19040.1 NbQ04g20020.1 NbQ05g03320.1 NbQ05g04660.1 NbQ05g05970.1 NbQ05g07500.1 NbQ05g08900.1 NbQ05g09760.1 NbQ05g10830.1 NbQ05g11150.1 NbQ05g11340.1 NbQ05g11510.1 NbQ05g11530.1 NbQ05g11780.1 NbQ05g16980.1 NbQ05g18190.1 NbQ05g21710.1 NbQ05g23400.1 NbQ06g01110.1 NbQ06g01430.1 NbQ06g04200.1 NbQ06g04440.1 NbQ06g05330.1 NbQ06g05770.1 NbQ06g05820.1 NbQ06g06700.1 NbQ06g08620.1 NbQ06g09190.1 NbQ06g10460.1 NbQ06g15220.1 NbQ06g15330.1 NbQ06g15700.1 NbQ06g16320.1 NbQ06g16590.1 NbQ06g17590.1 NbQ06g17670.1 NbQ06g20050.1 NbQ07g01030.1 NbQ07g02010.1 NbQ07g04350.1 NbQ07g04900.1 NbQ07g05610.1 NbQ07g06200.1 NbQ07g07110.1 NbQ07g07690.1 NbQ07g08640.1 NbQ07g10390.1 NbQ07g11920.1 NbQ07g14130.1 NbQ07g15590.1 NbQ07g15620.1 NbQ07g16910.1 NbQ07g17130.1 NbQ07g17950.1 NbQ08g00060.1 NbQ08g02240.1 NbQ08g02300.1 NbQ08g02310.1 NbQ08g03290.1 NbQ08g05330.1 NbQ08g09280.1 NbQ08g14890.1 NbQ08g15820.1 NbQ08g15950.1 NbQ08g19830.1 NbQ08g20150.1 NbQ08g22050.1 NbQ08g22620.1 NbQ09g02100.1 NbQ09g02620.1 NbQ09g03950.1 NbQ09g04200.1 NbQ09g06040.1 NbQ09g06640.1 NbQ09g08160.1 NbQ09g08330.1 NbQ09g09660.1 NbQ09g11220.1 NbQ09g13860.1 NbQ09g15180.1 NbQ09g15310.1 NbQ09g16530.1 NbQ09g17900.1 NbQ09g18100.1 NbQ09g18720.1 NbQ09g19280.1 NbQ09g21840.1 NbQ10g00480.1 NbQ10g01350.1 NbQ10g02870.1 NbQ10g03640.1 NbQ10g03730.1 NbQ10g08070.1 NbQ10g09510.1 NbQ10g11010.1 NbQ10g11760.1 NbQ10g12050.1 NbQ10g12060.1 NbQ10g12910.1 NbQ10g19200.1 NbQ10g19930.1 NbQ10g20390.1 NbQ10g20730.1 NbQ10g21080.1 NbQ10g21140.1 NbQ10g24010.1 NbQ11g00310.1 NbQ11g01210.1 NbQ11g01370.1 NbQ11g04610.1 NbQ11g04800.1 NbQ11g06060.1 NbQ11g07820.1 NbQ11g08390.1 NbQ11g09100.1 NbQ11g09350.1 NbQ11g13660.1 NbQ11g13930.1 NbQ11g16260.1 NbQ11g17360.1 NbQ11g18430.1 NbQ11g21080.1 NbQ11g23280.1 NbQ11g23990.1 NbQ11g25050.1 NbQ12g03770.1 NbQ12g04850.1 NbQ12g07340.1 NbQ12g09080.1 NbQ12g10820.1 NbQ12g12070.1 NbQ12g14750.1 NbQ12g15000.1 NbQ12g15230.1 NbQ12g20380.1 NbQ12g21080.1 NbQ12g21830.1 NbQ12g23960.1 NbQ13g01300.1 NbQ13g02350.1 NbQ13g03860.1 NbQ13g04410.1 NbQ13g08800.1 NbQ13g09850.1 NbQ13g10370.1 NbQ13g11700.1 NbQ13g12420.1 NbQ13g15780.1 NbQ13g16040.1 NbQ13g23160.1 NbQ13g24120.1 NbQ13g24540.1 NbQ13g25080.1 NbQ13g25490.1 NbQ13g28240.1 NbQ13g29770.1 NbQ14g01070.1 NbQ14g03950.1 NbQ14g05360.1 NbQ14g05410.1 NbQ14g06880.1 NbQ14g07270.1 NbQ14g07500.1 NbQ14g10290.1 NbQ14g10770.1 NbQ14g14320.1 NbQ14g17890.1 NbQ14g18710.1 NbQ14g20960.1 NbQ14g22890.1 NbQ15g00150.1 NbQ15g02300.1 NbQ15g02330.1 NbQ15g02350.1 NbQ15g03230.1 NbQ15g06190.1 NbQ15g07120.1 NbQ15g07750.1 NbQ15g09000.1 NbQ15g09050.1 NbQ15g11920.1 NbQ15g12650.1 NbQ15g12840.1 NbQ15g15670.1 NbQ15g15930.1 NbQ15g18670.1 NbQ15g19070.1 NbQ15g20620.1 NbQ15g22880.1 NbQ15g23000.1 NbQ15g26060.1 NbQ16g00880.1 NbQ16g04360.1 NbQ16g06490.1 NbQ16g09100.1 NbQ16g11020.1 NbQ16g11560.1 NbQ16g13810.1 NbQ16g13820.1 NbQ16g17040.1 NbQ16g17130.1 NbQ16g17340.1 NbQ16g18390.1 NbQ16g18430.1 NbQ16g23100.1 NbQ16g23570.1 NbQ16g24270.1 NbQ16g25200.1 NbQ16g25830.1 NbQ16g25880.1 NbQ16g25990.1 NbQ16g26610.1 NbQ16g26660.1 NbQ16g28010.1 NbQ16g28180.1 NbQ17g01150.1 NbQ17g01180.1 NbQ17g01570.1 NbQ17g01950.1 NbQ17g05460.1 NbQ17g05540.1 NbQ17g05980.1 NbQ17g07990.1 NbQ17g08300.1 NbQ17g09330.1 NbQ17g09400.1 NbQ17g10090.1 NbQ17g11220.1 NbQ17g13030.1 NbQ17g15460.1 NbQ17g16690.1 NbQ17g20980.1 NbQ17g22370.1 NbQ17g25040.1 NbQ17g28730.1 NbQ18g02140.1 NbQ18g02740.1 NbQ18g05440.1 NbQ18g06120.1 NbQ18g07470.1 NbQ18g12320.1 NbQ18g12530.1 NbQ18g12850.1 NbQ18g13840.1 NbQ18g14420.1 NbQ18g14930.1 NbQ18g15730.1 NbQ18g17750.1 NbQ18g17850.1 NbQ18g21060.1 NbQ19g01040.1 NbQ19g05480.1 NbQ19g06450.1 NbQ19g06510.1 NbQ19g08330.1 NbQ19g11840.1 NbQ19g11880.1 NbQ19g13750.1 NbQ19g14190.1 NbQ19g14210.1 NbQ19g14920.1 NbQ19g18540.1 NbQ19g19870.1 NbQ19g21020.1 NbQ19g21220.1 NbQ19g22080.1 NbQ19g22800.1 NbQ19g24690.1 NbQ19g24730.1 rna19561&#xA;OG0000001 Capann_59V1aChr01g048170.1 NbL01g00020.1 NbL01g00940.1 NbL01g02330.1 NbL01g03550.1 NbL01g03650.1 NbL01g04410.1 NbL01g04920.1 NbL01g06850.1 NbL01g16120.1 NbL01g19150.1 NbL01g20140.1 NbL01g20930.1 NbL01g22230.1 NbL01g24190.1 NbL01g24280.1 NbL01g24300.1 NbL02g00570.1 NbL02g00900.1 NbL02g01270.1 NbL02g02110.1 NbL02g02210.1 NbL02g02470.1 NbL02g03180.1 NbL02g04740.1 NbL02g04750.1 NbL02g06120.1 NbL02g06860.1 NbL02g07280.1 NbL02g07680.1 NbL02g07740.1 NbL02g09780.1 NbL02g11320.1 NbL02g12670.1 NbL02g13080.1 NbL02g14050.1 NbL02g14190.1 NbL02g15010.1 NbL02g15890.1 NbL02g16190.1 NbL02g16730.1 NbL02g17070.1 NbL02g17360.1 NbL02g18820.1 NbL02g19340.1 NbL02g20100.1 NbL02g23950.1 NbL02g24800.1 NbL03g01610.1 NbL03g01680.1 NbL03g01890.1 NbL03g02230.1 NbL03g02600.1 NbL03g03410.1 NbL03g04990.1 NbL03g05400.1 NbL03g08030.1 NbL03g08250.1 NbL03g08690.1 NbL03g10230.1 NbL03g11060.1 NbL03g13030.1 NbL03g14960.1 NbL03g15110.1 NbL03g16690.1 NbL03g16900.1 NbL03g18260.1 NbL03g18950.1 NbL03g21180.1 NbL03g21210.1 NbL03g21530.1 NbL03g22960.1 NbL03g24430.1 NbL04g01140.1 NbL04g01490.1 NbL04g02030.1 NbL04g02560.1 NbL04g03700.1 NbL04g04160.1 NbL04g05240.1 NbL04g05420.1 NbL04g05850.1 NbL04g12420.1 NbL04g12640.1 NbL04g13650.1 NbL04g13780.1 NbL04g14310.1 NbL04g16260.1 NbL04g17750.1 NbL04g18380.1 NbL04g18870.1 NbL04g19030.1 NbL04g19630.1 NbL05g00320.1 NbL05g03060.1 NbL05g03300.1 NbL05g04060.1 NbL05g07620.1 NbL05g08630.1 NbL05g09580.1 NbL05g10060.1 NbL05g11400.1 NbL05g12280.1 NbL05g13170.1 NbL05g16020.1 NbL05g17530.1 NbL05g17730.1 NbL05g18340.1 NbL05g18590.1 NbL05g18600.1 NbL05g19640.1 NbL05g20640.1 NbL05g21000.1 NbL05g21640.1 NbL05g22610.1 NbL06g00640.1 NbL06g00660.1 NbL06g02210.1 NbL06g03150.1 NbL06g03680.1 NbL06g04910.1 NbL06g07950.1 NbL06g09970.1 NbL06g11480.1 NbL06g12220.1 NbL06g12400.1 NbL06g12460.1 NbL06g12850.1 NbL06g13120.1 NbL06g14450.1 NbL06g14780.1 NbL06g16990.1 NbL06g17200.1 NbL06g17760.1 NbL06g20380.1 NbL07g02100.1 NbL07g02540.1 NbL07g02970.1 NbL07g03110.1 NbL07g04840.1 NbL07g05350.1 NbL07g06580.1 NbL07g07530.1 NbL07g08450.1 NbL07g09380.1 NbL07g09870.1 NbL07g10730.1 NbL07g10850.1 NbL07g11080.1 NbL07g12450.1 NbL07g12710.1 NbL07g13110.1 NbL07g13920.1 NbL07g14240.1 NbL07g15520.1 NbL07g16220.1 NbL07g17480.1 NbL08g01820.1 NbL08g02750.1 NbL08g02930.1 NbL08g03510.1 NbL08g03620.1 NbL08g03850.1 NbL08g03970.1 NbL08g04040.1 NbL08g06150.1 NbL08g06410.1 NbL08g06680.1 NbL08g06730.1 NbL08g07620.1 NbL08g08450.1 NbL08g08640.1 NbL08g09910.1 NbL08g10160.1 NbL08g11760.1 NbL08g12570.1 NbL08g13630.1 NbL08g13890.1 NbL08g15050.1 NbL08g15340.1 NbL08g18010.1 NbL08g18420.1 NbL08g19080.1 NbL08g19190.1 NbL09g00900.1 NbL09g02160.1 NbL09g02330.1 NbL09g02470.1 NbL09g04150.1 NbL09g05210.1 NbL09g07010.1 NbL09g09070.1 NbL09g10290.1 NbL09g10500.1 NbL09g11220.1 NbL09g13490.1 NbL09g15290.1 NbL09g15830.1 NbL09g17240.1 NbL09g19250.1 NbL09g19460.1 NbL09g20190.1 NbL09g21040.1 NbL09g21520.1 NbL09g23460.1 NbL10g00080.1 NbL10g03710.1 NbL10g04330.1 NbL10g04560.1 NbL10g05200.1 NbL10g06320.1 NbL10g07510.1 NbL10g07960.1 NbL10g08670.1 NbL10g08970.1 NbL10g11120.1 NbL10g11340.1 NbL10g11820.1 NbL10g13720.1 NbL10g14560.1 NbL10g14770.1 NbL10g16430.1 NbL10g18140.1 NbL10g18380.1 NbL10g19280.1 NbL10g19690.1 NbL10g21210.1 NbL10g22680.1 NbL10g23160.1 NbL10g23560.1 NbL10g24210.1 NbL11g00680.1 NbL11g00970.1 NbL11g01230.1 NbL11g01270.1 NbL11g01520.1 NbL11g01530.1 NbL11g02920.1 NbL11g03540.1 NbL11g03990.1 NbL11g05630.1 NbL11g08950.1 NbL11g08980.1 NbL11g09510.1 NbL11g10840.1 NbL11g11030.1 NbL11g11230.1 NbL11g12430.1 NbL11g13300.1 NbL11g15430.1 NbL11g16390.1 NbL11g16410.1 NbL11g17320.1 NbL11g18090.1 NbL11g21310.1 NbL11g21470.1 NbL11g21780.1 NbL11g21820.1 NbL11g22270.1 NbL11g22310.1 NbL11g23180.1 NbL11g24100.1 NbL12g00130.1 NbL12g01810.1 NbL12g02230.1 NbL12g02720.1 NbL12g02760.1 NbL12g04120.1 NbL12g04550.1 NbL12g06630.1 NbL12g07830.1 NbL12g09170.1 NbL12g10580.1 NbL12g12090.1 NbL12g12490.1 NbL12g12630.1 NbL12g12800.1 NbL12g13320.1 NbL12g13460.1 NbL12g14430.1 NbL12g14970.1 NbL12g15490.1 NbL12g17460.1 NbL12g18190.1 NbL12g18590.1 NbL12g19900.1 NbL12g20690.1 NbL12g22040.1 NbL12g22560.1 NbL13g00350.1 NbL13g01440.1 NbL13g02400.1 NbL13g03210.1 NbL13g03360.1 NbL13g04070.1 NbL13g05250.1 NbL13g08460.1 NbL13g09010.1 NbL13g09140.1 NbL13g10290.1 NbL13g11570.1 NbL13g13370.1 NbL13g14910.1 NbL13g18680.1 NbL13g19510.1 NbL13g23520.1 NbL13g24010.1 NbL13g24190.1 NbL13g24460.1 NbL13g26310.1 NbL13g26640.1 NbL13g26860.1 NbL13g27260.1 NbL13g27960.1 NbL14g02460.1 NbL14g02750.1 NbL14g08750.1 NbL14g08910.1 NbL14g09120.1 NbL14g09540.1 NbL14g09920.1 NbL14g11070.1 NbL14g11150.1 NbL14g12570.1 NbL14g14530.1 NbL14g14860.1 NbL14g15240.1 NbL14g15460.1 NbL14g16620.1 NbL14g16910.1 NbL14g17940.1 NbL14g21150.1 NbL14g21750.1 NbL14g21910.1 NbL15g00790.1 NbL15g01170.1 NbL15g02310.1 NbL15g04220.1 NbL15g05970.1 NbL15g06340.1 NbL15g06440.1 NbL15g07020.1 NbL15g07370.1 NbL15g07470.1 NbL15g09010.1 NbL15g13210.1 NbL15g14550.1 NbL15g14600.1 NbL15g17290.1 NbL15g18170.1 NbL15g19710.1 NbL15g21840.1 NbL15g21930.1 NbL15g23410.1 NbL15g23420.1 NbL15g23430.1 NbL15g25130.1 NbL15g25200.1 NbL16g01760.1 NbL16g02140.1 NbL16g04460.1 NbL16g05010.1 NbL16g05020.1 NbL16g06780.1 NbL16g07540.1 NbL16g07980.1 NbL16g09760.1 NbL16g10610.1 NbL16g12320.1 NbL16g13510.1 NbL16g14420.1 NbL16g15690.1 NbL16g17420.1 NbL16g17790.1 NbL16g17880.1 NbL16g18730.1 NbL16g18940.1 NbL16g19440.1 NbL16g20980.1 NbL16g23180.1 NbL16g23610.1 NbL16g23660.1 NbL16g23910.1 NbL16g24550.1 NbL16g24640.1 NbL16g25300.1 NbL16g25630.1 NbL16g26710.1 NbL17g01590.1 NbL17g02070.1 NbL17g02120.1 NbL17g02920.1 NbL17g03040.1 NbL17g03540.1 NbL17g03700.1 NbL17g03800.1 NbL17g05400.1 NbL17g07510.1 NbL17g08450.1 NbL17g08930.1 NbL17g10090.1 NbL17g14370.1 NbL17g14600.1 NbL17g15390.1 NbL17g15900.1 NbL17g16000.1 NbL17g16910.1 NbL17g17480.1 NbL17g18240.1 NbL17g20020.1 NbL17g20830.1 NbL17g21220.1 NbL17g21690.1 NbL17g25960.1 NbL18g00030.1 NbL18g00150.1 NbL18g00310.1 NbL18g00670.1 NbL18g00700.1 NbL18g01630.1 NbL18g02650.1 NbL18g04460.1 NbL18g05210.1 NbL18g05690.1 NbL18g07270.1 NbL18g07440.1 NbL18g07500.1 NbL18g09090.1 NbL18g09810.1 NbL18g09880.1 NbL18g10500.1 NbL18g10990.1 NbL18g12070.1 NbL18g13060.1 NbL18g17480.1 NbL19g00230.1 NbL19g04470.1 NbL19g04700.1 NbL19g07770.1 NbL19g07870.1 NbL19g09260.1 NbL19g09870.1 NbL19g13480.1 NbL19g14360.1 NbL19g14400.1 NbL19g14720.1 NbL19g17700.1 NbL19g18860.1 NbL19g22750.1 NbQ13g00370.1 NbQ14g10310.1 NbQ17g06050.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This script appears to have some problems with the file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;orthofinder_output = &amp;quot;../OrthoFinder-res/Results_Jan23/Orthogroups/Orthogroups-fixed.txt&amp;quot;&#xA;orthogroups = pd.read_csv(orthofinder_output, sep=' ', header=None)&#xA;&#xA;# Extract gene family information&#xA;expansions = {}&#xA;contractions = {}&#xA;for i, row in orthogroups.iterrows():&#xA;    if len(row) &amp;gt; 2:&#xA;        expansions[row[0]] = row[2:]&#xA;    else:&#xA;        contractions[row[0]] = row[1]&#xA;&#xA;# Print expansions and contractions&#xA;print(&amp;quot;Expansions:&amp;quot;, expansions)&#xA;print(&amp;quot;Contractions:&amp;quot;, contractions)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I got the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;geneFamilyExpansionsContractions.py:&#xA; 20: DtypeWarning:&#xA;   Columns (2,3,4,5,6,...,964,965,966,967,968) have mixed types.&#xA;   Specify dtype option on import or set low_memory=False.&#xA;  orthogroups = pd.read_csv(orthofinder_output, sep=' ', header=None)&#xA;...&#xA;Expansions:&#xA;...&#xA;Name: 63241, Length: 967, dtype: object, 'OG0063242':&#xA;2      NaN&#xA;3      NaN&#xA;4      NaN&#xA;5      NaN&#xA;6      NaN&#xA;      ... &#xA;964    NaN&#xA;965    NaN&#xA;966    NaN&#xA;967    NaN&#xA;968    NaN&#xA;Name: 63242, Length: 967, dtype: object}&#xA;Contractions: {}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can it be fixed?&lt;/p&gt;&#xA;" OwnerUserId="3329" LastEditorUserId="73" LastEditDate="2023-01-25T19:48:44.483" LastActivityDate="2023-01-25T19:48:44.483" Title="expansions and contractions from OrthoFinder" Tags="|python|proteins|pandas|orthofinder|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20436" PostTypeId="2" ParentId="20435" CreationDate="2023-01-24T22:01:01.313" Score="2" Body="&lt;p&gt;The answer is actually right there in the error message (warning message, actually):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;DtypeWarning: Columns (2,... 968) have mixed types. Specify dtype option on import or set low_memory=False.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Setting &lt;code&gt;low_memory-False&lt;/code&gt; makes the error go away:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;orthogroups = pd.read_csv(orthofinder_output, sep=' ', header=None, low_memory=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;According to &lt;a href=&quot;https://www.roelpeters.be/solved-dtypewarning-columns-have-mixed-types-specify-dtype-option-on-import-or-set-low-memory-in-pandas/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this random blog&lt;/a&gt; post I found when I searched for &amp;quot;pandas DtypeWarning: have mixed types&amp;quot;, this is because:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Pandas is really nice because instead of stopping altogether, it guesses which dtype a column has. However, by default, Pandas has the low_memory=True argument. This means that the CSV file gets split up in multiple chunks and the guess is done for every chunk, resulting in a column with multiple dtypes. Pandas is so kind to let you know it was confused and something might have happened.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The same blog also suggests that this isn't really the right approach, and that makes sense, since this is forcing Pandas to read the entire file and that might be a problem for huge files or low memory machines. So, reading on, the blog suggests the other option provided by the error, setting the &lt;code&gt;dtype&lt;/code&gt; manually.&lt;/p&gt;&#xA;&lt;p&gt;Now, I am afraid I have never used Pandas and don't really know much Python either, but a quick glance at the &lt;a href=&quot;https://pandas.pydata.org/pandas-docs/stable/user_guide/basics.html#basics-dtypes&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pandas documentation&lt;/a&gt; suggests that the &lt;code&gt;object&lt;/code&gt; data type is the most versatile and should work for everything, including strings. Since your file is nothing but a collection of strings, as far as I can tell, this seems like the best approach.&lt;/p&gt;&#xA;&lt;p&gt;So I changed your script to this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;import pandas as pd&#xA;import sys&#xA;&#xA;orthofinder_output = sys.argv[1]&#xA;orthogroups = pd.read_csv(orthofinder_output, sep=' ', header=None, low_memory=False)&#xA;# Extract gene family information&#xA;expansions = {}&#xA;contractions = {}&#xA;for i, row in orthogroups.iterrows():&#xA;    if len(row) &amp;gt; 2:&#xA;        expansions[row[0]] = row[2:]&#xA;    else:&#xA;        contractions[row[0]] = row[1]&#xA;&#xA;# Print expansions and contractions&#xA;print(&amp;quot;Expansions:&amp;quot;, expansions)&#xA;print(&amp;quot;Contractions:&amp;quot;, contractions)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And it seems to work just fine with no warnings shown.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-01-24T22:01:01.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20437" PostTypeId="1" AcceptedAnswerId="20438" CreationDate="2023-01-24T23:58:18.973" Score="2" ViewCount="343" Body="&lt;p&gt;I want to remove reads from FASTQ file that contain homopolymers &amp;gt; 10bp and remove reads with &amp;lt;35 average quality score across the entire read.&lt;/p&gt;&#xA;&lt;p&gt;To remove homopolymers &amp;gt; 10bp, I tried this on a Linux machine, but it only removes the sequence line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zcat file.fastq.gz | &#xA;  awk '!/A{10,}/&amp;amp;&amp;amp;!/C{10,}/&amp;amp;&amp;amp;!/G{10,}/&amp;amp;&amp;amp;!/T{10,}/ {print}' cleaned_file.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user16821" LastEditorDisplayName="user16821" LastEditDate="2023-01-26T23:32:56.190" LastActivityDate="2023-01-26T23:32:56.190" Title="Remove low quality reads" Tags="|fastq|awk|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20438" PostTypeId="2" ParentId="20437" CreationDate="2023-01-25T01:09:56.057" Score="4" Body="&lt;p&gt;Use an off the shelf tool for read preprocessing.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/ivars-silamikelis/fastq_qual_trimmer&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is one&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./fastq_qual_trimmer -i test.fq -m 35 -H 10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That one does exactly what you want but seems a little old/unmaintained, so here is &lt;a href=&quot;https://manpages.debian.org/testing/ea-utils/fastq-mcf.1.en.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;another&lt;/a&gt; that does more or less the same thing:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fastq-mcf --qual-mean 35 --homopolymer-pct {X} adapters.fa reads.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where &lt;code&gt;{X}&lt;/code&gt; is 10 / read length, adapters.fa is an adapter file (which I believe can be empty or filled with dummy sequences).&lt;/p&gt;&#xA;&lt;p&gt;You could also use a library like biopython or dnaio to write a quick script to do this, but it hardly seems worth it.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2023-01-25T23:04:24.560" LastActivityDate="2023-01-25T23:04:24.560" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20439" PostTypeId="2" ParentId="20435" CreationDate="2023-01-25T01:58:07.867" Score="1" Body="&lt;p&gt;The error right at the end is&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Specify &lt;code&gt;dtype&lt;/code&gt; option on import or set &lt;code&gt;low_memory=False&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt; Oh wait @terdon has already spotted this! :-D ... great minds and all that :)&lt;/p&gt;&#xA;&lt;p&gt;In addition, the &lt;code&gt;pandas&lt;/code&gt; print out it states ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dtype object&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This isn't good. &lt;code&gt;Name: 63242,&lt;/code&gt; is a bit unusual BTW worth checking.&lt;/p&gt;&#xA;&lt;p&gt;Looking at the values ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;NaN&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... that is really not good.&lt;/p&gt;&#xA;&lt;p&gt;What I think is happening is it's been assigned to a numerical datatype, but it's categorical data. Hence &lt;code&gt;NaN&lt;/code&gt; because like the alphabet can be none-base 10 numbers, in data compression used a lot, but it's got to be consistent pattern (I genuinely don't know if pandas can spot/work in none-base 10 number systems). &lt;code&gt;pandas&lt;/code&gt; is really &lt;code&gt;C&lt;/code&gt; (probably &lt;code&gt;C++&lt;/code&gt;) - its a static type backend - so those datatypes are the key to this.&lt;/p&gt;&#xA;&lt;p&gt;It makes sense because your data is categorical data and &lt;code&gt;NaN&lt;/code&gt; means its managed to confuse a large amount of it for e.g. &lt;code&gt;int64&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Looking the &lt;a href=&quot;https://pandas.pydata.org/docs/reference/api/pandas.read_csv.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; the alternative is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;orthogroups = pd.read_csv(orthofinder_output, sep=' ', header=None, dtype='category')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The documentation is&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;low_memorybool, default True&#xA;Internally process the file in chunks, resulting in lower memory use while parsing, but possibly mixed type inference. To ensure no mixed types either set False, or specify the type with the dtype parameter. Note that the entire file is read into a single DataFrame regardless, use the chunksize or iterator parameter to return the data in chunks. (Only valid with C parser).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;What you want is to issue a &lt;code&gt;df.dtypes&lt;/code&gt; command once it is loaded and the output will define the datatype per column,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;'A'    int64&#xA;'B'    int32&#xA;'C'   category&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you have mixed dtypes it &lt;em&gt;may&lt;/em&gt; (I stress &lt;em&gt;may&lt;/em&gt;) be able to correct it via,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df.astype('int64') # this is not your data&#xA;df.astype('category') # this is your data&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;HOWEVER, I don't think this will work, because &lt;code&gt;NaN&lt;/code&gt; indicates its trashed the data. Anyway the data is definitely categorical BTW.&lt;/p&gt;&#xA;&lt;p&gt;Anyway, good question and interesting stuff.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;As a bit of gossip, you can also switch to categorical ordered data I think this will work for dataframes, this example is &lt;code&gt;Series&lt;/code&gt;. There's other ways to order the data BTW.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from pandas.api.types import CategoricalDtype&#xA;my_dtype = CategoricalDtype(\&#xA;    categories=[2, 1], ordered=True)&#xA;df.dtype(my_dtype)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output is &lt;code&gt;1&lt;/code&gt; and &lt;code&gt;2&lt;/code&gt; now switch places because 2 &amp;lt; 1. Categorial datatypes look to be the right datatype for your inputs.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-25T02:27:02.827" LastActivityDate="2023-01-25T02:27:02.827" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20441" PostTypeId="1" CreationDate="2023-01-25T08:37:54.493" Score="2" ViewCount="41" Body="&lt;p&gt;I have the following design matrix:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mm_noreps.interactions &amp;lt;- model.matrix(~condition*TRAPed)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Both variables are factors condition has 4 levels and TRAPed has 2 levels.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pheatmap(mm_noreps.interactions)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WaSJ5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WaSJ5.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want to use the following contrasts:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;TRAP positive (just consider this a drug treatment) vs TRAP negative in all conditions,&#xA;c(-1,0,0,0,0,1,0,0)&lt;/li&gt;&#xA;&lt;li&gt;FearRecallpositive vs all other condition's positive cells c(0,0,-1/3,0,-1/3,-1/3,0,1), and&lt;/li&gt;&#xA;&lt;li&gt;FearRecallpostive vs all other conditions c(-1/7,-1/7,-1/7,-1/7,-1/7,-1/7,-1/7,1).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The main question I have is are there differentially expressed genes specific to the FearRecall:TRAPedpostive interaction.  Does contrast 3 address this? The other two are more sanity checks than actual contrasts of interest.&lt;/p&gt;&#xA;" OwnerUserId="2482" LastEditorUserId="2482" LastEditDate="2023-01-25T21:48:22.320" LastActivityDate="2023-01-25T21:48:22.320" Title="Does the contrast matrix I have made address the question I am trying to answer?" Tags="|rna-seq|edger|limma|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20442" PostTypeId="1" CreationDate="2023-01-25T02:38:33.607" Score="-1" ViewCount="1053" Body="&lt;p&gt;I have a fast file containing DNA sequences. I've already cleaned the file and has only DNA sequences with no gaps or errors or white space characters. I want to find open reading frames in the fasta file that are longer than 30 bp.&lt;/p&gt;&#xA;&lt;p&gt;Here's a little description: An open reading frame is defined by a sequence that&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;start with &lt;code&gt;ATG&lt;/code&gt;,&lt;/li&gt;&#xA;&lt;li&gt;ends with &lt;code&gt;TAG&lt;/code&gt;, &lt;code&gt;TAA&lt;/code&gt;, or &lt;code&gt;TGA&lt;/code&gt;,&lt;/li&gt;&#xA;&lt;li&gt;is a multiple of three.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;For each open reading frame, I want to print the start position in the sequence, the length of the open reading frame, and the open reading frame sequence to the screen. Then I want to print the total number of open reading frames found. I would appreciate some help.&lt;/p&gt;&#xA;&lt;p&gt;I can't use libraries.&lt;/p&gt;&#xA;&lt;p&gt;Also, here's the DNA sequence I am working with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GCATGCAATACAGTGACATATATATATACCCTAACACTACCCTAACCCTACCCTATTTCA&#xA;ACCCTTCCAACCTGTCTCTCAACTTACCCTCACATTACCCTACCTCTCCACTTGTTACCC&#xA;TGTCCCATTCAACCATACCACTCCCAACCACCATCCATCCCTCTACTTACTACCACCAAT&#xA;CAACCGTCCACCATAACCGTTACCCTCCAATTAGCCATATTCAACTTCACTACCACTTAC&#xA;CCTGCCATTACTCTACCATCCACCATCTGCTACTCACCATACTGTTGTTCTACCCTCCAT&#xA;ATTGAAACGTTAACAAATGATCGTAAATAATACACATATACTTACCCTACCACTTTATAC&#xA;CACCACACATCACATGCCATACTCACCTTCACTTGTATACTGATATGCCATACGCACACG&#xA;GATGCTACAGTATATACCACTCTCAAACTTACCCTACTCTCACATTCTACTCCACTCCAT&#xA;GACCCATCTCTCACTAAATCAGTACTAAATGCACCCACATCATTATGCACGGCACTTGCC&#xA;TCAGCGGTCTATACCCTGAGCCATTTACCCATAACTCCCACGATTATCCACATTTTAATA&#xA;TCTATATCTCATTCGGCGGGCCCAAATATTGTATAACTGCTCTTAATACATACGTTATAC&#xA;CACTTTTGCACCATATACTAACCACTCAAATTATATACACTTATGCCAATATAACCAAAA&#xA;AATTACCACTAAAATCACCTAAACATAAAAATTATTTATCTTTCAACTTTACGAAATAAA&#xA;CACACTCAATTGCGTATCTATACCACCATGACGTCATTAACGTAAAAGTTCCTTAATATT&#xA;ACCATTTGCTTGAACGGATACCATTTCAGAATATTTCTAACTTTCACAGACCATACATTA&#xA;GAATAATATGCCACCTCACTGTCGTAACACTCTATATTCACCGAGAAACAATACGGTAGT&#xA;GGCTCAAACTCATGCCGGTGCTATGATACAATTGTATCCTATTTCCATTCTCATATGCTA&#xA;TCCGCAATATCCTAAAAGCATAACTGATGCATCTTTAATCTTGTATGTGACACTACTCAT&#xA;ACGAAGAGACTATATCTAAAGAAGACGATACAGTGATATGTACGTTGTTTTTGTAGAATT&#xA;ATAATGAAACGTCAAATAACCCTACTATATTATAACTTATCAGCGCCGTATACTAAAACG&#xA;GACGTTACGATATTGTCTCACTTCATCTTACCACCCTCTATCTTATTGTTGATAAAACAC&#xA;TAACCCCTCAGCTTTATTTCTAGTTACAGTTACAACAAACTATCCCAAACCATAAATCTT&#xA;AATATTTTAGGTGTCAAAAAATGAGGATCTCCAAATGAGAGTTTGGTACCATGACTTGTA&#xA;ACTCCACTACCCTGATCTGCAATCTTGTTCTTAGAAGTGACGCATACTCTATATGGCCCG&#xA;ACGCGACGCGCCAAAAAATGAAAAAAGAAGCAGCGACTCATTTTTATGGAAGGACAAAGT&#xA;GCTGCGAAGTCATACGCTTCCAATTTCATTATTGTTTATTGGACATACTCTGTTAGCTTT&#xA;ATTACCGTCCACGCTTTTTCTACAATAGTGTAAAAGTTTCTTTCTTATGTTCATCGTATT&#xA;CATAAAATGCTTCACGAACACCGTCATTGATCAAATAGGTTTATAATATTAATATACATT&#xA;TATATAATCGGCGGTATTTATATCATCAAAAAAAGTAGTTTTTTATTTTATTTTTTCATT&#xA;ACTTTTCACTGTCTATGGATTTTCATTCGTAAAGGCATCACTCCCTAGTTTGCGATAGTG&#xA;TAGATACCGTCCTTGGATAGAGCACTGGAGATGGCTGGCTTTAATCTGCTGGAGTACCAT&#xA;GGAACACCGGTGATCATTCTGGTCACTTGGTCTGGGGCAATACCAGTCAACATGGTGGTG&#xA;AAGTCACCGTAGTTGAAAACGGCTTCAGCAACTTCGACTGGGTAGGTTTCAGTTGGGTGG&#xA;GCGGCTTGGAACATGTAGTATTGGGCCAAGTGAGCTCTGATATCAGAGACGTAGACACCC&#xA;AATTCCACCAAGTTGACTCTTTCGTCAGATTGAGCTAGAGTGGTGGTTGCAGAAAGCAGT&#xA;AGCAGCGATGGCAGCGACACCAGCAGCGATTGAAGTTAATTTGACCATTGTATTTGTTTT&#xA;GTTTGTTAGTGCTGATGTAATCTTAACAAGAAATAGTGAAATGAAAGCGCATACCTCAAA&#xA;GGCATATAGTTGAAGCAGCTCTATTTATACCCGTTCCTCCATCTGTCATCACTACTTAAA&#xA;CGATTCGTTAACAGACGCTCATTTAGCACCTCACATATTCTCCATATCTCATCTTTCACA&#xA;CAATCTCATTATCTCTATGGAGATGCTCTTGTTTCTGAACGAATCATACATCTTTCATAG&#xA;GTTTCGTATGTGGAGTATTGTTTTATGGCACTCATGTGTATTCGTATGCGCAGAATGTGG&#xA;GAATGCCTATTATAGGGGTGCCGGGGGGTGCCTTGAAAAACCCTTTTGCGCGCCTGTTAA&#xA;GTTTCCGTTTTCAGTCAAAAAGAATATCCGAATTTTAGATTTGGACCCTCGTTCAGAAGC&#xA;TTACTGTCTAAGCCATCATTTGGTGTGTCCTAAACGGTTTCCATGCAAAGCAACATCTTT&#xA;GTTACTCATTCCTGAAGGTTGAATAAAAAATAGCAACCATGTGCAGGGAGTCGTATATTG&#xA;TTAGATTCTAGAGACTTGTACGCATCATCAAAGCTGTAAATAGAATAAACATACGCAAGG&#xA;CGTCAAAAGTGCATAGTTAAGAAAATTCCTGACATGTGAAAATATGTGTTTATGAAATGT&#xA;GTCAAGGCCCGTCTATAGCGTAGTTAACCCCTCTGCAGGAGTAAGTGACTTTTTTTACGC&#xA;TCAAAAGGCAACGAGGGCACATACTTAAAAGTCATTTTCAAACACATCTGCAGTTTGCAA&#xA;CGACAGATAACAATATTATGATAGGATGGTATGATGTTATTAGCTGCCACATATTTTTAA&#xA;TAATAGCATTAGTCACGTCTCTTCAATTGTTGGGATGAAACTCTAAAATATCATTCCTTT&#xA;AGTAGTATTCCAGTTACCAGTATATTATCACATGCCGAAAAAGAAGATGACATAAAGATC&#xA;GACAAACAGTCTTCAAATATAATGGAAGCTGGAATGCAAGGATTGATAATGTAACAGGAT&#xA;ACTGAATGACAAAGTATAAATGAAAAAAAAAAAAAAAAAAAAGTAGTAATACTATTATGT&#xA;GGAAATACCGATTCCATTTTGAGGATTCCAATTGTTGGAATAAAAATCAACTATCATCTA&#xA;CTAACTAGTATTTACGTTACTAGTATATTATCATATACGGTGTTAGAAGATGACGCAAAT&#xA;GATGAGAAATAGTCATCTAAATTAGTGGAAGCTGAAACGCAAGGATTGATAATGTAATAG&#xA;GATCAATGAATATTAACATATAAAACGATGATAATAATATTTATAGAATTGTGTAGAATT&#xA;GCAGATTCCCTTTTATGGATTCCTAAATCCTGGAGGAGAACTTCTAGTATATCTACATAC&#xA;CTAATATTATAGCCTTAATCACAATGGAATCCCAACAATTACATCAAAATCCACATTCTC&#xA;TACACCAATACCATCGACGAGAGCTTCTAGTAAATTGTATACATAACAGTATAACCCTTA&#xA;CCAACAATGGAATCTCAAAGATTATTAAATTATTCACAGACTCTGAGGATTCGGGTAAAA&#xA;TAGGGTATTTAACTGGTTACCGGAAAGGTTTAGAAAATTCGTGGAGGGTTGGCCGAGTGG&#xA;TCTAAGGCGGCAGACTTAAGATCTGTTGGACGGTTGTCCGCGCGAGTTCGAACCTCGCAT&#xA;TCATTTTCCGGATCCACTTTATAGTGGGTCACTTCATCTCTTGTAGGGTTCACTGAAAGA&#xA;TAATCAATCAAATTATCATATGACCAATGGGTCACTTTTTTGATGAGACCCATGTTTATT&#xA;CTTCTATACGTTCGTGATACTGCACTTACGACTACCAGTAACATCAGAATAACCATAGAG&#xA;CTTATGCTATCTTGAAATATATCATCGCCGTAAAGTATGATGTGTAGCGCATTAGTCTTC&#xA;TGCATTCATATTATCAGTGGTATACCGTTTGGTATATGCTTTCCAATCATGAAGTGTTGG&#xA;CTCTACTCTATATAAATTCGACATCCATTGATTTTCTCAATAAATGAGTTACCCAAGTAA&#xA;GCTTTCCATTGATACAAGTGATCTACATTCTTGCGACGCCAAATTATAAAGCACTAAAAA&#xA;TCATATCATACCCAATTGCGGGCAGCACATGTATATATTATACACTACTTATAACTAACC&#xA;TTCTTCATATATAACAATATGCCTGATAATACTAATGCATTTAAATCATATCATGAAAGA&#xA;AAATTACATGGGTTTTATTGACATAATTGCATTTAGAATACATAAAATTCTAAAGAATTA&#xA;ATATATCCAAAGTATTAGACATAACCAAGAATAATAGTGAATAATTTTAGATTTTGTTAC&#xA;ATATAATTCTGCTTGCCTATCTCTTCCACTCTTTTCAAAACGTTGCATGTAAGCGTTACT&#xA;AATATTCCGCTTTATTTTGTTGCAATTCCTAATTTTTTCATTACATTATCTTGCGAGTAC&#xA;GGAAGCGATTAACGTTCTCCCAATAGAAGGAACAAACATAGATATTGAAGTTTTACTGCT&#xA;TTTGCTTACCTGACCTTTTTCAAATTTAATTTTTTCCCGCTAATAAGACCATAAACTACC&#xA;CCGAACCAAATTCTAAAAGATAGTCAGCTGGATTAGAGTTGTCATCTCCAAACATTAATT&#xA;TTGCATTATCTTCGGCTTCAATCAAATCGCCTGATAAGAACTCCTTTAATTCTTCATGAA&#xA;TGTTTGTATGTGGATGACTCTCCATAGTGCCAGCATGATTGTGGTTACCGACCGAATCAT&#xA;ATAACGGTGGCTCCCAATTGTGCAATTCAGCCTTACTATTTTGTTCATTCATCAAAGCAT&#xA;TTGGGACAGATCTAATATCTATAATTCTTTCCTCACTATTCTCGCTATTATTTTGCCCCG&#xA;AACTGGCATGGTGGTTATTGGTAAAAGGAGATAATGCTGCGACAGAACTTTTCTTCTCTT&#xA;CCAATTCTTTTATCTTCGCCAATAACGCCTTCTTTTTTCGTGCTTGTATTTCTAAAATTT&#xA;CGGCCAGGTATTGTAAATATTCGACCGTTCTATCCAAAATGATACCCTTATTTGGTTTGA&#xA;TTTGTTTACCTAGGTCATCGTAATTCAATAAAGATGGTGGAACCAACTGGCCGAGTTCTT&#xA;TTATCTTTTGCTTTATTAATTCTCTTCTTCTCCTTTCGACGGCATTATGAAACTCTCTTT&#xA;TGCGCCTCAGTTTCTCATCAGAAGTTAACCCGCCTAAAATCTTTGGAACACTTCCAGGTC&#xA;CTATATTTTCAGTCATATTGCTACTTATTGAAGTGTGTCTTGTTCTGGGTGTGTTTATGC&#xA;TACCATGCCTAAAAGAAGATGAAAGGAAACTTCCTGCACGAAATGACGATGATGGTGATC&#xA;TTACCTTTGGGGAATATGTGGAAGATACGGATGCTGGGCCCAAGCTTTGTGGATTATAAG&#xA;AAAATGATGATGAATATGTGTTTGGTGTCATCATATCAGAATTGATGCTAGAAGATAAAG&#xA;AGGAGCTCAAATCATCAGTTAAGTTATACATTGTGTCATCGGTGCCATGCTGAAATAAAA&#xA;AGTCATGTGGTAACTCAGCTTTTAAGCCTTCTTGTTGCTGTAATGGAGTATTCATTGCTC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="user556983" LastEditorUserId="73" LastEditDate="2023-01-25T19:52:20.783" LastActivityDate="2023-01-27T02:31:23.090" Title="Find open reading frames in a DNA sequence" Tags="|python|homework|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20443" PostTypeId="2" ParentId="20442" CreationDate="2023-01-25T14:17:27.777" Score="0" Body="&lt;p&gt;The code is,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.Seq import Seq&#xA;frame = 2&#xA;myseq = myseq[frame:]&#xA;bioseq = Seq(myseq)&#xA;fullaa = bioseq.translate()&#xA;for aa in fullaa.split(&amp;quot;*&amp;quot;):&#xA;    print (aa)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The sequence is below the &lt;code&gt;-----&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The output looks like open readingframe2. The first protein is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;MQYSDIYIYPNTTLTLPYFNPSNLSLNLPSHYPTSPLVTLSHSTIPLPTTIHPSTYYHQSTVHHNRYPPISHIQLHYHLPCHYSTIHHLLLTILLFYPPY&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This meets with the requirements ... the remaining output is below ... it looks like a eukaryotic with introns. It feels like a home-work question.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt; Blasted it, its a yeast protein&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;RecName: Full=Putative uncharacterized protein YBL109W [Saccharomyces cerevisiae S288C]&#xA;Sequence ID: P89495.3&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The hit is 100%&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Remaining output&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;NVNK&#xA;S&#xA;IIHIYLPYHFIPPHITCHTHLHLYTDMPYAHGCYSIYHSQTYPTLTFYSTP&#xA;PISH&#xA;ISTKCTHIIMHGTCLSGLYPEPFTHNSHDYPHFNIYISFGGPKYCITALNTYVIPLLHHILTTQIIYTYANITKKLPLKSPKHKNYLSFNFTK&#xA;THSIAYLYHHDVINVKVP&#xA;YYHLLERIPFQNISNFHRPYIRIICHLTVVTLYIHRETIR&#xA;WLKLMPVL&#xA;YNCILFPFSYAIRNILKA&#xA;LMHL&#xA;SCM&#xA;HYSYEETISKEDDTVICTLFL&#xA;NYNETSNNPTIL&#xA;LISAVY&#xA;NGRYDIVSLHLTTLYLIVDKTLTPQLYF&#xA;LQLQQTIPNHKS&#xA;YFRCQKMRISK&#xA;EFGTMTCNSTTLICNLVLRSDAYSIWPDATRQKMKKEAATHFYGRTKCCEVIRFQFHYCLLDILC&#xA;LYYRPRFFYNSVKVSFLCSSYS&#xA;NASRTPSLIK&#xA;VYNINIHLYNRRYLYHQKK&#xA;FFILFFHYFSLSMDFHS&#xA;RHHSLVCDSVDTVLG&#xA;STGDGWL&#xA;SAGVPWNTGDHSGHLVWGNTSQHGGEVTVVENGFSNFDWVGFSWVGGLEHVVLGQVSSDIRDVDTQFHQVDSFVRLS&#xA;SGGCRKQ&#xA;QRWQRHQQRLKLI&#xA;PLYLFCLLVLM&#xA;S&#xA;QEIVK&#xA;KRIPQRHIVEAALFIPVPPSVITT&#xA;TIR&#xA;QTLI&#xA;HLTYSPYLIFHTISLSLWRCSCF&#xA;TNHTSFIGFVCGVLFYGTHVYSYAQNVGMPIIGVPGGALKNPFARLLSFRFQSKRISEF&#xA;IWTLVQKLTV&#xA;AIIWCVLNGFHAKQHLCYSFLKVE&#xA;KIATMCRESYIVRF&#xA;RLVRIIKAVNRINIRKASKVHS&#xA;ENS&#xA;HVKICVYEMCQGPSIA&#xA;LTPLQE&#xA;VTFFTLKRQRGHILKSHFQTHLQFATTDNNIMIGWYDVISCHIFLIIALVTSLQLLG&#xA;NSKISFL&#xA;&#xA;YSSYQYIITCRKRR&#xA;HKDRQTVFKYNGSWNARIDNVTGY&#xA;MTKYK&#xA;KKKKKKSSNTIMWKYRFHFEDSNCWNKNQLSSTN&#xA;YLRY&#xA;YIIIYGVRR&#xA;RK&#xA;&#xA;EIVI&#xA;ISGS&#xA;NARIDNVIGSMNINI&#xA;NDDNNIYRIV&#xA;NCRFPFMDS&#xA;ILEENF&#xA;YIYIPNIIALITMESQQLHQNPHSLHQYHRRELLVNCIHNSITLTNNGISKIIKLFTDSEDSGKIGYLTGYRKGLENSWRVGRVV&#xA;GGRLKICWTVVRASSNLAFIFRIHFIVGHFISCRVH&#xA;KIINQIII&#xA;PMGHFFDETHVYSSIRS&#xA;YCTYDYQ&#xA;HQNNHRAYAILKYIIAVKYDV&#xA;RISLLHSYYQWYTVWYMLSNHEVLALLYINSTSIDFLNK&#xA;VTQVSFPLIQVIYILATPNYKALKIISYPIAGSTCIYYTLLITNLLHI&#xA;QYA&#xA;&#xA;Y&#xA;CI&#xA;IIS&#xA;KKITWVLLT&#xA;LHLEYIKF&#xA;RINISKVLDITKNNSE&#xA;F&#xA;ILLHIILLAYLFHSFQNVACKRY&#xA;YSALFCCNS&#xA;FFHYIILRVRKRLTFSQ&#xA;KEQT&#xA;ILKFYCFCLPDLFQI&#xA;FFPANKTINYPEPNSKR&#xA;SAGLELSSPNINFALSSASIKSPDKNSFNSS&#xA;MFVCG&#xA;LSIVPA&#xA;LWLPTESYNGGSQLCNSALLFCSFIKAFGTDLISIILSSLFSLLFCPELAWWLLVKGDNAATELFFSSNSFIFANNAFFFRACISKISARYCKYSTVLSKMIPLFGLICLPRSS&#xA;FNKDGGTNWPSSFIFCFINSLLLLSTAL&#xA;NSLLRLSFSSEVNPPKIFGTLPGPIFSVILLLIEVCLVLGVFMLPCLKEDERKLPARNDDDGDLTFGEYVEDTDAGPKLCGL&#xA;ENDDEYVFGVIISELMLEDKEELKSSVKLYIVSSVPC&#xA;NKKSCGNSAFKPSCCCNGVFIA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;pre&gt;&lt;code&gt;myseq = &amp;quot;GCATGCAATACAGTGACATATATATATACCCTAACACTACCCTAACCCTACCCTATTTCA\&#xA;ACCCTTCCAACCTGTCTCTCAACTTACCCTCACATTACCCTACCTCTCCACTTGTTACCC\&#xA;TGTCCCATTCAACCATACCACTCCCAACCACCATCCATCCCTCTACTTACTACCACCAAT\&#xA;CAACCGTCCACCATAACCGTTACCCTCCAATTAGCCATATTCAACTTCACTACCACTTAC\&#xA;CCTGCCATTACTCTACCATCCACCATCTGCTACTCACCATACTGTTGTTCTACCCTCCAT\&#xA;ATTGAAACGTTAACAAATGATCGTAAATAATACACATATACTTACCCTACCACTTTATAC\&#xA;CACCACACATCACATGCCATACTCACCTTCACTTGTATACTGATATGCCATACGCACACG\&#xA;GATGCTACAGTATATACCACTCTCAAACTTACCCTACTCTCACATTCTACTCCACTCCAT\&#xA;GACCCATCTCTCACTAAATCAGTACTAAATGCACCCACATCATTATGCACGGCACTTGCC\&#xA;TCAGCGGTCTATACCCTGAGCCATTTACCCATAACTCCCACGATTATCCACATTTTAATA\&#xA;TCTATATCTCATTCGGCGGGCCCAAATATTGTATAACTGCTCTTAATACATACGTTATAC\&#xA;CACTTTTGCACCATATACTAACCACTCAAATTATATACACTTATGCCAATATAACCAAAA\&#xA;AATTACCACTAAAATCACCTAAACATAAAAATTATTTATCTTTCAACTTTACGAAATAAA\&#xA;CACACTCAATTGCGTATCTATACCACCATGACGTCATTAACGTAAAAGTTCCTTAATATT\&#xA;ACCATTTGCTTGAACGGATACCATTTCAGAATATTTCTAACTTTCACAGACCATACATTA\&#xA;GAATAATATGCCACCTCACTGTCGTAACACTCTATATTCACCGAGAAACAATACGGTAGT\&#xA;GGCTCAAACTCATGCCGGTGCTATGATACAATTGTATCCTATTTCCATTCTCATATGCTA\&#xA;TCCGCAATATCCTAAAAGCATAACTGATGCATCTTTAATCTTGTATGTGACACTACTCAT\&#xA;ACGAAGAGACTATATCTAAAGAAGACGATACAGTGATATGTACGTTGTTTTTGTAGAATT\&#xA;ATAATGAAACGTCAAATAACCCTACTATATTATAACTTATCAGCGCCGTATACTAAAACG\&#xA;GACGTTACGATATTGTCTCACTTCATCTTACCACCCTCTATCTTATTGTTGATAAAACAC\&#xA;TAACCCCTCAGCTTTATTTCTAGTTACAGTTACAACAAACTATCCCAAACCATAAATCTT\&#xA;AATATTTTAGGTGTCAAAAAATGAGGATCTCCAAATGAGAGTTTGGTACCATGACTTGTA\&#xA;ACTCCACTACCCTGATCTGCAATCTTGTTCTTAGAAGTGACGCATACTCTATATGGCCCG\&#xA;ACGCGACGCGCCAAAAAATGAAAAAAGAAGCAGCGACTCATTTTTATGGAAGGACAAAGT\&#xA;GCTGCGAAGTCATACGCTTCCAATTTCATTATTGTTTATTGGACATACTCTGTTAGCTTT\&#xA;ATTACCGTCCACGCTTTTTCTACAATAGTGTAAAAGTTTCTTTCTTATGTTCATCGTATT\&#xA;CATAAAATGCTTCACGAACACCGTCATTGATCAAATAGGTTTATAATATTAATATACATT\&#xA;TATATAATCGGCGGTATTTATATCATCAAAAAAAGTAGTTTTTTATTTTATTTTTTCATT\&#xA;ACTTTTCACTGTCTATGGATTTTCATTCGTAAAGGCATCACTCCCTAGTTTGCGATAGTG\&#xA;TAGATACCGTCCTTGGATAGAGCACTGGAGATGGCTGGCTTTAATCTGCTGGAGTACCAT\&#xA;GGAACACCGGTGATCATTCTGGTCACTTGGTCTGGGGCAATACCAGTCAACATGGTGGTG\&#xA;AAGTCACCGTAGTTGAAAACGGCTTCAGCAACTTCGACTGGGTAGGTTTCAGTTGGGTGG\&#xA;GCGGCTTGGAACATGTAGTATTGGGCCAAGTGAGCTCTGATATCAGAGACGTAGACACCC\&#xA;AATTCCACCAAGTTGACTCTTTCGTCAGATTGAGCTAGAGTGGTGGTTGCAGAAAGCAGT\&#xA;AGCAGCGATGGCAGCGACACCAGCAGCGATTGAAGTTAATTTGACCATTGTATTTGTTTT\&#xA;GTTTGTTAGTGCTGATGTAATCTTAACAAGAAATAGTGAAATGAAAGCGCATACCTCAAA\&#xA;GGCATATAGTTGAAGCAGCTCTATTTATACCCGTTCCTCCATCTGTCATCACTACTTAAA\&#xA;CGATTCGTTAACAGACGCTCATTTAGCACCTCACATATTCTCCATATCTCATCTTTCACA\&#xA;CAATCTCATTATCTCTATGGAGATGCTCTTGTTTCTGAACGAATCATACATCTTTCATAG\&#xA;GTTTCGTATGTGGAGTATTGTTTTATGGCACTCATGTGTATTCGTATGCGCAGAATGTGG\&#xA;GAATGCCTATTATAGGGGTGCCGGGGGGTGCCTTGAAAAACCCTTTTGCGCGCCTGTTAA\&#xA;GTTTCCGTTTTCAGTCAAAAAGAATATCCGAATTTTAGATTTGGACCCTCGTTCAGAAGC\&#xA;TTACTGTCTAAGCCATCATTTGGTGTGTCCTAAACGGTTTCCATGCAAAGCAACATCTTT\&#xA;GTTACTCATTCCTGAAGGTTGAATAAAAAATAGCAACCATGTGCAGGGAGTCGTATATTG\&#xA;TTAGATTCTAGAGACTTGTACGCATCATCAAAGCTGTAAATAGAATAAACATACGCAAGG\&#xA;CGTCAAAAGTGCATAGTTAAGAAAATTCCTGACATGTGAAAATATGTGTTTATGAAATGT\&#xA;GTCAAGGCCCGTCTATAGCGTAGTTAACCCCTCTGCAGGAGTAAGTGACTTTTTTTACGC\&#xA;TCAAAAGGCAACGAGGGCACATACTTAAAAGTCATTTTCAAACACATCTGCAGTTTGCAA\&#xA;CGACAGATAACAATATTATGATAGGATGGTATGATGTTATTAGCTGCCACATATTTTTAA\&#xA;TAATAGCATTAGTCACGTCTCTTCAATTGTTGGGATGAAACTCTAAAATATCATTCCTTT\&#xA;AGTAGTATTCCAGTTACCAGTATATTATCACATGCCGAAAAAGAAGATGACATAAAGATC\&#xA;GACAAACAGTCTTCAAATATAATGGAAGCTGGAATGCAAGGATTGATAATGTAACAGGAT\&#xA;ACTGAATGACAAAGTATAAATGAAAAAAAAAAAAAAAAAAAAGTAGTAATACTATTATGT\&#xA;GGAAATACCGATTCCATTTTGAGGATTCCAATTGTTGGAATAAAAATCAACTATCATCTA\&#xA;CTAACTAGTATTTACGTTACTAGTATATTATCATATACGGTGTTAGAAGATGACGCAAAT\&#xA;GATGAGAAATAGTCATCTAAATTAGTGGAAGCTGAAACGCAAGGATTGATAATGTAATAG\&#xA;GATCAATGAATATTAACATATAAAACGATGATAATAATATTTATAGAATTGTGTAGAATT\&#xA;GCAGATTCCCTTTTATGGATTCCTAAATCCTGGAGGAGAACTTCTAGTATATCTACATAC\&#xA;CTAATATTATAGCCTTAATCACAATGGAATCCCAACAATTACATCAAAATCCACATTCTC\&#xA;TACACCAATACCATCGACGAGAGCTTCTAGTAAATTGTATACATAACAGTATAACCCTTA\&#xA;CCAACAATGGAATCTCAAAGATTATTAAATTATTCACAGACTCTGAGGATTCGGGTAAAA\&#xA;TAGGGTATTTAACTGGTTACCGGAAAGGTTTAGAAAATTCGTGGAGGGTTGGCCGAGTGG\&#xA;TCTAAGGCGGCAGACTTAAGATCTGTTGGACGGTTGTCCGCGCGAGTTCGAACCTCGCAT\&#xA;TCATTTTCCGGATCCACTTTATAGTGGGTCACTTCATCTCTTGTAGGGTTCACTGAAAGA\&#xA;TAATCAATCAAATTATCATATGACCAATGGGTCACTTTTTTGATGAGACCCATGTTTATT\&#xA;CTTCTATACGTTCGTGATACTGCACTTACGACTACCAGTAACATCAGAATAACCATAGAG\&#xA;CTTATGCTATCTTGAAATATATCATCGCCGTAAAGTATGATGTGTAGCGCATTAGTCTTC\&#xA;TGCATTCATATTATCAGTGGTATACCGTTTGGTATATGCTTTCCAATCATGAAGTGTTGG\&#xA;CTCTACTCTATATAAATTCGACATCCATTGATTTTCTCAATAAATGAGTTACCCAAGTAA\&#xA;GCTTTCCATTGATACAAGTGATCTACATTCTTGCGACGCCAAATTATAAAGCACTAAAAA\&#xA;TCATATCATACCCAATTGCGGGCAGCACATGTATATATTATACACTACTTATAACTAACC\&#xA;TTCTTCATATATAACAATATGCCTGATAATACTAATGCATTTAAATCATATCATGAAAGA\&#xA;AAATTACATGGGTTTTATTGACATAATTGCATTTAGAATACATAAAATTCTAAAGAATTA\&#xA;ATATATCCAAAGTATTAGACATAACCAAGAATAATAGTGAATAATTTTAGATTTTGTTAC\&#xA;ATATAATTCTGCTTGCCTATCTCTTCCACTCTTTTCAAAACGTTGCATGTAAGCGTTACT\&#xA;AATATTCCGCTTTATTTTGTTGCAATTCCTAATTTTTTCATTACATTATCTTGCGAGTAC\&#xA;GGAAGCGATTAACGTTCTCCCAATAGAAGGAACAAACATAGATATTGAAGTTTTACTGCT\&#xA;TTTGCTTACCTGACCTTTTTCAAATTTAATTTTTTCCCGCTAATAAGACCATAAACTACC\&#xA;CCGAACCAAATTCTAAAAGATAGTCAGCTGGATTAGAGTTGTCATCTCCAAACATTAATT\&#xA;TTGCATTATCTTCGGCTTCAATCAAATCGCCTGATAAGAACTCCTTTAATTCTTCATGAA\&#xA;TGTTTGTATGTGGATGACTCTCCATAGTGCCAGCATGATTGTGGTTACCGACCGAATCAT\&#xA;ATAACGGTGGCTCCCAATTGTGCAATTCAGCCTTACTATTTTGTTCATTCATCAAAGCAT\&#xA;TTGGGACAGATCTAATATCTATAATTCTTTCCTCACTATTCTCGCTATTATTTTGCCCCG\&#xA;AACTGGCATGGTGGTTATTGGTAAAAGGAGATAATGCTGCGACAGAACTTTTCTTCTCTT\&#xA;CCAATTCTTTTATCTTCGCCAATAACGCCTTCTTTTTTCGTGCTTGTATTTCTAAAATTT\&#xA;CGGCCAGGTATTGTAAATATTCGACCGTTCTATCCAAAATGATACCCTTATTTGGTTTGA\&#xA;TTTGTTTACCTAGGTCATCGTAATTCAATAAAGATGGTGGAACCAACTGGCCGAGTTCTT\&#xA;TTATCTTTTGCTTTATTAATTCTCTTCTTCTCCTTTCGACGGCATTATGAAACTCTCTTT\&#xA;TGCGCCTCAGTTTCTCATCAGAAGTTAACCCGCCTAAAATCTTTGGAACACTTCCAGGTC\&#xA;CTATATTTTCAGTCATATTGCTACTTATTGAAGTGTGTCTTGTTCTGGGTGTGTTTATGC\&#xA;TACCATGCCTAAAAGAAGATGAAAGGAAACTTCCTGCACGAAATGACGATGATGGTGATC\&#xA;TTACCTTTGGGGAATATGTGGAAGATACGGATGCTGGGCCCAAGCTTTGTGGATTATAAG\&#xA;AAAATGATGATGAATATGTGTTTGGTGTCATCATATCAGAATTGATGCTAGAAGATAAAG\&#xA;AGGAGCTCAAATCATCAGTTAAGTTATACATTGTGTCATCGGTGCCATGCTGAAATAAAA\&#xA;AGTCATGTGGTAACTCAGCTTTTAAGCCTTCTTGTTGCTGTAATGGAGTATTCATTGCTC&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-25T14:25:12.093" LastActivityDate="2023-01-25T14:25:12.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20444" PostTypeId="1" CreationDate="2023-01-25T16:14:55.373" Score="2" ViewCount="33" Body="&lt;p&gt;We have two 10X Genomics scRNA-seq and scATAC-seq datasets generated from similar cells, but in different experiments - separately, not using the multiome platform. Now, we are trying to use Pando (&lt;a href=&quot;https://github.com/quadbiolab/Pando&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/quadbiolab/Pando&lt;/a&gt;) to generate a gene regulatory network.&lt;/p&gt;&#xA;&lt;p&gt;We have successfully converted our CCA-embedded dataset into h5ad format (&lt;a href=&quot;https://mojaveazure.github.io/seurat-disk/articles/convert-anndata.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://mojaveazure.github.io/seurat-disk/articles/convert-anndata.html&lt;/a&gt;), then used bipartite matching (&lt;a href=&quot;https://github.com/quadbiolab/organoid_regulomes&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/quadbiolab/organoid_regulomes&lt;/a&gt;) to receive a .tsv file matching invidiual cells from both datasets - it gives us the barcodes of the cells in both the scRNA and scATAC datasets. Now, we need to actually generate metacells/pseudocells that contain both modalities inside the same cell to be able to use Pando.&lt;/p&gt;&#xA;&lt;p&gt;According to Fleck et al. 2022 (&lt;a href=&quot;https://www.nature.com/articles/s41586-022-05279-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41586-022-05279-8&lt;/a&gt;), they state&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The metacells were integrated using cluster similarity spectrum (CSS)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;which would refer to this technique: &lt;a href=&quot;https://github.com/quadbiolab/simspec/blob/master/vignette/vignette.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/quadbiolab/simspec/blob/master/vignette/vignette.md&lt;/a&gt;&#xA;However, that does not make use of the previously generated bipartite-matched cell list.&lt;/p&gt;&#xA;&lt;p&gt;I'm a little bit lost here - can someone point me in the right direction to how we can generate the metacells containing both modalities, scRNA and scATAC?&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot in advance!&lt;/p&gt;&#xA;" OwnerUserId="16826" LastActivityDate="2023-01-25T16:14:55.373" Title="Creating meta cells from scRNA-seq and scATAC-seq for GRN analysis" Tags="|scrnaseq|atac-seq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20445" PostTypeId="1" AcceptedAnswerId="20449" CreationDate="2023-01-25T18:12:18.933" Score="4" ViewCount="47" Body="&lt;p&gt;I need a quick way to trim multiple reads in a FASTA file. I need to trim everything that is 1kbp upstream of this sequence &lt;code&gt;AAGAGATGTTCAATCGTTTAAACAAATTCCAAGCTGCTTTAGCTTTGGCCCTTTACTCTCA&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I figure a quick python script might be the way to go but I'm not sure if there's a tool that already does that (i.e. Trimmomatic or something like it) that would be able to do that for me.&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="16067" LastActivityDate="2023-01-26T12:44:12.530" Title="Trim reads 1kb upstream of sequence" Tags="|python|trimming|multi-fasta|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20446" PostTypeId="2" ParentId="20426" CreationDate="2023-01-25T19:39:16.657" Score="2" Body="&lt;p&gt;&lt;em&gt;Answer from @terdon, @Ben_Ensembl, converted from &lt;a href=&quot;https://bioinformatics.stackexchange.com/help/privileges/comment&quot;&gt;comments&lt;/a&gt;:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Genes don't have exons, &lt;em&gt;transcripts&lt;/em&gt; have exons. There are &lt;a href=&quot;https://www.ensembl.org/Homo_sapiens/Transcript/Summary?db=core;g=ENSG00000034713;r=16:75566375-75577881;t=ENST00000565057&quot; rel=&quot;nofollow noreferrer&quot;&gt;five transcripts for ENSG00000034713&lt;/a&gt;, one with one exon, two with three exons, and two with four exons. No transcripts with five exons though.&lt;/p&gt;&#xA;&lt;p&gt;The count you retrieved from the GTF will be the number of unique exons for all transcripts of each gene (some unique to a single transcript, some common between two or more transcripts).&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;[please improve this answer if you can by editing it]&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-01-25T19:39:16.657" CommentCount="0" CommunityOwnedDate="2023-01-25T19:39:16.657" ContentLicense="CC BY-SA 4.0" />
  <row Id="20447" PostTypeId="2" ParentId="18351" CreationDate="2023-01-25T21:53:26.980" Score="3" Body="&lt;p&gt;It depends on the task. If you want to recover TCR and BCR repertoires along with gene expression - 5' is a must. Also, for an overview repertoire analysis you dont even have to perform target enrichment and use 5' RNA-seq daata to directly recover the repertoire.&lt;/p&gt;&#xA;&lt;p&gt;Its easy to do with MiXCR software in one line command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mixcr analyze 10x-5gex-full-length \&#xA; --species hsa \&#xA; sample_R1.fastq.gz \&#xA; sample_R2.fastq.gz \&#xA; sample_result&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Though MiXCR currently only supports repertoire analysis it will also support gene expression soon.&lt;/p&gt;&#xA;&lt;p&gt;More on that here:&#xA;&lt;a href=&quot;https://docs.milaboratories.com/mixcr/reference/overview-built-in-presets/#10xgenomics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.milaboratories.com/mixcr/reference/overview-built-in-presets/#10xgenomics&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Disclaimer: I work for the company that develops MiXCR.&lt;/p&gt;&#xA;" OwnerUserId="16818" LastEditorUserId="298" LastEditDate="2023-01-27T10:59:16.820" LastActivityDate="2023-01-27T10:59:16.820" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20448" PostTypeId="2" ParentId="20442" CreationDate="2023-01-25T22:44:53.120" Score="1" Body="&lt;p&gt;ok I have a Python attempt, used &lt;code&gt;Biopython&lt;/code&gt; only to load your DNA sequence;&#xA;working only on your sequence ( not calculating the reverse complement) and as per your instruction an ORF is something in between an ATG and a stop codon, so I am using every ATG as a start even if preceded by another ATG, lenght of ORFs is &amp;gt;30bp meaning &amp;gt;90 bases. Code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;# record = SeqIO.read(&amp;quot;sample_test.fasta&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;# record = SeqIO.read(&amp;quot;sample_test_2.fasta&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;&#xA;record = SeqIO.read(&amp;quot;sample.fasta&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;&#xA;# print(record.seq)&#xA;# print(len(sequence))&#xA;&#xA;sequence = record.seq&#xA;&#xA;orfs = []&#xA;&#xA;for i in range(len(sequence)-2):&#xA;    &#xA;    if sequence[i]+sequence[i+1]+sequence[i+2] == 'ATG':&#xA;    &#xA;        print('ATG at : ', i)&#xA;                &#xA;        orf = 'ATG'&#xA;        x = 3&#xA;        while sequence[i+x:i+3+x] not in ['TAG', 'TAA', 'TGA'] and i+x &amp;lt; len(sequence):                    &#xA;            orf += sequence[i+x:i+3+x]&#xA;            x += 3&#xA;        if len(orf) &amp;gt; 90:   &#xA;            orfs.append((i, orf , len(orf)))&#xA;&#xA;cnt = 1&#xA;for i in orfs:&#xA;    &#xA;    print('\nsequence number : ', cnt ,'\nstart : ', i[0] ,'\nframe : ',  ((len(sequence)-len(sequence[i[0]:]))%3)+1 ,&#xA;          '\nsequence : ', i[1],'\nlenght sequence :', i[2],'\n\n')&#xA;    cnt += 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Result, start is with a sequence that goes from 0 to len(sequence)-1:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sequence number :  1 &#xA;start :  2 &#xA;frame :  3 &#xA;sequence :  ATGCAATACAGTGACATATATATATACCCTAACACTACCCTAACCCTACCCTATTTCAACCCTTCCAACCTGTCTCTCAACTTACCCTCACATTACCCTACCTCTCCACTTGTTACCCTGTCCCATTCAACCATACCACTCCCAACCACCATCCATCCCTCTACTTACTACCACCAATCAACCGTCCACCATAACCGTTACCCTCCAATTAGCCATATTCAACTTCACTACCACTTACCCTGCCATTACTCTACCATCCACCATCTGCTACTCACCATACTGTTGTTCTACCCTCCATAT &#xA;lenght sequence : 300 &#xA;&#xA;&#xA;&#xA;sequence number :  2 &#xA;start :  524 &#xA;frame :  3 &#xA;sequence :  ATGCACGGCACTTGCCTCAGCGGTCTATACCCTGAGCCATTTACCCATAACTCCCACGATTATCCACATTTTAATATCTATATCTCATTCGGCGGGCCCAAATATTGTATAACTGCTCTTAATACATACGTTATACCACTTTTGCACCATATACTAACCACTCAAATTATATACACTTATGCCAATATAACCAAAAAATTACCACTAAAATCACCTAAACATAAAAATTATTTATCTTTCAACTTTACGAAA &#xA;lenght sequence : 252 &#xA;&#xA;&#xA;&#xA;sequence number :  3 &#xA;start :  907 &#xA;frame :  2 &#xA;sequence :  ATGCCACCTCACTGTCGTAACACTCTATATTCACCGAGAAACAATACGGTAGTGGCTCAAACTCATGCCGGTGCTATGATACAATTGTATCCTATTTCCATTCTCATATGCTATCCGCAATATCCTAAAAGCATAACTGATGCATCTTTAATCTTGTATGTGACACTACTCATACGAAGAGACTATATC &#xA;lenght sequence : 189 &#xA;&#xA;&#xA;&#xA;sequence number :  4 &#xA;start :  982 &#xA;frame :  2 &#xA;sequence :  ATGATACAATTGTATCCTATTTCCATTCTCATATGCTATCCGCAATATCCTAAAAGCATAACTGATGCATCTTTAATCTTGTATGTGACACTACTCATACGAAGAGACTATATC &#xA;lenght sequence : 114 &#xA;&#xA;&#xA;&#xA;sequence number :  5 &#xA;start :  1143 &#xA;frame :  1 &#xA;sequence :  ATGAAACGTCAAATAACCCTACTATATTATAACTTATCAGCGCCGTATACTAAAACGGACGTTACGATATTGTCTCACTTCATCTTACCACCCTCTATCTTATTGTTGATAAAACAC &#xA;lenght sequence : 117 &#xA;&#xA;&#xA;&#xA;sequence number :  6 &#xA;start :  1370 &#xA;frame :  3 &#xA;sequence :  ATGACTTGTAACTCCACTACCCTGATCTGCAATCTTGTTCTTAGAAGTGACGCATACTCTATATGGCCCGACGCGACGCGCCAAAAAATGAAAAAAGAAGCAGCGACTCATTTTTATGGAAGGACAAAGTGCTGCGAAGTCATACGCTTCCAATTTCATTATTGTTTATTGGACATACTCTGT &#xA;lenght sequence : 183 &#xA;&#xA;&#xA;&#xA;sequence number :  7 &#xA;start :  1432 &#xA;frame :  2 &#xA;sequence :  ATGGCCCGACGCGACGCGCCAAAAAATGAAAAAAGAAGCAGCGACTCATTTTTATGGAAGGACAAAGTGCTGCGAAGTCATACGCTTCCAATTTCATTATTGTTTATTGGACATACTCTGTTAGCTTTATTACCGTCCACGCTTTTTCTACAA &#xA;lenght sequence : 153 &#xA;&#xA;&#xA;&#xA;sequence number :  8 &#xA;start :  1457 &#xA;frame :  3 &#xA;sequence :  ATGAAAAAAGAAGCAGCGACTCATTTTTATGGAAGGACAAAGTGCTGCGAAGTCATACGCTTCCAATTTCATTATTGTTTATTGGACATACTCTGT &#xA;lenght sequence : 96 &#xA;&#xA;&#xA;&#xA;sequence number :  9 &#xA;start :  1485 &#xA;frame :  1 &#xA;sequence :  ATGGAAGGACAAAGTGCTGCGAAGTCATACGCTTCCAATTTCATTATTGTTTATTGGACATACTCTGTTAGCTTTATTACCGTCCACGCTTTTTCTACAATAGTG &#xA;lenght sequence : 105 &#xA;&#xA;&#xA;&#xA;sequence number :  10 &#xA;start :  1830 &#xA;frame :  1 &#xA;sequence :  ATGGCTGGCTTTAATCTGCTGGAGTACCATGGAACACCGGTGATCATTCTGGTCACTTGGTCTGGGGCAATACCAGTCAACATGGTGGTGAAGTCACCG &#xA;lenght sequence : 99 &#xA;&#xA;&#xA;&#xA;sequence number :  11 &#xA;start :  2107 &#xA;frame :  2 &#xA;sequence :  ATGGCAGCGACACCAGCAGCGATTGAAGTTAATTTGACCATTGTATTTGTTTTGTTTGTTAGTGCTGATGTAATCTTAACAAGAAATAGTGAAATGAAAGCGCATACCTCAAAGGCATATAGT &#xA;lenght sequence : 123 &#xA;&#xA;&#xA;&#xA;sequence number :  12 &#xA;start :  2356 &#xA;frame :  2 &#xA;sequence :  ATGGAGATGCTCTTGTTTCTGAACGAATCATACATCTTTCATAGGTTTCGTATGTGGAGTATTGTTTTATGGCACTCATGTGTATTCGTATGCGCAGAATGTGGGAATGCCTATTATAGGGGTGCCGGGGGGTGCCTTGAAAAACCCTTTTGCGCGCCTGTTAAGTTTCCGTTTTCAGTCAAAAAGAATATCCGAATTTTAGATTTGGACCCTCGTTCAGAAGCTTACTGTCTAAGCCATCATTTGGTGTGTCCTAAACGGTTTCCATGCAAAGCAACATCTTTGTTACTCATTCCTGAAGGT &#xA;lenght sequence : 303 &#xA;&#xA;&#xA;&#xA;sequence number :  13 &#xA;start :  2362 &#xA;frame :  2 &#xA;sequence :  ATGCTCTTGTTTCTGAACGAATCATACATCTTTCATAGGTTTCGTATGTGGAGTATTGTTTTATGGCACTCATGTGTATTCGTATGCGCAGAATGTGGGAATGCCTATTATAGGGGTGCCGGGGGGTGCCTTGAAAAACCCTTTTGCGCGCCTGTTAAGTTTCCGTTTTCAGTCAAAAAGAATATCCGAATTTTAGATTTGGACCCTCGTTCAGAAGCTTACTGTCTAAGCCATCATTTGGTGTGTCCTAAACGGTTTCCATGCAAAGCAACATCTTTGTTACTCATTCCTGAAGGT &#xA;lenght sequence : 297 &#xA;&#xA;&#xA;&#xA;sequence number :  14 &#xA;start :  2407 &#xA;frame :  2 &#xA;sequence :  ATGTGGAGTATTGTTTTATGGCACTCATGTGTATTCGTATGCGCAGAATGTGGGAATGCCTATTATAGGGGTGCCGGGGGGTGCCTTGAAAAACCCTTTTGCGCGCCTGTTAAGTTTCCGTTTTCAGTCAAAAAGAATATCCGAATTTTAGATTTGGACCCTCGTTCAGAAGCTTACTGTCTAAGCCATCATTTGGTGTGTCCTAAACGGTTTCCATGCAAAGCAACATCTTTGTTACTCATTCCTGAAGGT &#xA;lenght sequence : 252 &#xA;&#xA;&#xA;&#xA;sequence number :  15 &#xA;start :  2462 &#xA;frame :  3 &#xA;sequence :  ATGCCTATTATAGGGGTGCCGGGGGGTGCCTTGAAAAACCCTTTTGCGCGCCTGTTAAGTTTCCGTTTTCAGTCAAAAAGAATATCCGAATTT &#xA;lenght sequence : 93 &#xA;&#xA;&#xA;&#xA;sequence number :  16 &#xA;start :  3447 &#xA;frame :  1 &#xA;sequence :  ATGATAATAATATTTATAGAATTGTGTAGAATTGCAGATTCCCTTTTATGGATTCCTAAATCCTGGAGGAGAACTTCTAGTATATCTACATACCTAATATTA &#xA;lenght sequence : 102 &#xA;&#xA;&#xA;&#xA;sequence number :  17 &#xA;start :  3563 &#xA;frame :  3 &#xA;sequence :  ATGGAATCCCAACAATTACATCAAAATCCACATTCTCTACACCAATACCATCGACGAGAGCTTCTAGTAAATTGTATACATAACAGTATAACCCTTACCAACAATGGAATCTCAAAGATTATTAAATTATTCACAGACTCTGAGGATTCGGGTAAAATAGGGTATTTAACTGGTTACCGGAAAGGTTTAGAAAATTCGTGGAGGGTTGGCCGAGTGGTC &#xA;lenght sequence : 219 &#xA;&#xA;&#xA;&#xA;sequence number :  18 &#xA;start :  4056 &#xA;frame :  1 &#xA;sequence :  ATGATGTGTAGCGCATTAGTCTTCTGCATTCATATTATCAGTGGTATACCGTTTGGTATATGCTTTCCAATCATGAAGTGTTGGCTCTACTCTATA &#xA;lenght sequence : 96 &#xA;&#xA;&#xA;&#xA;sequence number :  19 &#xA;start :  4059 &#xA;frame :  1 &#xA;sequence :  ATGTGTAGCGCATTAGTCTTCTGCATTCATATTATCAGTGGTATACCGTTTGGTATATGCTTTCCAATCATGAAGTGTTGGCTCTACTCTATA &#xA;lenght sequence : 93 &#xA;&#xA;&#xA;&#xA;sequence number :  20 &#xA;start :  5073 &#xA;frame :  1 &#xA;sequence :  ATGCTGCGACAGAACTTTTCTTCTCTTCCAATTCTTTTATCTTCGCCAATAACGCCTTCTTTTTTCGTGCTTGTATTTCTAAAATTTCGGCCAGGTATTGTAAATATTCGACCGTTCTATCCAAAA &#xA;lenght sequence : 126 &#xA;&#xA;&#xA;&#xA;sequence number :  21 &#xA;start :  5253 &#xA;frame :  1 &#xA;sequence :  ATGGTGGAACCAACTGGCCGAGTTCTTTTATCTTTTGCTTTATTAATTCTCTTCTTCTCCTTTCGACGGCATTATGAAACTCTCTTTTGCGCCTCAGTTTCTCATCAGAAGTTAACCCGCCTAAAATCTTTGGAACACTTCCAGGTCCTATATTTTCAGTCATATTGCTACTTATTGAAGTGTGTCTTGTTCTGGGTGTGTTTATGCTACCATGCC &#xA;lenght sequence : 216 &#xA;&#xA;&#xA;&#xA;sequence number :  22 &#xA;start :  5456 &#xA;frame :  3 &#xA;sequence :  ATGCTACCATGCCTAAAAGAAGATGAAAGGAAACTTCCTGCACGAAATGACGATGATGGTGATCTTACCTTTGGGGAATATGTGGAAGATACGGATGCTGGGCCCAAGCTTTGTGGATTA &#xA;lenght sequence : 120 &#xA;&#xA;&#xA;&#xA;sequence number :  23 &#xA;start :  5478 &#xA;frame :  1 &#xA;sequence :  ATGAAAGGAAACTTCCTGCACGAAATGACGATGATGGTGATCTTACCTTTGGGGAATATGTGGAAGATACGGATGCTGGGCCCAAGCTTTGTGGATTATAAGAAAATGATGATGAATATGTGTTTGGTGTCATCATATCAGAAT &#xA;lenght sequence : 144 &#xA;&#xA;&#xA;&#xA;sequence number :  24 &#xA;start :  5502 &#xA;frame :  1 &#xA;sequence :  ATGACGATGATGGTGATCTTACCTTTGGGGAATATGTGGAAGATACGGATGCTGGGCCCAAGCTTTGTGGATTATAAGAAAATGATGATGAATATGTGTTTGGTGTCATCATATCAGAAT &#xA;lenght sequence : 120 &#xA;&#xA;&#xA;&#xA;sequence number :  25 &#xA;start :  5508 &#xA;frame :  1 &#xA;sequence :  ATGATGGTGATCTTACCTTTGGGGAATATGTGGAAGATACGGATGCTGGGCCCAAGCTTTGTGGATTATAAGAAAATGATGATGAATATGTGTTTGGTGTCATCATATCAGAAT &#xA;lenght sequence : 114 &#xA;&#xA;&#xA;&#xA;sequence number :  26 &#xA;start :  5511 &#xA;frame :  1 &#xA;sequence :  ATGGTGATCTTACCTTTGGGGAATATGTGGAAGATACGGATGCTGGGCCCAAGCTTTGTGGATTATAAGAAAATGATGATGAATATGTGTTTGGTGTCATCATATCAGAAT &#xA;lenght sequence : 111 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am sure there is some awk, perl, regex way to get the same in much less lines.&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2023-01-26T16:26:59.727" LastActivityDate="2023-01-26T16:26:59.727" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20449" PostTypeId="2" ParentId="20445" CreationDate="2023-01-26T12:31:19.623" Score="2" Body="&lt;p&gt;Here's one way using &lt;a href=&quot;/questions/tagged/biopython&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;biopython&amp;#39;&quot; aria-label=&quot;show questions tagged &amp;#39;biopython&amp;#39;&quot; rel=&quot;tag&quot; aria-labelledby=&quot;biopython-container&quot;&gt;biopython&lt;/a&gt;. Note that the &lt;a href=&quot;https://biopython.org/wiki/Seq&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;Seq&lt;/code&gt;&lt;/a&gt; object has a number of methods that act just like those of a Python string. One of these is the &lt;a href=&quot;https://biopython.org/docs/1.80/api/Bio.Seq.html#Bio.Seq.SequenceDataAbstractBaseClass.find&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;find()&lt;/code&gt;&lt;/a&gt; method, which returns the index of the first occurrence of the sequence if found. Replace with &lt;a href=&quot;https://biopython.org/docs/1.80/api/Bio.Seq.html#Bio.Seq.SequenceDataAbstractBaseClass.rfind&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;rfind()&lt;/code&gt;&lt;/a&gt; if you want the last occurrence if that makes more sense. This solution uses a generator to avoid storing the entire list of trimmed sequences in memory:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import argparse&#xA;import contextlib&#xA;import gzip&#xA;import sys&#xA;&#xA;from Bio import SeqIO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;def trim_fasta(fileobj, sequence, num_bases):&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    Trim num_bases upstream of the sequence provided&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    for record in SeqIO.parse(fileobj, 'fasta'):&#xA;        index = record.seq.find(sequence)&#xA;        if index != -1 and index &amp;gt; num_bases:&#xA;            record = record[index-num_bases:]&#xA;        yield record&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;def open_file(filename, mode='r'):&#xA;    if filename.endswith('.gz'):&#xA;        return gzip.open(filename, mode)&#xA;    else:&#xA;        return open(filename, mode)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;def get_argument_parser():&#xA;&#xA;    parser = argparse.ArgumentParser(add_help=False)&#xA;&#xA;    group = parser.add_argument_group('input options')&#xA;    group.add_argument(&#xA;        &amp;quot;fasta&amp;quot;,&#xA;        type=str,&#xA;        metavar=&amp;quot;FILE&amp;quot;,&#xA;        help=&amp;quot;The input FASTA file&amp;quot;,&#xA;    )&#xA;    group.add_argument(&#xA;        &amp;quot;sequence&amp;quot;,&#xA;        type=str,&#xA;        metavar=&amp;quot;STR&amp;quot;,&#xA;        help=&amp;quot;The sequence to find and trim&amp;quot;,&#xA;    )&#xA;&#xA;    group = parser.add_argument_group('trimming options')&#xA;    group.add_argument(&#xA;        &amp;quot;-n&amp;quot;,&#xA;        &amp;quot;--num_bases&amp;quot;,&#xA;        type=int,&#xA;        metavar=&amp;quot;INT&amp;quot;,&#xA;        default=1000,&#xA;        help=&amp;quot;Trim INT bases upstream of input sequence&amp;quot;,&#xA;    )&#xA;&#xA;    group = parser.add_argument_group('output options')&#xA;    group.add_argument(&#xA;        &amp;quot;-o&amp;quot;,&#xA;        &amp;quot;--output&amp;quot;,&#xA;        type=str,&#xA;        metavar=&amp;quot;FILE&amp;quot;,&#xA;        default='-',&#xA;        help=&amp;quot;Write the filtered output to FILE (default: stdout)&amp;quot;,&#xA;    )&#xA;    group.add_argument(&#xA;        &amp;quot;-f&amp;quot;,&#xA;        &amp;quot;--force&amp;quot;,&#xA;        action='store_true',&#xA;        help=&amp;quot;Overwrite the output file if it exists&amp;quot;,&#xA;    )&#xA;&#xA;    group = parser.add_argument_group('additional options')&#xA;    group.add_argument(&#xA;        &amp;quot;-h&amp;quot;,&#xA;        &amp;quot;--help&amp;quot;,&#xA;        action=&amp;quot;help&amp;quot;,&#xA;        help=&amp;quot;Show this help message and exit&amp;quot;,&#xA;    )&#xA;&#xA;    return parser&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;def main():&#xA;&#xA;    parser = get_argument_parser()&#xA;    args = parser.parse_args()&#xA;&#xA;    with contextlib.ExitStack() as stack:&#xA;        if args.output != '-':&#xA;            out = stack.enter_context(open_file(args.output, 'wt' if args.force else 'xt'))&#xA;        else:&#xA;            out = sys.stdout&#xA;&#xA;        input_fasta = stack.enter_context(open_file(args.fasta))&#xA;        trimmed_reads = trim_fasta(input_fasta, args.sequence, args.num_bases)&#xA;&#xA;        SeqIO.write(trimmed_reads, out, &amp;quot;fasta-2line&amp;quot;)&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;hr /&gt;&#xA;&lt;pre&gt;&lt;code&gt;usage: trim_reads.py [-n INT] [-o FILE] [-f] [-h] FILE STR&#xA;&#xA;input options:&#xA;  FILE                  The input FASTA file&#xA;  STR                   The sequence to find and trim&#xA;&#xA;trimming options:&#xA;  -n INT, --num_bases INT&#xA;                        Trim INT bases upstream of input sequence&#xA;&#xA;output options:&#xA;  -o FILE, --output FILE&#xA;                        Write the filtered output to FILE (default: stdout)&#xA;  -f, --force           Overwrite the output file if it exists&#xA;&#xA;additional options:&#xA;  -h, --help            Show this help message and exit&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2023-01-26T12:44:12.530" LastActivityDate="2023-01-26T12:44:12.530" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20450" PostTypeId="1" CreationDate="2023-01-26T15:52:00.973" Score="2" ViewCount="47" Body="&lt;p&gt;I have started analyzing methylation (EM-seq) data for the first time ~0.8M positions.&#xA;In this data set I have 27 samples of 20 patients. I want to perform a PCA of the dataset to check for possible batch effects or problems.&lt;/p&gt;&#xA;&lt;p&gt;Some publications and guides use the raw M-values to plot it without taking into account the coverage of each position. If I do that, I can detect a previously unknown source of variation, which is already great.&lt;/p&gt;&#xA;&lt;p&gt;Usually there is also the information of the coverage of each position (not the number of reads per sample). Example of the data, which some packages store in the same object:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Cov:&#xA;      A      B      C      D      E&#xA;     19     10     18     14     20&#xA;     13      8     20     12     12&#xA;     12      6      9     16      7&#xA;     10      7      9     16      8&#xA;      8      5      6     12      8&#xA;&#xA;M-values:&#xA;      A      B      C      D      E&#xA;   9.88   0.00   0.00   0.00   0.00&#xA;   7.93   0.00  10.00   3.00   0.00&#xA;   9.00   1.98   5.94  13.92   4.97&#xA;   9.00   2.94   3.96   8.96   4.96&#xA;   2.96   4.00   1.98   4.92   4.96&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see the coverage of each position is not the same for each sample.&#xA;However, I haven't found any specific method for calculating a PCA taking into account the coverage: there doesn't seem to be a method that normalizes the M-values by the coverage of each position.&lt;/p&gt;&#xA;&lt;p&gt;I checked the distribution of coverage and 91% is covered by less than 14 reads, but some positions are covered by more than 300 reads.&lt;/p&gt;&#xA;&lt;p&gt;What is the best way to normalize the position by coverage and plot the PCA ?&lt;/p&gt;&#xA;" OwnerUserId="48" LastEditorUserId="48" LastEditDate="2023-01-28T11:45:28.557" LastActivityDate="2023-01-28T11:45:28.557" Title="PCA of methylation-values normalized by coverage" Tags="|normalization|pca|methylation|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20451" PostTypeId="1" AcceptedAnswerId="20682" CreationDate="2023-01-26T16:16:52.710" Score="2" ViewCount="267" Body="&lt;p&gt;I want to use physico chemical descriptors of single stranded DNA/RNA-Oligonucleotides in order to model certain properties. To generate these descriptors i need a computer understandable representation of these oligonucleotides, e.g. the SMILES representation. So far i could not find of any method to generate these SMILES representations out of the primary structure of these oligonucleotides.&#xA;I am wondering if someone knows of a software, that takes the primary sequence of a single stranded DNA-Oligonucleotide with e.g. 50 nts as input and gives the SMILES representation as output.&#xA;So far i could find nothing that would accomplish this task, although, at least to me, it seems like a straight forward task.&lt;/p&gt;&#xA;&lt;p&gt;I will appreciate every constructive thought on that.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; Add some form of input and output file&lt;/p&gt;&#xA;&lt;p&gt;As requested here an example of input and output pair:&lt;/p&gt;&#xA;&lt;p&gt;input should be primary sequence:&#xA;TCA&lt;/p&gt;&#xA;&lt;p&gt;output should be the corresponding SMILES string:&#xA;O=C(C(C)=CN1[C@@]2([H])C([H])([H])C@@([H])C@([H])O2)NC1=O&lt;/p&gt;&#xA;&lt;p&gt;This example was generated with chemdraw 20.1.1.&lt;/p&gt;&#xA;&lt;p&gt;For my use case i need a software, that convert many (like 50k - 1000k) primary sequences of single stranded DNA/RNA with a length of about 50 nts into the corresponding SMILES string. Ideally would be some sort of python package but anything else would help also a lot!&lt;/p&gt;&#xA;" OwnerUserId="16620" LastEditorUserId="16620" LastEditDate="2023-01-30T15:52:30.160" LastActivityDate="2023-03-07T23:10:17.930" Title="Is it possible to create SMILES representation out of DNA/RNA-Oligonucleotides (~50 nts)?" Tags="|rna|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20452" PostTypeId="2" ParentId="20442" CreationDate="2023-01-26T17:00:37.280" Score="1" Body="&lt;p&gt;using an unwanted (regex library) I can steal code from &lt;a href=&quot;https://stackoverflow.com/questions/13114246/how-to-find-a-open-reading-frame-in-python&quot;&gt;https://stackoverflow.com/questions/13114246/how-to-find-a-open-reading-frame-in-python&lt;/a&gt; , code :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;&#xA;&#xA;from Bio import SeqIO&#xA;# record = SeqIO.read(&amp;quot;sample_test.fasta&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;# record = SeqIO.read(&amp;quot;sample_test_2.fasta&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;&#xA;record = SeqIO.read(&amp;quot;sample.fasta&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;&#xA;# print(record.seq)&#xA;# print(len(sequence))&#xA;&#xA;sequence = str(record.seq)&#xA;&#xA;&#xA;pattern = re.compile(r'(?=(ATG(?:...)*?)(?=TAG|TGA|TAA))')&#xA;&#xA;&#xA;revcompseq = sequence[::-1].maketrans(&amp;quot;ATGC&amp;quot;, &amp;quot;TACG&amp;quot;) #reverse complement&#xA;&#xA;# print (pattern.findall(sequence)) #forward search&#xA;# print (pattern.findall(sequence[::-1].translate(revcompseq))) #backward search&#xA;&#xA;&#xA;a = pattern.findall(sequence)&#xA;&#xA;# print(type(a))&#xA;&#xA;&#xA;cnt = 1&#xA;for i in a:&#xA;    if len(i) &amp;gt; 90:&#xA;        print('\nnumero = ',cnt,'\n',i ,'\nlenght = ', len(i),'\n---------------------------\n\n')&#xA;        cnt += 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;numero =  1 &#xA; ATGCAATACAGTGACATATATATATACCCTAACACTACCCTAACCCTACCCTATTTCAACCCTTCCAACCTGTCTCTCAACTTACCCTCACATTACCCTACCTCTCCACTTGTTACCCTGTCCCATTCAACCATACCACTCCCAACCACCATCCATCCCTCTACTTACTACCACCAATCAACCGTCCACCATAACCGTTACCCTCCAATTAGCCATATTCAACTTCACTACCACTTACCCTGCCATTACTCTACCATCCACCATCTGCTACTCACCATACTGTTGTTCTACCCTCCATAT &#xA;lenght =  300 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  2 &#xA; ATGCACGGCACTTGCCTCAGCGGTCTATACCCTGAGCCATTTACCCATAACTCCCACGATTATCCACATTTTAATATCTATATCTCATTCGGCGGGCCCAAATATTGTATAACTGCTCTTAATACATACGTTATACCACTTTTGCACCATATACTAACCACTCAAATTATATACACTTATGCCAATATAACCAAAAAATTACCACTAAAATCACCTAAACATAAAAATTATTTATCTTTCAACTTTACGAAA &#xA;lenght =  252 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  3 &#xA; ATGCCACCTCACTGTCGTAACACTCTATATTCACCGAGAAACAATACGGTAGTGGCTCAAACTCATGCCGGTGCTATGATACAATTGTATCCTATTTCCATTCTCATATGCTATCCGCAATATCCTAAAAGCATAACTGATGCATCTTTAATCTTGTATGTGACACTACTCATACGAAGAGACTATATC &#xA;lenght =  189 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  4 &#xA; ATGATACAATTGTATCCTATTTCCATTCTCATATGCTATCCGCAATATCCTAAAAGCATAACTGATGCATCTTTAATCTTGTATGTGACACTACTCATACGAAGAGACTATATC &#xA;lenght =  114 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  5 &#xA; ATGAAACGTCAAATAACCCTACTATATTATAACTTATCAGCGCCGTATACTAAAACGGACGTTACGATATTGTCTCACTTCATCTTACCACCCTCTATCTTATTGTTGATAAAACAC &#xA;lenght =  117 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  6 &#xA; ATGACTTGTAACTCCACTACCCTGATCTGCAATCTTGTTCTTAGAAGTGACGCATACTCTATATGGCCCGACGCGACGCGCCAAAAAATGAAAAAAGAAGCAGCGACTCATTTTTATGGAAGGACAAAGTGCTGCGAAGTCATACGCTTCCAATTTCATTATTGTTTATTGGACATACTCTGT &#xA;lenght =  183 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  7 &#xA; ATGGCCCGACGCGACGCGCCAAAAAATGAAAAAAGAAGCAGCGACTCATTTTTATGGAAGGACAAAGTGCTGCGAAGTCATACGCTTCCAATTTCATTATTGTTTATTGGACATACTCTGTTAGCTTTATTACCGTCCACGCTTTTTCTACAA &#xA;lenght =  153 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  8 &#xA; ATGAAAAAAGAAGCAGCGACTCATTTTTATGGAAGGACAAAGTGCTGCGAAGTCATACGCTTCCAATTTCATTATTGTTTATTGGACATACTCTGT &#xA;lenght =  96 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  9 &#xA; ATGGAAGGACAAAGTGCTGCGAAGTCATACGCTTCCAATTTCATTATTGTTTATTGGACATACTCTGTTAGCTTTATTACCGTCCACGCTTTTTCTACAATAGTG &#xA;lenght =  105 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  10 &#xA; ATGGCTGGCTTTAATCTGCTGGAGTACCATGGAACACCGGTGATCATTCTGGTCACTTGGTCTGGGGCAATACCAGTCAACATGGTGGTGAAGTCACCG &#xA;lenght =  99 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  11 &#xA; ATGGCAGCGACACCAGCAGCGATTGAAGTTAATTTGACCATTGTATTTGTTTTGTTTGTTAGTGCTGATGTAATCTTAACAAGAAATAGTGAAATGAAAGCGCATACCTCAAAGGCATATAGT &#xA;lenght =  123 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  12 &#xA; ATGGAGATGCTCTTGTTTCTGAACGAATCATACATCTTTCATAGGTTTCGTATGTGGAGTATTGTTTTATGGCACTCATGTGTATTCGTATGCGCAGAATGTGGGAATGCCTATTATAGGGGTGCCGGGGGGTGCCTTGAAAAACCCTTTTGCGCGCCTGTTAAGTTTCCGTTTTCAGTCAAAAAGAATATCCGAATTTTAGATTTGGACCCTCGTTCAGAAGCTTACTGTCTAAGCCATCATTTGGTGTGTCCTAAACGGTTTCCATGCAAAGCAACATCTTTGTTACTCATTCCTGAAGGT &#xA;lenght =  303 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  13 &#xA; ATGCTCTTGTTTCTGAACGAATCATACATCTTTCATAGGTTTCGTATGTGGAGTATTGTTTTATGGCACTCATGTGTATTCGTATGCGCAGAATGTGGGAATGCCTATTATAGGGGTGCCGGGGGGTGCCTTGAAAAACCCTTTTGCGCGCCTGTTAAGTTTCCGTTTTCAGTCAAAAAGAATATCCGAATTTTAGATTTGGACCCTCGTTCAGAAGCTTACTGTCTAAGCCATCATTTGGTGTGTCCTAAACGGTTTCCATGCAAAGCAACATCTTTGTTACTCATTCCTGAAGGT &#xA;lenght =  297 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  14 &#xA; ATGTGGAGTATTGTTTTATGGCACTCATGTGTATTCGTATGCGCAGAATGTGGGAATGCCTATTATAGGGGTGCCGGGGGGTGCCTTGAAAAACCCTTTTGCGCGCCTGTTAAGTTTCCGTTTTCAGTCAAAAAGAATATCCGAATTTTAGATTTGGACCCTCGTTCAGAAGCTTACTGTCTAAGCCATCATTTGGTGTGTCCTAAACGGTTTCCATGCAAAGCAACATCTTTGTTACTCATTCCTGAAGGT &#xA;lenght =  252 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  15 &#xA; ATGCCTATTATAGGGGTGCCGGGGGGTGCCTTGAAAAACCCTTTTGCGCGCCTGTTAAGTTTCCGTTTTCAGTCAAAAAGAATATCCGAATTT &#xA;lenght =  93 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  16 &#xA; ATGATAATAATATTTATAGAATTGTGTAGAATTGCAGATTCCCTTTTATGGATTCCTAAATCCTGGAGGAGAACTTCTAGTATATCTACATACCTAATATTA &#xA;lenght =  102 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  17 &#xA; ATGGAATCCCAACAATTACATCAAAATCCACATTCTCTACACCAATACCATCGACGAGAGCTTCTAGTAAATTGTATACATAACAGTATAACCCTTACCAACAATGGAATCTCAAAGATTATTAAATTATTCACAGACTCTGAGGATTCGGGTAAAATAGGGTATTTAACTGGTTACCGGAAAGGTTTAGAAAATTCGTGGAGGGTTGGCCGAGTGGTC &#xA;lenght =  219 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  18 &#xA; ATGATGTGTAGCGCATTAGTCTTCTGCATTCATATTATCAGTGGTATACCGTTTGGTATATGCTTTCCAATCATGAAGTGTTGGCTCTACTCTATA &#xA;lenght =  96 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  19 &#xA; ATGTGTAGCGCATTAGTCTTCTGCATTCATATTATCAGTGGTATACCGTTTGGTATATGCTTTCCAATCATGAAGTGTTGGCTCTACTCTATA &#xA;lenght =  93 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  20 &#xA; ATGCTGCGACAGAACTTTTCTTCTCTTCCAATTCTTTTATCTTCGCCAATAACGCCTTCTTTTTTCGTGCTTGTATTTCTAAAATTTCGGCCAGGTATTGTAAATATTCGACCGTTCTATCCAAAA &#xA;lenght =  126 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  21 &#xA; ATGGTGGAACCAACTGGCCGAGTTCTTTTATCTTTTGCTTTATTAATTCTCTTCTTCTCCTTTCGACGGCATTATGAAACTCTCTTTTGCGCCTCAGTTTCTCATCAGAAGTTAACCCGCCTAAAATCTTTGGAACACTTCCAGGTCCTATATTTTCAGTCATATTGCTACTTATTGAAGTGTGTCTTGTTCTGGGTGTGTTTATGCTACCATGCC &#xA;lenght =  216 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  22 &#xA; ATGCTACCATGCCTAAAAGAAGATGAAAGGAAACTTCCTGCACGAAATGACGATGATGGTGATCTTACCTTTGGGGAATATGTGGAAGATACGGATGCTGGGCCCAAGCTTTGTGGATTA &#xA;lenght =  120 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  23 &#xA; ATGAAAGGAAACTTCCTGCACGAAATGACGATGATGGTGATCTTACCTTTGGGGAATATGTGGAAGATACGGATGCTGGGCCCAAGCTTTGTGGATTATAAGAAAATGATGATGAATATGTGTTTGGTGTCATCATATCAGAAT &#xA;lenght =  144 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  24 &#xA; ATGACGATGATGGTGATCTTACCTTTGGGGAATATGTGGAAGATACGGATGCTGGGCCCAAGCTTTGTGGATTATAAGAAAATGATGATGAATATGTGTTTGGTGTCATCATATCAGAAT &#xA;lenght =  120 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  25 &#xA; ATGATGGTGATCTTACCTTTGGGGAATATGTGGAAGATACGGATGCTGGGCCCAAGCTTTGTGGATTATAAGAAAATGATGATGAATATGTGTTTGGTGTCATCATATCAGAAT &#xA;lenght =  114 &#xA;---------------------------&#xA;&#xA;&#xA;&#xA;numero =  26 &#xA; ATGGTGATCTTACCTTTGGGGAATATGTGGAAGATACGGATGCTGGGCCCAAGCTTTGTGGATTATAAGAAAATGATGATGAATATGTGTTTGGTGTCATCATATCAGAAT &#xA;lenght =  111 &#xA;---------------------------&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I can get the start position in my sequence of all the found ORFs with :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;b = [[m.start(1), m.end(1)] for m in pattern.finditer(sequence)]&#xA;&#xA;print(b, len(b))&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;result a list of tuples :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[(2, 302), (316, 400), (373, 400), (404, 479), (421, 505), &#xA;(478, 505), (508, 556), (524, 776), (702, 711), (807, 822), &#xA;(907, 1096), (971, 983), (982, 1096), (1014, 1032), (1046, 1055), &#xA;(1064, 1067), (1117, 1141), (1143, 1260), (1340, 1355), (1354, 1378), &#xA;(1370, 1553), (1432, 1585), (1457, 1553), (1485, 1590), (1606, 1663),&#xA; (1626, 1647), (1754, 1769), (1830, 1929),&#xA;&#xA;.........................................&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but cannot filter for &lt;code&gt;m.end() - m.start() &amp;gt; 90&lt;/code&gt; in the one liner &lt;code&gt;b&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;any help ??&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDITED :&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;b = [[m.start(1), m.end(1)] for m in pattern.finditer(sequence) if (m.end(1) - m.start(1)) &amp;gt; 90]&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;should work only cant get found string with &lt;code&gt;m.group()&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;as an example for first match I get:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;2, 302, &amp;lt;re.Match object; span=(2, 2), match=''&amp;gt;&lt;/code&gt; with &lt;code&gt;m.group()&lt;/code&gt; = &lt;code&gt;''&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDITED 2:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I can get the found substring using &lt;code&gt; m.groups()[0]&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2023-01-27T02:31:23.090" LastActivityDate="2023-01-27T02:31:23.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20453" PostTypeId="1" CreationDate="2023-01-26T17:01:55.640" Score="0" ViewCount="60" Body="&lt;p&gt;I want to generate descriptors from dot bracket notations of single stranded RNA/DNA secondary structures. So far i have come across this paper:&lt;a href=&quot;https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0097696#s5&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0097696#s5&lt;/a&gt;&#xA;In that, there is given a list of descriptors derived from the secondary structure of single stranded RNA but it is not explained how they derived these descriptors. Is there a Software out there, that would accomplish that task?&#xA;I wrote a script myself, to extract loop counts etc. from dot bracket notation but it would rather tedious to write a script all by myself for all descriptors possible.&lt;/p&gt;&#xA;&lt;p&gt;I would appreciate every constructive thought on that.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt; Summary of paper for relevance&lt;/p&gt;&#xA;&lt;p&gt;In this paper the secondary structures of single stranded RNA is parsed into many descriptors, whereby the secondary structure is given as dot bracket notation and where obtained by mfold. THis paper is nothing more than an example of the variety of descriptors that are describing the secondary structure of single stranded RNA/DNA present in dot bracket notation.&#xA;My question now is: Is there a general accepted software, that takes as input the dot bracket notation of an single stranded RND/DNA Oligonucleotide and returns many different descriptors? Such alike rdkit would do it for small organic molecules when given the SMILES string as input.&#xA;I hope this clarified my need.&lt;/p&gt;&#xA;" OwnerUserId="16620" LastEditorUserId="16620" LastEditDate="2023-01-30T15:50:52.640" LastActivityDate="2024-02-24T20:00:35.060" Title="Is there a software that extract molecular descriptors of single stranded RNAs secondary structure from dot bracket notation in nucleotide resolution?" Tags="|rna|secondary-structure|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20454" PostTypeId="1" CreationDate="2023-01-26T18:07:03.163" Score="1" ViewCount="65" Body="&lt;p&gt;I'm tryig to perform RKM normalization on raw counts for RNA-Seq Data:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;counts_data &amp;lt;- exprs181517[,-1]&#xA;#Setting title col to rownames for pheno data &#xA;rownames(pheno.data181517) &amp;lt;- NULL&#xA;rownames(counts_data) &amp;lt;- exprs181517[,1]&#xA;phenoColData &amp;lt;- tibble::column_to_rownames(pheno.data181517, &amp;quot;title&amp;quot;)&#xA;colnames(counts_data) &amp;lt;- rownames(phenoColData)&#xA;#Defining the design column&#xA;designCol &amp;lt;- as.data.frame(phenoColData$characteristics_ch1)&#xA;rownames(designCol) &amp;lt;- rownames(phenoColData)&#xA;&#xA;#Creating a summarized experiment&#xA;se &amp;lt;- SummarizedExperiment(list(counts=counts_data), colData=phenoColData)&#xA;&#xA;DESeqDataSetFromMatrix(se, design ~ designCol)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but I keep getting this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in h(simpleError(msg, call)) :&#xA;  error in evaluating the argument 'x' in selecting a method for function 'as.matrix':&#xA;    no method for coercing this S4 class to a vector&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16832" LastEditorUserId="73" LastEditDate="2023-01-27T19:25:00.090" LastActivityDate="2023-01-27T19:25:00.090" Title="Error while using SummarizedExperiment() in R" Tags="|rna-seq|bioconductor|normalization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20455" PostTypeId="1" CreationDate="2023-01-26T21:12:12.193" Score="2" ViewCount="653" Body="&lt;p&gt;I have a VCF file with 200 samples (mitochondrial genome of Plasmodium falciparum). Here is a pic to take a look at:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/M3wuG.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/M3wuG.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And a few relevant lines from the actual file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##INFO=&amp;lt;ID=AC,Number=A,Type=Integer,Description=&amp;quot;Allele count in genotypes, for each ALT allele, in the same order as listed&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=AF,Number=A,Type=Float,Description=&amp;quot;Allele Frequency, for each ALT allele, in the same order as listed&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=AN,Number=1,Type=Integer,Description=&amp;quot;Total number of alleles in called genotypes&amp;quot;&amp;gt;&#xA;#CHROM  POS     ID      REF     ALT     QUAL    FILTER  INFO    FORMAT  FP0008-C   FP0009-C&#xA;Pf_M76611       19      .       A       T       33196.8 MissingVQSLOD;Mitochondrion     AC=9;AF=0.0003019;AN=29816;ANN=T|intragenic_variant|MODIFIER|PF3D7_MIT00100|PF3D7_MIT00100|gene_variant|PF3D7_MIT00100|||n.19A&amp;gt;T||||||;AS_InbreedingCoeff=0.0437;AS_QD=20.96;BaseQRankSum=0.141;CDS;DP=559138;ExcessHet=0.0026;FS=2.892;InbreedingCoeff=0.52;MLEAC=8;MLEAF=0.0002683;MQ=60;MQRankSum=0;QD=20.96;ReadPosRankSum=0.856;RegionType=Mitochondrion;SOR=0.789;set=genotypegvcfs  GT:AD:DP:GQ:PL  0/0:52,0:52:99:0,120,1800  0/0:54,0:54:99:0,120,1800&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A small subset of samples I am working on can be found here:&#xA;&lt;a href=&quot;https://www.mediafire.com/file/6dkct6guk5zx83m/sample.vcf/file&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.mediafire.com/file/6dkct6guk5zx83m/sample.vcf/file&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am trying to calculate allele frequency (AF) for each variant in the dataset. The population is assumed to be all the samples in the file.&lt;/p&gt;&#xA;&lt;p&gt;In other words, even though the VCF file contains AF statistics from population data, I would like to know how to recalculate AF only from the VCF file. I believe I need to know how AC and AN are calculated first.&lt;/p&gt;&#xA;&lt;p&gt;AF is calculated this way: allele count (AC) / allele number (AN)&lt;/p&gt;&#xA;&lt;p&gt;I tried to calculate them from the information from GT (counting 0/0, 1/1 and 0/1 in all the 200 samples), but then I learned that this calculation is not suitable for mithocondrial data.&lt;/p&gt;&#xA;&lt;p&gt;All lines in the VCF file contain an AF statistic:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -v 'AF=' sample.vcf | grep -v '^#'&#xA;&amp;lt;No output&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The information is provided under INFO column, but how were they calculated? If I do not have AC and AN to begin with, what should I do?&#xA;Is there a way to calculate AC and AN from the other information provided in the dataset? such as the data mentioned in the FORMAT column?&#xA;GT: genotype&#xA;AD: unfiltered allele depth&#xA;DP : read depth at this position for this sample (Integer)&#xA;GQ : conditional genotype quality, encoded as a phred quality&#xA;PL : the phred-scaled genotype likelihoods rounded to the closest integer&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="16833" LastEditorUserId="16833" LastEditDate="2023-01-28T13:33:44.767" LastActivityDate="2023-01-30T15:36:57.197" Title="How to Recalculate Allele Frequency from a VCF File?" Tags="|vcf|allele-frequency|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20456" PostTypeId="2" ParentId="20454" CreationDate="2023-01-26T21:20:00.910" Score="2" Body="&lt;p&gt;DESeqDataSetFrom&lt;strong&gt;Matrix&lt;/strong&gt; but a SummarizedExperiment is not a matrix. Use the function &lt;code&gt;DESeqDataSet()&lt;/code&gt; instead and make sure the &lt;code&gt;design&lt;/code&gt; argument is followed by a &lt;code&gt;=&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-01-26T21:20:00.910" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20457" PostTypeId="2" ParentId="4226" CreationDate="2023-01-26T21:43:04.243" Score="3" Body="&lt;p&gt;I came across this thread while searching for a solution to a similar problem I was experiencing. I am inputting markers that are 20 basepairs in length as the query sequence and haven't been getting hits. As terdon mentioned in their excellent reply, setting word size to something smaller like 11 helps for these shorter query sequences.&lt;/p&gt;&#xA;&lt;p&gt;I didn't get any results on my 20 bp query sequences when using blastn against the sunflower genome, but I got plenty when setting word size to 11. So, why does setting word size for blastn to 11 improve the results despite documentation stating that default word size is 11? Well it turns out that this is very misleading. &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Class/MLACourse/Modules/BLAST/nucleotide_blast.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;This webpage&lt;/a&gt; explains that the default &lt;strong&gt;algorithm&lt;/strong&gt; used in blastn is actually megablast, not blastn. The default word length for this default megablast algorithm is actually 28!&lt;/p&gt;&#xA;&lt;p&gt;The documentation is quite unclear about this and many websites providing tutorials for blastn also seem to miss this key detail. Hope this helps anyone else who struggles with this!&lt;/p&gt;&#xA;" OwnerUserId="16834" LastActivityDate="2023-01-26T21:43:04.243" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20458" PostTypeId="1" AcceptedAnswerId="20461" CreationDate="2023-01-27T01:20:37.097" Score="3" ViewCount="96" Body="&lt;p&gt;I encountered the dichotomy in the context of PDBx/mmcif files, say, &lt;code&gt;6OSQ&lt;/code&gt;: each chain has a&#xA;&lt;code&gt;_entity_poly.pdbx_seq_one_letter_code&lt;/code&gt; but also &lt;code&gt;_entity_poly.pdbx_seq_one_letter_code_can&lt;/code&gt; which i assume stands for &amp;quot;canonical&amp;quot;. The &lt;em&gt;raw&lt;/em&gt; and the &lt;em&gt;canonical&lt;/em&gt; sequences.&lt;/p&gt;&#xA;&lt;p&gt;The difference seems to be that all the ligands, hetatoms and other junk (&lt;code&gt;(PSU)&lt;/code&gt;, &lt;code&gt;(MP4)&lt;/code&gt;, &lt;code&gt;(MG4)&lt;/code&gt;) have been removed from the the raw sequence to make it canonical i.e. it can now be packaged as &lt;code&gt;fasta&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My question is where does one find documentation for this process? Is making it amenable to fasta tools the primary motivation? What kind of hetero atoms &lt;em&gt;can&lt;/em&gt; occur there? Where is the logic written down for replacing them or deleting them when deriving the canonical sequence? Is the job of the structure's depositor or the PDB? Is there a tool for this?&lt;/p&gt;&#xA;&lt;p&gt;Below is chain &lt;code&gt;1&lt;/code&gt; of PDB&lt;code&gt;6OSQ&lt;/code&gt;. First is the raw seq, then is the canonical one.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1 polyribonucleotide no yes&#xA;;GGUUAAGCGACUAAGCGUACACGGUGGAUGCCCUGGCAGUCAGAGGCGAUGAAGGACGUGCUAAUCUGCGAUAAGCGUCG&#xA;GUAAGGUGAUAUGAACCGUUAUAACCGGCGAUUUCCGAAUGGGGAAACCCAGUGUGUUUCGACACACUAUCAUUAACUGA&#xA;AUCCAUAGGUUAAUGAGGCGAACCGGGGGAACUGAAACAUCUAAGUACCCCGAGGAAAAGAAAUCAACCGAGAUUCCCCC&#xA;AGUAGCGGCGAGCGAACGGGGAGCAGCCCAGAGCCUGAAUCAGUGUGUGUGUUAGUGGAAGCGUCUGGAAAGGCGCGCGA&#xA;UACAGGGUGACAGCCCCGUACACAAAAAUGCACAUGCUGUGAGCUCGAUGAGUAGGGCGGGACACGUGGUAUCCUGUCUG&#xA;AAUAUGGGGGGACCAUCCUCCAAGGCUAAAUACUCCUGACUGACCGAUAGUGAACCAGUACCGUGAGGGAAAGGCGAAAA&#xA;GAACCCCGGCGAGGGGAGUGAAAAAGAACCUGAAACCGUGUACGUACAAGCAGUGGGAGCACGCUUAGGCGUGUGACUGC&#xA;GUACCUUUUGUAUAAUGGGUCAGCGACUUAUAUUCUGUAGCAAGGUUAACCGAAUAGGGGAGCCGAAGGGAAACCGAGUC&#xA;UUAACUGGGCGUUAAGUUGCAGGGUAUAGACCCGAAACCCGGUGAUCUAGCCAUGGGCAGGUUGAAGGUUGGGUAACACU&#xA;AACUGGAGGACCGAACCGACUAAU(1MG)(PSU)(5MU)GAAAAAUUAGCGGAUGACUUGUGGCUGGGGGUGAAAGGCCA&#xA;AUCAAACCGGGAGAUAGCUGGUUCUCCCCGAAAGCUAUUUAGGUAGCGCCUCGUGAAUUCAUCUCCGGGGGUAGAGCACU&#xA;GUUUCGGCAAGGGGGUCAACCCGACUUACCAACCCGAUGCAAACUGCGAAUACCGGAGAAUGUUAUCACGGGAGACACAC&#xA;GGCGGG(PSU)GCUAACGUCCGUCGUGAAGAGGGAAACAACCCAGACCGCCAGCUAAGGUCCCAAAGUCAUGGUUAAGUG&#xA;GGAAACGAUGUGGGAAGGCCCAGACAGCCAGGAUGUUGGCUUAGAAGCAGCCAUCAUUUAAAGAAAGCGUAAUAGCUCAC&#xA;UGGUCGAGUCGGCCUGCGCGGAAGAUGUAACGGGGCUAAACCAUGCACCGAAGCUGCGGCAGCGACGCUUAUGCGUUGUU&#xA;GGGUAGGGGAGCGUUCUGUAAGCCUGCGAAGGUGUGCUGUGAGGCAUGCUGGAGGUAUCAGAAGUGCGAAUGCUGACAUA&#xA;AGUAACGAUAAAGCGGGUGAAAAGCCCGCUCGCCGGAAGACCAAGGGUUCCUGUCCAACGUUAAUCGGGGCAGGGUGAGU&#xA;CGACCCCUAAGGCGAGGCCGAAAGGCGUAGUCGAUGGGAAACAGGUUAAUAUUCCUGUACUUGGUGUUACUGCGAAGGGG&#xA;GGACGGAGAAGGCUAUGUUGGCCGGGCGACGGUUGUCCCGGUUUAAGCGUGUAGGCUGGUUUUCCAGGCAAAUCCGGAAA&#xA;AUCAAGGCUGAGGCGUGAUGACGAGGCACUACGGUGCUGAAGCAACAAAUGCCCUGCUUCCAGGAAAAGCCUCUAAGCAU&#xA;CAGGUAACAUCAAAUCGUACCCCAAACCGACAC(6MZ)GGUGGUCAGGUAGAGAAUACCAAGGCGCUUGAGAGAACUCGG&#xA;GUGAAGGAACUAGGCAAAAUGGUGCCGUAACUUCGGGAGAAGGCACGCUGAUAUGUAGGUGAGGUCCCUCGCGGAUGGAG&#xA;CUGAAAUCAGUCGAAGAUACCAGCUGGCUGCAACUGUUUAUUAAAAACACAGCACUGUGCAAACACGAAAGUGGACGUAU&#xA;ACGGUGUGACGCCU(2MG)CCCGGUGCCGGAAGGUUAAUUGAUGGGGUUAGCGCAAGCGAAGCUCUUGAUCGAAGCCCCG&#xA;GUAAACGGCGGCCG(PSU)AAC(3TD)A(PSU)AACGGUCCUAAGGUAGCGAAA(5MU)UCCUUGUCGGGUAAGUUCCGA&#xA;C(5MC)UGCACGAAUGGCGUAAUGAUGGCCAGGCUGUCUCCACCCGAGACUCAGUGAAAUUGAACUCGCUGUG(6MZ)AG&#xA;AUGCAGUGUACCCGCGGCAAGACGGAAAGACCCCGU(G7M)AACCUUUACUAUAGCUUGACACUGAACAUUGAGCCUUGA&#xA;UGUGUAGGAUAGGUGGGAGGCUUUGAAGUGUGGACGCCAGUCUGCAUGGAGCCGACCUUGAAAUACCACCCUUUAAUGUU&#xA;UGAUGUUCUAACGUUGACCCGUAAUCCGGGUUGCGGACAGUGUCUGGUGGGUAGUUUGACUG(OMG)GGCGGUCUCCUCC&#xA;UAAAGAGUAACGGAGGAGCACGAAGGUUGGCUAAUCCUGGUCGGACAUCAGGAGGUUAGUGCAAUGGCAUAAGCCAGCUU&#xA;GACUGCGAGCGUGACGGCGCGAGCAGGUGCGAAAGCAGGUCAUAGUGAUCCGGUGGUUCUGAAUGGAAGGGCCAUCGCUC&#xA;AACGGAUAAAAGGUACUCCG(2MG)GGAUAACAGGC(PSU)GAUACCGCCCAAGAGUUCAUAUCGACGGCGGUGUUUGGC&#xA;A(OMC)CUCG(2MA)(PSU)GUCGGCUCAUCACAUCCUGGGGCUGAAGUAGGUCCCAAGGGUAUGGC(OMU)GUUCGCCA&#xA;UUUAAAGUGGUACGCGAGC(PSU)GGGUUUAGAACGUCGUGAGACAGU(PSU)CGGUCCCUAUCUGCCGUGGGCGCUGGA&#xA;GAACUGAGGGGGGCUGCUCCUAGUACGAGAGGACCGGAGUGGACGCAUCACUGGUGUUCGGGUUGUCAUGCCAAUGGCAC&#xA;UGCCCGGUAGCUAAAUGCGGAAGAGAUAAGUGCUGAAAGCAUCUAAGCACGAAACUUGCCCCGAGAUGAGUUCUCCCUGA&#xA;CCCUUUAAGGGUCCUGAAGGAACGUUGAAGACGACGACGUUGAUAGGCCGGGUGUGUAAGCGCAGCGAUGCGUUGAGCUA&#xA;ACCGGUACUAAUGAACCGUGAGGCUUAACCU&#xA;;&#xA;;GGUUAAGCGACUAAGCGUACACGGUGGAUGCCCUGGCAGUCAGAGGCGAUGAAGGACGUGCUAAUCUGCGAUAAGCGUCG&#xA;GUAAGGUGAUAUGAACCGUUAUAACCGGCGAUUUCCGAAUGGGGAAACCCAGUGUGUUUCGACACACUAUCAUUAACUGA&#xA;AUCCAUAGGUUAAUGAGGCGAACCGGGGGAACUGAAACAUCUAAGUACCCCGAGGAAAAGAAAUCAACCGAGAUUCCCCC&#xA;AGUAGCGGCGAGCGAACGGGGAGCAGCCCAGAGCCUGAAUCAGUGUGUGUGUUAGUGGAAGCGUCUGGAAAGGCGCGCGA&#xA;UACAGGGUGACAGCCCCGUACACAAAAAUGCACAUGCUGUGAGCUCGAUGAGUAGGGCGGGACACGUGGUAUCCUGUCUG&#xA;AAUAUGGGGGGACCAUCCUCCAAGGCUAAAUACUCCUGACUGACCGAUAGUGAACCAGUACCGUGAGGGAAAGGCGAAAA&#xA;GAACCCCGGCGAGGGGAGUGAAAAAGAACCUGAAACCGUGUACGUACAAGCAGUGGGAGCACGCUUAGGCGUGUGACUGC&#xA;GUACCUUUUGUAUAAUGGGUCAGCGACUUAUAUUCUGUAGCAAGGUUAACCGAAUAGGGGAGCCGAAGGGAAACCGAGUC&#xA;UUAACUGGGCGUUAAGUUGCAGGGUAUAGACCCGAAACCCGGUGAUCUAGCCAUGGGCAGGUUGAAGGUUGGGUAACACU&#xA;AACUGGAGGACCGAACCGACUAAUGUUGAAAAAUUAGCGGAUGACUUGUGGCUGGGGGUGAAAGGCCAAUCAAACCGGGA&#xA;GAUAGCUGGUUCUCCCCGAAAGCUAUUUAGGUAGCGCCUCGUGAAUUCAUCUCCGGGGGUAGAGCACUGUUUCGGCAAGG&#xA;GGGUCAACCCGACUUACCAACCCGAUGCAAACUGCGAAUACCGGAGAAUGUUAUCACGGGAGACACACGGCGGGUGCUAA&#xA;CGUCCGUCGUGAAGAGGGAAACAACCCAGACCGCCAGCUAAGGUCCCAAAGUCAUGGUUAAGUGGGAAACGAUGUGGGAA&#xA;GGCCCAGACAGCCAGGAUGUUGGCUUAGAAGCAGCCAUCAUUUAAAGAAAGCGUAAUAGCUCACUGGUCGAGUCGGCCUG&#xA;CGCGGAAGAUGUAACGGGGCUAAACCAUGCACCGAAGCUGCGGCAGCGACGCUUAUGCGUUGUUGGGUAGGGGAGCGUUC&#xA;UGUAAGCCUGCGAAGGUGUGCUGUGAGGCAUGCUGGAGGUAUCAGAAGUGCGAAUGCUGACAUAAGUAACGAUAAAGCGG&#xA;GUGAAAAGCCCGCUCGCCGGAAGACCAAGGGUUCCUGUCCAACGUUAAUCGGGGCAGGGUGAGUCGACCCCUAAGGCGAG&#xA;GCCGAAAGGCGUAGUCGAUGGGAAACAGGUUAAUAUUCCUGUACUUGGUGUUACUGCGAAGGGGGGACGGAGAAGGCUAU&#xA;GUUGGCCGGGCGACGGUUGUCCCGGUUUAAGCGUGUAGGCUGGUUUUCCAGGCAAAUCCGGAAAAUCAAGGCUGAGGCGU&#xA;GAUGACGAGGCACUACGGUGCUGAAGCAACAAAUGCCCUGCUUCCAGGAAAAGCCUCUAAGCAUCAGGUAACAUCAAAUC&#xA;GUACCCCAAACCGACACAGGUGGUCAGGUAGAGAAUACCAAGGCGCUUGAGAGAACUCGGGUGAAGGAACUAGGCAAAAU&#xA;GGUGCCGUAACUUCGGGAGAAGGCACGCUGAUAUGUAGGUGAGGUCCCUCGCGGAUGGAGCUGAAAUCAGUCGAAGAUAC&#xA;CAGCUGGCUGCAACUGUUUAUUAAAAACACAGCACUGUGCAAACACGAAAGUGGACGUAUACGGUGUGACGCCUGCCCGG&#xA;UGCCGGAAGGUUAAUUGAUGGGGUUAGCGCAAGCGAAGCUCUUGAUCGAAGCCCCGGUAAACGGCGGCCGUAACXAUAAC&#xA;GGUCCUAAGGUAGCGAAAUUCCUUGUCGGGUAAGUUCCGACCUGCACGAAUGGCGUAAUGAUGGCCAGGCUGUCUCCACC&#xA;CGAGACUCAGUGAAAUUGAACUCGCUGUGAAGAUGCAGUGUACCCGCGGCAAGACGGAAAGACCCCGUGAACCUUUACUA&#xA;UAGCUUGACACUGAACAUUGAGCCUUGAUGUGUAGGAUAGGUGGGAGGCUUUGAAGUGUGGACGCCAGUCUGCAUGGAGC&#xA;CGACCUUGAAAUACCACCCUUUAAUGUUUGAUGUUCUAACGUUGACCCGUAAUCCGGGUUGCGGACAGUGUCUGGUGGGU&#xA;AGUUUGACUGGGGCGGUCUCCUCCUAAAGAGUAACGGAGGAGCACGAAGGUUGGCUAAUCCUGGUCGGACAUCAGGAGGU&#xA;UAGUGCAAUGGCAUAAGCCAGCUUGACUGCGAGCGUGACGGCGCGAGCAGGUGCGAAAGCAGGUCAUAGUGAUCCGGUGG&#xA;UUCUGAAUGGAAGGGCCAUCGCUCAACGGAUAAAAGGUACUCCGGGGAUAACAGGCUGAUACCGCCCAAGAGUUCAUAUC&#xA;GACGGCGGUGUUUGGCACCUCGAUGUCGGCUCAUCACAUCCUGGGGCUGAAGUAGGUCCCAAGGGUAUGGCUGUUCGCCA&#xA;UUUAAAGUGGUACGCGAGCUGGGUUUAGAACGUCGUGAGACAGUUCGGUCCCUAUCUGCCGUGGGCGCUGGAGAACUGAG&#xA;GGGGGCUGCUCCUAGUACGAGAGGACCGGAGUGGACGCAUCACUGGUGUUCGGGUUGUCAUGCCAAUGGCACUGCCCGGU&#xA;AGCUAAAUGCGGAAGAGAUAAGUGCUGAAAGCAUCUAAGCACGAAACUUGCCCCGAGAUGAGUUCUCCCUGACCCUUUAA&#xA;GGGUCCUGAAGGAACGUUGAAGACGACGACGUUGAUAGGCCGGGUGUGUAAGCGCAGCGAUGCGUUGAGCUAACCGGUAC&#xA;UAAUGAACCGUGAGGCUUAACCU&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6511" LastEditorUserId="6511" LastEditDate="2023-01-27T20:47:20.303" LastActivityDate="2023-01-28T08:27:59.910" Title="How is the &quot;canonical&quot; version (`_entity_poly.pdbx_seq_one_letter_code`) obtained in the PDB?" Tags="|sequence-alignment|pdb|file-formats|structural-biology|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20459" PostTypeId="1" CreationDate="2023-01-27T07:37:36.807" Score="1" ViewCount="32" Body="&lt;p&gt;Does any one have a condensed NCBI reference seq database with all the gbff files, which covers mostly all the bacteria and archaea. The current NCBI ReqSeq contains more than 30,000 gbff files. I am looking for a much more curated refseq with less than 30K files in it.&lt;/p&gt;&#xA;" OwnerUserId="993" LastActivityDate="2023-01-27T07:37:36.807" Title="Is there a smaller representative version of refseq with fewer than 30K files in it?" Tags="|bacteria|refseq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20460" PostTypeId="2" ParentId="20459" CreationDate="2023-01-27T07:37:36.807" Score="1" Body="&lt;p&gt;proGenomes includes a collection of genomes that are representative of different types - &lt;a href=&quot;https://progenomes.embl.de/download.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://progenomes.embl.de/download.cgi&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Or Genome Updater could be used to filter on unique taxonomy IDs and genome completeness etc.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/pirovc/genome_updater&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/pirovc/genome_updater&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="993" LastActivityDate="2023-01-27T07:37:36.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20461" PostTypeId="2" ParentId="20458" CreationDate="2023-01-27T10:46:13.867" Score="2" Body="&lt;p&gt;You can check specification of &lt;a href=&quot;https://mmcif.wwpdb.org/dictionaries/mmcif_pdbx_v50.dic/Items/_entity_poly.pdbx_seq_one_letter_code.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;pdbx_seq_one_letter_code&lt;/a&gt; and &lt;a href=&quot;https://mmcif.wwpdb.org/dictionaries/mmcif_pdbx_v50.dic/Items/_entity_poly.pdbx_seq_one_letter_code_can.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;pdbx_seq_one_letter_code_can&lt;/a&gt;. These items are generated from the depositor-provided sequence on the PDB side. I suppose that the software used is &lt;a href=&quot;https://sw-tools.rcsb.org/apps/MAXIT/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;MAXIT&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Note: these are not &lt;a href=&quot;https://en.wikipedia.org/wiki/Protein_Data_Bank_(file_format)&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB files&lt;/a&gt;. They are distributed by the PDB organization, but the file format is mmCIF (or, more formally, &lt;a href=&quot;https://mmcif.wwpdb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDBx/mmCIF&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="266" LastEditorUserId="73" LastEditDate="2023-01-27T19:06:50.680" LastActivityDate="2023-01-27T19:06:50.680" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20462" PostTypeId="1" AcceptedAnswerId="20468" CreationDate="2023-01-27T16:34:38.740" Score="2" ViewCount="85" Body="&lt;p&gt;When performing GSEA and many other gene set analysis, when gene sets with hierarchy such as KEGG collection or Reactome collection, the FDR could be skewed sometimes, and the ranking of gene sets by any metrics could be confusing because multiple subsets belonging to the same set would have similar ranking, pushing other gene sets down the list. For example, in reactome axon guidance (R-HSA-422475.6) contains semaphorin interaction (R-HSA-373755.1), which contains Sema4d in semaphorin signaling (R-HSA-400685.2). The enrichment of Sema4d gene set usually (though not necessarily) entails the enrichment of the other gene sets above it. That might lead to an over-adjustment of p-value if I am not mistaken.&lt;/p&gt;&#xA;&lt;p&gt;I am just wondering if there are ways to mitigate it, or it is not a major concern that I should have?&lt;/p&gt;&#xA;" OwnerUserId="15648" LastActivityDate="2023-01-30T15:30:42.527" Title="Is the potential skewing of FDR and ranking of gene set analysis with gene sets with hierarchical structure a concern?" Tags="|pathway|gsea|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20463" PostTypeId="2" ParentId="20462" CreationDate="2023-01-27T17:32:57.377" Score="2" Body="&lt;p&gt;Statisticians might kill me for it but I've always filtered long and redundant geneset lists to only contain what remotely makes biological sense in my setup. If I am investigating leukocyte transcriptomes I do not care about terms related to filtration processes in the kidney or signal transduction in the hypothalamus. I think the redundancy of common pathway databases in terms of genes contributing to many terms justifiers to eliminate certain terms and choose those that make sense and are not overly large (for example not more than 500 genes). Top-of-hierarchy terms like 'metabolism' eith more than 1000 genes are anyway unspecific and hard to interpret. That then reduces the multiple testing burden quite a lot.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-01-27T17:32:57.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20464" PostTypeId="1" AcceptedAnswerId="20465" CreationDate="2023-01-27T19:17:06.780" Score="1" ViewCount="141" Body="&lt;p&gt;I am using the following one-liner with &lt;code&gt;NCBI entrez&lt;/code&gt; to query their databases from the terminal (see &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez Direct: E-utilities on the Unix Command Line&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db assembly -query ${species name} |&#xA;    xtract -pattern ENTREZ_DIRECT -element Count&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This works a charm but occasionally it sends an xml error that messes up the output.&lt;/p&gt;&#xA;&lt;p&gt;Here is an example of the error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;xml&#xA;FAILURE ( Fri Jan 27 01:12:44 PM CST 2023 )&#xA;nquire -url https://eutils.ncbi.nlm.nih.gov/entrez/eutils/ esearch.fcgi -retmax 0 -usehistory y -db assembly -term &amp;quot;Crepis commutata&amp;quot; -tool edirect -edirect 18.7 -edirect_os Linux -email {}&#xA;&amp;lt;ErrorList&amp;gt;&#xA;  &amp;lt;PhraseNotFound&amp;gt;commutata&amp;lt;/PhraseNotFound&amp;gt;&#xA;  &amp;lt;PhraseNotFound&amp;gt;Crepis commutata[All Fields]&amp;lt;/PhraseNotFound&amp;gt;&#xA;&amp;lt;ErrorList&amp;gt;&#xA;SECOND ATTEMPT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Entrez, unfortunately, does not have a man page or &lt;code&gt;--help&lt;/code&gt; entry on my installation for some reason and I am not quite sure where I should change the one-liner to change this behavior.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have any clues?&lt;/p&gt;&#xA;" OwnerUserId="16699" LastEditorUserId="298" LastEditDate="2023-01-29T12:03:31.660" LastActivityDate="2023-01-29T12:03:31.660" Title="How to set Entrez Direct - ESearch NCBI entrez to pipe or ignore errors?" Tags="|database|entrez|efetch|ncbi|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20465" PostTypeId="2" ParentId="20464" CreationDate="2023-01-27T19:34:02.930" Score="1" Body="&lt;p&gt;This is because there are not assemblies for &lt;em&gt;Crepis commutate&lt;/em&gt; within NCBI's assembly database.&lt;/p&gt;&#xA;&lt;p&gt;The cross check is here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/assembly/?term=Crepis%20commutata&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/assembly/?term=Crepis+commutata&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The output is a bit clearer than Entrez&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The following terms were not found in Assembly: commutata, Crepis commutata[All Fields].&#xA;No items found.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;All you need do to correct it is issue a &lt;code&gt;next&lt;/code&gt; statement in the a loop of species above the code you've presented.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;if [[ &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$species&quot; == 'Crepis commutata' ]]; then&#xA;  continue&#xA;fi&#xA;esearch -db assembly -query &quot;$&lt;/span&gt;species&amp;quot; | xtract -pattern ENTREZ_DIRECT -element Count&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To answer the comments if this is done via a Python subprocess, you could simply issue the following code (which has not been checked)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import subprocess&#xA;&#xA;for species in List:&#xA;    try:&#xA;        subprocess.run([&amp;quot;esearch&amp;quot;, &amp;quot;-db&amp;quot; &amp;quot;assembly&amp;quot;, &amp;quot;-query&amp;quot;, f&amp;quot;${species}&amp;quot;, &amp;quot;|&amp;quot;, &amp;quot;xtract&amp;quot;, &amp;quot;-pattern&amp;quot;, &amp;quot;ENTREZ_DIRECT&amp;quot;, &amp;quot;-&amp;quot;, &amp;quot;element&amp;quot;], shell=True&amp;quot;)&#xA;    except ValueError as ve:&#xA;        print(f'Species {species}, is not present.')&#xA;        continue&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-27T20:34:21.710" LastActivityDate="2023-01-27T20:34:21.710" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20466" PostTypeId="2" ParentId="20458" CreationDate="2023-01-28T08:27:59.910" Score="2" Body="&lt;p&gt;So, i was little myopic when reading the PDB description the first few times, but roughly what i was looking for is this:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the non-canonical sequence(&lt;code&gt;pdbx_seq_one_letter_code&lt;/code&gt;) makes explicit the ligands present in the given chain.&lt;/li&gt;&#xA;&lt;li&gt;the full dictionary of these is called the &lt;a href=&quot;https://www.wwpdb.org/data/ccd#ligandExpo&quot; rel=&quot;nofollow noreferrer&quot;&gt;CCD&lt;/a&gt; and contains ~20000 entries, is curated and updated weekly by the PDB. That's basically their database (glorified text file over ftp) against which they check whether a given ligand has already been recorded.&lt;/li&gt;&#xA;&lt;li&gt;the process of chaffing them out of the EM dough as well as schema and more context canbe found here: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4243272/pdf/bau116.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4243272/pdf/bau116.pdf&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="6511" LastActivityDate="2023-01-28T08:27:59.910" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20467" PostTypeId="2" ParentId="19159" CreationDate="2023-01-30T03:08:19.223" Score="1" Body="&lt;p&gt;ClairS is now released for long-read somatic variant calling. &lt;a href=&quot;https://github.com/HKU-BAL/Clairs&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/HKU-BAL/Clairs&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16845" LastActivityDate="2023-01-30T03:08:19.223" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20468" PostTypeId="2" ParentId="20462" CreationDate="2023-01-30T15:30:42.527" Score="2" Body="&lt;p&gt;This is actually a known problem in the correction of nominal p-values in over-representation analyses that use the Gene Ontology categories (commonly but misleadingly called &amp;quot;GO enrichment&amp;quot; in many publications).&lt;/p&gt;&#xA;&lt;p&gt;Most currently used methods for p-value adjustment (like BH) assume that p-values are independent, which is not true for a case like the one you describe - a set contained by another set is basically by definition not independent. Even if you didn't have a hierarchical structure, you would still have many overlaps across all your genesets, which would still mean that you cannot assume independence. So in both cases, but I think especially in your case, a BH/FWER correction would be statistically wrong (and possibly overly conservative).&lt;/p&gt;&#xA;&lt;p&gt;There are a few ways to avoid this: one way would be to prune the directed acyclic graph of your genesets/categories, retaining only the categories that are enriched and have no enriched children.&#xA;In your case, for instance, if &amp;quot;axon guidance&amp;quot; and  &amp;quot;semaphorin interaction&amp;quot; are significant but &amp;quot;Sema4d in semaphorin signaling&amp;quot; is not, you would stop at &amp;quot;semaphorin signaling&amp;quot; removing the result for &amp;quot;axon guidance&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;This is kind of a strong correction, which is guaranteed to lower the multiple testing correction burden (unless you have no significant genesets).&lt;/p&gt;&#xA;&lt;p&gt;There are however more refined ways to address this:&lt;/p&gt;&#xA;&lt;p&gt;One of them is to remove from set A (parent) any genes that belong to a significantly over-represented/enriched set A' (child). This method is termed the &lt;strong&gt;elim method&lt;/strong&gt; and was described in this &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/22/13/1600/193669&quot; rel=&quot;nofollow noreferrer&quot;&gt;publication&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The same publication discusses another method called &lt;strong&gt;weight&lt;/strong&gt; which assigns a weight to a significant category based on the significance of its children.&lt;/p&gt;&#xA;&lt;p&gt;Another method is implemented in the gProfiler tool and introduced in this 2007 &lt;a href=&quot;https://academic.oup.com/nar/article/35/suppl_2/W193/2920757&quot; rel=&quot;nofollow noreferrer&quot;&gt;publication&lt;/a&gt;, which is the &lt;strong&gt;g:SCS method&lt;/strong&gt;. Namely it calculates p-value thresholds based on empirical, randomized p-values for sets of different sizes. According to the authors it is more conservative than BH but less than Bonferroni.&lt;/p&gt;&#xA;&lt;p&gt;The only issue with these approaches is that there are, to my knowledge, no ways to apply them to your data independently, i.e. they are coded within the respective tools and do not exist as stand-alone packages for multiple test correction. I hope I'm wrong and someone developed these versions, but for the time being the only thing you can do is to try to implement your own versions of these methods.&lt;/p&gt;&#xA;&lt;p&gt;Another relevant, more recent publication is the &lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-017-1571-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;SetRank&lt;/a&gt; one, which describes a method to do GSEA with a p-value calculation that accounts for overlaps, and has its own package on CRAN. If you prefer to apply a correction to your own results (like the ones you get from &lt;code&gt;fgsea&lt;/code&gt;), however, I do not know whether this package has methods you can use or recycle.&lt;/p&gt;&#xA;" OwnerUserId="16618" LastActivityDate="2023-01-30T15:30:42.527" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20469" PostTypeId="2" ParentId="20455" CreationDate="2023-01-30T15:36:57.197" Score="2" Body="&lt;p&gt;It sounds like your main confusion is how to calculate AF. I'll answer that right away, but I'll also suggest a Python library that you can use to do this and much more.&lt;/p&gt;&#xA;&lt;p&gt;You're correct, the AF is defined as the number of non-missing alternate alleles (AC) divided the number of non-missing alleles (AN). Calculating these are straightforward. Map each GT to its alternate alleles:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;0/0 =&amp;gt; 0&#xA;0/1 =&amp;gt; 1&#xA;1/1 =&amp;gt; 2&#xA;./0 =&amp;gt; 0&#xA;./1 =&amp;gt; 1&#xA;./. =&amp;gt; 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, compute the sum. That is the AC. For the AN, map each GT to its number of non-missing alleles:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;0/0 =&amp;gt; 2&#xA;0/1 =&amp;gt; 2&#xA;1/1 =&amp;gt; 2&#xA;./0 =&amp;gt; 1&#xA;./1 =&amp;gt; 1&#xA;./. =&amp;gt; 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, compute the sum. That is the AN. Now calculate AF as&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;AC / AN&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;All of this is kind of annoying to do though! I suggest you use &lt;a href=&quot;https://hail.is&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hail&lt;/a&gt; (disclaimer: I am a Hail developer).&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import hail as hl&#xA;&#xA;# Hail is not yet aware of Plasmodium falciparum reference genome&#xA;mt = hl.import_vcf('sample.vcf', reference_genome=None)&#xA;&#xA;# let's peek at our data&#xA;mt.show(n_rows=10, n_cols=5)&#xA;&#xA;# Let's get some information about the GT specifically:&#xA;mt.GT.summarize()&#xA;&#xA;# We can use Hail's built in variant_qc method to calculate AF for us:&#xA;mt = hl.variant_qc(mt)&#xA;&#xA;# Let's take a look at that&#xA;mt.variant_qc.show()&#xA;&#xA;# We can overwrite the old AF field and export back to a VCF file if necessary&#xA;mt = mt.annotate_rows(&#xA;    # AF[1] is the frequency of the first alternate allele&#xA;    # Your dataset appears to have some multi-allelics. The other&#xA;    # alternate allele frequencies are available in AF[2], ...&#xA;    info = mt.info.annotate(&#xA;        AF = mt.variant_qc.AF[1]&#xA;    )&#xA;)&#xA;hl.export_vcf(mt, 'sample-updated.vcf.bgz')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;This comment doesn't make sense to me:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I learned that this calculation is not suitable for mithocondrial data.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The definition of AF, AC, and AN doesn't change depending on the type of config (autosome, sex chromosome, mitochondrial). Perhaps the issue is ploidy encoding? A GT in a VCF can be:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1-ploidy, e.g. &lt;code&gt;0&lt;/code&gt; (reference), &lt;code&gt;1&lt;/code&gt; (first alternate), &lt;code&gt;2&lt;/code&gt; (second alternate)&lt;/li&gt;&#xA;&lt;li&gt;2-ploidy, e.g. &lt;code&gt;0/0&lt;/code&gt; (homozygous reference), &lt;code&gt;0/1&lt;/code&gt; (heterozygous), &lt;code&gt;1/1&lt;/code&gt; (homozygous first alternate)&lt;/li&gt;&#xA;&lt;li&gt;etc.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Sometimes people incorrectly encode a 1-ploidy config (for example the Y chromosome) as: &lt;code&gt;0/0&lt;/code&gt; and &lt;code&gt;1/1&lt;/code&gt; instead of &lt;code&gt;0&lt;/code&gt; and &lt;code&gt;1&lt;/code&gt;. That's not your fault though and it doesn't mean the definition of AF, AC, and AN change. It means someone gave you malformed data.&lt;/p&gt;&#xA;" OwnerUserId="1087" LastActivityDate="2023-01-30T15:36:57.197" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20470" PostTypeId="2" ParentId="20453" CreationDate="2023-01-30T16:04:41.667" Score="0" Body="&lt;p&gt;The authors use&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Kapetanovic IM (2008) Computer-aided drug discovery and development (CADDD): in silico-chemico-biological approach. Chem Biol Interact 171: 165–176.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This generates the descriptors, which are normally used in small molecule interactions and they obtained a moderate (in reality low for ML) correlation. That is the innovative part of the study. &amp;quot;Decision trees&amp;quot; used alone are not recommended because it will over tighten onto the data. The reason they used them was to maximise the fit - the risk is over tightening. When its says &amp;quot;the algorithm successfully ...&amp;quot;, well thats using a Decision Tree, any other ML algorithm will be lower.&lt;/p&gt;&#xA;&lt;p&gt;The very nice part of the paper is the combination of wet-lab and analytical work, that balance is often not logistically trivial to achieve.&lt;/p&gt;&#xA;&lt;p&gt;What is not clear is they talk about 6 descriptors out of 60 and also 27 descriptors out of 60. Which is the one they use? I would suggest the paper was not rigorously reviewed/edited to permit this sort of confusion for the reader. It could be (suspect) they performed two different analyses, course grain (6) and fine grain (27), but I don't think that is described very well.&lt;/p&gt;&#xA;&lt;p&gt;I will not critique further.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; The paper is commendable for using a small molecular physiochemical/ thermodynamic description algorithm onto RNA data. Thats cool. Wrapping it around a Decision Tree alone, I wouldn't agree with - nothing wrong with DT's but they can't be used in isolation.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;QSAR Model&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In order to find the mathematical relationship between aptamers’ structure and activity in the training set, two functions of MASS packages of R program were used (R 2.14.1 2011 The R Foundation). Firstly, a stepwise-selection with the function “stepAIC” (direction = both) focused on the most relevant 27 descriptors out of the 60 that can contribute to the prediction of the relative activity. Then, a second step with multiple-linear-regression function “lm” was applied on the selected descriptors to generate appropriate coefficients for the linear correlation equation. The quality of the output was evaluated according to the p-value of the null hypothesis, and multiple R2 indicated the correlation between actual and predicted values.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;QSAR&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Sixty descriptors quantifying structural and thermodynamic properties of the aptamers were produced and gradually the most relevant descriptors were selected to form a six descriptor- based algorithm, which can predict the binding of aptamers to influenza virus and thus anticipate its anti-viral activity. The algorithm successfully predicted the binding affinity in the training set of aptamers (R2 = 0.702) and the was validated with a test set (R2 = 0.66); moreover, the model’s sensitivity and specificity in selecting aptamers with enhanced relative binding were high, 89% and 87%, respectively.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Personally, I don't think that is the question the OP is asking - I think they are asking about a different technique in RNA secondary structure analysis to the technique authors performed here. That however is simply a guess. It's an interesting paper, there is no question about that - it's not how I personally would do it - but  it's interesting. The issue is the thermodynamics has been rigorously worked out historically for RNA folding - it's a large body of work - and I'm sure the one or two reviewers would have pointed this out. It's complicated and if the authors are using their predictions to recycle back to the wet-lab .... thats what it's all about.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Normally, folding is calculated using graph theory ... FYI.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;From there comments .... whats the program ... its Kapetanovic (2008) as cited above, which will be a classic small molecular binding algorithm the authors called &lt;code&gt;QSAR&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In cases there's any misunderstanding - we wouldn't use small molecule interaction based calculations (MD) to understand RNA folding/binding because there's an alternative body of theory for this specific calculation, partly thermodynamics, partly a separate factor. ML is heavily used but not via an MD style calculation - which is what the authors are doing. Thus if you use the method there is a risk of criticism.&lt;/p&gt;&#xA;&lt;p&gt;It's complicated - I understand the authors perspective, they are saying classical style hybridisation its just not that simple. It might be complicated but hybridisation shouldn't be ignored as the empirical assumption.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-30T16:54:23.173" LastActivityDate="2023-01-30T16:54:23.173" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20471" PostTypeId="1" CreationDate="2023-01-31T00:27:12.020" Score="1" ViewCount="507" Body="&lt;p&gt;I'm trying to figure out the right way to do differential gene expression analysis with a discrete variable with three groups. The context is, I want to assess differential expression as a function of whether samples are WT, have heterozygous loss, or homozygous loss for a particular gene. A key part of it is comparing the differences between the WT and het-loss samples with the differences between het-loss and hom-loss samples.&lt;/p&gt;&#xA;&lt;p&gt;I tried doing just doing differential expression with DESeq2, treating it as a categorical variable, but I got a strange result: differential expression between het and hom samples seemed to usually go in the opposite direction as in wt vs. het (so if a gene was upregulated in het relative to WT, it was downregulated in hom samples relative to het). This was very counterintuitive; why would losing the second copy of a gene consistently have the opposite effect on overall gene expression that losing the first copy has? So to test if something was going awry, I did the same analysis, but randomly assigning samples into three groups, and found the same thing: the log fold change comparing group 3 to 2 was strongly negatively correlated with the log FC comparing group 2 to 1, even though the groups are entirely random. So this must be some artifact of the model being used (note that I observe the same thing when used limma voom instead of DESeq 2). I include an example here from data loaded from TCGA via TCGAbiolinks:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(TCGAbiolinks)&#xA;library(DESeq2)&#xA;&#xA;#Loading KIRC data from TCGA:&#xA;query_TCGA = GDCquery(&#xA;  project = &amp;quot;TCGA-KICH&amp;quot;,&#xA;  data.type = &amp;quot;Gene Expression Quantification&amp;quot;,&#xA;  data.category = &amp;quot;Transcriptome Profiling&amp;quot;, # parameter enforced by GDCquery&#xA;  experimental.strategy = &amp;quot;RNA-Seq&amp;quot;,&#xA;  workflow.type = &amp;quot;STAR - Counts&amp;quot;)&#xA;download &amp;lt;- GDCdownload(query = query_TCGA)&#xA;dat &amp;lt;- GDCprepare(query = query_TCGA, save = FALSE)&#xA;rna &amp;lt;- as.data.frame(SummarizedExperiment::assay(dat))&#xA;&#xA;#Randomly assigning samples to groups 1, 2, and 3 and running DESeq2:&#xA;set.seed(1234)&#xA;xvar &amp;lt;- factor(sample(1:3, ncol(rna), replace = TRUE), levels = c(1,2,3))&#xA;columnData &amp;lt;- data.frame('Patient'=colnames(rna),'xvar'=xvar)&#xA;columnData&lt;span class=&quot;math-container&quot;&gt;$xvar &amp;lt;- factor(columnData$&lt;/span&gt;xvar, levels = c('1','2','3'))&#xA;deseq2Data &amp;lt;- DESeqDataSetFromMatrix(countData=rna, colData=columnData, design= ~     xvar)&#xA;deseq2Data &amp;lt;- deseq2Data[rowSums(counts(deseq2Data)) &amp;gt; 5, ]&#xA;deseq2Data &amp;lt;- DESeq(deseq2Data)&#xA;res.1_v_3 &amp;lt;- as.data.frame(results(deseq2Data, contrast = c('xvar','1','3')))&#xA;res.1_v_2 &amp;lt;- as.data.frame(results(deseq2Data, contrast = c('xvar','1','2')))&#xA;res.2_v_3 &amp;lt;- as.data.frame(results(deseq2Data, contrast = c('xvar','2','3')))&#xA;&#xA;#Merging comparisons of 1 v 2, 1 v 3, and 2 v 3 into one data frame:&#xA;merged &amp;lt;- merge(data.frame('Gene'=rownames(res.1_v_2),     'log2FC_1_v_2'=res.1_v_2&lt;span class=&quot;math-container&quot;&gt;$log2FoldChange, 'padj_1_v_2'=res.1_v_2$&lt;/span&gt;padj), &#xA;                data.frame('Gene'=rownames(res.2_v_3),     'log2FC_2_v_3'=res.2_v_3&lt;span class=&quot;math-container&quot;&gt;$log2FoldChange, 'padj_2_v_3'=res.2_v_3$&lt;/span&gt;padj), by='Gene')&#xA;merged &amp;lt;- merge(merged, data.frame('Gene'=rownames(res.1_v_3),     'log2FC_1_v_3'=res.1_v_3&lt;span class=&quot;math-container&quot;&gt;$log2FoldChange, &#xA;                                   'padj_1_v_3'=res.1_v_3$&lt;/span&gt;padj), by='Gene')&#xA;#Color coding for illustration:&#xA;merged&lt;span class=&quot;math-container&quot;&gt;$color &amp;lt;- 'gray'&#xA;merged$&lt;/span&gt;sig_1_v_2 &amp;lt;- merged&lt;span class=&quot;math-container&quot;&gt;$padj_1_v_2 &amp;lt; .05&#xA;merged$&lt;/span&gt;sig_1_v_3 &amp;lt;- merged&lt;span class=&quot;math-container&quot;&gt;$padj_1_v_3 &amp;lt; .05&#xA;merged$&lt;/span&gt;sig_2_v_3 &amp;lt;- merged&lt;span class=&quot;math-container&quot;&gt;$padj_2_v_3 &amp;lt; .05&#xA;merged$&lt;/span&gt;color[which(merged&lt;span class=&quot;math-container&quot;&gt;$padj_1_v_2 &amp;lt; .05 &amp;amp; merged$&lt;/span&gt;padj_2_v_3 &amp;gt; .05)] &amp;lt;- 'blue'&#xA;merged&lt;span class=&quot;math-container&quot;&gt;$color[which(merged$&lt;/span&gt;padj_1_v_2 &amp;gt; .05 &amp;amp; merged&lt;span class=&quot;math-container&quot;&gt;$padj_2_v_3 &amp;lt; .05)] &amp;lt;- 'red'&#xA;merged$&lt;/span&gt;color[which(merged&lt;span class=&quot;math-container&quot;&gt;$padj_1_v_2 &amp;lt; .05 &amp;amp; merged$&lt;/span&gt;padj_2_v_3 &amp;lt; .05)] &amp;lt;- 'purple'&#xA;maxim &amp;lt;- max(abs(c(merged&lt;span class=&quot;math-container&quot;&gt;$log2FC_1_v_2, merged$&lt;/span&gt;log2FC_2_v_3)))&#xA;coef &amp;lt;- cor(merged&lt;span class=&quot;math-container&quot;&gt;$log2FC_1_v_2, merged$&lt;/span&gt;log2FC_2_v_3)&#xA;p &amp;lt;- try(round(chisq.test(table(merged[,c('sig_1_v_2','sig_2_v_3')]))$p.value,     digits = 4), silent = TRUE)&#xA;if(class(p) == 'try-error'){p &amp;lt;- NA}&#xA;&#xA;plot(merged&lt;span class=&quot;math-container&quot;&gt;$log2FC_1_v_2, merged$&lt;/span&gt;log2FC_2_v_3, col=merged$color, pch=16,     cex=.5,main='Log2 FC',&#xA;     xlim=c(-maxim1,maxim), ylim=c(-maxim,maxim),xlab=c('LogFC:     Het/WT'),ylab=c('LogFC: Biallelic/Het'))&#xA;abline(a=0,b=coef1,col='red')&#xA;legend('topright',legend = c('Coefficient','Significant Group 2 v. Group 1',&#xA;                             'Significant Group 3 vs. group 2','Significant for     both comparisons'), &#xA;       lty=c(1, NA, NA,NA), pch=c(NA, 16, 16,16),     col=c('red','blue','red','purple'))&#xA;text(x=-maxim*.8, y=-maxim, labels=paste('p=',p,' (chi-sq. test',sep=''))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hDQxp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hDQxp.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is there something particular I'm doing wrong? And what would be the correct way to do differential expression analysis with an independent variable that is  essentially an ordinal variable? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="16872" LastEditorUserId="16872" LastEditDate="2023-01-31T07:44:39.417" LastActivityDate="2023-01-31T13:18:19.620" Title="Differential gene expression: how do I correctly compare three groups to each other?" Tags="|r|deseq2|differential-expression|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20472" PostTypeId="1" CreationDate="2023-01-31T01:25:00.887" Score="1" ViewCount="105" Body="&lt;p&gt;I have tumor rna seq samples. I want to find out percentage of blood contamination within these samples. What is the best possible way to find % of blood contamination within the rna seq samples?&lt;/p&gt;&#xA;" OwnerUserId="2040" LastActivityDate="2023-01-31T08:22:49.797" Title="blood contamination rna seq" Tags="|rna-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20473" PostTypeId="2" ParentId="20472" CreationDate="2023-01-31T08:22:49.797" Score="5" Body="&lt;p&gt;I assume by &amp;quot;blood contamination&amp;quot; you are interested in tumor purity and immune cell infiltration. For that you can use a deconvolution tool for bulk RNA-seq like &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5895181/&quot; rel=&quot;noreferrer&quot;&gt;CIBERSORT&lt;/a&gt; or &lt;a href=&quot;https://icbi.i-med.ac.at/software/quantiseq/doc/&quot; rel=&quot;noreferrer&quot;&gt;quanTIseq&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2023-01-31T08:22:49.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20474" PostTypeId="2" ParentId="20471" CreationDate="2023-01-31T13:18:19.620" Score="3" Body="&lt;p&gt;I will try to answer this open-ended question with general statements towards the analysis of larger cohorts:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Larger cohorts are usually not homogeneous.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Based on the provided annotations (&lt;code&gt;colData(dat)&lt;/code&gt;) there are cancer samples &lt;strong&gt;and&lt;/strong&gt; normal controls in this dataset, and they're not equal in size. It is 66 &lt;code&gt;Primary solid Tumor&lt;/code&gt; and 25 &lt;code&gt;Solid Tissue Normal&lt;/code&gt;. Running a PCA, for example via &lt;code&gt;plotPCA()&lt;/code&gt; after running &lt;code&gt;vst()&lt;/code&gt; will demonstrate that this is the main source of variation in this dataset, separating along PC1. That means a random split of the dataset will still have more or fewer cancers or normals in one or the other groups, and at that sample size and based on the expected strong expression differences between cancer and normal this will get DEGs almost certainly.&lt;/p&gt;&#xA;&lt;p&gt;Here the PCA via &lt;code&gt;plotPCA(vst(dds, blind=TRUE), &amp;quot;definition&amp;quot;)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Fvv8b.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Fvv8b.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;That having said, you need to inspect the data you deal with in depth before throwing any test on it. If not you might test some sort of confounders or factors that bias the analysis without noticing it. Based on the PCA there is also &lt;strong&gt;notable heterogeneity&lt;/strong&gt; in the cancer samples, and the annotations list additional factors that can contribute to this, be it &lt;strong&gt;gender&lt;/strong&gt;, &lt;strong&gt;smoking status&lt;/strong&gt;, &lt;strong&gt;tumor stage&lt;/strong&gt;, &lt;strong&gt;age&lt;/strong&gt; and others. You would need to ensure that your random assignment does not nest any of these factors or the tumor/normal type with the random groups.&lt;/p&gt;&#xA;&lt;p&gt;Data exploration is key. There was &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-022-02648-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;a recent paper&lt;/a&gt; that got quite some attention on Twitter claiming that common testing frameworks (incl DESeq2) lead to exaggerated false positive rates. However, the DESeq2 author (&lt;a href=&quot;https://github.com/mikelove/preNivolumabOnNivolumab/blob/main/preNivolumabOnNivolumab.knit.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;analysis here&lt;/a&gt;) showed that this paper missed aforementioned proper data exploration, and the dataset infact had unaddressed technical variation and samples forming groups within that dataset. Addressing that (he did it via &lt;code&gt;RUVSeq&lt;/code&gt;) then deemphasized the strong claims in that paper.&lt;/p&gt;&#xA;&lt;p&gt;Summarizing: Large cohorts, especially human ones, often have internal nesting and confounding with other factors that are unwanted. That can be the aforementioned obvious ones like age or gender (sex) but also more subtle ones often not recorded, such as dietary status, unreported drug consumption, amount of physical exercise the person does (if any), genetic predisposition, previous disease or accidents/trauma and so on. That having said, I am quite sure that the &amp;quot;random&amp;quot; DEGs you see are nested unwanted variation rather than anything &amp;quot;wrong&amp;quot; with the testing frameworks.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-01-31T13:18:19.620" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20475" PostTypeId="1" AcceptedAnswerId="20477" CreationDate="2023-01-31T15:46:05.247" Score="1" ViewCount="435" Body="&lt;p&gt;I have a BAM file generated from single-end data and I want to estimate the insert size using picard's &lt;code&gt;CollectInsertSizeMetrics&lt;/code&gt; as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar Picard/picard.jar CollectInsertSizeMetrics I=Ctrl_sorted.bam O=Ctrl.insert_sizes.txt H=Ctrl.insert_sizes.pdf M=0.5 MAX_RECORDS_IN_RAM=2000000 VALIDATION_STRINGENCY=LENIENT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It runs fine until at the very end I get the following message:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;WARNING 2023-01-31 15:37:13 CollectInsertSizeMetrics    All data categories were discarded because they contained &amp;lt; 0.5 of the total aligned paired data.&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Picard obviously thinks the data should be paired-end, so is there a way to tell it that it is single-end? I cannot find an option for this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar picard.jar CollectInsertSizeMetrics -h&#xA;USAGE: CollectInsertSizeMetrics [arguments]&#xA;&#xA;Collect metrics about the insert size distribution of a paired-end library. This tool provides useful metrics for&#xA;validating library construction including the insert size distribution and read orientation of paired-end&#xA;libraries.&amp;lt;/p&amp;gt;The expected proportions of these metrics vary depending on the type of library preparation used,&#xA;resulting from technical differences between pair-end libraries and mate-pair libraries. For a brief primer on&#xA;paired-end sequencing and mate-pair reads, see the &amp;lt;a&#xA;href='https://www.broadinstitute.org/gatk/guide/article?id=6327'&amp;gt;GATK Dictionary&amp;lt;/a&amp;gt;.&amp;lt;p&amp;gt;The CollectInsertSizeMetrics&#xA;tool outputs the percentages of read pairs in each of the three orientations (FR, RF, and TANDEM) as a histogram. In&#xA;addition, the insert size distribution is output as both a histogram (.insert_size_Histogram.pdf) and as a data table&#xA;(.insert_size_metrics.txt).&amp;lt;/p&amp;gt;&amp;lt;p&amp;gt;Note: Metrics labeled as percentages are actually expressed as fractions!&amp;lt;/p&amp;gt;&amp;lt;h4&amp;gt;Usage&#xA;example:&amp;lt;/h4&amp;gt;&amp;lt;pre&amp;gt;java -jar picard.jar CollectInsertSizeMetrics \&amp;lt;br /&amp;gt;      I=input.bam \&amp;lt;br /&amp;gt;     &#xA;O=insert_size_metrics.txt \&amp;lt;br /&amp;gt;      H=insert_size_histogram.pdf \&amp;lt;br /&amp;gt;      M=0.5&amp;lt;/pre&amp;gt;Note: If processing a small&#xA;file, set the minimum percentage option (M) to 0.5, otherwise an error may occur. &amp;lt;br /&amp;gt;&amp;lt;br /&amp;gt;Please see &amp;lt;a&#xA;href='https://broadinstitute.github.io/picard/picard-metric-definitions.html#InsertSizeMetrics'&amp;gt;InsertSizeMetrics&amp;lt;/a&amp;gt;&#xA;for detailed explanations of each metric.&amp;lt;hr /&amp;gt;&#xA;Version:2.23.8&#xA;&#xA;&#xA;Required Arguments:&#xA;&#xA;--Histogram_FILE,-H:File      File to write insert size Histogram chart to.  Required. &#xA;&#xA;--INPUT,-I:File               Input SAM or BAM file.  Required. &#xA;&#xA;--OUTPUT,-O:File              The file to write the output to.  Required. &#xA;&#xA;&#xA;Optional Arguments:&#xA;&#xA;--arguments_file:File         read one or more arguments files and add them to the command line  This argument may be&#xA;                              specified 0 or more times. Default value: null. &#xA;&#xA;--ASSUME_SORTED,-AS:Boolean   If true (default), then the sort order in the header file will be ignored.  Default value:&#xA;                              true. Possible values: {true, false} &#xA;&#xA;--COMPRESSION_LEVEL:Integer   Compression level for all compressed files created (e.g. BAM and VCF).  Default value: 5. &#xA;&#xA;--CREATE_INDEX:Boolean        Whether to create an index when writing VCF or coordinate sorted BAM output.  Default&#xA;                              value: false. Possible values: {true, false} &#xA;&#xA;--CREATE_MD5_FILE:Boolean     Whether to create an MD5 digest for any BAM or FASTQ files created.    Default value:&#xA;                              false. Possible values: {true, false} &#xA;&#xA;--DEVIATIONS:Double           Generate mean, sd and plots by trimming the data down to MEDIAN +&#xA;                              DEVIATIONS*MEDIAN_ABSOLUTE_DEVIATION. This is done because insert size data typically&#xA;                              includes enough anomalous values from chimeras and other artifacts to make the mean and sd&#xA;                              grossly misleading regarding the real distribution.  Default value: 10.0. &#xA;&#xA;--GA4GH_CLIENT_SECRETS:String Google Genomics API client_secrets.json file path.  Default value: client_secrets.json. &#xA;&#xA;--help,-h:Boolean             display the help message  Default value: false. Possible values: {true, false} &#xA;&#xA;--HISTOGRAM_WIDTH,-W:Integer  Explicitly sets the Histogram width, overriding automatic truncation of Histogram tail.&#xA;                              Also, when calculating mean and standard deviation, only bins &amp;lt;= Histogram_WIDTH will be&#xA;                              included.  Default value: null. &#xA;&#xA;--INCLUDE_DUPLICATES:Boolean  If true, also include reads marked as duplicates in the insert size histogram.  Default&#xA;                              value: false. Possible values: {true, false} &#xA;&#xA;--MAX_RECORDS_IN_RAM:Integer  When writing files that need to be sorted, this will specify the number of records stored&#xA;                              in RAM before spilling to disk. Increasing this number reduces the number of file handles&#xA;                              needed to sort the file, and increases the amount of RAM needed.  Default value: 500000. &#xA;&#xA;--METRIC_ACCUMULATION_LEVEL,-LEVEL:MetricAccumulationLevel&#xA;                              The level(s) at which to accumulate metrics.    This argument may be specified 0 or more&#xA;                              times. Default value: [ALL_READS]. Possible values: {ALL_READS, SAMPLE, LIBRARY,&#xA;                              READ_GROUP} &#xA;&#xA;--MIN_HISTOGRAM_WIDTH,-MW:Integer&#xA;                              Minimum width of histogram plots. In the case when the histogram would otherwise&#xA;                              betruncated to a shorter range of sizes, the MIN_HISTOGRAM_WIDTH will enforce a minimum&#xA;                              range.  Default value: null. &#xA;&#xA;--MINIMUM_PCT,-M:Float        When generating the Histogram, discard any data categories (out of FR, TANDEM, RF) that&#xA;                              have fewer than this percentage of overall reads. (Range: 0 to 1).  Default value: 0.05. &#xA;&#xA;--QUIET:Boolean               Whether to suppress job-summary info on System.err.  Default value: false. Possible&#xA;                              values: {true, false} &#xA;&#xA;--REFERENCE_SEQUENCE,-R:File  Reference sequence file.  Default value: null. &#xA;&#xA;--STOP_AFTER:Long             Stop after processing N reads, mainly for debugging.  Default value: 0. &#xA;&#xA;--TMP_DIR:File                One or more directories with space available to be used by this program for temporary&#xA;                              storage of working files  This argument may be specified 0 or more times. Default value:&#xA;                              null. &#xA;&#xA;--USE_JDK_DEFLATER,-use_jdk_deflater:Boolean&#xA;                              Use the JDK Deflater instead of the Intel Deflater for writing compressed output  Default&#xA;                              value: false. Possible values: {true, false} &#xA;&#xA;--USE_JDK_INFLATER,-use_jdk_inflater:Boolean&#xA;                              Use the JDK Inflater instead of the Intel Inflater for reading compressed input  Default&#xA;                              value: false. Possible values: {true, false} &#xA;&#xA;--VALIDATION_STRINGENCY:ValidationStringency&#xA;                              Validation stringency for all SAM files read by this program.  Setting stringency to&#xA;                              SILENT can improve performance when processing a BAM file in which variable-length data&#xA;                              (read, qualities, tags) do not otherwise need to be decoded.  Default value: STRICT.&#xA;                              Possible values: {STRICT, LENIENT, SILENT} &#xA;&#xA;--VERBOSITY:LogLevel          Control verbosity of logging.  Default value: INFO. Possible values: {ERROR, WARNING,&#xA;                              INFO, DEBUG} &#xA;&#xA;--version:Boolean             display the version number for this tool  Default value: false. Possible values: {true,&#xA;                              false} &#xA;&#xA;&#xA;Advanced Arguments:&#xA;&#xA;--showHidden,-showHidden:Boolean&#xA;                              display hidden arguments  Default value: false. Possible values: {true, false} &#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or is there another tool that can deal with single-end data to estimate the insert size?&lt;/p&gt;&#xA;" OwnerUserId="10386" LastEditorUserId="10386" LastEditDate="2023-01-31T15:51:35.440" LastActivityDate="2023-01-31T17:40:55.367" Title="Estimate insert size for single-end data with picard CollectInsertSizeMetrics" Tags="|ngs|sam|picard|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20477" PostTypeId="2" ParentId="20475" CreationDate="2023-01-31T17:40:55.367" Score="5" Body="&lt;p&gt;&lt;code&gt;CollectInsertSizeMetrics&lt;/code&gt; does not estimate but (as by the name) &lt;strong&gt;collects&lt;/strong&gt; insert size metrics, which is nothing different than parsing the TLEN field from the BAM file. It is intended to only be used for paired-end data, not single-end, as single-end data do not have a value in that field in the BAM file. It is calculated by the aligner for paired-end data.&lt;/p&gt;&#xA;&lt;p&gt;What kind of data do you have? For targeted assays such as ChIP-seq the &lt;code&gt;macs2&lt;/code&gt; peak caller has a module to estimate fragment size based on strand cross correlation analysis. I do not kow whether you can use that for any sort of assay.&lt;/p&gt;&#xA;&lt;p&gt;If this is your own data then you can ask the lab person who made the library to look up the size from the Bioanalyzer or TapeStation QC. If it is published data then check the methods text for the procedure/kit. Most NGS types have a relatively standard insert size depending on the exact assay.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-01-31T17:40:55.367" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20478" PostTypeId="1" AcceptedAnswerId="20480" CreationDate="2023-01-31T20:33:03.760" Score="2" ViewCount="144" Body="&lt;p&gt;Hi I am looking to sort a list of kmers into descending order. I used KMC to obtain a list of kmers from a fastq file separated by tab (e.g.; kmer count). From there I have written this python script to attempt to sort the kmers and their counts in descending order:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import csv&#xA;&#xA;with open(&amp;quot;6mers.tsv&amp;quot;) as kmer_file:&#xA;    &#xA;    tsv_reader = csv.DictReader(kmer_file, delimiter=&amp;quot;\t&amp;quot;)&#xA;    &#xA;    for kmer in tsv_reader:&#xA;        &#xA;        name = kmer[&amp;quot;kmer&amp;quot;]&#xA;        &#xA;        count = kmer[&amp;quot;count&amp;quot;]&#xA;        &#xA;        # section below attempts to sort the dictionary into descending order&#xA;        # taken from (https://www.freecodecamp.org/news/sort-dictionary-by-value-in-python/) &#xA;        sorted_kmers = sorted(kmer.items(), key=lambda x:x[1], reverse=True)&#xA;&#xA;        converted_dict = dict(sorted_kmers)&#xA;    &#xA;        print(converted_dict)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The code above outputs this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{'kmer': 'TGCCAA', 'count': '2'}&#xA;{'kmer': 'TGGACA', 'count': '2'}&#xA;{'kmer': 'TGGGAA', 'count': '2'}&#xA;{'kmer': 'TGGTAA', 'count': '5'}&#xA;{'kmer': 'TGTCAA', 'count': '5'}&#xA;{'kmer': 'TGTGAA', 'count': '2'}&#xA;{'kmer': 'TTCCAA', 'count': '13'}&#xA;{'kmer': 'TTGAAA', 'count': '6'}&#xA;{'kmer': 'TTGCAA', 'count': '4'}&#xA;{'kmer': 'TTTAAA', 'count': '6'}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Clearly I am creating a separate dictionary for each kmer and it's count - I'm not sure how but I would like a result that looks more like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{'TTCCAA', '13'}&#xA;{'TTGAAA', '6'}&#xA;{'TTTAAA', '6'}&#xA;{'TGGTAA', '5'}&#xA;{'TGTCAA', '5'}&#xA;{'TTGCAA', '4'}&#xA;{'TGCCAA', '2'}&#xA;{'TGGACA', '2'}&#xA;{'TGGGAA', '2'}&#xA;{'TGTGAA', '2'}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can someone help me fix my code so it works?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The .tsv file looks just like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TGCCAA  2&#xA;TGGACA  2&#xA;TGGGAA  2&#xA;TGGTAA  5&#xA;TGTCAA  5&#xA;TGTGAA  2&#xA;TTCCAA  13&#xA;TTGAAA  6&#xA;TTGCAA  4&#xA;TTTAAA  6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="16067" LastEditDate="2023-01-31T21:22:22.760" LastActivityDate="2023-02-01T00:22:31.807" Title="Sort dictionary in descending order" Tags="|python|k-mer|dictionary|tsv|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20479" PostTypeId="2" ParentId="20478" CreationDate="2023-01-31T21:17:55.077" Score="1" Body="&lt;p&gt;This is untested code ... without seeing to the raw file its tricky.&#xA;Could you try:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import csv&#xA;with open(&amp;quot;6mers.tsv&amp;quot;) as kmer_file:&#xA;    tsv_reader = csv.DictReader(kmer_file, delimiter=&amp;quot;\t&amp;quot;)&#xA;    mydict = {rows[0]:rows[1] for rows in tsv_reader if not rows[0] == 'kmer'}&#xA;&#xA;sorted_kmers = sorted(mydict.items(), key=lambda x:x[1], reverse=True)&#xA;print(sorted_kmers)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm assuming there is a header called 'kmer'&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The output didn't work. The rows['kmers'] infers that (somehow) tsv_reader is already a dictionary (that's not how I thought it worked). Thus try,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import csv&#xA;with open(&amp;quot;6mers.tsv&amp;quot;) as kmer_file:&#xA;    tsv_reader = csv.DictReader(kmer_file, delimiter=&amp;quot;\t&amp;quot;)&#xA;    mydict = {rows['kmers']:rows['count'] for rows in tsv_reader}&#xA;&#xA;sorted_kmers = sorted(mydict.items(), key=lambda x:x[1], reverse=True)&#xA;print(sorted_kmers)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Unless it's loaded as a singular dictionary with Kmer as the key and count as the value the sort line will not work. Once that format is achieved as a dictionary then it will work.&lt;/p&gt;&#xA;&lt;p&gt;What I don't understand is where it's getting 'kmer' or 'count' from if there are no headers in the csv file. Something isn't right if the above code works.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-01-31T21:54:40.403" LastActivityDate="2023-01-31T21:54:40.403" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="20480" PostTypeId="2" ParentId="20478" CreationDate="2023-02-01T00:22:31.807" Score="2" Body="&lt;p&gt;Just make sure to cast the count to an integer prior to sorting. Assuming you need to do something with the counts, you could use:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import csv&#xA;&#xA;from operator import itemgetter&#xA;from pprint import pprint&#xA;&#xA;&#xA;with open(&amp;quot;6mers.tsv&amp;quot;) as kmer_file:&#xA;&#xA;    reader = csv.reader(kmer_file, delimiter=&amp;quot;\t&amp;quot;)&#xA;    counts = ((kmer, int(count)) for kmer, count in reader)&#xA;&#xA;    sorted_kmers = sorted(counts, key=itemgetter(1), reverse=True)&#xA;&#xA;    pprint(sorted_kmers)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[('TTCCAA', 13),&#xA; ('TTGAAA', 6),&#xA; ('TTTAAA', 6),&#xA; ('TGGTAA', 5),&#xA; ('TGTCAA', 5),&#xA; ('TTGCAA', 4),&#xA; ('TGCCAA', 2),&#xA; ('TGGACA', 2),&#xA; ('TGGGAA', 2),&#xA; ('TGTGAA', 2)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-02-01T00:22:31.807" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20481" PostTypeId="1" CreationDate="2023-02-01T01:35:05.867" Score="1" ViewCount="41" Body="&lt;p&gt;Where/How can I download all proteins in the UniProt with signal peptide annotated? I downloaded the uniref50.xml. However, the annotation information is not contained.&lt;/p&gt;&#xA;&lt;p&gt;In an XML record of a protein, the signal peptide information is contained in &lt;code&gt;&amp;lt;feature type=&amp;quot;signal peptide&amp;quot; evidence=&amp;quot;2&amp;quot;&amp;gt; &amp;lt;/feature&amp;gt;&lt;/code&gt;. The uniref.xml does not have this information. uniprot annotates this feature with signalp mainly.&lt;/p&gt;&#xA;" OwnerUserId="16880" LastEditorUserId="73" LastEditDate="2023-02-02T19:08:46.770" LastActivityDate="2023-02-02T19:08:46.770" Title="Where/How can I download all proteins in the UniProt with signal peptide annotated?" Tags="|uniprot|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20485" PostTypeId="1" CreationDate="2023-02-01T10:30:08.920" Score="2" ViewCount="296" Body="&lt;p&gt;Good morning, I study bioinformatics and I need some help with a work. I have the NCBI accession numbers for the species I want for example: NC_037078.1 and I have the specific coordinates where I need to cut them to obtain the 16s gene but I don´t know how to do it. I'm using the language shell script. Is there some way to cut the genome into one specific region?&lt;/p&gt;&#xA;" OwnerUserId="16884" LastEditorUserId="16884" LastEditDate="2023-02-01T12:09:24.110" LastActivityDate="2023-02-01T23:54:05.087" Title="How do I cut a genome into one specific region?" Tags="|shell|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20486" PostTypeId="2" ParentId="20485" CreationDate="2023-02-01T12:28:03.937" Score="3" Body="&lt;p&gt;If you know the coordinates, you could just use &lt;a href=&quot;https://www.htslib.org/doc/samtools-faidx.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;samtools faidx&lt;/code&gt;&lt;/a&gt; to extract the corresponding subsequence from the FASTA file(s). Regions can be specified on the command line with the format: &lt;code&gt;RNAME[:STARTPOS[-ENDPOS]]&lt;/code&gt;. For example, to extract the first ten bases from &lt;code&gt;chr1&lt;/code&gt; in your reference FASTA file, you would use:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools faidx ref.fasta chr1:1-10&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or if you wanted the whole chromosome, you would use:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools faidx ref.fasta chr1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;With regards to your specific example, you could try:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools faidx NC_037078.1.fasta NC_037078.1:96630-98120&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;NC_037078.1:96630-98120&#xA;TCTCATGGAGAGTTCGATCCTGGCTCAGGATGAACGCTGGCGGCATGCTTAACACATGCA&#xA;AGTCGGACGGGAAGTGGTGTTTCCAGTGGCGGACGGGTGAGTAACGCGTAAGAACCTGCC&#xA;CTTGGGAGGGGAACAACAGCTGGAAACGGCTGCTAATACCCCGTAGGCTGAGGAGCAAAA&#xA;GGAGGAATCCGCCCGAGGAGGGGCTTGCGTCTGATTAGCTAGTTGGTGAGGCAATAGCTT&#xA;ACCAAGGCGATGATCAGTAGCTGGTCCGAGAGGATGATCAGCCACACTGGGACTGAGACA&#xA;CGGCCCAGACTCCTACGGGAGGCAGCAGTGGGGAATTTTCCGCAATGGGCGAAAGCCTGA&#xA;CGGAGCAATGCCGCGTGGAGGTAGAAGGCCCACGGGTCGTGAACTTCTTTTCCCGGAGAA&#xA;GAAGCAATGACGGTATCTGGGGAATAAGCATCGGCTAACTCTGTGCCAGCAGCCGCGGTA&#xA;AGACAGAGGATGCAAGCGTTATCCGGAATGATTGGGCGTAAAGCGTCTGTAGGTGGCTTT&#xA;TTAAGTCCGCCGTCAAATCCCAGGGCTCAACCCTGGACAGGCGGTGGAAACTACCAAGCT&#xA;GGAGTACGGTAGGGGCAGAGGGAATTTCCGGTGGAGCGGTGAAATGCGTAGAGATCGGAA&#xA;AGAACACCAACGGCGAAAGCACTCTGCTGGGCCGACACTGACACTGAGAGACGAAAGCTA&#xA;GGGGAGCGAATGGGATTAGATACCCCAGTAGTCCTAGCCGTAAACGATGGATACTGGGCG&#xA;CTGTGCGTATCGACCCGTGCAGTGCTGTAGCTAACGCGTTAAGTATCCCGCCTGGGGAGT&#xA;ACGTTCGCAAGAATGAAACTCAAAGGAATTGACGGGGGCCCGCACAAGCGGTGGAGCATG&#xA;TGGTTTAATTCGATGCAAAGCGAAGAACCTTACCAGGGCTTGACATGCCGCGAATCCTTT&#xA;TGAAAGAGAGGGGTGCCTTCGGGAACGCGGACACAGGTGGTGCATGGCTGTCGTCAGCTC&#xA;GTGCCGTAAGGTGTTGGGTTAAGTCCCGCAACGAGCGCAACCCTCGTGTTTAGTTGCCAT&#xA;CGTTGAATTTGGAACCCTGAACAGACTGCCGGTGATAAGCCGGAGGAAGGTGAGGATGAC&#xA;GTCAAGTCATCATGCCCCTTATGCCCTGGGCGACACACGTGCTACAATGGCCGGGACAAA&#xA;GGGTCGCGATCTCGCGAGGGTGAGCTAACCCCAAAAACCCGTCCTCAGTTCGGATTGCAG&#xA;GCTGCAACTCGCCTGCATGAAGCCGGAATCGCTAGTAATCGCCGGTCAGCCATACGGCGG&#xA;TGAATTCGTTCCCGGGCCTTGTACACACCGCCCGTCACACTATGGGAGCTGGCCATGCCC&#xA;GAAGTCGTTACCTTAACCGCAAGGAGGGGGATGCCGAAGGCAGGGCTAGTGACTGGAGTG&#xA;AAGTCGTAACAAGGTAGCCGTACTGGAAGGTGCGGCTGGATCACCTCCTTT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that one or more regions may be specified on the command line and all position coordinates are 1-based. If the index file (i.e. &lt;code&gt;&amp;lt;ref.fasta&amp;gt;.fai&lt;/code&gt;) doesn't exist, it will be created. If no regions are specified, &lt;code&gt;samtools faidx&lt;/code&gt; will simply create the index file and exit. This is useful if you need to create the index prior to querying it. Only uncompressed FASTA files or FASTA files compressed with &lt;a href=&quot;https://www.htslib.org/doc/bgzip.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bgzip&lt;/a&gt; can be indexed.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ samtools faidx -h&#xA;Usage: samtools faidx &amp;lt;file.fa|file.fa.gz&amp;gt; [&amp;lt;reg&amp;gt; [...]]&#xA;Option: &#xA; -o, --output FILE        Write FASTA to file.&#xA; -n, --length INT         Length of FASTA sequence line. [60]&#xA; -c, --continue           Continue after trying to retrieve missing region.&#xA; -r, --region-file FILE   File of regions.  Format is chr:from-to. One per line.&#xA; -i, --reverse-complement Reverse complement sequences.&#xA;     --mark-strand TYPE   Add strand indicator to sequence name&#xA;                          TYPE = rc   for /rc on negative strand (default)&#xA;                                 no   for no strand indicator&#xA;                                 sign for (+) / (-)&#xA;                                 custom,&amp;lt;pos&amp;gt;,&amp;lt;neg&amp;gt; for custom indicator&#xA;     --fai-idx      FILE  name of the index file (default file.fa.fai).&#xA;     --gzi-idx      FILE  name of compressed file index (default file.fa.gz.gzi).&#xA; -f, --fastq              File and index in FASTQ format.&#xA; -h, --help               This message.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2023-02-01T23:54:05.087" LastActivityDate="2023-02-01T23:54:05.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20487" PostTypeId="1" CreationDate="2023-02-01T15:36:14.993" Score="-1" ViewCount="26" Body="&lt;p&gt;I'm trying to alter this string: &lt;code&gt;NC_037078 1:122545 124035&lt;/code&gt; into this: &lt;code&gt;NC_037078.1:122545-124035&lt;/code&gt; but I don´t know how.&lt;/p&gt;&#xA;&lt;p&gt;The document is like this:\&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    1. rrn16 \&#xA;16S ribosomal RNA [Genlisea aurea]\&#xA;Other Aliases: C7J69_pgr001, BK572_Gau_0120\&#xA;Genomic context: Chloroplast\&#xA;Annotation:  NC_037078.1 (122545..124035, complement)\&#xA;ID:` 36166080&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The script until now looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;code=$(ls -l | grep &amp;quot;Annotation&amp;quot; data.txt | cut -d &amp;quot; &amp;quot; -f3,4 | sed 's/[(]/:/g' | sed 's/[)]/ /g' | sed 's/[,]/ /g' | sed 's/[ ]//g' | sed 's/[&amp;quot;.&amp;quot;]/ /g' )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can you help me solving this problem?&lt;/p&gt;&#xA;" OwnerUserId="16884" LastEditorUserId="3694" LastEditDate="2023-02-01T15:52:27.247" LastActivityDate="2023-02-01T15:57:14.227" Title="How can I tranform this String?" Tags="|shell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20488" PostTypeId="2" ParentId="20487" CreationDate="2023-02-01T15:57:14.227" Score="1" Body="&lt;p&gt;&lt;code&gt;echo NC_037078 1:122545 124035 | sed 's/ /./1' | sed 's/ /:/g'&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note:&#xA;Reference: &lt;a href=&quot;https://stackoverflow.com/questions/59047776/sed-replace-only-first-occurrence-then-only-second-occurrence-then-only-third&quot;&gt;https://stackoverflow.com/questions/59047776/sed-replace-only-first-occurrence-then-only-second-occurrence-then-only-third&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4001" LastActivityDate="2023-02-01T15:57:14.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20489" PostTypeId="1" CreationDate="2023-02-01T16:25:43.443" Score="-1" ViewCount="41" Body="&lt;p&gt;Good afternoon, I'm a bioinformatics student and I'm trying to put some strings between commas with a space. The output looks like this: &lt;code&gt;NC_037078.1:122545-124035,NC_037078.1:96630-98120 &lt;/code&gt;and I want it to be like this: &lt;code&gt;NC_037078.1:122545-124035, NC_037078.1:96630-98120 &lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The document is like this:&lt;br /&gt;&#xA;1. rrn16 &lt;br /&gt;&#xA;16S ribosomal RNA [Genlisea aurea]&lt;br /&gt;&#xA;Other Aliases: C7J69_pgr001, BK572_Gau_0120&lt;br /&gt;&#xA;Genomic context: Chloroplast&lt;br /&gt;&#xA;Annotation:  NC_037078.1 (122545..124035, complement)&lt;br /&gt;&#xA;ID:` 36166080&lt;/p&gt;&#xA;&lt;p&gt;The script until now looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;code=$(ls -l | grep &amp;quot;Annotation&amp;quot; data.txt | cut -d &amp;quot; &amp;quot; -f3,4 | sed 's/[(]/:/g' | sed 's/[)]/ /g' | sed 's/[,]/ /g' | sed 's/[ ]//g' | sed 's/[&amp;quot;.&amp;quot;]/ /g' | sed 's/ /./1' | sed 's/ /-/g' | sed 's/-//1' | tr &amp;quot;\n&amp;quot; &amp;quot;,&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16884" LastEditorUserId="16884" LastEditDate="2023-02-01T17:10:12.747" LastActivityDate="2023-02-01T17:10:12.747" Title="How can I transform this String?" Tags="|shell|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20490" PostTypeId="2" ParentId="20489" CreationDate="2023-02-01T17:03:39.783" Score="1" Body="&lt;p&gt;It seems a small change. Anyway try ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -p -e 's/(.*NC_[0-9]+\.[0-9]),(NC_[0-9]+\.[0-9].*)/&lt;span class=&quot;math-container&quot;&gt;$1,\s$&lt;/span&gt;2/g' data.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you have happy that this works, then repeat with an &lt;code&gt;-i&lt;/code&gt; flag&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -p -i -e 's/(.*NC_[0-9]+\.[0-9]),(NC_[0-9]+\.[0-9].*)/&lt;span class=&quot;math-container&quot;&gt;$1,\s$&lt;/span&gt;2/g' data.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've not checked the code, but it's worth checking whether &lt;code&gt;.*&lt;/code&gt; is really needed. Its a while since I've used Perl&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-02-01T17:03:39.783" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20491" PostTypeId="1" AcceptedAnswerId="20492" CreationDate="2023-02-01T21:59:07.107" Score="3" ViewCount="148" Body="&lt;p&gt;I have a list of kmers and their frequencies broken down like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TGCCAA  2&#xA;TGGACA  2&#xA;TGGGAA  2&#xA;TGGTAA  5&#xA;TGTCAA  5&#xA;TGTGAA  2&#xA;TTCCAA  13&#xA;TTGAAA  6&#xA;TTGCAA  4&#xA;TTTAAA  6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And I would like to use clustering to look at the most frequent and the most unique instances of kmers in a sample. I started off following the tutorial found here (&lt;a href=&quot;https://www.w3schools.com/python/python_ml_hierarchial_clustering.asp#:%7E:text=Hierarchical%20clustering%20is%20an%20unsupervised,need%20a%20%22target%22%20variable.&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.w3schools.com/python/python_ml_hierarchial_clustering.asp#:~:text=Hierarchical%20clustering%20is%20an%20unsupervised,need%20a%20%22target%22%20variable.&lt;/a&gt;) using the scikit-learn python package.&lt;/p&gt;&#xA;&lt;p&gt;However, when I go to implement my data I get a ValueError where it can not convert string to float, which seems obvious as the tutorial uses two sets of numbers and not strings.&lt;/p&gt;&#xA;&lt;p&gt;I was wondering if there was a known way to cluster kmers and their counts or simply just strings.&lt;/p&gt;&#xA;" OwnerUserId="16067" LastActivityDate="2023-02-02T16:09:40.310" Title="hierarchical clustering of kmers and their counts" Tags="|python|clustering|k-mer|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20492" PostTypeId="2" ParentId="20491" CreationDate="2023-02-01T23:05:12.517" Score="2" Body="&lt;p&gt;This isn't good data for clustering based on the count value because it needs to be multivariate, e.g. you'd need two or more 'count' columns (if that makes any biological sense). This data is univariate so I'd just use a standard bar chart, personally I would rank the data and consider the underlying distribution.&lt;/p&gt;&#xA;&lt;p&gt;You could make a tree of the sequence data and the data counts are the labels. If you want to make a tree (hierarchical clustering) then using nucleotide data its becomes a phylogenetic tree, which isn't hierarchical clustering - although clustering can be used to make phylogenetic trees.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;K-mers ... if these sites are not homologous a phylogenetic tree is not a good solution: they do look like homologous however. In terms of shifting to numerical variables the technical term is called &lt;code&gt;one-hot-key&lt;/code&gt; encoding and scikit-learn has a function for this. This is what Steve is saying. However, understanding what the k-mer represents is kinda important. To perform one-hot-key encoding its:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from sklearn.preprocessing import OneHotEncoder&#xA;hot = OneHotEncoder()&#xA;.... gets complicated&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alternative approach&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;df = pd.read_csv(&amp;quot;kmer.csv&amp;quot;)&#xA;one_hot_encode = pd.get_dummies(df, columns = ['kmers'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This code will transform your data to what Steve said .. keeping it as &lt;code&gt;pandas&lt;/code&gt; is sort of useful because hierarchical clustering will take a pandas input.&lt;/p&gt;&#xA;&lt;p&gt;I do have doubts ... the &lt;code&gt;counts&lt;/code&gt; column needs a unique identifier &lt;em&gt;in addition&lt;/em&gt; to the number, because you'll get identical labels otherwise (my original idea had limitations). Ultimately, I don't precisely know what the kmer value biologically means and thats essential. I would be cautious about dumping that straight into a hierarchical clustering algorithm (it will be in &lt;code&gt;scipy&lt;/code&gt; BTW not &lt;code&gt;scikit-learn&lt;/code&gt;) .&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The following would provide a phylogenetic tree input, but statistically its dodgy without understanding what the k-mer means&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;&#xA;kmer = ['TGCCAA', 'TGGACA', 'TGGGAA', 'TGGTAA', 'TGTCAA', 'TGTGAA',  'TTCCAA', 'TTGAAA', 'TTGCAA', 'TTTAAA']&#xA;&#xA;count = [2, 2,  2, 5, 5, 2, 13,  6,  4,  6]&#xA;&#xA;x = [chr(i) for i in range(ord('a'),ord('z')+1)]&#xA;n = 0&#xA;mydict = {}&#xA;for c,k in zip (count, kmer):&#xA;    n += 1&#xA;    mydict[str(c)+x[n]] = k&#xA;    if n &amp;gt; 25:&#xA;        n = 0 &#xA;for key in mydict:&#xA;    if mydict[key] == None:&#xA;        continue&#xA;    print ('&amp;gt;' + key + '\n' + mydict[key] + '\n')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;2b&#xA;TGCCAA&#xA;&amp;gt;2c&#xA;TGGACA&#xA;&amp;gt;2d&#xA;TGGGAA&#xA;&amp;gt;5e&#xA;TGGTAA&#xA;&amp;gt;5f&#xA;TGTCAA&#xA;&amp;gt;2g&#xA;TGTGAA&#xA;&amp;gt;13h&#xA;TTCCAA&#xA;&amp;gt;6i&#xA;TTGAAA&#xA;&amp;gt;4j&#xA;TTGCAA&#xA;&amp;gt;6k&#xA;TTTAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here's the tree&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/y3Nk0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/y3Nk0.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;All I can conclude is count and kmer - with one exception &lt;code&gt;TTTAAA&lt;/code&gt; and &lt;code&gt;TTGAAA&lt;/code&gt; are not correlated. &lt;code&gt;TTRAAA&lt;/code&gt; maybe be a result, possibly.&lt;/p&gt;&#xA;&lt;p&gt;Here's the matrix.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1fOzj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1fOzj.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&#xA;Actually there might be a result here: the k-mer representing '13' is a genetic outlier. Whilst it might be difficult to see in the tree I've presented that could proven more decisively. If a bigger sample size demonstrated comparable phenomena - that is a result for certain. So maybe it's not such a bad analysis after all - it could lead to notably interesting result. Hierarchical clustering BTW couldn't generate that results - its assumption base removes the outlier effect.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-02-02T16:09:40.310" LastActivityDate="2023-02-02T16:09:40.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20493" PostTypeId="2" ParentId="20491" CreationDate="2023-02-02T01:27:38.133" Score="3" Body="&lt;p&gt;I think you need some way to convert the k-mer strings into a numerical representation so that they can be clustered. There's a few ways to do this. For example, you could one-hot (or two-bit) encode each k-mer so that each nucleotide is represented as a set of binary values. Or you could just use a simple integer encoding. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def int_encode(kmer):&#xA;    two_bit = {'A': 0, 'C': 1, 'G': 2, 'T': 3}&#xA;    encoding = []&#xA;    for nucleotide in kmer:&#xA;        encoding.append(two_bit[nucleotide])&#xA;    return encoding&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You could then concatenate the output with the frequency counts of each k-mer. For your set of k-mers, your input for clustering would then look like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[3, 2, 1, 1, 0, 0, 2]&#xA;[3, 2, 2, 0, 1, 0, 2]&#xA;[3, 2, 2, 2, 0, 0, 2]&#xA;[3, 2, 2, 3, 0, 0, 5]&#xA;[3, 2, 3, 1, 0, 0, 5]&#xA;[3, 2, 3, 2, 0, 0, 2]&#xA;[3, 3, 1, 1, 0, 0, 13]&#xA;[3, 3, 2, 0, 0, 0, 6]&#xA;[3, 3, 2, 1, 0, 0, 4]&#xA;[3, 3, 3, 0, 0, 0, 6]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-02-02T01:27:38.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20494" PostTypeId="1" CreationDate="2023-02-02T05:51:05.420" Score="2" ViewCount="328" Body="&lt;p&gt;So I have a molecule (a dsRNA) , that has two conformations. I want to align it in pymol with molecules but I get the following error: ExecutiveAlign error: atomic alignment failed (mismatched identifiers? )&lt;/p&gt;&#xA;&lt;p&gt;So I try removing the conformations and maintain the spaces also, however no luck. INfact after editing and displaying the sequence in pymol, the chain ids seem to get interleaved as follows:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/HQhjX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HQhjX.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;I am writing here sample text from the molecule that has both the conformations:&lt;/p&gt;&#xA;&lt;p&gt;ATOM      1  O3'A  C C   1      -7.005  37.688  50.581  0.60 29.09           O&lt;br /&gt;&#xA;ATOM      2  O3'B  C C   1      -6.653  36.261  51.094  0.40 28.26           O&lt;br /&gt;&#xA;ATOM      3  P  A  C C   1      -8.284  38.506  50.584  0.60 27.34           P&lt;br /&gt;&#xA;ATOM      4  P  B  C C   1      -8.150  36.411  51.318  0.40 28.35           P&lt;br /&gt;&#xA;ATOM      5  O1PA  C C   1      -8.747  38.916  51.969  0.60 27.60           O&lt;br /&gt;&#xA;ATOM      6  O1PB  C C   1      -8.852  35.063  51.303  0.40 26.74           O&lt;/p&gt;&#xA;&lt;p&gt;and the molecule after removing the conformers, editing the identifier number, and maintaining the spaces as found in usual PDB files:&lt;/p&gt;&#xA;&lt;p&gt;From chain C:&#xA;ATOM     1  O3'   C C   1      -7.005  37.688  50.581  0.60 29.09           O&lt;br /&gt;&#xA;ATOM     2  P     C C   1      -8.284  38.506  50.584  0.60 27.34           P&lt;br /&gt;&#xA;ATOM     3  O1P   C C   1      -8.747  38.916  51.969  0.60 27.60           O&lt;br /&gt;&#xA;ATOM     4  O2P   C C   1      -8.241  39.648  49.584  0.60 28.40           O&lt;br /&gt;&#xA;ATOM     5  O5'   C C   1      -9.442  37.507  50.040  0.60 28.29           O&lt;/p&gt;&#xA;&lt;p&gt;From chain D:&#xA;TER    446        U C  21&#xA;ATOM   447  O3'   C D   1     -11.398  37.360  86.614  0.60 24.03           O&lt;br /&gt;&#xA;ATOM   448  P     C D   1     -11.903  36.706  87.889  0.60 26.14           P&lt;br /&gt;&#xA;ATOM   449  O1P   C D   1     -12.702  37.680  88.743  0.60 28.36           O&lt;/p&gt;&#xA;&lt;p&gt;Please help, at the end of the day I would like to align this molecule with other molecules in pymol without getting the error. Thanks&lt;/p&gt;&#xA;" OwnerUserId="16888" LastActivityDate="2023-02-02T05:51:05.420" Title="ExecutiveAlign error: atomic alignment failed (mismatched identifiers? )" Tags="|rna|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20495" PostTypeId="1" AcceptedAnswerId="20506" CreationDate="2023-02-02T18:15:37.293" Score="3" ViewCount="776" Body="&lt;p&gt;I'm plotting scRNA data using Violin Plots.  I need to re-order the plots and add median dots.  My code is:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;Vln_plot &amp;lt;- VlnPlot(object, &#xA;                    features = c(&amp;quot;Mcidas&amp;quot;), &#xA;                    cols = c(&amp;quot;coral1&amp;quot;, &amp;quot;cadetblue&amp;quot;, &amp;quot;cadetblue2&amp;quot;), &#xA;                    split.by = &amp;quot;genotype&amp;quot;,&#xA;                    assay = &amp;quot;RNA&amp;quot;,&#xA;                    pt.size = 0,&#xA;                    log = TRUE) + &#xA;  stat_summary(fun.y = median.stat, geom='point', size = 10, colour = &amp;quot;blue&amp;quot;,&#xA;               position = position_dodge(0.9)) &#xA;  &#xA;&#xA;Vln_plot&lt;span class=&quot;math-container&quot;&gt;$data$&lt;/span&gt;split&amp;lt;- factor(x = Vln_plot&lt;span class=&quot;math-container&quot;&gt;$data$&lt;/span&gt;split,&#xA;                                 levels = c(&amp;quot;WT&amp;quot;, &amp;quot;F7KO&amp;quot;, &amp;quot;F8HET&amp;quot;))&#xA;&#xA;Vln_plot&#xA;&#xA;median.stat &amp;lt;- function(x){&#xA;  out &amp;lt;- quantile(x, probs = c(0.5))&#xA;  names(out) &amp;lt;- c(&amp;quot;ymed&amp;quot;)&#xA;  return(out) &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and my output looks like:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/cRmvG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cRmvG.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is there any way to fix the legend so those dots aren't in each box?  I also need to figure out how to get rid of that x-axis tic mark and change the x axis label from Identity to something else.  Can anyone point me in the right direction on how to accomplish this?&lt;/p&gt;&#xA;" OwnerUserId="16894" LastEditorUserId="650" LastEditDate="2023-02-02T18:53:20.197" LastActivityDate="2023-02-08T00:53:29.307" Title="How do I change the legend for a Violin Plot with median dot" Tags="|r|seurat|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20496" PostTypeId="2" ParentId="18928" CreationDate="2023-02-03T04:23:16.163" Score="1" Body="&lt;p&gt;The -2 to 2 should be the normalized version of log2 data (maybe mean or median normalization). This is commonly used in heatmaps to show downregulated value vs. upregulated value.&lt;/p&gt;&#xA;" OwnerUserId="16898" LastActivityDate="2023-02-03T04:23:16.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20497" PostTypeId="1" CreationDate="2023-02-03T08:32:00.753" Score="1" ViewCount="358" Body="&lt;p&gt;When I use cellranger or cellbender to filter cell from raw RNA count matrix, the results they output are allways different. Each of then can call some cells not appearing in other's result. SO, how do cellranger and cellbender call cells? What is the difference between them?&lt;/p&gt;&#xA;" OwnerUserId="16899" LastActivityDate="2023-07-06T10:08:46.070" Title="How do cellranger and cellbender call cells? What is the difference between them?" Tags="|scrnaseq|cellranger|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20498" PostTypeId="1" CreationDate="2023-02-03T09:46:09.400" Score="4" ViewCount="178" Body="&lt;p&gt;Dealing whith a problematic sequencing run I found this over-represented sequence:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;GGAAGAGCACACGTCTGAACTCCAGTCAC&lt;/strong&gt;TAGCTT&lt;strong&gt;ATCTCGTATGGCGTCTTCTGCTTG&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It is clearly relate to the Illumina sequencing primer as shown by this read structure description (note the bold residues above):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/oL4Sr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/oL4Sr.png&quot; alt=&quot;illumina primers&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I searched the sequence in the nucleotide NR ncbi database, using blastn (standard parameters) and I found this as a first hit:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Severe acute respiratory syndrome coronavirus 2 genome assembly, chromosome: 1&#xA;Sequence ID: OV196492.1Length: 30243Number of Matches: 2&#xA;Range 1: 73 to 133GenBankGraphicsNext MatchPrevious Match&#xA;Alignment statistics for match #1&#xA;Score   Expect  Identities  Gaps    Strand&#xA;89.8 bits(48)   2e-14   57/61(93%)  2/61(3%)    Plus/Minus&#xA;Query  1    GGAAGAGCACACGTCTGAACTCCAGTCACTAG--CTTATCTCGTATGGCGTCTTCTGCTT  58&#xA;            ||||||||||||||||||||||||||||| ||  ||||||||||||| ||||||||||||&#xA;Sbjct  133  GGAAGAGCACACGTCTGAACTCCAGTCACCAGTGCTTATCTCGTATGCCGTCTTCTGCTT  74&#xA;&#xA;Query  59   G  59&#xA;            |&#xA;Sbjct  73   G  73&#xA;&#xA;&#xA;Range 2: 188 to 248GenBankGraphicsNext MatchPrevious MatchFirst Match&#xA;Alignment statistics for match #2&#xA;Score   Expect  Identities  Gaps    Strand&#xA;89.8 bits(48)   2e-14   57/61(93%)  2/61(3%)    Plus/Minus&#xA;Query  1    GGAAGAGCACACGTCTGAACTCCAGTCACTAG--CTTATCTCGTATGGCGTCTTCTGCTT  58&#xA;            ||||||||||||||||||||||||||||| ||  ||||||||||||| ||||||||||||&#xA;Sbjct  248  GGAAGAGCACACGTCTGAACTCCAGTCACCAGTGCTTATCTCGTATGCCGTCTTCTGCTT  189&#xA;&#xA;Query  59   G  59&#xA;            |&#xA;Sbjct  188  G  188&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There are also sequences of other virues that match very well.&lt;/p&gt;&#xA;&lt;p&gt;Do you know what is the relation between Illumina adapters/primers and viral seqiuences?&lt;/p&gt;&#xA;" OwnerUserId="3772" LastEditorUserId="3772" LastEditDate="2023-02-11T14:06:28.577" LastActivityDate="2023-02-11T14:06:28.577" Title="relation between Illumina sequencing primer and viral sequences" Tags="|covid-19|illumina|virus|coronavirus|adapter|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20499" PostTypeId="2" ParentId="20498" CreationDate="2023-02-03T12:21:47.713" Score="8" Body="&lt;p&gt;You'll unfortunately find adapter sequences contaminating a lot of entries in Genbank. In short, the sequence is most likely either an unassembled fragment where the input into the assembly step hadn't had adapters trimmed or (somewhat more rarely) the adapter sequence was incorporated into an assembled contig and then uploaded. You'll see a lot of lower quality sequences like this, especially from massive viral screens.&lt;/p&gt;&#xA;&lt;p&gt;Be aware that there are other quality issues facing particularly virus sequences such as extremely high NNN stretches (cf. the difficulties faced in creating the C-RVDB viral database once there were hundreds of thousands of apparently poor-quality coronavirus sequences automatically added to Genbank).&lt;/p&gt;&#xA;" OwnerUserId="77" LastActivityDate="2023-02-03T12:21:47.713" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20500" PostTypeId="1" AcceptedAnswerId="20501" CreationDate="2023-02-03T12:38:11.550" Score="1" ViewCount="30" Body="&lt;p&gt;ALT ID CN2, CN3,CN4,CN5 are regions of elevated copy number relative to the reference, Then what does the ID =DUP separately cover? I am new to these analyses. please help&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##ALT=&amp;lt;ID=DUP, Description=&amp;quot;Region of elevated copy number relative to the reference&amp;quot;&amp;gt;&#xA;##ALT=&amp;lt;ID=CNO, Description=&amp;quot;Copy number allele: 0 copies&amp;quot;&amp;gt;&#xA;##ALT=&amp;lt;ID=CN2, Description=&amp;quot;Copy number allele: 2 copies&amp;quot;&amp;gt;&#xA;##ALT=&amp;lt;ID=CN3, Description=&amp;quot;Copy number allele: 3 copies&amp;quot;&amp;gt;&#xA;##ALT=&amp;lt;ID=CN4, Description=&amp;quot;Copy number allele: 4 copies&amp;quot;&amp;gt;&#xA;##ALT=&amp;lt;ID=CN5, Description=&amp;quot;Copy number allele: 5 copies&amp;quot;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16900" LastEditorUserId="73" LastEditDate="2023-02-03T17:43:35.867" LastActivityDate="2023-02-04T11:42:38.663" Title="what does ALT ID DUP refer to when ALT ID for copy number are already specified?" Tags="|vcf|illumina|cnv|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20501" PostTypeId="2" ParentId="20500" CreationDate="2023-02-03T12:51:32.297" Score="1" Body="&lt;p&gt;The details will depend on the specific tool used to generate the VCF. Each tool has its own system and there is not much standardization for these fields. That said, the likeliest situation here is that if the CNV caller cannot determine the copy number, but only that there is a duplication, it just puts DUP. And if it can determine the specific copy number, then it uses one of the specific terms.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, if the copy number is beyond those numbers with a specific &lt;code&gt;CNN&lt;/code&gt; value (say you have 7 copies, or 65 or whatever), then it would presumably also use a &lt;code&gt;DUP&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="10775" LastEditDate="2023-02-04T11:42:38.663" LastActivityDate="2023-02-04T11:42:38.663" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20502" PostTypeId="1" AcceptedAnswerId="20654" CreationDate="2023-02-03T15:16:05.597" Score="1" ViewCount="59" Body="&lt;p&gt;I'm trying to work out is how reliable/accurate my eggNOG metagenomic outputs are. Critical for this is how its identifying orthologues.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; &lt;code&gt;eggNOG&lt;/code&gt; is commonly used in metagenomics gene identification. Take a sample, randomly sequence all the DNA therein, then &amp;quot;annotate&amp;quot; via eggNOG. It identifies the putative gene function against lots of databases e.g. NCBI, KEGG, Brite, Go ...&lt;/p&gt;&#xA;&lt;p&gt;eggNOG uses is own database for gene identification via 'seed orthology' each seed has its own KEGG ID.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; Does the comparison between the query (metagenomics data) and the &lt;em&gt;seeds&lt;/em&gt; take place using DNA-DNA comparisons? Alternatively, is it performing protein-protein comparisons? Algorithmically, by the easiest is DNA-DNA, however DNA-DNA would result in limitations of interpretation ('cause of complex evolutionary stuff). It could be using an optimised a protein database instead and returning the output as DNA.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-02-03T15:23:04.813" LastActivityDate="2023-03-02T17:06:59.687" Title="eggNOG: DNA-DNA or protein-protein orthologue identification?" Tags="|metagenome|kegg|go|eggnog|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20503" PostTypeId="1" CreationDate="2023-02-03T15:37:37.403" Score="-1" ViewCount="546" Body="&lt;p&gt;I have done DE analysis using &lt;a href=&quot;https://scanpy.readthedocs.io/en/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SCANpy&lt;/a&gt; on my single cell data and I have compared each cluster versus all the other clusters. One cluster seems particularly interesting so I wanted to do pathway analysis.&lt;/p&gt;&#xA;&lt;p&gt;I have already tried to do the pathway analysis but I got confused. I initially followed the instructions from here (&lt;a href=&quot;https://nbisweden.github.io/workshop-scRNAseq/labs/compiled/scanpy/scanpy_05_dge.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nbisweden.github.io/workshop-scRNAseq/labs/compiled/scanpy/scanpy_05_dge.html&lt;/a&gt;) and did the enrichment and then I also wanted to do GSEA so I used gseapy.prerank. However, then I was trying to find a way to visualize the network so I ended up going to the gseapy documentation.&lt;/p&gt;&#xA;&lt;p&gt;The steps of the enrichment analysis seemed the same but then I got confused with the GSEA portion. The first tutorial I found used gseapy.prerank but the second one uses gp.gsea. I was trying to understand the difference and it seems that since I already have the DE analysis I should be using the gseapy.prerank right bit I am not sure.&lt;/p&gt;&#xA;&lt;p&gt;I was wondering if doing the analysis trough KEGG would be easier.&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Gene set enrichment analysis (GSEA) is a commonly used algorithm for characterizing gene expression changes. However, the currently available tools used to perform GSEA have a limited ability to analyze large datasets, which is particularly problematic for the analysis of single-cell data. To overcome this limitation, we developed a GSEA package in Python (GSEApy), which could efficiently analyze large single-cell datasets.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="16902" LastEditorUserId="73" LastEditDate="2023-02-07T21:57:26.237" LastActivityDate="2023-02-07T21:57:26.237" Title="How to do pathway analysis after scanpy for single cell data after DE analysis?" Tags="|single-cell|pathway|gsea|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20504" PostTypeId="2" ParentId="20503" CreationDate="2023-02-03T17:36:08.093" Score="0" Body="&lt;p&gt;The original question was GSEApy vs KEGG. The response below is GSEApy.  The question has been edited and is now a  different question concerning options within GSEApy.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;em&gt;Response to original question&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;If GSEApy as described in the publication provides the output you require - that is much, much simpler. KEGG will be complex and you'd need its ids.&lt;/p&gt;&#xA;&lt;p&gt;However, surely GSEA will require to perform the full calculation of fold-change from scratch rather than be fed the output of scanpy? KEGG will require the output of scanpy. GSEApy primary function is fold-change rather identifying the precisely which pathways are regulating the phenotype under analysis.&lt;/p&gt;&#xA;&lt;p&gt;KEGG will provide a more accurate answer - in my humble opinion. GSEApy is a lot easier because it very much an &amp;quot;all-in-one&amp;quot; solution. If the answer isn't satisfactory you can always switch.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Just to note the GSEApy version I'm referring to is the latest &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/39/1/btac757/6847088&quot; rel=&quot;nofollow noreferrer&quot;&gt;publication&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;NOTE&lt;/strong&gt; There was a deleted post from the OP about an option within GSEA. This is a separate question in the SE format of questions and answers. Thats the way the site works.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-02-03T18:55:07.697" LastActivityDate="2023-02-03T18:55:07.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20506" PostTypeId="2" ParentId="20495" CreationDate="2023-02-03T18:03:54.490" Score="1" Body="&lt;p&gt;Your command looks like it creates a ggplot2 object, so it should be possible to remove the symbol via &lt;code&gt;guides&lt;/code&gt;, setting the associated aesthetic to &lt;code&gt;&amp;quot;none&amp;quot;&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Vln_plot &amp;lt;- VlnPlot(object, &#xA;                    features = c(&amp;quot;Mcidas&amp;quot;), &#xA;                    cols = c(&amp;quot;coral1&amp;quot;, &amp;quot;cadetblue&amp;quot;, &amp;quot;cadetblue2&amp;quot;), &#xA;                    split.by = &amp;quot;genotype&amp;quot;,&#xA;                    assay = &amp;quot;RNA&amp;quot;,&#xA;                    pt.size = 0,&#xA;                    log = TRUE) + &#xA;  stat_summary(fun.y = median.stat, geom='point', size = 10, colour = &amp;quot;blue&amp;quot;,&#xA;               position = position_dodge(0.9)) +&#xA;  guides(color = &amp;quot;none&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If that doesn't work, try telling the &lt;code&gt;stat_summary&lt;/code&gt; command to disable the legend via &lt;code&gt;show.legend = FALSE&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Vln_plot &amp;lt;- VlnPlot(object, &#xA;                    features = c(&amp;quot;Mcidas&amp;quot;), &#xA;                    cols = c(&amp;quot;coral1&amp;quot;, &amp;quot;cadetblue&amp;quot;, &amp;quot;cadetblue2&amp;quot;), &#xA;                    split.by = &amp;quot;genotype&amp;quot;,&#xA;                    assay = &amp;quot;RNA&amp;quot;,&#xA;                    pt.size = 0,&#xA;                    log = TRUE) + &#xA;  stat_summary(fun.y = median.stat, geom='point', size = 10, colour = &amp;quot;blue&amp;quot;,&#xA;               position = position_dodge(0.9), show.legend = FALSE) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-02-03T18:03:54.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20507" PostTypeId="1" CreationDate="2023-02-03T20:26:26.760" Score="2" ViewCount="35" Body="&lt;p&gt;I'm looking for a standard to access terminology services. A terminology service provides an generalized API to multiple terminologies like ICD10, MedDRA, Snomed, Loinc etc. But I'm looking for a generalized API over multiple terminology services. I don't want to implement an interface for each terminology service which the users need.&lt;/p&gt;&#xA;&lt;p&gt;I found some things like OMG Common Terminology Services v1.2 (uses SOAP) or HL7/FHIR (uses REST). But I'm not familiar with the current status of a global standard in this area. Is there such a standard?&lt;/p&gt;&#xA;&lt;p&gt;As an example of a terminology service I have found &lt;a href=&quot;https://digital.nhs.uk/services/terminology-servers&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://digital.nhs.uk/services/terminology-servers&lt;/a&gt;. It provides access to multiple terminologies. There are other such service providers with its own service.&lt;/p&gt;&#xA;&lt;p&gt;Every implementation of such a service around the world seems to use different ways (REST, SOAP returns JSON, XML) to retrieve data from the underlying terminologies. This is enough for an user who uses only one service. But I have to query a lot of such services because they provide different terminologies I need.&lt;/p&gt;&#xA;&lt;p&gt;This has nothing to do with hardware. I'm looking for a generalized standardized API for multiple terminologies in the medical/biomedical area like the &lt;a href=&quot;https://digital.nhs.uk/services/terminology-servers&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://digital.nhs.uk/services/terminology-servers&lt;/a&gt; example I gave above so I don't need to write a different interface for every source.&lt;/p&gt;&#xA;" OwnerUserId="16903" LastEditorUserId="298" LastEditDate="2023-02-07T12:02:40.717" LastActivityDate="2023-02-07T12:02:40.717" Title="Is there a standard API for terminology services?" Tags="|terminology|webservice|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20508" PostTypeId="2" ParentId="20498" CreationDate="2023-02-03T22:09:05.970" Score="0" Body="&lt;p&gt;SARS-CoV-2 NCBI no. OV196492.1&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; The relationship sought is simply the genome position because this is the UTR (UnTranslated Region).&lt;/p&gt;&#xA;&lt;p&gt;The linker is present starting at position 74 and is also present at position of 189 of the SARS-CoV-2 genome from 5' to 3' (I checked). This is where position 1 is not necessarily position 1 of the genome, but the start of the authors genome. So it's the UTR. I did suspect this was the location.&lt;/p&gt;&#xA;&lt;p&gt;There are three reasons for the drop in sequence quality at the UTR:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;overall quality, really the read depth, of short-read sequence data is reduced at the terminal regions of viruses &lt;em&gt;per se&lt;/em&gt;;&lt;/li&gt;&#xA;&lt;li&gt;there is extensive secondary structure in the UTR regions which is critical to its function, thus the folding will affect short-read sequencing approaches, e.g. primer binding will be more difficult because these sequences will preferentially self-anneal (preventing the primer from binding);&lt;/li&gt;&#xA;&lt;li&gt;QC annotation from the authors will focus on the coding regions, the UTR is outside the coding regions and therefore can get missed.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In this particular case point 3 is likely the underlying reason. The authors QC ignored the UTRs (the 3' UTR is also problematic). If a linker is caught in the coding domains the translation will flag it immediately ... obviously the UTR isn't translated. Thats simply why. For virologists, this isn't cool, but for clinicians they don't see its importance.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Overall&lt;/strong&gt;, this is simply an issue of &lt;em&gt;ad hoc&lt;/em&gt; QC because the NCBI SARS-CoV-2 database is extremely powerful and excellent for custom analysis. To &amp;quot;throw in the towel&amp;quot; on the power and flexibility of NCBI's SARS-CoV-2 database for what is essentially some personal QC coding isn't worth the trade-off in my personal opinion. It's a powerful portal, which enables an analysts to focus on downstream issues.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-02-03T23:13:53.527" LastActivityDate="2023-02-03T23:13:53.527" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20509" PostTypeId="1" CreationDate="2023-02-03T22:19:08.017" Score="4" ViewCount="1168" Body="&lt;p&gt;I have generated several FASTQ files and I would like to know the amount of reads for each of them.&#xA;I am planning to run &lt;a href=&quot;https://www.bioinformatics.babraham.ac.uk/projects/fastqc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FastQC&lt;/a&gt; on the files which I know would give me the number of reads per sample but it would be a lot easier for me if I could get one text file with the file name and number of reads for all samples together.&lt;/p&gt;&#xA;&lt;p&gt;When I check for one file I normally do &lt;code&gt;zcat filename_R1.fastq.gz | wc -l&lt;/code&gt; and then divide by 4.&#xA;Is there a way to do it for all my files at once and get a text file with the file name and reads at the end?&lt;/p&gt;&#xA;&lt;p&gt;For example I will have 71 R1.fastq.gz files and I would like to output the amount of reads to a text file. I just meant that I would prefer to use one command instead of having to use the command I mentioned above for each file.&lt;/p&gt;&#xA;" OwnerUserId="16902" LastEditorUserId="73" LastEditDate="2023-02-05T19:45:27.320" LastActivityDate="2023-02-05T20:24:51.693" Title="How to get a file with the number of reads for several fastq.gz files?" Tags="|fastq|linux|command-line|" AnswerCount="4" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20511" PostTypeId="2" ParentId="20509" CreationDate="2023-02-04T00:03:49.843" Score="1" Body="&lt;p&gt;You can replace &lt;code&gt;gunzip -c&lt;/code&gt; with &lt;code&gt;zcat&lt;/code&gt;.&lt;/p&gt;&#xA;   &#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;#USAGE: script.sh /runFolder&#xA;for i in &lt;span class=&quot;math-container&quot;&gt;$1/*.1.fastq.gz&#xA;do&#xA;   echo &quot;$&lt;/span&gt;i&amp;quot; &lt;span class=&quot;math-container&quot;&gt;$(gunzip -c $&lt;/span&gt;i | echo `wc -l`/4 | bc -l)&#xA;done&#xA;&#xA;#OR in the current folder, do this:&#xA;for i in ./*.1.fastq.gz&#xA;do&#xA;   echo &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$i&quot; $&lt;/span&gt;(gunzip -c $i | echo `wc -l`/4 | bc -l)&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14180" LastEditorUserId="14180" LastEditDate="2023-02-04T00:08:56.743" LastActivityDate="2023-02-04T00:08:56.743" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20512" PostTypeId="2" ParentId="20509" CreationDate="2023-02-04T12:50:39.777" Score="5" Body="&lt;p&gt;One way would be to use &lt;a href=&quot;https://bioinf.shenwei.me/seqkit/usage/#stats&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;seqkit stats&lt;/code&gt;&lt;/a&gt;, which can be installed using &lt;code&gt;conda&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c bioconda seqkit&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It has a number of useful options, including ways to parallelize the counting, using &lt;code&gt;-j&lt;/code&gt;, &lt;code&gt;--threads&lt;/code&gt; (default 4), and to skip errors using &lt;code&gt;-e&lt;/code&gt;, &lt;code&gt;--skip-err&lt;/code&gt;. The latter can be especially useful if you have a larger number of FASTQ files with millions of reads and you'd like to run the command overnight for example. There's also an option to produce a machine-friendly tabular output, using &lt;code&gt;-T&lt;/code&gt;, &lt;code&gt;--tabular&lt;/code&gt; which would allow you to parse the output without much effort. I'd be tempted to just write the output to a file using the &lt;code&gt;-o&lt;/code&gt;, &lt;code&gt;--out-file&lt;/code&gt; option:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqkit stats -To stats.tsv *.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Example results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat stats.tsv &#xA;file    format  type    num_seqs    sum_len min_len avg_len max_len&#xA;PHB_Rep1_R1.fastq.gz    FASTQ   RNA 118571  11857100    100 100.0   100&#xA;PHB_Rep1_R2.fastq.gz    FASTQ   RNA 118571  11857100    100 100.0   100&#xA;PHB_Rep2_R1.fastq.gz    FASTQ   RNA 144826  14482600    100 100.0   100&#xA;PHB_Rep2_R2.fastq.gz    FASTQ   RNA 144826  14482600    100 100.0   100&#xA;PHB_Rep3_R1.fastq.gz    FASTQ   RNA 129786  12978600    100 100.0   100&#xA;PHB_Rep3_R2.fastq.gz    FASTQ   RNA 129786  12978600    100 100.0   100&#xA;PUH_Rep1_R1.fastq.gz    FASTQ   RNA 227392  22739200    100 100.0   100&#xA;PUH_Rep1_R2.fastq.gz    FASTQ   RNA 227392  22739200    100 100.0   100&#xA;PUH_Rep2_R1.fastq.gz    FASTQ   RNA 162373  16237300    100 100.0   100&#xA;PUH_Rep2_R2.fastq.gz    FASTQ   RNA 162373  16237300    100 100.0   100&#xA;PUH_Rep3_R1.fastq.gz    FASTQ   RNA 185442  18544200    100 100.0   100&#xA;PUH_Rep3_R2.fastq.gz    FASTQ   RNA 185442  18544200    100 100.0   100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And then just parse the output to get what you need, if or when you need it. For your specific requirements, you could just use AWK to skip the header line and print the first and fourth columns:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN {FS=OFS=&amp;quot;\t&amp;quot; } NR&amp;gt;1 { print $1, $4 }' stats.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;PHB_Rep1_R1.fastq.gz    118571&#xA;PHB_Rep1_R2.fastq.gz    118571&#xA;PHB_Rep2_R1.fastq.gz    144826&#xA;PHB_Rep2_R2.fastq.gz    144826&#xA;PHB_Rep3_R1.fastq.gz    129786&#xA;PHB_Rep3_R2.fastq.gz    129786&#xA;PUH_Rep1_R1.fastq.gz    227392&#xA;PUH_Rep1_R2.fastq.gz    227392&#xA;PUH_Rep2_R1.fastq.gz    162373&#xA;PUH_Rep2_R2.fastq.gz    162373&#xA;PUH_Rep3_R1.fastq.gz    185442&#xA;PUH_Rep3_R2.fastq.gz    185442&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-02-04T12:50:39.777" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20513" PostTypeId="2" ParentId="20509" CreationDate="2023-02-04T17:22:06.437" Score="-4" Body="&lt;p&gt;If you're looking for a one-liner,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in $(ls *.fastq.gz); do count=$(zcat $i | grep &amp;quot;^@&amp;quot; | wc -l); printf &amp;quot;%b\t%b&amp;quot; $i $count; done &amp;gt; counts.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16908" LastActivityDate="2023-02-04T17:22:06.437" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20516" PostTypeId="2" ParentId="20509" CreationDate="2023-02-05T19:56:27.903" Score="0" Body="&lt;p&gt;&lt;code&gt;zcat&lt;/code&gt;, like &lt;code&gt;cat&lt;/code&gt;, works with multiple file names as arguments, up to the argument limit (on Linux, that's usually at least a thousand), so counting multiple lines in multiple files still works, assuming the fastq files are formatted in the expected way with only four lines per record:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zcat filename_R1.fastq.gz filename_R2.fastq.gz ... | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If there's a consistent file pattern, that can be used with file name glob wildcards:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zcat *.fastq.gz | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you have a file containing a list of files, this can be processed as shell arguments as well:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zcat $(cat list_of_files.txt) | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For an unlimited number of files, something like GNU parallel can be used to feed file names from piped input. It's also a bit safer for file names containing weird characters in them:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat list_of_files.txt | parallel zcat {} | wc -l&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... but [after checking the question again] given that you want counts for individual files, the file name needs to be added as well:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat list_of_files.txt | parallel echo {} $(zcat {} | wc -l)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;read&lt;/code&gt; function provides an alternate way to do the same thing:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat list_of_files.txt | while read fileName; do echo &lt;span class=&quot;math-container&quot;&gt;${fileName} $&lt;/span&gt;(zcat ${fileName} | wc -l); done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Assuming you are using bash, dividing by 4 can be done via the arithmetic substitution operator, &lt;code&gt;$((X))&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat list_of_files.txt | parallel echo {} &lt;span class=&quot;math-container&quot;&gt;$(( $&lt;/span&gt;(zcat {} | wc -l) / 4 ))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or with &lt;code&gt;read&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat list_of_files.txt | while read fileName; do echo &lt;span class=&quot;math-container&quot;&gt;${fileName} $&lt;/span&gt;(( &lt;span class=&quot;math-container&quot;&gt;$(zcat $&lt;/span&gt;{fileName} | wc -l) / 4 )); done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And, finally, this can be output to a file using &lt;code&gt;&amp;gt;&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(cat list_of_files.txt | while read fileName; do echo &lt;span class=&quot;math-container&quot;&gt;${fileName} $&lt;/span&gt;(( &lt;span class=&quot;math-container&quot;&gt;$(zcat $&lt;/span&gt;{fileName} | wc -l) / 4 )); done) &amp;gt; fastq_record_counts.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-02-05T20:24:51.693" LastActivityDate="2023-02-05T20:24:51.693" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20517" PostTypeId="2" ParentId="20503" CreationDate="2023-02-05T22:50:11.790" Score="1" Body="&lt;p&gt;An easy way to do an enrichment analysis in scanpy is using &lt;a href=&quot;https://scanpy.readthedocs.io/en/stable/generated/scanpy.queries.enrich.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;scanpy.queries.enrich()&lt;/code&gt;&lt;/a&gt;. This relies on &lt;a href=&quot;https://pypi.org/project/gprofiler-official/#description&quot; rel=&quot;nofollow noreferrer&quot;&gt;gprofiler&lt;/a&gt;, which uses KEGG, GO and other ontologies&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2023-02-05T22:50:11.790" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20519" PostTypeId="2" ParentId="20497" CreationDate="2023-02-06T08:31:12.337" Score="1" Body="&lt;p&gt;As mentioned in the comments, an example of your inputs, outputs and their difference would be a good addition to your question. Moreover, the cell calling changed in CellRanger v3 compared to V2.&lt;/p&gt;&#xA;&lt;p&gt;There are several differences in how CellRanger and CellBender filter out empty droplets and call cells. Both methods are either published or available as preprints, so you can learn much more about them especially if you're interested in the technical aspects.&lt;/p&gt;&#xA;&lt;p&gt;CellRanger v3 uses a procedure largely based on the emptyDrops algorithm (&lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1662-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lun et al. 2019&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;Briefly, the algorithm calculates an &amp;quot;ambient profile&amp;quot; by summing all the read counts belonging to barcodes with a total read count below a threshold. Then, it compares each barcode to this ambient profile, testing for the similarity of the distribution of a barcode's counts to the ambient's.&#xA;CellRanger v3 does not implement exactly the same thresholds and will give you slightly different results than emptyDrops. CellRanger v2, on the other hand, used hard thresholds (based on knee point or the inflection point of the barcode ranks) which is less precise and biologically justifiable.&lt;/p&gt;&#xA;&lt;p&gt;CellBender (&lt;a href=&quot;https://www.biorxiv.org/content/10.1101/791699v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Fleming et al. 2022&lt;/a&gt;) uses a different procedure, in that it uses a deep learning model to learn suitable priors for ambient count distribution, and uses a different statistical model to call cells. The authors of CellBender compare their method to CellRanger and emptyDrops, so you can see for yourself how different the results are (on control datasets).&lt;/p&gt;&#xA;&lt;p&gt;Additionally, one of the differences is that CellBender will give you &amp;quot;corrected counts&amp;quot;, i.e. it will subtract the estimated ambient profile from the &amp;quot;real&amp;quot; profile. In other words, it will remove the background noise and give you counts that are (supposedly) only coming from good cells.&lt;/p&gt;&#xA;&lt;p&gt;Just keep in mind that whether or not &amp;quot;corrected counts&amp;quot; are good to use for downstream applications (HVG identification, clustering, etc.) is debatable (see &lt;a href=&quot;http://bioconductor.org/books/3.13/OSCA.multisample/ambient-problems.html#subtracting-ambient-counts&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for a discussion).&lt;/p&gt;&#xA;&lt;p&gt;A reasonable compromise is to flag genes with high ambient contaminations so that you know whether to trust the result of e.g. a differential expression test or a marker finding procedure.&lt;/p&gt;&#xA;" OwnerUserId="16618" LastActivityDate="2023-02-06T08:31:12.337" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20520" PostTypeId="1" AcceptedAnswerId="20521" CreationDate="2023-02-06T14:22:22.290" Score="1" ViewCount="277" Body="&lt;p&gt;I have scRNA-seq from PBMC analyzed by Seurat. I am seeing for most of genes a lot of cells have a zero expression like this gene&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NcSV0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NcSV0.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So 268 single cells have the expression level == 0&lt;/p&gt;&#xA;&lt;p&gt;And I am more wondering when seeing zero expression for a housekeeping gene like&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1FyC7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1FyC7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For which 72 single cells do not express this gene&lt;/p&gt;&#xA;&lt;p&gt;Does this come from biology or technical artifact?&lt;/p&gt;&#xA;&lt;p&gt;Thanks for sharing your ideas here&lt;/p&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2023-02-08T07:40:05.547" Title="Cells with zero expression of a given gene" Tags="|scrnaseq|seurat|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20521" PostTypeId="2" ParentId="20520" CreationDate="2023-02-06T17:31:37.240" Score="6" Body="&lt;p&gt;This has been debated for a few years now as the &amp;quot;dropout&amp;quot; problem, which is actually a mixture of different issues.&lt;/p&gt;&#xA;&lt;p&gt;On one hand, you are sequencing a relatively low input at a depth that does not reach saturation (i.e. you don't measure all there is to measure). This is partially because, on droplet data - such as 10X - you never use the same sequencing depth per single cell that you would use for a bulk measurement. You produce fewer sequencing reads.&lt;/p&gt;&#xA;&lt;p&gt;Additionally, the way a library is generated within a droplet with such a low input implies that transcripts will compete for a relatively limited amount of enzymes and reagents.&lt;/p&gt;&#xA;&lt;p&gt;This means that a highly expressed gene (i.e. one of which a single cell has produced several transcript molecules) will be more likely to be detected than one with fewer molecules (there are some biases but they do not explain alone the phenomenon). This would hardly ever happen in a bulk RNA sequencing dataset produced at an appropriate depth, and it is why we regard count data from scRNA-seq as &amp;quot;sparse&amp;quot; (with many 0's).&lt;/p&gt;&#xA;&lt;p&gt;On the other hand, there is a third issue to consider that is more &amp;quot;biological&amp;quot; than 'technical&amp;quot;: the way we used to think about transcription was rather mechanistic (here I hope others will excuse this word).&lt;/p&gt;&#xA;&lt;p&gt;What I mean by that is that you would think of transcription as an ordered set of steps whereby a transcription factor binds a promoter aided by other cofactors and chromatin complexes, recruit the RNA polymerase and let it do its job for as long/as many molecules as needed. However more biophysically motivated models generated from both &lt;em&gt;in vitro&lt;/em&gt; and &lt;em&gt;in vivo&lt;/em&gt; lines of evidence have painted a different picture, in which transcription is bursty, has restarts, early stops, and does not necessarily work as a putting a needle on a vinyl record. DNA loci may be transcribed at low levels due to phenomena that don't immediately depend on transcriptional regulation, such as cell cycle dynamics. This is a big source of intercellular variability. If you think about it, these are large molecules operating in solution and associating/dissociating from other molecules according to statistical thermodynamics. If the &amp;quot;vinyl&amp;quot; view of transcription was good enough to make sense of most bulk RNA-seq results, where you are indeed observing the average expression of genes in a cell population/tissue, once we get at the single cell level such a model does not necessarily hold.&lt;/p&gt;&#xA;&lt;p&gt;You know that GAPDH is a &amp;quot;housekeeping&amp;quot; gene because on a population average measured by RT-qPCR or Western Blot you never* see its levels fluctuate significantly, and they stay high compared to many other genes. At the single cell level, however, this need not be true at all times and for all cells. Hence a proportion of your zero-count genes will come from detection limits and low input (&amp;quot;technical&amp;quot; dropouts) and another proportion will come from the stochastic nature of transcription itself.&lt;/p&gt;&#xA;&lt;p&gt;Now the important aspect that the field has reflected upon is how to model these dropout, or if we even need to. The goal of these models is to understand variability and noise in the signal, allowing us to extrapolate useful biological data. Should we use a specific distribution of counts that accounts for an excess of 0's in the data (zero inflation)? Should we model dropouts entirely separately? Are distributions used in bulk RNA-seq analysis sufficient to model the data in an actionable and computationally efficient way? What about modelling burstiness and other biophysical properties?&lt;/p&gt;&#xA;&lt;p&gt;I believe it can be argued that there is no &amp;quot;actual/correct statistical model&amp;quot; because models are choices and results should be interpreted in light of those choices and their assumptions. If you are looking to cluster your cells and find &amp;quot;markers&amp;quot; to characterize the composition of a tissue, a simple model may be sufficiently equipped to let you draw some interesting conclusions. If instead you want to look more in depth at dynamical properties (transitions between cell states, &amp;quot;RNA velocity&amp;quot;, splicing, etc.) you may need to use more rigorous, biophysically motivated models that are also more complex in their theoretical justifications and their implementation.&lt;/p&gt;&#xA;&lt;p&gt;Here are some recommended readings if you want to dive deeper in the issue of dropouts and count modelling. The list is not exhaustive but I hope it helps.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;stochastic/biophysical modelling of counts: &lt;a href=&quot;https://www.nature.com/articles/s41467-022-34857-7&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gorin et al. 2022&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;multinomial modelling of counts: &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1874-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hafemeister and Satija 2019&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;dropout modelling and usage for feature selection: &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/30590489/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Andrews et al. 2019&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;negative binomial modelling of counts: &lt;a href=&quot;https://www.nature.com/articles/s41587-019-0379-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;Svensson et al. 2020&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;naught all zero's are the same: &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/33101615/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Silverman et al. 2020&lt;/a&gt; - this is about microbiome data but is still interesting IMHO&lt;/li&gt;&#xA;&lt;li&gt;on the importance of using rigorous models for RNA velocity: &lt;a href=&quot;https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1010492https://pubmed.ncbi.nlm.nih.gov/30590489/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Gorin et al. 2022&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;*more precise measurements in different tissues/cell lines have shown housekeeping genes to be variable in bulk as well, but that's another issue.&lt;/p&gt;&#xA;" OwnerUserId="16618" LastEditorUserId="16618" LastEditDate="2023-02-08T07:40:05.547" LastActivityDate="2023-02-08T07:40:05.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20522" PostTypeId="1" AcceptedAnswerId="20524" CreationDate="2023-02-06T20:04:33.010" Score="4" ViewCount="64" Body="&lt;p&gt;I have a dataset with following information. Column C1_1 is chromosome number, C1_2 is SNP position and C3 is the p-value. I want to pick the most significant association within a genomic region of 3000 bp:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;C1  C2  C3&#xA;L01_005000  L002g034    1.5e-12&#xA;L01_003000  L001g045    2.3e-6&#xA;L02_145000  L004g034    1.1e-2&#xA;L02_155000  L002g050    1.1e-4&#xA;L02_148000  L002g001    1.1e-3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to split first column into two as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;C1_1    C1_2    C2  C3&#xA;L01 005000  L002g034    1.5e-12&#xA;L01 003000  L001g045    2.3e-6&#xA;L02 145000  L004g034    1.1e-2&#xA;L02 155000  L002g050    1.1e-4&#xA;L02 148000  L002g001    1.1e-3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then I want to sort subsets such as &lt;code&gt;L01&lt;/code&gt; and &lt;code&gt;L02&lt;/code&gt; separately based on &lt;code&gt;C1_2&lt;/code&gt; column:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;C1_1    C1_2    C2  C3&#xA;L01 003000  L001g045    2.3e-6&#xA;L01 005000  L002g034    1.5e-12&#xA;L02 145000  L004g034    1.1e-2&#xA;L02 148000  L002g001    1.1e-3&#xA;L02 155000  L002g050    1.1e-4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then I want to pick one value from each subset such as for &lt;code&gt;L02&lt;/code&gt; – I want to pick the most significant value based on &lt;code&gt;C3&lt;/code&gt; column under the difference of 3000. For example, &lt;code&gt;L02-148000&lt;/code&gt; is chosen as shown below as its &lt;code&gt;C3&lt;/code&gt; value is lower than &lt;code&gt;L02-145000&lt;/code&gt;, and then the next value &lt;code&gt;L02-155000&lt;/code&gt; is chosen as the difference is more than 3000 than the previous value of &lt;code&gt;L02-148000&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;C1_1    C1_2    C2  C3&#xA;L01 005000  L002g034    1.5e-12&#xA;L02 148000  L002g001    1.1e-3&#xA;L02 155000  L002g050    1.1e-4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have a long dataset with 500,000 rows of values and the subset goes from &lt;code&gt;L01-L24&lt;/code&gt;. I think it will need a loop to process these many values to get the final values.&lt;/p&gt;&#xA;&lt;p&gt;This is just an example, I have shown one for L01 because the difference in L01 203000 and L01 205000 is less than 3000. I have shown two values for L02 because the difference between L02 145000 and L02 148000 is not more than 3000. Whereas the difference between L02 155000 and L02 148000 is more than 3000.&lt;/p&gt;&#xA;" OwnerUserId="14034" LastEditorUserId="73" LastEditDate="2023-02-07T21:40:06.483" LastActivityDate="2023-02-07T22:33:13.167" Title="question to pick values based on conditions using a loop" Tags="|r|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20523" PostTypeId="2" ParentId="20522" CreationDate="2023-02-06T21:26:57.537" Score="1" Body="&lt;h3&gt;EDIT: Ignore this answer, it does not address one of your requirements - that of the 3000-long bin. You're going to need some partitioning operations which are not straightforward.&lt;/h3&gt;&#xA;&lt;h4&gt;@jared_marmot has cracked it in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/20524/650&quot;&gt;their solution&lt;/a&gt;.&lt;/h4&gt;&#xA;&lt;p&gt;This should be easy to do with dplyr/tidyr. Use&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;## Code not tested, might need tweaking&#xA;library(tidyr)&#xA;library(dplyr)&#xA;df_split &amp;lt;- df %&amp;gt;% separate(C1, sep = &amp;quot;_&amp;quot;, into = c(&amp;quot;C1_1&amp;quot;,&amp;quot;C1_2&amp;quot;))&#xA;df_aggr &amp;lt;- df_split %&amp;gt;% select(C1_1, C3) %&amp;gt;% group_by(C1_1) %&amp;gt;% summarise(C3 = min(C3)) %&amp;gt;% inner_join(df_split)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There are two problems that make your 3000 window complicated:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The start and end are not fixed: it's not 1 to n in steps of 3000; that'd be a straightforward bin.&lt;/li&gt;&#xA;&lt;li&gt;We cannot even go with the smallest coordinate as the start, as you may have data points at 1500, 2500, 7200, 7600. If those data points exist, a set of 3000-long windows starting at 1500 would be 1500-4500, 4500-7500, etc. making your data go in 3 bins whereas your data points belong in 2 bins. You need windows with independent start coordinates. I've never encountered this requirement in my career even as a puzzle so I'm starting to think this might be an X-Y problem that needs to be addressed at a different level.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="650" LastEditorUserId="650" LastEditDate="2023-02-07T18:35:58.290" LastActivityDate="2023-02-07T18:35:58.290" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20524" PostTypeId="2" ParentId="20522" CreationDate="2023-02-07T11:35:21.330" Score="4" Body="&lt;p&gt;This question is deceptively challenging, but here is a potential solution:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(tidyverse)&#xA;&#xA;df &amp;lt;- read.table(text = &amp;quot;C1  C2  C3&#xA;L01_005000  L002g034    1.5e-12&#xA;L01_003000  L001g045    2.3e-6&#xA;L02_145000  L004g034    1.1e-2&#xA;L02_155000  L002g050    1.1e-4&#xA;L02_148000  L002g001    1.1e-3&amp;quot;,&#xA;header = TRUE)&#xA;&#xA;df&#xA;#&amp;gt;           C1       C2      C3&#xA;#&amp;gt; 1 L01_005000 L002g034 1.5e-12&#xA;#&amp;gt; 2 L01_003000 L001g045 2.3e-06&#xA;#&amp;gt; 3 L02_145000 L004g034 1.1e-02&#xA;#&amp;gt; 4 L02_155000 L002g050 1.1e-04&#xA;#&amp;gt; 5 L02_148000 L002g001 1.1e-03&#xA;&#xA;df %&amp;gt;%&#xA;  separate(C1, into = c(&amp;quot;C1_1&amp;quot;, &amp;quot;C1_2&amp;quot;)) %&amp;gt;%&#xA;  mutate(C1_2 = as.numeric(C1_2)) %&amp;gt;%&#xA;  arrange(C1_1, C1_2) %&amp;gt;%&#xA;  group_by(C1_1) %&amp;gt;%&#xA;  mutate(subgroup = cumsum(C1_2 &amp;gt; lag(C1_2, default = first(C1_2)) + 3000)) %&amp;gt;%&#xA;  group_by(C1_1, subgroup) %&amp;gt;%&#xA;  slice_min(C3) %&amp;gt;%&#xA;  ungroup() %&amp;gt;%&#xA;  select(-subgroup)&#xA;#&amp;gt; # A tibble: 3 × 4&#xA;#&amp;gt;   C1_1    C1_2 C2            C3&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;  &amp;lt;dbl&amp;gt; &amp;lt;chr&amp;gt;      &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 L01     5000 L002g034 1.5e-12&#xA;#&amp;gt; 2 L02   148000 L002g001 1.1e- 3&#xA;#&amp;gt; 3 L02   155000 L002g050 1.1e- 4&#xA;&#xA;## additional example data&#xA;df &amp;lt;- read.table(text = &amp;quot;C1  C2  C3&#xA;L01_005000  L002g034    1.5e-12&#xA;L01_003000  L001g045    2.3e-6&#xA;L02_145000  L004g034    1.1e-2&#xA;L02_155000  L002g050    1.1e-4&#xA;L02_158000  L004g034    1.1e-2&#xA;L02_159000  L002g050    1.1e-4&#xA;L03_165000  L002g001    1.1e-3&amp;quot;,&#xA;header = TRUE)&#xA;&#xA;df&#xA;#&amp;gt;           C1       C2      C3&#xA;#&amp;gt; 1 L01_005000 L002g034 1.5e-12&#xA;#&amp;gt; 2 L01_003000 L001g045 2.3e-06&#xA;#&amp;gt; 3 L02_145000 L004g034 1.1e-02&#xA;#&amp;gt; 4 L02_155000 L002g050 1.1e-04&#xA;#&amp;gt; 5 L02_158000 L004g034 1.1e-02&#xA;#&amp;gt; 6 L02_159000 L002g050 1.1e-04&#xA;#&amp;gt; 7 L03_165000 L002g001 1.1e-03&#xA;&#xA;df %&amp;gt;%&#xA;  separate(C1, into = c(&amp;quot;C1_1&amp;quot;, &amp;quot;C1_2&amp;quot;)) %&amp;gt;%&#xA;  mutate(C1_2 = as.numeric(C1_2)) %&amp;gt;%&#xA;  arrange(C1_1, C1_2) %&amp;gt;%&#xA;  group_by(C1_1) %&amp;gt;%&#xA;  mutate(subgroup = cumsum(C1_2 &amp;gt; lag(C1_2, default = first(C1_2)) + 3000)) %&amp;gt;%&#xA;  group_by(C1_1, subgroup) %&amp;gt;%&#xA;  slice_min(C3) %&amp;gt;%&#xA;  ungroup() %&amp;gt;%&#xA;  select(-subgroup)&#xA;#&amp;gt; # A tibble: 5 × 4&#xA;#&amp;gt;   C1_1    C1_2 C2            C3&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;  &amp;lt;dbl&amp;gt; &amp;lt;chr&amp;gt;      &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 L01     5000 L002g034 1.5e-12&#xA;#&amp;gt; 2 L02   145000 L004g034 1.1e- 2&#xA;#&amp;gt; 3 L02   155000 L002g050 1.1e- 4&#xA;#&amp;gt; 4 L02   159000 L002g050 1.1e- 4&#xA;#&amp;gt; 5 L03   165000 L002g001 1.1e- 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;sup&gt;Created on 2023-02-07 with &lt;a href=&quot;https://reprex.tidyverse.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;reprex v2.0.2&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Edit&lt;/h2&gt;&#xA;&lt;p&gt;If you have two SNPs with the same C3 value (i.e. the top two rows in the example dataset below) you can select one of them using &lt;code&gt;slice_min(C3, with_ties = FALSE)&lt;/code&gt;, e.g. the code below will select the 'first' entry when two C3 values are the same within the same group/subgroup:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(tidyverse)&#xA;&#xA;df &amp;lt;- read.table(text = &amp;quot;C1  C2  C3&#xA;L01_005000  L002g034    1.5e-12&#xA;L01_003000  L001g045    1.5e-12&#xA;L02_145000  L004g034    1.1e-2&#xA;L02_155000  L002g050    1.1e-4&#xA;L02_158000  L004g034    1.1e-2&#xA;L02_159000  L002g050    1.1e-4&#xA;L03_165000  L002g001    1.1e-3&amp;quot;,&#xA;header = TRUE)&#xA;&#xA;df&#xA;#&amp;gt;           C1       C2      C3&#xA;#&amp;gt; 1 L01_005000 L002g034 1.5e-12&#xA;#&amp;gt; 2 L01_003000 L001g045 1.5e-12&#xA;#&amp;gt; 3 L02_145000 L004g034 1.1e-02&#xA;#&amp;gt; 4 L02_155000 L002g050 1.1e-04&#xA;#&amp;gt; 5 L02_158000 L004g034 1.1e-02&#xA;#&amp;gt; 6 L02_159000 L002g050 1.1e-04&#xA;#&amp;gt; 7 L03_165000 L002g001 1.1e-03&#xA;&#xA;df %&amp;gt;%&#xA;  separate(C1, into = c(&amp;quot;C1_1&amp;quot;, &amp;quot;C1_2&amp;quot;)) %&amp;gt;%&#xA;  mutate(C1_2 = as.numeric(C1_2)) %&amp;gt;%&#xA;  arrange(C1_1, C1_2) %&amp;gt;%&#xA;  group_by(C1_1) %&amp;gt;%&#xA;  mutate(subgroup = cumsum(C1_2 &amp;gt; lag(C1_2, default = first(C1_2)) + 3000)) %&amp;gt;%&#xA;  group_by(C1_1, subgroup) %&amp;gt;%&#xA;  slice_min(C3, with_ties = FALSE) %&amp;gt;%&#xA;  ungroup() %&amp;gt;%&#xA;  select(-subgroup)&#xA;#&amp;gt; # A tibble: 4 × 4&#xA;#&amp;gt;   C1_1    C1_2 C2            C3&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;  &amp;lt;dbl&amp;gt; &amp;lt;chr&amp;gt;      &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 L01     3000 L001g045 1.5e-12&#xA;#&amp;gt; 2 L02   145000 L004g034 1.1e- 2&#xA;#&amp;gt; 3 L02   155000 L002g050 1.1e- 4&#xA;#&amp;gt; 4 L03   165000 L002g001 1.1e- 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;sup&gt;Created on 2023-02-08 with &lt;a href=&quot;https://reprex.tidyverse.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;reprex v2.0.2&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="9098" LastEditorUserId="9098" LastEditDate="2023-02-07T22:33:13.167" LastActivityDate="2023-02-07T22:33:13.167" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20525" PostTypeId="1" AcceptedAnswerId="20526" CreationDate="2023-02-07T12:53:00.207" Score="0" ViewCount="329" Body="&lt;p&gt;I have a big &lt;code&gt;fasta.dataset&lt;/code&gt; file containing half a million proteins (1.0 GB). I have four lines for each protein code:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;line 1:the protein code&lt;/li&gt;&#xA;&lt;li&gt;line 2: protein length in amino acids&lt;/li&gt;&#xA;&lt;li&gt;line 3: amino acid sequence&lt;/li&gt;&#xA;&lt;li&gt;line 4: secondary structure&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Now, I am trying to open and read it in python (Biopython), and it does not work:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;filename = 'pdb.fasta_qual.dataset'&#xA;sequences = SeqIO.parse ( filename,'fasta')&#xA;for record in sequences:&#xA;    example = record&#xA;    break&#xA;print(example)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I read it in python and loop through the file to look at line 3  for each protein to count the sequence length and distribution?&lt;/p&gt;&#xA;&lt;p&gt;here is the first 5 lines of my file: which my file contains 500,000 proteins for each one has a 4 lines (name ,len of protein in amino acid,the seq represents by letters which what I would to calculate,the secondary structure)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;4LGTD&#xA;247&#xA;M       S       E       K       L       Q       K       V       L       A       R       A       G       H       G      T&#xA;.       .       E       E       H       H       H       H       H       H       H       T       T       S       S       .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to open and read the file and loop through line 3 for each protein to calculate the length of the sequences and plot a histogram ! to check the distribution.&#xA;,,,&lt;/p&gt;&#xA;&lt;p&gt;The output i am expecting is :&lt;/p&gt;&#xA;&lt;p&gt;The len for the first seq is  =&lt;/p&gt;&#xA;&lt;p&gt;The len for the second seq is  =&lt;/p&gt;&#xA;&lt;p&gt;Until&lt;/p&gt;&#xA;&lt;p&gt;the len of the last sequence  which is number (500.000)=&lt;/p&gt;&#xA;&lt;p&gt;And then i can plot a histogram for the len of the sequences&#xA;,,,&lt;/p&gt;&#xA;&lt;p&gt;NOTE: I have opened and read the file's info by Linux, but I could not by python.&lt;/p&gt;&#xA;" OwnerUserId="16923" LastEditorUserId="3848" LastEditDate="2023-02-19T20:50:41.570" LastActivityDate="2023-02-19T20:50:41.570" Title="open and read Fasta file (raw data)" Tags="|fasta|proteins|protein-structure|biopython|" AnswerCount="2" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="20526" PostTypeId="2" ParentId="20525" CreationDate="2023-02-07T14:34:12.377" Score="0" Body="&lt;p&gt;I better try and explain the issues a bit better. A dataset of 0.5 million at 1 GB will struggle to load into &lt;code&gt;Biopython&lt;/code&gt;. &lt;code&gt;Biopython&lt;/code&gt; isn't great at handling large numbers of sequences - for a regular desktop/laptop - and what will happen is the parser will freeze or else take an enormous amount of time (many hours). I call this a &amp;quot;RAM bottleneck&amp;quot;. A generator is an easy workaround.&lt;/p&gt;&#xA;&lt;p&gt;The other issue with the code is there are lots of bugs combining format and code. There is a clarity issue as well it looks like you want to calculate the secondary structure - rather than import the secondary structure alongside the alignment. I could be wrong on this though, certainly secondary structure is present in the &lt;em&gt;ad hoc&lt;/em&gt; alignment. There is an alignment format that handles both secondary structure and sequence data. You might think about restructuring the question and your approach into multiple questions.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Reformatting the &lt;em&gt;ad hoc&lt;/em&gt; alignment to permit import into &lt;code&gt;Biopython&lt;/code&gt; on a &lt;em&gt;pilot data set&lt;/em&gt; (not 0.5 million sequences). This is close to @terdon's answer.&lt;/li&gt;&#xA;&lt;li&gt;Code that will process very large files in &lt;code&gt;Biopython&lt;/code&gt;. This is the answer below.&lt;/li&gt;&#xA;&lt;li&gt;An alignment format via &lt;code&gt;Biopython&lt;/code&gt; (probably output - could be input) that integrates linear sequence with secondary structure.&lt;/li&gt;&#xA;&lt;li&gt;Plotting a histogram for 0.5 million sequences.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Points (Questions) 1. to 3. can be leveraged by Biopython. Point (Question) 4 can be done via Python too - graphing/histograms are pretty good. There seems an underlying issue of project management for coding based work.&lt;/p&gt;&#xA;&lt;p&gt;If this is a RAM bottleneck then solution is below.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The easiest approach is to use a generator (please see 'format').&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;example = (record for record in SeqIO.parse(filename,'fasta'))&#xA;print (next(example)) # example.id ?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;A generator will deal with the sequences one by one instead of trying to load the whole thing to memory. It may not be suitable for certain complex calculations, but it is by far the easiest approach to the problem (if the problem is the OP is struggling to load a million sequences into memory).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; I always used &lt;code&gt;yield&lt;/code&gt; commands but @Steve introduced the &lt;code&gt;()&lt;/code&gt; format and I definitely prefer it. Also note that manipulating the output of a generator can be different to a list, tuple or dictionary (calculation depending).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Format&lt;/strong&gt;&#xA;The format supplied isn't fasta so the &lt;code&gt;SeqIO&lt;/code&gt; will not work in &lt;code&gt;fasta&lt;/code&gt; mode and should throw an error/exception. The format you need to leverage &lt;code&gt;Biopython&lt;/code&gt; is very different. I would recommend a standard 'open' command and passing this to a custom dictionary. Also the output of &lt;code&gt;print(next(record))&lt;/code&gt; should produce an object symbol, not the actual data e.g. &lt;code&gt;record.id&lt;/code&gt; would be needed to see data: it does however demonstrate the generator is working.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-02-08T08:58:51.587" LastActivityDate="2023-02-08T08:58:51.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20527" PostTypeId="2" ParentId="20525" CreationDate="2023-02-07T17:00:58.623" Score="4" Body="&lt;p&gt;It sounds like all you really want is to print every 2nd line of a text file. If so, you don't need Python, let alone BioPython, you can do it with basic *nix tools:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ awk 'NR%4==2' pdb.fasta_qual.dataset &#xA;247&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;NR&lt;/code&gt; is the current line number, and &lt;code&gt;%&lt;/code&gt; is the &lt;a href=&quot;https://en.wikipedia.org/wiki/Modulo&quot; rel=&quot;nofollow noreferrer&quot;&gt;modulo&lt;/a&gt; operator. Therefore, &lt;code&gt;NR % 4&lt;/code&gt; will equal &lt;code&gt;2&lt;/code&gt; on the second line of each group of 4 lines in the file. In &lt;code&gt;awk&lt;/code&gt;, the default action when something evaluates to true is to print the current line, so this will print out all length lines from your input file (assuming you don't have blank lines anywhere, which we don't know since you only showed the first entry).&lt;/p&gt;&#xA;&lt;p&gt;If you want to add some sort of message to be printed, you could do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ awk 'NR%4==2{ print &amp;quot;The length of protein&amp;quot;,++c,&amp;quot;is:&amp;quot;,$0}' pdb.fasta_qual.dataset &#xA;The length of protein 1 is: 247&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you really must do this in Python, you could do something like this:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;import sys&#xA;&#xA;#my_file= sys.argv[1]&#xA;line_number = 0&#xA;with open(sys.argv[1]) as f:&#xA;    for line in f:&#xA;        line_number += 1&#xA;        if (line_number % 4 == 2):&#xA;            print(&amp;quot;The length of protein %d is: %d&amp;quot; % (line_number,int(line.strip())))&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And then you would run it like this (assuming you named the script &lt;code&gt;foo.py&lt;/code&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ python3 foo.py pdb.fasta_qual.dataset &#xA;The length of protein 2 is: 247&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Out of curiosity, I created a test file by repeating your example 10000000 times, which left me with a 2.4G file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ perl -e 'open(A,&amp;quot;pdb.fasta_qual.dataset&amp;quot;); @lines=&amp;lt;A&amp;gt;; print &amp;quot;@lines&amp;quot; x 10000000' &amp;gt; testFile&#xA;$ ls -lh testFile &#xA;-rw-r--r-- 1 terdon terdon 2.4G Feb  7 17:06 testFile&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then tested both approaches for speed. Unsurprisingly, &lt;code&gt;awk&lt;/code&gt; was  much faster:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;$ time awk 'NR%4==2' testFile &amp;gt; /dev/null &#xA;&#xA;real    0m6.447s&#xA;user    0m5.917s&#xA;sys     0m0.492s&#xA;&#xA;$ time foo.py testFile &amp;gt; /dev/null &#xA;&#xA;real    0m10.856s&#xA;user    0m10.111s&#xA;sys     0m0.523s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2023-02-07T17:08:16.537" LastActivityDate="2023-02-07T17:08:16.537" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20528" PostTypeId="1" CreationDate="2023-02-07T17:02:57.037" Score="1" ViewCount="89" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://github.com/malonge/RagTag/issues/152&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm trying to correct a long-read assembly with a short-read scaffold; I'm hoping to fill in the short gaps in the scaffold with the matching long-read sections. The RagTag &amp;quot;patch&amp;quot; function, if I understand correctly, is supposed to do exactly this. I use this command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ragtag.py patch --fill-only liberibacter_central_ragtag.fasta liberibacter_central_assembly.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But when I attempt to run it I get the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Tue Feb  7 09:55:17 2023 --- VERSION: RagTag v2.1.0&#xA;Tue Feb  7 09:55:17 2023 --- WARNING: This is a beta version of `ragtag patch`&#xA;Tue Feb  7 09:55:17 2023 --- CMD: ragtag.py patch --fill-only liberibacter_central_assembly.fasta liberibacter_central_ragtag.fasta&#xA;Tue Feb  7 09:55:17 2023 --- WARNING: Without '-u' invoked, some component/object AGP pairs might share the same ID. Some external programs/databases don't like this. To ensure valid AGP format, use '-u'.&#xA;Tue Feb  7 09:55:17 2023 --- INFO: Running: ragtag_splitasm.py -o /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.ctg.agp /xdisk/baltrus/moshesteyn/liberibacter_central_assembly.fasta &amp;gt; /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.ctg.fasta 2&amp;gt; /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.err&#xA;Tue Feb  7 09:55:17 2023 --- INFO: Finished running : ragtag_splitasm.py -o /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.ctg.agp /xdisk/baltrus/moshesteyn/liberibacter_central_assembly.fasta &amp;gt; /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.ctg.fasta 2&amp;gt; /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.err&#xA;Tue Feb  7 09:55:17 2023 --- INFO: Running: ragtag_rename.py /xdisk/baltrus/moshesteyn/liberibacter_central_ragtag.fasta -p qseq -o /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.rename.agp &amp;gt; /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.rename.fasta 2&amp;gt; /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.err&#xA;Tue Feb  7 09:55:17 2023 --- INFO: Finished running : ragtag_rename.py /xdisk/baltrus/moshesteyn/liberibacter_central_ragtag.fasta -p qseq -o /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.rename.agp &amp;gt; /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.rename.fasta 2&amp;gt; /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.err&#xA;Tue Feb  7 09:55:17 2023 --- INFO: Writing: /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.comps.fasta&#xA;Tue Feb  7 09:55:17 2023 --- INFO: Mapping the query genome to the target genome&#xA;Tue Feb  7 09:55:17 2023 --- INFO: Running: nucmer --maxmatch -l 100 -c 500 -p /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.asm /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.ctg.fasta /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.rename.fasta 2&amp;gt; /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.asm.delta.log&#xA;Tue Feb  7 09:55:19 2023 --- INFO: Finished running : nucmer --maxmatch -l 100 -c 500 -p /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.asm /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.ctg.fasta /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.rename.fasta &amp;gt; /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.asm.delta.log&#xA;Tue Feb  7 09:55:19 2023 --- INFO: Running: ragtag_delta2paf.py /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.asm.delta &amp;gt; /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.asm.paf 2&amp;gt; /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.err&#xA;Tue Feb  7 09:55:19 2023 --- INFO: Finished running : ragtag_delta2paf.py /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.asm.delta &amp;gt; /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.asm.paf 2&amp;gt; /xdisk/baltrus/moshesteyn/ragtag_output/ragtag.patch.err&#xA;Tue Feb  7 09:55:19 2023 --- INFO: Reading whole genome alignments&#xA;Tue Feb  7 09:55:19 2023 --- INFO: Filtering and merging alignments&#xA;Tue Feb  7 09:55:19 2023 --- INFO: Building a scaffold graph from the contig AGP file&#xA;Tue Feb  7 09:55:19 2023 --- INFO: Building a scaffold graph from the target/query mappings&#xA;Tue Feb  7 09:55:19 2023 --- INFO: Computing a matching solution to the scaffold graph&#xA;Traceback (most recent call last):&#xA;File &amp;quot;/home/u30/moshesteyn/.conda/envs/py382/bin/ragtag_patch.py&amp;quot;, line 518, in &amp;lt;module&amp;gt;&#xA;    main()&#xA;File &amp;quot;/home/u30/moshesteyn/.conda/envs/py382/bin/ragtag_patch.py&amp;quot;, line 497, in main&#xA;    match_psg = aln_psg.max_weight_matching()&#xA;File &amp;quot;/home/u30/moshesteyn/.conda/envs/py382/lib/python3.8/site-packages/ragtag_utilities/ScaffoldGraph.py&amp;quot;, line 824, in max_weight_matching&#xA;    for cc in nx.connected_components(G=cover_graph):&#xA;File &amp;quot;/home/u30/moshesteyn/.conda/envs/py382/lib/python3.8/site-packages/networkx/classes/backends.py&amp;quot;, line 134, in wrapper&#xA;    graph = args[0]&#xA;IndexError: tuple index out of range&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does anyone know what might be causing this error? No one else seems to have run into it on the github. I've run it with both files individually against a known reference, as well as reversing the order of the files and removing the &amp;quot;fill-only&amp;quot; option, and I have the same problem. I've looked at several of the output files, and they seem normal, at least in that they aren't empty or filled with zeroes.&lt;/p&gt;&#xA;" OwnerUserId="15345" LastEditorUserId="73" LastEditDate="2023-02-08T19:17:00.730" LastActivityDate="2023-02-08T19:17:00.730" Title="RagTag patch error--&quot;Tuple index out of range&quot;" Tags="|assembly|scaffold|ragtag|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="20529" PostTypeId="2" ParentId="20495" CreationDate="2023-02-07T23:10:21.947" Score="1" Body="&lt;p&gt;If you can provide a &lt;a href=&quot;https://stackoverflow.com/help/minimal-reproducible-example&quot;&gt;minimal reproducible example&lt;/a&gt; with a 'cut-down' version of your data it would be a lot easier to answer your question. I have an example from a similar question on stackoverflow (&lt;a href=&quot;https://stackoverflow.com/a/73696447/12957340&quot;&gt;https://stackoverflow.com/a/73696447/12957340&lt;/a&gt;) that illustrates one approach to your problem:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# Load libraries&#xA;library(tidyverse)&#xA;&#xA;# Load example data&#xA;raw_data &amp;lt;- structure(list(Type = c(&amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &#xA;                                    &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &#xA;                                    &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &#xA;                                    &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EY&amp;quot;, &amp;quot;EO&amp;quot;, &#xA;                                    &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &#xA;                                    &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;EO&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &#xA;                                    &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &#xA;                                    &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &#xA;                                    &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;, &amp;quot;Fe&amp;quot;), &#xA;                           Maturity = structure(c(2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, &#xA;                                                  2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, &#xA;                                                  2L, 2L, 2L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, &#xA;                                                  1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, &#xA;                                                  1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, 1L, &#xA;                                                  1L, 1L, 1L, 1L, 1L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, &#xA;                                                  2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L, 2L), .Label = c(&amp;quot;Before&amp;quot;, &#xA;                                                                                                      &amp;quot;After&amp;quot;), class = &amp;quot;factor&amp;quot;), Mn = c(49.65414679, 30.42267647, &#xA;                                                                                                                                          53.57468125, 58.38531743, 11.07088615, 45.10644112, 41.33779076, &#xA;                                                                                                                                          18.77426257, 60.36149117, 67.0260032, 38.13116392, 69.58713059, &#xA;                                                                                                                                          66.80825471, 30.34496525, 56.90875291, 72.37427455, 22.06855022, &#xA;                                                                                                                                          39.26030375, 35.31067394, 24.95698024, 24.88586385, 34.65821779, &#xA;                                                                                                                                          56.2750941, 19.56655377, 45.04117517, 44.04604616, 30.4130276, &#xA;                                                                                                                                          15.76824315, 21.50636001, 27.25777212, 17.08214603, 14.17493713, &#xA;                                                                                                                                          34.24593956, 12.39548126, 23.59787998, 38.42021152, 32.54843776, &#xA;                                                                                                                                          18.47119505, 26.05687922, 32.97977895, 16.91703485, 30.0626039, &#xA;                                                                                                                                          26.72724955, 17.69433512, 22.34314521, 16.73815161, 13.08749188, &#xA;                                                                                                                                          19.60771131, 24.08865106, 34.57472809, 14.94650052, 32.60745327, &#xA;                                                                                                                                          29.86439635, 20.71564742, 22.32568148, 18.37220522, 24.22830773, &#xA;                                                                                                                                          20.91660251, 20.96486868, 34.25266591, 35.57200211, 14.5736099, &#xA;                                                                                                                                          25.67187037, 27.7352539, 42.919751, 27.1949762, 18.94238981, &#xA;                                                                                                                                          25.25867502, 31.66215867, 18.1734287, 24.41910112, 12.07438938, &#xA;                                                                                                                                          12.85034127, 19.75012923, 35.96030792, 30.07986478, 38.91460672, &#xA;                                                                                                                                          41.18492241, 41.68085289, 60.93940724, 49.77930187, 17.56806894, &#xA;                                                                                                                                          37.13306953, 47.06926881, 85.62093648, 24.31500546, 22.049633, &#xA;                                                                                                                                          40.42923285, 87.53339143, 37.08699364, 38.29525586, 28.00272281, &#xA;                                                                                                                                          15.58855204, 37.14164977)), row.names = c(NA, -94L), class = c(&amp;quot;tbl_df&amp;quot;, &#xA;                                                                                                                                                                                                         &amp;quot;tbl&amp;quot;, &amp;quot;data.frame&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Functions:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;summary_df &amp;lt;- raw_data %&amp;gt;%&#xA;  mutate(Maturity = ifelse(Maturity == &amp;quot;Before&amp;quot;, &amp;quot;a&amp;quot;, &amp;quot;x&amp;quot;)) %&amp;gt;%&#xA;  group_by(Type, Maturity) %&amp;gt;%&#xA;  summarise(max_val = max(density(Mn)[[1]]) + 4) %&amp;gt;%&#xA;  ungroup()&#xA;&#xA;median.stat &amp;lt;- function(x){&#xA;  out &amp;lt;- quantile(x, probs = c(0.5))&#xA;  names(out) &amp;lt;- c(&amp;quot;ymed&amp;quot;)&#xA;  return(out) &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Similar 'problematic' violin plot:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;ggplot(raw_data, aes(x = Type, y = Mn, fill = Maturity)) +&#xA;  geom_violin(trim = FALSE, position=&amp;quot;dodge&amp;quot;) + &#xA;  geom_boxplot(width=.1, outlier.shape=NA, position=position_dodge(.9)) +  &#xA;  scale_y_continuous(breaks = seq(0, 100, 25)) +&#xA;  theme_classic() + &#xA;  theme(axis.title.x = element_blank(),&#xA;        axis.ticks.x = element_blank()) + &#xA;  labs(y=&amp;quot;Percent Maturity&amp;quot;, x= &amp;quot;&amp;quot;) +&#xA;  geom_text(data = summary_df,&#xA;            aes(label = Maturity,&#xA;                y = max_val,&#xA;                x = Type,&#xA;                group = Maturity),&#xA;            position = position_dodge(width = 0.9),&#xA;            check_overlap = TRUE,&#xA;            show.legend = FALSE,&#xA;            inherit.aes = FALSE) +&#xA;  stat_summary(fun.y = median.stat, geom='point', size = 10, colour = &amp;quot;blue&amp;quot;,&#xA;               position = position_dodge(0.9))&#xA;#&amp;gt; Warning: The `fun.y` argument of `stat_summary()` is deprecated as of ggplot2 3.3.0.&#xA;#&amp;gt; ℹ Please use the `fun` argument instead.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/HUpkO.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Remove dots:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;ggplot(raw_data, aes(x = Type, y = Mn, fill = Maturity)) +&#xA;  geom_violin(trim = FALSE, position=&amp;quot;dodge&amp;quot;) + &#xA;  geom_boxplot(width=.1, outlier.shape=NA, position=position_dodge(.9)) +  &#xA;  scale_y_continuous(breaks = seq(0, 100, 25)) +&#xA;  theme_classic() + &#xA;  theme(axis.title.x = element_blank(),&#xA;        axis.ticks.x = element_blank()) + &#xA;  labs(y=&amp;quot;Percent Maturity&amp;quot;, x= &amp;quot;&amp;quot;) +&#xA;  geom_text(data = summary_df,&#xA;            aes(label = Maturity,&#xA;                y = max_val,&#xA;                x = Type,&#xA;                group = Maturity),&#xA;            position = position_dodge(width = 0.9),&#xA;            check_overlap = TRUE,&#xA;            show.legend = FALSE,&#xA;            inherit.aes = FALSE) +&#xA;  stat_summary(fun.y = median.stat, geom='point', size = 10, colour = &amp;quot;blue&amp;quot;,&#xA;               position = position_dodge(0.9), key_glyph = &amp;quot;blank&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/UoS5n.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Remove dots, but keep 'colour' legend to specify what the dots represent:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;ggplot(raw_data, aes(x = Type, y = Mn, fill = Maturity)) +&#xA;  geom_violin(trim = FALSE, position=&amp;quot;dodge&amp;quot;) + &#xA;  geom_boxplot(width=.1, outlier.shape=NA,  position=position_dodge(.9), key_glyph = &amp;quot;boxplot&amp;quot;) +  &#xA;  scale_y_continuous(breaks = seq(0, 100, 25)) +&#xA;  theme_classic() + &#xA;  theme(axis.title.x = element_blank(),&#xA;        axis.ticks.x = element_blank()) + &#xA;  labs(y=&amp;quot;Percent Maturity&amp;quot;, x= &amp;quot;&amp;quot;) +&#xA;  geom_text(data = summary_df,&#xA;            aes(label = Maturity,&#xA;                y = max_val,&#xA;                x = Type,&#xA;                group = Maturity),&#xA;            position = position_dodge(width = 0.9),&#xA;            check_overlap = TRUE,&#xA;            show.legend = FALSE,&#xA;            inherit.aes = FALSE) +&#xA;  stat_summary(aes(colour = &amp;quot;Median&amp;quot;), fun.y = median.stat, &#xA;               geom='point', size = 10,&#xA;               position = position_dodge(0.9)) +&#xA;  scale_color_manual(values = &amp;quot;blue&amp;quot;, name = &amp;quot;Statistics&amp;quot;) +&#xA;  guides(fill = guide_legend(override.aes = list(shape = NA)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.stack.imgur.com/TCa7O.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;sup&gt;Created on 2023-02-08 with &lt;a href=&quot;https://reprex.tidyverse.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;reprex v2.0.2&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;p&gt;Hopefully this approach can be adapted to your use-case.&lt;/p&gt;&#xA;" OwnerUserId="9098" LastEditorUserId="9098" LastEditDate="2023-02-08T00:53:29.307" LastActivityDate="2023-02-08T00:53:29.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20531" PostTypeId="1" AcceptedAnswerId="20532" CreationDate="2023-02-08T18:08:25.307" Score="2" ViewCount="87" Body="&lt;p&gt;I'm trying to reproduce a complex manual search on NCBI using BioPython, but obviously something is not right. My command line is not working, but I'd especially like a way to have an accurate search where I can select only the genomes related to a published paper. Is there an easy way to do that?&lt;/p&gt;&#xA;&lt;p&gt;I want to limit my results to only those genomes that i) have an NCBI BioProject entry and ii) that BioProject entry contains a &lt;code&gt;Publications&lt;/code&gt; field.&lt;/p&gt;&#xA;&lt;p&gt;For example, let's say I want all the genomes of Microbacterium, the latest version, from RefSeq and not partial (this is what I tried to write but something is not working) and I would like to add one more filter. Take, among the remaining/selected genomes, only the ones related to published papers.&lt;/p&gt;&#xA;&lt;p&gt;For this strain, we have a related paper:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/bioproject/PRJNA669461/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/bioproject/PRJNA669461/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For this one, we do not:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/bioproject/PRJNA608230/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/bioproject/PRJNA608230/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and I really would like to not have to check 100 genomes manually, but I don't know how to manage it.&lt;/p&gt;&#xA;&lt;p&gt;I tried:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;query = &amp;quot;Microbacterium[Organism] AND latest refseq[filter] AND (all[filter] NOT&#xA;partial[filter])&amp;quot;&#xA;handle = Entrez.esearch(term=query, db=&amp;quot;Assembly&amp;quot;, retmax=600)&#xA;ids = Entrez.read(handle)[&amp;quot;IdList&amp;quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And after this, I'm just using a &lt;code&gt;for&lt;/code&gt; loop to get genomes, but my query is not good and it is not handling the problem of the publication.&lt;/p&gt;&#xA;&lt;p&gt;Thanks for any help!&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Edit : as I need to search in the db Assembly unfortunately the answer with the flag author or date is not working&lt;/p&gt;&#xA;" OwnerUserId="16929" LastEditorUserId="73" LastEditDate="2023-02-09T18:12:45.283" LastActivityDate="2023-02-09T18:12:45.283" Title="how to do manual search on NCBI with biopython and add link to publication" Tags="|python|biopython|ncbi|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20532" PostTypeId="2" ParentId="20531" CreationDate="2023-02-08T18:55:35.073" Score="0" Body="&lt;p&gt;Entrez has the tags [journal], [volume], [page number], [title], [year] and [author] to identify all sequences from a given publication. You will find a full list of tags &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/advanced&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; from the drop down menu.&lt;/p&gt;&#xA;&lt;p&gt;Thus,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;query = &amp;quot;Microbacterium[Organism] AND Nature[Journal] AND 528[volume] AND Bai[author] AND 364[page number]&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will identify all &lt;em&gt;Microbacterium&lt;/em&gt; genome sequences associated with Bai &lt;em&gt;et al&lt;/em&gt; (2015) Nature 528:364- and is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/?term=Microbacterium%5BOrganism%5D%20AND%20NATURE%5BJournal%5D%20AND%20nature%5BJournal%5D%20AND%20528%5Bvolume%5D%20AND%20Bai%5Bauthor%5D%20AND%20364%5Bpage%20number%5D&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For practical purposes I routinely search using the organism and author tags for their sequence output.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The example given (post-edit) was&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Kim KK et al., &amp;quot;Microbacterium aquimaris sp. nov., isolated from seawater.&amp;quot;, Int J Syst Evol Microbiol, 2008 Jul;58(Pt 7):1616-20&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;query = &amp;quot;Microbacterium[Organism] AND 58[volume] AND Kim[author] AND 1616[page number] AND 2008[year]&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I removed the [journal] tag because the abbreviation it uses is a bit unusual and replaced it with [year]. Still works &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/?term=Microbacterium%5BOrganism%5D%20AND%2058%5Bvolume%5D%20AND%20Kim%5Bauthor%5D%20AND%201616%5Bpage%20number%5D%20AND%202008%5Byear%5D&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;From the comments ... if only published sequences were required then using the &lt;code&gt;range&lt;/code&gt; function and &lt;code&gt;[publication date]&lt;/code&gt; is one way in ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;query = &amp;quot;Microbacterium[Organism] AND 2000/1/1:2020/1/1[publication date]&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The best strategy is to use the &lt;code&gt;[author]&lt;/code&gt; because data production tends to be dominated by a small number of groups.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Comments of Comments&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Why isn't searching working. The following search works&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;query = &amp;quot;Microbacterium[Organism] AND (all[filter] NOT partial[filter])&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The term it doesn't like is &lt;code&gt;latest refseq[filter]&lt;/code&gt; if that is removed it will work. &lt;code&gt;refseq&lt;/code&gt; works, but it doesn't like &lt;code&gt;latest&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-02-08T21:23:55.700" LastActivityDate="2023-02-08T21:23:55.700" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="20533" PostTypeId="1" CreationDate="2023-02-09T09:01:47.060" Score="2" ViewCount="382" Body="&lt;p&gt;i have two protein pdb files and want to calculate the backchain RMSD between them. As far as I know the GUI of Pymol can use align to calculate the RMSD (but I don't know if it is the backchain RMSD).I want to use the python code to use pymol to calculate the backchain RMSD. I know how to align them but don't know how to calculate the RMSD.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pymol2&#xA;with pymol2.PYMOL() as pymol:&#xA;    pymol.cmd.load('foo.pdb', 'foo')&#xA;    pymol.cmd.load('bar.pdb', 'bar')&#xA;    pymol.cmd.align('foo', 'bar')&#xA;    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16933" LastEditorUserId="73" LastEditDate="2023-02-09T18:10:02.500" LastActivityDate="2023-02-10T18:29:12.463" Title="calculate the backchain RMSD between two pdb files by pymol in Python" Tags="|python|pymol|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20534" PostTypeId="1" CreationDate="2023-02-09T09:08:19.640" Score="2" ViewCount="177" Body="&lt;p&gt;I have converted a VCF file to the arlequin file format. It consists of 20 samples that &lt;em&gt;should&lt;/em&gt; belong to three populations, along with a few thousands SNPs. However, when I load the arlequin file into the software, it shows I only have 1 population. Any way to fix this?&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="16934" LastEditorUserId="73" LastEditDate="2023-02-09T18:05:21.513" LastActivityDate="2024-03-05T19:08:26.093" Title="Arlequin - How to assign samples to populations?" Tags="|snp|format-conversion|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20535" PostTypeId="2" ParentId="20534" CreationDate="2023-02-09T10:17:57.540" Score="0" Body="&lt;p&gt;Arlequin format is historically extremely difficult to manually manage. The group leader eventually wrote software to allow easy Arlequin format management and conversion. This is achieve by Population Genetics Data (format) Spider, or PGDSpider by &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/28/2/298/198891?login=false&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lisher and Excoffier (2018)&lt;/a&gt;. All information is available &lt;a href=&quot;http://www.cmpg.unibe.ch/software/PGDSpider/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and the manual is &lt;a href=&quot;http://www.cmpg.unibe.ch/software/PGDSpider/PGDSpider%20manual_vers%202-1-1-5.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I would personally avoid attempting to manually rectify population designation.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-02-09T10:17:57.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20536" PostTypeId="1" CreationDate="2023-02-09T14:17:09.377" Score="2" ViewCount="304" Body="&lt;p&gt;I am trying to create a channel with a CSV file and use it as tuple input. Yet the output I obtain does not contain the entire file but only the first element of the file (1st row).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;the csv&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dig,val &#xA;1,One&#xA;2,Two&#xA;3,Three&#xA;4,Four&#xA;5,Five&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output :&#xA;&lt;code&gt;cat digOne.txt&lt;/code&gt;&#xA;&lt;code&gt;1 &lt;/code&gt;&#xA;instead of the whole collection&#xA;digOne, digTwo...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA; size_amp_ch = Channel&#xA;        .fromPath( params.file)&#xA;        .splitCsv(header:true).map{&#xA;            row  -&amp;gt; tuple(row.dig,row.val)&#xA;        }&#xA;&#xA;process{&#xA;    input:&#xA;&#xA;    tuple val(digits), val(values)&#xA;    &#xA;    output:&#xA;&#xA;&#xA;    path &amp;quot;dig_${values}.txt&amp;quot;&#xA;&#xA;    script:&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    echo &lt;span class=&quot;math-container&quot;&gt;$digits &amp;gt;&amp;gt; dig_$&lt;/span&gt;{values}.txt&#xA;&#xA;&#xA;}&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;thank you&lt;/p&gt;&#xA;" OwnerUserId="16935" LastEditorUserId="16935" LastEditDate="2023-02-09T14:17:38.003" LastActivityDate="2023-02-09T15:27:17.020" Title="Nextflow tuple from CSV cols" Tags="|nextflow|workflow-management|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20537" PostTypeId="2" ParentId="20536" CreationDate="2023-02-09T15:27:17.020" Score="1" Body="&lt;p&gt;I think the problem is just that you have extra whitespace at the end of the first line in your CSV file. Either remove the whitespace manually, or use the &lt;code&gt;strip&lt;/code&gt; parameter with the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#splitcsv&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;splitCsv&lt;/code&gt;&lt;/a&gt; operator to remove leading and trailing whitespace from values. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params.file = './file.csv'&#xA;&#xA;&#xA;process test_proc {&#xA;&#xA;    input:&#xA;    tuple val(digits), val(values)&#xA;&#xA;    output:&#xA;    path &amp;quot;dig_${values}.txt&amp;quot;&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    echo &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${digits}&quot; &amp;gt; &quot;dig_$&lt;/span&gt;{values}.txt&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;workflow { &#xA; &#xA;    Channel.fromPath( params.file ) \&#xA;        | splitCsv( header:true, strip:true ) \&#xA;        | map { row -&amp;gt;&#xA;            tuple( row.dig, row.val )&#xA;        } \&#xA;        | test_proc \&#xA;        | view()&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ nextflow run main.nf &#xA;N E X T F L O W  ~  version 22.10.0&#xA;Launching `main.nf` [stupefied_boyd] DSL2 - revision: 2a9d8dc414&#xA;executor &amp;gt;  local (5)&#xA;[4f/a73196] process &amp;gt; test_proc (2) [100%] 5 of 5 ✔&#xA;/path/to/work/9d/4905d0236788c1a3733da5358b7fc4/dig_Four.txt&#xA;/path/to/work/f9/d04e8a1c2265d73cde626703317df7/dig_Five.txt&#xA;/path/to/work/ff/522238f87a5641fdc7adce3e4d1cea/dig_Three.txt&#xA;/path/to/work/e9/9b7ec2a65d45986fd8944da4a2054c/dig_One.txt&#xA;/path/to/work/4f/a73196fbd56db3c9647fa7b072edfd/dig_Two.txt&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-02-09T15:27:17.020" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20538" PostTypeId="1" CreationDate="2023-02-09T15:48:24.750" Score="0" ViewCount="100" Body="&lt;p&gt;I have 3000 molecules which I need for docking but before that I need to perform energy minimization for all the molecules and it is taking a long time because I am doing it one by one. So is there a way that I can select multiple molecules and do the energy minimization&lt;/p&gt;&#xA;" OwnerUserId="16937" LastActivityDate="2023-02-10T19:10:35.787" Title="Energy minimization of molecules" Tags="|docking|molecular-dynamics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20539" PostTypeId="1" CreationDate="2023-02-10T06:22:18.427" Score="1" ViewCount="1651" Body="&lt;p&gt;&lt;em&gt;[this question is based on a question that was asked on &lt;a href=&quot;https://www.reddit.com/r/bioinformatics/comments/10ye0sr/costs_of_ont_minion/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reddit&lt;/a&gt;]&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;We're interested in doing whole-transcriptome cDNA sequencing of 24 mouse cell lines, and are deciding between Illumina and Nanopore sequencing. What are the costs involved in Nanopore sequencing, and how much hassle is it to analyse Nanopore data?&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-11-16T04:19:11.930" LastActivityDate="2023-11-16T04:19:11.930" Title="How much does Nanopore cDNA Sequencing Cost?" Tags="|rna-seq|nanopore|illumina|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20540" PostTypeId="2" ParentId="20539" CreationDate="2023-02-10T06:22:18.427" Score="6" Body="&lt;p&gt;The standard PCR-cDNA barcoding kit is \$750 USD, and has enough reagents for six runs, with each run using up to 24 samples (i.e. 144 samples total). Each run also needs one MinION flow cell (\$900 USD).&lt;/p&gt;&#xA;&lt;p&gt;The total cost including external reagents is naturally dependent on the number of samples, and gets cheaper as the number of samples increases. For 24 samples, it's about \$5200 NZD (\$3100 USD). See below for more details.&lt;/p&gt;&#xA;&lt;p&gt;If you could outsource this to a lab that has a PromethION (or P2 Solo), the sequencing would be a great lot cheaper. PromethION flow cells start at \$3600 for a pack of 4 (i.e. the same unit cost as MinION flow cells), but have about ten times the yield of MinION flow cells (which means more samples, or greater depth).&lt;/p&gt;&#xA;&lt;p&gt;From a bioinformatics perspective, there's more work to do with nanopore reads because there are no commonly used / established pipelines (so I've had to &lt;a href=&quot;https://www.protocols.io/view/stranded-transcript-count-table-generation-from-lo-5qpvonn2bl4o/v17&quot; rel=&quot;nofollow noreferrer&quot;&gt;create my own&lt;/a&gt;), but the results are much cleaner once they've been processed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Cost Breakdown&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;For cDNA sequencing in Aotearoa, MinION works out cheaper than Illumina when compared against a service centre, about two thirds the price for us. Here is the reagent list I wrote up a couple of months ago for sequencing from 30 whole transcriptome samples, mouse or human (prices in NZD). This is with an ideal read threshold of 1M reads per sample (which I have found to give good quantitative results for differential expression). You could probably get away with lower yield for microbial sequencing:&lt;/p&gt;&#xA;&lt;p&gt;ONT reagents:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;1 x PCR-cDNA Barcoding Kit 24 \$750 USD [\$1210 NZD]&lt;/li&gt;&#xA;&lt;li&gt;1 x Rapid Adapter Auxiliary V14 \$288 USD [\$481 NZD]&lt;/li&gt;&#xA;&lt;li&gt;1 x R10.4.1 flow cell @ \$900 USD [\$1503 NZD]&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Total ONT cost: \$3194&lt;/p&gt;&#xA;&lt;p&gt;Other reagents:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;10 mM dNTP NEB N0447 [\$145 NZD]&lt;/li&gt;&#xA;&lt;li&gt;Maxima H Minus RT ThermoFisher EP0751 [\$346 NZD]&lt;/li&gt;&#xA;&lt;li&gt;RNAclean or AmPure XP beads [AmPure XP: \$800 NZD]&lt;/li&gt;&#xA;&lt;li&gt;Exonuclease 1 NEB M0293 [\$160 NZD]&lt;/li&gt;&#xA;&lt;li&gt;RNAseOUT ThermoFisher 10777019 [\$569 NZD]&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Total external reagent cost (excludes nuclease-free water, tubes, pipette tips, gloves, ethanol, DNA QC): \$2020 NZD&lt;/p&gt;&#xA;&lt;p&gt;[As well as whatever you need to extract 200ng total RNA per sample]&lt;/p&gt;&#xA;&lt;p&gt;Our current protocols for this PCR-cDNA barcoding + V14 RA approach can be found &lt;a href=&quot;https://www.gringene.org/nanopore/protocols/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The comparative Illumina cost according to the service centre we asked a couple of months ago was \$185 NZD per sample + \$5000 NZD per flow cell (or $9440 NZDfor 24 samples).&lt;/p&gt;&#xA;&lt;p&gt;For almost all of these external reagents the amount ordered is an absolute minimum (from memory I think most are something like 50 reactions), so doing a second equivalent experiment would mostly only need the flow cells to be purchased.&lt;/p&gt;&#xA;&lt;p&gt;In addition to the cost, turnaround time with MinION is quicker (for us it's a couple of weeks versus a couple of months for Illumina [with most of that time spent waiting in a queue]), and the results are nicer to analyse: full length transcript sequences with very even coverage profiles.&lt;/p&gt;&#xA;&lt;p&gt;I'm aware that sequencing is more expensive in Aotearoa than it is in other countries, so the cost / time difference may not be as extreme elsewhere.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Bulk discounts for ONT flow cells&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The prices above are for ordering reagents and flow cells one at a time, using the most expensive reagents. I call this a &amp;quot;grant funding&amp;quot; budget, because it's the quote I would give to people who are looking to fund research. There are ways to reduce costs when organising sequencing in bulk, and I've already hinted above at the extra capacity in reagent kits.&lt;/p&gt;&#xA;&lt;p&gt;However, the biggest savings for nanopore sequencing can be achieved by ordering lots of flow cells at once. For example, ordering 48 MinION flow cells means that the per-flowcell price is \$500 USD instead of \$900 USD. That's 48 flow cells spread over the course of an entire year, &lt;em&gt;&lt;strong&gt;with costs paid on delivery&lt;/strong&gt;&lt;/em&gt;. If you can encourage multiple research groups to put in a combined bulk order, it's not too far out of the realm of possibility.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The long-read advantage&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I've been told that a researcher is able to get 50M paired-end Illumina reads from a sequencing service centre for about $100 CAD.&lt;/p&gt;&#xA;&lt;p&gt;50M paired end reads is about 10 Gb, which can be achieved on a MinION run (\$600 - \$1000), or about 1/10th of a PromethION run (\$850 - \$1500).&lt;/p&gt;&#xA;&lt;p&gt;Even given those prices, it doesn't take much of a long-read advantage for MinION to become the most economical option. My own studies suggest that 1M full-length reads from nanopore sequencing (i.e. about 1/10th of a MinION R9.4.1 flow cell) gives results that are at least as good as 20M short paired-end Illumina reads.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-11-16T04:18:37.933" LastActivityDate="2023-11-16T04:18:37.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20541" PostTypeId="1" CreationDate="2023-02-10T08:34:43.767" Score="2" ViewCount="132" Body="&lt;p&gt;I have a dataset of binary morphological characters and would like to know how many of them are parsimony informative. Is there a way to calculate this with an R package?&lt;/p&gt;&#xA;" OwnerUserId="2961" LastActivityDate="2023-11-08T00:07:30.833" Title="Is there a way to calculate the number of parsimony informative characters in a morphological dataset in R?" Tags="|r|phylogenetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20542" PostTypeId="1" CreationDate="2023-02-10T11:41:16.477" Score="2" ViewCount="77" Body="&lt;p&gt;&lt;em&gt;[this question is based on a question that was asked on &lt;a href=&quot;https://www.reddit.com/r/bioinformatics/comments/10y0bd4/fastest_way_to_check_e_coli_contamination_levels/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reddit&lt;/a&gt;]&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have sequenced some [mostly cell-free] DNA from a sputum sample on a nanopore sequencing machine, and would like to know what is in it (i.e. the proportion of different bacteria). The run produced 2 million reads with an average read length of about 200bp.&lt;/p&gt;&#xA;&lt;p&gt;What's the best way to go about doing a metagenomic classification of sequenced reads?&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-02-13T00:31:24.760" Title="How do I measure the proportion of different bacteria in a sample from a high-throughput sequencer?" Tags="|sequence-analysis|nanopore|metagenome|illumina|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20543" PostTypeId="2" ParentId="20542" CreationDate="2023-02-10T11:41:16.477" Score="4" Body="&lt;p&gt;If you've got a fast solid state drive with &amp;gt;200G of space, then kraken2 + bracken works really well on both long and short reads (e.g. Nanopore, Illumina). It's also specifically designed for metagenomic analysis, unlike other general aligners like STAR, bwa, bowtie2, etc., so you get tree-based classification which can deal with partial / incomplete matches.&lt;/p&gt;&#xA;&lt;p&gt;First, use &lt;a href=&quot;https://github.com/DerrickWood/kraken2/wiki&quot; rel=&quot;nofollow noreferrer&quot;&gt;kraken2&lt;/a&gt; to map reads to taxa in memory-mapped mode (to reduce system memory consumption):&lt;/p&gt;&#xA;&lt;p&gt;$ kraken2 --threads 10 --db /mnt/ultra_fast/kraken/metagenome --quick --memory-mapping reads/*.fastq.gz --report kraken2.report.txt &amp;gt; kraken2.out.txt&lt;/p&gt;&#xA;&lt;p&gt;Then, use &lt;a href=&quot;https://ccb.jhu.edu/software/bracken/index.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;bracken&lt;/a&gt; to adjust the counts based on mapping / library probabilities:&lt;/p&gt;&#xA;&lt;p&gt;$ ~/install/bracken/bracken -d /mnt/ultra_fast/kraken/metagenome -i kraken2.report.txt -o bracken.report.txt&lt;/p&gt;&#xA;&lt;p&gt;I use &lt;a href=&quot;https://hub.docker.com/r/florianbw/pavian&quot; rel=&quot;nofollow noreferrer&quot;&gt;pavian&lt;/a&gt; to look at the results, because it creates quite understandable Sankey plots:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/N3fxT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/N3fxT.png&quot; alt=&quot;Sankey plot showing metagenomic sequencing results. The three most abundant organisms are Escherichia coli, Shigella flexneri, and Salmonella enterica&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But inspecting the &lt;code&gt;_bracken_species.txt&lt;/code&gt; file is a pretty good text-based way to do it. If there's substantial amounts of a particular bacteria in the sample, it should come out in the results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;100.00  232939  0       R       1       root&#xA;99.99   232912  0       R1      131567    cellular organisms&#xA;68.52   159618  0       D       2           Bacteria&#xA;67.67   157635  0       P       1224          Proteobacteria&#xA;67.47   157152  0       C       1236            Gammaproteobacteria&#xA;61.82   144001  0       O       91347             Enterobacterales&#xA;61.79   143941  0       F       543                 Enterobacteriaceae&#xA;32.28   75183   0       G       561                   Escherichia&#xA;32.26   75147   75147   S       562                     Escherichia coli&#xA;0.02    35      35      S       1499973                 Escherichia marmotae&#xA;0.00    1       1       S       564                     Escherichia fergusonii&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;kraken2/bracken indexes can be downloaded from here (the build process takes &lt;em&gt;ages&lt;/em&gt;; it makes more sense to use the pre-built indexes):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://benlangmead.github.io/aws-indexes/k2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://benlangmead.github.io/aws-indexes/k2&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As far as I'm aware, kraken2 can't do strain-level assignment with the most commonly-used databases (although that may depend on what information that is stored in the database). For strain-level assignment, it seems like MetaMaps may work:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1038/s41467-019-10934-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1038/s41467-019-10934-2&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-02-13T00:31:24.760" LastActivityDate="2023-02-13T00:31:24.760" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20544" PostTypeId="2" ParentId="20541" CreationDate="2023-02-10T12:49:19.837" Score="1" Body="&lt;p&gt;The &lt;code&gt;ape&lt;/code&gt; library (Analyses of Phylogenetics and Evolution) using the &lt;code&gt;MPR&lt;/code&gt; method (Most Parsimonious Reconstruction).&lt;/p&gt;&#xA;&lt;p&gt;Information on &lt;code&gt;ape&lt;/code&gt; is &lt;a href=&quot;https://rdrr.io/cran/ape/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The format&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;install.packages(“ape”)&#xA;library(ape)&#xA;MPR(x, phy, outgroup)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;x&lt;/strong&gt; :   a vector of integers.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;phy&lt;/strong&gt; : an object of class &amp;quot;phylo&amp;quot;; the tree must be unrooted and fully dichotomous.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;outgroup&lt;/strong&gt; : an integer or a character string giving the tip of phy used as outgroup.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Rooting the tree is recommended. In the example rooting isn't used but an outgroup is used as taxa &lt;code&gt;t1&lt;/code&gt;, that will be fine.&lt;/p&gt;&#xA;&lt;p&gt;Generating random data,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;vectx &amp;lt;- rpois(50, 1) # make random data&#xA;tree &amp;lt;- rtree(50, rooted = FALSE)  # also rtree(50) make random tree&#xA;MPR(vectx, tree, &amp;quot;t1&amp;quot;) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thats it, there's nothing more. Data, tree = steps.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Clarification&lt;/strong&gt; the question is number of parsimonious characters. As a strict definition the number of informative characters is the same as the number of steps and thats what &lt;code&gt;MPR&lt;/code&gt; will describe ... thats easy.&lt;/p&gt;&#xA;&lt;p&gt;There are other possible meanings&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The number of parsimonious characters/steps to make a tree is calculated via &lt;code&gt;MPR&lt;/code&gt; as above.&lt;/li&gt;&#xA;&lt;li&gt;if its the actual changes at a given position in the alignment (or vector) for each node in the tree - thats the answer below - but thats complicated.&lt;/li&gt;&#xA;&lt;li&gt;If its simply is this site (not character) is parsimoniously informative and this one isn't then the &amp;quot;alignment&amp;quot; or vector could be used one character position at a time and subject to &lt;code&gt;MPR&lt;/code&gt; and the step length would determine how parsimoniously informative each site was ... thus point 3 is easily doable in &lt;code&gt;R&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; Calculating the actual sites on the alignment and associated changes per node is easily done outside &lt;code&gt;R&lt;/code&gt;, but within &lt;code&gt;R&lt;/code&gt; I don't know if thats doable.&lt;/p&gt;&#xA;&lt;p&gt;Wait, I'm wrong it's doable but I would envisage its difficult to implement. Calculating parsimonious characters for each given node in a tree - yes I do that - doing that in &lt;code&gt;R&lt;/code&gt; ... errhhh thats not me. There are Pythonic ways in, but I've not implemented them.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-02-10T23:51:58.327" LastActivityDate="2023-02-10T23:51:58.327" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20545" PostTypeId="2" ParentId="20533" CreationDate="2023-02-10T18:29:12.463" Score="2" Body="&lt;p&gt;The backbone can be specified using a selector for the names of these, which are &lt;code&gt;N&lt;/code&gt;, &lt;code&gt;CA&lt;/code&gt; (Cα) and &lt;code&gt;C&lt;/code&gt; (C-prime).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;out = pymol.cmd.align('foo and name N+CA+C and polymer', 'bar and name N+CA+C and polymer')&#xA;rmsd, n_atoms, n_cyles, n_rmsd_pre, n_atom_pre, score, n_res = out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you do not wish the proteins to be moved you can use &lt;code&gt;rms_cur&lt;/code&gt;, which is just the rmsd.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastActivityDate="2023-02-10T18:29:12.463" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20546" PostTypeId="1" CreationDate="2023-02-10T18:43:47.680" Score="2" ViewCount="73" Body="&lt;p&gt;I was asked to use palantir for trajectory analysis. I have 4 samples that I have preprocessed using scanpy. Can I run the palantir analysis on the merged object or do I need to unmerge them?&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="16902" LastActivityDate="2023-02-10T18:43:47.680" Title="Can I use palantir for trajectory analysis in a a merged object?" Tags="|single-cell|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20547" PostTypeId="2" ParentId="20538" CreationDate="2023-02-10T18:58:22.773" Score="1" Body="&lt;p&gt;As @M__ has mentioned, the terminology in your Q is a bit confusing.&lt;/p&gt;&#xA;&lt;p&gt;A molecule can be the small molecule (ligand, compound etc.) or a polypeptide chain (protein).&lt;/p&gt;&#xA;&lt;p&gt;Conformer generation of a compound is commonly not called 'energy minimisation' for technical reasons. For example, you normally generate in rdkit a conformer via &lt;code&gt;AllChem.EmbedMolecule(mol)&lt;/code&gt;, but you could do it via&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;p = AllChem.MMFFGetMoleculeProperties(mol, 'MMFF94')&#xA;ff = AllChem.MMFFGetMoleculeForceField(mol, p)&#xA;m = ff.Minimize()&#xA;assert m == 0, 'failure'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The latter would make sense if you want to add constraints (e.g. &lt;code&gt;ff.MMFFAddPositionConstraint(0, 0.1, 10)&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;There is also DFT (e.g. Gaussian or Schrödinger Jaguar), wherein the conformer is generated from a forcefield, but using a slimmed-down quantum theory to described orbitals as opposed to atoms. This is much slower than the above which use atoms as rigid particles tethered together (molecular mechanics).&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;EmbedMolecule&lt;/code&gt; uses ETKDG method, which is a stochastic search method that cheats by using knowledge derived from crystal structures (cf. CCDC), so is way faster. It will be based on crystal structures, where the latter makes the compound best suit the terms of the MMFF94 forcefield.&lt;/p&gt;&#xA;&lt;p&gt;If you are using a conformer generator, or similar, then the only quicker method is to retrieve the crystallographical structure.&lt;/p&gt;&#xA;&lt;p&gt;If you are talking about minimising a whole protein, every forcefield will give slightly different results and there is no DB you could use. However, if it is for a snapshot/pose calculation, a common hack is do a local minimisation around your target and use implicit solvent which will be much faster. You have added the tag &lt;code&gt;MD&lt;/code&gt;, which would suggest FEP calculations, for this local minimisation would not work.&lt;/p&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="3848" LastEditDate="2023-02-10T19:10:35.787" LastActivityDate="2023-02-10T19:10:35.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20548" PostTypeId="1" CreationDate="2023-02-10T20:28:22.327" Score="2" ViewCount="184" Body="&lt;p&gt;I would like to make a UMAP where the cells are colored by the average expression of the bulk signature genes but I am not confident that I did it correctly. I would like to use scanpy for it.&lt;/p&gt;&#xA;&lt;p&gt;I did the below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bulk_de_genes_up_list = bulk_de_genes['Gene'].tolist()&#xA;#Subset the data based on the list of genes&#xA;adata2 = adata[:, adata.var_names.isin(bulk_de_genes_up_list)]&#xA;average_expression = adata2.X.mean(axis=1)&#xA;adata2.var['bulk_de_gene_average'] = average_expression&#xA;sc.pl.umap(adata2, color='bulk_de_gene_average', cmap='viridis')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I do get a UMAP as an output but I am not sure if it is done correctly. I am mainly worried about average_expression = adata2.X.mean(axis=1)&lt;/p&gt;&#xA;&lt;p&gt;Is that the correct way of calculating the mean of the gene expression per cell?&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="16902" LastEditorUserId="16902" LastEditDate="2023-02-13T20:19:47.427" LastActivityDate="2023-02-13T20:19:47.427" Title="How to make a UMAP for single cell data and color cells by average expression of a list of genes in scanpy?" Tags="|python|scrnaseq|single-cell|umap|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20550" PostTypeId="1" CreationDate="2023-02-10T23:59:37.560" Score="2" ViewCount="59" Body="&lt;p&gt;Is there a database with all known proteins/genes of specific pathogens such as Staph aureus and annotated whether it is a known virulence or resistance factor that idealy can be downloaded in table format?&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="2108" LastEditDate="2023-02-16T19:07:40.027" LastActivityDate="2023-02-16T19:07:40.027" Title="Get a comprehensive list of virulence factors for certain or all pathogens?" Tags="|public-databases|microbial-genomics|pathogenicity|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20551" PostTypeId="2" ParentId="20550" CreationDate="2023-02-11T00:34:55.320" Score="4" Body="&lt;p&gt;I suggest that you are looking for a resource like &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwjfw6vom4z9AhV0IX0KHZFaAa4QFnoECAkQAQ&amp;amp;url=http%3A%2F%2Fwww.mgc.ac.cn%2FVFs%2F&amp;amp;usg=AOvVaw1AmmhEo5WZmOQGcDHHJ4gY&quot; rel=&quot;nofollow noreferrer&quot;&gt;VFDB&lt;/a&gt; (virulence factor database). That table does annotate by bacterial species, so you can filter as you like.&lt;/p&gt;&#xA;&lt;p&gt;You can download a description table or FASTA sequences from their &lt;a href=&quot;http://www.mgc.ac.cn/VFs/download.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;downloads page&lt;/a&gt; (see screenshot). Last update was 4 days ago, so they should be quite current.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VD47V.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VD47V.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For resistance genes, you could use something like &lt;a href=&quot;https://card.mcmaster.ca/&quot; rel=&quot;nofollow noreferrer&quot;&gt;CARD&lt;/a&gt; (Comprehensive Antibiotic Resistance Database), which also has a &lt;a href=&quot;https://card.mcmaster.ca/download&quot; rel=&quot;nofollow noreferrer&quot;&gt;downloads page&lt;/a&gt; where you can download a similar TSV table or FASTA sequences.&lt;/p&gt;&#xA;&lt;p&gt;Both resources include web tools for predicting the presence of the elements of interest in sequences.&lt;/p&gt;&#xA;&lt;p&gt;If you are looking for a state of the art prediction tool for such elements in genomes, you might look into &lt;a href=&quot;https://gitlab.com/treangenlab/seqscreen&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqscreen&lt;/a&gt; from Todd Treangen's lab.&lt;/p&gt;&#xA;&lt;p&gt;For a general purpose microbiological annotation and analysis resource, you might also look into &lt;a href=&quot;https://www.bv-brc.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BV-BRG&lt;/a&gt;, which succeeds PATRIC. I have only used PATRIC but BV-BRG seems very thorough.&lt;/p&gt;&#xA;&lt;p&gt;I think that you are unlikely to find a resource that has &lt;em&gt;non-virulent/non-resistance&lt;/em&gt; genes annotated.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2023-02-14T19:01:05.420" LastActivityDate="2023-02-14T19:01:05.420" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20552" PostTypeId="1" AcceptedAnswerId="20557" CreationDate="2023-02-11T11:58:29.447" Score="1" ViewCount="275" Body="&lt;p&gt;My goal:&#xA;I want to trim off the primers (Forward : CGAGAAGACCCTRTGRAGCT, Reverse : GTTGGGGYGACCNYGG) from a fasta file with a lot of dna sequences allowing for some (e.g. 3) mismatches (identity).&lt;/p&gt;&#xA;&lt;p&gt;Input sequences&#xA;Seq1:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;CGAGAAGACCCTATGGAGCT&lt;/strong&gt;TAAGGCGCCAGAACAGCTCACGTCAAACACCCCCGCATAAAGGGAATAAACCAAGTGGACCCTGCTCTAGTGTCTTTG&lt;strong&gt;GTTGGGGCGACCGCGG&lt;/strong&gt;AACGT&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Seq2:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;ATGGCCCATCCCTCACAGCTAGGATTTCAAGATGCAGCTTCCCCAGTTATAGAAGAACTTCTCCACTTTCACGACCATGCCCTAATAATCGTTTTTTTAATTAGTACACTAGTA&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Seq3:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;CATAAGA&lt;strong&gt;CGAAAAGACCCTATGGAGCT&lt;/strong&gt;TTAGACGTCAGAGCAGCTCATGTAAAGCACCCCTAAACAAAGGAAAAAACCAAATGAAATCTGCCCTAATGTCTTTG&lt;strong&gt;GTTCGGGCGACCGCGG&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Output sequences&lt;/p&gt;&#xA;&lt;p&gt;Seq1:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;TAAGGCGCCAGAACAGCTCACGTCAAACACCCCCGCATAAAGGGAATAAACCAAGTGGACCCTGCTCTAGTGTCTTTG&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Seq3:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;TTAGACGTCAGAGCAGCTCATGTAAAGCACCCCTAAACAAAGGAAAAAACCAAATGAAATCTGCCCTAATGTCTTTG&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Seq2 is absent because it has no match for the primers so it should be discarded.&lt;/p&gt;&#xA;&lt;p&gt;In seq 3 there is a mismatch in Forward (CGA&lt;strong&gt;A&lt;/strong&gt;AA...) and in Reverse (GTT&lt;strong&gt;C&lt;/strong&gt;GG...).  I don't need the primers to have 100% identity.  85-90% is OK.&lt;/p&gt;&#xA;&lt;p&gt;Issues:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;primer sequences can be anywhere in each sequence, not just in the beginning.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The sequence between the primers is not constant.  it is approximately 80 bases but not exactly 80.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I don't know how to put the primers' identity parameter.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;How can I search for R or N nucleotides (IUPAC)? Because I don't know how, I wrote the primers with specific nucleotides and not R or N.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This is the code I have so far, but it has the problems mentioned above:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;def trim_adaptors(records, adaptorF,adaptorR):&#xA;&#xA;    len_adaptorF = len(adaptorF)  &#xA;    len_adaptorR = len(adaptorR)&#xA;&#xA;    for record in records:&#xA;        indexF = record.seq.find(adaptorF)&#xA;        indexR = record.seq.find(adaptorR)&#xA;        if indexR == -1  or indexF == -1:&#xA;            # adaptor not found, so won't trim&#xA;            print(record.id, &amp;quot;no primer/s match&amp;quot;)&#xA;            continue&#xA;        else:&#xA;            # trim off the adaptor&#xA;            yield record[indexF + len_adaptorF:indexR + 1]&#xA;&#xA;original_reads = SeqIO.parse(&amp;quot;trimming_testfas.fas&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;&#xA;trimmed_reads = trim_adaptors(original_reads,&amp;quot;CGAGAAGACCCTATGGAGCT&amp;quot; ,&amp;quot;GTTGGGGCGACCGCGG&amp;quot;)&#xA;&#xA;count = SeqIO.write(trimmed_reads, &amp;quot;trimmed.fasta&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;&#xA;print(&amp;quot;Saved %i reads&amp;quot; % count)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This doesn't have to be a script in Python, I am also open to using existing tools.&lt;/p&gt;&#xA;&lt;p&gt;Thank you a lot in advance, hope you got me :)&lt;/p&gt;&#xA;" OwnerUserId="16945" LastEditorUserId="298" LastEditDate="2023-02-11T18:30:58.517" LastActivityDate="2023-02-12T01:18:35.723" Title="Primer trimming-fasta files" Tags="|python|trimming|primer|multi-fasta|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20553" PostTypeId="1" CreationDate="2023-02-11T13:23:43.633" Score="1" ViewCount="168" Body="&lt;p&gt;In my pipeline nextflow, I create a channel with channel.fromSRA and the channel contains lots of heavy files.fastq.gz. Then I have a first process to unzip files and transform them in files.fasta and a second process to count kmers. Is it possible to remove files.gz after first process and remove files.fasta after the second process. Can someone explain me how to do? They are too heavy to keep them in the work directory. Thanks.&lt;/p&gt;&#xA;&lt;p&gt;In my pipeline I use kmer-ssr to count kmers present on my genome sample.&lt;/p&gt;&#xA;&lt;p&gt;First I create a reads_ch = Channel.fromSRA&lt;/p&gt;&#xA;&lt;p&gt;In this channel there are 12 files (. fastq.gz) of 20 Gb each one.&lt;/p&gt;&#xA;&lt;p&gt;Then the first process unzips them and transforms them in files.fasta.&lt;/p&gt;&#xA;&lt;p&gt;Now in my work directory there are 12 more files of 80 Gb each one. So the work directory is now 1200 Gb.&lt;/p&gt;&#xA;&lt;p&gt;The second process counts kmer present in reads and creates 12 files of 100 Gb each one. The work directory will be 2200 Gb.&lt;/p&gt;&#xA;&lt;p&gt;The work directory is too heavy. I want to know how I can suppress the current file. fastq.gz after first process&lt;/p&gt;&#xA;" OwnerUserId="16948" LastEditorUserId="73" LastEditDate="2023-02-12T02:39:13.527" LastActivityDate="2024-03-08T05:03:20.140" Title="Remove files after process is terminated" Tags="|fasta|fastq|nextflow|k-mer|workflow-management|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20556" PostTypeId="1" CreationDate="2023-02-11T16:29:55.687" Score="1" ViewCount="83" Body="&lt;p&gt;I have received two NGS files from an NGS company, both are FASTQ files that correspond to reads from a tumor sample. I heavily suspect that MCPyV is present, and I am hoping to identify it.&lt;/p&gt;&#xA;&lt;p&gt;What downloadable software can I use for viral-host integration detection? I have access to both Windows and Mac OS, but have no experience with Linux.&lt;/p&gt;&#xA;" OwnerUserId="16949" LastActivityDate="2024-03-09T05:07:25.973" Title="How can I detect MCPyV in a FASTQ file?" Tags="|fastq|virus|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20557" PostTypeId="2" ParentId="20552" CreationDate="2023-02-11T18:28:41.077" Score="2" Body="&lt;p&gt;The IUPAC ambiguity codes can be thought of as regular expression character classes. &lt;code&gt;R&lt;/code&gt; matches any purine, so &lt;code&gt;[AG]&lt;/code&gt;, &lt;code&gt;Y&lt;/code&gt; matches any pyrimidine, so &lt;code&gt;[CT]&lt;/code&gt;, and &lt;code&gt;N&lt;/code&gt; matches anything at all, so &lt;code&gt;.&lt;/code&gt; or &lt;code&gt;[ACTG]&lt;/code&gt;. So those are easy to handle: instead of &lt;code&gt; CGAGAAGACCCTRTGRAGCT&lt;/code&gt; you can use the regular expression &lt;code&gt;CGAGAAGACCCT[AG]TG[AG]AGCT&lt;/code&gt;, and instead of &lt;code&gt;GTTGGGGYGACCNYGG&lt;/code&gt; you can use &lt;code&gt;GTTGGGG[CT]GACC.[CT]GG&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Allowing for random mismatches is trickier though and since you're open to using existing tools, I strongly urge you to use something like &lt;code&gt;cutadapt&lt;/code&gt; instead of trying to reinvent the wheel. So, install &lt;code&gt;cutadapt&lt;/code&gt; on your system (see &lt;a href=&quot;https://cutadapt.readthedocs.io/en/stable/installation.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for instructions) and then run:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cutadapt -a &amp;quot;CGAGAAGACCCTRTGRAGCT...GTTGGGGYGACCNYGG;max_error_rate=0.15;&amp;quot; --discard-untrimmed -o output.fa file.fa  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That assumes your input sequences are in fasta format in the file &lt;code&gt;file.fa&lt;/code&gt; and  will create the output file &lt;code&gt;output.fa&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat output.fa &#xA;&amp;gt;seq1&#xA;TAAGGCGCCAGAACAGCTCACGTCAAACACCCCCGCATAAAGGGAATAAACCAAGTGGACCCTGCTCTAGTGTCTTTG&#xA;&amp;gt;seq3&#xA;TTAGACGTCAGAGCAGCTCATGTAAAGCACCCCTAAACAAAGGAAAAAACCAAATGAAATCTGCCCTAATGTCTTTG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The options used are &lt;code&gt;-a $primer1..$primer2&lt;/code&gt; which tells &lt;code&gt;cutudapt&lt;/code&gt; these primers are linked adapters (&lt;a href=&quot;https://cutadapt.readthedocs.io/en/stable/guide.html#linked-adapters&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cutadapt.readthedocs.io/en/stable/guide.html#linked-adapters&lt;/a&gt;) and then we also add &lt;code&gt;max_error_rate=0.15;&lt;/code&gt; to allow up to 85% identity matches. Cutadapt already knows how to handle IUPAC ambiguity codes so nothing special is needed for those.&lt;/p&gt;&#xA;&lt;p&gt;I am not sure how this will handle cases where only one of the two sequences is present, you'll need to test for that. I expect that they will be discarded since we're specifying linked primers but I would double check to be sure. In any case, &lt;code&gt;cutadapt&lt;/code&gt; is a very powerful tool, so I'm sure there will be a way to handle that in the way you want if you check the documentation.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-02-11T18:28:41.077" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20558" PostTypeId="1" CreationDate="2023-02-11T21:49:06.847" Score="1" ViewCount="15" Body="&lt;p&gt;I have two NGS files from an NGS company corresponding to the sequencing data from a tumor sample as follows:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;TB_7710391_R1.FASTQ.gz&lt;/li&gt;&#xA;&lt;li&gt;TB_7710391_R2.FASTQ.gz&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I have downloaded the genome for MCPyV as a file called “MCPYV_Sequence.fasta”&lt;/p&gt;&#xA;&lt;p&gt;I have downloaded the Hg38 as a file called “hg_38.fa”&lt;/p&gt;&#xA;&lt;p&gt;I have input the host genome as hg_38.fa, the viral genome as “ MCPYV_Sequence.fasta” and the sequence data as both FASTQ files.&lt;/p&gt;&#xA;&lt;p&gt;I run the assembly in the cloud, and I receive back an error stating that the assembly as failed.&lt;/p&gt;&#xA;&lt;p&gt;What am I doing wrong?&lt;/p&gt;&#xA;" OwnerUserId="16949" LastEditorUserId="16949" LastEditDate="2023-02-11T21:49:40.260" LastActivityDate="2023-02-11T21:49:40.260" Title="DNASTAR viral-host integration assembly keeps failing" Tags="|ngs|assembly|fastq|virus|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20559" PostTypeId="2" ParentId="20553" CreationDate="2023-02-12T03:25:57.977" Score="0" Body="&lt;p&gt;Unfortunately, this remains an open issue with Nextflow: &lt;a href=&quot;https://github.com/nextflow-io/nextflow/issues/452&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nextflow-io/nextflow/issues/452&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, writing large amounts of uncompressed data to disk is usually best avoided. If it can't be (e.g. maybe the file needs to be read multiple times), you'd need to merge the two processes so that the writing of the temporary file and the running of the tool is done in a single process. This would then let you remove the temporary input file as soon as it's no longer required.&lt;/p&gt;&#xA;&lt;p&gt;Another way would be to pipe the uncompressed and FASTA formatted reads directly into your tool. Not all tools support this functionality obviously, but &lt;a href=&quot;https://github.com/ridgelab/Kmer-SSR&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kmer-SSR&lt;/a&gt; &lt;em&gt;can&lt;/em&gt; indeed receive input from STDIN, which completely obviates the need to write any temporary files. Note that by default, Kmer-SSR writes also to STDOUT and this lets you write a compressed output file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process kmer_ssr {&#xA;&#xA;    cpus 4&#xA;&#xA;    input:&#xA;    tuple val(sample), path(fastq)&#xA;&#xA;    output:&#xA;    tuple val(sample), path(&amp;quot;${sample}.tsv.gz&amp;quot;)&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    zcat -f &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${fastq}&quot; \\&#xA;        | sed -n '1~4s/^@/&amp;gt;/p; 2~4p' \\&#xA;        | kmer-ssr -t $&lt;/span&gt;{task.cpus} \\&#xA;        | gzip \\&#xA;        &amp;gt; &amp;quot;${sample}.tsv.gz&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Since the above uses a pipeline, make sure to add the &lt;code&gt;pipefail&lt;/code&gt; to your command. One way is to simply apply it to all processes by adding the following to your &lt;code&gt;nextflow.config&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process {&#xA;&#xA;  shell = [ '/bin/bash', '-euo', 'pipefail' ]&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From the &lt;code&gt;bash&lt;/code&gt; man page:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;pipefail&lt;/p&gt;&#xA;&lt;p&gt;If set, the return value of a pipeline is the value of the last&#xA;(rightmost) command to exit with a non-zero status, or zero if all&#xA;commands in the pipeline exit successfully. This option is disabled by&#xA;default.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-02-12T03:25:57.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20560" PostTypeId="1" CreationDate="2023-02-12T11:13:55.130" Score="1" ViewCount="43" Body="&lt;p&gt;I have data in paired-end as well as single-end format for the same sample.I would like to split those two files into two per each, where one contains sequences that have 5' end and the other one has 3' ends.&lt;/p&gt;&#xA;&lt;p&gt;So the paired end data has R1 and R2, if R1 is first, it has 5' ends and if not, it has 3' ends? And I'd do the same thing for the single end data.&lt;/p&gt;&#xA;&lt;p&gt;I'm using samtools and pysam.&lt;/p&gt;&#xA;" OwnerUserId="16983" LastEditorUserId="16983" LastEditDate="2023-02-12T11:14:11.257" LastActivityDate="2023-02-12T11:14:11.257" Title="How do I filter reads from a bamfile based on 5' and 3' ends?" Tags="|samtools|pysam|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20561" PostTypeId="2" ParentId="20556" CreationDate="2023-02-12T19:23:52.190" Score="0" Body="&lt;p&gt;You could start by trying out a metagenomics analysis program like kraken2. There are a few databases that combine bacteria, fungi, and viral searches.&lt;/p&gt;&#xA;&lt;p&gt;One benefit of using kraken2 is that it has options to output the most likely taxa of matches at the sub-read level, which can help with identifying integrated or chimeric sequences.&lt;/p&gt;&#xA;&lt;p&gt;I've written up the higher-level approach in more detail in this answer:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/20542/how-do-i-measure-the-proportion-of-different-bacteria-in-a-sample-from-a-high-th&quot;&gt;How do I measure the proportion of different bacteria in a sample from a high-throughput sequencer?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The sequence-level hypotheses about kmer origin can be found in the direct output of kraken2 (last column):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ kraken2 --threads 10 --db /media/minion/minion_temp/db/kraken/metagenome --memory-mapping fastq_pass/barcode0?/PAK83314_pass_barcode0?_*.fastq.gz --report kraken2.report.txt &amp;gt; kraken2.out.txt&#xA;$ head kraken2.out.txt&#xA;C       a3e0b7c9-e6f3-40bf-900c-b1c59eb4977d    9606    208     0:70 623:3 0:53 9606:1 0:6 9606:32 131567:5 2759:4&#xA;C       8030d33f-3838-4ed3-bab6-5f3b79fd059e    575614  164     0:68 623:1 0:46 575614:2 0:13&#xA;C       5d87a648-799d-451f-b747-32a0fdf2517a    9606    190     0:108 9606:5 0:43&#xA;C       8aba667d-4a5b-4f8b-9d18-1b5f761b87b2    9606    267     0:124 9606:9 0:99 1:1&#xA;C       1b3501c2-6766-4ee8-b984-9e92470f37da    9606    335     0:70 28901:5 0:12 35885:1 0:23 9606:16 0:83 9606:17 0:6 9606:2 0:9 9606:5 0:11 9606:41&#xA;U       75c2625d-ab87-4614-8d0d-c9d12c7d1e19    0       242     0:208&#xA;U       902b0ec1-80d9-4e07-80c8-29a79191d931    0       171     0:137&#xA;C       56b7c8a8-4cfd-44ec-8922-62871b29d31c    9606    338     0:103 9606:5 0:22 9606:5 0:1 9606:13 0:55 9606:27 0:38 9606:3 0:32&#xA;C       68bf6718-2692-46e7-ba2d-f45952fd6138    9606    197     0:68 1:9 0:38 9606:19 0:29&#xA;C       81f0f4ae-3bb5-4b94-89ff-152ff3b6f5d7    9606    238     0:146 9606:46 0:12&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm aware that you're interested in looking for integration of a &lt;em&gt;single&lt;/em&gt; viral genome. This can be specifically looked at with kraken2 because it can handle custom-built indexes as well. However, the benefit of using a comprehensive index (e.g. the &lt;code&gt;MinusB&lt;/code&gt; database from &lt;a href=&quot;https://benlangmead.github.io/aws-indexes/k2&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) is that it will make it much easier to exclude the other numerous endogenous viruses that are already integrated into the genome, whereas using a MCPyV-only index may lead to false-positive matches to sequences that are from another virus.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-02-13T00:30:15.037" LastActivityDate="2023-02-13T00:30:15.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20562" PostTypeId="2" ParentId="20556" CreationDate="2023-02-13T01:06:16.853" Score="0" Body="&lt;p&gt;FYI (non-OPs) it is very likely this sample has already been diagnosed and proven positive via a PCR test. I don't know the OP, but I am pretty familiar with oncovirus studies. To be clear, my guess is that PCR positive - doesn't mean cancer.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What I think is occurring (if I'm wrong hopefully the OP will say) - is the  OP has a cancer patient(s) and has good evidence that the underlying cause is MCPyV: which is a very specific oncovirus.&lt;/li&gt;&#xA;&lt;li&gt;What they want to do is identify viral integration in the patient genome as evidence of oncogenesis. To put this &lt;em&gt;general&lt;/em&gt; field in perspective I think this issue is literally the rise and fall of many investigators and companies in cancer (again not focusing on a singular virus &lt;em&gt;per se&lt;/em&gt; but generically on the oncoviruses). Keep in mind I have an explanatory bias towards viruses.&lt;/li&gt;&#xA;&lt;li&gt;The molecular genetics of integration is likely to be  complex.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In summary, the OP will particularly want to do (I suspect) is map the viral integration site. What is difficult is not diagnosis of the virus because at any time post-any sort of approximate assembly the genome can be locally &lt;code&gt;blasted&lt;/code&gt; against the reference virus and its likely to have already been PCR'd. It's getting an assembly good enough to understanding the integration site - to demonstrate integration has occurred and thats is likely difficult (but I hope I'm wrong). Inverse PCR was used for this sort of thing - but thats a wetlab approach and PCR-based strategies with traditional Sanger sequencing can still be very useful here, but its laborious but was/is a  traditional approach.&lt;/p&gt;&#xA;&lt;p&gt;Thus, I suspect the OP is from a classical virological background venturing into NGS. I noted the OP used a human reference genome hg38 in a previous post.&lt;/p&gt;&#xA;&lt;p&gt;If thats correct then ... the big issue in my opinion is obtaining a correct assembly of the putative virus-human genome &lt;strong&gt;O&lt;/strong&gt; Steps would be,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;QC&lt;/li&gt;&#xA;&lt;li&gt;Assembly e.g. via &lt;code&gt;bwa-mem2&lt;/code&gt; ... &lt;strong&gt;O&lt;/strong&gt; &lt;strong&gt;O&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;li&gt;Construct a local &lt;code&gt;makeblastdb&lt;/code&gt; database of the genome then ..&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;blastn&lt;/code&gt; between the human genome and a viral query to identify the integration site&lt;/li&gt;&#xA;&lt;li&gt;Perform something like a &lt;code&gt;gatk&lt;/code&gt; analysis to have a different perspective on the integration site via VCF&lt;/li&gt;&#xA;&lt;li&gt;hone in on the contig via &lt;code&gt;IGV&lt;/code&gt; for optical analysis and study how the virus has integrated&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;O&lt;/strong&gt; My eukaryotic genomics is outdated and the above approach is summarised in Genomics in the Cloud by Geraldine &lt;a href=&quot;https://www.oreilly.com/library/view/genomics-in-the/9781491975183/&quot; rel=&quot;nofollow noreferrer&quot;&gt;A. Van der Auwera, Brian D. O'Connor&lt;/a&gt; published through O'Reilly Media. I just focus on small genomes.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;O&lt;/strong&gt; &lt;strong&gt;O&lt;/strong&gt; and &lt;strong&gt;NOTE&lt;/strong&gt; I cannot remember if &lt;code&gt;bwa-mem2&lt;/code&gt; uses a reference genome e.g. hg38 type thing (Auwera and O'Connor will describe it). If it does the assembly of the integration site, in my opinion, would be compromised and alternative assembly strategies might (probably would) be required. I don't think this will be trivial - I think it would need a skilled eukaryotic bioinformatician who could be told about the biology of viral integration. Keep in mind, eukaryotic genome assembly is not my thing.&lt;/p&gt;&#xA;&lt;p&gt;However, riches await the investigator if this can be satisfactorily solved IMO.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To answer @gringers point, &lt;code&gt;Kraken2&lt;/code&gt; would be useful for viral integration &lt;em&gt;per se&lt;/em&gt;  and yeah the results would be interesting because there's quite of them ... but in this case - I think - its an exact genetic sequence of a single species, thus I think direct blast-type blasting or &lt;code&gt;minimap2&lt;/code&gt; mapping is a better strategy than &lt;code&gt;Kraken2&lt;/code&gt;. Just my opinion.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-02-13T02:28:14.697" LastActivityDate="2023-02-13T02:28:14.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20563" PostTypeId="1" CreationDate="2023-02-13T10:39:12.083" Score="1" ViewCount="37" Body="&lt;p&gt;Is anyone aware of a way to download all vector sequences from &lt;a href=&quot;https://www.addgene.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;addgene&lt;/a&gt; at once? As a non-profit, I'd assume they shouldn't be opposed to making this data available, but it doesn't seem to be possible. Might anyone have scraped it at some point?&lt;/p&gt;&#xA;" OwnerUserId="12306" LastEditorUserId="12306" LastEditDate="2023-02-13T10:39:37.707" LastActivityDate="2023-02-13T10:39:37.707" Title="Download Addgene database" Tags="|database|data-download|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20564" PostTypeId="1" CreationDate="2023-02-13T14:59:28.417" Score="1" ViewCount="600" Body="&lt;p&gt;I have a Seurat object (metadata) with the single R samples consisting of cell types and sample types columns. I am trying to make a table that has a sample and percentage of cell types for each sample so that I can run a t-test across the samples that came out of 3 prime vs multi-omics.&lt;/p&gt;&#xA;&lt;p&gt;I have a seuratobject meta data with singler_labels (cell labels like chondrocytes, astrocytes, macrophage...), sample names (S0030, S0031, S0032.....), cell id/barcodes, and everything a seurat object has. I want to make a table with samples and percent of cell types per sample just to see what sample has what percent types of cells.&lt;/p&gt;&#xA;&lt;p&gt;How do I do this?&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="16997" LastEditorUserId="73" LastEditDate="2023-02-15T19:34:18.817" LastActivityDate="2023-03-06T13:22:16.460" Title="How to calculate cell type percentage in every sample" Tags="|rna-seq|scrnaseq|seurat|single-cell|multi-omics|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20565" PostTypeId="1" AcceptedAnswerId="20567" CreationDate="2023-02-13T15:00:34.040" Score="2" ViewCount="125" Body="&lt;p&gt;This feels like a silly question, but I am testing parabricks bammetrics and nothing is being written to my output file. This is the command I am using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker run \&#xA;  --rm \&#xA;  --volume ~/workdir:/workdir \&#xA;  --volume ~/ouputdir:/outputdir \&#xA;  -w /workdir nvcr.io/nvidia/clara/clara-parabricks:4.0.1-1 \&#xA;  pbrun \&#xA;  bammetrics \&#xA;  --ref /workdir/reference.fasta \&#xA;  --bam /workdir/cram.cram \&#xA;  --interval-file /workdir/regions.bed \&#xA;  --out-metrics-file /outputdir/metrics.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But metrics.txt doesn't have anything written in it when I open it after the command is run. Am I doing something wrong? I am very new to parabricks, so I assume I am missing something simple&lt;/p&gt;&#xA;&lt;p&gt;I apologize if this is the wrong place to ask this question.&lt;/p&gt;&#xA;" OwnerUserId="16999" LastEditorUserId="3848" LastEditDate="2023-02-14T19:01:46.463" LastActivityDate="2023-02-15T01:22:32.967" Title="Why might parabricks not be writing output to the output file I provided?" Tags="|wgs|docker|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20566" PostTypeId="2" ParentId="18137" CreationDate="2023-02-13T15:21:52.763" Score="0" Body="&lt;p&gt;This information is now easily available in JSON format here: &lt;a href=&quot;https://github.com/corneliusroemer/pango-sequences/blob/main/data/pango-consensus-sequences_summary.json&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/corneliusroemer/pango-sequences/blob/main/data/pango-consensus-sequences_summary.json&lt;/a&gt;. It's one big dictionary of dictionaries with Pango lineage names as outer keys and corresponding Nextstrain clades accessible via inner key:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-json prettyprint-override&quot;&gt;&lt;code&gt;{&#xA; &amp;quot;A&amp;quot;: {&#xA;  &amp;quot;lineage&amp;quot;: &amp;quot;A&amp;quot;,&#xA;  &amp;quot;nextstrainClade&amp;quot;: &amp;quot;19B&amp;quot;,&#xA;  &amp;quot;nucSubstitutions&amp;quot;: [&#xA;   &amp;quot;C8782T&amp;quot;,&#xA;   &amp;quot;T28144C&amp;quot;&#xA;  ],&#xA;  ...&#xA; },&#xA; &amp;quot;A.1&amp;quot;: {&#xA;  &amp;quot;lineage&amp;quot;: &amp;quot;A.1&amp;quot;,&#xA;  &amp;quot;unaliased&amp;quot;: &amp;quot;A.1&amp;quot;,&#xA;  &amp;quot;nextstrainClade&amp;quot;: &amp;quot;19B&amp;quot;,&#xA;  ...&#xA; },&#xA; ...&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It's straightforward to convert this to a TSV mapping using just minimal scripting.&lt;/p&gt;&#xA;&lt;p&gt;Beware that not all Pango lineages pre-Omicron are contained therein. Small ones are missing currently.&lt;/p&gt;&#xA;&lt;p&gt;Disclaimer: I created the repo.&lt;/p&gt;&#xA;" OwnerUserId="12272" LastActivityDate="2023-02-13T15:21:52.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20567" PostTypeId="2" ParentId="20565" CreationDate="2023-02-13T16:36:09.310" Score="3" Body="&lt;p&gt;&lt;strong&gt;EDIT Summary&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;For future users who encounter the problem the solution is,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker run --platform linux/amd64 --rm -it -u 0 --entrypoint /bin/sh  nvcr.io/nvidia/clara/clara-parabricks:4.0.1-1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am unhappy with the practice of limiting Docker access via blocking standard runtime: there are statements of intent on SE as a strategy to attempt to restrict Docker access. Parabricks isn't an isolated incident of deploying this setup within bioinformatics. However, I checked their &lt;a href=&quot;https://forums.developer.nvidia.com/t/how-parabricks-hide-docker-container-content/192104&quot; rel=&quot;nofollow noreferrer&quot;&gt;forum&lt;/a&gt; and there is no statement of intent that it's deliberate. Parabrick's copyrights are &lt;a href=&quot;https://docs.nvidia.com/clara/parabricks/3.7.0/Documentation/ThirdPartySoftwareNotices.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. There would be issues if there was ever an official statement to this extent. They would likely argue it was for other reasons to automate stuff.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;em&gt;Original answer&lt;/em&gt; ...&lt;/p&gt;&#xA;&lt;p&gt;I don't know parabricks, but I know Docker. What you are attempting is to run Docker from the outside ... I would get on the inside to view the error log. Whats happening is the error is not being passed outside the container. Getting on the inside requires&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker run  --rm -it --name mydocker nvcr.io/nvidia/clara/clara-parabricks:4.0.1-1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This &lt;em&gt;should&lt;/em&gt; give you access to the container. If it doesn't post back and I'll forward a different command that will. It's impossible to block access to a Docker container albeit some people try.&lt;/p&gt;&#xA;&lt;p&gt;For simplicity is worth trying&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker run  --rm -it --name mydocker clara-parabricks:4.0.1-1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Anyway, if you are using this approach you'll need,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker cp *outside container* *insider container* nvcr.io/nvidia/clara/clara-parabricks:4.0.1-1 and vice versa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... this is an amazing command and allows you to move data in and out of the container. You've got to get the data on the inside of the docker and &lt;code&gt;docker cp&lt;/code&gt; does that very well. I would then look to running the whole thing from the inside of the container. Thats exactly how I run Dockers, then I exit and return when it's done. There might be a bit of clean via this approach, I'm not sure &lt;code&gt;--rm&lt;/code&gt; might take care of it.&lt;/p&gt;&#xA;&lt;p&gt;You'll also be able to look at the programs and copy it out onto hard drive. This last bit may be not advisable - the whole point of Docker is to protect your system from their algorithms ... but you can definitely do it and if you get it paired up with NextFlow thats cool.&lt;/p&gt;&#xA;&lt;p&gt;I don't particularly want to pull the Docker container because they're often huge and I've no interest in the algorithm albeit I've spotted its at &lt;code&gt;nvcr.io/nvidia/clara/&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Docker, to be honest it's lazy by the developers. Docker is - techie packaging. So it's good for reviewers and behind web firewalls. It's useful but like you've just experienced  if it flops you're stuck. The one positive thing about Docker (apart from DevOps) is it's generally tolerated to package someone else's algorithm as part of a pipeline and let a third party 'pull it'. This is because they'll be a Dockerfile which can reconstruct the whole thing without copyright infringement. So if that first developer objects ... the Docker developer would just issue the Dockerfile. &lt;code&gt;conda&lt;/code&gt; and stuff max out on that concept.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;There's a weird error as the OP described below.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Right&lt;/strong&gt; Docker uses by default a tag &lt;code&gt;latest&lt;/code&gt; that does not work for parabricks. The tag must be &lt;code&gt;4.0.1-1&lt;/code&gt;. Could you retry the above code please?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt; For good measure try the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker run  --rm -it --name mystuff  nvcr.io/nvidia/clara/clara-parabricks:4.0.1-1 --entrypoint /bin/bash&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Might not be bash, thus&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker run  --rm -it --name mystuff  nvcr.io/nvidia/clara/clara-parabricks:4.0.1-1 --entrypoint /bin/sh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;They shouldn't throw an error, but you might still find you are outside the container after running it. Lets see.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The error looks okay, because a runtime error is exactly what would be expected if they are attempting to prevent access. Therefore please try:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker run --platform linux/amd64 --rm -it -u 0 --entrypoint /bin/sh  nvcr.io/nvidia/clara/clara-parabricks:4.0.1-1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This, or a variant of it, should work.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-02-15T01:22:32.967" LastActivityDate="2023-02-15T01:22:32.967" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20568" PostTypeId="1" CreationDate="2023-02-13T17:31:04.057" Score="1" ViewCount="444" Body="&lt;p&gt;I would like to make a UMAP where the cells are colored by the average expression of the bulk signature genes but I am not confident that I did it correctly. I would like to use scanpy for it.&lt;/p&gt;&#xA;&lt;p&gt;I did the below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bulk_de_genes_up_list = bulk_de_genes['Gene'].tolist()&#xA;# Subset the data based on the list of genes&#xA;adata2 = adata[:, adata.var_names.isin(bulk_de_genes_up_list)]&#xA;average_expression = adata2.X.mean(axis=1)&#xA;adata2.obs['bulk_de_gene_average'] = average_expression&#xA;sc.pl.umap(adata2, color='bulk_de_gene_average', cmap='viridis')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I do get a UMAP as an output but I am not sure if it is done correctly. I am mainly worried about &lt;code&gt;average_expression = adata2.X.mean(axis=1)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is that the correct way of calculating the mean of the gene expression per cell?&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="16902" LastEditorUserId="73" LastEditDate="2023-02-15T19:27:10.790" LastActivityDate="2023-02-15T19:27:10.790" Title="How to plot average gene expression in scanpy?" Tags="|python|scrnaseq|umap|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20571" PostTypeId="1" AcceptedAnswerId="20591" CreationDate="2023-02-14T16:47:17.250" Score="2" ViewCount="89" Body="&lt;p&gt;My data = &lt;code&gt;data&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Model   A1BG    A1CF    A2M A2ML1   A3GALT2 A4GALT  foldchange&#xA;IM00499 N   L   L   G   N   N   0.285744&#xA;IM00499.1   L   N   D   G   D   D   0.005046&#xA;IM00980 N   N   N   N   N   N   -0.068949&#xA;IM01529 N   N   L   N   N   N   0.065914&#xA;IM01624 A   N   L   N   N   N   0.302100&#xA;IM01240 N   D   N   D   N   G   -0.428324&#xA;IM00807 N   L   A   N   D   N   0.257222&#xA;IM00999 N   N   N   N   N   D   0.056268&#xA;IM00983 N   N   N   N   N   N   0.107456&#xA;IM01076 N   N   D   A   N   N   0.218301&#xA;IM00803 N   D   N   N   N   D   -0.479368&#xA;IM00125 N   N   A   N   N   A   0.270428&#xA;IM00998 D   N   N   N   G   N   0.142748&#xA;IM00997 N   L   G   G   N   G   -0.154540&#xA;IM00996 N   L   N   N   N   L   -1.302023&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above data frame is divided into 5 categories = A, N, L, G and D&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;feature = ['A1BG', 'A1CF', 'A2M', 'A2ML1', 'A3GALT2', 'A4GALT']&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Feature_level_partitions are arrays of combination of classification values&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;feature_level_partitions = ['A', 'N' , 'L',  'G',  'D'], ['D'], ['L', 'G']&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I was using the below &lt;code&gt;list comprehension&lt;/code&gt; to calculate &lt;code&gt;Hedge's G&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;        hedges_g = np.array(&#xA;            [&#xA;                Hedges_g(data[feature].isin(feature_level_partition), y)&#xA;                for feature_level_partition in feature_level_partitions&#xA;            ],&#xA;            dtype=np.float64,&#xA;        )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;Hedges_g&lt;/code&gt; is defined as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def Hedges_g(x, y):&#xA;    difference_of_means = abs(np.mean(y[x])[0] - np.mean(y[~x])[0])&#xA;    s_pooled_weighted = math.sqrt((((n1 - 1) * s1 ** 2) + ((n2 - 1) * s2 ** 2)) / (n1 + n2 - 2))&#xA;    return difference_of_means / s_pooled_weighted&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am interested in knowing how many models there are within each category. So for example if I divide up the data into [&amp;quot;D&amp;quot;,&amp;quot;L&amp;quot;] on one side, and the rest on the other, I want to know how many models have either &amp;quot;D&amp;quot; OR &amp;quot;L&amp;quot; and how many models have any of the other categories and then don't calculate &lt;code&gt;Hegde's G&lt;/code&gt; if the number on either side is less than 2.&lt;/p&gt;&#xA;&lt;p&gt;To answer the above query I tried the following code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[data[feature].isin(feature_level_partition).sum()&#xA;    for feature_level_partition in feature_level_partitions]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above code gave me the number of models with the &lt;code&gt;feature_level_partitions&lt;/code&gt;&#xA;However I also want to know the number of models without the &lt;code&gt;feature_level_partitions&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I also want to create a python function which calculates both above numbers in one line and fit this code as a &lt;code&gt;if statement&lt;/code&gt; in my above &lt;code&gt;list comprehension&lt;/code&gt; function&lt;/p&gt;&#xA;" OwnerUserId="4783" LastEditorUserId="73" LastEditDate="2023-02-15T19:31:19.680" LastActivityDate="2023-02-18T18:16:48.997" Title="Python list comprehension to calculate the set of categories for each row" Tags="|python|pandas|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20573" PostTypeId="1" CreationDate="2023-02-15T03:55:23.307" Score="2" ViewCount="120" Body="&lt;p&gt;Recently I have ran a human WGS on the BGI DNBSEQ system, and their FASTQ quality scores seem to be quite impressive, where the Phred scores barely deteriorate along the read length when checked on FastQC.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/okf48.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/okf48.png&quot; alt=&quot;DNBSEQ_fastqc&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since the DNBSEQ technology is relatively new to me, I wonder if there's any bioinformatic approach to test whether the Phred scores listed in these FASTQ files are authentic or not? I'm kind of worried that their sequencer would just falsely assign high Phred scores to all bases.&lt;/p&gt;&#xA;" OwnerUserId="14961" LastActivityDate="2023-02-16T02:24:54.417" Title="Can we test the accuracy of Phred scores shown in FASTQ files" Tags="|fastq|wgs|fastqc|phred-scores|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20574" PostTypeId="2" ParentId="18795" CreationDate="2023-02-15T08:41:19.913" Score="1" Body="&lt;p&gt;You need to pass the fasta file instead of fa.fai or dict file as reference&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    gatk Funcotator \&#xA;    --variant chr21.vcf \&#xA;    --reference hg38.fa \  &#xA;    --ref-version hg38 \&#xA;    --data-sources-path funcotator_dataSources.v1.2.20180329 \&#xA;    --output chr21.funcotated.maf \&#xA;    --output-file-format MAF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The directory where you have hg38.fa must contain also the hg38.fa.fai and hg38.dict&lt;/p&gt;&#xA;&lt;p&gt;Hope this helps&lt;/p&gt;&#xA;&lt;p&gt;Zhan&lt;/p&gt;&#xA;" OwnerUserId="17011" LastActivityDate="2023-02-15T08:41:19.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20575" PostTypeId="2" ParentId="20573" CreationDate="2023-02-15T19:09:41.380" Score="3" Body="&lt;p&gt;Yes, reported accuracy can be tested and compared with actual accuracy by mapping reads from a reference sample to a known-perfect reference sequence. It's usually done in aggregate, at a read level rather than at single-base level.&lt;/p&gt;&#xA;&lt;p&gt;I like using Heng Li's &lt;a href=&quot;http://lh3.github.io/2018/11/25/on-the-definition-of-sequence-identity&quot; rel=&quot;nofollow noreferrer&quot;&gt;gap-compressed identity statistic&lt;/a&gt; for this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Y1V6J.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Y1V6J.jpg&quot; alt=&quot;mean base quality vs gap-compressed identity for Nanopore reads&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, fish out a few reads at random, and use BLAST, which reports an identity score.&lt;/p&gt;&#xA;&lt;p&gt;If the sample is not a reference sample with a precisely known sequence, then this approach will not work, because introduced variation will confound the comparison.&lt;/p&gt;&#xA;&lt;p&gt;While it might be possible to create a new reference sequence from the reads, this introduces the possibility of systematic error in the sequencer that would be hidden by the new reference sequence, and when you're looking at error rates in the realm of q30 and above (as in your plot), those small differences are going to matter a lot.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-02-16T02:24:54.417" LastActivityDate="2023-02-16T02:24:54.417" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20577" PostTypeId="2" ParentId="20564" CreationDate="2023-02-15T19:39:00.733" Score="1" Body="&lt;p&gt;If your cell types and sample names are in separate metadata variables attached to the Seurat object, then you can use &lt;code&gt;table&lt;/code&gt; to count up the pairings:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;table(pbmc.seurat&lt;span class=&quot;math-container&quot;&gt;$singler_labels, pbmc.seurat$&lt;/span&gt;sample_name)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will output a matrix containing these counts&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-02-15T19:39:00.733" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20579" PostTypeId="1" CreationDate="2023-02-16T08:00:04.107" Score="2" ViewCount="45" Body="&lt;p&gt;I seem to have two groups of samples based on hierarchical clustering as well as MDS plot analysis using the sample label from the tree cut of the hierarchial clustering. I am analyzing microarray data. Should I adjust for the two different groups of samples in the model? For context, my covariate of interest is a continuous measure (air pollutant measure). This is a human cohort and not a case-control model and hence I was wondering if I should adjust for the different groups of samples.&#xA;[&lt;img src=&quot;https://i.stack.imgur.com/20fhm.jpg&quot; alt=&quot;Hierarchial clustering][1]&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;MDS plot is &lt;a href=&quot;https://i.stack.imgur.com/O2Jch.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for the same samples:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/VuE7b.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VuE7b.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4307" LastEditorUserId="3848" LastEditDate="2023-08-31T18:53:54.617" LastActivityDate="2023-08-31T18:53:54.617" Title="Should I compensate for two clusters of samples in microarray by hierarchical clustering and MDS?" Tags="|phylogenetics|phylogeny|microarray|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20581" PostTypeId="2" ParentId="20579" CreationDate="2023-02-16T10:08:19.757" Score="1" Body="&lt;p&gt;Very interesting, if I understand correctly you are asking whether a straight hierarchical clustering should account for a continuous variable because it is resulting in discrete groups (clusters) which seems contradictory for a continuous variable?&lt;/p&gt;&#xA;&lt;p&gt;If that is the question the answer is strictly no. Do not use weighting for &lt;em&gt;this particular&lt;/em&gt; analysis (other regression style analysis yes - but not this one). Thus 'weights' are for regression analysis not for multivariate statistics. This clustering analysis is about assessing the correlation and not trying to impose it - thats regression.&lt;/p&gt;&#xA;&lt;p&gt;The 'weighting' that can be used is different for a tree based analysis - it accounts for rate variation within the expression data - you are assuming rate homogeneity. That doesn't always work - but it's complicated and your data looks good as is.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, I would suggest your have 4 clusters. What you want to do is map the continuous trait across all branches 'tips' of your clustering. This will break up your continuous variable (hopefully) and looking at the MDS very much into discrete groups. That is the result, in fact it may not be 2 clusters but 4 clusters - map all the 'tips' against the continuous variable. There're statistically randomisation tree methods to assess the robustness of the observation - you can also use bootstrapping.&lt;/p&gt;&#xA;&lt;p&gt;The question is whether these discrete groups are in fact the best description of the data. The MDS suggests that the continuous variable &amp;quot;air pollution&amp;quot; is in continuous within gene expression - using a combination of clustering and MDS you highlight this facet of the data. There alternative analyses namely regression or GLM to assess the relationship between the continuous variable and your results - the MDS is suggestive of that. However, it's not a 'and', 'or', 'right', 'wrong' relationship a continuous variable can be broken up into discrete groups: if there is a 1:1 relationship between the gene expression and the continuous variable &amp;quot;air pollution&amp;quot; (MDS analysis) - that is a better description, if that relationship doesn't hold breaking into discrete clusters is the best description &lt;em&gt;thus far&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Essentially this is statistical modelling and the question is what is the best model to fit all the data. I'm personally not a fan of hierarchical clustering as a tree description of the data - but in your data it looks good.&lt;/p&gt;&#xA;&lt;p&gt;At the heart of the question you are asking whether there are boundary effects in the data as described by the hierarchical clustering? Thus is there a certain level of pollution (or whatever) causing one gene expression boundary and then another and so on ... Perfectly possible - but central to that hypothesis is whether clustering is the best description of the data rather than a regression analysis, i.e. is it continuous or is it discrete.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Discrete vs. continuous&lt;/strong&gt;  If it is discrete then k-means clustering would be a really, really good follow-up analysis to confirm the discrete &amp;quot;boundary&amp;quot; effects within the data. GLM is an analysis that might trap the data if it's continuous, there a lot of other methods in regression analysis. Basically if you map each &amp;quot;pollution&amp;quot; exposure against the MDS and if it's a perfect fit - trapping the result in a regression model outside eigenvalues would be the traditional thing to do. However, these days &amp;quot;unsupervised learning&amp;quot; (of which MDS is one method) is trendy so it may not be needed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;k-means clustering&lt;/strong&gt; I would certainly consider k-means clustering in any case - this is because hierarchical clustering makes a rate assumption which might smash up the discrete groups. K-means can capture that without getting into complicated tree analysis. Basically, k-means you'll set 2 groups - assess the fit of the continuous variable, try 3 and then 4, how does the air-pollution variable fit? The data will also work well with supervised machine learning. However, it is genuinely continuous then regression analysis is best approach. With k-means you can also do parameterisation, i.e. whats the optimal number of groups represented by the data: if it comes back with '4' - which neatly segregate &amp;quot;air pollution&amp;quot; thats really good.&lt;/p&gt;&#xA;&lt;p&gt;It's an exciting result. The way you write the result in the context of the conclusion is really important - i.e. the continuous verse discrete best fit analysis - in case you encounter a statistical modeller in peer-review.&lt;/p&gt;&#xA;&lt;p&gt;My answer is a bit waffly, but is a representation of statistical modelling in this particular data, because two approaches may both be appropriate, but one is a better description than the other.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;, if I've misunderstood the question and some sort of standardisation is needed between experiments - thats a different answer. Thus if you are saying that the groups represent different experiments, e.g. performed on different days or different arrays and there's risk of experimental artefact: thats about standardisation and the answer would be absolutely. Standardisation is essential to rule out experimental bias - particularly in microarray data.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-02-16T16:09:22.400" LastActivityDate="2023-02-16T16:09:22.400" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20582" PostTypeId="1" AcceptedAnswerId="20586" CreationDate="2023-02-16T15:26:14.133" Score="2" ViewCount="84" Body="&lt;p&gt;I am using the &lt;a href=&quot;https://dataguide.nlm.nih.gov/edirect/xtract.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;xtract tool&lt;/a&gt; of the Entrez Direct (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;EDirect&lt;/a&gt;) package of NBCI. I have a list of Accession Numbers:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;U47804, U47803, U47802, U47801, U47800, U47799, U47798, X92938&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and try to get information about:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Name of organism (Org-ref_taxname tag)&lt;/li&gt;&#xA;&lt;li&gt;Product of gene (Prot-ref_name_E tag)&lt;/li&gt;&#xA;&lt;li&gt;Public year (Date-std_year tag)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I executed the following command:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;for i in {U47804 U47803 U47802 U47801 U47800 U47799 U47798 X92938}&#xA;do&#xA;  efetch -db &amp;quot;nucleotide&amp;quot; -id &amp;quot;$i&amp;quot; -format full; done |&#xA;    xtract -pattern Seq-entry -element Org-ref_taxname \&#xA;      -block Prot-ref_name -def &amp;quot;-&amp;quot; -element Prot-ref_name_E \&#xA;      -block Date-std -position first -element Date-std_year&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The result I got:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Human immunodeficiency virus 1  gp120   1996&#xA;Human immunodeficiency virus 1  gp120   1996&#xA;Human immunodeficiency virus 1  gp120   1996&#xA;Human immunodeficiency virus 1  gp120   1996&#xA;Human immunodeficiency virus 1  gp120   1996&#xA;Human immunodeficiency virus 1  gp120   1996&#xA;Human immunodeficiency virus 1  gp120   1996&#xA;Human immunodeficiency virus 1  1995&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Because the &lt;strong&gt;X92938&lt;/strong&gt; doesn't have &lt;em&gt;&amp;lt;Prot-ref_name&amp;gt;&lt;/em&gt;, so I was expecting to replace the missing fields with &lt;strong&gt;&amp;quot;-&amp;quot;&lt;/strong&gt; by the &lt;code&gt;-def&lt;/code&gt; tag of the extract tool. It means my desired result is as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Human immunodeficiency virus 1  gp120   1996&#xA;Human immunodeficiency virus 1  gp120   1996&#xA;Human immunodeficiency virus 1  gp120   1996&#xA;Human immunodeficiency virus 1  gp120   1996&#xA;Human immunodeficiency virus 1  gp120   1996&#xA;Human immunodeficiency virus 1  gp120   1996&#xA;Human immunodeficiency virus 1  gp120   1996&#xA;Human immunodeficiency virus 1    -     1995&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can someone have experience with the NBCI tool may help me? Thanks a lot.&lt;/p&gt;&#xA;" OwnerUserId="17025" LastEditorUserId="17025" LastEditDate="2023-02-17T12:01:44.880" LastActivityDate="2023-02-18T08:53:58.943" Title="Default placeholder for missing fields doesn't work with block in xtract tool" Tags="|bash|edirect|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20583" PostTypeId="1" CreationDate="2023-02-16T16:16:08.860" Score="-2" ViewCount="124" Body="&lt;p&gt;I have performed genomic sequencing using nanopore technology and upon analysing using BLAST in the command line I have identified the presence of E.coli. I was wondering if I can add some code to identify the known virulence genes of E.coli which are on the NCBI BLAST database? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="17027" LastActivityDate="2023-02-16T18:12:27.560" Title="How can I identify the virulence factors in my FASTQ files generated from nanopore sequencing bacteria DNA?" Tags="|genomics|blast|bacteria|command-line|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20584" PostTypeId="2" ParentId="20583" CreationDate="2023-02-16T18:04:02.840" Score="1" Body="&lt;p&gt;you could try to submit your sequences to &lt;a href=&quot;http://www.mgc.ac.cn/cgi-bin/VFs/v5/main.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;VFDB: a reference database for bacterial virulence factors&lt;/a&gt; ; paper here : &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC539962/&quot; rel=&quot;nofollow noreferrer&quot;&gt;reference paper&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;use the VFanalyzer tool&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;VFanalyzer provides an automatic analysis pipeline for a systematic screen of known/potential VFs in given complete/draft bacterial genomes. Users can perform the analysis by one of the publicly available bacterial genomes either from our pre-downloaded genome list (Option 1) or via any valid GenBank accession(s) (Option 2). Or alternatively, users can also analyse any private genome data (raw FASTA sequence, predicted proteins, or annotated GenBank format genome) using Option 3. For query with raw FASTA genome, predicted CDSs will be identified using GLIMMER3 prior to the run of VFanalyzer.&#xA;Please note that, VFanalyzer only accepts query genomes from the genera of bacterial pathogens with full information in VFDB, and it's designed for complete or nearly complete draft genomes. So you are at your own risk if the query genome are derived from other bacteria or only partially sequenced.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;NOTE: Due to connetion issue with NCBI, VFanalyzer can not fetch gbk file using Edirect correctly. Please download the file manually and use option3 to submit your jobs, we will find the other way to fix the problem ASAP.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Think you need to convert your FASTQ to FASTA first.&lt;/p&gt;&#xA;&lt;p&gt;Other resource is : &lt;a href=&quot;https://cge.food.dtu.dk/services/VirulenceFinder/&quot; rel=&quot;nofollow noreferrer&quot;&gt;VirulenceFinder 2.0&lt;/a&gt; or &lt;a href=&quot;https://cge.food.dtu.dk/services/PathogenFinder/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PathogenFinder 1.1&lt;/a&gt;;  paper ref : &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7512150/&quot; rel=&quot;nofollow noreferrer&quot;&gt;In Silico Genotyping of Escherichia coli Isolates for Extraintestinal Virulence Genes by Use of Whole-Genome Sequencing Data&lt;/a&gt;, &lt;a href=&quot;https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0077302&quot; rel=&quot;nofollow noreferrer&quot;&gt;PathogenFinder - Distinguishing Friend from Foe Using Bacterial Whole Genome Sequence Data&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2023-02-16T18:12:27.560" LastActivityDate="2023-02-16T18:12:27.560" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20585" PostTypeId="2" ParentId="20155" CreationDate="2023-02-17T02:03:07.523" Score="0" Body="&lt;p&gt;Actually, precompiled SHAPEIT 4.2.2 is available in &lt;a href=&quot;https://launchpad.net/ubuntu/+source/shapeit4&quot; rel=&quot;nofollow noreferrer&quot;&gt;a Ubuntu repository&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sudo apt install shapeit4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14597" LastEditorUserId="14597" LastEditDate="2023-02-17T12:19:41.890" LastActivityDate="2023-02-17T12:19:41.890" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20586" PostTypeId="2" ParentId="20582" CreationDate="2023-02-17T11:59:41.730" Score="2" Body="&lt;p&gt;After trying myself in many ways, I have found that the method of creating a checkmark column when a desired column is recorded is more efficient and easier to understand.&lt;/p&gt;&#xA;&lt;p&gt;Here is my bash script to solve my problem in a verbose way (I guess has others way shorter).&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/bash&#xA;&#xA;change_index(){&#xA;    local in=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$(&amp;lt; /dev/stdin)&quot; # Variable assignment from the stdin&#xA;    local col_num=$&lt;/span&gt;(head -n 1 &amp;lt;(echo &amp;quot;$in&amp;quot;) |awk -F&amp;quot;\t&amp;quot; '{print -1+NF/2}')  # Number of fields&#xA;&#xA;    &#xA;    for k in &lt;span class=&quot;math-container&quot;&gt;$(seq 0 $&lt;/span&gt;((&lt;span class=&quot;math-container&quot;&gt;$col_num-1))); do # Loop through number of field&#xA;        in=$&lt;/span&gt;(awk -v ia=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$k&quot; -F&quot;\t&quot;  '{sub(&quot;/[^&quot;ia&quot;]&quot;,&quot;/&quot;ia&quot;\t-\t/&quot;((ia+1)),$(2*ia+1)) ;print $0}' &amp;lt;(echo &quot;$&lt;/span&gt;in&amp;quot;)) # Supplement missing fields&#xA;    done&#xA;    &#xA;    in=&lt;span class=&quot;math-container&quot;&gt;$(awk -v ia=&quot;$&lt;/span&gt;col_num&amp;quot; -F&amp;quot;\t&amp;quot;  '{sub(&amp;quot;/[^&amp;quot;ia&amp;quot;]&amp;quot;,&amp;quot;/&amp;quot;ia&amp;quot;\t-&amp;quot;,&lt;span class=&quot;math-container&quot;&gt;$(2*ia+1)) ;print $&lt;/span&gt;0}' &amp;lt;(echo &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$in&quot;)) # Supplement missing the last fields&#xA;    in=$&lt;/span&gt;(sed 's/\/[[:digit:]]\+//g' &amp;lt;(echo &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$in&quot;)) # Remove markers&#xA;    echo &quot;$&lt;/span&gt;in&amp;quot;&#xA;}&#xA;&#xA;for i in {U47804 U47803 U47802 U47801 U47800 U47799 U47798 X92938}; do \&#xA;    efetch -db &amp;quot;nucleotide&amp;quot; -id &amp;quot;$i&amp;quot; -format full; done |    &#xA;    xtract -pattern Seq-entry \&#xA;        -lbl &amp;quot;/0&amp;quot; -element Org-ref_taxname \&#xA;        -block Prot-ref_name -lbl &amp;quot;/1&amp;quot; -element Prot-ref_name_E \&#xA;        -block Date-std -position first -lbl &amp;quot;/2&amp;quot; -element Date-std_year |&#xA;    change_index| column -t&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Explanation&lt;/h2&gt;&#xA;&lt;p&gt;The main idea is that if a field is recorded there will be a marker field lay before the recorded field. This action is done by &lt;code&gt;-lbl&lt;/code&gt; tag. And we have the following draft table:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996&#xA;/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996&#xA;/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996&#xA;/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996&#xA;/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996&#xA;/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996&#xA;/0  Human  immunodeficiency  virus  1  /1  gp120  /2  1996&#xA;/0  Human  immunodeficiency  virus  1  /2  1995       &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After having a draft table, and cleaning the table by &lt;code&gt;change_index()&lt;/code&gt; function I wrote.&lt;/p&gt;&#xA;" OwnerUserId="17025" LastEditorUserId="17025" LastEditDate="2023-02-18T08:53:58.943" LastActivityDate="2023-02-18T08:53:58.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20587" PostTypeId="1" CreationDate="2023-02-18T05:52:27.880" Score="1" ViewCount="584" Body="&lt;p&gt;I am at this point in the GitHub tutorial: &lt;a href=&quot;https://github.com/DerrickWood/kraken2/blob/master/docs/MANUAL.markdown#standard-kraken-2-database&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/DerrickWood/kraken2/blob/master/docs/MANUAL.markdown#standard-kraken-2-database&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It says to create the standard Kraken database, I use the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;kraken2-build --standard --db $DBNAME&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It then says to change &lt;code&gt;$DBNAME&lt;/code&gt; with the preferred database name/location. Is this something that I need to download on my own? Where can I find it, and how do I do it?&lt;/p&gt;&#xA;&lt;p&gt;To provide context, I want to use Kraken2 to identify whether or not there is the presence of MCPyV in a tumor sample.&lt;/p&gt;&#xA;" OwnerUserId="17033" LastActivityDate="2023-02-18T19:54:56.220" Title="How do I build the &quot;Standard Kraken 2 Database&quot;?" Tags="|ngs|linux|metagenome|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20589" PostTypeId="1" CreationDate="2023-02-18T15:50:17.190" Score="1" ViewCount="161" Body="&lt;p&gt;I am attempting to build just the standard Kraken2 Database by using the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;kraken2-build --standard --threads 24 --db $DBNAME&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am returned the following error a few minutes building it:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rsync_from_ncbi.pl: unexpected FTP path (new server?) for &#xA;https://ftp.ncbi.nlm.nih.gov/genomes/all/GCF/000/762/265/GCF_000762265.1_ASM76226v1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Anyone know why this is happening?&lt;/p&gt;&#xA;" OwnerUserId="16949" LastActivityDate="2023-10-01T06:57:42.487" Title="Kraken2 Standard Database failing to build (unexpected FTP path)" Tags="|ngs|metagenome|ncbi|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20590" PostTypeId="1" CreationDate="2023-02-18T16:42:39.580" Score="0" ViewCount="121" Body="&lt;p&gt;I am attempting to build my own custom database for Kraken2. I have two questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If I have the MCPyV genome in a file called MCPyV.fasta, how do I build a database with just this?&lt;/li&gt;&#xA;&lt;li&gt;How do I build the MinusB database? (&lt;a href=&quot;https://benlangmead.github.io/aws-indexes/k2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://benlangmead.github.io/aws-indexes/k2&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;For number one, I attempted to build this by using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;kraken2-build --add-to-library MCPyV.fasta --db my_db&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But was returned an error stating that there was no taxonomy information found. How do I make this MCPyV.fasta file amenable to Kraken2 while building the database?&lt;/p&gt;&#xA;&lt;p&gt;For number 2, I again attempted to build the MinusB database using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;kraken2-build --build --threads 4 --db MinusB&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and was returned an error regarding missing /taxonomy directory.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know how to build these databases properly?&lt;/p&gt;&#xA;" OwnerUserId="16949" LastActivityDate="2023-07-20T16:04:36.370" Title="How do I build the MinusB database for Kraken2? (Taxonomy issues)" Tags="|sequence-alignment|ngs|sequence-analysis|metagenome|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20591" PostTypeId="2" ParentId="20571" CreationDate="2023-02-18T18:05:16.127" Score="1" Body="&lt;p&gt;What I'm not clear about is the definition of &lt;code&gt;y&lt;/code&gt; - thats crucial to understanding what the code does. Just to note, although you've called the dataframe &lt;code&gt;data&lt;/code&gt;, I've called it &lt;code&gt;df&lt;/code&gt; (traditional shorthand). In terms of speed there will not be much in it between vectorised &lt;code&gt;pandas&lt;/code&gt; and &lt;code&gt;numpy&lt;/code&gt;. Explicitly using &lt;code&gt;iterrows&lt;/code&gt; or else &lt;code&gt;apply&lt;/code&gt; is when it slows down a lot.&lt;/p&gt;&#xA;&lt;p&gt;Possibilities of the definition of &lt;code&gt;y&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;1.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;y = df.drop(['Model', 'foldchange'], axis=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I think the above is more likely, or ...&lt;/p&gt;&#xA;&lt;p&gt;2.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;y = df['fold change']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What I think is occurring is &lt;code&gt;np.mean&lt;/code&gt; is performing a boolean operation on the feature columns. What I didn't understand though was this ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;abs(np.mean(y[x])[0] - np.mean(y[~x])[0])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;.. this is because for example model 'IM00980' albeit the mean is 0 this will always be 1 under the above equation. This is because &lt;code&gt;y[~x]&lt;/code&gt; will be 1 for all values and thus abs(0 - 1) is 1. Equally if all columns in a row were 1 the output would also be 1 because abs(1-0) is 1. Anyway, what I would do is explicitly perform the operation within &lt;code&gt;pandas&lt;/code&gt; rather than coercing &lt;code&gt;np&lt;/code&gt; into a boolean operation. This is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;df = pd.read_csv(r'/pathtofile/megha.csv', delimiter=&amp;quot;\t&amp;quot;)&#xA;&#xA;feature = ['A1BG', 'A1CF', 'A2M', 'A2ML1', 'A3GALT2', 'A4GALT']&#xA;df=df.drop(['foldchange'],axis=1)&#xA;feature_level_partitions = ['L', 'G']&#xA;filtered = df[feature].isin(feature_level_partitions).eq(True).mul(1)&#xA;df2 = df['Model'].to_frame().join(filtered)&#xA;df2['mean'] = df2[feature].mean(axis=1)&#xA;print(df2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here ~x would simply &lt;code&gt;1 - mean&lt;/code&gt; as a new pandas column, i.e. &lt;code&gt;df2['~mean'] = df2['mean'] - 1&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;The output of that code is:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;-&lt;/th&gt;&#xA;&lt;th&gt;Model&lt;/th&gt;&#xA;&lt;th&gt;A1BG&lt;/th&gt;&#xA;&lt;th&gt;A1CF&lt;/th&gt;&#xA;&lt;th&gt;A2M&lt;/th&gt;&#xA;&lt;th&gt;A2ML1&lt;/th&gt;&#xA;&lt;th&gt;A3GALT2&lt;/th&gt;&#xA;&lt;th&gt;A4GALT&lt;/th&gt;&#xA;&lt;th&gt;mean.&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;IM00499&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;1.&lt;/td&gt;&#xA;&lt;td&gt;1.&lt;/td&gt;&#xA;&lt;td&gt;1.&lt;/td&gt;&#xA;&lt;td&gt;0.&lt;/td&gt;&#xA;&lt;td&gt;0.&lt;/td&gt;&#xA;&lt;td&gt;0.500000&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;IM00499.1&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0.333333&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;td&gt;IM00980&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0.000000&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;3&lt;/td&gt;&#xA;&lt;td&gt;IM01529&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0.166667&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;4&lt;/td&gt;&#xA;&lt;td&gt;IM01624&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0.166667&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;5&lt;/td&gt;&#xA;&lt;td&gt;IM01240&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;0.166667&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;6&lt;/td&gt;&#xA;&lt;td&gt;IM00807&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0.166667&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;7&lt;/td&gt;&#xA;&lt;td&gt;IM00999&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0.000000&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;8&lt;/td&gt;&#xA;&lt;td&gt;IM00983&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0.000000&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;IM01076&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0.000000&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;10&lt;/td&gt;&#xA;&lt;td&gt;IM00803&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0.000000&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;11&lt;/td&gt;&#xA;&lt;td&gt;IM00125&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0.000000&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;12&lt;/td&gt;&#xA;&lt;td&gt;IM00998&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0.166667&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;13&lt;/td&gt;&#xA;&lt;td&gt;IM00997&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;0.666667&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;14&lt;/td&gt;&#xA;&lt;td&gt;IM00996&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;0.333333&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&lt;hr /&gt;&#xA;&lt;p&gt;If this is correct it provides the basis to move forward for all other operations. Basically, what I'm saying is I think is good to break open the &lt;code&gt;numpy&lt;/code&gt; into explicit &lt;code&gt;pandas&lt;/code&gt; operations, what is happening is there's a mix of &lt;code&gt;pandas&lt;/code&gt; and &lt;code&gt;numpy&lt;/code&gt; and I would recommend going down one route or the other.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-02-18T18:16:48.997" LastActivityDate="2023-02-18T18:16:48.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20592" PostTypeId="2" ParentId="20587" CreationDate="2023-02-18T19:54:56.220" Score="2" Body="&lt;p&gt;\$DBNAME is a folder location on your computer; you specify it yourself, for example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;kraken2-build --standard --db /media/ip80/externalDrive/kraken_indexes/betterThanGenbank&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-02-18T19:54:56.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20593" PostTypeId="1" CreationDate="2023-02-18T20:55:15.273" Score="0" ViewCount="97" Body="&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/9554899/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;&lt;strong&gt;Cross-posted on biostars&lt;/strong&gt;&lt;/em&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have trio_example.vcf file and I want to split them into male and female groups. I tried this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ bcftools query -l trio_example.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And it gave me:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;N1&#xA;N2&#xA;N3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After that I tried this command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for sample in `bcftools query -l trio_example.vcf.gz`; do vcf-subset --exclude-ref -c &lt;span class=&quot;math-container&quot;&gt;$sample trio_example.vcf.gz &amp;gt; $&lt;/span&gt;{sample}.vcf; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to split 3 samples. Now its showing same data in all 3 N files&lt;/p&gt;&#xA;&lt;p&gt;How can I solve this issue? Also I need to split the data into male and female. For splitting I am using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools query -f '%CHROM\t%POS\t%REF\t%ALT\t%AF\n' trio_example.vcf | awk '{if(&lt;span class=&quot;math-container&quot;&gt;$1 == &quot;X&quot; &amp;amp;&amp;amp; $&lt;/span&gt;5 &amp;gt;= 0.3 &amp;amp;&amp;amp; &lt;span class=&quot;math-container&quot;&gt;$5 &amp;lt;= 0.7) {print $6 &quot; is male&quot;} else if ($&lt;/span&gt;1 == &amp;quot;Y&amp;quot; &amp;amp;&amp;amp; &lt;span class=&quot;math-container&quot;&gt;$5 == 1) {print $6 &quot; is male&quot;} else if ($&lt;/span&gt;1 == &amp;quot;X&amp;quot; &amp;amp;&amp;amp; &lt;span class=&quot;math-container&quot;&gt;$5 == 1) {print $6 &quot; is female&quot;} else {print $&lt;/span&gt;6 &amp;quot; is of unknown sex&amp;quot;}}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But it also not give me exact result&lt;/p&gt;&#xA;" OwnerUserId="9269" LastEditorUserId="650" LastEditDate="2023-02-21T15:08:43.840" LastActivityDate="2023-02-21T15:08:43.840" Title="How to Split VCF File into male and Female groups" Tags="|vcf|shell|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="20594" PostTypeId="1" CreationDate="2023-02-19T10:39:35.557" Score="0" ViewCount="31" Body="&lt;p&gt;I want to use maxent to make some future distribution predictions. I downloaded future climate data from worldclim. In worldclim's page there are three variables (max temp, min temp and precipitation) as well as a fourth titled &amp;quot;bioclimatic  variables&amp;quot; which mentions that it contains the other three and some other new ones. My question is: if i download all four and use them for my analysis, will maxent produce reliable results? Or would the duplicate variables be a problem?&lt;/p&gt;&#xA;" OwnerUserId="16657" LastEditorUserId="2108" LastEditDate="2023-02-21T22:24:55.427" LastActivityDate="2023-02-22T22:45:57.833" Title="Would duplicate climatic variables impact maxent?" Tags="|modelling|ecology|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20596" PostTypeId="1" AcceptedAnswerId="20606" CreationDate="2023-02-20T11:26:34.073" Score="-1" ViewCount="137" Body="&lt;p&gt;I have a txt file (50k) proteins in it, which looks like this:&lt;/p&gt;&#xA;&lt;p&gt;line 1:the protein code&lt;/p&gt;&#xA;&lt;p&gt;line 2: protein length in amino acids&lt;/p&gt;&#xA;&lt;p&gt;line 3: amino acid sequence&lt;/p&gt;&#xA;&lt;p&gt;line 4: secondary structure&lt;/p&gt;&#xA;&lt;p&gt;line 5: XYZ coordinates(which is numbers).&lt;/p&gt;&#xA;&lt;p&gt;line 6:embty&lt;/p&gt;&#xA;&lt;p&gt;the input looks like this :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;4LGTD&#xA;247&#xA;M       S       E       K       L       Q       K       V       L       A       R       A       G       H       G       S       R   T&#xA;.       .       E       E       H       H       H       H       H       H       H       T       T       S       S       .       H   .&#xA;-34.3 -28.9 16.3        -33.0 -26.8 13.4        -29.4 -25.4 13.3        -27.9 -22.6 11.2        -26.9 -24.1 7.8 -23.3 -22.9 8.2 -23.0 -24.7 11.5     -24.1 -27.9 9.9 -21.8 -27.7 6.9 -18.8 -26.7 9.0 -19.5 -29.7 11.3        -19.6 -31.9 8.2 -16.2 -30.7 7.2 -16.7 -28.2 4.4      -15.0 -25.1 5.8 -15.4 -22.7 8.7 -18.8 -21.3 9.7 -17.7 -17.9 8.3 -16.3 -19.3 5.1 -19.5 -21.2 4.5 -21.5 -18.0 5.1 -19.7 -16.2 2.3      -20.8 -19.1 -0.0        -24.4 -19.2 1.2 -24.6 -15.4 0.6 -23.5 -16.0 -3.0        -26.4 -18.4 -3.6        -24.1 -21.4 -4.0    -25.9 -23.6 -1.5 -29.3 -25.2 -2.0        -31.4 -27.0 0.5 -34.0 -29.5 -0.8        -33.9 -27.8 -4.2        -34.1 -24.2 -3.0        -31.3 -21.7 -2.9     -30.7 -21.0 0.8 -30.2 -17.5 2.2 -27.9 -16.3 4.9 -29.4 -17.1 8.3 -31.3 -20.1 7.0 -31.7 -23.1 9.4 -32.1 -26.8 8.7 -32.6 -30.1 10.5     -30.2 -32.9 9.7 -32.0 -36.0 8.4 -31.6 -38.7 5.7 -34.0 -36.6 3.6 -32.0 -33.4 3.7 -30.4 -32.7 0.4 -27.7 -30.0 0.6 -25.8 -29.0 -2.5     -23.0 -26.6 -3.0        -22.5 -25.6 -6.6        -25.0 -28.3 -7.5        -22.9 -31.1 -5.9        -24.6 -33.1 -3.2    -23.0 -33.1 0.2  -23.1 -36.0 2.6 -23.7 -34.9 6.2 -23.8 -36.9 9.4 -27.2 -37.3 11.1        -28.5 -37.1 14.7        -19.2 -38.7 17.8    -17.7 -36.6 20.6 -14.0 -37.3 20.3        -11.2 -35.4 22.0        -7.4 -35.3 21.2 -4.7 -34.3 23.6 -1.9 -32.6 21.6 1.6 -31.6 22.7  2.2 -28.4 20.7       5.8 -27.5 20.1  7.1 -23.9 19.9  7.1 -22.6 16.3  3.6 -24.0 15.4  0.7 -21.6 14.7  -2.8 -22.2 16.0 -4.8 -21.4 12.8 -6.4 -23.3 9.9       -4.8 -20.9 7.5  -1.6 -18.8 8.1  -1.1 -16.0 5.5  2.7 -16.4 6.0   3.0 -20.1 5.2   4.2 -21.3 8.7   3.5 -24.7 10.3  0.1 -24.9 11.9       -1.1 -27.3 14.7 -3.6 -28.9 12.3 -1.0 -30.0 9.8  -0.7 -33.4 11.5 -4.0 -34.7 12.9 -6.1 -37.9 12.3 -8.5 -37.7 9.3  -12.2 -38.1 10.1     -15.3 -39.0 8.2 -17.5 -36.0 7.6 -16.2 -34.0 10.6        -13.2 -31.8 11.6        -10.9 -31.0 14.5        -11.7 -27.8 16.4     -8.8 -26.3 18.5 -9.8 -25.6 22.1 -8.4 -22.1 22.1 -5.0 -20.9 20.9 -1.5 -21.1 22.2 1.2 -18.6 21.2  3.8 -19.7 18.6  6.6 -20.0 21.1       4.5 -21.3 23.9  4.7 -25.1 24.5  2.3 -27.7 25.8  -1.3 -29.0 25.9 -3.8 -28.1 23.2 -7.0 -30.0 23.1 -8.9 -30.7 19.9 -12.5 -32.0 19.6     -14.9 -32.8 16.9        -17.6 -31.5 19.2        -17.9 -27.7 18.8        -20.2 -27.3 21.8        -17.6 -29.0 24.0        -15.0 -26.8 22.4     -16.8 -23.5 23.1        -17.7 -24.6 26.6        -14.1 -25.3 27.4        -12.9 -21.9 26.0        -15.6 -19.6 27.2    -14.8 -18.2 30.6 -18.4 -18.5 31.9        -17.5 -22.1 32.5        -14.3 -21.2 34.4        -12.2 -24.1 33.1        -8.7 -24.1 34.5 -5.7 -23.6 32.3      -2.5 -24.5 34.2 0.9 -23.2 33.2  4.4 -23.9 34.3  6.1 -20.6 34.0  9.8 -20.0 34.3  10.1 -16.4 35.4 13.0 -14.0 35.5 13.3 -11.2 38.1      12.7 -10.5 41.8 9.7 -12.2 43.3  8.4 -12.0 46.8  5.2 -13.0 48.6  3.7 -9.5 48.4   3.8 -9.8 44.6   2.2 -13.2 44.8  -0.6 -11.7 47.0      -1.0 -8.9 44.5  -1.3 -11.3 41.5 -3.9 -13.4 43.4 -6.1 -10.5 44.4 -6.1 -8.6 41.1  -3.9 -6.2 39.3  -4.5 -3.7 36.4  -3.6 -4.8 32.9       -3.8 -2.6 29.8  -7.0 -4.4 29.0  -8.5 -4.4 32.5  -8.2 -6.2 35.8  -6.8 -9.7 35.9  -5.7 -12.2 38.6 -4.2 -15.6 38.7 -6.6 -18.1 40.2      -3.8 -20.0 41.8 -0.0 -19.4 42.0 2.4 -21.9 43.4  6.2 -21.7 43.7  7.6 -24.7 41.8  11.4 -24.5 42.2 14.2 -22.0 42.2 17.4 -20.5 41.0      18.6 -17.0 41.7 16.7 -14.6 39.4 15.1 -17.6 37.8 12.2 -19.4 39.4 9.4 -21.8 38.3  5.8 -21.1 39.2  2.7 -23.1 38.4  -0.3 -20.8 37.8      -3.8 -21.1 36.7 -6.3 -18.9 34.9 -9.9 -19.4 34.1 -9.9 -17.1 31.1 -7.6 -16.7 28.1 -6.8 -13.1 27.2 -4.0 -12.5 24.7 -0.6 -13.1 26.3      -2.2 -12.9 29.7 0.3 -14.9 31.8  3.2 -12.8 30.5  1.3 -9.5 31.1   0.4 -10.5 34.6  4.0 -11.2 35.6  5.1 -8.0 33.8   2.5 -6.0 35.8        4.6 -6.8 38.9   8.0 -6.3 37.2   8.8 -9.9 36.2   9.7 -11.4 32.9  8.9 -14.8 31.5  11.6 -17.1 30.0 9.5 -20.2 29.5  5.8 -21.0 29.3       4.3 -24.5 29.1  0.6 -25.3 29.3  0.5 -28.5 31.4  -3.2 -28.9 32.0 -6.5 -27.9 30.5 -9.5 -28.7 32.7 -9.1 -32.2 34.0 -6.7 -33.3 31.2      -3.0 -32.9 32.1 -0.4 -33.0 29.4 -0.1 -36.8 28.8 2.9 -38.3 30.7  4.2 -40.6 28.0  3.5 -38.2 25.2  6.4 -36.4 23.5  6.5 -32.9 22.0       4.6 -32.8 18.6  2.5 -35.9 19.2  -1.1 -36.5 20.3 -3.4 -39.1 21.9 -7.1 -39.7 22.0 -9.2 -39.8 25.2 -11.5 -42.8 25.8    -15.2 -42.7 26.7 -14.8 -42.3 30.4        -12.3 -39.5 30.0        -14.2 -37.7 27.3        -17.3 -38.0 29.6        -15.4 -36.5 32.6    -14.3 -33.6 30.4 -17.9 -33.1 29.4        -19.2 -33.2 33.0        -16.7 -30.4 33.8        -18.7 -28.1 31.6        -22.2 -29.4 32.6    -22.6 -31.6 29.5 -24.2 -35.0 30.3        -22.3 -38.2 29.6        -22.5 -39.9 26.3        -22.8 -43.7 25.4&#xA;&#xA;1XXIJ&#xA;334&#xA;M       R       W       Y       P       W       L       R       P       D       F       E       K       L       V       A       S   L&#xA;.       .       .       .       T       T       H       H       H       H       H       H       H       H       H       T       T   .&#xA;65.7 -16.7 128.5        64.5 -19.9 130.3        66.9 -22.7 131.5        66.0 -25.8 129.7        67.1 -29.4 130.9        69.2 -30.4 127.8     71.4 -27.4 128.6        72.9 -29.3 131.5        75.3 -32.0 130.1        76.9 -29.2 128.1        76.9 -26.7 131.0        78.6 -29.4 133.1     81.8 -29.7 131.1        81.8 -26.0 130.4        82.2 -25.2 134.1        84.3 -28.3 134.6        87.0 -26.5 132.5    86.5 -23.1 134.2 87.2 -24.8 137.5        90.3 -26.9 136.4        91.9 -24.1 134.3        91.2 -25.8 130.9        88.8 -22.9 130.1    88.9 -21.4 126.6 89.0 -17.6 126.7        86.9 -17.5 123.5        83.8 -19.6 123.0        81.6 -19.9 119.9        78.4 -21.8 120.5    76.4 -23.0 117.5 72.8 -23.9 117.9        69.4 -23.9 116.3        66.9 -21.3 117.2        65.1 -21.8 120.5        67.7 -24.4 121.7    68.4 -22.5 124.8 71.4 -20.9 123.2        70.2 -17.9 125.1        70.3 -19.4 128.5        73.6 -21.3 128.3        75.7 -18.2 127.8    74.1 -16.3 130.7 74.1 -19.4 133.0        77.8 -19.8 132.3        78.8 -16.2 133.0        77.0 -16.2 136.4        79.4 -18.9 137.5    82.4 -16.9 136.6 81.1 -13.6 138.0        80.6 -14.9 141.5        82.7 -13.6 144.4        81.9 -16.6 146.7        80.7 -19.6 144.5    79.2 -22.6 146.2 80.7 -25.3 143.9        78.7 -25.6 140.7        76.6 -22.5 141.0        76.6 -19.1 142.8        75.6 -18.0 146.3    73.1 -15.2 145.4 74.5 -12.7 148.0        78.0 -11.6 146.9        78.5 -8.3 145.0 78.7 -10.2 141.8        75.6 -12.3 142.3        73.7 -9.2 143.5      74.8 -6.8 140.8 74.2 -9.8 138.5 70.6 -10.2 139.7        70.4 -6.4 139.2 71.7 -7.1 135.7 74.0 -4.0 135.8 77.6 -5.1 136.4      81.1 -3.7 135.7 82.4 -7.0 134.2 79.5 -8.2 132.2 79.4 -6.7 128.7 76.5 -7.3 126.2 77.5 -6.8 122.6 74.1 -6.4 121.0 74.1 -5.1 117.4      73.4 -1.3 116.8 69.7 -0.7 117.7 67.9 -0.2 114.4 70.3 -1.8 112.0 70.7 -5.6 111.3 74.4 -5.6 111.8 77.1 -4.7 114.4 79.7 -1.9 114.3      83.5 -1.7 114.3 82.9 1.0 117.0  81.1 -1.0 119.7 83.9 -3.6 119.5 86.8 -1.2 119.9 85.1 0.5 122.9  84.7 -2.8 124.7 88.3 -3.9 123.7      89.8 -0.9 125.5 87.2 -1.3 128.2 88.8 -4.6 129.1 92.3 -3.2 128.8 91.9 -1.8 132.3 91.6 -3.7 135.7 88.3 -5.5 136.6 86.3 -3.2 138.9      85.3 -6.1 141.2 88.7 -7.8 141.0 88.2 -11.1 139.2        87.1 -12.3 135.8        85.6 -10.1 132.9        83.1 -11.6 130.4    81.8 -10.7 126.9 78.9 -12.0 124.9        77.8 -11.6 121.3        74.2 -12.0 120.1        75.2 -13.4 116.7        78.7 -13.1 115.2    77.5 -13.8 111.7 75.7 -10.4 111.7        78.8 -8.2 112.1 80.6 -5.9 109.6 83.7 -7.5 107.9 85.8 -4.8 109.5 84.2 -5.5 112.9 85.6 -9.1 112.6      89.2 -8.1 112.3 88.8 -5.4 114.9 87.8 -8.1 117.3 90.6 -10.6 116.4        93.6 -8.6 117.7 92.2 -8.7 121.2 91.9 -12.5 121.2    95.5 -12.5 120.1 96.5 -10.2 123.0        94.2 -11.8 125.7        93.9 -9.5 128.8 94.7 -11.9 131.7        91.7 -13.9 133.1        89.1 -12.4 130.8     86.3 -14.5 129.1        84.5 -14.4 125.7        81.3 -15.9 124.4        80.5 -15.9 120.7        77.3 -17.4 119.5    76.0 -18.2 116.0 73.5 -20.3 114.2        75.8 -20.6 111.2        79.5 -20.8 112.0        80.6 -21.2 108.4        79.5 -17.8 107.3    82.1 -16.6 109.9 85.7 -15.6 109.1        87.9 -18.7 109.1        90.7 -16.8 110.8        88.2 -15.8 113.5        86.5 -19.2 114.1    90.0 -20.6 114.6 90.7 -18.5 117.7        87.6 -19.7 119.5        86.5 -23.1 120.6        83.4 -24.2 118.7        80.9 -26.1 120.8    77.9 -27.9 119.3 74.6 -27.5 121.1        73.0 -30.7 119.6        69.2 -30.8 118.8        68.0 -33.4 121.2        66.8 -36.7 119.7    63.3 -36.2 118.3 61.6 -38.6 120.7        62.8 -36.6 123.6        61.9 -33.2 122.0        58.4 -34.3 121.2        57.6 -35.6 124.7    59.0 -32.4 126.2 56.8 -30.5 123.8        53.6 -32.5 124.5        53.8 -30.9 127.9        54.2 -27.3 126.9        51.2 -27.5 124.5    48.6 -30.1 123.9 48.5 -30.9 120.2        48.7 -34.1 118.1        52.2 -35.6 118.2        52.1 -35.5 114.4        51.8 -31.8 114.5    54.3 -31.6 117.3 56.9 -33.4 115.1        56.1 -31.1 112.2        56.9 -28.0 114.3        60.2 -29.7 115.5        61.2 -30.5 111.9    60.2 -27.0 110.8 62.8 -25.7 113.3        65.7 -27.9 112.4        65.1 -29.8 115.6        65.1 -26.7 117.9        62.5 -27.5 120.5    62.4 -23.9 121.6 61.6 -22.5 118.2        59.2 -25.4 117.7        57.4 -24.4 120.9        57.2 -20.8 119.7        55.7 -22.0 116.5    52.5 -22.6 118.4 52.3 -19.7 120.9        51.0 -17.2 118.3        48.2 -18.8 116.5        50.9 -19.8 114.0        50.0 -23.4 113.8    46.4 -22.5 113.1 47.8 -20.1 110.4        49.7 -23.1 109.1        46.6 -25.3 109.0        45.0 -22.6 106.8        47.8 -23.0 104.4    47.6 -26.8 104.0 43.8 -26.5 103.4        44.3 -24.0 100.7        47.1 -26.1 99.2 44.9 -29.2 99.2 42.3 -27.4 97.2 44.7 -26.0 94.7 46.9 -29.1 94.1      44.1 -31.1 92.5 42.2 -28.1 91.1 45.1 -26.2 89.4 43.8 -22.9 90.9 47.0 -22.0 92.7 46.0 -18.4 92.8 43.6 -19.4 95.6 46.6 -19.4 98.0      46.7 -15.7 97.7 44.6 -15.2 100.8        46.6 -17.8 102.7        50.0 -16.3 101.8        49.4 -12.6 101.4        50.1 -11.5 104.9     53.0 -9.8 106.8 54.0 -13.4 107.8        54.5 -14.0 104.1        58.1 -15.2 104.4        57.5 -17.6 107.2        54.6 -18.9 105.2     56.4 -19.2 101.9        59.0 -21.3 103.7        56.2 -23.6 105.0        55.0 -24.0 101.4        58.6 -24.8 100.4    59.0 -27.6 102.9 56.0 -29.3 101.4        57.5 -28.8 97.9 60.7 -30.3 99.0 59.0 -33.4 100.5        57.0 -34.1 97.3 60.3 -33.9 95.5 62.0 -36.3 97.9      59.2 -38.8 97.5 59.9 -39.0 93.7 63.3 -40.3 94.6 61.9 -42.6 97.2 63.3 -40.7 100.2        62.2 -42.2 103.5        62.3 -39.0 105.6     59.6 -36.3 105.3        59.3 -33.1 107.2        55.6 -32.7 106.8        54.4 -35.9 108.5        51.5 -33.8 109.6    49.8 -34.3 106.2 51.3 -37.2 104.2        48.2 -37.2 102.1        49.2 -33.8 100.8        52.8 -34.5 99.9 51.5 -37.6 98.1 48.9 -36.0 95.9      51.3 -33.1 95.3 54.0 -35.5 94.1 51.3 -37.3 92.0 50.2 -34.2 90.0 53.6 -32.6 89.7 56.6 -34.0 87.9 59.9 -33.8 89.7 60.9 -31.0 87.3      57.8 -28.9 87.8 58.3 -29.2 91.5 62.0 -28.0 91.2 60.8 -25.0 89.2 58.2 -24.0 91.7 60.5 -24.7 94.7 62.8 -22.0 93.4 60.4 -19.2 92.5      58.9 -19.8 95.9 62.4 -19.6 97.2 63.7 -16.5 95.4 60.4 -14.7 95.9 60.3 -15.3 99.6 63.7 -13.7 99.7 62.8 -10.5 97.9 60.1 -10.1 100.5     62.5 -10.4 103.3        65.4 -8.5 101.8 64.0 -5.6 99.5  62.8 -2.7 101.7 59.2 -2.6 100.6 57.9 -4.3 97.4  54.2 -5.0 96.9  53.5 -8.2 98.9       50.4 -8.5 96.9  51.8 -8.1 93.4  54.8 -10.2 93.8 52.9 -13.1 95.2 50.1 -12.8 92.6 52.9 -13.0 90.1 54.5 -16.0 91.9 51.1 -17.7 91.9      50.0 -17.1 88.3 53.5 -18.0 87.3 53.3 -21.3 89.1 50.3 -22.1 87.0 52.1 -21.6 83.7 54.8 -24.0 84.7 52.0 -26.5 85.2 50.8 -26.1 81.6      52.0 -28.7 79.0 54.8 -27.5 76.7 56.2 -24.8 78.9 60.0 -24.6 79.4 60.6 -24.0 83.1 63.6 -22.0 84.5 67.2 -23.0 85.4 69.4 -21.4 88.1      71.8 -18.6 87.1 75.6 -17.8 87.3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and so on (the same structure)&lt;/p&gt;&#xA;&lt;p&gt;Please note in  line 5; There is a space between x&#xA;and y and y and z, but each XYZ group is tab-separated from the next.&lt;/p&gt;&#xA;&lt;p&gt;Now  I am unsure if the code down works and computing the distance between ( XYZ coordinates) correctly or not!&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '&#xA;NR%6==5 { printf &amp;quot;%f\n&amp;quot;, sqrt(($1-x)^2 + ($2-y)^2 + ($3-z)^2) }&#xA;       { x=$1; y=$2; z=$3 }' myfile.adataset &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I want the output to be like this, which is to print also the lines 1,2,3,4 , and the results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;4LGTD&#xA;247&#xA;M       S       E       K       L       Q       K       V       L       &#xA;.       .       E       E       H       H       H       H       H &#xA;the result&#xA;&#xA;1XXIJ&#xA;334&#xA;M       R       W       Y       P       W       L       R       P&#xA;        T       T       H       H       H       H       H&#xA;the result &#xA;&#xA;and so on &#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So after that, I want other code to delete blocks of lines(1,2,3,4,5)if the results of XYZ coordinates are bigger than (4.5)?&#xA;Any suggestions  code for first computing the distance and then deleting if the line 5 result is bigger than 4.5 by awk to do this?&lt;/p&gt;&#xA;" OwnerUserId="16923" LastEditorUserId="298" LastEditDate="2023-02-22T16:33:22.617" LastActivityDate="2023-02-23T22:37:52.223" Title="computing the distance between ( XYZ coordinates)" Tags="|protein-structure|linux|awk|" AnswerCount="1" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="20597" PostTypeId="2" ParentId="20590" CreationDate="2023-02-20T11:34:59.957" Score="1" Body="&lt;p&gt;I am not sure if I understand your first question correctly.&lt;/p&gt;&#xA;&lt;p&gt;Kraken is meant to be for the taxonomic classification of metagenomic sequencing data. Therefore, if you only have one target genome of interest there is no need to use Kraken. You can just use any sequence aligner and check the mapping rates.&lt;/p&gt;&#xA;&lt;p&gt;However, if you are trying to add a custom genome to an existing database then you need a taxonomy id for the sequence names in the genome and add it to your DB using the --add-to-library command. If this is what you are asking this tutorial would help you &lt;a href=&quot;https://bioinformaticsworkbook.org/dataAnalysis/Metagenomics/Kraken.html#gsc.tab=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioinformaticsworkbook.org/dataAnalysis/Metagenomics/Kraken.html#gsc.tab=0&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Regarding your second question, I truly believe those ref indexes are ready to go. You can download the tar file, extract it in a folder called MinusB and release the Kraken using, e.g., &lt;code&gt;kraken2 --db MinusB yourreads.fq&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Hope this helps&lt;/p&gt;&#xA;" OwnerUserId="15700" LastEditorUserId="15700" LastEditDate="2023-02-20T14:37:52.030" LastActivityDate="2023-02-20T14:37:52.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20598" PostTypeId="1" CreationDate="2023-02-20T12:33:25.613" Score="1" ViewCount="97" Body="&lt;p&gt;Im working on building a phylogeny from scratch with downloaded FASTA sequences from GeneBank. I think Im doing alright up until the multi sequence alignment in the &lt;code&gt;msa&lt;/code&gt; package where I get an error &lt;code&gt;Error in convertAlnRows(result$msa, type) : There is an invalid aln file!&lt;/code&gt; As always thanks in advance for taking a look: Here is the reproducible example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library (ape)&#xA;library (rentrez)&#xA;library (msa)&#xA;&#xA;#Download Species Data &#xA;# B_terrestris&#xA;B_terrestris &amp;lt;- entrez_fetch(db = &amp;quot;nucleotide&amp;quot;, &#xA;                                  id = &amp;quot;NC_045179.1&amp;quot;, &#xA;                                  rettype = &amp;quot;fasta&amp;quot;)&#xA;&#xA;# B_hypocrita&#xA;B_hypocrita &amp;lt;- entrez_fetch(db = &amp;quot;nucleotide&amp;quot;, &#xA;                             id = &amp;quot;NC_011923.1&amp;quot;, &#xA;                             rettype = &amp;quot;fasta&amp;quot;)&#xA;&#xA;#Vespa&#xA;Vespa &amp;lt;- entrez_fetch(db = &amp;quot;nucleotide&amp;quot;, &#xA;                      id = &amp;quot; MT137096.1&amp;quot;, &#xA;                      rettype = &amp;quot;fasta&amp;quot;)&#xA;&#xA;seq&amp;lt;- c(B_terrestris, B_hypocrita, Vespa) # gotta clean it &#xA;&#xA;#FASTA Cleaner Function &#xA;fasta_cleaner &amp;lt;- function(fasta_object, parse = TRUE)&#xA;{&#xA;  fasta_object &amp;lt;- sub(&amp;quot;^(&amp;gt;)(.*?)(\\n)(.*)(\\n\\n)&amp;quot;,&amp;quot;\\4&amp;quot;, fasta_object)&#xA;  fasta_object &amp;lt;- gsub(&amp;quot;\n&amp;quot;, &amp;quot;&amp;quot;, fasta_object)&#xA;  &#xA;  if(parse == TRUE){&#xA;    fasta_object &amp;lt;- stringr::str_split(fasta_object,&#xA;                                       pattern = &amp;quot;&amp;quot;,&#xA;                                       simplify = FALSE)&#xA;  }&#xA;  &#xA;  return(fasta_object[[1]])&#xA;}&#xA;&#xA;&#xA;#This should be ready to go! &#xA;clean.seq&amp;lt;-fasta_cleaner (seq, parse = FALSE)&#xA;&#xA;str (clean.seq)&#xA;&#xA;msa (clean.seq,  type=&amp;quot;dna&amp;quot;, method=&amp;quot;Muscle&amp;quot;, cluster=&amp;quot;neighborjoining&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17043" LastEditorUserId="3848" LastEditDate="2023-08-28T18:59:05.367" LastActivityDate="2023-08-28T18:59:05.367" Title="Phylogeny building in R from FASTA files:" Tags="|r|phylogenetics|sequence-alignment|phylogeny|fasta|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20599" PostTypeId="1" CreationDate="2023-02-20T12:39:36.133" Score="2" ViewCount="96" Body="&lt;p&gt;I have a problem with  nextflow -resume&lt;/p&gt;&#xA;&lt;p&gt;Here is my script :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Reads_ch = Channel.fromSRA(params.ids, apiKey: params.apikey)&#xA;         .flatMap { left, right -&amp;gt;&#xA;            def SRRCode = left&#xA;            switch(right) {&#xA;                case { right.getClass() == java.util.ArrayList &amp;amp;&amp;amp; right.size() == 2 }:&#xA;                    def forward = right[0]&#xA;                    def reverse = right[1]&#xA;                    return [tuple(SRRCode.concat(&amp;quot;_1&amp;quot;), forward), tuple(SRRCode.concat(&amp;quot;_2&amp;quot;), reverse)]&#xA;&#xA;                case { right.getClass() == java.util.ArrayList &amp;amp;&amp;amp; right.size() == 3 }:&#xA;                    def forward = right[1]&#xA;                    def reverse = right[2]&#xA;                    return [tuple(SRRCode.concat(&amp;quot;_1&amp;quot;), forward), tuple(SRRCode.concat(&amp;quot;_2&amp;quot;), reverse)]&#xA;&#xA;                case right.getClass() == nextflow.file.http.XPath:&#xA;                    def sequence = right&#xA;                    return [tuple(SRRCode, sequence)]&#xA;            }&#xA;      }&#xA;&#xA;process Unzip {&#xA;    tag &amp;quot;Unzip $fileCode reads files using unpigz&amp;quot;&#xA;    maxForks params.threads&#xA;    cpus params.threads_unzip&#xA;&#xA;    input:&#xA;    set fileCode, file(readsFile) from Reads_ch&#xA;&#xA;    output:&#xA;    tuple fileCode, file(&amp;quot;${fileCode}.fasta&amp;quot;) into Kmer_ch&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    unpigz -cp&lt;span class=&quot;math-container&quot;&gt;${task.cpus} $&lt;/span&gt;readsFile | paste - - - - | cut -f 1,2 | sed 's/^@/&amp;gt;/' | tr &amp;quot;\\t&amp;quot; &amp;quot;\\n&amp;quot; &amp;gt; ${fileCode}.fasta&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process Kmer {&#xA;    tag &amp;quot;Processing $fileCode reads files using kmer-ssr&amp;quot;&#xA;    maxForks params.threads&#xA;    cpus params.threads_kmer&#xA;&#xA;    input:&#xA;    set fileCode, file(readsFile) from Kmer_ch&#xA;&#xA;    output:&#xA;    tuple fileCode, file(&amp;quot;${fileCode}.tsv&amp;quot;) into ComputedReads_ch&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    kmer-ssr -d -t &lt;span class=&quot;math-container&quot;&gt;${task.cpus} -p $&lt;/span&gt;{params.kmerSSR.periodMin}-&lt;span class=&quot;math-container&quot;&gt;${params.kmerSSR.periodMax} -n $&lt;/span&gt;{params.kmerSSR.ssrSizeTreshold} -r &lt;span class=&quot;math-container&quot;&gt;${params.kmerSSR.repeatMin} -l $&lt;/span&gt;{params.kmerSSR.readLengthMin} -i &lt;span class=&quot;math-container&quot;&gt;${fileCode}.fasta -o $&lt;/span&gt;{fileCode}.tsv&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When the pipeline is broken, I try to run again with nextflow run -resume and the problem is that all sequences from SRA are downloaded again ?? and the pipeline starts again from the beginning instead of resuming the job where it was interrupted.&#xA;What is the problem, please ?&#xA;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="16948" LastActivityDate="2023-02-20T12:39:36.133" Title="Problem with nextflow -resume and path.fromSRA" Tags="|nextflow|data-download|sra|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20600" PostTypeId="1" CreationDate="2023-02-20T15:35:04.383" Score="2" ViewCount="47" Body="&lt;p&gt;I am currently working on my PhD and as part of my thesis, I intend to analyse gene expression within multiple sclerosis (MS) lesions by looking at RNA-seq datasets on Gene Expression Omnibus (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/&lt;/a&gt;)  , which I have selected already. Part of my analysis is using R to transform the count matrices provided on GEO into DGElists (differential gene expression lists), and I have been using both the edgeR handbook, as well as an R script provided by one of the groups whose dataset I am including in my analysis, to construct my own code (as I have had to teach myself R programming from scratch).&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;[MS lesions are multiple sclerosis lesions. There are multiple subtypes that are included in this group's analysis - Active, Chronic active, inactive, remyelinating (or repairing) lesions and Normal Appearing White Matter. These classifications relate to the physiology of the lesion - active lesions are undergoing active inflammation, chronic active means it goes through periods of inflammation, inactive means inflammation isn't occurring at the moment, and remyelinating relates to a lesion that is undergoing healing.]&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;My current issue is as follows: Following along with the script provided by the lab group, they appear to have calculated a &amp;quot;PCA Vector&amp;quot;, which they use in the creation of their design matrix. As far as I can tell, PCA, or Principal Component Analysis, is meant to account for the multi-dimensionality of a dataset, essentially transforming the data so that it can be plotted on a 2-dimensional set of axes and the variability of the points shown visually. Upon contacting the group, they said that they used PCA to find the biggest outliers as well as identify the largest patient bias, which they then normalised for.&lt;/p&gt;&#xA;&lt;p&gt;The group has also provided their data which can be opened in R. When I utilise their &amp;quot;PCA vector&amp;quot; in the construction of my design matrix, I get the same values as the group, however, any attempts at producing this PCA vector by performing PCA using R packages does not give me values that are in any way close to those in the group's DGE lists (also provided).&lt;/p&gt;&#xA;&lt;p&gt;My question therefore is this: Does anybody have an idea as to how one could create a &amp;quot;PCA vector&amp;quot; - a value of variation between patients - which could then be incorporated into a design matrix in order to account for patient variation? I feel I should mention - patient age, sex and MS lesion type are already accounted for as they are also used in the construction of the design matrix. Would anyone have any experience with this?&lt;/p&gt;&#xA;&lt;p&gt;I ask that you please be gentle/patient with me - I am still very new to the world of bioinformatics (and eager to learn!) and I apologise if I have not explained my issue very well. I will do my very utmost to clear up any confusion, and I would be incredibly grateful to anyone who can help!&lt;/p&gt;&#xA;&lt;p&gt;For those interested in seeing the journal article which includes links to the GEO repository which hosts their DGE lists and count matrix, as well as their github containing their R script and data, I have included the link below.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://actaneurocomms.biomedcentral.com/articles/10.1186/s40478-019-0855-7#availability-of-data-and-materials&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://actaneurocomms.biomedcentral.com/articles/10.1186/s40478-019-0855-7#availability-of-data-and-materials&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17045" LastEditorUserId="73" LastEditDate="2023-02-21T18:23:28.273" LastActivityDate="2023-02-21T18:23:28.273" Title="Using Multi-Dimensional Scaling (MDS) to produce a vector in order to account for patient bias when constructing DGE lists from RNA-seq datasets in R?" Tags="|r|statistics|normalization|edger|pca|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20601" PostTypeId="2" ParentId="20598" CreationDate="2023-02-20T17:11:34.430" Score="1" Body="&lt;p&gt;The formal question I've been given is what aligner would I use? There is an issue between forming a data pipeline for &lt;code&gt;ape&lt;/code&gt; and the latest and greatest. Its trade and the compromise would be &lt;code&gt;msaClustalOmega()&lt;/code&gt;, but the rationale is complicated.&lt;/p&gt;&#xA;&lt;p&gt;I strictly use &lt;a href=&quot;https://drive5.com/muscle5/manual/cmd_super5.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;muscle5&lt;/code&gt;&lt;/a&gt; or specifically &lt;code&gt;muscle -super5&lt;/code&gt; option as a standalone. I am certain &lt;code&gt;R&lt;/code&gt; &lt;code&gt;msa&lt;/code&gt; has not introduced &lt;code&gt;muscle5&lt;/code&gt; and I don't even know if it's published yet. It's on bioRxiv.org. The &lt;code&gt;super5&lt;/code&gt; option must be specified and once done its amongst the most incredible algorithms &lt;em&gt;per se&lt;/em&gt; notably in speed - no other aligner comes close - and personal benchmarks in accuracy. To give some idea, I was going to write an aligner as part of a package (its makes stuff simpler) and thought - whats the point with &lt;code&gt;muscle5&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Its competitor was MAFFT and published benchmarks demonstrated it was the strongest, before &lt;code&gt;muscle5&lt;/code&gt; was released.&lt;/p&gt;&#xA;&lt;p&gt;The dilemma is simply that &lt;code&gt;ape&lt;/code&gt; is cool and if you are wanting to have a seamless pipeline then greatest aligners don't fit with that. MAFFT isn't in &lt;code&gt;msa&lt;/code&gt; and seriously doubt &lt;code&gt;muscle5&lt;/code&gt; will be. HOWEVER Clustal Omega is in &lt;code&gt;msa&lt;/code&gt; and that is the strongest (most recent) of the options it has. Omega does have published benchmarks and its neither quite as fast nor quite as accurate as MAFFT.&lt;/p&gt;&#xA;&lt;p&gt;If you've got 50000 sequences - it's &lt;code&gt;muscle5&lt;/code&gt; no contest. If you've 50 sequences then keep the pipeline together via &lt;code&gt;msaClustalOmega&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The original question and code appeared to involve converting fasta inside &lt;code&gt;R&lt;/code&gt; onto &lt;code&gt;msa&lt;/code&gt;. Thats how subprocessing works outside &lt;code&gt;R&lt;/code&gt; (or &lt;code&gt;Python&lt;/code&gt;) but it's not the OO approach.&lt;/p&gt;&#xA;&lt;p&gt;My original answer was in comments and remains an unofficial answer because usually I run the code I'm recommending and I don't fancy subprocessing muscle via &lt;code&gt;R&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The error was the &lt;code&gt;msa&lt;/code&gt; algorithm was attempting to shift the object into a format that will subprocess via muscle and it doesn't understand fasta format (it wants the raw object). Thus, you appear to be wanting to pass &amp;quot;official fasta&amp;quot; into msa. It doesn't need it, it simply needs a sequence object. It's imported as fasta but then it should be a sequence object (it's no longer fasta). That should go straight into msa. Thus &lt;code&gt;seq&lt;/code&gt; should load straight into msa with &amp;quot;fasta&amp;quot; option removed. Finally could you remove msa and used &lt;code&gt;msaMuscle&lt;/code&gt; then simply &lt;code&gt;msaMuscle(seq)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-02-20T17:18:10.683" LastActivityDate="2023-02-20T17:18:10.683" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20602" PostTypeId="1" CreationDate="2023-02-20T19:41:59.750" Score="3" ViewCount="76" Body="&lt;p&gt;I downloaded the &lt;a href=&quot;http://www.candidagenome.org/download/gff/C_albicans_SC5314/Assembly22/&quot; rel=&quot;nofollow noreferrer&quot;&gt;gff&lt;/a&gt; and &lt;a href=&quot;http://www.candidagenome.org/download/sequence/C_albicans_SC5314/Assembly22/current/&quot; rel=&quot;nofollow noreferrer&quot;&gt;fasta&lt;/a&gt; files for candida albicans from &lt;a href=&quot;http://www.candidagenome.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.candidagenome.org&lt;/a&gt;.  I want to use &lt;code&gt;hisat2&lt;/code&gt; to align some fastq files against it.&lt;/p&gt;&#xA;&lt;p&gt;There are many files in each download :&lt;/p&gt;&#xA;&lt;p&gt;gff :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A22_ForcheSNPs.gff&#xA;A22_Historic_Assemblies.gff&#xA;A22_Jones_PMID_15123810_Polymorphisms.gff&#xA;A22_Jones_PMID_15123810_Polymorphisms.vcf&#xA;A22_Unannotated_transcripts_Bruno_et_al.gff&#xA;A22_Unannotated_transcripts_Sellam_et_al.gff&#xA;A22_Unannotated_transcripts_Tuch_et_al.gff&#xA;C_albicans_SC5314_A22_current_features.gff&#xA;C_albicans_SC5314_A22_current_features.gtf&#xA;C_albicans_SC5314_A22_current_features_with_chromosome_sequences.gff.gz&#xA;C_albicans_SC5314_A22_current_intergenic.gff&#xA;C_albicans_SC5314_haplotype_variations.gff&#xA;C_albicans_SC5314_version_A22-s07-m01-r179_features.gff&#xA;C_albicans_SC5314_version_A22-s07-m01-r179_features.gtf&#xA;C_albicans_SC5314_version_A22-s07-m01-r179_features_with_chromosome_sequences.gff.gz&#xA;C_albicans_SC5314_version_A22-s07-m01-r179_intergenic.gff&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;fasta :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;C_albicans_SC5314_A22_current_chromosomes.fasta.gz&#xA;C_albicans_SC5314_A22_current_default_coding.fasta.gz&#xA;C_albicans_SC5314_A22_current_default_genomic.fasta.gz&#xA;C_albicans_SC5314_A22_current_default_protein.fasta.gz&#xA;C_albicans_SC5314_A22_current_not_feature.fasta.gz&#xA;C_albicans_SC5314_A22_current_orf_coding.fasta.gz&#xA;C_albicans_SC5314_A22_current_orf_genomic_1000.fasta.gz&#xA;C_albicans_SC5314_A22_current_orf_genomic.fasta.gz&#xA;C_albicans_SC5314_A22_current_orf_plus_intergenic.fasta.gz&#xA;C_albicans_SC5314_A22_current_orf_trans_all.fasta.gz&#xA;C_albicans_SC5314_A22_current_other_features_genomic_1000.fasta.gz&#xA;C_albicans_SC5314_A22_current_other_features_genomic.fasta.gz&#xA;C_albicans_SC5314_A22_current_other_features_no_introns.fasta.gz&#xA;C_albicans_SC5314_A22_current_other_features_plus_intergenic.fasta.gz&#xA;C_albicans_SC5314_version_A22-s07-m01-r179_chromosomes.fasta.gz&#xA;C_albicans_SC5314_version_A22-s07-m01-r179_default_coding.fasta.gz&#xA;C_albicans_SC5314_version_A22-s07-m01-r179_default_genomic.fasta&#xA;C_albicans_SC5314_version_A22-s07-m01-r179_default_protein.fasta.gz&#xA;C_albicans_SC5314_version_A22-s07-m01-r179_not_feature.fasta.gz&#xA;C_albicans_SC5314_version_A22-s07-m01-r179_orf_coding.fasta.gz&#xA;C_albicans_SC5314_version_A22-s07-m01-r179_orf_genomic_1000.fasta.gz&#xA;C_albicans_SC5314_version_A22-s07-m01-r179_orf_genomic.fasta.gz&#xA;C_albicans_SC5314_version_A22-s07-m01-r179_orf_plus_intergenic.fasta&#xA;C_albicans_SC5314_version_A22-s07-m01-r179_orf_trans_all.fasta.gz&#xA;C_albicans_SC5314_version_A22-s07-m01-r179_other_features_genomic_1000.fasta.gz&#xA;C_albicans_SC5314_version_A22-s07-m01-r179_other_features_genomic.fasta.gz&#xA;C_albicans_SC5314_version_A22-s07-m01-r179_other_features_no_introns.fasta.gz&#xA;C_albicans_SC5314_version_A22-s07-m01-r179_other_features_plus_intergenic.fasta.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm pretty sure I'd like to use the version &lt;code&gt;*A22-s07-m01-r179*&lt;/code&gt; since it seems like it is a version that will is not transient like the &lt;code&gt;*current*&lt;/code&gt; files.&lt;/p&gt;&#xA;&lt;p&gt;QUESTION :&lt;/p&gt;&#xA;&lt;p&gt;I am looking at bulk gene expression derived from an RNA-Seq experiment run on an Illumina sequencer.  Which gff / fasta file should I use for alignment with &lt;code&gt;hisat&lt;/code&gt;?&lt;/p&gt;&#xA;" OwnerUserId="3886" LastEditorUserId="3886" LastEditDate="2023-02-21T16:05:19.900" LastActivityDate="2023-02-21T16:05:19.900" Title="Which candida albicans fasta and gff file should I use for alignment?" Tags="|rna-seq|fasta|gff|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20603" PostTypeId="2" ParentId="20602" CreationDate="2023-02-20T19:51:11.900" Score="0" Body="&lt;p&gt;The refseq is &lt;em&gt;Candida albicans&lt;/em&gt; SC5314.&lt;/p&gt;&#xA;&lt;p&gt;I assume you are performing a fasta reference based assembly.&lt;/p&gt;&#xA;&lt;p&gt;Its 8 chromosomes are  NC_032089.1 to NC_032096.1 inclusively from chromosome 1 to chromosome 7 (NC_032095.1) to chromosome R.&lt;/p&gt;&#xA;&lt;p&gt;Its &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/?term=Candida%20albicans%20SC5314&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Most of the files you downloaded are SC5314. So I dunno it depends what the project is re-annotation, reassembly?&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;vcf&lt;/code&gt; is a very interesting file, particularly if you are sequencing your own variants of &lt;em&gt;Candida&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Whats in all the files:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;C_albicans_SC5314_A22_current_orf_coding.fasta.gz proteins ignoring introns - useful&lt;/li&gt;&#xA;&lt;li&gt;C_albicans_SC5314_A22_current_chromosomes.fasta.gz is going to be NCBI files I described.&lt;/li&gt;&#xA;&lt;li&gt;C_albicans_SC5314_A22_current_default_protein.fasta.gz orfs splicing out the introns&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;... list goes on really it depends on the project. I would recommend &lt;code&gt;current&lt;/code&gt; only. &lt;code&gt;A22-s07-m01-r179&lt;/code&gt; will likely be the previous assembly / annotation.&lt;/p&gt;&#xA;&lt;p&gt;I guess the single file you want is C_albicans_SC5314_A22_current_chromosomes.fasta.gz for referenced based assemblies (presumably of &lt;em&gt;Candida&lt;/em&gt; isolates).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-02-20T21:51:15.153" LastActivityDate="2023-02-20T21:51:15.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20604" PostTypeId="2" ParentId="20602" CreationDate="2023-02-21T14:40:13.163" Score="1" Body="&lt;p&gt;First of all, you almost certainly want the &lt;code&gt;current&lt;/code&gt; files. The convention in pretty much all sequence repositories is that &lt;code&gt;foo.current&lt;/code&gt; is always the link pointing to the most recent and most up to date data. Those are what you want and not some &amp;quot;stable&amp;quot; version, you want the most accurate and that will usually be the &lt;code&gt;current&lt;/code&gt;. In any case, my bet is that you will find that the &lt;code&gt;*A22-s07-m01-r179*&lt;/code&gt; files are exactly the same as the corresponding &lt;code&gt;*current*&lt;/code&gt; files since the &lt;code&gt;current&lt;/code&gt; ones will just be symlinks.&lt;/p&gt;&#xA;&lt;p&gt;Next, in order to align your data, you need the genome. The genome is this one:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://www.candidagenome.org/download/sequence/C_albicans_SC5314/Assembly22/current/C_albicans_SC5314_A22_current_chromosomes.fasta.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;C_albicans_SC5314_A22_current_chromosomes.fasta.gz&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;That is the file that has the chromosomes in fasta format, so that's the one you want for alignment. Now, if you have done RNAseq and only care about annotated, known genes, you could speed things up a little by using one of the more targeted files (just scroll down on the page you linked to for an explanation of what each file has), but using the whole genome won't be wrong. And at 8.4M compressed, that's a tiny genome so I don't think it's worth trying to get much more speed out of what will already be a pretty quick process.&lt;/p&gt;&#xA;&lt;p&gt;As for which GFF, that will again depend on what you want, but most likely you want either of these, depending on whether you want GFF or GTF:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;C_albicans_SC5314_A22_current_features.gff&#xA;C_albicans_SC5314_A22_current_features.gtf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The main message is that you usually want the &amp;quot;current&amp;quot; files, just make a note of what version the &amp;quot;current&amp;quot; files are pointing to if you need to report that.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-02-21T14:40:13.163" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20605" PostTypeId="1" AcceptedAnswerId="20614" CreationDate="2023-02-21T18:51:09.093" Score="2" ViewCount="58" Body="&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;def correlations(group):&#xA;    # Remove columns&#xA;    group = (&#xA;        group.reset_index()&#xA;        .drop(columns=[&amp;quot;RNA1_Approved_Symbol&amp;quot;], level=0)&#xA;        .set_index(&amp;quot;RNA1_ID&amp;quot;)&#xA;    )&#xA;    # compute correlations&#xA;    correlation = group.T.corr()&#xA;&#xA;    # rename pair column&#xA;    correlation = correlation.reset_index().rename(&#xA;        columns={&amp;quot;RNA1_ID&amp;quot;: &amp;quot;RNA1_ID_new&amp;quot;}&#xA;    )&#xA;&#xA;    # Remove NAs&#xA;    corr_result = (&#xA;        correlation.melt(value_name=&amp;quot;Correlation&amp;quot;, id_vars=[&amp;quot;RNA1_ID_new&amp;quot;])&#xA;        .set_index([&amp;quot;RNA1_ID&amp;quot;, &amp;quot;RNA1_ID_new&amp;quot;])&#xA;        .dropna()&#xA;    )&#xA;    return corr_result&#xA;&#xA;&#xA;def correlation_1(data):&#xA;    ## Create Pivot table&#xA;    pivot_library = data.pivot(&#xA;        index=[&amp;quot;RNA1_Approved_Symbol&amp;quot;, &amp;quot;RNA1_ID&amp;quot;], columns=[&amp;quot;RNA2_ID&amp;quot;]&#xA;    )&#xA;&#xA;    # Measure correlations between four id's&#xA;    return (&#xA;        pivot_library.groupby(&amp;quot;RNA1_Approved_Symbol&amp;quot;)&#xA;        .apply(correlations)&#xA;        .reset_index()&#xA;        .dropna()&#xA;    )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am using the above function to calculate the correlation&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;data = df1&#xA;correlations = correlation_1(df1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to automate the above function when I am using different data frame where there is &lt;code&gt;RNA2_Approved_Symbol&lt;/code&gt; instead of &lt;code&gt;RNA1_Approved_Symbol&lt;/code&gt; and &lt;code&gt;RNA2_ID&lt;/code&gt; instead of &lt;code&gt;RNA1_ID&lt;/code&gt;. Like using index 1 and index 2.&lt;/p&gt;&#xA;" OwnerUserId="4845" LastEditorUserId="650" LastEditDate="2023-02-21T21:00:30.073" LastActivityDate="2023-02-22T21:13:06.633" Title="Automate the def function in pandas for correlation" Tags="|python|statistics|pandas|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20606" PostTypeId="2" ParentId="20596" CreationDate="2023-02-22T02:22:37.377" Score="2" Body="&lt;p&gt;Based on the example you've provided, your awk command is returning an incorrect result (&lt;a href=&quot;https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem&quot;&gt;XY problem&lt;/a&gt;), however one potential answer to your question is to print your desired outcome using two clauses:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;NR%6==0 (i.e. every 6 lines: 5 lines and 1 blank line) {do action}&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;NR%6!=0 (i.e. every other line) {do different action}&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Example data:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat test.txt&#xA;&#xA;4LGTD&#xA;247&#xA;M       S       E       K       L       Q       K       V       L       A       R       A       G       H       G       S       R   T&#xA;.       .       E       E       H       H       H       H       H       H       H       T       T       S       S       .       H   .&#xA;-34.3 -28.9 16.3        -33.0 -26.8 13.4        -29.4 -25.4 13.3        -27.9 -22.6 11.2        -26.9 -24.1 7.8 -23.3 -22.9 8.2 -23.0 -24.7 11.5     -24.1 -27.9 9.9 -21.8 -27.7 6.9 -18.8 -26.7 9.0 -19.5 -29.7 11.3        -19.6 -31.9 8.2 -16.2 -30.7 7.2 -16.7 -28.2 4.4      -15.0 -25.1 5.8 -15.4 -22.7 8.7 -18.8 -21.3 9.7 -17.7 -17.9 8.3 -16.3 -19.3 5.1 -19.5 -21.2 4.5 -21.5 -18.0 5.1 -19.7 -16.2 2.3      -20.8 -19.1 -0.0        -24.4 -19.2 1.2 -24.6 -15.4 0.6 -23.5 -16.0 -3.0        -26.4 -18.4 -3.6        -24.1 -21.4 -4.0    -25.9 -23.6 -1.5 -29.3 -25.2 -2.0        -31.4 -27.0 0.5 -34.0 -29.5 -0.8        -33.9 -27.8 -4.2        -34.1 -24.2 -3.0        -31.3 -21.7 -2.9     -30.7 -21.0 0.8 -30.2 -17.5 2.2 -27.9 -16.3 4.9 -29.4 -17.1 8.3 -31.3 -20.1 7.0 -31.7 -23.1 9.4 -32.1 -26.8 8.7 -32.6 -30.1 10.5     -30.2 -32.9 9.7 -32.0 -36.0 8.4 -31.6 -38.7 5.7 -34.0 -36.6 3.6 -32.0 -33.4 3.7 -30.4 -32.7 0.4 -27.7 -30.0 0.6 -25.8 -29.0 -2.5     -23.0 -26.6 -3.0        -22.5 -25.6 -6.6        -25.0 -28.3 -7.5        -22.9 -31.1 -5.9        -24.6 -33.1 -3.2    -23.0 -33.1 0.2  -23.1 -36.0 2.6 -23.7 -34.9 6.2 -23.8 -36.9 9.4 -27.2 -37.3 11.1        -28.5 -37.1 14.7        -19.2 -38.7 17.8    -17.7 -36.6 20.6 -14.0 -37.3 20.3        -11.2 -35.4 22.0        -7.4 -35.3 21.2 -4.7 -34.3 23.6 -1.9 -32.6 21.6 1.6 -31.6 22.7  2.2 -28.4 20.7       5.8 -27.5 20.1  7.1 -23.9 19.9  7.1 -22.6 16.3  3.6 -24.0 15.4  0.7 -21.6 14.7  -2.8 -22.2 16.0 -4.8 -21.4 12.8 -6.4 -23.3 9.9       -4.8 -20.9 7.5  -1.6 -18.8 8.1  -1.1 -16.0 5.5  2.7 -16.4 6.0   3.0 -20.1 5.2   4.2 -21.3 8.7   3.5 -24.7 10.3  0.1 -24.9 11.9       -1.1 -27.3 14.7 -3.6 -28.9 12.3 -1.0 -30.0 9.8  -0.7 -33.4 11.5 -4.0 -34.7 12.9 -6.1 -37.9 12.3 -8.5 -37.7 9.3  -12.2 -38.1 10.1     -15.3 -39.0 8.2 -17.5 -36.0 7.6 -16.2 -34.0 10.6        -13.2 -31.8 11.6        -10.9 -31.0 14.5        -11.7 -27.8 16.4     -8.8 -26.3 18.5 -9.8 -25.6 22.1 -8.4 -22.1 22.1 -5.0 -20.9 20.9 -1.5 -21.1 22.2 1.2 -18.6 21.2  3.8 -19.7 18.6  6.6 -20.0 21.1       4.5 -21.3 23.9  4.7 -25.1 24.5  2.3 -27.7 25.8  -1.3 -29.0 25.9 -3.8 -28.1 23.2 -7.0 -30.0 23.1 -8.9 -30.7 19.9 -12.5 -32.0 19.6     -14.9 -32.8 16.9        -17.6 -31.5 19.2        -17.9 -27.7 18.8        -20.2 -27.3 21.8        -17.6 -29.0 24.0        -15.0 -26.8 22.4     -16.8 -23.5 23.1        -17.7 -24.6 26.6        -14.1 -25.3 27.4        -12.9 -21.9 26.0        -15.6 -19.6 27.2    -14.8 -18.2 30.6 -18.4 -18.5 31.9        -17.5 -22.1 32.5        -14.3 -21.2 34.4        -12.2 -24.1 33.1        -8.7 -24.1 34.5 -5.7 -23.6 32.3      -2.5 -24.5 34.2 0.9 -23.2 33.2  4.4 -23.9 34.3  6.1 -20.6 34.0  9.8 -20.0 34.3  10.1 -16.4 35.4 13.0 -14.0 35.5 13.3 -11.2 38.1      12.7 -10.5 41.8 9.7 -12.2 43.3  8.4 -12.0 46.8  5.2 -13.0 48.6  3.7 -9.5 48.4   3.8 -9.8 44.6   2.2 -13.2 44.8  -0.6 -11.7 47.0      -1.0 -8.9 44.5  -1.3 -11.3 41.5 -3.9 -13.4 43.4 -6.1 -10.5 44.4 -6.1 -8.6 41.1  -3.9 -6.2 39.3  -4.5 -3.7 36.4  -3.6 -4.8 32.9       -3.8 -2.6 29.8  -7.0 -4.4 29.0  -8.5 -4.4 32.5  -8.2 -6.2 35.8  -6.8 -9.7 35.9  -5.7 -12.2 38.6 -4.2 -15.6 38.7 -6.6 -18.1 40.2      -3.8 -20.0 41.8 -0.0 -19.4 42.0 2.4 -21.9 43.4  6.2 -21.7 43.7  7.6 -24.7 41.8  11.4 -24.5 42.2 14.2 -22.0 42.2 17.4 -20.5 41.0      18.6 -17.0 41.7 16.7 -14.6 39.4 15.1 -17.6 37.8 12.2 -19.4 39.4 9.4 -21.8 38.3  5.8 -21.1 39.2  2.7 -23.1 38.4  -0.3 -20.8 37.8      -3.8 -21.1 36.7 -6.3 -18.9 34.9 -9.9 -19.4 34.1 -9.9 -17.1 31.1 -7.6 -16.7 28.1 -6.8 -13.1 27.2 -4.0 -12.5 24.7 -0.6 -13.1 26.3      -2.2 -12.9 29.7 0.3 -14.9 31.8  3.2 -12.8 30.5  1.3 -9.5 31.1   0.4 -10.5 34.6  4.0 -11.2 35.6  5.1 -8.0 33.8   2.5 -6.0 35.8        4.6 -6.8 38.9   8.0 -6.3 37.2   8.8 -9.9 36.2   9.7 -11.4 32.9  8.9 -14.8 31.5  11.6 -17.1 30.0 9.5 -20.2 29.5  5.8 -21.0 29.3       4.3 -24.5 29.1  0.6 -25.3 29.3  0.5 -28.5 31.4  -3.2 -28.9 32.0 -6.5 -27.9 30.5 -9.5 -28.7 32.7 -9.1 -32.2 34.0 -6.7 -33.3 31.2      -3.0 -32.9 32.1 -0.4 -33.0 29.4 -0.1 -36.8 28.8 2.9 -38.3 30.7  4.2 -40.6 28.0  3.5 -38.2 25.2  6.4 -36.4 23.5  6.5 -32.9 22.0       4.6 -32.8 18.6  2.5 -35.9 19.2  -1.1 -36.5 20.3 -3.4 -39.1 21.9 -7.1 -39.7 22.0 -9.2 -39.8 25.2 -11.5 -42.8 25.8    -15.2 -42.7 26.7 -14.8 -42.3 30.4        -12.3 -39.5 30.0        -14.2 -37.7 27.3        -17.3 -38.0 29.6        -15.4 -36.5 32.6    -14.3 -33.6 30.4 -17.9 -33.1 29.4        -19.2 -33.2 33.0        -16.7 -30.4 33.8        -18.7 -28.1 31.6        -22.2 -29.4 32.6    -22.6 -31.6 29.5 -24.2 -35.0 30.3        -22.3 -38.2 29.6        -22.5 -39.9 26.3        -22.8 -43.7 25.4&#xA;&#xA;1XXIJ&#xA;334&#xA;M       R       W       Y       P       W       L       R       P       D       F       E       K       L       V       A       S   L&#xA;.       .       .       .       T       T       H       H       H       H       H       H       H       H       H       T       T   .&#xA;65.7 -16.7 128.5        64.5 -19.9 130.3        66.9 -22.7 131.5        66.0 -25.8 129.7        67.1 -29.4 130.9        69.2 -30.4 127.8     71.4 -27.4 128.6        72.9 -29.3 131.5        75.3 -32.0 130.1        76.9 -29.2 128.1        76.9 -26.7 131.0        78.6 -29.4 133.1     81.8 -29.7 131.1        81.8 -26.0 130.4        82.2 -25.2 134.1        84.3 -28.3 134.6        87.0 -26.5 132.5    86.5 -23.1 134.2 87.2 -24.8 137.5        90.3 -26.9 136.4        91.9 -24.1 134.3        91.2 -25.8 130.9        88.8 -22.9 130.1    88.9 -21.4 126.6 89.0 -17.6 126.7        86.9 -17.5 123.5        83.8 -19.6 123.0        81.6 -19.9 119.9        78.4 -21.8 120.5    76.4 -23.0 117.5 72.8 -23.9 117.9        69.4 -23.9 116.3        66.9 -21.3 117.2        65.1 -21.8 120.5        67.7 -24.4 121.7    68.4 -22.5 124.8 71.4 -20.9 123.2        70.2 -17.9 125.1        70.3 -19.4 128.5        73.6 -21.3 128.3        75.7 -18.2 127.8    74.1 -16.3 130.7 74.1 -19.4 133.0        77.8 -19.8 132.3        78.8 -16.2 133.0        77.0 -16.2 136.4        79.4 -18.9 137.5    82.4 -16.9 136.6 81.1 -13.6 138.0        80.6 -14.9 141.5        82.7 -13.6 144.4        81.9 -16.6 146.7        80.7 -19.6 144.5    79.2 -22.6 146.2 80.7 -25.3 143.9        78.7 -25.6 140.7        76.6 -22.5 141.0        76.6 -19.1 142.8        75.6 -18.0 146.3    73.1 -15.2 145.4 74.5 -12.7 148.0        78.0 -11.6 146.9        78.5 -8.3 145.0 78.7 -10.2 141.8        75.6 -12.3 142.3        73.7 -9.2 143.5      74.8 -6.8 140.8 74.2 -9.8 138.5 70.6 -10.2 139.7        70.4 -6.4 139.2 71.7 -7.1 135.7 74.0 -4.0 135.8 77.6 -5.1 136.4      81.1 -3.7 135.7 82.4 -7.0 134.2 79.5 -8.2 132.2 79.4 -6.7 128.7 76.5 -7.3 126.2 77.5 -6.8 122.6 74.1 -6.4 121.0 74.1 -5.1 117.4      73.4 -1.3 116.8 69.7 -0.7 117.7 67.9 -0.2 114.4 70.3 -1.8 112.0 70.7 -5.6 111.3 74.4 -5.6 111.8 77.1 -4.7 114.4 79.7 -1.9 114.3      83.5 -1.7 114.3 82.9 1.0 117.0  81.1 -1.0 119.7 83.9 -3.6 119.5 86.8 -1.2 119.9 85.1 0.5 122.9  84.7 -2.8 124.7 88.3 -3.9 123.7      89.8 -0.9 125.5 87.2 -1.3 128.2 88.8 -4.6 129.1 92.3 -3.2 128.8 91.9 -1.8 132.3 91.6 -3.7 135.7 88.3 -5.5 136.6 86.3 -3.2 138.9      85.3 -6.1 141.2 88.7 -7.8 141.0 88.2 -11.1 139.2        87.1 -12.3 135.8        85.6 -10.1 132.9        83.1 -11.6 130.4    81.8 -10.7 126.9 78.9 -12.0 124.9        77.8 -11.6 121.3        74.2 -12.0 120.1        75.2 -13.4 116.7        78.7 -13.1 115.2    77.5 -13.8 111.7 75.7 -10.4 111.7        78.8 -8.2 112.1 80.6 -5.9 109.6 83.7 -7.5 107.9 85.8 -4.8 109.5 84.2 -5.5 112.9 85.6 -9.1 112.6      89.2 -8.1 112.3 88.8 -5.4 114.9 87.8 -8.1 117.3 90.6 -10.6 116.4        93.6 -8.6 117.7 92.2 -8.7 121.2 91.9 -12.5 121.2    95.5 -12.5 120.1 96.5 -10.2 123.0        94.2 -11.8 125.7        93.9 -9.5 128.8 94.7 -11.9 131.7        91.7 -13.9 133.1        89.1 -12.4 130.8     86.3 -14.5 129.1        84.5 -14.4 125.7        81.3 -15.9 124.4        80.5 -15.9 120.7        77.3 -17.4 119.5    76.0 -18.2 116.0 73.5 -20.3 114.2        75.8 -20.6 111.2        79.5 -20.8 112.0        80.6 -21.2 108.4        79.5 -17.8 107.3    82.1 -16.6 109.9 85.7 -15.6 109.1        87.9 -18.7 109.1        90.7 -16.8 110.8        88.2 -15.8 113.5        86.5 -19.2 114.1    90.0 -20.6 114.6 90.7 -18.5 117.7        87.6 -19.7 119.5        86.5 -23.1 120.6        83.4 -24.2 118.7        80.9 -26.1 120.8    77.9 -27.9 119.3 74.6 -27.5 121.1        73.0 -30.7 119.6        69.2 -30.8 118.8        68.0 -33.4 121.2        66.8 -36.7 119.7    63.3 -36.2 118.3 61.6 -38.6 120.7        62.8 -36.6 123.6        61.9 -33.2 122.0        58.4 -34.3 121.2        57.6 -35.6 124.7    59.0 -32.4 126.2 56.8 -30.5 123.8        53.6 -32.5 124.5        53.8 -30.9 127.9        54.2 -27.3 126.9        51.2 -27.5 124.5    48.6 -30.1 123.9 48.5 -30.9 120.2        48.7 -34.1 118.1        52.2 -35.6 118.2        52.1 -35.5 114.4        51.8 -31.8 114.5    54.3 -31.6 117.3 56.9 -33.4 115.1        56.1 -31.1 112.2        56.9 -28.0 114.3        60.2 -29.7 115.5        61.2 -30.5 111.9    60.2 -27.0 110.8 62.8 -25.7 113.3        65.7 -27.9 112.4        65.1 -29.8 115.6        65.1 -26.7 117.9        62.5 -27.5 120.5    62.4 -23.9 121.6 61.6 -22.5 118.2        59.2 -25.4 117.7        57.4 -24.4 120.9        57.2 -20.8 119.7        55.7 -22.0 116.5    52.5 -22.6 118.4 52.3 -19.7 120.9        51.0 -17.2 118.3        48.2 -18.8 116.5        50.9 -19.8 114.0        50.0 -23.4 113.8    46.4 -22.5 113.1 47.8 -20.1 110.4        49.7 -23.1 109.1        46.6 -25.3 109.0        45.0 -22.6 106.8        47.8 -23.0 104.4    47.6 -26.8 104.0 43.8 -26.5 103.4        44.3 -24.0 100.7        47.1 -26.1 99.2 44.9 -29.2 99.2 42.3 -27.4 97.2 44.7 -26.0 94.7 46.9 -29.1 94.1      44.1 -31.1 92.5 42.2 -28.1 91.1 45.1 -26.2 89.4 43.8 -22.9 90.9 47.0 -22.0 92.7 46.0 -18.4 92.8 43.6 -19.4 95.6 46.6 -19.4 98.0      46.7 -15.7 97.7 44.6 -15.2 100.8        46.6 -17.8 102.7        50.0 -16.3 101.8        49.4 -12.6 101.4        50.1 -11.5 104.9     53.0 -9.8 106.8 54.0 -13.4 107.8        54.5 -14.0 104.1        58.1 -15.2 104.4        57.5 -17.6 107.2        54.6 -18.9 105.2     56.4 -19.2 101.9        59.0 -21.3 103.7        56.2 -23.6 105.0        55.0 -24.0 101.4        58.6 -24.8 100.4    59.0 -27.6 102.9 56.0 -29.3 101.4        57.5 -28.8 97.9 60.7 -30.3 99.0 59.0 -33.4 100.5        57.0 -34.1 97.3 60.3 -33.9 95.5 62.0 -36.3 97.9      59.2 -38.8 97.5 59.9 -39.0 93.7 63.3 -40.3 94.6 61.9 -42.6 97.2 63.3 -40.7 100.2        62.2 -42.2 103.5        62.3 -39.0 105.6     59.6 -36.3 105.3        59.3 -33.1 107.2        55.6 -32.7 106.8        54.4 -35.9 108.5        51.5 -33.8 109.6    49.8 -34.3 106.2 51.3 -37.2 104.2        48.2 -37.2 102.1        49.2 -33.8 100.8        52.8 -34.5 99.9 51.5 -37.6 98.1 48.9 -36.0 95.9      51.3 -33.1 95.3 54.0 -35.5 94.1 51.3 -37.3 92.0 50.2 -34.2 90.0 53.6 -32.6 89.7 56.6 -34.0 87.9 59.9 -33.8 89.7 60.9 -31.0 87.3      57.8 -28.9 87.8 58.3 -29.2 91.5 62.0 -28.0 91.2 60.8 -25.0 89.2 58.2 -24.0 91.7 60.5 -24.7 94.7 62.8 -22.0 93.4 60.4 -19.2 92.5      58.9 -19.8 95.9 62.4 -19.6 97.2 63.7 -16.5 95.4 60.4 -14.7 95.9 60.3 -15.3 99.6 63.7 -13.7 99.7 62.8 -10.5 97.9 60.1 -10.1 100.5     62.5 -10.4 103.3        65.4 -8.5 101.8 64.0 -5.6 99.5  62.8 -2.7 101.7 59.2 -2.6 100.6 57.9 -4.3 97.4  54.2 -5.0 96.9  53.5 -8.2 98.9       50.4 -8.5 96.9  51.8 -8.1 93.4  54.8 -10.2 93.8 52.9 -13.1 95.2 50.1 -12.8 92.6 52.9 -13.0 90.1 54.5 -16.0 91.9 51.1 -17.7 91.9      50.0 -17.1 88.3 53.5 -18.0 87.3 53.3 -21.3 89.1 50.3 -22.1 87.0 52.1 -21.6 83.7 54.8 -24.0 84.7 52.0 -26.5 85.2 50.8 -26.1 81.6      52.0 -28.7 79.0 54.8 -27.5 76.7 56.2 -24.8 78.9 60.0 -24.6 79.4 60.6 -24.0 83.1 63.6 -22.0 84.5 67.2 -23.0 85.4 69.4 -21.4 88.1      71.8 -18.6 87.1 75.6 -17.8 87.3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Example command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'NR%6 != 0 {print} NR%6==0 {res=(sqrt((&lt;span class=&quot;math-container&quot;&gt;$1-x)^2 + ($&lt;/span&gt;2-y)^2 + (&lt;span class=&quot;math-container&quot;&gt;$3-z)^2)); if(res &amp;lt;= 4.5){printf &quot;%f\n&quot;, res} else{print}} {x=$1; y=$2; z=$&lt;/span&gt;3}'&#xA;&#xA;4LGTD&#xA;247&#xA;M       S       E       K       L       Q       K       V       L       A       R       A       G       H       G       S       R   T&#xA;.       .       E       E       H       H       H       H       H       H       H       T       T       S       S       .       H   .&#xA;-34.3 -28.9 16.3        -33.0 -26.8 13.4        -29.4 -25.4 13.3        -27.9 -22.6 11.2        -26.9 -24.1 7.8 -23.3 -22.9 8.2 -23.0 -24.7 11.5     -24.1 -27.9 9.9 -21.8 -27.7 6.9 -18.8 -26.7 9.0 -19.5 -29.7 11.3        -19.6 -31.9 8.2 -16.2 -30.7 7.2 -16.7 -28.2 4.4      -15.0 -25.1 5.8 -15.4 -22.7 8.7 -18.8 -21.3 9.7 -17.7 -17.9 8.3 -16.3 -19.3 5.1 -19.5 -21.2 4.5 -21.5 -18.0 5.1 -19.7 -16.2 2.3      -20.8 -19.1 -0.0        -24.4 -19.2 1.2 -24.6 -15.4 0.6 -23.5 -16.0 -3.0        -26.4 -18.4 -3.6        -24.1 -21.4 -4.0    -25.9 -23.6 -1.5 -29.3 -25.2 -2.0        -31.4 -27.0 0.5 -34.0 -29.5 -0.8        -33.9 -27.8 -4.2        -34.1 -24.2 -3.0        -31.3 -21.7 -2.9     -30.7 -21.0 0.8 -30.2 -17.5 2.2 -27.9 -16.3 4.9 -29.4 -17.1 8.3 -31.3 -20.1 7.0 -31.7 -23.1 9.4 -32.1 -26.8 8.7 -32.6 -30.1 10.5     -30.2 -32.9 9.7 -32.0 -36.0 8.4 -31.6 -38.7 5.7 -34.0 -36.6 3.6 -32.0 -33.4 3.7 -30.4 -32.7 0.4 -27.7 -30.0 0.6 -25.8 -29.0 -2.5     -23.0 -26.6 -3.0        -22.5 -25.6 -6.6        -25.0 -28.3 -7.5        -22.9 -31.1 -5.9        -24.6 -33.1 -3.2    -23.0 -33.1 0.2  -23.1 -36.0 2.6 -23.7 -34.9 6.2 -23.8 -36.9 9.4 -27.2 -37.3 11.1        -28.5 -37.1 14.7        -19.2 -38.7 17.8    -17.7 -36.6 20.6 -14.0 -37.3 20.3        -11.2 -35.4 22.0        -7.4 -35.3 21.2 -4.7 -34.3 23.6 -1.9 -32.6 21.6 1.6 -31.6 22.7  2.2 -28.4 20.7       5.8 -27.5 20.1  7.1 -23.9 19.9  7.1 -22.6 16.3  3.6 -24.0 15.4  0.7 -21.6 14.7  -2.8 -22.2 16.0 -4.8 -21.4 12.8 -6.4 -23.3 9.9       -4.8 -20.9 7.5  -1.6 -18.8 8.1  -1.1 -16.0 5.5  2.7 -16.4 6.0   3.0 -20.1 5.2   4.2 -21.3 8.7   3.5 -24.7 10.3  0.1 -24.9 11.9       -1.1 -27.3 14.7 -3.6 -28.9 12.3 -1.0 -30.0 9.8  -0.7 -33.4 11.5 -4.0 -34.7 12.9 -6.1 -37.9 12.3 -8.5 -37.7 9.3  -12.2 -38.1 10.1     -15.3 -39.0 8.2 -17.5 -36.0 7.6 -16.2 -34.0 10.6        -13.2 -31.8 11.6        -10.9 -31.0 14.5        -11.7 -27.8 16.4     -8.8 -26.3 18.5 -9.8 -25.6 22.1 -8.4 -22.1 22.1 -5.0 -20.9 20.9 -1.5 -21.1 22.2 1.2 -18.6 21.2  3.8 -19.7 18.6  6.6 -20.0 21.1       4.5 -21.3 23.9  4.7 -25.1 24.5  2.3 -27.7 25.8  -1.3 -29.0 25.9 -3.8 -28.1 23.2 -7.0 -30.0 23.1 -8.9 -30.7 19.9 -12.5 -32.0 19.6     -14.9 -32.8 16.9        -17.6 -31.5 19.2        -17.9 -27.7 18.8        -20.2 -27.3 21.8        -17.6 -29.0 24.0        -15.0 -26.8 22.4     -16.8 -23.5 23.1        -17.7 -24.6 26.6        -14.1 -25.3 27.4        -12.9 -21.9 26.0        -15.6 -19.6 27.2    -14.8 -18.2 30.6 -18.4 -18.5 31.9        -17.5 -22.1 32.5        -14.3 -21.2 34.4        -12.2 -24.1 33.1        -8.7 -24.1 34.5 -5.7 -23.6 32.3      -2.5 -24.5 34.2 0.9 -23.2 33.2  4.4 -23.9 34.3  6.1 -20.6 34.0  9.8 -20.0 34.3  10.1 -16.4 35.4 13.0 -14.0 35.5 13.3 -11.2 38.1      12.7 -10.5 41.8 9.7 -12.2 43.3  8.4 -12.0 46.8  5.2 -13.0 48.6  3.7 -9.5 48.4   3.8 -9.8 44.6   2.2 -13.2 44.8  -0.6 -11.7 47.0      -1.0 -8.9 44.5  -1.3 -11.3 41.5 -3.9 -13.4 43.4 -6.1 -10.5 44.4 -6.1 -8.6 41.1  -3.9 -6.2 39.3  -4.5 -3.7 36.4  -3.6 -4.8 32.9       -3.8 -2.6 29.8  -7.0 -4.4 29.0  -8.5 -4.4 32.5  -8.2 -6.2 35.8  -6.8 -9.7 35.9  -5.7 -12.2 38.6 -4.2 -15.6 38.7 -6.6 -18.1 40.2      -3.8 -20.0 41.8 -0.0 -19.4 42.0 2.4 -21.9 43.4  6.2 -21.7 43.7  7.6 -24.7 41.8  11.4 -24.5 42.2 14.2 -22.0 42.2 17.4 -20.5 41.0      18.6 -17.0 41.7 16.7 -14.6 39.4 15.1 -17.6 37.8 12.2 -19.4 39.4 9.4 -21.8 38.3  5.8 -21.1 39.2  2.7 -23.1 38.4  -0.3 -20.8 37.8      -3.8 -21.1 36.7 -6.3 -18.9 34.9 -9.9 -19.4 34.1 -9.9 -17.1 31.1 -7.6 -16.7 28.1 -6.8 -13.1 27.2 -4.0 -12.5 24.7 -0.6 -13.1 26.3      -2.2 -12.9 29.7 0.3 -14.9 31.8  3.2 -12.8 30.5  1.3 -9.5 31.1   0.4 -10.5 34.6  4.0 -11.2 35.6  5.1 -8.0 33.8   2.5 -6.0 35.8        4.6 -6.8 38.9   8.0 -6.3 37.2   8.8 -9.9 36.2   9.7 -11.4 32.9  8.9 -14.8 31.5  11.6 -17.1 30.0 9.5 -20.2 29.5  5.8 -21.0 29.3       4.3 -24.5 29.1  0.6 -25.3 29.3  0.5 -28.5 31.4  -3.2 -28.9 32.0 -6.5 -27.9 30.5 -9.5 -28.7 32.7 -9.1 -32.2 34.0 -6.7 -33.3 31.2      -3.0 -32.9 32.1 -0.4 -33.0 29.4 -0.1 -36.8 28.8 2.9 -38.3 30.7  4.2 -40.6 28.0  3.5 -38.2 25.2  6.4 -36.4 23.5  6.5 -32.9 22.0       4.6 -32.8 18.6  2.5 -35.9 19.2  -1.1 -36.5 20.3 -3.4 -39.1 21.9 -7.1 -39.7 22.0 -9.2 -39.8 25.2 -11.5 -42.8 25.8    -15.2 -42.7 26.7 -14.8 -42.3 30.4        -12.3 -39.5 30.0        -14.2 -37.7 27.3        -17.3 -38.0 29.6        -15.4 -36.5 32.6    -14.3 -33.6 30.4 -17.9 -33.1 29.4        -19.2 -33.2 33.0        -16.7 -30.4 33.8        -18.7 -28.1 31.6        -22.2 -29.4 32.6    -22.6 -31.6 29.5 -24.2 -35.0 30.3        -22.3 -38.2 29.6        -22.5 -39.9 26.3        -22.8 -43.7 25.4&#xA;&#xA;1XXIJ&#xA;334&#xA;M       R       W       Y       P       W       L       R       P       D       F       E       K       L       V       A       S   L&#xA;.       .       .       .       T       T       H       H       H       H       H       H       H       H       H       T       T   .&#xA;65.7 -16.7 128.5        64.5 -19.9 130.3        66.9 -22.7 131.5        66.0 -25.8 129.7        67.1 -29.4 130.9        69.2 -30.4 127.8     71.4 -27.4 128.6        72.9 -29.3 131.5        75.3 -32.0 130.1        76.9 -29.2 128.1        76.9 -26.7 131.0        78.6 -29.4 133.1     81.8 -29.7 131.1        81.8 -26.0 130.4        82.2 -25.2 134.1        84.3 -28.3 134.6        87.0 -26.5 132.5    86.5 -23.1 134.2 87.2 -24.8 137.5        90.3 -26.9 136.4        91.9 -24.1 134.3        91.2 -25.8 130.9        88.8 -22.9 130.1    88.9 -21.4 126.6 89.0 -17.6 126.7        86.9 -17.5 123.5        83.8 -19.6 123.0        81.6 -19.9 119.9        78.4 -21.8 120.5    76.4 -23.0 117.5 72.8 -23.9 117.9        69.4 -23.9 116.3        66.9 -21.3 117.2        65.1 -21.8 120.5        67.7 -24.4 121.7    68.4 -22.5 124.8 71.4 -20.9 123.2        70.2 -17.9 125.1        70.3 -19.4 128.5        73.6 -21.3 128.3        75.7 -18.2 127.8    74.1 -16.3 130.7 74.1 -19.4 133.0        77.8 -19.8 132.3        78.8 -16.2 133.0        77.0 -16.2 136.4        79.4 -18.9 137.5    82.4 -16.9 136.6 81.1 -13.6 138.0        80.6 -14.9 141.5        82.7 -13.6 144.4        81.9 -16.6 146.7        80.7 -19.6 144.5    79.2 -22.6 146.2 80.7 -25.3 143.9        78.7 -25.6 140.7        76.6 -22.5 141.0        76.6 -19.1 142.8        75.6 -18.0 146.3    73.1 -15.2 145.4 74.5 -12.7 148.0        78.0 -11.6 146.9        78.5 -8.3 145.0 78.7 -10.2 141.8        75.6 -12.3 142.3        73.7 -9.2 143.5      74.8 -6.8 140.8 74.2 -9.8 138.5 70.6 -10.2 139.7        70.4 -6.4 139.2 71.7 -7.1 135.7 74.0 -4.0 135.8 77.6 -5.1 136.4      81.1 -3.7 135.7 82.4 -7.0 134.2 79.5 -8.2 132.2 79.4 -6.7 128.7 76.5 -7.3 126.2 77.5 -6.8 122.6 74.1 -6.4 121.0 74.1 -5.1 117.4      73.4 -1.3 116.8 69.7 -0.7 117.7 67.9 -0.2 114.4 70.3 -1.8 112.0 70.7 -5.6 111.3 74.4 -5.6 111.8 77.1 -4.7 114.4 79.7 -1.9 114.3      83.5 -1.7 114.3 82.9 1.0 117.0  81.1 -1.0 119.7 83.9 -3.6 119.5 86.8 -1.2 119.9 85.1 0.5 122.9  84.7 -2.8 124.7 88.3 -3.9 123.7      89.8 -0.9 125.5 87.2 -1.3 128.2 88.8 -4.6 129.1 92.3 -3.2 128.8 91.9 -1.8 132.3 91.6 -3.7 135.7 88.3 -5.5 136.6 86.3 -3.2 138.9      85.3 -6.1 141.2 88.7 -7.8 141.0 88.2 -11.1 139.2        87.1 -12.3 135.8        85.6 -10.1 132.9        83.1 -11.6 130.4    81.8 -10.7 126.9 78.9 -12.0 124.9        77.8 -11.6 121.3        74.2 -12.0 120.1        75.2 -13.4 116.7        78.7 -13.1 115.2    77.5 -13.8 111.7 75.7 -10.4 111.7        78.8 -8.2 112.1 80.6 -5.9 109.6 83.7 -7.5 107.9 85.8 -4.8 109.5 84.2 -5.5 112.9 85.6 -9.1 112.6      89.2 -8.1 112.3 88.8 -5.4 114.9 87.8 -8.1 117.3 90.6 -10.6 116.4        93.6 -8.6 117.7 92.2 -8.7 121.2 91.9 -12.5 121.2    95.5 -12.5 120.1 96.5 -10.2 123.0        94.2 -11.8 125.7        93.9 -9.5 128.8 94.7 -11.9 131.7        91.7 -13.9 133.1        89.1 -12.4 130.8     86.3 -14.5 129.1        84.5 -14.4 125.7        81.3 -15.9 124.4        80.5 -15.9 120.7        77.3 -17.4 119.5    76.0 -18.2 116.0 73.5 -20.3 114.2        75.8 -20.6 111.2        79.5 -20.8 112.0        80.6 -21.2 108.4        79.5 -17.8 107.3    82.1 -16.6 109.9 85.7 -15.6 109.1        87.9 -18.7 109.1        90.7 -16.8 110.8        88.2 -15.8 113.5        86.5 -19.2 114.1    90.0 -20.6 114.6 90.7 -18.5 117.7        87.6 -19.7 119.5        86.5 -23.1 120.6        83.4 -24.2 118.7        80.9 -26.1 120.8    77.9 -27.9 119.3 74.6 -27.5 121.1        73.0 -30.7 119.6        69.2 -30.8 118.8        68.0 -33.4 121.2        66.8 -36.7 119.7    63.3 -36.2 118.3 61.6 -38.6 120.7        62.8 -36.6 123.6        61.9 -33.2 122.0        58.4 -34.3 121.2        57.6 -35.6 124.7    59.0 -32.4 126.2 56.8 -30.5 123.8        53.6 -32.5 124.5        53.8 -30.9 127.9        54.2 -27.3 126.9        51.2 -27.5 124.5    48.6 -30.1 123.9 48.5 -30.9 120.2        48.7 -34.1 118.1        52.2 -35.6 118.2        52.1 -35.5 114.4        51.8 -31.8 114.5    54.3 -31.6 117.3 56.9 -33.4 115.1        56.1 -31.1 112.2        56.9 -28.0 114.3        60.2 -29.7 115.5        61.2 -30.5 111.9    60.2 -27.0 110.8 62.8 -25.7 113.3        65.7 -27.9 112.4        65.1 -29.8 115.6        65.1 -26.7 117.9        62.5 -27.5 120.5    62.4 -23.9 121.6 61.6 -22.5 118.2        59.2 -25.4 117.7        57.4 -24.4 120.9        57.2 -20.8 119.7        55.7 -22.0 116.5    52.5 -22.6 118.4 52.3 -19.7 120.9        51.0 -17.2 118.3        48.2 -18.8 116.5        50.9 -19.8 114.0        50.0 -23.4 113.8    46.4 -22.5 113.1 47.8 -20.1 110.4        49.7 -23.1 109.1        46.6 -25.3 109.0        45.0 -22.6 106.8        47.8 -23.0 104.4    47.6 -26.8 104.0 43.8 -26.5 103.4        44.3 -24.0 100.7        47.1 -26.1 99.2 44.9 -29.2 99.2 42.3 -27.4 97.2 44.7 -26.0 94.7 46.9 -29.1 94.1      44.1 -31.1 92.5 42.2 -28.1 91.1 45.1 -26.2 89.4 43.8 -22.9 90.9 47.0 -22.0 92.7 46.0 -18.4 92.8 43.6 -19.4 95.6 46.6 -19.4 98.0      46.7 -15.7 97.7 44.6 -15.2 100.8        46.6 -17.8 102.7        50.0 -16.3 101.8        49.4 -12.6 101.4        50.1 -11.5 104.9     53.0 -9.8 106.8 54.0 -13.4 107.8        54.5 -14.0 104.1        58.1 -15.2 104.4        57.5 -17.6 107.2        54.6 -18.9 105.2     56.4 -19.2 101.9        59.0 -21.3 103.7        56.2 -23.6 105.0        55.0 -24.0 101.4        58.6 -24.8 100.4    59.0 -27.6 102.9 56.0 -29.3 101.4        57.5 -28.8 97.9 60.7 -30.3 99.0 59.0 -33.4 100.5        57.0 -34.1 97.3 60.3 -33.9 95.5 62.0 -36.3 97.9      59.2 -38.8 97.5 59.9 -39.0 93.7 63.3 -40.3 94.6 61.9 -42.6 97.2 63.3 -40.7 100.2        62.2 -42.2 103.5        62.3 -39.0 105.6     59.6 -36.3 105.3        59.3 -33.1 107.2        55.6 -32.7 106.8        54.4 -35.9 108.5        51.5 -33.8 109.6    49.8 -34.3 106.2 51.3 -37.2 104.2        48.2 -37.2 102.1        49.2 -33.8 100.8        52.8 -34.5 99.9 51.5 -37.6 98.1 48.9 -36.0 95.9      51.3 -33.1 95.3 54.0 -35.5 94.1 51.3 -37.3 92.0 50.2 -34.2 90.0 53.6 -32.6 89.7 56.6 -34.0 87.9 59.9 -33.8 89.7 60.9 -31.0 87.3      57.8 -28.9 87.8 58.3 -29.2 91.5 62.0 -28.0 91.2 60.8 -25.0 89.2 58.2 -24.0 91.7 60.5 -24.7 94.7 62.8 -22.0 93.4 60.4 -19.2 92.5      58.9 -19.8 95.9 62.4 -19.6 97.2 63.7 -16.5 95.4 60.4 -14.7 95.9 60.3 -15.3 99.6 63.7 -13.7 99.7 62.8 -10.5 97.9 60.1 -10.1 100.5     62.5 -10.4 103.3        65.4 -8.5 101.8 64.0 -5.6 99.5  62.8 -2.7 101.7 59.2 -2.6 100.6 57.9 -4.3 97.4  54.2 -5.0 96.9  53.5 -8.2 98.9       50.4 -8.5 96.9  51.8 -8.1 93.4  54.8 -10.2 93.8 52.9 -13.1 95.2 50.1 -12.8 92.6 52.9 -13.0 90.1 54.5 -16.0 91.9 51.1 -17.7 91.9      50.0 -17.1 88.3 53.5 -18.0 87.3 53.3 -21.3 89.1 50.3 -22.1 87.0 52.1 -21.6 83.7 54.8 -24.0 84.7 52.0 -26.5 85.2 50.8 -26.1 81.6      52.0 -28.7 79.0 54.8 -27.5 76.7 56.2 -24.8 78.9 60.0 -24.6 79.4 60.6 -24.0 83.1 63.6 -22.0 84.5 67.2 -23.0 85.4 69.4 -21.4 88.1      71.8 -18.6 87.1 75.6 -17.8 87.3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;NB. in both of these 'blocks' the equation is &amp;gt;4.5, so the entire line is printed. Is this your desired outcome?&lt;/p&gt;&#xA;" OwnerUserId="9098" LastEditorUserId="9098" LastEditDate="2023-02-23T22:37:52.223" LastActivityDate="2023-02-23T22:37:52.223" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20608" PostTypeId="1" AcceptedAnswerId="20611" CreationDate="2023-02-22T09:53:14.627" Score="3" ViewCount="81" Body="&lt;p&gt;I'm a beginner in bioinformatics working with environmental metabarcoding, and while reading up on different marker genes, I understand that COI is preferably used for animals thanks to its higher variability over 18S (although it depends on which taxonomic resolution fits best and which reference data is available). What I don't understand though, and can't find an answer to, is why COI metabarcoding isn't used for bacteria and archaea as well. Why is 16S preferred, although bacteria and archaea clearly have the COI gene? Is it not variable enough? Or are the primer binding sites too variable? Something else?&lt;/p&gt;&#xA;" OwnerUserId="17054" LastEditorUserId="3848" LastEditDate="2023-06-24T15:25:20.563" LastActivityDate="2023-06-24T15:25:20.563" Title="Why is COI metabarcoding not used for prokaryotes?" Tags="|phylogenetics|phylogeny|barcode|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20609" PostTypeId="1" CreationDate="2023-02-22T10:15:23.150" Score="1" ViewCount="72" Body="&lt;p&gt;I have next data:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;age - how old is person, e.g. 17 or 25&lt;/li&gt;&#xA;&lt;li&gt;bmi - body mass index,  e.g. 18.25 or 29.35&lt;/li&gt;&#xA;&lt;li&gt;rsXXX - results of genotyping in rs(point), e.g. rs10852521: C/C, C/T and T/T&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I devide all data in 2 groups &lt;code&gt;bmi_group&lt;/code&gt;, create new column bmi_group, remove all rows not in group&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;low - bmi&amp;lt;25.0&lt;/li&gt;&#xA;&lt;li&gt;high - bmi&amp;gt;=30.0&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;LG = 25.0&#xA;HG = 30.0&#xA;subdata = subset(all_data, bmi &amp;lt; LG | bmi &amp;gt;= HG)&#xA;subdata$bmi_group &amp;lt;- with(subdata, ifelse(bmi &amp;lt; LG, 'low', 'high'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I need to find out if a certain allelic variant affects the risk of obesity, taking into account age. I try next:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; model &amp;lt;- glm(imt_group ~ rs11075990+age, data = point.data, family = binomial)&#xA;&amp;gt; summary(model)&#xA;&#xA;Deviance Residuals: &#xA;    Min       1Q   Median       3Q      Max  &#xA;-2.4209  -0.6968   0.4239   0.6470   2.0662  &#xA;&#xA;Coefficients:&#xA;              Estimate Std. Error z value Pr(&amp;gt;|z|)    &#xA;(Intercept)    6.07127    0.56333  10.778   &amp;lt;2e-16 ***&#xA;rs11075990A/G -0.65093    0.28982  -2.246   0.0247 *  &#xA;rs11075990G/G -0.53818    0.35860  -1.501   0.1334    &#xA;age           -0.12783    0.01235 -10.351   &amp;lt;2e-16 ***&#xA;---&#xA;Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1&#xA;&#xA;(Dispersion parameter for binomial family taken to be 1)&#xA;&#xA;    Null deviance: 568.58  on 430  degrees of freedom&#xA;Residual deviance: 403.53  on 427  degrees of freedom&#xA;AIC: 411.53&#xA;&#xA;Number of Fisher Scoring iterations: 4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Problem is genotyping data not binominal - &lt;code&gt;Factor w/ 3 levels &amp;quot;A/A&amp;quot;,&amp;quot;A/G&amp;quot;,&amp;quot;G/G&amp;quot;:&lt;/code&gt;. What and how can I do?&lt;/p&gt;&#xA;" OwnerUserId="3108" LastActivityDate="2023-02-22T12:44:57.083" Title="GLM for multinomial using r - results of genotyping" Tags="|r|statistics|molecular-genetics|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20610" PostTypeId="2" ParentId="20609" CreationDate="2023-02-22T12:44:57.083" Score="3" Body="&lt;p&gt;I do not see a problem here. There are 3 levels corresponding to the three genotypes and genotype &lt;code&gt;A/A&lt;/code&gt; is the base (reference) level and the &amp;quot;effects&amp;quot; of the other two genotypes are stated &amp;quot;on top of the base level&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://courses.atlas.illinois.edu/spring2016/STAT/STAT200/RProgramming/RegressionFactors.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;This tutorial&lt;/a&gt; on linear regression with factor levels is very useful to understand how to conduct linear regression in general but also goes over the specifics of doing so with R.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, you created a new discrete (factor) variable out of a continuous variable when you bin BMI status as low or high. You can certainly do so but you can also use it as it is, as a continuous variable. And I guess you forgot to include this variable in your model.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2023-02-22T12:44:57.083" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20611" PostTypeId="2" ParentId="20608" CreationDate="2023-02-22T15:00:32.380" Score="2" Body="&lt;p&gt;The exact answer to your question is bacteria do not have a mitochondrion, therefore &lt;em&gt;cox1&lt;/em&gt; is not a universal gene combining both Archaebacteria and bacteria. In contrast 16S is universal for these two kingdoms AND it is also single copy. Moreover, bacteria have diverse tropisms beyond the Krebs or Calvin cycles. Extensive studies have been performed on 16S and it provides an excellent taxonomic target in bacteria.&lt;/p&gt;&#xA;&lt;p&gt;18S is a problematic target in eukaryotes because it can paralogous and that is not an ideal target in taxonomic studies, but has nicely conserved primer site: thus single copy genes are required/preferred. In addition, 18S is prone to indels (insertions/deletions) and these are difficult to model, align and understand the underlying molecular mechanism or selection advantage.&lt;/p&gt;&#xA;&lt;p&gt;Every animal on earth has a mitochondrion and therefore &lt;em&gt;cox1&lt;/em&gt;. Mitochondrion genes are guaranteed to be single copy in higher eukaryotes and &lt;em&gt;cox1&lt;/em&gt; has been extensively sequenced throughout metazoans. Thereby barcoding is appropriate because there's an exact reference to identify a given species is usually available. It has been extremely successful in insects which are the most biodiverse and species rich taxonomic Class on earth: there are more insects than all other higher eukaryotic species combined.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;In answer to the comments ...&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Whether mitochondrion were originally bacteria is a very difficult question and remains a theory (compartmentalisation theory - very old). I have not seen any rigorous phylogenetic support for this.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;em&gt;cox1&lt;/em&gt; (COI) or &lt;em&gt;cox2&lt;/em&gt; (COI) is not universal to bacteria, e.g. &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0005272820301547&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Its primary use is for aerobic organisms (oxygen consuming). Thus &lt;em&gt;cox1&lt;/em&gt; has a strong Krebs cycle association but there are a multitude of anaerobic pathways.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Anaerobic bacteria don't necessarily need &lt;em&gt;cox1&lt;/em&gt;/&lt;em&gt;cox2&lt;/em&gt; and there are a lot of obligate anaerobic bacteria in addition to non-obligatory anaerobic bacteria (which can do both).&lt;/p&gt;&#xA;&lt;p&gt;Marine metazoan are obligate aerobic organisms.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;There is no reason to suppose bacteria  are strictly  single copy &lt;em&gt;cox1&lt;/em&gt;. mtDNA is tiny and hence copy number heavily constrained, bacterial genomes by contrast are much larger and could easily accommodate duplication events.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;There isn't much work on bacterial &lt;em&gt;cox1&lt;/em&gt;, it has been pursued to attempt to augment antibiotic therapy. &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0005272820301547&quot; rel=&quot;nofollow noreferrer&quot;&gt;Degli Esposti (2020)&lt;/a&gt; is the closest I know to a comprehensive discussion on the bacterial &lt;em&gt;cox&lt;/em&gt; genes.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-02-23T09:32:46.120" LastActivityDate="2023-02-23T09:32:46.120" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20612" PostTypeId="1" CreationDate="2023-02-22T15:58:36.863" Score="1" ViewCount="24" Body="&lt;p&gt;I have done some gene expression work using two Nanostring panels (glial profiling and neuroinflammation) on the same 18 tissue samples. There are some overlapping genes between the panels, about 150 of 1400. Does anyone have any suggestions, possibly using a R package but open to other software, for how to normalize the gene expression data (raw count data) and deal with the overlapping genes between the two panels from the same samples. I have found some normalization packages (RUVSeq) for R but not yet something to handle the overlapping genes.&lt;/p&gt;&#xA;&lt;p&gt;Thank you,&lt;br /&gt;&#xA;Carolyn&lt;/p&gt;&#xA;" OwnerUserId="17057" LastEditorUserId="650" LastEditDate="2023-02-22T18:18:11.630" LastActivityDate="2023-02-22T18:18:11.630" Title="Normalization and Merging of Gene Expression Panels with Overlapping Genes from Same Cohort" Tags="|r|gene-expression|normalization|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20614" PostTypeId="2" ParentId="20605" CreationDate="2023-02-22T20:58:21.687" Score="1" Body="&lt;p&gt;If I have understood the question is to generalise the column names to permit a function to accept dataframes where you don't have the same column name, or it's different from the previous dataframe column names. Here the column could be called &lt;code&gt;RNA1_Approved_Symbol&lt;/code&gt; or &lt;code&gt;RNA2_Approved_Symbol&lt;/code&gt; in addition a different column could be called &lt;code&gt;RNA2_ID&lt;/code&gt; or &lt;code&gt;RNA_ID&lt;/code&gt;. No problem providing the column order is identical for the columns in question. If the orders are not identical see the code below the &lt;code&gt;----&lt;/code&gt; to correct that.&lt;/p&gt;&#xA;&lt;p&gt;If the order is identical the simple answer is &lt;code&gt;df.columns[i]&lt;/code&gt;. Thus it's obtaining the column name on its position within the dataframe (&lt;code&gt;columns&lt;/code&gt; command), where &lt;code&gt;i&lt;/code&gt; is the column location.&lt;/p&gt;&#xA;&lt;p&gt;Example, to rename &lt;code&gt;RNA1_ID&lt;/code&gt; to &lt;code&gt;RNA1_ID_new&lt;/code&gt;, or &lt;code&gt;RNA2_ID&lt;/code&gt; to &lt;code&gt;RNA2_ID_new&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import re&#xA;&#xA;df1 = pd.DataFrame({&amp;quot;perch&amp;quot;: [5,6,7], &amp;quot;RNA1_ID&amp;quot;: [1,2,4], &amp;quot;roach&amp;quot;: [30,42,51]})&#xA;df2 = pd.DataFrame({&amp;quot;perch&amp;quot;: [5,6,7], &amp;quot;RNA2_ID&amp;quot;: [1,2,4], &amp;quot;roach&amp;quot;: [30,42,51]})&#xA;&#xA;def renamecolumn(df):&#xA;    myname = df.columns[1]&#xA;    newname = re.sub(r'$','_new', myname)&#xA;    df = df.rename(columns={myname: newname})&#xA;    return df&#xA;&#xA;print (renamecolumn(df1))&#xA;print (renamecolumn(df2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output,&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;-&lt;/th&gt;&#xA;&lt;th&gt;perch&lt;/th&gt;&#xA;&lt;th&gt;RNA1_ID_new&lt;/th&gt;&#xA;&lt;th&gt;roach&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;5&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;30&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;6&lt;/td&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;td&gt;42&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;td&gt;7&lt;/td&gt;&#xA;&lt;td&gt;4&lt;/td&gt;&#xA;&lt;td&gt;51&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;-&lt;/th&gt;&#xA;&lt;th&gt;perch&lt;/th&gt;&#xA;&lt;th&gt;RNA2_ID_new&lt;/th&gt;&#xA;&lt;th&gt;roach&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;5&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;30&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;6&lt;/td&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;td&gt;42&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;td&gt;7&lt;/td&gt;&#xA;&lt;td&gt;4&lt;/td&gt;&#xA;&lt;td&gt;51&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;I noted there was a fair number of &lt;code&gt;set_index&lt;/code&gt; commands so the earlier the column name can be trapped inside a variable the better, because &lt;code&gt;set_index&lt;/code&gt; will change the column order.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Alternatively, the columns can be re-arranged to be in a given order if the original order between &lt;code&gt;df1&lt;/code&gt; and &lt;code&gt;df2&lt;/code&gt; is different. Here I'll move the column to the front.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mycols = df1.columns.tolist()&#xA;mycols = mycols[-2:] + mycols[:-2]&#xA;df1 = df1[mycols]&#xA;print (df1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;-&lt;/th&gt;&#xA;&lt;th&gt;RNA1_ID&lt;/th&gt;&#xA;&lt;th&gt;roach&lt;/th&gt;&#xA;&lt;th&gt;perch&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;30&lt;/td&gt;&#xA;&lt;td&gt;5&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;td&gt;42&lt;/td&gt;&#xA;&lt;td&gt;6&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;td&gt;4&lt;/td&gt;&#xA;&lt;td&gt;51&lt;/td&gt;&#xA;&lt;td&gt;7&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;Thus &lt;code&gt;RNA1_ID&lt;/code&gt; has been moved from the second to the first column. If &lt;code&gt;df2&lt;/code&gt; was in the last column repeat as above use &lt;code&gt;-1&lt;/code&gt; instead of &lt;code&gt;-2&lt;/code&gt; and both columns will now be at the start of the dataframe. Modify the core dataframe generalisation renaming accordingly, viz.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;myname = df.columns[0]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... sorted.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-02-22T21:13:06.633" LastActivityDate="2023-02-22T21:13:06.633" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20615" PostTypeId="2" ParentId="20594" CreationDate="2023-02-22T21:48:26.027" Score="1" Body="&lt;p&gt;Ok I understand better now so I'll write an answer.&lt;/p&gt;&#xA;&lt;p&gt;Look at the &lt;a href=&quot;https://www.worldclim.org/data/bioclim.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioclimatic data site&lt;/a&gt;. You can &lt;a href=&quot;https://www.worldclim.org/data/worldclim21.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;download them from the main page&lt;/a&gt;, but the suggestion to generate them from the time series data using an R package (&lt;code&gt;dismo&lt;/code&gt;). The bioclimatic variables are thus various linear combinations of the 3 variable time series input.&lt;/p&gt;&#xA;&lt;p&gt;Not having run this myself, I speculate, but it doesn't make sense to me that the bioclimatic variables are also time series, they appear to be summaries across the time series. So they're not &lt;em&gt;exactly&lt;/em&gt; identical in information content, but they're close enough that I'm not sure it's worth the effort of figuring out how to munge the monthly data into a format that works with the summaries.&lt;/p&gt;&#xA;&lt;p&gt;Someone already went to a lot of trouble to think about the biologically relevant representation of this data (e.g. what ANUCLIM does). I recommend relying on that work, or at the least &lt;a href=&quot;https://anuclimate.wordpress.com/anuclim-6-1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;looking at their arguments&lt;/a&gt; and deciding whether you agree enough to use their summaries.&lt;/p&gt;&#xA;&lt;p&gt;It doesn't particularly matter what software you use, I'd argue (probably MaxEnt the species distribution package, but you haven't clarified). Most methods are going to be doing similar things with this data. They are going to find the variables with the best explanation and weigth them more heavily. Throwing in a couple extra correlated variables might get you a little margin, but it is unlikely to increase explanatory power.&lt;/p&gt;&#xA;&lt;p&gt;If you want a formal methodology to compare the two, you could use something like &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwiR4-Kajar9AhVAHTQIHYFbCQAQFnoECBUQAQ&amp;amp;url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FAkaike_information_criterion&amp;amp;usg=AOvVaw06Nlj5xuEVqROYWL211LKF&quot; rel=&quot;nofollow noreferrer&quot;&gt;AIC&lt;/a&gt; or &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwia1Iyqjar9AhVPHjQIHfqoDKIQFnoECAoQAQ&amp;amp;url=https%3A%2F%2Fen.wikipedia.org%2Fwiki%2FBayesian_information_criterion&amp;amp;usg=AOvVaw2oneRWiP5wooH0bV9PrdiL&quot; rel=&quot;nofollow noreferrer&quot;&gt;BIC&lt;/a&gt; to compare the models formally with the 2 variable sets. But it hardly seems worth the effort of the data munging.&lt;/p&gt;&#xA;&lt;h2&gt;Assumption:&lt;/h2&gt;&#xA;&lt;p&gt;I am assuming that you are not talking about adding ~identical variables also into your model, e.g. &lt;a href=&quot;https://www.worldclim.org/data/worldclim21.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;files here&lt;/a&gt;, but are instead talking about the less identical monthly data &lt;a href=&quot;https://www.worldclim.org/data/worldclim21.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I'm &lt;em&gt;pretty sure&lt;/em&gt; this is right, but I wanted to clarify.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2023-02-22T22:45:57.833" LastActivityDate="2023-02-22T22:45:57.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20616" PostTypeId="1" CreationDate="2023-02-22T23:17:17.533" Score="1" ViewCount="50" Body="&lt;p&gt;When i started programming against PDB i had a mixture of confusion &amp;amp; frustration with the fact that certain &lt;code&gt;cif&lt;/code&gt; files contain two actual structures aka assemblies of the object; i take it for granted now.&lt;/p&gt;&#xA;&lt;p&gt;However, i observe now (from years of usage) that in the case of the ribosome XRAY/NMR structs frequently have exactly 2 structures aka assemblies stuck together with very little differnce between those two assemblies (~2-3 atom difference in a ~150,000 atom structure). Is there a reason for that? Does it have to do with the way microscopy is done or is it just the artifact of the deposition process? Why not deposit them as separate structures or just one if they are near nigh identical?&lt;/p&gt;&#xA;&lt;p&gt;Example is &lt;code&gt;4V5D&lt;/code&gt; with 2 assemblies: &lt;a href=&quot;https://data.rcsb.org/graphql/index.html?query=%7B%0A%20%20entry(entry_id%3A%224v5d%22)%7B%0A%20%20%20%20assemblies%7B%0A%20%20%20%20%20%20rcsb_assembly_info%20%7B%0A%20%20%20%20%20%20%20%20assembly_id%0A%20%20%20%20%20%20%20%20atom_count%0A%20%20%20%20%20%20%20%20branched_atom_count%0A%20%20%20%20%20%20%20%20branched_entity_count%0A%20%20%20%20%20%20%20%20branched_entity_instance_count%0A%20%20%20%20%20%20%20%20hydrogen_atom_count%0A%20%20%20%20%20%20%20%20modeled_polymer_monomer_count%0A%20%20%20%20%20%20%20%20na_polymer_entity_types%0A%20%20%20%20%20%20%20%20nonpolymer_atom_count%0A%20%20%20%20%20%20%20%20nonpolymer_entity_count%0A%20%20%20%20%20%20%20%20nonpolymer_entity_instance_count%0A%20%20%20%20%20%20%20%20num_heterologous_interface_entities%0A%20%20%20%20%20%20%20%20num_heteromeric_interface_entities%0A%20%20%20%20%20%20%20%20num_homomeric_interface_entities%0A%20%20%20%20%20%20%20%20num_interface_entities%0A%20%20%20%20%20%20%20%20num_interfaces%0A%20%20%20%20%20%20%20%20num_isologous_interface_entities%0A%20%20%20%20%20%20%20%20num_na_interface_entities%0A%20%20%20%20%20%20%20%20num_prot_na_interface_entities%0A%20%20%20%20%20%20%20%20num_protein_interface_entities%0A%20%20%20%20%20%20%20%20polymer_atom_count%0A%20%20%20%20%20%20%20%20polymer_composition%0A%20%20%20%20%20%20%20%20polymer_entity_count%0A%20%20%20%20%20%20%20%20polymer_entity_count_DNA%0A%20%20%20%20%20%20%20%20polymer_entity_count_RNA%0A%20%20%20%20%20%20%20%20polymer_entity_count_nucleic_acid%0A%20%20%20%20%20%20%20%20polymer_entity_count_nucleic_acid_hybrid%0A%20%20%20%20%20%20%20%20polymer_entity_count_protein%0A%20%20%20%20%20%20%20%20polymer_entity_instance_count%0A%20%20%20%20%20%20%20%20polymer_entity_instance_count_DNA%0A%20%20%20%20%20%20%20%20polymer_entity_instance_count_RNA%0A%20%20%20%20%20%20%20%20polymer_entity_instance_count_nucleic_acid%0A%20%20%20%20%20%20%20%20polymer_entity_instance_count_nucleic_acid_hybrid%0A%20%20%20%20%20%20%20%20polymer_entity_instance_count_protein%0A%20%20%20%20%20%20%20%20polymer_monomer_count%0A%20%20%20%20%20%20%20%20selected_polymer_entity_types%0A%20%20%20%20%20%20%20%20solvent_atom_count%0A%20%20%20%20%20%20%20%20solvent_entity_count%0A%20%20%20%20%20%20%20%20solvent_entity_instance_count%0A%20%20%20%20%20%20%20%20total_assembly_buried_surface_area%0A%20%20%20%20%20%20%20%20total_number_interface_residues%0A%20%20%20%20%20%20%20%20unmodeled_polymer_monomer_count%0A%20%20%20%20%20%20%7D%0A%20%20%20%20%7D%0A%20%20%7D%0A%7D&quot; rel=&quot;nofollow noreferrer&quot;&gt;Assemblies data.&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;PS.&#xA;The relevant gql snippet for rcsb data api:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;  entry(entry_id:&amp;quot;4v5d&amp;quot;){&#xA;    assemblies{&#xA;      rcsb_assembly_info {&#xA;        assembly_id&#xA;        atom_count&#xA;        branched_atom_count&#xA;        branched_entity_count&#xA;        branched_entity_instance_count&#xA;        hydrogen_atom_count&#xA;        modeled_polymer_monomer_count&#xA;        na_polymer_entity_types&#xA;        nonpolymer_atom_count&#xA;        nonpolymer_entity_count&#xA;        nonpolymer_entity_instance_count&#xA;        num_heterologous_interface_entities&#xA;        num_heteromeric_interface_entities&#xA;        num_homomeric_interface_entities&#xA;        num_interface_entities&#xA;        num_interfaces&#xA;        num_isologous_interface_entities&#xA;        num_na_interface_entities&#xA;        num_prot_na_interface_entities&#xA;        num_protein_interface_entities&#xA;        polymer_atom_count&#xA;        polymer_composition&#xA;        polymer_entity_count&#xA;        polymer_entity_count_DNA&#xA;        polymer_entity_count_RNA&#xA;        polymer_entity_count_nucleic_acid&#xA;        polymer_entity_count_nucleic_acid_hybrid&#xA;        polymer_entity_count_protein&#xA;        polymer_entity_instance_count&#xA;        polymer_entity_instance_count_DNA&#xA;        polymer_entity_instance_count_RNA&#xA;        polymer_entity_instance_count_nucleic_acid&#xA;        polymer_entity_instance_count_nucleic_acid_hybrid&#xA;        polymer_entity_instance_count_protein&#xA;        polymer_monomer_count&#xA;        selected_polymer_entity_types&#xA;        solvent_atom_count&#xA;        solvent_entity_count&#xA;        solvent_entity_instance_count&#xA;        total_assembly_buried_surface_area&#xA;        total_number_interface_residues&#xA;        unmodeled_polymer_monomer_count&#xA;      }&#xA;    }&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6511" LastEditorUserId="6511" LastEditDate="2023-02-24T00:45:12.597" LastActivityDate="2023-02-24T00:45:12.597" Title="Why do XRAY (but not CryoEM) structures of ribosome in PDB have 2 assemblies?" Tags="|protein-structure|assembly|pdb|ribosomal|" AnswerCount="0" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="20617" PostTypeId="1" CreationDate="2023-02-22T23:47:34.317" Score="2" ViewCount="424" Body="&lt;p&gt;I have data from a GWAS that provides me with the SNP, Chromosome, and base-pair. My data set has thousands of SNPs. What is the easiest way to find the nearest gene for each SNP using this information? My data looks like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gQBFO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gQBFO.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17059" LastActivityDate="2023-03-08T16:27:07.090" Title="Getting nearest gene from SNP data using SNP ID, CHR, and BP" Tags="|gene|snp|gwas|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20618" PostTypeId="1" CreationDate="2023-02-23T13:52:00.890" Score="1" ViewCount="27" Body="&lt;p&gt;Our study shows that gene A knockout will down-regulate cell cycle-related genes in some cell lines. My aim is to understand whether cell cycle-related genes are down regulated on the transcriptional or translational level upon gene A knockout.&lt;/p&gt;&#xA;&lt;p&gt;I plan to compare the transcript and protein abundance of cell cycle-related genes between two groups of cell lines (1st group: sensitive to gene A knockout; 2nd group: not sensitive to the knockout). May I ask how can I do this? Are there any tools, databases and papers related to my goal? Thank you very much!&lt;/p&gt;&#xA;" OwnerUserId="15643" LastActivityDate="2023-02-23T13:52:00.890" Title="How to investigate the correlation between transcript and protein abundance?" Tags="|correlation|data-mining|transcriptome-regulation|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20619" PostTypeId="1" CreationDate="2023-02-23T18:38:41.520" Score="1" ViewCount="405" Body="&lt;p&gt;I am analyzing publically available scRNA seq datasets on R using Seurat. I have created a seurat object, clustered and annotated different cell types. Now, I am interested in comparing the expression values and percentage of expressing cells within each cluster for a given set of genes I am interested in. I can  visualize their expression using DotPlots, feature plots, etc... but how can I generate a table with the expression values (log2FC) for my defined set of genes across the different clusters/cell types?&#xA;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="17068" LastActivityDate="2023-02-24T18:55:13.363" Title="Determine expression values for a defined list of genes across different clusters in scRNA data using seurat" Tags="|scrnaseq|seurat|gene-expression|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20620" PostTypeId="1" AcceptedAnswerId="20621" CreationDate="2023-02-23T22:19:02.767" Score="3" ViewCount="216" Body="&lt;p&gt;I am trying to run the &lt;strong&gt;Prokka annotation tool&lt;/strong&gt; on my linux machine and have installed &lt;strong&gt;Barrnap and added it to $PATH and given the folder chmod 777 permission&lt;/strong&gt;. However, when running Prokka, it gives the line &lt;code&gt;Could not parse version from 'sh: 1: barrnap: Permission denied'&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Running Barrnap solely through the command line with a valid input works fine but does not seem to work when another script calls barrnap. I have tried checking path variables and giving max permissions to barrnap and Prokka but it does not seem to change this error. Here is the full log file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[15:01:28] This is prokka 1.14.6&#xA;[15:01:28] Written by Torsten Seemann &amp;lt;torsten.seemann@gmail.com&amp;gt;&#xA;[15:01:28] Homepage is https://github.com/tseemann/prokka&#xA;[15:01:28] Local time is Thu Feb 23 15:01:28 2023&#xA;[15:01:28] You are saurab&#xA;[15:01:28] Operating system is linux&#xA;[15:01:28] You have BioPerl 1.7.8&#xA;[15:01:28] System has 16 cores.&#xA;[15:01:28] Will use maximum of 8 cores.&#xA;[15:01:28] Annotating as &amp;gt;&amp;gt;&amp;gt; Bacteria &amp;lt;&amp;lt;&amp;lt;&#xA;[15:01:28] Generating locus_tag from 'jobs/NC_004556.faa' contents.&#xA;[15:01:28] Setting --locustag ECJHPMJE from MD5 ec31963ed5d3d76e28501fa7e45ca273&#xA;[15:01:28] Creating new output folder: PROKKA_02232023&#xA;[15:01:28] Running: mkdir -p PROKKA_02232023&#xA;[15:01:28] Using filename prefix: PROKKA_02232023.XXX&#xA;[15:01:28] Setting HMMER_NCPU=1&#xA;[15:01:28] Writing log to: PROKKA_02232023/PROKKA_02232023.log&#xA;[15:01:28] Command: /home/saurab/BIOIN/basys2-app/sub_programs/prokka/bin/prokka jobs/NC_004556.faa&#xA;[15:01:28] Appending to PATH: /home/saurab/BIOIN/basys2-app/sub_programs/prokka/bin/../binaries/linux&#xA;[15:01:28] Appending to PATH: /home/saurab/BIOIN/basys2-app/sub_programs/prokka/bin/../binaries/linux/../common&#xA;[15:01:28] Appending to PATH: /home/saurab/BIOIN/basys2-app/sub_programs/prokka/bin&#xA;[15:01:28] Looking for 'aragorn' - found /home/saurab/BIOIN/phastest-app/sub_programs/aragorn&#xA;[15:01:28] Determined aragorn version is 001002 from 'ARAGORN v1.2   Dean Laslett'&#xA;[15:01:28] Looking for 'barrnap' - found /home/saurab/BIOIN/phastest-app/sub_programs/barrnap&#xA;[15:01:28] Coult not parse version from 'sh: 1: barrnap: Permission denied'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17070" LastActivityDate="2023-02-24T18:42:06.677" Title="Barrnap Bacterial rRNA Predictor script permission denied error when running Prokka" Tags="|sequence-annotation|perl|bacteria|functional-annotation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20621" PostTypeId="2" ParentId="20620" CreationDate="2023-02-23T22:57:28.020" Score="3" Body="&lt;p&gt;My suggestion is to run &lt;code&gt;prokka&lt;/code&gt; from within &lt;code&gt;conda&lt;/code&gt; by simply downloading &lt;a href=&quot;https://docs.conda.io/projects/conda/en/stable/user-guide/install/download.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;miniconda&lt;/a&gt; then&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda create --name prokka&#xA;conda activate prokka&#xA;conda install -c conda-forge -c bioconda -c defaults prokka&#xA;prokka --help&#xA;prokka --version&#xA;prokka mycontigs.fa # running prokka&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You could always perform a&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chmod 700 /home/saurab/BIOIN/phastest-app/sub_programs/barrnap&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or some variant like &lt;code&gt;chmod 500 ...&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;or possibly using a recursive &lt;code&gt;chmod -R 500&lt;/code&gt; that sort of thing. Eventually it will work.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;However&lt;/strong&gt;, it strikes me you are installing &lt;code&gt;prokka&lt;/code&gt; outside any virtual environment such as &lt;code&gt;conda&lt;/code&gt; or &lt;code&gt;brew&lt;/code&gt; and I would not do that personally, because you are giving significant permissions to the algorithm.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;conda&lt;/code&gt; (or &lt;code&gt;brew&lt;/code&gt;) will sort out all the permission needed and there's no need to do anything with &lt;code&gt;chmod&lt;/code&gt; stuff.&lt;/p&gt;&#xA;&lt;p&gt;Good luck with the annotation.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="73" LastEditDate="2023-02-24T18:42:06.677" LastActivityDate="2023-02-24T18:42:06.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20622" PostTypeId="1" AcceptedAnswerId="20626" CreationDate="2023-02-24T10:23:48.173" Score="2" ViewCount="74" Body="&lt;p&gt;I've been doing phylogenetics with large (hundreds) 16S rRNA sequences lately.&lt;/p&gt;&#xA;&lt;p&gt;Usually I'm focusing on one order, and using a combination of trees and sequence similarity to assess stuff like 'is this a genus', 'is this a separate genus', 'is this sequence actually in the order of interest'. It feels like a lot of what I do is repetitive and could be automated more. In particular, it feels like there should be a simpler way for having an assemble view at sequence similarity rather that calculating a Percentage Identity Matrix and then putting it in Excel with conditional formatting. I'm imagining something like this clustering diagram for species (&lt;a href=&quot;https://www.researchgate.net/publication/360738216/figure/fig4/AS:1182363421806594@1658908815491/The-t-SNE-clustering-of-bacterial-genomes-in-the-code-RA-space-at-different-taxonomic.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.researchgate.net/publication/360738216/figure/fig4/AS:1182363421806594@1658908815491/The-t-SNE-clustering-of-bacterial-genomes-in-the-code-RA-space-at-different-taxonomic.png&lt;/a&gt;), but for sequences instead (using sequence similarity values, at least). Still, I can't seem to find something like that for 16S rRNA that's as visually compelling (or even as visually helpful as my Excel sheet). There are a number of programs for clustering OTUs, but the ones I've encountered don't seem to have a graphical / interactive interface.&lt;/p&gt;&#xA;&lt;p&gt;Does something like this exist? Thank you for your time!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastEditorUserId="3848" LastEditDate="2023-08-23T01:44:18.747" LastActivityDate="2023-08-23T01:44:18.747" Title="Is there a graphical/interactive 16S rRNA clustering method?" Tags="|phylogenetics|phylogeny|sequence-analysis|rna|software-recommendation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20623" PostTypeId="2" ParentId="20619" CreationDate="2023-02-24T18:55:13.363" Score="1" Body="&lt;p&gt;Assuming you're looking for &lt;em&gt;differential&lt;/em&gt; expression values, the &lt;code&gt;FindMarkers&lt;/code&gt; function within Seurat would be a good thing to try first:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# find all markers distinguishing cluster 5 from clusters 0 and 3&#xA;cluster5.markers &amp;lt;- FindMarkers(pbmc, ident.1 = 5, ident.2 = c(0, 3), min.pct = 0.25)&#xA;head(cluster5.markers, n = 5)&#xA;##                       p_val avg_log2FC pct.1 pct.2     p_val_adj&#xA;## FCGR3A        8.246578e-205   4.261495 0.975 0.040 1.130936e-200&#xA;## IFITM3        1.677613e-195   3.879339 0.975 0.049 2.300678e-191&#xA;## CFD           2.401156e-193   3.405492 0.938 0.038 3.292945e-189&#xA;## CD68          2.900384e-191   3.020484 0.926 0.035 3.977587e-187&#xA;## RP11-290F20.3 2.513244e-186   2.720057 0.840 0.017 3.446663e-182 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;[Note the &lt;code&gt;avg_log2FC&lt;/code&gt; column]&lt;/p&gt;&#xA;&lt;p&gt;More details can be found at the bottom of the Seurat tutorial pages for the PBMC 3K Guided Tutorial, &lt;em&gt;Finding differentially expressed features (cluster biomarkers)&lt;/em&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://satijalab.org/seurat/articles/pbmc3k_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/articles/pbmc3k_tutorial.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-02-24T18:55:13.363" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20624" PostTypeId="1" CreationDate="2023-02-24T19:20:57.383" Score="2" ViewCount="130" Body="&lt;p&gt;I have a pandas dataframe that reads in a PAF file from minimap2. What I would like to do is take the first 5 columns of the data from to create a BED file.&lt;/p&gt;&#xA;&lt;p&gt;I used this to extract the first 5 columns:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pandas as pd&#xA;import numpy as np&#xA;df=pd.read_csv(tag1_paf, delimiter = &amp;quot;\t&amp;quot;)&#xA;#print(df[4])&#xA;&#xA;specific_columns=df.iloc[:,[0,1,2,3,4]]&#xA;print(specific_columns)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which outputs this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;read1  47215  20591  21087 +&#xA;read2  14111  3478   3973  +&#xA;read3  30861  21367  21855 +&#xA;read4  4647   767    1257  -&#xA;read5  11706  9569   9990  +&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But now I want to do math on the third and fourth columns depending on the strand its on. For example, if column 5 = &amp;quot;+&amp;quot; add 1000 to the third column.&lt;/p&gt;&#xA;&lt;p&gt;I can't figure it out on my own - I've tried looking it up and using the series add/sub methods from pandas (&lt;a href=&quot;https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.add.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.add.html&lt;/a&gt;) but I'm still confused.&lt;/p&gt;&#xA;&lt;p&gt;If anyone can help that would be very appreciated!&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="3848" LastEditDate="2023-07-28T21:55:50.740" LastActivityDate="2023-07-28T21:55:50.740" Title="Math on Pandas Columns" Tags="|python|phylogenetics|phylogeny|sequence-analysis|pandas|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20625" PostTypeId="2" ParentId="20624" CreationDate="2023-02-24T20:15:03.090" Score="4" Body="&lt;p&gt;The vectorised solution (fast) is,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;filepath = '/pathtodir/test/bioinfo.csv'&#xA;&#xA;df = pd.read_csv(filepath, delimiter=&amp;quot; &amp;quot;, header=None)&#xA;&#xA;df.loc[df[4] == &amp;quot;+&amp;quot;, 'new2'] = df[2] + 1000&#xA;df.loc[df[4] == &amp;quot;-&amp;quot;, 'new2'] = df[2]&#xA;df = df.drop([2], axis=1)&#xA;df = df[[0,1,'new2',3,4]]&#xA;print (df)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;-&lt;/th&gt;&#xA;&lt;th&gt;0&lt;/th&gt;&#xA;&lt;th&gt;1&lt;/th&gt;&#xA;&lt;th&gt;new2&lt;/th&gt;&#xA;&lt;th&gt;3&lt;/th&gt;&#xA;&lt;th&gt;4&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;read1&lt;/td&gt;&#xA;&lt;td&gt;47215&lt;/td&gt;&#xA;&lt;td&gt;21591.0&lt;/td&gt;&#xA;&lt;td&gt;21087&lt;/td&gt;&#xA;&lt;td&gt;+&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;read2&lt;/td&gt;&#xA;&lt;td&gt;14111&lt;/td&gt;&#xA;&lt;td&gt;4478.0&lt;/td&gt;&#xA;&lt;td&gt;3973&lt;/td&gt;&#xA;&lt;td&gt;+&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;td&gt;read3&lt;/td&gt;&#xA;&lt;td&gt;30861&lt;/td&gt;&#xA;&lt;td&gt;22367.0&lt;/td&gt;&#xA;&lt;td&gt;21855&lt;/td&gt;&#xA;&lt;td&gt;+&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;3&lt;/td&gt;&#xA;&lt;td&gt;read4&lt;/td&gt;&#xA;&lt;td&gt;4647&lt;/td&gt;&#xA;&lt;td&gt;767.0&lt;/td&gt;&#xA;&lt;td&gt;1257&lt;/td&gt;&#xA;&lt;td&gt;-&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;4&lt;/td&gt;&#xA;&lt;td&gt;read5&lt;/td&gt;&#xA;&lt;td&gt;11706&lt;/td&gt;&#xA;&lt;td&gt;10569.0&lt;/td&gt;&#xA;&lt;td&gt;9990&lt;/td&gt;&#xA;&lt;td&gt;+&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;An alternative method to re-order the columns then the answer is &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/20605/automate-the-def-function-in-pandas-for-correlation&quot;&gt;Automate the def function in pandas for correlation&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Original data&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;&lt;/th&gt;&#xA;&lt;th&gt;0&lt;/th&gt;&#xA;&lt;th&gt;1&lt;/th&gt;&#xA;&lt;th&gt;2&lt;/th&gt;&#xA;&lt;th&gt;3&lt;/th&gt;&#xA;&lt;th&gt;4&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;read1&lt;/td&gt;&#xA;&lt;td&gt;47215&lt;/td&gt;&#xA;&lt;td&gt;20591&lt;/td&gt;&#xA;&lt;td&gt;21087&lt;/td&gt;&#xA;&lt;td&gt;+&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;read2&lt;/td&gt;&#xA;&lt;td&gt;14111&lt;/td&gt;&#xA;&lt;td&gt;3478&lt;/td&gt;&#xA;&lt;td&gt;3973&lt;/td&gt;&#xA;&lt;td&gt;+&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;td&gt;read3&lt;/td&gt;&#xA;&lt;td&gt;30861&lt;/td&gt;&#xA;&lt;td&gt;21367&lt;/td&gt;&#xA;&lt;td&gt;21855&lt;/td&gt;&#xA;&lt;td&gt;+&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;3&lt;/td&gt;&#xA;&lt;td&gt;read4&lt;/td&gt;&#xA;&lt;td&gt;4647&lt;/td&gt;&#xA;&lt;td&gt;767&lt;/td&gt;&#xA;&lt;td&gt;1257&lt;/td&gt;&#xA;&lt;td&gt;-&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;4&lt;/td&gt;&#xA;&lt;td&gt;read5&lt;/td&gt;&#xA;&lt;td&gt;11706&lt;/td&gt;&#xA;&lt;td&gt;9569&lt;/td&gt;&#xA;&lt;td&gt;9990&lt;/td&gt;&#xA;&lt;td&gt;+&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&lt;hr /&gt;&#xA;&lt;p&gt;Final solution&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df = pd.read_csv(filepath, delimiter=&amp;quot; &amp;quot;, header=None)&#xA;df.loc[df[4] == &amp;quot;+&amp;quot;, 'new2'] = df[2] + 1000&#xA;df.loc[df[4] == &amp;quot;-&amp;quot;, 'new2'] = df[2] - 1000&#xA;df.loc[df['new2'] &amp;lt; 0, 'new2'] = 1&#xA;df = df.drop([2], axis=1)&#xA;df = df[[0,1,'new2',3,4]]&#xA;print (df)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;-&lt;/th&gt;&#xA;&lt;th&gt;0&lt;/th&gt;&#xA;&lt;th&gt;1&lt;/th&gt;&#xA;&lt;th&gt;new2&lt;/th&gt;&#xA;&lt;th&gt;3&lt;/th&gt;&#xA;&lt;th&gt;4&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;read1&lt;/td&gt;&#xA;&lt;td&gt;47215&lt;/td&gt;&#xA;&lt;td&gt;21591.0&lt;/td&gt;&#xA;&lt;td&gt;21087&lt;/td&gt;&#xA;&lt;td&gt;+&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;read2&lt;/td&gt;&#xA;&lt;td&gt;14111&lt;/td&gt;&#xA;&lt;td&gt;4478.0&lt;/td&gt;&#xA;&lt;td&gt;3973&lt;/td&gt;&#xA;&lt;td&gt;+&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;td&gt;read3&lt;/td&gt;&#xA;&lt;td&gt;30861&lt;/td&gt;&#xA;&lt;td&gt;22367.0&lt;/td&gt;&#xA;&lt;td&gt;21855&lt;/td&gt;&#xA;&lt;td&gt;+&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;3&lt;/td&gt;&#xA;&lt;td&gt;read4&lt;/td&gt;&#xA;&lt;td&gt;4647&lt;/td&gt;&#xA;&lt;td&gt;1.0&lt;/td&gt;&#xA;&lt;td&gt;1257&lt;/td&gt;&#xA;&lt;td&gt;-&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;4&lt;/td&gt;&#xA;&lt;td&gt;read5&lt;/td&gt;&#xA;&lt;td&gt;11706&lt;/td&gt;&#xA;&lt;td&gt;10569.0&lt;/td&gt;&#xA;&lt;td&gt;9990&lt;/td&gt;&#xA;&lt;td&gt;+&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;... worked too&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-02-24T23:39:21.790" LastActivityDate="2023-02-24T23:39:21.790" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20626" PostTypeId="2" ParentId="20622" CreationDate="2023-02-25T15:04:37.127" Score="1" Body="&lt;p&gt;My recommendation is &lt;a href=&quot;https://www.megasoftware.net&quot; rel=&quot;nofollow noreferrer&quot;&gt;MegaX&lt;/a&gt;. It's the closest thing in phylogenetics to an all encompassing GUI.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;This has a nice visual alignment editor which highlights conserved, variable and parsimonious sites.&lt;/li&gt;&#xA;&lt;li&gt;It will produce distance matrices based on any number of criteria, e.g. % homology, Jukes-Cantor, Kimura-2 parameter etc ...&lt;/li&gt;&#xA;&lt;li&gt;Good for bootstrapping distance-calculations as well as can perform ML based tree building methods with full parameterisation.&lt;/li&gt;&#xA;&lt;li&gt;Nice tree visualisation, which can be edited, switched to viewing bootstrapping (if performed), or to branch genetic distances (again under any distance criteria, e.g. %age homology etc ..). It does loads of other stuff, which in this case are not needed.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;You can export the tree and import into &lt;a href=&quot;http://evomics.org/resources/software/molecular-evolution-software/figtree/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FigTree&lt;/a&gt; to produce &lt;a href=&quot;https://www.researchgate.net/profile/Mariana-Louro/publication/350894395/figure/fig3/AS:1013091672281097@1618551285234/Maximum-likelihood-phylogenetic-condensed-tree-calculated-using-aligned-concatenated.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;collapsed clades represented by coloured triangles&lt;/a&gt;. In summary for easy GUI stuff MegaX, then FigTree.&lt;/p&gt;&#xA;&lt;p&gt;There are no PCA/MDA based analysis for 16S sequence data because eiganvalue-based calculations would override lots of theory involved in point mutation modelling and the resulting trees. I could replicated the &lt;a href=&quot;https://www.researchgate.net/publication/360738216/figure/fig4/AS:1182363421806594@1658908815491/The-t-SNE-clustering-of-bacterial-genomes-in-the-code-RA-space-at-different-taxonomic.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;PCA output&lt;/a&gt; in the question with sequence data, its not hard to do - but it's not publishable.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-02-25T15:12:38.183" LastActivityDate="2023-02-25T15:12:38.183" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20628" PostTypeId="1" CreationDate="2023-02-26T06:36:30.743" Score="-1" ViewCount="59" Body="&lt;p&gt;I have fasta multiple sequence alignment file of protein sequence. I want to look for mutation/substitution in amino acid between query with reference to one query. Is there any R package or any method I can go through.&lt;/p&gt;&#xA;" OwnerUserId="17079" LastEditorUserId="650" LastEditDate="2023-03-03T22:06:26.363" LastActivityDate="2023-03-03T22:06:26.363" Title="To find amino acid substitution from fasta alignment" Tags="|r|proteins|mutations|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20629" PostTypeId="2" ParentId="20628" CreationDate="2023-02-26T11:47:18.510" Score="0" Body="&lt;p&gt;Use &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PAGE_TYPE=BlastDocs&amp;amp;DOC_TYPE=Download&quot; rel=&quot;nofollow noreferrer&quot;&gt;local Blast&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;makeblastdb -in myproteins.fa -parse_seqids -blastdb_version 5  -out myfasta.db -title &amp;quot;protein db&amp;quot; -dbtype protein&#xA;&#xA;blastn -query reference.fas -db myfasta.db -evalue 1e-6 -num_threads 4 -out blasthits.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;-db&lt;/code&gt; made from the fasta alignment file (preferably without gaps)&#xA;&lt;code&gt;-query&lt;/code&gt; is the sequence your reference sequence&lt;/p&gt;&#xA;&lt;p&gt;Output there will be a pairwise comparison for every sequence against your reference.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-02-26T11:47:18.510" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20630" PostTypeId="1" CreationDate="2023-02-27T14:20:15.033" Score="1" ViewCount="23" Body="&lt;p&gt;I am in need of a simple tool that accepts an XML file generated by a BlastP as input and gives me back a file containing information about mismatches/indels/etc. Fortunately such a tool already exists, but it works specifically with XML files generated by BlastN: &lt;a href=&quot;http://lindenb.github.io/jvarkit/BlastNToSnp.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://lindenb.github.io/jvarkit/BlastNToSnp.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm trying to adapt the code so that it also reads BlastP files, but after adapting the code it gives me both information about matches and mismatches in the output, and since my knowledge of Java is quite limited I can't identify the problem. Can someone with more expertise in java help me?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;package com.github.lindenb.jvarkit.tools.blast;&#xA;&#xA;import java.io.File;&#xA;import java.io.PrintWriter;&#xA;import java.util.List;&#xA;&#xA;import gov.nih.nlm.ncbi.blast.Hit;&#xA;import gov.nih.nlm.ncbi.blast.Hsp;&#xA;import gov.nih.nlm.ncbi.blast.Iteration;&#xA;&#xA;import javax.xml.bind.JAXBContext;&#xA;import javax.xml.bind.JAXBElement;&#xA;import javax.xml.bind.JAXBException;&#xA;import javax.xml.bind.Marshaller;&#xA;import javax.xml.bind.Unmarshaller;&#xA;import javax.xml.namespace.QName;&#xA;import javax.xml.stream.XMLEventReader;&#xA;import javax.xml.stream.XMLInputFactory;&#xA;import javax.xml.stream.XMLResolver;&#xA;import javax.xml.stream.XMLStreamException;&#xA;import javax.xml.stream.events.XMLEvent;&#xA;&#xA;import htsjdk.samtools.util.CloserUtil;&#xA;&#xA;import com.github.lindenb.jvarkit.io.IOUtils;&#xA;import com.github.lindenb.jvarkit.util.bio.AcidNucleics;&#xA;&#xA;import com.beust.jcommander.Parameter;&#xA;import com.github.lindenb.jvarkit.util.jcommander.Launcher;&#xA;import com.github.lindenb.jvarkit.util.jcommander.Program;&#xA;import com.github.lindenb.jvarkit.util.log.Logger;&#xA;&#xA;@Program(name=&amp;quot;blastp2snp&amp;quot;,&#xA;    keywords={&amp;quot;blast&amp;quot;,&amp;quot;snp&amp;quot;},&#xA;    description=&amp;quot;print indel/mismatch in a blastp stream&amp;quot;,&#xA;    biostars={89151,455765},&#xA;    modificationDate=&amp;quot;20200831&amp;quot;,&#xA;    creationDate=&amp;quot;20131126&amp;quot;&#xA;    )&#xA;public class BlastPToSnp extends Launcher&#xA;{&#xA;    private static final Logger LOG = Logger.build(BlastPToSnp.class).make();&#xA;&#xA;&#xA;    @Parameter(names={&amp;quot;-o&amp;quot;,&amp;quot;--output&amp;quot;},description=OPT_OUPUT_FILE_OR_STDOUT)&#xA;    private File outputFile = null;&#xA;&#xA;&#xA;    @Parameter(names={&amp;quot;-n&amp;quot;,&amp;quot;--gapsize&amp;quot;},description=&amp;quot;min gap&amp;quot;)&#xA;    private int minGapSize = 3 ;&#xA;&#xA;    private Unmarshaller unmarshaller;&#xA;    private Marshaller marshaller;&#xA;    /* force javac to compile */&#xA;    @SuppressWarnings(&amp;quot;unused&amp;quot;)&#xA;    private gov.nih.nlm.ncbi.blast.ObjectFactory _ignore_for_javac=null;&#xA;    &#xA;    &#xA;    private Iteration peekIteration(final XMLEventReader r) throws XMLStreamException,JAXBException&#xA;        {&#xA;        while(r.hasNext())&#xA;            {&#xA;            final XMLEvent evt=r.peek();&#xA;            if(!(evt.isStartElement()))&#xA;                {&#xA;                r.next();&#xA;                continue;&#xA;                }&#xA;            final String name=evt.asStartElement().getName().getLocalPart();&#xA;            if(name.equals(&amp;quot;BlastOutput_program&amp;quot;))&#xA;                {&#xA;                r.next();&#xA;                String prog=r.getElementText();&#xA;                if(!&amp;quot;blastp&amp;quot;.equals(prog))&#xA;                    {&#xA;                    throw new XMLStreamException(&amp;quot;Not a blastp input:&amp;quot;+prog);&#xA;                    }&#xA;                continue;&#xA;                }&#xA;            &#xA;            if(!name.equals(&amp;quot;Iteration&amp;quot;))&#xA;                {&#xA;                r.next();&#xA;                continue;&#xA;                }&#xA;            &#xA;            return this.unmarshaller.unmarshal(r, Iteration.class).getValue();&#xA;            }&#xA;        return null;&#xA;        }&#xA;    &#xA;    &#xA;private void run(&#xA;        PrintWriter pw,&#xA;        XMLEventReader r&#xA;        )&#xA;        throws XMLStreamException,JAXBException&#xA;    {&#xA;     long numIterations=0L;&#xA;     long numPerfectMath=0L;&#xA;     long numNoHit=0L;&#xA;     &#xA;    &#xA;    pw.print(&amp;quot;#query&amp;quot;);&#xA;    pw.print('\t');&#xA;    pw.print(&amp;quot;hit&amp;quot;);&#xA;    pw.print('\t');&#xA;    pw.print((&amp;quot;hit-index&amp;quot;));&#xA;    pw.print('\t');&#xA;    pw.print(&amp;quot;hsp-index&amp;quot;);&#xA;    &#xA;    pw.print('\t');&#xA;    pw.print(&amp;quot;query-POS&amp;quot;);&#xA;    pw.print('\t');&#xA;    pw.print(&amp;quot;hit-POS&amp;quot;);&#xA;    pw.print('\t');&#xA;    pw.print(&amp;quot;STRAND&amp;quot;);&#xA;    &#xA;    pw.print('\t');&#xA;    pw.print(&amp;quot;REF(hit)&amp;quot;);&#xA;    pw.print('\t');&#xA;    pw.print(&amp;quot;ALT(query)&amp;quot;);&#xA;    pw.print('\t');&#xA;    pw.print(&amp;quot;blast.align_length&amp;quot;);&#xA;    pw.print('\t');&#xA;    pw.print(&amp;quot;blast.hit.var&amp;quot;);&#xA;    pw.print('\t');&#xA;    pw.print(&amp;quot;blast.query.var&amp;quot;);&#xA;    pw.print('\t');&#xA;    pw.print(&amp;quot;blast.mid.var&amp;quot;);&#xA;    &#xA;    pw.println();&#xA;    for(;;)&#xA;        {&#xA;&#xA;        Iteration iter1=peekIteration(r);&#xA;        if(iter1==null) break;&#xA;        ++numIterations;&#xA;        if(iter1.getIterationHits().getHit().isEmpty())&#xA;            {&#xA;            LOG.info(&amp;quot;No hit found for &amp;quot;+iter1.getIterationQueryDef());&#xA;            ++numNoHit;&#xA;            continue;&#xA;            }&#xA;        boolean found_mismatch=false;&#xA;        for(int hit_loop=0; hit_loop &amp;lt;  iter1.getIterationHits().getHit().size();++hit_loop)&#xA;            {&#xA;            final Hit hit=iter1.getIterationHits().getHit().get(hit_loop);&#xA;            for(int  hsp_loop =0;hsp_loop&amp;lt; hit.getHitHsps().getHsp().size();++hsp_loop)&#xA;                {&#xA;                final Hsp hsp = hit.getHitHsps().getHsp().get(hsp_loop);&#xA;                String hspQseq=hsp.getHspQseq();&#xA;                String hspMid=hsp.getHspMidline();&#xA;                String hspHSeq=hsp.getHspHseq();&#xA;                &#xA;                int hsp_query_from=Integer.parseInt(hsp.getHspQueryFrom());&#xA;                int hsp_query_to=Integer.parseInt(hsp.getHspQueryTo());&#xA;                int hsp_hit_from=Integer.parseInt(hsp.getHspHitFrom());&#xA;                int hsp_hit_to=Integer.parseInt(hsp.getHspHitTo());&#xA;                final int align_length=Integer.parseInt(hsp.getHspAlignLen());&#xA;                final int hit_shift=(hsp_hit_from&amp;gt;hsp_hit_to?-1:1);&#xA;                &#xA;                int i=0;&#xA;                int query_index = hsp_query_from;&#xA;                int hit_index = hsp_hit_from;&#xA;                &#xA;                while( i&amp;lt; align_length )&#xA;                    {&#xA;                    char ch= hspHSeq.charAt(i);&#xA;                    char cq= hspQseq.charAt(i);&#xA;                    char cm= hspMid.charAt(i);&#xA;                    if(cm=='|')&#xA;                        {&#xA;                        ++i;&#xA;                        query_index++;&#xA;                        hit_index+=hit_shift;&#xA;                        continue;&#xA;                        }&#xA;                    found_mismatch=true;&#xA;                    &#xA;                    int j=i+1;&#xA;                    for(;;)&#xA;                        {&#xA;                        int k=hspMid.indexOf(' ', j);&#xA;                        if(k==-1 || k-j&amp;gt; minGapSize) break;&#xA;                        j=k+1;&#xA;                        }&#xA;                    &#xA;                    &#xA;                    String ref=new String(hspHSeq.substring(i, j)).replaceAll(&amp;quot;[\\- ]&amp;quot;, &amp;quot;&amp;quot;);&#xA;                    String alt=new String(hspQseq.substring(i, j)).replaceAll(&amp;quot;[\\- ]&amp;quot;, &amp;quot;&amp;quot;);&#xA;                    &#xA;                    &#xA;                    &#xA;                    if(hit_shift&amp;lt;0)&#xA;                        {&#xA;                        StringBuilder sb=new StringBuilder(alt.length());&#xA;                        for(int x=alt.length()-1;x&amp;gt;=0;--x)&#xA;                            {&#xA;                            sb.append(AcidNucleics.complement(alt.charAt(x)));&#xA;                            }&#xA;                        alt=sb.toString();&#xA;                        &#xA;                        sb=new StringBuilder(ref.length());&#xA;                        for(int x=ref.length()-1;x&amp;gt;=0;--x)&#xA;                            {&#xA;                            sb.append(AcidNucleics.complement(ref.charAt(x)));&#xA;                            }&#xA;                        ref=sb.toString();&#xA;                        &#xA;                        &#xA;                        }&#xA;                        &#xA;                    pw.print(iter1.getIterationQueryDef());&#xA;                    pw.print('\t');&#xA;                    pw.print(hit.getHitDef());&#xA;                    pw.print('\t');&#xA;                    pw.print((1+hit_loop));&#xA;                    pw.print('\t');&#xA;                    pw.print((1+hsp_loop));&#xA;                    &#xA;                    pw.print('\t');&#xA;                    pw.print(query_index);&#xA;                    pw.print('\t');&#xA;                    pw.print(hit_index);&#xA;                    pw.print('\t');&#xA;                    pw.print(hit_shift==1?'+':'-');&#xA;                    &#xA;                    pw.print('\t');&#xA;                    pw.print(ref);&#xA;                    pw.print('\t');&#xA;                    pw.print(alt);&#xA;                    &#xA;                    &#xA;                    pw.print('\t');&#xA;                    pw.print(align_length);&#xA;                    pw.print('\t');&#xA;                    pw.print(hspHSeq.substring(i, j).replace(' ', '.'));&#xA;                    pw.print('\t');&#xA;                    pw.print(hspQseq.substring(i, j).replace(' ', '.'));&#xA;                    pw.print('\t');&#xA;                    pw.print(hspMid.substring(i, j).replace(' ', '.'));&#xA;&#xA;                    &#xA;                    pw.println();&#xA;                    &#xA;                    //marshaller.marshal(new JAXBElement&amp;lt;Hsp&amp;gt;(new QName(&amp;quot;Hsp&amp;quot;), Hsp.class, hsp), System.out);&#xA;                    //pw.println();&#xA;                    &#xA;                    while(i&amp;lt;j)&#xA;                        {&#xA;                        ch= hspHSeq.charAt(i);&#xA;                        cq= hspQseq.charAt(i);&#xA;&#xA;                        if(ch!='-' &amp;amp;&amp;amp; ch!=' ')&#xA;                            {&#xA;                            hit_index+=hit_shift;&#xA;                            }&#xA;                        if(cq!='-' &amp;amp;&amp;amp; cq!=' ')&#xA;                            {&#xA;                            query_index++;&#xA;                            }&#xA;                        ++i;&#xA;                        }&#xA;                    }&#xA;                if(hit_index-hit_shift!=hsp_hit_to)&#xA;                    {&#xA;                    marshaller.marshal(new JAXBElement&amp;lt;Hsp&amp;gt;(new QName(&amp;quot;Hsp&amp;quot;), Hsp.class, hsp),stderr());&#xA;                    throw new IllegalStateException(&amp;quot;Error expected hit-index= &amp;quot;+hit_index+&amp;quot;-&amp;quot;+hit_shift+&amp;quot; == hsp-hit-to=&amp;quot;+hsp_hit_to);&#xA;                    }&#xA;                if(query_index-1!=hsp_query_to)&#xA;                    {&#xA;                    marshaller.marshal(new JAXBElement&amp;lt;Hit&amp;gt;(new QName(&amp;quot;Hit&amp;quot;), Hit.class, hit),stderr());&#xA;                    throw new IllegalStateException(&amp;quot;query_index &amp;quot;+query_index+&amp;quot;(1 != hsp_query_to:&amp;quot;+hsp_query_to);&#xA;                    }&#xA;                if(pw.checkError()) break;&#xA;                }&#xA;            &#xA;            }//end loop Hit&#xA;        &#xA;        if(!found_mismatch)&#xA;            {&#xA;            numPerfectMath++;&#xA;            }&#xA;        &#xA;        }//end while read&#xA;    LOG.info(&amp;quot;ITERATIONS : &amp;quot; +numIterations);&#xA;    LOG.info(&amp;quot;ONLY_PERFECT_MATCH : &amp;quot; +numPerfectMath);&#xA;    LOG.info(&amp;quot;NO_HIT : &amp;quot; +numNoHit);&#xA;    }&#xA;    &#xA;    @Override&#xA;    public int doWork(final List&amp;lt;String&amp;gt; args) {&#xA;        PrintWriter pw=null;&#xA;        XMLEventReader rx=null;&#xA;        try&#xA;            {&#xA;            &#xA;            JAXBContext jc = JAXBContext.newInstance(&amp;quot;gov.nih.nlm.ncbi.blast&amp;quot;);&#xA;            this.unmarshaller=jc.createUnmarshaller();&#xA;            this.marshaller=jc.createMarshaller();&#xA;            this.marshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT,true);&#xA;            this.marshaller.setProperty(Marshaller.JAXB_FRAGMENT,true);&#xA;            XMLInputFactory xmlInputFactory=XMLInputFactory.newFactory();&#xA;            xmlInputFactory.setProperty(XMLInputFactory.IS_NAMESPACE_AWARE, Boolean.FALSE);&#xA;            xmlInputFactory.setProperty(XMLInputFactory.IS_COALESCING, Boolean.TRUE);&#xA;            xmlInputFactory.setProperty(XMLInputFactory.IS_REPLACING_ENTITY_REFERENCES, Boolean.TRUE);&#xA;            xmlInputFactory.setProperty(XMLInputFactory.IS_SUPPORTING_EXTERNAL_ENTITIES, Boolean.FALSE);&#xA;            xmlInputFactory.setXMLResolver(new XMLResolver()&#xA;                {&#xA;                @Override&#xA;                public Object resolveEntity(String arg0, String arg1, String arg2,&#xA;                        String arg3) throws XMLStreamException&#xA;                    {&#xA;                    LOG.info(&amp;quot;resolveEntity:&amp;quot; +arg0+&amp;quot;/&amp;quot;+arg1+&amp;quot;/&amp;quot;+arg2);&#xA;                    return null;&#xA;                    }&#xA;                });&#xA;            final String inputName= oneFileOrNull(args);&#xA;            if(inputName==null)&#xA;                {&#xA;                LOG.info(&amp;quot;Reading from stdin&amp;quot;);&#xA;                rx=xmlInputFactory.createXMLEventReader(stdin());&#xA;                }&#xA;            else&#xA;                {&#xA;                LOG.info(&amp;quot;Reading from &amp;quot;+inputName);&#xA;                rx=xmlInputFactory.createXMLEventReader(IOUtils.openURIForBufferedReading(inputName));&#xA;                }&#xA;            pw = super.openFileOrStdoutAsPrintWriter(outputFile);&#xA;            run(pw,rx);&#xA;            pw.flush();&#xA;            return RETURN_OK;&#xA;            }&#xA;        catch(Exception err)&#xA;            {&#xA;            LOG.error(err);&#xA;            return -1;&#xA;            }&#xA;        finally&#xA;            {&#xA;            CloserUtil.close(rx);&#xA;            CloserUtil.close(pw);&#xA;            }&#xA;        }&#xA;&#xA;    /**&#xA;     * @param args&#xA;     */&#xA;    public static void main(final String[] args) {&#xA;        new BlastPToSnp().instanceMainWithExit(args);&#xA;&#xA;    }&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17084" LastEditorUserId="298" LastEditDate="2023-02-28T11:06:18.053" LastActivityDate="2023-02-28T11:06:18.053" Title="Print indel/mismatch in a blastp stream" Tags="|blast|parsing|java|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20631" PostTypeId="1" CreationDate="2023-02-27T18:35:52.053" Score="0" ViewCount="145" Body="&lt;p&gt;I am trying to compare two ChIP-seq datasets for histone modifications, however, the Spearman and Pearson correlations between two particular datasets does not correspond to their peak overlap (peak overlap is very low, while both pearson and spearman tests are showing strong correlation). I am performing this analysis on Usegalaxy, first using multibamSummary on both bam files followed by plotCorrelation. What could be the error here?&lt;/p&gt;&#xA;" OwnerUserId="17086" LastActivityDate="2023-04-19T14:59:02.260" Title="I am trying to find correlation between two ChIP-seq datasets but the Pearson and Spearman test results do not correspond with the peak overlap" Tags="|chip-seq|correlation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20633" PostTypeId="2" ParentId="7229" CreationDate="2023-02-27T22:27:32.863" Score="2" Body="&lt;p&gt;Author of the package here, stumbled across this today.  The existing explanation is close, but not entirely correct.&lt;/p&gt;&#xA;&lt;p&gt;Our paper describes it like this: &amp;quot;We first created a fish plot of an AML patient with a chemotherapy-induced bottleneck that eliminated one subclone, while another survived and drove the relapse&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;In other words, the grey represents a tumor, starting from a single cell (point at far left). The tumor grows and expands, acquiring additional mutations (red) that increase the relative growth rate of a subclone.  When sampled at day 0 (when the patient showed up with cancer), the red subclone made up about a third of the tumor cells.  Chemotherapy was applied, shrinking the tumor dramatically and completely eliminating the red subclone (patient went into remission).&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, remission doesn't always mean &amp;quot;cured&amp;quot;, as this tumor persisted, and picked up more mutations (orange subclone), and then within that subclone, picked up even more (yellow subclone).  Each gave those cells sufficient growth advantage that at relapse, essentially all of the tumor cells were comprised of the orange subclone, with a subset of cells that contained both the orange and yellow mutations.&lt;/p&gt;&#xA;&lt;p&gt;There are also subtleties about whether the orange and yellow mutations pre-existed at very low levels (below the sensitivity of the sequencing that was performed), or were actually brand new mutations, but that's outside the scope of this explanation!&lt;/p&gt;&#xA;" OwnerUserId="74" LastActivityDate="2023-02-27T22:27:32.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20634" PostTypeId="1" CreationDate="2023-02-28T10:07:28.203" Score="1" ViewCount="240" Body="&lt;p&gt;I am relatively new to genome analysis and would like to compare 30+ vcf files, 1 sample per file. After filtering there are about ~20 variants per vcf file.&lt;/p&gt;&#xA;&lt;p&gt;I would like to evaluate to mutational landscape between samples and for each sample as well.&lt;/p&gt;&#xA;&lt;p&gt;One option that I found is a &lt;em&gt;&lt;strong&gt;mutation heatmap&lt;/strong&gt;&lt;/em&gt; where each row is a sample and each column a position in the genome. Is this the preferred way of displaying this type of information and how would I include gene information?&lt;/p&gt;&#xA;&lt;p&gt;At this point I am interested in getting an overview first of the affected positions and then dig deeper using e.g. zygosity.&lt;/p&gt;&#xA;&lt;p&gt;Thank you very much for helping out!&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;2nd edit&lt;/strong&gt;&#xA;I would also like to evaluate if there is a bias towards certain genes. For example are certain oncogenes mutated more frequently in this cohort or is the accumulation random and unique within each sample?&lt;/p&gt;&#xA;&lt;p&gt;Best,&#xA;M&lt;/p&gt;&#xA;" OwnerUserId="16874" LastEditorUserId="16874" LastEditDate="2023-03-02T14:44:08.110" LastActivityDate="2023-03-02T14:44:08.110" Title="variant visualisation of VCF files for 30+ samples" Tags="|vcf|variant-calling|visualization|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20636" PostTypeId="2" ParentId="20634" CreationDate="2023-02-28T17:12:49.200" Score="3" Body="&lt;p&gt;Single sample PVCF files are generally not combinable in the way you describe. PVCF stands for &amp;quot;project&amp;quot; VCF. As far as I know, PVCF isn't defined anywhere but, it refers to a multi-sample VCF that does not contain reference blocks. It only contains lines for which at least one sample has at least one alternate allele. We distinguish this from a GVCF which is a single-sample VCF that does contain reference blocks. We can't combine two PVCFs in a principled way because some variant might only appear in one PVCF. We lack the reference block information necessary to decide if a sample in the other VCF is 0/0 or &amp;quot;no call&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Do all the VCF files have the same set of variants? If all the VCF files have the same set of variants, you might try using &lt;a href=&quot;https://hail.is/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Hail&lt;/a&gt; (disclaimer: I'm a Hail maintainer):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import hail as hl&#xA;&#xA;vcf_files = ['sample1.vcf.bgz', ...]&#xA;mts = [hl.import_vcf(f) for f in vcf_files]&#xA;mt = hl.MatrixTable.union_cols(mts)  # assumes each file has the same set of variants&#xA;mt.write('combined.mt')  # save to Hail's native format so the following code is fast&#xA;&#xA;mt = hl.read_matrix_table('combined.mt')&#xA;mt.show(n_rows=5, n_cols=10)  # see a table of the first 5 rows and 10 cols&#xA;&#xA;mt = hl.variant_qc(mt)  # adds a bunch of variant QC annotations&#xA;mt.describe()  # inspect available annotations&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You mentioned &amp;quot;genomic distribution&amp;quot;. Are you referring to the allele frequency in this sample? You can see a genome-wide scatter plot of reference allele frequency like this:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from hail.ggplot import *&#xA;fig = ggplot(mt) + geom_point(aes(x=mt.locus, y=mt.variant_qc.AF[0]))&#xA;fig.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1087" LastEditorUserId="73" LastEditDate="2023-03-01T18:45:57.363" LastActivityDate="2023-03-01T18:45:57.363" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20637" PostTypeId="1" AcceptedAnswerId="20638" CreationDate="2023-02-28T18:48:10.693" Score="4" ViewCount="176" Body="&lt;p&gt;I have a dataset for mutation data and I want to calculate mutation frequencies across all genes&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;df&lt;/strong&gt; (This is only the small subset of data)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Gene name   Sample id   MUTATION_ID Mutation Description&#xA;ARID1B  2719660 171258500   Substitution - Missense&#xA;ARID1B  2719659     &#xA;ARID1B  2719661 171258501   Substitution - Missense&#xA;ARID1B  2719662     &#xA;ARID1B  2719663 171258501   Substitution - Nonsense&#xA;CD58    2878555 110346783   Substitution - Nonsense&#xA;CD58    2877956     &#xA;CD58    2878557     &#xA;CD58    2877958 110346784   Substitution - Nonsense&#xA;CD58    2878559 110346785   Substitution - Nonsense&#xA;CD58    2877960     &#xA;MRE11   2861617 123320443   Substitution - coding silent&#xA;MRE12   2861617 123320444   Substitution - coding &#xA;MRE13   2861617     &#xA;MRE14   2861617 123320445   Substitution - coding silent&#xA;MRE15   2861617     &#xA;MRE16   2861617 123320446   Substitution - coding &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The formula for calculating the mutation is&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Positives ÷ (Positives + Negatives) x 100&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;where,&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Positives&lt;/code&gt;  = No of samples where &lt;code&gt;MUTATION_ID&lt;/code&gt; is present&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Negative&lt;/code&gt; = No of samples &lt;code&gt;MUTATION_ID&lt;/code&gt; for the sample&lt;/p&gt;&#xA;&lt;p&gt;I want to calculate mutation frequency for every gene in the &lt;code&gt;column_1:Gene name&lt;/code&gt; with python script&lt;/p&gt;&#xA;&lt;p&gt;I tried the following code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df = df.groupby(&amp;quot;Gene name&amp;quot;).count()&#xA;Positives = df[&amp;quot;MUTATION_ID&amp;quot;]&#xA;Negatives = df[&amp;quot;Sample id&amp;quot;] - df[&amp;quot;MUTATION_ID&amp;quot;] &#xA;df['Mutation_Frequency'] = Positives / (Positives + Negatives) * 100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3766" LastEditorUserId="3766" LastEditDate="2023-02-28T20:05:17.713" LastActivityDate="2023-03-01T01:21:36.913" Title="calculating mutation frequencies for every gene" Tags="|python|mutations|pandas|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20638" PostTypeId="2" ParentId="20637" CreationDate="2023-02-28T22:17:41.100" Score="2" Body="&lt;p&gt;The code is fine. The only issue is you'd put \s characters in-between the column header names. I've place an additional &amp;quot;mutation_frequency&amp;quot; column because thats how I'd do it.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df = pd.read_csv('/pathtodir/test/bioinfo.csv',sep=&amp;quot;\s&amp;quot;,engine='python')&#xA;df = df.groupby(&amp;quot;Genename&amp;quot;).count()&#xA;df['Mutation_Frequency1'] = ((df[&amp;quot;MUTATION_ID&amp;quot;] / (df[&amp;quot;Sampleid&amp;quot;] + (df[&amp;quot;Sampleid&amp;quot;] - df[&amp;quot;MUTATION_ID&amp;quot;]))) * 100).round(2)&#xA;df['Mutation_Frequency2'] = (df[&amp;quot;MUTATION_ID&amp;quot;] / df[&amp;quot;Sampleid&amp;quot;])*100&#xA;df.drop(['Description', 'Mutation'], axis=1, inplace=True)&#xA;print(df)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;-&lt;/th&gt;&#xA;&lt;th&gt;Sampleid&lt;/th&gt;&#xA;&lt;th&gt;MUTATION_ID&lt;/th&gt;&#xA;&lt;th&gt;Mutation_Frequency1&lt;/th&gt;&#xA;&lt;th&gt;Mutation_Frequency2&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;ARID1B&lt;/td&gt;&#xA;&lt;td&gt;5&lt;/td&gt;&#xA;&lt;td&gt;3&lt;/td&gt;&#xA;&lt;td&gt;42.86&lt;/td&gt;&#xA;&lt;td&gt;60.0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;CD58&lt;/td&gt;&#xA;&lt;td&gt;6&lt;/td&gt;&#xA;&lt;td&gt;3&lt;/td&gt;&#xA;&lt;td&gt;33.33&lt;/td&gt;&#xA;&lt;td&gt;50.0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;MRE11&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;100.00&lt;/td&gt;&#xA;&lt;td&gt;100.0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;MRE12&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;100.00&lt;/td&gt;&#xA;&lt;td&gt;100.0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;MRE13&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0.00&lt;/td&gt;&#xA;&lt;td&gt;0.0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;MRE14&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;100.00&lt;/td&gt;&#xA;&lt;td&gt;100.0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;MRE15&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0.00&lt;/td&gt;&#xA;&lt;td&gt;0.0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;MRE16&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;100.00&lt;/td&gt;&#xA;&lt;td&gt;100.0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-03-01T01:21:36.913" LastActivityDate="2023-03-01T01:21:36.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20639" PostTypeId="1" CreationDate="2023-03-01T08:24:07.870" Score="2" ViewCount="50" Body="&lt;p&gt;I have 3 groups for the RNA-seq analysis (Control, treatment A and treatment B). There are 2 replicates for control and treatment A and 3 replicates for treatment B (lost 2 replicates due to a mistake in sample handling). May I ask how will the inconsistent replicate numbers impact the results (e.g. DEGs)?&lt;/p&gt;&#xA;" OwnerUserId="15643" LastActivityDate="2023-03-01T17:48:53.430" Title="Inconsistent replicate numbers in RNA-seq" Tags="|rna-seq|genomics|differential-expression|gene-expression|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20640" PostTypeId="1" CreationDate="2023-03-01T08:53:19.497" Score="1" ViewCount="152" Body="&lt;p&gt;I have MAG geneome sequence released in NCBI. In the released sequence there are 4915 contigs present. I want to get the full length genome sequence as FASTA file to proceed with further works like circular genome viewing and other molecular typing. Kindly let me know how to proceed in this regard. I am not much aware of bioinformatics so need even the basic information in this regard.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="17094" LastActivityDate="2023-03-02T12:02:36.867" Title="How to align multiple contigs? To generate full length genome" Tags="|fasta|wgs|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20641" PostTypeId="1" CreationDate="2023-03-01T13:02:02.307" Score="3" ViewCount="266" Body="&lt;p&gt;I am analyzing the &lt;a href=&quot;https://www.rcsb.org/3d-view/7VF2&quot; rel=&quot;nofollow noreferrer&quot;&gt;7VF2&lt;/a&gt; structure, from the RCSB-database. I noticed that some residues belonging to one of its chains interact with the other chains that are present within the assembly. My question is if there is a method to extract a list of interacting residues that are present in a PDB.&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/O39sF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/O39sF.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Just as an example, we have Lys 1616 belonging to ZC3H13 chain that interacts with Glu 621 of the protein virilizer homolog (the first chain)&lt;/p&gt;&#xA;&lt;p&gt;Mol* 3D visualizer can be used for major information in the 3D View in the link page I provided&lt;/p&gt;&#xA;" OwnerUserId="9142" LastEditorUserId="9142" LastEditDate="2023-03-01T13:38:15.513" LastActivityDate="2023-03-03T19:26:42.993" Title="How to extract a list of hydrogen bonds from a PDB?" Tags="|pdb|structural-biology|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20642" PostTypeId="1" AcceptedAnswerId="20645" CreationDate="2023-03-01T14:05:38.123" Score="3" ViewCount="683" Body="&lt;p&gt;I previously used albacore version 2.3.1 to make initial move tables, but then I re-squiggle using Tombo version 1.5.1 to fix the errors.&lt;/p&gt;&#xA;&lt;p&gt;Example of move table produced by albacore, in which all the moves have the same size:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/XhTUF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XhTUF.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;When it is re-squiggled by Tombo (which produces variable length moves/blocks):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MdHhr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MdHhr.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am starting a new project, and I can repeat the same pipeline, however, my concern is that albacore version 2.3.1 is too old now and I was wondering to ask if there are better pipelines/software for base-calling that produces initial move tables so that I can build on top of.&lt;/p&gt;&#xA;" OwnerUserId="15341" LastActivityDate="2023-03-02T21:51:14.480" Title="Which basecaller for nanopore is the best to produce event tables with information about the block size/move table?" Tags="|nanopore|base-calling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20643" PostTypeId="1" CreationDate="2023-03-01T15:55:52.233" Score="2" ViewCount="35" Body="&lt;p&gt;I am working with a whole genus of land snails, trying to predict its distribution in the coming decades and how it might get affected by climate change. I have already worked with MaxENT and the R package Biomod2 (I have tried out all the important algorithms within it as well). I was wondering if any of you can recommend another option thats reliable.&lt;/p&gt;&#xA;" OwnerUserId="16657" LastActivityDate="2024-03-30T21:02:50.467" Title="Can you recommend me some programs or R packages for species distribution modelling?" Tags="|modelling|ecology|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20644" PostTypeId="2" ParentId="20639" CreationDate="2023-03-01T17:48:53.430" Score="2" Body="&lt;p&gt;It will not do any harm other than a reduction in statistical power compared to more samples per group. Common analysis tools such as DESeq2, edgeR and limma will work on unequal sample size just fine.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-03-01T17:48:53.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20645" PostTypeId="2" ParentId="20642" CreationDate="2023-03-01T18:37:46.967" Score="3" Body="&lt;p&gt;ONT's newest basecaller, Dorado, can do this for SAM output with the &lt;code&gt;--emit-moves&lt;/code&gt; argument. It was added in &lt;a href=&quot;https://github.com/nanoporetech/dorado/commit/f5371d16cfd7eef71995edbfeeb86df7f4201440&quot; rel=&quot;nofollow noreferrer&quot;&gt;this commit&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Dorado can be downloaded from here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/nanoporetech/dorado&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nanoporetech/dorado&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here's an example command line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ ~/install/dorado/dorado-0.2.1-linux-x64/bin/dorado basecaller --emit-moves \&#xA;  ~/install/dorado/dorado-0.2.1-linux-x64/dna_r10.4.1_e8.2_260bps_sup@v4.1.0 DAE_pod5 | \&#xA;  samtools view -b - &amp;gt; called_DAE_gDNA_reads_all.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And a few lines of example output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@HD     VN:1.6  SO:unknown&#xA;@PG     ID:basecaller   PN:dorado       VN:0.2.1+c70423e        CL:dorado basecaller --emit-moves /home/minion/install/dorado/dorado-0.2.1-linux-x64/dna_r10.4.1_e8.2_260bps_sup@v4.1.0 DAE_pod5&#xA;@PG     ID:samtools     PN:samtools     PP:basecaller   VN:1.10 CL:samtools view -b -&#xA;@PG     ID:samtools.1   PN:samtools     PP:samtools     VN:1.10 CL:samtools view -h called_DAE_gDNA_reads_all.bam&#xA;@RG     ID:460fc479-3b30-4ed5-801f-a82bbe556101_dna_r10.4.1_e8.2_260bps_sup@v4.1.0      PU:PAK83314     PM:     DT:2023-02-08T10:22:53.256+00:00        PL:ONT  DS:basecall_model=dna_r10.4.1_e8.2_260bps_sup@v4.1.0 runid=460fc479-3b30-4ed5-801f-a82bbe556101 LB:DAE_gDNA     SM:DAE_gDNA&#xA;@RG     ID:227379b1-c5fc-465d-9e24-eb771133f11e_dna_r10.4.1_e8.2_260bps_sup@v4.1.0      PU:PAK83314     PM:     DT:2023-02-08T20:28:34.34+00:00 PL:ONT  DS:basecall_model=dna_r10.4.1_e8.2_260bps_sup@v4.1.0 runid=227379b1-c5fc-465d-9e24-eb771133f11e LB:no_sample    SM:no_sample&#xA;003d84eb-ce0f-47ea-9c7c-fe939fe8702e    4       *       0       0       *       *       0       110     GTGCCTCCTTGCTTTGGTTCAGTTCTATCCAGCCTTTAGATATCCCAGCCCTCACAGGGTTTGTGTGACCTTGTTGCATTATCATGCGAAACACTTTCGCGTTTTTAGTG  %%%%%(&amp;amp;%%&amp;amp;%%'((''))*.-,-''&amp;amp;&amp;amp;&amp;amp;'&amp;amp;&amp;amp;&amp;amp;(%$$$%&amp;amp;&amp;amp;&amp;amp;'%%%%&amp;amp;(*))'&amp;amp;&amp;amp;&amp;amp;&amp;amp;)-05&amp;gt;6;&amp;gt;&amp;gt;777887++'&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;%%%&amp;amp;*.....0820001:;;;&amp;lt;&amp;lt;87''&amp;amp;&amp;amp;  qs:i:7  du:f:0.526      ns:i:2104       ts:i:10 mx:i:4  ch:i:1037       st:Z:2023-02-09T07:07:00.271+00:00      rn:i:96929      f5:Z:PAK83314_fail_unclassified_227379b1_ea81734c_853.pod5      sm:f:98.674     sd:f:22.395     sv:Z:quantile   RG:Z:227379b1-c5fc-465d-9e24-eb771133f11e_dna_r10.4.1_e8.2_260bps_sup@v4.1.0    mv:B:c,5,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,1,0,0,1,1,0,0,1,0,0,1,0,0,0,0,1,0,1,1,0,1,0,1,0,1,0,1,0,0,1,0,1,1,0,0,1,0,1,0,0,1,0,0,1,0,0,1,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,0,0,1,0,1,0,1,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,1,1,0,0,1,0,0,0,0,0,1,0,1,0,1,1,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,0,1,0,0,1,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,0,1,0,1,0,0,0,1,0,0,0,1,0,1,0,1,0,0,1,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,1,0,1,0,0,0,0,0,1,0,0,0,0,0,0,0,1,0,0,1,0,0,1,0,1,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,1,0,0,1,0,1,1,0,0,1,0,1,0,0,1,0,1,0,1,0,0,0,0,1,0,0,0,1,0,0,1,0,0,1,1,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,0,0,1,0,1,0,1&#xA;002e48ca-84bf-4419-8370-943d1ca3ac64    4       *       0       0       *       *       0       107     TATGTCTAACTTACTTCGTTCAGTTGTGAAAACTGGGTGTTTAACCTTTGATTCTATCGTGTTTCCCTAGTTTTCGCATTTGACGTGAAACGCTTTCACATTTTGGT     &amp;amp;&amp;amp;'''&amp;amp;%%%%((((*+)***())((')&amp;amp;%%%%$%%%'--,,,,-6996/,+-5ABBBCCFBCE&amp;gt;88:1764424/--,,--&amp;amp;%$$%*08??;;;;&amp;lt;B&amp;lt;:((('()//     qs:i:9  du:f:0.92975    ns:i:3719       ts:i:10 mx:i:4  ch:i:1058       st:Z:2023-02-09T07:07:08.905+00:00      rn:i:62167      f5:Z:PAK83314_fail_unclassified_227379b1_ea81734c_853.pod5      sm:f:85.094     sd:f:23.944     sv:Z:quantile   RG:Z:227379b1-c5fc-465d-9e24-eb771133f11e_dna_r10.4.1_e8.2_260bps_sup@v4.1.0    mv:B:c,5,1,0,0,0,0,0,0,1,0,0,0,0,0,1,1,0,0,0,0,0,1,0,0,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,1,0,0,0,0,0,0,1,1,1,0,1,0,1,0,0,1,0,0,1,0,1,1,0,0,1,0,1,0,0,0,0,1,0,0,1,0,1,0,0,1,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,1,0,0,0,1,0,1,0,0,1,0,1,0,1,1,0,1,0,0,0,0,0,1,0,0,1,0,0,1,0,1,0,1,0,0,1,1,0,0,0,0,0,0,0,0,0,1,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,1,0,0,0,0,0,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,0,1,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,1,0,1,0,1,1,0,1,0,0,1,0,1,0,0,1,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,0,1,0,0,0,0,1,0,1,0,0,1,0,0,0,1,0,0&#xA;0138d83a-da9f-4468-bafc-b974c4e9eece    4       *       0       0       *       *       0       234     ATGTTTATGTGTTCTTCGTTAGTAAGCCACCGATATTGACTCCAGGGTTTGTGTAACCTTGTTTTCGCATTTATCGTGAAACGCTTTCGCGTTTTTCGTCGCTTCAAATAGTTTTAATTCAGTTGTCTAAAAATGTTAACATATGGGTATTATTTCTCTAGAACATTTTAGTTGTGTAATTTCTACCATTTAAATGGGGTGAATAAATAACCAAATCATTGAACAACTGGAATG      &amp;amp;&amp;amp;&amp;amp;')'&amp;amp;%$$##$&amp;amp;&amp;amp;''+,-)''&amp;amp;'')**'&amp;amp;%&amp;amp;''))&amp;amp;&amp;amp;&amp;amp;)'((*+8&amp;lt;AEEFGHJIJIJIICA@;843344:254,,,,CEE&amp;gt;====7&amp;lt;&amp;lt;&amp;lt;&amp;gt;&amp;gt;@=?+***'&amp;amp;&amp;amp;''(''&amp;amp;%%%%$%%&amp;amp;)1//..-+,+*'&amp;amp;'(+2+++45&amp;lt;ACCC66677@75556+((((55&amp;gt;@AEFHFHGFEHHJHHFHIKJGFFFKLHZEGEGGIDFDDCDDERKIEGDG&amp;gt;=&amp;gt;5332/'&amp;amp;&amp;amp;&amp;amp;&amp;amp;)**,2242.      qs:i:10 du:f:0.69975    ns:i:2799       ts:i:10 mx:i:4  ch:i:1933       st:Z:2023-02-09T07:07:01.924+00:00      rn:i:77788      f5:Z:PAK83314_fail_unclassified_227379b1_ea81734c_853.pod5      sm:f:97.397     sd:f:27.276     sv:Z:quantile   RG:Z:227379b1c5fc-465d-9e24-eb771133f11e_dna_r10.4.1_e8.2_260bps_sup@v4.1.0    mv:B:c,5,1,0,0,0,1,0,1,0,1,0,1,0,0,1,1,0,0,1,1,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,1,0,0,1,1,1,0,0,1,0,0,0,1,0,0,1,0,1,0,0,1,0,1,0,0,1,0,0,0,0,0,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,0,1,0,0,0,0,0,0,0,0,1,0,1,0,1,0,1,0,0,1,0,1,1,0,0,1,1,0,0,0,1,0,0,0,1,0,0,1,0,0,1,0,1,0,0,0,0,1,1,0,0,1,0,1,0,0,1,0,1,0,0,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,0,1,0,1,0,0,1,0,0,1,0,0,1,0,0,0,1,0,1,0,1,0,1,1,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,1,0,0,0,0,0,0,0,1,1,0,0,0,1,0,1,0,1,0,1,0,0,0,1,0,1,0,1,0,1,0,1,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,1,1,0,1,1,0,1,0,1,1,0,1,0,1,0,0,0,0,0,0,1,0,1,0,1,0,1,0,1,1,1,0,1,0,1,0,1,1,1,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,1,0,1,1,0,1,0,1,1,1,1,0,1,0,1,1,1,0,0,1,1,0,0,0,0,1,0,1,0,1,0,0,1,0,0,0,0,0,1,0,1,0,1,0,1,0,1,0,1,1,0,1,0,1,1,1,1,0,0,1,0,1,1,0,1,0,1,0,0,0,0,0,1,0,1,0,0,0,0,0,0,0,0,1,0,1,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,0,0,1,0,1,1,0,1,0,1,0,1,0,1,0,1,0,1,0,1,0,0,1,0,1,0,0,1,0,1,0,0,1,0,0,1,0,1,0,1,0,1,0,1,0,1,0,1,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,0,1,0,1,1,0,1,0,1,0,1,1,0,1,1,0,1,1,0,1,0,1,0,0,1,0,0,1,1,0,0,1,0,0,1,0,0,1,1,1,1,0,0,1,0,1,0,1,0,1,0,1,0,1,1,0,1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Based on the commit, the move table is attached to the &lt;code&gt;mv&lt;/code&gt; &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;tag&lt;/a&gt; for each read, indicating an array of 8-bit signed integers (represented as comma-separated values in the SAM output). I'd guess (based on how the calling has worked in the past) that these represent strides comprising a fixed number of consecutive samples, so (as before) it'll need correction to convert the stride width to variable width events.&lt;/p&gt;&#xA;&lt;p&gt;For the model I'm using here, my guess from the model toml file is that the stride width is 5:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ grep -B 8 stride ~/install/dorado/dorado-0.2.1-linux-x64/dna_r10.4.1_e8.2_260bps_sup@v4.1.0/config.toml &#xA;[encoder]&#xA;type = &amp;quot;serial&amp;quot;&#xA;[[encoder.sublayers]]&#xA;type = &amp;quot;convolution&amp;quot;&#xA;insize = 1&#xA;size = 16&#xA;bias = true&#xA;winlen = 5&#xA;stride = 1&#xA;--&#xA;max = 3.5&#xA;&#xA;[[encoder.sublayers]]&#xA;type = &amp;quot;convolution&amp;quot;&#xA;insize = 16&#xA;size = 16&#xA;bias = true&#xA;winlen = 5&#xA;stride = 1&#xA;--&#xA;max = 3.5&#xA;&#xA;[[encoder.sublayers]]&#xA;type = &amp;quot;convolution&amp;quot;&#xA;insize = 16&#xA;size = 1024&#xA;bias = true&#xA;winlen = 19&#xA;stride = 5&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-03-02T21:51:14.480" LastActivityDate="2023-03-02T21:51:14.480" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20646" PostTypeId="1" CreationDate="2023-03-01T20:47:11.943" Score="3" ViewCount="90" Body="&lt;p&gt;I am doing a project where I am reproducing the analysis from the article &amp;quot;sangeranalyseR: Simple and Interactive Processing of Sanger Sequencing Data in R&amp;quot;. Below is the example chunk for Sanger Alignment. Below are the package and datasets.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(sangeranalyseR)&#xA;data(&amp;quot;sangerAlignmentData&amp;quot;)&#xA;data(&amp;quot;sangerContigData&amp;quot;)&#xA;data(&amp;quot;sangerReadFData&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Below is the code for Sanger Alignment, similar to the code for Sanger Contig and Sanger Reading.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;rawDataDir &amp;lt;- system.file(&amp;quot;extdata&amp;quot;, package = &amp;quot;sangeranalyseR&amp;quot;)&#xA;fastaFN &amp;lt;- file.path(rawDataDir, &amp;quot;fasta&amp;quot;, &amp;quot;SangerAlignment&amp;quot;, &amp;quot;Sanger_all_reads.fa&amp;quot;)&#xA;my_sangerAlignmentFa &amp;lt;- new(&amp;quot;SangerAlignment&amp;quot;,&#xA;                            inputSource          = &amp;quot;FASTA&amp;quot;,&#xA;                            processMethod        = &amp;quot;REGEX&amp;quot;,&#xA;                            FASTA_File           = fastaFN,&#xA;                            REGEX_SuffixForward  = &amp;quot;_[0-9]*_F&lt;span class=&quot;math-container&quot;&gt;$&quot;,&#xA;                        REGEX_SuffixReverse  = &quot;_[0-9]*_R$&lt;/span&gt;&amp;quot;,&#xA;                            refAminoAcidSeq      = &amp;quot;SRQWLFSTNHKDIGTLYFIFGAWAGMVGTSLSILIRAELGHPGALIGDDQIYNVIVTAHAFIMIFFMVMPIMIGGFGNWLVPLMLGAPDMAFPRMNNMSFWLLPPALSLLLVSSMVENGAGTGWTVYPPLSAGIAHGGASVDLAIFSLHLAGISSILGAVNFITTVINMRSTGISLDRMPLFVWSVVITALLLLLSLPVLAGAITMLLTDRNLNTSFFDPAGGGDPILYQHLFWFFGHPEVYILILPGFGMISHIISQESGKKETFGSLGMIYAMLAIGLLGFIVWAHHMFTVGMDVDTRAYFTSATMIIAVPTGIKIFSWLATLHGTQLSYSPAILWALGFVFLFTVGGLTGVVLANSSVDIILHDTYYVVAHFHYVLSMGAVFAIMAGFIHWYPLFTGLTLNNKWLKSHFIIMFIGVNLTFFPQHFLGLAGMPRRYSDYPDAYTTWNIVSTIGSTISLLGILFFFFIIWESLVSQRQVIYPIQLNSSIEWYQNTPPAEHSYSELPLLTN&amp;quot;,&#xA;                            minReadsNum           = 2,&#xA;                            minReadLength         = 20,&#xA;                            minFractionCall       = 0.5,&#xA;                            maxFractionLost       = 0.5,&#xA;                            geneticCode           = GENETIC_CODE,&#xA;                            acceptStopCodons      = TRUE,&#xA;                            readingFrame          = 1,&#xA;                            processorsNum         = 1)&#xA;&#xA;launchApp(my_sangerAlignmentFa)&#xA;&#xA;writeFasta(my_sangerAlignmentFa,&#xA;           outputDir         = tempdir(),&#xA;           compress          = FALSE,&#xA;           compression_level = NA,&#xA;           selection         = &amp;quot;all&amp;quot;)&#xA;&#xA;generateReport(my_sangerAlignmentFa,&#xA;               outputDir           = tempdir(),&#xA;               includeSangerRead   = TRUE,&#xA;               includeSangerContig = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There are two problems that occurred for me. One is that the launchapp() function created an error for me stating&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;INFO [2023-03-01 14:25:52] Your input is 'SangerAlignment' S4 instance&#xA;INFO [2023-03-01 14:25:52] SangerAlignment with 'FASTA' inputSource&#xA;cannot run Shiny app (You don't need to do trimming or base calling)&#xA;NULL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I fix this error?&lt;/p&gt;&#xA;&lt;p&gt;Another error that occurred within knitting. I ran the same code without the launchapp() function and an error that I never seen before said this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Quitting from lines NA-219 (SangerRead_Report_fasta.Rmd)  Error in parse_block(g[-1], g[1], params.src, markdown_mode) : &#xA;Duplicate chunk label 'unnamed-chunk-1', which has been used for the chunk:&#xA;library(sangeranalyseR) library(rentrez) Calls: &amp;lt;Anonymous&amp;gt; ...&#xA;process_file -&amp;gt; split_file -&amp;gt; lapply -&amp;gt; FUN -&amp;gt; parse_block&#xA;&#xA;Execution halted&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is it my RStudio or is the code itself. Is there anything I am missing?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sessionInfo(sangeranalysesR)&#xA;R version 4.2.2 (2022-10-31 ucrt)&#xA;Platform: x86_64-w64-mingw32/x64 (64-bit)&#xA;Running under: Windows 10 x64 (build 22621)&#xA;&#xA;Matrix products: default&#xA;&#xA;locale:&#xA;[1] LC_COLLATE=English_United States.utf8 &#xA;[2] LC_CTYPE=English_United States.utf8   &#xA;[3] LC_MONETARY=English_United States.utf8&#xA;[4] LC_NUMERIC=C                          &#xA;[5] LC_TIME=English_United States.utf8    &#xA;&#xA;attached base packages:&#xA;character(0)&#xA;&#xA;other attached packages:&#xA;[1] sangeranalyseR_1.8.0&#xA;&#xA;loaded via a namespace (and not attached):&#xA;  [1] utils_4.2.2            nlme_3.1-161          &#xA;  [3] fs_1.6.1               bitops_1.0-7          &#xA;  [5] bit64_4.0.5            httr_1.4.4            &#xA;  [7] GenomeInfoDb_1.34.6    tools_4.2.2           &#xA;  [9] bslib_0.4.2            utf8_1.2.2            &#xA; [11] R6_2.5.1               DT_0.27               &#xA; [13] DBI_1.1.3              lazyeval_0.2.2        &#xA; [15] BiocGenerics_0.44.0    colorspace_2.1-0      &#xA; [17] rhdf5filters_1.10.0    ade4_1.7-20           &#xA; [19] withr_2.5.0            tidyselect_1.2.0      &#xA; [21] gridExtra_2.3          phangorn_2.11.1       &#xA; [23] bit_4.0.5              compiler_4.2.2        &#xA; [25] cli_3.5.0              datasets_4.2.2        &#xA; [27] shinyjs_2.1.0          plotly_4.10.1         &#xA; [29] ggdendro_0.1.23        base_4.2.2            &#xA; [31] sass_0.4.5             scales_1.2.1          &#xA; [33] quadprog_1.5-8         stringr_1.5.0         &#xA; [35] digest_0.6.31          rmarkdown_2.20        &#xA; [37] sangerseqR_1.34.0      XVector_0.38.0        &#xA; [39] pkgconfig_2.0.3        htmltools_0.5.4       &#xA; [41] sessioninfo_1.2.2      fastmap_1.1.0         &#xA; [43] excelR_0.4.0           grDevices_4.2.2       &#xA; [45] htmlwidgets_1.6.1      rlang_1.0.6           &#xA; [47] rstudioapi_0.14        RSQLite_2.2.20        &#xA; [49] shiny_1.7.4            jquerylib_0.1.4       &#xA; [51] generics_0.1.3         jsonlite_1.8.4        &#xA; [53] BiocParallel_1.32.5    zip_2.2.2             &#xA; [55] dplyr_1.1.0            RCurl_1.98-1.10       &#xA; [57] magrittr_2.0.3         GenomeInfoDbData_1.2.9&#xA; [59] biomformat_1.26.0      Matrix_1.5-3          &#xA; [61] Rhdf5lib_1.20.0        Rcpp_1.0.10           &#xA; [63] munsell_0.5.0          S4Vectors_0.36.1      &#xA; [65] fansi_1.0.4            logger_0.2.2          &#xA; [67] DECIPHER_2.26.0        ape_5.6-2             &#xA; [69] shinycssloaders_1.0.0  lifecycle_1.0.3       &#xA; [71] yaml_2.3.7             stringi_1.7.12        &#xA; [73] MASS_7.3-58.2          zlibbioc_1.44.0       &#xA; [75] rhdf5_2.42.0           plyr_1.8.8            &#xA; [77] grid_4.2.2             blob_1.2.3            &#xA; [79] parallel_4.2.2         promises_1.2.0.1      &#xA; [81] shinydashboard_0.7.2   crayon_1.5.2          &#xA; [83] methods_4.2.2          lattice_0.20-45       &#xA; [85] Biostrings_2.66.0      zeallot_0.1.0         &#xA; [87] knitr_1.42             pillar_1.8.1          &#xA; [89] igraph_1.3.5           seqinr_4.2-23         &#xA; [91] reshape2_1.4.4         codetools_0.2-19      &#xA; [93] stats4_4.2.2           fastmatch_1.1-3       &#xA; [95] reprex_2.0.2           BiocVersion_3.16.0    &#xA; [97] glue_1.6.2             evaluate_0.20         &#xA; [99] BiocManager_1.30.20    data.table_1.14.6     &#xA;[101] vctrs_0.5.2            httpuv_1.6.8          &#xA;[103] graphics_4.2.2         gtable_0.3.1          &#xA;[105] purrr_1.0.1            tidyr_1.3.0           &#xA;[107] cachem_1.0.6           ggplot2_3.4.0         &#xA;[109] openxlsx_4.2.5.1       xfun_0.36             &#xA;[111] mime_0.12              xtable_1.8-4          &#xA;[113] later_1.3.0            viridisLite_0.4.1     &#xA;[115] tibble_3.1.8           stats_4.2.2           &#xA;[117] memoise_2.0.1          IRanges_2.32.0        &#xA;[119] shinyWidgets_0.7.6     ellipsis_0.3.2        &#xA;[121] BiocStyle_2.26.0&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17097" LastEditorUserId="17097" LastEditDate="2023-03-03T07:46:30.690" LastActivityDate="2023-03-03T07:46:30.690" Title="Sanger Sequencing Knitting Error" Tags="|r|sequence-alignment|bioconductor|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20647" PostTypeId="2" ParentId="20646" CreationDate="2023-03-02T04:13:57.590" Score="3" Body="&lt;p&gt;Your code runs without error on my laptop (macOS 13.2.1; R v4.2.2; Rstudio 2022.07.2), so I have some further questions that might help with troubleshooting (too many to include in a comment):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Does the &lt;a href=&quot;https://github.com/roblanf/sangeranalyseR#quick-start-guide&quot; rel=&quot;nofollow noreferrer&quot;&gt;example from the docs&lt;/a&gt; run without errors/warnings on your system?&lt;/li&gt;&#xA;&lt;li&gt;Did you install the sangeranalyseR package via Bioconductor (per the &lt;a href=&quot;https://github.com/roblanf/sangeranalyseR#quick-start-guide&quot; rel=&quot;nofollow noreferrer&quot;&gt;docs&lt;/a&gt;)?&lt;/li&gt;&#xA;&lt;li&gt;Do you have the most up-to-date Bioconducter installed?&lt;/li&gt;&#xA;&lt;li&gt;Have you tried running the example code in a 'fresh' session?&lt;/li&gt;&#xA;&lt;li&gt;Are you able to please run the command &lt;code&gt;sessionInfo()&lt;/code&gt; and edit your question to include the output?&lt;/li&gt;&#xA;&lt;li&gt;If you run your example via the &lt;a href=&quot;https://reprex.tidyverse.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;reprex package&lt;/a&gt;, are there any differences between your output and my output?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The code from your question run on my laptop:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# BiocManager::install(&amp;quot;sangeranalyseR&amp;quot;)&#xA;library(sangeranalyseR)&#xA;#&amp;gt; Loading required package: stringr&#xA;#&amp;gt; Loading required package: ape&#xA;#&amp;gt; Loading required package: Biostrings&#xA;#&amp;gt; Loading required package: BiocGenerics&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'BiocGenerics'&#xA;#&amp;gt; The following objects are masked from 'package:stats':&#xA;#&amp;gt; &#xA;#&amp;gt;     IQR, mad, sd, var, xtabs&#xA;#&amp;gt; The following objects are masked from 'package:base':&#xA;#&amp;gt; &#xA;#&amp;gt;     anyDuplicated, aperm, append, as.data.frame, basename, cbind,&#xA;#&amp;gt;     colnames, dirname, do.call, duplicated, eval, evalq, Filter, Find,&#xA;#&amp;gt;     get, grep, grepl, intersect, is.unsorted, lapply, Map, mapply,&#xA;#&amp;gt;     match, mget, order, paste, pmax, pmax.int, pmin, pmin.int,&#xA;#&amp;gt;     Position, rank, rbind, Reduce, rownames, sapply, setdiff, sort,&#xA;#&amp;gt;     table, tapply, union, unique, unsplit, which.max, which.min&#xA;#&amp;gt; Loading required package: S4Vectors&#xA;#&amp;gt; Loading required package: stats4&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'S4Vectors'&#xA;#&amp;gt; The following objects are masked from 'package:base':&#xA;#&amp;gt; &#xA;#&amp;gt;     expand.grid, I, unname&#xA;#&amp;gt; Loading required package: IRanges&#xA;#&amp;gt; Loading required package: XVector&#xA;#&amp;gt; Loading required package: GenomeInfoDb&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'Biostrings'&#xA;#&amp;gt; The following object is masked from 'package:ape':&#xA;#&amp;gt; &#xA;#&amp;gt;     complement&#xA;#&amp;gt; The following object is masked from 'package:base':&#xA;#&amp;gt; &#xA;#&amp;gt;     strsplit&#xA;#&amp;gt; Loading required package: DECIPHER&#xA;#&amp;gt; Loading required package: RSQLite&#xA;#&amp;gt; Loading required package: parallel&#xA;#&amp;gt; Loading required package: reshape2&#xA;#&amp;gt; Loading required package: phangorn&#xA;#&amp;gt; Loading required package: sangerseqR&#xA;#&amp;gt; Loading required package: gridExtra&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'gridExtra'&#xA;#&amp;gt; The following object is masked from 'package:BiocGenerics':&#xA;#&amp;gt; &#xA;#&amp;gt;     combine&#xA;#&amp;gt; Loading required package: shiny&#xA;#&amp;gt; Loading required package: shinydashboard&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'shinydashboard'&#xA;#&amp;gt; The following object is masked from 'package:graphics':&#xA;#&amp;gt; &#xA;#&amp;gt;     box&#xA;#&amp;gt; Loading required package: shinyjs&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'shinyjs'&#xA;#&amp;gt; The following object is masked from 'package:shiny':&#xA;#&amp;gt; &#xA;#&amp;gt;     runExample&#xA;#&amp;gt; The following object is masked from 'package:RSQLite':&#xA;#&amp;gt; &#xA;#&amp;gt;     show&#xA;#&amp;gt; The following object is masked from 'package:Biostrings':&#xA;#&amp;gt; &#xA;#&amp;gt;     show&#xA;#&amp;gt; The following object is masked from 'package:GenomeInfoDb':&#xA;#&amp;gt; &#xA;#&amp;gt;     show&#xA;#&amp;gt; The following object is masked from 'package:XVector':&#xA;#&amp;gt; &#xA;#&amp;gt;     show&#xA;#&amp;gt; The following object is masked from 'package:IRanges':&#xA;#&amp;gt; &#xA;#&amp;gt;     show&#xA;#&amp;gt; The following object is masked from 'package:S4Vectors':&#xA;#&amp;gt; &#xA;#&amp;gt;     show&#xA;#&amp;gt; The following object is masked from 'package:stats4':&#xA;#&amp;gt; &#xA;#&amp;gt;     show&#xA;#&amp;gt; The following objects are masked from 'package:methods':&#xA;#&amp;gt; &#xA;#&amp;gt;     removeClass, show&#xA;#&amp;gt; Loading required package: data.table&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'data.table'&#xA;#&amp;gt; The following objects are masked from 'package:reshape2':&#xA;#&amp;gt; &#xA;#&amp;gt;     dcast, melt&#xA;#&amp;gt; The following object is masked from 'package:IRanges':&#xA;#&amp;gt; &#xA;#&amp;gt;     shift&#xA;#&amp;gt; The following objects are masked from 'package:S4Vectors':&#xA;#&amp;gt; &#xA;#&amp;gt;     first, second&#xA;#&amp;gt; Loading required package: plotly&#xA;#&amp;gt; Loading required package: ggplot2&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'plotly'&#xA;#&amp;gt; The following object is masked from 'package:ggplot2':&#xA;#&amp;gt; &#xA;#&amp;gt;     last_plot&#xA;#&amp;gt; The following object is masked from 'package:phangorn':&#xA;#&amp;gt; &#xA;#&amp;gt;     add_boxplot&#xA;#&amp;gt; The following object is masked from 'package:XVector':&#xA;#&amp;gt; &#xA;#&amp;gt;     slice&#xA;#&amp;gt; The following object is masked from 'package:IRanges':&#xA;#&amp;gt; &#xA;#&amp;gt;     slice&#xA;#&amp;gt; The following object is masked from 'package:S4Vectors':&#xA;#&amp;gt; &#xA;#&amp;gt;     rename&#xA;#&amp;gt; The following object is masked from 'package:stats':&#xA;#&amp;gt; &#xA;#&amp;gt;     filter&#xA;#&amp;gt; The following object is masked from 'package:graphics':&#xA;#&amp;gt; &#xA;#&amp;gt;     layout&#xA;#&amp;gt; Loading required package: DT&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'DT'&#xA;#&amp;gt; The following objects are masked from 'package:shiny':&#xA;#&amp;gt; &#xA;#&amp;gt;     dataTableOutput, renderDataTable&#xA;#&amp;gt; Loading required package: zeallot&#xA;#&amp;gt; Loading required package: excelR&#xA;#&amp;gt; Loading required package: shinycssloaders&#xA;#&amp;gt; Loading required package: ggdendro&#xA;#&amp;gt; Loading required package: shinyWidgets&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'shinyWidgets'&#xA;#&amp;gt; The following object is masked from 'package:shinyjs':&#xA;#&amp;gt; &#xA;#&amp;gt;     alert&#xA;#&amp;gt; Loading required package: openxlsx&#xA;#&amp;gt; Loading required package: tools&#xA;#&amp;gt; Loading required package: rmarkdown&#xA;#&amp;gt; Loading required package: knitr&#xA;#&amp;gt; Loading required package: seqinr&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'seqinr'&#xA;#&amp;gt; The following object is masked from 'package:shiny':&#xA;#&amp;gt; &#xA;#&amp;gt;     a&#xA;#&amp;gt; The following object is masked from 'package:sangerseqR':&#xA;#&amp;gt; &#xA;#&amp;gt;     read.abif&#xA;#&amp;gt; The following object is masked from 'package:Biostrings':&#xA;#&amp;gt; &#xA;#&amp;gt;     translate&#xA;#&amp;gt; The following objects are masked from 'package:ape':&#xA;#&amp;gt; &#xA;#&amp;gt;     as.alignment, consensus&#xA;#&amp;gt; Loading required package: BiocStyle&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'BiocStyle'&#xA;#&amp;gt; The following objects are masked from 'package:rmarkdown':&#xA;#&amp;gt; &#xA;#&amp;gt;     html_document, md_document, pdf_document&#xA;#&amp;gt; The following object is masked from 'package:shiny':&#xA;#&amp;gt; &#xA;#&amp;gt;     markdown&#xA;#&amp;gt; Loading required package: logger&#xA;#&amp;gt; Welcome to sangeranalyseR&#xA;data(&amp;quot;sangerAlignmentData&amp;quot;)&#xA;data(&amp;quot;sangerContigData&amp;quot;)&#xA;data(&amp;quot;sangerReadFData&amp;quot;)&#xA;&#xA;rawDataDir &amp;lt;- system.file(&amp;quot;extdata&amp;quot;, package = &amp;quot;sangeranalyseR&amp;quot;)&#xA;fastaFN &amp;lt;- file.path(rawDataDir, &amp;quot;fasta&amp;quot;, &amp;quot;SangerAlignment&amp;quot;, &amp;quot;Sanger_all_reads.fa&amp;quot;)&#xA;my_sangerAlignmentFa &amp;lt;- new(&amp;quot;SangerAlignment&amp;quot;,&#xA;                            inputSource          = &amp;quot;FASTA&amp;quot;,&#xA;                            processMethod        = &amp;quot;REGEX&amp;quot;,&#xA;                            FASTA_File           = fastaFN,&#xA;                            REGEX_SuffixForward  = &amp;quot;_[0-9]*_F&lt;span class=&quot;math-container&quot;&gt;$&quot;,&#xA;                        REGEX_SuffixReverse  = &quot;_[0-9]*_R$&lt;/span&gt;&amp;quot;,&#xA;                            refAminoAcidSeq      = &amp;quot;SRQWLFSTNHKDIGTLYFIFGAWAGMVGTSLSILIRAELGHPGALIGDDQIYNVIVTAHAFIMIFFMVMPIMIGGFGNWLVPLMLGAPDMAFPRMNNMSFWLLPPALSLLLVSSMVENGAGTGWTVYPPLSAGIAHGGASVDLAIFSLHLAGISSILGAVNFITTVINMRSTGISLDRMPLFVWSVVITALLLLLSLPVLAGAITMLLTDRNLNTSFFDPAGGGDPILYQHLFWFFGHPEVYILILPGFGMISHIISQESGKKETFGSLGMIYAMLAIGLLGFIVWAHHMFTVGMDVDTRAYFTSATMIIAVPTGIKIFSWLATLHGTQLSYSPAILWALGFVFLFTVGGLTGVVLANSSVDIILHDTYYVVAHFHYVLSMGAVFAIMAGFIHWYPLFTGLTLNNKWLKSHFIIMFIGVNLTFFPQHFLGLAGMPRRYSDYPDAYTTWNIVSTIGSTISLLGILFFFFIIWESLVSQRQVIYPIQLNSSIEWYQNTPPAEHSYSELPLLTN&amp;quot;,&#xA;                            minReadsNum           = 2,&#xA;                            minReadLength         = 20,&#xA;                            minFractionCall       = 0.5,&#xA;                            maxFractionLost       = 0.5,&#xA;                            geneticCode           = GENETIC_CODE,&#xA;                            acceptStopCodons      = TRUE,&#xA;                            readingFrame          = 1,&#xA;                            processorsNum         = 2)&#xA;#&amp;gt; Assessing frameshifts in nucleotide sequences:&#xA;#&amp;gt; ================================================================================&#xA;#&amp;gt; &#xA;#&amp;gt; Time difference of 0.05 secs&#xA;#&amp;gt; Assessing frameshifts in nucleotide sequences:&#xA;#&amp;gt; ================================================================================&#xA;#&amp;gt; &#xA;#&amp;gt; Time difference of 0.03 secs&#xA;#&amp;gt; Assessing frameshifts in nucleotide sequences:&#xA;#&amp;gt; ================================================================================&#xA;#&amp;gt; &#xA;#&amp;gt; Time difference of 0.03 secs&#xA;#&amp;gt; Assessing frameshifts in nucleotide sequences:&#xA;#&amp;gt; ================================================================================&#xA;#&amp;gt; &#xA;#&amp;gt; Time difference of 0.03 secs&#xA;&#xA;writeFasta(my_sangerAlignmentFa,&#xA;           outputDir         = tempdir(),&#xA;           compress          = FALSE,&#xA;           compression_level = NA,&#xA;           selection         = &amp;quot;all&amp;quot;)&#xA;&#xA;generateReport(my_sangerAlignmentFa,&#xA;               outputDir           = tempdir(),&#xA;               includeSangerRead   = TRUE,&#xA;               includeSangerContig = TRUE)&#xA;#&amp;gt; processing file: SangerRead_Report_fasta.Rmd&#xA;#&amp;gt; output file: SangerRead_Report_fasta.knit.md&#xA;#&amp;gt; /usr/local/bin/pandoc +RTS -K512m -RTS SangerRead_Report_fasta.knit.md --to html4 --from markdown+autolink_bare_uris+tex_math_single_backslash --output /private/var/folders/gf/3p_ynkts411bs238rtw3y0b40000gn/T/Rtmplj2Uy8/SangerAlignment/Achl_ACHLO006-09/Sanger_all_reads/SangerRead_Report_fasta.html --lua-filter /Library/Frameworks/R.framework/Versions/4.2/Resources/library/rmarkdown/rmarkdown/lua/pagebreak.lua --lua-filter /Library/Frameworks/R.framework/Versions/4.2/Resources/library/rmarkdown/rmarkdown/lua/latex-div.lua --embed-resources --standalone --variable bs3=TRUE --section-divs --table-of-contents --toc-depth 3 --variable toc_float=1 --variable toc_selectors=h1,h2,h3 --variable toc_collapsed=1 --variable toc_smooth_scroll=1 --variable toc_print=1 --template /Library/Frameworks/R.framework/Versions/4.2/Resources/library/rmarkdown/rmd/h/default.html --no-highlight --variable highlightjs=1 --variable theme=bootstrap --mathjax --variable 'mathjax-url=https://mathjax.rstudio.com/latest/MathJax.js?config=TeX-AMS-MML_HTMLorMML' --include-in-header /var/folders/gf/3p_ynkts411bs238rtw3y0b40000gn/T//Rtmplj2Uy8/rmarkdown-str3926187706b9.html&#xA;#&amp;gt; &#xA;#&amp;gt; Output created: /private/var/folders/gf/3p_ynkts411bs238rtw3y0b40000gn/T/RtmplI8nQX/SangerAlignment/SangerAlignment_Report.html&#xA;#&amp;gt; [1] &amp;quot;/private/var/folders/gf/3p_ynkts411bs238rtw3y0b40000gn/T/RtmplI8nQX/SangerAlignment/SangerAlignment_Report.html&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;sup&gt;Created on 2023-03-02 with &lt;a href=&quot;https://reprex.tidyverse.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;reprex v2.0.2&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Example from the docs:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(sangeranalyseR)&#xA;#&amp;gt; Loading required package: stringr&#xA;#&amp;gt; Loading required package: ape&#xA;#&amp;gt; Loading required package: Biostrings&#xA;#&amp;gt; Loading required package: BiocGenerics&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'BiocGenerics'&#xA;#&amp;gt; The following objects are masked from 'package:stats':&#xA;#&amp;gt; &#xA;#&amp;gt;     IQR, mad, sd, var, xtabs&#xA;#&amp;gt; The following objects are masked from 'package:base':&#xA;#&amp;gt; &#xA;#&amp;gt;     anyDuplicated, aperm, append, as.data.frame, basename, cbind,&#xA;#&amp;gt;     colnames, dirname, do.call, duplicated, eval, evalq, Filter, Find,&#xA;#&amp;gt;     get, grep, grepl, intersect, is.unsorted, lapply, Map, mapply,&#xA;#&amp;gt;     match, mget, order, paste, pmax, pmax.int, pmin, pmin.int,&#xA;#&amp;gt;     Position, rank, rbind, Reduce, rownames, sapply, setdiff, sort,&#xA;#&amp;gt;     table, tapply, union, unique, unsplit, which.max, which.min&#xA;#&amp;gt; Loading required package: S4Vectors&#xA;#&amp;gt; Loading required package: stats4&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'S4Vectors'&#xA;#&amp;gt; The following objects are masked from 'package:base':&#xA;#&amp;gt; &#xA;#&amp;gt;     expand.grid, I, unname&#xA;#&amp;gt; Loading required package: IRanges&#xA;#&amp;gt; Loading required package: XVector&#xA;#&amp;gt; Loading required package: GenomeInfoDb&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'Biostrings'&#xA;#&amp;gt; The following object is masked from 'package:ape':&#xA;#&amp;gt; &#xA;#&amp;gt;     complement&#xA;#&amp;gt; The following object is masked from 'package:base':&#xA;#&amp;gt; &#xA;#&amp;gt;     strsplit&#xA;#&amp;gt; Loading required package: DECIPHER&#xA;#&amp;gt; Loading required package: RSQLite&#xA;#&amp;gt; Loading required package: parallel&#xA;#&amp;gt; Loading required package: reshape2&#xA;#&amp;gt; Loading required package: phangorn&#xA;#&amp;gt; Loading required package: sangerseqR&#xA;#&amp;gt; Loading required package: gridExtra&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'gridExtra'&#xA;#&amp;gt; The following object is masked from 'package:BiocGenerics':&#xA;#&amp;gt; &#xA;#&amp;gt;     combine&#xA;#&amp;gt; Loading required package: shiny&#xA;#&amp;gt; Loading required package: shinydashboard&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'shinydashboard'&#xA;#&amp;gt; The following object is masked from 'package:graphics':&#xA;#&amp;gt; &#xA;#&amp;gt;     box&#xA;#&amp;gt; Loading required package: shinyjs&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'shinyjs'&#xA;#&amp;gt; The following object is masked from 'package:shiny':&#xA;#&amp;gt; &#xA;#&amp;gt;     runExample&#xA;#&amp;gt; The following object is masked from 'package:RSQLite':&#xA;#&amp;gt; &#xA;#&amp;gt;     show&#xA;#&amp;gt; The following object is masked from 'package:Biostrings':&#xA;#&amp;gt; &#xA;#&amp;gt;     show&#xA;#&amp;gt; The following object is masked from 'package:GenomeInfoDb':&#xA;#&amp;gt; &#xA;#&amp;gt;     show&#xA;#&amp;gt; The following object is masked from 'package:XVector':&#xA;#&amp;gt; &#xA;#&amp;gt;     show&#xA;#&amp;gt; The following object is masked from 'package:IRanges':&#xA;#&amp;gt; &#xA;#&amp;gt;     show&#xA;#&amp;gt; The following object is masked from 'package:S4Vectors':&#xA;#&amp;gt; &#xA;#&amp;gt;     show&#xA;#&amp;gt; The following object is masked from 'package:stats4':&#xA;#&amp;gt; &#xA;#&amp;gt;     show&#xA;#&amp;gt; The following objects are masked from 'package:methods':&#xA;#&amp;gt; &#xA;#&amp;gt;     removeClass, show&#xA;#&amp;gt; Loading required package: data.table&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'data.table'&#xA;#&amp;gt; The following objects are masked from 'package:reshape2':&#xA;#&amp;gt; &#xA;#&amp;gt;     dcast, melt&#xA;#&amp;gt; The following object is masked from 'package:IRanges':&#xA;#&amp;gt; &#xA;#&amp;gt;     shift&#xA;#&amp;gt; The following objects are masked from 'package:S4Vectors':&#xA;#&amp;gt; &#xA;#&amp;gt;     first, second&#xA;#&amp;gt; Loading required package: plotly&#xA;#&amp;gt; Loading required package: ggplot2&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'plotly'&#xA;#&amp;gt; The following object is masked from 'package:ggplot2':&#xA;#&amp;gt; &#xA;#&amp;gt;     last_plot&#xA;#&amp;gt; The following object is masked from 'package:phangorn':&#xA;#&amp;gt; &#xA;#&amp;gt;     add_boxplot&#xA;#&amp;gt; The following object is masked from 'package:XVector':&#xA;#&amp;gt; &#xA;#&amp;gt;     slice&#xA;#&amp;gt; The following object is masked from 'package:IRanges':&#xA;#&amp;gt; &#xA;#&amp;gt;     slice&#xA;#&amp;gt; The following object is masked from 'package:S4Vectors':&#xA;#&amp;gt; &#xA;#&amp;gt;     rename&#xA;#&amp;gt; The following object is masked from 'package:stats':&#xA;#&amp;gt; &#xA;#&amp;gt;     filter&#xA;#&amp;gt; The following object is masked from 'package:graphics':&#xA;#&amp;gt; &#xA;#&amp;gt;     layout&#xA;#&amp;gt; Loading required package: DT&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'DT'&#xA;#&amp;gt; The following objects are masked from 'package:shiny':&#xA;#&amp;gt; &#xA;#&amp;gt;     dataTableOutput, renderDataTable&#xA;#&amp;gt; Loading required package: zeallot&#xA;#&amp;gt; Loading required package: excelR&#xA;#&amp;gt; Loading required package: shinycssloaders&#xA;#&amp;gt; Loading required package: ggdendro&#xA;#&amp;gt; Loading required package: shinyWidgets&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'shinyWidgets'&#xA;#&amp;gt; The following object is masked from 'package:shinyjs':&#xA;#&amp;gt; &#xA;#&amp;gt;     alert&#xA;#&amp;gt; Loading required package: openxlsx&#xA;#&amp;gt; Loading required package: tools&#xA;#&amp;gt; Loading required package: rmarkdown&#xA;#&amp;gt; Loading required package: knitr&#xA;#&amp;gt; Loading required package: seqinr&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'seqinr'&#xA;#&amp;gt; The following object is masked from 'package:shiny':&#xA;#&amp;gt; &#xA;#&amp;gt;     a&#xA;#&amp;gt; The following object is masked from 'package:sangerseqR':&#xA;#&amp;gt; &#xA;#&amp;gt;     read.abif&#xA;#&amp;gt; The following object is masked from 'package:Biostrings':&#xA;#&amp;gt; &#xA;#&amp;gt;     translate&#xA;#&amp;gt; The following objects are masked from 'package:ape':&#xA;#&amp;gt; &#xA;#&amp;gt;     as.alignment, consensus&#xA;#&amp;gt; Loading required package: BiocStyle&#xA;#&amp;gt; &#xA;#&amp;gt; Attaching package: 'BiocStyle'&#xA;#&amp;gt; The following objects are masked from 'package:rmarkdown':&#xA;#&amp;gt; &#xA;#&amp;gt;     html_document, md_document, pdf_document&#xA;#&amp;gt; The following object is masked from 'package:shiny':&#xA;#&amp;gt; &#xA;#&amp;gt;     markdown&#xA;#&amp;gt; Loading required package: logger&#xA;#&amp;gt; Welcome to sangeranalyseR&#xA;rawDataDir &amp;lt;- system.file(&amp;quot;extdata&amp;quot;, package = &amp;quot;sangeranalyseR&amp;quot;)&#xA;parentDir &amp;lt;- file.path(rawDataDir, 'Allolobophora_chlorotica', 'ACHLO')&#xA;&#xA;ACHLO_contigs &amp;lt;- SangerAlignment(ABIF_Directory     = parentDir,&#xA;                                 REGEX_SuffixForward = &amp;quot;_[0-9]*_F.ab1&lt;span class=&quot;math-container&quot;&gt;$&quot;,&#xA;                             REGEX_SuffixReverse = &quot;_[0-9]*_R.ab1$&lt;/span&gt;&amp;quot;)&#xA;&#xA;launchApp(ACHLO_contigs)&#xA;writeFasta(ACHLO_contigs)&#xA;generateReport(ACHLO_contigs)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;sup&gt;Created on 2023-03-02 with &lt;a href=&quot;https://reprex.tidyverse.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;reprex v2.0.2&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;My sessionInfo output:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;sessionInfo()&#xA;#&amp;gt; R version 4.2.2 (2022-10-31)&#xA;#&amp;gt; Platform: x86_64-apple-darwin17.0 (64-bit)&#xA;#&amp;gt; Running under: macOS Big Sur ... 10.16&#xA;#&amp;gt; &#xA;#&amp;gt; Matrix products: default&#xA;#&amp;gt; BLAS:   /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRblas.0.dylib&#xA;#&amp;gt; LAPACK: /Library/Frameworks/R.framework/Versions/4.2/Resources/lib/libRlapack.dylib&#xA;#&amp;gt; &#xA;#&amp;gt; locale:&#xA;#&amp;gt; [1] en_AU.UTF-8/en_AU.UTF-8/en_AU.UTF-8/C/en_AU.UTF-8/en_AU.UTF-8&#xA;#&amp;gt; &#xA;#&amp;gt; attached base packages:&#xA;#&amp;gt;  [1] tools     parallel  stats4    stats     graphics  grDevices utils    &#xA;#&amp;gt;  [8] datasets  methods   base     &#xA;#&amp;gt; &#xA;#&amp;gt; other attached packages:&#xA;#&amp;gt;  [1] sangeranalyseR_1.8.0  logger_0.2.2          BiocStyle_2.26.0     &#xA;#&amp;gt;  [4] seqinr_4.2-23         knitr_1.42            rmarkdown_2.20       &#xA;#&amp;gt;  [7] openxlsx_4.2.5.2      shinyWidgets_0.7.6    ggdendro_0.1.23      &#xA;#&amp;gt; [10] shinycssloaders_1.0.0 excelR_0.4.0          zeallot_0.1.0        &#xA;#&amp;gt; [13] DT_0.27               plotly_4.10.1         ggplot2_3.4.1        &#xA;#&amp;gt; [16] data.table_1.14.8     shinyjs_2.1.0         shinydashboard_0.7.2 &#xA;#&amp;gt; [19] shiny_1.7.4           gridExtra_2.3         sangerseqR_1.34.0    &#xA;#&amp;gt; [22] phangorn_2.11.1       reshape2_1.4.4        DECIPHER_2.26.0      &#xA;#&amp;gt; [25] RSQLite_2.3.0         Biostrings_2.66.0     GenomeInfoDb_1.34.9  &#xA;#&amp;gt; [28] XVector_0.38.0        IRanges_2.32.0        S4Vectors_0.36.2     &#xA;#&amp;gt; [31] BiocGenerics_0.44.0   ape_5.7               stringr_1.5.0        &#xA;#&amp;gt; &#xA;#&amp;gt; loaded via a namespace (and not attached):&#xA;#&amp;gt;  [1] nlme_3.1-162           bitops_1.0-7           fs_1.6.1              &#xA;#&amp;gt;  [4] bit64_4.0.5            httr_1.4.5             R.cache_0.16.0        &#xA;#&amp;gt;  [7] bslib_0.4.2            utf8_1.2.3             R6_2.5.1              &#xA;#&amp;gt; [10] DBI_1.1.3              lazyeval_0.2.2         colorspace_2.1-0      &#xA;#&amp;gt; [13] ade4_1.7-22            withr_2.5.0            tidyselect_1.2.0      &#xA;#&amp;gt; [16] bit_4.0.5              compiler_4.2.2         cli_3.6.0             &#xA;#&amp;gt; [19] sass_0.4.5             scales_1.2.1           quadprog_1.5-8        &#xA;#&amp;gt; [22] digest_0.6.31          R.utils_2.12.2         pkgconfig_2.0.3       &#xA;#&amp;gt; [25] htmltools_0.5.4        styler_1.9.0           fastmap_1.1.1         &#xA;#&amp;gt; [28] htmlwidgets_1.6.1      rlang_1.0.6            rstudioapi_0.14       &#xA;#&amp;gt; [31] jquerylib_0.1.4        generics_0.1.3         jsonlite_1.8.4        &#xA;#&amp;gt; [34] zip_2.2.2              dplyr_1.1.0            R.oo_1.25.0           &#xA;#&amp;gt; [37] RCurl_1.98-1.10        magrittr_2.0.3         GenomeInfoDbData_1.2.9&#xA;#&amp;gt; [40] Matrix_1.5-3           Rcpp_1.0.10            munsell_0.5.0         &#xA;#&amp;gt; [43] fansi_1.0.4            lifecycle_1.0.3        R.methodsS3_1.8.2     &#xA;#&amp;gt; [46] stringi_1.7.12         yaml_2.3.7             MASS_7.3-58.2         &#xA;#&amp;gt; [49] zlibbioc_1.44.0        plyr_1.8.8             grid_4.2.2            &#xA;#&amp;gt; [52] blob_1.2.3             promises_1.2.0.1       crayon_1.5.2          &#xA;#&amp;gt; [55] lattice_0.20-45        pillar_1.8.1           igraph_1.4.1          &#xA;#&amp;gt; [58] codetools_0.2-19       fastmatch_1.1-3        reprex_2.0.2          &#xA;#&amp;gt; [61] glue_1.6.2             evaluate_0.20          BiocManager_1.30.20   &#xA;#&amp;gt; [64] vctrs_0.5.2            httpuv_1.6.9           gtable_0.3.1          &#xA;#&amp;gt; [67] purrr_1.0.1            tidyr_1.3.0            cachem_1.0.7          &#xA;#&amp;gt; [70] xfun_0.37              mime_0.12              xtable_1.8-4          &#xA;#&amp;gt; [73] later_1.3.0            viridisLite_0.4.1      tibble_3.1.8          &#xA;#&amp;gt; [76] memoise_2.0.1          ellipsis_0.3.2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9098" LastEditorUserId="9098" LastEditDate="2023-03-03T04:34:48.757" LastActivityDate="2023-03-03T04:34:48.757" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20648" PostTypeId="1" CreationDate="2023-03-02T08:18:20.140" Score="0" ViewCount="37" Body="&lt;p&gt;What software do you use to design LAMP reaction primers for alignment?  So far I have been using Primer Explorer v5 but I am looking for some alternatives.&lt;/p&gt;&#xA;&lt;p&gt;Alignment includes some species from the same virus family. So I am looking for a set of primers that will capture species from the given alignment.&lt;/p&gt;&#xA;" OwnerUserId="17099" LastEditorUserId="73" LastEditDate="2023-03-03T18:45:59.337" LastActivityDate="2023-03-03T18:45:59.337" Title="LAMP primers design software" Tags="|primer|design|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20649" PostTypeId="2" ParentId="20640" CreationDate="2023-03-02T12:02:36.867" Score="1" Body="&lt;p&gt;Simple answer is, it might not be possible to do it.&lt;/p&gt;&#xA;&lt;p&gt;The large number of contigs suggests to me that a short-read sequencing method (such as Illumina sequencing) was used. Alone, short-read sequencing technologies cannot resolve repetitive sequences that are longer than the length of the reads. This is why you have a large number of contigs.*&lt;/p&gt;&#xA;&lt;p&gt;*You might already have done this, but if you did not, it might be worth checking that your MAG is not actually a mixture of different genomes. This can also somewhat increase the contig number and, more importantly, can skew your interpretation if you assume it's only one organism. There are various ways to test this, but the simplest I know is the program CheckM. To run it you will need a way to use Unix/Linux (could be on a virtual machine). If you are not familiar with Unix, bash and conda, I would heartily suggest software carpentry lessons such as The Unix Shell (&lt;a href=&quot;https://swcarpentry.github.io/shell-novice/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://swcarpentry.github.io/shell-novice/&lt;/a&gt;) and possibly this conda lesson although it is in beta and I haven't tested it &lt;a href=&quot;https://carpentries-incubator.github.io/introduction-to-conda-for-data-scientists/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://carpentries-incubator.github.io/introduction-to-conda-for-data-scientists/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm not familiar with the term 'molecular typing', but it looks like you're interested in what organisms / genes there are in your sample? You probably might not need a full genome for that.&lt;/p&gt;&#xA;&lt;p&gt;Small probability of helping - but it might also be good to have a look at the sizes of your contigs. You might luck out and have a few large contigs (hundred thousand - million basepairs) which could be compared to existing database genomes.&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2023-03-02T12:02:36.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20650" PostTypeId="2" ParentId="20628" CreationDate="2023-03-02T12:46:03.480" Score="1" Body="&lt;p&gt;User-friendly alternative:&lt;/p&gt;&#xA;&lt;p&gt;Get the alignment into Jalview, set a colour scheme you like, and inspect alignment visually. You can move sequences up and down in the alignment with keyboard arrows, so you can focus on what you need.&lt;/p&gt;&#xA;&lt;p&gt;Online, no-install-needed alternative:&#xA;(same program as M_'s method)&#xA;blastn on NCBI website. Tick 'align two or more sequences'. Query in the top box, all the others in the bottom box.&#xA;(Would not use this method if you had&amp;gt; 20 seqences).&lt;/p&gt;&#xA;&lt;p&gt;If you have many sequences, I reccomend going with M_ 's advice instead.&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2023-03-02T12:46:03.480" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20651" PostTypeId="1" AcceptedAnswerId="20652" CreationDate="2023-03-02T14:07:46.603" Score="3" ViewCount="280" Body="&lt;p&gt;I was trying to left normalize my vcf file using following command:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;bcftools norm -f reference.fasta -m -any input.vcf &amp;gt; output.norm.vcf&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;However this gives the following error:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Non-ACGTN alternate allele at 1:278165 .. VCF:'-3AGG'&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;It looks like the variant caller returned all deletions and insertions like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1   5387285 .   C   +2TG&#xA;1   5387316 .   G   -8TGTGTATC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This file format was produced by SCCaller 2.0.0, there are several unconventional output fields produced by this caller but but the repository isn't maintained.&lt;/p&gt;&#xA;&lt;p&gt;Is this an issue of the vcf format or does this means that there is a non-ACGTN character on position 1:278165 in the fasta reference file? I don't see anyting out of the ordinary at this position using IGV.&lt;/p&gt;&#xA;&lt;p&gt;What would be the best way forward?&#xA;I tried the following command to extract non-ACGTN allels but it outputs all alleles:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;bcftools view -i 'ALT !~ &amp;quot;^[ACGTN]+$&amp;quot;' input.vcf.gz &amp;gt; errors.vcf&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;EDIT: including several lines of vcf file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CHROM   POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  SAMPLE1&#xA;1   89946   .   A   T   68  .   NS=1    GT:SO:AD:BI:GQ:PL   0/1:False:3,18:0.973:68:273,164,49,117&#xA;1   286158  .   A   +2TG    78  .   NS=1    GT:SO:AD:BI:GQ:PL   0/0:NA:11,10:0.982:78:154,97,175,456&#xA;1   900119  .   A   G   150 .   NS=1    GT:SO:AD:BI:GQ:PL   0/0:False:17,14:0.995:150:214,136,321,657&#xA;1   932613  .   G   -4TTTC  62  .   NS=1    GT:SO:AD:BI:GQ:PL   0/1:NA:4,18:0.998:62:274,165,103,173&#xA;1   1168310 .   T   +2CA    35  .   NS=1    GT:SO:AD:BI:GQ:PL   0/0:NA:18,5:0.985:35:77,55,91,712```&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16874" LastEditorUserId="73" LastEditDate="2023-03-03T18:44:36.607" LastActivityDate="2023-03-03T20:30:05.513" Title="Non-ACGTN alternate alleles in VCF" Tags="|vcf|variant-calling|bcftools|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20652" PostTypeId="2" ParentId="20651" CreationDate="2023-03-02T14:22:12.917" Score="1" Body="&lt;p&gt;That isn't a valid ALT field according to the &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VCF standard&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;ALT — alternate base(s): Comma-separated list of alternate non-reference alleles. These alleles do not have&#xA;to be called in any of the samples. Options are base Strings made up of the bases A,C,G,T,N (case insensitive)&#xA;or the ‘*’ symbol (allele missing due to overlapping deletion) or a MISSING value ‘.’ (no variant) or an&#xA;angle-bracketed ID String (“”) or a breakend replacement string as described in Section 5.4. If there&#xA;are no alternative alleles, then the MISSING value must be used. In other words, the ALT field must be a&#xA;symbolic allele, or a breakend replacement string, or match the regular expression ^([ACGTNacgtn]+|*|.)$.&#xA;Tools processing VCF files are not required to preserve case in the allele String, except for IDs, which are case&#xA;sensitive. (String; no whitespace, commas, or angle-brackets are permitted in the ID String itself)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So yes, you can have things other than ACTG, &lt;code&gt;N&lt;/code&gt; most commonly or &lt;code&gt;.&lt;/code&gt; or &lt;code&gt;*&lt;/code&gt;. And you can also have things like &lt;code&gt;&amp;lt;DEL&amp;gt;&lt;/code&gt; and &lt;code&gt;&amp;lt;DUP&amp;gt;&lt;/code&gt; for structural variants. But a &lt;code&gt;+&lt;/code&gt; or a digit make no sense and are not allowed so this is not a valid VCF file.&lt;/p&gt;&#xA;&lt;p&gt;So, filter them out. You can use something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zcat input.vcf.gz | awk -F'\t' '/^#/ || $5 ~ /^[ACGTNacgtn]+|\*|\.)$/' |&#xA;    bgzip &amp;gt; fixed.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would also report this as a bug to the developers of your variant caller.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-03-02T14:22:12.917" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20653" PostTypeId="1" CreationDate="2023-03-02T16:12:14.727" Score="1" ViewCount="55" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; eggNOG uses seed orthologues which correspond to its own internal database, so for example &lt;code&gt;J421_1876&lt;/code&gt; can be placed in its online search engine &lt;a href=&quot;http://eggnog5.embl.de/#/app/home&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. This will recover the function and species classification. Brief output&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Carbohydrate transport and metabolism gluconolactonase activity&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;.. and lots of associated species. Thats useful information but not very amenable to coded solutions.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Aim&lt;/strong&gt; I want to replicate this via its API, i.e. to be placed in a pipeline.&lt;/p&gt;&#xA;&lt;p&gt;Thus eggNOG API is &lt;a href=&quot;http://eggnog5.embl.de/#/app/api&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; - but I can't figure out, here's the instructions&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://eggnogapi5.embl.de/nog_data/%5Bdataformat%5D/%5Battributes%5D/%5Bnogname%5D&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://eggnogapi5.embl.de/nog_data/[dataformat]/[attributes]/[nogname]&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; Does anyone know how to use the API to interrogate using the &lt;code&gt;seed orthologue&lt;/code&gt; code?&lt;/p&gt;&#xA;&lt;p&gt;There are alternatives which involve LOADS of hard drive space (not very attractive).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-03-02T17:11:09.620" LastActivityDate="2023-03-02T17:11:09.620" Title="eggNOG API, how's it work?" Tags="|phylogenetics|bacteria|api|taxonomy|eggnog|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20654" PostTypeId="2" ParentId="20502" CreationDate="2023-03-02T17:06:59.687" Score="1" Body="&lt;p&gt;I can answer this now. I've been 'under the hood' of eggNOG and its using &amp;quot;hmm&amp;quot; (hidden Markov model) very likely, almost certain, the Eddie (sp?) group's &lt;code&gt;hmmer&lt;/code&gt; to cluster proteins associated. It represents the cluster members strictly via seed orthologue code, which then are aligned to form a 'raw' and 'trimmed' alignment. Each alignment is assigned a code which corresponds to its know function. This is a database of alignments. Once it gets a protein - protein hit against one of its orthologue (I suspect it will be using a nested system) it will then assign the query to that alignment - each of which is assigned to a protein function.&lt;/p&gt;&#xA;&lt;p&gt;It representative database not an all encompassing database, i.e. sequences are being omitted.&lt;/p&gt;&#xA;&lt;p&gt;Taxonomically it is simply declaring a the familial name. There will be an internal database mapping the &lt;code&gt;seed orthologue&lt;/code&gt; onto the species, but that is not being publicly disclosed.&lt;/p&gt;&#xA;&lt;p&gt;I make a better explanation than that if anyone wants it.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;There is some description in the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2238944/&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-03-02T17:06:59.687" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20655" PostTypeId="1" CreationDate="2023-03-02T18:42:14.227" Score="2" ViewCount="88" Body="&lt;p&gt;I have phyloseq object where sequence table were builded by using the function mergeSequenceTables from DADA2 package, to merge ten sequence tables from different runs.&#xA;The sample_data consist of two columns 1)containing sample names same as sequence table column names and 2)study name from which samples comes from.&lt;/p&gt;&#xA;&lt;p&gt;Using this information I want to subset sequences that have  average relative abundance &amp;gt; 0.1% at least in 3 studies.&lt;/p&gt;&#xA;&lt;p&gt;I tried this but it doesn't consider information about studies.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;prevalenceDF = data.frame(Prevalence = colSums(otu_table(ps.trans) &amp;gt; 0),&#xA;                      TotalAbundance = colSums(otu_table(ps.trans)),&#xA;                      tax_table(ps.trans)&#xA;)&#xA;&#xA;summary_prevalence &amp;lt;- plyr::ddply(prevalenceDF, &amp;quot;Genus&amp;quot;, function(df){ data.frame(mean_prevalence = mean(df&lt;span class=&quot;math-container&quot;&gt;$Prevalence),&#xA;             totalAbundance = sum(df$&lt;/span&gt;TotalAbundance, na.rm = T), stringsAsFactors = F) })&#xA;&#xA;&#xA;keepPhyla &amp;lt;- summary_prevalence&lt;span class=&quot;math-container&quot;&gt;$Phylum[summary_prevalence$&lt;/span&gt;totalAbundance/sum(summary_prevalence$totalAbundance)&#xA;&amp;gt;= 0.001] &#xA;ps.1 = subset_taxa(ps, Phylum %in% keepPhyla)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does anyone have any pointers haw can I count how many studies containing each genera and add as column to prevalenceDF data frame or other way to do filtering?&lt;/p&gt;&#xA;" OwnerUserId="17103" LastEditorUserId="17103" LastEditDate="2023-03-03T16:54:24.640" LastActivityDate="2023-03-03T16:54:24.640" Title="Filter phyloseq sequence table" Tags="|r|phyloseq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20656" PostTypeId="1" AcceptedAnswerId="20660" CreationDate="2023-03-02T22:15:16.217" Score="1" ViewCount="52" Body="&lt;p&gt;Could someone help me understand what the difference between transcript and primary transcript on phytozme is? For example, this dataset of &lt;a href=&quot;https://data.jgi.doe.gov/search?q=Arabidopsis+thaliana&amp;amp;superseded=Current&amp;amp;expanded=Phytozome-447&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;A.thaliana&lt;/em&gt;&lt;/a&gt; has &amp;quot;primary transcript CDS&amp;quot; vs CDS.&lt;/p&gt;&#xA;&lt;p&gt;Off the top of my head primary transcript represents the initial strand of RNA made from DNA and transcripts represent genes post-processing of the initial strand.&lt;/p&gt;&#xA;&lt;p&gt;The thing that is confusing me is, why is the transcript file larger than the primary transcript file? The transcript file has about 21k more headers than the &amp;quot;primary transcript&amp;quot;. The only explanation I can think of is that due to alternate splicing you could end up with a bunch of isoforms that lend more headers to the transcript file. Is that it?&lt;/p&gt;&#xA;" OwnerUserId="16699" LastActivityDate="2023-03-03T19:32:32.040" Title="Transcript vs Primary transcript on phytozome" Tags="|genomics|public-databases|transcriptome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20657" PostTypeId="1" CreationDate="2023-03-02T20:50:12.150" Score="3" ViewCount="25" Body="&lt;p&gt;For a project I'm working on, I need to figure out how to model noise that may be happening in real genomes due to alignment errors, contamination, etc. Specifically, short-read paired-end data either the fastqs or bams. The reason I want to do so is to distinguish this noise from signals of actual subclonal variants.&lt;/p&gt;&#xA;&lt;p&gt;When I say noise, I mean things like discordant pairs with abnormal orientation happening spuriously across the genome. This noise might be happening due to alignment error or something else, I'm not entirely sure yet.&lt;/p&gt;&#xA;&lt;p&gt;I've simulated data with SVEngine but the amount of noise (discordant pairs with 1 kb+ insert sizes) that don't overlap with simulated variants is minimal. A total of 11 pairs.&lt;/p&gt;&#xA;&lt;p&gt;Is there a tool or method I can use to add this noise, specifically discordant pair noise? I don't want to add subclonal variants with a low VAF as a replacement for noise, rather add something truly spurious.&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerDisplayName="Suraj Rajendran" LastActivityDate="2023-03-03T03:43:46.820" Title="Add or Simulate Noise in Short-Read Paired End Data" Tags="|sequence-alignment|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20658" PostTypeId="2" ParentId="5600" CreationDate="2023-03-03T14:22:47.180" Score="2" Body="&lt;p&gt;&lt;code&gt;samtools&lt;/code&gt; can do this natively too using &lt;code&gt;-N, --qname-file&lt;/code&gt; CLI option:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;samtools view -N read_names.txt in.bam &amp;gt; read_locs.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;❯ samtools --version&#xA;samtools 1.16.1&#xA;Using htslib 1.16&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15257" LastActivityDate="2023-03-03T14:22:47.180" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20659" PostTypeId="2" ParentId="20641" CreationDate="2023-03-03T19:24:15.723" Score="1" Body="&lt;p&gt;You can do that with GROMACS. You probably need to check the manual for this, but roughly these are the steps:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gmx pdb2gmx -f input.pdb -o output.gro -water spc&#xA;gmx solvate -cp output.gro -cs spc216.gro -o output_solv.gro -p output.top&#xA;gmx pdb2gmx -f output_solv.gro -o output_processed.gro -p output.top -ignh&#xA;gmx make_ndx -f output_processed.gro -o index.ndx&#xA;gmx hbond -f output_processed.gro -n index.ndx -s output.tpr -num hbond.xvg -hbn hbond.ndx&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;First, you convert the PDB file into a GROMACS topology file using the pdb2gmx command. Then you can solvate the system with water molecules and add hydrogen atoms. Then you cCreate an index file containing the groups of atoms you want to analyze. Then you can use the hbond command to calculate the hydrogen bonds.&lt;/p&gt;&#xA;&lt;p&gt;I have not used GROMACS in a while. Hope that provides a starting point. With GROMACS you can even run a simulation and monitor the h-bonds over time.&lt;/p&gt;&#xA;&lt;p&gt;You actually might need to run an energy minimziation first, as the PDB might not in an optimal state for the h-bonds detection.&lt;/p&gt;&#xA;" OwnerUserId="3061" LastEditorUserId="3061" LastEditDate="2023-03-03T19:26:42.993" LastActivityDate="2023-03-03T19:26:42.993" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20660" PostTypeId="2" ParentId="20656" CreationDate="2023-03-03T19:32:32.040" Score="1" Body="&lt;p&gt;According to a team lead at Phytozome:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Multi-exon genes can produce different transcripts via process of alternative splicing:  inclusion/exclusion of particular exons in the spliced transcript.Designation of one of these transcripts as “primary” will vary based on the genome under consideration:  the primary might be the one with the highest expression under “normal” conditions, it might be the one whose mutations was first studied, or it might simply be the transcript with the longest CDS.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In all cases, Number_of_transcripts &amp;gt;= Number_of_primary_transcripts                    Number_of_primary_transcripts == Number_of_genes&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="16699" LastActivityDate="2023-03-03T19:32:32.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20661" PostTypeId="2" ParentId="20651" CreationDate="2023-03-03T20:29:22.357" Score="0" Body="&lt;p&gt;I would personally use plink2 for this. Nice and simple and unerstandable by all.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plink2 --vcf in.vcf --snps-only just-acgt --export vcf --out out.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="3694" LastEditDate="2023-03-03T20:30:05.513" LastActivityDate="2023-03-03T20:30:05.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20663" PostTypeId="1" CreationDate="2023-03-05T09:19:49.573" Score="2" ViewCount="104" Body="&lt;p&gt;I am dealing with a transcriptome of a species and list of genes selected for a gene enrichment analysis. In order to see what kind of genes in my selected list are overrepresented compared with the transcriptome, I ran eggnog-mapper annotation.&lt;/p&gt;&#xA;&lt;p&gt;After running eggnog mapper, I extract the list of GO, KEGG and COG terms associated.&lt;/p&gt;&#xA;&lt;p&gt;From there, I would like to do a Fisher-test to highlight COG categories overrepresented in our querry selected list compared to our reference transcriptome.&lt;/p&gt;&#xA;&lt;p&gt;My questions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How do you deal with enrichment, this kind of test for COG categories?&lt;/li&gt;&#xA;&lt;li&gt;For the eggnog results KEGG, EC ...ETC, for example how to assign each KO number to metabolism, • genetic information processing • environmental information processing • cellular processes • organismal systems • human diseases • Drug development&#xA;and draw graphs?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17119" LastActivityDate="2023-03-05T09:19:49.573" Title="COG Enrichment, the overrepresentation test between annotated transcriptome and a set of annotated gene, eggnog-mapper annotation results" Tags="|go-enrichment|kegg|eggnog|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20664" PostTypeId="1" AcceptedAnswerId="20667" CreationDate="2023-03-06T11:17:10.817" Score="2" ViewCount="37" Body="&lt;p&gt;I am building primers for LAMP which I know will not match all my target sequences perfectly.&lt;/p&gt;&#xA;&lt;p&gt;To some extent, having primers with multiple variants at variable sites (degenerate primers) and longer sequences (tagged degenerate primers) will help. However, I can't just increase degeneracy (variability) forever because then I will have no amplification efficiency.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, especially at the 5' end which is less important from an amplification perspective, I will have some calculated mismatches. So I have this puzzle:&lt;/p&gt;&#xA;&lt;p&gt;If those positions of interest (closer to the 5' end) are likely to be any base with about the same frequency, which is either a) the base or b) the pair of bases, which are likely to clash the least?&lt;/p&gt;&#xA;&lt;p&gt;My instinctive reaction is that for a single base, a pyrimidine would fit better, because even if it doesn't match, it's smaller? And for a pair of bases, probably a purine and a pyrimidine? I would appreciate your thoughts!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastEditorUserId="73" LastEditDate="2023-03-07T18:29:08.340" LastActivityDate="2023-03-07T18:29:08.340" Title="Which non-matching DNA bases clash the least?" Tags="|sequence-analysis|primer|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20665" PostTypeId="2" ParentId="20564" CreationDate="2023-03-06T13:22:16.460" Score="1" Body="&lt;p&gt;The following package performs this type of analysis and can be directly used on a Seurat object:&lt;/p&gt;&#xA;&lt;p&gt;paper: &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/38/20/4720/6675456&quot; rel=&quot;nofollow noreferrer&quot;&gt;propeller: testing for differences in cell type proportions in single cell data&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;github: &lt;a href=&quot;https://github.com/phipsonlab/speckle&quot; rel=&quot;nofollow noreferrer&quot;&gt;Speckle&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You mention that you would like to do some statistics so keep in mind that this requires replicated measurements to handle the large variability in proportions between samples. Your metadata should contain:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;celltype, sample, group_to_test&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;once the package is installed you can simply run the propeller function and set group to technology (in your post you mention you want to evaluate 3' vs multi-omics):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;propel &amp;lt;- propeller(clusters = object@meta.data$celltype, &#xA;                    sample = object@meta.data$sample, &#xA;                    group = object@meta.data$technology)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will return the statistics: t-test for two groups or ANOVA for multiple testing.&lt;/p&gt;&#xA;&lt;p&gt;The following function will return the transformed proportions that can then be used in limma for example to test for more complex designs or simple sample-wise plotting using ggplot. For &lt;code&gt;transform&lt;/code&gt; you will offcourse have to decide which transformation is applicable to your data.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;getTransformedProps(clusters = object@meta.data$celltype, &#xA;                             sample = object@meta.data$sample, transform=&amp;quot;asin&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There are a variety of other packages that allow for more complex analysis and a recent benchmark can be found &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2023.02.24.529894v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16874" LastActivityDate="2023-03-06T13:22:16.460" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20666" PostTypeId="1" CreationDate="2023-03-06T14:38:17.677" Score="1" ViewCount="73" Body="&lt;p&gt;I ran cellranger multi on a TotalSeq B dataset and got this error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[error] Pipestance failed. Error log at:&#xA;sampleUnderscore2/SC_MULTI_CS/SC_MULTI_CORE/MULTI_CHEMISTRY_DETECTOR/_GEM_WELL_CHEMISTRY_DETECTOR/DETECT_COUNT_CHEMISTRY/fork0/chnk0-u0e6a05f224/_errors&#xA;&#xA;Log message:&#xA;There were not enough reads to auto detect the chemistry: Sample CITE_2 in &amp;quot;/scratch/palmer/E222532_PR_Run01_051222/outs/fastq_path/AAAVFT3HV&amp;quot;&#xA;Note that you can avoid auto detection by specifying the specific chemistry type and version.&#xA;- Minimum number of required reads = 10000&#xA;- Number of reads available = 1408&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm not sure what to specify the chemistry as. Does anyone know what it might be for CITEseq as part of TotalSeq B? Alternatively, can I reduce the minimum number of required reads?&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="17124" LastEditorUserId="17124" LastEditDate="2023-03-06T14:38:55.417" LastActivityDate="2023-03-06T20:48:11.720" Title="In cellranger, can I change the minimum number of reads for autodetecting chemistry?" Tags="|scrnaseq|cellranger|10x|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20667" PostTypeId="2" ParentId="20664" CreationDate="2023-03-06T15:41:15.340" Score="2" Body="&lt;p&gt;I would just use Primer(3)-Blast here:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/tools/primer-blast/index.cgi?GROUP_TARGET=on&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/tools/primer-blast/index.cgi?GROUP_TARGET=on&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would avoid the approach you outline because:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Primer design algorithms incorporating degeneracy are very good;&lt;/li&gt;&#xA;&lt;li&gt;I'm not sure you're right (but could be wrong).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The algorithm behind Primer-Blast is Primer3, and is legendary in the primer design business and of course Blast.&lt;/p&gt;&#xA;&lt;p&gt;I agree that strong conservation at 3' is key to the design, but based on homology rather than chemistry.&lt;/p&gt;&#xA;&lt;p&gt;The thermodynamic calculation ... I would suggest that a mismatch with AT is going to less severe than GC. GC is a stronger bond, but I'd let the algorithm sort this out and prioritise using personal judgement:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Conservation by homology at the 3' end.&lt;/li&gt;&#xA;&lt;li&gt;Conservation by homology for GC regions.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Ultimately, primers are cheap so ordering loads of algorithmic designs is usually the best approach.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Just to add an explanation on the subject of why is this bioinformatics?&lt;/p&gt;&#xA;&lt;p&gt;One of the issues of primer selection is self-hybridisation within the primer or annealing between the primer pair. This is bad news for a primer - because it might not work.&lt;/p&gt;&#xA;&lt;p&gt;Algorithm primer design programs are good at spotting and avoiding unwanted hybridisation. When degeneracy is involved this becomes complicated.&lt;/p&gt;&#xA;&lt;p&gt;What Primer-Blast does is assess all combinations of degeneracy against issues such as self-hybridisation and primer-annealing. Humanely it is not possible to assess all those combinations and that is why an algorithm is needed.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-03-07T00:00:59.977" LastActivityDate="2023-03-07T00:00:59.977" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20668" PostTypeId="1" AcceptedAnswerId="20707" CreationDate="2023-03-06T15:53:15.110" Score="3" ViewCount="993" Body="&lt;p&gt;Hi I am trying to get the cancer ontologies (obo_id and label) from EBI-OLS. Earlier I have used the below code to get the &lt;code&gt;obo_id&lt;/code&gt; terms and &lt;code&gt;labels&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;descendants = requests.get(&#xA;    &amp;quot;https://www.ebi.ac.uk/ols/api/ontologies/efo/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FMONDO_0004992/descendants&amp;quot;&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As there are more than 1 page so I created &lt;code&gt;total_pages&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;total_pages = descendants.json()[&amp;quot;page&amp;quot;][&amp;quot;totalElements&amp;quot;] // 1000 + 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;cancer_id = []&#xA;&#xA;for i in range(total_pages):&#xA;    ## Check the number of id's present in 'totalElements'&#xA;    des = {&amp;quot;page&amp;quot;: i, &amp;quot;size&amp;quot;: descendants.json()[&amp;quot;page&amp;quot;][&amp;quot;totalElements&amp;quot;]}&#xA;    ## Add the number of 'totalElements' and 'pages' in the new python request for descendants&#xA;    cancer_efo = requests.get(&#xA;        &amp;quot;https://www.ebi.ac.uk/ols/api/ontologies/efo/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FMONDO_0004992/descendants&amp;quot;,&#xA;        params=des,&#xA;    )&#xA;    ## Get the number of obo_id's present in the cancer_efo.json&#xA;    obo = [obj[&amp;quot;obo_id&amp;quot;] for obj in cancer_efo.json()[&amp;quot;_embedded&amp;quot;][&amp;quot;terms&amp;quot;]]&#xA;    ## Get the number of cancer labels present in the cancer_efo.json&#xA;    label = [obj[&amp;quot;label&amp;quot;] for obj in cancer_efo.json()[&amp;quot;_embedded&amp;quot;][&amp;quot;terms&amp;quot;]]&#xA;    data = zip(obo, label)&#xA;    cancer_id.extend(data)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above code worked pretty well with one link&#xA;Now the issue is that I want to get ontologies for 37 descendent links and maximum links have more than 1 page. I want to get one file (cancer_id) containing all obo_id's and labels from these all 37 links.&lt;/p&gt;&#xA;&lt;p&gt;I made a separate file called &lt;code&gt;list.txt&lt;/code&gt; (Right now I am copying only 3 here)&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;list.txt&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;https://www.ebi.ac.uk/ols/api/ontologies/ncit/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FNCIT_C4861/descendants&#xA;https://www.ebi.ac.uk/ols/api/ontologies/ncit/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FNCIT_C3171/descendants&#xA;https://www.ebi.ac.uk/ols/api/ontologies/ncit/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FNCIT_C3457/descendants&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried the below code but it is not working&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;&#xA;with open ('list.txt', 'r') as fin:&#xA;    links = fin.readlines()&#xA;&#xA;def requestIt(links): &#xA;    cancer_all = requests.get(links)&#xA;    return cancer_all.json() &#xA;&#xA;data = list(map(requestIt, links))&#xA;&#xA;obo = obj[&amp;quot;obo_id&amp;quot;] for obj in data&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The error is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;obo = obj[&amp;quot;obo_id&amp;quot;] for obj in data&#xA;                        ^&#xA;SyntaxError: invalid syntax&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4783" LastEditorUserId="73" LastEditDate="2023-03-07T18:24:46.023" LastActivityDate="2023-03-11T19:41:02.550" Title="How to run python request for list of url's with multiple page numbers?" Tags="|python|api|pandas|ontology|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20669" PostTypeId="1" AcceptedAnswerId="20671" CreationDate="2023-03-06T18:02:07.257" Score="1" ViewCount="35" Body="&lt;p&gt;I was trying to calculate the frequency of disease_present (&lt;code&gt;yes&lt;/code&gt;) when smoking status is &lt;code&gt;y&lt;/code&gt; (yes) for each group (A, B, C, D)&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt; The formula is = Number of samples with disease_present (yes) / No of samples with smoking status 'y' * 100&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;df&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Sample_code Group   smoking disease_present&#xA;1   A   y   yes&#xA;2   B   n   &#xA;3   C   n   yes&#xA;4   D   y   &#xA;5   B   n   yes&#xA;6   A   y   yes&#xA;7   C   n   &#xA;8   D   y   &#xA;9   A   n   yes&#xA;10  B   y   &#xA;11  D   n   yes&#xA;12  C   n   &#xA;13  A   y   yes&#xA;14  B   y   yes&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df.groupby(&amp;quot;Group&amp;quot;).count()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This gave the same number in every category&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Pandas by default turns empty cells into NaN&lt;/li&gt;&#xA;&lt;li&gt;I tried to replace NaN with empty strings with &lt;code&gt;df1 = df.fillna(&amp;quot;&amp;quot;)&lt;/code&gt;. I still got the same results&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="10684" LastActivityDate="2023-03-06T18:27:45.270" Title="How to calculate frequency of a category with respect to the value in another column?" Tags="|python|pandas|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20670" PostTypeId="2" ParentId="20643" CreationDate="2023-03-06T18:27:00.960" Score="0" Body="&lt;p&gt;Spatial species distribution always follow a negative binomial distribution at any point of &lt;em&gt;most&lt;/em&gt; other factors e.g. climate. When a species enters a human monoculture type environment - then deviation from the negative binomial can occur within that monoculture. The clumping will be approximated via trigonometry: 'cause the clumping can be approximated via a circle, very clumped - small circle, more dispersed bigger circle.&lt;/p&gt;&#xA;&lt;p&gt;The parameters of the negative binomial are likely to change in relationship to climate and thats what I'd model if I was doing this work.  The 'clumping' (which is what the negative binomial models) is likely to tightened as global warming increases. In practical terms the snails start drying out so they have to clump in the remaining pockets of moisture (this might be more true of slugs than snails ... at a guess). Alot of invertebrates that cannot or don't really fly - are prone to desiccation.&lt;/p&gt;&#xA;&lt;p&gt;Personally, I go to empirical stats rather than use a package. However, the packages you are using will follow these two principles. So you can do it from first principles. Its fairly simple ecology its not like phylogenetics that have been through decades of development.&lt;/p&gt;&#xA;&lt;p&gt;If you've DNA from the snails then the phylogenetic Bayesian spatial models around Beast provide a powerful basis for the investigation. That would be very strongly recommended if you've DNA sequence from a good proportion of the snails.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-03-06T18:27:00.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20671" PostTypeId="2" ParentId="20669" CreationDate="2023-03-06T18:27:45.270" Score="2" Body="&lt;p&gt;I am not able to reproduce the issue.&lt;/p&gt;&#xA;&lt;p&gt;However I think this is a &lt;a href=&quot;https://pandas.pydata.org/docs/reference/api/pandas.crosstab.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;crosstab&lt;/a&gt; problem:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; new_df = df.fillna(&amp;quot;no&amp;quot;)&#xA;&amp;gt;&amp;gt;&amp;gt; new_df&#xA;# prints:&#xA;    Sample_code Group smoking disease_present&#xA;0             1     A       y             yes&#xA;1             2     B       n              no&#xA;2             3     C       n             yes&#xA;3             4     D       y              no&#xA;4             5     B       n             yes&#xA;5             6     A       y             yes&#xA;6             7     C       n              no&#xA;7             8     D       y              no&#xA;8             9     A       n             yes&#xA;9            10     B       y              no&#xA;10           11     D       n             yes&#xA;11           12     C       n              no&#xA;12           13     A       y             yes&#xA;13           14     B       y             yes&#xA;&#xA;&amp;gt;&amp;gt;&amp;gt; pd.crosstab([new_df.disease_present, new_df.smoking], new_df.Group)&#xA;# prints&#xA;Group                    A  B  C  D&#xA;disease_present smoking            &#xA;no              n        0  1  2  0&#xA;                y        0  1  0  2&#xA;yes             n        1  1  1  1&#xA;                y        3  1  0  0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-03-06T18:27:45.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20673" PostTypeId="2" ParentId="20668" CreationDate="2023-03-06T19:04:56.693" Score="1" Body="&lt;p&gt;@Megha, I have not looked through the code in too much detail, however.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;obo = obj[&amp;quot;obo_id&amp;quot;] for obj in data&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This isn't formal Python code. The correction would simply be to place this in a comprehension ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;obo = [obj[&amp;quot;obo_id&amp;quot;] for obj in data]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... unless you want an iterator.&lt;/p&gt;&#xA;&lt;p&gt;I think there is a deep issue with &lt;code&gt;obj[&amp;quot;obo_id&amp;quot;]&lt;/code&gt;. If the syntax still throws an exception could you provide a print out of either &lt;code&gt;data&lt;/code&gt; please, or else ....&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;obj for obj in data:&#xA;    print (obj)&#xA;    print (obj.keys)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;data&lt;/code&gt; data structure for the output of &lt;code&gt;get.requests&lt;/code&gt; followed by &lt;code&gt;.json()&lt;/code&gt; is kinda key here&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-03-06T19:04:56.693" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20674" PostTypeId="2" ParentId="13524" CreationDate="2023-03-06T19:06:59.610" Score="2" Body="&lt;p&gt;As long as the lncRNA is annotated you can use &lt;a href=&quot;https://gtexportal.org/home/gene&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gtexportal.org/home/gene&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17125" LastActivityDate="2023-03-06T19:06:59.610" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20675" PostTypeId="1" CreationDate="2023-03-06T19:24:56.967" Score="1" ViewCount="43" Body="&lt;p&gt;I have a set of files based on which I expect to load RDS files like R1-Tn5-NHP-55_NA and R1-Tn5-NHP-42_NA but the code is producing just the name of the 55 files not both 55 and 42.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;#standardize N set names  &#xA;rename_KP_set &amp;lt;- function(sample_name_vector)  {&#xA; new_sample_name_vector &amp;lt;- apply(sample_name_vector, &#xA;      function(x){     &#xA;         if (grepl(&amp;quot;42&amp;quot;,x[3])){     &#xA;           c(x[1:3],'55',x[5])&#xA;         } else if (grepl(&amp;quot;55&amp;quot;,x[3])) {&#xA;           c(x[1:3],'NHP',x[5])    &#xA;         } else if (grepl(&amp;quot;55&amp;quot;,x[3])){&#xA;           c(x[1:3],'NHP',x[5])&#xA; } else&#xA;c(x[1:3],'55',x[5])&#xA; }  &#xA;}) %&amp;gt;% &#xA;     sapply(paste, collapse='-')   &#xA;     return(new_sample_name_vector) &#xA; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How I can get separate rds files having 42 and 55 numbers or for that matter any rds with some number attached to them?&lt;/p&gt;&#xA;&lt;p&gt;I have already &lt;a href=&quot;https://stackoverflow.com/questions/75548867/naming-function-in-r&quot;&gt;posted this question on Stack overflow on February 27th, 2023&lt;/a&gt; but have got no answers so I am reposting that in Stack exchange bioinformatics in search of an answer. Any help on this issue will be considered useful.&lt;/p&gt;&#xA;" OwnerUserId="9659" LastEditorUserId="650" LastEditDate="2023-03-06T20:50:30.147" LastActivityDate="2023-03-06T20:50:30.147" Title="Naming functions in R" Tags="|r|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20676" PostTypeId="2" ParentId="20666" CreationDate="2023-03-06T20:48:11.720" Score="2" Body="&lt;p&gt;I don't think you can reduce the number of reads required for autodetection, and it would be a bad idea anyway, because the guess could go wrong.&lt;/p&gt;&#xA;&lt;p&gt;From &lt;a href=&quot;https://kb.10xgenomics.com/hc/en-us/articles/360019665352-What-is-the-difference-between-TotalSeq-A-B-and-C-&quot; rel=&quot;nofollow noreferrer&quot;&gt;10X Q&amp;amp;A&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;TotalSeq B is specifically designed to be compatible with the Single Cell 3’ v3 and 3.1 workflows&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And from &lt;a href=&quot;https://support.10xgenomics.com/single-cell-gene-expression/software/pipelines/latest/using/count&quot; rel=&quot;nofollow noreferrer&quot;&gt;CellRanger support&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;--chemistry     Optional. Assay configuration. NOTE: by default the assay configuration is detected automatically, which is the recommended mode. You should only specify chemistry if there is an error in automatic detection. Select one of:&#xA;&#xA;    auto for auto-detection (default),&#xA;    threeprime for Single Cell 3′,&#xA;    fiveprime for Single Cell 5′,&#xA;    SC3Pv2 for Single Cell 3′ v2,&#xA;    SC3Pv3 for Single Cell 3′ v3,&#xA;    SC3Pv3LT for Single Cell 3′ v3 LT,&#xA;    SC3Pv3HT for Single Cell 3′ v3 HT,&#xA;    SC5P-PE for Single Cell 5′ paired-end (both R1 and R2 are used for alignment),&#xA;    SC5P-R2 for Single Cell 5′ R2-only (where only R2 is used for alignment).&#xA;    SC3Pv1 for Single Cell 3′ v1. NOTE: this mode cannot be auto-detected. It must be set explicitly with this option.&#xA;    ARC-v1 for analyzing the GEX portion of multiome data. NOTE: this mode cannot be auto-detected.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That leaves more than one possible chemistry option that could be applicable to your data. Check with the people who prepared the library if you don't know which library kit was used.&lt;/p&gt;&#xA;" OwnerUserId="15489" LastActivityDate="2023-03-06T20:48:11.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20677" PostTypeId="1" AcceptedAnswerId="20699" CreationDate="2023-03-07T07:46:48.813" Score="3" ViewCount="247" Body="&lt;p&gt;I have a dataset that I wish to convert to tree or phylo format like in the &lt;code&gt;ape&lt;/code&gt; package, in order to plot the phylogenetic tree. It is formatted like a list of edges and species names for the leaves. The dataset is quite large, but I have made an example dataset that represents it:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;df &amp;lt;- data.frame(c(8,8,7,6,6,7,9,9,9),&#xA;                c(1,2,3,4,5,6,7,8,9),&#xA;                c(9, 9, 12, 14, 17, 19, 21, 23, 9),&#xA;                c(&amp;quot;a&amp;quot;,&amp;quot;b&amp;quot;,&amp;quot;c&amp;quot;,&amp;quot;d&amp;quot;,&amp;quot;e&amp;quot;, &amp;quot;node1&amp;quot;, &amp;quot;node2&amp;quot;, &amp;quot;node3&amp;quot;, &amp;quot;node4&amp;quot;))&#xA;names(df) &amp;lt;- c(&amp;quot;parent&amp;quot;,&amp;quot;node&amp;quot;,&amp;quot;branch.length&amp;quot;, &amp;quot;label&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I wish to plot it as a rooted phylogenetic tree using the &lt;code&gt;ape&lt;/code&gt; package, but I can't figure out how to convert it to for example a newick string. I have tried to look at solutions both in R and Python, but I can't figure it out. Does anyone have any idea how this can be done?&lt;/p&gt;&#xA;&lt;p&gt;Additional information: I also have two other vectors, one contains the length of each edge, and one containing the age of the child node, equal to the sum of all edge length from that child to any leaf of the tree. It would be great if this could be included, but not neccessary for the plotting as it is only illustrative.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am using R version 4.1.2.&#xA;One of my attempts at translating my data to Newick code (as i understand this could include the branch lengths), using one of [these answers].&#xA;I don't understand what goes wrong, but I have not worked with Newick code before so I am not very experienced with this.&#xA;(&lt;a href=&quot;https://biology.stackexchange.com/questions/37057/construct-phylogeny-form-edgelist&quot;&gt;https://biology.stackexchange.com/questions/37057/construct-phylogeny-form-edgelist&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;df &amp;lt;- data.frame(c(8,8,7,6,6,7,9,9,9),&#xA;                c(1,2,3,4,5,6,7,8,9),&#xA;                c(9, 9, 12, 14, 17, 19, 21, 23, 9),&#xA;                c(&amp;quot;a&amp;quot;,&amp;quot;b&amp;quot;,&amp;quot;c&amp;quot;,&amp;quot;d&amp;quot;,&amp;quot;e&amp;quot;, &amp;quot;node1&amp;quot;, &amp;quot;node2&amp;quot;, &amp;quot;node3&amp;quot;, &amp;quot;node4&amp;quot;))&#xA;names(df) &amp;lt;- c(&amp;quot;parent&amp;quot;,&amp;quot;node&amp;quot;,&amp;quot;branch.length&amp;quot;, &amp;quot;label&amp;quot;)&#xA;tree_data &amp;lt;- df&#xA;str(tree_data)&#xA;&#xA;tree_data&lt;span class=&quot;math-container&quot;&gt;$newick = paste(tree_data$&lt;/span&gt;label,':',signif(tree_data&lt;span class=&quot;math-container&quot;&gt;$branch.length,3),sep=&quot;&quot;)&#xA;nparents = 2&#xA;  while(nparents &amp;gt; 1){&#xA;  leaf = setdiff(tree_data$node,tree_data$parent)&#xA;  parent_leaf = unique(tree_data$parent[!is.na(match(tree_data$node,leaf))])&#xA;  index_remove = integer(0)&#xA;  for(i in 1:length(parent_leaf)){&#xA;parent_children = tree_data$node[which(tree_data$parent == parent_leaf[i])]&#xA;all_children_in_leaf = all(parent_children %in% leaf)&#xA;if(all_children_in_leaf==T){&#xA;  index_remove = c(index_remove,match(parent_children,tree_data$node))&#xA;  tree_data$newick[which(tree_data$node==parent_leaf[i])] = paste(&quot;(&quot;,paste(tree_data$newick[match(parent_children,tree_data$node)],collapse=&quot; , &quot;),&quot;)&quot;,tree_data$newick[which(tree_data$node==parent_leaf[i])] ,sep = &quot; &quot;)&#xA;}&#xA;  }&#xA;  tree_data = tree_data[-index_remove,]&#xA;  parent_next = unique(tree_data$parent[!is.na(match(tree_data$node,leaf))])&#xA;  nparents = length(parent_next)&#xA;  if (nparents==1) newick_code = paste(&quot;(&quot;,paste(tree_data$&lt;/span&gt;newick,collapse=&amp;quot;,&amp;quot;),&amp;quot;)&amp;quot;,colnames(tree_data)[parent_next],&amp;quot;;&amp;quot;,sep = &amp;quot; &amp;quot;)&#xA;}&#xA;&#xA;&#xA;tree &amp;lt;- ape::read.tree(text = newick_code)&#xA;&#xA;plot.phylo(tree)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I don't get an error any more, but I get some unexpected output:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/NXj2q.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NXj2q.png&quot; alt=&quot;The output I get&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is what I would expect, all nodes that are not single letters are inner nodes, not leaves:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/G0lj5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/G0lj5.png&quot; alt=&quot;The output I want&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I really appreciate any help!&lt;/p&gt;&#xA;" OwnerUserId="17127" LastEditorUserId="17127" LastEditDate="2023-03-10T13:41:19.877" LastActivityDate="2023-03-10T16:27:22.670" Title="Plot phylogenetic tree from list of edges" Tags="|r|python|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="20678" PostTypeId="2" ParentId="20341" CreationDate="2023-03-07T09:32:53.053" Score="2" Body="&lt;p&gt;I think the answer to your question is the &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#input-repeaters-each&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;each&lt;/code&gt; input repeaters&lt;/a&gt;. You can also check &lt;a href=&quot;https://www.nextflow.io/docs/latest/faq.html#how-do-i-iterate-over-a-process-n-times&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nextflow FAQ&lt;/a&gt; for the question on how to iterate over a process or a file.&lt;/p&gt;&#xA;&lt;p&gt;Not in front of a PC now but you could try something like &lt;code&gt;each lineage_dataset&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="4413" LastActivityDate="2023-03-07T09:32:53.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20679" PostTypeId="1" AcceptedAnswerId="20680" CreationDate="2023-03-07T12:22:16.583" Score="3" ViewCount="63" Body="&lt;p&gt;I am interested in annotating a list of variants using VARAdb. My coordinates use the GRCh38 (hg19) reference genome. However, I do not know which coordinate system VARAdb uses.&lt;/p&gt;&#xA;&lt;p&gt;A section of the VARAdb publication reads:&#xA;&amp;quot;GWAS have provided a large amount of data&#xA;associating genetic variants with common phenotypes. We&#xA;collected risk SNPs fromfive sources, including theNHGRI&#xA;GWAS Catalog (29), GWASdb v2.0 (30), GAD (31), Johnson&#xA;and O’Donnell (32), and GRASP v2.0 (33). Then, we&#xA;filtered risk SNPs ofwhich ‘Variant ID’which are not ‘rsID’&#xA;and &lt;strong&gt;transformed locations of risk SNPs from the GRCh38&#xA;assembly version into the hg19 version in the GWAS Catalog&lt;/strong&gt;.&#xA;Finally, we obtained 1 515 001 risk SNPs associated&#xA;with diseases, traits and phenotypes.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;which makes me think that they may be using the hg19 genome, but I am not sure.&lt;/p&gt;&#xA;&lt;p&gt;So, my question is:&lt;/p&gt;&#xA;&lt;p&gt;Which human reference genome version does VARAdb use?&lt;/p&gt;&#xA;&lt;p&gt;VARAdb database:&#xA;&lt;a href=&quot;http://www.licpathway.net/VARAdb/search.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.licpathway.net/VARAdb/search.php&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;VARAdb publication:&#xA;&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/33095866/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pubmed.ncbi.nlm.nih.gov/33095866/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17128" LastActivityDate="2023-03-07T14:58:22.277" Title="Which human reference genome version does VARAdb use?" Tags="|sequence-annotation|database|reference-genome|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20680" PostTypeId="2" ParentId="20679" CreationDate="2023-03-07T14:47:58.037" Score="3" Body="&lt;p&gt;They are using hg19.&lt;/p&gt;&#xA;&lt;p&gt;I found this by simply going to &amp;quot;&lt;a href=&quot;http://www.licpathway.net/VARAdb/Browse_all_variation.php?select=Chr::chr1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Browse&lt;/a&gt;&amp;quot; then copying taking the first rsID in the list (rs1891805) and comparing the chromosomal position they give in VARAdb to what I see in dbSNP. VARAdb gives it as position 154904840 on chromosome 1 and &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/rs1891805#variant_details&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbSNP has&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Sequence name       Change&#xA;GRCh37.p13 chr 1    NC_000001.10:g.154904840T&amp;gt;C&#xA;GRCh38.p14 chr 1    NC_000001.11:g.154932364T&amp;gt;C&#xA;PMVK RefSeqGene     NG_053028.1:g.15287A&amp;gt;G&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;GRCh37 is hg19, and GRCh38 is hg38. So, as you can see above, the position 154904840 is from hg19 and the hg38 position for this rsID is 154932364.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2023-03-07T14:58:22.277" LastActivityDate="2023-03-07T14:58:22.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20681" PostTypeId="2" ParentId="20617" CreationDate="2023-03-07T22:38:58.290" Score="2" Body="&lt;p&gt;This answer assumes that you have a terminal that runs linux commands and python, and that you can install and run bedtools.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;step 1:&lt;/strong&gt; convert your input spreadsheet to &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwiN9KTU7sr9AhU3JjQIHWiHA5YQFnoECA4QAQ&amp;amp;url=https%3A%2F%2Fgenome.ucsc.edu%2FFAQ%2FFAQformat.html&amp;amp;usg=AOvVaw1YBDwZ04GPxtn2N95WP6Lr&quot; rel=&quot;nofollow noreferrer&quot;&gt;BED format&lt;/a&gt;.&#xA;There is probably an awk one liner that does this but something like this python script will do it based on csv input (assuming that your coordinates are 1-indexed):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# writes BED format to standard output, capture it in input_file.bed&#xA;# not actually run&#xA;with open(&amp;quot;input_file.csv&amp;quot;) as infile:&#xA;  for line in infile:&#xA;    fields = line.split(&amp;quot;,&amp;quot;)&#xA;    coord = int(fields[2])&#xA;    outline = f&amp;quot;chr{fields[0]}\t{coord - 1}\t{coord}\t{fields[1]}&amp;quot;&#xA;    print(outline)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;step 2:&lt;/strong&gt;&#xA;sort the BED.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sort -k1,1 -k2,2n input_file.bed &amp;gt; input_file.sorted.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;step 3:&lt;/strong&gt;&#xA;Get genome annotations. I am assuming that you are using the grch38 build of the human genome.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;wget https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_human/release_43/gencode.v43.annotation.gff3.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;step 4:&lt;/strong&gt;&#xA;Get nearest gene for each snp.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bedtools closest -a input_file.sorted.bed -b gencode.v43.annotation.gff3.gz &amp;gt; input_annotated_with_genes.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2023-03-08T16:27:07.090" LastActivityDate="2023-03-08T16:27:07.090" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20682" PostTypeId="2" ParentId="20451" CreationDate="2023-03-07T23:10:17.930" Score="3" Body="&lt;p&gt;It looks like various chemical software toolkits can export SMILES, for example &lt;a href=&quot;http://openbabel.org/wiki/SMILES&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenBabel&lt;/a&gt;, using the &lt;a href=&quot;http://openbabel.org/wiki/Babel&quot; rel=&quot;nofollow noreferrer&quot;&gt;babel command line tool.&lt;/a&gt; I am not readily seeing the best way to read e.g. a FASTA, though. Another option would be &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwi52YLa98r9AhVPMTQIHT8QCRkQFnoECAwQAQ&amp;amp;url=https%3A%2F%2Fperkinelmerinformatics.com%2Fproducts%2Fresearch%2Fchemdraw&amp;amp;usg=AOvVaw0BmVb9fWH4mKXs22TzVe39&quot; rel=&quot;nofollow noreferrer&quot;&gt;ChemDraw&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.researchgate.net/post/How_do_I_convert_my_peptides_into_SMILES_strings&quot; rel=&quot;nofollow noreferrer&quot;&gt;This researchgate thread&lt;/a&gt; has a lot of information on peptide-&amp;gt;SMILES conversion, I don't see nucleotides though.&lt;/p&gt;&#xA;&lt;p&gt;One option, though it would be a little finicky, would be to look up the nucleotides SMILES strings on &lt;a href=&quot;https://pubchem.ncbi.nlm.nih.gov/#query=adenosine&quot; rel=&quot;nofollow noreferrer&quot;&gt;PubChem&lt;/a&gt; and then write a script to string them together (with a monophosphate) based on an input sequence. It seems like &lt;a href=&quot;https://github.com/rdkit/rdkit/issues/3080&quot; rel=&quot;nofollow noreferrer&quot;&gt;there is some related tooling already in rdkit&lt;/a&gt;, but a convenient API doesn't seem to exist yet. &lt;a href=&quot;https://Born,%20Jannis.%20(2020).%20Re:%20How%20do%20I%20convert%20my%20peptides%20into%20SMILES%20strings?.%20Retrieved%20from:%20https://www.researchgate.net/post/How_do_I_convert_my_peptides_into_SMILES_strings/5eaff1007b75ea5bbf031b32/citation/download.&quot; rel=&quot;nofollow noreferrer&quot;&gt;The developer does have one suggestion for peptides&lt;/a&gt;, which would translate into something like this for nucleotides:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from rdkit import Chem&#xA;&#xA;nt_mol = Chem.MolFromFASTA(&amp;quot;ACATATAT&amp;quot;, flavor=9)  # flavor param notes as DNA&#xA;Chem.MolToSMILES(nt_mol)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To do this across many nucleotides you could hook it up to biopython to iterate across a FASTA file.&lt;/p&gt;&#xA;&lt;p&gt;The nucleotide &lt;code&gt;flavor&lt;/code&gt; option exists, but there is a note that only peptide is supported. I don't know whether that means the feature is experimental or is not implemented.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-03-07T23:10:17.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20683" PostTypeId="1" CreationDate="2023-03-08T02:07:09.667" Score="0" ViewCount="42" Body="&lt;p&gt;JSMol, the molecular modelling software, requires PDB file format to work and render the graphics. Currently, I am writing an annotation software - backend in Perl - that takes in a .faa file (fasta amino acids). Is there any way or tool out there that can easily convert to a PDB file? Many of the tools out there do not seem to give what I need.&lt;/p&gt;&#xA;&lt;p&gt;My workflow so far:&lt;/p&gt;&#xA;&lt;p&gt;input.fna -&amp;gt; Bakta -&amp;gt; bakta.faa -&amp;gt; &lt;strong&gt;covert tool&lt;/strong&gt; -&amp;gt; proteins.pdb -&amp;gt; JSMol&lt;/p&gt;&#xA;&lt;p&gt;Essentially, a tool exists called pdb2fasta that takes a pdb file and converts to a fasta file but can’t seem to find a proper tool that does it the other way around.&lt;/p&gt;&#xA;" OwnerUserId="17070" LastEditorUserId="73" LastEditDate="2023-03-09T19:49:05.363" LastActivityDate="2023-03-09T19:49:05.363" Title="What tool(s) can I use to use a .faa file to feed into JSMol?" Tags="|fasta|protein-structure|sequence-annotation|pdb|perl|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20684" PostTypeId="1" AcceptedAnswerId="20688" CreationDate="2023-03-08T14:08:07.140" Score="4" ViewCount="323" Body="&lt;p&gt;My problem is that I want to retrieve the sequences line + secondary structure line that match the protein names from the &lt;code&gt;header.txt&lt;/code&gt; file:&lt;/p&gt;&#xA;&lt;p&gt;The text file(&lt;code&gt;header.txt&lt;/code&gt;) contains the protein names that look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1XXIJ&#xA;1P0EA&#xA;1H9HI&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the other file(&lt;code&gt;pdb.txt&lt;/code&gt;) structured like this:&lt;/p&gt;&#xA;&lt;p&gt;Line 1:the protein name&lt;/p&gt;&#xA;&lt;p&gt;Line 2: protein length&lt;/p&gt;&#xA;&lt;p&gt;Line 3: amino acid sequence&lt;/p&gt;&#xA;&lt;p&gt;Line 4: secondary structure&lt;/p&gt;&#xA;&lt;p&gt;Line 5: empty&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;4LGTD&#xA;247&#xA;M       S       E       K       L       Q       K       V       L       A       R       &#xA;.       .       E       E       H       H       H       H       H       H       H       &#xA;&#xA;1XXIJ&#xA;334&#xA;M       R       W       Y       P       W       L       R       P       D       F       &#xA;.       .       .       .       T       T       H       H       H       H       H       &#xA;&#xA;6U02G&#xA;116&#xA;E       V       Q       L       V       E       S       G       G       G       L       &#xA;.       .       E       E       E       E       E       .       .       E       E       &#xA;&#xA;1P0EA&#xA;363&#xA;R       P       R       F       S       F       S       I       A       A       R       &#xA;.       .       S       E       E       E       E       E       E       E       E  &#xA; &#xA;1H9HI&#xA;30&#xA;G       C       P       R       I       L       I       R       C       K       Q       &#xA;.       E       E       .       .       .       .       B       S       S&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So my expected output is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1XXIJ&#xA;334&#xA;M       R       W       Y       P       W       L       R       P       D       F       &#xA;.       .       .       .       T       T       H       H       H       H       H       &#xA;&#xA;1P0EA&#xA;363&#xA;R       P       R       F       S       F       S       I       A       A       R       &#xA;.       .       S       E       E       E       E       E       E       E       E  &#xA; &#xA;1H9HI&#xA;30&#xA;G       C       P       R       I       L       I       R       C       K       Q       &#xA;.       E       E       .       .       .       .       B       S       S&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Do you guys have any suggestions for me by awk or any(Linux)code? Please let me know if you require additional information. Cheers!&lt;/p&gt;&#xA;" OwnerUserId="16923" LastEditorUserId="650" LastEditDate="2023-03-08T15:06:10.983" LastActivityDate="2023-03-08T23:55:16.893" Title="How to retrieve sequences from a txt file by protein names from other txt file?" Tags="|proteins|protein-structure|linux|awk|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20685" PostTypeId="2" ParentId="20684" CreationDate="2023-03-08T15:05:02.553" Score="2" Body="&lt;p&gt;I'm assuming that your second file is tab-delimited and has no commas. The easiest way to do this would be to transpose each record in your second file to a single line and then grep, then transform the result again:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -Ff header.txt &amp;lt;(paste -d, - - - - - &amp;lt; pdb.txt) | tr &amp;quot;,&amp;quot; &amp;quot;\n&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I just tested and it works:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;➜ cat pdb.txt&#xA;4LGTD&#xA;247&#xA;M       S       E       K       L       Q       K       V       L       A       R&#xA;.       .       E       E       H       H       H       H       H       H       H&#xA;&#xA;1XXIJ&#xA;334&#xA;M       R       W       Y       P       W       L       R       P       D       F&#xA;.       .       .       .       T       T       H       H       H       H       H&#xA;&#xA;6U02G&#xA;116&#xA;E       V       Q       L       V       E       S       G       G       G       L&#xA;.       .       E       E       E       E       E       .       .       E       E&#xA;&#xA;1P0EA&#xA;363&#xA;R       P       R       F       S       F       S       I       A       A       R&#xA;.       .       S       E       E       E       E       E       E       E       E&#xA;&#xA;1H9HI&#xA;30&#xA;G       C       P       R       I       L       I       R       C       K       Q&#xA;.       E       E       .       .       .       .       B       S       S&#xA;&#xA;➜ cat header.txt&#xA;1XXIJ&#xA;1P0EA&#xA;1H9HI&#xA;&#xA;➜ grep -Ff header.txt &amp;lt;(paste -d, - - - - - &amp;lt; pdb.txt) | tr &amp;quot;,&amp;quot; &amp;quot;\n&amp;quot;&#xA;1XXIJ&#xA;334&#xA;M       R       W       Y       P       W       L       R       P       D       F&#xA;.       .       .       .       T       T       H       H       H       H       H&#xA;&#xA;1P0EA&#xA;363&#xA;R       P       R       F       S       F       S       I       A       A       R&#xA;.       .       S       E       E       E       E       E       E       E       E&#xA;&#xA;1H9HI&#xA;30&#xA;G       C       P       R       I       L       I       R       C       K       Q&#xA;.       E       E       .       .       .       .       B       S       S&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastEditorUserId="298" LastEditDate="2023-03-08T16:45:35.017" LastActivityDate="2023-03-08T16:45:35.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20686" PostTypeId="1" AcceptedAnswerId="20687" CreationDate="2023-03-08T15:23:17.633" Score="4" ViewCount="214" Body="&lt;p&gt;I have 36 samples in total in the bam_files folder, with the name like this 20230306_CH_EP_C01.md.bam&lt;/p&gt;&#xA;&lt;p&gt;I expect the code to output all the 36 samples, one by one. But the run stopped after only running one sample.&#xA;I have searched online, but did not find a solution.&#xA;I am using nextflow version 2.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bam_chl = Channel&#xA;     .fromPath(&amp;quot;bam_files/*.md.bam&amp;quot;)&#xA;     .map{ item -&amp;gt; tuple(item.baseName.substring(0, 18), item) }&#xA;     &#xA;&#xA;     &#xA;ref_chl = Channel&#xA;      .fromPath(&amp;quot;sequences/hg38.fa&amp;quot;)&#xA;ref_index_chl = Channel&#xA;      .fromPath(&amp;quot;sequences/hg38.fa.fai&amp;quot;)&#xA;amplicon_chl = Channel&#xA;      .fromPath(&amp;quot;sequences/tp53_amplicon.hg38.interval_list&amp;quot;)&#xA;target_chl = Channel&#xA;      .fromPath(&amp;quot;sequences/TP53.hg38.interval_list&amp;quot;)&#xA;     &#xA;&#xA;&#xA;process get_pcr_metric {&#xA;&#xA;  publishDir path: 'pcr_metric', pattern: '*.csv'&#xA;  &#xA;  input:&#xA;  tuple val(pair_id), path(bam)&#xA;  path(ref)&#xA;  path(ref_index)&#xA;  path(amplicon)&#xA;  path(target)&#xA;&#xA;  output:&#xA;  path(&amp;quot;*.pcr_metrics.csv&amp;quot;)&#xA;&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  java -jar /picard.jar CollectTargetedPcrMetrics \&#xA;       I=&lt;span class=&quot;math-container&quot;&gt;$bam \&#xA;   O=$&lt;/span&gt;{pair_id}_pcr_metrics.txt \&#xA;       R=&lt;span class=&quot;math-container&quot;&gt;$ref \&#xA;   AMPLICON_INTERVALS=$&lt;/span&gt;amplicon \&#xA;       TARGET_INTERVALS=$target&#xA;       &#xA;  sed -n -e 7p -e 8p &lt;span class=&quot;math-container&quot;&gt;${pair_id}_pcr_metrics.txt |  sed 's/\t/,/g'   &amp;gt; $&lt;/span&gt;{pair_id}.pcr_metrics.csv&#xA;  &#xA;  &#xA;&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;&#xA;}&#xA;&#xA;&#xA;workflow {&#xA;&#xA;get_pcr_metric(bam_chl, ref_chl,ref_index_chl, amplicon_chl, target_chl)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14191" LastEditorUserId="3694" LastEditDate="2023-06-01T09:41:18.893" LastActivityDate="2023-06-01T09:41:18.893" Title="Why does this nextflow script finish after running one sample?" Tags="|nextflow|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20687" PostTypeId="2" ParentId="20686" CreationDate="2023-03-08T16:43:49.000" Score="3" Body="&lt;p&gt;The issue here is that the &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#frompath&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;fromPath&lt;/code&gt;&lt;/a&gt; factory method produces a &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#queue-channel&quot; rel=&quot;nofollow noreferrer&quot;&gt;queue channel&lt;/a&gt;, but you don't want all of your input channels to be queue channels. Most of the time, what you want is one queue channel and one or more &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#value-channel&quot; rel=&quot;nofollow noreferrer&quot;&gt;value&lt;/a&gt; channels when you require &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#process-multiple-input-channels&quot; rel=&quot;nofollow noreferrer&quot;&gt;multiple input channels&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You &lt;em&gt;could&lt;/em&gt; use the &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#value&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;value&lt;/code&gt;&lt;/a&gt; factory method to create the value channels, but a simpler way is to just pass in some file objects. The latter works because a value channel is implicitly created by a process when it is invoked with a simple value&lt;sup&gt;&lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#channel-type-value&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;&lt;/sup&gt;. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params.ref_fasta = &amp;quot;./sequences/hg38.fa&amp;quot;&#xA;params.amplicon_intervals = &amp;quot;./sequences/tp53_amplicon_intervals.hg38.interval_list&amp;quot;&#xA;params.target_intervals = &amp;quot;./sequences/TP53.hg38.interval_list&amp;quot;&#xA;&#xA;params.bam_files = &amp;quot;./bam_files/*.md.bam&amp;quot;&#xA;params.outdir = './results'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;process get_pcr_metric {&#xA;&#xA;    publishDir path: &amp;quot;${params.outdir}/pcr_metric&amp;quot;, mode: 'copy'&#xA;&#xA;    input:&#xA;    tuple val(pair_id), path(bam_file)&#xA;    path ref_fasta&#xA;    path amplicon_intervals&#xA;    path target_intervals&#xA;&#xA;    output:&#xA;    path &amp;quot;${pair_id}_pcr_metrics.txt&amp;quot;&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    picard CollectTargetedPcrMetrics \\&#xA;         I=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${bam_file}&quot; \\&#xA;         O=&quot;$&lt;/span&gt;{pair_id}_pcr_metrics.txt&amp;quot; \\&#xA;         R=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${ref_fasta}&quot; \\&#xA;         AMPLICON_INTERVALS=&quot;$&lt;/span&gt;{amplicon_intervals}&amp;quot; \\&#xA;         TARGET_INTERVALS=&amp;quot;${target_intervals}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow {&#xA;&#xA;    ref_fasta = file( params.ref_fasta )&#xA;    amplicon_intervals = file( params.amplicon_intervals )&#xA;    target_intervals = file( params.target_intervals )&#xA;&#xA;    Channel&#xA;        .fromPath( params.bam_files )&#xA;        .map { tuple( it.getBaseName(2), it ) }&#xA;        .set { bam_files }&#xA;&#xA;    get_pcr_metric( bam_files, ref_fasta, amplicon_intervals, target_intervals )&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-03-08T16:43:49.000" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20688" PostTypeId="2" ParentId="20684" CreationDate="2023-03-08T22:59:45.967" Score="2" Body="&lt;p&gt;If your file structure is consistent, one potential solution is to use GNU grep:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -A4 --no-group-separator -Fwf header.txt pdb.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This works with your example data and should be relatively fast on your 'real' data, due to &lt;code&gt;-Fw&lt;/code&gt; ('fixed' i.e. not regex, and only match whole words) but it requires GNU grep for the &lt;code&gt;--no-group-separator&lt;/code&gt; option (i.e. not BSD grep).&lt;/p&gt;&#xA;" OwnerUserId="9098" LastEditorUserId="9098" LastEditDate="2023-03-08T23:55:16.893" LastActivityDate="2023-03-08T23:55:16.893" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20689" PostTypeId="2" ParentId="20684" CreationDate="2023-03-08T23:15:08.697" Score="2" Body="&lt;p&gt;One way using &lt;a href=&quot;/questions/tagged/awk&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; aria-label=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; rel=&quot;tag&quot; aria-labelledby=&quot;tag-awk-tooltip-container&quot;&gt;awk&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'FNR==NR { a[$1]; next } FNR%5==1 { f = ($1 in a) ? 1 : 0 }f' header.txt pdb.txt &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This adds each of the protein names in &lt;em&gt;header.txt&lt;/em&gt; to an array. By adding the first field (&lt;code&gt;$1&lt;/code&gt;), rather than the complete record (&lt;code&gt;$0&lt;/code&gt;) to the array, we can ensure that we strip off any lagging whitespace which can sometimes cause problems depending on how the list of protein names was created. Next, we can check whether the record number (i.e. line number) of the second input file is a multiple of 5 with remainder 1. This condition returns true for the first line of each record in &lt;em&gt;pdb.txt&lt;/em&gt;. We can then use a ternary operator to set a flag variable to one if the first field of the current line (i.e., the protein name) is present in the a array, &lt;em&gt;or&lt;/em&gt; to zero to indicate that the current record is not one of the records in the array. The &lt;code&gt;f&lt;/code&gt; by itself is just shorthand for &lt;code&gt;f != 0 { print }&lt;/code&gt;, which prints the current line when the flag is non-zero.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2023-03-08T23:26:20.597" LastActivityDate="2023-03-08T23:26:20.597" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20690" PostTypeId="1" AcceptedAnswerId="20691" CreationDate="2023-03-09T00:57:21.560" Score="1" ViewCount="284" Body="&lt;p&gt;How can I get a csv or a list of the gene symbol names for all the protein coding genes in mice?  I have RNA sequencing data and I'm not interested in the non-coding stuff. I'm worried it could mess up my results.  Can GEO or blast provide this?&lt;/p&gt;&#xA;&lt;p&gt;Edit: I am not interested in EntrezID or Ensemble, I don't not want them I just am more interested in gene name like Fos or Penk. Etc.  I work in RNA seq I don't need to map any fastq files to a genome that has been done already I just need to filter out genes that are not protien coding.  If you are not familiar with what RNA seq data looks like it is simply rows of genes and columns of samples each entry giving the number of reads for that gene in that sample, single cell RNA seq is the same.  Have a look at GEO if you want more examples: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE145970&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE145970&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2482" LastEditorUserId="2482" LastEditDate="2023-03-09T09:24:02.197" LastActivityDate="2023-03-09T16:16:47.777" Title="Gene symbol list for all protein coding genes in mice" Tags="|rna-seq|blast|rna|geoquery|geo|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20691" PostTypeId="2" ParentId="20690" CreationDate="2023-03-09T09:31:04.740" Score="5" Body="&lt;p&gt;If you want &lt;em&gt;all&lt;/em&gt; known mouse protein coding genes, you can get the list from &lt;a href=&quot;https://www.ensembl.org/biomart&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl's BioMart&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;First, select the mouse genome:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Ohjp3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Ohjp3.png&quot; alt=&quot;BioMart screenshot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then, in &amp;quot;Filters&amp;quot; limit to protein coding genes only:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ovrFs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ovrFs.png&quot; alt=&quot;BioMart screenshot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And next, choose &amp;quot;Gene name&amp;quot; and, to have a more standardized and stable name and to ensure that you get allr esults (not all genes have a &amp;quot;name&amp;quot;, but they will all have IDs), include the &amp;quot;Gene stable ID&amp;quot;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pZpLA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pZpLA.png&quot; alt=&quot;BioMart screenshot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Finally, click on &amp;quot;Results&amp;quot;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mNF5M.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mNF5M.png&quot; alt=&quot;BioMart Screenshot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Select &amp;quot;File&amp;quot; and &amp;quot;CSV&amp;quot; and then click on &amp;quot;Go&amp;quot; to download the list:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Y1QM3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Y1QM3.png&quot; alt=&quot;BioMart screenshot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The end result is a file like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Gene stable ID,Gene name&#xA;ENSMUSG00000064341,mt-Nd1&#xA;ENSMUSG00000064345,mt-Nd2&#xA;ENSMUSG00000064351,mt-Co1&#xA;ENSMUSG00000064354,mt-Co2&#xA;ENSMUSG00000064356,mt-Atp8&#xA;ENSMUSG00000064357,mt-Atp6&#xA;ENSMUSG00000064358,mt-Co3&#xA;ENSMUSG00000064360,mt-Nd3&#xA;ENSMUSG00000065947,mt-Nd4l&#xA;ENSMUSG00000064363,mt-Nd4&#xA;ENSMUSG00000064367,mt-Nd5&#xA;ENSMUSG00000064368,mt-Nd6&#xA;ENSMUSG00000064370,mt-Cytb&#xA;ENSMUSG00000095019,&#xA;ENSMUSG00000094915,&#xA;ENSMUSG00000095041,&#xA;ENSMUSG00000095666,&#xA;ENSMUSG00000094741,Pramel39&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With 21849 gene IDs, of which 21772 have a &amp;quot;name&amp;quot; as well.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="650" LastEditDate="2023-03-09T16:16:47.777" LastActivityDate="2023-03-09T16:16:47.777" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20693" PostTypeId="2" ParentId="20668" CreationDate="2023-03-09T17:17:50.883" Score="2" Body="&lt;p&gt;@Megha&#xA;The requestIt(link) function returns a list of dictionary objects.&#xA;So your &amp;quot;data&amp;quot; variable is a list of dictionary objects.&lt;/p&gt;&#xA;&lt;p&gt;Every item on this list (a dict) has a key named &amp;quot;_embedded&amp;quot; whose value is also a dictionary.&#xA;This dictionary (the _embedded) has a key named &amp;quot;terms&amp;quot; which is a list of dictionaries.&#xA;The obo_ids you need are inside the dictionaries of the &amp;quot;terms&amp;quot; list.&lt;/p&gt;&#xA;&lt;p&gt;The problem with your code was that you were trying to access the &amp;quot;obo_id&amp;quot; key on the top level dictionary.&lt;/p&gt;&#xA;&lt;p&gt;So a simple solution would be like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;with open ('urls.txt', 'r') as fin:&#xA;    links = fin.readlines()&#xA;resps = [requestIt(link) for link in links] # this one holds all the data&#xA;obo_ids = []&#xA;for resp in resps:                          # for every response we got&#xA;    terms = resp['_embedded']['terms']      # choose the terms list&#xA;    for term in terms:                      # for every dict in this list&#xA;        obo_ids.append(term['obo_id'])      # extract obo_id and append it to our answer&#xA;print(obo_ids)                              # do stuff with obo_ids&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You should probably perform some exception handling inside the requestIt() function to handle broken links etc.&lt;/p&gt;&#xA;" OwnerUserId="17138" LastActivityDate="2023-03-09T17:17:50.883" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20694" PostTypeId="2" ParentId="20301" CreationDate="2023-03-09T22:40:45.137" Score="1" Body="&lt;p&gt;Here can use GraphiQL API provided by RCSB.&#xA;Here is an example&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import requests&#xA;from rdkit import Chem&#xA;pdbid = '7RDX'&#xA;ls_smi = []&#xA;&#xA;query = f'''{{&#xA;  entry(entry_id: &amp;quot;{pdbid}&amp;quot;) {{&#xA;    nonpolymer_entities {{&#xA;      nonpolymer_comp {{&#xA;        rcsb_id&#xA;        pdbx_chem_comp_descriptor {{&#xA;          type&#xA;          comp_id&#xA;          program&#xA;          descriptor&#xA;        }}&#xA;      }}&#xA;    }}&#xA;  }}&#xA;}}'''&#xA;url = &amp;quot;https://data.rcsb.org/graphql&amp;quot;&#xA;r = requests.post(url, json={'query': query})&#xA;for i in r.json()['data']['entry']['nonpolymer_entities']:&#xA;    if i['nonpolymer_comp']['pdbx_chem_comp_descriptor'][0]['type'] == 'SMILES':&#xA;        ls_smi.append(Chem.CanonSmiles(i['nonpolymer_comp']['pdbx_chem_comp_descriptor'][0]['descriptor']))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17141" LastEditorUserId="650" LastEditDate="2023-03-15T21:26:13.740" LastActivityDate="2023-03-15T21:26:13.740" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20695" PostTypeId="1" CreationDate="2023-03-10T00:18:15.010" Score="1" ViewCount="46" Body="&lt;p&gt;I have two samples A and B and I do have an RNAseq for both of them. Inside each file (for A and B) I do have a set of FPKMs that give me a fold change for each gene (sample vs control). It looks somewhat like that:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gen_name grp01 grp02 grp03 grp04 fold_change&#xA;gene1    10    10    5     5     2&#xA;gene2    11    11    11    11    1&#xA;gene3    5     5     10    10    0.5&#xA;gene4    ... &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;for sample A&#xA;and, say&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gen_name grp05 grp06 grp07 grp08 fold_change&#xA;gene1    100   100   5     5     20&#xA;gene2    110   110   110   110   1&#xA;gene3    5     5     100   100   0.05&#xA;gene4    ... &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;for sample B&lt;/p&gt;&#xA;&lt;p&gt;I need to show with PCA that, foe example, gene1 is sample B is much more overexpressed than in sample A, same for gene3, but it looks like gene2 is not very interesting.&lt;/p&gt;&#xA;&lt;p&gt;What data must be put in PCA analysis?&lt;/p&gt;&#xA;" OwnerUserId="15486" LastActivityDate="2023-03-10T00:18:15.010" Title="What is the best approach to do a PCA analysis to distinguish between two groups?" Tags="|r|differential-expression|pca|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20698" PostTypeId="1" CreationDate="2023-03-10T16:00:27.287" Score="4" ViewCount="61" Body="&lt;p&gt;I would like to identify the somatic mutations present in a cell line and characterise the genes that are potentially affected by those mutations. For example, are there oncogenes mutated in a subpopulation in the cell line?&lt;/p&gt;&#xA;&lt;p&gt;I have currently access to single cell WGS data and sequencing data of the bulk sample derived from the same clone. The data was generate after MDA amplification of the single cell's genome so I am currently evaluating SCCaller and Prosolo for variant calling. If I correctly understand single cell variant calling, mutations called in each cell are called relatively to the bulk background sample. So each cell separately doesn't provide much information on the cell line itself.&lt;/p&gt;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Do i need single cell WGS to quantify the overall 'genomic health' of a cell line or can I just use deep sequencing followed by somatic variant calling against a reference genome.&lt;/li&gt;&#xA;&lt;li&gt;What added value can single cell WGS provide here and how can I extract that information?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Links to papers, packages or statistical models more than welcome.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="16874" LastActivityDate="2024-04-05T18:07:10.883" Title="Identifying somatic mutations in cell lines" Tags="|single-cell|genomics|variant-calling|mutations|variants|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20699" PostTypeId="2" ParentId="20677" CreationDate="2023-03-10T16:27:22.670" Score="4" Body="&lt;p&gt;The following example uses python to generate the newick string. You can than import that in R for further analysis.&lt;/p&gt;&#xA;&lt;p&gt;Please keep in mind that you can not use self reference for node4 (9,9,9), so I changed this to 'Root'.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from ete3 import Tree&#xA;import pandas as pd&#xA;&#xA;df = pd.DataFrame({'parent': [8, 8, 7, 6, 6, 7, 9, 9, 'Root'],&#xA;                   'node': [1, 2, 3, 4, 5, 6, 7, 8, 9],&#xA;                   'branch.length': [9, 9, 12, 14, 17, 19, 21, 23, 9],&#xA;                   'label': ['a', 'b', 'c', 'd', 'e', 'node1', 'node2', 'node3', 'node4']})&#xA;&#xA;&#xA;parent_child_list = df[['parent', 'node', 'branch.length']].apply(tuple, axis=1).tolist()&#xA;tree = Tree.from_parent_child_table(parent_child_list)&#xA;newick_str = tree.write(format=1)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;newick_str:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;'(((3:12,(4:14,5:17)6:19)7:21,(1:9,2:9)8:23)9:9);'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16874" LastActivityDate="2023-03-10T16:27:22.670" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20700" PostTypeId="1" AcceptedAnswerId="20706" CreationDate="2023-03-11T00:59:15.417" Score="1" ViewCount="201" Body="&lt;p&gt;Is there an easy way to grab the rsids (hg38) of a VCF file? I know one of the tabs can contain RSIDs, but this file doesn't contain any.&lt;/p&gt;&#xA;&lt;p&gt;I have a software I created for mapping 23AndMe DTC testing to SNPedia, and I'm trying to integrate more DTC testing. (&lt;a href=&quot;https://www.github.com/mentatpsi/osgenome&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.github.com/mentatpsi/osgenome&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;The user found this &lt;a href=&quot;https://gist.github.com/KlausTrainer/26c2996cf32677e4c107bd8aaee67794&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gist.github.com/KlausTrainer/26c2996cf32677e4c107bd8aaee67794&lt;/a&gt; which is a script to use variant summary data to map to the RSIDs, but I'm hesitant to perform a SQL dump to find a table that contains&#xA;chromosome, postion, and rsID columns.&lt;/p&gt;&#xA;&lt;p&gt;As a bonus question, 23AndMe reports all positive orientations. How would I go about converting strands to match this orientation? Does the VCF contain information that would allow this action autonomously?&lt;/p&gt;&#xA;" OwnerUserId="17148" LastEditorUserId="17148" LastEditDate="2023-03-11T01:06:35.473" LastActivityDate="2023-03-13T10:49:08.293" Title="Converting a Dante Lab VCF file" Tags="|vcf|gene|snp|genotyping|rsid|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20702" PostTypeId="1" AcceptedAnswerId="20703" CreationDate="2023-03-11T14:17:13.283" Score="2" ViewCount="58" Body="&lt;p&gt;I have created a python code that prints out all lengths of ORFs in a DNA sequence (start codon - &amp;quot;ATG&amp;quot;, stop codons - &amp;quot;TAG&amp;quot;, &amp;quot;TAA&amp;quot; and &amp;quot;TGA&amp;quot;. It seems to give correct answers, when compared with specialized ORF-finding programs. The thing is that besides correct answers it also gives an Index Error in the end:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &amp;quot;scripts.py&amp;quot;, line 33, in &amp;lt;module&amp;gt;&#xA;    if list_start[i]&amp;gt;list_stop[n-1]:&#xA;IndexError: list index out of range&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, I have created a dictionary from a FASTA file where &lt;em&gt;key&lt;/em&gt; is a sequence name and &lt;em&gt;value&lt;/em&gt; is the sequence itself and want to find ORFs for all sequences in that dictionary using a &lt;em&gt;for&lt;/em&gt; loop and this code. However, the loop runs only once and I think that the Index Error to be the cause.&lt;/p&gt;&#xA;&lt;p&gt;The code is the following:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;line=&amp;quot;TCGGGCGAAGGCGGCAGCAAGTCGTCCACGCGCAGCGCGGCACCGCGGGCCTCTGCCGTGCGCTGCTTGGCCATGGCCTCCAGCGCACCGATCGGATCAAAGCCGCTGAAGCCTTCGCGCATCAGGCGGCCATAGTTGGCGCCAGTGACCGTACCAACCGCCTTGATGCGGCGCTCGGTCATCGCTGCATTGATCGAGTAGCCACCGCCGCCGCAAATGCCCAGCACGCCAATGCGTTCTTCATCCACATAGGGGAGCGTTACGAGGTAGTCGCAGACCACGCGGAAATCCTCGACGCGCAGTGTCGGGTCTTCGGTAAAACGTGGTTCGCCGCCGCTGGCACCCTGGAAGCTGGCGTCGAAGGCGATGACGACGAAACCTTCCTTGGCCAGCGCCTCGCCATACACGTTCCCCGATGTTTGCTCCTTGCAGCTGCCGATCGGATGCGCGCTGATGATGGCGGGATATTTCTTGCCTTCGTCGAAGTTCGGCGGGAAGTGGATGTCGGCTGCGATATCCCAATACACATTCTTGATCTTGACGCTTTTCATGACAGCTCCGTTCAGGGGGAGGGGGTAAGTTCGCCAGGCCGAATCGTTGGTAGCCAAGCGGCAACGACTCGAATATAGAGAGCCGATTGGAATTCCGTAAGATCGCAATCTGGACTACAGTGGTATCTTCAAATTGACAATGGCACCTACATGGATCCCTCACTGCTTCCGTCTCTCGCGTGGTTCGCCCACGTCGCACATCATCGTAGCTTCACGAAAGCGGCTGCGGAAATGGGCGTTTCTCGAGCAAACCTGTCGCAGAACGTGAAGGCGCTCGAACGCCGGTTGAACGTCAAGCTGCTGTATCGAACGACTCGCGACATGTCGCTGACCGAGGAGGGGCAGCGGCTCTACGAGGTGTGGTATCCCGCGCTGGTCGCGGTCGAGCGGACGGTCGACGCGCTGCACGAGGAGCGCGACGAGCCGTCGGGGTTGATTC&amp;quot;&#xA;r_frame=input('Enter a reading frame')&#xA;line=line[int(r_frame)-1:]&#xA;# making list of codons (3-nucleotide groups)&#xA;x=3&#xA;codon=[line[y-x:y] for y in range(x, len(line)+x,x)]&#xA;#defininig function  that will find all the indexes of codon in list of codons&#xA;def find_indices(list_to_check, item_to_find):&#xA;    indices = []&#xA;    for idx, value in enumerate(codon):&#xA;        if value == item_to_find:&#xA;            indices.append(idx)&#xA;    return indices&#xA;# makirng list with all the indexes of &amp;quot;ATG&amp;quot; (start codon)&#xA;list_start=find_indices(codon, &amp;quot;ATG&amp;quot;)&#xA;#making list with all the indexes of &amp;quot;TGA&amp;quot;, &amp;quot;TAA&amp;quot; or &amp;quot;TAG&amp;quot; (stop codons) and sorting it from smaller to bigger&#xA;list_stop=find_indices(codon, &amp;quot;TGA&amp;quot;)+find_indices(codon, &amp;quot;TAA&amp;quot;)+find_indices(codon, &amp;quot;TAG&amp;quot;)&#xA;list_stop.sort()&#xA;i=0 # variable for indexes in list_start&#xA;n=0 #variable for indexes in list_stop&#xA;j=0 # variable that determines numer of times searching for ORF will be repeated&#xA;#first round without loop so that i and n will be &amp;gt;0 and we can make  if list_start[i]&amp;gt;list_stop[n-1] without errors&#xA;if list_start[i]&amp;lt;list_stop[n]:&#xA;    print(3*(list_stop[n]-list_start[i])+3)&#xA;    j=j+1&#xA;    n=n+1&#xA;    i=i+1&#xA;else:&#xA;    n=n+1&#xA;#loop that will search for suitable start codon-stop codon pairs and print the length of ORF&#xA;while j&amp;lt;codon.count('ATG'):&#xA;    if list_start[i]&amp;gt;list_stop[n-1]:&#xA;        if list_start[i]&amp;lt;list_stop[n]:&#xA;            print(3*(list_stop[n]-list_start[i])+3) &#xA;#multiply by 3 'cause one item in list stands for 3 nucleotides and add 3 so the whole length of stop codon will be counted&#xA;            j=j+1&#xA;            n=n+1&#xA;            i=i+1&#xA;        else:&#xA;            n=n+1&#xA;    else: i=i+1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For sequence given in the code the ORFs lengths should be as following:&lt;/p&gt;&#xA;&lt;p&gt;Reading frame 1:63; 36; 36; 213; 129&lt;/p&gt;&#xA;&lt;p&gt;Reading frame 2: 39&lt;/p&gt;&#xA;&lt;p&gt;Reading frame 3: 186&lt;/p&gt;&#xA;&lt;p&gt;This code might seem weird as I'm new to python and bioinformatics. I will be very thankful if anyone gives me an advice how can I get rid of the Index Error here.&lt;/p&gt;&#xA;" OwnerUserId="17152" LastEditorUserId="17152" LastEditDate="2023-03-11T20:40:23.997" LastActivityDate="2023-03-11T20:40:23.997" Title="How to tackle an Index Error in my code for finding ORFs" Tags="|python|orf|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20703" PostTypeId="2" ParentId="20702" CreationDate="2023-03-11T15:11:46.827" Score="1" Body="&lt;p&gt;Firstly, there is a bug in the code,&lt;/p&gt;&#xA;&lt;p&gt;Original,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def find_indices(list_to_check, item_to_find):&#xA;    indices = []&#xA;    for idx, value in enumerate(codon):&#xA;        if value == item_to_find:&#xA;            indices.append(idx)&#xA;    return indices&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Corrected,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def find_indices(list_to_check, item_to_find):&#xA;    indices = []&#xA;    for idx, value in enumerate(list_to_check):&#xA;        if value == item_to_find:&#xA;            indices.append(idx)&#xA;    return indices&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You renamed the variable &lt;code&gt;codon&lt;/code&gt; for the function, but then retained its name as &lt;code&gt;codon&lt;/code&gt; within the function. It should be &lt;code&gt;list_to_check&lt;/code&gt;. Also it is convenient.&lt;/p&gt;&#xA;&lt;p&gt;If you are &lt;em&gt;certain&lt;/em&gt; your code is fine, the easiest way to silence the error is via a &lt;code&gt;try&lt;/code&gt; and &lt;code&gt;except&lt;/code&gt; statement. The critical variable is &lt;code&gt;line&lt;/code&gt; and I'm not sure where that is being declared. Without that its difficult to unpick the underlying error, which is an out of range error.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def find_indices(list_to_check, item_to_find):&#xA;        indices = []&#xA;        for idx, value in enumerate(list_to_check):&#xA;            if value == item_to_find:&#xA;                indices.append(idx)&#xA;        return indices&#xA;&#xA;&#xA;if __name__ == '__main__':&#xA;    #line= sample DNA nucleotide sequence&#xA;&#xA;    try:&#xA;        r_frame= input('Enter a reading frame')&#xA;        line = line[int(r_frame)-1:]&#xA;        line=line[int(r_frame)-1:]&#xA;        x=3&#xA;        codon=[line[y-x:y] for y in range(x, len(line)+x,x)]&#xA;        list_start=find_indices(codon, &amp;quot;ATG&amp;quot;)&#xA;        list_stop=find_indices(codon, &amp;quot;TGA&amp;quot;)+find_indices(codon, &amp;quot;TAA&amp;quot;)+find_indices(codon, &amp;quot;TAG&amp;quot;)&#xA;        list_stop.sort()&#xA;        i=0 # variable for indexes in list_start&#xA;        n=0 #variable for indexes in list_stop&#xA;        j=0 # variable that determines numer of times searching for ORF will be repeated&#xA;        if list_start[i]&amp;lt;list_stop[n]:&#xA;            print(3*(list_stop[n]-list_start[i])+3)&#xA;            j=j+1&#xA;            n=n+1&#xA;            i=i+1&#xA;        else:&#xA;            n=n+1&#xA;        while j&amp;lt;codon.count('ATG'):&#xA;            if list_start[i]&amp;gt;list_stop[n-1]:&#xA;                if list_start[i]&amp;lt;list_stop[n]:&#xA;                    print(3*(list_stop[n]-list_start[i])+3)&#xA;                    j=j+1&#xA;                    n=n+1&#xA;                    i=i+1&#xA;                else:&#xA;                    n=n+1&#xA;            else:&#xA;                i = i + 1&#xA;    except AssertionError as error:&#xA;        print (error)&#xA;    except:&#xA;        pass&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you want to see the error you simply deleted the late two lines of code.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;However&lt;/strong&gt;* to resolve the second bug is quite complicated, particularly without a &lt;code&gt;line&lt;/code&gt; declaration. We would simply use is &lt;code&gt;Biopython&lt;/code&gt;'s &lt;code&gt;.translate()&lt;/code&gt; method, we wouldn't do this from first principles. In a past question thats been done.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="298" LastEditDate="2023-03-11T15:12:48.930" LastActivityDate="2023-03-11T15:12:48.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20705" PostTypeId="1" CreationDate="2023-03-11T16:18:23.923" Score="3" ViewCount="23" Body="&lt;p&gt;older releases of PDB used to specify DSSP as the 2ary structure assignment algorithm used in PDB, right now no algorithm is explicitly stated. Where can I find a reference for it in the database?&lt;/p&gt;&#xA;" OwnerUserId="17154" LastActivityDate="2023-03-11T16:18:23.923" Title="which secondary structure assignment algorithm is used by PDB?" Tags="|pdb|secondary-structure|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20706" PostTypeId="2" ParentId="20700" CreationDate="2023-03-11T16:18:58.230" Score="1" Body="&lt;p&gt;If your file doesn't contain the rsIDs, you can use a tool like &lt;a href=&quot;https://grch37.ensembl.org/Tools/VEP&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl's VEP&lt;/a&gt; to annotate the file and get rsIDs as well. Of course, not all variants have an rsID so don't expect to find one for everything reported in your file.&lt;/p&gt;&#xA;&lt;p&gt;More importantly, if these are the raw, unfiltered data from commercial direct-to-consumer sequencing products like 23&amp;amp;me, you need to be aware that they are incredibly untrustworthy. This is why 23&amp;amp;me don't include them in their official report. They only show you the quality results that have been filtered. When working with the raw data, you are very likely to have completely bogus results and will need to have a full VCF file and filter on various quality criteria.&lt;/p&gt;&#xA;&lt;p&gt;I have looked at the raw 23&amp;amp;me data from 4 individuals, and &lt;em&gt;all&lt;/em&gt; of them had extremely rare pathogenic variants reported that would basically have killed them long before they learned to walk. They were clearly false positives.&lt;/p&gt;&#xA;&lt;p&gt;I stress that I am not suggesting these companies are producing crap: none of these variants was reported in the official report of 23&amp;amp;me, they were filtered out. The bad calls are only present when you work on the raw, unfiltered data but if that is what you are working on, you  need to be extremely careful.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-03-11T16:18:58.230" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20707" PostTypeId="2" ParentId="20668" CreationDate="2023-03-11T18:20:12.357" Score="1" Body="&lt;p&gt;Okay, its done. What you are looking for is as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;from pathlib import Path&#xA;&#xA;# Test code    &#xA;def get_all_keys(d):&#xA;    for key, value in d.items():&#xA;        yield key&#xA;        if isinstance(value, dict):&#xA;            yield from get_all_keys(value)&#xA;&#xA;def requestIt(links): &#xA;    cancer_all = requests.get(links)&#xA;    return cancer_all.json() &#xA;&#xA;if __name__ == '__main__':&#xA;&#xA;    path = Path('/PathToDir/test', 'link.html')&#xA;    with open (path, 'r') as fin:&#xA;        links = fin.readlines()&#xA;        &#xA;        &#xA;    data = list(map(requestIt, links))&#xA;&#xA;    # Test code&#xA;    for x in get_all_keys(data[0]['_embedded']['terms'][0]):&#xA;        print(x)&#xA;&#xA;    # Answer code&#xA;    for i,d in enumerate(data):&#xA;        print (f'Link {i + 1} ') &#xA;        for obo_id in d['_embedded']['terms']:&#xA;            print (obo_id['obo_id'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Link 1&lt;/strong&gt;&#xA;NCIT:C8263&#xA;NCIT:C9163&#xA;NCIT:C5630&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Link 2&lt;/strong&gt;&#xA;NCIT:C7175&#xA;NCIT:C68700&#xA;NCIT:C68696&#xA;NCIT:C68697&#xA;NCIT:C68698&#xA;NCIT:C68699&#xA;NCIT:C9155&#xA;NCIT:C9288&#xA;NCIT:C9287&#xA;NCIT:C9019&#xA;NCIT:C9018&#xA;NCIT:C36055&#xA;NCIT:C36058&#xA;NCIT:C36057&#xA;NCIT:C36056&#xA;NCIT:C38377&#xA;NCIT:C122688&#xA;NCIT:C156718&#xA;NCIT:C156719&#xA;NCIT:C169107&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Link 3&lt;/strong&gt;&#xA;NCIT:C7056&#xA;NCIT:C131911&#xA;NCIT:C165799&#xA;NCIT:C68684&#xA;NCIT:C4998&#xA;NCIT:C5089&#xA;NCIT:C5095&#xA;NCIT:C8154&#xA;NCIT:C8604&#xA;NCIT:C127840&#xA;NCIT:C8489&#xA;NCIT:C8852&#xA;NCIT:C69144&#xA;NCIT:C115349&#xA;NCIT:C115367&#xA;NCIT:C165240&#xA;NCIT:C8926&#xA;NCIT:C160230&#xA;NCIT:C160231&#xA;NCIT:C182035&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Where &lt;strong&gt;Link 1&lt;/strong&gt; is ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;https://www.ebi.ac.uk/ols/api/ontologies/ncit/terms/http%253A%252F%252Fpurl.obolibrary.org%252Fobo%252FNCIT_C4861/descendants&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... and so on.&lt;/p&gt;&#xA;&lt;p&gt;If the code is listed &lt;code&gt;# Test code&lt;/code&gt; that is what is used to interrogate the data structure. This doesn't contribute to your answer.&lt;/p&gt;&#xA;&lt;p&gt;However, given the way I've done it, it's important to understand the data structure. Thus output of &lt;code&gt;# Test code&lt;/code&gt; is at the base of this post. If you need another part of the data structure you simply switch &lt;code&gt;id_odo&lt;/code&gt; to whatever else you are looking for (at the base of this post).&lt;/p&gt;&#xA;&lt;p&gt;There will be an easy to use OOP method based out the &lt;code&gt;data&lt;/code&gt; object. I did't want to spend time investigating the object ... the question is specifically about &lt;code&gt;obo_id&lt;/code&gt; so it was easier to go for that.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Test code output&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;iri&#xA;lang&#xA;description&#xA;synonyms&#xA;annotation&#xA;Contributing_Source&#xA;Display_Name&#xA;Legacy Concept Name&#xA;Neoplastic_Status&#xA;Preferred_Name&#xA;Semantic_Type&#xA;UMLS_CUI&#xA;code&#xA;label&#xA;ontology_name&#xA;ontology_prefix&#xA;ontology_iri&#xA;is_obsolete&#xA;term_replaced_by&#xA;is_defining_ontology&#xA;has_children&#xA;is_root&#xA;short_form&#xA;obo_id  # &amp;lt;- its here&#xA;in_subset&#xA;obo_definition_citation&#xA;obo_xref&#xA;obo_synonym&#xA;is_preferred_root&#xA;_links&#xA;self&#xA;href&#xA;parents&#xA;href&#xA;ancestors&#xA;href&#xA;hierarchicalParents&#xA;href&#xA;hierarchicalAncestors&#xA;href&#xA;jstree&#xA;href&#xA;graph&#xA;href&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-03-11T19:41:02.550" LastActivityDate="2023-03-11T19:41:02.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20708" PostTypeId="1" AcceptedAnswerId="20725" CreationDate="2023-03-12T13:26:32.830" Score="0" ViewCount="72" Body="&lt;p&gt;I have multiple sets of files, so I grepped the list from a column in a seperate file such as file1.txt. I want to change all other files by using this file1.txt based on a matching string.&lt;/p&gt;&#xA;&lt;p&gt;for example I am pasting three files here&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;File 1:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr - seq1 NZ_JAHWGH010000010.1 0 60562 green_a4&#xA;chr - seq3 NZ_JAHWGH010000012.1 0 466573 green_a4&#xA;chr - seq5 NZ_JAHWGH010000013.1 0 125526 green_a4&#xA;chr - seq39 NZ_CAJGBF010000006.1 0 211154 green_a4&#xA;chr - seq40 NZ_CAJGBF010000007.1 0 199119 green_a4&#xA;chr - seq41 NZ_CAJGBF010000008.1 0 190934 green_a4&#xA;chr - seq42 NZ_CAJGBF010000009.1 0 178251 green_a4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I extracted the required information using grep; the command is given below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep &amp;quot;NZ_JAHWGH&amp;quot; file.txt |awk '{print &lt;span class=&quot;math-container&quot;&gt;$3}' &amp;gt;file1.txt&#xA;grep &quot;NZ_CAJGBF&quot; file.txt |awk '{print $&lt;/span&gt;3}' &amp;gt;file2.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now I wanted to use this file1.txt and file2.txt to change another files. Output will be like if list in file1.txt matches then all matched lists should be substituted as seq1 similarly using the second file2.txt I want to substitute all matching patterns as seq2.&lt;/p&gt;&#xA;&lt;p&gt;For example the file is given here to be changed by using information from the previous file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seq3 19301 20914 fill_color=green_a0&#xA;seq1 16726 18337 fill_color=green_a0&#xA;seq5 167934 169650 fill_color=green_a0&#xA;seq42 200621 202367 fill_color=red_a0&#xA;seq41 160164 161903 fill_color=green_a0&#xA;seq40 31356 33104 fill_color=green_a0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And final output after the change should be as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seq1 19301 20914 fill_color=green_a0&#xA;seq1 16726 18337 fill_color=green_a0&#xA;seq1 167934 169650 fill_color=green_a0&#xA;seq2 200621 202367 fill_color=red_a0&#xA;seq2 160164 161903 fill_color=green_a0&#xA;seq2 31356 33104 fill_color=green_a0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried using this command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in f2.txt; do grep -wf f1.txt &lt;span class=&quot;math-container&quot;&gt;$i|awk '{$1=&quot;seq1&quot;;print}'; grep -wf other.txt $&lt;/span&gt;i |awk '{$1=&amp;quot;seq2&amp;quot;;print}'; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;2nd attempt:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep -wf f1.txt f2.txt|sed -e 's/seq[0-9]*/seq1/g'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is this correct? If it is correct is there any more efficient way to do it? also it is only printing to the console. I want the output and also changes in the main file. Kindly improve my scripts.&lt;/p&gt;&#xA;&lt;p&gt;I appreciate yout time and help.&lt;/p&gt;&#xA;" OwnerUserId="16745" LastEditorUserId="73" LastEditDate="2023-03-12T19:29:49.567" LastActivityDate="2023-03-15T15:58:01.937" Title="Replace a string in a column of a text file based on matching string listed in another file using linux" Tags="|bash|linux|awk|sed|for-loop|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20709" PostTypeId="1" CreationDate="2023-03-12T15:26:38.673" Score="1" ViewCount="44" Body="&lt;p&gt;I have 1 &lt;code&gt;.bcf&lt;/code&gt; file called with individuals from two groups, one which is my samples, another which is a reference panel. Due to my samples having a limited amount data, when I run PLINK to generate &lt;code&gt;.eigenvec&lt;/code&gt;, none of the samples have any overlap. I believe this is due to the data being skewed. Therefore, I wish to generate a PCA from my reference panel and my samples separately, and then project my samples on top of the reference panel.&lt;/p&gt;&#xA;&lt;p&gt;I tried from the PLINK file from the original &lt;code&gt;.bcf&lt;/code&gt; file to extract the two groups, then identify which genotypes are equally well represented using parameters such as &lt;code&gt;--geno 0.1&lt;/code&gt; to generate SNP's overlapping between the groups. Then I tried the following after reading a lot of different suggestions online, which generates a nice plot with the points projected on top of each other.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;group1_eig &amp;lt;- fread(&amp;quot;Samples.eigenvec&amp;quot;)&#xA;group1_eig &amp;lt;- group1_eig[,3:ncol(group1_eig)]&#xA;&#xA;group1_freq &amp;lt;- fread(&amp;quot;Samples.frq&amp;quot;)&#xA;group1_eval &amp;lt;- fread(&amp;quot;Samples.eigenval&amp;quot;)&#xA;&#xA;# read in the files&#xA;group2_eig &amp;lt;- fread(&amp;quot;RefPanel.eigenvec&amp;quot;)&#xA;group2_eig &amp;lt;- group2_eig[,3:ncol(group2_eig)]&#xA;&#xA;group2_freq &amp;lt;- fread(&amp;quot;RefPanel.frq&amp;quot;)&#xA;group2_eval &amp;lt;- fread(&amp;quot;RefPanel.eigenval&amp;quot;)&#xA;&#xA;freq &amp;lt;- rbind(group1_freq, group2_freq)&#xA;eval &amp;lt;- rbind(group1_eval, group2_eval)&#xA;eig &amp;lt;- rbind(group1_eig, group2_eig)&#xA;pca &amp;lt;- prcomp(eig[,3:ncol(eig)], scale = TRUE)&#xA;group1_pca &amp;lt;- predict(pca, newdata = group1_eig[,3:ncol(group1_eig)], &#xA;                      type = &amp;quot;scores&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is this approach acceptable? What other approaches work? I have read you can also normalise one group using the square root of the eigenvalues for the other group. Or potentially use the MAF (&lt;code&gt;.frq&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="8630" LastEditorUserId="-1" LastEditDate="2023-04-22T19:14:50.017" LastActivityDate="2023-04-22T19:14:50.017" Title="Project PLINK eigenvector from one group on top of another group" Tags="|snp|plink|pca|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20710" PostTypeId="2" ParentId="20698" CreationDate="2023-03-12T15:35:04.273" Score="0" Body="&lt;p&gt;I am not actively following this field, so take my answer with a grain of salt.&lt;/p&gt;&#xA;&lt;p&gt;With bulk data, you can't easily tell how subclones differ from each other or characterize the mutation rates in subclones. It is also challenging to get accurate somatic mutation calls especially when you don't have the matched normal.&lt;/p&gt;&#xA;&lt;p&gt;With single-cell data, you usually get more accurate point mutation calls in 20-80% of the genome, depending on the amplification method. You can estimate the mutation rate and call large CNVs more accurately in each cell. Sometimes you can even reconstruct the evolution of subclones. I would say single-cell data is better than bulk data all around. The only problem is that the single-cell approach is more costly as you need to sequence many cells to see the pattern.&lt;/p&gt;&#xA;&lt;p&gt;On a side note, MDA is not that accurate for calling point mutations. This may not be a problem if the mutation rate is much higher than the error rate.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2023-03-12T15:35:04.273" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20711" PostTypeId="1" CreationDate="2023-03-12T18:50:33.733" Score="1" ViewCount="21" Body="&lt;p&gt;I'm trying to get all C. elegans genes from Wormbase programmatically, as well as all of their alternative gene names. To do this, I'm using &lt;a href=&quot;http://intermine.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;intermine&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I was able to obtain all genes, but not ALL alternative names. The code below retrieves the genes' primary identifiers, and their &amp;quot;secondary identifiers.&amp;quot;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from intermine.webservice import Service&#xA;service = Service(&amp;quot;http://intermine.wormbase.org/tools/wormmine/service&amp;quot;)&#xA;&#xA;query=service.new_query(&amp;quot;Gene&amp;quot;)&#xA;# 6239 is the taxonId for C. elegans&#xA;query.add_constraint(&amp;quot;organism.taxonId&amp;quot;,&amp;quot;=&amp;quot;,&amp;quot;6239&amp;quot;)&#xA;query.select(&amp;quot;*&amp;quot;)&#xA;&#xA;all_wormbase_genes_symbol = list()&#xA;all_wormbase_genes_sID = list()&#xA;&#xA;for row in query.rows(start=0):&#xA;    # get primary symbol/ID&#xA;    all_wormbase_genes_symbol.append(row[&amp;quot;symbol&amp;quot;])&#xA;    # get secondary ID&#xA;    all_wormbase_genes_sID.append(row[&amp;quot;secondaryIdentifier&amp;quot;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried to look for other keys in &lt;code&gt;row&lt;/code&gt; that might contain this information by first retrieving the list of all possible keys:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;k = set()&#xA;for row in query.rows(start=0):&#xA;    k.update(row.keys())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but none have what I want:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{'Gene.automatedDescription',&#xA; 'Gene.biotype',&#xA; 'Gene.briefDescription',&#xA; 'Gene.description',&#xA; 'Gene.geneticPosition',&#xA; 'Gene.id',&#xA; 'Gene.lastUpdated',&#xA; 'Gene.length',&#xA; 'Gene.name',&#xA; 'Gene.operon',&#xA; 'Gene.primaryIdentifier',&#xA; 'Gene.score',&#xA; 'Gene.scoreType',&#xA; 'Gene.secondaryIdentifier',&#xA; 'Gene.symbol'}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="8601" LastActivityDate="2023-03-12T18:50:33.733" Title="How can I programmatically retrieve all alternative gene names from Wormbase?" Tags="|wormbase|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20713" PostTypeId="1" CreationDate="2023-03-13T13:35:14.063" Score="2" ViewCount="1055" Body="&lt;p&gt;I recently used Dante to obtain a WGS and have downloaded several files produced during this process. There is a file that ends '.vcf' which I did think was a VCF file but I'm not sure now after a previous question. Part of that file is included below. Its about 1.1GB in size.&lt;/p&gt;&#xA;&lt;p&gt;It appears to me to follow VCF format at least to a degree but it is missing the rsid's. The field where an rsid would usually be in a VCF file is replaced with a period (.).&lt;/p&gt;&#xA;&lt;p&gt;I would like to backfill (annotate?) this file with known rsid's (where defined). There are nearly 4 million of them and online tools don't seem ideal, I would prefer to be able to do it locally, if that is possible. I am not looking for software to do this, but a data file which specifies which rsid is assigned to a particular chr:pos:allele. That may or may not be possible - perhaps I misunderstand and its far more complex than that. But from what I can tell, for a particular reference genome, and a particular build of dbsnp, there is a simple relationship as described.&lt;/p&gt;&#xA;&lt;p&gt;I want to do this, for instance, to be able to use various tools, like SNPedia. Dante appears to use GRCh37, vs SNPedia using mainly GRCh38, which I can see is an issue, and will probably require me to buy a GRCh38 conversion from Dante (or do it myself). But please ignore that particular issue right now.&lt;/p&gt;&#xA;&lt;p&gt;I don't quite understand what the dependencies are. I've seen comments that suggest that reference genome is not important to generating rsid's from VCF files and that only dbsnp build number needs to match the environment that the VCF file was generated in, but that does not seem to make much sense to me. From what I can see, if you are converting chr:pos information to rsid's, you need to be using the right reference genome. Does dbsnp build really matter? Should you not use the latest build, if it is continually being added to - surely rsid's are not reused? And then &lt;a href=&quot;https://annovar.openbioinformatics.org/en/latest/articles/dbSNP/&quot; rel=&quot;nofollow noreferrer&quot;&gt;a post on the ANNOVAR site&lt;/a&gt; seems to suggest that no one really knows what rsid's are and that to different people they are different things, further muddying the waters.&lt;/p&gt;&#xA;&lt;p&gt;So in terms of question - is there a file, downloadable from somewhere, that contains mappings from chr:pos to rsid, and if so, how does one select the correct file for a particular dataset (IE generated with a particular reference genome, with a particular dbsnp, to be used with references that have used a particular reference/dbsnp build like GRCh38 / dbsnp build 151)?&lt;/p&gt;&#xA;&lt;p&gt;This file (header first), downloaded from Dante, is named .filtered.snp.vcf&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    ##fileformat=VCFv4.2&#xA;    ##FILTER=&amp;lt;ID=PASS,Description=&amp;quot;All filters passed&amp;quot;&amp;gt;&#xA;    ##FORMAT=&amp;lt;ID=AD,Number=R,Type=Integer,Description=&amp;quot;Allelic depths (counting only informative reads out of the total reads) for the ref and alt alleles in the order listed&amp;quot;&amp;gt;&#xA;    ##FORMAT=&amp;lt;ID=AF,Number=A,Type=Float,Description=&amp;quot;Allele fractions for alt alleles in the order listed&amp;quot;&amp;gt;&#xA;    ##FORMAT=&amp;lt;ID=DP,Number=1,Type=Integer,Description=&amp;quot;Approximate read depth (reads with MQ=255 or with bad mates are filtered)&amp;quot;&amp;gt;&#xA;    ##FORMAT=&amp;lt;ID=F1R2,Number=R,Type=Integer,Description=&amp;quot;Count of reads in F1R2 pair orientation supporting each allele&amp;quot;&amp;gt;&#xA;    ##FORMAT=&amp;lt;ID=F2R1,Number=R,Type=Integer,Description=&amp;quot;Count of reads in F2R1 pair orientation supporting each allele&amp;quot;&amp;gt;&#xA;    ##FORMAT=&amp;lt;ID=GP,Number=G,Type=Float,Description=&amp;quot;Phred-scaled posterior probabilities for genotypes as defined in the VCF specification&amp;quot;&amp;gt;&#xA;    ##FORMAT=&amp;lt;ID=GQ,Number=1,Type=Integer,Description=&amp;quot;Genotype Quality&amp;quot;&amp;gt;&#xA;    ##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&amp;quot;Genotype&amp;quot;&amp;gt;&#xA;    ##FORMAT=&amp;lt;ID=MB,Number=4,Type=Integer,Description=&amp;quot;Per-sample component statistics to detect mate bias&amp;quot;&amp;gt;&#xA;    ##FORMAT=&amp;lt;ID=PL,Number=G,Type=Integer,Description=&amp;quot;Normalized, Phred-scaled likelihoods for genotypes as defined in the VCF specification&amp;quot;&amp;gt;&#xA;    ##FORMAT=&amp;lt;ID=PRI,Number=G,Type=Float,Description=&amp;quot;Phred-scaled prior probabilities for genotypes&amp;quot;&amp;gt;&#xA;    ##FORMAT=&amp;lt;ID=PS,Number=1,Type=Integer,Description=&amp;quot;Physical phasing ID information, where each unique ID within a given sample (but not across samples) connects records within a phasing group&amp;quot;&amp;gt;&#xA;    ##FORMAT=&amp;lt;ID=SB,Number=4,Type=Integer,Description=&amp;quot;Per-sample component statistics which comprise the Fisher's Exact Test to detect strand bias&amp;quot;&amp;gt;&#xA;    ##FORMAT=&amp;lt;ID=SQ,Number=A,Type=Float,Description=&amp;quot;Somatic quality&amp;quot;&amp;gt;&#xA;    ##DRAGENCommandLine=&amp;lt;ID=HashTableBuild,Version=&amp;quot;SW: 01.003.044.3.9.5, HashTableVersion: 8&amp;quot;,CommandLineOptions=&amp;quot;dragen --build-hash-table true --output-directory grch37 --ht-reference grch37.fasta&amp;quot;&amp;gt;&#xA;    ##DRAGENCommandLine=&amp;lt;ID=dragen,Version=&amp;quot;SW: 05.121.645.4.0.3, HW: 05.121.645&amp;quot;,Date=&amp;quot;Thu Mar 02 09:58:00 UTC 2023&amp;quot;,CommandLineOptions=&amp;quot;-f -c output.cfg -r /references/grch37&amp;quot;&amp;gt;&#xA;    ##INFO=&amp;lt;ID=AC,Number=A,Type=Integer,Description=&amp;quot;Allele count in genotypes, for each ALT allele, in the same order as listed&amp;quot;&amp;gt;&#xA;    ##INFO=&amp;lt;ID=AF,Number=A,Type=Float,Description=&amp;quot;Allele Frequency, for each ALT allele, in the same order as listed&amp;quot;&amp;gt;&#xA;    ##INFO=&amp;lt;ID=AN,Number=1,Type=Integer,Description=&amp;quot;Total number of alleles in called genotypes&amp;quot;&amp;gt;&#xA;    ##INFO=&amp;lt;ID=FS,Number=1,Type=Float,Description=&amp;quot;Phred-scaled p-value using Fisher's exact test to detect strand bias&amp;quot;&amp;gt;&#xA;    ##INFO=&amp;lt;ID=QD,Number=1,Type=Float,Description=&amp;quot;Variant Confidence/Quality by Depth&amp;quot;&amp;gt;&#xA;    ##INFO=&amp;lt;ID=SOR,Number=1,Type=Float,Description=&amp;quot;Symmetric Odds Ratio of 2x2 contingency table to detect strand bias&amp;quot;&amp;gt;&#xA;    ##INFO=&amp;lt;ID=DP,Number=1,Type=Integer,Description=&amp;quot;Approximate read depth (informative and non-informative); some reads may have been filtered based on mapq etc.&amp;quot;&amp;gt;&#xA;    ##INFO=&amp;lt;ID=FractionInformativeReads,Number=1,Type=Float,Description=&amp;quot;The fraction of informative reads out of the total reads&amp;quot;&amp;gt;&#xA;    ##INFO=&amp;lt;ID=MQ,Number=1,Type=Float,Description=&amp;quot;RMS Mapping Quality&amp;quot;&amp;gt;&#xA;    ##INFO=&amp;lt;ID=MQRankSum,Number=1,Type=Float,Description=&amp;quot;Z-score From Wilcoxon rank sum test of Alt vs. Ref read mapping qualities&amp;quot;&amp;gt;&#xA;    ##INFO=&amp;lt;ID=ReadPosRankSum,Number=1,Type=Float,Description=&amp;quot;Z-score from Wilcoxon rank sum test of Alt vs. Ref read position bias&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=DRAGENSnpHardQUAL,Description=&amp;quot;Set if true:QUAL &amp;lt; 3&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=DRAGENIndelHardQUAL,Description=&amp;quot;Set if true:QUAL &amp;lt; 3&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=LowDepth,Description=&amp;quot;Set if true:DP &amp;lt;= 1&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=PloidyConflict,Description=&amp;quot;Genotype call from variant caller not consistent with chromosome ploidy&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=base_quality,Description=&amp;quot;Site filtered because median base quality of alt reads at this locus does not meet threshold&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=filtered_reads,Description=&amp;quot;Site filtered because too large a fraction of reads have been filtered out&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=fragment_length,Description=&amp;quot;Site filtered because absolute difference between the median fragment length of alt reads and median fragment length of ref reads at this locus exceeds threshold&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=low_af,Description=&amp;quot;Allele frequency does not meet threshold&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=low_depth,Description=&amp;quot;Site filtered because the read depth is too low&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=low_frac_info_reads,Description=&amp;quot;Site filtered because the fraction of informative reads is below threshold&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=low_normal_depth,Description=&amp;quot;Site filtered because the normal sample read depth is too low&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=long_indel,Description=&amp;quot;Site filtered because the indel length is too long&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=mapping_quality,Description=&amp;quot;Site filtered because median mapping quality of alt reads at this locus does not meet threshold&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=multiallelic,Description=&amp;quot;Site filtered because more than two alt alleles pass tumor LOD&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=non_homref_normal,Description=&amp;quot;Site filtered because the normal sample genotype is not homozygous reference&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=no_reliable_supporting_read,Description=&amp;quot;Site filtered because no reliable supporting somatic read exists&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=panel_of_normals,Description=&amp;quot;Seen in at least one sample in the panel of normals vcf&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=read_position,Description=&amp;quot;Site filtered because median of distances between start/end of read and this locus is below threshold&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=RMxNRepeatRegion,Description=&amp;quot;Site filtered because all or part of the variant allele is a repeat of the reference&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=str_contraction,Description=&amp;quot;Site filtered due to suspected PCR error where the alt allele is one repeat unit less than the reference&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=too_few_supporting_reads,Description=&amp;quot;Site filtered because there are too few supporting reads in the tumor sample&amp;quot;&amp;gt;&#xA;    ##FILTER=&amp;lt;ID=weak_evidence,Description=&amp;quot;Somatic variant score does not meet threshold&amp;quot;&amp;gt;&#xA;    ##ML=PackageVersion=4,ModelVersion=4,ModelCRC=2921457924&#xA;    ##contig=&amp;lt;ID=1,length=249250621&amp;gt;&#xA;    ##contig=&amp;lt;ID=2,length=243199373&amp;gt;&#xA;    ##contig=&amp;lt;ID=3,length=198022430&amp;gt;&#xA;    ##contig=&amp;lt;ID=4,length=191154276&amp;gt;&#xA;    ##contig=&amp;lt;ID=5,length=180915260&amp;gt;&#xA;    ##contig=&amp;lt;ID=6,length=171115067&amp;gt;&#xA;    ##contig=&amp;lt;ID=7,length=159138663&amp;gt;&#xA;    ##contig=&amp;lt;ID=8,length=146364022&amp;gt;&#xA;    ##contig=&amp;lt;ID=9,length=141213431&amp;gt;&#xA;    ##contig=&amp;lt;ID=10,length=135534747&amp;gt;&#xA;    ##contig=&amp;lt;ID=11,length=135006516&amp;gt;&#xA;    ##contig=&amp;lt;ID=12,length=133851895&amp;gt;&#xA;    ##contig=&amp;lt;ID=13,length=115169878&amp;gt;&#xA;    ##contig=&amp;lt;ID=14,length=107349540&amp;gt;&#xA;    ##contig=&amp;lt;ID=15,length=102531392&amp;gt;&#xA;    ##contig=&amp;lt;ID=16,length=90354753&amp;gt;&#xA;    ##contig=&amp;lt;ID=17,length=81195210&amp;gt;&#xA;    ##contig=&amp;lt;ID=18,length=78077248&amp;gt;&#xA;    ##contig=&amp;lt;ID=19,length=59128983&amp;gt;&#xA;    ##contig=&amp;lt;ID=20,length=63025520&amp;gt;&#xA;    ##contig=&amp;lt;ID=21,length=48129895&amp;gt;&#xA;    ##contig=&amp;lt;ID=22,length=51304566&amp;gt;&#xA;    ##contig=&amp;lt;ID=X,length=155270560&amp;gt;&#xA;    ##contig=&amp;lt;ID=Y,length=59373566&amp;gt;&#xA;    ##contig=&amp;lt;ID=MT,length=16569&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000207.1,length=4262&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000226.1,length=15008&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000229.1,length=19913&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000231.1,length=27386&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000210.1,length=27682&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000239.1,length=33824&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000235.1,length=34474&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000201.1,length=36148&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000247.1,length=36422&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000245.1,length=36651&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000197.1,length=37175&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000203.1,length=37498&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000246.1,length=38154&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000249.1,length=38502&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000196.1,length=38914&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000248.1,length=39786&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000244.1,length=39929&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000238.1,length=39939&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000202.1,length=40103&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000234.1,length=40531&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000232.1,length=40652&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000206.1,length=41001&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000240.1,length=41933&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000236.1,length=41934&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000241.1,length=42152&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000243.1,length=43341&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000242.1,length=43523&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000230.1,length=43691&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000237.1,length=45867&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000233.1,length=45941&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000204.1,length=81310&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000198.1,length=90085&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000208.1,length=92689&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000191.1,length=106433&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000227.1,length=128374&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000228.1,length=129120&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000214.1,length=137718&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000221.1,length=155397&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000209.1,length=159169&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000218.1,length=161147&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000220.1,length=161802&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000213.1,length=164239&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000211.1,length=166566&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000199.1,length=169874&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000217.1,length=172149&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000216.1,length=172294&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000215.1,length=172545&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000205.1,length=174588&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000219.1,length=179198&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000224.1,length=179693&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000223.1,length=180455&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000195.1,length=182896&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000212.1,length=186858&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000222.1,length=186861&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000200.1,length=187035&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000193.1,length=189789&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000194.1,length=191469&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000225.1,length=211173&amp;gt;&#xA;    ##contig=&amp;lt;ID=GL000192.1,length=547496&amp;gt;&#xA;    ##contig=&amp;lt;ID=NC_007605,length=171823&amp;gt;&#xA;    ##contig=&amp;lt;ID=hs37d5,length=35477943&amp;gt;&#xA;    ##reference=file:///references/grch37/reference.bin&#xA;    ##bcftools_filterVersion=1.9+htslib-1.9&#xA;    ##bcftools_filterCommand=filter -i 'TYPE=&amp;quot;snp&amp;quot; ' /processing/sample.raw.vcf.gz; Date=Thu Mar  2 10:28:45 2023&#xA;    #CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  GFX0xxxxxx&#xA;    1   12783   .   G   A   6.16    PASS    AC=1;AF=0.5;AN=2;DP=37;FS=0;MQ=11.6;MQRankSum=-2.137;QD=0.45;ReadPosRankSum=2.878;SOR=1.949;FractionInformativeReads=0.973  GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    0/1:10,26:0.722:36:4,14:6,12:4:38,0,1:6.1614,2.5231,7.0198:0,34.77,37.77:10,0,26,0:4,6,14,12&#xA;    1   13868   .   A   G   4.69    PASS    AC=1;AF=0.5;AN=2;DP=61;FS=2.275;MQ=16.39;MQRankSum=2.368;QD=0.36;ReadPosRankSum=-1.078;SOR=1.136;FractionInformativeReads=1 GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    0/1:36,25:0.41:61:16,15:20,10:4:37,0,8:4.6918,2.1112,13.42:0,34.77,37.78:15,21,8,17:23,13,12,13&#xA;    1   13980   .   T   C   23.14   PASS    AC=2;AF=1;AN=2;DP=2;FS=0;MQ=5.7;MQRankSum=0;QD=6.5;ReadPosRankSum=0;SOR=2.303;FractionInformativeReads=1    GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    1/1:0,2:1:2:0,1:0,1:19:61,24,0:23.143,20.887,0.056833:0,34.77,37.77:0,0,2,0:0,0,1,1&#xA;    1   14464   .   A   T   14.11   PASS    AC=1;AF=0.5;AN=2;DP=48;FS=7.243;MQ=22.38;MQRankSum=3.14;QD=0.86;ReadPosRankSum=2.527;SOR=0.267;FractionInformativeReads=1   GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    0/1:11,37:0.771:48:8,16:3,21:7:77,6,0:14.113,0.98271,7.8591:0,34.77,37.77:3,8,19,18:4,7,20,17&#xA;    1   14907   .   A   G   3.67    PASS    AC=1;AF=0.5;AN=2;DP=77;FS=0.893;MQ=26.78;MQRankSum=-0.058;QD=0.41;ReadPosRankSum=5.956;SOR=0.532;FractionInformativeReads=1 GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    0/1:48,29:0.377:77:31,13:17,16:4:36,0,22:3.6675,2.453,27.555:0,34.77,37.77:27,21,15,14:28,20,15,14&#xA;    1   14930   .   A   G   3.26    PASS    AC=1;AF=0.5;AN=2;DP=82;FS=3.051;MQ=27.04;MQRankSum=-0.363;QD=0.37;ReadPosRankSum=-0.277;SOR=0.754;FractionInformativeReads=1    GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    0/1:51,31:0.378:82:30,16:21,15:3:35,0,21:3.2627,2.7897,26.606:0,34.77,37.77:28,23,14,17:29,22,17,14&#xA;    1   15211   .   T   G   6.66    PASS    AC=1;AF=0.5;AN=2;DP=70;FS=0;MQ=21.21;MQRankSum=0.533;QD=0.49;ReadPosRankSum=-0.905;SOR=0.743;FractionInformativeReads=1 GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    0/1:28,42:0.6:70:12,22:16,20:6:40,0,11:6.6596,1.2235,15.27:0,34.77,37.77:15,13,23,19:15,13,17,25&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then a couple of chunks from random bits of the file:&lt;/p&gt;&#xA;&lt;p&gt;offset 100000 lines:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    1   82279586    .   T   C   62.12   PASS    AC=2;AF=1;AN=2;DP=25;FS=0;MQ=250;MQRankSum=0;QD=4.89;ReadPosRankSum=0;SOR=0.941;FractionInformativeReads=1GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB  1/1:0,25:1:25:0,10:0,15:52:100,55,0:62.124,52.173,2.8992e-05:0,34.77,37.77:0,0,11,14:0,0,10,15&#xA;    1   82280153    .   G   A   62.73   PASS    AC=2;AF=1;AN=2;DP=31;FS=0;MQ=250;MQRankSum=0;QD=4.53;ReadPosRankSum=0;SOR=1.044;FractionInformativeReads=1GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB  1/1:0,31:1:31:0,15:0,16:57:101,61,0:62.731,57.663,9.319e-06:0,34.77,37.77:0,0,18,13:0,0,14,17&#xA;    1   82280683    .   C   T   54.83   PASS    AC=2;AF=1;AN=2;DP=17;FS=0;MQ=250;MQRankSum=0;QD=5.77;ReadPosRankSum=0;SOR=1.061;FractionInformativeReads=1GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB  1/1:0,17:1:17:0,8:0,9:45:93,49,0:54.832,45.914,0.00012581:0,34.77,37.77:0,0,10,7:0,0,11,6&#xA;    1   82280870    .   T   C   41.24   PASS    AC=1;AF=0.5;AN=2;DP=19;FS=1.808;MQ=250;MQRankSum=3.592;QD=2.58;ReadPosRankSum=1.858;SOR=1.14;FractionInformativeReads=1 GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    0/1:11,8:0.421:19:8,2:3,6:40:76,0,45:41.236,0.00039866,47.817:0,34.77,37.77:4,7,2,6:5,6,6,2&#xA;    1   82280975    .   A   T   40.52   PASS    AC=1;AF=0.5;AN=2;DP=24;FS=1.569;MQ=250;MQRankSum=4.07;QD=2.02;ReadPosRankSum=1.025;SOR=1.085;FractionInformativeReads=1 GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    0/1:14,10:0.417:24:5,8:9,2:40:75,0,50:40.524,0.00040643,53.142:0,34.77,37.77:7,7,4,6:8,6,4,6&#xA;    1   82281492    .   T   C   44.14   PASS    AC=1;AF=0.5;AN=2;DP=22;FS=6.923;MQ=250;MQRankSum=3.873;QD=2.2;ReadPosRankSum=-0.334;SOR=2.064;FractionInformativeReads=1    GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    0/1:13,9:0.409:22:9,4:4,5:44:79,0,51:44.143,0.00018328,54.3:0,34.77,37.77:6,7,7,2:6,7,6,3&#xA;    1   82282062    .   T   C   8.34    PASS    AC=1;AF=0.5;AN=2;DP=26;FS=1.974;MQ=247.84;MQRankSum=3.383;QD=1.04;ReadPosRankSum=-1.561;SOR=0.863;FractionInformativeReads=1    GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    0/1:21,5:0.192:26:9,3:12,2:8:42,0,29:8.3401,0.69098,32.665:0,34.77,37.77:9,12,3,2:9,12,1,4&#xA;    1   82282378    .   C   T   42.61   PASS    AC=1;AF=0.5;AN=2;DP=24;FS=1.685;MQ=250;MQRankSum=4.07;QD=2.02;ReadPosRankSum=2.898;SOR=1.23;FractionInformativeReads=1  GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    0/1:14,10:0.417:24:8,6:6,4:43:77,0,57:42.606,0.0002423,60.191:0,34.77,37.77:8,6,7,3:10,4,7,3&#xA;    1   82282788    .   C   T   48.99   PASS    AC=1;AF=0.5;AN=2;DP=30;FS=0;MQ=245.71;MQRankSum=2.889;QD=1.67;ReadPosRankSum=1.975;SOR=0.914;FractionInformativeReads=1 GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    0/1:14,16:0.533:30:6,9:8,7:46:84,0,46:48.985,0.00010665,49.217:0,34.77,37.77:8,6,10,6:6,8,7,9&#xA;    1   82282947    .   G   T   47.31   PASS    AC=1;AF=0.5;AN=2;DP=32;FS=10.606;MQ=250;MQRankSum=4.796;QD=1.56;ReadPosRankSum=0.68;SOR=0.725;FractionInformativeReads=1    GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    0/1:15,17:0.531:32:6,8:9,9:46:82,0,48:47.312,0.00011286,51.287:0,34.77,37.77:11,4,7,10:8,7,8,9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;offset 2000000 lines:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    8   142260995   .   T   C   65.76   PASS    AC=2;AF=1;AN=2;DP=28;FS=0;MQ=248.93;MQRankSum=0;QD=4.69;ReadPosRankSum=0;SOR=1.179;FractionInformativeReads=1   GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    1/1:0,28:1:28:0,14:0,14:53:104,56,0:65.759,52.946,2.3297e-05:0,34.77,37.77:0,0,11,17:0,0,13,15&#xA;    8   142261389   .   T   C   66.03   PASS    AC=2;AF=1;AN=2;DP=28;FS=0;MQ=250;MQRankSum=0;QD=4.67;ReadPosRankSum=0;SOR=0.997;FractionInformativeReads=1GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB  1/1:0,28:1:28:0,12:0,16:54:104,57,0:66.026,54.004,1.812e-05:0,34.77,37.77:0,0,16,12:0,0,14,14&#xA;    8   142261541   .   G   A   65.34   PASS    AC=2;AF=1;AN=2;DP=30;FS=0;MQ=250;MQRankSum=0;QD=4.51;ReadPosRankSum=0;SOR=1.765;FractionInformativeReads=1GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB  1/1:0,30:1:30:0,14:0,16:52:103,55,0:65.339,52.429,2.5886e-05:0,34.77,37.77:0,0,9,21:0,0,17,13&#xA;    8   142261618   .   A   G   64.46   PASS    AC=2;AF=1;AN=2;DP=27;FS=0;MQ=250;MQRankSum=0;QD=4.57;ReadPosRankSum=0;SOR=1.302;FractionInformativeReads=1GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB:PS   1|1:0,27:1:27:0,14:0,13:53:102,56,0:64.465,53.434,2.1227e-05:0,34.77,37.78:0,0,10,17:0,0,15,12:142261618&#xA;    8   142261621   .   C   T   60.33   PASS    AC=2;AF=1;AN=2;DP=25;FS=0;MQ=250;MQRankSum=0;QD=4.7;ReadPosRankSum=0;SOR=1.051;FractionInformativeReads=0.96    GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB:PS 1|1:0,24:1:24:0,13:0,11:48:98,51,0:60.326,48.044,7.2481e-05:0,34.77,37.78:0,0,10,14:0,0,13,11:142261618&#xA;    8   142261967   .   C   T   68.75   PASS    AC=2;AF=1;AN=2;DP=37;FS=0;MQ=250;MQRankSum=0;QD=4.26;ReadPosRankSum=0;SOR=0.859;FractionInformativeReads=1GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB  1/1:0,37:1:37:0,17:0,20:56:107,60,0:68.745,56.755,9.8367e-06:0,34.77,37.77:0,0,20,17:0,0,18,19&#xA;    8   142262327   .   A   G   68.48   PASS    AC=2;AF=1;AN=2;DP=42;FS=0;MQ=250;MQRankSum=0;QD=4.13;ReadPosRankSum=0;SOR=0.788;FractionInformativeReads=1GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB  1/1:0,42:1:42:0,21:0,21:56:106,60,0:68.476,56.786,9.8367e-06:0,34.77,37.77:0,0,22,20:0,0,11,31&#xA;    8   142262413   .   G   A   62.88   PASS    AC=2;AF=1;AN=2;DP=27;FS=0;MQ=231.86;MQRankSum=0;QD=4.75;ReadPosRankSum=0;SOR=1.302;FractionInformativeReads=1   GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    1/1:0,27:1:27:0,16:0,11:51:101,54,0:62.881,51.183,3.5205e-05:0,34.77,37.77:0,0,10,17:0,0,9,18&#xA;    8   142263074   .   G   A   51.24   PASS    AC=2;AF=1;AN=2;DP=20;FS=0;MQ=143.27;MQRankSum=0;QD=5.31;ReadPosRankSum=0;SOR=0.892;FractionInformativeReads=1   GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    1/1:0,20:1:20:0,9:0,11:36:89,39,0:51.237,35.702,0.0012015:0,34.77,37.77:0,0,11,9:0,0,10,10&#xA;    8   142263171   .   C   T   52.9    PASS    AC=2;AF=1;AN=2;DP=22;FS=0;MQ=122.2;MQRankSum=0;QD=4.81;ReadPosRankSum=0;SOR=0.693;FractionInformativeReads=0.909    GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    1/1:0,20:1:20:0,11:0,9:35:91,38,0:52.899,34.894,0.0014297:0,34.77,37.77:0,0,10,10:0,0,9,11&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17155" LastActivityDate="2023-08-21T05:44:18.753" Title="How to generate rsid values locally from VCF file" Tags="|vcf|rsid|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20714" PostTypeId="2" ParentId="20713" CreationDate="2023-03-13T18:36:20.603" Score="3" Body="&lt;p&gt;Download the dbSNP VCF file (latest version) and map to your input VCF using &lt;code&gt;bcftools annotate&lt;/code&gt;. You'll want to match reference sequence version (hg19/hg38) and use the latest version of dbSNP to get the most up to date information.&lt;/p&gt;&#xA;&lt;p&gt;dbSNP VCF FTP folder: &lt;a href=&quot;https://ftp.ncbi.nih.gov/snp/latest_release/VCF/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nih.gov/snp/latest_release/VCF/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-03-13T18:36:20.603" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20715" PostTypeId="2" ParentId="20713" CreationDate="2023-03-13T20:02:10.413" Score="2" Body="&lt;p&gt;As a partial answer to the specific questions about definitions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;rsIDs define a specific location and type of variation*, but the location of that variation can change within different reference genomes. This may include a change in chromosome, as well as a change in position.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Different dbSNP builds can add additional variation, but the same rsID will &lt;em&gt;always&lt;/em&gt; refer to the same variant in &lt;em&gt;all&lt;/em&gt; dbSNP builds that refer to that variation.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;rsIDs can be invalidated in new dbSNP builds for various reasons. It's a good idea to always use the latest dbSNP build when annotating variants.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;rsIDs are never recycled. There is a prescribed process for merging and splitting variants that ensures backwards compatibility. If a variation is merged, I think the approach is to always define the earliest-numbered variant as the primary variation. If a variation is split, I think the approach is to use a new rsID for each of the separate variations.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;* by &amp;quot;variation&amp;quot;, I'm referring to a type of modification in the DNA sequence, rather than a specific instance of that modification. This is explained in the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/docs/RefSNP_about/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbSNP documentation&lt;/a&gt; (an &amp;quot;ss&amp;quot; is a submitted variant, a submission made to dbSNP):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Each ss that maps to the same position and is the same variant type is assigned to an existing RefSNP (rs) or assigned a new one.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Although that isn't entirely clear, this suggests that if a new variant is submitted that falls at the same position of an exiting one and is of the same type as the existing one, then the new one may simply be added to the same rsID as the existing one. So, for example, if we have a single nucleotide G =&amp;gt; C variant at position N, and a new G =&amp;gt; T variant is reported at the same position, then the new one could have the same rsID as the old one.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;In response to an additional comment / question:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;a &lt;a href=&quot;https://annovar.openbioinformatics.org/en/latest/articles/dbSNP/&quot; rel=&quot;nofollow noreferrer&quot;&gt;post&lt;/a&gt; on the ANNOVAR site seems to suggest that no one really knows what rsid's are and that to different people they are different things, further muddying the waters.... Is this guy an authority on this or not?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I can't comment on authority for that guy; the authorship is unclear to me. What matters is how the dbSNP people understand dbSNP (i.e. the documentation link above), because they are the authority on it.&lt;/p&gt;&#xA;&lt;p&gt;My understanding of it was helped because many years ago I attended a close-knit workshop that included a talk by Mike Feolo, and I had been working on a human dataset with many oddities, so I was able to ask lots of questions about corner cases. Here's the text from a section on rsIDs that I added into the appendix of &lt;a href=&quot;https://researcharchive.vuw.ac.nz/handle/10063/1987&quot; rel=&quot;nofollow noreferrer&quot;&gt;my thesis&lt;/a&gt;; things have probably changed since then, but I expect the broad strokes are still correct:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Mike Feolo gave a talk at the [Working With The HapMap] Cambridge course on the dbSNP database. He talked about how new Single Nucleotide Polymorphisms (SNPs) were submitted, compared against currently existing SNPs, and merged if they were already in the database. The database RefSeq(rs) numbers are updated about twice a year, and whenever there is a new build of the human genome sequence. As a result of these builds, some rs numbers are merged to account for increased knowledge of mutations, a process that I was not aware of. This has an impact on my research, because I have received data from an Illumina 317k SNPchip, which I compare to the HapMap dataset. The rs numbers from the Illumina chip are for a specific dbSNP build, which may not be fully consistent with the most recent build (from which the HapMap data is retrieved). Mike has recommended that both the build number and the rs number are used when referring to a specific mutation. Sharma Buch pointed out that it was possible to ask Illumina for information about the merged rs numbers for their chips, which would allow datasets from different builds to be used together.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-03-15T02:15:53.657" LastActivityDate="2023-03-15T02:15:53.657" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20716" PostTypeId="1" CreationDate="2023-03-14T12:34:19.723" Score="1" ViewCount="144" Body="&lt;p&gt;I have a list of SMILES of small molecules and I want to be able to simulate these molecules with an Amber force field in Python. Currently, I use RDkit to convert the smile into a PDB file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from rdkit import Chem&#xA;from rdkit.Chem import AllChem &#xA;&#xA;molecule = Chem.MolFromSMiles('smile')&#xA;molecule = Chem.AddHs(molecule)&#xA;AllChem.EmbedMolecule(molecule)&#xA;Chem.MolToPDBFile(molecule, 'molecule.pdb')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then I use the ParmEd package:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import parmed as pmd&#xA;&#xA;pdb = pmd.load_file('molecule.pdb')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This pdb object for 'CC(C)C' for example, looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;Structure 14 atoms; 1 residues; 13 bonds; NOT parameterized&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then I want to create a prmtop file, which is needed to initialise an Amber force:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pdb.save('molecule.prmtop', overwrite=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This throws the error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;RuntimeError: atom types are not assigned&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm not sure why this error gets thrown, since the atom types do seem to be inside the PDB file. I'm new to trying to do these kinds of things, so I'm not completely sure if more information is needed.&lt;/p&gt;&#xA;&lt;p&gt;If this method in general is not preferable, what is the best way to achieve this (preferably on python)?&lt;/p&gt;&#xA;&lt;p&gt;Any help would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="17166" LastActivityDate="2023-03-14T12:34:19.723" Title="From SMILE get Amber force in Python" Tags="|python|rdkit|small-molecules|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20718" PostTypeId="1" CreationDate="2023-03-14T15:18:26.760" Score="1" ViewCount="90" Body="&lt;p&gt;I love that Deseq2 has altHypothesis=&amp;quot;lessAbs&amp;quot; !!!!!!! I've used it a ton for RNA-seq. However, now I'm working with mass spect data using limma. Is there a way to make limma do the same thing?&lt;/p&gt;&#xA;" OwnerUserId="17125" LastEditorUserId="15489" LastEditDate="2023-03-14T19:25:05.497" LastActivityDate="2023-03-14T19:25:05.497" Title="To find genes that don't change in RNA-seq, Deseq2 has altHypothesis=&quot;lessAbs&quot;. Is there a way to make limma do the same thing for proteomics?" Tags="|r|rna-seq|differential-expression|proteomics|limma|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20719" PostTypeId="1" CreationDate="2023-03-14T16:37:24.817" Score="0" ViewCount="93" Body="&lt;p&gt;I'm trying to map ONT long reads to a portion of a gene I'm looking at. The region is about 25bp long. When I search for the region in the document it pulls up the sequence in every read but when I map to reference with minimap2 it produces no contigs.&lt;/p&gt;&#xA;&lt;p&gt;Is 25bp too small to be a reference? I can potentially extend it but it's right in an area of conservation in this region of a gene and I would rather not extend much farther.&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="16067" LastEditDate="2023-11-01T21:17:53.260" LastActivityDate="2023-11-01T21:17:53.260" Title="Is my reference sequence too small?" Tags="|sequence-alignment|gene|nanopore|minimap2|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20720" PostTypeId="2" ParentId="20718" CreationDate="2023-03-14T18:18:25.930" Score="2" Body="&lt;p&gt;There is no direct support for this by best knowledge. Similar questions have been asked previously in the Bioconductor support forum, for example:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://support.bioconductor.org/p/64300/#64793&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.bioconductor.org/p/64300/#64793&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;They discuss options around the TOST concept.&lt;/p&gt;&#xA;&lt;p&gt;For edgeR:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://support.bioconductor.org/p/66283/#66287&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.bioconductor.org/p/66283/#66287&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;From these posts please go back and forth to find other posts in that forum to capture what the developers of limma/edgeR responded on the matter.&lt;/p&gt;&#xA;&lt;p&gt;Depending on your exact purpose it might even be enough to go for genes with large AveExpr and a nominal (not adj) p-value close to one.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-03-14T18:18:25.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20721" PostTypeId="1" AcceptedAnswerId="20722" CreationDate="2023-03-15T02:49:09.737" Score="2" ViewCount="148" Body="&lt;p&gt;Below is the code from nextflow site (&lt;a href=&quot;https://www.nextflow.io/docs/edge/operator.html#grouptuple&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nextflow.io/docs/edge/operator.html#grouptuple&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;For the output, the first is A B C, and the second is C, A.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to make the output like this&lt;/p&gt;&#xA;&lt;p&gt;[1, [A, B, C]]&#xA;[2, [A, C]]&lt;/p&gt;&#xA;&lt;p&gt;That is, the stuff inside [] is in some order.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Channel&#xA;    .of(&#xA;        [1, 'A'],&#xA;        [1, 'B'],&#xA;        [2, 'C'],&#xA;        [3, 'B'],&#xA;        [1, 'C'],&#xA;        [2, 'A'],&#xA;        [3, 'D']&#xA;    )&#xA;    .groupTuple()&#xA;    .view()&#xA;&#xA;[1, [A, B, C]]&#xA;[2, [C, A]]&#xA;[3, [B, D]]&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14191" LastActivityDate="2023-03-15T04:08:24.417" Title="file order after nextflow groupTuple" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20722" PostTypeId="2" ParentId="20721" CreationDate="2023-03-15T04:08:24.417" Score="2" Body="&lt;p&gt;You can change the sorting criteria for the grouped items using the &lt;code&gt;sort&lt;/code&gt; keyword. To perform a &lt;em&gt;natural&lt;/em&gt; sort, just use &lt;code&gt;sort: true&lt;/code&gt;. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Channel.of(&#xA;    [1, 'A'],&#xA;    [1, 'B'],&#xA;    [2, 'C'],&#xA;    [3, 'B'],&#xA;    [1, 'C'],&#xA;    [2, 'A'],&#xA;    [3, 'D']&#xA;)&#xA;.groupTuple(sort: true)&#xA;.view()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ nextflow run main.nf &#xA;N E X T F L O W  ~  version 22.04.0&#xA;Launching `main.nf` [special_darwin] DSL2 - revision: 2e641ccfd9&#xA;[1, [A, B, C]]&#xA;[2, [A, C]]&#xA;[3, [B, D]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Otherwise, a custom sorting criteria can be specified using a closure.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-03-15T04:08:24.417" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20723" PostTypeId="1" CreationDate="2023-03-15T10:13:46.493" Score="0" ViewCount="104" Body="&lt;p&gt;I performed a ROARY from 100 genome. I got several file including gene_presence_absence.csv  from the results. I want to know how many unique gene(unshared gene, based on blast homology, same gene (duplicate gene) may be unique if no homology is there between them) from presence absence csv file or other file. Kindly suggest me how to calculate total number of unique gene for each genome which is only present in one genome&lt;/p&gt;&#xA;" OwnerUserId="16745" LastEditorUserId="3694" LastEditDate="2023-03-15T15:07:44.293" LastActivityDate="2024-02-23T06:04:20.553" Title="Calculating total number of Unique gene in every genome from ROARY pangenome output" Tags="|r|phylogeny|reference-genome|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20724" PostTypeId="2" ParentId="18595" CreationDate="2023-03-15T14:07:57.547" Score="1" Body="&lt;p&gt;You need to make a transformed variable to the gene sets of interest. I would rescale the data on a range of 0-1. Then you can make a log2 ratio of the sum(set1)/sum(set2). You could explore other transformations, but this will give contrasting values for high set1 and high set2, but neutral if the expression is similar between the sets.&lt;/p&gt;&#xA;" OwnerUserId="17168" LastActivityDate="2023-03-15T14:07:57.547" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20725" PostTypeId="2" ParentId="20708" CreationDate="2023-03-15T15:58:01.937" Score="0" Body="&lt;p&gt;Here's one way using &lt;a href=&quot;/questions/tagged/awk&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; aria-label=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; rel=&quot;tag&quot; aria-labelledby=&quot;tag-awk-tooltip-container&quot;&gt;awk&lt;/a&gt;. This avoids the creation of the intermediary files by matching the alphabetic characters in the fourth column. If the prefix has not been seen, it is added to an array along with a counter, which is only incremented when the prefix has not yet been seen. We then store the seq ids and the replacement strings in another array. Once we've finished reading the first file, we can then query the first column of the second file in the arguments list. If the seq id in the first column exists in the array of seq ids, we can print the replacement string and remaining columns.&lt;/p&gt;&#xA;&lt;p&gt;With the following in a file called &lt;code&gt;script.awk&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BEGIN {&#xA;&#xA;   counter = 1&#xA;}&#xA;&#xA;FNR==NR {&#xA;    if( match(&lt;span class=&quot;math-container&quot;&gt;$4, /^NZ_[A-Z]+/) ) {&#xA;        prefix = substr($&lt;/span&gt;4, RSTART, RLENGTH)&#xA;&#xA;        if( !(prefix in seen) ) {&#xA;            seen[prefix] = counter++&#xA;        }&#xA;&#xA;        seq_ids[$3] = &amp;quot;seq&amp;quot; seen[prefix]&#xA;    }&#xA;&#xA;    next&#xA;}&#xA;&#xA;$1 in seq_ids {&#xA;&#xA;    print seq_ids[&lt;span class=&quot;math-container&quot;&gt;$1], $&lt;/span&gt;2, &lt;span class=&quot;math-container&quot;&gt;$3, $&lt;/span&gt;4&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Run using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk -f script.awk file1 file2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seq1 19301 20914 fill_color=green_a0&#xA;seq1 16726 18337 fill_color=green_a0&#xA;seq1 167934 169650 fill_color=green_a0&#xA;seq2 200621 202367 fill_color=red_a0&#xA;seq2 160164 161903 fill_color=green_a0&#xA;seq2 31356 33104 fill_color=green_a0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-03-15T15:58:01.937" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20726" PostTypeId="1" CreationDate="2023-03-15T17:55:25.450" Score="1" ViewCount="53" Body="&lt;p&gt;Given a set of genes in a FASTA-formatted file, for example: &lt;code&gt;exFasta.fa&lt;/code&gt;, what is the best way to determine how well conserved the genes in that file are among an order of organisms, for example: Orthoptera?&lt;/p&gt;&#xA;&lt;p&gt;BLAST is an obvious option, but I am not sure that it is the most efficient way considering there are likely thousands of organisms in that order. OrthoFinder also comes across as a valid option, but I am not sure how to implement such a thing considering the required inputs.&lt;/p&gt;&#xA;&lt;p&gt;Any advice would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="17169" LastEditorUserId="73" LastEditDate="2023-03-16T18:44:43.920" LastActivityDate="2023-03-16T18:55:01.423" Title="Given a FASTA file with a set of genes, how do I determine that set's conservation among a genetic order?" Tags="|python|blast|blastp|orthofinder|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20727" PostTypeId="1" CreationDate="2023-03-15T23:52:41.467" Score="2" ViewCount="238" Body="&lt;p&gt;I am trying to run --setupdb in Prokka, but I'm getting this error that I can't seem to find the answer for.&lt;/p&gt;&#xA;&lt;p&gt;When I run&#xA;&lt;code&gt;prokka --setupdb&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I get&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[18:57:31] Determined cmpress version is 001001 from '# INFERNAL 1.1.4 (Dec 2020)'&#xA;[18:57:31] Pressing CM database: /home/carol/.local/lib/prokka/db/cm/Viruses&#xA;[18:57:31] Running: cmpress \/home\/carol\/\.local\/lib\/prokka\/db\/cm\/Viruses&#xA;&#xA;Error: File existence/permissions problem in trying to open CM file /home/carol/.local/lib/prokka/db/cm/Viruses.&#xA;CM file /home/carol/.local/lib/prokka/db/cm/Viruses not found (nor an .i1m binary of it)&#xA;&#xA;[18:57:31] Could not run command: cmpress \/home\/carol\/\.local\/lib\/prokka\/db\/cm\/Viruses&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does anyone know how I can fix it?&lt;/p&gt;&#xA;" OwnerUserId="17175" LastActivityDate="2023-03-15T23:52:41.467" Title="PROKKA setupdb / cmpress error" Tags="|sequence-annotation|database|errors|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20728" PostTypeId="1" CreationDate="2023-03-15T23:58:36.087" Score="2" ViewCount="67" Body="&lt;p&gt;I am trying to do Cut and Tag analysis for a set of files and I want to perform a procedure called Tn5 normalization of samples.&#xA;The basic goal is to generate bedgraph files and does normalization with the weights of a sequencing depth. I have a set of function files that I have defined and then a set of Tn5 normalization scripts. The scripts are both quite long and hence I will be sharing parts of the scripts to get hold of my error and also provide as much information as possible.&lt;/p&gt;&#xA;&lt;p&gt;My bedgraph files look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;replicate species Msample serotype protein            sample      condition    group&#xA;1        R1     NHP      02     LK03      NA R1-NHP-02-LK03-NA R1-NHP-LK03-NA LK03-NHP&#xA;2        R1     NHP      04     LK03      NA R1-NHP-04-LK03-NA R1-NHP-LK03-NA LK03-NHP&#xA;3        R1     NHP      13      KP1      NA R1-NHP-13-KP1-NA R1-NHP-KP1-NA KP1-NHP&#xA;4        R1     NHP      42      KP1      NA R1-NHP-42-KP1-NA R1-NHP-KP1-NA KP1-NHP&#xA;5        R1     NHP      55     LK03      NA R1-NHP-55-LK03-NA R1-NHP-LK03-NA LK03-NHP&#xA;6        R1     NHP      57      KP1      NA R1-NHP-57-KP1-NA R1-NHP-KP1-NA KP1-NHP&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I must load a complete set of files called RDS which basically bears a list of the individual bed graph files.&lt;/p&gt;&#xA;&lt;p&gt;The read lengths for each of the files look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqnames start  end normcount&#xA;73694699     opt2     0   52 0.0000000&#xA;73694700     opt2    52   56 0.0243838&#xA;73694701     opt2    56   83 0.0731515&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The Tn5 R normalization scripts which are relevant&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(dplyr, lib.loc = &amp;quot;/scg/apps/software/r/4.1.2/lib64/R/library&amp;quot;)&#xA;library(gridExtra, lib.loc = &amp;quot;/scg/apps/software/r/4.1.2/lib64/R/library&amp;quot;)&#xA;library(Gviz, lib.loc = &amp;quot;/home/ag111/R/x86_64-pc-linux-gnu-library/4.1&amp;quot;) #needed to be installed&#xA;library(ggplot2)&#xA;library(RcppArmadillo, lib.loc = &amp;quot;/scg/apps/software/r/4.1.2/lib64/R/library&amp;quot;)&#xA;#library(stringi, lib.loc = &amp;quot;/scg/apps/software/r/4.0.3/lib64/R/library&amp;quot;)&#xA;library(tidyr, lib.loc = &amp;quot;/scg/apps/software/r/4.1.2/lib64/R/library&amp;quot;)&#xA;library(RColorBrewer, lib.loc = &amp;quot;/scg/apps/software/r/4.1.2/lib64/R/library&amp;quot;)&#xA;library(ggpubr, lib.loc = &amp;quot;/scg/apps/software/r/4.1.2/lib64/R/library&amp;quot;) #needed to be installed&#xA;library(GenomicFeatures, lib.loc = &amp;quot;/scg/apps/software/r/4.1.2/lib64/R/library&amp;quot;)&#xA;library(GenomicRanges, lib.loc = &amp;quot;/scg/apps/software/r/4.1.2/lib64/R/library&amp;quot;)&#xA;&#xA;source('AG_CnRT_functions.r')&#xA;dir.create(file.path(&amp;quot;Tn5exITR_InVivo_norm_full&amp;quot;))&#xA;&#xA;intermediates_dirs &amp;lt;- c(SeqDepth='intermediates/')&#xA;#intermediates_dirs &amp;lt;- c(SeqDepth='/labs/markay/Aranyak/Tn5_NHP_rename/Data_analysis/intermediates/')&#xA;&#xA;normalization_method &amp;lt;- 'SeqDepth'&#xA;&#xA;AAV_gsize &amp;lt;- 4410&#xA;AAV_anno &amp;lt;- read.delim('AAV_opt2_anno.txt')&#xA;&#xA;#figures_dir &amp;lt;- c(SeqDepth='Tn5exITR_InVivo_norm_full2/')&#xA;&#xA;figures_dir &amp;lt;- c(SeqDepth='Tn5exITR_InVivo_norm_full2/')&#xA;&#xA;############## Tn5 ###########&#xA;AAV_normbedgraphs &amp;lt;- readRDS(&amp;quot;/labs/markay/Aranyak/Tn5_NHP_rename/Data_analysis/intermediates/AAV_rmdup_SeqDepthNormbedgraphs.rds&amp;quot;)&#xA;&#xA;naming_order &amp;lt;- c('replicate','species',&amp;quot;Msample&amp;quot;,'serotype','protein') &#xA;&#xA;AAV_normbedgraphs_exITR &amp;lt;- AAV_normbedgraphs %% lapply(function(x){&#xA;  df &amp;lt;- x %% subset(start = 185 &amp;amp; end &amp;lt;=  4167)&#xA;  return(df)&#xA;})&#xA;&#xA;AAV_normcov_regions_meta_exITR &amp;lt;- list()&#xA;AAV_normcov_regions_meta_exITR[[normalization_method]] &amp;lt;- get_normcov_regions_meta_AG( &#xA;  AAV_normbedgraphs=AAV_normbedgraphs_exITR,&#xA;  naming_order=naming_order,&#xA;  AAV_gsize=AAV_gsize,&#xA;  AAV_anno=AAV_anno)&#xA;&#xA;&#xA;Merging data&#xA;AAV_normcov_regions_meta_exITR[[normalization_method]]&lt;span class=&quot;math-container&quot;&gt;$serotype[AAV_normcov_regions_meta_exITR[[normalization_method]]$&lt;/span&gt;serotype == &amp;quot;LK&amp;quot;] &amp;lt;- &amp;quot;LK03&amp;quot;&#xA;AAV_normcov_regions_meta_exITR[[normalization_method]]&lt;span class=&quot;math-container&quot;&gt;$serotype[AAV_normcov_regions_meta_exITR[[normalization_method]]$&lt;/span&gt;serotype == &amp;quot;iG&amp;quot;] &amp;lt;- &amp;quot;AM&amp;quot;&#xA;AAV_normcov_regions_meta_exITR[naming_order] &amp;lt;- lapply(AAV_normcov_regions_meta_exITR[naming_order], factor)&#xA;AAV_normcov_regions_meta_exITR[[normalization_method]]&lt;span class=&quot;math-container&quot;&gt;$serotype &amp;lt;- ordered(AAV_normcov_regions_meta_exITR[[normalization_method]]$&lt;/span&gt;serotype, levels = c(&amp;quot;AM&amp;quot;, &amp;quot;LK03&amp;quot;, &amp;quot;DJ&amp;quot;))&#xA;&#xA;AAV_normcov_regions_meta_exITR[[normalization_method]]&lt;span class=&quot;math-container&quot;&gt;$serotype[AAV_normcov_regions_meta_exITR[[normalization_method]]$&lt;/span&gt;serotype == &amp;quot;LK&amp;quot;] &amp;lt;- &amp;quot;LK03&amp;quot;&#xA;AAV_normcov_regions_meta_exITR[[normalization_method]]&lt;span class=&quot;math-container&quot;&gt;$serotype[AAV_normcov_regions_meta_exITR[[normalization_method]]$&lt;/span&gt;serotype == &amp;quot;iG&amp;quot;] &amp;lt;- &amp;quot;KP1&amp;quot;&#xA;AAV_normcov_regions_meta_exITR[naming_order] &amp;lt;- lapply(AAV_normcov_regions_meta_exITR[naming_order], factor)&#xA;AAV_normcov_regions_meta_exITR[[normalization_method]]&lt;span class=&quot;math-container&quot;&gt;$serotype &amp;lt;- ordered(AAV_normcov_regions_meta_exITR[[normalization_method]]$&lt;/span&gt;serotype, levels = c(&amp;quot;KP1&amp;quot;, &amp;quot;LK03&amp;quot;))&#xA;&#xA;&#xA;metadata &amp;lt;- get_metadata_AG3(AAV_normbedgraphs=AAV_normbedgraphs_exITR,&#xA;                             #naming_order=naming_order)&#xA;metadata &amp;lt;- get_metadata_KP(AAV_normbedgraphs=AAV_normbedgraphs_exITR,&#xA;                             naming_order=naming_order)&#xA;&#xA;head(metadata)&#xA;&#xA;#coverage separated by AAV regions and species&#xA;#to_plot &amp;lt;- c()&#xA;#to_plot &amp;lt;- AAV_normcov_regions_meta_exITR[[normalization_method]] %% gather('region','normcov',2:6) &#xA;#to_plot&lt;span class=&quot;math-container&quot;&gt;$region &amp;lt;- factor(to_plot$&lt;/span&gt;region, levels=c(as.character(AAV_anno_exITR&lt;span class=&quot;math-container&quot;&gt;$region),'AAV_normcov'))&#xA;#max_y &amp;lt;- ceiling(max( log2(to_plot$&lt;/span&gt;normcov1) ) )&#xA;#min_y &amp;lt;- floor( min( log2(to_plot$normcov1) ) ).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After loading of the rds file the error I am getting while I am executing the code block&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt; AAV_normcov_regions_meta_exITR &amp;lt;- list()&#xA; #AAV_normcov_regions_meta_exITR[[normalization_method]] &amp;lt;- get_normcov_regions_meta_AG3(&#xA; AAV_normcov_regions_meta_exITR[[normalization_method]] &amp;lt;- get_normcov_regions_meta_AG( &#xA;   AAV_normbedgraphs=AAV_normbedgraphs_exITR,&#xA;   naming_order=naming_order,&#xA;   AAV_gsize=AAV_gsize,&#xA;   AAV_anno=AAV_anno)&#xA;&#xA;Error in if (!all(AAV_normcov_regions&lt;span class=&quot;math-container&quot;&gt;$sample == metadata$&lt;/span&gt;sample)) { : &#xA;  missing value where TRUE/FALSE needed.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The code block which I think for the function file that is valid  is&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;AAV_normcov_regions[is.na(AAV_normcov_regions)] &amp;lt;- 0&#xA;  AAV_normcov_regions[is.null(AAV_normcov_regions)] &amp;lt;- 0&#xA;  AAV_normcov_regions[is.na(AAV_normcov_regions)] = 0&#xA;  AAV_normcov_regions &amp;lt;- AAV_normcov_regions[ match(metadata$sample,AAV_normcov_regions$sample) , ]&#xA;  if ( !all(AAV_normcov_regions$sample == metadata$sample) ){ &#xA;    stop('Unsuccessful matching to metadata') &#xA;  }    &#xA;  return(AAV_normcov_regions).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This function is working for another set of samples to generate a plot list.&#xA;The plot list I am having is to_plot&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;[1] sample      AAV_normcov replicate   species     Msample     serotype    protein     condition   group      &#xA;&amp;lt;0 rows&amp;gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(or 0-length row.names) where as what is expected is something like this&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;  sample   Enhancer        FLuc      polyA   Promoter       WPRE AAV_normcov replicate batch species drug serotype condition group&#xA;1  R1-LiD-H-N-DJ 0.42244801 0.290929537 0.43640203 0.22379898 0.34125606  0.26650388        R1   LiD       H    N       DJ      H-DJ  H-DJ&#xA;2  R1-LiD-H-N-iG 1.42930414 0.259549496 0.57248120 0.37425687 0.42189340  0.36092640        R1   LiD       H    N       AM      H-iG  H-iG&#xA;3  R1-LiD-H-N-LK 0.25754367 0.317786560 0.27373645 0.19470267 0.29818285  0.24753280        R1   LiD       H    N     LK03      H-LK  H-LK&#xA;4  R1-LiD-M-N-DJ 0.24578258 0.097169668 0.14489638 0.06470209 0.11091813  0.09308374        R1   LiD       M    N       DJ      M-DJ  M-DJ&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am happy to share other code blocks if it is required to answer the question.&lt;/p&gt;&#xA;" OwnerUserId="9659" LastEditorUserId="650" LastEditDate="2023-03-16T20:36:08.077" LastActivityDate="2023-03-17T12:28:17.623" Title="Cut and Tag analysis" Tags="|r|genomics|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20729" PostTypeId="1" CreationDate="2023-03-16T15:16:48.063" Score="2" ViewCount="163" Body="&lt;p&gt;I am trying to demultiplex some paired-end ddradseq data and am running into an issue with STACKS in that the program only seems to read 1 of my files for input (there are 2) and also results in over 90% of my barcodes identified as not found, and therefore dropped. As a result, my retained reads are extremely low (1.1%). I ran this same script before on the same dataset and had good results so I am not sure why I am coming across this issue now?&lt;/p&gt;&#xA;&lt;p&gt;The current script I am using is as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;module load stacks&#xA;&#xA;process_radtags -p /home/heather/scratch/raw_inputs/ -P -o /home/heather/scratch/processrad_out \&#xA; -b /home/heather/scratch/barcodes/lesp_barcodes_batch1.txt \&#xA; -c -q -r --inline_inline --renz_1 sbfI --renz_2 mluCI&#xA; &#xA;echo &amp;quot;Process radtags complete. Check log file to see dropped reads then proceed to alignment&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My input files are named HFL_R1_001.fastq.gz and HFL_R1_002.fastq.gz respectively (I have simplified these for the sake of this post, but the correct file names are specified in my scripts so this is not the issue). The program runs, but this is the output of the log file that I get which shows very low retained reads.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BEGIN per_file_raw_read_counts&#xA;File    Retained Reads  Low Quality     Barcode Not Found   RAD cutsite Not Found   Total&#xA;HFL_S1_L001_R1_001.fastq.gz 11092204        3150    1029697796  12760620        1053553770&#xA;END per_file_raw_read_counts&#xA;&#xA;BEGIN total_raw_read_counts&#xA;Total Sequences 1053553770&#xA;Barcode Not Found   1029697796  97.7%&#xA;Low Quality     3150    0.0%&#xA;RAD Cutsite Not Found   12760620        1.2%&#xA;Retained Reads  11092204        1.1%&#xA;END total_raw_read_counts&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have also tried running a script where I specified with the -1 and -2 flags which files I wanted the program to input specifically and get the same result, with the difference of the program seems to read data from both files, but still only finds the 1 input file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Processing paired-end data.&#xA;Using Phred+33 encoding for quality scores.&#xA;Found 1 paired input file(s).&#xA;Searching for single and paired-end, inlined barcodes.&#xA;Loaded 191 barcodes (4-6bp / 6bp).&#xA;Will attempt to recover barcodes with at most 1 / 1 mismatches.&#xA;Processing file 1 of 1 [HFL_S1_L001_R1_001.fastq.gz]&#xA;  Reading data from:&#xA;  /home/heather/scratch/raw_inputs/HFL_S1_L001_R1_001.fastq.gz and&#xA;  /home/heather/scratch/raw_inputs/HFL_S1_L001_R2_001.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The rest of the log file from this particular script looks almost identical to the one described above, where retained reads are around 1%.&lt;/p&gt;&#xA;&lt;p&gt;I have checked my barcode file and there are no unreadable characters and I can't imagine why I would be getting so many lost barcodes when this ran previously with no problem? If anyone has any insight it would be much appreciated. I am relatively new to bioinformatics and don't really know where to go from here.&lt;/p&gt;&#xA;" OwnerUserId="17177" LastEditorUserId="17177" LastEditDate="2023-03-16T15:18:59.530" LastActivityDate="2023-03-17T12:23:22.390" Title="STACKS: process_radtags only reads 1 input file and returns &gt;90% barcode not found drops" Tags="|genomics|demultiplexing|radseq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20730" PostTypeId="2" ParentId="20726" CreationDate="2023-03-16T18:55:01.423" Score="2" Body="&lt;p&gt;It's possible that &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/moleblast/moleblast.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;MOLE-BLAST&lt;/a&gt; could help for you:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;MOLE-BLAST is an experimental tool that helps taxonomists find closest database neighbors of submitted query sequences. It computes a multiple sequence alignment (MSA) between the query sequences along with their top BLAST database hits, and generates a phylogenetic tree. Query sequences in the tree are denoted with highlighted node labels. If the input sequences come from different genes or loci, MOLE-BLAST can cluster them and compute an MSA and a phylogenetic tree for each locus separately. This tool may eventually be incorporated into our supported suite of BLAST applications based on the feedback we receive.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I haven't actually used MOLE-BLAST, but I have used &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/smartblast/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Smart BLAST&lt;/a&gt;, which does a somewhat similar phylogenetic search (although on a restricted diverse taxonomic set) for protein queries.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-03-16T18:55:01.423" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20731" PostTypeId="2" ParentId="20719" CreationDate="2023-03-17T08:13:24.807" Score="3" Body="&lt;p&gt;That's not the right way to do it.&lt;/p&gt;&#xA;&lt;p&gt;You either have reference or not. If not, you need to assemble one (that problem is called genome assembly). If yes, you map all the reads to the whole reference - the process of mapping is made for this - placing individual WGS reads to their corresponding locations and as a matter of fact, missing portions of the reference can mess it up (that was one of the reasons why there was a push for T2T for human).&lt;/p&gt;&#xA;&lt;p&gt;Once the reads are mapped, you can use any genome browser (e.g. IGV) to zoom in to the exact location you are interested in and check the stack of reads mapping in there. Given how small the region is, manual inspection of the quality of the mapping is the best you got.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2023-03-17T08:13:24.807" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20737" PostTypeId="1" CreationDate="2023-03-17T13:09:38.170" Score="2" ViewCount="174" Body="&lt;p&gt;I have used Minimap2 to create a paf file by aligning a Fastq file against itself. Now from this Paf file I can see where the reads overlap, and I want to take these positions, and use them to extract the entire sequences between these positions from the paf file so like this:&lt;/p&gt;&#xA;&lt;p&gt;If read A starts on position 72 and ends on position 2041 and overlaps here with read B which starts on position 190 and ends on position 2741, I would like to extract these from the Fastq and merge into a contig. But I don't really know how to do that.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;query_name,query_length,query_start,query_end,strand,target_name,target_length,target_start,target_end,residue_matches,alignment_block_length,mapping_quality,tp,cm,s1,dv,rl&#xA;read=73,2677,81,2674,+,read=86140,3729,190,2779,2082,2627,0,S,4,2,0,6&#xA;read=73,2677,89,2674,-,read=78530,52349,15309,17890,1977,2623,0,S,4,1,0,6&#xA;read=73,2677,95,2674,-,read=8246,38338,32497,35071,1976,2623,0,S,4,1,0,6&#xA;read=73,2677,95,2674,-,read=82661,13059,8472,11069,1960,2636,0,S,4,1,0,6&#xA;read=73,2677,100,2669,+,read=79559,11152,5925,8498,1928,2617,0,S,4,1,0,6&#xA;read=73,2677,89,2674,+,read=75579,16131,10030,12576,1924,2609,0,S,4,1,0,6&#xA;read=73,2677,100,2669,+,read=92760,9659,7065,9617,1904,2613,0,S,4,1,0,6&#xA;read=73,2677,89,2674,+,read=95665,81160,4544,7111,1866,2624,0,S,4,1,0,6&#xA;read=73,2677,81,2660,+,read=75036,31663,27966,30529,1841,2619,0,S,4,1,0,6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is a snippet from the PAF file. It's comma delimitted, and below are some of the reads from the Fastq file, the reads vary a lot in length.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@9a65dba6-05c9-49d7-ac0e-2ede30be6036 runid=7f8eec05f1beed68c29c091f78bf84807beb3275 read=39 ch=1736 start_time=2021-01-27T22:53:24Z flow_cell_id=PAG49596 protocol_group_id=210105Chi sample_id=D21-130 barcode=barcode01&#xA;AGTATACTTCGTTTCAGTTACGTATTGCTAAGGTTAACACAAAGACACCGACAACTTTCTTCAGCACCTGGTGCTGAAGAAAGTTATCGGTATTTCTTTGTGATCAGCCGTAG&#xA;+&#xA;%))(&amp;amp;&amp;amp;''&amp;amp;((AA8&amp;gt;33??7++)'*+$$55(1@BFFF&amp;gt;?&amp;gt;=&amp;gt;684.-8:2)&amp;amp;.,11168=&amp;gt;,*/13;;;348:A&amp;gt;;7032)2312%*,&amp;amp;')(001123)),+&amp;amp;,*$%#$%$$$&#xA;@edaf3d67-a971-43ed-be5b-3816b985d53f runid=7f8eec05f1beed68c29c091f78bf84807beb3275 read=136 ch=1940 start_time=2021-01-27T22:54:06Z flow_cell_id=PAG49596 protocol_group_id=210105Chi sample_id=D21-130 barcode=barcode01&#xA;ATTACTTCGTTCAGTTACGTATTGCTAAGGTTAACACAAAGACACCACGACAACTTTCTTCGCACTATGGTGTTCGCTATTATCAGATTCCTGATGAAGAAGAACTAATGAAATTGGTTTCTATTATTTATTGACTTCGTTCTATAATAGAGCAAAGTTTTCAGCTGGAGAAAATTGGTGAAGAAAAAGCTGAGATTACTGCAAAAGCAACTTCTCTCGTGGTAAATCTGTCACCTCTGCAGAGCATTATCTGAAAGAATGAGGAACTTTCCTGAAGTTATTGTTCAATATGGTTGCTCTTCAAAATTGCTAAGCCAGAACAGCAAGATTGATAACATTCATGCATTAGAAGACGCTCTTTATAGTAAGTTGGTTGGCGTGGTACAGTAGACTGTATTAGTGAATATGATGGAGAATTAGCAGTTATTGACTTCAAGACTTCTAAAGCACCTAAGACCGAAAATGGATTGATGGTTATTTTGCAGCAATGGCATGTATTTATTCATTAGTTCATTGGCATGTGCTAAGAAACTTGTTGTATAATAATGGCATGTGAAGATGGAACCTGTAAAGTTTATGAAGGCGTAATAGAAACATCAACATCAGCAAATTACTCGTACATTACATTAGAAACTTTTTGGATTTTCAACTAGAATTAAATGGAAAATGAATTAACCAAAGCACTAGATAAAAAATTCATGAATGCTGCCAAAAAGTTTTCTCTTGAAGTAGAGAAAATTGTATTGGAAGAAAAAAGTGAATTATATTGATGCTATTGTTCTATTTTGTGGAAAAATAGTATTGAAGTGGACTCTGTTACTAAGTTGATTTCTAAACCTTTAAAGGAAAAATTAAAGGCAGATACTGTAATTTGAACTTCATGAAGAACCTCGTGCCGACTACCCATATAAATACTAGTATCTAGGAGGAATTATGTCTGATTTTTTGAATCTGAACAAGTTCCAAGAATCTATAAAGACATCAACAGATGCAAGAAAGAGATATATTCTAAAGTCTTGTTTGCTTTTTGACATTGCCTCAAAGAAAATGGATCATGTTGATTTATTAGAACAACTTCTAGAGAAGCAAAAGAAACTTCTACATGAGATTGGCTTCTGACGATCCTCGTGCTAAAGATATGATGGAGCAAATTCATCAATCTGCTCAGTTGATTGGATTCCCCTAAGGATTCATCACCTGATGTTTTATTTGAGAATATGAACCTTGGTCTTATTAGATTTGCTTGACATGGGAGCACTGTCGCCCTATAATAGACCAGTACACAAAAGCCCAAATCTAATTACAGAAATTTGGAAATCTATGTCTTTTGCATCACTAAAAAAACAGTCCTCTCTTGGATCTCTAACCGCAAAGTTGGTTAAAGAAGTAGAAGACCAATAAAGCAGGTAACGGGCTGATGAGCGTCTTTGACCAGAGAGTTGATAAAAAGCATGGAGCAGTCTGTTGCAGTTATTCGTTTCTTCCAGCACCAGATAAGAAGGATTCTTCCTTGGGCAAAACTTTATTCTCATGCATTTCAAGGATGAGTGGATGGTACGTGAAAATAACTACTGTTGGTAGTAAGGATCCTGTTTCAGAGTACAATACGGTCTCATGGATTCTGGTTCAGATGCCAATAAGCAGATCGCTCCTTCAATCAAAGGCGTAAGCTTTTCTTACTATAGCAACATCTATGTTGTAGAGGATATACAAATCCGAATGGAAGGAAAAATATGTATTTACAAATTTGGTAAGAAGATCTTGACAGATCATCGCCAATAATACAACCTGAGTTCAGGGATGAGTCACCAATCAATCCATTGATTTCACAA&#xA;+&#xA;%%&amp;amp;-79888F?9.-90/5400/0(&amp;amp;24105EEH?:39&amp;lt;7++999:8'+23695,,,..77*%&amp;amp;'*.'%).&amp;amp;)882))2*220.7,3CIHA7476935-2734D;:;8+--1;003))%'&amp;amp;$%$$&amp;amp;#%&amp;amp;&amp;amp;&amp;amp;&amp;amp;(-57:;44&amp;gt;:;B=&amp;lt;A4//.%+452,-4('(#$##%&amp;amp;++&amp;amp;,.9;???:CEHJ9GEA@&amp;lt;/10.:9&amp;lt;HH2&amp;lt;&amp;lt;;&amp;gt;IE10,45*79:1(3%1,3-*)'''*'''31*++40*$$$$)'689&amp;lt;?A5668:)4&amp;lt;=220#''%$$$$$%%%.++2/%#%$'(#+*)(&amp;amp;#&amp;amp;')#$')*+'&amp;gt;&amp;gt;&amp;lt;::&amp;lt;65&amp;lt;:,-:;8)&amp;amp;((--4;9&amp;gt;=ED?&amp;lt;&amp;lt;==;;886)6689&amp;lt;;&amp;amp;&amp;amp;)&amp;amp;%$&amp;quot;$-55;&amp;gt;=+9:29;)'('')#%%%$$&amp;amp;&amp;amp;&amp;amp;%**+112&amp;amp;).739:@=-0.-.290&amp;lt;&amp;gt;&amp;gt;@DABCFGFGABBDEIHD&amp;gt;&amp;gt;02'''(/-6977&amp;gt;64;62,+/33/&amp;amp;(#**+,.7?:--07;:74'')4.''((&amp;amp;%$((*100.)%%$##$##$#$%#%#%$$%*$&amp;amp;#$$$$#$%'&amp;amp;%((4-;&amp;lt;79992..++2589=B?;;&amp;lt;;=?@&amp;gt;@DE=FAE&amp;gt;H@?===ADBA=HK@&amp;gt;=;&amp;lt;+(830-1--($*+%$)%&amp;amp;$'#(&amp;amp;%%$$%.5569CH@=&amp;gt;==A@A=CGHJKKNJFEKGB0'(1/HLJA&amp;lt;&amp;lt;'4:7/?&amp;gt;99ABE??@CJNG==&amp;gt;3?EJFCCROOFEGD&amp;gt;EHJLWTQMMJIMNJ?@CFBBC@&amp;gt;=&amp;gt;9:2/.1&amp;gt;CEF?GAFDC;@5::&amp;amp;8(BBIFJHBCEFF;571*II?34=6;=BFFG:55=;??7.+*,56@KLFA@@@C&amp;lt;,(+/0,&amp;gt;E734867526/;&amp;lt;DB&amp;lt;DEDDG?=F:&amp;gt;D@?CI@EB@CMHEF@&amp;lt;@B@A&amp;gt;80,444=AC7781)%%&amp;amp;&amp;amp;%(%$#$&amp;amp;#&amp;amp;''(6-.35''7'%(()&amp;amp;)&amp;amp;()*&amp;gt;74&amp;lt;8&amp;lt;.)038./0.&amp;amp;@BDICA=&amp;gt;KE@&amp;lt;&amp;lt;CC&amp;gt;&amp;lt;7&amp;lt;;8747?=&amp;gt;;66&amp;lt;:;9ANFED4%4762/0//3;&amp;lt;8:DB8:=8+;&amp;lt;&amp;gt;:E?@D@/59&amp;lt;&amp;lt;956/13,=88&amp;lt;B6@?=',:8@&amp;gt;&amp;lt;89&amp;gt;@;)()+%.*&amp;amp;''())&amp;amp;./0/,%'(&amp;amp;&amp;amp;''(2263'''%07A&amp;lt;;&amp;lt;&amp;lt;C?:?===B&amp;lt;8:JD6;===?A@CB//2/.(%$14333@&amp;lt;1.)31,5643+.:;89=-/9:+&amp;amp;)-./022149892163/+;56&amp;gt;=;,'''2360-..;ABC@@98;&amp;lt;@&amp;gt;E=&amp;lt;?&amp;gt;AA83C&amp;gt;==C?*.*FGE@;;3-.---AD9;&amp;lt;.;89=??;;?HA/$$'81*.3+(%#$%($$#%%$'&amp;amp;$&amp;amp;$%&amp;amp;%*%*,92'')7;8:9;?;9=???8:@76;&amp;gt;=A&amp;gt;&amp;gt;A&amp;lt;;&amp;gt;8=9&amp;gt;;==A9&amp;lt;8?==;(1+).1AC=ECDFF501&amp;amp;*)&amp;amp;#(5258EAB&amp;gt;@&amp;amp;/285&amp;gt;@?3.;@@&amp;lt;:=&amp;lt;A@&amp;gt;&amp;lt;;;3009;5+556&amp;lt;AA?78=7C;&amp;gt;?;67+7:;75&amp;gt;;,/92./,'++.//(()/59&amp;lt;@&amp;gt;&amp;gt;CA8,&amp;amp;&amp;amp;8:894%%(.%/01//.0.,+',/'&amp;amp;,/10)%$$(/,&amp;amp;&amp;amp;$.,***#%%%&amp;amp;&amp;amp;$##$$%%##$'&amp;amp;&amp;amp;')26&amp;amp;1331:;A:CD?A=3666:/.300(,*/#'#'+#1;6444-+(+35&amp;lt;;89&amp;lt;7257,@@&amp;gt;1.++DDA4*&amp;amp;(*'*)*7&amp;lt;&amp;lt;&amp;lt;=873)#%%$#$--(''%(*+7968&amp;gt;6.41-7888:&amp;gt;B:;==FDC:6998;;8:+'&amp;amp;$$'+))'#(%+%')$(+2&amp;lt;?&amp;lt;56:3116708;8%445433.474/-1+--5465'&amp;amp;&amp;amp;%$&amp;amp;++,,088:5:;95;/&amp;amp;*::@99=A::;45;?)$$&amp;amp;+,,&amp;amp;$$#$%00--.0(),''(&amp;amp;''''&amp;amp;&amp;amp;&amp;amp;$$%##$%%&amp;amp;$'224(12;.001+$*)(()'%&amp;amp;%##$$&amp;amp;'%$%$#&amp;quot;$))$0+)'8;79:;93=A?;&amp;amp;%136-,,/;.+*3;&amp;lt;&amp;gt;8?&amp;gt;=:&amp;lt;9&amp;lt;&amp;lt;58?98.#$&amp;amp;%&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Another thing that is currently confusing me about my Paf file is the target length and query length which seems to make no sense. It's not the length of the overlap nor the length of the entire segment or read. Can anybody shed some light here on the Paf file and how to extract these positions in Python? Please note that I do not want to use external tools for this, only Python and Python libraries.&lt;/p&gt;&#xA;" OwnerUserId="17135" LastEditorUserId="298" LastEditDate="2023-03-17T14:16:20.417" LastActivityDate="2023-03-17T14:16:20.417" Title="Extracting positions from PAF files in order to extract sequences from a Fastq file with Python" Tags="|python|minimap2|paf|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20738" PostTypeId="2" ParentId="20257" CreationDate="2023-03-17T21:52:22.600" Score="1" Body="&lt;p&gt;I ended up using the following code to solve this issue:&lt;/p&gt;&#xA;&lt;p&gt;Change the &lt;code&gt;tsv&lt;/code&gt; file and the &lt;code&gt;tree&lt;/code&gt; file name accordingly. Be mindful of the format the &lt;code&gt;tsv&lt;/code&gt; file should be in (indicated in the code).&lt;/p&gt;&#xA;&lt;p&gt;Also change the tree and the font scales as needed.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Load required libraries&#xA;library(ape)&#xA;library(ggtree)&#xA;&#xA;# Read the TSV file with species and node names and numbers&#xA;# Make sure the TSV file has headers: &amp;quot;name&amp;quot;, and &amp;quot;number&amp;quot;&#xA;data &amp;lt;- read.table(&amp;quot;counts.tsv&amp;quot;, header = TRUE, sep = &amp;quot;\t&amp;quot;, fill = TRUE, colClasses = &amp;quot;character&amp;quot;)&#xA;&#xA;# Read the Newick file&#xA;newick_file &amp;lt;- &amp;quot;species_tree.newick&amp;quot;&#xA;tree &amp;lt;- read.tree(newick_file)&#xA;&#xA;# Add labels and numbers to the tree&#xA;add_labels &amp;lt;- function(tree, data) {&#xA;  missing_numbers &amp;lt;- c()&#xA;  &#xA;  for (i in 1:nrow(data)) {&#xA;    label_name &amp;lt;- data&lt;span class=&quot;math-container&quot;&gt;$name[i]&#xA;label_number &amp;lt;- data$&lt;/span&gt;number[i]&#xA;    if (label_name %in% tree&lt;span class=&quot;math-container&quot;&gt;$tip.label) {&#xA;  tip_index &amp;lt;- match(label_name, tree$tip.label)&#xA;  tree$tip.label[tip_index] &amp;lt;- paste(label_name, label_number, sep = &quot;:&quot;)&#xA;} else if (label_name %in% tree$&lt;/span&gt;node.label) {&#xA;      node_index &amp;lt;- match(label_name, tree&lt;span class=&quot;math-container&quot;&gt;$node.label)&#xA;  tree$&lt;/span&gt;node.label[node_index] &amp;lt;- paste(label_name, label_number, sep = &amp;quot;:&amp;quot;)&#xA;    } else {&#xA;      missing_numbers &amp;lt;- c(missing_numbers, label_name)&#xA;    }&#xA;  }&#xA;  &#xA;  return(list(tree = tree, missing_numbers = missing_numbers))&#xA;}&#xA;&#xA;# Modify the tree object to include labels and numbers&#xA;result &amp;lt;- add_labels(tree, data)&#xA;tree &amp;lt;- result&lt;span class=&quot;math-container&quot;&gt;$tree&#xA;missing_numbers &amp;lt;- result$&lt;/span&gt;missing_numbers&#xA;&#xA;# Write missing labels to a log file&#xA;if (length(missing_numbers) &amp;gt; 0) {&#xA;  writeLines(c(&amp;quot;Missing numbers for the following labels:&amp;quot;, missing_numbers), &amp;quot;missing_labels.log&amp;quot;)&#xA;}&#xA;&#xA;# Convert factors to characters&#xA;tree&lt;span class=&quot;math-container&quot;&gt;$tip.label &amp;lt;- as.character(tree$&lt;/span&gt;tip.label)&#xA;tree&lt;span class=&quot;math-container&quot;&gt;$node.label &amp;lt;- as.character(tree$&lt;/span&gt;node.label)&#xA;&#xA;# Visualize the tree&#xA;p &amp;lt;- ggtree(tree, layout = &amp;quot;rectangular&amp;quot;) +&#xA;  geom_tiplab(size = 3) +&#xA;  geom_nodelab(size = 3, na.rm = TRUE, nudge_x = 0.008) +&#xA;  theme_tree2() +&#xA;  theme(legend.position = &amp;quot;none&amp;quot;)&#xA;&#xA;# Save the tree as a PDF&#xA;ggsave(&amp;quot;species_tree_with_numbers.pdf&amp;quot;, p, width = 20, height = 20, dpi = 300)&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16699" LastActivityDate="2023-03-17T21:52:22.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20739" PostTypeId="2" ParentId="18642" CreationDate="2023-03-18T08:45:20.397" Score="2" Body="&lt;p&gt;In the &lt;code&gt;efetch.fcgi&lt;/code&gt; URL, you can set &lt;code&gt;rettype=fasta&lt;/code&gt; to download the nucleotide sequence or &lt;code&gt;rettype=fasta_cds_aa&lt;/code&gt; to download protein coding sequences:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ curl 'https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=nuccore&amp;amp;rettype=fasta&amp;amp;id=NC_045512.2'|head -n3&#xA;&amp;gt;NC_045512.2 Severe acute respiratory syndrome coronavirus 2 isolate Wuhan-Hu-1, complete genome&#xA;ATTAAAGGTTTATACCTTCCCAGGTAACAAACCAACCAACTTTCGATCTCTTGTAGATCTGTTCTCTAAA&#xA;CGAACTTTAAAATCTGTGTGGCTGTCACTCGGCTGCATGCTTAGTGCACTCACGCAGTATAATTAATAAC&#xA;$&lt;/span&gt; curl 'https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=nuccore&amp;amp;rettype=fasta_cds_aa&amp;amp;id=NC_045512.2'|head -n3&#xA;&amp;gt;lcl|NC_045512.2_prot_YP_009724389.1_1 [gene=ORF1ab] [locus_tag=GU280_gp01] [db_xref=GeneID:43740578] [protein=ORF1ab polyprotein] [exception=ribosomal slippage] [protein_id=YP_009724389.1] [location=join(266..13468,13468..21555)] [gbkey=CDS]&#xA;MESLVPGFNEKTHVQLSLPVLQVRDVLVRGFGDSVEEVLSEARQHLKDGTCGLVEVEKGVLPQLEQPYVF&#xA;IKRSDARTAPHGHVMVELVAELEGIQYGRSGETLGVLVPHVGEIPVAYRKVLLRKNGNKGAGGHSYGADL&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17190" LastActivityDate="2023-03-18T08:45:20.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20742" PostTypeId="1" CreationDate="2023-03-19T18:30:07.323" Score="2" ViewCount="116" Body="&lt;p&gt;I ran the following command trying to download a fasta file from NIH but the command doesn't seem to terminate and I get no output file in my repository:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fastq-dump SRR10134980&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have installed SRA Toolkit following the instructions on &lt;a href=&quot;https://github.com/ncbi/sra-tools/wiki/02.-Installing-SRA-Toolkit&quot; rel=&quot;nofollow noreferrer&quot;&gt;this page&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17197" LastEditorUserId="73" LastEditDate="2023-03-20T18:46:02.580" LastActivityDate="2023-03-20T18:46:02.580" Title="sra toolkit fastq-dump comand not terminating" Tags="|sratoolkit|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20743" PostTypeId="1" AcceptedAnswerId="20746" CreationDate="2023-03-19T23:38:47.023" Score="3" ViewCount="291" Body="&lt;p&gt;I'm attempting to run a nextflow pipeline and have begun getting unexpected output for a few of the processes in my pipeline. I believe this to be a syntax error, but I am not sure where to correct it. Any help is finding where my error is located is appreciated.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pxfDb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pxfDb.png&quot; alt=&quot;Error&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env nextflow&#xA; nextflow.enable.dsl=2&#xA; import groovy.io.*&#xA;&#xA;&#xA;&#xA;&#xA;  process hisat2Alignment {&#xA;    fastq1 = NS.1389.002.NEBNext_dual_i7_*_R1.fastq.gz&#xA;    fastq2 = NS.1389.002.NEBNext_dual_i7_*_R2.fastq.gz&#xA;    input:&#xA;    tuple path(&amp;quot;fastq1&amp;quot;), path(&amp;quot;fastq2&amp;quot;)&#xA;&#xA;    output:&#xA;    tuple path(&amp;quot;*.sam&amp;quot;), val(fastq1.baseName.substring(0, fastq1.baseName.indexOf(&amp;quot;_R1&amp;quot;)))&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    hisat2 -p 16 -x genome -1 &lt;span class=&quot;math-container&quot;&gt;${fastq1} -2 $&lt;/span&gt;{fastq2} -S mapped_${baseName}.sam&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;&#xA; }&#xA;&#xA;process Stringtier {&#xA;    publishDir &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${params.baseDir}/assembled/$&lt;/span&gt;baseName&amp;quot;, mode: &amp;quot;move&amp;quot;&#xA;    env {&#xA;     PATH = $PATH:/data/selena/stringtie-2.2.1&#xA;    }&#xA;&#xA;    input:&#xA;    tuple path(mapped_bam), val(baseName)&#xA;    output:&#xA;    tuple path(&amp;quot;*.gtf&amp;quot;), val(baseName)&#xA;&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    stringtie -e -G &lt;span class=&quot;math-container&quot;&gt;${params.annotation} -o $&lt;/span&gt;{baseName}.gtf ${mapped_bam}&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA; workflow {&#xA;    samples = new File(params.configFilePath).readLines()*.split('\t');&#xA;    samples  = Channel.fromList(samples);&#xA;    hisat2Alignment(samples);&#xA;    assembled = Stringtier(samples)&#xA;    &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17198" LastEditorUserId="17198" LastEditDate="2023-03-20T04:56:23.447" LastActivityDate="2023-03-20T13:20:15.533" Title="Nextflow Pipeline: Unexpected Input &quot;{&quot;" Tags="|rna-seq|nextflow|errors|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20746" PostTypeId="2" ParentId="20743" CreationDate="2023-03-20T06:44:59.540" Score="2" Body="&lt;p&gt;Nextflow does not have an &lt;code&gt;env&lt;/code&gt; process directive. If &lt;code&gt;stringtie&lt;/code&gt; is not already in your &lt;code&gt;$PATH&lt;/code&gt;, you'll need to append it before running &lt;code&gt;nextflow&lt;/code&gt;, for example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;export PATH=&amp;quot;${PATH}:/data/selena/stringtie-2.2.1&amp;quot;&#xA;nextflow run de-pipeline.nf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, you may prefer to avoid local tool installations altogether, and use either the &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#conda&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;conda&lt;/code&gt;&lt;/a&gt; or &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#container&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;container&lt;/code&gt;&lt;/a&gt; process directives. The latter will however require a container runtime, like &lt;a href=&quot;https://www.nextflow.io/docs/latest/container.html#docker&quot; rel=&quot;nofollow noreferrer&quot;&gt;Docker&lt;/a&gt; or &lt;a href=&quot;https://www.nextflow.io/docs/latest/container.html#singularity&quot; rel=&quot;nofollow noreferrer&quot;&gt;Singularity&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You may also consider including a 'sample name' or some sort of sample identifier in your config file, which I suspect is just a simple tab-separated file of filenames. This would greatly simplify your workflow. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params.samplesheet = './samples.tsv'&#xA;params.genome = './genomes/GRCh38/GRCh38'&#xA;params.gtf = './Homo_sapiens.GRCh38.gtf'&#xA;&#xA;genome_name = file( params.genome ).name&#xA;genome_path = file( params.genome ).parent&#xA;&#xA;params.outdir = './results'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;process HISAT2 {&#xA;&#xA;    tag { sample }&#xA;&#xA;    publishDir &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${params.outdir}/HISAT2/$&lt;/span&gt;{sample}&amp;quot;, mode: 'copy'&#xA;&#xA;    cpus 16&#xA;&#xA;    input:&#xA;    tuple val(sample), path(reads)&#xA;    path genome_dir&#xA;&#xA;    output:&#xA;    tuple val(sample), path(&amp;quot;${sample}.sam&amp;quot;)&#xA;&#xA;    script:&#xA;    def (r1, r2) = reads&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    hisat2 \\&#xA;        -p &lt;span class=&quot;math-container&quot;&gt;${task.cpus} \\&#xA;        -x &quot;$&lt;/span&gt;{genome_dir}/&lt;span class=&quot;math-container&quot;&gt;${genome_name}&quot; \\&#xA;        -1 &quot;$&lt;/span&gt;{r1}&amp;quot; \\&#xA;        -2 &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${r2}&quot; \\&#xA;        -S &quot;$&lt;/span&gt;{sample}.sam&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;process SAMtools_sort {&#xA;&#xA;    tag { sample }&#xA;&#xA;    input:&#xA;    tuple val(sample), path(bam)&#xA;&#xA;    output:&#xA;    tuple val(sample), path(&amp;quot;${sample}.sorted.bam&amp;quot;)&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    samtools sort \\&#xA;        -o &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample}.sorted.bam&quot; \\&#xA;        &quot;$&lt;/span&gt;{bam}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;process StringTie {&#xA;&#xA;    tag { sample }&#xA;&#xA;    publishDir &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${params.outdir}/StringTie/$&lt;/span&gt;{sample}&amp;quot;, mode: 'copy'&#xA;&#xA;    input:&#xA;    tuple val(sample), path(bam)&#xA;    path gtf&#xA;&#xA;    output:&#xA;    tuple val(sample), path(&amp;quot;${sample}.gtf&amp;quot;)&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    stringtie \\&#xA;        -e \\&#xA;        -G &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${gtf}&quot; \\&#xA;        -o &quot;$&lt;/span&gt;{sample}.gtf&amp;quot; \\&#xA;        &amp;quot;${bam}&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow {&#xA;&#xA;    gtf = file( params.gtf )&#xA;&#xA;    Channel&#xA;        .fromPath( params.samplesheet )&#xA;        .splitCsv( sep: '\t' )&#xA;        .map { sample, fq1, fq2 -&amp;gt;&#xA;            def reads = tuple( file(fq1), file(fq2) )&#xA;&#xA;            tuple( sample, reads )&#xA;        }&#xA;        .set { samples }&#xA;&#xA;    HISAT2( samples, genome_path )&#xA;&#xA;    SAMtools_sort( HISAT2.out )&#xA;&#xA;    StringTie( SAMtools_sort.out, gtf )&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Where the contents of the input sample sheet (e.g. &lt;code&gt;samples.tsv&lt;/code&gt;) looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sample_A    /path/to/reads/A_R1.fastq.gz    /path/to/reads/A_R2.fastq.gz&#xA;sample_B    /path/to/reads/B_R1.fastq.gz    /path/to/reads/B_R2.fastq.gz&#xA;sample_C    /path/to/reads/C_R1.fastq.gz    /path/to/reads/C_R2.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ nextflow run main.nf&#xA;N E X T F L O W  ~  version 22.04.0&#xA;Launching `main.nf` [adoring_poisson] DSL2 - revision: 43e8c59949&#xA;executor &amp;gt;  local (9)&#xA;[60/1cba2f] process &amp;gt; HISAT2 (sample_A)        [100%] 3 of 3 ✔&#xA;[0a/931a07] process &amp;gt; SAMtools_sort (sample_A) [100%] 3 of 3 ✔&#xA;[6d/7689a2] process &amp;gt; StringTie (sample_A)     [100%] 3 of 3 ✔&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="298" LastEditDate="2023-03-20T13:20:15.533" LastActivityDate="2023-03-20T13:20:15.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20747" PostTypeId="1" CreationDate="2023-03-20T07:37:21.833" Score="2" ViewCount="128" Body="&lt;p&gt;I have data that looks like this: 3 column SNPs their gene based on Annovar and a p-value for every SNP. What I would like is to aggregate the p values for every gene.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;snps &amp;lt;- data.frame(&#xA;  snp_id = c(&amp;quot;rs1&amp;quot;, &amp;quot;rs2&amp;quot;, &amp;quot;rs3&amp;quot;, &amp;quot;rs4&amp;quot;, &amp;quot;rs5&amp;quot;, &amp;quot;rs6&amp;quot;, &amp;quot;rs7&amp;quot;, &amp;quot;rs8&amp;quot;),&#xA;  gene = c(&amp;quot;gene1&amp;quot;, &amp;quot;gene1&amp;quot;, &amp;quot;gene1&amp;quot;, &amp;quot;gene1&amp;quot;, &amp;quot;gene2&amp;quot;, &amp;quot;gene2&amp;quot;, &amp;quot;gene2&amp;quot;, &amp;quot;gene2&amp;quot;),&#xA;  pvalue = c(0.7703884, 0.9648540, 0.9648540, 0.9648540, 0.54, 0.03, 0.03, 0.8)&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;what I need to do is :&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;For each gene, sort the p-values in ascending order.&lt;/li&gt;&#xA;&lt;li&gt;For each p-value in the sorted list, multiply it by the number of SNPs in that gene.&lt;/li&gt;&#xA;&lt;li&gt;Divide the result from step 2 by the rank of the p-value in the sorted list (i.e., its index in the list).&lt;/li&gt;&#xA;&lt;li&gt;Take the minimum value from step 3 as the new p-value for that gene.&lt;/li&gt;&#xA;&lt;li&gt;Adjust the p-values using the false discovery rate (FDR) correction method.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This is the formula for each gene :combined p-value &lt;span class=&quot;math-container&quot;&gt;$\min(n p_{(i)}/i)$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$p_{(1)}, …, p_{(n)}$&lt;/span&gt; are the ordered p-values.&lt;/p&gt;&#xA;&lt;p&gt;I tried this code  with the usage of &lt;code&gt;simes.test&lt;/code&gt; package in R:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(mppa)&#xA;&#xA;x=c(0.77,0.964,0.964,0.964)&#xA;simes.test(x)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The answer is 1.285. This is the source code of the &lt;code&gt;simes.test&lt;/code&gt; function:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;function (x, returnstat = FALSE) &#xA;{&#xA;    r = rank(x)&#xA;    T = min(length(x) * x/r)&#xA;    if (returnstat) &#xA;        c(T, T)&#xA;    else T&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How is it possible for the pvalue to be larger then 1? What is the problem? I would be happy if you could share your knowledge in this situation.&lt;/p&gt;&#xA;" OwnerUserId="17200" LastEditorUserId="650" LastEditDate="2023-03-20T19:17:52.190" LastActivityDate="2023-03-20T19:17:52.190" Title="Simes method in R" Tags="|r|statistics|p-values|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20748" PostTypeId="2" ParentId="20742" CreationDate="2023-03-20T13:36:57.033" Score="1" Body="&lt;p&gt;That's an 11GB file and fastq-dump is not a speedster. Without more info the likely answer is that it keeps downloading which might take several hours on  poor connection. You can get fastq download links from sra-explorer.info -- That is more stable than this tool.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastEditorUserId="16676" LastEditDate="2023-03-20T15:44:30.463" LastActivityDate="2023-03-20T15:44:30.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20749" PostTypeId="1" CreationDate="2023-03-20T16:47:24.393" Score="1" ViewCount="74" Body="&lt;p&gt;I have a tsv file generated from abricate (&lt;a href=&quot;https://github.com/tseemann/abricate&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/tseemann/abricate&lt;/a&gt;). I need to convert them to gff3 format with certain columns retained, certain columns reordered, while other columns deleted.&lt;/p&gt;&#xA;&lt;p&gt;We are trying to use these gff3 files for downstream applications and for piping into other applications. However, we could not solve it.&lt;/p&gt;&#xA;&lt;p&gt;Below are examples of my tsv files, what possibly we may need to do, and desired output files in gff3 format.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Any help will be much appreciated.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Input tsv file:&lt;/h2&gt;&#xA;&lt;p&gt;FILE SEQUENCE START END GENE COVERAGE COVERAGE_MAP GAPS %COVERAGE %IDENTITY DATABASE ACCESSION PRODUCT&lt;/p&gt;&#xA;&lt;p&gt;UBird_Cyou_D3.fna BJCZ01000001.1 1866608 1867417 cdtB 1-810/810 =============== 0/0 100 90 vfdb CAD48850 (cdtB) cytolethal distending toxin B [CDT (VF0185)] [Escherichia coli O157:H str. 493/89] UBird_Cyou_D3.fna BJCZ01000001.1 1867414 1868190 cdtA 1-777/777 =============== 0/0 100 90.61 vfdb CAD48849 (cdtA) cytolethal distending toxin A [CDT (VF0185)] [Escherichia coli O157:H str. 493/89] UBird_Cyou_D3.fna BJCZ01000001.1 2245186 2246238 ompA 1-1041/1041 ========/====== 1/12 100 94.11 vfdb AAF37887 (ompA) outer membrane protein A [OmpA (VF0236)] [Escherichia coli O18:K1:H7 str. RS218]&lt;/p&gt;&#xA;&lt;h2&gt;What we may need to do (there may be other ways too, I am not sure):&lt;/h2&gt;&#xA;&lt;pre&gt;&lt;code&gt;Row 1 (always starts with &amp;quot;#&amp;quot;) - Need to replace with the string &amp;quot;##gff-version 3&amp;quot;&#xA;Col 1 - get rid of &amp;quot;.fna&amp;quot; and retain other data&#xA;Insert new Col - print the string from Col 11 for all rows&#xA;Col 2 - get rid of entire column&#xA;Insert new Col - print &amp;quot;CDS&amp;quot; for all rows&#xA;Col 3 - retain data&#xA;Col 4 - retain data&#xA;Insert new Col and print &amp;quot;.&amp;quot; for all rows&#xA;Insert new Col and print &amp;quot;+&amp;quot; for all rows&#xA;Insert new Col and print &amp;quot;0&amp;quot; for all rows&#xA;Col 5 to Col 10 - get rid of all these columns and data&#xA;Col 11 - delete column&#xA;Col 13 - retain data except &amp;quot;(&amp;quot;, &amp;quot;)&amp;quot;, &amp;quot;[&amp;quot;, &amp;quot;]&amp;quot;&#xA;Add new Col - Starting with &amp;quot;ID=&amp;quot; followed by the string taken from Col 1 and a underscore added (for the example data &amp;quot;UBird_Cyou_D3_&amp;quot;) alongwith numerals starting from 1 and incrementing by 1. This column data needs to be appended by &amp;quot;product=&amp;quot; followed by data from the corresponding row of the modified Col 13. The separator between ID string and product string should be &amp;quot;;&amp;quot;. After completion this column should be like &amp;quot;ID=UBird_Cyou_D3_1;product=cdtB cytolethal distending toxin B CDT VF0185 Escherichia coli O157:H str. 493/89&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Desired final output (*.gff3) considering the example data:&lt;/h2&gt;&#xA;&lt;p&gt;##gff-version 3&lt;/p&gt;&#xA;&lt;p&gt;UBird_Cyou_D3 vfdb CDS 187 756 . + 0 ID=UBird_Cyou_D3_1;product=cdtB cytolethal distending toxin B CDT VF0185 Escherichia coli O157:H str. 493/89&lt;/p&gt;&#xA;" OwnerUserId="17202" LastActivityDate="2023-03-21T00:31:32.203" Title="Convert Abricate output tsv file to gff3 format" Tags="|gff3|tsv|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20752" PostTypeId="2" ParentId="20749" CreationDate="2023-03-21T00:31:32.203" Score="2" Body="&lt;p&gt;Here's one way using &lt;a href=&quot;/questions/tagged/awk&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; aria-label=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; rel=&quot;tag&quot; aria-labelledby=&quot;tag-awk-tooltip-container&quot;&gt;awk&lt;/a&gt;, that I think fulfills the requirements. It adds each of the column names (on the first line) to an array to make accessing each of the fields a bit easier. This approach isn't strictly necessary, but it does make for a more readable solution in my opinion. With the following in a file called &lt;code&gt;tsv2gff3.awk&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BEGIN {&#xA;    FS=OFS=&amp;quot;\t&amp;quot;&#xA;    c=1&#xA;}&#xA;&#xA;NR==1 &amp;amp;&amp;amp; sub(/^#/, &amp;quot;&amp;quot;) {&#xA;    for(i=1; i&amp;lt;=NF; i++) {&#xA;        a[$i]=i&#xA;    }&#xA;&#xA;    print &amp;quot;##gff-version 3&amp;quot;&#xA;    next&#xA;}&#xA;&#xA;{&#xA;    sub(/\.fna&lt;span class=&quot;math-container&quot;&gt;$/, &quot;&quot;, $&lt;/span&gt;a[&amp;quot;FILE&amp;quot;])&#xA;    gsub(/[\(\)\[\]]/, &amp;quot;&amp;quot;, $a[&amp;quot;PRODUCT&amp;quot;])&#xA;&#xA;    id = &amp;quot;ID=&amp;quot; &lt;span class=&quot;math-container&quot;&gt;$a[&quot;FILE&quot;] &quot;_&quot; c++&#xA;    product = &quot;product=&quot; $&lt;/span&gt;a[&amp;quot;PRODUCT&amp;quot;]&#xA;&#xA;    print \&#xA;        &lt;span class=&quot;math-container&quot;&gt;$a[&quot;FILE&quot;],&#xA;        $&lt;/span&gt;a[&amp;quot;DATABASE&amp;quot;],&#xA;        &amp;quot;CDS&amp;quot;,&#xA;        &lt;span class=&quot;math-container&quot;&gt;$a[&quot;START&quot;],&#xA;        $&lt;/span&gt;a[&amp;quot;END&amp;quot;],&#xA;        &amp;quot;.&amp;quot;,&#xA;        &amp;quot;+&amp;quot;,&#xA;        &amp;quot;0&amp;quot;,&#xA;        id &amp;quot;;&amp;quot; product&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the following in &lt;code&gt;file.tsv&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#FILE   SEQUENCE    START   END GENE    COVERAGE    COVERAGE_MAP    GAPS    %COVERAGE   %IDENTITY   DATABASE    ACCESSION   PRODUCT&#xA;UBird_Cyou_D3.fna   BJCZ01000001.1  1866608 1867417 cdtB    1-810/810   =============== 0/0 100 90  vfdb    CAD48850    (cdtB) cytolethal distending toxin B [CDT (VF0185)] [Escherichia coli O157:H str. 493/89]&#xA;UBird_Cyou_D3.fna   BJCZ01000001.1  1867414 1868190 cdtA    1-777/777   =============== 0/0 100 90.61   vfdb    CAD48849    (cdtA) cytolethal distending toxin A [CDT (VF0185)] [Escherichia coli O157:H str. 493/89]&#xA;UBird_Cyou_D3.fna   BJCZ01000001.1  2245186 2246238 ompA    1-1041/1041 ========/====== 1/12    100 94.11   vfdb    AAF37887    (ompA) outer membrane protein A [OmpA (VF0236)] [Escherichia coli O18:K1:H7 str. RS218]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Run using &lt;code&gt;awk -f tsv2gff3.awk file.tsv&lt;/code&gt;, results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##gff-version 3&#xA;UBird_Cyou_D3   vfdb    CDS 1866608 1867417 .   +   0   ID=UBird_Cyou_D3_1;product=cdtB cytolethal distending toxin B CDT VF0185 Escherichia coli O157:H str. 493/89&#xA;UBird_Cyou_D3   vfdb    CDS 1867414 1868190 .   +   0   ID=UBird_Cyou_D3_2;product=cdtA cytolethal distending toxin A CDT VF0185 Escherichia coli O157:H str. 493/89&#xA;UBird_Cyou_D3   vfdb    CDS 2245186 2246238 .   +   0   ID=UBird_Cyou_D3_3;product=ompA outer membrane protein A OmpA VF0236 Escherichia coli O18:K1:H7 str. RS218&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-03-21T00:31:32.203" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20755" PostTypeId="1" AcceptedAnswerId="20757" CreationDate="2023-03-21T02:13:57.663" Score="3" ViewCount="249" Body="&lt;p&gt;I'm reading Subramanian et. al's (2005) original GSEA &lt;a href=&quot;https://www.pnas.org/doi/10.1073/pnas.0506580102&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;. In the paper, the authors mention the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We noticed that the use of&#xA;weighted steps could cause the distribution of observed ES scores to&#xA;be asymmetric in cases where many more genes are correlated with one&#xA;of the two phenotypes. We therefore estimate the significance levels&#xA;by considering separately the positively and negatively scoring gene&#xA;sets&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I understand what they're saying, but I simply don't understand why it's a problem. Basically, they're saying that in some cases, most genes in the list we're testing could appear mostly at the top or bottom of the list. But I don't understand why having asymmetric distributions is a problem. What is the issue here?&lt;/p&gt;&#xA;&lt;p&gt;EDIT: Here are some example GSEA plots I found via a web search with such gene sets. Where is the problem?&#xA;&lt;a href=&quot;https://i.stack.imgur.com/qJdzT.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qJdzT.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;One thing I don't get though is why this problem would be introduced through &amp;quot;weighting&amp;quot; of the steps. I feel like it would also be a problem without weighting. The way it is phrased in the paper seems to suggest that it's the weighting that causes an issue.&lt;/p&gt;&#xA;" OwnerUserId="8601" LastEditorUserId="73" LastEditDate="2023-03-22T20:22:23.993" LastActivityDate="2023-05-09T21:17:54.697" Title="Why are asymmetric distributions an issue for GSEA?" Tags="|gsea|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20756" PostTypeId="1" CreationDate="2023-03-21T11:30:41.197" Score="0" ViewCount="182" Body="&lt;p&gt;I use the diffdock (&lt;a href=&quot;https://github.com/gcorso/DiffDock&quot; rel=&quot;nofollow noreferrer&quot;&gt;link to diffdock on github&lt;/a&gt;) to dock a peptide with protein and this tool give me a ligand in sdf file format. I traslate this ligand sdf format file to a pdb format file and find the pdbfile lack res information. I wonder if tools/code can automaticly complete this file. I need the UNK become the residue types like pdb.This my error pdb file（this structure is a peptide）&lt;a href=&quot;https://i.stack.imgur.com/lQjJR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lQjJR.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/lQjJR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/lQjJR.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;HETATM    1  C   UNK     0       9.797  22.501  -6.484  0.00  0.00           C &#xA;HETATM    2  C   UNK     0      10.366  21.086  -6.446  0.00  0.00           C &#xA;HETATM    3  O   UNK     0       9.550  20.218  -7.135  0.00  0.00           O &#xA;HETATM    4  N   UNK     0      11.710  21.084  -6.978  0.00  0.00           N &#xA;HETATM    5  C   UNK     0      11.793  20.201  -8.123  0.00  0.00           C &#xA;HETATM    6  C   UNK     0      11.151  20.760  -9.346  0.00  0.00           C &#xA;HETATM    7  C   UNK     0      11.586  20.031 -10.581  0.00  0.00           C &#xA;HETATM    8  C   UNK     0      13.000  20.308 -10.986  0.00  0.00           C &#xA;HETATM    9  C   UNK     0      13.397  19.716 -12.309  0.00  0.00           C &#xA;HETATM   10  N   UNK     0      12.636  18.523 -12.633  0.00  0.00           N &#xA;HETATM   11  C   UNK     0      11.861  18.392 -13.830  0.00  0.00           C &#xA;HETATM   12  O   UNK     0      11.837  19.364 -14.611  0.00  0.00           O &#xA;HETATM   13  C   UNK     0      11.148  17.131 -14.065  0.00  0.00           C &#xA;HETATM   14  C   UNK     0      11.800  16.246 -15.085  0.00  0.00           C &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16933" LastEditorUserId="10049" LastEditDate="2023-03-23T22:41:35.907" LastActivityDate="2023-03-24T11:25:00.343" Title="complete the res type in sdf" Tags="|pdb|peptide-shaker|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20757" PostTypeId="2" ParentId="20755" CreationDate="2023-03-21T21:00:04.993" Score="1" Body="&lt;p&gt;The problem is that the enrichment score is modified based on whether the sets are positively or negatively associated, which means that an unbalanced set via either associative statistic or gene count (as is almost always the case) will lead to an incorrect total score.&lt;/p&gt;&#xA;&lt;p&gt;Put in other words, the calculated score from the positive genes alters the calculated score from the negative genes.&lt;/p&gt;&#xA;&lt;p&gt;Given this, it's a better idea to either separately calculate scores for the positive set and the negative set (this is the easiest approach, and works with existing methods), or (equivalently) recalculate enrichment score modifiers for positive and negative genes separately (i.e. adjust the graph) so that the zero point for the enrichment score crosses the zero point for association.&lt;/p&gt;&#xA;&lt;p&gt;A common argument against this (when I have mentioned it to others) is that GSEA has &lt;em&gt;always&lt;/em&gt; been done on the total gene set, so it makes sense to do it on the total gene set because it has always been done that way. I'm not a huge fan of the status-quo argument.&lt;/p&gt;&#xA;&lt;p&gt;I alluded to this issue in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/15963/73&quot;&gt;this previous answer&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;With regards to your figures:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;a&lt;/strong&gt;. Zero-point crossover is at an enrichment score of ~0.42, so the negative set will be underemphasised. It's not contributing much anyway (except for the one most negative gene at the end), so wouldn't affect the outcome much.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;b&lt;/strong&gt;. Zero-point crossover is at an enrichment score of ~0.45. Similar situation as in &lt;strong&gt;a&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;c&lt;/strong&gt;. Zero-point crossover is at an enrichment score of ~0.32. There aren't really any positive genes with substantial association scores, so no issue.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;d&lt;/strong&gt;. Plot looks to be identical to &lt;strong&gt;c&lt;/strong&gt;, so the same comments apply.&lt;/p&gt;&#xA;&lt;p&gt;The scores are most affected when they have a lot of area on both sides of the Y axis. The top right plot of this sub-figure is an example of that, showing a sharp-ish tail on the negative side, so the calculated enrichment score for the negative peak is less extreme than it would be if only the negative genes were considered [&lt;a href=&quot;https://www.nature.com/articles/s41590-021-01067-0/figures/8&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;]:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gYveS.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gYveS.jpg&quot; alt=&quot;GSEA plots with zero points marked; most plots show a large steep curve for positive genes, with a small hook for negative genes&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The way it is phrased in the paper seems to suggest that it's the weighting that causes an issue.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes. If the positive genes are weighted separately from the negative genes, then the bias can be removed. I did briefly attempt a non-parametric weighting for GSEA (based on rank), but got too confused by the maths involved. Someone answered my question about it, but I haven't had to do any GSEA since then, so haven't actually verified if their solution works for me:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://math.stackexchange.com/questions/3567714/probability-that-the-nth-highest-of-a-set-of-k-numbers-selected-from-m-numbers-h&quot;&gt;https://math.stackexchange.com/questions/3567714/probability-that-the-nth-highest-of-a-set-of-k-numbers-selected-from-m-numbers-h&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-05-09T21:17:54.697" LastActivityDate="2023-05-09T21:17:54.697" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20758" PostTypeId="2" ParentId="20756" CreationDate="2023-03-22T07:25:21.180" Score="1" Body="&lt;p&gt;An SDF file lacks residue information, including atom name.&#xA;Although for the latter, the &lt;code&gt;molFileAlias&lt;/code&gt; atomic property is frequently used.&lt;/p&gt;&#xA;&lt;h2&gt;Adding residue info&lt;/h2&gt;&#xA;&lt;p&gt;There are several ways to do this, such as in RDKit (&lt;a href=&quot;https://blog.matteoferla.com/2020/03/atom-names-purely-in-rdkit.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;blog post&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from rdkit import Chem&#xA;from rdkit.Chem import AllChem&#xA;&#xA;from typing import List&#xA;&#xA;with Chem.SDMolSupplier('.sdf') as mol_store:&#xA;     mols: List[Chem.Mol] = list(mol_store)&#xA;&#xA;mol: Chem.Mol&#xA;for mol in mols:&#xA;    atom: Chem.Atom&#xA;    for i, atom in enumerate(mol.GetAtoms()):&#xA;        info = AllChem.AtomPDBResidueInfo(atomName=f'ATM{i}',&#xA;                                          residueName='LIG',&#xA;                                          residueNumber=1, &#xA;                                          chainId='X')&#xA;        atom.SetMonomerInfo(info)   &#xA;&#xA;mol: Chem.Mol&#xA;for mol in mols:&#xA;    name: str = mol.GetProp('_Name')&#xA;    Chem.MolToPDBFile(mol, f'{name}.pdb')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Notes:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Atom names do have somewhat of a standard format, C1, C2, C3. I am being lazy in the above. If your ligand is present in the RCSB PDB, it will have a chemical component info on the PDB: &lt;a href=&quot;https://www.rcsb.org/ligand/CFF&quot; rel=&quot;nofollow noreferrer&quot;&gt;example&lt;/a&gt; and it is often imperative to obey the atom names and atom index from there.&lt;/li&gt;&#xA;&lt;li&gt;PyMOL or pymol2 module can also add PDB information, therein the residue index is not an integer but a string as it combines w/ insertion code.&lt;/li&gt;&#xA;&lt;li&gt;In some tools, you lose the bond order, but if you have a SDF and not a PDB, then you are safe. But if you have a reference it's easy to fix.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from rdkit import Chem&#xA;from rdkit.Chem import AllChem&#xA;template: Chem.Mol = Chem.MolFromSmiles('')&#xA;AllChem.AssignBondOrdersFromTemplate(template, mol)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6322" LastEditorUserId="10049" LastEditDate="2023-03-24T11:25:00.343" LastActivityDate="2023-03-24T11:25:00.343" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20759" PostTypeId="1" CreationDate="2023-03-22T07:40:21.223" Score="1" ViewCount="21" Body="&lt;p&gt;This is from this &lt;a href=&quot;https://www.nature.com/articles/nature20598&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The LSC17 score is calculated for each patient as a linear combination of GE of these 17 genes weighted by regression coefficients that were estimated from the training data as follows: &lt;code&gt;LSC17 score = (DNMT3B × 0.0874) + (ZBTB46 × −0.0347) + (NYNRIN × 0.00865) + (ARHGAP22 × −0.0138) + (LAPTM4B × 0.00582) + (MMRN1 × 0.0258) + (DPYSL3 × 0.0284) + (KIAA0125 × 0.0196) + (CDK6 × −0.0704) + (CPXM1 × −0.0258) + (SOCS2 × 0.0271) + (SMIM24 × −0.0226) + (EMP1 × 0.0146) + (NGFRAP1 × 0.0465) + (CD34 × 0.0338) + (AKR1C3 × −0.0402) + (GPR56 × 0.0501).&lt;/code&gt; As above- and below-median scores in the training cohort were associated with adverse and favourable cytogenetic risk, respectively, a &lt;strong&gt;median threshold&lt;/strong&gt; was used to discretize scores into high and low groups.&lt;/p&gt;&#xA;&lt;p&gt;I'm unable to understand or apply the median threshold to create patient group specifically the R implementation here.&lt;/p&gt;&#xA;&lt;p&gt;In order to address the problem my small dummy data-frame&#xA;This is my dummy data&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;set.seed(123)&#xA;nr1 = 20; nr2 = 8; nr3 = 6; nr = nr1 + nr2 + nr3&#xA;nc1 = 16; nc2 = 8; nc3 = 10; nc = nc1 + nc2 + nc3&#xA;mat = cbind(rbind(matrix(rnorm(nr1*nc1, mean = 1,   sd = 0.5), nr = nr1),&#xA;                  matrix(rnorm(nr2*nc1, mean = 0,   sd = 0.5), nr = nr2),&#xA;                  matrix(rnorm(nr3*nc1, mean = 0,   sd = 0.5), nr = nr3)),&#xA;            rbind(matrix(rnorm(nr1*nc2, mean = 0,   sd = 0.5), nr = nr1),&#xA;                  matrix(rnorm(nr2*nc2, mean = 1,   sd = 0.5), nr = nr2),&#xA;                  matrix(rnorm(nr3*nc2, mean = 0,   sd = 0.5), nr = nr3)),&#xA;            rbind(matrix(rnorm(nr1*nc3, mean = 0.5, sd = 0.5), nr = nr1),&#xA;                  matrix(rnorm(nr2*nc3, mean = 0.5, sd = 0.5), nr = nr2),&#xA;                  matrix(rnorm(nr3*nc3, mean = 1,   sd = 0.5), nr = nr3))&#xA;)&#xA;mat = mat[sample(nr, nr), sample(nc, nc)] # random shuffle rows and columns&#xA;rownames(mat) = paste0(&amp;quot;gene&amp;quot;, seq_len(nr))&#xA;colnames(mat) = paste0(&amp;quot;LSC&amp;quot;, seq_len(nc))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;They have generated the LSC17 score by doing lasso,here in this case will create a dummy 5 gene score such as this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;LSC5 score = (gene1 × 0.0874) + (gene6 × −0.0347) + (gene10 × 0.00865) + (gene22 × −0.0138) + (gene28× 0.00582) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Now to generate or categorize the samples into above or below the median score.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;How to implement the above step in R which Im unable to do it?&#xA;One suggestion I received is this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; combined into a single variable when you make LSC17. After that, it's just quantile(LSC17,probs=c(0.3333,0.6666)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which I unsuccessfully followed as I don't know how to implement it in the whole data frame.&lt;/p&gt;&#xA;&lt;p&gt;Any help or suggestion would be really appreciated&lt;/p&gt;&#xA;&lt;p&gt;I got an elegant solution which I asked &lt;a href=&quot;https://stats.stackexchange.com/questions/610125/using-multiple-genes-building-gene-signature-and-survival-analysis&quot;&gt;here&lt;/a&gt; If there are other ways in R to do this I would love to know&lt;/p&gt;&#xA;" OwnerUserId="14154" LastEditorUserId="14154" LastEditDate="2023-03-22T09:59:38.363" LastActivityDate="2023-03-22T09:59:38.363" Title="Generating gene signature for sample classification and survival analysis patient cohort" Tags="|r|rna-seq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20760" PostTypeId="1" CreationDate="2023-03-22T10:51:52.843" Score="1" ViewCount="53" Body="&lt;p&gt;I have a set of RNAseq data for a couple of relatively under-studied species.&lt;/p&gt;&#xA;&lt;p&gt;They're fungi, of the order Hypocreales. The assemblies we're working with are de novo - reference seqs don't actually exist yet for most of our species. What the collaborator provided was updated genome assemblies in 60-90 fragments as opposed to the original 250-300, but obviously that's still not full resolution. As for the gene families, not a whole lot is known... in fact our species' genes are still not well described and we're mostly working off of predicted orthology.&lt;/p&gt;&#xA;&lt;p&gt;During the analysis, a collaborator created new DNA scaffolds for these species which we'd now like to use.&lt;/p&gt;&#xA;&lt;p&gt;However when I redid our RNAseq using the existing reads with the new scaffolds, one of our species had the number of differentially-expressed genes drop by anywhere from 20-50% per condition.&lt;/p&gt;&#xA;&lt;p&gt;As the reads themselves haven't changed, we weren't expecting significant changes in differential gene expression, and indeed for our other species changes have been minimal.&lt;/p&gt;&#xA;&lt;p&gt;Does anybody know what might cause this kind of change?&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="17213" LastEditorUserId="73" LastEditDate="2023-03-23T20:01:19.083" LastActivityDate="2023-03-23T20:01:19.083" Title="RNAseq: Why would using a more complete scaffold reduce DEGs?" Tags="|rna-seq|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20761" PostTypeId="1" CreationDate="2023-03-22T14:00:13.160" Score="2" ViewCount="137" Body="&lt;p&gt;Since such a neat solution &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/20749/convert-abricate-output-tsv-file-to-gff3-format&quot;&gt;(abricate tsv to gff3)&lt;/a&gt; was provided by &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/10775/steve&quot;&gt;Steve&lt;/a&gt;, here are few other steps that I am looking to add so that the script progress to logical maturity to be usable by many others.&lt;/p&gt;&#xA;&lt;p&gt;I have two files - (1) fasta file with .fna extension, and (2) a data file with .tsv extension generated from &lt;a href=&quot;https://github.com/tseemann/abricate&quot; rel=&quot;nofollow noreferrer&quot;&gt;abricate&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I need to write a gff3 file (containing sequences) by combining these two files (.fna and .tsv)&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;A .gff3 file generally has three or four components as below.&lt;/em&gt; Details &lt;a href=&quot;https://asia.ensembl.org/info/website/upload/gff3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Declaration of file type and version&lt;/li&gt;&#xA;&lt;li&gt;Declaration of sequence region(s) starting from Name of the sequence, followed by single space, then 1 separated by single space, ending in numerical value of the sequence length of the fasta sequence (which is the .fna file).&lt;/li&gt;&#xA;&lt;li&gt;Sequence features&lt;/li&gt;&#xA;&lt;li&gt;Fasta sequence (optional)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;See below a truncated example&lt;/p&gt;&#xA;&lt;p&gt;##gff-version 3&#xA;&lt;br&gt;##sequence-region NZ_BHEI01000001 1 13890&#xA;&lt;br&gt;NZ_BHEI01000001 Prodigal    CDS 29  1588    .   +   0   ID=CEMHHHDG_00001;product=hypothetical protein&#xA;&lt;br&gt;NZ_BHEI01000001 Prodigal    CDS 1585    2289    .   +   0   ID=CEMHHHDG_00002;product=Putative beta-lactamase HcpC&#xA;&lt;br&gt;....&#xA;&lt;br&gt;##FASTA&#xA;&lt;br&gt;&amp;gt;NZ_BHEI01000001&#xA;&lt;br&gt;AAAAAAAGAATGCTTGCCTACTGGAGTAGTGCCTGTTACTTTTAAACCCACTTTTTTGCG&#xA;&lt;br&gt;TTATGCGGATGAGTATTTTTTAAGAGTTGAATTTCAAGATGGAAGTGATGAAATCACTCA&#xA;&lt;br&gt;TATAGAGGAGTTGGCAAAATACACAGATCAAA..........&lt;/p&gt;&#xA;&lt;p&gt;Notice that the sequence names has to match in every instance (declaration of sequence regions, sequence features, and the fasta header at the end)&lt;/p&gt;&#xA;&lt;p&gt;The .tsv to .gff3 conversion was elegantly solved by &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/10775/steve&quot;&gt;Steve&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now we need to add the declaration of sequence region(s) in the .gff3 file. The declaration comprises of&#xA;&lt;br&gt;&amp;quot;##sequence-region&amp;quot;[space]the input file name without the file extension .fna[space]sequence start[space]sequence length[\n]&lt;/p&gt;&#xA;&lt;p&gt;For the last part, I have used the following awk code on the .fna file to generate the length in desired format.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '/^&amp;gt;/ {if (seqlen); seqlen=0;next; } { seqlen += length($0)}END{print &amp;quot;1 &amp;quot; seqlen}' example.fna&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This output needs to be appended in the declaration of sequence region line.&lt;/p&gt;&#xA;&lt;p&gt;Now we need to append the fasta sequence in the following format&lt;/p&gt;&#xA;&lt;p&gt;&lt;br&gt;##FASTA&#xA;&lt;br&gt;&amp;gt;Sequence name as in the parts above in the .gff3 file&#xA;&lt;br&gt;AAAAAAAGAATGCTTGCCTACTGGAGTAGTGCCTGTTACTTTTAAACCCACTTTTTTGCG&#xA;&lt;br&gt;TTATGCGGATGAGTATTTTTTAAGAGTTGAATTTCAAGATGGAAGTGATGAAATCACTCA&#xA;&lt;br&gt;TATAGAGGAGTTGGCAAAATACACAGATCAAA..........&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is there any help in sight, please?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="17202" LastActivityDate="2023-03-29T06:28:15.943" Title="Improving conversion of abricate tsv file to gff3 file" Tags="|gff3|tsv|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20763" PostTypeId="1" CreationDate="2023-03-22T16:15:33.417" Score="0" ViewCount="25" Body="&lt;p&gt;I need to run multinomial logistic regression on a smaller set of variants e.g. using Trinculo. May I know if there's any existing workflow on Terra that can do this? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="17216" LastActivityDate="2023-03-23T13:18:51.110" Title="Any existing workflow on Terra that can run multinomial logistic regression?" Tags="|logistic-regression|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20764" PostTypeId="1" AcceptedAnswerId="20768" CreationDate="2023-03-22T17:58:13.240" Score="3" ViewCount="152" Body="&lt;p&gt;I am using the following code to download all pubmed central article ids that have a keyword appearing in their titles/abstracts and also falling between two dates. The code seems to work when I select database = &amp;quot;pubmed&amp;quot; but when I switch to database = &amp;quot;pmc&amp;quot; I get empty list. For example, using the start and end dates as mentioned in the code I get 4 PubMed ids: ['36902449', '36880658', '36583446', '35239250'] and when I check manually, the first one among them (which is '36902449') is also present in PMC database, of course, the id is different (10003631) but same article. The code is expected to fetch that id. I'd appreciate it if someone could point out what I'm missing here.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import os&#xA;import datetime&#xA;from Bio import Entrez&#xA;&#xA;# Set the email address to identify yourself to NCBI&#xA;Entrez.email = &amp;quot;example@gmail.com&amp;quot;&#xA;&#xA;# Set the date range for the search&#xA;start_date = datetime.datetime(2023, 3, 1)&#xA;end_date = datetime.datetime(2023, 3, 22)&#xA;&#xA;# Format the dates as strings in the correct format&#xA;start_date_str = start_date.strftime(&amp;quot;%Y/%m/%d&amp;quot;)&#xA;end_date_str = end_date.strftime(&amp;quot;%Y/%m/%d&amp;quot;)&#xA;&#xA;# Build the search query with the date range&#xA;keyword = &amp;quot;dosage compensation&amp;quot;&#xA;# database = &amp;quot;pubmed&amp;quot;&#xA;database = &amp;quot;pmc&amp;quot;&#xA;&#xA;# Use esearch to fetch the list of matching article IDs&#xA;if database == &amp;quot;pubmed&amp;quot;:&#xA;    search_term = f'(&amp;quot;{start_date_str}&amp;quot;[Date - Publication] : &amp;quot;{end_date_str}&amp;quot;[Date - Publication]) AND {keyword}[Title/Abstract]'&#xA;    handle = Entrez.esearch(db=database,&#xA;                            term=search_term,&#xA;                            retmax=100,&#xA;                            api_key = os.environ['ENTREZ_API_KEY'])&#xA;elif database == &amp;quot;pmc&amp;quot;:&#xA;    search_term = f'(&amp;quot;{start_date_str}&amp;quot;[Publication Date] : &amp;quot;{end_date_str}&amp;quot;[Publication Date]) AND {keyword}[Title/Abstract] AND open_access[filter]'&#xA;    handle = Entrez.esearch(db=database,&#xA;                        term=search_term,&#xA;                        retmax=1000,&#xA;                        api_key = os.environ['ENTREZ_API_KEY'])&#xA;&#xA;record = Entrez.read(handle)&#xA;id_list = record['IdList']&#xA;handle.close()&#xA;&#xA;# Print the number of matching articles&#xA;print(f&amp;quot;Found {len(id_list)} articles published between {start_date_str} and {end_date_str} in {database}&amp;quot;)&#xA;# print(id_list)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7177" LastEditorUserId="650" LastEditDate="2023-03-22T21:49:27.703" LastActivityDate="2023-03-23T05:05:56.330" Title="Download all pubmed central article ids that have a keyword appearing in their titles/abstracts and also falling between two dates" Tags="|python|biopython|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20765" PostTypeId="2" ParentId="20763" CreationDate="2023-03-23T01:32:19.980" Score="1" Body="&lt;p&gt;What I think is occurring is Terra is the &lt;a href=&quot;https://www.broadinstitute.org/videos/access-data-run-analysis-tools-and-collaborate-cloud-terra&quot; rel=&quot;nofollow noreferrer&quot;&gt;Broad Institute's cloud service&lt;/a&gt;. Within that there is a Jupiter notebook and you then just use scikit-learn's (Python) &lt;a href=&quot;https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;LogisticRegression()&lt;/a&gt; using multinomial option. Then is simply ....&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from sklearn.linear_model import LogisticRegression&#xA;import pandas as pd&#xA;model = LogisticRegression(multi_class='multinomial', solver='lbfgs')&#xA;df = pd.DataFrame ({'colA':[1,2,3], 'colB':[1,2,5] ... etc ...)&#xA;X = df[['colA', 'colB', 'colC']]&#xA;y = df[['colD']]&#xA;model.fit(X, y)&#xA;preds = model.predict(X) &#xA;print (preds)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This sort of operation. Does that make sense?&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-03-23T13:18:51.110" LastActivityDate="2023-03-23T13:18:51.110" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20766" PostTypeId="1" AcceptedAnswerId="20767" CreationDate="2023-03-23T02:10:10.200" Score="3" ViewCount="632" Body="&lt;p&gt;I am trying to run my nextflow pipeline, and have gotten this error:&lt;br /&gt;&#xA;&lt;code&gt;samtools sort: failed to read header from &amp;quot;-&amp;quot;&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm not sure why this error is occurring.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env nextflow&#xA;nextflow.enable.dsl=2&#xA;import groovy.io.*&#xA;&#xA;params.t = 8&#xA;&#xA;process joinFastq {     &#xA;input:&#xA;path(ds_directory)&#xA;&#xA;output:&#xA;tuple path(&amp;quot;joined_&lt;span class=&quot;math-container&quot;&gt;${ds_directory.baseName}.fastq&quot;), val(&quot;$&lt;/span&gt;ds_directory.baseName&amp;quot;)&#xA;&#xA;script:&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;gunzip -c &lt;span class=&quot;math-container&quot;&gt;${ds_directory}/*/fastq_pass/*.fastq.gz &amp;gt; &#xA; joined_$&lt;/span&gt;{ds_directory.baseName}.fastq&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;}&#xA; process minimap2 {&#xA;cpus 8&#xA;memory &amp;quot;20 GB&amp;quot;&#xA;input:&#xA;tuple path(fastq), val(baseName)&#xA;&#xA;output:&#xA;tuple path(&amp;quot;*.bam&amp;quot;), val(baseName)&#xA;&#xA;  script:&#xA;   &amp;quot;&amp;quot;&amp;quot;&#xA;  minimap2 -ax splice -uf -k14 -t 8 &lt;span class=&quot;math-container&quot;&gt;${params.genome} $&lt;/span&gt;{fastq} &amp;gt; mapped_&lt;span class=&quot;math-container&quot;&gt;${baseName}.sam&#xA;    samtools view -h -b mapped_$&lt;/span&gt;{baseName}.sam &amp;gt; &lt;span class=&quot;math-container&quot;&gt;${baseName}.bam&#xA;   samtools sort -o $&lt;/span&gt;{baseName}.bam &amp;gt; ${baseName}_sorted.bam&#xA;   &amp;quot;&amp;quot;&amp;quot;&#xA;   }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have tried applying the &lt;code&gt;-h&lt;/code&gt; flag to &lt;code&gt;samtools sort&lt;/code&gt; in the script&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   samtools sort -h -o &lt;span class=&quot;math-container&quot;&gt;${baseName}.bam &amp;gt; $&lt;/span&gt;{baseName}_sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and the same error is produced. I'm not sure why the header flag didn't solve this issue.&lt;/p&gt;&#xA;" OwnerUserId="17198" LastEditorUserId="6251" LastEditDate="2023-03-23T18:31:11.500" LastActivityDate="2023-03-23T18:31:11.500" Title="Nextflow Error: failed to read header from &quot;-&quot;" Tags="|rna-seq|samtools|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20767" PostTypeId="2" ParentId="20766" CreationDate="2023-03-23T03:28:44.053" Score="3" Body="&lt;p&gt;This is not a Nextflow error. The problem is that without an input file, samtools &lt;a href=&quot;https://www.htslib.org/doc/samtools-sort.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;sort&lt;/code&gt;&lt;/a&gt; tries to read from stdin. Using a recent &lt;code&gt;samtools&lt;/code&gt;, you can however coordinate sort the SAM and write a sorted BAM using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools sort -o &amp;quot;${baseName}.bam&amp;quot; &amp;quot;mapped_${baseName}.sam&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You may have been intending to pipe the output to &lt;code&gt;samtools sort&lt;/code&gt;, which would avoid writing large SAM files and is usually preferable. A minimal example might look like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params.reads = './*.fastq.gz'&#xA;params.reference = './gencode.v43.transcripts.fa'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;process minimap2 {&#xA;&#xA;    tag { sample }&#xA;&#xA;    cpus 8&#xA;    memory 20.GB&#xA;&#xA;    input:&#xA;    tuple val(sample), path(fastq)&#xA;    path ref_fasta&#xA;&#xA;    output:&#xA;    tuple val(sample), path(&amp;quot;${sample}.bam&amp;quot;)&#xA;&#xA;    script:&#xA;    def cpus = task.cpus &amp;gt; 1 ? task.cpus - 1 : task.cpus&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    minimap2 \\&#xA;        -ax splice \\&#xA;        -uf \\&#xA;        -k14 \\&#xA;        -t &lt;span class=&quot;math-container&quot;&gt;${cpus} \\&#xA;        &quot;$&lt;/span&gt;{ref_fasta}&amp;quot; \\&#xA;        &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${fastq}&quot; |&#xA;    samtools sort \\&#xA;        -o &quot;$&lt;/span&gt;{sample}.bam&amp;quot; &#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow {&#xA;&#xA;    reads = Channel.fromFilePairs( params.reads, size: 1 )&#xA;    reference = file( params.reference )&#xA;&#xA;    minimap2( reads, reference )&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Also, be sure to avoid using absolute paths in your workflow scripts. For your workflow to be portable, processes should only access files in the working directory. The example above avoids this, by using a &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#value-channel&quot; rel=&quot;nofollow noreferrer&quot;&gt;value channel&lt;/a&gt;, to stage the reference FASTA file into the working directory.&lt;/p&gt;&#xA;&lt;p&gt;In case anyone is wondering, the ternary is to ensure that we do not oversubscribe the node(s) that these jobs land on. Otherwise, each job could use (up to) 900% (800% + 100% = 900%) CPU.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2023-03-23T03:58:42.217" LastActivityDate="2023-03-23T03:58:42.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20768" PostTypeId="2" ParentId="20764" CreationDate="2023-03-23T05:05:56.330" Score="1" Body="&lt;p&gt;There seems to be a problem with using the &lt;code&gt;[Title/Abstract]&lt;/code&gt; field. What you have looks correct and should produce the expected result, since it should capture articles where the search term appears in either the title or in the abstract, or in both. But it does not. Strangely, you get the expected result using &lt;code&gt;{keyword}[Abstract]&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;A workaround could be to &lt;code&gt;OR&lt;/code&gt; the fields with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;f'(&amp;quot;{keyword}&amp;quot;[Title] OR &amp;quot;{keyword}&amp;quot;[Abstract])'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Perhaps something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;search_term = ' AND '.join([&#xA;    f'(&amp;quot;{start_date_str}&amp;quot;[Publication Date] : &amp;quot;{end_date_str}&amp;quot;[Publication Date])',&#xA;    f'(&amp;quot;{keyword}&amp;quot;[Title] OR &amp;quot;{keyword}&amp;quot;[Abstract])',&#xA;    f'(open_access[filter])',&#xA;])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-03-23T05:05:56.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20769" PostTypeId="2" ParentId="20760" CreationDate="2023-03-23T13:48:51.743" Score="1" Body="&lt;p&gt;Fungi, ain't a kingdom most of us will know much about. The only thing I can guess at is that in bacteria there has resurgence in methods for &lt;em&gt;de novo&lt;/em&gt; assembly specifically aimed at reducing the total number of contigs in an assembly. The 'new wave' is summarised in this response here, &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/20336/improving-prokaryotic-assembly-with-other-contig-scaffold-level-data&quot;&gt;Improving prokaryotic assembly with other contig/scaffold-level data?&lt;/a&gt;. This is potentially very exciting, except the OP reported they didn't work ;-(&lt;/p&gt;&#xA;&lt;p&gt;A reasonable guess is your colleague has deployed these approaches to reduce the number of contigs. HOWEVER, what is appropriate for bacteria, may not be appropriate for fungi, alternatively it may be an improvement. Both bateria and fungi are small genomes, but beyond that there could notable differences.&lt;/p&gt;&#xA;&lt;p&gt;If fungi are dominated by multigene families these might be being removed in the new wave &lt;em&gt;de novo&lt;/em&gt; assemblies (they are designed for bacteria). The collapse of multigene families absolutely occurs in other eukaryotes with small genomes, but they have to be really big on multigene families (&amp;gt;10% of genes in their genome). If that was true you would probably know about it.&lt;/p&gt;&#xA;&lt;p&gt;Alternative, this could simply be poor gene identification in the first assembly resulting in an artificial jump in the number of genes.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Solution?&lt;/strong&gt; Either way you should be able to figure which genes are different between the assemblies and precisely what type of genes are differing, e.g. are these large multi-gene families. Personally, I would pipe the predicted genes from both assemblies through &lt;code&gt;eggNOG&lt;/code&gt;. This might be useful, but not really what it's designed for, but it will return the KEGG nos.  More explicit methods based around comparing the RNAseq reads against the assembly via e.g. &lt;code&gt;bedtools&lt;/code&gt; or &lt;a href=&quot;https://wikis.utexas.edu/display/bioiteam/Gene+Counting&quot; rel=&quot;nofollow noreferrer&quot;&gt;these guys&lt;/a&gt; here recommend &lt;code&gt; htseq&lt;/code&gt;. This might be a better way to rephrase the question.&lt;/p&gt;&#xA;&lt;p&gt;Thus I recommend:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Finding out which &lt;em&gt;de novo&lt;/em&gt; methods were used between first and second fungi genome assemblies. Do any of those methods match this post, &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/20336/improving-prokaryotic-assembly-with-other-contig-scaffold-level-data&quot;&gt;Improving prokaryotic assembly with other contig/scaffold-level data?&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Consider a more explicit genomics study to examine RNAseq reads against the genome, or comparison between predicted no. genes between genomes.&lt;/li&gt;&#xA;&lt;li&gt;Having said all that ... once you've done your DEG you  identify the KEGG no. and then simply compare the lists directly to find out which genes are different between the assemblies. This is an easy database operation to perform, e.g. via a &lt;code&gt;merge&lt;/code&gt; function (separate question).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I'd check point 1 and then go for point 3 - because it's something I easily know how to do. Thats just my opinion, a purist would likely explore this via point 2.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt; What I am certain about is that for a species where very little might be known it is better to perform an investigation rather than make an assumption that might prove later to be wrong.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-03-23T13:59:28.623" LastActivityDate="2023-03-23T13:59:28.623" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20770" PostTypeId="1" CreationDate="2023-03-23T14:12:39.537" Score="3" ViewCount="91" Body="&lt;p&gt;I have an input file looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; CD630_08510_19&#xA;Query  121  TGGCTTGGGTTTATATTTGATACTTTAGGAACATTTACAATGAGTAACATAGCCAACAGC  180&#xA;            |||||||| |||||||||||||||||||||||||||||||||||||||||||||||||||&#xA;Sbjct  121  TGGCTTGGTTTTATATTTGATACTTTAGGAACATTTACAATGAGTAACATAGCCAACAGC  180&#xA;&#xA;Query  241  GCAATTGTGCTTATGCTTTTTCATGCAAGTTGGGCTACTTTTGTCCTATACAAAGATGAT  300&#xA;            |||||||| |||||||||||||||||||||||||||||||||||||||||||||||||||&#xA;Sbjct  241  GCAATTGTACTTATGCTTTTTCATGCAAGTTGGGCTACTTTTGTCCTATACAAAGATGAT  300&#xA;&#xA;Query  361  CCATACTTTATAGGTATGTTTATTGGTATGGTTGGATAG  399&#xA;            ||||| ||||| ||||||||||||||||||| |||||||&#xA;Sbjct  361  CCATATTTTAT-GGTATGTTTATTGGTATGGCTGGATAG  398&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The input file is the result file after running the command line blast.&#xA;It's a txt file generated after blast alignment and in txt file header format are like this.&lt;/p&gt;&#xA;&lt;p&gt;I am trying to extract the specific nucleotide base for position 129, 249, 366, 372 and 392 from query and subject aligned sequence. python and perl solution will be fine as well. thank you.&lt;/p&gt;&#xA;&lt;p&gt;For example: I need my output to have format below like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Seq_name        base_position       query_base  subject_base    all_subject_base&#xA;CD630_08510_19  129                 G           T               TAT-C&#xA;CD630_08510_19  249                 G           A&#xA;CD630_08510_19  366                 C           T&#xA;CD630_08510_19  372                 A           -&#xA;CD630_08510_19  392                 T           C&#xA;&#xA;CD630_08510_20  129                 G           G               GATCC&#xA;CD630_08510_20  249                 G           A&#xA;CD630_08510_20  366                 C           T&#xA;CD630_08510_20  372                 A           C&#xA;CD630_08510_20  392                 T           C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;all_subject_base: are the bases from position 129, 249, 366, 372 and 392 of subject_base column. I want to keep this in one column thats why all_subject_base.&lt;/p&gt;&#xA;&lt;p&gt;I tried to extract this with grep but have not been able to. I am new in bioinformatics so any suggestion would be very helpful.&lt;/p&gt;&#xA;" OwnerUserId="17221" LastEditorUserId="73" LastEditDate="2023-03-23T19:42:06.923" LastActivityDate="2023-03-27T22:28:51.427" Title="Extract specific nucleotide base position for query and subject from text file" Tags="|blast|linux|shell|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20771" PostTypeId="1" CreationDate="2023-03-24T00:03:31.093" Score="0" ViewCount="37" Body="&lt;p&gt;I am trying to identify genes that are overall considered to be &amp;quot;actively transcribed&amp;quot; in my cell line of interest for some downstream analysis.&lt;/p&gt;&#xA;&lt;p&gt;What would the approach be for defining this subset of active genes from an RNA-seq experiments? Any genes with &amp;gt; 1 mapped fragment? Is there a minimum fpkm you would use? or are there some statistics that I can use to identify with some degree of certainty genes that are mostly active in my cell line?&lt;/p&gt;&#xA;&lt;p&gt;FYI, I am not doing a comparative experiment to identify DEGs, just trying to get a baseline of a group of genes that I can safely considered to be &amp;quot;active&amp;quot; in my cell line.&lt;/p&gt;&#xA;" OwnerUserId="9870" LastEditorUserId="9870" LastEditDate="2023-03-24T16:59:12.883" LastActivityDate="2023-03-24T16:59:12.883" Title="How would you define a set of active genes from a bulk RNA-seq experiment in one biological sample?" Tags="|rna-seq|" AnswerCount="0" CommentCount="3" ClosedDate="2023-03-24T13:52:16.567" ContentLicense="CC BY-SA 4.0" />
  <row Id="20772" PostTypeId="1" CreationDate="2023-03-24T06:31:13.947" Score="0" ViewCount="60" Body="&lt;p&gt;I have data that looks like this: 3 column SNPs their gene based on Annovar and a p-value for every SNP. What I would like is to aggregate the p values for every gene.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;snps &amp;lt;- data.frame(&#xA;  snp_id = c(&amp;quot;rs1&amp;quot;, &amp;quot;rs2&amp;quot;, &amp;quot;rs3&amp;quot;, &amp;quot;rs4&amp;quot;, &amp;quot;rs5&amp;quot;, &amp;quot;rs6&amp;quot;, &amp;quot;rs7&amp;quot;, &amp;quot;rs8&amp;quot;),&#xA;  Gene.refGene_ANNOVAR = c(&amp;quot;gene1&amp;quot;, &amp;quot;gene1&amp;quot;, &amp;quot;gene1&amp;quot;, &amp;quot;gene1&amp;quot;, &amp;quot;gene2&amp;quot;, &amp;quot;gene2&amp;quot;, &amp;quot;gene2&amp;quot;, &amp;quot;gene2&amp;quot;),&#xA;  p.value = c(0.7703884, 0.9648540, 0.9648540, 0.9648540, 0.54, 0.03, 0.03, 0.8)&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;above an example of the data .&#xA;I read about the SKAT method -&amp;gt; &lt;a href=&quot;https://www.hsph.harvard.edu/skat/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.hsph.harvard.edu/skat/&lt;/a&gt;&lt;br /&gt;&#xA;and figured it might do the work .&#xA;I read about the package here:&lt;a href=&quot;https://rdrr.io/cran/SKAT/man/SKAT.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rdrr.io/cran/SKAT/man/SKAT.html&lt;/a&gt;&#xA;tried to implement it on my data , but got lost as how to perform it correctly:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gene_pvals &amp;lt;- aggregate(p.value ~ Gene.refGene_ANNOVAR, data = df, FUN = function(x) SKAT::SKAT_Null(x)$p.value)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would be happy if you could share your knowledge in this situation.I don't have information about the correlations between the SNPs but I know that they are correlated, do I have enough data to complete the SKAT method?&#xA;also for every SNP i have data on how many patients had the hetro/homozygous encoding(0,1,2).&lt;/p&gt;&#xA;" OwnerUserId="17200" LastEditorUserId="17200" LastEditDate="2023-04-01T10:34:10.767" LastActivityDate="2023-04-01T10:34:10.767" Title="SKAT method in R" Tags="|r|gene|snp|p-values|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20773" PostTypeId="1" AcceptedAnswerId="20774" CreationDate="2023-03-24T11:02:01.783" Score="2" ViewCount="253" Body="&lt;pre&gt;&lt;code&gt;bwa mem ../01.data/GCF_000146045.2_R64_genomic.fna ../01.data/${sra}_1.fastq.gz ../01.data/${sra}_2.fastq.gz | \&#xA;    samtools view -S -b -t ../01.data/GCF_000146045.2_R64_genomic.fna.fai - | \&#xA;    samtools sort - &amp;gt;$sra.sorted.bam&#xA;samtools index $sra.sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am mapping the downloaded yeast sequences in sra format to the reference genome in a bash shell script.  And I am using samtools to change to bam file.  Then I try to sort the resulted bam file.&lt;/p&gt;&#xA;&lt;p&gt;I could follow the steps but I could not understand the use of hyphen after the .fai (index file) and the use of this hyphen before the pipe.  Moreover, what is the use of the hyphen after the samtools sort?&lt;/p&gt;&#xA;" OwnerUserId="17226" LastEditorUserId="3694" LastEditDate="2023-03-24T12:21:28.120" LastActivityDate="2023-03-25T10:24:44.920" Title="Problem with samtools command with a final hyphen" Tags="|samtools|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20774" PostTypeId="2" ParentId="20773" CreationDate="2023-03-24T11:13:49.880" Score="4" Body="&lt;p&gt;It allows you to chain commands together to avoid creating intermediate files and simplify your workflow:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Samtools is designed to work on a stream. It regards an input file `-'&#xA;as the standard input (stdin) and an output file `-' as the standard&#xA;output (stdout). Several commands can thus be combined with Unix&#xA;pipes. Samtools always output warning and error messages to the&#xA;standard error output (stderr).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is a common idiom in the *nix world, and is actually part of the &lt;a href=&quot;https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;POSIX specification&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Guideline 13:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;For utilities that use operands to represent files to be opened for either reading or writing, the '-' operand should be used to mean only standard input (or standard output when it is clear from context that an output file is being specified) or a file named -.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="298" LastEditDate="2023-03-24T13:01:56.060" LastActivityDate="2023-03-24T13:01:56.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20775" PostTypeId="2" ParentId="20773" CreationDate="2023-03-24T13:03:01.297" Score="4" Body="&lt;p&gt;The hyphen &amp;quot;-&amp;quot; character is commonly used as a placeholder in Unix commands to represent the standard input (stdin) or standard output (stdout) streams. From the &lt;a href=&quot;https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap12.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;POSIX specification&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Guideline 13:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;For utilities that use operands to represent files to be opened for&#xA;either reading or writing, the '-' operand should be used to mean only&#xA;standard input (or standard output when it is clear from context that&#xA;an output file is being specified) or a file named -.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In your pipeline, the hyphen after the index file name indicates that the output of the &lt;code&gt;bwa mem&lt;/code&gt; command should be piped to &lt;a href=&quot;https://www.htslib.org/doc/samtools-view.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;samtools view&lt;/code&gt;&lt;/a&gt;, which will then read it from the standard input stream. The hyphen after the &lt;a href=&quot;https://www.htslib.org/doc/samtools-sort.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;samtools sort&lt;/code&gt;&lt;/a&gt; command, is not strictly necessary since an input SAM/BAM/CRAM is entirely optional. Note that recent versions of &lt;code&gt;samtools&lt;/code&gt; let you coordinate sort the SAM and write a sorted BAM, so you can do away with the call to &lt;code&gt;samtools view&lt;/code&gt; entirely. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;set -euo pipefail&#xA;&#xA;bwa mem \&#xA;    ../01.data/GCF_000146045.2_R64_genomic.fna \&#xA;    &amp;quot;../01.data/&lt;span class=&quot;math-container&quot;&gt;${sra}_1.fastq.gz&quot; \&#xA;    &quot;../01.data/$&lt;/span&gt;{sra}_2.fastq.gz&amp;quot; |&#xA;samtools sort \&#xA;    -o &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sra}.sorted.bam&quot;&#xA;samtools index \&#xA;    &quot;$&lt;/span&gt;{sra}.sorted.bam&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you wanted to, you could replace the hyphens with the &lt;code&gt;/dev/stdin&lt;/code&gt; special file, which represents the standard input streams. Sometimes this can make more complex commands a bit more readable, but note that not all Unix systems support the &lt;code&gt;/dev/stdin&lt;/code&gt; and &lt;code&gt;/dev/stdout&lt;/code&gt; special files. Most modern Unix systems do however.&lt;/p&gt;&#xA;&lt;p&gt;The above uses &lt;code&gt;set -eo pipefail&lt;/code&gt; to ensure pipeline errors are not masked. This exits immediately if any command in the pipeline exits with a non-zero status.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2023-03-24T13:28:50.690" LastActivityDate="2023-03-24T13:28:50.690" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20776" PostTypeId="1" CreationDate="2023-03-25T07:54:56.447" Score="-3" ViewCount="118" Body="&lt;p&gt;How do we plot histogram in R\Python with two variables[BC and Control] and their corresponding gene names examples is below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/W6oID.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W6oID.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thank you,&lt;/p&gt;&#xA;" OwnerUserId="14059" LastEditorUserId="3848" LastEditDate="2023-03-31T11:23:15.407" LastActivityDate="2023-06-27T19:55:35.763" Title="Histogram plot using R/python with two variables[BC and Control] and their corresponding gene names" Tags="|r|python|proteins|ggplot2|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20778" PostTypeId="2" ParentId="20776" CreationDate="2023-03-25T15:16:55.687" Score="1" Body="&lt;p&gt;Real data is better, particularly if genes names are wanted (complicated simulation) .... anyway there you go. Simulated values under a negative binomial which are pretty realistic to the data supplied (that'll be negative binomial, just not ranked).&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;import numpy as np&#xA;import pandas as pd&#xA;&#xA;np.random.seed(2)&#xA;s1 = list(np.random.negative_binomial(1, 0.1, 500))&#xA;s2 = list(np.random.negative_binomial(1, 0.5, 500))&#xA;mydict = {'A':s1, 'B':s2}&#xA;df = pd.DataFrame(mydict)&#xA;fig, ax = plt.subplots()&#xA;&#xA;heights, bins = np.histogram(df['A'])&#xA;heights2, bins2 = np.histogram(df['B'], bins=bins)&#xA;&#xA;width = (bins[1] - bins2[0])/3&#xA;&#xA;ax.bar(bins[:-1], heights, width=width, facecolor='coral')&#xA;ax.bar(bins2[:-1]+width, heights2, width=width, facecolor='blue')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gYD7d.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gYD7d.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Here's horizontal bars (minor differences) with slightly different negative binomial values used ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# same imports as before &#xA;&#xA;np.random.seed(2)&#xA;s1 = list(np.random.negative_binomial(1, 0.1, 500))&#xA;s2 = list(np.random.negative_binomial(1, 0.3, 500))&#xA;mydict = {'A':s1, 'B':s2}&#xA;df = pd.DataFrame(mydict)&#xA;fig, ax = plt.subplots()&#xA;&#xA;heights, bins = np.histogram(df['A'])&#xA;heights2, bins2 = np.histogram(df['B'], bins=bins)&#xA;&#xA;width = (bins[1] - bins2[0])/3&#xA;&#xA;ax.barh(bins[:-1], heights, width, facecolor='coral')&#xA;ax.barh(bins2[:-1]+width, heights2, width, facecolor='blue')&#xA;ax.invert_yaxis()&#xA;plt.show()&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Kf2qA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Kf2qA.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To adapt it you just need to provide an appropriate set of values that will then be fitted to a dataframe.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="650" LastEditDate="2023-06-27T19:55:35.763" LastActivityDate="2023-06-27T19:55:35.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20779" PostTypeId="1" CreationDate="2023-03-26T18:35:17.013" Score="1" ViewCount="254" Body="&lt;p&gt;I can't seem to find a clear answer to this question, so here it goes:&lt;/p&gt;&#xA;&lt;p&gt;I have sequenced scRNASeq + scVDJSeq (TCR) data, which has been sequenced using feature barcoding from 10x genomics, via antibody capture.&lt;/p&gt;&#xA;&lt;p&gt;There is a very useful &lt;code&gt;cellranger multi&lt;/code&gt; command, which allows us to process both this data types in 'one go'.&lt;/p&gt;&#xA;&lt;p&gt;In order to run it one needs to generate a &lt;code&gt;csv&lt;/code&gt; file a priori, such that &lt;code&gt;cellranger&lt;/code&gt; is able to identify which files are pertinent to which file type - see this link:&lt;a href=&quot;https://support.10xgenomics.com/single-cell-vdj/software/pipelines/latest/using/multi&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.10xgenomics.com/single-cell-vdj/software/pipelines/latest/using/multi&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the link above, with the example given under &amp;quot;Example multi config CSVs&amp;quot;, it seems as though there are separate &lt;code&gt;fastq&lt;/code&gt; files for the feature barcode, versus for the actual gene expression samples. Whereas, in my case, I only have the standard &lt;code&gt;R1&lt;/code&gt; and &lt;code&gt;R2&lt;/code&gt; fastq files, which is creating some confusion as how to generate this file, and run cellranger.&lt;/p&gt;&#xA;&lt;p&gt;Now my question is, how does one run &lt;code&gt;cellranger multi&lt;/code&gt; with feature barcoding should there only be the standard 2 &lt;code&gt;fastq&lt;/code&gt; files? My understanding is that the feature barcode is usually in the &lt;code&gt;R2&lt;/code&gt; read? Is this correct? Would I need to Demultiplex? I am completely lost as to how to proceed. If you know, could you please provide a short example of what your &lt;code&gt;csv&lt;/code&gt; files would look like, should you not have separate &lt;code&gt;fastq&lt;/code&gt; files for the feature barcode files?&lt;/p&gt;&#xA;&lt;p&gt;Thanks,&lt;/p&gt;&#xA;" OwnerUserId="4001" LastEditorUserId="15489" LastEditDate="2023-03-28T20:54:41.060" LastActivityDate="2023-03-28T20:54:41.060" Title="Running cellranger on scRNASeq data with feature barcoding (10x + antibody capture)" Tags="|scrnaseq|fastq|10x-genomics|cellranger|demultiplexing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20780" PostTypeId="1" CreationDate="2023-03-27T00:14:28.603" Score="1" ViewCount="55" Body="&lt;p&gt;I installed the latest ucsc-bedgraphtobigwig (v377) with conda in a new enviroment enviroment inmy WSL2.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;conda install -c bioconda ucsc-bedgraphtobigwig&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, when I try using the package, I am getting his error.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;free(): invalid pointer&#xA;Aborted&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried downloading the package straight from UCSC, and when I run it, I also get some sort of memory error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;double free or corruption (out)&#xA;Aborted&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Has anyone experienced something similar? I tried downgrading to v357, and I am still getting the same issue.&lt;/p&gt;&#xA;" OwnerUserId="9870" LastActivityDate="2023-03-27T00:14:28.603" Title="ucsc-bedgraphtobigwig throwing memory allocation error" Tags="|ucsc|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20781" PostTypeId="1" AcceptedAnswerId="20786" CreationDate="2023-03-27T08:09:17.797" Score="0" ViewCount="59" Body="&lt;p&gt;I am currently working on a project where I have genetic data for patients classified into three different groups. Specifically, I have information on 26 SNPs for each patient. I have recorded each allele using letters. I am trying to decide on the best approach to analyze this data and would appreciate any guidance or suggestions.&#xA;Firstly, I need help formatting the SNP information for analysis. I have recorded each allele using a comma-separated format (e.g., &amp;quot;A,G&amp;quot;). However, other encoding methods may be more appropriate for analyzing SNP data. Could someone please suggest a suitable encoding method for my data?&#xA;Additionally, I am interested in performing machine learning and cluster analysis on this dataset to investigate the relationship between genotype and patient group. Please recommend any specific algorithms or packages that may be useful for this type of analysis in either Python or R.&#xA;Thank you in advance for your help and guidance.&lt;/p&gt;&#xA;" OwnerUserId="17236" LastActivityDate="2023-03-28T18:12:32.793" Title="Help with analyzing SNP data for three patient groups" Tags="|snp|correlation|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20782" PostTypeId="1" CreationDate="2023-03-27T09:19:17.047" Score="-1" ViewCount="53" Body="&lt;p&gt;I have the sequencing data of a Monkeypox virus sample. I have the list of variants extracted from Nextclade and I want to find the corresponding aminoacids substitution.&#xA;For example I have the variant C1092T (nucleotide annotation) and I want to automatically convert it into OPG001:D162N (aminoacid annotation).&lt;/p&gt;&#xA;" OwnerUserId="8933" LastEditorUserId="8933" LastEditDate="2023-03-27T11:27:53.863" LastActivityDate="2023-03-27T15:31:02.773" Title="Variants list to aminoacids" Tags="|ngs|snp|format-conversion|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20783" PostTypeId="1" AcceptedAnswerId="20813" CreationDate="2023-03-27T13:45:30.133" Score="4" ViewCount="98" Body="&lt;p&gt;Using the new &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Datasets&lt;/a&gt; platform, you can browse the collection of genomes associated with one or more taxa. For example, searching &lt;em&gt;Pseudomonas aeruginosa&lt;/em&gt; &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/data-hub/genome/?taxon=287&quot; rel=&quot;nofollow noreferrer&quot;&gt;returns 19,878 genomes&lt;/a&gt; as of 29 March 2023.&lt;/p&gt;&#xA;&lt;p&gt;In the search filtering tab, they give the option to &amp;quot;Exclude atypical genomes&amp;quot;. Applying this option to the previous search &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/data-hub/genome/?taxon=287&amp;amp;typical_only=true&quot; rel=&quot;nofollow noreferrer&quot;&gt;returns 19,737 genomes&lt;/a&gt;, implying that 141 genomes were discarded as atypical.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What criteria are they using to label genomes &amp;quot;atypical&amp;quot;?&lt;/strong&gt; I've read the documentation for the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genome/annotation_prok/process/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Prokaryotic Genome Annotation Process&lt;/a&gt;, and my hunch is that atypical genomes are those that are annotated with &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/29773659/&quot; rel=&quot;nofollow noreferrer&quot;&gt;atypical genes&lt;/a&gt;, though I'm not sure.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XFrzc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XFrzc.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3967" LastEditorUserId="3967" LastEditDate="2023-03-29T14:19:20.623" LastActivityDate="2023-03-31T14:58:29.060" Title="What is the NCBI's definition of an &quot;atypical genome&quot;?" Tags="|genome|data-download|ncbi|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20784" PostTypeId="2" ParentId="20782" CreationDate="2023-03-27T15:31:02.773" Score="1" Body="&lt;p&gt;Neither &lt;code&gt;C1092T&lt;/code&gt; nor &lt;code&gt;OPG001:D162N&lt;/code&gt; seem to follow HGVS conventions. You could try to convert them to HGVS nomenclature standards then use modules such as &lt;a href=&quot;https://hgvs.readthedocs.io/en/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;hgvs&lt;/a&gt; to get amino acid changes from nucleotide changes. You will need reference sequence identifiers though, so it might be easier to go from chromosomal level changes to AA changes, as cDNA changes depend on the transcript (which may or may not be an issue with viral transcripts).&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-03-27T15:31:02.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20785" PostTypeId="2" ParentId="20779" CreationDate="2023-03-27T15:38:22.457" Score="2" Body="&lt;p&gt;Check your FASTQ headers - if they've been demultiplexed, the index barcodes can be found in the header. The barcodes are usually separated by a &lt;code&gt;+&lt;/code&gt; in the comment part (second segment when the header is split by &lt;code&gt;&amp;quot; &amp;quot;&lt;/code&gt; blank space) of the header. If you see a match to &lt;code&gt;[ATGC]{10}[+][ATGC]{10}&lt;/code&gt; in the header, the first &lt;code&gt;[ATGC]{10}&lt;/code&gt; is the I1 barcode and the second is the I2 barcode. You can use &lt;code&gt;FFFFFFFFFF&lt;/code&gt; placeholder qual scores to generate &amp;quot;fake&amp;quot; I1 and I2 files and then use that with cellranger.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-03-27T15:38:22.457" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20786" PostTypeId="2" ParentId="20781" CreationDate="2023-03-27T19:43:52.493" Score="0" Body="&lt;p&gt;&lt;strong&gt;Overview&lt;/strong&gt; In the first instance unsupervised learning is needed - namely PCA - and strongly recommended to identify a training target. The point of this analysis is the resulting clusters of different 'patients' (?).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;How to do it?&lt;/strong&gt; The first part of the analysis is to perform one-hot-key coding.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;df = pd.read_csv({'A':['A', 'G', 'C'], 'B':['T', 'A', 'G']})&#xA;one_hot_encode = pd.get_dummies(df, columns = ['A', 'B'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This converts A, G, C, T to numerical values and then gets passed to unsupervised learning.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Supervised learning&lt;/strong&gt; Moving into supervised ML is not trivial, even with a suitable 'target', unless a simple algorithm such as naive Bayes is used. First I'd do the PCA analysis. ...&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt; the training target could simply be the three patient groups.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-03-28T18:12:32.793" LastActivityDate="2023-03-28T18:12:32.793" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20787" PostTypeId="2" ParentId="20770" CreationDate="2023-03-27T22:28:51.427" Score="2" Body="&lt;p&gt;Usually, when you run your BLAST alignment you specify the output format (e.g. as described in &amp;quot;outfmt&amp;quot; in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK279684/table/appendices.T.options_common_to_all_blast/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/books/NBK279684/table/appendices.T.options_common_to_all_blast/&lt;/a&gt;). If, for some reason, you can't re-run the BLAST command (e.g. if this output file was given to you, or if you can't access your query or reference sequence/s) you &lt;em&gt;can&lt;/em&gt; parse this 'readable' output format but it's not straightforward.&lt;/p&gt;&#xA;&lt;p&gt;Here is a potential solution using awk:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat positions.txt&#xA;129&#xA;249&#xA;366&#xA;372&#xA;392&#xA;&#xA;cat input.txt&#xA;&amp;gt; CD630_08510_19&#xA;Query  121  TGGCTTGGGTTTATATTTGATACTTTAGGAACATTTACAATGAGTAACATAGCCAACAGC  180&#xA;            |||||||| |||||||||||||||||||||||||||||||||||||||||||||||||||&#xA;Sbjct  121  TGGCTTGGTTTTATATTTGATACTTTAGGAACATTTACAATGAGTAACATAGCCAACAGC  180&#xA;&#xA;Query  241  GCAATTGTGCTTATGCTTTTTCATGCAAGTTGGGCTACTTTTGTCCTATACAAAGATGAT  300&#xA;            |||||||| |||||||||||||||||||||||||||||||||||||||||||||||||||&#xA;Sbjct  241  GCAATTGTACTTATGCTTTTTCATGCAAGTTGGGCTACTTTTGTCCTATACAAAGATGAT  300&#xA;&#xA;Query  361  CCATACTTTATAGGTATGTTTATTGGTATGGTTGGATAG  399&#xA;            ||||| ||||| ||||||||||||||||||| |||||||&#xA;Sbjct  361  CCATATTTTAT-GGTATGTTTATTGGTATGGCTGGATAG  398&#xA;&amp;gt; CD630_08510_20&#xA;Query  121  TGGCTTGGATTTATATTTGATACTTTAGGAACATTTACAATGAGTAACATAGCCAACAGC  180&#xA;            |||||||| |||||||||||||||||||||||||||||||||||||||||||||||||||&#xA;Sbjct  121  TGGCTTGGGTTTATATTTGATACTTTAGGAACATTTACAATGAGTAACATAGCCAACAGC  180&#xA;&#xA;Query  241  GCAATTGTGCTTATGCTTTTTCATGCAAGTTGGGCTACTTTTGTCCTATACAAAGATGAT  300&#xA;            |||||||| |||||||||||||||||||||||||||||||||||||||||||||||||||&#xA;Sbjct  241  GCAATTGTACTTATGCTTTTTCATGCAAGTTGGGCTACTTTTGTCCTATACAAAGATGAT  300&#xA;&#xA;Query  361  CCATACTTTATAGGTATGTTTATTGGTATGGTTGGATAG  399&#xA;            ||||| ||||| ||||||||||||||||||| |||||||&#xA;Sbjct  361  CCATATTTTAT-GGTATGTTTATTGGTATGGCTGGATAG  398&#xA;&#xA;awk 'BEGIN {&#xA;    OFS = &amp;quot;\t&amp;quot;&#xA;}&#xA;&#xA;NR == FNR {&#xA;    a[$0]&#xA;    next&#xA;}&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;$1 == &quot;&amp;gt;&quot; {&#xA;b[++c] = $&lt;/span&gt;2&#xA;}&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;$1 != &quot;&amp;gt;&quot; {&#xA;for (i in a) {&#xA;    if ($1 == &quot;Query&quot; &amp;amp;&amp;amp; i &amp;gt;= $2 &amp;amp;&amp;amp; i &amp;lt;= $4) {&#xA;        d[++e] = b[c] OFS i OFS substr($3, (i - $2) + 1, 1)&#xA;    } else if ($1 == &quot;Sbjct&quot; &amp;amp;&amp;amp; i &amp;gt;= $2 &amp;amp;&amp;amp; i &amp;lt;= $4) {&#xA;        f[++g] = substr($3, (i - $2) + 1, 1)&#xA;        h[c] = h[c] substr($3, (i - $&lt;/span&gt;2) + 1, 1)&#xA;        }&#xA;    }&#xA;}&#xA;&#xA;END {&#xA;    printf &amp;quot;%s\n&amp;quot;, &amp;quot;Seq_name\tbase_position\tquery_base\tsubject_base\tall_subject_base&amp;quot;&#xA;    for (j = 1; j &amp;lt;= e; j++) {&#xA;        printf &amp;quot;%s\t%s\t%s\n&amp;quot;, d[j], f[j], (j % length(a) == 1 ? h[int(j / length(a)) + 1] : &amp;quot;&amp;quot;)&#xA;    }&#xA;}' positions.txt input.txt&#xA;Seq_name    base_position   query_base  subject_base    all_subject_base&#xA;CD630_08510_19  129 G   T   TA-CT&#xA;CD630_08510_19  249 G   A&#xA;CD630_08510_19  372 A   -&#xA;CD630_08510_19  392 T   C&#xA;CD630_08510_19  366 C   T&#xA;CD630_08510_20  129 A   G   GA-CT&#xA;CD630_08510_20  249 G   A&#xA;CD630_08510_20  372 A   -&#xA;CD630_08510_20  392 T   C&#xA;CD630_08510_20  366 C   T&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you have more positions:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat positions.txt&#xA;129&#xA;249&#xA;366&#xA;372&#xA;373&#xA;374&#xA;375&#xA;376&#xA;392&#xA;&#xA;awk 'BEGIN {&#xA;    OFS = &amp;quot;\t&amp;quot;&#xA;}&#xA;&#xA;NR == FNR {&#xA;    a[$0]&#xA;    next&#xA;}&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;$1 == &quot;&amp;gt;&quot; {&#xA;b[++c] = $&lt;/span&gt;2&#xA;}&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;$1 != &quot;&amp;gt;&quot; {&#xA;for (i in a) {&#xA;    if ($1 == &quot;Query&quot; &amp;amp;&amp;amp; i &amp;gt;= $2 &amp;amp;&amp;amp; i &amp;lt;= $4) {&#xA;        d[++e] = b[c] OFS i OFS substr($3, (i - $2) + 1, 1)&#xA;    } else if ($1 == &quot;Sbjct&quot; &amp;amp;&amp;amp; i &amp;gt;= $2 &amp;amp;&amp;amp; i &amp;lt;= $4) {&#xA;        f[++g] = substr($3, (i - $2) + 1, 1)&#xA;        h[c] = h[c] substr($3, (i - $&lt;/span&gt;2) + 1, 1)&#xA;        }&#xA;    }&#xA;}&#xA;&#xA;END {&#xA;    printf &amp;quot;%s\n&amp;quot;, &amp;quot;Seq_name\tbase_position\tquery_base\tsubject_base\tall_subject_base&amp;quot;&#xA;    for (j = 1; j &amp;lt;= e; j++) {&#xA;        printf &amp;quot;%s\t%s\t%s\n&amp;quot;, d[j], f[j], (j % length(a) == 1 ? h[int(j / length(a)) + 1] : &amp;quot;&amp;quot;)&#xA;    }&#xA;}' positions.txt input.txt&#xA;Seq_name    base_position   query_base  subject_base    all_subject_base&#xA;CD630_08510_19  129 G   T   TA-GGTACT&#xA;CD630_08510_19  249 G   A&#xA;CD630_08510_19  372 A   -&#xA;CD630_08510_19  373 G   G&#xA;CD630_08510_19  374 G   G&#xA;CD630_08510_19  375 T   T&#xA;CD630_08510_19  376 A   A&#xA;CD630_08510_19  392 T   C&#xA;CD630_08510_19  366 C   T&#xA;CD630_08510_20  129 A   G   GA-GGTACT&#xA;CD630_08510_20  249 G   A&#xA;CD630_08510_20  372 A   -&#xA;CD630_08510_20  373 G   G&#xA;CD630_08510_20  374 G   G&#xA;CD630_08510_20  375 T   T&#xA;CD630_08510_20  376 A   A&#xA;CD630_08510_20  392 T   C&#xA;CD630_08510_20  366 C   T&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;NB this won't work if you only have a single position in positions.txt, but if that is a potential use-case you can modify the script to suit, e.g. replace &lt;code&gt;(j % length(a) == 1 ? h[int(j / length(a)) + 1] : &amp;quot;&amp;quot;)&lt;/code&gt; with &lt;code&gt;f[j]&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="9098" LastActivityDate="2023-03-27T22:28:51.427" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20788" PostTypeId="1" AcceptedAnswerId="20790" CreationDate="2023-03-28T13:39:04.243" Score="2" ViewCount="55" Body="&lt;p&gt;When visuliazing a &lt;a href=&quot;https://en.wikipedia.org/wiki/Phylogenetic_tree#Phylogram&quot; rel=&quot;nofollow noreferrer&quot;&gt;phylogram&lt;/a&gt; is it preferable to show ticks on the x axis or a scale bar?&lt;/p&gt;&#xA;&lt;p&gt;Given an example tree with branch lengths in newick format &lt;code&gt;(A:0.01,(B:0.02,C:0.05):0.02);&lt;/code&gt;, the two options are illustrated in the plot below:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XsQAC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XsQAC.png&quot; alt=&quot;A figure comprising two subplots, on the left a phylogram with ticks on the x axis, on the right a phylogram with a scale bar&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15054" LastEditorUserId="15054" LastEditDate="2023-03-28T16:08:35.613" LastActivityDate="2023-03-28T16:27:00.613" Title="Using ticks on the x axis or a scale bar when visualisizing a phylogram?" Tags="|phylogenetics|phylogeny|visualization|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20790" PostTypeId="2" ParentId="20788" CreationDate="2023-03-28T16:20:31.130" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/XsQAC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/XsQAC.png&quot; alt=&quot;A figure comprising two subplots, on the left a phylogram with ticks on the x axis, on the right a phylogram with a scale bar&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Both will work. The convention is the scale-bar and the rationale is below (which is a bit complicated).&lt;/p&gt;&#xA;&lt;p&gt;By convention:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the right-hand tree where there is a scale-bar and is the traditional diagram for phylograms.&lt;/li&gt;&#xA;&lt;li&gt;the left-hand tree scale-bar (&amp;quot;x-axis&amp;quot;) is used for molecular clock representations, i.e. not for phylograms, but dendrograms.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In terms of clarity the &amp;quot;x-axis&amp;quot; (or extended scale bar) appears clearer. However, firstly, for a seasoned phylogenecist it makes no difference and simplicity is preferred. Secondly, there's coalescent theory.&lt;/p&gt;&#xA;&lt;p&gt;The left-hand tree ... the thing is in a molecular clock representation the scale is a &lt;strong&gt;reverse axes&lt;/strong&gt; starting at the right-hand tips. Whether thats time or genetic diversity its a good representation (well for time its essential) &lt;em&gt;because&lt;/em&gt; coalescent theory means that there is zero genetic diversity for that taxa at time 0 (the idea is you are extrapolating, or reconstructing, the common ancestor). The representation you presented confuses that convention because 0 starts at the assumed node in the outgroup.&lt;/p&gt;&#xA;&lt;p&gt;Thus the scale-bar is better.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-03-28T16:27:00.613" LastActivityDate="2023-03-28T16:27:00.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20792" PostTypeId="1" AcceptedAnswerId="20793" CreationDate="2023-03-28T17:57:12.750" Score="2" ViewCount="104" Body="&lt;p&gt;I am using Manta (&lt;a href=&quot;https://github.com/Illumina/manta/blob/master/docs/userGuide/README.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Illumina/manta/blob/master/docs/userGuide/README.md&lt;/a&gt;) but whenever I try to specify the &lt;code&gt;--exome&lt;/code&gt; command using the following code I get the error below:&lt;/p&gt;&#xA;&lt;p&gt;The code in my script:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Accept path to reference fasta file as a parameter:&#xA;&#xA;reference=$1&#xA;&#xA;# Accept input and output directories as parameters:&#xA;&#xA;input_dir=$2&#xA;&#xA;output_dir=$3&#xA;&#xA;# Hard coded Manta Installation Path:&#xA;&#xA;## If someone installs Manta they must specify where the installation is placed:&#xA;&#xA;MANTA_INSTALL_PATH=&amp;quot;/path/to/manta-1.6.0.release_src/install&amp;quot;&#xA;&#xA;&#xA;# Main code:&#xA;for file in &lt;span class=&quot;math-container&quot;&gt;${input_dir}/*.bam&#xA;do&#xA;  # Create subdirectory named after bam file&#xA;  sub_dir=$&lt;/span&gt;{output_dir}/&lt;span class=&quot;math-container&quot;&gt;$(basename $&lt;/span&gt;{file} .bam)&#xA;  mkdir -p ${sub_dir}&#xA;&#xA;  # Use reference fasta path specified by user&#xA;  &lt;span class=&quot;math-container&quot;&gt;${MANTA_INSTALL_PATH}/bin/configManta.py \&#xA;    --bam $&lt;/span&gt;file \&#xA;    --referenceFasta &lt;span class=&quot;math-container&quot;&gt;${reference} \&#xA;--runDir $&lt;/span&gt;{sub_dir}&#xA;    --exome&#xA;done&#xA;&#xA;# Now run each `runWorkflow.py`&#xA;&#xA;find &amp;quot;$output_dir&amp;quot; -name &amp;quot;runWorkflow.py&amp;quot; -exec {} \;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;runDir&lt;/code&gt; is directly below the previous command however, it doesn't seem to write well in this post.&lt;/p&gt;&#xA;&lt;p&gt;I ran the script via the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./script.sh /path/to/reference.fa /path/to/bam_files /path/to/output_dir&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get the error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/path/to/my/sample/folder/sample1/runWorkflow.py&#xA;/path/to/this/job_scripts/number_by_HPC: line 40: --exome: command not found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How do I get it to run with the &lt;code&gt;--exome&lt;/code&gt; command and not get the following error (it works without the &lt;code&gt;--exome&lt;/code&gt; command)? I am using exome data so I believe I need to use the &lt;code&gt;--exome&lt;/code&gt; command&lt;/p&gt;&#xA;" OwnerUserId="17244" LastEditorUserId="650" LastEditDate="2023-03-29T22:02:23.923" LastActivityDate="2023-03-29T22:16:09.300" Title="Manta &quot;--exome: command not found&quot;" Tags="|bash|exome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20793" PostTypeId="2" ParentId="20792" CreationDate="2023-03-28T18:27:16.613" Score="4" Body="&lt;p&gt;You're missing a &lt;code&gt;\&lt;/code&gt; in the line before the one containing the &lt;code&gt;--exome&lt;/code&gt;, so shell treats it as a new command, which is why you see this error. Add the &lt;code&gt; \&lt;/code&gt; after &lt;code&gt;${sub_dir}&lt;/code&gt; and your script should not give you this error.&lt;/p&gt;&#xA;&lt;p&gt;With that correction added in, your code will look like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Accept path to reference fasta file as a parameter:&#xA;&#xA;reference=$1&#xA;&#xA;# Accept input and output directories as parameters:&#xA;&#xA;input_dir=$2&#xA;&#xA;output_dir=$3&#xA;&#xA;# Hard coded Manta Installation Path:&#xA;&#xA;## If someone installs Manta they must specify where the installation is placed:&#xA;&#xA;MANTA_INSTALL_PATH=&amp;quot;/path/to/manta-1.6.0.release_src/install&amp;quot;&#xA;&#xA;&#xA;# Main code:&#xA;for file in &lt;span class=&quot;math-container&quot;&gt;${input_dir}/*.bam&#xA;do&#xA;  # Create subdirectory named after bam file&#xA;  sub_dir=$&lt;/span&gt;{output_dir}/&lt;span class=&quot;math-container&quot;&gt;$(basename $&lt;/span&gt;{file} .bam)&#xA;  mkdir -p ${sub_dir}&#xA;&#xA;  # Use reference fasta path specified by user&#xA;  &lt;span class=&quot;math-container&quot;&gt;${MANTA_INSTALL_PATH}/bin/configManta.py \&#xA;    --bam $&lt;/span&gt;file \&#xA;    --referenceFasta &lt;span class=&quot;math-container&quot;&gt;${reference} \&#xA;--runDir $&lt;/span&gt;{sub_dir} \&#xA;    --exome&#xA;done&#xA;&#xA;# Now run each `runWorkflow.py`&#xA;&#xA;find &amp;quot;$output_dir&amp;quot; -name &amp;quot;runWorkflow.py&amp;quot; -exec {} \;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastEditorUserId="650" LastEditDate="2023-03-29T22:16:09.300" LastActivityDate="2023-03-29T22:16:09.300" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="20794" PostTypeId="1" AcceptedAnswerId="20795" CreationDate="2023-03-28T18:56:15.870" Score="3" ViewCount="44" Body="&lt;p&gt;Hi I am downloading the data from a website which requires authentication of email_id and password before the download. They then provide the download link which is used to download dataset. This download link is different every time&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;### Download Cosmic Files&#xA;&#xA;##First you need to login with your login details and the access key ID may vary with each login ID&#xA;&#xA;##STEP-1&#xA;echo &amp;quot;pr@gmail.com:abc123@&amp;quot; | base64&#xA;#The above lines gives a authorization key cHIxNEBzYW5&#xA;&#xA;##STEP-2&#xA;# get the download link using the authorization key from step 1&#xA;#curl -H &amp;quot;Authorization: Basic cHIxNEBzYW5&amp;quot; https://cancer.sanger.ac.uk/cosmic/file_download/GRCh38/cosmic/v97/CosmicSample.tsv.gz&#xA;&#xA;#The above line gives the download link where AWS access key changes everytime like in this case it was https://cog.sanger.ac.uk/cosmic/GRCh38/cosmic/v97/CosmicSample.tsv.gz?AWSAccessKeyId=KRV7P7QR9DL41J9EWGA2&amp;amp;Expires=1679591113&amp;amp;Signature=WPjwDQsMoaLtRA2utiGLNrOSlOA%3D&#xA;&#xA;##STEP-3&#xA;# Download file with the link from step 2&#xA;#curl -o CosmicSample.tsv &amp;quot;https://cog.sanger.ac.uk/cosmic/GRCh38/cosmic/v97/CosmicSample.tsv.gz?AWSAccessKeyId=KRV7P7QR9DL41J9EWGA2&amp;amp;Expires=1679591113&amp;amp;Signature=WPjwDQsMoaLtRA2utiGLNrOSlOA%3D&amp;quot;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I want to automate the script first by passing used_id and password as an string&lt;/li&gt;&#xA;&lt;li&gt;I want to replace the AWS key in the step 3 with a string that is generated from step 2&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="3766" LastEditorUserId="650" LastEditDate="2023-03-29T18:40:05.770" LastActivityDate="2023-03-29T18:40:05.770" Title="Passing strings for password, email authentication and download link" Tags="|database|bash|curl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20795" PostTypeId="2" ParentId="20794" CreationDate="2023-03-29T01:43:06.747" Score="3" Body="&lt;p&gt;The cURL command to obtain the download link returns a JSON containing the URL. This can be parsed on the command line using &lt;a href=&quot;https://stedolan.github.io/jq/&quot; rel=&quot;nofollow noreferrer&quot;&gt;jq&lt;/a&gt; and used to construct a command to download the file. For example, with a list of URLs in a file (one per line) called &lt;em&gt;urls.txt&lt;/em&gt;, you could try the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;while IFS= read -r url ; do&#xA;    &amp;gt;&amp;amp;2 echo &amp;quot;Downloading: ${url}&amp;quot;&#xA;&#xA;    curl -sSH @cosmic_header &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${url}&quot; |&#xA;    jq --arg base &quot;$&lt;/span&gt;(basename &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${url}&quot;)&quot; -r '&quot;curl -o \($&lt;/span&gt;base) \(.url | @sh)&amp;quot;' |&#xA;    sh&#xA;done &amp;lt; urls.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that the above stores the HTTP header in a file called &lt;em&gt;cosmic_header&lt;/em&gt;, which can be created using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;echo &amp;quot;Authorization: Basic $(echo 'username:password' | base64)&amp;quot; &amp;gt; cosmic_header&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This avoids exposing the authorization key on the command line when you go to download your list of files, which might be preferable if working in a shared environment.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-03-29T01:43:06.747" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20796" PostTypeId="2" ParentId="20761" CreationDate="2023-03-29T06:28:15.943" Score="1" Body="&lt;p&gt;ABRicate can handle multiple input files&lt;sup&gt;&lt;a href=&quot;https://github.com/tseemann/abricate#input&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;&lt;/sup&gt; and will set the &lt;em&gt;FILE&lt;/em&gt; column in it's output accordingly. Assuming all of the FASTA files in this column exist on disk, you can try the following &lt;a href=&quot;/questions/tagged/python&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;python&amp;#39;&quot; aria-label=&quot;show questions tagged &amp;#39;python&amp;#39;&quot; rel=&quot;tag&quot; aria-labelledby=&quot;tag-python-tooltip-container&quot;&gt;python&lt;/a&gt; solution. It uses &lt;a href=&quot;https://pypi.org/project/pyfaidx/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pyfaidx&lt;/a&gt;, which allows for fast random access to the sequences in the (indexed) FASTA file(s). If the FASTA indexes don't exist, they will be created. It loads only a minimal amount of the input FASTA file(s) into memory. With the following in a file called &lt;code&gt;tsv2gff3.py&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import argparse&#xA;import csv&#xA;import contextlib&#xA;import re&#xA;import sys&#xA;import textwrap&#xA;&#xA;from dataclasses import dataclass, astuple&#xA;from pathlib import Path&#xA;from typing import List, Literal, Tuple&#xA;&#xA;from pyfaidx import Fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;def get_argument_parser():&#xA;    parser = argparse.ArgumentParser(add_help=False)&#xA;    group = parser.add_argument_group('positional arguments')&#xA;    group.add_argument(&#xA;        &amp;quot;tsv&amp;quot;,&#xA;        type=str,&#xA;        metavar=&amp;quot;FILE&amp;quot;,&#xA;        help=&amp;quot;The ABRicate TSV file&amp;quot;,&#xA;    )&#xA;    group = parser.add_argument_group('output options')&#xA;    group.add_argument(&#xA;        &amp;quot;-o&amp;quot;,&#xA;        &amp;quot;--output&amp;quot;,&#xA;        type=str,&#xA;        metavar=&amp;quot;FILE&amp;quot;,&#xA;        default='-',&#xA;        help=&amp;quot;Write the GFF3 output to FILE (default: stdout)&amp;quot;,&#xA;    )&#xA;    group.add_argument(&#xA;        &amp;quot;-f&amp;quot;,&#xA;        &amp;quot;--force&amp;quot;,&#xA;        action='store_true',&#xA;        help=&amp;quot;Overwrite the output file if it exists&amp;quot;,&#xA;    )&#xA;    group = parser.add_argument_group('additional options')&#xA;    group.add_argument(&#xA;        &amp;quot;-h&amp;quot;,&#xA;        &amp;quot;--help&amp;quot;,&#xA;        action=&amp;quot;help&amp;quot;,&#xA;        help=&amp;quot;Show this help message and exit&amp;quot;,&#xA;    )&#xA;&#xA;    return parser&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;@dataclass&#xA;class Feature:&#xA;&#xA;    seq_id: str&#xA;    source: str&#xA;    type: str&#xA;    start: int&#xA;    end: int&#xA;    score: float&#xA;    strand: Literal['-', '+']&#xA;    phase: Literal['0', '1', '2']&#xA;    attrs: List[Tuple[str, str]]&#xA;&#xA;    @property&#xA;    def attributes(self):&#xA;        return ';'.join([f'{key}={val}' for key, val in self.attrs])&#xA;&#xA;    def __iter__(self):&#xA;        return iter(astuple(self))&#xA;&#xA;    def __str__(self):&#xA;        vals = list(map(str, astuple(self)[:-1])) + [self.attributes]&#xA;        return '\t'.join(vals) + '\n'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;def main():&#xA;&#xA;    parser = get_argument_parser()&#xA;    args = parser.parse_args()&#xA;&#xA;    with contextlib.ExitStack() as stack:&#xA;        if args.output != '-':&#xA;            out = stack.enter_context(open(args.output, 'wt' if args.force else 'xt'))&#xA;        else:&#xA;            out = sys.stdout&#xA;&#xA;        tsv = stack.enter_context(open(args.tsv, 'r'))&#xA;        first_line = tsv.readline().rstrip()&#xA;        fieldnames = first_line.lstrip('#').split('\t')&#xA;        reader = csv.DictReader(tsv, dialect='excel-tab', fieldnames=fieldnames)&#xA;&#xA;        fasta_files = dict()&#xA;        seq_ids = dict()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;        out.write('##gff-version 3\n')&#xA;        for idx, row in enumerate(reader, 1):&#xA;&#xA;            filename = Path(row['FILE'])&#xA;            if filename not in fasta_files:&#xA;                fasta_files[str(filename)] = stack.enter_context(Fasta(filename))&#xA;&#xA;            if filename.suffix in ('.fasta', '.fa', '.fna'):&#xA;                seq_id = filename.stem&#xA;            else:&#xA;                seq_id = filename.name&#xA;&#xA;            fasta = fasta_files[str(filename)]&#xA;            seq = fasta[row['SEQUENCE']]&#xA;&#xA;            if not seq_id in seq_ids:&#xA;                out.write(f'##sequence-region {seq_id} 1 {len(seq)}\n')&#xA;                seq_ids[seq_id] = [row['FILE'], row['SEQUENCE']]&#xA;&#xA;            database = row['DATABASE']&#xA;            product = re.sub(r'[\[\]\(\)]', '', row['PRODUCT'])&#xA;            start, end = int(row['START']), int(row['END'])&#xA;            attrs = [&#xA;                ('ID', f'{seq_id}_{idx}'),&#xA;                ('Product', product),&#xA;            ],&#xA;&#xA;            feature = Feature(seq_id, database, &amp;quot;CDS&amp;quot;, start, end, '.', '+', '0', *attrs)&#xA;&#xA;            out.write(str(feature))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;        out.write('##FASTA\n')&#xA;        for seq_id, (filename, seq_name) in seq_ids.items():&#xA;            fasta_rec = fasta_files[filename][seq_name]&#xA;            seq = textwrap.fill(str(fasta_rec), width=60)&#xA;&#xA;            out.write(f'&amp;gt;{seq_id}\n{seq}\n')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;if __name__ == '__main__':&#xA;    main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And &lt;code&gt;pyfaidx&lt;/code&gt; installed, the usage is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ python tsv2gff3.py -h&#xA;usage: tsv2gff3.py [-o FILE] [-f] [-h] FILE&#xA;&#xA;positional arguments:&#xA;  FILE                  The ABRicate TSV file&#xA;&#xA;output options:&#xA;  -o FILE, --output FILE&#xA;                        Write the GFF3 output to FILE (default: stdout)&#xA;  -f, --force           Overwrite the output file if it exists&#xA;&#xA;additional options:&#xA;  -h, --help            Show this help message and exit&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-03-29T06:28:15.943" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20797" PostTypeId="1" CreationDate="2023-03-29T18:09:02.863" Score="1" ViewCount="66" Body="&lt;p&gt;We came across a project in our lab that no one exactly knows how to approach. Since, I know a little bit of Python programming, this project was assigned to me.&lt;/p&gt;&#xA;&lt;p&gt;There is a data from a randomised controlled clinical trial with 60 participants. Half (30) are control and the other half are actual patients (case). From each group, biopsy samples were taken before (pre) and after (post) treatment with drug A that targets cell type B in the tissue.&lt;/p&gt;&#xA;&lt;p&gt;In each biopsy sample, the B cells were detected and classified either as “normal” or “malignant”.&lt;/p&gt;&#xA;&lt;p&gt;Associated metadata for each biopsy are displayed here (I have only included 12 records for this file):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;name    patient_id  arm treatment&#xA;111     0       control     pre&#xA;112     1       control     pre&#xA;113     2       control     pre&#xA;121     0       control     post&#xA;122     1       control     post&#xA;123     2       control     post&#xA;211     75      case        pre&#xA;212     76      case        pre&#xA;213     77      case        pre&#xA;221     75      case        post&#xA;222     76      case        post&#xA;223     77      case        post&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;●   name: spreadsheet file name&lt;/p&gt;&#xA;&lt;p&gt;●   patient_id: patient identity number&lt;/p&gt;&#xA;&lt;p&gt;●   arm: trial arm (‘case’ or ‘control)&lt;/p&gt;&#xA;&lt;p&gt;●   treatment: treatment condition (‘pre’ or ‘post’ treatment)&lt;/p&gt;&#xA;&lt;p&gt;Other files (I have only included 20 records for each file: 10 normal and 10 malignant) contain cell detection results from a single biopsy, and each row in a spreadsheet represents an individual detected cell:&lt;/p&gt;&#xA;&lt;p&gt;●   x: x coordinate&lt;/p&gt;&#xA;&lt;p&gt;●   y: y coordinate&lt;/p&gt;&#xA;&lt;p&gt;●   label: classified label (‘normal’ or ‘malignant’)&lt;/p&gt;&#xA;&lt;p&gt;Just showing an example, file 111 looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;x   y   label&#xA;730 724 normal&#xA;1962 450 normal&#xA;1511 817 normal&#xA;1244 455 normal&#xA;2529 397 normal&#xA;1878 262 normal&#xA;2248 369 normal&#xA;2007 273 normal&#xA;1531 878 normal&#xA;1729 834 normal&#xA;931  1270 malignant&#xA;1282 314 malignant&#xA;1630 839 malignant&#xA;1543 460 malignant&#xA;2493 237 malignant&#xA;1311 744 malignant&#xA;1999 366 malignant&#xA;737 1361 malignant&#xA;2252 448 malignant&#xA;2620 398 malignant&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The rest can be found here, but probably they will not be necessary: &lt;a href=&quot;https://www.mediafire.com/file/ka7r59kf0swnbnd/OtherFiles.rar/file&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.mediafire.com/file/ka7r59kf0swnbnd/OtherFiles.rar/file&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am trying to answer 3 questions here (if you can think of other questions, please let me know):&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;identifying post-treatment morphological changes due to the effect of drug A.&lt;/li&gt;&#xA;&lt;li&gt;proposing measures to quantify the changes.&lt;/li&gt;&#xA;&lt;li&gt;Using appropriate statistical analysis tools to provide insight whether the changes are due to chance or not.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="16833" LastEditorUserId="16833" LastEditDate="2023-03-29T22:53:54.447" LastActivityDate="2023-03-30T14:51:00.350" Title="Analysing the Effect of a Drug on the Morphological Changes of a Cell Type" Tags="|python|drugs|meta-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20799" PostTypeId="1" CreationDate="2023-03-29T19:30:05.843" Score="0" ViewCount="248" Body="&lt;p&gt;I have a dante labs dataset that I would like to cross reference with SNPedia but I am unclear about orientation. I've read the info on SNPedia and other places about it but I just don't get it yet. I realise this is my own stupidity, the explanations feel sound but they seem to be missing some key info that I can't find anywhere.&lt;/p&gt;&#xA;&lt;p&gt;Anyway without being too tl;dr about what confuses me, I wonder if my dante vcf file has the orientation problem or not because it shows what the reference genome base is next to the sample base. So for example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1 36571920 rs96067 G A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Chr:1, Pos: 36571920, rsid:rs96067, reference base:G, sample base: A, (another field later in the line defines that both bases are the alt base so this is an (A;A) where the reference had (G;G)).&lt;/p&gt;&#xA;&lt;p&gt;​On SNPedia the genotypes mentioned for rs96067 are C;C, C;T and T;T. Since the reference base in my dante file is a G, which does not show up as a possible reference base on the SNPedia page, does this mean that the dante file is flipped with respect to this gene compared to SNPedia and so I should flip one or other if I want to compare them (flip the base in my file or the data from SNPedia that is)? So I guess flip G-&amp;gt;C and A-&amp;gt;T meaning my AA becomes TT?&lt;/p&gt;&#xA;&lt;p&gt;​BTW what confuses me about the SNPedia explanation about orientation is this:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Unfortunately if this was instead a SNP where the two alleles are A or T the same flipping logic falls down. We don't (yet) have a way to know for sure if you should be flipped or not, since both forms of your flip rs1234(A;A) and rs1234(T;T) are possible.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Why is AA not the same as TT in this case? Surely this just reflects the direction that the bases were read from, so detection of an AA implicitly means that if we had read from the other direction we would have read TT? I appreciate this is wrong but I don't understand why.&lt;/p&gt;&#xA;&lt;p&gt;@terdon: Re where I was quoting from, &lt;a href=&quot;https://www.snpedia.com/index.php/Ambiguous_flip&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.snpedia.com/index.php/Ambiguous_flip&lt;/a&gt; (&lt;a href=&quot;https://www.snpedia.com/index.php/Orientation&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.snpedia.com/index.php/Orientation&lt;/a&gt; also has some info). I've been taking 'flip' as their word for inversion. Re the 'orientation problem' I meant the ambiguity re orientation mentioned in those pages. However you mentioned in your comment that the convention is to always use the + strand. I read up about that and understand now that VCFs use that convention, which answers my question (thanks!). Dbsnp states that it always uses +ve. So like can be compared with like, as long as you use SNP definitions for the correct reference build that your individual SNP VCF has been generated from, and you are using dbsnp as your database. The so called 'flipping' can happen between reference versions/builds and could trip you if you don't use the correct one. Thats what I have gathered to this point, do let me know if its not right.&lt;/p&gt;&#xA;&lt;p&gt;On SNPedia however many rsid's seem to be defined as minus orientation. They have 2 fields, orientation and stabilised orientation. To quote from the above linked pages:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Orientation indicates the orientation reported in the most current build (currently GRCh38), which is reported below the genotypes in the Reference field. The chromosome number and nucleotide position in that reference build is shown next. StabilizedOrientation is the orientation that is relevant to the genotypes that have been defined in SNPedia for each SNP...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So they are saying that GRCh38 has both plus and minus orientation within and each genotype includes an orientation field? How does this fit with the standard '+ve' convention you mentioned? Maybe these are 2 different things and I just misunderstand. But it would seem far more sensible to simply adjust GRCh38 to all +ve. Is this 'orientation problem' just SNPedias problem? What exactly do they mean on the above linked pages on orientation and ambigous flips?&lt;/p&gt;&#xA;&lt;p&gt;SNPedia seems pretty useful if it can be made to work - there is lots of associated data on each genotype page, the API is easy to use including for agregated queries etc, but it seems way less reliable than dbsnp despite apparently simply taking their data from dbsnp. Taking the above example I mentioned, rs96067 G&amp;gt;A, dbsnp defines this well, for GRCh37 it shows NC_000001.10:g.36571920G&amp;gt;A. There is also a 'COL8A2 RefSeqGene    NG_016245.2:g.23766C&amp;gt;T' entry which I don't fully understand. However on SNPedia as I said all that shows up are C;C, C;T and T;T genotypes. Maybe they are referring to the COL8A2 C&amp;gt;A entry on dbsnp - or did they scrape this SNP def badly? Or are they showing transcript change rather than genomic as Ram RS suggested?&lt;/p&gt;&#xA;" OwnerUserId="17155" LastEditorUserId="17155" LastEditDate="2023-03-31T21:55:23.457" LastActivityDate="2023-04-02T20:30:51.587" Title="Can you infer orientation from vcf file data?" Tags="|snp|" AnswerCount="4" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20800" PostTypeId="2" ParentId="20799" CreationDate="2023-03-29T20:29:05.523" Score="1" Body="&lt;p&gt;SNPedia seems to show transcript base change and not genomic base change, so their C&amp;gt;T is the same as the G&amp;gt;A you note.&lt;/p&gt;&#xA;&lt;p&gt;I don't understand SNPedia's allele notations completely, other sources are more straightforward. Does their &lt;code&gt;A;A&lt;/code&gt; mean &lt;code&gt;A&lt;/code&gt; on both strands? Then why do they talk about dbSNP showing &lt;code&gt;C;T&lt;/code&gt; when dbSNP would in fact show &lt;code&gt;C&amp;gt;T&lt;/code&gt; - they seem to use &lt;code&gt;;&lt;/code&gt; to mean alleles and base change interchangeably, and that is confusing. I recommend sticking to dbSNP for all information not found in SNPedia - something tells me SNPedia is not as trustworthy.&lt;/p&gt;&#xA;" OwnerUserId="650" LastEditorUserId="73" LastEditDate="2023-03-30T19:39:05.003" LastActivityDate="2023-03-30T19:39:05.003" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="20801" PostTypeId="1" AcceptedAnswerId="20806" CreationDate="2023-03-29T21:51:28.910" Score="2" ViewCount="55" Body="&lt;p&gt;I have Illumina RNAseq data and would like to maximize my power to find candidate genes that are differentially expressed genes between experimental conditions.&lt;/p&gt;&#xA;&lt;p&gt;Many of my (de novo assembled and annotated) gene models have multiple isoforms. Because I will only be using &amp;quot;uniquely&amp;quot; aligning read counts (as specified by DESeq2 documentation) I fear that I might be disadvantaging myself by aligning my reads back to the full set of gene annotations.&lt;/p&gt;&#xA;&lt;p&gt;Would I be wrong to first filter my gff file for only the longest isoform prior to read alignment?&lt;/p&gt;&#xA;" OwnerUserId="17251" LastActivityDate="2023-03-30T22:59:51.523" Title="RNAseq alignment: best practices for aligning to multiple isoforms?" Tags="|rna-seq|sequence-alignment|deseq2|differential-expression|isoform|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20802" PostTypeId="1" CreationDate="2023-03-29T21:52:26.273" Score="2" ViewCount="105" Body="&lt;p&gt;I am trying to figure out a way to remove the limiting factors that Manta says it cannot handle. To quote from the &lt;a href=&quot;https://github.com/Illumina/manta/blob/master/docs/userGuide/README.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;Manta page&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The following limitations exist on the input BAM or CRAM files provided to Manta:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Alignments cannot have an unknown read sequence (SEQ=&amp;quot;*&amp;quot;)&lt;/li&gt;&#xA;&lt;li&gt;Alignments cannot contain the &amp;quot;=&amp;quot; character in the SEQ field.&lt;/li&gt;&#xA;&lt;li&gt;Alignments cannot use the sequence match/mismatch (&amp;quot;=&amp;quot;/&amp;quot;X&amp;quot;) CIGAR notation&#xA;RG (read group) tags in the alignment records are ignored -- each file will be&#xA;treated as representing one sample.&lt;/li&gt;&#xA;&lt;li&gt;Alignments with basecall quality values greater than 70 are rejected (these are not&#xA;supported on the assumption that this indicates an offset error)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Would using a &lt;code&gt;find&lt;/code&gt; statement in this way and then making &lt;code&gt;.bai&lt;/code&gt; files be adequate to remove the limiting factors that Manta has?:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;find &lt;span class=&quot;math-container&quot;&gt;${input_dir} -name &quot;*.bam&quot; -type f -exec sh -c 'output_file=&quot;$&lt;/span&gt;{0%.bam}_filtered.bam&amp;quot;; temp_file=&lt;span class=&quot;math-container&quot;&gt;$(mktemp); samtools view -h -q 70 &quot;$&lt;/span&gt;0&amp;quot; | sed &amp;quot;/^\(@\|[^[:blank:]]\+[[:blank:]]\+4\)/!s/\t[=*]/\t/g&amp;quot; | samtools view -Sb - &amp;gt; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$temp_file&quot;; mv &quot;$&lt;/span&gt;temp_file&amp;quot; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$output_file&quot;; samtools index &quot;$&lt;/span&gt;output_file&amp;quot;; rm &amp;quot;$0&amp;quot;' {} \;&#xA;# Make the .bai files:&#xA;&#xA;for file in &lt;span class=&quot;math-container&quot;&gt;${input_dir}/*.bam; do&#xA;    samtools index &quot;$&lt;/span&gt;{file}&amp;quot; &amp;quot;${file%.*}.bai&amp;quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, I get the error (repeatedly for all my bam files):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;[main_samview] fail to read the header from &amp;quot;-&amp;quot;.&#xA;samtools index: &amp;quot;/path/to/MyBatch1And2/SampleNumber_filtered_filtered_filtered.bam&amp;quot; is in a format that cannot be usefully indexed&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I am not sure why I got this error once and then it did not occur again. Why?&#xA;Or would this not be enough remove the limiting factors (as there are more to be considered or the code is not written correctly) of Manta?&lt;/p&gt;&#xA;" OwnerUserId="17244" LastEditorUserId="17244" LastEditDate="2023-03-30T20:31:26.617" LastActivityDate="2023-03-30T20:31:26.617" Title="Dealing With Manta Limitations" Tags="|sam|bash|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20803" PostTypeId="2" ParentId="20797" CreationDate="2023-03-29T23:50:07.777" Score="1" Body="&lt;p&gt;Yes it's easy you load everything into a dataframe, so each variable is one column and perform supervised machine learning. That will solve all questions.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Hypothesis 1&lt;/strong&gt; The target is 'pre' 'post' &lt;strong&gt;if there is only one drug in the trial&lt;/strong&gt;. If there is more than one drug in the trial the drugs would become the target, e.g. if you wanted to know the efficacy between 'drug A' and 'drug B'. What you want to know is the effective of the drug. This would help hypothesis 2.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Hypothesis 2&lt;/strong&gt; Estimated cell size is the training target. This is more complicated because I'm not sure what x and y mean: but if x and y are cell measurements then an estimate of cell surface area should be made. If x and y can become a single variate of cell surface area than that can form categorical data ... super size, big, medium large, medium, medium-regular, regular etc ...&#xA;This could then become the training target and probably what you are looking for.&lt;/p&gt;&#xA;&lt;p&gt;The ML feature selection will assess cell size (if 'pre' and 'post' is the training target; or), or 'pre'-'post' (if 'cell size estimation' is the training target). In either case the expected result is it should provide the strongest weight in the analysis.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Finally&lt;/strong&gt; Finally and importantly, the more variables you can input the better, e.g. the gender of the patient, age of the patient as categorical data (bins), smoking, drinking ... anything you can think of, the more the better. ML should ignore anything that has not impact on 'pre' / 'post' or 'estimated cell size'.&lt;/p&gt;&#xA;&lt;p&gt;The ML algorithm is more complicated but start with naive Bayes and see how it goes. Beyond this it does start to get complicated however.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To address the comments,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Load everything into a dataframe&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;A dataframe is a basic unit of operation in &lt;code&gt;Python&lt;/code&gt; (&lt;code&gt;pandas&lt;/code&gt;) or &lt;code&gt;R&lt;/code&gt;. Python also uses &lt;code&gt;numpy&lt;/code&gt; arrays.&#xA;Everything, means everything - all the data you have, metadata all files. For the calculation the columns involving patient id and patient number, this might be better excluded. I'm not sure - it's complicated because you'll have duplication of records - I would do the calculations with and without this data. You can delete patient id (it's the same as patient number - from a calculation point of view). The x, y is a special case.&lt;/p&gt;&#xA;&lt;p&gt;Thus all the various tables you describe get loaded into one dataframe, this includes all the patient data such as gender, weight, age, social factors ... it's just like one giant excel table, with rows and columns. The columns are labelled according to the variable e.g. Drug therapy = 'pre' and 'post' data. The dataframe will feed directly to the ML algorithm - a good reason for using it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;x&lt;/strong&gt;&lt;/em&gt; and &lt;em&gt;&lt;strong&gt;y&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Personally I don't think splitting this data into two columns is a good idea. It will confuse the algorithms - its needs to be one column of data - a single variate. To achieve this will be doable. I would use &lt;strong&gt;π&lt;/strong&gt; to calculate the cell surface area. How that is done for an elongated circle - I don't know - but someone will have done it somewhere, you just need to look it up. Thus 2&lt;strong&gt;π&lt;/strong&gt;&lt;em&gt;r&lt;/em&gt;&lt;sup&gt;2&lt;/sup&gt; if its perfectly round where &lt;em&gt;r&lt;/em&gt; = diameter/2 ... elongation must have a derivation its just finding it.&lt;/p&gt;&#xA;&lt;p&gt;This will absolutely make a difference in your calculations - what I would do is make it to a categorical variable by plotting it and constructing a histogram - otherwise you are into transformations and thats complicated. Categorical variables will also permit logistic regression ML and that can be very useful, because it places each variable into positive or negative weights.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-03-30T14:51:00.350" LastActivityDate="2023-03-30T14:51:00.350" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20804" PostTypeId="2" ParentId="20802" CreationDate="2023-03-30T14:37:32.573" Score="2" Body="&lt;p&gt;Assuming you have &lt;strong&gt;paired-end&lt;/strong&gt; Illumina reads mapped using BWA or BWA-MEM (with default parameters), these limitations are usually not much of a concern:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Alignments cannot have an unknown read sequence (SEQ=&amp;quot;*&amp;quot;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Best practice for representing alignments in SAM format, suggests that the SEQ and QUAL of secondary alignments should be set to an asterisk to reduce file size. You can either filter these out after the fact, using something like &lt;code&gt;samtools view -F 256 &amp;lt;bam&amp;gt;&lt;/code&gt;, or simply avoid writing out any secondary alignments in the first place.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Alignments cannot contain the &amp;quot;=&amp;quot; character in the SEQ field.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;An &amp;quot;=&amp;quot; character can be used to denote that the base is identical to the reference base. I've not come across an aligner or downstream process that produces such output. If you know of such a tool or process, please comment below to let me know.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Alignments cannot use the sequence match/mismatch (&amp;quot;=&amp;quot;/&amp;quot;X&amp;quot;) CIGAR notation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Please see: &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/404/why-most-aligners-do-not-output-the-x-cigar-operation&quot;&gt;Why most aligners do not output the &amp;quot;X&amp;quot; CIGAR operation?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;RG (read group) tags in the alignment records are ignored -- each file will be treated as representing one sample.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If you have multiple libraries for a single sample, and they differ in some not so small way (perhaps you have a paired-end library and an old mate-pair library, or maybe the distribution of insert sizes/template lengths of each library varies wildly), you may want to consider running each library through separately.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Alignments with basecall quality values greater than 70 are rejected (these are not supported on the assumption that this indicates an offset error)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Short-read sequencing technologies do not produce such high base quality scores. Only long-read technologies, like PacBio and Oxford Nanopore, produce base quality scores above 70 (up to 93).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Ultimately, I don't think you really need to deal with any of these with some command pipeline if you choose your inputs wisely.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-03-30T14:37:32.573" CommentCount="13" ContentLicense="CC BY-SA 4.0" />
  <row Id="20805" PostTypeId="1" CreationDate="2023-03-30T15:15:43.573" Score="1" ViewCount="158" Body="&lt;p&gt;I'm trying to merge two VCF files, but I'm running into an error message that says &lt;code&gt;The REF prefixes differ: G vs A (1,1) Failed to merge alleles at 1:88143 in sheep2_50KOAR3.1_student_project.vcf.gz&lt;/code&gt;. I'm not sure how to fix this error. Can someone help me understand what is causing this error and how to resolve it?&lt;/p&gt;&#xA;&lt;p&gt;Here is the command I used to merge the files:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools merge sheep1_50KOAR3.1_student_project.vcf sheep2_50KOAR3.1_student_project.vcf -Ov -o sheepcombined_50KOAR3.1_student_project.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Both files have SNP data from the Illumina 50k Ovine SNP, and I believe were generated using the same reference genome. I'm not sure what could be causing this error. Any help would be greatly appreciated.&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="17255" LastEditorUserId="650" LastEditDate="2023-03-30T15:46:36.617" LastActivityDate="2023-03-30T15:46:36.617" Title="Title: Error merging VCF files: &quot;The REF prefixes differ: G vs A&quot;" Tags="|vcf|merge|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20806" PostTypeId="2" ParentId="20801" CreationDate="2023-03-30T19:35:23.313" Score="2" Body="&lt;p&gt;The fact that an alignment is unique or not is relative to the genome, not its annotation. In other words, a read will be considered a multimapper (and ignored by some programs) if it can align to several genomic positions, but a read will be considered properly mapped if it aligns to a single genomic position, that may or may not correspond to multiple isoforms.&lt;/p&gt;&#xA;&lt;p&gt;So if you're only interested in gene expression, there shouldn't be any problem with multimappers.&lt;/p&gt;&#xA;&lt;p&gt;Note that this answer could change depending on what you are aligning on, and how exactly you performed your &lt;em&gt;de novo&lt;/em&gt; assembly.&lt;/p&gt;&#xA;" OwnerUserId="9497" LastActivityDate="2023-03-30T19:35:23.313" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20807" PostTypeId="2" ParentId="20799" CreationDate="2023-03-30T20:01:29.700" Score="0" Body="&lt;p&gt;Strand flips are common when using different VCF generation software. You've got one of the easier-to-determine variants there, because it's pretty clear that the orientation is incorrect between SNPedia and your VCF file. Illumina is particularly frustrating in this regard because they have a &amp;quot;top&amp;quot;/&amp;quot;bottom&amp;quot; definition, which can sometimes be inconsistent with the &amp;quot;reference positive&amp;quot; convention.&lt;/p&gt;&#xA;&lt;p&gt;The reason strand for complementary variants can't be [easily] determined is because the usual way to fix strand in a bulk fashion is to check for which of the variant or it's complement are present in the reference database. If the variant is a complementary variant, then &lt;em&gt;both&lt;/em&gt; will be found (one as the reference allele, and the other as the alternate variant allele). There are some statistical ways to work round this (e.g. looking at the most common variant in your dataset, and comparing it to the highest allele frequency in the reference database), but those methods will fail if the ancestral background of your study population is substantially different from the reference population (which was the case for the population genetics work I was doing).&lt;/p&gt;&#xA;&lt;p&gt;The ultimate reference for RS numbers is &lt;a href=&quot;https://ncbi.nlm.nih.gov/snp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbSNP&lt;/a&gt;, because that's where the RS numbers are dished out for new SNPs.&lt;/p&gt;&#xA;&lt;p&gt;Illumina claims that their top/bottom definition allows them to precisely determine complementary strand orientation, but it's useless unless the full sequence context is provided (i.e. the surrounding sequence up until strand can be determined by reverse complementation). Here's the explanation from &lt;a href=&quot;https://knowledge.illumina.com/microarray/general/microarray-general-reference_material-list/000001521&quot; rel=&quot;nofollow noreferrer&quot;&gt;Illumina&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For SNPs that are not [A/T] or [G/C], A is always on the top strand and T is always the bottom strand. A and T nucleotides are the “A alleles”; G and C nucleotides are the “B alleles”.&#xA;If the SNP is [A/T] or [G/C]: Use sequence walking to determine TOP/BOT strands, then assign A/B alleles.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;rs903997 ...ATGAGA*A*AGT[C/G]TGA*G*AGTGCA...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;By Illumina's definition, the above sequence is designated &amp;quot;top&amp;quot;, because if you walk outwards from the variant location, you eventually get to a situation where A is present on one side, and C or G is present on the other side. The reverse complement of this sequence would be designated &amp;quot;bottom&amp;quot;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rs903997 ...TGCACT*C*TCA[C/G]ACT*T*TCTCAT...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is explained more exhaustively in Illumina's technical note:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.illumina.com/documents/products/technotes/technote_topbot.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.illumina.com/documents/products/technotes/technote_topbot.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;When I was puzzling over this, I gave up trying to work this out, and filtered out all complementary variants in my dataset.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-04-02T20:30:51.587" LastActivityDate="2023-04-02T20:30:51.587" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20808" PostTypeId="2" ParentId="20801" CreationDate="2023-03-30T22:59:51.523" Score="1" Body="&lt;p&gt;If you want to account for changing abundances of isoforms, use a pseudoaligner like kallisto for counting.  And use DEXSeq, not DESeq2, as DESeq2 is intended to be applied to gene counts, not transcript counts.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2023-03-30T22:59:51.523" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20809" PostTypeId="1" CreationDate="2023-03-31T04:58:00.663" Score="2" ViewCount="645" Body="&lt;p&gt;I have a bash script that I would like to parellelize to run on multiple nodes. My goal is to run my python sample_script.py script on pairwise comparisons of samples to see if their variants are a match or not (output is written by echo for the function so we can compare results). I unfortunately can’t post the contents of the Python script, but it is basically comparing germline variants between samples to determine if a sample is a match or not. I’m trying to figure out how to make the script run faster or run multiple pair wise comparisons at once then concatenate results. The samples I'm comparing are listed in the array. Running the script on multiple samples takes too long, so I'd like to qsub the process or run the script on multiple nodes, then concatenate the results. Can someone advise on the best way to tackle this? I don't have much experience with jobs using qsub. Should I consider an SGE array job and if so how would I go about this?&lt;/p&gt;&#xA;&lt;p&gt;Example code below using bash within jupyter notebook:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;%%bash&#xA;array=( A037375001 A037646101 A037710601 )&#xA;for i in &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${array[@]}&quot;&#xA;      do &#xA;        for j in &quot;$&lt;/span&gt;{array[@]}&amp;quot;&#xA;        do&#xA;        python3 sample_script.py --run-sample-id-A &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$i&quot; --run-sample-id-B &quot;$&lt;/span&gt;j&amp;quot;&#xA;        echo run-sample-id-A &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$i&quot; run-sample-id-B &quot;$&lt;/span&gt;j&amp;quot;&#xA;      done&#xA;    done &amp;gt; &amp;quot;/outputdir/out.csv&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="17258" LastEditorUserId="17258" LastEditDate="2023-04-06T17:36:23.117" LastActivityDate="2023-04-06T17:36:23.117" Title="Parallelize or qsub a bash script" Tags="|python|bash|parallel|" AnswerCount="4" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20811" PostTypeId="2" ParentId="20809" CreationDate="2023-03-31T10:11:57.803" Score="2" Body="&lt;p&gt;Consider using a workflow language such as Snakemake, Nextflow or Cromwell to handle distributing the jobs for you. It will take a little more time investment upfront to set them up, but it will save you many hours and headaches later on.&lt;/p&gt;&#xA;" OwnerUserId="3818" LastActivityDate="2023-03-31T10:11:57.803" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20812" PostTypeId="2" ParentId="20809" CreationDate="2023-03-31T12:52:54.183" Score="0" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt; Just for everyone else &lt;code&gt;SGE&lt;/code&gt; = Sun Grid Engine and is the &lt;code&gt;qsub&lt;/code&gt; system mentioned in the title of the question.&lt;/p&gt;&#xA;&lt;h3&gt;&lt;code&gt;qsub&lt;/code&gt;:&lt;/h3&gt;&#xA;&lt;p&gt;If by parallelising you mean multithreading a single job across loads of cores then the first thing is how many cores per node are there?&lt;/p&gt;&#xA;&lt;p&gt;Usually it is around 10 (but could be much more). That exact number is super important to know because if you request more cores than exist on a single node the job will never run. At best qsub will refuse the job, at worst it will forever be stuck in the queue. For multi-processing on &lt;code&gt;qsub&lt;/code&gt; its ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;qsub -pe omp 8  myscript.sh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here its requesting 8 cores and myscript.sh will contain the shell command to run the python script. If there's a node with 10-cores 2 are in use, it will then load the job to give its max. capacity.&lt;/p&gt;&#xA;&lt;p&gt;Thus, &lt;code&gt;qsub&lt;/code&gt; does not do &lt;strong&gt;course grain-parallelisation&lt;/strong&gt;, that is &lt;code&gt;MPI&lt;/code&gt;. Thus &lt;code&gt;qsub&lt;/code&gt; will only parallelise across the cores in a single &lt;code&gt;node&lt;/code&gt;. This is without question a limitation for cluster computing.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;qsub&lt;/code&gt; is more than just submitting to a queue, you need to monitor the queuing system pre- and post- submission. Firstly, this is for the number of existing jobs across that system. Secondly, this is to see whats happening to your stuff.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;qstat&lt;/code&gt; is the way to understand whats there:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;qstat -f # whats what on the cluster&#xA;qstat -q&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This lists all the queues.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;qstat -u username # looks at what a given user is doing&#xA;qstat long-queue&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Targets availability on a specific queue.&lt;/p&gt;&#xA;&lt;p&gt;A complete list of &lt;code&gt;qstat&lt;/code&gt; is &lt;a href=&quot;http://docs.adaptivecomputing.com/torque/4-1-3/Content/topics/commands/qstat.htm&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Also &lt;code&gt;qusage -l&lt;/code&gt; is useful.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;&amp;quot;Parallelising&amp;quot; loops&lt;/h3&gt;&#xA;&lt;p&gt;If you are submitting loads of jobs which are all working independently in parallel it better to use a different strategy on &lt;code&gt;qsub&lt;/code&gt;. It is better to submit each job separately to the queue via a &lt;code&gt;qsub&lt;/code&gt; loop. Thus there would be two scripts the first is the script in the question, the second is a submission loop: this would comprise a &lt;code&gt;qsub&lt;/code&gt; argument within a loop that would submit each job sequentially. Thus the array might be better in the &lt;code&gt;qsub&lt;/code&gt; submission loop.&lt;/p&gt;&#xA;&lt;p&gt;Monitoring the queue takes place as per normal via &lt;code&gt;qstat&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;h4&gt;Rationale&lt;/h4&gt;&#xA;&lt;p&gt;The reason for this is the way the queue works &lt;code&gt;qsub&lt;/code&gt; prioritises single jobs over multi-threaded parallelisation, thus you get your results much quicker.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-03-31T14:20:32.560" LastActivityDate="2023-03-31T14:20:32.560" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20813" PostTypeId="2" ParentId="20783" CreationDate="2023-03-31T14:58:29.060" Score="3" Body="&lt;p&gt;Biostars user &lt;a href=&quot;https://www.biostars.org/u/100756/&quot; rel=&quot;nofollow noreferrer&quot;&gt;MirianT_NCBI&lt;/a&gt; provided an answer on &lt;a href=&quot;https://www.biostars.org/p/9559023/#9559023&quot; rel=&quot;nofollow noreferrer&quot;&gt;my crosspost&lt;/a&gt;, which I'm copying here:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;You can find more information about what constitute an &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets/docs/v2/troubleshooting/faq/#why-are-some-genomes-flagged-with-a-warning-icon-and-message&quot; rel=&quot;nofollow noreferrer&quot;&gt;atypical genome in this FAQ&lt;/a&gt;. Here's the list of reasons for a genome to be marked as atypical:&lt;br /&gt;&#xA;Datasets flags genomes with a warning icon and message for the following genome problems:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;chimeric - sequences from two different organisms are joined together.&lt;/li&gt;&#xA;&lt;li&gt;contaminated - sequences from another organism, cloning vectors, linkers, adapters or primers are present in the assembly.&lt;/li&gt;&#xA;&lt;li&gt;genome length too large - total non-gapped sequence length of the assembly is more than 1.5 times that of the average for the genome in the Assembly resource from the same species, more than 15 Mbp, or is otherwise suspiciously long.&lt;/li&gt;&#xA;&lt;li&gt;genome length too small - total non-gapped sequence length of the assembly is less than half that of the average for the genomes in the Assembly resource from the same species, less than 300 Kbp, or is otherwise suspiciously short.&lt;/li&gt;&#xA;&lt;li&gt;hybrid - sequences from a hybrid between different species, strains or isolates.&lt;/li&gt;&#xA;&lt;li&gt;low quality sequence - long stretches of the sequence have a high proportion of ambiguous bases, are low complexity, or have some other indication that the sequence quality is low.&lt;/li&gt;&#xA;&lt;li&gt;mixed culture - sequences come from two or more organisms that were not cultured separately.&lt;/li&gt;&#xA;&lt;li&gt;misassembled - alignment to related genome assemblies or other evidence indicates the assembly is likely to have errors.&lt;/li&gt;&#xA;&lt;li&gt;partial - the assembly has only partial genome representation.&lt;/li&gt;&#xA;&lt;li&gt;sequence duplications - assembly has one or more large duplications.&lt;/li&gt;&#xA;&lt;li&gt;unverified source organism - the origin of the assembly is misidentified.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3967" LastActivityDate="2023-03-31T14:58:29.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20814" PostTypeId="2" ParentId="20809" CreationDate="2023-03-31T15:27:42.463" Score="5" Body="&lt;p&gt;What's best is quite subjective. My preference would be use &lt;a href=&quot;https://www.nextflow.io/&quot; rel=&quot;noreferrer&quot;&gt;Nextflow&lt;/a&gt; to abstract away the underlying job submission system and then use the &lt;a href=&quot;https://www.nextflow.io/docs/latest/executor.html#sge&quot; rel=&quot;noreferrer&quot;&gt;sge&lt;/a&gt; executor to submit the jobs to the cluster. The only issue I see, is that it looks like you're feeding in a list of sample IDs, which your script presumably uses to find some VCF files containing the variants. If the script could be refactored to accept file inputs, we can have Nextflow localize them into its working directory. This would ensure your workflow (however trivial) is portable and could be run on AWS Batch, for example. The following example uses the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#combine&quot; rel=&quot;noreferrer&quot;&gt;combine&lt;/a&gt; operator to create the cartesian product and the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#collectfile&quot; rel=&quot;noreferrer&quot;&gt;collectFile&lt;/a&gt; operator to gather the results. With the following in a file called &lt;code&gt;main.nf&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params.vcf_files = './path/to/*.vcf'&#xA;&#xA;&#xA;process compare_variants {&#xA;&#xA;    input:&#xA;    tuple(&#xA;        path(sample_A_vcf, stageAs: 'A/*'),&#xA;        path(sample_B_vcf, stageAs: 'B/*'),&#xA;    ) &#xA;&#xA;    output:&#xA;    stdout&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    your_script.py \\&#xA;        --sample-vcf-A &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_A_vcf}&quot; \\&#xA;        --sample-vcf-B &quot;$&lt;/span&gt;{sample_B_vcf}&amp;quot; &#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;workflow {&#xA;     &#xA;     samples = Channel.fromPath( params.vcf_files )&#xA;     product = samples.combine(samples)&#xA;&#xA;     compare_variants( product )&#xA;         .collectFile( name: 'results.txt' )&#xA;         .view { &amp;quot;Saved results to: $it&amp;quot; }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the following in a file called &lt;code&gt;nextflow.config&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process {&#xA;&#xA;  executor = 'sge'&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And your Python script in a folder called 'bin' in the root directory of your project repository. Nextflow automatically adds this folder to the $PATH in the execution environment. So make it executable, for example using &lt;code&gt;chmod +x your_script.py&lt;/code&gt;. Then, with Nextflow installed give it a test:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ nextflow run main.nf&#xA;N E X T F L O W  ~  version 22.10.7&#xA;Launching `main.nf` [chaotic_watson] DSL2 - revision: ac224d863b&#xA;executor &amp;gt;  local (484)&#xA;[bc/a39b27] process &amp;gt; compare_variants (482) [100%] 484 of 484 ✔&#xA;Saved results to: /path/to/work/tmp/3e/95330f32edec3f109b197bc836b2ca/results.txt&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-03-31T15:27:42.463" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20815" PostTypeId="2" ParentId="20809" CreationDate="2023-03-31T19:04:48.567" Score="2" Body="&lt;p&gt;You may use &lt;a href=&quot;https://github.com/lh3/asub/blob/master/asub&quot; rel=&quot;nofollow noreferrer&quot;&gt;asub&lt;/a&gt; (array job submitter) for this. Modify your script to &lt;code&gt;myscript.sh&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;array=( A037375001 A037646101 A037710601 A037710701 A037753001 A037844701 A038153001 A038172201 A038184401 A038272502 A038298602 A038533901 A038797201 A038944801 A038973401 A039034701 A039048101 A039089601 A039164101 A039303301 A039323801 A039538301 )&#xA;for i in &amp;quot;${array[@]}&amp;quot;; do&#xA;  for j in &amp;quot;${array[@]}&amp;quot;; do&#xA;    echo &amp;quot;python3 sample_script.py --run-sample-id-A $i --run-sample-id-B $j &amp;gt; out-$i-$j.csv&amp;quot;&#xA;  done&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that the main difference is this script prints command lines, not executing them. You then run asub with&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;bash myscript.sh | asub -q myqueue -j myjobname&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;asub will submit a job array. If you don't like asub, you may submit with&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;bash myscript.sh | xargs -i echo &amp;quot;echo '{}' | qsub -q myqueue&amp;quot; | sh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will submit separate jobs, not a job array. It will be slower and less convenient but you don't need to install anything or learn anything.&lt;/p&gt;&#xA;&lt;p&gt;With either approach, you need to run the following to collate all the results:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;cat out-*-*.csv &amp;gt; our.csv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr/&gt;&#xA;&lt;p&gt;On a side note, this approach can be adapted for &lt;a href=&quot;https://www.gnu.org/software/parallel/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GNU parallel&lt;/a&gt; to spawn multiple processes on the same machine:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;bash myscript.sh | parallel -j 16&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="37" LastEditorUserId="298" LastEditDate="2023-04-01T13:52:24.453" LastActivityDate="2023-04-01T13:52:24.453" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20816" PostTypeId="1" CreationDate="2023-03-31T20:27:32.187" Score="2" ViewCount="197" Body="&lt;p&gt;I have 2 sets of fastq files from another collaborator. The first is contains exome data and the second contains  RNAseq data. But both have the RNA in the name but a different ID. How do I differentiate between a fastq file between exome seq and RNAseq?&lt;/p&gt;&#xA;" OwnerUserId="17261" LastEditorUserId="3694" LastEditDate="2023-04-02T09:33:01.467" LastActivityDate="2023-04-03T20:49:34.407" Title="How to differentiate DNA fastq and RNA fastq files?" Tags="|rna-seq|fastq|exome|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20817" PostTypeId="2" ParentId="20799" CreationDate="2023-03-31T21:56:26.677" Score="1" Body="&lt;p&gt;To answer (a small) part of my own question 'Why is AA not the same as TT in this case?' - I believe the answer to this is that the flanking sequence is important here. We can't just flip an AA to a TT in the middle of a strand without having any effect unless we also flip the flanking sequence also. What matters is the relation between the particular base in question and its flanking sequences. I didn't appreciate that before for some reason but its pretty obvious.&lt;/p&gt;&#xA;" OwnerUserId="17155" LastActivityDate="2023-03-31T21:56:26.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20818" PostTypeId="2" ParentId="20799" CreationDate="2023-03-31T22:13:45.110" Score="1" Body="&lt;p&gt;SNPedia introduces its own orientation to cut the tie to a reference genome, such that the bulk data is independent of the reference genome in use. The intention is good. The implementation is ok. It is just a bit confusing.&lt;/p&gt;&#xA;&lt;p&gt;First of all, orientation is always relative; there is no absolute orientation. When you talk about plus or minus, you have to set a &amp;quot;reference&amp;quot; which could be a reference genome, transcription orientation or even something arbitrary.&lt;/p&gt;&#xA;&lt;p&gt;Then VCF. Orientation in VCF is always relative to the reference that is used to generate the VCF.&lt;/p&gt;&#xA;&lt;p&gt;In case of SNPedia, &amp;quot;Orientation&amp;quot; is secondary information that is relative to GRCh38, the current human reference genome (PS: this field changes with the reference genome but the internal primary strand system doesn't). rs96067 has two alleles C and T. Because its &amp;quot;Orientation&amp;quot; is &amp;quot;minus&amp;quot;, the alleles on the GRCh38 strand is G and A, consistent with the dbSNP information. I checked a few other SNPs. It seems that SNPedia doesn't tell you which is the allele in the reference genome. This is actually fine because what matters is the genotype.&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Stabilized&amp;quot; orientation is relative to an arbitrary orientation stored in the SNPedia database. It is not directly relevant to your question when both you and SNPedia use the same reference genome.&lt;/p&gt;&#xA;&lt;p&gt;How to use SNPedia? When you have a VCF in the GRCh38 coordinate, traverse rsIDs in the VCF. For each rsID, extract the genotype in the GRCh38 coordinate. Then get the SNPedia &amp;quot;Orientation&amp;quot; and &amp;quot;Geno&amp;quot; based on the rsID. If the orientation is &amp;quot;minus&amp;quot;, complement &amp;quot;Geno&amp;quot; (i.e. &amp;quot;A;C&amp;quot; to &amp;quot;T;G&amp;quot;, etc). Compare the genoptype from the VCF to the genotype association from SNPedia to predict the effect of genotype.&lt;/p&gt;&#xA;&lt;hr/&gt;&#xA;&lt;p&gt;PS (clarification in response to new comments): if you design a database that ties to GRCh38 from ground up, you may need to flip many genotypes when the reference genome is changed. This process is complex and error prone. With their current design, if the reference genome is changed, they only need to update &amp;quot;Orientation&amp;quot; without changing the core data. This is a thoughtful design, though the UI could be made clearer.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="73" LastEditDate="2023-04-01T01:03:48.453" LastActivityDate="2023-04-01T01:03:48.453" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20819" PostTypeId="1" AcceptedAnswerId="20820" CreationDate="2023-04-01T10:35:19.483" Score="1" ViewCount="131" Body="&lt;p&gt;I am trying to do alignment among orthologs and paralogs of a gene but I encountered some problems. I have came up with a workflow, but the sequences I extracted are not correct. They are of hugely varying length, and the alignment is very poor.&lt;/p&gt;&#xA;&lt;p&gt;I have attached the workflow below, could anyone suggest if I am missing out on anything or if I am interpreting the files wrongly? Really appreciate your help.&lt;/p&gt;&#xA;&lt;p&gt;My query is: with a list of target genes and bacteria genomes (.fna), for each gene, extract all sequences of it from all genomes and put together into one file for alignment&#xA;&lt;a href=&quot;https://i.stack.imgur.com/piLpj.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/piLpj.jpg&quot; alt=&quot;&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17265" LastActivityDate="2023-04-01T14:58:23.313" Title="How to extract sequences of a specific gene from genomes, with genomes' .fna, .gff and results from KofamScan?" Tags="|gff3|kegg|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20820" PostTypeId="2" ParentId="20819" CreationDate="2023-04-01T11:34:52.130" Score="1" Body="&lt;p&gt;The answer is very simple, just align from amino acids. Amino acids are not subject to saturation - which is what you are encountering and is smashing up the nucleotide alignment. Amino acid will result in a high quality alignment. Thus you think the sequences are wrong, I think the sequences are simply divergent.&lt;/p&gt;&#xA;&lt;p&gt;The other issues of variation in length - that depends on the how you've assembled the genes and exactly what project this is. &lt;em&gt;Sinorhizobium&lt;/em&gt; is a pretty specific bacteria, so it's pretty clear what the biology is - its quite possibly legume agriculture. However, the precise molecular study is not so clear. Basically, what I'm saying is 'variations' in length are not necessarily 'errors' if this is metagenomics.&lt;/p&gt;&#xA;&lt;p&gt;You could extract the amino acids from your &lt;code&gt;fna&lt;/code&gt; file. Or ... use gff format to get the amino acids via &lt;code&gt;gffread -y protein.fa&lt;/code&gt; that sort of thing.&lt;/p&gt;&#xA;&lt;p&gt;The following might not make much sense ... but hope it helps ... to obtain the DNA alignment ... you convert the amino acid alignment back to a nucleotide alignment by transferring the alignment from amino acids on to the nucleotide data using Emboss' &lt;code&gt;tranalign&lt;/code&gt;, amongst many other programs.&lt;/p&gt;&#xA;&lt;p&gt;The tranalign command is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tranalign -asequence [nucleotide filename (fasta)] -bsequence [aligned protein filename (fasta)] -outseq &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;tranalign is also available &lt;a href=&quot;https://www.bioinformatics.nl/cgi-bin/emboss/tranalign&quot; rel=&quot;nofollow noreferrer&quot;&gt;online&lt;/a&gt;, albeit the program is nothing special and its just as easy to write the code.&lt;/p&gt;&#xA;&lt;p&gt;You need a bit of code to translate the nucleotides to amino acids [I use Biopython &lt;code&gt;translate()&lt;/code&gt;] or just use &lt;code&gt;seqotron&lt;/code&gt; of &lt;code&gt;MEGAX&lt;/code&gt; translate function (if you've a Mac) for a GUI.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Warning&lt;/strong&gt; If you use this approach the taxa names (strain names) KOs etc ... need to be identical between amino acid and nucleotide files AND the order needs to be identical too. This can be a headache - but if you encounter errors here simply post back as a separate question and I'll forward the code to get around this 'glitch'.&lt;/p&gt;&#xA;&lt;p&gt;I hope this all sort of makes sense.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;HOWEVER, if you want to assess whether KOfamscan is giving the correct results (which is what you are questioning) you could use&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://academic.oup.com/bioinformatics/article/36/7/2251/5631907&quot; rel=&quot;nofollow noreferrer&quot;&gt;KofamKOALA: KEGG Ortholog assignment based on profile HMM and adaptive score threshold available&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is a webserver available &lt;a href=&quot;https://www.genome.jp/tools/kofamkoala/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; that will do the same thing and should give the same results. The authors consider it an upgrade, but I've not used it.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;There are ALOT of other approaches to this problem BTW and all have  over-reliance on KEGG, whilst it is the most comprehensive database available, is not perfect. KEGG &lt;em&gt;per se&lt;/em&gt; omits upwards of half of all genes. Nobody else needs to know that but as long as you are aware of this thats fine.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-04-01T14:58:23.313" LastActivityDate="2023-04-01T14:58:23.313" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20821" PostTypeId="2" ParentId="20816" CreationDate="2023-04-01T20:34:50.187" Score="5" Body="&lt;p&gt;welcome to the Bioinformatics StackExchange! This is one of those cases where the best solution is to just ask your collaborators. Don't bother with anything technical from the data itself because it's only going to waste your time.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2023-04-01T20:34:50.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20822" PostTypeId="1" CreationDate="2023-04-01T21:41:52.570" Score="1" ViewCount="23" Body="&lt;p&gt;I want to extract the amino acid sequences from KofamScan output, and my workflow is as attached in the picture:&lt;/p&gt;&#xA;&lt;p&gt;For the analysis I am doing, I need to get the animo acid sequences, align them, and do codon alignment with the corresponding nucleotide sequences, so that I can get aligned nucleotides in the end.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, for each copy of the gene, I need both the nucleotide sequence and aa sequence. I am extracting these sequences using the .gff file of the genome where the gene can be found, and for each row on the .gff file, there is a distinct combination of the IDs from the .faa and .fna files, and there is start and end position of the target gene that I can use to extract the sequence on the .fna file. (as shown in the 4th step in the workflow)&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;e.g. NZ_AJQO01000119.1 is the ID from .fna file for nucleotides, and&#xA;FIBHGADE03314 is the ID for aa in the .faa file. They are on the same&#xA;line in the .gff file, and in that line, that start and end positions&#xA;are 60 and 1025 respectively, using the ID NZ_AJQO01000119.1 (as shown in the last step on the workflow).&lt;br /&gt;&#xA;Therefore, I extract the nucleotide sequences based on the info on&lt;br /&gt;&#xA;start and end, and do codon alignment with the aligned aa sequences&lt;br /&gt;&#xA;retrieved using the ID FIBHGADE03314.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, the extracted nucleotide sequences do not seem correct, while the aa sequences looks good.&lt;/p&gt;&#xA;&lt;p&gt;I would like to ask if I am interpreting the ID-ID correspondence wrongly? Or am I interpreting the start and end positions wrongly? Are the numbers inclusive/exclusive?&lt;/p&gt;&#xA;&lt;p&gt;Thank you so much your help.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Hraa1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Hraa1.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17265" LastActivityDate="2023-04-01T21:41:52.570" Title="Extracting animo acid and nucleotide sequences from KofamScan output and codon alignment" Tags="|gff3|gff|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20824" PostTypeId="1" CreationDate="2023-04-02T18:21:19.560" Score="0" ViewCount="45" Body="&lt;p&gt;How to remove orig ident. already data normalized, scaled assigned cluster/ Example, I have 4 sample data. like 1-control, 2- treatment with A, 3- treatment with B, and 4- treatment with C,&#xA;I want to remove. I want to remove 3- treatment with B, and want to use all 3 ident for downstream analysis. without doing again rescaling and assigning cluster cell type. is it possible?&#xA;if yes please tell Sauret r coding so i can run analysis in R&lt;/p&gt;&#xA;" OwnerUserId="17274" LastActivityDate="2023-04-02T18:21:19.560" Title="How to remove orig ident. already data normalized, scaled assigned cluster/" Tags="|r|rna-seq|scrnaseq|seurat|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20825" PostTypeId="1" CreationDate="2023-04-03T06:08:21.727" Score="2" ViewCount="346" Body="&lt;p&gt;I am a software engineer and I am preparing a presentation to aspiring bioinformatics PhDs on how to use best-practice software engineering when publishing code (such as include documentation, modular design, include tests, ...). In particular my presentation will be focused on publishing &amp;quot;pipeline&amp;quot; code, that is, code that is mainly focused on transforming data to a suitable shape for analysis.&lt;/p&gt;&#xA;&lt;p&gt;I am trying to find good example of published bioinformatics pipelines that I can point students to, but as I am not a bioinformatician I am struggling to do so, so would like your help. It doesn't matter if the published pipeline is super-niche or not very popular so long as you think it is engineered well so that I can look it up.&lt;/p&gt;&#xA;&lt;p&gt;Specifically the published code should have: adequate documentation, testing methodology, modular design and be easy to install.&lt;/p&gt;&#xA;" OwnerUserId="17278" LastEditorUserId="298" LastEditDate="2023-04-04T14:03:16.747" LastActivityDate="2023-04-04T14:03:16.747" Title="Where can I find examples of published bioinformatics pipeline packages?" Tags="|data-preprocessing|data-retrieval|publishing|" AnswerCount="2" CommentCount="4" ClosedDate="2023-04-08T19:16:26.073" ContentLicense="CC BY-SA 4.0" />
  <row Id="20826" PostTypeId="2" ParentId="20825" CreationDate="2023-04-03T08:38:44.923" Score="4" Body="&lt;p&gt;I think the nextflow core (&lt;a href=&quot;https://nf-co.re&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nf-co.re&lt;/a&gt;) pipelines should satisfy most of your requirements. These pipelines are written in nextflow (&lt;a href=&quot;https://nextflow.io&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nextflow.io&lt;/a&gt;) which allows scaling from personal to hpc infrastructure rather efficiently.&#xA;The core pipelines are usually quite well documented, and have a module design. I have not really looked into the testing methodology, but I think the modules themselves are tested. Running them is easy once nextflow is installed on your system.&lt;/p&gt;&#xA;" OwnerUserId="17280" LastActivityDate="2023-04-03T08:38:44.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20827" PostTypeId="2" ParentId="20825" CreationDate="2023-04-03T09:55:36.763" Score="4" Body="&lt;p&gt;The GATK team maintains a GitHub repository with their pipelines: &lt;a href=&quot;https://github.com/gatk-workflows&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/gatk-workflows&lt;/a&gt;. They use the &lt;a href=&quot;https://cromwell.readthedocs.io/en/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cromwell Workflow Management System&lt;/a&gt;, an open source tool created by the Broad Institute.&lt;/p&gt;&#xA;&lt;p&gt;You are probably most interested in &lt;a href=&quot;https://github.com/gatk-workflows/gatk4-data-processing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/gatk-workflows/gatk4-data-processing&lt;/a&gt;, their workflow for aliging fastq files against a reference genome to produce a bam file which can be used for downstream analyses such as variant calling:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;h1&gt;gatk4-data-processing&lt;/h1&gt;&#xA;&lt;h2&gt;Purpose :&lt;/h2&gt;&#xA;&lt;p&gt;Workflows for processing high-throughput sequencing data for variant discovery with GATK4 and related tools.&lt;/p&gt;&#xA;&lt;h2&gt;processing-for-variant-discovery-gatk4 :&lt;/h2&gt;&#xA;&lt;p&gt;The processing-for-variant-discovery-gatk4 WDL pipeline implements data&#xA;pre-processing according to the GATK Best Practices. The workflow&#xA;takes as input an unmapped BAM list file (text file containing paths&#xA;to unmapped bam files) to perform preprocessing tasks such as mapping,&#xA;marking duplicates, and base recalibration. It produces a single BAM&#xA;file and its index suitable for variant discovery analysis using tools&#xA;such as Haplotypecaller.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you are starting with FASTQ files visit the seq-format-conversion repository for workflows to convert FASTQs to unmapped BAMS.&lt;/li&gt;&#xA;&lt;li&gt;The processing-for-variant-discovery-gatk4 provides quick and general processing for sequence data using the latest releases of&#xA;GATK. If users are interested in a more elaborate version of this&#xA;workflow with quality control tasks and routinely tested for validity&#xA;(useful in production environments) then visit the&#xA;gatk4-genome-processing-pipeline repository.&lt;/li&gt;&#xA;&lt;li&gt;The BAM output from processing-for-variant-discovery-gatk4 can be used to perform a variety of other analysis like somatic short variant&#xA;discovery, germline short variant discovery, or germline copy number&#xA;variant discovery. Visit the GATK Best Practices documentation to&#xA;determine what to do next with the BAM files.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-04-03T09:55:36.763" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20828" PostTypeId="1" CreationDate="2023-04-03T19:15:01.393" Score="2" ViewCount="53" Body="&lt;p&gt;I've been trying to run &lt;a href=&quot;https://github.com/priyamoorjani/DATES&quot; rel=&quot;nofollow noreferrer&quot;&gt;DATES&lt;/a&gt;, a software that estimates admixture time, on my Linux Mint machine but I keep running into problems. This is the code to run the program:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./dates -p pardates.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The program then runs smoothly, but right before it ends it comes into an error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Calling fit!...&#xA;dates_jackknife -p pardates.txt -l     0.450 -z Assyrian -r 77  -a &amp;gt;Assyrian:log&#xA;**sh: 1: dates_jackknife: not found**&#xA;##end of dates:       28.806 seconds cpu      627.270 Mbytes in use&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have tried the software on other Linux flavors but it keeps giving the same error, despite the fact I have exported the path to the bin directory every single time. I exported the path with this line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;export PATH=/ubuntu/home/Desktop/DATES/bin&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've also checked my version of perl script (dates jackknife's script) and it's up to date.&lt;/p&gt;&#xA;&lt;p&gt;I have contacted the authors about the issue but they told me it's an issue with my environment, which I'm pretty sure is not.&lt;/p&gt;&#xA;" OwnerUserId="17283" LastEditorUserId="73" LastEditDate="2023-04-04T19:51:28.793" LastActivityDate="2023-09-15T14:19:06.683" Title="DATES: sh: 1: dates_jackknife: not found" Tags="|genetics|admixture|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20829" PostTypeId="2" ParentId="20816" CreationDate="2023-04-03T20:49:34.407" Score="3" Body="&lt;p&gt;Absolutely agree with James that you should ask your collaborators, but purely as an academic exercise... assuming you have a good reference genome+annotation (e.g. human/mouse), whole exome seq should align to mostly annotated exonic, RNA-seq to exonic+intronic, and whole genome seq to exonic+intronic+intergenic sequences. Sequence duplication levels may also be higher for RNA-Seq with highly expressed genes often 'deeply' sequenced&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2023-04-03T20:49:34.407" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20830" PostTypeId="1" CreationDate="2023-04-04T01:12:23.800" Score="1" ViewCount="80" Body="&lt;p&gt;I am trying to replicate the heritability estimates based on the &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/35835914/&quot; rel=&quot;nofollow noreferrer&quot;&gt;insomnia GWAS&lt;/a&gt; summary statistics using &lt;a href=&quot;https://github.com/bulik/ldsc&quot; rel=&quot;nofollow noreferrer&quot;&gt;LDSC&lt;/a&gt;. However, I have encountered a problem as my estimates seem to be only about half of the original estimates listed in &lt;a href=&quot;https://static-content.springer.com/esm/art%3A10.1038%2Fs41588-022-01124-w/MediaObjects/41588_2022_1124_MOESM5_ESM.xlsx&quot; rel=&quot;nofollow noreferrer&quot;&gt;Table S1&lt;/a&gt;. Despite my efforts to locate the error, I have been unsuccessful in finding the source of the problem.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/8bPUf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8bPUf.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Below are the commands I used for female estimates.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zcat insomnia_female_ukb2b_EUR_sumstats_20190311_with_chrX_mac_100.txt.gz |cut -f 6,11,12,15,16,17 -d $'\t'|sed  &amp;quot;1s/STAT/Z/&amp;quot; &amp;gt; female_useful.txt&#xA;../munge_sumstats.py --sumstats  female_useful.txt --N-col NMISS --snp RSID_UKB   --a1 A1_UKB  --a2 A2_UKB --out female_insomnia_GWAS_2022 --merge-alleles LDSCORE_w_hm3.snplist --chunksize 500000&#xA;../ldsc.py --h2 female_insomnia_GWAS_2022.sumstats.gz --ref-ld-chr eur_w_ld_chr/ --w-ld-chr eur_w_ld_chr/ --out merged_GWAS_results_h2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The GWAS summary statistics were downloaded from&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://ctg.cncr.nl/documents/p1651/insomnia_female_ukb2b_EUR_sumstats_20190311_with_chrX_mac_100.txt.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ctg.cncr.nl/documents/p1651/insomnia_female_ukb2b_EUR_sumstats_20190311_with_chrX_mac_100.txt.gz&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can anyone tell me where am I wrong? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="17286" LastEditorUserId="17286" LastEditDate="2023-04-04T01:13:41.910" LastActivityDate="2023-04-04T01:13:41.910" Title="Heritability estimate in LD Score Regression (LDSC)" Tags="|statistics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20831" PostTypeId="1" CreationDate="2023-04-04T09:36:12.890" Score="1" ViewCount="65" Body="&lt;p&gt;I want to do a systematic literature review of genetic variant annotation. As part of this I am interested in all the databases available to annotate genetic variants.&lt;/p&gt;&#xA;&lt;p&gt;I want to export all the references cited by the article with PubMed ID 35943817 (&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/35943817/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pubmed.ncbi.nlm.nih.gov/35943817/&lt;/a&gt;), because many of these references are relevant to my review. I can see that they number 93. I want them in a CSV table with their PubMed ID, Title, etc., similar to CSV tables obtained in a search after clicking &amp;quot;Save citations to file&amp;quot;, Format &amp;quot;CSV&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="17128" LastEditorUserId="73" LastEditDate="2023-04-05T19:56:37.380" LastActivityDate="2023-04-06T15:35:01.110" Title="How can I export all and only the publications cited by a specific article?" Tags="|literature-search|" AnswerCount="1" CommentCount="2" ClosedDate="2023-04-08T19:14:07.123" ContentLicense="CC BY-SA 4.0" />
  <row Id="20832" PostTypeId="1" AcceptedAnswerId="20833" CreationDate="2023-04-04T18:40:07.510" Score="-1" ViewCount="66" Body="&lt;p&gt;I have been struggling to get the code I wrote for running GATK g CNV on my Linux machine (Steps Here: &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035531152&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360035531152&lt;/a&gt; ) to work. Specifically it seems to abort a run before running the main code (I even removed my DetermineGermlineContigPloidy step to no avail). Below is my code and then the error:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;    &#xA;## Paths To Be Set&#xA;&#xA;echo &amp;quot;My paths have been set&amp;quot;&#xA;&#xA;### Set Bam Directory:&#xA;#### Make sure to set a seperate directory for the batch you are doing&#xA;&#xA;BAM_DIR=/pathToThe/Batch_I_Am_Running&#xA;&#xA;### Set the path to the reference genome:&#xA;reference=/pathToMy/primary_assembly.fa&#xA;&#xA;#model_dir =/path/to/my/GATK-gCNV&#xA;&#xA;### Set the path to the singularity image:&#xA;SINGULARITY_CACHEDIR=/path/to/my/GATK&#xA;singularity_image=$SINGULARITY_CACHEDIR/gatk_latest.sif&#xA;&#xA;### Set Number Of Bins&#xA;#### Normally 100 to 200*&#xA;&#xA;NUM_BINS=200&#xA;&#xA;### Setting the number of threads ?? Like Cores ??&#xA;&#xA;NUM_THREADS=20&#xA;&#xA;&#xA;&#xA;#______________________________________________________________________________________________________________________________________________________________&#xA;    &#xA;## Main code&#xA;&#xA;echo &amp;quot;I am about to start the main code&amp;quot;&#xA;&#xA;for c in &amp;quot;${CHROMOSOME_NAMES[@]}&amp;quot;; do&#xA;&#xA;  # loop over the directories containing the BAM files&#xA;  for dir in &amp;quot;${BAM_DIR}&amp;quot;/*/; do&#xA;&#xA;    # get the name of the directory without the path&#xA;    dir_name=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$(basename &quot;$&lt;/span&gt;{dir}&amp;quot;)&amp;quot;&#xA;&#xA;    # get the name of the BAM file without the extension&#xA;    bam_name=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$(basename &quot;$&lt;/span&gt;{dir}&amp;quot;/*.bam .bam)&amp;quot;&#xA;&#xA;    echo &amp;quot;Processing file: $bam_name&amp;quot;&#xA;&#xA;    echo &amp;quot;I am starting Step 1: Preprocessing&amp;quot;&#xA;&#xA;    # Step 1: Preprocessing&#xA;    singularity exec &lt;span class=&quot;math-container&quot;&gt;$singularity_image PreprocessIntervals \&#xA;  -R $&lt;/span&gt;{reference} \&#xA;      -I &lt;span class=&quot;math-container&quot;&gt;${dir}/$&lt;/span&gt;{bam_name}.bam \&#xA;      -L targets.interval_list \&#xA;      --bin-length 0 \&#xA;      -imr OVERLAPPING_ONLY \&#xA;      -O targets.preprocessed.interval_list&#xA;&#xA;    echo &amp;quot;I am starting Step 2: AnnotateIntervals with GC content&amp;quot;&#xA;&#xA;    # Step 2: AnnotateIntervals with GC content&#xA;    singularity exec &lt;span class=&quot;math-container&quot;&gt;$singularity_image AnnotateIntervals \&#xA;  -L chr$&lt;/span&gt;{c}.interval_list \&#xA;      -I &lt;span class=&quot;math-container&quot;&gt;${dir}/$&lt;/span&gt;{bam_name}.bam \&#xA;      -R &lt;span class=&quot;math-container&quot;&gt;${reference} \&#xA;  -imr OVERLAPPING_ONLY \&#xA;  -O chr$&lt;/span&gt;{c}.annotated.tsv&#xA;&#xA;    echo &amp;quot;I am starting Step 3: CollectReadCounts&amp;quot;&#xA;&#xA;    # Step 3: Collect read counts for the samples&#xA;    singularity exec &lt;span class=&quot;math-container&quot;&gt;$singularity_image CollectReadCounts \&#xA;  -I $&lt;/span&gt;{dir}/&lt;span class=&quot;math-container&quot;&gt;${bam_name}.bam \&#xA;  -L targets.preprocessed.interval_list \&#xA;  --interval-merging-rule OVERLAPPING_ONLY \&#xA;  --format TSV \&#xA;  -O $&lt;/span&gt;{dir_name}.${bam_name}.counts.tsv&#xA;&#xA;    echo &amp;quot;I am starting Step 4: FilteringIntervals based on GC-content &amp;amp; read depth&amp;quot;&#xA;&#xA;    # Step 4: FilterIntervals based on GC-content and read depth&#xA;    singularity exec &lt;span class=&quot;math-container&quot;&gt;$singularity_image FilterIntervals \&#xA;  -L chr$&lt;/span&gt;{c}.interval_list \&#xA;      --annotated-intervals chr&lt;span class=&quot;math-container&quot;&gt;${c}.annotated.tsv \&#xA;  -imr OVERLAPPING_ONLY \&#xA;  &quot;$&lt;/span&gt;{dir}&amp;quot;/*.counts.tsv \&#xA;      --max-panel-norm 1.5 \&#xA;      --min-panel-norm 0.3 \&#xA;      -O chr${c}.filtered.interval_list&#xA;&#xA;    echo &amp;quot;Not doing Step 5 DetermineGermlineContigPloidy for now&amp;quot;&#xA;&#xA;    # Step 5: DetermineGermlineContigPloidy (Mandatory Step)&#xA;   # singularity exec &lt;span class=&quot;math-container&quot;&gt;$singularity_image DetermineGermlineContigPloidy \&#xA; # --model-parameters $&lt;/span&gt;{model_dir}/model_parameters.txt \&#xA;     # --contig-ploidy-calls &lt;span class=&quot;math-container&quot;&gt;${model_dir}/contig_ploidy_calls.tsv \&#xA; # --contig-ploidy-priors chr20XY_contig_ploidy_priors.tsv \&#xA; # --output-prefix $&lt;/span&gt;{model_dir}/ploidy \&#xA;     # --standardized-output \&#xA;     # &lt;span class=&quot;math-container&quot;&gt;$(find &quot;$&lt;/span&gt;{output_dir}&amp;quot; -name &amp;quot;*.calls.tsv&amp;quot; -printf '%p ') \&#xA;&#xA;#    for file in ${input_dir}/*.tsv&#xA;#    do&#xA;&#xA;    # Get sample ID from filename&#xA;#      sample=&lt;span class=&quot;math-container&quot;&gt;$(basename $&lt;/span&gt;{file} .tsv)&#xA;#      # Run DetermineGermlineContigPloidy command&#xA;#      singularity exec &lt;span class=&quot;math-container&quot;&gt;${singularity_image} DetermineGermlineContigPloidy \&#xA;#        --input $&lt;/span&gt;{file} \&#xA;#        --contig-ploidy-priors &lt;span class=&quot;math-container&quot;&gt;${contig_ploidy_priors} \&#xA;#        --output-prefix $&lt;/span&gt;{output_dir}/${sample}_ploidy \&#xA;#        --standardized-output&#xA;#    done&#xA;&#xA;    echo &amp;quot;Creating shards via bins&amp;quot;&#xA;&#xA;    # Creating shards via bins&#xA;    singularity exec &lt;span class=&quot;math-container&quot;&gt;$singularity_image IntervalListTools \&#xA;  SplitIntervals \&#xA;  -R $&lt;/span&gt;{reference} \&#xA;      -L chr&lt;span class=&quot;math-container&quot;&gt;${c}.interval_list \&#xA;  -scatter $&lt;/span&gt;{NUM_BINS} \&#xA;      -o &lt;span class=&quot;math-container&quot;&gt;${dir}/$&lt;/span&gt;{c}_shard.interval_list&#xA;&#xA;    echo &amp;quot;Starting the Shard Loop that will run GermlineCNVCaller and PostprocessGermlineCNVCalls&amp;quot;&#xA;&#xA;    # Loop over the shards and run GermlineCNVCaller and PostprocessGermlineCNVCalls&#xA;    for shard_file in &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${dir}/$&lt;/span&gt;{c}&amp;quot;*_shard.interval_list; do&#xA;&#xA;      echo &amp;quot;Starting Step 6: Run GermlineCNVCaller (Mandatory Step)&amp;quot;&#xA;&#xA;      # Step 6: Run GermlineCNVCaller (Mandatory Step)&#xA;      singularity exec &lt;span class=&quot;math-container&quot;&gt;$singularity_image GermlineCNVCaller \&#xA;    --run-mode COHORT \&#xA;    -L &quot;$&lt;/span&gt;{shard_file}&amp;quot; \&#xA;        -I &lt;span class=&quot;math-container&quot;&gt;$(find &quot;$&lt;/span&gt;{dir}&amp;quot; -name &amp;quot;*.tsv&amp;quot; -printf '%p ') \&#xA;        --minimum-interval-median-percentile 5.0 \&#xA;        --maximum-zeros-in-sample-percentage 40.0 \&#xA;        --annotation-cutoffs &amp;quot;QD=2.0&amp;quot; &amp;quot;SOR=2.0&amp;quot; &amp;quot;FS=10.0&amp;quot; &amp;quot;MQ=50.0&amp;quot; \&#xA;        --interval-merging-rule OVERLAPPING_ONLY \&#xA;        --output-prefix &lt;span class=&quot;math-container&quot;&gt;$(basename &quot;$&lt;/span&gt;{shard_file}&amp;quot; .interval_list)&#xA;&#xA;      echo &amp;quot;Starting Step 7: Run PostprocessGermlineCNVCalls (Mandatory Step)&amp;quot;&#xA;&#xA;      # Step 7: Run PostprocessGermlineCNVCalls (Mandatory Step)&#xA;      singularity exec &lt;span class=&quot;math-container&quot;&gt;$singularity_image PostprocessGermlineCNVCalls \&#xA;    --model-shard-path &quot;$&lt;/span&gt;{shard_file}&amp;quot; \&#xA;        --cnv-call-path &lt;span class=&quot;math-container&quot;&gt;$(basename &quot;$&lt;/span&gt;{shard_file}&amp;quot; .interval_list).calls.tsv \&#xA;        --output-dir postprocess-sm \&#xA;        --scatter-count ${NUM_THREADS} \&#xA;        --num-scatter-plot-columns 8 \&#xA;        --max-zeros-in-sample-percentage 40.0 \&#xA;        --annotation-cutoffs &amp;quot;QD=2.0&amp;quot; &amp;quot;SOR=2.0&amp;quot; &amp;quot;FS=10.0&amp;quot; &amp;quot;MQ=50.0&amp;quot; \&#xA;        --interval-merging-rule OVERLAPPING_ONLY \&#xA;        --minimum-contig-length 46709983 \&#xA;        --minimum-interval-median-percentile 5.0&#xA;&#xA;    done&#xA;&#xA;&#xA;  done&#xA;&#xA;&#xA;done&#xA;&#xA;    #______________________________________________________________________________________________________________________________________________________________&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &amp;quot;.out&amp;quot; file I received after the run aborted is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;My paths have been set&lt;/p&gt;&#xA;&lt;p&gt;I am about to start the main code&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Why would it abort so soon (I changed the number of Gs used and that did not have any effect)?&lt;/p&gt;&#xA;" OwnerUserId="17244" LastEditorUserId="298" LastEditDate="2023-04-04T18:45:00.573" LastActivityDate="2023-04-04T18:50:00.237" Title="GATK g CNV code Aborts Before Running Main Code" Tags="|gatk|cnv|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20833" PostTypeId="2" ParentId="20832" CreationDate="2023-04-04T18:50:00.237" Score="1" Body="&lt;p&gt;You are using &lt;code&gt;${CHROMOSOME_NAMES[@]}&lt;/code&gt; but you never define that, so when your code gets here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for c in &amp;quot;${CHROMOSOME_NAMES[@]}&amp;quot;; do&#xA;...&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It doesn't enter the &lt;code&gt;for&lt;/code&gt; loop because &lt;code&gt;&amp;quot;${CHROMOSOME_NAMES[@]}&amp;quot;&lt;/code&gt; is an empty array. It's the same thing as doing this in a shell session:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ for c in &amp;quot;${CHROMOSOME_NAMES[@]}&amp;quot;; do echo &amp;quot;This is chromosome $c&amp;quot;; done&#xA;$ &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Just like your script, that will produce no output since there is nothing for it to loop over.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-04-04T18:50:00.237" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20836" PostTypeId="1" CreationDate="2023-04-04T15:13:32.560" Score="5" ViewCount="105" Body="&lt;p&gt;The ladder distance (LD) between residue i and residue j is the minimum number of base pairs (created due to secondary structure of the ssRNA) one has to cross when going from i to j.&lt;/p&gt;&#xA;&lt;p&gt;The Maximum Ladder Distance of a ssRNA molecule is the maximum ladder distance between any two of the residues of this molecule[*].&lt;/p&gt;&#xA;&lt;p&gt;Given the secondary structure of a ssRNA molecule in &lt;a href=&quot;https://www.tbi.univie.ac.at/RNA/ViennaRNA/doc/html/rna_structure_notations.html&quot; rel=&quot;noreferrer&quot;&gt;dot-bracket notation&lt;/a&gt; (e.g. by RNAfold), &lt;strong&gt;how can I calculate the Maximum Ladder Distance of this molecule&lt;/strong&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Below is an illustration from [*]. The MLD is 11, since this is the number of basepairs on the yellow line.&lt;/p&gt;&#xA;&lt;p&gt;Edit: The dot-bracket notation for this molecule would be&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;..(((((.......))...((.((((((.......))))))..)).)).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SJXOg.png&quot; rel=&quot;noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SJXOg.png&quot; alt=&quot;Example MLD for a ssRNA molecule&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;*: Yoffe AM, Prinsen P, Gopal A, Knobler CM, Gelbart WM, et al. (2008)&#xA;Predicting the sizes of large RNA molecules. Proc Natl Acad Sci U S A 105:&#xA;16153–16158&lt;/p&gt;&#xA;" OwnerDisplayName="user36028" LastActivityDate="2023-04-05T14:01:34.343" Title="How to calculate the Maximum Ladder Distance (MLD) of ssRNA from Dot-Bracket notation?" Tags="|rna|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20837" PostTypeId="2" ParentId="20836" CreationDate="2023-04-05T10:05:40.963" Score="5" Body="&lt;p&gt;There are different implementations for calcuating the &lt;span class=&quot;math-container&quot;&gt;$\rm MLD$&lt;/span&gt;. One possible approach is to consider &lt;span class=&quot;math-container&quot;&gt;$\rm MLD$&lt;/span&gt; as the &lt;em&gt;heighest value of all possible &lt;a href=&quot;http://rna.tbi.univie.ac.at/RNAWebSuite/help.html#A7&quot; rel=&quot;noreferrer&quot;&gt;mountain plots&lt;/a&gt;&lt;/em&gt;, generated by shifting the beginning of your sequence. The highest value of a mountain plot tells you the maximal ladder distance that can be reached from &lt;em&gt;the first nucleotide&lt;/em&gt;. But the longest path might not begin at the first nucleotide, and that is why you have to go through all possible shifts of the sequence.&lt;/p&gt;&#xA;&lt;p&gt;Here is an example of this implementation in Python. The algorithm is as following:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Convert dot-bracket notation to a list of base pairs (&lt;code&gt;dot2pairs&lt;/code&gt;).&lt;/li&gt;&#xA;&lt;li&gt;Calculate the first mountain plot.&lt;/li&gt;&#xA;&lt;li&gt;Calculate required &lt;code&gt;shifts&lt;/code&gt;. &lt;em&gt;This is not strictly neccessary because you can swipe through all possible &lt;code&gt;shifts = list(range(0, length))&lt;/code&gt;, but it gives better performance.&lt;/em&gt;&lt;/li&gt;&#xA;&lt;li&gt;Iterate through all &lt;code&gt;shifts&lt;/code&gt;, calculate the &lt;code&gt;MLD&lt;/code&gt; at each step, and return the maximal one.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from collections import deque&#xA;&#xA;def dot2pairs(dot):&#xA;    stack, pairs = deque(), []&#xA;&#xA;    for i, n in enumerate(dot):&#xA;        if n == '(':&#xA;            stack.append(i)&#xA;        elif n == ')':&#xA;            pairs.append((stack.pop(), i))&#xA;    &#xA;    return np.array(pairs)&#xA;&#xA;def dot2MLD(dot):&#xA;    # Convert to a list of base pairs&#xA;    pairs0 = dot2pairs(dot)&#xA;&#xA;    if len(pairs0) == 0:&#xA;        return 0&#xA;&#xA;    MLD = 0&#xA;    shifts = [0]&#xA;    length = len(dot)&#xA;    seq = np.zeros(length, dtype=int)&#xA;    curr_mount = None&#xA;&#xA;    while shifts:        &#xA;        # Shift the sequence&#xA;        pairs = (pairs0 - shifts.pop()) % length&#xA;&#xA;        # Find positions of opening and closing nucleotides&#xA;        opens, closes = np.min(pairs, axis=1), np.max(pairs, axis=1)&#xA;&#xA;        seq.fill(0)&#xA;        seq[opens] = 1&#xA;        seq[closes] = -1&#xA;&#xA;        mountain = np.cumsum(seq)&#xA;&#xA;        MLD = max(MLD, int(np.max(mountain)))&#xA;        &#xA;        # In first iteration, calculate all required shifts&#xA;        if curr_mount is None:&#xA;            curr_mount = mountain[0]&#xA;            for i, m in enumerate(mountain):&#xA;                if m == curr_mount: &#xA;                    shifts.append(i)&#xA;                else:&#xA;                    curr_mount = m&#xA;&#xA;    return MLD&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Let's test it on your structure:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dot2MLD(&amp;quot;..(((((.......)))...((.((((((.......))))))..)).)).&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;11&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And on a &lt;a href=&quot;https://pastebin.com/gKTAgreQ&quot; rel=&quot;noreferrer&quot;&gt;larger structure&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dot2MLD(s)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;1141&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It took ~ 5 seconds for this 17481 nt long structure (and it takes ~ 20 seconds if you don't precalculate requires &lt;code&gt;shifts&lt;/code&gt;).&lt;/p&gt;&#xA;" OwnerUserId="14817" OwnerDisplayName="Domen" LastActivityDate="2023-04-05T10:05:40.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20838" PostTypeId="1" AcceptedAnswerId="20841" CreationDate="2023-04-05T17:55:20.780" Score="1" ViewCount="27" Body="&lt;p&gt;I was doing &lt;code&gt;pathway enrichment analysis&lt;/code&gt; using the below code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;p_value = scipy.stats.hypergeom.sf(len(intersection) - 1, geneset_size, len(pathway), len(hits))&#xA;        if p_value &amp;lt; 0.05:&#xA;            for gene in list(pathway):&#xA;                results = results.append({&amp;quot;Gene&amp;quot;: gene, &amp;quot;Model&amp;quot;: model, &amp;quot;Enriched Pathway&amp;quot;: Pathway_name}, ignore_index=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;hits = df[&amp;quot;Gene&amp;quot;][np.logical_and(df[&amp;quot;Model&amp;quot;] == model, df[&amp;quot;p_value_anova&amp;quot;] &amp;lt; 0.05)]&#xA;intersection = pathway.intersection(hits)&#xA;Model = model name (like A, B, C) in input file&#xA;Gene = Gene from the input gene&#xA;Enriched pathway=Pathways that are enriched from the Pathway input file provided&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now the problem is that earlier this checking pathway enrichment on the basis of p_value in &lt;code&gt;hits&lt;/code&gt; was working but now using cutoff (0.05) is resulting in loss of all pathways. I am looking to test the enrichment using the statistical test that will work on continous measure than binarizing the P-value.&lt;/p&gt;&#xA;" OwnerUserId="4783" LastActivityDate="2023-04-05T23:42:48.990" Title="How to get enriched pathways in the data using continous statistic measure?" Tags="|python|statistics|pathway|go-enrichment|pandas|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20839" PostTypeId="1" CreationDate="2023-04-05T18:05:21.460" Score="1" ViewCount="54" Body="&lt;p&gt;Sorry if the answer to this should be obvious.&lt;/p&gt;&#xA;&lt;p&gt;I have RNA-expression results from 24 samples which can be divided into 6 groups, (wildtype and two different mutants at two different ages) with a total of 24 samples. I have made a PCA plot and when describing it in my thesis said separation between samples is minimal. One of my examiners has said&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;You state that the separation between samples in the principal component analysis is minimal, this needs to be quantified to differentiate between minimal but statistically significant, and not significantly different.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I have no idea how he expects me to test for significance. I know how to test for significant differences between individual samples, and for differences per gene/principle component. I don't know how to group samples together while testing for significance against multiple things or if I should use genes or principle components (probably the latter).&lt;/p&gt;&#xA;&lt;p&gt;I'm writing it in R so if anyone knows which R function can do the kinds of statistical tests I need that would be a huge help. Alternatively is this something which I shouldn't really do and I should push back on?&lt;/p&gt;&#xA;&lt;p&gt;I'm already performing significance testing using DESeq2. To the best of my knowledge that only tests for whether a single gene is significant. It doesn't test for whether the samples are significantly different as a whole which is what I'm after.&lt;/p&gt;&#xA;" OwnerUserId="1843" LastEditorUserId="73" LastEditDate="2023-04-06T20:35:57.453" LastActivityDate="2023-04-06T20:35:57.453" Title="Test for differences between groups of samples" Tags="|r|rna-seq|pca|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20840" PostTypeId="2" ParentId="20839" CreationDate="2023-04-05T18:38:28.803" Score="2" Body="&lt;p&gt;PCA does not do any testing and has no p-values. You may state that a separation of less than &lt;code&gt;m&lt;/code&gt; units in PC &amp;gt; &lt;code&gt;i&lt;/code&gt; is not significant while a separation of greater than &lt;code&gt;n&lt;/code&gt; units in PC &amp;lt; &lt;code&gt;i&lt;/code&gt; is significant as PCs &amp;lt; &lt;code&gt;i&lt;/code&gt; explain X% of the variation (&lt;code&gt;i&lt;/code&gt; being 2 or 3). That's how you can back your statement on the separation being minimal.&lt;/p&gt;&#xA;&lt;p&gt;That is, in this sample plot:&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.ibb.co/7VqDbBc/image.png&quot; alt=&quot;sample PCA&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;PC2 distance of &amp;lt; 0.05 might be called trivial compared to PC1 distance &amp;gt; 0.5 if PC1 explains 80% of the variance. This is just 2 PCs though, so your actual case will need to be more robust.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-04-05T18:38:28.803" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20841" PostTypeId="2" ParentId="20838" CreationDate="2023-04-05T23:31:20.090" Score="1" Body="&lt;p&gt;Just to mention I personally think there is a higher loop above the loop stated above.&lt;/p&gt;&#xA;&lt;p&gt;I've not looked at each bit of code in detail but&#xA;there's a bug is on line 4:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;results = results.append({&amp;quot;Gene&amp;quot;: gene, &amp;quot;Model&amp;quot;: model, &amp;quot;Enriched Pathway&amp;quot;: Pathway_name}, ignore_index=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Firstly, this is fashionable coding because you are dumping straight into a &lt;code&gt;pandas&lt;/code&gt; dataframe (described below). The correction is simply ... declare &lt;code&gt;pandas&lt;/code&gt; in advance&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;results = pd.DataFrame()&#xA;p_value = # code missing&#xA;if p_value &amp;lt; 0.05:&#xA;    for gene in list(pathway):&#xA;        results.append({&amp;quot;Gene&amp;quot;: gene, &amp;quot;Model&amp;quot;: model, &amp;quot;Enriched Pathway&amp;quot;: Pathway_name}, ignore_index=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The thing is if there is a higher loop above &lt;code&gt;list(pathway)&lt;/code&gt; then &lt;code&gt;results = results.append(....&lt;/code&gt; will over-ride the last set of results. This would explain why results 'disappear' because if the last data in the loop is a blank - or very little output, all previous stuff gets deleted.&lt;/p&gt;&#xA;&lt;p&gt;I personally would use a dictionary here. Thus&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# declare the dictionary before all loops     &#xA;results = {&amp;quot;Gene&amp;quot;: [], &amp;quot;Model&amp;quot;: [], &amp;quot;Enriched Pathway&amp;quot;: []} &#xA;p_value = # code missing&#xA;if p_value &amp;lt; 0.05:&#xA;    for gene in list(pathway):&#xA;       genedict = {&amp;quot;Gene&amp;quot;: gene, &amp;quot;Model&amp;quot;: model, &amp;quot;Enriched Pathway&amp;quot;: Pathway_name}&#xA;       for k,v in mydicti.values():&#xA;           results[v].append(v)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... you could then transfer this to &lt;code&gt;pandas&lt;/code&gt; once the loops are complete via,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df = pd.DataFrame(genedict)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I do suspect there's other issues, but as a general rule &lt;code&gt;results = results.append(...&lt;/code&gt; is better declaring &lt;code&gt;results&lt;/code&gt; first and then appending to &lt;code&gt;results&lt;/code&gt;, unless you've a very specific reason (e.g. writing each loop into a separate file). The &lt;code&gt;pandas&lt;/code&gt; thing is kinda clever for big data sets, because &lt;code&gt;pandas&lt;/code&gt; is fast, whilst dictionaries are slow - for big data (like &amp;gt;&amp;gt;500 000 big). The interchangeability between &lt;code&gt;dictionaries&lt;/code&gt; and &lt;code&gt;pandas&lt;/code&gt; means it's kinda easier conceptually (and code-wise) to switch at the coders convenience in my personal opinion.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-04-05T23:42:48.990" LastActivityDate="2023-04-05T23:42:48.990" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20842" PostTypeId="2" ParentId="20831" CreationDate="2023-04-06T08:52:53.623" Score="2" Body="&lt;p&gt;Something like this will work, it did just now for me.  You might want to extract different info from the records though.&lt;br /&gt;&#xA;This exports to a table (bibliography.tsv) columns containing:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;pmid&lt;/li&gt;&#xA;&lt;li&gt;authors&lt;/li&gt;&#xA;&lt;li&gt;title&lt;/li&gt;&#xA;&lt;li&gt;journal&lt;/li&gt;&#xA;&lt;li&gt;year&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;max_attempts = 5&#xA;def fetch_pubmed_data(pmid):&#xA;    Entrez.email = &amp;quot;you@email.com&amp;quot;  # Replace with your email address    &#xA;    handle = Entrez.efetch(db=&amp;quot;pubmed&amp;quot;, id=pmid, rettype=&amp;quot;xml&amp;quot;, retmode=&amp;quot;text&amp;quot;)&#xA;    record = Entrez.read(handle)&#xA;    time.sleep(1)&#xA;    return record&#xA;&#xA;&#xA;def extract_cited_articles(pmid):&#xA;    &#xA;    record = fetch_pubmed_data(pmid)&#xA;    cited_pmids = []&#xA;&#xA;    record_article = record['PubmedArticle']&#xA;&#xA;    if &amp;quot;PubmedData&amp;quot; in record_article[0] and &amp;quot;ReferenceList&amp;quot; in record_article[0][&amp;quot;PubmedData&amp;quot;]:&#xA;        for reference in record_article[0][&amp;quot;PubmedData&amp;quot;][&amp;quot;ReferenceList&amp;quot;][0][&amp;quot;Reference&amp;quot;]:&#xA;            if &amp;quot;ArticleIdList&amp;quot; in reference and len(reference[&amp;quot;ArticleIdList&amp;quot;]) &amp;gt; 0:&#xA;                cited_pmids.append(reference[&amp;quot;ArticleIdList&amp;quot;][0])&#xA;&#xA;    return cited_pmids&#xA;&#xA;&#xA;def extract_bibliographic_info(pmid_list, max_attempts):&#xA;    bibliographic_info = {}&#xA;    bibliographic_info_ls = []&#xA;&#xA;    for pmid in pmid_list:&#xA;&#xA;        authors = title = journal = year = &amp;quot;&amp;quot;&#xA;        record = {}&#xA;&#xA;        for attempt in range(max_attempts):&#xA;            try:&#xA;                record = fetch_pubmed_data(pmid)&#xA;                break&#xA;&#xA;            except:&#xA;                time.sleep(5)&#xA;&#xA;        if 'PubmedArticle' in record:&#xA;            if len(record['PubmedArticle']) &amp;gt;0:&#xA;                article = record['PubmedArticle'][0][&amp;quot;MedlineCitation&amp;quot;][&amp;quot;Article&amp;quot;]&#xA;&#xA;                if &amp;quot;AuthorList&amp;quot; in article:&#xA;                    authors = &amp;quot;, &amp;quot;.join([f&amp;quot;{author['LastName']} {author['Initials']}&amp;quot; for author in article[&amp;quot;AuthorList&amp;quot;] if 'LastName' in author and 'Initials' in author])&#xA;                    title = article[&amp;quot;ArticleTitle&amp;quot;]&#xA;                    journal = article[&amp;quot;Journal&amp;quot;][&amp;quot;Title&amp;quot;]&#xA;                    year = article[&amp;quot;Journal&amp;quot;][&amp;quot;JournalIssue&amp;quot;][&amp;quot;PubDate&amp;quot;][&amp;quot;Year&amp;quot;]&#xA;&#xA;&#xA;        print (pmid, authors, title, journal, year)&#xA;        bibliographic_info_ls.append([pmid, authors, title, journal, year])&#xA;&#xA;    return bibliographic_info_ls&#xA;&#xA;&#xA;def main():&#xA;    pmid = input(&amp;quot;Enter the PubMed article ID: &amp;quot;)&#xA;    cited_pmids = extract_cited_articles(pmid)&#xA;    bibliographic_info_ls = extract_bibliographic_info(cited_pmids, max_attempts)&#xA;&#xA;    df = pd.DataFrame(bibliographic_info_ls, columns=[&amp;quot;pmid&amp;quot;, &amp;quot;authors&amp;quot;, &amp;quot;title&amp;quot;, &amp;quot;journal&amp;quot;, &amp;quot;year&amp;quot;])&#xA;    df.to_csv(&amp;quot;bibliography.tsv&amp;quot;, sep=&amp;quot;\t&amp;quot;, index=False)&#xA;    return df&#xA;    &#xA;if __name__ == &amp;quot;__main__&amp;quot;:&#xA;    df = main()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17293" LastEditorUserId="3694" LastEditDate="2023-04-06T15:35:01.110" LastActivityDate="2023-04-06T15:35:01.110" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20843" PostTypeId="1" CreationDate="2023-04-06T16:07:28.220" Score="1" ViewCount="96" Body="&lt;p&gt;I have about 10 fasta files that I need to pull from a giant folder of other fasta files (has about 100 files). I have a list of the files I need in a .txt file separated by line and I want to pull out the 10 fasta files listed and combine them into one big file.&lt;/p&gt;&#xA;&lt;p&gt;I tried this command at first:&#xA;&lt;code&gt;xargs &amp;lt; names.txt cat &amp;gt; names.fasta&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;But it gave me this error:&#xA;&lt;code&gt;cat: 'read_1.fasta'$'\r': No such file or directory&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;For all the files except the last read in the list which it managed to add to the &lt;code&gt;names.fasta&lt;/code&gt; .&lt;/p&gt;&#xA;&lt;p&gt;I'm not sure what the error message means given I can find the files in my folder so they should be there. I can do this manually but I know it'll take a while to do.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;: I changed the list to be separated by space and it actually pulled out all the files I needed! However, is there a way to account for reading in the names by newline instead of space??&lt;/p&gt;&#xA;" OwnerUserId="16067" LastActivityDate="2023-04-06T18:23:30.793" Title="Combine separate fasta files from a list.txt" Tags="|fasta|command-line|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20844" PostTypeId="2" ParentId="20843" CreationDate="2023-04-06T16:45:42.107" Score="2" Body="&lt;p&gt;This should work&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;xargs &amp;lt; names.txt cat &amp;gt; names.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or else try,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat names.txt | xargs cat &amp;gt; newdir/names.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What I think is within &lt;code&gt;names.txt&lt;/code&gt; there's a typo or a missing file and that's what it's complaining about.&lt;/p&gt;&#xA;&lt;p&gt;Perhaps try using&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ls &amp;gt; tmp&#xA;cat names.txt | sort &amp;gt; tmp2&#xA;diff tmp tmp2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If there are any differences that should pick it up.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-04-06T16:45:42.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20845" PostTypeId="2" ParentId="20843" CreationDate="2023-04-06T18:23:30.793" Score="1" Body="&lt;p&gt;Just to address an edge case, run &lt;code&gt;dos2unix names.txt&lt;/code&gt; first and then a modified version of @M__'s command&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat names.txt | xargs -I v_file cat v_file &amp;gt;&amp;gt; names.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Based on your edit, it looks like your working platform has different line endings compared to the platform on which &lt;code&gt;names.txt&lt;/code&gt; was last edited/saved.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-04-06T18:23:30.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20846" PostTypeId="1" AcceptedAnswerId="20848" CreationDate="2023-04-07T03:00:22.043" Score="1" ViewCount="338" Body="&lt;p&gt;I have this output which is going to do bwa next which is from join operator&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[DF122_SO11378_A03, DF122_SO11378, DF122_SO11378_A03.norm.merged.fq, DF122_SO11378_A03.final.gfa, DF122_SO11378_A03.final.fasta]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;in this case, &lt;code&gt;pair_id&lt;/code&gt; is &lt;code&gt;DF122_SO11378_A03&lt;/code&gt;,&#xA;&lt;code&gt;plate_id&lt;/code&gt; is &lt;code&gt;DF122_SO11378&lt;/code&gt;, &lt;code&gt;fq&lt;/code&gt; is &lt;code&gt;DF122_SO11378_A03.norm.merged.fq&lt;/code&gt;,&#xA;&lt;code&gt;DF122_SO11378_A03.final.gfa&lt;/code&gt;  is &lt;code&gt;gfa&lt;/code&gt;&#xA;&lt;code&gt;DF122_SO11378_A03.final.fasta&lt;/code&gt;  is &lt;code&gt;fa&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have following code like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process bwa {&#xA;&#xA;     input:&#xA;     tuple val(pair_id), val(plate_id), path(fq), path(gfa), path(fa) from align_in&#xA;&#xA;     output:&#xA;     path &amp;quot;*bam&amp;quot; into bam&#xA;     tuple val(pair_id), val(plate_id), path (&amp;quot;*.csv&amp;quot;) into metrics&#xA;&#xA;     &amp;quot;&amp;quot;&amp;quot;&#xA;     if [ -s &lt;span class=&quot;math-container&quot;&gt;$fa ]; then&#xA;     bwa index $&lt;/span&gt;fa&#xA;&#xA;     bwa mem &lt;span class=&quot;math-container&quot;&gt;$fa $&lt;/span&gt;fq \&#xA;      | samtools view -bh -F2048 - \&#xA;      | samtools sort &amp;gt; ${pair_id}.bam&#xA;&#xA;     samtools depth -a &lt;span class=&quot;math-container&quot;&gt;${pair_id}.bam &amp;gt; $&lt;/span&gt;{pair_id}.depth.csv&#xA;     samtools view -c &lt;span class=&quot;math-container&quot;&gt;${pair_id}.bam &amp;gt;$&lt;/span&gt;{pair_id}.count.csv&#xA;     &#xA;     else&#xA;     touch &lt;span class=&quot;math-container&quot;&gt;${pair_id}.depth.csv&#xA;     touch $&lt;/span&gt;{pair_id}.count.csv&#xA;     touch ${pair_id}.bam&#xA;     fi&#xA;     &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, now my upstream work flow changed a little, for some sample they have more than one gfa and fasta output like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[DF122_SO11378_A04, DF122_SO11378, DF122_SO11378_A04.norm.merged.fq, [DF122_SO11378_A04.2.final.gfa, DF122_SO11378_A04.final.gfa], [DF122_SO11378_A04.2.final.fasta, DF122_SO11378_A04.final.fasta]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;in this case, &lt;code&gt;pair_id&lt;/code&gt; is &lt;code&gt;DF122_SO11378_A04&lt;/code&gt;,&#xA;&lt;code&gt;plate_id&lt;/code&gt; is &lt;code&gt;DF122_SO11378&lt;/code&gt;, &lt;code&gt;fq&lt;/code&gt; is &lt;code&gt;DF122_SO11378_A04.norm.merged.fq&lt;/code&gt;,&#xA;&lt;code&gt;[DF122_SO11378_A04.2.final.gfa, DF122_SO11378_A04.final.gfa]&lt;/code&gt; is &lt;code&gt;gfa&lt;/code&gt;,&#xA;&lt;code&gt;[DF122_SO11378_A04.2.final.fasta, DF122_SO11378_A04.final.fasta]&lt;/code&gt; is &lt;code&gt;fa&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;But some samples still have one gfa and fasta as shown earlier.&lt;/p&gt;&#xA;&lt;p&gt;How can I adapt my bwa process to handle such situations?&lt;/p&gt;&#xA;&lt;p&gt;I am using DSL1.&lt;/p&gt;&#xA;" OwnerUserId="14191" LastEditorUserId="10775" LastEditDate="2023-04-07T13:50:58.727" LastActivityDate="2023-04-07T13:50:58.727" Title="how to handle variable output from nextflow" Tags="|nextflow|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20847" PostTypeId="1" CreationDate="2023-04-07T09:22:24.287" Score="3" ViewCount="133" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://biology.stackexchange.com/questions/111873/when-it-is-stated-in-your-consumer-genetics-testing-report-that-you-have-x-let&quot;&gt;Biology SE&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;When a raw number like 1.3 percent is stated, it is confusing without context,and can be misinterpreted. So does 1.3 percent mean that 1.3 percent of the letters/nucleotides/bases of my genome consists of established (likely/high confidence) Neanderthal markers or SNPS? Does it mean 1.3 percent of my entire genome consists of these Neanderthal markers, i.e 1.3 percent of 3.1 billion or 6.2 billion base pairs, or is it 1.3 percent of protein coding regions, or exons?**&lt;/p&gt;&#xA;&lt;p&gt;How do they obtain these markers for Neanderthal ancestry to begin with? I know the Neanderthal genome has been sequenced from bone fragments. Do they compare SNPs and variable regions in the genomes between Neanderthal and Modern Human population genomes to see the relative frequencies of these variations/SNPs. Then likely Neanderthal markers are flagged by statistical models and algorithms..for example combinations of SNPs that appear infrequently or hardly at all in the reference Sapien population genome but occur frequently in the Neanderthal reference genome..that marks these SNPs/variable regions as of likely Neanderthal origin? Could you please describe some of these statistical methods and algorithms? Can anything else explain the &amp;quot;markers&amp;quot;, or SNPs other than interbreeding with Neanderthal (such as mutations)? Please illustrate using some hypothetical alignments like AAGGGT..vs GAGGGT..Please give a clear, concise and authoritative answer of high confidence. It is okay to be detailed and technical, although building up to complexity so that it can be understood with relatively little familiarity with population genetics and Mathematics.&lt;/p&gt;&#xA;" OwnerUserId="17297" LastEditorUserId="3694" LastEditDate="2023-04-09T11:27:32.823" LastActivityDate="2023-05-11T07:41:15.223" Title="When it is stated in your consumer genetics testing report that you have x% (lets say 1.3 percent...) Neanderthal DNA, what is actually meant?" Tags="|snp|variation|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20848" PostTypeId="2" ParentId="20846" CreationDate="2023-04-07T13:30:24.567" Score="1" Body="&lt;p&gt;If you're looking for the transposition, you could use the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#transpose&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;transpose&lt;/code&gt;&lt;/a&gt; operator to get what you want. But to handle the variable output, you would need to first check what types you have. For this, you could use something like the following for example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;input_ch&#xA;    .map { pair_id, plate_id, fastq, gfa, fasta -&amp;gt;&#xA;&#xA;        def gfa_files = gfa instanceof Path ? [gfa] : gfa&#xA;        def fasta_files = fasta instanceof Path ? [fasta] : fasta&#xA;&#xA;        tuple( pair_id, plate_id, fastq, gfa_files, fasta_files )&#xA;    }&#xA;    .transpose()&#xA;    .view()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that the old DSL1 is no longer supported.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-04-07T13:30:24.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20849" PostTypeId="2" ParentId="20847" CreationDate="2023-04-07T14:54:12.863" Score="2" Body="&lt;p&gt;I ran a basic search of 23andme and it looks like they use known variant loci from the Neanderthal genome. Also, they expect a 2-2.5% overlap. See:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://customercare.23andme.com/hc/en-us/articles/212873707-Neanderthal-Ancestry-Report-Basics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://customercare.23andme.com/hc/en-us/articles/212873707-Neanderthal-Ancestry-Report-Basics&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://blog.23andme.com/articles/new-neanderthal-report&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blog.23andme.com/articles/new-neanderthal-report&lt;/a&gt; (Section &amp;quot;The Science behind the report&amp;quot;)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://permalinks.23andme.com/pdf/23-05_neanderthal_ancestry_inference.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://permalinks.23andme.com/pdf/23-05_neanderthal_ancestry_inference.pdf&lt;/a&gt; (Eric Durand's white paper on the technical details)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-04-07T14:54:12.863" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20850" PostTypeId="1" CreationDate="2023-04-08T01:13:29.800" Score="0" ViewCount="76" Body="&lt;p&gt;I attempt to do a selective sweep analysis by using the software &lt;a href=&quot;https://bioweb.pasteur.fr/packages/pack@XP-CLR@1.0&quot; rel=&quot;nofollow noreferrer&quot;&gt;XP-CLR1.0&lt;/a&gt;. My genotypic data is VCF format, so I wonder if someone with experience on this package could recommend any tool to convert from VCF to the input file format required by this software?&lt;/p&gt;&#xA;&lt;p&gt;Output files I need are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;geno file: contains the genotype of a single SNP, for example:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;1 0 1 1 9 9&#xA;1 1 1 0 0 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It contains two SNPs (rows) from three individuals (columns). First two columns are the two alleles of individual #1. Values 1 and 0 correspond to two allele types. 9 denotes missing data.&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;snp file: each row contains information of each SNP, for example:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;snp_name, chr#, genetic_distance(cM), Physical_distance, Ref_allele, Alt_allele&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17299" LastEditorUserId="17299" LastEditDate="2023-04-10T05:38:39.243" LastActivityDate="2023-04-10T05:38:39.243" Title="How to convert VCF files to input file format required by XP-CLR1.0 software?" Tags="|vcf|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20851" PostTypeId="2" ParentId="20847" CreationDate="2023-04-08T23:09:45.780" Score="2" Body="&lt;p&gt;Based on reading a similar post it seems like if the report says I am 1.3 percent Neanderthal  then it  does NOT  mean 1.3% of my  base pairs or SNPs of my genome (or exons or protein coding region or total number of my SNPs) has a &amp;quot;Neanderthal&amp;quot; label inferred, rather some kind of statistical inference is made as a whole.&lt;/p&gt;&#xA;&lt;p&gt;The  author suggests when it comes to 23AndMe  that my genome is segmented into chunks of a chromosome (containing a set of SNPs, 23AndMe does genotyping only reads parts of chromosome where bases are known to vary.)&lt;/p&gt;&#xA;&lt;p&gt;Technical details could be referred to the following sites:  &lt;a href=&quot;https://academic.oup.com/bib/article/19/5/765/3038466&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://academic.oup.com/bib/article/19/5/765/3038466&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.cd-genomics.com/the-principles-and-workflow-of-snp-microarray.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cd-genomics.com/the-principles-and-workflow-of-snp-microarray.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.illumina.com/techniques/popular-applications/genotyping/snp-snv-genotyping.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.illumina.com/techniques/popular-applications/genotyping/snp-snv-genotyping.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since 23andMe is using v5 genotyping chip, the fraction of Neanderthal variant could be calculated.  Moreover, the basis of v5 genotyping chip is related to the concept that DNA-based polymorphisms (SNPs or SNVs) which differ substantially in frequency among human populations can be employed to infer ancestry.  With 23andMe, an average of 236 Neanderthal SNPs/SNVs could be obtained with the analysis based on a selected group of consented customers.&lt;/p&gt;&#xA;&lt;p&gt;And these chunks are fed into a Machine learning classification  algorithm like Support vector machine (SVM). The Machine learning algorithm was trained chunks of genome of  reference population sets with chunks of known ancestry (like &amp;quot;Scandinavian&amp;quot; or &amp;quot;South European&amp;quot; or &amp;quot;Sub Saharan African&amp;quot;).&lt;/p&gt;&#xA;&lt;p&gt;Then the classification algorithm makes a multilabel prediction... lets say the algorithm predicts that a particular  segment is 95 percent likely to be human and 5 percent Neanderthal..and maybe the next segment has a  2% probability of being human and 98 percent Neanderthal..Then the algorithm looks for segments of high confidence (&amp;gt;90 percent or 70 percent whatever threshold you set) and then add them up.&lt;/p&gt;&#xA;&lt;p&gt;For example if in a toy example genome has 9 segments of the same size has a  greater than 90 percent probability of being estimated &amp;quot;human&amp;quot;, and 1 segment with greater than 90 percent of being estimated  Neanderthal by the classification algorithm than I would be  considered to be 10 percent Neanderthal...&lt;/p&gt;&#xA;&lt;p&gt;The author suggests this is the method used to infer ancestry like &amp;quot;Norwegian&amp;quot; and &amp;quot;South European&amp;quot; and &amp;quot;Sub Saharan African&amp;quot; by 23And Me..but will it be similar for inferring Neanderthal ancestry?&lt;/p&gt;&#xA;&lt;p&gt;Calculating ancestry percentage is not seem as  simple as counting up number of &amp;quot;Neanderthal&amp;quot; SNPs/Markers and dividing by your total SNPs or base pairs in your genome/exon...Rather an inference of ancestry mix needs to be made based on reading SNPs on segments of chromosomes and comparing it to reference data sets via classification algorithms.&lt;/p&gt;&#xA;" OwnerUserId="17297" LastEditorUserId="17300" LastEditDate="2023-04-10T11:56:05.863" LastActivityDate="2023-04-10T11:56:05.863" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20853" PostTypeId="1" CreationDate="2023-04-10T08:06:57.747" Score="-2" ViewCount="355" Body="&lt;p&gt;Organism under investigation is  &lt;em&gt;Plasmodium falciparum&lt;/em&gt;. How to calculate the allele frequency for each row?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Q6EOC.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Q6EOC.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I tried with this code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    import pandas as pd&#xA;&#xA;# Read CSV file&#xA;# df = pd.read_csv(&amp;quot;Pf_M76611.pf7.200_samples - Copy.csv&amp;quot;)&#xA;&#xA;df= pd.read_csv(&amp;quot;/content/Pf_M76611.pf7.200_samples - Copy.csv&amp;quot;)&#xA;&#xA;# Define function to calculate allele frequencies&#xA;def calc_allele_freq(row):&#xA;    # Get total number of samples&#xA;    num_samples = len(row) - 9&#xA;&#xA;    # Get reference allele&#xA;    ref_allele = row['REF']&#xA;&#xA;    # Get alternative alleles&#xA;    alt_alleles = row['ALT'].split(',')&#xA;&#xA;    # Initialize list to store allele frequencies&#xA;    allele_freqs = []&#xA;&#xA;    # Loop over alternative alleles&#xA;    for alt_allele in alt_alleles:&#xA;        # Initialize count for this allele&#xA;        count = 0&#xA;&#xA;        # Loop over samples&#xA;        for i in range(9, len(row)):&#xA;            # Get genotype for this sample&#xA;            gt = row[i].split(':')[0]&#xA;&#xA;            # Count this allele if it is present in the genotype&#xA;            if alt_allele in gt:&#xA;                count += 1&#xA;&#xA;        # Calculate frequency for this allele&#xA;        freq = count / num_samples&#xA;&#xA;        # Add frequency to list&#xA;        allele_freqs.append(freq)&#xA;&#xA;    # Add frequency for reference allele&#xA;    allele_freqs.insert(0, 1 - sum(allele_freqs))&#xA;&#xA;    # Return allele list and frequency list as a tuple&#xA;    allele_list = [ref_allele] + alt_alleles&#xA;    freq_list = [round(freq, 1) for freq in allele_freqs]&#xA;    return (allele_list, freq_list)&#xA;&#xA;# Apply function to each row of the dataframe&#xA;allele_freq_df = df.apply(calc_allele_freq, axis=1, result_type='expand')&#xA;&#xA;# Add columns to dataframe&#xA;df['Allele List'] = allele_freq_df[0]&#xA;df['Proportion List'] = allele_freq_df[1]&#xA;&#xA;# Select columns for output and drop duplicates&#xA;# output_df = df[['#CHROM', 'POS', 'Allele List', 'Proportion List']].drop_duplicates()&#xA;&#xA;# # Convert output dataframe to string and print&#xA;# print(output_df.to_string(index=False))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17284" LastEditorUserId="73" LastEditDate="2023-04-11T20:56:23.427" LastActivityDate="2023-04-12T00:09:29.843" Title="How to calculate Allele frequency from vcf file" Tags="|python|sequence-analysis|genomics|allele-frequency|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20854" PostTypeId="1" AcceptedAnswerId="20855" CreationDate="2023-04-10T21:01:01.717" Score="2" ViewCount="202" Body="&lt;p&gt;I ran the following code in bash (I added print statements for ease):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Module loading&#xA;&#xA;echo &amp;quot;I am loading the specified modules&amp;quot;&#xA;&#xA;module load Singularity/version_number&#xA;&#xA;# Define the path to the singularity:&#xA;&#xA;singularity_image=&amp;quot;/path/to/folder/GATK/gatk_latest.sif&amp;quot;&#xA;&#xA;echo &amp;quot;I am going to make the `.dict ` file&amp;quot;&#xA;&#xA;singularity exec $singularity_image /gatk/gatk --java-options -Xms8000m \&#xA;  CreateSequenceDictionary \&#xA;  -R /path/to/my/primary_assembly.fa \&#xA;  -O /path/to/the/primary_assembly.dict&#xA;&#xA;echo &amp;quot;I am all done&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, I get the following &lt;code&gt;out&lt;/code&gt; file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;I am loading the specified modules&#xA;job_scripts_number: line 23:&#xA;.dict: command not found I am going to make the  file &#xA;INFO  NativeLibraryLoader - Loading libgkl_compression.so from&#xA;jar:file:/gatk/gatk-package-4.4.0.0-local.jar!/path/to/libgkl_compression.so [Today's date] CreateSequenceDictionary --OUTPUT&#xA;path/to/my/primary_assembly.dict&#xA;--REFERENCE /path/to/my/primary_assembly.fa&#xA;--TRUNCATE_NAMES_AT_WHITESPACE true --NUM_SEQUENCES 2147483647 --VERBOSITY INFO --QUIET false --VALIDATION_STRINGENCY STRICT --COMPRESSION_LEVEL 2 --MAX_RECORDS_IN_RAM 999999 --CREATE_INDEX false --CREATE_MD5_FILE false --help false --version false --showHidden false --USE_JDK_DEFLATER false --USE_JDK_INFLATER false [Today's date] Executing as user@node_number on Linux version&#xA;; OpenJDK 64-Bit Server VM version;&#xA;Deflater: A; Inflater: A; Provider is available; Picard&#xA;version: Version:4.4.0.0 [Today's date]&#xA;picard.sam.CreateSequenceDictionary done. Elapsed time: 0.53 minutes.&#xA;Runtime.totalMemory()=8388608000 Tool returned: 0 Using GATK jar&#xA;/gatk/gatk-package-4.4.0.0-local.jar Running:&#xA;    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -Xms8000m -jar /gatk/gatk-package-4.4.0.0-local.jar CreateSequenceDictionary -R&#xA;/path/to/primary_assembly.fa&#xA;-O /path/to/primary_assembly.dict&#xA;I am all done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Why would I get the error &amp;quot;.dict: command not found I am going to make the  file &amp;quot; especially when I look at the &lt;code&gt;head&lt;/code&gt; of the .dict file and it seems to look the way it should?&lt;/p&gt;&#xA;" OwnerUserId="17244" LastEditorUserId="17244" LastEditDate="2023-05-26T21:00:02.683" LastActivityDate="2023-05-26T21:00:02.683" Title="Singularity &amp; GATK 4.4.0.0. CreateSequenceDictionary" Tags="|gatk|docker|dictionary|singularity|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20855" PostTypeId="2" ParentId="20854" CreationDate="2023-04-10T22:44:45.163" Score="5" Body="&lt;p&gt;This has nothing to do with Singularity/Docker or GATK. You get the error &lt;code&gt;.dict: command not found&lt;/code&gt; because you've used backticks to perform a &lt;a href=&quot;https://www.gnu.org/software/bash/manual/html_node/Command-Substitution.html&quot; rel=&quot;noreferrer&quot;&gt;command substitution&lt;/a&gt; in a prior &lt;code&gt;echo&lt;/code&gt; command:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;code&gt;echo &amp;quot;I am going to make the `.dict ` file&amp;quot;&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Backticks are legacy syntax, you could have also written:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;echo &amp;quot;I am going to make the $(.dict) file&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But neither of these makes any sense - there is no &lt;code&gt;.dict&lt;/code&gt; command. Of course single quotes would prevent the command substitution:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;echo 'I am going to make the `.dict` file'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But this might cause some confusion when the output is displayed. A better choice would be to use double quotes, but without the backticks. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;echo &amp;quot;I am going to make the '.dict' file&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2023-04-10T23:12:23.940" LastActivityDate="2023-04-10T23:12:23.940" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20857" PostTypeId="2" ParentId="19948" CreationDate="2023-04-11T05:34:31.557" Score="1" Body="&lt;p&gt;From the documentation at &lt;a href=&quot;https://kb.10xgenomics.com/hc/en-us/articles/115003646912-How-is-sequencing-saturation-calculated-&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://kb.10xgenomics.com/hc/en-us/articles/115003646912-How-is-sequencing-saturation-calculated-&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;All of these samflag 0x400 reads have an xf tag value of 17, which consist of bits 1 and 16. This also means these reads do not have the xf bit of 8, which mark representative reads from a group of duplicates. The converse isn't true though. The xf17 consist mostly of samflag 0x400 duplicate reads but also of samflag nonduplicate reads.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="17308" LastActivityDate="2023-04-11T05:34:31.557" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20858" PostTypeId="1" CreationDate="2023-04-11T11:27:52.070" Score="15" ViewCount="1938" Body="&lt;p&gt;We have a lot of Illumina sequenced exome data. Currently we are using &lt;a href=&quot;https://github.com/shubhamchandak94/Spring&quot; rel=&quot;noreferrer&quot;&gt;spring&lt;/a&gt; for its great lossless compression, but we are looking if there is anything better (and most preferably opensource) which can let us compress our fastq files.&lt;/p&gt;&#xA;&lt;p&gt;We also want to compress or reduce file size of BAM, but using CRAM in lossless mode didn't yield good results. The space savings were decent, roughly ~10% (IMO was expecting more), but the biggest hit came when using it with IGV. We have set up a http-server and stream bams to IGV (Helped us immensely as people wouldn't have to download and visualize bam files). With Cram, the loading times were significantly high, requiring users to commit more memory for IGV. Sometimes we would have to wait around a minute for reads to show up. With Bam it is near instantaneous.&lt;/p&gt;&#xA;&lt;p&gt;Can't use lossy compression as these BAM files might be subject to future downstream analysis (Ex. CNV calling, variant calling or something else).&lt;/p&gt;&#xA;&lt;p&gt;What are some good ways to compress fastq and bam files so we can maximize our storage?&lt;/p&gt;&#xA;" OwnerUserId="6561" LastEditorUserId="6251" LastEditDate="2023-04-12T19:42:43.220" LastActivityDate="2023-05-23T11:26:05.740" Title="Good / recommended way to archive fastq and bam files?" Tags="|sam|fastq|cram|storage|" AnswerCount="5" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20859" PostTypeId="2" ParentId="20858" CreationDate="2023-04-11T12:09:29.343" Score="3" Body="&lt;p&gt;You might want to consider &lt;a href=&quot;https://www.petagene.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PetaGene&lt;/a&gt;. This is a paid tool, and not open source, but it is something we use at work and it is really good! It can do lossless compression of gzip-compressed fastq files, bam files, and even cram files, and achieves really good size reduction (image from the PetaGene webpage):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8JqUZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8JqUZ.png&quot; alt=&quot;petagene compression performance bar graph&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In my own testing, using files of different sizes, I get an average size reduction of 62.1% (2.6 x smaller) for fastq.gz files (compared to the gzipped fastq file, not compared to the original, uncompressed fastq file) and an average reduction of 82.1% (5.6 x smaller) for bam files. I didn't test cram files.&lt;/p&gt;&#xA;&lt;p&gt;If you have the budget for a paid service, these guys do good work! The greatest advantage, apart from the very significant size reduction, is that they also provide a system that allows you to use the compressed files natively. So you can feed the petagene-compressed bam file directly to bwa for example. They create special symlinks (&lt;code&gt;foo.fastq.gz -&amp;gt; foo.fastq.petacompressed&lt;/code&gt;) and you can then use these symlinks and petagene handles the decompression on the fly.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The &lt;a href=&quot;https://en.wikipedia.org/wiki/CRAM_(file_format)&quot; rel=&quot;nofollow noreferrer&quot;&gt;Wikipedia page on CRAM&lt;/a&gt; has a nice table with a few alternatives to CRAM which I reproduce below. I haven't tried any except PetaGene though so cannot comment on their performance:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;Name&lt;/th&gt;&#xA;&lt;th&gt;Key advantage over CRAM&lt;/th&gt;&#xA;&lt;th&gt;Repo&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Deez&lt;/td&gt;&#xA;&lt;td&gt;None. An early approach to SAM compression, introduced 2014.&lt;/td&gt;&#xA;&lt;td&gt;&lt;a href=&quot;https://github.com/sfu-compbio/deez&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/sfu-compbio/deez&lt;/a&gt;&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Genozip&lt;/td&gt;&#xA;&lt;td&gt;Significantly better compression, but slower random access&lt;/td&gt;&#xA;&lt;td&gt;&lt;a href=&quot;https://github.com/divonlan/genozip&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/divonlan/genozip&lt;/a&gt;&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;GenomSys&lt;/td&gt;&#xA;&lt;td&gt;Utilizes ISO-standard MPEG-G, but compression inferior to CRAM 3.1&lt;/td&gt;&#xA;&lt;td&gt;Source code not publicly available&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;PetaSuite&lt;/td&gt;&#xA;&lt;td&gt;Virtual filesystem interface&lt;/td&gt;&#xA;&lt;td&gt;Source code not publicly available&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2023-05-23T11:26:05.740" LastActivityDate="2023-05-23T11:26:05.740" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20861" PostTypeId="2" ParentId="20858" CreationDate="2023-04-11T14:11:18.687" Score="3" Body="&lt;p&gt;An alternative to the good option of PetaGene that @terdon mentioned and also not open source is the &lt;a href=&quot;https://developer.illumina.com/news-updates/introducing-dragen-original-read-archive-ora&quot; rel=&quot;nofollow noreferrer&quot;&gt;DRAGEN ORA compression&lt;/a&gt;. This is loseless reference based compression and can be ~5 x smaller over fastq.gz, but it's only for fastq files and only those generated by Illumina machines (and will be easiest within the DRAGEN ecosystem).&lt;/p&gt;&#xA;&lt;p&gt;Another point to consider is where you store the data. Even with 'standard' compression you might get more benefits moving archival data to a system like &lt;a href=&quot;https://en.wikipedia.org/wiki/Amazon_S3_Glacier&quot; rel=&quot;nofollow noreferrer&quot;&gt;AWS S3 Glacier&lt;/a&gt;, depending on your current infastructure setup of course&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2023-04-11T14:11:18.687" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20862" PostTypeId="2" ParentId="19866" CreationDate="2023-04-11T14:56:23.830" Score="1" Body="&lt;p&gt;For proteins and molecules in general, and within the meaning of this paper, &amp;quot;internal coordinates&amp;quot; are any representation that are not dependent on the molecule rotation or translation in a Cartesian space.&lt;/p&gt;&#xA;&lt;p&gt;Calling them &amp;quot;internal coordinates&amp;quot; may be slightly confusing, and a more correct term could be &amp;quot;internal representation&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;That means, I guess, that internal coordinates &lt;strong&gt;could&lt;/strong&gt; be, as you mention, a list of vector in a Cartesian space, where the Cartesian space &amp;quot;orientation&amp;quot; would be decided by, say, principle component analysis (or any other arbitrary rule depending on the molecule). This however defeats the purpose of most internal representations: dimensionality reduction or faster computations.&lt;/p&gt;&#xA;&lt;p&gt;Other (more interesting) ways of representing a molecule in internal coordinates would be:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;As (&lt;a href=&quot;https://onlinelibrary.wiley.com/doi/full/10.1002/jcc.26372&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bayati, Lardan &amp;amp; Bardhan 2020&lt;/a&gt;) focus on: a set of bond distance, angle, and &lt;strong&gt;dihedrals&lt;/strong&gt; (see Ramachandran plots and rotamers). This is especially useful in the case of proteins. This representation is for example used by &lt;a href=&quot;https://doi.org/10.1146/annurev.biochem.77.062906.171838&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rosetta&lt;/a&gt;. Angles and bond distance can be considered as constant (depending on the atomic type), and so one can infer the Cartesian coordinates of multiple atoms with just one dihedral angle, hence providing a huge dimensionality reduction. In this case there is no need for vector. Also, in the case of internal energy computation there is always one energy term depending on dihedrals, so that is already done. Limit of such methods is that in molecular constructs there are energy terms due to long-range interactions, i.e van der waals and electrostatics. There is hence a need in such software to always switch between a Dihedral representation and a Cartesian representation. Another problem is that proteins backbone are then represented as a dihedral &amp;quot;chain&amp;quot;, and so the Cartesian coordinates of one atom becomes tightly linked to the previous one, which are linked to the previous one, etc. This is the problem this paper is trying to deal with.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;As &lt;a href=&quot;https://arxiv.org/pdf/2204.10673.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;(Tan et al. 2023)&lt;/a&gt; use, a &lt;strong&gt;graph representation&lt;/strong&gt;. &lt;a href=&quot;https://github.com/a-r-j/graphein&quot; rel=&quot;nofollow noreferrer&quot;&gt;Graphein&lt;/a&gt; is an interesting python library for converting and analyzing proteins and molecules from cartesian coordinates to graph representation. In this case instead of having a vector, each atoms position is defined as its distance with surrounding atoms. This kind of representation has received more and more attention from the protein structure prediction &amp;amp; design community as it is particularly suited for neural networks. For example I would say the protein representation in AlphaFold is a kind of graph representation (using &amp;quot;triangles&amp;quot; made by the distance between three atoms). In comparison to dihedral representation, graph representation packs less &amp;quot;assumption&amp;quot; about the system and are even worst than Cartesian representation in terms of dimension (if you do a &amp;quot;global&amp;quot; representation of N atoms then you end up with N! values vs only 3*N with Cartesian), but at least things like non-covalent interactions are implicitly represented.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I believe this answer gives enough material to look at. If this still appears unclear, skimming through Rosetta tutorials / papers figures / webinars should provide ample explanations, same for AlphaFold.&lt;/p&gt;&#xA;" OwnerUserId="12054" LastActivityDate="2023-04-11T14:56:23.830" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20864" PostTypeId="2" ParentId="20858" CreationDate="2023-04-11T18:10:10.360" Score="3" Body="&lt;p&gt;The only free and open source tool I know that can help is &lt;a href=&quot;https://en.wikipedia.org/wiki/Zstd&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;zstd&lt;/code&gt;&lt;/a&gt;. Their github repository's README describes it as:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Zstandard, or zstd as short version, is a fast lossless compression algorithm, targeting real-time compression scenarios at zlib-level and better compression ratios. It's backed by a very fast entropy stage, provided by Huff0 and FSE library.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;We read &lt;a href=&quot;http://www.bioinformaticszen.com/post/use-zstd-for-raw-fastq/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this blog post&lt;/a&gt; on using it to compress fastq files and were intrigued so we ran some tests. We couldn't reproduce the same level of compression that the article claims but that might be due to different input files or the way an index was/wasn't created.&lt;/p&gt;&#xA;&lt;p&gt;Even in our tests though, we did see that it provides better compression than gzip. Here are the specific values for one of the tests, run using an R1 file of a human WGS run:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: center;&quot;&gt;Uncompressed fastq&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;bgzip-compressed&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: center;&quot;&gt;zstd-compressed&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;179G&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;58G&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;47G&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;This was done using &lt;code&gt;-z 19&lt;/code&gt;, to set the compression level to 19 (we found this to be the best tradeoff between size and time to compress). We also saw that using a human genome fasta file to create a compression index, and then using that when compressing human fastq files, actually resulted in worse performance (larger compressed file) than not using the index. But it is entirely possible that we just did it wrong, we didn't spend a lot of time on it.&lt;/p&gt;&#xA;&lt;p&gt;So you might want to investigate &lt;code&gt;zstd&lt;/code&gt; as well, maybe you can get better performance if you play around with the indexing. On the plus side, it is free&#xA;and open source. On the other hand, it won't do much with bam or cram files.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2023-04-14T11:43:13.767" LastActivityDate="2023-04-14T11:43:13.767" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20865" PostTypeId="2" ParentId="19613" CreationDate="2023-04-11T18:56:17.427" Score="2" Body="&lt;p&gt;wish there was a more subtle answer, but this works.  Place this previous to your VlnPlot&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;update_geom_defaults(&amp;quot;violin&amp;quot;, aes(linewidth = 0))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17313" LastEditorUserId="650" LastEditDate="2023-04-11T19:25:29.093" LastActivityDate="2023-04-11T19:25:29.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20866" PostTypeId="2" ParentId="20858" CreationDate="2023-04-11T20:49:16.553" Score="5" Body="&lt;p&gt;I'd recommend reconsidering CRAM. It's an open standard that is most likely to still be accessible many years in the future. Regarding your point about lossy compression...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Can't use lossy [CRAM] compression as these BAM files might be subject to future downstream analysis (Ex. CNV calling, variant calling or something else).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;By default, CRAM compression is not lossy.&lt;/p&gt;&#xA;&lt;p&gt;CRAM does, however, have an optional archive settings mode (&lt;code&gt;samtools view --output-fmt cram --output-fmt-option archive ...&lt;/code&gt;) which &lt;em&gt;is&lt;/em&gt; a lossy compression, doing things like removing read names, removing additional accessory fields, and additional compression of quality scores. In all cases, the base sequence of the reads is preserved.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.htslib.org/doc/samtools.html&quot; rel=&quot;noreferrer&quot;&gt;https://www.htslib.org/doc/samtools.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-04-11T20:49:16.553" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20867" PostTypeId="2" ParentId="20853" CreationDate="2023-04-11T22:58:06.740" Score="0" Body="&lt;p&gt;Coding: first and foremost all operations should be performed directly in &lt;code&gt;pandas&lt;/code&gt;. It is much more efficient and alot easier. You don't need all the loops, &lt;code&gt;pandas&lt;/code&gt; does that. &lt;code&gt;apply&lt;/code&gt; is okay. I would use a very different strategy (below).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The actually contents of the &lt;code&gt;csv&lt;/code&gt; are not clear.&lt;/p&gt;&#xA;&lt;p&gt;This is what I think might have happened: there's   multiple VCFs against the same reference I assume these were loaded and a database &lt;code&gt;join&lt;/code&gt; operation performed using the reference genome column 'REF' as the index for the &lt;code&gt;join&lt;/code&gt; across all input VCFs. I've placed this here because I'm not 100% sure what the input file represents - if it's not what I think it is - thats how I'd get to the allelic frequency.&lt;/p&gt;&#xA;&lt;p&gt;Why do I think that?&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Without multiple strains the allelic frequency cannot be calculated.&lt;/li&gt;&#xA;&lt;li&gt;The file states '200 samples'&lt;/li&gt;&#xA;&lt;li&gt;The presence of multiple alleles in the 'ALT' column.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The problems are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;heterozygosity. This assumes falciparum is in its haploid state (it has a diploid stage). If its the diploid stage the allelic frequency cannot be calculated using this input file because the linkage was discarded (which is really important)&lt;/li&gt;&#xA;&lt;li&gt;The underlying problem is the number of actual strains isn't explicitly stated .&lt;/li&gt;&#xA;&lt;li&gt;the frequency of the 'ALT' doesn't appear to be declared&lt;/li&gt;&#xA;&lt;li&gt;even if the a frequency was declared (but not presented) that must be linked to the 'ALT' column, each allele needs a exact frequency, otherwise population genetics will not work on this data.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;First conclusion&lt;/strong&gt; The allelic frequency is performed across each row using the reference genome as the reference via &lt;code&gt;value_counts()&lt;/code&gt; divided by the number of strains and presented in its own column. Somehow the relevant columns are missing - will need to see them.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;However&lt;/strong&gt; Again on the information presented I don't think it can be done because if this represents 200 samples the frequency of a given allele is not declared in this data, or simply the number of 'ALT's across all VCFs. Without accessing that number the allelic frequency is not doable because it's entirely dependent on it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Possible alternative conclusion&lt;/strong&gt; Its possible the question needs to be redefined not as &lt;em&gt;allelic frequency&lt;/em&gt; - this has an exact definition within population genetics, but &lt;em&gt;genetic diversity&lt;/em&gt;, i.e. remove the term 'frequency' altogether. What I think is being calculated is genetic diversity of all samples in the file against a reference genome. If that is correct it would be good to change the question accordingly. These are different mathematical entities.&lt;/p&gt;&#xA;&lt;p&gt;Whatever operation is required is easy to calculate, but its obtaining that information from the input file and the precise nature of the investigation.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-04-12T00:09:29.843" LastActivityDate="2023-04-12T00:09:29.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20868" PostTypeId="2" ParentId="20858" CreationDate="2023-04-11T23:32:23.050" Score="12" Body="&lt;p&gt;EDIT: I am rewriting the answer in response to updates to the original question.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;TL;DR&lt;/strong&gt;: use CRAM&lt;/p&gt;&#xA;&lt;h3&gt;Background 1: quality binning and FASTQ compression&lt;/h3&gt;&#xA;&lt;p&gt;In the old days, base callers outputted base quality at full resolution – you could see quality from Q2 to Q40 in full range. As a result, quality strings were like semi-random strings and very difficult to compress. Later people gradually realized that keeping base quality in low resolution wouldn't affect downstream analysis. The Illumina basecaller started to output quality in 8 distinct values and later changed that to 4 bins. This change greatly simplified quality string and made them compressed better. For example, in old days, a 30X BAM would take ~100 GB. With quality binning, it would only take ~60 GB.&lt;/p&gt;&#xA;&lt;h3&gt;Background 2: GATK Base quality recalibration&lt;/h3&gt;&#xA;&lt;p&gt;In early 2010s, Illumina base quality was not calibrated well. GATK people introduced BQSR to correct that and observed noticeable improvement in SNP accuracy. Nonetheless, with improved Illumina base caller, their base quality became more accurate. Meanwhile, the world moved to 30X deep sequencing. The depth overwhelms slight inaccuracy in quality. I would say around 2015, BQSR was already unnecessary for data produced at the time.&lt;/p&gt;&#xA;&lt;p&gt;Does it hurt to apply BQSR? Yes. First, BQSR introduces subtle biases towards the reference and towards known SNPs. Second, BQSR distorts the data. At least for some datasets, I observed that SNP accuracy dropped with variant quality after BQSR; I didn't observe this with raw quality. Third, BQSR is slow. Fourth, for new sequencers producing data at higher quality, BQSR is likely to decrease data quality. Last, related to the question, BQSR added another semi-random quality string and made compression even harder. Nowadays, running BQSR is a waste of resource for worse results. The official GATK best practice no longer uses BQSR according to their &lt;a href=&quot;https://github.com/gatk-workflows/gatk4-germline-snps-indels/blob/master/haplotypecaller-gvcf-gatk4.wdl&quot; rel=&quot;nofollow noreferrer&quot;&gt;WDL file&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h3&gt;CRAM&lt;/h3&gt;&#xA;&lt;p&gt;These days a 30X human CRAM only takes ~15 GB (see &lt;a href=&quot;https://ftp.sra.ebi.ac.uk/vol1/run/ERR324/ERR3240144/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this file&lt;/a&gt;). This is a huge contrast to ~100 GB BAM in early 2010s. OP only saw ~10% saving probably due to a) BQSR and/or b) old data with full quality resolution.&lt;/p&gt;&#xA;&lt;p&gt;On encoding/decoding speed, CRAM was much slower than BAM. Not any more. The latest htslib implementation of CRAM is faster than BAM on encoding and only slightly slower on decoding. The poor performance of IGV on CRAM could be that the Java CRAM decoder is not as optimized.&lt;/p&gt;&#xA;&lt;p&gt;It is true that CRAM is not as widely supported as BAM. However, all the other alternatives are much worse. Petagene said they had &lt;a href=&quot;https://www.petagene.com/wp-content/uploads/2019/04/PetaGene-IGV-Windows-Mac-Press-Release-FINAL.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;IGV-PG&lt;/a&gt; (PDF) for their format. That is not official IGV and I couldn't find more recent update beyond the 2019 press release. I don't see other viable options.&lt;/p&gt;&#xA;&lt;p&gt;Note that the common practice is to keep all raw reads, mapped or not, in BAM or CRAM such that you can get raw reads back later. BAM/CRAM additionally keeps metadata like read group, sample name, run information etc and is actually more popular than FASTQ in large sequencing centers. Also note that you don't need to sort CRAM by coordinate. Unsorted CRAM is only a little larger than sorted CRAM.&lt;/p&gt;&#xA;&lt;h3&gt;CRAM and its competitors&lt;/h3&gt;&#xA;&lt;p&gt;The core CRAM developer, James Bonfield, is one of the most knowledgeable researchers on compression (and one of the best C programmers) in this field. He has done a lot of compression evaluation over the years. The conclusion is that on a fair benchmark, CRAM is comparable to the best tools so far in terms of compression ratio.&lt;/p&gt;&#xA;&lt;p&gt;Petagene could compress better in the plot @terdon showed mostly because it has a special treatment of the &lt;code&gt;OQ&lt;/code&gt; tag generated by GATK BQSR. It is a typical trick marketing people use to make their methods look better. With BQSR phased out, this plot is no longer relevant.&lt;/p&gt;&#xA;&lt;h3&gt;On commercial software&lt;/h3&gt;&#xA;&lt;p&gt;In general, I welcome commercial tools and think they are invaluable to users. I also have huge respect to DRAGEN developers. However, on FASTQ storage, I would strongly recommend against closed-source compressors. If those tools go under, you may lose your data. Not worth it.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2023-04-12T17:38:32.693" LastActivityDate="2023-04-12T17:38:32.693" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="20869" PostTypeId="1" AcceptedAnswerId="20870" CreationDate="2023-04-12T02:23:55.363" Score="3" ViewCount="65" Body="&lt;p&gt;I have 96  fasta files (A1, A2, A3...) from one  plasmid assembly pipeline, and I have another 96 fasta  files (B1, B2, B3 ...) from another plasmid assembly pipeline.&#xA;I would like to compare pair wise (A1 to B1, A2 to B2, A3 to B3 etc) and collect some metrics to see how this two assembly method differs.&lt;/p&gt;&#xA;&lt;p&gt;I am thinking of doing blastn -query A1 -subject B1, but this created a lot visual alignment, rather than detailed align metrics.&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your suggestions.&lt;/p&gt;&#xA;" OwnerUserId="14191" LastEditorUserId="3848" LastEditDate="2023-06-02T23:24:52.913" LastActivityDate="2023-06-02T23:24:52.913" Title="compare fasta sequences in pairs and collect metrics" Tags="|phylogenetics|fasta|phylogeny|assembly|blast|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20870" PostTypeId="2" ParentId="20869" CreationDate="2023-04-12T13:13:33.780" Score="3" Body="&lt;p&gt;It's easy; just use output &lt;code&gt;format 6&lt;/code&gt;, this gives you a nice table that's easy to parse:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastn -db plasmid2.db -query plasmid1.fasta -evalue 1e-6 -outfmt &amp;quot;6 pident evalue bitscore&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I don't know what parameters you prefer.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;... &amp;quot;6 pident evalue bitscore&amp;quot; ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You just edit the &lt;code&gt;pident&lt;/code&gt;, &lt;code&gt;evalue&lt;/code&gt; and &lt;code&gt;bitscore&lt;/code&gt; above.  You might want to place &lt;code&gt;staxid&lt;/code&gt; or something to tell you what it is. A selection of parameters are as follows:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;staxid ssciname qseqid sseqid pident length mismatch gapopen qstart qend sstart send evalue bitscore qcovs sstrand&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You might want to make the command a bit fancier&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastn -db plasmid2.db -query plasmid1.fasta -evalue 1e-7 -outfmt &amp;quot;6 pident evalue bitscore&amp;quot; | sort | uniq -c | sort -n &amp;gt;&amp;gt; myplasmid.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="298" LastEditDate="2023-04-12T16:20:13.797" LastActivityDate="2023-04-12T16:20:13.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20872" PostTypeId="1" CreationDate="2023-04-12T17:33:07.317" Score="2" ViewCount="47" Body="&lt;p&gt;I have the following question about the paper &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0196885815000135&quot; rel=&quot;nofollow noreferrer&quot;&gt;A parsimony-based metric for phylogenetic trees&lt;/a&gt; by V. Moulton and T. Wu.&#xA;In this paper, the authors define a &amp;quot;parsimony-distance&amp;quot; between phylogenetic trees on the same set of taxa. They study how this distance is related to other common distances used in phylogenetics, such as counting &lt;a href=&quot;https://en.wikipedia.org/wiki/Tree_rearrangement&quot; rel=&quot;nofollow noreferrer&quot;&gt;tree bisection and reconnection&lt;/a&gt; (TBR) moves.&lt;/p&gt;&#xA;&lt;p&gt;For each notion of distance, we can define the &amp;quot;neighborhood&amp;quot; of a tree as the set of trees which have distance one from the first specified tree.&#xA;It is claimed in the paper that the parsimony-distance neighborhood contains the TBR-distance neighborhood, for any tree.&lt;/p&gt;&#xA;&lt;p&gt;Lemma 6.2 states that only caterpillar trees have unit distance to some other tree. This seems to suggest that for any non-caterpillar tree, the parsimony-neighborhood should be empty.&#xA;However, the TBR-neighborhood of any tree should be non-empty, since it is possible to apply a TBR move and obtain a different tree.&#xA;Are there some assumptions I am missing about when the parsimony-neighborhood is empty, or when the TBR-neighborhood is contained in the parsimony neighborhood?&lt;/p&gt;&#xA;&lt;p&gt;Also, when applying the formula in Corollary 6.6 to the non-caterpillar tree on 6 leaves, I am unsure how to apply the formula to give the neighborhood size  &lt;span class=&quot;math-container&quot;&gt;$|N_p| = 0$&lt;/span&gt;, which should be implied by Lemma 6.2.&lt;/p&gt;&#xA;" OwnerUserId="17320" LastEditorUserId="3848" LastEditDate="2023-09-10T15:40:17.453" LastActivityDate="2024-02-07T17:02:30.330" Title="Parsimony-distance neighborhoods in tree space" Tags="|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20875" PostTypeId="2" ParentId="20872" CreationDate="2023-04-13T03:40:48.923" Score="0" Body="&lt;p&gt;The divergence in language makes it difficult to conceptualise and without that it's difficult to approach algebraic abstraction, albeit I've worked on tree space and its landscapes. I suspect I get point 3, but not much more.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&amp;quot;Caterpillar tree&amp;quot; is a term in graph theory. If that is the definition used here then every parsimony tree would be a caterpillar tree, there would never be a &amp;quot;non-caterpillar tree&amp;quot;. This is because each branch length defines one &amp;quot;step&amp;quot;: &amp;quot;parsimony's version of a mutation&amp;quot;, so each &amp;quot;step&amp;quot; within the branch could be considered to be a &amp;quot;branch-less node&amp;quot;, or else bifurcating node.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Ladderized tree I don't understand that. Maybe Np = 0 means a complete polytomy, i.e. a star tree, but could be a branch length = 0, i.e. genetically identical taxa.&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;The concept for an empty neighborhood ... possibly if all topologies are equally parsimonious this could degenerate into an empty tree neighborhood, again a star tree. What parsimony does is always define a bifurcating tree regardless, over replication all alternative solutions may result collectively in an &amp;quot;empty neighborhood&amp;quot;. This might be fine from an algebraic perspective, its weird from algorithm perspective though.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;When a TBR-neighborhood is equal to a parsimony neighborhood that actually makes sense. I define that as the TBR-neighborhood being &amp;quot;trapped&amp;quot; within the parsimony neighborhood and thats convergence. Here the &amp;quot;step length&amp;quot; has reached its minimum plateau and that tree is the most parsimonious tree ... that makes complete sense. This also makes sense within the context of the paper (I haven't read it - but I can see where they are heading).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;What the authors want to do is pick holes in TBR so they want to define the point at which TBR has reached its most parsimonious tree ... I assume they want to show TBR can make errors and their solution is better. Thus they need to define the &amp;quot;end point&amp;quot;. At this &amp;quot;end point&amp;quot; they want to show their measure outperforms TBR to reach a lower step size.&lt;/p&gt;&#xA;&lt;p&gt;Thus they will want to imply that TBR should be replaced with their derivation. Its a nice methodological way to basic (or &amp;quot;fundamental&amp;quot;) phylogentics.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-04-13T03:46:51.583" LastActivityDate="2023-04-13T03:46:51.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20876" PostTypeId="1" AcceptedAnswerId="20885" CreationDate="2023-04-13T14:37:08.753" Score="1" ViewCount="138" Body="&lt;p&gt;I notice when using tools like HOMER there are two different types of outputs: de novo motifs vs known motifs and  it seems no matter what the data is (type of ChIP etc.) there is always a &amp;quot;de novo&amp;quot; motif set in the results.&lt;/p&gt;&#xA;&lt;p&gt;I was hoping to reach out to experts on insight about how much of the space of motif based structure, within a genome, has been identified. Is there any consensus on what portion of the motif like patterns, for example in human or mouse genome, have been identified? When I find de novo motifs using HOMER in my ChIP-seq or ATAC peaks, what are the chances that in my common-place ChIP experiment (PU.1 for example) I find de novo motifs, not recorded in the databases (i.e. JASPER).&lt;/p&gt;&#xA;" OwnerUserId="15741" LastEditorUserId="15741" LastEditDate="2023-04-15T03:32:16.777" LastActivityDate="2023-04-16T15:06:05.270" Title="How much of the space of motifs have been discovered?" Tags="|motifs|homer|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20877" PostTypeId="2" ParentId="14983" CreationDate="2023-04-13T16:42:30.433" Score="1" Body="&lt;p&gt;Following the &lt;a href=&quot;https://jef.works/blog/2017/03/28/CIGAR-strings-for-dummies/&quot; rel=&quot;nofollow noreferrer&quot;&gt;basic rules&lt;/a&gt; and preferring the &lt;code&gt;AAG&lt;/code&gt; contiguous&#xA;we can write:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  01234&#xA;  AAGCT     (ref) &#xA;AAAAGGC     (query)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So the CIGAR string is &lt;code&gt;2I3M1I1D&lt;/code&gt; that is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;2I&lt;/code&gt; insert 2&lt;code&gt;A&lt;/code&gt;s  at the start:&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;3M&lt;/code&gt; match the 3 bases &lt;code&gt;AAG&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;1I&lt;/code&gt; insert the &lt;code&gt;G&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;1D&lt;/code&gt; delete the final &lt;code&gt;T&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17327" LastEditorUserId="17327" LastEditDate="2023-04-13T16:47:55.333" LastActivityDate="2023-04-13T16:47:55.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20878" PostTypeId="1" AcceptedAnswerId="20880" CreationDate="2023-04-13T19:54:12.630" Score="3" ViewCount="252" Body="&lt;p&gt;I am trying to run Picard in Singularity and try as I might I cannot seem to find the path to use. For example with GATK 4.4.0.0 I use the following path:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;singularity exec $singularity_image /gatk/gatk --java-options -Xms8000m \&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;where &lt;code&gt;singularity_image&lt;/code&gt; is a variable that encodes the path to the gatk.sif. However, I cannot seem to figure out the path I need to use for picard.&lt;/p&gt;&#xA;&lt;p&gt;When I look inside the Singularity file with the following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;singularity exec picard_latest.sif /bin/bash -c &amp;quot;ls /&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bin  boot  dev  environment  etc  home  lib  lib32  lib64  libx32  media  mnt  opt  proc  root  run  sbin  singularity  srv  sys  tmp  usr  var&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Attempt 1: Using Find&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;singularity exec picard_latest.sif find / -name &amp;quot;picard.jar&amp;quot; 2&amp;gt;/dev/null&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, this gave no results.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Attempt 2: Going Down The Opt Path&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;singularity exec picard_latest.sif /bin/bash -c &amp;quot;ls /opt/&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This led to a file called &lt;code&gt;java&lt;/code&gt; and that had only one file in it called &lt;code&gt;openjdk&lt;/code&gt; so that seemed to be where java was stored.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Attempt 3: Looking In Boot&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;singularity exec picard_latest.sif /bin/bash -c &amp;quot;ls /boot/&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This yielded no results as it seems to be an empty file.&lt;/p&gt;&#xA;&lt;p&gt;I have made other attempts but I cannot seem to establish the path I should use. I cannot change the java directly as it is preset on my cluster.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know what the path would be for the latest version of Picard or could point me to how to look for this as I cannot seem to find this?&lt;/p&gt;&#xA;" OwnerUserId="17244" LastEditorUserId="17244" LastEditDate="2023-05-26T20:59:21.980" LastActivityDate="2023-05-26T20:59:21.980" Title="Singularity &amp; Picard" Tags="|picard|singularity|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20879" PostTypeId="1" CreationDate="2023-04-13T23:58:06.683" Score="5" ViewCount="851" Body="&lt;p&gt;I am attempting to integrate different bulk RNA-Seq datasets. While this is not ideal, I'm trying to reduce the technical variability in these datasets by using data generated by similar protocols (for example, all of them have been sequenced with a Poly-A enrichment strategy and Illumina short read sequencing). I am aware the results I obtain from this analysis will be very limited, but still want to pursue the analysis. I'm looking for advice on two aspects of the analysis, but first some more info about the data:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;There are 5 bulk RNA-Seq datasets and 4 biological groups. Each datasets only comprises one biological group. Only two biological groups have two datasets, and such datasets overlap in PCA.&lt;/li&gt;&#xA;&lt;li&gt;In Principal Component Analysis (PCA) and the first PC (~26% of variation) is very correlated with the biological variable (cell differentiation), suggesting there is a biological signal among all the noise/technical bias.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Despite these preliminary results I'm not sure I'm accounting correctly for some aspects: The datasets have been sequenced with different sequencing coverage (in fact, different numbers and length of reads). Some are single read and some paired-end. My questions are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;For PCA, I'm normalizing with variance stabilizing transformation (VST). How should I account for the differences in read numbers and type between datasets?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For differential expression (DE) analysis I'm using DESeq2. How should I account for the differences in read numbers and type between datasets? There is no batch variable, as all datasets come from different sources (batches, laboratories)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I'm happy to provide more information if necessary. Thanks&lt;/p&gt;&#xA;" OwnerUserId="16896" LastActivityDate="2023-04-19T12:19:09.470" Title="Integrating bulk RNA-Seq data with different sequencing depths and from different sources" Tags="|rna-seq|rna|quality-control|data-preprocessing|public-dataset|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20880" PostTypeId="2" ParentId="20878" CreationDate="2023-04-14T00:10:14.167" Score="2" Body="&lt;p&gt;Assuming you've pulled the latest &lt;a href=&quot;https://hub.docker.com/r/broadinstitute/picard&quot; rel=&quot;nofollow noreferrer&quot;&gt;broadinstitute/picard&lt;/a&gt; container, your first attempt should have produced the expected result:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ singularity exec picard_latest.sif find / -name &amp;quot;picard.jar&amp;quot; 2&amp;gt;/dev/null&#xA;/usr/picard/picard.jar&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And then to run Picard and display the usage, you would use:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ singularity exec picard_latest.sif java -jar /usr/picard/picard.jar -h&#xA;USAGE: PicardCommandLine &amp;lt;program name&amp;gt; [-h]&#xA;&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that there's now a newer &lt;a href=&quot;https://github.com/broadinstitute/picard/releases&quot; rel=&quot;nofollow noreferrer&quot;&gt;release&lt;/a&gt; of Picard, which uses a much newer version of Java. There's no &lt;a href=&quot;https://hub.docker.com/r/broadinstitute/picard/tags&quot; rel=&quot;nofollow noreferrer&quot;&gt;tag&lt;/a&gt; for this on Docker Hub (yet). So if you wanted the 'latest', you could try using the quay.io biocontainer:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ singularity pull docker://quay.io/biocontainers/picard:3.0.0--hdfd78af_1&#xA;$&lt;/span&gt; singularity exec picard_3.0.0--hdfd78af_1.sif picard -h&#xA;USAGE: PicardCommandLine &amp;lt;program name&amp;gt; [-h]&#xA;&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that the Bioconda build system also builds Docker containers and these are uploaded to &lt;a href=&quot;https://quay.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;quay.io&lt;/a&gt; as part of the &lt;a href=&quot;https://quay.io/organization/biocontainers&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biocontainers project&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2023-04-14T00:13:28.940" LastActivityDate="2023-04-14T00:13:28.940" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20881" PostTypeId="2" ParentId="20879" CreationDate="2023-04-14T00:58:22.923" Score="2" Body="&lt;p&gt;Per-sample coverage / count differences are expected in all differential expression analysis, and correction for that is built into DESeq2. You shouldn't need to do any additional correction with respect to proportional counts when looking at a PCA derived from the VST.&lt;/p&gt;&#xA;&lt;p&gt;For accounting for different sources / types / etc, it's a good idea to add that information into the metadata, and test for differential expression with and without accounting for those variables to see if anything changes. This is recommended in the &lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DESeq2 manual&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If there is unwanted variation present in the data (e.g. batch effects) it is always recommend to correct for this, which can be accommodated in DESeq2 by including in the design any known batch variables or by using functions/packages such as svaseq in sva (Leek 2014) or the RUV functions in RUVSeq (Risso et al. 2014) to estimate variables that capture the unwanted variation. In addition, the ashr developers have a specific method for accounting for unwanted variation in combination with ashr (Gerard and Stephens 2017).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;See the multi-factor design section of the manual for more information (which has a &amp;quot;single-read&amp;quot; vs &amp;quot;paired-end&amp;quot; example):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#multi-factor-designs&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#multi-factor-designs&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If after accounting for the different variables there is not enough information to properly separate groups, DESeq should tell you. Ideally you should aim for six replicates of each state of each independent variable (easier to do when there are interacting variables), but it's quite rare to see that in differential expression analysis.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-04-16T00:54:40.487" LastActivityDate="2023-04-16T00:54:40.487" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20882" PostTypeId="1" CreationDate="2023-04-14T09:48:29.020" Score="0" ViewCount="71" Body="&lt;p&gt;After a long time not running EDirect, I got an error when I tried using it. It said,&amp;quot; curl: schannel: next InitializationSecurityContext failed: Unknown error (0×80090367) - There is no common application protocol between the client and server...&#xA;I tried using WiFi and hotspot, but it did not work.&#xA;Additional info:&#xA;1- I ran EDirect on Cygwin64 Terminal on my personal computer with Windows 11 operating system.&#xA;2- I first ran 'export PATH=&lt;span class=&quot;math-container&quot;&gt;${PATH}:$&lt;/span&gt;{HOME}/edirect'. After that, I tried 'esearch -db pubmed -query &amp;quot;stroke AND english [FILT] AND has abstract [FILT] NOT review [PTYP]&amp;quot; | efilter -mindate 2000 | efetch -format xml | xtract -pattern PubmedArticle -element PMID AbstractText &amp;gt; dm_cs.txt'&lt;/p&gt;&#xA;&lt;p&gt;I used to ran it successfully on the same PC over WiFi or hotspot three month ago.&lt;/p&gt;&#xA;" OwnerUserId="16414" LastEditorUserId="16414" LastEditDate="2023-04-19T02:05:10.687" LastActivityDate="2023-04-19T02:05:10.687" Title="No common application protocol between the client and server? An error when using Entrez Direct" Tags="|curl|edirect|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20883" PostTypeId="2" ParentId="20882" CreationDate="2023-04-14T11:59:18.803" Score="1" Body="&lt;p&gt;This happens.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;One possibility is it's simply NCBI blocking the request to avoid being overrun by e.g. bots. This occurs if you've been running the script repeatedly within a single day. It will be fine tomorrow in this scenario.&lt;/li&gt;&#xA;&lt;li&gt;The second possibility is you've not supplied an email address.&lt;/li&gt;&#xA;&lt;li&gt;The final possibility is if you're running a VPN. NCBI has taken a strong dislike to VPN's &lt;em&gt;per se&lt;/em&gt;, I've never figured out its algorithms - it seems to permit a fixed number of requests then pointedly refuses. Just turn off the VPN.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;If its point 2 then understanding how you've coded this would be useful to allow an email address to be sent. Biopython has a function for this. I just use Biopython BTW.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-04-14T15:05:57.593" LastActivityDate="2023-04-14T15:05:57.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20885" PostTypeId="2" ParentId="20876" CreationDate="2023-04-16T15:06:05.270" Score="2" Body="&lt;p&gt;De novo means that the software infers presence of motifs based on the provided sets of DNA sequences, most commonly peaks from ChIP-seq and similar experiments.&#xA;So essentially it stacks the sequences on top of each other and checks for presence of conserved nucleotides that form sequences of certain lengths, for example &amp;lt; 10bp.&lt;/p&gt;&#xA;&lt;p&gt;Known means that each peak is checked whether it contains a known motif (so is similar to a given known motif) and then in the end a per-motif statistic is calculated to decide whether enriched or not.&lt;/p&gt;&#xA;&lt;p&gt;If the investigated protein is well-studied then de novo and known is likely similar or identical. De novo does not mean that found motif is novel in terms or unknown or undescribed. It just means that it is derived from the data directly rather than matching known motifs onto the peaks. It's simply two different ways to approach the question. Hope that makes sense.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-04-16T15:06:05.270" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20887" PostTypeId="1" CreationDate="2023-04-17T18:14:18.940" Score="2" ViewCount="257" Body="&lt;p&gt;I am working with PAR-CLIP data where experimental procedure induces T to C mismatches. The other types of mismatches we can consider coming from artifact such as PCR errors. It could also be an existing SNP. For simplicity, let's consider T to C mismatches are experimentally induced and rest of the mismatches are background mismatch rate.&lt;/p&gt;&#xA;&lt;p&gt;From Illumina NGS seq after performing alignment, for each of he &lt;code&gt;read groups&lt;/code&gt; (reads collapsed in a 30-70 base region with minimum 10 read coverage). Up to this point its already done.&lt;/p&gt;&#xA;&lt;p&gt;I want to compare T to C mismatches vs background mismatches. To do so I need to calculate the rate of signal mismatches (T to C) and background mismatches.&lt;/p&gt;&#xA;&lt;p&gt;Using &lt;code&gt;samtools mpileup&lt;/code&gt; I can get total number of all kind of mismatches for a &lt;code&gt;read group&lt;/code&gt;, however, takes some time.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How do I calculate background mismatch rate from this?&lt;/li&gt;&#xA;&lt;li&gt;How do I calculate total mismatches in a bam?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I was thinking about something like&lt;br /&gt;&#xA;for a read group:&#xA;&lt;code&gt;mismatch_rate_group=total_mismatches_group/(total_read_group*total_base_group)&lt;/code&gt;&#xA;for a BAM file:&#xA;&lt;code&gt;mismatch_rate_global=total_mismatches_bam/(total_reads_aligned_bam*genome_size_base)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;What important factor I might missing in my approach?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I  am not using any variant caller (e.g. bcftools). Initially I tried with samtools mpileup and then used a shell script to figure out the mismatches. But I am not sure is this is the right thing to do. Or even there are any easier methods. If one read has a mutation, I consider that a mismatch and I presume it to be artifact and/or PCR errors.&lt;/p&gt;&#xA;" OwnerUserId="14423" LastEditorUserId="73" LastEditDate="2023-04-18T19:13:32.503" LastActivityDate="2024-01-13T22:02:45.310" Title="calculate mismatch frequency/rate from a BAM file" Tags="|ngs|sam|point-mutation|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20888" PostTypeId="1" AcceptedAnswerId="20901" CreationDate="2023-04-17T19:04:49.413" Score="1" ViewCount="49" Body="&lt;p&gt;I am looking at &lt;a href=&quot;https://github.com/Illumina/manta/tree/master/docs/userGuide&quot; rel=&quot;nofollow noreferrer&quot;&gt;Manta on Github&lt;/a&gt; and I ran a bit of code:&lt;/p&gt;&#xA;&lt;p&gt;The following variables were defined in the code:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;input_dir&lt;/code&gt; is the file path to a folder with less than 10 .bams as per the limitations of Manta&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;output_dir&lt;/code&gt; is the path to the output&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;MANTA_INSTALL_PATH&lt;/code&gt; is the path to the Manta installation&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;reference&lt;/code&gt; is the path to the reference fasta&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;    for file in &lt;span class=&quot;math-container&quot;&gt;${input_dir}/*.bam&#xA;do&#xA;  # Create subdirectory named after bam file&#xA;  sub_dir=$&lt;/span&gt;{output_dir}/&lt;span class=&quot;math-container&quot;&gt;$(basename $&lt;/span&gt;{file} .bam)&#xA;      mkdir -p ${sub_dir}&#xA;    &#xA;      # Use reference fasta path specified by user&#xA;      &lt;span class=&quot;math-container&quot;&gt;${MANTA_INSTALL_PATH}/bin/configManta.py \&#xA;    --bam $&lt;/span&gt;file \&#xA;        --referenceFasta &lt;span class=&quot;math-container&quot;&gt;${reference} \&#xA;    --runDir $&lt;/span&gt;{sub_dir} \&#xA;        --exome&#xA;    done&#xA;&#xA;# Now run each `runWorkflow.py`&#xA;&#xA;find &amp;quot;$output_dir&amp;quot; -name &amp;quot;runWorkflow.py&amp;quot; -exec {} \;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The run worked and each sample had a folder with the following files:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;results  runWorkflow.py  runWorkflow.py.config.pickle  workflow.error.log.txt  workflow.exitcode.txt  workflow.warning.log.txt  workspace&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Each &lt;code&gt;workflow.error.log.txt&lt;/code&gt; had nothing in it implying the the run had been successful. Each &lt;code&gt;results&lt;/code&gt; folder had 3 subfolders&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;evidence  stats  variants&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;stats&lt;/code&gt; and &lt;code&gt;variants&lt;/code&gt; folder each had outputs but the &lt;code&gt;evidence&lt;/code&gt; folder had nothing. Is this normal? Should it have something in this folder? Why is this folder there in Manta if nothing is in it?&lt;/p&gt;&#xA;" OwnerUserId="17244" LastActivityDate="2023-04-19T09:02:03.493" Title="Manta Evidence Folder Is Empty" Tags="|git|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20890" PostTypeId="1" CreationDate="2023-04-18T02:55:33.107" Score="3" ViewCount="2208" Body="&lt;p&gt;Can anyone suggest a strategy for speeding up VCF merging?&lt;/p&gt;&#xA;&lt;p&gt;I have ~44,000 single-sample VCFs that I am trying to merge into a multi-sample VCF with &lt;code&gt;bcftools merge&lt;/code&gt;, but the job keeps timing out on the cluster (just timed out after 4 days). I could obviously try running it for a couple of weeks, but I was hoping there is a better strategy someone can suggest?&lt;/p&gt;&#xA;&lt;p&gt;The exact command I am running is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools merge --no-version -m all -i 'DP:avg,RO:avg,AO:avg' -F + -o merged.vcf --file-list merge.fofn&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Happy to try any other tool that will give the same merge results.&lt;/p&gt;&#xA;" OwnerUserId="149" LastActivityDate="2023-04-19T23:32:39.223" Title="Strategy for merging many VCFs" Tags="|vcf|bcftools|merge|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20891" PostTypeId="2" ParentId="20890" CreationDate="2023-04-18T08:04:23.903" Score="4" Body="&lt;p&gt;We created this over the pandemic when we needed to merge loads of VCFs which all have the same header and first few columns (ie same variants)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/iqbal-lab-org/ivcfmerge&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/iqbal-lab-org/ivcfmerge&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;That page gives detailed instructions but i'll paste a couple of examples&#xA;of how you can use it&lt;/p&gt;&#xA;&lt;p&gt;== In Python&#xA;=4.1.1 If the number of input files is small (can be opened all at once)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from contextlib import ExitStack&#xA;from ivcfmerge import ivcfmerge&#xA;&#xA;filenames = [...]    # List/iterator of relative/absolute paths to input files&#xA;output_path = '...'  # Where to write the merged VCF to&#xA;&#xA;with ExitStack() as stack:&#xA;    files = map(lambda fname: stack.enter_context(open(fname)), filenames)&#xA;    with open(output_path) as outfile:&#xA;        ivcfmerge(files, outfile)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;=4.1.2 If the number of input files is big (cannot be opened all at once)&#xA;from ivcfmerge import ivcfmerge_batch&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;filenames = [...]    # List/iterator of relative/absolute paths to input files&#xA;output_path = '...'  # Where to write the merged VCF to&#xA;batch_size = 1000    # How many files to open and merge at once&#xA;&#xA;ivcfmerge_batch(filenames, output_path, batch_size)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Pretty sure we initially did batched bcftools merge, I can't honestly remember why we moved to this. Thankfully not having to do these merges very often&lt;/p&gt;&#xA;" OwnerUserId="17341" LastEditorUserId="17341" LastEditDate="2023-04-18T08:16:02.730" LastActivityDate="2023-04-18T08:16:02.730" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20893" PostTypeId="1" CreationDate="2023-04-18T11:30:17.053" Score="2" ViewCount="53" Body="&lt;p&gt;I am looking for instructions on creating a custom-made kraken2 catalogue from the sequences of my choice and the corresponding phylogenetic tree in newick notation.&lt;/p&gt;&#xA;&lt;p&gt;I will appreciate an input on whether/how it can be done. &lt;a href=&quot;https://github.com/DerrickWood/kraken2/wiki/Manual#custom-databases&quot; rel=&quot;nofollow noreferrer&quot;&gt;The prescription provided in the kraken2 manual&lt;/a&gt; seems to assume that sequences and phylogeny originate from the NCBI. &lt;a href=&quot;https://github.com/leylabmpi/Struo2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Struo2&lt;/a&gt; extends this to GTDB sequences. However, in my case the sequences are collections of concatenated genes (i.e., not full genomes), provided by a third party.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Remark:&lt;/strong&gt; (in response to comments) I don't &lt;em&gt;need&lt;/em&gt; to work with newick format - I have the tree in newick format. So converting it to &lt;code&gt;dmp&lt;/code&gt; might be a part of the answer.&lt;/p&gt;&#xA;" OwnerUserId="1813" LastEditorUserId="1813" LastEditDate="2023-04-19T05:34:08.477" LastActivityDate="2023-04-19T05:34:08.477" Title="Custom-made kraken2 catalogue" Tags="|gene|k-mer|ncbi|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20895" PostTypeId="2" ParentId="17534" CreationDate="2023-04-18T15:52:55.760" Score="1" Body="&lt;p&gt;Yes! Multiome seems to have cells with much higher mitochondrial gene content. I don't understand the biology behind this. But in using a cut off of 5% (i.e. remove cells with &amp;gt;5% MT genes) I lose over 80% of cells in multiome data. Also, here is the Signac tutorial where they use a 20% cut off for multiome data - &lt;a href=&quot;https://satijalab.org/seurat/articles/weighted_nearest_neighbor_analysis.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/articles/weighted_nearest_neighbor_analysis.html&lt;/a&gt; (&amp;quot;WNN analysis of 10x Multiome, RNA + ATAC&amp;quot;)&lt;/p&gt;&#xA;" OwnerUserId="17344" LastActivityDate="2023-04-18T15:52:55.760" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20897" PostTypeId="2" ParentId="20890" CreationDate="2023-04-18T17:55:31.967" Score="1" Body="&lt;p&gt;Perform a tree based merge of the files.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Sort the files into groups that can easily be merged in a measurable amount of time.&lt;/li&gt;&#xA;&lt;li&gt;Merge each group into a separate Variant Call Format (VCF) file&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Has the benefit that files can also be grouped by type / source / run&lt;/li&gt;&#xA;&lt;li&gt;Further benefit that small merges can be checked to ensure you like the output before proceeding to the next stage.&lt;/li&gt;&#xA;&lt;li&gt;Further benefit that there are also subgroup files available to be distributed or shared separately.&lt;/li&gt;&#xA;&lt;li&gt;Downside, extra storage space until you have a final product.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;Merge subfiles upward in stages until you arrive at a complete 44,000 entry file&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If you're happy with the earlier test results, process can be automated with relatively simple iterative shell scripts.&lt;/li&gt;&#xA;&lt;li&gt;Possibly faster, as the bulk merge needs to load every single file into memory, and you may be hitting a capacity bottleneck.&lt;/li&gt;&#xA;&lt;li&gt;Worked in Fluid Dynamics for years, and often ran into a similar issue.  Could run many small cases representing a large case faster than running the large case, because large case would overwhelm memory and transfer capacity.&lt;/li&gt;&#xA;&lt;li&gt;In some clusters you may also be causing nodes to silently crash, which then stalls the entire process without a visible notification.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17346" LastActivityDate="2023-04-18T17:55:31.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20898" PostTypeId="2" ParentId="20887" CreationDate="2023-04-18T19:31:21.230" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;I am not using any variant caller&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Use a variant caller. &lt;code&gt;bcftools&lt;/code&gt;, at least, will output variant frequency per variant:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools mpileup -ugf ref.fa sample.bam | bcftools call -mv &amp;gt; output.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As an alternative approach, I wrote &lt;a href=&quot;https://gitlab.com/gringer/bioinfscripts/-/blob/master/readstomper.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;some code&lt;/a&gt; to parse the mpileup output and extract counts for different alleles; this output may be easier for you to parse and calculate your desired frequencies:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# readstomper.pl -- generate base-packed proportion statistics for&#xA;#   output from 'samtools mpileup'&#xA;#&#xA;# example use:&#xA;# samtools mpileup -B -Q 0 -f circ-Nb-ec3-mtDNA.fasta LAST_OL_132394_vs_mtDNA.bam | \&#xA;#   /bioinf/scripts/readstomper.pl &amp;gt; LAST_OLmpileupProp_132394_vs_mtDNA.txt.gz&#xA;#&#xA;# example output:&#xA;# Assembly,Position,Coverage,ref,cR,pR,A,C,G,T,d,i,InsMode&#xA;# Nb_mtDNA,11456,86,T,79.00,0.92,0.0000,0.0465,0.0000,0.0000,0.03,0.01,A;1.00&#xA;# Nb_mtDNA,11457,86,A,81.00,0.94,0.0000,0.0000,0.0233,0.0000,0.03,0.02,TA;0.50&#xA;# Nb_mtDNA,11458,86,G,79.00,0.92,0.0349,0.0233,0.0000,0.0116,0.01,0.00&#xA;# Nb_mtDNA,11459,86,A,78.00,0.92,0.0000,0.0000,0.0235,0.0471,0.01,0.00&#xA;# Nb_mtDNA,11460,86,T,5.00,0.06,0.0233,0.8140,0.0233,0.0000,0.08,0.00&#xA;# Nb_mtDNA,11461,86,G,57.00,0.66,0.1512,0.1163,0.0000,0.0465,0.02,0.01,AATAACAACACGTAACCGA;1.00&#xA;# Nb_mtDNA,11462,86,G,78.00,0.91,0.0349,0.0349,0.0000,0.0116,0.01,0.00&#xA;# Nb_mtDNA,11463,86,G,66.00,0.77,0.0465,0.0698,0.0000,0.0116,0.10,0.01,GCAATA;1.00&#xA;# Nb_mtDNA,11464,86,T,74.00,0.86,0.0000,0.0814,0.0233,0.0000,0.03,0.00&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-04-18T19:31:21.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20899" PostTypeId="2" ParentId="14799" CreationDate="2023-04-19T01:46:42.810" Score="0" Body="&lt;p&gt;from cmapPy.pandasGEXpress.parse import parse&lt;/p&gt;&#xA;&lt;p&gt;from cmapPy.pandasGEXpress.write import write&lt;/p&gt;&#xA;&lt;p&gt;parse(&amp;quot;C:/Users/User/Desktop/gctx/CAS9_BASELINE_LEVEL3_BING__n10721x10174.gctx&amp;quot;)&lt;/p&gt;&#xA;" OwnerUserId="17348" LastEditorUserId="17348" LastEditDate="2023-04-19T01:47:13.393" LastActivityDate="2023-04-19T01:47:13.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20900" PostTypeId="2" ParentId="20890" CreationDate="2023-04-19T01:52:02.690" Score="3" Body="&lt;p&gt;I ended up using an iterative approach (inspired by &lt;a href=&quot;https://shicheng-guo.github.io/bioinformatics/1923/02/28/bcftools-merge&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt;), which took 1.5 hours - so a massive speed up (and that was just the sequential version).&lt;/p&gt;&#xA;&lt;p&gt;Note: &lt;a href=&quot;https://github.com/sharkdp/fd&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;fd&lt;/code&gt;&lt;/a&gt; is analogous to &lt;code&gt;find&lt;/code&gt;, but way nicer to use.&lt;/p&gt;&#xA;&lt;p&gt;First, merge subsets of &lt;em&gt;n&lt;/em&gt; VCFs&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# size of the subsets&#xA;n=200&#xA;# create file of filenames for each subset&#xA;fd -e vcf . path/to/vcfs/ | split -l $n --additional-suffix '.fofn' - subset_vcfs&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;-e vcf&lt;/code&gt; tells &lt;code&gt;fd&lt;/code&gt; to find all files with a &lt;code&gt;.vcf&lt;/code&gt; extension, you can obviously change this to &lt;code&gt;.bcf&lt;/code&gt;/&lt;code&gt;.vcf.gz&lt;/code&gt; etc. if necessary. &lt;code&gt;split&lt;/code&gt; splits the list of all VCFs files into subsets of size &lt;code&gt;-l&lt;/code&gt; (200) and creates the file of filenames with the prefix &lt;code&gt;subset_vcfs&lt;/code&gt;. It will add a random suffix to this - e.g. &lt;code&gt;subset_vcfsav&lt;/code&gt;. Plus the additional suffix &lt;code&gt;.fofn&lt;/code&gt;. See the &lt;a href=&quot;https://www.man7.org/linux/man-pages/man1/split.1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;split&lt;/code&gt; manual&lt;/a&gt; for all options.&lt;/p&gt;&#xA;&lt;p&gt;Now iterate through these subsets and merge the VCFs listed in them.&lt;/p&gt;&#xA;&lt;h2&gt;Subset merge option 1 - sequential&lt;/h2&gt;&#xA;&lt;p&gt;This just merges the VCFs one at a time&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;for sub in *.fofn;&#xA;do&#xA;    echo &amp;quot;Merging subset ${sub}...&amp;quot;&#xA;    out_vcf=&amp;quot;merge.${sub}.bcf&amp;quot;&#xA;    bcftools merge --file-list &amp;quot;$sub&amp;quot; -o &amp;quot;$out_vcf&amp;quot;&#xA;    bcftools index &amp;quot;$out_vcf&amp;quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Subset merge option 2 - parallel&lt;/h2&gt;&#xA;&lt;p&gt;You can easily parallelise the previous step using &lt;code&gt;fd&lt;/code&gt; on the subset files. See &lt;a href=&quot;https://mbh.sh/2020-06-22-cheap-parallelisation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this post&lt;/a&gt; for more details. Or basic &lt;a href=&quot;https://unix.stackexchange.com/questions/103920/parallelize-a-bash-for-loop&quot;&gt;bash &lt;code&gt;for&lt;/code&gt; loop parallelisation&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Create a script (&lt;code&gt;merge_subset.sh&lt;/code&gt;) of what we want to run on each subset&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;#!/usr/bin/env bash&#xA;sub=&amp;quot;$1&amp;quot;&#xA;echo &amp;quot;Merging subset ${sub}...&amp;quot;&#xA;out_vcf=&amp;quot;merge.${sub}.bcf&amp;quot;&#xA;bcftools merge --file-list &amp;quot;$sub&amp;quot; -o &amp;quot;$out_vcf&amp;quot;&#xA;bcftools index &amp;quot;$out_vcf&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now we run this script with each subset file of filenames using &lt;code&gt;fd&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# number of processes to use. remove this option to use all available&#xA;jobs=4&#xA;fd -j $jobs -e fofn -x bash merge_subset.sh '{}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Final merge&lt;/h2&gt;&#xA;&lt;p&gt;Now we merge all of these VCFs into our final VCF&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ls merge.*.bcf &amp;gt; merged.fofn&#xA;bcftools merge -o merged.bcf --file-list merged.fofn&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="149" LastEditorUserId="149" LastEditDate="2023-04-19T23:32:39.223" LastActivityDate="2023-04-19T23:32:39.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20901" PostTypeId="2" ParentId="20888" CreationDate="2023-04-19T09:02:03.493" Score="1" Body="&lt;p&gt;The documentation on the GitHub page you linked to &lt;a href=&quot;https://github.com/Illumina/manta/tree/master/docs/userGuide#generating-evidence-bams&quot; rel=&quot;nofollow noreferrer&quot;&gt;has this to say&lt;/a&gt; about the &lt;code&gt;evidence&lt;/code&gt; folder:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Generating evidence bams&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Using the &lt;code&gt;--generateEvidenceBam&lt;/code&gt; option, Manta can be configured to&#xA;generate bam files of evidence reads for SVs listed in the candidate&#xA;vcf file.&lt;/p&gt;&#xA;&lt;p&gt;It is recommended to use this option together with the --region&#xA;option, so that the analysis is limited to relatively small genomic&#xA;regions for debugging purposes.&lt;/p&gt;&#xA;&lt;p&gt;The evidence bam files are provided in&#xA;&lt;code&gt;${MANTA_ANALYSIS_PATH}/results/evidence&lt;/code&gt;, with a naming format&#xA;&lt;code&gt;evidence_*.*.bam&lt;/code&gt;. There is one such file for each input bam of the&#xA;analysis, containing evidence reads of the candidate SVs identified&#xA;from that input bam. Each read in an evidence bam keeps all&#xA;information from the original bam, and it contains also a customized&#xA;tag in the format:&#xA;&lt;code&gt;ZM:Z:${MANTA_SV_ID_1}|${EVIDENCE_TYPE},${MANTA_SV_ID_2}|${EVIDENCE_TYPE}&lt;/code&gt;.&#xA;For example, &lt;code&gt;ZM:Z:MantaINV:5:0:1:0:0:0|PR|SRM,MantaDEL:5:1:2:0:0:0|SR&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;One read can have more than one of the three evidence types: PR for&#xA;paired reads, SR for split reads, and SRM for split read mates. One&#xA;read can be evidence for multiple SVs, which are separated by commas&#xA;in the tag. Notice that the number of evidence reads for a particular&#xA;SV in the evidence bam files could be more than the evidence counts&#xA;(PR and SR) in the final vcf files. This is because more stringent&#xA;criteria are applied for generating evidence counts in the final vcf&#xA;files.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So it looks to me like you need the &lt;code&gt;--generateEvidenceBam&lt;/code&gt; option to generate evidence, but that the directory &lt;code&gt;evidence&lt;/code&gt; is created regardless of whether you use the option. Try running the analysis again, but using &lt;code&gt;--generateEvidenceBam&lt;/code&gt; this time.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-04-19T09:02:03.493" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20902" PostTypeId="2" ParentId="20879" CreationDate="2023-04-19T12:19:09.470" Score="0" Body="&lt;p&gt;In case of different sequencing depth one performs &lt;em&gt;rarefaction&lt;/em&gt;, to make the data comparable.&lt;/p&gt;&#xA;&lt;p&gt;The standard approach is &lt;em&gt;rarefaction by size&lt;/em&gt;, when one takes the number of reads in the smallest sample, and samples without replacement the same number of reads from the other samples. Potentially, one could carry out upsizing to a greater number of reads (e.g., if the smallest sample is an outlier in terms of the number of reads, and downsizing may result in losing too much information.) I suggest reading the &lt;a href=&quot;https://cran.r-project.org/web/packages/momr/momr.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;tutorial for MOMR package&lt;/a&gt; for more detailed description (the R code is also available.)&lt;/p&gt;&#xA;&lt;p&gt;There are more sophisticated methods of rarefaction, such as &lt;em&gt;rarefaction by coverage&lt;/em&gt; - these are of significant scientific interests, but tricky to use and not very widespread. See, e.g., &lt;a href=&quot;https://esajournals.onlinelibrary.wiley.com/doi/10.1890/13-0133.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt; (it deals with sampling species, but generalization to reads is straightforward... though reads are not species.)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Remark:&lt;/strong&gt; Of course many good packages (like those mentioned by @gringer) might already perform rarefaction by default - but it is always a good idea to check the documentation.&lt;/p&gt;&#xA;" OwnerUserId="1813" LastActivityDate="2023-04-19T12:19:09.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20903" PostTypeId="2" ParentId="20631" CreationDate="2023-04-19T14:59:02.260" Score="1" Body="&lt;p&gt;Correlations like this between ChIP-seq datasets is common.&lt;/p&gt;&#xA;&lt;p&gt;If you're calculating the correlation between BAM files across the entire genome, those are largely going to be the same (hence the need for input controls in ChIP-seq for performing the peak calling in the first place).&#xA;Similarly, many ChIP-seq experiments only have 30 - 60% of reads within their peaks (FRiP), which means that 40 - 70% of reads don't fall within peak regions.&#xA;This is going to heavily skew any correlation calculations that don't exclusively focus on the peaks themselves.&lt;/p&gt;&#xA;&lt;p&gt;To more clearly compare ChIP-seq data against each other, global metrics like FRiP can help you diagnose measurement/sequencing issues.&#xA;Looking at the correlation across ChIP-seq signal within peaks may be a more informative calculation than looking at all reads across the genome.&#xA;Similarly, principal component analysis on signal within peaks can be helpful here.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/DiffBind.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;DiffBind&lt;/a&gt; has some helper functions to perform these types of calculations that you can look at.&lt;/p&gt;&#xA;" OwnerUserId="240" LastActivityDate="2023-04-19T14:59:02.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20904" PostTypeId="1" AcceptedAnswerId="20912" CreationDate="2023-04-19T17:47:03.383" Score="1" ViewCount="60" Body="&lt;p&gt;Does &lt;a href=&quot;https://github.com/brentp/bwa-meth&quot; rel=&quot;nofollow noreferrer&quot;&gt;bwameth&lt;/a&gt; accept &lt;code&gt;bam&lt;/code&gt; (unaligned) type as input or I will have to convert unaligned bam to fastq (&lt;code&gt;samtools fastq bam&lt;/code&gt;) to use it as input?&lt;/p&gt;&#xA;" OwnerUserId="2970" LastActivityDate="2023-04-21T08:56:48.417" Title="Can bwameth accept unaligned Bam?" Tags="|sam|bwa|methylation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20905" PostTypeId="2" ParentId="15720" CreationDate="2023-04-20T01:40:57.027" Score="0" Body="&lt;p&gt;It is possible that some of your samples have too few cells, so it is recommended to check the data and remove samples with low cell counts.&lt;/p&gt;&#xA;" OwnerUserId="17352" LastActivityDate="2023-04-20T01:40:57.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20906" PostTypeId="1" AcceptedAnswerId="20908" CreationDate="2023-04-20T05:23:08.283" Score="5" ViewCount="521" Body="&lt;p&gt;What does the IGV refseq genes with three different thicknesses of lines mean?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Q6IBK.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Q6IBK.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17353" LastEditorUserId="12518" LastEditDate="2023-05-10T15:28:18.563" LastActivityDate="2023-05-12T08:31:47.967" Title="What does the IGV refseq genes with three different thicknesses of lines mean?" Tags="|igv|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20907" PostTypeId="1" AcceptedAnswerId="20909" CreationDate="2023-04-20T09:44:28.133" Score="1" ViewCount="168" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9561238&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am trying to download and analyze a miRNA expression dataset from NCBI GEO &lt;code&gt;(GSE25631)&lt;/code&gt;. I specifically want non-normalized data to perform normalization and my own set of other analyses later on. Accordingly, I downloaded the &lt;code&gt;Series Matrix File&lt;/code&gt; (to match the Sample IDs with their phenotype) and the non-normalized data &lt;code&gt;GSE25631_non-normalized_data.txt.gz&lt;/code&gt; programmatically using the &lt;code&gt;GEOquery&lt;/code&gt; package in R. To normalize the data I'm using the &lt;code&gt;DESeq2&lt;/code&gt; package. When I run the &lt;code&gt;DESeqDataSetFromMatrix&lt;/code&gt; function I get this error:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error in DESeqDataSet(se, design = design, ignoreRank) :&#xA;some values in assay are negative&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I went back to the non-normalized data and observed that there are a lot of negative values. I need suggestions on how to deal with this issue. Should I replace all such values with NA? I also observed that every probe has a detection p-value associated with it. Should I also filter out the expression values which have a p-value higher than 0.05?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(raw_counts)&#xA;        gbm-0001 Detection Pval gbm-0011 Detection Pval gbm-0013 Detection Pval gbm-0024 Detection Pval gbm-0058 Detection Pval&#xA;        ILMN_3167304   27.875      0.3071961 -24.9375      0.6689599    14.25       0.350027   4.3125      0.4723163     8.25      0.3966338&#xA;        ILMN_3168038  211.875       6.42E-05   1.0625      0.4925717    99.25    0.003645935  60.3125      0.1657064    72.25     0.01086376&#xA;        ILMN_3167890 13839.88       3.68E-38 23507.06       3.68E-38 22345.25       3.68E-38 20482.31       3.68E-38 21419.25       3.68E-38&#xA;        ILMN_3167526    5.875      0.4577177 105.0625     0.03279042     8.25      0.4117529 -29.6875       0.683706    11.25      0.3604082&#xA;        ILMN_3167209   82.875     0.06708142 182.0625      0.0007096    60.25     0.05167192   9.3125      0.4403947    72.25     0.01086376&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here's the R script which I used for all the mentioned steps.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(GEOquery)&#xA;library(DESeq2)&#xA;library(tidyverse)&#xA;&#xA;&#xA;data &amp;lt;- getGEO(GEO = &amp;quot;GSE25631&amp;quot;, &#xA;           destdir = &amp;quot;E:\\GSE25631&amp;quot;,&#xA;           GSEMatrix = TRUE,&#xA;           AnnotGPL = FALSE,&#xA;           getGPL = FALSE,&#xA;           parseCharacteristics = TRUE)&#xA;&#xA;clindata &amp;lt;- data$GSE25631_series_matrix.txt.gz@phenoData@data&#xA;&#xA;url = &amp;quot;https://www.ncbi.nlm.nih.gov/geo/download/? &#xA;acc=GSE25631&amp;amp;format=file&amp;amp;file=GSE25631%5Fnon%2Dnormalized%5Fdata%2Etxt%2Egz&amp;quot;&#xA;download.file(url, &amp;quot;E:\\GSE25631\\GSE25631_raw_counts.gz&amp;quot;)&#xA;raw_counts &amp;lt;-read.delim(&amp;quot;E:\\GSE25631\\GSE25631_raw_counts.gz&amp;quot;, &#xA;                     stringsAsFactors = FALSE, sep = &amp;quot;\t&amp;quot;)&#xA;colnames(raw_counts) &amp;lt;- raw_counts[5,]&#xA;raw_counts &amp;lt;- raw_counts[-(1:5),]&#xA;raw_counts &amp;lt;- raw_counts %&amp;gt;% remove_rownames %&amp;gt;% column_to_rownames(var = &amp;quot;ID_REF&amp;quot;)&#xA;raw_counts &amp;lt;- raw_counts[ , c(TRUE, FALSE)]&#xA;&#xA;&#xA;colnames(raw_counts) &amp;lt;- clindata&lt;span class=&quot;math-container&quot;&gt;$title&#xA;rownames(clindata) &amp;lt;- clindata$&lt;/span&gt;title&#xA;colnames(clindata)[colnames(clindata) == &amp;quot;disease state:ch1&amp;quot;] &amp;lt;- &amp;quot;Condition&amp;quot;&#xA;clindata&lt;span class=&quot;math-container&quot;&gt;$Condition[clindata$&lt;/span&gt;Condition == &amp;quot;primary glioblastoma multiform (brain tumor)&amp;quot;] &amp;lt;- &amp;quot;GBM&amp;quot;&#xA;clindata&lt;span class=&quot;math-container&quot;&gt;$Condition[clindata$&lt;/span&gt;Condition == &amp;quot;normal&amp;quot;] &amp;lt;- &amp;quot;Normal&amp;quot;&#xA;clindata&lt;span class=&quot;math-container&quot;&gt;$Condition &amp;lt;- as.factor(clindata$&lt;/span&gt;Condition)&#xA;&#xA;dds &amp;lt;- DESeqDataSetFromMatrix(countData = raw_counts,&#xA;                          colData = clindata,&#xA;                          design = ~ Condition)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14724" LastEditorUserId="73" LastEditDate="2023-04-20T20:39:09.570" LastActivityDate="2023-04-20T20:40:25.893" Title="DESeq2 throwing error while normalizing raw microarray expression data due to presence of negative values" Tags="|deseq2|normalization|microarray|geoquery|geo|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20908" PostTypeId="2" ParentId="20906" CreationDate="2023-04-20T11:06:11.477" Score="5" Body="&lt;p&gt;The thinkness indicates whether something is an intron, a coding exon, or a UTR. The very thin lines are introns, the full thickness boxes are coding exons and the thinner boxes are UTRs.&lt;/p&gt;&#xA;&lt;p&gt;UTRs, untranslated regions, are exons, they are included in the final mRNA, but they don't code for protein. So, by convention, they are shown as thinner exons to give the user a visual indication that although this is still an exonic region, it's a special one.&lt;/p&gt;&#xA;&lt;p&gt;I don't really know where this is documented, it's pretty standard across all genome browsers and bam visualizers I know though so it's a common idiom.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2023-05-12T08:31:47.967" LastActivityDate="2023-05-12T08:31:47.967" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20909" PostTypeId="2" ParentId="20907" CreationDate="2023-04-20T11:07:26.637" Score="2" Body="&lt;p&gt;Answered at biostars: &lt;a href=&quot;https://www.biostars.org/p/9561238/#9561240&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/9561238/#9561240&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;DESeq2 is meant for digital counts (RNA-seq and similar assays). Arrays are not supported, nor is the implemented method meaningful for array intensity values.&lt;/p&gt;&#xA;&lt;p&gt;Guides on how to handle these data are probably in here:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="16676" LastEditorUserId="73" LastEditDate="2023-04-20T20:40:25.893" LastActivityDate="2023-04-20T20:40:25.893" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20910" PostTypeId="1" AcceptedAnswerId="20917" CreationDate="2023-04-21T03:27:13.673" Score="2" ViewCount="175" Body="&lt;p&gt;I want to search GenBank entries by gene product and taxon id.&lt;/p&gt;&#xA;&lt;p&gt;Taking the 16s rRNA as an example for a gene and Piscirickettsia salmonis (taxon:1238) for a taxon:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/219846389&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/219846389&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How do I retrieve all GenBank entries that have the following properties &lt;strong&gt;/product&lt;/strong&gt;=&amp;quot;16S ribosomal RNA&amp;quot; and &lt;strong&gt;/db_xref&lt;/strong&gt;=&amp;quot;taxon:1238&amp;quot;?&lt;/p&gt;&#xA;&lt;p&gt;Update: It would be appreciated if you also showed me how to retrieve GenBank entries like &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/KC473400.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;, which don't have an exact match to the &amp;quot;16S ribosomal RNA&amp;quot; product.&lt;/p&gt;&#xA;" OwnerUserId="17357" LastEditorUserId="73" LastEditDate="2023-04-23T06:13:15.070" LastActivityDate="2023-04-23T06:13:15.070" Title="How do i search by these gene properties in NCBI's GenBank?" Tags="|genbank|sequence|16rrna|ncbi|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20911" PostTypeId="2" ParentId="20910" CreationDate="2023-04-21T06:54:53.313" Score="-1" Body="&lt;p&gt;To specifically answer the question the search that is used is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;((1000:1500[Sequence Length]) AND Piscirickettsia salmonis[Organism]) AND 16S&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That is place &lt;a href=&quot;https://www.ncbi.nlm.nih.gov&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; and select 'nucleotide'.&lt;/p&gt;&#xA;&lt;p&gt;Genbank uses taxon id internally, it's a database reference not a biological code, and can't be used as input - its completely unique to Genbank. Anyway the search results in 276 records, and are shown &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/?term=((1000%3A1500%5BSequence%20Length%5D)%20AND%20Piscirickettsia%20salmonis%5BOrganism%5D)%20AND%2016s&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, this is near full length 16S thats cool.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Clarification&lt;/strong&gt; &lt;code&gt;[taxon id]&lt;/code&gt; is not a field in Genbank, it's a database code, the tag is &lt;code&gt;[organism]&lt;/code&gt;.  Genbank knows what 16S is - its the most famous genetic marker in microbiology - so there is no need for a &lt;code&gt;[tag]&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;At the base of the page there is a &amp;quot;Send to:&amp;quot; option (right hand corner). Click and then select &amp;quot;File&amp;quot;. The 276 sequences will then download.&lt;/p&gt;&#xA;&lt;p&gt;If &lt;code&gt;[Sequence length]&lt;/code&gt; is removed this result 3820 sequences &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/?term=(Piscirickettsia%20salmonis%5BOrganism%5D)%20AND%2016s&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. There are some 16S-23S sequences on the database result in a 2,198 bp length, so extending the sequence length will trap these by &lt;code&gt;1000:2200[Sequence Length]&lt;/code&gt;. There are 431 bp sequences - which are not cool at all, probably a lot.&lt;/p&gt;&#xA;&lt;p&gt;The approach used below identifies nearly 1000 is complicated it traps everything of relevance which is close to full length and cuts out the 16S from  2,198 bp length and beyond. The 431 bp partial 16S sequences have been removed. Thus the numbers below are the idealised data set, so its about 1000 sequences and the tree will guide you into major groups - which is probably what is wanted, i.e. to place a 16S from a local salmon farm into the broader clade diversity of the species.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The answer is for this species is there are around a thousand sequences on Genbank for the species of interest which are split into 6 identical genotypic groups. The demonstration of the species is represented by the scale bar, i.e. it is very little genetic diversity.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OgU6r.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OgU6r.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Cycloclausticus sp&lt;/em&gt; is the outgroup and each member of the in-group is &lt;em&gt;Piscirickettsia salmonis&lt;/em&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Just to address the comments&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&amp;quot;Cycloclasticus&amp;quot;? The question is not specific to /db_xref=&amp;quot;taxon:1238&amp;quot;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;It is specific there's a fundamental principle of trees. A tree needs a root, all the ingroups are all &lt;em&gt;Piscirickettsia salmonis&lt;/em&gt;. Without a root, i.e. something that is not &lt;em&gt;Piscirickettsia salmonis&lt;/em&gt; there is no direction in the tree. &lt;em&gt;Cycloclasticus&lt;/em&gt; is a good or reasonable root.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;how do i access this interface?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Its complicated to explain and its a different question.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-04-21T18:20:10.277" LastActivityDate="2023-04-21T18:20:10.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20912" PostTypeId="2" ParentId="20904" CreationDate="2023-04-21T08:56:48.417" Score="2" Body="&lt;p&gt;No, it's a bwa mem wrapper for bisulfite data essentially and bwa mem accepts fastq only.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-04-21T08:56:48.417" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20913" PostTypeId="1" CreationDate="2023-04-21T17:07:11.553" Score="1" ViewCount="38" Body="&lt;p&gt;I've been trying to find out a way to retrieve the Objective Power (or Magnification) of a Whole Slide Image file without downloading it first.&#xA;I've seen that some WSI's Objective power might be 40x and others 20x, and I need a way to filter the ones I want to retrieve according with this parameter.&#xA;However, I cannot figure out the query I need for this.&#xA;Can anyone help me&lt;/p&gt;&#xA;&lt;p&gt;Some details:&lt;/p&gt;&#xA;&lt;p&gt;I'm using the GDC API (&lt;a href=&quot;https://gdc.cancer.gov/developers/gdc-application-programming-interface-api&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gdc.cancer.gov/developers/gdc-application-programming-interface-api&lt;/a&gt;) to retrieve Whole Slide Image files (in this case .svs files).&lt;/p&gt;&#xA;&lt;p&gt;I want to retrieve this files according with its Objective Power, since images in the TCGA dataset (&lt;a href=&quot;https://www.genome.gov/Funded-Programs-Projects/Cancer-Genome-Atlas&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.genome.gov/Funded-Programs-Projects/Cancer-Genome-Atlas&lt;/a&gt;) might have 20x or 40x of objective power.&lt;/p&gt;&#xA;&lt;p&gt;One way to do this is to retrieve the whole image using the data endpoint of the GDC API and then use a library like Openslide to check the objective power:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import requests&#xA;from io import BytesIO&#xA;import openslide&#xA;&#xA;# Set up the GDC API endpoint&#xA;endpoint = 'https://api.gdc.cancer.gov/data'&#xA;&#xA;# Set up the query parameters&#xA;query = {&#xA;    'ids': ['my_file_id'],&#xA;}&#xA;&#xA;# Send the request to the GDC API&#xA;response = requests.get(endpoint, params=query)&#xA;&#xA;# Retrieve the binary data from the response&#xA;binary_data = BytesIO(response.content)&#xA;&#xA;# Open the WSI file using the Openslide library&#xA;slide = openslide.open_slide(binary_data)&#xA;&#xA;# Retrieve the Objective Power of the WSI&#xA;objective_power = slide.properties.get(openslide.PROPERTY_NAME_OBJECTIVE_POWER)&#xA;&#xA;print('Objective Power:', objective_power)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, retrieving the whole file takes too much time, so I would like to know if anybody knows of a field in the GDC API that allows me to check the objective power of a WSI without retrieving the file itself.&lt;/p&gt;&#xA;" OwnerUserId="17359" LastEditorUserId="17359" LastEditDate="2023-04-21T19:44:26.690" LastActivityDate="2023-04-21T19:44:26.690" Title="GDC API: Obtain the Objective Power (or Magnification) of a WSI file" Tags="|python|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20915" PostTypeId="1" AcceptedAnswerId="20916" CreationDate="2023-04-22T05:11:13.340" Score="2" ViewCount="190" Body="&lt;p&gt;Consider:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;seqkit grep -nrp &amp;quot;;s__nosocomialis;&amp;quot;  ribogrove_11.217_sequences.fasta&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Using this command lets me select all sequences related to the organism &amp;quot;&lt;em&gt;A. nosocomialis&lt;/em&gt;&amp;quot; in the &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/35217168/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RiboGrove&lt;/a&gt; database.&lt;/p&gt;&#xA;&lt;p&gt;How do I save these selected sequences to a new &amp;quot;.fasta&amp;quot; file?&lt;/p&gt;&#xA;" OwnerUserId="17357" LastEditorUserId="1775" LastEditDate="2023-04-22T19:14:42.337" LastActivityDate="2023-04-23T14:09:42.487" Title="How do I save selected sequences in seqkit to a file?" Tags="|seqkit|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20916" PostTypeId="2" ParentId="20915" CreationDate="2023-04-22T06:13:09.460" Score="4" Body="&lt;p&gt;Based on &lt;a href=&quot;https://bioinf.shenwei.me/seqkit/usage/#grep&quot; rel=&quot;noreferrer&quot;&gt;the docs&lt;/a&gt;, seqkit grep has a &lt;code&gt;-o&lt;/code&gt; &amp;quot;output file&amp;quot; option, so if you get the correct output with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqkit grep -nrp &amp;quot;;s__nosocomialis;&amp;quot; ribogrove_11.217_sequences.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then this command will (in theory) give you your desired outcome:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqkit grep -nrp &amp;quot;;s__nosocomialis;&amp;quot; ribogrove_11.217_sequences.fasta -o new.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9098" LastActivityDate="2023-04-22T06:13:09.460" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20917" PostTypeId="2" ParentId="20910" CreationDate="2023-04-22T06:16:09.063" Score="1" Body="&lt;p&gt;This can be done using the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK25498/#chapter3.ESearch__ESummaryEFetch&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI eutils command-line tools&lt;/a&gt;. For taxa, the &lt;a href=&quot;https://www.biostars.org/p/58955/&quot; rel=&quot;nofollow noreferrer&quot;&gt;correct incantation&lt;/a&gt; for the search string is &lt;code&gt;txid&amp;lt;taxID&amp;gt;[Organism]&lt;/code&gt; (assuming you want that taxon and all its children), but I haven't found the right one for the product field. However, it can be approximated by searching for a string in the tile line via &lt;code&gt;&amp;lt;search&amp;gt;[TITL]&lt;/code&gt;. This seems to work fairly well, assuming you want the fasta files (the &lt;code&gt;-format&lt;/code&gt; argument can be changed to &lt;code&gt;genbank&lt;/code&gt; if you want the genbank record instead):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ esearch -db nuccore -query &amp;quot;txid1238[Organism] AND 16s[TITL]&amp;quot; | efetch -format fasta | grep -A 1 '^&amp;gt;' | head&#xA;&amp;gt;MW433822.1 Piscirickettsia salmonis strain HRV16 16S ribosomal RNA gene, partial sequence&#xA;TCAGATTGAACGCTGGTGGCATGCTTAACACATGCAAGTCGAACGGTAACATGACTAGCTTGCTAGTTGA&#xA;--&#xA;&amp;gt;MW433821.1 Piscirickettsia salmonis strain HRV13 16S ribosomal RNA gene, partial sequence&#xA;AATTTCAGGTGTAGCGGTGAAATGCGTAGATATCTGAAGGAACACCGGTGGCGAAGGCGACTTTCTGGAT&#xA;--&#xA;&amp;gt;MN023100.1 Piscirickettsia salmonis clone Ps18621-6 16S ribosomal RNA gene, partial sequence&#xA;GCTTGCTAGATGCTGACGAGTGGCGGACGGGTGAGTAACGCGTGGGAATTTACCTTTTAGTGGGGGATAA&#xA;--&#xA;&amp;gt;MN023099.1 Piscirickettsia salmonis clone Ps18429-3 16S ribosomal RNA gene, partial sequence&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Unfortunately I can't find anything for searching specifically in the features of an NCBI record. There'll be some way to do it; I just don't know how.&lt;/p&gt;&#xA;&lt;p&gt;Based on the updated request, it's possible to do a full-text search through the entire metadata (e.g. for &lt;code&gt;16S ribosomal RNA&lt;/code&gt;) by adding the search term without specifying a field. Depending on what you want to do with the results, and how many were produced, it could be easy enough to manually unwanted sequences (e.g. whole genome shotgun assemblies, or chimeric products):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ esearch -db nuccore -query &amp;quot;txid1238[Organism:exp] AND (16S ribosomal RNA)&amp;quot; | efetch -format fasta | grep -A 1 '^&amp;gt;' --no-group-separator | tail -n 12&#xA;&amp;gt;EU170254.1 Piscirickettsia salmonis strain NVI 5692 16S ribosomal RNA gene, partial sequence&#xA;CACATGCAAGTCGAACGGTAGCAGATCTAGCTTGCTAGATGCTGACGAGTGGCGGACGGGTGAGTAACGC&#xA;&amp;gt;U36941.1 Piscirickettsia salmonis 16S ribosomal RNA gene, complete sequence&#xA;AGAGTTTGATCCTGGCTCAGATTGAACGCTGGTGGCATGCTTAACACATGCAAGTCGAACGGTAGCAGAT&#xA;&amp;gt;U55015.1 Piscirickettsia salmonis 16S ribosomal RNA gene, complete sequence&#xA;AGAGTTTGATCCTGGCTCAGATTGAACGCTGGTGGCATGCTTAACACATGCAAGTCGAACGGTAGCAGAT&#xA;&amp;gt;U36942.1 Piscirickettsia salmonis strain NOR-92 16S ribosomal RNA gene, complete sequence&#xA;AGAGTTTGATCCTGGCTCAGATTGAACGCTGGTGGCATGCTTAACACATGCAAGTCGAACGGTAGCAGAT&#xA;&amp;gt;U36940.1 Piscirickettsia salmonis strain EM-90 16S ribosomal RNA gene, complete sequence&#xA;AGAGTTTGATCATGGCTCAGATTGAACGCTGGTGGCATGCTTAACACATGCAAGTCGAACGGTAGCAGAT&#xA;&amp;gt;U36915.1 Piscirickettsia salmonis 16S ribosomal RNA gene, complete sequence&#xA;AGAGTTTGATCATGGCTCAGATTGAACGCTGGTGGCATGCTTAACACATGCAAGTCGAACGGTAGCAGAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;FWIW, here's the list of all the searchable fields in the &lt;code&gt;nuccore&lt;/code&gt; database:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ einfo -db nuccore -fields&#xA;ACCN    Accession&#xA;ALL     All Fields&#xA;ASSM    Assembly&#xA;AUTH    Author&#xA;BIOS    BioSample&#xA;BRD     Breed&#xA;COMP    Component Accession&#xA;CULT    Cultivar&#xA;DIV     Division&#xA;ECNO    EC/RN Number&#xA;FILT    Filter&#xA;FKEY    Feature key&#xA;GENE    Gene Name&#xA;GPRJ    BioProject&#xA;ISOL    Isolate&#xA;ISS     Issue&#xA;JOUR    Journal&#xA;KYWD    Keyword&#xA;MDAT    Modification Date&#xA;ORGN    Organism&#xA;PACC    Primary Accession&#xA;PAGE    Page Number&#xA;PDAT    Publication Date&#xA;PORG    Primary Organism&#xA;PROP    Properties&#xA;PROT    Protein Name&#xA;SLEN    Sequence Length&#xA;SQID    SeqID String&#xA;STRN    Strain&#xA;SUBS    Substance Name&#xA;TITL    Title&#xA;UID     UID&#xA;VOL     Volume&#xA;WORD    Text Word&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-04-23T04:47:44.170" LastActivityDate="2023-04-23T04:47:44.170" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20918" PostTypeId="2" ParentId="4534" CreationDate="2023-04-22T07:55:29.393" Score="1" Body="&lt;p&gt;Your issue is that the pattern &lt;code&gt;.&lt;/code&gt; matches a single character. To match the whole header, use &lt;code&gt;.*&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;seqkit replace -p &amp;quot;(.*)&amp;quot; --replacement &amp;quot;{kv}&amp;quot; --kv-file keyValues.txt mySequence.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alternatively as pointed out by others you may not really need &lt;code&gt;keyValues.txt&lt;/code&gt; here as you're just removing the &lt;code&gt;chr_&lt;/code&gt; prefix:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;seqkit replace -p &amp;quot;chr_(.*)&amp;quot; --replacement '$1' mySequence.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(Don't forget the single quotes for &lt;code&gt;$1&lt;/code&gt; else the shell will expand it)&lt;/p&gt;&#xA;" OwnerUserId="15257" LastActivityDate="2023-04-22T07:55:29.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20919" PostTypeId="2" ParentId="20915" CreationDate="2023-04-23T14:09:42.487" Score="1" Body="&lt;p&gt;For any program that prints output to the terminal (standard output), you can always redirect to a file using &lt;code&gt;&amp;gt;&lt;/code&gt; or &lt;code&gt;&amp;gt;&amp;gt;&lt;/code&gt;. Either of the two will create the output file if it doesn't exist, but &lt;code&gt;&amp;gt;&lt;/code&gt; will overwrite any contents of an already existing file, while &lt;code&gt;&amp;gt;&amp;gt;&lt;/code&gt; will append to it without overwriting. So you could just do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqkit grep -nrp &amp;quot;;s__nosocomialis;&amp;quot; ribogrove_11.217_sequences.fasta &amp;gt; out.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-04-23T14:09:42.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20920" PostTypeId="1" AcceptedAnswerId="20926" CreationDate="2023-04-24T00:47:50.100" Score="2" ViewCount="2175" Body="&lt;p&gt;I have a R script that subsets out the tumor cells from 6 different individual seurat objects.&lt;/p&gt;&#xA;&lt;p&gt;It gets a list of file paths for all RDS files containing the expression data for multiple datasets in a directory.&#xA;It creates two empty lists, datasets and dataset_index, to hold the subsets of tumor cells from each dataset and their corresponding indices, respectively.&#xA;It loops through each file path, reads the RDS file, and subsets the tumor cells from the expression data.&#xA;If the dataset does not contain any tumor cells, it prints a message indicating that fact.&#xA;The subsets of tumor cells are appended to the datasets list and their corresponding indices are appended to the dataset_index list.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;scfiles = list.files('/LungDataSets/',pattern = '.*SCTransform_Symphony.*.RDS',recursive=TRUE)&#xA;&#xA;scfiles = paste('/LungDataSets/',scfiles,sep='')&#xA;&#xA;datasets = list()&#xA;dataset_index = list()&#xA;&#xA;for (i in 1:length(scfiles)) {&#xA;    scData = readRDS(scfiles[i])&#xA;    &#xA;    if (is.element('Tumor', unique(scData$cell_ann)))  {&#xA;        tumorData = subset(scData,subset = cell_ann=='Tumor')&#xA;        datasets = append(datasets,tumorData)&#xA;        dataset_index = append(dataset_index,toString(i))&#xA;    }&#xA;    else {&#xA;        print(c(scfiles[i], 'does not contain Tumor cells'))&#xA;    }&#xA;        &#xA;    &#xA;    rm(scData)&#xA;    &#xA;    &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now I want to merge all those tumor cells from the 6 datasets into one seurat object.&lt;/p&gt;&#xA;&lt;p&gt;I've tried &lt;code&gt;merge&lt;/code&gt; and &lt;code&gt;rbind&lt;/code&gt; but they don't work.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;megaObject &amp;lt;- merge(datasets)&#xA;Error in as.data.frame.default(x[[i]], optional = TRUE) : &#xA;  cannot coerce class ‘structure(&amp;quot;Seurat&amp;quot;, package = &amp;quot;SeuratObject&amp;quot;)’ to a data.frame&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6827" LastActivityDate="2023-04-24T18:18:34.043" Title="Merging multiple seurat datalist together" Tags="|r|scrnaseq|seurat|merge|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20921" PostTypeId="1" CreationDate="2023-04-24T06:35:26.887" Score="2" ViewCount="45" Body="&lt;p&gt;&lt;em&gt;&lt;strong&gt;This question has been cross-posted on &lt;a href=&quot;https://www.biostars.org/p/9561427/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt; but has not received any answers yet, so I'm posting it here. I'll update here if it gets answered there.&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have downloaded a miRNA expression dataset from NCBI GEO &lt;code&gt;(GSE25631)&lt;/code&gt; to study differential gene expression and perform other analyses. As mentioned in GEO, this profiling was performed on GPL8179 Illumina Human v2 MicroRNA expression beadchip. Accordingly, I installed the &lt;code&gt;illuminaHumanv2.db&lt;/code&gt; package to annotate the Illumina probe IDs. However, when I tried to convert the Illumina probe IDs to their corresponding gene symbols, it returned &lt;code&gt;NA&lt;/code&gt; for all the probes. Can anybody identify the issue with this approach? Here's the R script which I used.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(GEOquery)&#xA;library(limma)&#xA;library(illuminaHumanv2.db)&#xA;&#xA;Sys.setenv(VROOM_CONNECTION_SIZE = 256000)&#xA;&#xA;data &amp;lt;- getGEO(GEO = &amp;quot;GSE25631&amp;quot;, &#xA;           destdir = &amp;quot;E:\\GSE25631&amp;quot;,&#xA;           GSEMatrix = TRUE,&#xA;           AnnotGPL = FALSE,&#xA;           getGPL = FALSE,&#xA;           parseCharacteristics = TRUE)&#xA;&#xA;data &amp;lt;- data$GSE25631_series_matrix.txt.gz&#xA;&#xA;raw_intensity &amp;lt;- exprs(data)&#xA;&#xA;samples &amp;lt;- as.character(pData(data)[,&amp;quot;title&amp;quot;])&#xA;&#xA;raw_intensity_log &amp;lt;- log2(raw_intensity)&#xA;colnames(raw_intensity_log) &amp;lt;- c(rep(&amp;quot;GBM&amp;quot;, 82), rep(&amp;quot;Normal&amp;quot;, 5))&#xA;&#xA;probe_id &amp;lt;- rownames(raw_intensity_log)&#xA;mapping &amp;lt;- data.frame(Gene = unlist(mget(x = probe_id, envir = illuminaHumanv2SYMBOL, ifnotfound = NA)))&#xA;&#xA;&#xA;&#xA;head(mapping, 10)&#xA;               Gene&#xA;ILMN_3166935   NA&#xA;ILMN_3166938   NA &#xA;ILMN_3166940   NA&#xA;ILMN_3166941   NA &#xA;ILMN_3166943   NA&#xA;ILMN_3166944   NA&#xA;ILMN_3166945   NA&#xA;ILMN_3166948   NA&#xA;ILMN_3166952   NA&#xA;ILMN_3166953   NA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;GSE25631_RAW.tar&lt;/code&gt; file in the supplementary section has all the Probe ID to Gene mappings but I'm curious to know why this approach didn't work.&lt;/p&gt;&#xA;" OwnerUserId="14724" LastEditorUserId="3694" LastEditDate="2023-04-24T09:21:53.913" LastActivityDate="2023-04-24T09:21:53.913" Title="illuminaHumanv2.db returning NA for all Illumina probe ID to gene symbol conversion" Tags="|r|bioconductor|microarray|illumina|geoquery|" AnswerCount="0" CommentCount="3" ClosedDate="2023-04-25T11:56:10.177" ContentLicense="CC BY-SA 4.0" />
  <row Id="20922" PostTypeId="2" ParentId="20920" CreationDate="2023-04-24T06:36:04.613" Score="0" Body="&lt;p&gt;I can't reproduce things quickly without a &lt;a href=&quot;https://stackoverflow.com/help/minimal-reproducible-example&quot;&gt;minimum reproducible example (e.g. with sample data from the package)&lt;/a&gt;. However, the error is telling you that R cannot convert the object to a dataframe when trying to use &lt;code&gt;merge()&lt;/code&gt; (at least I suspect that this error occurred when you ran &lt;code&gt;merge()&lt;/code&gt; since you posted it right under it). I suggest you convert &lt;em&gt;everything you need&lt;/em&gt; to a dataframe (there are quite some questions, e.g., on biostars regarding this conversion). &lt;em&gt;Everything you need&lt;/em&gt; because a SeuratObject cannot simply be cast to a dataframe, only parts of it, e.g., &lt;code&gt;data_transcripts_seurat@assays$SCT@counts %&amp;gt;% as.matrix() %&amp;gt;% t() %&amp;gt;% as.data.frame()&lt;/code&gt; (&lt;a href=&quot;https://www.biostars.org/p/482473/&quot; rel=&quot;nofollow noreferrer&quot;&gt;from this biostars question&lt;/a&gt;). However, you might need some other information. Therefore, I suggest you explore the SeuratObject using RStudio and check what information you want to draw from it. After the (interesting) parts of the SeuratObject have been cast to dataframes, e.g., in the for-loop, you can go ahead with merging.&lt;/p&gt;&#xA;" OwnerUserId="15933" LastActivityDate="2023-04-24T06:36:04.613" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20923" PostTypeId="1" CreationDate="2023-04-24T06:42:45.580" Score="0" ViewCount="12" Body="&lt;p&gt;First question: I found that there is no galaxy.yml in the config directory of the running version of docker galaxy, only galaxy.yml.sample file and galaxy.yml.interactivatetools file, so do I need to create it, or do I need to create a new one using the cat command? galaxy.yml file?&#xA;The second question is about these initial notes, where can I go deeper?&lt;/p&gt;&#xA;" OwnerUserId="17353" LastActivityDate="2023-04-24T06:42:45.580" Title="Do you need galaxy.yml when you start creating galaxy?" Tags="|galaxy|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20924" PostTypeId="2" ParentId="20920" CreationDate="2023-04-24T09:01:57.350" Score="2" Body="&lt;p&gt;You should be able to use something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Reduce(&#xA;    f = function(x, y) {merge(x, y, merge.data = FALSE)},&#xA;    x = datasets # list of Seurat objects&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will create a new Seurat object based on the multiple seurat objects in your list. It might be good idea to store the &amp;quot;sample&amp;quot; information within the metadata slots of individual objects. Probably a better approach is to have cell barcodes of each sample prefixed with the sample id.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2023-04-24T09:01:57.350" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20925" PostTypeId="1" AcceptedAnswerId="20929" CreationDate="2023-04-24T12:41:06.730" Score="1" ViewCount="121" Body="&lt;p&gt;I am trying to create a GATK g CNV pipeline and I need to make shards before running the &lt;code&gt;GermlineCNVCaller&lt;/code&gt; therefore, I believe I need to use &lt;code&gt;SplitIntervals&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I have the following predefined variables:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;singularity_image&lt;/code&gt; is the path to the GATK sif file&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;bam_file&lt;/code&gt; is a bam file from a defined bam directory&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;reference&lt;/code&gt; is a path to a .fa file&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;targets_interval_list&lt;/code&gt; is a interval list created by Picard (ending with .interval_list)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;NUM_BINS&lt;/code&gt; is a manually preset value from 100 to 200.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;  singularity exec $singularity_image /gatk/gatk --java-options -Xms8000m \&#xA;  SplitIntervals \&#xA;  -I ${bam_file} \&#xA;  -R ${reference} \&#xA;  -L ${targets_interval_list} \&#xA;  --scatter-count ${NUM_BINS} \&#xA;  -O ${output_dir}/Interval_list_${bam_filename}_.shard_\%${NUM_BINS}\%_\%{shard_index}\%.interval_list&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have tried change the specifications under &lt;code&gt;SplitIntervals&lt;/code&gt; but for whatever reason I keep getting the following error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;***********************************************************************&#xA;&#xA;A USER ERROR has occurred: but no positional argument is defined for this tool.&#xA;&#xA;***********************************************************************&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Why is this error occurring?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The documentation I am using is here:&lt;/strong&gt; &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360036899592-SplitIntervals&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360036899592-SplitIntervals&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17244" LastEditorUserId="17244" LastEditDate="2023-06-08T20:06:57.063" LastActivityDate="2023-06-08T20:06:57.063" Title="GATK SplitIntervals: A USER ERROR has occurred: but no positional argument is defined for this tool" Tags="|gatk|singularity|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20926" PostTypeId="2" ParentId="20920" CreationDate="2023-04-24T18:18:34.043" Score="3" Body="&lt;p&gt;The first parameter of &lt;code&gt;merge&lt;/code&gt; should be a Seurat object, the second (&lt;code&gt;y&lt;/code&gt;) can be one Seurat object or a list of several. Try:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;merge(x = datasets[[1]], y = datasets[-1])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;See the &lt;a href=&quot;https://satijalab.org/seurat/articles/merge_vignette.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;merge vignette&lt;/a&gt; for more details. You may want to use the &lt;code&gt;add.cell.ids&lt;/code&gt; option to be able to tell which dataset each cell originated from.&lt;/p&gt;&#xA;" OwnerUserId="15489" LastActivityDate="2023-04-24T18:18:34.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20927" PostTypeId="1" CreationDate="2023-04-24T19:59:52.217" Score="3" ViewCount="108" Body="&lt;p&gt;I am seeking advice on whether or not to incorporate incomplete data into my &lt;strong&gt;BEAST&lt;/strong&gt; analysis.&lt;/p&gt;&#xA;&lt;p&gt;I have HIV data consisting of two partitions of the &lt;em&gt;pol&lt;/em&gt; gene: one that is 1.2 &lt;em&gt;kb&lt;/em&gt; in length and present in all isolates, &amp;amp; another that is 2 &lt;em&gt;kb&lt;/em&gt; in length but only present in one-third of the isolates. I am wondering if I should include the second partition in my analysis, despite its incomplete nature.&lt;/p&gt;&#xA;&lt;p&gt;I have found some publications discussing the use of incomplete data in phylogenetic analyses, such as &lt;a href=&quot;https://doi.org/10.1016/j.ympev.2015.02.002&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yuchi Zheng &amp;amp; John J. Wiens (2015)&lt;/a&gt;, &lt;a href=&quot;https://doi.org/10.1016/j.ympev.2013.06.004&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rasmus Hovmöller et al. (2013)&lt;/a&gt;, &lt;a href=&quot;https://doi.org/10.1371/journal.pone.0042925&quot; rel=&quot;nofollow noreferrer&quot;&gt;John J. Wiens &amp;amp; Jonathan Tiu (2012)&lt;/a&gt;. However, I remain uncertain about the best course of action.&lt;/p&gt;&#xA;&lt;p&gt;If you have experience working with incomplete data in BEAST analyses or any insights on this matter, I would greatly appreciate your advice.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance for your help.&lt;/p&gt;&#xA;&lt;p&gt;Best regards,&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h2&gt;Update&lt;/h2&gt;&#xA;&lt;p&gt;As seen from the answer &amp;amp; comments, there has been a misinterpretation of my partitions' structure &amp;amp; their incompleteness. So hereinbelow is a short description of my data, the subject topic, &amp;amp; a diagram.&lt;/p&gt;&#xA;&lt;p&gt;There are 1009 isolates of the &lt;em&gt;pol&lt;/em&gt; gene of HIV from the Republic of Armenia. They were sequenced from the start of the gene by two means: Sanger &amp;amp; an NGS; which caused two groups of length: 1.2 kb &amp;amp; &amp;gt; 3 kb, respectively.&lt;/p&gt;&#xA;&lt;p&gt;This way the situation may be demonstrated via the next artificial miniature diagram:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/f7wnI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/f7wnI.png&quot; alt=&quot;diagram with a little situation example&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Based on the picture like above, I have thought it rational to divide the data into two staple partitions: the first third of &lt;em&gt;pol&lt;/em&gt; &amp;amp; its last part. The first partition has all sequences completed, and the second one has mostly full-length complete sequences for a third of isolates and just &lt;code&gt;NNN…NNN&lt;/code&gt;s for the other two-thirds.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;N&lt;/code&gt;s are automatically added at the alignment step. Sanger sequences are just short and end from 1200 to 1500. I currently do not have access to the source data. They were provided to me already built and aligned at the reference.&lt;/p&gt;&#xA;&lt;p&gt;In summary, the question is, &lt;strong&gt;&amp;quot;Should only the complete first partition be used in subsequent BEAST analysis or should the incomplete second one be included as well?&amp;quot;&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="15161" LastEditorUserId="73" LastEditDate="2023-04-27T00:06:24.487" LastActivityDate="2023-04-27T00:06:24.487" Title="What is the limit of partition incompleteness in Bayesian MCMC?" Tags="|phylogenetics|phylogeny|virus|beast|hiv|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20928" PostTypeId="1" CreationDate="2023-04-24T20:04:12.333" Score="0" ViewCount="9" Body="&lt;p&gt;I am currently working on verifying the subgenotyping of our sequences and a subsample from the &lt;strong&gt;Los Alamos NL&lt;/strong&gt; database. My supervisor has recommended using &lt;strong&gt;REGA&lt;/strong&gt; &amp;amp; &lt;strong&gt;COMET&lt;/strong&gt; for this purpose. While our dataset, which consists of slightly &amp;gt; 1,000 sequences, can be easily subgenotyped using the online versions of these tools, the &lt;strong&gt;LANL&lt;/strong&gt; dataset is significantly larger, prompting me to consider local installations. Unfortunately, however, I have been unable to find the source repositories for &lt;strong&gt;REGA&lt;/strong&gt; &amp;amp; &lt;strong&gt;COMET&lt;/strong&gt;.&lt;br /&gt;&#xA;I did come across a link to the entire &lt;a href=&quot;https://rega.kuleuven.be/cev/regadb/download&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;REGA DB&lt;/strong&gt;&lt;/a&gt; but I am unsure if it contains the subgenotyping tool or if the last can be installed separately. I am also interested in exploring other locally installable tools of comparable quality.&lt;br /&gt;&#xA;In my search, I found &lt;a href=&quot;http://jphmm.gobics.de/jphmm.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;jpHMM&lt;/strong&gt;&lt;/a&gt;, which is available for download and free use. However, it seems to be less frequently used in publications compared to the aforementioned ones.&lt;/p&gt;&#xA;&lt;p&gt;I would appreciate any insights on the quality of &lt;strong&gt;jpHMM&lt;/strong&gt; or any other recommended tools for local installation (even as packages, e.g. for &lt;em&gt;Python&lt;/em&gt;).&lt;/p&gt;&#xA;&lt;p&gt;If you have experience with these subgenotyping tools or any relevant advice, please share your thoughts! Your help is greatly appreciated!&lt;/p&gt;&#xA;&lt;p&gt;Best regards,&lt;/p&gt;&#xA;" OwnerUserId="15161" LastActivityDate="2023-04-24T20:04:12.333" Title="Is there available REGA or COMET versions for downloading &amp; using on a local machine?" Tags="|python|hiv|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20929" PostTypeId="2" ParentId="20925" CreationDate="2023-04-24T20:45:34.107" Score="1" Body="&lt;p&gt;This solution where the &lt;code&gt;-O&lt;/code&gt; is just set to the output directory seems the most easy way to handle the issue:&lt;/p&gt;&#xA;&lt;p&gt;Keep the following variables the same:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;singularity_image&lt;/code&gt; is the path to the GATK sif file&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;bam_file&lt;/code&gt; is a bam file from a defined bam directory&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;reference&lt;/code&gt; is a path to a .fa file&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;targets_interval_list&lt;/code&gt; is a interval list created by Picard (ending with .interval_list)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;NUM_BINS&lt;/code&gt; is a manually preset value from 100 to 200.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The code with the modified &lt;code&gt;-O&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  # run SplitIntervals with shard name&#xA;  singularity exec &lt;span class=&quot;math-container&quot;&gt;$singularity_image /gatk/gatk --java-options -Xms8000m \&#xA;  SplitIntervals \&#xA;  -R $&lt;/span&gt;{reference} \&#xA;  -L &lt;span class=&quot;math-container&quot;&gt;${targets_interval_list} \&#xA;  --scatter-count $&lt;/span&gt;{NUM_BINS} \&#xA;  -O ${output_dir}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;While it doesn't name the shards in a very elegant way it seems to work. The shards are made from the interval list and therefore, should not be named after the .bams that will be processed in other steps that are used (for in this case a GATK g CNV pipeline).&lt;/p&gt;&#xA;" OwnerUserId="17244" LastActivityDate="2023-04-24T20:45:34.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20930" PostTypeId="2" ParentId="20927" CreationDate="2023-04-24T21:21:16.600" Score="1" Body="&lt;p&gt;The original answer was &amp;quot;criticised&amp;quot; for being obtuse.&lt;/p&gt;&#xA;&lt;p&gt;The basic very simple answer is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;If the data you present in the diagram (alignment) is left EXACTLY in the format used to make the tree - it honestly does not matter whether you have 1 partition or 2 partitions. It makes no practical difference. The posterior likelihood and stuff will change a bit - basically the result is the same. The reason is given in the very long explanation below. I admit 2-partitions looks cooler and there's reason not to do it, computationally it will be a bit slower.&#xA;&lt;strong&gt;Partitions&lt;/strong&gt; I am happy to explain why a partition is used  - but thats different answer or question.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What must not be done with HIV is changing every '-' to '?' or changing every '-' to 'N'. Whether this is 1-partition or 2-partitions. With higher eukaryotes that is fine and that will work very nicely, but HIV no. So Weins et al (2012) can do this (almost certainly have done), but you can't.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I know it seems a very small change replacing every '-' with a '?' or 'N', but what it represents seismic. I have attempted to explain why it is so important below the line, but it's about indels (insertions/deletions). I recognise my explanation may not be perfect.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Confusion&lt;/strong&gt; The confusing bit is you mention 'NNNNNs' and thats the worry. If NNNNs are changed to '-' thats fine, if they are left as 'NNNNs' that raises the complicated spectre of 'indels' and will get very complicated. It probably can be done but its really complicated.&lt;/p&gt;&#xA;&lt;p&gt;To make the point clear if I perform a &lt;code&gt;trimal&lt;/code&gt; analysis on my alignment&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;trimal -in infile -out outfile -nogap -fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It will remove all sites with '-' in them, i.e. it will remove the entire site, not just the '-' character. I think it's really cool. Anyway, the tree from a &lt;code&gt;trimal&lt;/code&gt; alignment will be basically the same as '1-partition' or '2-partitions' with all the '-' retained, even though the alignments look very different. That sounds weird but it is a consequence of how Bayes (i.e. Beast) or ML calculate phylogenetic information, which I tried to explain below. HOWEVER, if there is an alignment with 'NNNNs' instead of '-' that will produce a very different tree.&lt;/p&gt;&#xA;&lt;p&gt;So for simplicity the answer is that 2-partitions is fine for '-', but not if those '-' are represented as 'NNNNN'. Thus for '-' alignments it looks cool, but makes little difference. 2-partitions can make a HUGE difference with different data particularly with loads and loads of NNNNNs instead of '-'.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; HIV is a special case because &lt;em&gt;env&lt;/em&gt; in particular - and possibly all the HIV genes - are known to contain extensive indels. Indels in &lt;em&gt;env&lt;/em&gt; are so prevalent in HIV they can be approximated via distribution (did Ziheng Yang do that?). So the short answer, Weins &lt;em&gt;et al&lt;/em&gt; (2012) can do it because they're dealing with higher eukaryotes and genes with negligible indels (in addition they could remove known indel sites), in HIV you would need independent evidence the missing data site was not going to be an indel (i.e. that particular site has never had a known indel in any HIV data set).&lt;/p&gt;&#xA;&lt;p&gt;The crux is there is a difference between missing data and absent data. Absent data &lt;em&gt;can be&lt;/em&gt; a no-go (below; context depending - some times it makes no difference [immediately below]), it can be detrimental to the phylogenetic signal. Missing data is very interesting.&lt;/p&gt;&#xA;&lt;p&gt;Basically, if you are going to leave the missing 2kb gene as '-' ... thats perfectly fine. It will not add any phylogenetic information - but it will not hurt the calculation. If 'N' or '?' are used instead to represent a missing 2Kb locus for HIV instead  of '-' thats different.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Weins &lt;em&gt;et al&lt;/em&gt; (2012) is a very interesting study.&lt;/p&gt;&#xA;&lt;p&gt;There are four issues&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Sample skews&lt;/li&gt;&#xA;&lt;li&gt;Absent data which causes blockwise deletions in Bayes and ML&lt;/li&gt;&#xA;&lt;li&gt;Missing data which &lt;em&gt;might&lt;/em&gt; recover phylogenetic information at a given site if correctly annotated.&#xA;4.Quality of data&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The traditional view is that it is better to remove the site/ or sample, if the quality of data is in doubt or for example an entire gene is missing. The traditional view is it is better to be certain of the data regardless of possible risk of a sample skew. However, &lt;em&gt;sample skews&lt;/em&gt; are the Achilles heal of population genetics and a lesser extent to phylogenetics. Thats what Weins &lt;em&gt;et al&lt;/em&gt; (2012) is concluding, it's really cool. I like it.&lt;/p&gt;&#xA;&lt;p&gt;We all know that an analyst can't deliberately cause a sample skew (i.e. you can't omit samples without good reason) - that goes with out saying. Right?&lt;/p&gt;&#xA;&lt;p&gt;The context of the question is &amp;quot;if a sample has &lt;em&gt;pol&lt;/em&gt; and its other gene wasn't sequenced - do you fill in the absent gene with &amp;quot;Ns&amp;quot; or &amp;quot;?&amp;quot; to trap the phylogenetic information in &lt;em&gt;pol&lt;/em&gt; and the samples which have data in the gene/locus/partition with limited coverage. &lt;strong&gt;It's a really good question.&lt;/strong&gt; My suspicion for HIV is I'm afraid you probably can't, but that might just be &lt;em&gt;env&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Absent data&lt;/strong&gt;&#xA;Bayesian and ML have no '-' character in their matrix. If an indel is present it will trigger that site to be excluded from carrying any phylogenetic information specifically for Bayes and ML. Thus&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;                Site1&#xA;Sample 1         A&#xA;Sample 2         A&#xA;Sample 3         A&#xA;....&#xA;Sample 9999      -&#xA;Sample 10000     A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Site 1 will not carry any phylogenetic information for Bayes or ML because the single '-' is not represented in the mutation matrix. The entire site for 9999 nucleotides is excluded. A pairwise analysis would be needed, e.g. distance methods.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Missing data&lt;/strong&gt; However &amp;quot;Sample 9999, Site 1&amp;quot; could be represented as 'N' instead of a '-' because that means any nucleotide in the matrix. &lt;em&gt;That works!&lt;/em&gt; Site 1 is now phylogenetically informative under Bayes and ML. Hooray! However, are we &lt;em&gt;certain&lt;/em&gt; Sample 9999, Site 1 is a nucleotide rather than an indel? If indels are rewritten as nucleotides, i.e. an absent nucleotide is converted to a nucleotide of unknown type ('AGCT'), i.e. missing data - non-nucleotide to nucleotide - thats cheating.&lt;/p&gt;&#xA;&lt;p&gt;This creates issues &lt;strong&gt;particularly&lt;/strong&gt; for HIV &lt;em&gt;env&lt;/em&gt; where there are loads of indels. If an indel was replaced with an 'N' - the analysis might work nicely, but there is real risk it is not being true to the data.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Possibly explanation&lt;/strong&gt; Sample skews in phylogeny are really interesting. What can happen is long internal branches can cause changes to the branching order. However, if those internal branches are shorten by more rigorous taxon sampling then these shortened internal branches can recover the true true. I think it was Jo Felsenstein that discovered this (the guy who invented maximum likelihood as a method for phylogenetics in place of parsimony).&lt;/p&gt;&#xA;&lt;p&gt;One explanation of Weins &lt;em&gt;et al&lt;/em&gt; (2012) is that the 'missing data' samples are shortening the distance between internal branches - by providing a more rigorous sample - and that is resulting in recovery of the correct tree. So it is probably a result inline with Jo Felsensteins original observation.&lt;/p&gt;&#xA;&lt;p&gt;So if &lt;em&gt;&lt;strong&gt;pol&lt;/strong&gt;&lt;/em&gt; never has indels - ever - you can do it. This would also help population dynamics and all shorts of calculations. If there's a risk that you are replacing indels with nucleotides it is very much, much better to avoid it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt; Everyone in phylogentics knows the issues of sample skews - a possible sample skew is permitted if the data is genuinely never been produced - it's not in existence (obviously). However, it is not permitted if the data exists but the analyst hasn't included it - for whatever reason. Its also preferred if the quality is in question - its better to err on the side of caution.&lt;/p&gt;&#xA;&lt;p&gt;If you fill in an absent gene with &amp;quot;?&amp;quot; or &amp;quot;N&amp;quot; to obtain phylogenetic information from genes that are present for that sample ... that is permitted providing it doesn't change the possible nature of the underlying data. &lt;em&gt;The rationale for doing this is the risk of sample skews.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;So would I adopt a &amp;quot;non-traditional&amp;quot; approach? As a general rule for viruses no and this is a particular special case regarding indels.&lt;/p&gt;&#xA;&lt;p&gt;If this was a higher eukaryote and like - that critical sample and  &lt;em&gt;that&lt;/em&gt; gene - didn't give sequence. This has happened. So would I dump the sample (or in this case the locus) - no I'd caveat the tree and explain and empirically check the assumptions.&lt;/p&gt;&#xA;&lt;p&gt;If I was in a situation where 2/3 of the data were missing, for a higher eukaryote (DNA with great proof-reading), personally I would do a full scale investigation, i.e. phylogenies with and without the gene/locus/partition in question. That is pretty much was Weins et al (2012) have done.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-04-26T21:16:24.553" LastActivityDate="2023-04-26T21:16:24.553" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20931" PostTypeId="1" CreationDate="2023-04-25T06:31:04.147" Score="4" ViewCount="131" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/9561109&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am looking forward to getting a valuable suggestion for a bioinformatic problem.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Currently, I am performing a de novo whole genome assembly. At the stage of barcode correction, I lost nearly half of all the reads due to erroneous barcodes.&lt;/p&gt;&#xA;&lt;p&gt;Barcodes, in this context, are 18-nucleotide-long DNA sequences that were added to the DNA insert during NGS library preparation. Now, I have a file that contains all the 18-base barcodes used in my projects. These barcodes are important because they are unique for reads that came from a DNA molecule and therefore helpful in generating long-range information.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hdwIJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hdwIJ.png&quot; alt=&quot;Barcode statistics&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;While library construction, 18-base molecular barcodes were used for the linked-read generation, and the barcodes have a high incidence of &amp;quot;N&amp;quot; in positions between 12 and 18, as you may see below.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/W70ih.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W70ih.png&quot; alt=&quot;Erroneous barcodes&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There is a very large occurrence of Ns in positions 12 to 18 of the barcodes and for this reason, a large pool of erroneous barcodes were filtered.&lt;/p&gt;&#xA;&lt;p&gt;Now, I have been asked by the company to do one of two things, to recover the lost reads:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Either replace the barcodes(barcode fastq file from sequencing core) with the correct ones from a pool of barcodes (This is a text file I received from the company that contains the pool of all 18 base barcode sequences)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Since most &amp;quot;N&amp;quot;s are seen in positions 12 to 18 of the 18-base barcode sequences (barcode fastq file from the sequencing core), replace the 11-mers in the barcode file, and replace them with the 11-mers from the correct barcode file (text file).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I have decided to use Python code to execute the first method.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;#Define the path where fastq file is stored.&#xA;import os&#xA;from Bio.SeqIO.QualityIO import FastqGeneralIterator &#xA;path=os.path.join(os.path.expanduser(&amp;quot;~&amp;quot;), &amp;quot;Desktop&amp;quot;, &amp;quot;SVA1_S1_R2_001.fastq&amp;quot;)&#xA;&#xA;#create a new file handle to write the file&#xA;path_new=os.path.join(os.path.expanduser(&amp;quot;~&amp;quot;), &amp;quot;Desktop&amp;quot;, &amp;quot;trimmedfastq.fastq&amp;quot;)&#xA;path_ust_barcode=os.path.join(os.path.expanduser(&amp;quot;~&amp;quot;), &amp;quot;Desktop&amp;quot;, &amp;quot;UST40MBarcodes.txt&amp;quot;)&#xA;write_file=open(path_new, &amp;quot;w&amp;quot;)&#xA;&#xA;#open the text file containg the correct pool of barcodes&#xA;&#xA;with open(path_ust_barcode, &amp;quot;r&amp;quot;) as fh:&#xA;   barcodes = fh.readlines()&#xA;&#xA;handle=open(path, &amp;quot;r&amp;quot;)  &#xA;output = []&#xA;&#xA;#===================================================&#xA;# Return the Hamming distance between string1 and string2.&#xA;def hamming_distance(string1, string2): &#xA;    # Start with a distance of zero, and count up&#xA;    distance = 0&#xA;    # Loop over the indices of the string&#xA;    L = len(string1)&#xA;    for i in range(L):&#xA;        # Add 1 to the distance if these two characters are not equal&#xA;        if string1[i] != string2[i]:&#xA;            distance += 1&#xA;    # Return the final count of differences&#xA;    return distance&#xA;#===================================================&#xA;&#xA;#set correct barcode to &amp;quot;None&amp;quot;&#xA;corr_barcode=None&#xA;#set the minimum hamming distance to &amp;quot;None&amp;quot;&#xA;min_distance=19&#xA;#open the fastq file&#xA;&#xA;for title, sequence, qual in FastqGeneralIterator(handle):&#xA;&#xA;    # if a perfect match exists, just output as is&#xA;    if sequence in barcodes:&#xA;        output.append(&amp;quot;@%s\n%s+\n%s\n&amp;quot; % (title, corr_barcode, qual))&#xA;        continue&#xA;&#xA;    for read in barcodes:&#xA;&#xA;        h_distance=hamming_distance(sequence, read)&#xA;&#xA;        # since if there was a perfect match we won't be here, &#xA;        # if distance is 1, we'll never do better&#xA;        if h_distance == 1:&#xA;            corr_barcode = read&#xA;            break&#xA;&#xA;        if  h_distance &amp;lt;= min_distance:&#xA;            min_distance=h_distance &#xA;            corr_barcode=read&#xA;&#xA;    output.append(&amp;quot;@%s\n%s+\n%s\n&amp;quot; % (title, corr_barcode, qual))&#xA;&#xA;&#xA;with open(path_new, &amp;quot;a&amp;quot;) as wf:&#xA;    for line in output:&#xA;        wf.write(line)&#xA;&#xA;handle.close()&#xA;print(&amp;quot;Done&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would like to know if this is the only Pythonic way to tackle this problem. This method is dead slow. My erroneous barcode fastq file is 30+ Gb, and the UST10x barcode file (the text file that contains all the correct barcode pool) is of size 750Mb. I have split this text file into 1000 smaller files, each about 750kb and having ~40,000 lines per file. It's been the third day and the program is still running without outputting anything. I am looking forward to getting valuable suggestions here, with respect to the code, especially how I can make the process faster.&lt;/p&gt;&#xA;&lt;p&gt;NB:  I've tried replacing&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;with open(path_ust_barcode, &amp;quot;r&amp;quot;) as fh:&#xA;   barcodes = fh.readlines()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;with&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;barcodes = list()&#xA;with open(path_ust_barcode, &amp;quot;r&amp;quot;) as fh:&#xA;   for b in fh:&#xA;      barcodes.append(b.rstrip())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Still, the code runs extremely slowly.&lt;/p&gt;&#xA;&lt;p&gt;I am using an IBM server with a 197GM RAM, and an octa-core processor. In my initial code, it was iterating through all the correct barcodes of the text file (UST10x file), for each erroneous barcode in the fastq file, at the end of every iteration, it opened a text file in append mode and wrote the correct version of fastq reads. Since my server has high memory capacity, I rewrote the code in such a way that it stored all the correct barcodes in the memory, and finally wrote it in a text file, by bringing the write function outside of the loop.&lt;/p&gt;&#xA;&lt;p&gt;Python is not absolutely necessary. I'm not familiar with C or C++, but that being said, I would really love to get this process done. I'm using an IBM server with Ubuntu (Jammy), and 197GB Ram, and the current local storage has only 150GM free space. if you can provide me suggestions as to how to execute it in C or C++, I can do it.&lt;/p&gt;&#xA;" OwnerUserId="16940" OwnerDisplayName="VIJITH KUMAR" LastEditorUserId="73" LastEditDate="2023-04-25T23:50:54.820" LastActivityDate="2023-05-04T21:16:49.017" Title="How can I make this Biopython program (to correct erroneous barcodes) run faster, and is there any alternative method?" Tags="|sequence-analysis|biopython|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20932" PostTypeId="1" AcceptedAnswerId="20933" CreationDate="2023-04-25T14:45:42.633" Score="2" ViewCount="187" Body="&lt;p&gt;I'm looking at RNAseq data from CCLE. The data is paired-end.&lt;/p&gt;&#xA;&lt;p&gt;Take the cell line Hs578T and the gene &lt;em&gt;HRAS&lt;/em&gt; as an example.&lt;/p&gt;&#xA;&lt;p&gt;The cell line carries a G12D mutation (c.35G&amp;gt;A), so the change in cds is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ggc ggtgtgggca agagtgcgct g - Wildtype CDS&#xA;gAc ggtgtgggca agagtgcgct g - Mutant CDS&#xA; ^&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My question is, when I &lt;code&gt;grep&lt;/code&gt; the mutant CDS &lt;code&gt;gAcggtgtgggcaagagtgcgctg&lt;/code&gt;, I do not get a match in my .bam file. But when I &lt;code&gt;grep&lt;/code&gt; the reverse complement (i.e., anti-sense sequence), I get matches coming from both mates.&lt;/p&gt;&#xA;&lt;p&gt;My question is, why are there reads in the .bam file that correspond to the anti-sense strand of the DNA? Shouldn't all the reads of mRNA be the CDS?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I got the answer -&#xA;The punch line is that the sequence in .bam files is ALWAYS '+' strand of the reference, no matter what mate the read came from, or which strand the gene sits on.&lt;/p&gt;&#xA;" OwnerUserId="7056" LastEditorUserId="7056" LastEditDate="2023-04-25T16:22:50.870" LastActivityDate="2023-04-25T16:22:50.870" Title="Why is there antisense sequence in RNAseq data" Tags="|rna-seq|strandedness|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20933" PostTypeId="2" ParentId="20932" CreationDate="2023-04-25T14:59:09.827" Score="3" Body="&lt;p&gt;Yes, the mRNA results should include CDSs (not only, don't forget you also have UTR sequences which are non-coding so are not part of the CDS), but there is no reason why they should all be from genes on the + strand of the genome. Remember that the distinction between &amp;quot;sense/+&amp;quot; and &amp;quot;antisense/-&amp;quot; is essentially a human invention; it doesn't imply that the cell considers one strand to be in any way superior than the other, and genes are pretty randomly distributed across both strands.&lt;/p&gt;&#xA;&lt;p&gt;This means that when you align to a human reference genome, you will always have some sequences aligning to the + strand, if the RNAs were transcribed from genes that happen to be on the sense strand, and others will align to the - strand, if they were transcribed from genes found in the antisense strand.&lt;/p&gt;&#xA;&lt;p&gt;This is completely normal and expected, there is nothing wrong with your data. For example, human &lt;a href=&quot;https://genome-euro.ucsc.edu/cgi-bin/hgTracks?db=hg38&amp;amp;lastVirtModeType=default&amp;amp;lastVirtModeExtraState=&amp;amp;virtModeType=default&amp;amp;virtMode=0&amp;amp;nonVirtPosition=&amp;amp;position=chr17%3A7668421%2D7687490&amp;amp;hgsid=296456572_3scecBBFuWBIs3UAbZqJhMdmIY8A&quot; rel=&quot;nofollow noreferrer&quot;&gt;tP53&lt;/a&gt; is on the antisense strand (note how the arrows are all pointing to the left):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RaA7e.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RaA7e.png&quot; alt=&quot;screnshot of the UCSC genome browser showing tp53&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-04-25T14:59:09.827" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="20935" PostTypeId="1" AcceptedAnswerId="20937" CreationDate="2023-04-25T20:30:39.723" Score="0" ViewCount="232" Body="&lt;p&gt;I used hashsolo to demultiplex my data and I am a bit confused as to how to continue the analysis.&#xA;Following hashsolo my samples look like below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A1&#xA;AnnData object with n_obs × n_vars = 8684 × 18197&#xA;    obs: 'most_likely_hypothesis', 'Classification', 'n_genes'&#xA;    var: 'gene_ids', 'feature_types', 'genome', 'n_counts', 'n_cells'&#xA;    layers: 'counts'&#xA;&#xA;A2&#xA;AnnData object with n_obs × n_vars = 9901 × 20862&#xA;    obs: 'most_likely_hypothesis', 'Classification'&#xA;    var: 'gene_ids', 'feature_types', 'genome', 'n_counts'&#xA;    layers: 'counts'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have a few questions regarding the analysis. My initial plan was to merge the 2 samples. However, when I do that and I print the Classification the merged object seems to add the classification counts because the same hashtag names were used.&lt;/p&gt;&#xA;&lt;p&gt;For example hashtag 1 has 949 in sample A1 and 1286 in sample A2.&#xA;When I do print&lt;code&gt;(adata.obs['Classification'].value_counts())&lt;/code&gt; hashtag1 has 2235 counts. Will that be a problem with the analysis?&#xA;I merged the object like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# add some metadata&#xA;A1.obs['type']=&amp;quot;C&amp;quot;&#xA;A2.obs['type']=&amp;quot;T&amp;quot;&#xA;&#xA;# merge into one object.&#xA;adata = A1.concatenate(A2)&#xA;&#xA;adata&#xA;AnnData object with n_obs × n_vars = 18585 × 18064&#xA;    obs: 'most_likely_hypothesis', 'Classification', 'n_genes', 'type', 'batch'&#xA;    var: 'gene_ids', 'feature_types', 'genome', 'n_counts-0', 'n_cells-0', 'n_counts-1'&#xA;    layers: 'counts'&#xA;&#xA;print(adata.obs['Classification'].value_counts())&#xA;Hashtag_2_Antibody    5134&#xA;Hashtag_3_Antibody    3150&#xA;Hashtag_4_Antibody    2000&#xA;Hashtag_1_Antibody    2235&#xA;Doublet                                         1598&#xA;Negative                                           9&#xA;Name: Classification, dtype: int64&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My hashtags also show up wen I do &lt;code&gt;sc.pl.highest_expr_genes(adata, n_top=20, )&lt;/code&gt;. Is that a problem?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/D4C0p.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/D4C0p.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;I would also like to plot a UMAP following leiden for each hashtagged sample but I am a bit confused on how to do that. Normally, I would add information such as sample or type on metadata and color by sample for example. But if I try to color by classification it will merge the hashtags from both samples.&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="16902" LastEditorUserId="16902" LastEditDate="2023-04-25T20:48:10.707" LastActivityDate="2023-04-25T21:01:03.937" Title="How to analyze hashtagged data with scanpy?" Tags="|rna|umap|hashing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20937" PostTypeId="2" ParentId="20935" CreationDate="2023-04-25T21:01:03.937" Score="0" Body="&lt;p&gt;You could add something like &lt;code&gt;A1.obs[&amp;quot;Classification_obj&amp;quot;] = [c+&amp;quot;_A1&amp;quot; for c in adata.obs[&amp;quot;Classification&amp;quot;]]&lt;/code&gt; but the classification should also be clear in the merged adata object from the &amp;quot;Classification&amp;quot; and &amp;quot;batch&amp;quot; labels together&lt;/p&gt;&#xA;&lt;p&gt;More importantly, after running hashsolo, you must delete your hashtag feature_types from adata.var, something like &lt;code&gt;A1 = A1[:, [g for g in A1.var.index if g not in list_of_Ab_hashtags]]&lt;/code&gt; . Once you have the cell type classifications and are happy with them (can be worth doing further QC first), you can throw away that Ab count data for the downstream standard GEX analysis&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2023-04-25T21:01:03.937" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20938" PostTypeId="1" CreationDate="2023-04-26T04:48:23.167" Score="1" ViewCount="34" Body="&lt;p&gt;Dear Bioinformatics Community,&lt;/p&gt;&#xA;&lt;p&gt;I hope you're all doing well. I am currently working on a project that requires designing some 10 nucleotide single-stranded DNAs, (preferably), with specific criteria.&lt;/p&gt;&#xA;&lt;p&gt;The criteria for the DNA sequences are as follows:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;An equal mix of nucleotides (A, C, G, and T)&lt;/li&gt;&#xA;&lt;li&gt;Thermodynamically stable sequences&lt;/li&gt;&#xA;&lt;li&gt;Sequences that do not self-hybridize or hybridize with each other&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I have been exploring the NUPACK software suite to achieve these goals, but I am finding it challenging to use the tools effectively for this specific task. I understand that NUPACK is primarily designed for generating sequences based on target secondary structures, which may not directly align with my requirements.&lt;/p&gt;&#xA;&lt;p&gt;I would greatly appreciate any advice, suggestions, or alternative methods you can provide to help me design these DNA sequences. Are there any other tools or resources that might be better suited for this purpose?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance for your insights and expertise. Your guidance will be invaluable as I work to complete this project.&lt;/p&gt;&#xA;&lt;p&gt;Best regards,&lt;/p&gt;&#xA;" OwnerUserId="17376" LastEditorUserId="2108" LastEditDate="2024-02-02T20:45:20.767" LastActivityDate="2024-02-02T20:45:20.767" Title="Seeking Guidance on Designing DNA Sequences with Specific Criteria" Tags="|computational-biochemistry|design|dna|thermodynamics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20939" PostTypeId="1" CreationDate="2023-04-26T08:00:27.273" Score="1" ViewCount="103" Body="&lt;p&gt;I have a bam file of whole exome human data. I now want to check coverage of a list of genes in my data. Basically, I want to get output like, for example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Gene    Percentage of coding region covered&#xA;A1BG               100 %&#xA;A1BG-AS1           100%&#xA;A1CF                99.94%&#xA;NEK8                100%&#xA;FANCI              100%&#xA;A2ML1             100%&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to find out at which depth how much percentage of given genes are covered. for example AT 30X NEK8 gene covered 100%.&#xA;Interested in the canonical transcript and if canonical transcript is not present then we can consider other transcript in that case.&lt;/p&gt;&#xA;" OwnerUserId="17380" LastEditorUserId="17380" LastEditDate="2023-04-29T11:24:30.933" LastActivityDate="2023-04-29T11:24:30.933" Title="How can I calculate the coverage of a gene in exome sequencing / calculate coverage per gene?" Tags="|gene|coverage|exome|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20940" PostTypeId="1" CreationDate="2023-04-26T08:30:12.393" Score="0" ViewCount="13" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Restrict from EMBOSS Suit version 6.3.1 with the following&#xA;parameters: snucleotide1, sitelen = 4, rformat = table, enzymes =&#xA;enzymes.txt.&#xA;From the 4379 enzymes present in REBASE, we selected the 650 restriction enzymes that were commercially available, since this assay is meant to be used in any laboratory. From the 650 enzymes, 152 digest all P. salmonis sequences and only 65 recognized conserved restriction sites in the complete set of sequences, generating the same/similar restriction pattern (same number of bands and similar sizes).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fmicb.2016.00643/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.frontiersin.org/articles/10.3389/fmicb.2016.00643/full&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If my reading is correct, these researchers figured out the conservation of the restriction sites from the resulting virtual patterns.&lt;/p&gt;&#xA;&lt;p&gt;The coordinates of a cut can be different in two related sequences even if the restriction site is conserved. This is because base insertions/deletions could occur before/after the restriction site.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;How do i figure out if a recognition sequence is conserved or not, if the coordinates of the cut are not the same in the sequences? How do i automate this?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;We can consider this as an example:&lt;/p&gt;&#xA;&lt;p&gt;I have two sequences of the 16s rRNA gene from two strains of a bacterial species. When i run EMBOSS's restrict by the following command:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;restrict -sitelen 4 -enzymes HaeIII -snucleotide_sequence 1 -sequence&#xA;sequences.fasta -rformat excel -fragments true -outfile patterns.xls&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://pastebin.com/0ua0dw30&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sequences link&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The result is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aKv0I.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aKv0I.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17357" LastEditorUserId="17357" LastEditDate="2023-04-26T10:15:46.960" LastActivityDate="2023-04-26T10:15:46.960" Title="Automated method to figure out whether a recognition site is conserved or not using the results of simulated restriction digestion" Tags="|sequence-alignment|16rrna|emboss|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20942" PostTypeId="1" CreationDate="2023-04-26T20:06:34.393" Score="1" ViewCount="136" Body="&lt;p&gt;I am trying to use hashsolo and I want to make sure that I have done things correctly.&lt;/p&gt;&#xA;&lt;p&gt;I did the below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#import files&#xA;A= sc.read_10x_h5('filtered_feature_bc_matrix.h5', gex_only=False)&#xA;A.var_names_make_unique()&#xA;A.layers[&amp;quot;counts&amp;quot;] = A.X.copy()&#xA;sc.pp.filter_genes(A, min_counts=1)&#xA;&#xA;#hashsolo&#xA;hto=A[:,A.var_names.str.startswith('HTO')].copy()&#xA;hashsolo.hashsolo(hto)&#xA;hashsolo.plot_qc_checks_cell_hashing(cell_hashing_data) #nothing gets plotted&#xA;A.obs['most_likely_hypothesis'] = hto.obs['most_likely_hypothesis']&#xA;A.obs['Classification'] = hto.obs['Classification']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then since I have two samples for which I did hashsolo (A and B) I concatenated the objects and continued with the scanpy analysis like shown below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sc.pl.highest_expr_genes(A_B, n_top=20, )&#xA;sc.pp.filter_cells(A_B, min_genes=200)&#xA;sc.pp.filter_genes(A_B, min_cells=3)&#xA;print(A_B.n_obs, A_B.n_vars)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and I continued with the mitochondria QC and the rest of the analysis.&lt;/p&gt;&#xA;&lt;p&gt;Is that correct?&lt;/p&gt;&#xA;&lt;p&gt;I am wondering if I should have done it differently. Like shown below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A= sc.read_10x_h5('filtered_feature_bc_matrix.h5', gex_only=False)&#xA;A.var_names_make_unique() Then the &#xA;sc.pl.highest_expr_genes(A, n_top=20, )&#xA;sc.pp.filter_cells(A, min_genes=200)&#xA;sc.pp.filter_genes(A, min_cells=3)&#xA;print(A.n_obs, A.n_vars)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and following the QC (filtering and mitochondria) and make the hto list and use hashsolo and then concatenate the 2 samples (A and B).&lt;/p&gt;&#xA;" OwnerUserId="16902" LastActivityDate="2023-04-26T20:06:34.393" Title="How to use hashsolo for demultiplexing hashtags?" Tags="|rna-seq|python|hashing|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20944" PostTypeId="1" AcceptedAnswerId="20945" CreationDate="2023-04-28T06:44:08.813" Score="4" ViewCount="111" Body="&lt;p&gt;I am trying to do mapping with multiple sequences against a reference genome. I tried the following code: The outputfile is .fastq.sam. I need only .sam. How do I get it?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in test/*.fastq; &#xA;do&#xA; ./minimap2 -ax map-ont reference.fna &lt;span class=&quot;math-container&quot;&gt;$i &amp;gt; $&lt;/span&gt;i.sam;&#xA; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17391" LastEditorUserId="17391" LastEditDate="2023-04-28T06:59:32.283" LastActivityDate="2023-04-29T04:17:18.933" Title="mapping nucleotide sequences" Tags="|read-mapping|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20945" PostTypeId="2" ParentId="20944" CreationDate="2023-04-28T07:06:11.663" Score="5" Body="&lt;p&gt;The command is fine, it is just a suffix. If you want to trim the suffix then use &lt;code&gt;${i%.fastq}.sam&lt;/code&gt; for the output, or even better convert the output from minimap2 to bam directly as this saves space:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./minimap2 -ax map-ont reference.fna &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${i}&quot; | samtools view -o &quot;$&lt;/span&gt;{i%.fastq}&amp;quot;.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16676" LastEditorUserId="16676" LastEditDate="2023-04-28T11:53:23.417" LastActivityDate="2023-04-28T11:53:23.417" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20946" PostTypeId="1" CreationDate="2023-04-28T08:09:19.837" Score="0" ViewCount="31" Body="&lt;p&gt;How do you get a specific 100 SRA sequences from a Bioproject which contains 400 runs, when you don't need all of them?&lt;/p&gt;&#xA;" OwnerUserId="17392" LastActivityDate="2023-04-28T08:09:19.837" Title="How to access specific SRA sequences within an SRP file?" Tags="|linux|ncbi|sra|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20947" PostTypeId="1" CreationDate="2023-04-28T08:40:22.403" Score="0" ViewCount="35" Body="&lt;p&gt;In publications the GEO code is usually shared and not the BioProject/study code.&lt;/p&gt;&#xA;&lt;p&gt;I want to get the BioProject/Study Accession code of a GEO dataset (to later download the data from ENA).&lt;br /&gt;&#xA;I know that I can go to the website and see find it but I would like to automate the process.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/doi:10.18129/B9.bioc.GEOquery&quot; rel=&quot;nofollow noreferrer&quot;&gt;GEOquery&lt;/a&gt; didn't provide it, and the data doesn't seem to be available via &lt;a href=&quot;ftp://ftp.ncbi.nlm.nih.gov/geo/README.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;the FTP&lt;/a&gt;.&lt;br /&gt;&#xA;Is there a programmatic way to obtain it?&lt;/p&gt;&#xA;" OwnerUserId="48" LastActivityDate="2023-04-29T23:27:13.363" Title="Get BioProject code of a GEO dataset" Tags="|public-databases|data-download|geo|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20949" PostTypeId="1" AcceptedAnswerId="20962" CreationDate="2023-04-28T17:38:02.527" Score="3" ViewCount="93" Body="&lt;p&gt;I am trying to download raw sequence data from the SRA database using the SRA toolkit (v.3.0.2). Unfortunately, the authors of the article I am reading gave SRA numbers (i.e. with the SRA prefix, not the SRR). This does not seem to work with the prefetch command:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;prefetch --max-size 100000000 SRAxxxxxx -O downloads_SRAxxxxxx/&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I ended up going on NCBI and manually finding the relevant SRR number for each sample; however, I was wondering if I could have done it programmatically. I appreciate your help.&lt;/p&gt;&#xA;&lt;p&gt;Edit: Thank you for your comments, here are a couple of examples:&lt;/p&gt;&#xA;&lt;p&gt;SRA030738 ; SRA030736 ;&lt;/p&gt;&#xA;&lt;p&gt;Searching for them on SRA I found the SRR numbers, respectively:&lt;/p&gt;&#xA;&lt;p&gt;SRR135604 ; SRR135602.&lt;/p&gt;&#xA;&lt;p&gt;The paper is Hasan et al, 2012:  DOI: 10.1073/pnas.1207359109&lt;/p&gt;&#xA;" OwnerUserId="3536" LastEditorUserId="3536" LastEditDate="2023-05-01T14:28:55.790" LastActivityDate="2023-05-08T21:52:11.577" Title="SRA toolkit and SRA- prefix" Tags="|sratoolkit|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20950" PostTypeId="2" ParentId="20944" CreationDate="2023-04-28T22:35:18.483" Score="1" Body="&lt;p&gt;Honestly, I wouldn't do it with a one-liner like this.  Make a proper bash script, so you can spend a couple of lines to get the naming just right.  Then you can use the bash script over and over again.  Easier to document too if the command line used is in a file, not just in terminal history.&lt;/p&gt;&#xA;&lt;p&gt;Also, yes, piping to convert to bam is almost certainly what you want to do.  But piping into a sort commmand is better.  And I think samtools sort will output a bam by itself if you ask it to.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2023-04-28T22:35:18.483" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20951" PostTypeId="2" ParentId="20944" CreationDate="2023-04-29T04:17:18.933" Score="1" Body="&lt;p&gt;I would recommend&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;ls test/*.fastq|sed s,.fastq$,,|xargs -i echo ./minimap2 -ax map-ont reference.fna {}.fastq \&amp;gt; {}.sam|sh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Replace trailing &amp;quot;sh&amp;quot; with &lt;a href=&quot;https://www.gnu.org/software/parallel/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GNU parallel&lt;/a&gt; for concurrent jobs on the same node; replace with &lt;a href=&quot;https://github.com/lh3/asub&quot; rel=&quot;nofollow noreferrer&quot;&gt;asub&lt;/a&gt; for submitting array jobs to Slurm/LSF/SGE.&lt;/p&gt;&#xA;&lt;p&gt;Once you get used to this, you can do a variety of tasks quickly, like renaming files in batch, gzipping all files in parallel, format conversion in batch, aligning 100 files on a cluster, etc.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2023-04-29T04:17:18.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20952" PostTypeId="1" CreationDate="2023-04-29T14:19:08.173" Score="1" ViewCount="105" Body="&lt;p&gt;I want to check the correlation between the expression of a given &amp;quot;test_gene&amp;quot;* in each cluster within a scRNA-seq database with the expression of a signature (i.e., gene sets whose expression is known and validated to be a “marker” of a specific phenotype) in a specific cluster, that is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Corr.: ssGSEA of signature in cluster X vs test_gene in cluster A&lt;/li&gt;&#xA;&lt;li&gt;Corr.: ssGSEA of signature in cluster X vs test_gene in cluster B&lt;/li&gt;&#xA;&lt;li&gt;Corr.: ssGSEA of signature in cluster X vs test_gene in cluster C&lt;/li&gt;&#xA;&lt;li&gt;and so on...&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Since I don't have much experience in transcriptomic data analysis, I was wondering what would be the best method to achieve this. The database I'm using (public) contains data from 19 different samples (patients), so I was thinking of doing a &amp;quot;sample-wise&amp;quot; Spearman's test. See this dummy example to exemplify my point:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;test_gene in cluster_A&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;ssGSEA score of signature in cluster_X&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Sample_1&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;2.1&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;2.3&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Sample_2&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;1.63&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;1.76&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Sample_3&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;0.39&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;0.45&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;. . .&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;. . .&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;. . .&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Sample_19&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;0.95&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;0.81&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;test_gene in cluster_B&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;ssGSEA score of signature in cluster_X&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Sample_1&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;1.91&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;1.21&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Sample_2&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;0.03&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;1.85&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Sample_3&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;3.13&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;0.9&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;. . .&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;. . .&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;. . .&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Sample_19&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;0.09&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;2.12&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;And so on with clusters C, D, E, etc...&lt;/p&gt;&#xA;&lt;p&gt;In this example (not real data) you can tell that gene_a's expression in cluster_A looks like could be linearly correlated with the signature's expression in cluster_X (they go &amp;quot;hand in hand&amp;quot;), but the same is not true for cluster_B. Here, a Spearman's test should output a p-value lower than 0.05 (signif.) for &amp;quot;test_gene in cluster_A vs signature_in_X&amp;quot;, and higher than 0.05 (not signif.) for &amp;quot;test_gene in cluster_B vs signature_in_X&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;All the samples in the actual database are from treatment-naïve renal cell carcinoma (RCC) patients, all of them with the same sub-type. Concerning the test_gene, it is actually a &amp;quot;placeholder&amp;quot; for each of the 20+ genes I want to test for a correlation. The cluster_X here would actually be the malignant cell cluster, being all the other clusters (A, B, etc.) the many cell types in the tumor micro-environment (TME). As for the data points, there would be 19, one per sample, but in the example I only put 3 just to make the point. There is one sample per patient, so sample = patient.&lt;/p&gt;&#xA;&lt;p&gt;Are there any special considerations (like the type of normalization or data treatment) I should have to achieve this?&lt;/p&gt;&#xA;&lt;p&gt;*The average of the normalized expression for that gene in all cells from the same cluster and sample.&lt;/p&gt;&#xA;" OwnerUserId="17398" LastEditorUserId="17398" LastEditDate="2023-05-03T17:15:02.050" LastActivityDate="2023-05-03T17:15:02.050" Title="Correlate ssGSEA score with Seurat's scaled expression data" Tags="|scrnaseq|seurat|correlation|gsea|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20956" PostTypeId="1" CreationDate="2023-04-30T17:51:55.650" Score="2" ViewCount="27" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9562091/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am trying to set up &lt;a href=&quot;https://rega.kuleuven.be/cev/regadb&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;RegaDB&lt;/strong&gt;&lt;/a&gt; on my &lt;strong&gt;CentOS&lt;/strong&gt; system, and I have installed &lt;strong&gt;PostgreSQL&lt;/strong&gt; via &lt;code&gt;conda&lt;/code&gt;. I followed the &lt;strong&gt;RegaDB&lt;/strong&gt; installation manual to configure the &lt;strong&gt;PostgreSQL&lt;/strong&gt; database. However, when I have applied the &lt;em&gt;postgresSchema.sql&lt;/em&gt; schema from the &lt;strong&gt;RegaDB&lt;/strong&gt; software package, I noticed several &lt;code&gt;NOTICE&lt;/code&gt; messages about truncated identifiers.&lt;/p&gt;&#xA;&lt;p&gt;Here is a sample of the output I received:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;psql:postgresSchema.sql:25: NOTICE:  identifier &amp;quot;query_definition_parameter_type_query_definition_parameter_type_ii_seq&amp;quot; will be truncated to &amp;quot;query_definition_parameter_type_query_definition_parameter_type&amp;quot;&#xA;psql:postgresSchema.sql:27: NOTICE:  identifier &amp;quot;query_definition_run_parameter_query_definition_run_parameter_ii_seq&amp;quot; will be truncated to &amp;quot;query_definition_run_parameter_query_definition_run_parameter_i&amp;quot;&#xA;psql:postgresSchema.sql:29: NOTICE:  identifier &amp;quot;resistance_interpretation_template_resistance_interpretation_template_ii_seq&amp;quot; will be truncated to &amp;quot;resistance_interpretation_template_resistance_interpretation_te&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I understand that &lt;strong&gt;PostgreSQL&lt;/strong&gt; has a maximum identifier length of &lt;em&gt;63&lt;/em&gt; bytes, and identifiers exceeding this limit will be truncated. My question is whether these truncated identifiers will cause any issues with the proper functioning of &lt;strong&gt;RegaDB&lt;/strong&gt;. If so, what would be the best approach to fix this issue?&lt;/p&gt;&#xA;&lt;p&gt;I would appreciate any guidance on this matter, and if there are any additional precautions or testing measures I should take to ensure &lt;strong&gt;RegaDB&lt;/strong&gt; functions as expected.&lt;/p&gt;&#xA;&lt;p&gt;Best regards,&lt;/p&gt;&#xA;" OwnerUserId="15161" LastEditorUserId="73" LastEditDate="2023-04-30T21:15:19.713" LastActivityDate="2023-04-30T21:15:19.713" Title="Is it normal to get truncated identifiers after applying SQL scema of RegaDB?" Tags="|hiv|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20957" PostTypeId="1" CreationDate="2023-05-01T15:39:33.067" Score="1" ViewCount="65" Body="&lt;p&gt;The website &lt;a href=&quot;https://maayanlab.cloud/sigcom-lincs/#/SignatureSearch/UpDown&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://maayanlab.cloud/sigcom-lincs/#/SignatureSearch/UpDown&lt;/a&gt; can be used to &amp;quot;Identify reversers and mimickers from over 1 million signatures by entering up and down gene sets .. &amp;quot; I am looking to get these &amp;quot;reversers&amp;quot; and &amp;quot;mimickers&amp;quot;, aka chemical perturbagens.&lt;/p&gt;&#xA;&lt;p&gt;So on this website, I paste my list of a selection of Up-regulated genes, in their respective box, and the same for my list of down-regulated genes. These genes are from an RNA sequencing experiment.&lt;/p&gt;&#xA;&lt;p&gt;After the website has done its thing, I expand the search for chemical perturbagens (LINCS L1000 Chemical Perturbations (2021)). One of the data tables I can download contain something called a Gene Rank Matrix. This table consist of one row for each gene and columns of the chemical perturbagens found. The values are integers.&lt;/p&gt;&#xA;&lt;p&gt;A small subsection of the data table would look like this:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/DO322.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DO322.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;My questions are what these integers (12, 144, 411, 641) represent. Are they how much the perturbagens (In this case REP.B025_YAPC_24H_K01_resiquimod_2.22uM) will affect the genes?&lt;/p&gt;&#xA;&lt;p&gt;The paper describing this website searches (&lt;a href=&quot;https://academic.oup.com/nar/article/50/W1/W697/6582159&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://academic.oup.com/nar/article/50/W1/W697/6582159&lt;/a&gt;) mentions &amp;quot;The expression tables provide the computed CD differential gene expression coefficients for each signature, while the rank matrices provide the integer gene ranks for each signature as determined by the coefficients&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Matrices that represent the gene rankings of the L1000 signatures are stored as HashMaps by the data API in random access memory (RAM). HashMaps are data structures designed to efficiently search for key-value pairs, and this implementation drastically cuts the fetch times of the gene rankings by UUIDs, thus improving the speed of the search&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Does this mean that the scores found wont have any meaning in regards to how well the perturbagens found will affect the genes? Or can some meaning be derived from this?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="17424" LastEditorUserId="3848" LastEditDate="2023-05-03T16:55:32.790" LastActivityDate="2023-05-03T17:05:08.983" Title="Gene rank scores from LINCS database: Evangelista et al. 2022" Tags="|rna-seq|public-databases|machine-learning|drugs|deep-learning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20958" PostTypeId="1" CreationDate="2023-05-01T20:57:31.800" Score="1" ViewCount="60" Body="&lt;p&gt;I have a bunch of .pdb files that consist of protein-peptide structures. However, before start working with them I need to clean them. Now, I can easily remove water, ion, or other stuff like BMET, BASN, BARG info, and anisotropic temperature information. Now, if the PDB flies consist of more than one instance of protein-peptide then it will be hard for me to work with them. The chains of protein and peptide numbers are arbitrary. For example, in a PDB file, a protein may have 4 chains(A, B, C, D) of proteins and 4 peptide chains(E, F, G, H). In that case, it may have 4 instances and we can select A and E to get a single instance from it. But, in pdb file, protein may have 4 chains(A,B,C,D) of proteins and 2 peptide chains(E,F). In that case, it may have 2 instances.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Now the question is, how to select one instance of protein-peptide from the PDB files without doing manual curation?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="10296" LastActivityDate="2023-05-01T20:57:31.800" Title="Detect monomer from from multimer structure in PDB files?" Tags="|protein-structure|pdb|data-preprocessing|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20959" PostTypeId="1" CreationDate="2023-05-01T22:55:50.953" Score="0" ViewCount="57" Body="&lt;p&gt;I have a filtered VCF file that I want to subset for the 500 most differentiated SNPs based on estimated Fst values (calculated in vcftools). I am not completely sure how to approach this. I have calculated the fst values across all populations (10 in total), but am not sure how to specify that I only want 500 SNPs with the highest fst value.&lt;/p&gt;&#xA;&lt;p&gt;I used VCFTools, but would BCFTools be more appropriate?&lt;/p&gt;&#xA;" OwnerUserId="17177" LastEditorUserId="3694" LastEditDate="2023-05-02T14:30:11.900" LastActivityDate="2023-05-02T14:30:11.900" Title="Filter VCF file for most differentiated SNPs (Fst) in VCFTools" Tags="|snp|filtering|genetics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20960" PostTypeId="1" CreationDate="2023-05-02T06:38:09.173" Score="0" ViewCount="44" Body="&lt;p&gt;I want to export the list of soft-clipped regions into a csv file from a region of interest in IGV. Additional information regarding the soft-clipped sequence, coordinate and nucleotide length will be helpful.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/2lM8V.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2lM8V.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;PFB the output file format&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;Gene&lt;/th&gt;&#xA;&lt;th&gt;Variant&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;FLT3&lt;/td&gt;&#xA;&lt;td&gt;chr13:28034082C&amp;gt;CAAACTCTAAATTTTCTCTTGGAAACTCCCATTTGAGATCATATAG&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;FLT3&lt;/td&gt;&#xA;&lt;td&gt;chr13:28034113T&amp;gt;TTTGAGATCATATTCATATTCTCTGAAATCAACGTAGAAGTACTCATTATCTGAGGAGCCGGTCACCTGTACCACC&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;" OwnerUserId="17425" LastEditorUserId="17425" LastEditDate="2023-05-03T12:19:12.193" LastActivityDate="2023-05-03T12:19:12.193" Title="How to export the list of soft-clipped regions from a region of interest in IGV" Tags="|sam|igv|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20961" PostTypeId="1" CreationDate="2023-05-02T21:11:35.707" Score="1" ViewCount="55" Body="&lt;p&gt;I'm trying to simulate the evolution of binary traits on phylogenies in R to end up with a dataset of the traits held by each tip. Starting at the root, at every timestep, new traits are added with a certain probability, and existing traits are lost with another probability.  This happens independently for every branch that exists at that timestep before moving to the next timestep. Those traits are added/removed from the final trait vectors of each species that is descended from the current branch/timestep.&lt;/p&gt;&#xA;&lt;p&gt;The issue is I don't know how to identify which tips (final species) originate from which current branches (at a specific timestep).&lt;/p&gt;&#xA;&lt;p&gt;Is there a function for identifying, on a phylogeny, at a specific time, tips that share a current ancestor/tips that are descended from the currently selected internal branch?&lt;/p&gt;&#xA;&lt;p&gt;Or, do you have a better way of organizing the trait data through time?&lt;/p&gt;&#xA;&lt;p&gt;I've tried using other software (TraitLab in Matlab) but not sure that it does exactly what I want, which is why I'd rather code it in R. If you have other suggestions of software to use, that would also be much appreciated! Unfortunately most trait evolution simulation software require you to know the total number of traits from the start, whereas I want new traits to appear and be lost.&lt;/p&gt;&#xA;&lt;p&gt;I want to do something similar but different to getMRCA. Starting with one selected species, at a specific time in the phylogeny, I want to identify all other species from which it hasn't yet split.&lt;/p&gt;&#xA;&lt;p&gt;Traits are gained under this model. The way I planned it is that every species has a vector of traits, each position in the vector being for 1 unique trait (the positions indicate the same traits across all species). So when a trait is gained, a &amp;quot;1&amp;quot; is added to all vectors of species descended from that branch, and a &amp;quot;0&amp;quot; is added to all other species. When a trait is lost, the &amp;quot;1&amp;quot; for that trait in all species descended from that branch is replaced with a &amp;quot;0&amp;quot;. I'm using the ape library, I just can't figure out how to identify which species are descended from a specific branch.&lt;/p&gt;&#xA;&lt;p&gt;Do you know of a function that can do that?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="17428" LastEditorUserId="73" LastEditDate="2023-05-03T19:44:38.953" LastActivityDate="2023-05-03T19:44:38.953" Title="How to identify all descendents of a branch on a phylogeny" Tags="|r|phylogenetics|phylogeny|simulated-data|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20962" PostTypeId="2" ParentId="20949" CreationDate="2023-05-03T00:21:11.017" Score="2" Body="&lt;p&gt;Using &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Entrez Direct E-utils&lt;/a&gt;, run the following command for your two examples:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db SRA -query 'SRA030738 OR SRA030736' | efetch -format docsum | xtract -pattern DocumentSummary -element Runs | xtract -pattern Run -element Run@acc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(I know, it seems strange to run &lt;code&gt;xtract&lt;/code&gt; twice, but the formatting on the plain xml output is dodgy. The first pass parsed out the unwanted &lt;code&gt;&amp;amp;gt&lt;/code&gt; or &lt;code&gt;&amp;amp;lt&lt;/code&gt; or &lt;code&gt;&amp;amp;quot&lt;/code&gt; characters. The second pass worked on the re-formatted &lt;code&gt;stdout&lt;/code&gt;)&lt;/p&gt;&#xA;" OwnerUserId="3381" LastActivityDate="2023-05-03T00:21:11.017" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20963" PostTypeId="1" AcceptedAnswerId="20982" CreationDate="2023-05-03T01:00:41.783" Score="3" ViewCount="268" Body="&lt;p&gt;I would like to download a large .vcf file containing many (hundreds or thousands) of samples. Ideally, I would download different population-specific .vcf files, but the ability to sort/filter by ancestry group is fine. Where do I get such a file? I prefer GRCh37 for consistency with other files I'm using.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Update: I tried the HGDP file--I'm trying the 100 genomes one now, in addition to trying &lt;a href=&quot;http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000_genomes_project/release/20190312_biallelic_SNV_and_INDEL/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000_genomes_project/release/20190312_biallelic_SNV_and_INDEL/&lt;/a&gt; now.&lt;/p&gt;&#xA;&lt;p&gt;I downloaded the file, but when I opened it it seemed to have no samples (instead of having many samples). Here is a full row:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; 1  10145   .   AAC A   151.43  PASS    AC=4;AF=0.0127389;AN=314;BaseQRankSum=0.72;ClippingRankSum=0;DP=24144;ExcessHet=160.275;FS=0;InbreedingCoeff=-0.1558;MLEAC=5;MLEAF=0.002604;MQ=1.63;MQRankSum=0.764;NEGATIVE_TRAIN_SITE;QD=3.52;ReadPosRankSum=0.085;SOR=0.458;VQSLOD=-0.9391;culprit=QD;VQSRMODE=INDEL;NS=157;ExcHet=0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/9562337/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="17430" LastEditorUserId="17430" LastEditDate="2023-05-06T06:28:59.937" LastActivityDate="2023-06-11T16:00:25.050" Title="Where do I get a large reference VCF?" Tags="|vcf|genome|1000genomes|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20964" PostTypeId="1" CreationDate="2023-05-03T11:29:05.287" Score="3" ViewCount="328" Body="&lt;p&gt;Not going to lie, my head hurts when I read the NextFlow documentation (No dig on the documentation, I think my head is losing its processing capacity lol).&lt;/p&gt;&#xA;&lt;p&gt;I had written a pipeline in Bash, essentially iterates over a text file containing full paths of R1 and R2 reads, followed by a series of commands and liberal use of gnu parallel to improve efficiency.&lt;/p&gt;&#xA;&lt;p&gt;Is there any template I can follow, where I can slot in my commands and get things running quickly?&lt;/p&gt;&#xA;&lt;p&gt;EDIT - The skeleton of pipeline goes as follows (As I'm not sure if I can put our inhouse pipeline in its entirety here) -&lt;/p&gt;&#xA;&lt;p&gt;Bash code which loops through R1 and R2, aligns with BWA-mem2 and sorts the bam, then using gnu parallel, I essentially follow the steps as outlined in GATK Best Practices (looking to drop BQSR, but the steps remain) to obtain a gvcf file. The steps in Best Practices have been turned to functions, rather than each step being called sequentially.&lt;/p&gt;&#xA;" OwnerUserId="6561" LastEditorUserId="73" LastEditDate="2023-05-03T19:32:53.580" LastActivityDate="2023-05-30T07:10:57.053" Title="Easier / lazy way to convert an existing bash pipeline to nextflow pipeline?" Tags="|bash|nextflow|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20965" PostTypeId="1" AcceptedAnswerId="20991" CreationDate="2023-05-03T14:48:46.987" Score="1" ViewCount="42" Body="&lt;p&gt;I am analysing some WGS data in MEGAN and would like to do some additional analysis in Python/R&lt;/p&gt;&#xA;&lt;p&gt;I am having trouble Tidying the Taxonomic data in a format which would be conducive to this. Originally I tried exporting Taxon Path to count and using the Split function in Pandas to break up taxonomy into new columns as this worked well for KEGG Pathways. Due to varying specificity of taxonomic assignments this didn't work.&lt;/p&gt;&#xA;&lt;p&gt;I've had more luck Exporting Data as a BIOM file format and converting it to TSV resulting in a file with the following column containing all taxonomic info (See below)&lt;/p&gt;&#xA;&lt;p&gt;Ideally , I'd like to generate columns for; Domain, Phylum , Class , Order, Family and Species by accessing the  features contained in Taxonomy. I'm unaware of any functions which would let me do that.&lt;/p&gt;&#xA;&lt;p&gt;an idea of what I'm trying to achived is&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;Domain&lt;/th&gt;&#xA;&lt;th&gt;Phylum&lt;/th&gt;&#xA;&lt;th&gt;Class&lt;/th&gt;&#xA;&lt;th&gt;Order&lt;/th&gt;&#xA;&lt;th&gt;Family&lt;/th&gt;&#xA;&lt;th&gt;Species&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Bacteria&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;td&gt;uncultured bacterium&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Bacteria&lt;/td&gt;&#xA;&lt;td&gt;Bacteroidetes&lt;/td&gt;&#xA;&lt;td&gt;Bacteroidia&lt;/td&gt;&#xA;&lt;td&gt;Bacteroidaceae&lt;/td&gt;&#xA;&lt;td&gt;Bacteroides&lt;/td&gt;&#xA;&lt;td&gt;Bacteroides acidifaciens&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;From&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;Taxonomy&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; s__uncultured bacterium&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides acidifaciens&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides caccae&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides cellulosilyticus&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides faecis&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides finegoldii&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides fragilis&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides intestinalis&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae;&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae;&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides salyersiae&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides stercoris&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides thetaiotaomicron&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides uniformis&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides xylanisolvens&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides cellulosilyticus CAG:158&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__uncultured Bacteroides sp.&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides sp. 2_2_4&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides sp. 3_1_23&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d__Bacteria; p__Bacteroidetes; c__Bacteroidia; o__Bacteroidales; f__Bacteroidaceae; g__Bacteroides; s__Bacteroides sp. D2&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;" OwnerUserId="15713" LastEditorUserId="15713" LastEditDate="2023-05-03T15:25:00.393" LastActivityDate="2023-05-09T03:08:42.953" Title="Tidying MEGAN Taxonomy for Python/R Analysis" Tags="|r|python|pandas|dataframe|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20966" PostTypeId="2" ParentId="20957" CreationDate="2023-05-03T15:39:30.677" Score="1" Body="&lt;p&gt;&lt;strong&gt;Summary answer&lt;/strong&gt;&#xA;Although I'm unhappy about how the paper is written and possibly the methodology, the following sentence is likely (almost certainly) to be the answer.&lt;/p&gt;&#xA;&lt;p&gt;The coefficient (aka weight) is how important that gene is for the classification of that signature. Thats it! :-)&lt;/p&gt;&#xA;&lt;p&gt;Essentially, there are loads of genes involved in identifying which genes represents which signature. Not all those genes are equally informative. The positive or negative skews in that calculation, called a &amp;quot;classification&amp;quot;, are identified through their weights (aka coefficients). Thats simply the answer. The bigger the coefficient the more important that gene is in the classification, i.e. its associated with the signature.&lt;/p&gt;&#xA;&lt;p&gt;There is no need to be more complicated than that. If you are interested in a critique of that paper, which I had to do to get to the answer and thereby understand the analytics of that paper, this is presented below.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Critique of &lt;a href=&quot;https://academic.oup.com/nar/article/50/W1/W697/6582159&quot; rel=&quot;nofollow noreferrer&quot;&gt;Evangelista &lt;em&gt;et al&lt;/em&gt;. 2022: Nucleic Acids 50:W697–W709&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note 1&lt;/strong&gt; I don't have time to read the paper in absolute detail, particularly the appendices, where some of the information needed may have be placed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note 2&lt;/strong&gt; In the answer below, a &amp;quot;expression coefficient&amp;quot; is what I've called an &amp;quot;expression weight&amp;quot; its the same thing.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Weightings and analytical concerns&lt;/strong&gt; I do understand weightings in deep learning, particularly machine learning. I read the paper and the ANN (artificial neural network) was very basic. I don't believe it had multiple layers, because it would have said so and it was a &lt;strong&gt;very odd&lt;/strong&gt; description for a deep learning algorithm. It would also be odd to have a single layer in deep learning, &lt;em&gt;very odd&lt;/em&gt;. The loss function is also not described at all which is &lt;em&gt;extremely important&lt;/em&gt; in deep learning (it will not work otherwise). Essentially, what the authors call &amp;quot;deep learning&amp;quot; appears to a machine learning neural network using an Adams gradient descent.  Adams gradient descent is a standard approach in deep-learning admittedly. Thereafter in the paper the details were very sketchy, maybe they are in the appendix (should be checked). Any deep-learning, machine-learning requires a demonstration of the robustness of the training and this appears absent. The two basic stats are accuracy and AUC-ROC, which were definitely absent. Universal stats are used to quickly understand how good the algorithm is in comparison to other deep-,machine- learning algorithms for other data sets.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Analytical overview&lt;/strong&gt; Generally I would assume the authors have performed a supervised learning algorithm, but this may not be correct (below). If it is supervised learning they have performed they would have identified their 'signatures' against a very large data set, then trained the different types of signature against that data and tested it. The learning could be terrible - but it's just not stated at all. I'm surprised because Nucleic Acids is at the pinnacle of algorithm publications.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary.&lt;/strong&gt; The authors performed deep-learning (which looked like a neural network machine learning algorithm). The description is at best poor, but further information could be present in the appendix. Therefore its not absolutely clear what a &amp;quot;weight&amp;quot; would represent because there are two possibilities.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Weights possibility: unsupervised learning&lt;/strong&gt; The one thing that might have happened is they've performed an unsupervised learning algorithm and the weights are a sort of &amp;quot;maximum variance&amp;quot;. This very trendy. That would explain the lack of accuracy and AUC-ROC type stuff. &lt;strong&gt;IF&lt;/strong&gt; this is true and I don't believe it's explained - I could be wrong - then the weights represent &lt;strong&gt;maximum information&lt;/strong&gt;. What does that mean? Good question, its context specific, its the gene which is most important in sectioning up the data - e.g. housekeeping gene which is required 'all over the place', not just part of the cell cycle is an example. Basically, in unsupervised weighted output high ranking weights are a mix of really boring and really brilliant stuff. So you just go through it with biological understanding, is this gene boring yeah/neigh, is this one - wait wow! unexpected ... That is actually fashionable analytics :-)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Weights possibility: supervised learning&lt;/strong&gt; Is already answered at the beginning. This is almost certainly the correct answer - it would be the orthodox approach - and we just accept the robustness checks for the algorithm and possibly the algorithm itself were simply insufficient.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Why isn't the paper clear (at best)?&lt;/strong&gt; What I assume has happened is the team have brilliant frontend and genomics team backend. The analyst in the middle is unlikely to be a specialist and thats why it's poorly described, or possibly performed. I assume if the reviewers were of similar composition then that's how its got through Nucleic Acids. Thus, I would personally treat the output with caution because deep learning (machine-learning) will be the essential bottleneck between their backend development and your data. Maybe &amp;quot;bottleneck&amp;quot; is not quite an accurate description, because its speeding up the classification, so maybe a &amp;quot;gate-keeper&amp;quot; is a better description.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-05-03T17:05:08.983" LastActivityDate="2023-05-03T17:05:08.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20967" PostTypeId="2" ParentId="20964" CreationDate="2023-05-03T17:14:06.207" Score="2" Body="&lt;p&gt;The gist would be something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fastqs = Channel.of(...some annoying groovy code to make this a list of tuples...)&#xA;workflow{&#xA;  align(fastqs)&#xA;  makeGVCF(align.out.bams)&#xA;  ... collect() the output from above joint call them all...&#xA;}&#xA;&#xA;process align {&#xA;  input:&#xA;    tuple val(sampleName), path(reads)&#xA;  output:&#xA;    tuple val(sampleName), path(&amp;quot;${sampleName}.bam&amp;quot;), emit: bams&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  bwa ... | samtools sort ...&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process makeGVCF{&#xA;  input:&#xA;    tuple val(sampleName), path(bam)&#xA;  output:&#xA;    path &amp;quot;${sampleName}.gvcf&amp;quot;, emit: gvcfs&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  ...haplotype caller code...&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="77" LastActivityDate="2023-05-03T17:14:06.207" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20968" PostTypeId="1" CreationDate="2023-05-04T12:16:09.437" Score="2" ViewCount="135" Body="&lt;p&gt;Familial defective apolipoprotein B-100 (FDB) is an autosomal dominant disorder associated with increased plasma cholesterol concentration and may thus increase the risk of premature coronary heart disease. To date, three apo B mutations, apo B R3500Q, apo B R3531C and apo B R3500W that cause defective apo B, have been reported.&lt;/p&gt;&#xA;&lt;p&gt;I have my raw DNA data from 23andMe (where each line corresponds to a single SNP). How can I search it for those related Apo B gene mutations?&lt;/p&gt;&#xA;&lt;p&gt;A single line in my DNA data looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rs12567639 1 756268 AA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have a programming background, but none in the bioinformatics area.&lt;/p&gt;&#xA;" OwnerUserId="17435" LastEditorUserId="73" LastEditDate="2023-05-04T20:30:15.307" LastActivityDate="2023-06-07T09:46:52.217" Title="How can I detect the Apo B gene mutations in my raw dna data?" Tags="|snp|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20970" PostTypeId="1" CreationDate="2023-05-04T19:36:03.933" Score="1" ViewCount="470" Body="&lt;p&gt;For some of our snRNA-seq samples we are finding low fraction of reads detected in single-nuclei rna-seq samples from cellranger, while the other metrics are perfect.&lt;/p&gt;&#xA;&lt;p&gt;While I understand this could be caused by library preparation and abundance of ambient RNA. Just wondering if this behavior is expected for single-nuclei datasets. My confusion is specifically coming from the fact that 10X website also has such a sample that has low number of fraction reads in cells.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://cf.10xgenomics.com/samples/cell-exp/7.0.0/5k_mouse_brain_CNIK_3pv3/5k_mouse_brain_CNIK_3pv3_web_summary.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cf.10xgenomics.com/samples/cell-exp/7.0.0/5k_mouse_brain_CNIK_3pv3/5k_mouse_brain_CNIK_3pv3_web_summary.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Would love to know if this behavior is expected or seen often who are more experienced in snRNA-seq. The metric documentation page is &lt;a href=&quot;https://kb.10xgenomics.com/hc/en-us/articles/360003919491-How-to-interpret-the-Fraction-Reads-in-Cells-metric-&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is snapshot from 10X report&#xA;&lt;a href=&quot;https://i.stack.imgur.com/VPMGY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VPMGY.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2025" LastEditorUserId="2025" LastEditDate="2023-05-06T16:55:02.560" LastActivityDate="2023-05-07T18:15:42.207" Title="Low fraction reads in cell metric in snRNA-seq data" Tags="|scrnaseq|transcriptome|cellranger|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20971" PostTypeId="1" CreationDate="2023-05-04T19:37:13.833" Score="4" ViewCount="49" Body="&lt;p&gt;I've generated a quant seq data that I intend to use to compare male and female gene expresion, with a focus on sexual chromosome.&lt;/p&gt;&#xA;&lt;p&gt;For my species (three-spined stickleback), it is a classic XY sex determination system, and i have a male and a female reference transcriptome, including or not the Y chromosome.&lt;/p&gt;&#xA;&lt;p&gt;I am not sure how to handle that with SALMON, and I have 2 questions in particular:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Given that I use two different transcriptomes depending of the sex, can I still compare the obtained normalized pseudocount of would it affect the estimation too much?&lt;/li&gt;&#xA;&lt;li&gt;I want to compare expression levels of genes present in both the X and Y reference (i.e genes that are not recombining but have not been lost or degenerated on the Y). For that, I'd need to add the gene counts of the Y and X copy. Is it Okay to add  pseudo count from different gene copies, or will that be issue? I do not understand the pseudo-count statistical model enough to have an idea about that&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My other option is to go with a STAR / HTseq pipeline, but my mapping rates seems about 20% lower with this tools.&lt;/p&gt;&#xA;" OwnerUserId="17437" LastEditorUserId="17437" LastEditDate="2023-05-05T15:17:28.063" LastActivityDate="2023-05-05T15:17:28.063" Title="Salmon Pseudo count when dealing with male and female RNA-seq data" Tags="|rna-seq|salmon|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20972" PostTypeId="1" CreationDate="2023-05-04T20:37:04.453" Score="3" ViewCount="46" Body="&lt;p&gt;I am working with preprocessed data (IsoSeq PacBio) and I cannot understand one of the steps:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -b -e [rq] &amp;gt;= 0.9 flnc.bam &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What exactly was filtered from the bam file?&lt;/p&gt;&#xA;&lt;p&gt;To obtain flnc they used &lt;a href=&quot;https://isoseq.how/getting-started.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;the standard IsoSeq pipeline&lt;/a&gt; (lima -&amp;gt; isoseq3 refine).&lt;/p&gt;&#xA;" OwnerUserId="17438" LastEditorUserId="298" LastEditDate="2023-05-05T10:00:01.190" LastActivityDate="2023-05-05T10:00:01.190" Title="regex: samtools command to &quot;refine&quot; PacBio IsoSeq data?" Tags="|samtools|pacbio|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20973" PostTypeId="2" ParentId="20931" CreationDate="2023-05-04T21:16:49.017" Score="2" Body="&lt;p&gt;Try profiling (benchmarking) your code to work out where it is slow.&lt;/p&gt;&#xA;&lt;p&gt;My other suggestions are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;replace your simple Pure Python hamming_distance function with something optimised like RapidFuzz - see &lt;a href=&quot;https://github.com/maxbachmann/RapidFuzz&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/maxbachmann/RapidFuzz&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;cache the corrections you are making - you have tonnes of RAM so a building a dictionary mapping the fragments with N to the corrected fragment ought to be possible. And then for each new sequence, check if you already solved it and if so reuse the same choice.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="793" LastActivityDate="2023-05-04T21:16:49.017" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20974" PostTypeId="1" CreationDate="2023-05-04T22:29:25.340" Score="2" ViewCount="22" Body="&lt;p&gt;I made an R function to grab rsID genotypes from dbSNP using the entrez API. For example, here is the xml output that I am trying to parse:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=snp&amp;amp;id=rs2656176&amp;amp;retmode=xml&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi?db=snp&amp;amp;id=rs2656176&amp;amp;retmode=xml&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;the 'SPDI' tag contains the genotype info. so I can easily use &lt;code&gt;xpathSApply&lt;/code&gt; to get this tag for many IDs. But for this dbSNP there are two different values, T&amp;gt;C and T&amp;gt;A.&lt;/p&gt;&#xA;&lt;p&gt;The T&amp;gt;C change is the one that has more support in all pop freq databases, and this is easily grabbed from the dbSNP webpage:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/rs2656176&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/snp/rs2656176&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The &amp;quot;Alt Allele&amp;quot; frequencies show A being nearly 0, while C is the major alt allele.&lt;/p&gt;&#xA;&lt;p&gt;However, I don't understand how I can parse the data from the XML from above using this... the 'global maf' tag in the XML shows some MAF values, but the 1000G value is only for the reference allele, not the alt.. There are other dbs that show the alt &amp;quot;C&amp;quot; allele but that might not always be the case.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have an idea of what tags I should look at? Or is there a better way to hit a different API to get this data instead?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="4174" LastActivityDate="2023-05-04T22:29:25.340" Title="How do you go about grabbing proper genotyping infomation from dbSNP using entrez API?" Tags="|snp|ncbi|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20975" PostTypeId="2" ParentId="20972" CreationDate="2023-05-05T01:32:53.523" Score="2" Body="&lt;p&gt;The &lt;a href=&quot;http://www.htslib.org/doc/samtools-view.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;samtools view&lt;/code&gt;&lt;/a&gt; command you've provided just uses a &lt;a href=&quot;http://www.htslib.org/doc/samtools.html#FILTER_EXPRESSIONS&quot; rel=&quot;nofollow noreferrer&quot;&gt;filter expression&lt;/a&gt; to discard records that do not meet the specified expression criteria. In this case, the expression is looking for alignments where the value of the 'rq' (read quality) tag is greater or equal to 0.9. The &lt;a href=&quot;https://isoseq.how/isoseq-tags.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;isoseq tags&lt;/a&gt; documentation says that the &lt;em&gt;read quality&lt;/em&gt; of the alignment is really the &lt;em&gt;predicted accuracy of the polished isoform&lt;/em&gt;. Note that the command produces BAM output, so be sure to redirect stdout or write to a file for example using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -b -e '[rq] &amp;gt;= 0.9' -o out.bam flnc.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-05-05T01:32:53.523" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20976" PostTypeId="1" CreationDate="2023-05-05T12:33:17.523" Score="1" ViewCount="173" Body="&lt;p&gt;An error occurred while I was running the program maxquant (2.0.3):&lt;/p&gt;&#xA;&lt;p&gt;Reading search engine results (SP)&lt;/p&gt;&#xA;&lt;p&gt;Unhandled Exception:&#xA;System.Exception: Exception during execution of external process: 1228404&#xA;Unhandled Exception:&#xA;System.IndexOutOfRangeException: Index was outside the bounds of the array.&#xA;at MsLib.Search.AndromedaPeptide.FromString (System.String line, BaseLibS.Ms.Data.Protein.ProteinSet proteinSet) [0x000a9] in :0&#xA;at MsLib.Search.AndromedaQuerySp.Add (System.Int32 count, System.String line, BaseLibS.Ms.Data.Protein.ProteinSet proteinSet, System.Int32 minLength) [0x00001] in :0&lt;/p&gt;&#xA;&lt;p&gt;[ERROR] FATAL UNHANDLED EXCEPTION: System.IndexOutOfRangeException: Index was outside the bounds of the array.&#xA;at MsLib.Search.AndromedaPeptide.FromString (System.String line, BaseLibS.Ms.Data.Protein.ProteinSet proteinSet) [0x000a9] in :0&lt;/p&gt;&#xA;&lt;p&gt;how can i fix it, thank your very much!&lt;/p&gt;&#xA;" OwnerUserId="17442" LastActivityDate="2023-06-07T12:06:05.723" Title="maxquant error: Unhandled Exception: System.IndexOutOfRangeException: Index was outside the bounds of the array" Tags="|maxquant|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20977" PostTypeId="2" ParentId="20964" CreationDate="2023-05-05T18:58:02.023" Score="2" Body="&lt;p&gt;Since you asked about templates, if your task is something rather generic, there is a good chance that a module, or several, that you can use to build your workflow exists in &lt;a href=&quot;https://nf-co.re/modules&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nf-co.re/modules&lt;/a&gt;&#xA;Generally I would recommend taking a look at the source of nf-core pipelines (e.g. &lt;a href=&quot;https://nf-co.re/sarek&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nf-co.re/sarek&lt;/a&gt; for variant calling) to get an idea how to get your samplesheets in.&lt;/p&gt;&#xA;" OwnerUserId="17280" LastEditorUserId="17280" LastEditDate="2023-05-05T19:06:12.770" LastActivityDate="2023-05-05T19:06:12.770" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20978" PostTypeId="2" ParentId="20968" CreationDate="2023-05-05T21:47:58.210" Score="3" Body="&lt;p&gt;I think you mean these.  The amino acid numbering has changed, but that happens as we discover more transcript variants.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://useast.ensembl.org/Homo_sapiens/Variation/Mappings?db=core;r=2:21005788-21006788;v=rs5742904;vdb=variation;vf=184237596&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://useast.ensembl.org/Homo_sapiens/Variation/Mappings?db=core;r=2:21005788-21006788;v=rs5742904;vdb=variation;vf=184237596&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://useast.ensembl.org/Homo_sapiens/Variation/Mappings?db=core;r=2:21005696-21006696;v=rs12713559;vdb=variation;vf=186851457&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://useast.ensembl.org/Homo_sapiens/Variation/Mappings?db=core;r=2:21005696-21006696;v=rs12713559;vdb=variation;vf=186851457&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://useast.ensembl.org/Homo_sapiens/Variation/Explore?db=core;g=ENSG00000084674;r=2:21001429-21044073;v=rs144467873;vdb=variation;vf=251617275&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://useast.ensembl.org/Homo_sapiens/Variation/Explore?db=core;g=ENSG00000084674;r=2:21001429-21044073;v=rs144467873;vdb=variation;vf=251617275&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2023-05-05T21:47:58.210" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20981" PostTypeId="1" CreationDate="2023-05-06T06:12:14.330" Score="1" ViewCount="8" Body="&lt;p&gt;I want to wrap galaxy into a docker image, and I want to delete files that are older than a certain date on a regular basis, so I want to know how to generate the files and where to store them.&lt;/p&gt;&#xA;" OwnerUserId="17353" LastActivityDate="2023-05-06T06:12:14.330" Title="Where is the galaxy results file path?" Tags="|galaxy|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20982" PostTypeId="2" ParentId="20963" CreationDate="2023-05-06T06:29:59.973" Score="1" Body="&lt;p&gt;Solution: Use &lt;a href=&quot;https://bochet.gcc.biostat.washington.edu/beagle/1000_Genomes_phase3_v5a/b37.vcf/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bochet.gcc.biostat.washington.edu/beagle/1000_Genomes_phase3_v5a/b37.vcf/&lt;/a&gt;, then to get the ancestry information (i.e., the ancestry group of each sample, e.g., &amp;quot;HG00566&amp;quot; or &amp;quot;NA19758&amp;quot;) use &lt;a href=&quot;https://www.internationalgenome.org/data-portal/sample&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.internationalgenome.org/data-portal/sample&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17430" LastActivityDate="2023-05-06T06:29:59.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20983" PostTypeId="1" CreationDate="2023-05-06T07:22:00.240" Score="1" ViewCount="185" Body="&lt;p&gt;I would like to find the mapping statistics from the sorted bam files.&#xA;Samtools flagstat gives the output only for a single file. What is the easiest way to find the mapping statistics for all sequences? How do I find the mapping percentage for all sequences? Is it possible to use the samtools flagstat if I merge all the files in to a single bamfile?&lt;/p&gt;&#xA;" OwnerUserId="17452" LastEditorUserId="17452" LastEditDate="2023-05-07T06:29:14.827" LastActivityDate="2023-05-07T06:29:14.827" Title="Mapping statistics from the bam files" Tags="|sequence-alignment|samtools|read-mapping|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="20984" PostTypeId="1" CreationDate="2023-05-07T05:12:02.840" Score="1" ViewCount="124" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9562084/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am running CibersortX in high-resolution mode (which yields estimates of gene values per sample).&lt;/p&gt;&#xA;&lt;p&gt;After that, I want to perform DE between two conditions on the resulting gene estimates.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;What statistical test would one need to apply to perform DE after high-resolution mode de-convolution?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If I use the output of DE for ranking purposes only (in pre-ranked GSEA), does it make sense to use a simple t-test?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The data is in TPM (CibersortX received TPMs as its input). I transform to log2 scale before t-test. There are only 7 samples available (4 control, 3 treatment)&lt;/p&gt;&#xA;&lt;p&gt;Here are some genes which get the smallest p-value after t-test (but do not pass FDR)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ETFBKMT 0.08727244  0.2549699   0.1277664   0.000000    0.597810    0.4984627   0.4672444&#xA;AFF3    3.28551359  2.9969676   2.8793547   3.013827    2.483018    2.5070328   2.3119873&#xA;PDLIM1  4.26118568  4.0109785   3.9223976   3.856899    3.388990    3.5155023   3.2931998&#xA;MYOF    6.17754486  5.9286025   5.7452668   5.845756    5.276884    5.3981318   5.1251924&#xA;LINC02210 3.22922278    3.1785117   3.5162067   3.307364    2.736807    2.9091521   2.8052582&#xA;CD274   5.20308488  5.0063801   4.8715090   4.922224    4.440247    4.6192961   4.5082217&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And these are the values before log&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ETFBKMT 1.062360    1.193311    1.092601    1.000000    1.513417    1.412707    1.382466&#xA;AFF3    9.750753    7.983202    7.358209    8.077041    5.590659    5.684498    4.965666&#xA;PDLIM1  19.175412   16.122220   15.162099   14.489126   10.475814   11.435934   9.802840&#xA;MYOF    72.381287   60.909803   53.641094   57.510614   38.770421   42.169610   34.900902&#xA;LINC02210 9.377626  9.053727    11.441519   9.899558    6.665934    7.511766    6.989834&#xA;CD274   36.837031   32.141830   29.273209   30.320557   21.709387   24.578008   22.756735&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And that is the a distribution of the expression values (after deconvolution) before log2, in one of the samples&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; Min.   :    1.00   &#xA; 1st Qu.:    5.01 &#xA; Median :   18.03&#xA; Mean   :   89.08 &#xA; 3rd Qu.:   47.82 &#xA; Max.   :34406.87  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6317" LastEditorUserId="6317" LastEditDate="2023-05-11T04:10:46.217" LastActivityDate="2024-02-05T06:01:30.157" Title="What statistical test to apply for DE after CibersortX deconvolution?" Tags="|rna-seq|differential-expression|deconvolution|t-test|cibersortx|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20985" PostTypeId="2" ParentId="20970" CreationDate="2023-05-07T18:15:42.207" Score="1" Body="&lt;p&gt;Yes, &amp;quot;low fraction reads in cells&amp;quot; is more common with snRNA-seq from frozen samples, I wouldn't worry about it by itself (you're just throwing away some sequencing data). You should still check though that the other QC metrics are good. In particular, sometimes such samples have an unclear seperation of cells from empty droplets on the UMI rank plot and the consequent estimated cell count is off, requiring alternative approaches to distinguish cellular droplets from ambient RNA etc.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2023-05-07T18:15:42.207" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20986" PostTypeId="2" ParentId="20984" CreationDate="2023-05-07T21:53:12.257" Score="0" Body="&lt;p&gt;To specifically answer the question - a T-test is a parametric test which requires a normal distribution. So it's not a good idea in context to this data. However, the non-parametric equivalent of the T-test is the Wilcoxon and thats absolutely perfect and described below together with the entire rationale for parametrics versus non-parametrics. Basically, I agree with your approach, except I would simply shift to non-parametric statistics.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I noted the response on Biostars from the link recommends &lt;strong&gt;DSeq2&lt;/strong&gt; and &lt;strong&gt;EdgeR&lt;/strong&gt;. Both tests along with the additional test described below, along with the T-test, are used to identify whether each gene in the analysis is significantly up-, or down-regulated and can then be ranked. The underlying issue is the validity of the tests associated &lt;em&gt;P-values&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Both DSeq2 and EdgeR have been notably criticised by &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-022-02648-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Li &lt;em&gt;et al&lt;/em&gt; 2022&lt;/a&gt;, in a paper entitled&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Li &lt;em&gt;et al&lt;/em&gt; 2022 &lt;em&gt;Exaggerated false positives by popular differential expression methods when analyzing human population samples&lt;/em&gt;&amp;quot; Genome Biology: 79&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Li &lt;em&gt;et al&lt;/em&gt; (2022) needs to be read carefully before making a choice. Li &lt;em&gt;et al&lt;/em&gt; 2022 is recommending using a non-parametric test, via Wilcoxon, which is implemented in the method &lt;strong&gt;NOIseq&lt;/strong&gt; from the R-Cran Bioc library (described below). Genome Biology has a current impact factor of 18 (exceptional). I was told that one member of the DSeq2 team produced a later paper advocating non-parametrics methods.&lt;/p&gt;&#xA;&lt;p&gt;The underlying issue is the dependency on a parametric distribution for both DSeq2 and EdgeR. Basically, the central issue is the validity of parametric statistics (assumed distribution) versus non-parametrics statics (no assumptions about the distribution).&lt;/p&gt;&#xA;&lt;p&gt;In parametric statistics outside DE, i.e. &lt;em&gt;in general&lt;/em&gt;, when these are used numerous tests are performed to assess the fit of the data set to the parametric distribution assumed. For example, for ANOVA it's the Fmax test. For a T-test there's a number of tests for the normal distribution. If the data set fails the test, a multitude of transformations are possible usually aimed at normalising the variance.&lt;/p&gt;&#xA;&lt;p&gt;From this perspective the concern is no comparative tests were devised for DSeq2 or EdgeR given the age of the papers DSeq2 is from 2014 &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-014-0550-8&quot; rel=&quot;nofollow noreferrer&quot;&gt;Love &lt;em&gt;et al&lt;/em&gt; (2014)&lt;/a&gt;. If there's no test how could the data use alternative transformations to fit the assumptions? Again this is unusual given the age of papers because such as tests would be expected.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Non-parameteric DEG&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The non-parametric method mentioned, &lt;strong&gt;NOISeq&lt;/strong&gt;, is from the R/ &lt;strong&gt;Bioc&lt;/strong&gt; package by &lt;a href=&quot;https://academic.oup.com/nar/article/43/21/e140/2468096&quot; rel=&quot;nofollow noreferrer&quot;&gt;Tarazona &lt;em&gt;et al&lt;/em&gt; 2015, Nucleic Acids:43&lt;/a&gt;. This was the approach use by Li &lt;em&gt;et al&lt;/em&gt; 2015 and compared against both DSeq2 and EdgeR. There are more recent examples, again I believe one of the DSeq2 team produced a non-parametric DEG as well.&lt;/p&gt;&#xA;&lt;p&gt;The GitHub page for Li &lt;em&gt;et al&lt;/em&gt; 2022, is &lt;a href=&quot;https://github.com/xihuimeijing/DEGs_Analysis_FDR&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/xihuimeijing/DEGs_Analysis_FDR&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I understand the criticism recently stated about Wilcoxon (non-parametrics in general) is their reduced power (response on Biostars). What happens is you get fewer significant 'hits' that are under or over-expressed genes. This is the basic trade-off between parametrics vs. non-parametrics. The key issue is if you've trapped your key results within a non-parametric test, &lt;em&gt;that is a very solid result&lt;/em&gt;. If the equivalent results are trapped using EdgeR or DSeq2 there is always going to be question mark whether it's a false positive.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;When to use EdgeR/DSeq2&lt;/em&gt;: if you are doing alternative statistical tests on the expression data which are directly comparable with under- and over-expression (e.g. ML and DL analysis) thats okay because we know Edge/DSeq2 gives false positives. When I've done this there's a 'centre ground' which can give conflicting result, its the central quartile type area, i.e. 25 - 75% region (to be honest it can be a bit outside that).&lt;/p&gt;&#xA;&lt;p&gt;If you are singly dependent on EdgeR, DSeq2 or NOISeq it is better to be conservative in my opinion, i.e. fewer hits which you are more certain about.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Final point&lt;/strong&gt; The final issue is simply how close to the DSeq2 and EdgeR data sets is the data you are working with, i.e. same species? What I would caution is these distributions have been applied broadly across species. The extent of the false positive rate must vary in my opinion.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-05-10T00:46:15.767" LastActivityDate="2023-05-10T00:46:15.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20987" PostTypeId="2" ParentId="20976" CreationDate="2023-05-08T11:54:21.017" Score="1" Body="&lt;p&gt;****this was the xml file that i configured to maxquant. i changed modification and added the fastq and raw file. Other parameters were default. however, 'Unhandled Exception: System.IndexOutOfRangeException: Index was outside the bounds of the array' was reported when I run the command '&lt;em&gt;maquant mq.xml&lt;/em&gt;'. but, When I only kept the N modification, the program ran successfully, but a small amount of peptides were identified.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/wCo8e.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wCo8e.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/zUA57.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zUA57.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17442" LastActivityDate="2023-05-08T11:54:21.017" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20988" PostTypeId="1" CreationDate="2023-05-08T12:58:07.553" Score="2" ViewCount="47" Body="&lt;p&gt;I'm finding a web server (or standalone program) that allows the upload of a custom PDB for protein fold topology search.&lt;/p&gt;&#xA;&lt;p&gt;Do you know any of them?&lt;/p&gt;&#xA;&lt;p&gt;More details:&lt;/p&gt;&#xA;&lt;p&gt;Actually I started off with Foldseek but have a question in mind:&lt;/p&gt;&#xA;&lt;p&gt;how do I know if I have a novel fold? (because similarity search will often return partially similar hits).&lt;/p&gt;&#xA;&lt;p&gt;Then, I tried to think of topology, as an illustration:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UE70k.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UE70k.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;which simplifies structures to spatial arrangements of secondary structures (or topology).&lt;/p&gt;&#xA;&lt;p&gt;I want to know if there's any database of protein fold topologies and tools for topology similarity search.&lt;/p&gt;&#xA;&lt;p&gt;ADD COMMENT • link • edit •&lt;/p&gt;&#xA;" OwnerUserId="14169" LastEditorUserId="14169" LastEditDate="2023-05-09T10:54:41.773" LastActivityDate="2023-05-09T11:57:34.657" Title="Protein fold topology search web-server" Tags="|protein-structure|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20990" PostTypeId="2" ParentId="20949" CreationDate="2023-05-08T21:52:11.577" Score="3" Body="&lt;p&gt;@Max_IT and @user3479780 Thanks for bringing this issue to our attention. It has been fixed in EDirect 19.5, which is now available on our public ftp site (See installation instructions &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK179288/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). Please update your installation and try again.&lt;/p&gt;&#xA;&lt;p&gt;The problem was that using capitalized &lt;code&gt;-db &amp;quot;SRA&amp;quot;&lt;/code&gt; bypassed normalization of the record. This was easily solved within EDirect by lower-casing the &lt;code&gt;-db&lt;/code&gt; argument value to make it case-insensitive.&lt;/p&gt;&#xA;&lt;p&gt;Normalization unescapes the XML-encoded ExpXml and Runs field contents to integrate them as regular XML within the docsum in situ, before passing them to the user. Thus:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;efetch -db SRA -format docsum -id SRA030738,SRA030736 |&#xA;xtract -pattern DocumentSummary -element Run@acc&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;will now work properly, without the need for chaining two &lt;code&gt;xtract&lt;/code&gt; commands together.&lt;/p&gt;&#xA;&lt;p&gt;When creating structured records, NCBI internal software enforces conformation to the ASN.1 specification or XML document type definition. This is normally beneficial, blocking syntactically invalid records (and preventing new software bugs from silently corrupting production pipelines), but it limits the ability to add experimental structured data that is not (yet) in the spec.&lt;/p&gt;&#xA;&lt;p&gt;Since &lt;code&gt;xtract&lt;/code&gt; does not use a spec or DTD, it can find any element within the newly-unescaped contents by object name. (It still checks for properly-balanced opening and closing tags.)&lt;/p&gt;&#xA;&lt;p&gt;Note also that you can skip the &lt;code&gt;esearch&lt;/code&gt; step if you know the identifiers, and can instead pass them directly to &lt;code&gt;efetch&lt;/code&gt; in an &lt;code&gt;-id&lt;/code&gt; argument. For larger sets of identifiers, you can now pipe them from a file, one ID per line, without the need for an &lt;code&gt;epost&lt;/code&gt; step.&lt;/p&gt;&#xA;&lt;p&gt;EDirect has logic to split large lists into smaller chunks, sized appropriately for each database, and will internally loop through the fetch requests until all of the IDs have been processed, automatically retrying in the event of transient network or server failure.&lt;/p&gt;&#xA;" OwnerUserId="3577" LastActivityDate="2023-05-08T21:52:11.577" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20991" PostTypeId="2" ParentId="20965" CreationDate="2023-05-09T03:08:42.953" Score="3" Body="&lt;p&gt;Save the following as a python script (e.g., named &lt;code&gt;splitty.py&lt;/code&gt;). Make the script executable by doing &lt;code&gt;chmod +x splitty.py&lt;/code&gt;. Then, if your &lt;code&gt;MEGAN&lt;/code&gt; data are in a file called &lt;code&gt;taxonfile.csv&lt;/code&gt;, run &lt;code&gt;splitty.py&lt;/code&gt; by doing &lt;code&gt;./splitty.py taxonfile.csv&lt;/code&gt;. Get help on the script by doing &lt;code&gt;splitty.py -h&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;Splitty&#xA;&#xA;Use this to split up a taxonomy table&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;def argsparser():&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    argsparser&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    import argparse&#xA;    parser = argparse.ArgumentParser(prog=&amp;quot;splitty&amp;quot;,&#xA;                                    formatter_class=argparse.ArgumentDefaultsHelpFormatter,&#xA;                                    description=&amp;quot;Uset splitty to get taxon table.&amp;quot;)&#xA;    parser.add_argument(&amp;quot;table&amp;quot;, help=&amp;quot;Taxon table in semicolon separated format.&amp;quot;)&#xA;    return parser&#xA;&#xA;def parseline(line_in):&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    Parse each line_in as return a dict&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    transl = {&amp;quot;d&amp;quot;: &amp;quot;Domain&amp;quot;,&#xA;             &amp;quot;p&amp;quot;: &amp;quot;Phylum&amp;quot;,&#xA;             &amp;quot;c&amp;quot;: &amp;quot;Class&amp;quot;,&#xA;             &amp;quot;o&amp;quot;: &amp;quot;Order&amp;quot;,&#xA;             &amp;quot;f&amp;quot;: &amp;quot;Family&amp;quot;,&#xA;             &amp;quot;g&amp;quot;: &amp;quot;Genus&amp;quot;,&#xA;             &amp;quot;s&amp;quot;: &amp;quot;Species&amp;quot;}&#xA;    return {transl[val[0]]: val[1] for val in [i.split(&amp;quot;__&amp;quot;) for i in line_in]}&#xA;&#xA;def main(args):&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    The main routine.&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    import pandas as pd&#xA;    df_list = []&#xA;    with open(args.table, &amp;quot;r&amp;quot;) as input_handle:&#xA;        for line in input_handle.readlines():&#xA;            linelist = parseline(list(filter(None, [i.strip() for i in line.rstrip().split(&amp;quot;;&amp;quot;)]))[1:])&#xA;            df_list.append(pd.DataFrame(linelist, index=[&amp;quot;Taxon&amp;quot;]))&#xA;    print(pd.concat(df_list).to_csv(sep=&amp;quot;\t&amp;quot;, index=False))&#xA;&#xA;if __name__ == &amp;quot;__main__&amp;quot;:&#xA;    main(argsparser().parse_args())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3381" LastActivityDate="2023-05-09T03:08:42.953" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="20992" PostTypeId="1" CreationDate="2023-05-09T05:56:14.780" Score="2" ViewCount="62" Body="&lt;p&gt;I recently received whole exome sequencing samples that were sequenced on an MGI sequencing instrument, DNBSEQ-T7. I am interested in somatic variant-calling on the paired tumor-normal samples. As this isn't the canonical Illumina sequencer, an issue that I am concerned with is that the technical artifacts arising from the sequencer are not yet well characterised and this may lead to spurious artifacts in the variants.&lt;/p&gt;&#xA;&lt;p&gt;I asked the company that did the sequencing for a panel of normals (PON) for the instrument to run Mutect2, but they simply replied that they use the standard GATK PON for their somatic variant calls.&lt;/p&gt;&#xA;&lt;p&gt;How should I go about assessing whether the variants are &amp;quot;biased&amp;quot; in any way, as a result of the instrument's technical properties? Is there a way to quantify the magnitude of these technical errors from the VCF files?&lt;/p&gt;&#xA;" OwnerUserId="15940" LastEditorUserId="298" LastEditDate="2023-05-10T15:30:28.890" LastActivityDate="2023-05-10T15:30:28.890" Title="WES variant calling with DNBSEQ-T7: technical quality assessment" Tags="|variant-calling|gatk|wes|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20993" PostTypeId="1" AcceptedAnswerId="21027" CreationDate="2023-05-09T06:42:03.147" Score="0" ViewCount="192" Body="&lt;p&gt;I had created databases of different sets of metagenome datasets - one with 6 runs, other with 48 runs, another with 100 runs, etc using the accession list for each of these datasets and makeblastdb command. Since I wanted to check the abundance of one specific 16S rRNA gene sequence, I ran a blastn command using this 16S sequence (F7) as the query against each of the created databases with a txt file (F7_droughtS.txt, F7_Dy.txt, so on) as the output. Ideally I should have got differently sized files with different number of OTU matches. I did get different files since the SRR ids were different for each database, but the results did not go beyond 500 lines for any database search. Can anyone explain why this is so?&lt;/p&gt;&#xA;&lt;p&gt;Following is the command I used for blastn of:&#xA;&lt;code&gt;sequence.fasta&lt;/code&gt; against database &lt;code&gt;firstdb&lt;/code&gt; :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastn -query sequence.fasta -db firstdb -outfmt 6 -out output1.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Also ran the following to know the default number of lines :&#xA;&lt;code&gt;blastn -help -max_target_seqs&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Number of database sequences to show one-line descriptions for&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Not applicable for outfmt &amp;gt; 4 Default = 500'     * Incompatible with:  max_target_seqs  -num_alignments &amp;lt;Integer, &amp;gt;=0&amp;gt;    Number of database sequences to show alignments for    Default = 250' * Incompatible with: max_target_seqs&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="17392" LastEditorUserId="3848" LastEditDate="2023-05-16T14:49:49.367" LastActivityDate="2023-05-16T14:57:35.893" Title="Blast output file only shows 500 lines -outfmt 6" Tags="|phylogenetics|phylogeny|metagenome|blastn|makeblastdb|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="20994" PostTypeId="2" ParentId="20988" CreationDate="2023-05-09T11:57:34.657" Score="1" Body="&lt;p&gt;There are a few, but the main candidates for this would be :&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;CATH - &lt;a href=&quot;http://www.cathdb.info/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.cathdb.info/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;SCOP - &lt;a href=&quot;https://scop.mrc-lmb.cam.ac.uk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://scop.mrc-lmb.cam.ac.uk/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Ptgl - &lt;a href=&quot;http://ptgl.uni-frankfurt.de/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://ptgl.uni-frankfurt.de/&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;CATH and Ptgl have search services on their website, but not sure about SCOP.&lt;/p&gt;&#xA;" OwnerUserId="745" LastActivityDate="2023-05-09T11:57:34.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20995" PostTypeId="2" ParentId="20992" CreationDate="2023-05-09T12:23:06.033" Score="1" Body="&lt;p&gt;Okay it seems the company has given you the VCF (maybe), you want to re-run it via Mutect2.  Mutect2 is part of the GATK suite, it's an upgrade, and they are refusing to run the appropriate PON. I think this is more personnel management than a major technical hurdle.&lt;/p&gt;&#xA;&lt;p&gt;I would reply to the tech and CC the technical director  of the company explaining Mutect2 vs. regular GATK PON. You simply use a standard process of escalation within technical justification. Maybe send them the commend to demonstrate how simple it is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;     gatk Mutect2 \&#xA;     -R reference.fa \&#xA;     -I tumor.bam \&#xA;     -I normal.bam \&#xA;     -normal normal_sample_name \&#xA;     --germline-resource af-only-gnomad.vcf.gz \&#xA;     --panel-of-normals pon.vcf.gz \&#xA;     -O somatic.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The company will have to justify their position regarding PONs or accommodate your requirements either now or the future. There may simply be a technical misunderstanding between you and the tech.&lt;/p&gt;&#xA;&lt;p&gt;You've paid for the service - thats the standard you expect. It's easier for them to assist you than all the agro of a complaints procedure and I'm pretty sure they'll help.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-05-09T13:05:29.167" LastActivityDate="2023-05-09T13:05:29.167" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="20996" PostTypeId="2" ParentId="15197" CreationDate="2023-05-09T15:00:48.047" Score="1" Body="&lt;p&gt;as for wdl version 1.0 (other versions untested)&lt;/p&gt;&#xA;&lt;p&gt;you can pack your reference and related files into a struct:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;struct Reference {&#xA;    File fasta&#xA;    File dict&#xA;    File fai&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You will pass the &lt;code&gt;Reference&lt;/code&gt; to your tasks instead of &lt;code&gt;File&lt;/code&gt;.&#xA;Inside your workflow/config define:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Ref example = {&#xA;    &amp;quot;fasta&amp;quot; : &amp;quot;reference.fasta&amp;quot;,&#xA;    &amp;quot;fai&amp;quot; : &amp;quot;reference.fasta.fai&amp;quot;,&#xA;    &amp;quot;dict&amp;quot; : &amp;quot;reference.dict&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Inside the task you just use the file want use by &lt;code&gt;example.fasta&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17468" LastEditorUserId="17468" LastEditDate="2023-05-09T15:01:18.343" LastActivityDate="2023-05-09T15:01:18.343" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="20997" PostTypeId="1" CreationDate="2023-05-09T15:23:14.000" Score="1" ViewCount="52" Body="&lt;p&gt;I wanted to generate 3D Cartesian coordinates of a peptide sequence for fixed bond lengths and bond angles for a given set of backbone dihedral angles and side dihedral angles. I also wanted to compute non-bonded van der Waal interactions (1-4 atoms) for each of the peptide generated from the dihedral angles. I initially tried frag builder module in python although it works fine. There are few issues,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I could not fix bond length and bond angle of my own choice.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;It generates PDB file and save it inside the hard disk. So I need to read it again and parse it to compute non-bonded van der Waal interactions (1-4 atom). It consumes lot of time&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Also I find fragbuilder is not that fast in generating peptides.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reading , parsing and computing the interactions takes around 30 mins to complete the task for 200,000 3D ensemble of structures of given peptide sequence of length 18.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Kindly suggest me a better module in python which does the job faster and also give reference to it.&lt;/p&gt;&#xA;" OwnerUserId="17469" LastActivityDate="2023-06-09T14:02:50.370" Title="Generating 3D cartesian coordinates of a peptide and computing van der Waal interactions (1-4 atoms) for a given set of dihedral angles?" Tags="|python|protein-structure|pdb|molecular-dynamics|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="20999" PostTypeId="2" ParentId="20997" CreationDate="2023-05-10T13:05:01.123" Score="1" Body="&lt;p&gt;The PDB module in Biopython is worth checking out for obtaining angles, distances, torsion angles. It is described from p208 at &lt;a href=&quot;http://biopython.org/DIST/docs/tutorial/Tutorial.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://biopython.org/DIST/docs/tutorial/Tutorial.pdf&lt;/a&gt;. It doesn't do van der Waal interactions.&lt;/p&gt;&#xA;&lt;p&gt;For example,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.PDB.PDBParser import PDBParser&#xA;from Bio.PDB.Chain import Chain&#xA;from Bio.PDB.internal_coords import *&#xA;from Bio.PDB.PICIO import write_PIC, read_PIC, read_PIC_seq&#xA;from Bio.PDB.ic_rebuild import write_PDB, IC_duplicate, structure_rebuild_test&#xA;from Bio.PDB.SCADIO import write_SCAD&#xA;from Bio.PDB.PDBIO import PDBIO&#xA;&#xA;parser = PDBParser(PERMISSIVE=1)&#xA;myProtein = parser.get_structure(&amp;quot;1a8o&amp;quot;, &amp;quot;1A8O.pdb&amp;quot;)&#xA;myChain = myProtein[0][&amp;quot;A&amp;quot;]&#xA;IC_Chain.MaxPeptideBond = 4.0&#xA;myChain.internal_coord = None&#xA;myChain.atom_to_internal_coordinates(verbose=True)&#xA;r2 = myChain.child_list[1]&#xA;r2ic = r2.internal_coord&#xA;r2chi1 = r2ic.get_angle(&amp;quot;chi1&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-05-10T13:05:01.123" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21000" PostTypeId="1" AcceptedAnswerId="21005" CreationDate="2023-05-10T18:27:41.577" Score="1" ViewCount="58" Body="&lt;p&gt;One of the columns in my R data frame looks like this:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;[1] &amp;quot;SVLEN=1007&amp;quot;   &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=204&amp;quot;    &amp;quot;SVLEN=-379&amp;quot;  &#xA; [7] &amp;quot;SVLEN=-492&amp;quot;   &amp;quot;SVLEN=-825&amp;quot;   &amp;quot;SVLEN=-713&amp;quot;   &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=3553&amp;quot;   &amp;quot;SVLEN=432391&amp;quot;&#xA;[13] &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=-52&amp;quot;    &amp;quot;SVLEN=&amp;gt;1615&amp;quot;  &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=6899&amp;quot;   &amp;quot;SVLEN=.&amp;quot;     &#xA;[19] &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=4468&amp;quot;   &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=627&amp;quot;   &#xA;[25] &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=1171&amp;quot;   &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=625&amp;quot;   &#xA;[31] &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=-237&amp;quot;   &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=-30&amp;quot;   &#xA;[37] &amp;quot;SVLEN=39&amp;quot;     &amp;quot;SVLEN=.&amp;quot;      &amp;quot;SVLEN=-30&amp;quot;    &amp;quot;SVLEN=-377&amp;quot;   &amp;quot;SVLEN=120497&amp;quot; &amp;quot;SVLEN=.&amp;quot;     &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and I would like it to look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[1] &amp;quot;1007&amp;quot;   &amp;quot;.&amp;quot;      &amp;quot;.&amp;quot;      &amp;quot;.&amp;quot;      &amp;quot;204&amp;quot;    &amp;quot;-379&amp;quot;  &#xA; [7] &amp;quot;-492&amp;quot;   &amp;quot;-825&amp;quot;   &amp;quot;-713&amp;quot;   &amp;quot;.&amp;quot;      &amp;quot;3553&amp;quot;   &amp;quot;432391&amp;quot;&#xA;[13] &amp;quot;.&amp;quot;      &amp;quot;-52&amp;quot;    &amp;quot;&amp;gt;1615&amp;quot;  &amp;quot;.&amp;quot;      &amp;quot;6899&amp;quot;   &amp;quot;.&amp;quot;     &#xA;[19] &amp;quot;.&amp;quot;      &amp;quot;4468&amp;quot;   &amp;quot;.&amp;quot;      &amp;quot;.&amp;quot;      &amp;quot;.&amp;quot;      &amp;quot;627&amp;quot;   &#xA;[25] &amp;quot;.&amp;quot;      &amp;quot;.&amp;quot;      &amp;quot;.&amp;quot;      &amp;quot;1171&amp;quot;   &amp;quot;.&amp;quot;      &amp;quot;625&amp;quot;   &#xA;[31] &amp;quot;.&amp;quot;      &amp;quot;.&amp;quot;      &amp;quot;-237&amp;quot;   &amp;quot;.&amp;quot;      &amp;quot;.&amp;quot;      &amp;quot;-30&amp;quot;   &#xA;[37] &amp;quot;39&amp;quot;     &amp;quot;.&amp;quot;      &amp;quot;-30&amp;quot;    &amp;quot;-377&amp;quot;   &amp;quot;120497&amp;quot; &amp;quot;.&amp;quot;         &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I could go in to the excel file I got it from and edit it but is there an easy way to do this in R?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;ANSWER&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;I found a solution to this probelm going back to the R documentation (&lt;a href=&quot;https://www.statology.org/gsub-r/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.statology.org/gsub-r/&lt;/a&gt;) use &lt;code&gt;gsub&lt;/code&gt; to section off characters in a string. My command for this was: &lt;code&gt;gsub(&amp;quot;.*=&amp;quot;,&amp;quot;\\1&amp;quot;,df$SVLEN_INFO)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;And that worked!&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="650" LastEditDate="2023-05-10T23:13:42.480" LastActivityDate="2023-05-17T18:36:14.880" Title="editing dataframe in r" Tags="|r|dataframe|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21001" PostTypeId="2" ParentId="20847" CreationDate="2023-05-11T04:43:59.220" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;How do they obtain these markers for Neanderthal ancestry to begin with? I know the Neanderthal genome has been sequenced from bone fragments&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Since ancient nuclear DNA is scarce, enough preserved molecules are needed.&lt;/li&gt;&#xA;&lt;li&gt;The level of scarcity about ancient nuclear DNA : often, 1% or fewer of the total DNA fragments from an extraction among soil microbes and other sources&lt;/li&gt;&#xA;&lt;li&gt;After extraction and sequencing, distinguish the genuine DNA fragments from those of modern contaminants and from those of microorganisms that are co-extracted from the sample&lt;/li&gt;&#xA;&lt;li&gt;Since ancient DNA has characteristic damage patterns, e.g. a high degree of cytosine deaminations on the five prime ends, and at the three prime end, there is a high frequency of Gs changing to As.&lt;/li&gt;&#xA;&lt;li&gt;Match these ancient DNA fragments to a map of the complete human genome reference sequence; this allowed the book of the target's genome to be assembled from the millions of random sequence fragments recovered from the bones&lt;/li&gt;&#xA;&lt;li&gt;The more times we had a read confirmed by multiple fragments, the more confident the sequencing could be in distinguishing the true DNA sequence, it is the sequencing depth issue&lt;/li&gt;&#xA;&lt;li&gt;Then genotype samples to discover, filter, and analyze variants.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17474" LastEditorUserId="3694" LastEditDate="2023-05-11T07:41:15.223" LastActivityDate="2023-05-11T07:41:15.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21002" PostTypeId="1" AcceptedAnswerId="21010" CreationDate="2023-05-11T09:37:04.137" Score="2" ViewCount="104" Body="&lt;p&gt;It is a lot easier in my work to do blast searches against protein databases when I can, since I work with distantly related organisms. But sometimes, I need to work with the actual nucleotide sequences. Until now, I've often done it manually, but it is getting unfeasible due to increasing data amounts.&lt;/p&gt;&#xA;&lt;p&gt;The NCBI website is structured such that I suspect automating the process (e.g. input a list of protein accession numbers, get out a fasta file with the corresponding nucleotide seqs) should be possible. At the same time, I am not quite sure what tools to use (biopython?), and if a tool already exists, I would rather not reinvent the wheel. Would you happen to know of anything like this? Thank you for your time!&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Update and WIP:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have tried Entrez Utils. In part it works, but I am having some issues scaling it up. For example,&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;elink -db protein -target nuccore -id WP_244350345.1 | efetch -format fasta_cds_na | awk '/^&amp;gt;/ {printf(&amp;quot;\n%s\n&amp;quot;,$0);next; } { printf(&amp;quot;%s&amp;quot;,$0);}  END {printf(&amp;quot;\n&amp;quot;);}' | grep WP_244350345.1 -A 1&lt;/code&gt;&lt;br /&gt;&#xA;works. (elinks to connect the protein accession WP_244350345.1 with the nucleotide database | efetch to grab the nucleotide sequence, which is a bit of a pain because it grabs the entire genome rather than the cds I want | so I need a bit of cleaning, including using awk to turn the resulting multi-line fasta to single-line fasta | and grep to actually get the cds sequence associated with WP_244350345.1. As I said, it works, and it results in:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;lcl|NZ_JAFIRA010000020.1_cds_WP_244350345.1_35 [gene=bchM] [locus_tag=JX360_RS09120] [protein=magnesium protoporphyrin IX methyltransferase] [protein_id=WP_244350345.1] [location=35695..36360] [gbkey=CDS]&#xA;ATGCAAGAGAAAAGCATCGTCCGCGACTATTTCAACTCCATTGGCTTCGACCGTTGGCGGCGCATCTACGGAGAGGATGAGGTCAATTTTGTCCAAAAAGACATTCGCTCCGGCCATGCCCGTACCGTAGCCACTGTGTTGGAATGGCTGGGGGATCCCTCAGGTTTGCAGATTTGTGATGCGGGGTGTGGGGTCGGTAGTCTTAGTTTGCCGTTGGCTGCCAAAGGGGCCCGCGTCTTCGCCAGCGACATTTCCGAACAAATGGTGAACGAAGCCCGCCGTCGCCAGCAAATCCAGTTGGGATCCACCGAGAACCCCCAATTTCGGGTCTCGGACTTAGAGGAACTAAGCGGAGAATATGATGCCGTCATTTGCCTGGATGTGATGATCCACTACCCAGAAGCCGATGCCCTGCGCATGCTGGAACATCTAACCCGCCTGGCCCGCTCTCGCTTGATCTTCAGTTTTGCCCCCAAAACCCCTCTGTTGACGTTGCTAAAAAAAGTGGGAGAGTTCTTCCCCGGCCCCAGCAAAGCCACCCGTGCCTACCAACACCGAGAAGCGATTTTGGTGAGCAAATTAACCGAATTGGGCTGGAACGTTCAGCAACGACAGACCATTCGTAGCCGCTTTTACTTTGCCTGGATTCTGGATTTGGCCCGGTGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;However&lt;/em&gt;, trying to read multiple accession numbers from a file, in a loop, is not working. My current code (essentially the same as before, just replacing WP_244350345.1 with ${LINE}:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;while IFS='' read -r LINE || [ -n &amp;quot;${LINE}&amp;quot; ]; do&#xA;echo &amp;quot;${LINE}&amp;quot;&#xA;elink -db protein -target nuccore -id &amp;quot;${LINE}&amp;quot; | efetch -format fasta_cds_na | awk '/^&amp;gt;/ {printf(&amp;quot;\n%s\n&amp;quot;,$0);next; } { printf(&amp;quot;%s&amp;quot;,$0);}  END {printf(&amp;quot;\n&amp;quot;);}' | grep &amp;quot;${LINE}&amp;quot; -A 1&#xA;done &amp;lt; &amp;quot;results.txt&amp;quot; | more&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;echo &amp;quot;${LINE}&amp;quot;&lt;/code&gt; part works fine, the rest does not. It has also worked perfectly fine with other parts of Entrez Utils (see below), so I am confused and I would appreciate some help.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;while IFS='' read -r LINE || [ -n &amp;quot;${LINE}&amp;quot; ]; do&#xA;organism=$(elink -db protein -target taxonomy -id &amp;quot;${LINE}&amp;quot; | efetch | tr --delete '\n' | sed 's/    species, /_/g'| sed 's/^.*\. //g' | sed 's/ /_/g')&#xA;echo &amp;quot;${LINE}&amp;quot;&#xA;echo &amp;quot;$organism&amp;quot;&#xA;done &amp;lt; results.txt | more&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;results in:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;WP_244350345.1&#xA;Thermostichus_vulcanus_cyanobacteria&#xA;WP_073547903.1&#xA;Chroogloeocystis_siderophila_cyanobacteria&#xA;WP_088429095.1&#xA;Halomicronema_hongdechloris_cyanobacteria&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4401" LastEditorUserId="4401" LastEditDate="2023-05-11T16:02:28.133" LastActivityDate="2023-05-11T16:19:50.040" Title="How to get the nucleotide sequence for protein NCBI accession code, programatically?" Tags="|proteins|gene|ncbi|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21003" PostTypeId="1" CreationDate="2023-05-11T11:32:16.007" Score="3" ViewCount="81" Body="&lt;p&gt;For example I have 3 bed files:&lt;/p&gt;&#xA;&lt;p&gt;window.bed:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1    0   1000&#xA;chr1    1000    2000&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;lp1.bed:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1    20  50&#xA;chr1    30  40&#xA;chr1    1500    1900&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;lp2.bed:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1    68  786&#xA;chr1    899 987&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I create this output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1    0   1000    2   2&#xA;chr1    1000    2000    1   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In this output columns 1, 2, 3 are the standard bed format cols, of the chromosome, start, and end of the features taken from window.bed above. Column 4 is the number of intersections between window.bed and lp1.bed per feature in window.bed. For example, the first element in column 4 is '2' because there are two intersections between window.bed and lp1.bed. Column 5 is the same but for the number of intersections between window.bed and lp2.bed for each feature in window.bed.&lt;/p&gt;&#xA;&lt;p&gt;In the case that a feature is a subset of another feature in either lp1.bed, or lp2.bed, they should be counted as two intersections. For example, the second feature of lp1.bed is a subset of the first feature. This is reflected in the first element of column 4 of the output.&lt;/p&gt;&#xA;&lt;p&gt;I suspect that it is doable with bedtools intersect, but I cannot work out the syntax.&lt;/p&gt;&#xA;" OwnerUserId="17477" LastEditorUserId="17477" LastEditDate="2023-05-11T13:56:07.153" LastActivityDate="2023-05-15T19:06:55.030" Title="Create bed file where each column is the number of intersections with another bed file" Tags="|bedtools|bed|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21004" PostTypeId="1" CreationDate="2023-05-11T11:39:53.920" Score="1" ViewCount="246" Body="&lt;p&gt;I have an Apple latest arm64 machine. Running the latest R 4.3.0 version. However, I'm unable to install an R package - &lt;code&gt;S4Arrays&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/X497b.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/X497b.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Doesn't look like there is an arm64 build for this package. Anybody know how to resolve it?&lt;/p&gt;&#xA;" OwnerUserId="174" LastActivityDate="2023-05-11T12:14:08.297" Title="Unable to install an R Bioconductor package on Apple M1/M2" Tags="|r|bioconductor|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21005" PostTypeId="2" ParentId="21000" CreationDate="2023-05-11T12:12:14.560" Score="2" Body="&lt;p&gt;Adding an answer for now:&lt;/p&gt;&#xA;&lt;p&gt;You can use &lt;code&gt;sub&lt;/code&gt; to remove the &lt;code&gt;SVLEN=&lt;/code&gt; part. You don't need &lt;code&gt;gsub&lt;/code&gt; and you don't need regular expressions, let alone capture groups.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;df$SVLEN_INFO &amp;lt;- sub(x = df$SVLEN_INFO, pattern = &amp;quot;SVLEN=&amp;quot;, replacement = &amp;quot;&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastEditorUserId="650" LastEditDate="2023-05-11T17:59:58.467" LastActivityDate="2023-05-11T17:59:58.467" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21006" PostTypeId="2" ParentId="21004" CreationDate="2023-05-11T12:14:08.297" Score="2" Body="&lt;p&gt;The easiest solution is to install the x86 version of R in your M1/M2 Mac &lt;a href=&quot;https://cran.r-project.org/bin/macosx/big-sur-x86_64/base/R-4.3.0-x86_64.pkg&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&#xA;It will be then emulated by the Rosetta. After that you will have no problems with Bioconductor packages&lt;/p&gt;&#xA;" OwnerUserId="11884" LastActivityDate="2023-05-11T12:14:08.297" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21007" PostTypeId="1" AcceptedAnswerId="21057" CreationDate="2023-05-11T14:50:26.387" Score="-2" ViewCount="199" Body="&lt;p&gt;I have a json file of protein data of mouse and human like&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-json prettyprint-override&quot;&gt;&lt;code&gt;{&#xA;    &amp;quot;name&amp;quot;: &amp;quot;P04202&amp;quot;,&#xA;    &amp;quot;full_name&amp;quot;: &amp;quot;Transforming growth factor beta-1 proprotein&amp;quot;,&#xA;    &amp;quot;symbol&amp;quot;: &amp;quot;TGFB1_MOUSE&amp;quot;,&#xA;    &amp;quot;length&amp;quot;: 390,&#xA;    &amp;quot;species&amp;quot;: &amp;quot;mouse&amp;quot;,&#xA;    &amp;quot;function&amp;quot;: &amp;quot;Transforming growth factor beta-1 proprotein: Precursor of the Latency-associated peptide (LAP) and Transforming growth factor beta-1 (TGF-beta-1) chains, which constitute the regulatory and active subunit of TGF-beta-1, respectively.&amp;quot;,&#xA;    &amp;quot;nr_modifications&amp;quot;: 2821,&#xA;    &amp;quot;residues&amp;quot;: [&#xA;        {&#xA;            &amp;quot;pos&amp;quot;: 1,&#xA;            &amp;quot;residue&amp;quot;: &amp;quot;M&amp;quot;,&#xA;            &amp;quot;modifications&amp;quot;: []&#xA;        },&#xA;        {&#xA;            &amp;quot;pos&amp;quot;: 2,&#xA;            &amp;quot;residue&amp;quot;: &amp;quot;P&amp;quot;,&#xA;            &amp;quot;modifications&amp;quot;: []&#xA;        },&#xA;        ...&#xA;        {&#xA;            &amp;quot;pos&amp;quot;: 60,&#xA;            &amp;quot;residue&amp;quot;: &amp;quot;A&amp;quot;,&#xA;            &amp;quot;modifications&amp;quot;: [&#xA;                {&#xA;                    &amp;quot;mod&amp;quot;: &amp;quot;[127] Fluoro&amp;quot;,&#xA;                    &amp;quot;type&amp;quot;: &amp;quot;Chemical derivative&amp;quot;,&#xA;                    &amp;quot;prob&amp;quot;: 1&#xA;                },&#xA;                {&#xA;                    &amp;quot;mod&amp;quot;: &amp;quot;[53] HNE&amp;quot;,&#xA;                    &amp;quot;type&amp;quot;: &amp;quot;Post-translational&amp;quot;,&#xA;                    &amp;quot;prob&amp;quot;: 0.25&#xA;                }&#xA;            ]&#xA;        },&#xA;        ...&#xA;        ]&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How do I create a scatter plot [![enter image description here][1]][1]&lt;/p&gt;&#xA;&lt;p&gt;Thanks for any idea&lt;/p&gt;&#xA;&lt;p&gt;So far I gathered a code to fetch the data from an online json&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-js prettyprint-override&quot;&gt;&lt;code&gt;export const load = ({ fetch }) =&amp;gt; {&#xA;  const fetchProteins = async () =&amp;gt; {&#xA;    const res = await fetch('https:address/assets/ptm.json')&#xA;    const data = await res.json()&#xA;    return data&#xA;  }&#xA;&#xA;  return {&#xA;    Proteins: fetchProteins()&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This code generates a scatter plot but I will need to hover around the dots&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;script&amp;gt;&#xA;    import { extent } from 'd3-array';&#xA;    import { scaleLinear } from 'd3-scale';&#xA;    export let data = [];&#xA;&#xA;    const margin = 30;&#xA;&#xA;    $: xScale = scaleLinear()&#xA;        .domain(extent(data.proteins.map((d) =&amp;gt; d.length)))&#xA;        .range([0, 800 - (margin * 2)]);&#xA;    &#xA;    $: yScale = scaleLinear()&#xA;        .domain(extent(data.proteins.map((d) =&amp;gt; d.nr_modifications)))&#xA;        .range([0, 400 - (margin * 2)]);&#xA;&amp;lt;/script&amp;gt;&#xA;  &#xA;  &#xA;&amp;lt;svg width=&amp;quot;800&amp;quot; height=&amp;quot;400&amp;quot;&amp;gt;&#xA;    &amp;lt;g transform=&amp;quot;translate({margin}, {margin})&amp;quot;&amp;gt;&#xA;        {#each data.proteins as datapoint}&#xA;            &amp;lt;circle&#xA;                cx={xScale(datapoint.length)}&#xA;                cy={yScale(datapoint.nr_modifications)}&#xA;                class:human={datapoint.species === 'human'}&#xA;                r=&amp;quot;20&amp;quot;&#xA;            /&amp;gt;&#xA;        {/each}&#xA;        &amp;lt;circle&#xA;                cx={10}&#xA;                cy={10}&#xA;                class=&amp;quot;human&amp;quot;&#xA;                r=&amp;quot;10&amp;quot;&#xA;            /&amp;gt;&#xA;            &amp;lt;circle&#xA;            cx={10}&#xA;            cy={40}&#xA;            class=&amp;quot;mouse&amp;quot;&#xA;            r=&amp;quot;10&amp;quot;&#xA;        /&amp;gt;&#xA;          &#xA;    &amp;lt;/g&amp;gt;&#xA;&amp;lt;/svg&amp;gt;&#xA;&#xA;&amp;lt;style&amp;gt;&#xA;    svg {&#xA;        border: 1px;&#xA;        border-style: solid;&#xA;    }&#xA;    circle {&#xA;        fill: steelblue;&#xA;        fill-opacity: 0.5;&#xA;    }&#xA;    .human {&#xA;        fill: red;&#xA;    }&#xA;&amp;lt;/style&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2023-05-12T09:07:43.170" LastActivityDate="2023-05-23T08:21:23.300" Title="Creating a scatterplot by a json data in js" Tags="|proteins|graphs|json|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21008" PostTypeId="2" ParentId="21003" CreationDate="2023-05-11T15:10:56.047" Score="2" Body="&lt;p&gt;The relevant &lt;code&gt;bedtools&lt;/code&gt; option seems to be &lt;code&gt;-c&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    -c  For each entry in A, report the number of overlaps with B.&#xA;        - Reports 0 for A entries that have no overlap with B.&#xA;        - Overlaps restricted by -f, -F, -r, and -s.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We can get the number of overlaps per region per file with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ bedtools intersect -c -a window.bed -b lp1.bed &#xA;chr1    0   1000    2&#xA;chr1    1000    2000    1&#xA;&#xA;$ bedtools intersect -c -a window.bed -b lp2.bed &#xA;chr1    0   1000    2&#xA;chr1    1000    2000    0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So we're almost there, we just need to combine them, and that's a job for &lt;code&gt;paste&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;$ paste  &amp;lt;(bedtools intersect -c -a window.bed -b lp2.bed) \&#xA;         &amp;lt;(bedtools intersect -c -a window.bed -b lp1.bed)&#xA;chr1    0   1000    2   chr1    0   1000    2&#xA;chr1    1000    2000    0   chr1    1000    2000    1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But that gives us extra fields, so we can use &lt;code&gt;cut&lt;/code&gt; to remove them:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ paste  &amp;lt;(bedtools intersect -c -a window.bed -b lp1.bed) \&#xA;         &amp;lt;(bedtools intersect -c -a window.bed -b lp2.bed | cut -f 4)&#xA;chr1    0   1000    2   2&#xA;chr1    1000    2000    1   0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;paste&lt;/code&gt; command just joins lines from its two input files in the order it receives them, but the bedtools commands will always output the same order as the lines in &lt;code&gt;window.bed&lt;/code&gt; so this should be fine.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;&amp;lt;(command)&lt;/code&gt; construct is called &lt;a href=&quot;https://www.gnu.org/software/bash/manual/html_node/Process-Substitution.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&amp;quot;process substitution&amp;quot;&lt;/a&gt; and is a way of passing the output of a command as a file to another command that expects files as input. Note that it isn't supported by all shells, but if you're using bash or zsh which are the most common these days, it should work.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-05-11T15:10:56.047" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21009" PostTypeId="2" ParentId="21002" CreationDate="2023-05-11T15:35:29.673" Score="0" Body="&lt;p&gt;You can use R &lt;a href=&quot;https://cran.r-project.org/web/packages/rentrez/vignettes/rentrez_tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Rentrez&lt;/a&gt;, in the link you can find a tutorial for the package usage.&lt;/p&gt;&#xA;" OwnerUserId="11884" LastActivityDate="2023-05-11T15:35:29.673" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21010" PostTypeId="2" ParentId="21002" CreationDate="2023-05-11T16:14:19.860" Score="1" Body="&lt;p&gt;Yes its easy using Biopython.&lt;/p&gt;&#xA;&lt;p&gt;What I'm requesting is the sequence must be &amp;gt;9500 otherwise it's discarded. In your case this value will be MUCH bigger, because these are bacterial genomes.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import Entrez&#xA;from Bio import SeqIO, AlignIO&#xA;&#xA;def getgenbank (smallGenDict):&#xA;    bacteriadb = []&#xA;    for bacteria in smallGenDict.keys():&#xA;        Entrez.email = &amp;quot;name@email.com&amp;quot;&#xA;        with Entrez.efetch(&#xA;                db=&amp;quot;nucleotide&amp;quot;, rettype=&amp;quot;gb&amp;quot;, retmode=&amp;quot;text&amp;quot;, id=bacteria&#xA;                ) as handle:&#xA;            seq_record = SeqIO.read(handle, &amp;quot;gb&amp;quot;)&#xA;            # record = get_CDS(seq_record, bacteria, smallGenDict.keys)&#xA;            record = seq_record # better to remove the above function because its outside the question&#xA;            if len(record.seq) &amp;gt; 9500:&#xA;                bacteriadb.append(record)&#xA;    return bacteriadb&#xA;&#xA;def get_CDS(record, bacteria, smallGenDict):&#xA;    for feature in record.features:&#xA;        # custom manipulation of input sequences &#xA;        # better to forget this bit of code&#xA;        # this whole part of the code has been skipped&#xA;        # its capturing the coding sequence (not relevant)&#xA;return record&#xA;&#xA;def main(smallGenDict):&#xA;    db = getgenbank(smallGenDict)&#xA;    SeqIO.write(db, '/Users/username/Desktop/myoutput.fasta', &amp;quot;fasta&amp;quot;)&#xA;&#xA;if __name__ == '__main__':&#xA;    smallGenDict = {'ABxxxxxx':'AGACTGATA', 'ABxxxxx2:'GAGATGA'} # better to automate this bit (different question)&#xA;    main(smallGenDict)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thats it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt; In the code below, the file smallGenDict is a dictionary of Genbank accession No.s and their sequence.&#xA;&lt;code&gt;{'ABxxxxxx':'AGACTGATA', 'ABxxxxx2:'GAGATGA'}&lt;/code&gt;&#xA;This better written here as a list&#xA;&lt;code&gt;['ABxxxxxx','ABxxxxx2' ...]&lt;/code&gt; say call it &lt;code&gt;smallGenList&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Where it says &lt;code&gt;smallGenDict.keys():&lt;/code&gt; just replace it with &lt;code&gt;smallGenList&lt;/code&gt; and remove the method &lt;code&gt;.keys()&lt;/code&gt;, otherwise the code will through a bug.&#xA;The function &lt;code&gt;get_CDS&lt;/code&gt; is processing the sequence file to remove the non-coding regions. I've not included it because its specific stuff.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-05-11T16:19:50.040" LastActivityDate="2023-05-11T16:19:50.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21011" PostTypeId="1" CreationDate="2023-05-11T17:10:40.130" Score="2" ViewCount="281" Body="&lt;p&gt;I am trying to get computeMatrix for bigwig file in specific genomic region using deeptools.&lt;br /&gt;&#xA;Below is the code I am using&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;computeMatrix reference-point --referencePoint TSS \&#xA;-b 1000 -a 1000 \&#xA;-R ~/Desktop/ATAC/ATAC/Inducible_elements_Greenberg.bed \&#xA;-S ~/Desktop/ATAC/Control.mRp.clN.bigWig \&#xA;--skipZeros \&#xA;-o ~/Desktop/ATAC/matrix_controlmRPATACBasalcomputematrix.gz \&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I get the following error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;File &amp;quot;/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/numpy/__init__.py&amp;quot;, line 305, in __getattr__&#xA;    raise AttributeError(__former_attrs__[attr])&#xA;AttributeError: module 'numpy' has no attribute 'float'.&#xA;`np.float` was a deprecated alias for the builtin `float`. To avoid this error in existing code, use `float` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.float64` here.&#xA;The aliases was originally deprecated in NumPy 1.20; for more details and guidance see the original release note at:&#xA;    https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations. Did you mean: 'cfloat'?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These are the packages in my Conda environment&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# packages in environment at /Users/abhinavsoni/anaconda3/envs/ATAC_Seq:&#xA;#&#xA;# Name                    Version                   Build  Channel&#xA;bzip2                     1.0.8                h3422bc3_4    conda-forge&#xA;ca-certificates           2023.5.7             hf0a4a13_0    conda-forge&#xA;libblas                   3.9.0           16_osxarm64_openblas    conda-forge&#xA;libcblas                  3.9.0           16_osxarm64_openblas    conda-forge&#xA;libcxx                    16.0.3               h4653b0c_0    conda-forge&#xA;libffi                    3.4.2                h3422bc3_5    conda-forge&#xA;libgfortran               5.0.0           12_2_0_hd922786_31    conda-forge&#xA;libgfortran5              12.2.0              h0eea778_31    conda-forge&#xA;liblapack                 3.9.0           16_osxarm64_openblas    conda-forge&#xA;libopenblas               0.3.21          openmp_hc731615_3    conda-forge&#xA;libsqlite                 3.41.2               hb31c410_1    conda-forge&#xA;libzlib                   1.2.13               h03a7124_4    conda-forge&#xA;llvm-openmp               16.0.3               h1c12783_0    conda-forge&#xA;ncurses                   6.3                  h07bb92c_1    conda-forge&#xA;numpy                     1.23.0          py310h0a343b5_0    conda-forge&#xA;openssl                   3.1.0                h53f4e23_3    conda-forge&#xA;pip                       23.1.2             pyhd8ed1ab_0    conda-forge&#xA;python                    3.10.11         h3ba56d0_0_cpython    conda-forge&#xA;python_abi                3.10                    3_cp310    conda-forge&#xA;readline                  8.2                  h92ec313_1    conda-forge&#xA;setuptools                67.7.2             pyhd8ed1ab_0    conda-forge&#xA;tk                        8.6.12               he1e0b03_0    conda-forge&#xA;tzdata                    2023c                h71feb2d_0    conda-forge&#xA;wheel                     0.40.0             pyhd8ed1ab_0    conda-forge&#xA;xz                        5.2.6                h57fd34a_0    conda-forge&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What could be its possible work around?&lt;/p&gt;&#xA;" OwnerUserId="17480" LastActivityDate="2023-05-12T21:13:40.383" Title="Error while running computeMatrix command in Deeptools" Tags="|python|chip-seq|atac-seq|deeptools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21012" PostTypeId="1" AcceptedAnswerId="21013" CreationDate="2023-05-11T20:07:59.593" Score="2" ViewCount="91" Body="&lt;p&gt;I am trying to run some code for my colleague in bash&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/path/to/cnvkit.py batch /path/to/my/folder/with/bams/*.bam \&#xA;  --normal --targets ${bed_file} \&#xA;  --fasta path/to/my/resources_broad_hg38_v0_Homo_sapiens_assembly38.fasta \&#xA;  --output-reference /path/to/my/CD_BATCH1_reference.cnn \&#xA;  --output-dir /path/to/my/Group_1 --scatter&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;however, I keep getting this truly peculiar error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &amp;quot;/path/to/cnvkit.py&amp;quot;, line 4, in &amp;lt;module&amp;gt;&#xA;    from . import commands&#xA;ImportError: cannot import name 'commands' from '__main__' (/path/to/cnvkit.py)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Attempt 1&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I checked if the path was correct and multiple times I found that it was correct. It did not change the error.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Attempt 2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Perhaps I need to &lt;code&gt;chmod +x *&lt;/code&gt; everything in the CNV kit directory and that will help. It did not change the error.&lt;/p&gt;&#xA;&lt;p&gt;Why is this error occurring? How do I solve this issue?&lt;/p&gt;&#xA;" OwnerUserId="17244" LastEditorUserId="298" LastEditDate="2023-05-12T10:56:21.690" LastActivityDate="2023-05-12T14:29:51.863" Title="CNV Kit ` from . import commands ImportError: cannot import name 'commands' from '__main__'`" Tags="|bash|cnv|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21013" PostTypeId="2" ParentId="21012" CreationDate="2023-05-11T20:33:54.967" Score="1" Body="&lt;p&gt;Could you download it from &lt;code&gt;conda&lt;/code&gt; and try again? Its here, &lt;a href=&quot;https://docs.conda.io/en/main/miniconda.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.conda.io/en/main/miniconda.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda create --name myenv&#xA;conda activate myenv&#xA;conda install -c bioconda cnvkit&#xA;cnvkit batch /path/to/my/folder/with/bams/*.bam --normal --targets ${bed_file} --fasta path/to/my/resources_broad_hg38_v0_Homo_sapiens_assembly38.fasta --output-reference /path/to/my/CD_BATCH1_reference.cnn --output-dir /path/to/my/Group_1 --scatter &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The path to the file is not used, the command is simply &lt;code&gt;cnvkit&lt;/code&gt;&#xA;This avoids all the privilege stuff and ensures you're running a virtual environment to protect your system Python.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To address the comments. &amp;quot;conda&amp;quot; is slow and doesn't ever load.&lt;/p&gt;&#xA;&lt;p&gt;There is an alternative to conda called &lt;strong&gt;mamba&lt;/strong&gt;, this is fast and designed to address this exact issue. I have never needed to use it so the following is a rough guide only.&lt;/p&gt;&#xA;&lt;p&gt;Download and install via&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda create --name myenv&#xA;conda activate myenv&#xA;conda install -c conda-forge mamba&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The documentation is here, &lt;a href=&quot;https://mamba.readthedocs.io/en/latest/user_guide/mamba.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://mamba.readthedocs.io/en/latest/user_guide/mamba.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I've never used it but the idea is ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mamba create -n mambaenv &amp;lt;list of packages&amp;gt;&#xA;mamba activate mambaenv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The list of packages could be left blank for now, then,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mamba install &amp;quot;Python&amp;gt;=3.10.0&amp;quot; numpy pandas pathlib&#xA;mamba install cnvkit # SEE NOTE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can also do&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mamba install -n mambaenv cnvkit&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;NOTE&lt;/strong&gt; cnvkit exists strictly in -c bioconda. You do not seem to search and install via channels in mamba&lt;/p&gt;&#xA;&lt;p&gt;You search for a package and its dependencies via&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mamba repoquery search cnvkit&#xA;mamba repoquery depends -a cnvkit&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It is important to place a virtual environment between the system Python and package Python (don't want a system corruption). More to the question, this would solve all the permission problems without changing your system in anyway (kinda important).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-05-12T14:29:51.863" LastActivityDate="2023-05-12T14:29:51.863" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21014" PostTypeId="1" AcceptedAnswerId="21015" CreationDate="2023-05-11T23:59:28.750" Score="2" ViewCount="35" Body="&lt;p&gt;I'm working with a system that does local BLAST searches using some preconfigured BLAST databases.  From what I see, these were updated within the last year, but I cannot find any current databases available that have those same names.  One specific set is a non-redundant set of human protein or nucleotide sequences.  The databases are named human_nr and human_nt (with various extensions as per the BLAST db format).  There are others names up_sp_human (Uniprot-Swissprot human?) and up_tbl_human (Uniprot-Tremble human?), but a cannot find any similar databases on the NCBI site or other sites.&lt;/p&gt;&#xA;&lt;p&gt;Where else should I look for such preconfigured databases?&lt;/p&gt;&#xA;" OwnerUserId="4491" LastEditorUserId="3848" LastEditDate="2023-06-10T23:38:22.260" LastActivityDate="2023-06-10T23:38:22.260" Title="NCBI Blast databases - cannot find some databases" Tags="|phylogenetics|phylogeny|blast|public-databases|ncbi|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21015" PostTypeId="2" ParentId="21014" CreationDate="2023-05-12T00:45:17.817" Score="3" Body="&lt;p&gt;I don't think there is a human_nr or human_nt database for NCBI.&lt;/p&gt;&#xA;&lt;p&gt;An equivalent database is accessible via the standard &lt;code&gt;nr/nt&lt;/code&gt; database and simply restrict the organism parameter to &lt;code&gt;human&lt;/code&gt; or to taxa id &lt;code&gt;9606 &lt;/code&gt;. This essentially will do the same thing.&lt;/p&gt;&#xA;&lt;p&gt;Or maybe this sort of thing?,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastdbcmd -db nr -entry all -outfmt &amp;quot;%g %T&amp;quot; | awk ' { if (&lt;span class=&quot;math-container&quot;&gt;$2 == 9606) { print $&lt;/span&gt;1 } } ' &amp;gt; human_flat-list.db&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-05-12T01:07:16.590" LastActivityDate="2023-05-12T01:07:16.590" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21016" PostTypeId="1" CreationDate="2023-05-12T07:55:50.177" Score="2" ViewCount="13" Body="&lt;p&gt;I want to study the ligand-receptor interaction and I am searching for lipidic ligands( mediators), I have found 16 so far. Does anyone know any database where I could find more lipidic ligands. I was searching on LIPID MAPS website but I couldn't know how to filter search for the lipidic ligands. Any recommendation is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="16457" LastActivityDate="2023-05-12T07:55:50.177" Title="Find Lipidic Mediator receptors" Tags="|single-cell|literature-search|interactions|lipid-bilayer|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21018" PostTypeId="2" ParentId="21011" CreationDate="2023-05-12T12:43:58.970" Score="2" Body="&lt;p&gt;It worked, I installed a the newer version of deeptools, from GitHub:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/deeptools/deepTools&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/deeptools/deepTools&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16676" LastEditorUserId="73" LastEditDate="2023-05-12T21:13:40.383" LastActivityDate="2023-05-12T21:13:40.383" CommentCount="0" CommunityOwnedDate="2023-05-12T12:43:58.970" ContentLicense="CC BY-SA 4.0" />
  <row Id="21019" PostTypeId="2" ParentId="20963" CreationDate="2023-05-12T15:25:28.617" Score="1" Body="&lt;p&gt;For anyone who arrives here looking for a large reference VCF but &lt;em&gt;not&lt;/em&gt; wanting GRCh37, you might consider the HGDP+1KG subset of the Genome Aggregation Database's dataset. You can find a VCF file per chromosome aligned, I believe, to GRCh38, &lt;a href=&quot;https://gnomad.broadinstitute.org/downloads#v3-hgdp-1kg&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1087" LastActivityDate="2023-05-12T15:25:28.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21020" PostTypeId="2" ParentId="2649" CreationDate="2023-05-14T13:25:43.300" Score="2" Body="&lt;p&gt;Just suggested improvements to the awk scripts in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2651/16755&quot;&gt;@terdon's answer&lt;/a&gt;, using any POSIX awk:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat FastaToTbl&#xA;#!/usr/bin/env bash&#xA;&#xA;awk -v OFS='\t' '&#xA;    {&#xA;        if ( /^&amp;gt;/ ) { out = (NR&amp;gt;1 ? ORS : &amp;quot;&amp;quot;) substr(&lt;span class=&quot;math-container&quot;&gt;$0,2) OFS }&#xA;        else        { out = $0 }&#xA;        printf &quot;%s&quot;, out&#xA;    }&#xA;    END { print &quot;&quot; }&#xA;' &quot;$&lt;/span&gt;{@:--}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat TblToFasta&#xA;#!/usr/bin/env bash&#xA;&#xA;awk -F'\t' '&#xA;    {&#xA;        gsub(/.{60}/,&amp;quot;&amp;amp;&amp;quot;ORS,&lt;span class=&quot;math-container&quot;&gt;$2)&#xA;        sub(ORS&quot;$&lt;/span&gt;&amp;quot;,&amp;quot;&amp;quot;,&lt;span class=&quot;math-container&quot;&gt;$2)&#xA;        print &quot;&amp;gt;&quot; $&lt;/span&gt;1 ORS &lt;span class=&quot;math-container&quot;&gt;$2&#xA;    }&#xA;' &quot;$&lt;/span&gt;{@:--}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ ./FastaToTbl file.fa&#xA;sequence1       ATGCGGAGCTTAGATTCTCGAGATCTCGATATCGCGCTTATAAAAGGCCCGGATTAGGGCTAGCTAGATATCGCGATAGCTAGGGATATCGAGATGCGATACG&#xA;sequence2       GTACTCGATACGCTACGCGATATTGCGCGATACGCATAGCTAACGATCGACTAGTGATGCATAGAGCTAGATCAGCTACGATAGCATCGATCGACTACGATCAGCATCAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ ./FastaToTbl file.fa | ./TblToFasta&#xA;&amp;gt;sequence1&#xA;ATGCGGAGCTTAGATTCTCGAGATCTCGATATCGCGCTTATAAAAGGCCCGGATTAGGGC&#xA;TAGCTAGATATCGCGATAGCTAGGGATATCGAGATGCGATACG&#xA;&amp;gt;sequence2&#xA;GTACTCGATACGCTACGCGATATTGCGCGATACGCATAGCTAACGATCGACTAGTGATGC&#xA;ATAGAGCTAGATCAGCTACGATAGCATCGATCGACTACGATCAGCATCAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;FastaToTbl&lt;/code&gt; will actually work in any awk but &lt;code&gt;TblToFasta&lt;/code&gt; requires a POSIX awk for support of regexp intervals like &lt;code&gt;{60}&lt;/code&gt;. If you have a very old, pre-POSIX awk that doesn't support regexp intervals then &lt;strong&gt;get a new awk&lt;/strong&gt; but if that's impossible for some reason then change &lt;code&gt;TblToFasta&lt;/code&gt; to the following and then it'll also work in any awk:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat TblToFasta&#xA;#!/usr/bin/env bash&#xA;&#xA;awk -F'\t' '&#xA;    BEGIN {&#xA;        dots = sprintf(&amp;quot;%&amp;quot;60&amp;quot;s&amp;quot;,&amp;quot;&amp;quot;)&#xA;        gsub(/ /,&amp;quot;.&amp;quot;,dots)&#xA;    }&#xA;    {&#xA;        gsub(dots,&amp;quot;&amp;amp;&amp;quot;ORS,&lt;span class=&quot;math-container&quot;&gt;$2)&#xA;        sub(ORS&quot;$&lt;/span&gt;&amp;quot;,&amp;quot;&amp;quot;,&lt;span class=&quot;math-container&quot;&gt;$2)&#xA;        print &quot;&amp;gt;&quot; $&lt;/span&gt;1 ORS &lt;span class=&quot;math-container&quot;&gt;$2&#xA;    }&#xA;' &quot;$&lt;/span&gt;{@:--}&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16755" LastEditorUserId="16755" LastEditDate="2023-05-14T14:00:01.083" LastActivityDate="2023-05-14T14:00:01.083" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21021" PostTypeId="2" ParentId="11757" CreationDate="2023-05-14T17:20:44.520" Score="1" Body="&lt;p&gt;For comprehensive microbial profiling of your 16S rRNA gene sequence data from Illumina MiSeq, I recommend using the web app Microbioma16S (&lt;a href=&quot;http://www.microbioma16s.it&quot; rel=&quot;nofollow noreferrer&quot;&gt;www.microbioma16s.it&lt;/a&gt;). This powerful tool offers an end-to-end pipeline that covers everything from preprocessing raw data to conducting alpha and beta diversity analysis. With Microbioma16S, you can explore interactive results of taxonomy associated with features, taxa relative abundances, and perform comparisons between sample groups. It's a user-friendly solution for your microbial profiling needs.&lt;/p&gt;&#xA;" OwnerUserId="17488" LastEditorUserId="17488" LastEditDate="2023-05-25T13:20:53.277" LastActivityDate="2023-05-25T13:20:53.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21022" PostTypeId="1" CreationDate="2023-05-15T11:26:39.547" Score="2" ViewCount="64" Body="&lt;p&gt;I have two PyRanges objects. PR1 is a set of regions of interest across a chromosome, PR2 tiles the genome and has a score associated with each range.&lt;/p&gt;&#xA;&lt;p&gt;I need a way to get for each range in PR1 its score based on its overlap with PR2. If it is contained within just one range in PR2, it inherits the score of that range. If it overlaps multiple ranges on PR2, the score should be a weighted average of those ranges based on the size of the overlap.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MYdJ7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MYdJ7.png&quot; alt=&quot;drawing of the calculations I wish to make&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Applying this to the image above, it would be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;score(A) = score(X)&#xA;score(B) = (OverlapBX * score(X) + OverlapBY * score(Y)) / sizeB&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've been browsing the PyRanges docs and I see a lot of functions that do almost that, but most of them don't take into account a score.&lt;/p&gt;&#xA;&lt;p&gt;I had done this before in R using GRanges, where I could iterate over the overlap object to piece together scores, but I can't find a direct port of R's &lt;code&gt;findOverlaps()&lt;/code&gt; in PyRanges.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The inputs I'm using for this are a list of mutation positions and a map of recombination rates over the chromosome they're on. So if two mutations are in a section with a constant recombination rate, the section between them inherits that recombination rate, but if the map says there's different rates between them, I need to take that into account.&lt;/p&gt;&#xA;&lt;p&gt;Here's a pastebin with the data I'm working with: &lt;a href=&quot;https://pastebin.com/sBYsjRQB&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/sBYsjRQB&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The Unfun Cat's first suggestion:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dist_to_pos = distances.join(positions, report_overlap=True).drop(like=&amp;quot;_b&amp;quot;)&#xA;df = dist_to_pos.df&#xA;g = df.groupby(&amp;quot;Id2&amp;quot;)&#xA;avg_score = g.cMperMb.sum() / g.Overlap.first()&#xA;avg_score.name = &amp;quot;AvgScore&amp;quot;&#xA;j_with_avg_score = pr.PyRanges(df.set_index(&amp;quot;Id2&amp;quot;).join(avg_score))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The results this give are... curious. Here's what it gives when applied to my data:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;j_with_avg_score &#xA;&#xA;Start   cMperMb     cMatstart   End     Chromosome  Id1     Overlap     AvgScore&#xA;0   2916415     1.223681    3.272236    2917484     chr1    0   1   12.236813&#xA;1   2916415     1.223681    3.272236    2917484     chr1    1   24  12.236813&#xA;2   2916415     1.223681    3.272236    2917484     chr1    2   5   12.236813&#xA;3   2916415     1.223681    3.272236    2917484     chr1    3   74  12.236813&#xA;4   2916415     1.223681    3.272236    2917484     chr1    4   110     12.236813&#xA;...     ...     ...     ...     ...     ...     ...     ...     ...&#xA;947     2960324     0.269208    3.291457    2960462     chr1    913     138     0.001951&#xA;948     2960462     0.352644    3.291494    2960790     chr1    913     77  0.018319&#xA;949     2960462     0.352644    3.291494    2960790     chr1    914     6   0.018319&#xA;950     2960462     0.352644    3.291494    2960790     chr1    915     15  0.018319&#xA;951     2960462     0.352644    3.291494    2960790     chr1    916     71  0.018319&#xA;&#xA;952 rows × 8 columns&#xA;## input data contains 917 sections --&amp;gt; mismatch!&#xA;&#xA;res_dic = {}&#xA;for x,y in zip(j_with_avg_score.Start, j_with_avg_score.AvgScore):&#xA;    res_dic[x] = y&#xA;res_dic&#xA;&#xA;{2916415: 12.236813262999997,&#xA; 2917484: 0.02265139573888889,&#xA; 2917538: 0.06125764894,&#xA; 2917608: 0.10651243958260867,&#xA; 2918281: 0.6792060563708334,&#xA; 2918836: 1.1527867390640016,&#xA; 2921187: 4.176119918333334,&#xA; 2921274: 1.2543028034,&#xA; 2921300: 0.9058800047555555,&#xA; 2921976: 3.26162101122,&#xA; 2922306: 0.4522015679581395,&#xA; 2923593: 0.7409518310923077,&#xA; 2924172: 8.386433179800006,&#xA; 2925614: 0.31748233525000014,&#xA; 2926340: 0.17712928483599996,&#xA; 2926675: 1.4567026153333333,&#xA; 2926730: 5.063891720272731,&#xA; 2927172: 2.42991683355,&#xA; 2927410: 0.9370814839,&#xA; 2927411: 0.1303501307571428,&#xA; 2928125: 0.20534225654799998,&#xA; 2928557: 0.19228154158888885,&#xA; 2928849: 1.5796778519999994,&#xA; 2929269: 1.9946231915199983,&#xA; 2931597: 0.15826380113170732,&#xA; 2933043: 0.010025524,&#xA; 2934387: 0.05559167711694925,&#xA; 2938888: 0.0023231045055555556,&#xA; 2940194: 0.01882597932,&#xA; 2941104: 0.1670264778,&#xA; 2941694: 0.0197562495886076,&#xA; 2942700: 0.3931709166833331,&#xA; 2946526: 1.6788888500999994,&#xA; 2947460: 0.19597836537931007,&#xA; 2951262: 5.532616244500001,&#xA; 2956899: 0.44889279189999975,&#xA; 2960324: 0.0019507828376811595,&#xA; 2960462: 0.018319161953246753}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Something's getting chewed up here, due to the number of unique sections in the output, the duplicates, and the start and end positions not matching the input data.&lt;/p&gt;&#xA;&lt;p&gt;However, the first value calculated is exactly the same as what I had gotten in my R script (with a few magnitude adjustments): &lt;code&gt;2916415: 12.236813262999997&lt;/code&gt; vs &lt;code&gt;2917178: 0.00012236813263&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;So the calculations happening are most likely correct, I think it's mainly the start and end positions that are getting messed up.&lt;/p&gt;&#xA;" OwnerUserId="6316" LastEditorUserId="6316" LastEditDate="2023-05-23T11:48:21.847" LastActivityDate="2023-05-23T11:48:21.847" Title="Assign score to PyRanges from another PyRanges object" Tags="|python|genomicranges|pyranges|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21023" PostTypeId="2" ParentId="17867" CreationDate="2023-05-15T16:38:00.643" Score="0" Body="&lt;p&gt;First and foremost, the fastcodeml guys didn't replace codeml via refactorising all the functionality of codeml, they refactorised one particular calculation and hence the confusion. fastcodeml isn't a particularly name helpful in truth, because it is not a faster version of all the functionality codeml, its just one of the calculations.&lt;/p&gt;&#xA;&lt;p&gt;In the authors defence they (notably) improved the runtime of the most intensive calculation in codeml and thats their point - its proof of concept.&lt;/p&gt;&#xA;&lt;p&gt;Thus, basically no fastcodeml can not be used in this context. fastcodeml runs the branch-site model of codeml only.&lt;/p&gt;&#xA;&lt;p&gt;What the OP describes isn't how the branch-sites model works, they are describing the branch model.&lt;/p&gt;&#xA;&lt;p&gt;The branch-site null model sets dN/dS to 1, and the LRT compares dN/dS likelihood between the designated branch(es) between dN/dS fixed to neutral against allowing dN/dS to be any value. Thus it's measuring whether the branch (branches) deviate from neutrality. What the OP describes via, model 0 one rate for the tree versus model 2 meaning two or more rates for the tree (basically two rates) and that is a different analysis and it is called a branch model.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;If fastcodeml can be shoehorned to perform sites and branch-models that would be good to know.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;There is an issue about 'tip selection' but the key answer concerns the basic difference between codeml and fastcodeml.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-05-15T17:00:57.780" LastActivityDate="2023-05-15T17:00:57.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21024" PostTypeId="2" ParentId="21003" CreationDate="2023-05-15T19:06:55.030" Score="2" Body="&lt;p&gt;Via &lt;code&gt;bedmap --count&lt;/code&gt;, which gives the option to specify a custom value when regions do not map:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;paste &amp;lt;(bedmap --echo --count --unmapped-val 0 --delim '\t' window.bed lp1.bed) &amp;lt;(bedmap --count --unmapped-val 0 window.bed lp2.bed) &amp;gt; answer.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Ref. &lt;a href=&quot;https://bedops.readthedocs.io/en/latest/content/reference/statistics/bedmap.html#element-and-overlap-statistics&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bedops.readthedocs.io/en/latest/content/reference/statistics/bedmap.html#element-and-overlap-statistics&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="776" LastActivityDate="2023-05-15T19:06:55.030" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21025" PostTypeId="1" AcceptedAnswerId="21026" CreationDate="2023-05-15T20:07:05.807" Score="0" ViewCount="279" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://stackoverflow.com/questions/76221113/refactoring-pandas-using-an-iterator-via-chunksize&quot;&gt;Stack Overflow&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Bioinformatics rationale&lt;/strong&gt; eggNOG files can be very big and sump all available RAM for regular to medium sized desktops.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I am looking for advice on using a pandas iterator.&lt;/p&gt;&#xA;&lt;p&gt;I performed a parsing operation using Python pandas, the size of the input files is  eggNOG a program used in metagenomic gene function identification but has broader application. The files are large and parsing is resulting in 'RAM bottleneck' phenomenon and will not processing/parse the file.&lt;/p&gt;&#xA;&lt;p&gt;The obvious solution is to shift to an iterator, which for pandas is the  chunksize option&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;import numpy as np&#xA;&#xA;df = pd.read_csv('myinfile.csv', sep=&amp;quot;\t&amp;quot;, chunksize=100)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Whats changed with the original code is the &lt;code&gt;chunksize=100&lt;/code&gt; bit, forcing an iterator.&lt;/p&gt;&#xA;&lt;p&gt;The next step is just to perform a simple operation, dropping a few columns and moving all '-' characters to np.nan then writing the whole file to disk.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df.drop(['score', 'evalue', 'Description', 'EC', 'PFAMs'],axis=1).replace('-', np.nan)&#xA;df.to_csv('my.csv',sep='\t',index=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How is the step performed under an iterator?&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="73" LastEditDate="2023-05-17T01:51:55.943" LastActivityDate="2023-06-16T02:07:15.870" Title="Refactoring pandas using an iterator via chunksize" Tags="|python|phylogenetics|phylogeny|pandas|kegg|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21026" PostTypeId="2" ParentId="21025" CreationDate="2023-05-15T20:26:29.150" Score="0" Body="&lt;p&gt;The solution posted by @el_oso and @Corralien &lt;a href=&quot;https://stackoverflow.com/questions/76221113/refactoring-pandas-using-an-iterator-via-chunksize&quot;&gt;here&lt;/a&gt; was as follows:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;if you're already having memory issues reading the file, the below will actually read and write to the new file in chunks.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;cols_to_keep = {'col1', 'col2', 'col3'}&#xA;df = pd.read_csv('myinfile.csv', chunksize=100, usecols=cols_to_keep)&#xA;with open('my.csv', 'w') as f:&#xA;    f.write('\t'.join(cols_to_keep) + '\n')&#xA;    for chunk in df:&#xA;        #do your processing here, appending that chunk to the file&#xA;        chunk.to_csv(f, header=None, sep='\t') &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;f is _io.TextIOWrapper i.e. a stream you can consistently write to in small memory chunks&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;This is very elegant because there are two components which are both cool and highlight why &lt;code&gt;pandas&lt;/code&gt; is so widely used.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Instead of loading a large file and then dumping stuff thats not wanted: don't load junk at point source.&lt;/li&gt;&#xA;&lt;li&gt;Simply open the write file outside the chunking loop will then stream the chunks to straight to disk and once the final chunking loop completes the file snaps shut.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The process is&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Load a chunk of data, but don't load any 'junk'.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Process the chunk of data and write it to disk.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Load another chunk of data repeating the above two steps.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Continue until the entire file is processed.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The &amp;quot;do processing&amp;quot; is an OO module (there's alot going on) and the chunk is actually &lt;code&gt;self.chunk&lt;/code&gt;. None of the original code needs to change its simply restructured inside the &lt;code&gt;open&lt;/code&gt; loop to stream chunks to disk.&lt;/p&gt;&#xA;&lt;p&gt;The beauty of this solution is &lt;code&gt;chunksize&lt;/code&gt; is a variable, here &lt;code&gt;100&lt;/code&gt; is just the example. Python's iterator called &lt;code&gt;yield&lt;/code&gt; is one line at a time iteration, but this the automates a defined 'chunk' at a time. Thus the runtime speed can be tuned against the &amp;quot;RAM bottleneck&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Testing&lt;/strong&gt; The chunked outfile was then reduced via &lt;code&gt;groupby&lt;/code&gt; to remove the &amp;quot;duplicates&amp;quot; and frequencies summed (code not shown). The &lt;code&gt;chunking&lt;/code&gt; OO module output file is identical to &amp;quot;non-chunked&amp;quot; OO module and written back into the module.&lt;/p&gt;&#xA;&lt;p&gt;Its worth upvoting the SO answers.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-05-15T20:32:24.040" LastActivityDate="2023-05-15T20:32:24.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21027" PostTypeId="2" ParentId="20993" CreationDate="2023-05-16T14:22:50.287" Score="1" Body="&lt;p&gt;I've reproduced the issue. The solution is to simply set &lt;code&gt;-max_target_seqs&lt;/code&gt; to whatever number of hits you want. E.g. &lt;code&gt;-max_target_seqs 1000&lt;/code&gt; gives 1000 and so on ... The problem  is specific to the &lt;code&gt;-outfmt&lt;/code&gt; option and not default searches.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;blastn version&lt;/strong&gt;  I'm using v2.12.0, Feb  8 2022&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Proof&lt;/strong&gt;&#xA;My database contains around 260000 fasta sequences.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;makeblastdb -in mystuff.aln.0.fa   -parse_seqids -blastdb_version 5  -out test.db -title &amp;quot;test&amp;quot; -dbtype nucl &#xA;blastn -query query.fas -db test.db -evalue 1e-6 -num_threads 4 -out blasthits.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then &lt;code&gt;wc blasthits.txt&lt;/code&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;gt;5000 hits&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If we now use &lt;code&gt;-outfmt 6&lt;/code&gt;, but remove the stringency of &lt;code&gt;-evalue 1e-6&lt;/code&gt; then ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastn -query query.fa -db test.db -outfmt 6 -num_threads 4 -out blasthits.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now perform &lt;code&gt;wc blasthits.txt&lt;/code&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;500&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So the OP is correct, I didn't know &lt;code&gt;-outfmt 6&lt;/code&gt; restricts the output to 500.&lt;/p&gt;&#xA;&lt;p&gt;If we now introduce the &lt;code&gt;max_target_seqs 1000&lt;/code&gt; parameter, suggested by @Laura then,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastn -query query.fa -db test.db -outfmt 6 -max_target_seqs 1000 -num_threads 4 -out blasthits.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then &lt;code&gt;wc blasthits.txt&lt;/code&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;1000&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If its 550 &lt;code&gt;wc ...&lt;/code&gt; is 550 etc...&lt;/p&gt;&#xA;&lt;p&gt;Thats it. Note in my blasts I usually parallelise across 4 processes i.e. &lt;code&gt;-num_threads 4&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-05-16T14:57:35.893" LastActivityDate="2023-05-16T14:57:35.893" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21028" PostTypeId="2" ParentId="21022" CreationDate="2023-05-17T07:47:54.363" Score="2" Body="&lt;p&gt;I have some inspiration below. It computes a weighted score based on overlap.&lt;/p&gt;&#xA;&lt;p&gt;I need to know what exactly you want the overlap BX and BY to be. Could you post some example data and expected output?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# initialize test data&#xA;import numpy as np&#xA;np.random.seed(0)&#xA;&#xA;import pyranges as pr&#xA;a1 = pr.data.aorta().drop()&#xA;a1.Id1 = list(range(len(a1)))&#xA;a2 = pr.data.aorta2().drop()&#xA;a2.Id2 = list(range(len(a2)))&#xA;a1.Score1 = np.random.random(len(a1)) * 100&#xA;a2.Score2 = np.random.random(len(a2)) * 100&#xA;&#xA;# compute weighted average based on overlaps&#xA;j = a1.join(a2, report_overlap=True).drop(like=&amp;quot;_b&amp;quot;)&#xA;&#xA;df = j.df&#xA;&#xA;g = df.groupby(&amp;quot;Id2&amp;quot;)&#xA;avg_score = g.Score2.sum() / g.Overlap.first()&#xA;avg_score.name = &amp;quot;AvgScore&amp;quot;&#xA;j_with_avg_score = pr.PyRanges(df.set_index(&amp;quot;Id2&amp;quot;).join(avg_score))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2688" LastActivityDate="2023-05-17T07:47:54.363" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21029" PostTypeId="2" ParentId="21000" CreationDate="2023-05-17T17:58:28.593" Score="-1" Body="&lt;p&gt;Following gringer's advice I am also adding this solution:&lt;/p&gt;&#xA;&lt;p&gt;Used gsub to section off characters in a string.&lt;/p&gt;&#xA;&lt;p&gt;My command for this was: &lt;code&gt;gsub(&amp;quot;.*=&amp;quot;,&amp;quot;\\1&amp;quot;,df$SVLEN_INFO)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thanks for all the help!&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="650" LastEditDate="2023-05-17T18:36:14.880" LastActivityDate="2023-05-17T18:36:14.880" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21030" PostTypeId="1" CreationDate="2023-05-17T18:08:50.557" Score="-1" ViewCount="160" Body="&lt;p&gt;I'm trying to make a plot that looks like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/p7vGO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/p7vGO.png&quot; alt=&quot;excel plot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is a rudimentary plot I did in excel. It's a plot of reads by length. Very simple nothing crazy to it. The important thing to note is that it's plotting the data in increasing order of length.&lt;/p&gt;&#xA;&lt;p&gt;When I try to use ggplot to make this same plot in R it does something like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/l5ueD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/l5ueD.png&quot; alt=&quot;r plot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where it orients the read names in alphabetical order not in ascending order of the lengths. The dataframe/table I have read into R is already listed in ascending length order. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1   c6dd66b4-dab5-484b-92b6-c4ecdcc155b1   1570&#xA;2   ecf15d4f-15b1-4636-a97d-e0928281e311   1739   &#xA;3   461f05c7-0fc6-4564-95df-6188966fe616   1790       &#xA;4   e989c32c-491e-4af1-a37d-4d4f5c87a3d9   1797       &#xA;5   ce94c2b3-442b-4b8c-bd59-6d7c23d5fe3d   1824       &#xA;6   ed15ea16-a28d-42ca-9a07-2818d90ddd4b   1832 &#xA;7   481550cd-6f87-44c7-915f-f0f3d638ffe4   1853      &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see the lengths increase as you go down and the read names are in no particular order. Can someone help me read in the correct order to make this plot actually work in ggplot?&lt;/p&gt;&#xA;" OwnerUserId="16067" LastActivityDate="2023-05-18T02:26:27.537" Title="change the order in ggplot" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21031" PostTypeId="2" ParentId="21030" CreationDate="2023-05-17T18:42:17.077" Score="-2" Body="&lt;p&gt;You need to enforce the order you need on the read-name column. Let's say &lt;code&gt;df&lt;/code&gt; is your data frame with &lt;code&gt;read.name&lt;/code&gt; and &lt;code&gt;length&lt;/code&gt; as the two columns.&lt;/p&gt;&#xA;&lt;p&gt;This is a fairly easy operation.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;df &amp;lt;- df[order(df$length),]&#xA;df$read.name &amp;lt;- factor(df$read.name, levels = df$read.name, ordered = TRUE)&#xA;ggplot(df, ...) #Your current ggplot code goes here&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastEditorUserId="3848" LastEditDate="2023-05-18T02:26:27.537" LastActivityDate="2023-05-18T02:26:27.537" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21032" PostTypeId="1" AcceptedAnswerId="21040" CreationDate="2023-05-17T20:39:17.773" Score="1" ViewCount="258" Body="&lt;p&gt;I'm trying to do simple filter for single data that is stored in &lt;code&gt;h5ad&lt;/code&gt; file format using this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import scanpy as sc&#xA;&#xA;sc.pp.filter_cells(adata, min_genes=500)&#xA;sc.pp.filter_genes(adata, min_cells=10)&#xA;print(adata.n_obs, adata.n_vars)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;when I saved the filtered data and see the size of my original data compare it, the size of the filtered data is more, but the dimension etc are reduced due to filtering.&lt;/p&gt;&#xA;&lt;p&gt;So is it something wrong am I doing?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;adata = sc.read_h5ad('/Single_cell/cerebellar_development/508cf892-174c-45ab-b2dc-05f54f1ee7ed/aldinger20.processed.h5ad')&#xA;adata&#xA;&#xA;AnnData object with n_obs × n_vars = 69174 × 49166&#xA;    obs: 'orig.ident', 'nCount_RNA', 'nFeature_RNA', 'sample_id', 'percent.mito', 'S.Score', 'G2M.Score', 'Phase', 'CC.Difference', 'nCount_SCT', 'nFeature_SCT', 'age', 'figure_clusters', 'sex', 'type', 'experiment', 'fig_cell_type'&#xA;    var: 'name'&#xA;    obsm: 'X_pca', 'X_umap', 'X_tsne'&#xA;&#xA;&#xA;sc.pp.filter_cells(adata, min_genes=500)&#xA;sc.pp.filter_genes(adata, min_cells=10)&#xA;print(adata.n_obs, adata.n_vars)&#xA;&#xA;results_file = 'adata.h5ad'&#xA;adata.write(results_file)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;FULL CODE&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;h1&gt;updated version: h5ad filter, added layers to make the filtered file usable for seurat to read and make a correct seurat object&lt;/h1&gt;&#xA;&lt;pre&gt;&lt;code&gt;#https://github.com/satijalab/seurat/issues/2485&#xA;&#xA;import scanpy as sc&#xA;import numpy as np&#xA;import pandas as pd&#xA;import time&#xA;from ipywidgets import interact, interactive, widgets&#xA;from IPython.display import display, HTML&#xA;&#xA;def filter_and_save_h5ad(h5ad_file, min_genes, min_cells):&#xA;    # h5ad file read&#xA;    adata = sc.read_h5ad(h5ad_file)&#xA;&#xA;    # cells and genes filter parameter from user&#xA;    sc.pp.filter_cells(adata, min_genes=min_genes)&#xA;    sc.pp.filter_genes(adata, min_cells=min_cells)&#xA;&#xA;    # Add 'counts' layer&#xA;    adata.layers['counts'] = adata.X.copy()&#xA;&#xA;    # add 'winsorization' layer&#xA;    adata.layers['winsorized'] = adata.X.copy()&#xA;&#xA;&#xA;    # filtered dataset save&#xA;    filtered_file = h5ad_file.replace('.h5ad', '_filtered.h5ad')&#xA;    adata.write(filtered_file)&#xA;&#xA;    print(f&amp;quot;Filtered dataset saved to {filtered_file}&amp;quot;)&#xA;&#xA;# input widgets&#xA;h5ad_file_widget = widgets.Text(description='File_add:')&#xA;min_genes_widget = widgets.IntText(description='Min_Genes:')&#xA;min_cells_widget = widgets.IntText(description='Min_Cells:')&#xA;button = widgets.Button(description='Filter_Save')&#xA;&#xA;# timer widget&#xA;time_widget = widgets.HTML(value=&amp;quot;&amp;lt;p&amp;gt;Time passed away: 0 seconds&amp;lt;/p&amp;gt;&amp;quot;)&#xA;&#xA;def on_button_clicked(click_me):&#xA;    h5ad_file = h5ad_file_widget.value.strip()&#xA;    min_genes = min_genes_widget.value&#xA;    min_cells = min_cells_widget.value&#xA;    &#xA;    # timer&#xA;    start_time = time.time()&#xA;    &#xA;    filter_and_save_h5ad(h5ad_file, min_genes, min_cells)&#xA;    &#xA;    # Calculate and update the elapsed time&#xA;    elapsed_time = time.time() - start_time&#xA;    time_widget.value = f&amp;quot;&amp;lt;p&amp;gt;Time passed away: {elapsed_time} seconds&amp;lt;/p&amp;gt;&amp;quot;&#xA;&#xA;button.on_click(on_button_clicked)&#xA;&#xA;# Display the widgets and button&#xA;display(h5ad_file_widget, min_genes_widget, min_cells_widget, button, time_widget)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14154" LastEditorUserId="104" LastEditDate="2023-06-08T20:34:57.133" LastActivityDate="2023-06-08T20:34:57.133" Title="h5ad file format filter" Tags="|scrnaseq|single-cell|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21033" PostTypeId="2" ParentId="21032" CreationDate="2023-05-18T02:33:53.890" Score="1" Body="&lt;p&gt;Scanpy's &lt;code&gt;h5ad&lt;/code&gt; is in fact hdf5 format. This format is used for rapid read/write (i/o) large amounts of data and thats its purpose. Its fairly easy to use from scratch.&lt;/p&gt;&#xA;&lt;p&gt;Its purpose is not to compress data. If you are concerned with the contents of &lt;code&gt;h5ad&lt;/code&gt; why not just read them directly via the &lt;code&gt;hdf5&lt;/code&gt; viewer. You can download it here. &lt;a href=&quot;https://www.hdfgroup.org/downloads/hdfview/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.hdfgroup.org/downloads/hdfview/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The file will then open up as a &amp;quot;Explorer&amp;quot; (Windows) or &amp;quot;Finder&amp;quot; (OSX) GUI file system and you can open the various files and see whats in them.&lt;/p&gt;&#xA;&lt;p&gt;My suspicion is that the original data is been written into the file. 'cause it can i/o rapidly and access precisely what it needs then file size could be a secondary consideration. Anyway you can peer in and find out by looking at the original file and the hdf5 save you've done.&lt;/p&gt;&#xA;&lt;p&gt;An example from the website is here&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zhtSf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zhtSf.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Cool right?&lt;/p&gt;&#xA;&lt;p&gt;Personally, I think HDF5 is overrated and &lt;code&gt;feather&lt;/code&gt; outperforms it for speed (i/o) - but thats just for dataframes. Its not much faster than pickling in case you are interested and pickling in addition does compression. parquet is the format that does rapid i/o but specialises in compression, again mostly for dataframes.&lt;/p&gt;&#xA;&lt;p&gt;HDF5 however does permit very pretty GUI interfaces.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Also if you want to load up and look at various files, that just uses the Python &lt;code&gt;hdf5&lt;/code&gt; module.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-05-18T03:09:54.613" LastActivityDate="2023-05-18T03:09:54.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21034" PostTypeId="1" CreationDate="2023-05-18T15:48:30.457" Score="1" ViewCount="827" Body="&lt;p&gt;I'm working with matched single cell data, where we have treated and untreated samples for the same patient. I ran CNV analysis using the &lt;code&gt;infercnv&lt;/code&gt; package.&lt;/p&gt;&#xA;&lt;p&gt;I've followed the tutorial:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# data matrix&#xA;counts_matrix &amp;lt;- scData@assays$RNA@counts&#xA;meta = data.frame(labels = Idents(scData), row.names = names(Idents(scData)))&#xA;unique(meta$labels) # check the cell labels&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Create the infercnv object&#xA;infercnv_obj &amp;lt;- CreateInfercnvObject(raw_counts_matrix = counts_matrix,&#xA;                                     annotations_file = meta,&#xA;                                     delim = &amp;quot;\t&amp;quot;,&#xA;                                     gene_order_file = &amp;quot;hg38_gencode_v27.txt&amp;quot;,&#xA;                                     ref_group_names = NULL)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;# perform infercnv operations to reveal cnv signal&#xA;infercnv_obj = infercnv::run(infercnv_obj,&#xA;                             cutoff=1,  # use 1 for smart-seq, 0.1 for 10x-genomics&#xA;                             out_dir=&amp;quot;output_dir&amp;quot;,  # dir is auto-created for storing outputs&#xA;                             cluster_by_groups=T,   # cluster&#xA;                             denoise=T,&#xA;                             HMM=T&#xA;                             )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, I'm not sure how to plot the results. Any help would be appreciated.&lt;/p&gt;&#xA;&lt;p&gt;When running the below recommendation I get this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;INFO [2023-05-19 12:49:22] ::plot_cnv:Start&#xA;INFO [2023-05-19 12:49:22] ::plot_cnv:Current data dimensions (r,c)=34108,17624 Total=82463956 Min=0 Max=55099.&#xA;INFO [2023-05-19 12:49:22] ::plot_cnv:Depending on the size of the matrix this may take a moment.&#xA;&amp;lt;sparse&amp;gt;[ &amp;lt;logic&amp;gt; ]: .M.sub.i.logical() maybe inefficient&#xA;Warning: sparse-&amp;gt;dense coercion: allocating vector of size 4.5 GiBINFO [2023-05-19 12:50:12] plot_cnv(): auto thresholding at: (-2.725632 , 3.000000)&#xA;INFO [2023-05-19 12:50:35] plot_cnv_observation:Start&#xA;INFO [2023-05-19 12:50:35] Observation data size: Cells= 17624 Genes= 34108&#xA;INFO [2023-05-19 12:50:35] clustering observations via method: ward.D&#xA;INFO [2023-05-19 12:50:35] Number of cells in group(1) is 331&#xA;INFO [2023-05-19 12:50:36] group size being clustered:  331,34108&#xA;Error in parallelDist(data_to_cluster, threads = infercnv.env$GLOBAL_NUM_THREADS) : &#xA;  x must be a matrix or a list of matrices.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6827" LastEditorUserId="6827" LastEditDate="2023-05-19T16:54:00.120" LastActivityDate="2024-03-30T16:08:26.003" Title="Plotting infercnv results" Tags="|r|single-cell|cnv|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21035" PostTypeId="1" CreationDate="2023-05-18T16:11:31.033" Score="0" ViewCount="55" Body="&lt;p&gt;My experiment is based on a quantitative continuous variable levels (B12 vitamin) in a sample of patients retrospectively selected based on values that can take (under our criteria):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;normal&lt;/li&gt;&#xA;&lt;li&gt;low levels: under the physiological or normal reference values but not enough to be considered deficiency&lt;/li&gt;&#xA;&lt;li&gt;deficiency (which is associated with poor nutrition, gastric disease, etc.). These data are excluded.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;We are interested in &lt;strong&gt;low levels&lt;/strong&gt;. Our hypothesis is &lt;strong&gt;to prove if mutations in 10 genes can be associated with these physiological low levels.&lt;/strong&gt; These physiological low levels are quantified under the format of maximum and minimum value of B12. The genes can take 3 possible categories:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;wt: absence of mutation&lt;/li&gt;&#xA;&lt;li&gt;het: half of mutation&lt;/li&gt;&#xA;&lt;li&gt;hom: full mutation&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Hypothesis:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I would like, if possible, to try something like a genetic risk score, &lt;strong&gt;if multiple het or hom states&lt;/strong&gt; in multiple genes can be additive. I don't know how to approach this, because there can be multiple permutations, and maybe permutation 1 has a stronger association with low levels than permutation 2.&lt;/li&gt;&#xA;&lt;li&gt;Any ideas on how to carry this out? I don't really know if mutations are additive or opposite.&#xA;Not sure if some clustering approach can be helpful here, like PCA or euclidean distance, linear discriminant analysis... Any advice is welcome.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;My idea is to execute a summary of levels of B12 for each gene and see in a boxplot possible differences, perhaps look for some association (or correlation), but I doubt it.&lt;/p&gt;&#xA;&lt;p&gt;Any idea or previous input or experiment like this would be tremendously welcome.&lt;/p&gt;&#xA;" OwnerUserId="16535" LastEditorUserId="298" LastEditDate="2023-05-19T18:00:13.033" LastActivityDate="2023-10-11T15:39:33.887" Title="How to associate point mutations to quantitative protein levels" Tags="|point-mutation|polygenic-risk-score|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21036" PostTypeId="2" ParentId="21034" CreationDate="2023-05-19T02:38:49.023" Score="0" Body="&lt;p&gt;Personally I would not use &lt;code&gt;intercnv_obj&lt;/code&gt; throughout, because is there was a bug in line 2, i.e. using &lt;code&gt;=&lt;/code&gt; instead of &lt;code&gt;&amp;lt;-&lt;/code&gt; then the created &lt;code&gt;intercnv_obj&lt;/code&gt; might be trying to be passed into &lt;code&gt;plot_cnv&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Could you try&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;data(infercnv_obj) &#xA;&#xA;plot_cnv(&#xA;  infercnv_obj,&#xA;  out_dir = &amp;quot;.&amp;quot;,&#xA;  title = &amp;quot;inferCNV&amp;quot;,&#xA;  obs_title = &amp;quot;Observations (Cells)&amp;quot;,&#xA;  ref_title = &amp;quot;References (Cells)&amp;quot;,&#xA;  cluster_by_groups = TRUE,&#xA;  cluster_references = TRUE,&#xA;  plot_chr_scale = FALSE,&#xA;  chr_lengths = NULL,&#xA;  k_obs_groups = 1,&#xA;  contig_cex = 1,&#xA;  x.center = mean(infercnv_obj@expr.data),&#xA;  x.range = &amp;quot;auto&amp;quot;,&#xA;  hclust_method = &amp;quot;ward.D&amp;quot;,&#xA;  custom_color_pal = NULL,&#xA;  color_safe_pal = FALSE,&#xA;  output_filename = &amp;quot;infercnv&amp;quot;,&#xA;  output_format = &amp;quot;png&amp;quot;,&#xA;  png_res = 300,&#xA;  dynamic_resize = 0,&#xA;  ref_contig = NULL,&#xA;  write_expr_matrix = FALSE,&#xA;  write_phylo = FALSE,&#xA;  useRaster = TRUE&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;If the above doesn't work ...&lt;/p&gt;&#xA;&lt;p&gt;What I think is happening is the input data is the problem. This precise error is already known and was reported on Broad's GitHub here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/broadinstitute/infercnv/issues/415&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/broadinstitute/infercnv/issues/415&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The responder (Broad Institute) believed it was the version problem, i.e. the version was outdated. However a separate respondee believed it was the input data format and suggested shifting away from a sparse matrix. That would make sense in context to the error being thrown.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-07-04T14:57:28.973" LastActivityDate="2023-07-04T14:57:28.973" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21037" PostTypeId="1" CreationDate="2023-05-19T07:08:33.150" Score="3" ViewCount="36" Body="&lt;p&gt;I am trying to reproduce an analysis result, and there is a requirement regarding the format of the result: the '3' rule ': all mutations should be expressed in a position close to the end of the gene transcription direction (3' end).&lt;/p&gt;&#xA;&lt;p&gt;The result I obtained is(annotate from VEP)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Chromosome  Start_Position  End_Position    Strand  Variant_Classification  Variant_Type    Reference_Allele    Tumor_Seq_Allele1   Tumor_Seq_Allele2   HGVSc   HGVSp   HGVSp_Short Transcript_ID&#xA;10  89692835    89692840    +   In_Frame_Del    DEL GATCTT  GATCTT  -   c.321_326del    p.Leu108_Asp109del  p.L108_D109del  NM_000314.8&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;the standard answer&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr10   89692837    89692842    TCTTGA      PTEN    NM_00314    c.321_326del    p.Leu108_Asp109del&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I speculate that ：&#xA;the HGVS standard followed by VEP includes the 3'rule . So ultimately, there is the same &lt;code&gt;p.Leu108_ Asp109del&lt;/code&gt;。 But the output of vep did not adjust Start_ Position and End_ Position？（Perhaps it thinks that after obtaining the correct amino acid mutation , it doesn't matter anymore）&#xA;But for me, I hope the ultimate Start_ Position and End_ The Position also complies with the regulations, which seems a bit troublesome because the different  transcripts may have different directions and always rely on the IGV to correct it , which is also tiring.&lt;/p&gt;&#xA;&lt;p&gt;Thanks for any suggestions&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Eyk9F.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Eyk9F.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have added an IGV image to better illustrate the problem. Reference genome regions 89692830 to 89692845 for the example in the figure.&#xA;bwa(or other software，like bowtie2) believes that the position of del should be&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ref:GTGAAGATCTTGACCA&#xA;alt:GTGAA------GACCA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But 3'rule believes that the position of del should be&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ref:GTGAAGATCTTGACCA&#xA;alt:GTGAAGA------CCA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To verify whether the results match 3'rule, it is necessary to know the direction of the transcripts, and each gene always has multiple transcripts, which is where I am confused&lt;/p&gt;&#xA;" OwnerUserId="15063" LastEditorUserId="298" LastEditDate="2023-05-19T11:53:31.803" LastActivityDate="2023-05-19T11:53:31.803" Title="How to normalise indel of different transcript direction?" Tags="|indel|vep|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21038" PostTypeId="1" AcceptedAnswerId="21039" CreationDate="2023-05-19T18:01:01.587" Score="4" ViewCount="253" Body="&lt;p&gt;Inspired by &lt;a href=&quot;https://www.nature.com/articles/d41586-023-01047-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;amateur variant hunters&lt;/a&gt;, I would like to join the Pango lineage proposal community and help contribute to variant surveillance.&lt;/p&gt;&#xA;&lt;p&gt;However, I cannot seem to get access to GISAID, the platform that exclusively hosts a large share of SARS-CoV-2 sequences from around the world.&lt;/p&gt;&#xA;&lt;p&gt;I applied/registered for an account back in summer 2022 (almost a year ago) using this form: &lt;a href=&quot;https://gisaid.org/register/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gisaid.org/register/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But since then, nothing has happened. I have send an email and filled out the contact form on the website but never heard back.&lt;/p&gt;&#xA;&lt;p&gt;I am very confused how GISAID can say they are a &amp;quot;public&amp;quot; database, while at the same time not giving accounts to people like me, not even responding to emails.&lt;/p&gt;&#xA;&lt;p&gt;How does everyone else get access? Is there a secret email you need to contact? Or do you need to pay? I've read &lt;a href=&quot;https://www.science.org/content/article/invented-persona-behind-key-pandemic-database&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; which describes the founder of GISAID in detail.&lt;/p&gt;&#xA;&lt;p&gt;Any help would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="17518" LastEditorUserId="3848" LastEditDate="2023-07-18T21:47:47.227" LastActivityDate="2023-07-18T21:47:47.227" Title="How to get a GISAID account? I registered months ago, still no reply!" Tags="|phylogenetics|phylogeny|public-databases|sars-cov-2|variants|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21039" PostTypeId="2" ParentId="21038" CreationDate="2023-05-19T18:20:20.300" Score="3" Body="&lt;p&gt;I've got a GISAID account and it's subject to an approval process. It was pretty simple and is awarded quickly. There are definitely no fees needed. I personally didn't see the advantages over NCBI's really very impressive SARS-CoV-2 database system and thats what I'd recommend.&lt;/p&gt;&#xA;&lt;p&gt;Their resource is here:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/labs/virus/vssi/#/virus?SeqType_s=Nucleotide&amp;amp;VirusLineage_ss=SARS-CoV-2,%20taxid:2697049&amp;amp;ids=MZ039713%20MZ039682%20OV522182%20&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/labs/virus/vssi/#/virus?SeqType_s=Nucleotide&amp;amp;VirusLineage_ss=SARS-CoV-2,%20taxid:2697049&amp;amp;ids=MZ039713%20MZ039682%20OV522182%20&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Its perfect for interrogation, it has the full spectrum of epidemiological metadata, and further details are in this post here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/18642/how-to-programatically-download-sars-cov-2-fasta-from-ncbi-genbank-via-api/18643#18643&quot;&gt;How to programatically download SARS-CoV-2 fasta from NCBI/Genbank via API?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The Science journal page seemed a very good write up of GISAID BTW.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;If you are wanting GISAID for networking, thats a bit tricky because NCBI is not going to do that.&#xA;You can reapply for GISAID access, but at the time I was much more impressed by NCBI. The speed at which NCBI produced an awesome database was truly jaw-dropping and they'd implemented full pangolineages classification at an early stage. Thus I just use NCBI.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-05-19T21:51:08.203" LastActivityDate="2023-05-19T21:51:08.203" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21040" PostTypeId="2" ParentId="21032" CreationDate="2023-05-19T20:20:01.800" Score="1" Body="&lt;p&gt;You didn't show us how the original .h5ad object was generated, how you read it in, how you wrote it, and if you did anything else in-between besides filtering. &lt;a href=&quot;https://anndata.readthedocs.io/en/latest/generated/anndata.AnnData.write.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;AnnData.write()&lt;/code&gt;&lt;/a&gt; takes a &lt;code&gt;compression&lt;/code&gt; arg that might explain the difference. As also mentioned in that link, ensuring you are writing a sparse matrix can make a substantial difference to file size (the matrix in &lt;code&gt;adata.X&lt;/code&gt; is generally by far the largest part of the object)&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2023-05-19T20:20:01.800" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21041" PostTypeId="1" CreationDate="2023-05-20T00:11:45.333" Score="1" ViewCount="13" Body="&lt;p&gt;I can calculate total rosetta energy score of a peptide-protein complex. However, when using flexpepdock module from pyrosetta I didn't find anything direct parameter like pep_sc or pep_sc_noref in the pyrosetta module. I try to write a workaround for pep_sc.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from pyrosetta import rosetta&#xA;&#xA;# Initialize PyRosetta&#xA;pyrosetta.init()&#xA;&#xA;# Load the protein-peptide complex from the PDB file&#xA;pose = rosetta.core.import_pose.pose_from_file(&amp;quot;./all_pdb/clean_pdb/1d4t.clean.pdb&amp;quot;)&#xA;&#xA;# Create a score function&#xA;scorefxn = rosetta.core.scoring.ScoreFunction()&#xA;&#xA;# Set up the FlexPepDock protocol&#xA;flexpep_dock = rosetta.protocols.flexpep_docking.FlexPepDockingProtocol()&#xA;&#xA;# Apply the FlexPepDock protocol&#xA;flexpep_dock.apply(pose)&#xA;&#xA;# Get the energy terms of interest&#xA;score = pose.energies().total_energy()&#xA;hbond_sc = pose.energies().residue_total_energies(1)[rosetta.core.scoring.hbond_sc]&#xA;rama = pose.energies().residue_total_energies(1)[rosetta.core.scoring.rama]&#xA;pep_sc = pose.energies().residue_total_energies(2)[rosetta.core.scoring.fa_atr] + \&#xA;         pose.energies().residue_total_energies(2)[rosetta.core.scoring.fa_rep] + \&#xA;         pose.energies().residue_total_energies(2)[rosetta.core.scoring.fa_sol] + \&#xA;         pose.energies().residue_total_energies(2)[rosetta.core.scoring.fa_elec]&#xA;         &#xA;fa_atr = pose.energies().residue_total_energies(1)[rosetta.core.scoring.fa_atr]&#xA;fa_pair = pose.energies().residue_total_energies(1)[rosetta.core.scoring.fa_pair]&#xA;#pep_sc_noref = pose.energies().residue_total_energies(1)[rosetta.core.scoring.pep_sc_noref]&#xA;fa_rep = pose.energies().residue_total_energies(1)[rosetta.core.scoring.fa_rep]&#xA;fa_dun = pose.energies().residue_total_energies(1)[rosetta.core.scoring.fa_dun]&#xA;fa_sol = pose.energies().residue_total_energies(1)[rosetta.core.scoring.fa_sol]&#xA;#startRMSbb = pose.energies().residue_total_energies(1)[rosetta.core.scoring.startRMSbb]&#xA;# Calculate I_sc (Interface energy)&#xA;complex_energy = pose.energies().total_energies()[rosetta.core.scoring.total_score]&#xA;protein_energy = pose.energies().residue_total_energies(1)[rosetta.core.scoring.total_score]&#xA;peptide_energy = pose.energies().residue_total_energies(2)[rosetta.core.scoring.total_score]&#xA;I_sc = complex_energy - (protein_energy + peptide_energy)&#xA;&#xA;# Print the energy terms&#xA;print(&amp;quot;Score:&amp;quot;, score)&#xA;print(&amp;quot;hbond_sc:&amp;quot;, hbond_sc)&#xA;print(&amp;quot;rama:&amp;quot;, rama)&#xA;print(&amp;quot;I_sc:&amp;quot;, I_sc)&#xA;print(&amp;quot;pep_sc:&amp;quot;, pep_sc)&#xA;print(&amp;quot;fa_atr:&amp;quot;, fa_atr)&#xA;print(&amp;quot;fa_pair:&amp;quot;, fa_pair)&#xA;print(&amp;quot;fa_rep:&amp;quot;, fa_rep)&#xA;print(&amp;quot;fa_dun:&amp;quot;, fa_dun)&#xA;print(&amp;quot;fa_sol:&amp;quot;, fa_sol)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can anyone tell me how to calculate pep_sc and pep_sc_noref for this code? Also, can anyone tell me what is the formula for pep_sc and pep_sc_noref if there is no direct reference .&lt;/p&gt;&#xA;" OwnerUserId="10296" LastActivityDate="2023-05-20T00:11:45.333" Title="Define and calculate pep_sc and pep_sc_noref in pyrosetta" Tags="|docking|rosetta|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21042" PostTypeId="1" AcceptedAnswerId="21052" CreationDate="2023-05-20T10:18:28.167" Score="1" ViewCount="124" Body="&lt;p&gt;I have run OrthoFinder on a set of 11 genomes and got the results in a folder. From the output folder, I saved all the single-copy gene sequences in a single FASTA file and now wish to remove all gaps using &lt;strong&gt;Gblocks&lt;/strong&gt; before using them for constructing a phylogenomic tree. But I don't know the command line used for running &lt;strong&gt;Gblocks&lt;/strong&gt;. Any help will be appreciated!&lt;/p&gt;&#xA;" OwnerUserId="17392" LastActivityDate="2023-05-22T14:27:13.477" Title="How to use Gblocks for trimming single-copy gene sequences?" Tags="|phylogenetics|phylogeny|linux|orthologues|orthofinder|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21043" PostTypeId="1" CreationDate="2023-05-20T22:37:11.933" Score="2" ViewCount="54" Body="&lt;p&gt;I'm trying to create a shiny app for single cell data visualization and then label cells with genes till this step it works, the final step when I try to export cluster it is not exporting the desired cluster or i might be doing something wrong which Im not sure&lt;/p&gt;&#xA;&lt;p&gt;This the code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(Seurat)&#xA;library(BiocParallel)&#xA;library(reticulate)&#xA;library(Matrix)&#xA;library(shiny)&#xA;library(dplyr)&#xA;&#xA;# Set the maximum file upload size to 3 GB&#xA;options(shiny.maxRequestSize = 3e+9)&#xA;&#xA;# Define the UI&#xA;ui &amp;lt;- fluidPage(&#xA;  titlePanel(&amp;quot;Single-cell RNA-seq Analysis&amp;quot;),&#xA;  sidebarLayout(&#xA;    sidebarPanel(&#xA;      fileInput(&amp;quot;h5ad_file&amp;quot;, &amp;quot;Upload H5AD File&amp;quot;, accept = &amp;quot;.h5ad&amp;quot;),&#xA;      actionButton(&amp;quot;run_button&amp;quot;, &amp;quot;Run Analysis&amp;quot;),&#xA;      selectInput(&amp;quot;gene_input&amp;quot;, &amp;quot;Select Gene for Feature Plot:&amp;quot;, choices = NULL),&#xA;      downloadButton(&amp;quot;export_button&amp;quot;, &amp;quot;Export Cluster&amp;quot;)&#xA;    ),&#xA;    mainPanel(&#xA;      plotOutput(&amp;quot;feature_plot&amp;quot;)&#xA;    )&#xA;  )&#xA;)&#xA;&#xA;# Define the server&#xA;server &amp;lt;- function(input, output, session) {&#xA;  &#xA;  # Function to perform the analysis&#xA;  performAnalysis &amp;lt;- function(h5ad_file) {&#xA;    sc &amp;lt;- import(&amp;quot;scanpy&amp;quot;)&#xA;    adata &amp;lt;- sc$read_h5ad(h5ad_file)&#xA;    &#xA;    counts &amp;lt;- t(adata&lt;span class=&quot;math-container&quot;&gt;$layers[&quot;counts&quot;])&#xA;    colnames(counts) &amp;lt;- adata$&lt;/span&gt;obs_names&lt;span class=&quot;math-container&quot;&gt;$to_list()&#xA;    rownames(counts) &amp;lt;- adata$&lt;/span&gt;var_names$to_list()&#xA;    counts &amp;lt;- Matrix::Matrix(as.matrix(counts), sparse = TRUE)&#xA;    &#xA;    data &amp;lt;- t(adata&lt;span class=&quot;math-container&quot;&gt;$layers[&quot;winsorized&quot;])&#xA;    colnames(data) &amp;lt;- adata$&lt;/span&gt;obs_names&lt;span class=&quot;math-container&quot;&gt;$to_list()&#xA;    rownames(data) &amp;lt;- adata$&lt;/span&gt;var_names$to_list()&#xA;    data &amp;lt;- Matrix::Matrix(as.matrix(data), sparse = TRUE)&#xA;    &#xA;    seurat &amp;lt;- CreateSeuratObject(counts)&#xA;    seurat &amp;lt;- SetAssayData(seurat, &amp;quot;data&amp;quot;, data)&#xA;    seurat &amp;lt;- AddMetaData(seurat, adata$obs)&#xA;    &#xA;    seurat[[&amp;quot;percent.mt&amp;quot;]] &amp;lt;- PercentageFeatureSet(seurat, pattern = &amp;quot;^MT[-\\.]&amp;quot;)&#xA;    seurat &amp;lt;- NormalizeData(seurat)&#xA;    seurat &amp;lt;- FindVariableFeatures(seurat, nfeatures = 500)&#xA;    seurat &amp;lt;- ScaleData(seurat)&#xA;    seurat &amp;lt;- ScaleData(seurat, vars.to.regress = c(&amp;quot;nFeature_RNA&amp;quot;, &amp;quot;percent.mt&amp;quot;))&#xA;    seurat &amp;lt;- RunPCA(seurat, npcs = 50)&#xA;    seurat &amp;lt;- RunUMAP(seurat, dims = 1:20)&#xA;    seurat &amp;lt;- FindNeighbors(seurat, dims = 1:20)&#xA;    seurat &amp;lt;- FindClusters(seurat, resolution = 1)&#xA;    cl_markers &amp;lt;- FindAllMarkers(seurat, only.pos = TRUE, min.pct = 0.25, logfc.threshold = log(1.2))&#xA;    top_gene &amp;lt;- cl_markers %&amp;gt;% group_by(cluster) %&amp;gt;% top_n(n = 2, wt = avg_log2FC)&#xA;    &#xA;    return(seurat)&#xA;  }&#xA;  &#xA;  # Reactive value to store the Seurat object&#xA;  seurat_object &amp;lt;- reactiveVal(NULL)&#xA;  &#xA;  # Perform analysis when the Run Analysis button is clicked&#xA;  observeEvent(input&lt;span class=&quot;math-container&quot;&gt;$run_button, {&#xA;    req(input$h5ad_file)&#xA;    h5ad_file &amp;lt;- input$h5ad_file$&lt;/span&gt;datapath&#xA;    seurat &amp;lt;- performAnalysis(h5ad_file)&#xA;    seurat_object(seurat)&#xA;    &#xA;    # Update gene choices for feature plot&#xA;    gene_choices &amp;lt;- rownames(seurat@assays$RNA@counts)&#xA;    updateSelectInput(session, &amp;quot;gene_input&amp;quot;, choices = gene_choices)&#xA;  })&#xA;  &#xA;  # Render the feature plot&#xA;  output&lt;span class=&quot;math-container&quot;&gt;$feature_plot &amp;lt;- renderPlot({&#xA;    seurat &amp;lt;- seurat_object()&#xA;    if (!is.null(seurat)) {&#xA;      gene &amp;lt;- input$&lt;/span&gt;gene_input&#xA;      FeaturePlot(seurat, features = gene, label = TRUE)&#xA;    }&#xA;  })&#xA;  &#xA;  # Export cluster when the Export Cluster button is clicked&#xA;  observeEvent(input&lt;span class=&quot;math-container&quot;&gt;$export_button, {&#xA;    seurat &amp;lt;- seurat_object()&#xA;    if (!is.null(seurat)) {&#xA;      cluster_data &amp;lt;- cbind(Cluster = seurat@meta.data$cluster, seurat@assays$&lt;/span&gt;RNA@counts)&#xA;      write.csv(cluster_data, &amp;quot;cluster.csv&amp;quot;, row.names = TRUE)&#xA;      updateActionButton(session, &amp;quot;export_button&amp;quot;, label = &amp;quot;Download Cluster&amp;quot;, href = &amp;quot;cluster.csv&amp;quot;)&#xA;    }&#xA;  })&#xA;  &#xA;  &#xA;  &#xA;  &#xA;  # Function to end the session&#xA;  endSession &amp;lt;- function() {&#xA;    stopApp()  # This will end the Shiny session and shut down the server&#xA;  }&#xA;  &#xA;  # Call the endSession function when the session ends (browser window closed)&#xA;  session$onSessionEnded(endSession)&#xA;}&#xA;&#xA;# Run the Shiny app&#xA;shinyApp(ui = ui, server = server)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;strong&gt;seurat&lt;/strong&gt; object which is generated its structure is like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;str(seurat)&#xA;Formal class 'Seurat' [package &amp;quot;SeuratObject&amp;quot;] with 13 slots&#xA;  ..@ assays      :List of 1&#xA;  .. ..&lt;span class=&quot;math-container&quot;&gt;$ RNA:Formal class 'Assay' [package &quot;SeuratObject&quot;] with 8 slots&#xA;  .. .. .. ..@ counts       :Formal class 'dgCMatrix' [package &quot;Matrix&quot;] with 6 slots&#xA;  .. .. .. .. .. ..@ i       : int [1:2238732] 29 73 80 148 163 184 186 227 229 230 ...&#xA;  .. .. .. .. .. ..@ p       : int [1:2639] 0 779 2131 3260 4220 4741 5522 6304 7094 7626 ...&#xA;  .. .. .. .. .. ..@ Dim     : int [1:2] 13714 2638&#xA;  .. .. .. .. .. ..@ Dimnames:List of 2&#xA;  .. .. .. .. .. .. ..$&lt;/span&gt; : chr [1:13714] &amp;quot;AL627309.1&amp;quot; &amp;quot;AP006222.2&amp;quot; &amp;quot;RP11-206L10.2&amp;quot; &amp;quot;RP11-206L10.9&amp;quot; ...&#xA;  .. .. .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ : chr [1:2638] &quot;AAACATACAACCAC-1&quot; &quot;AAACATTGAGCTAC-1&quot; &quot;AAACATTGATCAGC-1&quot; &quot;AAACCGTGCTTCCG-1&quot; ...&#xA;  .. .. .. .. .. ..@ x       : num [1:2238732] 1 1 2 1 1 1 1 41 1 1 ...&#xA;  .. .. .. .. .. ..@ factors : list()&#xA;  .. .. .. ..@ data         :Formal class 'dgCMatrix' [package &quot;Matrix&quot;] with 6 slots&#xA;  .. .. .. .. .. ..@ i       : int [1:2238732] 29 73 80 148 163 184 186 227 229 230 ...&#xA;  .. .. .. .. .. ..@ p       : int [1:2639] 0 779 2131 3260 4220 4741 5522 6304 7094 7626 ...&#xA;  .. .. .. .. .. ..@ Dim     : int [1:2] 13714 2638&#xA;  .. .. .. .. .. ..@ Dimnames:List of 2&#xA;  .. .. .. .. .. .. ..$&lt;/span&gt; : chr [1:13714] &amp;quot;AL627309.1&amp;quot; &amp;quot;AP006222.2&amp;quot; &amp;quot;RP11-206L10.2&amp;quot; &amp;quot;RP11-206L10.9&amp;quot; ...&#xA;  .. .. .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ : chr [1:2638] &quot;AAACATACAACCAC-1&quot; &quot;AAACATTGAGCTAC-1&quot; &quot;AAACATTGATCAGC-1&quot; &quot;AAACCGTGCTTCCG-1&quot; ...&#xA;  .. .. .. .. .. ..@ x       : num [1:2238732] 1.64 1.64 2.23 1.64 1.64 ...&#xA;  .. .. .. .. .. ..@ factors : list()&#xA;  .. .. .. ..@ scale.data   : num [1:500, 1:2638] 1.5265 -0.4982 -0.0324 -0.3129 -0.4631 ...&#xA;  .. .. .. .. ..- attr(*, &quot;dimnames&quot;)=List of 2&#xA;  .. .. .. .. .. ..$&lt;/span&gt; : chr [1:500] &amp;quot;MRPL20&amp;quot; &amp;quot;RER1&amp;quot; &amp;quot;TNFRSF9&amp;quot; &amp;quot;SRM&amp;quot; ...&#xA;  .. .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ : chr [1:2638] &quot;AAACATACAACCAC-1&quot; &quot;AAACATTGAGCTAC-1&quot; &quot;AAACATTGATCAGC-1&quot; &quot;AAACCGTGCTTCCG-1&quot; ...&#xA;  .. .. .. ..@ key          : chr &quot;rna_&quot;&#xA;  .. .. .. ..@ assay.orig   : NULL&#xA;  .. .. .. ..@ var.features : chr [1:500] &quot;PPBP&quot; &quot;LYZ&quot; &quot;S100A9&quot; &quot;IGLL5&quot; ...&#xA;  .. .. .. ..@ meta.features:'data.frame':  13714 obs. of  5 variables:&#xA;  .. .. .. .. ..$&lt;/span&gt; vst.mean                 : num [1:13714] 0.00341 0.00114 0.0019 0.00114 0.00682 ...&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ vst.variance             : num [1:13714] 0.0034 0.00114 0.00189 0.00114 0.00678 ...&#xA;  .. .. .. .. ..$&lt;/span&gt; vst.variance.expected    : num [1:13714] 0.00365 0.00114 0.00197 0.00114 0.00748 ...&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ vst.variance.standardized: num [1:13714] 0.933 0.992 0.963 0.992 0.906 ...&#xA;  .. .. .. .. ..$&lt;/span&gt; vst.variable             : logi [1:13714] FALSE FALSE FALSE FALSE FALSE FALSE ...&#xA;  .. .. .. ..@ misc         : list()&#xA;  ..@ meta.data   :'data.frame':    2638 obs. of  10 variables:&#xA;  .. ..&lt;span class=&quot;math-container&quot;&gt;$ orig.ident     : Factor w/ 1 level &quot;SeuratProject&quot;: 1 1 1 1 1 1 1 1 1 1 ...&#xA;  .. ..$&lt;/span&gt; nCount_RNA     : num [1:2638] 2419 4903 3147 2639 980 ...&#xA;  .. ..&lt;span class=&quot;math-container&quot;&gt;$ nFeature_RNA   : int [1:2638] 779 1352 1129 960 521 781 782 790 532 550 ...&#xA;  .. ..$&lt;/span&gt; n_genes        : num [1:2638] 781 1352 1131 960 522 ...&#xA;  .. ..&lt;span class=&quot;math-container&quot;&gt;$ percent_mito   : num [1:2638] 0.0302 0.0379 0.0089 0.0174 0.0122 ...&#xA;  .. ..$&lt;/span&gt; n_counts       : num [1:2638] 2419 4903 3147 2639 980 ...&#xA;  .. ..&lt;span class=&quot;math-container&quot;&gt;$ louvain        : Factor w/ 8 levels &quot;CD4 T cells&quot;,..: 1 3 1 2 5 4 4 4 1 6 ...&#xA;  .. ..$&lt;/span&gt; percent.mt     : num [1:2638] 3.02 3.79 0.89 1.74 1.22 ...&#xA;  .. ..&lt;span class=&quot;math-container&quot;&gt;$ RNA_snn_res.1  : Factor w/ 8 levels &quot;0&quot;,&quot;1&quot;,&quot;2&quot;,&quot;3&quot;,..: 1 3 1 6 4 1 5 5 5 6 ...&#xA;  .. ..$&lt;/span&gt; seurat_clusters: Factor w/ 8 levels &amp;quot;0&amp;quot;,&amp;quot;1&amp;quot;,&amp;quot;2&amp;quot;,&amp;quot;3&amp;quot;,..: 1 3 1 6 4 1 5 5 5 6 ...&#xA;  ..@ active.assay: chr &amp;quot;RNA&amp;quot;&#xA;  ..@ active.ident: Factor w/ 8 levels &amp;quot;0&amp;quot;,&amp;quot;1&amp;quot;,&amp;quot;2&amp;quot;,&amp;quot;3&amp;quot;,..: 1 3 1 6 4 1 5 5 5 6 ...&#xA;  .. ..- attr(*, &amp;quot;names&amp;quot;)= chr [1:2638] &amp;quot;AAACATACAACCAC-1&amp;quot; &amp;quot;AAACATTGAGCTAC-1&amp;quot; &amp;quot;AAACATTGATCAGC-1&amp;quot; &amp;quot;AAACCGTGCTTCCG-1&amp;quot; ...&#xA;  ..@ graphs      :List of 2&#xA;  .. ..&lt;span class=&quot;math-container&quot;&gt;$ RNA_nn :Formal class 'Graph' [package &quot;SeuratObject&quot;] with 7 slots&#xA;  .. .. .. ..@ assay.used: chr &quot;RNA&quot;&#xA;  .. .. .. ..@ i         : int [1:52760] 0 102 187 274 457 833 1147 1503 1506 1550 ...&#xA;  .. .. .. ..@ p         : int [1:2639] 0 17 24 30 33 40 61 83 85 96 ...&#xA;  .. .. .. ..@ Dim       : int [1:2] 2638 2638&#xA;  .. .. .. ..@ Dimnames  :List of 2&#xA;  .. .. .. .. ..$&lt;/span&gt; : chr [1:2638] &amp;quot;AAACATACAACCAC-1&amp;quot; &amp;quot;AAACATTGAGCTAC-1&amp;quot; &amp;quot;AAACATTGATCAGC-1&amp;quot; &amp;quot;AAACCGTGCTTCCG-1&amp;quot; ...&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ : chr [1:2638] &quot;AAACATACAACCAC-1&quot; &quot;AAACATTGAGCTAC-1&quot; &quot;AAACATTGATCAGC-1&quot; &quot;AAACCGTGCTTCCG-1&quot; ...&#xA;  .. .. .. ..@ x         : num [1:52760] 1 1 1 1 1 1 1 1 1 1 ...&#xA;  .. .. .. ..@ factors   : list()&#xA;  .. ..$&lt;/span&gt; RNA_snn:Formal class 'Graph' [package &amp;quot;SeuratObject&amp;quot;] with 7 slots&#xA;  .. .. .. ..@ assay.used: chr &amp;quot;RNA&amp;quot;&#xA;  .. .. .. ..@ i         : int [1:245992] 0 69 102 172 187 189 248 274 276 315 ...&#xA;  .. .. .. ..@ p         : int [1:2639] 0 79 118 179 253 323 424 541 574 650 ...&#xA;  .. .. .. ..@ Dim       : int [1:2] 2638 2638&#xA;  .. .. .. ..@ Dimnames  :List of 2&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ : chr [1:2638] &quot;AAACATACAACCAC-1&quot; &quot;AAACATTGAGCTAC-1&quot; &quot;AAACATTGATCAGC-1&quot; &quot;AAACCGTGCTTCCG-1&quot; ...&#xA;  .. .. .. .. ..$&lt;/span&gt; : chr [1:2638] &amp;quot;AAACATACAACCAC-1&amp;quot; &amp;quot;AAACATTGAGCTAC-1&amp;quot; &amp;quot;AAACATTGATCAGC-1&amp;quot; &amp;quot;AAACCGTGCTTCCG-1&amp;quot; ...&#xA;  .. .. .. ..@ x         : num [1:245992] 1 0.0811 0.2903 0.1765 0.1765 ...&#xA;  .. .. .. ..@ factors   : list()&#xA;  ..@ neighbors   : list()&#xA;  ..@ reductions  :List of 2&#xA;  .. ..&lt;span class=&quot;math-container&quot;&gt;$ pca :Formal class 'DimReduc' [package &quot;SeuratObject&quot;] with 9 slots&#xA;  .. .. .. ..@ cell.embeddings           : num [1:2638, 1:50] -3.544 -5.892 -3.544 10.139 0.932 ...&#xA;  .. .. .. .. ..- attr(*, &quot;dimnames&quot;)=List of 2&#xA;  .. .. .. .. .. ..$&lt;/span&gt; : chr [1:2638] &amp;quot;AAACATACAACCAC-1&amp;quot; &amp;quot;AAACATTGAGCTAC-1&amp;quot; &amp;quot;AAACATTGATCAGC-1&amp;quot; &amp;quot;AAACCGTGCTTCCG-1&amp;quot; ...&#xA;  .. .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ : chr [1:50] &quot;PC_1&quot; &quot;PC_2&quot; &quot;PC_3&quot; &quot;PC_4&quot; ...&#xA;  .. .. .. ..@ feature.loadings          : num [1:500, 1:50] 0.02701 0.16566 0.17077 -0.00786 -0.03674 ...&#xA;  .. .. .. .. ..- attr(*, &quot;dimnames&quot;)=List of 2&#xA;  .. .. .. .. .. ..$&lt;/span&gt; : chr [1:500] &amp;quot;PPBP&amp;quot; &amp;quot;LYZ&amp;quot; &amp;quot;S100A9&amp;quot; &amp;quot;IGLL5&amp;quot; ...&#xA;  .. .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ : chr [1:50] &quot;PC_1&quot; &quot;PC_2&quot; &quot;PC_3&quot; &quot;PC_4&quot; ...&#xA;  .. .. .. ..@ feature.loadings.projected: num[0 , 0 ] &#xA;  .. .. .. ..@ assay.used                : chr &quot;RNA&quot;&#xA;  .. .. .. ..@ global                    : logi FALSE&#xA;  .. .. .. ..@ stdev                     : num [1:50] 5 3.39 3.2 2.78 1.94 ...&#xA;  .. .. .. ..@ key                       : chr &quot;PC_&quot;&#xA;  .. .. .. ..@ jackstraw                 :Formal class 'JackStrawData' [package &quot;SeuratObject&quot;] with 4 slots&#xA;  .. .. .. .. .. ..@ empirical.p.values     : num[0 , 0 ] &#xA;  .. .. .. .. .. ..@ fake.reduction.scores  : num[0 , 0 ] &#xA;  .. .. .. .. .. ..@ empirical.p.values.full: num[0 , 0 ] &#xA;  .. .. .. .. .. ..@ overall.p.values       : num[0 , 0 ] &#xA;  .. .. .. ..@ misc                      :List of 1&#xA;  .. .. .. .. ..$&lt;/span&gt; total.variance: num 457&#xA;  .. ..&lt;span class=&quot;math-container&quot;&gt;$ umap:Formal class 'DimReduc' [package &quot;SeuratObject&quot;] with 9 slots&#xA;  .. .. .. ..@ cell.embeddings           : num [1:2638, 1:2] -3.44 -7.94 -1.77 8.79 -1.51 ...&#xA;  .. .. .. .. ..- attr(*, &quot;scaled:center&quot;)= num [1:2] 1.64 -1.3&#xA;  .. .. .. .. ..- attr(*, &quot;dimnames&quot;)=List of 2&#xA;  .. .. .. .. .. ..$&lt;/span&gt; : chr [1:2638] &amp;quot;AAACATACAACCAC-1&amp;quot; &amp;quot;AAACATTGAGCTAC-1&amp;quot; &amp;quot;AAACATTGATCAGC-1&amp;quot; &amp;quot;AAACCGTGCTTCCG-1&amp;quot; ...&#xA;  .. .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ : chr [1:2] &quot;UMAP_1&quot; &quot;UMAP_2&quot;&#xA;  .. .. .. ..@ feature.loadings          : num[0 , 0 ] &#xA;  .. .. .. ..@ feature.loadings.projected: num[0 , 0 ] &#xA;  .. .. .. ..@ assay.used                : chr &quot;RNA&quot;&#xA;  .. .. .. ..@ global                    : logi TRUE&#xA;  .. .. .. ..@ stdev                     : num(0) &#xA;  .. .. .. ..@ key                       : chr &quot;UMAP_&quot;&#xA;  .. .. .. ..@ jackstraw                 :Formal class 'JackStrawData' [package &quot;SeuratObject&quot;] with 4 slots&#xA;  .. .. .. .. .. ..@ empirical.p.values     : num[0 , 0 ] &#xA;  .. .. .. .. .. ..@ fake.reduction.scores  : num[0 , 0 ] &#xA;  .. .. .. .. .. ..@ empirical.p.values.full: num[0 , 0 ] &#xA;  .. .. .. .. .. ..@ overall.p.values       : num[0 , 0 ] &#xA;  .. .. .. ..@ misc                      : list()&#xA;  ..@ images      : list()&#xA;  ..@ project.name: chr &quot;SeuratProject&quot;&#xA;  ..@ misc        : list()&#xA;  ..@ version     :Classes 'package_version', 'numeric_version'  hidden list of 1&#xA;  .. ..$&lt;/span&gt; : int [1:3] 4 1 3&#xA;  ..@ commands    :List of 7&#xA;  .. ..&lt;span class=&quot;math-container&quot;&gt;$ NormalizeData.RNA       :Formal class 'SeuratCommand' [package &quot;SeuratObject&quot;] with 5 slots&#xA;  .. .. .. ..@ name       : chr &quot;NormalizeData.RNA&quot;&#xA;  .. .. .. ..@ time.stamp : POSIXct[1:1], format: &quot;2023-05-21 03:58:21&quot;&#xA;  .. .. .. ..@ assay.used : chr &quot;RNA&quot;&#xA;  .. .. .. ..@ call.string: chr &quot;NormalizeData(seurat)&quot;&#xA;  .. .. .. ..@ params     :List of 5&#xA;  .. .. .. .. ..$&lt;/span&gt; assay               : chr &amp;quot;RNA&amp;quot;&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ normalization.method: chr &quot;LogNormalize&quot;&#xA;  .. .. .. .. ..$&lt;/span&gt; scale.factor        : num 10000&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ margin              : num 1&#xA;  .. .. .. .. ..$&lt;/span&gt; verbose             : logi TRUE&#xA;  .. ..&lt;span class=&quot;math-container&quot;&gt;$ FindVariableFeatures.RNA:Formal class 'SeuratCommand' [package &quot;SeuratObject&quot;] with 5 slots&#xA;  .. .. .. ..@ name       : chr &quot;FindVariableFeatures.RNA&quot;&#xA;  .. .. .. ..@ time.stamp : POSIXct[1:1], format: &quot;2023-05-21 03:58:22&quot;&#xA;  .. .. .. ..@ assay.used : chr &quot;RNA&quot;&#xA;  .. .. .. ..@ call.string: chr &quot;FindVariableFeatures(seurat, nfeatures = 500)&quot;&#xA;  .. .. .. ..@ params     :List of 12&#xA;  .. .. .. .. ..$&lt;/span&gt; assay              : chr &amp;quot;RNA&amp;quot;&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ selection.method   : chr &quot;vst&quot;&#xA;  .. .. .. .. ..$&lt;/span&gt; loess.span         : num 0.3&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ clip.max           : chr &quot;auto&quot;&#xA;  .. .. .. .. ..$&lt;/span&gt; mean.function      :function (mat, display_progress)  &#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ dispersion.function:function (mat, display_progress)  &#xA;  .. .. .. .. ..$&lt;/span&gt; num.bin            : num 20&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ binning.method     : chr &quot;equal_width&quot;&#xA;  .. .. .. .. ..$&lt;/span&gt; nfeatures          : num 500&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ mean.cutoff        : num [1:2] 0.1 8&#xA;  .. .. .. .. ..$&lt;/span&gt; dispersion.cutoff  : num [1:2] 1 Inf&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ verbose            : logi TRUE&#xA;  .. ..$&lt;/span&gt; ScaleData.RNA           :Formal class 'SeuratCommand' [package &amp;quot;SeuratObject&amp;quot;] with 5 slots&#xA;  .. .. .. ..@ name       : chr &amp;quot;ScaleData.RNA&amp;quot;&#xA;  .. .. .. ..@ time.stamp : POSIXct[1:1], format: &amp;quot;2023-05-21 03:58:23&amp;quot;&#xA;  .. .. .. ..@ assay.used : chr &amp;quot;RNA&amp;quot;&#xA;  .. .. .. ..@ call.string: chr [1:2] &amp;quot;ScaleData(seurat, vars.to.regress = c(\&amp;quot;nFeature_RNA\&amp;quot;, &amp;quot; &amp;quot;    \&amp;quot;percent.mt\&amp;quot;))&amp;quot;&#xA;  .. .. .. ..@ params     :List of 11&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ features          : chr [1:500] &quot;PPBP&quot; &quot;LYZ&quot; &quot;S100A9&quot; &quot;IGLL5&quot; ...&#xA;  .. .. .. .. ..$&lt;/span&gt; assay             : chr &amp;quot;RNA&amp;quot;&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ vars.to.regress   : chr [1:2] &quot;nFeature_RNA&quot; &quot;percent.mt&quot;&#xA;  .. .. .. .. ..$&lt;/span&gt; model.use         : chr &amp;quot;linear&amp;quot;&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ use.umi           : logi FALSE&#xA;  .. .. .. .. ..$&lt;/span&gt; do.scale          : logi TRUE&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ do.center         : logi TRUE&#xA;  .. .. .. .. ..$&lt;/span&gt; scale.max         : num 10&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ block.size        : num 1000&#xA;  .. .. .. .. ..$&lt;/span&gt; min.cells.to.block: num 2638&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ verbose           : logi TRUE&#xA;  .. ..$&lt;/span&gt; RunPCA.RNA              :Formal class 'SeuratCommand' [package &amp;quot;SeuratObject&amp;quot;] with 5 slots&#xA;  .. .. .. ..@ name       : chr &amp;quot;RunPCA.RNA&amp;quot;&#xA;  .. .. .. ..@ time.stamp : POSIXct[1:1], format: &amp;quot;2023-05-21 03:58:25&amp;quot;&#xA;  .. .. .. ..@ assay.used : chr &amp;quot;RNA&amp;quot;&#xA;  .. .. .. ..@ call.string: chr &amp;quot;RunPCA(seurat, npcs = 50)&amp;quot;&#xA;  .. .. .. ..@ params     :List of 10&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ assay          : chr &quot;RNA&quot;&#xA;  .. .. .. .. ..$&lt;/span&gt; npcs           : num 50&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ rev.pca        : logi FALSE&#xA;  .. .. .. .. ..$&lt;/span&gt; weight.by.var  : logi TRUE&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ verbose        : logi TRUE&#xA;  .. .. .. .. ..$&lt;/span&gt; ndims.print    : int [1:5] 1 2 3 4 5&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ nfeatures.print: num 30&#xA;  .. .. .. .. ..$&lt;/span&gt; reduction.name : chr &amp;quot;pca&amp;quot;&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ reduction.key  : chr &quot;PC_&quot;&#xA;  .. .. .. .. ..$&lt;/span&gt; seed.use       : num 42&#xA;  .. ..&lt;span class=&quot;math-container&quot;&gt;$ RunUMAP.RNA.pca         :Formal class 'SeuratCommand' [package &quot;SeuratObject&quot;] with 5 slots&#xA;  .. .. .. ..@ name       : chr &quot;RunUMAP.RNA.pca&quot;&#xA;  .. .. .. ..@ time.stamp : POSIXct[1:1], format: &quot;2023-05-21 03:58:35&quot;&#xA;  .. .. .. ..@ assay.used : chr &quot;RNA&quot;&#xA;  .. .. .. ..@ call.string: chr &quot;RunUMAP(seurat, dims = 1:20)&quot;&#xA;  .. .. .. ..@ params     :List of 26&#xA;  .. .. .. .. ..$&lt;/span&gt; dims                : int [1:20] 1 2 3 4 5 6 7 8 9 10 ...&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ reduction           : chr &quot;pca&quot;&#xA;  .. .. .. .. ..$&lt;/span&gt; assay               : chr &amp;quot;RNA&amp;quot;&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ slot                : chr &quot;data&quot;&#xA;  .. .. .. .. ..$&lt;/span&gt; umap.method         : chr &amp;quot;uwot&amp;quot;&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ return.model        : logi FALSE&#xA;  .. .. .. .. ..$&lt;/span&gt; n.neighbors         : int 30&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ n.components        : int 2&#xA;  .. .. .. .. ..$&lt;/span&gt; metric              : chr &amp;quot;cosine&amp;quot;&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ learning.rate       : num 1&#xA;  .. .. .. .. ..$&lt;/span&gt; min.dist            : num 0.3&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ spread              : num 1&#xA;  .. .. .. .. ..$&lt;/span&gt; set.op.mix.ratio    : num 1&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ local.connectivity  : int 1&#xA;  .. .. .. .. ..$&lt;/span&gt; repulsion.strength  : num 1&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ negative.sample.rate: int 5&#xA;  .. .. .. .. ..$&lt;/span&gt; uwot.sgd            : logi FALSE&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ seed.use            : int 42&#xA;  .. .. .. .. ..$&lt;/span&gt; angular.rp.forest   : logi FALSE&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ densmap             : logi FALSE&#xA;  .. .. .. .. ..$&lt;/span&gt; dens.lambda         : num 2&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ dens.frac           : num 0.3&#xA;  .. .. .. .. ..$&lt;/span&gt; dens.var.shift      : num 0.1&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ verbose             : logi TRUE&#xA;  .. .. .. .. ..$&lt;/span&gt; reduction.name      : chr &amp;quot;umap&amp;quot;&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ reduction.key       : chr &quot;UMAP_&quot;&#xA;  .. ..$&lt;/span&gt; FindNeighbors.RNA.pca   :Formal class 'SeuratCommand' [package &amp;quot;SeuratObject&amp;quot;] with 5 slots&#xA;  .. .. .. ..@ name       : chr &amp;quot;FindNeighbors.RNA.pca&amp;quot;&#xA;  .. .. .. ..@ time.stamp : POSIXct[1:1], format: &amp;quot;2023-05-21 03:58:35&amp;quot;&#xA;  .. .. .. ..@ assay.used : chr &amp;quot;RNA&amp;quot;&#xA;  .. .. .. ..@ call.string: chr &amp;quot;FindNeighbors(seurat, dims = 1:20)&amp;quot;&#xA;  .. .. .. ..@ params     :List of 17&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ reduction      : chr &quot;pca&quot;&#xA;  .. .. .. .. ..$&lt;/span&gt; dims           : int [1:20] 1 2 3 4 5 6 7 8 9 10 ...&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ assay          : chr &quot;RNA&quot;&#xA;  .. .. .. .. ..$&lt;/span&gt; k.param        : num 20&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ return.neighbor: logi FALSE&#xA;  .. .. .. .. ..$&lt;/span&gt; compute.SNN    : logi TRUE&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ prune.SNN      : num 0.0667&#xA;  .. .. .. .. ..$&lt;/span&gt; nn.method      : chr &amp;quot;annoy&amp;quot;&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ n.trees        : num 50&#xA;  .. .. .. .. ..$&lt;/span&gt; annoy.metric   : chr &amp;quot;euclidean&amp;quot;&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ nn.eps         : num 0&#xA;  .. .. .. .. ..$&lt;/span&gt; verbose        : logi TRUE&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ force.recalc   : logi FALSE&#xA;  .. .. .. .. ..$&lt;/span&gt; do.plot        : logi FALSE&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ graph.name     : chr [1:2] &quot;RNA_nn&quot; &quot;RNA_snn&quot;&#xA;  .. .. .. .. ..$&lt;/span&gt; l2.norm        : logi FALSE&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ cache.index    : logi FALSE&#xA;  .. ..$&lt;/span&gt; FindClusters            :Formal class 'SeuratCommand' [package &amp;quot;SeuratObject&amp;quot;] with 5 slots&#xA;  .. .. .. ..@ name       : chr &amp;quot;FindClusters&amp;quot;&#xA;  .. .. .. ..@ time.stamp : POSIXct[1:1], format: &amp;quot;2023-05-21 03:58:36&amp;quot;&#xA;  .. .. .. ..@ assay.used : chr &amp;quot;RNA&amp;quot;&#xA;  .. .. .. ..@ call.string: chr &amp;quot;FindClusters(seurat, resolution = 1)&amp;quot;&#xA;  .. .. .. ..@ params     :List of 10&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ graph.name      : chr &quot;RNA_snn&quot;&#xA;  .. .. .. .. ..$&lt;/span&gt; modularity.fxn  : num 1&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ resolution      : num 1&#xA;  .. .. .. .. ..$&lt;/span&gt; method          : chr &amp;quot;matrix&amp;quot;&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ algorithm       : num 1&#xA;  .. .. .. .. ..$&lt;/span&gt; n.start         : num 10&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ n.iter          : num 10&#xA;  .. .. .. .. ..$&lt;/span&gt; random.seed     : num 0&#xA;  .. .. .. .. ..&lt;span class=&quot;math-container&quot;&gt;$ group.singletons: logi TRUE&#xA;  .. .. .. .. ..$&lt;/span&gt; verbose         : logi TRUE&#xA;  ..@ tools       : list()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My objective is&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Generate feature plot&lt;/li&gt;&#xA;&lt;li&gt;Then label the cells based on only variable genes,right now I see all the genes I would like to only label based on top genes which I'm storing at &lt;code&gt;top_gene&lt;/code&gt; but Im not sure how to incorporate it instead of every genes which I now see in my drop down menu&lt;/li&gt;&#xA;&lt;li&gt;The after labelling cells i want to export the labelled cluster as csv file&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;What Now Im getting is html file which not what I want&lt;/p&gt;&#xA;&lt;p&gt;Any help or suggestion is really helpful&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;UPDATE&lt;/strong&gt;&#xA;The right function I have to use is this &lt;code&gt;FetchData()&lt;/code&gt; but Im not sure how to pass the gene which i choose to label the cells and use the same gene as input for exporting cluster&lt;/p&gt;&#xA;" OwnerUserId="14154" LastEditorUserId="14154" LastEditDate="2023-05-22T07:30:10.773" LastActivityDate="2023-05-22T07:30:10.773" Title="shiny app for single cell cluster export issue" Tags="|single-cell|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21045" PostTypeId="2" ParentId="18918" CreationDate="2023-05-21T21:18:28.477" Score="1" Body="&lt;p&gt;Additional answer on asymetric IUPAC score matrix.&lt;/p&gt;&#xA;&lt;p&gt;To get the scores with an asymentric score matrix. This is also implemented in the Bioconductor package MSA2dist (&lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/MSA2dist.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioconductor.org/packages/release/bioc/html/MSA2dist.html&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(Biostrings)&#xA;library(MSA2dist)&#xA;&#xA;# consensus sequence and subject sequences&#xA;seqs &amp;lt;- setNames(DNAStringSet(c(&amp;quot;GTTGTGATTTGCTTTCRAATTAGTATCTTTGAACCATTGAAAACAAC&amp;quot;,&#xA;&amp;quot;GTTGTGATTTGCTTTCAAATTAGTATCTTTGAACCATTGAAAACAAC&amp;quot;)), c(&amp;quot;con&amp;quot;, &amp;quot;seq&amp;quot;))&#xA;&#xA;# define score matrix&#xA;scoreMatrix &amp;lt;- MSA2dist::iupacMatrix()&#xA;scoreMatrix[&amp;quot;A&amp;quot;, &amp;quot;R&amp;quot;]&#xA;# [1] 0.5&#xA;scoreMatrix[&amp;quot;R&amp;quot;, &amp;quot;A&amp;quot;] &amp;lt;- 1&#xA;&#xA;MSA2dist::dnastring2dist(seqs, score=scoreMatrix)&#xA;MSA2dist::dnastring2dist(rev(seqs), score=scoreMatrix)&#xA;&#xA;# or here using the asymetric score matrix from Biostrings&#xA;&#xA;ansm &amp;lt;- nucleotideSubstitutionMatrix(symmetric = FALSE)&#xA;&#xA;#note: MSA2dist score is normalized by analysed sites&#xA;MSA2dist::dnastring2dist(seqs, score=ansm)&#xA;MSA2dist::dnastring2dist(rev(seqs), score=ansm)&#xA;&#xA;pairwiseAlignment(seqs[[1]], seqs[[2]], substitutionMatrix = ansm)&#xA;pairwiseAlignment(seqs[[2]], seqs[[1]], substitutionMatrix = ansm)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Since &lt;code&gt;MSA2dist::dnastring2dist&lt;/code&gt; was until now calculating all possible pairwise combinations from top to bottom (leaving out the already seen combinations), now I will include the option to really calculate all pairwise combinations to take into account asymetric scores and sequence position.&lt;/p&gt;&#xA;" OwnerUserId="17525" LastEditorUserId="17525" LastEditDate="2023-05-21T21:19:57.427" LastActivityDate="2023-05-21T21:19:57.427" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21046" PostTypeId="2" ParentId="458" CreationDate="2023-05-21T22:08:42.160" Score="0" Body="&lt;p&gt;I filtered my bams out of curiosity. I had around 500 peaks before filtering. When i called on the filtered bams, i lost 100 (from the blacklist) and gained 38. The peak call was using macs2. Looking at the gained peaks, they are legit and a binding partner of my TF had peaks there.&#xA;Note however i ran out of ram for bam &amp;gt; 8GB using bedtools intersect to filter. And if you do it, sort the bam and make sure the chromosome names are ordered using the same tool between blacklist bed and your bams.&lt;/p&gt;&#xA;" OwnerUserId="17526" LastActivityDate="2023-05-21T22:08:42.160" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21047" PostTypeId="1" CreationDate="2023-05-22T05:42:19.337" Score="1" ViewCount="80" Body="&lt;p&gt;I would like to make an evolutionary tree using the SNP data from DArTseq. Are there any software or packages available for this purpose? I came across the software SNphylo, but its home page is not updated. Your suggestions would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="17527" LastActivityDate="2023-06-21T16:01:50.217" Title="Evolutionary tree from SNP data" Tags="|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21048" PostTypeId="1" AcceptedAnswerId="21050" CreationDate="2023-05-22T07:07:02.880" Score="1" ViewCount="238" Body="&lt;p&gt;When we run the OrthoFinder analysis tool on a group of genomes to get the orthologues shared by them one of the output files include a folder named 'Species_Tree' that contains a text file named 'SpeciesTree_rooted.txt' . These are its contents (This is an example I have used for the question - 11 genomes of the &lt;em&gt;Metabacillus&lt;/em&gt; genus) :&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;((Mlacus:0.244776,Mmangrovi:0.284609)1:0.0277466,(((Mdong:0.0284157,Midriensis:0.0250137)1:0.0553715,Mindicus:0.0904605)1:0.148105,(Mfastidiosus:0.207986,(Mniabensis:0.142138,((Mcrassostreae:0.0124653,Mlitoralis:0.0139468)1:0.116616,(Mhalosaccharovorans:0.0408989,Mschmidt:0.0422379)1:0.0872204)1:0.0402912)1:0.0849362)1:0.0750784)1:0.0277466);&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I used iTOL to construct the tree using this file but I could only see the branch lengths and no way to add any bootstrap values if there had been any, to the tree as seen in most research papers - 99, 98, 100 etc. Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="17392" LastEditorUserId="298" LastEditDate="2023-05-22T11:26:51.213" LastActivityDate="2023-05-22T14:03:10.553" Title="How to add bootstrap values to the phylogenetic tree generated by OrthoFinder?" Tags="|phylogenetics|sequence-alignment|phylogeny|taxonomy|orthofinder|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21049" PostTypeId="1" CreationDate="2023-05-22T13:04:23.513" Score="2" ViewCount="16" Body="&lt;p&gt;I wanted to assign secondary structure to set of pdb files. However, DSSP and STRIDE cannot assign for single peptides if it is falling inside single long beta strand or polyprolene region as it does not have any hydrogen bond.DSSP and STRIDE work fine as long as the peptides form hydrogen bonds.&lt;/p&gt;&#xA;&lt;p&gt;Can someone suggest me a way to assign secondary structure for peptides which also fall in the region of PPII and beta strand region ?&lt;/p&gt;&#xA;" OwnerUserId="17469" LastActivityDate="2023-05-22T13:04:23.513" Title="Is there any python module to assign secondary structures which includes both Polyprolene region as well as individual strand as well?" Tags="|protein-structure|pdb|secondary-structure|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21050" PostTypeId="2" ParentId="21048" CreationDate="2023-05-22T13:57:16.787" Score="1" Body="&lt;p&gt;Bootstrapping is easy just use &lt;strong&gt;iqtree&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;iqtree -s file -m GTR+I+G -T 6 -B 1000 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will parallelise across 6 CPUs and results in 1000 bootstraps, whilst GTR is the mutation model, which models invariant sites under a 4 category gamma distribution. This is a robust model.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;file&lt;/code&gt; is aligned fasta format (its all data pipeline so I forget if I've shifted the format) - although this can be changed to any input format&lt;/p&gt;&#xA;&lt;p&gt;The result will be in the treefile and is a professional quality tree, complete with bootstraps.&lt;/p&gt;&#xA;&lt;p&gt;The following up analysis is a Bayesian tree and is more complicated.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; You must resolve/delete gaps using iqtree because of the way the authors are calling gaps - this related to your question here - &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/21042/how-to-use-gblocks-for-trimming-single-copy-gene-sequences&quot;&gt;How to use Gblocks for trimming single-copy gene sequences?&lt;/a&gt; . Basically columns with gaps should be deleted.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Question, are you sure this is not a Bayesian analysis? It would make sense if it was a Bayesian analysis that is 100% robust (there are robustness scores there). This would make sense for this tree and this data. As far  I am aware iTOL is used for viewing and manipulating trees rather than creating them.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-05-22T14:03:10.553" LastActivityDate="2023-05-22T14:03:10.553" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="21051" PostTypeId="2" ParentId="21047" CreationDate="2023-05-22T14:07:13.827" Score="1" Body="&lt;p&gt;At present I would simply use iqtree&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;iqtree -s file -m GTR+I+G -T 6 -B 1000 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will parallelise across 6 CPUs and results in 1000 bootstraps, whilst GTR is the mutation model, which models invariant sites under a 4 category gamma distribution. This is a robust model.&lt;/p&gt;&#xA;&lt;p&gt;You can view the tree in &lt;a href=&quot;http://tree.bio.ed.ac.uk/software/figtree/&quot; rel=&quot;nofollow noreferrer&quot;&gt;FigTree&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The DNA file is in fasta format&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; seq1&#xA;AGACGT&#xA;&amp;gt;seq2&#xA;AGAAAT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If there's an 'unknown' format bug let me know because I only work in  data pipeline so I forget if I've shifted the format.&lt;/p&gt;&#xA;&lt;p&gt;The result will be in the treefile and is a professional quality tree, complete with bootstraps.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;I&lt;/code&gt; parameter here could omitted for this dataset because I assume all none of the sites are invariant.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-05-22T15:32:22.807" LastActivityDate="2023-05-22T15:32:22.807" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21052" PostTypeId="2" ParentId="21042" CreationDate="2023-05-22T14:15:13.943" Score="1" Body="&lt;p&gt;Basically this is 16S data. If its proteins there's a different strategy - even if the data set needed is nucleotides (separate question). I would simply use trimal which is available on &lt;strong&gt;conda&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;trimal -in &amp;lt;inputfile&amp;gt; -out &amp;lt;outputfile&amp;gt; -nogaps&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The input file is aligned fasta here and will output as aligned fasta.&lt;/p&gt;&#xA;&lt;p&gt;Any site (alignment position) with an indel &lt;code&gt;-&lt;/code&gt; will be removed. This is definitely preferred if performing an &lt;strong&gt;iqtree&lt;/strong&gt; analysis, because of the way it handles indels. This will also clean up poorly aligned, alignments.&lt;/p&gt;&#xA;&lt;p&gt;e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;---ATG&#xA;---A-G&#xA;--TATG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will result in...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;AG&#xA;AG&#xA;AG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Ultimately for alignments - there's no magic bullet - and manual editing and inspection is expected e.g. using &lt;strong&gt;seqotron&lt;/strong&gt;. There are very good reasons for this concerning circular mathematics. However, modern aligners such as &lt;strong&gt;muscle5&lt;/strong&gt; are extremely good.&lt;/p&gt;&#xA;&lt;p&gt;Trimal is available via conda&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install -c bioconda trimal &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In terms of using GBlock I'd need to know whether it was simply deleting indel sites, deleting indel sites at &amp;gt;50% frequency or whether it was performing a genetic diversity calculation per site. I don't think it is doing the latter. It is better to perform diversity saturation calculation explicitly rather than using an unknown automate because thats kinda important. If its deleting sites with excessive &amp;gt;50% indels - honestly its better to do a blockwise deletion rather than do something half-half. Within SSU there is a reason might be being done because there can be excessive indels at deep taxonomic levels with this data type. Again using iqtree just do it 'blockwise' which is the trimal output.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Note&lt;/em&gt; I've not read the original GBlock paper, but if you summarise its application I could consider it against &lt;strong&gt;trimal&lt;/strong&gt;. However, I definitely prefer &lt;strong&gt;trimal&lt;/strong&gt; considering the downstream application.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-05-22T14:27:13.477" LastActivityDate="2023-05-22T14:27:13.477" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21054" PostTypeId="1" CreationDate="2023-05-23T00:13:07.593" Score="1" ViewCount="167" Body="&lt;p&gt;Hi guys =) I'm sorry if this is a repeat but I haven't been able to find an answer using the search field or google.&lt;/p&gt;&#xA;&lt;p&gt;I am trying to edit a fastq file that has a corrupt line. The read in question is VH00362:5:AAAV5KTHV:1:1210:64850:6869 where the read is just a massive repeat – not the expected 110bp. I am trying to remove two lines from this file and asking around  I've been told to use &lt;code&gt;sed&lt;/code&gt; as in&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;gzip -c file.fastq.gz | sed &amp;quot;45500973,45500974d&amp;quot; &amp;gt; new.fastq.gz&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;but when I run &lt;code&gt;wc -l &lt;/code&gt; to check the number of lines the results are incoherent, as if it has deleted much more than the two lines I asked. Another issue that I'm seeing with this file is that every time I run &lt;code&gt;gzip&lt;/code&gt; I get a &lt;code&gt;invalid compressed data--format violated&lt;/code&gt; error, which tells me the file is corrupted but I'm doing my best to salvage whatever I can from the file as it is an irreplaceable sample.&lt;/p&gt;&#xA;&lt;p&gt;Is there another way to simply delete these two lines? Any tool or command you could recommend? Any help is deeply appreciated.&lt;/p&gt;&#xA;" OwnerUserId="17536" LastActivityDate="2023-05-24T15:34:27.357" Title="How can I delete these lines in my fastq file?" Tags="|rna-seq|data-preprocessing|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21055" PostTypeId="2" ParentId="21054" CreationDate="2023-05-23T00:26:52.767" Score="2" Body="&lt;p&gt;This is not the thing I do these days but ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gunzip -c file.fastq.gz | sed &amp;quot;45500973,45500974d&amp;quot; &amp;gt; new.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You were using &lt;code&gt;gzip&lt;/code&gt; instead of &lt;code&gt;gunzip&lt;/code&gt;, and that the source of the error. Finally the out file isn't compressed, so you probably want:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gunzip -c file.fastq.gz | sed &amp;quot;45500973,45500974d&amp;quot; | gzip &amp;gt; new.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That should work (I tested it and yes it works).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="298" LastEditDate="2023-05-23T11:17:34.113" LastActivityDate="2023-05-23T11:17:34.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21056" PostTypeId="1" CreationDate="2023-05-23T07:35:26.883" Score="1" ViewCount="52" Body="&lt;p&gt;I'm wondering which tool or program is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Most updated; doesn't use a superseded algorithm&lt;/li&gt;&#xA;&lt;li&gt;Runs very quickly (ideally, under 30s per VCF file)&lt;/li&gt;&#xA;&lt;li&gt;Able to be automated in some way for a large number of VCFs&lt;/li&gt;&#xA;&lt;li&gt;I need to at least have these groups: Africans, East Asians, Europeans, and South Asians. More specific groups are fine.&lt;/li&gt;&#xA;&lt;li&gt;Able to be run locally, as I have a lot of data&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I am not looking to visualize the data or graph a PCA. My goal is to get a percentage match, or any closeness measure, for each group for each VCF file (assuming one sample per file). All VCFs are generated from microarray data.&lt;/p&gt;&#xA;&lt;p&gt;For example, Eurogenes k13 and MDLP World-22, especially with oracle, have sufficient results but are only available as a web app.&lt;/p&gt;&#xA;&lt;p&gt;To clarify, I am asking if there exists &lt;em&gt;any&lt;/em&gt; such local program that exists that is peer-reviewed or validated, fast, and automate-able.&lt;/p&gt;&#xA;" OwnerUserId="17430" LastEditorUserId="17430" LastEditDate="2023-05-29T17:54:27.703" LastActivityDate="2023-05-29T17:54:27.703" Title="What is the current state-of-the-art for classifying the ancestry groups of VCF files?" Tags="|vcf|genome|admixture|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="21057" PostTypeId="2" ParentId="21007" CreationDate="2023-05-23T08:21:23.300" Score="3" Body="&lt;p&gt;It seems like you are already on the right track. The code you provided fetches the protein data from the JSON file and generates a scatter plot with D3.js based on two properties: &lt;code&gt;length&lt;/code&gt; and &lt;code&gt;nr_modifications&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To add an interactive hover event, you can add a tooltip to your scatterplot. The tooltip will display additional information when you hover over each dot.&lt;/p&gt;&#xA;&lt;p&gt;Below is an example of how you can add a tooltip:&lt;/p&gt;&#xA;&lt;p&gt;First, add a &lt;code&gt;&amp;lt;div&amp;gt;&lt;/code&gt; element in your HTML that will be used as the tooltip:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-html prettyprint-override&quot;&gt;&lt;code&gt;&amp;lt;div id=&amp;quot;tooltip&amp;quot; style=&amp;quot;position: absolute; opacity: 0;&amp;quot;&amp;gt;&#xA;    &amp;lt;p&amp;gt;&amp;lt;strong&amp;gt;Protein:&amp;lt;/strong&amp;gt; &amp;lt;span id=&amp;quot;protein&amp;quot;&amp;gt;&amp;lt;/span&amp;gt;&amp;lt;/p&amp;gt;&#xA;    &amp;lt;p&amp;gt;&amp;lt;strong&amp;gt;Modifications:&amp;lt;/strong&amp;gt; &amp;lt;span id=&amp;quot;modifications&amp;quot;&amp;gt;&amp;lt;/span&amp;gt;&amp;lt;/p&amp;gt;&#xA;    &amp;lt;p&amp;gt;&amp;lt;strong&amp;gt;Length:&amp;lt;/strong&amp;gt; &amp;lt;span id=&amp;quot;length&amp;quot;&amp;gt;&amp;lt;/span&amp;gt;&amp;lt;/p&amp;gt;&#xA;&amp;lt;/div&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then, in your script, add the code to handle the mouseover and mouseout events for each circle:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-javascript prettyprint-override&quot;&gt;&lt;code&gt;{#each data.proteins as datapoint}&#xA;    &amp;lt;circle&#xA;        cx={xScale(datapoint.length)}&#xA;        cy={yScale(datapoint.nr_modifications)}&#xA;        class:human={datapoint.species === 'human'}&#xA;        r=&amp;quot;20&amp;quot;&#xA;        on:mouseover=&amp;quot;{(e) =&amp;gt; handleMouseover(e, datapoint)}&amp;quot;&#xA;        on:mouseout=&amp;quot;{(e) =&amp;gt; handleMouseout(e)}&amp;quot;&#xA;    /&amp;gt;&#xA;{/each}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Define the &lt;code&gt;handleMouseover&lt;/code&gt; and &lt;code&gt;handleMouseout&lt;/code&gt; functions in your script. These functions will show and hide the tooltip respectively. For the &lt;code&gt;handleMouseover&lt;/code&gt; function, it also updates the tooltip content with the protein data:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-javascript prettyprint-override&quot;&gt;&lt;code&gt;&amp;lt;script&amp;gt;&#xA;    // ... your existing code ...&#xA;&#xA;    const handleMouseover = (event, datapoint) =&amp;gt; {&#xA;        // Update the tooltip content&#xA;        document.getElementById('protein').textContent = datapoint.symbol;&#xA;        document.getElementById('modifications').textContent = datapoint.nr_modifications;&#xA;        document.getElementById('length').textContent = datapoint.length;&#xA;&#xA;        // Show the tooltip&#xA;        document.getElementById('tooltip').style.opacity = 1;&#xA;&#xA;        // Position the tooltip&#xA;        document.getElementById('tooltip').style.left = event.pageX + 'px';&#xA;        document.getElementById('tooltip').style.top = event.pageY + 'px';&#xA;    };&#xA;&#xA;    const handleMouseout = () =&amp;gt; {&#xA;        // Hide the tooltip&#xA;        document.getElementById('tooltip').style.opacity = 0;&#xA;    };&#xA;&amp;lt;/script&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In standard HTML or JavaScript, the {#each} syntax and usage of {} for expressions and variable binding won't work as they are specific to Svelte. Same applies to the usage of on:mouseover and on:mouseout. In standard HTML or JavaScript, we use onmouseover and onmouseout.&lt;/p&gt;&#xA;&lt;p&gt;Here are some things which I haven't checked, but you might want to consider looking at:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It looks like you are trying to load JSON data from an external&#xA;source, but the function fetchProteins() is never called and its&#xA;result (the promise returned) is not being awaited. The way it's&#xA;structured now, Proteins in the returned object will hold a Promise,&#xA;not the actual data.&lt;/li&gt;&#xA;&lt;li&gt;The code assumes that the returned JSON data is an array stored in&#xA;data.proteins, but the provided JSON example suggests that the data&#xA;structure might be different. It doesn't show any 'proteins' key in&#xA;the JSON. The code needs to align with the actual structure of the&#xA;returned JSON data. &lt;code&gt;data.proteins&lt;/code&gt; implies that &lt;code&gt;data&lt;/code&gt; is an object and &lt;code&gt;proteins&lt;/code&gt; is a property of that object containing an array.&lt;/li&gt;&#xA;&lt;li&gt;In your SVG transform attribute, you are using braces {} though the correct syntax for a&#xA;translate transformation in SVG is translate(x,y).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You might want to try looking at the following changes:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-js prettyprint-override&quot;&gt;&lt;code&gt;export const load = ({ fetch }) =&amp;gt; {&#xA;  const fetchProteins = async () =&amp;gt; {&#xA;    const res = await fetch('https:address/assets/ptm.json')&#xA;    const data = await res.json()&#xA;    return data&#xA;  }&#xA;&#xA;  return {&#xA;    Proteins: fetchProteins&#xA;  }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre class=&quot;lang-js prettyprint-override&quot;&gt;&lt;code&gt;  $: xScale = scaleLinear()&#xA;        .domain(extent(data.map((d) =&amp;gt; d.length)))&#xA;        .range([0, 800 - (margin * 2)]);&#xA;    &#xA;    $: yScale = scaleLinear()&#xA;        .domain(extent(data.map((d) =&amp;gt; d.nr_modifications)))&#xA;        .range([0, 400 - (margin * 2)]);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre class=&quot;lang-html prettyprint-override&quot;&gt;&lt;code&gt;&amp;lt;g transform=&amp;quot;translate({margin}px, {margin}px)&amp;quot;&amp;gt;&#xA;        {#each data as datapoint}&#xA;            &amp;lt;circle&#xA;                cx={xScale(datapoint.length)}&#xA;                cy={yScale(datapoint.nr_modifications)}&#xA;                class=&amp;quot;{datapoint.species === 'human' ? 'human' : ''}&amp;quot;&#xA;                r=&amp;quot;20&amp;quot;&#xA;            /&amp;gt;&#xA;        {/each}&#xA;        &amp;lt;!-- The rest of your code --&amp;gt;&#xA;&amp;lt;/g&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17430" LastActivityDate="2023-05-23T08:21:23.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21058" PostTypeId="2" ParentId="11465" CreationDate="2023-05-23T08:32:04.360" Score="2" Body="&lt;p&gt;You can also do this with &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/content/tools/intersect.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;bedtools intersect&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;bedtools intersect -abam input.bam -b bedfile.bed &amp;gt; output.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This should be identical to the &lt;code&gt;samtools view&lt;/code&gt; answer. One further feature though is you can output all reads that &lt;strong&gt;don't&lt;/strong&gt; overlap with the regions in &lt;code&gt;bedfile.bed&lt;/code&gt; by adding the &lt;code&gt;-v&lt;/code&gt; flag.&lt;/p&gt;&#xA;" OwnerUserId="15257" LastActivityDate="2023-05-23T08:32:04.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21059" PostTypeId="1" CreationDate="2023-05-23T14:44:47.197" Score="2" ViewCount="25" Body="&lt;p&gt;My samples have been sequenced with the Twist Exome 2.0 sequencing panel, and I want to assess how efficient the sequencing has been. By efficient, I mean examining metrics such as the uniformity (fold-80 score), coverage, on/off-target % etc. as documented in their webpage. I checked the % off-target as follows:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Count the # of reads in the bam file (&lt;code&gt;samtools view -c &amp;quot;$dir/$base.bam&lt;/code&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Count the # of reads that intersect with the bed region (&lt;code&gt;bedtools intersect -a &amp;quot;$dir/$base.bam&amp;quot; -b $bed -u | samtools view -c -&lt;/code&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Calculate the % off-target rate as (&lt;code&gt;bc -l &amp;lt;&amp;lt;&amp;lt; &amp;quot;(1 - ($bed_reads / $total_reads)) * 100&amp;quot;&lt;/code&gt;)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I have two questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Is there a cleaner/simpler way to calculate the off-target rate? I know I can calculate the on-target rate with &lt;code&gt;bedtools coverage&lt;/code&gt;, but I am not sure of how to calculate the off-target rate with a single command line.&lt;/li&gt;&#xA;&lt;li&gt;I obtained an off-target rate of approximately 20-25% for my samples. Is this within the &amp;quot;good&amp;quot; off-target % range?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="15940" LastEditorUserId="298" LastEditDate="2023-05-23T17:00:00.243" LastActivityDate="2023-05-23T17:00:00.243" Title="Off-target % for whole-exome sequencing panel" Tags="|ngs|sam|wes|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21061" PostTypeId="1" CreationDate="2023-05-23T16:53:08.377" Score="1" ViewCount="211" Body="&lt;p&gt;I have 6 VCF files, where I would like to obtain the SNPs that are common (by position) in all the 6 files. I have tried this command&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools isec  -p trial2 -n=6 -c all A2_SNP.vcf.gz A_SNP.vcf.gz B_SNP.vcf.gz C_SNP.vcf.gzD_SNP.vcf.gz E_SNP.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But this produces 6 files, and through the README file, I think each of the VCF contains the unique SNPs for each of the 6 files, and I could only get sites.txt file that ocntain the positions that are common. Are there any way to get a VCF file that have the common SNPs ?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="17543" LastActivityDate="2023-05-23T17:59:19.130" Title="Getting VCF file that contain common SNPs from 6 VCF file using isec" Tags="|rna-seq|snp|variant-calling|rna|gatk|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21062" PostTypeId="2" ParentId="21061" CreationDate="2023-05-23T17:59:19.130" Score="2" Body="&lt;p&gt;You've used the right tool and gotten the right results. What you need to understand is that VCF contains variant loci as well as the genotypes for &amp;gt;=1 sample(s) at those loci. The intersect operation will extract common loci, but it does not know which genotypes you're interested in, which is why it generates one file per input, giving you the genotypes from each VCF for the common loci.&lt;/p&gt;&#xA;&lt;p&gt;You could use the sites file to do a &lt;code&gt;bcftools merge&lt;/code&gt; operation, limiting to just the common sites using the &lt;code&gt;-R &amp;lt;regions-file&amp;gt;&lt;/code&gt; option so you'd get a VCF with all the samples across all 6 input VCFs but only for the variant loci.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-05-23T17:59:19.130" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21063" PostTypeId="1" CreationDate="2023-05-23T20:08:17.327" Score="3" ViewCount="88" Body="&lt;p&gt;I'm wondering if imputation, specifically &lt;a href=&quot;http://faculty.washington.edu/browning/beagle/beagle.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Beagle&lt;/a&gt;, needs a reference panel that matches the sample's ancestry group. For example, Beagle documentation suggests the 1000 Genomes Project phase 3 reference panel, which contains all ancestry groups. Is this acceptable to use to impute the genome of, say, an East Asian, African, and European?&lt;/p&gt;&#xA;" OwnerUserId="17430" LastEditorUserId="3694" LastEditDate="2023-05-24T08:50:04.290" LastActivityDate="2023-05-29T22:20:39.133" Title="Should genotype imputation be ancestry specific?" Tags="|imputation|1000genomes|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21064" PostTypeId="2" ParentId="21063" CreationDate="2023-05-23T20:21:06.983" Score="0" Body="&lt;p&gt;in my experience you should use a publicly cited reference genome like GRCh37 or GRCh38&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0888754317300058&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sciencedirect.com/science/article/pii/S0888754317300058&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035890951-Human-genome-reference-builds-GRCh38-or-hg38-b37-hg19&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360035890951-Human-genome-reference-builds-GRCh38-or-hg38-b37-hg19&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17544" LastActivityDate="2023-05-23T20:21:06.983" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21066" PostTypeId="2" ParentId="21063" CreationDate="2023-05-24T08:53:37.907" Score="1" Body="&lt;p&gt;Basically, yes, you will get better results if you use a reference panel which contains samples with similar ancestry to your targets. Imputation works by matching haplotypes in your target samples to those in a reference panel. If the haplotypes in the target don’t exist in the reference panel, the variants won’t be imputed accurately.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We demonstrated that using TOPMed sequencing data as the imputation&#xA;reference panel improves genotype imputation quality in these&#xA;populations[Hispanic/Latino and African], which subsequently enhanced gene-mapping power for&#xA;complex traits. For rare variants with minor allele frequency (MAF) &amp;lt;&#xA;0.5%, we observed a 2.3- to 6.1-fold increase in the number of well-imputed variants, with 11-34% improvement in average imputation&#xA;quality, compared to the state-of-the-art 1000 Genomes Project Phase 3&#xA;and Haplotype Reference Consortium reference panels.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;From &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/31869403/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="3694" LastEditDate="2023-05-29T22:20:39.133" LastActivityDate="2023-05-29T22:20:39.133" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="21067" PostTypeId="1" AcceptedAnswerId="21074" CreationDate="2023-05-24T12:21:09.533" Score="1" ViewCount="42" Body="&lt;p&gt;I am trying to generate several snakemake rules. However, I would like to include some restrictions due to storage limitations&lt;/p&gt;&#xA;&lt;p&gt;I first would like to copy the the bamfiles, 1 for each sample, to a staging area. However, while I have over 50 samples, I can only process three samples at a time.&lt;/p&gt;&#xA;&lt;p&gt;Then I want to compare the bam file to chunked candidate files, 1 for each chromosome. This however, I would like to run on 23 cores per sample so that once each bam file is copied 69 cores are working.&lt;/p&gt;&#xA;&lt;p&gt;finally, if all chunks for a given sample have been processed I want to clean up the staging area, copy the bamfile for the next sample and process the chunks for that sample.&lt;/p&gt;&#xA;&lt;p&gt;My current implementation, see below, just continues copying bamfiles and crashes when I run out of strorage;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule copy_bam:&#xA;    input:&#xA;        bamfile=os.path.join(config['bam_folder'], &amp;quot;{sample}.gatk.sorted.MD.bam&amp;quot;),&#xA;        bamindex=os.path.join(config['bam_folder'], &amp;quot;{sample}.gatk.sorted.MD.bam.bai&amp;quot;)&#xA;    output:&#xA;        staging_bamfile=&amp;quot;{staging_area}/{sample}.bam&amp;quot;,&#xA;        staging_bamindex=&amp;quot;{staging_area}/{sample}.bam.bai&amp;quot;&#xA;    wildcard_constraints:&#xA;        sample=&amp;quot;[A-Za-z0-9_]+&amp;quot;&#xA;    resources:&#xA;        bam_copy=1&#xA;    shell:&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        cp {input.bamfile} {output.staging_bamfile}&#xA;        cp {input.bamindex} {output.staging_bamindex}&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;rule process_chunks:&#xA;    input:&#xA;        chunkfile=&amp;quot;{staging_area}/chunks/{chunk}_with_header.bcf.gz&amp;quot;,&#xA;        bamfile=rules.copy_bam.output.staging_bamfile&#xA;    output:&#xA;        dynamic(&amp;quot;{staging_area}/calls/{sample}/{chunk}.bcf&amp;quot;)&#xA;&#xA;    threads: 23&#xA;    shell:&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        process chunks&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;rule cleanup:&#xA;    input:&#xA;        bamfile=&amp;quot;{staging_area}/{sample}.bam&amp;quot;,&#xA;        bamindex=&amp;quot;{staging_area}/{sample}.bam.bai&amp;quot;,&#xA;        processed=lambda wildcards: expand(&amp;quot;{staging_area}/calls/{sample}/{chunk}.bcf&amp;quot;, chunk=get_chunks()[wildcards.sample], staging_area=config['staging_area'], sample=wildcards.sample)&#xA;    output:&#xA;        touch(&amp;quot;{staging_area}/{sample}/cleanup_done.txt&amp;quot;)&#xA;    shell:&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        rm {input.bamfile}&#xA;        rm {input.bamindex}&#xA;        rm {input.processed}&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;run with: &lt;code&gt;snakemake --resources bam_copy=3 --cores 70&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="16874" LastEditorUserId="240" LastEditDate="2023-05-25T15:15:22.030" LastActivityDate="2023-05-25T15:15:22.030" Title="snakemake restricting resources for specific rule" Tags="|snakemake|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21068" PostTypeId="1" CreationDate="2023-05-24T15:18:42.547" Score="0" ViewCount="278" Body="&lt;p&gt;I have an excel file that I created from VCF file for common SNPs across 6 samples. This excel have the chromosomes and the position of the SNPs only (see example table1) &lt;a href=&quot;https://i.stack.imgur.com/dMsTq.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dMsTq.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;Now I would like to obtain the other information (eg. allels, Genotype, depth, etc) from the VCF files of the 6 samples (i.e. the one that contains these positions).&#xA;I tried using AWK command like here for position 23432 on chr. 1 for the 6 file :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk -F &amp;quot; &amp;quot; '&lt;span class=&quot;math-container&quot;&gt;$1==&quot;1&quot; &amp;amp;&amp;amp; $&lt;/span&gt;2==&amp;quot;23432&amp;quot;' file1.vcf&#xA;awk -F &amp;quot; &amp;quot; '&lt;span class=&quot;math-container&quot;&gt;$1==&quot;1&quot; &amp;amp;&amp;amp; $&lt;/span&gt;2==&amp;quot;23432&amp;quot;' file2.vcf&#xA;awk -F &amp;quot; &amp;quot; '&lt;span class=&quot;math-container&quot;&gt;$1==&quot;1&quot; &amp;amp;&amp;amp; $&lt;/span&gt;2==&amp;quot;23432&amp;quot;' file3.vcf&#xA;awk -F &amp;quot; &amp;quot; '&lt;span class=&quot;math-container&quot;&gt;$1==&quot;1&quot; &amp;amp;&amp;amp; $&lt;/span&gt;2==&amp;quot;23432&amp;quot;' file4.vcf&#xA;awk -F &amp;quot; &amp;quot; '&lt;span class=&quot;math-container&quot;&gt;$1==&quot;1&quot; &amp;amp;&amp;amp; $&lt;/span&gt;2==&amp;quot;23432&amp;quot;' file5.vcf&#xA;awk -F &amp;quot; &amp;quot; '&lt;span class=&quot;math-container&quot;&gt;$1==&quot;1&quot; &amp;amp;&amp;amp; $&lt;/span&gt;2==&amp;quot;23432&amp;quot;' file6.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;he issue is that these SNPs I have are thousands positions, so I need an automated way to do this&lt;/p&gt;&#xA;&lt;p&gt;Could you advise on that ?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="17543" LastEditorUserId="3694" LastEditDate="2023-05-24T20:47:28.353" LastActivityDate="2023-05-24T20:47:28.353" Title="Extracting information from VCF file for many specific positions in specific chromosomes?" Tags="|vcf|snp|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21069" PostTypeId="2" ParentId="21054" CreationDate="2023-05-24T15:34:27.357" Score="1" Body="&lt;p&gt;A &lt;code&gt;fastq&lt;/code&gt; file has four line-separated fields per sequence. If you suspect that a sequence is corrupted, then you would want to delete all four lines related to that sequence. If you just delete the first two lines (&lt;code&gt;sequence identifier&lt;/code&gt; and &lt;code&gt;raw sequence letters&lt;/code&gt;), the file will most probably throw an error down the road. In your case, the sequence would look something like this -&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@VH00362:5:AAAV5KTHV:1:1210:64850:6869&#xA;GATTTGGGGTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTTGTTCAACTCACAGTTT&#xA;+&#xA;!''*((((***+))%%%++)(%%%%).1***-+*''))**55CCF&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;gt;CCCCCCC65&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, the code to delete these four lines from a &lt;code&gt;fastq.gz&lt;/code&gt; file would be -&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gunzip -c file.fastq.gz | sed &amp;quot;45500973,45500976d&amp;quot; | gzip &amp;gt; new.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This &lt;code&gt;sed&lt;/code&gt; command specifies a range of lines and says that delete lines from line number &lt;code&gt;45500973&lt;/code&gt; to &lt;code&gt;45500976&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="17495" LastActivityDate="2023-05-24T15:34:27.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21070" PostTypeId="1" CreationDate="2023-05-24T17:59:27.943" Score="2" ViewCount="59" Body="&lt;p&gt;I am analyzing PacBio IsoSeq data and I am using StringTie to assemble transcripts. I noticed the difference in the output files when I add a reference genome annotation (-G option in StringTie). In general I have less transcripts assembled when I give as imput reference genome annotation and I was wondering why. I know that the annotation of the genome is not the best (Borelian orangutang) and I would rather trust the data rather than the annotation provided.&lt;/p&gt;&#xA;&lt;p&gt;I am attaching an example for one of the genes (IGV): the first track is reference annotation; the second is StringTie output when no reference annotation is provided; the third is StringTie output when reference annotation is provided (in the third case two novel assembled isoforms are missing)&lt;a href=&quot;https://i.stack.imgur.com/aORrt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aORrt.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17438" LastActivityDate="2023-05-27T22:25:36.980" Title="StringTie discovers less novel isoforms when reference annotation is provided" Tags="|pacbio|isoform|stringtie|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21071" PostTypeId="1" CreationDate="2023-05-24T18:20:41.837" Score="0" ViewCount="59" Body="&lt;p&gt;From the link &lt;a href=&quot;https://gtexportal.org/home/datasets&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gtexportal.org/home/datasets&lt;/a&gt;, under V7, I'm trying to do R/Python analyses on the Gene TPM and Transcript TPM files. But in these files (and to open them I had to use Universal Viewer since the files are too large to view with an app like NotePad), I'm seeing a bunch of ID's for samples (i.e. &lt;code&gt;GTEX-1117F-0226-SM-5GZZ7&lt;/code&gt;), followed by transcript ID's like ENSG00000223972.4, and then a bunch of numbers like 0.02865 (and they take up like 99% of the large files). Can someone help me decipher what the numbers mean, please? And are the numbers supposed to be assigned to a specific sample ID? (The amount of letters far exceed the amount of samples, btw). I tried opening these files as tables in R but I do not think R is categorizing the contents of the file correctly.&lt;/p&gt;&#xA;&lt;p&gt;For context, I am planning to match males with females for sex comparison but in order to do that, I need to get R to categorize everything correctly. (I know that females have &amp;quot;F&amp;quot; where &lt;code&gt;####-#####-####-#x-#####&lt;/code&gt; where &lt;code&gt;x&lt;/code&gt; is and males have &amp;quot;M&amp;quot;).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Example subsets&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;For gene TPM:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#1.2 56202 11688&#xA;Name Description GTEX-1117F-0226-SM-5GZZ7 GTEX-111CU-1826-SM-5GZYN GTEX-111FC-0226-SM-5N9B8 GTEX-111VG-2326-SM-5N9BK GTEX-111YS-2426-SM-5GZZQ GTEX-1122O-2026-SM-5NQ91 GTEX-1128S-2126-SM-5H12U GTEX-113IC-0226-SM-5HL5C 7C-0005-SM-57WDL GTEX-ZVE2-0006-SM-51MRW GTEX-ZVZQ-0006-SM-51MR8 GTEX-ZXES-0005-SM-57WCB GTEX-ZXG5-0005-SM-57WCN&#xA;ENSG00000223972.4 DDX11L1 0.1082 0.1158 0.02104 0.02329 0 0.04641 0.03076 0.09358 0.121 0.02859 0 0.02373 0.08024 0.02231 0.0189 0.09309 0.07772 0.1596 0.1173 0.07969 0.07763 0.02062 0.07831 0.05159 0.1181 0.04187&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For transcript TPM:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;transcript_id gene_id GTEX-1117F-0226-SM-5GZZ7 GTEX-1117F-0426-SM-5EGHI GTEX-1117F-0526-SM-5EGHJ GTEX-1117F-0626-SM-5N9CS GTEX-1117F-0726-SM-5GIEN GTEX-1117F-1326-SM-5EGHH GTEX-1117F-2226-SM-5N9CH GTEX-1117F-2426-SM-5EGGH GTEX-111CU-0226-SM-5GZXC GTEX-111CU-0326-SM-5GZXO&#xA;ENST00000373020.4 ENSG00000000003.10 26.84 4.13 13.54 31.17 6.87 39.75 32.25 30.81 19.69 46.42 9.07 27.83 9.04 12.87 18.18 11.12 11.28 7.86 5.52 24.67 6.48 8.13 21.41 5.39 7.98 8.56 51.61 12.51 70.01 17.48 2.1 14.16 3.96 11.39 1.29 6.5 29.93 4.74 3.1 20.82 16.81 14.82 50.8 16.9 9.35 39.3 3.61 2.87 11.73&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17550" LastEditorUserId="73" LastEditDate="2023-05-24T20:34:06.037" LastActivityDate="2023-05-24T21:02:43.923" Title="What do the numbers mean in these RNA-Seq gene/transcript TPM files?" Tags="|r|rna-seq|sequence-analysis|genomics|tpm|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21072" PostTypeId="2" ParentId="21071" CreationDate="2023-05-24T21:02:43.923" Score="1" Body="&lt;p&gt;The files have pretty self-explanatory headers.&lt;/p&gt;&#xA;&lt;p&gt;The first file has EnsEMBL gene ID, HGNC symbol and then TPM for each sample. The second file has EnsEMBL gene ID, EnsEMBL transcript ID and TPM for each sample. Each header beginning with &lt;code&gt;GTEX&lt;/code&gt; is a sample.&lt;/p&gt;&#xA;&lt;p&gt;You should be using linux command line to look at this, not basic Windows utilities like Notepad. At least use Notepad++, basic Windows utilities are not equipped to handle large text files.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-05-24T21:02:43.923" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21073" PostTypeId="1" CreationDate="2023-05-25T07:42:52.657" Score="3" ViewCount="125" Body="&lt;p&gt;&lt;strong&gt;Setup&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I ran picard's CollectGcBiasMetrics tool for my WES sample, with the intention of assessing the performance of the Exome capture panel in terms of the uniformity of read coverage at various %GC windows. I obtained the gc bias plot attached below.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/uh9iN.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uh9iN.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am a bit puzzled by the normalised coverage values in the windows containing higher GC % and need help with interpretation - in particular, whether it indicates that something has gone awry with the sample, or if it is merely the way the normalised coverage is computed in picard. Why do the normalised coverage values seem to peak at around 50-55 GC%?&lt;/p&gt;&#xA;&lt;p&gt;Digging into the &lt;code&gt;gc_bias_metrics.txt&lt;/code&gt; file:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/OlaUe.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OlaUe.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The #windows, #read starts and normalised coverage values are highlighted in the red, blue and yellow boxes respectively. Here, I notice that the # reads / # windows ratio is high for the tail end GC content, which probably inflates the normalised coverage value.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Is this something unexpected for a WES sample&lt;/em&gt;, i.e. should I be concerned?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Follow-up question&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;If the coverage values provided by picard isn't that useful, are there other useful tools for me to calculate the uniformity of read coverage across various GC%? I am looking at plots like Figure 3 in the &lt;a href=&quot;https://www.twistbioscience.com/products/ngs/fixed-panels/exome2?tab=data&quot; rel=&quot;nofollow noreferrer&quot;&gt;twist biosciences data page&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="15940" LastActivityDate="2023-05-25T07:42:52.657" Title="Picard's CollectGcBiasMetrics gc-bias plot- help with interpretation" Tags="|picard|wes|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21074" PostTypeId="2" ParentId="21067" CreationDate="2023-05-25T12:57:39.303" Score="2" Body="&lt;p&gt;One Snakemake feature you can use is &lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#protected-and-temporary-files&quot; rel=&quot;nofollow noreferrer&quot;&gt;temporary outputs&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you make your outputs from the &lt;code&gt;copy_bam&lt;/code&gt; rule temporary, they'll be cleared up after no other rule requires that file as an input.&#xA;Your Snakefile would look something like this:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;rule copy_bam:&#xA;    input:&#xA;        bam = os.path.join(config['bam_folder'], &amp;quot;{sample}.gatk.sorted.MD.bam&amp;quot;),&#xA;        idx = os.path.join(config['bam_folder'], &amp;quot;{sample}.gatk.sorted.MD.bam.bai&amp;quot;),&#xA;    output:&#xA;        bam = temp(&amp;quot;{staging_area}/{sample}.bam&amp;quot;),&#xA;        idx = temp(&amp;quot;{staging_area}/{sample}.bam.bai&amp;quot;),&#xA;    wildcard_constraints:&#xA;        sample=&amp;quot;[A-Za-z0-9_]+&amp;quot;&#xA;    resources:&#xA;        bam_copy=1&#xA;    shell:&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        cp {input.bam} {output.bam}&#xA;        cp {input.idx} {output.idx}&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;# omitted for brevity&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can then specify how many jobs run concurrently with the &lt;code&gt;--cores&lt;/code&gt;/&lt;code&gt;--jobs&lt;/code&gt; options to keep the space usage small.&lt;/p&gt;&#xA;" OwnerUserId="240" LastEditorUserId="240" LastEditDate="2023-05-25T15:14:19.433" LastActivityDate="2023-05-25T15:14:19.433" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21075" PostTypeId="1" CreationDate="2023-05-25T16:25:55.863" Score="1" ViewCount="43" Body="&lt;p&gt;I modelled a macromolecule/receptor using Avogadro and exported it as receptor.pdb. Many entries in this file are CONECT records. If I try to run AutoDock Vina with it, this happens:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/CU7EJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CU7EJ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, if I prepare this receptor again for docking, but, this time, removing the CONECT records (as I've seen many people do), my receptor file now exhibits broken bonds. Like this:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Zn4jA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Zn4jA.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I've tried converting the .pdb file to .pdbqt with OpenBabel — removing all but the largest contiguous fragment and ignoring CONECT records as well —, but nothing works. I've also looked into it, and it looks like Vina doesn't have any parameters to handle CONECT records. Isn't there any way at all to do this?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="17555" LastEditorUserId="17555" LastEditDate="2023-05-27T16:46:29.940" LastActivityDate="2023-05-27T16:46:29.940" Title="Is there any way to handle CONECT records in AutoDock Vina?" Tags="|docking|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21077" PostTypeId="1" AcceptedAnswerId="21078" CreationDate="2023-05-26T13:53:19.683" Score="1" ViewCount="192" Body="&lt;p&gt;I would like to try an use Singularity with CNV kit as the HPC I use seems to work better with Singularity:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Initial Attempt:&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;singularity exec ~/CNV_kit/cnvkit_latest.sif /bin/bash -c &amp;quot;ls /&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;Result&lt;/em&gt;:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;bin  boot  dev  environment  etc  home  lib  lib32  lib64  libx32  media  mnt  opt  proc  root  run  sbin  singularity  srv  sys  tmp  usr  var&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Attempt 1:&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;singularity exec ~/CNV_kit/cnvkit_latest.sif /bin/bash -c &amp;quot;cd /proc/ &amp;amp;&amp;amp; ls&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This results in a folder with a series of numbers but not the cnvkit py files.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Attempt 2:&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;singularity exec ~/CNV_kit/cnvkit_latest.sif /bin/bash -c &amp;quot;cd /singularity/ &amp;amp;&amp;amp; ls&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This results in &lt;code&gt;/bin/bash: line 1: cd: /singularity/: Not a directory&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Attempt 3:&lt;/strong&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; singularity exec ~/CNV_kit/cnvkit_latest.sif /bin/bash -c &amp;quot;cd /media/ &amp;amp;&amp;amp; ls&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This results in nothing being printed to the screen. I tried this command three times to be sure.&lt;/p&gt;&#xA;&lt;p&gt;How can I find the Singularity path I would need to use with &lt;code&gt;singularity exec&lt;/code&gt; to run from the Singularity image.&lt;/p&gt;&#xA;" OwnerUserId="17244" LastEditorUserId="17244" LastEditDate="2023-05-26T20:57:09.530" LastActivityDate="2023-06-05T21:14:00.870" Title="Singularity And CNV Kit" Tags="|bash|singularity|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21078" PostTypeId="2" ParentId="21077" CreationDate="2023-05-26T16:29:57.220" Score="2" Body="&lt;p&gt;I like this question a lot. Firstly I don't know singularity, but I know Docker and it will be similar. &lt;code&gt;docker exec&lt;/code&gt; is very similar.&lt;/p&gt;&#xA;&lt;p&gt;So for example the reasons are likely as follows:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;media&lt;/code&gt; could be empty and hence no output.&lt;/li&gt;&#xA;&lt;li&gt;the Python script may not be in &lt;code&gt;/proc&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;singularity is likely to be a file rather than a directory.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;One solution is,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;singularity exec ~/CNV_kit/cnvkit_latest.sif /bin/bash -c &amp;quot;ls -l /&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;i.e. ask it which are the directories. Another solution is..&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;singularity exec ~/CNV_kit/cnvkit_latest.sif /bin/bash -c &amp;quot;find / -name 'cnvkit*'&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will locate &lt;code&gt;cnvkit.py&lt;/code&gt;, note the use of ' and &amp;quot; for this sort of command.&lt;/p&gt;&#xA;&lt;p&gt;Personally, I would dive into the container and issue the command of interest, manually. This way you get a feel for how it's constructed and don't need to keep issuing commands from outside the container. In Docker this is easy and it's impossible to keep a third party out of a container (even if the developer tries). In singularity it is likely similar.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;From the notes&lt;/strong&gt; I would simply access the singularity container directly.&lt;/p&gt;&#xA;&lt;p&gt;Accessing a singularity container is different from Docker (I looked it up) ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;singularity shell ~/CNV_kit/cnvkit_latest.sif &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Once inside you'll be able to look around, try out the &lt;code&gt;find&lt;/code&gt; command, look in the &lt;code&gt;bin&lt;/code&gt; etc... There will be no need to issue commands from outside the container to get it working.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;From further comments,&lt;/p&gt;&#xA;&lt;p&gt;A node on an HPC is a group of usually 8 to 32+ CPUs. What you have is a 'master' or 'head' node, where you log into and keep your data, and the stuff you do the work on is called 'nodes'. It has a more specific name, but it's not very cool, so we don't use it. The HPC will comprise a large number of nodes, e.g. 100+, but can be smaller particularly with the massive CPU machines around now. A node concept is a bit different on a NVidia machine if its part of a cluster.&lt;/p&gt;&#xA;&lt;p&gt;The issue is someone could have called a script 'node', which would be confusing. It's complicated, because someone could have their own research machine which is attached and maintained on. the HPC, but only they can access and called it e.g. 'Joes_node', of 'Jills_node'.&lt;/p&gt;&#xA;&lt;p&gt;Personally, I would sit down with your administrator and ask them to explain singularity and this the comments herein and this. They will know that at admin level (thats their job).&lt;/p&gt;&#xA;&lt;p&gt;From my point of view, if I'd access to a cluster using singularity I could figure it out. I use GCP now, it's geeky but cool, so my information is out of date on non-Cloud clusters.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;RamRS comments&lt;/strong&gt;&#xA;What RamRS is saying and what I'm trying to assess whether you are within a singularity container. Again, I don't know the architecture.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Merging With Accepted Answer:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The main file in cnvkit is &lt;code&gt;cnvkit.py&lt;/code&gt; in order to find that you need to look using &lt;code&gt;singularity exec&lt;/code&gt;. The following command gives the path to the cnvkit.py file:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;singularity exec cnvkit_latest.sif /bin/bash -c &amp;quot;cd /usr/local/bin/ &amp;amp;&amp;amp; ls&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;An example of how to use this path is below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Directory containing sample files:&#xA;directory=&amp;quot;/path/to/bam/files/folder&amp;quot;&#xA;&#xA;# Define path to singularity image:&#xA;cnvkit_container=&amp;quot;/path/to/cnvkit_latest.sif&amp;quot;&#xA;&#xA;cnvkit_path=&amp;quot;/usr/local/bin/cnvkit.py&amp;quot;&#xA;&#xA;singularity exec &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$cnvkit_container&quot; &quot;$&lt;/span&gt;cnvkit_path&amp;quot; call &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$directory/sample.cns&quot; -o &quot;$&lt;/span&gt;directory/sample.call.cns&amp;quot;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="17244" LastEditDate="2023-06-05T21:14:00.870" LastActivityDate="2023-06-05T21:14:00.870" CommentCount="23" ContentLicense="CC BY-SA 4.0" />
  <row Id="21079" PostTypeId="2" ParentId="21070" CreationDate="2023-05-27T22:25:36.980" Score="1" Body="&lt;p&gt;Not very familiar with what StringTie is doing, but in general reference annotations are helpful in providing a prior for what the tool expects.&lt;/p&gt;&#xA;&lt;p&gt;I would suggest looking at the &lt;a href=&quot;http://ccb.jhu.edu/software/stringtie/index.shtml?t=manual&quot; rel=&quot;nofollow noreferrer&quot;&gt;manual&lt;/a&gt; and using some of the provided options and outputs to probe the coverage of the various transcript species (I believe these are in the &amp;quot;attributes&amp;quot; section of the GTF output).&lt;/p&gt;&#xA;&lt;p&gt;If the transcripts that are missing when you use &lt;code&gt;-G&lt;/code&gt; have a coverage of just a couple of reads, they are likely to be less interesting. On the other hand, if they have relatively high coverage, then you have more reason to be skeptical of the existing annotation.&lt;/p&gt;&#xA;&lt;p&gt;My guess is that the tool is more heavily weighting existing transcripts based on the annotation, and therefore excluding the extra transcripts as being below some threshold.&lt;/p&gt;&#xA;&lt;p&gt;I'd recommend consulting the &amp;quot;recommended workflow&amp;quot; in the manual that supposedly allows for novel isoforms. Ensure that you are using this workflow and not the more streamlined workflow, which does not allow for novel isoforms.&lt;/p&gt;&#xA;&lt;p&gt;See also the comment suggesting comparing to other IsoSeq-oriented tools.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-05-27T22:25:36.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21081" PostTypeId="1" CreationDate="2023-05-29T22:08:15.590" Score="2" ViewCount="26" Body="&lt;p&gt;Hi everyone I had a question on if it is possible to calculate (Site Frequency Spectrum)SFS using STR data. Please mention if you know any programm that can be used for that!&#xA;I have tried Arlequin but it doesn't offer this option for STR. Personally I haven't find anything that can do this type of calculation with STR!&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="17573" LastActivityDate="2023-05-29T22:08:15.590" Title="Calculation of Site Frequency Spectrum (SFS) using STR data" Tags="|genetics|biopython|populations|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21082" PostTypeId="1" CreationDate="2023-05-29T22:55:32.793" Score="1" ViewCount="28" Body="&lt;p&gt;Trying to load the XML data from this url&#xA;URL: &lt;a href=&quot;https://www.senate.gov/general/contact_information/senators_cfm.xml&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.senate.gov/general/contact_information/senators_cfm.xml&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Load required library&#xA;library(RCurl)&#xA;&#xA;# Define the URL&#xA;url &amp;lt;- &amp;quot;https://www.senate.gov/general/contact_information/senators_cfm.xml&amp;quot;&#xA;&#xA;# Retrieve XML content from the URL&#xA;xml_content &amp;lt;- getURL(url)&#xA;&#xA;# Load XML library&#xA;library(XML)&#xA;&#xA;# Parse the XML content&#xA;xml_parsed &amp;lt;- xmlParse(xml_content)&#xA;&#xA;# Extract data from XML and convert it to a data frame&#xA;data &amp;lt;- xmlToDataFrame(xml_parsed)&#xA;&#xA;# Display the dimensions of the data&#xA;dim(data)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But my last row is all NA's. I know the dim is suppose to be 100 due to the number of senators per state but it's currently 101 due to that empty row.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; data[1:5]&#xA;            member_full    last_name     first_name party state&#xA;1        Baldwin (D-WI)      Baldwin          Tammy     D    WI&#xA;2       Barrasso (R-WY)     Barrasso           John     R    WY&#xA;3         Bennet (D-CO)       Bennet     Michael F.     D    CO&#xA;4      Blackburn (R-TN)    Blackburn         Marsha     R    TN&#xA;5     Blumenthal (D-CT)   Blumenthal        Richard     D    CT&#xA;6         Booker (D-NJ)       Booker        Cory A.     D    NJ&#xA;7        Boozman (R-AR)      Boozman           John     R    AR&#xA;8          Braun (R-IN)        Braun           Mike     R    IN&#xA;9          Britt (R-AL)        Britt     Katie Boyd     R    AL&#xA;10         Brown (D-OH)        Brown        Sherrod     D    OH&#xA;11          Budd (R-NC)         Budd            Ted     R    NC&#xA;12      Cantwell (D-WA)     Cantwell          Maria     D    WA&#xA;13        Capito (R-WV)       Capito  Shelley Moore     R    WV&#xA;14        Cardin (D-MD)       Cardin    Benjamin L.     D    MD&#xA;15        Carper (D-DE)       Carper      Thomas R.     D    DE&#xA;16         Casey (D-PA)        Casey Robert P., Jr.     D    PA&#xA;17       Cassidy (R-LA)      Cassidy           Bill     R    LA&#xA;18       Collins (R-ME)      Collins       Susan M.     R    ME&#xA;19         Coons (D-DE)        Coons Christopher A.     D    DE&#xA;20        Cornyn (R-TX)       Cornyn           John     R    TX&#xA;21  Cortez Masto (D-NV) Cortez Masto      Catherine     D    NV&#xA;22        Cotton (R-AR)       Cotton            Tom     R    AR&#xA;23        Cramer (R-ND)       Cramer          Kevin     R    ND&#xA;24         Crapo (R-ID)        Crapo           Mike     R    ID&#xA;25          Cruz (R-TX)         Cruz            Ted     R    TX&#xA;26        Daines (R-MT)       Daines          Steve     R    MT&#xA;27     Duckworth (D-IL)    Duckworth          Tammy     D    IL&#xA;28        Durbin (D-IL)       Durbin     Richard J.     D    IL&#xA;29         Ernst (R-IA)        Ernst           Joni     R    IA&#xA;30     Feinstein (D-CA)    Feinstein         Dianne     D    CA&#xA;31     Fetterman (D-PA)    Fetterman           John     D    PA&#xA;32       Fischer (R-NE)      Fischer            Deb     R    NE&#xA;33    Gillibrand (D-NY)   Gillibrand     Kirsten E.     D    NY&#xA;34        Graham (R-SC)       Graham        Lindsey     R    SC&#xA;35      Grassley (R-IA)     Grassley          Chuck     R    IA&#xA;36       Hagerty (R-TN)      Hagerty           Bill     R    TN&#xA;37        Hassan (D-NH)       Hassan  Margaret Wood     D    NH&#xA;38        Hawley (R-MO)       Hawley           Josh     R    MO&#xA;39      Heinrich (D-NM)     Heinrich         Martin     D    NM&#xA;40  Hickenlooper (D-CO) Hickenlooper        John W.     D    CO&#xA;41        Hirono (D-HI)       Hirono       Mazie K.     D    HI&#xA;42        Hoeven (R-ND)       Hoeven           John     R    ND&#xA;43    Hyde-Smith (R-MS)   Hyde-Smith          Cindy     R    MS&#xA;44       Johnson (R-WI)      Johnson            Ron     R    WI&#xA;45         Kaine (D-VA)        Kaine            Tim     D    VA&#xA;46         Kelly (D-AZ)        Kelly           Mark     D    AZ&#xA;47       Kennedy (R-LA)      Kennedy           John     R    LA&#xA;48          King (I-ME)         King  Angus S., Jr.     I    ME&#xA;49     Klobuchar (D-MN)    Klobuchar            Amy     D    MN&#xA;50      Lankford (R-OK)     Lankford          James     R    OK&#xA;51           Lee (R-UT)          Lee           Mike     R    UT&#xA;52        LujÃ¡n (D-NM)       LujÃ¡n        Ben Ray     D    NM&#xA;53        Lummis (R-WY)       Lummis     Cynthia M.     R    WY&#xA;54       Manchin (D-WV)      Manchin       Joe, III     D    WV&#xA;55        Markey (D-MA)       Markey      Edward J.     D    MA&#xA;56      Marshall (R-KS)     Marshall          Roger     R    KS&#xA;57     McConnell (R-KY)    McConnell          Mitch     R    KY&#xA;58      Menendez (D-NJ)     Menendez         Robert     D    NJ&#xA;59       Merkley (D-OR)      Merkley           Jeff     D    OR&#xA;60         Moran (R-KS)        Moran          Jerry     R    KS&#xA;61        Mullin (R-OK)       Mullin      Markwayne     R    OK&#xA;62     Murkowski (R-AK)    Murkowski           Lisa     R    AK&#xA;63        Murphy (D-CT)       Murphy    Christopher     D    CT&#xA;64        Murray (D-WA)       Murray          Patty     D    WA&#xA;65        Ossoff (D-GA)       Ossoff            Jon     D    GA&#xA;66       Padilla (D-CA)      Padilla           Alex     D    CA&#xA;67          Paul (R-KY)         Paul           Rand     R    KY&#xA;68        Peters (D-MI)       Peters        Gary C.     D    MI&#xA;69          Reed (D-RI)         Reed           Jack     D    RI&#xA;70      Ricketts (R-NE)     Ricketts           Pete     R    NE&#xA;71         Risch (R-ID)        Risch       James E.     R    ID&#xA;72        Romney (R-UT)       Romney           Mitt     R    UT&#xA;73         Rosen (D-NV)        Rosen          Jacky     D    NV&#xA;74        Rounds (R-SD)       Rounds           Mike     R    SD&#xA;75         Rubio (R-FL)        Rubio          Marco     R    FL&#xA;76       Sanders (I-VT)      Sanders        Bernard     I    VT&#xA;77        Schatz (D-HI)       Schatz          Brian     D    HI&#xA;78       Schmitt (R-MO)      Schmitt           Eric     R    MO&#xA;79       Schumer (D-NY)      Schumer     Charles E.     D    NY&#xA;80         Scott (R-FL)        Scott           Rick     R    FL&#xA;81         Scott (R-SC)        Scott            Tim     R    SC&#xA;82       Shaheen (D-NH)      Shaheen         Jeanne     D    NH&#xA;83        Sinema (I-AZ)       Sinema        Kyrsten     I    AZ&#xA;84         Smith (D-MN)        Smith           Tina     D    MN&#xA;85      Stabenow (D-MI)     Stabenow         Debbie     D    MI&#xA;86      Sullivan (R-AK)     Sullivan            Dan     R    AK&#xA;87        Tester (D-MT)       Tester            Jon     D    MT&#xA;88         Thune (R-SD)        Thune           John     R    SD&#xA;89        Tillis (R-NC)       Tillis           Thom     R    NC&#xA;90    Tuberville (R-AL)   Tuberville          Tommy     R    AL&#xA;91    Van Hollen (D-MD)   Van Hollen          Chris     D    MD&#xA;92         Vance (R-OH)        Vance           J.D.     R    OH&#xA;93        Warner (D-VA)       Warner        Mark R.     D    VA&#xA;94       Warnock (D-GA)      Warnock     Raphael G.     D    GA&#xA;95        Warren (D-MA)       Warren      Elizabeth     D    MA&#xA;96         Welch (D-VT)        Welch          Peter     D    VT&#xA;97    Whitehouse (D-RI)   Whitehouse        Sheldon     D    RI&#xA;98        Wicker (R-MS)       Wicker       Roger F.     R    MS&#xA;99         Wyden (D-OR)        Wyden            Ron     D    OR&#xA;100        Young (R-IN)        Young           Todd     R    IN&#xA;101                &amp;lt;NA&amp;gt;         &amp;lt;NA&amp;gt;           &amp;lt;NA&amp;gt;  &amp;lt;NA&amp;gt;  &amp;lt;NA&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16911" LastActivityDate="2023-05-29T22:55:32.793" Title="NA values from a URL import" Tags="|r|rcurl|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21083" PostTypeId="2" ParentId="20964" CreationDate="2023-05-30T07:10:57.053" Score="5" Body="&lt;p&gt;If your objective is to convert Bash scripts into Nextflow code, you might find Viash (&lt;a href=&quot;https://viash.io&quot; rel=&quot;noreferrer&quot;&gt;https://viash.io&lt;/a&gt;) a particularly useful open source tool. It automatically generates independent Nextflow modules from each of your scripts, along with the corresponding Docker containers for each module.&#xA;Docker containers are generated on the fly based on your config description in Viash.&#xA;These modules can be coupled together to build a (modular) Nextflow pipeline.&lt;/p&gt;&#xA;" OwnerUserId="17574" LastActivityDate="2023-05-30T07:10:57.053" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21085" PostTypeId="2" ParentId="21077" CreationDate="2023-05-30T16:36:53.477" Score="0" Body="&lt;p&gt;Thanks to OP for sharing that their image is from &lt;a href=&quot;https://hub.docker.com/r/etal/cnvkit&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://hub.docker.com/r/etal/cnvkit&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I downloaded this and it seems pretty simple.&lt;/p&gt;&#xA;&lt;p&gt;My tests:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;singularity pull docker://etal/cnvkit:latest&#xA;$ singularity shell ./cnvkit_latest.sif&#xA;INFO:    Converting SIF file to temporary sandbox...&#xA;bash: warning: setlocale: LC_ALL: cannot change locale (en_US.UTF-8)&#xA;Singularity&amp;gt; ls /&#xA;bin  boot  dev  environment  etc  home  lib  lib32  lib64  libx32  media  mnt  opt  proc  root  run  sbin  singularity  srv  sys  tmp  usr  var&#xA;Singularity&amp;gt; ls /singularity&#xA;/singularity&#xA;Singularity&amp;gt; cnvkit.py&#xA;usage: cnvkit.py [-h]&#xA;                 {batch,target,access,antitarget,autobin,coverage,reference,fix,segment,call,diagram,scatter,heatmap,breaks,genemetrics,gainloss,sex,gender,metrics,segmetrics,bintest,import-picard,import-seg,import-theta,import-rna,export,version}&#xA;                 ...&#xA;&#xA;CNVkit, a command-line toolkit for copy number analysis.&#xA;&#xA;positional arguments:&#xA;  {batch,target,access,antitarget,autobin,coverage,reference,fix,segment,call,diagram,scatter,heatmap,breaks,genemetrics,gainloss,sex,gender,metrics,segmetrics,bintest,import-picard,import-seg,import-theta,import-rna,export,version}&#xA;                        Sub-commands (use with -h for more info)&#xA;    batch               Run the complete CNVkit pipeline on one or more BAM files.&#xA;    target              Transform bait intervals into targets more suitable for CNVkit.&#xA;    access              List the locations of accessible sequence regions in a FASTA file.&#xA;    antitarget          Derive off-target (&amp;quot;antitarget&amp;quot;) bins from target regions.&#xA;    autobin             Quickly calculate reasonable bin sizes from BAM read counts.&#xA;    coverage            Calculate coverage in the given regions from BAM read depths.&#xA;    reference           Compile a coverage reference from the given files (normal samples).&#xA;    fix                 Combine target and antitarget coverages and correct for biases. Adjust raw coverage data according to the given reference, correct potential biases and re-center.&#xA;    segment             Infer copy number segments from the given coverage table.&#xA;    call                Call copy number variants from segmented log2 ratios.&#xA;    diagram             Draw copy number (log2 coverages, segments) on chromosomes as a diagram. If both the raw probes and segments are given, show them side-by-side on each chromosome (segments on the left side, probes on the right&#xA;                        side).&#xA;    scatter             Plot probe log2 coverages and segmentation calls together.&#xA;    heatmap             Plot copy number for multiple samples as a heatmap.&#xA;    breaks              List the targeted genes in which a copy number breakpoint occurs.&#xA;    genemetrics         Identify targeted genes with copy number gain or loss.&#xA;    sex                 Guess samples' sex from the relative coverage of chromosomes X and Y.&#xA;    metrics             Compute coverage deviations and other metrics for self-evaluation.&#xA;    segmetrics          Compute segment-level metrics from bin-level log2 ratios.&#xA;    bintest             Test for single-bin copy number alterations.&#xA;    import-picard       Convert Picard CalculateHsMetrics tabular output to CNVkit .cnn files. The input file is generated by the PER_TARGET_COVERAGE option in the CalculateHsMetrics script in Picard tools. If 'antitarget' is in the&#xA;                        input filename, the generated output filename will have the suffix '.antitargetcoverage.cnn', otherwise '.targetcoverage.cnn'.&#xA;    import-seg          Convert a SEG file to CNVkit .cns files.&#xA;    import-theta        Convert THetA output to a BED-like, CNVkit-like tabular format. Equivalently, use the THetA results file to convert CNVkit .cns segments to integer copy number calls.&#xA;    import-rna          Convert a cohort of per-gene log2 ratios to CNVkit .cnr format.&#xA;    export              Convert CNVkit output files to another format.&#xA;    version             Display this program's version.&#xA;&#xA;options:&#xA;  -h, --help            show this help message and exit&#xA;&#xA;See the online manual for details: https://cnvkit.readthedocs.io&#xA;Singularity&amp;gt; which cnvkit.py&#xA;/usr/local/bin/cnvkit.py&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see, since &lt;code&gt;cnvkit.py&lt;/code&gt; is in &lt;code&gt;/usr/local/bin/&lt;/code&gt;, simply typing &lt;code&gt;cnvkit.py&lt;/code&gt; within the shell works. Let's try an exec with no frills:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ singularity exec cnvkit_latest.sif cnvkit.py&#xA;INFO:    Converting SIF file to temporary sandbox...&#xA;usage: cnvkit.py [-h]&#xA;                 {batch,target,access,antitarget,autobin,coverage,reference,fix,segment,call,diagram,scatter,heatmap,breaks,genemetrics,gainloss,sex,gender,metrics,segmetrics,bintest,import-picard,import-seg,import-theta,import-rna,export,version}&#xA;                 ...&#xA;&#xA;CNVkit, a command-line toolkit for copy number analysis.&#xA;&#xA;positional arguments:&#xA;  {batch,target,access,antitarget,autobin,coverage,reference,fix,segment,call,diagram,scatter,heatmap,breaks,genemetrics,gainloss,sex,gender,metrics,segmetrics,bintest,import-picard,import-seg,import-theta,import-rna,export,version}&#xA;                        Sub-commands (use with -h for more info)&#xA;    batch               Run the complete CNVkit pipeline on one or more BAM files.&#xA;    target              Transform bait intervals into targets more suitable for CNVkit.&#xA;    access              List the locations of accessible sequence regions in a FASTA file.&#xA;    antitarget          Derive off-target (&amp;quot;antitarget&amp;quot;) bins from target regions.&#xA;    autobin             Quickly calculate reasonable bin sizes from BAM read counts.&#xA;    coverage            Calculate coverage in the given regions from BAM read depths.&#xA;    reference           Compile a coverage reference from the given files (normal samples).&#xA;    fix                 Combine target and antitarget coverages and correct for biases. Adjust raw coverage data according to the given reference, correct potential biases and re-center.&#xA;    segment             Infer copy number segments from the given coverage table.&#xA;    call                Call copy number variants from segmented log2 ratios.&#xA;    diagram             Draw copy number (log2 coverages, segments) on chromosomes as a diagram. If both the raw probes and segments are given, show them side-by-side on each chromosome (segments on the left side, probes on the right&#xA;                        side).&#xA;    scatter             Plot probe log2 coverages and segmentation calls together.&#xA;    heatmap             Plot copy number for multiple samples as a heatmap.&#xA;    breaks              List the targeted genes in which a copy number breakpoint occurs.&#xA;    genemetrics         Identify targeted genes with copy number gain or loss.&#xA;    sex                 Guess samples' sex from the relative coverage of chromosomes X and Y.&#xA;    metrics             Compute coverage deviations and other metrics for self-evaluation.&#xA;    segmetrics          Compute segment-level metrics from bin-level log2 ratios.&#xA;    bintest             Test for single-bin copy number alterations.&#xA;    import-picard       Convert Picard CalculateHsMetrics tabular output to CNVkit .cnn files. The input file is generated by the PER_TARGET_COVERAGE option in the CalculateHsMetrics script in Picard tools. If 'antitarget' is in the&#xA;                        input filename, the generated output filename will have the suffix '.antitargetcoverage.cnn', otherwise '.targetcoverage.cnn'.&#xA;    import-seg          Convert a SEG file to CNVkit .cns files.&#xA;    import-theta        Convert THetA output to a BED-like, CNVkit-like tabular format. Equivalently, use the THetA results file to convert CNVkit .cns segments to integer copy number calls.&#xA;    import-rna          Convert a cohort of per-gene log2 ratios to CNVkit .cnr format.&#xA;    export              Convert CNVkit output files to another format.&#xA;    version             Display this program's version.&#xA;&#xA;options:&#xA;  -h, --help            show this help message and exit&#xA;&#xA;See the online manual for details: https://cnvkit.readthedocs.io&#xA;INFO:    Cleaning up image...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The answer seems to be a lot easier than having to find &lt;code&gt;cnvkit.py&lt;/code&gt;, as it is already in the PATH.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-05-30T16:36:53.477" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21086" PostTypeId="1" CreationDate="2023-05-30T18:00:50.473" Score="1" ViewCount="18" Body="&lt;p&gt;I am trying to run Tax4Fun in MicrobiomeAnalyst, but am running into trouble with my table formatting. From the qiime2 pathway, I have an OTU, metadata and taxonomic table formatted like so: &lt;a href=&quot;https://i.stack.imgur.com/tLAA3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tLAA3.png&quot; alt=&quot;feature table&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pFHTb.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pFHTb.png&quot; alt=&quot;metadata&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/9E47f.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/9E47f.png&quot; alt=&quot;taxonomic table&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In order to use Tax4Fun, I understand that I will need to reformat my OTU table to include sequence IDs so it looks like this example:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/tJGkj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/tJGkj.png&quot; alt=&quot;sample seq table&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, I cannot figure out how to merge my sequence files from the qiime2 pathway with the OTU table so it looks like that. Any suggestions would be much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="17577" LastActivityDate="2023-05-30T18:00:50.473" Title="How Can You Reformat an OTU Table from qiime2 pathway to Include Sequence IDs?" Tags="|sequence-annotation|genomics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21087" PostTypeId="1" CreationDate="2023-05-30T19:06:46.863" Score="0" ViewCount="49" Body="&lt;p&gt;I have a data frame columns are pathways and genes are rows&lt;/p&gt;&#xA;&lt;p&gt;Like&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;head(a[1:2,1:2])&#xA;  Release.of.Cancer.Cell.Antigens Cancer.Antigen.Presentation&#xA;1                          ARID1A                        ARG2&#xA;2                             ATM                        ATF3&#xA;&#xA;str(a[1:2,1:2])&#xA;'data.frame':   2 obs. of  2 variables:&#xA; &lt;span class=&quot;math-container&quot;&gt;$ Release.of.Cancer.Cell.Antigens: chr  &quot;ARID1A&quot; &quot;ATM&quot;&#xA; $&lt;/span&gt; Cancer.Antigen.Presentation    : chr  &amp;quot;ARG2&amp;quot; &amp;quot;ATF3&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to make a list of pathways containing their genes, something like&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;str(h.sets)&#xA;List of 50&#xA; $ HALLMARK_ADIPOGENESIS                     : chr [1:208] &amp;quot;ABCA1&amp;quot; &amp;quot;ABCB8&amp;quot; &amp;quot;ACAA2&amp;quot; &amp;quot;ACADL&amp;quot; ...&#xA; $ HALLMARK_ALLOGRAFT_REJECTION              : chr [1:332] &amp;quot;AARS1&amp;quot; &amp;quot;ABCE1&amp;quot; &amp;quot;ABI1&amp;quot; &amp;quot;ACHE&amp;quot; ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I done so&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;h.names &amp;lt;- unique(colnames(a))&#xA;h.sets &amp;lt;- vector(&amp;quot;list&amp;quot;,length=length(h.names))&#xA;names(h.sets) &amp;lt;- h.names&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But here I get error here&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;for (i in names(h.sets)) {&#xA;    h.sets[[i]] &amp;lt;- pull(a[colnames(a)==i])&#xA;}&#xA;&#xA;Error in `pull()`:&#xA;! Can't extract columns past the end.&#xA;ℹ Location 1 doesn't exist.&#xA;ℹ There are only 0 columns.&#xA;Run `rlang::last_trace()` to see where the error occurred.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any help please&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="650" LastEditDate="2023-05-30T19:44:37.917" LastActivityDate="2023-05-30T19:44:37.917" Title="Making a list of lists in R" Tags="|r|pathway|gsea|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21088" PostTypeId="2" ParentId="5002" CreationDate="2023-05-30T21:47:29.100" Score="1" Body="&lt;p&gt;In case someone finds this years later: OP's problem was the use of the &amp;quot;-noint&amp;quot; flag. It stands for &amp;quot;no interspersed&amp;quot;, so the OP explicitly asked RepeatMasker  NOT to look for transposable elements. If they omitted the flag, RepeatMasker would have automatically downloaded mammal TE sequences (nowadays it uses Dfam). Even if they had a custom library like suggested above, they would still need to delete &amp;quot;-noint&amp;quot;. Check your commands!&lt;/p&gt;&#xA;" OwnerUserId="17578" LastActivityDate="2023-05-30T21:47:29.100" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21089" PostTypeId="1" CreationDate="2023-05-31T05:10:10.020" Score="1" ViewCount="28" Body="&lt;p&gt;I have whole-exome sequencing data of an immortalised non-tumor (normal) cell line that I wish to assess for the presence/absence of APC/Wnt mutations. This is to double check that the cell line is sufficiently &amp;quot;clean&amp;quot;, without mutations in key regions, for further experimental analysis.&lt;/p&gt;&#xA;&lt;p&gt;I understand that without a matched normal sample, I won't be able to distinguish if a variant is a somatic variant or a rare inherited germline mutation. As I just want to check for mutations in those regions, it is not important to distinguish between these types.&lt;/p&gt;&#xA;&lt;p&gt;Question: Should I use the GATK haplotypecaller or a somatic variant caller such as Mutect2 without a matched-normal sample?&lt;/p&gt;&#xA;" OwnerUserId="15940" LastActivityDate="2023-05-31T05:10:10.020" Title="Which type of variant caller should I use in a WES normal cell line sample?" Tags="|variant-calling|wes|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21091" PostTypeId="1" CreationDate="2023-05-31T15:49:04.083" Score="1" ViewCount="136" Body="&lt;p&gt;I have a couple of MAGs of interest that I am trying to submit to the NCBI (with annotations).&lt;/p&gt;&#xA;&lt;p&gt;The process of submitting the annotations is rather challenging to me. I have the output of Prokka, which includes multiple files, but the one I think I need is the .gff (which is GFF3 format).&lt;/p&gt;&#xA;&lt;p&gt;The NCBI website descibes how to obtain the required file format (.sqn) from .gff files (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genbank/genomes_gff/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/genbank/genomes_gff/&lt;/a&gt;). I downloaded the required file (table2asn), and ran the following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;table2asn&amp;quot; -M n -Z -J -c w -t &amp;quot;template.sbt&amp;quot; -V b -j &amp;quot;[organism=Organism] [strain=strain] [gcode=11]&amp;quot; -i &amp;quot;fasta.fna&amp;quot; -f &amp;quot;annotations.gff&amp;quot; -o output.sqn&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, I got an error:&#xA;&lt;code&gt;GFF annotation requires locus tags, which are missing from one or more genes, so the command line argument -locus-tag-prefix is needed.&lt;/code&gt; Which was a bit strange, since I could see that the original file had a locus_tag entry. The format liiked kind of like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;S6C7    Prodigal:002006 CDS 244817  246085  .   -   0   ID=AJNLIECJ_06723;inference=ab initio prediction:Prodigal:002006,similar to AA sequence:ISfinder:ISNpu7;locus_tag=AJNLIECJ_06723;product=IS701 family transposase ISNpu7&#xA;S6C7    Prodigal:002006 CDS 246142  246702  .   -   0   ID=AJNLIECJ_06724;inference=ab initio prediction:Prodigal:002006;locus_tag=AJNLIECJ_06724;product=hypothetical protein&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Anyway, I added &lt;code&gt;-locus-tag-prefix AJNLIECJ&lt;/code&gt; to my code, which at least got me an output, but now the output is a touch weird:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; gene            complement(246142..246702)&#xA;                 /locus_tag=&amp;quot;AJNLIECJ_006724&amp;quot;&#xA; CDS             complement(246142..246702)&#xA;                 /locus_tag=&amp;quot;AJNLIECJ_006724&amp;quot;&#xA;                 /inference=&amp;quot;ab initio prediction:Prodigal:002006&amp;quot;&#xA;                 /codon_start=1&#xA;                 /transl_table=11&#xA;                 /product=&amp;quot;hypothetical protein&amp;quot;&#xA;                 /protein_id=&amp;quot;AJNLIECJ:AJNLIECJ_06724&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So basically, the /protein_id is repeated, and sometimes the numbers between locus and protein ID don't even match (some of the protein ids 'go ahead' for some reason).&lt;/p&gt;&#xA;&lt;p&gt;My thoughts:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I have no idea how to fix this&lt;/li&gt;&#xA;&lt;li&gt;Is it even necessary to fix it? Since all NCBI protein IDs tend to be in a consistent format anyway, so no matter what I add, it will get updated.&lt;/li&gt;&#xA;&lt;li&gt;Should I remove the /protein_id field? How?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thank you for your time!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2024-02-08T14:55:02.333" Title="Prokka + table2asn (NCBI submission)" Tags="|sequence-annotation|text-processing|ncbi|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21092" PostTypeId="1" CreationDate="2023-05-31T16:01:06.137" Score="2" ViewCount="26" Body="&lt;p&gt;I am searching a protein sequences against a nucleotide database (&amp;quot;tblastn&amp;quot;) using mmseqs and need nucleotide sequence alignments as an output.&lt;/p&gt;&#xA;&lt;p&gt;I start by the following (pg 23 of &lt;a href=&quot;https://mmseqs.com/latest/userguide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://mmseqs.com/latest/userguide.pdf&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;mmseqs search ecoli_proteins ecoli_genome alnDB tmp &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and then get alignments as follows:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;mmseqs convertalis ecoli_proteins ecoli_genome alnDB alnDB.html --format-mode 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, this gives me protein sequence alignments, when I really want the nucleotide base alignments (i.e. would be aligned codon-by-codon).&lt;/p&gt;&#xA;&lt;p&gt;What is the proper way to get this output from mmseqs?&lt;/p&gt;&#xA;" OwnerUserId="5049" LastActivityDate="2023-05-31T16:01:06.137" Title="Nucleotide alignments from mmseqs &quot;tblastn&quot;" Tags="|genomics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21093" PostTypeId="1" AcceptedAnswerId="21098" CreationDate="2023-06-01T01:18:45.073" Score="2" ViewCount="288" Body="&lt;p&gt;I have up and down-regulated genes from bulk RNA result from DeSeq2. Genes that are differential in chromatin accessibility between two conditions from DiffBind. I would like to find the genes overlap between bulk RNA and ATAC-seq assay as these papers but don't know the code to get it. Would you please have a suggestion? Thank you so much  .&#xA;&lt;a href=&quot;https://i.stack.imgur.com/NbLvn.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NbLvn.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fnut.2021.742672/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper 1&lt;/a&gt;&#xA;&lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fvets.2022.925590/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;paper 2&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;&lt;em&gt;in silico&lt;/em&gt; DEG details&lt;/strong&gt;&#xA;I used intersect function in R which can do the same job. Which I am not sure is the experiment try to find transcription factors that cause the difference in phenotype between 2 conditions. DEG in bulk-RNA from DeSeq2 (log2 fold change &amp;gt;2.5 or log2 fold change &amp;lt; -2.5) intersect with genes differently in chromatin accessibility from DiffBind (log2 fold change &amp;gt;2.5 or log2 fold change &amp;lt; -2.5). I narrowed down to 142 genes.&lt;/p&gt;&#xA;&lt;p&gt;For atac-seq result, after applying filter log2fold &amp;gt; 2.5 or log2fold &amp;lt; -2.5 and p value &amp;lt; 0.00001, I still got around 12k genes that are different between 2 conditions. Meanwhile bulk-RNA after apply filter has only around 300 genes. The number of genes in human and pig are similar so I am not sure about the number of gene I have in atac-seq&lt;/p&gt;&#xA;&lt;p&gt;What do you think about this approach because I don't use MEME or TOMTOM?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I share with you what the author did.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot; In order to screen new transcription factors by combining with star transcription factors. First, we used MEME software to predict the motif of Duroc pigs' peaks. Second, we compared the transcription factor database with TOMTOM to predict all transcription factors that may bind to the chromatin open regions . Third, we found all possible binding sites of MYOD, MYOG, MYF5, MYF6 and MEF2 families and did a union. A total of 99 sites were found.&#xA;&amp;quot;Fourthly, we counted other transcription factors that may bind to these 99 sites and sorted them by the number of binding sites. All of the above statistics were manually counted.&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Will read more to give you a good reply. Thank you so much &lt;/p&gt;&#xA;" OwnerUserId="17585" LastEditorUserId="3848" LastEditDate="2023-06-02T02:14:00.817" LastActivityDate="2023-06-02T04:15:56.773" Title="Integrate bulk RNA and ATAC-seq genes" Tags="|rna-seq|deseq2|atac-seq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21094" PostTypeId="2" ParentId="21093" CreationDate="2023-06-01T07:29:28.533" Score="2" Body="&lt;p&gt;If I am getting your question right, all you need is &lt;a href=&quot;https://bioinfogp.cnb.csic.es/tools/venny/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioinfogp.cnb.csic.es/tools/venny/&lt;/a&gt;. I assume that your two lists corresponding to RNA-seq and ATAC-seq use the same set of gene identifiers (gene symbol, Entrez ID, Ensembl ID etc).&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;Results&lt;/code&gt; section is populated with entries that belongs to the corresponding &amp;quot;region&amp;quot; upon clicking on the regions of the Venn diagram.&lt;/p&gt;&#xA;&lt;p&gt;Personally I have never used it but my wet-lab colleagues use it all the time.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2023-06-01T08:42:28.067" LastActivityDate="2023-06-01T08:42:28.067" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21095" PostTypeId="2" ParentId="20686" CreationDate="2023-06-01T09:21:54.513" Score="2" Body="&lt;p&gt;The code can be made to work on &lt;em&gt;all&lt;/em&gt; samples by changing the interface of the process and using the &lt;code&gt;combine&lt;/code&gt; operator:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ref_chl = Channel&#xA;      .fromPath(&amp;quot;sequences/hg38.fa&amp;quot;)&#xA;ref_index_chl = Channel&#xA;      .fromPath(&amp;quot;sequences/hg38.fa.fai&amp;quot;)&#xA;amplicon_chl = Channel&#xA;      .fromPath(&amp;quot;sequences/tp53_amplicon.hg38.interval_list&amp;quot;)&#xA;target_chl = Channel&#xA;      .fromPath(&amp;quot;sequences/TP53.hg38.interval_list&amp;quot;)&#xA;&#xA;bam_chl = Channel&#xA;     .fromPath(&amp;quot;bam_files/*.md.bam&amp;quot;)&#xA;     .map{ item -&amp;gt; tuple(item.baseName.substring(0, 18), item) }&#xA;     .combine(ref_chl)&#xA;     .combine(ref_index_chl)&#xA;     .combine(amplicon_chl)&#xA;     .combine(target_chl)&#xA;&#xA;process get_pcr_metric {&#xA;&#xA;  publishDir path: 'pcr_metric', pattern: '*.csv'&#xA;  &#xA;  input:&#xA;  tuple val(pair_id), path(bam), path(ref), path(ref_index), path(amplicon), path(target)&#xA;&#xA;  output:&#xA;  path(&amp;quot;*.pcr_metrics.csv&amp;quot;)&#xA;&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;  java -jar /picard.jar CollectTargetedPcrMetrics \&#xA;       I=&lt;span class=&quot;math-container&quot;&gt;$bam \&#xA;   O=$&lt;/span&gt;{pair_id}_pcr_metrics.txt \&#xA;       R=&lt;span class=&quot;math-container&quot;&gt;$ref \&#xA;   AMPLICON_INTERVALS=$&lt;/span&gt;amplicon \&#xA;       TARGET_INTERVALS=$target&#xA;       &#xA;  sed -n -e 7p -e 8p &lt;span class=&quot;math-container&quot;&gt;${pair_id}_pcr_metrics.txt |  sed 's/\t/,/g'   &amp;gt; $&lt;/span&gt;{pair_id}.pcr_metrics.csv&#xA;  &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;workflow {&#xA;  get_pcr_metric(bam_chl)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;combine&lt;/code&gt; operator as the name implies generates all combinations. The different file objects are then in one tuple.&lt;/p&gt;&#xA;&lt;p&gt;Since only one &lt;code&gt;Channel&lt;/code&gt; is passed to the process, it's now also possible to rewrite the workflow code as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow {&#xA;&#xA;  refs = [&#xA;    &amp;quot;sequences/hg38.fa&amp;quot;,&#xA;    &amp;quot;sequences/hg38.fa.fai&amp;quot;,&#xA;    &amp;quot;sequences/tp53_amplicon.hg38.interval_list&amp;quot;,&#xA;    &amp;quot;sequences/TP53.hg38.interval_list&amp;quot;&#xA;  ]&#xA;&#xA;  ref_chl = Channel.from(refs)&#xA;   | map{ pointer -&amp;gt; file(pointer) }&#xA;   | collect&#xA;&#xA;  Channel.fromPath(&amp;quot;bam_files/*.md.bam&amp;quot;)&#xA;   | map{ item -&amp;gt; tuple(item.baseName.substring(0, 18), item) }&#xA;   | combine( ref_chl )&#xA;   | get_pcr_metric&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17587" LastActivityDate="2023-06-01T09:21:54.513" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21096" PostTypeId="1" CreationDate="2023-06-01T10:01:32.800" Score="1" ViewCount="198" Body="&lt;p&gt;i dont know how to build/pull this container (&lt;a href=&quot;https://codebase.helmholtz.cloud/proteinprompt/ProteinPrompt&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://codebase.helmholtz.cloud/proteinprompt/ProteinPrompt&lt;/a&gt;) can you advice me how to install it?&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&lt;p&gt;My soft - Ubuntu 20.04 lts&lt;/p&gt;&#xA;&lt;p&gt;Thanks to &lt;a href=&quot;https://bioinformatics.stackexchange.com/users/3848/m&quot;&gt;@M__&lt;/a&gt; I manage to build image (almost). But I have an issue :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;roboczy@marcin-B550-AORUS-ELITE-AX-V2:/home/marcin/ProteinPrompt$ docker build -t dockerfile .&#xA;[+] Building 1.0s (15/15) FINISHED                                              &#xA; =&amp;gt; [internal] load build definition from Dockerfile                       0.0s&#xA; =&amp;gt; =&amp;gt; transferring dockerfile: 1.19kB                                     0.0s&#xA; =&amp;gt; [internal] load .dockerignore                                          0.0s&#xA; =&amp;gt; =&amp;gt; transferring context: 2B                                            0.0s&#xA; =&amp;gt; [internal] load metadata for docker.io/continuumio/miniconda3:4.10.3-  0.9s&#xA; =&amp;gt; [ 1/10] FROM docker.io/continuumio/miniconda3:4.10.3-alpine@sha256:e0  0.0s&#xA; =&amp;gt; [internal] load build context                                          0.0s&#xA; =&amp;gt; =&amp;gt; transferring context: 650B                                          0.0s&#xA; =&amp;gt; CACHED [ 2/10] RUN rm /bin/sh &amp;amp;&amp;amp; ln -s /bin/bash /bin/sh               0.0s&#xA; =&amp;gt; CACHED [ 3/10] RUN conda update -n base conda                          0.0s&#xA; =&amp;gt; CACHED [ 4/10] COPY ./data/proteinPrompt.yml /                         0.0s&#xA; =&amp;gt; CACHED [ 5/10] RUN conda env update --name root -f /proteinPrompt.yml  0.0s&#xA; =&amp;gt; CACHED [ 6/10] RUN mkdir /proteinPrompt/                               0.0s&#xA; =&amp;gt; CACHED [ 7/10] COPY ./proteinPrompt/*py /proteinPrompt/                0.0s&#xA; =&amp;gt; CACHED [ 8/10] RUN mkdir /model/                                       0.0s&#xA; =&amp;gt; CACHED [ 9/10] RUN wget https://zenodo.org/record/5708224/files/rf_hu  0.0s&#xA; =&amp;gt; CACHED [10/10] COPY ./data/all.proteins.HUMAN.autocorrelation.pbz2 /m  0.0s&#xA; =&amp;gt; exporting to image                                                     0.0s&#xA; =&amp;gt; =&amp;gt; exporting layers                                                    0.0s&#xA; =&amp;gt; =&amp;gt; writing image sha256:ae93ff408fb6468280a5f0344bb2ac5ac811df2e5d32a  0.0s&#xA; =&amp;gt; =&amp;gt; naming to docker.io/library/dockerfile                              0.0s&#xA;WARNING: buildx: failed to read current commit information with git rev-parse --is-inside-work-tree&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and image is named 'dockerfile'? and it do not work&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;roboczy@marcin-B550-AORUS-ELITE-AX-V2:/home/marcin/ProteinPrompt$ docker images&#xA;REPOSITORY    TAG           IMAGE ID       CREATED          SIZE&#xA;dockerfile    latest        ae93ff408fb6   11 minutes ago   2.02GB&#xA;alphafold     latest        c41a4b5e9d48   13 days ago      10GB&#xA;nvidia/cuda   11.0.3-base   4f3e95122fab   23 months ago    122MB&#xA;&#xA;roboczy@marcin-B550-AORUS-ELITE-AX-V2:/home/marcin/ProteinPrompt&lt;span class=&quot;math-container&quot;&gt;$ docker run  -v $&lt;/span&gt;(pwd)/data:/data proteinprompt search -f data/input.fa -o data/output.csv&#xA;Unable to find image 'proteinprompt:latest' locally&#xA;docker: Error response from daemon: pull access denied for proteinprompt, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.&#xA;See 'docker run --help'.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;EDIT 2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;I try to edit dockerfile (in several places)&#xA;&#xA;FROM continuumio/miniconda3:4.10.3-alpine&#xA;MAINTAINER Sebastian Canzler &amp;lt;sebastian.canzler@ufz.de&amp;gt;&#xA;LABEL authors=&amp;quot;sebastian.canzler@ufz.de&amp;quot; \&#xA;    description=&amp;quot;Docker image of ProteinPrompt - a tool to predict protein-protein interactions based on amino acid sequences.&amp;quot;&#xA;RUN apt-get update &amp;amp;&amp;amp; apt-get install -y git #&amp;lt;-----HERE&#xA;&#xA;#Try This to get a bash in order to run 'source activate modfinder'&#xA;RUN rm /bin/sh &amp;amp;&amp;amp; ln -s /bin/bash /bin/sh&#xA;&#xA;RUN conda update -n base conda&#xA;COPY ./data/proteinPrompt.yml /&#xA;RUN conda env update --name root -f /proteinPrompt.yml &amp;amp;&amp;amp; \&#xA;    rm -rf /opt/conda/pkgs/*&#xA;ENV PATH /opt/conda/envs/proteinPrompt/bin:&lt;span class=&quot;math-container&quot;&gt;$PATH&#xA;# copy the proteinPrompt files&#xA;RUN mkdir /proteinPrompt/&#xA;COPY ./proteinPrompt/*py /proteinPrompt/&#xA;ENV PATH /proteinPrompt:$&lt;/span&gt;PATH&#xA;&#xA;# get the random forest model from zenodo&#xA;RUN mkdir /model/&#xA;#COPY ./data/rf_human_750_inv.rev.pkl /model/random_forest.pkl&#xA;RUN wget https://zenodo.org/record/5708224/files/rf_human_750_inv.rev.pkl?download=1 -O /model/random_forest.pkl&#xA;COPY ./data/all.proteins.HUMAN.autocorrelation.pbz2 /model/all.proteins.HUMAN.autocorrelation.pbz2&#xA;&#xA;&#xA;&#xA;# make proteinPrompt executable&#xA;ENTRYPOINT [&amp;quot;python&amp;quot;, &amp;quot;/proteinPrompt/ProteinPrompt.py&amp;quot;]&#xA;#CMD [&amp;quot;-h&amp;quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but there is a bash problem:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;roboczy@marcin-B550-AORUS-ELITE-AX-V2:/home/marcin/ProteinPrompt&lt;span class=&quot;math-container&quot;&gt;$ docker build -t dockerfile .&#xA;[+] Building 1.6s (10/15)                                                       &#xA; =&amp;gt; [internal] load .dockerignore                                          0.0s&#xA; =&amp;gt; =&amp;gt; transferring context: 2B                                            0.0s&#xA; =&amp;gt; [internal] load build definition from Dockerfile                       0.0s&#xA; =&amp;gt; =&amp;gt; transferring dockerfile: 1.23kB                                     0.0s&#xA; =&amp;gt; [internal] load metadata for docker.io/continuumio/miniconda3:4.10.3-  1.0s&#xA; =&amp;gt; [ 1/11] FROM docker.io/continuumio/miniconda3:4.10.3-alpine@sha256:e0  0.0s&#xA; =&amp;gt; [internal] load build context                                          0.0s&#xA; =&amp;gt; =&amp;gt; transferring context: 650B                                          0.0s&#xA; =&amp;gt; CACHED [ 2/11] RUN rm /bin/sh &amp;amp;&amp;amp; ln -s /bin/bash /bin/sh               0.0s&#xA; =&amp;gt; CACHED [ 3/11] RUN conda update -n base conda                          0.0s&#xA; =&amp;gt; CACHED [ 4/11] COPY ./data/proteinPrompt.yml /                         0.0s&#xA; =&amp;gt; CACHED [ 5/11] RUN conda env update --name root -f /proteinPrompt.yml  0.0s&#xA; =&amp;gt; ERROR [ 6/11] RUN apt-get update &amp;amp;&amp;amp; apt-get install -y git             0.5s&#xA;------&#xA; &amp;gt; [ 6/11] RUN apt-get update &amp;amp;&amp;amp; apt-get install -y git:&#xA;#0 0.537 /bin/ash: apt-get: not found&#xA;------&#xA;WARNING: buildx: failed to read current commit information with git rev-parse --is-inside-work-tree&#xA;Dockerfile:15&#xA;--------------------&#xA;  13 |         rm -rf /opt/conda/pkgs/*&#xA;  14 |     ENV PATH /opt/conda/envs/proteinPrompt/bin:$&lt;/span&gt;PATH&#xA;  15 | &amp;gt;&amp;gt;&amp;gt; RUN apt-get update &amp;amp;&amp;amp; apt-get install -y git&#xA;  16 |     &#xA;  17 |     # copy the proteinPrompt files&#xA;--------------------&#xA;ERROR: failed to solve: process &amp;quot;/bin/ash -eo pipefail -c apt-get update &amp;amp;&amp;amp; apt-get install -y git&amp;quot; did not complete successfully: exit code: 127&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;EDIT 3:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;roboczy@marcin-B550-AORUS-ELITE-AX-V2:/home/marcin/ProteinPrompt$ docker pull registry.hzdr.de/proteinprompt/proteinprompt:latest&#xA;latest: Pulling from proteinprompt/proteinprompt&#xA;5843afab3874: Already exists &#xA;72c585c386a3: Already exists &#xA;87d798ddb41c: Already exists &#xA;316f048b0253: Pull complete &#xA;ae11eb6b6512: Pull complete &#xA;d3dc3767a31c: Pull complete &#xA;ea3dbaf26ce1: Pull complete &#xA;061c354f8f77: Pull complete &#xA;60eb5484e32a: Pull complete &#xA;0625cf0fe317: Pull complete &#xA;442818fdc663: Pull complete &#xA;203550cafc1f: Pull complete &#xA;Digest: sha256:01f0251f34313437e5fcc2dbb26d1ec1103d50aef6335b2261a2048be8ed0813&#xA;Status: Downloaded newer image for registry.hzdr.de/proteinprompt/proteinprompt:latest&#xA;registry.hzdr.de/proteinprompt/proteinprompt:latest&#xA;&#xA;&#xA;roboczy@marcin-B550-AORUS-ELITE-AX-V2:/home/marcin/ProteinPrompt$ docker images&#xA;REPOSITORY                                     TAG           IMAGE ID       CREATED         SIZE&#xA;alphafold                                      latest        c41a4b5e9d48   13 days ago     10GB&#xA;registry.hzdr.de/proteinprompt/proteinprompt   latest        cd99d22f3cb2   18 months ago   1.93GB&#xA;nvidia/cuda                                    11.0.3-base   4f3e95122fab   23 months ago   122MB&#xA;&#xA;&#xA;roboczy@marcin-B550-AORUS-ELITE-AX-V2:/home/marcin/ProteinPrompt&lt;span class=&quot;math-container&quot;&gt;$ docker run  -v $&lt;/span&gt;(pwd)/data:/data proteinprompt search -f data/input.fa -o data/output.csv&#xA;Unable to find image 'proteinprompt:latest' locally&#xA;docker: Error response from daemon: pull access denied for proteinprompt, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.&#xA;See 'docker run --help'.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Do you know what is wrong?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Final approach,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker pull registry.hzdr.de/proteinprompt/proteinprompt:latest&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1224" LastEditorUserId="3848" LastEditDate="2023-06-03T17:08:10.597" LastActivityDate="2023-06-03T17:08:10.597" Title="ProteinPromt installation" Tags="|software-installation|protein-protein-interaction|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21097" PostTypeId="2" ParentId="21096" CreationDate="2023-06-01T11:27:26.750" Score="1" Body="&lt;p&gt;The docker file is here &lt;a href=&quot;https://codebase.helmholtz.cloud/proteinprompt/ProteinPrompt/-/blob/main/Dockerfile&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://codebase.helmholtz.cloud/proteinprompt/ProteinPrompt/-/blob/main/Dockerfile&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You need to download and install a copy of Docker (below).&lt;/p&gt;&#xA;&lt;p&gt;These are the instructions how to build the container locally from the remote Docker file. Copy and paste this code into a file called &lt;code&gt;Dockerfile&lt;/code&gt;. Here I placed it in a &lt;code&gt;tmp&lt;/code&gt; directory, then simply (see &lt;strong&gt;note&lt;/strong&gt;)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; docker build -t tmp/dockerfile .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Just to note the authors may have specific instructions for the build command.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;How to install Docker&lt;/strong&gt; Before this command is run you need to install docker here &lt;a href=&quot;https://www.docker.com/products/docker-desktop/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.docker.com/products/docker-desktop/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;After installation you need a special command to run Docker in sudo-less mode is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sudo usermod -aG docker&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The first command I gave will then build the Dockerfile. There quite a lot of commands BTW to run/execute a Dockerfile, but you can also ask more questions.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;em&gt;Why am I recommending this way&lt;/em&gt;?&lt;/p&gt;&#xA;&lt;p&gt;Simple, because normally downloading docker is done via &lt;a href=&quot;https://docker.com&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docker.com&lt;/a&gt; and you would issue a &lt;code&gt;pull&lt;/code&gt; request. For some weird reason the authors haven't done this and I don't know what the outcome of an external download is. If it is built via the Dockerfile then Docker will recognise the Docker container. Once the container is built locally you can upload it to your own Docker site and delete it locally and access it any time via a &lt;code&gt;pull&lt;/code&gt; request or a &lt;code&gt;run&lt;/code&gt; request (will automate a &lt;code&gt;pull&lt;/code&gt; request).&lt;/p&gt;&#xA;&lt;p&gt;Docker is very cool BTW, worth learning.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Comments&lt;/strong&gt;, following bug report&lt;/p&gt;&#xA;&lt;p&gt;I don't think you've got git installed (&lt;code&gt;apt-get install -y git&lt;/code&gt;). Once that is installed I would would try again. &lt;strong&gt;BUT FIRST&lt;/strong&gt; I would delete the current build docker rmi dockerfile. If docker rmi ... fails (I don't think it will) then remove all containers before deleting the image with &lt;code&gt;docker rm $(docker ps -qa)&lt;/code&gt;. The use &lt;code&gt;rmi&lt;/code&gt; and then install git &lt;code&gt;apt-get install -y git&lt;/code&gt; and do the docker build command.&lt;/p&gt;&#xA;&lt;p&gt;If this doesn't work (same error) ..... you need to alter the Dockerfile and build it again from the new file .. Its a bug by the authors simply add the line &lt;code&gt;RUN apt-get update &amp;amp;&amp;amp; apt-get install -y git&lt;/code&gt; ... This is because there's a chance that git needs to be on the inside of the container not the outside.&lt;/p&gt;&#xA;&lt;p&gt;If all the above fails, I would download the entire image/container from the website the authors have loaded it. I don't know how to pull from a non-docker website, I've never done it. Note I'll place these comments in an updated post. I am honestly skeptical about the authors ability to write/develop Docker.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-06-03T17:05:33.260" LastActivityDate="2023-06-03T17:05:33.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21098" PostTypeId="2" ParentId="21093" CreationDate="2023-06-02T04:03:09.260" Score="1" Body="&lt;p&gt;You requested I make a response. Okay, I'm reluctant to do this firstly, all my  eukaryotes genomics is out of date except for immunology stuff and secondly, this is a hardcore eukaryote gene expression question. I am really not sure about the pig-human side of this experiment.&lt;/p&gt;&#xA;&lt;p&gt;What you have is an extremely good data set. What you appear to be doing is assessing the complexity of chromatin based gene expression, using very hardcore data. What we don't know is the variation e.g. in experiments leading to the RNA-seq. I would guess the RNA-seq is varying and being compared against atac-seq.&lt;/p&gt;&#xA;&lt;p&gt;I have not read the papers, read the abstracts you posted.&#xA;MEME and TOMTOM don't seem to be impressive in comparison to an absolute biological experiment RNA-seq vs. atac-seq. Predictions are never perfect (the very, very best predictions are 80%) and you have an absolute data set - why even be concerned about predictions based analysis?&lt;/p&gt;&#xA;&lt;p&gt;I do know DEG and the approach you are using is very conservative. It's good to visualise this on a volcano plot. Personally I just use a probabilistic threshold and then assess the log2 for each gene group. You are using both -/+2.5 log2 and &amp;lt;1E-6 . Personally I would be concerned with the stringency and also look at results with e.g. just &amp;lt;1E-6 or &amp;lt;1E-6 and -/+ 0.5log2. Keep in mind I don't do eukaryotes.&lt;/p&gt;&#xA;&lt;p&gt;The threshold criteria will be the number 1 issue that affects your results and that is the most important information I can give. So do your approach and also try a more relaxed threshold. Personally I wouldn't relax the p&amp;lt;1E-6, but I would relax the log2. The reasons for this are a bit complicated, but they are statistically sound.&lt;/p&gt;&#xA;&lt;p&gt;Your intersect approach seems fine, viz.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;intersect(object1, object2) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;@haci's method is cool, the above does the job too. The question is the genes vs thresholds and the experimental set ups and then the comparison between experimental setups, i.e. which genes are switching under chromatin control.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;What I am not clear about is the experimental design and how you are performing DSeq2,&lt;/em&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I assume you are using an internal control for RNAseq and a comparable one for atac-seq. I assume you are not comparing atac-seq verse RNAseq in the DEG.&lt;/li&gt;&#xA;&lt;li&gt;The alternative is atac-seq vs. RNAseq DEG and the comparison is pig versus human. I honestly dunno.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Anway the caveat I would add is an &lt;code&gt;intersect&lt;/code&gt; SHOULD NOT ignore whether it is +log2 or -log2 ... you need to do four intersects&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;+ve verse +ve&lt;/li&gt;&#xA;&lt;li&gt;-v verse -ve&lt;/li&gt;&#xA;&lt;li&gt;+ve verse -ve&lt;/li&gt;&#xA;&lt;li&gt;-ve verse +ve&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Again its four separate intersects.&lt;/p&gt;&#xA;&lt;p&gt;Good luck looks a really interesting experiment and analytics look simple but powerful in context.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-06-02T04:15:56.773" LastActivityDate="2023-06-02T04:15:56.773" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21100" PostTypeId="1" CreationDate="2023-06-02T13:44:45.040" Score="2" ViewCount="39" Body="&lt;p&gt;I've docked a large inorganic compound (receptor) with an organic molecule that isn't a protein (ligand). I wanted to see the interaction between them in a 2D diagram, but BIOVIA Discovery Studio (DS) doesn't work well with pdb files converted from pdbqt. And since the receptor isn't a protein or a fragment of DNA/RNA, DS also won't show the diagrams (the same with PLIP and Proteins Plus). I've also tried LigPlot, LeView and PoseView, but none of them worked as well. Does anyone know another  software that draws 2D diagrams, and, if not, is there any way to do this in R???&lt;/p&gt;&#xA;" OwnerUserId="17555" LastEditorUserId="17555" LastEditDate="2023-06-02T13:51:39.890" LastActivityDate="2023-06-02T13:51:39.890" Title="2D diagram of intermolecular interactions in R" Tags="|r|docking|interactions|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21101" PostTypeId="1" AcceptedAnswerId="21110" CreationDate="2023-06-03T22:10:58.723" Score="0" ViewCount="60" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9565378/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;In other words, are NCBI37 and hg19 synonymous? I can't find much information for what NCBI37 is in a human (not mouse) genome.&lt;/p&gt;&#xA;&lt;p&gt;The VCF is from a lab sequencing human genomes, and they're no longer able to be reached out to. I can't provide a link to NCBI37 because I don't know what it is and I can't find much information on it.&lt;/p&gt;&#xA;&lt;p&gt;Relevant line from VCF:&#xA;&amp;quot;##reference=NCBI37&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Here is part of the VCF header:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.2&#xA;##source=apt-format-result:2.10.2&#xA;##reference=NCBI37&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&amp;quot;Genotype&amp;quot;&amp;gt;&#xA;##contig=&amp;lt;ID=1&amp;gt;&#xA;##contig=&amp;lt;ID=10&amp;gt;&#xA;##contig=&amp;lt;ID=11&amp;gt;&#xA;##contig=&amp;lt;ID=12&amp;gt;&#xA;##contig=&amp;lt;ID=19&amp;gt; [omitted some chromosomes for brevity]&#xA;##contig=&amp;lt;ID=2&amp;gt;&#xA;##contig=&amp;lt;ID=20&amp;gt;&#xA;##contig=&amp;lt;ID=MT&amp;gt;&#xA;##contig=&amp;lt;ID=X&amp;gt;&#xA;##contig=&amp;lt;ID=Y&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17430" LastEditorUserId="73" LastEditDate="2023-06-05T05:03:55.443" LastActivityDate="2023-06-05T05:11:19.950" Title="Is a human genome file with reference as NCBI37 the same as hg19?" Tags="|vcf|genome|reference-genome|hg19|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21102" PostTypeId="1" CreationDate="2023-06-04T03:50:48.830" Score="1" ViewCount="29" Body="&lt;p&gt;I'm referring to the text described &lt;a href=&quot;https://github.com/verilylifesciences/genomewarp&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;These tools [NCBI remap, CrossMap] operate only on the sites present in an input VCF, and return the representation of those sites in a new genome assembly. This does not capture all variation, however. Consider an individual who has sequence reads that indicate they match the GRCh37 reference genome assembly at position GRCh37.chr1:169,519,049 (i.e. the individual's genotype is T/T). Because the individual is homozygous reference at that site, there will be no variation present in their VCF file created on GRCh37. However, the analogous position on the updated GRCh38 reference genome assembly, position GRCh38.chr1:169,549,811, has the reference base C. Consequently, if the individual's read data were analyzed on GRCh38, they would be identified as homozygous for a C-&amp;gt;T SNP. Because this site is not present in the input GRCh37 VCF, it is never added when creating a GRCh38 VCF by these other tools.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How important is this? Do you think it is relevant to any types of analysis?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/VGsoO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/VGsoO.png&quot; alt=&quot;Performance of liftover methods&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17430" LastEditorUserId="17430" LastEditDate="2023-06-04T19:58:23.993" LastActivityDate="2023-06-04T20:33:48.230" Title="How important are the homozygous variants that get unnecessarily deleted using liftover?" Tags="|genome|assembly|liftover|ncbi|hg19|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21103" PostTypeId="1" AcceptedAnswerId="21130" CreationDate="2023-06-04T07:14:19.250" Score="1" ViewCount="109" Body="&lt;p&gt;My question is exactly the one in the title. I add a particular example for more clarity. If I want to study the sequence of protein &lt;a href=&quot;https://en.wikipedia.org/wiki/I%CE%BAB%CE%B1&quot; rel=&quot;nofollow noreferrer&quot;&gt;IkBα&lt;/a&gt;, do I find only one sequence or is there a database that contains many sequences from many human host species ?&lt;/p&gt;&#xA;&lt;p&gt;EDIT: The analysis that I am going to do is to use NLP models to apply it to protein sequences to extrapolate, for instance, protein function knowledge.&lt;/p&gt;&#xA;&lt;p&gt;Here is a reference of a research paper &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S2405471221002039&quot; rel=&quot;nofollow noreferrer&quot;&gt;Learning the protein language: Evolution, structure, and function&lt;/a&gt; that considers this application by taking into account the problem of low number of sequences data.&lt;/p&gt;&#xA;&lt;p&gt;So I think I should have at least 10 protein sequences.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="10594" LastEditorUserId="10594" LastEditDate="2023-06-04T15:44:18.310" LastActivityDate="2023-06-07T18:20:10.973" Title="How many protein sequences do we usually have for each species?" Tags="|proteins|sequence-analysis|public-databases|public-dataset|deep-learning|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21105" PostTypeId="1" AcceptedAnswerId="21113" CreationDate="2023-06-04T18:15:59.177" Score="1" ViewCount="79" Body="&lt;p&gt;I am trying to using &lt;a href=&quot;https://illumina.github.io/NirvanaDocumentation/introduction/getting-started&quot; rel=&quot;nofollow noreferrer&quot;&gt;Illumina's Nirvana&lt;/a&gt; however I can't seem to get it to work despite having the right Singularity path.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I pulled Nirvana down using&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;singularity pull docker://annotation/nirvana:3.14&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;I found the path to the .dll using the following command:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;singularity exec nirvana_3.14.sif /bin/bash -c &amp;quot;cd /opt/nirvana/ &amp;amp;&amp;amp; ls&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;Tried to run the downloader using:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;## Define path to singularity image:&#xA;sif_container=&amp;quot;/path/to/nirvana_3.14.sif&amp;quot;&#xA;&#xA;## Path in the singularity file:&#xA;sif_path=&amp;quot;/opt/nirvana/Downloader.dll&amp;quot;&#xA;&#xA;## Data to be placed in&#xA;Data=&amp;quot;path/to/Nirvana/GRCh38/&amp;quot;&#xA;&#xA;#_____________________________________________________________________________________________________________________________________&#xA;&#xA;singularity exec &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$sif_container&quot; dotnet &quot;$&lt;/span&gt;sif_path&amp;quot; --ga GRCh38 --out &amp;quot;$Data&amp;quot;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;yet it keeps saying &lt;code&gt;ERROR: The top-level output directory directory was not specified. Please use the --out parameter.&lt;/code&gt;. I have checked the paths and they are correct and I have specified the data path yet it does not seem to recognize it? I have even moved around the container and the path variables to no avail?&lt;/p&gt;&#xA;" OwnerUserId="17244" LastActivityDate="2023-06-09T13:41:50.980" Title="Singularity and Illumina's Nirvana" Tags="|illumina|singularity|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21106" PostTypeId="1" AcceptedAnswerId="21111" CreationDate="2023-06-04T18:41:18.873" Score="-3" ViewCount="66" Body="&lt;p&gt;I have antibody data the first column is heavy chain sequence and the second column is the light chain&lt;/p&gt;&#xA;&lt;p&gt;I want to create a third column by combining two columns like&lt;/p&gt;&#xA;&lt;p&gt;If &lt;code&gt;CARGRGIAVAGTTGFDYW&lt;/code&gt; is a heavy chain and &lt;code&gt;CQQYNSYSLTF&lt;/code&gt; a light chain, in third column I want &lt;code&gt;CARGRGIAVAGTTGFDYW_CQQYNSYSLTF&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;if a heavy or light chain is absent, I want something like &lt;code&gt;NA_CQQYNSYSLTF&lt;/code&gt; or &lt;code&gt;CARGRGIAVAGTTGFDYW_NA&lt;/code&gt; depends of absent part&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CDR3.light.aa   CDR3.heavy.aa&#xA;-              CAGDLSVGGVLAGGVPHLRHFDPW&#xA;-              CPETPRYCSGGFCYGEFDNW&#xA;&#xA;CSSYAGSNTFGLF   -&#xA;CSSYAGSNTGLF    -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So far I have done this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;a&lt;span class=&quot;math-container&quot;&gt;$a &amp;lt;- paste(a$&lt;/span&gt;CDR3.heavy.aa, a$CDR3.light.aa, sep= &amp;quot;_&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I noticed a lot of mess and manual works&lt;/p&gt;&#xA;&lt;p&gt;Any help please?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2023-06-05T09:50:16.133" LastActivityDate="2023-06-05T10:22:25.740" Title="Make a new column in R or bash" Tags="|r|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21107" PostTypeId="2" ParentId="21102" CreationDate="2023-06-04T20:33:48.230" Score="2" Body="&lt;p&gt;All this is saying is that the tool doesn't lift over variants which aren't present in the vcf. I can't imagine this being a big deal in any analysis I've done and it's exactly what I'd expect the tool to do. There's no rule that invariant sites (with respect to the reference) can't be in a vcf, so if you are concerned then just retain the variants in the vcf file at the calling stage&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2023-06-04T20:33:48.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21108" PostTypeId="1" CreationDate="2023-06-04T20:51:30.553" Score="0" ViewCount="259" Body="&lt;p&gt;I am trying to install Singularity on my MacOS but it doesn't seem to pick up &lt;code&gt;which singularity&lt;/code&gt; or &lt;code&gt;singularity --version&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Main Attempt:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Install using Homebrew (and install Home-brew):&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;/bin/bash -c &amp;quot;$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)&amp;quot;&#xA;brew install --cask --no-quarantine singularity&#xA;singularity --version&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;resulting in&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;==&amp;gt; Downloading https://bitbucket.org/router_gray/singularityviewer/downloads/Singularity_Alpha_1_8_9_8709_x86_64.dmg&#xA;Already downloaded: /Users/Username/Library/Caches/Homebrew/downloads/d3015dad9cc2d3ef25603a68114f3caf63cb0dd90e03f159405999262392baec--Singularity_Alpha_1_8_9_8709_x86_64.dmg&#xA;==&amp;gt; Uninstalling Cask singularity&#xA;==&amp;gt; Backing App 'SingularityAlpha.app' up to '/opt/homebrew/Caskroom/singularity/1.8.9.8709/SingularityAlpha.app'&#xA;==&amp;gt; Removing App '/Applications/SingularityAlpha.app'&#xA;==&amp;gt; Purging files for version 1.8.9.8709 of Cask singularity&#xA;==&amp;gt; Installing Cask singularity&#xA;==&amp;gt; Moving App 'SingularityAlpha.app' to '/Applications/SingularityAlpha.app'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then test using &lt;code&gt;singularity --version&lt;/code&gt; with the result&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;singularity not found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Attempt 2:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Maybe it wants singularity spelled with a capital Singularity instead of lowercase:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;which Singularity&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;resulting in:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Singularity not found&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Attempt 3:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Export the path to the bash_profile and the source it&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;export PATH=&amp;quot;/Applications/SingularityAlpha.app/Contents/MacOS:$PATH&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, this resulted in the same issue when &lt;code&gt;singularity --version&lt;/code&gt; was run. It could not recognize that the singularity was installed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Attempt 4:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Explore inside the Singularity app which is installed under '/Applications/SingularityAlpha.app'&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The path &lt;code&gt;/Applications/SingularityAlpha.app/Contents/MacOS&lt;/code&gt; lead to a file called SingularityAlpha.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Exporting this path in the bash profile and sourcing still lead to the same issue when trying to see if it even recoginzed the singularity using &lt;code&gt;singularity --version&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I have also tried reinstalling it and it still has the same issue. Is there any suggestion for how to install/use Singularity locally as many packages will have to be tested locally on my machine before running it on my HPC?&lt;/p&gt;&#xA;" OwnerUserId="17244" LastEditorUserId="17244" LastEditDate="2023-06-04T20:59:38.140" LastActivityDate="2023-06-09T22:47:22.957" Title="Singularity And MacOS" Tags="|singularity|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21110" PostTypeId="2" ParentId="21101" CreationDate="2023-06-05T05:11:19.950" Score="1" Body="&lt;p&gt;Most likely yes, they are the same.&lt;/p&gt;&#xA;&lt;p&gt;However, &amp;quot;NCBI37&amp;quot; is an odd/unusual notation, because it doesn't indicate the species (human / mouse). Assuming it's a human genome (based on you comparing it to &lt;em&gt;hg&lt;/em&gt;19 rather than &lt;em&gt;mm&lt;/em&gt;19, or something else), then the correct name for the NCBI reference should be GRCh37.&lt;/p&gt;&#xA;&lt;p&gt;As it's a different name, we can only make guesses as to what the people who generated the VCF meant. You'll need to talk directly to them in order to be sure.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-06-05T05:11:19.950" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21111" PostTypeId="2" ParentId="21106" CreationDate="2023-06-05T10:22:25.740" Score="2" Body="&lt;p&gt;Your approach was correct, you just have to transform your dataframe first to replace the character &amp;quot;-&amp;quot; by NA :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(stringr)&#xA;library(dplyr)&#xA;df=data.frame(CDR3.light.aa=c(&amp;quot;-&amp;quot;,&amp;quot;-&amp;quot;,&amp;quot;CSSYAGSNTFGLF&amp;quot;,'CSSYAGSNTGLF'),&#xA;              CDR3.heavy.aa=c(&amp;quot;CAGDLSVGGVLAGGVPHLRHFDPW&amp;quot;,&amp;quot;-&amp;quot;,&amp;quot;CPETPRYCSGGFCYGEFDNW&amp;quot;,&amp;quot;-&amp;quot;))&#xA;&#xA;df=df %&amp;gt;% &#xA;  mutate_all(funs(str_replace(., &amp;quot;-&amp;quot;, &amp;quot;NA&amp;quot;)))      &#xA;&#xA;&#xA;df&lt;span class=&quot;math-container&quot;&gt;$a &amp;lt;- paste(df$&lt;/span&gt;CDR3.heavy.aa, df$CDR3.light.aa, sep= &amp;quot;_&amp;quot;)&#xA;&#xA;&#xA; df&#xA;  CDR3.light.aa            CDR3.heavy.aa                                  a&#xA;1            NA CAGDLSVGGVLAGGVPHLRHFDPW        CAGDLSVGGVLAGGVPHLRHFDPW_NA&#xA;2            NA                       NA                              NA_NA&#xA;3 CSSYAGSNTFGLF     CPETPRYCSGGFCYGEFDNW CPETPRYCSGGFCYGEFDNW_CSSYAGSNTFGLF&#xA;4  CSSYAGSNTGLF                       NA                    NA_CSSYAGSNTGLF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14639" LastActivityDate="2023-06-05T10:22:25.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21112" PostTypeId="2" ParentId="21108" CreationDate="2023-06-05T15:18:52.413" Score="3" Body="&lt;p&gt;&lt;em&gt;&lt;strong&gt;Answer:&lt;/strong&gt;&lt;/em&gt; Currently there is not a good way for Singularity to be installed on MacOS. Perhaps, there will be in the future.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I've tried and failed to install Singularity on macOS - it needs VirtualBox  or some such virtualization software and a command line utility that makes virtual containers - ultimately there were so many problems that I just gave up. Docker is far easier to install on macOS and there is not a lot of difference IMO. You can use docker to build and test images and then pull them to a place where you have access to singularity and test singularity specific stuff there.&lt;/p&gt;&#xA;&lt;p&gt;I think Singularity is a lot easier to install on Linux. See here: &lt;a href=&quot;https://docs.sylabs.io/guides/3.0/user-guide/installation.html#mac&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.sylabs.io/guides/3.0/user-guide/installation.html#mac&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT-2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Singularity-Alpha is a Second Life video game related cask (&lt;a href=&quot;https://github.com/singularity-viewer/SingularityViewer&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/singularity-viewer/SingularityViewer&lt;/a&gt;), it is not the Singularity software. Don't waste your time on it, I just tried to test it and realized I'd already tested and discarded it as irrelevant.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;EDIT: To address your homebrew issue, note that you're installing a cask and it's moving an App to /Applications/. Just typing in commands that should work is not the same as proper testing. Try opening the SingularityAlpha application to see what that does. I'm not saying it'll work, just that you can test it better. Your attempts only show a lack of understanding in how macOS Applications work and how you can(not) access them from the command line or add locations to $PATH like you can on Linux.&lt;/p&gt;&#xA;" OwnerUserId="650" LastEditorUserId="17244" LastEditDate="2023-06-09T22:47:22.957" LastActivityDate="2023-06-09T22:47:22.957" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21113" PostTypeId="2" ParentId="21105" CreationDate="2023-06-05T15:25:15.183" Score="3" Body="&lt;p&gt;From your previous questions, it is apparent that your HPC has an odd singularity setup. You could ask you sysadmin about how to navigate it.&lt;/p&gt;&#xA;&lt;h2&gt;EDIT&lt;/h2&gt;&#xA;&lt;p&gt;After working with OP on chat (and based on previous experience with them), it looks like their sysadmin has &lt;code&gt;mount hostfs&lt;/code&gt; enabled in the &lt;code&gt;singularity.conf&lt;/code&gt; file, which messes with every container every time. I guess it's easier than letting people bind their directories of choice but anyway, the way to circumvent that is to use &lt;code&gt;--no-mount hostfs&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;@Indira, always run &lt;code&gt;singularity [exec|shell|run] --no-mount hostfs&lt;/code&gt; instead of plain &lt;code&gt;singularity [exec|shell|run]&lt;/code&gt;. All containers will then run well and if they error out, it's probably because either the container internally encountered an error or it was unable to access an external resource. What you're running into now is a mix that obscures the real problems and creates a no-win scenario for you.&lt;/p&gt;&#xA;" OwnerUserId="650" LastEditorUserId="650" LastEditDate="2023-06-09T13:41:50.980" LastActivityDate="2023-06-09T13:41:50.980" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21114" PostTypeId="1" CreationDate="2023-06-05T15:31:31.960" Score="0" ViewCount="16" Body="&lt;p&gt;For an analysis of my data, I have a transcriptome and a list of sequences obtained from the transcriptome. I would like to perform a functional enrichment analysis. I have annotated both sets of data using eggnog mapper. Currently, I want to perform a test between the two functional annotations, specifically COGs (Clusters of Orthologous Groups). I have tried using the R code &lt;a href=&quot;https://yulab-smu.top/biomedical-knowledge-mining-book/enrichment-overview.html#gsea-algorithm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://yulab-smu.top/biomedical-knowledge-mining-book/enrichment-overview.html#gsea-algorithm&lt;/a&gt;&#xA;with clusterProfiler, but it seems that it may not work. With which tools or code can I perform this test, please?&lt;/p&gt;&#xA;&lt;p&gt;exemple somme of my data :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qNATr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qNATr.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17119" LastActivityDate="2023-06-05T15:31:31.960" Title="overrepresentation test, between transcriptome and candidates sequences obtained from the transcriptome" Tags="|statistics|transcriptome|go-enrichment|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21115" PostTypeId="1" AcceptedAnswerId="21499" CreationDate="2023-06-05T19:48:40.967" Score="1" ViewCount="97" Body="&lt;p&gt;I am trying to use &lt;a href=&quot;https://lbgi.fr/AnnotSV/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AnnotSV&lt;/a&gt; to annotate some Manta outputs in the following code with the &lt;a href=&quot;https://hpo.jax.org/app/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HPO term&lt;/a&gt; changed:&lt;/p&gt;&#xA;&lt;p&gt;Where:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;input_dir is the input directory&lt;/li&gt;&#xA;&lt;li&gt;singularity_container is the path to the AnnotSV singularity container&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;# Loop through the files in the input directory&#xA;for file in &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$input_dir&quot;/*.vcf; do&#xA;    # Get the base name of the file without extension&#xA;    filename=$&lt;/span&gt;(basename &amp;quot;$file&amp;quot; .vcf)&#xA;&#xA;    # Create the output subdirectory within the input directory&#xA;    output_dir=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$input_dir/AnnotSV_Batch_Run&quot;&#xA;mkdir -p &quot;$&lt;/span&gt;output_dir&amp;quot;&#xA;&#xA;    # Set the output file path with &amp;quot;_AnnotSV&amp;quot; appended&#xA;    output_file=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$output_dir/$&lt;/span&gt;{filename}_AnnotSV_&amp;quot;&#xA;&#xA;    # Run AnnotSV via Singularity with the base name as output prefix&#xA;    singularity exec &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$singularity_container&quot; AnnotSV -SVinputFile &quot;$&lt;/span&gt;file&amp;quot; -genomeBuild GRCh38 --hpo &amp;quot;HP:0000001&amp;quot;  -outputFile &amp;quot;$output_file&amp;quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The HPO term I am using is in the jax.org site but when I run it - I get the most peculiar error &lt;code&gt; option not known.&lt;/code&gt; but the HPO term is specified in the &lt;a href=&quot;https://lbgi.fr/AnnotSV/Documentation/README.AnnotSV_latest.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;AnnotSV manual&lt;/a&gt; on page 49 (according to the number on the bottom). As for trying I tried with &lt;code&gt;--HPOterm&lt;/code&gt; and &lt;code&gt;hpoterm&lt;/code&gt; I tried with and without the &lt;code&gt;:&lt;/code&gt; and with and without quotes as well. I am not sure what the issue is but I cannot seem to find the solution. Has anyone perchance used this software before with HPO terms as it seems not to be accepting the input I give it?&lt;/p&gt;&#xA;" OwnerUserId="17244" LastActivityDate="2023-08-28T23:10:50.193" Title="Singularity, AnnotSV and HPO" Tags="|singularity|" AnswerCount="1" CommentCount="15" ContentLicense="CC BY-SA 4.0" />
  <row Id="21116" PostTypeId="1" CreationDate="2023-06-05T20:37:09.350" Score="2" ViewCount="129" Body="&lt;p&gt;I am very new to ATAC-seq pipeline from Encode and trying to run on a HPC but not sure the steps after reading their instruction. &lt;a href=&quot;https://github.com/ENCODE-DCC/atac-seq-pipeline&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/ENCODE-DCC/atac-seq-pipeline&lt;/a&gt;&#xA;Here is my json file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;{&#xA;    &amp;quot;atac.title&amp;quot; : “atac)”,&#xA;    &amp;quot;atac.description&amp;quot; : “Encode”,&#xA;&#xA;    &amp;quot;atac.pipeline_type&amp;quot; : &amp;quot;atac&amp;quot;,&#xA;    &amp;quot;atac.align_only&amp;quot; : false,&#xA;    &amp;quot;atac.true_rep_only&amp;quot; : false,&#xA;&#xA;    &amp;quot;atac.genome_tsv&amp;quot; : &amp;quot;https://storage.googleapis.com/encode-pipeline-genome-data/genome_tsv/v4/hg38.tsv&amp;quot;,&#xA;&#xA;    &amp;quot;atac.paired_end&amp;quot; : true,&#xA;&#xA;    &amp;quot;atac.fastqs_wt_rep1_R1&amp;quot; : [ &amp;quot;rep1_R1_L1.fastq.gz&amp;quot; ],&#xA;    &amp;quot;atac.fastqs_wt_rep1_R2&amp;quot; : [ &amp;quot;rep1_R2_L1.fastq.gz&amp;quot;],&#xA;    &amp;quot;atac.fastqs_wt_rep2_R1&amp;quot; : [ &amp;quot;rep2_R1_L1.fastq.gz&amp;quot; ],&#xA;    &amp;quot;atac.fastqs_wt_rep2_R2&amp;quot; : [ &amp;quot;rep2_R2_L1.fastq.gz&amp;quot; ],&#xA;    &amp;quot;atac.fastqs_di_rep1_R1&amp;quot; : [ &amp;quot;rep1_R1_L1.fastq.gz&amp;quot; ],&#xA;    &amp;quot;atac.fastqs_di_rep1_R2&amp;quot; : [ &amp;quot;rep1_R2_L1.fastq.gz&amp;quot;],&#xA;    &amp;quot;atac.fastqs_di_rep2_R1&amp;quot; : [ &amp;quot;rep2_R1_L1.fastq.gz&amp;quot; ],&#xA;    &amp;quot;atac.fastqs_di_rep2_R2&amp;quot; : [ &amp;quot;rep2_R2_L1.fastq.gz&amp;quot; ],&#xA;&#xA;    &amp;quot;atac.auto_detect_adapter&amp;quot; : true,&#xA;&#xA;&#xA;    &amp;quot;atac.multimapping&amp;quot; : 4&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also need input file so what it should be?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Input files&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;IMPORTANT: Our pipeline considers a replicate (rep) as a biological replicate. You can still define technical replicates for each bio replicate. Tech replicates will be merged together to make a single FASTQ for each bio replicate.&lt;/p&gt;&#xA;&lt;p&gt;IMPORTANT: Our pipeline supports up to 10 bio replicates.&lt;/p&gt;&#xA;&lt;p&gt;IMPORTANT: Our pipeline has cross-validation analyses (IDR/overlap) comparing every pair of all replicates. Number of tasks for such analyses will be like nC2. This number will be 45 for 10 bio replicates. It's recommended to keep number of replicates &amp;lt;= 4.&lt;/p&gt;&#xA;&lt;p&gt;Pipeline can start from any of the following data types (FASTQ, BAM, NODUP_BAM and TAG-ALIGN).&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Starting from FASTQs&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Technical replicates for each bio-rep will be MERGED in the very early stage of the pipeline. Each read end R1 and R2 have separate arrays atac.fastqs_repX_R1 and atac.fastqs_repX_R2. Do not define R2 array for single-ended replicates.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Example of 3 paired-ended biological replicates and 2 technical replicates for each bio rep. Two technical replicates BIOREPX_TECHREP1.R1.fq.gz and BIOREPX_TECHREP2.R1.fq.gz for each bio replicate will be merged.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;       {&#xA;           &amp;quot;atac.paired_end&amp;quot; : true,&#xA;           &amp;quot;atac.fastqs_rep1_R1&amp;quot; : [&amp;quot;BIOREP1_TECHREP1.R1.fq.gz&amp;quot;, &amp;quot;BIOREP1_TECHREP2.R1.fq.gz&amp;quot;],&#xA;           &amp;quot;atac.fastqs_rep1_R2&amp;quot; : [&amp;quot;BIOREP1_TECHREP1.R2.fq.gz&amp;quot;, &amp;quot;BIOREP1_TECHREP2.R2.fq.gz&amp;quot;],&#xA;           &amp;quot;atac.fastqs_rep2_R1&amp;quot; : [&amp;quot;BIOREP2_TECHREP1.R1.fq.gz&amp;quot;, &amp;quot;BIOREP2_TECHREP2.R1.fq.gz&amp;quot;],&#xA;           &amp;quot;atac.fastqs_rep2_R2&amp;quot; : [&amp;quot;BIOREP2_TECHREP1.R2.fq.gz&amp;quot;, &amp;quot;BIOREP2_TECHREP2.R2.fq.gz&amp;quot;],&#xA;           &amp;quot;atac.fastqs_rep3_R1&amp;quot; : [&amp;quot;BIOREP3_TECHREP1.R1.fq.gz&amp;quot;, &amp;quot;BIOREP3_TECHREP2.R1.fq.gz&amp;quot;],&#xA;           &amp;quot;atac.fastqs_rep3_R2&amp;quot; : [&amp;quot;BIOREP3_TECHREP1.R2.fq.gz&amp;quot;, &amp;quot;BIOREP3_TECHREP2.R2.fq.gz&amp;quot;]&#xA;       }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;How should I modify this command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;caper hpc submit atac.wdl -i &amp;quot;${INPUT_JSON}&amp;quot; --conda --leader-job-name ANY_GOOD_LEADER_JOB_NAME&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I appreciate your help !&lt;/p&gt;&#xA;" OwnerUserId="17585" LastEditorUserId="73" LastEditDate="2023-06-06T21:38:26.143" LastActivityDate="2023-06-06T21:38:26.143" Title="Help with running ATAC-seq using Encode pipeline" Tags="|atac-seq|encode|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21117" PostTypeId="1" AcceptedAnswerId="21120" CreationDate="2023-06-06T10:16:13.923" Score="1" ViewCount="43" Body="&lt;p&gt;I would like to read iteratively the same BAM file twice using &lt;code&gt;htslib&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My option is to use &lt;code&gt;hts_open&lt;/code&gt;, &lt;code&gt;sam_hdr_read&lt;/code&gt;, then &lt;code&gt;sam_read1&lt;/code&gt; as much as needed to read the file until EOF.&lt;/p&gt;&#xA;&lt;p&gt;Is there a possibility to &amp;quot;rewind&amp;quot;, and use &lt;code&gt;sam_read1&lt;/code&gt; again from the beginning of the file, without closing the file, destroying the object, and opening it again?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;My aim is group the reads by the barcode.  The first pass would allocate the specific sizes first (the number of reads by barcode), then store the reads.&lt;/p&gt;&#xA;&lt;p&gt;I know I could sort the reads first (using &lt;code&gt;samtools sort -t BX&lt;/code&gt;), but I would like to skip this step using pre-allocation.&lt;/p&gt;&#xA;" OwnerUserId="17606" LastEditorUserId="17606" LastEditDate="2023-06-06T14:47:52.300" LastActivityDate="2023-06-06T15:53:53.750" Title="Reading same BAM file twice with htslib" Tags="|htslib|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21118" PostTypeId="1" CreationDate="2023-06-06T14:54:46.103" Score="0" ViewCount="118" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9565421/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;How can I calculate the variant allelic fraction from the VEP file for multiple human samples?&lt;/p&gt;&#xA;&lt;p&gt;I used the ensembl-vep and it is human data.&lt;/p&gt;&#xA;" OwnerUserId="16772" LastEditorUserId="73" LastEditDate="2023-06-06T21:26:50.717" LastActivityDate="2023-06-06T21:28:03.273" Title="How to calculate the variant allelic fraction (VAF) from VEP output?" Tags="|python|biopython|vep|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21119" PostTypeId="2" ParentId="21118" CreationDate="2023-06-06T15:12:50.017" Score="2" Body="&lt;p&gt;If you just have a VEP output (i.e. VCF) without raw genotypes then you cannot calculate the allelic fraction.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="73" LastEditDate="2023-06-06T21:28:03.273" LastActivityDate="2023-06-06T21:28:03.273" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21120" PostTypeId="2" ParentId="21117" CreationDate="2023-06-06T15:53:53.750" Score="1" Body="&lt;p&gt;The best solution is to reopen the file. For a coordinate sorted file you have the option to jump back to the start of the first chromosome with index but that is more complex, less flexible and won't be faster.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2023-06-06T15:53:53.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21121" PostTypeId="1" AcceptedAnswerId="21124" CreationDate="2023-06-06T15:59:38.577" Score="3" ViewCount="114" Body="&lt;p&gt;Is there an out-of-the-box function in the &lt;code&gt;Bio.SeqIO&lt;/code&gt; module (in &lt;code&gt;biopython&lt;/code&gt;) that can print a protein sequence (in command line) in multiple lines along with the numbering of the residues (either left and right or in-between the lines)?&lt;/p&gt;&#xA;&lt;p&gt;In particular I have a &lt;code&gt;.fasta&lt;/code&gt; file which I can read into a &lt;code&gt;SeqRecord&lt;/code&gt; object like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;record = SeqIO.read(&amp;quot;protein.fasta&amp;quot;, &amp;quot;fasta&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to then print the &lt;code&gt;Seq&lt;/code&gt; class with the numbering, not just as a string as shown below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print(record.seq)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have already tried:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SeqIO.write(record, sys.stdout, 'gb')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But it is not working for me. The error was &lt;code&gt;ValueError: missing molecule_type in annotations&lt;/code&gt;. I googled it and it appears to be related to newer biopython versions (see &lt;a href=&quot;https://github.com/linsalrob/PhiSpy/issues/43&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/linsalrob/PhiSpy/issues/43&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://sequenceconversion.bugaco.com/converter/biology/sequences/fasta_to_genbank.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;This on-line tool&lt;/a&gt; seems to do what I want, but one needs to provide it with an alphabet. How do I do that with code? Does a &lt;code&gt;SeqRecord&lt;/code&gt; object have an alphabet?&lt;/p&gt;&#xA;" OwnerUserId="17607" LastEditorUserId="73" LastEditDate="2023-06-06T21:22:43.187" LastActivityDate="2023-06-06T21:54:46.320" Title="Residue numbering in biopython" Tags="|sequence|biopython|command-line|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21122" PostTypeId="1" CreationDate="2023-06-06T16:55:51.760" Score="-1" ViewCount="98" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/9565667/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have two bam files from single-cell RNA sequencing mapped to the reference genome using CellRanger, I can view them in IGV and I have a particular region where the pattern of reads mapped to the reference genome are different between the two bam files but when I try the following code to plot trackplots with coverage, the resulting plot is not consistent with what I see in IGV. I am not very familiar with this, aren't I supposed to normalize the data as well? please help me.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;txdb &amp;lt;-TxDb.Hsapiens.UCSC.hg38.knownGene&#xA;txTr &amp;lt;- GeneRegionTrack(txdb, chromosome = &amp;quot;chr1&amp;quot;, &#xA;                        start = 153561336,  end = 153561564)&#xA;dTrack &amp;lt;- DataTrack(range = bam.file, genome = &amp;quot;hg38&amp;quot;, type = &amp;quot;l&amp;quot;, &#xA;                     name = &amp;quot;Coverage&amp;quot;, window = -1, &#xA;                     chromosome = &amp;quot;chr1&amp;quot;)&#xA;&#xA;dTrack0 &amp;lt;- DataTrack(range = bam.file.0, genome = &amp;quot;hg38&amp;quot;, type = &amp;quot;l&amp;quot;, &#xA;                     name = &amp;quot;Coverage&amp;quot;, window = -1, &#xA;                     chromosome = &amp;quot;chr1&amp;quot;, col=&amp;quot;red&amp;quot;)&#xA;&#xA;plotTracks(c(dTrack,dTrack0, txTr), from = 153561357, to = 153561585)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16344" LastEditorUserId="73" LastEditDate="2023-07-09T20:23:46.120" LastActivityDate="2023-08-08T21:07:41.010" Title="Gviz Coverage Plots" Tags="|r|scrnaseq|sam|cellranger|gviz|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21123" PostTypeId="2" ParentId="21121" CreationDate="2023-06-06T18:25:30.947" Score="0" Body="&lt;p&gt;Given the criteria required the closest solution is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;seq_obj = ... # Biopython seq_obj&#xA;with open(&amp;quot;mysequence.gb&amp;quot;, &amp;quot;w&amp;quot;) as fout:&#xA;    SeqIO.write(seq_obj, fout, &amp;quot;gb&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If this fails I would substitute &lt;code&gt;gb&lt;/code&gt; for &lt;code&gt;fasta&lt;/code&gt; and try again. If that fails, i.e. no fasta, there's a problem with the sequence object. If this succeeds but the &lt;code&gt;gb&lt;/code&gt; fails, just supply a reproducible example. It will work.&lt;/p&gt;&#xA;&lt;p&gt;I've used &lt;code&gt;gb&lt;/code&gt; to successfully write APIs against NCBI Genbank and trap the meta-data. Genbank format is really cool.&lt;/p&gt;&#xA;&lt;p&gt;Oh wait, sorry the answer is  ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seq_obj for records:&#xA;    print(seq_obj.format(&amp;quot;gb&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;From the comments &amp;quot;molecule type&amp;quot;...&lt;/p&gt;&#xA;&lt;p&gt;Which are you using&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for record in SeqIO.parse(filename, &amp;quot;fasta&amp;quot;):&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for record in SeqIO.read(filename, &amp;quot;fasta&amp;quot;):&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which ever is used I'd to the other method.&lt;/p&gt;&#xA;&lt;p&gt;Just to comment on @RamRS solution, thats cool I didn't know.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for record in SeqIO.parse(filename, &amp;quot;fasta&amp;quot;):&#xA;    record.annotations = {&amp;quot;molecule_type&amp;quot; : &amp;quot;protein&amp;quot; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What is happening is the instance is being manually set. However, Biopython should detected the data type automatically. My suspicion is that &lt;code&gt;read&lt;/code&gt; or &lt;code&gt;parse&lt;/code&gt; one comprises and automated data type setter.&lt;/p&gt;&#xA;&lt;p&gt;In any case, RamRS's solution is useful and will work. In past coding I would have read the object via Genbank.&lt;/p&gt;&#xA;&lt;p&gt;The old &lt;code&gt;alphabet&lt;/code&gt; method is deprecated and shouldn't be attempted.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;For future reference RamRS's code is described &lt;a href=&quot;https://github.com/biopython/biopython/issues/3156&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-06-06T21:54:46.320" LastActivityDate="2023-06-06T21:54:46.320" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21124" PostTypeId="2" ParentId="21121" CreationDate="2023-06-06T19:42:11.633" Score="3" Body="&lt;p&gt;@M__ is right - there's a &lt;code&gt;molecule_type&lt;/code&gt; error when you try to write and this needs to be fixed. Here's my end to end approach:&lt;/p&gt;&#xA;&lt;p&gt;The input file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat xyz.fa&#xA;&amp;gt;NP_000537.3 cellular tumor antigen p53 isoform a [Homo sapiens]&#xA;MEEPQSDPSVEPPLSQETFSDLWKLLPENNVLSPLPSQAMDDLMLSPDDIEQWFTEDPGPDEAPRMPEAA&#xA;PPVAPAPAAPTPAAPAPAPSWPLSSSVPSQKTYQGSYGFRLGFLHSGTAKSVTCTYSPALNKMFCQLAKT&#xA;CPVQLWVDSTPPPGTRVRAMAIYKQSQHMTEVVRRCPHHERCSDSDGLAPPQHLIRVEGNLRVEYLDDRN&#xA;TFRHSVVVPYEPPEVGSDCTTIHYNYMCNSSCMGGMNRRPILTIITLEDSSGNLLGRNSFEVRVCACPGR&#xA;DRRTEEENLRKKGEPHHELPPGSTKRALPNNTSSSPQPKKKPLDGEYFTLQIRGRERFEMFRELNEALEL&#xA;KDAQAGKEPGGSRAHSSHLKSKKGQSTSRHKKLMFKTEGPDSD&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My (bad) code:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;with open(&amp;quot;xyz.fa&amp;quot;) as handle:&#xA;    for record in SeqIO.parse(handle, &amp;quot;fasta&amp;quot;):&#xA;        record.annotations = { &amp;quot;molecule_type&amp;quot; : &amp;quot;protein&amp;quot; }&#xA;SeqIO.write(record, &amp;quot;xyz.gb&amp;quot;, &amp;quot;gb&amp;quot;) # moved two indentations to the left&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat xyz.gb&#xA;&#xA;LOCUS       NP_000537.3              393 aa                     UNK 01-JAN-1980&#xA;DEFINITION  NP_000537.3 cellular tumor antigen p53 isoform a [Homo sapiens].&#xA;ACCESSION   NP_000537&#xA;VERSION     NP_000537.3&#xA;KEYWORDS    .&#xA;SOURCE      .&#xA;  ORGANISM  .&#xA;            .&#xA;FEATURES             Location/Qualifiers&#xA;ORIGIN&#xA;        1 meepqsdpsv epplsqetfs dlwkllpenn vlsplpsqam ddlmlspddi eqwftedpgp&#xA;       61 deaprmpeaa ppvapapaap tpaapapaps wplsssvpsq ktyqgsygfr lgflhsgtak&#xA;      121 svtctyspal nkmfcqlakt cpvqlwvdst pppgtrvram aiykqsqhmt evvrrcphhe&#xA;      181 rcsdsdglap pqhlirvegn lrveylddrn tfrhsvvvpy eppevgsdct tihynymcns&#xA;      241 scmggmnrrp iltiitleds sgnllgrnsf evrvcacpgr drrteeenlr kkgephhelp&#xA;      301 pgstkralpn ntssspqpkk kpldgeyftl qirgrerfem frelnealel kdaqagkepg&#xA;      361 gsrahsshlk skkgqstsrh kklmfktegp dsd&#xA;//&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I used the code from &lt;a href=&quot;https://github.com/biopython/biopython/issues/3156&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/biopython/biopython/issues/3156&lt;/a&gt; to fix the &lt;code&gt;ValueError: missing molecule_type in annotations&lt;/code&gt; that I faced when I'd skipped the &lt;code&gt;record.annotations = {&amp;quot;molecule_type&amp;quot; : &amp;quot;protein&amp;quot; }&lt;/code&gt; line.&lt;/p&gt;&#xA;&lt;p&gt;There is definitely a better way to write the python code instead of using a loop or at least writing to a static file from within a loop, but that's not central to the problem here so I'm choosing to ignore it. I welcome any improvements to the code.&lt;/p&gt;&#xA;" OwnerUserId="650" LastEditorUserId="3848" LastEditDate="2023-06-06T19:50:53.963" LastActivityDate="2023-06-06T19:50:53.963" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21125" PostTypeId="2" ParentId="21122" CreationDate="2023-06-06T23:05:31.257" Score="0" Body="&lt;p&gt;I would recommend reading the following study by &lt;a href=&quot;https://translational-medicine.biomedcentral.com/articles/10.1186/s12967-021-02936-w&quot; rel=&quot;nofollow noreferrer&quot;&gt;Zhao &lt;em&gt;et al&lt;/em&gt; (2021)&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The traditional simplest measure is CPM otherwise known as TPM. This is counts/transcripts per million. FPKM (fragments per kilobase of transcript per million fragments mapped) is used for metagenomics. I don't know its application in your context ... I don't know your context.&lt;/p&gt;&#xA;&lt;p&gt;The discussions for the pros- and cons- of FPKM and TPM/CPM are discussed therein.&lt;/p&gt;&#xA;&lt;p&gt;The authors in the study favour normalisation - &lt;em&gt;in their context&lt;/em&gt; - not in general. The authors use the coefficient of variation, and intraclass correlation coefficient. I would see this as a form of standardisation. It's worth keeping an open mind because its a pivotal step in the calculation but all too easily overlooked area of investigation.&lt;/p&gt;&#xA;&lt;p&gt;From RamRS's comments that may have changed already. In general statistics standardisation is a pivotal step, usually followed by the removal of outliers.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-06-06T23:37:31.940" LastActivityDate="2023-06-06T23:37:31.940" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21126" PostTypeId="2" ParentId="21103" CreationDate="2023-06-07T05:08:35.613" Score="2" Body="&lt;p&gt;I'm going to start by assuming that you don't actually want 10 protein sequences from the same species, even though you write this in your comment on M__'s (deleted) answer. This will not be a very informative sample of variation in a given protein, except for extreme examples like antibodies.&lt;/p&gt;&#xA;&lt;p&gt;To get a good sample of variation you almost certainly want to look across species. The paper that you link to does this by using the UniRef database, they then cluster proteins at the 90% similarity threshold. That is well beyond the variation that you'd expect within a species. See their section &amp;quot;Training datasets&amp;quot;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We train our masked language models on a large corpus of protein sequences, UniRef90 (Suzek et al., 2007), retrieved in July 2018. This dataset contains 76,215,872 protein sequences filtered to 90% sequence identity.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You could use an approach like that in the paper to get similar-ish clusters of proteins whose variation you can then analyze. You can use this, but I am somewhat suspicious of that, without reading the paper in too much depth, because we know rather a lot about orthology/paralogy and how it affects function in these cases. Especially in these very elaborate not very introspectable models I worry about what they will do with inadequately curated data.&lt;/p&gt;&#xA;&lt;p&gt;So instead I'd probably want to first identify the &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1832-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;orthology groups&lt;/a&gt; (gene families) of interest. These are generally manually curated groupings of proteins.&lt;/p&gt;&#xA;&lt;p&gt;Once you have the orthology groups that you care about, you can use a database such as &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwjAz-_Tr7D_AhVMAzQIHRULDHYQFnoECA0QAQ&amp;amp;url=https%3A%2F%2Fwww.genome.jp%2Fkegg%2Fko.html&amp;amp;usg=AOvVaw2_1ohJtdx9xARgBfI0LjJv&quot; rel=&quot;nofollow noreferrer&quot;&gt;KEGG&lt;/a&gt; or &lt;a href=&quot;https://omabrowser.org/oma/home/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OMA&lt;/a&gt; to identify the specific orthologs (genes) that belong to the group. Then you will have your dataset.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-06-07T05:08:35.613" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21127" PostTypeId="2" ParentId="20968" CreationDate="2023-06-07T09:46:52.217" Score="2" Body="&lt;p&gt;If you are interested in ApoB gene variants you can:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;go to &lt;a href=&quot;https://www.ensembl.org/Homo_sapiens/Gene/Variation_Gene/Table?db=core;g=ENSG00000084674;r=2:21001429-21044073&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ensembl.org/Homo_sapiens/Gene/Variation_Gene/Table?db=core;g=ENSG00000084674;r=2:21001429-21044073&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;sort the table by Clin.Sig. column&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;select the name of the variant, i.e. rs5742904&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;search for that variant in your 23andMe file&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;compare the variants from the ENSEMBL with what you got in your result&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ensembl.org/Homo_sapiens/Variation/Explore?db=core;g=ENSG00000084674;r=2:21001429-21044073;v=rs5742904;vdb=variation;vf=184237596&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ensembl.org/Homo_sapiens/Variation/Explore?db=core;g=ENSG00000084674;r=2:21001429-21044073;v=rs5742904;vdb=variation;vf=184237596&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But I assume all of this is moot since 23andMe should give you some summary of the results and not just the raw file, right?&lt;/p&gt;&#xA;" OwnerUserId="6314" LastActivityDate="2023-06-07T09:46:52.217" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21129" PostTypeId="2" ParentId="19200" CreationDate="2023-06-07T13:34:33.653" Score="3" Body="&lt;p&gt;Came across this looking for a solution myself last night but now I have created my own I will share it.&lt;/p&gt;&#xA;&lt;p&gt;Rather then using &lt;code&gt;Bio&lt;/code&gt;python (which is a Python package I use a lot), it is more computationally efficient to parse the &lt;code&gt;.fasta&lt;/code&gt; files from the protein data bank (&lt;a href=&quot;https://www.rcsb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;RCSB&lt;/a&gt; in this case) which as far as I can see, contain this information. For example, this is &lt;a href=&quot;https://www.rcsb.org/structure/2P16&quot; rel=&quot;nofollow noreferrer&quot;&gt;2P16&lt;/a&gt;'s &lt;code&gt;.fasta&lt;/code&gt; file (with 2 unique chains):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;2P16_1|Chain A|Coagulation factor X (EC 3.4.21.6) (Stuart factor) (Stuart-Prower factor)|Homo sapiens (9606)&#xA;IVGGQECKDGECPWQALLINEENEGFCGGTILSEFYILTAAHCLYQAKRFKVRVGDRNTEQEEGGEAVHEVEVVIKHNRFTKETYDFDIAVLRLKTPITFRMNVAPACLPERDWAESTLMTQKTGIVSGFGRTHEKGRQSTRLKMLEVPYVDRNSCKLSSSFIITQNMFCAGYDTKQEDACQGDSGGPHVTRFKDTYFVTGIVSWGEGCARKGKYGIYTKVTAFLKWIDRSMKT&#xA;&amp;gt;2P16_2|Chain B[auth L]|Coagulation factor X (EC 3.4.21.6) (Stuart factor) (Stuart-Prower factor)|Homo sapiens (9606)&#xA;KLCSLDNGDCDQFCHEEQNSVVCSCARGYTLADNGKACIPTGPYPCGKQTLE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and &lt;a href=&quot;https://www.rcsb.org/structure/6won&quot; rel=&quot;nofollow noreferrer&quot;&gt;6WON&lt;/a&gt;'s (with 8 of the same chains):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;6WON_1|Chains A, B, C, D, E, F, G, H|YohF|Salmonella typhimurium (strain LT2 / SGSC1412 / ATCC 700720) (99287)&#xA;QGMTKVAIVTASDSGIGKACALLLAQNGFDIGITWHSDERGAQETAKKAAQFGVRAETIHLDLSQLPEGAQAIEHLIQRLGRVDVLVNNAGAMTKSAFIDMPFTQWRQIFTVDVDGAFLCAQIAARHMIKQGEGGRIINITSVHEHTPLPQASAYTAAKHALGGLTKSMALELIEHHILVNAVAPGAIATPMNDMDDSDIEPGSEPSIPIARPGSTHEIASLVAWLCSEGASYTTGQSLIVDGGFMLANPQFNAK&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So far this always seems to be the case and parsing the &lt;code&gt;.fasta&lt;/code&gt; is a lot easier then using BLAST or sequence similarity such as you are or were.&lt;/p&gt;&#xA;&lt;p&gt;For the implementation, whereas the &lt;code&gt;.pdb&lt;/code&gt;, &lt;code&gt;.cif&lt;/code&gt; files etc use the same download URL (&lt;code&gt;https://files.rcsb.org/download/&lt;/code&gt;), the &lt;code&gt;.fasta&lt;/code&gt; files use this link: &lt;code&gt;https://www.rcsb.org/fasta/entry/&lt;/code&gt;. Parsing the downloaded &lt;code&gt;.fasta&lt;/code&gt; files and only reading the lines that begin with &lt;code&gt;&amp;gt;&lt;/code&gt;, stripping out unnecessary chains (in my specific case I just want the first unique chains in the PDB so &lt;code&gt;A, B&lt;/code&gt; being the same and &lt;code&gt;C, D&lt;/code&gt; being the same would return me a list with [&lt;code&gt;A, C&lt;/code&gt;], you can modify this script to your own ends) and running the Python code gives this output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;First unique chains in 1d2z (2 unique chains): ['B', 'A']&#xA;First unique chains in 2p16 (2 unique chains): ['A', 'B']&#xA;First unique chains in 6won (1 unique chain):  ['A']&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There was an issue with some entries having &lt;code&gt;Chain&lt;/code&gt; or &lt;code&gt;Chains&lt;/code&gt; so both had to be considered. Additionally, there are commas that can be removed from all but the last chain ID but for my purposes I do not need to strip it, you can put something in &lt;code&gt;_fasta_helper()&lt;/code&gt; to this end before &lt;code&gt;return&lt;/code&gt;. This is the Python code and hopefully it helps someone out in the future:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import re&#xA;from typing import Union&#xA;import urllib.request&#xA;from urllib.error import (HTTPError,&#xA;                          URLError,&#xA;                          ContentTooShortError as ContentError)&#xA;&#xA;def rcsb_downloader(pdb: str, ext: str = None, down_path: str = None) -&amp;gt; str:&#xA;    &amp;quot;&amp;quot;&amp;quot;Downloads the PDB from the protein databank RCSB.&#xA;&#xA;    Parameters&#xA;    ----------&#xA;    pdb : str&#xA;        Protein databank accession 4 letter code i.e. '6won'&#xA;    ext : str, default = None&#xA;        Extension of the file to download i.e. '.fasta', '.cif' etc. Note: if&#xA;        None argument then ext = '.pdb'&#xA;&#xA;    Returns&#xA;    -------&#xA;    str | bool&#xA;        Returns str path if download successful, False if failure&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;    if ext is None:&#xA;        ext = '.pdb'&#xA;    if down_path is None:&#xA;        download_path = ('/your/path/to/dir/' + pdb + ext)&#xA;    else:&#xA;        download_path = down_path + pdb + ext&#xA;&#xA;    rcsb_path = 'https://files.rcsb.org/download/' + pdb + ext&#xA;&#xA;    if ext == '.fasta': # fasta has a different web search then others&#xA;        rcsb_path = 'https://www.rcsb.org/fasta/entry/' + pdb&#xA;&#xA;    try:&#xA;        urllib.request.urlretrieve(rcsb_path, download_path)&#xA;        return download_path&#xA;    # Try again in case there was a temporary issue with the first attempt&#xA;    except (ContentError, HTTPError, URLError):&#xA;        try:&#xA;            urllib.request.urlretrieve(rcsb_path, download_path)&#xA;            return download_path&#xA;        except (ContentError, HTTPError, URLError):&#xA;            print(f&amp;quot;PDB failed to download {pdb}&amp;quot;)&#xA;            return False&#xA;&#xA;    def _fasta_helper(self, line: str, single_unique_chain: bool = True) -&amp;gt; Union[str, list]:&#xA;        &amp;quot;&amp;quot;&amp;quot;Separate the first chain ID from the .fasta comment line.&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;        # [1] some of the chains had [auth C] etc after so this is removed&#xA;        # [2] was originally a list comprehension that filtered based on the word&#xA;        # chain, however some of the fasta entries had &amp;quot;Chain&amp;quot; or &amp;quot;Chains&amp;quot; in their&#xA;        # description which broke this. The first arrow is removed, split at |, the&#xA;        # second entry (1st in Python) entry contains the chain information and the&#xA;        # first entry removed (which so far has always been chain, anticipate their&#xA;        # being edge cases with older PDB entries). This appears to hold up.&#xA;&#xA;        clean_chain = [&#xA;            x.replace(&amp;quot;,&amp;quot;, &amp;quot;&amp;quot;)&#xA;            for x in list(&#xA;                re.sub(r&amp;quot;\[.*?\]&amp;quot;, &amp;quot;&amp;quot;, line).lstrip(&amp;quot;&amp;gt;&amp;quot;).split(&amp;quot;|&amp;quot;)[1].split(&amp;quot; &amp;quot;)[1:]  # [1]  # [2]&#xA;            )&#xA;        ]&#xA;&#xA;        if not single_unique_chain:&#xA;            return clean_chain&#xA;        return clean_chain[0]&#xA;&#xA;def fasta_parse(path_: str) -&amp;gt; list:&#xA;    &amp;quot;&amp;quot;&amp;quot;Parse .fasta files and analyse lines beginning with '&amp;gt;'.&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;    with open(path_, 'r', encoding='utf-8') as fasta:&#xA;        total_unique_chains = 0  # Use as a flag to test if more than one unique chain&#xA;        unique_chains       = []&#xA;        for line in fasta:&#xA;            if line[0] == '&amp;gt;' and total_unique_chains == 0:&#xA;                total_unique_chains += 1&#xA;                unique_chains.append(_fasta_helper(line))&#xA;                continue&#xA;            elif line[0] == '&amp;gt;' and total_unique_chains != 0:&#xA;                unique_chains.append(_fasta_helper(line))&#xA;    return unique_chains&#xA;&#xA;pdb_diff_chain  = '2p16' # Chain ...&#xA;pdb_diff_chains = '1d2z' # Chains ...&#xA;pdb_same_chains = '6won' # Chains ...&#xA;&#xA;download_p_diff  = rcsb_downloader(pdb_diff_chain, '.fasta')&#xA;download_p_diffs = rcsb_downloader(pdb_diff_chains, '.fasta')&#xA;download_p_same  = rcsb_downloader(pdb_same_chains, '.fasta')&#xA;&#xA;unique_s  = fasta_parse(download_p_same)&#xA;unique_d  = fasta_parse(download_p_diffs)&#xA;unique_ds = fasta_parse(download_p_diff)&#xA;&#xA;print(f'First unique chains in {pdb_diff_chains} (2 unique chains): {unique_ds}\n'&#xA;      f'First unique chains in {pdb_diff_chain} (2 unique chains): {unique_d}\n'&#xA;      f'First unique chains in {pdb_same_chains} (1 unique chain):  {unique_s}')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16234" LastEditorUserId="16234" LastEditDate="2023-07-15T15:47:23.660" LastActivityDate="2023-07-15T15:47:23.660" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21130" PostTypeId="2" ParentId="21103" CreationDate="2023-06-07T17:33:15.610" Score="1" Body="&lt;p&gt;Just to add to Maximilian Press's answer. I happen to know about human immunology and I think Maximilian Press has a good idea.&lt;/p&gt;&#xA;&lt;p&gt;Genetic studies and immunological studies &lt;em&gt;per se&lt;/em&gt; in cellular immunology  tend to be restricted within humans. There are limited exceptions to this namely chimpanzees and humanised mammals, most notably humanised mice. In cellular immunology these mammals retain immunologically distinct behaviour but are sufficiently similar to be useful models.&lt;/p&gt;&#xA;&lt;p&gt;The protein in question is a B-cell repressor, thats more complicated ...&lt;/p&gt;&#xA;&lt;p&gt;However, using trans-species homology of the protein of interest for&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;chimpanzees&lt;/li&gt;&#xA;&lt;li&gt;humanised mice&lt;/li&gt;&#xA;&lt;li&gt;regular mice&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;.. represents a good diversity of proteins to use what appears to be a deep-learning function prediction if this was a traditional cellular immunological study. Any immunologist looking at the biological rationale would recognise the model systems deployed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Method: Blast&lt;/strong&gt; Generating diversity for given protein is easy, except for humans. Just use online Blast and restrict the search to the species of interest and 20000 hits (?I forget the upper limit) is the maximum and select for more diverse genes. This would work for chimps and (possibly) mice. Thus blast a human gene and restrict the output to the species of interest. Human gene diversity - this much bigger than an online blast could be targeted in accordance with human genetic diversity. The cellular immune system for humans does not conform to background diversity &lt;em&gt;in general&lt;/em&gt;. Some geographic human populations do have a HLA predominance but there are no absolute association.&lt;/p&gt;&#xA;&lt;p&gt;The final option is local blasting the human database for that gene, that is doable and objectively used to derive representative diversity - for this purpose is complicated and would advise against it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-06-07T18:20:10.973" LastActivityDate="2023-06-07T18:20:10.973" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21131" PostTypeId="1" CreationDate="2023-06-07T19:00:17.287" Score="1" ViewCount="62" Body="&lt;p&gt;So for context, I have a set of TPM values for multiple genes for different samples, and I need to calculate the differential expression for RNA-seq values. I've been using this website as a guide: &lt;a href=&quot;https://raw.githubusercontent.com/ucdavis-bioinformatics-training/2019_March_UCSF_mRNAseq_Workshop/master/differential_expression/DE_Analysis.Rmd&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://raw.githubusercontent.com/ucdavis-bioinformatics-training/2019_March_UCSF_mRNAseq_Workshop/master/differential_expression/DE_Analysis.Rmd&lt;/a&gt;. The overall aim of this calculation is so that I can use PANDA to map TM motifs/combine gene expression data to create networks representing interactions between transcription factors and genes, and GSEA to analyze genes ranked by fold change or differential expression p-value.&lt;/p&gt;&#xA;&lt;p&gt;So what I need to do is match one male to one female of the same age group (which I already did prior; the number of males outnumbered the number of females so I filtered the number of males so that there would be an equal number, and of the same age distribution). But it seems that I need to derive two factors and create a new variable &amp;quot;group&amp;quot; that combines factor1 and factor2. In this case, it would be the gender I would use as the group factors, right?&lt;/p&gt;&#xA;" OwnerUserId="17613" LastEditorUserId="17613" LastEditDate="2023-06-08T06:36:27.533" LastActivityDate="2023-06-08T06:36:27.533" Title="How do I calculate differential expression for RNA-seq values with the &quot;limma&quot; package and the &quot;ebayes&quot; function?" Tags="|differential-expression|tpm|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21132" PostTypeId="1" CreationDate="2023-06-07T20:10:27.280" Score="2" ViewCount="125" Body="&lt;p&gt;Entry level Bioinformatician here.&#xA;I am trying to map my single cell data with standard annotation file or assign my Seurat cluster to cell types. I have used the singleR and celldex package to map it but my problem is the single cell only consist of endothelial cells. I haven't found paper that only consist of rats endothelial cells. The single cell data is for rats.  I am stuck. What is the best thing that could be done here?&lt;/p&gt;&#xA;" OwnerUserId="17614" LastEditorUserId="17614" LastEditDate="2023-06-07T20:13:43.113" LastActivityDate="2023-06-07T20:52:44.807" Title="Mapping single cell data with annotation" Tags="|scrnaseq|clustering|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21133" PostTypeId="2" ParentId="21132" CreationDate="2023-06-07T20:52:44.807" Score="3" Body="&lt;p&gt;It seems like you can annotate all your cells as endothelial cells, problem solved ;) But seriously, not clear what your expected cell types are (subtypes of endothelial cells perhaps) or the tissue type. If you can't find a suitable reference dataset for the reference mapping approach, then you can always resort to the 'old fashioned' way of unsupervised clustering (e.g leiden) plus checking marker gene expression to annotate the clusters.&lt;/p&gt;&#xA;&lt;p&gt;An alternative approach, could be finding a suitable human/mouse reference and  then converting the gene ids to rat (ideally via 1:1 ortholog mapping) but this could get messy, so I suggest the former simpler way, at least as a starting point&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2023-06-07T20:52:44.807" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21134" PostTypeId="1" CreationDate="2023-06-08T04:13:51.360" Score="2" ViewCount="123" Body="&lt;p&gt;I want to create a phylogenetic tree using a 3.8MB FASTA file with ambiguous code. How can I perform multiple sequence alignment and the construction of phylogenetic tree with these ambiguous codes?&lt;/p&gt;&#xA;&lt;p&gt;Could you please suggest some programs or R packages to complete this task?&lt;/p&gt;&#xA;&lt;p&gt;How does MEGA11 software handle ambiguous codes? Will the alignment consider them to be mismatches?&lt;/p&gt;&#xA;" OwnerUserId="17615" LastEditorUserId="17615" LastEditDate="2023-06-09T02:23:56.147" LastActivityDate="2023-06-09T02:23:56.147" Title="Phylogenetic tree from a large FASTA file" Tags="|r|phylogenetics|fasta|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21135" PostTypeId="1" CreationDate="2023-06-08T06:35:47.310" Score="-1" ViewCount="210" Body="&lt;p&gt;So for context, I have a set of TPM values for multiple genes for different samples, and I need to calculate the differential expression for RNA-seq values. I've been using this website as a guide: &lt;a href=&quot;https://raw.githubusercontent.com/ucdavis-bioinformatics-training/2019_March_UCSF_mRNAseq_Workshop/master/differential_expression/DE_Analysis.Rmd&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://raw.githubusercontent.com/ucdavis-bioinformatics-training/2019_March_UCSF_mRNAseq_Workshop/master/differential_expression/DE_Analysis.Rmd&lt;/a&gt;. The overall aim of this calculation is so that I can use PANDA to map TM motifs/combine gene expression data to create networks representing interactions between transcription factors and genes, and GSEA to analyze genes ranked by fold change or differential expression p-value.&lt;/p&gt;&#xA;&lt;p&gt;Note that I am using log2(TPM+1) values. How do I calculate what the filtering cutoff should be? Note that I already excluded all genes that had any values of 0 for any of the samples, and I will use the Benjamini-Hochberg method to adjust p-values for multiple testing afterwards.&lt;/p&gt;&#xA;" OwnerUserId="17613" LastEditorUserId="73" LastEditDate="2023-06-08T21:11:30.863" LastActivityDate="2023-06-12T16:06:01.410" Title="How do I calculate DEG cutoffs?" Tags="|differential-expression|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21137" PostTypeId="1" CreationDate="2023-06-08T09:00:01.743" Score="2" ViewCount="321" Body="&lt;p&gt;I want to zip the files generated by fasterq-dump simultaneously as they are generated. The code that I tried is this.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat SRR_Acc_List.txt | while read i &#xA;do  fasterq-dump ${i} --split-spot -v --skip-technical  -e 15| gzip -k  ${i}&amp;gt; ${i}.gz &#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17618" LastEditorUserId="73" LastEditDate="2023-06-08T21:08:50.630" LastActivityDate="2023-06-12T19:33:32.853" Title="ZIP the fasterq-dump files in loop" Tags="|ncbi|sra|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21138" PostTypeId="2" ParentId="21137" CreationDate="2023-06-08T13:26:53.297" Score="1" Body="&lt;p&gt;Haven't tried, but I believe you may add &lt;code&gt;-Z&lt;/code&gt; to fasterq-dump and remove &lt;code&gt;-k ${i}&lt;/code&gt; from gzip. In addition, I would recommend the following:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;cat SRR_Acc_List.txt | xargs -i echo fasterq-dump {} --split-spot -Z -v --skip-technical -e 15 \| gzip &amp;gt; {}.gz | sh&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It is simpler and more flexible. For example, replacing &lt;code&gt;sh&lt;/code&gt; with &lt;code&gt;parallel&lt;/code&gt; would allow you to download in parallel.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2023-06-08T13:26:53.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21139" PostTypeId="1" CreationDate="2023-06-08T15:43:28.317" Score="0" ViewCount="283" Body="&lt;p&gt;I'm working with cellranger count to annotate some publicly available fastq files (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra/SRX7888073%5Baccn%5D&quot; rel=&quot;nofollow noreferrer&quot;&gt;SRX7888073&lt;/a&gt;). For each sample, 4 runs are available, which I assume represent technical replicates.&lt;/p&gt;&#xA;&lt;p&gt;How do you recommend to handle them in cellranger? Are they 4 different samples (i.e. 4 separate runs), or 1 sample with 4 lanes?&lt;/p&gt;&#xA;&lt;p&gt;Thank you !&lt;/p&gt;&#xA;" OwnerUserId="17619" LastActivityDate="2023-06-09T17:06:02.280" Title="How to handle technical replicates in cellranger count?" Tags="|scrnaseq|cellranger|" AnswerCount="3" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21140" PostTypeId="2" ParentId="21139" CreationDate="2023-06-08T17:55:53.600" Score="0" Body="&lt;p&gt;After some digging, it seems like each of the 4 runs has I1, R1 and R2 files - they're replicates.&lt;/p&gt;&#xA;&lt;p&gt;You can either&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;run them separately and then perform &lt;code&gt;cellranger aggr&lt;/code&gt;; or&lt;/li&gt;&#xA;&lt;li&gt;link to all 12 files using soft-links that follow a proper naming convention (the files need to have the same prefix, lane numbers need to be different per replicate), ensure all softlinks are in the same directory, then use the directory with &lt;code&gt;cellranger count&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I'd recommend the former approach as it will probably save a little compute time, is parallelize-able and will be easier to debug/drill down into.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-06-08T17:55:53.600" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21141" PostTypeId="2" ParentId="21139" CreationDate="2023-06-08T21:04:14.083" Score="2" Body="&lt;p&gt;If they are truly technical replicates in the most specific sense (e.g. from the same extraction and library prep, just split up for sequencing), then the counts for each identified cell can be combined (i.e. added together). One way to do this is to concatenate the files prior to running any data processing scripts.&lt;/p&gt;&#xA;&lt;p&gt;If there is anything different about the runs, then they should be treated as non-technical replicates and properly integrated:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://satijalab.org/seurat/articles/integration_rpca.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://satijalab.org/seurat/articles/integration_rpca.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;We had a situation with Rhapsody single cell data where a library was split into three on three different cell barcode plates, and was showing a small amount of library preparation bias that influenced clustering. Carrying out an integration of the libraries helped to reduce this bias.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-06-08T21:04:14.083" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21142" PostTypeId="1" AcceptedAnswerId="21144" CreationDate="2023-06-09T00:24:01.293" Score="0" ViewCount="29" Body="&lt;p&gt;I am using the following &lt;a href=&quot;https://github.com/anthonyozerov/variant-filtering/tree/main&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github repository&lt;/a&gt; that I cloned. One apparent issue is that it is using an old/defunct link for the phenotype_to_genes.txt that is no longer active and after several tries I found what seems to be the &lt;a href=&quot;http://purl.obolibrary.org/obo/hp/hpoa/phenotype_to_genes.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;correct link&lt;/a&gt;. However, I cannot seem to get it to work as it seems to be throwing up an error that makes no sense to me when I try and run it via a bash script:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Variable I defined in advance:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;pedigree_file&lt;/code&gt; has the path to the pedigree file with 4 columns (Family ID, individual_ID,Status [Child,Father,Mother,Sibling], Sex [Male or Female] and Phenotype [Affected or Unaffected]&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;Mendelian&lt;/code&gt; a path to a file created via Mendelian filtering&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;output_dir&lt;/code&gt; path to the output directory&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;phenotype_path&lt;/code&gt; a path to a file with 2 columns group/family and then HPO&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;VC_Python_directory&lt;/code&gt; path to the variant filtering directory&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;Map_File&lt;/code&gt; path to the phenotype_to_genes.txt&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;family_names&lt;/code&gt; a list of family/group names&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;The bash script:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;cd &amp;quot;$VC_Python_directory&amp;quot;&#xA;&#xA;IFS=',' read -ra family_array &amp;lt;&amp;lt;&amp;lt; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$family_names&quot;&#xA;for family_name in &quot;$&lt;/span&gt;{family_array[@]}&amp;quot;; do&#xA;&#xA;    # Concatenate the file path&#xA;    file_path=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$output_dir/$&lt;/span&gt;family_name.txt&amp;quot;&#xA;    &#xA;    # Echo the family name and file path&#xA;    echo &amp;quot;Family Name: &lt;span class=&quot;math-container&quot;&gt;${family_name}&quot;&#xA;echo &quot;File Path: $&lt;/span&gt;{file_path}&amp;quot;&#xA;    &#xA;    # Run the Python script with the correct arguments&#xA;    python main.py -p &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$pedigree_file&quot; --data &quot;$&lt;/span&gt;Mendelian&amp;quot; --output &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$file_path&quot; --family &quot;$&lt;/span&gt;family_name&amp;quot; -ph &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$phenotype_path&quot; --mapfile &quot;$&lt;/span&gt;Map_File&amp;quot;&#xA;done&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;The error:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Family Name: Sample_1&#xA;File Path: /path/to/output_directory/Sample_1.txt&#xA;Path/to/variant-filtering/utils.py:77: DtypeWarning: Columns (2) have mixed types. Specify dtype option on import or set low_memory=False.&#xA;  phen_to_genes = pd.read_csv(mapfile, sep = '\t', header = None, comment = '#')&#xA;Getting relevant genes for family phenotypes...&#xA;Traceback (most recent call last):&#xA;  File “path/to/variant-filtering/main.py&amp;quot;, line 26, in &amp;lt;module&amp;gt;&#xA;    load_phen(families, args.phenfile, args.mapfile)&#xA;  File “path/to/variant-filtering/utils.py&amp;quot;, line 80, in load_phen&#xA;    phen_to_genes.columns = [&amp;quot;HPO-id&amp;quot;, &amp;quot;HPO label&amp;quot;, &amp;quot;gene-id&amp;quot;,&#xA;  File “path/to/python3.10/site-packages/pandas/core/generic.py&amp;quot;, line 6002, in __setattr__&#xA;    return object.__setattr__(self, name, value)&#xA;  File &amp;quot;pandas/_libs/properties.pyx&amp;quot;, line 69, in pandas._libs.properties.AxisProperty.__set__&#xA;  File “/path/to/generic.py&amp;quot;, line 730, in _set_axis&#xA;    self._mgr.set_axis(axis, labels)&#xA;  File “/path/to/managers.py&amp;quot;, line 225, in set_axis&#xA;    self._validate_set_axis(axis, new_labels)&#xA;  File “/path/to/base.py&amp;quot;, line 70, in _validate_set_axis&#xA;    raise ValueError(&#xA;ValueError: Length mismatch: Expected axis has 5 elements, new values have 7 elements&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have checked over the pedigree file and map file and phenotype path files but I am really unsure of how to go about this pipeline as I have never seen this kind of error before. I am not sure I can ask this here but does anyone perchance have an inkling as to what is causing the rather peculiar error?&lt;/p&gt;&#xA;" OwnerUserId="17244" LastEditorUserId="17244" LastEditDate="2023-06-09T21:00:39.553" LastActivityDate="2023-06-09T21:00:39.553" Title="Variant Filtering Using AnthonyOzerov/variant-filtering Github Repository And Python Errors" Tags="|python|bash|github|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21143" PostTypeId="1" CreationDate="2023-06-09T08:23:08.800" Score="3" ViewCount="335" Body="&lt;p&gt;I wish to convert Aperio AFI microscope .svs files from a pathology lab microscope into 3DHistech .mrxs format. Lab ppl have some pipeline X and they asked me to look into the matter.&lt;/p&gt;&#xA;&lt;p&gt;I can read the SVS files via &lt;a href=&quot;https://github.com/openslide/openslide-python&quot; rel=&quot;nofollow noreferrer&quot;&gt;openslide&lt;/a&gt; or &lt;a href=&quot;https://www.openmicroscopy.org/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;openmicroscopy&lt;/a&gt;. Conversion to 'regular' pictures (tiff, for instance). I figure out that since &lt;a href=&quot;https://github.com/openslide/openslide-python&quot; rel=&quot;nofollow noreferrer&quot;&gt;openslide&lt;/a&gt; can read mrxs, writing this format should be feasible too, but can't find any (free) software to do this. I don't care about any specialties, just need the end result to be readable by Histech readers.&lt;/p&gt;&#xA;&lt;p&gt;Can anyone help with the conversion? Lacking full software, I could try to code writer if resulting file description is available.&lt;/p&gt;&#xA;" OwnerUserId="17627" LastEditorUserId="3694" LastEditDate="2023-06-18T17:14:11.303" LastActivityDate="2024-01-19T00:46:18.323" Title="Converting convert .svs files into .mrxs format" Tags="|file-formats|format-conversion|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21144" PostTypeId="2" ParentId="21142" CreationDate="2023-06-09T11:14:28.047" Score="1" Body="&lt;p&gt;Apparently the format of the &lt;code&gt;phenotype_to_genes.txt&lt;/code&gt; File changed, while at the time when this pipeline was published the file consisted of 7 columns, it only has 5 now. That's what the error message tells you. It's best to file an issue at the github repository to get in touch with the authors. 1) so that other potential users are also aware of this issue and 2) the authors might be willing to fix this issue/help you get an updated version of the pipeline or an older version of the phenotype-file.&lt;/p&gt;&#xA;" OwnerUserId="17629" LastActivityDate="2023-06-09T11:14:28.047" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21145" PostTypeId="2" ParentId="21139" CreationDate="2023-06-09T17:00:02.367" Score="3" Body="&lt;p&gt;It depends what you mean by &amp;quot;technical replicates&amp;quot;.  If it is the exact same library prep spread out over two lanes, or two flowcells, they must be combined at the fastq stage.  Cellranger will naturally do this with fastqs that keep their original Illumina-assigned names, but just have the files split by lane.&lt;/p&gt;&#xA;&lt;p&gt;If one file has a read from a gene with a particular cell barcode and UMI, and another read in a separate file but &lt;strong&gt;the same library prep&lt;/strong&gt; has a read from the same gene, with the same barcode and same UMI, those two must &lt;strong&gt;not&lt;/strong&gt; be both counted. So if you combine the counts, that's wrong.&lt;/p&gt;&#xA;&lt;p&gt;A NextSeq naturally has 4 lanes, so that's almost certainly what you are looking at.  The read names would confirm that, unless they've been altered.  Cat them all together, or alter their names so that they match the original Illumina specs with lanes 1-4.  You might have to do that anyway, cellranger used to be very picky about that, and might still be.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastEditorUserId="2388" LastEditDate="2023-06-09T17:06:02.280" LastActivityDate="2023-06-09T17:06:02.280" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21146" PostTypeId="1" CreationDate="2023-06-09T20:34:42.213" Score="1" ViewCount="128" Body="&lt;p&gt;I am new to pacbio sequencing data. I just did some alignment of pacbio HiFI data using pbmm2. I have the bam now, I would like to collect some metrics on the alignment. I used to work on the illumina data, I used picard CollectAlignmentSummaryMetrics, CollectGcBiasMetrics, CollectInsertSizeMetrics to collect a series of alignment metrics.&#xA;I was wondering if there is similar tools for pacbio data. I searched and did not find an answer.&lt;/p&gt;&#xA;&lt;p&gt;We are mainly interested in aligned rate, sequence length, coverage, accuracy, and GC content. So I write some scripts for it. These scripts are inside the nextflow, the input is the bam file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools flagstat &lt;span class=&quot;math-container&quot;&gt;$bam \&#xA; | head -8 \&#xA; | tr &quot; &quot; &quot;,&quot;  \&#xA; | tr &quot;+&quot; &quot; &quot; \&#xA; | awk -F',' '{ print $1, $3}' \&#xA; | paste - - - - - - - -  \&#xA; | awk '{print $1, $3, $5, $7, $9, $11, $2, $4, $6, $8, $10, $&lt;/span&gt;12 }' &amp;gt;  align_prp&#xA; &#xA; samtools depth &lt;span class=&quot;math-container&quot;&gt;$bam \&#xA; |  awk '{sum+=$3; sumsq+=$3*$&lt;/span&gt;3} END { print sum/NR, sqrt(sumsq/NR - (sum/NR)**2)}' &amp;gt; coverage&#xA; &#xA; samtools view &lt;span class=&quot;math-container&quot;&gt;$bam \&#xA; | awk '{ $35=length($10); sum+=$35; sumsq+=$35*$&lt;/span&gt;35} END { print sum/NR, sqrt(sumsq/NR - (sum/NR)**2)}' &amp;gt; length&#xA; &#xA;&#xA; &#xA; samtools view  &lt;span class=&quot;math-container&quot;&gt;$bam \&#xA; | cut -f 10 \&#xA; | fold -w 1 \&#xA; | awk '($&lt;/span&gt;1==&amp;quot;G&amp;quot; || $1==&amp;quot;C&amp;quot;) {N++;} END {print (N/(1.0*NR)*100);}' &amp;gt; GC_content&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I am still interested to know if there are some package available there.&lt;/p&gt;&#xA;" OwnerUserId="14191" LastEditorUserId="14191" LastEditDate="2023-06-10T16:59:22.513" LastActivityDate="2023-06-10T16:59:22.513" Title="Pacbio HIFI pbmm2 alignment metrics" Tags="|sequence-alignment|pacbio|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21147" PostTypeId="1" AcceptedAnswerId="21150" CreationDate="2023-06-10T02:01:28.213" Score="1" ViewCount="39" Body="&lt;p&gt;I'm trying to conduct &lt;code&gt;lmFit&lt;/code&gt; on my &lt;code&gt;fread::table&lt;/code&gt; matrix:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;fit &amp;lt;- lmFit(matchedgeneTPM)[,-c(1,2)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It's giving me the error that expression object should be numeric and that there are 2 non-numeric columns (because the first two columns are the labels of the genes and gene ID's I'm working with). I thought the last portion of my code would allow it to skip over the first two columns but apparently it's not the right way to do it. Can someone show me how to format the code correctly, please?&lt;/p&gt;&#xA;" OwnerUserId="17613" LastEditorUserId="650" LastEditDate="2023-06-12T13:22:58.770" LastActivityDate="2023-06-12T13:22:58.770" Title="How do I fix this limma line?" Tags="|r|rna|limma|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21150" PostTypeId="2" ParentId="21147" CreationDate="2023-06-10T14:09:01.413" Score="2" Body="&lt;p&gt;My guess is you need to index &lt;code&gt;matchedgeneTPM&lt;/code&gt; before you run &lt;code&gt;lmFit()&lt;/code&gt;, not after:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fit &amp;lt;- lmFit(matchedgeneTPM[,-c(1:2)])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2023-06-10T14:09:01.413" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21151" PostTypeId="1" CreationDate="2023-06-10T15:29:48.037" Score="1" ViewCount="48" Body="&lt;pre&gt;&lt;code&gt;&amp;gt;003197.2:c1179912-1178219_reverse_complement&#xA;AGGCTTTTAAAAGCCTCCTGGGTTTTTAGTGAAGCTGAGTGATAGAAGCTCTGTATTTGCATAGCGTTTT&#xA;GTCGGGCGCTTTAGCCGGCGCTTTGCCCTGGCCTGAGAGAATCTGCATTCCGTTGTATAAGGTTTTTTGT&#xA;CCTTCTGATGCTGTAGATAATTCCCCCATGTCGCGCCGGGTTCTCGCAGGACAATAATTTCCGGGCGAGC&#xA;AGAACGGTTGCCGCGACGGTAAGAAGATCGCGCTGTAAGTTATAGAGGTTATGCAGCGAGTGGTTAGACG&#xA;CCGGTCCGCTTTAACTTTGGCTAACTCCATTTGGTTTGCCATTGACGTTAGAACCGGGTCTTGTTTACCC&#xA;GTGCTGCAATAAGTTCGATAAGATTTTTCTTCAATGCTTTTGCCGCGGCTTCTTCTTGTTTTGTTGCTGG&#xA;GCATCTCTAAACGCTACTGCCGCAAAGCGATGAGCTTCTTTTGCAGGTAAGCCATCCTGCTGCTGAGTGC&#xA;GTGATGCCCGTTATGCGTGAGTGTATTTTTTATGGTTTGCCAGGGCTGGTTATTAAGCTGCTTGACCTGA&#xA;CATAAGCACTGGGAAAGATATCTTTTGCGCCGATTTTCATCTCTGCTGCAGGGAGCTGCGTGTTGGTATA&#xA;ACCGTGGACATCCACAAATTATTGGCGTGATGAATATTCTTGGTATCCCAACTGCATACGCCCTTTCCCT&#xA;ATAGGGGGAAAGCACGCCATGACGTATCCCGCAAAAAAGCGTTTTATCTTTACCGTCCTCATGCACACTC&#xA;GTGCCGCAGTTAATACTTCTTTGGCTTTGTTTTCAGCGCCGACGTGACGCAGAAGCGGATCTTTTTCATG&#xA;ACGGATACCAGTTTCAGGCTTACCGCCTCGCCCGCTAAGGCTTTGTTAAGCAACTCAGGTTTACTAAAAA&#xA;CCATGCGCGCATTTGGTCCTCGACCATCGTTCCCTCTTTGCCGAAAATATTCGACGCGGTGAGTAACCCG&#xA;CCGTCTGTAGATCGCCATCTTTATTGCGGATTTTTAAATGAATCATTTTTCCCGGCTGGGTCAACGATTG&#xA;CCAAAGCCGAGCTTGAGCGCCAGCTCATTAACACCCACATTAAATGCGGCGACGTCCGGTTTTATTTTTA&#xA;TTCAGGGCGTAAATCATTGCCTAATAACTGATGTAGCGCCTCGGCATTATAGCTATCCGATGCCTTAAGG&#xA;ATGTATTGACGACCTCATAATTATCCGGGTATTGCGCCAGCCATTCGCCAACCCAGCCACCTGGTCTGGC&#xA;AGTTTATAGGGTTCGCCGCCATCTTTATGATGTTGGTTATTTTTCCATATATCCTTAATCTGGCGCGCTA&#xA;TTTGCCGCTTTTACAATTCCAGGCGGGTACCGCGTCAATTTCATGGGCTAACATGGCAAGGCGTTGTGCG&#xA;TTAACATATGGGTCTGATGTAAGGAAATGATCTCTCGCTTGATTTCTGAATCCATCATCCCTGTACGATC&#xA;TTACCGCTATTCAGTAATACTTTTTGGAAAATTTTCTGTCCACCGCTATCCGGAAGACTACCAGGCGCAC&#xA;CACCTCTGGCGATAAATTTTTCATTACTTTGTTTCCCGCCCCGCCCGTATTTTGTTTCTGAATCTCCAGG&#xA;ATGAAGAAATGCCTTTTACTGACTGCCAAATATTTTCATCCCCAACTCGTTTTTGATAGGAAAGATTGAG&#xA;TCAAGATGTGATTA&#xA;&amp;gt;003197.2:c3026750-3024686_reverse_complement&#xA;TTTGATCTCGGTCTGCATACCTGGCATTATGACGGGGGGCTGAGTCCTTACACTTGTAACCATTATTAAT&#xA;CCCCTGACAGCGTCGCTGTGGCACTTTCTCTGACTGCGGAAAGATTCGCCGCAAGATTCGTGGCCTGCGT&#xA;TTAAACAATGCATCCAGACTCGCCTGTTTGATCAGCGCGGGAAAATCTTCCAGTTGCGGGCCTTTAATTT&#xA;CGCTTTCTTACCGGCGACATTATTTGAATTGGTAAAAACTTCTTTTAACGCCTCAGCGTCTTTCCCGCAT&#xA;GCGTCTTCGCCTCAGGAGAATCACTGGTGGCGTTGAGCGCTGAACGAAAGAGCCCGGCAAACTCCATTAT&#xA;AAAGCTGACTTTTCTTTCAGGCCATCTTTTATGATTTGCGCGGTATACTCTGCCCCCACCTTCATCAGTA&#xA;ATCTAACAGCTGCTTTTCCAGGTTTTCCAGCTTCGCTTCCGCTTTCTTTGTTTCTGGCAGCCATGGCCCA&#xA;GCATCACCTCCTGCATAACAAGATTCGTCGATAATTTACTTAATTCACCGCCAGTGTTATTTTTGATAAT&#xA;GTAAGATTGCTGCGGGTTAACGGATCTAACGTACAGCCAAAGTTATGTTCAATGCAGCTGGCAATATAGG&#xA;TTGCGTTAATTTTTGGTGGCACATATCAAGCGCCTCAACAGCCTTCGCCGCCGCTTTGTCAACAAGGTGC&#xA;GTAGCAGCAGGGGTATCAACGTTTGCATTTCAAGGTGCCGGGCTTCCCGTCCTACGCTGGTACCCTGCTC&#xA;GGTCCAGCCGCAGGCTCCGGAATTTTAGGGTCAGGCAGTTTATCTGCAGGAATTTGTGCAAAAACATTAC&#xA;GTTACTGCTATCAATATTAATATTGATATTAATACCCGTAGGTTCTGCTTTTTTCCCACCATCAGGTACT&#xA;CGATATGGCGCTGGCTATTATCGACATGGCTTCGGCTATTGTTAATATGCTTACTGTTATCCACGCTATG&#xA;CGTTGGCTGTTATCAATAAATGTTCGGCTATTATCAATCGTCTTCCGGCTATTATCATGGTTGCTATTAT&#xA;AGTCGAATCCACACGCGAATGACTATGACTTACGTTGCTGCTGTTAGTCGAATGGTGTGACTCGCCATTG&#xA;CAATTTTACCCGCGATGCCATGGTCAATTGCCCCTGTGCTGGCGCTGTGTGCCGTCTCCGTTTGATGCGT&#xA;ACCACCTTTCCATCTTTACTTTCGGATGAAGCGTTGGTCACAGCCTCTGCCGTCGCATGAGCCGTGACGT&#xA;GCTCTTGCTGGTTACGCCATCGACTTCATCAAAAGAGGTTGTTTCACCCGTAGTGCCCTTTTCTGACGTG&#xA;CTGCCGTCTGGCTTTGCTGTTTATTATCATCAACGCCGTGCACCGTGGCCTGTACCGGCTTGCCGATAAT&#xA;TGTAAGGTATCGGTTTTAACATTCTCGACGCCAGCCAGTTGACTGGCAAGCGGCTTCACATTCACAATCT&#xA;CTTATCTGTCTGACTATTATCATTATCGGTCGTACCGGCTTTATTACCTGTAATTACCGTCGTGTCTGAT&#xA;TATTCGTAACGCTCGGTACTGTTGAGAGGAAACTATTTTGCTTAAACTTCAAACCAGAAAATGGGCCAGT&#xA;GTATCTGGCTCCAGGGCTTTACGTATCACACCCAAAAACGTCTCACGGGCGTCGAAATGCATTGAATGCA&#xA;TATCGTGTCATTGGGTAATATTTCCGCACGCAAGCCGTCAAACGCTCTGCGTACAGGGAACGGTGTGGAG&#xA;TCTCCATTTTCGCTTTCAATTCTGGGTGCTTGTCAATATCGCTACATTGCGCTTTCAACGCTGCACTCTT&#xA;CTGGCAAACTGCAGTGCAATCTCAGCCAGTTTTTCTTTTTGTGGATGATGAGTAATGACCTCTTTGAGAG&#xA;GATAAGGCCGTCCAGCACATTACTTAACACATAGTCAGTTTCCCCTTTTAGCCTGGTCAGCCCCGCCTCT&#xA;TAACCCGATCCACGCCAGGTTTATTCAGGTATGACTCGTAAGCCGGGCCAGCTCGCCAGCTACCGTCTCC&#xA;TTAACGCTGCATGTGCAAGCCATCAACGGTAGTAA&#xA;&amp;gt;003197.2:c3010816-3009905_reverse_complement&#xA;TATTACGAGTCGTAGGACTTAGCATACTTTTCCCTCCACATGATAGCTCCTGCACCGAAAATATCATCTT&#xA;ACAATACAGGTCAATGCTTCGGTGTTTAGCTCCCCGCTGACGTCAGGCTGAGGCGATAAACTCGCCCCCA&#xA;TTCGAGATCGGTTTCCTCGTAACTGATATGAATCCAGCTGCCATCTATAATTTCACTTTCCAGACGCTCA&#xA;CTGACATAACAAATTTAAGATTTAACTCTGGATACTTCTTATTGGCCTGTGCAACAAGAAAAGGCATCAT&#xA;TTTACGAGAAATCGGGCAGATTCAACGCCTTCCATTCTGGTCTCCTTTATAGAGGAAACAAGCTCATGGA&#xA;AATCACCGATGTCTTTCCGTCCATAACTCTAAAATCGACGACTGAAATATGGATACCTGAACTTCCCATA&#xA;CAAGAGCTGCTTTGGTCCTCAACGCCAGTAAAGCAGGTCCAAAAGCGCTACACGCTGCTGGTTCGAACAA&#xA;CATTCAGAAGAGCTTCGTTGAATGTCCAGCTCGACCATAGCAAAATAACAATCAGGCAGTTGTTCACGTT&#xA;TTCATGAATTTTTACTAAGTTCATAAATTCAAGCTGAAGTTTTTTGGCGAGCGCCAGGCTAAAAATACCG&#xA;GCAGATAGCGGTCTGCTTTATCGGACGGGAGAAAAGGTTCTTCACCACACAGACGTTCACAAATATTATC&#xA;GCCGCCGGATTGGCCTCCACATATTCATTTAATCGTTGTGCGCCTTGAGTATGTTTGTAAAAACTCACCG&#xA;TAGCATAACGGCATTGTTATCGAATCGCTCATAAAGCGTTTCATTCTTTTTGTTTACTATACTGCTTCCC&#xA;ACGGTTTAAGTAAAGACTTATATTCAGCTATCCTTTTTTTATGAGCGGATATAGAGAGTTTTTTATCGTT&#xA;TT&#xA;&amp;gt;003277.2:c27903-27178_reverse_complement&#xA;CCCCATCATAAGCAGCTACAATATTCAAAGGAGGGATGTTTAGATTTAGATTAGGCCTATTTATGGGCAT&#xA;ATCTTCCGCATTTGGTTGTGCAAGGAGTTGAAATTATTTTTCAGGTGCTTATTTGTAGATGGTATTTCCG&#xA;AAAGAAACCACTCTGGCGCATCCCTGAATAGTCAGGCACATCCAGCGCCTCTTTAAAGTGGGATACCGGC&#xA;GTGATTCTCGACGAGCATGGATGAAAACATCGTAACCATGGTTATTCAGCTGAAAACCTTGGCTCATAGC&#xA;GCTTGTGGCACCATATCCCTGAGCACACTGATGTGGAACTTGTCACCGGCAAATTTGCCCTGAGACTGAG&#xA;CATATCGGTCACTTTCCACTTATCTACCGGACTGTCCTCTGAAAACAGCAATCCGGACAGCGCTTGAAAT&#xA;CCTGGTCTGGTTTTATATACAACGTGAACTGAGCGCCCAGGCTAACACGGGCTTGTTGAACGACCTTCTC&#xA;AGTCTGGATTCCAGACACTCTATAAATTGCCGTGTCTTGTGGAGAAACGACGCACTGTACTGCGAATTTT&#xA;GAGATATTTCCAATTTTCAGGATGAACGTCTGACTCAGGACACTGTCCTGAAATAACCCCATTTTCGGAT&#xA;GTTCCTCACGTAAAGCCTGTCTCTGCATTTCGCCACCATCACGCCCACTTCGTAGTTCATTACGATAACT&#xA;TTACTCTGTCATCAAACGATAAAACG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17632" LastEditorUserId="73" LastEditDate="2023-06-10T20:57:13.887" LastActivityDate="2023-07-12T14:01:45.280" Title="how to search and calculate occurence of certain word (like TAA) in fastafile" Tags="|python|fasta|multi-fasta|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21152" PostTypeId="2" ParentId="21135" CreationDate="2023-06-11T18:08:05.207" Score="0" Body="&lt;p&gt;Two solutions - the underlying issue is called Type 1 error (below).&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If you are performing studies on humans then the Benjamini-Hochberg method is without question the method of choice, because this includes the &lt;em&gt;false detection&lt;/em&gt; rate for EdgeR and DeSeq2. Thats been independently calculated.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thus, the paper that caused a stir was &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-022-02648-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;Li &lt;em&gt;et al&lt;/em&gt; (2022)&lt;/a&gt; who wrote,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Particularly, the actual FDRs of DESeq2 and edgeR sometimes exceed 20% when the target FDR is 5%.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;strong&gt;Benjamini-Hochberg (BH) method solves that for humans (EdgeR and DeSeq2)&lt;/strong&gt;&lt;/em&gt;, because the problem with the BH method is you need a false discovery rate, Li &lt;em&gt;et al&lt;/em&gt; 2022, calculated that its about 20% ... so for EdgeR and DeSeq2 the problem is solved. The BH tests will simply use the estimate for the FDR by Li &lt;em&gt;et al&lt;/em&gt; (2022) instead of the FDR by EdgeR and DeSeq2. Cool? It is :-) it wasn't what Li &lt;em&gt;et al&lt;/em&gt; (2022) intended but it is the solution to a difficult conundrum (concerning a statistical issue or parametrics vs. non-parametrics). &lt;strong&gt;So yeah it's cool&lt;/strong&gt; (needs count data).&lt;/p&gt;&#xA;&lt;p&gt;To perform BH you simply rank the observed probabilities (e.g. from Limma), and there are significant if each is below (or equal to) the BH rank. The BH rank is the total number of tests as denominator, the rank position as numerator and this is multiplied by the false detection rate as a frequency.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;HOWEVER&lt;/em&gt;, you are using Limma and I don't how the model works. I know the underlying stats behind EdgeR, DeSeq2 and NOISeq.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Okay so the false discovery rate is not known ... in the way Li &lt;em&gt;et al&lt;/em&gt; (2022) calculated it. What now? Limma is likely to calculate an FDR, you could use that via BH. My system is so far from humans there's no way Li &lt;em&gt;et al&lt;/em&gt; (2022) FDR is relevant.&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;Bonferoni's approach - I don't know how mainstream this is, but its what I use (I've a background in classical stats), because I've no idea what the true FDR is. This approach can be exemplified via Volcano plots is to visualise the Type 1 error issues.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Here's a volcano plot which permits online viewing via links &lt;a href=&quot;https://www.htgmolecular.com/blog/2022-08-25/understanding-volcano-plots&quot; rel=&quot;nofollow noreferrer&quot;&gt;obtained here&lt;/a&gt;. I can't change the size BTW (Bioconductor and loads more BTW don't permit this),&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mFeN2.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mFeN2.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is using a flat probability - which in the picture is very low - it's the horizontal threshold (&lt;em&gt;y&lt;/em&gt;-axis). So the P value is one parameter approach to significance ....&lt;/p&gt;&#xA;&lt;p&gt;The other parameter is the log&lt;sub&gt;2&lt;/sub&gt; threshold ...&lt;/p&gt;&#xA;&lt;p&gt;Here's a two-parameter approach obtained from &lt;a href=&quot;https://www.genekitr.fun/plot-others-1.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;GeneKitR's site&lt;/a&gt;, who do permit images to be link viewed (thank you genekitr).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gnq9l.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gnq9l.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here the authors have used log&lt;sub&gt;2&lt;/sub&gt; as the threshold (&lt;em&gt;x&lt;/em&gt;-axis). The threshold is probably log&lt;sub&gt;2&lt;/sub&gt; &amp;gt; 1 and log&lt;sub&gt;2&lt;/sub&gt; &amp;lt; -1, which is reasonable. They have set the P-value threshold to be really low, looks to be about 5%. The plot reveals the effective P-value is 1E-2.5, which is low, but nothing really starts happening until 1E-3, which ... okay. Although this is a two-parameter approach the guys really just relied on log&lt;sub&gt;2&lt;/sub&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Type 1 error and a 1-parameter method threshold&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The massive problem is the shear number of tests performed and this will result in significant values PURELY BY CHANCE ...artefacts of critical probability. BH is one approach, Bonferoni's is an alternative way in.&lt;/p&gt;&#xA;&lt;p&gt;Put simply for your data to calculate the Bonferoni threshold if a 5% critical probability is sought then its ...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;0.05 / 19000 = 2.6e-6&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Which is okay. So any single result &amp;lt;2.6E-6 is significant. This okay, but you will lose results so its vulnerable to Type 2 error. Why do you lose stuff ...&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Type 2 error and 1-parameter method threshold&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The biological question is e.g. is 0.1log&lt;sub&gt;2&lt;/sub&gt; biological sensible? What does it actually mean? Honestly I don't have a clue ... and the problem with a 1-parameter approach, it's very conservative but infers weird stuff. An expression difference of 10% isn't really serious, but thats what it's measuring. ... because its measuring this it means the probability via Bonferoni's is set high - &lt;em&gt;for meaningless stuff&lt;/em&gt; and you can be missing important shifts in log&lt;sub&gt;2&lt;/sub&gt;, e.g. &amp;gt;1 because they are falling under the high threshold.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Type 1 error and a 2-parameter method thresholds&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is where it all starts getting interesting. As soon as a +/- log&lt;sub&gt;2&lt;/sub&gt; threshold is set the number of total tests will drop A LOT. That means the P-value threshold raises. For example, if selecting &amp;gt; (or &amp;lt;) +/- 1log&lt;sub&gt;2&lt;/sub&gt; as used in the above example, supposing the number of observations above/below the +/- 1log&lt;sub&gt;2&lt;/sub&gt; was 5000. The P-value now raises via Bonferoni, e.g.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;0.05/5000 = 1e-05&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;That looks better there'll be more results. Supposing we set a threshold of  +/- 1.5log&lt;sub&gt;2&lt;/sub&gt; and obtained 500 results ... then the new Bonferoni derived threshold is,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;0.05/500 = 1e-04 (0.0001)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;That looks fabulous giving loads of significant results ... Wait isn't this all getting a bit crazy, because it depends on the biological assumption of what a reasonable log&lt;sub&gt;2&lt;/sub&gt; is. Yep, indeed, agreed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Type 1 error and a 2-parameter method thresholds: weaknesses&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The issue is that a reasonable log&lt;sub&gt;2&lt;/sub&gt; is sensible, because there's a point where log&lt;sub&gt;2&lt;/sub&gt; expression differences becoming utterly meaningless. However there's a point up to -/+ 2.5 log&lt;sub&gt;2&lt;/sub&gt; (which is very high) where you could choose anything - that shifts the critical probability and that shifts how many results are positive.&lt;/p&gt;&#xA;&lt;p&gt;So how do you ...&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Set a log&lt;sub&gt;2&lt;/sub&gt; within a 2-parameter threshold?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I've just solved that. I was really happy. The practical answer is it's complicated. My contractual basis is I can't reveal that because public disclosure isn't permitted without top-level approval.&lt;/p&gt;&#xA;&lt;p&gt;The approach is to use an independent method to calculation DEG  (if you check my posts you'll see I do AI). You can then perform parameter tuning of DEG against the independent method and statistically assess the association. In my case I was surprised by the answer - intuitively I wouldn't have gone there, but my system ain't humans.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;In your case,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;simply check the literature to see what other log&lt;sub&gt;2&lt;/sub&gt;  thresholds are in use, use that and adjust the P-value threshold accordingly as described above and cite that work.&lt;/li&gt;&#xA;&lt;li&gt;look at the biological reality of the results at different log&lt;sub&gt;2&lt;/sub&gt; thresholds. For example, an operon should behave like an operon. If the log&lt;sub&gt;2&lt;/sub&gt; threshold is smashing-up that operon - that doesn't make sense and adjust log&lt;sub&gt;2&lt;/sub&gt; threshold by lowering.&lt;/li&gt;&#xA;&lt;li&gt;alternatively use BH, this however assumes Limma's FDR calculation is correct and the problem is there no independent study examining that.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Either of the above 3 approaches will be fine your context. This is not cutting-edge statistical methodology it simply needs to be reasonable.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Ultra-conservative parameters and Type 2 error&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;To avoid all the complexities some investigators simply use &amp;gt; (or &amp;lt;) +/- 2.5log&lt;sub&gt;2&lt;/sub&gt; and P-value = 1e-6. These values in ANY system are extreme when used together, but there is no chance of Type 1 error. The danger is Type 2 error , where significant values are either omitted or considered non-significant. This could happen here from two fronts:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;If this results e.g. 100 observations, and DEGs with P-values &amp;lt; 5e-4 but &amp;gt; 1e-6 are omitted&lt;/li&gt;&#xA;&lt;li&gt;DEGs &amp;lt; (or &amp;gt;) +/- 2.5 log&lt;sub&gt;2&lt;/sub&gt; and above the lower threshold are omitted.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;However, it will depend on the question.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; Much of the theory in &lt;em&gt;point 2&lt;/em&gt;, would also apply to the BH-method. However, I've worked this theory exclusively for Bonferoni's correction and I've not given much thought to BH. This is because it's not relevant to my purpose.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;em&gt;See comments below your post about your TPM approach&lt;/em&gt;. This post is cut-off criteria.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Just to finish this post, here's a volcano with thanks to &lt;a href=&quot;https://www.britannica.com/science/volcano&quot; rel=&quot;nofollow noreferrer&quot;&gt;Britannica&lt;/a&gt;  for permitting the link-view. I think the idea of a volcano plot refers to the &amp;quot;plume&amp;quot; of ash, dust and occasionally lava (which some of us have real-world experience).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/I74jU.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/I74jU.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-06-12T16:06:01.410" LastActivityDate="2023-06-12T16:06:01.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21153" PostTypeId="2" ParentId="21151" CreationDate="2023-06-11T20:53:30.947" Score="1" Body="&lt;p&gt;Since you tag python, I suggest a simple Biopython solution:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;print(sum(s.seq.count(&amp;quot;TAA&amp;quot;) for s in SeqIO.parse(&amp;quot;file.fa&amp;quot;, &amp;quot;fasta&amp;quot;)))&#xA;# 97 on your input file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you wanted an overlapping sequence count, Biopython has a method for this too:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;print(sum(s.seq.count_overlap(&amp;quot;TAA&amp;quot;) for s in SeqIO.parse(&amp;quot;file.fa&amp;quot;, &amp;quot;fasta&amp;quot;)))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here they are the same, but for a string like &amp;quot;TAT&amp;quot; they could be different:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; from Bio.Seq import Seq&#xA;&amp;gt;&amp;gt;&amp;gt; Seq(&amp;quot;TATAT&amp;quot;).count(&amp;quot;TAT&amp;quot;)&#xA;1&#xA;&amp;gt;&amp;gt;&amp;gt; Seq(&amp;quot;TATAT&amp;quot;).count_overlap(&amp;quot;TAT&amp;quot;)&#xA;2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="104" LastEditorUserId="650" LastEditDate="2023-06-12T13:23:41.303" LastActivityDate="2023-06-12T13:23:41.303" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21154" PostTypeId="2" ParentId="21135" CreationDate="2023-06-11T22:23:32.780" Score="1" Body="&lt;p&gt;tl:dr use limma (not voom, that's for raw counts) and refer to previous questions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://support.bioconductor.org/p/98820/#98875&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.bioconductor.org/p/98820/#98875&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://support.bioconductor.org/p/125144/#125156&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.bioconductor.org/p/125144/#125156&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The problem you have is not so much the threshold but the method to infer any metric to be used for this threshold. The arguably most common methods in R for DEG analysis are DESeq2, edgeR and limma, based on number of citations. The former two require raw counts which you apparently do not have, but if you do then use this with any of the mentioned tools.&lt;/p&gt;&#xA;&lt;p&gt;limma is what is left and could be used here with TPM. Note that TPM has the problem that counts are divided by a length factor. A brief background: Any statistical test aims to decide whether the given variance between two groups (basically the logFC) is greater than expected by chance, which is the variance within group. In RNA-seq the variance is not equal across the range of counts (low, middle, high, for simplicity). On log2 count scale variance is greater for genes with low(er) counts than for higher counts. Usually, using raw counts the method (DESeq2, edgeR, limma, and others) learn from the data what is the expected variance at the given average expression level, and incorporate that into the model to make adequate inference even at small sample size. That is by the way where the common assumption &amp;quot;most genes do not change&amp;quot; comes from. Assuming that most genes are not DE, then the mean-variance trend represents the expected variance at the given average count range, and this can then be used to help decide whether a gene is DE or not. With TPM you basically throw this out of the window because the actual counts (as they come from the sequencer) are manipulated by dividing by length and the entire mean-variance model gets distorted. Another factor is that TPM, which corrects by total library size can fail to properly normalize counts, I wrote a summary here:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.biostars.org/p/9465851/#9465854&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/9465851/#9465854&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Long story short: TPM is not good for DEG.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;If it's all you have then use what is called &amp;quot;limma-trend&amp;quot;, see the limma manual at Bioconductor, which will directly model the log2-TPMs and give some logFC and FDR. You can then use standard cutoffs, such as FDR &amp;lt; 0.05 &amp;amp; logFC &amp;gt;/&amp;lt; 1.5. These cutoffs are not perfect, but common, and given that TPMs are bad anyway you can and should stick to standard cutoffs as much as possible, rather than infering your own ones. This at least gives some standardization to this suboptimal analysis.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-06-11T22:23:32.780" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21155" PostTypeId="2" ParentId="21137" CreationDate="2023-06-12T09:11:52.670" Score="1" Body="&lt;p&gt;I tried this and it worked&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat SRR_Acc_List.txt | while read i ; do  fasterq-dump ${i} --split-spot -v --skip-technical  -e 20 ; for j in i ; do   gzip ${i}.fastq  ; done; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17618" LastEditorUserId="650" LastEditDate="2023-06-12T19:33:32.853" LastActivityDate="2023-06-12T19:33:32.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21156" PostTypeId="1" CreationDate="2023-06-12T10:16:30.827" Score="0" ViewCount="22" Body="&lt;p&gt;i am struggling to find the correlation between two isoforms and their survival outcome. i do have a dataset that is a cohort for thousands of genes and am only interesting in two which they should be isofroms to each other. However, i thought the ratio between these two might be a prognostic marker. i need to find a way where i can see if this is possible by interpreting the clinical outcome which is the overall survival. many thanks guys.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Crossposted: &lt;a href=&quot;https://www.biostars.org/p/9566172/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/9566172/&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="17638" LastEditorUserId="3694" LastEditDate="2023-06-12T12:55:55.410" LastActivityDate="2023-06-12T12:55:55.410" Title="How to find the correlation between two genes and their separate overall survival outcome?" Tags="|gene|differential-expression|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21157" PostTypeId="1" CreationDate="2023-06-12T13:37:08.000" Score="0" ViewCount="25" Body="&lt;p&gt;Through fasterq-dump, I am generating 2 files in 1 run of loop. I want to club these two files (A_1.fastq &amp;amp; A_2.fastq) in one folder and zip the it.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat SRR_Acc_List.txt | while read i ; do  fasterq-dump &lt;span class=&quot;math-container&quot;&gt;${i} --split-files -v --skip-technical  -e 15 ; for j in 2 ; do   gzip $&lt;/span&gt;{i}_j.fastq  ; done; done&#xA;cat SRR_Acc_List.txt | while read i ; do  fasterq-dump &lt;span class=&quot;math-container&quot;&gt;${i} --split-files -v --skip-technical  -e 15 ; for(j = 1;j&amp;lt;2;j++) ; do   gzip $&lt;/span&gt;{i}_j.fastq  ; done; done&#xA;&#xA;cat SRR_Acc_List.txt | while read i ; do  fasterq-dump &lt;span class=&quot;math-container&quot;&gt;${i} --split-files -v --skip-technical  -e 15 ; for j in 1 2 ; do   gzip $&lt;/span&gt;{i}_${j}.fastq  ; done; done&#xA;&#xA;cat SRR_Acc_List.txt | while read i ; do  fasterq-dump &lt;span class=&quot;math-container&quot;&gt;${i} --split-files -v --skip-technical  -e 15 ; for j in 2 ; do   gzip $&lt;/span&gt;{i}_&lt;span class=&quot;math-container&quot;&gt;${j}.fastq  ; done; done&#xA;cat SRR_Acc_List.txt | while read i ; do  fasterq-dump $&lt;/span&gt;{i} --split-files -v --skip-technical  -e 15 | mkdir {i} | for j in i; do   gzip ${i}  ; done; done&#xA;&#xA;cat SRR_Acc_List.txt | while read i ; do  fasterq-dump &lt;span class=&quot;math-container&quot;&gt;${i} --split-files -v --skip-technical  -e 15 | mkdir $&lt;/span&gt;{i} ; for j in i; do   gzip &lt;span class=&quot;math-container&quot;&gt;${i}  ; done; done&#xA;cat SRR_Acc_List.txt | while read i ; do  fasterq-dump $&lt;/span&gt;{i} --split-files -v --skip-technical  -e 15 ; mkdir &lt;span class=&quot;math-container&quot;&gt;${i} ; mv $&lt;/span&gt;{i}_*.fastq &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${i}&quot;; for j in i ; do gzip $&lt;/span&gt;{i} ; done;done&#xA;&#xA;cat SRR_Acc_List.txt | while read i ; do  fasterq-dump &lt;span class=&quot;math-container&quot;&gt;${i} --split-files -v --skip-technical  -e 15 ; mkdir $&lt;/span&gt;{i} ; mv &lt;span class=&quot;math-container&quot;&gt;${i}_*.fastq &quot;$&lt;/span&gt;{i}&amp;quot;;done; for j in i ; do zip ${i} ; done&#xA;&#xA;cat SRR_Acc_List.txt | while read i ; do  fasterq-dump &lt;span class=&quot;math-container&quot;&gt;${i} --split-files -v --skip-technical  -e 15 ; mkdir $&lt;/span&gt;{i} ; mv &lt;span class=&quot;math-container&quot;&gt;${i}_*.fastq &quot;$&lt;/span&gt;{i}&amp;quot; ;  zip ${i} ; done&#xA;&#xA;cat SRR_Acc_List.txt | while read i ; do  fasterq-dump &lt;span class=&quot;math-container&quot;&gt;${i} --split-files -v --skip-technical  -e 15 ; mkdir $&lt;/span&gt;{i} ; mv &lt;span class=&quot;math-container&quot;&gt;${i}_*.fastq &quot;$&lt;/span&gt;{i}&amp;quot; ; tar -cv &lt;span class=&quot;math-container&quot;&gt;${i} | gzip &amp;gt; $&lt;/span&gt;{i}.tar.gz ; done&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;None of them is working.&lt;/p&gt;&#xA;" OwnerUserId="17618" LastEditorUserId="3694" LastEditDate="2023-06-12T19:21:14.753" LastActivityDate="2023-06-12T19:21:14.753" Title="adding multiple files in folder and zipping in loop" Tags="|fasta|ncbi|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21158" PostTypeId="1" CreationDate="2023-06-12T13:39:18.510" Score="3" ViewCount="60" Body="&lt;p&gt;I am trying to transform a distance matrix from the qiime2 pathway using a centred log ratio in R. I uploaded the matrix directly from a qiime2 output file, and have converted to a data matrix and have checked that it is symmetrical.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;structure(list(X1 = c(0, 0.2177609998, 0.2133361674, 0.1549136105, 0.1400395799), X11 = c(0.2177609998, 0, 0.07805820645, 0.1418994689, 0.1934668819), X12 = c(0.2133361674, 0.07805820645, 0, 0.1475390242, 0.1857477705), X13 = c(0.1549136105, 0.1418994689, 0.1475390242, 0, 0.1046740994), X14 = c(0.1400395799, 0.1934668819, 0.1857477705, 0.1046740994, 0)), row.names = c(&amp;quot;X1&amp;quot;, &amp;quot;X11&amp;quot;, &amp;quot;X12&amp;quot;, &amp;quot;X13&amp;quot;, &amp;quot;X14&amp;quot;), class = &amp;quot;data.frame&amp;quot;)&#xA;dm &amp;lt;- read.csv(file = &amp;quot;dm.csv&amp;quot;, header = TRUE, row.names = 1)&#xA;isSymmetric(as.matrix(dm)) [1] TRUE&#xA;dm_matrix &amp;lt;- (data.matrix(dm, rownames.force = NA))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then tried to transform the dataset using the &lt;code&gt;dm_matrix_clr &amp;lt;- data.frame(clr(dm_matrix))&lt;/code&gt; function from the 'compositions' package. This produced an output, however it was not symmetrical. From my understanding, the transformed distance matrix should be symmetric like the input matrix. The same numbers on opposing sides of the diagonal are being transformed, but are resulting in different outputs.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ENWBr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ENWBr.png&quot; alt=&quot;transformed distance matrix&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Any help on how resolve this problem so I have a symmetrical transformed distance matrix would be greatly appreciated.&lt;/p&gt;&#xA;" OwnerUserId="17577" LastEditorUserId="3694" LastEditDate="2023-06-12T15:31:43.370" LastActivityDate="2023-06-15T17:34:14.410" Title="Why is my centred log ratio transformation on a distance matrix in R not symmetrical?" Tags="|r|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21159" PostTypeId="1" AcceptedAnswerId="21166" CreationDate="2023-06-12T22:04:45.597" Score="2" ViewCount="85" Body="&lt;p&gt;I have a simple dataframe that includes points where the x values are 0, 1, 2, 3, 4, and 7. When I go to plot the data this is what ggplot2 gives me:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/8JfDJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/8JfDJ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I only want to include the points mentioned above. How do I take out the instance of 5 and 6 in the bar plot?&lt;/p&gt;&#xA;&lt;p&gt;(I have tried googling the answer and nothing is coming up with my actual need so I figured I'd ask the community which might give an answer sooner - please be kind I know this is bound to have a simple solution)&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="73" LastEditDate="2023-06-14T21:19:53.447" LastActivityDate="2023-06-14T21:19:53.447" Title="Removing missing categories from geom_bar" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21160" PostTypeId="1" AcceptedAnswerId="21163" CreationDate="2023-06-13T08:07:23.753" Score="1" ViewCount="128" Body="&lt;p&gt;Here is part of my data table, which I will call &amp;quot;df&amp;quot;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    Subject.ID    Sex           V1           V2           V3           V4           V5&#xA;1   GTEX-1117F female   4.24944534   0.18307358   1.99276843   0.21785110   0.74777407&#xA;2   GTEX-1128S male    3.286585483  0.165944088  1.843983844  0.444455046  0.307311939&#xA;3   GTEX-11EMC female   3.31947343   0.27846061   1.46519089   0.33753984   0.91708799&#xA;4   GTEX-11GSP female  3.232353768  0.215492230  1.778208576  0.166458372  0.644225472&#xA;5   GTEX-11TTK male    2.934705200  0.256406854  1.712815854  0.256165277  0.948301812&#xA;6   GTEX-11ZUS male     3.77188558   0.18434378   2.43189035   0.37373172   1.14339342&#xA;7   GTEX-12WSD female   4.22032995   0.21933892   0.93900085   0.08687886   1.06901468&#xA;8   GTEX-12ZZX female   3.43616184   0.39473363   2.73205207   0.33525457   1.51399598&#xA;9   GTEX-1313W male     3.66334462   0.39418485   2.47248777   0.53545580   0.64634702&#xA;10  GTEX-131XW female  2.956614288  0.276317993  1.977096712  0.167743280  1.528071165&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How do I correctly format the following code to account for the kind of dataframe I'm working with? I'm using sex as the factors to be interacted. Here is what I have so far:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;design &amp;lt;- model.matrix(~ Sex)&#xA;fit &amp;lt;- lmFit(df, design)&#xA;fit &amp;lt;- eBayes(fit)&#xA;topTable(fit)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The first line is giving me the error &amp;quot;Error in eval(predvars, data, env) : object 'Sex' not found&amp;quot; (and idk why I'm getting this error since there is a &amp;quot;Sex&amp;quot; column in df). The second line gives me the error &amp;quot;Expression object should be numeric, instead it is a data.frame with n non-numeric columns&amp;quot; (and I know why I'm getting this error; I'm not sure how to handle the error, though).&lt;/p&gt;&#xA;" OwnerUserId="17613" LastEditorUserId="3694" LastEditDate="2023-06-13T08:09:14.293" LastActivityDate="2023-06-13T10:33:58.900" Title="How do I correctly format my limma/eBayes code?" Tags="|r|limma|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21161" PostTypeId="1" CreationDate="2023-06-13T08:42:46.017" Score="0" ViewCount="23" Body="&lt;p&gt;I was wondering if anyone know of a repository for C. elegans data sets, specifically in L1 larvae stage.&lt;/p&gt;&#xA;&lt;p&gt;Are there any papers or GEO-sets for this organism at this stage?&lt;/p&gt;&#xA;&lt;p&gt;thanks&lt;/p&gt;&#xA;&lt;p&gt;Assa&lt;/p&gt;&#xA;" OwnerUserId="1440" LastActivityDate="2024-03-11T00:02:45.300" Title="Looking for datasets of RNA-Seq (or scRNA-Seq) for L1 larvae C. elegans" Tags="|rna-seq|scrnaseq|wormbase|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21162" PostTypeId="1" AcceptedAnswerId="21168" CreationDate="2023-06-13T09:01:37.600" Score="0" ViewCount="124" Body="&lt;p&gt;I'm running &lt;strong&gt;BEAST v1.10.4&lt;/strong&gt; with &lt;strong&gt;BEAGLE v4.0.0&lt;/strong&gt;. My dataset has one large partition and I have access to a machine with many CPUs so I would like to optimize performance by parallelizing the process. Following the &lt;a href=&quot;https://beast.community/performance&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt; and various (quite old and probably outdated) help Q&amp;amp;A pages I tried&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;beast -beagle -beagle_CPU -beagle_instances X -beagle_thread_count Y -threads Z mydata.xml&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;with different values for XYZ but BEAST is ever only using a single CPU regardless.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;How do I make BEAST use on more than one CPU?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;We have a 'server' in-house, that is essentially a big computer with 144 CPUs that is shared by all users. That's why it would be important to limit the max amount of threads with the -threads option. No queuing system, no GPUs. I want to do some tip date sampling which afaik is not available in Beast2.&lt;/p&gt;&#xA;" OwnerUserId="9564" LastEditorUserId="73" LastEditDate="2023-06-14T21:10:34.543" LastActivityDate="2023-07-14T22:03:10.407" Title="How to parallelize BEAST" Tags="|phylogenetics|phylogeny|beast|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21163" PostTypeId="2" ParentId="21160" CreationDate="2023-06-13T10:33:58.900" Score="2" Body="&lt;p&gt;You need to look at how to create a model matrix : &lt;a href=&quot;https://www.rdocumentation.org/packages/stats/versions/3.6.2/topics/model.matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rdocumentation.org/packages/stats/versions/3.6.2/topics/model.matrix&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In your code, you did not specify the data from which the Sex columns should be taken, or you did not created the Sex variable&lt;/p&gt;&#xA;&lt;p&gt;Either create &lt;code&gt;Sex=df$Sex&lt;/code&gt; or specify it in the model.matrix function &lt;code&gt;model.matrix(~ Sex, data=df)&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="14639" LastActivityDate="2023-06-13T10:33:58.900" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21164" PostTypeId="1" AcceptedAnswerId="21652" CreationDate="2023-06-13T12:15:36.750" Score="1" ViewCount="72" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9566425/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt; 3 months ago, i.e. a long time - so its perfectly reasonable for the question to be asked here&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I started studying bioinformatics and I enjoy it a lot (I'm a software developer with around 15 years of experience but mostly in backend, scripting, architecture, and security) and I want to do, as a project, a database regarding natural compounds. When I saw that on NCBI there are a lot of plant genomes I thought, that, with software, somehow we can &amp;quot;guess&amp;quot; (probabilities, not certainties) what a plant can contain or not.&lt;/p&gt;&#xA;&lt;p&gt;I'm wondering, if, by bioinformatic means only, it is possible to &amp;quot;extract&amp;quot; genes and, after that proteins and natural compounds from plant genome data that is available on NCBI (please see &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets/genome/GCA_029618835.1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/datasets/genome/GCA_029618835.1/&lt;/a&gt; as an example)? Example: just by processing the data from the link above we can get to Flavoxanthin or any other substance.&lt;/p&gt;&#xA;&lt;p&gt;I was wondering if a kind of &amp;quot;de-novo&amp;quot; annotation can be made (if that's the correct term).&lt;/p&gt;&#xA;&lt;p&gt;What other useful information from a chemical compound perspective can be found from digging on the genome itself?&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="17646" LastEditorUserId="3848" LastEditDate="2023-10-09T14:35:05.417" LastActivityDate="2023-10-09T14:35:05.417" Title="Extract data from genome" Tags="|genome|gene|sequence-annotation|genomics|plants|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21166" PostTypeId="2" ParentId="21159" CreationDate="2023-06-13T14:37:40.617" Score="4" Body="&lt;p&gt;I was able to get it to work by converting the column in the dataframe to a character using this command:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;df$day &amp;lt;- as.character(df$day)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;and it worked like a charm! Thanks for your advice @haci&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;You can give a try by converting the column of numbers to a factor variable.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Aotfy.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Aotfy.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="73" LastEditDate="2023-06-14T21:18:19.183" LastActivityDate="2023-06-14T21:18:19.183" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21167" PostTypeId="1" CreationDate="2023-06-13T15:45:43.513" Score="2" ViewCount="120" Body="&lt;p&gt;&lt;em&gt;This question has been asked and answered on &lt;a href=&quot;https://stackoverflow.com/questions/76466744/nextflow-pass-reads-variable-to-process-in-another-script&quot;&gt;Stack Overflow&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm new to nextflow, channel, process. Kindly bear with me. I'd like to create a pipeline, pass paired reads from one to another process. I am unable to do it.&lt;/p&gt;&#xA;&lt;p&gt;script1.nf&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params.outdir = 'results'&#xA;&#xA;params.reads1 = &amp;quot;/Users/user/Downloads/tiny/normal/*_R1_xxx.fastq.gz&amp;quot;&#xA;&#xA;params.reads2 = &amp;quot;/Users/user/Downloads/tiny/normal/*_R2_xxx.fastq.gz&amp;quot;&#xA;&#xA;println &amp;quot;reads: $params.reads1&amp;quot;&#xA;&#xA;println &amp;quot;reads: $params.reads2&amp;quot;&#xA;&#xA;Channel.fromPath(params.reads1,checkIfExists: true).view()&#xA;&#xA;Channel.fromPath(params.reads2,checkIfExists: true).view()&#xA;&#xA;include { fastp } from '/Users/name/Documents/name/nextflow_scripts/fastp.nf'&#xA;&#xA;&#xA;fqpairs_ch=channel.fromFilePairs('/Users/name/Downloads/tiny/normal/*_R{1,2}_xxx.fastq.gz')&#xA;&#xA;&#xA;process check {&#xA;&#xA;    output: &#xA;    stdout&#xA;&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot; &#xA;    echo &amp;quot;hello from command line \n&amp;quot;&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;}&#xA;&#xA;workflow {&#xA;&#xA;    check().view(). /// this works&#xA;&#xA;    fastp(fqpairs_ch).view() // it gives error&#xA;&#xA;}&#xA; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;fastp.nf&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params.outdir = &amp;quot;./results&amp;quot;&#xA;&#xA;process fastp() {&#xA;&#xA;tag &amp;quot;fastp on $sample_id&amp;quot;&#xA;&#xA;&#xA;input:&#xA;        tuple val(sample_id), file(x), file(x1) from fqpairs_ch&#xA;&#xA;&#xA;output:&#xA;println (&amp;quot;$sample_id&amp;quot;)&#xA;&#xA;&#xA;    script:&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot; &#xA;&#xA;mkdir fastp_trimmed&#xA;&#xA;        fastp \&#xA;        --correction \&#xA;    --cut_tail \&#xA;    --disable_trim_poly_g \&#xA;    --length_required 50 \&#xA;    --qualified_quality_phred 20 \&#xA;    --thread 12 \&#xA;    --trim_poly_x \&#xA;    --unqualified_percent_limit 20 \&#xA;        -i &lt;span class=&quot;math-container&quot;&gt;${x[0]} -I $&lt;/span&gt;{x[1]} \&#xA;        -o fastp_trimmed/joinedfiles.dat_&lt;span class=&quot;math-container&quot;&gt;${x[0]} -O fastp_trimmed/joinedfiles.dat_$&lt;/span&gt;{x[1]} &#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Error:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;ERROR ~ No such variable: fqpairs_ch&lt;/p&gt;&#xA;&lt;p&gt;-- Check script 'fastp.nf' at line: 13 or see '.nextflow.log' file&#xA;for more details&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I'm unable to wrap by head around tag, val. Also, it is difficult for me to understand how to pass variable from one script to another. I've watched Youtube tutorial but with limited success.&lt;/p&gt;&#xA;" OwnerUserId="862" LastEditorUserId="73" LastEditDate="2023-06-14T21:06:45.027" LastActivityDate="2023-06-14T21:06:45.027" Title="Nextflow: pass reads variable to process in another script" Tags="|nextflow|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21168" PostTypeId="2" ParentId="21162" CreationDate="2023-06-13T17:02:40.013" Score="1" Body="&lt;p&gt;The machine is important because of the introduction of M1 type chips which will run GPU.&lt;/p&gt;&#xA;&lt;p&gt;From historical use of beast1 the following will work ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;beast -overwrite -beagle_auto -seed 2001 mydata.xml &amp;gt; myfile.runtime.log&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will parallelise happily within the number of processors available. I would recommend it. This is because it may not parallelise across 144 CPUs. It depends on the machine architecture. Also from memory (mine not RAM) it dynamically readjusts, so even if it was 144 CPUs parallelised when another job lands on a CPU that drops to 143 CPUs for beast1.&lt;/p&gt;&#xA;&lt;p&gt;Anyway, I use the Cloud so I no longer keep up with computer architecture.&lt;/p&gt;&#xA;&lt;p&gt;If this floods 144 processors and there's no dynamic readjustment I would suggest,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; beast -beagle -beagle_SSE -beagle_CPU -threads 10 -overwrite mydata.xml &amp;gt; myfile.runtime.log&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would limit the number of threads to get it working before considering e.g. 40.&lt;/p&gt;&#xA;&lt;p&gt;However, shifting to GPU is strongly recommended, this will solve any beast2 issues and its very fast. However, I don't know about beast1 for GPU, I've never used it for that.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;If none of the above work and you've not get any sort of M1 or higher chip (to move to GPU) then you need to look at the error log, especially if you see this error.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Error 122 parsing the xml input file&#xA;Cannot create class: ThreadedTreeLikelihood.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; although you've got a nice CPU heavy machine, simple fairly modern Mac will do the same operation probably quicker via GPU, at least for beast2, beast1 I dunno&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Just to note I'm acting in solidarity with an SE strike until an internal matter is resolved about ChatGPT and requested a meeting with staff to explain why. I've formally notified SE. I'll be answering very select questions and doing only essential admin  (breaches of CoC and spam). Hopefully all is back to normal shortly.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-06-13T23:53:49.743" LastActivityDate="2023-06-13T23:53:49.743" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21170" PostTypeId="1" CreationDate="2023-06-13T20:53:02.820" Score="0" ViewCount="20" Body="&lt;p&gt;I have been thinking about to what extent a biological entity’s characteristics can be decomposed into discrete sections of the genome that are parsed and mapped one-to-one to a physical structure. It is fascinating in this regard how indistinguishable biology is from computational linguistics and formal systems theory in general - DNA is a formal language; and we are still trying to unravel the grammar. Right?&lt;/p&gt;&#xA;&lt;p&gt;I think the following question depends on what type of “grammar” DNA has. I don’t know the term in the field, but I have been using the term “holographic” in my own thinking, when a sequence in a code is interpreted twice - by two different grammars, or two different parts or stages of a grammar, contributing meaning to multiple realizations in the output. (“Holographic”, as in, a 2D surface appears to be 3D so long as the information is encoded on it in such a way to be parsible / reconstructible - the information is not lost, in the mix, but is able to be separated and mathematically resolved (some sort of non-associativity under some function, is what I suspect).&lt;/p&gt;&#xA;&lt;p&gt;My question is about to what extent “kinship” is indicated by “having similar genes”.&lt;/p&gt;&#xA;&lt;p&gt;Genes are &lt;em&gt;virtual&lt;/em&gt; (I think) - pure information. It seems like the same genes can come into being without them having replicated from the same ancestor (convergent evolution). The presence of the same gene in two biological entities does not strictly imply closer “kinship” when compared to two biological entities without that shared gene. Conversely, we need not expect entities of close kinship to share the same genes (a sibling can inherit one allele of some gene, another sibling the other one, dominant or recessive, etc.)&lt;/p&gt;&#xA;&lt;p&gt;Perhaps there is an analogy between language and biology, that I haven’t clearly spelled out, yet.&lt;/p&gt;&#xA;&lt;p&gt;-&amp;gt; Organisms are to DNA what sentences are to “logical form” (in linguistics).&lt;/p&gt;&#xA;&lt;p&gt;-&amp;gt; DNA is to mechanisms that build DNA (RNA transcription, protein synthesis, etc.) what logical form is to “universal grammar” (or, the actual system cradling the human aptitude for languages, itself).&lt;/p&gt;&#xA;&lt;p&gt;-&amp;gt; (The odd part): Biological reproduction is when a “sentence” of language directly, physically transforms (or, contributes a part of itself) into an instance of a “grammar” (a valid grammar, as specified by the parameters/constraints/dictates of “universal grammar”). It is intriguing how the analogy gets challenged here, perhaps to be probed further. For now, it could be seen to stretch to how human language can &lt;em&gt;express&lt;/em&gt; grammars (like humans can define grammars - constructed languages, as one example - mathematics - or any formal system at all.)&lt;/p&gt;&#xA;&lt;p&gt;-&amp;gt; Lastly, biological evolution has some analogy to language evolution (perhaps, a fourth “layer” or “grammar” (a specification of means by which something can change - rules, constraints) of the system), in that there is some stochastic generation which gets selected for some (chaotically changing, varying) context (ie, natural selection; as for humans and language, it might be hard to specify the various reasons, including highly social ones, why language features, or cultural features, are either retained or dissipate (Dawkins’s concept of “memes”)).&lt;/p&gt;&#xA;&lt;p&gt;I think there are two competing dynamics.&lt;/p&gt;&#xA;&lt;p&gt;One says that similar genes are not “informationally significant” because the “universal grammar” &lt;em&gt;and&lt;/em&gt; the transformation rules/rounds/generations on universal grammars themselves (the ways the system can vary from generation to generation, in the space of possible evolutionary trajectories) can easily generate the same structure with multiplicity, independently.&lt;/p&gt;&#xA;&lt;p&gt;Another competing force says that presence of similar genes indicates a likelihood of kinship because it is improbable that two people who collectively have a high degree of similarity between their total collection of genes, perhaps (this is how ethnicities arise - a group exchanging genes internally, within a single evolutionary path, pass the same genes around amongst each other; the same with human languages).&lt;/p&gt;&#xA;&lt;p&gt;I wonder if there is any “equilibrium” theory, where the extent to which a system’s propensity to permit random duplicates of a form, is somehow balanced with the (evolutionary) dynamic where memes (genes, language features) cluster with regards to proximity of interaction.&lt;/p&gt;&#xA;&lt;p&gt;So we at the end of the day have a situation where similar overall gene profiles is actually a probable indicator of “poly”-kinship (a kinship vector where any two entities have some last common ancestor for each of their genes, not just their last common “parent” - and those kinship vectors can have unique distributions, from homogeneous to with wide range - but the same average).&lt;/p&gt;&#xA;&lt;p&gt;But, there are still outliers where a person is able to have a similar phenotype because of sufficient randomness in the way genes work, without it indicating a high-value (normed) kinship-vector.&lt;/p&gt;&#xA;&lt;p&gt;(This could apply to linguistics where it would be fallacious to assume Japanese and Turkish must be related to each other just because they share strikingly agglutinative morphology features.)&lt;/p&gt;&#xA;&lt;p&gt;How is this modeled mathematically, in bioinformatics?&lt;/p&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="16692" LastActivityDate="2023-06-13T20:53:02.820" Title="The mathematics of kinship (degrees and distributions of relatedness on individual and group levels)" Tags="|statistics|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21171" PostTypeId="1" AcceptedAnswerId="21213" CreationDate="2023-06-14T06:44:53.300" Score="3" ViewCount="131" Body="&lt;p&gt;I want a process by which I can identify genes within plant genomes; i.e. a process that works for all plants, not just for a single species.&lt;/p&gt;&#xA;&lt;p&gt;I'm wondering, if, by bioinformatic means only, it is possible to &amp;quot;extract&amp;quot; genes and, after that proteins and natural compounds from plant genome data that is available on NCBI (please see &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets/genome/GCA_029618835.1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/datasets/genome/GCA_029618835.1/&lt;/a&gt; as an example)?&#xA;Example: just by processing the data from the link above we can get to Flavoxanthin or any other substance.&lt;/p&gt;&#xA;&lt;p&gt;What other useful information from a chemical compound perspective can be found from digging into the genome itself?&lt;/p&gt;&#xA;&lt;p&gt;Any hints will be greatly appreciated.&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="17646" LastEditorUserId="73" LastEditDate="2023-06-15T05:15:02.727" LastActivityDate="2023-06-22T15:54:25.043" Title="Extract compounds/proteins from plant genomic data" Tags="|genome|software-recommendation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21172" PostTypeId="1" CreationDate="2023-06-14T09:23:08.760" Score="0" ViewCount="20" Body="&lt;p&gt;I have performed NGS on 34 patient samples and 11 controls. Now I have variants from both groups but there is obvious difference in number of variants.&#xA;How can I compare variants of both groups statistically?&#xA;Can I do association analysis?&lt;/p&gt;&#xA;" OwnerUserId="15560" LastActivityDate="2023-06-14T09:23:08.760" Title="What statistical analysis should I perform on DNA variants from unequal number of case (34) and control (11) samples?" Tags="|ngs|statistics|sequence-analysis|variants|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21173" PostTypeId="1" CreationDate="2023-06-14T10:02:00.340" Score="1" ViewCount="130" Body="&lt;p&gt;I am just starting on my atac-seq analysis (not very experienced, so my apologies for a stupid question), and I am wanting to trim the adapters of off my paired-end reads.&lt;/p&gt;&#xA;&lt;p&gt;I am following the galaxy atac-seq protocol that says to enter in the custom 3' adapter sequence. I took the primer sequences from Buenrostro.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;Ad1_noMX&#xA;AATGATACGGCGACCACCGAGATCTACACTCGTCGGCAGCGTCAGATGTG&#xA;&amp;gt;Ad2.7_CTCTCTAC&#xA;CAAGCAGAAGACGGCATACGAGATGTAGAGAGGTCTCGTGGGCTCGGAGATGT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Following the ATAC protocol, I see that the primer sequences they used are the reverse complement sequences of the end of the provided sequences (ACATCTCCGAGCCCACGAGAC/CACATCTGACGCTGCCGACGA). I took this part and added the reverse complement index primer of 2.7 (GTAGAGAG) to the front.&lt;/p&gt;&#xA;&lt;p&gt;So for my R1 my sequence is:&#xA;GTAGAGAGACATCTCCGAGCCCACGAGAC&#xA;And for my R2 my sequence is:&#xA;GTAGAGAGCACATCTGACGCTGCCGACGA&lt;/p&gt;&#xA;&lt;p&gt;However, when I run the command I only get 1.4% read1/2 with adapter. Could anyone suggest where I am going wrong?&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="17650" LastEditorUserId="73" LastEditDate="2023-06-14T20:52:48.053" LastActivityDate="2023-06-14T20:53:22.297" Title="How to identify index sequences for cutadapt for atac-seq" Tags="|atac-seq|cutadapt|galaxy|indexing|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21174" PostTypeId="1" AcceptedAnswerId="21179" CreationDate="2023-06-14T10:06:32.570" Score="2" ViewCount="74" Body="&lt;p&gt;How are the &amp;quot;effect_allele&amp;quot; and &amp;quot;other_allele&amp;quot; bases chosen in a PGS file from the PGS Catalog?&#xA;For example, for PGS002723 (&lt;a href=&quot;https://www.pgscatalog.org/score/PGS002723/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.pgscatalog.org/score/PGS002723/&lt;/a&gt;), if I set the &amp;quot;effect_allele&amp;quot; to &amp;quot;ref&amp;quot;, then &amp;quot;ref&amp;quot; is equal to the reference&#xA;base of hg38 in 98.60% of the cases on chromosome 22.&#xA;On chromosome 22, there are 205 PGS variants where the &amp;quot;effect_allele&amp;quot; does not match the&#xA;reference base of hg38. In all of these 205 cases, the &amp;quot;other_allele&amp;quot; matches the hg38&#xA;base.&lt;/p&gt;&#xA;&lt;p&gt;Why does PGS Catalog encode the hg38 reference base in 14,477 of cases for chromosome 22&#xA;as &amp;quot;effect_allele&amp;quot; and in 205 of cases as &amp;quot;other_allele&amp;quot;?&lt;/p&gt;&#xA;&lt;p&gt;Finally, how can I deal with this encoding if I want to compute the polygenic score on a certain cohort?&#xA;Can I simply invert the sign of the &amp;quot;effect_weight&amp;quot; column for these 205 cases where the &amp;quot;other_allele&amp;quot; matches the reference?&lt;/p&gt;&#xA;&lt;p&gt;I paste my reproducible code below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;suppressMessages(library(BSgenome.Hsapiens.UCSC.hg38))&#xA;suppressMessages(library(data.table))&#xA;suppressMessages(library(R.utils))&#xA;suppressMessages(library(stringr))&#xA;&#xA;setwd('/cluster/home/criccio/PRS/')&#xA;&#xA;source('scripts/get_hg38_base.R')&#xA;&#xA;if (!dir.exists('output/table'))  dir.create('output/table')&#xA;&#xA;PGS_ID &amp;lt;- 'PGS002723'&#xA;&#xA;# Example to ask questions online&#xA;system(paste0('wget -P output/ --recursive --no-parent ftp://ftp.ebi.ac.uk/pub/databases/spot/pgs/scores/', PGS_ID, '/ScoringFiles/'))&#xA;&#xA;scoring_file &amp;lt;- data.table::fread(paste0('output/ftp.ebi.ac.uk/pub/databases/spot/pgs/scores/', PGS_ID, '/ScoringFiles/Harmonized/', PGS_ID, '_hmPOS_GRCh38.txt.gz'))&#xA;data.table::setnames(scoring_file, c('chr_name', 'hm_pos', 'effect_allele', 'other_allele'), c('chr', 'pos', 'ref', 'alt'))&#xA;&#xA;pgs_variants_per_chrom &amp;lt;- scoring_file[, .N, chr]&#xA;data.table::setnames(pgs_variants_per_chrom, 'N', 'nb_pgs_in_scoring_file')&#xA;&#xA;chromosome_number &amp;lt;- 22&#xA;&#xA;scoring_file[chr == chromosome_number, ref_hg38 := sapply(scoring_file[chr == chromosome_number, pos], get_hg38_chr22_base)]&#xA;scoring_file[chr == chromosome_number, ref_hg38 := toupper(ref_hg38)]&#xA;scoring_file[chr == chromosome_number, table(ref == ref_hg38)] # 14477 TRUE and 205 FALSE&#xA;scoring_file[chr == chromosome_number, 100 * mean(ref == ref_hg38, na.rm = TRUE)] # 98.60%&#xA;&#xA;scoring_file_mismatch &amp;lt;- scoring_file[chr == chromosome_number &amp;amp; ref != ref_hg38] &#xA;scoring_file_mismatch[chr == chromosome_number, mean(alt == ref_hg38)] # 100% match&#xA;data.table::fwrite(scoring_file_mismatch, paste0('output/table/', PGS_ID, '_chr_pos_ref_mismatching_hg38_chr', chromosome_number, '.csv'), sep = ',')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;get_hg38_base.R&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(BSgenome.Hsapiens.UCSC.hg38)&#xA;Hsapiens_chr22 &amp;lt;- strsplit(as.character(Hsapiens$chr22), '')[[1]]&#xA;get_hg38_chr22_base &amp;lt;- function(pos) Hsapiens_chr22[pos]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17128" LastActivityDate="2023-06-14T20:36:18.490" Title="How are the &quot;effect_allele&quot; and &quot;other_allele&quot; bases chosen in a PGS file from the PGS Catalog?" Tags="|genomics|gwas|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21175" PostTypeId="2" ParentId="21173" CreationDate="2023-06-14T11:31:49.710" Score="2" Body="&lt;p&gt;The sequence to trim for ATAC-seq is &lt;code&gt;CTGTCTCTTATACACATCT&lt;/code&gt;. It's the Nextera adapter sequence. The sequence is the reverse complement of the 3' ends of your primers, both forward and reverse.&lt;/p&gt;&#xA;&lt;p&gt;You trim adapters which come before any index sequence so trimming adapter (and by this everything beyond it) is sufficient.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastEditorUserId="73" LastEditDate="2023-06-14T20:53:22.297" LastActivityDate="2023-06-14T20:53:22.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21176" PostTypeId="1" CreationDate="2023-06-14T12:48:23.293" Score="2" ViewCount="15" Body="&lt;p&gt;My aim is to associate CNVs to DNAm, but I don't know where to find datasets which contain both (sequencing + methylation). Do you know any?&lt;/p&gt;&#xA;" OwnerUserId="14030" LastActivityDate="2023-06-14T12:48:23.293" Title="datasets or biobanks with both (WGS or genotype intensity) and methylation data" Tags="|cnv|methylation|public-dataset|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21177" PostTypeId="1" CreationDate="2023-06-14T12:51:33.427" Score="2" ViewCount="15" Body="&lt;p&gt;I found that GTEx contains several SNPs eQTLs, but I'm looking for previously reported CNV-eQTLs, where can I find such data (freely available)?&lt;/p&gt;&#xA;" OwnerUserId="14030" LastActivityDate="2023-06-14T12:51:33.427" Title="consortium, biobank or browser with CNV eQTLs" Tags="|cnv|public-dataset|qtls|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21178" PostTypeId="2" ParentId="3249" CreationDate="2023-06-14T16:47:25.117" Score="0" Body="&lt;p&gt;I recommend using NCBI's tool for this called &lt;code&gt;update_blastdb.pl&lt;/code&gt;, shipped with the blast suite (I use the conda environment from &lt;a href=&quot;https://bioconda.github.io/recipes/blast/README.html?highlight=blast#package-package%20%27blast%27&quot; rel=&quot;nofollow noreferrer&quot;&gt;bioconda&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;cd&lt;/code&gt; to the directory you want nt DB in and run:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;update_blastdb.pl --decompress nt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Pros: if a download fails for whatever reason it is retried, downloads are md5sum checked and re-running in a folder with an existing DB will (as the name suggests) only update the relevant DB portions. More information &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK569850/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;!&lt;/p&gt;&#xA;" OwnerUserId="15257" LastActivityDate="2023-06-14T16:47:25.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21179" PostTypeId="2" ParentId="21174" CreationDate="2023-06-14T19:42:06.980" Score="2" Body="&lt;p&gt;Effect allele is almost always the reference allele. It theoretically does not have to be, so it's good to not rely on that fact. However, in practice it is typically a safe bet. At first glance, the answer to your question could be this:&lt;/p&gt;&#xA;&lt;p&gt;The polygenic score from the PGS catalog is using GRCh37. Your genome build is hg38. I expect that some alleles in the score swapped ref/alt positions between builds, resulting in your 205 cases where the effect allele matches the alt allele.&lt;/p&gt;&#xA;&lt;p&gt;To confirm, could you list a few examples of cases that matched as expected and a few examples of cases that didn't match?&lt;/p&gt;&#xA;&lt;p&gt;You don't need to deal with this encoding if you want to compute a polygenic score on a cohort. You can match based on the allele itself. (Also, there are existing tools which will calculate polygenic scores for you.) Here is an example (just a snippet, you'd have to iterate through your files, etc. for this to work):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;            if PGS_row scoring file identifier like rsID or position == vcf_row identifier: # When the SNP is in both files&#xA;            effect_allele = PGS_row[2] # Wherever the affect allele is in your scoring file&#xA;            effect_weight = float(PGS_row[4])&#xA;            dosage (of ref allele) = (depends on your VCF formatting)&#xA;&#xA;            if effect_allele == vcf_row[3]: # Effect allele matches reference allele&#xA;                running_total += (2 - dosage) * effect_weight # We assume dosage of alt is 2 minus dosage of ref. So if you have 2 alt alleles, 2-2 = 0 and there will be nothing added for this SNP.&#xA;                num_additions += 1&#xA;            elif effect_allele == vcf_row[4]: # Effect allele matches alternate allele&#xA;                running_total += dosage * effect_weight&#xA;                num_additions += 1&#xA;            else:&#xA;                nomatch = nomatch + 1&#xA;&#xA;# Calculate final output score&#xA;final_output_score = running_total / num_additions&#xA;percent_matched = (num_additions)/((nomatch)+(num_additions))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You shouldn't invert the sign of the &amp;quot;effect_weight&amp;quot; column for these 205 cases where the &amp;quot;other_allele&amp;quot; matches the reference. Not having the effect allele doesn't add the opposite beta to the score, it simply doesn't change it.&lt;/p&gt;&#xA;&lt;p&gt;Try this:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Do you still see those 205 cases if you use a genome build matching the PGS file? (I.e., GRCh37.)&lt;/li&gt;&#xA;&lt;li&gt;Do you still see those 205 cases if you use the LiftOver PGS scoring file with hg38 coordinates? (Download it &lt;a href=&quot;https://ftp.ebi.ac.uk/pub/databases/spot/pgs/scores/PGS002723/ScoringFiles/Harmonized/PGS002723_hmPOS_GRCh38.txt.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="17430" LastEditorUserId="17430" LastEditDate="2023-06-14T20:36:18.490" LastActivityDate="2023-06-14T20:36:18.490" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21180" PostTypeId="1" CreationDate="2023-06-14T19:57:46.260" Score="1" ViewCount="262" Body="&lt;p&gt;I am imputing human genome samples, all hg19, using the Michigan Imputation Server. The input of the data is one VCF per autosome, which each VCF containing a few dozen samples.&lt;/p&gt;&#xA;&lt;p&gt;The samples within the VCF are from highly heterogenous sources (various genotype arrays, from multiple labs). I have cleaned and standardized the data (removed indels, multiallelic sites, duplicates, and non-standard genotypes). I merged the samples into a single file for each chromosome. The VCF may not be left-aligned.&lt;/p&gt;&#xA;&lt;p&gt;It passes all QC with no issues, and then fails and gives no error message beyond &amp;quot;unfortunately, your job failed&amp;quot; and &amp;quot;Imputation on chromosome 14 failed. Imputation was stopped.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;As a test, I imputed chromosome 1, chromosome 13, and chromosome 14  by themselves. Chromosome 1 and 13 worked perfectly, but chromosome 14 failed imputation. I have no idea why chromosome 14 would be special or different. Also, imputing chromosomes 1-13 failed on chromosome 9, so it appears only certain chromosomes are causing issues.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have ideas as to why this might be occurring, or how to fix this issue?&lt;/p&gt;&#xA;&lt;p&gt;I dislike using hg19. I am using that because that is the build that some of the genome data I received is on. Labs are still using hg19 in the present day, for some reason. However, I have used hg19 successfully on the Michigan Imputation Server before, and it supports both hg19 and hg38, so I don't think build is the issue. I have a different batch of hg38 data that I will run through the server later.&lt;/p&gt;&#xA;&lt;p&gt;Here is a comparison between the QC statistics:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;Chromosome:&lt;/th&gt;&#xA;&lt;th&gt;13 (success)&lt;/th&gt;&#xA;&lt;th&gt;14 (failure)&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;SNPs&lt;/td&gt;&#xA;&lt;td&gt;75,675&lt;/td&gt;&#xA;&lt;td&gt;69,049&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Chunks&lt;/td&gt;&#xA;&lt;td&gt;6&lt;/td&gt;&#xA;&lt;td&gt;6&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Datatype&lt;/td&gt;&#xA;&lt;td&gt;unphased&lt;/td&gt;&#xA;&lt;td&gt;unphased&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Build&lt;/td&gt;&#xA;&lt;td&gt;hg19&lt;/td&gt;&#xA;&lt;td&gt;hg19&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Reference Panel&lt;/td&gt;&#xA;&lt;td&gt;apps@1000g-phase-3-v5 (hg19)&lt;/td&gt;&#xA;&lt;td&gt;apps@1000g-phase-3-v5 (hg19)&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Alternative allele frequency&lt;/td&gt;&#xA;&lt;td&gt;&amp;gt; 0.5 sites: 14,015&lt;/td&gt;&#xA;&lt;td&gt;&amp;gt; 0.5 sites: 11,776&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Reference Overlap&lt;/td&gt;&#xA;&lt;td&gt;99.18%&lt;/td&gt;&#xA;&lt;td&gt;99.25%&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Match&lt;/td&gt;&#xA;&lt;td&gt;44,261&lt;/td&gt;&#xA;&lt;td&gt;39,947&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Allele switch&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Strand flip&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Strand flip and allele switch&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;A/T, C/G genotypes&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Filter flag set&lt;/td&gt;&#xA;&lt;td&gt;367&lt;/td&gt;&#xA;&lt;td&gt;342&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Invalid alleles&lt;/td&gt;&#xA;&lt;td&gt;20,567&lt;/td&gt;&#xA;&lt;td&gt;18,022&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Multiallelic sites&lt;/td&gt;&#xA;&lt;td&gt;12&lt;/td&gt;&#xA;&lt;td&gt;16&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Duplicated sites&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;NonSNP sites&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Monomorphic sites&lt;/td&gt;&#xA;&lt;td&gt;9,963&lt;/td&gt;&#xA;&lt;td&gt;10,302&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Allele mismatch&lt;/td&gt;&#xA;&lt;td&gt;138&lt;/td&gt;&#xA;&lt;td&gt;116&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;SNPs call rate &amp;lt; 90%&lt;/td&gt;&#xA;&lt;td&gt;43,096&lt;/td&gt;&#xA;&lt;td&gt;38,760&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Excluded sites in total&lt;/td&gt;&#xA;&lt;td&gt;74,143&lt;/td&gt;&#xA;&lt;td&gt;67,558&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Remaining sites in total&lt;/td&gt;&#xA;&lt;td&gt;1,165&lt;/td&gt;&#xA;&lt;td&gt;1,187&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Typed only sites&lt;/td&gt;&#xA;&lt;td&gt;367&lt;/td&gt;&#xA;&lt;td&gt;304&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Execution status&lt;/td&gt;&#xA;&lt;td&gt;successful&lt;/td&gt;&#xA;&lt;td&gt;successful&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;Per the Michigan Imputation Server documentation, I've checked my input VCF files with CheckVCF.&lt;/p&gt;&#xA;&lt;p&gt;Here are the results on the chromosome VCF file that failed:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;---------------     REPORT     ---------------&#xA;Total [ 69095 ] lines processed&#xA;Examine [ 49 ] VCF header lines, [ 69046 ] variant sites, [ 21 ] samples&#xA;[ 0 ] duplicated sites&#xA;[ 0 ] NonSNP site are outputted to [ test.check.nonSnp ]&#xA;[ 0 ] Inconsistent reference sites are outputted to [ test.check.ref ]&#xA;[ 0 ] Variant sites with invalid genotypes are outputted to [ test.check.geno ]&#xA;[ 11947 ] Alternative allele frequency &amp;gt; 0.5 sites are outputted to [ test.check.af ]&#xA;[ 34408 ] Monomorphic sites are outputted to [ test.check.mono ]&#xA;---------------     ACTION ITEM     ---------------&#xA;* No error found by checkVCF.py, thank you for cleanning VCF file.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here are the results on the chromosome VCF file that succeeded:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;---------------     REPORT     ---------------&#xA;Total [ 75718 ] lines processed&#xA;Examine [ 49 ] VCF header lines, [ 75669 ] variant sites, [ 21 ] samples&#xA;[ 0 ] duplicated sites&#xA;[ 1 ] NonSNP site are outputted to [ test.check.nonSnp ]&#xA;[ 0 ] Inconsistent reference sites are outputted to [ test.check.ref ]&#xA;[ 0 ] Variant sites with invalid genotypes are outputted to [ test.check.geno ]&#xA;[ 14059 ] Alternative allele frequency &amp;gt; 0.5 sites are outputted to [ test.check.af ]&#xA;[ 37553 ] Monomorphic sites are outputted to [ test.check.mono ]&#xA;---------------     ACTION ITEM     ---------------&#xA;* No error found by checkVCF.py, thank you for cleanning VCF file.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17430" LastEditorUserId="3694" LastEditDate="2023-06-18T15:41:05.203" LastActivityDate="2023-06-18T15:41:05.203" Title="How do I fix chromosome-specific Michigan Imputation Server error (Imputation on chromosome 14 failed. Imputation was stopped)" Tags="|vcf|imputation|" AnswerCount="0" CommentCount="14" ContentLicense="CC BY-SA 4.0" />
  <row Id="21181" PostTypeId="2" ParentId="14937" CreationDate="2023-06-14T21:06:52.180" Score="3" Body="&lt;p&gt;Yes. LiftOver tools, if they work on VCF data (some only take BED as input) work on GWAS summary statistics. You could use&#xA;&lt;a href=&quot;https://pypi.org/project/liftover/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pypi.org/project/liftover/&lt;/a&gt;, for example.&lt;/p&gt;&#xA;" OwnerUserId="17430" LastActivityDate="2023-06-14T21:06:52.180" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21182" PostTypeId="2" ParentId="21161" CreationDate="2023-06-14T21:16:24.120" Score="0" Body="&lt;p&gt;See WormBase, which has a number of different resources for different nematodes:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://wormbase.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://wormbase.org&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-06-14T21:16:24.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21183" PostTypeId="2" ParentId="15839" CreationDate="2023-06-14T21:21:20.570" Score="1" Body="&lt;p&gt;The &amp;quot;OR/SD&amp;quot; metric is somewhat arbitrary. It's just convenient that it is 1 SD; there is no exact reason. If you wanted to make it OR per 2 SDs, you could just multiply the OR by 2.&lt;/p&gt;&#xA;&lt;p&gt;Imagine you have a distribution from 1st to 100th percentile of PRS, and you have a linear association between percentile of PRS and probability of disease. How would you quantify this? OR/SD is a quick and easily interpretable way to say how sensitive the PRS is. In other words, how &amp;quot;fast&amp;quot; the OR increases as you increase PRS.&lt;/p&gt;&#xA;&lt;p&gt;The method for obtaining this metric usually involves converting your raw scores into Z-scores. The slope of your regression where Y is OR and X is Z-score I think would correspond to OR/SD.&lt;/p&gt;&#xA;" OwnerUserId="17430" LastActivityDate="2023-06-14T21:21:20.570" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21184" PostTypeId="2" ParentId="19528" CreationDate="2023-06-14T21:26:03.903" Score="0" Body="&lt;p&gt;For the tools you mentioned, you should convert to BED format. Other tools will accept VCF. It should work either way, combined or not, though you may have to adjust your pipeline depending on which you prefer. I'd imagine a combined file would be easier to work with.&lt;/p&gt;&#xA;" OwnerUserId="17430" LastActivityDate="2023-06-14T21:26:03.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21185" PostTypeId="2" ParentId="14640" CreationDate="2023-06-14T21:29:29.550" Score="1" Body="&lt;p&gt;In a sample with all 40, determine the accuracy of the PRS. Then in the same sample, remove the 20 you are missing in the other set, and then determine the accuracy with the missing variants. This will give you an idea of the level of attenuation. However, you may not have access to such a dataset. In that case, I would recommend imputing your data before calculating a polygenic score. (It's likely that the GWAS the PRS is from used imputation itself.)&lt;/p&gt;&#xA;" OwnerUserId="17430" LastActivityDate="2023-06-14T21:29:29.550" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21186" PostTypeId="1" CreationDate="2023-06-15T14:10:04.610" Score="4" ViewCount="357" Body="&lt;p&gt;I am trying to write a nextflow script below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/*&#xA; * pipeline input parameters&#xA; */&#xA;params.reads = &amp;quot;/path/to/fq/files/&amp;quot;&#xA;params.guide_library = &amp;quot;/path/to/guidelibrary/&amp;quot;&#xA;params.outdir = &amp;quot;results&amp;quot;&#xA;&#xA;log.info &amp;quot;&amp;quot;&amp;quot;\&#xA;    DUAL GUIDE CRISPR QC - N F   P I P E L I N E&#xA;    ===================================&#xA;    guide library: &lt;span class=&quot;math-container&quot;&gt;${params.guide_library}&#xA;reads        : $&lt;/span&gt;{params.reads}&#xA;    outdir       : ${params.outdir}&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    .stripIndent(true)&#xA;&#xA;/*&#xA; * define the `index` process that creates a binary index&#xA; * given the transcriptome file&#xA; */&#xA;&#xA;&#xA;process READCOUNT {&#xA;    input:&#xA;    path guide_library&#xA;    tuple val(sample_name) path(reads)&#xA;&#xA;    output:&#xA;    path &amp;quot;$sample_name&amp;quot;&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    exactcounts.py -inputlibrary &lt;span class=&quot;math-container&quot;&gt;$guide_library -read1 $&lt;/span&gt;{reads[0]} -read2 &lt;span class=&quot;math-container&quot;&gt;${reads[1]} -prefix $&lt;/span&gt;sample_name&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;workflow {&#xA;    Channel&#xA;        .fromFilePairs(params.reads, checkIfExists: true)&#xA;        .set { read_pairs_ch }&#xA;   count_ch = READCOUNT(params.guide_library,read_pairs_ch)&#xA;    &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But am getting the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Process 'READCOUNT' has been already used -- If you need to reuse the same component, include it with a different name or include it in a different workflow context&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17657" LastActivityDate="2023-06-15T17:09:24.247" Title="Nextflow - Process has already been used" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21187" PostTypeId="2" ParentId="21186" CreationDate="2023-06-15T15:47:31.360" Score="5" Body="&lt;p&gt;I believe adding a comma after &lt;code&gt;val(sample_name)&lt;/code&gt; on line &lt;code&gt;tuple val(sample_name) path(reads)&lt;/code&gt; will solve the issue.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: Also documented here: &lt;a href=&quot;https://github.com/nextflow-io/nextflow/discussions/2714&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nextflow-io/nextflow/discussions/2714&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2023-06-15T17:09:24.247" LastActivityDate="2023-06-15T17:09:24.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21188" PostTypeId="2" ParentId="21158" CreationDate="2023-06-15T17:34:14.410" Score="1" Body="&lt;p&gt;Based on conversation in the comments.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;CLR is intended for compositional data. Distance data are not compositional, and compositions are (generally) not symmetric. So you should perform your normalization differently.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;If you want to transform your data for normalization purposes on the features themselves (in your case, the organismal relative abundances), you should do it on the compositional input from which you generate your distances. This is the intended application for CLR. For a reference on normalization approaches for compositional data (including CLR), see &lt;a href=&quot;https://www.nature.com/articles/s41522-020-00160-w&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;If you want to transform your distance matrix (e.g. the relationships between samples, irrespective of the underlying relative abundances), then you should use &lt;a href=&quot;https://forum.qiime2.org/t/transforms-for-correcting-negative-eigenvalues-for-pcoa-on-semi-metric-distances/17863&quot; rel=&quot;nofollow noreferrer&quot;&gt;one of the recommended methods&lt;/a&gt; for transforming distance data.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Note that the applications are very different. Application (1) is transforming the underlying organismal frequency data, such that all of the relative abundances are on the same scale. This will make all downstream analyses more stable / interpretable, including ordination analysis.&lt;/p&gt;&#xA;&lt;p&gt;Application (2) is saying basically, &amp;quot;my ordination / PCoA on my set of samples &lt;em&gt;still&lt;/em&gt; looks weird, even after all that normalization I did on my relative abundances. I want to fix it.&amp;quot; Examples of &amp;quot;looks weird&amp;quot; given include negative eigenvalues.&lt;/p&gt;&#xA;&lt;p&gt;You can't substitute (2) for (1). They are working with different data and have different consequences, though both can lead similar issues in your outputs and ordination plots&lt;/p&gt;&#xA;&lt;p&gt;It seems like your fundamental problem is that qiime2 outputs are difficult to work with. If that is the case, I would suggest asking a separate question about how to handle qiime2 outputs (or contacting the qiime2 authors about how they do compositional normalization).&lt;/p&gt;&#xA;&lt;p&gt;It looks like the qiime2 forum &lt;a href=&quot;https://forum.qiime2.org/t/normalization-of-feature-table/13184/5&quot; rel=&quot;nofollow noreferrer&quot;&gt;already has questions about the right way to apply CLR or related transformations in their workflow&lt;/a&gt;. &lt;a href=&quot;https://forum.qiime2.org/t/normalisation-of-features/10715/3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is another one&lt;/a&gt;. I would suggest following those recommendations and spending some quality time with the tutorials and user forum.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-06-15T17:34:14.410" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21189" PostTypeId="2" ParentId="16096" CreationDate="2023-06-16T09:55:58.470" Score="4" Body="&lt;p&gt;This proved to be so simple that it took a while to discover: Just use colnames.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;x = readVcf(&amp;quot;in.vcf&amp;quot;)&#xA;colnames(x) &amp;lt;- c(&amp;quot;new1&amp;quot;, &amp;quot;new2&amp;quot;)&#xA;# At this point, colnames(x) and samples(header(x)) will show different names&#xA;&#xA;# ... but if you write it out, it should use the colnames for both:&#xA;writeVcf(x, &amp;quot;out.vcf&amp;quot;)&#xA;y = readVcf(&amp;quot;out.vcf&amp;quot;)&#xA;&#xA;# samples(header(y)) and colnames(y) should both be &amp;quot;new1&amp;quot;,&amp;quot;new2&amp;quot; here&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17661" LastEditorUserId="650" LastEditDate="2023-06-16T14:11:35.043" LastActivityDate="2023-06-16T14:11:35.043" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21190" PostTypeId="1" CreationDate="2023-06-16T09:57:24.130" Score="2" ViewCount="51" Body="&lt;p&gt;I want to run a simulation with COBRApy for my bacterial Genome Scale Metabolic Model (GEM). I want it to run on different concentration of medium composition such as 0% NaCl, 10% NaCl, and 20% NaCl.&lt;/p&gt;&#xA;&lt;p&gt;How to implement this type of simulation in COBRApy?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="14560" LastActivityDate="2023-06-18T10:25:29.120" Title="Running Genome Scale Metabolic Model but with different concentration of medium composition" Tags="|genome|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21191" PostTypeId="1" CreationDate="2023-06-16T17:51:43.243" Score="2" ViewCount="113" Body="&lt;p&gt;I recently performed single cell RNA seq using core services and the web summary output from 10x that they have given shows that there are very few cells detected (100-300) cells per sample. Once this same issue happened in 1 out of 3 samples and this time it happened with 4 out of 4 samples.&lt;/p&gt;&#xA;&lt;p&gt;What might be the issue? The core says the cell preparation was not good but the cells were sorted for viability and then used to make the library. Any help?&lt;/p&gt;&#xA;&lt;p&gt;Full Summary snap:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ywJQT.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ywJQT.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have uploaded the full html file here : &lt;a href=&quot;https://drive.google.com/file/d/11eQduL4OC4kjC0BfhGp3Nhoci69bRAy6/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://drive.google.com/file/d/11eQduL4OC4kjC0BfhGp3Nhoci69bRAy6/view?usp=sharing&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;For one sample, Cell ranger 7.0.1 gave 284 cells while 6.8.2 gave 2200 cells.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17664" LastEditorUserId="73" LastEditDate="2023-06-17T09:38:00.490" LastActivityDate="2023-07-18T08:03:05.357" Title="Cell Ranger returns too few cells!" Tags="|single-cell|cellranger|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21192" PostTypeId="1" AcceptedAnswerId="21201" CreationDate="2023-06-16T18:29:39.457" Score="2" ViewCount="178" Body="&lt;p&gt;I am running a blastp search:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;time ~/bin/ncbi-blast-2.14.0+-src/c++/ReleaseMT/bin/blastp -db nr -query X.fasta -out nr.X -num_descriptions 4000 -remote&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but I have 2 problems with the output:&lt;/p&gt;&#xA;&lt;p&gt;I. Under &lt;code&gt;Sequences producing significant alignments:&lt;/code&gt;,&#xA;the output is truncated, and I can't read the exact species name:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;XP_002906761.1 elongation factor 3, putative [Phytophthora infest...  848     0.0  &#xA;KAG3108573.1 Elongation factor 3 [Phytophthora idaei] &amp;gt;gb|KAG3151...  845     0.0  &#xA;KAG2772535.1 Elongation factor 3 [Phytophthora cactorum] &amp;gt;gb|KAG2...  845     0.0  &#xA;KAG6976665.1 hypothetical protein JG688_00001147 [Phytophthora al...  844     0.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;II.&lt;/p&gt;&#xA;&lt;p&gt;The above sequences do not show up on the detailed sequence alignments below.&lt;/p&gt;&#xA;&lt;p&gt;I have tried numerous variations of &amp;quot;outfmt&amp;quot; variations, but now am getting warnings that I'm using the system too much.  So far, output format &lt;code&gt;0&lt;/code&gt; is the best.&lt;/p&gt;&#xA;&lt;p&gt;How can I get the full names and not the truncations?&lt;/p&gt;&#xA;&lt;p&gt;Why aren't the above species showing up later on, and how can I get the full alignment to show?&lt;/p&gt;&#xA;" OwnerUserId="7115" LastEditorUserId="7115" LastEditDate="2023-06-16T21:03:09.460" LastActivityDate="2023-06-20T22:13:22.877" Title="How to get full alignment output from BLASTP?" Tags="|blast|blastp|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21193" PostTypeId="2" ParentId="21192" CreationDate="2023-06-16T23:09:55.073" Score="1" Body="&lt;p&gt;Hmmm, firstly can you attempt multi-threading on a remote command via &lt;code&gt; -num_threads 8&lt;/code&gt;...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastp -db nr -query X.fasta -out nr.X -num_threads 8 -num_descriptions 4000 -remote&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It's a worth a try. Otherwise it could likely be a function of the network access or remote server resources. I don't think this is a function of the local RAM. Therefore performing a remote blast to download relevant sections of the NCBI database, convert that into a local database and then perform local blast seems the best strategy.&lt;/p&gt;&#xA;&lt;p&gt;Multi-threading and &lt;code&gt;-outfmt&lt;/code&gt; work well with local blast.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastp -query query.fa -db test.db -outfmt 6 -max_target_seqs 1000 -num_threads 8 -out blasthits.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The alignment is a function of the output formats in the above example its &lt;code&gt;-outfmt 6&lt;/code&gt; that is not useful, but &lt;code&gt;1&lt;/code&gt; to &lt;code&gt;4&lt;/code&gt; are relevant as follows.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; -outfmt &amp;lt;String&amp;gt;&#xA;   alignment view options:&#xA;     0 = pairwise,&#xA;     1 = query-anchored showing identities,&#xA;     2 = query-anchored no identities,&#xA;     3 = flat query-anchored, show identities,&#xA;     4 = flat query-anchored, no identities,&#xA;     5 = XML Blast output,&#xA;     6 = tabular,&#xA;     7 = tabular with comment lines,&#xA;     8 = Text ASN.1,&#xA;     9 = Binary ASN.1,&#xA;    10 = Comma-separated values,&#xA;    11 = BLAST archive format (ASN.1) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Multi-threading/parallelisation is the way to overcome memory issues, but really effective only in local operations not remote operations. Again, I think there's too much traffic/ or too many resources requested on NCBI server - and thats where the memory error is occurring.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-06-16T23:19:12.110" LastActivityDate="2023-06-16T23:19:12.110" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21195" PostTypeId="1" AcceptedAnswerId="21198" CreationDate="2023-06-18T16:53:17.403" Score="3" ViewCount="85" Body="&lt;p&gt;I am trying to run a function from the ProtParam script in the Bio.SeqUtils package. The function is for calculating the instability index of a protein. The code I am using in my script with the pre-built function is the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# entry - user inputted string taking in amino acid letters&#xA;entry = e1.get()&#xA;prot_data = ProteinAnalysis(entry)&#xA;result = prot_data.instability_index()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I run the code, I get the following key error related to the instability_index function.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;File &amp;quot;/Genome_Toolkit/venv/lib/python3.9/site- packages/Bio/SeqUtils/ProtParam.py&amp;quot;, line 152, in instability_index&#xA;dipeptide_value = index[this][next]&#xA;KeyError: '\n'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The instability_index() function in the ProtParam script looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def instability_index(self):&#xA;    index = ProtParamData.DIWV&#xA;    score = 0.0&#xA;&#xA;    for i in range(self.length - 1):&#xA;        this, next = self.sequence[i: i + 2]&#xA;        dipeptide_value = index[this][next]&#xA;        score += dipeptide_value&#xA;&#xA;    return (10.0 / self.length) * score&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I looked at the script myself, specifically the line the error is referring to but couldn't pinpoint the issue. Any guidance is appreciated. Thanks!&lt;/p&gt;&#xA;&lt;p&gt;the output of &lt;code&gt;print(entry)&lt;/code&gt; and &lt;code&gt;print(repr(entry))&lt;/code&gt; looks like this, which is exactly what the entry was in the input:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MALWMRLLPLLALLALWGPDPAAAFVNQHLCGSHLVEALYLVCGERGFFYTPKTRREAEDLQVGQVELGG&#xA;GPGAGSLQPLALEGSLQKRGIVEQCCTSICSLYQLENYCN&#xA;'MALWMRLLPLLALLALWGPDPAAAFVNQHLCGSHLVEALYLVCGERGFFYTPKTRREAEDLQVGQVELGG\nGPGAGSLQPLALEGSLQKRGIVEQCCTSICSLYQLENYCN'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14422" LastEditorUserId="73" LastEditDate="2023-06-20T22:19:47.487" LastActivityDate="2023-06-20T22:19:47.487" Title="KeyError '/n' in Bio SeqUtils Package ProtParam Script" Tags="|proteins|errors|biopython|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21196" PostTypeId="1" CreationDate="2023-06-19T03:02:21.493" Score="1" ViewCount="253" Body="&lt;p&gt;&lt;em&gt;This question has also been asked on &lt;a href=&quot;https://www.biostars.org/p/9566771/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I wonder how can I join multiple large .tsv files (over 20 GB each) by using two identifiers (columns) which I used to do using the &lt;code&gt;full_join&lt;/code&gt; function in &lt;code&gt;dplyr&lt;/code&gt; package for comparatively smaller files. All files are in &lt;code&gt;bed format&lt;/code&gt;, where ref_f.bed will be mapped by the other files by col1 and col2.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;#merging the files&#xA;dflist &amp;lt;- list(ref_f, file_1, file_2, file_3,....file_n)&#xA;&#xA;Test_out &amp;lt;- dflist %&amp;gt;% reduce(full_join, by=c(&amp;quot;chr&amp;quot;, &amp;quot;start&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;head ref_f.bed&#xA;chr1 1 100 .&#xA;chr1 101 200 .&#xA;chr1 201 300 .&#xA;chr1 301 400 .&#xA;chr1 401 500 .&#xA;chr1 501 600 .&#xA;chr1 601 700 .&#xA;chr1 701 800 .&#xA;chr1 801 900 .&#xA;&#xA;head file_1.bed&#xA;chr1 10101 10200 10158 10177 Scerevisiae-UniPROBE-Tbf1.UP00299&#xA;chr1 10101 10200 10158 10177 Scerevisiae-UniPROBE-Tbf1.UP00299&#xA;chr1 10101 10200 10164 10183 Scerevisiae-UniPROBE-Tbf1.UP00299&#xA;chr1 10101 10200 10164 10183 Scerevisiae-UniPROBE-Tbf1.UP00299&#xA;chr1 10101 10200 10175 10194 Scerevisiae-UniPROBE-Tbf1.UP00299&#xA;chr1 10101 10200 10175 10194 Scerevisiae-UniPROBE-Tbf1.UP00299&#xA;chr1 10101 10200 10175 10194 Scerevisiae-UniPROBE-Tbf1.UP00299&#xA;chr1 10101 10200 10181 10200 Scerevisiae-UniPROBE-Tbf1.UP00299&#xA;chr1 10101 10200 10181 10200 Scerevisiae-UniPROBE-Tbf1.UP00299&#xA;   &#xA;head file_2.bed&#xA;chr1 10101 10200 10158 10177 Scerevisiae-UniPROBE-Tbf1.UP00299&#xA;chr1 10101 10200 10158 10177 Scerevisiae-UniPROBE-Tbf1.UP00299&#xA;chr1 10101 10200 10164 10183 Scerevisiae-UniPROBE-Tbf1.UP00299&#xA;chr1 10101 10200 10164 10183 Scerevisiae-UniPROBE-Tbf1.UP00299&#xA;chr1 10101 10200 10175 10194 Scerevisiae-UniPROBE-Tbf1.UP00299&#xA;chr1 10101 10200 10175 10194 Scerevisiae-UniPROBE-Tbf1.UP00299&#xA;chr1 10101 10200 10175 10194 Scerevisiae-UniPROBE-Tbf1.UP00299&#xA;chr1 10101 10200 10181 10200 Scerevisiae-UniPROBE-Tbf1.UP00299&#xA;chr1 10101 10200 10181 10200 Scerevisiae-UniPROBE-Tbf1.UP00299&#xA;&#xA;head file_3.bed&#xA;chr1 29101 29200 29111 29124 Hsapiens-HOCOMOCOv10-AP2D_HUMAN.H10MO.D&#xA;chr1 29101 29200 29132 29152 Scerevisiae-UniPROBE-Cha4.UP00300&#xA;chr1 29101 29200 29116 29133 Scerevisiae-UniPROBE-Ecm22.UP00305&#xA;chr1 29101 29200 29126 29143 Scerevisiae-UniPROBE-Hal9.UP00315&#xA;chr1 29101 29200 29126 29143 Scerevisiae-UniPROBE-Hal9.UP00315&#xA;chr1 29101 29200 29106 29125 Scerevisiae-UniPROBE-Rpn4.UP00272&#xA;chr1 29101 29200 29110 29130 Scerevisiae-UniPROBE-Rsc30.UP00279&#xA;chr1 29101 29200 29112 29132 Scerevisiae-UniPROBE-Rsc30.UP00279&#xA;chr1 29101 29200 29133 29152 Scerevisiae-UniPROBE-Stp2.UP00338&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;R needs to import all files before performing the task because the file size for more than 20 files surpasses 20 GB, making this task appear difficult for R. This is the first time I have encountered this kind of issue, and I previously used this method in R for merging smaller size files. Therefore, I'm seeking a better solution to this issue.&lt;/p&gt;&#xA;&lt;p&gt;It would be a great assistance if anyone kindly advised how to overcome this issue.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="12770" LastEditorUserId="73" LastEditDate="2023-06-19T22:50:18.817" LastActivityDate="2023-06-21T03:44:46.767" Title="How to join multiple large .tsv (&gt; 20 GB, bed file) files by two identifier?" Tags="|r|python|bash|bed|tsv|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21197" PostTypeId="2" ParentId="21196" CreationDate="2023-06-19T07:04:40.510" Score="4" Body="&lt;p&gt;If you are open to a GNU/Linux solution, the following should be a good start:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '{printf(&amp;quot;%s-%s %s %s %s\n&amp;quot;, &lt;span class=&quot;math-container&quot;&gt;$1, $&lt;/span&gt;2, &lt;span class=&quot;math-container&quot;&gt;$1, $&lt;/span&gt;2, &lt;span class=&quot;math-container&quot;&gt;$3, $&lt;/span&gt;4);}' ref_f.bed | LC_ALL=C sort -k 1 &amp;gt; sorted_modified_ref_f.txt&#xA;awk '{printf(&amp;quot;%s-%s %s %s %s %s %s %s\n&amp;quot;, &lt;span class=&quot;math-container&quot;&gt;$1, $&lt;/span&gt;2, &lt;span class=&quot;math-container&quot;&gt;$1, $&lt;/span&gt;2, &lt;span class=&quot;math-container&quot;&gt;$3, $&lt;/span&gt;4, &lt;span class=&quot;math-container&quot;&gt;$5, $&lt;/span&gt;6);}' file_1.bed | LC_ALL=C sort -k 1 &amp;gt; sorted_modified_file_1.txt&#xA;join sorted_modified_ref_f.txt sorted_modified_file_1.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;awk&lt;/code&gt; commands are creating a &amp;quot;first column&amp;quot; that is a combination of columns 1 and 2 and this is required because &lt;code&gt;join&lt;/code&gt; &amp;quot;joins&amp;quot; on a single column.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2023-06-20T15:07:37.557" LastActivityDate="2023-06-20T15:07:37.557" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21198" PostTypeId="2" ParentId="21195" CreationDate="2023-06-19T12:05:50.020" Score="3" Body="&lt;p&gt;When the user is pushing &lt;code&gt;enter&lt;/code&gt; on their keyboard, it is likely inserting a newline character (&lt;code&gt;\n&lt;/code&gt;) at the end of the user input &lt;code&gt;entry&lt;/code&gt; and causing the error you have included in your question. You can strip the &lt;code&gt;\n&lt;/code&gt; by using &lt;code&gt;entry.rstrip('\n')&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;# entry - user inputted string taking in amino acid letters&#xA;entry = e1.get()&#xA;entry = entry.rstrip('\n')&#xA;prot_data = ProteinAnalysis(entry)&#xA;result = prot_data.instability_index()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;From the comments where the &lt;code&gt;print(entry)&lt;/code&gt; was given as&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;code&gt;MALWMRLLPLLALLALWGPDPAAAFVNQHLCGSHLVEALYLVCGERGFFYTPKTRREAEDLQVGQVELGG GPGAGSLQPLALEGSLQKRGIVEQCCTSICSLYQLENYCN&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;and &lt;code&gt;print(repr(entry))&lt;/code&gt; is given as&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;code&gt;'MALWMRLLPLLALLALWGPDPAAAFVNQHLCGSHLVEALYLVCGERGFFYTPKTRREAEDLQVGQVELGG\nGPGAGSLQPLALEGSLQKRGIVEQCCTSICSLYQLENYCN'&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;the &lt;code&gt;\n&lt;/code&gt; character is not at the end of the line so &lt;code&gt;.rstrip('\n')&lt;/code&gt; cannot be used (it only works at the end of the string).&lt;/p&gt;&#xA;&lt;p&gt;Try instead &lt;code&gt;.replace('\n', '')&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;# entry - user inputted string taking in amino acid letters&#xA;entry = e1.get()&#xA;entry = entry.replace('\n', '')&#xA;prot_data = ProteinAnalysis(entry)&#xA;result = prot_data.instability_index()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which on my system (removing the &lt;code&gt;prot_data&lt;/code&gt; and &lt;code&gt;result&lt;/code&gt; lines) outputs without the &lt;code&gt;\n&lt;/code&gt; character and &lt;code&gt;print(repr(entry))&lt;/code&gt; is the same as &lt;code&gt;print(entry)&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16234" LastEditorUserId="16234" LastEditDate="2023-06-20T16:16:41.647" LastActivityDate="2023-06-20T16:16:41.647" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21199" PostTypeId="1" AcceptedAnswerId="21202" CreationDate="2023-06-19T12:56:26.513" Score="3" ViewCount="55" Body="&lt;p&gt;I have single cell data from two samples : normal and pathological and&#xA;I would like to track the progression of the cells (and find the genes that drive it) from normal to pathological.&lt;/p&gt;&#xA;&lt;p&gt;Upon first glance, it seems most reasonable to define the trajectory &lt;strong&gt;in advance&lt;/strong&gt; in such a way that it would have a starting point, an end point, and a straight route between these two.&#xA;Is it possible, and using what package if so?&lt;/p&gt;&#xA;&lt;p&gt;P.S. In contrast to the question above - I started with the &lt;em&gt;destiny&lt;/em&gt; Bioconductor package. However, it always yields three tips for the diffusion pseudotime, and I am not sure how to determine the starting and the end tip of the trajectory...&lt;/p&gt;&#xA;" OwnerUserId="6317" LastActivityDate="2023-06-19T21:20:11.630" Title="What trajectory analysis method allows to set the form of the trajectory?" Tags="|bioconductor|single-cell|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21201" PostTypeId="2" ParentId="21192" CreationDate="2023-06-19T19:28:41.957" Score="-2" Body="&lt;p&gt;The problem I had was solved by &lt;code&gt;num_alignments&lt;/code&gt;, which is more complete than the first section.  I do not think that the text truncation is solvable.  However, parsing of the latter half solved the problem.&lt;/p&gt;&#xA;&lt;p&gt;the correct way to get the most complete output is:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;time ~/bin/ncbi-blast-2.14.0+-src/c++/ReleaseMT/bin/blastp -db nr -query X.fasta -out nr.X.0 -outfmt 0 -num_descriptions 4000 -remote -num_alignments 4000&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="7115" LastEditorUserId="73" LastEditDate="2023-06-20T22:13:22.877" LastActivityDate="2023-06-20T22:13:22.877" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21202" PostTypeId="2" ParentId="21199" CreationDate="2023-06-19T21:20:11.630" Score="4" Body="&lt;p&gt;slingshot (Bioconductor) accepts a priori starts and ends. For differential expression along the trajectory you may check tradeSeq.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-06-19T21:20:11.630" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21203" PostTypeId="2" ParentId="21196" CreationDate="2023-06-19T23:20:38.723" Score="2" Body="&lt;p&gt;For joining / merging / intersecting bed files, you're better off using &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bedtools&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;It's unclear what you're trying to do with these steps, but I'm guessing it's something like the following:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The bedtools sub-commands include:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[ Genome arithmetic ]&#xA;    intersect     Find overlapping intervals in various ways.&#xA;    window        Find overlapping intervals within a window around an &amp;gt; interval.&#xA;    ...&#xA;    merge         Combine overlapping/nearby intervals into a single interval.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If you're interested in combining multiple lines of different files in R, have a look at the &lt;code&gt;map_dfr&lt;/code&gt; function (reference &lt;a href=&quot;https://purrr.tidyverse.org/reference/map_dfr.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;), which does a row-wise concatenation of the results of a function mapped to a list.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-06-19T23:20:38.723" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21204" PostTypeId="1" CreationDate="2023-06-20T11:28:08.007" Score="0" ViewCount="39" Body="&lt;p&gt;I built the matrix as the gmt example in &lt;a href=&quot;https://docs.gsea-msigdb.org/#GSEA/Data_Formats/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.gsea-msigdb.org/#GSEA/Data_Formats/&lt;/a&gt; but It still has errors when I try to run it.&#xA;When I try the gene set from &lt;a href=&quot;https://www.gsea-msigdb.org/gsea/msigdb/human/geneset/HALLMARK_ADIPOGENESIS.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.gsea-msigdb.org/gsea/msigdb/human/geneset/HALLMARK_ADIPOGENESIS.html&lt;/a&gt;&#xA;in .gmt format and load it with GSEA appears this error:&lt;/p&gt;&#xA;&lt;p&gt;Thank you so much&lt;/p&gt;&#xA;&lt;p&gt;Nerea&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;---- Full Error Message ----&#xA;col:34 &amp;gt; matrix's fColCnt:34&#xA;&#xA;---- Stack Trace ----&#xA;# of exceptions: 1&#xA;------col:34 &amp;gt; matrix's fColCnt:34------&#xA;java.lang.ArrayIndexOutOfBoundsException: col:34 &amp;gt; matrix's fColCnt:34&#xA;at org.gsea_msigdb.gsea/edu.mit.broad.genome.math.Matrix.getColumnV(Matrix.java:261)&#xA;at org.gsea_msigdb.gsea/edu.mit.broad.genome.objects.DefaultDataset.getColumn(DefaultDataset.java:289)&#xA;at org.gsea_msigdb.gsea/edu.mit.broad.genome.objects.TemplateFactory.extract(TemplateFactory.java:85)&#xA;at org.gsea_msigdb.gsea/edu.mit.broad.genome.alg.DatasetGenerators.extract(DatasetGenerators.java:292)&#xA;at org.gsea_msigdb.gsea/edu.mit.broad.genome.alg.DatasetGenerators.extract(DatasetGenerators.java:288)&#xA;at org.gsea_msigdb.gsea/xtools.gsea.AbstractGsea2Tool.execute_one(AbstractGsea2Tool.java:79)&#xA;at org.gsea_msigdb.gsea/xtools.gsea.AbstractGsea2Tool.execute_one_with_reporting(AbstractGsea2Tool.java:103)&#xA;at org.gsea_msigdb.gsea/xtools.gsea.Gsea.execute(Gsea.java:166)&#xA;at org.gsea_msigdb.gsea/edu.mit.broad.xbench.tui.TaskManager$ToolRunnable.run(TaskManager.java:391)&#xA;at java.base/java.lang.Thread.run(Unknown Source)&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17679" LastActivityDate="2023-06-20T11:28:08.007" Title="Run gmt format download from GSEA" Tags="|gsea|errors|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21205" PostTypeId="2" ParentId="21196" CreationDate="2023-06-21T03:39:15.283" Score="0" Body="&lt;p&gt;Thanks for all the responses, however, I got a relatable outcome from (&lt;a href=&quot;https://unix.stackexchange.com/questions/113898/how-to-merge-two-files-based-on-the-matching-of-two-columns&quot;&gt;https://unix.stackexchange.com/questions/113898/how-to-merge-two-files-based-on-the-matching-of-two-columns&lt;/a&gt;) by using&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;$ awk 'NR==FNR {h[$2] = $6; next} {print $1,$2,$3,h[$2]}' file_1 ref_f &amp;gt; out_1&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;For &lt;code&gt;file_1&lt;/code&gt; (&lt;code&gt;NR==FNR&lt;/code&gt; is only true for the first file argument). Save column 6 in hash-array using column 2 as key: &lt;code&gt;h[$2] = $6&lt;/code&gt;. Then for &lt;code&gt;ref_f&lt;/code&gt; and output all three columns &lt;code&gt;$1,$2,$3&lt;/code&gt;, appending the corresponding saved column from hash-array &lt;code&gt;h[$2]&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, it does not print &lt;code&gt;col4&lt;/code&gt; and &lt;code&gt;col5&lt;/code&gt; from &lt;code&gt;file_1&lt;/code&gt; and must be repeated for each subsequent file. I hope that the answer could be improved further.&lt;/p&gt;&#xA;" OwnerUserId="12770" LastEditorUserId="12770" LastEditDate="2023-06-21T03:44:46.767" LastActivityDate="2023-06-21T03:44:46.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21206" PostTypeId="1" AcceptedAnswerId="21207" CreationDate="2023-06-21T05:03:18.070" Score="0" ViewCount="50" Body="&lt;p&gt;I want to convert a .tsv file table in binary format (1 and 0).  In which all cells containing text would become &lt;code&gt;1&lt;/code&gt; and blank cells would become &lt;code&gt;0&lt;/code&gt;,  except first three columns.&lt;/p&gt;&#xA;&lt;p&gt;The given .tsv file has 6 fields&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head adrbvbeT_copy.tsv &#xA;chr1    1   100         &#xA;chr1    101 200         &#xA;chr1    201 300         &#xA;chr1    301 400         &#xA;chr1    401 500         &#xA;chr1    501 600         &#xA;chr1    601 700         &#xA;chr1    701 800         &#xA;chr1    801 900         &#xA;chr1    901 1000    &#xA;&#xA;tail adrbvbeT.tsv &#xA;chrY    57226501    57226600            &#xA;chrY    57226601    57226700    Mmusculus-jaspar2016-Tcf21-MA0832.1     &#xA;chrY    57226701    57226800    Mmusculus-HOCOMOCOv10-TF7L2_MOUSE.H10MO.A       &#xA;chrY    57226801    57226900    Mmusculus-HOCOMOCOv10-STAT1_MOUSE.H10MO.A       &#xA;chrY    57226901    57227000    Mmusculus-HOCOMOCOv10-ZEP2_MOUSE.H10MO.D        &#xA;chrY    57227001    57227100    NA-SwissRegulon-SMARCC2.SwissRegulon    Mmusculus-HOCOMOCOv10-PBX2_MOUSE.H10MO.C    &#xA;chrY    57227101    57227200    Mmusculus-HOCOMOCOv10-ZIC3_MOUSE.H10MO.C    Mmusculus-HOCOMOCOv10-TBX5_MOUSE.H10MO.D    Mmusculus-HOCOMOCOv10-PIT1_MOUSE.H10MO.C&#xA;chrY    57227201    57227300    Mmusculus-HOCOMOCOv10-MAFB_MOUSE.H10MO.D    Mmusculus-UniPROBE-Tbp.UP00029  Mmusculus-HOCOMOCOv10-MAFB_MOUSE.H10MO.D&#xA;chrY    57227301    57227400    Mmusculus-HOCOMOCOv10-PITX2_MOUSE.H10MO.D   Hsapiens-jaspar2016-TGIF2-MA0797.1  Mmusculus-HOCOMOCOv10-PITX2_MOUSE.H10MO.D&#xA;chrY    57227401    57227415    Mmusculus-HOCOMOCOv10-TFDP1_MOUSE.H10MO.D   Mmusculus-HOCOMOCOv10-TEAD4_MOUSE.H10MO.D   Mmusculus-HOCOMOCOv10-ZBTB6_MOUSE.H10MO.D&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;First, I converted all blank cells to &lt;code&gt;0&lt;/code&gt;. Now I'm wondering how I can replace the text in the cells with &lt;code&gt;1&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN { FS = OFS = &amp;quot;\t&amp;quot; } { for(i=1; i&amp;lt;=NF; i++) if(&lt;span class=&quot;math-container&quot;&gt;$i ~ /^ *$&lt;/span&gt;/) $i = 0 }; 1' adrbvbeT_copy.tsv &amp;gt; adrbvbeT_copyN0.tsv&#xA;&#xA;head adrbvbeT_copyN0.tsv &#xA;chr1    1   100 0   0   0&#xA;chr1    101 200 0   0   0&#xA;chr1    201 300 0   0   0&#xA;chr1    301 400 0   0   0&#xA;chr1    401 500 0   0   0&#xA;chr1    501 600 0   0   0&#xA;chr1    601 700 0   0   0&#xA;chr1    701 800 0   0   0&#xA;chr1    801 900 0   0   0&#xA;chr1    901 1000    0   0   0&#xA;&#xA;&#xA;tail adrbvbeT_copyN0.tsv &#xA;chrY    57226501    57226600    0   0   0&#xA;chrY    57226601    57226700    Mmusculus-jaspar2016-Tcf21-MA0832.1 0   0&#xA;chrY    57226701    57226800    Mmusculus-HOCOMOCOv10-TF7L2_MOUSE.H10MO.A   0   0&#xA;chrY    57226801    57226900    Mmusculus-HOCOMOCOv10-STAT1_MOUSE.H10MO.A   0   0&#xA;chrY    57226901    57227000    Mmusculus-HOCOMOCOv10-ZEP2_MOUSE.H10MO.D    0   0&#xA;chrY    57227001    57227100    NA-SwissRegulon-SMARCC2.SwissRegulon    Mmusculus-HOCOMOCOv10-PBX2_MOUSE.H10MO.C    0&#xA;chrY    57227101    57227200    Mmusculus-HOCOMOCOv10-ZIC3_MOUSE.H10MO.C    Mmusculus-HOCOMOCOv10-TBX5_MOUSE.H10MO.D    Mmusculus-HOCOMOCOv10-PIT1_MOUSE.H10MO.C&#xA;chrY    57227201    57227300    Mmusculus-HOCOMOCOv10-MAFB_MOUSE.H10MO.D    Mmusculus-UniPROBE-Tbp.UP00029  Mmusculus-HOCOMOCOv10-MAFB_MOUSE.H10MO.D&#xA;chrY    57227301    57227400    Mmusculus-HOCOMOCOv10-PITX2_MOUSE.H10MO.D   Hsapiens-jaspar2016-TGIF2-MA0797.1  Mmusculus-HOCOMOCOv10-PITX2_MOUSE.H10MO.D&#xA;chrY    57227401    57227415    Mmusculus-HOCOMOCOv10-TFDP1_MOUSE.H10MO.D   Mmusculus-HOCOMOCOv10-TEAD4_MOUSE.H10MO.D   Mmusculus-HOCOMOCOv10-ZBTB6_MOUSE.H10MO.D&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12770" LastEditorUserId="12770" LastEditDate="2023-06-21T05:09:45.163" LastActivityDate="2023-06-21T14:11:12.553" Title="How can I change all text in a .tsv file to 1, except the first three fields?" Tags="|bash|bed|awk|text-processing|command-line|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21207" PostTypeId="2" ParentId="21206" CreationDate="2023-06-21T05:46:56.003" Score="3" Body="&lt;p&gt;Not an expert, but :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN { FS = OFS = &amp;quot;\t&amp;quot; } { for(i=4; i&amp;lt;=6; i++) if(&lt;span class=&quot;math-container&quot;&gt;$i ~ /^ *$&lt;/span&gt;/) {&lt;span class=&quot;math-container&quot;&gt;$i = 0} else {$&lt;/span&gt;i = 1} }; 1'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Since you have a very specific file format, this code starts from field 4, checking whether the fields after it are empty or not. I assume also that know that you file has 6 fields max exactly, so I &amp;quot;hardcoded&amp;quot; this knowledge.&lt;/p&gt;&#xA;" OwnerUserId="6317" LastActivityDate="2023-06-21T05:46:56.003" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21210" PostTypeId="1" AcceptedAnswerId="21211" CreationDate="2023-06-21T17:43:19.913" Score="-2" ViewCount="57" Body="&lt;p&gt;I have data frame each row is a pathway and it involved genes like&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Z7BC5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Z7BC5.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; head(h)&#xA;    Release.of.Cancer.Cell.Antigens.1  ARID1A    ATM  BIRC3   BLM BNIP3   BRCA1   BRCA2 BRD3  BRD4 BRIP1 CCNA1 CCND1&#xA;V1        Cancer Antigen Presentation    ARG2   ATF3    B2M BATF3   BLK   CCL22 CCL3/L1 CCL4  CCL5  CCR5  CD14  CD19&#xA;V2      T-cell Priming and Activation ADORA2A  BATF3   BTLA  CCL2 CCL22 CCL3/L1    CCL4 CCL5  CCR2  CCR5  CD1C   CD2&#xA;V3 Immune Cell Localization to Tumors     A2M ADGRE1 ANGPT2  APOE  ATF3     AXL   BATF3  BLK CCL13 CCL14 CCL18 CCL19&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to convert that like&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gene.sets &amp;lt;- list(Release.of.Cancer.Cell.Antigens = c(&amp;quot;ARID1A&amp;quot;, &amp;quot;ATM&amp;quot;,  &amp;quot;BIRC3&amp;quot;,    &amp;quot;BLM&amp;quot;),&#xA;            Cancer_Antigen_Presentation = c(&amp;quot;ARG2&amp;quot;,&amp;quot;ATF3&amp;quot;,&amp;quot;B2M&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have not done anything so far as I am not certain what exactly I should google for.&lt;/p&gt;&#xA;&lt;p&gt;By code the user kindly provided I get this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gene.sets = lapply(1:nrow(h), function(x) unname(unlist(h[x,-1])))&#xA;names(gene.sets) = h[,1]&#xA;&#xA;[91] &amp;quot;IGF2R&amp;quot;     &amp;quot;IHH&amp;quot;       &amp;quot;IL11&amp;quot;      &amp;quot;IL11RA&amp;quot;    &amp;quot;IL12RB2&amp;quot;   &amp;quot;IL15&amp;quot;      &amp;quot;IL17A&amp;quot;     &amp;quot;IL1A&amp;quot;      &amp;quot;IL1B&amp;quot;     &#xA;[100] &amp;quot;IL22RA1&amp;quot;   &amp;quot;IL2RA&amp;quot;     &amp;quot;IL2RB&amp;quot;     &amp;quot;IL6&amp;quot;       &amp;quot;IL7R&amp;quot;      &amp;quot;IRF1&amp;quot;      &amp;quot;IRF2&amp;quot;      &amp;quot;IRF3&amp;quot;      &amp;quot;IRF4&amp;quot;     &#xA;[109] &amp;quot;IRF7&amp;quot;      &amp;quot;IRF9&amp;quot;      &amp;quot;ISG15&amp;quot;     &amp;quot;ITGAE&amp;quot;     &amp;quot;JAK1&amp;quot;      &amp;quot;JAK2&amp;quot;      &amp;quot;JAK3&amp;quot;      &amp;quot;KIT&amp;quot;       &amp;quot;LAG3&amp;quot;     &#xA;[118] &amp;quot;LAIR1&amp;quot;     &amp;quot;LIF&amp;quot;       &amp;quot;LTB&amp;quot;       &amp;quot;MX1&amp;quot;       &amp;quot;NCAM1&amp;quot;     &amp;quot;NKG7&amp;quot;      &amp;quot;OAS1&amp;quot;      &amp;quot;OAS2&amp;quot;      &amp;quot;OAS3&amp;quot;     &#xA;[127] &amp;quot;PDCD1&amp;quot;     &amp;quot;PDCD1LG2&amp;quot;  &amp;quot;PF4&amp;quot;       &amp;quot;PIK3CA&amp;quot;    &amp;quot;PIK3CG&amp;quot;    &amp;quot;PIK3R5&amp;quot;    &amp;quot;PRF1&amp;quot;      &amp;quot;PRLR&amp;quot;      &amp;quot;PROM1&amp;quot;    &#xA;[136] &amp;quot;PSMB10&amp;quot;    &amp;quot;PSMB5&amp;quot;     &amp;quot;PSMB9&amp;quot;     &amp;quot;PTEN&amp;quot;      &amp;quot;REN&amp;quot;       &amp;quot;RORC&amp;quot;      &amp;quot;RSAD2&amp;quot;     &amp;quot;SELL&amp;quot;      &amp;quot;SELP&amp;quot;     &#xA;[145] &amp;quot;SIRPA&amp;quot;     &amp;quot;SOX11&amp;quot;     &amp;quot;SOX2&amp;quot;      &amp;quot;SPRY4&amp;quot;     &amp;quot;STAT1&amp;quot;     &amp;quot;STAT2&amp;quot;     &amp;quot;STAT4&amp;quot;     &amp;quot;TBX21&amp;quot;     &amp;quot;TIGIT&amp;quot;    &#xA;[154] &amp;quot;TNF&amp;quot;       &amp;quot;TNFAIP3&amp;quot;   &amp;quot;TNFRSF10B&amp;quot; &amp;quot;TNFRSF10C&amp;quot; &amp;quot;TNFRSF11A&amp;quot; &amp;quot;TNFRSF11B&amp;quot; &amp;quot;TNFRSF14&amp;quot;  &amp;quot;TNFRSF17&amp;quot;  &amp;quot;TNFRSF18&amp;quot; &#xA;[163] &amp;quot;TNFRSF1A&amp;quot;  &amp;quot;TNFRSF1B&amp;quot;  &amp;quot;TNFRSF4&amp;quot;   &amp;quot;TNFRSF8&amp;quot;   &amp;quot;TNFRSF9&amp;quot;   &amp;quot;TNFSF10&amp;quot;   &amp;quot;TNFSF12&amp;quot;   &amp;quot;TNFSF13&amp;quot;   &amp;quot;TNFSF13B&amp;quot; &#xA;[172] &amp;quot;TNFSF18&amp;quot;   &amp;quot;TNFSF4&amp;quot;    &amp;quot;TNFSF8&amp;quot;    &amp;quot;TPSAB1/B2&amp;quot; &amp;quot;TRAF1&amp;quot;     &amp;quot;TSLP&amp;quot;      &amp;quot;VEGFA&amp;quot;     &amp;quot;WNT7B&amp;quot;     &amp;quot;&amp;quot;         &#xA;[208] &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;         &#xA;[217] &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;         &#xA;[226] &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;         &#xA;[235] &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;         &#xA;[244] &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;         &#xA;[253] &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;          &amp;quot;&amp;quot;    &#xA;&#xA;x=lapply(gene.sets, function(z){ z[!is.na(z) &amp;amp; z != &amp;quot;&amp;quot;]})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="15519" LastEditDate="2023-06-27T02:01:58.970" LastActivityDate="2023-06-27T02:01:58.970" Title="Converting a data frame to multiple lists" Tags="|r|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21211" PostTypeId="2" ParentId="21210" CreationDate="2023-06-21T18:09:46.767" Score="2" Body="&lt;p&gt;I don't know whether this will actually work since you didn't provide a reproducible example...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gene.sets = lapply(1:nrow(h), function(x) unname(unlist(h[x,-1])))&#xA;names(gene.sets) = h[,1]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2023-06-21T18:09:46.767" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21212" PostTypeId="1" CreationDate="2023-06-22T13:57:15.207" Score="1" ViewCount="337" Body="&lt;p&gt;I want to analyse the single-cell ATAC-seq data from 10X downloaded from GSE158398. The files attached to this dataset are &lt;code&gt;matrix.mtx.gz&lt;/code&gt;, &lt;code&gt;fragments.tsv.gz&lt;/code&gt;, and &lt;code&gt;barcodes.tsv.gz&lt;/code&gt; files, in addition to a &lt;code&gt;peaks.bed.gz&lt;/code&gt; file.&lt;/p&gt;&#xA;&lt;p&gt;How do I load this into R and make a Seurat object? Normally there is a matrix, barcodes and features files as output from scATAC-seq 10X data, so how do I handle only having a fragments file instead?&lt;/p&gt;&#xA;&lt;p&gt;Thank you very much for your help with this!&lt;/p&gt;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&lt;p&gt;This is the code I would normally run to load the ATAC counts (matrix), barcodes and features and make the Seurat object. However, with this dataset, I do not have the features.tsv file, but a fragments.tsv file instead and so I am unsure how to change my code to accommodate this change in input data.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;ATAC_dir &amp;lt;- &amp;quot;~/GSE169246_PDL1/ATAC/&amp;quot;&#xA;&#xA;barcode_path_ATAC &amp;lt;- paste0(ATAC_dir, &amp;quot;barcodes.tsv.gz&amp;quot;)&#xA;features_path_ATAC &amp;lt;- paste0(ATAC_dir, &amp;quot;features.tsv.gz&amp;quot;)&#xA;matrix_path_ATAC &amp;lt;- paste0(ATAC_dir, &amp;quot;matrix.mtx.gz&amp;quot;)&#xA;&#xA;barcode_names_ATAC &amp;lt;- read.delim(barcode_path_ATAC, &#xA;                            header = FALSE, &#xA;                            stringsAsFactors = FALSE)$V1&#xA;&#xA;feature_names_ATAC &amp;lt;- read.delim(features_path_ATAC, &#xA;                                 header = FALSE, &#xA;                                 stringsAsFactors = FALSE)$V1&#xA;&#xA;mat_ATAC &amp;lt;- readMM(file = matrix_path_ATAC)&#xA;&#xA;colnames(mat_ATAC) &amp;lt;- barcode_names_ATAC&#xA;rownames(mat_ATAC) &amp;lt;- feature_names_ATAC&#xA;&#xA;&#xA;chrom_assay &amp;lt;- CreateChromatinAssay(&#xA;  counts = mat_ATAC,&#xA;  sep = c(&amp;quot;:&amp;quot;, &amp;quot;-&amp;quot;),&#xA;  genome = 'hg38',&#xA;  fragments = &amp;quot;peaks.bed.gz&amp;quot;,&#xA;  min.cells = 10,&#xA;  min.features = 200&#xA;)&#xA;&#xA;seurat_ATAC &amp;lt;- CreateSeuratObject(&#xA;  counts = chrom_assay,&#xA;  assay = &amp;quot;peaks&amp;quot;&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17686" LastEditorUserId="650" LastEditDate="2023-06-23T15:00:59.903" LastActivityDate="2023-06-23T15:00:59.903" Title="Loading scATAC-seq data containing matrix.mtx, fragments.tsv and barcodes.tsv files from GEO into R?" Tags="|seurat|single-cell|10x-genomics|atac-seq|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21213" PostTypeId="2" ParentId="21171" CreationDate="2023-06-22T15:47:23.227" Score="2" Body="&lt;p&gt;I believe I understand the project, to obtain a biochemical/metabolic profile of a large number of poorly annotated plant genomes. There isn't a perfect solution, like Prokka for bacteria. It is clearly an area subject to recent development and interest.&lt;/p&gt;&#xA;&lt;p&gt;I would recommend using Gene Ontology Meta Annotator for Plants (GOMAP) which is described in &amp;quot;&lt;a href=&quot;https://plantmethods.biomedcentral.com/articles/10.1186/s13007-021-00754-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Plant Methods&lt;/a&gt;&amp;quot; journal.&lt;/p&gt;&#xA;&lt;p&gt;There are limitations:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;KEGG is not used (below);&lt;/li&gt;&#xA;&lt;li&gt;its a singularity container - and thats its only distribution.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Learning Singularlity&lt;/strong&gt; You will need to learn running the container through Singularity and that will be limiting if you are not familiar with containers. However, this is the best of the available &amp;quot;out of the box&amp;quot; solutions I can see in plants. A Singularity HPC is a good call here, tricky if you don't have access to one. I know Docker, Singularity is similar but different.  The information on Singularity is &lt;a href=&quot;https://docs.sylabs.io/guides/3.0/user-guide/quick_start.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I would start reading from &amp;quot;Executing Commands&amp;quot;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;singularity pull docker://godlovedc/lolcow # get the container &#xA;singularity exec lolcow_latest.sif cowsay moo # run it&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(the cow is a famous teaching tool for containers)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;KEGG&lt;/strong&gt; One approach that can be used is to perform COG (NCBI/Genbank code) and KEGG no. identification (e.g. eggNOG). Tools such as eggNOG are setup for metagenomics data however. The COG/KEGG no. output is only the starting point, interrogating each of these to assign gene function and metabolic pathways will require a KEGG database and custom code. Unless someone has published, e.g. KEGG database in SQL.&lt;/p&gt;&#xA;&lt;p&gt;In my experience the Kyoto Encyclopaedia of Genes and Genomes (KEGG) is the best singular functional database available, thats why I see GOMAPs exclusion of this as a limitation. COG doesn't provide a particularly useful information functional description IMO. eggNOG, or any other method to identify COGs  and KEGG numbers, interrogated against the KEGG database works for plants very well. I know this because I'm not focused on plants but often recover plant  defence mechanisms genes from KEGG. Basically, KEGG identifies &amp;gt;&amp;gt; half of all genes with known COG nomenclature and notably higher than other functional databases.Thus I see GOMAPs exclusion of KEGG as a limitation. However, to reiterate to go identify COG/KEGG -&amp;gt; KEGG database interrogation and analysis is a lot of custom code.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Databases&lt;/strong&gt; GOMAP implements &lt;em&gt;Arabidopsis&lt;/em&gt; annotation, which may be cool and provides a starting point. It's also running UniProt, UniProt-plants, interpro, GO databases and those &lt;strong&gt;+&lt;/strong&gt; &lt;em&gt;Arabidopsis&lt;/em&gt; annotated DB could be really cool. It definitely sounds promising.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Processor time&lt;/strong&gt; Any method of gene identification takes AGES and I have no doubt that GOMAP will be REALLY slow, looking at the pipeline it's using - there are a lot of databases its using. This is GOMAPs justification for using Singularity, it going to take ages to run therefore containerise it in a format often used by modern HPCs. To be honest it's a bit lazy, especially as it was published in 2021 (should be uploaded e.g. to conda). The authors talk about Python and R ... hmmmm, it looks to be a MySQL backend, thats where its speed will come from. The container will be large BTW hopefully is &amp;lt;1GB, but it will be &amp;gt;200MB.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; &lt;a href=&quot;https://docs.sylabs.io/guides/3.0/user-guide/quick_start.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Its worth learning Singularity&lt;/a&gt; just keep to the basic commands to run GOMAP. You will need access to a large machine and the output will be okay but they're better &lt;em&gt;general&lt;/em&gt; databases. However, I don't know how useful the &lt;em&gt;Arabidopsis&lt;/em&gt; database it - thats about the biology, i.e. what you are looking for and whether &lt;em&gt;Arabidopsis&lt;/em&gt; is a good model of that investigation&lt;/p&gt;&#xA;&lt;p&gt;Note, Springer Nature refused a link image request for the container overview, but the image I wanted to share is &lt;a href=&quot;https://media.springernature.com/full/springer-static/image/art%3A10.1186%2Fs13007-021-00754-1/MediaObjects/13007_2021_754_Fig1_HTML.png?as=webp&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-06-22T15:54:25.043" LastActivityDate="2023-06-22T15:54:25.043" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21214" PostTypeId="1" CreationDate="2023-06-22T18:29:13.500" Score="0" ViewCount="76" Body="&lt;p&gt;As far as I know, we as humans share 99.9 similarity in our genomes. Further, we inherit 50% of the genome from our parents. So, it is correct to say that parent children have a 99.95% similarity in their genome?&lt;/p&gt;&#xA;" OwnerUserId="17688" LastEditorUserId="3694" LastEditDate="2023-08-30T07:22:58.580" LastActivityDate="2023-08-30T07:22:58.580" Title="Human genome similarities" Tags="|genome|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21215" PostTypeId="2" ParentId="21214" CreationDate="2023-06-22T19:01:06.657" Score="2" Body="&lt;p&gt;The answer is no if the question is read at face value,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I don't know the metric used (is this just SNPs?), nor which part of the genome is included in the calculation, just the exome?&lt;/li&gt;&#xA;&lt;li&gt;Moreover, if the genetic identity between parents is unknown, then the similarity of their offspring to either parent cannot be calculated.&lt;/li&gt;&#xA;&lt;li&gt;Your calculation excludes &lt;em&gt;de novo&lt;/em&gt; mutations, particular at neutral sites and copy number variation, e.g. microsatellites, however its not clear whether the data is simply the exome.&lt;/li&gt;&#xA;&lt;li&gt;Homologous recombination is excluded, i.e. particularly occurring in the parental germ-line, there must be stochastic effects even in the first generation.&lt;/li&gt;&#xA;&lt;li&gt;Finally, mtDNA is maternally inherited.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-06-23T02:11:49.320" LastActivityDate="2023-06-23T02:11:49.320" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21216" PostTypeId="1" AcceptedAnswerId="21217" CreationDate="2023-06-22T22:58:44.813" Score="1" ViewCount="52" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Rv8MG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Rv8MG.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;Hi all, I am checking if a gene which around 50k base pairs has mutation. So is that a variant from A to G? Would you please tell me how to do so? Thank you so much!&lt;/p&gt;&#xA;&lt;p&gt;The reference sequence from GRCh38, sequenced using Illumina technology.&lt;/p&gt;&#xA;" OwnerUserId="17585" LastEditorUserId="73" LastEditDate="2023-06-23T04:49:34.023" LastActivityDate="2023-06-23T04:49:34.023" Title="Check mutation of a gene" Tags="|sam|vcf|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21217" PostTypeId="2" ParentId="21216" CreationDate="2023-06-22T23:12:42.833" Score="1" Body="&lt;p&gt;Assuming this is a diploid genome, this looks like it could be a heterozygous variant (i.e. A is present on one chromosome, and G is present on the other chromosome).&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-06-22T23:12:42.833" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21218" PostTypeId="2" ParentId="21214" CreationDate="2023-06-23T00:49:15.667" Score="2" Body="&lt;p&gt;The common wording is that the difference between two human &lt;em&gt;haplotypes&lt;/em&gt; is 0.1%. This number only counts base substitutions, not insertions/deletions (indels) or structural variations (SVs). SVs together affects more bases than base substitutions. Also, the number is smaller is non-Africans (~0.07%) and higher in Africans (~0.12%). Anyway, let's denote this number by &lt;span class=&quot;math-container&quot;&gt;$\theta$&lt;/span&gt;, also known as the scaled mutation rate.&lt;/p&gt;&#xA;&lt;p&gt;Because human is diploid, you need to count genotype differences when you compare two human individuals. Under the Wright-Fisher model, the rate of genotype difference on autosomes is&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\theta\cdot\left(1+\frac{1}{2}\cdot\frac{1}{2}+\frac{1}{3}\right)=1.58\theta$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;The rate between a child and his/her parent is, I think:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\theta\cdot\left[1\cdot\frac{1}{2}+\frac{1}{2}\cdot\left(\frac{1}{3}+\frac{2}{3}\cdot\frac{1}{2}\right)+\frac{1}{3}\cdot\frac{1}{2}\right]=\theta$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note that I am sure the rate can be theoretically calculated but I am not confident in my calculation. Also, the rate of genotype difference on sex chromosomes or mitochondrial DNA is different.&lt;/p&gt;&#xA;" OwnerUserId="37" LastEditorUserId="37" LastEditDate="2023-06-23T00:56:13.797" LastActivityDate="2023-06-23T00:56:13.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21219" PostTypeId="1" CreationDate="2023-06-23T05:25:22.577" Score="1" ViewCount="130" Body="&lt;p&gt;I would like to know the level of inbreeding for a single sample VCF file. I'm agnostic to the specific method of measuring inbreeding; any should work. However, I'd like to avoid having to specify a &amp;quot;background&amp;quot; or &amp;quot;reference&amp;quot; population. Ideally, the only input should be the single VCF file containing one human genome sample (WGS, hg38). (I will run this multiple times on different samples, but I'd like to run them individually.)&lt;/p&gt;&#xA;&lt;p&gt;The existing tools seem to all require multiple VCFs.&lt;/p&gt;&#xA;&lt;p&gt;Relatedly, is runs of homozygosity an acceptable metric for evaluating inbreeding in this context? And are there any special considerations to consider for various ethnic groups, like Ashkenazi Jews, that might influence the interpretation?&lt;/p&gt;&#xA;" OwnerUserId="17430" LastEditorUserId="3694" LastEditDate="2023-06-23T07:03:16.103" LastActivityDate="2024-03-19T15:37:25.970" Title="How do I calculate inbreeding on a single VCF file?" Tags="|vcf|genome|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21220" PostTypeId="2" ParentId="21219" CreationDate="2023-06-23T05:45:57.203" Score="0" Body="&lt;p&gt;&lt;strong&gt;Possible solution&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;for record in vcf_reader:&#xA;    if not record.is_snp or len(record.ALT) != 1:&#xA;        continue  # skip if not a biallelic SNP&#xA;&#xA;    gt = record.genotype('Sample')['GT']&#xA;&#xA;    if gt == '0/1' or gt == '1/0':&#xA;        obs_hets += 1&#xA;    elif gt == '0/0' or gt == '1/1':&#xA;        obs_hom += 1&#xA;&#xA;num_variants = obs_hom + obs_hets&#xA;exp_hets = num_variants * 0.5  # under the assumption of Hardy-Weinberg Equilibrium&#xA;&#xA;f = 1 - (obs_hets / exp_hets)  # calculate inbreeding coefficient&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Example output: &amp;quot;-0.25&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Interpretation: We observed more heterozygosity than we expected.&lt;/p&gt;&#xA;&lt;p&gt;I don't know if this method is valid, however.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;A different way to calculate it&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;for record in vcf_reader:&#xA;    if not record.is_snp or len(record.ALT) != 1:&#xA;        continue  # skip if not a biallelic SNP&#xA;&#xA;    gt = record.genotype('Sample')['GT']&#xA;&#xA;    if gt == '0/1' or gt == '1/0':&#xA;        obs_hets += 1&#xA;        allele_counts['A'] += 1&#xA;        allele_counts['a'] += 1&#xA;    elif gt == '0/0':&#xA;        allele_counts['A'] += 2&#xA;    elif gt == '1/1':&#xA;        allele_counts['a'] += 2&#xA;&#xA;total_alleles = sum(allele_counts.values())&#xA;&#xA;# calculate allele frequencies&#xA;p = allele_counts['A'] / total_alleles&#xA;q = allele_counts['a'] / total_alleles&#xA;&#xA;# calculate expected heterozygosity&#xA;exp_hets = 2 * p * q * total_alleles / 2  # the total number of variants&#xA;&#xA;f = 1 - (obs_hets / exp_hets)  # calculate inbreeding coefficient&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Example output: &amp;quot;-0.45&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Interpretation: We observed more heterozygosity than we expected. I don't have a good idea of the meaning of this number.&lt;/p&gt;&#xA;" OwnerUserId="17430" LastEditorUserId="650" LastEditDate="2024-03-19T15:37:25.970" LastActivityDate="2024-03-19T15:37:25.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21221" PostTypeId="1" CreationDate="2023-06-23T20:31:50.813" Score="1" ViewCount="37" Body="&lt;p&gt;I have 4 samples which are 4 conditions but when I made this plot, I got only 2 conditions.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Pk0LH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Pk0LH.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here is the code I used:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;samples &amp;lt;- read.csv(&amp;quot;path/to/sample_sheet_DiffBind.csv&amp;quot;)&#xA;result &amp;lt;- dba(sampleSheet=samples)&#xA;result &amp;lt;- dba.blacklist(result) &#xA;result &amp;lt;- dba.count(result, bParallel=FALSE)&#xA;result &amp;lt;- dba.normalize(result)&#xA;result &amp;lt;- dba.contrast(result, minMembers=2)&#xA;result &amp;lt;- dba.analyze(result)&#xA;dba.plotProfile(profiles)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I followed DiffBind manual but not sure why. Thank you so much! This is the plot before analyzing.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rVhJc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rVhJc.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17585" LastEditorUserId="17585" LastEditDate="2023-06-23T22:06:38.517" LastActivityDate="2023-06-23T22:06:38.517" Title="Missing samples when using function plotProfile() from DiffBind for ATAC-seq" Tags="|atac-seq|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21222" PostTypeId="1" CreationDate="2023-06-24T00:27:12.643" Score="0" ViewCount="54" Body="&lt;p&gt;This is my first time posting to stack exchange, as well as to using data-driven approaches, so please correct me if my questions is not clear enough.&lt;/p&gt;&#xA;&lt;p&gt;I am trying to parse 200+ &lt;em&gt;.pdb&lt;/em&gt; files to extract residues within 15 angstroms of a certain ligand. I have managed to extract the resnames in a text file.&lt;/p&gt;&#xA;&lt;p&gt;In order to do a sanity check, I first ran my code on the first 5 pdb files of this list and compared the outputs to expected output. However, I was wondering if there are any other steps I can take to make my check more robust.&lt;/p&gt;&#xA;&lt;p&gt;Are there any 'gold standard'/good  practices related to this?&lt;/p&gt;&#xA;" OwnerUserId="17696" LastActivityDate="2023-06-24T00:27:12.643" Title="Good practices for doing sanity checks when parsing multiple PDB files" Tags="|proteins|database|pdb|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21224" PostTypeId="1" AcceptedAnswerId="21225" CreationDate="2023-06-24T23:10:53.940" Score="2" ViewCount="67" Body="&lt;p&gt;My goal is to understand the phylogeny of genes involved in bacterial restriction-modification systems, in psychrophilic bacteria.&lt;/p&gt;&#xA;&lt;p&gt;To do so here is what I have done.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Downloaded the protein sequences of all RM genes in psychrophilic bacteria from REBASE. I also downloaded a set of genomes I'd like understand the phylogeny of from NCBI.&lt;/li&gt;&#xA;&lt;li&gt;Clustered these genes using usearch's &lt;code&gt;uclust&lt;/code&gt; method.&lt;/li&gt;&#xA;&lt;li&gt;Aligned the resulting centroids using MUSCLE5.&lt;/li&gt;&#xA;&lt;li&gt;Built an HMM from the aligned centroids using HMMER3.&lt;/li&gt;&#xA;&lt;li&gt;Ran &lt;code&gt;hmmsearch&lt;/code&gt; to get hits from my HMM on each genome (AA).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I am unsure how to proceed now. I now a phylogenetic tree requires an alignment, but I'm not sure where my HMM hits fit-in. Am I meant to use &lt;code&gt;hmmalign&lt;/code&gt; on the output of &lt;code&gt;hmmsearch&lt;/code&gt; and then feed the resulting alignment to IQ-TREE or similar?&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your help as I learn bioinformatics.&lt;/p&gt;&#xA;&lt;p&gt;PS: Reading through the forum it seems like MOLE-BLAST does exactly this, however I have too many genes and their length is too long to use the MOLE-BLAST as hosted by NCBI at least.&lt;/p&gt;&#xA;&lt;p&gt;I should mention I set the identity threshold of search to 50% for proteins. Also, I got this method from this paper which achieved something similar (&lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/30893937/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pubmed.ncbi.nlm.nih.gov/30893937/&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="10792" LastEditorUserId="10792" LastEditDate="2023-06-26T23:50:09.273" LastActivityDate="2023-06-26T23:50:09.273" Title="How to determine the phylogeny of a specific set of genes" Tags="|phylogenetics|phylogeny|hmmer|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21225" PostTypeId="2" ParentId="21224" CreationDate="2023-06-25T01:12:34.027" Score="0" Body="&lt;p&gt;Your strategy is fine but the order is wrong.&lt;/p&gt;&#xA;&lt;p&gt;What I would do is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;2 -&amp;gt; checked by 4 and 6&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Run &lt;em&gt;usearch&lt;/em&gt; and check the output of that in HMMER3. You can then use this to kick out stuff from usearch, that are not real homologues.  Thats what HMMER3, hmmeralign (I dunno hmmersearch), are doing (below). It's also possible the HMMER approaches could replace usearch.&lt;/p&gt;&#xA;&lt;p&gt;Once you have identified your homologues with confidence you can then align each separate group via &lt;em&gt;muscle5&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Your way&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;What you are doing is using HMMER3 to assess the quality of the alignment. What could be a problem is feeding &lt;em&gt;usearch&lt;/em&gt; straight into muscle5 ... a non-homologous protein will get you a bad alignment. The bad alignment will not give you a good tree. &lt;em&gt;Both aligners and trees need nice homologues, otherwise they don't work.&lt;/em&gt; The errors between aligner (muscle5) and tree are also synergistic. Thus if that bad sequences isn't removed by hmmer stuff then alignment errors compound onto sequence errors in the bad sequence giving a really bad tree.&lt;/p&gt;&#xA;&lt;p&gt;You could of course align, HMMER3, filter stuff then align again (thats much better a bit complicated).&lt;/p&gt;&#xA;&lt;p&gt;I don't know how good usearch is - but you are relying on it alot, supposing it gives you orthologues at &amp;lt;25% identity that could be any protein. Basically you are feeding that stuff straight into muscle5 and if HMMER3 doesn't trap it that will go straight onto a tree. That could be a bad alignment and that would be a bad tree.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What you have to understand is phylogenetic trees are very sensitive to bad sequences, i.e. stuff that is not true homologues. The aligner will also be sensitive. There will also be bad synergistic relationships between the two.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;HMMER3 and is associated methods (hmmeralign) are trying to clean stuff up before that stage, rather than simply relying on &lt;em&gt;usearch&lt;/em&gt;. Prof Eddie (HMMER and pfam[?] author) has an excellent reputation in this area - not in tree building but identifying protein families and thats what you are doing here.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;hmmeralign doesn't replace muscle5, but it will provide a very good assessment of the protein families around a given seed. It's going to give you the homologues. That output would then go to muscle5 to get homologous sites.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The tree needs &lt;em&gt;both&lt;/em&gt; homologues and homologous sites.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;The whole hmmer project has a really good reputation in the area of protein family identification and thats why its been recommended.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Probable simple answer&lt;/strong&gt; I think the answer to the question is you're are confusing hmmeralign with muscle5. They are complimentary but distinct. Again hmmeralign identifies homologues and the muscle5 designates homologous sites.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Answering the comments&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;50% cut-off&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It looks like &lt;em&gt;usearch&lt;/em&gt; clusters according to a %age identity threshold.&lt;/p&gt;&#xA;&lt;p&gt;There are three weaknesses:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Gene duplication can take place about 50% ... it depends whether thats cool, usually it isn't, but it's your call.&lt;/li&gt;&#xA;&lt;li&gt;There can be complete change of function &amp;gt;50% identity and this is about the definition of a homologue. If you are using a functional definition its not a homologue&lt;/li&gt;&#xA;&lt;li&gt;Homologues can have &amp;lt;&amp;lt;50% identity and thats very common certainly in viruses I don't know about bacteria.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;hmmersearch is going to rank your data according to the hmmer algorithms. It's going to be like blast,but all hmmer algorithms are focused around defining protein families (it's a better approach than blast &lt;em&gt;for this&lt;/em&gt; application). So it provides another option to assess the output of &lt;em&gt;usearch&lt;/em&gt;. Thus if there a gene duplication for example you should be able to spot it (it will be a sharp break in the continuous output). This will help you assess you 50% cut-off.&lt;/p&gt;&#xA;&lt;p&gt;hmmeralign would be quite cool, because you can take a representation of &lt;em&gt;usearch&lt;/em&gt; and then search that via hmmeralign and see if you get a similar output. It is extremely important for assessing 50% cut-off, does the homology drop below 50%? hmmeralign will tell you, but it must see the complete original database (not the stuff extracted from &lt;em&gt;usearch&lt;/em&gt;. You can cross check that with hmmersearch to assess where hmmeralign is making the cut-off. So hmmeralign will independently (or &lt;em&gt;usearch&lt;/em&gt;) will find you homologues and hmmersearch will give you an insight about how it's done it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="73" LastEditDate="2023-06-25T22:06:30.633" LastActivityDate="2023-06-25T22:06:30.633" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21226" PostTypeId="1" CreationDate="2023-06-25T11:42:34.890" Score="3" ViewCount="133" Body="&lt;p&gt;Mean and median identical with no variation in FASTQC results&#xA;After using FASTQC, in the section of &amp;quot;Per Base Sequence Quality&amp;quot; I encountered a problem when I saw the Box &amp;amp; Whisker type plot. The median value and the mean quality are the same and I have no yellow box, as shown here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/g90gY.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/g90gY.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17704" LastEditorUserId="17704" LastEditDate="2023-06-28T07:58:32.213" LastActivityDate="2023-06-28T07:58:32.213" Title="Mean and median identical with no variation in FASTQC results" Tags="|fastqc|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21228" PostTypeId="1" CreationDate="2023-06-25T23:08:01.753" Score="1" ViewCount="54" Body="&lt;p&gt;I did sequence alignment of a large peptide multi-fasta (n= 4991 sequences). The peptide alignment has sequences with the same length and pal2nal went through just fine... except some of the codon sequences are at half length. If average is X then some sequences are X/2. This is choking IQ-Tree.&lt;/p&gt;&#xA;&lt;p&gt;I have tried both MUSCLE super5 and MAFFT. The error remain the same (i.e. MUSCLE or MAFFT both lead to some sequences having half of average length) except for different average lengths and average half length in MUSCLE and MAFFT codon sequences. I have pulled out and played with the sequences causing the issue and they seem to be in frame.&lt;/p&gt;&#xA;&lt;p&gt;Example of peptide sequence not causing an issue: &lt;code&gt;RKVEAFLLFKEMGERGCQPNVHTYTVLIDSFCKERNLDDARKLFDDMFKKGLVPSVVTYNALIDGYCKEGMTEAALEILGMMESKKCNPNARTYNELICGFCKAK&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;corresponding cds&#xA;&lt;code&gt;AGGAAAGTGGAAGCTTTTCTACTTTTTAAAGAAATGGGTGAAAGAGGTTGTCAGCCTAATGTTCATACATACACTGTGCTTATTGATTCCTTCTGTAAGGAAAGGAATCTTGATGATGCCAGGAAATTGTTTGATGACATGTTTAAGAAAGGTTTGGTTCCCAGTGTGGTCACTTATAATGCTTTAATTGATGGGTATTGTAAAGAGGGAATGACTGAAGCTGCATTAGAAATTTTAGGTATGATGGAATCAAAGAAATGCAACCCTAATGCTCGGACCTACAATGAATTGATCTGTGGATTTTGTAAAGCTAAA&lt;/code&gt;'&lt;/p&gt;&#xA;&lt;p&gt;Example of peptide causing issue:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;GLCKGGRLNDAWEIFQYLLAKGYQLNVHTYNAMVHGFCKEGLLDEAISLLYKMEENGCVPNSVTFNVVL&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;corresponding cds&#xA;&lt;code&gt;GGTTTGTGCAAAGGTGGTAGATTAAATGATGCGTGGGAGATTTTTCAGTATCTTTTAGCGAAAGGTTATCAACTAAATGTCCATACATATAATGCGATGGTTCATGGTTTTTGCAAAGAAGGTTTGCTTGATGAAGCAATCTCCCTGCTTTATAAAATGGAAGAGAATGGTTGTGTCCCTAATTCTGTAACTTTTAATGTAGTCCTT&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Any idea what might be going on?&lt;/p&gt;&#xA;" OwnerUserId="16699" LastEditorUserId="3848" LastEditDate="2023-08-22T02:49:14.350" LastActivityDate="2023-08-22T02:49:14.350" Title="Pal2nal translation of large multi-fasta files produces a codon translated file where some sequences half length of the average" Tags="|phylogenetics|sequence-alignment|phylogeny|proteins|codon|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21229" PostTypeId="1" AcceptedAnswerId="21255" CreationDate="2023-06-26T16:29:06.973" Score="1" ViewCount="63" Body="&lt;p&gt;I am new to use of GEO and having trouble figuring out a straightforward way to grab the names of the files,  (through programmable handles like GEOparse or otherwise) that is the entire and complete list of all files fastq and otherwise (NOT downloading the actual data, but the names and descriptions) within a specific GEO archive (GSEXXX). Albeit with the intention of categorizing the data into  more understandable and friendly to browsing segments, the data presentation in a GEO archive seems very fragmented, making it hard to find what you are looking for. When you submit your data to GEO, they require you to have a spreadsheet, that is essentially the list of files you are submitting with their corresponding meta-data descriptions. Is it possible to get that exact submission spreadsheet associated with a data archive?&lt;/p&gt;&#xA;" OwnerUserId="15741" LastEditorUserId="15741" LastEditDate="2023-06-26T23:54:15.943" LastActivityDate="2023-07-06T06:13:36.083" Title="How do you extract the full list of files in a GEO archive?" Tags="|chip-seq|geoquery|geo|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21230" PostTypeId="1" CreationDate="2023-06-26T18:28:49.333" Score="3" ViewCount="70" Body="&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;: I have been working on a human variant calling pipeline, from whole exome reads to variant annotation.&#xA;For variant annotation I used the &lt;a href=&quot;https://www.ensembl.org/info/docs/tools/vep/script/vep_cache.html#:%7E:text=For%20most%20of%20the%20functionality,database%20(e.g.%20homo_sapiens_core_109_38)%20installed.&quot; rel=&quot;nofollow noreferrer&quot;&gt;Variant Effect predictor (VEP)&lt;/a&gt;, which is &lt;a href=&quot;https://www.ensembl.org/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl&lt;/a&gt; based.&lt;/p&gt;&#xA;&lt;p&gt;I have used the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI&lt;/a&gt; reference genome GRCh38.p14 and the older version GRCh37.p13. The latter version was for comparison with older variant calling pipelines. It is about the latter geneome version I want to continue the discussion.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Problem&lt;/strong&gt;: I sparked a little discussion due to the use of different versions of &amp;quot;hg19&amp;quot; genome (GRCH19, B37, NCBI, Ensembl) in a research project. This is related to &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035890711-GRCh37-hg19-b37-humanG1Kv37-Human-Reference-Discrepancies&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;. Due to discrepancies in these versions, I also wholeheartedly believe these cannot be used interchangeably. But this also got me thinking about using VEP, based on Ensembl and likely their reference genome, while I also used in my pipeline the reference genome of NCBI. When I look at just two example genes, you can already notice the difference:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/UCUcq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/UCUcq.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 1&lt;/strong&gt;: is VEP able to correctly annotate variants likely based on the Ensembl reference genome, while I am using the NCBI reference genome?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 2&lt;/strong&gt;: any thought/knowledge on using the one or the other reference genome, Ensembl or NCBI?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 3&lt;/strong&gt;: Follow-up on Question 2, how do these versions relate to the UCSC version? There does not seem to be a cache for this version?&lt;/p&gt;&#xA;" OwnerUserId="16204" LastEditorUserId="3694" LastEditDate="2023-06-28T14:31:46.723" LastActivityDate="2023-06-28T14:31:46.723" Title="Different human reference genomes from NCBI and ENSEMBL, and variant annotation with Variant Effect Predictor (VEP)" Tags="|variant-calling|reference-genome|ensembl|ncbi|vep|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21232" PostTypeId="2" ParentId="21228" CreationDate="2023-06-27T02:36:06.950" Score="1" Body="&lt;p&gt;I had a quick blast of the example sequences and this is the &lt;em&gt;Acacia&lt;/em&gt; so ancient vascular plant. It forms a locus within the protein in the first part of the question. This protein is comprises a large repeat amino acid motif (check NCBI), the second &amp;quot;issue sequence&amp;quot; described in the question was close to the size of the repeat locus, but start/stop of the locus is not in accordance with the repeat structure. Thus I don't think the repeat locus is part of the issue.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Genome assemblies&lt;/strong&gt; I noted the examples you gave were all draft genomes and this suggest the problem was with the original genome assemblies. If this was true the way to overcome this would be to reassemble the sequence using a reference genome, &lt;em&gt;bwa-mem2&lt;/em&gt; type thing. Thus, here the error is occurring way before the sequences is even downloaded. Just to note, repeat structures can often be collapsed on a draft genome assembly, but I don't think thats happened here.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Reading frame&lt;/strong&gt; what I think you are doing is performing an alignment in the proteins and then using that alignment as the template which is then transferred to the sequence data. So now the nucleotide data is nicely aligned including all indels. That appears to be the goal of &lt;em&gt;pal2nal&lt;/em&gt;. A common error here is reading frame error, particularly at the beginning of the sequence. I use &lt;em&gt;transalign&lt;/em&gt; regularly for this purpose and do encounter that error. I don't know how &lt;em&gt;pal2nal&lt;/em&gt; deals with reading frame errors so I can't comment, but thats why I wanted to see the nucleotide input to &lt;em&gt;pal2nal&lt;/em&gt; to check this.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Trees&lt;/strong&gt; the reason this is taking place is because &lt;em&gt;iqtree&lt;/em&gt; doesn't permit a tree. That I can help with, I know about tree building. If you post your &lt;em&gt;iqtree&lt;/em&gt; command as a separate question , together with the &lt;em&gt;iqtree&lt;/em&gt; error, I can probably answer that. What I'll identify is how to build a tree, this likely to exclude sequence, or could be addressed as two trees. Dunno, but if the bottleneck is can't build a tree - thats easily fixable. In this case, is the sequence issue an assembly error, reading frame error ... something else. There's not enough data that I could point to as the underlying source.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-06-27T02:42:00.060" LastActivityDate="2023-06-27T02:42:00.060" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21233" PostTypeId="1" CreationDate="2023-06-27T08:45:22.447" Score="0" ViewCount="64" Body="&lt;p&gt;Does anybody know of a C/C++ based implementation of the Louvain or Leiden algorithms that will take gene expression looking csv input files, with the genes in columns and the samples in rows, and perform community clustering on it? Something like&#xA;@lh3lh3 &lt;a href=&quot;https://github.com/lh3/bhtsne&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/lh3/bhtsne&lt;/a&gt; but for the community clustering algorithms.&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2023-06-27T08:45:22.447" Title="Louvain/Leiden community clustering implementation in C/C++ for gene expression data" Tags="|gene-expression|clustering|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21236" PostTypeId="1" AcceptedAnswerId="21238" CreationDate="2023-06-27T21:31:42.017" Score="4" ViewCount="315" Body="&lt;p&gt;I have a FASTA header that looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;7c8250ef-c89f-4d42-9d48-12c8fe245fb2 runid=606f271fc97598006ba5a922136a2c304cef75a5 sampleid=Pool12-1 read=19008 ch=301 start_time=2021-07-03T08:48:18Z barcode=barcode01&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And I am able to change it to the desired output here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;7c8250ef-c89f-4d42-9d48-12c8fe245fb2_001&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Using this command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep '^&amp;gt;' 001_old.fasta | cut -d ' ' -f 1,8 | sed '/^&amp;gt;/s/$/_001/' &amp;gt; 001_new.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However it completely deletes the DNA sequence for that read and only outputs the names of the headers to the new fasta file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;7c8250ef-c89f-4d42-9d48-12c8fe245fb2_001&#xA;&amp;gt;815440d5-e033-446c-b7dd-5fad4105184a_001&#xA;&amp;gt;d36a0fea-101d-4bd9-bee4-adec7fe4d43c_001&#xA;&amp;gt;38ca9eb8-bbd2-4d0a-ab06-79d134ad46f1_001&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I think it has something to do with the &lt;code&gt;cut&lt;/code&gt; command I'm using. I tried just doing 1 but that gave the same output so I figured if I included the 8th position of text after a space it would work but again no dice. If someone could help me to include the DNA sequence under the FASTA header in this new file that would be great.&lt;/p&gt;&#xA;" OwnerUserId="16067" LastActivityDate="2023-06-28T03:49:15.417" Title="Keeping DNA sequence after changing FASTA header on command line" Tags="|fasta|grep|multi-fasta|command-line|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21237" PostTypeId="2" ParentId="21236" CreationDate="2023-06-27T21:51:46.257" Score="3" Body="&lt;p&gt;The problem is in the &lt;code&gt;grep&lt;/code&gt; command you're using, which only sends the fasta file headers to the output. Here's my guess at what the commands in your pipeline do:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;grep '^&amp;gt;' 001_old.fasta&lt;/code&gt; - extract lines starting with '&amp;gt;' from &lt;code&gt;001_old.fasta&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;cut -d ' ' -f 1,8&lt;/code&gt; - assume file contains space-delimited records; extract only the first and eighth field from each line&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;sed '/^&amp;gt;/s/$/_001/'&lt;/code&gt; - in lines that begin with a '&amp;gt;', replace the end of the line with '_001'&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;code&gt;&amp;gt; 001_new.fasta&lt;/code&gt; - write result out to &lt;code&gt;001_new.fasta&lt;/code&gt;&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Of all of these, I think only the &lt;code&gt;sed&lt;/code&gt; and output parts are important, but it needs to be slightly modified to work as intended:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sed '/^&amp;gt;/s/ .*$/_001/' 001_old.fasta &amp;gt; 001_new.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Assuming I've got the syntax right, this should search for lines beginning with '&amp;gt;', and replace everything from the first space up to the end of the line with '_001'.&lt;/p&gt;&#xA;&lt;p&gt;I'm more familiar with perl, so here's a perl one-liner that should do the same thing:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -pe '/^&amp;gt;/ &amp;amp;&amp;amp; s/ .*$/_001/' 001_old.fasta &amp;gt; 001_new.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-06-27T21:51:46.257" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21238" PostTypeId="2" ParentId="21236" CreationDate="2023-06-28T00:42:22.460" Score="5" Body="&lt;p&gt;The &lt;code&gt;grep&lt;/code&gt;, &lt;code&gt;cut&lt;/code&gt;, &lt;code&gt;sed&lt;/code&gt; pipeline could also be replaced with the following &lt;a href=&quot;/questions/tagged/awk&quot; class=&quot;post-tag&quot; title=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; aria-label=&quot;show questions tagged &amp;#39;awk&amp;#39;&quot; rel=&quot;tag&quot; aria-labelledby=&quot;tag-awk-tooltip-container&quot;&gt;awk&lt;/a&gt; command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '/^&amp;gt;/ { print $1 &amp;quot;_001&amp;quot;; next }1' file.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The general syntax of an AWK command is &lt;code&gt;condition { action }&lt;/code&gt; where &lt;code&gt;condition&lt;/code&gt; specifies a condition or regular expression, and &lt;code&gt;action&lt;/code&gt; specifies a block of code or set of commands to be run when the condition is met. Both &lt;code&gt;condition&lt;/code&gt; and &lt;code&gt;action&lt;/code&gt; are &lt;em&gt;optional&lt;/em&gt;. Here, we use a regular expression to match lines beginning with a &amp;quot;&amp;gt;&amp;quot; symbol, and print the first whitespace-separated field followed by &amp;quot;_001&amp;quot;. We then use &lt;code&gt;next&lt;/code&gt; to skip the remaining actions and move to the next line, which avoids any further processing of the current line. The &lt;code&gt;1&lt;/code&gt; on the end is actually another condition that always evaluates to true. Since there's no action associated with this condition, AWK performs the default action which is to print the entire line. This effectively prints all lines that do not begin with a &amp;quot;&amp;gt;&amp;quot; symbol.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2023-06-28T03:49:15.417" LastActivityDate="2023-06-28T03:49:15.417" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21239" PostTypeId="1" CreationDate="2023-06-28T16:52:51.687" Score="2" ViewCount="226" Body="&lt;p&gt;I am writing to you to take an input or may be you can provide a different perspective. I am at wits end :(&lt;/p&gt;&#xA;&lt;p&gt;So I have nearly 200 samples. These are separated into two groups (group I treated with drug1 &amp;amp; group II treated with drug II) of course these two groups have sub groups of responders to drug and non responders.&lt;/p&gt;&#xA;&lt;p&gt;I have used DESeq2 to analyze this data. The first thing that I notice about this data is that the in the correlation plot the samples clustering across the groups. In past I have seen some samples clustering across the groups but this a lot. There is no cluster specific to one group.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QTEJ0.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QTEJ0.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Secondly, I get like 5 or 6 DEG that are statistically significant and have fold change of 2.&lt;/p&gt;&#xA;&lt;p&gt;I have been part of similar studies with same tissue and comparable drugs but have never noticed something like this.&lt;/p&gt;&#xA;&lt;p&gt;I am not sure what is happening here and why it is happening.&lt;/p&gt;&#xA;&lt;p&gt;Any help or insights are sincerely appreciated.&lt;/p&gt;&#xA;" OwnerUserId="5062" LastEditorUserId="3848" LastEditDate="2023-06-29T18:42:45.813" LastActivityDate="2023-06-29T18:42:45.813" Title="Correlation heatmap of RNA-seq clusters all samples together leading to very low no. of DEGs" Tags="|rna-seq|phylogenetics|phylogeny|deseq2|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21240" PostTypeId="2" ParentId="21239" CreationDate="2023-06-28T18:52:04.893" Score="4" Body="&lt;p&gt;This answer has two stages,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Diagnostics, what analytically has gone wrong?&lt;/li&gt;&#xA;&lt;li&gt;Where to go from here.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I am pretty confident about point 1. I've been in comparable situations and I know trees. Stage 2, is more open to opinion, however there's no point criticising something (point 1) without pointing out how it can be corrected (point 2).&lt;/p&gt;&#xA;&lt;p&gt;If your question is within group heterogeneity, excepting for the large group of non-responders associated with D1, thats a different answer.&lt;/p&gt;&#xA;&lt;p&gt;Low number of DEGs is about your control group? Difficult to comment because the DEG calculation is not declared, nor volcano plots, cut-offs, FDR etc...&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;You have to keep in mind this data mining and the there are some interesting groups emerging. For first stage analytics looks reasonable (low DEGs dunno).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Result&lt;/strong&gt; You have 9-10 clusters and those are informative.  You can see this because they are forming a checker-board across the heat-map.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Issue&lt;/strong&gt; The issue is that the deeper branches below the 9-10 clusters are not informative. Do see? The deep branches are likely to be repositioned in different orders in different analyses. Those alternative order might make better biological sense. On the heat-map shown there's a risk of attaching two much important to deep branching orders for a UPMGA tree for a correlation dendrogram.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Heterogeneity across the matrix is not permitted in UPGMA and causes artefacts - because you are trying to say there's a uniform rate when in most biology that assumption doesn't work. This will usually result in incorrect branching orders.&lt;/li&gt;&#xA;&lt;li&gt;You are forcing the data into a bifurcating tree: it might not be at all, could be a large polytomy (star tree).&lt;/li&gt;&#xA;&lt;li&gt;Even data sets designed for trees (correlation matrixes are not good data sets for trees) ... branching orders go wrong for loads of reasons.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The second stage analysis would be to take the correlation matrix and apply the neighbour-joining (nj) algorithm to it and see whether the deeper branch align differently. There are also &lt;em&gt;much&lt;/em&gt; better tree methods, but that starts getting complicated fast, because there are alternative approaches, such as co-variance. Advanced tree methods (Bayes and ML) will model shifts heterogeneity - however it needs some expertise to get it working.&lt;/p&gt;&#xA;&lt;p&gt;I'd start assessing the assumptions UPGMA vs neighbour-joining does that take me closer to a biological explanation, then a 9x9 checkboard? Thus would the number of squares in a checkerboard by greatly reduced via nj?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Machine learning&lt;/strong&gt; Supervised learning (ML), is the trendy way into the data set and in this day an age thats probably the way I'd go. What I'd do is use the 9-10 clusters here as an initial assessment of the training targets. I'd then perform whats called a &amp;quot;feature selection analysis&amp;quot; and assess whether that output makes biological sense. The feature selection should start identifying what parts of your data are causing the shifts between those 9-10 clusters.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-06-28T20:17:02.020" LastActivityDate="2023-06-28T20:17:02.020" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21241" PostTypeId="2" ParentId="19533" CreationDate="2023-06-28T23:34:57.817" Score="1" Body="&lt;p&gt;I've come across the same error which seems to be a common issue when using the &lt;code&gt;subset()&lt;/code&gt; function with Seurat objects. Sometimes, I encountered the same error and couldn't figure out the reason behind it, checking the variable's names was not a remedy in my case as I specified the exact same name for subsetting as to what I could see in my object metadata. However, I found a workaround to subset cells in a Seurat object using a different approach.&lt;/p&gt;&#xA;&lt;p&gt;Instead of using &lt;code&gt;subset()&lt;/code&gt;, you can try the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seurat_obj_subset &amp;lt;- seurat_obj[, &amp;lt;condition to be met&amp;gt;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For example, if you want to subset a Seurat object called 'pbmc' based on conditions like having more than 1000 features and more than 4000 counts, you can use the following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pbmc = pbmc[, pbmc&lt;span class=&quot;math-container&quot;&gt;$nFeature &amp;gt; 1000 &amp;amp; pbmc$&lt;/span&gt;nCount &amp;gt; 4000]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17726" LastActivityDate="2023-06-28T23:34:57.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21242" PostTypeId="1" CreationDate="2023-06-29T00:13:55.130" Score="1" ViewCount="273" Body="&lt;p&gt;Here are the first few columns of my VCF v4.2 file. As you can see, the alt allele is missing. The VCF's refs and alts don't correspond to the reference genome.&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;#CHROM&lt;/th&gt;&#xA;&lt;th&gt;POS&lt;/th&gt;&#xA;&lt;th&gt;ID&lt;/th&gt;&#xA;&lt;th&gt;REF&lt;/th&gt;&#xA;&lt;th&gt;ALT&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;77051&lt;/td&gt;&#xA;&lt;td&gt;rs9408670&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;C&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;114350&lt;/td&gt;&#xA;&lt;td&gt;rs12685480&lt;/td&gt;&#xA;&lt;td&gt;A&lt;/td&gt;&#xA;&lt;td&gt;.&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;116581&lt;/td&gt;&#xA;&lt;td&gt;rs558835286&lt;/td&gt;&#xA;&lt;td&gt;G&lt;/td&gt;&#xA;&lt;td&gt;.&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;117550&lt;/td&gt;&#xA;&lt;td&gt;rs267602077&lt;/td&gt;&#xA;&lt;td&gt;A&lt;/td&gt;&#xA;&lt;td&gt;.&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;117720&lt;/td&gt;&#xA;&lt;td&gt;rs10959293&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;.&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;126903&lt;/td&gt;&#xA;&lt;td&gt;rs62533602&lt;/td&gt;&#xA;&lt;td&gt;G&lt;/td&gt;&#xA;&lt;td&gt;A&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;133828&lt;/td&gt;&#xA;&lt;td&gt;rs78804900&lt;/td&gt;&#xA;&lt;td&gt;C&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;154736&lt;/td&gt;&#xA;&lt;td&gt;rs200683662&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;.&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;154933&lt;/td&gt;&#xA;&lt;td&gt;rs17795008&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;G&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;174425&lt;/td&gt;&#xA;&lt;td&gt;rs2998944&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;C&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;174500&lt;/td&gt;&#xA;&lt;td&gt;rs142581442&lt;/td&gt;&#xA;&lt;td&gt;G&lt;/td&gt;&#xA;&lt;td&gt;.&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;175050&lt;/td&gt;&#xA;&lt;td&gt;rs62532020&lt;/td&gt;&#xA;&lt;td&gt;G&lt;/td&gt;&#xA;&lt;td&gt;A&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;185632&lt;/td&gt;&#xA;&lt;td&gt;rs2992854&lt;/td&gt;&#xA;&lt;td&gt;C&lt;/td&gt;&#xA;&lt;td&gt;A&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;188894&lt;/td&gt;&#xA;&lt;td&gt;rs3951559&lt;/td&gt;&#xA;&lt;td&gt;G&lt;/td&gt;&#xA;&lt;td&gt;A&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;191843&lt;/td&gt;&#xA;&lt;td&gt;rs3008128&lt;/td&gt;&#xA;&lt;td&gt;G&lt;/td&gt;&#xA;&lt;td&gt;A&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;202061&lt;/td&gt;&#xA;&lt;td&gt;rs117675283&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;.&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;204201&lt;/td&gt;&#xA;&lt;td&gt;rs10964134&lt;/td&gt;&#xA;&lt;td&gt;G&lt;/td&gt;&#xA;&lt;td&gt;A&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;The missing alt allele happens when the genotype is 0/0. The VCF file isn't saying anything incorrect. However, there is a tool I'm trying to use that filters out &amp;quot;0/.&amp;quot;genotypes.&lt;/p&gt;&#xA;&lt;p&gt;A better way of formatting them (which I'm trying to achieve) would have the ref/alt alleles correspond to the reference genome, and just have a redundant letter in the case of a homozygous genotype.&lt;/p&gt;&#xA;&lt;p&gt;I'd like my VCF to display the correct alt/ref for that position even if there are no samples with a genotype corresponding to the alt allele. How do I do this? I'm not sure what this process is called.&lt;/p&gt;&#xA;&lt;p&gt;Ideal output:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;#CHROM&lt;/th&gt;&#xA;&lt;th&gt;POS&lt;/th&gt;&#xA;&lt;th&gt;ID&lt;/th&gt;&#xA;&lt;th&gt;REF&lt;/th&gt;&#xA;&lt;th&gt;ALT&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;77051&lt;/td&gt;&#xA;&lt;td&gt;rs9408670&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;C&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;114350&lt;/td&gt;&#xA;&lt;td&gt;rs12685480&lt;/td&gt;&#xA;&lt;td&gt;A&lt;/td&gt;&#xA;&lt;td&gt;G&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;116581&lt;/td&gt;&#xA;&lt;td&gt;rs558835286&lt;/td&gt;&#xA;&lt;td&gt;G&lt;/td&gt;&#xA;&lt;td&gt;A&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;" OwnerUserId="17430" LastEditorUserId="73" LastEditDate="2023-06-30T00:14:44.473" LastActivityDate="2023-07-02T04:58:23.573" Title="How do I add reference/alternate alleles to my vcf file?" Tags="|vcf|genome|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21243" PostTypeId="1" AcceptedAnswerId="21244" CreationDate="2023-06-29T15:45:29.010" Score="4" ViewCount="200" Body="&lt;p&gt;I'm expecting to get 17 different paired-end fastq files (34 in total), so I want to make a bash script to just run my code through all the fastq files in a directory at once. How can I change the name of the input and output files each time the script runs through each file? So when it moves to the file_002, all names have file_002 at the beginning instead of file_001, and so on. And also, when merging the R1 and R2 reads how can I make that it only merges the correspondant files with a loop? for examples merging only file_001_R1 with file_001_R2, file_002_R1 with file_002_R2, file_003_R1 with file_003_R2, and so on.&lt;/p&gt;&#xA;&lt;p&gt;This is my code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for file in directory_name&#xA;do&#xA;pear -f file_001_R1.fastq.gz -r file_001_R2.fastq.gz -o file_001.fastq&#xA;cutadapt -g TGATAACAATTGGAGCAGCCTC...GGATCGACCAAGAACCAGCA -o file_001_barcode.fastq file_001.fastq&#xA;cutadapt -g GTGTACAAATAATTGTCAAC...CTGTCTCTTATACACATCTC -o file_001_UMI.fastq file_001.fastq&#xA;seqkit concat file_001_barcode.fastq file_001_UMI.fastq &amp;gt; file_001_concatenation.fastq&#xA;seqkit rmdup -s file_001_concatenation.fastq -o file_001_unique_pairs.fastq&#xA;seqkit subseq -r file_001_unique_pairs.fastq &amp;gt; file_001_unique_barcodes.fasta&#xA;bowtie -q --suppress 1,2,4,6,7,8 -x ref_index file_001_unique_barcodes.fasta &amp;gt; file_001_barcodes_allignment.bowtie&#xA;sort file_001_barcodes_allignment.bowtie | uniq -c &amp;gt; file_001_barcode_counts.txt&#xA;awk 'BEGIN{print &amp;quot;Barcode,TF_variant,Code&amp;quot;}{print $3&amp;quot;,&amp;quot;$2&amp;quot;,&amp;quot;$1}' file_001_barcode_counts.txt &amp;gt; file_001_barcode_counts.csv&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="17729" LastActivityDate="2023-06-30T00:11:19.947" Title="How can I give different names to files in a directory with a for loop in a bash script?" Tags="|bash|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21244" PostTypeId="2" ParentId="21243" CreationDate="2023-06-29T18:35:19.640" Score="3" Body="&lt;p&gt;Here's what I use - along with an example you can modify as needed:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FASTQ_DIR=&amp;quot;/directory/containing/FASTQs/&amp;quot;&#xA;&#xA;for R1_FASTQ in &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$FASTQ_DIR&quot;*R1*.fastq.gz&#xA;do&#xA;                         BASE=$&lt;/span&gt;(basename &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$R1_FASTQ&quot; _R1.fastq.gz)&#xA;                         R2_FASTQ=&quot;$&lt;/span&gt;{R1_FASTQ/R1/R2}&amp;quot;&#xA;                         R3_FASTQ=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${R1_FASTQ/R1/R3}&quot;&#xA;                         echo &quot;BASE: $&lt;/span&gt;BASE&amp;quot;&#xA;                         echo &amp;quot;R1: &lt;span class=&quot;math-container&quot;&gt;$R1_FASTQ&quot;&#xA;                         echo &quot;R2: $&lt;/span&gt;R2_FASTQ&amp;quot;&#xA;                         echo &amp;quot;R3: &lt;span class=&quot;math-container&quot;&gt;$R3_FASTQ&quot;&#xA;                         # process R1/R2 here using &quot;$&lt;/span&gt;R1_FASTQ&amp;quot; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$R2_FASTQ&quot; for input, and &quot;$&lt;/span&gt;{BASE}_XXXXXXX.fastq&amp;quot; for output&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Basically, the strategy is to iterate over all the R1 FASTQs in a directory, calculate a &lt;code&gt;BASE&lt;/code&gt; name that can be used to rename output files with arbitrary extensions (like &lt;code&gt;.sam&lt;/code&gt;, &lt;code&gt;.bam&lt;/code&gt;, etc...), and use BASH shell parameter expansion to calculate and set variables to the R2 (and R3 if necessary) reads present in an Illumina run that uses standard Illumina naming conventions.&lt;/p&gt;&#xA;&lt;p&gt;This may need modification depending on the naming conventions you are using.&lt;/p&gt;&#xA;" OwnerUserId="92" LastEditorUserId="92" LastEditDate="2023-06-29T18:40:25.670" LastActivityDate="2023-06-29T18:40:25.670" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21245" PostTypeId="2" ParentId="21243" CreationDate="2023-06-30T00:11:19.947" Score="2" Body="&lt;p&gt;Something like this? This uses &lt;code&gt;printf&lt;/code&gt; to create a zero-padded 3-digit number corresponding to 17 numbers from 1 to 17:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for fileBase in $(printf 'file_%03d\n' $(seq 1 17));&#xA;   do echo ${fileBase}_R1.fq.gz ${fileBase}_R2.fq.gz;&#xA;   pear -f ${fileBase}_R1.fastq.gz -r ${fileBase}_R2.fastq.gz -o ${fileBase}.fastq&#xA;   ...&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-06-30T00:11:19.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21246" PostTypeId="2" ParentId="15059" CreationDate="2023-06-30T17:44:52.640" Score="1" Body="&lt;p&gt;Ok went back to this question having &lt;code&gt;pymol&lt;/code&gt; within a hand's reach this time. Question is not really clear to me so I am posting a reasonable (to me) version of the question into a new code snippet;&lt;/p&gt;&#xA;&lt;p&gt;I am using two PDBs containing &lt;code&gt;ATP&lt;/code&gt; as ligand &lt;code&gt;'3dgl.pdb', '2p09.pdb'&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pymol&#xA;&#xA;&#xA;list_pdbs = ['3dgl.pdb', '2p09.pdb']&#xA;&#xA;&#xA;pymol.finish_launching()&#xA;&#xA;&#xA;&#xA;for _i in list_pdbs :&#xA;    &#xA;    pymol.cmd.load( _i, _i.rstrip('.pdb'))&#xA;&#xA;&#xA;objs = pymol.cmd.get_object_list(selection='(all)')&#xA;&#xA;print('structure loaded : ' , objs)&#xA;&#xA;&#xA;pymol.cmd.remove('resn HOH')&#xA;&#xA;for _i in objs:&#xA;    &#xA;    &#xA;    pymol.cmd.create(_i+'_ATP_copy' ,  _i + ' and resn ATP')&#xA;    &#xA;    # print('after create ')&#xA;    &#xA;    area = pymol.cmd.get_area(_i+'_ATP_copy', quiet = 0)&#xA;    &#xA;    pymol.cmd.show('dots' , _i+'_ATP_copy')&#xA;    &#xA;    print(_i+'ATP_copy' + ' area ---&amp;gt;  ' + str(area))&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;structure loaded :  ['3dgl', '2p09']&#xA; cmd.get_area: 1013.704 Angstroms^2.&#xA;3dglATP_copy area ---&amp;gt;  1013.7040405273438&#xA; cmd.get_area: 1013.704 Angstroms^2.&#xA;2p09ATP_copy area ---&amp;gt;  1013.7040405273438&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;picture of the &lt;code&gt;ATP&lt;/code&gt; surfaces in Pymol itself,&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jBaSs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jBaSs.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;you can play with  &lt;code&gt;cmd.get_area(...)&lt;/code&gt; parameters see &lt;a href=&quot;https://pymolwiki.org/index.php/Get_area&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pymolwiki.org/index.php/Get_area&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10049" LastActivityDate="2023-06-30T17:44:52.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21247" PostTypeId="1" CreationDate="2023-06-30T20:07:12.770" Score="1" ViewCount="204" Body="&lt;p&gt;The vignette of R package &lt;code&gt;ieugwasr&lt;/code&gt; describes a &lt;code&gt;plink&lt;/code&gt; based wrapper function for LD clumping GWAS data using the &lt;em&gt;1000 genomes reference dataset&lt;/em&gt; (&lt;a href=&quot;http://fileserve.mrcieu.ac.uk/ld/1kg.v3.tgz&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://fileserve.mrcieu.ac.uk/ld/1kg.v3.tgz&lt;/a&gt;). I would like LD clump GRCh38 based GWAS data. I am under the impression that the previous LD reference dataset is based on GRCh37. Therefore, I kindly ask:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Is &lt;code&gt;1kg.v3.tgz&lt;/code&gt; based on GRCh37?&lt;/li&gt;&#xA;&lt;li&gt;Can you use GRCh37 based reference to LD clump GRCh38 GWAS results?&lt;/li&gt;&#xA;&lt;li&gt;Is there a GRCh38 reference version of 1000 genomes that is split to different super populations (EUR etc)?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17735" LastEditorUserId="17735" LastEditDate="2023-06-30T20:09:29.770" LastActivityDate="2024-01-04T11:42:00.207" Title="LD clump GRCh38 GWAS results" Tags="|gwas|reference-genome|1000genomes|ld|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21248" PostTypeId="1" CreationDate="2023-07-01T14:06:28.350" Score="0" ViewCount="23" Body="&lt;p&gt;Is there a comprehensive database available online of gene expression across the brain taken from individuals with psychological conditions? Resources like the Allen Institute's Human Brain Map (link here for reference) feature heatmaps of gene expression taken from healthy volunteers, and I'm looking for similar datasets from patients with particular conditions, especially sensory processing disorders such as ASD and ADHD. Any and all data is appreciated!&lt;/p&gt;&#xA;" OwnerUserId="17738" LastActivityDate="2023-07-01T14:06:28.350" Title="Gene Expression Database for Psychological Disorders" Tags="|rna-seq|genome|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21250" PostTypeId="1" CreationDate="2023-07-01T22:40:33.837" Score="1" ViewCount="47" Body="&lt;p&gt;I have a MS-lipidomics dataset - from human samples split into a treatment group (n = 2) and control (n = 2) - which contains several detected targets that matched to the same lipid. I don't have access to any of the raw data, only the Tier 1 and Tier 2-matched lipid names and ID numbers.&lt;/p&gt;&#xA;&lt;p&gt;There are around 300 duplicates out of a total of 1000 detected markers, so I'm reluctant to exclude them entirely, but I couldn't find any descriptions of best-practices for what to do in this situation. I'm considering testing the correlation between duplicates and removing the ones lie below some threshold (e.g: 0.5), then averaging the remaining duplicates - but I'm not sure if this would be a valid way of approaching this.&lt;/p&gt;&#xA;&lt;p&gt;Are there any good ways of addressing these duplicate values?&lt;/p&gt;&#xA;" OwnerUserId="17741" LastEditorUserId="73" LastEditDate="2023-07-09T20:00:20.057" LastActivityDate="2024-04-04T23:04:15.503" Title="Handling duplicates in untargeted lipidomics" Tags="|proteomics|multi-omics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21251" PostTypeId="1" AcceptedAnswerId="21257" CreationDate="2023-07-02T02:00:43.150" Score="2" ViewCount="59" Body="&lt;p&gt;I am interested in running an analysis on the genome of every psychrophilic bacteria. Here defined as bacteria that have an optimal growth temperature at less than 15°C.&lt;/p&gt;&#xA;&lt;p&gt;What would be the best way to go about collecting these?&lt;/p&gt;&#xA;&lt;p&gt;I have yet to find a &amp;quot;growth temperature&amp;quot; filter on databases like NCBI. Though REBASE does include that (however obtaining the genomes from that site does not seem straightforward).&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="10792" LastActivityDate="2023-07-04T00:21:03.493" Title="What would be the best method to obtain every prokaryotic psychrophile genome?" Tags="|metagenome|wgs|bacteria|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21252" PostTypeId="2" ParentId="21251" CreationDate="2023-07-02T02:21:26.523" Score="2" Body="&lt;p&gt;The database to use is BactoTraits. This will help either in the entirety or at least provide a solid starting point. Its here&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S1470160X21007123&quot; rel=&quot;nofollow noreferrer&quot;&gt;BactoTraits – A functional trait database to evaluate how natural and man-induced changes influence the assembly of bacterial communities&lt;/a&gt;&#xA;by Aurélie Cébron (2021) Ecological Indicators 130&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;We have created a user-friendly database of bacterial functional traits, thanks to the properties of &lt;strong&gt;19,455 bacterial strains&lt;/strong&gt;. This database has been called BactoTraits. For example, oxygen preference, size and shape of bacteria, motility, optimum and range of pH and &lt;strong&gt;temperature&lt;/strong&gt;, genome GC percent and trophic type are among the 19 traits included in BactoTraits. Based on the best-informed strains in the database, we identified five functional groups (i.e. mesophiles, competitors, colonizers, stress-sensitives and stress-tolerants) exhibiting a wide strain taxonomic diversity but with quite similar trait profile combinations.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Thus it's nearly 20000 species so your batteria should be there.&lt;/p&gt;&#xA;&lt;p&gt;You can download the exact database &lt;a href=&quot;https://ordar.otelo.univ-lorraine.fr/record?id=10.24396/ORDAR-53&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. This is Univ de Lorraine's site, France. Its 9.4MB and called&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;BactoTraits_databaseV2_Jun2022.csv&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The supplementary information in the paper gives all the details for interpreting the database columns.&lt;/p&gt;&#xA;&lt;p&gt;Given your skills in development you should have no trouble converting the &lt;strong&gt;csv&lt;/strong&gt; format into a functional database and interrogating/querying it. My understanding of the paper is that the authors are microbiologists rather than coders, which in context is perfect.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Linking NCBI&lt;/p&gt;&#xA;&lt;p&gt;BactoTraits contains two strain designations&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;DSM no. (&lt;a href=&quot;https://www.dsmz.de/collection/catalogue/microorganisms/catalogue&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dsmz.de/collection/catalogue/microorganisms/catalogue&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;ATCC no. for example  ATCC BAA 1260 is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?mode=Info&amp;amp;id=364410&quot; rel=&quot;nofollow noreferrer&quot;&gt;here on NCBI&lt;/a&gt; &lt;em&gt;Granulibacter bethesdensis&lt;/em&gt;, which checks out on BactoTraits and is DSM 17861&lt;/li&gt;&#xA;&lt;li&gt;txid364410[Organism:exp]  is the SRA number from the above link ... so its all there, and here &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra/?term=txid364410%5BOrganism:exp%5D&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/sra/?term=txid364410[Organism:exp]&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So ATCC links to NCBI links to the SRA and genome.&lt;/p&gt;&#xA;&lt;p&gt;DSM are not necessarily available on NCBI, but ATCC will there. NCBI do have lots of databases for download, a database comprising ATCC to NCBI genome ... dunno, but &lt;em&gt;Entrez&lt;/em&gt; will be able to link the ATCC (American Type Culture Collection)&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-07-02T04:07:39.093" LastActivityDate="2023-07-02T04:07:39.093" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21253" PostTypeId="2" ParentId="21242" CreationDate="2023-07-02T04:58:23.573" Score="1" Body="&lt;p&gt;Let me know if you see any potential issues with this solution. It just replaces the dots with random letters (easier/faster than aligning to reference genome).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def add_ref():&#xA;    # Directory path&#xA;    dir_path = '/path/'&#xA;&#xA;    # Function to process genotypes in each row&#xA;    def process_genotypes(row):&#xA;        genotypes = np.array([x.split(&amp;quot;:&amp;quot;)[0] for x in row[9:]])&#xA;        if row[4] == &amp;quot;.&amp;quot; and not any(val in [&amp;quot;1/1&amp;quot;, &amp;quot;0/1&amp;quot;, &amp;quot;1/0&amp;quot;] for val in genotypes): # to make sure I'm not introducing an allele that isn't present&#xA;            # also can do if row[3] in [&amp;quot;A&amp;quot;, &amp;quot;T&amp;quot;]: then random.choice([&amp;quot;C&amp;quot;, &amp;quot;G&amp;quot;])&#xA;            if row[3] == &amp;quot;A&amp;quot;:&#xA;                row[4] = &amp;quot;C&amp;quot;&#xA;            elif row[3] == &amp;quot;T&amp;quot;:&#xA;                row[4] = &amp;quot;G&amp;quot;&#xA;            elif row[3] == &amp;quot;C&amp;quot;:&#xA;                row[4] = random.choice([&amp;quot;T&amp;quot;]) # I know&#xA;            elif row[3] == &amp;quot;G&amp;quot;:&#xA;                row[4] = random.choice([&amp;quot;A&amp;quot;])&#xA;        return row&#xA;&#xA;    # Get all .vcf.gz files in the directory&#xA;    files = glob.glob(dir_path + '*.vcf.gz')&#xA;&#xA;    for file in files:&#xA;        print(f&amp;quot;Processing {file}...&amp;quot;)&#xA;        # Name of the new file&#xA;        new_file = os.path.join(dir_path, 'new_' + os.path.basename(file))&#xA;        with gzip.open(file, 'rt') as in_f, gzip.open(new_file, 'wt') as out_f:&#xA;            for line in in_f:&#xA;                if line.startswith('#'):&#xA;                    out_f.write(line)&#xA;                else:&#xA;                    row = line.strip().split('\t')&#xA;                    new_row = process_genotypes(row)&#xA;                    out_f.write('\t'.join(new_row) + '\n')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17430" LastActivityDate="2023-07-02T04:58:23.573" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21254" PostTypeId="1" CreationDate="2023-07-02T06:32:01.117" Score="1" ViewCount="78" Body="&lt;p&gt;How do I generate a custom gtf file format for  a organism of my interest similar to human gtf file, in the custom gtf file I want to add entrez id to the new custom gtf.&lt;/p&gt;&#xA;&lt;p&gt;My organism is canis_familaris&lt;/p&gt;&#xA;&lt;p&gt;I  have to use is this &lt;a href=&quot;https://ftp.ncbi.nih.gov/gene/DATA/GENE_INFO/Mammalia/&quot; rel=&quot;nofollow noreferrer&quot;&gt;source&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;How do I start, is there any tools there to build custom gtf file from the GENE_INFO file?&lt;/p&gt;&#xA;&lt;p&gt;From the gene_info file I extracted the ensembl and the entrezID column so here as an example I have taken only the top 10 head of the file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;entrezID   dbXrefs&#xA;399518   VGNC:VGNC:43937|Ensembl:ENSCAFG00845006432&#xA;399530   VGNC:VGNC:40330|Ensembl:ENSCAFG00845002136&#xA;399544   VGNC:VGNC:48329|Ensembl:ENSCAFG00845029798&#xA;399545   VGNC:VGNC:41624|Ensembl:ENSCAFG00845011460&#xA;399653   VGNC:VGNC:44358|Ensembl:ENSCAFG00845001610&#xA;403152   -&#xA;403157   VGNC:VGNC:38980|Ensembl:ENSCAFG00845013158&#xA;403168   VGNC:VGNC:37699|Ensembl:ENSCAFG00845014982&#xA;403169   VGNC:VGNC:54270&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My input gtf file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!genome-build ROS_Cfam_1.0&#xA;#!genome-version ROS_Cfam_1.0&#xA;#!genome-date 2020-09&#xA;#!genome-build-accession GCA_014441545.1&#xA;#!genebuild-last-updated 2020-10&#xA;X       ensembl gene    24550462        24552226        .       -       .       gene_id &amp;quot;ENSCAFG00845015183&amp;quot;; gene_version &amp;quot;1&amp;quot;; gene_source &amp;quot;ensembl&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;;&#xA;X       ensembl transcript      24550462        24552226        .       -       .       gene_id &amp;quot;ENSCAFG00845015183&amp;quot;; gene_version &amp;quot;1&amp;quot;; transcript_id &amp;quot;ENSCAFT00845027108&amp;quot;; transcript_version &amp;quot;1&amp;quot;; gene_source &amp;quot;ensembl&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_source &amp;quot;ensembl&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;; tag &amp;quot;Ensembl_canonical&amp;quot;;&#xA;X       ensembl exon    24552206        24552226        .       -       .       gene_id &amp;quot;ENSCAFG00845015183&amp;quot;; gene_version &amp;quot;1&amp;quot;; transcript_id &amp;quot;ENSCAFT00845027108&amp;quot;; transcript_version &amp;quot;1&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_source &amp;quot;ensembl&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_source &amp;quot;ensembl&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;; exon_id &amp;quot;ENSCAFE00845128634&amp;quot;; exon_version &amp;quot;1&amp;quot;; tag &amp;quot;Ensembl_canonical&amp;quot;;&#xA;X       ensembl CDS     24552206        24552226        .       -       0       gene_id &amp;quot;ENSCAFG00845015183&amp;quot;; gene_version &amp;quot;1&amp;quot;; transcript_id &amp;quot;ENSCAFT00845027108&amp;quot;; transcript_version &amp;quot;1&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_source &amp;quot;ensembl&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_source &amp;quot;ensembl&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;; protein_id &amp;quot;ENSCAFP00845021332&amp;quot;; protein_version &amp;quot;1&amp;quot;; tag &amp;quot;Ensembl_canonical&amp;quot;;&#xA;X       ensembl start_codon     24552224        24552226        .       -       0       gene_id &amp;quot;ENSCAFG00845015183&amp;quot;; gene_version &amp;quot;1&amp;quot;; transcript_id &amp;quot;ENSCAFT00845027108&amp;quot;; transcript_version &amp;quot;1&amp;quot;; exon_number &amp;quot;1&amp;quot;; gene_source &amp;quot;ensembl&amp;quot;; gene_biotype &amp;quot;protein_coding&amp;quot;; transcript_source &amp;quot;ensembl&amp;quot;; transcript_biotype &amp;quot;protein_coding&amp;quot;; tag &amp;quot;Ensembl_canonical&amp;quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So the final goal is to add entrezID to the modified gtf file keeping the format same as it was in the input file.&lt;/p&gt;&#xA;&lt;p&gt;Here in my small gtf input the gene_id is only for ENSCAFG00845015183 as due to space constraint i didn't put more lines.&lt;/p&gt;&#xA;&lt;p&gt;So now how do I map my reference file entrezID to the gtf file gene_ID as this pattern &lt;code&gt;ENSCAFG&lt;/code&gt; which is common in both the file.&lt;/p&gt;&#xA;&lt;p&gt;Now in my mapping file I dont have entrezID for ENSCAFG00845015183 since due to first 10 row i have used to show.&lt;/p&gt;&#xA;&lt;p&gt;So how do I map these two file where the modified gtf fill will have new attributes as entrezID&lt;/p&gt;&#xA;" OwnerUserId="285" LastEditorUserId="285" LastEditDate="2023-07-03T12:30:43.180" LastActivityDate="2023-07-03T12:30:43.180" Title="building custom gtf file with entrez ID for any organism" Tags="|rna-seq|gtf|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21255" PostTypeId="2" ParentId="21229" CreationDate="2023-07-02T06:40:47.657" Score="1" Body="&lt;p&gt;Unlikely that you will be able to get the original submission spreadsheet, but you can use &lt;a href=&quot;http://geofetch.databio.org/en/latest/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Geofetch&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6317" LastEditorUserId="6317" LastEditDate="2023-07-06T06:13:36.083" LastActivityDate="2023-07-06T06:13:36.083" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21256" PostTypeId="2" ParentId="4586" CreationDate="2023-07-03T23:35:08.380" Score="1" Body="&lt;p&gt;As the question mentions python, here is a python example using &lt;a href=&quot;https://github.com/biopython/biopython&quot; rel=&quot;nofollow noreferrer&quot;&gt;biopython&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import json&#xA;from Bio import Entrez&#xA;&#xA;&#xA;def accession2taxid(acc: str, db=&amp;quot;nucleotide&amp;quot;) -&amp;gt; str:&#xA;    handle = Entrez.esearch(db=db, term=acc)&#xA;    record = Entrez.read(handle)&#xA;    gi = record[&amp;quot;IdList&amp;quot;][0]&#xA;    handle = Entrez.esummary(db=db, id=gi, retmode=&amp;quot;json&amp;quot;)&#xA;    result = json.load(handle)[&amp;quot;result&amp;quot;]&#xA;    taxid = result[gi][&amp;quot;taxid&amp;quot;]&#xA;    return str(taxid)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So if you want to use an assembly accession, as in your example&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; accession2taxid(&amp;quot;GCA_000005845.2&amp;quot;, db=&amp;quot;assembly&amp;quot;)&#xA;'511145'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In your example, you said that &lt;code&gt;GCA_000005845.2&lt;/code&gt; should return &lt;code&gt;79781&lt;/code&gt;, but that Taxon ID is for &amp;quot;uncultured archaeon AOC672&amp;quot;. The taxid in the example I just showed returns &amp;quot;Escherichia coli str. K-12 substr. MG1655&amp;quot;, which is what you want.&lt;/p&gt;&#xA;&lt;p&gt;Your question also mentions GenBank accessions, so here is another example&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; accession2taxid(&amp;quot;CU458896.1&amp;quot;, db=&amp;quot;nucleotide&amp;quot;)&#xA;'561007'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="149" LastActivityDate="2023-07-03T23:35:08.380" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21257" PostTypeId="2" ParentId="21251" CreationDate="2023-07-04T00:21:03.493" Score="2" Body="&lt;p&gt;I ended up copying the GenBank (primary) column from the REBASE search results into an excel sheet. From there, I created a file with one accession per line called &lt;code&gt;Psych_Accessions.txt&lt;/code&gt;. I then ran the following command using NCBI's command line tools.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;epost -db nuccore -input Psych_Accessions.txt -format acc | efetch -format fasta &amp;gt; psych_genomes.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10792" LastActivityDate="2023-07-04T00:21:03.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21258" PostTypeId="2" ParentId="21034" CreationDate="2023-07-04T09:54:04.310" Score="0" Body="&lt;p&gt;If you haven't already, I recommend checking out the infercnv YouTube tutorial (&lt;a href=&quot;https://github.com/broadinstitute/inferCNV/wiki/Running-InferCNV&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/broadinstitute/inferCNV/wiki/Running-InferCNV&lt;/a&gt;) for detailed instructions on running InferCNV. Additionally, you can explore Uphyloplot2 (&lt;a href=&quot;https://github.com/harbourlab/uphyloplot2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/harbourlab/uphyloplot2&lt;/a&gt;) for building a tree from the output generated by InferCNV.&lt;/p&gt;&#xA;&lt;p&gt;I noticed that you didn't use the untreated sample as the normal reference for your CNV analysis. I'm curious to understand your reasoning behind this. In my case, I also have an untreated sample that isn't diploid, along with two treated samples. Initially, I planned to use the untreated sample as the &amp;quot;normal&amp;quot; reference. However, considering that InferCNV assumes a normal diploid reference, I'm now uncertain if using the untreated control would be appropriate.&lt;/p&gt;&#xA;&lt;p&gt;Please let me know your thoughts, any help would be much appreciated!&lt;/p&gt;&#xA;" OwnerUserId="17756" LastActivityDate="2023-07-04T09:54:04.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21259" PostTypeId="1" AcceptedAnswerId="21265" CreationDate="2023-07-04T15:08:50.587" Score="1" ViewCount="40" Body="&lt;p&gt;I am currently working on DE analysis of coding as well as non-coding features of &lt;em&gt;A. thaliana&lt;/em&gt; using the &lt;code&gt;edgeR&lt;/code&gt; package.&lt;/p&gt;&#xA;&lt;p&gt;Is the negative binomial method that is normally used for the DE analysis of genes suitable for non-coding features such as miRNA, lncRNA, snRNA, snoRNA, pseudogenes, transposons, uORFs and 5' and 3' UTRs? Or are there special statistical methods that are more suitable for modelling each of these, like I have read that the zero-inflated model used by &lt;code&gt;lncDIFF&lt;/code&gt; can better model lncRNA counts? I see that my miRNA counts matrix is abundant in zero counts and very low counts of genes. Should I use lncDIFF for that?&lt;/p&gt;&#xA;&lt;p&gt;Also, among the various DE test methods in edgeR such as &lt;code&gt;exactTest()&lt;/code&gt;, &lt;code&gt;glmFit()&lt;/code&gt; and &lt;code&gt;glmLRT()&lt;/code&gt;, &lt;code&gt;glmQLFit()&lt;/code&gt; and &lt;code&gt;glmQLFTest()&lt;/code&gt;, which is/are the one(s) most suitable for running DE tests on such non-coding features?&lt;/p&gt;&#xA;" OwnerUserId="17761" LastEditorUserId="17761" LastEditDate="2023-07-04T15:54:00.910" LastActivityDate="2023-07-05T15:21:08.993" Title="Statistical methods suitable for DE analysis of non coding RNAs" Tags="|rna-seq|differential-expression|edger|non-coding|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21261" PostTypeId="1" CreationDate="2023-07-04T16:15:27.307" Score="1" ViewCount="59" Body="&lt;p&gt;I am trying to do DE analysis of non-coding features of &lt;em&gt;A. thaliana&lt;/em&gt;. I find in the miRNA and lncRNA counts file that they are abundant in zero counts, and most of the non-zero counts are very low. Now, for the DE analysis of coding features, an inbuilt filtering method to remove the extremely low counts is available in edgeR, called &lt;code&gt;filterByExpr()&lt;/code&gt;. Is this function suitable for only those features with high counts like coding sequences, or is it suitable for non-coding features with low counts and low library sizes too?&#xA;If not, then what kind of filtering criteria should I use for filtering out genes with insignificant counts? Or should I filter out only the ones with row sums 0?&lt;/p&gt;&#xA;" OwnerUserId="17761" LastActivityDate="2024-03-31T20:03:20.367" Title="Filtering criteria for non-coding features with very low counts" Tags="|rna-seq|differential-expression|filtering|edger|non-coding|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21262" PostTypeId="1" CreationDate="2023-07-04T19:25:21.423" Score="0" ViewCount="134" Body="&lt;p&gt;I am new to Gene Ontology (GO) analysis and would like to classify and visualize the annotations. Here is the background and the steps I have taken so far:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I annotated the bacterial whole genome using Prokka. I performed&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Interproscan on the annotated genome using the Galaxy server.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The Interproscan results are in a tabular format, and column 14&#xA;contains the GO IDs.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Example GO IDs from column 14:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;column 14&#xA;GO:0003677&#xA;GO:0003677&#xA;GO:0005524|GO:0016887&#xA;GO:0005524|GO:0016020|GO:0055085&#xA;GO:0003723|GO:0046872&#xA;GO:0004521|GO:0004534|GO:0006396|GO:0008270|GO:0090501&#xA;etc &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to classify these annotations into the Cellular Component, Biological Process, and Molecular Function categories, and then plot a graph similar to the provided image.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/PgIqk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PgIqk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am comfortable using either R or Python for this analysis. Could you please provide guidance on how to achieve this using the preferred language?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance for your help!&lt;/p&gt;&#xA;" OwnerUserId="16745" LastEditorUserId="16745" LastEditDate="2023-07-04T19:38:43.657" LastActivityDate="2023-07-04T19:38:43.657" Title="Gene Ontology (GO) analysis and classification of bacterial whole genome annotations" Tags="|r|python|gene-ontology|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21265" PostTypeId="2" ParentId="21259" CreationDate="2023-07-05T15:21:08.993" Score="1" Body="&lt;p&gt;There is no question that &lt;strong&gt;lncDIFF&lt;/strong&gt; is the approach to use, especially if the alternative is performing &lt;strong&gt;edgeR&lt;/strong&gt; directly, and thats demonstrable (below). I don't think anyone would question the legitimacy of your approach. If you look at Fig 1 for the paper describing the test, &lt;a href=&quot;https://media.springernature.com/full/springer-static/image/art%3A10.1186%2Fs12864-019-5926-4/MediaObjects/12864_2019_5926_Fig1_HTML.png?as=webp&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; there is no question about it &lt;strong&gt;edgeR&lt;/strong&gt; used directly does not perform for this task at all. The paper describing lncDIFF is here,&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bmcgenomics.biomedcentral.com/articles/10.1186/s12864-019-5926-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bmcgenomics.biomedcentral.com/articles/10.1186/s12864-019-5926-4&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You present &lt;strong&gt;lncDIFF&lt;/strong&gt; as an option within &lt;strong&gt;edgeR&lt;/strong&gt;. This I don't know, but &lt;strong&gt;lncDIFF&lt;/strong&gt;  can be directly downloaded from GitHub, &lt;a href=&quot;https://github.com/qianli10000/lncDIFF&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/qianli10000/lncDIFF&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;em&gt;The one thing you have to be careful about is you are using plants, whilst these tests are designed around humans, see below for details.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;How it's tested: GLM methods versus an Exact test, e.g. Fisher's. Both are all valid tests. However, GLM methods are preferred  over an exact test, in my statistical experience.&lt;/p&gt;&#xA;&lt;p&gt;First, you need to obtain a &lt;code&gt;fit&lt;/code&gt; before you can perform a test, which ever GLM you choose.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;x &amp;lt;- model.matrix(...)&#xA;y &amp;lt;- estimateDisp(...)&#xA;fit &amp;lt;- glmFit(y, x, robust=TRUE)&#xA;lrt &amp;lt;- glmLRT(fit, contrast=c(-1,1,0,0,0)) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The basic issue is GLM likelihood ratio test versus quasi likelihood test, which the right approach? The clear answer is QLT because the authors of &lt;strong&gt;lncDIFF&lt;/strong&gt; developed the test around QLT (see the paper). That shouldn't be general advice outside &lt;strong&gt;lncDIFF&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The correct approach is to try both, if there major differences thats an issue: I would say in that scenario is consider  QLT - but only for &lt;strong&gt;lncDIFF&lt;/strong&gt;, as described. If there are no major differences you are good to go.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-07-05T15:21:08.993" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21266" PostTypeId="2" ParentId="21261" CreationDate="2023-07-05T15:32:02.980" Score="0" Body="&lt;p&gt;Partially answered before. I am assuming this is a &lt;strong&gt;lncDIFF&lt;/strong&gt; calculation. For the &lt;strong&gt;lncDIFF&lt;/strong&gt; paper it is unlikely to work as the following figure displays,&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bmcgenomics.biomedcentral.com/articles/10.1186/s12864-019-5926-4/figures/4&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bmcgenomics.biomedcentral.com/articles/10.1186/s12864-019-5926-4/figures/4&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, don't take my word for this, test it. You should test the option explicitly via &lt;code&gt;glmQLFTest&lt;/code&gt; and &lt;code&gt;glmLRT&lt;/code&gt;. If you perform either of these tests with or without the filter the better fit (one not rejected) is the correct answer. I strongly predict the filtered data will be rejected and unfiltered data  (hopefully) will pass.&lt;/p&gt;&#xA;&lt;p&gt;The question of what is better &lt;code&gt;glmQLFTest&lt;/code&gt; versus &lt;code&gt;glmLRT&lt;/code&gt; is discussed in the previous answer, here &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/21259/statistical-methods-suitable-for-de-analysis-of-non-coding-rnas&quot;&gt;Statistical methods suitable for DE analysis of non coding RNAs&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Just keep in mind this is plant data and these tests are developed for humans.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-07-05T17:02:37.490" LastActivityDate="2023-07-05T17:02:37.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21267" PostTypeId="1" AcceptedAnswerId="21278" CreationDate="2023-07-05T17:02:23.690" Score="3" ViewCount="52" Body="&lt;p&gt;I need an algorithm to process protein chains. The volume size would probably not exceed 30x30x30 angstrom^3.&lt;/p&gt;&#xA;&lt;p&gt;Say, I have a point cloud of 50,000 points in a 3D space.&lt;/p&gt;&#xA;&lt;p&gt;I want to &lt;strong&gt;count&lt;/strong&gt; the number of neighbors &lt;strong&gt;each&lt;/strong&gt; point {(x&lt;sub&gt;1&lt;/sub&gt;,y&lt;sub&gt;1&lt;/sub&gt;,z&lt;sub&gt;1&lt;/sub&gt;), (x&lt;sub&gt;2&lt;/sub&gt;,y&lt;sub&gt;2&lt;/sub&gt;,z&lt;sub&gt;2&lt;/sub&gt;), (x&lt;sub&gt;3&lt;/sub&gt;,y&lt;sub&gt;3&lt;/sub&gt;,z&lt;sub&gt;3&lt;/sub&gt;), ... ...,(x&lt;sub&gt;N&lt;/sub&gt;,y&lt;sub&gt;N&lt;/sub&gt;,z&lt;sub&gt;N&lt;/sub&gt;)} has at radii {r&lt;sub&gt;1&lt;/sub&gt;, r&lt;sub&gt;2&lt;/sub&gt;, r&lt;sub&gt;3&lt;/sub&gt;, ..., r&lt;sub&gt;M&lt;/sub&gt;}.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Which algorithm should I use?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Additional information:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you need to do this count online/sequentially or all at once?&lt;br /&gt;&#xA;All once.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;What restrictions might there be on the radii?&lt;br /&gt;&#xA;The radii will be 4.0, 4.5, 5.0, 5.5, 6.0, 6.5, 7.0 angstrom.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Do you need to do this just once in this particular case or do you need a general algorithm?&lt;br /&gt;&#xA;I will use this to process protein sequences. Not DNA/RNA, though.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For a general algorithm, what's the likely range of the number of points?&lt;br /&gt;&#xA;The range would be 50,000 at max (although, the maximum length of available protein is 38,000 residues).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are you expecting there to be relatively few or relatively many neighbors?&lt;br /&gt;&#xA;I am expecting relatively few neighbors, in the range of 0-50.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;NOTE:&lt;/strong&gt; I prefer a hash-based algorithm as it doesn't require additional data structures like KD tree, or octree.&lt;/p&gt;&#xA;" OwnerUserId="12686" LastActivityDate="2023-07-07T22:14:51.453" Title="Algorithm for counting neighbors" Tags="|proteins|protein-structure|algorithms|3d-structure|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21268" PostTypeId="1" CreationDate="2023-07-05T17:40:49.487" Score="-2" ViewCount="78" Body="&lt;p&gt;I have a dataset that contains IDs and associated Gene Ontology (GO) IDs. Some of the IDs have multiple GO IDs assigned to them, and I'm unsure how to handle this situation. I would like to determine whether Should I split all Go id or I should use the first or last GO ID in the hierarchy when assigning the GO ID to the ID.&lt;/p&gt;&#xA;&lt;p&gt;Here is an example of my data:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;3667 FEHLGCFF_05007                                             GO:0006637|GO:0047617&#xA;3673 FEHLGCFF_05919                                  GO:0016020|GO:0022857|GO:0055085&#xA;3684 FEHLGCFF_00369                                             GO:0022857|GO:0055085&#xA;3695 FEHLGCFF_00369                                  GO:0016020|GO:0022857|GO:0055085&#xA;3696 FEHLGCFF_00369                                                        GO:0022857&#xA;3702 FEHLGCFF_06641                                                        GO:0016627&#xA;3707 FEHLGCFF_06641                                                        GO:0003995&#xA;3721 FEHLGCFF_04414                                                        GO:0046872&#xA;3725 FEHLGCFF_02968                                  GO:0003700|GO:0006352|GO:0006355&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How should I handle cases where an ID has multiple GO IDs assigned to it? Should I split all Go id or Should I use the first GO ID, the last GO ID, or is there another approach I should consider?&lt;/p&gt;&#xA;&lt;p&gt;Any guidance or best practices would be greatly appreciated. Thank you!&lt;/p&gt;&#xA;&lt;p&gt;Here is My code using R, it it Ok&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(tidyverse)&#xA;&#xA;# Read the go-basic.obo file&#xA;go_data &amp;lt;- readLines(&amp;quot;go-basic.obo&amp;quot;)&#xA;&#xA;# Extract the GO ID and Gene ID from the unique_data dataframe&#xA;unique_data &amp;lt;- unique_data %&amp;gt;%&#xA;  separate_rows(GO_ID, sep = &amp;quot;\\|&amp;quot;) %&amp;gt;%&#xA;  select(V1, GO_ID) # Replace with the correct column names from your unique_data dataframe&#xA;&#xA;# Create an empty dataframe to store the results&#xA;result &amp;lt;- data.frame(Gene_ID = character(), GO_ID = character(), Name = character(), Namespace = character(), stringsAsFactors = FALSE)&#xA;&#xA;# Process each line in the go-basic.obo file&#xA;for (i in 1:(length(go_data) - 1)) {&#xA;  line &amp;lt;- go_data[i]&#xA;  &#xA;  if (grepl(&amp;quot;^id:&amp;quot;, line)) {&#xA;    go_id &amp;lt;- strsplit(line, &amp;quot; &amp;quot;)[[1]][2]&#xA;    &#xA;    # Check if the current GO ID exists in the unique_data dataframe&#xA;    if (go_id %in% unique_data$GO_ID) {&#xA;      # Filter the unique_data dataframe for the current GO ID&#xA;      filtered_data &amp;lt;- unique_data %&amp;gt;% filter(GO_ID == go_id)&#xA;      &#xA;      # Extract the name and namespace information&#xA;      name &amp;lt;- &amp;quot;&amp;quot;&#xA;      namespace &amp;lt;- &amp;quot;&amp;quot;&#xA;      &#xA;      while (!grepl(&amp;quot;^is_a:&amp;quot;, go_data[i])) {&#xA;        if (grepl(&amp;quot;^name:&amp;quot;, go_data[i])) {&#xA;          name &amp;lt;- strsplit(go_data[i], &amp;quot;name: &amp;quot;)[[1]][2]&#xA;        } else if (grepl(&amp;quot;^namespace:&amp;quot;, go_data[i])) {&#xA;          namespace &amp;lt;- strsplit(go_data[i], &amp;quot;namespace: &amp;quot;)[[1]][2]&#xA;        }&#xA;        i &amp;lt;- i + 1&#xA;      }&#xA;      &#xA;      # Add the information to the result dataframe&#xA;      temp_df &amp;lt;- data.frame(Gene_ID = filtered_data$V1, GO_ID = go_id, Name = name, Namespace = namespace, stringsAsFactors = FALSE) # Replace with the correct column names&#xA;      result &amp;lt;- bind_rows(result, temp_df)&#xA;    }&#xA;  }&#xA;}&#xA;&#xA;# Print the result dataframe&#xA;result&#xA;nrow(result)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16745" LastEditorUserId="3694" LastEditDate="2023-09-07T11:49:07.300" LastActivityDate="2023-09-07T11:49:07.300" Title="Assigning multiple GO IDs to an ID: Should I split or use the first or last ID in the hierarchy?" Tags="|r|gene-ontology|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21270" PostTypeId="1" CreationDate="2023-07-06T12:08:52.050" Score="0" ViewCount="111" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/2W5L6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/2W5L6.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I tried to quantify the number of cells of Fzd9+ Macrophage.&lt;/p&gt;&#xA;&lt;p&gt;how can I know the cell count?&lt;/p&gt;&#xA;&lt;p&gt;thank you&lt;/p&gt;&#xA;" OwnerUserId="17771" LastActivityDate="2023-07-06T12:51:34.640" Title="is it possible to count cell in Violin plot in seurat?" Tags="|rna-seq|scrnaseq|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21271" PostTypeId="2" ParentId="21270" CreationDate="2023-07-06T12:51:34.640" Score="3" Body="&lt;p&gt;This can be a good starting point: &lt;code&gt;table(your_seurat_object@meta.data$the_column_that_has_the_labels_of_interest)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Basically you will be counting the number of rows (each single cell is a row) in the &lt;code&gt;meta.data&lt;/code&gt; slot of your Seurat object for each level of your column of interest (for example cell type annotations).&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2023-07-06T12:51:34.640" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21272" PostTypeId="1" CreationDate="2023-07-06T22:14:23.530" Score="2" ViewCount="24" Body="&lt;p&gt;I have bulk RNAseq data from two strains of mice from Jackson Lab: C57BL/6 and B6.SJL.&lt;/p&gt;&#xA;&lt;p&gt;The former expresses a &lt;code&gt;Ptprc-b&lt;/code&gt; allele and the latter expresses a &lt;code&gt;Ptprc-a&lt;/code&gt; allele.&lt;/p&gt;&#xA;&lt;p&gt;They are supposedly isogenic at all other loci. But to the best of my knowledge, there is no empirical evidence of this.&lt;/p&gt;&#xA;&lt;p&gt;My question is, how would I go about finding all the &lt;strong&gt;homozygous&lt;/strong&gt; &lt;em&gt;single nucleotide&lt;/em&gt; &lt;strong&gt;variations&lt;/strong&gt; &lt;strong&gt;(if there are any)&lt;/strong&gt; in the coding sequences of the two strains from my RNAseq data.&lt;/p&gt;&#xA;&lt;p&gt;I should mention that the RNAseq data are paired-end illumina reads ~150bp long.&lt;/p&gt;&#xA;" OwnerUserId="7056" LastActivityDate="2023-07-06T22:14:23.530" Title="Feasible to find genetic variations of two samples using RNAseq data?" Tags="|rna-seq|snp|variant-calling|variation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21273" PostTypeId="1" AcceptedAnswerId="21291" CreationDate="2023-07-07T04:25:17.420" Score="2" ViewCount="58" Body="&lt;p&gt;I am currently working with the non-coding features of &lt;em&gt;A. thaliana&lt;/em&gt;, and trying to get the DE features. Among the three DE test methods in &lt;code&gt;edgeR&lt;/code&gt; such as &lt;code&gt;exactTest()&lt;/code&gt;, &lt;code&gt;glmFit()&lt;/code&gt; and &lt;code&gt;glmLRT()&lt;/code&gt;, &lt;code&gt;glmQLFit()&lt;/code&gt; and &lt;code&gt;glmQLFTest()&lt;/code&gt;, which is/are the one(s) most suitable for running DE tests on miRNA, snRNA, snoRNA, pseuogenes, transposons and 5' and 3'-UTRs?&#xA;Should I be using any other DE analysis tool instead of &lt;code&gt;edgeR&lt;/code&gt; for some specific non-coding feature, like there's lncDIFF for lncRNA?&lt;/p&gt;&#xA;" OwnerUserId="17761" LastActivityDate="2023-07-12T15:52:54.050" Title="DE analysis tool and method for non-coding features" Tags="|rna-seq|differential-expression|edger|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21274" PostTypeId="2" ParentId="21267" CreationDate="2023-07-07T04:39:52.797" Score="0" Body="&lt;p&gt;K Dimensional Tree (KD-Tree) data structure might help:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biorxiv.org/content/10.1101/605907v2.full.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biorxiv.org/content/10.1101/605907v2.full.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6649" LastActivityDate="2023-07-07T04:39:52.797" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21275" PostTypeId="1" CreationDate="2023-07-07T12:48:53.360" Score="3" ViewCount="117" Body="&lt;p&gt;I am using the Deseq2 package to perform a DE analysis on multiple factors.&lt;/p&gt;&#xA;&lt;p&gt;So, basically to perform the analysis I consider patients characterized by the Brugada Syndrome type 1 and other patients that instead are controls.&lt;/p&gt;&#xA;&lt;p&gt;In order to have equal numbers of PZ and CTRs I included in my analysis all the PZ of type 1, that in this case are 8 samples, and I selected 8 random CTRs among the full dataset the CTRs that I have for this batch (in total are 40).&lt;/p&gt;&#xA;&lt;p&gt;To select the random CTRs I used the sample_n function:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;random_CTR_samples_IGATECH&amp;lt;-sample_n(samples_IGATECH_ALL_CTR,8)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Anyway I created the design matrix that looks like this including also the Age variable as well as the Gender one since I want to test also the effect of those in my data, (NON_INDUCIBLE)= CTR:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;       Sample BrS_BASELINE_PATTERN Gender Age&#xA;1  DG1340                    1      M  44&#xA;2  DG1383                    1      F  35&#xA;3  DG1453                    1      F  55&#xA;4  DG1455                    1      F  53&#xA;5  DG1473                    1      M  33&#xA;6  DG1557                    1      M  45&#xA;7  DG1578                    1      F  57&#xA;8  DG1590                    1      M  42&#xA;9  DG1380       NON_INDUCIBILE      M  20&#xA;10 DG1381       NON_INDUCIBILE      F  52&#xA;11 DG1382       NON_INDUCIBILE      M  49&#xA;12 DG1565       NON_INDUCIBILE      M  30&#xA;13 DG1594       NON_INDUCIBILE      F  42&#xA;14 DG1620       NON_INDUCIBILE      M  40&#xA;15 DG1633       NON_INDUCIBILE      F  62&#xA;16 DG1652       NON_INDUCIBILE      M  21&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and I filtered the Count matrix for the samples present in the design one. So I have&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;head(Counts_IGATECH_ALL_1_random_CTR)&#xA;            DG1340 DG1383 DG1453 DG1455 DG1473 DG1557 DG1578 DG1590 DG1380 DG1381 DG1382 DG1565 DG1594 DG1620 DG1633 DG1652&#xA;ENSG00000114771      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0&#xA;ENSG00000129673      0      1      0      0      1      0      0      1      0      1      0      0      0      0      0      0&#xA;ENSG00000144452      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0&#xA;ENSG00000143921      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0&#xA;ENSG00000173357      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0&#xA;ENSG00000161103      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0      0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I run the &lt;code&gt;DESeqDataSetFromMatrix&lt;/code&gt; function I get this error:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;converting counts to integer mode&#xA;factor levels were dropped which had no samples&#xA;Error in checkFullRank(modelMatrix) : &#xA;  the model matrix is not full rank, so the model cannot be fit as specified.&#xA;  One or more variables or interaction terms in the design formula are linear&#xA;  combinations of the others and must be removed.&#xA;&#xA;  Please read the vignette section 'Model matrix not full rank':&#xA;&#xA;  vignette('DESeq2')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and the function i ran is:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;dds_IGATECH_1_random_CTR&amp;lt;- DESeqDataSetFromMatrix(countData = round(Counts_IGATECH_ALL_1_random_CTR),&#xA;                                               colData = design_IGATECH_1_random_CTR,&#xA;                                               design = ~ Age + Gender + BrS_BASELINE_PATTERN)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I revised my design matrix multiple times and tried to change also the combinations of the CTRs to have a different one and see if that would solve the problem but it didn't work, and I cannot figure out what it can be.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance for any suggestion.&lt;/p&gt;&#xA;&lt;p&gt;Update&lt;/p&gt;&#xA;&lt;p&gt;Design matrix:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;dput(design_IGATECH_1_random_CTR)&#xA;&#xA;structure(list(Sample = c(&amp;quot;DG1340&amp;quot;, &amp;quot;DG1383&amp;quot;, &amp;quot;DG1453&amp;quot;, &amp;quot;DG1455&amp;quot;, &#xA;&amp;quot;DG1473&amp;quot;, &amp;quot;DG1557&amp;quot;, &amp;quot;DG1578&amp;quot;, &amp;quot;DG1590&amp;quot;, &amp;quot;DG1376&amp;quot;, &amp;quot;DG1379&amp;quot;, &amp;quot;DG1573&amp;quot;, &#xA;&amp;quot;DG1575&amp;quot;, &amp;quot;DG1579&amp;quot;, &amp;quot;DG1598&amp;quot;, &amp;quot;DG1627&amp;quot;, &amp;quot;DG1633&amp;quot;), BrS_BASELINE_PATTERN = structure(c(1L, &#xA;1L, 1L, 1L, 1L, 1L, 1L, 1L, 3L, 3L, 3L, 3L, 3L, 3L, 3L, 3L), levels = c(&amp;quot;1&amp;quot;, &#xA;&amp;quot;2&amp;quot;, &amp;quot;NON_INDUCIBILE&amp;quot;), class = &amp;quot;factor&amp;quot;), Gender = structure(c(2L, &#xA;1L, 1L, 1L, 2L, 2L, 1L, 2L, 1L, 2L, 1L, 1L, 2L, 2L, 2L, 1L), levels = c(&amp;quot;F&amp;quot;, &#xA;&amp;quot;M&amp;quot;), class = &amp;quot;factor&amp;quot;), Age = structure(c(23L, 15L, 32L, 31L, &#xA;13L, 24L, 34L, 21L, 29L, 27L, 30L, 10L, 19L, 28L, 18L, 37L), levels = c(&amp;quot;17&amp;quot;, &#xA;&amp;quot;20&amp;quot;, &amp;quot;21&amp;quot;, &amp;quot;22&amp;quot;, &amp;quot;23&amp;quot;, &amp;quot;24&amp;quot;, &amp;quot;25&amp;quot;, &amp;quot;26&amp;quot;, &amp;quot;28&amp;quot;, &amp;quot;29&amp;quot;, &amp;quot;30&amp;quot;, &amp;quot;32&amp;quot;, &#xA;&amp;quot;33&amp;quot;, &amp;quot;34&amp;quot;, &amp;quot;35&amp;quot;, &amp;quot;37&amp;quot;, &amp;quot;38&amp;quot;, &amp;quot;39&amp;quot;, &amp;quot;40&amp;quot;, &amp;quot;41&amp;quot;, &amp;quot;42&amp;quot;, &amp;quot;43&amp;quot;, &amp;quot;44&amp;quot;, &#xA;&amp;quot;45&amp;quot;, &amp;quot;46&amp;quot;, &amp;quot;48&amp;quot;, &amp;quot;49&amp;quot;, &amp;quot;50&amp;quot;, &amp;quot;51&amp;quot;, &amp;quot;52&amp;quot;, &amp;quot;53&amp;quot;, &amp;quot;55&amp;quot;, &amp;quot;56&amp;quot;, &amp;quot;57&amp;quot;, &#xA;&amp;quot;58&amp;quot;, &amp;quot;61&amp;quot;, &amp;quot;62&amp;quot;, &amp;quot;63&amp;quot;, &amp;quot;64&amp;quot;, &amp;quot;69&amp;quot;), class = &amp;quot;factor&amp;quot;)), row.names = c(NA, &#xA;-16L), class = &amp;quot;data.frame&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The NON_INDUCIBLE data are different since i have generated a new design matrix again, but the format is the same&lt;/p&gt;&#xA;" OwnerUserId="17776" LastEditorUserId="650" LastEditDate="2023-07-07T15:37:48.003" LastActivityDate="2023-07-07T15:37:48.003" Title="Issue with Differential Gene expression analysis with Deseq2" Tags="|r|rna-seq|differential-expression|deseq2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21276" PostTypeId="2" ParentId="21275" CreationDate="2023-07-07T13:54:04.643" Score="4" Body="&lt;p&gt;This typically happens when Age is a factor rather than numeric column, so each unique row is nested with itself. Design &lt;code&gt;~BrS_BASELINE_PATTERN + Gender&lt;/code&gt; is fine, there is no nesting, so this narrows it down to the Age column. Cannot check as you do not provide reproducible data via &lt;code&gt;dput()&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-07-07T13:54:04.643" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21277" PostTypeId="1" CreationDate="2023-07-07T15:52:13.170" Score="5" ViewCount="91" Body="&lt;p&gt;I came across this gene &lt;strong&gt;Pepc&lt;/strong&gt; (peptidase C)&lt;/p&gt;&#xA;&lt;p&gt;NCBI gene database has a page for it here &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene/109616&quot; rel=&quot;noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/gene/109616&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But it does not contain any information on its coding sequence or genomic location. It just says it sits on chromosome 1.&lt;/p&gt;&#xA;&lt;p&gt;Can someone familiar with how the reference genome is annotated explain why this is?&lt;/p&gt;&#xA;" OwnerUserId="7056" LastEditorUserId="3694" LastEditDate="2023-07-07T18:54:52.017" LastActivityDate="2023-08-23T22:54:34.387" Title="Why is there gene without annotated location?" Tags="|sequence-annotation|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21278" PostTypeId="2" ParentId="21267" CreationDate="2023-07-07T22:14:51.453" Score="4" Body="&lt;p&gt;The usual method is dividing the space into cells, in your case 7Å cubes, assigning atoms to cells, and searching for neighbors only in 3x3x3=27 cells. This method is described in Wikipedia as &lt;a href=&quot;https://en.wikipedia.org/wiki/Cell_lists&quot; rel=&quot;nofollow noreferrer&quot;&gt;cell lists&lt;/a&gt;, although that name is not as widely used as the method itself.&lt;/p&gt;&#xA;&lt;p&gt;If you analyze experimental protein structures note that they may have alternative conformations (alternative locations of the same atom), and that the structure in the file represents an &lt;a href=&quot;https://pdb101.rcsb.org/learn/guide-to-understanding-pdb-data/biological-assemblies#Anchor-Asym&quot; rel=&quot;nofollow noreferrer&quot;&gt;asymmetric unit&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2023-07-07T22:14:51.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21279" PostTypeId="1" CreationDate="2023-07-08T07:13:35.883" Score="1" ViewCount="53" Body="&lt;p&gt;I have been looking all over the web to find some answers to my problem but unfortunately, I was unsuccessful. I wish to determine whether an a priori defined set of genes in my case genes associated with Epidermolysis bullosa shows statistically significant, concordant differences between two biological states (e.g. phenotypes).&lt;/p&gt;&#xA;&lt;p&gt;I have a list of genes from my own experiment and a predefined list of genes associated with Epidermolysis bullosa, and want to find out if the genes of Epidermolysis bullosa are overrepresented in my list.&lt;/p&gt;&#xA;&lt;p&gt;Here is an article about GSEA, which is related to what I want to do:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.pnas.org/doi/epdf/10.1073/pnas.0506580102&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.pnas.org/doi/epdf/10.1073/pnas.0506580102&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/xRqZF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xRqZF.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;so I have the following data:&#xA;I'm researching patients with severe or mild disease; in this research, I have found a list of  200 genes after conducting a SKAT test in R (&lt;a href=&quot;https://www.rdocumentation.org/packages/SKAT/versions/2.2.4/topics/SKAT&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rdocumentation.org/packages/SKAT/versions/2.2.4/topics/SKAT&lt;/a&gt;) with a p-value &amp;lt;0.05 ( which I presume can cause a severe disease instead of mild  ) I want to focus on this 200 significant genes that can perhaps affect the disease level, this genes can be  -&amp;gt; the L list as described in the article above.&#xA;Also, the S list of predefined genes ( as is called in the article above )is associated with Epidermolysis bullosa disease. and I want to know if the genes from the Epidermolysis bullosa are overrepresented in my L list.&#xA;After reading the article and many more resources, I understood that to perform GSEA  (&lt;a href=&quot;https://www.gsea-msigdb.org/gsea/index.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.gsea-msigdb.org/gsea/index.jsp&lt;/a&gt;), I had to have gene expression data which  i don't have .&lt;/p&gt;&#xA;&lt;p&gt;so I wish to understand if there is a tool that could do a gene set comparison without having the expression data whether it's in R or Python that uses my gene list and predefined list ( in my case associated with Epidermolysis bullosa)  ,thank you :)&lt;/p&gt;&#xA;" OwnerUserId="17779" LastEditorUserId="73" LastEditDate="2023-07-10T05:58:50.773" LastActivityDate="2023-07-10T21:23:59.913" Title="Gene Set Comparison Without Expression Data" Tags="|r|gene|statistics|gsea|go-enrichment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21280" PostTypeId="2" ParentId="21250" CreationDate="2023-07-08T08:10:15.497" Score="0" Body="&lt;p&gt;I don't know lipids, but if 1/3 of your data set comprises duplication thats a skewed distribution with a low mean and high variance (of the duplications). It is likely to follow a negative binomial distribution. You need to plot a histogram of your data and you will find that 0 (no duplicates) is massively represented, 1 duplicate is next in frequency, 2 duplicates are lower frequency ... and on until a very small proportion of the data comprises a large number of duplicates. This is a very common distribution throughout biology.&lt;/p&gt;&#xA;&lt;p&gt;The advice is:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Do not remove the duplicates because they are a vital part of the distribution, i.e. biological information.&lt;/li&gt;&#xA;&lt;li&gt;Construct a histogram of each data set.&lt;/li&gt;&#xA;&lt;li&gt;Don't take &amp;quot;an average&amp;quot;, i.e. don't take the mean of skewed data, thats very important if it has a high variance.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Easy solution&lt;/strong&gt; The easiest way to deal with this data is take a median and construct upper and lower quartiles around the median (IQR), especially the frequency (i.e. the distribution duplicates). This will give an approximate comparison between data sets. This alone might work and certainly gives you &amp;quot;cut-offs&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;If it is a negative binomial distribution then testing the significance between two negative binomial distributions beyond the IQR is an advanced area of statistics and a separate question.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-07-09T15:36:21.173" LastActivityDate="2023-07-09T15:36:21.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21281" PostTypeId="1" CreationDate="2023-07-08T10:52:06.113" Score="1" ViewCount="16" Body="&lt;p&gt;We have a cohort of around 3k exomes, most of them from different capture kits, such as Agilent v6, v8, Nextera, Twist, Clinical Research exome and many others. Right now we are creating genome db on a per chromosome basis and combining the vcf.&lt;/p&gt;&#xA;&lt;p&gt;We were wondering the following questions -&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Will the mix of capture kits introduce more false positive variant calls in the recalibration step&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If we restrict the genomics DB to only include regions covered by capture kit, will we see a performance boost (Ex. instead of entire chr2, we restrict it to regions covered by all capture kits).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="6561" LastActivityDate="2023-07-08T10:52:06.113" Title="GenomicsDB and recalibration recommended for cohort of mixed exome (GATK)" Tags="|genomics|gatk|exome|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21282" PostTypeId="1" CreationDate="2023-07-09T17:56:30.020" Score="1" ViewCount="65" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.reddit.com/r/bioinformatics/comments/14vph0q/why_does_my_hwe_qq_plot_have_extreme_deviation/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reddit&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have recently completed my thesis and one of the comments was that I report on why this graph looks this way. I have tried to find a reason but the closest I can come to is that it means that there are false positives? Or that this is possibly due to population stratification?&lt;/p&gt;&#xA;&lt;p&gt;The trait of interest was alcohol dependency with covariates of sex and age, in a non african american population.&lt;/p&gt;&#xA;&lt;p&gt;I am also unsure as to how this will impact my research or the implications of this?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/eKv98.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/eKv98.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I don't understand why this population (non-African American) &amp;quot;reacted&amp;quot; this way - do I provide a different null model? The previous populations had &amp;quot;relatively normal&amp;quot; plots. They were all run through the same pipeline.&lt;/p&gt;&#xA;&lt;p&gt;How the data was calculated:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;There were 1061 individuals in the non-African American dataset. At QC phase 0, there were 2 individuals who had discordant sex information and 87 duplicate SNPs. When the MAF value was set to 0.000015, 125708 SNPs were removed when the MAF. Both the mind and geno parameters were set to 0.2, resulting in 127357 and 0 SNPs to be removed respectively. When setting the HWE to &lt;span class=&quot;math-container&quot;&gt;$1 \times 10^{-12}$&lt;/span&gt;, 4495 SNPs were removed. After all these parameters were set and the pipeline completed, there were 851773 SNPs left from 1044 individuals.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="5066" LastEditorUserId="73" LastEditDate="2023-07-10T22:23:23.977" LastActivityDate="2023-07-10T22:26:53.493" Title="Why does my HWE QQ plot have extreme deviation and what does it mean?" Tags="|genome|snp|gwas|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21283" PostTypeId="2" ParentId="21282" CreationDate="2023-07-09T19:58:29.503" Score="1" Body="&lt;p&gt;This much deviation from the expected distribution looks like a modeling issue, i.e. that the statistical model use to represent the null hypothesis is incorrect.&lt;/p&gt;&#xA;&lt;p&gt;I can't see &lt;em&gt;any&lt;/em&gt; values that lie along the expected line. It's more common to see a generally linear trend, with a few outliers just at the end.&lt;/p&gt;&#xA;&lt;p&gt;It could very likely be due to population stratification. You have defined this population as &amp;quot;non-African American&amp;quot;, which seems to me to be quite a vague definition that would capture a lot of different population groups with distinct genetic histories.&lt;/p&gt;&#xA;&lt;p&gt;In my own research &lt;a href=&quot;https://doi.org/10.1007/s10038-006-0094-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;many years ago&lt;/a&gt;, I found substantially different haplotypes within the ADH gene cluster between Māori and European individuals, and I could imagine that not adjusting for population structure in other populations would have an impact on association scores.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-07-10T22:26:53.493" LastActivityDate="2023-07-10T22:26:53.493" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21284" PostTypeId="2" ParentId="21279" CreationDate="2023-07-09T22:17:14.600" Score="1" Body="&lt;p&gt;You can't do a GSEA on a set alone because the differentiating statistic (or, occasionally, the rank) is used to calculate the enrichment score that is produced by the GSEA algorithm.&lt;/p&gt;&#xA;&lt;p&gt;The other gene list type tools that I'm aware of (that my biologist colleagues use) are &lt;a href=&quot;https://david.ncifcrf.gov/tools.jsp&quot; rel=&quot;nofollow noreferrer&quot;&gt;DAVID&lt;/a&gt; and &lt;a href=&quot;http://pantherdb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PANTHER&lt;/a&gt;. But those are general discovery algorithms, not for comparing specific lists.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-07-10T21:23:59.913" LastActivityDate="2023-07-10T21:23:59.913" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21285" PostTypeId="1" CreationDate="2023-07-10T09:38:20.877" Score="3" ViewCount="330" Body="&lt;p&gt;I have a list of about 100,000 rsids and I want to get their genomic coordinates on the GRCh38 genome build. Is there a command line tool that allows me to do this? If yes, which one?&lt;/p&gt;&#xA;&lt;p&gt;I have tried bcftools but, given the error message I got, I believe that it does not support search by rsid:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view -R rsids_chr1_subset.txt -f PASS -v snps dbSNP_v156_GRCh38.p14.vcf.gz &amp;gt; rsids_chr1.vcf&#xA;[E::bcf_sr_regions_init] Could not parse 1-th line of file rsids_chr1_subset.txt, using the columns 1,2[,-1]&#xA;Failed to read the regions: rsids_chr1_subset.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17128" LastActivityDate="2023-10-07T05:39:56.463" Title="How can I retrieve the GRCh38 coordinates of a list of rsids?" Tags="|bcftools|rsid|" AnswerCount="3" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21286" PostTypeId="2" ParentId="21285" CreationDate="2023-07-10T11:02:00.933" Score="2" Body="&lt;p&gt;rsids_chr1.txt contains a list of rsids, one per line.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view --include ID==@rsids_chr1.txt dbSNP_v156_GRCh38.p14.vcf.gz -Ou | bcftools query -f '%CHROM  %ID %POS  %REF  %ALT{0}\n' &amp;gt; dbSNP_v156_GRCh38.p14_AF.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17128" LastActivityDate="2023-07-10T11:02:00.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21287" PostTypeId="2" ParentId="21285" CreationDate="2023-07-10T13:14:09.930" Score="5" Body="&lt;p&gt;Several years ago I created rsidx to address this particular use case. &lt;a href=&quot;https://github.com/bioforensics/rsidx&quot; rel=&quot;noreferrer&quot;&gt;https://github.com/bioforensics/rsidx&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# VCF should be sorted by genomic coordinates and indexed by tabix&#xA;rsidx index dbSNP151_GRCh38.vcf.gz dbSNP151_GRCh38.rsidx&#xA;rsidx search dbSNP151_GRCh38.vcf.gz dbSNP151_GRCh38.rsidx rs3114908 rs10756819&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="96" LastActivityDate="2023-07-10T13:14:09.930" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21288" PostTypeId="2" ParentId="21282" CreationDate="2023-07-10T14:13:51.450" Score="0" Body="&lt;p&gt;The level of discordance is extreme, particularly for humans. This is either a publication in Nature/Science and massive advance in the field (especially humans), or the genetic factors have massive confounders.&lt;/p&gt;&#xA;&lt;p&gt;In theory, I'm aware there must be a genetic component to alcoholism, there is a biochemical component about the rate of processing alcohol toxins. However, non-genetic (especially social/community) affects of alcoholism could predominate.&lt;/p&gt;&#xA;&lt;p&gt;What I personally think is happening is there are very strong non-genetic confounders which are disrupting the model and this is getting confused with neutral HWE. For example, if alcoholism predominates in a community or social group the transmission is social not genetic, so the confounder is extreme because they are genetically related, but the gene(s) are not being inherited. In phylogenetic terms it will make the &amp;quot;gene&amp;quot; appear far higher up (far deeper) in the tree than it actually is. The &amp;quot;lone&amp;quot; drinker is probably the model, but group behaviour and peer-pressure is probably more realistic. Equally community effects could be suppressing alcoholism, such as religious belief.&lt;/p&gt;&#xA;&lt;p&gt;Thus if a non-genetic model of alcoholism, (loads of social factors) resulted in a stronger fit by for example 3 logs difference, whilst the genetic model is 50 logs difference, then that is the answer and points to the need for careful population sampling for GWAS. Upvotes J Human Gen is cool.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;As a disclaimer, I do not understand not human genetics outside immunology. I do understand multiple linear regression models and failing to account for a pivotal factor(s) causes unusual effects. An obvious example in this case is religious belief: thus alcoholism genes could be present but the environment is not presented. For example, you might conclude long hair caused a reduction in human growth because they tightly correlated in humans. However, once gender is a factored into the model, the result is clear hair-length associates with gender not height.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-07-10T16:02:57.867" LastActivityDate="2023-07-10T16:02:57.867" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21289" PostTypeId="2" ParentId="18881" CreationDate="2023-07-10T16:39:27.927" Score="3" Body="&lt;pre&gt;&lt;code&gt;seqkit replace -p &amp;quot;$&amp;quot; -r &amp;quot;;&amp;quot; my.fasta &amp;gt; my.renamed.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Where &lt;code&gt;-p&lt;/code&gt; is the pattern to match the end of the header text &lt;code&gt;$&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;Where &lt;code&gt;-r&lt;/code&gt; is the replacement pattern&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinf.shenwei.me/seqkit/usage/#replace&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioinf.shenwei.me/seqkit/usage/#replace&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Seqkit is an extremely powerful toolkit for working with fasta/fastq files. It can a lot with a single cli command.&lt;/p&gt;&#xA;" OwnerUserId="490" LastActivityDate="2023-07-10T16:39:27.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21290" PostTypeId="1" CreationDate="2023-07-10T20:02:13.713" Score="2" ViewCount="148" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9568785/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt; and the &lt;a href=&quot;https://support.bioconductor.org/p/9153107/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioconductor Forum&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I want to find differential accessibility between conditions (WS, WF, MS, MF). I can compare any pair conditions without error except this pair MS vs MF. Data is the bam and broadpeak files from ATAC seq pipeline and the code is like a pipeline of the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/DiffBind/inst/doc/DiffBind.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DiffBind&lt;/a&gt; R package. The sample is human and uses Illumina sequencing technology.&lt;/p&gt;&#xA;&lt;p&gt;Would you please suggest what wrong in this case? I pretty much run the same code as I used for another data set and didn't have this error. Thank you so much!&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;result &amp;lt;- dba(sampleSheet=samples)&#xA;result &amp;lt;- dba.blacklist(result) &#xA;result &amp;lt;- dba.count(result, bParallel=FALSE)&#xA;result &amp;lt;- dba.count(result)&#xA;result &amp;lt;- dba.normalize(result)&#xA;result &amp;lt;- dba.contrast(result, minMembers=2)&#xA;result &amp;lt;- dba.analyze(result)&#xA;result &amp;lt;- dba.plotProfile(result)&#xA;&#xA;Generating report-based DBA object... Error: No valid contrasts/methods specified.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There is post about this error but I still got error when trying those solutions. Such as the order of the function, the path to the bam and broad peak files in sample sheet.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;dba.show(result, bContrasts=TRUE)&#xA;     Factor    Group Samples  Group2 Samples2 DB.DESeq2&#xA;1 Condition diseased       2 control        2         0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I don't know why the value of DB.DESeq2 in this case was 0 which maybe the reason of the error.&lt;/p&gt;&#xA;&lt;p&gt;This is the result after running dba.contrast:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;4 Samples, 89585 sites in matrix:&#xA;   ID Tissue   Factor Condition Replicate    Reads FRiP&#xA;1  MS    non ATAC-seq   control         1 23731398 0.17&#xA;2 MSR    non ATAC-seq   control         2 10499000 0.16&#xA;3  MF    non ATAC-seq  diseased         1 31535510 0.18&#xA;4 MFR    non ATAC-seq  diseased         2 18488559 0.18&#xA;&#xA;Design: [~Condition] | 1 Contrast:&#xA;     Factor    Group Samples  Group2 Samples2&#xA;1 Condition diseased       2 control        2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If I try to compare MS vs MF, I will get that error but if I remove MS and replace by other sample, I don't have that error anymore.&lt;/p&gt;&#xA;&lt;p&gt;I don't have that error if I add more samples and the value of DB.SESeq2 at the first row from dba.show() is not 0.&lt;/p&gt;&#xA;&lt;p&gt;Full samplesheet:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SampleID,Tissue,Factor,Condition,Treatment,Replicate,bamReads,ControlID,bamControl,Peaks,PeakCallers&#xA;ws1,non,ATAC-seq,ws,,1,ATAC_seq_output/bwa/merged_library/ws_REP1.mLb.clN.sorted.bam,,,ATAC_seq_output/bwa/merged_library/macs2/broad_peak/ws_REP1.mLb.clN_peaks.broadPeak,bed&#xA;ws2,non,ATAC-seq,ws,,2,ATAC_seq_output/bwa/merged_library/ws_REP2.mLb.clN.sorted.bam,,,ATAC_seq_output/bwa/merged_library/macs2/broad_peak/ws_REP2.mLb.clN_peaks.broadPeak,bed&#xA;wf1,non,ATAC-seq,wf,,1,ATAC_seq_output/bwa/merged_library/wf_REP1.mLb.clN.sorted.bam,,,ATAC_seq_output/bwa/merged_library/macs2/broad_peak/wf_REP1.mLb.clN_peaks.broadPeak,bed&#xA;wf2,non,ATAC-seq,wf,,2,ATAC_seq_output/bwa/merged_library/wf_REP2.mLb.clN.sorted.bam,,,ATAC_seq_output/bwa/merged_library/macs2/broad_peak/wf_REP2.mLb.clN_peaks.broadPeak,bed&#xA;ms1,non,ATAC-seq,ms,,1,ATAC_seq_output/bwa/merged_library/ms_REP1.mLb.clN.sorted.bam,,,ATAC_seq_output/bwa/merged_library/macs2/broad_peak/ms_REP1.mLb.clN_peaks.broadPeak,bed&#xA;ms2,non,ATAC-seq,ms,,2,ATAC_seq_output/bwa/merged_library/ms_REP2.mLb.clN.sorted.bam,,,ATAC_seq_output/bwa/merged_library/macs2/broad_peak/ms_REP2.mLb.clN_peaks.broadPeak,bed&#xA;mf1,non,ATAC-seq,mf,,1,ATAC_seq_output/bwa/merged_library/ws_REP1.mLb.clN.sorted.bam,,,ATAC_seq_output/bwa/merged_library/macs2/broad_peak/ws_REP1.mLb.clN_peaks.broadPeak,bed&#xA;mf2,non,ATAC-seq,mf,,2,ATAC_seq_output/bwa/merged_library/ws_REP2.mLb.clN.sorted.bam,,,ATAC_seq_output/bwa/merged_library/macs2/broad_peak/ws_REP2.mLb.clN_peaks.broadPeak,bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3D2rz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3D2rz.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17585" LastEditorUserId="73" LastEditDate="2023-07-20T00:35:00.270" LastActivityDate="2023-07-20T00:35:00.270" Title="Help with error DiffBind" Tags="|r|bioconductor|atac-seq|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21291" PostTypeId="2" ParentId="21273" CreationDate="2023-07-11T03:45:15.110" Score="3" Body="&lt;p&gt;The post here is useful ... &lt;a href=&quot;https://support.bioconductor.org/p/76790/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.bioconductor.org/p/76790/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;ExactTest&lt;/code&gt; is really cool and completely different from &lt;code&gt;glmQLFit&lt;/code&gt; &lt;code&gt;glmFit&lt;/code&gt; etc ... you'll need two controls to check it works (it could just reject everything). Its described here &lt;a href=&quot;https://rdrr.io/bioc/edgeR/man/exactTest.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rdrr.io/bioc/edgeR/man/exactTest.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There is a concern, some of the &lt;strong&gt;lncDIFF&lt;/strong&gt; data looked more like a lognormal to me in the publication, but I didn't read it in sufficient detail. Anyway your data will probably approximate to a negative binomial distribution and it's statistically tricky to  assess whether two negative binomial distributions are significantly different ... the &lt;code&gt;ExactTest&lt;/code&gt; will do this.&lt;/p&gt;&#xA;&lt;p&gt;Is the total output of the control significantly different to the test sample? ... the &lt;code&gt;ExactTest&lt;/code&gt; will tell you. You want &lt;em&gt;p&lt;/em&gt; &amp;lt; 0.05, preferably a lot lower than 0.05 to be convinced. However, you need to compare two control samples against each other, where &lt;em&gt;p&lt;/em&gt; &amp;gt; 0.05 for the subsequent tests to be valid.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;So really nice BUT you need to check whether it is a negative binomial FIRST. Thats what the other tests are doing. Please do check whether &lt;strong&gt;lncDIFF&lt;/strong&gt; identified a negative binomial, because if it ain't &lt;code&gt;ExactTest&lt;/code&gt; will not work, they could be using a different distribution.&lt;/p&gt;&#xA;&lt;p&gt;The other tests you mentioned are different to the &lt;code&gt;ExactTest&lt;/code&gt; because they simply assess whether the data is negative binomial. The distribution is the frequency of each RNA transcript BTW. edgeR's &lt;strong&gt;lnfDIFF&lt;/strong&gt; requires that distribution and thats how it can assign probabilities to the difference between the frequency distribution of two identical transcripts under difference experimental conditions (or other conditions). So you need a test to say &amp;quot;yeah I'm a negative binomial&amp;quot; (assuming it is negative binomial), then everything else via &lt;strong&gt;lncDIFF&lt;/strong&gt; is then valid.&lt;/p&gt;&#xA;&lt;p&gt;First you've got to fit you data to a model ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;y &amp;lt;-glmGLMFit(...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... is the function and that uses general linear models to fit the negative binomial. The fit will result in something called &amp;quot;a residual&amp;quot; and that is very important, the smaller the better. Testing whether that residual is small enough for a negative binomial to be declared is down to ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;glmLRT(y ...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is a likelihood ratio test, to test the observed likelihood against the model (p &amp;gt; 0.05). You can also use&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gof(y ...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;this is goodness of fit, as the link says the residual follows a chi-squared distribution, i.e. its tested on the chi2 tables and thats easy to understand. Again you need a probability &amp;gt; 0.05 to declare a fit.&lt;/p&gt;&#xA;&lt;p&gt;HOWEVER, the &lt;strong&gt;lnfDIFF&lt;/strong&gt; guys developed a quasi-likelihood to assess the distribution&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;z&amp;lt;-glmQLFit(...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Fits the data via quasi-likelihood, I honestly have no empirical idea how that works.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;glmQLTest(z ...)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;tests the fit to the distribution. You need a probability &amp;gt; 0.05. Again I have not empirical idea whats going on here.&lt;/p&gt;&#xA;&lt;p&gt;ANY non-significant result is good, doesn't matter if its GLM with gof, GLM with LRT, GLM-QL (whatever that is) because all of these are solid tests.  Which would I use? The guys who developed &lt;strong&gt;lncDIFF&lt;/strong&gt; used QL (quasi-likelihood) to assess the fit (might be a good idea). However, statistically I honestly don't know whats going on, so I would also look at the GLM and fit it against the goodness of fit - because thats easy to understand.&lt;/p&gt;&#xA;&lt;p&gt;What do I do to assess the fit? I use machine learning, but it's way more complicated than anything here.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;From the comments.&lt;/p&gt;&#xA;&lt;p&gt;It's important to note the context. The OP is performing non-coding RNA work, &lt;strong&gt;lncDIFF&lt;/strong&gt; is much better than normal &lt;strong&gt;edgeR&lt;/strong&gt; for this type of data described here &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/21259/statistical-methods-suitable-for-de-analysis-of-non-coding-rnas&quot;&gt;Statistical methods suitable for DE analysis of non coding RNAs&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;lncDIFF&lt;/strong&gt; is now available for &lt;strong&gt;R&lt;/strong&gt; &lt;a href=&quot;https://cran.r-project.org/web/packages/lncDIFF/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;... so the methods of edgeR should be available to it. &lt;code&gt;glmQLFit()&lt;/code&gt; is a general function in R. Providing the &lt;strong&gt;lncDIFF&lt;/strong&gt; object will be accepted by the tests, by performing everything in &lt;strong&gt;R&lt;/strong&gt; this should solve this.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;I would advise using&lt;/em&gt; &lt;strong&gt;lnfDIFF&lt;/strong&gt;'s QL directly, i.e.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;y&amp;lt;-ZIQMLfit(...) # fit the distribution&#xA;ZIQML.LRT(y ...) # test the fit&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The manual is here&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/qianli10000/lncDIFF/blob/master/lncDIFF-manual.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/qianli10000/lncDIFF/blob/master/lncDIFF-manual.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I hope that makes sense these are pretty advanced stats so it can be tricky to understand them. Whilst I can teach code, I don't teach complex stats outside machine learning.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-07-12T15:52:54.050" LastActivityDate="2023-07-12T15:52:54.050" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21292" PostTypeId="1" CreationDate="2023-07-11T14:11:26.250" Score="2" ViewCount="56" Body="&lt;p&gt;I have a gene ontology obo file, and would like to rearrange the obo format file in a single column to a text file in three columns comprised of 'parent (id:GO:xxxxxxx)', 'child (is_a: GO:xxxxxxx, part_of GO:xxxxxxx, or positively/negatively_regulates GO:xxxxxxx. For example...&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your help in advance!&lt;/p&gt;&#xA;&lt;p&gt;Original file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[Term]&#xA;id: GO:0000001 # parent&#xA;name: mitochondrion inheritance&#xA;namespace: biological_process&#xA;def: &amp;quot;The distribution of mitochondria, including the mitochondrial genome, into daughter cells after mitosis or meiosis, mediated by interactions between mitochondria and the cytoskeleton.&amp;quot; [GOC:mcc, PMID:10873824, PMID:11389764]&#xA;synonym: &amp;quot;mitochondrial inheritance&amp;quot; EXACT []&#xA;is_a: GO:0048308 ! organelle inheritance # child&#xA;is_a: GO:0048311 ! mitochondrion distribution # child&#xA;&#xA;[Term]&#xA;id: GO:0000015 # parent&#xA;name: phosphopyruvate hydratase complex&#xA;namespace: cellular_component&#xA;def: &amp;quot;A multimeric enzyme complex, usually a dimer or an octamer, that catalyzes the conversion of 2-phospho-D-glycerate to phosphoenolpyruvate and water.&amp;quot; [GOC:jl, ISBN:0198506732]&#xA;subset: goslim_metagenomics&#xA;synonym: &amp;quot;enolase complex&amp;quot; EXACT []&#xA;is_a: GO:1902494 ! catalytic complex # child&#xA;relationship: part_of GO:0005829 ! cytosol # child&#xA;&#xA;[Term]&#xA;id: GO:0000122 # parent&#xA;name: negative regulation of transcription by RNA polymerase II&#xA;namespace: biological_process&#xA;alt_id: GO:0010553&#xA;alt_id: GO:0045816&#xA;def: &amp;quot;Any process that stops, prevents, or reduces the frequency, rate or extent of transcription mediated by RNA polymerase II.&amp;quot; [GOC:go_curators, GOC:txnOH]&#xA;synonym: &amp;quot;down regulation of global transcription from RNA polymerase II promoter&amp;quot; RELATED []&#xA;synonym: &amp;quot;down regulation of transcription from RNA polymerase II promoter&amp;quot; EXACT []&#xA;synonym: &amp;quot;down-regulation of global transcription from RNA polymerase II promoter&amp;quot; RELATED []&#xA;synonym: &amp;quot;down-regulation of transcription from RNA polymerase II promoter&amp;quot; EXACT []&#xA;synonym: &amp;quot;downregulation of global transcription from RNA polymerase II promoter&amp;quot; RELATED []&#xA;synonym: &amp;quot;downregulation of transcription from RNA polymerase II promoter&amp;quot; EXACT []&#xA;synonym: &amp;quot;inhibition of global transcription from RNA polymerase II promoter&amp;quot; RELATED []&#xA;synonym: &amp;quot;inhibition of transcription from RNA polymerase II promoter&amp;quot; EXACT []&#xA;synonym: &amp;quot;negative regulation of gene-specific transcription from RNA polymerase II promoter&amp;quot; RELATED []&#xA;synonym: &amp;quot;negative regulation of global transcription from Pol II promoter&amp;quot; RELATED []&#xA;synonym: &amp;quot;negative regulation of transcription from Pol II promoter&amp;quot; EXACT []&#xA;synonym: &amp;quot;negative regulation of transcription from RNA polymerase II promoter&amp;quot; EXACT []&#xA;synonym: &amp;quot;negative regulation of transcription from RNA polymerase II promoter, global&amp;quot; RELATED []&#xA;is_a: GO:0006357 ! regulation of transcription by RNA polymerase II # child&#xA;is_a: GO:0045892 ! negative regulation of DNA-templated transcription # child&#xA;relationship: negatively_regulates GO:0006366 ! transcription by RNA polymerase II # child&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Desired format:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Parent       Child       Relationship&#xA;GO:0000001   GO:0048308  is_a&#xA;GO:0000001   GO:0048311  is_a&#xA;GO:0000015   GO:1902494  is_a&#xA;GO:0000015   GO:0005829  part_of&#xA;GO:0000122   GO:0006357  is_a&#xA;GO:0000122   GO:0045892  is_a&#xA;GO:0000122   GO:0006366  negatively_regulates&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17793" LastEditorUserId="17793" LastEditDate="2023-07-11T15:39:50.630" LastActivityDate="2023-07-12T03:56:19.827" Title="How to rearrange gene ontology obo file?" Tags="|gene|gene-ontology|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21293" PostTypeId="1" CreationDate="2023-07-11T14:34:00.130" Score="1" ViewCount="68" Body="&lt;p&gt;I want to delete an amino acid residue from a loop and then join the loose ends. I intend to increase the thermal stability of the protein by reducing the size of the loop and its agitation. Pymol just deletes the residue and breaks the loop.&lt;/p&gt;&#xA;&lt;p&gt;I intend to increase the optimal temperature of the protein, even if it costs some reduction in activity. The objective is to reduce protein agitation due to temperature increase. I will try point mutations that increase hydrogen interactions and eventually delete a loop located glycine to reduce its length and consequently its thermal agitation. All this work is aimed at preserving the catalytic site, but some loss of activity is acceptable.&lt;/p&gt;&#xA;&lt;p&gt;As a complement to my question, I present an excerpt from the paper by Mahdie Rahban et al (2022) ; Thermal stability enhancement: Fundamental concepts of protein engineering strategies to manipulate the flexible structure.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Loop deletion or shortening by insertion of proline or deletion of&#xA;Gly, coupled with loop anchoring by increasing the number of salt&#xA;bridges, hydrogen bonds, or hydrophobicity, are good strategies to&#xA;enhance the overall rigidity of a protein conformation leading to&#xA;better compactness and a decreased number of internal cavities.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="17794" LastEditorUserId="73" LastEditDate="2023-07-11T22:14:22.090" LastActivityDate="2023-12-12T01:03:10.343" Title="How do I delete an amino acid residue from a loop and then join the loose ends?" Tags="|proteins|protein-structure|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21294" PostTypeId="1" AcceptedAnswerId="21295" CreationDate="2023-07-11T17:26:51.573" Score="1" ViewCount="34" Body="&lt;p&gt;I have a tsv file that lists the reads and read lengths from a FASTA file but some reads are duplicated - that's just from the analysis I did previously - but I want to only take one instance of the read and have it be the SHORTER length than the other duplicate.&lt;/p&gt;&#xA;&lt;p&gt;For instance, here is the beginning of the tsv file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;read                                    length&#xA;91290b97-5ca1-489d-b488-17afc1d217e7    3584&#xA;dad08436-3cd6-40a1-9af1-b846bd24a815    3787&#xA;f4a7bbdf-de62-48a8-a8b0-06076e251820    3637&#xA;f4a7bbdf-de62-48a8-a8b0-06076e251820    4717&#xA;21ddb2f2-654e-4124-8226-91c5a8d11a97    3786&#xA;6a2de460-a661-4c9e-ac1d-adfc3e5841e2    3520&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can see that &lt;code&gt;f4a7bbdf-de62-48a8-a8b0-06076e251820&lt;/code&gt; is found twice with different lengths &lt;code&gt;3637&lt;/code&gt; and &lt;code&gt;4717&lt;/code&gt;. I want to go through and take the shorter length of that read and include it in the final tsv file, resulting in this structure:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;read                                    length&#xA;91290b97-5ca1-489d-b488-17afc1d217e7    3584&#xA;dad08436-3cd6-40a1-9af1-b846bd24a815    3787&#xA;f4a7bbdf-de62-48a8-a8b0-06076e251820    3637&#xA;21ddb2f2-654e-4124-8226-91c5a8d11a97    3786&#xA;6a2de460-a661-4c9e-ac1d-adfc3e5841e2    3520&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm trying to do this in python and have started with making the tsv file a list:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;newlist = []&#xA;duplist = []&#xA;&#xA;with open(tsv_path) as file:&#xA;    tsv_file = csv.reader(file, delimiter=&amp;quot;\t&amp;quot;)&#xA;    &#xA;    for line in tsv_file:&#xA;        if line[0] not in newlist:&#xA;            newlist.append(line[0])&#xA;        else:&#xA;            duplist.append(line)&#xA;            &#xA;    print(&amp;quot;All the duplicate reads: &amp;quot;, duplist)&#xA;    print(&amp;quot;The normal reads: &amp;quot;, newlist)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which works to contain the first instance of a read in one list and the second/third instance of the read to go in a separate list with their lengths. However, the shorter length of a read is not always the first instance of a read. &lt;strong&gt;Do I need to include another loop that looks at the lengths and compares each instance of a read? Is there an easier way to do this?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="650" LastEditDate="2023-07-12T16:14:01.960" LastActivityDate="2023-07-12T16:16:09.527" Title="Getting rid of duplicates in a dictionary" Tags="|python|file-formats|pandas|duplications|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21295" PostTypeId="2" ParentId="21294" CreationDate="2023-07-11T17:48:56.983" Score="3" Body="&lt;p&gt;&lt;s&gt;Not sure how you'd do this in python but&lt;/s&gt; you need a group-by operation here. Group by the read name and pick the min length per group.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: With pandas, you can do:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;df.groupby(['read'])['length'].agg('min')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-07-11T17:48:56.983" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21296" PostTypeId="1" AcceptedAnswerId="21305" CreationDate="2023-07-11T21:12:48.467" Score="4" ViewCount="127" Body="&lt;p&gt;I'm developing pipeline in nextflow it uses tools: fastp, bwa mem index, bwamem, gatk mark duplicates, gatk setupnmd, gatk applybqsr, gatk recalibrate. I do this using FASTQ.gz of normal and tumor DNA samples.&lt;br /&gt;&#xA;I'm interested to use mutect2, lancet and strelka in nextsteps. However, I am unable to write code in the pipeline that makes sense. I use loads of shell scripting in mutect2 to use tumor-normal.&lt;br /&gt;&#xA;To expand, I've input in FASTQ as tiny_n, tiny_t, where _n is normal and _t is tumor. I run pipeline until applybqsr for each individually. However, as I reach mutect2, I check if sample contains &lt;strong&gt;_n&lt;/strong&gt; or otherwise, I then create an empty file for the output as required in the mutect2 process.&lt;/p&gt;&#xA;&lt;p&gt;How do I overcome creating dummy/empty file?&lt;/p&gt;&#xA;&lt;p&gt;Please see below code of &lt;strong&gt;main.nf&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params.raw_files=&amp;quot;/sc/arion/projects/tiny/all_tumors_normals/tiny_*{R1,R2}.fastq&amp;quot;&#xA;&#xA;include { FASTP} from './fastp_process.nf'&#xA;//include {bwa_index} from './index_process.nf'&#xA;include { align_bwa_mem} from './bwamem_process_already_index.nf'&#xA;include { gatk_markduplicates} from './gatk_markduplicates_process.nf'&#xA;include { recalibrate_bam } from './recalibratebam_process.nf'&#xA;include {setupnmdtags} from './setupnmdtags_process.nf'&#xA;include { applybqsr } from './applybqsr_process.nf'&#xA;include { mutect2 } from './mutect2_process.nf'&#xA;workflow {&#xA;&#xA;        read_pairs_ch = Channel.fromFilePairs(params.raw_files) &#xA;&#xA;        FASTP(read_pairs_ch) &#xA;&#xA;        align_bwa_mem(FASTP.out.reads) &#xA;        gatk_markduplicates(align_bwa_mem.out.sorted_bams)&#xA;        setupnmdtags(gatk_markduplicates.out.sorted_bam_mark_duplicates)&#xA;        recalibrate_bam(setupnmdtags.out.setupnmdtags_bam)&#xA;&#xA;applybqsr(gatk_markduplicates.out.sorted_bam_mark_duplicates,recalibrate_bam.out.recalib_table)&#xA;&#xA;&#xA;mutect2(applybqsr.out.recal_bam)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;below is code of mutect2 to handle only one sample, run for normal, create dummy/empty file for tumor sample, &lt;strong&gt;mutect2_process.nf&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process mutect2 {&#xA;        maxForks 3&#xA;        debug true&#xA;        tag &amp;quot;${sample_id}&amp;quot;&#xA; memory '15 GB'&#xA;&#xA;        input:&#xA;        tuple val(sample_id),path(bqsrbam)&#xA;        &#xA;        output:&#xA;        tuple val(sample_id),path(&amp;quot;${sample_id}.mutect2out.vcf&amp;quot;)&#xA;        &#xA;&#xA;        script:&#xA;                         def avail_mem = task.memory ? task.memory.toGiga() : 0&#xA;                def java_options = [&#xA;        avail_mem ? &amp;quot;-Xmx${avail_mem}G&amp;quot; : &amp;quot;&amp;quot;,&#xA;        &amp;quot;-XX:+UseSerialGC&amp;quot;,&#xA;    ]&#xA;&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        echo &amp;quot;${sample_id}\n&amp;quot;&#xA;&#xA;        grep_value=\\\$(echo &amp;quot;${sample_id}|  | grep &amp;quot;_n&amp;quot; &amp;quot;)&#xA;&#xA;        if [[ ! -z &amp;quot;\\\${grep_value}&amp;quot; ]]&#xA;        then &#xA;&#xA;                identifier=\\\$(echo ${sample_id} | awk -F&amp;quot;_&amp;quot; '{print \\\$1}')&#xA;                echo &amp;quot;\\\${identifier}&amp;quot;_t.sorted.markdup.recal.bam&amp;quot;&amp;quot;&#xA;&#xA;                ml gatk/4.1.3.0&#xA;gatk --java-options &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${java_options.join(' ')}&quot; Mutect2 -L $&lt;/span&gt;params.target -I &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${params.outdir}/applybqsr/&quot;&quot;\\\${identifier}&quot;_t.sorted.markdup.recal.bam&quot;&quot; -I  &quot;$&lt;/span&gt;{params.outdir}/applybqsr/&amp;quot;&amp;quot;\\\${identifier}&amp;quot;_n.sorted.markdup.recal.bam&amp;quot;&amp;quot; -normal \\\${identifier}&amp;quot;_n&amp;quot; -tumor \\\${identifier}&amp;quot;_t&amp;quot; --output &lt;span class=&quot;math-container&quot;&gt;${sample_id}&quot;.mutect2out.vcf&quot; -R $&lt;/span&gt;params.hg38genome&#xA;&#xA;        else &#xA;                echo &amp;quot;Nothing to do\n&amp;quot;&#xA;                touch ${sample_id}&amp;quot;.mutect2out.vcf&amp;quot;&#xA; &#xA;        fi&#xA;&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        }&#xA;&#xA;&#xA;workflow.onComplete { &#xA;        log.info ( workflow.success ? &amp;quot;Done mutect2!&amp;quot; : &amp;quot;Oops .. something went wrong in mutect2&amp;quot; )&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There is a load of unix/shell code that does trivial checks. It runs gatk mutect2 oly when _n is found.&#xA;How do I overcome: output defined &lt;code&gt;tuple val(sample_id),path(&amp;quot;${sample_id}.mutect2out.vcf&amp;quot;)&lt;/code&gt; ?&lt;br /&gt;&#xA;Is there a better way to do this?&lt;/p&gt;&#xA;" OwnerUserId="862" LastEditorUserId="862" LastEditDate="2023-07-11T21:30:05.137" LastActivityDate="2023-07-14T00:51:33.487" Title="nextflow: tumor normal sample - how to make code organic" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21297" PostTypeId="2" ParentId="21294" CreationDate="2023-07-11T22:12:41.737" Score="2" Body="&lt;p&gt;You can do this with the &lt;code&gt;sort&lt;/code&gt; command-line tool. This command pipe sequence sorts numerically by the second field, then chooses unique elements based on the first field&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sort -r -k 2n,2 read_lengths.txt | sort -r -u -k 1,1 &amp;gt; unique_lengths.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;[the additional &lt;code&gt;-r&lt;/code&gt; bits are not strictly necessary; they help make sure that the headings stay at the top]&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="650" LastEditDate="2023-07-12T16:16:09.527" LastActivityDate="2023-07-12T16:16:09.527" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21298" PostTypeId="2" ParentId="21292" CreationDate="2023-07-12T02:17:46.353" Score="2" Body="&lt;p&gt;One potential solution using awk:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN {&#xA;    OFS = &amp;quot;\t&amp;quot;&#xA;    print &amp;quot;Parent&amp;quot;, &amp;quot;Child&amp;quot;, &amp;quot;Relationship&amp;quot;&#xA;}&#xA;&#xA;/GO:[[:digit:]]{1,}/ {&#xA;    if (&lt;span class=&quot;math-container&quot;&gt;$1 ~ /^id:/) {&#xA;    parent_id = $2&#xA;} else if ($&lt;/span&gt;1 ~ /^is_a:/) {&#xA;        gsub(&amp;quot;:&amp;quot;, &amp;quot;&amp;quot;, &lt;span class=&quot;math-container&quot;&gt;$1)&#xA;    print parent_id, $&lt;/span&gt;2, &lt;span class=&quot;math-container&quot;&gt;$1&#xA;} else if ($&lt;/span&gt;1 ~ /^relationship:/) {&#xA;        print parent_id, &lt;span class=&quot;math-container&quot;&gt;$3, $&lt;/span&gt;2&#xA;    }&#xA;}' test.txt&#xA;Parent      Child       Relationship&#xA;GO:0000001  GO:0048308  is_a&#xA;GO:0000001  GO:0048311  is_a&#xA;GO:0000015  GO:1902494  is_a&#xA;GO:0000015  GO:0005829  part_of&#xA;GO:0000122  GO:0006357  is_a&#xA;GO:0000122  GO:0045892  is_a&#xA;GO:0000122  GO:0006366  negatively_regulates&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Including comments:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk 'BEGIN {                      # before processing the first line&#xA;    OFS = &amp;quot;\t&amp;quot;                    # set the output field separator to tab&#xA;    print &amp;quot;Parent&amp;quot;, &amp;quot;Child&amp;quot;, &amp;quot;Relationship&amp;quot;    # and print the header&#xA;}&#xA;&#xA;/GO:[[:digit:]]{1,}/ {    # for any line that contains a GO term&#xA;    if (&lt;span class=&quot;math-container&quot;&gt;$1 ~ /^id:/) {            # if the line starts with id:&#xA;    parent_id = $1                # set the parent id&#xA;} else if ($&lt;/span&gt;1 ~ /^is_a/) {        # then, for lines that start with is_a &#xA;        gsub(&amp;quot;:&amp;quot;, &amp;quot;&amp;quot;, &lt;span class=&quot;math-container&quot;&gt;$1)         # remove the colon&#xA;    print parent_id, $&lt;/span&gt;2, &lt;span class=&quot;math-container&quot;&gt;$1       # and print the parent, child and relationship&#xA;} else if ($&lt;/span&gt;1 ~ /^relationship:/) {  # then, for lines that start with relationship&#xA;        print parent_id, &lt;span class=&quot;math-container&quot;&gt;$3, $&lt;/span&gt;2   # print the parent, child, and relationship&#xA;    }&#xA;}' test.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9098" LastEditorUserId="9098" LastEditDate="2023-07-12T03:56:19.827" LastActivityDate="2023-07-12T03:56:19.827" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21299" PostTypeId="1" CreationDate="2023-07-12T06:15:31.217" Score="0" ViewCount="23" Body="&lt;p&gt;As part of a larger analysis, I need to evaluate pairs of genomic regions (&lt;code&gt;regionsA&lt;/code&gt; and &lt;code&gt;regionsB&lt;/code&gt;) and perform different calculations for each pair depending on whether:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;They're completely separate (no overlap)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;regionsB&lt;/code&gt; partially overlaps &lt;code&gt;regionsA&lt;/code&gt; (start or end of &lt;code&gt;regionsB&lt;/code&gt; does not overlap; both are handled separately)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;regionsB&lt;/code&gt; is fully contained in &lt;code&gt;regionsA&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;At this time, &lt;code&gt;regionsA&lt;/code&gt; and &lt;code&gt;regionsB&lt;/code&gt; are represented as Pandas DataFrames. Likewise, I'm doing all possible combinations:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;if (region_b.start &amp;lt; region_a.start and&#xA;    region_b.end &amp;lt; region_b.start):&#xA;[...]&#xA;elif (region_b.start &amp;lt; region_a.start and&#xA;     (region_b.end &amp;gt; region_a.start&#xA;     and region_b.end &amp;lt; region_a.end )):&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Given that there are many conditions I check for, this makes the code long, a little hard to read, and too verbose. Is there anything I can use (without making the code harder to read) to make these checks simpler, something like PyRanges (or PyRanges itself, although I don't think there's API for it)?&lt;/p&gt;&#xA;&lt;p&gt;If not, I'll just live with what I have.&lt;/p&gt;&#xA;" OwnerUserId="6814" LastActivityDate="2023-07-12T06:15:31.217" Title="Compare in Python / PyRanges two genomic intervals to determine whether they're separate, partially overlapping, or contained" Tags="|python|pandas|interval|pyranges|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21300" PostTypeId="1" CreationDate="2023-07-12T13:09:18.167" Score="1" ViewCount="32" Body="&lt;p&gt;I have a bunch of samples of a non-model organism divided in two groups: parental generation and offspring. I do not have a pedigree so I do not actually know who is a parent of who, and which are the sibships. I have a bunch of SNPs that I can use to create a kinship matrix, but I cannot resolve families just using it.&lt;/p&gt;&#xA;&lt;p&gt;I want to perform linkage disequilibrium (LD) between the SNPs. I can calculate the correlation between them, but I feel I am losing a lot of power because there is kinship between samples. Is there a program that could take the kinship matrix in account when calculating the LD?&lt;/p&gt;&#xA;&lt;p&gt;I could, in theory, reconstruct the pedigree from the markers. I run multiple instance of &lt;a href=&quot;https://www.zsl.org/about-zsl/resources/software/colony&quot; rel=&quot;nofollow noreferrer&quot;&gt;Colony&lt;/a&gt; but the results are underwhelming -- most samples are not allocated (admittedly this raises questions on the SNP situation, but this is a whole different issue).&lt;/p&gt;&#xA;" OwnerUserId="10565" LastEditorUserId="3848" LastEditDate="2023-07-13T12:38:10.873" LastActivityDate="2023-07-13T15:34:16.370" Title="Finding linkage disequilibrium when I have a kinship matrix but not pedigree" Tags="|phylogenetics|phylogeny|plink|pedigree|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21301" PostTypeId="1" AcceptedAnswerId="21324" CreationDate="2023-07-12T13:27:14.177" Score="0" ViewCount="91" Body="&lt;p&gt;I have .vcf file with over 78 000 variations in 95 experiment subjects. I used vcftools to extract fixation index score for general differentiation between Case and Control groups.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./vcftools --vcf SCH_freebayes_sort_ostE_SnpEff_SnpSift.vcf --weir-fst-pop Case.txt --weir-fst-pop Control.txt --out FstExon_Case_vs_Control&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;VCFtools - 0.1.17&#xA;(C) Adam Auton and Anthony Marcketta 2009&#xA;&#xA;Parameters as interpreted:&#xA;    --vcf SCH_freebayes_sort_ostE_SnpEff_SnpSift.vcf&#xA;    --weir-fst-pop Case.txt&#xA;    --weir-fst-pop Control.txt&#xA;    --keep Case.txt&#xA;    --keep Control.txt&#xA;    --out Fst_TestData_nochange&#xA;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=AF,Number=A,Type=Float,Description=&amp;quot;Estimated allele frequency in the range (0,1]&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=PRO,Number=1,Type=Float,Description=&amp;quot;Reference allele observation count, with partial observations recorded fractionally&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=PAO,Number=A,Type=Float,Description=&amp;quot;Alternate allele observations, with partial observations recorded fractionally&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=SRP,Number=1,Type=Float,Description=&amp;quot;Strand balance probability for the reference allele: Phred-scaled upper-bounds estimate of the probability of observing the deviation between SRF and SRR given E(SRF/SRR) ~ 0.5, derived using Hoeffding's inequality&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=SAP,Number=A,Type=Float,Description=&amp;quot;Strand balance probability for the alternate allele: Phred-scaled upper-bounds estimate of the probability of observing the deviation between SAF and SAR given E(SAF/SAR) ~ 0.5, derived using Hoeffding's inequality&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=AB,Number=A,Type=Float,Description=&amp;quot;Allele balance at heterozygous sites: a number between 0 and 1 representing the ratio of reads showing the reference allele to all reads, considering only reads from individuals called as heterozygous&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=ABP,Number=A,Type=Float,Description=&amp;quot;Allele balance probability at heterozygous sites: Phred-scaled upper-bounds estimate of the probability of observing the deviation between ABR and ABA given E(ABR/ABA) ~ 0.5, derived using Hoeffding's inequality&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=RPP,Number=A,Type=Float,Description=&amp;quot;Read Placement Probability: Phred-scaled upper-bounds estimate of the probability of observing the deviation between RPL and RPR given E(RPL/RPR) ~ 0.5, derived using Hoeffding's inequality&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=RPPR,Number=1,Type=Float,Description=&amp;quot;Read Placement Probability for reference observations: Phred-scaled upper-bounds estimate of the probability of observing the deviation between RPL and RPR given E(RPL/RPR) ~ 0.5, derived using Hoeffding's inequality&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=EPP,Number=A,Type=Float,Description=&amp;quot;End Placement Probability: Phred-scaled upper-bounds estimate of the probability of observing the deviation between EL and ER given E(EL/ER) ~ 0.5, derived using Hoeffding's inequality&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=EPPR,Number=1,Type=Float,Description=&amp;quot;End Placement Probability for reference observations: Phred-scaled upper-bounds estimate of the probability of observing the deviation between EL and ER given E(EL/ER) ~ 0.5, derived using Hoeffding's inequality&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=TYPE,Number=A,Type=String,Description=&amp;quot;The type of allele, either snp, mnp, ins, del, or complex.&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=TYPE,Number=A,Type=String,Description=&amp;quot;The type of allele, either snp, mnp, ins, del, or complex.&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=TYPE,Number=A,Type=String,Description=&amp;quot;The type of allele, either snp, mnp, ins, del, or complex.&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=TYPE,Number=A,Type=String,Description=&amp;quot;The type of allele, either snp, mnp, ins, del, or complex.&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=TYPE,Number=A,Type=String,Description=&amp;quot;The type of allele, either snp, mnp, ins, del, or complex.&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=CIGAR,Number=A,Type=String,Description=&amp;quot;The extended CIGAR representation of each alternate allele, with the exception that '=' is replaced by 'M' to ease VCF parsing.  Note that INDEL alleles do not have the first matched base (which is provided by default, per the spec) referred to by the CIGAR.&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in FORMAT entry: ID=GQ,Number=1,Type=Float,Description=&amp;quot;Genotype Quality, the Phred-scaled marginal (or unconditional) probability of the called genotype&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in FORMAT entry: ID=GL,Number=G,Type=Float,Description=&amp;quot;Genotype Likelihood, log10-scaled likelihoods of the data given the called genotype for each possible genotype generated from the reference and alternate alleles given the sample ploidy&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=WTD,Number=0,Type=Flag,Description=&amp;quot;Is Withdrawn by submitter If one member ss is withdrawn by submitter, then this bit is set.  If all member ss' are withdrawn, then the rs is deleted to SNPHistory&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=WTD,Number=0,Type=Flag,Description=&amp;quot;Is Withdrawn by submitter If one member ss is withdrawn by submitter, then this bit is set.  If all member ss' are withdrawn, then the rs is deleted to SNPHistory&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=TOPMED,Number=.,Type=String,Description=&amp;quot;An ordered, comma delimited list of allele frequencies based on TOPMed, starting with the reference allele followed by alternate alleles as ordered in the ALT column. The TOPMed minor allele is the second largest value in the list.&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=TOPMED,Number=.,Type=String,Description=&amp;quot;An ordered, comma delimited list of allele frequencies based on TOPMed, starting with the reference allele followed by alternate alleles as ordered in the ALT column. The TOPMed minor allele is the second largest value in the list.&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=WGT,Number=1,Type=Integer,Description=&amp;quot;Weight, 00 - unmapped, 1 - weight 1, 2 - weight 2, 3 - weight 3 or more&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=WGT,Number=1,Type=Integer,Description=&amp;quot;Weight, 00 - unmapped, 1 - weight 1, 2 - weight 2, 3 - weight 3 or more&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=WGT,Number=1,Type=Integer,Description=&amp;quot;Weight, 00 - unmapped, 1 - weight 1, 2 - weight 2, 3 - weight 3 or more&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=WGT,Number=1,Type=Integer,Description=&amp;quot;Weight, 00 - unmapped, 1 - weight 1, 2 - weight 2, 3 - weight 3 or more&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=MTP,Number=0,Type=Flag,Description=&amp;quot;Microattribution/third-party annotation(TPA:GWAS,PAGE)&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=NOC,Number=0,Type=Flag,Description=&amp;quot;Contig allele not present in variant allele list. The reference sequence allele at the mapped position is not present in the variant allele list, adjusted for orientation.&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=CAF,Number=.,Type=String,Description=&amp;quot;An ordered, comma delimited list of allele frequencies based on 1000Genomes, starting with the reference allele followed by alternate alleles as ordered in the ALT column. Where a 1000Genomes alternate allele is not in the dbSNPs alternate allele set, the allele is added to the ALT column. The minor allele is the second largest value in the list, and was previuosly reported in VCF as the GMAF. This is the GMAF reported on the RefSNP and EntrezSNP pages and VariationReporter&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=CAF,Number=.,Type=String,Description=&amp;quot;An ordered, comma delimited list of allele frequencies based on 1000Genomes, starting with the reference allele followed by alternate alleles as ordered in the ALT column. Where a 1000Genomes alternate allele is not in the dbSNPs alternate allele set, the allele is added to the ALT column. The minor allele is the second largest value in the list, and was previuosly reported in VCF as the GMAF. This is the GMAF reported on the RefSNP and EntrezSNP pages and VariationReporter&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=CAF,Number=.,Type=String,Description=&amp;quot;An ordered, comma delimited list of allele frequencies based on 1000Genomes, starting with the reference allele followed by alternate alleles as ordered in the ALT column. Where a 1000Genomes alternate allele is not in the dbSNPs alternate allele set, the allele is added to the ALT column. The minor allele is the second largest value in the list, and was previuosly reported in VCF as the GMAF. This is the GMAF reported on the RefSNP and EntrezSNP pages and VariationReporter&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=CAF,Number=.,Type=String,Description=&amp;quot;An ordered, comma delimited list of allele frequencies based on 1000Genomes, starting with the reference allele followed by alternate alleles as ordered in the ALT column. Where a 1000Genomes alternate allele is not in the dbSNPs alternate allele set, the allele is added to the ALT column. The minor allele is the second largest value in the list, and was previuosly reported in VCF as the GMAF. This is the GMAF reported on the RefSNP and EntrezSNP pages and VariationReporter&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=MUT,Number=0,Type=Flag,Description=&amp;quot;Is mutation (journal citation, explicit fact): a low frequency variation that is cited in journal and other reputable sources&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=SAO,Number=1,Type=Integer,Description=&amp;quot;Variant Allele Origin: 0 - unspecified, 1 - Germline, 2 - Somatic, 3 - Both&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=SAO,Number=1,Type=Integer,Description=&amp;quot;Variant Allele Origin: 0 - unspecified, 1 - Germline, 2 - Somatic, 3 - Both&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=SAO,Number=1,Type=Integer,Description=&amp;quot;Variant Allele Origin: 0 - unspecified, 1 - Germline, 2 - Somatic, 3 - Both&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=SSR,Number=1,Type=Integer,Description=&amp;quot;Variant Suspect Reason Codes (may be more than one value added together) 0 - unspecified, 1 - Paralog, 2 - byEST, 4 - oldAlign, 8 - Para_EST, 16 - 1kg_failed, 1024 - other&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=SSR,Number=1,Type=Integer,Description=&amp;quot;Variant Suspect Reason Codes (may be more than one value added together) 0 - unspecified, 1 - Paralog, 2 - byEST, 4 - oldAlign, 8 - Para_EST, 16 - 1kg_failed, 1024 - other&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=SSR,Number=1,Type=Integer,Description=&amp;quot;Variant Suspect Reason Codes (may be more than one value added together) 0 - unspecified, 1 - Paralog, 2 - byEST, 4 - oldAlign, 8 - Para_EST, 16 - 1kg_failed, 1024 - other&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=SSR,Number=1,Type=Integer,Description=&amp;quot;Variant Suspect Reason Codes (may be more than one value added together) 0 - unspecified, 1 - Paralog, 2 - byEST, 4 - oldAlign, 8 - Para_EST, 16 - 1kg_failed, 1024 - other&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=SSR,Number=1,Type=Integer,Description=&amp;quot;Variant Suspect Reason Codes (may be more than one value added together) 0 - unspecified, 1 - Paralog, 2 - byEST, 4 - oldAlign, 8 - Para_EST, 16 - 1kg_failed, 1024 - other&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=SSR,Number=1,Type=Integer,Description=&amp;quot;Variant Suspect Reason Codes (may be more than one value added together) 0 - unspecified, 1 - Paralog, 2 - byEST, 4 - oldAlign, 8 - Para_EST, 16 - 1kg_failed, 1024 - other&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=PM,Number=0,Type=Flag,Description=&amp;quot;Variant is Precious(Clinical,Pubmed Cited)&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=AF_popmax,Number=A,Type=Float,Description=&amp;quot;Maximum allele frequency across populations (excluding samples of Ashkenazi, Finnish, and indeterminate ancestry)&amp;quot;&amp;gt;&#xA;Warning: Expected at least 2 parts in INFO entry: ID=AF_popmax,Number=A,Type=Float,Description=&amp;quot;Maximum allele frequency across populations (excluding samples of Ashkenazi, Finnish, and indeterminate ancestry)&amp;quot;&amp;gt;&#xA;Keeping individuals in 'keep' list&#xA;After filtering, kept 95 out of 95 Individuals&#xA;Outputting Weir and Cockerham Fst estimates.&#xA;    Fst: Only using diploid sites.&#xA;Weir and Cockerham mean Fst estimate: -0.00031927&#xA;Weir and Cockerham weighted Fst estimate: -0.00047509&#xA;After filtering, kept 78758 out of a possible 78758 Sites&#xA;Run Time = 4.00 seconds&#xA;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But when I tried to open .weir.fst files with notepad or excel, I lost over 9 000 rows of data. I am sure that there is problem with how I am opening the .weir.fst files.&lt;/p&gt;&#xA;&lt;p&gt;Please, how are these files opened properly?&lt;/p&gt;&#xA;" OwnerUserId="17800" LastEditorUserId="17800" LastEditDate="2023-07-12T17:07:46.230" LastActivityDate="2023-07-16T00:10:44.227" Title="How to open .weir.fst files witout losing data?" Tags="|vcftools|populations|population-genetics|fst|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="21302" PostTypeId="1" AcceptedAnswerId="21309" CreationDate="2023-07-12T19:18:08.807" Score="1" ViewCount="47" Body="&lt;p&gt;If I have multiple SAM files without header, and a header saved in a separate file, how do I convert that to a BAM file?&lt;/p&gt;&#xA;&lt;p&gt;One solution is to create intermediate files:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for samfile in *.sam&#xA;do&#xA;  cat my_header &amp;gt; tmp.sam&#xA;  cat $samfile &amp;gt;&amp;gt; tmp.sam&#xA;  samtools view -b tmp.sam &amp;gt; $samfile.bam&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But that seems inelegant and I feel like I'm missing the better way.&lt;/p&gt;&#xA;" OwnerUserId="9497" LastEditorUserId="9497" LastEditDate="2023-07-12T19:42:35.100" LastActivityDate="2023-07-13T14:20:40.307" Title="Create bam from separate sam and header" Tags="|samtools|bash|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21304" PostTypeId="1" CreationDate="2023-07-12T20:29:20.943" Score="1" ViewCount="67" Body="&lt;p&gt;I have 133 aligned and sorted BAM files that I am trying to run through the ref_map.pl script from the STACKS pipeline. My script looks as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;module load stacks&#xA;&#xA;ref_map.pl -T 15 --popmap B2_filenames.txt --samples /home/heather/scratch/03_sort_and_map/filtered_and_sorted/B2/ --out-path /home/heather/scratch/ref_map_results/full_data&#xA;&#xA;echo &amp;quot;ref_map.pl for B2 dataset completed&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I keep running into the issue where gstacks is unable to read the BAM files. This is what the log file looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  Configuration for this run:&#xA;  Input mode: reference-based&#xA;  Population map: 'B2_filenames.txt'&#xA;  Input files: 133, e.g. '/home/heather/scratch/03_sort_and_map/filtered_and_sorted/B2/1401-47750.sort_filt.bam'&#xA;  Output to: '/home/heather/scratch/ref_map_results/full_data/'&#xA;  Model: marukilow (var_alpha: 0.01, gt_alpha: 0.05)&#xA;&#xA;Reading BAM headers...&#xA;Processing all loci...&#xA;Error: No BAM records.&#xA;Error: (At the 0th record in file '/home/heather/scratch/03_sort_and_map/filtered_and_sorted/B2/1401-47754_B2.sort_filt.bam'.)&#xA;Aborted.&#xA;&#xA;ref_map.pl: Aborted because the last command failed (1).&#xA;Last command executed by ref_map.pl was:&#xA;  /cvmfs/soft.computecanada.ca/easybuild/software/2020/avx2/Core/stacks/2.64/bin/gstacks -I /home/heather/scratch/03_sort_and_map/filtered_and_sorted/B2/ -M B2_filenames.txt -O /home/heather/scratch/ref_map_results/full_data -t 15&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I checked the BAM file with samtools - the headers look ok, and I don't get any output when I run samtools quickcheck, so I can't quite figure out what is causing the issue. I am following the same pipeline as I did with a previous batch of data and can't understand whats going wrong.&lt;/p&gt;&#xA;" OwnerUserId="17177" LastActivityDate="2023-07-18T20:32:36.023" Title="STACKS ref_map.pl no BAM records" Tags="|sam|genomics|genetics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21305" PostTypeId="2" ParentId="21296" CreationDate="2023-07-13T01:28:23.290" Score="2" Body="&lt;p&gt;I think you have a design problem here. A better way would be perform this operation in your &lt;em&gt;workflow&lt;/em&gt; block. This would allow you to feed in the tumor and normal BAMs (and avoid accessing files outside of the working directory as previously discussed). You can use the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#map&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;map&lt;/code&gt;&lt;/a&gt; operator to extract the patient or case id from the sample names and the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#filter&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;filter&lt;/code&gt;&lt;/a&gt; operator to separate the samples into two channels. You could also use the &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#branch&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;branch&lt;/code&gt;&lt;/a&gt; operator for this like in the example below. The &lt;a href=&quot;https://www.nextflow.io/docs/latest/operator.html#join&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;join&lt;/code&gt;&lt;/a&gt; operator can then be used to join together the items emitted in a consistent way. For example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow {&#xA;&#xA;    ref_fasta = file( params.ref_fasta )&#xA;    targets = file( params.targets )&#xA;&#xA;    ...&#xA;&#xA;    applybqsr.out.recal_bam&#xA;        .branch { sample_id, bam -&amp;gt;&#xA;&#xA;            def case_id = sample_id.substring(0, sample_id.lastIndexOf('_'))&#xA;&#xA;            tumor: sample_id.toUpperCase().endsWith('_T')&#xA;                return tuple( case_id, sample_id, bam )&#xA;&#xA;            normal: sample_id.toUpperCase().endsWith('_N')&#xA;                return tuple( case_id, sample_id, bam )&#xA;&#xA;        }&#xA;        .set { cases_ch }&#xA;&#xA;    joined_ch = cases_ch.tumor.join( cases_ch.normal )&#xA;&#xA;    mutect2( joined_ch, ref_fasta, targets )&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;process mutect2 {&#xA;&#xA;    tag { sample_id }&#xA;&#xA;    module 'gatk/4.1.3.0'&#xA;&#xA;    input:&#xA;    tuple val(case_id), val(tumor), path(tumor_bam), val(normal), path(normal_bam)&#xA;    path reference_fasta&#xA;    path intervals_bed&#xA;&#xA;    output:&#xA;    tuple val(case_id), path(&amp;quot;${case_id}.mutect2.vcf&amp;quot;)&#xA;&#xA;    script:&#xA;    def avail_mem = task.memory ? task.memory.toGiga() : 0&#xA;    def java_options = [&#xA;        avail_mem ? &amp;quot;-Xmx${avail_mem}G&amp;quot; : &amp;quot;&amp;quot;,&#xA;        &amp;quot;-XX:+UseSerialGC&amp;quot;,&#xA;    ]&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    gatk \\&#xA;        --java-options &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${java_options.join(' ')}&quot; \\&#xA;    Mutect2 \\&#xA;        -R &quot;$&lt;/span&gt;{reference_fasta}&amp;quot; \\&#xA;        -L &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${intervals_bed}&quot; \\&#xA;        -I &quot;$&lt;/span&gt;{tumor_bam}&amp;quot; \\&#xA;        -I &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${normal_bam}&quot; \\&#xA;        -normal &quot;$&lt;/span&gt;{normal}&amp;quot; \\&#xA;        -tumor &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${tumor}&quot; \\&#xA;        -O &quot;$&lt;/span&gt;{case_id}&amp;quot;.mutect2.vcf&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There's also likely an issue with how &lt;em&gt;applybqsr&lt;/em&gt; is invoked. If you require &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#process-multiple-input-channels&quot; rel=&quot;nofollow noreferrer&quot;&gt;multiple input channels&lt;/a&gt;, most of the time what you'll want is one &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#queue-channel&quot; rel=&quot;nofollow noreferrer&quot;&gt;queue channel&lt;/a&gt; and one or more &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#value-channel&quot; rel=&quot;nofollow noreferrer&quot;&gt;value channels&lt;/a&gt;. I suspect the inputs for &lt;em&gt;applybqsr&lt;/em&gt; also need to be joined into a single queue channel.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastEditorUserId="10775" LastEditDate="2023-07-14T00:51:33.487" LastActivityDate="2023-07-14T00:51:33.487" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21306" PostTypeId="2" ParentId="21300" CreationDate="2023-07-13T01:54:00.833" Score="2" Body="&lt;p&gt;I originally thought you were working on humans. I spotted &amp;quot;non model organism&amp;quot;, that changes things alot, albeit I guess humans are a non-model organism in a way. The question is what the organism. LD has been applied to every kingdom of life, except viruses. Arlequin isn't bad for relatively small to medium data sets (but tricky to get the format right - thats a separate question). If it's a genome sequence then I still suspect its plink. There are a number of recent LD programs BTW.&lt;/p&gt;&#xA;&lt;p&gt;I did notice Colony was written in Fortran! Wow, the author must be a physicist at ZSL anyway...&lt;/p&gt;&#xA;&lt;p&gt;If you have a defined number of SNPs - sounds small to medium data set - then I'd use Arlquein &lt;a href=&quot;http://cmpg.unibe.ch/software/arlequin35/Arl35Methods.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://cmpg.unibe.ch/software/arlequin35/Arl35Methods.html&lt;/a&gt;, Arlquin does loads of pop gen BTW its very famous for AMOVA BTW (very cool calculation). HAPLOVIEW has been used for LD, these authors used it &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4641910/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, what surprised me was they avoided using Arlequin for the LD but used HAPLOVIEW. There's a plethora of recent LD algorithms, never used them so can' say, e.g. &lt;a href=&quot;https://onlinelibrary.wiley.com/doi/10.1111/1755-0998.13438&quot; rel=&quot;nofollow noreferrer&quot;&gt;quickLD&lt;/a&gt;, stuff in R.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;What I am definitely saying is the LD works on from raw alleles&lt;/em&gt; Thats the most important part of this response. I've attempted to describe the rationale below.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;If this is human genomes, or large genome data I'd would recommend plink analysis performed directly on your data, as described here &lt;a href=&quot;https://www.cog-genomics.org/plink/1.9/ld&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cog-genomics.org/plink/1.9/ld&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Using any sort of matrix will remove the linkage disequilibrium signal, &lt;strong&gt;IF&lt;/strong&gt; its a single matrix for all your data. If its multiple matrices you can perform a correlation statistic between the matrices - but its better just to do a straight linkage disequilibrium calculation, e.g. plink.&lt;/p&gt;&#xA;&lt;p&gt;The reason why correlation matrices will fail is because LD needs to identify incongruence/disparity and whether that incongruence occurs within random expectations.&lt;/p&gt;&#xA;&lt;p&gt;Basically you are making a phylogenetic tree - thats what the correlation matrix is about, but the tree assumes clonality. LD is about deviation from random expectations (equilibrium), so either a random distribution, e.g. Poison or a randomisation process is needed. What I'm attempting to say is &amp;quot;a tree&amp;quot; (single matrix) and LD are very different calculations. Whilst, technically you can do LD via matrices (plural), its better to do a LD calculation from the raw alleles.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;From the comments, it appears a &amp;quot;tree like structure&amp;quot; (ancestors) is wanted to direct an LD calculation.&lt;/p&gt;&#xA;&lt;p&gt;What you can do is subdivide the populations and perform LD on each subdivision. A tree (here using a correlation or kinship matrix) is a continuum so it's at what point do you call the the subdivision? Its doable but the information is not presented here.&lt;/p&gt;&#xA;&lt;p&gt;What you can do is use &amp;quot;&lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fgene.2013.00098/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;Structure&lt;/a&gt;&amp;quot; - this has been upgraded alot. This is a HWE, I think using Bayes and it designates how many populations (at LD) are present in your data. You can then perform a specific LD directly on each subdivision.&lt;/p&gt;&#xA;&lt;p&gt;If you compare the output of &amp;quot;Structure&amp;quot;,basically Bayesian HWE you should see a correlation with your kinship matrix. What you are inferring is a partial LD parameter that will define ancestry and whilst its doable it has never been calculated in this manner that I'm aware of: its random or it ain't and thats the criteria of significance. This is definitely how LD works in small genomes (everything below higher eukaryotes).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-07-13T15:34:16.370" LastActivityDate="2023-07-13T15:34:16.370" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21307" PostTypeId="2" ParentId="21290" CreationDate="2023-07-13T06:25:06.360" Score="2" Body="&lt;p&gt;I have very little idea what's going on here, but in the absence of any other answers or help, I'll try to work through this from a no-knowledge start point to see if I can hopefully help shift this problem closer towards a solution.&lt;/p&gt;&#xA;&lt;p&gt;The specific error you have encountered is &lt;code&gt;No valid contrasts/methods specified&lt;/code&gt;. Looking at the &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/vignettes/DiffBind/inst/doc/DiffBind.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;DiffBind Vignette&lt;/a&gt;, it provides this information about defining contrasts:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Before running the differential analysis, we need to tell DiffBind how to model the data, including which comparison(s) we are interested in. This is done using the &lt;code&gt;dba.contrast&lt;/code&gt; function, as follows:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; tamoxifen &amp;lt;- dba.contrast(tamoxifen, reorderMeta=list(Condition=&amp;quot;Responsive&amp;quot;))  &#xA;&amp;gt; tamoxifen&#xA;11 Samples, 2845 sites in matrix:&#xA;       ID Tissue Factor  Condition  Treatment Replicate   Reads FRiP&#xA; 1 BT4741  BT474     ER  Resistant Full-Media         1  652697 0.16&#xA; 2 BT4742  BT474     ER  Resistant Full-Media         2  663370 0.15&#xA; 3  MCF71   MCF7     ER Responsive Full-Media         1  346429 0.31&#xA; 4  MCF72   MCF7     ER Responsive Full-Media         2  368052 0.19&#xA; 5  MCF73   MCF7     ER Responsive Full-Media         3  466273 0.25&#xA; 6  T47D1   T47D     ER Responsive Full-Media         1  399879 0.11&#xA; 7  T47D2   T47D     ER Responsive Full-Media         2 1475415 0.06&#xA; 8 MCF7r1   MCF7     ER  Resistant Full-Media         1  616630 0.22&#xA; 9 MCF7r2   MCF7     ER  Resistant Full-Media         2  593224 0.14&#xA;10  ZR751   ZR75     ER Responsive Full-Media         1  706836 0.33&#xA;11  ZR752   ZR75     ER Responsive Full-Media         2 2575408 0.22&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After showing your contrast result, you explained a partial removal of the error:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If I try to compare MS vs MF, I will get that error but if I remove MS and replace by other sample, I don't have that error anymore.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Specifically relating to MS vs MF, I can see why there would be an issue with the condition contrast, especially if it's using DESeq2 in the background. The short answer is that there is not enough variation within the groups.&lt;/p&gt;&#xA;&lt;p&gt;DESeq2 cares about making sure that conditions added to the model are separable from samples. This is further explained in the DESeq2 manual &lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#model-matrix-not-full-rank&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The issue is that there's no way to distinguish between the control condition and the MS sample (or, similarly, between the diseased condition and the MF sample). In order to get past this error, you need both control and diseased in your MS group or in your MF group (or both, if you want to be &lt;em&gt;really&lt;/em&gt; rigorous), and ideally a few more replicates of each condition.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-07-14T01:00:04.230" LastActivityDate="2023-07-14T01:00:04.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21308" PostTypeId="1" CreationDate="2023-07-13T06:30:59.510" Score="0" ViewCount="47" Body="&lt;p&gt;Can we do protein-protein simulation through Gromacs? if yes, can anyone teach me the correct procedure to get this done.&lt;/p&gt;&#xA;&lt;p&gt;Thank you,&lt;/p&gt;&#xA;" OwnerUserId="14059" LastActivityDate="2023-07-13T06:30:59.510" Title="Protein-Protein Simulation using Gromacs" Tags="|docking|protein-protein-interaction|simulated-data|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21309" PostTypeId="2" ParentId="21302" CreationDate="2023-07-13T14:20:40.307" Score="1" Body="&lt;p&gt;I'd use GNU parallel. I've not tested this with real files but try it out, ensure the &lt;code&gt;{}&lt;/code&gt; is passed to the process substitution:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mkdir with_header&#xA;parallel samtools view -b &amp;lt;(cat my_header {}) -o with_header/{} ::: *.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The command could be better - out file name could be done better but you get the concept.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-07-13T14:20:40.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21310" PostTypeId="1" AcceptedAnswerId="21312" CreationDate="2023-07-13T18:02:57.103" Score="1" ViewCount="164" Body="&lt;p&gt;I've normal-tumor human FASTQ file(s). Currently, I work with only one sample (both normal and tumor fastq R1 and R2), only.&lt;br /&gt;&#xA;I have alignment gatk mark duplicates, setupnmdtags, applybqsr, recalibrate steps before calling mutect2.&lt;/p&gt;&#xA;&lt;p&gt;My input files are:&lt;br /&gt;&#xA;tiny_tumor_R1.fastq.gz&lt;br /&gt;&#xA;tiny_tumor_R2.fastq.gz&lt;br /&gt;&#xA;tiny_normal_R1.fastq.gz&lt;br /&gt;&#xA;tiny_normal_R2.fastq.gz&lt;/p&gt;&#xA;&lt;p&gt;The expected output from the mutect2 is &lt;strong&gt;tiny.mutect2out.vcf&lt;/strong&gt; that is ${sample_name}&amp;quot;.mutect2out.vcf&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;I split &lt;code&gt;tiny_normal&lt;/code&gt; using nextflow split and take first element of the split array, that is tiny.&lt;/p&gt;&#xA;&lt;p&gt;However, I get error as:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Caused by:   Missing value declared as output parameter: sample_name&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I understand that the variable was defined in the script step. How do I retrieve at the output level?&lt;/p&gt;&#xA;&lt;p&gt;Below is code of the mutect2_process.nf&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process mutect2 {&#xA;&#xA;        maxForks 3&#xA;        debug true&#xA;&#xA; memory '15 GB'&#xA;&#xA;        input:&#xA;        tuple val(sample_id_tumor),path(bqsrbam_tumor)&#xA;        tuple val(sample_id_normal),path(bqsrbam_normal)&#xA;        &#xA;        output:&#xA;        tuple val(sample_name),path(${sample_name}&amp;quot;.mutect2out.vcf&amp;quot;)&#xA;        &#xA;&#xA;        script:&#xA;        def avail_mem = task.memory ? task.memory.toGiga() : 0&#xA;                def java_options = [&#xA;        avail_mem ? &amp;quot;-Xmx${avail_mem}G&amp;quot; : &amp;quot;&amp;quot;,&#xA;        &amp;quot;-XX:+UseSerialGC&amp;quot;,&#xA;    ]&#xA;        def sample_name=sample_id_normal.split('_normal')[0]&#xA;&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;                &#xA;        ml gatk/4.1.3.0&#xA;gatk --java-options &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${java_options.join(' ')}&quot; Mutect2 -L $&lt;/span&gt;params.target -I &lt;span class=&quot;math-container&quot;&gt;${bqsrbam_tumor} -I $&lt;/span&gt;{bqsrbam_normal} -normal &lt;span class=&quot;math-container&quot;&gt;${sample_name} -tumor $&lt;/span&gt;{sample_name} --output &lt;span class=&quot;math-container&quot;&gt;${sample_name}&quot;.mutect2out.vcf&quot; -R $&lt;/span&gt;params.hg38genome&#xA;&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="862" LastActivityDate="2023-07-14T01:31:47.550" Title="Nextflow - Missing value declared as output parameter - output variable defined in script" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21311" PostTypeId="1" CreationDate="2023-07-13T23:28:55.107" Score="1" ViewCount="365" Body="&lt;p&gt;I'm trying to download pdf from pubmed central using selenium webdriver in python. The code is provided below which downloads the pdf fine but, it does not quit automatically. i.e. it is sort of stuck executing &lt;code&gt;browser.get(href)&lt;/code&gt; line in &lt;code&gt;download_pmc_pdf()&lt;/code&gt;. What could be the reason?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;from selenium import webdriver&#xA;from selenium.webdriver.common.by import By&#xA;from selenium.webdriver.firefox.options import Options&#xA;from selenium.webdriver.support.ui import WebDriverWait&#xA;from selenium.webdriver.support import expected_conditions as EC&#xA;ncbi_url = &amp;quot;https://www.ncbi.nlm.nih.gov/&amp;quot;&#xA;&#xA;download_dir = &amp;quot;pdf_downloaded&amp;quot;&#xA;&#xA;if not os.path.exists(download_dir):&#xA;    os.mkdir(download_dir)&#xA;    &#xA;options = Options()&#xA;options.set_preference('permissions.default.stylesheet', 2)&#xA;options.set_preference('permissions.default.image', 2)&#xA;options.set_preference('browser.download.folderList', 2)&#xA;options.set_preference('browser.download.dir', os.path.abspath(download_dir))&#xA;options.set_preference('browser.download.manager.showWhenStarting', False)&#xA;options.set_preference('browser.helperApps.neverAsk.saveToDisk', 'application/pdf')&#xA;options.set_preference('pdfjs.disabled', True)&#xA;&#xA;browser = webdriver.Firefox(options=options)&#xA;&#xA;###############################################################################&#xA;def download_pmc_pdf(pmcid_key):&#xA;    pmc_url = ncbi_url + &amp;quot;pmc/articles/&amp;quot; + pmcid_key + &amp;quot;/&amp;quot;&#xA;    browser.get(pmc_url)&#xA;    try:&#xA;        wait = WebDriverWait(browser, 10)&#xA;        pdf_links = wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, '.pmc-sidebar__formats li a')))&#xA;        &#xA;        # Iterate over all links and navigate to the link containing '.pdf' in the href attribute&#xA;        for link in pdf_links:&#xA;            href = link.get_attribute('href')&#xA;            if '.pdf' in href:&#xA;                browser.get(href)&#xA;                break  # exit the loop as we have found the PDF link&#xA;        print(&amp;quot;line beofre browser.quit()&amp;quot;)&#xA;        browser.quit()  # quit the browser outside the loop&#xA;    except Exception as e:&#xA;        print(e)&#xA;&#xA;&#xA;    &#xA;###############################################################################&#xA;pmcid = &amp;quot;2788555&amp;quot;            &#xA;download_pmc_pdf(pmcid)&#xA;browser.quit()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7177" LastActivityDate="2023-07-17T06:09:16.797" Title="selenium webdriver is stuck after downloading and saving a pdf file from pubmed central website" Tags="|python|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21312" PostTypeId="2" ParentId="21310" CreationDate="2023-07-14T01:31:47.550" Score="1" Body="&lt;p&gt;You would like to use the &lt;em&gt;sample_name&lt;/em&gt; variable outside the &lt;code&gt;script&lt;/code&gt; block, but the &lt;code&gt;def&lt;/code&gt; keyword defines a variable locally. All you'd need to do is remove the &lt;code&gt;def&lt;/code&gt; keyword to make it work, but usually what you'd want to do is to define your &lt;em&gt;sample_name&lt;/em&gt; variable outside of your process block in your workflow block like in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/21305/10775&quot;&gt;my previous answer&lt;/a&gt;. I would also call this variable something else like &lt;em&gt;pair_id&lt;/em&gt;, &lt;em&gt;donor_name&lt;/em&gt;, &lt;em&gt;case&lt;/em&gt; or &lt;em&gt;patient&lt;/em&gt; perhaps. I would say you actually have two sample names here: one tumor sample (i.e. &lt;em&gt;tiny_tumor&lt;/em&gt;) and one normal sample (i.e. &lt;em&gt;tiny_normal&lt;/em&gt;) from the same patient/donor/case.&lt;/p&gt;&#xA;&lt;p&gt;You may also run into issues if &lt;em&gt;both&lt;/em&gt; of your input channels are &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#queue-channel&quot; rel=&quot;nofollow noreferrer&quot;&gt;queue channels&lt;/a&gt;. You may not notice any issues if you're testing only a single pair of samples. Try just copying your tiny files a couple of times and giving your workflow a try, for example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in tiny_{tumor,normal}_R{1,2}.fastq.gz ; do&#xA;    for j in foo bar baz; do&#xA;        cp &amp;quot;${i}&amp;quot; &amp;quot;${j}_${i#*_}&amp;quot;&#xA;    done&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Most of the time what you'll want is one &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#queue-channel&quot; rel=&quot;nofollow noreferrer&quot;&gt;queue channel&lt;/a&gt; and one or more &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#value-channel&quot; rel=&quot;nofollow noreferrer&quot;&gt;value channels&lt;/a&gt; when you require &lt;a href=&quot;https://www.nextflow.io/docs/latest/process.html#process-multiple-input-channels&quot; rel=&quot;nofollow noreferrer&quot;&gt;multiple input channels&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-07-14T01:31:47.550" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21313" PostTypeId="1" CreationDate="2023-07-14T15:10:59.903" Score="1" ViewCount="78" Body="&lt;p&gt;I have some highly contaminated ancient DNA sequences. I have adapter removed and collapsed these and run them through Kraken2. The Kraken reports show multiple levels of taxa with good numbers of reads in each.&lt;/p&gt;&#xA;&lt;p&gt;However, when I run Bracken - I keep getting an error saying it can't find any reads.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error: no reads found. Please check your Kraken report&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have tried adjusting the taxonomic level (all the way up to Phylum) and get the same result. I have tried using different databases to run the Kraken analysis through and updating both packages.&lt;/p&gt;&#xA;&lt;p&gt;Any ideas what could be going wrong?&lt;/p&gt;&#xA;&lt;p&gt;Update:&lt;/p&gt;&#xA;&lt;p&gt;Here's the head of one of the report files:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/5nrA0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5nrA0.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am also using the latest version of Bracken downloaded via Bioconda. My command to run Bracken is via a bash script:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#SBATCH --partition=day&#xA;#SBATCH --output=slurm_bracken_job%J.out&#xA;#SBATCH --mem-per-cpu=10G&#xA;#SBATCH --cpus-per-task=32&#xA;&#xA;# we load kraken2 and bracken into our environment:&#xA;ml kraken2&#xA;ml bracken&#xA;&#xA;NAME=$1&#xA;&#xA;srun bracken -d /workspaces/groups/database/nt-taxonomy-2021-02-04_braken/ -i &lt;span class=&quot;math-container&quot;&gt;${NAME}.kreport -o $&lt;/span&gt;{NAME}.bracken -r 75 -l S&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is then run using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in &lt;span class=&quot;math-container&quot;&gt;$(ls -1 ./*.report | sed 's/.report//'); do sbatch Test_Kraken.sh &quot;$&lt;/span&gt;i&amp;quot;; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17816" LastEditorUserId="650" LastEditDate="2023-11-09T15:39:46.670" LastActivityDate="2023-11-09T15:39:46.670" Title="Bracken not working" Tags="|sequence-analysis|metagenome|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21314" PostTypeId="2" ParentId="21311" CreationDate="2023-07-14T15:57:30.880" Score="2" Body="&lt;p&gt;I had to execute a JavaScript code that opens the pdf link in a new window/tab. After that, just closed this newly opened window/tab and continued with the script execution. Here is the updated code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import os&#xA;import time&#xA;import csv&#xA;&#xA;from selenium import webdriver&#xA;from selenium.webdriver.common.by import By&#xA;from selenium.webdriver.firefox.options import Options&#xA;from selenium.webdriver.support.ui import WebDriverWait&#xA;from selenium.webdriver.support import expected_conditions as EC&#xA;&#xA;ncbi_url = &amp;quot;https://www.ncbi.nlm.nih.gov/&amp;quot;&#xA;&#xA;download_dir = &amp;quot;pdf_downloaded&amp;quot;&#xA;&#xA;if not os.path.exists(download_dir):&#xA;    os.mkdir(download_dir)&#xA;    &#xA;options = Options()&#xA;options.set_preference('permissions.default.stylesheet', 2)&#xA;options.set_preference('permissions.default.image', 2)&#xA;options.set_preference('browser.download.folderList', 2)&#xA;options.set_preference('browser.download.dir', os.path.abspath(download_dir))&#xA;options.set_preference('browser.download.manager.showWhenStarting', False)&#xA;options.set_preference('browser.helperApps.neverAsk.saveToDisk', 'application/pdf')&#xA;options.set_preference('pdfjs.disabled', True)&#xA;&#xA;###############################################################################&#xA;def download_pmc_pdf(pmcid_key):&#xA;    browser = webdriver.Firefox(options=options)&#xA;    pmc_url = ncbi_url + &amp;quot;pmc/articles/&amp;quot; + pmcid_key + &amp;quot;/&amp;quot;&#xA;    browser.get(pmc_url)&#xA;    try:&#xA;        wait = WebDriverWait(browser, 10)&#xA;        pdf_links = wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR, '.pmc-sidebar__formats li a')))&#xA;&#xA;        # Iterate over all links and navigate to the link containing '.pdf' in the href attribute&#xA;        for link in pdf_links:&#xA;            href = link.get_attribute('href')&#xA;            if '.pdf' in href:&#xA;                # Open the pdf link in a new window&#xA;                browser.execute_script(f'window.open(&amp;quot;{href}&amp;quot;,&amp;quot;_blank&amp;quot;);')&#xA;                break&#xA;&#xA;        time.sleep(15)  # delay for 10 seconds&#xA;        # After the file is downloaded:&#xA;        with open(f'{download_dir}/file_mapping.csv', 'a', newline='') as file:&#xA;            writer = csv.writer(file)&#xA;            writer.writerow([pmcid_key, os.path.basename(href)])&#xA;        ## Switch to the new window and close it &#xA;        ## (this is a hack to deal with bowser freezing upon downloading)&#xA;        browser.switch_to.window(browser.window_handles[-1])&#xA;        browser.close()&#xA;&#xA;    except Exception as e:&#xA;        print(e)&#xA;    finally:&#xA;        browser.quit()  # quit the browser outside the loop    &#xA;###############################################################################&#xA;pmcid = &amp;quot;2788555&amp;quot;   &#xA;download_pmc_pdf(pmcid)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="7177" LastEditorUserId="7177" LastEditDate="2023-07-14T16:02:37.583" LastActivityDate="2023-07-14T16:02:37.583" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21315" PostTypeId="2" ParentId="18299" CreationDate="2023-07-14T17:08:11.123" Score="3" Body="&lt;p&gt;M__'s approach is good, but, as explained, some subset of the unambiguous DNA sequences will not be recovered because of the ambiguity in reverse-translation due to the degeneracy of the genetic code. I don't know a &lt;code&gt;python&lt;/code&gt; approach, but here's an &lt;code&gt;R&lt;/code&gt; function that returns a vector of all the possible unambiguous DNA strings:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;make_unambiguous &amp;lt;- function(dna) {&#xA;  require(tidyverse)&#xA;  require(S4Vectors)&#xA;  iupac &amp;lt;- tibble(code = c(&amp;quot;A&amp;quot;, &amp;quot;C&amp;quot;, &amp;quot;G&amp;quot;, &amp;quot;T&amp;quot;,&#xA;                           &amp;quot;R&amp;quot;, &amp;quot;Y&amp;quot;, &amp;quot;S&amp;quot;, &amp;quot;W&amp;quot;,&#xA;                           &amp;quot;K&amp;quot;, &amp;quot;M&amp;quot;, &amp;quot;B&amp;quot;, &amp;quot;D&amp;quot;,&#xA;                           &amp;quot;H&amp;quot;, &amp;quot;V&amp;quot;, &amp;quot;N&amp;quot;),&#xA;                  base = c(&amp;quot;A&amp;quot;, &amp;quot;C&amp;quot;, &amp;quot;G&amp;quot;, &amp;quot;T&amp;quot;,&#xA;                           &amp;quot;AG&amp;quot;, &amp;quot;CT&amp;quot;, &amp;quot;GC&amp;quot;, &amp;quot;AT&amp;quot;,&#xA;                           &amp;quot;GT&amp;quot;, &amp;quot;AC&amp;quot;, &amp;quot;CGT&amp;quot;, &amp;quot;AGT&amp;quot;,&#xA;                           &amp;quot;ACT&amp;quot;, &amp;quot;ACG&amp;quot;, &amp;quot;ACGT&amp;quot;))&#xA;&#xA;  tibble(dna) |&amp;gt;&#xA;    separate_rows(dna, sep = '(?&amp;lt;=.)(?=.)') |&amp;gt;&#xA;    left_join(iupac, by = c(&amp;quot;dna&amp;quot; = &amp;quot;code&amp;quot;)) |&amp;gt;&#xA;    pull(base) |&amp;gt;&#xA;    str_split(&amp;quot;&amp;quot;) |&amp;gt;&#xA;    expand.grid(stringsAsFactors = FALSE) |&amp;gt;&#xA;    unite(col = sequence, sep = &amp;quot;&amp;quot;) |&amp;gt;&#xA;    as_vector()&#xA;}&#xA;&#xA;make_unambiguous(dna = &amp;quot;AAYGANAGYCARAGYAAR&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3967" LastEditorUserId="3967" LastEditDate="2023-08-16T16:26:04.057" LastActivityDate="2023-08-16T16:26:04.057" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21317" PostTypeId="1" CreationDate="2023-07-14T20:24:55.223" Score="2" ViewCount="109" Body="&lt;p&gt;I'm following along with some pre written code to learn how to do differential expression. I am using the same data set and have gone back through the code and copy and pasted what they used and the follow along gets the two headings rather than the single one that I am getting, so I'm a bit confused how I'm getting different results.&lt;/p&gt;&#xA;&lt;p&gt;I have some data which I have put through DESeq but the output for resultsNames isn't what I expect. The code is&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData = exprs(bottomly.eset), colData = pData(bottomly.eset), design = ~ strain)&#xA;dds &amp;lt;- DESeq(dds)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but this code produces this output &lt;code&gt;&amp;quot;Intercept&amp;quot;  &amp;quot;strain_DBA.2J_vs_C57BL.6J&amp;quot;&lt;/code&gt; whereas the output that I intended was &lt;code&gt;&amp;quot;Intercept&amp;quot;  &amp;quot;strain_DBA.2J&amp;quot;   &amp;quot;C57BL.6J&amp;quot;&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Is there a simple way to change this within the code that I have or is this something related to the dataset I used?&lt;/p&gt;&#xA;" OwnerUserId="17820" LastEditorUserId="73" LastEditDate="2023-07-15T19:53:08.297" LastActivityDate="2023-07-15T19:53:08.297" Title="Understanding DESeq2 output for resultNames" Tags="|differential-expression|deseq2|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21318" PostTypeId="2" ParentId="21293" CreationDate="2023-07-14T23:38:56.750" Score="0" Body="&lt;p&gt;It depends on what software you have in hand and want to use. But basically, when you delete/add a residue, you can't just stick two parts together since it affects the whole conformation.&lt;/p&gt;&#xA;&lt;p&gt;I suggest using AlphaFold2, and input it with different sequences (including deletion mutation) to see how it affects the structure.&lt;/p&gt;&#xA;&lt;p&gt;Here is a link to AF2 colab:&#xA;&lt;a href=&quot;https://colab.research.google.com/github/sokrypton/ColabFold/blob/main/AlphaFold2.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://colab.research.google.com/github/sokrypton/ColabFold/blob/main/AlphaFold2.ipynb&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17819" LastActivityDate="2023-07-14T23:38:56.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21319" PostTypeId="2" ParentId="21317" CreationDate="2023-07-15T00:20:18.407" Score="2" Body="&lt;p&gt;DESeq2 is a package designed for calculating &lt;em&gt;differential&lt;/em&gt; expression, in other words for comparing expression between two states. The primary results it produces are therefore comparative values (e.g. comparing DBA.2J and C57BL.6J in your case).&lt;/p&gt;&#xA;&lt;p&gt;Given that this does not seem to be your intended calculation, it would be helpful if you could provide more context around what you want to use DESeq2 for. If it's not for differential expression, there's likely a better tool / approach for solving your problem.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-07-15T00:20:18.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21320" PostTypeId="2" ParentId="21317" CreationDate="2023-07-15T10:59:50.467" Score="2" Body="&lt;p&gt;So I was following along with a coding lesson from Youtube learning how to do differential expression in R, if the data had been my own I probably would have been able to figure out what needed to be done.&#xA;The code that worked for me was&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData = exprs(bottomly.eset), colData = pData(bottomly.eset), design = ~ strain)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;dds &amp;lt;- DESeq(dds, betaPrior = TRUE)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I was then able to use this code below which was resulting in the error previously&#xA;&lt;code&gt;contrast.deseq2 &amp;lt;- list(c(&amp;quot;strainC57BL.6J&amp;quot;, &amp;quot;strainDBA.2J&amp;quot;))&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thank you to everyone who commented on this question&lt;/p&gt;&#xA;" OwnerUserId="17820" LastActivityDate="2023-07-15T10:59:50.467" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21321" PostTypeId="1" AcceptedAnswerId="21322" CreationDate="2023-07-15T15:14:45.173" Score="1" ViewCount="37" Body="&lt;p&gt;Doing complex sql query on Chembl database and it takes 40 minutes on the machine from lab.&#xA;I was thinking using dask, extract each table and export to parquet, when do the data processing .&#xA;Is this a good approach or there is a better alternative?&lt;/p&gt;&#xA;" OwnerUserId="17825" LastActivityDate="2023-07-15T16:17:52.060" Title="Sql query slow on Chembl postgres database" Tags="|python|database|data-retrieval|cheminformatics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21322" PostTypeId="2" ParentId="21321" CreationDate="2023-07-15T15:41:05.110" Score="0" Body="&lt;p&gt;I suspect the issue is you want to parallelise (multi-threading really) the job to reduce run time.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;parquet&lt;/code&gt; is recommended. It is simply an archive format that allows compression and is often used with a &lt;code&gt;pandas&lt;/code&gt; dataframe. Use &lt;code&gt;parquet&lt;/code&gt; to store data by all means, but thats easy via &lt;code&gt;pyarrow&lt;/code&gt; and the &lt;code&gt;pandas&lt;/code&gt; command&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    df.to_parquet('/User/OP/location/file,parquet' ,engine='pyarrow', compression='gzip')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You will need to install &lt;code&gt;pyarrow&lt;/code&gt; via pip or conda, there is a faster version (forget its name, something like &lt;code&gt;fastparquet&lt;/code&gt; or something used by default). Yeah it's a very good archiving format, in fact its industrial standard and I should use it more.&lt;/p&gt;&#xA;&lt;p&gt;Python has loads of ways to parallelise of which dask is one way. dask is an entire framework. To leverage dask to parallelise postgres I would refer to this &lt;a href=&quot;https://stackoverflow.com/questions/75825264/parallel-and-distribute-across-npartitions-with-dask-and-postgresql-db&quot;&gt;post&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I don't use dask, I am aware it is a solution in high performance Python. Personally I would use this approach directly via &lt;a href=&quot;https://medium.com/nerd-for-tech/postgres-major-version-upgrade-parallel-analyze-with-python-af9931ffd087&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;multiprocessor&lt;/code&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from psycopg2 import connect&#xA;import multiprocessor&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;... etc ...&lt;/p&gt;&#xA;&lt;p&gt;I would suggest its horses for courses, I simply know how to multiprocess in Python so I don't bother with dask.&lt;/p&gt;&#xA;&lt;p&gt;To answer the question in summary&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;parquet is the best, if SPEED is not important. If speed is important then &lt;code&gt;feather&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;dask is mainstream within high performance Python and its doable. There's no reason not to&lt;/li&gt;&#xA;&lt;li&gt;There are other multithread Postgress approaches and thats what I would do, but if you just getting into parallelisation you may as well use dask. It will take some learning though.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;If you're not into parallelisation and you're not writing a data pipeline for publication I would simply consider getting a machine with more RAM. It shouldn't be taking that length of time, parallelisation will solve the run time - no question - but it's the time it will take to get to grips with a parallel solution. What is likely occurring is the job and database size is exceeding the RAM capacity of the machine. At a wild random guess a 16 or 32 Meg RAM chip would collapse the runtime to a few minutes and thats what you are looking for.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;From the comments.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;parquet&lt;/code&gt; is really for data compression and archiving. The issue is I/o is reasonably fast but not lightening fast. It is probably slower than &lt;code&gt;pickle&lt;/code&gt; (it will be slower if you're compressing via gzip as well).&lt;/p&gt;&#xA;&lt;p&gt;For a 'temporary' save (couple months) and really fast i/o then its &lt;code&gt;feather&lt;/code&gt;. There is no dispute about that.&lt;/p&gt;&#xA;&lt;p&gt;If you are keeping your data a long time and need to save space its &lt;code&gt;parquet&lt;/code&gt;. Performance-wise &lt;code&gt;feather&lt;/code&gt; is way faster than &lt;code&gt;parquet&lt;/code&gt;. I've never used &lt;code&gt;feather&lt;/code&gt; BTW I use &lt;code&gt;pickle&lt;/code&gt; or &lt;code&gt;parquet&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;To give some idea a 500Meg flat file compressed an archived via &lt;code&gt;parquet&lt;/code&gt; resulting in a 50Meg file, and &lt;em&gt;will take several minutes to write/import&lt;/em&gt;. Thats 'cause it's doing loads of stuff. My suspicion is that this too slow for you.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-07-15T16:17:52.060" LastActivityDate="2023-07-15T16:17:52.060" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21323" PostTypeId="1" CreationDate="2023-07-15T20:42:01.170" Score="1" ViewCount="70" Body="&lt;p&gt;I am doing amplicon sequencing of a virus across many different regions. Lets say I have 20k unique variants of the same virus that I put into my pcr assay and after sequencing and amplifications I am left with, 19k variants that appear. But many of the variants appear at really low read count and I don't know if that read is real or just noise. I have a threshold T that I say if a variant appears above T I'm counting it as real. Out of the 20k variants about only 15% of variants appear to be &amp;quot;real&amp;quot;. I have some data on different input species counts (2k, 10k, 20k, 40k), and their corresponding fraction of &amp;quot;real&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;My question is how do I determine which input variants count is best? I only looked at 4 different values and maybe the correct value isn't sampled, I obviously can't try everything between 2k and 40k. I think I want a balance between largest fraction of &amp;quot;real&amp;quot; variants and total number of variants.&lt;/p&gt;&#xA;&lt;p&gt;Is there any research (or better yet code) that answers/discusses this problem?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advanced.&lt;/p&gt;&#xA;" OwnerUserId="17826" LastEditorUserId="3848" LastEditDate="2023-07-16T05:21:21.660" LastActivityDate="2023-07-19T07:49:56.943" Title="Choosing the best number of species for assay" Tags="|ngs|variant-calling|virus|pcr|qpcr|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21324" PostTypeId="2" ParentId="21301" CreationDate="2023-07-16T00:10:44.227" Score="0" Body="&lt;p&gt;Oh this is fairly easy. The file size has exceed the capacity of Excel (50000 rows??). This doesn't affect downstream analysis because the rows are there you just can't visualise them.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;BBEdit&lt;/strong&gt; will open it if this is Mac (free from AppStore - really good BTW). If this is Linux then &lt;code&gt;vi&lt;/code&gt; or &lt;code&gt;vim&lt;/code&gt;. A Windows version of BBEdit is &lt;strong&gt;Notepad++&lt;/strong&gt;, please not Notepad.&lt;/p&gt;&#xA;&lt;p&gt;You can request the number of lines (bash Linux) via&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;wc weir.fst &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will return the number of lines in the file, if it's &amp;gt;50000, i.e. beyond the number of the Excel limit, that's your answer. You just need to find an alternative viewer&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-07-16T00:10:44.227" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21325" PostTypeId="2" ParentId="21323" CreationDate="2023-07-16T03:10:58.563" Score="1" Body="&lt;p&gt;The underlying answer to the question is have you experimentally controlled for &lt;em&gt;in vitro&lt;/em&gt; artefacts? Thats taking a cloned virus and performing the same RT-PCR. If not thats makes things very difficult. If there isn't a cloned virus thats a problem. You might need to ask your wet-lab colleagues what old school &amp;quot;cloning a virus&amp;quot; means, or Biology SE will explain this. The reason this is important is given below.&lt;/p&gt;&#xA;&lt;p&gt;The first step to the question is simply to plot a histogram against copy number. What you will find is a heavily skewed distribution where the copy number of one variant will represent &amp;gt;40% (at a wild guess) of all the viral copy number. Moreover, a handful (less than 100) of variants will comprise 90% of all the copy number. Once that is established it is much easier to assess what is a true variant. What I'm saying is the difference is one log minimum between what you think you are looking at and what is there.&lt;/p&gt;&#xA;&lt;p&gt;The sources of artefact:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;RT-PCR artefacts, particularly if there is an RT step: Taq polymerase alone has an error rate of 1 in 1000 and RT has a very high error rate, worse it's the first step.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;What DNA polymerase was used?&lt;/p&gt;&#xA;&lt;p&gt;The key issue is separating the RT-PCR artefacts from naturally occurring RNA polymerase errors &lt;em&gt;in situ&lt;/em&gt; is not possible without a control. If a frequency analysis cut-off strategy is used it needs to be an ultra-conservative threshold, i.e. I mean ridiculously high and this means lot of variants are going to be dumped.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Why?&lt;/strong&gt; Again it's back to the problem whether the variants were generated &lt;em&gt;in vitro&lt;/em&gt;, i.e. artefacts, or &lt;em&gt;in situ&lt;/em&gt;, i.e. resulting from RNA polymerase. Generating a cut-off within a skewed distribution is not trivial &lt;em&gt;per se&lt;/em&gt; but extremely difficult when the ratio between artefact and genuine variant has not been controlled.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;There appears some sort of control. Maximising the threshold: thats a function of the control, so the control needs to accommodate a model of the different numbers of input variables; it will not be a linear function. Basically what input will give me the best representation of non-artifactual variants, that is what the control has to find out. Thus, for every input, you want a control.&lt;/p&gt;&#xA;&lt;p&gt;Again, if the control is lacking, in any context, then the rule of thumb is to be ultra-conservative, not trying to maximise the number of variants, but minimise them, particularly if the control isn't perfect. Most virological reviewers raise concern with the numbers been reported here on the information given (obviously we don't know the virus).&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;NGS sequencing is another source of artefact - I should have mentioned there are some famous disasters here. I'm sure you NGS is really good, but a small number of investigators have deposited a disproportionate number of artefact ridden sequences - that everyone knows about.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;from comments (again)&lt;/p&gt;&#xA;&lt;p&gt;A control of 4 is tiny in data science especially a GLM approach. However, other studies would have been performed - possibly lots of them if this was e.g. HIV - and controls would have been run. There are two major strategies (really 3) for biologically cloning a virus. Really you'll need 100+ controls, that may not be feasible but as many within 4 to 100 as are available. GLM would be a good approach IMO. It would need stating that this does not truly control the rate of artefacts, but it's much closer.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Final comments ... don't want to assess thresholds.&lt;/p&gt;&#xA;&lt;p&gt;I gave this question some thought and I think it would work via machine learning (what I specialise in). However, it may be difficult to find appropriate published controls and/or augmentation. Personally, I'm satisfied this would work if done correctly, but implementation is fairly advance ML. Given the amount of time I've spent on thinking this question through I wouldn't use an alternative approach, because I think it would be valid. Augmentation BTW is a generic term in machine learning and deep learning to increase the size of the training set.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-07-19T07:49:56.943" LastActivityDate="2023-07-19T07:49:56.943" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21327" PostTypeId="2" ParentId="21311" CreationDate="2023-07-17T06:09:16.797" Score="1" Body="&lt;p&gt;Pubmed Central stores pdfs for all available open access articles on a public-facing ftp server (&lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/&lt;/a&gt;). You can query the database using the OA Web Service API, e.g. with cURL&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# for pubmed id &amp;quot;10026098&amp;quot;&#xA;curl https://www.ncbi.nlm.nih.gov/pmc/utils/oa/oa.fcgi?id=10026098&#xA;&amp;lt;OA&amp;gt;&amp;lt;responseDate&amp;gt;2023-07-17 01:39:13&amp;lt;/responseDate&amp;gt;&amp;lt;request id=&amp;quot;PMC10026098&amp;quot;&amp;gt;https://www.ncbi.nlm.nih.gov/pmc/utils/oa/oa.fcgi?id=10026098&amp;lt;/request&amp;gt;&amp;lt;records returned-count=&amp;quot;2&amp;quot; total-count=&amp;quot;2&amp;quot;&amp;gt;&amp;lt;record id=&amp;quot;PMC10026098&amp;quot; citation=&amp;quot;Ther Adv Med Oncol. 2023 Mar 18; 15:17588359231161409&amp;quot; license=&amp;quot;CC BY-NC&amp;quot; retracted=&amp;quot;no&amp;quot;&amp;gt;&amp;lt;link format=&amp;quot;tgz&amp;quot; updated=&amp;quot;2023-03-23 12:10:30&amp;quot; href=&amp;quot;ftp://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_package/83/eb/PMC10026098.tar.gz&amp;quot; /&amp;gt;&amp;lt;link format=&amp;quot;pdf&amp;quot; updated=&amp;quot;2023-03-20 09:34:33&amp;quot; href=&amp;quot;ftp://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/83/eb/10.1177_17588359231161409.PMC10026098.pdf&amp;quot; /&amp;gt;&amp;lt;/record&amp;gt;&amp;lt;/records&amp;gt;&amp;lt;/OA&amp;gt;&#xA;&#xA;# get the pdf link from the API (ftp storage)&#xA;curl https://www.ncbi.nlm.nih.gov/pmc/utils/oa/oa.fcgi?id=10026098 | sed -n 's/.*\&amp;quot;\(ftp.*.pdf\)\&amp;quot;.*/\1/p'&#xA;ftp://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_pdf/83/eb/10.1177_17588359231161409.PMC10026098.pdf&#xA;&#xA;# download the pdf&#xA;pdf=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$(curl https://www.ncbi.nlm.nih.gov/pmc/utils/oa/oa.fcgi?id=10026098 | sed -n 's/.*\&quot;\(ftp.*.pdf\)\&quot;.*/\1/p')&quot;&#xA;curl -O &quot;$&lt;/span&gt;f&amp;quot;&#xA;&#xA;# download many pdfs from a list of ids&#xA;cat list_of_ids.txt&#xA;10026098&#xA;9341183&#xA;&#xA;while read -r id_from_list&#xA;do&#xA;    pdf=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$(curl https://www.ncbi.nlm.nih.gov/pmc/utils/oa/oa.fcgi?id=$&lt;/span&gt;id_from_list | sed -n 's/.*\&amp;quot;\(ftp.*.pdf\)\&amp;quot;.*/\1/p')&amp;quot;&#xA;    curl -O &amp;quot;$pdf&amp;quot;&#xA;done &amp;lt; list_of_ids.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This approach is preferable to selenium scraping for many reasons, and if you have any problems the NCBI provides documentation (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/tools/ftp/#indart&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/tools/ftp/#indart&lt;/a&gt;) and offers support (you can email them at pubmedcentral@ncbi.nlm.nih.gov).&lt;/p&gt;&#xA;" OwnerUserId="9098" LastActivityDate="2023-07-17T06:09:16.797" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21328" PostTypeId="2" ParentId="9225" CreationDate="2023-07-17T11:25:26.033" Score="2" Body="&lt;p&gt;As of 2023 (or Seurat &amp;gt;4 or &amp;gt;3?), simply use &lt;code&gt;merge&lt;/code&gt; and write&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;combined.obj &amp;lt;- merge(x = ls.Seurat[[1]], y = ls.Seurat[2:l(ls.Seurat)], add.cell.ids = vec_sample_IDs)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Since &lt;code&gt;y&lt;/code&gt; can take a list.&lt;/p&gt;&#xA;" OwnerUserId="1365" LastActivityDate="2023-07-17T11:25:26.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21329" PostTypeId="1" AcceptedAnswerId="21330" CreationDate="2023-07-17T19:56:44.370" Score="1" ViewCount="82" Body="&lt;p&gt;I've a pipeline with many processes.&lt;br /&gt;&#xA;One of the steps is to run lancet tool. Lancet tool runs per chromosome (1-22, x,y). The input in the lancet step is output from the applybqsr (gatk).&lt;/p&gt;&#xA;&lt;p&gt;There are tumor and normal bam generated along with the index. human genome FASTA has a index. When I run lancet through the main.nf I run into an error&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;ERROR: index not found for BAM file&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I've a config file as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params {&#xA;&#xA;        hg38genome=&amp;quot;/sc/arion/reference_genome/human/hg38/Homo_sapiens_assembly38.fasta.gz&amp;quot;&#xA;        outdir = '/sc/arion/output_pipeline_nextflow/test_tiny_datasets/'&#xA;        dbsnp=&amp;quot;/sc/arion/projects/Daphni2_Pipeline_resources/resourceDBsnp146/dbsnp_146.hg38.vcf.gz&amp;quot;&#xA;        target=&amp;quot;/sc/arion/projects/Twist_Exome_Target_hg38.padded.interval_list.bed/Twist_Exome_Target_hg38.padded.interval_list.bed&amp;quot;&#xA;&#xA;}&#xA;        withName: applybqsr {&#xA;        publishDir = [&#xA;                [  &#xA;                path: &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${params.outdir}/applybqsr/normal/&quot;,&#xA;                mode: 'copy',&#xA;                pattern:'*normal*'&#xA;                ],&#xA;                [&#xA;                path: &quot;$&lt;/span&gt;{params.outdir}/applybqsr/tumor/&amp;quot;,&#xA;                mode: 'copy',&#xA;                pattern:'*tumor*'&#xA;                ]       &#xA;        ]&#xA;        }&#xA;        //applybqsr ends        &#xA;&#xA;        &#xA;        withName: lancet {&#xA;        publishDir = [  &#xA;                path: &amp;quot;${params.outdir}/lancet/&amp;quot;,&#xA;                mode: 'copy'&#xA;        ]&#xA;        }&#xA;        //lancet ends&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have applybqsr as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process applybqsr{&#xA;&#xA;        maxForks 3&#xA;        debug true&#xA;&#xA;        memory '6 GB'&#xA;&#xA;        input:&#xA;        tuple val(sample_id_tumor), path(markbam_tumor)&#xA;        tuple val(sample_id_tumor), path(recaltable_tumor)&#xA;        tuple val(sample_id_normal), path(markbam_normal)&#xA;        tuple val(sample_id_normal), path(recaltable_normal)&#xA;&#xA;        output:&#xA;        tuple val(sample_id_tumor), path(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_id_tumor}.sorted.markdup.recal.bam&quot;), emit: recal_bam_tumor&#xA;        tuple val(sample_id_tumor), path(&quot;$&lt;/span&gt;{sample_id_tumor}.sorted.markdup.recal.bai&amp;quot;),  emit: recal_bai_tumor&#xA;        tuple val(sample_id_normal), path(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_id_normal}.sorted.markdup.recal.bam&quot;), emit: recal_bam_normal&#xA;        tuple val(sample_id_normal), path(&quot;$&lt;/span&gt;{sample_id_normal}.sorted.markdup.recal.bai&amp;quot;),  emit: recal_bai_normal&#xA;        &#xA;        script:&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;        ml gatk/4.1.3.0 samtools/1.9&#xA;        samtools index &lt;span class=&quot;math-container&quot;&gt;${markbam_tumor}&#xA;        samtools index $&lt;/span&gt;{markbam_normal}&#xA;&#xA;        gatk ApplyBQSR \\&#xA;        -R &lt;span class=&quot;math-container&quot;&gt;$params.hg38genome \\&#xA;        -I $&lt;/span&gt;{markbam_tumor} \\&#xA;        --bqsr-recal-file &lt;span class=&quot;math-container&quot;&gt;${recaltable_tumor} \\&#xA;        -O $&lt;/span&gt;{sample_id_tumor}.sorted.markdup.recal.bam \\&#xA;        -L &lt;span class=&quot;math-container&quot;&gt;$params.target \\&#xA;        --create-output-bam-index true \\&#xA;        --java-options -Xmx$&lt;/span&gt;{task.memory.toGiga()}g&#xA;&#xA;        gatk ApplyBQSR \\&#xA;        -R &lt;span class=&quot;math-container&quot;&gt;$params.hg38genome \\&#xA;        -I $&lt;/span&gt;{markbam_normal} \\&#xA;        --bqsr-recal-file &lt;span class=&quot;math-container&quot;&gt;${recaltable_normal} \\&#xA;        -O $&lt;/span&gt;{sample_id_normal}.sorted.markdup.recal.bam \\&#xA;        -L &lt;span class=&quot;math-container&quot;&gt;$params.target \\&#xA;        --create-output-bam-index true \\&#xA;        --java-options -Xmx$&lt;/span&gt;{task.memory.toGiga()}g&#xA;&#xA;&#xA;        &amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;}&#xA;workflow.onComplete { &#xA;        log.info ( workflow.success ? &amp;quot;Done applyBQSR!&amp;quot; : &amp;quot;Oops .. something went wrong in applyBQSR&amp;quot; )&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've lancet.nf as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process lancet {&#xA;        maxForks 3&#xA;        debug true&#xA;&#xA;        input:&#xA;        tuple val(sample_id_tumor),path(bqsrbam_tumor)&#xA;        tuple val(sample_id_normal),path(bqsrbam_normal)&#xA;        val chromosome&#xA;&#xA;        output:&#xA;        path &amp;quot;CHR&lt;span class=&quot;math-container&quot;&gt;${chromosome}_$&lt;/span&gt;{sample_name}.lancet.vcf&amp;quot;&#xA;&#xA;        script:&#xA;&#xA;        sample_name=sample_id_normal.split('_normal')[0]&#xA;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;ml lancet&#xA;&#xA;lancet --tumor &lt;span class=&quot;math-container&quot;&gt;${bqsrbam_tumor} --normal $&lt;/span&gt;{bqsrbam_normal} --ref &lt;span class=&quot;math-container&quot;&gt;$params.hg38genome --reg &quot;chr&quot;$&lt;/span&gt;chromosome --num-threads 8 &amp;gt; &amp;quot;CHR&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${chromosome}&quot;_$&lt;/span&gt;{sample_name}.lancet.vcf&amp;quot; &#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;}&#xA;&#xA;workflow.onComplete {&#xA;          log.info(workflow.success ? &amp;quot;Done lancet!&amp;quot; : &amp;quot;Oops .. something went wrong in lancet&amp;quot; )&#xA; }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have main.nf as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params.outdir_index=&amp;quot;/sc/arion/output_pipeline_nextflow/test_tiny_datasets/bwa_index&amp;quot;&#xA;params.raw_files_normal=&amp;quot;/sc/arion/tiny/all_tumors_normals/*_normal_*{R1,R2}.fastq&amp;quot;&#xA;params.raw_files_tumor=&amp;quot;/sc/arion/tiny/all_tumors_normals/*_tumor_*{R1,R2}.fastq&amp;quot;&#xA;&#xA;include { FASTP} from './fastp_process.nf'&#xA;include {bwa_index} from './index_process.nf'&#xA;&#xA;include { align_bwa_mem} from './bwamem_process_already_index.nf'&#xA;include { gatk_markduplicates} from './gatk_markduplicates_process.nf'&#xA;include {setupnmdtags} from './setupnmdtags_process.nf'&#xA;include { recalibrate_bam } from './recalibratebam_process.nf'&#xA;include { applybqsr } from './applybqsr_process.nf'&#xA;include { mutect2 } from './mutect2_process.nf'&#xA;include { lancet } from './lancet_process.nf'&#xA;&#xA;workflow {&#xA;&#xA;        normal_read_pairs=Channel.fromFilePairs(params.raw_files_normal)&#xA;        tumor_read_pairs=Channel.fromFilePairs(params.raw_files_tumor)&#xA;        //FASTP   align_bwa_mem        gatk_markduplicates         setupnmdtags        recalibrate_bam &#xA;        &#xA;        applybqsr(gatk_markduplicates.out.sorted_bam_mark_duplicates_tumor,recalibrate_bam.out.recalib_table_tumor,gatk_markduplicates.out.sorted_bam_mark_duplicates_normal,recalibrate_bam.out.recalib_table_normal)&#xA;&#xA;        mutect2(applybqsr.out.recal_bam_tumor,applybqsr.out.recal_bam_normal)&#xA;        &#xA;        chr=Channel.of( 1..22, 'X', 'Y' )&#xA;        lancet(applybqsr.out.recal_bam_tumor,applybqsr.out.recal_bam_normal,chr)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Lancet runs fine if I run it hard coded into a process code block with same input (tumor-normal bam) and human genome FASTA. How do I enable index files to be staged into the lancet process generated one step before?&lt;/p&gt;&#xA;" OwnerUserId="862" LastActivityDate="2023-07-18T07:08:48.877" Title="nextflow: unable to find index" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21330" PostTypeId="2" ParentId="21329" CreationDate="2023-07-18T07:08:48.877" Score="2" Body="&lt;p&gt;I am reluctant to go through this wall of code, but generally it is advisable to always keep bam files and its index in a tuple so the index is staged with its bam file.&lt;/p&gt;&#xA;&lt;p&gt;For example a process that creates the bam and now emits a tuple with bam and bai file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(...)&#xA;output:&#xA;tuple path(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${id}.bam&quot;), path(&quot;$&lt;/span&gt;{id}.bam.bai&amp;quot;), emit: tuple_bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and then a process that receives such a &lt;code&gt;tuple_bam&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(...)&#xA;input:&#xA;tuple path(bam), path(bai)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The first entry is the bam, the second the index.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-07-18T07:08:48.877" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21331" PostTypeId="1" AcceptedAnswerId="21450" CreationDate="2023-07-18T14:21:53.060" Score="2" ViewCount="81" Body="&lt;p&gt;I would like to plot using estimateCellCounts as I do not know my cell counts for the blood samples I am using however, it seem that I cannot get estimateCellCounts to work with EPIC arrays.&lt;/p&gt;&#xA;&lt;p&gt;Detailing Attempts:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Attempt 1&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I first attempted it with EPIC arrays but it wouldn't take it.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;estimateCellCounts(RGset, &#xA;                   compositeCellType = &amp;quot;Blood&amp;quot;,&#xA;                   processMethod = &amp;quot;auto&amp;quot;, &#xA;                   probeSelect = &amp;quot;auto&amp;quot;,&#xA;                   cellTypes = c(&amp;quot;Bcell&amp;quot;, &#xA;                                 &amp;quot;CD4T&amp;quot;, &#xA;                                 &amp;quot;CD8T&amp;quot;, &#xA;                                 &amp;quot;Eos&amp;quot;, &#xA;                                 &amp;quot;Gran&amp;quot;, &#xA;                                 &amp;quot;Mono&amp;quot;, &#xA;                                 &amp;quot;Neu&amp;quot;, &#xA;                                 &amp;quot;NK&amp;quot;&#xA;                                ),&#xA;                   referencePlatform = &amp;quot;IlluminaHumanMethylationEPIC&amp;quot;,&#xA;                   returnAll = TRUE,&#xA;                   meanPlot = TRUE,&#xA;                   verbose = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the associated error:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;Loading required package: FlowSorted.Blood.EPIC&#xA;Loading required package: ExperimentHub&#xA;Loading required package: AnnotationHub&#xA;Loading required package: BiocFileCache&#xA;Loading required package: dbplyr&#xA;&#xA;Attaching package: 'AnnotationHub'&#xA;&#xA;The following object is masked from 'package:Biobase':&#xA;    cache&#xA;&#xA;Warning message in data(list = referencePkg):&#xA;&amp;quot;data set 'FlowSorted.Blood.EPIC' not found&amp;quot;&#xA;Error in get(referencePkg): object 'FlowSorted.Blood.EPIC' not found&#xA;Traceback:&#xA;1. estimateCellCounts(RGset, compositeCellType = &amp;quot;Blood&amp;quot;, processMethod = &amp;quot;auto&amp;quot;, &#xA; .     probeSelect = &amp;quot;auto&amp;quot;, cellTypes = c(&amp;quot;Bcell&amp;quot;, &amp;quot;CD4T&amp;quot;, &amp;quot;CD8T&amp;quot;, &#xA; .         &amp;quot;Eos&amp;quot;, &amp;quot;Gran&amp;quot;, &amp;quot;Mono&amp;quot;, &amp;quot;Neu&amp;quot;, &amp;quot;NK&amp;quot;), referencePlatform = &amp;quot;IlluminaHumanMethylationEPIC&amp;quot;, &#xA; .     returnAll = TRUE, meanPlot = TRUE, verbose = TRUE)&#xA;2. get(referencePkg)&#xA;3. get(referencePkg)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Adding the &lt;code&gt;library(&amp;quot;FlowSorted.Blood.EPIC&amp;quot;)&lt;/code&gt; did not stop the error &lt;code&gt;&amp;quot;data set 'FlowSorted.Blood.EPIC' not found&amp;quot;&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Attempt 2&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Then, I looked up the error I got first on &lt;a href=&quot;https://github.com/hansenlab/minfi/issues/185&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github&lt;/a&gt; and found that converting to 450k was recommended. So I used the &lt;code&gt;convertArray&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Then it threw up an error saying  &lt;code&gt;&amp;quot;FlowSorted.Blood.450k&amp;quot;&lt;/code&gt; was not installed.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Attempt 3&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;So I installed and loaded the library and re-ran to get the code below and the error below. However, this does not change the issue that it does not seem to run. The code I used and the error is below - any suggestions are appreciated:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(&amp;quot;FlowSorted.Blood.450k&amp;quot;)&#xA;&#xA;RGset_450k = convertArray(RGset, &#xA;                          outType = c(&amp;quot;IlluminaHumanMethylation450k&amp;quot;),&#xA;                          verbose = TRUE&#xA;                         ) &#xA;&#xA;estimateCellCounts(RGset_450k, &#xA;                   compositeCellType = &amp;quot;Blood&amp;quot;,&#xA;                   processMethod = &amp;quot;auto&amp;quot;, &#xA;                   probeSelect = &amp;quot;auto&amp;quot;,&#xA;                   cellTypes = c(&amp;quot;Bcell&amp;quot;, &#xA;                                 &amp;quot;CD4T&amp;quot;, &#xA;                                 &amp;quot;CD8T&amp;quot;, &#xA;                                 &amp;quot;Eos&amp;quot;, &#xA;                                 &amp;quot;Gran&amp;quot;, &#xA;                                 &amp;quot;Mono&amp;quot;, &#xA;                                 &amp;quot;Neu&amp;quot;, &#xA;                                 &amp;quot;NK&amp;quot;&#xA;                                ),&#xA;                   referencePlatform = &amp;quot;IlluminaHumanMethylation450k&amp;quot;,&#xA;                   returnAll = TRUE,&#xA;                   meanPlot = TRUE,&#xA;                   verbose = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Error&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;Error in p[trainingProbes, ]: subscript out of bounds&#xA;Traceback:&#xA;&#xA;1. estimateCellCounts(RGset_450k, compositeCellType = &amp;quot;Blood&amp;quot;, processMethod = &amp;quot;auto&amp;quot;, &#xA; .     probeSelect = &amp;quot;auto&amp;quot;, cellTypes = c(&amp;quot;Bcell&amp;quot;, &amp;quot;CD4T&amp;quot;, &amp;quot;CD8T&amp;quot;, &#xA; .         &amp;quot;Eos&amp;quot;, &amp;quot;Gran&amp;quot;, &amp;quot;Mono&amp;quot;, &amp;quot;Neu&amp;quot;, &amp;quot;NK&amp;quot;), referencePlatform = &amp;quot;IlluminaHumanMethylation450k&amp;quot;, &#xA; .     returnAll = TRUE, meanPlot = TRUE, verbose = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How would I make it get around this error as it does not output any results? Any suggestions/solutions are appreciated.&lt;/p&gt;&#xA;" OwnerUserId="17244" LastEditorUserId="650" LastEditDate="2023-07-20T16:12:55.593" LastActivityDate="2023-08-18T13:15:42.527" Title="Minfi and estimateCellCounts Error: in p[trainingProbes, ]: subscript out of bounds" Tags="|r|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21332" PostTypeId="1" AcceptedAnswerId="21336" CreationDate="2023-07-18T14:27:10.710" Score="0" ViewCount="49" Body="&lt;p&gt;I am very new to bioinformatics but have some background in coding with R. I have been asked to find a data set and complete differential expression using the Limma package in R.&#xA;I have tried a few data sets but haven't been able to get any code to work.&lt;/p&gt;&#xA;&lt;p&gt;What kind of data do I need to have in order for Differential Expression to work with Limma R? I have tried load and read.table, is there a read function that I am missing?&lt;/p&gt;&#xA;&lt;p&gt;I was advised to look on Gene Expression Omnibus to find published data sets to work on but again I am struggling to find any that will work.&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="17820" LastActivityDate="2023-07-19T08:01:10.443" Title="What kind of data do I need to be able to complete Differential Expression using Limma" Tags="|r|differential-expression|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21333" PostTypeId="1" CreationDate="2023-07-18T15:29:59.627" Score="1" ViewCount="41" Body="&lt;p&gt;I am trying to write the right command for my association testing using PLINK 1.9 but I could not find a clear answer so far and not sure how to wrtie it correct.&lt;/p&gt;&#xA;&lt;p&gt;I have genotype variants called and in PLINK bed formatted files under geno_data folder.&lt;/p&gt;&#xA;&lt;p&gt;I have phenotype.txt file. First column as &amp;quot;Sample_ID&amp;quot;, second column as &amp;quot;Treatment&amp;quot; (1 for Placebo and 2 for CompoundX) and third column has the Response variable as &amp;quot;ResX&amp;quot;, which has integer number between 0 to 9.&lt;/p&gt;&#xA;&lt;p&gt;I also have PCs file as cov.txt. The first and second columns are the same and includes Sample_IDs. The column names are &amp;quot;FID&amp;quot; and &amp;quot;IID&amp;quot;. The third, fouth and fifth columns have PCs,with the column names &amp;quot;PC1&amp;quot;, &amp;quot;PC2&amp;quot; and &amp;quot;PC3&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;The Plink documentation did not help me as I am new in this. Therefore I need some examples command to clarify for various models as described below.&lt;/p&gt;&#xA;&lt;p&gt;I need the correct PLINK (1.9) command (if not possible PLINK2 is also OK) for the following linear models:&lt;/p&gt;&#xA;&lt;p&gt;Model 1:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ResX ~ Variant * Treatment + PC1 + PC2 + PC3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Second command I need is for this model:&lt;/p&gt;&#xA;&lt;p&gt;Model 2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ResX ~ Variant + Treatment + PC1 + PC2 + PC3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Model 3:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ResX ~ Variant * Treatment&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note I had asked this question on Biostar 11 days ago but did not receive any answer or comment yet.&#xA;Thank you&lt;/p&gt;&#xA;" OwnerUserId="2536" LastEditorUserId="3694" LastEditDate="2023-07-18T19:29:25.570" LastActivityDate="2023-07-18T19:29:25.570" Title="PLINK linear association testing command with an interaction and 2 covariates term" Tags="|gwas|plink|linear-regression|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21335" PostTypeId="2" ParentId="21268" CreationDate="2023-07-18T21:04:43.447" Score="1" Body="&lt;p&gt;You might want to look at &lt;a href=&quot;http://revigo.irb.hr&quot; rel=&quot;nofollow noreferrer&quot;&gt;Revigo&lt;/a&gt; to simplify multiple GO terms into their parent terms. I've used &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/rrvgo.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;rrvgo&lt;/a&gt; extensively to simplify enrichment results - this is the R implementation of Revigo.&lt;/p&gt;&#xA;&lt;p&gt;It's not clear how exactly you are obtaining these GO assignments. If these are the significant hits for each feature, I'd take the union of all significant GO terms (i.e. everything in your second column combined) and pass it to rrvgo. Then, you can select the parent term associated with each row. I don't guarantee each row will be collapsed to a single term, but it should make whatever you are doing downstream much more tractable.&lt;/p&gt;&#xA;" OwnerUserId="9558" LastEditorUserId="9558" LastEditDate="2023-07-18T21:07:41.477" LastActivityDate="2023-07-18T21:07:41.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21336" PostTypeId="2" ParentId="21332" CreationDate="2023-07-19T07:30:58.317" Score="0" Body="&lt;p&gt;Limma is relatively generic in what it accepts. Typically it is either data on log2 scale that are either already normalized or need some of the inbuilt normalization methods (such as quantile normalization), or in case of RNA-seq you can provide raw counts which then can run through the limma-voom pipeline.&lt;/p&gt;&#xA;&lt;p&gt;The format is always a matrix of counts/values, with genes/observations in rows and samples/specimen in columns.&lt;/p&gt;&#xA;&lt;p&gt;For practice, you could follow the RNA-seq/limma-voom section (chapter 15) of the &lt;a href=&quot;http://www.bioconductor.org/packages/current/bioc/vignettes/limma/inst/doc/usersguide.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;limma manual&lt;/a&gt; which has a RNA-seq case study available at &lt;a href=&quot;https://subread.sourceforge.net/RNAseqCaseStudy.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://subread.sourceforge.net/RNAseqCaseStudy.html&lt;/a&gt; with example data and code.&lt;/p&gt;&#xA;&lt;p&gt;GEO has a lot of published data, often microarray gene expression values or RNA-seq, but lets discuss this in a separate question if you need it to keep this here focused on limma.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastEditorUserId="16676" LastEditDate="2023-07-19T08:01:10.443" LastActivityDate="2023-07-19T08:01:10.443" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21337" PostTypeId="1" CreationDate="2023-07-19T14:32:57.290" Score="1" ViewCount="73" Body="&lt;p&gt;I received paired end amplicon sequence from a LAB with different lengths for R1 (320) and R2(280). Should I trust this lab to sequence other samples?&lt;/p&gt;&#xA;&lt;p&gt;Also, I had to do a trimming over the R2 at 220(because Phred score was below 30) in the qiime2/DADA2 pipeline. Do you think trimming almost 60 bp from R2 won't result in false positives? I'm looking for precise results from my analysis. The lab claims this will give quality results as this is a specific experimental design of their lab. Should I trust the results I get from this data, also the lab? I have more samples to get sequenced.&lt;/p&gt;&#xA;" OwnerUserId="17845" LastEditorUserId="650" LastEditDate="2023-07-19T15:35:17.260" LastActivityDate="2023-07-19T15:35:17.260" Title="Can files with different R1 and R2 lengths be trusted?" Tags="|sequence-analysis|metagenome|illumina|trimming|microbiome|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="21338" PostTypeId="1" AcceptedAnswerId="21339" CreationDate="2023-07-19T21:42:51.960" Score="2" ViewCount="91" Body="&lt;p&gt;I'd like to merge files from different lanes for forward and reverse reads. For e.g., if a sample A has three lanes in forward, and three reverse, ultimately one reverse and one forward is to be generated.  I have normal and tumor for each sample. N as normal and T as tumor.&lt;br /&gt;&#xA;The lanes files are for e.g. as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MM-265-DNA-N-01-01_L003_R1_001.fastq.gz&#xA;MM-265-DNA-N-01-01_L003_R2_001.fastq.gz&#xA;MM-265-DNA-N-01-01_L004_R1_001.fastq.gz&#xA;MM-265-DNA-N-01-01_L004_R2_001.fastq.gz&#xA;&#xA;MM-265-DNA-T-01-01_L003_R1_001.fastq.gz&#xA;MM-265-DNA-T-01-01_L003_R2_001.fastq.gz&#xA;MM-265-DNA-T-01-01_L004_R1_001.fastq.gz&#xA;MM-265-DNA-T-01-01_L004_R2_001.fastq.gz&#xA;&#xA;MM-465-DNA-N-01-01_L003_R1_001.fastq.gz&#xA;MM-465-DNA-N-01-01_L003_R2_001.fastq.gz&#xA;MM-465-DNA-N-01-01_L004_R1_001.fastq.gz&#xA;MM-465-DNA-N-01-01_L004_R2_001.fastq.gz&#xA;&#xA;MM-465-DNA-T-01-01_L003_R1_001.fastq.gz&#xA;MM-465-DNA-T-01-01_L003_R2_001.fastq.gz&#xA;MM-465-DNA-T-01-01_L004_R1_001.fastq.gz&#xA;MM-465-DNA-T-01-01_L004_R2_001.fastq.gz  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The final output needs to be as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MM-465_normal_R1.fastq MM-465_normal_R2.fastq&#xA;MM-465_tumor_R1.fastq MM-465_tumor_R2.fastq&#xA;MM-265_normal_R1.fastq MM-265_normal_R2.fastq&#xA;MM-265_tumor_R1.fastq MM-265_tumor_R2.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've code below broken in three parts:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;regular expression&lt;/li&gt;&#xA;&lt;li&gt;process that iterates over files&lt;/li&gt;&#xA;&lt;li&gt;workflow that calls the above two code blocks&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;regex:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def get_sample_id( file ) {&#xA;def matcher = (file =~ /(MM-\d+)-DNA-N-.+/)&#xA;    if (matcher) {&#xA;        return matcher[0][1]&#xA;    } else {&#xA;        return ''&#xA;    }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;process:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process file_print {&#xA;&#xA;debug true&#xA;&#xA;input:&#xA;&#xA;tuple val(sample_id), path(datasetFile)&#xA;val(sample_status) &#xA;val(read_value)&#xA;&#xA;output:&#xA;publishDir '/arion/learn_new_concepts/', mode:'copy'&#xA;&#xA;tuple val (sample_id), path(&amp;quot;${output_file_name}&amp;quot;)&#xA;&#xA;script:&#xA;&#xA;output_file_name = &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_id}_$&lt;/span&gt;{sample_status}_${read_value}.fastq&amp;quot;&#xA;&#xA;for (g in datasetFile){&#xA;println(g)&#xA;&#xA;&amp;quot;zcat &lt;span class=&quot;math-container&quot;&gt;${g} &amp;gt;&amp;gt; $&lt;/span&gt;{output_file_name} &amp;quot;&#xA;&#xA;}&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Workflow:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow {&#xA;&#xA;normal_reads=Channel.fromFilePairs(&amp;quot;/sc/arion/MM-*-N-*_L00*_R{1,2}_*.fastq.gz&amp;quot;)&#xA;//MM-2658-DNA-N-01-01_L004_R2_001.fastq.gz  //&#xA;&#xA;normal_forward_reads=normal_reads&#xA;        .map{ file -&amp;gt; return tuple( get_sample_id(file), file[1][0] )}.groupTuple(sort:true ) &#xA;&#xA;file_print(normal_forward_reads,&amp;quot;normal&amp;quot;,&amp;quot;R1&amp;quot;).view() /// similar for reverse [1][1], R2 .. for tumor pass reads, &amp;quot;tumor&amp;quot;, &amp;quot;R1&amp;quot;/R2&amp;quot;&#xA;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have not put code for reverse, but I get error as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Caused by:   Process `file_print (1)` terminated with an error exit status (127) Command executed:   null&#xA;&#xA;Command exit status:   127&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried many things such as &lt;code&gt;${ output_file_name }&lt;/code&gt;, &lt;code&gt;{ output_file_name }&lt;/code&gt;, &lt;code&gt;$output_file_name&lt;/code&gt; but in vain.&lt;/p&gt;&#xA;&lt;p&gt;I'd like to improve few things in code:&lt;/p&gt;&#xA;&lt;p&gt;a) The iteration/loop over file in &lt;code&gt;datasetFile&lt;/code&gt;   I think something like &lt;code&gt;each&lt;/code&gt; could help, but I need order to be maintained&lt;/p&gt;&#xA;&lt;p&gt;b) Order in which loop of &lt;code&gt;datasetFile&lt;/code&gt; is created. For instance, I'd like to keep lanes L003, L004, L005 in this order. Or, L001, L002, L003 order, I hope this is clear.&lt;/p&gt;&#xA;&lt;p&gt;I cannot move ahead or able to sort this by myself.&lt;/p&gt;&#xA;&lt;p&gt;I do not quite understand how and when to use &lt;code&gt;.map{}&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Edited: after output added tuple for val and &lt;code&gt;path&lt;/code&gt;. I still get same error.&lt;/p&gt;&#xA;" OwnerUserId="862" LastEditorUserId="650" LastEditDate="2023-07-20T13:34:17.187" LastActivityDate="2023-07-20T13:34:17.187" Title="nextflow: merge lane files from samples forward and reverse read" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21339" PostTypeId="2" ParentId="21338" CreationDate="2023-07-20T05:53:46.630" Score="2" Body="&lt;p&gt;I think you are on the right track using &lt;code&gt;groupTuple(sort: true)&lt;/code&gt;. Another way to get what you want would be to instead parse the &lt;code&gt;fromFilePairs&lt;/code&gt; grouping key. Unless you're looking to do more than just concatenate (and decompress) FASTQ files, I think all the process really requires is an output prefix (in addition to the FASTQ files of course), for example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params.reads = &amp;quot;/sc/arion/MM-*_R{1,2}_001.fastq.gz&amp;quot;&#xA;&#xA;&#xA;process concat_reads {&#xA;&#xA;    tag { prefix }&#xA;&#xA;    input:&#xA;    tuple val(prefix), path(reads, stageAs: 'reads/*')&#xA;&#xA;    output:&#xA;    tuple val(prefix), path(&amp;quot;${prefix}.fastq&amp;quot;)&#xA;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    zcat -f &lt;span class=&quot;math-container&quot;&gt;${reads} &amp;gt; &quot;$&lt;/span&gt;{prefix}.fastq&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;workflow {&#xA;&#xA;    Channel.fromFilePairs( params.reads ) \&#xA;        | branch { key, fastqs -&amp;gt;&#xA;&#xA;            def matcher = key =~ /^(MM-\w+)-DNA-(\w+)-.*_L(\d+).*$/&#xA;&#xA;            matches: matcher&#xA;                def (_, sample_id, sample_type, lane) = matcher[0].findAll()&#xA;&#xA;                def new_sample_id = &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_id}_$&lt;/span&gt;{sample_type}&amp;quot;&#xA;&#xA;                return tuple( new_sample_id, lane as int, *fastqs )&#xA;&#xA;            other: true&#xA;                return tuple( key, *fastqs )&#xA;        } \&#xA;        | set { samples }&#xA;&#xA;    samples.matches \&#xA;        | groupTuple( sort: true ) \&#xA;        | flatMap { sample_id, lanes, fq1s, fq2s -&amp;gt;&#xA;            [&#xA;                tuple( &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_id}_R1&quot;, fq1s ),&#xA;                tuple( &quot;$&lt;/span&gt;{sample_id}_R2&amp;quot;, fq2s ),&#xA;            ]&#xA;        } \&#xA;        | concat_reads \&#xA;        | branch { prefix, fastq -&amp;gt;&#xA;&#xA;            def idx = prefix.lastIndexOf('_')&#xA;&#xA;            def sample_id = prefix.substring(0, idx)&#xA;            def suffix = prefix.substring(idx + 1)&#xA;&#xA;            read1: suffix == 'R1'&#xA;                return tuple( sample_id, fastq )&#xA;            read2: suffix == 'R2'&#xA;                return tuple( sample_id, fastq )&#xA;        } \&#xA;        | set { result }&#xA;&#xA;    result.read1 \&#xA;        | join( result.read2 ) \&#xA;        | map { sample_id, fastq1, fastq2 -&amp;gt;&#xA;&#xA;            tuple( sample_id, [ fastq1, fastq2 ] )&#xA;        } \&#xA;        | view()&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ nextflow run main.nf&#xA;N E X T F L O W  ~  version 23.04.1&#xA;Launching `main.nf` [jolly_noether] DSL2 - revision: 4a81f7a875&#xA;executor &amp;gt;  local (8)&#xA;[b8/a0b534] process &amp;gt; concat_reads (MM-265_T_R2) [100%] 8 of 8 ✔&#xA;[MM-465_T, [/path/to/work/9c/8b6fd328c848379423a0aba098cb74/MM-465_T_R1.fastq, /path/to/work/34/913780b1da902b5035f3bf56aac038/MM-465_T_R2.fastq]]&#xA;[MM-465_N, [/path/to/work/69/595120a1ee0a42217d06855ca9d1bc/MM-465_N_R1.fastq, /path/to/work/8f/31c406c5180073de577502fdb88fbf/MM-465_N_R2.fastq]]&#xA;[MM-265_N, [/path/to/work/1b/9c17cefb5aa51b3ee69832dc65d915/MM-265_N_R1.fastq, /path/to/work/d0/aac9f46e4a7f7165693b2537ad5630/MM-265_N_R2.fastq]]&#xA;[MM-265_T, [/path/to/work/d5/6295ff109df3ccb829a6f0a97b6e45/MM-265_T_R1.fastq, /path/to/work/b8/a0b53499ca13a523d811ecf772449b/MM-265_T_R2.fastq]]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that unless the FASTQ files are small, I would try to avoid decompressing the FASTQ files if possible. Most bioinformatics tools that deal with large FASTQ files will likely have gzip support, meaning you can replace the &lt;code&gt;zcat -f&lt;/code&gt; with a &lt;code&gt;cat&lt;/code&gt; in the example above (and remember to add the &lt;code&gt;.gz&lt;/code&gt; extension to the output filename).&lt;/p&gt;&#xA;" OwnerUserId="10775" LastActivityDate="2023-07-20T05:53:46.630" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="21341" PostTypeId="1" CreationDate="2023-07-21T09:55:00.930" Score="2" ViewCount="59" Body="&lt;p&gt;I'm working for a project using GEO, in order to analyze some gene expression data. The SuperSeries I am referring to is GSE73304, but I found some problems working on the SubSeries GSE73303.&lt;/p&gt;&#xA;&lt;p&gt;I started my analysis Microarray expression dataset and everything was fine. Now I have to investigate the other dataset, the one target as Bisulfite-Seq. But when I try to see inside it, I find there are correctly 20 patients, but 0 features, so seems it is empty. I attach R output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(GEOquery)&#xA;gsemet &amp;lt;- getGEO(&amp;quot;GSE73303&amp;quot;)&#xA;&#xA;&amp;gt; gsemet&#xA;$GSE73303_series_matrix.txt.gz&#xA;ExpressionSet (storageMode: lockedEnvironment)&#xA;assayData: 0 features, 20 samples &#xA;element names: exprs &#xA;protocolData: none&#xA;phenoData&#xA;sampleNames: GSM1890515 GSM1890516 ... GSM1890534 (20 total)&#xA;varLabels: title geo_accession ... Sex:ch1 (44 total)&#xA;varMetadata: labelDescription&#xA;featureData: none&#xA;experimentData: use 'experimentData(object)'&#xA;pubMedIds: 27437034&#xA;28883895 &#xA;Annotation: GPL11154 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Could someone help me to find where I'm wrong? Or maybe, if the file is empty, pass me the right file?&lt;/p&gt;&#xA;&lt;p&gt;Thank you very much in advance&lt;/p&gt;&#xA;" OwnerUserId="17860" LastActivityDate="2023-07-21T12:55:49.517" Title="GEO Methylation dataset &quot;GSE73303&quot; shows 0 features" Tags="|r|bioconductor|genetics|geo|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21342" PostTypeId="2" ParentId="21341" CreationDate="2023-07-21T12:55:49.517" Score="1" Body="&lt;p&gt;It's NGS data, while GEOquery only returns data for arrays. You will either need to use the data they provude in the supplement of the GEO submission (if they do, it's not mandatory), or download and process fastq from scratch.&lt;/p&gt;&#xA;&lt;p&gt;It's not gene expression by the way, it's methylation data, bisulfite sequencing.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-07-21T12:55:49.517" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21343" PostTypeId="1" CreationDate="2023-07-21T13:27:49.370" Score="1" ViewCount="28" Body="&lt;p&gt;I'm using DAVID tool online, in order to perform functional annotation chart. I extracted the most relevant genes from a dataset (GSE73304), but have the problem that DAVID is not able to understand the most part.&lt;/p&gt;&#xA;&lt;p&gt;I tried to put my 250 most relevant genes, such as&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A_33_P3230339&#xA;A_21_P0014879&#xA;A_23_P373568&#xA;A_23_P206371&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;into David, using as identifier &amp;quot;AGILENT_ID&amp;quot;, as I found in the website. However, David is able to recognize just 89 of them (which are classified correctly as &amp;quot;homo sapiens&amp;quot;). I have the same problem if I try to do the gene_id conversion.&lt;/p&gt;&#xA;&lt;p&gt;This might be not a David's problem, since I tried also on R withoud good results.&lt;/p&gt;&#xA;&lt;p&gt;If someone knows the reason of this issue and how to solve it, it would be really appreciated&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="17860" LastActivityDate="2023-07-21T13:27:49.370" Title="David tool online not converting most genes" Tags="|r|bioconductor|sequence-annotation|identifiers|geoquery|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21344" PostTypeId="1" AcceptedAnswerId="21346" CreationDate="2023-07-21T13:54:17.653" Score="3" ViewCount="123" Body="&lt;p&gt;I am new to glmnet and other ML techniques, so apologise in advance if this sounds a trivial question. However, your guidance is very much appreciated.&lt;/p&gt;&#xA;&lt;p&gt;So, I have nearly 700 genes with their expression profile across 4 conditions. I want to identify signature genes for each of the 4 conditions. I am using glmnet but I am not getting anywhere near the results.&lt;/p&gt;&#xA;&lt;p&gt;so my data looks like this&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;           Subtype     HES4   FNDC10    PRKCZ PRKCZ.AS1&#xA;Sample1     NRT 5.452955 6.241387 8.546059  6.260245&#xA;Sample2     RT 5.125350 7.038719 8.900024  6.267325&#xA;Sample3     NRT 5.689560 5.903811 8.029434  4.213996&#xA;Sample4     NRP 5.834468 5.722589 8.061339  5.072274&#xA;Sample5     RP 5.401623 7.272111 9.108342  6.599629&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;purely in terms of numbers, the subtypes have following number of samples.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;NRT 52&#xA;RT 42&#xA;NRP 30&#xA;RP 7&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Q1. May be the groups are imbalanced and caution should be taken?&lt;/p&gt;&#xA;&lt;p&gt;I have applied glmnet as following&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;modellinglasso=read.table(&amp;quot;LRMatrix.txt&amp;quot;, header=T, row.names = 1, sep=&amp;quot;\t&amp;quot; )&#xA;modellinglasso$Subtype=as.factor(modellinglasso$Subtype)&#xA;modellinglasso$Subtype&#xA;#build a single pass (single fold) lasso-penalised model&#xA;lassoModel &amp;lt;- glmnet(&#xA;  x=data.matrix(modellinglasso[,-1]),&#xA;  y=modellinglasso$Subtype,&#xA;  standardize=TRUE,&#xA;  alpha=1.0,&#xA;  family=&amp;quot;multinomial&amp;quot;)&#xA;plot(lassoModel, xvar=&amp;quot;lambda&amp;quot;)&#xA;Warning message:&#xA;In lognet(xd, is.sparse, ix, jx, y, weights, offset, alpha, nobs,  :&#xA;  one multinomial or binomial class has fewer than 8  observations; dangerous ground&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/0lUP0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/0lUP0.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;#perform 10-fold cross validation&#xA;cv.lassoModel &amp;lt;- cv.glmnet(&#xA;  x=data.matrix(modellinglasso[,-1]),&#xA;  y=modellinglasso$Subtype,&#xA;  standardize=TRUE,&#xA;  alpha=1.0,&#xA;  nfolds=10,&#xA;  family=&amp;quot;multinomial&amp;quot;,&#xA;  parallel=TRUE)&#xA;# plot variable deviances vs. shrinkage parameter, λ (lambda)&#xA;plot(cv.lassoModel)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/E02wY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/E02wY.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;idealLambda &amp;lt;- cv.lassoModel&lt;span class=&quot;math-container&quot;&gt;$lambda.min&#xA;idealLambda1se &amp;lt;- cv.lassoModel$&lt;/span&gt;lambda.1se&#xA;print(idealLambda); print(idealLambda1se)&#xA;&#xA;[1] 0.1186125&#xA;[1] 0.1186125&#xA;&#xA;# derive coefficients for each gene&#xA;co &amp;lt;- coef(cv.lassoModel, s=idealLambda, exact=TRUE)&#xA;head(co)&#xA;&#xA;co.se &amp;lt;- coef(cv.lassoModel, s=idealLambda1se, exact=TRUE)&#xA;head(co.se)&#xA;&#xA;# identify predictors for each sub-type&#xA;rownames(co.se&lt;span class=&quot;math-container&quot;&gt;$PR)[which(co.se$&lt;/span&gt;NRT != 0)]&#xA;&#xA;[1] &amp;quot;(Intercept)&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output for&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;co.se$RT!=0&#xA;6 x 1 sparse Matrix of class &amp;quot;lgCMatrix&amp;quot;&#xA;            1&#xA;(Intercept) |&#xA;HES4        .&#xA;FNDC10      .&#xA;PRKCZ       .&#xA;PRKCZ.AS1   .&#xA;MMEL1       .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It doesn't matter which subtype I put, I get the same output &amp;quot;(Intercept)&amp;quot; instead of gene names.&lt;/p&gt;&#xA;&lt;p&gt;Any help to get the gene signatures for the subtypes is sincerely appreciated.&lt;/p&gt;&#xA;&lt;p&gt;Thank  you&lt;/p&gt;&#xA;" OwnerUserId="5062" LastEditorUserId="5062" LastEditDate="2023-07-22T09:45:02.010" LastActivityDate="2023-07-22T16:23:48.990" Title="Applying glmnet to identify predictors for subtypes" Tags="|r|rna-seq|machine-learning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21345" PostTypeId="1" CreationDate="2023-07-21T18:41:43.973" Score="0" ViewCount="39" Body="&lt;p&gt;I'm working with UK Biobank data. I made a cohort with the Cohort Browser and tried to extract the data with this code  &lt;code&gt;df = participant.retrieve_fields(names=field_names, filter_sql = cohort.sql, coding_values = &amp;quot;replace&amp;quot;, engine=dxdata.connect())&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This is part of the text of the error: &lt;code&gt;Py4JJavaError: An error occurred while calling 04100.showString. org.apache.spark. SparkException: Exception thrown in awaitResult:&lt;/code&gt;. The full error message is cut off.&lt;/p&gt;&#xA;&lt;p&gt;However, if I remove the filter - &lt;code&gt;df = participant.retrieve_fields(names=field_names, coding_values = &amp;quot;replace&amp;quot;, engine=dxdata.connect())&lt;/code&gt; – there is no issue.&lt;/p&gt;&#xA;&lt;p&gt;What kind of error is this and why would filtering for a cohort help?&lt;/p&gt;&#xA;" OwnerUserId="17862" LastEditorUserId="17862" LastEditDate="2023-07-21T19:39:56.163" LastActivityDate="2023-07-24T17:01:36.297" Title="Filtering cohorts in PySpark returns Py4JJavaError" Tags="|python|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21346" PostTypeId="2" ParentId="21344" CreationDate="2023-07-22T04:39:07.903" Score="2" Body="&lt;p&gt;Final answer. Keep in mind I work in Python so I'm trying to translate here.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The output of &lt;code&gt;coef(cv.lassoModel)&lt;/code&gt; ... those are your genes of interest thats your &lt;strong&gt;feature selection&lt;/strong&gt;. &lt;code&gt;head&lt;/code&gt; I assume gives you the top values ... I think thats right (I need to check its not &lt;code&gt;coef(lassoModel)&lt;/code&gt;) ... Basically thats where your results are.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This might be over simplistic but &lt;code&gt;co.se&lt;/code&gt; without the &lt;code&gt;head&lt;/code&gt; is your answer. Not every gene will carry a weight BTW. If the entirety of that is (700) &lt;code&gt;.&lt;/code&gt; ... try point 2 below and then point 4. I don't think that will happen (see &lt;strong&gt;Whats happened&lt;/strong&gt;)&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;I would redo this with &lt;code&gt;alpha=0&lt;/code&gt; (ridge regression) and &lt;code&gt;alpha=0.5&lt;/code&gt; elasticnet, but you don't need to&lt;/li&gt;&#xA;&lt;li&gt;Generate an accuracy score&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;    y_predicted &amp;lt;- predict(best_model, s = best_lambda, newx = x)&#xA;    sst &amp;lt;- sum((y - mean(y))^2)&#xA;    sse &amp;lt;- sum((y_predicted - y)^2)&#xA;    rsq &amp;lt;- 1 - sse/sst&#xA;    rsq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Hopefully this is large e.g. around 0.8 (in Python this automatic BTW). You could ignore that, but its what I'd do&lt;/p&gt;&#xA;&lt;ol start=&quot;4&quot;&gt;&#xA;&lt;li&gt;The intercept is simply the weight when your gene expression score is 0.&lt;/li&gt;&#xA;&lt;li&gt;Lasso regression handles sparse data - so no worries about &lt;code&gt;&amp;quot;.&amp;quot;&lt;/code&gt;. I don't whether that Python's version of &amp;quot;nan&amp;quot; (not a number), or if its zero.  ... @haci knows what this means and his answer is authoritative for the second part of your question.&lt;/li&gt;&#xA;&lt;li&gt;In Python transformations are common, but that doesn't seem an option here, so find providing your accuracy score is reasonable, personally I'm happy.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In Python what you want is feature selection&#xA;&lt;code&gt;print(pd.DataFrame({'Features': X_train.columns, 'Coefficients': lasso.coef_}))&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Whats happened?&lt;/strong&gt; The coefficients for &lt;code&gt;head(co.se)&lt;/code&gt; have been shrunk to zero, thats only the top 10? results, you've 700 weights. The full list &lt;em&gt;should&lt;/em&gt; (famous last words) contain a minority of really big coefficients ... those are really important and those are your answer. So by printing only the top few results ... not the whole lot the results are still there you've just not seen them. Thats it, basically you were almost there and you could happily settle on that as the final answer. Basically, alot of genes ain't doing much a few genes are doing all the work - thats the answer. Below I stress repeatedly I don't think anything is wrong here and I personally like the result.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Background&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Lasso regression&lt;/strong&gt;, it's a cousin of ridge regression and key parameter is lambda (who'd guessed). What your plots are doing is tuning the parameter and finding the associated weight individually or collectively. Fairly straight forward. So like whats the value of lambda for your data set ('answered' below)?  It's worth doing some reading on how lambda is calculated to understand the plot, if you're not sure how it works.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Imbalance&lt;/strong&gt;, well to be honest I'm not convinced there's anything wrong with your analysis. Okay there's a lack of stationarity for some of your weights ... but some are fine and gist is pretty clear. Yeah okay there's a problem with stabilisation of lambda - thats not necessarily imbalance.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Imbalance&lt;/strong&gt; is when it's 1 in 10 and over. You've got 1 in 6, 1 in 7, not huge. However, - &lt;em&gt;you might be right&lt;/em&gt; - and if you're worried simply repeat the analysis excluding &lt;code&gt;RP&lt;/code&gt; (value 7). Then check whether the parameterisation plot (lambda against weight) is more stable or more &amp;quot;homogeneous&amp;quot; (multinomial plot), i.e. the genes behaviour in a more uniform way. If that is does work - it gives you nicer results AND &lt;code&gt;RP&lt;/code&gt; is essential to the analysis thats a separate question about augmentation. That is complicated. For example, the multinomial plot with a 3 target categorical model might give a clear value for lambda.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Transformation&lt;/strong&gt; Here, I would suggest if you want to stabilise the multinomial plot (if it's possible) I would look at a transformation. The data you presented doesn't look like it needs a transformation, but we can't see the whole data. In Python there's a &lt;em&gt;max-min&lt;/em&gt; transformation - thats where I might start.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Overall good results&lt;/strong&gt; Personally, I'd say these were good results, except the weights ain't homogeneous for some genes (most are fine BTW), but the patterns clear. The &lt;em&gt;red&lt;/em&gt; (top), &lt;em&gt;blue&lt;/em&gt; (bottom) and cyan (negative y-axis) are real results showing that within the 700 genes &lt;strong&gt;3 genes&lt;/strong&gt; are having a massive impact. Sure you have positive and negative and thats how it works one gene is positively contributing to the classification, others are negatively contributing to it. Then you've the black and 'deep pink' lines which have stabilised around 5E4. &lt;strong&gt;From 700 genes down to 5 genes&lt;/strong&gt;. Whats wrong with that?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Whats the answer to the data set?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Its ...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;NRT&#xA;RT&#xA;NRP&#xA;RP&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;... is due to 'red', 'blue', 'cyan', 'black' and 'deep pink' genes. Thats it.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Frank assessment&lt;/strong&gt; To be perfectly honest, looks good. No-one is going to say &amp;quot;Wooaaa lambda ain't stable, destroy! ;-)&amp;quot;, they'll say &amp;quot;Wow thats cool and clear&amp;quot;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;How I'd do it..&lt;/strong&gt; I would look at basic ML robustness statistics and use this to assess the model rather than the parameterisation of lambda. Its horses for courses as they say. Upvotes.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; I use Python not R for ML, both work well however. &lt;code&gt;glmnet&lt;/code&gt; within ML is new to me BTW.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-07-22T16:23:48.990" LastActivityDate="2023-07-22T16:23:48.990" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21347" PostTypeId="1" CreationDate="2023-07-22T11:46:11.853" Score="1" ViewCount="82" Body="&lt;p&gt;I'm trying to analyise methylation data using GEO, in order to conduct a multi-omics analysis. The SubSeries I am referring to is GSE73303. Unfortunately, the code used to inspect it produces empty file, so I downloaded tar files, as I found on bioconductor forum (ref. &lt;a href=&quot;https://support.bioconductor.org/p/70963/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;I was able to download the tar files and extract thenm but afterwards I got a list of .bigwig files. I tried to read them on R, but actually I feel very lost and I can't understand how to trace back methylation data from these files.&lt;/p&gt;&#xA;&lt;p&gt;If someone can guide me into this process, it would be really appreciated&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance&lt;/p&gt;&#xA;" OwnerUserId="17860" LastActivityDate="2023-07-22T13:21:29.583" Title="Read methylation data from tar files on GEO (GSE73303)" Tags="|r|bioconductor|gene-expression|methylation|geo|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21348" PostTypeId="2" ParentId="21347" CreationDate="2023-07-22T13:21:29.583" Score="1" Body="&lt;p&gt;As I've already told you here &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/21341/geo-methylation-dataset-gse73303-shows-0-features&quot;&gt;GEO Methylation dataset &amp;quot;GSE73303&amp;quot; shows 0 features&lt;/a&gt; GEO/GEOquery does not provide ready-to-use processed NGS data. It can be that the authors provide processed data, but if not (apparently not the case in this submission) then you have to start from the fastq files. Please see the many tutorials online on downloading fastq and processing bisulfite data, or try to use existing pipelines (nf-core methylseq might be an option) or seek a collaboration.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-07-22T13:21:29.583" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21349" PostTypeId="1" CreationDate="2023-07-22T16:44:59.147" Score="0" ViewCount="30" Body="&lt;p&gt;I used commandline progressiveMauve to align two genome .gbk files. The alignment was successful. But when I try to visualize the final .xmfa file with Mauve, I get the following error.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Exception in thread &amp;quot;Thread-1&amp;quot; java.lang.RuntimeException: Similarity index is too large.&#xA;at org.gel.mauve.histogram.ZoomHistogram.allocateIndex(Unknown Source)&#xA;at org.gel.mauve.histogram.ZoomHistogram.&amp;lt;init&amp;gt;(Unknown Source)&#xA;at org.gel.mauve.SimilarityIndex.&amp;lt;init&amp;gt;(Unknown Source)&#xA;at org.gel.mauve.XmfaViewerModel.init(Unknown Source)&#xA;at org.gel.mauve.XmfaViewerModel.&amp;lt;init&amp;gt;(Unknown Source)&#xA;at org.gel.mauve.ModelBuilder.buildModel(Unknown Source)&#xA;at org.gel.mauve.gui.FrameLoader.loadFile(Unknown Source)&#xA;at org.gel.mauve.gui.FrameLoader.run(Unknown Source)&#xA;at java.lang.Thread.run(Thread.java:745)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I solve this?&#xA;Is there another way to visualize the xmfa file.&#xA;My aim is to visualize an inversion in one of the genomes.&lt;/p&gt;&#xA;" OwnerUserId="9089" LastActivityDate="2023-07-22T16:44:59.147" Title="Error while opening xmfa generated by progressiveMauve" Tags="|sequence-alignment|genome|java|mauve|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21350" PostTypeId="2" ParentId="19581" CreationDate="2023-07-22T22:41:27.810" Score="1" Body="&lt;p&gt;Ok, quite late.... my code stolen from &lt;a href=&quot;https://pymolwiki.org/index.php/Spectrum&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pymolwiki.org/index.php/Spectrum&lt;/a&gt; as per in my comment:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Not an expert here, but you could try to create a pdb with only c-alpha where B factor column has value according to bond lenght (proportional to it) and then color by b factor spectrum, I am sure there is some command about this : pymolwiki.org/index.php/Spectrum –&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;code using a fake peptide created with &lt;code&gt;cmd.fab('AFGVAL', 'peptide_1')&lt;/code&gt; :&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;peptide_1.pdb&lt;/code&gt; :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATOM      1  N   ALA     1      19.613   5.728  -6.281  1.00  0.00           N  &#xA;ATOM      2  CA  ALA     1      20.289   7.022  -6.281  1.00  0.00           C  &#xA;ATOM      3  C   ALA     1      21.789   6.848  -6.281  1.00  0.00           C  &#xA;ATOM      4  O   ALA     1      22.320   5.731  -6.292  1.00  0.00           O  &#xA;ATOM      5  CB  ALA     1      19.781   7.814  -5.064  1.00  0.00           C  &#xA;ATOM      6  H   ALA     1      20.159   4.796  -6.281  1.00  0.00           H  &#xA;ATOM      7  HA  ALA     1      20.021   7.561  -7.209  1.00  0.00           H  &#xA;ATOM      8 1HB  ALA     1      18.685   7.964  -5.100  1.00  0.00           H  &#xA;ATOM      9 2HB  ALA     1      20.005   7.300  -4.109  1.00  0.00           H  &#xA;ATOM     10 3HB  ALA     1      20.237   8.819  -5.007  1.00  0.00           H  &#xA;ATOM     11  N   PHE     2      22.540   7.897  -6.270  1.00  0.00           N  &#xA;ATOM     12  CA  PHE     2      23.990   7.730  -6.270  1.00  0.00           C  &#xA;ATOM     13  C   PHE     2      24.690   9.068  -6.255  1.00  0.00           C  &#xA;ATOM     14  O   PHE     2      24.050  10.156  -6.244  1.00  0.00           O  &#xA;ATOM     15  CB  PHE     2      24.437   6.942  -7.538  1.00  0.00           C  &#xA;ATOM     16  CG  PHE     2      23.824   5.546  -7.729  1.00  0.00           C  &#xA;ATOM     17  CD1 PHE     2      22.610   5.411  -8.413  1.00  0.00           C  &#xA;ATOM     18  CD2 PHE     2      24.430   4.416  -7.174  1.00  0.00           C  &#xA;ATOM     19  CE1 PHE     2      22.002   4.165  -8.523  1.00  0.00           C  &#xA;ATOM     20  CE2 PHE     2      23.826   3.167  -7.294  1.00  0.00           C  &#xA;ATOM     21  CZ  PHE     2      22.611   3.043  -7.966  1.00  0.00           C  &#xA;ATOM     22  H   PHE     2      22.130   8.821  -6.260  1.00  0.00           H  &#xA;ATOM     23  HA  PHE     2      24.287   7.177  -5.358  1.00  0.00           H  &#xA;ATOM     24 2HB  PHE     2      25.541   6.842  -7.539  1.00  0.00           H  &#xA;ATOM     25 3HB  PHE     2      24.233   7.549  -8.444  1.00  0.00           H  &#xA;ATOM     26 1HD  PHE     2      22.116   6.283  -8.820  1.00  0.00           H  &#xA;ATOM     27 2HD  PHE     2      25.362   4.509  -6.633  1.00  0.00           H  &#xA;ATOM     28 1HE  PHE     2      21.050   4.077  -9.026  1.00  0.00           H  &#xA;ATOM     29 2HE  PHE     2      24.295   2.298  -6.857  1.00  0.00           H  &#xA;ATOM     30  HZ  PHE     2      22.134   2.079  -8.049  1.00  0.00           H  &#xA;ATOM     31  N   GLY     3      25.980   9.099  -6.253  1.00  0.00           N  &#xA;ATOM     32  CA  GLY     3      26.656  10.393  -6.239  1.00  0.00           C  &#xA;ATOM     33  C   GLY     3      28.155  10.220  -6.239  1.00  0.00           C  &#xA;ATOM     34  O   GLY     3      28.685   9.105  -6.251  1.00  0.00           O  &#xA;ATOM     35  H   GLY     3      26.513   8.242  -6.261  1.00  0.00           H  &#xA;ATOM     36 3HA  GLY     3      26.365  10.908  -5.371  0.00  0.00           H  &#xA;ATOM     37  HA  GLY     3      26.378  10.953  -7.151  1.00  0.00           H  &#xA;ATOM     38  N   VAL     4      28.906  11.269  -6.227  1.00  0.00           N  &#xA;ATOM     39  CA  VAL     4      30.356  11.101  -6.228  1.00  0.00           C  &#xA;ATOM     40  C   VAL     4      31.055  12.440  -6.212  1.00  0.00           C  &#xA;ATOM     41  O   VAL     4      30.429  13.507  -6.201  1.00  0.00           O  &#xA;ATOM     42  CB  VAL     4      30.798  10.251  -7.484  1.00  0.00           C  &#xA;ATOM     43  CG1 VAL     4      32.326  10.136  -7.737  1.00  0.00           C  &#xA;ATOM     44  CG2 VAL     4      30.276   8.794  -7.470  1.00  0.00           C  &#xA;ATOM     45  H   VAL     4      28.497  12.192  -6.217  1.00  0.00           H  &#xA;ATOM     46  HA  VAL     4      30.654  10.571  -5.305  1.00  0.00           H  &#xA;ATOM     47  HB  VAL     4      30.366  10.744  -8.381  1.00  0.00           H  &#xA;ATOM     48 1HG1 VAL     4      32.862   9.673  -6.888  1.00  0.00           H  &#xA;ATOM     49 2HG1 VAL     4      32.559   9.537  -8.638  1.00  0.00           H  &#xA;ATOM     50 3HG1 VAL     4      32.799  11.120  -7.921  1.00  0.00           H  &#xA;ATOM     51 1HG2 VAL     4      29.177   8.752  -7.361  1.00  0.00           H  &#xA;ATOM     52 2HG2 VAL     4      30.510   8.264  -8.413  1.00  0.00           H  &#xA;ATOM     53 3HG2 VAL     4      30.710   8.201  -6.643  1.00  0.00           H  &#xA;ATOM     54  N   ALA     5      32.345  12.471  -6.211  1.00  0.00           N  &#xA;ATOM     55  CA  ALA     5      33.021  13.765  -6.196  1.00  0.00           C  &#xA;ATOM     56  C   ALA     5      34.521  13.592  -6.197  1.00  0.00           C  &#xA;ATOM     57  O   ALA     5      35.052  12.475  -6.219  1.00  0.00           O  &#xA;ATOM     58  CB  ALA     5      32.511  14.544  -4.971  1.00  0.00           C  &#xA;ATOM     59  H   ALA     5      32.879  11.614  -6.220  1.00  0.00           H  &#xA;ATOM     60  HA  ALA     5      32.753  14.314  -7.119  1.00  0.00           H  &#xA;ATOM     61 1HB  ALA     5      31.415  14.694  -5.007  1.00  0.00           H  &#xA;ATOM     62 2HB  ALA     5      32.734  14.020  -4.022  1.00  0.00           H  &#xA;ATOM     63 3HB  ALA     5      32.967  15.549  -4.903  1.00  0.00           H  &#xA;ATOM     64  N   LEU     6      35.271  14.641  -6.173  1.00  0.00           N  &#xA;ATOM     65  CA  LEU     6      36.722  14.473  -6.173  1.00  0.00           C  &#xA;ATOM     66  C   LEU     6      37.420  15.811  -6.143  1.00  0.00           C  &#xA;ATOM     67  O   LEU     6      36.792  16.881  -6.122  1.00  0.00           O  &#xA;ATOM     68  CB  LEU     6      37.174  13.638  -7.404  1.00  0.00           C  &#xA;ATOM     69  CG  LEU     6      36.580  12.214  -7.580  1.00  0.00           C  &#xA;ATOM     70  CD1 LEU     6      37.231  11.514  -8.782  1.00  0.00           C  &#xA;ATOM     71  CD2 LEU     6      36.743  11.343  -6.323  1.00  0.00           C  &#xA;ATOM     72  H   LEU     6      34.862  15.564  -6.154  1.00  0.00           H  &#xA;ATOM     73  HA  LEU     6      37.013  13.950  -5.245  1.00  0.00           H  &#xA;ATOM     74 2HB  LEU     6      38.279  13.552  -7.384  1.00  0.00           H  &#xA;ATOM     75 3HB  LEU     6      36.958  14.216  -8.325  1.00  0.00           H  &#xA;ATOM     76  HG  LEU     6      35.495  12.316  -7.792  1.00  0.00           H  &#xA;ATOM     77 1HD1 LEU     6      37.097  12.092  -9.716  1.00  0.00           H  &#xA;ATOM     78 2HD1 LEU     6      38.318  11.367  -8.642  1.00  0.00           H  &#xA;ATOM     79 3HD1 LEU     6      36.787  10.518  -8.962  1.00  0.00           H  &#xA;ATOM     80 1HD2 LEU     6      37.799  11.259  -6.005  1.00  0.00           H  &#xA;ATOM     81 2HD2 LEU     6      36.175  11.759  -5.470  1.00  0.00           H  &#xA;ATOM     82 3HD2 LEU     6      36.357  10.318  -6.476  1.00  0.00           H  &#xA;TER   &#xA;END&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;peptide_2_moved.pdb&lt;/code&gt; , same pdb with some just carbon alpha moved by hand :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATOM      1  N   ALA     1      19.613   5.728  -6.281  1.00  0.00           N  &#xA;ATOM      2  CA  ALA     1      20.289   7.022  -6.281  1.00  0.00           C  &#xA;ATOM      3  C   ALA     1      21.789   6.848  -6.281  1.00  0.00           C  &#xA;ATOM      4  O   ALA     1      22.320   5.731  -6.292  1.00  0.00           O  &#xA;ATOM      5  CB  ALA     1      19.781   7.814  -5.064  1.00  0.00           C  &#xA;ATOM      6  H   ALA     1      20.159   4.796  -6.281  1.00  0.00           H  &#xA;ATOM      7  HA  ALA     1      20.021   7.561  -7.209  1.00  0.00           H  &#xA;ATOM      8 1HB  ALA     1      18.685   7.964  -5.100  1.00  0.00           H  &#xA;ATOM      9 2HB  ALA     1      20.005   7.300  -4.109  1.00  0.00           H  &#xA;ATOM     10 3HB  ALA     1      20.237   8.819  -5.007  1.00  0.00           H  &#xA;ATOM     11  N   PHE     2      22.540   7.897  -6.270  1.00  0.00           N  &#xA;ATOM     12  CA  PHE     2      24.369   7.337  -6.515  1.00  0.00           C  &#xA;ATOM     13  C   PHE     2      24.690   9.068  -6.255  1.00  0.00           C  &#xA;ATOM     14  O   PHE     2      24.050  10.156  -6.244  1.00  0.00           O  &#xA;ATOM     15  CB  PHE     2      24.437   6.942  -7.538  1.00  0.00           C  &#xA;ATOM     16  CG  PHE     2      23.824   5.546  -7.729  1.00  0.00           C  &#xA;ATOM     17  CD1 PHE     2      22.610   5.411  -8.413  1.00  0.00           C  &#xA;ATOM     18  CD2 PHE     2      24.430   4.416  -7.174  1.00  0.00           C  &#xA;ATOM     19  CE1 PHE     2      22.002   4.165  -8.523  1.00  0.00           C  &#xA;ATOM     20  CE2 PHE     2      23.826   3.167  -7.294  1.00  0.00           C  &#xA;ATOM     21  CZ  PHE     2      22.611   3.043  -7.966  1.00  0.00           C  &#xA;ATOM     22  H   PHE     2      22.130   8.821  -6.260  1.00  0.00           H  &#xA;ATOM     23  HA  PHE     2      24.287   7.177  -5.358  1.00  0.00           H  &#xA;ATOM     24 2HB  PHE     2      25.541   6.842  -7.539  1.00  0.00           H  &#xA;ATOM     25 3HB  PHE     2      24.233   7.549  -8.444  1.00  0.00           H  &#xA;ATOM     26 1HD  PHE     2      22.116   6.283  -8.820  1.00  0.00           H  &#xA;ATOM     27 2HD  PHE     2      25.362   4.509  -6.633  1.00  0.00           H  &#xA;ATOM     28 1HE  PHE     2      21.050   4.077  -9.026  1.00  0.00           H  &#xA;ATOM     29 2HE  PHE     2      24.295   2.298  -6.857  1.00  0.00           H  &#xA;ATOM     30  HZ  PHE     2      22.134   2.079  -8.049  1.00  0.00           H  &#xA;ATOM     31  N   GLY     3      25.980   9.099  -6.253  1.00  0.00           N  &#xA;ATOM     32  CA  GLY     3      26.656  10.393  -6.239  1.00  0.00           C  &#xA;ATOM     33  C   GLY     3      28.155  10.220  -6.239  1.00  0.00           C  &#xA;ATOM     34  O   GLY     3      28.685   9.105  -6.251  1.00  0.00           O  &#xA;ATOM     35  H   GLY     3      26.513   8.242  -6.261  1.00  0.00           H  &#xA;ATOM     36 3HA  GLY     3      26.365  10.908  -5.371  0.00  0.00           H  &#xA;ATOM     37  HA  GLY     3      26.378  10.953  -7.151  1.00  0.00           H  &#xA;ATOM     38  N   VAL     4      28.906  11.269  -6.227  1.00  0.00           N  &#xA;ATOM     39  CA  VAL     4      30.719  10.841  -6.509  1.00  0.00           C  &#xA;ATOM     40  C   VAL     4      31.055  12.440  -6.212  1.00  0.00           C  &#xA;ATOM     41  O   VAL     4      30.429  13.507  -6.201  1.00  0.00           O  &#xA;ATOM     42  CB  VAL     4      30.798  10.251  -7.484  1.00  0.00           C  &#xA;ATOM     43  CG1 VAL     4      32.326  10.136  -7.737  1.00  0.00           C  &#xA;ATOM     44  CG2 VAL     4      30.276   8.794  -7.470  1.00  0.00           C  &#xA;ATOM     45  H   VAL     4      28.497  12.192  -6.217  1.00  0.00           H  &#xA;ATOM     46  HA  VAL     4      30.654  10.571  -5.305  1.00  0.00           H  &#xA;ATOM     47  HB  VAL     4      30.366  10.744  -8.381  1.00  0.00           H  &#xA;ATOM     48 1HG1 VAL     4      32.862   9.673  -6.888  1.00  0.00           H  &#xA;ATOM     49 2HG1 VAL     4      32.559   9.537  -8.638  1.00  0.00           H  &#xA;ATOM     50 3HG1 VAL     4      32.799  11.120  -7.921  1.00  0.00           H  &#xA;ATOM     51 1HG2 VAL     4      29.177   8.752  -7.361  1.00  0.00           H  &#xA;ATOM     52 2HG2 VAL     4      30.510   8.264  -8.413  1.00  0.00           H  &#xA;ATOM     53 3HG2 VAL     4      30.710   8.201  -6.643  1.00  0.00           H  &#xA;ATOM     54  N   ALA     5      32.345  12.471  -6.211  1.00  0.00           N  &#xA;ATOM     55  CA  ALA     5      32.965  13.942  -6.049  1.00  0.00           C  &#xA;ATOM     56  C   ALA     5      34.521  13.592  -6.197  1.00  0.00           C  &#xA;ATOM     57  O   ALA     5      35.052  12.475  -6.219  1.00  0.00           O  &#xA;ATOM     58  CB  ALA     5      32.511  14.544  -4.971  1.00  0.00           C  &#xA;ATOM     59  H   ALA     5      32.879  11.614  -6.220  1.00  0.00           H  &#xA;ATOM     60  HA  ALA     5      32.753  14.314  -7.119  1.00  0.00           H  &#xA;ATOM     61 1HB  ALA     5      31.415  14.694  -5.007  1.00  0.00           H  &#xA;ATOM     62 2HB  ALA     5      32.734  14.020  -4.022  1.00  0.00           H  &#xA;ATOM     63 3HB  ALA     5      32.967  15.549  -4.903  1.00  0.00           H  &#xA;ATOM     64  N   LEU     6      35.271  14.641  -6.173  1.00  0.00           N  &#xA;ATOM     65  CA  LEU     6      37.139  14.337  -6.385  1.00  0.00           C  &#xA;ATOM     66  C   LEU     6      37.420  15.811  -6.143  1.00  0.00           C  &#xA;ATOM     67  O   LEU     6      36.792  16.881  -6.122  1.00  0.00           O  &#xA;ATOM     68  CB  LEU     6      37.174  13.638  -7.404  1.00  0.00           C  &#xA;ATOM     69  CG  LEU     6      36.580  12.214  -7.580  1.00  0.00           C  &#xA;ATOM     70  CD1 LEU     6      37.231  11.514  -8.782  1.00  0.00           C  &#xA;ATOM     71  CD2 LEU     6      36.743  11.343  -6.323  1.00  0.00           C  &#xA;ATOM     72  H   LEU     6      34.862  15.564  -6.154  1.00  0.00           H  &#xA;ATOM     73  HA  LEU     6      37.013  13.950  -5.245  1.00  0.00           H  &#xA;ATOM     74 2HB  LEU     6      38.279  13.552  -7.384  1.00  0.00           H  &#xA;ATOM     75 3HB  LEU     6      36.958  14.216  -8.325  1.00  0.00           H  &#xA;ATOM     76  HG  LEU     6      35.495  12.316  -7.792  1.00  0.00           H  &#xA;ATOM     77 1HD1 LEU     6      37.097  12.092  -9.716  1.00  0.00           H  &#xA;ATOM     78 2HD1 LEU     6      38.318  11.367  -8.642  1.00  0.00           H  &#xA;ATOM     79 3HD1 LEU     6      36.787  10.518  -8.962  1.00  0.00           H  &#xA;ATOM     80 1HD2 LEU     6      37.799  11.259  -6.005  1.00  0.00           H  &#xA;ATOM     81 2HD2 LEU     6      36.175  11.759  -5.470  1.00  0.00           H  &#xA;ATOM     82 3HD2 LEU     6      36.357  10.318  -6.476  1.00  0.00           H  &#xA;TER   &#xA;END&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;code :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pymol&#xA;&#xA;import math&#xA;&#xA;from pymol import (&#xA;                    &#xA;                    cmd ,&#xA;                    stored &#xA;                    )&#xA;&#xA;print('########## PYMOL VERSION ##########################################')&#xA;print('         ',  cmd.get_version() )&#xA;print('###################################################################')&#xA;&#xA;pymol.finish_launching()&#xA;&#xA;&#xA;cmd.set('internal_gui_width' ,  '500')&#xA;&#xA;cmd.load('peptide_1.pdb' , 'peptide_1')&#xA;&#xA;cmd.load('peptide_2_moved.pdb' , 'peptide_2_moved')&#xA;&#xA;diff_len = lambda x,y : math.sqrt((x[0]-y[0])*(x[0]-y[0]) + (x[1]-y[1])*(x[1]-y[1]) + (x[2]-y[2])*(x[2]-y[2]))&#xA;&#xA;&#xA;&#xA;cmd.select('peptide_1_CA' ,'peptide_1 and name CA')&#xA;&#xA;cmd.select('peptide_2_moved_CA' ,'peptide_2_moved and name CA')&#xA;&#xA;stored.peptide_1_CA_coord = []&#xA;&#xA;stored.peptide_2_moved_CA_coord = []&#xA;&#xA;cmd.iterate_state('1' , 'peptide_1_CA' , 'stored.peptide_1_CA_coord.append((x,y,z))')&#xA;            &#xA;cmd.iterate_state('1' , 'peptide_2_moved_CA' , 'stored.peptide_2_moved_CA_coord.append((x,y,z))')&#xA;&#xA;&#xA;print('stored.peptide_1_CA_coord : ' , stored.peptide_1_CA_coord)&#xA;&#xA;print('stored.peptide_2_moved_CA_coord : ' , stored.peptide_2_moved_CA_coord)&#xA;&#xA;stored.newB = list(map(lambda x,y: diff_len(x,y), stored.peptide_2_moved_CA_coord, stored.peptide_1_CA_coord))&#xA;&#xA;print('\nstorednewB to be passed : ', stored.newB , '\n')&#xA;&#xA;cmd.alter('peptide_2_moved_CA', 'b=stored.newB.pop(0)')&#xA;&#xA;&#xA;cmd.save('peptide_2_moved_CA.pdb' , 'peptide_2_moved_CA')&#xA;&#xA;cmd.spectrum('b', 'rainbow_rev' , 'peptide_2_moved_CA' , '0' , '1')&#xA;&#xA;&#xA;cmd.color('lightblue' , 'peptide_1' )&#xA;&#xA;cmd.set('cartoon_transparency' , '0.5', 'peptide_1')&#xA;&#xA;cmd.alphatoall('peptide_2_moved')&#xA;&#xA;&#xA;cmd.save('peptide_2_moved_CA_alphatoall.pdb' , 'peptide_2_moved')&#xA;&#xA;&#xA;cmd.load('peptide_2_moved_CA_alphatoall.pdb' , 'peptide_2_moved_CA_alphatoall')&#xA;&#xA;cmd.spectrum('b', 'rainbow_rev' , 'peptide_2_moved_CA_alphatoall' , '0' , '1')&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;########## PYMOL VERSION ##########################################&#xA;          ('2.3.0'&#xA;...&#xA;stored.peptide_1_CA_coord :  [(20.288999557495117, 7.021999835968018, -6.281000137329102), (23.989999771118164, 7.730000019073486, -6.269999980926514), (26.6560001373291, 10.392999649047852, -6.238999843597412), (30.356000900268555, 11.10099983215332, -6.228000164031982), (33.020999908447266, 13.765000343322754, -6.196000099182129), (36.72200012207031, 14.472999572753906, -6.172999858856201)]&#xA;stored.peptide_2_moved_CA_coord :  [(20.288999557495117, 7.021999835968018, -6.281000137329102), (24.368999481201172, 7.336999893188477, -6.514999866485596), (26.6560001373291, 10.392999649047852, -6.238999843597412), (30.7189998626709, 10.840999603271484, -6.508999824523926), (32.96500015258789, 13.942000389099121, -6.048999786376953), (37.138999938964844, 14.336999893188477, -6.385000228881836)]&#xA;&#xA;storednewB to be passed :  [0.0, 0.5984269572074781, 0.0, 0.5275691186189471, 0.23679966390593804, 0.48716415818795433] &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;showing in pymol window:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xHmvs.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xHmvs.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;that is the CAs trace of moved (rainbow colored) vs original (lightblue [lightgray out of ray to png]) according to Palette &lt;a href=&quot;https://pymolwiki.org/index.php/Palette_Colorbars&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pymolwiki.org/index.php/Palette_Colorbars&lt;/a&gt; &lt;code&gt;rainbow_rev&lt;/code&gt; :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/TvWOH.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/TvWOH.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;that goes from red = 0 to BLUE = 1 as set out in&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;cmd.spectrum('b', 'rainbow_rev' , 'peptide_2_moved_CA' , '0' , '1')&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;againg transferred calculated CAs to all atom using &lt;code&gt;AlphaToAll&lt;/code&gt; :&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;cmd.alphatoall('peptide_2_moved')&lt;/code&gt; (check it into code) &lt;a href=&quot;https://i.stack.imgur.com/MmExF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MmExF.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;To use &lt;code&gt;spectrum&lt;/code&gt; from min to max of the actual distance differences use in code above:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;.....&#xA;&#xA;stored.min = min(stored.newB)&#xA;&#xA;stored.max = max(stored.newB)&#xA;&#xA;print('\nmin of storednewB : ' , stored.min)&#xA;&#xA;print('\nmax of storednewB : ' , stored.max)&#xA;&#xA;print('\n')&#xA;&#xA;&#xA;cmd.alter('peptide_2_moved_CA', 'b=stored.newB.pop(0)')&#xA;&#xA;cmd.save('peptide_2_moved_CA.pdb' , 'peptide_2_moved_CA')&#xA;&#xA;&#xA;cmd.spectrum('b', 'rainbow_rev' , 'peptide_2_moved_CA' , stored.min , stored.max)&#xA;....&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IbD7h.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IbD7h.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10049" LastActivityDate="2023-07-22T22:41:27.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21351" PostTypeId="2" ParentId="21345" CreationDate="2023-07-23T04:34:20.253" Score="0" Body="&lt;p&gt;Generally remote biological servers, for example UCSC genome server, will not perform SQL queries for a free account. They'll allow downloads, but not queries. That is probably the same reason here.&lt;/p&gt;&#xA;&lt;p&gt;The reason is simply these are often huge databases and queries can take up a lot of their computational resources, particularly if everyone is doing them. I don't know BioBank UK, but suggest its common practice.&lt;/p&gt;&#xA;&lt;p&gt;If thats true the only difference with Biobank and UCSC is the UCSC server refusals are stated very clearly, &amp;quot;this operation ain't permitted for free accounts&amp;quot; (paraphrasing). What might be happening in an API error is not being picked up and relayed to you. Depends how the API is written so it's just reporting a timeout.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-07-23T12:38:09.787" LastActivityDate="2023-07-23T12:38:09.787" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21353" PostTypeId="1" AcceptedAnswerId="21355" CreationDate="2023-07-24T13:06:14.137" Score="2" ViewCount="90" Body="&lt;p&gt;Consider a sorted, indexed, only mapped reads containing BAM file.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to get a sub BAM based on read line numbers?&lt;/p&gt;&#xA;&lt;p&gt;For example get a sub BAM file which contains read line numbers [1, 3, 8, 70] of the original BAM.&lt;/p&gt;&#xA;&lt;p&gt;This can be done iteratively using a counter but its too slow. Is there a way to do this using samtools view or some tool like it.&lt;/p&gt;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Size of the BAM file is ~7GB. Its human exome.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I'm using Julia (&lt;a href=&quot;https://github.com/BioJulia/XAM.jl/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/BioJulia/XAM.jl/&lt;/a&gt;) to iterate over the BAM and it takes me ~300 seconds to iterate over all the records. I feel this is too slow.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I'm running a function on each read of the BAM file and it decides which bucket the read has to go into. So this is specific subsampling.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I think &lt;code&gt;samtools view -&lt;/code&gt; and &lt;code&gt;awk&lt;/code&gt; route seems viable.&lt;/p&gt;&#xA;" OwnerUserId="17846" LastEditorUserId="17846" LastEditDate="2023-07-24T17:23:55.253" LastActivityDate="2023-07-24T21:03:19.187" Title="How to subsample BAM based on read line number?" Tags="|sam|samtools|pysam|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21354" PostTypeId="2" ParentId="21345" CreationDate="2023-07-24T17:01:36.297" Score="1" Body="&lt;p&gt;It looks like the issue comes from one of the fields I was filtering for. When I remove that filter the code works again. Still need to investigate why that happened.&lt;/p&gt;&#xA;" OwnerUserId="17862" LastActivityDate="2023-07-24T17:01:36.297" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21355" PostTypeId="2" ParentId="21353" CreationDate="2023-07-24T21:03:19.187" Score="2" Body="&lt;p&gt;Samtools+awk:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;samtools view -h aln.bam \&#xA;  | awk 'BEGIN{while((getline&amp;lt;&amp;quot;lineno.txt&amp;quot;)&amp;gt;0)a[$1]=1}/^@/{++n;print}!/^@/&amp;amp;&amp;amp;a[NR-n]' \&#xA;  | samtools view -b --threads=4 &amp;gt; sub.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would recommend &lt;a href=&quot;https://invisible-island.net/mawk/&quot; rel=&quot;nofollow noreferrer&quot;&gt;mawk&lt;/a&gt; which is much faster than other awk.&lt;/p&gt;&#xA;&lt;p&gt;I believe using Julia should be faster as it would not need to convert BAM to SAM and then convert back. Five min to read through a 7GB BAM file sounds about right. That is not slow.&lt;/p&gt;&#xA;" OwnerUserId="37" LastActivityDate="2023-07-24T21:03:19.187" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21356" PostTypeId="1" CreationDate="2023-07-25T03:24:33.517" Score="2" ViewCount="32" Body="&lt;p&gt;I am using bcftools to merge 5000+ vcf files  using this command&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools merge -l list_of_vcfs.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am getting the following error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; Warning: The row size is too big for FORMAT/PL at chr1:23899810, requires 12204615120 bytes, skipping.&#xA;The REF prefixes differ: C vs N (1,1)&#xA;Failed to merge alleles at chr1:53128382 in &amp;lt;sample&amp;gt;.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any help on how to fix this issue would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="17889" LastEditorUserId="3848" LastEditDate="2023-07-25T19:26:52.040" LastActivityDate="2023-07-25T19:26:52.040" Title="bcftool merge of 5K samples makes ID a large string" Tags="|vcf|bcftools|merge|command-line|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21357" PostTypeId="1" CreationDate="2023-07-25T12:54:06.820" Score="0" ViewCount="64" Body="&lt;p&gt;I have a challenge with how to correctly code indels in plink (I plan to convert my binary file set to VCF and use it for lift over in downstream analysis).&lt;/p&gt;&#xA;&lt;p&gt;For instance, my dataset has an indel with rsID rs72552763 and, alleles ATGAT and AT. Its position is chr6: 160139849 - 160139853 (GRCh 38.p14).&lt;/p&gt;&#xA;&lt;p&gt;In the .map file, which bp position should I indicate?&lt;/p&gt;&#xA;&lt;p&gt;With respect to the .ped file, how should I code the alleles/genotypes?&lt;/p&gt;&#xA;" OwnerUserId="17894" LastActivityDate="2023-07-25T12:54:06.820" Title="Correctly coding indels for ped/map files in plink and VCF" Tags="|vcf|plink|indel|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21358" PostTypeId="1" CreationDate="2023-07-25T16:26:32.197" Score="3" ViewCount="126" Body="&lt;p&gt;I am looking for an R based solution to the following problem:&lt;/p&gt;&#xA;&lt;p&gt;Trypsin does not cleave perfectly in proteomics experiments. So starting from a protein sequence FASTA I want to generate all possible peptides which COULD be generated if trypsin was allowed n number of &amp;quot;missed cleavages&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;In informatics terms I would describe this as skipping a split during a &lt;code&gt;strsplit()&lt;/code&gt; function&lt;/p&gt;&#xA;&lt;p&gt;Example to clarify:&lt;/p&gt;&#xA;&lt;p&gt;We start out with the protein sequence: &lt;code&gt;GRGKA&lt;/code&gt; which is usually split by &lt;code&gt;strsplit()&lt;/code&gt; (digested by trypsin) after every K and R&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;quot;GRGKA&amp;quot; %&amp;gt;% strsplit(., &amp;quot;(?&amp;lt;=K|R)&amp;quot;, perl = TRUE)&#xA;&#xA;[[1]]&#xA;[1] &amp;quot;GR&amp;quot; &amp;quot;GK&amp;quot; &amp;quot;A&amp;quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;now imagine that trypsin did not cleave perfectly, for example missing always one K or R&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;quot;GRGK&amp;quot; &amp;quot;GKA&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;would be generated in this case. Or if trypsin digested this protein really badly and would end up missing two cleavages we would end up with the sequence:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;GRGKA&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would love to have a generalizable solution that allows to generate all combinations of sequences that arise with n number of missed cleavages.&lt;/p&gt;&#xA;&lt;p&gt;If you need any more information please feel free to ask, sorry if I missed something!&lt;/p&gt;&#xA;&lt;p&gt;jay.sf's requested example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;FLGERTCTRNERQP&amp;quot;  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;perfect digest:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;FLGER&amp;quot; &amp;quot;TCTR&amp;quot; &amp;quot;NER&amp;quot; &amp;quot;QP&amp;quot;   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;one missed cleavage all combinations:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;FLGERTCTR&amp;quot; &amp;quot;TCTRNER&amp;quot; &amp;quot;NERQP&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;two missed cleavages all combinations:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;FLGERTCTRNER&amp;quot; &amp;quot;QP&amp;quot; &amp;quot;FLGER&amp;quot; &amp;quot;TCTRNERQP&amp;quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;three missed cleavages (there are a total of three K and Rs so trypsin would miss all)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;FLGERTCTRNERQP&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For longer sequences I think this becomes a frameshift like problem: If we count the K and Rs in the sequence:&lt;/p&gt;&#xA;&lt;p&gt;FLGE&lt;strong&gt;R&lt;/strong&gt;TCT&lt;strong&gt;R&lt;/strong&gt;NE&lt;strong&gt;R&lt;/strong&gt;QPFLGE&lt;strong&gt;R&lt;/strong&gt;TCT&lt;strong&gt;R&lt;/strong&gt;NE&lt;strong&gt;R&lt;/strong&gt;QP&lt;br /&gt;&#xA;FLGE&lt;strong&gt;1&lt;/strong&gt;TCT&lt;strong&gt;2&lt;/strong&gt;NE&lt;strong&gt;3&lt;/strong&gt;QPFLGE&lt;strong&gt;4&lt;/strong&gt;TCT&lt;strong&gt;5&lt;/strong&gt;NE&lt;strong&gt;6&lt;/strong&gt;QP&lt;br /&gt;&#xA;FLGE&lt;strong&gt;R&lt;/strong&gt;_TCT&lt;strong&gt;R&lt;/strong&gt;_NE&lt;strong&gt;R&lt;/strong&gt;_QPFLGE&lt;strong&gt;R&lt;/strong&gt;_TCT&lt;strong&gt;R&lt;/strong&gt;_NE&lt;strong&gt;R&lt;/strong&gt;_QP (perfect digest)&lt;br /&gt;&#xA;FLGE&lt;strong&gt;R&lt;/strong&gt;TCT&lt;strong&gt;R&lt;/strong&gt;_NE&lt;strong&gt;R&lt;/strong&gt;QPFLGE&lt;strong&gt;R&lt;/strong&gt;_TCT&lt;strong&gt;R&lt;/strong&gt;NE&lt;strong&gt;R&lt;/strong&gt;_QP (one missed cleavage and always the uneven R and Ks)&lt;br /&gt;&#xA;FLGE&lt;strong&gt;R&lt;/strong&gt;_TCT&lt;strong&gt;R&lt;/strong&gt;NE&lt;strong&gt;R&lt;/strong&gt;_QPFLGE&lt;strong&gt;R&lt;/strong&gt;TCT&lt;strong&gt;R&lt;/strong&gt;_NE&lt;strong&gt;R&lt;/strong&gt;QP  (one missed cleavage and always the even R and Ks)&lt;/p&gt;&#xA;&lt;p&gt;So I think for 1 missed cleavage we have two frames over the whole sequence and for two missed cleavages we would have three frames and so on....&lt;/p&gt;&#xA;" OwnerUserId="17896" LastEditorUserId="14153" LastEditDate="2023-07-27T08:00:50.133" LastActivityDate="2023-07-27T18:48:34.297" Title="missed cleavages combinations generation" Tags="|r|proteins|sequence-analysis|" AnswerCount="2" CommentCount="3" FavoriteCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21359" PostTypeId="1" CreationDate="2023-07-25T22:20:53.463" Score="0" ViewCount="21" Body="&lt;p&gt;I'm using this tutorial (&lt;a href=&quot;https://github.com/netZoo/netZooM/blob/master/tutorials/panda/panda.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/netZoo/netZooM/blob/master/tutorials/panda/panda.pdf&lt;/a&gt;) to use PANDA to infer gene regulatory networks, but without ppi values (which I've been told can be done easily using MATLAB). I replicated the codes given in the tutorial but I omitted all the portions with ppi. After doing so, I'm getting a lot of warnings and errors that makes me think I've made syntax errors in the process of taking out the ppi portions. Here's what I have:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;addpath(genpath('path/to/netZooM'))&#xA;&#xA;load(&amp;quot;myMat.mat&amp;quot;)&#xA;myTable = struct2table(myTable);&#xA;addpath(genpath('path/to/netZooM'));&#xA;fileparts(which('netZooM'));&#xA;&#xA;exp_file = 'tests/test_data/expression.txt';&#xA;motif_file = 'tests/test_data/motifTest.txt';&#xA;panda_out = '';&#xA;lib_path = 'C:/Users/ual-laptop/Documents/bioinformatics.m/netZooM';&#xA;save_temp = '';&#xA;&#xA;alpha = 0.1;&#xA;save_pairs = 0;&#xA;&#xA;expressionData = readTable('expression.txt');&#xA;size(expressionData)&#xA;&#xA;[TF, gene, weight] = textread(motif_file, '%s%s%f');&#xA;TFNames = unique(TF);&#xA;&#xA;disp('Reading in expression data!');&#xA;fid = fopen(exp file, 'r');&#xA;headings = fget1(fid)&#xA;n = length(regexp(headings, '\t'));&#xA;frewind(fid);&#xA;Exp = textscan(fid, ['%s', repmat('%f', 1, n)], 'delimiter', '\t');&#xA;fclose(fid);&#xA;GeneNames = Exp{1};&#xA;Exp = cat(2, Exp{2:end});&#xA;[NumGenes, NumConditions] = size(Exp);&#xA;fprintf('%d genes and %d conditions!\n', NumGenes, NumConditions);&#xA;Exp = Exp';&#xA;&#xA;disp('Computing coexpression network:');&#xA;GeneCoReg = Coexpression(Exp);&#xA;&#xA;disp('Reading in motif data!');&#xA;[TF, gene, weight] = textread(motif_file, '%s%s%f');&#xA;TFNames = unique(TF);&#xA;NumTFs = length(TFNames);&#xA;[~,i] = ismember(TF, TFNames);&#xA;[~,j] = ismember(gene, GeneNames);&#xA;RegNet = zeros(NumTFs, NumGenes);&#xA;RegNet(sub2ind([NumTFs, NumGenes], i, j));&#xA;fprintf('%d TFs and %d edges!\n', NumTFs, length(weight));&#xA;&#xA;disp('Normalizing Networks:');&#xA;RegNet = NormalizeNetwork(RegNet);&#xA;GeneCoReg = NormalizeNetwork(GeneCoReg);&#xA;TFCoop = NormalizeNetwork(TFCoop);&#xA;&#xA;disp('Running PANDA algorithm:');&#xA;AgNet = PANDA(RegNet, GeneCoReg, TFCoop, alpha);&#xA;&#xA;AgNet = panda_run(lib_path, exp_file, motif_file, panda_out, save_temp, alpha);&#xA;&#xA;sum(AgNet, 2)&#xA;sum(Agnet, 1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Are there any glaring syntax errors in my code? I'm thinking my errors could also stem from maybe cloning the github repository into the wrong place (which is a separate topic) but it's definitely possible that there are syntax errors since this is my first time doing this kind of stuff.&lt;/p&gt;&#xA;" OwnerUserId="17613" LastActivityDate="2023-07-25T22:20:53.463" Title="Using PANDA to infer gene regulatory networks without ppi" Tags="|differential-expression|pandas|matlab|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21361" PostTypeId="1" CreationDate="2023-07-26T08:45:24.333" Score="2" ViewCount="94" Body="&lt;p&gt;I have bulk data from 4 samples of triple negative breast cancer, which 2 of them are control and the other 2 are treated. The goal here is to estimate the average macrophages signatures, for M0,M1 and M2 in each of the 4 samples.&lt;/p&gt;&#xA;&lt;p&gt;My bulk data:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Gene    MCtrl1_TPM  MCtrl2_TPM  MTreat1_TPM MTreat2_TPM&#xA;KHDC1L  0.0 75.15   0.0 0.0&#xA;CD24    1.53    41.56   0.11    0.06&#xA;MAMLD1  5.2 11.94   0.0 0.0&#xA;SMIM31  4.1 10.24   0.0 0.06&#xA;C16orf95    7.9 13.24   0.0 0.08&#xA;OCLN    2.69    7.06    0.06    0.03&#xA;RBM14-RBM4  16.97   1.95    0.0 0.0&#xA;RPS10-NUDT3 0.0 0.0 31.91   12.06&#xA;NDUFC2-KCTD14   15.06   49.98   0.41    0.0&#xA;ATP9A   2.7 10.05   0.01    0.0&#xA;UST 0.38    10.63   0.0 0.01&#xA;LOC100996709    12.25   30.58   0.0 0.0&#xA;ADAM8   10.13   2.14    0.03    0.0&#xA;ADGRG2  0.05    21.49   0.0 0.0&#xA;MBNL2   4.61    5.04    0.0 0.0&#xA;PIGBOS1 56.73   51.33   0.6 0.09&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To do that, I thought was to use the CIBERSORTx software more specifically, the option - Impute Gene Expression, High-Resolution Mode (tutorial 5) with my bulk data and signature matrix file, the LM22 signature matrix file of 22 immune cells provided by CIBERSORTx, but with the selected columns correspondent to the 3 types of macrophages.&lt;/p&gt;&#xA;&lt;p&gt;Signature matrix file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Gene    M0  M1  M2&#xA;ABCB4   7.951360487 27.55771004 121.4322774&#xA;ABCB9   56.25188749 29.38087813 40.39026017&#xA;ACAP1   48.97221651 30.97425375 23.05515036&#xA;ACHE    13.6929629  1373.06962  48.58576071&#xA;ACP5    13644.1723  1062.475909 7633.960182&#xA;ADAM28  54.96648703 274.639997  360.6820442&#xA;ADAMDEC1    4464.262133 8524.022801 5004.969419&#xA;ADAMTS3 45.56049098 32.34016681 33.35142773&#xA;ADRB2   143.0178779 787.6239741 363.1850058&#xA;AIF1    1841.269508 1611.228571 11610.29035&#xA;AIM2    181.9571    2965.888162 148.2845223&#xA;ALOX15  6.621226619 5.412216318 430.3353048&#xA;ALOX5   788.5397751 83.30089783 248.3724724&#xA;AMPD1   76.54456664 62.39764428 59.95269372&#xA;ANGPT4  39.33257003 21.218127   11.8097012&#xA;ANKRD55 123.600429  96.20392388 48.55417359&#xA;APOBEC3A    85.2400852  4501.140479 148.7993403&#xA;APOBEC3G    278.7681406 1036.993881 258.3004281&#xA;APOL3   238.5980956 11009.29392 437.5272083&#xA;APOL6   13.10943681 967.6451456 7.784612131&#xA;AQP9    4583.500675 4309.341662 399.4746192&#xA;ARHGAP22    175.4574354 351.5922058 272.9890572&#xA;ARRB1   333.3962724 946.0138184 395.5562302&#xA;ASGR1   108.7218916 25.23626149 155.4913385&#xA;ASGR2   36.11978652 26.3738691  760.7257707&#xA;ATHL1   52.25820981 167.5032595 132.631945&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I encountered this error, trying to do that:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## Formatted for easier reading&#xA;Error in runCIBERSORTxGEP(weights, mixture, classes, QN, maxsamples, label, : &#xA;Abort! The number of mixture samples is not greater than the number of cell types. &#xA;Calls: CIBERSORTxHiRes -&amp;gt; CIBERSORTxGEP -&amp;gt; runCIBERSORTxGEP Execution halted&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How this error is possible, when I have 3 types of macrophages and 4 samples? There's another way to estimate the average of the 3 types of macrophages signatures? Specifically in a graphic like this?If yes, how?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yHiYJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yHiYJ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17902" LastEditorUserId="650" LastEditDate="2023-07-26T15:48:43.100" LastActivityDate="2023-10-25T15:04:08.493" Title="How to do average macrophages signature boxplots, with Bulk RNA data from 2 control and 2 treatment samples and LM22 signature matrix from cibersort?" Tags="|r|cibersortx|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21362" PostTypeId="1" CreationDate="2023-07-26T17:55:17.833" Score="-1" ViewCount="55" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ObuEr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ObuEr.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;a href=&quot;https://i.stack.imgur.com/4OrwO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4OrwO.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;Is there a data analysis software (free) or code that I can use to view normalized CDR3 amino acid length distributions?&lt;/p&gt;&#xA;&lt;p&gt;Regarding code, preferably using Python or R.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: Added a picture of the non-normalized CDR3 amino acid length distribution. The issue is that ES0017_metastasis has more samples than all other individuals which is why it has the highest peak at every amino acid length from 10-20. Was wondering if normalization was possible? Plot generated via Immunarch package in R.&lt;/p&gt;&#xA;&lt;p&gt;EDIT2: Now able to access OriginPro and Graphpad Prism via university.&lt;/p&gt;&#xA;&lt;p&gt;EDIT3:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/DFcuR.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DFcuR.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;EDIT4:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/zAUs4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zAUs4.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;Here are the results, what do you think...&#xA;The first chart is the log transformation (one extra as well, don't know how to delete). The bottom chart is the min-max transformation.&lt;/p&gt;&#xA;&lt;p&gt;The x-axis refers to amino acid length. Each discrete block is an amino acid length, e.g. 12 amino acids long, etc.&lt;/p&gt;&#xA;" OwnerUserId="17903" LastEditorUserId="17903" LastEditDate="2023-07-28T13:55:17.797" LastActivityDate="2023-08-27T20:07:16.050" Title="Is there a data analysis software (free) or code that I can use to view normalized CDR3 amino acid length distributions?" Tags="|rna-seq|sequence-analysis|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21364" PostTypeId="2" ParentId="21358" CreationDate="2023-07-27T05:57:46.090" Score="2" Body="&lt;p&gt;Essentially we want to combine &lt;em&gt;1, 2, ..., n - 1&lt;/em&gt; splits, and if the &lt;code&gt;var&lt;/code&gt;iance of split lengths is greater than one we have a second solution. Here is a function that,  after the &lt;code&gt;strsplit&lt;/code&gt;, first creates indices &lt;code&gt;cb&lt;/code&gt; to later &lt;code&gt;paste&lt;/code&gt; the respective splits together. I'm not quite sure what the maximum number of splits is, but this might be a start.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mccg &amp;lt;- function(x) {&#xA;  ## missed cleavages combinations generation ++++++++++++++++++++++++++++++++++&#xA;  gs &amp;lt;- \(n, k, length=n*k, right=FALSE) {&#xA;    ## helper, similar to `gl()` but w/o factorization&#xA;    if (right) {&#xA;      cbr &amp;lt;- gs(n, k, n, right=FALSE)&#xA;      rev(max(cbr) - cbr + 1L)&#xA;    } else {&#xA;      rep_len(rep.int(seq_len(n), rep.int(k, n)), length)&#xA;    }&#xA;  }&#xA;  ## split string after [K, R]&#xA;  spl_x &amp;lt;- el(strsplit(x, '(?&amp;lt;=K|R)', perl=TRUE))&#xA;  len &amp;lt;- length(spl_x)&#xA;  res &amp;lt;- if (len &amp;lt;= 1L) {&#xA;    ## if [K, R] at the end or missing&#xA;    x&#xA;  } else {&#xA;    ## nested list of combinations&#xA;    cb &amp;lt;- lapply(1:(len - 1L) |&amp;gt; {\(.) setNames(., . - 1L)}(), \(i, .len=len) {&#xA;      cb &amp;lt;- list(gs(.len, i, .len))&#xA;      if (i != 1L &amp;amp;&amp;amp; var(tabulate(el(cb)))) {&#xA;        ## second solution, starting from right&#xA;        c(cb, list(gs(.len, i, .len, right=TRUE)))&#xA;      } else {&#xA;        cb&#xA;      }&#xA;    }) |&amp;gt; c(setNames(list(rep_len(1L, len)), len - 1L))&#xA;    ## paste according to sequences&#xA;    rapply(cb, \(x) vapply(unname(split(spl_x, x)), paste, collapse='', &#xA;                           character(1)), &#xA;           how='list')&#xA;  }&#xA;  return(res)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Usage&lt;/h2&gt;&#xA;&lt;p&gt;&lt;em&gt;Two splits:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mccg(x='GRGKA')&#xA;# &lt;span class=&quot;math-container&quot;&gt;$`0`&#xA;# [1] &quot;GR&quot; &quot;GK&quot; &quot;A&quot; &#xA;# &#xA;# $&lt;/span&gt;`1`&#xA;# &lt;span class=&quot;math-container&quot;&gt;$`1`[[1]]&#xA;# [1] &quot;GRGK&quot; &quot;A&quot;   &#xA;# &#xA;# $&lt;/span&gt;`1`[[2]]&#xA;# [1] &amp;quot;GR&amp;quot;  &amp;quot;GKA&amp;quot;&#xA;# &#xA;# &#xA;# $`2`&#xA;# [1] &amp;quot;GRGKA&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;Three splits:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mccg(x='FLGERTCTRNERQP')&#xA;# &lt;span class=&quot;math-container&quot;&gt;$`0`&#xA;# [1] &quot;FLGER&quot; &quot;TCTR&quot;  &quot;NER&quot;   &quot;QP&quot;   &#xA;# &#xA;# $&lt;/span&gt;`1`&#xA;# [1] &amp;quot;FLGERTCTR&amp;quot; &amp;quot;NERQP&amp;quot;    &#xA;# &#xA;# &lt;span class=&quot;math-container&quot;&gt;$`2`&#xA;# $&lt;/span&gt;`2`[[1]]&#xA;# [1] &amp;quot;FLGERTCTRNER&amp;quot; &amp;quot;QP&amp;quot;          &#xA;# &#xA;# &lt;span class=&quot;math-container&quot;&gt;$`2`[[2]]&#xA;# [1] &quot;FLGER&quot;     &quot;TCTRNERQP&quot;&#xA;# &#xA;# &#xA;# $&lt;/span&gt;`3`&#xA;# [1] &amp;quot;FLGERTCTRNERQP&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;More splits:&lt;/em&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mccg(x='FLGERTCTRNERQPFLGERTCTRNERQP')&#xA;# &lt;span class=&quot;math-container&quot;&gt;$`0`&#xA;# [1] &quot;FLGER&quot;   &quot;TCTR&quot;    &quot;NER&quot;     &quot;QPFLGER&quot; &quot;TCTR&quot;    &quot;NER&quot;     &quot;QP&quot;     &#xA;# &#xA;# $&lt;/span&gt;`1`&#xA;# &lt;span class=&quot;math-container&quot;&gt;$`1`[[1]]&#xA;# [1] &quot;FLGERTCTR&quot;  &quot;NERQPFLGER&quot; &quot;TCTRNER&quot;    &quot;QP&quot;        &#xA;# &#xA;# $&lt;/span&gt;`1`[[2]]&#xA;# [1] &amp;quot;FLGER&amp;quot;       &amp;quot;TCTRNER&amp;quot;     &amp;quot;QPFLGERTCTR&amp;quot; &amp;quot;NERQP&amp;quot;      &#xA;# &#xA;# &#xA;# &lt;span class=&quot;math-container&quot;&gt;$`2`&#xA;# $&lt;/span&gt;`2`[[1]]&#xA;# [1] &amp;quot;FLGERTCTRNER&amp;quot;   &amp;quot;QPFLGERTCTRNER&amp;quot; &amp;quot;QP&amp;quot;            &#xA;# &#xA;# &lt;span class=&quot;math-container&quot;&gt;$`2`[[2]]&#xA;# [1] &quot;FLGER&quot;          &quot;TCTRNERQPFLGER&quot; &quot;TCTRNERQP&quot;     &#xA;# &#xA;# &#xA;# $&lt;/span&gt;`3`&#xA;# &lt;span class=&quot;math-container&quot;&gt;$`3`[[1]]&#xA;# [1] &quot;FLGERTCTRNERQPFLGER&quot; &quot;TCTRNERQP&quot;          &#xA;# &#xA;# $&lt;/span&gt;`3`[[2]]&#xA;# [1] &amp;quot;FLGERTCTRNER&amp;quot;     &amp;quot;QPFLGERTCTRNERQP&amp;quot;&#xA;# &#xA;# &#xA;# &lt;span class=&quot;math-container&quot;&gt;$`4`&#xA;# $&lt;/span&gt;`4`[[1]]&#xA;# [1] &amp;quot;FLGERTCTRNERQPFLGERTCTR&amp;quot; &amp;quot;NERQP&amp;quot;                  &#xA;# &#xA;# &lt;span class=&quot;math-container&quot;&gt;$`4`[[2]]&#xA;# [1] &quot;FLGERTCTR&quot;           &quot;NERQPFLGERTCTRNERQP&quot;&#xA;# &#xA;# &#xA;# $&lt;/span&gt;`5`&#xA;# &lt;span class=&quot;math-container&quot;&gt;$`5`[[1]]&#xA;# [1] &quot;FLGERTCTRNERQPFLGERTCTRNER&quot; &quot;QP&quot;                        &#xA;# &#xA;# $&lt;/span&gt;`5`[[2]]&#xA;# [1] &amp;quot;FLGER&amp;quot;                   &amp;quot;TCTRNERQPFLGERTCTRNERQP&amp;quot;&#xA;# &#xA;# &#xA;# $`6`&#xA;# [1] &amp;quot;FLGERTCTRNERQPFLGERTCTRNERQP&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Simulation&lt;/h2&gt;&#xA;&lt;p&gt;Here are some toy data,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;set.seed(42)&#xA;n &amp;lt;- 1e3&#xA;min_chr &amp;lt;- 5; max_chr &amp;lt;- 50&#xA;strings &amp;lt;- replicate(n, {&#xA;  repeat {&#xA;    smp &amp;lt;- sample(setdiff(LETTERS, c(&amp;quot;B&amp;quot;, &amp;quot;Z&amp;quot;, &amp;quot;J&amp;quot;, &amp;quot;O&amp;quot;, &amp;quot;U&amp;quot;, &amp;quot;X&amp;quot;)),&#xA;                  sample(min_chr:max_chr, 1L), replace=TRUE)&#xA;    if (any(grepl('K|R', smp))) break&#xA;  }&#xA;  paste(smp, collapse='')&#xA;}) |&amp;gt; setNames(nm=_)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;over which the execution of the function can be tested in a loop.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;r &amp;lt;- lapply(strings, \(x) tryCatch(mccg(x), error=\(e) NA))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14153" LastEditorUserId="14153" LastEditDate="2023-07-27T18:48:34.297" LastActivityDate="2023-07-27T18:48:34.297" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21365" PostTypeId="2" ParentId="21362" CreationDate="2023-07-27T12:04:41.297" Score="0" Body="&lt;h3&gt;Final update&lt;/h3&gt;&#xA;&lt;p&gt;Okay the answer below is based on the second graph from the top. Looking at the update the correct answer is closer to the first answer presented. &lt;strong&gt;EDIT 3&lt;/strong&gt; is normalised data. It appears that the y-axis is probably natural log transformed (log&lt;sub&gt;e&lt;/sub&gt;) or log&lt;sub&gt;2&lt;/sub&gt; and this transformation has normalised it.&lt;/p&gt;&#xA;&lt;p&gt;Thus there is not need to normalise the data, the program has already done it in EDIT3 and in the very top graph now presented.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;This looks impressive, thanks. I get it. There are two way to do this. If you are normalising the frequency (y-axis) is what you are wanting to normalise then either (in this instance its the wrong answer [see below the ----]):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a log10 transformation (even Excel will log this data)&lt;/li&gt;&#xA;&lt;li&gt;a max-min transformation (thats in Python, likely in R - I dunno a general package maybe SPSS??, dunno)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;You then test via the normal distribution using with &lt;a href=&quot;https://en.wikipedia.org/wiki/Shapiro%E2%80%93Wilk_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;Shapiro–Wilk test&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kolmogorov–Smirnov test&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I suspect the max-min transformation will work. There are other ways in but it will involve discarding data that is &amp;gt;3SD from the mean. Try the above and see how it works.&lt;/p&gt;&#xA;&lt;p&gt;Note, a specialist statistician will request you perform a QQ plot, because the above tests of normalisation are not perfect.&lt;/p&gt;&#xA;&lt;p&gt;This is not standardisation this is conformation to the normal distribution.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;This could be about bin size if you have a discrete range for gene length and doesn't need to be very big - normally you need to try a few values, in this case its just 1 amino acid bin ... This can be used as a sliding window. It will pull the data into a normal distribution. That may not be very well explained, but I'm pretty sure that will work.&lt;/p&gt;&#xA;&lt;p&gt;1 amino acid sounds weird but part the failure to see a normal distribution is a result of your calculation: you are splitting an amino acid into fractions and thats whats causing the irregularities in the data. Thus between values 10 and 20 on the x-axis (presumably size range difference) you should only have 20 bars - but you've got loads &amp;gt; 100 which isn't possible.&lt;/p&gt;&#xA;&lt;p&gt;Do that try SW and KS tests and you're sorted.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-07-28T19:53:03.880" LastActivityDate="2023-07-28T19:53:03.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21366" PostTypeId="1" CreationDate="2023-07-27T18:07:30.193" Score="1" ViewCount="35" Body="&lt;p&gt;I am working with yeast strains (specifically S. cerevisiae) that have been cultured and evolved over many generations in the lab. I am using the reference S288C strain when I do certain analyses but I want to be able to construct my own reference using the &amp;quot;in-house&amp;quot; strains. Ultimately I want to be able to call variants when mapping strains to this &amp;quot;in-house&amp;quot; reference to make it easier to visualize/understand the difference in strains.&lt;/p&gt;&#xA;&lt;p&gt;I am trying to use &lt;a href=&quot;https://github.com/vgteam/vg&quot; rel=&quot;nofollow noreferrer&quot;&gt;VG-tools&lt;/a&gt; to create a pangenome graph using 10 different strains all sharing the same reference and I'm honestly a little confused on how to go about doing it. I was able to make a &amp;quot;vg&amp;quot; file for each sample individually but I know downstream I want to be able to map other strains to this pangenome as a reference and call variants to it rather than the standard reference sequence.&lt;/p&gt;&#xA;&lt;p&gt;I'm mainly looking to see if anyone has built a &amp;quot;pangenome graph&amp;quot; using multiple samples and how they went about doing it. If anyone could give me some tips/tricks/advice for this kind of task that would be great!&lt;/p&gt;&#xA;&lt;p&gt;I have a phylogenetic tree of all the strains but how would I use that to see the differences in variation?&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="73" LastEditDate="2023-07-29T03:29:51.433" LastActivityDate="2023-07-29T03:29:51.433" Title="Constructing a pangenome graph using multiple samples" Tags="|reference-genome|graphs|pangenome|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21367" PostTypeId="1" CreationDate="2023-07-28T08:10:51.350" Score="2" ViewCount="39" Body="&lt;p&gt;I am using the PGS Calculator (&lt;a href=&quot;https://github.com/PGScatalog/pgsc_calc/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/PGScatalog/pgsc_calc/&lt;/a&gt;) to compute the polygenic scores of individuals based on a scoring file.&lt;/p&gt;&#xA;&lt;p&gt;PGS Calculator works wonderfully and outputs the aggregated score for each individual.&lt;/p&gt;&#xA;&lt;p&gt;However, I am interested in the contribution of each variant to the score of each individual.&lt;/p&gt;&#xA;&lt;p&gt;At the moment, I have:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;IID&lt;/th&gt;&#xA;&lt;th&gt;PGS&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;33432984732&lt;/td&gt;&#xA;&lt;td&gt;10&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;32432809751&lt;/td&gt;&#xA;&lt;td&gt;20&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;I would like the following table:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;IID&lt;/th&gt;&#xA;&lt;th&gt;Variant 1&lt;/th&gt;&#xA;&lt;th&gt;Variant 2&lt;/th&gt;&#xA;&lt;th&gt;Variant 3&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;33432984732&lt;/td&gt;&#xA;&lt;td&gt;3&lt;/td&gt;&#xA;&lt;td&gt;4&lt;/td&gt;&#xA;&lt;td&gt;3&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;32432809751&lt;/td&gt;&#xA;&lt;td&gt;6&lt;/td&gt;&#xA;&lt;td&gt;8&lt;/td&gt;&#xA;&lt;td&gt;6&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;in which I can see the contribution of each variant to the overall score.&lt;/p&gt;&#xA;&lt;p&gt;Is this possible with PGS Calculator? If not, is there a tool other than PGS Calculator that I can use? Or should I use my own scripts?&lt;/p&gt;&#xA;" OwnerUserId="17128" LastEditorUserId="650" LastEditDate="2023-07-28T14:55:22.833" LastActivityDate="2024-03-05T07:00:55.063" Title="Can the PGS Calculator give the contribution of each variant to the overall PGS?" Tags="|polygenic-risk-score|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21368" PostTypeId="2" ParentId="18961" CreationDate="2023-07-28T20:06:07.397" Score="2" Body="&lt;h2&gt;tl;dr&lt;/h2&gt;&#xA;&lt;p&gt;Use an awk script (below) or &amp;quot;one-liner&amp;quot;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '{name = &amp;quot;&amp;gt; &amp;quot;; field = 2; while ( &lt;span class=&quot;math-container&quot;&gt;$field + 0 != $&lt;/span&gt;field ) { name = name &amp;quot; &amp;quot; &lt;span class=&quot;math-container&quot;&gt;$field; field = field+1 } ; field = NF; if ( $&lt;/span&gt;field + 0 == &lt;span class=&quot;math-container&quot;&gt;$field ) { field = field -1 }; while ( $&lt;/span&gt;field + 0 != &lt;span class=&quot;math-container&quot;&gt;$field ) { field = field - 1 }; field = field - 1; seq = $&lt;/span&gt;field; gsub(/-/, &amp;quot;&amp;quot;, seq); print name; print seq}' the_m8_file.m8 &amp;gt; the_fasta_file.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Caveat emptor: I've tested this on my own Foldseek outputs and it works for my pedestrian purposes of multisequence alignment with COBALT. I haven't tested it with PROSS but expect it should work for you too.&lt;/p&gt;&#xA;&lt;h2&gt;Long answer:&lt;/h2&gt;&#xA;&lt;p&gt;Using this definition of m8 (which is a modification of the BLAST m8 format):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/steineggerlab/foldseek#output-search&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/steineggerlab/foldseek#output-search&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And this definition of FASTA:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://en.wikipedia.org/wiki/FASTA_format&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/FASTA_format&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;We see we want to take lines like this (I've added [...] here to show what I've cut off for brevity):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;job.pdb_A   Some matched structure name 51.100  311 138 8   2   305 25  328 1.000   2.790E-35   1272    306 334 VDVSVSIFINKIYGVNTLEQTYKVDGYIVAQWTGKPR--KTPGDKPLIVENTQIERWINNGLWVPALEFINVVGSPDTGNKRLMLFPDGRVIYNARFLGSFSNDMDFRLFPFDRQQFVLELEPFSYNNQQLRF--SDIQVYTENIDN[...] -27.488,-8.287,-35.993,-26.838,-4.500,-36.421,-25.278,-3.958,-32.938,-22.398,-6.436,-33.555,-21.375,-4.861,-36.892,-21.373,-1.326,-35.385,-19.237,-2.585,--18.044,-1.289,8.073,-14.345,-2.072,7.367,-15.099,-5.282,5.328,-16.872,-3.358,2.522,-16.876,-5.018,-0.77 [...] MIKKEILSAIASLVWFCSPALYAQYVVDTSVSINKIYGVNTVDQTYKVDGYLVTSWDEPDPTYRPASGERVYENHHADQLMNDGLWVPAFEFINIIGQRQTANRRVVISADGRVTYNERFQGLFTTTMDFRQFPFDYQTFVLMMEPFSYEQKQL[...]   1918945 Vibrio spartinae&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and map them to entries like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;some matched structure name&#xA;MIKKEILSAIASLVWFCSPALYAQYVVDTSVSINKIYGVNTVDQTYKVDGYLVTSWDEPDPTYRPASGERVYENHHADQLMNDGLWVPAFEFINIIGQRQTANRRVVISADGRVTYNERFQGLFTTTMDFRQFPFDYQTFVLMMEPFSYEQKQL[...]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;We can do this with an awk script (The awk manual is your friend here &lt;a href=&quot;https://www.gnu.org/software/gawk/manual/html_node/Long.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.gnu.org/software/gawk/manual/html_node/Long.html&lt;/a&gt;). Put the following in a file somewhere handy and call it using &lt;code&gt;awk -f m8_to_fasta.awk my_m8_file.m8 &amp;gt; my_fasta_file.fasta&lt;/code&gt;&#xA;you may want to combine this with head, tail, and/or grep to reduce the number of sequences that end up in your output.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#! /bin/awk    &#xA;{&#xA;        name = &amp;quot;&amp;gt; &amp;quot;&#xA;        # start on the second field, foldseek will put the job name in &lt;span class=&quot;math-container&quot;&gt;$1&#xA;        field = 2 &#xA;        # Check each field until we see numbers&#xA;        while ( $&lt;/span&gt;field + 0 != &lt;span class=&quot;math-container&quot;&gt;$field ) { &#xA;           name = name &quot; &quot; $&lt;/span&gt;field&#xA;           field = field+1 &#xA;        }&#xA;        # name now contains the name string which is the first line of a fasta entry&#xA;        &#xA;        field = NF&#xA;        # Now find the matched sequence&#xA;        # Sometimes an entry ENDS in a number, so we strip that&#xA;        if ( &lt;span class=&quot;math-container&quot;&gt;$field + 0 == $&lt;/span&gt;field ) { field = field -1 }&#xA;        while ( &lt;span class=&quot;math-container&quot;&gt;$field + 0 != $&lt;/span&gt;field ) {&#xA;            field = field - 1&#xA;        }&#xA;        # field is now pointing at some undocumented number. Decrement once more to get the sequence&#xA;        field = field - 1&#xA;        # field is now pointing at the sequence&#xA;        seq = $field&#xA;        &#xA;        #Now I/we want to sanitize the sequence to remove any gaps&#xA;        gsub(/-/, &amp;quot;&amp;quot;, seq)&#xA;        &#xA;        print name&#xA;        print seq&#xA;    }&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or we can coerce this into a monstrous one-liner:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; awk '{name = &amp;quot;&amp;gt; &amp;quot;; field = 2; while ( &lt;span class=&quot;math-container&quot;&gt;$field + 0 != $&lt;/span&gt;field ) { name = name &amp;quot; &amp;quot; &lt;span class=&quot;math-container&quot;&gt;$field; field = field+1 } ; field = NF; if ( $&lt;/span&gt;field + 0 == &lt;span class=&quot;math-container&quot;&gt;$field ) { field = field -1 }; while ( $&lt;/span&gt;field + 0 != &lt;span class=&quot;math-container&quot;&gt;$field ) { field = field - 1 }; field = field - 1; seq = $&lt;/span&gt;field; gsub(/-/, &amp;quot;&amp;quot;, seq); print name; print seq}' the_m8_file.m8 &amp;gt; the_fasta_file.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17909" LastEditorUserId="650" LastEditDate="2023-07-28T20:28:31.797" LastActivityDate="2023-07-28T20:28:31.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21369" PostTypeId="1" CreationDate="2023-07-28T20:27:21.523" Score="1" ViewCount="33" Body="&lt;p&gt;I have the following list of intervals&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;Chr&lt;/th&gt;&#xA;&lt;th&gt;start&lt;/th&gt;&#xA;&lt;th&gt;end&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;5&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;5&lt;/td&gt;&#xA;&lt;td&gt;10&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;10&lt;/td&gt;&#xA;&lt;td&gt;15&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;15&lt;/td&gt;&#xA;&lt;td&gt;20&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;20&lt;/td&gt;&#xA;&lt;td&gt;25&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;25&lt;/td&gt;&#xA;&lt;td&gt;30&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;and the following CpGs I'd like to intersect with the intervals and aggregate the last two columns using sum.&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;chr&lt;/th&gt;&#xA;&lt;th&gt;start&lt;/th&gt;&#xA;&lt;th&gt;end&lt;/th&gt;&#xA;&lt;th&gt;end -1&lt;/th&gt;&#xA;&lt;th&gt;coverage&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;3&lt;/td&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;4&lt;/td&gt;&#xA;&lt;td&gt;6&lt;/td&gt;&#xA;&lt;td&gt;5&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;7&lt;/td&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;8&lt;/td&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;10&lt;/td&gt;&#xA;&lt;td&gt;12&lt;/td&gt;&#xA;&lt;td&gt;11&lt;/td&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;13&lt;/td&gt;&#xA;&lt;td&gt;15&lt;/td&gt;&#xA;&lt;td&gt;14&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;16&lt;/td&gt;&#xA;&lt;td&gt;18&lt;/td&gt;&#xA;&lt;td&gt;17&lt;/td&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;19&lt;/td&gt;&#xA;&lt;td&gt;21&lt;/td&gt;&#xA;&lt;td&gt;20&lt;/td&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;22&lt;/td&gt;&#xA;&lt;td&gt;24&lt;/td&gt;&#xA;&lt;td&gt;23&lt;/td&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;25&lt;/td&gt;&#xA;&lt;td&gt;27&lt;/td&gt;&#xA;&lt;td&gt;26&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;I ran &lt;code&gt;bedtools map -a cgis.bed -b s.bed -c 4 -o sum&lt;/code&gt;, but when a CpG falls&#xA;across two regions, bedtools counts it in both. Here the 5 - 10 intervals should have only the second row of the CpG table: a sum of 2. However, since the second row spans 5, the 4 - 6 interval falls into both the 0 - 5 and the 5 - 10.&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;chr&lt;/th&gt;&#xA;&lt;th&gt;start&lt;/th&gt;&#xA;&lt;th&gt;end&lt;/th&gt;&#xA;&lt;th&gt;bedtools sum coverage&lt;/th&gt;&#xA;&lt;th&gt;correct sum coverage&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;5&lt;/td&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;5&lt;/td&gt;&#xA;&lt;td&gt;10&lt;/td&gt;&#xA;&lt;td&gt;3&lt;/td&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;10&lt;/td&gt;&#xA;&lt;td&gt;15&lt;/td&gt;&#xA;&lt;td&gt;3&lt;/td&gt;&#xA;&lt;td&gt;3&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;15&lt;/td&gt;&#xA;&lt;td&gt;20&lt;/td&gt;&#xA;&lt;td&gt;4&lt;/td&gt;&#xA;&lt;td&gt;4&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;20&lt;/td&gt;&#xA;&lt;td&gt;25&lt;/td&gt;&#xA;&lt;td&gt;4&lt;/td&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;chr1&lt;/td&gt;&#xA;&lt;td&gt;25&lt;/td&gt;&#xA;&lt;td&gt;30&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;Can I use bedtools to place it in the lower interval only and not both? Intersects should be start inclusive, but end exclusive. This does work as I'd like if I reduce every end column element in the second CpG table by one.&lt;/p&gt;&#xA;" OwnerUserId="15855" LastEditorUserId="15855" LastEditDate="2023-08-26T17:23:11.100" LastActivityDate="2023-08-26T17:23:11.100" Title="`bedtools`: Intersecting/map into only one interval when there are multiple matches?" Tags="|bedtools|interval|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21371" PostTypeId="1" CreationDate="2023-07-29T18:12:08.140" Score="1" ViewCount="113" Body="&lt;p&gt;I have managed to install IDR (&lt;a href=&quot;https://github.com/nboley/idr&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nboley/idr&lt;/a&gt;) correctly (idr --version returns 2.0.3), but I can't get it to run on my peak files (*.bed) without crashing.&lt;/p&gt;&#xA;&lt;p&gt;My peak files are generated using HOMER (findPeaks) and they are converted to proper *.bed format via HOMER/pos2bed.pl.  But when I run idr on my *.bed peak files as described in the IDR guide page like below:&lt;/p&gt;&#xA;&lt;p&gt;idr --samples peak1.bed peak2.bed&lt;/p&gt;&#xA;&lt;p&gt;it always returns an error inside the function bed_load() :&#xA;signal = float(data[signal_index])&#xA;IndexError: list index out of range&lt;/p&gt;&#xA;&lt;p&gt;I suspect it's related to peak files (likely columns) not being in the format the program idr.py is expecting, and I couldn't find much about the required &amp;quot;input&amp;quot; file format in the idr help pages.&lt;/p&gt;&#xA;&lt;p&gt;Can someone provide examples of an idr.py friendly peak file, maybe just a couple of rows with the proper/expected column types and numbers? Thank you in advance.&lt;/p&gt;&#xA;&lt;p&gt;I am including the guesstimate guidance from ChatGPT, which is not necessarily correct, but maybe a pointer in the right direction - it's suggesting the IDR format has to be consistent with narrowPeak formats having 10 columns, and the actual column that IDR uses in it's analysis is column 7 - which it calls it the &amp;quot;signal value&amp;quot;. But what exactly is this measure (the peak strength, the peak height etc.)? Any insight would be appreciated? Also, on where this type of information may have been discussed, seeing how critical it is to running the tool successfully.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RrQ3U.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RrQ3U.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The IDR readme on usage describes very clearly what the OUTPUT format is, but is not clear what the INPUT format is (column descriptions) - which is what one needs to successfully form the INPUT and run IDR. My typical HOMER/pos2bed.pl converted row is less than 8 columns which at this point, I am certain is the cause of IDR failures. I am attempting to figure out which column IDR actually uses, and which are not essential for it's considerations (they just get duplicated in the output).&lt;/p&gt;&#xA;&lt;p&gt;Here are the error messages I get:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/nboley/idr/issues/41&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nboley/idr/issues/41&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15741" LastEditorUserId="73" LastEditDate="2023-07-30T20:43:38.953" LastActivityDate="2023-07-30T20:44:56.593" Title="What is the required input peak file format for IDR (Irreproducible Discovery Rate)" Tags="|chip-seq|atac-seq|peak-calling|macs2|homer|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21372" PostTypeId="2" ParentId="21371" CreationDate="2023-07-29T23:17:57.123" Score="2" Body="&lt;p&gt;As mentioned in the IDR readme on &lt;a href=&quot;https://github.com/nboley/idr#usage&quot; rel=&quot;nofollow noreferrer&quot;&gt;usage&lt;/a&gt;, there are examples in the IDR github repository &lt;a href=&quot;https://github.com/nboley/idr/tree/master/tests/data&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. Here's one of them:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr16   31085416    31086009    .   0   .   362.546677850757    -1  4.12368834166759    250&#xA;chr12   109490129   109490582   .   0   .   360.329496957805    -1  4.12368834166759    215&#xA;chr13   41345146    41345561    .   0   .   358.624215585062    -1  4.12368834166759    207&#xA;chr19   50083722    50084001    .   0   .   266.603941224671    -1  4.5777559727315 186&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From the looks of it, it seems that the output is a line-filtered version of the input format (i.e. output and input formats are identical).&lt;/p&gt;&#xA;&lt;p&gt;I have no idea how IDR works (never used it before), but the error you've added to your post suggests there's a range mismatch. My guess based on that error message is that the two input files don't have overlapping specified regions.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-07-30T20:44:56.593" LastActivityDate="2023-07-30T20:44:56.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21373" PostTypeId="1" CreationDate="2023-07-30T13:56:35.640" Score="1" ViewCount="232" Body="&lt;p&gt;I am using Autodock and I'm getting an error in the step where we get the glg file (i.e. Run--&amp;gt; Autogrid) I am not sure what should I do...Please help me if someone knows how to tackle this. Plus yep there's no space in the folder file name and I have all exe and dat files in the same one folder.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;                         &lt;span class=&quot;math-container&quot;&gt;$Revision: 1.109 $&lt;/span&gt;&#xA;&#xA;Compilation parameters:  NUM_RECEPTOR_TYPES=20 NEINT=2048&#xA;&#xA;MAX_MAPS=16 NDIEL=16384 MAX_ATOM_TYPES=14&#xA;&#xA;energy_lookup table has   655360 entries of size 8&#xA;&#xA;e_vdW_Hb table has   401408 entries of size 8&#xA;&#xA;Maximum number of maps that can be computed = 16 (defined by MAXMAPS in &amp;quot;autocomm.h&amp;quot;).&#xA;&#xA;GPF&amp;gt; npts 18 12 24                        # num.grid points in xyz&#xA;&#xA;Number of grid points in x-direction:    19&#xA;&#xA;Number of grid points in y-direction:    13&#xA;&#xA;Number of grid points in z-direction:    25&#xA;&#xA;GPF&amp;gt; gridfld 98 Alanine_over_Phenylalanine.maps.fld # grid_data_file&#xA;&#xA;C:/Bioinformatics/Docking/autogrid4.exe: ERROR:  Grid data file needs the extension &amp;quot;.fld&amp;quot; for AVS input&#xA;&#xA;C:/Bioinformatics/Docking/autogrid4.exe: Unsuccessful Completion.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xERnA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xERnA.png&quot; alt=&quot;my gpf file which I used to generate further glg step, do you think there's something wrong in it?&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17915" LastEditorUserId="3694" LastEditDate="2023-07-30T18:10:40.490" LastActivityDate="2023-07-30T18:10:40.490" Title="Errors in AutoDock" Tags="|docking|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21374" PostTypeId="1" CreationDate="2023-07-30T22:14:31.427" Score="0" ViewCount="14" Body="&lt;p&gt;I have access to pre-processed VCF files from a genetic dataset, and am trying to use the MoChA tool to calculate mosaic chromosomal alterations in this dataset. However, MoChA requires BAF and log2ratio values for each of the variants, which are missing from the VCF files given, and the original genome array CEL files are unavailable so I cannot redo the variant calling and include these statistics&lt;/p&gt;&#xA;&lt;p&gt;Instead, they have provided us with nexus format CNV files (from the &lt;a href=&quot;https://www.thermofisher.com/ca/en/home/life-science/microarray-analysis/microarray-analysis-instruments-software-services/microarray-analysis-software/axiom-cnv-summary-tools-software.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Axiom CNV summary tools&lt;/a&gt;) for each of the samples with a host of log2 ratio and BAF files across the different Axiom probes.&lt;/p&gt;&#xA;&lt;p&gt;I am just a bit confused about how we can use these files to import the BAF and log2ratio values into the VCF for MoChA. Any suggestions?&lt;/p&gt;&#xA;&lt;p&gt;A couple sample rows of the CNV file are included here (I've removed the sample identifier and replaced with ---).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Sample  ProbeSetName    Chromosome      Position        Log2Ratio       BAF&#xA;---     AFFX-KIT-000001 9       101258881       -0.440535940556599      0.482727669963674&#xA;---     AFFX-KIT-000002 4       164934874       -0.106227164241716      0.057035393165642&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17917" LastEditorUserId="17917" LastEditDate="2023-07-30T22:18:07.850" LastActivityDate="2023-07-30T22:18:07.850" Title="Adding BAF and Log2 ratio to VCF file" Tags="|vcf|microarray|cnv|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21375" PostTypeId="1" CreationDate="2023-08-01T06:11:07.293" Score="-1" ViewCount="71" Body="&lt;p&gt;I am trying to calculate the N50 value from the assembled FASTA file. I used stats.sh from the BBmap package. I executed the following command&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./stats.sh  in=/Home/Downloads/BBMap_39.01/bbmap/resources/contig.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I got the following error.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Exception in thread &amp;quot;main&amp;quot; java.lang.RuntimeException: Input file does not appear to be valid: /Home/Downloads/BBMap_39.01/bbmap/resources/contig.fasta&#xA;    at jgi.AssemblyStats2.process(AssemblyStats2.java:248)&#xA;    at jgi.AssemblyStats2.main(AssemblyStats2.java:40)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How to resolve this error?&lt;/p&gt;&#xA;&lt;p&gt;The sequences are shown below. My file also contains some lengthy sequences.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;NODE_1_length_3013_cov_4.540906&#xA;TGGGGGATCATGTAACTCGCCTTGATCGTTGGGAACCGGAGCTGAATGAAGCCATACCAA&#xA;ACGACGAGCGTGACACCACGATGCCTGTAGCAATGGCAACAACGTTGCGCAAACTATTAA&#xA;CTGGCGAACTACTTACTCTAGCTTCCCGGCAACAATTAATAGACTGGATGGAGGCGGATA&#xA;AAGTTGCAGGACCACTTCTGCGCTCGGCCCTTCCGGCTGGCTGGTTTATTGCTGATAAAT&#xA;CTGGAGCCGGTGAGCGTGGGTCTCGCGGTATCATTGCAGCACTGGGGCCAGATGGTAAGC&#xA;CCTCCCGTATCGTAGTTATCTACACGACGGGGAGTCAGGCAACTATGGATGAACGAAATA&#xA;GACAGATCGCTGAGATAGGTGCCTCACTGATTAAGCATTGGTAACTGTCAGACCAAGTTT&#xA;&amp;gt;NODE_2_length_2292_cov_16.637908&#xA;CAAAGGAATAATAATTGACTGATGATTGGTTCATTAATTACTTCATTAGTTAATAGATTG&#xA;CAGTCTTTATAAATAAGTTTATAGTTGTATTCCACAGAATTTCTGATCCTGTCTCTGCCC&#xA;TACCCAATGTTTTATCATTAAACTTCATTTACATAAGATAGTATGTTCAACAAATATAAA&#xA;GATTTGCTGGAATGGAAGGCACAGTAAGATATTTCACAGGATAAAATTAAAGTAGATGTC&#xA;AACAGAGTAAATCAAAGTGAAATTTAATAGAGTTAAACAGTAAGTGTTTTAAATACCAGG&#xA;ATTAAATTCCATGAAGCTGCAGCTTTTAAAAAAAGGCACAAAGTGGAATAAGAAAAAATG&#xA;ACTTTCAGAAAGTTAATGTAAAGAAAGCTCACGGTTTTCATTGGCTGCCTGCTCAGTTTG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17923" LastEditorUserId="3848" LastEditDate="2023-08-02T12:14:16.163" LastActivityDate="2023-08-02T12:14:16.163" Title="stats.sh error in BBmap package" Tags="|fasta|sequence-analysis|linux|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21377" PostTypeId="1" CreationDate="2023-08-01T16:20:22.323" Score="1" ViewCount="122" Body="&lt;p&gt;Im trying to follow this pipeline &lt;a href=&quot;https://github.com/epifluidlab/cragr&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/epifluidlab/cragr&lt;/a&gt; on cfDNA WGS. I have bam files from paired end reads aligned with bwa mem.&lt;/p&gt;&#xA;&lt;p&gt;The workflow requires bgzipped and indexed fragment data files as input.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/CJ4Gt.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/CJ4Gt.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have been searching how can I obtain BED files from fragments coordinates instead of reads coordinates, and came up with no answer.&lt;/p&gt;&#xA;&lt;p&gt;Does someone know how can I obtain such bed file?&lt;/p&gt;&#xA;" OwnerUserId="17927" LastEditorUserId="17927" LastEditDate="2023-08-01T16:21:33.673" LastActivityDate="2023-08-02T04:29:38.607" Title="obtaining a fragment data file bed from bam" Tags="|sam|bed|bedtools|wgs|epigenetics|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21378" PostTypeId="1" CreationDate="2023-08-01T18:17:30.613" Score="0" ViewCount="107" Body="&lt;p&gt;I am getting the following error when I run FragPipe on my Ubuntu machine 22.04 how do I fix this?&#xA;&lt;a href=&quot;https://i.stack.imgur.com/X0i7W.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/X0i7W.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="14059" LastActivityDate="2023-08-01T18:17:30.613" Title="running fragpipe" Tags="|genomics|proteomics|systems-biology|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21379" PostTypeId="2" ParentId="21377" CreationDate="2023-08-02T04:29:38.607" Score="2" Body="&lt;p&gt;There's an example in the README (&lt;a href=&quot;https://github.com/epifluidlab/cragr#example-datasets&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/epifluidlab/cragr#example-datasets&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For you information, below are steps from the BAM file to the&#xA;fragment file:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;samtools sort -n -o qsorted.bam -@ 4 sample.bam&#xA;samtools view -h -f 3 -F 3852 -G 48 --incl-flags 48 qsorted.bam |\&#xA;  bamToBed -bedpe -mate1 -i stdin |\&#xA;  awk -F'\t' -v OFS=&amp;quot;\t&amp;quot; '{if ($1!=$4) next; if ($9==&amp;quot;+&amp;quot;) {s=$2;e=$6} else {s=$5;e=$3} if (e&amp;gt;s) print $1,s,e,$7,$8,$9}' |\&#xA;  sort -k1,1V -k2,2n |\&#xA;  bgzip &amp;gt; sample.frag.bed.gz&#xA;tabix -p bed sample.frag.bed.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9098" LastActivityDate="2023-08-02T04:29:38.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21380" PostTypeId="2" ParentId="21375" CreationDate="2023-08-02T07:38:26.680" Score="1" Body="&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;./stats.sh in=notExistingFile.fa&#xA;Exception in thread &amp;quot;main&amp;quot; java.lang.RuntimeException: Input file does not appear to be valid: notExistingFile.fa&#xA;        at jgi.AssemblyStats2.process(AssemblyStats2.java:248)&#xA;        at jgi.AssemblyStats2.main(AssemblyStats2.java:40)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The error comes when the file does not exist. My best guess is that it is a typo on your side. Is the path really &lt;code&gt;/Home&lt;/code&gt; with capital &amp;quot;H&amp;quot;? Most OS like Linux and macOS use lowercase. Anyway, make sure the file exists.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-08-02T07:38:26.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21382" PostTypeId="1" AcceptedAnswerId="21400" CreationDate="2023-08-02T13:01:10.373" Score="3" ViewCount="98" Body="&lt;p&gt;I want to do mapping with bwa mem. I downloaded a reference genome from ncbi database which is (Danaus plexippus). I got one &lt;code&gt;ncbi_database.zip&lt;/code&gt; file which contain 2 folders and 3 files of doc,assembly_report_json file. I read one file named &lt;code&gt;read.md&lt;/code&gt; which came after downloading but its not helping. What should be done in this case?&lt;/p&gt;&#xA;&lt;p&gt;I don't have an issue with using bwa-mem; the problem is with the reference genome from NCBI. I downloaded it from &lt;a href=&quot;https://ncbi.nlm.nih.gov/datasets/genome/GCF_009731565.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ncbi.nlm.nih.gov/datasets/genome/GCF_009731565.1&lt;/a&gt; but after that I checked the downloaded file and I got 5 folders into it and I'm confused which one to use. 2 files have &lt;code&gt;.fna&lt;/code&gt; extension and the rest are reports that are in the json file format.&lt;/p&gt;&#xA;" OwnerUserId="17934" LastEditorUserId="650" LastEditDate="2023-08-04T15:06:37.293" LastActivityDate="2023-08-07T05:41:05.270" Title="Reference genome (bwa -mem)" Tags="|read-mapping|reference-genome|bwa|ncbi|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21383" PostTypeId="2" ParentId="21382" CreationDate="2023-08-02T20:03:15.353" Score="1" Body="&lt;p&gt;What you need is the genome sequence in fasta format, typically suffixed &lt;code&gt;.fa&lt;/code&gt;, for example from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets/genome/GCF_009731565.1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/datasets/genome/GCF_009731565.1/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This file needs to be indexed via &lt;code&gt;bwa index&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-08-02T20:03:15.353" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21384" PostTypeId="1" AcceptedAnswerId="21386" CreationDate="2023-08-03T06:42:17.753" Score="3" ViewCount="55" Body="&lt;p&gt;What does this command implies in bwa mem&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bwa mem -t 8 index/referencegenome.fna read1.fastq.gz read2.fastq.gz &amp;gt; output.sam&amp;quot;&amp;quot;&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What does &lt;code&gt;-t 8&lt;/code&gt; stands for here?&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="17934" LastEditorUserId="650" LastEditDate="2023-08-04T15:05:09.430" LastActivityDate="2023-08-04T15:05:09.430" Title="what does 8 stands for in bwa mem command line" Tags="|bwa|command-line|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21385" PostTypeId="2" ParentId="21384" CreationDate="2023-08-03T08:07:45.620" Score="1" Body="&lt;p&gt;When you have questions on software then the first thing is to do your due diligence and read the manual:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bio-bwa.sourceforge.net/bwa.shtml&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bio-bwa.sourceforge.net/bwa.shtml&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then simply checking for the &amp;quot;-t&amp;quot; argument will tell you that it is the number of threads used for the alignment process.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-08-03T08:07:45.620" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21386" PostTypeId="2" ParentId="21384" CreationDate="2023-08-03T09:26:55.963" Score="0" Body="&lt;p&gt;&lt;code&gt;-t&lt;/code&gt; is an important parameter here and is important to understand. bwa is a CPU calculation and the more CPUs the job (calculation) is spread across the faster the calculation will take to complete and the less chance of the calculation exceeding the RAM capacity of the machine. &amp;quot;Exceeding RAM capacity&amp;quot; (I don't know the technical term), is when the process &amp;quot;grinds to a halt&amp;quot; and never completes.&lt;/p&gt;&#xA;&lt;p&gt;Basically what can happen is the indexing step instead of taking ~3 hours, could easily be still running after 24 hours if the machine has little RAM and &lt;code&gt;-t&lt;/code&gt; isn't set sufficiently high.&lt;/p&gt;&#xA;&lt;p&gt;One CPU is known as &amp;quot;one core&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;For a regular desktop if you set &lt;code&gt;-t X&lt;/code&gt; where X is number of CPUs - 1 in my opinion that is good practice. So if the machine has 8 CPUs then X = 7. If you are working on a cluster, e.g. GCP or AWS, then X is the maximum number of cores (CPUs) available.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-08-03T09:26:55.963" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21387" PostTypeId="1" AcceptedAnswerId="21480" CreationDate="2023-08-03T09:27:33.923" Score="3" ViewCount="158" Body="&lt;p&gt;After aligning paired-end reads to a reference genome, I am getting low percentage:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;34636166 + 0 in total (QC-passed reads + QC-failed reads)&#xA;34346076 + 0 primary&#xA;0 + 0 secondary&#xA;290090 + 0 supplementary&#xA;0 + 0 duplicates&#xA;0 + 0 primary duplicates&#xA;5706057 + 0 mapped (16.47% : N/A)&#xA;5415967 + 0 primary mapped (15.77% : N/A)&#xA;34346076 + 0 paired in sequencing&#xA;17173038 + 0 read1&#xA;17173038 + 0 read2&#xA;3807784 + 0 properly paired (11.09% : N/A)&#xA;3996748 + 0 with itself and mate mapped&#xA;1419219 + 0 singletons (4.13% : N/A)&#xA;177284 + 0 with mate mapped to a different chr&#xA;65388 + 0 with mate mapped to a different chr (mapQ&amp;gt;=5)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What has to be done to increase the mapping percentage?&lt;/p&gt;&#xA;&lt;p&gt;I'm attaching the fastqc report before trimming and after trimming too&#xA;first-  R1 without trimming&#xA;second- R2 without trimming&lt;/p&gt;&#xA;&lt;p&gt;third- forward paired&#xA;fourth- reverse paried&lt;/p&gt;&#xA;&lt;p&gt;rest are overrepresentations&lt;/p&gt;&#xA;&lt;p&gt;After using trimmomatic the quality is getting more bad and even the mapping is of low percentage&lt;/p&gt;&#xA;&lt;p&gt;Mash screen result in the last image&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/hoxQE.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hoxQE.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wFmzf.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wFmzf.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Lfk3U.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Lfk3U.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/BRqka.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BRqka.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ajBWX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ajBWX.png&quot; alt=&quot;&#xA;&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/U6wRS.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/U6wRS.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/A1pU5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/A1pU5.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17934" LastEditorUserId="17934" LastEditDate="2023-08-17T04:44:23.320" LastActivityDate="2023-08-23T21:58:17.010" Title="low mapping percentage in bwa mem" Tags="|sam|samtools|reference-genome|bwa|" AnswerCount="1" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="21388" PostTypeId="2" ParentId="411" CreationDate="2023-08-03T10:26:47.190" Score="1" Body="&lt;p&gt;I recommend using &lt;a href=&quot;https://github.com/rpetit3/fastq-dl&quot; rel=&quot;nofollow noreferrer&quot;&gt;fastq-dl&lt;/a&gt;: downloads from either ENA or SRA, multithreaded, automatic retries.&lt;/p&gt;&#xA;" OwnerUserId="15257" LastActivityDate="2023-08-03T10:26:47.190" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21389" PostTypeId="1" AcceptedAnswerId="21390" CreationDate="2023-08-03T11:58:01.763" Score="3" ViewCount="127" Body="&lt;p&gt;I am using bedtools getfasta to convert genomic coordinates to sequences. I feed it a bedfile containing say chr1 s.start s.end. When I check the sequence output, it does not include the s.start base.&#xA;For example if the bedfile entry is chr1 23456 23678, it gets the sequence from 23457-23678. Is this a feature and I should use a tag to make it include the first base?&#xA;The command I am using is &lt;code&gt;bedtools getfasta -fi genome.fa -fo gene.fa -bed input.bed&lt;/code&gt;&#xA;I tried this in bedtools versions 2.27.1 and 2.30.1&lt;/p&gt;&#xA;" OwnerUserId="9089" LastEditorUserId="73" LastEditDate="2023-08-03T23:58:24.170" LastActivityDate="2023-08-04T06:38:16.237" Title="Why doesn't bedtools getfasta pick up the first base?" Tags="|genome|bed|bedtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21390" PostTypeId="2" ParentId="21389" CreationDate="2023-08-03T12:10:28.930" Score="3" Body="&lt;p&gt;BED files are 0-based. I link this old but gold &lt;a href=&quot;https://www.biostars.org/p/84686/&quot; rel=&quot;nofollow noreferrer&quot;&gt;biostars&lt;/a&gt; thread for more background. Essentially, from normal counting with start 1 you subtract 1 from the start coordinate.&lt;/p&gt;&#xA;&lt;p&gt;For example in the sequence &lt;code&gt;ATCGGG&lt;/code&gt; the &lt;code&gt;TCGGG&lt;/code&gt; substring would have &amp;quot;normal&amp;quot; (aka human-readable) coordinates 2-6, but in BED format that would be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chrSomething    1    6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here is the bedtools &amp;quot;proof&amp;quot;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$ cat foo.bed&#xA;chrSomething    1       6&#xA;&#xA;$ cat foo.fa&#xA;&amp;gt;chrSomething&#xA;ATCGGG&#xA;&#xA;$ bedtools getfasta -fi foo.fa -bed foo.bed&#xA;&amp;gt;chrSomething:1-6&#xA;TCGGG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16676" LastEditorUserId="16676" LastEditDate="2023-08-04T06:38:16.237" LastActivityDate="2023-08-04T06:38:16.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21391" PostTypeId="1" CreationDate="2023-08-03T13:56:53.533" Score="0" ViewCount="38" Body="&lt;p&gt;I have a set of real data containing &lt;strong&gt;labels&lt;/strong&gt;. However, in some biclustering algorithms (e.g., &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/10977070/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Cheng and Church Algorithm&lt;/a&gt;), originally authors applied gene expression data like &lt;a href=&quot;https://arep.med.harvard.edu/biclustering/yeast.matrix&quot; rel=&quot;nofollow noreferrer&quot;&gt;Yeast&lt;/a&gt; without having labels. My question is that what are appropriate &lt;em&gt;evaluation measures&lt;/em&gt; in this scenario?&lt;/p&gt;&#xA;" OwnerUserId="17940" LastActivityDate="2024-01-01T15:02:20.407" Title="How evaluate biclustering algorithm over real datasets containing labels?" Tags="|gene-expression|machine-learning|algorithms|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21392" PostTypeId="1" CreationDate="2023-08-03T16:58:38.673" Score="1" ViewCount="43" Body="&lt;p&gt;&lt;em&gt;Apologies if the question is out of the scope of the bioinformatics SE. Please feel free to close it if it is the case.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Recently, I have been going into a rabbit hole of papers that discuss the progress of sequencing technologies, from the earliest attempts to Sanger, massively parallel sequencing, and long-read sequencing.&lt;/p&gt;&#xA;&lt;p&gt;To my surprise, I was not able to find any recent reference that direclty highlights Illumina's market dominance. The closest I could find was the &lt;a href=&quot;https://academic.oup.com/nar/article/39/suppl_1/D19/2505848&quot; rel=&quot;nofollow noreferrer&quot;&gt;2010 SRA paper&lt;/a&gt; (&amp;quot;Almost 80% [...]&amp;quot;) that briefly describes the proportion of records per platform or &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/gb4168&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; that alludes to Illumina's monopoly. How come it seems so difficult to find recent reports on the use of each sequencing platform?&lt;/p&gt;&#xA;&lt;p&gt;Am I missing something there, or am I just looking for something that is not that interesting? To be fair, it's probably easier to query the SRA directly to get my answer.&lt;/p&gt;&#xA;" OwnerUserId="17941" LastEditorUserId="17941" LastEditDate="2023-08-03T16:59:52.800" LastActivityDate="2023-08-03T16:59:52.800" Title="Journal article reference for Illumina's &quot;near monopoly&quot;" Tags="|illumina|sra|" AnswerCount="0" CommentCount="3" ClosedDate="2023-08-05T13:35:13.177" ContentLicense="CC BY-SA 4.0" />
  <row Id="21393" PostTypeId="1" CreationDate="2023-08-03T22:47:59.480" Score="1" ViewCount="69" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.reddit.com/r/bioinformatics/comments/15hikqy/what_are_the_output_files_of_rnaseq_from_facility/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Reddit&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am new in our lab and I am going to do bulk RNA-Seq. What type of files will we get from the company (Genewiz)? Will it be a bunch of Fastq files? or they give something like bam files?&lt;/p&gt;&#xA;" OwnerUserId="17943" LastEditorUserId="73" LastEditDate="2023-08-04T06:52:55.603" LastActivityDate="2024-01-01T08:09:01.027" Title="What are the output files of RNA-Seq from facility?" Tags="|rna-seq|ngs|fastq|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21394" PostTypeId="2" ParentId="21393" CreationDate="2023-08-04T06:37:04.810" Score="1" Body="&lt;p&gt;This cannot be predicted unless you have interacted with them previously. Most commonly a sequencing facility will send raw data as gzipped fastq files, often together with md5 checksums so the user can check file integrity after transfer/download from them.&lt;/p&gt;&#xA;&lt;p&gt;We worked with a facility before who sent data in unaligned bam format, which then for typical workflows first had to be converted back to fastq.&lt;/p&gt;&#xA;&lt;p&gt;After all, you're a paying customer so feel free to shoot them an email and ask whatever question comes up. Ask what exactly they will provide so that you can prepare your workflows accordingly. Most likely it will be fastq.gz.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-08-04T06:37:04.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21395" PostTypeId="2" ParentId="21391" CreationDate="2023-08-04T11:49:36.477" Score="0" Body="&lt;p&gt;Briefly evaluation measures, which I presume means &amp;quot;features&amp;quot;, are determined using supervised learning using a technique called &amp;quot;feature selection&amp;quot;. Unsupervised learning generally is used to identify categories for a supervised learning algorithm. The orthodoxy has changed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&amp;quot;Unsupervised learning feature selection&amp;quot;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;To identify suitable features using unsupervised learning &lt;em&gt;a priori&lt;/em&gt; then genome location which must have a major outcome, thereafter it would be a case of deleting a given feature and re-evaluating the clusters. If the clusters disintegrate then that is a key feature (not very cool but would work for datasets with limited numbers of features).&lt;/li&gt;&#xA;&lt;li&gt;Weighted unsupervised learning is popular, this is basically &amp;quot;feature selection&amp;quot;, i.e. which features contribute to the clustering. This is basically the formal version of point 1. Personally I have had little success with this approach, i.e. the results were not informative, but it does work. The results are not discriminatory between different conditions, which is what I was seeking.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Regarding labels, labels are not used in unsupervised learning approach - except weighted unsupervised learning - hence their presence or absence usually makes little difference. The only &amp;quot;label&amp;quot; used is the sample type, i.e. the &amp;quot;id&amp;quot; and not the &amp;quot;feature&amp;quot;. Without intrinsically understanding the experiment it is difficult to forward an accurate &lt;em&gt;a priori&lt;/em&gt; assessment.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-08-04T13:33:23.890" LastActivityDate="2023-08-04T13:33:23.890" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21396" PostTypeId="1" AcceptedAnswerId="21397" CreationDate="2023-08-05T12:53:02.253" Score="3" ViewCount="335" Body="&lt;p&gt;I have a very basic objective which I want to give list of id to a ncbi-srafastq tool kit which will prefetch the id which is in .sra file extension and then it will run fastq-dump on the file to split the files into fastq files, single file if there is single end file or paired end file if its paired end or two set of files.&lt;/p&gt;&#xA;&lt;p&gt;This is my code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;input=$1&#xA;&#xA;a=&lt;span class=&quot;math-container&quot;&gt;$(basename &quot;$&lt;/span&gt;input&amp;quot;)&#xA;id=${a/.id/}&#xA;&#xA;# Download SRA data&#xA;prefetch &amp;quot;$id&amp;quot;&#xA;&#xA;# Extract SRA ID from the downloaded data&#xA;#sraID=&lt;span class=&quot;math-container&quot;&gt;$(ls -1 | grep -v run.sh)&#xA;sraID=$&lt;/span&gt;(ls -1 | grep -v run.sh | sed 's/\.sra&lt;span class=&quot;math-container&quot;&gt;$//')&#xA;# Validate the downloaded SRA data&#xA;vdb-validate &quot;$&lt;/span&gt;sraID&amp;quot;&#xA;&#xA;# Dump fastq data from the SRA file&#xA;fastq-dump &amp;quot;$sraID&amp;quot;  --gzip&#xA;&#xA;# Concatenate fastq files for read1, read2, and unpaired reads (if available)&#xA;for s in &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$sraID&quot;&#xA;do&#xA;  echo &quot;$&lt;/span&gt;s&amp;quot;&#xA;  if [ &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$s&quot; != &quot;$&lt;/span&gt;id&amp;quot; ]&#xA;  then&#xA;    [ -f &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${s}_1.fastq&quot; ] &amp;amp;&amp;amp; cat &quot;$&lt;/span&gt;{s}_1.fastq&amp;quot; &amp;gt;&amp;gt; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${id}_R1.fastq&quot;&#xA;    [ -f &quot;$&lt;/span&gt;{s}_2.fastq&amp;quot; ] &amp;amp;&amp;amp; cat &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${s}_2.fastq&quot; &amp;gt;&amp;gt; &quot;$&lt;/span&gt;{id}_R2.fastq&amp;quot;&#xA;    [ -f &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${s}.fastq&quot; ] &amp;amp;&amp;amp; cat &quot;$&lt;/span&gt;{s}.fastq&amp;quot; &amp;gt;&amp;gt; &amp;quot;${id}.fastq&amp;quot;&#xA;  fi&#xA;done&#xA;&#xA;# Compress fastq files using pigz&#xA;#for i in &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${id}&quot;*.fastq&#xA;#do&#xA;  #echo &quot;$&lt;/span&gt;i&amp;quot;&#xA; # pigz &amp;quot;$i&amp;quot; &amp;amp; sleep 1&#xA;#done&#xA;#wait&#xA;&#xA;# Remove the original uncompressed fastq file if both read1 and read2 files are present&#xA;[ -f &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${id}_R1.fastq.gz&quot; ] &amp;amp;&amp;amp; [ -f &quot;$&lt;/span&gt;{id}_R2.fastq.gz&amp;quot; ] &amp;amp;&amp;amp; [ -f &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${id}.fastq.gz&quot; ] &amp;amp;&amp;amp; rm &quot;$&lt;/span&gt;{id}.fastq.gz&amp;quot;&#xA;&#xA;# Test deleting intermediate folder&#xA;echo &amp;quot;Space before tmp folder delete&amp;quot;&#xA;echo &amp;quot;Removing tmp folder &lt;span class=&quot;math-container&quot;&gt;$sraID &quot;&#xA;ls *&#xA;rm -rf &quot;$&lt;/span&gt;sraID&amp;quot;*&#xA;ls *&#xA;echo 'done'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I run using as small example file as such&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./run.sh SRR12332899&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The error I get is this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;2023-08-05T12:37:32 prefetch.3.0.6: Current preference is set to retrieve SRA Normalized Format files with full base quality scores.&#xA;2023-08-05T12:37:33 prefetch.3.0.6: 1) Downloading 'SRR12332899'...&#xA;2023-08-05T12:37:33 prefetch.3.0.6: SRA Normalized Format file is being retrieved, if this is different from your preference, it may be due to current file availability.&#xA;2023-08-05T12:37:33 prefetch.3.0.6:  Downloading via HTTPS...&#xA;2023-08-05T12:40:48 prefetch.3.0.6:  HTTPS download succeed&#xA;2023-08-05T12:40:48 prefetch.3.0.6:  'SRR12332899' is valid&#xA;2023-08-05T12:40:48 prefetch.3.0.6: 1) 'SRR12332899' was downloaded successfully&#xA;2023-08-05T12:40:48 prefetch.3.0.6: 'SRR12332899' has 0 unresolved dependencies&#xA;2023-08-05T12:40:48 vdb-validate.3.0.6 info: Database 'SRR12332899.sra' metadata: md5 ok&#xA;2023-08-05T12:40:48 vdb-validate.3.0.6 info: Table 'SEQUENCE' metadata: md5 ok&#xA;2023-08-05T12:40:48 vdb-validate.3.0.6 info: Column 'ALTREAD': checksums ok&#xA;2023-08-05T12:40:48 vdb-validate.3.0.6 info: Column 'QUALITY': checksums ok&#xA;2023-08-05T12:40:48 vdb-validate.3.0.6 info: Column 'READ': checksums ok&#xA;2023-08-05T12:40:48 vdb-validate.3.0.6 info: Column 'SPOT_GROUP': checksums ok&#xA;2023-08-05T12:40:48 vdb-validate.3.0.6 info: Database 'sra/SRR12332899.sra' contains only unaligned reads&#xA;2023-08-05T12:40:48 vdb-validate.3.0.6 info: Database 'SRR12332899.sra' is consistent&#xA;2023-08-05T12:40:49 fastq-dump.3.0.6 err: item not found while constructing within virtual database module - the path 'sra' cannot be opened as database or table&#xA;fastq-dump quit with error code 3&#xA;sra&#xA;Space before tmp folder delete&#xA;Removing tmp folder sra&#xA;run.sh&#xA;&#xA;sra:&#xA;SRR12332899.sra&#xA;run.sh&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If I do just&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;prefetch SRR12332899&lt;/code&gt; and then &lt;code&gt;fastq-dump SRR12332899&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I get a SRR12332899.fastq  SRR12332899.sra&lt;/p&gt;&#xA;&lt;p&gt;This is for one file since I want to download list of files using the script which is not working.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion would be really helpful&lt;/p&gt;&#xA;" OwnerUserId="14154" LastActivityDate="2023-08-05T21:40:09.380" Title="ncbi fastq dump error in loop" Tags="|fastq|shell|ncbi|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21397" PostTypeId="2" ParentId="21396" CreationDate="2023-08-05T21:40:09.380" Score="3" Body="&lt;p&gt;The main problem is that your script is inadvertently treating a set of file and directory names as a single SRA accession number, all in one variable.  You should update it to work with each accession separately and make sure they really are accessions.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;On this line you're taking all the names of things in your working directory (minus the script), each on a separate line, and trimming .sra off the end of each line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sraID=$(ls -1 | grep -v run.sh | sed 's/\.sra$//')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In your case that list was apparently just a directory named &amp;quot;sra&amp;quot;.  (Even though I don't have that directory on my end, you can see it in your output, from the first of two &lt;code&gt;ls *&lt;/code&gt; at the end, where you have &lt;code&gt;sra/SRR12332899.sra&lt;/code&gt;.)  So later on it tries to &lt;code&gt;fastq-dump sra&lt;/code&gt;, which gives you that &amp;quot;cannot be opened as database or table&amp;quot; error.  (&lt;code&gt;vdb-validate&lt;/code&gt; can take a directory as input, so it doesn't complain.)  If I run your script from scratch in an empty directory with SRR12332899, it actually seems to run OK, aside from removing everything it creates at the end.  But if you run it with a second accession you see the problem, since it then tries things like &lt;code&gt;vdb-validate &amp;quot;several accessions and filenames separated by newlines&amp;quot;&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I think a proper fix would involve a bunch of changes, but here are some tips, at least:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;use subdirectories to keep things organized so you don't have to juggle multiple types of file in one place&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;set -x&lt;/code&gt; at the start of a script will show you each command as it happens, which can be very helpful for troubleshooting&lt;/li&gt;&#xA;&lt;li&gt;be careful with quotes and whitespace in bash.  If you want to loop over each word in a variable you can do &lt;code&gt;for thing in $var; do ...; done&lt;/code&gt; (unquoted) but if you run into filenames with whitespace, it breaks them up.  Another pattern is &lt;code&gt;command | while read thing; do ...; done&lt;/code&gt; which will go line-by-line.  (In that case if you were working with a variable storing the command output, you might do something like &lt;code&gt;echo &amp;quot;$output&amp;quot; | while...&lt;/code&gt;, quoted that time, since you'd want to preserve the newlines.)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;ls *&lt;/code&gt; will expand each file/directory name as separate arguments (so bash really ends up calling &lt;code&gt;ls &amp;quot;sra&amp;quot; ...whatever else...&lt;/code&gt;), and when ls sees directory names is lists their contents too.  If you just want to show all the files/directories you can leave off the asterisk altogether, or if you wanted to give directory names but not have it list their contents, you can use &lt;code&gt;ls -d&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;you don't necessarily have to call &lt;code&gt;ls&lt;/code&gt; to store the names as a block of text.  You can loop over each filename, including those with spaces, directly in bash (e.g. &lt;code&gt;for thing in *; do echo &amp;quot;path: '$thing'&amp;quot;; done&lt;/code&gt;)&lt;/li&gt;&#xA;&lt;li&gt;be careful with the rm command, especially combined with wildcards, and especially with &lt;code&gt;-rf&lt;/code&gt;.  If &lt;code&gt;sraID&lt;/code&gt; happened to be empty somehow, think of what &lt;code&gt;rm -rf &amp;quot;$sraID&amp;quot;*&lt;/code&gt; would actually mean.  (And yes, scripts can remove themselves!)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2023-08-05T21:40:09.380" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21398" PostTypeId="1" AcceptedAnswerId="21399" CreationDate="2023-08-06T21:14:13.497" Score="0" ViewCount="84" Body="&lt;p&gt;I am using clusterProfiler for differential gene expression using this code:&#xA;&lt;a href=&quot;https://yulab-smu.top/biomedical-knowledge-mining-book/enrichplot.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://yulab-smu.top/biomedical-knowledge-mining-book/enrichplot.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have a question about the third line of code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; de &amp;lt;- names(geneList)[abs(geneList) &amp;gt; 2]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here all gene names are taken with absolute fold change &amp;gt;2. As an example, if in a hypothetical extreme case you have 100 downregulated genes (log2 fold change &amp;lt;-2) and zero upregulated genes (log2 fold change &amp;gt;2), then you may incorrectly think that identified pathways are activated, but they are actually repressed. How is this accounted for? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="7354" LastActivityDate="2023-08-07T03:02:23.450" Title="Interpretation of clusterProfiler results" Tags="|r|differential-expression|gene-expression|pathway|clusterprofiler|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21399" PostTypeId="2" ParentId="21398" CreationDate="2023-08-06T22:10:33.040" Score="1" Body="&lt;p&gt;The scenario you described would be reported by the code because:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;... [abs(geneList) &amp;gt; 2]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;abs&lt;/code&gt; will change any negative value to positive and retain all positive values as positive. Therefore values &amp;lt;-2 will be reported as well as values &amp;gt;2. &lt;code&gt;abs&lt;/code&gt; = absolute and to take the absolute value of a number is exactly the operation just described.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;From comments. Just remove &lt;code&gt;abs&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;negative &amp;lt;- names(geneList)[geneList &amp;lt; -2]&#xA;positive &amp;lt;- names(geneList)[geneList &amp;gt; 2]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-08-07T03:02:23.450" LastActivityDate="2023-08-07T03:02:23.450" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21400" PostTypeId="2" ParentId="21382" CreationDate="2023-08-07T05:41:05.270" Score="0" Body="&lt;p&gt;I downloaded genome from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets/genome/GCF_009731565.1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/datasets/genome/GCF_009731565.1/&lt;/a&gt; I got 5 files in it so if you are confused which one to use in that so GCF.fna having genomic data with fna extension is used as an reference genome if you are also new to this things this might be useful.&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="17934" LastActivityDate="2023-08-07T05:41:05.270" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21401" PostTypeId="1" CreationDate="2023-08-07T08:30:37.883" Score="0" ViewCount="34" Body="&lt;p&gt;While generating the predictions using spliceAI from vep, I am not getting any values.&lt;/p&gt;&#xA;&lt;p&gt;I used this command to generate the output,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;vep \&#xA;  -i input.vcf \&#xA;  -o output.vcf \&#xA;  --cache \&#xA;  --force_overwrite \&#xA;  --tab \&#xA;  --no_check_variants_order \&#xA;  --force \&#xA;  --plugin SpliceAI,snv=spliceai_scores.raw.snv.hg38.vcf.gz,indel=spliceai_scores.raw.indel.hg38.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Below, are the examples of entries in the output file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ENSEMBL VARIANT EFFECT PREDICTOR v109.3  &#xA;Output produced at 2023-08-07 09:05:43  &#xA;Connected to homo_sapiens_core_109_38 on ensembldb.ensembl.org  &#xA;Using cache in /.vep/homo_sapiens/109_GRCh38  &#xA;Using API version 109, DB version 109  &#xA;ensembl-io version 109.4946a86  &#xA;ensembl version 109.10baaec  &#xA;ensembl-funcgen version 109.cba2db8  &#xA;ensembl-variation version 109.18a12b6  &#xA;...  &#xA;Uploaded_variation Location Allele Gene Feature Feature_type Consequence cDNA_position CDS_position Protein_position Amino_acids Codons Existing_variation IMPACT DISTANCE STRAND FLAGS SpliceAI_pred  &#xA;100017682 chr1:100017682 A ENSG00000117620 ENST00000370153 Transcript missense_variant,splice_region_variant 981 880 294 A/T Gca/Aca - MODERATE - -     &#xA;100017682 chr1:100017682 A ENSG00000117620 ENST00000370155 Transcript splice_region_variant,3_prime_UTR_variant,NMD_transcript_variant 882 - - - - LOW - 1 - -&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is there any wrong in the input file? Could anyone help here?&lt;/p&gt;&#xA;" OwnerUserId="17962" LastEditorUserId="650" LastEditDate="2023-08-07T14:27:17.593" LastActivityDate="2023-08-07T14:27:17.593" Title="spliceAI not giving output value while running using vep (Variant Ensemble Predictor)" Tags="|variants|rna-splicing|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21402" PostTypeId="1" CreationDate="2023-08-07T08:52:09.130" Score="-1" ViewCount="167" Body="&lt;p&gt;I'm completely new to scRNA-seq analysis (and to most of Bioinformatics as well) so I apologize if this was asked a million times before.&lt;/p&gt;&#xA;&lt;p&gt;I am trying to get the single cell expression data of lymph node and bone marrow from &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE159929&quot; rel=&quot;nofollow noreferrer&quot;&gt;this GEO entry&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In Samples, there are multiple organs and accessing any of them gets me a CSV file compressed in gz format. The CSV file appears to contain the barcodes and the counts, but I do not know where to go from there. I need to annotate each cell type and I am not sure how. Seurat tutorials indicate that I have to have a few other file types which do not seem to be available in this repository. Does this mean I cannot use it, or is there a way to get this data to count/cell type just using what's available? I imagine that I don't need to perform any alignment since the rows already contain gene names.&lt;/p&gt;&#xA;&lt;p&gt;Any and all guidance would be very much appreciated. Thank you so much in advance.&lt;/p&gt;&#xA;" OwnerUserId="17963" LastEditorUserId="650" LastEditDate="2023-08-07T14:26:04.807" LastActivityDate="2023-08-07T14:26:04.807" Title="Using CSV files continaing scRNA-seq count data from GEO" Tags="|rna-seq|scrnaseq|single-cell|" AnswerCount="1" CommentCount="2" ClosedDate="2023-08-11T12:18:20.803" ContentLicense="CC BY-SA 4.0" />
  <row Id="21403" PostTypeId="2" ParentId="21402" CreationDate="2023-08-07T10:25:34.903" Score="1" Body="&lt;p&gt;Please understand that your question is open-ended and essentially you're asking for a complete guidance through your analysis which is not feasible. I recommend to go through either the Seurat tutorials or the &lt;a href=&quot;https://bioconductor.org/books/release/OSCA.basic/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bioconductor single-cell workflow&lt;/a&gt; to learn the basics, such as typical workflows, formats of required input files etc.&lt;/p&gt;&#xA;&lt;p&gt;With this knowledge you can then start finding out which sort of data are provided in GEO and start getting your analysis going. You're welcome to ask specific questions here, but please be precise and not open-ended.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-08-07T10:25:34.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21404" PostTypeId="2" ParentId="18763" CreationDate="2023-08-07T16:58:14.523" Score="1" Body="&lt;p&gt;My understanding is from &lt;a href=&quot;http://dx.doi.org/10.1038/gim.2017.26&quot; rel=&quot;nofollow noreferrer&quot;&gt;Whiffin et al. 2017&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Imagine a dominant mendelian disease A with complete penetrance, caused only by one allele, which is currently unknown. In your search for that unknown allele, a &amp;quot;credible&amp;quot; candidate variants is one whose population allele frequency is lower than the prevalence of that disease in the population. In that sense,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\text{max. credible. AF}_\text{A} = \text{prevalence}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In a more realistic scenario, a similar disease B can be caused by more than one allele. A credible candidate variant would therefore have a lower expected allele frequency, because only some proportion of diseases in the population should be attributable to that variant (the &amp;quot;allelic contribution&amp;quot; of that variant),&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\text{max. credible AF}_\text{B} = \text{prevalence} \times \text{max. allelic contrib.}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Finally, consider a similar disease C with incomplete penetrance. Intuitively, a credible candidate variant would be allowed to have higher population allele frequency than the disease prevalence, due to the presence of unaffected carriers,&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$$\text{max. credible AF}_\text{C} = \text{prevalence} \times \text{max. allelic contrib.} \times 1/\text{penetrance}$$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thus far, we have talked about the maximum credible &lt;em&gt;population&lt;/em&gt; allele frequency. Population reference databases like gnomAD only have a sample of the population. Whiffin and colleagues observe that one can set a value for the maximum credible allele frequency, and subsequently compute by modelling a Poisson distribution, the one-tailed 95% confidence interval for the expected allele count in a sample of a given size.&lt;/p&gt;&#xA;&lt;p&gt;Conversely, they suggest, you could work backwards from an allele count and find the highest possible value for the maximum credible population allele frequency that gives the allele count as the 95% confidence interval. I believe this is the FAF95 recorded in gnomAD, or at least its definition should be similar.&lt;/p&gt;&#xA;&lt;p&gt;In other words: find a reasonable estimate for &lt;span class=&quot;math-container&quot;&gt;$\text{max. credible AF}_\text{C}$&lt;/span&gt;, and filter FAF95 against that.&lt;/p&gt;&#xA;" OwnerUserId="6520" LastActivityDate="2023-08-07T16:58:14.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21405" PostTypeId="1" AcceptedAnswerId="21406" CreationDate="2023-08-07T19:49:09.120" Score="2" ViewCount="36" Body="&lt;p&gt;I am new to both coding and GEO so this may be a silly question.&lt;/p&gt;&#xA;&lt;p&gt;I need the raw_counts data set for the GEO data set I am using but data in the downloads box is not the raw_counts. How would I be able to find the raw counts?&lt;/p&gt;&#xA;&lt;p&gt;As I am new to both I decided to use GEO2R to run an analysis and then to see if I could create the same results using code in R as I new that I could look at the R script for GEO2R if needed.&lt;/p&gt;&#xA;&lt;p&gt;I was struggling with which of the files I needed to download so I looked at the R script used in GEO2R but the files that are being used are completely different to the ones that I have downloaded.&lt;/p&gt;&#xA;&lt;p&gt;I am using the data set for Series GSE178646 &amp;quot;CRISPR gene editing and inducible pluripotent stem cell neuronal disease modelling for rare disease diagnosis: EMHM1 genetic variant analysis in Kleefstra Syndrome&amp;quot; and am only interested in CF7-12 but the files that I have downloaded are called &lt;code&gt;&amp;quot;GSM5395222_CF7_kallisto_abundance.tsv.gz&amp;quot;&lt;/code&gt; where as in the R script in GEO2R the code has used the file &lt;code&gt;&amp;quot;GSE178646_raw_counts_GRCh38.p13_NCBI.tsv.gz&amp;quot;&lt;/code&gt; and I am really not sure where I could access this file.&lt;/p&gt;&#xA;" OwnerUserId="17820" LastEditorUserId="17820" LastEditDate="2023-08-08T08:19:12.490" LastActivityDate="2023-08-08T10:38:02.460" Title="How to find the file names referred to in the R script produced by GEO2R in GEO Accession" Tags="|geo|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21406" PostTypeId="2" ParentId="21405" CreationDate="2023-08-07T22:05:46.947" Score="2" Body="&lt;p&gt;All the data associated with this project is here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gds/?term=GSE178646&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/gds/?term=GSE178646&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The actual download page and project description is here&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE178646&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE178646&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The dataset you seek &lt;strong&gt;GSE178646_raw_counts&lt;/strong&gt; is here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE178646&amp;amp;format=file&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/download/?acc=GSE178646&amp;amp;format=file&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The NPC EHMT1_SNV CF7 cells for &lt;strong&gt;GSM5395222_CF7_kallisto_abundance.tsv.gz&lt;/strong&gt; is on the same page &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE178646&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If you now click on the &lt;strong&gt;GEO2 analysis&lt;/strong&gt; on this page it will take your &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/geo2r/?acc=GSE178646&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; where you select your samples and begin the GEO2 analysis.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;From the comments,&lt;/p&gt;&#xA;&lt;p&gt;How to do this?&lt;/p&gt;&#xA;&lt;p&gt;Its easy simply go to &lt;a href=&quot;https://www.ncbi.nlm.nih.gov&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the check box search for the option &lt;strong&gt;GEO datasets&lt;/strong&gt;, in this example I queried &lt;strong&gt;GSM5395222&lt;/strong&gt; in the box. This goes to the search results &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gds/?term=GSM5395222&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, the top hit called &amp;quot;&lt;strong&gt;1&lt;/strong&gt;&amp;quot; is document you want &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE178646&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The very top hit (above the &lt;strong&gt;1&lt;/strong&gt;) is actual data in question, minus the project information page.&lt;/p&gt;&#xA;&lt;p&gt;All the information is within this document and simply use a &amp;quot;search text in webpage&amp;quot; using &lt;strong&gt;GSM5395222&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-08-08T10:38:02.460" LastActivityDate="2023-08-08T10:38:02.460" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21407" PostTypeId="1" AcceptedAnswerId="21416" CreationDate="2023-08-08T00:46:07.140" Score="3" ViewCount="175" Body="&lt;p&gt;So here I'm trying to donwload multiple fastq files based on the GSM ID input the script works fine when then input is only SRR id but it runs into error when it is given GSM ID&lt;/p&gt;&#xA;&lt;p&gt;My small GSM ID input set&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat srr_id.txt&#xA;GSM5369701&#xA;GSM4663476&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14154" LastEditorUserId="16676" LastEditDate="2023-08-09T19:00:25.357" LastActivityDate="2023-08-10T10:37:04.753" Title="Multiple SRR ID downloaded for single GSM IDs input issue" Tags="|fastq|shell|sratoolkit|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21408" PostTypeId="1" CreationDate="2023-08-08T11:17:36.147" Score="0" ViewCount="38" Body="&lt;p&gt;I'm looking for some kind of software, ideally FOSS, that will allow us to plot, compare, annotate and add additional points to time series data from bioprocessing runs (bioreactors).&#xA;Many of the bioreactors come with some of their own quite limited software to do this, but it cannot be installed on other computers.&lt;/p&gt;&#xA;&lt;p&gt;I have looked around a fair amount but can only expensive-looking biopharma software. Is there any academic software to achieve this?&#xA;I was considering that writing it myself wouldn't be that difficult with python, but I don't want to waste effort.&lt;/p&gt;&#xA;&lt;p&gt;(maybe this SE is not the best for bioprocessing, not sure where else to ask.)&lt;/p&gt;&#xA;" OwnerUserId="17972" LastActivityDate="2024-01-05T20:07:15.827" Title="Bioprocess data visualisation and analysis" Tags="|software-recommendation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21409" PostTypeId="2" ParentId="21408" CreationDate="2023-08-08T13:16:03.003" Score="0" Body="&lt;p&gt;As a general answer the technique here is called &amp;quot;Internet of Things&amp;quot; (IoT), where a machine, or an &amp;quot;IoT&amp;quot; device reports to either the cloud or another computer. In bioinformatics this technology has not embraced at present outside the industrial sector.&lt;/p&gt;&#xA;&lt;p&gt;Google Cloud Platform (GCP) does have an IoT platform where the devices can be registered and data is streamed to the cloud. However, doing this operation across a LAN (local area network) is automatic and can be directed via &lt;code&gt;cron&lt;/code&gt; notably in automating and collecting the data and analysing it periodically. For a generic IoT stack consider &lt;a href=&quot;https://www.mdpi.com/2076-3417/11/24/11932&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.mdpi.com/2076-3417/11/24/11932&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-08-08T18:13:56.400" LastActivityDate="2023-08-08T18:13:56.400" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21410" PostTypeId="1" AcceptedAnswerId="21411" CreationDate="2023-08-08T13:42:34.883" Score="1" ViewCount="41" Body="&lt;p&gt;I have a species tree to which I am trying to add species, which includes a tip &amp;quot;Geissospermum laeve&amp;quot;. When I run:&#xA;&lt;code&gt;with_polytomes = phytools::add.species.to.genus(tree, &amp;quot;Geissospermum polytome1&amp;quot;, where='root')&lt;/code&gt;, the original tree is returned with the error message: 'could not match your species to a genus  check spelling, including case'.&lt;/p&gt;&#xA;&lt;p&gt;Reproducible example:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;test_tree = ape::rtree(5)&#xA;test_tree$tip.label = c(&amp;quot;t3 sp1&amp;quot;, &amp;quot;t4 sp1&amp;quot;, &amp;quot;t5 sp1&amp;quot;, &amp;quot;t2 sp1&amp;quot;, &amp;quot;t1 sp1&amp;quot;)&#xA;with_polytomes = phytools::add.species.to.genus(test_tree, &amp;quot;t1 polytome1&amp;quot;, where='root')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I don't understand why this is happening, if anyone could shed some light on what's causing this error that would be great&lt;/p&gt;&#xA;" OwnerUserId="17673" LastEditorUserId="650" LastEditDate="2023-08-08T15:25:14.870" LastActivityDate="2023-08-09T13:02:54.567" Title="phytools add.species.to.genus could not match your species to a genus" Tags="|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21411" PostTypeId="2" ParentId="21410" CreationDate="2023-08-08T18:11:56.470" Score="1" Body="&lt;p&gt;Two things to try,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;setting &lt;code&gt;genus=NULL&lt;/code&gt;, The logic being it doesn't know which genus to add it to. Just as a precaution setting all blank space within a taxa id in the tree to &lt;code&gt;_&lt;/code&gt; ...&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;add.species.to.genus(test_tree, &amp;quot;t1_polytome1&amp;quot;, genus=NULL, where=c(&amp;quot;root&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Maybe replace &lt;code&gt;root&lt;/code&gt; with &lt;code&gt;random&lt;/code&gt; to get it working.&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;alternative using &lt;code&gt;tree&lt;/code&gt; instead&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;genus.to.species.tree(test_tree, &amp;quot;t1_polytome1&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="17673" LastEditDate="2023-08-09T13:02:54.567" LastActivityDate="2023-08-09T13:02:54.567" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21412" PostTypeId="2" ParentId="15506" CreationDate="2023-08-08T19:12:24.533" Score="1" Body="&lt;p&gt;As stated in &lt;a href=&quot;https://github.com/broadinstitute/cromwell/issues/4395&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/broadinstitute/cromwell/issues/4395&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The problem may be because you can't hardlink folders on your system for various reasons (i.e. filesystem doesn't allow hardlinking folders, only files,  or you are trying to link between different filesystems).&lt;/p&gt;&#xA;&lt;p&gt;You can try overriding the default way glob works.&lt;/p&gt;&#xA;&lt;p&gt;i.e. add this to your cromwell.config&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;glob-link-command = &amp;quot;ls -L GLOB_PATTERN 2&amp;gt; /dev/null | xargs -I ? ln -s ? GLOB_DIRECTORY&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;backend&#xA;{&#xA;  default = &amp;quot;Slurm&amp;quot;&#xA;  providers&#xA;  {&#xA;    Htcondor {&#xA;      actor-factory = &amp;quot;cromwell.backend.impl.sfs.config.ConfigBackendLifecycleActorFactory&amp;quot;&#xA;&#xA;      config&#xA;      {&#xA;        glob-link-command = &amp;quot;find GLOB_PATTERN 2&amp;gt; /dev/null | xargs -I ? ln -s ? GLOB_DIRECTORY&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16656" LastActivityDate="2023-08-08T19:12:24.533" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21413" PostTypeId="1" AcceptedAnswerId="21415" CreationDate="2023-08-09T08:44:32.450" Score="2" ViewCount="34" Body="&lt;p&gt;I have a bunch of RNA sequences (and their optimal secondary structures) and their corresponding energy values (measured by mean free energy) and I'm trying to find a way to identify features (patterns in their structures or sequences) common between samples of similar energy values. the overall goal is to try and identify sequence or structural elements in the RNA samples that tend to correlate with low mean free energy values. would be grateful if anyone could recommend me algorithms to look into using - I'm assuming this would be an unsupervised learning project and I only have experience with supervised stuff, but I'm looking into PCA right now and not sure if that'd be useful. should I be looking into something else?&lt;/p&gt;&#xA;&lt;p&gt;Intuitively I'm imagining it as like a clustering + feature extraction problem where I have a bunch of dots, each representing an RNA sample, and then the axis represents energy so the dots could be clustered in energy value similarity and then within each energy value cluster patterns/relationships could be found between the sequences and structures of the samples. but not sure if an ML algo exists to handle this and PCA seems like not what I'm looking for because the axes would be principal components and not energy...&lt;/p&gt;&#xA;" OwnerUserId="17976" LastEditorUserId="73" LastEditDate="2023-08-10T00:13:54.223" LastActivityDate="2023-08-10T00:13:54.223" Title="Help with finding an unsupervised learning algorithm for RNA sequence/structure pattern identification" Tags="|rna|clustering|features|unsupervised-learning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21414" PostTypeId="1" CreationDate="2023-08-09T08:58:20.853" Score="2" ViewCount="36" Body="&lt;p&gt;we've done Whole Genome Sequencing with Illumina on DNA extracted from cell lines, where some are suspected to have been infected by something (viral, bacterial, not sure).&lt;/p&gt;&#xA;&lt;p&gt;I've tried running metaphlan4 (version from github) and got zero hits in all of them.&lt;/p&gt;&#xA;&lt;p&gt;What other tools could I be trying? Thx.&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2023-10-12T10:32:54.237" Title="detecting DNA of possible infection on cell line" Tags="|metagenome|whole-genome|dna|" AnswerCount="3" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21415" PostTypeId="2" ParentId="21413" CreationDate="2023-08-09T13:30:34.170" Score="0" Body="&lt;p&gt;It depends what is meant by &amp;quot;features&amp;quot;. Features can be a particular structure component in the RNA structure. In ML/DL these &amp;quot;features&amp;quot; are called instances (or observations). &amp;quot;Features&amp;quot; is also a technical word used in machine/deep learning, which can be completely different, it means the data or statistical variables associated with an instance.&lt;/p&gt;&#xA;&lt;p&gt;Feature weight extraction (ML/DL definition) using unsupervised learning is complicated, but doable. This is where the features corresponding to each entry are weighted, i.e. quantified. Under traditional unsupervised learning this is not achievable, unless you want to include/exclude &amp;quot;features&amp;quot; and assess cluster integrity. I personally considered it a lot of work for no real interpretable results.&lt;/p&gt;&#xA;&lt;p&gt;If the features are components of the RNA structure (what would be technically called 'instances' in ML/DL) then the general traditional unsupervised learning following PCA analysis is tSNE. However, tSNE is often used to get better resolution the PCA feature reduction output, i.e. tightens the clusters. PCA-&amp;gt;tSNE is a cool technique and works well. PCA and tSNE can give different clusters by the way. As a rule of thumb I consider PCA &amp;gt; tSNE.&lt;/p&gt;&#xA;&lt;p&gt;Multi-dimensional scaling etc ... needs a defined analytics aim.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Information on tSNE and PCA this online resource looks fine&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.datacamp.com/tutorial/introduction-t-sne&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.datacamp.com/tutorial/introduction-t-sne&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The above is if you want to do tSNE and compare the results to PCA this link is good.&lt;/p&gt;&#xA;&lt;p&gt;In terms of using tSNE to sharpen PCA clusters I always used O'Reilly's &amp;quot;Introduction to Machine Learning with Python&amp;quot;. There is a very clear example of Python code of the technique there, towards the end of the book and why it works.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-08-09T15:12:53.113" LastActivityDate="2023-08-09T15:12:53.113" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21416" PostTypeId="2" ParentId="21407" CreationDate="2023-08-09T14:12:26.727" Score="1" Body="&lt;p&gt;Basically you're asking why a given GEO accession (GSM...) has many Runs (SRR...). That is because the submitters uploaded multiple fastq files for the same sample, which is the case if a library has been sequenced over multiple lanes or flowcells, often to increase depth. You can use &lt;code&gt;cat&lt;/code&gt; to combine these &amp;quot;technical replicates&amp;quot; into a single fastq file for both R1 and R2 file in case of paired-end data, or one single fastq file in case of single-end data.&lt;/p&gt;&#xA;&lt;p&gt;For example, if you have &lt;code&gt;file1.fq.gz&lt;/code&gt;, &lt;code&gt;file2.fq.gz&lt;/code&gt; and &lt;code&gt;file3.fq.gz&lt;/code&gt; just do:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;cat file1.fq.gz file2.fq.gz file3.fq.gz &amp;gt; combined.fq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16676" LastEditorUserId="16676" LastEditDate="2023-08-10T10:37:04.753" LastActivityDate="2023-08-10T10:37:04.753" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21417" PostTypeId="1" AcceptedAnswerId="21424" CreationDate="2023-08-09T15:30:27.323" Score="1" ViewCount="108" Body="&lt;p&gt;I've done multiple regression modeling in R using the below script. Now I'm wondering how can I make a table for the coefficients of each model of all models from lm, glm, and rf separately.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&#xA;all_variables &amp;lt;- names(PaC09c_w20_dat3)&#xA;response_variables &amp;lt;- all_variables[c(1:3)]&#xA;predictors &amp;lt;- all_variables[-c(1:3)]&#xA;&#xA;lm_model &amp;lt;- lapply(&#xA;  response_variables,&#xA;  function(x) lm(reformulate(termlabels = predictors, response = x), data = PaC09c_w20_dat3)&#xA;) |&amp;gt;&#xA;  setNames(response_variables)&#xA;&#xA;glm_model &amp;lt;- lapply(&#xA;  response_variables,&#xA;  function(x) glm(reformulate(termlabels = predictors, response = x), data = PaC09c_w20_dat3)&#xA;) |&amp;gt;&#xA;  setNames(response_variables)&#xA;&#xA;rf_ranger &amp;lt;- lapply(&#xA;  response_variables,&#xA;  function(x) ranger(reformulate(termlabels = predictors, response = x), data = PaC09c_w20_dat3, importance=&amp;quot;impurity&amp;quot;)&#xA;) |&amp;gt;&#xA;  setNames(response_variables)&#xA;&#xA;save.image(file=&amp;quot;PaC09c_w20_Reg.RData&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Input data looks like (Original Table is very large),&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  head(PaC09c_w20_dat3)&#xA;        EE87865ln1   EE87866ln1   EE87895ln1  blood_vessel_w20 adrenal_gland_w20 bone_element_w20 bronchus_w20 &#xA;               &amp;lt;dbl&amp;gt;       &amp;lt;dbl&amp;gt;      &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;             &amp;lt;dbl&amp;gt;            &amp;lt;dbl&amp;gt;        &amp;lt;dbl&amp;gt;         &#xA;        1    0.00391     0.00326    0.00332       0                 0                1            0             &#xA;        2    0.00139     0.00116    0.00132       0                 0                0            0             &#xA;        3    0.00360     0.00270    0.00469       1                 1                0            1             &#xA;        4    0.00323     0.00348    0.00339       0                 0                1            0             &#xA;        5    0.00323     0.00330    0.00382       0                 1                0            0             &#xA;        6    0.00278     0.00208    0.00214       0                 0                1            0   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The expected output would look like this,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;                           EE87865_lmcof   EE87866_lmcof&#xA;blood_vessel_w20           6.332633e-06    5.916615e-06&#xA;adrenal_gland_w20         -2.159470e-06   -2.399176e-06&#xA;bone_element_w20           8.578804e-06    8.106153e-06&#xA;bronchus_w20              -2.956291e-05   -2.152231e-05       &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12770" LastEditorUserId="12770" LastEditDate="2023-08-09T17:59:27.570" LastActivityDate="2023-08-11T00:46:39.117" Title="How to get coefficient tables from multiple regression model result!" Tags="|r|linear-regression|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21418" PostTypeId="2" ParentId="21417" CreationDate="2023-08-09T20:37:38.723" Score="1" Body="&lt;p&gt;To get the coefficients alone&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;lm_model&lt;span class=&quot;math-container&quot;&gt;$coefficients&#xA;glm_model$&lt;/span&gt;coefficients&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Everything including coefficients its,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;summary(lm_model)&#xA;summary(glm_model)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I assume this is the same for &lt;code&gt;summary(rf_ranger)&lt;/code&gt;, albeit not sure whether &lt;code&gt;coefficients&lt;/code&gt; would work here&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-08-09T20:37:38.723" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21419" PostTypeId="1" AcceptedAnswerId="21420" CreationDate="2023-08-09T21:54:39.603" Score="2" ViewCount="51" Body="&lt;p&gt;Reading through &lt;a href=&quot;https://www.datacamp.com/tutorial/introduction-t-sne&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt; t-sne blogpost and I'm a little unclear on how i'd use either of these algorithms here for my goals. i get it'd cluster my samples based on similarity in terms of first and second principal components/t-SNEs, but what could i then use identify structural or sequence patterns within each cluster? please let me know if I'm missing something - thanks!&lt;/p&gt;&#xA;&lt;p&gt;I'm also curious as to how I would apply PCA or t-SNE on my data considering its non-numerical nature. I was thinking of applying one-hot encoding, but is there a generally more suggested approach? I've attached a screenshot of a few of my dataset rows for context (for the sake of having the table fit, I chose rows with simpler structures/zero MFE values, but most rows have complex structures and non-zero MFE values). Thanks.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GbuKU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GbuKU.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;This is based on my previous post:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I have a bunch of RNA sequences (and their optimal secondary structures) and their corresponding energy values (measured by mean free energy) and I'm trying to find a way to identify features (patterns in their structures or sequences) common between samples of similar energy values. the overall goal is to try and identify sequence or structural elements in the RNA samples that tend to correlate with low mean free energy values. would be grateful if anyone could recommend me algorithms to look into using - I'm assuming this would be an unsupervised learning project and I only have experience with supervised stuff, but I'm looking into PCA right now and not sure if that'd be useful. should I be looking into something else?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Intuitively I'm imagining it as like a clustering + feature extraction problem where I have a bunch of dots, each representing an RNA sample, and then the axis represents energy so the dots could be clustered in energy value similarity and then within each energy value cluster patterns/relationships could be found between the sequences and structures of the samples. but not sure if an ML algo exists to handle this and PCA seems like not what I'm looking for because the axes would be principal components and not energy...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="17976" LastEditorUserId="73" LastEditDate="2023-08-10T00:10:30.657" LastActivityDate="2023-08-10T00:10:30.657" Title="Applying PCA or tSNE on non-numerical data" Tags="|rna|clustering|unsupervised-learning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21420" PostTypeId="2" ParentId="21419" CreationDate="2023-08-09T23:06:02.383" Score="1" Body="&lt;p&gt;I'd recommend PCA, tSNE the PCA (first)+ tSNE (second).&lt;/p&gt;&#xA;&lt;p&gt;The sequence data - one hot key encoding is a good idea.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;datadict = {'UTR':[.....],'Sequence':['AGAA','ACAG' ...], 'Optimal Secondary Structure':[....]}&#xA;df = pd.DataFrame(datadict)&#xA;df_onehot = pd.get_dummies(df['Sequence'])&#xA;df = pd.concat([df, df_onehot], axis=1)     # add the one hot encoded column to master dataframe&#xA;df.drop(...) # remove the original 'Sequence' column&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The other issue is vectorising the &amp;quot;&lt;strong&gt;dots&lt;/strong&gt;&amp;quot;. This is really a separate question, here's what I'd do ...&lt;/p&gt;&#xA;&lt;p&gt;For optimal secondary structure, maybe simply count the dots?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;count = df_series.str.count(&amp;quot;.&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This only works on a series so you'll need to do something like &lt;code&gt;df_list = df['Dots column'].to_list()&lt;/code&gt; and then &lt;code&gt;df_series = pd.Series(df_list)&lt;/code&gt;. Finally the &lt;code&gt;count&lt;/code&gt; will need to be &lt;code&gt;concat&lt;/code&gt; back onto the &lt;code&gt;df&lt;/code&gt; and the original &lt;code&gt;Dots&lt;/code&gt; deleted.&lt;/p&gt;&#xA;&lt;p&gt;The thing I'm not clear about is your 'instances' that could be 'UTR_info' thats the stuff you are calling you data or &amp;quot;features&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-08-09T23:26:16.923" LastActivityDate="2023-08-09T23:26:16.923" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21421" PostTypeId="2" ParentId="21414" CreationDate="2023-08-10T00:20:21.900" Score="3" Body="&lt;p&gt;I've had some success looking for contamination in Illumina and Nanopore reads using &lt;a href=&quot;https://github.com/DerrickWood/kraken2&quot; rel=&quot;nofollow noreferrer&quot;&gt;kraken2&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Kraken2 has a phylogenetic-based approach, which in the first instance tries to find the most specific taxa that the target read likely comes from (going further up in the tree if there's ambiguity), and also produces a summary table indicating the approximate proportion of reads that come from particular taxa. It can be paired with &lt;a href=&quot;https://ccb.jhu.edu/software/bracken/&quot; rel=&quot;nofollow noreferrer&quot;&gt;bracken&lt;/a&gt; for additional reliability.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-08-10T23:34:46.017" LastActivityDate="2023-08-10T23:34:46.017" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21422" PostTypeId="1" CreationDate="2023-08-10T01:06:33.503" Score="1" ViewCount="39" Body="&lt;p&gt;To put it super simply: I have RNA sequences, their optimal secondary structures, and the MFE values of those structures. I'm trying to computationally identity what structural elements and/or sequence motifs tend to be present in RNAs with lower MFEs.&lt;/p&gt;&#xA;&lt;p&gt;Was thinking about using unsupervised learning for this, but is this the wrong approach?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/21419/applying-pca-or-tsne-on-non-numerical-data&quot;&gt;Past version of this question&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17976" LastActivityDate="2024-01-14T03:02:15.237" Title="What algorithm should I use to find what RNA sequence/structural elements lead to different labelled MFE values?" Tags="|rna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21424" PostTypeId="2" ParentId="21417" CreationDate="2023-08-10T04:32:36.290" Score="1" Body="&lt;p&gt;Well, I found a solution,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; coefs_lm&amp;lt;-as.data.frame(lapply(lm_model,function(x)coef(x)[2:5]))&#xA;&#xA; coefs_glm&amp;lt;-as.data.frame(lapply(glm_model,function(x)coef(x)[2:5]))&#xA;&#xA;var_imp &amp;lt;- function(object, ...) {&#xA;  var_imp_r &amp;lt;- object$variable.importance&#xA;  var_imp_r&#xA;}&#xA;&#xA;coefs_rf&amp;lt;-as.data.frame(lapply(rf_ranger,function(x)var_imp(x)[1:4]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;2:5&lt;/code&gt; for &lt;code&gt;lm&lt;/code&gt; and &lt;code&gt;glm&lt;/code&gt; is the number of samples &lt;code&gt;EE..&lt;/code&gt;, where 1 is intercept.&lt;/p&gt;&#xA;" OwnerUserId="12770" LastEditorUserId="12770" LastEditDate="2023-08-11T00:46:39.117" LastActivityDate="2023-08-11T00:46:39.117" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21425" PostTypeId="2" ParentId="13237" CreationDate="2023-08-10T08:50:07.313" Score="1" Body="&lt;p&gt;I had some trouble installing rpy2 because of binary incompatibility issues. To solve this problem, I took the following steps:&lt;/p&gt;&#xA;&lt;p&gt;First, I wrote the necessary functions in R. Then, I created a straightforward REST-API using &lt;code&gt;library(plumber)&lt;/code&gt; and started the server on my local machine. On the Python side, I utilized &lt;code&gt;requests&lt;/code&gt; to call the functions and receive responses.&lt;/p&gt;&#xA;&lt;p&gt;I know that this solution is somewhat ugly, but it has works for me&lt;/p&gt;&#xA;" OwnerUserId="17985" LastActivityDate="2023-08-10T08:50:07.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21426" PostTypeId="1" AcceptedAnswerId="21429" CreationDate="2023-08-10T13:16:46.593" Score="2" ViewCount="54" Body="&lt;p&gt;I have pulled GATK docker on my computer but when running a GATK command, can not locate my input file&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My OS version&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat /etc/os-release&#xA;NAME=&amp;quot;Ubuntu&amp;quot;&#xA;VERSION=&amp;quot;18.04.6 LTS (Bionic Beaver)&amp;quot;&#xA;ID=ubuntu&#xA;ID_LIKE=debian&#xA;PRETTY_NAME=&amp;quot;Ubuntu 18.04.6 LTS&amp;quot;&#xA;VERSION_ID=&amp;quot;18.04&amp;quot;&#xA;HOME_URL=&amp;quot;https://www.ubuntu.com/&amp;quot;&#xA;SUPPORT_URL=&amp;quot;https://help.ubuntu.com/&amp;quot;&#xA;BUG_REPORT_URL=&amp;quot;https://bugs.launchpad.net/ubuntu/&amp;quot;&#xA;PRIVACY_POLICY_URL=&amp;quot;https://www.ubuntu.com/legal/terms-and-policies/privacy-policy&amp;quot;&#xA;VERSION_CODENAME=bionic&#xA;UBUNTU_CODENAME=bionic&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Files in my working directory&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fi1d18@RBGO-Server2:/data/Continuum/WES/vcf&lt;span class=&quot;math-container&quot;&gt;$ ls&#xA;GCA_000001405.15_GRCh38_no_alt_analysis_set.fna.gz&#xA;NG-27280_CLTSS_LTS_001A_lib506241_7636_2_MarkedDup.bam&#xA;NG-27280_CLTSS_LTS_001A_lib506241_7636_2_MarkedDup.bam.bai&#xA;NG-27280_CLTSS_LTS_001A_lib506241_7636_2_MarkedDuplicates.txt&#xA;NG-27280_CLTSS_LTS_001B_lib506242_7915_2_MarkedDup.bam&#xA;NG-27280_CLTSS_LTS_001B_lib506242_7915_2_MarkedDup.bam.bai&#xA;NG-27280_CLTSS_LTS_001B_lib506242_7915_2_MarkedDuplicates.txt&#xA;NG-27280_CLTSS_LTS_002B_lib488377_7494_1_MarkedDup.bam&#xA;NG-27280_CLTSS_LTS_002B_lib488377_7494_1_MarkedDup.bam.bai&#xA;NG-27280_CLTSS_LTS_002B_lib488377_7494_1_MarkedDuplicates.txt&#xA;fi1d18@RBGO-Server2:/data/Continuum/WES/vcf$&lt;/span&gt; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then pulled GATK docker&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fi1d18@RBGO-Server2:/data/Continuum/WES/vcf$ docker run -v ~/vcf:/gatk/vcf -it broadinstitute/gatk:latest&#xA;(gatk) root@8b2af4d7aadf:/gatk# &#xA;&#xA;&#xA;(gatk) root@8b2af4d7aadf:/gatk# ls&#xA;GATKConfig.EXAMPLE.properties  gatk-package-4.4.0.0-local.jar  gatkPythonPackageArchive.zip  scripts&#xA;README.md              gatk-package-4.4.0.0-spark.jar  gatkcondaenv.yml          vcf&#xA;gatk                   gatk-spark.jar              gatkenv.rc&#xA;gatk-completion.sh         gatk.jar                run_unit_tests.sh&#xA;(gatk) root@8b2af4d7aadf:/gatk# &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you see, folder seems to be empty because I can not connect my local working directory to container&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(gatk) root@6d3d3ff69e21:/gatk# cd vcf&#xA;(gatk) root@6d3d3ff69e21:/gatk/vcf# ls&#xA;(gatk) root@6d3d3ff69e21:/gatk/vcf# &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have tried all of these but did not work&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fi1d18@RBGO-Server2:/data/Continuum/WES/vcf$ docker run -v /data/Continuum/WES/vc:/gatk/ -it broadinstitute/gatk:latest&#xA;&#xA;fi1d18@RBGO-Server2:/data/Continuum/WES/vcf$ docker run -v ~/data/Continuum/WES/vc:/gatk/ -it broadinstitute/gatk:latest&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2023-08-10T16:31:54.260" LastActivityDate="2023-08-10T19:00:00.933" Title="Importing local files to a GATK docker image" Tags="|gatk|docker|wes|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21427" PostTypeId="1" CreationDate="2023-08-10T16:59:20.367" Score="1" ViewCount="150" Body="&lt;p&gt;I assembled Nanopore sequenced reads with Flye and visualized the GFA graph in Bandage but I don't really know how to interpret the result. For context, this is a yeast (DNA) genome. My goal is to figure out areas of repeats, ploidy level, and overall structure of the strain. So far I understand that the bubbles are repetitive areas but I mainly want to know how to interpret a graph like this.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4EART.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4EART.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm new to interpreting these graphs and I couldn't find any sort of guide online so if someone has any resources they'd be willing to share or if they could help interpret this graph I would really appreciate it!&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="16067" LastEditDate="2023-08-10T17:36:07.520" LastActivityDate="2023-08-25T21:21:53.853" Title="Interpreting GFA graph visualized in Bandage" Tags="|assembly|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21428" PostTypeId="2" ParentId="21414" CreationDate="2023-08-10T17:54:07.370" Score="1" Body="&lt;p&gt;Maybe less advanced but worth mentioning, I've had good results with &lt;a href=&quot;https://www.bioinformatics.babraham.ac.uk/projects/fastq_screen/&quot; rel=&quot;nofollow noreferrer&quot;&gt;fastq_screen&lt;/a&gt;. It's simply trying to align a subset of your sequences against various genomes (comes with a few genomes, you can add more).&lt;/p&gt;&#xA;" OwnerUserId="9497" LastActivityDate="2023-08-10T17:54:07.370" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21429" PostTypeId="2" ParentId="21426" CreationDate="2023-08-10T19:00:00.933" Score="1" Body="&lt;p&gt;@terdon answered that&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker run -v /data/Continuum/WES/:/gatk/vcf -it broadinstitute/gatk:latest&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastActivityDate="2023-08-10T19:00:00.933" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21431" PostTypeId="1" CreationDate="2023-08-11T19:54:57.563" Score="1" ViewCount="37" Body="&lt;p&gt;I have one function that is for reading .h5ad files&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ReadH5AD &amp;lt;- function(file) {&#xA;  # Load libraries&#xA;  library(anndata)&#xA;  library(reticulate)&#xA;  &#xA;  # Check if the file has the .h5ad extension&#xA;  if (tolower(tools::file_ext(file)) != &amp;quot;h5ad&amp;quot;) {&#xA;    stop(&amp;quot;Only h5ad file paths are accepted!&amp;quot;)&#xA;  }&#xA;  &#xA;  # Import and read the h5ad file&#xA;  ad &amp;lt;- import(&amp;quot;anndata&amp;quot;, convert = FALSE)&#xA;  h5ad &amp;lt;- ad$read_h5ad(file)&#xA;  &#xA;  # Extract counts matrix as a dense matrix&#xA;  counts &amp;lt;- as.matrix(py_to_r(h5ad$X))&#xA;  &#xA;  # Extract cell names&#xA;  cell_names &amp;lt;- py_to_r(h5ad&lt;span class=&quot;math-container&quot;&gt;$obs$&lt;/span&gt;index)&#xA;  &#xA;  # Extract gene names&#xA;  gene_names &amp;lt;- py_to_r(h5ad&lt;span class=&quot;math-container&quot;&gt;$var$&lt;/span&gt;index)&#xA;  &#xA;  # Transpose the matrix&#xA;  counts &amp;lt;- t(counts)&#xA;  &#xA;  # Set cell names and gene names as row and column names&#xA;  colnames(counts) &amp;lt;- cell_names&#xA;  rownames(counts) &amp;lt;- gene_names&#xA;  &#xA;  return(counts)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now there is another function which is used for file which are .mtx type along with barcode.tsv and gene.tsv.&lt;/p&gt;&#xA;&lt;p&gt;In order to detect organism type this function is used&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;detectOrganism &amp;lt;- function(outdir,data){&#xA;  gene.loc &amp;lt;- file.path(outdir, list.files(path=outdir, pattern='*genes\\.tsv', recursive=FALSE)[1])&#xA;  features.loc &amp;lt;- file.path(outdir, list.files(path=outdir, pattern='*features\\.tsv\\.gz', recursive=FALSE)[1])&#xA;  tryCatch({&#xA;    # Flag to indicate if data is from CellRanger &amp;gt;= 3.0&#xA;    checkGeneFile &amp;lt;- file.exists(gene.loc)&#xA;    if (!checkGeneFile) {&#xA;      genes &amp;lt;- read.table(file = features.loc, header = FALSE, sep = '\t',nrow=1, row.names = NULL)&#xA;    }&#xA;    else{&#xA;      genes &amp;lt;- read.table(file = gene.loc, header = FALSE, sep = '\t',nrow=1, row.names=NULL)&#xA;    }        &#xA;    if(grepl(&amp;quot;^ENSMUSG&amp;quot;,genes[1,1])){data&lt;span class=&quot;math-container&quot;&gt;$organism=&quot;Mouse&quot;}&#xA;    else if(grepl(&quot;^ENSG&quot;,genes[1,1])){data$organism=&quot;Human&quot;}&#xA;    else{data$&lt;/span&gt;organism=&amp;quot;NA&amp;quot;;stop()}&#xA;  }, &#xA;  interrupt = function(x){errorOccured&amp;lt;&amp;lt;-TRUE; speciesErrorNotification()},&#xA;  error = function(x){errorOccured&amp;lt;&amp;lt;-TRUE; speciesErrorNotification()}&#xA;  )&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This works when the input type is matrix.mtx,gene.tsv along with barcode.tsv. As this can detect using grepl from the gene.tsv file.&lt;/p&gt;&#xA;&lt;p&gt;I would like to implement the same for .h5ad input.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  gene_names &amp;lt;- py_to_r(h5ad&lt;span class=&quot;math-container&quot;&gt;$var$&lt;/span&gt;index)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This part can extract the gene_names from a given .h5ad input file but how do I pass this to the detectOrganism function.&lt;/p&gt;&#xA;&lt;p&gt;I have here basically two file types .h5ad and .mtx.&lt;/p&gt;&#xA;&lt;p&gt;So my simple goal is the add the same functionality when the input is .h5ad instead of .mtx file type.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion or help would be really appreciated&lt;/p&gt;&#xA;" OwnerUserId="285" LastActivityDate="2023-08-11T19:54:57.563" Title="Parsing gene names to detect organism from h5ad input" Tags="|scrnaseq|single-cell|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21432" PostTypeId="1" CreationDate="2023-08-11T22:57:59.610" Score="0" ViewCount="67" Body="&lt;p&gt;After assembly of genome, some protocols sometimes call for removal of scaffolds shorter than 500 or 1000 (some papers have one number while the other has the other.) Is this simply to remove the artefact and to make later analysis smoother, without the extroneous data? Also how do we know how where is our cut off point for size?&lt;/p&gt;&#xA;" OwnerUserId="17997" LastActivityDate="2023-10-06T18:11:54.513" Title="Why do we delete scaffolds shorter than 500 - 1000 bp from the assembled genome?" Tags="|assembly|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21433" PostTypeId="1" CreationDate="2023-08-13T14:34:04.067" Score="-1" ViewCount="336" Body="&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;plotDendroAndColors(bwnet$dendrograms[[1]], cbind(bwnet$unmergedColors, bwnet$colors),&#xA;                    c(&amp;quot;unmerged&amp;quot;, &amp;quot;merged&amp;quot;),&#xA;                    dendroLabels = FALSE,&#xA;                    addGuide = TRUE,&#xA;                    hang= 0.03,&#xA;                    guideHang = 0.05)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am getting the following error. colrs modules are not coming in  dendrogram graph. I am using 4gb ram laptop. is this problem because of laptop or my mistake in understanding code.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;Error in .plotOrderedColorSubplot(order = order, colors = colors,&#xA;rowLabels = rowLabels, : Length of colors vector not compatible with&#xA;number of objects in 'order'. madam getting this error.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My total genes are 15456. Under blockwisemodule I chose 5000&lt;/p&gt;&#xA;" OwnerUserId="18005" LastEditorUserId="650" LastEditDate="2023-08-16T19:40:34.010" LastActivityDate="2023-08-18T14:56:39.983" Title="WGCNA error: Length of colors vector not compatible with number of objects in 'order'" Tags="|r|bioconductor|wgcna|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21435" PostTypeId="1" CreationDate="2023-08-14T16:02:33.127" Score="0" ViewCount="59" Body="&lt;p&gt;I have a VCF file that is of the following format:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.1&#xA;##fileDate=20210329-231145&#xA;##AnalysisPipelineName=NGS5&#xA;##AnalysisPipelineVersion=V5.2.1&#xA;##AnalysisConfigurationName=NGS5_Exome&#xA;##AnalysisConfigurationVersion=5.2.1&#xA;##NGSPanelName=SureSelect_ExomeV7_plus_720G&#xA;##NGSPanelVersion=V1.0&#xA;##BatchID=WES_032721_A&#xA;##Lane=Lane1&#xA;##reference=hg38&#xA;##PathogenicSilentFile=Pathogenic_Silent.20200730.vcf&#xA;##FlowcellName=210327_A01233_0050_AHYFJHDSXY&#xA;##VariantDefDateTime=20210323-070701&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&amp;quot;Genotype&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=VF,Number=1,Type=Float,Description=&amp;quot;Variant Frequency&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=AD,Number=.,Type=Integer,Description=&amp;quot;Allele Depth&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=SB,Number=1,Type=String,Description=&amp;quot;Strand Bias with read numbers (Fw/Rv)&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=SA,Number=4,Type=Integer,Description=&amp;quot;Number of 1) forward ref alleles; 2) reverse ref; 3) forward non-ref; 4) reverse non-ref alleles, used in variant calling&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=SP,Number=1,Type=Float,Description=&amp;quot;Phred of Chi-square p-value for strand bias&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=SO,Number=1,Type=String,Description=&amp;quot;abs(log10( (RefFw*AltRv)/(RefRv*AltFw) )) for strand bias&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=DP,Number=1,Type=Integer,Description=&amp;quot;Total Depth&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=TI,Number=.,Type=String,Description=&amp;quot;Transcript ID&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=GI,Number=.,Type=String,Description=&amp;quot;Gene ID&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=LO,Number=.,Type=String,Description=&amp;quot;Genetic location&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=FC,Number=.,Type=String,Description=&amp;quot;Functional Consequence&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=PC,Number=.,Type=String,Description=&amp;quot;Protein level change&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=DC,Number=.,Type=String,Description=&amp;quot;DNA level change&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=EXON,Number=.,Type=String,Description=&amp;quot;Exon number&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=CI,Number=.,Type=String,Description=&amp;quot;Clinical impact&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=dbSNP,Number=.,Type=String,Description=&amp;quot;dbSNP version&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=1KG,Number=0,Type=String,Description=&amp;quot;Existing in the 1000 Genome phase 3 data&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=R8,Description=&amp;quot;IndelRepeatLength is greater than 8&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=R8.1,Description=&amp;quot;IndelRepeatLength of a monomer is greater than 8&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=R8.2,Description=&amp;quot;IndelRepeatLength of a dimer is greater than 8&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=sb,Description=&amp;quot;Variant strand bias high&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=sb.s,Description=&amp;quot;Variant strand bias significantly high (only for SNV)&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=rs,Description=&amp;quot;Variant with rs (dbSNP) number in a non-core gene&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=FP,Description=&amp;quot;Possibly false positives due to high similarity to off-target regions&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=NC,Description=&amp;quot;Noncoding INDELs on non-core genes&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=lowDP,Description=&amp;quot;low depth variant&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=Benign,Description=&amp;quot;Benign variant&amp;quot;&amp;gt;&#xA;##contig=&amp;lt;ID=chrM,length=16569&amp;gt;&#xA;##contig=&amp;lt;ID=chr1,length=248956422&amp;gt;&#xA;##contig=&amp;lt;ID=chr2,length=242193529&amp;gt;&#xA;##contig=&amp;lt;ID=chr3,length=198295559&amp;gt;&#xA;##contig=&amp;lt;ID=chr4,length=190214555&amp;gt;&#xA;##contig=&amp;lt;ID=chr5,length=181538259&amp;gt;&#xA;##contig=&amp;lt;ID=chr6,length=170805979&amp;gt;&#xA;##contig=&amp;lt;ID=chr7,length=159345973&amp;gt;&#xA;##contig=&amp;lt;ID=chr8,length=145138636&amp;gt;&#xA;##contig=&amp;lt;ID=chr9,length=138394717&amp;gt;&#xA;##contig=&amp;lt;ID=chr10,length=133797422&amp;gt;&#xA;##contig=&amp;lt;ID=chr11,length=135086622&amp;gt;&#xA;##contig=&amp;lt;ID=chr12,length=133275309&amp;gt;&#xA;##contig=&amp;lt;ID=chr13,length=114364328&amp;gt;&#xA;##contig=&amp;lt;ID=chr14,length=107043718&amp;gt;&#xA;##contig=&amp;lt;ID=chr15,length=101991189&amp;gt;&#xA;##contig=&amp;lt;ID=chr16,length=90338345&amp;gt;&#xA;##contig=&amp;lt;ID=chr17,length=83257441&amp;gt;&#xA;##contig=&amp;lt;ID=chr18,length=80373285&amp;gt;&#xA;##contig=&amp;lt;ID=chr19,length=58617616&amp;gt;&#xA;##contig=&amp;lt;ID=chr20,length=64444167&amp;gt;&#xA;##contig=&amp;lt;ID=chr21,length=46709983&amp;gt;&#xA;##contig=&amp;lt;ID=chr22,length=50818468&amp;gt;&#xA;##contig=&amp;lt;ID=chrX,length=156040895&amp;gt;&#xA;##contig=&amp;lt;ID=chrY,length=57227415&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_GL000195v1,length=182896&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_GL000213v1,length=164239&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_GL000214v1,length=137718&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_GL000216v2,length=176608&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_GL000218v1,length=161147&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_GL000219v1,length=179198&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_GL000220v1,length=161802&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_GL000224v1,length=179693&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_GL000226v1,length=15008&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270302v1,length=2274&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270303v1,length=1942&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270304v1,length=2165&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270305v1,length=1472&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270310v1,length=1201&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270311v1,length=12399&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270312v1,length=998&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270315v1,length=2276&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270316v1,length=1444&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270317v1,length=37690&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270320v1,length=4416&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270322v1,length=21476&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270329v1,length=1040&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270330v1,length=1652&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270333v1,length=2699&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270334v1,length=1368&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270335v1,length=1048&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270336v1,length=1026&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270337v1,length=1121&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270338v1,length=1428&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270340v1,length=1428&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270362v1,length=3530&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270363v1,length=1803&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270364v1,length=2855&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270366v1,length=8320&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270371v1,length=2805&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270372v1,length=1650&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270373v1,length=1451&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270374v1,length=2656&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270375v1,length=2378&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270376v1,length=1136&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270378v1,length=1048&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270379v1,length=1045&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270381v1,length=1930&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270382v1,length=4215&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270383v1,length=1750&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270384v1,length=1658&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270385v1,length=990&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270386v1,length=1788&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270387v1,length=1537&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270388v1,length=1216&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270389v1,length=1298&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270390v1,length=2387&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270391v1,length=1484&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270392v1,length=971&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270393v1,length=1308&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270394v1,length=970&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270395v1,length=1143&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270396v1,length=1880&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270411v1,length=2646&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270412v1,length=1179&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270414v1,length=2489&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270417v1,length=2043&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270418v1,length=2145&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270419v1,length=1029&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270420v1,length=2321&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270422v1,length=1445&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270423v1,length=981&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270424v1,length=2140&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270425v1,length=1884&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270429v1,length=1361&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270435v1,length=92983&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270438v1,length=112505&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270442v1,length=392061&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270448v1,length=7992&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270465v1,length=1774&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270466v1,length=1233&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270467v1,length=3920&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270468v1,length=4055&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270507v1,length=5353&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270508v1,length=1951&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270509v1,length=2318&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270510v1,length=2415&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270511v1,length=8127&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270512v1,length=22689&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270515v1,length=6361&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270516v1,length=1300&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270517v1,length=3253&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270518v1,length=2186&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270519v1,length=138126&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270521v1,length=7642&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270522v1,length=5674&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270528v1,length=2983&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270529v1,length=1899&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270530v1,length=2168&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270538v1,length=91309&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270539v1,length=993&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270544v1,length=1202&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270548v1,length=1599&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270579v1,length=31033&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270580v1,length=1553&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270581v1,length=7046&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270582v1,length=6504&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270583v1,length=1400&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270584v1,length=4513&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270587v1,length=2969&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270588v1,length=6158&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270589v1,length=44474&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270590v1,length=4685&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270591v1,length=5796&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270593v1,length=3041&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270741v1,length=157432&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270742v1,length=186739&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270743v1,length=210658&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270744v1,length=168472&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270745v1,length=41891&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270746v1,length=66486&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270747v1,length=198735&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270748v1,length=93321&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270749v1,length=158759&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270750v1,length=148850&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270751v1,length=150742&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270752v1,length=27745&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270753v1,length=62944&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270754v1,length=40191&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270755v1,length=36723&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270756v1,length=79590&amp;gt;&#xA;##contig=&amp;lt;ID=chrUn_KI270757v1,length=71251&amp;gt;&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  TN21-116928&#xA;chr1    6197766 rs2294714   T   C   0.0 rs  DP=1931;TI=NM_000983.3;GI=RPL22;FC=Silent;DC=c.13-10A&amp;gt;G;LO=INTRON;EXON=2    GT:VF:AD:SB:SA:SP:SO    0/1:0.483:999,932:883/1048:451/548,432/500:2:0.0&#xA;chr1    7677739 rs3011926   A   G   0.0 rs  DP=851;TI=NM_015215.3;GI=CAMTA1;FC=Silent;DC=c.2914+6A&amp;gt;G;LO=INTRON;EXON=11  GT:VF:AD:SB:SA:SP:SO    1/1:0.999:1,850:309/542:0/1,309/541:0:INF&#xA;chr1    9724365 rs11121484  C   T   0.0 rs  DP=1664;TI=NM_005026.4;GI=PIK3CD;FC=Synonymous;PC=Y936=;DC=c.2808C&amp;gt;T;LO=EXON;EXON=22    GT:VF:AD:SB:SA:SP:SO    0/1:0.455:907,757:842/822:448/459,394/363:6:0.0&#xA;chr1    10258594    rs12402052  C   G   0.0 rs  DP=1400;TI=NM_015074.3;GI=KIF1B;FC=Synonymous;PC=A95=;DC=c.285C&amp;gt;G;LO=EXON;EXON=4    GT:VF:AD:SB:SA:SP:SO    0/1:0.462:751,647:678/722:375/376,302/345:6:0.1&#xA;chr1    10295776    rs3753037   C   T   0.0 rs  DP=1050;TI=NM_015074.3;GI=KIF1B;FC=Silent;DC=c.1639+10C&amp;gt;T;LO=INTRON;EXON=17 GT:VF:AD:SB:SA:SP:SO    0/1:0.477:549,501:395/655:211/338,184/317:2:0.0&#xA;chr1    10375266    rs11121552  C   A   0.0 rs  DP=1487;TI=NM_015074.3;GI=KIF1B;FC=Synonymous;PC=T1721=;DC=c.5163C&amp;gt;A;LO=EXON;EXON=46    GT:VF:AD:SB:SA:SP:SO    0/1:0.484:767,719:870/617:443/324,427/292:3:0.0&#xA;chr1    11121270    rs11121691  C   T   0.0 .   DP=1092;TI=NM_004958.3;GI=MTOR;FC=Synonymous;PC=L2303=;DC=c.6909G&amp;gt;A;LO=EXON;EXON=49 GT:VF:AD:SB:SA:SP:SO    0/1:0.483:565,527:664/428:346/219,318/209:1:0.0&#xA;chr1    11130589    rs2275527   G   A   0.0 .   DP=1048;TI=NM_004958.3;GI=MTOR;FC=Synonymous;PC=S1851=;DC=c.5553C&amp;gt;T;LO=EXON;EXON=39 GT:VF:AD:SB:SA:SP:SO    0/1:0.509:515,533:572/476:275/240,297/236:3:0.0&#xA;chr1    11145001    rs1057079   C   T   0.0 .   DP=1392;TI=NM_004958.3;GI=MTOR;FC=Synonymous;PC=A1577=;DC=c.4731G&amp;gt;A;LO=EXON;EXON=33 GT:VF:AD:SB:SA:SP:SO    0/1:0.511:681,711:703/689:352/329,351/360:4:0.0&#xA;chr1    11228701    rs1064261   G   A   0.0 .   DP=1859;TI=NM_004958.3;GI=MTOR;FC=Synonymous;PC=N999=;DC=c.2997C&amp;gt;T;LO=EXON;EXON=19  GT:VF:AD:SB:SA:SP:SO    0/1:0.490:948,911:1059/800:536/412,523/388:2:0.0&#xA;chr1    11241657    rs1135172   A   G   0.0 .   DP=1643;TI=NM_004958.3;GI=MTOR;FC=Synonymous;PC=D479=;DC=c.1437T&amp;gt;C;LO=EXON;EXON=10  GT:VF:AD:SB:SA:SP:SO    0/1:0.512:801,842:793/850:376/425,417/425:5:0.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Unfortunately, when I try to utilize the following Python code to validate my VCF file:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pysam&#xA;&#xA;def validate_vcf(vcf_file_path):&#xA;    try:&#xA;        # Open the VCF file&#xA;        vcf_file = pysam.VariantFile(vcf_file_path)&#xA;&#xA;        # Iterate through the records to check if they can be read&#xA;        for record in vcf_file:&#xA;            pass&#xA;&#xA;        print(&amp;quot;VCF file is valid.&amp;quot;)&#xA;    except Exception as e:&#xA;        print(f&amp;quot;Error validating VCF file: {str(e)}&amp;quot;)&#xA;&#xA;# Path to the VCF file you want to validate&#xA;vcf_file_path = &amp;quot;path/to/yourfile.vcf&amp;quot;&#xA;&#xA;validate_vcf(vcf_file_path)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I receive the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[W::bcf_hrec_check] Invalid tag name: &amp;quot;1KG&amp;quot;&#xA;[E::vcf_parse_format] Invalid character '/' in 'SA' FORMAT field at chr1:6197766&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does anyone know what I can do to fix this?&lt;/p&gt;&#xA;" OwnerUserId="18008" LastEditorUserId="650" LastEditDate="2023-08-14T19:52:15.220" LastActivityDate="2023-08-15T00:09:14.617" Title="Invalid tag name: &quot;1KG&quot; and Invalid character '/' in 'SA' FORMAT field at chr1:6197766" Tags="|python|ngs|vcf|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21436" PostTypeId="1" CreationDate="2023-08-14T16:37:42.320" Score="1" ViewCount="30" Body="&lt;p&gt;I thought that when you linked tasks in a WDL workflow e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;call task1 {&#xA;    input: in = file1&#xA;}&#xA;&#xA;call task2 {&#xA;    input: in = task1.out&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That the first time you run it task1 and task2 would run, but the second time you run it nothing would run because the inputs haven't changed i.e. like GNU make&lt;/p&gt;&#xA;&lt;p&gt;Also, if you delete the output of task2 and run it then only task2 would run as the output from task1 is still there and hasn't changed.&lt;/p&gt;&#xA;&lt;p&gt;Basically, I am trying to use WDL like GNU make to only re-run the parts of my workflow that need rerunning depending on which inputs have changed.&lt;/p&gt;&#xA;&lt;p&gt;I suspect I have misunderstood and WDL cannot be used like this?&lt;/p&gt;&#xA;" OwnerUserId="18009" LastEditorUserId="650" LastEditDate="2023-08-14T19:52:35.993" LastActivityDate="2023-08-14T19:52:35.993" Title="WDL dependencies" Tags="|wdl|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21437" PostTypeId="2" ParentId="21435" CreationDate="2023-08-15T00:09:14.617" Score="1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Given issues I've had in various programming languages that use numbers at the start of a variable, the tag issue might be related to it starting with a numeric value. Is it possible to change that from &lt;code&gt;1KG&lt;/code&gt; to something like &lt;code&gt;ph3_1KG&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;According to the information in the VCF header, the SA field should have 4 values:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##FORMAT=&amp;lt;ID=SA,Number=4,Type=Integer,Description=&amp;quot;Number of 1) forward ref alleles; 2) reverse ref; 3) forward non-ref; 4) reverse non-ref alleles, used in variant calling&amp;quot;&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;While that location does have four numerical values in the actual data for that variant location, it seems like the validation code isn't expecting the forward and reverse alleles to be separated by &lt;code&gt;/&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  TN21-116928&#xA;chr1    6197766 rs2294714   T   C   0.0 rs  DP=1931;TI=NM_000983.3;GI=RPL22;FC=Silent;DC=c.13-10A&amp;gt;G;LO=INTRON;EXON=2    GT:VF:AD:SB:SA:SP:SO    0/1:0.483:999,932:883/1048:451/548,432/500:2:0.0&#xA;                                                                                                                                 ^^                                     ^^^^^^^^^^^^^^^&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Whether or not these errors are actually important will depend on the program that this VCF file is used for.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-08-15T00:09:14.617" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21438" PostTypeId="1" CreationDate="2023-08-15T12:27:58.257" Score="3" ViewCount="69" Body="&lt;p&gt;I received Whole Exome Sequencing data from an NGS company (CARIS, specifically). I received R1 and R2 FASTQ files, a BAM file aligned to hg38, and a VCF file.&lt;/p&gt;&#xA;&lt;p&gt;I used CNVPytor to create a CNV plot (Manhattan plot), by following this example Jupyter Notebook here: &lt;a href=&quot;https://github.com/abyzovlab/CNVpytor/blob/master/examples/PythonLibraryGuide.ipynb&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/abyzovlab/CNVpytor/blob/master/examples/PythonLibraryGuide.ipynb&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, when I run this code on my data, I get the following graph:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ipFXd.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ipFXd.jpg&quot; alt=&quot;Graph1&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, I tried to use CNVKit and used the following steps/code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cnvkit.py batch TN21-116928.DNA.bam --normal -m hybrid --fasta hg38.fa --targets targets.bed --output-reference my_reference.cnn&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Where &amp;quot;targets.bed&amp;quot; was a file of the following form:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr1    33306766    33321098    A3GALT2&#xA;chr22   42692121    42721298    A4GALT&#xA;chr3    138123713   138132390   A4GNT&#xA;chr12   53307456    53324864    AAAS&#xA;chr12   125065434   125143333   AACS&#xA;chr3    151814073   151828488   AADAC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The graph created from this is the following:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/GLX2G.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GLX2G.png&quot; alt=&quot;Graph 2&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know where I am going wrong? Any pointers?&lt;/p&gt;&#xA;" OwnerUserId="18008" LastActivityDate="2023-08-15T12:27:58.257" Title="Issue creating CNV plot from WES data" Tags="|python|ngs|cnv|wes|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21439" PostTypeId="1" CreationDate="2023-08-15T17:11:37.213" Score="-1" ViewCount="84" Body="&lt;p&gt;I have about 100 data sets with over 100,000 rows and 27 predictor variables and multiple response variable on each dataset. I want to select the most essential Features for these datasets. I used Boruta() from the package Boruta, but it takes an extremely long time (about 3 hours for a single sample). Could someone kindly tell me the quickest technique to select key features from a large dataset?&lt;/p&gt;&#xA;&lt;p&gt;This is the extension from my previous questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://stackoverflow.com/questions/76824529/how-to-do-regression-modelling-in-r-multiple-one-at-a-time-to-multiple-column&quot;&gt;https://stackoverflow.com/questions/76824529/how-to-do-regression-modelling-in-r-multiple-one-at-a-time-to-multiple-column&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/21417/how-to-get-coefficient-tables-from-multiple-regression-model-result/&quot;&gt;How to get coefficient tables from multiple regression model result!&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="12770" LastEditorUserId="650" LastEditDate="2023-08-16T19:38:38.530" LastActivityDate="2023-08-16T19:38:38.530" Title="What are the fastest way of feature selection in R for large dataset in Machine Learning?" Tags="|r|modelling|linear-regression|features|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21440" PostTypeId="2" ParentId="21439" CreationDate="2023-08-15T19:12:09.563" Score="0" Body="&lt;p&gt;Feature selection: the fastest thing moving (i.e. the simplest) is naive Bayes ML. I don't use it but KNN (K-nearest neighbor) I do use, which is also computationally simple, i.e. fast. I think it gives better results, i.e. the best results for a simple algorithm.&lt;/p&gt;&#xA;&lt;p&gt;R ML has improved decision trees BTW - quite popular, but you'd never use this in Python.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-08-15T19:12:09.563" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21442" PostTypeId="1" CreationDate="2023-08-16T11:29:47.757" Score="0" ViewCount="18" Body="&lt;p&gt;I was wondering if you can suggest something on the current status of the human genome annotation efforts. I know about ENSEMBL/HAVANA/GENCODE, and I've read the relevant papers; however, I was wondering if RefSeq or UCSC are still actively developed, and are there any competing efforts on T2T assembly, human pan-genome, etc etc?&lt;/p&gt;&#xA;&lt;p&gt;I know there are quite a few projects that generate both short and long read RNA-seq and other &amp;quot;evidence&amp;quot; from lots of cell lines etc. I guess I was looking for a good and up-to-date perspective or a review type of article.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="16324" LastActivityDate="2023-08-16T11:29:47.757" Title="The current status of human genome annotation" Tags="|sequence-annotation|human-genome|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21444" PostTypeId="2" ParentId="14388" CreationDate="2023-08-16T13:06:22.213" Score="2" Body="&lt;p&gt;I found fdnadist in embassy-phylip package &lt;a href=&quot;https://anaconda.org/bioconda/embassy-phylip&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://anaconda.org/bioconda/embassy-phylip&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18016" LastActivityDate="2023-08-16T13:06:22.213" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21445" PostTypeId="1" CreationDate="2023-08-16T17:12:53.433" Score="0" ViewCount="169" Body="&lt;p&gt;I'm in the following situation and I came here to see if I could get some help.&lt;/p&gt;&#xA;&lt;p&gt;I'd like to use IgBLAST for an analysis on some Alpaca antibody data, so I received some FASTA sequences from a colleague, &lt;code&gt;imgt_gapped_alpaca_IGHV.fasta&lt;/code&gt;, &lt;code&gt;imgt_gapped_alpaca_IGHD.fasta&lt;/code&gt;, etc.&lt;/p&gt;&#xA;&lt;p&gt;I used the &lt;code&gt;edit_imgt_file.pl&lt;/code&gt; tool to appropriately structure each &lt;code&gt;.fasta&lt;/code&gt; file, and to make a BLAST database for each (V, D, J) segment:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Process V segment sequences with edit_imgt_file&#xA;edit_imgt_file.pl \&#xA;~/VHH/data/imgt-data/imgt_gapped_alpaca_IGHV.fasta &amp;gt; ~/ncbi-igblast-1.21.0/internal_data/alpaca/imgt_alpaca_ig_v.fasta&#xA;&#xA;makeblastdb -parse_seqids -dbtype nucl \&#xA;    -in ~/ncbi-igblast-1.21.0/internal_data/alpaca/imgt_alpaca_ig_v.fasta \&#xA;    -out ~/ncbi-igblast-1.21.0/internal_data/alpaca/alpaca_V&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After doing that for each of the segments, I went to put all the pieces together and perform the following query:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;igblastn \&#xA;    -germline_db_V ~/ncbi-igblast-1.21.0/internal_data/alpaca/alpaca_V \&#xA;    -germline_db_D ~/ncbi-igblast-1.21.0/internal_data/alpaca/alpaca_D \&#xA;    -germline_db_J ~/ncbi-igblast-1.21.0/internal_data/alpaca/alpaca_J \&#xA;    -auxiliary_data ~/ncbi-igblast-1.21.0/optional_file/alpaca_gl.aux \&#xA;    -domain_system imgt -ig_seqtype Ig -organism alpaca \&#xA;    -outfmt '7 std qseq sseq btop' \&#xA;    -query ~/VHH/data/ALPACA1-BC1.fastq \&#xA;    -out ~/VHH/data/igblast-out/ALPACA1-BC1.fmt7&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Unfortunately, the following error is returned to me:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BLAST query/options error: Germline annotation database alpaca/alpaca_V could not be found in [internal_data] directory&#xA;Please refer to the BLAST+ user manual.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've searched the internet high and low, but this appears to be a bit of a niche problem, and the BLAST+ user manual promised a little more than it was able to deliver in terms of helpful suggestions.&lt;/p&gt;&#xA;&lt;p&gt;Any questions/comments/ideas/diagnostics would be very welcome-- if you see any low hanging fruit, please let me know-- thank you so much for your time!&lt;/p&gt;&#xA;" OwnerUserId="18019" LastActivityDate="2023-08-16T17:12:53.433" Title="IgBLAST Error: Germline annotation database could not be found in [internal_data] directory" Tags="|blast|blastn|makeblastdb|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21446" PostTypeId="1" CreationDate="2023-08-16T17:15:25.333" Score="1" ViewCount="38" Body="&lt;p&gt;I haven't found any relevant posts, so here is my question (it's more a question on statistics I believe):&lt;/p&gt;&#xA;&lt;p&gt;I have the deseq2 output of DE genes( around 25000 genes) , and among them 1000 are significantly differentiated(and I want to check them). Now, I group these 25000 genes by the epigenetic states of my data(i divide the full genome into 3 groups and I also have 4th group which are regions whose group I couldn’t identify due to lack of coverage etc ). So now I have my 25,000 genes where roughly 12000 belong to group a, 8000 belong to group 4 unidentified, 4500 belong to group b and 500 belong to group c. Now, what i do , from these 25000 genes I define my 4 gene sets where each group is one term so 4 in total, then i take my 1000 DEGs , i rank them and then I use prerank approach to get the enrichment results. I also hesitate to just stick ot hypergeometric test instead, but then not sure what I should consider in this case.&lt;/p&gt;&#xA;&lt;p&gt;My question is: How can I check if there are more differentially expressed genes in one group compared to others? I am particularly interested in group A, i expect it to have more DE genes (i don't know if upregulated or downregulated or both mixed).&lt;/p&gt;&#xA;&lt;p&gt;So far I've decided to go with hypergeometric test, but if my hypothesis is that one group is overrepresented with DE than others, i'm not sure it's a good approach; I tried GSEA too but because i have very few large gene sets ( where i define by a gene set all the genes that fall into one group), i hesitate if it's a suitable approach.&lt;/p&gt;&#xA;&lt;p&gt;I would greatly appreciate any help and tips answers and comments.  Thanks a lot everyone!!&lt;/p&gt;&#xA;&lt;p&gt;best regards,&#xA;alexa&lt;/p&gt;&#xA;" OwnerUserId="18020" LastEditorUserId="-1" LastEditDate="2023-08-17T09:44:50.973" LastActivityDate="2023-08-17T09:44:50.973" Title="enrichment of DE genes within wide regions not pathways, hypergeometric test or GSEA?" Tags="|statistics|deseq2|gsea|go-enrichment|t-test|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21447" PostTypeId="2" ParentId="21422" CreationDate="2023-08-16T23:07:05.657" Score="0" Body="&lt;p&gt;I would maintain traditional unsupervised learning is a very good method here (PCA and/or tSNE). The number 1 question is which column is the observation (label)? That depends on the aim of the study. The absence of a &amp;quot;target&amp;quot; column is the thing that separate unsupervised learning from supervised learning.&lt;/p&gt;&#xA;&lt;p&gt;The &amp;quot;observation&amp;quot; (label), i.e. the column that provides the name, BUT doesn't take part in the calculation, will either be a gene or an isolate/species/taxonomic-group and the aim is to see which genes or isolates cluster together. The implication is that the MFE (MLE?) is the observation (label) ... can be I guess. Again its about the aim of the study.&lt;/p&gt;&#xA;&lt;p&gt;The trick with unsupervised learning is to get as many &amp;quot;features&amp;quot; (columns of data) for each &amp;quot;observation&amp;quot; (probably gene name or taxonomic group). MLE (MFE?) energy (entropy?) has a value, but there are other values (which I forget but other entropy values), those can be included. Dumbie variables are used for e.g. sequence data, but if there are counts will work and experimental data measurements can be included. There are a few rules:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;When you have a cateogorical variable as a column in a dataframe it must be designated as such, pandas will allow this. Similarly if you have continuous variable, MLE that must be designated as a float and counts will be designated as integers. You don't want a float to appear as a categorical variable, that will be disastrous. You need to set the data type of each column (see below).&lt;/li&gt;&#xA;&lt;li&gt;In PCA a reasonable number of components must comprise a large amount (sum) of the variance, e.g. &amp;gt;70%.&lt;/li&gt;&#xA;&lt;li&gt;You can only see realistically two components at any one time on a graph, just plot them out, e.g. PC1 versus PC2, then PC1 vs PC3. Look at which give nice clusters. Don't just look at PC1 and PC2 and believe thats the only answer (which is what everyone does).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Point 1&lt;/strong&gt; is important if the PCA or tSNE may not permit mixed data types (I can't remember), but if so it will complain.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Point 2&lt;/strong&gt; If the sum of the first three principle components exceed 70% of the variance - thats very, very good. If its alot of then you might  consider performing tSNE on the PCs to tighten the result.&lt;/p&gt;&#xA;&lt;p&gt;Thats about it, what you are looking for is clusters that give some indication about the way the data is behaving. This can be the end point of the analysis or it can be the hypothesis for further testing.&lt;/p&gt;&#xA;&lt;p&gt;Very clear distinct cluster which make biological sense is a good result with unsupervised learning. Obviously the issue is that the clusters are not significant, but if they look good and make sense that's the key to the analysis.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-08-17T01:48:58.357" LastActivityDate="2023-08-17T01:48:58.357" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21449" PostTypeId="1" CreationDate="2023-08-17T13:31:35.187" Score="2" ViewCount="79" Body="&lt;p&gt;I'm trying to programmatically download files from SRA (with prefetch in sra-tools) starting from a GSE number, using Python.&lt;/p&gt;&#xA;&lt;p&gt;There is an explanation for using entrez-direct on the CLI, using &lt;a href=&quot;https://www.biostars.org/p/97782/&quot; rel=&quot;nofollow noreferrer&quot;&gt;esearch and the efetch&lt;/a&gt;. But I want to use Python. This is what I have now:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio import Entrez&#xA;Entrez.email = 'my_name@email.me'&#xA;handle = Entrez.esearch(db='gds', query='GSE134195', term=&amp;quot;biopython&amp;quot;)&#xA;record = Entrez.read(handle)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which gives:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-json prettyprint-override&quot;&gt;&lt;code&gt;record = Entrez.read(handle)&#xA;record&#xA;&#xA;{'Count': '17', 'RetMax': '17', 'RetStart': '0', 'IdList': ['200066260', '200045602', '200037064', '200030751', '100013939', '300762998', '300762997', '300762996', '300762995', '300762994', '300762993', '300762992', '300762991', '300762990', '300762989', '300762988', '300762987'], 'TranslationSet': [], 'TranslationStack': [{'Term': 'biopython[All Fields]', 'Field': 'All Fields', 'Count': '17', 'Explode': 'N'}, 'GROUP'], 'QueryTranslation': 'biopython[All Fields]'}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But what are these numbers? IdList looks useful, but there are 17 numbers where I expect 14. How do I get to SRR (SRA) files? The files I want can be found manually here: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE134195&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE134195&lt;/a&gt;, click on SRA run selector (at the bottom), this bring you here: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Traces/study/?acc=PRJNA554267&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/Traces/study/?acc=PRJNA554267&lt;/a&gt;, there I can download the Accession list that I want. But I'd like to automate that process.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="3964" LastActivityDate="2023-08-17T13:31:35.187" Title="Get SRR numbers from a GSE number" Tags="|entrez|ncbi|biopython|sra|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21450" PostTypeId="2" ParentId="21331" CreationDate="2023-08-18T13:15:42.527" Score="2" Body="&lt;p&gt;I am the developer of FlowSorted.Blood.EPIC, I will try to provide you with some guidance.&#xA;First, the FlowSorted.Blood.EPIC is located on ExperimentHub, which means that you need to ask the program to download the database first before running estimateCellCounts (minfi) or estimateCellCounts2 (FlowSorted.Blood.EPIC).&lt;/p&gt;&#xA;&lt;p&gt;For estimateCellCounts2, I explain this here if you are using a 450K array &lt;a href=&quot;https://github.com/immunomethylomics/FlowSorted.Blood.EPIC/issues/10&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub FlowSorted.Blood.EPIC&lt;/a&gt;. If you are using EPIC you can read the help file of &lt;a href=&quot;https://www.bioconductor.org/packages/release/data/experiment/vignettes/FlowSorted.Blood.EPIC/inst/doc/FlowSorted.Blood.EPIC.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;estimateCellCounts2&lt;/a&gt;. Although we believe our method improves the cell estimates, you can load the library and use minfi's &amp;quot;auto&amp;quot; selection or the same option on estimateCellCounts2 (that is your decision as a user of the library).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(FlowSorted.Blood.EPIC)&#xA;FlowSorted.Blood.EPIC &amp;lt;-&#xA;    libraryDataGet(&amp;quot;FlowSorted.Blood.EPIC&amp;quot;)&#xA;FlowSorted.Blood.EPIC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then you can run estimateCellCounts according to minfi's help.&lt;/p&gt;&#xA;&lt;p&gt;Good luck.&lt;/p&gt;&#xA;" OwnerUserId="18030" LastActivityDate="2023-08-18T13:15:42.527" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21451" PostTypeId="1" CreationDate="2023-08-18T14:33:11.623" Score="1" ViewCount="11" Body="&lt;p&gt;In &lt;a href=&quot;https://www.imgt.org/download/LIGM-DB/&quot; rel=&quot;nofollow noreferrer&quot;&gt;IMGT ftp-download (flat files) page&lt;/a&gt;, there are 2 optional fasta files (imgtblast and imgtrefseq) with their corresponding blast databases to download. Is the only difference the syntax and additional annotation in the refseq files?&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="2970" LastActivityDate="2023-08-18T14:33:11.623" Title="imgtrefseq vs imgtblast" Tags="|untagged|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21452" PostTypeId="2" ParentId="21433" CreationDate="2023-08-18T14:56:39.983" Score="1" Body="&lt;p&gt;Please follow the code in &lt;a href=&quot;https://horvath.genetics.ucla.edu/html/CoexpressionNetwork/Rpackages/WGCNA/Tutorials/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;WGCNA tutorial&lt;/a&gt; section 2c. Since the analysis ran in blockwise manner, you need to plot the dendrograms for each block separately and subset the module colors to each block. The example code is in the tutorial.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2023-08-18T14:56:39.983" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21453" PostTypeId="1" CreationDate="2023-08-19T21:44:49.400" Score="2" ViewCount="53" Body="&lt;p&gt;Bracken is the recommended post-processer of Kraken2 outputs and specifically is expected to improve relative abundance estimates. However, Bracken's Github has over 100 outstanding issues and the code seems to rely on Python 2, which is outdated. While many of the github issues are likely not due to bugs, it does seem like the software does not seem to be supported strongly at this time. I am curious what other alternatives exist. Are folks moving from kraken2/bracken to another solution for quick taxonomic hypothesis generation of metagenomics data?&lt;/p&gt;&#xA;" OwnerUserId="14844" LastActivityDate="2023-08-19T23:01:57.927" Title="What alternatives are there to Bracken?" Tags="|metagenome|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21454" PostTypeId="2" ParentId="21453" CreationDate="2023-08-19T23:01:57.927" Score="1" Body="&lt;p&gt;No, Braken is the &lt;em&gt;tour de force&lt;/em&gt; of first stage post-hoc Kraken2 analysis.&lt;/p&gt;&#xA;&lt;p&gt;Whether the code is supported or not it corrects bias in Kraken2 population frequency distributions. I'm not aware of any theoretical concerns.&lt;/p&gt;&#xA;&lt;p&gt;Generally, Python2 isn't a good idea, but it is overshadowed with the bigger concern of population bias, which is what Braken is there to adjust.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-08-19T23:01:57.927" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21455" PostTypeId="1" CreationDate="2023-08-20T11:13:58.390" Score="0" ViewCount="37" Body="&lt;p&gt;Cross-posted on biostars: &lt;a href=&quot;https://www.biostars.org/p/9572744/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/9572744/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am trying to understand better what number to be used and how to put it in the function MRcoefs within the Metagenomeseq package in R for calculating differentially expressed OTUs between two condition:&#xA;my code looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MRcoefs(fit, coef = 2, group = 3, number = 143, eff = 5.5) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here I am setting the eff argument to 5.5 because I have calculated the effectiv_size using this code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Effective_size(fit) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which result in table of effective size for each feature (OTU), and then I got the mean of these&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mean(Effective_size(fit))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which was 5.5&#xA;When I run this, it produced an error:&#xA;prop outside [0,1], I understand it should be between 0-1, but why ? is not that number in eff is the mean that I calculated which bring 5.5 ?&#xA;Also when I tried to run it with .5 as the document suggested as below :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MRcoefs(fit, coef = 2, group = 3, number = 143, eff = .5) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;it run nicely, no error , but it produced the same OTUs as when I did not include eff, I was expecting that some OTUs will be removed as they were below this threshold&lt;/p&gt;&#xA;&lt;p&gt;Could you please help me ?&lt;/p&gt;&#xA;" OwnerUserId="17543" LastEditorUserId="650" LastEditDate="2023-08-21T19:34:11.203" LastActivityDate="2024-01-19T14:04:25.557" Title="Try to understand better how to set out an effective sample size within the MetagenomSeq package in microbiome" Tags="|r|microbial-genomics|16rrna|microbiome|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21456" PostTypeId="1" AcceptedAnswerId="21459" CreationDate="2023-08-20T11:52:57.197" Score="0" ViewCount="105" Body="&lt;p&gt;I have some bam files in this directory&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/data/Continuum/WES/results/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to run &lt;code&gt;GATK&lt;/code&gt; mutation calling over bam files&lt;/p&gt;&#xA;&lt;p&gt;I googled and I realised for one function, I can do this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cd /data/Continuum/WES/vcf/&#xA;&#xA;for file in *.bam ; do ./gatk CollectSequencingArtifactMetrics -I /data/Continuum/WES/results/NG-27280_CLTSS_LTS_0017_lib506243_7661_2_MarkedDup.bam -O NG-27280_CLTSS_LTS_0017_lib506243_7661_2_MarkedDup --FILE_EXTENSION .txt -R resources_broad_hg38_v0_Homo_sapiens_assembly38.fasta done;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have five functions to run but I do not know how to include all in the script&lt;/p&gt;&#xA;&lt;p&gt;These are my functions&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./gatk CollectSequencingArtifactMetrics -I /data/Continuum/WES/results/NG-27280_CLTSS_LTS_0017_lib506243_7661_2_MarkedDup.bam -O NG-27280_CLTSS_LTS_0017_lib506243_7661_2_MarkedDup --FILE_EXTENSION .txt -R resources_broad_hg38_v0_Homo_sapiens_assembly38.fasta&#xA;&#xA;&#xA;./gatk GetPileupSummaries -I /data/Continuum/WES/results/NG-27280_CLTSS_LTS_0017_lib506243_7661_2_MarkedDup.bam -O NG-27280_CLTSS_LTS_0017_lib506243_7661_2_MarkedDup.targeted_sequencing.table -V af-only-gnomad.hg38.vcf.gz -L resources_broad_hg38_v0_wgs_calling_regions.hg38.interval_list -R resources_broad_hg38_v0_Homo_sapiens_assembly38.fasta&#xA;&#xA;&#xA;./gatk CalculateContamination -I NG-27280_CLTSS_LTS_0017_lib506243_7661_2_MarkedDup.targeted_sequencing.table -O NG-27280_CLTSS_LTS_0017_lib506243_7661_2_MarkedDup.targeted_sequencing.contamination.table&#xA;&#xA;./gatk GetSampleName -I /data/Continuum/WES/results/NG-27280_CLTSS_LTS_0017_lib506243_7661_2_MarkedDup.bam -O NG-27280_CLTSS_LTS_0017_lib506243_7661_2_MarkedDup.targeted_sequencing.sample_name&#xA;&#xA;./gatk Mutect2 -R resources_broad_hg38_v0_Homo_sapiens_assembly38.fasta -I /data/Continuum/WES/results/NG-27280_CLTSS_LTS_0017_lib506243_7661_2_MarkedDup.bam -O NG-27280_CLTSS_LTS_0017_lib506243_7661_2_MarkedDup.vcf -tumor NG-27280_CLTSS_LTS_0017_lib506243_7661_2_MarkedDup.targeted_sequencing.sample_name --af-of-alleles-not-in-resource 2.5e-06 --germline-resource af-only-gnomad.hg38.vcf.gz -pon 1000g_pon.hg38.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I thought to use &lt;code&gt;|&lt;/code&gt; to include five functions but I do not know how to change input and output for each iteration&lt;/p&gt;&#xA;&lt;p&gt;I tried this for the first function,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    FILES=&amp;quot;/data/Continuum/WES/testAlignmentBROADGenome2/results/.MarkedDup.bam*&amp;quot;&#xA;for f in &lt;span class=&quot;math-container&quot;&gt;$FILES&#xA;do&#xA;  ./gatk CollectSequencingArtifactMetrics -I $&lt;/span&gt;f -O $f --FILE_EXTENSION .txt -R resources_broad_hg38_v0_Homo_sapiens_assembly38.fasta&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but returned this error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SAMException: Cannot read non-existent file: file:///data/Continuum/WES/testAlignmentBROADGenome2/results/.MarkedDup.bam*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Anyone, please help me in develop this?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2023-08-20T15:18:22.287" LastActivityDate="2023-08-21T03:18:54.700" Title="A bash script for running on a bunch of bam files" Tags="|bash|gatk|wes|mutect2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21457" PostTypeId="1" CreationDate="2023-08-20T23:23:57.000" Score="4" ViewCount="78" Body="&lt;p&gt;I doing the alignment of samples following the GATK pipeline, and doing the MergeBamAlignment,like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MergeBamAlignment \&#xA;  -ALIGNED $path/file.unsorted.bam \&#xA;  -UNMAPPED $path/file.unmapped.bam \&#xA;  -O $path/file.merged.bam \&#xA;  -R $references/GRCh38.primary_assembly.genome.fa \&#xA;  -SO coordinate \&#xA;  -TMP_DIR $path/tmp/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I got this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Exception in thread &amp;quot;main&amp;quot; java.lang.IllegalArgumentException: Do not use this function to merge dictionaries with different sequences in them. Sequences must be in the same order as well. Found [] and [chr1, chr2, chr3, chr4, chr5, chr6, chr7, chr8, chr9, chr10, chr11, chr12, chr13, chr14, chr15, chr16, chr17, chr18, chr19, chr20, chr21, chr22, chrX, chrY, chrM, GL000008.2, GL000009.2, GL000194.1, GL000195.1, GL000205.2, GL000208.1, GL000213.1, GL000214.1, GL000216.2, GL000218.1, GL000219.1, GL000220.1, GL000221.1, GL000224.1, GL000225.1, GL000226.1, KI270302.1, KI270303.1, KI270304.1, KI270305.1, KI270310.1, KI270311.1, KI270312.1, KI270315.1, KI270316.1, KI270317.1, KI270320.1, KI270322.1, KI270329.1, KI270330.1, KI270333.1, KI270334.1, KI270335.1, KI270336.1, KI270337.1, KI270338.1, KI270340.1, KI270362.1, KI270363.1, KI270364.1, KI270366.1, KI270371.1, KI270372.1, KI270373.1, KI270374.1, KI270375.1, KI270376.1, KI270378.1, KI270379.1, KI270381.1, KI270382.1, KI270383.1, KI270384.1, KI270385.1, KI270386.1, KI270387.1, KI270388.1, KI270389.1, KI270390.1, KI270391.1, KI270392.1, KI270393.1, KI270394.1, KI270395.1, KI270396.1, KI270411.1, KI270412.1, KI270414.1, KI270417.1, KI270418.1, KI270419.1, KI270420.1, KI270422.1, KI270423.1, KI270424.1, KI270425.1, KI270429.1, KI270435.1, KI270438.1, KI270442.1, KI270448.1, KI270465.1, KI270466.1, KI270467.1, KI270468.1, KI270507.1, KI270508.1, KI270509.1, KI270510.1, KI270511.1, KI270512.1, KI270515.1, KI270516.1, KI270517.1, KI270518.1, KI270519.1, KI270521.1, KI270522.1, KI270528.1, KI270529.1, KI270530.1, KI270538.1, KI270539.1, KI270544.1, KI270548.1, KI270579.1, KI270580.1, KI270581.1, KI270582.1, KI270583.1, KI270584.1, KI270587.1, KI270588.1, KI270589.1, KI270590.1, KI270591.1, KI270593.1, KI270706.1, KI270707.1, KI270708.1, KI270709.1, KI270710.1, KI270711.1, KI270712.1, KI270713.1, KI270714.1, KI270715.1, KI270716.1, KI270717.1, KI270718.1, KI270719.1, KI270720.1, KI270721.1, KI270722.1, KI270723.1, KI270724.1, KI270725.1, KI270726.1, KI270727.1, KI270728.1, KI270729.1, KI270730.1, KI270731.1, KI270732.1, KI270733.1, KI270734.1, KI270735.1, KI270736.1, KI270737.1, KI270738.1, KI270739.1, KI270740.1, KI270741.1, KI270742.1, KI270743.1, KI270744.1, KI270745.1, KI270746.1, KI270747.1, KI270748.1, KI270749.1, KI270750.1, KI270751.1, KI270752.1, KI270753.1, KI270754.1, KI270755.1, KI270756.1, KI270757.1].&#xA;    at htsjdk.samtools.SAMSequenceDictionary.mergeDictionaries(SAMSequenceDictionary.java:340)&#xA;    at picard.sam.SamAlignmentMerger.getDictionaryForMergedBam(SamAlignmentMerger.java:197)&#xA;    at picard.sam.AbstractAlignmentMerger.mergeAlignment(AbstractAlignmentMerger.java:384)&#xA;    at picard.sam.SamAlignmentMerger.mergeAlignment(SamAlignmentMerger.java:181)&#xA;    at picard.sam.MergeBamAlignment.doWork(MergeBamAlignment.java:368)&#xA;    at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:309)&#xA;    at picard.cmdline.PicardCommandLine.instanceMain(PicardCommandLine.java:103)&#xA;    at picard.cmdline.PicardCommandLine.main(PicardCommandLine.java:113)&#xA;&#xA;srun: error: compute-12: task 0: Exited with exit code 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Before I also confirmed that the files that I'm trying to merge, are with the same reference sequences. So I don't understand what could be the problem.&lt;/p&gt;&#xA;" OwnerUserId="17902" LastEditorUserId="298" LastEditDate="2023-08-22T11:11:29.750" LastActivityDate="2023-08-22T15:30:40.180" Title="MergeBamAlignment error" Tags="|gatk|cancer|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21458" PostTypeId="1" CreationDate="2023-08-21T01:12:28.357" Score="-1" ViewCount="40" Body="&lt;p&gt;I want to build a machine learning model to predict colorectal cancer based on 16S rRNA microbiome data (stool samples).&lt;/p&gt;&#xA;&lt;p&gt;I have filtered the data using filtering approach by Duvallet(removing samples with fewer than 100 reads and OTUs with fewer than 10 reads,as well as OTUs which were present in fewer than 1% within the study) What do I have to do after the preprocessing step? How can I do the feature selection for the phenotype data and the microbiome data?&lt;/p&gt;&#xA;" OwnerUserId="18044" LastEditorUserId="18044" LastEditDate="2023-08-21T19:09:46.973" LastActivityDate="2023-08-21T19:09:46.973" Title="Feature-selection microbiome data to build a model to predict cancer" Tags="|python|metagenome|machine-learning|microbiome|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21459" PostTypeId="2" ParentId="21456" CreationDate="2023-08-21T03:00:46.823" Score="6" Body="&lt;p&gt;There are a couple ways to approach this, one is to create output files based on the &lt;code&gt;BAM&lt;/code&gt; input file name - the other approach is to make a new folder (based on the input &lt;code&gt;BAM&lt;/code&gt; filename containing result files with the same name for each input. I generally use the first method, but either work.&lt;/p&gt;&#xA;&lt;p&gt;Here is a script that may run the 5 &lt;code&gt;GATK&lt;/code&gt; functions. It calculates a prefix to be used for derivative output files &lt;code&gt;BASE&lt;/code&gt;, and should run your pipeline on all the &lt;code&gt;BAM&lt;/code&gt; files within the &lt;code&gt;INPUT_FOLDER&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;set -e&#xA;&#xA;INPUT_FOLDER=&amp;quot;/data/Continuum/WES/results/&amp;quot;&#xA;&#xA;for BAM_FILE in &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$INPUT_FOLDER/&quot;*.bam&#xA;do&#xA;        BASE=$&lt;/span&gt;(basename &amp;quot;$BAM_FILE&amp;quot; .bam) # NG-27280_CLTSS_LTS_0017_lib506243_7661_2_MarkedDup&#xA;&#xA;        # function 1&#xA;        ./gatk CollectSequencingArtifactMetrics \&#xA;                -I &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$BAM_FILE&quot; \&#xA;                -O &quot;$&lt;/span&gt;BASE&amp;quot; \&#xA;                --FILE_EXTENSION .txt \&#xA;                -R resources_broad_hg38_v0_Homo_sapiens_assembly38.fasta&#xA;&#xA;        # function 2&#xA;        ./gatk GetPileupSummaries \&#xA;                -I &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$BAM_FILE&quot; \&#xA;                -O &quot;$&lt;/span&gt;BASE.targeted_sequencing.table&amp;quot; \&#xA;                -V af-only-gnomad.hg38.vcf.gz \&#xA;                -L resources_broad_hg38_v0_wgs_calling_regions.hg38.interval_list \&#xA;                -R resources_broad_hg38_v0_Homo_sapiens_assembly38.fasta&#xA;&#xA;        # function 3&#xA;        ./gatk CalculateContamination \&#xA;                -I &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$BASE.targeted_sequencing.table&quot; \&#xA;                -O &quot;$&lt;/span&gt;BASE.targeted_sequencing.contamination.table&amp;quot; \&#xA;&#xA;        # function 4&#xA;        ./gatk GetSampleName \&#xA;                -I &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$BAM_FILE&quot; \&#xA;                -O &quot;$&lt;/span&gt;BASE.targeted_sequencing.sample_name&amp;quot;&#xA;&#xA;        # function 5&#xA;        ./gatk Mutect2 \&#xA;                -R resources_broad_hg38_v0_Homo_sapiens_assembly38.fasta \&#xA;                -I &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$BAM_FILE&quot; \&#xA;                -O &quot;$&lt;/span&gt;BASE&amp;quot;.vcf \&#xA;                -tumor &amp;quot;$BASE.targeted_sequencing.sample_name&amp;quot; \&#xA;                --af-of-alleles-not-in-resource 2.5e-06 \&#xA;                --germline-resource af-only-gnomad.hg38.vcf.gz \&#xA;                -pon 1000g_pon.hg38.vcf.gz&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It may need some adjustments - I did not test this, and used the &lt;code&gt;GATK&lt;/code&gt; incantations you provided (thank you for including all of your steps!)&lt;/p&gt;&#xA;&lt;p&gt;It includes the addition of the command &lt;code&gt;set -e&lt;/code&gt;, which depending on GATKs return status may stop the script if any command fails - otherwise the script will happily run through a failed command.&lt;/p&gt;&#xA;&lt;p&gt;You also may need to adjust the location of your output, and the directory the script is in upon execution, as you are running &lt;code&gt;GATK&lt;/code&gt; from the current directory (&lt;code&gt;./gatk&lt;/code&gt;)&lt;/p&gt;&#xA;&lt;p&gt;In order to make these long lines readable, I use the method of placing a &lt;code&gt;\&lt;/code&gt; at the end of a line - this allows the command to wrap to the next line. Be sure that the &lt;code&gt;\&lt;/code&gt; character is at the end of your line for this to function properly.&lt;/p&gt;&#xA;" OwnerUserId="92" LastEditorUserId="92" LastEditDate="2023-08-21T03:18:54.700" LastActivityDate="2023-08-21T03:18:54.700" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21460" PostTypeId="2" ParentId="21458" CreationDate="2023-08-21T03:20:09.430" Score="0" Body="&lt;p&gt;I think the question needs improving. As the question stands pre-processing to feature selection ML (that's whats being asked).&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;put the species diversity into a pandas dataframe the last column is the target (cancer vs healthy).&lt;/li&gt;&#xA;&lt;li&gt;assess a variety of ML algorithms (because they give different answers). Naive bayes, linear regression and K-nearest neighbour (KNN) are simple algorithms, easy to run and at this level fine.&lt;/li&gt;&#xA;&lt;li&gt;each have a &amp;quot;feature selection&amp;quot; - all features will be positive.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;At a personal level I like KNN, but avoid LR and NB.&lt;/p&gt;&#xA;&lt;p&gt;I personally would simply suggest unsupervised learning.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-08-21T03:20:09.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21461" PostTypeId="2" ParentId="3881" CreationDate="2023-08-21T04:56:37.607" Score="2" Body="&lt;p&gt;You need to understand a few key points for your example given:&lt;/p&gt;&#xA;&lt;p&gt;(a) Ancestry (and 23andMe) will use the letters I &lt;strong&gt;and&lt;/strong&gt; D to indicate it is an InDel/DELINS. You have to read the rsID definition to determine if the variant is an Insert or Delete from the reference.  And therefore which letter is used to mean positive / variant and which for negative / reference.  They simply use the other letter to indicate negative / reference.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/?term=rs267607236&quot; rel=&quot;nofollow noreferrer&quot;&gt;rs267607236&lt;/a&gt; is defined as a deletion. So the I's mean the reference was detected in this case. Therefore the Ancestry file indicates no deletion of the 4 base-pairs at coordinates 50515271-4.&lt;/p&gt;&#xA;&lt;p&gt;(b) Ancestry and dbSNP VCF annotation files uses a &amp;quot;left-aligned&amp;quot; coordinate for ALL InDel's and generally designate the coordinate BEFORE the variant.&lt;/p&gt;&#xA;&lt;p&gt;This makes sense for Inserts. It gives the coordinate of the last base-pair in the reference just before the insert starts. That coordinate plus 1 is the next base-pair after the insert in the reference.&lt;/p&gt;&#xA;&lt;p&gt;This same method is used for deletions.  But because the deleted values have coordinates in the reference, this makes deletions appear &amp;quot;one off&amp;quot;. You could use the starting coordinate of the deletion when defining the variant (e.g. &lt;code&gt;[TGAG/-]&lt;/code&gt;). Otherwise, you need the coordinate before if defining the deletion as &lt;code&gt;[CTGAG/C]&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/snp/?term=rs267607236&quot; rel=&quot;nofollow noreferrer&quot;&gt;rs267607236&lt;/a&gt; is listed in the OP's information as coordinate 50515270. Because it is a deletion, the deleted sequence actually starts with the base-pair at coordinate 50515270+1 or 50515271.&lt;/p&gt;&#xA;&lt;p&gt;(c) As it ends up, for rs267607236 in question here, the dbSNP VCF and the dbSNP online website entry are different (both latest release 156 in this case). This could happen but still be consistent because there is a &lt;code&gt;T&lt;/code&gt; flanking the 4 base-pair deletion.  The starting reference sequence &lt;code&gt;...C TGAGT A...&lt;/code&gt; becomes &lt;code&gt;CTA&lt;/code&gt; after the deletion.  But which &lt;code&gt;T&lt;/code&gt; remains is not really known. The VCF file defines the deletion by retaining the trailing &lt;code&gt;T&lt;/code&gt;. The online website retains the leading &lt;code&gt;T&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;And in fact, it appears the website entry starting coordinate is incorrect.  Given the way they defined this variant within the flanking sequence, the online website should use the coordinate 22:50515270 (like in the Ancestry file) and not 22:50515271 as it does now. This may have existed with that coordinate in an earlier release used by Illumina / Ancestry when defining the pipeline.  But why the VCF and online website are different now is confusing.&lt;/p&gt;&#xA;&lt;p&gt;There are numerous other nuances, conventions and actual errors that have to be understand to properly convert a microarray raw data file to a VCF.&lt;/p&gt;&#xA;&lt;p&gt;Here are all the entries that we referenced / consulted brought together for your perusal:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rs6010164      22    50515270    T    T&#xA;rs267607236    22    50515270    I    I&#xA;&#xA;GRCh37 Reference sequence around rs267607236:&#xA;TTAC TGAGT AAGA  (70-74)&#xA;&#xA;dbSNP Release 156 GRCh37 VCF:&#xA;NC_000022.10    50515270        rs6010164       T       C&#xA;NC_000022.10    50515269        rs267607236     CTGAG   C       # Note: leading T deleted&#xA;&#xA;dbSNP web page definition:&#xA;Alleles:T&amp;gt;C             for rs6010164)&#xA;TTAC [T/C] GAGTAAGAT&#xA;22:50515270 (GRCh37)&#xA;NC_000022.10:50515270:T:C&#xA;&#xA;Alleles:TGAGT&amp;gt;T         for rs267607236&#xA;TTACT [GAGT/-] AAGAT                                            # Note: trailing T deleted&#xA;22:50515271 (GRCh37)                # for consistency with flanking sequence, this should be ..270&#xA;NC_000022.10:50515271:TGAGT:T&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It is not clear to me what Ancestry does when the same rsID has an insert and a deletion at the same position. For example, the line from dbSNP VCF for rs768019142 is:&lt;br /&gt;&#xA;&lt;code&gt;NC_000001.11    10055   rs768019142     TA      T,TAA&lt;/code&gt;&lt;br /&gt;&#xA;which says there are two variants in the rsID. (This is common, especially for SNPs with multiple variants possible).  A deletion at 10056 OR a single letter insert after 10056. To represent this, instead of a single letter for the reference, they have given two base-pairs as the reference sequence. They probably should have defined two rsIDs at two adjacent locations. UCSC's versions of the VCF files are filtered and cleanup to simplify items like this.&lt;/p&gt;&#xA;" OwnerUserId="17844" LastActivityDate="2023-08-21T04:56:37.607" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21462" PostTypeId="2" ParentId="20713" CreationDate="2023-08-21T05:44:18.753" Score="1" Body="&lt;p&gt;Generally, consumer WGS companies do not deliver annotated VCF's. Annotations include rsIDs, gene names or similar area designations, and others.  But this can be done using the NIH &lt;a href=&quot;https://ftp.ncbi.nih.gov/snp/latest_release/VCF/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbSNP VCF&lt;/a&gt; files and the &lt;a href=&quot;https://samtools.github.io/bcftools/howtos/annotate.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;bcftools annotate&lt;/a&gt; command.  The &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTrackUi?db=hg38&amp;amp;g=dbSnp153Composite&quot; rel=&quot;nofollow noreferrer&quot;&gt;annotation files at UCSC&lt;/a&gt; are modified for their GenomeBrowser and are often filtered and cleaned up.  While better and more consistent, they are usually not the latest release available from NIH.&lt;/p&gt;&#xA;&lt;p&gt;Just make sure to match the build model and sequence (chromosome) naming convention between your annotation file and the original VCF. Dante usually delivers using the hs37d5 reference model. So use the .25 dbSNP VCFs files.  But dbSNP VCFs use accession names whereas Dante's hs37d5 reference model uses GRCh numeric ones.  So you will have to rename sequence names in one file or the other to make it work. Also, hopefully, the vendor has left aligned the InDels as that is what the dbSNP file has them defined as and what bcftools expects as well. Dante traditionally uses the DRAGEN pipeline with their Illumina sequencers; which does this. But they have recently switched to MGI DNB sequencers and the megaBOLT pipeline for customers outside the USA. We are not yet sure how they are processing InDels. Finally, Dante has historically delivered a separate SNP and InDel file.  You will often want to merge them or annotate them both separately then merge. bcftools can do this.&lt;/p&gt;&#xA;&lt;p&gt;The dbSNP VCF file has well over 1 billion rsIDs now defined. And many fields of additional information that can be annotated in.  Often, if you just want to add a simple subset of the data, you can process the dbSNP VCF file to a simpler tab-separated (TSV, tab) file of entries. It can cut down the annotation file from 25 GB to 5 GB or less. See the bcftools annotation page for hints on using that tool to do this as well.&lt;/p&gt;&#xA;&lt;p&gt;Personally, I find it easier to use online tools or services that have been tweaked.  Especially if they have nice browsers.  &lt;a href=&quot;https://sequencing.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sequencing dot com&lt;/a&gt;, &lt;a href=&quot;https://usegalaxy.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;usegalaxy.org&lt;/a&gt; and &lt;a href=&quot;https://iobio.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;iobio.io&lt;/a&gt; are a few that come to mind.  Not all allow the export of the annotated VCF.&lt;/p&gt;&#xA;&lt;p&gt;This and similar topics of utilizing your results are heavily discussed in the &lt;a href=&quot;https://www.facebook.com/groups/consumerwgs&quot; rel=&quot;nofollow noreferrer&quot;&gt;consumerWGS Facebook group&lt;/a&gt;; if you also wish to participate there.&lt;/p&gt;&#xA;" OwnerUserId="17844" LastActivityDate="2023-08-21T05:44:18.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21464" PostTypeId="1" CreationDate="2023-08-21T16:52:22.320" Score="2" ViewCount="62" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9572854/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have two samples where the sequencing was done on 3-lanes paired end reads. The two samples are A44943_1 and A44944_1.&lt;/p&gt;&#xA;&lt;p&gt;I would like to generate a for in loop to merge the R1 files of sample A44943_1 from the 3-lanes,  and then do the same for R2 files of this sample, then I want to merge R1 files of sample A44944_1 from the 3-lanes and then do the same for R2 files of this sample. So I thought about for in loop. My fastq files looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;R0480-S0001_182602S1P1_A44943_1_H77YFDRXY_CGCAACTA-GAATCCGA_L001_R1_trimmed.fastq&#xA;R0480-S0001_182602S1P1_A44943_1_H77YFDRXY_CGCAACTA-GAATCCGA_L001_R2_trimmed.fastq&#xA;R0480-S0001_182602S1P1_A44943_1_H77YFDRXY_CGCAACTA-GAATCCGA_L002_R1_trimmed.fastq&#xA;R0480-S0001_182602S1P1_A44943_1_H77YFDRXY_CGCAACTA-GAATCCGA_L002_R2_trimmed.fastq&#xA;R0480-S0001_182602S1P1_A44943_1_H7YKTDRXY_CGCAACTA-GAATCCGA_L003_R1_trimmed.fastq&#xA;R0480-S0001_182602S1P1_A44943_1_H7YKTDRXY_CGCAACTA-GAATCCGA_L003_R2_trimmed.fastq&#xA;R0480-S0002_182602S1P2_A44944_1_H77YFDRXY_CACAGACT-TGGTACAG_L001_R1_trimmed.fastq&#xA;R0480-S0002_182602S1P2_A44944_1_H77YFDRXY_CACAGACT-TGGTACAG_L001_R2_trimmed.fastq&#xA;R0480-S0002_182602S1P2_A44944_1_H77YFDRXY_CACAGACT-TGGTACAG_L002_R1_trimmed.fastq&#xA;R0480-S0002_182602S1P2_A44944_1_H77YFDRXY_CACAGACT-TGGTACAG_L002_R2_trimmed.fastq&#xA;R0480-S0002_182602S1P2_A44944_1_H7YKTDRXY_CACAGACT-TGGTACAG_L003_R1_trimmed.fastq&#xA;R0480-S0002_182602S1P2_A44944_1_H7YKTDRXY_CACAGACT-TGGTACAG_L003_R2_trimmed.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Could you advise me on  a for in loop to do this task?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="17543" LastEditorUserId="73" LastEditDate="2023-08-23T01:00:49.790" LastActivityDate="2023-08-23T01:00:49.790" Title="How to loop over certain files within a directory that belongs to certain samples" Tags="|linux|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21465" PostTypeId="2" ParentId="21464" CreationDate="2023-08-21T18:43:16.560" Score="5" Body="&lt;p&gt;You can certainly use a for loop for this if desired, but an easier method is to use the &lt;code&gt;cat&lt;/code&gt; command - something like this:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;cat R0480-S0001_182602S1P1_A44943_1_H77YFDRXY_CGCAACTA-GAATCCGA_L00[123]_R1_trimmed.fastq &amp;gt; R0480-S0001_182602S1P1_A44943_1_H77YFDRXY_CGCAACTA-GAATCCGA_R1_trimmed.fastq&#xA;cat R0480-S0001_182602S1P1_A44943_1_H77YFDRXY_CGCAACTA-GAATCCGA_L00[123]_R2_trimmed.fastq &amp;gt; R0480-S0001_182602S1P1_A44943_1_H77YFDRXY_CGCAACTA-GAATCCGA_R2_trimmed.fastq&#xA;cat R0480-S0002_182602S1P2_A44944_1_H77YFDRXY_CACAGACT-TGGTACAG_L00[123]_R1_trimmed.fastq &amp;gt; R0480-S0002_182602S1P2_A44944_1_H77YFDRXY_CACAGACT-TGGTACAG_R1_trimmed.fastq&#xA;cat R0480-S0002_182602S1P2_A44944_1_H77YFDRXY_CACAGACT-TGGTACAG_L00[123]_R2_trimmed.fastq &amp;gt; R0480-S0002_182602S1P2_A44944_1_H77YFDRXY_CACAGACT-TGGTACAG_R2_trimmed.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="92" LastEditorUserId="650" LastEditDate="2023-08-22T15:41:51.643" LastActivityDate="2023-08-22T15:41:51.643" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21466" PostTypeId="2" ParentId="21455" CreationDate="2023-08-22T02:27:57.617" Score="0" Body="&lt;p&gt;Set &lt;code&gt;numberEff = True&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MRcoefs(fit, coef = 2, group = 3, number = 143, eff = 5.5, numberEff = True) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I suspect you've got the quantile value set, hence it will throw an error above 1.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-08-22T13:27:35.093" LastActivityDate="2023-08-22T13:27:35.093" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21467" PostTypeId="1" CreationDate="2023-08-22T15:12:38.897" Score="0" ViewCount="53" Body="&lt;p&gt;I have a gene set containing a set of 1500 genes in which I did differential gene expression using limma voom pipeline. After this step I wanted to analyze a specific set of pathways the metabolism set from KEGG database.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;hs_kegg_df &amp;lt;- msigdbr(species = &amp;quot;Homo sapiens&amp;quot;) %&amp;gt;%&#xA; dplyr::filter(&#xA;gs_cat == &amp;quot;C2&amp;quot;, # This is to filter only to the C2 curated gene sets&#xA;gs_subcat == &amp;quot;CP:KEGG&amp;quot; # This is because we only want KEGG pathways&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then since I only want to analyze metabolism I did this (keep only the pathways envolved in metabolism):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;hs_kegg_df_with_gs_source &amp;lt;- subset(hs_kegg_df, grepl(&amp;quot;^hsa00|^hsa01|^hsa02&amp;quot;, gs_exact_source))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My problem is that, for example, for the pathway  KEGG arginine and proline metabolism the set contains 54 genes in which my kit only covers 4, and these for 4 genes are Differentially expressed yet when doing: ( I decided to do ORA because with 1500 genes GSEA is not recommended).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;em &amp;lt;- enricher(genes, minGSSize=1, maxGSSize = 20,  universe = names(gene_list), TERM2GENE=hs_kegg_df,pAdjustMethod = &amp;quot;fdr&amp;quot;)  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This set does not appear with significant p value. I also tried passing to the enricher just that pathway in particular, the p value is 1. Because in hypergeometric test, the background genes are the same as the as DGE genes in all pathways giving this way 1-0=1.&lt;/p&gt;&#xA;&lt;p&gt;How can I analyze just one pathway in particular given this specific conditions?&lt;/p&gt;&#xA;&lt;p&gt;These are the 4 genes DGE and included in my kit:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  gs_name                              gene_symbol&#xA;  &amp;lt;chr&amp;gt;                                &amp;lt;chr&amp;gt;      &#xA;1 KEGG_ARGININE_AND_PROLINE_METABOLISM ALDH2      &#xA;2 KEGG_ARGININE_AND_PROLINE_METABOLISM ARG2       &#xA;3 KEGG_ARGININE_AND_PROLINE_METABOLISM NOS1       &#xA;4 KEGG_ARGININE_AND_PROLINE_METABOLISM ODC1    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And this are all the 59.&lt;/p&gt;&#xA;&lt;p&gt;here all the genes dge and all the genes in pathway:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; genes_dge &amp;lt;- c(&amp;quot;ALDH2&amp;quot;, &amp;quot;ARG2&amp;quot;, &amp;quot;NOS1&amp;quot;, &amp;quot;ODC1&amp;quot;)&#xA; all_genes &amp;lt;- c(&amp;quot;ACY1&amp;quot;, &amp;quot;AGMAT&amp;quot;, &amp;quot;ALDH18A1&amp;quot;, &amp;quot;ALDH1B1&amp;quot;, &amp;quot;ALDH2&amp;quot;, &amp;quot;ALDH3A2&amp;quot;, &amp;quot;ALDH4A1&amp;quot;, &amp;quot;ALDH7A1&amp;quot;, &amp;quot;ALDH9A1&amp;quot;, &amp;quot;AMD1&amp;quot;, &amp;quot;AOC1&amp;quot;, &amp;quot;ARG1&amp;quot;, &amp;quot;ARG2&amp;quot;, &amp;quot;ASL&amp;quot;, &amp;quot;ASS1&amp;quot;, &amp;quot;AZIN2&amp;quot;, &amp;quot;CKB&amp;quot;, &amp;quot;CKM&amp;quot;, &amp;quot;CKMT1A&amp;quot;, &amp;quot;CKMT1B&amp;quot;,s&amp;quot;CKMT2&amp;quot;, &amp;quot;CPS1&amp;quot;, &amp;quot;DAO&amp;quot;, &amp;quot;GAMT&amp;quot;, &amp;quot;GATM&amp;quot;, &amp;quot;GLS&amp;quot;, &amp;quot;GLS2&amp;quot;, &amp;quot;GLUD1&amp;quot;, &amp;quot;GLUD2&amp;quot;, &amp;quot;GLUD2&amp;quot;, &amp;quot;GLUL&amp;quot;, &amp;quot;GOT1&amp;quot;, &amp;quot;GOT2&amp;quot;, &amp;quot;LAP3&amp;quot;, &amp;quot;MAOA&amp;quot;, &amp;quot;MAOB&amp;quot;, &amp;quot;NAGS&amp;quot;, &amp;quot;NOS1&amp;quot;, &amp;quot;NOS2&amp;quot;, &amp;quot;NOS3&amp;quot;, &amp;quot;OAT&amp;quot;, &amp;quot;ODC1&amp;quot;, &amp;quot;OTC&amp;quot;, &amp;quot;P4HA1&amp;quot;, &amp;quot;P4HA2&amp;quot;, &amp;quot;P4HA3&amp;quot;, &amp;quot;PRODH&amp;quot;, &amp;quot;PRODH2&amp;quot;, &amp;quot;PYCR1&amp;quot;, &amp;quot;PYCR2&amp;quot;, &amp;quot;PYCR3&amp;quot;, &amp;quot;PYCR3&amp;quot;, &amp;quot;SAT1&amp;quot;,&amp;quot;SAT2&amp;quot;, &amp;quot;SMS&amp;quot;, &amp;quot;SRM&amp;quot;) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18056" LastEditorUserId="73" LastEditDate="2023-08-23T00:58:59.390" LastActivityDate="2023-08-23T12:38:24.233" Title="Pathway Enrichment Analysis for one specific pathway?" Tags="|differential-expression|limma|gsea|clusterprofiler|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21469" PostTypeId="1" CreationDate="2023-08-22T18:56:39.300" Score="0" ViewCount="98" Body="&lt;p&gt;I am trying to plot by using these commands , but this is the error I am getting, and I tried multiple times, even by making the window size larger, the [lot tab is empty/blank too.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;plot1 &amp;lt;- VariableFeaturePlot(nsclc.seurat.obj)&#xA;LabelPoints(plot = plot1, points = top10, repel = TRUE)***&#xA;*When using repel, set xnudge and ynudge to 0 for optimal results&#xA;Error in grid.Call(C_convert, x, as.integer(whatfrom), as.integer(whatto),  : &#xA;  Viewport has zero dimension(s)&#xA;In addition: Warning message:&#xA;Transformation introduced infinite values in continuous x-axis* &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18061" LastEditorUserId="650" LastEditDate="2023-08-24T16:06:52.553" LastActivityDate="2023-08-24T16:06:52.553" Title="Plot error and blank tab of plot" Tags="|seurat|ggplot2|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21470" PostTypeId="1" AcceptedAnswerId="21471" CreationDate="2023-08-22T20:38:20.933" Score="0" ViewCount="56" Body="&lt;p&gt;I downloaded some raw RNA-seq reads from NCBI using the sra-toolkit (Accession numbers: SRR2053159-64). The reads were generated using ABI SOLiD, and in colour space. I downloaded them in base space in fastq format using the &lt;code&gt;-B&lt;/code&gt; option in &lt;code&gt;fastq-dump&lt;/code&gt;. But after that, when I am looking at the data after running FastQC, I find that the forward read length is 50 while the reverse reads length is 35. How is this possible? This is causing all the reverse reads to fail the filtering criteria set in Trimmomatic &lt;code&gt;MINLEN:36&lt;/code&gt;. What is the correct way to work around this?&lt;/p&gt;&#xA;&lt;p&gt;I have already set &lt;code&gt;LEADING:20 TRAILING:20 AVGQUAL:25 SLIDINGWINDOW:10:30 MINLEN:36&lt;/code&gt; Would it be a good idea to entirely remove the &lt;code&gt;MINLEN:36&lt;/code&gt;, or would it be better if I make the threshold 25?&lt;/p&gt;&#xA;&lt;p&gt;Actually I am trying to compare the transcriptomes of different bacteria under a certain stress using raw reads from NCBI. While for the rest of the set of bacteria the sequencing was done using either Illumina or Solexa, this one was done using ABI SOLiD. Is it a good idea to include this ABI SOLiD data along with the others for analysis?&lt;/p&gt;&#xA;" OwnerUserId="17761" LastEditorUserId="17761" LastEditDate="2023-08-24T15:49:16.777" LastActivityDate="2023-08-24T15:49:16.777" Title="Paired end reads with different read lengths" Tags="|rna-seq|trimming|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21471" PostTypeId="2" ParentId="21470" CreationDate="2023-08-23T00:56:12.450" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;I find that the forward read length is 50 while the reverse reads length is 35. How is this possible?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Forward cycle length was longer on SOLiD than reverse cycle length; this is common for SOLiD data. If you want to work with your data, you'll need to adjust your trimming settings appropriately to account for this (e.g. getting rid of the minimum length entirely, or setting it to a lower value). Trimmomatic does have other trimming options that are not length-based. Given the options that you've mentioned, it should be fine to just drop that minimum length parameter.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is it a good idea to include this ABI SOLiD data along with the others for analysis?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;No, especially not if you're converting to base space, then mapping. SOLiD data should be mapped in colorspace, so that small changes in base transitions don't have a huge effect on the mapped sequence. SOLiD data converted to base space sequence is not appropriate for mapping because it mixes different error modes (i.e. sequencing error, and base transition error). Bowtie (as in Bowtie v1) can map to a colorspace reference, and will correct errors based on that reference in the resultant SAM output. This means that if you trust your reference, you can use Bowtie as a data cleaner for colorspace data.&lt;/p&gt;&#xA;&lt;p&gt;However, on a practical level, the output of ABI SOLiD machines for existing datasets was low enough that (coupled with the bioinformatics issues) it makes more financial sense to resequence samples than to try to navigate around the pitfalls of colorspace representation.&lt;/p&gt;&#xA;&lt;p&gt;I have &lt;a href=&quot;https://www.seqanswers.com/forum/bioinformatics/bioinformatics-aa/38324-problem-with-solid-data?p=248208#post248208&quot; rel=&quot;nofollow noreferrer&quot;&gt;previously&lt;/a&gt; offered the following steps for dealing with colorspace data:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Transfer all the colorspace files onto an external hard disk&lt;/li&gt;&#xA;&lt;li&gt;Delete all other copies of the colorspace files&lt;/li&gt;&#xA;&lt;li&gt;Remove the hard drive from the computer&lt;/li&gt;&#xA;&lt;li&gt;Use a sledgehammer or similar to squash the disk platters closer together&lt;/li&gt;&#xA;&lt;li&gt;Withdraw \$500 from the bank&lt;/li&gt;&#xA;&lt;li&gt;Place the \$500 on top of the hard drive&lt;/li&gt;&#xA;&lt;li&gt;Return the hard drive (with the money) back to the client&lt;/li&gt;&#xA;&lt;li&gt;Report to the client that there was insufficient data for a suitable analysis, and recommend that the experiment is repeated using a base-space sequencer&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h2&gt;A More detailed Explanation&lt;/h2&gt;&#xA;&lt;p&gt;&lt;em&gt;[from &lt;a href=&quot;https://www.seqanswers.com/forum/bioinformatics/bioinformatics-aa/24489-the-subread-aligner-fast-accurate-and-scalable-read-mapping-by-seed-and-vote?p=228626#post228626&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;]&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;In colorspace sequences that are converted to base space, any error in the sequence will cause all the following bases to be incorrect. Any colour-space to base-space conversion needs to take into account (and correct) errors so that the base-space sequences are correct. When there is a sequence difference, the conversion needs to make sure that only that position is changed in the base-space version.&lt;/p&gt;&#xA;&lt;p&gt;Consider the following sequences that map to the same position:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;reference: G101320112&#xA;sequence1: X101120112&#xA;sequence2: X101312011&#xA;sequence3: X201320312&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;[I chucked an INDEL in there to make things a bit harder]&lt;/p&gt;&#xA;&lt;p&gt;A naive base-space conversion would convert these sequences as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;reference: GTTGCTTGTC&#xA;sequence1: GTTGTCCACT&#xA;sequence2: GTTGCAGGTG&#xA;sequence3: GAACGAATGA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;[apologies if my conversion is incorrect. Fixes appreciated]&lt;/p&gt;&#xA;&lt;p&gt;Very similar colour-space sequences, but very different base-space sequences.&lt;/p&gt;&#xA;&lt;p&gt;A more correct conversion would notice where the errors were in the sequences relative to the index, and modify the next colour-space base as well to something that looks appropriate:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;reference: G101320112&#xA;sequence1: X101100112&#xA;sequence2: X101313011&#xA;sequence3: X231320332&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This would end up with these converted sequences:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;reference: GTTGCTTGTC&#xA;sequence1: GTTGTTTGTC&#xA;sequence2: GTTGCATTGG&#xA;sequence3: GATGCTTATC&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which look considerably better.&lt;/p&gt;&#xA;&lt;p&gt;I hate colorspace because the conversions are very unintuitive, and difficult to explain to other people. About the only nice thing is that reverse complement is just the reverse, but this also means that aligners and assemblers need to be modified to account for that when working in colorspace (or double-encoded colorspace), and you can get weird unexpected chimeras (e.g. poly-A tails and poly-T heads merging). You can save a lot of pain and confusion by sticking with a base-space sequencer.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-08-24T04:13:37.070" LastActivityDate="2023-08-24T04:13:37.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21472" PostTypeId="2" ParentId="21467" CreationDate="2023-08-23T01:37:58.600" Score="1" Body="&lt;p&gt;Two questions&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Significance&lt;/li&gt;&#xA;&lt;li&gt;Lack of consistency between enzymes (log2) within the arginine and proline metabolism pathway.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;.  1. We've dealt with significance in a past post on Limma (it was not voom though). I'll try and find the post and link it here.&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;The answer is straight forward: whilst all four enzymes include Arginine and proline metabolism pathway&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;only ARG and ODC1 share exactly the same pathways. Thus these should give very similar results.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;NOS1 is a key enzyme in 6 different pathways.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;ALDH2 is involved in 13 different pathways.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Therefore NOS1 and ALDH2 would have &amp;quot;confounders&amp;quot; (from an experimental perspective, not host metabolism) because they are more complex than singly arginine and proline metabolism pathways.&lt;/p&gt;&#xA;&lt;p&gt;The other factor is in cancer patients the transcriptomics will get messed up, ALDH2 could go all over the place.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Prediction&lt;/em&gt;  ARG = ORD &amp;lt; NOS1 &amp;lt; ALDH2&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;**ALDH2** aldehyde dehydrogenase 2 &#xA;Pathways&#xA;Glycolysis / Gluconeogenesis&#xA;Ascorbate and aldarate metabolism&#xA;Fatty acid degradation&#xA;Valine, leucine and isoleucine degradation&#xA;Lysine degradation&#xA;**Arginine and proline metabolism**&#xA;Histidine metabolism&#xA;Tryptophan metabolism&#xA;beta-Alanine metabolism&#xA;Glycerolipid metabolism&#xA;Pyruvate metabolism&#xA;Pantothenate and CoA biosynthesis&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;**ARG** arginase&#xA;Pathways&#xA;**Arginine and proline metabolism**&#xA;Glutathione metabolism&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;**NOS1** nitric oxide synthase &#xA;Pathways&#xA;Arginine biosynthesis&#xA;**Arginine and proline metabolism**&#xA;Calcium signaling pathway&#xA;Phagosome&#xA;Apelin signaling pathway&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;**ODC1** ornithine decarboxylase 1&#xA;Pathways&#xA;**Arginine and proline metabolism**&#xA;Glutathione metabolism&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-08-23T12:38:24.233" LastActivityDate="2023-08-23T12:38:24.233" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21473" PostTypeId="1" AcceptedAnswerId="21474" CreationDate="2023-08-23T08:05:09.187" Score="1" ViewCount="140" Body="&lt;p&gt;I am visualizing the alignment of a particular genomic region across different genomes with mauve. In certain genomes the region of interest is in the negative strand and in others the positive strand. What tool can I use to reverse complement my genbank files without losing the features and annotation?&lt;/p&gt;&#xA;" OwnerUserId="9089" LastActivityDate="2023-11-03T20:07:58.160" Title="How to reverse complement a genbank file?" Tags="|genomics|genbank|mauve|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21474" PostTypeId="2" ParentId="21473" CreationDate="2023-08-23T09:04:35.773" Score="2" Body="&lt;p&gt;I have not really used Mauve, so I don't know whether it might have an in-built setting for that.&lt;/p&gt;&#xA;&lt;p&gt;SnapGene Viewer should be able to do that - it can open genbank files and export them, and View &amp;gt; Flip Sequence gets you the reverse complement without affecting the annotations. SnapGene can also do that; I only specified Viewer because it's free.&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2023-08-23T09:04:35.773" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21475" PostTypeId="1" AcceptedAnswerId="21476" CreationDate="2023-08-23T10:59:59.150" Score="2" ViewCount="83" Body="&lt;p&gt;To make a STAR reference genome used for alignment you need a genome fasta (the resources fasta provided by the Broad Institute in my case) and a GTF fasta. I tried to use the Ensembl human genome fasta fro hg38 but that does not work. I have looked in the &lt;a href=&quot;https://console.cloud.google.com/storage/browser/genomics-public-data/resources/broad/hg38/v0/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GATK resources package&lt;/a&gt; but there is no &lt;code&gt;.gtf&lt;/code&gt; file there. I really don't know what to do in this case? Is there anyone who can provide some guidance as to what &lt;code&gt;.gtf&lt;/code&gt; file I should use?&lt;/p&gt;&#xA;" OwnerUserId="17244" LastActivityDate="2023-08-26T10:10:17.280" Title="GTF To Use With Broad Institute Fasta In STAR" Tags="|star|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21476" PostTypeId="2" ParentId="21475" CreationDate="2023-08-23T11:05:38.870" Score="1" Body="&lt;p&gt;If you are looking for a genome and its corresponding annotations, you can use the following, it is from &lt;code&gt;nf-core&lt;/code&gt; (&lt;a href=&quot;https://nf-co.re/docs/usage/reference_genomes&quot; rel=&quot;nofollow noreferrer&quot;&gt;link to the original doc&lt;/a&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA; &#xA;VERSION=108&#xA;wget -L ftp://ftp.ensembl.org/pub/release-&lt;span class=&quot;math-container&quot;&gt;$VERSION/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna_sm.primary_assembly.fa.gz&#xA;wget -L ftp://ftp.ensembl.org/pub/release-$&lt;/span&gt;VERSION/gtf/homo_sapiens/Homo_sapiens.GRCh38.$VERSION.gtf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;BTW the most up-to-date version as of today is 109.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2023-08-23T11:05:38.870" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21478" PostTypeId="1" CreationDate="2023-08-23T10:39:13.323" Score="2" ViewCount="119" Body="&lt;p&gt;I wrote the .yaml file which was according the instruction from zUMIs:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;###########################################&#xA;#Welcome to zUMIs&#xA;#below, please fill the mandatory inputs&#xA;#We expect full paths for all files.&#xA;###########################################&#xA;&#xA;#define a project name that will be used to name output files&#xA;project: SmartSeq3&#xA;&#xA;#Sequencing File Inputs:&#xA;#For each input file, make one list object &amp;amp; define path and barcode ranges &#xA;#base definition vocabulary: BC(n) UMI(n) cDNA(n).&#xA;#Barcode range definition needs to account for all ranges.You can  give several comma-separated ranges for BC &amp;amp; UMI sequences, eg.BC(1-6,20-26)&#xA;#you can specify between 1 and 4 input files&#xA;&#xA;sequence_files:&#xA;  file1:&#xA;    name: /home/isidora/IRCCS_Candiolo/Fastq/*_R1_001_val_1.fq ##/home/isidora/IRCCS_Candiolo/Fastq/zUMIsFiles1/ #path to first file&#xA;    base_definition:&#xA;      - cDNA(1-150)&#xA;      - BC(1-8)&#xA;      - UMI(12-19)&#xA;  file2:&#xA;    name: /home/isidora/IRCCS_Candiolo/Fastq/*_R2_001_val_2.fq ## /home/isidora/IRCCS_Candiolo/Fastq/zUMIsFiles2/ #path to second file&#xA;    base_definition:&#xA;      - cDNA(1-150)&#xA;      - BC(1-8)&#xA;      - UMI(12-19)&#xA;&#xA;#reference genome setup&#xA;reference:&#xA;  STAR_index: /home/isidora/IRCCS_Candiolo/Fastq/ #path to STAR genome index&#xA;  GTF_file: /home/isidora/IRCCS_Candiolo/Fastq/hg38.refGene.gtf #path to gene annotation file in GTF format&#xA;  exon_extension: no #extend exons by a certain width?&#xA;  extension_length: 0 #number of bp to extend exons by&#xA;  scaffold_length_min: 0 #minimal scaffold/chromosome length to consider (0 = all)&#xA;  additional_files: /home/isidora/IRCCS_Candiolo/Fastq/hg38.fa #Optional parameter. It is possible to give additonal reference sequences here, eg ERCC.fa&#xA;  additional_STAR_params: pGe.sjdbOverhang &amp;gt; 0 #Optional parameter. you may add custom mapping parameters to STAR here&#xA;  #output directory&#xA;out_dir: /home/isidora/IRCCS_Candiolo/Fastq/zUMIs/ #specify the full path to the output directory&#xA;&#xA;###########################################&#xA;#below, you may optionally change default parameters&#xA;###########################################&#xA;&#xA;#number of processors to use&#xA;num_threads: 20&#xA;mem_limit: null #Memory limit in Gigabytes, null meaning unlimited RAM usage&#xA;#barcode &amp;amp; UMI filtering options&#xA;#number of bases under the base quality cutoff that should be filtered out.&#xA;#Phred score base-cutoff for quality control.&#xA;filter_cutoffs:&#xA;  BC_filter:&#xA;    num_bases: 3&#xA;    phred: 20&#xA;  UMI_filter:&#xA;    num_bases: 2&#xA;    phred: 20&#xA;&#xA;#Options for Barcode handling&#xA;#You can give either number of top barcodes to use or give an annotation of cell barcodes&#xA;#If you leave both barcode_num and barcode_file empty, zUMIs will perform automatic cell barcode selection for you!&#xA;barcodes:&#xA;  barcode_num: null&#xA;  barcode_file: /home/isidora/IRCCS_Candiolo/Fastq/SampleSheet.txt&#xA;  barcode_sharing: null #Optional for combining several barcode sequences per cell (see github wiki)  &#xA;  automatic: yes #Give yes/no to this option. If the cell barcodes should be detected automatically. If the barcode file is given in combination with automatic barcode detection, the list of giiven barcodes will be used as whitelist.&#xA;  BarcodeBinning: 1 #Hamming distance binning of close cell barcode sequences.  &#xA;  nReadsperCell: 100 #Keep only the cell barcodes with atleast n number of reads.  &#xA;  demultiplex: yes #produce per-cell demultiplexed bam files.&#xA;&#xA;#Options related to counting of reads towards expression profiles&#xA;counting_opts:&#xA;  introns: yes #can be set to no for exon-only counting.&#xA;  intronProb: no #perform an estimation of how likely intronic reads are to be derived from mRNA by comparing to intergenic counts.&#xA;  downsampling: '0' #Number of reads to downsample to. This value can be a fixed number of reads (e.g. 10000) or a desired range (e.g  10000-20000) Barcodes with less than &amp;lt;d&amp;gt; will not be reported. 0 means adaptive downsampling. Default: 0&#xA;  strand: 0 #Is the library stranded? 0 = unstranded, 1 = positively stranded, 2 = negatively stranded &#xA;  Ham_Dist: 1 #Hamming distance collapsing of UMI sequences.&#xA;  velocyto: no #Would you like velocyto to do counting of intron-exon spanning reads  &#xA;  primaryHit: yes #Do you want to count the primary Hits of multimapping reads towards gene expression levels?&#xA;  multi_overlap: no #Do you want to assign reads overlapping to multiple features?  &#xA;  fraction_overlap: 0 #minimum required fraction of the read overlapping with the gene for read assignment to genes  &#xA;  twoPass: yes #perform basic STAR twoPass mapping&#xA;&#xA;#produce stats files and plots?&#xA;make_stats: yes&#xA;&#xA;&#xA;#Start zUMIs from stage. Possible TEXT(Filtering, Mapping, Counting,Summarising). Default: Filtering.&#xA;which_Stage: Filtering&#xA;&#xA;#define dependencies program paths&#xA;samtools_exec: samtools #samtools executable&#xA;Rscript_exec: Rscript #Rscript executable&#xA;STAR_exec: STAR #STAR executable&#xA;pigz_exec: pigz #pigz executable&#xA;&#xA;#below, fqfilter will add a read_layout flag defining SE or PE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I start the program with this code &lt;code&gt;zUMIs/zUMIs.sh -c -y ~/IRCCS_Candiolo/Fastq/zUMIs/zUMIs.yaml&lt;/code&gt;, I get errors:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Using miniconda environment for zUMIs!&#xA; note: internal executables will be used instead of those specified in the YAML file!&#xA;&#xA;&#xA; You provided these parameters:&#xA; YAML file:     /home/isidora/IRCCS_Candiolo/Fastq/zUMIs/zUMIs.yaml&#xA; zUMIs directory:               /home/isidora/IRCCS_Candiolo/Fastq/zUMIs&#xA; STAR executable                STAR&#xA; samtools executable            samtools&#xA; pigz executable                pigz&#xA; Rscript executable             Rscript&#xA; RAM limit:   null&#xA; zUMIs version 2.9.7e&#xA;&#xA;&#xA;Wed Aug 23 12:06:25 CEST 2023&#xA;WARNING: The STAR version used for mapping is 2.7.3a and the STAR index was created using the version 2.7.4a. This may lead to an error while mapping. If you encounter any errors at the mapping stage, please make sure to create the STAR index using STAR 2.7.3a.&#xA;Filtering...&#xA;sh: 1: Syntax error: Unterminated quoted string&#xA;sh: 1: Syntax error: Unterminated quoted string&#xA;Wed Aug 23 12:06:26 CEST 2023&#xA;Error in eval(bysub, parent.frame(), parent.frame()) :&#xA;  object 'XC' not found&#xA;Calls: cellBC -&amp;gt; [ -&amp;gt; [.data.table -&amp;gt; eval -&amp;gt; eval&#xA;In addition: Warning message:&#xA;In data.table::fread(bccount_file, header = FALSE, col.names = c(&amp;quot;XC&amp;quot;,  :&#xA;  File '/home/isidora/IRCCS_Candiolo/Fastq/zUMIs//SmartSeq3.BCstats.txt' has size 0. Returning a NULL data.table.&#xA;Execution halted&#xA;Mapping...&#xA;[1] &amp;quot;2023-08-23 12:06:28 CEST&amp;quot;&#xA;Warning message:&#xA;In data.table::fread(cmd = paste(samtools, &amp;quot;view&amp;quot;, filtered_bams[1],  :&#xA;  File '/tmp/RtmpKSSqU2/file57f5393a8fde' has size 0. Returning a NULL data.table.&#xA;&#xA;EXITING because of fatal PARAMETERS error: pGe.sjdbOverhang &amp;lt;=0 while junctions are inserted on the fly with --sjdbFileChrStartEnd or/and --sjdbGTFfile&#xA;SOLUTION: specify pGe.sjdbOverhang&amp;gt;0, ideally readmateLength-1&#xA;Aug 23 12:07:18 ...... FATAL ERROR, exiting&#xA;Wed Aug 23 12:07:18 CEST 2023&#xA;Counting...&#xA;[1] &amp;quot;2023-08-23 12:07:30 CEST&amp;quot;&#xA;Error in fread(paste0(opt$out_dir, &amp;quot;/zUMIs_output/&amp;quot;, opt$project, &amp;quot;kept_barcodes_binned.txt&amp;quot;)) :&#xA;  File '/home/isidora/IRCCS_Candiolo/Fastq/zUMIs//zUMIs_output/SmartSeq3kept_barcodes_binned.txt' does not exist or is non-readable. getwd()=='/home/isidora/IRCCS_Candiolo/Fastq/zUMIs'&#xA;Execution halted&#xA;Wed Aug 23 12:07:30 CEST 2023&#xA;Loading required package: yaml&#xA;Loading required package: Matrix&#xA;[1] &amp;quot;loomR found&amp;quot;&#xA;Error in gzfile(file, &amp;quot;rb&amp;quot;) : cannot open the connection&#xA;Calls: rds_to_loom -&amp;gt; readRDS -&amp;gt; gzfile&#xA;In addition: Warning message:&#xA;In gzfile(file, &amp;quot;rb&amp;quot;) :&#xA;  cannot open compressed file '/home/isidora/IRCCS_Candiolo/Fastq/zUMIs//zUMIs_output/expression/SmartSeq3.dgecounts.rds', probable reason 'No such file or directory'&#xA;Execution halted&#xA;Wed Aug 23 12:07:32 CEST 2023&#xA;Descriptive statistics...&#xA;[1] &amp;quot;I am loading useful packages for plotting...&amp;quot;&#xA;[1] &amp;quot;2023-08-23 12:07:33 CEST&amp;quot;&#xA;Error in data.table::fread(paste0(opt$out_dir, &amp;quot;/zUMIs_output/&amp;quot;, opt$project,  :&#xA;  File '/home/isidora/IRCCS_Candiolo/Fastq/zUMIs//zUMIs_output/SmartSeq3kept_barcodes.txt' does not exist or is non-readable. getwd()=='/home/isidora/IRCCS_Candiolo/Fastq/zUMIs'&#xA;Execution halted&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am new at bioinformatic field and I do not understand the meaning of these errors and the things I should correct.&#xA;Can anyone help me please?&lt;/p&gt;&#xA;" OwnerUserId="18069" OwnerDisplayName="Isidora Pantovic" LastEditorUserId="3694" LastEditDate="2023-08-23T18:03:07.153" LastActivityDate="2023-08-24T19:35:34.050" Title="Analysis of RNA sequencing with zUMIs" Tags="|command-line|bash|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21479" PostTypeId="1" CreationDate="2023-08-23T18:25:12.883" Score="2" ViewCount="102" Body="&lt;p&gt;I'm working on an ML model to predict colorectal cancer using data from the human microbiome (collected from fecal samples).&lt;/p&gt;&#xA;&lt;p&gt;My dataset is made up of metadata (containing phenotypic information on the N=156 population on which the study is being carried out,&#xA;&lt;a href=&quot;https://i.stack.imgur.com/MkOVB.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MkOVB.png&quot; alt=&quot;Metadata features&quot; /&gt;&lt;/a&gt; and an OTU table &lt;a href=&quot;https://i.stack.imgur.com/AArhv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AArhv.png&quot; alt=&quot;OTU table have 129 column containing SUBJECT IDs and the 1st column contain taxa&quot; /&gt;&lt;/a&gt;containing a taxonomy of bacteria found and their abundance for each sample from one person (each person has one and only one sample).&lt;/p&gt;&#xA;&lt;p&gt;I filtered the OTU table by eliminating sparse data (I eliminated OTUs below a certain threshold) after which I calculated the relative abundance for each OTU.&lt;br /&gt;&#xA;I deleted samples with missing values.&lt;br /&gt;&#xA;I made the metadata join (Subject ID    Sample ID    Age (years)    Gender    BMI (kg/m²)    Country of Residence    Diagnosis    TNM Stage    AJCC Stage    Localization) and the OTU table.&lt;/p&gt;&#xA;&lt;p&gt;In the end, I ended up with 114 rows × 221 columns .&lt;/p&gt;&#xA;&lt;p&gt;I normalized the data and then built an SVM(SVC) model with the rbf kernel with C=0.05 and gamma=0.001 I got&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Training Accuracy: 0.967032967032967&#xA;Test Accuracy: 0.782608695652174&#xA;SVC(AUC=0.64)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I try to optimize the model using the code below, the values I get for gamma and C give me overfitting (trainset accuracy is 100%).&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;# import packages&#xA;from sklearn.svm import SVC&#xA;from sklearn.model_selection import GridSearchCV&#xA;# range of values to try for gamma and C&#xA;Cs = [0.00001,0.0001,0.001,0.1,1,2,5,10,100,1000]&#xA;gammas = [0.00001,0.0001,0.001,0.1,1,10,100,10000] &#xA;# number of segments for cross validation&#xA;nfolds = 5 # number of segments for class validation&#xA;param_grid = {'C':Cs ,'gamma': gammas}&#xA;# search for the best parameters using cross-validation &#xA;grid_search = GridSearchCV(SVC(kernel='rbf'),param_grid,cv=nfolds)&#xA;grid_search.fit(X_train,y_train)&#xA;grid_search.best_params_&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18044" LastEditorUserId="18044" LastEditDate="2023-08-25T11:44:07.053" LastActivityDate="2024-01-25T23:08:46.133" Title="Model optimization causes overfitting" Tags="|machine-learning|modelling|microbial-genomics|microbiome|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21480" PostTypeId="2" ParentId="21387" CreationDate="2023-08-23T21:58:17.010" Score="1" Body="&lt;p&gt;Just answering, following extensive back and forth in the comments.&lt;/p&gt;&#xA;&lt;p&gt;I think that your reads are fine:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;FASTQC doesn't see anything too weird. You see some noise at the first few basecalls but I am not too alarmed about that.&lt;/li&gt;&#xA;&lt;li&gt;FASTQC finds overrepresented only a &lt;a href=&quot;https://www.nature.com/articles/s41598-021-92705-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;telomeric repeat&lt;/a&gt; and its reverse complement (TTAGG). That is something you'd expect to find in WGS reads.&lt;/li&gt;&#xA;&lt;li&gt;Mash screen finds that the only match, with a small but non-trivial number of k-mer matches, is the monarch butterfly &lt;em&gt;Danaus plexippus&lt;/em&gt;, e.g. another lepidoptera. So that makes sense.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Therefore, the issue is likely that your reference genome has issues or has been swapped out with something else.&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Where did your reference assembly come from?&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Did it get mixed up somehow?&lt;/li&gt;&#xA;&lt;li&gt;What does it look like when you run &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwjh8vLv4vOAAxWbMDQIHSP9AnAQFnoECA0QAQ&amp;amp;url=https%3A%2F%2Fquast.sourceforge.net%2F&amp;amp;usg=AOvVaw0icJdHdIreg_bswF_XMDH3&amp;amp;opi=89978449&quot; rel=&quot;nofollow noreferrer&quot;&gt;QUAST&lt;/a&gt;?&lt;/li&gt;&#xA;&lt;li&gt;What does it look like when you run &lt;code&gt;mash dist&lt;/code&gt; of your assembly against the same mash database? Is it maybe a bunch of endosymbionts or other contaminants?&lt;/li&gt;&#xA;&lt;li&gt;What happens if you map your reads against a high-quality lepidopteran assembly like the monarch butterfly or &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/35294789/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this&lt;/a&gt;?&lt;/li&gt;&#xA;&lt;li&gt;What happens if you map your assembly against a high-quality lepidopteran assembly? Is it syntenic?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-08-23T21:58:17.010" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21481" PostTypeId="2" ParentId="21277" CreationDate="2023-08-23T22:54:34.387" Score="1" Body="&lt;p&gt;This gene/protein appears to have been described exclusively as an isozyme / electrophoretic mobility assay of a protein.&lt;/p&gt;&#xA;&lt;p&gt;See for example &lt;a href=&quot;https://www.jax.org/strain/000824&quot; rel=&quot;nofollow noreferrer&quot;&gt;this PepC mutant page&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The Pepc locus controls electrophoretic mobility of the enzyme peptidase C of red blood cells. This LT/J congenic strain carries the Pepca allele from C57BL/6 that determines a slow anodally migrating electrophoretic band.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The relevant publications describing mutants are from the 70s and 80s, e.g. &lt;a href=&quot;https://www.informatics.jax.org/reference/allele/MGI:4819771?typeFilter=Literature&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, when there were no genomic resources for mice.&lt;/p&gt;&#xA;&lt;p&gt;However, linkage mapping of electrophoretic variants could be used to establish linkage to loci on chromosome 1, apparently.&lt;/p&gt;&#xA;&lt;p&gt;Very likely PepC is encoded by &lt;em&gt;some&lt;/em&gt; annotated gene on chr1, but no one knows which one exactly because they don't really know much about the protein other than its electrophoretic variability. But that's just a guess by me.&lt;/p&gt;&#xA;&lt;p&gt;When the databases were set up, they had to accommodate 1) mouse mutant strains, and 2) published articles referencing the protein, so they did their best.&lt;/p&gt;&#xA;&lt;p&gt;However, no one ever followed up to link everything together, so we just have these orphan entries.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-08-23T22:54:34.387" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21482" PostTypeId="2" ParentId="21478" CreationDate="2023-08-24T06:03:44.193" Score="3" Body="&lt;p&gt;I'd advise you to keep away from ZUMIs unless you're doing the same workflow that it was designed for. I spent many days trying to sort out issues with that program, and eventually gave up; it wasn't created with the expectation of errors or unexpected input, and had too many programming minefields for the data I was working with.&lt;/p&gt;&#xA;&lt;p&gt;ZUMIs should be stopping on the first error and explaining what's wrong. As demonstrated by your situation, it's not; it's trying to keep on processing, barreling through and dragging errors along for the ride. It's not exactly beginner-friendly code.&lt;/p&gt;&#xA;&lt;p&gt;In the end, I changed to using Salmon/Alevin, which had a more fail-resistant interface, and (eventually) similar barcode flexibility:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://combine-lab.github.io/alevin-fry-tutorials/2021/custom-geo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://combine-lab.github.io/alevin-fry-tutorials/2021/custom-geo/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-08-24T19:35:34.050" LastActivityDate="2023-08-24T19:35:34.050" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21483" PostTypeId="2" ParentId="21475" CreationDate="2023-08-24T06:15:12.023" Score="1" Body="&lt;p&gt;I've had success in the past generating STAR indexes using the Gencode GTF files available from here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.gencodegenes.org/human/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.gencodegenes.org/human/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Comprehensive gene annotation (ALL regions)&lt;/li&gt;&#xA;&lt;li&gt;Genome sequence (ALL)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If you make sure that you're getting both the genome and the GTF files from the same source, it's more likely that they will work well together.&lt;/p&gt;&#xA;&lt;p&gt;Note that the ALL annotation includes assembly patches and alternate haplotypes. The alternate haplotypes mean that population variation is captured, which usually increases the alignment rate of reads.&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, most bioinformatics workflows don't properly handle alternate assemblies. As one example, reads aligning to those areas may end up classified as mapping to multiple genomic locations, even though they actually aren't. So it's a tradeoff of a more accurate alignment vs more hassle dealing with those alignments downstream.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-08-26T10:10:17.280" LastActivityDate="2023-08-26T10:10:17.280" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21486" PostTypeId="1" CreationDate="2023-08-24T22:59:04.350" Score="0" ViewCount="48" Body="&lt;p&gt;I'm editing a time-scaled tree using ggtree. The dates are correctly adjusted when I apply the 'mrsd' command and 'theme_tree2()', but when I try to rotate the tree with 'layout_dendrogram()', the scale still remains on the x-axis with no dates. Is there a way to place the date scale on the y-axis in a tree rotated with 'layout_dendrogram()'?&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# Code without layout_dendrogram()&#xA;ggtree(my_tree, mrsd='2022-12-31') %&amp;lt;+% &#xA;  metadato_arbol + &#xA;  geom_tippoint(size = 2) + &#xA;  theme_tree2()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/X6tco.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/X6tco.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# Using layout_dendrogram()&#xA;ggtree(my_tree, mrsd='2022-12-31') %&amp;lt;+% &#xA;  metadato_arbol + &#xA;  geom_tippoint(size = 2) + &#xA;  theme_tree2() + layout_dendrogram()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/71BGk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/71BGk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12707" LastEditorUserId="650" LastEditDate="2023-08-25T14:01:54.893" LastActivityDate="2024-01-22T14:08:36.007" Title="Scale on the y-axis in a tree rotated with 'layout_dendrogram()' using ggtree" Tags="|r|phylogenetics|ggtree|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21487" PostTypeId="2" ParentId="21486" CreationDate="2023-08-25T09:16:26.440" Score="0" Body="&lt;p&gt;Maybe try the following via gridExtra,&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(gridExtra)&#xA;library(ggtree)&#xA;library(ape)&#xA;set.seed(2023-08-23)&#xA;my_tree &amp;lt;- rtree(30)&#xA;p &amp;lt;- ggtree(my_tree, mrsd='2022-12-31') %&amp;lt;+% &#xA;  metadato_arbol + &#xA;  geom_tippoint(size = 2) + &#xA;  theme_tree2() # + geom_text(aes(label=node))&#xA;gridExtra::grid.arrange(p, p %&amp;gt;% rotate(33) %&amp;gt;% rotate(52), ncol=2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="650" LastEditDate="2023-08-25T14:01:44.147" LastActivityDate="2023-08-25T14:01:44.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21488" PostTypeId="2" ParentId="21479" CreationDate="2023-08-25T10:34:32.957" Score="0" Body="&lt;p&gt;You are missing a validation split triggering data leakage. The error is here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grid_search.fit(X_train,y_train)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can't use &lt;code&gt;X_train&lt;/code&gt; and &lt;code&gt;y_train&lt;/code&gt; here. The correction is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grid_search.fit(X_val,y_val)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Generating &lt;code&gt;X_val&lt;/code&gt; and &lt;code&gt;y_val&lt;/code&gt; is described below.&lt;/p&gt;&#xA;&lt;p&gt;The code used to generate&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Training Accuracy: 0.967032967032967&#xA;Test Accuracy: 0.782608695652174&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;...  included one &lt;code&gt;test_split_train&lt;/code&gt; and you need to perform a second &lt;code&gt;test_split_train&lt;/code&gt; to obtain &lt;code&gt;X_val&lt;/code&gt;, and &lt;code&gt;y_val&lt;/code&gt;. The original code gave a pretty good as a result, the training score will always be higher.&lt;/p&gt;&#xA;&lt;p&gt;The diagnosis of the code presented is there is 100% data leakage. What is missing is a second &lt;code&gt;test_train_split&lt;/code&gt;. Again one &lt;code&gt;test_train_split&lt;/code&gt; is present in the original code and there would have been a,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;from sklearn.model_selection import train_test_split&#xA;&#xA;X = # a dataframe containing the features remove the target y (below)&#xA;y = # a same dataframe but a single column which is the target&#xA;X_train,X_test,y_train,y_test = train_test_split(X,y,test_size=0.2,random_state=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above will give a training proportion at 80% and test proportion at 20%. It's called an 80:20 split. That code will have been in the original code for the first calculation, I know that because the results are pretty standard for a routine ML &lt;code&gt;test_train_split&lt;/code&gt; operation. Whats happening is the training set is being held absolutely separately to the test set. There is 100% compartmentalisation between the two data sets. Thats critical because if they got confused the calculation falls over, you will be testing the exact data thats just been trained. That confusion would be term &amp;quot;data leakage&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;In the code you presented what has happened is the same data used for parameterisation is exactly the same data for training (&lt;code&gt;X_train&lt;/code&gt;, &lt;code&gt;y_train&lt;/code&gt;). Again it's called data leakage and parameter grids are very sensitive to this.&lt;/p&gt;&#xA;&lt;p&gt;To correct this when a parameter grid is used it must have its own data set. That is instead of having a &lt;code&gt;X_train,X_test,y_train,y_test&lt;/code&gt; the &lt;code&gt;test_train_split&lt;/code&gt; is modified to generate &lt;code&gt;X_train,X_test,y_train,y_test, X_val, y_val&lt;/code&gt;. The code to generate this is,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;from sklearn.model_selection import train_test_split&#xA; X_train, X_test, y_train, y_test &#xA;    = train_test_split(X, y, test_size=0.2, random_state=1)&#xA;&#xA; X_train, X_val, y_train, y_val &#xA;    = train_test_split(X_train, y_train, test_size=0.25, random_state=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This approach is singly leveraging &lt;code&gt;test_train_split&lt;/code&gt; by splitting the training data set twice. This example creates a 60:20:20 split. Some say that 70:20:10 split is used for a parameter split (10 for the parameter search). Matter of opinion (you just adjust the proportions).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Thus&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;the grid search is strictly for the parameter split (X_val, y_val)&lt;/li&gt;&#xA;&lt;li&gt;and those parameters at then using in training split (X_train, y_train)&lt;/li&gt;&#xA;&lt;li&gt;the testing is then performed (X_test, y_test)&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;What this process does is ensure no part of &lt;code&gt;X_val&lt;/code&gt;, &lt;code&gt;y_val&lt;/code&gt; is ever seen in the training set. If that happens again - data leakage. Here only the optimised parameters are seen by the training set.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; Introducing validation parameter split into the calculation isn't trivial and there is nothing wrong with your original result. Thus you don't necessarily need it. Generally speaking 80% accuracy is fine, yours is 1% lower (can be overlooked).&lt;/p&gt;&#xA;&lt;p&gt;Parameter tuning will increase the accuracy score but it will be 85% rather than 79% and if it's really gone well then it might be 90%. What I'm saying is it will improve things, but will not absolutely solve the accuracy issue, it's just a general and accepted issue within ML.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Just to finally add there are biological reasons in cancer prediction why accuracy is a key parameter, but thats a different question.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Comments: causes lower scores&lt;/p&gt;&#xA;&lt;p&gt;What I'm saying the correct approach. I recognise that &lt;a href=&quot;https://www.geeksforgeeks.org/svm-hyperparameter-tuning-using-gridsearchcv-ml/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Geek for Geeks&lt;/strong&gt;&lt;/a&gt; uses your approach - but they are wrong.&lt;/p&gt;&#xA;&lt;p&gt;A separate validation split will certainly give result in lower scores than &lt;code&gt;grid_search.fit(X_train,y_train)&lt;/code&gt; thats inevitable, but it is correct in stopping overfitting.&lt;/p&gt;&#xA;&lt;p&gt;If the validation split causes its lower accuracy than &lt;code&gt;0.8&lt;/code&gt; I'd look at &lt;code&gt;gammas = ... 0.0001,0.001,0.1 ...&lt;/code&gt; because you have missed &lt;code&gt;0.01&lt;/code&gt; and that will affect your model. It shouldn't give a lower score than the default parameters - it's not possible and something is wrong with the code and  application of parameters.&lt;/p&gt;&#xA;&lt;p&gt;Once you've the initial parameters you can refine the parameter search around the initial values - nothing wrong with that and that can even be coded (automated). The only minor issue is reducing the size of the training split (60% is a bit low). That can cause issues in which case the split is 70:20:10, i.e. 10 for validation.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-08-28T21:11:11.087" LastActivityDate="2023-08-28T21:11:11.087" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21489" PostTypeId="2" ParentId="21427" CreationDate="2023-08-25T21:21:53.853" Score="1" Body="&lt;p&gt;I would suggest making a &lt;a href=&quot;https://dgenies.toulouse.inra.fr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dot plot of your assembly&lt;/a&gt; against a reference (related) yeast genome assembly.&lt;/p&gt;&#xA;&lt;p&gt;I'd suggest also running &lt;a href=&quot;https://github.com/ablab/quast&quot; rel=&quot;nofollow noreferrer&quot;&gt;QUAST&lt;/a&gt; against the yeast reference to get a finer grained idea of the comparison.&lt;/p&gt;&#xA;&lt;p&gt;Bandage is more of an assembly QC tool for understanding whether there are issues with the assembly. It will not tell you anything about ploidy or that kind of thing.&lt;/p&gt;&#xA;&lt;p&gt;Off the top of my head, I find it worrisome that 90% of your sequence is in a single scaffold/contig. Presumably there's more than a single chromosome. It does look like the repeats are messing you up.&lt;/p&gt;&#xA;&lt;p&gt;I'd suggest using &lt;a href=&quot;https://github.com/chhylp123/hifiasm&quot; rel=&quot;nofollow noreferrer&quot;&gt;hifiasm&lt;/a&gt; instead and seeing if it makes different decisions.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-08-25T21:21:53.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21490" PostTypeId="1" CreationDate="2023-08-25T22:16:34.023" Score="0" ViewCount="91" Body="&lt;p&gt;I've got a moderately large set of PBMCs, over 1M cells. That means I can't easily do a grid search of dimensionality reduction/clustering parameters/methods. Some examples results I'm getting with scanpy default parameters include PCA -- a big, dense grouping with lobes radiating outwards, and UMAP -- &amp;quot;better&amp;quot; groupings in some sense, but many many tiny clusters of cells, more than are biologically plausible (hundreds) as well.&lt;/p&gt;&#xA;&lt;p&gt;Basically I'm looking for best dimred/clustering practices for such a dataset, other than what's on offer from scanpy/Seurat. My metrics such as they are are that the major cell types should be clearly separated, and that the overall plot have some biological plausibility in terms of number of visually distinct clusters.&lt;/p&gt;&#xA;&lt;p&gt;I'm aware of the Xiang et al Frontier Genetics 2021 paper, but the number of cells their example datasets use is much smaller AFAIK. They concluded that tSNE is the best choice of method according to their metrics, though UMAP also has some stability advantages. The Chari and Pachter PLOS Comp. Bio. 2023 paper raises some good points (a highly dimensional space can't really accurately have its relations preserved in 2 dimensions, and there are other issues of noise, interpretability, and so on. They recommend clustering from the higher dimensional data, which I do agree with. A different paper rec could be great.&lt;/p&gt;&#xA;&lt;p&gt;PS: SCVI seems like a good option, I'm playing with that at the moment.&lt;/p&gt;&#xA;" OwnerUserId="18080" LastEditorUserId="3848" LastEditDate="2023-08-30T14:42:16.053" LastActivityDate="2024-01-27T15:08:00.903" Title="scRNA: What are good dimensionality reduction/clustering parameters to get biologically plausible groupings?" Tags="|rna-seq|single-cell|clustering|pca|dimensionality-reduction|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21491" PostTypeId="2" ParentId="13255" CreationDate="2023-08-26T04:24:22.900" Score="1" Body="&lt;p&gt;This worked for me today:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# Load required packages&#xA;BiocManager::install(&amp;quot;Biostrings&amp;quot;)&#xA;&#xA;library(Biostrings)&#xA;&#xA;# Read the multi-genome FASTA file&#xA;multi_genome_fasta &amp;lt;- readDNAStringSet(&amp;quot;*.fasta&amp;quot;)&#xA;&#xA;# Extract individual genomes and save as separate FASTA files&#xA;for (i in 1:length(multi_genome_fasta)) {&#xA;    header &amp;lt;- names(multi_genome_fasta)[i]&#xA;    sequence &amp;lt;- as.character(multi_genome_fasta[i])&#xA;    &#xA;    # Combine header and sequence&#xA;    combined_data &amp;lt;- paste(&amp;quot;&amp;gt;&amp;quot;, header, sequence)&#xA;    &#xA;    # Save the combined data as a separate FASTA file&#xA;    file_name &amp;lt;- paste0(header, &amp;quot;.fasta&amp;quot;)&#xA;    writeLines(combined_data, file_name)&#xA;    &#xA;    cat(&amp;quot;Saved&amp;quot;, header, &amp;quot;as&amp;quot;, file_name, &amp;quot;\n&amp;quot;)&#xA;}&#xA;&#xA;cat(&amp;quot;Separation completed.\n&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18081" LastEditorUserId="650" LastEditDate="2023-08-26T21:00:23.683" LastActivityDate="2023-08-26T21:00:23.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21492" PostTypeId="1" AcceptedAnswerId="21501" CreationDate="2023-08-26T14:38:23.960" Score="2" ViewCount="42" Body="&lt;p&gt;All, I have a workflow that in some rules requires the sample's read-lengths. For this I always ran a script that would parse all fastq files and would add the read lengths to the sample table. Rules could then take the read lengths for each sample from there during execution.&lt;/p&gt;&#xA;&lt;p&gt;However, I recently made a workflow/pipeline that downloads files from SRA, so I need to compute the read lengths at a later stage during workflow execution (after prefetch/fasterq-dump).&lt;/p&gt;&#xA;&lt;p&gt;I added a step to update the sample table, however, I think the sample table is only read at the very beginning of the workflow.&lt;/p&gt;&#xA;&lt;p&gt;What would be the nicest way, the Snakemake preferred way, to re-read the samples.tsv file before a rule is executed?&lt;/p&gt;&#xA;&lt;p&gt;In general I like storing meta data in the sample table so, I'll probably use this mechanism more often in the future, if I find a nice methods to re-parse the samples table.&lt;/p&gt;&#xA;&lt;p&gt;Right now I do this in rules:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;rule deletion_profile:&#xA;    input:&#xA;        rseqc_input_bam,&#xA;        rules.add_max_read_length.output.done_file&#xA;    output:&#xA;        os.path.join(rseqc_dir, '{sample}.deletion_profile.txt')&#xA;    conda:&#xA;        &amp;quot;../envs/rseqc.yaml&amp;quot;&#xA;    params:&#xA;        read_length = lambda wildcards: samples.loc[wildcards.sample, 'read_length']&#xA;    shell:&#xA;        '''&#xA;        deletion_profile.py -i {input} -l {params.read_length} -o {rseqc_dir}/{wildcards.sample}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thanx.&lt;/p&gt;&#xA;" OwnerUserId="3964" LastActivityDate="2023-08-29T15:45:40.870" Title="Snakemake: Re-read the samples.tsv file midway through the workflow" Tags="|snakemake|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21493" PostTypeId="2" ParentId="21490" CreationDate="2023-08-27T18:01:01.637" Score="0" Body="&lt;p&gt;&lt;strong&gt;Aim&lt;/strong&gt; It's important to define what all these calculations are doing: they're attempting to extract maximum &amp;quot;multi-dimensional&amp;quot; variance from a data set. The key thing here is that the methodology is old, but it is very well established (long before RNAseq ever existed). I get the point empirical stats is boring, but it's essential to consider whats the underbelly of the methods being used.&lt;/p&gt;&#xA;&lt;p&gt;To answer the question directly&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;PCA+tSNE is worth considering&lt;/li&gt;&#xA;&lt;li&gt;SCVI-tools is in a different league, and totalIV looks very interesting (what I'm tentatively recommending).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;em&gt;Note, the commercial sector is often more advanced than public sector: I developed ML methods which intersect it with classical approaches for this basic style of calculation. The scvi-tools however is &amp;quot;full-on&amp;quot; via a DL approach.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Note2, I've not read the papers cited except the scvi-tools documentation&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Data set size&lt;/strong&gt; I don't do PMBC calculations - I do perform PMBC &lt;em&gt;post-hoc&lt;/em&gt; analytics BTW. I personally don't see the data set size as being prohibitive for an unsupervised learning calculation. Pyscan should be able to handle this.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;PCA&lt;/strong&gt; is the goto approach stage 1 - but there's a caveat, which is what you're hinting at. First, it's how much variance is trapped within each component. Scanpy will permit this calculation. What you want is &amp;gt;70% variance, classical stats for example PC1+PC2+PC3+PC4 &amp;gt; 70%. However, from past posts on this site then for PMBCs this might be unattainable, like no-can- exceed the variance threshold for lots and lots of PCs. That is an issue.&lt;/p&gt;&#xA;&lt;p&gt;To go through your papers&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Chari and Pachter PLOS Comp. Bio. 2023&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Sure, I've not read the paper but on your report, simply taking PC1 and PC2 on bivariate axes is not cool at all. Very little of the variance can be represented. Nobody disputes this. Even if 10 PCs trapped &amp;gt; 70% of the variance - thats a lot of bivariate plots to assess, e.g. PC1 vs PC3, PC1 vs PC4 etc ... At least you've trapped a good proportion of the variance.&lt;/p&gt;&#xA;&lt;p&gt;The problem becomes what happens when very little of the variance is captured. This is where Xiang &lt;em&gt;et al&lt;/em&gt; Frontier Genetics 2021 are likely to be coming from. It's better at that point to use tSNE, but its not ideal.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Whats the solution?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Approach 1&lt;/strong&gt;&#xA;In classical stats (not in RNAseq stuff) the output of PCA would be placed into tSNE and this will cause the variance spread across large numbers of PCs to be trapped by tSNE. This will give you nice clusters. I've not seen this approach at all in RNAseq - it is a common approach. The assumption is that loads and loads of PCs from the PCA trap sufficient amounts of the variance and tSNE will nicely represent that in 2 dimensions. It's possible that is what Xiang &lt;em&gt;et al&lt;/em&gt; Frontier Genetics 2021 have done, but they may have simply gone straight to tSNE (see note2).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Approach 2&lt;/strong&gt;&#xA;scvi-tool is impressive and I would recommend totalVI for dimensionality reduction, which they place straight into scanpy.&lt;/p&gt;&#xA;&lt;p&gt;The method and usage (not the robustness) for totalIV scvi-tools is described &lt;a href=&quot;https://docs.scvi-tools.org/en/stable/user_guide/models/totalvi.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. They are deploying a &amp;quot;hidden Markov&amp;quot;/latent variable style model. See the note below on pss. They've used &amp;quot;CITE-seq RNA&amp;quot;, which I don't exactly know what that is but hopefully that will remain relevant.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;It's important to note scvi-tools incorporates lots of other approaches including semi-supervised learning (is that scANVI???), transfer-learning (assume its CNN-style calculations), a standard ANN type model (I think thats scVI). I'm simply proposing what I would guess would work in your case.&lt;/li&gt;&#xA;&lt;li&gt;It is worth noting there are imperfections with the authors' methods - in general. Computer vision is a useful example, this used very similar approaches and it took years before serious levels of accuracy were achieved. Trapping absolute accuracy within a single deep learning model &amp;quot;off-the-cuff&amp;quot; I suggest is unlikely.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Semi-supervised learning its basically unsupervised learning but quantifying the weights of each feature - you can use it but I have never got good results from semi-supervised learning methods in general, this might be different though.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;ps.&lt;/strong&gt; If I've time (i.e. might not happen) I'll pop the link into the related question, where the low amounts of variance between PCs, and how to optimise that is partially answered.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;pss.&lt;/strong&gt; Overall scvi-kit for the models outlined need published demonstration they are robust. This part of their work appears partially missing, basically there appears a lot more functionality than published papers. I assume their other stuff is working its way through peer-review. &lt;em&gt;The totalIV approach: I performed a brief search but was unable to find anything published&lt;/em&gt; - whilst I'm recommending it - &lt;strong&gt;it appears untested&lt;/strong&gt;. What I believe is they've stolen a march on the rest of the field in general, so I'm making a best guess.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-08-28T12:30:48.523" LastActivityDate="2023-08-28T12:30:48.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21494" PostTypeId="1" AcceptedAnswerId="21495" CreationDate="2023-08-27T21:32:55.260" Score="4" ViewCount="78" Body="&lt;p&gt;I am trying to remove samples from a chromosome vcf file. I wrote a function that takes chromosome number and a list of samples to remove. When I try to run bcftools using subprocess module it only runs bcftools, as if I was running &lt;code&gt;subprocess.run([bcftools], shell=True, env=my_env)&lt;/code&gt; instead of:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;subprocess.run(['bcftools', 'view', in_vcf, '--samples', drop_samples, '-Oz', '&amp;gt;', out_vcf], shell=True, env=my_env)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The function:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;def remove_individuals(chr_samples):&#xA;    my_env = os.environ.copy()&#xA;    my_env['PATH'] = f'/home/aydar/bin/bcftools_1.11/bin/:{my_env[&amp;quot;PATH&amp;quot;]}'&#xA;    def remove_indiv(chr, indiv):&#xA;        in_vcf = f'data.filtered-updated-{chr}.vcf.gz'&#xA;        out_vcf = f'data.filtered-updated-{chr}-faulty-samp-rem.vcf.gz'&#xA;        drop_samples = &amp;quot;\'^&amp;quot;+&amp;quot;,&amp;quot;.join(indiv)+&amp;quot;\'&amp;quot;&#xA;        proc = subprocess.run(&#xA;            ['bcftools', 'view', in_vcf, &#xA;             '--samples', drop_samples, '-Oz', '&amp;gt;', out_vcf],&#xA;            shell=True,&#xA;            env=my_env&#xA;        )&#xA;        print(proc.args)&#xA;    chr_samples.apply(lambda x: remove_indiv(x.index[0], list(x.values[0])))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;function input:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;13,[2500000075_2500000075,2000000171_2000000171,1800000179_1800000179,330134048_330134048]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I run the same command in the terminal everything works.&lt;/p&gt;&#xA;&lt;p&gt;The command I test with in terminal:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view data.filtered-updated-13.vcf.gz --samples '^2500000075_2500000075,2000000171_2000000171,1800000179_1800000179,330134048_330134048' -Oz &amp;gt; out_test_vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18090" LastEditorUserId="650" LastEditDate="2023-08-28T17:08:04.620" LastActivityDate="2023-08-28T17:08:32.300" Title="Can not launch bcftools using python's subprocess module, as it only accepts first command of commands list" Tags="|python|bcftools|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21495" PostTypeId="2" ParentId="21494" CreationDate="2023-08-27T21:44:24.477" Score="3" Body="&lt;p&gt;Never mind, I figured it out. As it turned out I had to write to file like so and extra quotes &lt;code&gt;(&amp;quot;\'&amp;quot;)&lt;/code&gt; around samples were not needed. Also shell=True does not work well with lists, only strings:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;def remove_individuals(chr_samples):&#xA;    my_env = os.environ.copy()&#xA;    my_env['PATH'] = f'/home/aydar/bin/bcftools_1.11/bin/:{my_env[&amp;quot;PATH&amp;quot;]}'&#xA;    def remove_indiv(chr, indiv):&#xA;        in_vcf = f'data.filtered-updated-{chr}.vcf.gz'&#xA;        out_vcf = f'data.filtered-updated-{chr}-faulty-samp-rem.vcf.gz'&#xA;        drop_samples = &amp;quot;^&amp;quot;+&amp;quot;,&amp;quot;.join(indiv)&#xA;        out = subprocess.run(&#xA;            ['bcftools', 'view', in_vcf, &#xA;             '--samples', drop_samples, '-Oz'],&#xA;            env=my_env,&#xA;            capture_output=True&#xA;        )&#xA;        with open(out_vcf, 'wb') as f:&#xA;            f.write(out.stdout)&#xA;    chr_samples.apply(lambda x: remove_indiv(x.index[0], list(x.values[0])))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Remember to use &lt;code&gt;'wb'&lt;/code&gt; if you like me and use &lt;code&gt;Oz&lt;/code&gt; output option, because it returns bgzipped file.&lt;/p&gt;&#xA;" OwnerUserId="18090" LastEditorUserId="650" LastEditDate="2023-08-28T17:08:32.300" LastActivityDate="2023-08-28T17:08:32.300" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21496" PostTypeId="1" AcceptedAnswerId="21497" CreationDate="2023-08-28T10:34:59.880" Score="1" ViewCount="64" Body="&lt;p&gt;I have a low-diversity metagenome (~11 bins &amp;gt; 80% completion). Out of the bins, 3 are of interest to me. None of the lower-completion bins that can be identified are from the group of interest. So let's call the bins of interest 1, 2 and 3.&lt;/p&gt;&#xA;&lt;p&gt;The raw reads had been assembled separately with MEGAHIT and SPAdes (meta). Out of them, only the SPAdes assembly was binned / refined. There are 3 16S rRNA in the MEGAHIT assembly (x, y, z) and 3 16S rRNA in the SPAdes assembly (a, b, c).&lt;/p&gt;&#xA;&lt;p&gt;Bin 1 is associated with sequence 'a', which is identical to 'x'.&lt;/p&gt;&#xA;&lt;p&gt;Bin 2 is linked with SPAdes sequence 'b', which has no MEGAHIT equivalent. Bin 3 is not associated with any 16S rRNA (even with MarkerMAG).&lt;/p&gt;&#xA;&lt;p&gt;Sequences 'c' and 'z' are very similar. Sequence 'y' is an outlier.&lt;/p&gt;&#xA;&lt;p&gt;There are small hints from phylogenetics that bin 3 could be associated with sequences c/z. But then, could sequence 'y' just be a misassembly? Or is it that somehow my final bins are missing an entire genome of interest? If the 16S rRNA for 'y' is real, I would expect a genome divergent enough from the others that it wouldn't get binned together with them. This is what I am trying to understand. Would you happen to have any advice / suggestions on what I could to to test this?&lt;/p&gt;&#xA;&lt;p&gt;Initial approach: assessing scaffold quality with the BBMap stats.sh . It's a bit difficult to tell, since SPAdes has about 10x more contigs. Worse N50/N90 but more long contigs and a better 'general assembly score'.&lt;/p&gt;&#xA;&lt;p&gt;Additional thoughts: The new bins seem too divergent from known genomes to use a reference for misassembly estimation.&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your time!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastEditorUserId="3848" LastEditDate="2023-08-28T11:29:19.010" LastActivityDate="2023-08-28T11:40:12.573" Title="Variation in 16S rRNA between assemblers - how do I know which is real?" Tags="|phylogeny|software-recommendation|metagenome|16rrna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21497" PostTypeId="2" ParentId="21496" CreationDate="2023-08-28T11:27:23.687" Score="2" Body="&lt;p&gt;This is easy @Laura - I'm a bit surprised by the results Spades meta didn't have a good reputation, but the criteria assessed was not what you are looking for.&lt;/p&gt;&#xA;&lt;p&gt;Anyway take all outputs and place them in large multi-species alignment,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;e.g. for each output perform a Blast for maximum genetic diversity for each sequence and sample across the large Blast  output. Take note if there are multiple partial hits against diverse bacteria (online Blast has a diagrammatical summary).&lt;/li&gt;&#xA;&lt;li&gt;You have a good understanding of your taxonomic group and intersperse that with other stuff, i.e. fill in the diversity.&lt;/li&gt;&#xA;&lt;li&gt;Assemble it, e.g. muscle5 superfast option.&lt;/li&gt;&#xA;&lt;li&gt;Perform a split decomposition phylogeny or equivalent which identifies phylogenetic networks.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://uni-tuebingen.de/en/fakultaeten/mathematisch-naturwissenschaftliche-fakultaet/fachbereiche/informatik/lehrstuehle/algorithms-in-bioinformatics/software/splitstree/&quot; rel=&quot;nofollow noreferrer&quot;&gt;SplitsTree is the historic package&lt;/a&gt; for this and it turns out that split decomposition was not the best, I think &amp;quot;neighornet&amp;quot; was superior. What this is does is identify mosaic gene signal in your data. It provides a visual representation. Thus if your data is a artifactual mix of two or more taxa this will cause a network effect between the two parents and your newly mis-assembled sequence will sit between the two in a two &amp;quot;diamond same&amp;quot;. This is the antonym of a bifurcating phylogeny.&lt;/p&gt;&#xA;&lt;p&gt;The final step is the assembly that produces sequences that minimises phylogenetic networks is the true sequence. If you've not encountered split-decomposition/networks before a verbal explanation might seem confusing but when you run through a few examples and try it on your data you'll get the grasp.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-08-28T11:40:12.573" LastActivityDate="2023-08-28T11:40:12.573" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21498" PostTypeId="1" CreationDate="2023-08-28T14:42:02.077" Score="1" ViewCount="25" Body="&lt;p&gt;Similar to &lt;a href=&quot;https://www.biostars.org/p/379046/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this unanswered question on Biostars&lt;/a&gt;, I am using MaSuRCA for the first time and want to know how other MaSuRCA users are determining fragment mean and fragment stdev. My understanding is that FastQC measures read length, which differs from fragment length.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://diposit.ub.edu/dspace/bitstream/2445/195686/1/727000.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;This paper&lt;/a&gt; that used MaSuRCA for a bird genome assembly mentions the parameters they used but not how those parameters were determined: &amp;quot;MaSuRCA was run applying the following parameters: fragment mean (422), fragment stdev (312), and estimated genome size (1.2 Gbp).&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;I am using Illumina, Nanopore, and PacBio reads for fungal genome assembly. Any advice on which tool(s) to use to determine fragment mean and stdev for MaSuRCA would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="17690" LastEditorUserId="17690" LastEditDate="2023-08-28T15:01:19.613" LastActivityDate="2023-08-28T15:01:19.613" Title="Determining fragment mean and fragment stdev for MaSuRCA config file" Tags="|assembly|reads|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21499" PostTypeId="2" ParentId="21115" CreationDate="2023-08-28T23:10:50.193" Score="1" Body="&lt;p&gt;It seems that currently AnnotSV does not support HPO terms usage an Singularity.&lt;/p&gt;&#xA;" OwnerUserId="17244" LastActivityDate="2023-08-28T23:10:50.193" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21501" PostTypeId="2" ParentId="21492" CreationDate="2023-08-29T15:45:40.870" Score="3" Body="&lt;p&gt;You're right, it doesn't re-run things again for each job, so your original definition of &lt;code&gt;samples&lt;/code&gt; is still the one it's working with.&lt;/p&gt;&#xA;&lt;p&gt;It seems like it goes against the grain of Snakemake to overwrite a file in-place that acts as input to a rule, so I think the most Snakemake-y way you could do it would be to treat your samples.tsv as part of the input/output graph like everything else.  You could have a rule with samples.tsv as input and samples.somethingelse.tsv as output where that latter one has read lengths and whatever else gets automatically added.  (Alternatively, you could just have your updated samples.tsv read within the rule itself rather than relying on a shared copy loaded earlier, which is kind of halfway to the above idea.)&lt;/p&gt;&#xA;&lt;p&gt;Either way, maybe something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule deletion_profile:&#xA;    input:&#xA;        bam=rseqc_input_bam,&#xA;        samples=&amp;quot;samples.extended.tsv&amp;quot;&#xA;    output:&#xA;        os.path.join(rseqc_dir, '{sample}.deletion_profile.txt')&#xA;    conda:&#xA;        &amp;quot;../envs/rseqc.yaml&amp;quot;&#xA;    run:&#xA;        samples = whatever_loads_samples(input.samples)&#xA;        read_length = samples.loc[wildcards.sample, 'read_length']&#xA;        shell(&amp;quot;deletion_profile.py -i {input.bam} -l {read_length} -o {output}&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(On a side note, you can use &lt;code&gt;{output}&lt;/code&gt; rather than repeating yourself for that part.)&lt;/p&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2023-08-29T15:45:40.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21502" PostTypeId="1" AcceptedAnswerId="21503" CreationDate="2023-08-29T18:50:01.167" Score="2" ViewCount="67" Body="&lt;p&gt;I am working on a snake-make workflow that follows the guidelines of formatting and linting so that it can be published on the official Snake-make repository.&#xA;Whenever I push my code on GitHub, the GitHub actions are run and the pipeline is tested &lt;a href=&quot;https://github.com/snakemake/snakemake-github-action&quot; rel=&quot;nofollow noreferrer&quot;&gt;as indicated here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The problem is related to the execution of some time-consuming rules, such as the genome indexing generation for bowtie. These rules are run every time and take plenty of time.&#xA;I was wondering whether there was a way to bypass the execution of these rules and test the pipeline anyway. Do you have any suggestions about this?&lt;/p&gt;&#xA;&lt;p&gt;A problem is that I have a rule in my snakemake workflow that downloads the human genome and that is needed to index and then for the alignment. I need a workaround to avoid this behaviour, since has you said it is not feasible to test on the whole genome.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="8911" LastEditorUserId="73" LastEditDate="2023-08-31T01:22:15.537" LastActivityDate="2023-08-31T01:22:46.500" Title="Snakemake and github actions" Tags="|python|snakemake|github|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21503" PostTypeId="2" ParentId="21502" CreationDate="2023-08-30T07:29:21.697" Score="2" Body="&lt;p&gt;The answer for any testing purpose is generally to put together a small and minimal dataset so the test completes in seconds or minutes while still being representative. It makes (to me) limited sense to have a Actions/CI running using for example the entire human genome as input. The point of testing is to check that steps are executed properly, arguments are passed correctly, errors are detected. For this use either a small genome (such as from a bacterium or yeast), or make an artificial one consisting of very few basepairs.&lt;/p&gt;&#xA;&lt;p&gt;What I often do (for example for a peak calling workflow) is to take a full dataset, then subset to a region where I see a) many peaks and b) no peaks, then extract this genomic range (a few thousand bp) to build an artificial genome, plus extract from the bam/fastq the reads contributing to this region. That way I have basically a read-world dataset but very limited in size. For CI testing that does very well and CI finished in seconds, allowing to test even different configurations of this workflow on a standard / unpaid Actions runner.&lt;/p&gt;&#xA;&lt;p&gt;I personally prefer to not automate downloads but rather to have the user provide an actual file, or provide the file pre-downloaded in the workflow bundle. Connections might be bad or broken that day or the server/HPC is not connected to internet, and the entire workflows fails. Maybe write the relevant section in a way that it either accepts a URL for download OR a local file, and then write a testing profile that then reads the file to ensure it always is accessable to the Actions runner.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastEditorUserId="73" LastEditDate="2023-08-31T01:22:46.500" LastActivityDate="2023-08-31T01:22:46.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21504" PostTypeId="1" CreationDate="2023-08-30T15:09:13.263" Score="2" ViewCount="184" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://stackoverflow.com/q/77003956/12369569&quot;&gt;stack overflow&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have QIIME2 outputs (feature table, tree, taxonomy, and metadata) that I've imported into R using qiime2R (v. 0.99.6) ‘qza_to_phyloseq’ to create a phyloseq object. My QIIME feature table shows reasonable values for ASV abundance across all samples (n=16; feature table available upon request).&lt;/p&gt;&#xA;&lt;p&gt;Problem: The otu_table the ASV values for an entire study (NCBI SRA accession PRJNA217520, named AS_Briggs_NGHP_17A in my data) are shown as 0. This is untrue according to QIIME2view outputs and core-metrics-results (generated following the moving pictures tutorial) and a barchart created using the phyloseq object. Additionally, some stats (alpha diversity) are calculated accurately yet other stats (core members of the microbiome) return errors citing the otu_table.&lt;/p&gt;&#xA;&lt;p&gt;Ideal solution(s): 1) the otu_table from the phyloseq object is accurate, 2) I debug stats issues, and/or 3) all of the above.&lt;/p&gt;&#xA;&lt;p&gt;Please see the following code and files for reproducibility below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(&amp;quot;qiime2R&amp;quot;)&#xA;library(&amp;quot;phyloseq&amp;quot;)&#xA;&#xA;pyro_physeq&amp;lt;-qza_to_phyloseq(&#xA;  features=&amp;quot;/path/pyro_sg.table.qza&amp;quot;,&#xA;  tree=&amp;quot;/path/pyro_sg.r.tree.qza&amp;quot;,&#xA;  taxonomy = &amp;quot;/path/pyro_sg.taxonomy.qza&amp;quot;,&#xA;  metadata = &amp;quot;/path/pyro_sg.met.short.txt&amp;quot;)&#xA;&#xA;#section out otu_table for ease&#xA;otu_table &amp;lt;- (otu_table(pyro_physeq))&#xA;otu_table #all SRRs are the study &amp;quot;AS_Briggs_NGHP_17A&amp;quot; except for the first two, SRR1539064 and SRR1795474&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am not expecting the majority of these values to be 0's. I can create barcharts with the same phyloseq object, which use the otu_table as the input.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plot_bar(pyro_physeq, fill=&amp;quot;Class&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alternatively, I've tried to section out the study &amp;quot;AS_Briggs_NGHP_17A&amp;quot; to visualize alone, and the barchart is how I'd expect. Similarly, I tried a quick shannon diversity analysis which was also calculated (and returned believable values).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;phy_briggs &amp;lt;- subset_samples(pyro_physeq, name == &amp;quot;AS_Briggs_NGHP_17A&amp;quot;)&#xA;plot_bar(phy_briggs, fill=&amp;quot;Class&amp;quot;)&#xA;div.sh &amp;lt;- data.frame(diversity(otu_ord, index = &amp;quot;shannon&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;HOWEVER, if I extract that otu_table it's all 0's. If I attempt to calculate the core microbiome using 'core_members' (&lt;a href=&quot;https://stackoverflow.com/questions/77003956/otu-table-from-phyloseq-object-does-not-have-correct-abundance-all-asv-are-0?noredirect=1#comment135748718_77003956:%7E:text=R%20package%20microbiome&quot;&gt;R package microbiome&lt;/a&gt;), the 0's for all ASVs return errors.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pseq.core &amp;lt;- core_members(phy_briggs,prevalence = 50/100,detection = 1/100)&#xA;pseq.core&#xA;# returns 'character(0)', or to see the error:&#xA;pseq.core &amp;lt;- core(phy_briggs,prevalence = 50/100,detection = 1/100)&#xA;Error in validObject(.Object) : invalid class “otu_table” object: &#xA; OTU abundance data must have non-zero dimensions.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This also occurs for the full phyloseq object (pyro_physeq). Thoughts?&lt;/p&gt;&#xA;&lt;p&gt;Data&lt;/p&gt;&#xA;&lt;p&gt;metadata: &lt;a href=&quot;https://www.dropbox.com/scl/fi/wloiq8dze3znplmoqytzt/pyro_sg.met.short.txt?rlkey=bfsnrdqt1peh52xc2cqb3tgnd&amp;amp;dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/scl/fi/wloiq8dze3znplmoqytzt/pyro_sg.met.short.txt?rlkey=bfsnrdqt1peh52xc2cqb3tgnd&amp;amp;dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;tree: &lt;a href=&quot;https://www.dropbox.com/scl/fi/oliaj1qb9annxxptlmuix/pyro_sg.r.tree.qza?rlkey=ijmjeyt77tvk63cn1tovbcgfr&amp;amp;dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/scl/fi/oliaj1qb9annxxptlmuix/pyro_sg.r.tree.qza?rlkey=ijmjeyt77tvk63cn1tovbcgfr&amp;amp;dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;feature table: &lt;a href=&quot;https://www.dropbox.com/scl/fi/goav68y9osdhu6zoam2e0/pyro_sg.table.qza?rlkey=vzmbksfcpf1wvfyqy1frozxcs&amp;amp;dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/scl/fi/goav68y9osdhu6zoam2e0/pyro_sg.table.qza?rlkey=vzmbksfcpf1wvfyqy1frozxcs&amp;amp;dl=0&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;taxonomy: &lt;a href=&quot;https://www.dropbox.com/scl/fi/3nmg3aobl8ck04uvm827u/pyro_sg.taxonomy.qza?rlkey=ujs4tjtzih5i6rhzj9xl4icri&amp;amp;dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.dropbox.com/scl/fi/3nmg3aobl8ck04uvm827u/pyro_sg.taxonomy.qza?rlkey=ujs4tjtzih5i6rhzj9xl4icri&amp;amp;dl=0&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18104" LastEditorUserId="73" LastEditDate="2023-08-31T01:20:05.537" LastActivityDate="2023-08-31T01:20:05.537" Title="otu_table from phyloseq object does not have correct abundance: all asv are &quot;0&quot; for specific group of samples" Tags="|r|16rrna|phyloseq|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21505" PostTypeId="1" AcceptedAnswerId="21506" CreationDate="2023-08-30T15:47:41.160" Score="4" ViewCount="182" Body="&lt;p&gt;I am new to the scRNA-seq field and I have been doing some experiments of visualization of UMAP using different numbers of PCA components for initialization. The process involves projecting scRNA-seq data (count matrix) onto various numbers of PCA components, followed by non-linear dimension reduction using UMAP. It seems that the UMAP results are exactly the same across different initializations, regardless of the number of PCA components used. Are there any theoretical reasons for this? I am using Dynamo and their function &lt;code&gt;dyn.tl.reduceDimension(adata, n_pca_components=30)&lt;/code&gt; for the experiments.&lt;/p&gt;&#xA;&lt;p&gt;The results can be reproduced using the following code. Here I used the build-in zebrafish dataset with &lt;code&gt;n_obs × n_vars = 4181 × 16940&lt;/code&gt;. I am just trying to see how the PCA initialization affects the Umap results.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import warnings&#xA;warnings.filterwarnings('ignore')&#xA;&#xA;import dynamo as dyn &#xA;&#xA;dyn.configuration.set_figure_params('dynamo', background='white')&#xA;&#xA;adata = dyn.sample_data.zebrafish()&#xA;&#xA;preprocessor = dyn.pp.Preprocessor(cell_cycle_score_enable=True)&#xA;preprocessor.preprocess_adata(adata, recipe='monocle')&#xA;&#xA;dyn.pl.variance_explained(adata)&#xA;&#xA;dyn.tl.reduceDimension(adata, n_pca_components=1, enforce=True)&#xA;dyn.pl.umap(adata, color='Cell_type')&#xA;&#xA;dyn.tl.reduceDimension(adata, n_pca_components=30, enforce=True)&#xA;dyn.pl.umap(adata, color='Cell_type')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here is the picture that shows the variance carried by PCA:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/DMDb2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DMDb2.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;And the final umap results using n_pca_components= 1 and n_pca_components= 30:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/wIbSC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/wIbSC.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18105" LastEditorUserId="650" LastEditDate="2023-08-31T15:59:42.997" LastActivityDate="2023-08-31T15:59:42.997" Title="Question about umap using different numbers of pca components as initialization" Tags="|rna-seq|pca|umap|dimensionality-reduction|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21506" PostTypeId="2" ParentId="21505" CreationDate="2023-08-30T16:15:05.403" Score="0" Body="&lt;p&gt;It seems you are performing PCA -&amp;gt; UMAP. The results you are reporting actually look cool. It's a good answer and there's nothing to worry about. It also shows good analytics, 'cause you're questioning the robustness of your data.&lt;/p&gt;&#xA;&lt;p&gt;The limiting factor is the variance carried by PCA - which needs to be calculated (it may be low, e.g. &amp;lt;50%). Well I mean you've calculated it via the PCs - but its not declared, its better to know how much it is, i.e. how much variance each PC holds. The UMAP is doing nothing more than representing the established variance identified by PCA.&lt;/p&gt;&#xA;&lt;p&gt;If you change the number of components which UMAP is assessing &lt;em&gt;then&lt;/em&gt; this will result in changes in the clusters observed, usually.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;However&lt;/strong&gt; if you are changing the no. PCs -&amp;gt; UMAP and there's little effect then the PCs are not introducing new information to cut-up the data set. Thats a fairly robust answer: it's attained &amp;quot;stationarity&amp;quot;. Very desirable output, just make sure there isn't a bug in your code, but it demonstrates there's no hidden or partial information that might give a different result. So the clusters you are seeing are a good representation of the data (thats good).&lt;/p&gt;&#xA;&lt;p&gt;Its not a complete analysis - as hinted - but I'd be happy with that result.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Overall, there's very heavy dispersion of the variance between PCs. I've never had to teach this stuff, beyond a few line explanations. An educator experienced with this theory would give a more succinct explanation. However, the discussion below arrives at a common understanding between the OP and I. If there's demand I'll summarise the discussion as a singular paragraph: the final comments should be the introduction, i.e. what PCA is doing at basic level.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Comments&lt;/strong&gt; (discussion)&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;@Zach&lt;/strong&gt; even when using n_pca_components=1 and n_pca_components=30, the results remain unchanged. The first PCA component should only capture a part of the overall information. Is this correct?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Response&lt;/strong&gt; Number 0 (on your graph) is the first principle component (16%). PC = 1 is the second PC (5%). Thus PCA = 1 means 21% of the variance. This far exceeds the PC 3 at 2%. Thats why it works, so all information is PC1 and PC2, nothing beyond that discriminates against the data.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;@Zach&lt;/strong&gt; So, n_pca_components = 3 could capture almost all the information and adding more components doesn't really make any difference to the results. But if we only consider n_pca_components = 1, why there are still no differences compared to n_pca_components = 30?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Response&lt;/strong&gt; its easy .. because if you add all the PCs from PC 4 + ... + PC 30 it will be &amp;lt;5% of the total variance. PC1 + PC2 + PC3 = 24% of the variance, so there is nothing to see beyond PC3, when the maximum is PC 30.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;@Zach&lt;/strong&gt; Why there are no differences between n_pca_components = 1 (using only PC1) and n_pca_components = 30 (using PC1+PC2+...+PC30)?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Response&lt;/strong&gt;&#xA;PC1 + PC2 = 24% and PC1 ... + PC30 &amp;lt;30%. There is no information in the variance for between group clusters for PC3 to PC30, there is likely to be within group differences but you are not measuring those.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;@Zach&lt;/strong&gt; there is little difference between PC1 + PC2 = 21% and PC1+...+PC30. I mean if we only consider PC1 = 16% and PC1+...+PC30 &amp;gt; 25%, do they have any differences between PC1 and PC1+...+PC30?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Response&lt;/strong&gt; PCA works on individual elements it, it doesn't work on groups. The analysis of groups is post-hoc (usually via colour coding). PC=1 in your UMAP calculation is PC1 + PC2 (PC1 is automatic, its position 0 in your graph). . You are only looking at between group variance and in your data that is resolved essentially for PC1 + PC2 = 21%. The within group positioning will change so, yes there will be difference against PC1+PC2 vs PC1 + .... + PC 30 but we can't see them. Again PCA is individual elements, it's not using group information. So if you did sub-group analysis you might see changes PC1+PC2 vs PC1 + ... PC 30. So to put it simply your group delineation is too big to spot in any differences (but that only matters if it is biologically relevant).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;@Zach&lt;/strong&gt; Actually what I mean is PC0 = 16% and PC0+...+PC30 &amp;gt; 25%. But your explanation makes sense to me and I appreciate your response.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-08-31T02:12:17.820" LastActivityDate="2023-08-31T02:12:17.820" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21507" PostTypeId="2" ParentId="21505" CreationDate="2023-08-31T01:17:28.290" Score="0" Body="&lt;p&gt;It is a bit odd that different numbers of principal components are giving identical results, especially given that even the same initialisation settings with different UMAP epoch numbers will give different results.&lt;/p&gt;&#xA;&lt;p&gt;I'm not familiar with the use of python for single cell data analysis, but it looks like the &lt;code&gt;adata&lt;/code&gt; information might be being modified when running &lt;code&gt;reduceDimension&lt;/code&gt;. If this is the case, then it's possible that the first PCA reduction is reducing the dataset as far as it will go, and the second PCA reduction doesn't do anything. This could be checked by changing the order of operations (from a fresh run) and seeing if the output is changed:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dyn.tl.reduceDimension(adata, n_pca_components=30, enforce=True)&#xA;dyn.pl.umap(adata, color='Cell_type')&#xA;&#xA;dyn.tl.reduceDimension(adata, n_pca_components=1, enforce=True)&#xA;dyn.pl.umap(adata, color='Cell_type')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It's also possible that the &lt;code&gt;umap&lt;/code&gt; command is doing its own reduction and ignoring the result of &lt;code&gt;dyn.tl.reduceDimension&lt;/code&gt;. Maybe the UMAP plotting function doesn't actually do a UMAP reduction as well, and the UMAP result (or initial condition) is calculated earlier (e.g. in &lt;code&gt;preprocessor.preprocess_adata&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;It's also possible, as @M__ suggests, that you've reached a stationary result after however many UMAP iterations are set as default. But... I find that unlikely given how variable UMAP has been for me. I'd want to do a lot more exploration before settling on that as an explanation.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-08-31T01:17:28.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21508" PostTypeId="1" CreationDate="2023-09-02T00:03:09.537" Score="0" ViewCount="91" Body="&lt;p&gt;I am trying to understand the Zuker Algorithm.&lt;/p&gt;&#xA;&lt;p&gt;What is happening during initialisation? I know in Nussinov the diagonal is filled with 0s. Are the diagonals filled with infinite energy during Zuker´s initialisation?&lt;/p&gt;&#xA;&lt;p&gt;Also, what is happening during recursion?&lt;/p&gt;&#xA;&lt;p&gt;I know unlike Nussinov, Zuker pays attention to stacking energies and loop structures, not individual basepairs. But how? How does the algorithm calculate the energies for stacking and loop structures at position 1 if there is no info about its neighbors yet?&lt;/p&gt;&#xA;&lt;p&gt;I understand there are 2 matrices - V and W. Is it correct that V is part of W? And is it correct that V keeps information about stack interactions and how much energy a stack of base pairs contributes - like keeping score on a multiplier?&lt;/p&gt;&#xA;&lt;p&gt;Is there a code or example (I would prefer R), which demonstrates how the algorithm goes over a sequence? Like a demonstration. I thought of one myself, but I am not sure if its correct:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sequence &amp;lt;- c(1, 2, 3, 4, 5,6,7,8,9,10,11,12,13,14,15)&#xA;&#xA;n &amp;lt;- length(sequence)&#xA;&#xA;for (l in 2:n) {&#xA;   for (i in 1:(n - l + 1)) {&#xA;      j &amp;lt;- i + l - 1&#xA;      cat(&amp;quot;i =&amp;quot;, i, &amp;quot;j =&amp;quot;, j, &amp;quot;\n&amp;quot;)&#xA;   }&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18117" LastEditorUserId="298" LastEditDate="2023-09-11T11:55:43.800" LastActivityDate="2023-09-11T11:55:43.800" Title="What is happening in the Zuker-Algorithm" Tags="|r|rna|algorithms|rna-structure|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21509" PostTypeId="1" AcceptedAnswerId="21517" CreationDate="2023-09-02T15:25:06.300" Score="1" ViewCount="32" Body="&lt;p&gt;I am a newbie in Bioinformatics, I want to extract the GO terms from yeast-cyc,&#xA;which is items in the Go terms tab of this page &lt;a href=&quot;https://yeast.biocyc.org/gene?orgid=YEAST&amp;amp;id=G3O-29622#tab=GO&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://yeast.biocyc.org/gene?orgid=YEAST&amp;amp;id=G3O-29622#tab=GO&lt;/a&gt; , not one page but one database like &lt;em&gt;Saccharomyces cerevisiae S288c&lt;/em&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Is there an API or python lib can help us finish this task? I had found biocyc in pypi, but I did not found Go terms was involved in this API.&lt;/p&gt;&#xA;&lt;p&gt;If the API is not convinient, can I download some dataset in which my required items is include?&lt;/p&gt;&#xA;" OwnerUserId="18119" LastEditorUserId="18119" LastEditDate="2023-09-02T16:16:15.257" LastActivityDate="2023-09-05T05:00:12.490" Title="How to access yeast-cyc in a programmatic way to extract the GO terms?" Tags="|python|genomics|public-dataset|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21510" PostTypeId="1" CreationDate="2023-09-03T00:05:25.740" Score="1" ViewCount="37" Body="&lt;p&gt;We are going to look at the likely N- and O- glycosylation sites within MUC16 (&lt;a href=&quot;https://www.uniprot.org/uniprotkb/Q8WXI7/entry&quot; rel=&quot;nofollow noreferrer&quot;&gt;Q8WXI7 · MUC16_HUMAN&lt;/a&gt;)§ by in house long-read DNA sequencing data (PacBio).&lt;/p&gt;&#xA;&lt;p&gt;Counting the number of tandem repeats is a task which could be done with bioinformatic tools like &lt;code&gt;ad-codeVNTR&lt;/code&gt;as shown in the below figure. Another task is identifying mutations within these repeats. My question is: after predicting &lt;code&gt;glycosylation sites&lt;/code&gt; using tools like &lt;code&gt;PA2DE&lt;/code&gt; within these tandem repeats in &lt;em&gt;MUC16&lt;/em&gt; gene, how could I identify any point mutations in these &lt;code&gt;glycosylation sites&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;I googled a lot to get some idea; Shall we do germline analysis in &lt;em&gt;MUC16&lt;/em&gt; glycosylation predicted sites? Is this what we should use to look at glycosylation sites that are ‘rejected’ to understand the sites that can’t be glycosylated (mutated)?&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have such an experience how to identify mutation in glycosylation sites in MUC16 which known to have variable number tandem repeat (VNTR)?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/RBIzU.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/RBIzU.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2023-09-03T17:16:32.650" LastActivityDate="2023-09-03T17:16:32.650" Title="Finding mutations in glycosylation sites" Tags="|protein-structure|sequence-analysis|pacbio|mass-spectrometry|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21511" PostTypeId="1" CreationDate="2023-09-04T02:39:07.260" Score="1" ViewCount="65" Body="&lt;p&gt;I typically see several ways of visualising DNA sequences and I would like to know when (best) to use them?&lt;/p&gt;&#xA;&lt;p&gt;For example, I sometimes see the letter codes GATC, sometimes I see colours, sometimes I see a representation of the double-helix. I think I saw a circular representation.&lt;/p&gt;&#xA;&lt;p&gt;Additionally, what other options exist for visualising DNA and when should they be used? E.g. for papers, analysis in software, etc.&lt;/p&gt;&#xA;&lt;p&gt;What do people tend to use when presenting DNA sequences?&lt;/p&gt;&#xA;" OwnerUserId="17924" LastActivityDate="2023-09-05T10:02:09.953" Title="What options are there for visualising DNA sequences?" Tags="|genome|visualization|dna|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21512" PostTypeId="2" ParentId="3477" CreationDate="2023-09-04T08:19:47.297" Score="0" Body="&lt;p&gt;From the old VCF manual,&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://vcftools.sourceforge.net/man_latest.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://vcftools.sourceforge.net/man_latest.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://manpages.debian.org/testing/vcftools/vcf-subset.1.en.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://manpages.debian.org/testing/vcftools/vcf-subset.1.en.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;cat in.vcf | vcf-subset -r -t indels -e -c SAMPLE1 &amp;gt; out.vcf&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;comma-separated list of variant types ( indels ) could be obtained.&lt;/p&gt;&#xA;" OwnerUserId="17226" LastEditorUserId="17226" LastEditDate="2023-09-06T03:53:14.520" LastActivityDate="2023-09-06T03:53:14.520" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21513" PostTypeId="1" CreationDate="2023-09-04T14:54:28.143" Score="4" ViewCount="61" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bf0BC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bf0BC.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;0&lt;/p&gt;&#xA;&lt;p&gt;I have to establish the penetration profile of particles in tumour spheroids.&lt;/p&gt;&#xA;&lt;p&gt;For this I have spheroids composed of cells which were exposed to particles, both of which are fluorescently labeled. The spheroids were then imaged using a confocal microscope by taking z-stacks of all spheroids. For analysis, I determined the intensity profile of each focal plane using an ImageJ plugin, namely &amp;quot;distribution_shapes&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;What should be done next?&lt;/p&gt;&#xA;&lt;p&gt;I currently did not find any step-by-step guide for this analysis for manual assessment and any type of help would be greatly appreciated.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uzcq2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uzcq2.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18125" LastEditorUserId="18125" LastEditDate="2023-11-01T15:30:44.690" LastActivityDate="2023-11-02T20:24:26.793" Title="How to analyse qualitatively the penetration ability of particles in spheroids using fluorescent z-stacks?" Tags="|3d-structure|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21514" PostTypeId="1" CreationDate="2023-09-04T23:51:03.523" Score="1" ViewCount="292" Body="&lt;p&gt;I am trying to use &lt;strong&gt;dorado&lt;/strong&gt; basecaller to process ONT data. I need to make the move table and I am doing it with &lt;strong&gt;--emit-move&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dorado basecaller dna_r9.4.1_e8_sup@v3.3/ /pod5_files/ --recursive --emit-moves --modified-bases 5mCG_5hmCG -v &amp;gt; /bams/mod_stride1.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The main purpose of the move table for me is to calculate the number of ONT device measurements before basecaller emits one base (calculate &lt;strong&gt;translocation time per base&lt;/strong&gt;). But the stride value in config.toml file is fixed to 5 (stride=5). I need to calculate the instantaneous translocation time per base (in seconds), because I am looking to find a measure similar to IPD values in PacBio, when we can get the time for sequencing each base.&lt;/p&gt;&#xA;&lt;p&gt;I was wondering to &lt;strong&gt;change the stride value in config.toml file&lt;/strong&gt; in the model folder, from stride=5 to stride=1, to directly infer the number of measurements without using Tombo for correcting and re-squiggling.&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, even though this command works perfectly fine when the stride=5, when I set stride=1 it always gives me the following error: &lt;strong&gt;&amp;quot;CUDA out of memory&amp;quot;&lt;/strong&gt;, even when I triple the size of RAM for the job.&lt;/p&gt;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Does setting stride=1 make sense for this purpose? Or do I have to do something else? (I know we have some K-mer models too)&lt;/li&gt;&#xA;&lt;li&gt;Is this even possible to change the stride value on config.toml file?&lt;/li&gt;&#xA;&lt;li&gt;How should I figure out how much RAM I need for the job?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="15341" LastEditorUserId="73" LastEditDate="2023-09-05T20:58:39.623" LastActivityDate="2023-09-05T21:04:50.430" Title="How should I use dorado basecaller to calculate translocation time? Can I change models' config files?" Tags="|nanopore|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21515" PostTypeId="2" ParentId="21514" CreationDate="2023-09-05T02:18:14.733" Score="1" Body="&lt;p&gt;In pod5 files the number of samples prior to the adapter/sequence should be encoded in the metadata, and can be looked at using ONT's &lt;code&gt;pod5 inspect&lt;/code&gt; tool:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ pod5 inspect reads pod5_file.pod5&#xA;&#xA;read_id,channel,well,pore_type,read_number,start_sample,end_reason,median_before,calibration_offset,calibration_scale,sample_count,byte_count,signal_compression_ratio&#xA;00445e58-3c58-4050-bacf-3411bb716cc3,908,1,not_set,100776,374223800,signal_positive,205.3,-240.0,0.1,65582,58623,0.447&#xA;00520473-4d3d-486b-86b5-f031c59f6591,220,1,not_set,7936,16135986,signal_positive,192.0,-233.0,0.1,167769,146495,0.437&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The average translocation speed can be worked out by comparing the &lt;code&gt;sample_count&lt;/code&gt; to the number of called bases.&lt;/p&gt;&#xA;&lt;p&gt;More details &lt;a href=&quot;https://pod5-file-format.readthedocs.io/en/latest/docs/tools.html#pod5-inspect&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, due to the way the basecaller models work, I don't think there's any easy way to create a 1-sample lookup table for precise instantaneous translocation calculations.&lt;/p&gt;&#xA;&lt;p&gt;If it is just an out of memory error, then it should work (but very slowly) if you set the basecaller batch size to 1 (i.e. &lt;code&gt;-b 1&lt;/code&gt;). If even that doesn't work, something else is going on and it's unlikely to be a memory problem.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-09-05T21:04:50.430" LastActivityDate="2023-09-05T21:04:50.430" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21516" PostTypeId="2" ParentId="21511" CreationDate="2023-09-05T02:58:35.827" Score="2" Body="&lt;p&gt;DNA sequences have a massive range in length, so the approach of visualisation is dependent on the purpose of that visualisation.&lt;/p&gt;&#xA;&lt;p&gt;I have been experimenting with visualising the repetitive structure of DNA in a way that represents repeats at both short range (tens of bases) and long range (millions of bases, or longer). You can see my attempts at that &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/375/finding-the-location-and-unit-length-of-repetitive-sequences-within-a-long-read&quot;&gt;here&lt;/a&gt;. An associated presentation about DNA visualisation (with emphasis on repetitive sequences) can be found &lt;a href=&quot;https://www.youtube.com/watch?v=u-v57M7Ldtk&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-09-05T10:02:09.953" LastActivityDate="2023-09-05T10:02:09.953" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21517" PostTypeId="2" ParentId="21509" CreationDate="2023-09-05T05:00:12.490" Score="1" Body="&lt;p&gt;You can get the GO terms for yeast from UniProt, e.g. &lt;a href=&quot;https://uniprot.org/uniprotkb?query=(taxonomy_id:559292)&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://uniprot.org/uniprotkb?query=(taxonomy_id:559292)&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Using the UniProt website you can generate an API endpoint URL for your query, e.g. here are the GO terms for all yeast genes: &lt;a href=&quot;https://rest.uniprot.org/uniprotkb/stream?fields=accession%2Cid%2Cprotein_name%2Cgene_names%2Cgo&amp;amp;format=tsv&amp;amp;query=%28%28taxonomy_id%3A559292%29%29&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rest.uniprot.org/uniprotkb/stream?fields=accession%2Cid%2Cprotein_name%2Cgene_names%2Cgo&amp;amp;format=tsv&amp;amp;query=%28%28taxonomy_id%3A559292%29%29&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can download this data using &lt;code&gt;wget&lt;/code&gt; or &lt;code&gt;cURL&lt;/code&gt;, e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;wget -O yeast_go_terms.tsv &amp;quot;https://rest.uniprot.org/uniprotkb/stream?fields=accession%2Cid%2Cprotein_name%2Cgene_names%2Cgo&amp;amp;format=tsv&amp;amp;query=%28%28taxonomy_id%3A559292%29%29&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9098" LastActivityDate="2023-09-05T05:00:12.490" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21518" PostTypeId="2" ParentId="4544" CreationDate="2023-09-05T08:07:19.447" Score="0" Body="&lt;p&gt;This post on the Bioconductor mailing list explains the difference: &lt;a href=&quot;https://support.bioconductor.org/p/100463/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://support.bioconductor.org/p/100463/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="1105" LastActivityDate="2023-09-05T08:07:19.447" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21519" PostTypeId="1" AcceptedAnswerId="21521" CreationDate="2023-09-05T09:48:04.283" Score="0" ViewCount="88" Body="&lt;p&gt;I want to plot a heat map in header I have the average of gene expression validated in lab for each sample&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;dput(heat[1:5,1:5])&#xA;&#xA;structure(c(3.05574677859363, -0.451189292623471, -0.398066832453666, &#xA;-0.456927931251448, -0.393418432279742, -0.270234667496526, -0.451189292623471, &#xA;-0.398066832453666, -0.456927931251448, -0.393418432279742, -0.270234667496526, &#xA;-0.451189292623471, -0.398066832453666, -0.456927931251448, -0.393418432279742, &#xA;-0.270234667496526, -0.451189292623471, -0.398066832453666, 2.0338091136055, &#xA;-0.393418432279742, -0.270234667496526, 1.76078923919148, 1.96896026174603, &#xA;1.97960246116808, -0.393418432279742), .Dim = c(5L, 5L), .Dimnames = list(&#xA;    c(&amp;quot;hsa-miR-548b-5p&amp;quot;, &amp;quot;hsa-miR-181c-5p&amp;quot;, &amp;quot;hsa-miR-181d-5p&amp;quot;, &#xA;    &amp;quot;hsa-miR-125a-3p&amp;quot;, &amp;quot;hsa-miR-4286&amp;quot;), c(&amp;quot;30566-001&amp;quot;, &amp;quot;30566-002&amp;quot;, &#xA;    &amp;quot;30566-003&amp;quot;, &amp;quot;30566-004&amp;quot;, &amp;quot;30566-005&amp;quot;)))&#xA;&#xA;&#xA;dput(m[1:5,c(2,7)])&#xA;&#xA;structure(list(HealthStatus = c(&amp;quot;Healthy&amp;quot;, &amp;quot;Healthy&amp;quot;, &amp;quot;Healthy&amp;quot;, &#xA;&amp;quot;Healthy&amp;quot;, &amp;quot;Healthy&amp;quot;), Average = c(108.915677, 103.148873, 40.3359607, &#xA;88.4329528, 87.7756577)), row.names = c(&amp;quot;30566-001&amp;quot;, &amp;quot;30566-002&amp;quot;, &#xA;&amp;quot;30566-003&amp;quot;, &amp;quot;30566-004&amp;quot;, &amp;quot;30566-005&amp;quot;), class = &amp;quot;data.frame&amp;quot;)&#xA;&#xA;&#xA;pheatmap(heat, cluster_rows=T, show_rownames=T,&#xA;         cluster_cols=FALSE, annotation_col=m[,c(2,7)],&#xA;         annotation_colors=annot_colors)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How I can order &lt;code&gt;average&lt;/code&gt; in a descending way? &lt;a href=&quot;https://i.stack.imgur.com/nsyYq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nsyYq.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="650" LastEditDate="2023-09-05T15:21:49.993" LastActivityDate="2023-09-08T11:22:59.960" Title="Ordering average of gene expression in heat map" Tags="|heatmap|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21521" PostTypeId="2" ParentId="21519" CreationDate="2023-09-05T23:33:33.710" Score="4" Body="&lt;p&gt;If you order your matrix/data.frame you can sort your columns by decreasing &amp;quot;Average&amp;quot;, e.g.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(pheatmap)&#xA;&#xA;heat &amp;lt;- structure(c(3.05574677859363, -0.451189292623471, -0.398066832453666, &#xA;                    -0.456927931251448, -0.393418432279742, -0.270234667496526, -0.451189292623471, &#xA;                    -0.398066832453666, -0.456927931251448, -0.393418432279742, -0.270234667496526, &#xA;                    -0.451189292623471, -0.398066832453666, -0.456927931251448, -0.393418432279742, &#xA;                    -0.270234667496526, -0.451189292623471, -0.398066832453666, 2.0338091136055, &#xA;                    -0.393418432279742, -0.270234667496526, 1.76078923919148, 1.96896026174603, &#xA;                    1.97960246116808, -0.393418432279742), .Dim = c(5L, 5L), .Dimnames = list(&#xA;                      c(&amp;quot;hsa-miR-548b-5p&amp;quot;, &amp;quot;hsa-miR-181c-5p&amp;quot;, &amp;quot;hsa-miR-181d-5p&amp;quot;, &#xA;                        &amp;quot;hsa-miR-125a-3p&amp;quot;, &amp;quot;hsa-miR-4286&amp;quot;), c(&amp;quot;30566-001&amp;quot;, &amp;quot;30566-002&amp;quot;, &#xA;                                                              &amp;quot;30566-003&amp;quot;, &amp;quot;30566-004&amp;quot;, &amp;quot;30566-005&amp;quot;)))&#xA;&#xA;m &amp;lt;- structure(list(HealthStatus = c(&amp;quot;Healthy&amp;quot;, &amp;quot;Healthy&amp;quot;, &amp;quot;Healthy&amp;quot;, &#xA;                                     &amp;quot;Healthy&amp;quot;, &amp;quot;Healthy&amp;quot;), Average = c(108.915677, 103.148873, 40.3359607, &#xA;                                                                        88.4329528, 87.7756577)), row.names = c(&amp;quot;30566-001&amp;quot;, &amp;quot;30566-002&amp;quot;, &#xA;                                                                                                                &amp;quot;30566-003&amp;quot;, &amp;quot;30566-004&amp;quot;, &amp;quot;30566-005&amp;quot;), class = &amp;quot;data.frame&amp;quot;)&#xA;&#xA;pheatmap(heat[,order(m[,&amp;quot;Average&amp;quot;], decreasing = TRUE)], &#xA;         cluster_rows=TRUE, show_rownames=T,&#xA;         cluster_cols=FALSE, annotation_col=m)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.imgur.com/LLbnkUG.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;sup&gt;Created on 2023-09-06 with &lt;a href=&quot;https://reprex.tidyverse.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;reprex v2.0.2&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="9098" LastEditorUserId="9098" LastEditDate="2023-09-08T11:22:59.960" LastActivityDate="2023-09-08T11:22:59.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21523" PostTypeId="2" ParentId="13255" CreationDate="2023-09-07T02:48:32.470" Score="1" Body="&lt;p&gt;There are many ways to do it. Here is one way to do in Python.&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;import os&#xA;&#xA;with open(&amp;quot;input.fasta&amp;quot;) as f:&#xA;    for line in f:&#xA;        if line.startswith('&amp;gt;'):&#xA;            record_name = line.lstrip(&amp;quot;&amp;gt;&amp;quot;)&#xA;            outfilename = f&amp;quot;{(record_name.rstrip())}.fasta&amp;quot;&#xA;            if not os.path.exists(outfilename):&#xA;                out = open(outfilename, 'w+')       &#xA;            else:&#xA;                print('File already exists!')&#xA;        out.write(line)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2023-09-07T02:48:32.470" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21524" PostTypeId="1" CreationDate="2023-09-07T02:54:31.387" Score="3" ViewCount="203" Body="&lt;p&gt;I have the assembly file of a certain lizard species, I wanted to recreate Fig. 2 of the following paper, which compares the GC content distribution over 5kb windows, how would I go about doing this?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bmcecolevol.biomedcentral.com/articles/10.1186/s12862-022-02086-7#Sec10&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bmcecolevol.biomedcentral.com/articles/10.1186/s12862-022-02086-7#Sec10&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/q4uns.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/q4uns.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18137" LastEditorUserId="298" LastEditDate="2023-09-08T12:18:41.973" LastActivityDate="2023-09-08T12:18:41.973" Title="How to analyze GC content of fasta file?" Tags="|fasta|genome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21525" PostTypeId="1" CreationDate="2023-09-07T03:28:42.310" Score="3" ViewCount="37" Body="&lt;p&gt;I need to match the external gene names of a table I have called (which has 1,000,000 entries) with their respective ensemble gene IDs.&lt;/p&gt;&#xA;&lt;p&gt;Here's a small subset of &lt;code&gt;df&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;V1&lt;/th&gt;&#xA;&lt;th&gt;external_gene_name&lt;/th&gt;&#xA;&lt;th&gt;V3&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;M0085&lt;/td&gt;&#xA;&lt;td&gt;A1BG&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;M0085&lt;/td&gt;&#xA;&lt;td&gt;SNX8&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;and here's a small subset of &lt;code&gt;ensembl_ids&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;ensembl_gene_id&lt;/th&gt;&#xA;&lt;th&gt;external_gene_name&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;ENSG00000288263&lt;/td&gt;&#xA;&lt;td&gt;GRK1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;ENSG00000291428&lt;/td&gt;&#xA;&lt;td&gt;BRSK2&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;I also included a line to see the number of entries in &lt;code&gt;external_gene_name&lt;/code&gt; which do not have corresponding ensemble names. Here is my code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;merged_df &amp;lt;- merge(df, ensembl_ids, by = &amp;quot;external_gene_name&amp;quot;)&#xA;elements_not_in &amp;lt;- df$external_gene_name[!(df$external_gene_name %in% ensembl_ids$external_gene_name)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But what's confusing me is that when I add the number of rows in &lt;code&gt;merged_df&lt;/code&gt; with that of &lt;code&gt;elements_not_in&lt;/code&gt;, the total exceeds 1,000,000. Shouldn't the total be 1,000,000? I'm wondering if there's a flaw in my code.&lt;/p&gt;&#xA;" OwnerUserId="17613" LastEditorUserId="298" LastEditDate="2023-09-08T11:27:49.147" LastActivityDate="2023-09-08T11:27:49.147" Title="Why does the subset of rows that match a condition added by the rows that don't match a condition not add up to the total number of rows?" Tags="|r|ensembl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21526" PostTypeId="2" ParentId="21525" CreationDate="2023-09-07T14:23:39.597" Score="4" Body="&lt;p&gt;From a brief glance, the gap is not in your code but in your expectations. You're operating based on an assumption of 1:1 mapping between ENSG IDs and HGNC symbols (your &lt;code&gt;external_gene_name&lt;/code&gt; attribute). In reality, multiple ENSG IDs will map to the same HGNC symbol so the merge will produce multiple rows per &lt;code&gt;external_gene_name&lt;/code&gt; in the merge product. This is why you're seeing a total greater than the sum of number of unique entries when you compare the merge product + the anti-join result (i.e., the &lt;code&gt;elements_not_in&lt;/code&gt; dataset) to the initial individual datasets.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-09-07T14:23:39.597" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21527" PostTypeId="2" ParentId="21524" CreationDate="2023-09-07T21:13:05.297" Score="4" Body="&lt;p&gt;I would suggest using &lt;code&gt;bedtools makewindows&lt;/code&gt; and &lt;code&gt;bedtools nuc&lt;/code&gt; to compute the GC content for genomic windows as shown in Fig 2 of the linked paper.&lt;/p&gt;&#xA;&lt;p&gt;See related question &lt;a href=&quot;https://www.biostars.org/p/9522847/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;This will slice the genome into 5Kbp (or whatever size specified to &lt;code&gt;makewindows&lt;/code&gt;) pieces and compute the GC content of each piece, which can be plotted as a distribution as in the figure.&lt;/p&gt;&#xA;&lt;p&gt;Something like this would work for the 5000 bp example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# assemble a bedtools compatible genome file:&#xA;# see https://www.biostars.org/p/70795/ for hints how to do this.&#xA;bedtools makewindows -w 5000 -g my_genome.genome &amp;gt; windows.bed # you just made this .genome file manually&#xA;bedtools nuc -bed windows.bed -fi my_genome.fasta &amp;gt; windows_gc.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The bed output file will contain the GC content of each window in the &lt;code&gt;7_pct_gc&lt;/code&gt; column (according to my understanding of the output file).&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-09-07T21:13:05.297" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21528" PostTypeId="2" ParentId="21513" CreationDate="2023-09-07T23:51:46.053" Score="1" Body="&lt;p&gt;I am not sure if this is exactly what you're looking for, but I was able to find &lt;a href=&quot;https://github.com/yakavetsiv/distribution-macro&quot; rel=&quot;nofollow noreferrer&quot;&gt;this ImageJ script&lt;/a&gt; that estimates a penetration profile in tumor spheroids.&lt;/p&gt;&#xA;&lt;p&gt;I am not an ImageJ person but I suspect that with some minimal changes this could be used to answer your question.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-09-07T23:51:46.053" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21531" PostTypeId="2" ParentId="21361" CreationDate="2023-09-10T12:48:05.947" Score="2" Body="&lt;p&gt;You can use &lt;code&gt;IOBR&lt;/code&gt; R package for this purpose. &lt;code&gt;IOBR&lt;/code&gt; is an R package to perform comprehensive analysis of tumor microenvironment and signatures for immuno-oncology.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;if (!requireNamespace(&amp;quot;remotes&amp;quot;, quietly = TRUE))&#xA;install.packages(&amp;quot;remotes&amp;quot;)&#xA;&#xA;if (!requireNamespace(&amp;quot;IOBR&amp;quot;, quietly = TRUE))&#xA;remotes::install_github(&amp;quot;IOBR/IOBR&amp;quot;)&#xA;&#xA;library(IOBR)&#xA;library(tidyverse)&#xA;&#xA;file = read.delim(file.choose(),row.names=1)        #bulk data from 4 samples&#xA; &#xA;# estimating the average macrophages signatures using cibersort method&#xA;cibersort&amp;lt;-deconvo_tme(eset = file, method = &amp;quot;cibersort&amp;quot;, arrays = TRUE, perm = 100 )&#xA;&#xA;head(cibersort)&#xA;&#xA;&#xA;#barplot&#xA;res&amp;lt;-cell_bar_plot(input = cibersort[,c(1,15:17)], title = &amp;quot;CIBERSORT Cell Fraction&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15931" LastActivityDate="2023-09-10T12:48:05.947" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21533" PostTypeId="1" CreationDate="2023-09-11T19:11:40.497" Score="2" ViewCount="269" Body="&lt;p&gt;I'm trying to perform Principal Component Analysis using R on a proteomics dataset.&lt;/p&gt;&#xA;&lt;p&gt;As the dataset contains a lot missing values I tried different approaches.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I ran PCA using &lt;code&gt;prcomp()&lt;/code&gt; and &lt;code&gt;autoplot()&lt;/code&gt; after deleting all samples with missing values. This worked out nicely, but the plots are not representative as I had to delete more than 70% of the samples.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I imputed the missing values and ran a PCA (again &lt;code&gt;prcomp()&lt;/code&gt; and &lt;code&gt;autoplot()&lt;/code&gt;) which worked out, but provided me with a PCA plot showing a strong horseshoe effect (which seems off as the PCA I performed on the complete samples of my data doesn't show such a distribution).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I then came across this article (&lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S1574954121000261&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.sciencedirect.com/science/article/pii/S1574954121000261&lt;/a&gt;) which introduces a modified PCA, InDaPCA (PCA of Incomplete Data). I transformed the code provided by the authors according to my data but again got errors/warnings because of missing values.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;This is the structure of my dataframe, the code I used to perform InDaPCA as well as the resulting error:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;num [1:261, 1:1132] 13.18 13.18 14.14 8.32 9.11 ...&#xA; - attr(*, &amp;quot;dimnames&amp;quot;)=List of 2&#xA;  ..&lt;span class=&quot;math-container&quot;&gt;$ : NULL&#xA;  ..$&lt;/span&gt; : chr [1:1132] &amp;quot;protein.A0A075B6H9&amp;quot; &amp;quot;protein.A0A075B6I0&amp;quot; &amp;quot;protein.A0A075B6I1&amp;quot; &amp;quot;protein.A0A075B6I9&amp;quot; ...&#xA;&#xA;InDaPCA &amp;lt;- function(protein_data_filtered) {&#xA;  &#xA;  #scaling&#xA;  X &amp;lt;- scale(protein_data, center = TRUE, scale = TRUE)&#xA;  &#xA;  #correlation&#xA;  C &amp;lt;- cor(X, use = &amp;quot;pairwise.complete.obs&amp;quot;)&#xA;  &#xA;  #Eigenvalue&#xA;  Eigenvalues &amp;lt;- eigen(C)$values&#xA;  Eigenvalues.pos &amp;lt;- Eigenvalues[Eigenvalues &amp;gt; 0]&#xA;  Eigenvalues.pos.as.percent &amp;lt;- 100 * Eigenvalues.pos / sum(Eigenvalues.pos)&#xA;  &#xA;  #Eigenvectors&#xA;  V &amp;lt;- eigen(C)$vectors&#xA;  &#xA;  #Principal components&#xA;  X2 &amp;lt;- X&#xA;  X2[is.na(X2)] &amp;lt;- 0&#xA;  PC &amp;lt;- as.matrix(X2) %*% V&#xA;  &#xA;  #object.standardized&#xA;  PCstand1 &amp;lt;- PC[, Eigenvalues &amp;gt; 0] / sqrt(Eigenvalues.pos)[col(PC[, Eigenvalues &amp;gt; 0])]&#xA;  PCstand2 &amp;lt;- PCstand1 / sqrt(nrow(PC) - 1)&#xA;  &#xA;  #loadings&#xA;  loadings &amp;lt;- cor(X, PC, use = &amp;quot;pairwise.complete.obs&amp;quot;)&#xA;  &#xA;  #arrows for biplot&#xA;  arrows &amp;lt;- cor(X, PC, use = &amp;quot;pairwise.complete.obs&amp;quot;) * sqrt(nrow(X) - 1)&#xA;  &#xA;  #output&#xA;  PCA &amp;lt;- list()&#xA;  PCA&lt;span class=&quot;math-container&quot;&gt;$Correlation.matrix &amp;lt;- C&#xA;  PCA$&lt;/span&gt;Eigenvalues &amp;lt;- Eigenvalues&#xA;  PCA&lt;span class=&quot;math-container&quot;&gt;$Positive.Eigenvalues &amp;lt;- Eigenvalues.pos&#xA;  PCA$&lt;/span&gt;Positive.Eigenvalues.as.percent &amp;lt;- Eigenvalues.pos.as.percent&#xA;  PCA&lt;span class=&quot;math-container&quot;&gt;$Square.root.of.eigenvalues &amp;lt;- sqrt(Eigenvalues.pos)&#xA;  PCA$&lt;/span&gt;Eigenvectors &amp;lt;- V&#xA;  PCA&lt;span class=&quot;math-container&quot;&gt;$Component.scores &amp;lt;- PC&#xA;  PCA$&lt;/span&gt;Variable.scores &amp;lt;- loadings&#xA;  PCA&lt;span class=&quot;math-container&quot;&gt;$Biplot.objects &amp;lt;- PCstand2&#xA;  PCA$&lt;/span&gt;Biplot.variables &amp;lt;- arrows&#xA;  &#xA;  return(PCA)&#xA;}&#xA;&#xA;pca_result &amp;lt;- InDaPCA(protein_data_filtered)&#xA;&#xA;Error in eigen(C) : unendliche oder fehlende Werte in 'x'&#xA;Called from: eigen(C)  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would highly appreciate any suggestions on how to perform PCA of datasets with many NA values and/or how to handle the horseshoe effect.&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot in advance!&lt;/p&gt;&#xA;" OwnerUserId="18164" LastEditorUserId="650" LastEditDate="2023-09-12T17:21:33.357" LastActivityDate="2023-09-12T17:21:33.357" Title="How to perform PCA on proteomic data set" Tags="|r|proteomics|pca|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21534" PostTypeId="1" AcceptedAnswerId="21537" CreationDate="2023-09-11T21:25:44.637" Score="3" ViewCount="108" Body="&lt;p&gt;I have one file with multiple SRA accession called &lt;code&gt;Dataframe_with_accession.txt&lt;/code&gt;, for this example I just put one SRA in that file called :&lt;code&gt;SRR8933535&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;And the idea is to create a nextflow pipeline to download the sra files in the &lt;code&gt;Dataframe_with_accession.txt&lt;/code&gt;, zip the files  using &lt;code&gt;pigz&lt;/code&gt; and remove the previous &lt;code&gt;fastq file&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;To do that I used this code :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;params.inputFile = &amp;quot;/crex/proj/Dataframe_with_accession.txt&amp;quot;&#xA;params.outputDir = &amp;quot;/crex/proj/Output&amp;quot;&#xA;&#xA;&#xA;process DOWNLOAD_FASTQ {&#xA;&#xA;    publishDir params.outputDir, mode: 'symlink'&#xA;&#xA;    input:&#xA;    each (accession)&#xA;&#xA;    output:&#xA;    tuple path(&amp;quot;params.outputDir/&lt;span class=&quot;math-container&quot;&gt;${accession}_1.fastq.gz&quot;)&#xA;    path(&quot;params.outputDir/$&lt;/span&gt;{accession}_2.fastq.gz&amp;quot;)&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    fasterq-dump --threads 12 --outdir &lt;span class=&quot;math-container&quot;&gt;${params.outputDir}  $&lt;/span&gt;{accession}&#xA;    pigz -p12 &lt;span class=&quot;math-container&quot;&gt;${params.outputDir}/$&lt;/span&gt;{accession}_1.fastq&#xA;    pigz -p12 &lt;span class=&quot;math-container&quot;&gt;${params.outputDir}/$&lt;/span&gt;{accession}_2.fastq&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;workflow {&#xA;    // Read accessions from the input file&#xA;    accList = file(params.inputFile).readLines()&#xA;&#xA;    DOWNLOAD_FASTQ(accList)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then I run this nextflow file.&lt;/p&gt;&#xA;&lt;p&gt;But I got the following error message :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[01/298f4b] process &amp;gt; DOWNLOAD_FASTQ (1) [100%] 1 of 1, failed: 1 ✘&#xA;Error executing process &amp;gt; 'DOWNLOAD_FASTQ (1)'&#xA;&#xA;Caused by:&#xA;  Missing output file(s) `SRR8933535_1.fastq` expected by process `DOWNLOAD_FASTQ (1)`&#xA;&#xA;Command executed:&#xA;&#xA;  fasterq-dump --threads 12 --outdir /crex/proj/Output  SRR8933535&#xA;&#xA;Command exit status:&#xA;  0&#xA;&#xA;Command output:&#xA;  (empty)&#xA;&#xA;Command error:&#xA;  spots read      : 8,389,148&#xA;  reads read      : 16,778,296&#xA;  reads written   : 16,778,296&#xA;&#xA;Work dir:&#xA;  /crex/proj/Output/work/01/298f4be6dde2ee1856df31db169bec&#xA;&#xA;Tip: you can try to figure out what's wrong by changing to the process work dir and showing the script file named `.command.sh`&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And I do not undestrand what is going wrong here ? It says&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Command output:&#xA;  (empty)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But the file is created&lt;/p&gt;&#xA;" OwnerUserId="2328" LastActivityDate="2023-09-12T11:25:28.027" Title="Error output in nextflow pipeline using fasterq-dump" Tags="|nextflow|fastq-dump|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21535" PostTypeId="1" CreationDate="2023-07-25T15:25:53.010" Score="0" ViewCount="13" Body="&lt;p&gt;I need to generate a list of sequences from a database, in which a split pattern is skipped up to n-times. As I only know R, I would appreciate an R solution.&lt;/p&gt;&#xA;&lt;p&gt;Example:&#xA;character string:&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;GRGKA&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;I could use a simple&lt;br /&gt;&#xA;&lt;code&gt;strsplit()&lt;/code&gt;&lt;br /&gt;&#xA;function but this would result in a &amp;quot;perfect&amp;quot; split pattern.&lt;/p&gt;&#xA;&lt;p&gt;What I am looking for is all combinations that are generated when n number of splits are skipped.&lt;/p&gt;&#xA;&lt;p&gt;I specifically want to split the string after K or R (so that K and R are still included in the reported split sequence)&lt;/p&gt;&#xA;&lt;p&gt;so a normal split would yield:&#xA;&amp;quot;GR&amp;quot; &amp;quot;GK&amp;quot; &amp;quot;A&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;however with one &amp;quot;skipped&amp;quot; split it would look like this:&#xA;&amp;quot;GRGK&amp;quot; &amp;quot;GKA&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;and with two skipped splits:&#xA;&amp;quot;GRGKA&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;I was not able to find something online. Sorry if I missed something or did not provide enough info.&lt;/p&gt;&#xA;&lt;p&gt;Here is the code that I started out with and which provides a normal split without skipping K or R&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;`&amp;quot;GRGKA&amp;quot; %&amp;gt;% strsplit(., &amp;quot;(?&amp;lt;=K|R)&amp;quot;, perl = TRUE)`&#xA;&#xA;[[1]]&#xA;[1] &amp;quot;GR&amp;quot; &amp;quot;GK&amp;quot; &amp;quot;A&amp;quot; &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17896" OwnerDisplayName="Klemens Fröhlich" LastActivityDate="2023-09-12T06:33:39.883" Title="skip splits in character string" Tags="|r|" AnswerCount="0" CommentCount="0" FavoriteCount="0" ClosedDate="2023-09-12T06:36:59.737" ContentLicense="CC BY-SA 4.0" />
  <row Id="21536" PostTypeId="2" ParentId="21358" CreationDate="2023-07-25T16:46:52.253" Score="0" Body="&lt;p&gt;&lt;strong&gt;1)&lt;/strong&gt; gsubfn is like gsub except the replacement can be a proto object containing a &lt;code&gt;fun&lt;/code&gt; proto method which is a function whose first argument receives the proto object and which inherits a &lt;code&gt;count&lt;/code&gt; (and other) built-ins.  The match is input to the function as the second argument and is replaced by the output of the function.  Here we replace each R or K with itself and if it is more than the k-th occurrence then with a semicolon as well.  Then split by semicolon.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(gsubfn)&#xA;&#xA;# inputs&#xA;x &amp;lt;- c(&amp;quot;GRGKA&amp;quot;, &amp;quot;GRGKAK&amp;quot;)  # vector of inputs&#xA;k &amp;lt;- 1 # no of splits to skip&#xA;&#xA;p &amp;lt;- proto(fun = function(self, x) paste0(x, if (count &amp;gt; k) &amp;quot;;&amp;quot;))&#xA;strsplit(gsubfn(&amp;quot;[RK]&amp;quot;, p, x), &amp;quot;;&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;giving&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[[1]]&#xA;[1] &amp;quot;GRGK&amp;quot; &amp;quot;A&amp;quot;   &#xA;&#xA;[[2]]&#xA;[1] &amp;quot;GRGK&amp;quot; &amp;quot;AK&amp;quot;  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;2)&lt;/strong&gt; Insert a semicolon after each R or K and then remove the first k.  &lt;code&gt;strsplit&lt;/code&gt; what is left.  This uses base R only.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;xx &amp;lt;- gsub(&amp;quot;([RK])&amp;quot;, &amp;quot;\\1;&amp;quot;, x)&#xA;for(i in seq_len(k)) xx &amp;lt;- sub(&amp;quot;;&amp;quot;, &amp;quot;&amp;quot;, xx)&#xA;strsplit(xx, &amp;quot;;&amp;quot;)&#xA;# same output as above&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerDisplayName="G. Grothendieck" LastActivityDate="2023-07-26T12:39:33.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21537" PostTypeId="2" ParentId="21534" CreationDate="2023-09-12T11:25:28.027" Score="2" Body="&lt;p&gt;Each nextflow process looks in its working directory for the files listed in &lt;code&gt;output:&lt;/code&gt; . In your &lt;code&gt;script:&lt;/code&gt; you already direct the output to its eventual destination. After running the script nextflow tries to find the output files and stops, because it can't find them. Solution: keep the output files in your workdir, nextflow will detect them and use the &lt;code&gt;publishDir&lt;/code&gt; directive to copy  them to their eventual destination.&lt;/p&gt;&#xA;" OwnerUserId="3818" LastActivityDate="2023-09-12T11:25:28.027" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21538" PostTypeId="1" CreationDate="2023-09-12T11:31:17.097" Score="1" ViewCount="92" Body="&lt;p&gt;Most of my experience has been working with genomic data, this time I am working on data from Smart-seq3. RNA excreted by cells in culture medium were processed with Smart-seq3 (From the looks of it, to capture as many fragments as possible). I have to perform differential expression analysis and check for any significant genes. I have the fastq files and trimmed them for adapters and polyA using trim_galore.&lt;/p&gt;&#xA;&lt;p&gt;The protocol suggested processing fastq file using zUMIs, however the data isn't traditional single cell sequencing, so as per one comment of their procotol, they replied that for bulk sequencing-esque applications, &lt;a href=&quot;https://www.protocols.io/view/smart-seq3-protocol-36wgq5rjxgk5/v3/comments?q=trim&quot; rel=&quot;nofollow noreferrer&quot;&gt;we can just trim out the UMIs&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I have not found a decent way to remove UMIs from fastq file, are there any tools to remove them? From what I've read, UMI_tools can extract the UMI information from the fastq file, but how I remove it from the fastq file?&lt;/p&gt;&#xA;" OwnerUserId="6561" LastActivityDate="2023-09-12T11:31:17.097" Title="Working with Smart-seq3 data, have some questions regarding UMI and bar coding" Tags="|rna-seq|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21539" PostTypeId="1" CreationDate="2023-09-12T19:42:05.317" Score="0" ViewCount="76" Body="&lt;p&gt;Im trying to run ProtDeform on ubuntu 20.04&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3125161/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3125161/&lt;/a&gt;&#xA;I need scoring function :/ to my project&lt;/p&gt;&#xA;&lt;p&gt;But error occurs:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(base) marcin@marcin-B550-AORUS-ELITE-AX-V2:~/ProtDeformV1.0$ ./install &#xA;rm: cannot remove 'ProtDeform': No such file or directory&#xA;Compiling...&#xA;gcc -c  -g Matras.c Amino.c Vector.c GotohDPdis.c DiffStr.c BetaTopo.c RegressL.c Jacobi3.c SecStrEle.c ScoreENV.c ScoreDIS.c ScoreSSE.c BuildUpL.c DiheDis.c RMS.c OutAln.c StrLibL.c OutLibL.c LlistIO.c CalZsc.c LlistAlign.c MergeSort.c PairAli.c IO.c&#xA;Matras.c:77:1: warning: return type defaults to ‘int’ [-Wimplicit-int]&#xA;   77 | main(argc,argv)&#xA;      | ^~~~&#xA;Matras.c: In function ‘main’:&#xA;Matras.c:126:3: warning: implicit declaration of function ‘Set_AAnum’ [-Wimplicit-function-declaration]&#xA;  126 |   Set_AAnum();&#xA;      |   ^~~~~~~~~&#xA;Matras.c:390:27: warning: zero-length gnu_printf format string [-Wformat-zero-length]&#xA;  390 |         sprintf(envscfile,&amp;quot;&amp;quot;);&#xA;      |                           ^~&#xA;Matras.c:431:23: warning: zero-length gnu_printf format string [-Wformat-zero-length]&#xA;  431 |   sprintf(PAR.COMMAND,&amp;quot;&amp;quot;);&#xA;      |                       ^~&#xA;Matras.c:436:29: warning: implicit declaration of function ‘Get_Date_String’; did you mean ‘Get_Time_String’? [-Wimplicit-function-declaration]&#xA;  436 |   printf(&amp;quot;#&amp;lt;DATE&amp;gt; \&amp;quot;%s\&amp;quot;\n&amp;quot;,Get_Date_String());&#xA;      |                             ^~~~~~~~~~~~~~~&#xA;      |                             Get_Time_String&#xA;Matras.c:436:22: warning: format ‘%s’ expects argument of type ‘char *’, but argument 2 has type ‘int’ [-Wformat=]&#xA;  436 |   printf(&amp;quot;#&amp;lt;DATE&amp;gt; \&amp;quot;%s\&amp;quot;\n&amp;quot;,Get_Date_String());&#xA;      |                     ~^      ~~~~~~~~~~~~~~~~~&#xA;      |                      |      |&#xA;      |                      char * int&#xA;      |                     %d&#xA;Matras.c:442:10: warning: too many arguments for format [-Wformat-extra-args]&#xA;  442 |   printf(&amp;quot;# QueFile \&amp;quot;%s\&amp;quot; LibFile \&amp;quot;%s\&amp;quot;\n&amp;quot;,MODE,QueFile,LibFile);&#xA;      |          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;Matras.c:498:4: warning: implicit declaration of function ‘Pairwise_3D_Alignment’ [-Wimplicit-function-declaration]&#xA;  498 |    Pairwise_3D_Alignment(&amp;amp;proA,&amp;amp;proB,&amp;amp;DmapA,&amp;amp;DmapB,&amp;amp;AHead,&amp;amp;SHead,inalifile,inssefile);&#xA;      |    ^~~~~~~~~~~~~~~~~~~~~&#xA;Matras.c:502:4: warning: implicit declaration of function ‘mainPD’; did you mean ‘main’? [-Wimplicit-function-declaration]&#xA;  502 |    mainPD(&amp;amp;proA, &amp;amp;proB, AHead.next, inalifile ); /* JR ProtDeform */&#xA;      |    ^~~~~~&#xA;      |    main&#xA;Matras.c:529:5: warning: implicit declaration of function ‘Output_Path_Horizontal_ClustalW’; did you mean ‘Output_Path_Horizontal_HTMLcolor’? [-Wimplicit-function-declaration]&#xA;  529 |     Output_Path_Horizontal_ClustalW(&amp;amp;proA,&amp;amp;proB,an,outclwfile);&#xA;      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;      |     Output_Path_Horizontal_HTMLcolor&#xA;Matras.c:726:6: warning: implicit declaration of function ‘Cal_Raw_MeanSD_Llist’; did you mean ‘Cal_Raw_Zscore_Llist’? [-Wimplicit-function-declaration]&#xA;  726 |      Cal_Raw_MeanSD_Llist(&amp;amp;ALibHead,&amp;amp;Mean,&amp;amp;SD);&#xA;      |      ^~~~~~~~~~~~~~~~~~~~&#xA;      |      Cal_Raw_Zscore_Llist&#xA;Matras.c:512:26: warning: ‘sprintf’ may write a terminating nul past the end of the destination [-Wformat-overflow=]&#xA;  512 |     sprintf(OutHead,&amp;quot;%s%d&amp;quot;,OheadStr,an-&amp;gt;num);&#xA;      |                          ^&#xA;Matras.c:512:5: note: ‘sprintf’ output between 2 and 139 bytes into a destination of size 128&#xA;  512 |     sprintf(OutHead,&amp;quot;%s%d&amp;quot;,OheadStr,an-&amp;gt;num);&#xA;      |     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;Matras.c:587:43: warning: ‘%s’ directive writing up to 127 bytes into a region of size between 113 and 240 [-Wformat-overflow=]&#xA;  587 |      { sprintf(command,&amp;quot;perl %s/rotSTR.pl %s.mat %s -nc A -S B &amp;gt; %s.pdb&amp;quot;,&#xA;      |                                           ^~&#xA;  588 |              BASE_DIR,OutHead,proAfile,OutHead);&#xA;      |                       ~~~~~~~              &#xA;Matras.c:587:8: note: ‘sprintf’ output between 40 and 484 bytes into a destination of size 256&#xA;  587 |      { sprintf(command,&amp;quot;perl %s/rotSTR.pl %s.mat %s -nc A -S B &amp;gt; %s.pdb&amp;quot;,&#xA;      |        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;  588 |              BASE_DIR,OutHead,proAfile,OutHead);&#xA;      |              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;Matras.c:590:62: warning: ‘%s’ directive writing up to 127 bytes into a region of size between 33 and 223 [-Wformat-overflow=]&#xA;  590 |        sprintf(command,&amp;quot;perl %s/rotSTR.pl - %s -nc B -S B &amp;gt;&amp;gt; %s.pdb&amp;quot;,BASE_DIR,proBfile,OutHead);&#xA;      |                                                              ^~                        ~~~~~~~&#xA;Matras.c:590:8: note: ‘sprintf’ output between 38 and 355 bytes into a destination of size 256&#xA;  590 |        sprintf(command,&amp;quot;perl %s/rotSTR.pl - %s -nc B -S B &amp;gt;&amp;gt; %s.pdb&amp;quot;,BASE_DIR,proBfile,OutHead);&#xA;      |        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;Matras.c:594:43: warning: ‘%s’ directive writing up to 127 bytes into a region of size between 113 and 240 [-Wformat-overflow=]&#xA;  594 |      { sprintf(command,&amp;quot;perl %s/rotSTR.pl %s.mat %s -nc A -S P -ch %c -H W &amp;gt; %s.pdb&amp;quot;,&#xA;      |                                           ^~&#xA;  595 |            BASE_DIR,OutHead,proAfile,proA.chain,OutHead);&#xA;      |                     ~~~~~~~                &#xA;Matras.c:594:8: note: ‘sprintf’ output between 51 and 495 bytes into a destination of size 256&#xA;  594 |      { sprintf(command,&amp;quot;perl %s/rotSTR.pl %s.mat %s -nc A -S P -ch %c -H W &amp;gt; %s.pdb&amp;quot;,&#xA;      |        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;  595 |            BASE_DIR,OutHead,proAfile,proA.chain,OutHead);&#xA;      |            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;Matras.c:597:73: warning: ‘%s’ directive writing up to 127 bytes into a region of size between 23 and 213 [-Wformat-overflow=]&#xA;  597 |       sprintf(command,&amp;quot;perl %s/rotSTR.pl - %s -nc B -S P -ch %c -H W&amp;gt;&amp;gt; %s.pdb&amp;quot;,&#xA;      |                                                                        ^~&#xA;&#xA;  598 |         BASE_DIR,proBfile,proB.chain,OutHead);&#xA;      |                                      ~~~~~~~                            &#xA;&#xA;Matras.c:597:8: note: ‘sprintf’ output between 48 and 365 bytes into a destination of size 256&#xA;  597 |        sprintf(command,&amp;quot;perl %s/rotSTR.pl - %s -nc B -S P -ch %c -H W&amp;gt;&amp;gt; %s.pdb&amp;quot;,&#xA;      |        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;  598 |          BASE_DIR,proBfile,proB.chain,OutHead);&#xA;      |          ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;Amino.c: In function ‘Amino_Name’:&#xA;Amino.c:104:11: warning: implicit declaration of function ‘strncmp’ [-Wimplicit-function-declaration]&#xA;  104 |       if (strncmp(t,&amp;quot;ALA&amp;quot;,3)==0)  r = 'A';&#xA;      |           ^~~~~~~&#xA;Amino.c:104:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  104 |       if (strncmp(t,&amp;quot;ALA&amp;quot;,3)==0)  r = 'A';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:105:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  105 |  else if (strncmp(t,&amp;quot;GLU&amp;quot;,3)==0)  r = 'E';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:106:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  106 |  else if (strncmp(t,&amp;quot;GLN&amp;quot;,3)==0)  r = 'Q';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:107:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  107 |  else if (strncmp(t,&amp;quot;ASP&amp;quot;,3)==0)  r = 'D';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:108:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  108 |  else if (strncmp(t,&amp;quot;ASN&amp;quot;,3)==0)  r = 'N';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:109:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  109 |  else if (strncmp(t,&amp;quot;LEU&amp;quot;,3)==0)  r = 'L';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:110:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  110 |  else if (strncmp(t,&amp;quot;GLY&amp;quot;,3)==0)  r = 'G';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:111:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  111 |  else if (strncmp(t,&amp;quot;LYS&amp;quot;,3)==0)  r = 'K';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:112:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  112 |  else if (strncmp(t,&amp;quot;SER&amp;quot;,3)==0)  r = 'S';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:113:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  113 |  else if (strncmp(t,&amp;quot;VAL&amp;quot;,3)==0)  r = 'V';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:114:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  114 |  else if (strncmp(t,&amp;quot;ARG&amp;quot;,3)==0)  r = 'R';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:115:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  115 |  else if (strncmp(t,&amp;quot;THR&amp;quot;,3)==0)  r = 'T';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:116:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  116 |  else if (strncmp(t,&amp;quot;PRO&amp;quot;,3)==0)  r = 'P';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:117:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  117 |  else if (strncmp(t,&amp;quot;ILE&amp;quot;,3)==0)  r = 'I';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:118:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  118 |  else if (strncmp(t,&amp;quot;MET&amp;quot;,3)==0)  r = 'M';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:119:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  119 |  else if (strncmp(t,&amp;quot;PHE&amp;quot;,3)==0)  r = 'F';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:120:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  120 |  else if (strncmp(t,&amp;quot;TYR&amp;quot;,3)==0)  r = 'Y';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:121:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  121 |  else if (strncmp(t,&amp;quot;CYS&amp;quot;,3)==0)  r = 'C';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:122:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  122 |  else if (strncmp(t,&amp;quot;TRP&amp;quot;,3)==0)  r = 'W';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:123:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  123 |  else if (strncmp(t,&amp;quot;HIS&amp;quot;,3)==0)  r = 'H';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:124:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  124 |  else if (strncmp(t,&amp;quot;UNK&amp;quot;,3)==0)  r = 'u';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:125:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  125 |  else if (strncmp(t,&amp;quot;ACE&amp;quot;,3)==0)  r = 'a';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;Amino.c:126:27: warning: ‘strncmp’ argument 3 type is ‘int’ where ‘long unsigned int’ is expected in a call to built-in function declared without prototype [-Wbuiltin-declaration-mismatch]&#xA;  126 |  else if (strncmp(t,&amp;quot;FOR&amp;quot;,3)==0)  r = 'f';&#xA;      |                           ^&#xA;&amp;lt;built-in&amp;gt;: note: built-in ‘strncmp’ declared here&#xA;GotohDPdis.c: In function ‘GotohDP_Dscore_Iterative’:&#xA;GotohDPdis.c:202:3: warning: implicit declaration of function ‘ScoreDistanceSPair’; did you mean ‘ScoreDistanceS’? [-Wimplicit-function-declaration]&#xA;  202 |   ScoreDistanceSPair(proA,proB,an,DmapA,DmapB);&#xA;      |   ^~~~~~~~~~~~~~~~~~&#xA;      |   ScoreDistanceS&#xA;GotohDPdis.c: In function ‘GotohDP_Dscore’:&#xA;GotohDPdis.c:223:33: warning: format ‘%s’ expects argument of type ‘char *’, but argument 2 has type ‘int’ [-Wformat=]&#xA;  223 |   { printf(&amp;quot;#ERROR:PRE.MaxNali %s is less than Nali %d\n&amp;quot;,PRE.MaxNali,an-&amp;gt;Nali);&#xA;      |                                ~^                         ~~~~~~~~~~~&#xA;      |                                 |                            |&#xA;      |                                 char *                       int&#xA;      |                                %d&#xA;DiffStr.c: In function ‘Number_Sec’:&#xA;DiffStr.c:202:5: warning: type of ‘sym’ defaults to ‘int’ [-Wimplicit-int]&#xA;  202 | int Number_Sec(sym,Nstr)&#xA;      |     ^~~~~~~~~~&#xA;SecStrEle.c: In function ‘Output_SSE_PDB’:&#xA;SecStrEle.c:355:13: warning: too many arguments for format [-Wformat-extra-args]&#xA;  355 |  fprintf(fp,&amp;quot;REMARK tFactor : 0 for Cend of SSE, 200 for  helix, 100 for sheet\n&amp;quot;,pr-&amp;gt;pname);&#xA;      |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;ScoreENV.c: In function ‘Number_Sec’:&#xA;ScoreENV.c:313:5: warning: type of ‘sym’ defaults to ‘int’ [-Wimplicit-int]&#xA;  313 | int Number_Sec(sym,Nstr)&#xA;      |     ^~~~~~~~~~&#xA;ScoreENV.c: In function ‘Read_Env_Score’:&#xA;ScoreENV.c:264:22: warning: ‘sprintf’ may write a terminating nul past the end of the destination [-Wformat-overflow=]&#xA;  264 |  sprintf(Fname,&amp;quot;%s/%s&amp;quot;,SCORE_DIR,fname);&#xA;      |                      ^&#xA;ScoreENV.c:264:2: note: ‘sprintf’ output 2 or more bytes (assuming 129) into a destination of size 128&#xA;  264 |  sprintf(Fname,&amp;quot;%s/%s&amp;quot;,SCORE_DIR,fname);&#xA;      |  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;BuildUpL.c: In function ‘Build_Up_LL’:&#xA;BuildUpL.c:256:2: warning: implicit declaration of function ‘Merge_Sort_Double_Linked_List’ [-Wimplicit-function-declaration]&#xA;  256 |  Merge_Sort_Double_Linked_List(head,'R');&#xA;      |  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;BuildUpL.c: In function ‘Read_SSE_Alignment’:&#xA;BuildUpL.c:677:4: warning: implicit declaration of function ‘Get_Part_Of_Line’ [-Wimplicit-function-declaration]&#xA;  677 |    Get_Part_Of_Line(num1,word1,1,strlen(word1));&#xA;      |    ^~~~~~~~~~~~~~~~&#xA;OutAln.c: In function ‘Write_Rasmol_Color_Alignment’:&#xA;OutAln.c:564:34: warning: implicit declaration of function ‘Get_Date_String’ [-Wimplicit-function-declaration]&#xA;  564 |  fprintf(fp,&amp;quot;# DATE      '%s'\n&amp;quot;,Get_Date_String());&#xA;      |                                  ^~~~~~~~~~~~~~~&#xA;OutAln.c:564:28: warning: format ‘%s’ expects argument of type ‘char *’, but argument 3 has type ‘int’ [-Wformat=]&#xA;  564 |  fprintf(fp,&amp;quot;# DATE      '%s'\n&amp;quot;,Get_Date_String());&#xA;      |                           ~^     ~~~~~~~~~~~~~~~~~&#xA;      |                            |     |&#xA;      |                            |     int&#xA;      |                            char *&#xA;      |                           %d&#xA;OutAln.c: In function ‘Write_Rasmol_Color_Alignment_Each_Protein’:&#xA;OutAln.c:715:27: warning: format ‘%s’ expects argument of type ‘char *’, but argument 3 has type ‘int’ [-Wformat=]&#xA;  715 |  fprintf(fp,&amp;quot;# DATE      %s\n&amp;quot;,Get_Date_String());&#xA;      |                          ~^    ~~~~~~~~~~~~~~~~~&#xA;      |                           |    |&#xA;      |                           |    int&#xA;      |                           char *&#xA;      |                          %d&#xA;OutAln.c:731:10: warning: implicit declaration of function ‘Ncompare’ [-Wimplicit-function-declaration]&#xA;  731 |  Ncomp = Ncompare(a);&#xA;      |          ^~~~~~~~&#xA;OutAln.c: In function ‘Write_Molscript_Alignment_Each_Protein’:&#xA;OutAln.c:880:27: warning: format ‘%s’ expects argument of type ‘char *’, but argument 3 has type ‘int’ [-Wformat=]&#xA;  880 |  fprintf(fp,&amp;quot;! DATE      %s\n&amp;quot;,Get_Date_String());&#xA;      |                          ~^    ~~~~~~~~~~~~~~~~~&#xA;      |                           |    |&#xA;      |                           |    int&#xA;      |                           char *&#xA;      |                          %d&#xA;OutAln.c:905:35: warning: implicit declaration of function ‘Remove_Space’ [-Wimplicit-function-declaration]&#xA;  905 |        if (OutProteinType == 'A') Remove_Space(rsta,pA-&amp;gt;Rnum[a-&amp;gt;nalA[i]]);&#xA;      |                                   ^~~~~~~~~~~~&#xA;OutAln.c: In function ‘Find_nal’:&#xA;OutAln.c:962:5: warning: type of ‘q’ defaults to ‘int’ [-Wimplicit-int]&#xA;  962 | int Find_nal(q,Nali,nalA)&#xA;      |     ^~~~~~~~&#xA;OutAln.c: In function ‘Output_Alignment_Information’:&#xA;OutAln.c:1058:2: warning: implicit declaration of function ‘Get_Part_Of_Line’ [-Wimplicit-function-declaration]&#xA; 1058 |  Get_Part_Of_Line(ln,pA-&amp;gt;compnd,0,40);&#xA;      |  ^~~~~~~~~~~~~~~~&#xA;StrLibL.c: In function ‘Read_Library_List’:&#xA;StrLibL.c:75:5: warning: implicit declaration of function ‘Get_First_Field’ [-Wimplicit-function-declaration]&#xA;   75 |     Get_First_Field(name,tax,buffer);&#xA;      |     ^~~~~~~~~~~~~~~&#xA;StrLibL.c: In function ‘Check_Library_List’:&#xA;StrLibL.c:113:3: warning: implicit declaration of function ‘Get_Part_Of_Line’ [-Wimplicit-function-declaration]&#xA;  113 |   Get_Part_Of_Line(dir,bn-&amp;gt;Name,1,2);&#xA;      |   ^~~~~~~~~~~~~~~~&#xA;OutLibL.c: In function ‘Output_Library_ClustalW’:&#xA;OutLibL.c:234:4: warning: implicit declaration of function ‘Get_Part_Of_Line’ [-Wimplicit-function-declaration]&#xA;  234 |    Get_Part_Of_Line(ln,bn-&amp;gt;sim-&amp;gt;Name,0,16);&#xA;      |    ^~~~~~~~~~~~~~~~&#xA;LlistIO.c: In function ‘Read_BSSP_File_LinearList’:&#xA;LlistIO.c:161:2: warning: implicit declaration of function ‘Set_SSE’ [-Wimplicit-function-declaration]&#xA;  161 |  Set_SSE(pr);&#xA;      |  ^~~~~~~&#xA;LlistIO.c:162:2: warning: implicit declaration of function ‘Set_Dis_bwn_SSE’ [-Wimplicit-function-declaration]&#xA;  162 |  Set_Dis_bwn_SSE(pr,'C');&#xA;      |  ^~~~~~~~~~~~~~~&#xA;LlistIO.c: In function ‘Translate_BSSP_File_LinearList’:&#xA;LlistIO.c:218:51: warning: implicit declaration of function ‘tolower’ [-Wimplicit-function-declaration]&#xA;  218 |             if (X-&amp;gt;line[75]=='C') pr-&amp;gt;Seq[Rnum] = tolower(pr-&amp;gt;Seq[Rnum]);  /* for CIS conf */&#xA;      |                                                   ^~~~~~~&#xA;PairAli.c: In function ‘Pairwise_3D_Alignment’:&#xA;PairAli.c:188:3: warning: implicit declaration of function ‘Set_Value_AlignNode’; did you mean ‘Tail_Value_AlignNode’? [-Wimplicit-function-declaration]&#xA;  188 |   Set_Value_AlignNode(Ahead,'S');    /* JR  It was  D */&#xA;      |   ^~~~~~~~~~~~~~~~~~~&#xA;      |   Tail_Value_AlignNode&#xA;PairAli.c:189:3: warning: implicit declaration of function ‘Merge_Sort_Double_Linked_List’ [-Wimplicit-function-declaration]&#xA;  189 |   Merge_Sort_Double_Linked_List(Ahead,'R');&#xA;      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;IO.c: In function ‘Read_BSSP_File’:&#xA;IO.c:705:3: warning: implicit declaration of function ‘Set_SSE’ [-Wimplicit-function-declaration]&#xA;  705 |   Set_SSE(pr);&#xA;      |   ^~~~~~~&#xA;IO.c:706:3: warning: implicit declaration of function ‘Set_Dis_bwn_SSE’ [-Wimplicit-function-declaration]&#xA;  706 |   Set_Dis_bwn_SSE(pr,'C');&#xA;      |   ^~~~~~~~~~~~~~~&#xA;IO.c: In function ‘Read_Sequence’:&#xA;IO.c:886:13: warning: implicit declaration of function ‘isalpha’ [-Wimplicit-function-declaration]&#xA;  886 |    else if (isalpha(sym)!=0)&#xA;      |             ^~~~~~~&#xA;IO.c: In function ‘Read_Matras_Environment_File’:&#xA;IO.c:87:23: warning: ‘%s’ directive writing up to 63 bytes into a region of size between 0 and 127 [-Wformat-overflow=]&#xA;   87 |   sprintf(Matfile,&amp;quot;%s/%s&amp;quot;,homedir,matfile);&#xA;      |                       ^~          ~~~~~~~&#xA;IO.c:87:3: note: ‘sprintf’ output between 2 and 192 bytes into a destination of size 128&#xA;   87 |   sprintf(Matfile,&amp;quot;%s/%s&amp;quot;,homedir,matfile);&#xA;      |   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;IO.c: In function ‘Read_PDB_File’:&#xA;IO.c:378:20: warning: ‘%s’ directive writing up to 3 bytes into a region of size between 0 and 127 [-Wformat-overflow=]&#xA;  378 |  sprintf(Fname,&amp;quot;%s/%s/%s&amp;quot;,PDB_DIR,dir,fname);&#xA;      |                    ^~             ~~~&#xA;IO.c:378:2: note: ‘sprintf’ output 3 or more bytes (assuming 133) into a destination of size 128&#xA;  378 |  sprintf(Fname,&amp;quot;%s/%s/%s&amp;quot;,PDB_DIR,dir,fname);&#xA;      |  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~&#xA;g++ -c  -g vector.cpp TransSeq.cpp Geometry.cpp WriteDeformPDB.cpp Evaluation.cpp  -w&#xA;Geometry.cpp:30:31: error: ‘&amp;lt;unnamed enum&amp;gt; GEOMETRY’, declared using unnamed type, is used but never defined [-fpermissive]&#xA;   30 | extern enum { RIGID, AFFINE } GEOMETRY;&#xA;      |                               ^~~~~~~~&#xA;WriteDeformPDB.cpp:16:31: error: ‘&amp;lt;unnamed enum&amp;gt; GEOMETRY’, declared using unnamed type, is used but never defined [-fpermissive]&#xA;   16 | extern enum { RIGID, AFFINE } GEOMETRY;&#xA;      |                               ^~~~~~~~&#xA;make: *** [Makefile:125: ../ProtDeform] Error 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've contacted the program developers about this, but am still waiting for answer from them.&lt;/p&gt;&#xA;&lt;p&gt;Can you help?&lt;/p&gt;&#xA;&lt;p&gt;EDIT1:&#xA;ox&#xA;I've checked&lt;/p&gt;&#xA;&lt;p&gt;Ubuntu 16.04 on virtualbox&lt;/p&gt;&#xA;&lt;p&gt;Ubuntu 20.04 with i7&lt;/p&gt;&#xA;&lt;p&gt;Ubuntu 20.04 with Ryzen 9&lt;/p&gt;&#xA;&lt;p&gt;same error...&lt;/p&gt;&#xA;&lt;p&gt;EDIT 2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;I have found hidden file .matras:&#xA;###############################&#xA;### MATRAS ENVIRONMENT FILE ###&#xA;###############################&#xA;&#xA;BASE_DIR .&#xA;SCORE_DIR  data_sc/ROM-04JAN29&#xA;TMP_DIR    .&#xA;BSSP_DIR  /DB/BSSP&#xA;PDB_DIR   /DB/PDB&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm checking to see if it simply requires PDB&lt;/p&gt;&#xA;" OwnerUserId="1224" LastEditorUserId="1224" LastEditDate="2023-09-16T12:26:35.443" LastActivityDate="2023-09-16T12:26:35.443" Title="Old program make error ProtDeform" Tags="|proteins|protein-structure|bash|perl|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21540" PostTypeId="1" CreationDate="2023-09-12T21:15:59.583" Score="2" ViewCount="68" Body="&lt;p&gt;I have a quick question about paired-end short reads. I have multiple genomes that were sequenced with paired-end Illumina NextSeq 200 technology, resulting in two fastq files per sample: &lt;code&gt;001_R1.fastq.gz&lt;/code&gt; and &lt;code&gt;001_R2.fastq.gz&lt;/code&gt;, typical for short read data.&lt;/p&gt;&#xA;&lt;p&gt;I would like to use a k-mer counting and clustering approach to put these genomes together on a phylogenetic tree. I was able to use both reads in one go to count k-mers but, as you can probably guess, this resulted in a very high frequency of k-mers identified and accounted for.&lt;/p&gt;&#xA;&lt;p&gt;For this specific use case (not interested in changing my approach), would it make more sense to just use a single file for kmer counting (i.e. only the &lt;code&gt;001_R1.fastq.gz&lt;/code&gt; file)?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&#xA;For more context, I'm trying to do an alignment-free approach to produce a phylogenetic distance comparison of these strains. I was using long reads before and it worked fine but Nanopore reads tend to be very messy. I have short-read data for all of the strains and would like to use that instead but that comes at the price of an overload of data.&lt;/p&gt;&#xA;&lt;p&gt;I suppose my main concern/question is how do I pre-process these short reads without assembling or aligning to a reference to count kmers? Or is that not possible without at least alignment?&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="16067" LastEditDate="2023-09-13T15:49:06.453" LastActivityDate="2023-09-14T15:14:36.497" Title="paired-end short reads: will one file suffice?" Tags="|sequence-analysis|illumina|k-mer|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21541" PostTypeId="2" ParentId="14113" CreationDate="2023-09-12T23:17:13.713" Score="-1" Body="&lt;p&gt;@Matteo Ferla wrote in the comments,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;It sounds like you'd want a MSA of the whole set and not a set of pairwise ones. If global MSA is what you actually want, try ebi.ac.uk/Tools/msa/muscle. Then to trim the ends before phylogeny there are dozens of tools.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The reason is because Blast aligns locally, i.e. a window, while a global alignment is the whole entirety of the sequence&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-09-12T23:24:24.687" LastActivityDate="2023-09-12T23:24:24.687" CommentCount="0" CommunityOwnedDate="2023-09-12T23:17:20.260" ContentLicense="CC BY-SA 4.0" />
  <row Id="21542" PostTypeId="2" ParentId="21540" CreationDate="2023-09-13T01:52:47.653" Score="1" Body="&lt;p&gt;The problem with using single-end short reads is that they're often not long enough to uniquely match to a database (especially for highly-conserved genes, like the ribosomal RNAs). Whether or not this is a problem for your particular situation is unclear; you haven't provided enough context / information for me to work that out. For example, a single end of a 2x50bp run on a NextSeq 2000 might cause problems (especially for medium to large eukaryote genomes), but a 2x300bp run on a NextSeq 2000 is probably okay.&lt;/p&gt;&#xA;&lt;p&gt;FWIW, Kraken2 has a &lt;code&gt;paired&lt;/code&gt; mode, which combines the results from both read ends from the same template source:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Usage of --paired also affects the --classified-out and --unclassified-out options; users should provide a # character in the filenames provided to those options, which will be replaced by kraken2 with &amp;quot;_1&amp;quot; and &amp;quot;_2&amp;quot; with mates spread across the two files appropriately. For example:&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;kraken2 --paired --classified-out cseqs#.fq seqs_1.fq seqs_2.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;See the Wiki / manual here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/DerrickWood/kraken2/wiki/Manual#classification&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/DerrickWood/kraken2/wiki/Manual#classification&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-09-13T01:52:47.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21543" PostTypeId="1" CreationDate="2023-09-13T12:08:36.657" Score="1" ViewCount="59" Body="&lt;p&gt;Utilizing a reference panel, I want to assign most likely population label to each individual in the study. Following are the files I have:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Reference panel population labels:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;HG00121 EUR&#xA;HG00122 EUR&#xA;HG00123 EUR&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Reference panel PC (Principal Component) coordinates:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;HG00121 -0.0108487      0.0259052       -0.0099605      0.0168545&#xA;HG00122 -0.0115789      0.0264642       -0.01722        0.0157885&#xA;HG00123 -0.00996687     0.0266983       -0.0155755      0.0155963&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Study individual PC coordinates:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Study_individual1 -0.0108487      0.0259052       -0.0099605      0.0168545&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;How do I go about assigning a population label to the study individual based of the reference panel? Most of the softwares I have come across such as evaluate_check_ancestry(plinkQC), FRAPOSA rely on inputs provided to PLINK which is not helpful in my case.&lt;/p&gt;&#xA;" OwnerUserId="4850" LastEditorUserId="4850" LastEditDate="2023-09-13T19:20:31.340" LastActivityDate="2023-09-13T19:20:31.340" Title="Predict population based on PC coordinates" Tags="|pca|population-genetics|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21544" PostTypeId="2" ParentId="21543" CreationDate="2023-09-13T16:09:15.353" Score="1" Body="&lt;p&gt;I apologise for being negative about this, but I am afraid you cannot use principle components for population inference other than assigning clusters within a bivariate or 3D graph. Eigen values are complex derivatives of data to achieve dimensionality reduction.&lt;/p&gt;&#xA;&lt;p&gt;For population genetics you require a sample size =&amp;gt; 10. I don't know the organism nor the question but goal is to derive allelic frequencies for each genome position. A common metric is F-statistics, for example Fst or Hardy-Weinberg equilibrium.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-09-13T16:09:15.353" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21545" PostTypeId="2" ParentId="21543" CreationDate="2023-09-13T16:16:53.847" Score="1" Body="&lt;p&gt;Here is a very simple way to get your started. Of course you can make it more complex:&lt;/p&gt;&#xA;&lt;p&gt;Take the mean PC1 and PC2 coordinates for each 'reference panel population label' and then calculate the reference panel population which has the closest Cartesian distance to your target, i.e. &lt;span class=&quot;math-container&quot;&gt;$\sqrt{{(PC1_{r} - PC1_{t})^2 + (PC2_{r} - PC2_{t})^2}}$&lt;/span&gt;, where &lt;span class=&quot;math-container&quot;&gt;$PC1_{r}$&lt;/span&gt; is the mean PC1 for a ref population and &lt;span class=&quot;math-container&quot;&gt;$PC1_{t}$&lt;/span&gt; is PC1 for the target.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastEditorUserId="650" LastEditDate="2023-09-13T17:57:37.063" LastActivityDate="2023-09-13T17:57:37.063" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21546" PostTypeId="1" CreationDate="2023-09-13T16:48:01.910" Score="1" ViewCount="46" Body="&lt;p&gt;I feel like this might be a really silly question and I've been going back and forth for awhile and I think confusing myself- so sorry if this is dumb.&lt;/p&gt;&#xA;&lt;p&gt;I am using Seqmonk to visualize genes in an assembly (but I suspect the issue would be the same in any visualization software). I am trying to identify promoters of specific genes to determine if they are differentially methylated. Genes on the forward strand (highlighted in red), the promoter would inherently be upstream- in this case to the left (i.e. if my start site was 5000 and the end site 6000, I would assume I could find my promoter to the left of the start site between 3000 and 5000 bases.&lt;/p&gt;&#xA;&lt;p&gt;My confusion is what happens with genes that are on the reverse strand (in blue here). Start site and end site could be the same (5000 and 6000- as end sites are always bigger than the start site). Does this mean that in the software, the promoter is expected to be on the right hand side (i.e. bases 6000-8000)? This has been my going thought but now I wonder if having bases that seem downstream (because they're bigger) makes sense, even though it really does because downstream on the forward strand would be upstream on the reverse strand. Or if I need to always do 3000-5000 bases as the promoter, regardless of the the strand of origin.&lt;/p&gt;&#xA;" OwnerUserId="18170" LastActivityDate="2023-09-13T17:58:50.480" Title="Identifying the promoter on the reverse strand using visualization software (seqmonk)" Tags="|genome|sequence-annotation|visualization|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21547" PostTypeId="2" ParentId="21540" CreationDate="2023-09-13T16:58:03.980" Score="3" Body="&lt;p&gt;I suspect you won't be able to avoid assembling or aligning. If you are working with strains that don't have their own, well studied reference genome and so want to avoid aligning so as not to bias your data towards whichever strain has been chosen as the reference for your species (please correct me if this isn't what's going on), then you will need to assemble a denovo genome from these reads. At the very least you will need to build the longest consensus contigs you can. Unfortunately, this isn't something I have experience with so I can't help you do this beyond telling you that, as I'm sure you already know, there are tools out there that will try to assemble your reads into contigs.&lt;/p&gt;&#xA;&lt;p&gt;I will instead try to explain why I think you can't avoid assembly. Short read fastq files, paired or not, contain short sequences of ~150-200 base pairs each. Now, depending on the depth of coverage, each region of your genome will be covered by multiple such short reads. Therefore, the data you have in your fastq files will be extremely repetitive and redundant since each of your genome's regions will be represented by multiple reads, and any attempt to find k-mers will be dogged by this redundancy. This is why you got so many more results than you expected and I don't see how using one file only would help at all.&lt;/p&gt;&#xA;&lt;p&gt;If you had sequenced using an assay that includes &lt;a href=&quot;https://www.illumina.com/techniques/sequencing/ngs-library-prep/multiplexing/unique-molecular-identifiers.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;unique molecular identifiers&lt;/a&gt; (UMIs), then you would be able to extract non-redundant, representative reads for all regions but I cannot think of a way to do this without UMIs off the top of my head. Not saying it isn't possible, only that you need someone more clever than I.&lt;/p&gt;&#xA;&lt;p&gt;That said, the best I can come up with is that &lt;strong&gt;if&lt;/strong&gt; your samples were sequenced in the same sequencing run and so can be assumed to have the same sequencing biases, you might be able to build your trees despite the redundancy in the data. My thinking here is that what you are really interested in isn't the k-mers themselves or their level of over-representation in each species, but only the difference in the k-mer content across your samples. If so, given that you sequenced all samples in the same run, they should all have the same level of over-representation and any differences might be enough for you to be able to cluster/build your trees.&lt;/p&gt;&#xA;&lt;p&gt;This seems worth a try to me, but I haven't worked on something like this in many years so if you go down this route, make sure to double check and ensure I'm not leading you astray!&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2023-09-14T15:14:36.497" LastActivityDate="2023-09-14T15:14:36.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21548" PostTypeId="2" ParentId="21546" CreationDate="2023-09-13T17:58:50.480" Score="2" Body="&lt;p&gt;Have a look at this UCSC browser shot.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/ylq48.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ylq48.png&quot; alt=&quot;UCSC browser shot of WDR92 showing promoter histone marks to the right of the transcribed unit on human chromosome 2&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Looking at the &lt;em&gt;WDR92&lt;/em&gt; transcript, note that it is transcribed right to left. This would indicate that the promoter is on the right. Indeed, we see a variety of promoter histone marks on the right of the gene, but not on the left.&lt;/p&gt;&#xA;&lt;p&gt;In other words, yes, for genes on the reverse strand, the larger coordinate will be at the beginning of the gene, and the promoter will include a larger coordinate than the rightmost transcribed base.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-09-13T17:58:50.480" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21549" PostTypeId="1" AcceptedAnswerId="21775" CreationDate="2023-09-13T23:00:04.167" Score="4" ViewCount="44" Body="&lt;p&gt;I have (human) Illumina genome wide SNP array copy number data. For each SNP genomewide, I have Log R Ratio (LRR) and B Allele Frequency (BAF).&lt;/p&gt;&#xA;&lt;p&gt;What tool(s) can I use to get the integer copy numbers (either -3 to +3 or 0 to inf) for each cytoband and for each gene in the genome?&lt;/p&gt;&#xA;&lt;p&gt;I assume that people routinely do this? Are there any R packages/pipelines/code to do this?&lt;/p&gt;&#xA;&lt;p&gt;Do people use &lt;code&gt;DNAcopy&lt;/code&gt; and/or &lt;code&gt;GISTIC&lt;/code&gt; for this?&lt;/p&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="7354" LastActivityDate="2023-10-29T16:32:34.087" Title="How to get cytoband and gene level copy number from genome wide SNP array copy number data?" Tags="|r|genome|illumina|copy-number|snp-chip|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21550" PostTypeId="1" CreationDate="2023-09-14T19:26:05.103" Score="1" ViewCount="33" Body="&lt;p&gt;My objective is to take a g.vcf.gz file and from 25-30 unmapped contigs with titles like &amp;quot;NW_020192317.1&amp;quot;, I want to make a subset of ~10k variants from each of the unmapped contigs and make one final g.vcf file that includes the header from the original g.vcf.gz file.&lt;/p&gt;&#xA;&lt;p&gt;From the post &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/3565/subset-smaller-bam-to-contain-several-thousand-rows-from-multiple-chromosomes&quot;&gt;here&lt;/a&gt;, I have tried running:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -bo output.g.vcf -s 123.4 inputfile.g.vcf.gz NW_020192317.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and I keep receiving the output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Aborted (core dumped)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm not too picky about the number of variants per contig, as long as it is in the thousands, that's why I tried using &amp;quot;.4&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, I tried using the code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view input.g.vcf.gz &amp;quot;NW_020192317.1:1-2&amp;quot; &amp;gt; output.g.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and I recieved the same response of &amp;quot;Aborted (core dumped)&amp;quot; with nothing else. I am confused what is wrong here and why it is throwing an error.&lt;/p&gt;&#xA;" OwnerUserId="18178" LastActivityDate="2023-09-22T17:18:58.923" Title="I am trying to create a subset of 10k variants from 25-30 unmapped contigs of a g.vcf file including the header" Tags="|vcf|samtools|gatk|subset|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21551" PostTypeId="2" ParentId="402" CreationDate="2023-09-15T00:17:45.753" Score="0" Body="&lt;p&gt;It have used both sambamba and samtools. Either one works great to downsample to a desired read depth after calculating the downsample factor.&lt;/p&gt;&#xA;&lt;p&gt;Sambamba can use multiple threads by using &lt;code&gt;-t INT&lt;/code&gt;, where INT is the number of threads. It can also set the compression level for the BAM file using &lt;code&gt;-l&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BAM=&amp;quot;Sample.bam&amp;quot;&#xA;READS=&amp;quot;10000000&amp;quot;&#xA;fraction=&lt;span class=&quot;math-container&quot;&gt;$(samtools idxstats $&lt;/span&gt;BAM | cut -f3 | awk -v ct=&lt;span class=&quot;math-container&quot;&gt;$myReads 'BEGIN {total=0} {total += $&lt;/span&gt;1} END {print ct/total}')&#xA;&#xA;# With samtools&#xA;samtools view -b -s  &lt;span class=&quot;math-container&quot;&gt;${fraction} $&lt;/span&gt;{BAM} &amp;gt; output.bam&#xA;&#xA;# With sambamba&#xA;sambamba view -f bam -l 5 -s  &lt;span class=&quot;math-container&quot;&gt;${fraction} $&lt;/span&gt;{BAM} &amp;gt; output.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9870" LastActivityDate="2023-09-15T00:17:45.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21552" PostTypeId="1" CreationDate="2023-09-15T00:25:50.353" Score="2" ViewCount="170" Body="&lt;p&gt;I am a beginner in snakemake and I'm trying to develop a working flow.&lt;/p&gt;&#xA;&lt;p&gt;Imagine I have multiple folders and in each folder I have a tumour and normal sample files:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SRC120/ &#xA;      SRC120-T1.fastq&#xA;      SRC120-N1.fastq&#xA;&#xA;SRC121/ &#xA;      SRC121-T1.fastq&#xA;      SRC121-N1.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and so on (in total I have more than 100 SRC folders).&lt;/p&gt;&#xA;&lt;p&gt;I am trying to make a snakemake rule to make a symlink of these fastq files.&lt;/p&gt;&#xA;&lt;p&gt;What I have written so far is:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import os&#xA;&#xA;SAMPLES=os.listdir(&amp;quot;/analysis/Anna/snakemake-demo/data/samples_fastq-test1/&amp;quot;)                                               &#xA;&#xA;def getFastqFilesForTissu(wildcards):&#xA;    fastqs = list()&#xA;    # Beware no other files than fastqs should be there&#xA;    for s in os.listdir(&amp;quot;/analysis/Anna/snakemake-demo/data/samples_fastq-test1/&amp;quot;+wildcards.sample):&#xA;        fastqs.append(os.path.join(&amp;quot;/analysis/Anna/snakemake-demo/data/samples_fastq-test1/&amp;quot;,wildcards.sample,s))&#xA;    return fastqs&#xA;&#xA;&#xA;rule all:                                                                        &#xA;    input:                                                                       &#xA;        expand(&amp;quot;results/{sample}/{sample}.fastq&amp;quot;, sample=SAMPLES)                   &#xA;&#xA;&#xA;rule foo:&#xA;    input:&#xA;        fastq_files = getFastqFilesForTissu &#xA;    output:&#xA;        &amp;quot;results/{sample}/{sample}.fastq&amp;quot;&#xA;    shell:&#xA;        'ln -s {input} {output}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This works if I have one file in each folder and when the file name matches with the folder's name (i.e., when there is no T1 or N1 suffix).&lt;/p&gt;&#xA;&lt;p&gt;Is there any way I could modify my rule to get it working for all samples in SRC directories? Or a better way to handle this issue?&lt;/p&gt;&#xA;" OwnerUserId="1749" LastEditorUserId="650" LastEditDate="2023-09-15T21:41:53.163" LastActivityDate="2023-10-02T07:09:28.767" Title="How can I develop a snakemake rule for all files in multiple directories?" Tags="|python|snakemake|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21553" PostTypeId="2" ParentId="20828" CreationDate="2023-09-15T14:19:06.683" Score="1" Body="&lt;p&gt;I suspect the path should not be 'ubuntu/home/Desktop/DATES/bin'. It should be /home/username/DATES/bin, where username would be your ubuntu ID/name&lt;/p&gt;&#xA;" OwnerUserId="18181" LastActivityDate="2023-09-15T14:19:06.683" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21554" PostTypeId="1" AcceptedAnswerId="21579" CreationDate="2023-09-15T18:48:45.257" Score="1" ViewCount="89" Body="&lt;p&gt;I believe one of the site members is an expert in Cyanobacteria hence the question.&lt;/p&gt;&#xA;&lt;p&gt;I'm looking for a comprehensive 'phylogenetic tree' of the phylum Cyanobacteria and want to identify all the &lt;strong&gt;terrestrial&lt;/strong&gt; members/groups. For example, &lt;em&gt;Nodosilinea&lt;/em&gt; is one of the terrestrial members.&lt;/p&gt;&#xA;&lt;p&gt;Part of my rationale for the question is that the Orders can be polyphyletic. From an outsider's point of view this is a difficult taxonomic group to understand.&lt;/p&gt;&#xA;&lt;p&gt;Orders&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Chroococcales&#xA;Chroococcidiopsidales&#xA;Gloeobacterales&#xA;Nostocales&#xA;Oscillatoriales&#xA;Pleurocapsales&#xA;Spirulinales&#xA;Synechococcales&#xA;Incertae sedis&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background&lt;/strong&gt;&#xA;Cyanobacteria is a massive phylum and important to both terrestrial and aquatic habitats, but what I need to understand is how terrestrial members are distributed across the tree. The reason they are interesting (both aquatic and terrestrial) is that they are efficient at removing CO&lt;sub&gt;2&lt;/sub&gt; from the atmosphere. Reducing a carbon footprint is an area of growing commercial interest.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="298" LastEditDate="2023-09-19T11:44:16.960" LastActivityDate="2024-02-17T00:07:40.577" Title="Genetic relationships between Cyanobacteria: terrestrial vs aquatic" Tags="|phylogenetics|phylogeny|bacteria|taxonomy|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21555" PostTypeId="1" CreationDate="2023-09-15T22:09:27.500" Score="1" ViewCount="235" Body="&lt;p&gt;I was reading through the &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/37/15/2112/6128680&quot; rel=&quot;nofollow noreferrer&quot;&gt;DNABERT&lt;/a&gt; paper and found that their input features were k-mers. This is essentially using rolling window features.&lt;/p&gt;&#xA;&lt;p&gt;Why train a transformer-like model on sequences of the form  &lt;span class=&quot;math-container&quot;&gt;${a_n,..,a_{n+k}}$&lt;/span&gt; rather than training it on the complete sequence &lt;span class=&quot;math-container&quot;&gt;${a_1,...,a_{m}}$&lt;/span&gt; with positional encoding?&lt;/p&gt;&#xA;&lt;p&gt;To directly quote the paper:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Instead of regarding each base as a single token, we tokenized a DNA&#xA;sequence with the k-mer representation, an approach that has been&#xA;widely used in analyzing DNA sequences. The k-mer representation&#xA;incorporates richer contextual information for each deoxynucleotide&#xA;base by concatenating it with its following ones. The concatenation of&#xA;them is called a k-mer.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;There doesn't seem to be any ablation study in this paper showing that this really is of any benefit.&lt;/p&gt;&#xA;&lt;p&gt;I would appreciate any more information: I just don't see how the 3-mers  {ATG, TGG, GGC, GCT} contain more information than the sequence {A,T,G,G,C,T}, 1-mers, with positional encoding.&lt;/p&gt;&#xA;" OwnerUserId="14306" LastEditorUserId="14306" LastEditDate="2023-09-23T10:33:08.760" LastActivityDate="2023-09-23T10:33:08.760" Title="Why does DNABert use overlapping k-mers as input?" Tags="|k-mer|deep-learning|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21556" PostTypeId="2" ParentId="20025" CreationDate="2023-09-16T12:53:31.283" Score="1" Body="&lt;p&gt;Just to feedback, the GitHub ticket was:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/etetoolkit/ete/releases/tag/4.0.0-beta&quot; rel=&quot;nofollow noreferrer&quot;&gt;ETE4.0.0&lt;/a&gt; beta contains an old ETE manual (2009) and a README file for ETE4. Is there an updated ETE4 manual beyond &lt;a href=&quot;https://etetoolkit.org/docs/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;etetoolkit.org/docs/latest/index.html&lt;/a&gt; or the README of ETE4?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;lm-jkominek wrote,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I know it's a bit of necro, but I wanted to second this request  - while the current docs work fine enough for ETE3, they have diverged quite far from ETE4 and there's a bunch of errors that come through when using the latest version that have no explanation...&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Thus I'm not alone ... (nothing to do with UAPs BTW)&lt;/p&gt;&#xA;&lt;p&gt;jhcepas wrote,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;absolutely. We are finishing some final changes to the core of ETE4 and will start with the documentation. In the mean time, @jordibc has a list of most important changes regarding compatibility. Could you share somewhere as a wiki page @jordibc ?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;jordibc replied,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Yes, we are working on an updated version of the manual to be released with ete4. In the meantime, most of the old code can be run with some of the changes mentioned in the part of the wiki on migrating from ete3 to ete4. @lm-jkominek and @sarscov2 thanks and hope this helps for the moment!&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The GitHub wiki was updated &lt;a href=&quot;https://github.com/etetoolkit/ete/wiki/3to4&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The Wiki page is kept up to date,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&amp;quot;Jordi Burguet-Castell edited this page 4 days ago · 2 revisions&amp;quot;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The actual additional features (why ETE4 is needed) and how to access them are not listed here. However the changes in syntax are listed. Examples of changes ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# ETE3&#xA;t = Tree(filename) &#xA;# ETE4&#xA;t = Tree(open(filename))&#xA;# ETE3&#xA;Tree(name='Hi', dist=1, support=1) &#xA;# ETE4 &#xA;Tree({'name': 'Hi', 'dist': 1, 'support': 1})&#xA;# ETE3&#xA;Tree(nw, format=1) &#xA;# ETE4 &#xA;Tree(nw, parser=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In summary there's different syntax. So we can start using ETE4&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-09-16T12:53:31.283" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21557" PostTypeId="1" AcceptedAnswerId="21562" CreationDate="2023-09-16T18:49:21.880" Score="0" ViewCount="263" Body="&lt;p&gt;I have two data sets&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Mutations in 10 genes&lt;/li&gt;&#xA;&lt;li&gt;Gene expression in 18 cancer 3D models&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;em&gt;For every (mutation, gene) and gene expression pair we run an ANOVA test&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is my data&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&#xA;&amp;gt; mut=read.delim(&amp;quot;mut.tsv&amp;quot;,row.names = 1)&#xA;&amp;gt; expr=as.data.frame(t(read.delim(&amp;quot;expr.tsv&amp;quot;,row.names = 1)))&#xA;&#xA;dput(expr)&#xA;&#xA;structure(list(GeneA_KO = c(0.0756383119639074, -0.162841965682723, &#xA;-0.110703100204517, -0.457154616239399, -0.505815193885352, 0.366072775033448, &#xA;0.288650458935083, -0.0287646498640309, -0.690732824311905, 0.157366122620995, &#xA;-1.29451546737684, 0.124167425460344, -0.168045313861751, 0.0186504943306465, &#xA;-0.495880827400706, -0.206668627339351, -0.148609490335447, 0.066844529089804&#xA;), GeneB_KO = c(-0.870157285953621, -0.35900872537626, -0.990406888128838, &#xA;-0.120656981579292, -1.41427631791049, -0.866421095836781, -0.816936112146099, &#xA;-0.894597236248507, -1.00084584207823, -0.850754488133456, -1.05314239761653, &#xA;-1.07580675428372, -1.24708324665688, -1.04098460460899, -0.232388675985086, &#xA;-1.48513703581865, -1.48373348870346, -1.76484058374354), class = &amp;quot;data.frame&amp;quot;, row.names = c(&amp;quot;ModelM&amp;quot;, &#xA;&amp;quot;ModelO&amp;quot;, &amp;quot;ModelS&amp;quot;, &amp;quot;ModelA&amp;quot;, &amp;quot;ModelI&amp;quot;, &amp;quot;ModelC&amp;quot;, &amp;quot;ModelT&amp;quot;, &amp;quot;ModelH&amp;quot;, &#xA;&amp;quot;ModelE&amp;quot;, &amp;quot;ModelR&amp;quot;, &amp;quot;ModelA2&amp;quot;, &amp;quot;ModelP&amp;quot;, &amp;quot;ModelE2&amp;quot;, &amp;quot;ModelU&amp;quot;, &#xA;&amp;quot;ModelT2&amp;quot;, &amp;quot;ModelC2&amp;quot;, &amp;quot;ModelS2&amp;quot;, &amp;quot;ModelX&amp;quot;))&#xA;&#xA;dput(mut)&#xA;&#xA;structure(list(ModelM = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, &#xA;0L), ModelO = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelS = c(0L, &#xA;0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelA = c(0L, 0L, 0L, 1L, &#xA;0L, 0L, 0L, 0L, 0L, 0L), ModelI = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, &#xA;0L, 0L, 0L), ModelC = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L&#xA;), ModelT = c(0L, 0L, 0L, 1L, 0L, 1L, 0L, 1L, 1L, 0L), ModelH = c(1L, &#xA;1L, 1L, 1L, 0L, 1L, 1L, 1L, 0L, 0L), ModelE = c(1L, 0L, 0L, 0L, &#xA;1L, 0L, 0L, 0L, 0L, 1L), ModelR = c(1L, 0L, 0L, 1L, 0L, 0L, 1L, &#xA;0L, 1L, 0L), ModelA2 = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, &#xA;0L), ModelP = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L), ModelE2 = c(1L, &#xA;0L, 0L, 1L, 0L, 1L, 0L, 0L, 0L, 0L), ModelU = c(1L, 0L, 1L, 1L, &#xA;0L, 1L, 0L, 1L, 1L, 1L), ModelT2 = c(0L, 0L, 0L, 1L, 0L, 0L, &#xA;0L, 0L, 0L, 0L), ModelI2 = c(1L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, &#xA;0L, 0L), ModelC2 = c(1L, 1L, 0L, 0L, 0L, 1L, 0L, 1L, 0L, 0L), &#xA;    ModelS2 = c(0L, 1L, 0L, 1L, 1L, 1L, 1L, 0L, 0L, 1L)), class = &amp;quot;data.frame&amp;quot;, row.names = c(&amp;quot;Gene1_mut&amp;quot;, &#xA;&amp;quot;Gene2_mut&amp;quot;, &amp;quot;Gene3_mut&amp;quot;, &amp;quot;Gene4_mut&amp;quot;, &amp;quot;Gene5_mut&amp;quot;, &amp;quot;Gene6_mut&amp;quot;, &#xA;&amp;quot;Gene7_mut&amp;quot;, &amp;quot;Gene8_mut&amp;quot;, &amp;quot;Gene9_mut&amp;quot;, &amp;quot;Gene10_mut&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To test this, for the first 3D cancer model as instance, I have done like below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;           GeneA_KO   GeneB_KO&#xA;ModelM   0.07563831 -0.8701573&#xA;ModelO  -0.16284197 -0.3590087&#xA;ModelS  -0.11070310 -0.9904069&#xA;ModelA  -0.45715462 -0.1206570&#xA;ModelI  -0.50581519 -1.4142763&#xA;ModelC   0.36607278 -0.8664211&#xA;ModelT   0.28865046 -0.8169361&#xA;ModelH  -0.02876465 -0.8945972&#xA;ModelE  -0.69073282 -1.0008458&#xA;ModelR   0.15736612 -0.8507545&#xA;ModelA2 -1.29451547 -1.0531424&#xA;ModelP   0.12416743 -1.0758068&#xA;ModelE2 -0.16804531 -1.2470832&#xA;ModelU   0.01865049 -1.0409846&#xA;ModelT2 -0.49588083 -0.2323887&#xA;ModelC2 -0.20666863 -1.4851370&#xA;ModelS2 -0.14860949 -1.4837335&#xA;ModelX   0.06684453 -1.7648406&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;           ModelM ModelO ModelS ModelA ModelI ModelC ModelT ModelH ModelE&#xA;Gene1_mut       0      0      0      0      0      0      0      1      1&#xA;Gene2_mut       0      0      0      0      0      0      0      1      0&#xA;Gene3_mut       0      0      0      0      0      0      0      1      0&#xA;Gene4_mut       1      1      1      1      1      1      1      1      0&#xA;Gene5_mut       0      0      0      0      0      0      0      0      1&#xA;Gene6_mut       0      0      0      0      0      0      1      1      0&#xA;Gene7_mut       0      0      0      0      0      0      0      1      0&#xA;Gene8_mut       0      0      0      0      0      0      1      1      0&#xA;Gene9_mut       0      0      0      0      0      0      1      0      0&#xA;Gene10_mut      0      0      0      0      0      0      0      0      1&#xA;           ModelR ModelA2 ModelP ModelE2 ModelU ModelT2 ModelI2 ModelC2 ModelS2&#xA;Gene1_mut       1       0      0       1      1       0       1       1       0&#xA;Gene2_mut       0       0      0       0      0       0       0       1       1&#xA;Gene3_mut       0       0      0       0      1       0       0       0       0&#xA;Gene4_mut       1       1      1       1      1       1       1       0       1&#xA;Gene5_mut       0       0      0       0      0       0       0       0       1&#xA;Gene6_mut       0       0      0       1      1       0       0       1       1&#xA;Gene7_mut       1       0      0       0      0       0       0       0       1&#xA;Gene8_mut       0       0      0       0      1       0       0       1       0&#xA;Gene9_mut       1       0      0       0      1       0       0       0       0&#xA;Gene10_mut      0       0      1       0      1       0       0       0       1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; my_data=cbind(mut[,1],expr[,1])&#xA;&amp;gt; colnames(my_data)=c(&amp;quot;mut&amp;quot;,&amp;quot;expr&amp;quot;)&#xA;&amp;gt; res.aov &amp;lt;- aov(expr ~ mut, data = as.data.frame(my_data))&#xA;&amp;gt; # Summary of the analysis&#xA;&amp;gt; summary(res.aov)&#xA;            Df Sum Sq Mean Sq F value Pr(&amp;gt;F)&#xA;mut          1  0.145  0.1448   0.192  0.673&#xA;Residuals    8  6.022  0.7527&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now I do a for loop for all of the 18 cancer cell lines&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; for(i in 1:ncol(g))&#xA;+ {&#xA;+   my_data=as.data.frame(cbind(m[,1],g[,1]))&#xA;+   #tidy will summarise and return neat format&#xA;+   print(broom::tidy(aov(my_data[,2] ~ my_data[,1], data = my_data)))&#xA;+ }&#xA;# A tibble: 2 × 6&#xA;  term            df sumsq meansq statistic p.value&#xA;  &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;1 my_data[, 1]     1 0.145  0.145     0.192   0.673&#xA;2 Residuals        8 6.02   0.753    NA      NA    &#xA;# A tibble: 2 × 6&#xA;  term            df sumsq meansq statistic p.value&#xA;  &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;1 my_data[, 1]     1 0.145  0.145     0.192   0.673&#xA;2 Residuals        8 6.02   0.753    NA      NA   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I done this in R but of course I got when doing in Python&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Python&lt;/em&gt;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import os&#xA;import pandas as pd&#xA;from statsmodels.formula.api import ols&#xA;import statsmodels.api as sm&#xA;os.getcwd()&#xA;os.chdir(&amp;quot;/Users//Downloads/test&amp;quot;)&#xA;os.getcwd()&#xA;m = pd.read_csv(&amp;quot;mut.tsv&amp;quot;, sep=&amp;quot;\t&amp;quot;, index_col=0,delim_whitespace=True)&#xA;m=pd.DataFrame(m)&#xA;m&#xA;g = pd.read_csv(&amp;quot;expr.tsv&amp;quot;, sep=&amp;quot;\t&amp;quot;,index_col=0,delim_whitespace=True).T&#xA;g=pd.DataFrame(g)&#xA;g&#xA;lst = list(g.columns)&#xA;&#xA;lst&#xA;dfs = []&#xA;for Model in lst:&#xA;   dfs.append(pd.concat([m.iloc[:, 0].reset_index(drop=True), g.iloc[0,:].reset_index(drop=True)],axis=1))&#xA;   dfs.append(df.dropna())&#xA;   dfs.append(df.columns==['mut','expr'])&#xA;   model = ols('expr ~ C(mut)', data=df).fit()&#xA;   aov_table = sm.stats.anova_lm(model, typ=2)&#xA;   aov_table&#xA;   final = pd.concat(dfs, ignore_index=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;    for Model in lst:&#xA;      Input In [165]&#xA;        for Model in lst:&#xA;                         ^&#xA;    IndentationError: expected an indented block&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GYIIn.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GYIIn.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="15519" LastEditDate="2023-09-23T10:26:25.893" LastActivityDate="2023-09-24T15:27:20.803" Title="Looping over a list of cell models in Python" Tags="|python|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21559" PostTypeId="2" ParentId="21555" CreationDate="2023-09-16T21:27:58.337" Score="1" Body="&lt;p&gt;NLP and vectorisation is used to trap non-linearity which is highly applicable to sentences written in English.&#xA;Vectorisation of k-mers for an overlapping sliding window is weird because it's trying to assess the functionality of non-coding sequence, i.e. &amp;quot;silent&amp;quot; and non-cds mutations will be the dominant mutation class, but it would identify localised co-mutations, i.e. within a k-mer.&lt;/p&gt;&#xA;&lt;p&gt;The only application I can think is stuff like identifying promoters or mutations in promoters, i.e. non-coding regions with a known function and can be trained against appropriate independent experimental observations, e.g. gene expression. If the k-mers were large it might work for RNA secondary structure because it would trap compensatory mutations. Vectorisation for individual nucleotides, i.e. dumbie variables, would not be successful at trapping compensatory mutations.&lt;/p&gt;&#xA;&lt;p&gt;In summary it has niche application.&lt;/p&gt;&#xA;&lt;p&gt;BERT is a famous large language model BTW.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;If k-mer =3 that's about non-linearity of proteins, but it depends how the sliding window is deployed. It depends on the step size of the sliding window.&lt;/p&gt;&#xA;&lt;p&gt;To try and answer this better, proteins fold back on themself triggering &amp;quot;co-evolution&amp;quot; or more accurately &amp;quot;covariance&amp;quot; between non-linear sites. This defines the triplet codon:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;in machine learning this is needed because it will struggle to find it by itself&lt;/li&gt;&#xA;&lt;li&gt;it will help it identify co-evolving mutations at non-linear locations across the gene&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;It might also identify selection events better.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; If you've sat down and thought the equation the RNN is attempting to unravel its mind-boggling. More worrisome does it really do it? It's really complicated to simply discuss, but when you  give it key information the ML/DL gets there quicker. The authors will be aware of that in their training and testing.&lt;/p&gt;&#xA;&lt;p&gt;Honestly, we're at the limits of reasonable application of ML/DL without going into RasNet type infrastructure. My system is different the idea is the same - give the training model the key biological information, it gets to the answer quicker, a lot quicker.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-09-19T12:17:24.950" LastActivityDate="2023-09-19T12:17:24.950" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21560" PostTypeId="1" CreationDate="2023-09-18T16:04:18.773" Score="1" ViewCount="22" Body="&lt;p&gt;I have a 15k library of variants (OTUs) from a genome region. I want to compare the evenness of this library and understand of the library bias.&lt;/p&gt;&#xA;&lt;p&gt;I tried different methods to calculate evenness or uniformity of this library such as Shannon evenness index, relative abundance rank plot, kurtosis , skewness. I am confused with the results and why they are not aligned. For example my Shannon evenness index is 0.98 which means library is very even, while my kurtosis is very large (120), indication not uniform distribution.&lt;/p&gt;&#xA;&lt;p&gt;I should note that for calculating kurtosis I use relative abundance which is read count of every variant /total read count of all variants in that library. My question is what metrics I can use to have a better understanding of library evenness or uniformity, other that above metrics, and why my results are so contradictory?&lt;/p&gt;&#xA;" OwnerUserId="17826" LastActivityDate="2023-09-18T16:04:18.773" Title="Metrics to use to estimate variant library evenness, uniformity, or bias" Tags="|statistics|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21562" PostTypeId="2" ParentId="21557" CreationDate="2023-09-19T06:23:56.657" Score="4" Body="&lt;p&gt;Here is one potential approach:&lt;/p&gt;&#xA;&lt;p&gt;Example data:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;expr1 &amp;lt;- structure(list(GeneA_KO = c(0.0756383119639074, -0.162841965682723, -0.110703100204517, -0.457154616239399, -0.505815193885352, 0.366072775033448, 0.288650458935083, -0.0287646498640309, -0.690732824311905, 0.157366122620995, -1.29451546737684, 0.124167425460344, -0.168045313861751, 0.0186504943306465, -0.495880827400706, -0.206668627339351, -0.148609490335447, 0.066844529089804), GeneB_KO = c(-0.870157285953621, -0.35900872537626, -0.990406888128838, -0.120656981579292, -1.41427631791049, -0.866421095836781, -0.816936112146099, -0.894597236248507, -1.00084584207823, -0.850754488133456, -1.05314239761653, -1.07580675428372, -1.24708324665688, -1.04098460460899, -0.232388675985086, -1.48513703581865, -1.48373348870346, -1.76484058374354)), class = &amp;quot;data.frame&amp;quot;, row.names = c(&amp;quot;ModelM&amp;quot;, &amp;quot;ModelO&amp;quot;, &amp;quot;ModelS&amp;quot;, &amp;quot;ModelA&amp;quot;, &amp;quot;ModelI&amp;quot;, &amp;quot;ModelC&amp;quot;, &amp;quot;ModelT&amp;quot;, &amp;quot;ModelH&amp;quot;, &amp;quot;ModelE&amp;quot;, &amp;quot;ModelR&amp;quot;, &amp;quot;ModelA2&amp;quot;, &amp;quot;ModelP&amp;quot;, &amp;quot;ModelE2&amp;quot;, &amp;quot;ModelU&amp;quot;, &amp;quot;ModelT2&amp;quot;, &amp;quot;ModelC2&amp;quot;, &amp;quot;ModelS2&amp;quot;, &amp;quot;ModelX&amp;quot;))&#xA;mut &amp;lt;- structure(list(ModelM = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelO = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelS = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelA = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelI = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelC = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelT = c(0L, 0L, 0L, 1L, 0L, 1L, 0L, 1L, 1L, 0L), ModelH = c(1L, 1L, 1L, 1L, 0L, 1L, 1L, 1L, 0L, 0L), ModelE = c(1L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 1L), ModelR = c(1L, 0L, 0L, 1L, 0L, 0L, 1L, 0L, 1L, 0L), ModelA2 = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelP = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L), ModelE2 = c(1L, 0L, 0L, 1L, 0L, 1L, 0L, 0L, 0L, 0L), ModelU = c(1L, 0L, 1L, 1L, 0L, 1L, 0L, 1L, 1L, 1L), ModelT2 = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelI2 = c(1L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelC2 = c(1L, 1L, 0L, 0L, 0L, 1L, 0L, 1L, 0L, 0L),     ModelS2 = c(0L, 1L, 0L, 1L, 1L, 1L, 1L, 0L, 0L, 1L)), class = &amp;quot;data.frame&amp;quot;, row.names = c(&amp;quot;Gene1_mut&amp;quot;, &amp;quot;Gene2_mut&amp;quot;, &amp;quot;Gene3_mut&amp;quot;, &amp;quot;Gene4_mut&amp;quot;, &amp;quot;Gene5_mut&amp;quot;, &amp;quot;Gene6_mut&amp;quot;, &amp;quot;Gene7_mut&amp;quot;, &amp;quot;Gene8_mut&amp;quot;, &amp;quot;Gene9_mut&amp;quot;, &amp;quot;Gene10_mut&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;R code:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;my_data=cbind(mut[,1],expr1[1:10,1])&#xA;colnames(my_data)=c(&amp;quot;mut&amp;quot;,&amp;quot;expr1&amp;quot;)&#xA;res.aov &amp;lt;- aov(expr1 ~ mut, data = as.data.frame(my_data))&#xA;summary(res.aov)&#xA;#&amp;gt;             Df Sum Sq Mean Sq F value Pr(&amp;gt;F)&#xA;#&amp;gt; mut          1 0.1364  0.1364   1.114  0.322&#xA;#&amp;gt; Residuals    8 0.9789  0.1224&#xA;&#xA;for (j in 1:nrow(mut)) {&#xA;  for(i in 1:ncol(expr1)) {&#xA;    my_data=as.data.frame(cbind(mut[,j],expr1[1:10,i]))&#xA;    print(broom::tidy(aov(my_data[,2] ~ my_data[,1], data = my_data)))&#xA;  }&#xA;}&#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.136  0.136      1.11   0.322&#xA;#&amp;gt; 2 Residuals        8 0.979  0.122     NA     NA    &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.541 0.541       7.38  0.0264&#xA;#&amp;gt; 2 Residuals        8 0.587 0.0733     NA    NA     &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.136  0.136      1.11   0.322&#xA;#&amp;gt; 2 Residuals        8 0.979  0.122     NA     NA    &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.541 0.541       7.38  0.0264&#xA;#&amp;gt; 2 Residuals        8 0.587 0.0733     NA    NA     &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.136  0.136      1.11   0.322&#xA;#&amp;gt; 2 Residuals        8 0.979  0.122     NA     NA    &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.541 0.541       7.38  0.0264&#xA;#&amp;gt; 2 Residuals        8 0.587 0.0733     NA    NA     &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.136  0.136      1.11   0.322&#xA;#&amp;gt; 2 Residuals        8 0.979  0.122     NA     NA    &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.541 0.541       7.38  0.0264&#xA;#&amp;gt; 2 Residuals        8 0.587 0.0733     NA    NA     &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.136  0.136      1.11   0.322&#xA;#&amp;gt; 2 Residuals        8 0.979  0.122     NA     NA    &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.541 0.541       7.38  0.0264&#xA;#&amp;gt; 2 Residuals        8 0.587 0.0733     NA    NA     &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.136  0.136      1.11   0.322&#xA;#&amp;gt; 2 Residuals        8 0.979  0.122     NA     NA    &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.541 0.541       7.38  0.0264&#xA;#&amp;gt; 2 Residuals        8 0.587 0.0733     NA    NA     &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df  sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.0612 0.0612     0.465   0.515&#xA;#&amp;gt; 2 Residuals        8 1.05   0.132     NA      NA    &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df  sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.0637 0.0637     0.479   0.508&#xA;#&amp;gt; 2 Residuals        8 1.06   0.133     NA      NA    &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.246  0.246      2.26   0.171&#xA;#&amp;gt; 2 Residuals        8 0.869  0.109     NA     NA    &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.313  0.313      3.07   0.118&#xA;#&amp;gt; 2 Residuals        8 0.815  0.102     NA     NA    &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df   sumsq  meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.00108 0.00108   0.00777   0.932&#xA;#&amp;gt; 2 Residuals        8 1.11    0.139    NA        NA    &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.220  0.220      1.94   0.201&#xA;#&amp;gt; 2 Residuals        8 0.907  0.113     NA     NA    &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df  sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.0529 0.0529     0.398   0.546&#xA;#&amp;gt; 2 Residuals        8 1.06   0.133     NA      NA    &#xA;#&amp;gt; # A tibble: 2 × 6&#xA;#&amp;gt;   term            df  sumsq meansq statistic p.value&#xA;#&amp;gt;   &amp;lt;chr&amp;gt;        &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;     &amp;lt;dbl&amp;gt;   &amp;lt;dbl&amp;gt;&#xA;#&amp;gt; 1 my_data[, 1]     1 0.0901 0.0901     0.695   0.429&#xA;#&amp;gt; 2 Residuals        8 1.04   0.130     NA      NA&#xA;&#xA;# write.csv(mut, &amp;quot;mut.csv&amp;quot;, quote = FALSE, row.names = FALSE)&#xA;# write.csv(expr1[1:10,], &amp;quot;expr1.csv&amp;quot;, quote = FALSE, row.names = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;sup&gt;Created on 2023-09-19 with &lt;a href=&quot;https://reprex.tidyverse.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;reprex v2.0.2&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Python code:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import os&#xA;import pandas as pd&#xA;from statsmodels.formula.api import ols&#xA;import statsmodels.api as sm&#xA;m = pd.read_csv(&amp;quot;mut.csv&amp;quot;, sep=&amp;quot;,&amp;quot;)&#xA;m=pd.DataFrame(m)&#xA;m.info()&#xA;g = pd.read_csv(&amp;quot;expr1.csv&amp;quot;, sep=&amp;quot;,&amp;quot;)&#xA;g=pd.DataFrame(g)&#xA;&#xA;lst = list(g.columns)&#xA;m_lst = list(m.columns)&#xA;&#xA;dfs = pd.DataFrame({})&#xA;for Mut in m_lst:&#xA;    for Model in lst:&#xA;        df = pd.concat([m[[Mut]].reset_index(drop=True), g[[Model]].reset_index(drop=True)],axis=1)&#xA;        df = df.rename(columns={Mut: &amp;quot;mut&amp;quot;, Model: &amp;quot;g&amp;quot;})&#xA;        model = ols('g ~ C(mut)', data=df).fit()&#xA;        aov_table = sm.stats.anova_lm(model, typ=2)&#xA;        aov_table[['Model']] = Model&#xA;        aov_table[['Mut']] = Mut&#xA;        dfs = dfs.append(aov_table)&#xA;&#xA;with pd.option_context('display.max_rows', None, 'display.max_columns', None):&#xA;    print(dfs)&#xA;            sum_sq   df         F    PR(&amp;gt;F)     Model      Mut&#xA;C(mut)    0.136365  1.0  1.114460  0.321950  GeneA_KO   ModelM&#xA;Residual  0.978877  8.0       NaN       NaN  GeneA_KO   ModelM&#xA;C(mut)    0.540949  1.0  7.375925  0.026417  GeneB_KO   ModelM&#xA;Residual  0.586718  8.0       NaN       NaN  GeneB_KO   ModelM&#xA;C(mut)    0.136365  1.0  1.114460  0.321950  GeneA_KO   ModelO&#xA;Residual  0.978877  8.0       NaN       NaN  GeneA_KO   ModelO&#xA;C(mut)    0.540949  1.0  7.375925  0.026417  GeneB_KO   ModelO&#xA;Residual  0.586718  8.0       NaN       NaN  GeneB_KO   ModelO&#xA;C(mut)    0.136365  1.0  1.114460  0.321950  GeneA_KO   ModelS&#xA;Residual  0.978877  8.0       NaN       NaN  GeneA_KO   ModelS&#xA;C(mut)    0.540949  1.0  7.375925  0.026417  GeneB_KO   ModelS&#xA;Residual  0.586718  8.0       NaN       NaN  GeneB_KO   ModelS&#xA;C(mut)    0.136365  1.0  1.114460  0.321950  GeneA_KO   ModelA&#xA;Residual  0.978877  8.0       NaN       NaN  GeneA_KO   ModelA&#xA;C(mut)    0.540949  1.0  7.375925  0.026417  GeneB_KO   ModelA&#xA;Residual  0.586718  8.0       NaN       NaN  GeneB_KO   ModelA&#xA;C(mut)    0.136365  1.0  1.114460  0.321950  GeneA_KO   ModelI&#xA;Residual  0.978877  8.0       NaN       NaN  GeneA_KO   ModelI&#xA;C(mut)    0.540949  1.0  7.375925  0.026417  GeneB_KO   ModelI&#xA;Residual  0.586718  8.0       NaN       NaN  GeneB_KO   ModelI&#xA;C(mut)    0.136365  1.0  1.114460  0.321950  GeneA_KO   ModelC&#xA;Residual  0.978877  8.0       NaN       NaN  GeneA_KO   ModelC&#xA;C(mut)    0.540949  1.0  7.375925  0.026417  GeneB_KO   ModelC&#xA;Residual  0.586718  8.0       NaN       NaN  GeneB_KO   ModelC&#xA;C(mut)    0.061205  1.0  0.464539  0.514753  GeneA_KO   ModelT&#xA;Residual  1.054037  8.0       NaN       NaN  GeneA_KO   ModelT&#xA;C(mut)    0.063734  1.0  0.479234  0.508361  GeneB_KO   ModelT&#xA;Residual  1.063933  8.0       NaN       NaN  GeneB_KO   ModelT&#xA;C(mut)    0.245964  1.0  2.263619  0.170857  GeneA_KO   ModelH&#xA;Residual  0.869278  8.0       NaN       NaN  GeneA_KO   ModelH&#xA;C(mut)    0.312937  1.0  3.072788  0.117706  GeneB_KO   ModelH&#xA;Residual  0.814730  8.0       NaN       NaN  GeneB_KO   ModelH&#xA;C(mut)    0.001082  1.0  0.007771  0.931920  GeneA_KO   ModelE&#xA;Residual  1.114159  8.0       NaN       NaN  GeneA_KO   ModelE&#xA;C(mut)    0.220171  1.0  1.940909  0.201060  GeneB_KO   ModelE&#xA;Residual  0.907496  8.0       NaN       NaN  GeneB_KO   ModelE&#xA;C(mut)    0.052891  1.0  0.398296  0.545575  GeneA_KO   ModelR&#xA;Residual  1.062350  8.0       NaN       NaN  GeneA_KO   ModelR&#xA;C(mut)    0.090105  1.0  0.694741  0.428739  GeneB_KO   ModelR&#xA;Residual  1.037562  8.0       NaN       NaN  GeneB_KO   ModelR&#xA;C(mut)    0.136365  1.0  1.114460  0.321950  GeneA_KO  ModelA2&#xA;Residual  0.978877  8.0       NaN       NaN  GeneA_KO  ModelA2&#xA;C(mut)    0.540949  1.0  7.375925  0.026417  GeneB_KO  ModelA2&#xA;Residual  0.586718  8.0       NaN       NaN  GeneB_KO  ModelA2&#xA;C(mut)    0.004637  1.0  0.033399  0.859537  GeneA_KO   ModelP&#xA;Residual  1.110605  8.0       NaN       NaN  GeneA_KO   ModelP&#xA;C(mut)    0.276724  1.0  2.601573  0.145422  GeneB_KO   ModelP&#xA;Residual  0.850943  8.0       NaN       NaN  GeneB_KO   ModelP&#xA;C(mut)    0.044310  1.0  0.331000  0.580888  GeneA_KO  ModelE2&#xA;Residual  1.070932  8.0       NaN       NaN  GeneA_KO  ModelE2&#xA;C(mut)    0.170278  1.0  1.422852  0.267110  GeneB_KO  ModelE2&#xA;Residual  0.957389  8.0       NaN       NaN  GeneB_KO  ModelE2&#xA;C(mut)    0.001687  1.0  0.012120  0.915049  GeneA_KO   ModelU&#xA;Residual  1.113555  8.0       NaN       NaN  GeneA_KO   ModelU&#xA;C(mut)    0.008679  1.0  0.062048  0.809563  GeneB_KO   ModelU&#xA;Residual  1.118988  8.0       NaN       NaN  GeneB_KO   ModelU&#xA;C(mut)    0.136365  1.0  1.114460  0.321950  GeneA_KO  ModelT2&#xA;Residual  0.978877  8.0       NaN       NaN  GeneA_KO  ModelT2&#xA;C(mut)    0.540949  1.0  7.375925  0.026417  GeneB_KO  ModelT2&#xA;Residual  0.586718  8.0       NaN       NaN  GeneB_KO  ModelT2&#xA;C(mut)    0.017610  1.0  0.128353  0.729416  GeneA_KO  ModelI2&#xA;Residual  1.097631  8.0       NaN       NaN  GeneA_KO  ModelI2&#xA;C(mut)    0.260821  1.0  2.407079  0.159387  GeneB_KO  ModelI2&#xA;Residual  0.866846  8.0       NaN       NaN  GeneB_KO  ModelI2&#xA;C(mut)    0.191207  1.0  1.655405  0.234206  GeneA_KO  ModelC2&#xA;Residual  0.924035  8.0       NaN       NaN  GeneA_KO  ModelC2&#xA;C(mut)    0.033474  1.0  0.244741  0.634101  GeneB_KO  ModelC2&#xA;Residual  1.094193  8.0       NaN       NaN  GeneB_KO  ModelC2&#xA;C(mut)    0.044621  1.0  0.333425  0.579533  GeneA_KO  ModelS2&#xA;Residual  1.070620  8.0       NaN       NaN  GeneA_KO  ModelS2&#xA;C(mut)    0.096956  1.0  0.752533  0.410940  GeneB_KO  ModelS2&#xA;Residual  1.030711  8.0       NaN       NaN  GeneB_KO  ModelS2&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9098" LastEditorUserId="15519" LastEditDate="2023-09-24T15:27:20.803" LastActivityDate="2023-09-24T15:27:20.803" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="21563" PostTypeId="1" AcceptedAnswerId="21571" CreationDate="2023-09-19T18:09:17.380" Score="1" ViewCount="269" Body="&lt;p&gt;Within a Nextflow workflow, some process runs a shell script that creates multiple variables, e.g.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;var_a=$(echo &amp;quot;1&amp;quot;)&#xA;var_b=$(echo &amp;quot;2&amp;quot;)&#xA;var_c=$(echo &amp;quot;3&amp;quot;)&#xA;var_d=$(echo &amp;quot;4&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These variables contain statistics (e.g. number of input reads, quality scores, ...) that I want to keep until the end of the pipeline, to then save them and perform QC. So I'd like to export them out of this process and carry them to the end of the workflow. To be clear, the number of variables is not unknown, it's just too many for convenience if I need to list them all every time.&lt;/p&gt;&#xA;&lt;p&gt;I &lt;a href=&quot;https://stackoverflow.com/questions/70862345/how-to-declare-a-new-nextflow-variable-in-a-shell-block&quot;&gt;understand&lt;/a&gt; that I can use the &lt;code&gt;env&lt;/code&gt; qualifier or an intermediate file to export the variables individually:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process get_vars {&#xA;  output:&#xA;    tuple env(var_a), env(var_b), env(var_c), env(var_d)&#xA;  &#xA;  shell:&#xA;  '''&#xA;  var_a=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;1&quot;)&#xA;  var_b=$&lt;/span&gt;(echo &amp;quot;2&amp;quot;)&#xA;  var_c=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;3&quot;)&#xA;  var_d=$&lt;/span&gt;(echo &amp;quot;4&amp;quot;)&#xA;  '''&#xA;}&#xA;&#xA;workflow{&#xA;  get_vars | view&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;with output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[ea/a145b4] process &amp;gt; get_vars [100%] 1 of 1 ✔&#xA;[1, 2, 3, 4]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However that becomes very impractical if I have a lot of such variables (that I do want to continue passing along to later processes, along with a much longer tuple of other relevant variables).&lt;/p&gt;&#xA;&lt;p&gt;My question: is there a convenient way to &amp;quot;encapsulate&amp;quot; all these variables in the output channel, for example with a Groovy map similar to the &lt;code&gt;meta&lt;/code&gt; map in &lt;a href=&quot;https://nf-co.re/docs/contributing/modules#what-is-the-meta-map&quot; rel=&quot;nofollow noreferrer&quot;&gt;nf-core&lt;/a&gt;? So I could directly write something like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process get_vars {&#xA;  output:&#xA;    val(my_vars)&#xA;  &#xA;  script:&#xA;  '''&#xA;  var_a=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;1&quot;)&#xA;  var_b=$&lt;/span&gt;(echo &amp;quot;2&amp;quot;)&#xA;  var_c=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;3&quot;)&#xA;  var_d=$&lt;/span&gt;(echo &amp;quot;4&amp;quot;)&#xA;  '''&#xA;  &#xA;  my_vars = [:]&#xA;  my_vars.a = var_a&#xA;  my_vars.b = var_b&#xA;  my_vars.c = var_c&#xA;  my_vars.d = var_d&#xA;}&#xA;&#xA;workflow{&#xA;  get_vars | view&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which obviously fails because &lt;code&gt;var_a&lt;/code&gt; is not defined outside of the &lt;code&gt;'''&lt;/code&gt; block.&lt;/p&gt;&#xA;" OwnerUserId="9497" LastEditorUserId="73" LastEditDate="2023-09-20T01:34:39.167" LastActivityDate="2023-09-21T09:46:05.803" Title="Export multiple variables from shell script into a Nextflow output channel" Tags="|nextflow|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21564" PostTypeId="1" CreationDate="2023-09-19T19:40:04.413" Score="1" ViewCount="560" Body="&lt;p&gt;I am using the software &lt;a href=&quot;https://www.rbvi.ucsf.edu/chimerax/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chimera X&lt;/a&gt; to visualize the 3d composition of molecules. I am very new to Chimera and only use it for visualization. The files I'm using are cmm files (xml-formatted) basically looking like below:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-xml prettyprint-override&quot;&gt;&lt;code&gt;...&#xA;&amp;lt;marker id=&amp;quot;14&amp;quot; x=&amp;quot;0.018&amp;quot; y=&amp;quot;-0.180&amp;quot; z=&amp;quot;0.215&amp;quot; r=&amp;quot;1.000000&amp;quot; g=&amp;quot;0.254900&amp;quot; b=&amp;quot;0.050980&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.018754&amp;quot; /&amp;gt;&#xA;&amp;lt;marker id=&amp;quot;15&amp;quot; x=&amp;quot;-0.032&amp;quot; y=&amp;quot;-0.191&amp;quot; z=&amp;quot;0.241&amp;quot; r=&amp;quot;1.000000&amp;quot; g=&amp;quot;0.254900&amp;quot; b=&amp;quot;0.050980&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.018754&amp;quot; /&amp;gt;&#xA;&amp;lt;marker id=&amp;quot;16&amp;quot; x=&amp;quot;-0.033&amp;quot; y=&amp;quot;-0.117&amp;quot; z=&amp;quot;0.246&amp;quot; r=&amp;quot;1.000000&amp;quot; g=&amp;quot;0.254900&amp;quot; b=&amp;quot;0.050980&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.018754&amp;quot; /&amp;gt;&#xA;&amp;lt;marker id=&amp;quot;17&amp;quot; x=&amp;quot;0.020&amp;quot; y=&amp;quot;-0.138&amp;quot; z=&amp;quot;0.244&amp;quot; r=&amp;quot;1.000000&amp;quot; g=&amp;quot;0.254900&amp;quot; b=&amp;quot;0.050980&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.018754&amp;quot; /&amp;gt;&#xA;&amp;lt;marker id=&amp;quot;18&amp;quot; x=&amp;quot;0.047&amp;quot; y=&amp;quot;-0.083&amp;quot; z=&amp;quot;0.280&amp;quot; r=&amp;quot;1.000000&amp;quot; g=&amp;quot;0.254900&amp;quot; b=&amp;quot;0.050980&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.018754&amp;quot; /&amp;gt;&#xA;&amp;lt;marker id=&amp;quot;19&amp;quot; x=&amp;quot;0.073&amp;quot; y=&amp;quot;-0.061&amp;quot; z=&amp;quot;0.240&amp;quot; r=&amp;quot;1.000000&amp;quot; g=&amp;quot;0.254900&amp;quot; b=&amp;quot;0.050980&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.018754&amp;quot; /&amp;gt;&#xA;&amp;lt;marker id=&amp;quot;20&amp;quot; x=&amp;quot;0.010&amp;quot; y=&amp;quot;-0.068&amp;quot; z=&amp;quot;0.271&amp;quot; r=&amp;quot;1.000000&amp;quot; g=&amp;quot;0.254900&amp;quot; b=&amp;quot;0.050980&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.018754&amp;quot; /&amp;gt;&#xA;...&#xA;&amp;lt;link id1=&amp;quot;0&amp;quot; id2=&amp;quot;1&amp;quot; r=&amp;quot;0.800000&amp;quot; g=&amp;quot;0.800000&amp;quot; b=&amp;quot;0.000000&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.006251&amp;quot;/&amp;gt;&#xA;&amp;lt;link id1=&amp;quot;1&amp;quot; id2=&amp;quot;2&amp;quot; r=&amp;quot;0.800000&amp;quot; g=&amp;quot;0.800000&amp;quot; b=&amp;quot;0.000000&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.006251&amp;quot;/&amp;gt;&#xA;&amp;lt;link id1=&amp;quot;2&amp;quot; id2=&amp;quot;3&amp;quot; r=&amp;quot;0.800000&amp;quot; g=&amp;quot;0.800000&amp;quot; b=&amp;quot;0.000000&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.006251&amp;quot;/&amp;gt;&#xA;&amp;lt;link id1=&amp;quot;3&amp;quot; id2=&amp;quot;4&amp;quot; r=&amp;quot;0.800000&amp;quot; g=&amp;quot;0.800000&amp;quot; b=&amp;quot;0.000000&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.006251&amp;quot;/&amp;gt;&#xA;&amp;lt;link id1=&amp;quot;4&amp;quot; id2=&amp;quot;5&amp;quot; r=&amp;quot;0.800000&amp;quot; g=&amp;quot;0.800000&amp;quot; b=&amp;quot;0.000000&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.006251&amp;quot;/&amp;gt;&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;i.e. just a simple polymer consisting of alot of markers (spheres) and links between them (cylinders), and with colors for each one specified in rgb (see &lt;a href=&quot;https://www.cgl.ucsf.edu/chimerax/docs/user/markers.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;). This is all well and good, but now I would like to set transparency for a subset of the markers and associated links. Preferably I would set this directly in the attributes in the cmm files, but I cannot find how to do this. The manual does mention that colours in general can be set with rgba where a means alpha level, but when trying to set a like seen in the xml block above, nothing happens (I tried values close to both 0 and 1; no difference)&lt;/p&gt;&#xA;&lt;p&gt;The manual documents a function called 'transparency' which I tried to use with for example &lt;code&gt;transparency 50,a #0:10-1200&lt;/code&gt;&#xA;to make atoms 10-1200 50% transparent. But I just get the error: &amp;quot;Missing or invalid &amp;quot;percent&amp;quot; argument: Expected a number&amp;quot;. If I only type &lt;code&gt;transparency 50&lt;/code&gt; everything does turn transparent, however, I wouldn't really call it transparent since I cannot really see through them to see the other molecules on the other side, they just turn bleak.&lt;/p&gt;&#xA;&lt;p&gt;Questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;em&gt;(Would someone explain how I could simply change the actual transparency of a large set of markers and links)&lt;/em&gt; (preferably by loading some file with specified marker id:s)? Also, is it possible to assign this directly in the cmm files, and if so, how?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;UPDATE: The command &lt;code&gt;transparency #1:2492-end 90 'All'&lt;/code&gt; worked to set the transparency of markers e.g. 2492-end. What I would still like to know is if it is possible to set this directly in the cmm-files.&lt;/p&gt;&#xA;" OwnerUserId="18207" LastEditorUserId="18207" LastEditDate="2023-09-21T07:20:20.707" LastActivityDate="2024-02-18T09:06:10.767" Title="Changing transparency of markers in Chimera X" Tags="|visualization|3d-structure|software-usage|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21565" PostTypeId="1" AcceptedAnswerId="21566" CreationDate="2023-09-20T11:47:11.240" Score="1" ViewCount="76" Body="&lt;p&gt;I am trying to design primers for a single stranded DNA virus.&#xA;But I am having a dilemma as whether we design the primers same way as double stranded DNA?&lt;br /&gt;&#xA;do we have to design forward and reverse primers both or just a single primer forward or reverse works?&lt;/p&gt;&#xA;" OwnerUserId="16682" LastActivityDate="2023-09-20T13:14:16.207" Title="Do we design primers for single stranded DNA with same criteria as we do for double stranded DNA?" Tags="|primer|pcr|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21566" PostTypeId="2" ParentId="21565" CreationDate="2023-09-20T13:14:16.207" Score="3" Body="&lt;p&gt;If you've e.g. a parvovirus you design primers as normal, i.e. must be a pair of primers as for double stranded DNA. Albeit there's only one strand present after the first round of PCR there both strands will be present and without the reverse primer the PCR can't work.&#xA;If you want evidence of this, you've had a SARS-CoV-2 diagnosis? Well thats a forward and reverse primer PCR starting with ssRNA converted to cDNA, i.e. single copy and then a pair of primers PCR the resulting locus.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-09-20T13:14:16.207" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21567" PostTypeId="1" AcceptedAnswerId="21574" CreationDate="2023-09-20T20:46:36.900" Score="3" ViewCount="81" Body="&lt;p&gt;&lt;strong&gt;Question and Background&lt;/strong&gt; I'm using NCBI's Blast Perl API to interrogate Genbank for example &lt;code&gt;blastp&lt;/code&gt;. I don't get the throughput using their point and click web stuff.&lt;/p&gt;&#xA;&lt;p&gt;I tried Biopython's &lt;code&gt;Bio.Blast import NCBIWWW&lt;/code&gt; but its under access restrictions from the host, notably attempting to use it during the day (works sporadically). The API however gets through and runs as fast as a web search ... that's good.&lt;/p&gt;&#xA;&lt;p&gt;Here's an example of the command I'm running:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CMD=Put&amp;amp;PROGRAM=blastp&amp;amp;DATABASE=nr&amp;amp;HITLIST_SIZE=2500&amp;amp;ALIGNMENTS=2500&amp;amp;QUERY=%3Emytest%0AMSILANKDTR%0A&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above query gives default Blast output. What I'm trying to do is increase the number of hit recovered to the maximum (5000). The options for this are &lt;code&gt;HITLIST_SIZE&lt;/code&gt;, &lt;code&gt;ALIGNMENTS&lt;/code&gt;, &lt;code&gt;DESCRIPTIONS&lt;/code&gt;. In the above example I'm attempting a &lt;strong&gt;hit_size&lt;/strong&gt; of 2500 ... but the same output is returned as for default. More generally  I can't get any of the &amp;quot;number of hits/descriptions/alignments returned&amp;quot; options to work, the default is returned each time.&lt;/p&gt;&#xA;&lt;p&gt;Has anybody solved this?&lt;/p&gt;&#xA;&lt;p&gt;Typical options for the API are given below&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Typical options&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Parameter   Definition  Type    CMD Allowed values&#xA;QUERY       Search query    String  Put *   Accession, GI, or FASTA.&#xA;DATABASE    BLAST database  String  Put *   Database from appendix 2 or one uploaded to blastdb_custom (see appendix 4)&#xA;PROGRAM     BLAST program   String  Put *   One of blastn, blastp, blastx, tblastn, tblastx. To enable megablast, use PROGRAM=blastn&amp;amp;MEGABLAST=on.&#xA;FILTER      Low complexity filtering    String  Put F to disable. T or L to enable. Prepend “m” for mask at lookup (e.g., mL)&#xA;FORMAT_TYPE Report type String  Put, Get    HTML, Text, XML, XML2, JSON2, or Tabular. HTML is the default.&#xA;EXPECT      Expect value    Double  Put Number greater than zero.&#xA;HITLIST_SIZE    Number of databases sequences to keep   Integer Put,Get Integer greater than zero.&#xA;DESCRIPTIONS    Number of descriptions to print (applies to HTML and Text)  Integer Put,Get Integer greater than zero.&#xA;ALIGNMENTS      Number of alignments to print (applies to HTML and Text)    Integer Put,Get Integer greater than zero.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Just to feedback on the answer and comment&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Using &lt;code&gt;-remote&lt;/code&gt; by locally installing blast as recommended by @Scott. This is a valid answer so definitely upvoted and recommended its further upvoted. I had tried this but not mentioned it in the question (really sorry), but it times out due to traffic issues. The API overcomes traffic issues. Again, @Scott's is a valid solution (considering I'd not explained I'd tried it).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;More details ...&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;The Perl API is here &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/docs/web_blast.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blast.ncbi.nlm.nih.gov/docs/web_blast.pl&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Line 95 below ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$args = &quot;CMD=Put&amp;amp;PROGRAM=$&lt;/span&gt;program&amp;amp;DATABASE=&lt;span class=&quot;math-container&quot;&gt;$database&amp;amp;QUERY=&quot; . &#xA;$&lt;/span&gt;encoded_query;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is changed to,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$args = &amp;quot;CMD=Put&amp;amp;PROGRAM=$program&amp;amp;DATABASE=$database&amp;amp;HITLIST_SIZE=$maxhits&amp;amp;QUERY=&amp;quot; . $encoded_query;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Also need to add an additional line at line 73&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$maxhits = shift;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From the command line,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl ./web-blast.pl blastp nr 2500 ./example.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(Note, The variable order is switched in the program).&lt;/p&gt;&#xA;&lt;p&gt;example.fa - e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;rleguminosarum&#xA;WGFDGSSTQQAEGRSSDCVLKPVAIYPDPARTNGALVMCEVMMPDGVTPHASNARATILDDEDAWFGFEQ&#xA;EYFFYQNGRPLGFPEQGYPAPQGPYYTGVGYSNVGDVAREIVEEHLDLCLAAGINHEGINAEVAKGQWEF&#xA;QIFGKGSKKAADQIWMARYLLQRLTEKYGIDI&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm proficient in Perl5, i.e. there's no error in changes to the code in the API, albeit my expertise is object oriented and high performance Python.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Limit on the request&lt;/strong&gt; ... I'm testing that now.&lt;/p&gt;&#xA;&lt;p&gt;First results &lt;em&gt;eeek&lt;/em&gt; @terdon might be right ... :-(&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-09-22T14:00:20.073" LastActivityDate="2023-09-22T18:14:45.070" Title="Remote NCBI's Blast Perl API: maximising hits returned via command line" Tags="|blast|perl|api|parameters|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21568" PostTypeId="1" AcceptedAnswerId="21578" CreationDate="2023-09-21T01:38:14.567" Score="0" ViewCount="73" Body="&lt;p&gt;Is there a way to look up the amino acid sequence of a given protein? For example, what amino acids are used to produce Amylase? I spent hours googling but to no avail. If that's impossible, how can I find the sequence of A, C, G, U of the mRNA that instructs the ribosome to produce Amylase?&lt;/p&gt;&#xA;" OwnerUserId="18215" LastEditorUserId="18215" LastEditDate="2023-09-21T06:32:49.317" LastActivityDate="2023-09-22T08:41:14.230" Title="How to find amino acid sequence of a given protein" Tags="|proteins|sequence-analysis|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21569" PostTypeId="1" CreationDate="2023-09-21T07:03:21.330" Score="0" ViewCount="28" Body="&lt;p&gt;Just like, PSI-BLAST is avaialble to generate a PSSM matrix for peptides, is there a web server where we can generate PSSM for DNA/RNA? I want to find homologous sequences to a given RNA sequence.&lt;/p&gt;&#xA;" OwnerUserId="16888" LastEditorUserId="16888" LastEditDate="2023-09-22T05:50:55.853" LastActivityDate="2023-09-22T05:50:55.853" Title="Is there a method/web server to generate Position-specific scoring matrix (PSSM) for nucleotide sequences (DNA/RNA)?" Tags="|rna|pssm|dna|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21570" PostTypeId="2" ParentId="21564" CreationDate="2023-09-21T07:41:40.300" Score="0" Body="&lt;p&gt;Ok, this is not an answer, but to better visualize the problem I recreated the question/answer on &lt;code&gt;Chimera&lt;/code&gt; that has a different syntax for the &lt;code&gt;transparency&lt;/code&gt;  command see : &lt;a href=&quot;https://www.cgl.ucsf.edu/chimera/docs/UsersGuide/midas/transparency.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cgl.ucsf.edu/chimera/docs/UsersGuide/midas/transparency.html&lt;/a&gt; and &lt;a href=&quot;https://www.cgl.ucsf.edu/chimerax/docs/user/commands/transparency.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cgl.ucsf.edu/chimerax/docs/user/commands/transparency.html&lt;/a&gt; .&lt;/p&gt;&#xA;&lt;p&gt;Input file is &lt;code&gt;marker.cmm&lt;/code&gt; , to be opened in &lt;code&gt;FILE MENU&lt;/code&gt; of &lt;code&gt;VOLUME TRACER&lt;/code&gt; from &lt;code&gt;VOLUME MENU&lt;/code&gt; :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;marker_set name=&amp;quot;marker set 1&amp;quot;&amp;gt;&#xA;&amp;lt;marker id=&amp;quot;14&amp;quot; x=&amp;quot;0.018&amp;quot; y=&amp;quot;-0.180&amp;quot; z=&amp;quot;0.215&amp;quot; r=&amp;quot;1.000000&amp;quot; g=&amp;quot;0.254900&amp;quot; b=&amp;quot;0.050980&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.018754&amp;quot; /&amp;gt;&#xA;&amp;lt;marker id=&amp;quot;15&amp;quot; x=&amp;quot;-0.032&amp;quot; y=&amp;quot;-0.191&amp;quot; z=&amp;quot;0.241&amp;quot; r=&amp;quot;1.000000&amp;quot; g=&amp;quot;0.254900&amp;quot; b=&amp;quot;0.050980&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.018754&amp;quot; /&amp;gt;&#xA;&amp;lt;marker id=&amp;quot;16&amp;quot; x=&amp;quot;-0.033&amp;quot; y=&amp;quot;-0.117&amp;quot; z=&amp;quot;0.246&amp;quot; r=&amp;quot;1.000000&amp;quot; g=&amp;quot;0.254900&amp;quot; b=&amp;quot;0.050980&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.018754&amp;quot; /&amp;gt;&#xA;&amp;lt;marker id=&amp;quot;17&amp;quot; x=&amp;quot;0.020&amp;quot; y=&amp;quot;-0.138&amp;quot; z=&amp;quot;0.244&amp;quot; r=&amp;quot;1.000000&amp;quot; g=&amp;quot;0.254900&amp;quot; b=&amp;quot;0.050980&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.018754&amp;quot; /&amp;gt;&#xA;&amp;lt;marker id=&amp;quot;18&amp;quot; x=&amp;quot;0.047&amp;quot; y=&amp;quot;-0.083&amp;quot; z=&amp;quot;0.280&amp;quot; r=&amp;quot;1.000000&amp;quot; g=&amp;quot;0.254900&amp;quot; b=&amp;quot;0.050980&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.018754&amp;quot; /&amp;gt;&#xA;&amp;lt;marker id=&amp;quot;19&amp;quot; x=&amp;quot;0.073&amp;quot; y=&amp;quot;-0.061&amp;quot; z=&amp;quot;0.240&amp;quot; r=&amp;quot;1.000000&amp;quot; g=&amp;quot;0.254900&amp;quot; b=&amp;quot;0.050980&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.018754&amp;quot; /&amp;gt;&#xA;&amp;lt;marker id=&amp;quot;20&amp;quot; x=&amp;quot;0.010&amp;quot; y=&amp;quot;-0.068&amp;quot; z=&amp;quot;0.271&amp;quot; r=&amp;quot;1.000000&amp;quot; g=&amp;quot;0.254900&amp;quot; b=&amp;quot;0.050980&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.018754&amp;quot; /&amp;gt;&#xA;&amp;lt;link id1=&amp;quot;14&amp;quot; id2=&amp;quot;15&amp;quot; r=&amp;quot;0.800000&amp;quot; g=&amp;quot;0.800000&amp;quot; b=&amp;quot;0.000000&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.006251&amp;quot;/&amp;gt;&#xA;&amp;lt;link id1=&amp;quot;15&amp;quot; id2=&amp;quot;16&amp;quot; r=&amp;quot;0.800000&amp;quot; g=&amp;quot;0.800000&amp;quot; b=&amp;quot;0.000000&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.006251&amp;quot;/&amp;gt;&#xA;&amp;lt;link id1=&amp;quot;17&amp;quot; id2=&amp;quot;18&amp;quot; r=&amp;quot;0.800000&amp;quot; g=&amp;quot;0.800000&amp;quot; b=&amp;quot;0.000000&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.006251&amp;quot;/&amp;gt;&#xA;&amp;lt;link id1=&amp;quot;18&amp;quot; id2=&amp;quot;19&amp;quot; r=&amp;quot;0.800000&amp;quot; g=&amp;quot;0.800000&amp;quot; b=&amp;quot;0.000000&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.006251&amp;quot;/&amp;gt;&#xA;&amp;lt;link id1=&amp;quot;19&amp;quot; id2=&amp;quot;20&amp;quot; r=&amp;quot;0.800000&amp;quot; g=&amp;quot;0.800000&amp;quot; b=&amp;quot;0.000000&amp;quot; a=&amp;quot;0&amp;quot; radius=&amp;quot;0.006251&amp;quot;/&amp;gt;&#xA;&amp;lt;/marker_set&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Image before command :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Kb0kq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Kb0kq.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Image after &lt;code&gt;transparency  50,a,b #0:16-end&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;(#0 being 0 model as the only loaded model on my &lt;code&gt;Chimera&lt;/code&gt; instance) :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/v3Ny2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/v3Ny2.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Better transparency visualizations :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/yaYRC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/yaYRC.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/V7AXk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/V7AXk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Dont get why a='50' or a='0.5' in &lt;code&gt;marker&lt;/code&gt;,&lt;code&gt;link&lt;/code&gt; definition dont work with &lt;code&gt;chimera&lt;/code&gt; too&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2023-09-21T08:17:42.907" LastActivityDate="2023-09-21T08:17:42.907" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21571" PostTypeId="2" ParentId="21563" CreationDate="2023-09-21T09:46:05.803" Score="2" Body="&lt;p&gt;I am not sure if i get what you mean by encapsulate, you can give your tuple a name using emit?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;process get_vars {&#xA;  output:&#xA;    tuple env(var_a), env(var_b), env(var_c), env(var_d), emit: vars&#xA;&#xA;  script:&#xA;  '''&#xA;  var_a=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;1&quot;)&#xA;  var_b=$&lt;/span&gt;(echo &amp;quot;2&amp;quot;)&#xA;  var_c=&lt;span class=&quot;math-container&quot;&gt;$(echo &quot;3&quot;)&#xA;  var_d=$&lt;/span&gt;(echo &amp;quot;4&amp;quot;)&#xA;  '''&#xA;}&#xA;&#xA;workflow {&#xA;  get_vars()&#xA;  get_vars.out.vars.view()&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That gives:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; [1, 2, 3, 4]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17280" LastActivityDate="2023-09-21T09:46:05.803" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21572" PostTypeId="1" AcceptedAnswerId="21590" CreationDate="2023-09-21T15:37:24.113" Score="1" ViewCount="62" Body="&lt;p&gt;Sometimes I am trying to see where a (bacterial) genus ends and where another one begins based on 16S rRNA phylogenies. Of course, the length and support of the branch matter a lot, but this question in particular is about sequence similarity.&lt;/p&gt;&#xA;&lt;p&gt;As far as I understand it, in general, sequences within the same genus are expected to have roughly &amp;gt;94-95% sequence similarity at the 16S rRNA level. But that leaves out a lot of variation. For example, in doodle A below, there may be &amp;lt;90% similarity between sequence 'a' and sequence 'e' ; however, there is always &amp;gt;95% similarity between, say, a and b, b and c, c and d, d and e...So all would belong to the same genus.&#xA;&lt;a href=&quot;https://i.stack.imgur.com/JRPc0.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/JRPc0.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;Previously I had considered just comparing every sequence with every other sequence in my group of interest, get an average and a standard deviation, and see if the average &amp;gt; 94-95%. However, it occured to me that if I have a tree shaped like that in doodle B (a couple of closely-related sequences from one genus and one very distantly-related sequence from another genus), I could get the very same average (and only slight changes in the standard deviation).&lt;/p&gt;&#xA;&lt;p&gt;It feels like there should be a simple mathematical property to distinguish between A and B based on similarity values, and/or point out outliers like sequence 'a' in B, but I do not know what that would be. Help would be appreciated!&lt;/p&gt;&#xA;&lt;p&gt;Mock dataset A:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;sequence1&lt;/th&gt;&#xA;&lt;th&gt;sequence2&lt;/th&gt;&#xA;&lt;th&gt;sequence similarity (%)&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;a&lt;/td&gt;&#xA;&lt;td&gt;b&lt;/td&gt;&#xA;&lt;td&gt;99&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;a&lt;/td&gt;&#xA;&lt;td&gt;c&lt;/td&gt;&#xA;&lt;td&gt;93&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;a&lt;/td&gt;&#xA;&lt;td&gt;d&lt;/td&gt;&#xA;&lt;td&gt;92&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;a&lt;/td&gt;&#xA;&lt;td&gt;e&lt;/td&gt;&#xA;&lt;td&gt;95&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;b&lt;/td&gt;&#xA;&lt;td&gt;c&lt;/td&gt;&#xA;&lt;td&gt;99&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;b&lt;/td&gt;&#xA;&lt;td&gt;d&lt;/td&gt;&#xA;&lt;td&gt;98&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;b&lt;/td&gt;&#xA;&lt;td&gt;e&lt;/td&gt;&#xA;&lt;td&gt;97&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;c&lt;/td&gt;&#xA;&lt;td&gt;d&lt;/td&gt;&#xA;&lt;td&gt;95&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;c&lt;/td&gt;&#xA;&lt;td&gt;e&lt;/td&gt;&#xA;&lt;td&gt;94&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d&lt;/td&gt;&#xA;&lt;td&gt;e&lt;/td&gt;&#xA;&lt;td&gt;88&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;Mock dataset B:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;sequence1&lt;/th&gt;&#xA;&lt;th&gt;sequence2&lt;/th&gt;&#xA;&lt;th&gt;sequence similarity (%)&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;a&lt;/td&gt;&#xA;&lt;td&gt;b&lt;/td&gt;&#xA;&lt;td&gt;90&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;a&lt;/td&gt;&#xA;&lt;td&gt;c&lt;/td&gt;&#xA;&lt;td&gt;90&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;a&lt;/td&gt;&#xA;&lt;td&gt;d&lt;/td&gt;&#xA;&lt;td&gt;90&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;a&lt;/td&gt;&#xA;&lt;td&gt;e&lt;/td&gt;&#xA;&lt;td&gt;90&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;b&lt;/td&gt;&#xA;&lt;td&gt;c&lt;/td&gt;&#xA;&lt;td&gt;99&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;b&lt;/td&gt;&#xA;&lt;td&gt;d&lt;/td&gt;&#xA;&lt;td&gt;99&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;b&lt;/td&gt;&#xA;&lt;td&gt;e&lt;/td&gt;&#xA;&lt;td&gt;99&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;c&lt;/td&gt;&#xA;&lt;td&gt;d&lt;/td&gt;&#xA;&lt;td&gt;99&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;c&lt;/td&gt;&#xA;&lt;td&gt;e&lt;/td&gt;&#xA;&lt;td&gt;97&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;d&lt;/td&gt;&#xA;&lt;td&gt;e&lt;/td&gt;&#xA;&lt;td&gt;97&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;" OwnerUserId="4401" LastActivityDate="2023-09-25T13:10:48.273" Title="Using average of sequence similarities when delimiting genera - how to deal with outliers?" Tags="|phylogenetics|statistics|16rrna|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21573" PostTypeId="2" ParentId="21555" CreationDate="2023-09-21T19:45:32.263" Score="0" Body="&lt;p&gt;[I'm not an expert in this field, this is only my &amp;quot;best try&amp;quot;. Maybe you could get better mathematically motivated answers on ai.stackexchange]&lt;/p&gt;&#xA;&lt;p&gt;First note that, as pointed in the DNA BERT paper, k-mers are a very common approach to working with sequences. This is true both for older approaches as well as newer Transformer-type models. For example the recent &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2023.01.11.523679v3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Nucleotide Transformer&lt;/a&gt;, &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2023.07.11.548628v2&quot; rel=&quot;nofollow noreferrer&quot;&gt;DNAGPT&lt;/a&gt; or &lt;a href=&quot;https://academic.oup.com/gigascience/article/doi/10.1093/gigascience/giad054/7230465&quot; rel=&quot;nofollow noreferrer&quot;&gt;MuLan-Methyl&lt;/a&gt; also use k-mers.&lt;/p&gt;&#xA;&lt;p&gt;While I'm not aware of a published systematic comparison between k-mers and single bases, I'm sure I've seen papers that vary &lt;code&gt;k&lt;/code&gt; as a hyperparameter (so would notice if &lt;code&gt;k=1&lt;/code&gt; gives best results), and I have to believe someone somewhere at some point has thought about not using k-mers.&lt;/p&gt;&#xA;&lt;p&gt;There are some Deep Learning papers that use individual one hot-encoded nucleotides, for example &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S0092867418316295&quot; rel=&quot;nofollow noreferrer&quot;&gt;SpliceAI&lt;/a&gt;, but they typically only look at relatively short, fixed-length, sequences (&lt;em&gt;relatively&lt;/em&gt;: this paper does go up to 10k).&lt;/p&gt;&#xA;&lt;p&gt;As to the &amp;quot;why&amp;quot;, my (limited) understanding is that there are two factors. If you think of NLP, a token is typically a word from the language. Taking each nucleotide as a token means you consider a language with only 4 words, that is a very limited amount of information. It does make sense to use more meaningful tokens: to respond to your comment, even ViT uses patches (and does not attempt to work on individual pixels). Based on our knowledge of biology, we can expect that most sequences have meaningful &amp;quot;motifs&amp;quot; that are made of a few consecutive nucleotides. If you feed the nucleotides to the Transformer, it has to use attention to detect these motifs as well as their long-range interaction. If instead you use a &amp;quot;patch&amp;quot; (or k-mer) that captures entire motifs, this can be learned by an initial linear projection and you only use the attention for long-range interactions.&lt;/p&gt;&#xA;&lt;p&gt;Finally, the other reason is computing power, as hinted to in the DNAGPT paper:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;In the non-overlapped k-mers strategy, the shift is equal to K, resulting in N/k tokens for an N-length sequence and improving the efficiency by k times.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Maybe taking 1-mers could work just as well as 6-mers, but the number of relationships to consider increases even more, a non-overlapping k-mer strategy does decrease the of interactions to consider, and an overlapping k-mer strategy where each patch is preprocessed (e.g. with a linear model) can also increase efficiency (so to speak the model does not need to consider short-range interactions which are already accounted for).&lt;/p&gt;&#xA;" OwnerUserId="9497" LastActivityDate="2023-09-21T19:45:32.263" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="21574" PostTypeId="2" ParentId="21567" CreationDate="2023-09-22T06:07:52.520" Score="2" Body="&lt;p&gt;While I haven't used your exact method above, using the BLAST command line tools return the desired number of target sequences&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastp  -query query.fa \&#xA;        -task blastp-short \&#xA;        -db nr \&#xA;        -max_target_seqs 5000 \&#xA;        -outfmt 6 \&#xA;        -out query.out \&#xA;        -remote&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;query.fa&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;mytest&#xA;MSILANKDTR&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The resulting &lt;code&gt;query.out&lt;/code&gt; file does contain 5000 unique hits.&lt;/p&gt;&#xA;" OwnerUserId="92" LastActivityDate="2023-09-22T06:07:52.520" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21575" PostTypeId="1" CreationDate="2023-09-22T06:16:12.317" Score="3" ViewCount="203" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9575709/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am performing a &lt;em&gt;de novo&lt;/em&gt; genome assembly using Illumina paired-end short reads, sequenced on a NovaSeq X by our collaborator at UCLA.&lt;/p&gt;&#xA;&lt;p&gt;At present, I am in the stage of trimming the adapters.&#xA;Here, you can have a look at the basic statistics and information on the adapter content obtained from the Fast QC report, for R1.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;FOR RAW READS&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nb6Eg.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nb6Eg.jpg&quot; alt=&quot;Basic Statistics&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/HAolu.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/HAolu.jpg&quot; alt=&quot;Adapter content&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I used Trimmomatic for trimming the adapter.&#xA;The following is the &lt;strong&gt;Trimmomatic Settings&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;ILLUMINACLIP:~/adapters/TruSeq3-PE.fa:2:30:10 MINLEN:36&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here, you can see the basic statistics and adapter content of the Trimmed reads.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Eg58h.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Eg58h.jpg&quot; alt=&quot;Trimmed_Basic Statistics&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rBfKj.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rBfKj.jpg&quot; alt=&quot;Trimmed Reads Adapter Content&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here, the output was:&lt;/p&gt;&#xA;&lt;p&gt;Both surviving: 566832403&#xA;Forward only surviving: 39244376&#xA;Reverse only surving: 0.00&lt;/p&gt;&#xA;&lt;p&gt;Dropped reads: &amp;lt;1%&lt;/p&gt;&#xA;&lt;p&gt;Here you can go through the full FastQC report for:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://drive.google.com/file/d/1uNeBYN-DrqMawQFmNgH-ePr1VL7B6FVy/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Raw Reads&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://drive.google.com/file/d/1ETz1jNW1H23rg-OsraIBUlcmi2AhsIpJ/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;Trimmed Reads&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Now I have two questions:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 1&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can I go ahead with the assembly process, for there is zero-adapter-presence in the reads? Should I mind the loss of reads?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I see that there are over-represented sequences, both in read 1 and read 2. I doubt if I can leave them be, or I should trim them too. I would like to know if the over-represented sequences need to be tossed, or if they can be left untouched and proceed with the assembly. If I need to remove these sequences, can Trimmomatic do it?&lt;/p&gt;&#xA;&lt;p&gt;If I were to trim the over-represented sequences, can I use an adapter &amp;quot;fasta&amp;quot; file as follows?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The following are the over-represented sequences for R1.&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/4jLyr.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4jLyr.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The following are the over-represented sequences for R2.&lt;/strong&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/IOryC.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IOryC.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can I have my fasta file as follows, to remove the over-represented sequences:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;PrefixPE/1&#xA;-------sequence1--------------&#xA;-------sequence2--------------&#xA;-------sequence3--------------&#xA;-------sequence4--------------&#xA;&amp;gt;PrefixPE/2&#xA;-------sequence1--------------&#xA;&#xA;java -jar trimmomatic-0.30.jar PE s_1_1_sequence.txt.gz s_1_2_sequence.txt.gz &#xA;lane1_forward_paired.fq.gz lane1_forward_unpaired.fq.gz lane1_reverse_paired.fq.gz &#xA;lane1_reverse_unpaired.fq.gz ILLUMINACLIP:overRepresentedSeqs.fa:2:30:10  MINLEN:36&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;NB: Sorry about the lengthy post. Since the file sizes are large, it takes over a day for the Trimmomatic to finish the task, so that it is not feasible to go with multiple rounds of trial and error check.&lt;/p&gt;&#xA;" OwnerUserId="18059" LastEditorUserId="73" LastEditDate="2023-09-26T10:11:23.633" LastActivityDate="2023-09-27T12:02:59.433" Title="Trimmomatic QC report shows drop in the reads and presence of overrepresented sequences" Tags="|ngs|sequence-analysis|illumina|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21576" PostTypeId="1" CreationDate="2023-09-22T06:28:03.573" Score="1" ViewCount="105" Body="&lt;p&gt;I am trying to calculate log2FoldChange in DESeq2 by hand because I am trying to reproduce my results and understand how DESeq2 works.&lt;/p&gt;&#xA;&lt;p&gt;What I want to know is:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Are the counts first fitted to a negative binomial? If so, how can I get access to those values?&lt;/li&gt;&#xA;&lt;li&gt;Regarding log2FoldChanges, I read that the coefficents are the estimates for the log2 foldchanges for each sample group. However, these estimates do not account for the large dispersion we observe with low read counts. To avoid this, the log2 fold changes calculated by the model need to be adjusted.&#xA;Why? Didn't we just fit the counts to a negative binomial, which should take into account the dispersion.&lt;/li&gt;&#xA;&lt;li&gt;Finally, how are the log2FoldChanges calculated? It's not possible to figure this out using the raw code because most of the real calculations call C scripts.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="18227" LastActivityDate="2023-12-08T13:06:52.400" Title="how is log2foldchange calculated in DESeq2" Tags="|deseq2|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21577" PostTypeId="1" CreationDate="2023-09-22T07:56:34.553" Score="1" ViewCount="60" Body="&lt;p&gt;Im analyzing methylation data from the new illumina's EPIC V2 array (900K).&lt;/p&gt;&#xA;&lt;p&gt;I would like to perform enrichment analysis on DMPs. In the past, I have always used missMethyl but since there are a lot of differences between EPIC V1 and EPIC V2 arrays I'm unsure if missMethyl can be used (Im afraid results might be biased)&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know any enrichment tool already available for the EPIC V2 array or alternatives to missMethyl?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="17927" LastActivityDate="2023-09-22T07:56:34.553" Title="Enrichment analysis of DMPs | new Ilumina EPIC V2 900K" Tags="|methylation|go-enrichment|epigenetics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21578" PostTypeId="2" ParentId="21568" CreationDate="2023-09-22T08:41:14.230" Score="2" Body="&lt;p&gt;Usually you go and search in a database. NCBI and Uniprot (specialized on proteins) are ways to go here.&lt;/p&gt;&#xA;&lt;p&gt;These two offer tons of other related information, links to other databases, advanced functionalities like similarity search based on sequence (e.g. blastp). etc.&lt;/p&gt;&#xA;&lt;p&gt;To narrow your search, you would need to specify which species you are interested. Here a quick result for amylase in humans, using &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/protein/?term=Amylase+homo+sapiens&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI&lt;/a&gt; and &lt;a href=&quot;https://www.uniprot.org/uniprotkb/P0DTE8/entry&quot; rel=&quot;nofollow noreferrer&quot;&gt;Uniprot&lt;/a&gt;. The nucleotide sequence is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/M28443.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;linked&lt;/a&gt; in both cases.&lt;/p&gt;&#xA;" OwnerUserId="475" LastActivityDate="2023-09-22T08:41:14.230" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21579" PostTypeId="2" ParentId="21554" CreationDate="2023-09-22T09:58:50.777" Score="2" Body="&lt;p&gt;Ah! Finally, a question I can answer.&lt;/p&gt;&#xA;&lt;p&gt;There has been some pretty good work about this recently. In particular, I would reccomend this paper by Chen et al. , which groups cyanobacteria into terrestrial / freshwater / marine and...well, the title says the rest. Some are edge cases by necessity (e.g. some coastal biofilm-forming cyanobacteria may be more similar to terrestrial ones than to many marine ones) but that's just nature.&lt;/p&gt;&#xA;&lt;p&gt;Chen, MY., Teng, WK., Zhao, L. et al. Comparative genomics reveals insights into cyanobacterial evolution and habitat adaptation. ISME J 15, 211–227 (2021). &lt;a href=&quot;https://doi.org/10.1038/s41396-020-00775-z&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1038/s41396-020-00775-z&lt;/a&gt; (&lt;a href=&quot;https://rdcu.be/dmJpm&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rdcu.be/dmJpm&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;I regularly use this in my work, mainly because they have their trees posted on iTOL (in the 'Data availability' section) and this makes it really easy to search for specific strains.&lt;/p&gt;&#xA;&lt;p&gt;Which brings us the the main problem. Right now there are about three main ways of separating the cyanobacterial phylum into orders. There is the NCBI database classification (largely based on 16S rRNA), like for example the Chen et al. paper uses. This may sometimes appear polyphyletic when overimposed on a genome tree. Then there is the GTDB classification (&lt;a href=&quot;https://gtdb.ecogenomic.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gtdb.ecogenomic.org/&lt;/a&gt;), which is based on genomes. This can be seen in Alcorta et al. 2020 (Alcorta J, Alarcón-Schumacher T, Salgado O and Díez B (2020) Taxonomic Novelty and Distinctive Genomic Features of Hot Spring Cyanobacteria. Front. Genet. 11:568223. doi: 10.3389/fgene.2020.568223) - on average, the tree shape is similar to the one in Chen et al., but the orders make more sense. The GTDB classification is good phylogenetically, and well-integrated with the database (for example, you can just get the list of all genomes/metagenomes available for an order). But because it's not focused on the taxonomy (the naming of things), the actual names may not always fit with the Bacterial/Botanical Code. One research group has been kind enough to go through the absolute mess that is cyanobacteria taxonomy, and now this paper exists.&lt;/p&gt;&#xA;&lt;p&gt;Strunecký, O., Ivanova, A. P., &amp;amp; Mareš, J. (2023). An updated classification of cyanobacterial orders and families based on phylogenomic and polyphasic analysis. Journal of Phycology, 59(1), 12-51. (&lt;a href=&quot;https://onlinelibrary.wiley.com/share/A5UXIHBUCIH9JRTIWFSE?target=10.1111/jpy.13304&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://onlinelibrary.wiley.com/share/A5UXIHBUCIH9JRTIWFSE?target=10.1111/jpy.13304&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;As far as I've seen, their orders more or less match with GTDB, just under different names (except for GTDB Cyanobacteriales with is split into more orders in this paper). They also have a much larger 16S rrNA phylogeny constrained by the genome one.&lt;/p&gt;&#xA;&lt;p&gt;Hower, that solves only part of the problem. Because at the smaller scale (genus, species), you still have strains labeled by their morphologies, and culture collections are (for understandable reasons) hesitant to change their naming system. For particularly visually non-descript groups like 'Synechococcus' or 'Leptolyngbya', there is a good chance that a particular strain could be nearly anywhere in the phylogenetic tree. I would always get a 16S rRNA (if not already avaiable), BLAST it, and then check where related strains appear in a genome tree.&lt;/p&gt;&#xA;&lt;p&gt;Edit: I have also heard of Cydrasil (&lt;a href=&quot;https://www.cydrasil.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cydrasil.org/&lt;/a&gt;), an online tool which can place a 16S rrNA sequence within the context of the cyanobacterial tree, but I haven't tested it yet.&lt;/p&gt;&#xA;&lt;p&gt;Hope that helps!&lt;/p&gt;&#xA;" OwnerUserId="4401" LastEditorUserId="4401" LastEditDate="2023-09-22T10:34:56.880" LastActivityDate="2023-09-22T10:34:56.880" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21580" PostTypeId="1" AcceptedAnswerId="21594" CreationDate="2023-09-22T11:12:08.417" Score="0" ViewCount="222" Body="&lt;p&gt;I have a simple Python script with this header&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from flask import Flask&#xA;app = Flask(__name__)&#xA;&#xA;@app.route('/')&#xA;import os&#xA;import pandas as pd&#xA;from statsmodels.formula.api import ols&#xA;import statsmodels.api as sm&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have installed GitHub Desktop, Docker Desktop&lt;/p&gt;&#xA;&lt;p&gt;I have created a folder in that I have&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(.venv) (base) angel@Angels-MacBook-Pro docker % ls&#xA;Dockerfile      app.py          compose.yaml        requirements.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In &lt;code&gt;requirements.txt&lt;/code&gt; I have&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(.venv) (base) angel@Angels-MacBook-Pro docker % cat requirements.txt &#xA;blinker==1.6.2&#xA;click==8.1.6&#xA;colorama==0.4.6&#xA;Flask==2.3.2&#xA;itsdangerous==2.1.2&#xA;Jinja2==3.1.2&#xA;MarkupSafe==2.1.3&#xA;Werkzeug==2.3.6&#xA;gunicorn==21.2.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the commands I have run so far&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker init&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then this one returns error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(.venv) (base) angel@Angels-MacBook-Pro docker % docker compose up --build                         &#xA;[+] Building 3.9s (14/14) FINISHED                                                                                   docker:desktop-linux&#xA; =&amp;gt; [server internal] load .dockerignore                                                                                             0.1s&#xA; =&amp;gt; =&amp;gt; transferring context: 680B                                                                                                    0.1s&#xA; =&amp;gt; [server internal] load build definition from Dockerfile                                                                          0.1s&#xA; =&amp;gt; =&amp;gt; transferring dockerfile: 1.63kB                                                                                               0.1s&#xA; =&amp;gt; [server] resolve image config for docker.io/docker/dockerfile:1                                                                  2.3s&#xA; =&amp;gt; [server auth] docker/dockerfile:pull token for registry-1.docker.io                                                              0.0s&#xA; =&amp;gt; CACHED [server] docker-image://docker.io/docker/dockerfile:1@sha256:ac85f380a63b13dfcefa89046420e1781752bab202122f8f50032edf31b  0.0s&#xA; =&amp;gt; [server internal] load metadata for docker.io/library/python:3.9.12-slim                                                         1.3s&#xA; =&amp;gt; [server auth] library/python:pull token for registry-1.docker.io                                                                 0.0s&#xA; =&amp;gt; [server base 1/5] FROM docker.io/library/python:3.9.12-slim@sha256:364ada4cb4f943b603d330cbb3d26f689c8b54de51cb12bad1202de7dba3  0.0s&#xA; =&amp;gt; [server internal] load build context                                                                                             0.0s&#xA; =&amp;gt; =&amp;gt; transferring context: 1.39kB                                                                                                  0.0s&#xA; =&amp;gt; CACHED [server base 2/5] WORKDIR /app                                                                                            0.0s&#xA; =&amp;gt; CACHED [server base 3/5] RUN adduser     --disabled-password     --gecos &amp;quot;&amp;quot;     --home &amp;quot;/nonexistent&amp;quot;     --shell &amp;quot;/sbin/nologi  0.0s&#xA; =&amp;gt; CACHED [server base 4/5] RUN --mount=type=cache,target=/root/.cache/pip     --mount=type=bind,source=requirements.txt,target=re  0.0s&#xA; =&amp;gt; [server base 5/5] COPY . .                                                                                                       0.0s&#xA; =&amp;gt; [server] exporting to image                                                                                                      0.0s&#xA; =&amp;gt; =&amp;gt; exporting layers                                                                                                              0.0s&#xA; =&amp;gt; =&amp;gt; writing image sha256:c207ecf929eb3dc4f172db22e56a19e7b89892fcdc31da80501cb3524e137f9d                                         0.0s&#xA; =&amp;gt; =&amp;gt; naming to docker.io/library/docker-server                                                                                     0.0s&#xA;[+] Running 1/1&#xA; ✔ Container docker-server-1  Recreated                                                                                              0.1s &#xA;Attaching to docker-server-1&#xA;docker-server-1  | [2023-09-22 11:09:20 +0000] [8] [INFO] Starting gunicorn 21.2.0&#xA;docker-server-1  | [2023-09-22 11:09:20 +0000] [8] [INFO] Listening at: http://0.0.0.0:8000 (8)&#xA;docker-server-1  | [2023-09-22 11:09:20 +0000] [8] [INFO] Using worker: sync&#xA;docker-server-1  | [2023-09-22 11:09:20 +0000] [9] [INFO] Booting worker with pid: 9&#xA;docker-server-1  | [2023-09-22 11:09:20 +0000] [9] [ERROR] Exception in worker process&#xA;docker-server-1  | Traceback (most recent call last):&#xA;docker-server-1  |   File &amp;quot;/usr/local/lib/python3.9/site-packages/gunicorn/arbiter.py&amp;quot;, line 609, in spawn_worker&#xA;docker-server-1  |     worker.init_process()&#xA;docker-server-1  |   File &amp;quot;/usr/local/lib/python3.9/site-packages/gunicorn/workers/base.py&amp;quot;, line 134, in init_process&#xA;docker-server-1  |     self.load_wsgi()&#xA;docker-server-1  |   File &amp;quot;/usr/local/lib/python3.9/site-packages/gunicorn/workers/base.py&amp;quot;, line 146, in load_wsgi&#xA;docker-server-1  |     self.wsgi = self.app.wsgi()&#xA;docker-server-1  |   File &amp;quot;/usr/local/lib/python3.9/site-packages/gunicorn/app/base.py&amp;quot;, line 67, in wsgi&#xA;docker-server-1  |     self.callable = self.load()&#xA;docker-server-1  |   File &amp;quot;/usr/local/lib/python3.9/site-packages/gunicorn/app/wsgiapp.py&amp;quot;, line 58, in load&#xA;docker-server-1  |     return self.load_wsgiapp()&#xA;docker-server-1  |   File &amp;quot;/usr/local/lib/python3.9/site-packages/gunicorn/app/wsgiapp.py&amp;quot;, line 48, in load_wsgiapp&#xA;docker-server-1  |     return util.import_app(self.app_uri)&#xA;docker-server-1  |   File &amp;quot;/usr/local/lib/python3.9/site-packages/gunicorn/util.py&amp;quot;, line 371, in import_app&#xA;docker-server-1  |     mod = importlib.import_module(module)&#xA;docker-server-1  |   File &amp;quot;/usr/local/lib/python3.9/importlib/__init__.py&amp;quot;, line 122, in import_module&#xA;docker-server-1  |     raise TypeError(msg.format(name))&#xA;docker-server-1  | TypeError: the 'package' argument is required to perform a relative import for '.venv.lib.python3.9.site-packages.eventlet.wsgi'&#xA;docker-server-1  | [2023-09-22 11:09:20 +0000] [9] [INFO] Worker exiting (pid: 9)&#xA;docker-server-1  | [2023-09-22 11:09:20 +0000] [8] [ERROR] Worker (pid:9) exited with code 3&#xA;docker-server-1  | [2023-09-22 11:09:20 +0000] [8] [ERROR] Shutting down: Master&#xA;docker-server-1  | [2023-09-22 11:09:20 +0000] [8] [ERROR] Reason: Worker failed to boot.&#xA;docker-server-1 exited with code 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;yaml&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Comments are provided throughout this file to help you get started.&#xA;# If you need more help, visit the Docker compose reference guide at&#xA;# https://docs.docker.com/compose/compose-file/&#xA;&#xA;# Here the instructions define your application as a service called &amp;quot;server&amp;quot;.&#xA;# This service is built from the Dockerfile in the current directory.&#xA;# You can add other services your application may depend on here, such as a&#xA;# database or a cache. For examples, see the Awesome Compose repository:&#xA;# https://github.com/docker/awesome-compose&#xA;services:&#xA;  server:&#xA;    build:&#xA;      context: .&#xA;    ports:&#xA;      - 8000:8000&#xA;&#xA;# The commented out section below is an example of how to define a PostgreSQL&#xA;# database that your application can use. `depends_on` tells Docker Compose to&#xA;# start the database before your application. The `db-data` volume persists the&#xA;# database data between container restarts. The `db-password` secret is used&#xA;# to set the database password. You must create `db/password.txt` and add&#xA;# a password of your choosing to it before running `docker compose up`.&#xA;#     depends_on:&#xA;#       db:&#xA;#         condition: service_healthy&#xA;#   db:&#xA;#     image: postgres&#xA;#     restart: always&#xA;#     user: postgres&#xA;#     secrets:&#xA;#       - db-password&#xA;#     volumes:&#xA;#       - db-data:/var/lib/postgresql/data&#xA;#     environment:&#xA;#       - POSTGRES_DB=example&#xA;#       - POSTGRES_PASSWORD_FILE=/run/secrets/db-password&#xA;#     expose:&#xA;#       - 5432&#xA;#     healthcheck:&#xA;#       test: [ &amp;quot;CMD&amp;quot;, &amp;quot;pg_isready&amp;quot; ]&#xA;#       interval: 10s&#xA;#       timeout: 5s&#xA;#       retries: 5&#xA;# volumes:&#xA;#   db-data:&#xA;# secrets:&#xA;#   db-password:&#xA;#     file: db/password.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Dockerfile&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FROM python:3.9&#xA;WORKDIR /docker&#xA;COPY . /docker&#xA;RUN pip install -r requirements.txt&#xA;CMD [&amp;quot;python&amp;quot;, &amp;quot;./app.py&amp;quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="650" LastEditDate="2023-09-25T14:06:21.680" LastActivityDate="2023-09-26T07:17:36.523" Title="How to pack up a Python script in the form a Dockerfile" Tags="|python|docker|git|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21581" PostTypeId="2" ParentId="21567" CreationDate="2023-09-22T15:33:21.397" Score="0" Body="&lt;p&gt;Right here's the response from NCBI,&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Please send your future blast related questions to blast-help@ncbi.nlm.nih.gov directly. The gb-admin alias is for submitters with questions related to their sequence deposition/submission.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For your issue, you should look at the following two parameters:&#xA;DESCRIPTIONS  Number of descriptions to print (applies to HTML and Text)  Integer Put,Get Integer greater than zero.&#xA;ALIGNMENTS    Number of alignments to print (applies to HTML and Text)    Integer Put,Get Integer greater than zero.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Do please note that asking for this level of hits saved/returned could be problematic for the search. They could make the search exceeds the cpu time allocation and lead to error.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I tried this ... it was the same as &amp;quot;HITLIST_SIZE&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I figured the answer ...&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Why its happening?&lt;/strong&gt; The API uses a default parameter for the *&lt;em&gt;maximum number of hits&lt;/em&gt;. It's imposing a limit on the number of hits returned, but this wasn't stated in the documentation of the parameters.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Whats the limit?&lt;/strong&gt; It's &amp;gt;70 but &amp;lt;= 100. I suspect the limit is 100 hits. Thus anything above this its set at 100.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; The reason why the API is working and other approaches simply time out is NCBI is imposing an upper limit on the number of hits for the API and the Blast servers are not permitting requests above this threshold at peak time. Thus because other approaches are not imposing the restrictions on the NCBI server, the API gets through, whilst the other methods (blastp directly and Biopython) are timing out.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;NCBI are recommending the API is because it is automating a 100 hit cut-off and the servers are deprioritising limits beyond this&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have filed a note of advice to NCBI because it should be clear in the documentation there's an upper limit on the HITLIST_SIZE parameter. It took me at least a full half day to figure this out (which isn't cool for limited documentation).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Proof of concept (untested)&lt;/strong&gt; Thus, using @Scott's solution&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastp  -query query.fa \&#xA;        -task blastp-short \&#xA;        -db nr \&#xA;        -max_target_seqs 100 \&#xA;        -outfmt 6 \&#xA;        -out query.out \&#xA;        -remote&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will work with the &lt;code&gt;-max_target_seqs 100&lt;/code&gt; and thats all the API is doing.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-09-22T18:14:45.070" LastActivityDate="2023-09-22T18:14:45.070" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21582" PostTypeId="2" ParentId="21550" CreationDate="2023-09-22T17:18:58.923" Score="2" Body="&lt;p&gt;After some guidance from my advisor, this command allowed me to get the output file I needed.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools view -o newvcfFileName.g.vcf -s 123.2 locationOfTheFile chrom1 chrom2 ... chrom23(mito DNA) --force-samples&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I listed every chromosome's title until I typed --force-samples. I found it simpler to copy and paste the names in.&lt;/p&gt;&#xA;" OwnerUserId="18178" LastActivityDate="2023-09-22T17:18:58.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21583" PostTypeId="1" CreationDate="2023-09-22T20:37:16.770" Score="2" ViewCount="25" Body="&lt;p&gt;I have about ~3000 genomes of organisms and presence (recorded as 1)/absence (recorded as 0) data of ~300 genes in them. Among 3000 genomes, we have experimental data on 20 organisms, which were represented by 2 different phenotype groups (a, b). According to a previous study, the 300 genes I mentioned are believed to be associated with phenotypes 'a' and 'b'.&lt;/p&gt;&#xA;&lt;p&gt;To predict the phenotypes of 2980 genomes with no phenotype information, I did hierarchical clustering of 3000 genomes based on the presence/absence of the 300 genes. I cut the resulting dendrogram at a certain height that allowed distinction of clusters that included either group 'a' or 'b', but not both. This gave me 120 clusters, including clusters with no group 'a' or 'b' organisms, clusters with at least one group 'a' organisms, and clusters with at least one group 'b' organisms. Among these 120 clusters, there are 15 clusters that had either group 'a' or 'b' organisms.&lt;/p&gt;&#xA;&lt;p&gt;I am questioning which collections of genes are responsible for causing phenotypes 'a' or 'b'. To address this, I am thinking about assigning different names for the 15 clusters that contain group 'a' or 'b' organisms, and make a classification tree using these 15 clusters. Does this sound like an appropriate application of classification tree approach? If not, could there be a more sound way to address this question?&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="18113" LastActivityDate="2023-09-22T22:17:55.803" Title="Is a classification tree appropriate method to use for my project?" Tags="|clustering|machine-learning|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21584" PostTypeId="2" ParentId="21583" CreationDate="2023-09-22T22:17:55.803" Score="1" Body="&lt;p&gt;This isn't a cool strategy because dendrograms are very poor representations of the relationships between genomes, for example they use UPGMA style metrics fail to account for mutation rate variation between genomes and prone to long-branch attraction artefacts.&lt;/p&gt;&#xA;&lt;p&gt;I would recommend doing whole genome alignments using e.g. muscle5 -super option and then performing a formal phylogenetic tree using e.g. iqtree or even just MEGAX (distance methods) and using bootstrapping to identify the robustness of clades comprising a and b.&lt;/p&gt;&#xA;&lt;p&gt;If two clades are supported by bootstraps &amp;gt; 80% then your hypothesis looks fine. At present your downstream analysis could be an artefact of the dendrogram phylogeny.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-09-22T22:17:55.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21585" PostTypeId="1" CreationDate="2023-09-23T00:48:54.707" Score="3" ViewCount="114" Body="&lt;p&gt;If I want to efetch two records, I can do the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;efetch -db nuccore -id AB610939,AB610940 -format fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have hundreds of records to fetch. Can I put all the IDs in a file and efetch. If so, how?&lt;/p&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2023-09-23T19:10:24.720" Title="efetch multiple records at once" Tags="|edirect|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21587" PostTypeId="2" ParentId="21585" CreationDate="2023-09-23T04:06:23.677" Score="2" Body="&lt;p&gt;If you're happy looping through each id in turn and each id in the single file (&lt;code&gt;idfile.txt&lt;/code&gt;) is separate by a newline:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;while read -r x; do&#xA;  efetch -db nuccore -id  &amp;quot;$x&amp;quot; -format fasta&#xA;done &amp;lt; idfile.txt &amp;gt; fetchout.fa &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;Speculation&lt;/em&gt; If the ids are not separated by a newline but a comma on a single line that might give the simultaneous efetch you're wanting. I strongly suspect that each id would need to be surround in quotation marks in such a file. Again this is just speculation and the above solution is preferred IMO.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="298" LastEditDate="2023-09-23T19:10:24.720" LastActivityDate="2023-09-23T19:10:24.720" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21588" PostTypeId="2" ParentId="21585" CreationDate="2023-09-23T15:01:10.657" Score="3" Body="&lt;p&gt;Yes, &lt;code&gt;efetch&lt;/code&gt; can be given a file with IDs. See &lt;code&gt;efetch -h&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ efetch -h | grep -m1 file&#xA;  -input         Read identifier(s) from file instead of stdin&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So you can pass your file of IDs (e.g. &lt;code&gt;ids&lt;/code&gt;) directly:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat ids&#xA;AB610939&#xA;AB610940&#xA;&#xA;$ efetch -db nuccore -input ids -format fasta&#xA;&amp;gt;AB610939.1 Synthetic construct RNA, spike-in microarray control RNA CRM 6204-a, clone: 6204-a-500-1&#xA;CTCGACTAGTTAATACGGTACAGGATAACCGATCGGCTTGCAACATAACGGCGTTAAGAATGCGGGAGTG&#xA;CAGTTTCCGATTCTCACATCAATCGCCAATAAGGCCTTGTCGCAATATAGACTCAACGGTTCTAGTAGCT&#xA;GATCGGTATTACGTGACGCAACCGATTAGACATGCACAATTCCTTGGTCGCTATACTACGGAAATCGTCA&#xA;GGTACTATAACCCGTCGCAGGCCTAATACGTGTCGTCACATCGCCAACCTATCGTCAGTCGGAAAGACGT&#xA;TGCTGTCTACCATCGAAACTATTTACCGCTCCGAGATTCACGAGTACGAACTCACGAGGAAGTTGCCCTA&#xA;TGTAAGGTATCACTCCAGGTACTGCGCCGATAGTACCAGGTGATCAAACGGTTGCAAGAAGGCCACGACG&#xA;TATCGGGCTCTTTAGACGTACGCTCGAGATTAAACGCGCACTGATTCACTTTAGCCCGGAATGTCTCGGT&#xA;GCGATGTAGA&#xA;&amp;gt;AB610940.1 Synthetic construct RNA, spike-in microarray control RNA CRM 6204-a, clone: CRM6204-a-500-2&#xA;AGACTAAATCTCGGCGTCGGTTCATACGCGCGATCGTTTGCTGTCAGGGCATACTCGAATCCGGACTCCG&#xA;ACAATTATAGGCCATCCTGAATAGCCGATCATGCGAGTCACGATAAGGCAGGCTCTGCGATATCCCGATA&#xA;TACTGGAGAAGCTGAATCCCACCTAGAGCGAACTGTCAGAGGATCGACCTCAGGCTCGCTATCATCATAA&#xA;CGGCGGACGACCTGTGTCACATTCCGAACGCTACGTGACGATATTATCTGTCGAAAGGCATAGAACGCCG&#xA;GTCAATATCCTGCGGCATTCTCTTTATCACCGGCTATAACTACTAGGTTCCGCAGATATAGACTGCGCAC&#xA;GGAACATGTAGATAGATCGAGTAGGGTAGCGATTTAACGACTCGACTTACAGACAGAGACGTAGAACGTC&#xA;AGACGAGTGGTATGCCCACCAGAGGCGATACAGGCTGTACCTGCGTAGCACTAGAGTCGTGCGTCATGCG&#xA;GACCCTATCT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If that were not possible, another option would be to put all your IDs as a comma separated string. If your IDs are in a file named &lt;code&gt;ids&lt;/code&gt;, one per line, you can convert that to a comma-separated string like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ tr '\n' , &amp;lt; ids | sed 's/,$/\n/'&#xA;AB610939,AB610940&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That assumes GNU &lt;code&gt;sed&lt;/code&gt;. If your &lt;code&gt;sed&lt;/code&gt; doesn't handle &lt;code&gt;\n&lt;/code&gt;, you can use this instead:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;printf &amp;quot;$(tr '\n' , &amp;lt; ids | sed 's/,$//')&amp;quot; &amp;quot;$(cat ids)&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Armed with this, you can use &lt;a href=&quot;https://www.gnu.org/software/bash/manual/html_node/Command-Substitution.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;command substitution&lt;/a&gt; to pass the comma-separated list to &lt;code&gt;efetch&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;efetch -db nuccore -id $(tr '\n' , &amp;lt; ids | sed 's/,$/\n/') -format fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If there are limits to te number of IDs you can fetch at once, or as may well be the case, if this results in a command that is &lt;a href=&quot;https://unix.stackexchange.com/q/120642/22222&quot;&gt;too long&lt;/a&gt;, then you can use a loop as suggested in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/21587/298&quot;&gt;another answer&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;However, be aware that while looping seems to work for &lt;code&gt;efetch&lt;/code&gt;, for reasons that are not clear to me, the &lt;code&gt;esearch&lt;/code&gt; developers have chosen to make it consume stdin. This means that the obvious, straightforward approach fails with &lt;code&gt;esearch&lt;/code&gt;. See &lt;a href=&quot;https://unix.stackexchange.com/q/682748/22222&quot;&gt;https://unix.stackexchange.com/q/682748/22222&lt;/a&gt; for details.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-09-23T15:01:10.657" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21589" PostTypeId="1" CreationDate="2023-09-23T23:36:04.473" Score="4" ViewCount="47" Body="&lt;p&gt;Assume that I wish to find all complete human mitochondrial genome records on GenBank (or rather, NCBI nuccore) that also have an entry in NCBI's Biosample database.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MYQUERY=&amp;quot;(mitochondrion[TITLE] OR mitochondrial[TITLE]) \&#xA;    AND complete genome[TITLE] \&#xA;    AND (human[TITLE] or homo sapiens[TITLE])&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I know that 61,719 such records exist on NCBI nuccore as of Sep-23-2023.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db nuccore -query &amp;quot;$MYQUERY&amp;quot; | \&#xA;    xtract -pattern ENTREZ_DIRECT -element Count&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also know that 452 such records exist on NCBI Biosample as of Sep-23-2023.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db biosample -query &amp;quot;$MYQUERY&amp;quot; | \&#xA;    xtract -pattern ENTREZ_DIRECT -element Count&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, these record sets either do not have an intersection or are not correctly linked (or I am using the edirect tools incorrectly), as 0 hits are identified.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db nuccore -query &amp;quot;$MYQUERY&amp;quot; | \&#xA;    elink -target biosample | \&#xA;    xtract -pattern ENTREZ_DIRECT -element Count&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;On the other hand, first esearching in the biosample database and then linking the hits to the nuccore database does seem to give 271 hits.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db biosample -query &amp;quot;$MYQUERY&amp;quot; | \&#xA;    elink -target nuccore | \&#xA;    xtract -pattern ENTREZ_DIRECT -element Count&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What is the reason for this seemingly unpredictable behavior?&lt;/p&gt;&#xA;" OwnerUserId="3748" LastEditorUserId="298" LastEditDate="2023-09-24T09:59:27.853" LastActivityDate="2023-09-24T09:59:27.853" Title="Linking GenBank records to biosamples (and vice versa) using edirect" Tags="|genbank|ncbi|edirect|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21590" PostTypeId="2" ParentId="21572" CreationDate="2023-09-24T17:07:40.690" Score="1" Body="&lt;p&gt;The basic question is where do genera start and stop within phylogenetics and the 16S 5% rule? This is not a trivial question within molecular taxonomy, especially for bacteria.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The key is to define the tree structure for 16S&lt;/li&gt;&#xA;&lt;li&gt;Define the tree structure for other genes&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;strong&gt;Background and tree stats&lt;/strong&gt; The difference between Tree A and Tree B is referred to as 'balance'. Tree A is unbalanced, whilst Tree B is balanced and that is a formal description. I would personally use the term Tree A was &amp;quot;heavily paraphyletic&amp;quot;, or &amp;quot;extensive paraphyly&amp;quot; - at a technical level that is accepted, I have used frequently. Equally I'd describe Tree B as a &amp;quot;singular monophyly&amp;quot;. The statistical basis for tree balance is called the Colless value.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;&amp;quot;Colless statistic&amp;quot;&lt;/strong&gt;. It's fairly easy to calculate. The reason it is not often used is because the tree needs to be phylogenetically robust (below). Not much importance is given to it because if a branch isn't robust thats a headache for this statistic (it can't tell the difference between robust and non-robust).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Difference between branches&lt;/strong&gt; The statistical difference between taxa A and taxa B is measured by bootstrapping (I'm sure you know this), where &amp;gt;= 80% is bifurcation is robust.&lt;/p&gt;&#xA;&lt;p&gt;Colless statistic might be used, but its fairly obvious, bootstrapping is needed and &lt;strong&gt;iqtree&lt;/strong&gt; is best package for this operation.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The second more difficult part of the question is the delineation of a genus. The 5% threshold of 16S has always been seen as a guide. I have never seen it rigidly enforced and it creates a conflict with phylogenetic theory as your example points out. Telling what is and ain't a genus ... you are right that a key is the tree structure.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;Tree structure and bootstrapping&lt;/strong&gt; ...&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;A declared monophyly must be supported by a bootstrap &amp;gt; 80%&lt;/li&gt;&#xA;&lt;li&gt;A declared paraphyly ... each paraphyletic branch must be supported by a bootstrap &amp;gt; 80%&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;strong&gt;16S versus other genes and tree structure&lt;/strong&gt;&#xA;The tree structure needs confirming particularly as the bootstrap values 16S will often be low. The starting point is to consider the traditional MLST loci (I forget what they are but stuff like atpD, etc ...), build trees from them and :&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Don't combine 16S with protein genes (they evolve at different rates), although it could be done in a partitioned model&lt;/li&gt;&#xA;&lt;li&gt;MLST protein genes and core-house keeping genes can be combined without partitioning the data (there's a test, but it'll basically work).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Take the MLST genes repeat the tree and assess the bootstraps. Basically it will produce the same tree, but: a) confirms the tree structure; b) can provide stronger bootstrap values (which confirm the tree).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;5% stuff&lt;/strong&gt;&#xA;All the above needs doing before any 5% stuff is considered. It is my personal view that using 5% 16S as a &lt;strong&gt;mean value&lt;/strong&gt; is sort of bending the numbers, particularly in a paraphyly where the divergence is e.g. between 2 to 10%.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Monophyly vs paraphyly&lt;/strong&gt; This where the tree structure is needed. Basically, a robust monophyly would be regarded as a taxonomic singular unit. In this case if its close to the 5% threshold genus rule:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;even if the pairwise distance between taxa within the monophyly is say 8% via 16S;&lt;/li&gt;&#xA;&lt;li&gt;and this is supported by other protein loci;&lt;/li&gt;&#xA;&lt;li&gt;and there's no suspicion this could represent a family.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;In the above criteria I'd say its evidence of a genus. 10% is a bit big distance however.&lt;/p&gt;&#xA;&lt;p&gt;If the tree structure is paraphyletic and the maximum different is say 10% between two taxa - one at the top of the paraphyly and the other as a most recent child ... then thats tricky. It could easily be two genera within the paraphyletic tree structure. For example, the vertebrates as paraphyletic, viz. amphibian, reptile, bird, edentates, marsupial, mammals and that represents 6 monophyletic Classes. Thus if that was my result I would leave open the possible interpretations: maybe this is a single genus; alternatively it could be two (or more) genera. The key to this is defining the nodes (internal branches) via bootstraps to be confident of the tree structure.&lt;/p&gt;&#xA;&lt;p&gt;Generally a single robust paraphyletic tree structure, i.e. heavily imbalanced, where each paraphyletic branch is robustly supported would not be regarded as a single genus especially when 16S genetic differences is &amp;gt;&amp;gt; 5%. Taxonomically, this scenario would need further phylogenetic analysis (separate question) and classical bacterial taxonomists would want phenotypes before splitting a genus.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Reviewing the question against the answer - what I would do for the monophyly (diversity 10%) is split off the basal groups (at 10%) view robust bootstrap support. Then forward the in-group as the genus and the full monophyly as a putative genus. So that 10% is not the divergence unequivocally defining a single genus. So thats quite easy on the picture drawn. Thus the in-group is &amp;lt;8% because 10% is a bit high.&lt;/p&gt;&#xA;&lt;p&gt;It is harder to do this with a paraphyletic tree structure and thats a separate question on the advanced techniques needed to investigate this.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-09-25T13:10:48.273" LastActivityDate="2023-09-25T13:10:48.273" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21592" PostTypeId="1" CreationDate="2023-09-25T20:55:42.153" Score="2" ViewCount="22" Body="&lt;p&gt;I am running this command, using the &lt;a href=&quot;https://hgdownload.soe.ucsc.edu/downloads.html#utilities_downloads&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC utilities&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bigWigSummary -type=coverage ./galGal6.phastCons77way.bw chr4 25526708 25528708 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to get phastCons77way coverage from 25526708 to 25528708 on chromosome 4 of the chicken genome.&lt;/p&gt;&#xA;&lt;p&gt;The file I am using can be downloaded from &lt;a href=&quot;https://hgdownload.soe.ucsc.edu/goldenPath/galGal6/phastCons77way/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. I assumed &amp;quot;coverage&amp;quot; would be the ratio of loci with phastCons scores (so if a region of 2,000 bp has phastCons for every bp, the coverage should be 1). I think the phastCons was calculated vis multi-alignment, so it's not like each of the 77 species would have a score for that locus on the chicken genome, but rather they together has one score calculated for each bp when available.&lt;/p&gt;&#xA;&lt;p&gt;However, I got 1.1345 as output. I am confused, 'cause I thought coverage cannot be more than 1, aka 100%.&lt;/p&gt;&#xA;" OwnerUserId="5029" LastEditorUserId="298" LastEditDate="2023-09-26T20:37:35.207" LastActivityDate="2023-09-26T20:37:35.207" Title="How come bigWigSummary shows coverage greater than 1 for some region of the genome?" Tags="|phylogenetics|coverage|ucsc|bigwig|genome-browser|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21593" PostTypeId="2" ParentId="20723" CreationDate="2023-09-26T04:59:13.927" Score="0" Body="&lt;p&gt;Did you try query_pan_genome?&lt;/p&gt;&#xA;&lt;p&gt;Ref. &lt;a href=&quot;https://sanger-pathogens.github.io/Roary/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://sanger-pathogens.github.io/Roary/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18250" LastActivityDate="2023-09-26T04:59:13.927" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21594" PostTypeId="2" ParentId="21580" CreationDate="2023-09-26T07:17:36.523" Score="3" Body="&lt;p&gt;Not sure which tutorial you're following, but I think it's overly complicated. You do not need the compose file, and the Dockerfile has some issues:&lt;/p&gt;&#xA;&lt;p&gt;First, specify what you want to copy into the container. That is &lt;code&gt;requirements.txt&lt;/code&gt; and &lt;code&gt;app.py&lt;/code&gt;. Then also, tell pip where the requirements file is. This works in terms of putting a container together. I am not testing the python script itself, this is on you.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$ cat Dockerfile&#xA;FROM python:3.9&#xA;WORKDIR /docker&#xA;COPY [&amp;quot;app.py&amp;quot;, &amp;quot;requirements.txt&amp;quot; ,&amp;quot;/docker/&amp;quot;]&#xA;RUN pip install -r /docker/requirements.txt&#xA;CMD [&amp;quot;python&amp;quot;, &amp;quot;./app.py&amp;quot;]&#xA;&#xA;$ cat app.py&#xA;from flask import Flask&#xA;app = Flask(__name__)&#xA;&#xA;@app.route('/')&#xA;import os&#xA;import pandas as pd&#xA;from statsmodels.formula.api import ols&#xA;import statsmodels.api as sm&#xA;&#xA;$ cat requirements.txt&#xA;blinker==1.6.2&#xA;click==8.1.6&#xA;colorama==0.4.6&#xA;Flask==2.3.2&#xA;itsdangerous==2.1.2&#xA;Jinja2==3.1.2&#xA;MarkupSafe==2.1.3&#xA;Werkzeug==2.3.6&#xA;gunicorn==21.2.0&#xA;&#xA;$ docker build -t atpoint/foo:1.0 .&#xA;[+] Building 5.8s (9/9) FINISHED&#xA; =&amp;gt; [internal] load build definition from Dockerfile                                                                                                                                                                                                                                       0.0s&#xA; =&amp;gt; =&amp;gt; transferring dockerfile: 38B                                                                                                                                                                                                                                                        0.0s&#xA; =&amp;gt; [internal] load .dockerignore                                                                                                                                                                                                                                                          0.0s&#xA; =&amp;gt; =&amp;gt; transferring context: 2B                                                                                                                                                                                                                                                            0.0s&#xA; =&amp;gt; [internal] load metadata for docker.io/library/python:3.9                                                                                                                                                                                                                              5.7s&#xA; =&amp;gt; [1/4] FROM docker.io/library/python:3.9@sha256:031b2a713079b2436e6c2102dbffbce109c9c2582ad0867c380e3c8d7fed477f                                                                                                                                                                        0.0s&#xA; =&amp;gt; [internal] load build context                                                                                                                                                                                                                                                          0.0s&#xA; =&amp;gt; =&amp;gt; transferring context: 64B                                                                                                                                                                                                                                                           0.0s&#xA; =&amp;gt; CACHED [2/4] WORKDIR /docker                                                                                                                                                                                                                                                           0.0s&#xA; =&amp;gt; CACHED [3/4] COPY [app.py, requirements.txt ,/docker/]                                                                                                                                                                                                                                 0.0s&#xA; =&amp;gt; CACHED [4/4] RUN pip install -r /docker/requirements.txt                                                                                                                                                                                                                               0.0s&#xA; =&amp;gt; exporting to image                                                                                                                                                                                                                                                                     0.0s&#xA; =&amp;gt; =&amp;gt; exporting layers                                                                                                                                                                                                                                                                    0.0s&#xA; =&amp;gt; =&amp;gt; writing image sha256:6a6f1b08c8343608d2eacaa8463a1638491b504a3230ab00d45a7b32a41e1a43                                                                                                                                                                                               0.0s&#xA; =&amp;gt; =&amp;gt; naming to docker.io/atpoint/foo:1.0                                                                                                                                                                                                                                                 0.0s&#xA;&#xA;Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Your container is now available as &lt;code&gt;atpoint/foo:1.0&lt;/code&gt; which reads as username (atpoint), name of container (foo) and a version number / tag (1.0), all of which is free to choose. This is the most basic way of packing a container.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-09-26T07:17:36.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21595" PostTypeId="1" CreationDate="2023-09-26T08:12:38.807" Score="1" ViewCount="93" Body="&lt;p&gt;Hay, this question is sort of an extension of &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/21538/working-with-smart-seq3-data-have-some-questions-regarding-umi-and-bar-coding&quot;&gt;previous post&lt;/a&gt;. We have sequenced extra cellular RNA from culture media. Since the number of cells were limited, we did not have enough material for biological or technical replicates, so each sample has only been sequenced once, and there are essentially three groups and each group has 2 subgroups, main groups are control, group A and group B. I think the table below should help -&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;group&lt;/th&gt;&#xA;&lt;th&gt;subgroup&lt;/th&gt;&#xA;&lt;th&gt;Number of unique samples&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Control&lt;/td&gt;&#xA;&lt;td&gt;Day 3&lt;/td&gt;&#xA;&lt;td&gt;~30&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Control&lt;/td&gt;&#xA;&lt;td&gt;Day 5&lt;/td&gt;&#xA;&lt;td&gt;~5&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Treatment_1&lt;/td&gt;&#xA;&lt;td&gt;Day 3&lt;/td&gt;&#xA;&lt;td&gt;~30&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Treatment_1&lt;/td&gt;&#xA;&lt;td&gt;Day 5&lt;/td&gt;&#xA;&lt;td&gt;~50&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Treatment_2&lt;/td&gt;&#xA;&lt;td&gt;Day 3&lt;/td&gt;&#xA;&lt;td&gt;~40&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Treatment_2&lt;/td&gt;&#xA;&lt;td&gt;Day 5&lt;/td&gt;&#xA;&lt;td&gt;~20&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;We want to compare the gene expression (Or rather abundance of RNA) between samples based on the Day subgroup.&lt;/p&gt;&#xA;&lt;p&gt;Since the same sample / culture batch has not been sequenced with replicates, and there is variance in number of samples per group, I was wondering if DESeq2 would be a valid option, as the documentation does mention that without replicates, the output from DESeq2 should be considered only for exploratory purpose.&lt;/p&gt;&#xA;&lt;p&gt;If DESeq2 is not appropriate, which other tool would be better?&lt;/p&gt;&#xA;&lt;p&gt;EDIT - I came across &lt;a href=&quot;https://www.bioconductor.org/packages/release/bioc/html/NOISeq.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;NOISeq&lt;/a&gt; which I will try once I get any output from DESEq2, if anyone has any other tools / papers which can potentially help me out, do let me know!&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="6561" LastEditorUserId="6561" LastEditDate="2023-09-28T12:37:41.467" LastActivityDate="2023-09-28T17:12:42.990" Title="Would DESeq2 be appropriate or should I use another tool for differential gene expression?" Tags="|differential-expression|rna|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21597" PostTypeId="2" ParentId="21575" CreationDate="2023-09-26T10:22:18.730" Score="1" Body="&lt;p&gt;Given that it's a NovaSeq, these are almost certainly &amp;quot;no signal&amp;quot; reads as a result of the &lt;a href=&quot;https://sequencing.qcfail.com/articles/illumina-2-colour-chemistry-can-overcall-high-confidence-g-bases/&quot; rel=&quot;nofollow noreferrer&quot;&gt;two-colour chemistry&lt;/a&gt; and should probably be excluded (or at least ignored).&lt;/p&gt;&#xA;&lt;p&gt;A long polyG sequence is unlikely to affect mapping by much, because they're fairly rare in most genomes. Unfortunately you can't exclude them using Trimmomatic alone.&lt;/p&gt;&#xA;&lt;p&gt;PolyG tail trimming can be done using &lt;a href=&quot;https://github.com/OpenGene/fastp#polyg-tail-trimming&quot; rel=&quot;nofollow noreferrer&quot;&gt;fastp&lt;/a&gt;. I don't have any experience using fastp, so you'll need to read the documentation yourself.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-09-27T12:02:59.433" LastActivityDate="2023-09-27T12:02:59.433" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21599" PostTypeId="1" CreationDate="2023-09-26T14:14:46.003" Score="0" ViewCount="66" Body="&lt;p&gt;Could you help me bioinformatics SE people.&lt;/p&gt;&#xA;&lt;p&gt;So I have been duckduckgo-ing up a storm and even consulted the AI overlords and haven't come up with a solid way to do this, but it is so fundamental to bioinformatics that it has to already exists and I don't need to be re-inventing the wheel...&lt;/p&gt;&#xA;&lt;p&gt;So imagine you have a fasta/q file with a &amp;quot;reference&amp;quot; sequence:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;AATTTTAAAAAAACCCCAAAAAAAAAAAGGGGGAAACCACAAAA&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;And a list of short oligo sequences:&#xA;&lt;code&gt;TTTT, CCCC, GGGG&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;How would be the best way to map (if that is the right terminology) the short oligos to the reference seq so that a configurable mismatch amount is allowed:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  TTTT       CCCC           GGGG    CCCC    &#xA;AATTTTAAAAAAACCCCAAAAAAAAAAAGGGGAAAACCACAAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Blast, BWA and Bowtie2 from my understanding aren't ideal for this because they aren't built for really short sequences, something about them using seeded reference lookup?&#xA;Using Smith-Watterman or Needleman-Wunsch technically gets the local aligning right but the mismach allowance is based on score not X total allowed. Plus, iterating over hundreds of oligos and aligning against dozens of large references seems inefficient.&#xA;The closest thing I have found to what I'm trying to do is &lt;a href=&quot;https://www.bioinformatics.org/sms2/primer_map.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sequence Manipulation Suites' Primer Map&lt;/a&gt;. This tool is almost perfect except a) it is a web tool, not a module I could import into python or incorporate into a CLI and b) does not appear to offer any tolerance for mismatches.&lt;/p&gt;&#xA;&lt;p&gt;Am I just searching with the wrong vocabulary? This has to be an already solved bioinformatics process right?&lt;/p&gt;&#xA;" OwnerUserId="10807" LastEditorUserId="3848" LastEditDate="2023-09-26T19:26:04.720" LastActivityDate="2024-02-23T20:08:41.000" Title="Mapping List of Probes/Primers/Short Oligos to a Reference Fasta/q" Tags="|sequence-alignment|read-mapping|multiple-sequence-alignment|primer|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21600" PostTypeId="2" ParentId="21599" CreationDate="2023-09-26T18:57:33.763" Score="0" Body="&lt;p&gt;Both of @haci's solutions are very cool:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/paulstothard/sequence_manipulation_suite/blob/master/docs/scripts/primer_map.js&quot; rel=&quot;nofollow noreferrer&quot;&gt;Primer Map&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://github.com/primer3-org/primer3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Primer3 CLI&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I wasn't aware that Primer3 was available as a CLI (command line interface).&lt;/p&gt;&#xA;&lt;p&gt;The solution I had in mind was &lt;em&gt;in silico&lt;/em&gt; PCR one example being &lt;a href=&quot;https://github.com/PaulGreenfieldOz/WorkingDogs/tree/master/Kelpie&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kelpie&lt;/a&gt; and the associated paper is &lt;a href=&quot;https://peerj.com/articles/6174/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, however I am not exactly sure about the application of the &amp;quot;primer mapping&amp;quot;, but most &amp;quot;primer&amp;quot; applications concern PCR.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-09-26T19:24:34.613" LastActivityDate="2023-09-26T19:24:34.613" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21601" PostTypeId="1" CreationDate="2023-09-26T21:29:43.983" Score="3" ViewCount="140" Body="&lt;p&gt;I have two GSM ID as test case where both of them are having data when I checked through sra explorer tool but when I try to fetch through a script only one of them returns output where the other one fails.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;GSM ID&lt;/strong&gt;&#xA;GSM3603268&#xA;GSM2683458&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Output&lt;/strong&gt;&#xA;For GSM3603268&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat GSM3603268.test.fastq&#xA;@GSM3603268.1 1 length=100&#xA;TTTTTTAAAGAAAACTTGAGCTTTTGGAGCATGGCAACCTAGCCTGCAGACACCGTATCCCCTTGTCCACTTCCCCCTGCAAACCATAAGTCCATTCCTA&#xA;+GSM3603268.1 1 length=100&#xA;=@@DDDDA++2CDABCEG@GEHHEHGA9EDFHGHD3::B??&amp;lt;DBFFBFB===)5;CA;(7=?;?CE;7)7;&amp;gt;C((583;&amp;lt;9&amp;gt;(,89@@@@CCCCDD:BC:&#xA;@GSM3603268.1 1 length=100&#xA;AGTTCCACGAGTTTTCTTTTTTTTAAGTGGTAGGAATGGACTTATGGGTTGCAGGGGGAAGTGGACAAGGGGCTACGGTGTCTGCAGGCTTGGTTGCCAT&#xA;+GSM3603268.1 1 length=100&#xA;:?=BD:D?C)&amp;lt;CDGH4+&amp;lt;CCCGHI)??D&amp;lt;D?00?D*?39C&amp;lt;CG4BFE28;4(.7AB############################################&#xA;&#xA;$ cat GSM3603268.test.log&#xA;2023-09-26T21:18:46 fastq-dump.3.0.1 int: string unexpected while executing query within virtual file system module - multiple response SRR URLs for the same service 's3'&#xA;2023-09-26T21:18:46 fastq-dump.3.0.1 int: string unexpected while executing query within virtual file system module - multiple response SRR URLs for the same service 's3'&#xA;2023-09-26T21:18:46 fastq-dump.3.0.1 int: string unexpected while executing query within virtual file system module - multiple response SRR URLs for the same service 's3'&#xA;Read 1 spots for GSM3603268&#xA;Written 1 spots for GSM3603268&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output for GSM2683458&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat GSM2683458.test.fastq&#xA;    bash-5.1#&#xA;&#xA;$ cat GSM2683458.test.log&#xA;2023-09-26T21:16:55 fastq-dump.3.0.1 int: string unexpected while executing query within virtual file system module - multiple response SRR URLs for the same service 's3'&#xA;2023-09-26T21:16:55 fastq-dump.3.0.1 int: string unexpected while executing query within virtual file system module - multiple response SRR URLs for the same service 's3'&#xA;2023-09-26T21:16:55 fastq-dump.3.0.1 int: string unexpected while executing query within virtual file system module - multiple response SRR URLs for the same service 's3'&#xA;2023-09-26T21:17:11 fastq-dump.3.0.1 warn: directory not found while opening manager within virtual file system module - can't open NC_000076.5 as a RefSeq or as a WGS&#xA;2023-09-26T21:17:11 fastq-dump.3.0.1 err: directory not found while opening manager within virtual file system module - failed GSM2683458&#xA;&#xA;=============================================================&#xA;An error occurred during processing.&#xA;A report was generated into the file '/root/ncbi_error_report.txt'.&#xA;If the problem persists, you may consider sending the file&#xA;to 'sra-tools@ncbi.nlm.nih.gov' for assistance.&#xA;=============================================================&#xA;&#xA;fastq-dump quit with error code 3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The script&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;#!/bin/sh&#xA;&#xA;set -x&#xA;PS4='[\\d \\t] '&#xA;&#xA;# Check parameter for error&#xA;check=0&#xA;# Print fastq-dump executable path&#xA;echo $(which fastq-dump)&#xA;&#xA;# Function to download FastQ with retries&#xA;download_with_retry() {&#xA;    local id=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$1&quot;&#xA;local retries=5&#xA;for attempt in $&lt;/span&gt;(seq &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$retries&quot;); do&#xA;    # Download start of fastq&#xA;    fastq-dump $&lt;/span&gt;(get_ngc) -X 1 -Z --split-spot &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$id&quot; &amp;gt; &quot;$&lt;/span&gt;{id}.test.fastq&amp;quot; 2&amp;gt; &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${id}.test.log&quot;&#xA;    numLines=$&lt;/span&gt;(cat &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${id}.test.fastq&quot; | wc -l)&#xA;    if [ $&lt;/span&gt;numLines -gt 0 ]; then&#xA;            echo &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${id} has data... OK&quot;&#xA;        return 0&#xA;    else&#xA;        echo &quot;$&lt;/span&gt;{id} does not have data on attempt &lt;span class=&quot;math-container&quot;&gt;$attempt... Retrying in 5 seconds...&quot;&#xA;        sleep 5&#xA;    fi&#xA;done&#xA;echo &quot;$&lt;/span&gt;{id} could not be downloaded after $retries attempts... ERROR&amp;quot;&#xA;    check=1&#xA;    return 1&#xA;}&#xA;&#xA;# Loop through all parameters to check validity&#xA;for file in &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$@&quot;; do&#xA;cp &quot;$&lt;/span&gt;{file}&amp;quot; .&#xA;    # Extract filename for sampleID&#xA;    file_basename=&lt;span class=&quot;math-container&quot;&gt;$(basename &quot;$&lt;/span&gt;{file}&amp;quot;)&#xA;    id=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${file_basename%&quot;.id&quot;}&quot;&#xA;# Start validation with retries&#xA;echo &quot;Checking $&lt;/span&gt;{id}...&amp;quot;&#xA;    download_with_retry &amp;quot;$id&amp;quot;&#xA;done&#xA;&#xA;# Exit with error if some FastQs are not accessible&#xA;if [ $check -gt 0 ]; then&#xA;    echo &amp;quot;ERROR: One or more samples have inaccessible FastQs.. exiting&amp;quot;&#xA;    exit 1&#xA;fi&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What exactly is going wrong in the case of the other ID? No matter how many retries I give, it fails.&lt;/p&gt;&#xA;&lt;p&gt;Any suggestion or help would be really appreciated.&lt;/p&gt;&#xA;&lt;p&gt;The error log:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ cat ncbi_error_report.txt&#xA;&amp;lt;Report&amp;gt;&#xA; &amp;lt;Run&amp;gt;&#xA;  &amp;lt;Date&amp;gt;&#xA;   &amp;lt;Start value=&amp;quot;Tue Sep 26 2023 9:16:55 PM&amp;quot;/&amp;gt;&#xA;   &amp;lt;End value=&amp;quot;Tue Sep 26 2023 9:17:11 PM&amp;quot;/&amp;gt;&#xA;  &amp;lt;/Date&amp;gt;&#xA;  &amp;lt;Home name=&amp;quot;HOME&amp;quot; value=&amp;quot;/root&amp;quot;/&amp;gt;&#xA;  &amp;lt;Cwd&amp;gt;/tmp&amp;lt;/Cwd&amp;gt;&#xA;  &amp;lt;CommandLine argc=&amp;quot;6&amp;quot;&amp;gt;&#xA;   &amp;lt;Arg index=&amp;quot;0&amp;quot; value=&amp;quot;fastq-dump&amp;quot;/&amp;gt;&#xA;   &amp;lt;Arg index=&amp;quot;1&amp;quot; value=&amp;quot;-X&amp;quot;/&amp;gt;&#xA;   &amp;lt;Arg index=&amp;quot;2&amp;quot; value=&amp;quot;1&amp;quot;/&amp;gt;&#xA;   &amp;lt;Arg index=&amp;quot;3&amp;quot; value=&amp;quot;-Z&amp;quot;/&amp;gt;&#xA;   &amp;lt;Arg index=&amp;quot;4&amp;quot; value=&amp;quot;--split-spot&amp;quot;/&amp;gt;&#xA;   &amp;lt;Arg index=&amp;quot;5&amp;quot; value=&amp;quot;GSM2683458&amp;quot;/&amp;gt;&#xA;  &amp;lt;/CommandLine&amp;gt;&#xA;  &amp;lt;Result rc=&amp;quot;RC(rcVFS,rcMgr,rcOpening,rcDirectory,rcNotFound)&amp;quot;/&amp;gt;&#xA;  &amp;lt;User admin=&amp;quot;true&amp;quot;/&amp;gt;&#xA; &amp;lt;/Run&amp;gt;&#xA; &amp;lt;Configuration&amp;gt;&#xA;  &amp;lt;Files count=&amp;quot;2&amp;quot;&amp;gt;&#xA;   &amp;lt;File name=&amp;quot;/etc/ncbi/settings.kfg&amp;quot;/&amp;gt;&#xA;   &amp;lt;File name=&amp;quot;/root/.ncbi/user-settings.mkfg&amp;quot;/&amp;gt;&#xA;  &amp;lt;/Files&amp;gt;&#xA;  &amp;lt;refseq state=&amp;quot;not found&amp;quot;/&amp;gt;&#xA;  &amp;lt;krypto state=&amp;quot;pwfile: not found&amp;quot;/&amp;gt;&#xA;  &amp;lt;sra&amp;gt;&#xA;   &amp;lt;quality_type&amp;gt;raw_scores&amp;lt;/quality_type&amp;gt;&#xA;  &amp;lt;/sra&amp;gt;&#xA;  &amp;lt;Config&amp;gt;&#xA;  &amp;lt;ConfigurationFiles&amp;gt;&#xA;/etc/ncbi/settings.kfg&#xA;/root/.ncbi/user-settings.mkfg&#xA;    &amp;lt;/ConfigurationFiles&amp;gt;&#xA;    &amp;lt;APPNAME&amp;gt;&amp;quot;fastq-dump&amp;quot;&amp;lt;/APPNAME&amp;gt;&#xA;    &amp;lt;APPPATH&amp;gt;&amp;quot;/tmp/&amp;quot;&amp;lt;/APPPATH&amp;gt;&#xA;    &amp;lt;BUILD&amp;gt;&amp;quot;RELEASE&amp;quot;&amp;lt;/BUILD&amp;gt;&#xA;    &amp;lt;HOME&amp;gt;&amp;quot;/root&amp;quot;&amp;lt;/HOME&amp;gt;&#xA;    &amp;lt;HOST&amp;gt;&amp;lt;/HOST&amp;gt;&#xA;    &amp;lt;LIBS&amp;gt;&#xA;      &amp;lt;GUID&amp;gt;&amp;quot;119c217a-7b81-47e8-91d6-56d19c8c9f15&amp;quot;&amp;lt;/GUID&amp;gt;&#xA;      &amp;lt;IMAGE_GUID&amp;gt;&amp;quot;119c217a-7b81-47e8-91d6-62229c64ee59&amp;quot;&amp;lt;/IMAGE_GUID&amp;gt;&#xA;    &amp;lt;/LIBS&amp;gt;&#xA;    &amp;lt;NCBI_HOME&amp;gt;&amp;quot;/root/.ncbi&amp;quot;&amp;lt;/NCBI_HOME&amp;gt;&#xA;    &amp;lt;NCBI_SETTINGS&amp;gt;&amp;quot;/root/.ncbi/user-settings.mkfg&amp;quot;&amp;lt;/NCBI_SETTINGS&amp;gt;&#xA;    &amp;lt;OS&amp;gt;&amp;quot;linux&amp;quot;&amp;lt;/OS&amp;gt;&#xA;    &amp;lt;PWD&amp;gt;&amp;quot;/tmp&amp;quot;&amp;lt;/PWD&amp;gt;&#xA;    &amp;lt;USER&amp;gt;&amp;lt;/USER&amp;gt;&#xA;    &amp;lt;VDB_CONFIG&amp;gt;&amp;lt;/VDB_CONFIG&amp;gt;&#xA;    &amp;lt;VDB_ROOT&amp;gt;&amp;lt;/VDB_ROOT&amp;gt;&#xA;    &amp;lt;kfg&amp;gt;&#xA;      &amp;lt;arch&amp;gt;&#xA;        &amp;lt;bits&amp;gt;&amp;quot;64&amp;quot;&amp;lt;/bits&amp;gt;&#xA;        &amp;lt;name&amp;gt;&amp;quot;56d19c8c9f15&amp;quot;&amp;lt;/name&amp;gt;&#xA;      &amp;lt;/arch&amp;gt;&#xA;      &amp;lt;dir&amp;gt;&amp;quot;/root/.ncbi&amp;quot;&amp;lt;/dir&amp;gt;&#xA;      &amp;lt;name&amp;gt;&amp;quot;user-settings.mkfg&amp;quot;&amp;lt;/name&amp;gt;&#xA;    &amp;lt;/kfg&amp;gt;&#xA;    &amp;lt;libs&amp;gt;&#xA;      &amp;lt;cloud&amp;gt;&#xA;        &amp;lt;report_instance_identity&amp;gt;&amp;quot;true&amp;quot;&amp;lt;/report_instance_identity&amp;gt;&#xA;      &amp;lt;/cloud&amp;gt;&#xA;    &amp;lt;/libs&amp;gt;&#xA;    &amp;lt;repository&amp;gt;&#xA;      &amp;lt;user&amp;gt;&#xA;        &amp;lt;ad&amp;gt;&#xA;          &amp;lt;public&amp;gt;&#xA;            &amp;lt;apps&amp;gt;&#xA;              &amp;lt;file&amp;gt;&#xA;                &amp;lt;volumes&amp;gt;&#xA;                  &amp;lt;flat&amp;gt;&amp;lt;/flat&amp;gt;&#xA;                  &amp;lt;flatAd&amp;gt;&amp;quot;.&amp;quot;&amp;lt;/flatAd&amp;gt;&#xA;                &amp;lt;/volumes&amp;gt;&#xA;              &amp;lt;/file&amp;gt;&#xA;              &amp;lt;refseq&amp;gt;&#xA;                &amp;lt;volumes&amp;gt;&#xA;                  &amp;lt;refseqAd&amp;gt;&amp;quot;.&amp;quot;&amp;lt;/refseqAd&amp;gt;&#xA;                &amp;lt;/volumes&amp;gt;&#xA;              &amp;lt;/refseq&amp;gt;&#xA;              &amp;lt;sra&amp;gt;&#xA;                &amp;lt;volumes&amp;gt;&#xA;                  &amp;lt;sraAd&amp;gt;&amp;quot;.&amp;quot;&amp;lt;/sraAd&amp;gt;&#xA;                &amp;lt;/volumes&amp;gt;&#xA;              &amp;lt;/sra&amp;gt;&#xA;              &amp;lt;sraPileup&amp;gt;&#xA;                &amp;lt;volumes&amp;gt;&#xA;                  &amp;lt;ad&amp;gt;&amp;quot;.&amp;quot;&amp;lt;/ad&amp;gt;&#xA;                &amp;lt;/volumes&amp;gt;&#xA;              &amp;lt;/sraPileup&amp;gt;&#xA;              &amp;lt;sraRealign&amp;gt;&#xA;                &amp;lt;volumes&amp;gt;&#xA;                  &amp;lt;ad&amp;gt;&amp;quot;.&amp;quot;&amp;lt;/ad&amp;gt;&#xA;                &amp;lt;/volumes&amp;gt;&#xA;              &amp;lt;/sraRealign&amp;gt;&#xA;              &amp;lt;wgs&amp;gt;&#xA;                &amp;lt;volumes&amp;gt;&#xA;                  &amp;lt;wgsAd&amp;gt;&amp;quot;.&amp;quot;&amp;lt;/wgsAd&amp;gt;&#xA;                &amp;lt;/volumes&amp;gt;&#xA;              &amp;lt;/wgs&amp;gt;&#xA;            &amp;lt;/apps&amp;gt;&#xA;            &amp;lt;root&amp;gt;&amp;quot;.&amp;quot;&amp;lt;/root&amp;gt;&#xA;          &amp;lt;/public&amp;gt;&#xA;        &amp;lt;/ad&amp;gt;&#xA;      &amp;lt;/user&amp;gt;&#xA;    &amp;lt;/repository&amp;gt;&#xA;    &amp;lt;sra&amp;gt;&#xA;      &amp;lt;quality_type&amp;gt;&amp;quot;raw_scores&amp;quot;&amp;lt;/quality_type&amp;gt;&#xA;    &amp;lt;/sra&amp;gt;&#xA;    &amp;lt;vdb&amp;gt;&#xA;      &amp;lt;lib&amp;gt;&#xA;        &amp;lt;paths&amp;gt;&#xA;          &amp;lt;kfg&amp;gt;&amp;quot;/usr/local/bin&amp;quot;&amp;lt;/kfg&amp;gt;&#xA;        &amp;lt;/paths&amp;gt;&#xA;      &amp;lt;/lib&amp;gt;&#xA;    &amp;lt;/vdb&amp;gt;&#xA;  &amp;lt;/Config&amp;gt;&#xA;  &amp;lt;RemoteAccess available=&amp;quot;false&amp;quot;/&amp;gt;&#xA;  &amp;lt;CurrentProtectedRepository found=&amp;quot;false&amp;quot;/&amp;gt;&#xA; &amp;lt;/Configuration&amp;gt;&#xA; &amp;lt;Object path=&amp;quot;https://sra-pub-run-odp.s3.amazonaws.com/sra/SRR5755657/SRR5755657&amp;quot; type=&amp;quot;database&amp;quot; fs_type=&amp;quot;unexpected&amp;quot;&amp;gt;&#xA;  &amp;lt;Dependencies&amp;gt;&#xA;   &amp;lt;List count=&amp;quot;22&amp;quot; missing=&amp;quot;22&amp;quot;&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;0&amp;quot; seq_id=&amp;quot;NC_000067.5&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;1&amp;quot; seq_id=&amp;quot;NC_000068.6&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;2&amp;quot; seq_id=&amp;quot;NC_000069.5&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;3&amp;quot; seq_id=&amp;quot;NC_000070.5&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;4&amp;quot; seq_id=&amp;quot;NC_000071.5&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;5&amp;quot; seq_id=&amp;quot;NC_000072.5&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;6&amp;quot; seq_id=&amp;quot;NC_000073.5&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;7&amp;quot; seq_id=&amp;quot;NC_000074.5&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;8&amp;quot; seq_id=&amp;quot;NC_000075.5&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;9&amp;quot; seq_id=&amp;quot;NC_000076.5&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;10&amp;quot; seq_id=&amp;quot;NC_000077.5&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;11&amp;quot; seq_id=&amp;quot;NC_000078.5&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;12&amp;quot; seq_id=&amp;quot;NC_000079.5&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;13&amp;quot; seq_id=&amp;quot;NC_000080.5&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;14&amp;quot; seq_id=&amp;quot;NC_000081.5&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;15&amp;quot; seq_id=&amp;quot;NC_000082.5&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;16&amp;quot; seq_id=&amp;quot;NC_000083.5&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;17&amp;quot; seq_id=&amp;quot;NC_000084.5&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;18&amp;quot; seq_id=&amp;quot;NC_000085.5&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;19&amp;quot; seq_id=&amp;quot;NC_000086.6&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;20&amp;quot; seq_id=&amp;quot;NC_000087.6&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;    &amp;lt;Dependency index=&amp;quot;21&amp;quot; seq_id=&amp;quot;NC_005089.1&amp;quot; local=&amp;quot;false&amp;quot; path=&amp;quot;&amp;quot;/&amp;gt;&#xA;   &amp;lt;/List&amp;gt;&#xA;  &amp;lt;/Dependencies&amp;gt;&#xA; &amp;lt;/Object&amp;gt;&#xA; &amp;lt;SOFTWARE&amp;gt;&#xA;  &amp;lt;VDBLibrary vers=&amp;quot;2.7.47&amp;quot;/&amp;gt;&#xA;  &amp;lt;Build static=&amp;quot;true&amp;quot;&amp;gt;&#xA;   &amp;lt;Module name=&amp;quot;&amp;quot;/&amp;gt;&#xA;  &amp;lt;/Build&amp;gt;&#xA;  &amp;lt;Tool date=&amp;quot;Nov 18 2022&amp;quot; name=&amp;quot;fastq-dump&amp;quot; vers=&amp;quot;3.0.1&amp;quot;&amp;gt;&#xA;   &amp;lt;Binary path=&amp;quot;/usr/local/bin/fastq-dump&amp;quot; type=&amp;quot;alias&amp;quot; md5=&amp;quot;c461c39bfa514aff3c4f7c0416ced617&amp;quot;&amp;gt;&#xA;    &amp;lt;Alias resolved=&amp;quot;fastq-dump.3&amp;quot;&amp;gt;&#xA;     &amp;lt;Alias resolved=&amp;quot;fastq-dump.3.0.1&amp;quot;&amp;gt;&#xA;      &amp;lt;Alias resolved=&amp;quot;sratools.3.0.1&amp;quot;/&amp;gt;&#xA;     &amp;lt;/Alias&amp;gt;&#xA;    &amp;lt;/Alias&amp;gt;&#xA;   &amp;lt;/Binary&amp;gt;&#xA;  &amp;lt;/Tool&amp;gt;&#xA; &amp;lt;/SOFTWARE&amp;gt;&#xA; &amp;lt;Env&amp;gt;&#xA; &amp;lt;/Env&amp;gt;&#xA;&amp;lt;/Report&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="285" LastEditorUserId="298" LastEditDate="2023-09-27T07:57:28.900" LastActivityDate="2023-10-05T00:47:56.850" Title="Shell script to validate fastq issue" Tags="|fastq|sratoolkit|ncbi|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21603" PostTypeId="1" AcceptedAnswerId="21632" CreationDate="2023-09-27T04:10:18.297" Score="2" ViewCount="48" Body="&lt;p&gt;I hope that the question has its place on this board.&lt;/p&gt;&#xA;&lt;p&gt;I am trying to run the mapping of cis eQTL using an interaction term with tensorQTL. I am running into the following index error when running the software:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; cis.map_nominal(genotype_df, variant_df,&#xA;...                 phenotype_df,&#xA;...                 phenotype_pos_df,&#xA;...                 prefix = prefix,&#xA;...                 covariates_df=covariates_df,&#xA;...                 interaction_df=interaction_s,&#xA;...                 maf_threshold_interaction=0.05,&#xA;...                 run_eigenmt=True,&#xA;...                 output_dir=output,&#xA;...                 write_top=True,&#xA;...                 write_stats=True)&#xA;cis-QTL mapping: nominal associations for all variant-phenotype pairs&#xA;  * 569 samples&#xA;  * 24665 phenotypes&#xA;  * 68 covariates&#xA;  * 69763935 variants&#xA;  * including 1 interaction term(s)&#xA;    * using 0.05 MAF threshold&#xA;  * cis-window: ±1,000,000&#xA;  * checking phenotypes: 24665/24665&#xA;  * Computing associations&#xA;    Mapping chromosome 1&#xA;    processing phenotype 137/24665Traceback (most recent call last):&#xA;  File &amp;quot;/QRISdata/Q5776/software/tensorqtl/tensorqtl/core.py&amp;quot;, line 197, in calculate_interaction_nominal&#xA;    Xinv = torch.matmul(torch.transpose(X_t, 1, 2), X_t).inverse() # ng x (1+2*ni) x (1+2*ni)&#xA;torch._C._LinAlgError: linalg.inv: (Batch element 2704): The diagonal element 3 is zero, the inversion could not be completed because the input matrix is singular.&#xA;&#xA;During handling of the above exception, another exception occurred:&#xA;&#xA;Traceback (most recent call last):&#xA;  File &amp;quot;&amp;lt;stdin&amp;gt;&amp;quot;, line 1, in &amp;lt;module&amp;gt;&#xA;  File &amp;quot;/QRISdata/Q5776/software/tensorqtl/tensorqtl/cis.py&amp;quot;, line 294, in map_nominal&#xA;    res = calculate_interaction_nominal(genotypes_t, phenotype_t.unsqueeze(0), interaction_t,&#xA;  File &amp;quot;/QRISdata/Q5776/software/tensorqtl/tensorqtl/core.py&amp;quot;, line 200, in calculate_interaction_nominal&#xA;    i = int(re.findall('For batch (\d+)', str(e))[0])&#xA;IndexError: list index out of range&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, when I run the same command but only on 50 genes, the code runs properly.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; cis.map_nominal(genotype_df, variant_df,&#xA;...                 phenotype_df.iloc[1:50],&#xA;...                 phenotype_pos_df.iloc[1:50],&#xA;...                 prefix = prefix,&#xA;...                 covariates_df=covariates_df,&#xA;...                 interaction_df=interaction_s,&#xA;...                 maf_threshold_interaction=0.05,&#xA;...                 run_eigenmt=True,&#xA;...                 output_dir=output,&#xA;...                 write_top=True,&#xA;...                 write_stats=True)&#xA;cis-QTL mapping: nominal associations for all variant-phenotype pairs&#xA;  * 569 samples&#xA;  * 49 phenotypes&#xA;  * 68 covariates&#xA;  * 69763935 variants&#xA;  * including 1 interaction term(s)&#xA;    * using 0.05 MAF threshold&#xA;  * cis-window: ±1,000,000&#xA;  * checking phenotypes: 49/49&#xA;  * Computing associations&#xA;    Mapping chromosome 1&#xA;    processing phenotype 49/49&#xA;    time elapsed: 0.44 min&#xA;    * writing output&#xA;done.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The interaction factor is a categorical variable (0/1), I am not sure why the issue is being raised and how to fix&lt;/p&gt;&#xA;&lt;p&gt;I would appreciate any help/direction to go forward.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="18258" LastActivityDate="2023-10-03T23:01:51.743" Title="tensorQTL interaction issue" Tags="|genomics|genetics|qtls|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21604" PostTypeId="1" CreationDate="2023-09-27T06:20:12.720" Score="1" ViewCount="98" Body="&lt;p&gt;I have two enzymes, with heme cofactors. but their overall structure is very different.&lt;/p&gt;&#xA;&lt;p&gt;I use Pymol regularly to visualize protein pdb structures.&lt;/p&gt;&#xA;&lt;p&gt;I want to align both structures, but the structures don't align well due to very low homology. However, their active sites/substrate entrance channels may be more similar than the rest of the structure.&#xA;so,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;I want to align only the substrate channels from both enzymes and compare their structure.&#xA;or&lt;/li&gt;&#xA;&lt;li&gt;align both proteins only using the heme as a reference point. So both hemes from two enzymes align and then I can manually do the rest.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I am more interested in how the substrate would be accessible in both enzymes differently.&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance.&lt;/p&gt;&#xA;&lt;p&gt;Edit: Sorry for being a bit unclear. We can pick two proteins like cytochrome P450 BM3 with pdb id 6H1L or 6H1O. The second protein can be a similar peroxygenase UPO, e.g. 7pn4.&#xA;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="18012" LastEditorUserId="18012" LastEditDate="2023-10-02T15:59:41.287" LastActivityDate="2023-10-02T15:59:41.287" Title="Align two different structures with respect to their active sites or heme cofactor" Tags="|pdb|pymol|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21605" PostTypeId="1" CreationDate="2023-09-27T18:12:15.773" Score="-1" ViewCount="97" Body="&lt;p&gt;I have mutation status (&lt;code&gt;0&lt;/code&gt;,&lt;code&gt;1&lt;/code&gt;) for ten genes in ten cancer 3D models&lt;/p&gt;&#xA;&lt;p&gt;I have also gene expression in these 3D models, like&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;expr1 &amp;lt;- structure(list(GeneA_KO = c(0.0756383119639074, -0.162841965682723, -0.110703100204517, -0.457154616239399, -0.505815193885352, 0.366072775033448, 0.288650458935083, -0.0287646498640309, -0.690732824311905, 0.157366122620995, -1.29451546737684, 0.124167425460344, -0.168045313861751, 0.0186504943306465, -0.495880827400706, -0.206668627339351, -0.148609490335447, 0.066844529089804), GeneB_KO = c(-0.870157285953621, -0.35900872537626, -0.990406888128838, -0.120656981579292, -1.41427631791049, -0.866421095836781, -0.816936112146099, -0.894597236248507, -1.00084584207823, -0.850754488133456, -1.05314239761653, -1.07580675428372, -1.24708324665688, -1.04098460460899, -0.232388675985086, -1.48513703581865, -1.48373348870346, -1.76484058374354)), class = &amp;quot;data.frame&amp;quot;, row.names = c(&amp;quot;ModelM&amp;quot;, &amp;quot;ModelO&amp;quot;, &amp;quot;ModelS&amp;quot;, &amp;quot;ModelA&amp;quot;, &amp;quot;ModelI&amp;quot;, &amp;quot;ModelC&amp;quot;, &amp;quot;ModelT&amp;quot;, &amp;quot;ModelH&amp;quot;, &amp;quot;ModelE&amp;quot;, &amp;quot;ModelR&amp;quot;, &amp;quot;ModelA2&amp;quot;, &amp;quot;ModelP&amp;quot;, &amp;quot;ModelE2&amp;quot;, &amp;quot;ModelU&amp;quot;, &amp;quot;ModelT2&amp;quot;, &amp;quot;ModelC2&amp;quot;, &amp;quot;ModelS2&amp;quot;, &amp;quot;ModelX&amp;quot;))&#xA;mut &amp;lt;- structure(list(ModelM = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelO = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelS = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelA = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelI = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelC = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelT = c(0L, 0L, 0L, 1L, 0L, 1L, 0L, 1L, 1L, 0L), ModelH = c(1L, 1L, 1L, 1L, 0L, 1L, 1L, 1L, 0L, 0L), ModelE = c(1L, 0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 1L), ModelR = c(1L, 0L, 0L, 1L, 0L, 0L, 1L, 0L, 1L, 0L), ModelA2 = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelP = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 1L), ModelE2 = c(1L, 0L, 0L, 1L, 0L, 1L, 0L, 0L, 0L, 0L), ModelU = c(1L, 0L, 1L, 1L, 0L, 1L, 0L, 1L, 1L, 1L), ModelT2 = c(0L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelI2 = c(1L, 0L, 0L, 1L, 0L, 0L, 0L, 0L, 0L, 0L), ModelC2 = c(1L, 1L, 0L, 0L, 0L, 1L, 0L, 1L, 0L, 0L),     ModelS2 = c(0L, 1L, 0L, 1L, 1L, 1L, 1L, 0L, 0L, 1L)), class = &amp;quot;data.frame&amp;quot;, row.names = c(&amp;quot;Gene1_mut&amp;quot;, &amp;quot;Gene2_mut&amp;quot;, &amp;quot;Gene3_mut&amp;quot;, &amp;quot;Gene4_mut&amp;quot;, &amp;quot;Gene5_mut&amp;quot;, &amp;quot;Gene6_mut&amp;quot;, &amp;quot;Gene7_mut&amp;quot;, &amp;quot;Gene8_mut&amp;quot;, &amp;quot;Gene9_mut&amp;quot;, &amp;quot;Gene10_mut&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;mut.csv&lt;/code&gt; and &lt;code&gt;expr1.csv&lt;/code&gt; exported for use in Python:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;write.csv(mut, &amp;quot;mut.csv&amp;quot;, quote = FALSE, row.names = FALSE)&#xA;write.csv(expr1[1:10,], &amp;quot;expr1.csv&amp;quot;, quote = FALSE, row.names = FALSE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to do a one-way ANOVA in Python so that calculate P-VALUE for each pair of mutation-expression in each pair of gene-model&lt;/p&gt;&#xA;&lt;p&gt;This code returns error&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import os&#xA;import pandas as pd&#xA;from statsmodels.formula.api import ols&#xA;import statsmodels.api as sm&#xA;m = pd.read_csv(&amp;quot;mut.csv&amp;quot;, sep=&amp;quot;,&amp;quot;)&#xA;m=pd.DataFrame(m)&#xA;m.info()&#xA;g = pd.read_csv(&amp;quot;expr1.csv&amp;quot;, sep=&amp;quot;,&amp;quot;)&#xA;g=pd.DataFrame(g)&#xA;&#xA;lst = list(g.columns)&#xA;m_lst = list(m.columns)&#xA;&#xA;dfs = pd.DataFrame({})&#xA;for Mut in m_lst:&#xA;    for Model in lst:&#xA;        df = pd.concat([m[[Mut]].reset_index(drop=True), g[[Model]].reset_index(drop=True)],axis=1)&#xA;        df = df.rename(columns={Mut: &amp;quot;mut&amp;quot;, Model: &amp;quot;g&amp;quot;})&#xA;        model = ols('g ~ C(mut)', data=df).fit()&#xA;ValueError: endog has evaluated to an array with multiple columns that has shape (10, 18)&#xA;&#xA;        aov_table = sm.stats.anova_lm(model, typ=2)&#xA;        aov_table[['Model']] = Model&#xA;        aov_table[['Mut']] = Mut&#xA;        dfs = dfs.append(aov_table)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The ideal output is something like&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;with pd.option_context('display.max_rows', None, 'display.max_columns', None):&#xA;    print(dfs)&#xA;            sum_sq   df         F    PR(&amp;gt;F)     Model      Mut&#xA;C(mut)    0.136365  1.0  1.114460  0.321950  GeneA_KO   ModelM&#xA;Residual  0.978877  8.0       NaN       NaN  GeneA_KO   ModelM&#xA;C(mut)    0.540949  1.0  7.375925  0.026417  GeneB_KO   ModelM&#xA;Residual  0.586718  8.0       NaN       NaN  GeneB_KO   ModelM&#xA;C(mut)    0.136365  1.0  1.114460  0.321950  GeneA_KO   ModelO&#xA;Residual  0.978877  8.0       NaN       NaN  GeneA_KO   ModelO&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="650" LastEditDate="2023-09-28T15:24:16.607" LastActivityDate="2023-09-28T15:24:16.607" Title="For loop in Python goes wrong" Tags="|python|for-loop|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21606" PostTypeId="2" ParentId="21605" CreationDate="2023-09-27T18:35:08.180" Score="1" Body="&lt;p&gt;I can see what you're wanting to do, I can see bugs but the key bit of output is missing (below). It seems a bit weird to me the nested loop in context, but thats fine if you're happy.&lt;/p&gt;&#xA;&lt;p&gt;The critical step is this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;df = pd.concat([m[[Mut]].reset_index(drop=True), g[[Model]].reset_index(drop=True)],axis=1)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What I'm concerned about is the &lt;code&gt;pd.concat&lt;/code&gt;. Are you sure &lt;code&gt;pd.concat&lt;/code&gt; is the right command here? I thought &lt;code&gt;m&lt;/code&gt; is one data table, e.g. mutation and &lt;code&gt;g&lt;/code&gt; is another e.g. expression. &lt;code&gt;pd.concat&lt;/code&gt; isn't going to work here, are you sure this isn't a &lt;code&gt;merge&lt;/code&gt; or &lt;code&gt;join&lt;/code&gt; command?&lt;/p&gt;&#xA;&lt;p&gt;Anyway, the fitting is complaining the &lt;code&gt;.shape()&lt;/code&gt; is wrong. Needs reshaping? Thats the error. Printing &lt;code&gt;df&lt;/code&gt; at that point would be needed.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;ValueError: endog has evaluated to an array with multiple columns that has shape (10, 18)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;To summarise normally the error you have received is about re-shaping the data. I've not put forward that solution here because,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;I am not sure what the data represents&lt;/li&gt;&#xA;&lt;li&gt;If I was doing this I wouldn't use &lt;code&gt;pd.concat&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;It seems a lot of columns for a 1-way ANOVA, but could be right, I've not done 1-way ANOVA across 10 columns.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Bug 1&lt;/em&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;m = pd.read_csv(&amp;quot;mut.csv&amp;quot;, sep=&amp;quot;,&amp;quot;)&#xA;m=pd.DataFrame(m)&#xA;m.info()&#xA;g = pd.read_csv(&amp;quot;expr1.csv&amp;quot;, sep=&amp;quot;,&amp;quot;)&#xA;g=pd.DataFrame(g)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This should simply be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;m = pd.read_csv(&amp;quot;mut.csv&amp;quot;)&#xA;g = pd.read_csv(&amp;quot;expr1.csv&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;Bug 2?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dfs = pd.DataFrame({})&#xA;...&#xA;dfs = dfs.append(aov_table)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will work given &lt;code&gt;aov_table&lt;/code&gt; output is a pandas dataframe (I'm not at all sure it is). You need &lt;code&gt;aov_table.info()&lt;/code&gt; to be sure. You can always place it in a dataframe however.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-09-27T23:01:10.543" LastActivityDate="2023-09-27T23:01:10.543" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21608" PostTypeId="2" ParentId="21595" CreationDate="2023-09-28T16:31:15.807" Score="1" Body="&lt;p&gt;Definitely a fan of NOIseq :-) (long story) ... there's &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-022-02648-4&quot; rel=&quot;nofollow noreferrer&quot;&gt;criticism it lacks power&lt;/a&gt;. What you are looking at is the significance of the observation against Type 1 error. The general criticism of non-parametrics is that they have low power, so few results. Personally, I'd use it and given how widely used DSeq2 is perform the same analysis via Dseq2. I'm not a fan of parametrics in this context, but in this field they dominate and DSeq2 is the best. Thus I'd use a parametric (DSeq2) and non-parametric NOIseq. The justification is you are using one of each. Personally, I do neither, I use machine learning to direct the thresholds of a parametric - but thats outside the public domain.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;The time series is more complicated.&lt;/em&gt; Its essentially a 2-way problem: within a time point and between time points. Binary log difference is a good analysis between two time points in this situation.&lt;/p&gt;&#xA;&lt;p&gt;I assume there was treatment between day 3 and day 5 for the treatment samples but not the control.&lt;/p&gt;&#xA;&lt;p&gt;The issue of replicates. Personally what I would attempt to take a short cut with replicates: if Treatment 1 and 2 have a shared impact on the sample, then treating them as replicates and compare them against the control would be the first stage of analytics. This way if you go to the expense of producing the replicates described here you can be sure you've got a result before conducting further wet-lab work.&lt;/p&gt;&#xA;&lt;p&gt;Clearly I don't know what Treatment 1 nor Treatment 2 are, but they could be drugs attacking the same metabolic pathway. Basically to do this you need a biological rationale for combining Treatments 1 and 2 to be considered replicates. I've attempted to re-describe this approach to biological analytics below the &amp;quot;-----&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;The post-hoc analysis is more complex. The post-hoc analysis is a database operation assessing the relative binary log fold difference between time points for treatment versus control.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;@haci likely represent the orthodox approach (e.g. at review level). What I would attempt is to provide biological justification that Treatment 1 and 2 were replicates. That depends on precise justification, i.e. this time series models the impact of generic interruption of this generic class of drug on pathway X, i.e. something that biochemically combines treatment 1 and 2.&lt;/p&gt;&#xA;&lt;p&gt;The other analysis is a within time point analysis and if you did that between Treatment 1 and Treatment 2 for day 3 and day 5 separately (2 analyses), this would be used as justification if there ain't much different within a time point. &lt;strong&gt;In addition&lt;/strong&gt; if treatment is post day 3, then Treatment 1, 2, and control are replicates, i.e. a triplicate (that's unequivocal). That provides a basis for moving forward.&lt;/p&gt;&#xA;&lt;p&gt;However, The interaction @haci is describing is likely a formal model because this is a two-way problem. I rarely disagree but you've got to be careful here because its a time-series which aren't  independent events and most &amp;quot;interaction&amp;quot; based models require independence. Thus day 5 is a result of day 3, therefore they are not independent. I've encountered this as an editor and it can go very wrong. Summary analysis between time series - those are independent, analysis between data points within a time-series are not independent and interaction mixes those up.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt;&#xA;Time series experiments are always complex and performing wet-lab replicates would keep things simple (but obviously more expensive). So, if you've good analytics support or you're an analyst then you probably do it without further wet-lab, otherwise send it back to the wet-lab and then follow the recipe.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-09-28T17:12:42.990" LastActivityDate="2023-09-28T17:12:42.990" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21609" PostTypeId="2" ParentId="21604" CreationDate="2023-09-28T20:32:57.607" Score="1" Body="&lt;p&gt;OK, trying to answer question 2, think I need to be sure that both enzymes have same identical heme cofactor and then find a rotation/translation matrix that applied to heme of protein A moved exactly on top of heme of protein B , than I have to move the entire protein backbone of protein A using that matrix, is it reasonable ? Am I getting it wrong ? With &lt;strong&gt;PyMOL&lt;/strong&gt; I tried to use &lt;code&gt;Pair/Pair_fit&lt;/code&gt; command with two heme proteins :&#xA;&lt;code&gt;2d0t&lt;/code&gt; (&lt;a href=&quot;https://i.stack.imgur.com/IVUTq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/structure/2d0t&lt;/a&gt; and &lt;code&gt;4zv8&lt;/code&gt; (&lt;a href=&quot;https://i.stack.imgur.com/uAsRi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/structure/4ZV8&lt;/a&gt;) :&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;pair hem_2d0t , hem_4zv8,&lt;/code&gt;  same as &lt;code&gt;pair_fit hem_2d0t , hem_4zv8&lt;/code&gt;, where &lt;code&gt;hem_2d0t&lt;/code&gt;,&lt;code&gt;hem_4zv8&lt;/code&gt;  are the heme of the two proteins [&lt;code&gt;select model 2d0t and resn HEM&lt;/code&gt; and &lt;code&gt;set_name sele, hem_2d0t&lt;/code&gt; or &lt;code&gt;select hem_2d0t, model 2d0t and resn HEM&lt;/code&gt;].&lt;/p&gt;&#xA;&lt;p&gt;See results, superposed heme:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IVUTq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IVUTq.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Superposed Proteins :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uAsRi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uAsRi.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;As for question 1 I tried to answer using the tools available to me.&#xA;I picked the two same proteins : &lt;code&gt;2d0t&lt;/code&gt; (&lt;a href=&quot;https://i.stack.imgur.com/IVUTq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/structure/2d0t&lt;/a&gt; and &lt;code&gt;4zv8&lt;/code&gt; (&lt;a href=&quot;https://i.stack.imgur.com/uAsRi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/structure/4ZV8&lt;/a&gt;). Using &lt;code&gt;parKVFinder&lt;/code&gt; (&lt;a href=&quot;https://i.stack.imgur.com/qqzr4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/LBC-LNBio/parKVFinder&lt;/a&gt; , &lt;a href=&quot;https://i.stack.imgur.com/73rfc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.softxjournal.com/article/S2352-7110(20)30319-8/fulltext&lt;/a&gt;) &lt;strong&gt;pymol plugin&lt;/strong&gt; on both proteins stripped of all their HETATM, I calculated their cavities. As an example , here a png of &lt;code&gt;2d0t&lt;/code&gt;  cavities :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qqzr4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qqzr4.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then I extracted the residues surrounding the cavity using :&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;select model 2d0t_merged near_to 5.0 of resn KAF&lt;/code&gt; ; [named &lt;code&gt;sele_2d0t&lt;/code&gt;]&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;select model 4zv8_merged near_to 5.0 of resn KAB&lt;/code&gt; ; [named &lt;code&gt;sele_4zv8&lt;/code&gt;]&lt;/p&gt;&#xA;&lt;p&gt;being &lt;code&gt;KAF&lt;/code&gt; and &lt;code&gt;KAB&lt;/code&gt; cavities described as non_bounded atom in pdb format as output of &lt;code&gt;parKVFinder&lt;/code&gt; .&lt;br /&gt;&#xA;I believe &lt;code&gt;parKVFinder&lt;/code&gt; has is own output for the residues surrounding cavities, but wanted to try &lt;code&gt;PyMOL&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Example for &lt;code&gt;2d0t&lt;/code&gt; :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/73rfc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/73rfc.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then I used&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;cealign sele_4zv8 , sele_2d0t&lt;/code&gt; (&lt;a href=&quot;https://pymolwiki.org/index.php/Cealign&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pymolwiki.org/index.php/Cealign&lt;/a&gt;) to align the residues of the two cavities. (&lt;code&gt;cealign&lt;/code&gt; uses just CA of the selections provided).&lt;/p&gt;&#xA;&lt;p&gt;Results , only cavities and surrounding residues :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/aIEa5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aIEa5.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;the two heme protein cavities are not alignrd even if the HIS (2dt0, red)  and CYS (4zv8, yellow) are on the same side :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/knQt2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/knQt2.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;NOTE:&lt;/p&gt;&#xA;&lt;p&gt;2dt0 in magenta&lt;/p&gt;&#xA;&lt;p&gt;4zv8 in cyan&lt;/p&gt;&#xA;&lt;p&gt;I choose 2 different heme protein (could be more extreme though) , just because the question was open to broad attempts, could be interesting too see for example results on not too distant cytochromes&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2023-10-01T10:45:20.583" LastActivityDate="2023-10-01T10:45:20.583" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21610" PostTypeId="1" CreationDate="2023-09-28T22:18:31.810" Score="0" ViewCount="42" Body="&lt;p&gt;I downloaded the TCGA pathology slides for the SKCM project and the files are all a name such as &amp;quot;TCGA-3N-A9WB-01Z-00-DX1.A9950ED4-9480-455C-AE0D-8E076D4DA432.svs&amp;quot;.&#xA;How do you get the clinical metadata for this dataset? When I click &amp;quot;clinical&amp;quot; for the project, the .tsv file has no case IDs that match to my file names.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://portal.gdc.cancer.gov/exploration?filters=%7B%22content%22%3A%5B%7B%22content%22%3A%7B%22field%22%3A%22cases.case_id%22%2C%22value%22%3A%5B%22set_id%3AYavV3YoBvzAafV-N5zuW%22%5D%7D%2C%22op%22%3A%22IN%22%7D%5D%2C%22op%22%3A%22AND%22%7D&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://portal.gdc.cancer.gov/exploration?filters=%7B%22content%22%3A%5B%7B%22content%22%3A%7B%22field%22%3A%22cases.case_id%22%2C%22value%22%3A%5B%22set_id%3AYavV3YoBvzAafV-N5zuW%22%5D%7D%2C%22op%22%3A%22IN%22%7D%5D%2C%22op%22%3A%22AND%22%7D&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18274" LastActivityDate="2023-09-28T22:18:31.810" Title="How do you download metadata for TCGA pathology slides" Tags="|genome|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21611" PostTypeId="1" CreationDate="2023-09-29T07:01:40.627" Score="0" ViewCount="59" Body="&lt;p&gt;With &lt;a href=&quot;https://huttenhower.sph.harvard.edu/humann&quot; rel=&quot;nofollow noreferrer&quot;&gt;HUMAnN3&lt;/a&gt;, I got the result of the KEGG KO annotation (by &lt;em&gt;humann_regroup_table&lt;/em&gt;:humann_regroup_table --input hmp_subset_genefamilies.tsv --groups uniref90_ko --output hmp_subset_genefamilies_uniref90_ko.tsv).&lt;/p&gt;&#xA;&lt;p&gt;I need to quantify/transform the KEGG KO annotation to KEGG module definitions.&lt;/p&gt;&#xA;&lt;p&gt;However, I did not find the relevant guidance/instruction. Are there any relevant tutorials available for reference?&lt;/p&gt;&#xA;&lt;p&gt;Just like the packages (&amp;quot;clusterProfiler&amp;quot;) using R for transform to KEGG pathway (KEGG pathway enrichment analysis).&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot!&lt;/p&gt;&#xA;&lt;p&gt;Any help will get me out of trouble!&lt;/p&gt;&#xA;" OwnerUserId="18277" LastEditorUserId="18277" LastEditDate="2023-10-06T02:14:53.033" LastActivityDate="2023-10-06T02:14:53.033" Title="How to obtain the KEGG mudule based on KEGG KO? (KEGG mudule enrichment)" Tags="|clustering|pathway|kegg|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21612" PostTypeId="1" CreationDate="2023-09-29T08:26:18.240" Score="2" ViewCount="72" Body="&lt;p&gt;I'm writing a function (python) that merges overlapping paired reads of a fragment, and a corresponding reference sequence. There's quite a lot of edge cases (e.g. when the reads disagree on indels) so I wonder if there's some standard library that does that?&lt;/p&gt;&#xA;&lt;p&gt;Context - the input is pysam AlignedSegment DNA reads. The output is fragment, reference - both strings corresponding to the merged fragment and its corresponding reference.&lt;/p&gt;&#xA;&lt;p&gt;e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;read1:           ACGTACGTACGT&#xA;read2:                CGTACGTACGT&#xA;expected output: ACGTACGTACGTACGT&#xA;&#xA;read1:           ACGTANGTACGT&#xA;read2:               ACGTACGTACGT&#xA;expected output: ACGTACGTACGTACGT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;this is just an illustration. The idea is to merge the overlapping DNA read sequences into the fragment and its corresponding reference using the information in the AlignedSegment (i.e. query_sequence, reference_start, query_qualities, cigartuples, and get_reference_sequence())&lt;/p&gt;&#xA;&lt;p&gt;Thanks.&lt;/p&gt;&#xA;" OwnerUserId="18278" LastEditorUserId="298" LastEditDate="2023-09-29T10:11:21.797" LastActivityDate="2024-02-27T00:02:25.270" Title="Merge fragment reads" Tags="|pysam|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21614" PostTypeId="1" CreationDate="2023-09-29T15:23:59.697" Score="1" ViewCount="115" Body="&lt;p&gt;Suppose I have a protein PDB file and want to compute energy using the following formula from the coordinates in the file.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qRIFG.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qRIFG.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;How can I do that?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Sometimes people suggest using the &lt;code&gt;CONECT&lt;/code&gt; field for that purpose. However, some PDB files do not have this field.&lt;/p&gt;&#xA;&lt;p&gt;Example: &lt;a href=&quot;https://www.rcsb.org/structure/4osk&quot; rel=&quot;nofollow noreferrer&quot;&gt;4OSK.pdb (Crystal structure of TAL effector reveals the recognition between asparagine and guanine)&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="12686" LastEditDate="2023-10-02T10:51:08.187" LastActivityDate="2023-10-02T10:51:08.187" Title="How can I compute energy from a protein PDB file?" Tags="|proteins|pdb|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21616" PostTypeId="2" ParentId="21612" CreationDate="2023-09-29T22:54:41.397" Score="0" Body="&lt;p&gt;I'd suggest looking at similar tools that already exist, e.g. &lt;a href=&quot;https://github.com/ponnhide/flashpy&quot; rel=&quot;nofollow noreferrer&quot;&gt;here is one that I hadn't heard of that lives on github&lt;/a&gt;. Just glancing at the code, it seems to use a scoring heuristic to decide how to derive a merge point, so it's using that to get around the absence of unambiguous points, probably at some rate of error.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/284637/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioPython has some functions&lt;/a&gt; that will create consensus sequences for you as well based on a pairwise alignment. (&lt;code&gt;Bio.Align.AlignInfo.SummaryInfo.dumb_consensus&lt;/code&gt;, linked in that post.)&lt;/p&gt;&#xA;&lt;p&gt;As a command line tool &lt;a href=&quot;https://pandaseq-tutorial.readthedocs.io/en/latest/tutorial/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pandaseq&lt;/a&gt; does the same thing, I'd suggest looking at how they handle edge cases.&lt;/p&gt;&#xA;&lt;p&gt;There are similar utilities in &lt;a href=&quot;http://qiime.org/scripts/join_paired_ends.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;QIIME&lt;/a&gt;, &lt;a href=&quot;https://dash.harvard.edu/bitstream/handle/1/42661733/MS_9.pdf?sequence=10&quot; rel=&quot;nofollow noreferrer&quot;&gt;NGMerge&lt;/a&gt;, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5657622/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BBMerge&lt;/a&gt;, [illumina-utils]- this is a pretty well-trodden path, I think all of those will be illuminating to look at in terms of reusing or writing original code.&lt;/p&gt;&#xA;&lt;p&gt;I'm guessing it will be easier to merge FASTQs and then align, rather than align and then merge. (Note that these tools don't seem to operate on alignments, because that is trickier).&lt;/p&gt;&#xA;&lt;p&gt;Alternately, if your real goal is to create a consensus genome sequence from your reference based on your alignments, there are a large number of tools that do that, notably &lt;code&gt;samtools&lt;/code&gt; (see e.g. &lt;a href=&quot;https://www.biostars.org/p/419927/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/419927/&lt;/a&gt;). No new code needed, you just input your BAM file and reference sequence.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-09-29T22:54:41.397" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21617" PostTypeId="1" AcceptedAnswerId="21618" CreationDate="2023-09-30T12:29:37.667" Score="3" ViewCount="128" Body="&lt;p&gt;I have a list containing the results of a MSA:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;msa = [&#xA;    '-G-ATG-CGGATACCG-', # reference sequence&#xA;    'CGTATCTCG--TACCGA',&#xA;    '-GTATCTCGT-GACCG-',&#xA;    'CG-AT--CGG-GACCGA',&#xA;    '---AT--CG--GACCGA',&#xA;    '-GTAT---GC-GACCG-',&#xA;    'CG-AT--CGG-GACCGA',&#xA;    'CGTAT--CG--GA CG-',&#xA;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to generate the CIGAR strings of those alignments (against the reference sequence); the expected result would be something like:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;cigars = [&#xA;    '13M',&#xA;    '1I1M1I3M1I2M2D5M1I',&#xA;    '1M1I3M1I3M1D5M',&#xA;    '1I3M1D3M1D5M1I',&#xA;    '1D2M1D2M2D5M1I',&#xA;    '1M1I2M2D2M1D5M',&#xA;    '1I3M1D3M1D5M1I',&#xA;    '1I1M1I2M1D2M2D5M',&#xA;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I don't know what would be a good approach for this. I managed to get the &amp;quot;aligned pairs&amp;quot; of each alignment but I don't think that'll help for determining the cigar strings:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;# mapping = position in alignment =&amp;gt; position in reference&#xA;mapping = {y: x for x,y in enumerate(y for y,nuc in enumerate(msa[0]) if nuc != '-')}&#xA;&#xA;for aln in msa:&#xA;    aligned_pairs = [(y, mapping[z]) for y,z in enumerate(z for z,nuc in enumerate(aln) if nuc != '-') if z in mapping]&#xA;    print(aligned_pairs)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;[(0, 0), (1, 1), (2, 2), (3, 3), (4, 4), (5, 5), (6, 6), (7, 7), (8, 8), (9, 9), (10, 10), (11, 11), (12, 12)]&#xA;[(1, 0), (3, 1), (4, 2), (5, 3), (7, 4), (8, 5), (9, 8), (10, 9), (11, 10), (12, 11), (13, 12)]&#xA;[(0, 0), (2, 1), (3, 2), (4, 3), (6, 4), (7, 5), (8, 6), (9, 8), (10, 9), (11, 10), (12, 11), (13, 12)]&#xA;[(1, 0), (2, 1), (3, 2), (4, 4), (5, 5), (6, 6), (7, 8), (8, 9), (9, 10), (10, 11), (11, 12)]&#xA;[(0, 1), (1, 2), (2, 4), (3, 5), (4, 8), (5, 9), (6, 10), (7, 11), (8, 12)]&#xA;[(0, 0), (2, 1), (3, 2), (4, 5), (5, 6), (6, 8), (7, 9), (8, 10), (9, 11), (10, 12)]&#xA;[(1, 0), (2, 1), (3, 2), (4, 4), (5, 5), (6, 6), (7, 8), (8, 9), (9, 10), (10, 11), (11, 12)]&#xA;[(1, 0), (3, 1), (4, 2), (5, 4), (6, 5), (7, 8), (8, 9), (9, 10), (10, 11), (11, 12)]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I generate the CIGAR strings efficiently?&lt;/p&gt;&#xA;" OwnerUserId="18190" LastEditorUserId="18190" LastEditDate="2023-10-09T17:35:16.443" LastActivityDate="2023-10-09T17:35:16.443" Title="Generating CIGAR strings for MSA with reference" Tags="|python|cigar|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21618" PostTypeId="2" ParentId="21617" CreationDate="2023-10-01T01:45:39.707" Score="4" Body="&lt;p&gt;If you loop through the characters of two aligned sequences &lt;strong&gt;in parallel&lt;/strong&gt; then you can easily determine the operations that compose the CIGAR. Then you just have to &amp;quot;collapse&amp;quot; the generated list of operations:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import itertools&#xA;&#xA;ref = msa[0]&#xA;&#xA;for aln in msa:&#xA;    cig = []&#xA;&#xA;    for i,nuc in enumerate(aln):&#xA;        if nuc != '-':&#xA;            if ref[i] != '-':&#xA;                cig.append('M')&#xA;            else:&#xA;                cig.append('I')&#xA;        elif ref[i] != '-':&#xA;                cig.append('D')&#xA;&#xA;    cig = [(len(list(it)), op) for op,it in itertools.groupby(cig)]&#xA;&#xA;    print( ''.join(str(n) + op for n,op in cig) )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;13M&#xA;1I1M1I3M1I2M2D5M1I&#xA;1M1I3M1I3M1D5M&#xA;1I3M1D3M1D5M1I&#xA;1D2M1D2M2D5M1I&#xA;1M1I2M2D2M1D5M&#xA;1I3M1D3M1D5M1I&#xA;1I1M1I2M1D2M2D5M&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18190" LastEditorUserId="18190" LastEditDate="2023-10-02T19:45:47.970" LastActivityDate="2023-10-02T19:45:47.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21619" PostTypeId="2" ParentId="20589" CreationDate="2023-10-01T06:57:42.487" Score="1" Body="&lt;p&gt;I have experienced the same issue yesterday and after trying the &lt;code&gt;--use-ftp&lt;/code&gt; argument without luck (this is proposed as an alternative in the forums), I have downloaded the database from &lt;a href=&quot;https://benlangmead.github.io/aws-indexes/k2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kraken 2, KrakenUniq and Bracken indexes&lt;/a&gt; (&lt;a href=&quot;https://github.com/DerrickWood/kraken2/issues/753#issuecomment-1692288857&quot; rel=&quot;nofollow noreferrer&quot;&gt;ref&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2023-10-01T06:57:42.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21620" PostTypeId="1" CreationDate="2023-10-01T10:16:57.723" Score="0" ViewCount="15" Body="&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/21614/how-can-i-retrieve-bond-information-from-a-protein-pdb-file&quot;&gt;How can I compute energy from a protein PDB file?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I asked the above question.&lt;/p&gt;&#xA;&lt;p&gt;Then I thought, Why don't I check how BioJava obtains bond information from a protein PDB file?&lt;/p&gt;&#xA;&lt;p&gt;Surprisingly, BioJava's tutorial &lt;a href=&quot;https://github.com/biojava/biojava-tutorial/blob/master/structure/README.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;doesn't have any discussion on Bonds&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I checked &lt;a href=&quot;https://github.com/biojava/biojava&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioJava's GitHub repo&lt;/a&gt;, but I failed to find any such information.&lt;/p&gt;&#xA;&lt;p&gt;How does BioJava retrieve bond information from a PDB file?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="650" LastEditDate="2023-10-02T13:43:27.940" LastActivityDate="2023-10-02T13:43:27.940" Title="How does BioJava retrieve bond information from a PDB file?" Tags="|pdb|java|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21621" PostTypeId="1" CreationDate="2023-10-01T12:48:50.063" Score="0" ViewCount="23" Body="&lt;p&gt;Does anyone know of a Ramachandran plot server (URL please) that allows you to display, for example, just the leucine residues in a protein structure rather than displaying all the amino acids in the pdb file? Thanks!&lt;/p&gt;&#xA;&lt;p&gt;Paul&lt;/p&gt;&#xA;" OwnerUserId="18286" LastActivityDate="2023-10-01T12:48:50.063" Title="Ramachandran Plot for specific amino acids in a pdb file" Tags="|pdb|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21622" PostTypeId="1" AcceptedAnswerId="21623" CreationDate="2023-10-01T16:54:48.517" Score="1" ViewCount="263" Body="&lt;p&gt;I'm leveraging &lt;strong&gt;minimap2&lt;/strong&gt; to map select genes from short-read microbial fastq metagenomics zip files.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 --heap-sort=yes --splice-flank=no metagenome.fastq.gz fastaDB.fas &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What I noticed is the low tolerance of mismatch when I need to maximise mismatch because microbes have loads of genetic diversity. Thus the collection of references sequences are never really a reference, in fact the &amp;quot;microbe reference genome&amp;quot; is a misnomer, because &lt;em&gt;in situ&lt;/em&gt; diversity of microbes is huge.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question&lt;/strong&gt; How do you relax the mapping stringency criteria for minimap2? If you are familiar with Blast parameters this would help alot here. I know blast but I've never parameterised it.&lt;/p&gt;&#xA;&lt;p&gt;The options are this are:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;&lt;a href=&quot;https://lh3.github.io/minimap2/minimap2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Alignment options&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;-A Matching score [2]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;-B INT    Mismatching penalty [4]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;-O INT1[,INT2] Gap open penalty [4,24].&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;-s INT    Minimal peak DP alignment score to output [40].&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;--score-N INT&#xA;Score of a mismatch involving ambiguous bases &lt;a href=&quot;https://lh3.github.io/minimap2/minimap2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Anything else that is relevant.&lt;/p&gt;&#xA;&lt;p&gt;Note, -f is a famous option, not sure it helps here.&lt;/p&gt;&#xA;&lt;p&gt;I'm going to code up a hyperparameterisation unit-test, but understanding the parameters first would really, really help.&lt;/p&gt;&#xA;&lt;p&gt;Alternative approaches welcome as well.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;em&gt;Plugging minimap2&lt;/em&gt;. I code high performance (hadoop-style, back-tracking, graph, traverse tall buildings in a stride  etc ...). The coder that wrote minimap has knowledge beyond, i.e. very fast run times directly from gzip file. I assume they are indexing then chunking the raw zip file then unzipping each 'chunk' JIT (just in time).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Thanks @MaximilianPress great answer&lt;/strong&gt;. I'll fill in the backstory.&lt;/p&gt;&#xA;&lt;p&gt;I'm trapping total genetic diversity across Genbank and the metagenomes. My protocol to do both:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Blasting out Genbank and then&lt;/li&gt;&#xA;&lt;li&gt;piping the output across metagenomes via in this case minimap2.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Hence speed is key. So I ain't replacing Blast with minimap2 but leveraging minimap2 via the output of blast.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;However&lt;/em&gt; on reflection of @MaximilianPress's discussion the answer is to generate a gold standard of diversity to assess alternative approaches.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-10-02T10:55:25.557" LastActivityDate="2023-10-03T16:09:08.267" Title="Parameter optimisation of minimap2" Tags="|fastq|blast|read-mapping|minimap2|parameters|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21623" PostTypeId="2" ParentId="21622" CreationDate="2023-10-01T20:38:57.250" Score="2" Body="&lt;h2&gt;First of all, use short read option&lt;/h2&gt;&#xA;&lt;p&gt;The short read preset (&lt;code&gt;-x sr&lt;/code&gt;) for minimap2 is heavily recommended for doing such mapping:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;minimap2 -x sr&lt;/code&gt;&lt;/p&gt;&#xA;&lt;h2&gt;Heng Li on minimap2 for related genomes&lt;/h2&gt;&#xA;&lt;p&gt;Heng Li (minimap2 author) has some suggestions on &lt;a href=&quot;https://github.com/lh3/minimap2/issues/521&quot; rel=&quot;nofollow noreferrer&quot;&gt;a somewhat related github issue&lt;/a&gt; (though it is now 4 years old):&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Minimap2 is not really intended for this purpose. For short reads he recommends &lt;a href=&quot;https://github.com/uwb-linux/stampy&quot; rel=&quot;nofollow noreferrer&quot;&gt;stampy&lt;/a&gt;, but it is clearly quite old (python2.7) and does not seem to have much recent developer history.&lt;/li&gt;&#xA;&lt;li&gt;Failing that, he suggests an option set such as follows:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If you want to give minimap2 a try anyway, you can use the following setting:  &lt;code&gt;minimap2 -axsr -A2 -B4 -O4,24 -E2,1&lt;/code&gt;&#xA;If minimap2 is not sensitive enough, reduce -k and -w at the cost of mapping speed. Reducing -w should have relatively lower impact on speed, but reducing -k should be more sensitive.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I would suggest engaging with Heng, he is quite responsive in general and may have more suggestions.&lt;/p&gt;&#xA;&lt;h2&gt;Other aligners for metagenomics&lt;/h2&gt;&#xA;&lt;p&gt;I would suggest that you look for more possibilities for alignment, or possibly &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwjRub2x2NWBAxUXGDQIHShcD8kQFnoECBoQAQ&amp;amp;url=https%3A%2F%2Fbmcbioinformatics.biomedcentral.com%2Farticles%2F10.1186%2Fs12859-022-05103-0&amp;amp;usg=AOvVaw1qd9PVqzz2yA3iLh2KNjhk&amp;amp;opi=89978449&quot; rel=&quot;nofollow noreferrer&quot;&gt;not using alignment in the first place&lt;/a&gt;. However, this is very sensitive to your ultimate application, which I'm not sure that I understand from your post.&lt;/p&gt;&#xA;&lt;p&gt;I was pleased to see that &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9040843/&quot; rel=&quot;nofollow noreferrer&quot;&gt;others have reported good results using minimap2 for metagenomics&lt;/a&gt;, which I was skeptical about, having been out of the field for a bit. Usually &lt;a href=&quot;https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0105776&quot; rel=&quot;nofollow noreferrer&quot;&gt;in metagenomics there is an expected tradeoff&lt;/a&gt; between sensitivity and specificity for the reasons that you state (this paper is old but it uses BLAST, so it is a sort of best case for sensitivity on unrelated genomes at the level of divergence that they test).&lt;/p&gt;&#xA;&lt;p&gt;There are of course new shotgun metagenomic aligners coming out all the time, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9650795/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here is one recent example&lt;/a&gt; that I know nothing about but which may be interesting as a comparison.&lt;/p&gt;&#xA;&lt;p&gt;If you are really looking for a rich parameter set that allows low-stringency mappings, I would suggest going back to BLAST, though it will be much slower. minimap2 and related algorithms are to some extent dependent on the ability to find short seed matches, so there will necessarily be less than perfect sensitivity in your application.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-10-01T20:38:57.250" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21624" PostTypeId="2" ParentId="21552" CreationDate="2023-10-02T07:09:28.767" Score="2" Body="&lt;p&gt;I suggest using package &lt;a href=&quot;https://docs.python.org/3/library/glob.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;glob&lt;/a&gt;, which supports unix-style pattern search. For instance finding all .py files in my repo:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import glob &#xA;all_python_files = glob.glob(r'c:\personal\coderepo\**\*.py', recursive=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Which outputs full paths:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;'c:\personal\coderepo\SimpleHTR\data\checkDirs.py', &lt;br&gt;&#xA;'c:\personal\coderepo\....&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I typically like to input the glob.glob to &lt;a href=&quot;https://pandas.pydata.org/docs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Pandas&lt;/a&gt; dataframes:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;all_files = glob.glob(f'{base_folder}/*')&#xA;all_files = pd.DataFrame(all_files, columns=['full_path'])&#xA;all_files['my_filename'] = all_files['full_path'].apply(lambda x: os.path.basename(x))&#xA;&#xA;#do whatever filtering is required:&#xA;all_files = all_files[all_files['my_file_name'].str.contains('regex_here')]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then perform what ever I need to do in loop:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for idx, _ in all_files.iterrows():&#xA;    do_something(_['full_path'])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17627" LastActivityDate="2023-10-02T07:09:28.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21625" PostTypeId="1" CreationDate="2023-10-02T10:55:20.467" Score="2" ViewCount="178" Body="&lt;p&gt;As you know, ncbi has several databases, including nt, nr, etc.&#xA;What I am going to do is to blast a sequence of approximately 30 base pairs to the nr database and extract all similar sequences.&#xA;For this purpose, with the help of YouTube, I downloaded and installed Command Line Blast on my Windows PC.&lt;/p&gt;&#xA;&lt;p&gt;Then, from the database section I downloaded all the nr databases, which were 83 zip files and 200 GB in size.  Then, I unzipped it and stored it to a folder on my computer.&lt;/p&gt;&#xA;&lt;p&gt;Because I saw that the size of the first two zip files that came out of the zip increased from 25 GB to 50 GB, I only unzipped two files to see if my command line works or not.&lt;/p&gt;&#xA;&lt;p&gt;So my database became a folder of almost 50 GB.&lt;/p&gt;&#xA;&lt;p&gt;Then I ran the simplest command in the cmd of my computer.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastn -query D:\query.fa -out D:\results.txt -db D:\nrDataBase&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It's been almost two weeks now that I'm stuck day and night with this simple line of code and it doesn't work.&lt;/p&gt;&#xA;&lt;p&gt;It says once that the query file is not accessible. Once it says that no suitable files were found in the database, etc.&lt;/p&gt;&#xA;&lt;p&gt;I tested every solution that came to my mind, I used the text format for the query, but it didn't work&lt;/p&gt;&#xA;&lt;p&gt;I downloaded one or two files from the nt database, I said maybe there is a problem with the nr database, but it still didn't work.&#xA;I made a small fasta file, introduced it as a database, but it still didn't accept.&lt;/p&gt;&#xA;&lt;p&gt;I tried also on the Macbook, it still didn't work, either the query got an error or the database.&lt;/p&gt;&#xA;&lt;p&gt;The blast has been well installed, When using -h it shows the full info of the tool. It doesn’t work means that I encounter different errors like: The query is not acceptable Or The alias or index file is found in the database I encounter these errors while e. g., the query file is a normal fasta file. I mean, the errors don’t seem to be logical, it looks like something else is going wrong.&lt;/p&gt;&#xA;&lt;p&gt;I'm really upset, I told you to send me a message to see if you can help me.&lt;/p&gt;&#xA;&lt;p&gt;Thank you very much and sorry for the long message.&lt;/p&gt;&#xA;" OwnerUserId="18298" OwnerDisplayName="Farzad Beikpour" LastEditorUserId="73" LastEditDate="2023-10-04T02:50:54.237" LastActivityDate="2023-11-13T20:28:38.113" Title="BLAST+ not working on my Command Line" Tags="|blast|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21626" PostTypeId="1" AcceptedAnswerId="21634" CreationDate="2023-10-02T16:14:52.300" Score="2" ViewCount="337" Body="&lt;p&gt;How do the Oxford Nanopore Flongle flow cells perform compared to the standard MinION flow cells, when it comes to reliability and error rates? I found &lt;a href=&quot;https://doi.org/10.1101/2022.07.13.499863&quot; rel=&quot;nofollow noreferrer&quot;&gt;this comparison article&lt;/a&gt; from last year, from where the below graph is taken, but they used the old chemistry. How is the situation today with the new R10.4.1 nanopores and Kit 14 chemistry? Does anyone have first-hand experience, or a good reference?&lt;/p&gt;&#xA;&lt;p&gt;My use case would mainly be metabarcoding, i.e. amplicon sequencing. If anyone has experience of Flongle flow cells for specifically metabarcoding, I would be happy to hear your thoughts.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mtEqK.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mtEqK.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17054" LastEditorUserId="73" LastEditDate="2023-10-09T04:20:40.343" LastActivityDate="2023-10-18T13:52:48.600" Title="Nanopore Flongle vs MinION" Tags="|nanopore|minion|barcode|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21627" PostTypeId="2" ParentId="21622" CreationDate="2023-10-03T12:56:50.487" Score="1" Body="&lt;p&gt;Just to follow-up on the performance of &lt;strong&gt;minimap2&lt;/strong&gt;: the solution @MaximillianPress described works. Thousands of hits are obtained for comparatively obscure genes (some will be duplicates). Parameterisation gives a 2-log&lt;sub&gt;10&lt;/sub&gt; increase (astonishing).&lt;/p&gt;&#xA;&lt;p&gt;The issue is that its:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;relatively slow - precisely as described by @MP;&lt;/li&gt;&#xA;&lt;li&gt;very, very sensitive to parameterisation, just a single integer changes the output by a couple of thousand hits.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I can see why there was concern about using it - such notable sensitive is an issue - and alternatively its clear why other investigators have had great success with it - hit the &amp;quot;sweet-spot&amp;quot; and its a bonanza. For my purposes it's fine - it's not a quantitation of complete genetic diversity, as long as I capture a reasonable proportion of diversity that is fine. Clearly, they'll be times the &amp;quot;sweet-spot&amp;quot; is missed but thats still okay (famous last words :-) ).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-10-03T16:09:08.267" LastActivityDate="2023-10-03T16:09:08.267" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21628" PostTypeId="1" AcceptedAnswerId="21642" CreationDate="2023-10-03T15:13:29.570" Score="2" ViewCount="43" Body="&lt;p&gt;In Snakemake I want to use two input files for comparison. The one file has a sample name (e.g. 526846), with an experimental id (e.g. EF_S13' and 'MF_S5'). The file I want to compare it to has the same sample name only. I want to compare each sample with experiment ID (first files) to the file with corresponding sample name (second file). These files are in different directories. I thought I would be able to use two expand functions with two lists, as shown in the code below.&#xA;Note: these are example lists and the lists in reality is muchlarger.&lt;/p&gt;&#xA;&lt;p&gt;Relevant code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;SAMPLE_SET = ['12345', '56789', '54321']&#xA;ID_SET = ['EF_S13', 'MF_S5', 'EF_S11', 'MF_S3', 'EF_S10', 'MF_S2']&#xA;&#xA;rule all:&#xA;    input:&#xA;        array = expand(&amp;quot;results/{sample}_{samp_id}_array/{sample}_{samp_id}.summary.csv&amp;quot;, zip, sample = SAMPLE_SET, samp_id = ID_SET)&#xA;&#xA;&#xA;rule happy_rtg_array:&#xA;    &amp;quot;&amp;quot;&amp;quot; Normalizes and compares vcf files with array samples&amp;quot;&amp;quot;&amp;quot;&#xA;    input:&#xA;        query_vcf = expand(&amp;quot;WESP_run/results/variant_calling/{sample}_{samp_id}.vcf&amp;quot;, zip, sample = SAMPLE_SET, samp_id = ID_SET),&#xA;        truth_vcf = expand(os.path.join(config[&amp;quot;truth_dir_vcf_gsa&amp;quot;],&amp;quot;{sample}.vcf.gz&amp;quot;), sample = SAMPLE_SET),&#xA;    output: &#xA;        &amp;quot;WESPValidation/results/{sample}_{samp_id}_array/{sample}_{samp_id}.summary.csv&amp;quot;&#xA;[more code]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, when I look at the output log file with --dry-run, I get as input for the job a huge list with all the inputs in one go, rather then per file/comparison, and one output file (see relevant output), and that times the amount of the (right) combinations of the file name and id (counted from the output).&lt;/p&gt;&#xA;&lt;p&gt;Also the wildcards are not what I thought they should be. I might be confused with the expand function, as I thought how I used them now should &amp;quot;lock in&amp;quot; the wildcards and become placeholders filled in by the lists.&lt;/p&gt;&#xA;&lt;p&gt;Relevant output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[Tue Oct  3 14:31:20 2023]&#xA;rule happy_rtg_array:&#xA;    input: 12345_EF_S13.vcf, 12345_MF_S5.vcf, 56789_EF_S11.vcf, 56789_MF_S3.vcf, 54321_EF_S10.vcf, 54321_MF_S2.vcf, 12345.vcf, 56789.vcf, 54321.vcf&#xA;    output: results/12345_EF_S13_array/12345_EF_S13.summary.csv&#xA;    log: happy_rtg_array/12345_EF_S13.log&#xA;    jobid: 6&#xA;    reason: Missing output files: WESPValidation/results/12345_EF_S13_array/12345_EF_S13.summary.csv&#xA;    wildcards: sample=12345_EF, samp_id=S13&#xA;    threads: 10&#xA;    resources: mem_mb=10000, disk_mb=7855, tmpdir=/tmp, threads=1, runtime_min=120&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I doubt this is how it should be, and wonder what I am doing wrong. Is there a better/correct way to do this.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance&lt;/p&gt;&#xA;" OwnerUserId="16204" LastEditorUserId="16204" LastEditDate="2023-10-05T11:17:23.507" LastActivityDate="2023-10-05T12:28:22.310" Title="Compare files with similar names using placeholders/wildcards in Snakemake" Tags="|python|snakemake|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21629" PostTypeId="1" AcceptedAnswerId="21630" CreationDate="2023-10-03T16:27:20.563" Score="1" ViewCount="532" Body="&lt;p&gt;I'm trying to use a Python script to plot with a rolling window using pandas and seaborn. This code worked for the longest time but now it's giving me an error that I don't know how to fix.&lt;/p&gt;&#xA;&lt;p&gt;Here is the code I'm using:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pandas as pd&#xA;import matplotlib.pyplot as plt&#xA;import numpy as np&#xA;import seaborn as sns&#xA;&#xA;heterozygosity_df = pd.read_csv(&amp;quot;file.tsv&amp;quot;, sep=&amp;quot;\t&amp;quot;, header=None).set_axis(['chr', 'pos', 'het'], axis=1, copy=False)&#xA;&#xA;heterozygosity_df['het'] = heterozygosity_df['het'].transform(lambda x: -1*(x-1))&#xA;&#xA;windows = [25, 50, 100, 200]&#xA;steps = [5]&#xA;&#xA;for window in windows:&#xA;    for step in steps:&#xA;        heterozygosity_ratio = {}&#xA;        for chr_heterozygosity_df in heterozygosity_df.groupby('chr'):&#xA;            heterozygosity_ratio[chr_heterozygosity_df[0]] = chr_heterozygosity_df[1].set_index('pos').rolling(window=window, step=step, center=True).mean()&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There is more after this but that's just to configure the resulting plot with matplotlib. The error I get when I use the above code is this:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &amp;quot;/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py&amp;quot;, line 368, in _prep_values&#xA;    values = ensure_float64(values)&#xA;             ^^^^^^^^^^^^^^^^^^^^^^&#xA;  File &amp;quot;pandas/_libs/algos_common_helper.pxi&amp;quot;, line 42, in pandas._libs.algos.ensure_float64&#xA;ValueError: could not convert string to float: 'chrI'&#xA;&#xA;The above exception was the direct cause of the following exception:&#xA;&#xA;Traceback (most recent call last):&#xA;  File &amp;quot;/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py&amp;quot;, line 485, in _apply_blockwise&#xA;    arr = self._prep_values(arr)&#xA;          ^^^^^^^^^^^^^^^^^^^^^^&#xA;  File &amp;quot;/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py&amp;quot;, line 370, in _prep_values&#xA;    raise TypeError(f&amp;quot;cannot handle this type -&amp;gt; {values.dtype}&amp;quot;) from err&#xA;TypeError: cannot handle this type -&amp;gt; object&#xA;&#xA;The above exception was the direct cause of the following exception:&#xA;&#xA;Traceback (most recent call last):&#xA;  File &amp;quot;plot_heterozygosity_window.py&amp;quot;, line 29, in &amp;lt;module&amp;gt;&#xA;    heterozygosity_ratio[chr_heterozygosity_df[0]] = chr_heterozygosity_df[1].set_index('pos').rolling(window=window, step=step, center=True).mean()&#xA;                                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^&#xA;  File &amp;quot;/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py&amp;quot;, line 2111, in mean&#xA;    return super().mean(&#xA;           ^^^^^^^^^^^^^&#xA;  File &amp;quot;/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py&amp;quot;, line 1507, in mean&#xA;    return self._apply(window_func, name=&amp;quot;mean&amp;quot;, numeric_only=numeric_only)&#xA;           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^&#xA;  File &amp;quot;/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py&amp;quot;, line 615, in _apply&#xA;    return self._apply_blockwise(homogeneous_func, name, numeric_only)&#xA;           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^&#xA;  File &amp;quot;/opt/homebrew/lib/python3.11/site-packages/pandas/core/window/rolling.py&amp;quot;, line 487, in _apply_blockwise&#xA;    raise DataError(&#xA;pandas.errors.DataError: Cannot aggregate non-numeric type: object&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have tried converting the 'chr' column directly from a string to a float:&#xA;&lt;code&gt;heterozygosity_df['chr'] = heterozygosity_df['chr'].astype(float)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have tried converting the entire dataframe to a float:&#xA;&lt;code&gt;heterozygosity_df = heterozygosity_df.astype(float, errors = 'raise')&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;But everything keeps giving me the &lt;code&gt;ValueError: could not convert string to float: 'chrI'&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Does someone know an easy way to fix this? Like I said, I have been using this code for months, and it's always worked - have they deprecated something in the pandas package to make it so you can't read in columns with strings and integers?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;For reference the dataframe is structured like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chrI  123  1&#xA;chrI  234  0&#xA;chrI  237  1&#xA;chrII 456  1 &#xA;chrII 567  1 &#xA;chrII 678  1 &#xA;...&#xA;chrXVI 123 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Where the first column designates the chromosome, the second column is the position, and the third column is the heterozygosity indicator.&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="16067" LastEditDate="2023-10-03T19:15:40.903" LastActivityDate="2023-10-03T20:25:34.827" Title="ValueError: count not convert string to float" Tags="|python|errors|pandas|seaborn|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21630" PostTypeId="2" ParentId="21629" CreationDate="2023-10-03T18:23:21.107" Score="1" Body="&lt;p&gt;The following would be useful,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print (heterozygosity_df.columns)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This looks like static typing issue within pandas. What I suspect is &lt;code&gt;heterozygosity_df.['chrI']&lt;/code&gt; is a column in the dataframe. What I think has happened is there's a mix of strings and floats within this column. pandas has set this as a &amp;quot;string&amp;quot; but you are wanting to perform numerical operations. Thus the solution is simply&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;print(heterozygosity_df.dtypes) # this should state 'chrI' is a &amp;quot;category&amp;quot; or &amp;quot;string&amp;quot;&#xA;heterozygosity_df['chrI'] = heterozygosity_df['chrI'].astype(float)&#xA;print(heterozygosity_df.dtypes)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you have multiple changes the syntax is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;heterozygosity_df = heterozygosity_df.astype({'chrI':'float', 'egColumn':'category'})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I suspect there will be other errors, e.g. the header is the first row of the data column. This is because again for pandas to automatically assign a column to a &amp;quot;string&amp;quot; means there must be a string value within the column.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;From the comments. I see whats happening. The easy solution is ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;replacement = {&#xA;    &amp;quot;chrI&amp;quot;: 1,&#xA;    &amp;quot;chrII&amp;quot;: 2,&#xA;    &amp;quot;chrIII&amp;quot;: 3,&#xA;    &amp;quot;chrIV&amp;quot;: 4,&#xA;    ....&#xA;}&#xA;&#xA;heterozygosity_df['chr'] = heterozygosity_df['chr'].str.replace(replacement, regex=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;From the comments ... good it works ... this is what I would have personally done ..&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;replacement = {&#xA;    &amp;quot;chrI&amp;quot;: 1,&#xA;    &amp;quot;chrII&amp;quot;: 2,&#xA;    &amp;quot;chrIII&amp;quot;: 3,&#xA;    &amp;quot;chrIV&amp;quot;: 4, # continue for all chromosomes&#xA;}&#xA;&#xA;heterozygosity_df = pd.read_csv(&amp;quot;file.tsv&amp;quot;, sep=&amp;quot;\t&amp;quot;, header=None).set_axis(['chr', 'pos', 'het'], axis=1, copy=False)&#xA;heterozygosity_df['chr'] = heterozygosity_df['chr'].str.replace(replacement, regex=True).astype('int')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Normally &lt;code&gt;str&lt;/code&gt; should be in place because when &lt;code&gt;'chr'&lt;/code&gt; is imported its an object. However, if it works thats the only thing that counts. &lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-10-03T20:25:34.827" LastActivityDate="2023-10-03T20:25:34.827" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="21631" PostTypeId="2" ParentId="21629" CreationDate="2023-10-03T20:22:52.803" Score="2" Body="&lt;p&gt;Ah! I finally got it to work! I used your command but I had to change it up a little bit to work with my data. The final command I used is this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;replacement = {&#xA;    &amp;quot;chrI&amp;quot;: 1,&#xA;    &amp;quot;chrII&amp;quot;: 2,&#xA;    &amp;quot;chrIII&amp;quot;: 3,&#xA;    &amp;quot;chrIV&amp;quot;: 4,&#xA;    &amp;quot;chrV&amp;quot;: 5,&#xA;    &amp;quot;chrVI&amp;quot;: 6,&#xA;    &amp;quot;chrVII&amp;quot;: 7,&#xA;    &amp;quot;chrVIII&amp;quot;: 8,&#xA;    &amp;quot;chrIX&amp;quot;: 9,&#xA;    &amp;quot;chrX&amp;quot;: 10,&#xA;    &amp;quot;chrXI&amp;quot;: 11,&#xA;    &amp;quot;chrXII&amp;quot;: 12,&#xA;    &amp;quot;chrXIII&amp;quot;: 13,&#xA;    &amp;quot;chrXIV&amp;quot;: 14,&#xA;    &amp;quot;chrXV&amp;quot;: 15,&#xA;    &amp;quot;chrXVI&amp;quot;: 16,&#xA;    &amp;quot;chrmt&amp;quot;: 17&#xA;}&#xA;&#xA;heterozygosity_df['chr'] = heterozygosity_df['chr'].replace(replacement, regex=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And with that I was able to generate plots showing all of the chromosomes! (I have to figure out how to change the axis labels from 1,2,3,4...10 back to the chromosome but that's a future problem). Thank you so much for all your help!!&lt;/p&gt;&#xA;" OwnerUserId="16067" LastActivityDate="2023-10-03T20:22:52.803" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21632" PostTypeId="2" ParentId="21603" CreationDate="2023-10-03T23:01:51.743" Score="1" Body="&lt;p&gt;Writing a solution for anyone interested. The issue stems from a dosage error, some variant contains only heterozygous individuals and no homozygous ones needed for QTL calculation.&lt;/p&gt;&#xA;&lt;p&gt;I used plink2 to calculate genotypes for all variant using the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plink2  --pfile PATH \&#xA;     --geno-counts  \&#xA;     --out OUT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then removed variant that contains less than 10 Homozygous invidivuals (HOM_REF_CT &amp;lt; 10) using plink then was able to run tensorQTL interaction mode.&lt;/p&gt;&#xA;&lt;p&gt;Hope it helps.&lt;/p&gt;&#xA;" OwnerUserId="18258" LastActivityDate="2023-10-03T23:01:51.743" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21633" PostTypeId="2" ParentId="2242" CreationDate="2023-10-03T23:45:15.427" Score="3" Body="&lt;p&gt;I recently developed &lt;a href=&quot;https://github.com/alejandrogzi/bed2gff&quot; rel=&quot;nofollow noreferrer&quot;&gt;bed2gff&lt;/a&gt; to quickly convert .bed files to a gff3 format, a tool written in Rust. Could be of help here!&lt;/p&gt;&#xA;" OwnerUserId="18305" LastActivityDate="2023-10-03T23:45:15.427" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21634" PostTypeId="2" ParentId="21626" CreationDate="2023-10-04T02:42:09.517" Score="4" Body="&lt;p&gt;For us, R10.4.1 Flongle flow cells are running at reduced yield due to lower numbers of working pores (something like 10-20 pores instead of 30-60 pores for R9.4.1 flow cells). This doesn't really affect the stuff we're using Flongles for, because I make sure we've got plenty of headroom built in when designing experiments.&lt;/p&gt;&#xA;&lt;p&gt;Here are the yield assumptions I use for flow cells, which are basically the lower ranges that we've seen when using them:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Flongle - 200 Mb&lt;/li&gt;&#xA;&lt;li&gt;MinION - 5 Gb&lt;/li&gt;&#xA;&lt;li&gt;PromethION - 50 Gb&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For Flongle, this has been fine for us to do sequencing and assembly of 12 plasmids on one flow cell using the rapid barcoding kit, and the improved accuracy from the Kit14 + R10.4.1 combination makes reliable assembly easier (especially where homopolymers are involved). I expect it will work similarly well for amplicon sequencing.&lt;/p&gt;&#xA;&lt;p&gt;Error rates are pretty similar across platforms, because it's the pore that matters, not the package. Here's a comparison plot of mapped accuracy vs predicted accuracy (from the basecaller) that I did recently on a TrackIt 1kb Plus DNA ladder, with bead cleanup using LFB, run on a PromethION flow cell with the V14 ligation kit (sequenced on a P2 Solo), then called using the new bacterial model. The reads represented in this plot have been filtered to only keep (at most) the longest 2000 matches for each target. The run produced 1,128,671 reads (1.95 Gb) over 1.5 hours:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/u8Bn7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/u8Bn7.png&quot; alt=&quot;Predicted vs mapped quality distribution(PromethION), demonstrating that the mapped quality is usually higher than the predicted quality from the base calling model&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here's a comparison plot of mapped accuracy vs predicted accuracy (from the basecaller) for the same DNA ladder sample run on a Flongle flow cell (sequenced on a MinION Mk1b), with bead cleanup using SFB, also taking at most 2000 longest sequences from each mapping. The run produced 465,144 reads (191 Mb) over 24 hours:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QX80C.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QX80C.png&quot; alt=&quot;Predicted vs mapped quality distribution (Flongle), demonstrating that the mapped quality is usually higher than the predicted quality from the base calling model&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The predicted accuracy is taken from the basecaller (i.e. fastq quality string), and is calculated by summing up the quality values after converting to a linear representation, then dividing by the number of bases and converting back to a log representation:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sub getMeanQual {&#xA;  my ($qual) = @_;&#xA;  if(length($qual) == 0){&#xA;    return(0);&#xA;  }&#xA;  my $qBase = 33;&#xA;  my @qspl = split(&amp;quot;&amp;quot;, $qual);&#xA;  my $qualTotal = 0;&#xA;  my $qualCount = 0;&#xA;  foreach my $qual (@qspl) {&#xA;    $qualTotal += 10 ** (-(ord($qual) - $qBase) / 10);&#xA;    $qualCount++;&#xA;  }&#xA;  return(int(-10 * log($qualTotal / $qualCount) / log(10)));&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;According to the trained models that LAST generated, there's not much different between the called reads between Flongle and PromethION flow cells. Here's the PromethION base mismatch matrix (q45 SNPs):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# substitution percent identity: 99.9971&#xA;#last -Q 1&#xA;#last -t4.38507&#xA;#last -a 23&#xA;#last -A 19&#xA;#last -b 2&#xA;#last -B 4&#xA;#last -S 1&#xA;# score matrix (query letters = columns, reference letters = rows):&#xA;       A      C      G      T&#xA;A      6    -45    -35    -46&#xA;C    -45      6    -45    -44&#xA;G    -33    -45      6    -46&#xA;T    -46    -45    -46      6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And here's the Flongle base mismatch matrix (q41 SNPs):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# substitution percent identity: 99.9922&#xA;#last -Q 1&#xA;#last -t4.36483&#xA;#last -a 20&#xA;#last -A 20&#xA;#last -b 4&#xA;#last -B 5&#xA;#last -S 1&#xA;# score matrix (query letters = columns, reference letters = rows):&#xA;       A      C      G      T&#xA;A      6    -45    -36    -46&#xA;C    -45      6    -44    -45&#xA;G    -32    -45      6    -46&#xA;T    -46    -45    -46      6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So I guess you could say that the Flongle has slightly lower accuracy, but I wouldn't call that difference significant for any practical purposes.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-10-18T13:52:48.600" LastActivityDate="2023-10-18T13:52:48.600" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21635" PostTypeId="1" CreationDate="2023-10-04T08:28:24.790" Score="2" ViewCount="52" Body="&lt;p&gt;I'm currently pondering on the best approach to handle and standardize variant IDs within our department to mitigate the challenges associated with rsids and the potential ambiguity of just using chromosomal positions.&lt;/p&gt;&#xA;&lt;p&gt;The GWAS glue (&lt;a href=&quot;https://mrcieu.github.io/gwasglue2/articles/Strategy.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://mrcieu.github.io/gwasglue2/articles/Strategy.html&lt;/a&gt;) packagae, state oa method where a variant ID is crafted by integrating the chromosomal position (chr), effect allele (ea), and non-effect allele (nea). It assumes the format: chr:pos_ea_nea. For alleles that exceed 10 characters, like certain indels, they apply the murmur32 hashing algorithm from the R digest package.&lt;/p&gt;&#xA;&lt;p&gt;Given our objectives:&lt;/p&gt;&#xA;&lt;p&gt;We likely prefer not to have excessively lengthy IDs.&#xA;Yet, using hashed IDs for certain long variants might be perplexing for some who aren't informed about the approach.&#xA;Hashing every ID could yield consistency, but perhaps might not be as intuitive for regular use.&#xA;I'm keen on collecting insights:&lt;/p&gt;&#xA;&lt;p&gt;Thoughts on the described method from Bristol?&#xA;What might be the maximum length for an indel we could encounter?&#xA;Is there a broader effort underway to address this concern?&#xA;Would adopting the Bristol approach set a potential standard, or should we explore or develop alternative methods? For example should the genome build be built into the id? or species?&#xA;I appreciate the community's feedback as we strive for clarity and efficiency in our variant naming conventions.&lt;/p&gt;&#xA;&lt;p&gt;For clarity:&#xA;I want to store data on variants. For example GWAS summary statistics.&#xA;I want to be able to match variants from differnt studies so I want to have an identifier for the variants.&#xA;I could use rs numbers&#xA;I could use chr:pos&#xA;I could use chr:pos:ref:alt&#xA;I could use chr:pos:effect:noneffect&#xA;I could use chr:pos:population:effect:noneffect&#xA;or studyid:chr:pos:populatoin:effect:noneffect&#xA;or etc&lt;/p&gt;&#xA;&lt;p&gt;There are pros and cons for each of these but I would like to here what people on here think these are.&#xA;I am also intrested in is there work out there to do this already. e.g. to update the idea of rsids to consider multiple alleles and effects in populations?&lt;/p&gt;&#xA;" OwnerUserId="228" LastEditorUserId="228" LastEditDate="2023-10-04T12:44:43.187" LastActivityDate="2023-10-30T14:28:49.170" Title="Standardizing Variant IDs best practise" Tags="|identifiers|variants|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21636" PostTypeId="1" AcceptedAnswerId="21638" CreationDate="2023-10-04T11:42:39.473" Score="2" ViewCount="57" Body="&lt;p&gt;I recently tried using eggNOG-mapper for pan-genome and core genome analysis of files from OrthoFinder, specifically the 'Orthogroup_Sequences' which contains a huge list of protein files in fasta format. As instructed, the folder was compressed into a tar.gz zip file and uploaded. If the upload was successful, an email is sent to your mail ID where you can check the details of your job and decide whether to START the job or not. Every time I START the job, it fails within a few minutes. Initially I presumed that it was due to the file size and hence reduced the number of sequences submitted, but in vain. Can anyone help me?&lt;/p&gt;&#xA;&lt;p&gt;eggNOG-mapper : &lt;a href=&quot;http://eggnog-mapper.embl.de/&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://eggnog-mapper.embl.de/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17392" LastActivityDate="2023-10-15T08:18:38.330" Title="Error with eggNOG-mapper jobs" Tags="|orthologues|orthofinder|embl|eggnog|pangenome|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21637" PostTypeId="2" ParentId="21628" CreationDate="2023-10-04T11:46:22.997" Score="1" Body="&lt;p&gt;Using &lt;code&gt;expand&lt;/code&gt; in your rule's &lt;code&gt;input:&lt;/code&gt;  causes snakemake to aggregate all vcfs into one list and leads to your undesired behaviour. By simply removing the &lt;code&gt;expand&lt;/code&gt; calls from the &lt;code&gt;input:&lt;/code&gt; you should get your desired result.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#the-expand-function&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://snakemake.readthedocs.io/en/stable/snakefiles/rules.html#the-expand-function&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3818" LastActivityDate="2023-10-04T11:46:22.997" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21638" PostTypeId="2" ParentId="21636" CreationDate="2023-10-04T14:51:36.097" Score="2" Body="&lt;p&gt;There is a known bug between eggNOG-mapper2 and eggNOG6. This is described here:&#xA;&lt;a href=&quot;https://github.com/eggnogdb/eggnog-mapper/issues/428&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/eggnogdb/eggnog-mapper/issues/428&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In your case, I'm not sure on what strategy you are deploying after OrthoFinder, but there's a chance it's not solvable at present and the best place for information is the eggnog GitHub.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-10-04T14:51:36.097" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21639" PostTypeId="1" CreationDate="2023-10-04T20:04:31.223" Score="2" ViewCount="69" Body="&lt;p&gt;I am working on a new sponge genome and I have produced the genomescope plot from the 21-mer kmer frequencies. I have hard time interpreting the plot. Can someone please help me?&#xA;Thank you.&lt;a href=&quot;https://i.stack.imgur.com/uiiC2.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uiiC2.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18311" LastActivityDate="2023-10-05T19:57:19.880" Title="How to interpret a GenomeScope plot?" Tags="|ngs|genome|sequence-annotation|illumina|genome-sequencing|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21641" PostTypeId="2" ParentId="21601" CreationDate="2023-10-05T00:47:56.850" Score="2" Body="&lt;p&gt;Use the SRR accessions, not the sample-IDs you are using.&lt;/p&gt;&#xA;&lt;p&gt;With entrez e-utils, I found the SRR of GSM2683458:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;esearch -db sra -query GSM2683458 | efetch -format xml | xtract -pattern IDENTIFIERS -if PRIMARY_ID -contains SRR -element PRIMARY_ID&lt;/code&gt; is &lt;code&gt;SRR5755657&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;For GSM3603268, using &lt;code&gt;esearch -db sra -query GSM3603268 | efetch -format xml | xtract -pattern IDENTIFIERS -if PRIMARY_ID -contains SRR -element PRIMARY_ID&lt;/code&gt;, the SRR is: &lt;code&gt;SRR8571942&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then just do &lt;code&gt;fasterq-dump SRR5755657; fasterq-dump SRR8571942&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3381" LastActivityDate="2023-10-05T00:47:56.850" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21642" PostTypeId="2" ParentId="21628" CreationDate="2023-10-05T12:28:22.310" Score="0" Body="&lt;p&gt;After some trial and error, I fixed it with the expand function only in the rule all and removing the expand from the input in the rule happy_rtg_array (thanks to @Pallie), and by changing the output file paths:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SAMPLE_SET = ['12345', '56789', '54321']&#xA;ID_SET = ['EF_S13', 'MF_S5', 'EF_S11', 'MF_S3', 'EF_S10', 'MF_S2']&#xA;&#xA;rule all:&#xA;    input:&#xA;        array = expand(&amp;quot;results/{sample}/{samp_id}/summary.csv&amp;quot;, zip, sample = SAMPLE_SET, samp_id = ID_SET)&#xA;&#xA;&#xA;rule happy_rtg_array:&#xA;    &amp;quot;&amp;quot;&amp;quot; Normalizes and compares vcf files with array samples&amp;quot;&amp;quot;&amp;quot;&#xA;    input:&#xA;        query_vcf = &amp;quot;_&amp;quot;.join([&amp;quot;WESP_run/results/variant_calling/{sample}&amp;quot;,&amp;quot;{samp_id}.vcf&amp;quot;]),&#xA;        truth_vcf = os.path.join(config[&amp;quot;truth_dir_vcf_gsa&amp;quot;],&amp;quot;{sample}.vcf.gz&amp;quot;),&#xA;    output: &#xA;        &amp;quot;WESPValidation/results/{sample}/{samp_id}/summary.csv&amp;quot;&#xA;[more code]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Basically, I replaced all the underscores with slashes. For some reason these underscores are messing up the wildcards when trying to connect them with it. Maybe someone more knowledgeable knows why this is.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[Tue Oct  5 12:24:10 2023]&#xA;rule happy_rtg_array:&#xA;    input: 12345_EF_S13.vcf&#xA;    output: results/12345/EF_S13/summary.csv&#xA;    log: happy_rtg_array/12345_EF_S13.log&#xA;    jobid: 6&#xA;    reason: Missing output files: WESPValidation/results/12345/EF_S13/summary.csv&#xA;    wildcards: sample=12345, samp_id=EF_S13&#xA;    threads: 10&#xA;    resources: mem_mb=10000, disk_mb=7855, tmpdir=/tmp, threads=1, runtime_min=120&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16204" LastActivityDate="2023-10-05T12:28:22.310" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21644" PostTypeId="2" ParentId="21639" CreationDate="2023-10-05T19:57:19.880" Score="1" Body="&lt;p&gt;Based on the numbers shown above the plot, and the appearance of the curve, this looks like reads from a diploid genome approximately 113Mb in length that was sequenced at around 174X coverage.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-10-05T19:57:19.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21646" PostTypeId="2" ParentId="7019" CreationDate="2023-10-06T15:59:59.727" Score="1" Body="&lt;p&gt;Within the R/Bioconductor ecosystem you could use the &lt;code&gt;DECIPHER&lt;/code&gt; package (&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/DECIPHER.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/html/DECIPHER.html&lt;/a&gt;) to align your sequences and then continue with either &lt;code&gt;ape&lt;/code&gt; or &lt;code&gt;phangorn&lt;/code&gt;  (&lt;a href=&quot;https://klausvigo.github.io/phangorn/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://klausvigo.github.io/phangorn/&lt;/a&gt;) packages to compute a tree. You can then visualize your tree using for example the &lt;code&gt;ggtree&lt;/code&gt; package (&lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/ggtree.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/html/ggtree.html&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="17280" LastActivityDate="2023-10-06T15:59:59.727" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21647" PostTypeId="1" CreationDate="2023-10-06T16:09:29.353" Score="2" ViewCount="56" Body="&lt;p&gt;I am looking for methods to compare substitution matrices to determine which one is better for a given set of data / type of protein.&lt;br /&gt;&#xA;Do you know such methods / tools?&lt;/p&gt;&#xA;&lt;p&gt;e.g.  BLOSUM62 vs PHAT for 50k - 100k+ sequences&lt;/p&gt;&#xA;" OwnerUserId="9651" LastEditorUserId="3848" LastEditDate="2023-10-09T14:29:46.063" LastActivityDate="2023-10-09T14:29:46.063" Title="Compare substitution matrices" Tags="|phylogenetics|phylogeny|matrix|substitution|amino-acids|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21648" PostTypeId="2" ParentId="21432" CreationDate="2023-10-06T17:08:15.137" Score="2" Body="&lt;p&gt;I recognise my answer is late but I hope it helps. The following has no emperical proof, just anecdotal from working with bacterial genomes:&lt;/p&gt;&#xA;&lt;p&gt;I'm not sure what kind of genome you're working with, but from my work with bacteria, we often delete scaffolds shorter than 1000 - 3000 bp because these are considered &amp;quot;noise&amp;quot;. Essentially the scaffolds are too small to gather any particularly useful information from as they may only carry a single gene (or a handful of little genes). If you've added in long read data with your short, half the time those small scaffolds are just artifacts from sequencing errors.&lt;/p&gt;&#xA;" OwnerUserId="18327" LastEditorUserId="18327" LastEditDate="2023-10-06T18:11:54.513" LastActivityDate="2023-10-06T18:11:54.513" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21649" PostTypeId="2" ParentId="21285" CreationDate="2023-10-07T05:17:27.253" Score="1" Body="&lt;p&gt;This was part of a custom liftover function which handled multiple alleles and reverse compliments. I do not know if it will be useful. I believe there was a separate thing somewhere with lookup that I may add. It handled apparent non-1:1 mappings in addition to platform-specific &amp;quot;rsIDs.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;converter = get_lifter(current, target)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;def convert_coordinates(input_file, output_file, converter):&#xA;        line_count = 0&#xA;        skipped = 0&#xA;        strand_flips = 0&#xA;        with open(input_file, 'r') as infile, open(output_file, 'w') as outfile:&#xA;            for line in infile:&#xA;                # Skip headers&#xA;                if line.startswith(&amp;quot;#&amp;quot;):&#xA;                    outfile.write(line)&#xA;                    continue&#xA;&#xA;                # VCF files are tab-separated&#xA;                parts = line.strip().split('\t')&#xA;&#xA;                # Get the chromosome and position&#xA;                chrom = parts[0]&#xA;                pos = int(parts[1])&#xA;&#xA;                # Convert the coordinates&#xA;                try:&#xA;                    new_coords = converter.query(chrom, pos)&#xA;                except KeyError:&#xA;                    #print(f&amp;quot;Chromosome {chrom} not found in the liftover chain file.&amp;quot;)&#xA;                    skipped += 1&#xA;                    continue&#xA;&#xA;                # If there's no new coordinates, we skip this variant&#xA;                if not new_coords:&#xA;                    skipped += 1&#xA;                    continue&#xA;&#xA;                # Get the new chromosome, position and strand&#xA;                new_chrom, new_pos, strand = new_coords[0]&#xA;&#xA;                # If the new strand is negative, reverse complement the REF and ALT alleles&#xA;                if strand == '-':&#xA;                    parts[3] = reverse_complement(parts[3])&#xA;                    parts[4] = reverse_complement(parts[4])&#xA;                    strand_flips += 1&#xA;&#xA;                # Replace the old chromosome and position with the new ones&#xA;                parts[0] = new_chrom&#xA;                parts[1] = str(new_pos)&#xA;&#xA;                # Write the line back into the file&#xA;                outfile.write('\t'.join(parts))&#xA;                outfile.write('\n')&#xA;&#xA;                # Print a progress statement every 1000 lines&#xA;                line_count += 1&#xA;                if line_count % 100000 == 0:&#xA;                    print(f&amp;quot;Processed {line_count} lines&amp;quot;)&#xA;&#xA;        # Calculate and print the percentage of variants skipped&#xA;        percentage_skipped = (skipped / line_count) * 100&#xA;        percentage_flips = (strand_flips / line_count) * 100&#xA;        print(f&amp;quot;\nFinished processing. Out of {line_count} total lines:&amp;quot;)&#xA;        print(f&amp;quot;{percentage_skipped}% of variants were skipped due to missing chromosomal data.&amp;quot;)&#xA;        print(f&amp;quot;{percentage_flips}% of variants required a strand flip.&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17430" LastEditorUserId="17430" LastEditDate="2023-10-07T05:39:56.463" LastActivityDate="2023-10-07T05:39:56.463" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21650" PostTypeId="2" ParentId="21367" CreationDate="2023-10-07T05:56:53.803" Score="0" Body="&lt;p&gt;This would be a hard feature to support natively, because some PRSs have 10 million+ SNPs.&lt;/p&gt;&#xA;&lt;p&gt;There is a tool that does this, but I don't want to recommend it because it will require debugging their source code upon running, and it is less accurate than what you are using here.&lt;/p&gt;&#xA;&lt;p&gt;I suggest doing --scorefile &amp;quot;my_custom_scores/*.txt&amp;quot; if you are interested in only a few variants. Add only one score to this file. This may cause some error especially if some of your samples do not have all positions defined.&lt;/p&gt;&#xA;&lt;p&gt;If you have more variants, there is a way to combine multiple scores in a single scorefile instead of a directory.&lt;/p&gt;&#xA;" OwnerUserId="17430" LastActivityDate="2023-10-07T05:56:53.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21651" PostTypeId="1" CreationDate="2023-10-08T20:20:41.253" Score="0" ViewCount="24" Body="&lt;p&gt;Does anybody know if there is a way to find out what are the magnification levels of the TCGA pathology slides? Are they all 40x? I can't seem to find a label of that anywhere&lt;/p&gt;&#xA;" OwnerUserId="18274" LastActivityDate="2023-10-08T20:20:41.253" Title="TCGA pathology slides magnification" Tags="|deep-learning|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21652" PostTypeId="2" ParentId="21164" CreationDate="2023-10-08T21:37:50.027" Score="2" Body="&lt;p&gt;This sounds like you are wanting to perform a machine learning calculation. However, for this question leveraging KEGG, Kyoto Encyclopaedia of Genome and Gene and building an SQL database around it is much more effective. It represents around 60% of all functional genes. KEGG is the best gene annotation out there IMO, particularly for flavonoids etc... Fantastic and all their linking pathways, everything links to everything, just amazing. KEGG represents an enormous volume of work, much of it being manually crunching publications for each gene.&lt;/p&gt;&#xA;&lt;p&gt;The entire database of KEGG keys is downloadable. Connecting a gene to a class of function is easy via KEGG the relationships are defined its just SQL-ing the whole thing together (querying all the tables). Again, it's totally and completely brilliant. Sure they'll be stuff out there has SQL-ed KEGG key tables, but if your a developer its better to build it independently - this way you're certain whats in there and what its doing.&lt;/p&gt;&#xA;&lt;p&gt;However, linking raw sequence data to a KEGG gene code is more challenging. &lt;a href=&quot;https://www.sciencedirect.com/science/article/pii/S002228361500649X&quot; rel=&quot;nofollow noreferrer&quot;&gt;lastKOALA and GhostKOALA&lt;/a&gt; is ... yeah well an approach no more. There's a main stream approach but with current limitations in implementation. KEGG however isn't limiting, it's linking a gene to a KEGG id is the challenge. Getting hold of the raw sequence for KEGG would be great, fantastic in fact, but I'm yet to find it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-10-08T22:21:27.300" LastActivityDate="2023-10-08T22:21:27.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21654" PostTypeId="1" CreationDate="2023-10-09T07:00:34.937" Score="2" ViewCount="495" Body="&lt;p&gt;I am trying EnrichKEGG on a few gene samples of genes to check it works before inputting the full genome:&lt;/p&gt;&#xA;&lt;p&gt;This is my data input as test.csv&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;id,fc&#xA;AJAP_14870,-0.04&#xA;AJAP_14875,0.32&#xA;AJAP_14880,-0.06&#xA;AJAP_14885,0.15&#xA;AJAP_14890,-0.52&#xA;AJAP_14895,-0.39&#xA;AJAP_14900,0.16&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;geneList = test[,2]&#xA;names(geneList) = as.character(test[,1])&#xA;geneList = sort(geneList, decreasing = TRUE)&#xA;&#xA;mygene &amp;lt;- names(geneList)[abs(geneList) &amp;gt; 2]&#xA;&#xA;kk &amp;lt;- enrichKEGG(gene         = mygene,&#xA;                 organism     = 'aja',&#xA;                 pvalueCutoff = 0.05)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There are no spaces in the gene names, I tested via&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;print(head(test$ID), quote=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I get the error&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;--&amp;gt; No gene can be mapped....&#xA;--&amp;gt; Expected input gene ID:&#xA;--&amp;gt; return NULL...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also tried installing through other means&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;getOption(&amp;quot;clusterProfiler.download.method&amp;quot;)&#xA;#[1] &amp;quot;libcurl&amp;quot;&#xA;install.packages(&amp;quot;R.utils&amp;quot;)&#xA;library(R.utils)&#xA;R.utils::setOption(&amp;quot;clusterProfiler.download.method&amp;quot;,&amp;quot;auto&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but that gives me the same error when I run the code. Any ideas how to fix this?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="18341" LastEditorUserId="650" LastEditDate="2023-10-09T14:53:38.617" LastActivityDate="2023-10-09T16:25:51.287" Title="How to fix &quot;No gene can be mapped, Expected input gene ID, return NULL...&quot; error in clusterProfiler?" Tags="|r|bioconductor|clusterprofiler|" AnswerCount="2" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="21655" PostTypeId="2" ParentId="21654" CreationDate="2023-10-09T07:28:28.173" Score="1" Body="&lt;p&gt;&lt;code&gt;enrichKEGG()&lt;/code&gt; expects entrez gene IDs so you will need to convert your gene identifiers to entrez identifiers first.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: Please see the answer by @Basti.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="3541" LastEditDate="2023-10-09T09:40:25.373" LastActivityDate="2023-10-09T09:40:25.373" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21656" PostTypeId="2" ParentId="21654" CreationDate="2023-10-09T09:19:51.687" Score="3" Body="&lt;p&gt;This issue may arise with earlier versions of clusterProfiler.&lt;/p&gt;&#xA;&lt;p&gt;A solution is to re-install clusterProfiler : &lt;code&gt;BiocManager::install(&amp;quot;clusterProfiler&amp;quot;, force = TRUE)&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;Pay attention that updating your R or Bioconductor (and restart your R session) version may be needed as noted in the comments&lt;/p&gt;&#xA;" OwnerUserId="14639" LastEditorUserId="650" LastEditDate="2023-10-09T16:25:51.287" LastActivityDate="2023-10-09T16:25:51.287" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21657" PostTypeId="2" ParentId="21164" CreationDate="2023-10-09T12:41:58.123" Score="1" Body="&lt;p&gt;I've used Augustus stand alone (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1160219/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1160219/&lt;/a&gt;) to find genes in a given eukaryotic genome. I agree you can then use things like KofamSCAN to categorize the genes against the KEGG database. If you want to compare against the NCBI database, diamond blast (&lt;a href=&quot;https://github.com/bbuchfink/diamond&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/bbuchfink/diamond&lt;/a&gt;) is probably the quickest way to go :)&lt;/p&gt;&#xA;" OwnerUserId="18327" LastActivityDate="2023-10-09T12:41:58.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21658" PostTypeId="2" ParentId="21647" CreationDate="2023-10-09T14:20:10.240" Score="2" Body="&lt;p&gt;The technique is phylogenetics is called likelihood ratio and for a regular amino acid matrix is 2Δ&lt;em&gt;ln&lt;/em&gt; which is assessed on Chi&lt;sup&gt;2&lt;/sup&gt; tables where Δ&lt;em&gt;ln&lt;/em&gt; is difference in likelihood between model 1 and model 2. The degrees of freedom hmmmm... is the number of parameters that are constrained, this involves the number of taxa minus the number of parameter differences between the models. This makes performing 50 000 taxa ... tricky because the degrees of freedom will be outside the Chi&lt;sup&gt;2&lt;/sup&gt; table.&lt;/p&gt;&#xA;&lt;p&gt;iqtree will perform a tree and provide a likelihood for 50+K taxa. There are two limitations,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;No PHAT model&lt;/li&gt;&#xA;&lt;li&gt;degrees of freedom (as mentioned)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Overcoming df is simply to say whatever is the lowest likelihood is the best model and simply state Δ&lt;em&gt;ln&lt;/em&gt; without bothering with significance testing. If the difference between models is large you'll get away with that (needs to be big) and for 50+K taxa it will be need to be huge.&lt;/p&gt;&#xA;&lt;p&gt;Getting PHAT (the model below the ---- might be PHAT) into iqtree is certainly doable because of &lt;a href=&quot;https://academic.oup.com/sysbio/article/70/5/1046/6146362&quot; rel=&quot;nofollow noreferrer&quot;&gt;QMaker&lt;/a&gt;. However, how this is done I don't know, because I never done this calculation with iqtree, you'll need to read the manual. For traditional packages such as PAML it's easy ... you make a tree then specific the custom amino acid model and it spits out the likelihood. For the number of taxa involved, PAML is not possible, it's impossibly big for PAML, even 1000 is a lot of PAML.&lt;/p&gt;&#xA;&lt;p&gt;iqtree does have LG, WAG and JTT amino acid models and will almost certainly have all the BLOSUM matrices and you just specify those at the command line. Again a comparison with PHAT needs to specify a custom matrix and I don't know how to do that in iqtree. The only thing I know is it must be doable.&lt;/p&gt;&#xA;&lt;p&gt;iqtree is the most efficient of the maximum likelihood phylogenetic methods by a very long way. Hence for dealing with 50+K is why I'm recommending it here.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; The theory about whats going on and why the whole thing needs phylogenetics is very complicated. Basically its an entire lecture on phylogenetic theory.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;This I believe is PHAT (needs checking)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/biopython/biopython/pull/283&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/biopython/biopython/pull/283&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note the code is copied and pasted from the above link, therefore it should be strictly regarded as a quote for copyright purposes.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;matrix = {&#xA;('W', 'F') :  0.0, ('L', 'R') : -6.0, ('I', 'I') :  5.0, ('Q', 'Q') :  9.0,&#xA;('W', 'N') : -5.0, ('V', 'I') :  3.0, ('H', 'T') : -4.0, ('H', 'P') : -6.0,&#xA;('W', 'V') : -4.0, ('Q', 'E') :  1.0, ('W', 'R') : -7.0, ('Q', 'A') : -3.0,&#xA;('H', 'H') : 11.0, ('H', 'D') : -1.0, ('L', 'N') : -3.0, ('Y', 'M') : -2.0,&#xA;('Y', 'I') : -3.0, ('Y', 'E') : -2.0, ('E', 'S') : -3.0, ('Y', 'A') : -3.0,&#xA;('Y', 'Y') : 11.0, ('T', 'C') : -1.0, ('E', 'C') : -7.0, ('Y', 'Q') :  0.0,&#xA;('E', 'G') : -3.0, ('V', 'A') :  1.0, ('C', 'C') :  7.0, ('M', 'R') : -6.0,&#xA;('P', 'T') : -4.0, ('V', 'E') : -5.0, ('P', 'P') : 13.0, ('I', 'T') : -1.0,&#xA;('K', 'S') : -5.0, ('R', 'G') : -5.0, ('I', 'P') : -4.0, ('R', 'C') : -8.0,&#xA;('A', 'T') :  0.0, ('K', 'K') :  5.0, ('A', 'P') :  3.0, ('V', 'M') :  1.0,&#xA;('I', 'D') : -5.0, ('K', 'C') :-10.0, ('K', 'G') : -5.0, ('R', 'S') : -6.0,&#xA;('F', 'Q') : -2.0, ('F', 'A') : -1.0, ('V', 'V') :  4.0, ('M', 'N') : -2.0,&#xA;('F', 'E') : -5.0, ('D', 'N') :  2.0, ('F', 'I') :  0.0, ('F', 'M') :  0.0,&#xA;('M', 'S') : -2.0, ('S', 'S') :  6.0, ('L', 'Q') : -3.0, ('W', 'E') : -7.0,&#xA;('W', 'A') : -4.0, ('W', 'M') : -4.0, ('H', 'S') : -2.0, ('W', 'I') : -4.0,&#xA;('S', 'C') :  1.0, ('L', 'A') : -1.0, ('L', 'E') : -5.0, ('W', 'Q') :  1.0,&#xA;('H', 'G') : -4.0, ('Q', 'N') :  2.0, ('H', 'C') : -7.0, ('L', 'M') :  2.0,&#xA;('W', 'Y') :  1.0, ('Y', 'N') :  2.0, ('E', 'P') : -5.0, ('Y', 'F') :  4.0,&#xA;('E', 'T') : -5.0, ('A', 'A') :  5.0, ('I', 'N') : -3.0, ('G', 'A') :  1.0,&#xA;('Y', 'V') : -3.0, ('E', 'D') :  6.0, ('W', 'H') : -3.0, ('Y', 'R') : -6.0,&#xA;('M', 'Q') : -1.0, ('P', 'S') : -3.0, ('R', 'H') : -4.0, ('A', 'C') :  1.0,&#xA;('R', 'D') : -7.0, ('K', 'P') : -4.0, ('L', 'D') : -5.0, ('K', 'T') : -6.0,&#xA;('V', 'N') : -3.0, ('M', 'A') : -1.0, ('K', 'H') : -5.0, ('V', 'R') : -7.0,&#xA;('P', 'C') : -8.0, ('M', 'E') : -5.0, ('A', 'S') :  2.0, ('T', 'T') :  3.0,&#xA;('R', 'T') : -6.0, ('I', 'G') : -2.0, ('R', 'P') : -7.0, ('K', 'D') : -5.0,&#xA;('I', 'C') : -3.0, ('F', 'R') : -7.0, ('F', 'V') : -1.0, ('L', 'C') : -2.0,&#xA;('F', 'F') :  6.0, ('D', 'A') : -5.0, ('F', 'N') : -1.0, ('W', 'D') : -7.0,&#xA;('L', 'P') : -5.0, ('Q', 'S') : -1.0, ('N', 'C') : -2.0, ('N', 'G') : -1.0,&#xA;('H', 'N') :  4.0, ('W', 'T') : -7.0, ('Q', 'G') : -2.0, ('W', 'P') : -6.0,&#xA;('Q', 'C') : -5.0, ('N', 'S') :  1.0, ('L', 'H') : -4.0, ('L', 'L') :  4.0,&#xA;('G', 'T') : -1.0, ('M', 'M') :  6.0, ('G', 'P') : -3.0, ('Y', 'K') : -4.0,&#xA;('Y', 'G') : -3.0, ('Y', 'C') : -1.0, ('E', 'A') : -5.0, ('E', 'E') : 12.0,&#xA;('Y', 'S') : -2.0, ('M', 'P') : -5.0, ('V', 'C') : -2.0, ('M', 'T') :  0.0,&#xA;('V', 'G') : -2.0, ('R', 'E') : -6.0, ('V', 'K') : -8.0, ('K', 'Q') : -1.0,&#xA;('R', 'A') : -6.0, ('I', 'R') : -6.0, ('N', 'A') : -2.0, ('V', 'S') : -2.0,&#xA;('M', 'D') : -5.0, ('M', 'H') : -4.0, ('K', 'A') : -7.0, ('R', 'Q') : -2.0,&#xA;('K', 'E') : -4.0, ('F', 'S') : -2.0, ('I', 'K') : -7.0, ('D', 'P') : -5.0,&#xA;('D', 'T') : -5.0, ('I', 'M') :  3.0, ('F', 'C') :  0.0, ('W', 'L') : -3.0,&#xA;('F', 'G') : -2.0, ('F', 'K') : -7.0, ('F', 'T') : -2.0, ('D', 'D') : 12.0,&#xA;('Q', 'T') : -3.0, ('W', 'G') : -5.0, ('Q', 'P') : -3.0, ('W', 'C') : -4.0,&#xA;('W', 'K') : -8.0, ('H', 'Q') :  2.0, ('Q', 'D') :  0.0, ('W', 'W') : 11.0,&#xA;('V', 'L') :  1.0, ('L', 'G') : -2.0, ('W', 'S') : -5.0, ('L', 'K') : -7.0,&#xA;('N', 'P') : -4.0, ('H', 'E') : -1.0, ('N', 'T') : -1.0, ('H', 'A') : -3.0,&#xA;('Y', 'L') : -2.0, ('Y', 'H') :  3.0, ('G', 'S') :  1.0, ('Y', 'D') : -4.0,&#xA;('V', 'Q') : -3.0, ('L', 'T') : -1.0, ('G', 'G') :  9.0, ('G', 'C') : -2.0,&#xA;('E', 'N') :  0.0, ('Y', 'T') : -3.0, ('Y', 'P') : -5.0, ('R', 'N') : -3.0,&#xA;('V', 'D') : -5.0, ('K', 'R') : -1.0, ('V', 'H') : -5.0, ('I', 'Q') : -3.0,&#xA;('V', 'P') : -4.0, ('M', 'C') : -2.0, ('K', 'N') : -2.0, ('V', 'T') :  0.0,&#xA;('M', 'G') : -1.0, ('T', 'S') :  1.0, ('I', 'E') : -5.0, ('M', 'K') : -6.0,&#xA;('I', 'A') :  0.0, ('N', 'N') : 11.0, ('R', 'R') :  9.0, ('F', 'P') : -5.0,&#xA;('L', 'I') :  2.0, ('I', 'S') : -2.0, ('D', 'S') : -4.0, ('L', 'S') : -2.0,&#xA;('I', 'H') : -5.0, ('F', 'D') : -5.0, ('D', 'C') : -7.0, ('F', 'H') : -2.0,&#xA;('D', 'G') : -2.0, ('F', 'L') :  1.0&#xA;}&#xA;&#xA;chars = [aa[0] for aa in matrix] + [aa[1] for aa in matrix]&#xA;chars = sorted(list(set(chars)))&#xA;&#xA;output = &amp;quot; ,%s\n&amp;quot; % &amp;quot;,&amp;quot;.join(chars)&#xA;&#xA;for i, aa1 in enumerate(chars):&#xA;    output += aa1&#xA;    for j, aa2 in enumerate(chars):&#xA;        if (aa1, aa2) in matrix:&#xA;            output += &amp;quot;,&amp;quot; + str(matrix[(aa1, aa2)])&#xA;        else:&#xA;            output += &amp;quot;,&amp;quot; + str(matrix[(aa2, aa1)])&#xA;    output += &amp;quot;\n&amp;quot;&#xA;print(output)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-10-09T14:26:00.857" LastActivityDate="2023-10-09T14:26:00.857" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21659" PostTypeId="1" CreationDate="2023-10-09T23:24:24.427" Score="4" ViewCount="117" Body="&lt;p&gt;I want a list of all variants,  i.e. sites which are known to vary between human to human. For example, it should ideally cover all sites in &lt;a href=&quot;http://ftp.1000genomes.ebi.ac.uk/vol1/ftp/data_collections/1000G_2504_high_coverage/working&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, but without samples.&lt;/p&gt;&#xA;&lt;p&gt;I don't want a giant reference panel with many samples, nor a .fa which has no major/minor allele information.&lt;/p&gt;&#xA;&lt;p&gt;I just want a file with the following information at each locus:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Location (chromosome position and chromosome)&lt;/li&gt;&#xA;&lt;li&gt;Bonus (not necessary): rsID&lt;/li&gt;&#xA;&lt;li&gt;Major allele (i.e., A, C, T, or G) at that position&lt;/li&gt;&#xA;&lt;li&gt;Minor allele at that position&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I'm not picky about how major/minor alleles are defined. Usually they are defined as more common in the general population vs. less common (loosely). Conforming to b37 or b38 would be ideal.&lt;/p&gt;&#xA;&lt;p&gt;I'm looking for one such file that doesn't have samples in it, but does contain all possible positions (not all VCF files have this)&lt;/p&gt;&#xA;&lt;p&gt;Where can I download something like this? I like working with VCFs but any format is good.&lt;/p&gt;&#xA;" OwnerUserId="17430" LastEditorUserId="73" LastEditDate="2023-10-11T01:03:43.183" LastActivityDate="2023-10-20T16:58:11.767" Title="Where to download a file with major and minor alleles at every position?" Tags="|vcf|data-download|rsid|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21660" PostTypeId="1" AcceptedAnswerId="21672" CreationDate="2023-10-10T04:08:50.603" Score="0" ViewCount="52" Body="&lt;p&gt;If so, by about how much? I want to know the downsides of skipping the phasing step on human microarray and WGS data.&lt;/p&gt;&#xA;" OwnerUserId="17430" LastEditorUserId="3694" LastEditDate="2023-10-10T08:35:56.740" LastActivityDate="2023-10-11T09:23:58.907" Title="Does phasing improve imputation quality?" Tags="|microarray|wgs|imputation|phasing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21661" PostTypeId="2" ParentId="21659" CreationDate="2023-10-10T07:10:35.107" Score="3" Body="&lt;p&gt;Sounds like you're looking for the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/projects/SNP/snp_summary.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbSNp&lt;/a&gt; files. You haven't specified a species, but you can download the data for human from the &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/snp/&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbSNP FTP site&lt;/a&gt;. Specifically, you want:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;wget https://ftp.ncbi.nlm.nih.gov/snp/organisms/human_9606/VCF/00-All.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This file has lines like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1   10848   rs1271898990    C   G   .   .   RS=1271898990;RSPOS=10848;dbSNPBuildID=151;SSR=0;SAO=0;VP=0x050000020005000002000100;GENEINFO=DDX11L1:100287102;WGT=1;VC=SNV;R5;ASP;TOPMED=0.99998407237512742,0.00001592762487257&#xA;1   10851   rs1340958514    G   A   .   .   RS=1340958514;RSPOS=10851;dbSNPBuildID=151;SSR=0;SAO=0;VP=0x050000020005000002000100;GENEINFO=DDX11L1:100287102;WGT=1;VC=SNV;R5;ASP;TOPMED=0.99996018093781855,0.00003981906218144&#xA;1   10854   rs1222740568    G   A,C .   .   RS=1222740568;RSPOS=10854;dbSNPBuildID=151;SSR=0;SAO=0;VP=0x050000020005000002000100;GENEINFO=DDX11L1:100287102;WGT=1;VC=SNV;R5;ASP;TOPMED=0.99967348369011213,0.00019113149847094,0.00013538481141692&#xA;1   10857   rs1487378566    C   G   .   .   RS=1487378566;RSPOS=10857;dbSNPBuildID=151;SSR=0;SAO=0;VP=0x050000020005000002000100;GENEINFO=DDX11L1:100287102;WGT=1;VC=SNV;R5;ASP;TOPMED=0.99933900356778797,0.00066099643221202&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, it sounds like you might be underestimating the complexity here. There isn't just one minor allele, many sites have multiple minor alleles. The example above shows one such multiallelic site (you can split these into separate lines using &lt;code&gt;bcftools&lt;/code&gt;). But you also have cases like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;1   10786   rs1464509424    CGGCGCA C   .   .   RS=1464509424;RSPOS=10787;dbSNPBuildID=151;SSR=0;SAO=0;VP=0x050000020005000002000200;GENEINFO=DDX11L1:100287102;WGT=1;VC=DIV;R5;ASP;TOPMED=0.99991239806320081,0.00008760193679918&#xA;1   10786   rs1258377526    CGGCGCAGGCGCAGACACATGCTAGCGCGTCGGGGTGGAGGCGT    C   .   .   RS=1258377526;RSPOS=10787;dbSNPBuildID=151;SSR=0;SAO=0;VP=0x050000020005000002000200;GENEINFO=DDX11L1:100287102;WGT=1;VC=DIV;R5;ASP&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Don't expect just one major and one minor allele per location.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-10-10T07:10:35.107" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21662" PostTypeId="2" ParentId="19684" CreationDate="2023-10-10T07:11:31.453" Score="1" Body="&lt;p&gt;Both existing answers seem to work to my eye. An alternative approach:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;&#xA;a = [&amp;quot;house&amp;quot;,&amp;quot;garden&amp;quot;, &amp;quot;living room&amp;quot;, &amp;quot;dog&amp;quot;,&amp;quot;cat&amp;quot;]&#xA;b = [&amp;quot;cat&amp;quot;,&amp;quot;dog&amp;quot;, &amp;quot;chicken&amp;quot;]&#xA;c = [&amp;quot;house&amp;quot;, &amp;quot;garden&amp;quot;,&amp;quot;bathroom&amp;quot;, &amp;quot;unique&amp;quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;First we generate all options available. This way your 'a' list is not restricting the options. We use &lt;a href=&quot;https://docs.python.org/3/library/stdtypes.html#set-types-set-frozenset&quot; rel=&quot;nofollow noreferrer&quot;&gt;sets&lt;/a&gt; to get unique values.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;options = list(set(a + b + c))&#xA;wanted = pd.DataFrame({'options' : options} )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Next we loop over all the lists. We use &lt;a href=&quot;https://docs.python.org/3/tutorial/datastructures.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;tuples&lt;/a&gt; to assign both name and variable.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for name, _ in [('a',a),('b',b),('c',c)]:&#xA;    wanted.loc[wanted['options'].isin(_), name ] = 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Finally we fill the missing values with zeroes.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;wanted = wanted.fillna('0')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In case you have large number of lists, you can iterate over ascii_lowercase letters:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;my_tuples = ()&#xA;from string import ascii_lowercase&#xA;&#xA;for _ in ascii_lowercase[:3]: #&amp;lt;-magic number for number of lists here&#xA;     my_tuples = my_tuples + (str(_), eval(_))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then change for loop as:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for name, _ in my_tuples:&#xA;     ....&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17627" LastEditorUserId="17627" LastEditDate="2023-10-10T07:51:13.817" LastActivityDate="2023-10-10T07:51:13.817" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21663" PostTypeId="1" CreationDate="2023-10-10T09:15:42.673" Score="1" ViewCount="58" Body="&lt;p&gt;I have clusters of DNA sequences that contain some mutations. All the sequences in a cluster should contain the same mutations.&lt;/p&gt;&#xA;&lt;p&gt;As the mutations aren't known, those clusters of sequences were pairwise-aligned against the non-mutated reference with &lt;code&gt;Minimap2&lt;/code&gt;. Then an interesting fact cropped-up: a few mutations close to each other will make the alignment go awry in that zone.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;Zone with a few SNPs:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;ref  GTATGCGGATACCG&#xA;&#xA;sq1  GTATCGTGACCG&#xA;sq2  GTATCTCGTGACCG&#xA;sq3  GTGTCTATTCGTGACCG&#xA;sq4  GTATCTCGTGACCG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Independent pairwise alignments against the reference (I removed the insertions from the output):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;ref  GTATGCGGATACCG&#xA;&#xA;sq1  GTAT-CGG--ACCG&#xA;sq2  GTATCTCGTGACCG&#xA;sq3  GTATTCGG--ACCG&#xA;sq4  GTATCTCGTGACCG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So I made a MSA of this small cluster (adding the reference in it) with &lt;code&gt;Muscle5&lt;/code&gt;; the difference is noticeable, as we can clearly &amp;quot;see&amp;quot; the mutations now:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;MSA alignment that includes the reference:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;ref  GTATG---CGGATACCG&#xA;&#xA;sq1  GTAT-----CGTGACCG&#xA;sq2  GTATC---TCGTGACCG&#xA;sq3  GTGTCTATTCGTGACCG&#xA;sq4  GTATC---TCGTGACCG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Same MSA, but trimmed:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;ref  GTATGCGGATACCG&#xA;&#xA;sq1  GTAT--CGTGACCG&#xA;sq2  GTATCTCGTGACCG&#xA;sq3  GTGTCTCGTGACCG&#xA;sq4  GTATCTCGTGACCG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I would say calculating the MSA seems to be the way to go but the clusters sizes range from 2 to 1000 sequences, the sequences are 3000nt long, and there's 2,000,000 sequences in total; we cannot afford to do the MSA of each cluster on the full size of the sequences.&#xA;&lt;br&gt;&#xA;On the other hand, we could probably manage to do a MSA of a few small zones in the sequences.&lt;/p&gt;&#xA;&lt;p&gt;So, given a cluster and the pairwise alignments of its sequences against the reference, &lt;strong&gt;how would you determine the zones that need to be re-aligned?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;note:&lt;/strong&gt; I'm using Muscle5 because there's a Python binding of it and the results seem good. I also tried FAMSA, which is very fast, but it didn't yield good alignments.&lt;/p&gt;&#xA;" OwnerUserId="18190" LastEditorUserId="18190" LastEditDate="2023-10-10T13:15:44.947" LastActivityDate="2023-10-10T13:15:44.947" Title="SNP How to detect regions of erratic pairwise alignment?" Tags="|python|sequence-alignment|snp|multiple-sequence-alignment|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21664" PostTypeId="2" ParentId="18302" CreationDate="2023-10-10T13:03:40.663" Score="0" Body="&lt;p&gt;&lt;code&gt;Idents(data.sc, cells=colnames(data.sc) %in% colnames(data.sub.1))&lt;/code&gt; gives the same output as &lt;code&gt;Idents(data.sc, cells=colnames(data.sc) %in% colnames(data.sub.2))&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Both are returning the cellIDs of all cells.&lt;/p&gt;&#xA;&lt;p&gt;The correct code to generate a list of colnames of data.sc is as follows:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;Idents(data.sc, cells=colnames(data.sc)[colnames(data.sc) %in% colnames(data.sub.1)]) &amp;lt;- paste0(&amp;quot;sub1_&amp;quot;, Idents(data.sub.1))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, you can directly add the new cluster names to a metadata column in the Seurat object using the following code:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;## Set up new metadata column to store subcluster IDs later ##&#xA;data.sc$subcluster &amp;lt;- Idents(data.sc)&#xA;&#xA;## subclustering steps (from a previous comment)##&#xA;data.sub.1 &amp;lt;- subset(data.sc, subset = seurat_clusters == 1)&#xA;data.sub.1 &amp;lt;- FindVariableFeatures(data.sub.1)&#xA;data.sub.1 &amp;lt;- ScaleData(data.sub.1)&#xA;data.sub.1 &amp;lt;- RunPCA(data.sub.1, verbose = FALSE)&#xA;data.sub.1 &amp;lt;- RunUMAP(data.sub.1, reduction = &amp;quot;pca&amp;quot;, dims = 1:30)&#xA;data.sub.1 &amp;lt;- FindNeighbors(data.sub.1, reduction = &amp;quot;pca&amp;quot;, dims = 1:30)&#xA;data.sub.1 &amp;lt;- FindClusters(data.sub.1, resolution = 0.5)&#xA;&#xA;## merging the subclusster names back to original clusters ##&#xA;data.sc&lt;span class=&quot;math-container&quot;&gt;$subcluster[colnames(dta.sc)[colnames(data.sc) %in% colnames(data.sub.1)]] &amp;lt;- paste0(&quot;sub1_&quot;, data.sub.1$&lt;/span&gt;seurat_clusters)&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18347" LastEditorUserId="18347" LastEditDate="2023-10-10T16:40:49.040" LastActivityDate="2023-10-10T16:40:49.040" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21665" PostTypeId="1" AcceptedAnswerId="21668" CreationDate="2023-10-10T13:20:47.787" Score="3" ViewCount="136" Body="&lt;p&gt;I would like to use the program BayesTraitV4. I have my phylogenetic tree in NEXUS format but the program does not accept it. I checked the example data and while they are also in NEXUS format, they seemed different. I will provide a part of my data and the example data respectively.&lt;/p&gt;&#xA;&lt;p&gt;My tree:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#NEXUS&#xA;begin taxa;&#xA;    dimensions ntax=85;&#xA;    taxlabels&#xA;    LevantinaFC4779&#xA;    Assyriella_naegelei&#xA;    Ceu_LAG_2&#xA;    CeuMist2&#xA;    CeuMist1&#xA;    Ceu_PAN_2&#xA;    Ceu_PAN_1&#xA;    Ceu_PAN_3&#xA;    Ceu_PAN_4&#xA;    CcMath1&#xA;    CcNav1&#xA;    CcNav2&#xA;    Cc_Mal_1&#xA;    Cc_KalR2_2&#xA;    CcRod1&#xA;    CcRod2&#xA;    CcRod3&#xA;    CcRod4&#xA;    CcHr1&#xA;    CeST6_1&#xA;    CeST6_4&#xA;    CeST6_5&#xA;    CeAA7_1&#xA;    CeST6_3&#xA;    CeST6_2&#xA;    Cg_MelR7_1&#xA;    Cg_MelR3_1&#xA;    Cg_MelR6_2&#xA;    Cg_MelR7_2&#xA;    Cg_MelR6_1&#xA;    Cg_MelR7_3&#xA;    Cg_MelR5_1&#xA;    Cg_MelR4_2&#xA;    Cg_MelR1_1&#xA;    Cg_MelR2_1&#xA;    Cg_MelR4_1&#xA;    Cg_MelR7_4&#xA;    Cg_MelR5_2&#xA;    ChTM4_2&#xA;    ChTM5_1&#xA;    ChTM4_3&#xA;    ChTM4_1&#xA;    Ch_MEN_5&#xA;    Ch_MEN_13&#xA;    Ch_MEN_8&#xA;    Ch_LEV_3&#xA;    Ch_LEV_5&#xA;    Ch_LEV_4&#xA;    Ch_MEN_6&#xA;    Ch_MEN_10&#xA;    Ch_LEV_2&#xA;    Ch_MEN_4&#xA;    Ch_MEN_14&#xA;    Ch_LEV_1&#xA;    Ch_MEN_11&#xA;    Ch_MEN_3&#xA;    Ch_MEN_9&#xA;    Ci_MSP_3&#xA;    Ci_MSP_2&#xA;    Ci_MSP_1&#xA;    Ci_DIA_1&#xA;    Ci_DIA_2&#xA;    CiPLAN1&#xA;    CiPLAN3&#xA;    CiPLAN2&#xA;    Ci_MEN_1&#xA;    Ci_MEN_2&#xA;    Ci_MEN_12&#xA;    CpGravR6_1&#xA;    CpGravR3&#xA;    CpGravR4&#xA;    CpGio_2&#xA;    CpGravR5_2&#xA;    CpGravR2&#xA;    CpGravR7&#xA;    CpGio_1&#xA;    CpGio_3&#xA;    CpGravR1&#xA;    CpGravR6_2&#xA;    CpGravR5_1&#xA;    CpThe_1&#xA;    CpSka_1&#xA;    Cn_IG_2&#xA;    Cn_IG_3&#xA;    Cn_IG_1&#xA;;&#xA;end;&#xA;&#xA;begin trees;&#xA;    tree TREE1 = [&amp;amp;R] ((LevantinaFC4779[&amp;amp;rate_range={7.888601174550756E-4,0.2727277039812744},height_95%_HPD={0.0,1.5711876244495215E-12},length_range={0.6843282088933,6.197383612374},height_median=6.004086117172847E-13,length_95%_HPD={1.080900382604,3.875114824373},height=6.614724410322848E-13,rate=0.049900260878735954,height_range={0.0,5.799805080641818E-12},rate_median=0.045084434131968686,length=2.411967820837161,length_median=2.3472043999879997,rate_95%_HPD={0.0108358382756707,0.10141567250363237}]:2.4119678208371664,Assyriella_naegelei[&amp;amp;rate_range={2.4004562386929353E-4,0.19480057878691465},height_95%_HPD={0.0,1.5711876244495215E-12},length_range={0.6843282088933,6.197383612374},height_median=6.004086117172847E-13,length_95%_HPD={1.080900382604,3.875114824373},height=6.614724410322848E-13,rate=0.03027613080757391,height_range={0.0,5.799805080641818E-&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Example tree:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#NEXUS&#xA;Begin trees;&#xA;    translate&#xA;        1   Opossum,&#xA;        2   Diprotodontian,&#xA;        3   Sloth,&#xA;        4   Armadillo,&#xA;        5   Anteater,&#xA;        6   Hedgehog,&#xA;        7   Mole,&#xA;        8   Shrew,&#xA;        9   Tenrecid,&#xA;        10  Golden_Mole,&#xA;        11  Sirenian,&#xA;        12  Hyrax,&#xA;        13  Elephant,&#xA;        15  Lo_Ear_Ele_shrew,&#xA;        17  Sciurid,&#xA;        18  Mouse,&#xA;        19  Rat,&#xA;        20  Hystricid,&#xA;        21  Caviomorph,&#xA;        22  Rabbit,&#xA;        23  Pika,&#xA;        24  Flying_Lemur,&#xA;        25  Tree_Shrew,&#xA;        26  Strepsirrhine,&#xA;        28  Phyllostomid,&#xA;        29  Free_tailed_bat,&#xA;        30  False_vampire_bat,&#xA;        31  Flying_Fox,&#xA;        32  Rousette_Fruitbat,&#xA;        33  Whale,&#xA;        35  Hippo,&#xA;        36  Llama,&#xA;        37  Ruminant,&#xA;        38  Pig,&#xA;        39  Horse,&#xA;        40  Rhino,&#xA;        41  Tapir,&#xA;        42  Cat,&#xA;        43  Caniform,&#xA;        44  Pangolin;&#xA;        tree tree.164000.208997.409491 = ((1:0.07628,2:0.065151):0.1546885,(((15:0.133631,(9:0.157087,10:0.077841):0.007403):0.005959,(11:0.0406,(12:0.092769,13:0.052284):0.001099):0.01453):0.029813,((4:0.05807,(3:0.054688,5:0.062757):0.008238):0.029441,(((26:0.083487,(24:0.064103,25:0.110063):0.003501):0.000775,((22:0.058663,23:0.116805):0.045375,(17:0.0898,((18:0.039267,19:0.043896):0.125808,(20:0.083297,21:0.089139):0.067194):0.006543):0.011062):0.005229):0.00803,((7:0.090648,(6:0.150383,8:0.133021):0.006503):0.01198,((36:0.062932,(38:0.066525,(37:0.076473,(35:0.039222,33:0.031429):0.00357):0.008228):0.005968):0.026714,((44:0.081521,(42:0.046822,43:0.059428):0.024607):0.004257,((39:0.046001,(40:0.029049,41:0.029533):0.003965):0.022397,((28:0.093611,29:0.050095):0.013432,(30:0.061026,(31:0.018601,32:0.018701):0.036144):0.004989):0.015808):0.001216):0.000872):0.002868):0.009567):0.010713):0.00234):0.1546885);&#xA;        tree tree.165000.208997.199794 = ((1:0.07628,2:0.065151):0.1546885,(((15:0.133631,(9:0.157087,10:0.077841):0.007403):0.005959,(11:0.0406,(12:0.092769,13:0.052284):0.000485):0.01453):0.029813,((4:0.05807,(3:0.054688,5:0.062757):0.008238):0.029441,(((26:0.083487,(24:0.064103,25:0.110063):0.002826):0.000775,((22:0.058663,23:0.116805):0.045375,(17:0.0898,((18:0.039267,19:0.043896):0.125808,(20:0.083297,21:0.089139):0.067194):0.006543):0.011062):0.003736):0.00803,((7:0.090648,(6:0.150383,8:0.133021):0.010132):0.01198,((44:0.084486,(42:0.046822,43:0.059428):0.024607):0.004257,((36:0.062932,(38:0.066525,(37:0.076473,(35:0.039222,33:0.031429):0.004674):0.008228):0.005968):0.026714,((39:0.046001,(40:0.029049,41:0.029533):0.003965):0.022397,((28:0.093611,29:0.050095):0.008576,(30:0.061026,(31:0.018601,32:0.018701):0.036144):0.004989):0.015808):0.001216):0.000872):0.002868):0.009567):0.010713):0.00234):0.1546885);&#xA;;&#xA;&#xA;&#xA;end;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Why are these formats different and how can I make my file to look like the example one?&#xA;Keep in mind the files are not presented in their entirety because I'm limited with the amount of characters.&lt;/p&gt;&#xA;&lt;p&gt;Edit: I will be providing my tree in this link &lt;a href=&quot;https://drive.google.com/file/d/1lYmzW9htwaclC_yYs3fQom-Z3zbjk8wH/view?usp=sharing&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://drive.google.com/file/d/1lYmzW9htwaclC_yYs3fQom-Z3zbjk8wH/view?usp=sharing&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16657" LastEditorUserId="16657" LastEditDate="2023-10-10T17:40:58.377" LastActivityDate="2023-10-11T13:45:45.107" Title="How can I convert my tree to the correct nexus format?" Tags="|phylogenetics|phylogeny|file-formats|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21666" PostTypeId="1" CreationDate="2023-10-10T13:44:17.497" Score="2" ViewCount="37" Body="&lt;p&gt;I want to compare/validate variants called from sequencing data with array (plink format) variant data. I converted the plink files (.bim, .bed, and .fam files) with plink1 to vcf files.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plink --bfile prefix_plink --recode vcf-iid --out prefix_out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, the plink vcf files have &amp;quot;I&amp;quot; and &amp;quot;D&amp;quot; values for INDEL variants that cannot be compared and break the tools downstream (I am now using hap.py with rtgtools to validate). Is there a way to convert plink files to vcf, with some transformation of these I and D variants to a usable value? A &amp;quot;D&amp;quot; I could translate by hand to &amp;quot;&amp;quot; which is a vcf accepted format. But insertions require I guess a more complex procedure, or are even impossible (because they need to be compared to a reference genome)? Or am I missing some flag somewhere?&lt;/p&gt;&#xA;&lt;p&gt;I would love to get some pointers and tips for this.&lt;/p&gt;&#xA;&lt;p&gt;Kind regards,&lt;/p&gt;&#xA;" OwnerUserId="16204" LastActivityDate="2023-10-10T13:44:17.497" Title="INDELS in PLINK files converted to VCF" Tags="|vcf|variant-calling|plink|indel|validation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21668" PostTypeId="2" ParentId="21665" CreationDate="2023-10-10T18:05:41.377" Score="2" Body="&lt;p&gt;Okay ... this will probably work&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#NEXUS&#xA;begin trees;&#xA;    tree TREE1 = [&amp;amp;R] ((LevantinaFC4779:2.411968,Assyriella_naegelei:2.411968):5.752052,((Cn_IG_1:0.126901,(Cn_IG_2:0.075971,Cn_IG_3:0.075971):0.05093):7.4805,(((CpThe_1:0.122189,CpSka_1:0.122189):2.663216,((CpGio_2:0.137489,(CpGravR6_1:0.087156,(CpGravR3:0.038234,CpGravR4:0.038234):0.048922):0.050334):0.257308,(CpGravR5_1:0.28159,((CpGravR7:0.099712,(CpGravR5_2:0.020255,CpGravR2:0.020255):0.079458):0.15617,(CpGravR6_2:0.109222,(CpGio_1:0.068711,(CpGio_3:0.049345,CpGravR1:0.049345):0.019367):0.040511):0.14666):0.025708):0.113207):2.390607):1.772627,(((Ci_MEN_12:0.072368,(Ci_MEN_1:0.022725,Ci_MEN_2:0.022725):0.049643):1.78894,((CiPLAN2:0.253487,(CiPLAN1:0.067509,CiPLAN3:0.067509):0.185978):0.386279,((Ci_MSP_3:0.061614,Ci_MSP_2:0.061614):0.231268,(Ci_MSP_1:0.113777,(Ci_DIA_1:0.047073,Ci_DIA_2:0.047073):0.066704):0.179105):0.346885):1.221542):1.580745,(((Ceu_LAG_2:1.580984,(CeuMist2:0.198982,CeuMist1:0.198982):1.382002):0.341226,((Ceu_PAN_2:0.046898,Ceu_PAN_1:0.046898):0.20368,(Ceu_PAN_3:0.018814,Ceu_PAN_4:0.018814):0.231764):1.671632):1.226475,((CcHr1:1.247104,((CcRod1:0.403751,(CcRod4:0.276104,(CcRod2:0.158977,CcRod3:0.158977):0.117127):0.127647):0.308704,((Cc_Mal_1:0.138232,Cc_KalR2_2:0.138232):0.395435,(CcMath1:0.338389,(CcNav1:0.091762,CcNav2:0.091762):0.246627):0.195278):0.178788):0.534649):1.609976,(((CeST6_5:0.056918,(CeST6_1:0.030107,CeST6_4:0.030107):0.026812):0.733078,((CeST6_2:0.143014,(CeAA7_1:0.097814,CeST6_3:0.097814):0.0452):0.349431,((Cg_MelR7_1:0.136715,(Cg_MelR3_1:0.088065,(Cg_MelR6_2:0.047987,(Cg_MelR7_2:0.019268,Cg_MelR6_1:0.019268):0.028719):0.040078):0.04865):0.074941,(Cg_MelR5_2:0.143401,(Cg_MelR7_3:0.119131,(Cg_MelR5_1:0.090065,((Cg_MelR4_1:0.020938,Cg_MelR7_4:0.020938):0.051613,(Cg_MelR4_2:0.052913,(Cg_MelR1_1:0.021387,Cg_MelR2_1:0.021387):0.031526):0.019639):0.017514):0.029066):0.02427):0.068255):0.280789):0.297551):0.793423,(((ChTM4_2:0.049871,ChTM5_1:0.049871):0.111934,(ChTM4_3:0.023134,ChTM4_1:0.023134):0.138672):0.228306,(Ch_MEN_9:0.174548,((Ch_MEN_11:0.021817,Ch_MEN_3:0.021817):0.10954,((Ch_MEN_8:0.057141,(Ch_MEN_5:0.022973,Ch_MEN_13:0.022973):0.034168):0.053065,((Ch_LEV_3:0.031056,Ch_LEV_5:0.031056):0.061225,((Ch_LEV_4:0.029359,(Ch_MEN_6:0.017456,Ch_MEN_10:0.017456):0.011904):0.034407,((Ch_LEV_2:0.017911,Ch_MEN_4:0.017911):0.040741,(Ch_MEN_14:0.03419,Ch_LEV_1:0.03419):0.024462):0.005114):0.028515):0.017925):0.021151):0.043191):0.215563):1.193307):1.27366):0.291605):0.293368):1.115978):3.04937):0.556619);&#xA;end;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Alternatively, I'm not sure about the root, just remove &lt;code&gt;[&amp;amp;R]&lt;/code&gt; in the above.&lt;/p&gt;&#xA;&lt;p&gt;Finally there could be an issue that you haven't obtained the sample directly from the raw Beast output, instead you've summarised it. It might complain about that. Thus it might need all those trees from the raw Beast output. Thats not a format issue it's a data input issue. BayesTraits might be perform the calculation against the total MCMCMC sample minus the burn-in.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;from comments&lt;/p&gt;&#xA;&lt;p&gt;Read it into Rs &lt;code&gt;ape::read.nexus()&lt;/code&gt; and then write via &lt;code&gt;phytools::writeNexus()&lt;/code&gt;. This should produce your translate table. Biopython recognises this format but doesn't write to it.&lt;/p&gt;&#xA;&lt;p&gt;The closest Biopython can achieve is a taxablock here&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#NEXUS&#xA;Begin Taxa;&#xA; Dimensions NTax=85;&#xA; TaxLabels LevantinaFC4779 Assyriella_naegelei Cn_IG_1 Cn_IG_2 Cn_IG_3 CpThe_1 CpSka_1 CpGio_2 CpGravR6_1 CpGravR3 CpGravR4 CpGravR5_1 CpGravR7 CpGravR5_2 CpGravR2 CpGravR6_2 CpGio_1 CpGio_3 CpGravR1 Ci_MEN_12 Ci_MEN_1 Ci_MEN_2 CiPLAN2 CiPLAN1 CiPLAN3 Ci_MSP_3 Ci_MSP_2 Ci_MSP_1 Ci_DIA_1 Ci_DIA_2 Ceu_LAG_2 CeuMist2 CeuMist1 Ceu_PAN_2 Ceu_PAN_1 Ceu_PAN_3 Ceu_PAN_4 CcHr1 CcRod1 CcRod4 CcRod2 CcRod3 Cc_Mal_1 Cc_KalR2_2 CcMath1 CcNav1 CcNav2 CeST6_5 CeST6_1 CeST6_4 CeST6_2 CeAA7_1 CeST6_3 Cg_MelR7_1 Cg_MelR3_1 Cg_MelR6_2 Cg_MelR7_2 Cg_MelR6_1 Cg_MelR5_2 Cg_MelR7_3 Cg_MelR5_1 Cg_MelR4_1 Cg_MelR7_4 Cg_MelR4_2 Cg_MelR1_1 Cg_MelR2_1 ChTM4_2 ChTM5_1 ChTM4_3 ChTM4_1 Ch_MEN_9 Ch_MEN_11 Ch_MEN_3 Ch_MEN_8 Ch_MEN_5 Ch_MEN_13 Ch_LEV_3 Ch_LEV_5 Ch_LEV_4 Ch_MEN_6 Ch_MEN_10 Ch_LEV_2 Ch_MEN_4 Ch_MEN_14 Ch_LEV_1;&#xA;End;&#xA;Begin Trees;&#xA; Tree tree1=((LevantinaFC4779:2.41197,Assyriella_naegelei:2.41197):5.75205,((Cn_IG_1:0.12690,(Cn_IG_2:0.07597,Cn_IG_3:0.07597):0.05093):7.48050,(((CpThe_1:0.12219,CpSka_1:0.12219):2.66322,((CpGio_2:0.13749,(CpGravR6_1:0.08716,(CpGravR3:0.03823,CpGravR4:0.03823):0.04892):0.05033):0.25731,(CpGravR5_1:0.28159,((CpGravR7:0.09971,(CpGravR5_2:0.02025,CpGravR2:0.02025):0.07946):0.15617,(CpGravR6_2:0.10922,(CpGio_1:0.06871,(CpGio_3:0.04934,CpGravR1:0.04934):0.01937):0.04051):0.14666):0.02571):0.11321):2.39061):1.77263,(((Ci_MEN_12:0.07237,(Ci_MEN_1:0.02272,Ci_MEN_2:0.02272):0.04964):1.78894,((CiPLAN2:0.25349,(CiPLAN1:0.06751,CiPLAN3:0.06751):0.18598):0.38628,((Ci_MSP_3:0.06161,Ci_MSP_2:0.06161):0.23127,(Ci_MSP_1:0.11378,(Ci_DIA_1:0.04707,Ci_DIA_2:0.04707):0.06670):0.17910):0.34688):1.22154):1.58075,(((Ceu_LAG_2:1.58098,(CeuMist2:0.19898,CeuMist1:0.19898):1.38200):0.34123,((Ceu_PAN_2:0.04690,Ceu_PAN_1:0.04690):0.20368,(Ceu_PAN_3:0.01881,Ceu_PAN_4:0.01881):0.23176):1.67163):1.22647,((CcHr1:1.24710,((CcRod1:0.40375,(CcRod4:0.27610,(CcRod2:0.15898,CcRod3:0.15898):0.11713):0.12765):0.30870,((Cc_Mal_1:0.13823,Cc_KalR2_2:0.13823):0.39543,(CcMath1:0.33839,(CcNav1:0.09176,CcNav2:0.09176):0.24663):0.19528):0.17879):0.53465):1.60998,(((CeST6_5:0.05692,(CeST6_1:0.03011,CeST6_4:0.03011):0.02681):0.73308,((CeST6_2:0.14301,(CeAA7_1:0.09781,CeST6_3:0.09781):0.04520):0.34943,((Cg_MelR7_1:0.13672,(Cg_MelR3_1:0.08807,(Cg_MelR6_2:0.04799,(Cg_MelR7_2:0.01927,Cg_MelR6_1:0.01927):0.02872):0.04008):0.04865):0.07494,(Cg_MelR5_2:0.14340,(Cg_MelR7_3:0.11913,(Cg_MelR5_1:0.09007,((Cg_MelR4_1:0.02094,Cg_MelR7_4:0.02094):0.05161,(Cg_MelR4_2:0.05291,(Cg_MelR1_1:0.02139,Cg_MelR2_1:0.02139):0.03153):0.01964):0.01751):0.02907):0.02427):0.06825):0.28079):0.29755):0.79342,(((ChTM4_2:0.04987,ChTM5_1:0.04987):0.11193,(ChTM4_3:0.02313,ChTM4_1:0.02313):0.13867):0.22831,(Ch_MEN_9:0.17455,((Ch_MEN_11:0.02182,Ch_MEN_3:0.02182):0.10954,((Ch_MEN_8:0.05714,(Ch_MEN_5:0.02297,Ch_MEN_13:0.02297):0.03417):0.05307,((Ch_LEV_3:0.03106,Ch_LEV_5:0.03106):0.06123,((Ch_LEV_4:0.02936,(Ch_MEN_6:0.01746,Ch_MEN_10:0.01746):0.01190):0.03441,((Ch_LEV_2:0.01791,Ch_MEN_4:0.01791):0.04074,(Ch_MEN_14:0.03419,Ch_LEV_1:0.03419):0.02446):0.00511):0.02851):0.01792):0.02115):0.04319):0.21556):1.19331):&#xA;1.27366):0.29161):0.29337):1.11598):3.04937):0.55662):0.00000;&#xA;End;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The solution could be coded from the taxa block if the above doesn't work, but easier to use &lt;strong&gt;R&lt;/strong&gt; &lt;code&gt;ape&lt;/code&gt; and &lt;code&gt;phytools&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-10-11T13:45:45.107" LastActivityDate="2023-10-11T13:45:45.107" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21669" PostTypeId="2" ParentId="14272" CreationDate="2023-10-11T05:37:31.383" Score="1" Body="&lt;p&gt;&lt;code&gt;seqkit stats fastq.gz&lt;/code&gt;. Get SeqKit from &lt;a href=&quot;https://bioinf.shenwei.me/seqkit/usage/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioinf.shenwei.me/seqkit/usage/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3381" LastActivityDate="2023-10-11T05:37:31.383" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21670" PostTypeId="1" CreationDate="2023-10-11T06:43:37.667" Score="1" ViewCount="91" Body="&lt;p&gt;&lt;em&gt;This is adapted from a question posted on &lt;a href=&quot;https://www.reddit.com/r/bioinformatics/comments/174m7p3/16s_vs_full_genome_sequencing/&quot; rel=&quot;nofollow noreferrer&quot;&gt;reddit&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;In my work, I occasionally get asked about metagenomic sequencing, which (after further questioning) I discover means that the person wants to work out the taxonomic composition of a sample.&lt;/p&gt;&#xA;&lt;p&gt;In the past, people have gone straight to 16S sequencing for this, and various manufacturers have created targeted 16S kits for this purpose. But does it still make sense to do that? Is 16S still cheaper and better than other options?&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-10-12T02:48:49.953" Title="Is 16S sequencing cheaper than whole-genome sequencing for taxonomic assignment?" Tags="|phylogenetics|metagenome|hts|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21671" PostTypeId="2" ParentId="21670" CreationDate="2023-10-11T06:43:37.667" Score="2" Body="&lt;p&gt;When comparing anything targeted to whole-genome sequencing for the purposes of assembly, the targeted approach will always be cheaper, because the total length that needs to be assembled is shorter. But that's like comparing a video to a photograph.&lt;/p&gt;&#xA;&lt;p&gt;If you're just looking to determine the phylogenetic composition of a bacterial sample, then there's no reason why targeted 16S sequencing should be cheaper than whole-genome shotgun sequencing, and the whole genome approach may end up better due to the highly conserved nature of the 16S sequence, and the abundance of horizontal gene transfer.&lt;/p&gt;&#xA;&lt;p&gt;Previously, 16S might have been better because the taxonomic databases were more complete, but now that there are so many microbial genomes available, the whole genome databases are comprehensive enough for most purposes.&lt;/p&gt;&#xA;&lt;p&gt;@terdon linked to &lt;a href=&quot;https://pubmed.ncbi.nlm.nih.gov/18682527/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this 2008 paper&lt;/a&gt;, in which they compare random sequence reads to 16S for estimating the biodiversity of a metagenomic library, finding that 16S wasn't worth it and they could get perfectly good results with random reads.&lt;/p&gt;&#xA;&lt;p&gt;In a situation where samples are abundant and no amplification is required, PCR can be excluded from whole-genome sequencing, making it the cheaper option at the sample preparation side of things.&lt;/p&gt;&#xA;&lt;p&gt;Just to put numbers to things to demonstrate how cheap whole-genome sequencing can be, here are the big consumable costs for nanopore (prices in USD):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;RBK114.24 rapid barcoding kit* - \$700 (24 samples, 12 runs)&lt;/li&gt;&#xA;&lt;li&gt;Flongle flow cell - \$810 (12 flow cells)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So the marginal cost of one run is \$68 for the flow cell, and \$60 for the sequencing kit, processing 24 samples at a cost of about $11 per sample (rounding up).&lt;/p&gt;&#xA;&lt;p&gt;[FWIW, ONT also provide a rapid PCR barcoding kit for the same cost (i.e. \$700, but polymerase is not included), which can be used together with PCR for low-yield samples.]&lt;/p&gt;&#xA;&lt;p&gt;Flongle flow cells will usually generate &amp;gt;200 Mb per run (up to about 2 Gb in rare cases), so that's about 8 Mb of sequence per sample, which should be sufficient to do proportional counting down to about 1% of the sample (assuming &amp;gt;1000 reads per sample).&lt;/p&gt;&#xA;&lt;p&gt;* There is a 96-barcode kit available for \$990, but I don't think that would have enough coverage to be useful when running samples on a Flongle flow cell.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-10-12T02:48:49.953" LastActivityDate="2023-10-12T02:48:49.953" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21672" PostTypeId="2" ParentId="21660" CreationDate="2023-10-11T09:23:58.907" Score="5" Body="&lt;p&gt;Yes. &lt;a href=&quot;https://www.mdpi.com/2075-1729/12/12/2030#B16-life-12-02030&quot; rel=&quot;noreferrer&quot;&gt;Phasing data leads to a better imputation accuracy&lt;/a&gt;. More specifically:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Improves allele matching: Correct phased data ensures that alleles are matched correctly between the reference panel and the target dataset.&lt;/li&gt;&#xA;&lt;li&gt;Using phased data, the computational time of imputation is significantly reduced.&lt;/li&gt;&#xA;&lt;li&gt;Having haplotypes (phased data) helps imputing variants that were not genotyped (missing variants/genotypes).&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/ncomms9111&quot; rel=&quot;noreferrer&quot;&gt;This paper&lt;/a&gt; based on the UK10K data has some actual numbers describing imputation accuracy of phased vs unphased data.&lt;/p&gt;&#xA;" OwnerUserId="475" LastActivityDate="2023-10-11T09:23:58.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21673" PostTypeId="1" AcceptedAnswerId="21674" CreationDate="2023-10-11T12:08:15.273" Score="0" ViewCount="34" Body="&lt;p&gt;I am using this package &lt;a href=&quot;https://pypi.org/project/nsdpy/&quot; rel=&quot;nofollow noreferrer&quot;&gt;nsdpy&lt;/a&gt; to download genome sequences from &lt;strong&gt;NCBI nucleotide database&lt;/strong&gt;.&lt;br /&gt;&#xA;Specifically I am interested in the &lt;strong&gt;whole mitochondrial genome&lt;/strong&gt; of different species, here I will use a subset of species just as an example.&lt;/p&gt;&#xA;&lt;p&gt;My Query is:&lt;br /&gt;&#xA;&lt;code&gt;&amp;quot;{mitochondrion[Title]&amp;quot; -i -t -L subset_species_list.txt &lt;/code&gt;&lt;br /&gt;&#xA;Where :&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;-i&lt;/code&gt; Adds the taxonomic information to the information lines of the sequences written in the output files (not relevant for this example)&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;-t&lt;/code&gt; option produces a tsv file with the following fields: Taxon name, SequenceID, TaxID, Taxonomic lineage, Sequence length, Sequence.&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;subset_species_list.txt&lt;/code&gt; is a list of species, below you can see its content&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;Xenopus laevis&#xA;Gasterosteus aculeatus&#xA;Chironomus riparius&#xA;Rasbora heteromorpha&#xA;Planktolyngbya limnetica&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output file is quite long and includes the actual sequences. Here I will post the same file where I dropped the column for the sequence and the lineage to keep it compact&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4QcyKl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4QcyKl.png&quot; alt=&quot;table&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;My aim is to obtain phylogenetic distances between species.&lt;/p&gt;&#xA;&lt;p&gt;My questions are:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What do the sequences for the same species represent?&lt;/li&gt;&#xA;&lt;li&gt;Which sequences are best suited for an alignment and how do I know that?&lt;/li&gt;&#xA;&lt;li&gt;Should I also look into the orientation of the sequences or are they all on the forward strand?&lt;/li&gt;&#xA;&lt;li&gt;Is it better if I filter for Coding sequences (CDS) instead of whole genomes&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Also if you noticed any rookie mistake in my process feel free to point that out. It is the first time for me working with DNA sequences and I really want to learn to correct way to do that.&lt;/p&gt;&#xA;" OwnerUserId="18351" LastEditorUserId="18351" LastEditDate="2023-10-11T12:51:34.803" LastActivityDate="2023-10-11T14:47:05.880" Title="What does the different sequences represent?" Tags="|sequence-alignment|sequence-analysis|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21674" PostTypeId="2" ParentId="21673" CreationDate="2023-10-11T14:47:05.880" Score="0" Body="&lt;p&gt;Welcome! I have some quick basic answers for the questions you're asking so hopefully this helps:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Sequences for the same species represent variation/diversity/evolutionary changes. Think of humans; how do different people's DNA sequences represent the sequence of a human? This is especially important for phylogenetic studies.&lt;/li&gt;&#xA;&lt;li&gt;For the most part, all sequences are best suited for alignment. The main thing to think about is the amount of data you have (if it's a lot of GB the alignment will take a lot of computing power), the type of sequnece data you have (simple rule of thumb is to not mix short read with long read sequences), and the tool you're using for alignment.&lt;/li&gt;&#xA;&lt;li&gt;Always a good idea to think about orientation but most tools will account for orientation when aligning sequences.&lt;/li&gt;&#xA;&lt;li&gt;This depends on what you're looking for but more often than not using the whole genomes is best as it lets you look at the whole picture instead of just a small portion but that's definitely a lot of computing power. If you're just looking at the mitochondria just focus on that region&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="16067" LastActivityDate="2023-10-11T14:47:05.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21676" PostTypeId="2" ParentId="18748" CreationDate="2023-10-11T15:34:49.173" Score="2" Body="&lt;p&gt;This is a common issue. Instead of lifting over your genomes, it might be computationally and conceptually easier just to lift over the summary statistics to GRCh37. Alternatively (and preferably), this is &lt;strong&gt;done for you&lt;/strong&gt; for every GWAS published in the GWAS catalog. Simply click on the &amp;quot;harmonized&amp;quot; folder after clicking on &amp;quot;download&amp;quot; and you can download whatever build aligns with your genotype data.&lt;/p&gt;&#xA;" OwnerUserId="17430" LastActivityDate="2023-10-11T15:34:49.173" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21677" PostTypeId="2" ParentId="21035" CreationDate="2023-10-11T15:39:33.887" Score="1" Body="&lt;p&gt;Yes, you can run an additive model (i.e., B*AA = 2*B*Aa, aa = B*AA - 4*Aa) or a dominance model (B*AA = B*Aa). You can test both of these and see which is a better fit to your data.&lt;/p&gt;&#xA;&lt;p&gt;However, it is practically known already that the model will be at least predominantly additive.&lt;/p&gt;&#xA;" OwnerUserId="17430" LastActivityDate="2023-10-11T15:39:33.887" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21678" PostTypeId="1" CreationDate="2023-10-11T16:42:11.223" Score="1" ViewCount="72" Body="&lt;p&gt;I have a general question about building phylogenetic trees without alignment:&lt;/p&gt;&#xA;&lt;p&gt;I want to build a tree using an alignment-free approach by counting kmers. I have found a lot of information in the literature about this type of practice but I'm having trouble finding approaches that use Illumina short read sequences (which is what I have).&lt;/p&gt;&#xA;&lt;p&gt;In my first attempt, I was able to generate a dendrogram by counting kmers and clustering them, but it did not group the sequences I had in the “correct/expected” way (these are known strains that are part of specific subclasses related to S. cerevisiae - in theory, this kmer clustering approach should’ve been able to group the strains based on their subclass pretty easily).&lt;/p&gt;&#xA;&lt;p&gt;There are 21 samples in total with 5 specific subclasses. Does anyone know a way to either cluster or simply calculate the distance between kmers to generate a tree of 20+ Illumina short-read samples?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&#xA;The main reason for doing this in this way is to eliminate any bias from the reference sequence and maintain all the alleles (most of the samples are tetraploid) so I really want to avoid collapsing the data with any type of preprocessing alignment or assembly.&lt;/p&gt;&#xA;&lt;p&gt;With that being said I would want to use the whole-genome short-read sequences of these 21 samples to count the kmers across the genome for phylogenetic analysis. If it gets too complicated, I would be okay with assembling the reads first before kmer counting.&lt;/p&gt;&#xA;&lt;p&gt;I used &lt;a href=&quot;https://github.com/natapol/kitsune&quot; rel=&quot;nofollow noreferrer&quot;&gt;Kitsune&lt;/a&gt; to determine the optimal kmer size for the samples which comes out to 15mer. I was able to count 15mers with all the samples using three methods: pure raw reads, assembling the reads with SPAdes, and downsampling the raw reads to 5x coverage. Using Jellyfish and custom scripts I wrote, which didn't end up being too taxing on my computer/server.&lt;/p&gt;&#xA;&lt;p&gt;Currently, I have 21 '.tsv' files that list the kmer and the kmer frequency for each individual sample and I have one giant matrix that combines that data together so I have the kmer and the kmer count for each sample in one file:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;&lt;/th&gt;&#xA;&lt;th&gt;sample_1&lt;/th&gt;&#xA;&lt;th&gt;sample_2&lt;/th&gt;&#xA;&lt;th&gt;sample_3&lt;/th&gt;&#xA;&lt;th&gt;...&lt;/th&gt;&#xA;&lt;th&gt;sample_21&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;AAAAA&lt;/td&gt;&#xA;&lt;td&gt;123432&lt;/td&gt;&#xA;&lt;td&gt;345432&lt;/td&gt;&#xA;&lt;td&gt;56765&lt;/td&gt;&#xA;&lt;td&gt;...&lt;/td&gt;&#xA;&lt;td&gt;55677&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;AAAAG&lt;/td&gt;&#xA;&lt;td&gt;1023&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;350655&lt;/td&gt;&#xA;&lt;td&gt;...&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;AAAAT&lt;/td&gt;&#xA;&lt;td&gt;462&lt;/td&gt;&#xA;&lt;td&gt;741&lt;/td&gt;&#xA;&lt;td&gt;2136&lt;/td&gt;&#xA;&lt;td&gt;...&lt;/td&gt;&#xA;&lt;td&gt;121&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;AAAAC&lt;/td&gt;&#xA;&lt;td&gt;342&lt;/td&gt;&#xA;&lt;td&gt;123&lt;/td&gt;&#xA;&lt;td&gt;3453&lt;/td&gt;&#xA;&lt;td&gt;...&lt;/td&gt;&#xA;&lt;td&gt;12345&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;------------&lt;/td&gt;&#xA;&lt;td&gt;----------&lt;/td&gt;&#xA;&lt;td&gt;----------&lt;/td&gt;&#xA;&lt;td&gt;---------------&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;td&gt;&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;I have looked briefly at kWIP. Ideally, I would use their distance calculation function to generate a .dist file for my giant matrix - I'll look into what inputs can be used for that function.&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="73" LastEditDate="2023-10-12T02:44:39.763" LastActivityDate="2023-10-12T02:44:39.763" Title="Approach for alignment-free phylogenetic tree generation using kmers" Tags="|phylogenetics|phylogeny|clustering|k-mer|distance|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21679" PostTypeId="2" ParentId="21678" CreationDate="2023-10-11T18:52:02.217" Score="1" Body="&lt;p&gt;A very simple solution is to use a correlation &lt;em&gt;r&lt;/em&gt; or &lt;em&gt;r&lt;/em&gt;&lt;sup&gt;2&lt;/sup&gt;. This establishes a pairwise matrix which is then resolved under neighbour-joining (don't use UPGMA or similar). MegaX will import a pairwise matrix and produce a tree in neighbour-joining. You could even produce a bootstrapped solution (that would require custom code) - bit tricky.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;I definitely recommend neighbour-joining for the tree&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;MegaX is a good implementation of neighbour-joining (it was originally Nei's idea and that was his package). The reason for this are complex and happy to address this in a separate question.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Algorithm&lt;/strong&gt; Lets say &lt;em&gt;r&lt;/em&gt; or &lt;em&gt;r&lt;/em&gt;&lt;sup&gt;2&lt;/sup&gt; with Sample 1 versus Sample 2 ... the correlation value goes in one half of a triangular matrix with zeros running down the centre of the matrix.  You then proceed for every pairwise comparison until you get a triangle with Sample 1, Sample 2, Sample 3, etc ... along the top and  Sample 1, Sample 2, Sample 3, etc  ... along the side. That will import to MegaX (its a long while since I've used it BTW).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Distance Methods&lt;/strong&gt; If I was performing this I wouldn't use  &lt;em&gt;r&lt;/em&gt; or &lt;em&gt;r&lt;/em&gt;&lt;sup&gt;2&lt;/sup&gt;: I would use either Jaccard, Euclidean or Cosine (very trendy right now) distances. Cosine is trendy because taps into tensors (Tensorflow and all that). &lt;strong&gt;Scipy&lt;/strong&gt; will almost certainly do the distance methods listed here for your data and output a nice triangular matrix.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Personally if I was coding this I would absolutely perform a bootstrap, but this is a lot of coding.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Issues&lt;/strong&gt; The issue with distance metrics without bootstrapping is that they give different solutions and there's no way identify which tree is the best tree. Bootstrapping helps a lot here and values &amp;gt; 80% are considered robust topologies. I've never personally known two distance metrics produce an incongruence for the same data under bootstrapping. The distance matrix issue was why everyone moved (and rigidly stuck with) formal models via maximum likelihood or Bayes - but these require alignments.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; Think you can go alignment free? Well yeah you can, but remember there's no such thing as a free lunch ;-)&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-10-11T19:32:48.603" LastActivityDate="2023-10-11T19:32:48.603" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21680" PostTypeId="1" AcceptedAnswerId="21681" CreationDate="2023-10-12T04:43:51.343" Score="0" ViewCount="82" Body="&lt;p&gt;I wonder how can I calculate the average of column 7 (WPS_win) when consecutive cells in column 2, column 3, and column 1 are the same. The file is in bed format (tab-separated format). The original file is very large. The first few lines of the file look like this,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chrom   start_pr    end_pr  width_pr    start_win   end_win WPS_win .overlap&#xA;chr1    981918  984117  2200    981900  982000  -5  82&#xA;chr1    981918  984117  2200    982000  982100  -1  100&#xA;chr1    981918  984117  2200    982200  982300  -2  100&#xA;chr1    981918  984117  2200    982300  982400  2   100&#xA;chr1    981918  984117  2200    982400  982500  -3  100&#xA;chr1    981918  984117  2200    982500  982600  -1  100&#xA;chr1    981918  984117  2200    982600  982700  -3  100&#xA;chr1    981918  984117  2200    982700  982800  1   100&#xA;chr1    981918  984117  2200    982800  982900  -5  100&#xA;chr1    981918  984117  2200    982900  983000  0   100&#xA;chr1    981918  984117  2200    983000  983100  0   100&#xA;chr1    981918  984117  2200    983100  983200  -5  100&#xA;chr1    981918  984117  2200    983200  983300  -1  100&#xA;chr1    981918  984117  2200    983300  983400  -3  100&#xA;chr1    981918  984117  2200    983400  983500  1   100&#xA;chr1    981918  984117  2200    983500  983600  -2  100&#xA;chr1    981918  984117  2200    983600  983700  1   100&#xA;chr1    981918  984117  2200    983700  983800  -3  100&#xA;chr1    981918  984117  2200    983800  983900  1   100&#xA;chr1    981918  984117  2200    983900  984000  -4  100&#xA;chr1    981918  984117  2200    984000  984100  -1  100&#xA;chr1    981918  984117  2200    984100  984200  -2  17&#xA;chr1    999138  1001337 2200    999100  999200  1   62&#xA;chr1    999138  1001337 2200    999200  999300  -1  100&#xA;chr1    999138  1001337 2200    999700  999800  -1  100&#xA;chr1    999138  1001337 2200    999800  999900  -2  100&#xA;chr1    999138  1001337 2200    999900  1000000 -2  100&#xA;chr1    999138  1001337 2200    1000000 1000100 1   100&#xA;chr1    999138  1001337 2200    1000100 1000200 -2  100&#xA;chr1    999138  1001337 2200    1000200 1000300 0   100&#xA;chr1    999138  1001337 2200    1000300 1000400 -1  100&#xA;chr1    999138  1001337 2200    1000400 1000500 -2  100&#xA;chr1    999138  1001337 2200    1000500 1000600 -2  100&#xA;chr1    999138  1001337 2200    1000600 1000700 2   100&#xA;chr1    999138  1001337 2200    1000700 1000800 -1  100&#xA;chr1    999138  1001337 2200    1000800 1000900 -2  100&#xA;chr1    999973  1002172 2200    999900  1000000 -2  27&#xA;chr1    999973  1002172 2200    1000000 1000100 1   100&#xA;chr1    999973  1002172 2200    1000100 1000200 -2  100&#xA;chr1    999973  1002172 2200    1000200 1000300 0   100&#xA;chr1    999973  1002172 2200    1000300 1000400 -1  100&#xA;chr1    999973  1002172 2200    1000400 1000500 -2  100&#xA;chr1    999973  1002172 2200    1000500 1000600 -2  100&#xA;chr1    999973  1002172 2200    1000600 1000700 2   100&#xA;chr1    999973  1002172 2200    1000700 1000800 -1  100&#xA;chr1    999973  1002172 2200    1000800 1000900 -2  100&#xA;chr1    999973  1002172 2200    1001600 1001700 -1  100&#xA;chr1    999973  1002172 2200    1001700 1001800 -2  100&#xA;chr1    999973  1002172 2200    1001800 1001900 1   100&#xA;chr1    999973  1002172 2200    1001900 1002000 -1  100&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In the above table, the first 22 rows of cols 1, 2, and 3 are the same. Therefore, The expected output would look like this,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chrom   start_pr    end_pr  Mean_WPS&#xA;chr1    981918  984117  -1.590909091&#xA;chr1    999138  1001337 -0.857142857&#xA;chr1    999973  1002172 -0.857142857&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm open to both R and Bash. Any suggestion to overcome this issue would be heartily appreciated.&lt;/p&gt;&#xA;" OwnerUserId="12770" LastActivityDate="2023-10-12T07:47:27.773" Title="How to calculate average from a column when consecutive cells are similar in different columns?" Tags="|r|genomics|bash|tsv|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21681" PostTypeId="2" ParentId="21680" CreationDate="2023-10-12T06:11:07.893" Score="2" Body="&lt;pre&gt;&lt;code&gt;library(dplyr)&#xA;cols_of_interest &amp;lt;- c(&amp;quot;chrom&amp;quot;, &amp;quot;start_pr&amp;quot;, &amp;quot;end_pr&amp;quot;)&#xA;&#xA;test_data |&amp;gt;&#xA;  group_by(across(all_of(cols_of_interest))) |&amp;gt;&#xA;  summarize(Mean_WPS = mean(WPS_win))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;gives the following with your test data above:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# A tibble: 3 × 4&#xA;# Groups:   chrom, start_pr [3]&#xA;  chrom start_pr  end_pr Mean_WPS&#xA;  &amp;lt;chr&amp;gt;    &amp;lt;int&amp;gt;   &amp;lt;int&amp;gt;    &amp;lt;dbl&amp;gt;&#xA;1 chr1    981918  984117   -1.59 &#xA;2 chr1    999138 1001337   -0.857&#xA;3 chr1    999973 1002172   -0.857&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/a/34487771/7236472&quot;&gt;Ref for the code used above&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2023-10-12T06:11:07.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21682" PostTypeId="2" ParentId="21680" CreationDate="2023-10-12T07:39:58.153" Score="0" Body="&lt;p&gt;I use &lt;code&gt;bedtools merge&lt;/code&gt; with a little trick to combine the first three columns into a single one to &amp;quot;mock&amp;quot; each of the unique intervals into one &amp;quot;chromosome&amp;quot; so bedtools easily knows that these are the same:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;cat example.bed \&#xA;| awk 'NR &amp;gt; 1 {OFS=&amp;quot;\t&amp;quot;; print &lt;span class=&quot;math-container&quot;&gt;$1&quot;--&quot;$&lt;/span&gt;2&amp;quot;--&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$3&quot;--&quot;, 1, 1, $&lt;/span&gt;7}' \&#xA;| bedtools merge -i - -c 4 -o mean \&#xA;| awk 'BEGIN {OFS=&amp;quot;\t&amp;quot;; print &amp;quot;chrom&amp;quot;, &amp;quot;start_pr&amp;quot;, &amp;quot;end_pr&amp;quot;, &amp;quot;Mean_WPS&amp;quot;} {split(&lt;span class=&quot;math-container&quot;&gt;$1,x,&quot;--&quot;); print x[1], x[2], x[3], $&lt;/span&gt;4}'&#xA;&#xA;chrom   start_pr        end_pr  Mean_WPS&#xA;chr1    981918  984117  -1.590909091&#xA;chr1    999138  1001337 -0.8571428571&#xA;chr1    999973  1002172 -0.8571428571&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;First &lt;code&gt;awk&lt;/code&gt; command: Skips first line, combines first three columns into one and prints this together with a single position for start and end plus the score.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;bedtools&lt;/code&gt; command: Merges the identical intervals, taking the mean of the fourth column.&lt;/p&gt;&#xA;&lt;p&gt;Second &lt;code&gt;awk&lt;/code&gt; command: Splits the first column again into the former chrom-start-end and puts the average score in column 4, and also adds a header line.&lt;/p&gt;&#xA;&lt;p&gt;Memory footprint of this should be minimal.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastEditorUserId="16676" LastEditDate="2023-10-12T07:47:27.773" LastActivityDate="2023-10-12T07:47:27.773" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21684" PostTypeId="1" AcceptedAnswerId="21711" CreationDate="2023-10-12T10:30:35.340" Score="0" ViewCount="149" Body="&lt;p&gt;I have a use case where I want to annotate the variants from running Oxford Nanopore's Medaka on a very simple reference, a single sequence starting on an ATG and finishing on a stop codon (no introns).&lt;/p&gt;&#xA;&lt;p&gt;Which of the variant effect prediction tools would be the easiest to use to programmatically create the .gtf annotation file and build a reference upon which one can then call the variant annotation, and get the different types of variant consequences on the vcf file (synonymous, non-synonymous with the pair of aa's involved, frame-shifts, etc.)&lt;/p&gt;&#xA;&lt;p&gt;So far I have looked at:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;snpEff: it seems to have a way of building the reference, although it's not straightforward to match the built java .jar with the one in the system.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;annovar: I couldn't find the documentation on how to supply your own annotations, only how to download the human GRCh38 or equivalents.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;VEP: I couldn't find the documentation on how to supply your own annotations, only how to download the human GRCh38 or equivalents.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2023-10-16T10:17:46.547" Title="which SNP effect software, such as VEP, annovar or snpEff, allows the user to submit their own annotations in gtf format?" Tags="|variant-calling|snpeff|annovar|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21685" PostTypeId="2" ParentId="21414" CreationDate="2023-10-12T10:32:54.237" Score="1" Body="&lt;p&gt;I didn't manage to try the ones reported by @gringer and @alexlok here, but I did manage to perform an analysis using the OneCodex webapp: &lt;a href=&quot;https://app.onecodex.com/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://app.onecodex.com/&lt;/a&gt; and based on the experimental information for the samples being analysed, the results were concordant with the expectation. You can do quite a lot on the &amp;quot;free tier&amp;quot;, so at the moment I didn't upgrade to the paid version.&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2023-10-12T10:32:54.237" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21686" PostTypeId="1" AcceptedAnswerId="21698" CreationDate="2023-10-12T10:38:18.077" Score="2" ViewCount="37" Body="&lt;p&gt;I have a dataset with lots of DNA sequences (~10 M seqs, each ~6 KBp long).&#xA;I want to detect presence of a fixed given protein (~600 Bp / 200 aa) within each sequence and, if detected, obtain the frame and position of the protein within the sequence. However, the protein may be coded arbitrarily and differently in each sequence. In addition, I need to account for frameshifts induced by sequencing errors.&lt;/p&gt;&#xA;&lt;p&gt;I've tried to use &lt;code&gt;tfasty36&lt;/code&gt; from the &lt;a href=&quot;https://github.com/wrpearson/fasta36&quot; rel=&quot;nofollow noreferrer&quot;&gt;FASTA36&lt;/a&gt; package. This is basically a &lt;code&gt;tblastn&lt;/code&gt; equivalent (protein query search against DNA database in all frames). Due to the statistical filtering built into these tools, I've come to the conclusion that &lt;code&gt;tfasty36&lt;/code&gt; and &lt;code&gt;tblastn&lt;/code&gt; are not the right tools for this job.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;tfasty36&lt;/code&gt; didn't work because it requires fiddling with statistical tolerances to show all matches. Also, the matches are quite fragmented (I guess, it uses heuristics instead of dynamic programming). For &lt;code&gt;tblastn&lt;/code&gt;, building the database seems to be unnecessary (given that it may have the same problems as &lt;code&gt;tfasty36&lt;/code&gt; afterwards).&lt;/p&gt;&#xA;&lt;p&gt;I've looked into alignment tools, but couldn't find any that can align a protein sequence to a DNA sequence while taking frameshifts into account.&lt;/p&gt;&#xA;&lt;p&gt;How can I solve this problem?&lt;/p&gt;&#xA;" OwnerUserId="18357" LastEditorUserId="298" LastEditDate="2023-10-13T13:46:42.287" LastActivityDate="2023-10-13T13:46:42.287" Title="Find protein in DNA sequences with arbitrary encoding and possible frameshifts" Tags="|sequence-alignment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21687" PostTypeId="1" CreationDate="2023-10-12T11:07:50.823" Score="1" ViewCount="37" Body="&lt;p&gt;I've encountered an issue that I'm hoping someone can help with. I constructed a phylogenetic tree using &lt;strong&gt;BEAST1&lt;/strong&gt; and then extracted the Maximum Credibility Tree using treeannotator. I'm trying to import it into &lt;strong&gt;R&lt;/strong&gt; for editing, but I run into an error when using the &lt;code&gt;read.beast()&lt;/code&gt; function from the &lt;strong&gt;treeio&lt;/strong&gt; package:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MCC_tree &amp;lt;- read.beast(&amp;quot;/home/MyPC/Downloads/MCC_beast.tre&amp;quot;)&#xA;Error in FUN(X[[i]], ...) : &#xA;  numbers of left and right parentheses in Newick string not equal&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Interestingly, when I attempt to visualize the tree using figtree, it works fine. Likewise, importing into R using &lt;code&gt;read.nexus()&lt;/code&gt; works without issues. However, I want to use &lt;code&gt;read.beast()&lt;/code&gt; specifically because I need to annotate the &amp;quot;posterior&amp;quot;, which I can't do when I import it using &lt;code&gt;read.nexus()&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Has anyone encountered a similar issue or have any suggestions on how to resolve this?&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="12707" LastEditorUserId="3848" LastEditDate="2023-10-12T12:11:37.703" LastActivityDate="2024-03-10T14:06:15.867" Title="Error importing Maximum Credibility Tree from BEAST into R using treeio's read.beast()" Tags="|r|phylogenetics|phylogeny|bioconductor|ggtree|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21688" PostTypeId="2" ParentId="21687" CreationDate="2023-10-12T12:11:18.287" Score="0" Body="&lt;ol&gt;&#xA;&lt;li&gt;if you read it into FigTree then in the export in Nexus and perform a &amp;quot;save all trees&amp;quot;  it should then read into &lt;code&gt;read.beast()&lt;/code&gt;.&lt;/li&gt;&#xA;&lt;li&gt;Use the beast file output before you performed tree annotator operation.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;1 or 2. should be read directly into &lt;code&gt;read.beast()&lt;/code&gt;.&#xA;I would try option 2 first.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-10-12T13:23:07.753" LastActivityDate="2023-10-12T13:23:07.753" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21689" PostTypeId="2" ParentId="21684" CreationDate="2023-10-12T14:36:35.157" Score="2" Body="&lt;p&gt;All of them have ways you can add your own annotation dataset but I've found the easiest way to be to use &lt;a href=&quot;https://samtools.github.io/bcftools/bcftools.html#annotate&quot; rel=&quot;nofollow noreferrer&quot;&gt;bcftools annotate&lt;/a&gt;. You can give it a tab-delimited (GTF is tab-delimited, but might need some pre-processing to get relevant values from column-9) or VCF file and have it pick specific columns/attributes, match the locus by any subset of CHR, POS, REF, ALT and add the annotation in a variety of custom manners. &lt;a href=&quot;https://github.com/brentp/vcfanno&quot; rel=&quot;nofollow noreferrer&quot;&gt;vcfanno&lt;/a&gt; is another option.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-10-12T14:36:35.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21690" PostTypeId="1" AcceptedAnswerId="21691" CreationDate="2023-10-12T17:10:41.883" Score="0" ViewCount="37" Body="&lt;p&gt;I want a script in R, which creates &lt;code&gt;M&lt;/code&gt; fragments of DNA with a length of &lt;code&gt;I&lt;/code&gt; but with &lt;strong&gt;GC content&lt;/strong&gt; of &amp;gt;70%&lt;/p&gt;&#xA;&lt;p&gt;I know this gives something but not exactly what I am looking for&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;require(Biostrings)&#xA;&#xA;set = NULL&#xA;for (i in 1:4)&#xA;  set = c(set, paste(sample(DNA_ALPHABET[1:4], 3, replace=T), collapse=&amp;quot;&amp;quot;))&#xA;rep(set,2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any idea please?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="650" LastEditDate="2023-10-12T18:58:19.790" LastActivityDate="2023-10-12T19:07:20.660" Title="A function to produce M fragments of DNA with length of I" Tags="|r|for-loop|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21691" PostTypeId="2" ParentId="21690" CreationDate="2023-10-12T18:52:56.560" Score="3" Body="&lt;p&gt;There are different ways to approach the GC bias, the easiest is probably to decide beforehand how many &amp;quot;A/T&amp;quot; and how many &amp;quot;G/C&amp;quot; we need in each sequence, then choose them randomly.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# set global parameters&#xA;M &amp;lt;- 7&#xA;I &amp;lt;- 15&#xA;&#xA;# for each sequence, choose a proportion of GC&#xA;prop_gc &amp;lt;- runif(M,&#xA;                 min = .7,&#xA;                 max = 1)&#xA;&#xA;# randomly choose bases, ensure we have a number that gives us at least `prop_gc`&#xA;bases &amp;lt;- lapply(prop_gc,&#xA;                \(.prop_gc){&#xA;                  c(&#xA;                    sample(c(&amp;quot;A&amp;quot;,&amp;quot;T&amp;quot;),&#xA;                           size = floor(I*(1 - .prop_gc)), replace = TRUE),&#xA;                    sample(c(&amp;quot;G&amp;quot;,&amp;quot;C&amp;quot;),&#xA;                           size = ceiling(I*.prop_gc), replace = TRUE)&#xA;                  )&#xA;                })&#xA;&#xA;# now we can simply take our bases and combine them (in a random order)&#xA;sequences &amp;lt;- sapply(bases,&#xA;                    \(.nt) sample(.nt) |&amp;gt; paste(collapse = &amp;quot;&amp;quot;))&#xA;&#xA;sequences&#xA;#&amp;gt; [1] &amp;quot;CGGCTTCCGCGCGGGT&amp;quot; &amp;quot;CTACCCACGGCGCCGG&amp;quot; &amp;quot;GGGCGGTGTATGTGCC&amp;quot; &amp;quot;GGCCCGGGTGCCCAGC&amp;quot;&#xA;#&amp;gt; [5] &amp;quot;GCGGGCGCGGGCGGGT&amp;quot; &amp;quot;CACCCCCTCCCGGGAC&amp;quot; &amp;quot;GTCGAAGCGCGCCAGC&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;sup&gt;Created on 2023-10-12 with &lt;a href=&quot;https://reprex.tidyverse.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;reprex v2.0.2&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;p&gt;Or same thing with a &lt;code&gt;for&lt;/code&gt; loop (since you gave it as a tag):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;M &amp;lt;- 7&#xA;I &amp;lt;- 15&#xA;&#xA;sequences &amp;lt;- character(length = M)&#xA;for(i in seq_len(M)){&#xA;  prop_gc &amp;lt;- runif(1,&#xA;                   min = .7,&#xA;                   max = 1)&#xA;  &#xA;  bases &amp;lt;- c(&#xA;    sample(c(&amp;quot;A&amp;quot;,&amp;quot;T&amp;quot;),&#xA;           size = floor(I * (1 - prop_gc)), replace = TRUE),&#xA;    sample(c(&amp;quot;G&amp;quot;,&amp;quot;C&amp;quot;),&#xA;           size = ceiling(I * prop_gc), replace = TRUE)&#xA;  )&#xA;  &#xA;  sequences[[i]] &amp;lt;- sample(bases) |&amp;gt; paste(collapse = &amp;quot;&amp;quot;)&#xA;  &#xA;}&#xA;sequences&#xA;#&amp;gt; [1] &amp;quot;CCCCGGTTGCGTTTCC&amp;quot; &amp;quot;CCTCGCTGCGCGTGGC&amp;quot; &amp;quot;CAGTCTCCCGCGGGGC&amp;quot; &amp;quot;GGGCCCGCCGAAAGGC&amp;quot;&#xA;#&amp;gt; [5] &amp;quot;GCGCGGGGCCGGTCAG&amp;quot; &amp;quot;CGGGCGCCCTGGACAC&amp;quot; &amp;quot;ACTCTGCGGGGGAGTC&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;sup&gt;Created on 2023-10-12 with &lt;a href=&quot;https://reprex.tidyverse.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;reprex v2.0.2&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;p&gt;And we can doublecheck the results:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;M &amp;lt;- 7000&#xA;I &amp;lt;- 500&#xA;&#xA;[...]&#xA;&#xA;# check&#xA;gc_prop &amp;lt;- function(x){&#xA;  toupper(x) |&amp;gt;&#xA;    strsplit(&amp;quot;&amp;quot;) |&amp;gt;&#xA;    sapply(\(.char) mean(.char == &amp;quot;C&amp;quot; | .char == &amp;quot;G&amp;quot;))&#xA;}&#xA;&#xA;sapply(sequences, nchar) |&amp;gt; table()&#xA;#&amp;gt; &#xA;#&amp;gt;  500 &#xA;#&amp;gt; 7000&#xA;&#xA;min(gc_prop(sequences))&#xA;#&amp;gt; [1] 0.702&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;sup&gt;Created on 2023-10-12 with &lt;a href=&quot;https://reprex.tidyverse.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;reprex v2.0.2&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;" OwnerUserId="9497" LastEditorUserId="9497" LastEditDate="2023-10-12T19:07:20.660" LastActivityDate="2023-10-12T19:07:20.660" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21692" PostTypeId="1" AcceptedAnswerId="21694" CreationDate="2023-10-12T23:16:31.137" Score="1" ViewCount="68" Body="&lt;p&gt;I have a pairwise matrix that I've manually formatted into a &amp;quot;.mega&amp;quot; file following the guide in the MegaX documentation but when I go to import it into the application it gives this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Unable to find the command terminator '&#xA;' (in line 48)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I don't understand what the terminator is supposed to be I can't find anything in the documentation or online. This is what my file looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#mega&#xA;TITLE: Lower-left triangular matrix&#xA;&#xA;#sample1&#xA;#sample2&#xA;#sample3&#xA;#sample4&#xA;#sample5&#xA;#sample6&#xA;#sample7&#xA;#sample8&#xA;#sample9&#xA;#sample10&#xA;#sample11&#xA;#sample12&#xA;#sample13&#xA;#sample14&#xA;#sample15&#xA;#sample16&#xA;#sample17&#xA;#sample18&#xA;#sample19&#xA;#sample20&#xA;#sample21&#xA;#sample22&#xA;        &#xA;6779.024&#xA;4010.596    5085.012&#xA;4444.925    3965.719    3597.515&#xA;3824.67     4435.878    2140.942    2855.545&#xA;2698.035    5877.724    4274.36     3721.735    3562.716&#xA;5746.71     4991.678    3103.77     4587.394    3318.78     6141.058&#xA;2898.099    7345.178    5461.126    5000.322    4846.293    2354.302    7366.433&#xA;4667.862    2949.764    3986.194    2832.598    3118.544    3551.237    5007.972    5056.938&#xA;3956.69     5110.169    2489.003    3530.347    2310.696    4285.076    3286.188    5301.894    3994.313&#xA;4086.841    5636.588    2851.673    3681.433    3078.399    4197.933    4416.542    5139.341    4347.211    3250.879&#xA;3595.868    6237.556    3924.761    3907.091    3624.064    3081.475    5770.594    3985.626    4223.117    4013.377    3136.24&#xA;4574.567    5763.568    2420.15     4433.579    3203.997    5015.929    3445.061    6063.541    4820.971    3131.669    2940.224    4132.314&#xA;4621.723    3265.735    3518.476    2528.629    2685.442    3581.311    4361.296    5144.475    2322.969    3536.469    3942.852    4218.651    4431.758&#xA;4914.659    3750.498    2719.845    3117.886    2456.598    4411.935    3294.878    5908.595    3261.36     3157.398    3657.63     4540.83     3583.543    2199.326&#xA;5662.093    3090.623    3190.918    3289.562    2772.944    5317.999    2559.264    6786.94     3426.171    3247.437    4233.766    5294.141    3960.851    2656.578    2155.925&#xA;5559.707    3073.796    4997.709    3838.601    4294.455    4486.788    5728.029    5962.611    2303.294    5051.106    5270.304    5020.468    5639.198    3504.092    4111.158    4282.95&#xA;3906.494    4028.613    3618.28     2618.348    2835.36     2743.097    4941.507    4259.628    2191.873    3599.004    3847.302    3506.872    4432.612    2061.23     2947.972    3583.198    3342.373&#xA;6582.561    1663.359    4830.862    3730.956    4161.803    5594.268    4860.298    7160.793    2675.27     4859.183    5448.895    5895.575    5537.159    3011.78     3374.505    2881.87     2979.42     3464.931&#xA;4706.715    4004.427    4618.035    3537.473    3842.462    3773.964    5703.134    5002.92     2263.311    4592.613    4872.922    4301.443    5290.578    3530.202    4175.393    4494.308    2427.841    2677.65     3599.691&#xA;5703.386    2214.959    4515.044    3273.191    3766.439    4681.361    5022.943    6197.46     1837.686    4544.365    5026.108    5096.364    5257.581    2833.977    3419.825    3267.254    2034.999    2778.249    1661.474    2332.169&#xA;6689.288    11865.223   8990.382    9113.452    8897.297    7007.562    10983.956   6142.025    9502.64     9014.072    8056.924    6431.134    8821.925    9843.732    10166.579   11038.532   10009.974   8748.955    11573.491   8952.942    10488.639&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt;&#xA;I copied the contents of the &lt;code&gt;.meg&lt;/code&gt; file over to an Excel sheet to make it easier to see but I changed the &lt;code&gt;#Mega&lt;/code&gt; to &lt;code&gt;#mega&lt;/code&gt; and I made sure to only include the correct elements (there are 22 samples in total, only 21 values present in the first line of the matrix). I also made sure to keep a line between the last sample name and the first value of the matrix. However, it's &lt;em&gt;still&lt;/em&gt; giving me the same error.&lt;/p&gt;&#xA;&lt;p&gt;Is there something I'm supposed to add at the end of the file to make it read the file correctly? I'm  trying to import this distance matrix so I can generate a NJ tree. I'm not sure what I'm doing differently than what is mentioned &lt;a href=&quot;https://www.megasoftware.net/mega1_manual/DataFormat.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="16067" LastEditDate="2023-10-13T18:50:44.447" LastActivityDate="2023-10-14T01:28:16.340" Title="MegaX Pairwise Matrix Error" Tags="|megax|distance-matrix|pairwise|neighbor-joining|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21693" PostTypeId="1" CreationDate="2023-10-12T23:55:39.177" Score="5" ViewCount="320" Body="&lt;p&gt;I successfully completed &lt;a href=&quot;https://choishingwan.github.io/PRS-Tutorial/&quot; rel=&quot;noreferrer&quot;&gt;Nature PRS tutorial&lt;/a&gt;, which is based on PLINK. Turning to my real data, I downloaded &lt;a href=&quot;https://gwas.mrcieu.ac.uk/datasets/ukb-d-20544_1/&quot; rel=&quot;noreferrer&quot;&gt;ukb-d-20544_1.vcf.gz&lt;/a&gt;. Now I'm facing the problem that I seem to be unable to use it in PLINK or find the correct data format to download at all, and I am a bit confused about the correct general summary statistics-download procedure.&lt;/p&gt;&#xA;&lt;h3&gt;1. Convert .vcf format to a .txt file, PLINK can understand&lt;/h3&gt;&#xA;&lt;p&gt;In the tutorial they use a &lt;code&gt;.txt.gz&lt;/code&gt; base data file aka summary statistics that looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CHR BP  SNP A1  A2  N   SE  P   OR  INFO    MAF&#xA;1   756604  rs3131962   A   G   388028  0.00301666  0.483171    0.997886915712657   0.890557941364774   0.369389592764921&#xA;1   768448  rs12562034  A   G   388028  0.00329472  0.834808    1.00068731609353    0.895893511351165   0.336845754096289&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Having this &lt;code&gt;.vcf.gz&lt;/code&gt; (please see excerpt at the very bottom), I unzipped and, in R, used &lt;code&gt;MungeSumstats::format_sumstats(&amp;quot;ukb-d-20544_1.vcf&amp;quot;, ref_genome = &amp;quot;GRCh37&amp;quot;)&lt;/code&gt;&lt;a href=&quot;https://stackoverflow.com/a/77257300/6574038&quot;&gt;[1]&lt;/a&gt;, to get something out of it similar to what they use in the .txt file in the tutorial. However, the column names won't match exactly. I know that I can use &lt;em&gt;BETA&lt;/em&gt; instead of &lt;em&gt;OR&lt;/em&gt;, and &lt;em&gt;FRQ&lt;/em&gt; might correspond to &lt;em&gt;MAF&lt;/em&gt;, but I can't seem to find the imputation information score &lt;em&gt;INFO&lt;/em&gt;. Here is what &lt;code&gt;MungeSumstats::format_sumstats&lt;/code&gt; produced:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SNP CHR BP  A1  A2  END FILTER  FRQ BETA    SE  LP  N   P&#xA;rs12238997  1   693731  A   G   693731  PASS    0.116734    0.00058531  0.000756397 0.357493    117706  0.439042942120051&#xA;rs371890604 1   707522  G   C   707522  PASS    0.0983981   0.000793177 0.000848559 0.456023    117706  0.349926634613021&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I need to have the columns corresponding to MAF and INFO as per the tutorial.&lt;/p&gt;&#xA;&lt;p&gt;The next step in the tutorial involves filtering based on specific criteria, such as MAF and INFO values. However, I can't proceed because these columns are missing.&lt;/p&gt;&#xA;&lt;p&gt;I would like to primarily use the terminal and only turn to R for statistical analysis. Could you guide me on the correct approach to convert the .vcf file to .txt so that I can continue following the tutorial with PLINK?&lt;/p&gt;&#xA;&lt;h3&gt;2. General approach&lt;/h3&gt;&#xA;&lt;p&gt;In addition to addressing the conversion issue, I'm also seeking guidance on the appropriate data sources for genetic analysis. I'm curious about whether OpenGWAS is a reliable source for publication or if there are other recommended sources like dbGaP. Additionally, what is the ideal data format to obtain?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The beginning of the received .vcf file looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.2&#xA;##FILTER=&amp;lt;ID=PASS,Description=&amp;quot;All filters passed&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=AF,Number=A,Type=Float,Description=&amp;quot;Allele Frequency&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=ES,Number=A,Type=Float,Description=&amp;quot;Effect size estimate relative to the alternative allele&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=SE,Number=A,Type=Float,Description=&amp;quot;Standard error of effect size estimate&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=LP,Number=A,Type=Float,Description=&amp;quot;-log10 p-value for effect estimate&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=AF,Number=A,Type=Float,Description=&amp;quot;Alternate allele frequency in the association study&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=SS,Number=A,Type=Float,Description=&amp;quot;Sample size used to estimate genetic effect&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=EZ,Number=A,Type=Float,Description=&amp;quot;Z-score provided if it was used to derive the EFFECT and SE fields&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=SI,Number=A,Type=Float,Description=&amp;quot;Accuracy score of summary data imputation&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=NC,Number=A,Type=Float,Description=&amp;quot;Number of cases used to estimate genetic effect&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=ID,Number=1,Type=String,Description=&amp;quot;Study variant identifier&amp;quot;&amp;gt;&#xA;##META=&amp;lt;ID=TotalVariants,Number=1,Type=Integer,Description=&amp;quot;Total number of variants in input&amp;quot;&amp;gt;&#xA;##META=&amp;lt;ID=VariantsNotRead,Number=1,Type=Integer,Description=&amp;quot;Number of variants that could not be read&amp;quot;&amp;gt;&#xA;##META=&amp;lt;ID=HarmonisedVariants,Number=1,Type=Integer,Description=&amp;quot;Total number of harmonised variants&amp;quot;&amp;gt;&#xA;##META=&amp;lt;ID=VariantsNotHarmonised,Number=1,Type=Integer,Description=&amp;quot;Total number of variants that could not be harmonised&amp;quot;&amp;gt;&#xA;##META=&amp;lt;ID=SwitchedAlleles,Number=1,Type=Integer,Description=&amp;quot;Total number of variants strand switched&amp;quot;&amp;gt;&#xA;##META=&amp;lt;ID=TotalControls,Number=1,Type=Integer,Description=&amp;quot;Total number of controls in the association study&amp;quot;&amp;gt;&#xA;##META=&amp;lt;ID=TotalCases,Number=1,Type=Integer,Description=&amp;quot;Total number of cases in the association study&amp;quot;&amp;gt;&#xA;##META=&amp;lt;ID=StudyType,Number=1,Type=String,Description=&amp;quot;Type of GWAS study [Continuous or CaseControl]&amp;quot;&amp;gt;&#xA;##SAMPLE=&amp;lt;ID=ukb-d-20544_1,TotalVariants=9938650,VariantsNotRead=0,HarmonisedVariants=9938650,VariantsNotHarmonised=0,SwitchedAlleles=209,StudyType=Continuous&amp;gt;&#xA;##contig=&amp;lt;ID=1,length=249250621,assembly=HG19/GRCh37&amp;gt;&#xA;##contig=&amp;lt;ID=2,length=243199373,assembly=HG19/GRCh37&amp;gt;&#xA;...&#xA;##contig=&amp;lt;ID=22,length=51304566,assembly=HG19/GRCh37&amp;gt;&#xA;##contig=&amp;lt;ID=X,length=155270560,assembly=HG19/GRCh37&amp;gt;&#xA;##contig=&amp;lt;ID=Y,length=59373566,assembly=HG19/GRCh37&amp;gt;&#xA;##contig=&amp;lt;ID=MT,length=16569,assembly=HG19/GRCh37&amp;gt;&#xA;##contig=&amp;lt;ID=GL000207.1,length=4262,assembly=HG19/GRCh37&amp;gt;&#xA;##contig=&amp;lt;ID=GL000226.1,length=15008,assembly=HG19/GRCh37&amp;gt;&#xA;...&#xA;##contig=&amp;lt;ID=GL000192.1,length=547496,assembly=HG19/GRCh37&amp;gt;&#xA;##gwas_harmonisation_command=--json /mnt/storage/private/mrcieu/research/scratch/IGD/data/dev/ukb-d-import/processed/ukb-d-20544_1/ukb-d-20544_1_data.json --ref /mnt/storage/private/mrcieu/research/scratch/IGD/data/dev/QC/genomes/b37/human_g1k_v37.fasta; 1.1.1&#xA;##file_date=2019-11-25T15:12:05.258952&#xA;##bcftools_annotateVersion=1.9-74-g6af271c+htslib-1.9-64-g226b4a8&#xA;##bcftools_annotateCommand=annotate -a /mnt/storage/home/gh13047/mr-eve/vcf-reference-datasets/dbsnp/dbsnp.v153.b37.vcf.gz -c ID -o /mnt/storage/private/mrcieu/research/scratch/IGD/data/dev/ukb-d-import/processed/ukb-d-20544_1/ukb-d-20544_1.vcf.gz -O z /mnt/storage/private/mrcieu/research/scratch/IGD/data/dev/ukb-d-import/processed/ukb-d-20544_1/ukb-d-20544_1_data.vcf.gz; Date=Mon Nov 25 15:37:00 2019&#xA;#CHROM  POS     ID      REF     ALT     QUAL    FILTER  INFO    FORMAT  ukb-d-20544_1&#xA;1       692794  rs530212009     CA      C       .       PASS    AF=0.111693     ES:SE:LP:AF:SS:ID       0.000379398:0.000798869:0.197332:0.111693:117706:1_692794_CA_C&#xA;1       693731  rs12238997      A       G       .       PASS    AF=0.116734     ES:SE:LP:AF:SS:ID       0.00058531:0.000756397:0.357493:0.116734:117706:rs12238997&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14153" LastEditorUserId="14153" LastEditDate="2023-10-13T00:04:16.767" LastActivityDate="2023-10-18T17:01:47.113" Title="Converting VCF format to text for use with PLINK and understanding column mapping" Tags="|vcf|public-databases|plink|format-conversion|polygenic-risk-score|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21694" PostTypeId="2" ParentId="21692" CreationDate="2023-10-13T00:42:38.480" Score="1" Body="&lt;p&gt;Criteria, matrix. There must be a blank line between the last sample name and the start of the matrix. Also better to use #mega.&lt;/p&gt;&#xA;&lt;p&gt;The tree from the matrix in the question is.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FRhO4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FRhO4.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It's simply the carriage returns &lt;strong&gt;must&lt;/strong&gt; by in Windows format &lt;code&gt;\r&lt;/code&gt; (even if its Linux or Mac) and &lt;strong&gt;not&lt;/strong&gt; Mac or Linux &lt;code&gt;\n&lt;/code&gt;. In Mac the converter is BBEdit.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-10-14T01:28:16.340" LastActivityDate="2023-10-14T01:28:16.340" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21695" PostTypeId="1" AcceptedAnswerId="21703" CreationDate="2023-10-13T05:09:54.520" Score="2" ViewCount="79" Body="&lt;p&gt;I have a list of organisms with their full scientific names and Taxonomy IDs, total ~1500. I want to BLAST my protein against this 1500 genome or proteome. How can I do that?&lt;/p&gt;&#xA;&lt;p&gt;Here a sample protein that I want to search the following database:&#xA;&lt;a href=&quot;http://togodb.org/db/thermobase&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://togodb.org/db/thermobase&lt;/a&gt;&#xA;Please note that database can be downloades as CSV file. I am very comfortable using web tools or programmatic tools such as Perl, Python, bash, or Entrez Programming Utilities.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;QCC44558.1 chitinase [Halobacterium salinarum] TaxID 641522631&#xA;MPHDRRSYLRTSSAVIASLLAASTPTSAADTPPEWDPDTVYTDGDKATFDGYVWEAKWWTKGDKPGADEW&#xA;GPWTQRRPVDGSPGGPTAAFTVTDTTVEPGTAVTVDASDTTGSVDSYEWAFGDGTTAAGVTASHTYDAAG&#xA;EYTIELTVTTGDGTTDTATQQITATTSPGDDEFKVVGYYPSWKGTDDYDFYPADVPFDQVTDVLYAFLDV&#xA;QPDGTVVLPDSDVDHESLLASFASLKQDRAADTRLKLSIGGWGLSPGFEDAAADQASRERFATTAVDLMR&#xA;TYDFDGIDVDWEHPGPRRGKCECGSAQGPANHVALLETVRDRLNDAEVEDGRTYDLSVANGGSDWNAALI&#xA;NHREVAAVVDDIYMMAYDFTGVWHGTAGLNAPIYGTPPDYPPSGDAQQYTLETTLTIWKEQGYWVDWMEW&#xA;EDHGDPVDDPGTLVLGMPFYGRGCNVENGIWDTFSLSEWQQGDPKYQNDVIPPGTWNDLRGPDDANTGAF&#xA;DYGDLAANYEGADAWTKQRNEQGGVPYLWNDSEGVFISYDDPTSIAAKVELAVAEDLGGVMIWELSQDYD&#xA;GTLLETINQHTP&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14180" LastEditorUserId="475" LastEditDate="2023-10-14T11:11:18.437" LastActivityDate="2023-10-14T13:37:29.497" Title="How to BLAST a protein against a large set of organisms?" Tags="|blast|ncbi|blastp|makeblastdb|e-utilities|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21696" PostTypeId="2" ParentId="21693" CreationDate="2023-10-13T08:18:55.337" Score="2" Body="&lt;p&gt;For the first part of question 2:  if there are other recommended sources like dbGaP.&lt;/p&gt;&#xA;&lt;p&gt;Please refer to this Databases of GWAS summary statistics:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: center;&quot;&gt;Database&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: center;&quot;&gt;Content&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;GWAS Catalog&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;GWAS summary statistics and GWAS lead SNPs reported in GWAS papers&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;GeneAtlas&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;UK Biobank GWAS summary statistics&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;Pan UKBB&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;UK Biobank GWAS summary statistics&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;GWAS Atlas&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;Collection of publicly available GWAS summary statistics with follow-up in silico analysis&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;FinnGen results&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;GWAS summary statistics released from FinnGen, a project that collected biological samples from many sources in Finland&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;Pheweb.jp&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: center;&quot;&gt;GWAS summary statistics of Biobank Japan and cross-population meta-analyses&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;&lt;code&gt;https://www.nature.com/articles/s43586-021-00056-9/tables/3&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="17226" LastActivityDate="2023-10-13T08:18:55.337" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21697" PostTypeId="1" CreationDate="2023-10-13T11:42:20.850" Score="1" ViewCount="88" Body="&lt;p&gt;I have done the limma association analysis using below model design but not sure how to  evaluate Potential systematic biases using Q-Q plots and the genomic inflation factor (k). Could you please help with this?&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;#model matrix&#xA;var&amp;lt;-model.matrix(~BMI + as.factor(Gender) + Age +CD8T +CD4T +NK + Bcell +Mono ,data=targets2)&#xA;fit&amp;lt;-lmFit(mval,var)&#xA;fit2&amp;lt;-eBayes(fit,trend=TRUE, robust=TRUE)&#xA;probe&amp;lt;-topTable(fit2,adjust=&amp;quot;BH&amp;quot;,coef=2,num=Inf)&#xA;&#xA;write.table(probe, file = &amp;quot;limma.output.txt&amp;quot;)&#xA;data &amp;lt;- read.delim(&amp;quot;limma.output.txt&amp;quot;)&#xA;head(data)&#xA;&#xA;               logFC       AveExpr.    t         P.Value  adj.P.Val         B&#xA;cg03546163  0.41018776  -0.8013218  7.85104192  1.13E-12    8.65E-07    12.3051291&#xA;cg17944885  -0.4884859  2.31974279  -5.7319645  6.33E-08    0.02431349  5.15237109&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;#For Genomic inflation factor&#xA;&#xA;# For p-values, calculate chi-squared statistic&#xA;chisq &amp;lt;- qchisq(1-data$P.Value,1)&#xA;&#xA;#Calculate lambda gc (λgc)&#xA;median(chisq)/qchisq(0.5,1)&#xA;&#xA;#QQ PLOT&#xA;&#xA;library(gaston)&#xA;pdf(&amp;quot;qq.plot.pdf&amp;quot;, width = 12, height = 12)&#xA;qqplot.pvalues(data$P.Value)&#xA;dev.off()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="1008" LastEditDate="2023-10-20T16:41:00.303" LastActivityDate="2023-10-20T16:41:00.303" Title="Q-Q plots and the genomic inflation factor (k)" Tags="|r|statistics|limma|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21698" PostTypeId="2" ParentId="21686" CreationDate="2023-10-13T12:03:12.847" Score="2" Body="&lt;p&gt;If you need to model introns and splicing, then you should use a tool that is designed for that. One of the limitations of blast and its cousins is that they only look at sequence similarity and do not attempt to join &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK62051/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HSPs&lt;/a&gt; into possible transcript models.&lt;/p&gt;&#xA;&lt;p&gt;I haven't worked on this in a long time, but back when I did, the programs I used were:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.ebi.ac.uk/about/vertebrate-genomics/software/exonerate&quot; rel=&quot;nofollow noreferrer&quot;&gt;exonerate&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Exonerate is a generic tool for pairwise sequence comparison. It allows you to align sequences using a many alignment models, either exhaustive dynamic programming or a variety of heuristics.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In this case, you would use your protein sequence as the query and the set of  DNA sequences as the target and try to align using the protein2genome model. Something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  exonerate --model protein2genome protein.pep dna.fa &amp;gt; out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will result in (many) alignments of your query protein agains the various genomic sequences. Here's an example I had handy:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ exonerate --model protein2genome -n 1 frog.histone.pep c.elegans.chrI.fa &#xA;Command line: [exonerate --model protein2genome -n 1 frog.histone.pep c.elegans.chrI.fa]&#xA;Hostname: [oregano]&#xA;&#xA;C4 Alignment:&#xA;------------&#xA;         Query: frog&#xA;        Target: CM041006.1 Caenorhabditis elegans strain DL226 chromosome I, whole genome shotgun sequence:[revcomp]&#xA;         Model: protein2genome:local&#xA;     Raw score: 732&#xA;   Query range: 32 -&amp;gt; 300&#xA;  Target range: 3039620 -&amp;gt; 3034073&#xA;&#xA;      33 : LeuGlnLysLysLeuGluGluLeuGluLeuAspGluGlnGlnArgLysArgLeuGl :      51&#xA;           |||:!!:!!:!!|||:!!:!!|||! !|||   |||! !|||! !:!!||||||..&#xA;           LeuGluGluGlnLeuLysLysLeuGlyLeuThrGluProGlnThrGlnArgLeuSe&#xA; 3039620 : CTAGAAGAACAGCTGAAAAAGCTCGGGCTCACGGAGCCACAAACTCAACGGCTCAG : 3039566&#xA;&#xA;      52 : uAlaPheLeuThrGlnLysGlnLysValGlyGluLeuLysAspAspAspPheGluL :      70&#xA;           .! !||||||  !   |||:!!   :!!  !||||||!..!!:|||   .!.:!!!&#xA;           rGluPheLeuGlnValLysGluGlyIleLysGluLeuSerGluAspMetLeuGlnT&#xA; 3039565 : CGAATTCTTGCAGGTCAAGGAGGGCATCAAGGAGTTGAGTGAGGATATGCTACAGA : 3039509&#xA;&#xA;      71 : ysValSerGluLeuGlyAlaGlyAsnGlyGlyValValPheLys  &amp;gt;&amp;gt;&amp;gt;&amp;gt; Targe :      85&#xA;             ! !.!!|||||||||  !||||||||||||||||||  !|||            &#xA;           hrGluGlyGluLeuGlyHisGlyAsnGlyGlyValValAsnLys++          &#xA; 3039508 : CTGAAGGAGAACTTGGCCATGGAAATGGAGGAGTTGTTAATAAGgt.......... : 3039462&#xA;&#xA;      86 : t Intron 1 &amp;gt;&amp;gt;&amp;gt;&amp;gt;  ValSerHisLysProThrSerLeuIleMetAlaArgLys :      97&#xA;           55 bp              !  !|||::!  !|||.!!:!!|||||||||||||||&#xA;                          ++CysValHisArgLysThrGlyValIleMetAlaArgLys&#xA; 3039461 : ...............agTGTGTTCATCGAAAAACCGGTGTAATTATGGCTCGGAAG : 3039373&#xA;&#xA;      98 : LeuIleHisLeuGluIleLysProAlaIleArgAsnGlnIleIleArgGluLeuGl :     116&#xA;           |||:!!||||||||||||||||||:!!:!!|||.!.||||||:!!!:!||||||  &#xA;           LeuValHisLeuGluIleLysProSerValArgGlnGlnIleValLysGluLeuAl&#xA; 3039372 : CTTGTACACCTCGAGATCAAGCCGTCAGTCCGACAACAGATCGTCAAAGAGCTCGC : 3039316&#xA;&#xA;     117 : nValLeuHisGluCysAsnSerProTyrIleValGlyPheTyrGlyAlaPheTyrS :     135&#xA;            |||||||||:!!||||||||||||!:!||||||||||||||||||||||||   .&#xA;           aValLeuHisLysCysAsnSerProPheIleValGlyPheTyrGlyAlaPheValA&#xA; 3039315 : CGTACTTCACAAGTGCAATAGTCCATTCATTGTTGGATTCTACGGGGCTTTTGTGG : 3039259&#xA;&#xA;     136 : erAspGlyGluIleSerIleCysMetGluHisMetAspGlyGlySerLeuAspGln :     153&#xA;           .!:!!..!!!:||||||||||||||||||:!!|||||||||  !|||||||||  !&#xA;           spAsnAsnAspIleSerIleCysMetGluTyrMetAspGlyLeuSerLeuAspIle&#xA; 3039258 : ACAATAACGATATATCAATTTGTATGGAGTATATGGATGGGCTTTCACTTGATATA : 3039205&#xA;&#xA;     154 : ValLeuLysLys  &amp;gt;&amp;gt;&amp;gt;&amp;gt; Target Intron 2 &amp;gt;&amp;gt;&amp;gt;&amp;gt;  AlaGlyLysIlePro :     162&#xA;           ||||||||||||           3837 bp           !.!|||::::!!|||&#xA;           ValLeuLysLys++                         ++ValGlyArgLeuPro&#xA; 3039204 : GTTTTGAAAAAGgt.........................agGTTGGCCGCCTCCCG : 3035341&#xA;&#xA;     163 : GluLysIleLeuGlyLysValSerIleAlaValIleLysGlyLeuThrTyrLeuAr :     181&#xA;           ||||||.!!:!!|||::!:!!|||:!!||||||:!!:::|||||||||||||||!:&#xA;           GluLysPheValGlyArgIleSerValAlaValValArgGlyLeuThrTyrLeuLy&#xA; 3035340 : GAAAAGTTTGTCGGACGAATCTCGGTTGCAGTTGTCCGTGGATTGACGTACTTGAA : 3035284&#xA;&#xA;     182 : gGluLysHisLysIleMetHisArg{A}  &amp;gt;&amp;gt;&amp;gt;&amp;gt; Target Intron 3 &amp;gt;&amp;gt;&amp;gt;&amp;gt;  :     190&#xA;           !!!::!!  !||||||:!!||||||{|}            50 bp           &#xA;           sAspGluIleLysIleLeuHisArg{A}++                         +&#xA; 3035283 : AGATGAAATTAAGATTTTGCATAGA{G}gt.........................a : 3035208&#xA;&#xA;     191 :  {sp}ValLysProSerAsnIleLeuValAsnSerArgGlyGluIle--LysLeuC :     207&#xA;            {||}|||||||||||||||!!:||||||||||||!.!|||||||||##   ||||&#xA;           +{sp}ValLysProSerAsnMetLeuValAsnSerAsnGlyGluIle##---LeuC&#xA; 3035207 : g{AT}GTTAAACCATCGAATATGCTTGTCAATAGTAACGGAGAAATTAA---TTAT : 3035157&#xA;&#xA;     208 : ysAspPheGlyValSerGlyGlnLeuIleAspSerMetAlaAsnSerPheValGly :     225&#xA;           ||||||||||||||||||||..!|||||||||||||||||||||||||||||||||&#xA;           ysAspPheGlyValSerGlyMetLeuIleAspSerMetAlaAsnSerPheValGly&#xA; 3035156 : GCGATTTTGGAGTCTCTGGAATGTTGATTGATTCGATGGCCAACTCATTTGTCGGA : 3035103&#xA;&#xA;     226 : ThrArgSerTyrMetSer  &amp;gt;&amp;gt;&amp;gt;&amp;gt; Target Intron 4 &amp;gt;&amp;gt;&amp;gt;&amp;gt;  ProGluArg :     234&#xA;           |||||||||||||||:!!            799 bp           |||||||||&#xA;           ThrArgSerTyrMetAla++                         ++ProGluArg&#xA; 3035102 : ACGAGGAGTTATATGGCGgt.........................agCCCGAACGA : 3034277&#xA;&#xA;     235 : LeuGlnGlyThrHisTyrSerValGlnSerAspIleTrpSerMetGlyLeuSerLe :     253&#xA;           |||  !|||:!!||||||!!!:!!..!|||||||||||||||.!.|||||||||||&#xA;           LeuThrGlySerHisTyrThrIleSerSerAspIleTrpSerPheGlyLeuSerLe&#xA; 3034276 : CTCACAGGATCCCACTATACAATTTCATCGGATATCTGGTCATTTGGATTATCACT : 3034220&#xA;&#xA;     254 : uValGluMetAlaIleGlyArgTyrProIleProProProAspAlaLysGluLeuG :     272&#xA;           |||||||:!:  !|||||||||||||||:!:||| !!|||..!  !  !|||! !!&#xA;           uValGluLeuLeuIleGlyArgTyrProValProAlaProSerGlnAlaGluTyrA&#xA; 3034219 : TGTCGAACTTCTTATTGGACGATATCCAGTGCCTGCACCGTCTCAAGCAGAATATG : 3034163&#xA;&#xA;     273 : luLeuIlePheGlyCysSerVal&amp;lt;-&amp;gt;GluGlyAspProAlaSerSerGluLeuAla :     289&#xA;               !!!:|||..!   :!!! !   |||  !!!:! !|||..!|||     !! !&#xA;           laThrMetPheAsnValAlaGluAsnGluIleGluLeuAlaAspSerLeuGluGlu&#xA; 3034162 : CTACAATGTTTAATGTGGCTGAAAATGAGATTGAACTGGCTGATAGTCTTGAAGAG : 3034109&#xA;&#xA;     290 : ProArgProArgProProGlyArgProIleSer :     300&#xA;           |||!.!  !!.!||||||.!!!.!|||   |||&#xA;           ProAsnTyrHisProProSerAsnProAlaSer&#xA; 3034108 : CCCAACTATCATCCGCCGAGCAATCCGGCGTCA : 3034074&#xA;&#xA;vulgar: frog 32 300 . CM041006.1 3039620 3034073 - 732 M 52 156 5 0 2 I 0 51 3 0 2 M 73 219 5 0 2 I 0 3833 3 0 2 M 32 96 S 0 1 5 0 2 I 0 46 3 0 2 S 1 2 M 14 42 F 0 2 G 1 0 M 26 78 5 0 2 I 0 795 3 0 2 M 48 144 G 0 3 M 21 63&#xA;-- completed exonerate analysis&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In addition, exonerate lets you control the output you want. Check out the &lt;code&gt;--ryo&lt;/code&gt; (roll your own output) option in &lt;code&gt;man exonerate&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;&lt;a href=&quot;https://www.ebi.ac.uk/Tools/psa/genewise/&quot; rel=&quot;nofollow noreferrer&quot;&gt;genewise&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;GeneWise compares a protein sequence to a genomic DNA sequence, allowing for introns and frameshifting errors.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Unlike exonerate, genewise &lt;em&gt;only&lt;/em&gt; does protein to DNA alignment, but like exonerate it models introns and splicing. 20 years ago, when I did this every day, my impression was that genewise was slower but slightly more sensitive than the default exonerate options (exonerate can be tweaked to be made more sensitive). So I would usually try exonerate first and fall back to genewise if it failed to find a good match.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-10-13T12:03:12.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21699" PostTypeId="2" ParentId="21695" CreationDate="2023-10-13T15:58:22.000" Score="1" Body="&lt;p&gt;In the interface of &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PROGRAM=blastp&amp;amp;PAGE_TYPE=BlastSearch&amp;amp;LINK_LOC=blasthome&quot; rel=&quot;nofollow noreferrer&quot;&gt;blastp&lt;/a&gt;, you can limit the search space to organisms you want to. Yo can do this by introducing name species or Tax IDs, you can add more than one, but I bet you don't wanna introduce the 1,500 one by one.&lt;/p&gt;&#xA;&lt;p&gt;The only other way to do this, is by using blastp in its &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/doc/blast-help/downloadblastdata.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;command-line version&lt;/a&gt;. You would need to build a custom database to be used as the searching space, and then run blast on your computer or server using this database. Bulding it is a pain in the ass, as far as I know. You need first to download all the proteins in all organisms in fasta format and then use this compilation as a database (makeblastdb command). I am not aware of a way to download all proteins from various organism given a list of taxonomic IDs, but if any, it has to be NCBI, Uniprot or similar. Then you can use blastp locally together with your custom database.&lt;/p&gt;&#xA;&lt;p&gt;But why limiting the search space is such a way? You are biasing your results to what you want to get, instead of finding the stuff that actually is closest to your protein of interest. My suggestion is to use the online version of blastp with one of the various databases available and rather than pre-selecting organisms, you can exclude the ones from which you definitely don't want to get results from, and you can also exclude entire families/genera such as primates (taxid:9443), to name an example.&lt;/p&gt;&#xA;" OwnerUserId="475" LastEditorUserId="475" LastEditDate="2023-10-14T08:43:38.267" LastActivityDate="2023-10-14T08:43:38.267" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21701" PostTypeId="1" AcceptedAnswerId="21704" CreationDate="2023-10-14T01:40:38.213" Score="2" ViewCount="29" Body="&lt;p&gt;My current job is rate-limited chiefly by the IO overhead of writing reads to their corresponding bam files (they are split up by both cell barcode and contig). The only reason I need to write these mini-split bams is so that I can then use the pysam coverage function to assess read depth at certain locations. I already have all of these reads in memory as a list of AlignedSegments. Is it necessary to write them to a .bam file in order to be able to index and use the coverage function? I feel like there should be a way to have the AlignedSegments be treated as a bam object in memory, possibly even indexed in memory, and then treated as if they were just streamed from a bamfile with an index in order to be able to use the coverage method. Thanks in advance, if this is possible it would increase my processing speed x10. Or, given a list of AlignedSegments, is there a way to get coverage at specific positions across all those AlignedSegments in my list (that would get me what I need).&lt;/p&gt;&#xA;" OwnerUserId="18364" LastActivityDate="2023-10-16T09:20:47.210" Title="Is it possible to, all in memory without writing a file, given a list of AlignedSegments and a bam header, build a bam, index and get coverage?" Tags="|coverage|pysam|indexing|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21702" PostTypeId="1" CreationDate="2023-10-14T10:28:53.863" Score="1" ViewCount="130" Body="&lt;p&gt;I wish to download a fasta sequence from NCBI using &amp;quot;rentrez&amp;quot; package and then save it in an object, then I want to have this object as a DNAStringSet object to work on the sequence using &amp;quot;Biostrings&amp;quot; capabilities.&lt;/p&gt;&#xA;&lt;p&gt;I leave my incorrect code here so I would be grateful if you could help me out.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;cov2 &amp;lt;- entrez_fetch(db=&amp;quot;nucleotide&amp;quot; , rettype = &amp;quot;fasta&amp;quot;, id = &amp;quot;NC_045512&amp;quot;)&#xA;cov2 &amp;lt;- readDNAStringSet(cov2)&#xA;...&#xA;Error in .Call2(&amp;quot;new_input_filexp&amp;quot;, filepath, PACKAGE = &amp;quot;XVector&amp;quot;) : &#xA;  cannot open file '&amp;gt;NC_045512.2 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18298" LastEditorUserId="131" LastEditDate="2023-10-17T09:49:08.533" LastActivityDate="2023-10-17T15:12:48.453" Title="Making a DNAStringSet object" Tags="|r|fasta|bioconductor|biostrings|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21703" PostTypeId="2" ParentId="21695" CreationDate="2023-10-14T11:01:20.963" Score="1" Body="&lt;p&gt;My solution is simply.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;perform a web-based Blastp using the following parameters:&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;most dissimilar&lt;/li&gt;&#xA;&lt;li&gt;5000 hits (maximum limit)&lt;/li&gt;&#xA;&lt;li&gt;Restrict to Archaea (which is what you samples appear to be)*&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;If you hit the limit of 5000 results, I would repeat the search for each class or phylum within Archaea. I can't remember if it was re-classified as a kingdom or was just another phylum. Concatenate all hits into a single fasta file.&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Reg-ex your extremophiles database (personally I'd use pandas), so that the species name of each bacterium is recovered no other information is present. &lt;em&gt;Or&lt;/em&gt; this is an easily accessible value within a hash/database (this is a better solution).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Use Biopython to loop through your blast fasta file and filter using &lt;code&gt;re&lt;/code&gt; search against the species from your database. If you get a hit save that sequence.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Ideally, the point 2 and 3 would be a pandas merge operation, but looping is easier.&lt;/p&gt;&#xA;&lt;p&gt;*, If there are extremophiles which are not Archaea (I don't know much about them), then take each general taxonomic group and repeat the operation for the search criteria &amp;quot;restrict the blastp to ... &lt;em&gt;specific group&lt;/em&gt;&amp;quot;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Issues&lt;/strong&gt; The issues with command line blastp, not via the web, is that access is greatly restricted during US working hours. If you are searching more than 100 hits and carrying out the search during the working day US time - they will not let it through. 100 hits or less is fine however. For &amp;gt;100 hits (e.g. 5000) it's got to be done well outside US working hours/ weekends. Web-based blastp doesn't carry this restriction of access during US working hours. Basically, the API route is deprioritised for the web-based approach. I assume this is to stop bots - albeit of course a bot can automate through NCBI's webpage.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; The original solution by @terdon might easily work as a remote blastp because each query is restricted to &amp;lt;100 - so it will always get through during working hours. What I am saying is if you're using Blast API it needs a carefully defined strategy, else shift to the web for blanket approaches.&lt;/p&gt;&#xA;&lt;p&gt;Entrez operations from a commandline do work ok during US working day time - albeit it's got to be coded to deal with &amp;quot;random&amp;quot; server refusals otherwise it will break the code (sporadic queries are refused). This is just my recent experience nothing more.&lt;/p&gt;&#xA;&lt;p&gt;Just to be clear - this is potentially a medium sized project. Once the strategy has been decided then specific code that supports this approach can be given in separate questions.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="298" LastEditDate="2023-10-14T13:37:29.497" LastActivityDate="2023-10-14T13:37:29.497" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21704" PostTypeId="2" ParentId="21701" CreationDate="2023-10-14T13:33:52.703" Score="3" Body="&lt;p&gt;Assuming you are on an operating system that supports them, like Linux, you can try using &lt;a href=&quot;https://en.wikipedia.org/wiki/Named_pipe&quot; rel=&quot;nofollow noreferrer&quot;&gt;named pipes&lt;/a&gt; in a ramdisk or &lt;a href=&quot;https://en.wikipedia.org/wiki/Tmpfs&quot; rel=&quot;nofollow noreferrer&quot;&gt;tmpfs&lt;/a&gt;. Not the most elegant approach, but I don't know if what you are asking for is possible (it may very well be, I just don't know) and this can be a decent workaround.&lt;/p&gt;&#xA;&lt;p&gt;First, check that you do have a mounted tmpfs. On my system, I have a few:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ mount | awk '$1==&amp;quot;tmpfs&amp;quot;'&#xA;tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,inode64)&#xA;tmpfs on /tmp type tmpfs (rw,nosuid,nodev,size=16278892k,nr_inodes=1048576,inode64)&#xA;tmpfs on /run/user/1000 type tmpfs (rw,nosuid,nodev,relatime,size=3255776k,nr_inodes=813944,mode=700,uid=1000,gid=1000,inode64)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;These filesystems are actually stored in RAM and should be significantly faster. For instance, on my machine, which has an SSD as the main storage, I still get significant speedup when using &lt;code&gt;/dev/shm&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## SSD&#xA;$ time seq 1000000000 &amp;gt; file&#xA;real    0m13.687s&#xA;user    0m5.623s&#xA;sys     0m7.516s&#xA;&#xA;$ time wc -l &amp;lt; file&#xA;1000000000&#xA;&#xA;real    0m4.972s&#xA;user    0m0.101s&#xA;sys     0m2.248s&#xA;&#xA;## tmpfs, RAM&#xA;$ time seq 1000000000 &amp;gt; /dev/shm/file&#xA;&#xA;real    0m6.755s&#xA;user    0m4.982s&#xA;sys     0m1.763s&#xA;&#xA;$ time wc -l &amp;lt; /dev/shm/file&#xA;1000000000&#xA;&#xA;real    0m1.024s&#xA;user    0m0.120s&#xA;sys     0m0.902s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see, using &lt;code&gt;/dev/shm&lt;/code&gt; is significantly faster. And using named pipes can give you a little boost too:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ time ( seq 1000000000 &amp;gt; /dev/shm/file;  wc -l &amp;lt; /dev/shm/file)&#xA;1000000000&#xA;&#xA;real    0m8.341s&#xA;user    0m5.277s&#xA;sys     0m3.044s&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;$ rm /dev/shm/file &amp;amp;&amp;amp; mkfifo /dev/shm/file &#xA;$&lt;/span&gt; time ( seq 1000000000 &amp;gt; /dev/shm/file &amp;amp;  wc -l &amp;lt; /dev/shm/file)&#xA;1000000000&#xA;&#xA;real    0m6.564s&#xA;user    0m5.426s&#xA;sys     0m3.965s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So, this is essentially a way of doing everything in RAM without needing to code it as doing everything in RAM. You can create these files (the FIFOs, the named pipes) and treat them as files, but since you're creating them in a &lt;code&gt;tmpfs&lt;/code&gt; filesystem, they're actually in RAM and by using FIFOs you not only get a further speed jump but also minimise the actual data that need to be stored.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2023-10-16T09:20:47.210" LastActivityDate="2023-10-16T09:20:47.210" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21705" PostTypeId="1" AcceptedAnswerId="21735" CreationDate="2023-10-15T01:27:26.747" Score="4" ViewCount="106" Body="&lt;p&gt;Does anyone here know how to calculate a value for the asterisk (*) code that appears in substitution matrices?&lt;br /&gt;&#xA;From my observation, to all pairs with a one asterisk, the lowest value from the matrix is set. For &lt;code&gt;**&lt;/code&gt; it's &lt;code&gt;1&lt;/code&gt;. &lt;a href=&quot;https://bio.davidson.edu/courses/genomics/2008/Simpson/Tutorial.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Example: BLOSUM62&lt;/a&gt;.&#xA;But, it's just my observation, I couldn't find any source or explanation how it was calculated (for the exemplary BLOSUM62 or any other substitution matrix).&lt;/p&gt;&#xA;&lt;p&gt;I'd appreciate sources, especially scientific papers.&lt;/p&gt;&#xA;" OwnerUserId="9651" LastEditorUserId="9651" LastEditDate="2023-10-15T19:17:25.800" LastActivityDate="2023-10-22T15:32:07.063" Title="Asterisk (*) calculation method" Tags="|amino-acids|substitution|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21706" PostTypeId="2" ParentId="21636" CreationDate="2023-10-15T08:18:38.330" Score="2" Body="&lt;p&gt;I'd like to answer my own question as I have found the solution for this and thought it might help anyone else trying to use eggNOG-mapper.&#xA;The prerequisite for uploading the input file with the required collection of .faa files, is that, the file extension has to end with .gz. The site mentions that too. I had made the mistake of uploading a .tar.gz which is what my system produced as a compressed file of my folder of sequences. Although the upload was successful, the runs failed as the file had some unwanted lines owing to the .tar.gz format.&#xA;I randomly uploaded a concatenated .faa file which worked well and got the results too! The trick is that, the input file has to be strictly a .gz and NOT .tar.gz or it could also be a .faa file with all the sequences.&lt;/p&gt;&#xA;" OwnerUserId="17392" LastActivityDate="2023-10-15T08:18:38.330" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21707" PostTypeId="1" CreationDate="2023-10-15T08:45:22.807" Score="1" ViewCount="42" Body="&lt;p&gt;I wanted to know the HGT genes in a Metabacillus strain and I noticed that several papers use HGTector for this purpose. I installed HGTector through conda and tried to download the database for Bacillota using the example codes given on the hgtector github page -&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;hgtector database -c bacteria -o db1 -t 1239&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The code for Bacillota is 1239. It took an awfully long time (approx. 13h) for the download to end and several warning messages were displayed saying so and so files could not be downloaded. When I checked &amp;quot;db1&amp;quot; only 1000 or so files were there, signifying that the download was incomplete as there are 9112 files under Bacillota. I am not sure if the problem is with my code or because there is not enough disk space or something else. Following are my system and Wifi details :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Lenovo Ideapad, 16GB memory&#xA;12th Gen i5 - 1235U x 12 Processor&#xA;1 TB Disk capacity&#xA;OS = Ubuntu 22.04.3 LTS&#xA;Wifi speed = 32Mb/s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have posted this on github about 4 days ago but no response yet. If  my disk space is not enough could anyone suggest any other tool to study horizontally transferred genes? I have scanned the web but did not find anything satisfactory.&lt;/p&gt;&#xA;&lt;p&gt;I ran &lt;code&gt;df -h&lt;/code&gt; and it shows ~700 GB of free disk space even after the download of 1000 files which should be around 2 GB.&lt;/p&gt;&#xA;&lt;p&gt;I've tried again at 'family' level with similar results. I don't think I can use HGTector for now. What do you think of RecentHGT? This is the paper for it - &lt;a href=&quot;https://www.frontiersin.org/articles/10.3389/fmicb.2018.00973/full&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="17392" LastEditorUserId="73" LastEditDate="2023-10-16T05:39:36.557" LastActivityDate="2023-10-16T05:39:36.557" Title="How to use HGTECTOR tool for identifying HGT genes in a bacterial genome?" Tags="|genome|bacteria|ecology|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21708" PostTypeId="2" ParentId="17966" CreationDate="2023-10-15T16:06:15.363" Score="1" Body="&lt;p&gt;OK my attempt, input pdb created in &lt;strong&gt;pymol&lt;/strong&gt; with &lt;code&gt;fab AVGLWPT , AVGLWPT , hydro=0&lt;/code&gt; and saved as &lt;code&gt;avgwpt.pdb&lt;/code&gt; :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATOM      1  N   ALA     1      67.846  51.727  15.161  1.00  0.00           N  &#xA;ATOM      2  CA  ALA     1      68.522  53.021  15.161  1.00  0.00           C  &#xA;ATOM      3  C   ALA     1      70.022  52.847  15.161  1.00  0.00           C  &#xA;ATOM      4  O   ALA     1      70.553  51.730  15.150  1.00  0.00           O  &#xA;ATOM      5  CB  ALA     1      68.014  53.813  16.378  1.00  0.00           C  &#xA;ATOM      6  N   VAL     2      70.773  53.896  15.173  1.00  0.00           N  &#xA;ATOM      7  CA  VAL     2      72.223  53.728  15.173  1.00  0.00           C  &#xA;ATOM      8  C   VAL     2      72.922  55.066  15.188  1.00  0.00           C  &#xA;ATOM      9  O   VAL     2      72.296  56.134  15.198  1.00  0.00           O  &#xA;ATOM     10  CB  VAL     2      72.665  52.877  13.916  1.00  0.00           C  &#xA;ATOM     11  CG1 VAL     2      74.193  52.762  13.664  1.00  0.00           C  &#xA;ATOM     12  CG2 VAL     2      72.143  51.420  13.932  1.00  0.00           C  &#xA;ATOM     13  N   GLY     3      74.212  55.098  15.190  1.00  0.00           N  &#xA;ATOM     14  CA  GLY     3      74.888  56.392  15.204  1.00  0.00           C  &#xA;ATOM     15  C   GLY     3      76.388  56.218  15.203  1.00  0.00           C  &#xA;ATOM     16  O   GLY     3      76.917  55.104  15.192  1.00  0.00           O  &#xA;ATOM     17  N   LEU     4      77.138  57.267  15.215  1.00  0.00           N  &#xA;ATOM     18  CA  LEU     4      78.589  57.099  15.215  1.00  0.00           C  &#xA;ATOM     19  C   LEU     4      79.288  58.437  15.230  1.00  0.00           C  &#xA;ATOM     20  O   LEU     4      78.660  59.507  15.241  1.00  0.00           O  &#xA;ATOM     21  CB  LEU     4      79.040  56.251  13.992  1.00  0.00           C  &#xA;ATOM     22  CG  LEU     4      78.445  54.825  13.832  1.00  0.00           C  &#xA;ATOM     23  CD1 LEU     4      79.094  54.113  12.637  1.00  0.00           C  &#xA;ATOM     24  CD2 LEU     4      78.610  53.968  15.098  1.00  0.00           C  &#xA;ATOM     25  N   TRP     5      80.577  58.469  15.232  1.00  0.00           N  &#xA;ATOM     26  CA  TRP     5      81.254  59.763  15.246  1.00  0.00           C  &#xA;ATOM     27  C   TRP     5      82.754  59.589  15.245  1.00  0.00           C  &#xA;ATOM     28  O   TRP     5      83.281  58.478  15.233  1.00  0.00           O  &#xA;ATOM     29  CB  TRP     5      80.817  60.551  16.512  1.00  0.00           C  &#xA;ATOM     30  CG  TRP     5      79.341  60.958  16.536  1.00  0.00           C  &#xA;ATOM     31  CD1 TRP     5      78.304  60.258  17.187  1.00  0.00           C  &#xA;ATOM     32  CD2 TRP     5      78.731  62.003  15.872  1.00  0.00           C  &#xA;ATOM     33  CE2 TRP     5      77.340  61.929  16.130  1.00  0.00           C  &#xA;ATOM     34  CE3 TRP     5      79.257  63.017  15.031  1.00  0.00           C  &#xA;ATOM     35  NE1 TRP     5      77.049  60.848  16.950  1.00  0.00           N  &#xA;ATOM     36  CZ2 TRP     5      76.464  62.876  15.556  1.00  0.00           C  &#xA;ATOM     37  CZ3 TRP     5      78.372  63.946  14.482  1.00  0.00           C  &#xA;ATOM     38  CH2 TRP     5      76.996  63.879  14.742  1.00  0.00           C  &#xA;ATOM     39  N   PRO     6      83.505  60.638  15.257  1.00  0.00           N  &#xA;ATOM     40  CA  PRO     6      84.959  60.560  15.257  1.00  0.00           C  &#xA;ATOM     41  C   PRO     6      85.498  60.036  16.599  1.00  0.00           C  &#xA;ATOM     42  O   PRO     6      84.724  59.735  17.507  1.00  0.00           O  &#xA;ATOM     43  CB  PRO     6      85.354  62.018  14.978  1.00  0.00           C  &#xA;ATOM     44  CG  PRO     6      84.201  62.829  15.588  1.00  0.00           C  &#xA;ATOM     45  CD  PRO     6      82.967  61.980  15.271  1.00  0.00           C  &#xA;ATOM     46  N   THR     7      86.770  59.913  16.774  1.00  0.00           N  &#xA;ATOM     47  CA  THR     7      87.263  59.414  18.054  1.00  0.00           C  &#xA;ATOM     48  C   THR     7      88.771  59.342  18.062  1.00  0.00           C  &#xA;ATOM     49  O   THR     7      89.436  59.668  17.100  1.00  0.00           O  &#xA;ATOM     50  CB  THR     7      86.639  58.014  18.376  1.00  0.00           C  &#xA;ATOM     51  CG2 THR     7      87.080  57.358  19.702  1.00  0.00           C  &#xA;ATOM     52  OG1 THR     7      85.224  58.110  18.479  1.00  0.00           O  &#xA;TER   &#xA;END&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;pic :&#xA;&lt;a href=&quot;https://i.stack.imgur.com/NZNZY.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/NZNZY.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;code :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python3&#xA;# -*- coding: utf-8 -*-&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;molecule create in pymol with fab AVGLWPT , AVGLWPT , hydro=0&#xA;&#xA;&amp;quot;&amp;quot;&amp;quot;&#xA;&#xA;import Bio&#xA;print('########## Biopython  VERSION ##########################################')&#xA;print('\n\n BIOPYTHON_VERSION : ', Bio.__version__)&#xA;print('###################################################################')&#xA;&#xA;from Bio.PDB import (&#xA;                    PDBParser,&#xA;                    Atom,&#xA;                    PDBIO&#xA;                    )&#xA;&#xA;&#xA;structure_code = 'avgwpt'&#xA;&#xA;structure_name = structure_code&#xA;&#xA;parser_pdb= PDBParser(QUIET=False)    ### shows warnings&#xA;&#xA;structure_whole = parser_pdb.get_structure(structure_name, structure_name+&amp;quot;.pdb&amp;quot;)  &#xA;&#xA;&#xA;last_serial = list(structure_whole.get_atoms())[-1].serial_number&#xA;&#xA;print(&amp;quot; last_serial : &amp;quot;,last_serial)&#xA;&#xA;for model in structure_whole:&#xA;    &#xA;    for chain in model:&#xA;        &#xA;        for resi in chain:&#xA;        &#xA;            print('1 : ', resi)&#xA;            &#xA;            print('2 : ',resi.id)&#xA;            &#xA;            try :&#xA;            &#xA;                print('3 : ',chain[resi.id[1]-1])&#xA;                &#xA;                doable = True&#xA;                &#xA;            except :&#xA;                &#xA;                print('3 : ','residue -1 missing !!!!!!!!!!!!!!!!!!!!!!1')&#xA;                &#xA;                doable = False&#xA;                &#xA;            &#xA;            print('4 : ',[i for i in resi.get_atoms()])&#xA;            &#xA;            if doable:&#xA;                &#xA;                print('5 : ','ok')&#xA;            &#xA;            else:&#xA;                &#xA;                print('5 : ','NO')&#xA;                &#xA;            &#xA;            if doable:&#xA;                &#xA;                &#xA;                print(chain[resi.id[1]-1]['C'])&#xA;                &#xA;                print(chain[resi.id[1]-1]['C'].get_vector(), type(chain[resi.id[1]-1]['C'].get_vector()))&#xA;                &#xA;                print(chain[resi.id[1]]['N'].get_vector())&#xA;                &#xA;                c = chain[resi.id[1]-1]['C'].get_vector()&#xA;                &#xA;                n = chain[resi.id[1]]['N'].get_vector()&#xA;                &#xA;                ca = chain[resi.id[1]]['CA'].get_vector()&#xA;                &#xA;                nc = n-c&#xA;                &#xA;                nca = n-ca&#xA;                &#xA;                print('vector BN : ', nc)&#xA;                &#xA;                nc.normalize()&#xA;                &#xA;                print('vector BN normalize : ', nc)&#xA;                &#xA;                print('vector B2B : ', nca)&#xA;                &#xA;                nca.normalize()&#xA;                &#xA;                print('vector B2B normalize : ', nca)&#xA;                &#xA;                print('sum of nc + nca : ' , nsum := nc + nca)&#xA;                &#xA;                nsum.normalize()&#xA;                &#xA;                nsum**1.023&#xA;                &#xA;                nsum += n&#xA;                &#xA;                print(nsum)&#xA;                &#xA;                print(nsum.get_array(), type(nsum.get_array()))&#xA;                &#xA;                resi.add(Atom.Atom(name=' H  ', coord = nsum.get_array(), bfactor=0., occupancy=1., altloc=' ', fullname=' H  ', serial_number = last_serial ,element='H'))&#xA;            &#xA;                last_serial += 1&#xA;&#xA;io=PDBIO() &#xA;&#xA;io.set_structure(structure_whole ) &#xA;&#xA;io.save(structure_name+&amp;quot;_H_added.pdb&amp;quot; , preserve_atom_numbering = True) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output , saved as &lt;code&gt;avgwpt_H_added.pdb&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ATOM      1  N   ALA     1      67.846  51.727  15.161  1.00  0.00           N  &#xA;ATOM      2  CA  ALA     1      68.522  53.021  15.161  1.00  0.00           C  &#xA;ATOM      3  C   ALA     1      70.022  52.847  15.161  1.00  0.00           C  &#xA;ATOM      4  O   ALA     1      70.553  51.730  15.150  1.00  0.00           O  &#xA;ATOM      5  CB  ALA     1      68.014  53.813  16.378  1.00  0.00           C  &#xA;ATOM      6  N   VAL     2      70.773  53.896  15.173  1.00  0.00           N  &#xA;ATOM      7  CA  VAL     2      72.223  53.728  15.173  1.00  0.00           C  &#xA;ATOM      8  C   VAL     2      72.922  55.066  15.188  1.00  0.00           C  &#xA;ATOM      9  O   VAL     2      72.296  56.134  15.198  1.00  0.00           O  &#xA;ATOM     10  CB  VAL     2      72.665  52.877  13.916  1.00  0.00           C  &#xA;ATOM     11  CG1 VAL     2      74.193  52.762  13.664  1.00  0.00           C  &#xA;ATOM     12  CG2 VAL     2      72.143  51.420  13.932  1.00  0.00           C  &#xA;ATOM     52  H   VAL     2      70.368  54.810  15.182  1.00  0.00           H  &#xA;ATOM     13  N   GLY     3      74.212  55.098  15.190  1.00  0.00           N  &#xA;ATOM     14  CA  GLY     3      74.888  56.392  15.204  1.00  0.00           C  &#xA;ATOM     15  C   GLY     3      76.388  56.218  15.203  1.00  0.00           C  &#xA;ATOM     16  O   GLY     3      76.917  55.104  15.192  1.00  0.00           O  &#xA;ATOM     53  H   GLY     3      74.741  54.249  15.182  1.00  0.00           H  &#xA;ATOM     17  N   LEU     4      77.138  57.267  15.215  1.00  0.00           N  &#xA;ATOM     18  CA  LEU     4      78.589  57.099  15.215  1.00  0.00           C  &#xA;ATOM     19  C   LEU     4      79.288  58.437  15.230  1.00  0.00           C  &#xA;ATOM     20  O   LEU     4      78.660  59.507  15.241  1.00  0.00           O  &#xA;ATOM     21  CB  LEU     4      79.040  56.251  13.992  1.00  0.00           C  &#xA;ATOM     22  CG  LEU     4      78.445  54.825  13.832  1.00  0.00           C  &#xA;ATOM     23  CD1 LEU     4      79.094  54.113  12.637  1.00  0.00           C  &#xA;ATOM     24  CD2 LEU     4      78.610  53.968  15.098  1.00  0.00           C  &#xA;ATOM     54  H   LEU     4      76.733  58.181  15.224  1.00  0.00           H  &#xA;ATOM     25  N   TRP     5      80.577  58.469  15.232  1.00  0.00           N  &#xA;ATOM     26  CA  TRP     5      81.254  59.763  15.246  1.00  0.00           C  &#xA;ATOM     27  C   TRP     5      82.754  59.589  15.245  1.00  0.00           C  &#xA;ATOM     28  O   TRP     5      83.281  58.478  15.233  1.00  0.00           O  &#xA;ATOM     29  CB  TRP     5      80.817  60.551  16.512  1.00  0.00           C  &#xA;ATOM     30  CG  TRP     5      79.341  60.958  16.536  1.00  0.00           C  &#xA;ATOM     31  CD1 TRP     5      78.304  60.258  17.187  1.00  0.00           C  &#xA;ATOM     32  CD2 TRP     5      78.731  62.003  15.872  1.00  0.00           C  &#xA;ATOM     33  CE2 TRP     5      77.340  61.929  16.130  1.00  0.00           C  &#xA;ATOM     34  CE3 TRP     5      79.257  63.017  15.031  1.00  0.00           C  &#xA;ATOM     35  NE1 TRP     5      77.049  60.848  16.950  1.00  0.00           N  &#xA;ATOM     36  CZ2 TRP     5      76.464  62.876  15.556  1.00  0.00           C  &#xA;ATOM     37  CZ3 TRP     5      78.372  63.946  14.482  1.00  0.00           C  &#xA;ATOM     38  CH2 TRP     5      76.996  63.879  14.742  1.00  0.00           C  &#xA;ATOM     55  H   TRP     5      81.105  57.620  15.224  1.00  0.00           H  &#xA;ATOM     39  N   PRO     6      83.505  60.638  15.257  1.00  0.00           N  &#xA;ATOM     40  CA  PRO     6      84.959  60.560  15.257  1.00  0.00           C  &#xA;ATOM     41  C   PRO     6      85.498  60.036  16.599  1.00  0.00           C  &#xA;ATOM     42  O   PRO     6      84.724  59.735  17.507  1.00  0.00           O  &#xA;ATOM     43  CB  PRO     6      85.354  62.018  14.978  1.00  0.00           C  &#xA;ATOM     44  CG  PRO     6      84.201  62.829  15.588  1.00  0.00           C  &#xA;ATOM     45  CD  PRO     6      82.967  61.980  15.271  1.00  0.00           C  &#xA;ATOM     56  H   PRO     6      83.072  61.539  15.267  1.00  0.00           H  &#xA;ATOM     46  N   THR     7      86.770  59.913  16.774  1.00  0.00           N  &#xA;ATOM     47  CA  THR     7      87.263  59.414  18.054  1.00  0.00           C  &#xA;ATOM     48  C   THR     7      88.771  59.342  18.062  1.00  0.00           C  &#xA;ATOM     49  O   THR     7      89.436  59.668  17.100  1.00  0.00           O  &#xA;ATOM     50  CB  THR     7      86.639  58.014  18.376  1.00  0.00           C  &#xA;ATOM     51  CG2 THR     7      87.080  57.358  19.702  1.00  0.00           C  &#xA;ATOM     52  OG1 THR     7      85.224  58.110  18.479  1.00  0.00           O  &#xA;ATOM     57  H   THR     7      87.409  60.156  16.044  1.00  0.00           H  &#xA;TER      58      THR     7                                                       &#xA;END   &#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;pic : &lt;a href=&quot;https://i.stack.imgur.com/WmGL1.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WmGL1.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Not sure I got the code above (in question) right, can't read it , had to use comments as trace.&lt;/p&gt;&#xA;&lt;p&gt;Spent a while banging my head with : &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2724740/&quot; rel=&quot;nofollow noreferrer&quot;&gt;HAAD: A Quick Algorithm for Accurate Prediction of Hydrogen Atoms in Protein Structures&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and trying to figure out their alghoritm, [code (&lt;a href=&quot;https://seq2fun.dcmb.med.umich.edu//HAAD/haad.f90&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://seq2fun.dcmb.med.umich.edu//HAAD/haad.f90&lt;/a&gt;) is in FORTRAN90] from the paper , but no luck, if anybody wants to help eplaining this bit:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For constructing the sp2H2 and sp2H1 H-atoms, we first decide on the orientation of the conjugated plane or the aromatic ring with respect to the neighboring heavy atoms; the normal vector of the conjugated plane is determined by taking the cross product of two vectors between the heavy atoms. For the sp2H2 H-atoms (illustrated in Fig. 1b), the normal vector of the conjugated plane is the cross product of the unit vectors B→N and B2→B; then the two H-atoms are placed at positions A1 and H, which are within the conjugated plane respected to the B→N vector with the exact bond angle from CHARMM22 force field. For sp2H1 H-atoms, two conformations are possible. The first is to place the H-atom in the peptide plane as illustrated in Fig. 1b, where A1 and B1 represent the alpha carbon atoms. The position of H in this case is decided by using the same method as the one used to determine the position of the sp2H2 H-atoms while holding the trans-conformation.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;it refers to Fig.1 of linked freely available article&lt;/p&gt;&#xA;" OwnerUserId="10049" LastActivityDate="2023-10-15T16:06:15.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21709" PostTypeId="1" AcceptedAnswerId="21712" CreationDate="2023-10-15T18:45:22.020" Score="3" ViewCount="85" Body="&lt;p&gt;I would like to add some 3D elements (balls and lines) to the PyMOL visualization to highlight some &amp;quot;holes&amp;quot; in the structure. Say I &lt;code&gt;fetch 1tqn&lt;/code&gt; and I want to add a yellow ball somewhere (to specific coordinates, say &lt;code&gt;[-19, -23, -14]&lt;/code&gt;) in the view. Or a bar from - to coordinates.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1sSQv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1sSQv.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I was going through wiki, did a online search but have not found a way how to add these simple 3D entities into the scene. I guess it should be trivial and I'm just using wrong queries. Because I need to add lots of these, I need to be able to do that via scripting interface.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance for any help!&lt;/p&gt;&#xA;" OwnerUserId="2137" LastActivityDate="2023-10-16T10:20:28.550" Title="how to add a custom elements into PyMOL visualization?" Tags="|pymol|modelling|3d-structure|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21710" PostTypeId="1" AcceptedAnswerId="21714" CreationDate="2023-10-16T10:15:58.230" Score="3" ViewCount="44" Body="&lt;p&gt;I have a phylogenetic tree with a lot of different populations and I wanted to removes most of them and apply new names to the labels. I wrote this code:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(ape)&#xA;library(phylotools)&#xA;tree&amp;lt;- read.tree(&amp;quot;Consensus_COMBINED_Cod3genesRatesC_trees.trees&amp;quot;)&#xA;&#xA;tree&amp;lt;- drop.tip(tree, tip = c(&amp;quot;CeuMist2&amp;quot;,&amp;quot;CeuMist1&amp;quot;,&amp;quot;Ceu_PAN_2&amp;quot;,&amp;quot;Ceu_PAN_1&amp;quot;,&amp;quot;Ceu_PAN_3&amp;quot;,&amp;quot;Ceu_PAN_4&amp;quot;))&#xA;tree&amp;lt;- drop.tip(tree, tip= c(&amp;quot;CcNav1&amp;quot;,&amp;quot;CcNav2&amp;quot;,&amp;quot;Cc_Mal_1&amp;quot;,&amp;quot;Cc_KalR2_2&amp;quot;,&amp;quot;CcRod1&amp;quot;,&amp;quot;CcRod2&amp;quot;,&amp;quot;CcRod3&amp;quot;,&amp;quot;CcRod4&amp;quot;,&amp;quot;CcHr1&amp;quot;))&#xA;tree&amp;lt;- drop.tip(tree, tip= c(&amp;quot;CeST6_4&amp;quot;,&amp;quot;CeST6_5&amp;quot;,&amp;quot;CeAA7_1&amp;quot;,&amp;quot;CeST6_3&amp;quot;,&amp;quot;CeST6_2&amp;quot;))&#xA;tree&amp;lt;- drop.tip(tree, tip= c(&amp;quot;Cg_MelR3_1&amp;quot;,&amp;quot;Cg_MelR6_2&amp;quot;,&amp;quot;Cg_MelR7_2&amp;quot;,&amp;quot;Cg_MelR6_1&amp;quot;,&amp;quot;Cg_MelR7_3&amp;quot;,&amp;quot;Cg_MelR5_1&amp;quot;,&amp;quot;Cg_MelR4_2&amp;quot;,&amp;quot;Cg_MelR1_1&amp;quot;,&amp;quot;Cg_MelR2_1&amp;quot;,&amp;quot;Cg_MelR4_1&amp;quot;,&amp;quot;Cg_MelR7_4&amp;quot;,&amp;quot;Cg_MelR5_2&amp;quot;))&#xA;tree&amp;lt;- drop.tip(tree, tip= c(&amp;quot;ChTM5_1&amp;quot;,&amp;quot;ChTM4_3&amp;quot;,&amp;quot;ChTM4_1&amp;quot;,&amp;quot;Ch_MEN_5&amp;quot;,&amp;quot;Ch_MEN_13&amp;quot;,&amp;quot;Ch_MEN_8&amp;quot;,&amp;quot;Ch_LEV_3&amp;quot;,&amp;quot;Ch_LEV_5&amp;quot;,&amp;quot;Ch_LEV_4&amp;quot;,&amp;quot;Ch_MEN_6&amp;quot;,&amp;quot;Ch_MEN_10&amp;quot;,&amp;quot;Ch_LEV_2&amp;quot;,&amp;quot;Ch_MEN_4&amp;quot;,&amp;quot;Ch_MEN_14&amp;quot;,&amp;quot;Ch_LEV_1&amp;quot;,&amp;quot;Ch_MEN_11&amp;quot;,&amp;quot;Ch_MEN_3&amp;quot;,&amp;quot;Ch_MEN_9&amp;quot;))&#xA;tree&amp;lt;- drop.tip(tree, tip= c(&amp;quot;Ci_MSP_2&amp;quot;,&amp;quot;Ci_MSP_1&amp;quot;,&amp;quot;Ci_DIA_1&amp;quot;,&amp;quot;Ci_DIA_2&amp;quot;,&amp;quot;CiPLAN1&amp;quot;,&amp;quot;CiPLAN3&amp;quot;,&amp;quot;CiPLAN2&amp;quot;,&amp;quot;Ci_MEN_1&amp;quot;,&amp;quot;Ci_MEN_2&amp;quot;,&amp;quot;Ci_MEN_12&amp;quot;))&#xA;tree&amp;lt;- drop.tip(tree, tip= c(&amp;quot;CpGravR3&amp;quot;,&amp;quot;CpGravR4&amp;quot;,&amp;quot;CpGio_2&amp;quot;,&amp;quot;CpGravR5_2&amp;quot;,&amp;quot;CpGravR2&amp;quot;,&amp;quot;CpGravR7&amp;quot;,&amp;quot;CpGio_1&amp;quot;,&amp;quot;CpGio_3&amp;quot;,&amp;quot;CpGravR1&amp;quot;,&amp;quot;CpGravR6_2&amp;quot;,&amp;quot;CpGravR5_1&amp;quot;,&amp;quot;CpThe_1&amp;quot;,&amp;quot;CpSka_1&amp;quot;))&#xA;tree&amp;lt;- drop.tip(tree, tip= c(&amp;quot;Cn_IG_2&amp;quot;,&amp;quot;Cn_IG_3&amp;quot;,&amp;quot;Cn_IG_1&amp;quot;,&amp;quot;LevantinaFC4779&amp;quot;,&amp;quot;Assyriella_naegelei&amp;quot;))&#xA;&#xA;newlabels&amp;lt;- c(&amp;quot;C. eucineta&amp;quot;, &amp;quot;C. codringtonii&amp;quot;, &amp;quot;C. elisabethae&amp;quot;,&amp;quot;C. gittenbergeri&amp;quot;, &amp;quot;C. helenae&amp;quot;, &amp;quot;C. intusplicata&amp;quot;, &amp;quot;C. parnassia&amp;quot;)&#xA;tree$tip.label&amp;lt;- newlabels&#xA;&#xA;write.nexus(tree, &amp;quot;CodTree.trees&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I use the &lt;code&gt;write.nexus&lt;/code&gt; command I get the following error:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;Error in y$tip.label : $ operator is invalid for atomic vectors&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I circumvent this error and export the tree in nexus format?&lt;/p&gt;&#xA;&lt;p&gt;I will also provide the tree here: &lt;a href=&quot;https://drive.google.com/file/d/1lYmzW9htwaclC_yYs3fQom-Z3zbjk8wH/view?usp=drive_link&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://drive.google.com/file/d/1lYmzW9htwaclC_yYs3fQom-Z3zbjk8wH/view?usp=drive_link&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16657" LastEditorUserId="650" LastEditDate="2023-10-16T14:25:36.303" LastActivityDate="2023-10-16T15:50:14.943" Title="How can I export a pruned phylogenetic tree in nexus format in R?" Tags="|r|phylogenetics|file-formats|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21711" PostTypeId="2" ParentId="21684" CreationDate="2023-10-16T10:17:46.547" Score="0" Body="&lt;p&gt;I managed to solve the issue by using a slightly different approach to @Ram RS, which is &lt;code&gt;bcftools csq&lt;/code&gt; instead of &lt;code&gt;bcftools annotate&lt;/code&gt;. The formatting is via GFF3, and the tool is picky about how to format it, but it turns out Torsten @tseemann had already gone through the pain of matching what the tool expects with the GFF3 flavour, and then it works:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/samtools/bcftools/issues/530#issuecomment-377637623&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/samtools/bcftools/issues/530#issuecomment-377637623&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;An example gff3 for a single CDS of length=1463 is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;seq1    geneX   gene    1       1463    .       +       .       ID=gene:gene1;Name=gene1;biotype=protein_coding                                            &#xA;seq1    geneX   mRNA    1       1463    .       +       .       ID=transcript:transcript1;Name=transcript1;Parent=gene:gene1;biotype=protein_coding        &#xA;seq1    geneX   CDS     1       1463    .       +       0       ID=CDS:cds1;Name=CDS1;Parent=transcript:transcript1;biotype=protein_coding                  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then call it with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools csq -p m -f $fastaref -g $gff3 $inputvcf -o $outvcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Thank you all for commenting.&lt;/p&gt;&#xA;" OwnerUserId="180" LastActivityDate="2023-10-16T10:17:46.547" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21712" PostTypeId="2" ParentId="21709" CreationDate="2023-10-16T10:20:28.550" Score="2" Body="&lt;p&gt;my attempt :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pymol&#xA;&#xA;from pymol.cgo import *&#xA;&#xA;from pymol import (&#xA;                    &#xA;                    cmd ,&#xA;                    )&#xA;&#xA;&#xA;print('########## PYMOL VERSION ##########################################')&#xA;print('         ',  cmd.get_version() )&#xA;print('###################################################################')&#xA;&#xA;&#xA;pymol.finish_launching()&#xA;&#xA;cmd.fetch('1tqn')&#xA;&#xA;cmd.remove('solvent')&#xA;&#xA;pos = cmd.get_position()&#xA;&#xA;r = 3&#xA;&#xA;print('###############################################################################\n')&#xA;&#xA;print('### POSITION : ' , pos,'  ###\n')&#xA;&#xA;print('###############################################################################\n')&#xA;&#xA;&#xA;&#xA;spherelist = [&#xA;    [COLOR,    1.000,    1.000,    0.000,&#xA;    SPHERE,   pos[0] , pos[1] , pos[2]  , r ],&#xA;&#xA;    ]&#xA;&#xA;for i in spherelist:&#xA;    &#xA;    a = str(spherelist.index(i))&#xA;&#xA;    cmd.load_cgo(i, 'sphere_'+a )&#xA;&#xA;&#xA;cmd.zoom()&#xA;&#xA;cmd.set('cartoon_transparency' , '0.5')&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;result:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Etqeo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Etqeo.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10049" LastActivityDate="2023-10-16T10:20:28.550" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21713" PostTypeId="1" CreationDate="2023-10-16T15:05:15.770" Score="3" ViewCount="591" Body="&lt;p&gt;&lt;em&gt;A question similar to this has also been asked on &lt;a href=&quot;https://www.biostars.org/p/9577731/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I understand GWAS is done with R, but are any written in Python?&#xA;I'm used to python and java, acclimating to R will take a bit of time and my advisor wants some data analyzed. I plan on using R but given the deadline, I want to finish the analysis in Python.&lt;/p&gt;&#xA;" OwnerUserId="18372" LastEditorUserId="73" LastEditDate="2023-10-17T03:15:58.793" LastActivityDate="2023-10-17T03:15:58.793" Title="GWAS in python?" Tags="|python|gwas|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21714" PostTypeId="2" ParentId="21710" CreationDate="2023-10-16T15:50:14.943" Score="3" Body="&lt;p&gt;So after digging I found the solution. What worked for me was adding the &lt;code&gt;file = &lt;/code&gt; in the &lt;code&gt;write.nexus&lt;/code&gt; command. The final line should be&#xA;&lt;code&gt;write.nexus(tree, file =  &amp;quot;Cod.trees&amp;quot;) &lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="16657" LastActivityDate="2023-10-16T15:50:14.943" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21715" PostTypeId="2" ParentId="21713" CreationDate="2023-10-16T16:56:57.977" Score="5" Body="&lt;p&gt;It doesn't (really) matter what language a piece of GWAS software is written in. SAIGE is written in a mix of R and C++. BOLT is written in C++. You shouldn't be writing a GWAS analysis from scratch, as it ends up being a very complex piece of software. You will almost certainly be interacting with the command line, when running a GWAS, so my advice is to find the software which is most suitable for your piece of analysis, not one which is written in a programming language you like.&lt;/p&gt;&#xA;" OwnerUserId="3694" LastActivityDate="2023-10-16T16:56:57.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21716" PostTypeId="2" ParentId="21702" CreationDate="2023-10-16T21:16:10.583" Score="1" Body="&lt;p&gt;Your issue is that &lt;code&gt;entrez_fetch()&lt;/code&gt; returns the fasta record(s) as a single string with the definition lines and sequences concatenated. Also, newline characters (&lt;code&gt;\n&lt;/code&gt;) appear in place of line breaks, with double newlines (&lt;code&gt;\n\n&lt;/code&gt;) separating different records. Here's a function that creates a &lt;code&gt;DNAStringSet&lt;/code&gt; given a list of nucleotide identifiers without writing each fasta to a file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;entrez2dss &amp;lt;-&#xA;  function(id_list) {&#xA;    &#xA;    require(rentrez)    # v1.2.3&#xA;    require(Biostrings) # v2.66.0&#xA;    require(stringr)    # v1.5.0&#xA;    &#xA;    # fetch all sequences, trim empty elements&#xA;    raw_ &amp;lt;-&#xA;      entrez_fetch(db = &amp;quot;nucleotide&amp;quot;,&#xA;                   rettype = &amp;quot;fasta&amp;quot;,&#xA;                   id = id_list) |&amp;gt;&#xA;      str_split_1(pattern = &amp;quot;\n\n&amp;quot;) |&amp;gt;&#xA;      head(-1)&#xA;    &#xA;    # get sequence names&#xA;    names_ &amp;lt;- &#xA;      gsub(pattern = &amp;quot;\n.*&amp;quot;,&#xA;           replacement = &amp;quot;&amp;quot;,&#xA;           x = raw_) |&amp;gt;&#xA;      gsub(pattern = &amp;quot;^&amp;gt;&amp;quot;,&#xA;           replacement  = &amp;quot;&amp;quot;)&#xA;    &#xA;    # get sequences&#xA;    seqs_ &amp;lt;-&#xA;      sub(pattern = &amp;quot;.*?\n&amp;quot;,&#xA;          replacement = &amp;quot;&amp;quot;,&#xA;          x = raw_) |&amp;gt;&#xA;      gsub(pattern = &amp;quot;\n&amp;quot;,&#xA;           replacement = &amp;quot;&amp;quot;)&#xA;    &#xA;    # create named DNAStringSet object&#xA;    dss &amp;lt;- DNAStringSet(x = seqs_)&#xA;    names(dss) &amp;lt;- names_&#xA;    dss&#xA;    &#xA;  }&#xA;&#xA;seqlist &amp;lt;- c(&amp;quot;NC_045512&amp;quot;, &amp;quot;OR558592&amp;quot;)&#xA;cov2 &amp;lt;- entrez2dss(seqlist)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3967" LastEditorUserId="3967" LastEditDate="2023-10-17T13:54:39.363" LastActivityDate="2023-10-17T13:54:39.363" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21717" PostTypeId="1" CreationDate="2023-10-16T21:28:41.660" Score="2" ViewCount="159" Body="&lt;p&gt;I would like to know the steps of this question and how to determine the forward and the reverse primers and also the length of the sequence, without using NCBI. If you could explain and give an answer step by step I would appreciate it. Thank you&lt;img src=&quot;https://i.stack.imgur.com/t2Goz.png&quot; alt=&quot;the sequence is in the attached image&quot; /&gt;&lt;/p&gt;&#xA;" OwnerUserId="18373" LastEditorUserId="73" LastEditDate="2023-10-17T03:00:35.103" LastActivityDate="2023-10-17T20:04:54.163" Title="1.What size PCR product will be generated using the primers (bold and underlined) in the sequence below?" Tags="|homework|sequence|primer|pcr|dna|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21718" PostTypeId="2" ParentId="21717" CreationDate="2023-10-17T03:12:43.423" Score="3" Body="&lt;p&gt;Without giving too much of the &amp;quot;what they want you to do yourself&amp;quot; bits of this away, here are a few explanations of the image that may help you answer this question:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;As mentioned in the text, the forward and reverse primers are underlined in the image.&lt;/li&gt;&#xA;&lt;li&gt;The &lt;code&gt;-&lt;/code&gt; and &lt;code&gt;+&lt;/code&gt; symbols below the forward-direction sequence help determine the positions of the bases above and below. A &lt;code&gt;+&lt;/code&gt; appears every 10 bases.&lt;/li&gt;&#xA;&lt;li&gt;The first base in the sequence is at position 1441 (as indicated by the numbers) to the left of the sequence. The first &lt;code&gt;+&lt;/code&gt; to the right of that first base represents position 1450.&lt;/li&gt;&#xA;&lt;li&gt;Primers are usually represented as a sequence in a &amp;quot;5-prime to 3-prime direction&amp;quot;, which is almost always the left to right direction of the top sequence in an image like this, and the right to left direction of the bottom sequence.&lt;/li&gt;&#xA;&lt;li&gt;The length of a PCR product includes the primer sequence.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For help beyond that explanation... this looks like a canned question. Unless you give more context around what bioinformatics project you're working on and how this question relates to your project (i.e. explain why it's a real-world problem rather than a text-book question), I'm going to leave answers to the specific questions asked as an exercise to the reader.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-10-17T20:04:54.163" LastActivityDate="2023-10-17T20:04:54.163" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21719" PostTypeId="2" ParentId="21702" CreationDate="2023-10-17T08:49:25.983" Score="1" Body="&lt;p&gt;The previous error message was because the &lt;code&gt;entrez_fetch()&lt;/code&gt; function returns the sequence as a character string, not a file path. To save the sequence to a file, I can use the &lt;code&gt;writeLines()&lt;/code&gt; function. For example, the following code will save the sequence to a file named &lt;code&gt;cov2.fasta&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# Download the FASTA sequence&#xA;cov2 &amp;lt;- entrez_fetch(db = &amp;quot;nucleotide&amp;quot;, rettype = &amp;quot;fasta&amp;quot;, id = c(&amp;quot;NC_045512&amp;quot;, &amp;quot;OR558592&amp;quot;))&#xA;# Save the sequence to a file&#xA;writeLines(cov2, &amp;quot;cov2.fasta&amp;quot;)&#xA;# Read the FASTA sequence into a DNAStringSet object&#xA;cov2_dna &amp;lt;- readDNAStringSet(&amp;quot;cov2.fasta&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18298" LastEditorUserId="650" LastEditDate="2023-10-17T15:12:48.453" LastActivityDate="2023-10-17T15:12:48.453" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21721" PostTypeId="1" AcceptedAnswerId="21723" CreationDate="2023-10-17T18:55:46.747" Score="4" ViewCount="56" Body="&lt;p&gt;I am facing the following issue:&lt;/p&gt;&#xA;&lt;p&gt;I have a rule in Snakemake that looks something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule somerule:&#xA;input:&#xA;    tables = expand(&amp;quot;results/{tables}_table.txt&amp;quot;, tables = [&amp;quot;1&amp;quot;, &amp;quot;2&amp;quot;]),&#xA;output:&#xA;    edit= expand(&amp;quot;results/{tables}_edited.txt&amp;quot;, tables = [&amp;quot;1&amp;quot;, &amp;quot;2&amp;quot;]),&#xA;conda:&#xA;    &amp;quot;../envs/r.yaml&amp;quot;&#xA;script:&#xA;    &amp;quot;../scripts/script.R&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The script used in this rule is as follows:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library()&#xA;&#xA;tab &amp;lt;- read.table(snakemake@input[[&amp;quot;tables&amp;quot;]], sep = &amp;quot;\t&amp;quot;, header = TRUE)&#xA;edit &amp;lt;- some_function(tab)&#xA;write.table(edit, file = snakemake@output[[&amp;quot;edit&amp;quot;]], sep = &amp;quot;\t&amp;quot;, quote = FALSE, row.names = TRUE, col.names = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I execute the code, I get this error:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;Error in file(file, 'rt'): invalid 'description' argument&#xA;Called from: read.table -&amp;gt; file&#xA;Execution halted&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, when I input the data individually (not as wildcards), the script works fine and produces the correct output files. Where could the problem be? My suspicion is that Snakemake is passing all the input data to the R script simultaneously instead of running them individually, but I'm not sure if that's the case and even less sure how to fix it.&lt;/p&gt;&#xA;" OwnerUserId="18385" LastEditorUserId="3694" LastEditDate="2023-10-21T17:55:26.440" LastActivityDate="2023-10-21T17:55:26.440" Title="How can I incorporate wildcards in Snakemake in an R script?" Tags="|r|snakemake|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21722" PostTypeId="1" CreationDate="2023-10-17T21:17:18.693" Score="1" ViewCount="41" Body="&lt;p&gt;I want to genotype samples in small batches with a common microarray (e.g., the GSA). However, the cost is very high for small batches.&lt;/p&gt;&#xA;&lt;p&gt;With a higher sample throughput/volume, per sample cost becomes more reasonable as the processing cost for a single run is spread out over more samples.&lt;/p&gt;&#xA;&lt;p&gt;I'd like to process no more than 16 samples per run, ideally even one or four samples per run. (Being able to submit a single sample for microarray analysis without having to pay to run the rest of the empty chip would be great.)&lt;/p&gt;&#xA;&lt;p&gt;It is possible that some company or core offers &amp;quot;pooled&amp;quot; microarray services, where I could submit a small number of samples that would be processed with other groups' samples on the same microarray run. This would mean there are more samples per run, so it could potentially cost much less.&lt;/p&gt;&#xA;&lt;p&gt;Looking on core and company websites, I don't see this offered anywhere.&lt;/p&gt;&#xA;&lt;p&gt;Does this service exist?&lt;/p&gt;&#xA;" OwnerUserId="17430" LastEditorUserId="17430" LastEditDate="2023-10-29T15:47:31.743" LastActivityDate="2023-10-29T15:47:31.743" Title="Does pooled sample genotyping microarray analysis exist?" Tags="|genomics|microarray|illumina|genotyping|snp-chip|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21723" PostTypeId="2" ParentId="21721" CreationDate="2023-10-18T13:42:30.567" Score="4" Body="&lt;p&gt;Your problem is that Snakemake is passing all the input data to the R script simultaneously instead of running them individually :)&lt;/p&gt;&#xA;&lt;p&gt;As written your rule takes a list of input files and gives a list of output files (&lt;code&gt;expand&lt;/code&gt; is just a formatting helper function that returns a list based on its arguments).  Instead maybe have one rule that calls the R script with one input and one output, and another rule that asks for all those outputs as an easy way run the whole thing at once.&lt;/p&gt;&#xA;&lt;p&gt;Maybe something like this?  I changed &lt;code&gt;tables&lt;/code&gt; to &lt;code&gt;table&lt;/code&gt; here just to emphasize it's one by one.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule all:&#xA;    input: expand(&amp;quot;results/{table}_edited.txt&amp;quot;, table = [&amp;quot;1&amp;quot;, &amp;quot;2&amp;quot;])&#xA;&#xA;rule somerule:&#xA;    input:&#xA;        table = &amp;quot;results/{table}_table.txt&amp;quot;&#xA;    output:&#xA;        edit = &amp;quot;results/{table}_edited.txt&amp;quot;&#xA;    conda: &amp;quot;../envs/r.yaml&amp;quot;&#xA;    script: &amp;quot;../scripts/script.R&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4980" LastEditorUserId="4980" LastEditDate="2023-10-21T14:57:34.473" LastActivityDate="2023-10-21T14:57:34.473" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21724" PostTypeId="2" ParentId="21693" CreationDate="2023-10-18T17:01:47.113" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;However, the column names won't match&#xA;exactly. I know that I can use BETA instead of OR, and FRQ might&#xA;correspond to MAF, but I can't seem to find the imputation information&#xA;score INFO.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes. There is not a standard format that everyone reports GWAS summary statistics in, unfortunately. First, check the paper that you are sourcing it from to see if they already filtered on imputation quality. Do ctrl+f and look in the methods section for imputation information (you may also try searching for imputation tools, like minimac or Beagle).&lt;/p&gt;&#xA;&lt;p&gt;The good news is that if the sumstats have many alleles, imputation accuracy doesn't matter much. Errors get averaged out and each error only has a small effect since there are many alleles. Usually it is expected that reported sumstats will be pre-QC'd in some way, especially if they lack INFO. It would be safe to proceed without the INFO filter step.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The next step in the tutorial involves filtering based on specific&#xA;criteria, such as MAF and INFO values. However, I can't proceed&#xA;because these columns are missing.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Sumstat formatting is whatever the author wants, and can be a bit annoying to parse. However, FRQ is basically MAF.&lt;/p&gt;&#xA;&lt;p&gt;See here (ctrl+f &amp;quot;frq&amp;quot;):&#xA;&lt;a href=&quot;https://bioconductor.riken.jp/packages/3.14/bioc/vignettes/MungeSumstats/inst/doc/MungeSumstats.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.riken.jp/packages/3.14/bioc/vignettes/MungeSumstats/inst/doc/MungeSumstats.html&lt;/a&gt;&#xA;&lt;a href=&quot;https://academic.oup.com/bioinformatics/article/37/23/4593/6380562&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://academic.oup.com/bioinformatics/article/37/23/4593/6380562&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I would like to primarily use the terminal and only turn to R for&#xA;statistical analysis. Could you guide me on the correct approach to&#xA;convert the .vcf file to .txt so that I can continue following the&#xA;tutorial with PLINK?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Did you try unzipping and simply replacing .vcf with .txt in the filename?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;General approach In addition to addressing the conversion issue, I'm also seeking guidance on the appropriate data sources for genetic&#xA;analysis. I'm curious about whether OpenGWAS is a reliable source for&#xA;publication or if there are other recommended sources like dbGaP.&#xA;Additionally, what is the ideal data format to obtain?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This aligns with your first question. If you use a database, sumstats will be standardized (usually) to the database's common format, so there will be less confusion over inconsistent column names. The GWAS catalog pretty good in my opinion, and all of them are acceptable.&lt;/p&gt;&#xA;" OwnerUserId="17430" LastActivityDate="2023-10-18T17:01:47.113" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21725" PostTypeId="1" CreationDate="2023-10-18T17:48:40.473" Score="3" ViewCount="106" Body="&lt;p&gt;I am aligning my Small RNA Seq data with Bowtie 2. Although the alignment performs well, the only information I obtain after finishing running the alignment is the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;## Time loading reference: 00:00:00&#xA;## Time loading forward index: 00:00:00&#xA;## Time loading mirror index: 00:00:00&#xA;## Multiseed full-index search: 00:00:00&#xA;## Time searching: 00:00:00&#xA;## Overall time: 00:00:00&#xA;## Time loading reference: 00:00:00&#xA;## Time loading forward index: 00:00:00&#xA;## Time loading mirror index: 00:00:00&#xA;## Multiseed full-index search: 00:00:01&#xA;## Time searching: 00:00:01&#xA;## Overall time: 00:00:01&#xA;## Time loading reference: 00:00:00&#xA;## Time loading forward index: 00:00:00&#xA;## Time loading mirror index: 00:00:00&#xA;## Multiseed full-index search: 00:00:00&#xA;## Time searching: 00:00:00&#xA;## Overall time: 00:00:00&#xA;## Time loading reference: 00:00:00&#xA;## Time loading forward index: 00:00:00&#xA;## Time loading mirror index: 00:00:00&#xA;## Multiseed full-index search: 00:00:00&#xA;## Time searching: 00:00:00&#xA;## Overall time: 00:00:00&#xA;## Time loading reference: 00:00:00&#xA;## Time loading forward index: 00:00:00&#xA;## Time loading mirror index: 00:00:00&#xA;## Multiseed full-index search: 00:00:00&#xA;## Time searching: 00:00:00&#xA;## Overall time: 00:00:00&#xA;## Time loading reference: 00:00:00&#xA;## Time loading forward index: 00:00:00&#xA;## Time loading mirror index: 00:00:00&#xA;## Multiseed full-index search: 00:00:00&#xA;## Time searching: 00:00:00&#xA;## Overall time: 00:00:00&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is my command line to build the index:&#xA;bowtie2-build -o 3 -t 5 Nuevofastaorina human_mirs&lt;/p&gt;&#xA;&lt;p&gt;And this is my command line to do the alignment:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bowtie2 -x human_mirs -U 1.lane.clean.gz -N 1 --norc --no-unal --no-head --ignore-quals –-quiet &amp;gt; sr1prueba.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Does anyone know how to obtain an alignment summary instead of the above information? What I mean by alignment summary is the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;20000 reads; of these:&#xA;  20000 (100.00%) were unpaired; of these:&#xA;    1247 (6.24%) aligned 0 times&#xA;    18739 (93.69%) aligned exactly 1 time&#xA;    14 (0.07%) aligned &amp;gt;1 times&#xA;93.77% overall alignment rate&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18391" LastEditorUserId="650" LastEditDate="2023-10-18T21:24:48.400" LastActivityDate="2023-10-18T21:24:48.400" Title="Cannot obtain alignment summary after running Bowtie2" Tags="|rna-seq|sequence-alignment|bowtie2|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21726" PostTypeId="2" ParentId="21725" CreationDate="2023-10-18T17:56:40.377" Score="2" Body="&lt;p&gt;Get &lt;a href=&quot;http://www.htslib.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;samtools&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Then run:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools stats file.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and/or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools flagstat file.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For example outputs see &lt;a href=&quot;https://www.biostars.org/p/412621/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-10-18T17:56:40.377" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21727" PostTypeId="2" ParentId="21725" CreationDate="2023-10-18T21:09:21.823" Score="5" Body="&lt;p&gt;Remove the &lt;code&gt;--quiet&lt;/code&gt; option, and capture STDERR as well as STDOUT:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bowtie2 -x human_mirs -U 1.lane.clean.gz -N 1 --norc --no-unal --no-head --ignore-quals 1&amp;gt;sr1prueba.sam 2&amp;gt;sr1prueba.summary.log&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;See also:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bowtie-bio.sourceforge.net/bowtie2/manual.shtml#alignment-summary&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;bowtie2&lt;/code&gt; manual: alignment summary&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6251" LastActivityDate="2023-10-18T21:09:21.823" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21728" PostTypeId="1" CreationDate="2023-10-19T07:28:33.577" Score="1" ViewCount="84" Body="&lt;p&gt;I'm trying to add polar hydrogens to protein (PDB ID 3l3l cleaned from water and ligands) and convert it to PDBQT using OpenBabel by:&#xA;&lt;code&gt;obabel -ipdb target.pdb -opdbqt -O target.pdbqt -xr --partialcharge gasteiger --addpolarh&lt;/code&gt;&#xA;It's working well, but all hydrogens are appending to the end of file. I want to renumber atoms so that residues in PDB would go in increasing order and hydrogens were located &amp;quot;between&amp;quot; corresponding heavy atoms, like AutoDock Tools do this.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to renumber them using OpenBabel? Or any other command-line utilities?&lt;/p&gt;&#xA;" OwnerUserId="18394" LastActivityDate="2023-10-19T07:28:33.577" Title="Renumbering atoms in PDB using OpenBabel" Tags="|proteins|docking|format-conversion|" AnswerCount="0" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21729" PostTypeId="1" CreationDate="2023-10-19T18:59:19.693" Score="3" ViewCount="47" Body="&lt;p&gt;My RNAseq analysis pipeline is as follows: fastqc (read quality is good, some overrepresentation of adaptor sequence) → trimmomatic (trimmed adaptor sequence, qc report after trimming suggests the overrepresented adaptors are gone) → HISAT2 (used a forward stranded alignment protocol) → featurecounts&lt;/p&gt;&#xA;&lt;p&gt;My HISAT 2 command line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;hisat2 --rna-strandness F -x HISAT2/grch38/genome -U &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$fastq_file&quot; | samtools sort -o &quot;HISAT2/$&lt;/span&gt;{sample_name}_aligned.bam&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My featurecounts command line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;featureCounts -s 2 -a hg38/gencode.v44.chr_patch_hapl_scaff.annotation.gtf -o &amp;quot;quant/&lt;span class=&quot;math-container&quot;&gt;${sample_name}_featureCounts.txt&quot; &quot;$&lt;/span&gt;bam_file&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, the strange thing is that if I use a forward stranded protocol in feature counts (with the forward stranded alignment I generated with HISAT2), the assignment rate is 5%, with the majority of unassigned reads being having no features. But if I use a reverse stranded protocol in feature counts, the assignment rate is 70%. Does this suggest my data is reverse stranded?&lt;/p&gt;&#xA;&lt;p&gt;My library was prepared by the NEB directional RNA library prep kit that claims to produce forward stranded libraries, so I’m really confused by this result. How should I proceed with this data? Should I run an alignment with a reverse strand protocol and just proceed with a reverse strand assignment? Or should I proceed with the forward alignment and forward strand assignment data because that’s what the library is supposed to be?&lt;/p&gt;&#xA;&lt;p&gt;Thank you so much for your help!&lt;/p&gt;&#xA;" OwnerUserId="18398" LastActivityDate="2023-10-21T13:04:38.247" Title="Discrepancy with featurecounts analysis using a forward stranded and reverse stranded protocol" Tags="|rna-seq|rna-alignment|featurecounts|strandedness|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21730" PostTypeId="2" ParentId="21659" CreationDate="2023-10-20T16:58:11.767" Score="2" Body="&lt;p&gt;I agree with @terdon that perhaps dbSNP is what you may want to look at. The dbSNP155 RefSNP VCF file in GRCh37 version may help. It can be downloaded from &lt;a href=&quot;https://ftp.ncbi.nlm.nih.gov/snp/latest_release/vcf/GCF_000001405.25.gz&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ftp.ncbi.nlm.nih.gov/snp/latest_release/vcf/GCF_000001405.25.gz&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You may also fine the 1000G genome website helpful. &lt;a href=&quot;https://www.internationalgenome.org/data-portal/data-collection&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.internationalgenome.org/data-portal/data-collection&lt;/a&gt; contains much of its release both in &lt;code&gt;.fa&lt;/code&gt; format and in VCF format. I understand that a large number of samples included increase the file size tremendously, but it is necessary as you're not just looking at what the major and minor allele is, but also their frequency, which may vary greatly between populations. It's important to check which population we're seeing when using their genomic information.&lt;/p&gt;&#xA;&lt;p&gt;Hope my answer helps. Regards.&lt;/p&gt;&#xA;" OwnerUserId="16537" LastActivityDate="2023-10-20T16:58:11.767" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21731" PostTypeId="2" ParentId="21247" CreationDate="2023-10-20T18:49:17.073" Score="2" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Is 1kg.v3.tgz based on GRCh37?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is there a GRCh38 reference version of 1000 genomes that is split to different super populations (EUR etc)?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Yes, the genome build for 1000 Genomes phase 3 release is in GRCh37. Luckily ISGR has reanalyzed their original sequencing data based on GRCh38. The data is provided both at the ISGR website &lt;a href=&quot;https://www.internationalgenome.org/data-portal/data-collection/30x-grch38&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.internationalgenome.org/data-portal/data-collection/30x-grch38&lt;/a&gt; and it is also made available by Christopher at PLINK2 website &lt;a href=&quot;https://www.cog-genomics.org/plink/2.0/resources&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.cog-genomics.org/plink/2.0/resources&lt;/a&gt;. However I suspect the data was not split into super populations.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Can you use GRCh37 based reference to LD clump GRCh38 GWAS results?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I wouldn't recommend doing so. Some variants identified in GRCh37 has been merged into others in GRCh38. Some variants are assigned different rsIDs. The position for the variant may also differ in different genome builds. Therefore clumping GRCh38 GWAS results with GRCh37 based reference may lead to fewer variants correctly mapped, which may influence your furthur analysis.&lt;/p&gt;&#xA;&lt;p&gt;Hope you find it helpful. Best regards.&lt;/p&gt;&#xA;" OwnerUserId="16537" LastActivityDate="2023-10-20T18:49:17.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21732" PostTypeId="1" CreationDate="2023-10-21T08:36:25.507" Score="1" ViewCount="65" Body="&lt;p&gt;running this code on R, using the Biostrings package,&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;vmatchPattern(&amp;quot;GAAGACCGAGGCCACGCGGAGTACGAACGAGGGTACAGTCTTC&amp;quot;, mySeq)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;in the console, it writes:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;...&#xA;&amp;lt;22 more elements&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I see all the results? The &lt;code&gt;print&lt;/code&gt; function returns the same result.&lt;/p&gt;&#xA;" OwnerUserId="18298" LastEditorUserId="650" LastEditDate="2023-10-23T19:55:30.367" LastActivityDate="2023-10-23T19:55:30.367" Title="seeing full results on R" Tags="|r|biostrings|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21733" PostTypeId="2" ParentId="21732" CreationDate="2023-10-21T13:02:18.287" Score="4" Body="&lt;p&gt;It returns an IRanges object. You can transform it to a data.frame using &lt;code&gt;as.data.frame()&lt;/code&gt; and then either write as a file with something like &lt;code&gt;write.table()&lt;/code&gt; to view outside of R or use &lt;code&gt;View()&lt;/code&gt; in RStudio to explore it.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-10-21T13:02:18.287" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21734" PostTypeId="2" ParentId="21729" CreationDate="2023-10-21T13:04:38.247" Score="3" Body="&lt;p&gt;This kit uses the dUTP method to preserve strandedness. That (to my knowledge) always produces a reverse library. Not sure where you have your information of a forward library from, but we used this kit for many years and it is reverse.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-10-21T13:04:38.247" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21735" PostTypeId="2" ParentId="21705" CreationDate="2023-10-22T01:28:00.690" Score="2" Body="&lt;p&gt;&lt;em&gt;Its really easy&lt;/em&gt;. No formula is needed.&lt;/p&gt;&#xA;&lt;p&gt;Protein matrices are based on observed frequencies from nucleotide alignments for a variety of genes. They are not based on &lt;em&gt;a priori&lt;/em&gt; formulas (it's not like nucleotides). Later matrices incorporated phylogenetic criteria using maximum likelihood - it's really complicated how it works.&lt;/p&gt;&#xA;&lt;p&gt;Blosum used odds-ratio (OR) I think without a phylogenetic tree. I suspect raw OR. Explaining with/without a phylogenetic tree is a full lecture in phylogenetic theory, just think it'll be a raw value calculated directly.&lt;/p&gt;&#xA;&lt;p&gt;So lets just run with raw OR based matrix ...&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;So&lt;/em&gt; ... &lt;code&gt;*&lt;/code&gt; is a stop-codon it carries the lowest weight in the matrix because each protein only has one of them. &lt;em&gt;So&lt;/em&gt; a stop-codon frequency in comparison to any amino acid is going to be extremely low - because for that aligned position there's only stop-codons (i.e. no other amino acids).  Thus its &lt;code&gt;-4&lt;/code&gt; here (lowest value in the matrix).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What about the 1?&lt;/strong&gt;&#xA;&lt;em&gt;However&lt;/em&gt;, when you compare a stop codon frequency ... between proteins, well its occurrence is always 1 to 1, right? Each protein has one stop codon, compare it to another protein ... thats got one stop-codon too. So one million proteins which are nicely aligned to be homologues (thats how its calculated) have one million stop codons ... thus its &lt;code&gt;1&lt;/code&gt; no matter how many protein there are in the alignment. The alignment position is always the same because stop codons (at least in Blosum) are always homologous.&lt;/p&gt;&#xA;&lt;p&gt;Once more, comparing a stop codon (in a protein), with the other stop codon in another protein some the same homologous &amp;quot;amino acid&amp;quot; site - no matter which protein it is - its always going to be &lt;code&gt;1&lt;/code&gt; because its a universal feature of all proteins.&lt;/p&gt;&#xA;&lt;p&gt;Thats how it works and I would simply cite&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Henikoff, S.; Henikoff, J.G. (1992). &amp;quot;Amino Acid Substitution Matrices from Protein Blocks&amp;quot;. PNAS. 89 (22): 10915–10919&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I've checked the paper and the authors didn't use stop codons, nor did they use any phylogenetic criteria (that was easy to guess). I know they didn't include stop codons because because,&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The Sigma function for matrix operations has a &lt;code&gt;20&lt;/code&gt; above it (i.e. 20 amino acids), there's no allowance for stop codons.&lt;/li&gt;&#xA;&lt;li&gt;The matrices they present omit stop codons.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;When NCBI are leveraging this matrix they need answers for all possibilities, the idea that &lt;code&gt;X&lt;/code&gt; (any amino acid) or &lt;code&gt;*&lt;/code&gt; is omitted would not be cool for them. Thus they could have approached the authors to fill in this information, this would probably mean redoing the alignments to include the stop codon (which the authors wouldn't have liked), or NCBI could have added it after the event (its a pretty safe addition) because the log-ratio of an invariant site is always &lt;code&gt;1&lt;/code&gt;. The authors/NCBI would just need to be sure the  stop codon was in the same position for all protein alignments in Blosum.&lt;/p&gt;&#xA;&lt;p&gt;You might find the wiki more understandable  &lt;a href=&quot;https://en.wikipedia.org/wiki/BLOSUM&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://en.wikipedia.org/wiki/BLOSUM&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-10-22T15:32:07.063" LastActivityDate="2023-10-22T15:32:07.063" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21736" PostTypeId="1" AcceptedAnswerId="21739" CreationDate="2023-10-23T10:31:49.557" Score="4" ViewCount="126" Body="&lt;p&gt;I would like to create a phylogenetic tree for the most species in my dataset.&lt;/p&gt;&#xA;&lt;p&gt;I'm starting with around 1200 species, but since it's not good practice to align short and long sequences I tried filtering only for the species with the [15k-18k bp] range in length, but this leaves me with around 350 species in total.&lt;/p&gt;&#xA;&lt;p&gt;Those sequences vary a lot in size, here's an histogram of the number of species I have data for&#xA;&lt;a href=&quot;https://i.stack.imgur.com/aYzXX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/aYzXX.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;As you can see I have a lot of sequences with length [12k-15k bp] and [15k-18k bp] (values on the x axis are scaled by a factor of 3000 and then rounded).&#xA;I would like maximise the species genetic information.&lt;br /&gt;&#xA;To do so I would like to extract the part of genome that is shared the most among all the species.&lt;/p&gt;&#xA;&lt;p&gt;Is this achievable by using a regex?&lt;br /&gt;&#xA;I am not sure on the length and type of the query to be sure of not extracting non-homologous regions as it would create a big problem later on for the phylogenetic tree creation.&lt;/p&gt;&#xA;&lt;p&gt;I can't make an alignment for all species because of the lack of computational power. That is why I'm trying to subset a piece of DNA that is the most shared (prior to the alignment, but I don't know if that is doable or logic)&lt;/p&gt;&#xA;&lt;p&gt;My question is: how would you proceed to get the most phylogenetic data out of this dataset?&lt;/p&gt;&#xA;&lt;p&gt;If you would use a regex, what type of query would you use?&lt;/p&gt;&#xA;&lt;h1&gt;More Info&lt;/h1&gt;&#xA;&lt;h3&gt;Alignment&lt;/h3&gt;&#xA;&lt;p&gt;When I try to align 10 sequences drawn from the [15k-18k bp] range I get this result (the numbers on the right are the TaxID of the species).&lt;br /&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/W53d5.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/W53d5.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;We can't see much here so I used an online MSA viewer&#xA;[https://alignmentviewer.org/]&#xA;&lt;a href=&quot;https://i.stack.imgur.com/1UKNq.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1UKNq.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;h3&gt;Data&lt;/h3&gt;&#xA;&lt;p&gt;My data is one Fasta file containing mitochondrial genomes for all the species.&lt;/p&gt;&#xA;&lt;p&gt;To do this preliminary analysis I only used 10 randomly chosen species with a sequence of length in the range [15k-18k bp].&lt;/p&gt;&#xA;" OwnerUserId="18351" LastEditorUserId="73" LastEditDate="2023-10-26T20:37:36.323" LastActivityDate="2023-10-26T20:37:36.323" Title="How to create a phylogenetic tree from diverse mitochondrial genomes" Tags="|phylogenetics|sequence-alignment|phylogeny|multiple-sequence-alignment|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21737" PostTypeId="2" ParentId="21736" CreationDate="2023-10-24T03:14:01.213" Score="2" Body="&lt;p&gt;It looks like these are random mitochondrial sequences pulled from NCBI, which is probably why the alignments aren't working well.&lt;/p&gt;&#xA;&lt;p&gt;As a starting point, I recommend you make sure that all the mitochondrial DNA sequences are rotated and reverse-complemented as necessary to make sure that the same gene matches position 1 of the sequence. I prefer doing this for COX1, because it's a well-conserved gene. After doing that, it'll be much easier to look at the alignment of that gene alone, which will give you an idea of the complexity of extending the task to the full length of the mitochondrial genome.&lt;/p&gt;&#xA;&lt;p&gt;I don't expect you'll find a long exact-match sequence that is consistently conserved across all species, even within COX1. It wouldn't surprise me if the longest conserved sequence were less than 16bp, with a few repeats of that sequence throughout the genome. If that is the case, you need to think about what to do with that information. Given the complexity of this intermediate problem you've asked about, it would be a good idea to think first about what you would do after you discovered there were no such sequence.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-10-24T03:19:09.880" LastActivityDate="2023-10-24T03:19:09.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21738" PostTypeId="1" CreationDate="2023-10-24T13:42:26.927" Score="3" ViewCount="69" Body="&lt;p&gt;I have RNA-seq Data from lung cancer immunotherapy patients but i don't have any single cell data from my sample, so i wanted to do a deconvolution with a public single cell data. I thought of doing the analysis with the cibersort website plataform but i wanted to obtain the results in R, to be easy to work with them.So i followed the steps in this article, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5895181/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5895181/&lt;/a&gt; :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(e1071)&#xA;library(parallel)&#xA;library(preprocessCore)&#xA;source(&amp;quot;CIBERSORT.R&amp;quot;)&#xA;results &amp;lt;- CIBERSORT(‘sig_matrix_file.txt’, ‘mixture_file.txt’, perm=100, QN=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But in when i did source(&amp;quot;CIBERSORT.R&amp;quot;), i obtain this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;quot;Error in file(filename, &amp;quot;r&amp;quot;, encoding = encoding) : &#xA;  cannot open the connection&#xA;In addition: Warning message:&#xA;In file(filename, &amp;quot;r&amp;quot;, encoding = encoding) :&#xA;  cannot open file 'CIBERSORT.R': No such file or directory&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I don't understand i was supposed to have a file?&lt;/p&gt;&#xA;" OwnerUserId="17902" LastActivityDate="2023-10-24T13:42:26.927" Title="how to install cibersort in R, to do deconvolution using my RNA-seq Data with public single cell data" Tags="|rna-seq|cibersort|deconvolution|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21739" PostTypeId="2" ParentId="21736" CreationDate="2023-10-24T14:42:38.233" Score="2" Body="&lt;p&gt;Your overall approach is cool, given caveats (below) to adapt the method. I can fairly authoritatively say your specific methodological approach will not work regarding its phylogenetic strategy without changing it. This is due to a phenomenon where the number of mutations becomes so great it is impossible to recover the ancestral state, basically it randomises the phylogenetic signal, this especially occurs at the 3rd codon nucleotide positions, i.e. 1 in every 3 nucleotides, 33.3% of the entire data set. This violates a central tenant of phylogenetic theory, therefore I am afraid that you cannot build a tree using this approach, but again you can adapt it to work (below).&lt;/p&gt;&#xA;&lt;p&gt;If you want only to use nucleotides then 16S is the only target available. However, 16S has been a good target for amphibians, therefore just aligning that gene and building a tree is cool. Its perfectly solid given prior work in vertebrates ...&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1186/s12862-022-01994-y&quot; rel=&quot;nofollow noreferrer&quot;&gt;Chan, K.O., Hertwig, S.T., Neokleous, D.N. et al. Widely used, short 16S rRNA mitochondrial gene fragments yield poor and erratic results in phylogenetic estimation and species delimitation of amphibians. BMC Ecol Evo 22, 37 (2022).&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If you wish to use protein coding genes you need to switch amino acids and align those. In terms of conservation there alot of established tools rather than &amp;quot;reinvent the wheel&amp;quot;. One program you could consider is &lt;strong&gt;AAcon&lt;/strong&gt;, &lt;a href=&quot;http://www.compbio.dundee.ac.uk/aacon/&quot; rel=&quot;nofollow noreferrer&quot;&gt;available here&lt;/a&gt;. This would give you can empirical basis for deducing which gene is&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the most conserved&lt;/li&gt;&#xA;&lt;li&gt;the most represented, i.e. &amp;quot;read-depth&amp;quot;, or species representation.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;... after its aligned, thereby yielding a genetic marker for phylogenetics. It uses a large number of metrics therefore most scenarios would be accommodated.&lt;/p&gt;&#xA;&lt;p&gt;COX1 is good target and likely gene to emerge from AAcon analysis.&lt;/p&gt;&#xA;&lt;p&gt;Building protein phylogenetic trees is a separate question.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Lacking computational power to align&lt;/strong&gt;. That is solvable, even on small computers, and you can place that as a separate question stating your computer capacity (RAM and CPUs). It's not difficult to align 1000s even on a small machine, it's simply the strategy to do this. Algorithm performance and accuracy change rapidly.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-10-24T15:43:20.290" LastActivityDate="2023-10-24T15:43:20.290" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21742" PostTypeId="1" CreationDate="2023-10-25T08:07:20.680" Score="-1" ViewCount="39" Body="&lt;p&gt;I have developed an R function to download the &amp;quot;organism&amp;quot; name of the numerous NCBI accession numbers which are already saved in an object.&#xA;The function:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;get_organism_names &amp;lt;- function(accession_numbers) {&#xA;  organism_names &amp;lt;- sapply(accession_numbers, function(accession) {&#xA;    record &amp;lt;- entrez_summary(db = &amp;quot;nucleotide&amp;quot;, id = accession)&#xA;    return(record$organism)&#xA;  }, USE.NAMES = FALSE)&#xA;  names(organism_names) &amp;lt;- accession_numbers&#xA;  return(organism_names)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Although it works for some accession numbers, for others it encounters an error stating for example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Warning: Esummary includes error message: Invalid uid KU253616.1 at position= 0Error: No esummary records found in file&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I either solve this error or ask R to skip the problematic accession and keep downloading the rest?&lt;/p&gt;&#xA;" OwnerUserId="18298" LastEditorUserId="298" LastEditDate="2023-10-25T12:53:10.900" LastActivityDate="2023-11-02T09:47:51.477" Title="error in downloading the annotation of sequences on NCBI" Tags="|r|ncbi|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21743" PostTypeId="1" CreationDate="2023-10-25T08:16:54.527" Score="0" ViewCount="45" Body="&lt;p&gt;I want to annotate 25,000+ proteins with GO_ID, TERM, and CATEGORY prior to doing enrichment analysis. Can I extract this information by submitting my sequences? Meanwhile, my species is a fish, and it's a non-model organism. Since my species is a non-model species, I have annotated the proteins from the whole genome by myself using interproscan.  There I was able to find GO IDs but there was no term or category which is not helpful for Enrichment analysis.&lt;/p&gt;&#xA;&lt;p&gt;Seq type:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;maker-ctg000210_np1212-snap-gene-104.0-mRNA-1 p/t AED:0.26 eAED:0.29 QI:38|0.66|0.5|1|0.33|0.25|4|0|342&#xA;MPCPSKRSRAAKKRKADLRSTEVPSKRSPVAQVDLVSCDEVGRPRKKAHTQSSDAIQSSDCQLQCQLQCQ&#xA;LQCQLQSSDAIQANSTNSQTNSTNSLNMCHDNNATDCCFLNESVDFPERSVLMGSFHQGDSRFGNVRNKQ&#xA;CGAISLTAVLKSKMKNVWSWETGDLDDVLIKGTVLYSTPRFEVTGVTAVTHATAQMTDSSSCATSSVNYT&#xA;VPGKDVKQNVCRPLYVHEVGVTLNRDCASKDENRSCECNVRKPETKQDKQSFNESTTDDVILVNVETHKQ&#xA;SLKRNLQATDEVIISDVRFHKQVFKPLTRHDQDALCTCLDLQNENVYLGLMATTDLDDMGCP&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After annotation:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;maker-ctg001000_np1212-snap-gene-144.133-mRNA-1       710fdf6795f6f6bfbb960a017c734103        1139    ProSiteProfiles PS50056 Tyrosine specific protein phosphatases domain profile.  354     408     12.21363        T       20-02-2023      IPR000387       Tyrosine-specific protein phosphatases domain   GO:0016311|GO:0016791&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I want to use my protein sequences and the following output I need:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Seq_id    GO_ID    GO_term    GO_categoty&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would appreciate any of your suggestions. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="12579" LastEditorUserId="298" LastEditDate="2023-10-25T15:15:59.140" LastActivityDate="2024-03-24T04:07:35.853" Title="GO Enrichment Analysis" Tags="|go-enrichment|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21745" PostTypeId="2" ParentId="2242" CreationDate="2023-10-25T17:05:10.397" Score="2" Body="&lt;p&gt;If anybody needs a simple bed to gff converter, simple in the sense that only takes a bed with a single type of feature, lets say peaks and translate it to GFF I recently wrote this, is Bash and as far as I know all the tools it uses should be present in most Linux / Mac operating systems.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/UBMI-IFC/bed2gff/tree/main&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/UBMI-IFC/bed2gff/tree/main&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It cannot deal, however with multi featured Bed files as it ignores everything from the 7th column and beyond.&lt;/p&gt;&#xA;" OwnerUserId="18427" LastEditorUserId="18427" LastEditDate="2023-10-25T17:07:17.143" LastActivityDate="2023-10-25T17:07:17.143" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21746" PostTypeId="1" CreationDate="2023-10-26T00:12:51.480" Score="2" ViewCount="53" Body="&lt;p&gt;I am working on using WGCNA for a bulk RNA-seq experiment. I have three experimental conditions (control, treatment 1, and treatment 2) and have a total sample size of 18 (6 per group). Using the online WGCNA tutorials, I could analyze my datasets after variance stabilizing transformation (vst). I got the summary of network analysis results as a CSV with Gene significance (GS), p.GS, module membership(MM), and associated p-value (p.MM).&lt;/p&gt;&#xA;&lt;p&gt;I want to extract a set of significant genes from the module with the highest positive correlation to my treatment conditions. As I am just getting acquainted with WGCNA interpretation, which of these parameters in the CSV should I use for filtering further (GS, p.GS, and module membership and associated p-value (p.MM))?&lt;/p&gt;&#xA;&lt;p&gt;Also, how would I choose the limits of the criteria? Should it be using the scatter plot (MM vs GS)?&lt;/p&gt;&#xA;&lt;p&gt;As I am pretty new to WGCNA, any help is appreciated. Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="18429" LastEditorUserId="18429" LastEditDate="2023-10-26T00:39:49.353" LastActivityDate="2023-10-29T16:11:27.123" Title="How do I select a subset of genes for functional enrichment(GO/KEGG) analysis from WGCNA results?" Tags="|networks|wgcna|go-enrichment|kegg|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21747" PostTypeId="2" ParentId="21743" CreationDate="2023-10-26T03:27:57.933" Score="0" Body="&lt;p&gt;It's unclear what you're asking. Can you please give some more information about what you will do with this output once you've found it.&lt;/p&gt;&#xA;&lt;p&gt;The single annotation you have provided has GO IDs, and using &lt;a href=&quot;https://amigo.geneontology.org/amigo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;AmiGO&lt;/a&gt;, I get the following results:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://amigo.geneontology.org/amigo/term/GO:0016311&quot; rel=&quot;nofollow noreferrer&quot;&gt;GO:0016311&lt;/a&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Name: dephosphorylation&lt;/li&gt;&#xA;&lt;li&gt;Ontology: biological_process&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://amigo.geneontology.org/amigo/term/GO:0016791&quot; rel=&quot;nofollow noreferrer&quot;&gt;GO:0016791&lt;/a&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Name: phosphatase activity&lt;/li&gt;&#xA;&lt;li&gt;Ontology: molecular_function&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Bear in mind that GO is a hierarchical annotation, and most GO enrichment analysis programs ignore this hierarchy when carrying out their analysis, which is even more important for a less studied non-model species, where annotation is unlikely to be complete at any level.&lt;/p&gt;&#xA;&lt;p&gt;Using a standard enrichment program that ignores the hierarchy would require fully populating the GO tree for each protein. For example, using the inferred tree view from AmiGO for that one protein:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;GO:0016311 -&amp;gt; GO:0008150|GO:0009987|GO:0008152|GO:0044237|GO:0006793|GO:0006796|GO:0016311&lt;/li&gt;&#xA;&lt;li&gt;GO:0016791 -&amp;gt;  GO:0003674|GO:0003824|GO:0016787|GO:0016788|GO:0042578|GO:0016791&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Consider this when thinking about the type of output you want. This will not be a single GO term/category per sequence; each sequence will have multiple GO IDs, multiple terms, and multiple categories, with many overlaps and lots of gaps.&lt;/p&gt;&#xA;&lt;p&gt;Further information about exploring the GO database (both automated and manual) can be found here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://amigo.geneontology.org/amigo/software_list&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://amigo.geneontology.org/amigo/software_list&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-10-26T03:27:57.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21749" PostTypeId="1" CreationDate="2023-10-26T07:24:48.520" Score="0" ViewCount="36" Body="&lt;p&gt;I have a 32-bp query that I want to perform BLAST+. my query represents different RNA viruses and I do expect that my hits get more than 500,000.&#xA;but in practice, when I perform it I get around only 700 hits. would you please kindly help me how to increase the efficacy of my BLAST+?&#xA;I leave my syntax here.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;blastn -task megablast -db nr -outfmt 7 -remote -max_target_seqs 99999999 -perc_identity 80 -query D:/Query.fas  -out D:/result.txt &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18298" LastEditorUserId="298" LastEditDate="2023-10-26T12:47:20.280" LastActivityDate="2023-10-28T11:25:33.853" Title="how to use BLAST+ in an optimal way" Tags="|blast|blastn|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21751" PostTypeId="1" CreationDate="2023-10-26T11:11:42.437" Score="0" ViewCount="77" Body="&lt;p&gt;I am facing an issue when running snakemake with a container and conda.&#xA;Let me explain better.&#xA;I am developing a workflow (locally) in snakemake, where some rules use a conda env (defined in /envs/env.yaml) and other rules use wrappers (so they also are based on conda envs).&lt;/p&gt;&#xA;&lt;p&gt;An example here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;rule clean_spike:&#xA;    input:&#xA;        sample_ref=&amp;quot;{}results/bam/{{id}}.tmp.bam&amp;quot;.format(outdir),&#xA;        sample_spike=&amp;quot;{}results/bam_spike/{{id}}_spike.tmp.bam&amp;quot;.format(outdir),&#xA;        sample_ref_index=&amp;quot;{}results/bam/{{id}}.tmp.bam.bai&amp;quot;.format(outdir),&#xA;        sample_spike_index=&amp;quot;{}results/bam_spike/{{id}}_spike.tmp.bam.bai&amp;quot;.format(outdir),&#xA;    output:&#xA;        sample_ref=&amp;quot;{}results/bam/{{id}}.clean.bam&amp;quot;.format(outdir),&#xA;        sample_spike=&amp;quot;{}results/bam_spike/{{id}}_spike.clean.bam&amp;quot;.format(outdir),&#xA;    conda:&#xA;        &amp;quot;../envs/pysam.yaml&amp;quot;&#xA;    log:&#xA;        &amp;quot;{}results/logs/spike/{{id}}.removeSpikeDups&amp;quot;.format(outdir),&#xA;    script:&#xA;        &amp;quot;../scripts/remove_spikeDups.py&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Locally, the pipeline runs smoothly.&lt;/p&gt;&#xA;&lt;p&gt;Now, I wanted to test the pipeline on another machine (server from the institute), and so I run the command&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;snakemake --containerize &amp;gt; Dockerfile&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I got a dockerfile and  I generated an image that was uploaded on dockerhub &lt;a href=&quot;https://hub.docker.com/repository/docker/davidebrex/chiprxsnakemake/general&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;At this point, I copied the workflow on the other machine, I added to the Snakefile the line:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;containerized: &amp;quot;docker://davidebrex/chiprxsnakemake:latest&amp;quot;&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;and tried to run the pipeline with: &lt;code&gt;snakemake -c 10 --use-conda --use-singularity.&lt;/code&gt;&#xA;Ideally, snakemake should run the conda envs within the docker container for each rule&lt;/p&gt;&#xA;&lt;p&gt;The problem is that i get an error related to the fact that conda is missing..&#xA;I checked the docker image and I can run conda and see the installed envs.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Any suggestions about what is going on?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;&lt;p&gt;This is the error I get:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;/usr/bin/bash: conda: command not found&#xA;Traceback (most recent call last):&#xA;  File &amp;quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/__init__.py&amp;quot;, line 792, in snakemake&#xA;    success = workflow.execute(&#xA;              ^^^^^^^^^^^^^^^^^&#xA;  File &amp;quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/workflow.py&amp;quot;, line 1246, in execute&#xA;    raise e&#xA;  File &amp;quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/workflow.py&amp;quot;, line 1242, in execute&#xA;    success = self.scheduler.schedule()&#xA;              ^^^^^^^^^^^^^^^^^^^^^^^^^&#xA;  File &amp;quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/scheduler.py&amp;quot;, line 636, in schedule&#xA;    self.run(runjobs)&#xA;  File &amp;quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/scheduler.py&amp;quot;, line 685, in run&#xA;    executor.run_jobs(&#xA;  File &amp;quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py&amp;quot;, line 158, in run_jobs&#xA;    self.run(&#xA;  File &amp;quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py&amp;quot;, line 582, in run&#xA;    future = self.run_single_job(job)&#xA;             ^^^^^^^^^^^^^^^^^^^^^^^^&#xA;  File &amp;quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py&amp;quot;, line 635, in run_single_job&#xA;    self.cached_or_run, job, run_wrapper, *self.job_args_and_prepare(job)&#xA;                                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^&#xA;  File &amp;quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/executors/__init__.py&amp;quot;, line 623, in job_args_and_prepare&#xA;    self.workflow.conda_base_path,&#xA;    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^&#xA;  File &amp;quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/workflow.py&amp;quot;, line 456, in conda_base_path&#xA;    return Conda().prefix_path&#xA;           ^^^^^^^&#xA;  File &amp;quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/deployment/conda.py&amp;quot;, line 653, in __init__&#xA;    shell.check_output(self._get_cmd(&amp;quot;conda info --json&amp;quot;), text=True)&#xA;  File &amp;quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/site-packages/snakemake/shell.py&amp;quot;, line 61, in check_output&#xA;    return sp.check_output(cmd, shell=True, executable=executable, **kwargs)&#xA;           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^&#xA;  File &amp;quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/subprocess.py&amp;quot;, line 466, in check_output&#xA;    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,&#xA;           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^&#xA;  File &amp;quot;/home/davide.bressan-1/mambaforge/envs/new_snakemake/lib/python3.12/subprocess.py&amp;quot;, line 571, in run&#xA;    raise CalledProcessError(retcode, process.args,&#xA;subprocess.CalledProcessError: Command 'conda info --json' returned non-zero exit status 127.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried to change the snakemake command and remove --use-conda but then I get an error related to the fact that the tools are missing.&#xA;I checked out other questions on stackoverflow but they were not helpful in my case (like this one: &lt;a href=&quot;https://stackoverflow.com/questions/64000711/how-to-use-singularity-and-conda-wrappers-in-snakemake&quot;&gt;here&lt;/a&gt;).&lt;/p&gt;&#xA;" OwnerUserId="8911" LastActivityDate="2023-10-26T11:11:42.437" Title="Error in snakemake when running conda envs in container" Tags="|python|snakemake|conda|singularity|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21752" PostTypeId="1" CreationDate="2023-10-26T12:25:45.667" Score="0" ViewCount="29" Body="&lt;p&gt;I have a microarray dataset with two mutants dataset that has already been normalised, and the fold change values for each gene in each mutant versus the wild type have been calculated. I'm interested in determining the statistical significance of these fold changes.&lt;/p&gt;&#xA;&lt;p&gt;What I've tried: I initially considered using limma, but I wasn't sure if it was appropriate for pre-calculated fold change values.&lt;/p&gt;&#xA;&lt;p&gt;What I was expecting: I was hoping to obtain a list of genes with their corresponding p-values and adjusted p-values, indicating their significance. Ideally, this would help me identify genes that are significantly upregulated or downregulated in the mutant compared to the wild type.&lt;/p&gt;&#xA;&lt;p&gt;Given that I only have the fold change values for two biological replicates (and no raw intensity or count data), what would be the best approach to achieve this? Any guidance, recommended tools, or R packages would be greatly appreciated!&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="18433" LastActivityDate="2023-10-26T12:25:45.667" Title="Calculating p-value and adjusted p-value for pre-normalized microarray data with fold change precalculated" Tags="|r|rna-seq|transcriptome|microarray|limma|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21753" PostTypeId="2" ParentId="2242" CreationDate="2023-10-26T12:38:25.933" Score="0" Body="&lt;p&gt;You could also use &lt;code&gt;agat_convert_bed2gff&lt;/code&gt;. &lt;code&gt;agat&lt;/code&gt; is generally a very useful tool when needing to convert from / to / between gff formats. See here how to use it to convert from bed to gff. &lt;a href=&quot;https://agat.readthedocs.io/en/latest/tools/agat_convert_bed2gff.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://agat.readthedocs.io/en/latest/tools/agat_convert_bed2gff.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17280" LastActivityDate="2023-10-26T12:38:25.933" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21754" PostTypeId="1" CreationDate="2023-10-26T13:52:00.407" Score="1" ViewCount="94" Body="&lt;p&gt;Sorry for the cross post but I'm not getting a response on bcbio github:&lt;/p&gt;&#xA;&lt;p&gt;I'm trying to run bcbio on 20 samples and it's only running one at a time as it steps through the yaml seemingly. I'm using slurm. I was wondering if someone can look at this bcbio snippet and if they see anything wrong?&lt;/p&gt;&#xA;&lt;p&gt;Slurm script&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;#SBATCH --job-name=bcbiopipeline_GPC3_PDx&#xA;#SBATCH -t 1-00:00&#xA;#SBATCH -c 1&#xA;#SBATCH -p core -n 32&#xA;#SBATCH --mem-per-cpu=40G&#xA;#SBATCH --error=job.%J.err&#xA;#SBATCH --output=job.%J.out&#xA;#SBATCH --export=ALL&#xA;&#xA;echo &amp;quot;Load bcbio...&amp;quot;&#xA;module use /projects/ngs_prodifx/local/software/modules/&#xA;module load bcbio-nextgen/1.2.9&#xA;&#xA;bcbio_nextgen.py seqc.yaml -n 72 -t ipython -s slurm -q core -r t=0-72:00 -r conmem=20 --timeout 3000 --tag &amp;quot;rna&amp;quot;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;yaml snippet:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;resources:&#xA;  default:&#xA;    memory: 30G&#xA;    cores: 72&#xA;details:&#xA;- algorithm:&#xA;    aligner: star&#xA;    disambiguate: mm10&#xA;    expression_caller:&#xA;    - salmon&#xA;    - kallisto&#xA;    quality_format: standard&#xA;    strandedness: auto&#xA;    transcriptome_fasta: /swiftcache/ngs/oncology/analysis/todd_creasy/working/reference/GRCh38.primary_assembly.genome_GPC3CART.fa&#xA;    transcriptome_gtf: /swiftcache/ngs/oncology/analysis/todd_creasy/working/reference/gencode.v29.annotation_GPC3CART.gtf&#xA;  analysis: RNA-seq&#xA;  description: 10_LI6612_Baseline_8743&#xA;  files:&#xA;  - /swiftcache/ngs/oncology/analysis/todd_creasy/working/fastq/SM-745/SM-745-10_R1_001.fastq.gz&#xA;  - /swiftcache/ngs/oncology/analysis/todd_creasy/working/fastq/SM-745/SM-745-10_R2_001.fastq.gz&#xA;  genome_build: hg38&#xA;  metadata:&#xA;    panel: Baseline&#xA;- algorithm:&#xA;    aligner: star&#xA;    disambiguate: mm10&#xA;    expression_caller:&#xA;    - salmon&#xA;    - kallisto&#xA;    quality_format: standard&#xA;    strandedness: auto&#xA;    transcriptome_fasta: /swiftcache/ngs/oncology/analysis/todd_creasy/working/reference/GRCh38.primary_assembly.genome_GPC3CART.fa&#xA;    transcriptome_gtf: /swiftcache/ngs/oncology/analysis/todd_creasy/working/reference/gencode.v29.annotation_GPC3CART.gtf&#xA;  analysis: RNA-seq&#xA;  description: 11_LI6612_Baseline_8765&#xA;  files:&#xA;  - /swiftcache/ngs/oncology/analysis/todd_creasy/working/fastq/SM-745/SM-745-11_R1_001.fastq.gz&#xA;  - /swiftcache/ngs/oncology/analysis/todd_creasy/working/fastq/SM-745/SM-745-11_R2_001.fastq.gz&#xA;  genome_build: hg38&#xA;  metadata:&#xA;    panel: Baseline&#xA;- algorithm:&#xA;    aligner: star&#xA;    disambiguate: mm10&#xA;    expression_caller:&#xA;    - salmon&#xA;    - kallisto&#xA;    quality_format: standard&#xA;    strandedness: auto&#xA;    transcriptome_fasta: /swiftcache/ngs/oncology/analysis/todd_creasy/working/reference/GRCh38.primary_assembly.genome_GPC3CART.fa&#xA;    transcriptome_gtf: /swiftcache/ngs/oncology/analysis/todd_creasy/working/reference/gencode.v29.annotation_GPC3CART.gtf&#xA;  analysis: RNA-seq&#xA;  description: 12_LI6612_Baseline_8770&#xA;  files:&#xA;  - /swiftcache/ngs/oncology/analysis/todd_creasy/working/fastq/SM-745/SM-745-12_R1_001.fastq.gz&#xA;  - /swiftcache/ngs/oncology/analysis/todd_creasy/working/fastq/SM-745/SM-745-12_R2_001.fastq.gz&#xA;  genome_build: hg38&#xA;  metadata:&#xA;    panel: Baseline&#xA;fc_name: seqc&#xA;upload:&#xA;  dir: ../output/final&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18434" LastActivityDate="2024-03-28T18:07:05.850" Title="bcbio run not running samples in parallel" Tags="|python|slurm|yaml|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21755" PostTypeId="1" CreationDate="2023-10-27T00:43:56.970" Score="1" ViewCount="30" Body="&lt;p&gt;A paper I'm reading from 2021&lt;sup&gt;[1]&lt;/sup&gt; mentions doing some analysis on data from the PhyLoTA database, with this reference citation:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;M. J. Sanderson et. al. 2008 &lt;a href=&quot;https://doi.org/10.1080/10635150802158688&quot; rel=&quot;nofollow noreferrer&quot;&gt;The PhyLoTa Browser: processing GenBank for molecular phylogenetics research&lt;/a&gt;, &lt;em&gt;Syst. Biol.&lt;/em&gt; 57(3) 335-346.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;This paper abstract claims to have an online interface at &lt;a href=&quot;http://loco.biosci.arizona.edu/pb&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://loco.biosci.arizona.edu/pb&lt;/a&gt;, but this link is not currently responsive. Does anyone know what happened to this project, and if it is currently accessible at some other location?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;[1] S. M. Harrington et. al. 2021 &lt;a href=&quot;https://doi.org/10.1093/molbev/msaa295&quot; rel=&quot;nofollow noreferrer&quot;&gt;Properties of Markov chain Monte Carlo performance across many empirial alignments&lt;/a&gt; MBE&lt;/p&gt;&#xA;" OwnerUserId="17320" LastEditorUserId="3848" LastEditDate="2023-10-28T14:41:58.733" LastActivityDate="2023-10-28T20:14:42.587" Title="What happened to the PhyLoTA browser?" Tags="|phylogenetics|phylogeny|public-databases|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21757" PostTypeId="2" ParentId="21754" CreationDate="2023-10-27T16:05:14.423" Score="0" Body="&lt;p&gt;I understand (see &lt;strong&gt;Slurm array job solution&lt;/strong&gt; below), what I can definitely help with is the automated generation of the &lt;code&gt;seqc.yaml&lt;/code&gt; file. This avoids the repetition (and risk of error) in copy and pasting. The following one-liner exactly reproduces the &lt;code&gt;seqc.yaml&lt;/code&gt; file and needs the &lt;code&gt;chunk.yaml&lt;/code&gt; below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for ((x=10;x&amp;lt;=13;x++)); do while IFS='' read -r a; do echo &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${a//SM-745-10/SM-745-$&lt;/span&gt;{x}}&amp;quot;; done &amp;lt; chunk.yaml &amp;gt;&amp;gt; seqc.yaml; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This could be generalised to 3 command lines in bash,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;y='SM-745-10'&#xA;z='M-745'&#xA;for ((x=10;x&amp;lt;=13;x++)); \&#xA;do while IFS='' read -r a; do echo &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${a//${y}/${z}$&lt;/span&gt;{x}}&amp;quot;;\&#xA;done &amp;lt; chunk.yaml &amp;gt;&amp;gt; seqc.yaml; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here everything from &lt;code&gt;for&lt;/code&gt; is a bash one-liner. Obviously &lt;code&gt;((x=10,x&amp;lt;=13;x++))&lt;/code&gt; would be &lt;code&gt;((x=1,x&amp;lt;=20;x++))&lt;/code&gt; for 20 fastq forward and reverse files. This will ensure the separate files are correctly named in the yaml file, in this example, viz.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# first pair&#xA;SM-745-10_R1_001.fastq.gz&#xA;SM-745-10_R2_001.fastq.gz&#xA;# second pair&#xA;SM-745-11_R1_001.fastq.gz&#xA;SM-745-11_R2_001.fastq.gz&#xA;# third pair&#xA;SM-745-12_R1_001.fastq.gz&#xA;SM-745-12_R2_001.fastq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The &lt;code&gt;chunk.yaml&lt;/code&gt; is,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;details:&#xA;- algorithm:&#xA;    aligner: star&#xA;    disambiguate: mm10&#xA;    expression_caller:&#xA;    - salmon&#xA;    - kallisto&#xA;    quality_format: standard&#xA;    strandedness: auto&#xA;    transcriptome_fasta: /swiftcache/ngs/oncology/analysis/todd_creasy/working/reference/GRCh38.primary_assembly.genome_GPC3CART.fa&#xA;    transcriptome_gtf: /swiftcache/ngs/oncology/analysis/todd_creasy/working/reference/gencode.v29.annotation_GPC3CART.gtf&#xA;  analysis: RNA-seq&#xA;  description: 10_LI6612_Baseline_8743&#xA;  files:&#xA;  - /swiftcache/ngs/oncology/analysis/todd_creasy/working/fastq/SM-745/SM-745-10_R1_001.fastq.gz&#xA;  - /swiftcache/ngs/oncology/analysis/todd_creasy/working/fastq/SM-745/SM-745-10_R2_001.fastq.gz&#xA;  genome_build: hg38&#xA;  metadata:&#xA;    panel: Baseline&#xA;-&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;bcbio&lt;/h3&gt;&#xA;&lt;p&gt;To summarise the &lt;a href=&quot;https://github.com/bcbio/bcbio-nextgen/issues/3719&quot; rel=&quot;nofollow noreferrer&quot;&gt;response from the Git support team&lt;/a&gt; is there's conflict occurring  ...&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The issue is here for the slurm script ...&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;#SBATCH -p core -n 32&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I assume this is &lt;code&gt;-N 1&lt;/code&gt; ,i.e. &amp;gt;= 32 core node, if it was 16 cores per node then &lt;code&gt;-N 2&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, &lt;strong&gt;bcbio&lt;/strong&gt; is taking care of this and the restriction &lt;code&gt;32&lt;/code&gt; is conflicting with &lt;code&gt;72&lt;/code&gt; from &lt;code&gt;bcbio&lt;/code&gt;, i.e. it creates its own Slurm controller script.&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;in &lt;strong&gt;bcbio&lt;/strong&gt; yaml&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;    default:&#xA;    memory: 30G&#xA;    cores: 72&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;2a. Firstly &lt;code&gt;72&lt;/code&gt; is specified in Slurm's CLI here ... &lt;code&gt;-n 72&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;2b. This isn't needed because,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#SBATCH --mem-per-cpu=40G &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;--mem-per-cpu&lt;/code&gt;  This is a very recent feature BTW.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;h3&gt;Slurm array job solution&lt;/h3&gt;&#xA;&lt;p&gt;What I considered was using the &lt;code&gt;chunk.yaml&lt;/code&gt; above with an array job - this solution isn't needed because &lt;strong&gt;bcbio&lt;/strong&gt; is dynamically reading and parallelising &lt;code&gt;seqc.yaml&lt;/code&gt; (maybe deploying a wrapper for &lt;code&gt;sinfo&lt;/code&gt;, &lt;code&gt;squeue&lt;/code&gt;). In the solution below what I'm doing is placing each fastq pair directly into the Slurm queue. This would then parallelise everything directly. Again whilst it would work, it's unnecessary.&lt;/p&gt;&#xA;&lt;p&gt;Thus for all 20 fastq files assuming &lt;code&gt;SM-745-1_R1_001.fastq.gz&lt;/code&gt; ... &lt;code&gt;SM-745-20_R1_001.fastq.gz&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;#SBATCH --job-name=bcbiopipeline_GPC3_PDx&#xA;#SBATCH -t 1-00:00:00 # 1- is one day. 00:00:00 is a wall clock. Does &amp;quot;1-00:00&amp;quot; work?&#xA;#SBATCH -c 1&#xA;#SBATCH --array=1-20 # this is now in $SLURM_ARRAY_TASK_ID&#xA;#SBATCH --mem-per-cpu=40G&#xA;#SBATCH --error=job.%J.err&#xA;#SBATCH --output=job.%J.out&#xA;#SBATCH --export=ALL&#xA;&#xA;echo &amp;quot;Load bcbio...&amp;quot;&#xA;module use /projects/ngs_prodifx/local/software/modules/  &#xA;module load bcbio-nextgen/1.2.9 &#xA;&#xA;# SM-745-10 and SM-745 could be assigned as variables y and z as above&#xA;&#xA;while IFS='' read -r a; \&#xA;do echo &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${a//SM-745-10/SM-745-$&lt;/span&gt;SLURM_ARRAY_TASK_ID&amp;quot;; \&#xA;done &amp;lt; chunk.yaml &amp;gt; seqc.yaml; done&#xA;bcbio_nextgen.py seqc.yaml -n 72 -t ipython -s slurm -q core -r t=0-72:00 -r conmem=20 --timeout 3000 --tag &amp;quot;rna&amp;quot;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; the &lt;code&gt;-n 72&lt;/code&gt; parameter, i.e. number of cores deployed, will likely be reduced under this solution.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Alternative&lt;/strong&gt; The alternative is to issue a slight modification of the bash one-liner described above to produce 3 (or 20) separate yaml files via bash &lt;code&gt;while&lt;/code&gt; referring the different fastq paired files and use array job to cycle through them and delete each in turn when done.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-10-30T17:43:36.270" LastActivityDate="2023-10-30T17:43:36.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21758" PostTypeId="1" AcceptedAnswerId="21761" CreationDate="2023-10-27T16:27:34.183" Score="2" ViewCount="284" Body="&lt;p&gt;I'm using Scipy's pdist function to calculate a distance matrix which is working great but it gives me a symmetric matrix and I want to convert it into a lower-left triangular matrix.&lt;/p&gt;&#xA;&lt;p&gt;This is what it prints out for me:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  a   b   c   d&#xA;a 0 ### ### ###&#xA;b ### 0 ### ###&#xA;c ### ### 0 ###&#xA;d ### ### ### 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is what I want it to be:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;###&#xA;### ###&#xA;### ### ###&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I know there's a function in R to do this (upper.tri) but it just replaces everything to the right of the 0 with more 0s. Is there a better/easier way to get it like this? I'm opening the file in excel and removing everything to the right with the 0 manually right now.&lt;/p&gt;&#xA;" OwnerUserId="16067" LastActivityDate="2023-10-27T20:40:49.520" Title="Convert Symmetrix Matrix into Lower Triangular Matrix" Tags="|format-conversion|scipy|distance-matrix|megax|pdist|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21759" PostTypeId="2" ParentId="21758" CreationDate="2023-10-27T16:36:36.283" Score="1" Body="&lt;p&gt;There seems to be a scipy method that retains the lower triangle but with the 0s. See: &lt;a href=&quot;https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.tril.html#scipy.sparse.tril&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.scipy.org/doc/scipy/reference/generated/scipy.sparse.tril.html#scipy.sparse.tril&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-10-27T16:36:36.283" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21761" PostTypeId="2" ParentId="21758" CreationDate="2023-10-27T20:40:49.520" Score="4" Body="&lt;p&gt;NumPy has a convenient built-in function for accessing the lower-triangular or upper-trianglar entries of an array.&#xA;To get the lower-triangular entries, you can use &lt;a href=&quot;https://numpy.org/doc/stable/reference/generated/numpy.tril_indices.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;numpy.tril_indices&lt;/code&gt;&lt;/a&gt;&#xA;to get the indices of these entries, and then use these indices to extract the entries.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;&#xA;ar = np.array(&#xA;    [[0, 1, 2],&#xA;    [3, 4, 5],&#xA;    [6, 7, 8]]&#xA;)&#xA;idx = np.tril_indices(3, -1)&#xA;ar[idx]&#xA;# outputs array([3, 6, 7])&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This doesn't automatically produce the spacing you want, but formatting the final display is likely context-dependent.&lt;/p&gt;&#xA;" OwnerUserId="17320" LastActivityDate="2023-10-27T20:40:49.520" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21762" PostTypeId="1" AcceptedAnswerId="21763" CreationDate="2023-10-27T20:43:13.840" Score="2" ViewCount="35" Body="&lt;p&gt;I am trying to store a GenBank file that could be read by Geneious later.&#xA;I found an example how to create the GenBank file &lt;a href=&quot;https://stackoverflow.com/questions/65216225/writing-and-saving-genbank-files-with-biobython-seqio-module&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;But now I run into &lt;code&gt;Alphabet&lt;/code&gt; issues.&#xA;I tried all protein classes from the &lt;a href=&quot;https://biopython.org/docs/1.75/api/Bio.Alphabet.IUPAC.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;documentation&lt;/a&gt;, but all fail with the same error:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;TypeError: Invalid alphabet found, &amp;lt;class 'Bio.Alphabet.ProteinAlphabet'&amp;gt;.&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I dont know what else to do here? Using &lt;code&gt;biopython==1.73&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    record = SeqRecord.SeqRecord(&#xA;        Seq.Seq(&amp;quot;MIRQALAVAALLLAGTAQADGLIDN&amp;quot;, alphabet=Alphabet.ProteinAlphabet),&#xA;        id=&amp;quot;FooBar&amp;quot;,&#xA;    )&#xA;&#xA;    with NamedTemporaryFile(mode=&amp;quot;w&amp;quot;) as genbank:&#xA;        SeqIO.write(record, genbank, &amp;quot;genbank&amp;quot;)&#xA;        pass&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17804" LastActivityDate="2023-10-27T20:59:10.280" Title="BioPython SeqIO fails with Invalid Alphabet found" Tags="|python|genbank|biopython|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21763" PostTypeId="2" ParentId="21762" CreationDate="2023-10-27T20:53:26.817" Score="1" Body="&lt;p&gt;Quickly ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.Alphabet import generic_protein&#xA;from Bio.Seq import Seq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;then ..&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;myprot = Seq('EXAMPLEPROT', alphabet=generic_protein)&#xA;    &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-10-27T20:59:10.280" LastActivityDate="2023-10-27T20:59:10.280" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21764" PostTypeId="2" ParentId="8583" CreationDate="2023-10-27T21:54:34.977" Score="0" Body="&lt;p&gt;my attempt, will get only non water (H2O) or non aa (aminoacid) present in the PDB .... tricky if the ligand is an aa :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.PDB import PDBParser&#xA;&#xA;from Bio.PDB import Selection&#xA;&#xA;d = {'CYS': 'C', 'ASP': 'D', 'SER': 'S', 'GLN': 'Q', 'LYS': 'K',&#xA;     'ILE': 'I', 'PRO': 'P', 'THR': 'T', 'PHE': 'F', 'ASN': 'N', &#xA;     'GLY': 'G', 'HIS': 'H', 'LEU': 'L', 'ARG': 'R', 'TRP': 'W', &#xA;     'ALA': 'A', 'VAL':'V', 'GLU': 'E', 'TYR': 'Y', 'MET': 'M'}&#xA;&#xA;pdb_file = '5to8'&#xA;&#xA;struct = PDBParser(QUIET = True).get_structure(pdb_file, pdb_file+'.pdb')&#xA;&#xA;res_list = Selection.unfold_entities(struct , 'R')&#xA;&#xA;for i in res_list :&#xA;    &#xA;    if i.resname != 'HOH' and i.resname not in d :&#xA;    &#xA;        print(i, i.resname)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;lt;Residue 7FM het=H_7FM resseq=701 icode= &amp;gt; 7FM&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="10049" LastActivityDate="2023-10-27T21:54:34.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21765" PostTypeId="2" ParentId="8583" CreationDate="2023-10-27T22:22:43.167" Score="1" Body="&lt;p&gt;copying from &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/20301/how-to-download-ligands-for-pdb-structure/20333#20333&quot;&gt;How to download ligands for PDB structure&lt;/a&gt; one of the answers, one of mines, code  :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;import requests&#xA;&#xA;import json&#xA;&#xA;&#xA;&#xA;parsed_lig_dict = {}&#xA;&#xA;def main(entry_id: str):    &#xA;    risposta = requests.get(f&amp;quot;https://data.rcsb.org/rest/v1/core/entry/{entry_id}/&amp;quot;)   &#xA;    parsed = json.loads(risposta.content.decode())    &#xA;    # print(risposta.status_code)    &#xA;    # print((parsed))    &#xA;    # print(parsed[&amp;quot;rcsb_entry_container_identifiers&amp;quot;][&amp;quot;non_polymer_entity_ids&amp;quot;])&#xA;    # print('\n', type(parsed))&#xA;    &#xA;    parsed_lig = parsed[&amp;quot;rcsb_entry_container_identifiers&amp;quot;][&amp;quot;non_polymer_entity_ids&amp;quot;]    &#xA;    # print('\n\n',parsed_lig, type(parsed_lig))&#xA;    &#xA;    # for i in parsed:        &#xA;    #     print('\n', i  , parsed[i])&#xA;                  &#xA;    for lig in parsed_lig:    &#xA;        risposta = requests.get(f&amp;quot;https://data.rcsb.org/rest/v1/core/nonpolymer_entity/{entry_id}/{lig}&amp;quot;)    &#xA;        parsed = json.loads(risposta.content.decode())    &#xA;        # print('\n',risposta.status_code, risposta.content)        &#xA;        &#xA;        # print(parsed[&amp;quot;rcsb_nonpolymer_entity_container_identifiers&amp;quot;][&amp;quot;asym_ids&amp;quot;], &#xA;              # parsed[&amp;quot;pdbx_entity_nonpoly&amp;quot;][&amp;quot;comp_id&amp;quot;])&#xA;                &#xA;        parsed_lig_dict[lig] = [parsed[&amp;quot;pdbx_entity_nonpoly&amp;quot;][&amp;quot;comp_id&amp;quot;] , &#xA;                                dict.fromkeys(parsed[&amp;quot;rcsb_nonpolymer_entity_container_identifiers&amp;quot;][&amp;quot;asym_ids&amp;quot;])]&#xA;    &#xA;    for lig in parsed_lig_dict:        &#xA;        for chain in parsed_lig_dict[lig][1]:    &#xA;            risposta = requests.get(f&amp;quot;https://data.rcsb.org/rest/v1/core/nonpolymer_entity_instance/{entry_id}/{chain}&amp;quot;)        &#xA;            # print(risposta.status_code)&#xA;        &#xA;            parsed = json.loads(risposta.content.decode())        &#xA;            # print('\n',risposta.status_code, risposta.content)            &#xA;            # print(chain , '--&amp;gt; ',parsed[&amp;quot;rcsb_nonpolymer_entity_instance_container_identifiers&amp;quot;][&amp;quot;auth_seq_id&amp;quot;])&#xA;            &#xA;            parsed_lig_dict[lig][1][chain] = parsed[&amp;quot;rcsb_nonpolymer_entity_instance_container_identifiers&amp;quot;][&amp;quot;auth_seq_id&amp;quot;]        &#xA;               &#xA;    &#xA;    # print(parsed_lig_dict)&#xA;    &#xA;    # for i in parsed_lig_dict:&#xA;    #     print(parsed_lig_dict[i])&#xA;    &#xA;    # for i in parsed_lig_dict:&#xA;    #     for y in parsed_lig_dict[i][1]:&#xA;    #         print(i,parsed_lig_dict[i][0], y , parsed_lig_dict[i][1][y])&#xA;                        &#xA;    cnt = 1     &#xA;    for i in parsed_lig_dict:        &#xA;        for y in parsed_lig_dict[i][1]:&#xA;            seq_id = parsed_lig_dict[i][1][y]            &#xA;            chain = y            &#xA;            comp_id = parsed_lig_dict[i][0]            &#xA;            print(seq_id , y)&#xA;&#xA;            risposta = requests.get(f&amp;quot;https://models.rcsb.org/v1/{pdb_file}/ligand?auth_seq_id={seq_id}&amp;amp;label_asym_id={chain}&amp;amp;encoding=sdf&amp;quot; ,                              &#xA;                              allow_redirects=True)             &#xA;            print(risposta.status_code)&#xA;                        &#xA;            filename = f&amp;quot;lig_{comp_id}_{chain}_{seq_id}_{cnt}.sdf&amp;quot;            &#xA;            open(filename, 'wb').write(risposta.content)            &#xA;            cnt += 1&#xA;&#xA;    &#xA;if __name__ == '__main__':  &#xA;&#xA;   pdb_file = '5to8'   &#xA;       &#xA;   main(pdb_file)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output file : &lt;code&gt;lig_7FM_B_701_1.sdf&lt;/code&gt; :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;7FM&#xA;  ModelServer 0.9.10&#xA;&#xA; 39 43  0  0  1  0  0  0  0  0  0&#xA;   -2.0610  -10.3590   40.5700 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -3.1170  -11.7950   42.6720 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -3.8640  -12.6100   43.6800 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -0.6970  -10.3290   36.5750 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -1.8210  -10.9080   35.9890 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -1.6600   -8.1940   35.9530 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    2.0300   -9.0340   41.4650 N   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    0.9390   -9.6680   41.0120 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    0.7980  -10.0800   39.7250 N   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    1.7720   -9.8430   38.8300 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    2.9200   -9.1930   39.2330 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    2.9910   -8.8020   40.5580 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    4.0340   -8.8690   38.3150 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -0.0830   -9.8810   41.9510 N   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -1.3380  -10.4670   41.7760 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -3.3020  -10.9670   40.4240 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -3.8290  -11.6870   41.4760 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -1.8800  -11.1950   42.8370 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -5.0400  -12.4130   41.5840 N   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -5.2460  -12.6990   43.0180 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    1.5160  -10.2930   37.5490 N   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    0.4060  -11.1430   37.2090 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -2.8420  -10.2200   35.4050 N   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -2.7360   -8.8690   35.4000 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -0.6240   -8.9330   36.5380 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -1.9670  -12.3410   35.9950 N   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -2.5680  -13.0870   37.1550 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -1.4690  -13.1930   34.5670 S   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -4.0440  -12.7310   37.3480 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -5.9670  -12.8650   40.6430 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -6.9720  -13.5050   40.9690 O   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -4.6200  -13.4130   38.5900 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -5.8010  -12.6200   39.1510 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -0.5520  -12.3690   33.7940 O   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -1.1430  -14.5640   34.8900 O   0  0  0  0  0  0  0  0  0  0  0  0&#xA;   -3.0530  -13.2140   33.7590 C   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    5.0490   -8.1440   38.8710 F   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    4.6120   -9.9430   37.7390 F   0  0  0  0  0  0  0  0  0  0  0  0&#xA;    3.5350   -8.1080   37.3100 F   0  0  0  0  0  0  0  0  0  0  0  0&#xA;  1 16  2  0  0  0  0&#xA;  1 15  1  0  0  0  0&#xA;  2 17  2  0  0  0  0&#xA;  2 18  1  0  0  0  0&#xA;  2  3  1  0  0  0  0&#xA;  3 20  1  0  0  0  0&#xA;  4  5  2  0  0  0  0&#xA;  4 25  1  0  0  0  0&#xA;  4 22  1  0  0  0  0&#xA;  5 23  1  0  0  0  0&#xA;  5 26  1  0  0  0  0&#xA;  6 24  1  0  0  0  0&#xA;  6 25  2  0  0  0  0&#xA;  7 12  2  0  0  0  0&#xA;  7  8  1  0  0  0  0&#xA;  8  9  2  0  0  0  0&#xA;  8 14  1  0  0  0  0&#xA;  9 10  1  0  0  0  0&#xA; 10 21  1  0  0  0  0&#xA; 10 11  2  0  0  0  0&#xA; 11 13  1  0  0  0  0&#xA; 11 12  1  0  0  0  0&#xA; 13 39  1  0  0  0  0&#xA; 13 38  1  0  0  0  0&#xA; 13 37  1  0  0  0  0&#xA; 14 15  1  0  0  0  0&#xA; 15 18  2  0  0  0  0&#xA; 16 17  1  0  0  0  0&#xA; 17 19  1  0  0  0  0&#xA; 19 30  1  0  0  0  0&#xA; 19 20  1  0  0  0  0&#xA; 21 22  1  0  0  0  0&#xA; 23 24  2  0  0  0  0&#xA; 26 28  1  0  0  0  0&#xA; 26 27  1  0  0  0  0&#xA; 27 29  1  0  0  0  0&#xA; 28 36  1  0  0  0  0&#xA; 28 34  2  0  0  0  0&#xA; 28 35  2  0  0  0  0&#xA; 29 32  1  0  0  0  0&#xA; 30 33  1  0  0  0  0&#xA; 30 31  2  0  0  0  0&#xA; 32 33  1  0  0  0  0&#xA;M  END&#xA;&amp;gt; &amp;lt;model_server_result.job_id&amp;gt;&#xA;RFDalM1AmREvgkv0PqdfQw&#xA;&#xA;&amp;gt; &amp;lt;model_server_result.datetime_utc&amp;gt;&#xA;2023-10-27 22:13:14&#xA;&#xA;&amp;gt; &amp;lt;model_server_result.server_version&amp;gt;&#xA;0.9.10&#xA;&#xA;&amp;gt; &amp;lt;model_server_result.query_name&amp;gt;&#xA;ligand&#xA;&#xA;&amp;gt; &amp;lt;model_server_result.source_id&amp;gt;&#xA;pdb-bcif&#xA;&#xA;&amp;gt; &amp;lt;model_server_result.entry_id&amp;gt;&#xA;5to8&#xA;&#xA;&amp;gt; &amp;lt;model_server_params.name&amp;gt;&#xA;atom_site&#xA;&#xA;&amp;gt; &amp;lt;model_server_params.value&amp;gt;&#xA;{&amp;quot;label_asym_id&amp;quot;:&amp;quot;B&amp;quot;,&amp;quot;auth_seq_id&amp;quot;:701}&#xA;&#xA;&amp;gt; &amp;lt;model_server_stats.io_time_ms&amp;gt;&#xA;5&#xA;&#xA;&amp;gt; &amp;lt;model_server_stats.parse_time_ms&amp;gt;&#xA;76&#xA;&#xA;&amp;gt; &amp;lt;model_server_stats.create_model_time_ms&amp;gt;&#xA;2&#xA;&#xA;&amp;gt; &amp;lt;model_server_stats.query_time_ms&amp;gt;&#xA;244&#xA;&#xA;&amp;gt; &amp;lt;model_server_stats.encode_time_ms&amp;gt;&#xA;0&#xA;&#xA;&amp;gt; &amp;lt;model_server_stats.element_count&amp;gt;&#xA;39&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;$$$$&lt;/span&gt;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;picture of .sdf file :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/xDGpr.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/xDGpr.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10049" LastActivityDate="2023-10-27T22:22:43.167" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21766" PostTypeId="1" AcceptedAnswerId="21772" CreationDate="2023-10-28T10:06:52.970" Score="1" ViewCount="120" Body="&lt;p&gt;So I wanted to download massive amounts of COX1 (or COI) sequences from any available database. I found this script in &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/13187/16657&quot;&gt;https://bioinformatics.stackexchange.com/a/13187/16657&lt;/a&gt; this comment that achieves just that using the &lt;code&gt;bold&lt;/code&gt; R package. I changed the script to fit my data preference:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# load packages&#xA;library(tidyverse)&#xA;library(rentrez)&#xA;library(bold)    # API interface to BOLD&#xA;library(taxize)  # for NCBI taxonomy lookup&#xA;library(seqinr)  # for FASTA output&#xA;&#xA;set_entrez_key()&#xA;&#xA;# get class-level taxa within &amp;quot;Mollusca&amp;quot; from NCBI taxonomy&#xA;taxa &amp;lt;- downstream(&amp;quot;Mollusca&amp;quot;, db = &amp;quot;ncbi&amp;quot;, downto = &amp;quot;order&amp;quot;)&#xA;# check if taxa present in BOLD&#xA;checks &amp;lt;- bold_tax_name(taxa&lt;span class=&quot;math-container&quot;&gt;$Mollusca$&lt;/span&gt;childtaxa_name)&#xA;taxa_bold &amp;lt;- checks[!is.na(checks&lt;span class=&quot;math-container&quot;&gt;$taxon),]$&lt;/span&gt;taxon&#xA;&#xA;# Download sequences from BOLD for each class-level taxon&#xA;sequences &amp;lt;- map(taxa_bold, bold_seq, marker = 'COI-5P') %&amp;gt;%&#xA;  flatten() %&amp;gt;%&#xA;  bind_rows()&#xA;&#xA;# Convert the list of sequences to a data frame&#xA;sequences_df &amp;lt;- do.call(rbind, sequences)&#xA;&#xA;&#xA;# Write sequences to a file&#xA;write.fasta(&#xA;  sequences = as.list(sequences_df&lt;span class=&quot;math-container&quot;&gt;$sequence), &#xA;  names = as.list(sequences_df$&lt;/span&gt;id), &#xA;  nbchar = 80, &#xA;  file.out = 'coi5p.fasta'&#xA;)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In the aforementioned thread, this script worked but now that I ran it I got the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error in `bind_rows()`:&#xA;! Argument 1 must be a data frame or a named atomic vector.&#xA;Run `rlang::last_trace()` to see where the error occurred.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am not sure what to do for this. I commented on the original thread but since it is kind of old I decided to post about it here.&lt;/p&gt;&#xA;&lt;p&gt;Note: the &lt;code&gt;set_entrez_key()&lt;/code&gt; requires a NCBI API key which you can get by registering on NCBI.&lt;/p&gt;&#xA;&lt;p&gt;Edit #1: The &lt;code&gt;sequences&lt;/code&gt; object before the &lt;code&gt;flatten()&lt;/code&gt; command is a large list containing 53 elements. After the command, the object becomes a large list with 257 elements.&lt;/p&gt;&#xA;" OwnerUserId="16657" LastEditorUserId="16657" LastEditDate="2023-10-29T07:54:50.660" LastActivityDate="2023-10-29T15:49:12.197" Title="How can I resolve `bind_rows()` error?" Tags="|database|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21767" PostTypeId="2" ParentId="21749" CreationDate="2023-10-28T11:25:33.853" Score="1" Body="&lt;p&gt;Megablast is designed to find very similar sequences. It is highly specific but has a very low sensitivity. This is because it has a very large &lt;a href=&quot;https://www.metagenomics.wiki/tools/blast/default-word-size&quot; rel=&quot;nofollow noreferrer&quot;&gt;word size&lt;/a&gt;. By default, I think it will use a &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK279684/table/appendices.T.blastn_application_options/&quot; rel=&quot;nofollow noreferrer&quot;&gt;word size of 28 nt&lt;/a&gt;. That's almost your entire sequence, so if your target sequence doesn't have 28 contiguous nucleotides that are &lt;em&gt;identical&lt;/em&gt; to 28 contiguous nucleotides of your query, no hit will be found.&lt;/p&gt;&#xA;&lt;p&gt;If you are looking for distant homologous sequences, you should use simple &lt;code&gt;blastn&lt;/code&gt; which defaults to a word size of 11. That will still likely be too small, you might want to change it to something lower (your blast tool should have an option for that), but it will give you more results than megablast.&lt;/p&gt;&#xA;&lt;p&gt;Finally, if these 32nt are translated into protein, you can consider using tblastn instead.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-10-28T11:25:33.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21768" PostTypeId="2" ParentId="21755" CreationDate="2023-10-28T14:30:27.513" Score="1" Body="&lt;p&gt;Phylota was implemented in R and is present on &lt;a href=&quot;https://github.com/ropensci/phylotaR&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub here&lt;/a&gt;, described by &lt;a href=&quot;https://doi.org/10.3390/life8020020&quot; rel=&quot;nofollow noreferrer&quot;&gt;Bennett &lt;em&gt;et al&lt;/em&gt; (2018)&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Here's the code...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(phylotaR)&#xA;wd &amp;lt;- '[FILEPATH TO WORKING DIRECTORY]'&#xA;ncbi_dr &amp;lt;- '[FILEPATH TO COMPILED BLAST+ TOOLS]'&#xA;txid &amp;lt;- 9443  # primates ID&#xA;setup(wd = wd, txid = txid, ncbi_dr = ncbi_dr)&#xA;run(wd = wd)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Sanderson &lt;em&gt;et al&lt;/em&gt;  was 15 years ago, albeit Sys. Biol. is a very reputable journal in phylogenetics. I've deployed solutions published there with outstanding results.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-10-28T20:14:42.587" LastActivityDate="2023-10-28T20:14:42.587" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21770" PostTypeId="1" CreationDate="2023-10-28T21:02:20.173" Score="1" ViewCount="25" Body="&lt;p&gt;There is an R code example on &lt;a href=&quot;https://rest.ensembl.org/documentation/info/vep_region_post&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ensembl's website&lt;/a&gt; on how to execute a POST &amp;quot;vep/:species/region&amp;quot; request for variants that only have one alternate allele. The part that holds the variant data in the POST request is structured as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;..., body = '{ &amp;quot;variants&amp;quot; : [&amp;quot;21  26960070  rs116645811 G A . . .&amp;quot;, &amp;quot;21  26965148  rs1135638 G A . . .&amp;quot; ] }'...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Where, for the first variant, 21 is the chromosome number, 26960070 is the start point coordinate on the chromosome, rs116645811 is the variant identifier, &amp;quot;G&amp;quot; is the reference allele and &amp;quot;A&amp;quot; is the alternate allele.&lt;/p&gt;&#xA;&lt;p&gt;Alternatively, you can substitute the variant identifiers - rs116645811 and rs1135638, in the example above - for the endpoint coordinate of the variant. I use this option myself and can personally confirm that it works.&lt;/p&gt;&#xA;&lt;p&gt;As mentioned before, the example above shows a POST request of variants that have only single alternate alleles.&#xA;&lt;em&gt;Does anybody, on the other hand, know how (one should adjust the structure in the example above) to execute a POST request that accommodates multiple alternate alleles for a single variant?&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you explain your answer with an example like the one above, please use an example with the endpoint coordinate (and not the variant ID).&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="18448" LastEditorUserId="18448" LastEditDate="2023-10-28T21:21:49.087" LastActivityDate="2023-10-28T21:21:49.087" Title="How to make a POST &quot;vep/:species/region&quot; request that is able to accomodate multiple alternate alleles for a single variant?" Tags="|r|ensembl|variants|vep|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21771" PostTypeId="1" CreationDate="2023-10-29T12:48:29.820" Score="0" ViewCount="21" Body="&lt;p&gt;I have a database of whole genome of African swine fever virus (about 190kb/genome) downloaded from GenBank. But these sequences do not start at the same place on the genome. When I align the sequence to build a phylogenetic tree. The sequences at the 3' and 5' ends have many gaps and do not match. The length of the entire data set is over 220kb due to many gaps.&#xA;Kindly suggest me a solution to the problem.&lt;/p&gt;&#xA;" OwnerUserId="18452" LastActivityDate="2023-10-29T12:48:29.820" Title="Problem with edit and align viral whole genomes" Tags="|sequence-alignment|whole-genome|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21772" PostTypeId="2" ParentId="21766" CreationDate="2023-10-29T15:49:12.197" Score="1" Body="&lt;p&gt;I managed to find the solution. After @gringer mentioned that they had not seen the &lt;code&gt;flatten&lt;/code&gt; command I decided to dig deeper. Turns out the command has been superseded  in purrr 1.0.0. So I will be posting the new script for anyone who seeks an updated version of the one in the old thread.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# load packages&#xA;library(tidyverse)&#xA;library(rentrez)&#xA;library(bold)    # API interface to BOLD&#xA;library(taxize)  # for NCBI taxonomy lookup&#xA;library(seqinr)  # for FASTA output&#xA;library(rBLAST)  &#xA;&#xA;&#xA;set_entrez_key(&amp;quot;d00fee673c4499df4c2479ba5fd71850b308&amp;quot;) #my ncbi api key, this is essential for pulling the taxonomy through ncbi&#xA;&#xA;# get class-level taxa within &amp;quot;Mollusca&amp;quot; from NCBI taxonomy&#xA;taxa &amp;lt;- downstream(&amp;quot;Mollusca&amp;quot;, db = &amp;quot;ncbi&amp;quot;, downto = &amp;quot;class&amp;quot;) #setting the taxonomic level of the search. it will pull sequences for all mollusca but seperately for each class. if the search is large change to a lower taxonomic level&#xA;&#xA;# check if taxa present in BOLD&#xA;checks &amp;lt;- bold_tax_name(taxa&lt;span class=&quot;math-container&quot;&gt;$Mollusca$&lt;/span&gt;childtaxa_name) &#xA;taxa_bold &amp;lt;- checks[!is.na(checks&lt;span class=&quot;math-container&quot;&gt;$taxon),]$&lt;/span&gt;taxon&#xA;&#xA;# Download sequences from BOLD for each class-level taxon&#xA;sequences &amp;lt;- map(taxa_bold, bold_seq, marker = 'COI-5P') &#xA;# Convert the list of sequences to a data frame&#xA;sequences_df &amp;lt;- do.call(rbind, sequences)&#xA;&#xA;&#xA;&#xA;&#xA;# Filter sequences based on the &amp;quot;COI-5P&amp;quot; marker. Despite the markes parameter, some files may have integrated multiple loci. The other sequences are removed from the R object&#xA;coi5p_sequences &amp;lt;- sequences_df %&amp;gt;% &#xA;  filter(marker == &amp;quot;COI-5P&amp;quot;)&#xA;&#xA;# Write sequences to a file&#xA;write.fasta(&#xA;  sequences = as.list(coi5p_sequences&lt;span class=&quot;math-container&quot;&gt;$sequence), &#xA;  names = as.list(coi5p_sequences$&lt;/span&gt;id), &#xA;  nbchar = 60, &#xA;  file.out = 'coi5p_Mollusca.fasta'&#xA;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16657" LastActivityDate="2023-10-29T15:49:12.197" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21773" PostTypeId="2" ParentId="21746" CreationDate="2023-10-29T16:11:27.123" Score="1" Body="&lt;p&gt;Consider filtering on every parameter: Gene Significance (GS), p-value for Gene Significance (p.GS), Module Membership (MM), and p-value for Module Membership (p.MM).&lt;/p&gt;&#xA;&lt;p&gt;This doesn't mean you need to necessarily filter on every parameter, but it depends on what exactly you want to do, and you might reasonably choose any/all.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;For GS, you may want to ensure you're capturing genes strongly associated with your experimental conditions.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For p.GS, you may want to ensure statistical relevance, (so filter out genes with high p-values). Similar for p.MM.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For MM, you may want it to be high to make sure the genes you select are strongly co-expressed and are relevant members of the module.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Plot MM vs GS to see if you can see an obvious filter threshold.&lt;/p&gt;&#xA;" OwnerUserId="17430" LastActivityDate="2023-10-29T16:11:27.123" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21774" PostTypeId="2" ParentId="21635" CreationDate="2023-10-29T16:30:36.953" Score="1" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Thoughts on the described method from Bristol?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;I don't immediately see the benefit to this over rsIDs.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;What might be the maximum length for an indel we could encounter?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Potentially very long. However, why does it matter? This will be rare.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Is there a broader effort underway to address this concern?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;dbSNP comes to mind.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Would adopting the Bristol approach set a potential standard, or&#xA;should we explore or develop alternative methods? For example should&#xA;the genome build be built into the id? or species?&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It depends on the type of analysis you're planning on doing. Are you doing cross-species analyses? And if so, what types? Do you expect to be comparing genomes with different genome builds?&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;For example GWAS summary statistics. I want to be able to match&#xA;variants from different studies so I want to have an identifier for&#xA;the variants.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Both chr:pos as well as rsID work for this, but just make sure that the genome build matches if you go with chr:pos.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;I could use rs numbers I could use chr:pos I could use chr:pos:ref:alt&#xA;I could use chr:pos:effect:noneffect I could use&#xA;chr:pos:population:effect:noneffect or&#xA;studyid:chr:pos:populatoin:effect:noneffect or etc&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;If you're sure your input data has rs numbers and always will, that might be easiest. However, you may find that sometimes it doesn't have rs numbers (e.g. some Affymetrix data), and it's surpassingly annoying to add them. I'd choose chr:pos, personally, and then consider the effect/non-effect/ref/alt/ on a per-analysis basis, because I would want to still identify the same position if there was a different ref/alt for example.&lt;/p&gt;&#xA;&lt;p&gt;Edit:&#xA;Based on the comment (OP wants to match effect direction), I would suggest chr:pos:effect_allele, but of course this means there is an effect allele (i.e. the position has been studied).&lt;/p&gt;&#xA;" OwnerUserId="17430" LastEditorUserId="17430" LastEditDate="2023-10-30T14:28:49.170" LastActivityDate="2023-10-30T14:28:49.170" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21775" PostTypeId="2" ParentId="21549" CreationDate="2023-10-29T16:32:34.087" Score="2" Body="&lt;p&gt;To obtain cytoband and gene-level copy numbers from genome-wide SNP array copy number data, you might want to consider using the ASCAT tool, as it makes allele-specific copy number segment data and derived integer gene-level copy number values from SNP array data. The SNP6 copy number analysis pipeline (ASCAT2) I believe is specifically for this purpose.&lt;/p&gt;&#xA;&lt;p&gt;See also:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;CNVkit, which has a method for copy number detection via using targeted reads and the non-specifically captured off-target reads to infer copy number evenly across the genome.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Array Comparative Genomic Hybridization (aCGH) and SNP Genotyping Technologies&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;CNVannotator&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="17430" LastActivityDate="2023-10-29T16:32:34.087" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21776" PostTypeId="1" CreationDate="2023-10-29T18:40:23.000" Score="0" ViewCount="29" Body="&lt;p&gt;I am trying to check my variant generation pipeline for humans. The pipeline is for NGS. I am searching for any dataset(.fastq+.bam+.vcf) for any specific sample. Now, GIAB has a nice dataset for that. However, the problem is that it's not a tiny dataset. And it's not feasible to test my pipeline with this data every time. I tried to generate a synthetic dataset with some read simulators(NEAT, MASON2). However, their aligned read files(.bam) files seem not very good. You can check my Issue for NEAT in &lt;a href=&quot;https://github.com/ncsa/NEAT/issues/88&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;.&#xA;So, my question is, &lt;strong&gt;Is there any publicly available tiny dataset(.fastq,.bam, .vcf) with which I can validate my pipeline fast? Or, is there any simulator with which I can simulate .bam, .fastq files from a given input .vcf? I am using fastqc,bwa,samtools,GATK4 mainly and GRCH38 reference in my pipeline.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="10296" LastActivityDate="2023-10-29T21:08:03.470" Title="Generating simulated gold standard NGS dataset/ Any publicly available smal gold standard NGS dataset" Tags="|ngs|public-dataset|simulated-data|validation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21777" PostTypeId="2" ParentId="21776" CreationDate="2023-10-29T21:08:03.470" Score="2" Body="&lt;p&gt;I would suggest subsampling an existing dataset such as the &lt;a href=&quot;https://www.nist.gov/programs-projects/genome-bottle&quot; rel=&quot;nofollow noreferrer&quot;&gt;GIAB&lt;/a&gt; resource that you've already indicated.&lt;/p&gt;&#xA;&lt;p&gt;Use a utility such as &lt;a href=&quot;https://github.com/lh3/seqtk&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqtk&lt;/a&gt; to subsample the appropriate number of reads from the dataset, and use that.&lt;/p&gt;&#xA;&lt;p&gt;Example with &lt;code&gt;seqtk&lt;/code&gt; to get 1 million reads each from paired fastqs while preserving pairing:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  # note -s100 random seed which should be used for both to keep pairs&#xA;  seqtk sample -s100 giab_read1.fq 1000000 &amp;gt; giab_sub_r1.fq&#xA;  seqtk sample -s100 giab_read2.fq 1000000 &amp;gt; giab_sub_r2.fq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-10-29T21:08:03.470" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21779" PostTypeId="1" CreationDate="2023-10-30T02:22:42.603" Score="2" ViewCount="20" Body="&lt;p&gt;I have DNA sequences from a single gecko species for a set of genes (SULF1, SOX9, SAL1), I wanted to compare the putative promoter sequences for each gene, and compare them to the promoter regions of other related species. Would something like HOMER work for say finding TF binding motifs in around 20 different sequences for 20 different species, or would I need a larger dataset?&lt;/p&gt;&#xA;" OwnerUserId="18137" LastActivityDate="2023-10-30T02:22:42.603" Title="How to identify TF binding motifs?" Tags="|fasta|homer|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21781" PostTypeId="1" CreationDate="2023-10-30T17:53:19.813" Score="0" ViewCount="25" Body="&lt;p&gt;I found the human genome alignment sets here:&#xA;&lt;a href=&quot;https://www.ensembl.org/info/genome/compara/multiple_genome_alignments.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ensembl.org/info/genome/compara/multiple_genome_alignments.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;but they doesn't include, for example, ancient human genomes, e.g. those here:&#xA;&lt;a href=&quot;https://reich.hms.harvard.edu/allen-ancient-dna-resource-aadr-downloadable-genotypes-present-day-and-ancient-dna-data&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://reich.hms.harvard.edu/allen-ancient-dna-resource-aadr-downloadable-genotypes-present-day-and-ancient-dna-data&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;or, for example, any of the other 'pan genome' whole genome alignments here:&#xA;&lt;a href=&quot;https://humanpangenome.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://humanpangenome.org/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, those latter two resources don't include whole genome alignments...&lt;/p&gt;&#xA;&lt;p&gt;About 10 years ago I remember reading a paper that used 'lots' of genomes to uncover novel features using different patterns of conservation, demonstrating the power of the approach.&lt;/p&gt;&#xA;&lt;p&gt;Have I missed a new, more up to date, whole genome alignment resource?&lt;/p&gt;&#xA;&lt;p&gt;Many thanks,&lt;/p&gt;&#xA;" OwnerUserId="3233" LastActivityDate="2023-10-31T02:23:52.500" Title="Current 'most comprehensive' whole genome alignment resource for human?" Tags="|ensembl|multiple-sequence-alignment|pangenome|whole-genome|ancient-dna|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21782" PostTypeId="1" CreationDate="2023-10-30T17:58:35.647" Score="2" ViewCount="28" Body="&lt;p&gt;I'm trying to replicate an existing polygenic score (i.e. test the accuracy in a new sample), and want to know if matching the original study's microarray chip will improve the accuracy (that is, variance explained in a new test set) of the polygenic score.&lt;/p&gt;&#xA;&lt;p&gt;The labs I have access to have different multiple microarray chips to choose from (e.g., GSA, GDA, UK Biobank Axiom Array, UK BiLEVE Axiom array, GeneChip 2.0 array, Infinium Core-24 Kit, etc.). Will using a chip that is different than the chip used in the study that produced the polygenic score impact the explained variance (accuracy) of the polygenic score? If so, by how much?&lt;/p&gt;&#xA;&lt;p&gt;Example: a study on heart disease runs a GWAS on 900,000 individuals who are genotyped with the UK BiLEVE Axiom array. This study uses these summary statistics to produce weights for a polygenic score. Lab A, B, and C are conducting polygenic score validation/replication studies, and they each have a random sample of 10,000 individuals drawn from the general population, and they all use the same summary statistics from the original study. After imputing the genotype data of the 10,000 people, they calculate the polygenic score for each person, and then evaluate how much variance in the phenotype they are able to explain. The only difference is that Lab A genotypes their 10,000 people with the UK BiLEVE Axiom array, whereas lab B uses the Illumina GSA, and lab C uses the Illumina GDA.&lt;/p&gt;&#xA;&lt;p&gt;Do we have any reason to suspect that there will be a substantial difference in variance explained (r^2) by the polygenic score between labs? Are there any studies which have analyzed something that could be informative to this question?&lt;/p&gt;&#xA;&lt;p&gt;I've been unable to find research on this question. This should not be dependent on which polygenic score is being studied; there is no reason to specify a particular phenotype.&lt;/p&gt;&#xA;&lt;p&gt;Note: it is standard to impute both in the original study and the replication.&lt;/p&gt;&#xA;" OwnerUserId="17430" LastActivityDate="2024-03-29T07:04:50.387" Title="Does using a microarray chip that matches the chip used in training lead to higher explained variance of a polygenic score?" Tags="|snp|microarray|illumina|imputation|polygenic-risk-score|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21783" PostTypeId="2" ParentId="21781" CreationDate="2023-10-31T02:23:52.500" Score="2" Body="&lt;p&gt;The obvious thing missing from your list is the &lt;a href=&quot;https://www.science.org/doi/10.1126/science.abj6987&quot; rel=&quot;nofollow noreferrer&quot;&gt;telomere-to-telomere genome assemblies&lt;/a&gt; that have been coming out in the last couple years.&lt;/p&gt;&#xA;&lt;p&gt;Assembly link &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwjVuJqem5-CAxV1IDQIHU2mASIQFnoECCQQAQ&amp;amp;url=https%3A%2F%2Fwww.ncbi.nlm.nih.gov%2Fassembly%2F11828891&amp;amp;usg=AOvVaw1Q8aP_L5RFstLQ0JtKVX6M&amp;amp;opi=89978449&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;There are now a &lt;a href=&quot;https://www.nature.com/articles/s41422-023-00849-5&quot; rel=&quot;nofollow noreferrer&quot;&gt;few others as well&lt;/a&gt; that are similar.&lt;/p&gt;&#xA;&lt;p&gt;It's been demonstrated that &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9336181/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this T2T assembly improves mapping rates&lt;/a&gt;, in some cases pretty dramatically.&lt;/p&gt;&#xA;&lt;p&gt;I am not sure entirely what you want from whole genome alignments, but UCSC if not others &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTracks?db=hub_2395475_t2t-chm13-v1.0&amp;amp;lastVirtModeType=default&amp;amp;lastVirtModeExtraState=&amp;amp;virtModeType=default&amp;amp;virtMode=0&amp;amp;nonVirtPosition=&amp;amp;position=chr22%3A3977324%2D3977530&amp;amp;hgsid=1749469962_FUCnGU4vZLseVNNvfdKfFDr4aeOU&quot; rel=&quot;nofollow noreferrer&quot;&gt;has incorporated the T2T assembly&lt;/a&gt;. &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-022-02735-6&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is a related commentary&lt;/a&gt; (which I haven't read in any detail).&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-10-31T02:23:52.500" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21784" PostTypeId="2" ParentId="21782" CreationDate="2023-10-31T04:37:57.697" Score="0" Body="&lt;p&gt;Genotyping can be validated by whole-genome sequencing, and this has been done on the &lt;a href=&quot;https://www.internationalgenome.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;International Genome Sample Resource&lt;/a&gt;. You should be less concerned about the &lt;em&gt;reproducibility&lt;/em&gt; of genotyping results, and more concerned about the &lt;em&gt;correctness&lt;/em&gt; or &lt;em&gt;validity&lt;/em&gt; of genotyping results. Companies will often revisit old SNPchips to mark invalid spots, update rs numbers, and add new genotyping probes that are better on a population-wide scale. In that sense, choosing a SNPchip platform that has an established track record of QC and information about version updates would be a better idea than trying to match a previous study as precisely as possible.&lt;/p&gt;&#xA;&lt;p&gt;However, with regards to replicating existing studies, a bigger concern for you should be the variation in sampled populations, which plays a big role in the outcome of a polygenic test.&lt;/p&gt;&#xA;&lt;p&gt;This variation is so extreme that directions of associations can flip for the same variant, especially for low-frequency variants with extremely low p-values (i.e. the ones that often show up as &lt;a href=&quot;https://twitter.com/SbotGwa/status/1321074069194887168&quot; rel=&quot;nofollow noreferrer&quot;&gt;manhattan plot peaks&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;The issues around population and study replication are explained well in the following paper from 2001:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1006/tpbi.2001.1543&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1006/tpbi.2001.1543&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is not something that can be fixed by attempting to replicate studies in more populations. As Cecile Janssens says, larger sample size only affects random error, not systematic error.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://twitter.com/cecilejanssens/status/1119235476567801856&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://twitter.com/cecilejanssens/status/1119235476567801856&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-10-31T04:45:33.037" LastActivityDate="2023-10-31T04:45:33.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21785" PostTypeId="1" CreationDate="2023-10-31T10:12:48.637" Score="0" ViewCount="30" Body="&lt;p&gt;I have a list of genes and their associated pathways. The data was generated using Partek. The data looks like this -&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;Gene_in_list&lt;/th&gt;&#xA;&lt;th&gt;Gene_set&lt;/th&gt;&#xA;&lt;th&gt;Description&lt;/th&gt;&#xA;&lt;th&gt;Enrichment score&lt;/th&gt;&#xA;&lt;th&gt;P.value&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Gene1&lt;/td&gt;&#xA;&lt;td&gt;hsa&lt;/td&gt;&#xA;&lt;td&gt;Disease C&lt;/td&gt;&#xA;&lt;td&gt;69&lt;/td&gt;&#xA;&lt;td&gt;0.005&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Gene1&lt;/td&gt;&#xA;&lt;td&gt;hsa&lt;/td&gt;&#xA;&lt;td&gt;Disease A&lt;/td&gt;&#xA;&lt;td&gt;45&lt;/td&gt;&#xA;&lt;td&gt;0.05&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Gene2&lt;/td&gt;&#xA;&lt;td&gt;hsa&lt;/td&gt;&#xA;&lt;td&gt;Disease B&lt;/td&gt;&#xA;&lt;td&gt;23&lt;/td&gt;&#xA;&lt;td&gt;0.007&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Gene2&lt;/td&gt;&#xA;&lt;td&gt;hsa&lt;/td&gt;&#xA;&lt;td&gt;Disease Y&lt;/td&gt;&#xA;&lt;td&gt;40&lt;/td&gt;&#xA;&lt;td&gt;0.009&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Gene2&lt;/td&gt;&#xA;&lt;td&gt;hsa&lt;/td&gt;&#xA;&lt;td&gt;Disease C&lt;/td&gt;&#xA;&lt;td&gt;69&lt;/td&gt;&#xA;&lt;td&gt;0.0001&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;I want to 'split open' the Description column (Which I think means going long to wide in tidyverse terminology?), and transform it to the below table -&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;Gene_in_list&lt;/th&gt;&#xA;&lt;th&gt;Disease A&lt;/th&gt;&#xA;&lt;th&gt;Disease B&lt;/th&gt;&#xA;&lt;th&gt;Disease C&lt;/th&gt;&#xA;&lt;th&gt;Disease Y&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Gene 1&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Gene 2&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;So that I can use the online circos plot to essentially show, which genes are involved in the pathways.&lt;/p&gt;&#xA;&lt;p&gt;I was also debating if Enrichment score / p.value could be used as the thickness of connections in circos, but I think it might look too thin / uneven.&lt;/p&gt;&#xA;&lt;p&gt;Any help appreciated, thanks!&lt;/p&gt;&#xA;" OwnerUserId="6561" LastActivityDate="2023-10-31T10:12:48.637" Title="Need to make circos plot from Enrichment data" Tags="|gene|circos|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21786" PostTypeId="1" CreationDate="2023-10-31T16:19:03.287" Score="1" ViewCount="43" Body="&lt;p&gt;I'm using the Python package &lt;em&gt;lifelines&lt;/em&gt; to build a CoxPH model where I reduce the step size to 0.1, like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cph = CoxPHFitter()&#xA;cph.fit(df, 'time', 'event', fit_options = {&amp;quot;step_size&amp;quot;:0.1})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I used k_fold_cross_validation to build the model, but I don't know how to add in step size. I expected to add it to fitter_kwargs, like this, but this does not work:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;k_fold_cross_validation(cph, df, duration_col='time', event_col='event', k=5, fitter_kwargs={fit_options:{&amp;quot;step_size&amp;quot;:0.1}})&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18470" LastActivityDate="2023-10-31T16:19:03.287" Title="Adding step size to cross validation in Python package lifelines" Tags="|python|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21787" PostTypeId="1" CreationDate="2023-10-31T18:29:25.227" Score="2" ViewCount="41" Body="&lt;ol&gt;&#xA;&lt;li&gt;is there any way to retrieve a BLAST request made using the Biopython library using the request's query_id?&lt;/li&gt;&#xA;&lt;li&gt;Alternatively is it possible to retrieve a query_id before the blast result is complete, and checking on it later?&lt;/li&gt;&#xA;&lt;li&gt;(bit off-topic) I seem to only retrieve 60 results even at higher &lt;code&gt; hitlist_size&lt;/code&gt; although i get much more results when blasting the same entry on uniprot&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I'm working on a script which processes a BLAST result and having to re-request the search every time is quite temporally cumbersome since i'm still learning the library. currently i'm using: &lt;code&gt;   blastResult = NCBIWWW.qblast(&amp;quot;blastp&amp;quot;, &amp;quot;swissprot&amp;quot;, targetAminoAcids,format_type=&amp;quot;Tabular&amp;quot;, hitlist_size=200)&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="18471" LastActivityDate="2023-11-13T14:46:30.227" Title="retrieving past BLAST requests in bioPython" Tags="|python|blast|biopython|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21788" PostTypeId="1" AcceptedAnswerId="21800" CreationDate="2023-11-01T15:22:06.993" Score="2" ViewCount="56" Body="&lt;p&gt;I'm trying to get ALL Proteins for a Taxonomic ID from Entrez using Esearch/Efetch and the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db &amp;quot;protein&amp;quot; -query &amp;quot;txid374666[Organism]&amp;quot; | efetch -format fasta &amp;gt; txid_374666.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can somebody tell if this command will get ALL proteins or not? Besides, this appears to be a bit slow.&lt;/p&gt;&#xA;&lt;p&gt;Can somebody point out any Biopython workflow, particularly how to get ALL Proteins&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="6649" LastActivityDate="2023-11-03T16:46:23.990" Title="Getting ALL Protein entries from Entrez for a particular taxonomic id" Tags="|sequence|entrez|biopython|efetch|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21789" PostTypeId="1" AcceptedAnswerId="21790" CreationDate="2023-11-01T16:40:44.043" Score="1" ViewCount="72" Body="&lt;p&gt;I am trying to create a simple pipeline that takes a file containing various short sequences, and run them through blast. I have an example sequence;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;some_random_sequence&#xA;AAGGTTCGGTCCAAATTGAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which returns a hit for FUT3 (using the RefSeq_Gene DB and using grch38 as an entrez_query). However, when I try to run this using NCBIWWW (Biopython) I don't get any results and I would like to figure out why.&lt;/p&gt;&#xA;&lt;p&gt;A snippet that contains the call that I am using, is listed here;&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio.Blast import NCBIWWW&#xA;from io import StringIO&#xA;&#xA;sequence=['&amp;gt;some_random_sequence\n','AAGGTTCGGTCCAAATTGAA']&#xA;&#xA;lines = &amp;quot;&amp;quot;.join(sequence)&#xA;file = StringIO(lines)&#xA;results = NCBIWWW.qblast(&#xA;    program=&amp;quot;blastn&amp;quot;,&#xA;    database=&amp;quot;RefSeq_Gene&amp;quot;,&#xA;    entrez_query=&amp;quot;grch38&amp;quot;,&#xA;    sequence=file.read(),&#xA;    # Manually setting the parameters because short_query checks the length&#xA;    # of the sequence, but for some reason includes the name (resulting in&#xA;    # a length of 42, and thus defeating the entire purpose of using&#xA;    # short_query (which does a if len(sequence) &amp;lt; 31).&#xA;    expect=1000,&#xA;    word_size=7,&#xA;    nucl_reward=1,&#xA;    filter=None,&#xA;    lcase_mask=None,&#xA;    # short_query=True,&#xA;)&#xA;with open('test_output.xml', 'w') as save_file:&#xA;    blast_results = results.read()&#xA;    save_file.write(blast_results)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18475" LastEditorUserId="650" LastEditDate="2023-11-01T16:47:18.950" LastActivityDate="2023-11-02T16:16:06.957" Title="Blastn through NCBIWWW not giving any results, as opposed to the online BLAST" Tags="|blast|biopython|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21790" PostTypeId="2" ParentId="21789" CreationDate="2023-11-01T17:29:59.197" Score="2" Body="&lt;p&gt;This is easy. If you set&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;expect=1000,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;expect=100,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It will work. NCBI servers impose a tighter restriction on the CLI whilst the web restriction is 5000.&lt;/p&gt;&#xA;&lt;p&gt;If you require a &lt;code&gt;1000&lt;/code&gt; restriction then out of (US) hours blasting is required, e.g. weekends.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The Perl Blast API is here&#xA;&lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/docs/web_blast.pl&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blast.ncbi.nlm.nih.gov/docs/web_blast.pl&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The documentation is here &lt;a href=&quot;https://ncbi.github.io/blast-cloud/dev/api.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ncbi.github.io/blast-cloud/dev/api.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Normally you need to modify it, that would be a separate question. This will work in default mode.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-11-02T16:16:06.957" LastActivityDate="2023-11-02T16:16:06.957" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21792" PostTypeId="2" ParentId="21742" CreationDate="2023-11-02T09:47:51.477" Score="1" Body="&lt;p&gt;You could use the &lt;code&gt;tryCatch&lt;/code&gt; &lt;a href=&quot;http://127.0.0.1:30714/library/base/html/conditions.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;function&lt;/a&gt; to handle inputs that throw an error. A good example on how to write them can be found &lt;a href=&quot;https://stackoverflow.com/questions/12193779/how-to-write-trycatch-in-r&quot;&gt;here&lt;/a&gt;. Though the message indicates it's a warning, so does the function stop running when you get this message?&lt;/p&gt;&#xA;" OwnerUserId="18402" LastActivityDate="2023-11-02T09:47:51.477" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21793" PostTypeId="1" CreationDate="2023-11-02T18:18:47.337" Score="-1" ViewCount="30" Body="&lt;p&gt;I'm trying to use DNABERT to predict promoter regions. Right now, I have a df with 500k rows, 1st column is the DNA sequence (all different lengths), 2nd column says &amp;quot;promoter&amp;quot; vs. &amp;quot;non-promoter&amp;quot;. I'm trying to understand the GitHub, read the paper, as well as other background reading, but I'm still confused.&lt;/p&gt;&#xA;&lt;p&gt;If I want to get started, they said it's optional to pre-train (so you can skip to step 3). This is where I got tripped up: &amp;quot;Note that the sequences are in kmer format, so you will need to convert your sequences into that.&amp;quot; From what I understand, you need to do this so that all of the sequences are the same length? So kmer=6 means all of the sequences are length 6? Someone suggested that I take the first nucleotide in the promoter and grab 3 nucleotides before and 3 nucleotides after (+/-3 bases). I don't think that's how the kmer thing works though? I tried replicating how I think it works down below (I got confused on the last row of the 'after' df). Please correct me if I'm wrong!&#xA;Also, if there is a good package that does this for me, please let me know!&lt;/p&gt;&#xA;&lt;p&gt;Before:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seq                          label &#xA;AAAGAGGT                      1 [promoter]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seq                          label &#xA;AAAGAG                         1&#xA;AAGAGG                         1&#xA;AGAGGT                         1&#xA;??[GAGGT]??                    1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18482" LastActivityDate="2023-11-03T14:44:20.943" Title="New to DNABERT confused about kmers input" Tags="|python|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21794" PostTypeId="2" ParentId="21513" CreationDate="2023-11-02T20:24:26.793" Score="1" Body="&lt;p&gt;The next step depends heavily on what your question is. Some general ideas, not specific to tumor particle penetration.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Your current visualization seems OK to me, if you want to make some features more obvious you can try density estimation.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you have a generative model (for example you believe this is a mixture of a &amp;quot;shallow&amp;quot; and &amp;quot;deep&amp;quot; Gaussian distribution, or you have reason to believe the particles follow an exponential distribution), you can fit your data to this distribution and estimate its parameters.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you want to compare conditions without too strong assumptions on the distributions, you could compute the cumulative distribution of the particles. There are statistical approaches to compare such curves.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;To compare conditions, you might even be able to come up with a simpler metric, for example &amp;quot;total fluorescence in the core of the spheroid&amp;quot; normalized by &amp;quot;total fluorescence in its periphery&amp;quot;.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Ultimately it all depends on your research question (are there treatments to compare to each others, biological replicates whose consistency you want to check, known structure within the tumor spheroid, a time course, ...). My suggestions above would be easier to implement in R/Matlab/Python than pure ImageJ.&lt;/p&gt;&#xA;" OwnerUserId="9497" LastActivityDate="2023-11-02T20:24:26.793" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21795" PostTypeId="1" CreationDate="2023-11-02T21:16:53.103" Score="0" ViewCount="120" Body="&lt;p&gt;My library is unstranded and the code I'm trying to use is this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Run alignment with unstranded method&#xA;hisat2 -q --rna-strandness none -x HISAT2/grch38/genome_index_grch38 -U hipsci/Kolf-1_R1_001.fastq | samtools sort -o HISAT2/Kolf-1_R1_001.bam&#xA;&#xA;echo &amp;quot;HISAT2 finished running!&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, I get this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(base) ~ % RNASeq_pipline/data/script/RNASeqpipline.sh&#xA;&#xA;Error: should be one of F, R, FR, or RF &#xA;Error: Encountered internal HISAT2 exception (#1)&#xA;Command: /usr/local/bin/hisat2-align-s --wrapper basic-0 -q --rna-strandness none -x &#xA;HISAT2/grch38/genome_index_grch38 -U hipsci/Kolf-1_R1_001.fastq &#xA;(ERR): hisat2-align exited with value 1&#xA;[W::hts_set_opt] Cannot change block size for this format&#xA;samtools sort: failed to read header from &amp;quot;-&amp;quot;&#xA;HISAT2 finished running!&#xA;0 minutes and 0 seconds elapsed.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Looking at the HISAT2 Manual:&lt;/p&gt;&#xA;&lt;p&gt;For single-end reads:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Use 'F' if the read corresponds to a transcript.&lt;br /&gt;&#xA;Use 'R' if the read corresponds to the reverse complemented counterpart of a transcript.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;For paired-end reads:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Use 'FR' if the first read in the pair corresponds to a transcript on the forward strand, and the second read corresponds to the reverse strand.&lt;br /&gt;&#xA;Use 'RF' if the first read in the pair corresponds to a transcript on the reverse strand, and the second read corresponds to the forward strand.&lt;/p&gt;&#xA;&lt;p&gt;When you use the &lt;code&gt;--rna-strandness&lt;/code&gt; option with either 'FR' or 'RF' for paired-end reads, HISAT2 will assign an XS attribute tag to each read alignment, indicating whether the read belongs to a transcript on the '+' (plus) or '-' (minus) strand of the genome.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;It doesn't say what to input in the case of an unstranded library. Any help would be greatly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="18483" LastEditorUserId="650" LastEditDate="2023-11-02T21:23:03.497" LastActivityDate="2023-11-04T04:31:59.240" Title="HISAT2: RNA strandedness" Tags="|rna-seq|rna|rna-alignment|strandedness|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21796" PostTypeId="2" ParentId="21795" CreationDate="2023-11-03T03:58:40.420" Score="2" Body="&lt;p&gt;According to the help produced when I type &lt;code&gt;hisat2 --help&lt;/code&gt;, the default strandedness when running HISAT2 is unstranded:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;  --rna-strandness &amp;lt;string&amp;gt;          specify strand-specific information (unstranded)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So taking out the &lt;code&gt;rna-strandedness&lt;/code&gt; option should map the reads as if they were unstranded:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Run alignment with unstranded method&#xA;hisat2 -q -x HISAT2/grch38/genome_index_grch38 -U hipsci/Kolf-1_R1_001.fastq | samtools sort -o HISAT2/Kolf-1_R1_001.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If this is a problem for you, I recommend you create an issue about it on the HISAT2 github page:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/DaehwanKimLab/hisat2/issues/new&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/DaehwanKimLab/hisat2/issues/new&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-11-04T04:31:59.240" LastActivityDate="2023-11-04T04:31:59.240" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21797" PostTypeId="1" CreationDate="2023-11-03T10:33:53.293" Score="1" ViewCount="113" Body="&lt;p&gt;I would like to perform blast+, using the command line, here is the code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-task blastn -db nr -outfmt 7 -query myQuery.fas -out result.txt -remote.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have to exclude SARS-CoV-2 hits from my results, so I must include &lt;code&gt;-negative_taxids&lt;/code&gt; mentioning SARS-CoV-2 NCBI taxid number. However, the problem is, &lt;code&gt;-negative_taxids&lt;/code&gt; is NOT compatible with &lt;code&gt;-remote&lt;/code&gt; command and my command line encounters an error.&#xA;I have to use &lt;code&gt;-remote&lt;/code&gt; as I cannot download entire +400 GB NR database to run it locally.&#xA;Could you please kindly help me out with this contradiction in my work?&lt;/p&gt;&#xA;&lt;p&gt;My output txt file for this command line is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Fields: query acc.ver, subject acc.ver, % identity, alignment length, mismatches, gap opens, q. start, q. end, s. start, s. end, evalue, bit score&#xA;# 600483 hits found&#xA;AstV-VA1    MN464146.1  100.000 32  0   0   1   32  6362    6393    7.14e-06    59.0&#xA;AstV-VA1    MN148428.1  100.000 32  0   0   1   32  6685    6716    7.14e-06    59.0&#xA;AstV-VA1    MK671313.1  100.000 32  0   0   1   32  6682    6713    7.14e-06    59.0&#xA;AstV-VA1    MK671312.1  100.000 32  0   0   1   32  6682    6713    7.14e-06    59.0&#xA;AstV-VA1    MK671311.1  100.000 32  0   0   1   32  6682    6713    7.14e-06    59.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18298" LastEditorUserId="298" LastEditDate="2023-11-04T15:54:57.700" LastActivityDate="2023-11-05T14:33:18.217" Title="How can I limit blast results to exclude a specific taxonomic ID?" Tags="|blast|sars-cov-2|ncbi|command-line|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21798" PostTypeId="1" CreationDate="2023-11-03T10:44:04.007" Score="1" ViewCount="22" Body="&lt;p&gt;We measured blood DNA methylation using epic array in 100 no-diabetic and 100 type 1 diatbetic patients matched for age, sex and smoking and also measured arsenic exposure using ICP-MS. Could you please suggest best way to find CpG sites associated with arsenic exposure and diabetes. We know that toxic element leads to health risk but we want to study this by using DNA methylation as mediator for disease risk.&lt;/p&gt;&#xA;&lt;p&gt;Could you please suggest any R package for doing these types of analysis. I have used previously limma for toxic element exposure and DNA methylation association analysis but not sure how make connections with health risk phenotype?&lt;/p&gt;&#xA;" OwnerUserId="1008" LastActivityDate="2023-11-03T10:44:04.007" Title="DNA methylation mediator analysis for environmental exposure and health outcome" Tags="|r|statistics|methylation|epigenetics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21799" PostTypeId="2" ParentId="21793" CreationDate="2023-11-03T14:44:20.943" Score="0" Body="&lt;p&gt;The next line in the manual speaks of a python function they provide that can get k-mers from your sequence, so you don't need to worry about doing this a different way. The function slices the sequence starting from the first base with a sliding window of k to obtain k-sized pieces of the sequence and then space-separates these pieces into a single string.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;seq=&amp;quot;ATGCCGTA&amp;quot;&#xA;k=2&#xA;[seq[x:x+k] for x in range(len(seq)+1-k)]&#xA;['AT', 'TG', 'GC', 'CC', 'CG', 'GT', 'TA']&#xA;&#xA;&amp;quot; &amp;quot;.join([seq[x:x+k] for x in range(len(seq)+1-k)])&#xA;'AT TG GC CC CG GT TA'&#xA;&#xA;k=3&#xA;&amp;quot; &amp;quot;.join([seq[x:x+k] for x in range(len(seq)+1-k)])&#xA;'ATG TGC GCC CCG CGT GTA'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-11-03T14:44:20.943" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21800" PostTypeId="2" ParentId="21788" CreationDate="2023-11-03T16:46:23.990" Score="3" Body="&lt;p&gt;As suggested by somebody, the datasets tools from NCBI  is blazing fast:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;datasets download genome taxon 187492  --include protein&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets/docs/v2/download-and-install/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Datasets Tool&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6649" LastActivityDate="2023-11-03T16:46:23.990" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21801" PostTypeId="1" CreationDate="2023-11-03T20:01:36.320" Score="1" ViewCount="25" Body="&lt;p&gt;I am writing a project proposal for my boss and it is on codon usage bias in the dystrophin protein and the spectrin-like repeats found in a hmmscan file using the longest dystrophin isoform protein sequence which I got from a GenBank file!&lt;/p&gt;&#xA;&lt;p&gt;There is a lot of research on how mRNA and tRNA impact codon usage bias and codon frequency.  Long story short, I work at a gene therapy lab as a bioinformatician doing research for muscular dystrophy.  Our lab is very interested in miRNA, and I have found a certain amount of articles that show miRNA may impact codon usage bias since miRNA can either cleave or translationally repress mRNA.&lt;/p&gt;&#xA;&lt;p&gt;I found a miRNA database called &lt;a href=&quot;http://www.tools4mirs.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;www.tools4mirs.org&lt;/a&gt;. It seems like a potentially good database, but I don't know if it's the right database geared for this project.  We saw some unusual lack of protein expression when there should be a lot of protein expression and our lab is really focused on an interested in miRNAs.&lt;/p&gt;&#xA;&lt;p&gt;Are there any other miRNA databases/tools that you can recommend?&lt;/p&gt;&#xA;&lt;p&gt;I code in Python and know some AI/ML and would like to make this project a AI/ML bioinformatics project, but I do not know what some good miRNA databases are, since I want to pull large amounts of miRNA data for my project that is relative to muscular dystrophy.&lt;/p&gt;&#xA;&lt;p&gt;So to sum up my question, what are some really heavily and well-curated miRNA databases besides tools4mirs.org ? I am kind of new to this field, and would like to hear any suggestion since I am still learning the inner workings of the field of bioinformatics and the possibilities and directions it can go in.&lt;/p&gt;&#xA;" OwnerUserId="18487" LastEditorUserId="73" LastEditDate="2023-11-05T11:21:29.683" LastActivityDate="2023-11-05T11:21:29.683" Title="What are some good miRNA databases?" Tags="|python|proteins|genomics|rna|genetics|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21802" PostTypeId="2" ParentId="21473" CreationDate="2023-11-03T20:07:58.160" Score="0" Body="&lt;p&gt;From the biology side of bioinformatics and the easier option, I would use Mauve.  If you want to make the project more specific and tailored to your needs, and you have python skills, I would write Python code to complete this task.  I would specifically use the bioinformatics Python library called Biopython: &lt;a href=&quot;https://biopython.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biopython.org&lt;/a&gt;&#xA;pip install biopython&lt;/p&gt;&#xA;&lt;p&gt;The second option is more involved but it gives you more control on your project/research since you're the one creating it.  I personally would go with the 2nd option if you have coding skills.  If you don't have coding skills, stick to the 1st option.  Im sure there are also tools you can use that are found on Galaxy:&#xA;&lt;a href=&quot;https://usegalaxy.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://usegalaxy.org&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am not sure what tools there are that would be good for your task and needs, but from using Galaxy in the past, im sure you can find something! :)&lt;/p&gt;&#xA;" OwnerUserId="18487" LastActivityDate="2023-11-03T20:07:58.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21806" PostTypeId="2" ParentId="21797" CreationDate="2023-11-04T12:59:15.487" Score="1" Body="&lt;p&gt;This isn't particularly elegant, but it should work. First, extract the accessions into a file. Next, get the taxid for each of the accessions. Once you have that, you can identify the accessions that are for SARS-CoV-2 sequences, and remove them from your output. Something like this:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Extract the accessions into a new file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '!/^#/{print $2}' blast.out &amp;gt; names&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Map those accessions to their taxids.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;while read -r accession; do&#xA;   printf '%s\t%s\n' &amp;quot;$accession&amp;quot;  \&#xA;       $(curl -s &amp;quot;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=nuccore&amp;amp;id=${accession}&amp;amp;rettype=summary&amp;amp;retmode=text&amp;quot; | &#xA;          awk -F'[&amp;lt;&amp;gt;]' '/Name=&amp;quot;TaxId&amp;quot;/{print $3}'); &#xA;done &amp;lt; names &amp;gt; taxids&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;With your example input, that produces:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;$ while read -r accession; do&#xA;&amp;gt;    printf '%s\t%s\n' &amp;quot;$accession&amp;quot;  \&#xA;&amp;gt;        $(curl -s &amp;quot;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=nuccore&amp;amp;id=${accession}&amp;amp;rettype=summary&amp;amp;retmode=text&amp;quot; | &#xA;&amp;gt;           awk -F'[&amp;lt;&amp;gt;]' '/Name=&amp;quot;TaxId&amp;quot;/{print $3}'); &#xA;&amp;gt; done &amp;lt; names&#xA;MN464146.1   1435485&#xA;MN148428.1   2661902&#xA;MK671313.1   1239566&#xA;MK671312.1   1239566&#xA;MK671311.1   1239566&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, this will probably be way too slow, doing them one at a time. So if you have many IDs, try and combine multiple queries into one. Like this:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;perl -ne 'chomp; print $. % 50 == 0 ? &amp;quot;$_\n&amp;quot; : &amp;quot;$_,&amp;quot; ' names | &#xA; | sed 's/,$/\n/' | &#xA;    while read accession; do &#xA;      curl -s &amp;quot;https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=nuccore&amp;amp;id=${accession}&amp;amp;rettype=summary&amp;amp;retmode=text&amp;quot; | &#xA;        awk -F'[&amp;lt;&amp;gt;]' '/Name=&amp;quot;Caption&amp;quot;/{ acc=$3 }&#xA;                   /Name=&amp;quot;TaxId&amp;quot;/{printf &amp;quot;%s\t%s\n&amp;quot;, acc, $3}'&#xA;    done  &amp;lt; names &amp;gt; taxids&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here, the first Perl command will read through the list of accessions and print the accession followed by a comma and, once every 50 lines (&lt;code&gt;$. % 50 == 0&lt;/code&gt;), it will instead print it with a newline. The &lt;code&gt;sed&lt;/code&gt; command will replace a trailing comma with a newline for the very last line of output where, unless the number of accessions is an exact multiple of 50, you will have a line ending with a comma instead of a newline.&lt;/p&gt;&#xA;&lt;p&gt;The result is a list of 50 comma-separated identifiers per line (fewer, on the last line). This is then fed to the &lt;code&gt;curl&lt;/code&gt; command and then a naive awk that parses it and prints out the accession and the taxid.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Important&lt;/strong&gt;: this approach loses the version information. So &lt;code&gt;MK671311.1&lt;/code&gt; becomes &lt;code&gt;MK671311&lt;/code&gt;. This shouldn't be a problem, and we will take it into account later, but it should be understood. This is what the output looks like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MN464146    1435485&#xA;MN148428    2661902&#xA;MK671313    1239566&#xA;MK671312    1239566&#xA;MK671311    1239566&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Remove the SARS-CoV-2 sequences.&lt;/p&gt;&#xA;&lt;p&gt;The taxid for SARS-CoV-2 is &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi?id=2697049&quot; rel=&quot;nofollow noreferrer&quot;&gt;2697049&lt;/a&gt;, so you can get the list of its accessions with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk -F'\t' '$2==&amp;quot;2697049&amp;quot;{print $1}' taxids &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can now filter your file to remove them using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;awk '(NR==FNR){ &#xA;       if($2==&amp;quot;2697049&amp;quot;){&#xA;         sars[$1]&#xA;       } &#xA;       next&#xA;      }&#xA;      { &#xA;        acc=$2; &#xA;        sub(/\.[0-9]*$/, &amp;quot;&amp;quot;, acc); &#xA;        if(acc in sars){ next } &#xA;          print&#xA;      }' taxids blast.out &amp;gt; blast.filtered.out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This awk script will read the first input file (&lt;code&gt;NR&lt;/code&gt; is the current input line number while &lt;code&gt;FNR&lt;/code&gt; the line number of the file being read; the two are only equal while reading the first file) and if its second field is the desired taxid, it saves the first field, the accession, in the array &lt;code&gt;sars&lt;/code&gt;. The &lt;code&gt;next&lt;/code&gt; ensures we don't do anything else while reading the first file.&lt;/p&gt;&#xA;&lt;p&gt;When we move to the second file, we make a copy of the second field, the accession, and save it as &lt;code&gt;acc&lt;/code&gt;, and then we remove the version information (&lt;code&gt; sub(/\.[0-9]*$/, &amp;quot;&amp;quot;, acc)&lt;/code&gt;. If the resulting accession is found in the &lt;code&gt;sars&lt;/code&gt; array, we skip the line with &lt;code&gt;next&lt;/code&gt; and if not, we print.&lt;/p&gt;&#xA;&lt;p&gt;This assumes you are using the faster, batch retrieval I gave above, the one that removes the versioning information. If you instead use the very slow first one, you can filter with&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; awk '(NR==FNR){ if($2==&amp;quot;2697049&amp;quot;){ sars[$1]} next }&#xA;      { if($2 in sars){next} print}' taxids blast.out &amp;gt; blast.filtered.out&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;That should give you a filtered blast output file that no loner has sequences from taxid 2697049. &lt;strong&gt;However&lt;/strong&gt;, please make sure to test this, count the number of input and output lines and make sure everything makes sense. I don't have access to your actual data and while this does work with the example you provided, I cannot guarantee it will work out of the box for your real data. As always, you need to be careful!&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2023-11-04T18:38:31.503" LastActivityDate="2023-11-04T18:38:31.503" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21808" PostTypeId="1" CreationDate="2023-11-04T20:25:49.360" Score="1" ViewCount="67" Body="&lt;p&gt;I'm trying to download a list of all of the non-redundant protein chain IDs in the RSCB-PDB (not the actual FASTA entries, just the IDs), filtered by some % identity.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know how to do this?  I haven't been able to figure it out with the &lt;a href=&quot;https://www.rcsb.org/search&quot; rel=&quot;nofollow noreferrer&quot;&gt;advanced search function&lt;/a&gt; .&lt;/p&gt;&#xA;" OwnerUserId="18494" LastEditorUserId="298" LastEditDate="2023-11-05T11:14:19.860" LastActivityDate="2023-11-05T13:47:07.143" Title="Downloading a list of all RCSB-PDB entries" Tags="|proteins|protein-structure|pdb|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="21810" PostTypeId="1" CreationDate="2023-11-05T03:03:17.780" Score="1" ViewCount="30" Body="&lt;p&gt;I'm currently doing my thesis with the topic of phylogenetic analysis and is taking references from the previous person in my university (who have done the same topics but different species).&lt;/p&gt;&#xA;&lt;p&gt;They used the largest contigs comprising well known markers such as matK, rbcL, and ITS. However my largest contigs doesn't have them.&lt;/p&gt;&#xA;&lt;p&gt;Is it possible for me to use the 2nd/3rd largest contigs that have those markers?&lt;/p&gt;&#xA;" OwnerUserId="18495" LastEditorUserId="3848" LastEditDate="2023-11-05T17:13:15.923" LastActivityDate="2023-12-05T18:08:51.137" Title="Annotation result for phylogenetic analysis shows no common evolutionary gene in contigs, can i change the contigs?" Tags="|phylogenetics|phylogeny|genome|sequence-annotation|scaffold|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21811" PostTypeId="2" ParentId="21797" CreationDate="2023-11-05T03:10:32.087" Score="0" Body="&lt;p&gt;I would personally look at is the version number of the blast you are using.&lt;/p&gt;&#xA;&lt;p&gt;Bugs around blast+ versions &amp;gt;= 2.9.0 (April 2019) for &lt;code&gt;-taxids&lt;/code&gt; and I would assume &lt;code&gt;-negative_taxids&lt;/code&gt; flag should have been fixed as described in the the Blast+ release notes &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK131777/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Fix taxID filtering combined with mask-based alias BLAST databases.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Bug fix descriptions are usually short, hence the lack of further detail.&lt;/p&gt;&#xA;&lt;p&gt;Blast &lt;code&gt;taxids&lt;/code&gt; can be implemented via lists here &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK569846/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/books/NBK569846/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-11-05T14:33:18.217" LastActivityDate="2023-11-05T14:33:18.217" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21812" PostTypeId="2" ParentId="21808" CreationDate="2023-11-05T13:47:07.143" Score="1" Body="&lt;p&gt;Did you try RCSB &lt;a href=&quot;https://www4.rcsb.org/search/advanced/sequence&quot; rel=&quot;nofollow noreferrer&quot;&gt;sequence similarity search&lt;/a&gt;?&#xA;These two announcement highlight the options you ask about:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.rcsb.org/news/5764422199cccf72e74ca34f&quot; rel=&quot;nofollow noreferrer&quot;&gt;Improved Searching with New Sequence Identity Cutoff&lt;/a&gt; (2017)&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.rcsb.org/news/feature/61b8e5e869569d045d35a9a3&quot; rel=&quot;nofollow noreferrer&quot;&gt;Remove Redundancy in Search Results&lt;/a&gt; (2015)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Alternatively, if you'd like to calculate identity on your own, you can download sequences of all the PDB entries from:&#xA;&lt;a href=&quot;https://files.wwpdb.org/pub/pdb/derived_data/pdb_seqres.txt&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://files.wwpdb.org/pub/pdb/derived_data/pdb_seqres.txt&lt;/a&gt;&#xA;(270MB).&lt;/p&gt;&#xA;" OwnerUserId="266" LastActivityDate="2023-11-05T13:47:07.143" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21813" PostTypeId="2" ParentId="21810" CreationDate="2023-11-05T14:20:43.077" Score="1" Body="&lt;p&gt;ITS: what is needed is the aim of the phylogenetic investigation. ITS evolves very quickly so it's good for a few closely related species and sub-species. Understanding the behaviour of a locus is as important as the volume of the locus analysed. Thus, the answer is yes.&lt;/p&gt;&#xA;&lt;p&gt;I have a feeling these are insects/arthropods simply because of the fragmented nature of the genome: they're hard to assemble and genetically diverse so there's often no reference genome. The issue of genome vs locus was addressed by David Penny years ago: genome trees can be misleading and that would be justification for using the contigs to derive select loci. The reason why genome trees can be misleading is complex and a separate question.&lt;/p&gt;&#xA;&lt;p&gt;Moreover, if you can map the loci to a chromosome and demonstrate the contigs are on different chromosomes you can use differences/similarities in trees between chromosomes as the first stage of disequilibrium analysis, i.e. selection acting via Mendelian exchange.&lt;/p&gt;&#xA;&lt;p&gt;To reiterate there is good phylogenetic justification for switching contigs because the loci are more important than the size of the contig (you'll need supporting references however).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-11-05T14:59:59.787" LastActivityDate="2023-11-05T14:59:59.787" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21815" PostTypeId="1" CreationDate="2023-11-06T08:19:40.257" Score="1" ViewCount="25" Body="&lt;p&gt;I am attempting to install and use the Bioconductor package DSS on Ubuntu 22.04.&lt;/p&gt;&#xA;&lt;p&gt;Following the instructions here: &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/DSS.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioconductor.org/packages/release/bioc/html/DSS.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I get to this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;hfile_libcurl.c:45:10: fatal error: curl/curl.h: No such file or directory     &#xA;   45 | #include &amp;lt;curl/curl.h&amp;gt;                                                                                        &#xA;      |          ^~~~~~~~~~~~~                                                                                        &#xA;compilation terminated.                                                                                               &#xA;make[1]: *** [Makefile.Rhtslib:128: hfile_libcurl.o] Error 1                 &#xA;make[1]: Leaving directory '/tmp/RtmpNry6uq/R.INSTALL10673560786c68/Rhtslib/src/htslib-1.7'&#xA;make: *** [Makevars.common:24: htslib] Error 2  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="180" LastActivityDate="2023-11-06T08:21:17.740" Title="installing and using differential methylation DSS software" Tags="|dss|differential-methylation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21816" PostTypeId="2" ParentId="21815" CreationDate="2023-11-06T08:21:17.740" Score="1" Body="&lt;p&gt;In Ubuntu 22.04, install:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;sudo apt install libcurl4-openssl-dev&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then reattempt:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;if (!require(&amp;quot;BiocManager&amp;quot;, quietly = TRUE))&#xA;    install.packages(&amp;quot;BiocManager&amp;quot;)&#xA;&#xA;BiocManager::install(&amp;quot;DSS&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then upon completion:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(DSS)&#xA;Loading required package: Biobase&#xA;Loading required package: BiocGenerics&#xA;&#xA;Attaching package: ‘BiocGenerics’                                                                                     &#xA;&#xA;The following objects are masked from ‘package:stats’:                                                                &#xA;                                                                                                                      &#xA;    IQR, mad, sd, var, xtabs                                                                                          &#xA;...                                                                                                                      &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="180" LastActivityDate="2023-11-06T08:21:17.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21817" PostTypeId="1" CreationDate="2023-11-06T10:24:41.950" Score="2" ViewCount="51" Body="&lt;p&gt;Not sure if anyone can help me with a Racon issue as the developers are not very responsive on GitHub. I am trying to assemble a genome without a reference. I first did basecalling using dorado, then converted from bam to fastq using samtools, then trimmed using nanofilt from there I used flye for assembly and then ran minimap2 to get my sam file. For some reason when I then run Racon, I get the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[racon::Polisher::initialize] loaded target sequences 0.000226 s&#xA;[racon::Polisher::initialize] loaded sequences 0.034854 s&#xA;[racon::Polisher::initialize] error: empty overlap set!&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I used the following command to run the assembly:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;flye --nano-raw sample_id.trimmed.fastq -o assembly -I 3 -t 4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I used the following command to run minimap2:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 -ax map-ont -t 4 sample_id.trimmed1.fastq assembly/assembly.fasta &amp;gt; sample_id.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then used the following command to run racon:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;racon -m 8 -x -8 -g -6 -w 500 -t 4 sample_id.trimmed1.fastq sample_id.sam assembly/assembly.fasta &amp;gt; Racon_polished.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Could I please get some assistance on this issue as I am not sure what I am doing wrong. Thank you&lt;/p&gt;&#xA;&lt;p&gt;First lines of the assembled fasta file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;contig_1&#xA;TAACAGTGAAATTCAATGGTCTGGTATCCATCTAAACATTGAATGAGCAAGAAAAATACA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;First lines of the trimmed fastq file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@bf7e12fd-b52d-498e-9e61-39bfe2eacc80&#xA;TGTACCCTAGTGGAGAATTCACCAACGCAGCAATACCTCCGGTGTTCTGTCTACAGCGTTACTGTGTCT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;First lines of the sam file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;contig_1    0   08ef4187-c2ab-4756-a269-275f6dfe99af    1   60  93179S96M1I81M1D29M1I10M2I420M1I118M1I107M1D63M1I18M4I660M2I242M CTAACTGTATTGCTGGGAAACAAATCAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18501" LastEditorUserId="73" LastEditDate="2023-11-15T09:46:39.997" LastActivityDate="2023-11-15T09:46:39.997" Title="Racon: empty overlap set!" Tags="|bash|nextflow|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21818" PostTypeId="1" AcceptedAnswerId="21819" CreationDate="2023-11-06T13:25:36.897" Score="0" ViewCount="57" Body="&lt;p&gt;I have a vcf file that I annoteted with VEP, for human data. I have run VEP to annotate my files with some additional parameters (as shown below in the ##VEP-command-line).&lt;/p&gt;&#xA;&lt;p&gt;However, my output is rather strange (mainly the INFO column).&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##VEP=&amp;quot;v108&amp;quot; time=&amp;quot;2023-04-27 15:13:08&amp;quot; cache=&amp;quot;workflow/resources/variants/cache_vep/homo_sapiens/108_GRCh38&amp;quot; ensembl-funcgen=108.56bb136 ensembl-variation=108.a885ada ensembl-io=108.58d13c1 ensembl=108.d8a9c80 1000genomes=&amp;quot;phase3&amp;quot; COSMIC=&amp;quot;96&amp;quot; ClinVar=&amp;quot;202205&amp;quot; HGMD-PUBLIC=&amp;quot;20204&amp;quot; assembly=&amp;quot;GRCh38.p13&amp;quot; dbSNP=&amp;quot;154&amp;quot; gencode=&amp;quot;GENCODE 42&amp;quot; genebuild=&amp;quot;2014-07&amp;quot; gnomADe=&amp;quot;r2.1.1&amp;quot; gnomADg=&amp;quot;v3.1.2&amp;quot; polyphen=&amp;quot;2.2.2&amp;quot; regbuild=&amp;quot;1.0&amp;quot; sift=&amp;quot;sift5.2.2&amp;quot;&#xA;##INFO=&amp;lt;ID=CSQ,Number=.,Type=String,Description=&amp;quot;Consequence annotations from Ensembl VEP. Format: Allele|Consequence|IMPACT|SYMBOL|Gene|Feature_type|Feature|BIOTYPE|EXON|INTRON|HGVSc|HGVSp|cDNA_position|CDS_position|Protein_position|Amino_acids|Codons|Existing_variation|DISTANCE|STRAND|FLAGS|SYMBOL_SOURCE|HGNC_ID|GENE_PHENO|SIFT|PolyPhen|AF|AFR_AF|AMR_AF|EAS_AF|EUR_AF|SAS_AF|gnomADe_AF|gnomADe_AFR_AF|gnomADe_AMR_AF|gnomADe_ASJ_AF|gnomADe_EAS_AF|gnomADe_FIN_AF|gnomADe_NFE_AF|gnomADe_OTH_AF|gnomADe_SAS_AF|CLIN_SIG|SOMATIC|PHENO&amp;quot;&amp;gt;&#xA;##VEP-command-line='vep --dir_cache workflow/resources/variants/cache_vep --vcf -i WESP_run/results/variant_calling/VP_DBS_25_7_S7.vcf -o WESP_run/results/variant_annotation/VP_DBS_25_7_S7.annotated.vcf.gz --af --af_1kg --af_gnomade --sift b --polyphen b --gene_phenotype --fork 24 --offline --compress_output gzip'&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  VP_DBS_25_7_S7&#xA;chr1    13273   .   G   C   25.3    PASS    CSQ=C|non_coding_transcript_exon_variant|MODIFIER|DDX11L1|ENSG00000223972|Transcript|ENST00000450305|transcribed_unprocessed_pseudogene|5/6||||313|||||rs531730856||1||HGNC|HGNC:37102||||0.0950|0.0204|0.1455|0.0625|0.1471|0.1401||||||||||||,C|non_coding_transcript_exon_variant|MODIFIER|DDX11L2|ENSG00000290825|Transcript|ENST00000456328|lncRNA|3/3||||521|||||rs531730856||1||EntrezGene|||||0.0950|0.0204|0.1455|0.0625|0.1471|0.1401||||||||||||,C|downstream_gene_variant|MODIFIER|WASH7P|ENSG00000227232|Transcript|ENST00000488147|unprocessed_pseudogene||||||||||rs531730856|1131|-1||HGNC|HGNC:38034||||0.0950|0.0204|0.1455|0.0625|0.1471|0.1401||||||||||||,C|downstream_gene_variant|MODIFIER|MIR6859-1|ENSG00000278267|Transcript|ENST00000619216|miRNA||||||||||rs531730856|4096|-1||HGNC|HGNC:50039||||0.0950|0.0204|0.1455|0.0625|0.1471|0.1401|||||||||||| GT:GQ:DP:AD:VAF:PL  0/1:23:55:28,27:0.490909:25,0,27&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For instance, at the first location/variant I look at the Ensembl gene id and see multiple genes on the same locations and strand. This is very strange to me, as I would understand different transcripts, but different genes on the same strand? Also, apart from that, the corresponding locations/regions of genes on the Ensembl website do not agree with the location in the vcf file.&#xA;For instance, the location for this SNP is 13273, but the annotations are (corresponding ensembl archive version 108):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Agrees: &lt;a href=&quot;http://oct2022.archive.ensembl.org/Homo_sapiens/Gene/Summary?db=core;g=ENSG00000223972;r=1:12010-13670;t=ENST00000450305&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://oct2022.archive.ensembl.org/Homo_sapiens/Gene/Summary?db=core;g=ENSG00000223972;r=1:12010-13670;t=ENST00000450305&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Agrees: &lt;a href=&quot;http://oct2022.archive.ensembl.org/Homo_sapiens/Gene/Summary?db=core;g=ENSG00000290825;r=1:11869-14409;t=ENST00000456328&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://oct2022.archive.ensembl.org/Homo_sapiens/Gene/Summary?db=core;g=ENSG00000290825;r=1:11869-14409;t=ENST00000456328&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Does not agree: &lt;a href=&quot;http://oct2022.archive.ensembl.org/Homo_sapiens/Gene/Summary?db=core;g=ENSG00000227232;r=1:14404-29570;t=ENST00000488147&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://oct2022.archive.ensembl.org/Homo_sapiens/Gene/Summary?db=core;g=ENSG00000227232;r=1:14404-29570;t=ENST00000488147&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;Does not agree: &lt;a href=&quot;http://oct2022.archive.ensembl.org/Homo_sapiens/Gene/Summary?db=core;g=ENSG00000278267;r=1:17369-17436;t=ENST00000619216&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://oct2022.archive.ensembl.org/Homo_sapiens/Gene/Summary?db=core;g=ENSG00000278267;r=1:17369-17436;t=ENST00000619216&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;What am I missing here?&lt;/p&gt;&#xA;&lt;p&gt;Kind regards,&#xA;Eva&lt;/p&gt;&#xA;" OwnerUserId="16204" LastEditorUserId="16204" LastEditDate="2023-11-06T13:36:44.240" LastActivityDate="2023-11-06T18:21:02.950" Title="VEP annotation INFO field Ensembl IDs and locations" Tags="|vcf|ensembl|identifiers|vep|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21819" PostTypeId="2" ParentId="21818" CreationDate="2023-11-06T18:21:02.950" Score="1" Body="&lt;p&gt;You need to set your expectations clearly instead of just saying &amp;quot;Does not agree&amp;quot;. I'm taking a guess here - your statement is that the 2/4 of the gene co-ordinates do not overlap the variant location.&lt;/p&gt;&#xA;&lt;p&gt;This is because you're viewing just part of the picture - the co-ordinates - and not seeing another important part, which is the kind of effect. For the latter two genes, VEP clearly says the effect is &lt;code&gt;downstream_gene_variant&lt;/code&gt;, which &lt;a href=&quot;http://www.sequenceontology.org/browser/current_svn/term/SO:0001632&quot; rel=&quot;nofollow noreferrer&quot;&gt;Sequence Ontology defines&lt;/a&gt; as &amp;quot;A sequence variant located 3' of a gene.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Putting those two pieces together, the entire picture makes sense. Variants affect different transcripts (of possibly different genes) in different manners. Looking at just 1 of those 2 components will of course make things seem odd.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-11-06T18:21:02.950" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21820" PostTypeId="1" CreationDate="2023-11-06T18:44:35.647" Score="0" ViewCount="68" Body="&lt;p&gt;I'm trying to use the HaplotypeCaller from the GATK toolkit but I keep getting an error. I pulled GATK through Docker and am using this command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker run -v /Users/rimo/ -it broadinstitute/gatk:latest gatk HaplotypeCaller -R /Users/rimo/reference.fasta -I /Users/rimo/sample1.bam -O /Users/rimo/sample1.g.vcf.gz -ERC GVCF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;code&gt;/Users/rimo&lt;/code&gt; is my home directory it's where the files are currently located on my computer and it's where I want GATK to look for files to use in the command.&lt;/p&gt;&#xA;&lt;p&gt;but then I get this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;A USER ERROR has occurred: The specified fasta file (file:///Users/rimo/reference.fasta) does not exist.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I don't fully understand how Docker commands work but I found a similar problem listed with this &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/21426/importing-local-files-to-a-gatk-docker-image&quot;&gt;issue&lt;/a&gt;. I tried to use their solution but it's still giving me the same error as listed above.&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="16067" LastEditDate="2023-11-28T18:13:59.723" LastActivityDate="2023-11-28T18:13:59.723" Title="How to run a GATK Docker Image with local files?" Tags="|variant-calling|gatk|docker|command-line|troubleshooting|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21821" PostTypeId="2" ParentId="21820" CreationDate="2023-11-06T21:03:51.473" Score="3" Body="&lt;p&gt;Read the manual please: &lt;a href=&quot;https://docs.docker.com/storage/bind-mounts/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.docker.com/storage/bind-mounts/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;code&gt;-v&lt;/code&gt; or &lt;code&gt;--volume&lt;/code&gt;: Consists of three fields, separated by colon characters (:). The fields must be in the correct order, and the meaning of each field is not immediately obvious.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;In the case of bind mounts, the first field is the path to the file or directory on the host machine.&lt;/li&gt;&#xA;&lt;li&gt;The second field is the path where the file or directory is mounted in the container.&lt;/li&gt;&#xA;&lt;li&gt;The third field is optional, and is a comma-separated list of options, such as ro, z, and Z. These options are discussed below.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Try&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker run -v /Users/rimo/:/myhome ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Also, why are you combining a &lt;code&gt;-it&lt;/code&gt; run with an actual command? I thought the &lt;code&gt;-it&lt;/code&gt; is to start an interactive shell, which is the opposite of running a single command.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2023-11-06T21:03:51.473" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21822" PostTypeId="2" ParentId="21820" CreationDate="2023-11-06T22:38:03.423" Score="0" Body="&lt;p&gt;I went back to the documentation listed on the GATK website instead of Docker because the Docker documentation was a bit confusing. And was able to figure it out with this command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;docker run -v ~/data:/gatk/my_data -it broadinstitute/gatk:latest&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And for those who might use this as a resource because they don't understand how to use local files on a Docker image the breakdown is as follows:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;~/data = path where the data is located on the local machine&lt;/li&gt;&#xA;&lt;li&gt;/gatk/my_data = path where the data will be located on the Docker image (folder will be created when the command is executed)&lt;/li&gt;&#xA;&lt;li&gt;this link saved me: &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035889991&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360035889991&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="16067" LastActivityDate="2023-11-06T22:38:03.423" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21823" PostTypeId="1" CreationDate="2023-11-07T12:17:25.300" Score="1" ViewCount="91" Body="&lt;p&gt;I would be thankful to you if you can help with how I can use MR from my case-control study to link DNA methylation (epic array) with toxic element exposure (arsenic) and health outcome (Cardiovascular diseases).&lt;/p&gt;&#xA;&lt;p&gt;I do have data from 125 Cardiovascular diseases and 125 non-Cardiovascular diseases individuals matched with age, sex, and smoking.I do have phenotypes files in a csv format having sex, age, smoking, disease, samples id, iAs (arsenic) values in log format and then a separate files for methylation data. we also have SNP data for these samples.&lt;/p&gt;&#xA;&lt;p&gt;Many thanks,&lt;/p&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="1008" LastEditDate="2023-11-08T12:17:07.493" LastActivityDate="2023-11-13T14:24:26.233" Title="Mendelian randomization" Tags="|r|statistics|epigenetics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21824" PostTypeId="1" CreationDate="2023-11-07T12:39:08.487" Score="1" ViewCount="31" Body="&lt;p&gt;This is kind of pedantic, but I'm not sure where to look...&lt;/p&gt;&#xA;&lt;p&gt;For GRCh38 (and a lot of work...) I have the following...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Chr          Length         Ns   ns&#xA;chr1    248,956,422 18,475,229  181&#xA;chr2    242,193,529   1645,291   10&#xA;chr3    198,295,559    195,420    4&#xA;chr4    190,214,555    461,888    0&#xA;chr5    181,538,259    272,881    0&#xA;chr6    170,805,979    727,255  202&#xA;chr7    159,345,973    375,841    1&#xA;chr8    145,138,636    370,500    0&#xA;chr9    138,394,717 16,604,127   40&#xA;chr10   133,797,422    534,287  173&#xA;chr11   135,086,622    552,880    0&#xA;chr12   133,275,309    137,270  223&#xA;chr13   114,364,328 163,81,200    3&#xA;chr14   107,043,718 164,75,569    0&#xA;chr15   101,991,189 173,49,841   23&#xA;chr16    90,338,345   8532,401    1&#xA;chr17    83,257,441    336,367  870&#xA;chr18    80,373,285    283,627   53&#xA;chr19    58,617,616    176,858    0&#xA;chr20    64,444,167    499,586  324&#xA;chr21    46,709,983  6,621,300   64&#xA;chr22    50,818,468 11,658,691    0&#xA;chrX    156,040,895  1,147,765  101&#xA;chrY     57,227,415 30,812,232  140&#xA;chrM         16,569          1    0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(Where) could I have looked this up?&lt;/p&gt;&#xA;&lt;p&gt;What are the counts for GRCh37?&lt;/p&gt;&#xA;" OwnerUserId="3233" LastEditorUserId="650" LastEditDate="2023-11-07T15:20:46.600" LastActivityDate="2023-11-08T15:45:49.903" Title="How many Ns and ns in GRCh37 / GRCh38 per 'canonical' chromosome?" Tags="|human-genome|repeat|grch37|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21825" PostTypeId="2" ParentId="21823" CreationDate="2023-11-07T14:11:40.197" Score="1" Body="&lt;p&gt;I understand linkage disequilibrium (LD) versus panmixia, which the technical term for Mendelian randomisation and for humans PLINK is a population package. The best outcome is a correlation between LD and cardiovascular disease. It is however an unlikely outcome.&lt;/p&gt;&#xA;&lt;p&gt;This really is a machine learning question and you would need to understand &amp;quot;feature selection&amp;quot; and will require very high accuracy scores &amp;gt;0.95 - however you could triple the size of our control group quite easily (recommended). This is a separate question and I'm happy to answer this there.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Just to address the comments. Inheritance probability, if that is the intended statistics, I personally don't think it would work because a 125 sample is low and the patients would need to be in cohorts of familial generations. LD is unlikely the explanation with humans but other non-humans species that might work (selection advantage).&lt;/p&gt;&#xA;&lt;p&gt;My personal view is this is a straight machine learning calculation, but it's a separate question. If your sample is heavily stratified across each familial generation then Hardy-Weinberg might work, but my experience of patient cohorts is that is really unusual and there could still be confounders. Supervised learning (subset of ML) is 100% the way forward.&lt;/p&gt;&#xA;&lt;p&gt;.... possibly use GWAS, but you've multiple features/training targets. GWAS is usually when there's one phenotype, ML would account for interaction in a way GWAS could not (it couldn't account for non-sequence features, i.e. mixed data).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-11-13T14:24:26.233" LastActivityDate="2023-11-13T14:24:26.233" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="21826" PostTypeId="1" CreationDate="2023-11-07T16:21:26.467" Score="-2" ViewCount="84" Body="&lt;p&gt;I have list of samples as a txt file and want to only keep these samples from this large file; column name is samples name in the large file.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(readr)&#xA;library(data.table)&#xA;&#xA;df1 &amp;lt;- read_csv(&amp;quot;control.csv&amp;quot;, col_types = cols(Chromosome = col_skip(), Start = col_skip(), End = col_skip(), Strand = col_skip()))&#xA;&#xA;df1[1:5,1:5]&#xA;# A tibble: 5 × 5&#xA;  ID          DN54 DC146 DDN44 DC179&#xA;  &amp;lt;chr&amp;gt;      &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;&#xA;1 cg26928153 0.848 0.879 0.872 0.897&#xA;2 cg16269199 0.731 0.732 0.759 0.775&#xA;3 cg13869341 0.842 0.863 0.819 0.896&#xA;4 cg24669183 0.824 0.866 0.822 0.910&#xA;5 cg26679879 0.425 0.363 0.420 0.358&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="650" LastEditDate="2023-11-13T16:37:58.120" LastActivityDate="2023-11-13T16:37:58.120" Title="Filtering columns from large file based on a column list" Tags="|r|bash|" AnswerCount="3" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21827" PostTypeId="2" ParentId="21826" CreationDate="2023-11-07T18:53:40.147" Score="3" Body="&lt;p&gt;You can save time and memory by preprocessing your file before loading it into R.&lt;/p&gt;&#xA;&lt;p&gt;Use &lt;a href=&quot;https://miller.readthedocs.io/&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;miller&lt;/code&gt;&lt;/a&gt; toolkit to manipulate csv or tab-delimited files using column names.&lt;/p&gt;&#xA;&lt;p&gt;Below is a one-liner that reads csv file, and writes columns specified in option &lt;code&gt;-f&lt;/code&gt;, in the order (&lt;code&gt;-o&lt;/code&gt;) listed on the command line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mlr --csv cut -o -f col3,col1 in.csv &amp;gt; out.csv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To read the list of columns from a file, 1 column name per line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mlr --csv cut -o -f $(paste -s -d, columns.txt) in.csv &amp;gt; out.csv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;See also:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Note that &lt;a href=&quot;https://github.com/johnkerl/miller&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;miller&lt;/code&gt;&lt;/a&gt; can be easily installed, for example, using &lt;a href=&quot;https://conda.io/projects/conda/en/latest/user-guide/install/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;conda&lt;/code&gt;&lt;/a&gt;, specifically &lt;code&gt;miniconda&lt;/code&gt;, like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda create --channel conda-forge --name miller miller&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6251" LastActivityDate="2023-11-07T18:53:40.147" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21828" PostTypeId="1" CreationDate="2023-11-07T20:02:45.537" Score="0" ViewCount="12" Body="&lt;p&gt;I am trying to identify the flux of IgG antibody in CHO ovary cells with optimised biomass and glucose input ranging from 0 to 10, but I am not sure when I run the FBA analysis what should I define my lower and upper bounds for glucose.&lt;/p&gt;&#xA;" OwnerUserId="18509" LastActivityDate="2023-11-07T20:02:45.537" Title="Metabolic Modeling" Tags="|biopython|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21829" PostTypeId="2" ParentId="21826" CreationDate="2023-11-08T05:51:01.347" Score="3" Body="&lt;p&gt;It's really just &lt;code&gt;count[,col$V1]&lt;/code&gt;. Please learn about basic subsetting of columns and rows, it's simple yet powerful.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-11-08T05:51:01.347" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21830" PostTypeId="1" CreationDate="2023-11-08T08:29:13.760" Score="0" ViewCount="25" Body="&lt;p&gt;I was trying to run bases2fastq software downloaded from the element bioscience platform. I was running this software through WDL workflow named list_fastq. For some reason, maybe syntax, my cromwell execution failed. However, I haven't been able to read the logs as they are empty. Below is the content of the 'stderr.background' file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mkfifo: cannot create fifo '/mnt/d/element-bioscience/cromwell-executions/list_fastq/5e9a780d-c010-41f7-bba0-bcb42c870891/call-list_samples/tmp.05c20794/out.78994': Operation not supported&#xA;mkfifo: cannot create fifo '/mnt/d/element-bioscience/cromwell-executions/list_fastq/5e9a780d-c010-41f7-bba0-bcb42c870891/call-list_samples/tmp.05c20794/err.78994': Operation not supported&#xA;/mnt/d/element-bioscience/cromwell-executions/list_fastq/5e9a780d-c010-41f7-bba0-bcb42c870891/call-list_samples/execution/script: line 17: /mnt/d/element-bioscience/cromwell-executions/list_fastq/5e9a780d-c010-41f7-bba0-bcb42c870891/call-list_samples/tmp.05c20794/out.78994: No such file or directory&#xA;/mnt/d/element-bioscience/cromwell-executions/list_fastq/5e9a780d-c010-41f7-bba0-bcb42c870891/call-list_samples/execution/script: line 18: /mnt/d/element-bioscience/cromwell-executions/list_fastq/5e9a780d-c010-41f7-bba0-bcb42c870891/call-list_samples/tmp.05c20794/err.78994: No such file or directory&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Please guide me to the steps on being able to prevent cromwell from deleting the contents of log files, if that's what's happening.&lt;/p&gt;&#xA;" OwnerUserId="18513" LastActivityDate="2023-11-08T08:29:13.760" Title="Unable to access error logs of failed cromwell execution (WDL) in local machine running in ubuntu environment" Tags="|errors|wdl|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21831" PostTypeId="2" ParentId="21824" CreationDate="2023-11-08T13:53:17.557" Score="1" Body="&lt;p&gt;I don't know where you could look this up and of course this will depend on the versions of the relevant genomes and on which exact assemblies you are working with. I doubt you would have found this specific number, how many Ns anywhere, but it's really not very hard to get them. For instance, I have hg19 and hg38 versions locally:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ for genome in hg19 hg38 ; do ls $genome/$genome.fa; done&#xA;hg19/hg19.fa&#xA;hg38/hg38.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This means I can get the Ns for both using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for genome in hg19 hg38 ; do &#xA;  FastaToTbl &amp;quot;$genome&amp;quot;/$genome.fa | &#xA;    perl -ne 'print if /^(chr)?[0-9MXY]*\s/' | &#xA;      awk -F&amp;quot;\t&amp;quot; '{ print $1, split($2,a,/[nN]/)}' &amp;gt; $genome.ns&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And then combine into a nice table with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ printf 'chr\thg19\thg38\n-----------------------\n'; &#xA;  join -j1 -t ' ' &amp;lt;( sed 's/chr//' hg19 | sort -gk1,1 ) \&#xA;                  &amp;lt;( sed 's/chr//'  hg38 | sort -gk1,1) | &#xA;                   column -t | sed 's/^/chr/'&#xA;chr hg19    hg38&#xA;-----------------------&#xA;chrM   1         2&#xA;chrX   4170001   1147867&#xA;chrY   33720001  30812373&#xA;chr1   23970001  18475411&#xA;chr2   4994856   1645302&#xA;chr3   3225296   195425&#xA;chr4   3492601   461889&#xA;chr5   3220001   272882&#xA;chr6   3720002   727458&#xA;chr7   3785001   375843&#xA;chr8   3475101   370501&#xA;chr9   21070001  16604168&#xA;chr10  4220010   534461&#xA;chr11  3877001   552881&#xA;chr12  3370503   137494&#xA;chr13  19580001  16381204&#xA;chr14  19060001  16475570&#xA;chr15  20836627  17349865&#xA;chr16  11470001  8532403&#xA;chr17  3400001   337238&#xA;chr18  3420020   283681&#xA;chr19  3320001   176859&#xA;chr20  3520001   499911&#xA;chr21  13023254  6621365&#xA;chr22  16410022  11658692&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The fancy sed stuff is just to ensure that sorting works correctly by removing and then re-adding the &lt;code&gt;chr&lt;/code&gt; string. Note that this is counting both &lt;code&gt;N&lt;/code&gt; and &lt;code&gt;n&lt;/code&gt;. The FastaToTbl script can be found &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2651/298&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2023-11-08T15:45:49.903" LastActivityDate="2023-11-08T15:45:49.903" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21832" PostTypeId="1" CreationDate="2023-11-08T14:26:17.710" Score="0" ViewCount="20" Body="&lt;p&gt;Given a specific &lt;code&gt;PyRanges&lt;/code&gt; object, such as&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pyranges as pr&#xA;gr = pr.from_dict({&amp;quot;Chromosome&amp;quot;: [1, 1, 1, 1], &amp;quot;Start&amp;quot;: [1, 25000, 30000, 40000],&#xA;                   &amp;quot;End&amp;quot;: [24000, 29500, 37000, 45000], &amp;quot;ratio&amp;quot;: [0.5, 0.3, 0.5, 0.4],&#xA;                   &amp;quot;ID&amp;quot;: (0,1,2,3), &amp;quot;Strand&amp;quot;: &amp;quot;+&amp;quot;})&#xA;&#xA;gr&#xA;&#xA;+--------------+-----------+-----------+-------------+-----------+--------------+&#xA;|   Chromosome |     Start |       End |       ratio |        ID | Strand       |&#xA;|   (category) |   (int64) |   (int64) |   (float64) |   (int64) | (category)   |&#xA;|--------------+-----------+-----------+-------------+-----------+--------------|&#xA;|            1 |         1 |     24000 |         0.5 |         0 | +            |&#xA;|            1 |     25000 |     29500 |         0.3 |         1 | +            |&#xA;|            1 |     30000 |     37000 |         0.5 |         2 | +            |&#xA;|            1 |     40000 |     45000 |         0.4 |         3 | +            |&#xA;+--------------+-----------+-----------+-------------+-----------+--------------+&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to get rid of one interval here (not necessarily the first or the last: for this exercise, for example the third one, &lt;em&gt;without&lt;/em&gt; relying on the presence of the &lt;code&gt;ID&lt;/code&gt; column.  So I would need to drop the record with &lt;code&gt;Chromosome&lt;/code&gt; 1, &lt;code&gt;Start&lt;/code&gt; 30000 and &lt;code&gt;End&lt;/code&gt; 37000. The use case is that I need to replace it with a different record.&lt;/p&gt;&#xA;&lt;p&gt;Replacement of the record is somewhat expensive but not a problem (&lt;code&gt;pr.concat&lt;/code&gt; followed by a sort to put everything in order), but I can't figure, after reading docstrings and searching around, how to &lt;em&gt;remove&lt;/em&gt; a record.&lt;/p&gt;&#xA;&lt;p&gt;I can only think of making the record I want to drop a second &lt;code&gt;PyRanges&lt;/code&gt; object then using &lt;code&gt;overlap()&lt;/code&gt; with &lt;code&gt;invert=True&lt;/code&gt; to remove the record. Is there a more elegant and less convoluted way to do this?&lt;/p&gt;&#xA;" OwnerUserId="6814" LastEditorUserId="650" LastEditDate="2023-11-08T16:16:03.887" LastActivityDate="2023-11-08T16:16:03.887" Title="PyRanges: drop specific record from an object" Tags="|python|pyranges|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21833" PostTypeId="1" AcceptedAnswerId="21847" CreationDate="2023-11-08T14:50:49.387" Score="3" ViewCount="73" Body="&lt;p&gt;I want to retrieve a sequence for many species from the Nucleotide database in NCBI.&lt;/p&gt;&#xA;&lt;p&gt;I'm using a command line approach and I have to figure out what is the best query that will return exclusively the region of DNA I am interested in and filter out unwanted noisy sequences.&lt;/p&gt;&#xA;&lt;p&gt;I am using a wide range of species (that I have stored as a list of TaxIDs of about 2000 species): it includes small crustaceans, invertebrates, some algae and small vertebrates (reptiles, amphibians and few mammals).&lt;/p&gt;&#xA;&lt;p&gt;My final goal is to obtain a phylogenetic tree for all or most of the species.&lt;/p&gt;&#xA;&lt;p&gt;I have been suggested to use these genes to create such phylogeny&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;CO1 mithocondrial DNA&lt;/li&gt;&#xA;&lt;li&gt;16S rRNA&lt;/li&gt;&#xA;&lt;li&gt;18S rRNA&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I want to formulate a specific query that will return exclusively those sequences.&lt;/p&gt;&#xA;&lt;p&gt;I'm using the GenBank query builder to visually check for the accuracy of my search and when I find a good query I will use it in the API.&lt;/p&gt;&#xA;&lt;p&gt;So far I came up with the following queries:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;(COX1[Title] OR CO1[Title]) AND complete[Title]&lt;/code&gt; --&amp;gt; 63/2000 Species&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;16S[title] AND complete[title] AND rRNA[title] NOT partial[title]&lt;/code&gt; --&amp;gt; 9/2000 species&lt;/li&gt;&#xA;&lt;li&gt;&lt;code&gt;18S[title] AND complete[title] AND rRNA[title] NOT partial[title]&lt;/code&gt; --&amp;gt; 15/2000 species&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;As you can see the number of species that I get sequence for is very low compared to the initial 2000 species. I doubt that we have so little available sequences (especially for COX1 that is used for barcoding)&lt;/p&gt;&#xA;&lt;p&gt;Can you help me understanding whether my queries are good or not? And if possible suggest a better alternative&lt;/p&gt;&#xA;&lt;h3&gt;More Info&lt;/h3&gt;&#xA;&lt;p&gt;A subset of 10 of my species of interest is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Rasbora heteromorpha&#xA;Elasmopus rapax&#xA;Gasterosteus aculeatus&#xA;Palaemonetes pugio&#xA;Catostomus commersoni&#xA;Daphnia magna&#xA;Oryzias latipes&#xA;Xenopus laevis&#xA;Tigriopus japonicus&#xA;Oncorhynchus mykiss&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Out of these 10 only 2 have sequences available using the COX1 query&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&#xA;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;Name&lt;/th&gt;&#xA;&lt;th&gt;SeqID&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Oncorhynchus mykiss&lt;/td&gt;&#xA;&lt;td&gt;EU186789.1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Xenopus laevis&lt;/td&gt;&#xA;&lt;td&gt;AB278691.1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&#xA;&lt;/div&gt;&#xA;&lt;p&gt;But Daphnia magna is one of the most commonly used organisms in lab and I found &lt;a href=&quot;https://www.tandfonline.com/doi/full/10.1080/23802359.2020.1750995&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; regarding the complete mitogenome of a specific strain of D. magna thus implying that the complete mitogenome of the species is already known. That means that there must be a way to retrieve all the other species that have full mitogenome but no COX1 gene mentioned in their GenBank title.&lt;/p&gt;&#xA;" OwnerUserId="18351" LastEditorUserId="18351" LastEditDate="2023-11-09T16:38:30.530" LastActivityDate="2023-11-13T16:02:07.980" Title="What is the best Query to retrieve DNA from NCBI?" Tags="|phylogenetics|phylogeny|sequence-annotation|genbank|barcode|" AnswerCount="2" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21834" PostTypeId="1" AcceptedAnswerId="21836" CreationDate="2023-11-08T18:37:52.833" Score="2" ViewCount="386" Body="&lt;p&gt;I did some calculations using a PDB file as input. As a result, I got a file with the atomic partial charges (only the values in one column, no other information) and the output PDB file structure without the charge information.&lt;/p&gt;&#xA;&lt;p&gt;My question: Is there a tool to add the charges to the output PDB file using the separate file?&lt;/p&gt;&#xA;&lt;p&gt;My PDB file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    TITLE     sistema05&#xA;    REMARK   4      COMPLIES WITH FORMAT V. 3.0, 1-DEC-2006&#xA;    REMARK 888&#xA;    REMARK 888 WRITTEN BY MAESTRO (A PRODUCT OF SCHRODINGER, LLC)&#xA;    HETATM    1  C1  BAS A   1      10.151  -3.071  -1.881  1.00  0.00           C  &#xA;    HETATM    2  C2  BAS A   1       9.389  -3.683  -0.874  1.00  0.00           C  &#xA;    HETATM    3  C3  BAS A   1       9.976  -3.936   0.382  1.00  0.00           C  &#xA;    HETATM    4  C4  BAS A   1       9.194  -3.742   1.530  1.00  0.00           C  &#xA;    HETATM    5  C5  BAS A   1       9.781  -3.201   2.695  1.00  0.00           C  &#xA;    HETATM    6  C6  BAS A   1       9.029  -2.357   3.510  1.00  0.00           C  &#xA;    HETATM    7  C7  BAS A   1       9.651  -1.313   4.236  1.00  0.00           C  &#xA;    HETATM    8  C8  BAS A   1       8.948  -0.139   4.488  1.00  0.00           C  &#xA;    HETATM    9  C9  BAS A   1       9.635   1.100   4.502  1.00  0.00           C  &#xA;    HETATM   10  C10 BAS A   1       9.001   2.259   4.094  1.00  0.00           C  &#xA;    HETATM   11  C11 BAS A   1       9.724   3.177   3.329  1.00  0.00           C  &#xA;    HETATM   12  C12 BAS A   1       9.148   3.815   2.228  1.00  0.00           C  &#xA;    HETATM   13  C13 BAS A   1       9.909   3.945   1.081  1.00  0.00           C  &#xA;    HETATM   14  C14 BAS A   1       9.332   3.754  -0.198  1.00  0.00           C  &#xA;    HETATM   15  C15 BAS A   1      10.098   3.266  -1.245  1.00  0.00           C  &#xA;    HETATM   16  C16 BAS A   1       9.501   2.445  -2.232  1.00  0.00           C  &#xA;    HETATM   17  C17 BAS A   1      10.234   1.431  -2.847  1.00  0.00           C  &#xA;    HETATM   18  C18 BAS A   1       9.575   0.238  -3.215  1.00  0.00           C  &#xA;    HETATM   19  C19 BAS A   1      10.249  -0.986  -3.121  1.00  0.00           C  &#xA;    HETATM   20  C20 BAS A   1       9.529  -2.124  -2.713  1.00  0.00           C  &#xA;...&#xA;...&#xA;...  &#xA;    CONECT    1  212    2   20&#xA;    CONECT    1   20&#xA;    CONECT    2    1    3   21&#xA;    CONECT    2   21&#xA;    CONECT    3    2  213    4&#xA;    CONECT    3    4&#xA;    CONECT    4    3    5   23&#xA;    CONECT    4    3&#xA;    CONECT    5    4  214    6&#xA;    CONECT    5    6&#xA;    CONECT    6    5    7   25&#xA;    CONECT    6    5&#xA;    CONECT    7    6  215    8&#xA;    CONECT    7    8&#xA;    CONECT    8    7    9   27&#xA;    CONECT    8    7&#xA;    CONECT    9    8  216   10&#xA;    CONECT   10    9   11   29&#xA;    CONECT   11   10  217   12&#xA;    CONECT   12   11   13   31&#xA;    CONECT   13   12  218   14&#xA;    CONECT   14   13   15   33&#xA;    CONECT   14   15&#xA;    CONECT   15   14  219   16&#xA;    CONECT   15   14&#xA;    CONECT   16   15   17   35&#xA;    CONECT   16   17&#xA;    CONECT   17   16  220   18&#xA;    CONECT   17   16&#xA;    CONECT   18   17   19   37&#xA;    CONECT   18   19&#xA;    CONECT   19   18  221   20&#xA;    CONECT   19   18&#xA;    CONECT   20    1   19   39&#xA;    CONECT   20    1&#xA;    CONECT   21    2   22   40&#xA;    END   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My CHARGE file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;   -0.02474906&#xA;    0.02291960&#xA;   -0.02789302&#xA;    0.02227110&#xA;   -0.02831805&#xA;    0.02230034&#xA;   -0.02876149&#xA;    0.02225527&#xA;   -0.02774076&#xA;    0.02216971&#xA;   -0.02074528&#xA;    0.02275309&#xA;   -0.02662750&#xA;    0.02177531&#xA;   -0.03007329&#xA;    0.02283631&#xA;   -0.02779055&#xA;    0.02312780&#xA;   -0.02312680&#xA;    0.02315866   &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9085" LastEditorUserId="9085" LastEditDate="2023-11-08T20:57:08.097" LastActivityDate="2023-11-27T19:54:06.120" Title="How to add charges to a PDB file?" Tags="|pdb|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21835" PostTypeId="2" ParentId="15591" CreationDate="2023-11-08T21:26:45.220" Score="1" Body="&lt;p&gt;I think the term &amp;quot;breakend&amp;quot; is derived specifically from the VCF file spec:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.2.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://samtools.github.io/hts-specs/VCFv4.2.pdf&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Starting on page 12:&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;5.4 Specifying complex rearrangements with breakends&lt;/p&gt;&#xA;&lt;p&gt;An arbitrary rearrangement event can be summarized as a set of novel adjacencies. Each adjacency ties together 2 breakends. The two breakends at either end of a novel adjacency are called mates.&lt;/p&gt;&#xA;&lt;p&gt;There is one line of VCF (i.e. one record) for each of the two breakends in a novel adjacency. A breakend record is identified with the tag “SVTYPE=BND” in the INFO field. The REF field of a breakend record indicates a base or&#xA;sequence s of bases beginning at position POS, as in all VCF records. The ALT field of a breakend record indicates a replacement for s. This “breakend replacement” has three parts:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;The string t that replaces places s. The string t may be an extended version of s if some novel bases are inserted during the formation of the novel adjacency.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The position p of the mate breakend, indicated by a string of the form “chr:pos”. This is the location of the first mapped base in the piece being joined at this novel adjacency.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;The direction that the joined sequence continues in, starting from p. This is indicated by the orientation of square brackets surrounding p. These 3 elements are combined in 4 possible ways to create the ALT. In each of the 4 cases, the assertion is that s is replaced with t, and then some piece starting at position p is joined to t. The cases are:&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;REF ALT Meaning&lt;/p&gt;&#xA;&lt;p&gt;s t[p[ piece extending to the right of p is joined after t&lt;/p&gt;&#xA;&lt;p&gt;s t]p] reverse comp piece extending left of p is joined after t&lt;/p&gt;&#xA;&lt;p&gt;s ]p]t piece extending to the left of p is joined before t&lt;/p&gt;&#xA;&lt;p&gt;s [p[t reverse comp piece extending right of p is joined before t&lt;/p&gt;&#xA;&lt;p&gt;The example in Figure 1 shows a 3-break operation involving 6 breakends. It exemplifies all possible orientations of breakends in adjacencies. Notice how the ALT field expresses the orientation of the breakends.&lt;/p&gt;&#xA;&lt;p&gt;#CHROM POS ID REF ALT QUAL FILTER INFO&lt;/p&gt;&#xA;&lt;p&gt;2 321681 bnd W G G]17:198982] 6 PASS SVTYPE=BND&lt;/p&gt;&#xA;&lt;p&gt;2 321682 bnd V T ]13:123456]T 6 PASS SVTYPE=BND&lt;/p&gt;&#xA;&lt;p&gt;13 123456 bnd U C C[2:321682[ 6 PASS SVTYPE=BND&lt;/p&gt;&#xA;&lt;p&gt;13 123457 bnd X A [17:198983[A 6 PASS SVTYPE=BND&lt;/p&gt;&#xA;&lt;p&gt;17 198982 bnd Y A A]2:321681] 6 PASS SVTYPE=BND&lt;/p&gt;&#xA;&lt;p&gt;17 198983 bnd Z C [13:123457[C 6 PASS SVTYPE=BND&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="18518" LastActivityDate="2023-11-08T21:26:45.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21836" PostTypeId="2" ParentId="21834" CreationDate="2023-11-09T01:26:18.040" Score="3" Body="&lt;p&gt;OK given , &lt;code&gt;input_connect.pdb&lt;/code&gt; , using &lt;strong&gt;Biopython&lt;/strong&gt; library&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TITLE     sistema05&#xA;REMARK   4      COMPLIES WITH FORMAT V. 3.0, 1-DEC-2006&#xA;REMARK 888&#xA;REMARK 888 WRITTEN BY MAESTRO (A PRODUCT OF SCHRODINGER, LLC)&#xA;HETATM    1  C1  BAS A   1      10.151  -3.071  -1.881  1.00  0.00           C  &#xA;HETATM    2  C2  BAS A   1       9.389  -3.683  -0.874  1.00  0.00           C  &#xA;HETATM    3  C3  BAS A   1       9.976  -3.936   0.382  1.00  0.00           C  &#xA;HETATM    4  C4  BAS A   1       9.194  -3.742   1.530  1.00  0.00           C  &#xA;HETATM    5  C5  BAS A   1       9.781  -3.201   2.695  1.00  0.00           C  &#xA;HETATM    6  C6  BAS A   1       9.029  -2.357   3.510  1.00  0.00           C  &#xA;HETATM    7  C7  BAS A   1       9.651  -1.313   4.236  1.00  0.00           C  &#xA;HETATM    8  C8  BAS A   1       8.948  -0.139   4.488  1.00  0.00           C  &#xA;HETATM    9  C9  BAS A   1       9.635   1.100   4.502  1.00  0.00           C  &#xA;HETATM   10  C10 BAS A   1       9.001   2.259   4.094  1.00  0.00           C  &#xA;HETATM   11  C11 BAS A   1       9.724   3.177   3.329  1.00  0.00           C  &#xA;HETATM   12  C12 BAS A   1       9.148   3.815   2.228  1.00  0.00           C  &#xA;HETATM   13  C13 BAS A   1       9.909   3.945   1.081  1.00  0.00           C  &#xA;HETATM   14  C14 BAS A   1       9.332   3.754  -0.198  1.00  0.00           C  &#xA;HETATM   15  C15 BAS A   1      10.098   3.266  -1.245  1.00  0.00           C  &#xA;HETATM   16  C16 BAS A   1       9.501   2.445  -2.232  1.00  0.00           C  &#xA;HETATM   17  C17 BAS A   1      10.234   1.431  -2.847  1.00  0.00           C  &#xA;HETATM   18  C18 BAS A   1       9.575   0.238  -3.215  1.00  0.00           C  &#xA;HETATM   19  C19 BAS A   1      10.249  -0.986  -3.121  1.00  0.00           C  &#xA;HETATM   20  C20 BAS A   1       9.529  -2.124  -2.713  1.00  0.00           C&#xA;CONECT    1  212    2   20&#xA;CONECT    1   20&#xA;CONECT    2    1    3   21&#xA;CONECT    2   21&#xA;CONECT    3    2  213    4&#xA;CONECT    3    4&#xA;CONECT    4    3    5   23&#xA;CONECT    4    3&#xA;CONECT    5    4  214    6&#xA;CONECT    5    6&#xA;CONECT    6    5    7   25&#xA;CONECT    6    5&#xA;CONECT    7    6  215    8&#xA;CONECT    7    8&#xA;CONECT    8    7    9   27&#xA;CONECT    8    7&#xA;CONECT    9    8  216   10&#xA;CONECT   10    9   11   29&#xA;CONECT   11   10  217   12&#xA;CONECT   12   11   13   31&#xA;CONECT   13   12  218   14&#xA;CONECT   14   13   15   33&#xA;CONECT   14   15&#xA;CONECT   15   14  219   16&#xA;CONECT   15   14&#xA;CONECT   16   15   17   35&#xA;CONECT   16   17&#xA;CONECT   17   16  220   18&#xA;CONECT   17   16&#xA;CONECT   18   17   19   37&#xA;CONECT   18   19&#xA;CONECT   19   18  221   20&#xA;CONECT   19   18&#xA;CONECT   20    1   19   39&#xA;CONECT   21    2   22   40&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and charges list &lt;code&gt;input_charge.txt&lt;/code&gt; :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-0.02474906&#xA; 0.02291960&#xA;-0.02789302&#xA; 0.02227110&#xA;-0.02831805&#xA; 0.02230034&#xA;-0.02876149&#xA; 0.02225527&#xA;-0.02774076&#xA; 0.02216971&#xA;-0.02074528&#xA; 0.02275309&#xA;-0.02662750&#xA; 0.02177531&#xA;-0.03007329&#xA; 0.02283631&#xA;-0.02779055&#xA; 0.02312780&#xA;-0.02312680&#xA; 0.02315866&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;with code :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import Bio&#xA;print('########## Biopython  VERSION ##########################################')&#xA;print('\n\n BIOPYTHON_VERSION : ', Bio.__version__)&#xA;print('###################################################################')&#xA;&#xA;from Bio.PDB import (&#xA;                    PDBParser,&#xA;                    Atom,&#xA;                    &#xA;                    )&#xA;                    &#xA;                    # PDBIO)&#xA;&#xA;import PDBIOcopy&#xA;&#xA;# from Bio.PDB import  PDBIO &#xA;&#xA;pdb = 'input'&#xA;&#xA;charges = 'input_charge.txt'&#xA;&#xA;charges_list = []&#xA;&#xA;with open(charges , 'r') as handle :&#xA;    &#xA;    for i in handle.readlines() :&#xA;&#xA;        charges_list.append(i.strip())&#xA;        &#xA;print(charges_list)&#xA;&#xA;parser = PDBParser(PERMISSIVE=True , QUIET = False)&#xA;&#xA;&#xA;structure = parser.get_structure(pdb, pdb+'.pdb')&#xA;&#xA;&#xA;atoms = structure.get_atoms()&#xA;&#xA;&#xA;idx = 0&#xA;&#xA;for atom in atoms :&#xA;    &#xA;    print(atom)&#xA;    &#xA;    atom.set_charge(charges_list[idx])&#xA;    &#xA;    idx += 1&#xA;    &#xA;&#xA;atoms = structure.get_atoms()&#xA;&#xA;for atom in atoms :&#xA;    &#xA;    print(atom , atom.get_charge())&#xA;    &#xA;&#xA;struct_to_save = PDBIOcopy.PDBIO()&#xA;&#xA;# struct_to_save = PDBIO()&#xA;&#xA;struct_to_save.set_structure(structure)&#xA;&#xA;struct_to_save.save('result_input.pdb')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and modifying &lt;code&gt;Bio.PDB.PDBIO&lt;/code&gt; as :&lt;/p&gt;&#xA;&lt;p&gt;line 22 :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;_ATOM_FORMAT_STRING = (&#xA;    &amp;quot;%s%5i %-4s%c%3s %c%4i%c   %8.3f%8.3f%8.3f%s%6.2f      %4s%2s%12s\n&amp;quot;&#xA;)  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;line 164 :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# charge=&amp;quot;  &amp;quot;, #deleted&#xA;pqr_charge     # insertes&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;line 213:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;charge = atom.pqr_charge  #inserted&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;line 389:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;atom.pqr_charge,  #inserted&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;output &lt;code&gt;result_input.pdb&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;HETATM    1  C1  BAS A   1      10.151  -3.071  -1.881  1.00  0.00           C -0.02474906&#xA;HETATM    2  C2  BAS A   1       9.389  -3.683  -0.874  1.00  0.00           C  0.02291960&#xA;HETATM    3  C3  BAS A   1       9.976  -3.936   0.382  1.00  0.00           C -0.02789302&#xA;HETATM    4  C4  BAS A   1       9.194  -3.742   1.530  1.00  0.00           C  0.02227110&#xA;HETATM    5  C5  BAS A   1       9.781  -3.201   2.695  1.00  0.00           C -0.02831805&#xA;HETATM    6  C6  BAS A   1       9.029  -2.357   3.510  1.00  0.00           C  0.02230034&#xA;HETATM    7  C7  BAS A   1       9.651  -1.313   4.236  1.00  0.00           C -0.02876149&#xA;HETATM    8  C8  BAS A   1       8.948  -0.139   4.488  1.00  0.00           C  0.02225527&#xA;HETATM    9  C9  BAS A   1       9.635   1.100   4.502  1.00  0.00           C -0.02774076&#xA;HETATM   10  C10 BAS A   1       9.001   2.259   4.094  1.00  0.00           C  0.02216971&#xA;HETATM   11  C11 BAS A   1       9.724   3.177   3.329  1.00  0.00           C -0.02074528&#xA;HETATM   12  C12 BAS A   1       9.148   3.815   2.228  1.00  0.00           C  0.02275309&#xA;HETATM   13  C13 BAS A   1       9.909   3.945   1.081  1.00  0.00           C -0.02662750&#xA;HETATM   14  C14 BAS A   1       9.332   3.754  -0.198  1.00  0.00           C  0.02177531&#xA;HETATM   15  C15 BAS A   1      10.098   3.266  -1.245  1.00  0.00           C -0.03007329&#xA;HETATM   16  C16 BAS A   1       9.501   2.445  -2.232  1.00  0.00           C  0.02283631&#xA;HETATM   17  C17 BAS A   1      10.234   1.431  -2.847  1.00  0.00           C -0.02779055&#xA;HETATM   18  C18 BAS A   1       9.575   0.238  -3.215  1.00  0.00           C  0.02312780&#xA;HETATM   19  C19 BAS A   1      10.249  -0.986  -3.121  1.00  0.00           C -0.02312680&#xA;HETATM   20  C20 BAS A   1       9.529  -2.124  -2.713  1.00  0.00           C  0.02315866&#xA;TER      21      BAS A   1                                                       &#xA;END  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That unfortunately is not a pdb file format anymore:&lt;/p&gt;&#xA;&lt;p&gt;PDB Format uses for RECORD ATOM CHARGE columns 79-80 , see &lt;a href=&quot;https://www.wwpdb.org/documentation/file-format-content/format33/v3.3.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Protein Data Bank Contents Guide: Atomic Coordinate Entry Format Description&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;At the end of all of this I realized that using python and just adding the charge values at the end of the pdb file read line by line would be a lot easier &amp;amp; faster but anyway I had the chance to explore &lt;code&gt;PDBIO&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you want to add the charges in the &lt;code&gt;bfactor&lt;/code&gt; or &lt;code&gt;occupancy&lt;/code&gt; columns just use :&lt;code&gt;atom.set_bfactor(charges_list[idx])&lt;/code&gt; or &lt;code&gt;atom.set_occupancy(charges_list[idx])&lt;/code&gt; be sure to modify &lt;code&gt;PDBIO&lt;/code&gt; to make space for your values&lt;/p&gt;&#xA;&lt;p&gt;There are other examples on SO about same problem too :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://stackoverflow.com/questions/53938256/how-to-modify-columns-contents-in-a-protein-pdb-file-by-keys-and-values-stored-i&quot;&gt;How to modify columns contents in a protein PDB file by keys and values stored in a text file using only bash tools&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Just Python code :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pdb = 'input_connect.pdb'&#xA;&#xA;charges = 'input_charge.txt'&#xA;&#xA;charges_list = []&#xA;&#xA;with open(charges , 'r') as handle :&#xA;    &#xA;    for i in handle.readlines() :&#xA;&#xA;        charges_list.append(i.strip())&#xA;        &#xA;print(charges_list)&#xA;&#xA;&#xA;&#xA;with open('input_connect.pdb' , 'r') as handle :&#xA;    &#xA;    pdb_file = handle.readlines()&#xA;    &#xA;&#xA;print(pdb_file)&#xA;&#xA;&#xA;idx = 0&#xA;&#xA;with open('result_python.pdb' , 'w') as handle :&#xA;    &#xA;    for i in pdb_file :&#xA;        &#xA;        i = i.strip()&#xA;        &#xA;        if i[0:6] == 'HETATM' : &#xA;            &#xA;            i = i +'  '+  charges_list[idx] +'\n'&#xA;            &#xA;            idx += 1&#xA;            &#xA;            handle.writelines(i)&#xA;            &#xA;        else :&#xA;            &#xA;            i = i + '\n'&#xA;            &#xA;            handle.writelines(i)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Am sure there is a bash or perl one liner for the code above.&lt;/p&gt;&#xA;&lt;p&gt;... &lt;a href=&quot;https://stackoverflow.com/questions/16394176/how-to-merge-two-files-consistently-line-by-line&quot;&gt;How to merge two files consistently line by line&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2023-11-25T14:09:11.843" LastActivityDate="2023-11-25T14:09:11.843" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21837" PostTypeId="2" ParentId="289" CreationDate="2023-11-09T08:52:32.393" Score="1" Body="&lt;p&gt;The MAPQ score appears to be calculated in the mem_approx_mapq_se function. I tried porting it to Python and inserting comments using ChatGPT 4, though I haven't tested it to ensure the output is identical to the original.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import math&#xA;&#xA;# Constants&#xA;MEM_MAPQ_COEFFICIENT = 30.0&#xA;MEM_MAPQ_MAXIMUM = 60&#xA;&#xA;def calculate_mapping_quality(alignment_options, alignment_region):&#xA;    # Determine the suboptimal alignment score, which is the second-best SW alignment score if available,&#xA;    # otherwise calculated from minimum seed length and match score.&#xA;    suboptimal_alignment_score = (alignment_region.suboptimal_score if alignment_region.suboptimal_score &#xA;                                  else alignment_options.minimum_seed_length * alignment_options.match_score)&#xA;&#xA;    # Update the suboptimal alignment score to the higher of the two: current suboptimal score or tandem hit score.&#xA;    suboptimal_alignment_score = (alignment_region.tandem_hit_score if alignment_region.tandem_hit_score &amp;gt; suboptimal_alignment_score &#xA;                                  else suboptimal_alignment_score)&#xA;&#xA;    # Early exit with zero mapping quality if the suboptimal score is greater than or equal to the alignment score.&#xA;    if suboptimal_alignment_score &amp;gt;= alignment_region.best_alignment_score:&#xA;        return 0&#xA;&#xA;    # Calculate the maximum alignment length between the query and reference sequences.&#xA;    alignment_length = max(alignment_region.query_end - alignment_region.query_start, &#xA;                           alignment_region.reference_end - alignment_region.reference_start)&#xA;&#xA;    # Calculate the identity of the alignment.&#xA;    identity = (1.0 - (alignment_length * alignment_options.match_score - alignment_region.best_alignment_score) &#xA;                / (alignment_options.match_score + alignment_options.mismatch_penalty) / alignment_length)&#xA;&#xA;    # Compute the mapping quality based on the alignment score, identity, and other factors.&#xA;    if alignment_region.best_alignment_score == 0:&#xA;        mapping_quality = 0&#xA;    elif alignment_options.mapQ_coef_length &amp;gt; 0:&#xA;        # Adjust mapping quality based on alignment length and identity.&#xA;        length_factor = (1.0 if alignment_length &amp;lt; alignment_options.mapQ_coef_length &#xA;                         else alignment_options.mapQ_coef_fac / math.log(alignment_length))&#xA;        length_factor *= identity * identity&#xA;        mapping_quality = int(6.02 * (alignment_region.best_alignment_score - suboptimal_alignment_score) &#xA;                              / alignment_options.match_score * length_factor * length_factor + 0.499)&#xA;    else:&#xA;        # Use the constant MEM_MAPQ_COEFFICIENT for mapping quality calculation.&#xA;        mapping_quality = int(MEM_MAPQ_COEFFICIENT * (1.0 - suboptimal_alignment_score / alignment_region.best_alignment_score) &#xA;                              * math.log(alignment_region.seed_coverage) + 0.499)&#xA;        mapping_quality = int(mapping_quality * identity * identity + 0.499) if identity &amp;lt; 0.95 else mapping_quality&#xA;&#xA;    # Penalize the mapping quality for the presence of suboptimal hits.&#xA;    if alignment_region.suboptimal_hit_count &amp;gt; 0:&#xA;        mapping_quality -= int(4.343 * math.log(alignment_region.suboptimal_hit_count + 1) + 0.499)&#xA;&#xA;    # Ensure the mapping quality does not exceed the maximum allowed value.&#xA;    mapping_quality = min(max(mapping_quality, 0), MEM_MAPQ_MAXIMUM)&#xA;&#xA;    # Adjust the mapping quality based on the fraction of repetitive regions in the alignment.&#xA;    mapping_quality = int(mapping_quality * (1.0 - alignment_region.repetitive_region_fraction) + 0.499)&#xA;&#xA;    # Return the final mapping quality value.&#xA;    return mapping_quality&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18519" LastActivityDate="2023-11-09T08:52:32.393" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21838" PostTypeId="1" CreationDate="2023-11-09T09:58:35.613" Score="0" ViewCount="94" Body="&lt;p&gt;I have seen in several articles, the use of &lt;a href=&quot;https://www.nature.com/articles/s41587-019-0114-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;CIBERSORT&lt;/a&gt; to calculate the proportion of immune system cells using gene expression data, I would like to try it but I have not found how to install it for R, I have tried to register on the website but it doesn't allow me to enter after introduing my e-mail and password.&lt;/p&gt;&#xA;" OwnerUserId="15698" LastEditorUserId="73" LastEditDate="2023-11-10T04:22:55.377" LastActivityDate="2023-11-10T04:22:55.377" Title="how to use CIBERSORT" Tags="|rna-seq|cibersort|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21840" PostTypeId="1" CreationDate="2023-11-09T21:08:17.540" Score="1" ViewCount="51" Body="&lt;p&gt;Can someone help hint me how I can interpret ordinal logistic regression coefficients and how I can use the .L, .Q and .C terms to calculate probabilities? I am analysing a dataset, where people complain of chest or not pain when they receive external supplements for a vitamin X or not. so I performed a logistic regression, using categorized and ordered (so it is an ordinal categorical variable) values of vitamin X from these observations. So I have grouped these vitamin X values within my data as very low, sub-optimal, optimal and high values and ordered that way.&lt;/p&gt;&#xA;&lt;p&gt;The data are as such:&lt;/p&gt;&#xA;&lt;p&gt;chest_pain_status: binary, 0 means not having chest pain, 1 means having chest pain&lt;/p&gt;&#xA;&lt;p&gt;suppl = binary, 0 means not supplementing and 1 means supplementing&lt;/p&gt;&#xA;&lt;p&gt;vitx_levels: very_low &amp;lt; suboptimal &amp;lt; optimal&amp;lt; high&lt;/p&gt;&#xA;&lt;p&gt;My approach is as follows:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;mod &amp;lt;- glm(chest_pain_status ~ suppl*vitx_levels, dataset = data,family = binomial())&#xA;&#xA;summary(mod)&#xA;                        Estimate       std.Error     z value          Pr( &amp;lt;|z|)&#xA;(intercept)             -2.7816        0.2993        -9.2927          0.000&#xA;suppl1                   0.9822        0.3209         3.0803          0.0021&#xA;vitx_levels.L           -0.1751        0.0634        -2.7391          0.0062&#xA;vitx_levels.Q           -0.1537        0.0552        -2.7689          0.0053&#xA;vitx_levels.C            0.0164        0.0460         0.3579          0.7203&#xA;suppl1:vitx_levels.L     0.16821       0.0739         2.7472          0.0229&#xA;suppl1:vitx_levels.Q     0.2433        0.0641         3.7932          0.00015&#xA;suppl1:vitx_levels.C    -0.0096        0.0537        -0.1787          0.8582&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question1&lt;/strong&gt;: Please how do I interpret coefficients of &lt;code&gt;vitx_levels.L&lt;/code&gt;, &lt;code&gt;vitx_levels.Q&lt;/code&gt; and &lt;code&gt;vitx_levels.C&lt;/code&gt;? From my googling I can understand that R tries to test linear, quadratic and cubic relationships between &lt;code&gt;vitx_levels&lt;/code&gt; and the log odds ratios of developing chest pain. It therefore means that there is a significant linear and negative relationship between &lt;code&gt;vitx_levels&lt;/code&gt; and &lt;code&gt;chest_pain&lt;/code&gt;. Same for the quadratic term. I hope I interpreted it right? Any additional ideas?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question2&lt;/strong&gt;: What values could I substitute in to &lt;code&gt;vitx_levels.L&lt;/code&gt;, &lt;code&gt;vitx_levels.Q&lt;/code&gt; and &lt;code&gt;vitx_levels.C&lt;/code&gt; If I wanted to create a formula like the one below where I will later transform y to get probabilities:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;y &amp;lt;- -2.7816 + 0.9822 * suppl1 - 0.1751 * vitx_levels.L - 0.1537 * vitx_levels.Q + 0.0164 * vitx_levels.C +  0.16821 * suppl1:vitx_levels.L + 0.2433 * suppl1:vitx_levels.Q - 0.0096 * suppl1:vitx_levels.C&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So for the function above, what values could I fit into &lt;code&gt;vitx_levels.L&lt;/code&gt;, &lt;code&gt;vitx_levels.Q&lt;/code&gt; and &lt;code&gt;vitx_levels.C&lt;/code&gt; to be able to compute the value of y?&lt;/p&gt;&#xA;" OwnerUserId="3968" LastEditorUserId="3968" LastEditDate="2023-11-10T21:10:28.167" LastActivityDate="2023-11-10T21:10:28.167" Title="Intepreting and applying ordinal logistic regression coefficients to calculate probabilities?" Tags="|r|statistics|ggplot2|machine-learning|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21841" PostTypeId="1" AcceptedAnswerId="21857" CreationDate="2023-11-10T04:34:02.893" Score="1" ViewCount="30" Body="&lt;p&gt;This might be something very trivial to ask, but I am new to this and was confused as in how to identify or differentiate between very closely related isolates from clonal isolate from the whole genome/core-genome phylogeny?&lt;/p&gt;&#xA;&lt;p&gt;For instace, I am drawing a core-genome phylogeny of Staphylococcus aureus genomes taken from NCBI as well as from my own sequenced ones. Now, my isolates have grouped with several isolates from NCBI (Bottom four greens with four red ones). I want to know, weather these isolates are actually closely related to NCBI ones or they are just those strains themselves?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/diGK7.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/diGK7.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thank you all in advance!&lt;/p&gt;&#xA;" OwnerUserId="18524" LastEditorUserId="18524" LastEditDate="2023-11-15T05:05:16.777" LastActivityDate="2023-11-15T09:36:20.190" Title="Confirming Identity of clone in Phylogeny" Tags="|phylogeny|genomics|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21842" PostTypeId="1" CreationDate="2023-11-10T05:12:22.730" Score="1" ViewCount="10" Body="&lt;p&gt;I am very new to single-cell metabolomics and today I want to jump in this area. I have been reading many review papers related to identification of metabolites at single-cell resolution from spectral data through spectral library search and I have a question that cannot find out a solution.&lt;/p&gt;&#xA;&lt;p&gt;The common workflow to identify mebolites I have discovered so far is that starting from, let say, 1000 live cells, we extract the content of each cell, put them to, let say, GC-TOF-MS technology to have spectral graphs (x-axis presents m/z values and y-axis presents intensities), and finally compare the spectral graph of each cell against a certain spectral library to infer metabolites.&lt;/p&gt;&#xA;&lt;p&gt;The thing is we know that a single cell has multiple metabolites regulating cellular functions of that cell. However, the data I see show that each cell has only one spectrum (the figure below) =&amp;gt; we can only find out 1 metabolite for one cell??? Is that true? Or something I am missing?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BANLp.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BANLp.jpg&quot; alt=&quot;spectral data&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18525" LastActivityDate="2023-11-10T05:12:22.730" Title="Identification of metabolites through spectral library search" Tags="|single-cell|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21843" PostTypeId="1" CreationDate="2023-11-10T14:19:53.813" Score="1" ViewCount="60" Body="&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pandas as pd&#xA;import requests&#xA;import xml.etree.ElementTree as ET&#xA;df = pd.read_excel('file.xlsx')&#xA;smiles = df['SMILES'].to_list()&#xA;metabolite =  df['Plant_metabolite'].to_list()&#xA;def downloader(smile):&#xA;    url = &amp;quot;https://bindingdb.org/axis2/services/BDBService/getTargetByCompound?smiles={SMILES}&amp;amp;cutoff={similarity_cutoff}&amp;quot;&#xA;    if type(smile) != str:&#xA;        return None&#xA;    else:&#xA;        similarity_cutoff = &amp;quot;0.85&amp;quot;&#xA;&#xA;        url = url.replace(&amp;quot;{SMILES}&amp;quot;, smile)&#xA;        url = url.replace(&amp;quot;{similarity_cutoff}&amp;quot;, similarity_cutoff)&#xA;&#xA;        response = requests.get(url)&#xA;&#xA;        if response.status_code == 200:&#xA;            response = response.text&#xA;&#xA;        else:&#xA;            return None&#xA;        return response&#xA;for i in range(0,len(smiles)):&#xA;    resp = downloader(smiles[i])&#xA;    if resp == None:&#xA;        pass&#xA;    else:&#xA;        tree = ET.fromstring(resp)&#xA;        dictionary = {}&#xA;        for j in range(3,len(tree)):&#xA;            for x in tree[j]:&#xA;                if x.tag[29:] not in dictionary.keys(): &#xA;                    dictionary[x.tag[29:]] = []&#xA;                dictionary[x.tag[29:]].append(x.text)&#xA;        df = pd.DataFrame(dictionary)&#xA;        if len(df.columns) &amp;gt; 0:&#xA;            df = df.loc[df['tanimoto'] &amp;gt; &amp;quot;0.85&amp;quot;]&#xA;            df = df.drop_duplicates(subset='smiles',keep = 'first')&#xA;            df.replace({'na': pd.NA}, inplace=True)&#xA;            df = df.dropna()&#xA;            name = &amp;quot;Valeriana jatamansi/{}.csv&amp;quot;.format(metabolite[i])&#xA;            df.to_csv(name,index = False)&#xA;        else:&#xA;            pass&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This my code which I am using to download targets for my compound, but there is a difference between the output returned by the API and in the online database? Like the names of the targets and other stuff...&#xA;For example: For alpha-cadanol (SMILES: CC1=CC2C(CC1)C@@(O)CC[C@@H]2C(C)C)&lt;/p&gt;&#xA;&lt;p&gt;This is the downloaded version:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/dMlPh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dMlPh.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This is the online version:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/DWkxO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/DWkxO.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&#xA;Is there something wrong in the code, or is something else the problem here? Is there any other method to do the same thing?&#xA;The similarity cutoff is 0.85 both in scripts and online, and affinity value for online portal is set up as 10000, no affinity filter is provided by the API.&lt;/p&gt;&#xA;" OwnerUserId="9772" LastEditorUserId="9772" LastEditDate="2023-11-11T08:20:58.003" LastActivityDate="2023-11-11T08:20:58.003" Title="Need help with binding DB API" Tags="|python|database|public-databases|api|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21844" PostTypeId="2" ParentId="21840" CreationDate="2023-11-10T18:02:12.363" Score="1" Body="&lt;p&gt;Its a while since I've done GLM because ML is the method of choice.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question1&lt;/strong&gt; Yes multi-linear regression is an approximate description of GLM.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Generally&lt;/em&gt; the weights simply mean whether a feature is positively or negatively associated with the phenomena. It's important to be sure it's the right way around (negatives and positives can flip, if that makes sense).&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;suppl1&lt;/code&gt; is significantly and relatively strongly associated with symptoms, whilst &lt;code&gt;L&lt;/code&gt; and &lt;code&gt;Q&lt;/code&gt; &lt;code&gt;vitamin&lt;/code&gt; levels appear to have some &lt;em&gt;protective&lt;/em&gt; effects.&#xA;&lt;code&gt;Suppl1&lt;/code&gt; has a dose dependent effect with &lt;code&gt;L&lt;/code&gt; and &lt;code&gt;Q&lt;/code&gt; having a lower correlation with the observed symptoms.&lt;/p&gt;&#xA;&lt;p&gt;The dose dependency result therefore confirm suppl1 as associated with the symptoms. In all cases levels &lt;code&gt;C&lt;/code&gt; are not significant, I assume this is a low level supplement.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; &lt;code&gt;vitx_levels&lt;/code&gt; appear to have some protective effect.&lt;/p&gt;&#xA;&lt;p&gt;I do stress it is important to ensure the &amp;quot;negative versus positive&amp;quot; is the right way round. The residual is important and I suspect you've omitted this in the GLM. The lower the residual the better the fit of the model.&lt;/p&gt;&#xA;&lt;p&gt;BTW LD is different from GLM although they are related (title versus body of the question).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 2&lt;/strong&gt; Yes that is correct &lt;code&gt;0.9822 * 1&lt;/code&gt; I assume you mean &lt;code&gt;0.9822 * suppl1&lt;/code&gt; ... However all &lt;code&gt;C&lt;/code&gt; weights are not significant, so you could omit them.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;residual versus intercept ... I'm pretty certain that these are the same thing (its the &lt;em&gt;c&lt;/em&gt; variable) and the intercept here appears quite large.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-11-10T18:57:16.277" LastActivityDate="2023-11-10T18:57:16.277" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21845" PostTypeId="1" CreationDate="2023-11-11T08:47:35.633" Score="0" ViewCount="69" Body="&lt;p&gt;I used a Genotyping software called T1K to generate KIR genotyping, the output file contains genes and alleles for each sample, how can i convert the results to a vcf file or other formats which can be used in plink?&#xA;The file i have for now looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Gene_name Allele_1 Allele_2&#xA;KIR2DL1 KIR2DL1*004 KIR2DL1*042&#xA;KIR2DL4 KIR2DL4*005 KIR2DL4*005&#xA;KIR3DL1 KIR3DL1*005 KIR3DL1*005&#xA;KIR3DP1 KIR3DP1*006 KIR3DP1*010&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My thought is to first convert the above file to a vcf file and use plink to generate bedbimfam from it, a standard vcf is like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.3&#xA;##fileDate=20090805&#xA;##source=myImputationProgramV3.1&#xA;##reference=file:///seq/references/1000GenomesPilot-NCBI36.fasta&#xA;##contig=&amp;lt;ID=20,length=62435964,assembly=B36,md5=f126cdf8a6e0c7f379d618ff66beb2da,species=&amp;quot;Homo sapiens&amp;quot;,taxonomy=x&amp;gt;&#xA;##phasing=partial&#xA;##INFO=&amp;lt;ID=NS,Number=1,Type=Integer,Description=&amp;quot;Number of Samples With Data&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=DP,Number=1,Type=Integer,Description=&amp;quot;Total Depth&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=AF,Number=A,Type=Float,Description=&amp;quot;Allele Frequency&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=AA,Number=1,Type=String,Description=&amp;quot;Ancestral Allele&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=DB,Number=0,Type=Flag,Description=&amp;quot;dbSNP membership, build 129&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=H2,Number=0,Type=Flag,Description=&amp;quot;HapMap2 membership&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=q10,Description=&amp;quot;Quality below 10&amp;quot;&amp;gt;&#xA;##FILTER=&amp;lt;ID=s50,Description=&amp;quot;Less than 50% of samples have data&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&amp;quot;Genotype&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GQ,Number=1,Type=Integer,Description=&amp;quot;Genotype Quality&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=DP,Number=1,Type=Integer,Description=&amp;quot;Read Depth&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=HQ,Number=2,Type=Integer,Description=&amp;quot;Haplotype Quality&amp;quot;&amp;gt;&#xA;#CHROM POS      ID         REF   ALT    QUAL  FILTER   INFO                             &#xA;FORMAT       NA00001         NA00002          NA00003&#xA;20     14370    rs6054257  G     A      29    PASS    NS=3;DP=14;AF=0.5;DB;H2           GT:GQ:DP:HQ  0|0:48:1:51,51  1|0:48:8:51,51   1/1:43:5:.,.&#xA;20     17330    .          T     A      3     q10     NS=3;DP=11;AF=0.017               GT:GQ:DP:HQ  0|0:49:3:58,50  0|1:3:5:65,3     0/0:41:3&#xA;20     1110696  rs6040355  A     G,T    67    PASS    NS=2;DP=10;AF=0.333,0.667;AA=T;DB GT:GQ:DP:HQ  1|2:21:6:23,27  2|1:2:0:18,2     2/2:35:4&#xA;20     1230237  .          T     .      47    PASS    NS=3;DP=13;AA=T                   GT:GQ:DP:HQ  0|0:54:7:56,60  0|0:48:4:51,51   0/0:61:2&#xA;20     1234567  microsat1  GTC   G,GTCT 50    PASS    NS=3;DP=9;AA=G                    GT:GQ:DP     0/1:35:4        0/2:17:2         1/1:40:3&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any tools can be used to realize it? For now i only have Gene names and allele names for each sample, How to obtain other genetic informtation a vcf file needed?&lt;/p&gt;&#xA;" OwnerUserId="18532" LastEditorUserId="18532" LastEditDate="2023-11-12T08:48:21.163" LastActivityDate="2023-11-12T08:48:21.163" Title="Convert genotyping results to vcf or other plink formats" Tags="|vcf|gene|plink|genotyping|imputation|" AnswerCount="0" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="21846" PostTypeId="2" ParentId="21833" CreationDate="2023-11-11T13:53:43.673" Score="2" Body="&lt;p&gt;There are two main problems I can see in your approach. First, gene names are not standardized across species and the &amp;quot;same&amp;quot; gene (what that means is a whole different discussion) can have different names. For example, human  &amp;quot;Cytochrome c oxidase assembly factor 1&amp;quot; &lt;a href=&quot;https://www.genenames.org/data/gene-symbol-report/#!/hgnc_id/HGNC:21868&quot; rel=&quot;nofollow noreferrer&quot;&gt;has the symbol &amp;quot;COA1&amp;quot;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Next, I don't quite understand why you expect the word &amp;quot;complete&amp;quot; to be in the title. That is actually why you don't find the &lt;em&gt;D. magna&lt;/em&gt; sequence: it isn't complete, the start codon hasn't been correctly annotated as you can see on the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene/24147205/&quot; rel=&quot;nofollow noreferrer&quot;&gt;relevant NCBI gene page&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7xM2k.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7xM2k.png&quot; alt=&quot;Screenshot showing that the start codon is not determined&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would instead try something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;((chrM[Chromosome]) OR (chromosome M[Chromosome]) OR (MT[Chromosome]) OR (chromosome MT[Chromosome])) AND ((CO1[Gene Name]) OR (COX1[Gene Name]) OR ((cytochrome[Gene Name]) AND (oxidase[Gene Name]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;or&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;((16S[Gene/Protein Name]) and (ribosomal[Gene/Protein Name]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;All that said, trying to rely on gene annotations is rarely the best approach. There just isn't enough standardization. I would instead look at databases that provide lists of homologous genes. For instance, NCBI's &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/research/cog/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Clusters of Orthologous Genes (COGs)&lt;/a&gt;, which provides grouped homologous sequences.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2023-11-13T16:02:07.980" LastActivityDate="2023-11-13T16:02:07.980" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21847" PostTypeId="2" ParentId="21833" CreationDate="2023-11-11T17:11:36.337" Score="1" Body="&lt;p&gt;The search term you need is &lt;strong&gt;[gene name]&lt;/strong&gt; rather than &lt;em&gt;[title]&lt;/em&gt;. For example,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(Gasterosteus aculeatus[Organism] AND cox1[Gene Name]) OR (Gasterosteus aculeatus[Organism] AND coi[Gene Name]) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Results in &lt;strong&gt;266&lt;/strong&gt; hits which all contain COX1 COI&lt;/p&gt;&#xA;&lt;p&gt;Using Daphnia magna ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(Daphnia magna[Organism] AND cox1[Gene Name]) OR (Daphnia magna[Organism] AND coi[Gene Name]) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This results in &lt;strong&gt;775&lt;/strong&gt; hits.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Daphina magna&lt;/em&gt; comprises &lt;strong&gt;61&lt;/strong&gt; mtDNA genomes, you perform the above search and sort by sequence length and set the sequences per page to 200(easy).&lt;/p&gt;&#xA;&lt;p&gt;Hope that resolves the issue.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-11-11T17:19:14.627" LastActivityDate="2023-11-11T17:19:14.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21848" PostTypeId="2" ParentId="21826" CreationDate="2023-11-13T09:58:24.000" Score="1" Body="&lt;p&gt;Below code works perfectly of me.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(readr)&#xA;library(data.table)&#xA;&#xA;df1 &amp;lt;- read_csv(&amp;quot;control.csv&amp;quot;, col_types = cols(Chromosome = col_skip(), Start = col_skip(), End = col_skip(), Strand = col_skip()))&#xA;&#xA;df1[1:5,1:5]&#xA;# A tibble: 5 × 5&#xA;  ID          DN54 DC146 DDN44 DC179&#xA;  &amp;lt;chr&amp;gt;      &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;&#xA;1 cg26928153 0.848 0.879 0.872 0.897&#xA;2 cg16269199 0.731 0.732 0.759 0.775&#xA;3 cg13869341 0.842 0.863 0.819 0.896&#xA;4 cg24669183 0.824 0.866 0.822 0.910&#xA;5 cg26679879 0.425 0.363 0.420 0.358&#xA;&#xA;col &amp;lt;- unlist(fread(&amp;quot;samples.csv&amp;quot;, header=F))&#xA;col &amp;lt;- c(&amp;quot;ID&amp;quot;, col)&#xA;extracted &amp;lt;- df1[,col, with=F]&#xA;&#xA;write.table(extracted, file=&amp;quot;control.beta.txt&amp;quot;,sep=&amp;quot;,&amp;quot;, quote=F, col.names=TRUE,row.names=FALSE)&#xA;&#xA;df &amp;lt;- read_csv(&amp;quot;control.beta.txt&amp;quot;)&#xA;df[1:5,1:5]&#xA;# A tibble: 5 × 5&#xA;  ID         DC103R DC110R DC115 DC120R&#xA;  &amp;lt;chr&amp;gt;       &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt; &amp;lt;dbl&amp;gt;  &amp;lt;dbl&amp;gt;&#xA;1 cg26928153  0.929  0.885 0.862  0.877&#xA;2 cg16269199  0.836  0.826 0.604  0.739&#xA;3 cg13869341  0.826  0.903 0.830  0.841&#xA;4 cg24669183  0.767  0.719 0.808  0.812&#xA;5 cg26679879  0.388  0.366 0.428  0.360&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="650" LastEditDate="2023-11-13T16:37:24.150" LastActivityDate="2023-11-13T16:37:24.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21849" PostTypeId="2" ParentId="21787" CreationDate="2023-11-13T14:46:30.227" Score="1" Body="&lt;p&gt;1 and 2 —&amp;gt; Yes, but you need to have the query ID in order to do so: having the description would be a bonus, but not required.  If you go to 8.1.1  Query Result of the Biopython manual, it will tell you how to get a query id, if you don’t already have it!  Go to section 7.1 as a reminder of how to run  BLAST via the internet.  Go to section 7.2 to. Run BLAST locally.   Weigh your options out with running via Internet or locally.  Running BLAST locally is faster usually and gives you more user freedom to create your own database with your own parameters, etcetera., but certain tools will need to be downloaded via the command line/terminal.  Follow this link for the BLAST command line user manual:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK279690/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/books/NBK279690/&lt;/a&gt;&#xA;I will paste the link to the Biopython user manual if you do not already have it:&#xA;&lt;a href=&quot;http://biopython.org/DIST/docs/tutorial/Tutorial.html#sec123&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://biopython.org/DIST/docs/tutorial/Tutorial.html#sec123&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;3 —&amp;gt; Try the Protein Data Bank (PDB) instead of UniProt or you can combine both UniProt and PDB (I recommend this route as it’s a combination of the 2 databases).  You can use these two databases to process a BLAST result!  I have found in my experience, combining data from more than 1 database has been beneficial in my case!&lt;/p&gt;&#xA;" OwnerUserId="18487" LastActivityDate="2023-11-13T14:46:30.227" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21850" PostTypeId="1" CreationDate="2023-11-13T16:04:51.160" Score="2" ViewCount="48" Body="&lt;p&gt;We are doing a comparison between two outcomes (positive and negative). We could not have any positive controls as we do not have any &amp;quot;control&amp;quot; data to set as baseline, either from literature or otherwise. No spike-ins have been used. We have some samples which had no resolved outcome, hence we are skeptic of using those as Controls.&lt;/p&gt;&#xA;&lt;p&gt;We have few negative control samples which contain just the blank culture media with no RNA. We want to remove these signals from the RNA-seq data, but I would like to know the most ideal way to perform this. I would assume just normalizing and subtracting counts wouldn't be the most ideal.&lt;/p&gt;&#xA;&lt;p&gt;So here's what the &lt;code&gt;coldata&lt;/code&gt; looks like -&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Sample_ID   Outcome&#xA;S1  Positive&#xA;S2  Negative&#xA;S3  Unknown&#xA;S4  Negative&#xA;…   …&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My previous post has been about the &lt;a href=&quot;https://support.bioconductor.org/p/9154760/&quot; rel=&quot;nofollow noreferrer&quot;&gt;same experiment&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here are the counts for the top 5 genes with high counts in negative samples (RIF1, GOLGB1, C10orf67, MAP3K15, F8):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Geneid  S1  S2  S3  S4  S5  S6  S7  S8  S9  S10 S11 S12 S13 S14 S15 S16 S17 S18 S19 S20 S21&#xA;ENSG00000189283.11  2284    2986    4577    2447    1290    1882    1953    1366    2443    3932    2640    1078    491 2919    1248    1627    1805    2094    2229    2789    190&#xA;ENSG00000173230.17  1000    1219    1324    1485    722 1142    870 943 1390    1617    1093    564 86  924 778 1059    1144    1248    1169    1828    50&#xA;ENSG00000180815.15  641 723 1118    1758    529 865 683 878 1554    2421    1571    414 136 1076    599 924 1098    1199    1405    1327    54&#xA;ENSG00000179133.15  369 433 592 488 305 366 335 252 455 2119    1357    118 159 314 200 284 376 333 517 550 17&#xA;ENSG00000256637.9   429 491 523 550 250 410 406 359 552 438 339 176 38  357 296 433 517 514 631 779 24&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6561" LastEditorUserId="73" LastEditDate="2023-11-15T07:11:10.793" LastActivityDate="2023-11-15T07:11:10.793" Title="How to incorporate negative controls in DESeq2" Tags="|rna-seq|differential-expression|deseq2|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21851" PostTypeId="2" ParentId="4206" CreationDate="2023-11-13T20:18:23.347" Score="2" Body="&lt;p&gt;The previous answers suggest Chimera and FoldX, which require a license for commercial use.&lt;/p&gt;&#xA;&lt;p&gt;Commercial users can use &lt;a href=&quot;https://github.com/openmm/pdbfixer/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pdbfixer&lt;/a&gt; or a similar tool to perform mutations (using e.g. &lt;a href=&quot;https://github.com/openmm/pdbfixer/blob/master/pdbfixer/pdbfixer.py#L729&quot; rel=&quot;nofollow noreferrer&quot;&gt;applyMutation&lt;/a&gt; in the API).&lt;/p&gt;&#xA;&lt;p&gt;You can then use &lt;a href=&quot;https://openmm.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenMM&lt;/a&gt; to &lt;a href=&quot;http://docs.openmm.org/latest/userguide/application/02_running_sims.html#a-first-example&quot; rel=&quot;nofollow noreferrer&quot;&gt;refine the mutated structure&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Granted, this requires some comfort writing Python scripts to use the APIs.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-11-13T20:18:23.347" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21855" PostTypeId="1" CreationDate="2023-11-14T21:11:41.570" Score="3" ViewCount="40" Body="&lt;p&gt;I wanted to use the code to convert mouse to human symbol, but I have problem with the mirror, I've tried to establish another one, but continue to have the same problem.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;musGenes &amp;lt;- c(&amp;quot;Hmmr&amp;quot;, &amp;quot;Tlx3&amp;quot;, &amp;quot;Cpeb4&amp;quot;)&#xA;&#xA;# Basic function to convert mouse to human gene names&#xA;convertMouseGeneList &amp;lt;- function(x){&#xA;    require(&amp;quot;biomaRt&amp;quot;)&#xA;    human = useMart(&amp;quot;ensembl&amp;quot;, dataset = &amp;quot;hsapiens_gene_ensembl&amp;quot;)&#xA;    mouse = useMart(&amp;quot;ensembl&amp;quot;, dataset = &amp;quot;mmusculus_gene_ensembl&amp;quot;)&#xA;    genesV2 = getLDS(attributes = c(&amp;quot;mgi_symbol&amp;quot;), filters = &amp;quot;mgi_symbol&amp;quot;, values = x , mart = mouse, attributesL = c(&amp;quot;hgnc_symbol&amp;quot;), martL = human, uniqueRows=T)&#xA;    humanx &amp;lt;- unique(genesV2[, 2])&#xA;&#xA;    # Print the first 6 genes found to the screen&#xA;    print(head(humanx))&#xA;    return(humanx)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15698" LastEditorUserId="650" LastEditDate="2023-11-15T14:52:32.733" LastActivityDate="2023-11-16T20:29:27.437" Title="biomarRT conversion of symbols" Tags="|r|biomart|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21856" PostTypeId="1" CreationDate="2023-11-15T08:00:55.093" Score="2" ViewCount="94" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9580264/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I was wondering if any of you have encountered a situation for bulk RNA-Seq where, possibly due to low sample quality or presence of dead cells, mitochondrial genes are expressed to a very large degree relative to other genes, thus skewing TPM values of all nuclear genes (by effectively scaling them down).&lt;/p&gt;&#xA;&lt;p&gt;In such a situation, what could I potentially do to alleviate this beyond preparing fresh samples? Could I, for example, exclude 'MT-' genes from samples and then recalculate TPM based on this filtered set of genes?&lt;/p&gt;&#xA;&lt;p&gt;My aim was to get some idea of within-samples expression levels, hence why going for TPMs as I know that counts normalised by DESeq2, edgeR and the like do not correct for gene length and are inappropriate for within-samples comparisons.&lt;/p&gt;&#xA;&lt;p&gt;Many thanks in advance for any insight&lt;/p&gt;&#xA;" OwnerUserId="18557" LastEditorUserId="73" LastEditDate="2023-11-19T06:04:17.443" LastActivityDate="2023-11-19T06:04:17.443" Title="Mitochondrial genes - TPM calculation bulk RNA-Seq" Tags="|rna-seq|tpm|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21857" PostTypeId="2" ParentId="21841" CreationDate="2023-11-15T09:36:20.190" Score="0" Body="&lt;p&gt;Your isolates are both robustly part of the NCBI lineage of 4, i.e. now making a lineage of 8 taxa and genetically very close to it, but slightly distinct.&lt;/p&gt;&#xA;&lt;p&gt;The robustness is given by the bootstrap values (100) and the genetic proximity is given by the genetic distances for each branch. What you want to do is look at the epidemiological information for your strains and those in Genbank, there should be some sort of recognised association, e.g. country, possibly locality, anything else?&lt;/p&gt;&#xA;&lt;p&gt;The association has epidemiological significance, thus it's not random, if that is your question. There will definitely something connecting this lineage of 8.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-11-15T09:36:20.190" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21858" PostTypeId="2" ParentId="21855" CreationDate="2023-11-15T14:57:04.080" Score="1" Body="&lt;p&gt;My background in coding is more extensive in Python.  If you have Python skills, I can say for sure, use the Bioservices library:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from bioservices import BioMart&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Install Bioservices via pip:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;pip install bioservices&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can also use the web-based tool on Ensembl called BioMart.  It does the same thing, just easier to use if you continue to have difficulties with Python or in your case, R.  I looked up the R package for BioMart, it is called: biomaRt&lt;/p&gt;&#xA;&lt;p&gt;Here's the link for the Ensembl BioMart web-based tool if you need to use it and are in a crunch for time on your project:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://useast.ensembl.org/info/data/biomart/how_to_use_biomart.html#introduction&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://useast.ensembl.org/info/data/biomart/how_to_use_biomart.html#introduction&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I hope this helps! :)&lt;/p&gt;&#xA;" OwnerUserId="18487" LastEditorUserId="650" LastEditDate="2023-11-16T20:29:27.437" LastActivityDate="2023-11-16T20:29:27.437" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21859" PostTypeId="1" CreationDate="2023-11-15T17:21:19.660" Score="1" ViewCount="21" Body="&lt;p&gt;I was wondering if anyone knows off-hand if there is a way to run BiG-SCAPE without producing the svg or html files? All I want is the networking files (which I can play with in CytoScape).&lt;/p&gt;&#xA;&lt;p&gt;There isn't anything obvious from reading the parameters section in &lt;a href=&quot;https://github.com/medema-group/BiG-SCAPE/wiki/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the wiki&lt;/a&gt; but perhaps I just missed it? Maybe like a minimal mode or something similar?&lt;/p&gt;&#xA;&lt;p&gt;Normally I would simply delete them, but I'm running around 20 000 BGCs through it. And I thought it would save time and disk space if those files are just never written out. I'm simultaneously trying out bigslice but the documentation is surprisingly tricky to follow.&lt;/p&gt;&#xA;" OwnerUserId="18327" LastEditorUserId="73" LastEditDate="2023-11-16T04:25:39.267" LastActivityDate="2023-11-16T04:25:39.267" Title="Producing just network files from BiG-SCAPE" Tags="|clustering|parameters|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21861" PostTypeId="1" CreationDate="2023-11-15T20:23:45.510" Score="2" ViewCount="68" Body="&lt;p&gt;I am doing research on a new method of optimizing sequence alignment process (Needleman - Wunsch algorithm) but the idea would only work with sequences that have the same length. I am wondering if that will have any real-life utility. Specifically, how often do we have those situations where we are aligning same-length sequences (or at least sub-sequences) and what are those situations?&lt;/p&gt;&#xA;" OwnerUserId="18561" LastEditorUserId="298" LastEditDate="2023-11-16T09:01:54.380" LastActivityDate="2023-11-17T08:10:30.650" Title="Sequence Alignment for sequences with the same length" Tags="|sequence-alignment|rna|sequence|dna|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21862" PostTypeId="2" ParentId="21861" CreationDate="2023-11-16T01:15:42.920" Score="1" Body="&lt;p&gt;I see ... the application is if two genes had the same number of indels* at different homologous positions within the gene. This happens but equally one gene might have an indel and another gene might lack it, in which case these genes would be different  lengths.&lt;/p&gt;&#xA;&lt;p&gt;So just to reiterate your method would find genes with an identical number of indels at different homologous positions in a gene. Its application is protein modelling. This is because indels cause a large change in the entropy of a protein, so even if these two genes had identical sequence (barring the indels obviously) they would be structurally different.&lt;/p&gt;&#xA;&lt;p&gt;Beyond this however the application is limited because apart from the known changes to protein structure, little is understood about quantifying the function of indels. Certainly in phylogenetics we specifically avoid them, because we struggle to fit them into a known model.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;*, indel = insertion deletion event. In a protein they occur in triplets.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-11-16T01:15:42.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21863" PostTypeId="2" ParentId="21856" CreationDate="2023-11-16T04:39:19.087" Score="1" Body="&lt;p&gt;Yes, this is a common situation. Not just for mitochondrial genes, but for any gene that may be an outlier (e.g. the B-cell receptor gene in a contaminant/unexpected B-cell population). The way to get around this is to not use TPM for differential expression analysis.&lt;/p&gt;&#xA;&lt;p&gt;DESeq2 by default uses a quantile-based normalisation method for bulk differential expression calculations, which works well when the majority of genes are not differentially expressed. When this assumption is broken (e.g. for a targeted panel), a control gene list can be supplied, or custom size factors can be applied to each sample.&lt;/p&gt;&#xA;&lt;p&gt;More information about using DESeq2 for differential expression can be found in the DESeq2 manual:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;For comparing expression from different genes, I normalise variance-stabilised data using the longest annotated gene length from transcripts.&lt;/p&gt;&#xA;&lt;p&gt;See &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/15218/73&quot;&gt;this answer&lt;/a&gt; and &lt;a href=&quot;https://doi.org/10.1084/jem.20160470&quot; rel=&quot;nofollow noreferrer&quot;&gt;this paper&lt;/a&gt; for more details.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-11-16T11:05:40.913" LastActivityDate="2023-11-16T11:05:40.913" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21864" PostTypeId="1" CreationDate="2023-11-16T08:10:59.610" Score="0" ViewCount="26" Body="&lt;p&gt;I have a data frame on R, called qIns8, with 14 columns and 204 rows. the column called &amp;quot;subject tax id&amp;quot; has NCBI taxIDs for the corresponding row, I need a script to add another column to the data frame called Organism for which the script has to get the organism name (NCBI taxname) from the column subject tax id.&#xA;I already tried this script using Taxize package, but it did not work:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;getTAXname &amp;lt;- function(tax_id) {&#xA;  name &amp;lt;- tryCatch({&#xA;    tax_name &amp;lt;- sci_name(tax_id)&#xA;    return(tax_name)&#xA;  }, error = function(e) {&#xA;    return(NA)&#xA;  })&#xA;  return(name)&#xA;}&#xA;&#xA;# Apply the function to each row of the dataframe&#xA;qIns8&lt;span class=&quot;math-container&quot;&gt;$Organism &amp;lt;- sapply(qIns8$&lt;/span&gt;subject_tax_id, getTAXname)&#xA;&#xA;# Check the updated dataframe&#xA;head(qIns8)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also tried this from Taxonomizr package and it did not work either:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;getTaxonomy(qIns8$`subject tax id`, sqlFile = qIns8$`Organism`)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18298" LastEditorUserId="650" LastEditDate="2023-11-16T20:28:54.840" LastActivityDate="2023-11-16T20:28:54.840" Title="getting NCBI TAXname by R" Tags="|r|taxonomy|ncbi|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21865" PostTypeId="1" CreationDate="2023-11-16T11:02:38.683" Score="0" ViewCount="102" Body="&lt;p&gt;I would be thankful to you if you can help with the statistical approach for case-control study to link DNA methylation (epic array) with toxic element exposure (arsenic) and health outcome (Cardiovascular diseases).&lt;/p&gt;&#xA;&lt;p&gt;I do have data from 125 Cardiovascular diseases and 125 non-Cardiovascular diseases individuals matched with age, sex, and smoking.I do have phenotypes files in a csv format having sex, age, smoking, disease, samples id, iAs (arsenic) values in log format and then a separate files for methylation data.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;The sample size is small.&lt;/li&gt;&#xA;&lt;li&gt;Maybe structural equation modelling could be used to find CpG sites associated with arsenic exposure and disease condition as I have very small size samples&lt;/li&gt;&#xA;&lt;li&gt;R is a preferred language.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="3848" LastEditDate="2023-11-18T23:45:11.093" LastActivityDate="2023-12-19T00:03:05.400" Title="Statistical approach to link DNA methylation with toxic element exposure and health outcome" Tags="|r|statistics|linear-regression|epigenetics|epidemiology|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21866" PostTypeId="2" ParentId="21864" CreationDate="2023-11-16T16:42:10.660" Score="1" Body="&lt;p&gt;I developed the function and it worked perfectly,&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;Converst_taxID_to_taxName &amp;lt;- function(tax_id) {&#xA;  tax_info &amp;lt;- entrez_summary(db=&amp;quot;taxonomy&amp;quot;, id=tax_id)&#xA;  return(tax_info$scientificname)&#xA;}&#xA;&#xA;# usage example&#xA;myDataFrame &amp;lt;- myDataFrame %&amp;gt;%&#xA;mutate(Organism = sapply(`subject tax id`, Converst_taxID_to_taxName))&#xA;&#xA;# in case one tax ID has a problem or cannot be proceeded, it puts &amp;quot;NA&amp;quot; and automatically goes the next ID&#xA;Converst_taxID_to_taxName2 &amp;lt;- function(tax_id) {&#xA;  tryCatch({&#xA;    tax_info &amp;lt;- entrez_summary(db=&amp;quot;taxonomy&amp;quot;, id=tax_id)&#xA;    return(tax_info$scientificname)&#xA;  }, error = function(e) {&#xA;    return(NA)&#xA;  })&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18298" LastEditorUserId="650" LastEditDate="2023-11-16T20:26:23.270" LastActivityDate="2023-11-16T20:26:23.270" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21868" PostTypeId="2" ParentId="21861" CreationDate="2023-11-17T08:10:30.650" Score="0" Body="&lt;p&gt;My 2 cents: Sequence alignment is a well studied problem in bioinformatics with some great existing methods that do handle uneven length sequences (i.e. indels). With tons of genomic data the trend these days is towards faster methods with more heuristics like pseudo-alignment or k-mer based approaches rather than theoretically optimal but slow and unscalable approaches like Needleman Wunsch. There might be some niche applications on DNA motifs or short peptide analyses but I think these are quite limited.&lt;/p&gt;&#xA;" OwnerUserId="104" LastActivityDate="2023-11-17T08:10:30.650" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21869" PostTypeId="1" CreationDate="2023-11-17T13:47:56.363" Score="0" ViewCount="17" Body="&lt;p&gt;After inputting a list of gene and running an analysis in &lt;a href=&quot;https://www.pantherdb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.pantherdb.org/&lt;/a&gt; (such as &amp;quot;functional classification&amp;quot;), I get a result table that is limited  by an X number of entries per page. I can select some entries from the table and export that subset as a text. But, Is there a simple way of exporting/downloading the whole functional classification for my gene list as a txt?&lt;/p&gt;&#xA;" OwnerUserId="9870" LastActivityDate="2023-11-17T13:47:56.363" Title="Exporting Panther functional classification of gene list" Tags="|public-databases|data-download|functional-annotation|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21870" PostTypeId="1" CreationDate="2023-11-17T14:16:54.583" Score="0" ViewCount="17" Body="&lt;p&gt;Any suggestions for the best way to estimate a metric for evolutionary rate of viral sequences of the same species?&lt;/p&gt;&#xA;&lt;p&gt;I was using terminal branch length from a phylogeny and would quite like to try sub rates but i am really new to this&lt;/p&gt;&#xA;" OwnerUserId="18574" LastEditorUserId="18574" LastEditDate="2023-11-17T14:39:16.860" LastActivityDate="2023-11-17T14:39:16.860" Title="Estimating rate of evolution from a phylogeny" Tags="|sequence-alignment|phylogeny|virus|beast|substitution-model|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21871" PostTypeId="1" CreationDate="2023-11-17T15:18:56.943" Score="1" ViewCount="299" Body="&lt;p&gt;I have single cell data which I want to annotate with the seurat package, so I ran this code in R:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;lung.data &amp;lt;- Read10X(data.dir =&amp;quot;C:/Users/Desktop/SINGLE-CELL/DATA&amp;quot;,gene.column = 1)&#xA;&#xA;lung &amp;lt;- CreateSeuratObject(counts = lung.data, project = &amp;quot;lung&amp;quot;, min.cells = 3, min.features = 200)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And the output of the features of this seurat object is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;    &amp;gt;rownames(lung)&#xA;    [1] &amp;quot;ENSG00000000003&amp;quot; &amp;quot;ENSG00000000419&amp;quot;&#xA; [3] &amp;quot;ENSG00000000457&amp;quot; &amp;quot;ENSG00000000460&amp;quot;&#xA; [5] &amp;quot;ENSG00000000938&amp;quot; &amp;quot;ENSG00000000971&amp;quot;&#xA; [7] &amp;quot;ENSG00000001036&amp;quot; &amp;quot;ENSG00000001084&amp;quot;&#xA; [9] &amp;quot;ENSG00000001167&amp;quot; &amp;quot;ENSG00000001460&amp;quot;&#xA;[11] &amp;quot;ENSG00000001461&amp;quot; &amp;quot;ENSG00000001497&amp;quot;&#xA;[13] &amp;quot;ENSG00000001561&amp;quot; &amp;quot;ENSG00000001617&amp;quot;&#xA;[15] &amp;quot;ENSG00000001626&amp;quot; &amp;quot;ENSG00000001629&amp;quot;&#xA;[17] &amp;quot;ENSG00000001630&amp;quot; &amp;quot;ENSG00000001631&amp;quot;&#xA;[19] &amp;quot;ENSG00000002016&amp;quot; &amp;quot;ENSG00000002079&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I change the row names of this Seurat Object from Ensembl to gene name, knowing that not all the ensemble genes have conversion to gene name, in order to obtain the seurat object with the gene names as row names and cell names as column names?&lt;/p&gt;&#xA;" OwnerUserId="17902" LastEditorUserId="298" LastEditDate="2023-11-17T17:26:51.043" LastActivityDate="2023-11-21T14:11:06.600" Title="Convert ensemble genes to gene names in my sigle cell signature matrix annotated with seurat in R?" Tags="|seurat|single-cell|ensembl|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21872" PostTypeId="2" ParentId="21871" CreationDate="2023-11-17T15:58:52.043" Score="1" Body="&lt;p&gt;I think BioMart will do the gene ID -&amp;gt; gene name conversion you're after:&#xA;&lt;a href=&quot;https://www.ensembl.org/info/data/biomart/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ensembl.org/info/data/biomart/index.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;To use BioMart to convert the gene IDs to gene names, you can use the BioMaRt bioconductor R package. Installation and running instructions can be found here: ensembl.org/info/data/biomart/biomart_r_package.html&lt;/p&gt;&#xA;&lt;p&gt;For this query, you will need to use the human genes dataset:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ensembl = useEnsembl(biomart=&amp;quot;ensembl&amp;quot;, dataset=&amp;quot;hsapiens_gene_ensembl&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then use your list of gene IDs as the 'Filter', with list of IDs as the values:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;filters = 'ensembl_gene_id' &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Most human protein-coding genes have an associated HGNC symbol, and these are used in Ensembl. Therefore, you can select the 'gene_name' attribute to retrieve the human gene names, and this will usually correspond to the HGNC gene names, as stored in the Ensembl database.&lt;/p&gt;&#xA;&lt;p&gt;More information on gene names can be found in the Ensembl documentation: &lt;a href=&quot;https://ensembl.org/info/genome/genebuild/gene_names.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://ensembl.org/info/genome/genebuild/gene_names.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6500" LastEditorUserId="298" LastEditDate="2023-11-21T14:11:06.600" LastActivityDate="2023-11-21T14:11:06.600" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21874" PostTypeId="1" CreationDate="2023-11-17T21:32:16.507" Score="1" ViewCount="33" Body="&lt;p&gt;Does anyone know of a way to programmatically get panther protein class terms and IDs for a gene list, in a way that the classification structure is preserved (PARENT CLASS + CHILD CLASS)?&lt;/p&gt;&#xA;&lt;p&gt;i.e. For a given gene, I would want to get the list of classes, in away that I can then use the top classification for downstream analyses.&lt;/p&gt;&#xA;&lt;p&gt;GENEX, PC00244 zinc finger transcription factor &amp;gt; PC00218 DNA-binding transcription factor &amp;gt; PC00264 gene-specific transcriptional regulator&lt;/p&gt;&#xA;" OwnerUserId="9870" LastActivityDate="2023-11-23T00:36:46.120" Title="PANTHER FUNCTIONAL CLASSIFICATION in R" Tags="|r|public-databases|functional-annotation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21875" PostTypeId="1" CreationDate="2023-11-17T21:52:06.083" Score="0" ViewCount="53" Body="&lt;p&gt;After reading through papers and the code, I'm still not clear if AlphaFold and the likes do actually calculate the 3d structure. My current feeling they are more like information retrieval engines. They find statistical 'similarity' features between the primary structures of the query protein and a database protein (or a family of those) with a known 3d structure, and that known fold is the 'calculated' 3d structure. This would mean that if there is no 'similar' protein with a known 3d fold then the program cannot 'predict' anything.&#xA;I realize this could be a stupid layman question but here it goes.&lt;/p&gt;&#xA;" OwnerUserId="18577" LastEditorUserId="2108" LastEditDate="2023-11-19T20:03:31.937" LastActivityDate="2023-12-20T11:07:31.037" Title="Does AlphaFold actually calculates the fold?" Tags="|protein-structure|alphafold|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21876" PostTypeId="1" CreationDate="2023-11-17T21:56:39.360" Score="1" ViewCount="55" Body="&lt;p&gt;I need to get a fasta sequence of a certain gene for a certain worm strain that is different from reference. I have a reference genome, BAM for the strain of interest, and coordinates of the gene. I know that vcftools can convert bam to fasta, but I do not see if I can do it only for a certain coordinates (like, for example, chr1:10100500-10200500). I converted the whole bam to fasta and then I tried using bedtools getfasta, but my system ran out of space. Is there a more rational way to do it?&lt;/p&gt;&#xA;" OwnerUserId="18578" LastEditorUserId="18578" LastEditDate="2023-11-17T22:02:45.240" LastActivityDate="2023-11-20T10:56:45.683" Title="Get a certain gene sequence from bam/vcf and reference" Tags="|fasta|samtools|bedtools|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21877" PostTypeId="1" CreationDate="2023-11-18T14:34:12.317" Score="2" ViewCount="78" Body="&lt;p&gt;I have some gwas summary statistics in GRCh38 that I want to lift to GRCh37. I am trying to liftover in R using this code:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(tidyverse)&#xA;library(magrittr)&#xA;library(data.table)&#xA;library(rtracklayer)&#xA;library(GenomicRanges)&#xA;&#xA;rm(list=ls())&#xA;&#xA;gwas_data &amp;lt;- fread(&amp;quot;/gwas_sumstats_allchr.txt&amp;quot;)&#xA;chain_file &amp;lt;- &amp;quot;/chain_files/hg38ToHg19.over.chain&amp;quot;&#xA;chain &amp;lt;- import.chain(chain_file)&#xA;&#xA;# Convert to GRanges object (assuming GENPOS is 1-based)&#xA;gwas_ranges &amp;lt;- GRanges(&#xA;  seqnames = Rle(gwas_data&lt;span class=&quot;math-container&quot;&gt;$CHROM),&#xA;  ranges = IRanges(start = gwas_data$&lt;/span&gt;GENPOS - 1, end = gwas_data$GENPOS) # Convert to 0-based start&#xA;)&#xA;&#xA;gwas_ranges&lt;span class=&quot;math-container&quot;&gt;$seqnames &amp;lt;- paste0(&quot;chr&quot;, gwas_ranges$&lt;/span&gt;seqnames)&#xA;&#xA;# Perform the liftover&#xA;liftover_output &amp;lt;- liftOver(gwas_ranges, chain)&#xA;&#xA;# Extract successful liftover&#xA;liftover_success &amp;lt;- liftover_output$`output`&#xA;&#xA;# Convert the results back to a data frame (convert positions back to 1-based)&#xA;result &amp;lt;- data.frame(&#xA;  CHROM = as.character(seqnames(liftover_success)),&#xA;  GENPOS = start(liftover_success) + 1  # Convert back to 1-based position&#xA;)&#xA;&#xA;# Print or save the result&#xA;print(head(result))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However this doesn't work and I am getting:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;liftover_output &amp;lt;- liftOver(gwas_ranges, chain)&#xA;Discarding unchained sequences: 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 1, 20, 21, 22, 2, 3, 4, 5, 6, 7, 8, 9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My chain file seems fine on checking it (downloaded from UCSC genome browser), I align my chromosomes to have &amp;quot;chr&amp;quot; like the 38 chain does, and my gwas data is indeed in GRCh38 - what else could be going wrong?&lt;/p&gt;&#xA;&lt;p&gt;My &lt;code&gt;gwas_ranges&lt;/code&gt; looks like:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;head(gwas_ranges)&#xA;GRanges object with 6 ranges and 1 metadata column:&#xA;seqnames      ranges strand |    seqnames&#xA;&amp;lt;Rle&amp;gt;   &amp;lt;IRanges&amp;gt;  &amp;lt;Rle&amp;gt; | &amp;lt;character&amp;gt;&#xA;[1]       10 10708-10709      * |         chr&#xA;[2]       10 10904-10905      * |         chr&#xA;[3]       10 10942-10943      * |         chr&#xA;[4]       10 12112-12113      * |         chr&#xA;[5]       10 13850-13851      * |         chr&#xA;[6]       10 14537-14538      * |         chr&#xA;-------   seqinfo: 22 sequences from an unspecified genome; no seqlengths&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I also tried doing the liftover in python using hail. This ran without error, but the liftover of SNPs on chromosome 10 position 1234 would lift to chromosome 18 12369 . Here's my hail attempt:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;import hail as hl&#xA;hl.stop()&#xA;hl.init()&#xA;&#xA;# Load the reference genomes and add the liftover chain file&#xA;rg38 = hl.get_reference(&amp;quot;GRCh38&amp;quot;)&#xA;rg37 = hl.get_reference(&amp;quot;GRCh37&amp;quot;)&#xA;local_chain_file = &amp;quot;/references_grch38_to_grch37.over.chain.gz&amp;quot; #downloaded from Hail's GCP&#xA;rg38.add_liftover(local_chain_file, rg37)&#xA;&#xA;# Import GWAS data&#xA;gwas = hl.import_table('/gwas_sumstats_allchr.txt', &#xA;                       types={'CHROM': 'str', 'GENPOS': 'int'})&#xA;&#xA;# Standardize chromosome names in your GWAS data to match GRCh38&#xA;gwas = gwas.annotate(CHROM=hl.if_else(hl.literal('chr').contains(gwas.CHROM), gwas.CHROM, hl.literal('chr') + gwas.CHROM))&#xA;&#xA;# Perform the liftover from GRCh38 to GRCh37&#xA;gwas = gwas.annotate(&#xA;    locus_grch38=hl.locus(gwas.CHROM, gwas.GENPOS, reference_genome='GRCh38'),&#xA;    locus_grch37=hl.liftover(hl.locus(gwas.CHROM, gwas.GENPOS, reference_genome='GRCh38'), 'GRCh37')&#xA;)&#xA;&#xA;# Add new columns for CHROM_38 and GENPOS_38&#xA;gwas = gwas.annotate(&#xA;    CHROM_38=hl.if_else(hl.is_defined(gwas.locus_grch37), gwas.locus_grch37.contig, hl.null('str')),&#xA;    GENPOS_38=hl.if_else(hl.is_defined(gwas.locus_grch37), gwas.locus_grch37.position, hl.null('int'))&#xA;)&#xA;&#xA;gwas = gwas.drop('locus_grch38', 'locus_grch37')&#xA;&#xA;# Export the results&#xA;gwas.export('/gwas_sumstats_allchr_GRCh37.txt')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Here's some of the output I get from the hail run:&lt;/p&gt;&#xA;&lt;p&gt;Head of &lt;code&gt;gwas_sumstats_GRCh38txt&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CHROM   GENPOS  ID  ALLELE0 ALLELE1 A1FREQ  INFO    N   TEST    BETA    SE  CHISQ   LOG10P  p   MAF&#xA;10  10709   10:10709:G:A    G   A   0.0140284   0.420904    56210   ADD 0.143464    0.0384744   13.904  3.71581 0.000192393324862147    0.0140284&#xA;10  10905   10:10905:G:A    G   A   0.0352519   0.418554    56210   ADD -0.000990946    0.0246439   0.0016169   0.0141582   0.967925206925404   0.0352519&#xA;10  10943   10:10943:G:C    G   C   0.0653494   0.469298    56210   ADD -0.0130032  0.0173743   0.56013 0.342745    0.454208230910946   0.0653494&#xA;10  12113   10:12113:AC:A   AC  A   0.0390721   0.431839    56210   ADD 0.0386709   0.0230572   2.81293 1.02915 0.0935082652260488  0.0390721&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Head of &lt;code&gt;gwas_sumstats_GRCh37.txt&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CHROM   GENPOS  ID  A1FREQ  INFO    N   TEST    BETA    SE  CHISQ   LOG10P  p   MAF original_locus&#xA;18  10905   18:10905:G:A    0.0140284   0.420904    56210   ADD 0.143464    0.0384744   13.904  3.71581 0.000192393324862147    0.0140284   chr10:10709&#xA;18  11258   18:11258:G:A    0.0352519   0.418554    56210   ADD -0.000990946    0.0246439   0.0016169   0.0141582   0.967925206925404   0.0352519   chr10:10905&#xA;18  11296   18:11296:G:C    0.0653494   0.469298    56210   ADD -0.0130032  0.0173743   0.56013 0.342745    0.454208230910946   0.0653494   chr10:10943&#xA;18  12467   18:12467:AC:A   0.0390721   0.431839    56210   ADD 0.0386709   0.0230572   2.81293 1.02915 0.0935082652260488  0.0390721   chr10:12113&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4242" LastEditorUserId="650" LastEditDate="2023-11-20T19:30:00.677" LastActivityDate="2024-04-04T11:30:15.150" Title="How to perform liftover from 38 to 37 in R?" Tags="|r|python|genomics|gwas|liftover|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21878" PostTypeId="2" ParentId="18760" CreationDate="2023-11-18T20:43:33.953" Score="0" Body="&lt;p&gt;OK , visualizing both proteins in &lt;strong&gt;PyMOL&lt;/strong&gt; , using &lt;code&gt;set cartoon_gap_cutoff = 0 &lt;/code&gt; ,&lt;/p&gt;&#xA;&lt;p&gt;I get :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/shOJ8.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/shOJ8.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Apparently the two conformations , have more than one difference in various loops.&lt;/p&gt;&#xA;&lt;p&gt;I modify the code above , thanks to your hint :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;internal_coords.IC_Chain.MaxPeptideBond = 5.0&#xA;ic_chain = internal_coords.IC_Chain(structure, verbose=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My code :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import Bio&#xA;&#xA;print('########## Biopython  VERSION ##########################################')&#xA;print('\n                BIOPYTHON_VERSION : ', Bio.__version__)&#xA;print('\n########################################################################\n\n')&#xA;&#xA;&#xA;from Bio.PDB import *&#xA;&#xA;&#xA;def extract_torsion_angles(structure):&#xA;    &#xA;    print('\nSTRUCTURE ---&amp;gt; ' , structure)&#xA;    &#xA;    ic_chain = internal_coords.IC_Chain(structure, verbose=True)&#xA;    &#xA;    ic_chain.atom_to_internal_coordinates()&#xA; &#xA;    d = ic_chain.dihedra&#xA; &#xA;    &amp;quot;&amp;quot;&amp;quot; &#xA;    Update the keys of a dihedra dictionary to make searching easier.&#xA; &#xA;    The keys of the dihedra dictionary (as obtained from &#xA;    extract_torsion_angles()) are 4-tuples where each element is&#xA;    an AtomKey containing 5 fields (respos, icode, resname, atm,&#xA;    altloc, occ). We use the residue position, residue name and &#xA;    atom name in order to create a new representation for each&#xA;    AtomKey. The final dihedra dictionary will have 4-tuple keys&#xA;    where each element of the tuple is one such string.&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA; &#xA;    new_d = {}&#xA;    for key, value in sorted(d.items(), key=lambda x: x[0]):&#xA;        new_key = []&#xA;        for atom_key in key:&#xA;            # Keep: residue position + resiude name + atom name.&#xA;            atom_key_str = (atom_key.akl[0] + '_' + &#xA;                atom_key.akl[2] + '_' + atom_key.akl[3])&#xA; &#xA;            new_key.append(atom_key_str)&#xA; &#xA;        # new_d[tuple(new_key)] = value&#xA;        &#xA;        new_d[tuple(new_key)] = tuple((value , structure.get_full_id()))&#xA; &#xA;    return new_d&#xA; &#xA; &#xA;f1 = '18760-BIOINFORMATICS-protein1.pdb'&#xA;f2 = '18760-BIOINFORMATICS-protein2.pdb'&#xA;&#xA; &#xA;parser = PDBParser(PERMISSIVE=1, QUIET=0)&#xA;s1 = parser.get_structure('protein1', f1)&#xA;s2 = parser.get_structure('protein2', f2)&#xA; &#xA;c1 = list(s1.get_chains())&#xA;c2 = list(s2.get_chains())&#xA; &#xA;print('Chains: ', len(c1), ' ', len(c2))  &#xA; &#xA;r1 = list(c1[0].get_residues())&#xA;r2 = list(c2[0].get_residues())&#xA; &#xA;print('Residues: ', len(r1), ' ', len(r2))&#xA; &#xA;for i in range(len(r1)):&#xA;    resname1 = r1[i].get_resname()&#xA;    resname2 = r2[i].get_resname()&#xA; &#xA;    if resname1 != resname2:&#xA;        print('Residues not matching')&#xA;        break&#xA; &#xA;    atoms1 = list(r1[i].get_atoms())&#xA;    atoms2 = list(r2[i].get_atoms())&#xA; &#xA;    atom_names1 = sorted([atom.get_id() for atom in atoms1])&#xA;    atom_names2 = sorted([atom.get_id() for atom in atoms2])&#xA; &#xA;    if atom_names1 != atom_names2:&#xA;        print('Different atoms!')&#xA; &#xA;&#xA;&#xA;def get_no_tors_diff(structure1 , structure2) :&#xA;    &#xA;    pept_bond = 1.4&#xA;    &#xA;    while pept_bond &amp;lt; 10.0 :&#xA;        &#xA;        pept_bond = round(pept_bond , 2)&#xA;        &#xA;        print('\n\n Peptide Bond Lenght = ' , pept_bond)&#xA;        &#xA;        internal_coords.IC_Chain.MaxPeptideBond = pept_bond&#xA;        &#xA;        d1 = extract_torsion_angles(s1)&#xA;        d2 = extract_torsion_angles(s2)&#xA;         &#xA;        print('Torsion angles: ', len(d1), ' ', len(d2))&#xA;         &#xA;        diff_keys = sorted((set(d1.keys()) ^ set(d2.keys())) , key = lambda tup : tup[0].split('_')[0])&#xA;&#xA;        print('diff_keys lenght : ', len(diff_keys))&#xA;&#xA;        print(diff_keys , type(diff_keys))&#xA;&#xA;        for i in diff_keys :&#xA;            &#xA;            print(i , i[0].split('_')[0] , type(i))&#xA;&#xA;&#xA;&#xA;        diff_keys_values = {}&#xA;&#xA;        for i in diff_keys :&#xA;            &#xA;            if i in d1.keys() :&#xA;                &#xA;                diff_keys_values[i] = d1[i]&#xA;                &#xA;            &#xA;            if i in d2.keys()  :&#xA;                &#xA;                diff_keys_values[i] = d2[i]&#xA;                &#xA;            else:&#xA;                &#xA;                pass&#xA;&#xA;&#xA;        print('diff_keys_values lenght : ', len(diff_keys_values))&#xA;            &#xA;        for i in diff_keys_values :&#xA;            &#xA;            print(i , diff_keys_values[i][1])&#xA;    &#xA;            &#xA;        print('\n\n')&#xA;        &#xA;        if len(diff_keys) == 0 :&#xA;            &#xA;            print('\n\n Peptide Bond Lenght without differences = ' , pept_bond)&#xA;            &#xA;            return pept_bond&#xA;        &#xA;        pept_bond += 0.1&#xA;        &#xA;        &#xA;    print('\n\n Cannot get rid of differences up to 10.0A in Peptide Bond ')&#xA;    &#xA;    return None&#xA;    &#xA;&#xA;&#xA;print('get_no_tors_diff(structure1 , structure2)' , get_no_tors_diff(s1 , s2))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;First and last bit of output :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Chains:  1   1&#xA;Residues:  313   313&#xA;&#xA;&#xA; Peptide Bond Lenght =  1.4&#xA;&#xA;STRUCTURE ---&amp;gt;  &amp;lt;Structure id=protein1&amp;gt;&#xA;chain break at GLY  6  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at LEU  37  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at ARG  38  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at LEU  75  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at LEU  84  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at VAL  92  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at PHE  160  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at PRO  163  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at VAL  164  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at HIS  198  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at PHE  224  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at PHE  245  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at LYS  246  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at LEU  251  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at LYS  279  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at ALA  285  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at ARG  296  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;&#xA;STRUCTURE ---&amp;gt;  &amp;lt;Structure id=protein2&amp;gt;&#xA;chain break at PHE  1  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at GLY  6  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at LEU  37  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at ARG  38  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at LEU  75  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at LEU  84  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at VAL  92  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at PHE  160  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at PRO  163  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at VAL  164  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at HIS  198  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at PHE  224  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at PHE  245  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at LYS  246  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at LEU  251  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at ALA  278  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at ALA  285  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;chain break at ARG  296  due to MaxPeptideBond (1.4 angstroms) exceeded&#xA;Torsion angles:  2504   2501&#xA;diff_keys lenght :  9&#xA;[('0_N_C', '1_F_N', '1_F_CA', '1_F_C'), ('0_N_CA', '0_N_C', '1_F_N', '1_F_CA'), ('0_N_N', '0_N_CA', '0_N_C', '1_F_N'), ('277_D_C', '278_A_N', '278_A_CA', '278_A_C'), ('277_D_N', '277_D_CA', '277_D_C', '278_A_N'), ('277_D_CA', '277_D_C', '278_A_N', '278_A_CA'), ('278_A_N', '278_A_CA', '278_A_C', '279_K_N'), ('278_A_C', '279_K_N', '279_K_CA', '279_K_C'), ('278_A_CA', '278_A_C', '279_K_N', '279_K_CA')] &amp;lt;class 'list'&amp;gt;&#xA;('0_N_C', '1_F_N', '1_F_CA', '1_F_C') 0 &amp;lt;class 'tuple'&amp;gt;&#xA;('0_N_CA', '0_N_C', '1_F_N', '1_F_CA') 0 &amp;lt;class 'tuple'&amp;gt;&#xA;('0_N_N', '0_N_CA', '0_N_C', '1_F_N') 0 &amp;lt;class 'tuple'&amp;gt;&#xA;('277_D_C', '278_A_N', '278_A_CA', '278_A_C') 277 &amp;lt;class 'tuple'&amp;gt;&#xA;('277_D_N', '277_D_CA', '277_D_C', '278_A_N') 277 &amp;lt;class 'tuple'&amp;gt;&#xA;('277_D_CA', '277_D_C', '278_A_N', '278_A_CA') 277 &amp;lt;class 'tuple'&amp;gt;&#xA;('278_A_N', '278_A_CA', '278_A_C', '279_K_N') 278 &amp;lt;class 'tuple'&amp;gt;&#xA;('278_A_C', '279_K_N', '279_K_CA', '279_K_C') 278 &amp;lt;class 'tuple'&amp;gt;&#xA;('278_A_CA', '278_A_C', '279_K_N', '279_K_CA') 278 &amp;lt;class 'tuple'&amp;gt;&#xA;diff_keys_values lenght :  9&#xA;('0_N_C', '1_F_N', '1_F_CA', '1_F_C') ('protein1',)&#xA;('0_N_CA', '0_N_C', '1_F_N', '1_F_CA') ('protein1',)&#xA;('0_N_N', '0_N_CA', '0_N_C', '1_F_N') ('protein1',)&#xA;('277_D_C', '278_A_N', '278_A_CA', '278_A_C') ('protein1',)&#xA;('277_D_N', '277_D_CA', '277_D_C', '278_A_N') ('protein1',)&#xA;('277_D_CA', '277_D_C', '278_A_N', '278_A_CA') ('protein1',)&#xA;('278_A_N', '278_A_CA', '278_A_C', '279_K_N') ('protein2',)&#xA;('278_A_C', '279_K_N', '279_K_CA', '279_K_C') ('protein2',)&#xA;('278_A_CA', '278_A_C', '279_K_N', '279_K_CA') ('protein2',)&#xA;&#xA;.........................&#xA;.................&#xA;..........&#xA;......&#xA; Peptide Bond Lenght =  4.6&#xA;&#xA;STRUCTURE ---&amp;gt;  &amp;lt;Structure id=protein1&amp;gt;&#xA;chain break at PHE  160  due to MaxPeptideBond (4.6 angstroms) exceeded&#xA;chain break at PHE  245  due to MaxPeptideBond (4.6 angstroms) exceeded&#xA;chain break at ARG  296  due to MaxPeptideBond (4.6 angstroms) exceeded&#xA;&#xA;STRUCTURE ---&amp;gt;  &amp;lt;Structure id=protein2&amp;gt;&#xA;chain break at LEU  84  due to MaxPeptideBond (4.6 angstroms) exceeded&#xA;chain break at PHE  160  due to MaxPeptideBond (4.6 angstroms) exceeded&#xA;chain break at PHE  245  due to MaxPeptideBond (4.6 angstroms) exceeded&#xA;chain break at ARG  296  due to MaxPeptideBond (4.6 angstroms) exceeded&#xA;Torsion angles:  2546   2543&#xA;diff_keys lenght :  3&#xA;[('83_K_CA', '83_K_C', '84_L_N', '84_L_CA'), ('83_K_N', '83_K_CA', '83_K_C', '84_L_N'), ('83_K_C', '84_L_N', '84_L_CA', '84_L_C')] &amp;lt;class 'list'&amp;gt;&#xA;('83_K_CA', '83_K_C', '84_L_N', '84_L_CA') 83 &amp;lt;class 'tuple'&amp;gt;&#xA;('83_K_N', '83_K_CA', '83_K_C', '84_L_N') 83 &amp;lt;class 'tuple'&amp;gt;&#xA;('83_K_C', '84_L_N', '84_L_CA', '84_L_C') 83 &amp;lt;class 'tuple'&amp;gt;&#xA;diff_keys_values lenght :  3&#xA;('83_K_CA', '83_K_C', '84_L_N', '84_L_CA') ('protein1',)&#xA;('83_K_N', '83_K_CA', '83_K_C', '84_L_N') ('protein1',)&#xA;('83_K_C', '84_L_N', '84_L_CA', '84_L_C') ('protein1',)&#xA;&#xA;&#xA;&#xA;&#xA;&#xA; Peptide Bond Lenght =  4.7&#xA;&#xA;STRUCTURE ---&amp;gt;  &amp;lt;Structure id=protein1&amp;gt;&#xA;chain break at PHE  160  due to MaxPeptideBond (4.7 angstroms) exceeded&#xA;chain break at PHE  245  due to MaxPeptideBond (4.7 angstroms) exceeded&#xA;chain break at ARG  296  due to MaxPeptideBond (4.7 angstroms) exceeded&#xA;&#xA;STRUCTURE ---&amp;gt;  &amp;lt;Structure id=protein2&amp;gt;&#xA;chain break at PHE  160  due to MaxPeptideBond (4.7 angstroms) exceeded&#xA;chain break at PHE  245  due to MaxPeptideBond (4.7 angstroms) exceeded&#xA;chain break at ARG  296  due to MaxPeptideBond (4.7 angstroms) exceeded&#xA;Torsion angles:  2546   2546&#xA;diff_keys lenght :  0&#xA;[] &amp;lt;class 'list'&amp;gt;&#xA;diff_keys_values lenght :  0&#xA;&#xA;&#xA;&#xA;&#xA;&#xA; Peptide Bond Lenght without differences =  4.7&#xA;get_no_tors_diff(structure1 , structure2) 4.7&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Using &lt;code&gt;internal_coords.IC_Chain.MaxPeptideBond = 4.7&lt;/code&gt; both models have same Chain Breaks at resi 160, 245 e 296 and are described by 2546 dihedral angles.&lt;/p&gt;&#xA;&lt;p&gt;My question now is how, if feasible , to force PyMOL to paint petide bonds of such lenght both in cartoons and lines/sticks mode.&lt;/p&gt;&#xA;&lt;p&gt;Second question if Biopython has a tool to detect chain breaks in Structures other than just printing it out with &lt;code&gt;ic_chain = internal_coords.IC_Chain(structure, verbose=True)&lt;/code&gt; , asking this because setting &lt;code&gt;internal_coords.IC_Chain.MaxPeptideBond = 50&lt;/code&gt; gets rid of all the chainbreaks but don't have a way to monitor their disappearing. Pics below show chain breaks at 4.7.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/sWe0l.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sWe0l.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2023-11-18T21:32:42.120" LastActivityDate="2023-11-18T21:32:42.120" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21879" PostTypeId="2" ParentId="21865" CreationDate="2023-11-18T23:58:06.893" Score="1" Body="&lt;p&gt;Having given some thought to this, I'd still use machine learning. I would attempt to augment the negative controls however. Is there a suitable published data set? R program is caret but it high barrier to entry without appropriate support.&lt;/p&gt;&#xA;&lt;p&gt;250 samples is just about enough. If you can augment that to 350 with negatives ... should be enough. What you do is dump all your data (&amp;quot;features&amp;quot;) into a single record, including the methylation data and use toxic element exposure as the training target. The output you want is called &amp;quot;feature selection&amp;quot;, what you are wanting is methylation sites to carry more &amp;quot;weight&amp;quot; than any other feature. Then you start other analytics to see what it's associating with. Its not trivial getting it working and a sceptic would complain about the lack of significance in the results, but its a good starting point.&lt;/p&gt;&#xA;&lt;p&gt;GWAS might work, just considering methylation vs toxic element exposure. SEM, I've never used it so I don't know.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-11-18T23:58:06.893" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21880" PostTypeId="1" CreationDate="2023-11-19T01:23:37.887" Score="0" ViewCount="49" Body="&lt;p&gt;Semi-supervised learning is when a portion of the training data set is labeled, and the rest is unlabeled; the unlabeled data is used to improved learning performance / accuracy.&lt;/p&gt;&#xA;&lt;p&gt;Are there (i.e., have there been) applications of semi-supervised learning to genomics? If so, what are they? If not, is there a reason why not?&lt;/p&gt;&#xA;" OwnerUserId="17430" LastEditorUserId="3694" LastEditDate="2023-11-19T21:36:34.523" LastActivityDate="2023-11-20T00:33:55.313" Title="Are there applications of semi-supervised learning to genomics?" Tags="|genomics|machine-learning|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21881" PostTypeId="2" ParentId="21876" CreationDate="2023-11-19T12:01:35.780" Score="4" Body="&lt;p&gt;The first approach that comes to mind is to cut the bam file and then convert the already cut bam to fasta:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -hb chr1:10100500-10200500 &amp;gt; small.bam&#xA;samtools fasta small.bam &amp;gt; small.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(note that I am not certain about the &lt;code&gt;samtools fasta&lt;/code&gt; command, I have never used it and I don't know if it will produce many sequences, one for each haplotype supported by reads; please test this.)&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2023-11-20T10:56:45.683" LastActivityDate="2023-11-20T10:56:45.683" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21882" PostTypeId="2" ParentId="21880" CreationDate="2023-11-20T00:28:53.900" Score="1" Body="&lt;p&gt;&lt;strong&gt;Personal view&lt;/strong&gt; Semi-supervised learning is basically unsupervised learning with weights, so the &amp;quot;label&amp;quot; or &amp;quot;category&amp;quot; can be quantified rather than approximately clustered as occurs in unsupervised learning.&lt;/p&gt;&#xA;&lt;p&gt;Unsupervised learning is PCA and tSNE and other multi-variate approaches. The advantages of semi-supervised learning is unlike unsupervised learning the feature in the data that shows greatest proportionate variance can be precisely identified and all features ranked; thus there aren't loads of components where the result could be anywhere.&lt;/p&gt;&#xA;&lt;p&gt;Sure I've interpreted semi-supervised learning and found it was not useful for any comparative purposes. Basically the same variance Eigen-vector weight observed in one gene in one unique environment, is the strongest Eignen-vector weight for the same gene in a completely different environment. However, these are two independent semi-supervised learning analyses. Thus, it's not measuring comparative difference but relative variance between all genes in that given experiment.&lt;/p&gt;&#xA;&lt;p&gt;Thus I personally didn't find it useful for my metric, HOWEVER such a result is good for some people where the overall structure of the variance identified by Eigen vectors is very similar  between contrasting environments. Basically it's a comparison to all other genes in the analysis. To get it working you'd need to combine all genes in all environments into a single data set.&lt;/p&gt;&#xA;&lt;p&gt;That would work, but ensure that the label for each feature (e.g. gene expression) comprised the environment and the gene. What the statistical theoretical validity of such a result would be - I haven't a clue - but it should give some interesting results. To be honest, if I'm not sure about the theoretical validity, I will not be alone.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-11-20T00:33:55.313" LastActivityDate="2023-11-20T00:33:55.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21883" PostTypeId="1" CreationDate="2023-11-20T10:11:07.940" Score="0" ViewCount="58" Body="&lt;p&gt;I have the Uniref90 dataset containing around 76M sequences. I have made modifications to all sequences based on a reduced amino acid alphabet.&lt;/p&gt;&#xA;&lt;p&gt;I am trying to cluster it again at 90% identity. I am using  &lt;code&gt;mmseq cluster&lt;/code&gt; on a 64 core, 500GB RAM machine. It has been more than 12h now.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Starting prefiltering scores calculation (step 1 of 1)&#xA;Query db start 1 to 74390789&#xA;Target db start 1 to 74390789&#xA;[=&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How long does it normally take to cluster such a database? I know I can use &lt;code&gt;linclust&lt;/code&gt; instead to cluster in linear time, but it might not be what I need in this case.&lt;/p&gt;&#xA;&lt;p&gt;I am using &lt;code&gt;mmseq cluster&lt;/code&gt; as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mmseqs createdb &amp;quot;uniref90-$1.fasta&amp;quot; &amp;quot;uniref90-DB-$1&amp;quot;&#xA;mmseqs cluster &amp;quot;uniref90-DB-$1&amp;quot;  &amp;quot;uniref90-DB-$1_clu&amp;quot; tmp/ --cov-mode 2 -c 0.8 --min-seq-id 0.9&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18584" LastEditorUserId="18584" LastEditDate="2023-11-20T16:31:42.730" LastActivityDate="2023-12-21T14:00:55.077" Title="How long should it take to cluster Uniref?" Tags="|proteins|clustering|uniprot|" AnswerCount="2" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21884" PostTypeId="2" ParentId="21875" CreationDate="2023-11-20T10:36:03.430" Score="0" Body="&lt;p&gt;Alphafold calculates the &amp;quot;folds&amp;quot; throughout a protein, thus its protein structural prediction. It uses convolution-based calculation, which is a modification of co-variance. The reason it doesn't use co-variance directly is complicated, but can result in artefacts.&lt;/p&gt;&#xA;&lt;p&gt;Thus it looks for &amp;quot;simultaneous&amp;quot; amino acid mutations in protein alignments and believes those represent close spatial proximity - based on previous supervised learning of proteins, comprising an alignment and a known structure.&lt;/p&gt;&#xA;&lt;p&gt;It has won two CASP competitions (world competition for protein folding prediction). The training is likely deeper both in data sets and ANN/CNN/RNN &amp;quot;layers&amp;quot; (deep learning jargon) than previous protein structural prediction models and hence its success. &amp;quot;Deepmind&amp;quot; the company behind it will be Tensorflow experts, rather than structural biologists.&lt;/p&gt;&#xA;&lt;p&gt;I don't use it but have seen its structural accuracy first-hand. The accuracy statistic RMSD was impressive.&lt;/p&gt;&#xA;&lt;p&gt;It is important to note it is built on the back of a long term effort by this community as to calculating protein structural prediction. So it's not &amp;quot;radical technology&amp;quot;, they've generally increased the accuracy of established methods. Again they've likely done this with more extensive training.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-11-20T10:52:16.933" LastActivityDate="2023-11-20T10:52:16.933" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21885" PostTypeId="1" AcceptedAnswerId="21886" CreationDate="2023-11-20T11:28:10.730" Score="3" ViewCount="1251" Body="&lt;p&gt;I am currently using the WGCNA package for some analysis and it seems the Horvath lab site is down. Does anyone know of anywhere else I can access the tutorial documents?&lt;/p&gt;&#xA;" OwnerUserId="18396" LastActivityDate="2024-01-18T16:33:22.283" Title="Where to access the WGCNA tutorial documents: Horvath lab site down" Tags="|wgcna|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21886" PostTypeId="2" ParentId="21885" CreationDate="2023-11-20T14:31:32.857" Score="10" Body="&lt;p&gt;Apologies for the web site trouble, we are working with UCLA to restore the web pages. In the meantime, I uploaded the tutorials to Dropbox at &lt;a href=&quot;https://www.dropbox.com/scl/fo/4vqfiysan6rlurfo2pbnk/h?rlkey=thqg8wlpdn4spu3ihjuc1kmlu&amp;amp;dl=0&quot; rel=&quot;noreferrer&quot;&gt;https://www.dropbox.com/scl/fo/4vqfiysan6rlurfo2pbnk/h?rlkey=thqg8wlpdn4spu3ihjuc1kmlu&amp;amp;dl=0&lt;/a&gt;. I suggest downloading the entire folder, unzipping and opening the file index.html in a browser. You should see a stripped down but usable version of the web page and all tutorial sections should be accessible.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2023-11-20T14:31:32.857" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21887" PostTypeId="1" CreationDate="2023-11-20T18:00:30.540" Score="2" ViewCount="24" Body="&lt;p&gt;I'm a newcomer to the field of antibodies, and I'm seeking guidance on obtaining 3D structures for my molecular simulations. After browsing the PDB databank, I noticed that there are Fab and Fc structures available. My specific interest lies in investigating the structural dynamics in the entire antibody (AB) and on the hinge region.&lt;/p&gt;&#xA;&lt;p&gt;I have a few questions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it common practice to attempt building the entire antibody structure and hinge regions from PDB files for fragments, or are there recommended alternatives?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Are there comprehensive PDB structures available for different IgG and therapeutic antibodies that include information on the whole structure and hinge regions? Especially for Bevacizumab and Rituximab?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Does it make sense to select single segments, e.g., CH1, VH, VL, for analysis? Would one expect these structures to be reasonably stable?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Additionally, I'd appreciate any insights into common pitfalls and any recommendations you might have for someone new to this field.&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance for your time and assistance.&lt;/p&gt;&#xA;&lt;p&gt;Best regards,&#xA;Albert&lt;/p&gt;&#xA;" OwnerUserId="18586" LastActivityDate="2023-11-20T18:00:30.540" Title="(Complete) Stuctural Models of Antibodies: Combining Fc and Fab from pdb to Build Hinge Region? Or what?" Tags="|pdb|structural-biology|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21889" PostTypeId="2" ParentId="21874" CreationDate="2023-11-21T07:22:20.887" Score="1" Body="&lt;p&gt;One approach is to get the gene's sequence (DNA/AA) and use the interproscan API  (e.g., &lt;a href=&quot;https://www.ebi.ac.uk/Tools/common/tools/help/index.html?tool=iprscan5&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/Tools/common/tools/help/index.html?tool=iprscan5&lt;/a&gt;) with the httr2 library in R to POST gene sequence(s) and GET the resulting Panther annotation. Make sure to include &lt;code&gt;Panther&lt;/code&gt; in the &lt;code&gt;appl&lt;/code&gt; parameter for the post request.&lt;/p&gt;&#xA;&lt;p&gt;Here's a function that does this for a single sequence.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;submit_fasta &amp;lt;- function(&#xA;    fasta,&#xA;    path_outfolder,&#xA;    email,&#xA;    stype = &amp;quot;n&amp;quot;,&#xA;    appl = c(&amp;quot;Panther&amp;quot;),&#xA;    title = 'stackoverflow'&#xA;    ) {&#xA;  stopifnot(&amp;quot;stype must be 'n' for nucleotide seqs or 'p' for protein seqs&amp;quot; = stype %in% c('n', 'p'))&#xA;  url_post &amp;lt;- &amp;quot;https://www.ebi.ac.uk/Tools/services/rest/iprscan5/run&amp;quot;&#xA;  chr_seq &amp;lt;- as.character(fasta)&#xA;  req &amp;lt;- httr2::request(url_post)&#xA;  req &amp;lt;- req |&amp;gt; httr2::req_body_form(&#xA;    &amp;quot;email&amp;quot; = email,&#xA;    &amp;quot;title&amp;quot; = title,&#xA;    &amp;quot;goterms&amp;quot; = &amp;quot;false&amp;quot;,&#xA;    &amp;quot;pathways&amp;quot; = &amp;quot;false&amp;quot;,&#xA;    &amp;quot;stype&amp;quot; = stype,&#xA;    &amp;quot;appl&amp;quot; = paste0(appl, collapse = &amp;quot;,&amp;quot;),&#xA;    &amp;quot;sequence&amp;quot; = chr_seq&#xA;  )&#xA;  req &amp;lt;- req |&amp;gt; httr2::req_headers(&amp;quot;Accept&amp;quot; = &amp;quot;text/plain&amp;quot;, &amp;quot;Content-type&amp;quot; = &amp;quot;application/x-www-form-urlencoded&amp;quot;)&#xA;  resp_post &amp;lt;- req |&amp;gt; httr2::req_perform()&#xA;  chr_retrieval_code &amp;lt;- resp_post$body |&amp;gt; rawToChar()&#xA;  url_get &amp;lt;- &amp;quot;https://www.ebi.ac.uk/Tools/services/rest/iprscan5/result&amp;quot;&#xA;  req &amp;lt;- httr2::request(url_get)&#xA;  req &amp;lt;- req |&amp;gt; httr2::req_url_path_append(chr_retrieval_code)&#xA;  req &amp;lt;- req |&amp;gt; httr2::req_url_path_append(&amp;quot;tsv&amp;quot;)&#xA;  req &amp;lt;- req |&amp;gt; httr2::req_retry(&#xA;    max_tries = 15L, backoff = ~ 30L,&#xA;    is_transient = function(resp) {httr2::resp_status(resp) == 400L}&#xA;  )&#xA;  req &amp;lt;- req |&amp;gt; httr2::req_headers(Accept = NULL)&#xA;  req |&amp;gt; httr2::req_dry_run(quiet = FALSE)&#xA;  resp_get &amp;lt;- req |&amp;gt; httr2::req_perform()&#xA;  cat('status code:', resp_get$status_code, '\n')&#xA;  result &amp;lt;- resp_get$body |&amp;gt; rawToChar()&#xA;  result &amp;lt;- read.table(text = result, sep = '\t')&#xA;  return(result)&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Importantly, the fasta input parameter is expected to be XString object from the Biostrings package. However, the code could easily be adjusted. The sequence eventually needs to be coerced to a string for the POST request.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#fasta &amp;lt;- Biostrings::readDNAStringSet('seq.fa')&#xA;#resp &amp;lt;- submit_fasta(fasta = fasta_single, path_outfolder = 'x', email = 'test@gmail.com', title = 'stackoverflow')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;edit&lt;/strong&gt;: set the &lt;code&gt;is_transient&lt;/code&gt; param to handle the actual transient error case where the data is not yet available (response code = 400). Also, it's important to note there's a &amp;quot;status&amp;quot; endpoint &lt;a href=&quot;https://www.ebi.ac.uk/Tools/common/tools/help/index.html?tool=iprscan5#/Status&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ebi.ac.uk/Tools/common/tools/help/index.html?tool=iprscan5#/Status&lt;/a&gt;) which is probably much more appropriate than pinging the data endpoint until either success or reaching max tries. However, I hope this answer suffices!&lt;/p&gt;&#xA;" OwnerUserId="16938" LastEditorUserId="16938" LastEditDate="2023-11-23T00:36:46.120" LastActivityDate="2023-11-23T00:36:46.120" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21890" PostTypeId="2" ParentId="21883" CreationDate="2023-11-21T08:05:33.697" Score="0" Body="&lt;p&gt;When I want to estimate the time taken to do something big, I subset my input data to something that takes a few minutes or less, and calculate forward from there. If there's alignment involved, it's usually a good idea to work it out for multiple dataset sizes, as there could be a linear relationship between input size and time taken, square relationship, or worse.&lt;/p&gt;&#xA;&lt;p&gt;Have you looked at how long it takes for, say, 1000 sequences, 2000 sequences, and 5000 sequences?&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-11-21T08:05:33.697" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21891" PostTypeId="2" ParentId="21883" CreationDate="2023-11-21T12:51:00.787" Score="0" Body="&lt;p&gt;The &lt;strong&gt;mmseqs2&lt;/strong&gt; code should be specifically compiled and optimised for your system. &lt;strong&gt;mmseqs2&lt;/strong&gt; optimisation is automated and might be available via Spack* (I'm not sure its actually done for mmseqs2), it is definitely available for &lt;strong&gt;EasyBuild&lt;/strong&gt;. I wouldn't advise attempting a manual optimisation unless you've prior experience.&lt;/p&gt;&#xA;&lt;p&gt;How much time this will save is more difficult to assess but 50% faster is a conservative assessment. This approach is good practice for a job like this size and its good for the associated report, its standard HPC practice. If you're doing cloud computation (i.e. $$$), HPC optimisation is essential. AWS make serious money from suboptimal HPC practice (in fairness to them they also do a lot of development work to minimise costs).&lt;/p&gt;&#xA;&lt;p&gt;Finally, if you're an expert in optimisation you can save &lt;strong&gt;serious&lt;/strong&gt; computational time, but this is a major investment of your personal development time and technique. The rationale of how and why is complicated.&lt;/p&gt;&#xA;&lt;p&gt;*, its very likely available on AWS.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;You might simply consider its been running for a few days already so HPC technique isn't needed for this specific job. However, you'll know for the future.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-11-21T13:17:11.010" LastActivityDate="2023-11-21T13:17:11.010" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21892" PostTypeId="1" CreationDate="2023-11-22T15:34:24.930" Score="1" ViewCount="86" Body="&lt;p&gt;I need to create a barplot with error bars, my data consists of 2 groups called &amp;quot;Genot&amp;quot;, with 3 sub-groups &amp;quot;Suberin&amp;quot; for each, associated with a value &amp;quot;Percent&amp;quot;.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;summary_data &amp;lt;- FY %&amp;gt;%&#xA;  group_by(Genot, Suberin) %&amp;gt;%&#xA;  summarise(&#xA;    Mean_Percent = mean(Percent),&#xA;    SD_Percent = sd(Percent))&#xA;&#xA;&#xA;# plot&#xA;ggplot(summary_data, aes(x = Genot, y = Mean_Percent, fill=Suberin)) +&#xA;  geom_bar(stat = 'identity', aes(fill = Suberin)) +&#xA;  geom_errorbar(data = summary_data,&#xA;                aes(x = Genot, ymax = Mean_Percent + SD_Percent, ymin = Mean_Percent - SD_Percent), &#xA;                , position = &amp;quot;identity&amp;quot;,width = 0.2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/qBK8J.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/qBK8J.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;My main problem is that the error bars are placed from 0, but I have to order my data as the following to get the graph in the order I want.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;FY$Suberin &amp;lt;- ordered(FY$Suberin,&#xA;                    levels = c(&amp;quot;Con&amp;quot;, &amp;quot;Patchy&amp;quot;,&amp;quot;Non&amp;quot; ))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I can't figure out a way to describe the positions&lt;/p&gt;&#xA;" OwnerUserId="18598" LastEditorUserId="650" LastEditDate="2023-12-04T20:28:35.490" LastActivityDate="2023-12-04T20:28:35.490" Title="stacked barplot with error bars" Tags="|r|ggplot2|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21893" PostTypeId="1" CreationDate="2023-11-23T14:02:44.013" Score="1" ViewCount="104" Body="&lt;p&gt;I need help. I am not able to find any reference for Eigen_phred_coding prediction tools score from &lt;a href=&quot;http://database.liulab.science/dbNSFP&quot; rel=&quot;nofollow noreferrer&quot;&gt;dbNSFP&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I know that it may be scaled in a similar manner to CADD_phred - ranking raw values from highest to smallest; get rank of that value in the queue; phred scale that value with the same formula as sequencing quality score.&lt;/p&gt;&#xA;&lt;p&gt;But I couldn't find the phred scaling of Eigen values directly, not to mention its suggested cutoff (same as CADD_phred &amp;gt; 15 or different?).&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;How are the Eigen scores transformed into Eigen_phred?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Is it in same manner as Cadd_phred?&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;If so, can same cutoff be used in both cases?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="14021" LastEditorUserId="298" LastEditDate="2023-11-23T14:25:48.733" LastActivityDate="2023-11-23T14:25:48.733" Title="Eigen phred score interpretation" Tags="|variation|dbnsfp|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21895" PostTypeId="1" CreationDate="2023-11-23T17:56:09.183" Score="0" ViewCount="44" Body="&lt;p&gt;I want to benchmark my DNA sequencing pipeline. In order to do that I want some gold-standard files in every step(.vcf,.bam,.fastq). I want to generate/simulate a bam file of reads for a given set of vcf. How to do that?&lt;/p&gt;&#xA;&lt;p&gt;The GIAB project has gold standard vcf, bam, and fastq. But they are not perfect for pipeline testing. Their size is quite big. Also, they are real data. My question was how to generate simulated Bam from vcf.&lt;/p&gt;&#xA;" OwnerUserId="10296" LastEditorUserId="73" LastEditDate="2023-11-24T05:28:04.753" LastActivityDate="2023-11-24T05:28:04.753" Title="Generate simulated bam for certain snps" Tags="|sam|vcf|format-conversion|simulated-data|conversion|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21897" PostTypeId="1" CreationDate="2023-11-24T17:39:33.870" Score="0" ViewCount="53" Body="&lt;p&gt;I ran the &amp;quot;funcotator&amp;quot; step from the GATK somatic short variant discovery (snvs + indels) pipeline and used the output of &lt;code&gt;filtermutectcalls&lt;/code&gt; as input, but I obtain this error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;17:00:18.816 INFO  NativeLibraryLoader - Loading libgkl_compression.so from jar:file:/gatk/gatk-package-4.3.0.0-local.jar!/com/intel/gkl/native/libgkl_compression.so&#xA;17:00:19.105 INFO  Funcotator - ------------------------------------------------------------&#xA;17:00:19.106 INFO  Funcotator - The Genome Analysis Toolkit (GATK) v4.3.0.0&#xA;17:00:19.106 INFO  Funcotator - For support and documentation go to https://software.broadinstitute.org/gatk/&#xA;17:00:19.106 INFO  Funcotator - Executing as rita.soares@compute-19 on Linux v5.4.0-148-generic amd64&#xA;17:00:19.106 INFO  Funcotator - Java runtime: OpenJDK 64-Bit Server VM v1.8.0_242-8u242-b08-0ubuntu3~18.04-b08&#xA;17:00:19.106 INFO  Funcotator - Start Date/Time: November 24, 2023 5:00:18 PM GMT&#xA;17:00:19.106 INFO  Funcotator - ------------------------------------------------------------&#xA;17:00:19.106 INFO  Funcotator - ------------------------------------------------------------&#xA;17:00:19.107 INFO  Funcotator - HTSJDK Version: 3.0.1&#xA;17:00:19.107 INFO  Funcotator - Picard Version: 2.27.5&#xA;17:00:19.107 INFO  Funcotator - Built for Spark Version: 2.4.5&#xA;17:00:19.107 INFO  Funcotator - HTSJDK Defaults.COMPRESSION_LEVEL : 2&#xA;17:00:19.107 INFO  Funcotator - HTSJDK Defaults.USE_ASYNC_IO_READ_FOR_SAMTOOLS : false&#xA;17:00:19.107 INFO  Funcotator - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_SAMTOOLS : true&#xA;17:00:19.108 INFO  Funcotator - HTSJDK Defaults.USE_ASYNC_IO_WRITE_FOR_TRIBBLE : false&#xA;17:00:19.108 INFO  Funcotator - Deflater: IntelDeflater&#xA;17:00:19.108 INFO  Funcotator - Inflater: IntelInflater&#xA;17:00:19.108 INFO  Funcotator - GCS max retries/reopens: 20&#xA;17:00:19.108 INFO  Funcotator - Requester pays: disabled&#xA;17:00:19.108 INFO  Funcotator - Initializing engine&#xA;17:00:19.577 INFO  FeatureManager - Using codec VCFCodec to read file file:///mnt/nfs/lobo/IMM-NFS/cfranco/MCPP-Lung/DNA/Variant_Calling/FilterMutectCalls/10LC_tumor_S2.filtered.vcf.gz&#xA;17:00:19.880 INFO  Funcotator - Done initializing engine&#xA;17:00:19.880 INFO  Funcotator - Skipping sequence dictionary validation.&#xA;17:00:19.880 INFO  Funcotator - Processing user transcripts/defaults/overrides...&#xA;17:00:19.881 INFO  Funcotator - Initializing data sources...&#xA;17:00:19.883 INFO  DataSourceUtils - Initializing data sources from directory: /mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s&#xA;17:00:19.953 INFO  DataSourceUtils - Data sources version: 1.7.2020429s&#xA;17:00:19.953 INFO  DataSourceUtils - Data sources source: ftp://gsapubftp-anonymous@ftp.broadinstitute.org/bundle/funcotator/funcotator_dataSources.v1.7.20200429s.tar.gz&#xA;17:00:19.953 INFO  DataSourceUtils - Data sources alternate source: gs://broad-public-datasets/funcotator/funcotator_dataSources.v1.7.20200429s.tar.gz&#xA;17:00:20.435 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/gencode.v34.annotation.REORDERED.gtf -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/gencode/hg38/gencode.v34.annotation.REORDERED.gtf&#xA;17:00:20.437 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/gencode.v34.pc_transcripts.fa -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/gencode/hg38/gencode.v34.pc_transcripts.fa&#xA;17:00:20.513 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/hg38_All_20180418.vcf.gz -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/dbsnp/hg38/hg38_All_20180418.vcf.gz&#xA;17:00:20.584 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/Familial_Cancer_Genes.no_dupes.tsv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/familial/hg38/Familial_Cancer_Genes.no_dupes.tsv&#xA;17:00:20.711 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/dnaRepairGenes.20180524T145835.csv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/dna_repair_genes/hg38/dnaRepairGenes.20180524T145835.csv&#xA;17:00:20.779 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/cosmic_fusion.tsv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/cosmic_fusion/hg38/cosmic_fusion.tsv&#xA;17:00:20.911 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/hgnc_download_Nov302017.tsv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/hgnc/hg38/hgnc_download_Nov302017.tsv&#xA;17:00:20.967 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/simple_uniprot_Dec012014.tsv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/simple_uniprot/hg38/simple_uniprot_Dec012014.tsv&#xA;17:00:21.034 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/CancerGeneCensus_Table_1_full_2012-03-15.txt -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/cancer_gene_census/hg38/CancerGeneCensus_Table_1_full_2012-03-15.txt&#xA;17:00:21.073 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/achilles_lineage_results.import.txt -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/achilles/hg38/achilles_lineage_results.import.txt&#xA;17:00:21.161 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/clinvar_20180429_hg38.vcf -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/clinvar/hg38/clinvar_20180429_hg38.vcf&#xA;17:00:21.171 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/gencode_xhgnc_v90_38.hg38.tsv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/gencode_xhgnc/hg38/gencode_xhgnc_v90_38.hg38.tsv&#xA;17:00:21.187 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/Cosmic.db -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/cosmic/hg38/Cosmic.db&#xA;17:00:21.341 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/oreganno.tsv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/oreganno/hg38/oreganno.tsv&#xA;17:00:21.357 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/gencode_xrefseq_v90_38.tsv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/gencode_xrefseq/hg38/gencode_xrefseq_v90_38.tsv&#xA;17:00:21.378 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/cosmic_tissue.tsv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/cosmic_tissue/hg38/cosmic_tissue.tsv&#xA;17:00:21.379 INFO  Funcotator - Finalizing data sources (this step can be long if data sources are cloud-based)...&#xA;17:00:21.379 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/gencode.v34.annotation.REORDERED.gtf -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/gencode/hg38/gencode.v34.annotation.REORDERED.gtf&#xA;17:00:21.380 INFO  DataSourceUtils - Setting lookahead cache for data source: Gencode : 100000&#xA;17:00:21.553 INFO  FeatureManager - Using codec GencodeGtfCodec to read file file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/gencode/hg38/gencode.v34.annotation.REORDERED.gtf&#xA;17:00:21.691 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/gencode.v34.pc_transcripts.fa -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/gencode/hg38/gencode.v34.pc_transcripts.fa&#xA;17:00:27.190 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/hg38_All_20180418.vcf.gz -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/dbsnp/hg38/hg38_All_20180418.vcf.gz&#xA;17:00:27.190 INFO  DataSourceUtils - Setting lookahead cache for data source: dbSNP : 100000&#xA;17:00:27.412 INFO  FeatureManager - Using codec VCFCodec to read file file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/dbsnp/hg38/hg38_All_20180418.vcf.gz&#xA;17:00:27.563 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/hg38_All_20180418.vcf.gz -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/dbsnp/hg38/hg38_All_20180418.vcf.gz&#xA;17:00:27.666 INFO  FeatureManager - Using codec VCFCodec to read file file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/dbsnp/hg38/hg38_All_20180418.vcf.gz&#xA;17:00:27.814 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/Familial_Cancer_Genes.no_dupes.tsv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/familial/hg38/Familial_Cancer_Genes.no_dupes.tsv&#xA;17:00:27.837 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/dnaRepairGenes.20180524T145835.csv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/dna_repair_genes/hg38/dnaRepairGenes.20180524T145835.csv&#xA;17:00:27.853 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/cosmic_fusion.tsv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/cosmic_fusion/hg38/cosmic_fusion.tsv&#xA;17:00:27.875 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/hgnc_download_Nov302017.tsv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/hgnc/hg38/hgnc_download_Nov302017.tsv&#xA;17:00:28.142 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/simple_uniprot_Dec012014.tsv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/simple_uniprot/hg38/simple_uniprot_Dec012014.tsv&#xA;17:00:28.400 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/CancerGeneCensus_Table_1_full_2012-03-15.txt -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/cancer_gene_census/hg38/CancerGeneCensus_Table_1_full_2012-03-15.txt&#xA;17:00:28.426 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/achilles_lineage_results.import.txt -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/achilles/hg38/achilles_lineage_results.import.txt&#xA;17:00:28.449 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/clinvar_20180429_hg38.vcf -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/clinvar/hg38/clinvar_20180429_hg38.vcf&#xA;17:00:28.449 INFO  DataSourceUtils - Setting lookahead cache for data source: ClinVar_VCF : 100000&#xA;17:00:28.502 INFO  FeatureManager - Using codec VCFCodec to read file file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/clinvar/hg38/clinvar_20180429_hg38.vcf&#xA;17:00:28.702 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/clinvar_20180429_hg38.vcf -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/clinvar/hg38/clinvar_20180429_hg38.vcf&#xA;17:00:28.780 INFO  FeatureManager - Using codec VCFCodec to read file file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/clinvar/hg38/clinvar_20180429_hg38.vcf&#xA;17:00:28.835 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/gencode_xhgnc_v90_38.hg38.tsv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/gencode_xhgnc/hg38/gencode_xhgnc_v90_38.hg38.tsv&#xA;17:00:29.651 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/Cosmic.db -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/cosmic/hg38/Cosmic.db&#xA;17:00:29.807 INFO  DataSourceUtils - Setting lookahead cache for data source: Oreganno : 100000&#xA;17:00:29.828 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/oreganno.tsv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/oreganno/hg38/oreganno.tsv&#xA;17:00:29.871 INFO  FeatureManager - Using codec XsvLocatableTableCodec to read file file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/oreganno/hg38/oreganno.config&#xA;17:00:30.155 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/oreganno.tsv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/oreganno/hg38/oreganno.tsv&#xA;17:00:30.158 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/oreganno.tsv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/oreganno/hg38/oreganno.tsv&#xA;WARNING 2023-11-24 17:00:30     AsciiLineReader Creating an indexable source for an AsciiFeatureCodec using a stream that is neither a PositionalBufferedStream nor a BlockCompressedInputStream&#xA;17:00:30.168 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/gencode_xrefseq_v90_38.tsv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/gencode_xrefseq/hg38/gencode_xrefseq_v90_38.tsv&#xA;17:00:30.284 INFO  DataSourceUtils - Resolved data source file path: file:///home/rita.soares/cosmic_tissue.tsv -&amp;gt; file:///mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s/cosmic_tissue/hg38/cosmic_tissue.tsv&#xA;17:00:30.398 INFO  Funcotator - Initializing Funcotator Engine...&#xA;17:00:30.400 INFO  FuncotatorEngine - Using given VCF and Reference.  No conversion required.&#xA;17:00:30.402 INFO  Funcotator - Creating a MAF file for output: file:/mnt/nfs/lobo/IMM-NFS/cfranco/MCPP-Lung/DNA/Variant_Calling/FUNCOTATOR/10LC_tumor_S2.FiltPASS.maf&#xA;17:00:30.420 INFO  ProgressMeter - Starting traversal&#xA;17:00:30.420 INFO  ProgressMeter -        Current Locus  Elapsed Minutes    Variants Processed  Variants/Minute&#xA;17:00:42.145 INFO  ProgressMeter -         chr1:1129652              0.2                  1000           5117.3&#xA;17:00:52.621 INFO  ProgressMeter -         chr1:1544800              0.4                  2000           5405.4&#xA;17:01:03.582 INFO  ProgressMeter -         chr1:2088265              0.6                  3000           5427.9&#xA;17:01:23.573 INFO  ProgressMeter -         chr1:2868977              0.9                  5000           5644.2&#xA;17:01:45.697 INFO  ProgressMeter -         chr1:3708722              1.3                  7000           5579.4&#xA;17:01:47.516 ERROR GencodeFuncotationFactory - Problem creating a GencodeFuncotation on transcript ENST00000581512.2 for variant: chr1:3783110-3783112(TAA* -&amp;gt; T): Variant overlaps transcript but is not completely contained within it. Funcotator cannot currently handle this case. Transcript: ENST00000581512.2  Variant: [VC Unknown @ chr1:3783110-3783112 Q. of type=INDEL alleles=[TAA*, T] attr={AS_FilterStatus=[weak_evidence, contamination], AS_SB_TABLE=[0, 0|0, 0], DP=1, ECNT=2, GERMQ=1, MBQ=[0, 31], MFRL=[0, 259], MMQ=[60, 60], MPOS=66, POPAF=2.21, ROQ=93, RPA=[15, 13], RU=A, STR=true, STRQ=93, TLOD=4.20} GT=GT:AD:AF:DP:F1R2:F2R1:FAD:PGT:PID:PS:SB     0|1:0,1:0.667:1:0,0:0,1:0,1:0|1:3783099_C_T:3783099:0,0,1,0 filters=haplotype,germline,contamination,weak_evidence&#xA;&#xA;... (CODE OMITTED BECAUSE OF CHARACTERS LIMINATION OF THE BODY OF THE QUESTION)&#xA;&#xA;17:06:03.884 ERROR GencodeFuncotationFactory - Problem creating a GencodeFuncotation on transcript ENST00000375071.4 for variant: chr1:20589096-20589097(TC* -&amp;gt; T): Variant overlaps transcript but is not completely contained within it. Funcotator cannot currently handle this case. Transcript: ENST00000375071.4  Variant: [VC Unknown @ chr1:20589096-20589097 Q. of type=INDEL alleles=[TC*, T] attr={AS_FilterStatus=SITE, AS_SB_TABLE=[108, 95|79, 72], DP=369, ECNT=2, GERMQ=1, MBQ=[20, 20], MFRL=[119, 123], MMQ=[60, 60], MPOS=26, PON=true, POPAF=0.256, ROQ=93, RPA=[3, 2], RU=C, STR=true, STRQ=93, TLOD=206.22} GT=GT:AD:AF:DP:F1R2:F2R1:FAD:SB   0/1:203,151:0.422:354:57,38:55,42:114,83:108,95,79,72 filters=panel_of_normals,germline&#xA;17:06:03.885 WARN  GencodeFuncotationFactory - Creating default GencodeFuncotation on transcript ENST00000375071.4 for problem variant: chr1:20589096-20589097(TC* -&amp;gt; T)&#xA;17:06:20.587 INFO  ProgressMeter -        chr1:21789645              5.8                 38000           6511.2&#xA;17:06:38.318 INFO  ProgressMeter -        chr1:22910935              6.1                 40000           6523.5&#xA;17:06:55.667 INFO  ProgressMeter -        chr1:24196391              6.4                 42000           6541.3&#xA;17:07:12.288 INFO  ProgressMeter -        chr1:25333744              6.7                 44000           6569.3&#xA;17:07:29.628 INFO  ProgressMeter -        chr1:26561968              7.0                 46000           6583.9&#xA;17:08:54.624 INFO  ProgressMeter -        chr1:27258140              8.4                 47000           5593.0&#xA;17:09:11.523 INFO  ProgressMeter -        chr1:28509457              8.7                 49000           5641.9&#xA;17:09:14.750 ERROR GencodeFuncotationFactory - Problem creating a GencodeFuncotation on transcript ENST00000294409.2 for variant: chr1:28714873-28714874(TA* -&amp;gt; T): Variant overlaps transcript but is not completely contained within it. Funcotator cannot currently handle this case. Transcript: ENST00000294409.2  Variant: [VC Unknown @ chr1:28714873-28714874 Q. of type=INDEL alleles=[TA*, T] attr={AS_FilterStatus=SITE, AS_SB_TABLE=[1, 8|0, 5], DP=14, ECNT=1, GERMQ=1, MBQ=[29, 29], MFRL=[277, 190], MMQ=[60, 60], MPOS=19, PON=true, POPAF=1.96, ROQ=93, RPA=[13, 12], RU=A, STR=true, STRQ=1, TLOD=6.90} GT=GT:AD:AF:DP:F1R2:F2R1:FAD:SB   0/1:9,5:0.377:14:1,1:4,1:8,5:1,8,0,5 filters=panel_of_normals,germline,slippage&#xA;17:09:14.750 WARN  GencodeFuncotationFactory - Creating default GencodeFuncotation on transcript ENST00000294409.2 for problem variant: chr1:28714873-28714874(TA* -&amp;gt; T)&#xA;17:09:28.156 INFO  ProgressMeter -        chr1:29742106              9.0                 51000           5690.5&#xA;17:09:44.363 INFO  ProgressMeter -        chr1:31000060              9.2                 53000           5740.7&#xA;17:10:02.660 INFO  ProgressMeter -        chr1:32309371              9.5                 55000           5766.8&#xA;17:10:21.404 INFO  ProgressMeter -        chr1:33875394              9.8                 57000           5787.0&#xA;17:10:39.604 INFO  ProgressMeter -        chr1:35564548             10.2                 59000           5811.1&#xA;17:11:13.558 INFO  ProgressMeter -        chr1:36959618             10.7                 61000           5690.8&#xA;17:11:30.930 INFO  ProgressMeter -        chr1:38301837             11.0                 63000           5722.9&#xA;17:11:48.409 INFO  ProgressMeter -        chr1:39672835             11.3                 65000           5752.3&#xA;17:12:04.997 INFO  ProgressMeter -        chr1:40888483             11.6                 67000           5787.7&#xA;17:12:22.171 INFO  ProgressMeter -        chr1:42389559             11.9                 69000           5816.6&#xA;17:13:07.894 INFO  ProgressMeter -        chr1:43856329             12.6                 71000           5624.0&#xA;17:13:25.958 INFO  ProgressMeter -        chr1:45154774             12.9                 73000           5647.7&#xA;17:13:43.676 INFO  ProgressMeter -        chr1:46612646             13.2                 75000           5672.8&#xA;17:14:00.829 INFO  ProgressMeter -        chr1:48197446             13.5                 77000           5700.8&#xA;17:14:19.050 INFO  ProgressMeter -        chr1:50725644             13.8                 79000           5720.3&#xA;17:14:29.607 ERROR GencodeFuncotationFactory - Problem creating a GencodeFuncotation on transcript ENST00000361556.9 for variant: chr1:51729808-51729817(GGGGGGTGAA* -&amp;gt; G): Variant overlaps transcript but is not completely contained within it. Funcotator cannot currently handle this case. Transcript: ENST00000361556.9  Variant: [VC Unknown @ chr1:51729808-51729817 Q. of type=INDEL alleles=[GGGGGGTGAA*, G] attr={AS_FilterStatus=SITE, AS_SB_TABLE=[44, 1|46, 0], DP=110, ECNT=1, GERMQ=1, MBQ=[37, 37], MFRL=[163, 167], MMQ=[60, 60], MPOS=35, POPAF=1.89, ROQ=93, RPA=[2, 1], RU=GGGGGTGAA, STR=true, STRQ=93, TLOD=175.52} GT=GT:AD:AF:DP:F1R2:F2R1:FAD:SB 0/1:45,46:0.514:91:23,24:12,22:44,46:44,1,46,0 filters=germline&#xA;17:14:29.607 WARN  GencodeFuncotationFactory - Creating default GencodeFuncotation on transcript ENST00000361556.9 for problem variant: chr1:51729808-51729817(GGGGGGTGAA* -&amp;gt; G)&#xA;17:14:29.715 INFO  ProgressMeter -        chr1:51744810             14.0                 80000           5719.1&#xA;17:14:47.317 INFO  ProgressMeter -        chr1:53112621             14.3                 82000           5741.6&#xA;17:15:03.760 INFO  ProgressMeter -        chr1:54313159             14.6                 84000           5770.9&#xA;17:15:20.710 INFO  ProgressMeter -        chr1:55702719             14.8                 86000           5795.9&#xA;17:15:38.363 INFO  ProgressMeter -        chr1:57618601             15.1                 88000           5815.3&#xA;17:15:55.381 INFO  ProgressMeter -        chr1:59379903             15.4                 90000           5838.1&#xA;17:16:13.770 INFO  ProgressMeter -        chr1:61650616             15.7                 92000           5851.5&#xA;17:16:29.978 INFO  ProgressMeter -        chr1:63193764             16.0                 94000           5877.7&#xA;17:17:46.249 INFO  ProgressMeter -        chr1:64973681             17.3                 96000           5560.8&#xA;17:18:29.297 INFO  ProgressMeter -        chr1:65743799             18.0                 97000           5394.5&#xA;17:18:44.743 INFO  VcfFuncotationFactory - ClinVar_VCF 20180429_hg38 cache hits/total: 0/138&#xA;17:18:44.746 INFO  VcfFuncotationFactory - dbSNP 9606_b151 cache hits/total: 0/63294&#xA;17:18:44.748 INFO  Funcotator - Shutting down engine&#xA;[November 24, 2023 5:18:44 PM GMT] org.broadinstitute.hellbender.tools.funcotator.Funcotator done. Elapsed time: 18.43 minutes.&#xA;Runtime.totalMemory()=2054684672&#xA;&#xA;org.broadinstitute.hellbender.exceptions.GATKException: Unable to query the database for geneName: LINC01702&#xA;        at org.broadinstitute.hellbender.tools.funcotator.dataSources.cosmic.CosmicFuncotationFactory.createFuncotationsOnVariant(CosmicFuncotationFactory.java:320)&#xA;        at org.broadinstitute.hellbender.tools.funcotator.DataSourceFuncotationFactory.determineFuncotations(DataSourceFuncotationFactory.java:245)&#xA;        at org.broadinstitute.hellbender.tools.funcotator.DataSourceFuncotationFactory.createFuncotations(DataSourceFuncotationFactory.java:211)&#xA;        at org.broadinstitute.hellbender.tools.funcotator.FuncotatorEngine.createFuncotationMapForVariant(FuncotatorEngine.java:178)&#xA;        at org.broadinstitute.hellbender.tools.funcotator.Funcotator.enqueueAndHandleVariant(Funcotator.java:924)&#xA;        at org.broadinstitute.hellbender.tools.funcotator.Funcotator.apply(Funcotator.java:878)&#xA;        at org.broadinstitute.hellbender.engine.VariantWalker.lambda&lt;span class=&quot;math-container&quot;&gt;$traverse$&lt;/span&gt;0(VariantWalker.java:104)&#xA;        at java.util.stream.ForEachOps&lt;span class=&quot;math-container&quot;&gt;$ForEachOp$&lt;/span&gt;OfRef.accept(ForEachOps.java:183)&#xA;        at java.util.stream.ReferencePipeline&lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt;1.accept(ReferencePipeline.java:193)&#xA;        at java.util.stream.ReferencePipeline&lt;span class=&quot;math-container&quot;&gt;$2$&lt;/span&gt;1.accept(ReferencePipeline.java:175)&#xA;        at java.util.stream.ReferencePipeline&lt;span class=&quot;math-container&quot;&gt;$3$&lt;/span&gt;1.accept(ReferencePipeline.java:193)&#xA;        at java.util.Iterator.forEachRemaining(Iterator.java:116)&#xA;        at java.util.Spliterators&lt;span class=&quot;math-container&quot;&gt;$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)&#xA;        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)&#xA;        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)&#xA;        at java.util.stream.ForEachOps$&lt;/span&gt;ForEachOp.evaluateSequential(ForEachOps.java:150)&#xA;        at java.util.stream.ForEachOps&lt;span class=&quot;math-container&quot;&gt;$ForEachOp$&lt;/span&gt;OfRef.evaluateSequential(ForEachOps.java:173)&#xA;        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)&#xA;        at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)&#xA;        at org.broadinstitute.hellbender.engine.VariantWalker.traverse(VariantWalker.java:102)&#xA;        at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1095)&#xA;        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:140)&#xA;        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:192)&#xA;        at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:211)&#xA;        at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)&#xA;        at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)&#xA;        at org.broadinstitute.hellbender.Main.main(Main.java:289)&#xA;Caused by: org.sqlite.SQLiteException: [SQLITE_IOERR_LOCK]  I/O error in the advisory file locking logic (disk I/O error)&#xA;        at org.sqlite.core.DB.newSQLException(DB.java:1030)&#xA;        at org.sqlite.core.DB.newSQLException(DB.java:1042)&#xA;        at org.sqlite.core.DB.execute(DB.java:884)&#xA;        at org.sqlite.core.CoreStatement.exec(CoreStatement.java:80)&#xA;        at org.sqlite.jdbc3.JDBC3Statement.executeQuery(JDBC3Statement.java:68)&#xA;        at org.broadinstitute.hellbender.tools.funcotator.dataSources.cosmic.CosmicFuncotationFactory.createFuncotationsOnVariant(CosmicFuncotationFactory.java:288)&#xA;        ... 26 more&#xA;Using GATK jar /gatk/gatk-package-4.3.0.0-local.jar&#xA;Running:&#xA;    java -Dsamjdk.use_async_io_read_samtools=false -Dsamjdk.use_async_io_write_samtools=true -Dsamjdk.use_async_io_write_tribble=false -Dsamjdk.compression_level=2 -jar /gatk/gatk-package-4.3.0.0-local.jar Funcotator -R /mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/bwa_prefiles/GRCh38.primary_assembly.genome.fa -V /mnt/nfs/lobo/IMM-NFS/cfranco/MCPP-Lung/DNA/Variant_Calling/FilterMutectCalls/10LC_tumor_S2.filtered.vcf.gz -O /mnt/nfs/lobo/IMM-NFS/cfranco/MCPP-Lung/DNA/Variant_Calling/FUNCOTATOR/10LC_tumor_S2.FiltPASS.maf --output-file-format MAF --ref-version hg38 --data-sources-path /mnt/nfs/lobo/SALMEIDA-NFS/lcosta/MCPP-Lung/gatk_files/funcotator_dataSources.v1.7.20200521s --disable-sequence-dictionary-validation&#xA;srun: error: compute-19: task 0: Exited with exit code 3&#xA;srun: launch/slurm: _step_signal: Terminating StepId=2949215.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17902" LastEditorUserId="298" LastEditDate="2023-11-24T18:15:44.947" LastActivityDate="2023-11-24T18:15:44.947" Title="ERROR IN FUNCOTATOR: [SQLITE_IOERR_LOCK] I/O error in the advisory file locking logic (disk I/O error)" Tags="|variant-calling|gatk|somatic|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21898" PostTypeId="1" CreationDate="2023-11-25T06:15:52.713" Score="1" ViewCount="58" Body="&lt;p&gt;I would like to know the acceptable error margin for the lowest free energy of a protein. In other words, say, both Group A and B have determined the lowest free energy of the same protein in their respective labs. What is an appropriate error margin so that I can compare these two results?&lt;/p&gt;&#xA;&lt;p&gt;I just came across &lt;a href=&quot;https://www.criver.com/eureka/free-energy-calculations-in-drug-discovery&quot; rel=&quot;nofollow noreferrer&quot;&gt;this article&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Now let me quote the following from the article.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Note that we generally require predictions to be within 1 kcal/mol to&#xA;be useful, since 1.4 kcal/mol equates to a 10-fold difference in&#xA;binding affinity.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Can we use this 1 kcal/mol as the acceptable error margin?&lt;/p&gt;&#xA;&lt;p&gt;Edited:&lt;/p&gt;&#xA;&lt;p&gt;from *(lehninger principles of biochemistry) and &lt;a href=&quot;https://www.quora.com/Is-Kd-the-dissociation-constant-for-a-given-protein-ligand-pair-directly-correlated-to-binding-affinity-kcal-mol-Rosetta-Energy-Units-etc&quot; rel=&quot;nofollow noreferrer&quot;&gt;Is Kd (the dissociation constant) for a given protein-ligand pair directly correlated to binding affinity (kcal/mol, Rosetta Energy Units, etc.)?&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/uCjDo.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/uCjDo.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18597" LastEditorUserId="10049" LastEditDate="2023-11-25T13:34:10.983" LastActivityDate="2023-11-25T13:34:10.983" Title="Energy error margin for protein structures" Tags="|proteins|protein-structure|proteomics|structural-biology|alphafold|" AnswerCount="0" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21899" PostTypeId="1" CreationDate="2023-11-25T07:14:26.693" Score="1" ViewCount="73" Body="&lt;p&gt;I am working on RNA seq analysis and I would like to know the following things:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Current Methods:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;downloaded genome fasta file for non-coding rna from ensembl and got the gtf file for hg38&lt;/li&gt;&#xA;&lt;li&gt;performed hist2 and got 17% alignment for my sample against ncrna genome&lt;/li&gt;&#xA;&lt;li&gt;sorted the file and then ran feature counts with the hg38 gtf file but got zero counts&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;what could be the issue that is causing zero counts?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;My bam file after sorting has the SRR ID at first whereas the GTF file has chromosome no: . Could this be the issue?&lt;/p&gt;&#xA;" OwnerUserId="18603" LastEditorUserId="16067" LastEditDate="2023-11-28T20:36:48.927" LastActivityDate="2023-11-29T16:02:24.407" Title="issue with rna seq analysis" Tags="|rna-seq|ngs|rna|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21902" PostTypeId="1" CreationDate="2023-11-26T21:55:14.050" Score="0" ViewCount="36" Body="&lt;p&gt;Is this possible to generate force fields for any residue in my .pdb file? I am trying to generate conformations for the proteins with arbitrary residues binded to it from .pdb files using OpenMM, but there are always errors like 'missing residue VAL, missing residue HEM, etc'. I tried to use pdbfixer from openmm library, skip\replace missing\nonstandard residues using GROMACS and generate .top .gro, but even after that it shows me errors or generates nonsence if I replace residues. I read some articles and looks like generating force fields is a complicated task. But still wondering is there any open-source framework\library\neural network that can generate such force fields?&lt;/p&gt;&#xA;&lt;p&gt;For example I follow this guide&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;http://docs.openmm.org/latest/userguide/application/02_running_sims.html#a-first-example&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://docs.openmm.org/latest/userguide/application/02_running_sims.html#a-first-example&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Whenever I try to run the following code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from openmm.app import *&#xA;from openmm import *&#xA;from openmm.unit import *&#xA;from sys import stdout&#xA;&#xA;pdb = PDBFile('input.pdb')&#xA;forcefield = ForceField('amber14-all.xml', 'amber14/tip3pfb.xml')&#xA;system = forcefield.createSystem(pdb.topology, nonbondedMethod=PME,&#xA;        nonbondedCutoff=1*nanometer, constraints=HBonds)&#xA;integrator = LangevinMiddleIntegrator(300*kelvin, 1/picosecond, 0.004*picoseconds)&#xA;simulation = Simulation(pdb.topology, system, integrator)&#xA;simulation.context.setPositions(pdb.positions)&#xA;simulation.minimizeEnergy()&#xA;simulation.reporters.append(PDBReporter('output.pdb', 1000))&#xA;simulation.reporters.append(StateDataReporter(stdout, 1000, step=True,&#xA;        potentialEnergy=True, temperature=True))&#xA;simulation.step(10000)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It shows&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;No template found for residue (number) (residue_name)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;for every .pdb file that I am trying to run and every ForceField parameter permutation.&#xA;As an example I tried to run simulations on this protein&#xA;&lt;a href=&quot;https://www.rcsb.org/structure/1a3n&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/structure/1a3n&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18610" LastEditorUserId="18610" LastEditDate="2023-11-28T16:21:41.707" LastActivityDate="2023-11-28T16:21:41.707" Title="Generating force-fields for ANY residue" Tags="|proteins|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21903" PostTypeId="2" ParentId="21834" CreationDate="2023-11-27T15:10:43.197" Score="1" Body="&lt;p&gt;The existing answer works I'm sure, however there are existing tools that do this or something quite similar.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;OpenBabel is able to &lt;a href=&quot;https://open-babel.readthedocs.io/en/latest/Command-line_tools/babel.html#options&quot; rel=&quot;nofollow noreferrer&quot;&gt;add charges&lt;/a&gt; using the option &lt;code&gt;--partialcharge&lt;/code&gt;, though the &lt;a href=&quot;https://open-babel.readthedocs.io/en/latest/Charges/charges.html?highlight=gasteiger&quot; rel=&quot;nofollow noreferrer&quot;&gt;docs are not impressive&lt;/a&gt;.&lt;/li&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://open-babel.readthedocs.io/en/latest/Charges/charges.html?highlight=gasteiger&quot; rel=&quot;nofollow noreferrer&quot;&gt;Autodock vina&lt;/a&gt; has a utility script that adds charges and extends the pdb to pdbqt, which is admittedly not exactly what you want.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;From the comments: @pippo1980 suggests this page which shows how to use the openbabel Python API to set charges&lt;/p&gt;&#xA;&lt;p&gt;Openbabel models &lt;strong&gt;(note that it includes &lt;code&gt;fromfile&lt;/code&gt; meaning you can supply your own)&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;obabel -L charges&#xA;eem    Assign Electronegativity Equilization Method (EEM) atomic partial charges. Bultinck B3LYP/6-31G*/MPA&#xA;eem2015ba    Assign Electronegativity Equilization Method (EEM) atomic partial charges. Cheminf B3LYP/6-311G/AIM&#xA;eem2015bm    Assign Electronegativity Equilization Method (EEM) atomic partial charges. Cheminf B3LYP/6-311G/MPA&#xA;eem2015bn    Assign Electronegativity Equilization Method (EEM) atomic partial charges. Cheminf B3LYP/6-311G/NPA&#xA;eem2015ha    Assign Electronegativity Equilization Method (EEM) atomic partial charges. Cheminf HF/6-311G/AIM&#xA;eem2015hm    Assign Electronegativity Equilization Method (EEM) atomic partial charges. Cheminf HF/6-311G/MPA&#xA;eem2015hn    Assign Electronegativity Equilization Method (EEM) atomic partial charges. Cheminf HF/6-311G/NPA&#xA;eqeq    Assign EQEq (charge equilibration) partial charges.&#xA;fromfile    Assign charges from file containing {'atom-name', charge} pairs&#xA;gasteiger    Assign Gasteiger-Marsili sigma partial charges&#xA;mmff94       Assign MMFF94 partial charges&#xA;none    Clear all partial charges&#xA;qeq    Assign QEq (charge equilibration) partial charges (Rappe and Goddard, 1991)&#xA;qtpie    Assign QTPIE (charge transfer, polarization and equilibration) partial charges (Chen and Martinez, 2007)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2023-11-27T19:54:06.120" LastActivityDate="2023-11-27T19:54:06.120" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21905" PostTypeId="1" CreationDate="2023-11-28T21:10:38.890" Score="1" ViewCount="34" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.reddit.com/r/bioinformatics/comments/18epsi0/what_are_some_bioinformatics_applications_for/&quot; rel=&quot;nofollow noreferrer&quot;&gt;reddit&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Tries are a data structure that can be used to efficiently store and search for strings.&#xA;Tries created from an &lt;em&gt;ordered sequence of strings&lt;/em&gt;  differ from the regular Tries in the following way:&lt;/p&gt;&#xA;&lt;p&gt;If there is a string of a certain character &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; at a character position &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; and is between two other strings that have both the same character &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt; where &lt;span class=&quot;math-container&quot;&gt;$p \neq&#xA; w$&lt;/span&gt; at position &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt;(and all their characters before position &lt;span class=&quot;math-container&quot;&gt;$x$&lt;/span&gt; are the same in all three strings) then the Trie must have an edge for the first occurrence of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;, an edge for &lt;span class=&quot;math-container&quot;&gt;$w$&lt;/span&gt; and an edge for the second occurrence of &lt;span class=&quot;math-container&quot;&gt;$p$&lt;/span&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I have recently written a paper regarding a novel data structure that is related to tries and I wish to connect it to some applications in bioinformatics since the paper itself presents only one application related to string algorithms.&lt;/p&gt;&#xA;&lt;p&gt;Since bioinformatics makes some use of string algorithms and since the structure is related to Tries I wish to ask you about the possible applications of Tries so I may find some application to my related data structure.&lt;/p&gt;&#xA;&lt;p&gt;What are the applications of Tries(data structure) of an &lt;em&gt;ordered&#xA;sequence of strings&lt;/em&gt; in bioinformatics?&lt;/p&gt;&#xA;" OwnerUserId="16115" LastEditorUserId="73" LastEditDate="2023-12-10T05:41:07.077" LastActivityDate="2023-12-10T05:41:07.077" Title="What are the applications of Tries(data structure) of an ordered sequence of strings in bioinformatics?" Tags="|phylogenetics|sequence-alignment|genomics|algorithms|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21906" PostTypeId="1" CreationDate="2023-11-29T01:42:40.283" Score="1" ViewCount="113" Body="&lt;p&gt;I am trying to align a bunch of paired sample fastq files using bwa mem.&lt;/p&gt;&#xA;&lt;p&gt;My original command was:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bwa mem -t 8 hg38.fa sample_read1.fq.gz sample_read2.fq.gz &amp;gt; sample_paired.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am running this on a HPC cluster.&lt;/p&gt;&#xA;&lt;p&gt;These files have approx. 25 million reads, so I initially anticipated that they might take a little bit of time; however I noticed that the programs seemed to hang - the sam file that I am redirecting the output to never became populated with any entries other than the header comments, even after hours of waiting.&lt;/p&gt;&#xA;&lt;p&gt;Noticing this, I decided to first test whether the most bare bones bwa mem functionality was working, and so I took just the first 1000 lines off one of the fastq files and tested whether it was working with:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bwa mem hg38.fa test_1000lines.fq.gz &amp;gt; test_output.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So this works. This creates a proper sam file almost instantaneously. However, I kept testing this behavior by increasing the number of lines I took off the fastq file from 1000 to 2000,3000,etc., and eventually realized that any fastq files I input with more than approximately 8400-8500 lines will make the program hang.&lt;/p&gt;&#xA;&lt;p&gt;If I run the following with a fastq file containing 8400 lines, the program will run and the sam file will be created in a few seconds:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bwa mem hg38.fa test_8400lines.fq.gz &amp;gt; test_output8400.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and this is the stdout output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[M::bwa_idx_load_from_disk] read 0 ALT contigs&#xA;[M::process] read 2100 sequences (313733 bp)...&#xA;[M::mem_process_seqs] Processed 2100 reads in 1.051 CPU sec, 1.054 real sec&#xA;[main] Version: 0.7.17-r1188&#xA;[main] CMD: bwa mem supporting_files/hg38.fa test_8-4k.fq.gz&#xA;[main] Real time: 4.231 sec; CPU: 4.178 sec&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, if I run the following with just 100 more lines in the fastq file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bwa mem hg38.fa test_8500lines.fq.gz &amp;gt; test_output8500.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The command will hang (at least, that is what it appears like it's doing) and all I will see on the terminal is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[M::bwa_idx_load_from_disk] read 0 ALT contigs&#xA;[M::process] read 2125 sequences (317468 bp)...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;followed by a blinking cursor.&lt;/p&gt;&#xA;&lt;p&gt;I've monitored the CPU and MEM usage with the &amp;quot;top&amp;quot; command while I was conducting these tests, and while the memory never goes above something like 15%, the CPU usage always goes to 100% when this hanging behavior happens. When the .sam files were created properly and the program didn't hang, the CPU usage was (for the very short amount of time it took for the program to run) like 60-70%.&lt;/p&gt;&#xA;&lt;p&gt;Whether the CPU usage going to 100% is causing the hanging, or the hanging it causing the CPU usage to 100%, I'm not sure.&lt;/p&gt;&#xA;&lt;p&gt;I'm completely out of ideas what the issue is here. Any ideas?&lt;/p&gt;&#xA;&lt;p&gt;EDIT: Although I originally suspected this to be an issue with my fastq file, I observed the stalling behavior with a different fastq file from a completely different source and ruled that out. It turns out the problem is something related to the reference index, since when I use a different set of reference indices (the hs38DH set) instead of the hg38 set that I had been using, the problem disappears.&lt;/p&gt;&#xA;" OwnerUserId="18628" LastEditorUserId="650" LastEditDate="2023-12-04T20:27:18.417" LastActivityDate="2023-12-04T20:27:18.417" Title="bwa mem hangs after a few thousand reads" Tags="|sequence-alignment|genome|assembly|variant-calling|bwa|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="21908" PostTypeId="2" ParentId="21899" CreationDate="2023-11-29T16:02:24.407" Score="3" Body="&lt;p&gt;Several issues:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;There is no non-coding RNA genome. There are non-coding genes but the genome fasta files from Ensembl are always the entire genome and this is what you should align against. Probably you downloaded a transcriptome fata file.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;If you aligned against a transcriptome fasta file (like ncRNAs) then it makes no sense to use a GTF file since GTF files have genome coordinates but you mapped against a selection of a transcriptome fasta, hence you have (specific) transcriptome coordinates. GTF and these coordinates are not compatible.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Solution: Align your data against the entire genome (hg38), then use the GTF to get counts for all annotated genes, then do subsetting to specific genes if that is what you need.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-11-29T16:02:24.407" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21909" PostTypeId="2" ParentId="21892" CreationDate="2023-11-29T19:57:31.253" Score="1" Body="&lt;p&gt;(Following from comments)&lt;/p&gt;&#xA;&lt;p&gt;I am 99% sure that, in the second code block, you are reordering only that one column and the rest of the columns are remaining unordered. That means that that column will be permuted with respect to the rest of the data, which I doubt is what you intend. I'd suggest instead running this, which will reorder the whole dataframe:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# not actually run cause I don't have the data, check it!&#xA;FY &amp;lt;- FY[&#xA;         order(factor(FY$Suberin,&#xA;               levels = c(&amp;quot;Con&amp;quot;, &amp;quot;Patchy&amp;quot;,&amp;quot;Non&amp;quot; )&#xA;         ),&#xA;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Consider this analogous example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; test = data.frame(&#xA;  &amp;quot;one&amp;quot;=c(1,2,3), &#xA;  &amp;quot;two&amp;quot;=c(6,5,4), &#xA;  &amp;quot;three&amp;quot;=c(&amp;quot;this&amp;quot;, &amp;quot;that&amp;quot;, &amp;quot;which&amp;quot;)&#xA;)&#xA;&amp;gt; test&#xA;  one two three&#xA;1   1   6  this&#xA;2   2   5  that&#xA;3   3   4 which&#xA;&amp;gt; test&lt;span class=&quot;math-container&quot;&gt;$three = ordered(test$&lt;/span&gt;three, levels=c(&amp;quot;that&amp;quot;, &amp;quot;this&amp;quot;, &amp;quot;which&amp;quot;))&#xA;# only column &amp;quot;three&amp;quot; is reordered!!!&#xA;&amp;gt; test&#xA;  one two three&#xA;1   1   6  this&#xA;2   2   5  that&#xA;3   3   4 which&#xA;&#xA;# THIS reorders the whole dataframe&#xA;&amp;gt; test[order(test$three),]&#xA;  one two three&#xA;2   2   5  that&#xA;1   1   6  this&#xA;3   3   4 which&#xA;&#xA;# impose a specific ordering by making it a factor, maybe there's a better way&#xA;&amp;gt; test[order(factor(test$three, levels=c(&amp;quot;that&amp;quot;, &amp;quot;which&amp;quot;, &amp;quot;this&amp;quot;))),]&#xA;  one two three&#xA;2   2   5  that&#xA;3   3   4 which&#xA;1   1   6  this&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I don't know that that impacts your error bar issue. But the underlying data will certainly be affected and possibly made inaccurate by permuting only one column.&lt;/p&gt;&#xA;&lt;p&gt;Another thing that's a little weird is that you're computing &lt;code&gt;Mean_Percent&lt;/code&gt; in two places: first, in your &lt;code&gt;summarise&lt;/code&gt; expression, and then again when you actually make the plot with &lt;code&gt;geom_bar(stat=&amp;quot;identity&amp;quot;...&lt;/code&gt;, which will fill the axes you've set up with newly estimated values based on the relative frequencies of the &lt;code&gt;Suberin&lt;/code&gt; column. So you aren't plotting the &lt;code&gt;Mean_Percent&lt;/code&gt; values that you are getting from &lt;code&gt;summarise&lt;/code&gt;, as far as I can tell.&lt;/p&gt;&#xA;&lt;p&gt;I'm a little scared of the way that you are computing mean/sd percentages and stacking them, especially not knowing what the data look like. I'd recommend confirming that your &lt;code&gt;Mean_Percent&lt;/code&gt; that you compute with &lt;code&gt;summarise&lt;/code&gt; adds up to 100%; for the actual bars in the barplot you are letting &lt;code&gt;geom_bar&lt;/code&gt; recompute the proportions directly from the data, so I don't think the fact that it apparently adds up to 100% in the plot is trustworthy. To summarize: it'd make sense to confirm that what's in your dataframe matches what's in the plot since they are separately computed.&lt;/p&gt;&#xA;&lt;p&gt;I don't know exactly that either of these issues is leading to your error bars having the wrong central tendencies, but investigating a couple of these lines may lead you to discover something.&lt;/p&gt;&#xA;&lt;p&gt;I'm not a &lt;code&gt;ggplot&lt;/code&gt; person so I can't diagnose just by looking at your code, and obviously I don't have your data so I can't tinker with it. Someone with more experience there might be a good idea. However, there are some &lt;a href=&quot;https://stackoverflow.com/questions/30872977/how-to-stack-error-bars-in-a-stacked-bar-plot-using-geom-errorbar&quot;&gt;related questions on SO&lt;/a&gt; that &lt;a href=&quot;https://stackoverflow.com/questions/42329858/stacked-percentage-barplot-with-error-bars-in-ggplot2?rq=3&quot;&gt;may be worth reading over&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Note again that none other than Hadley Wickham &lt;a href=&quot;https://github.com/tidyverse/ggplot2/issues/1079&quot; rel=&quot;nofollow noreferrer&quot;&gt;finds this kind of plot confusing&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-11-29T19:57:31.253" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21910" PostTypeId="1" CreationDate="2023-11-29T22:42:58.337" Score="1" ViewCount="118" Body="&lt;p&gt;I'm trying to run RNA-seq on single-end mouse RNA data (50bp reads) but have very limited RAM (&amp;lt;4 gb) - I don't think I really have a way to work around this memory restriction, and wanted to try Kallisto's pseudoalignment (I conda installed it) as indexing and/or aligning with other tools like STAR and HISAT2 have resulted in the processes being killed. Building the index with &lt;code&gt;kallisto index&lt;/code&gt; still results in the process being killed, so I downloaded the index files from &lt;a href=&quot;https://github.com/pachterlab/kallisto-transcriptome-indices/releases/tag/v1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/pachterlab/kallisto-transcriptome-indices/releases/tag/v1&lt;/a&gt; instead. I ran &lt;code&gt;kallisto quant -i index.idx -o output --single -l 180 -s 20 sample1.fastq&lt;/code&gt; (I don't entirely know what the mean and sd arguments should be for my data), but this resulted in the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[quant] fragment length distribution is truncated gaussian with mean = 180, sd = 20&#xA;Error: incompatible indices. Found version 13, expected version 10&#xA;Rerun with index to regenerate&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I've seen that other people have come across this error before when quantifying with kallisto, but it seems that they are building the index themselves while I was trying to use the pre-generated index from the unpacked &lt;code&gt;mouse_index_standard.tar.xz&lt;/code&gt; file downloaded from the link above.&lt;/p&gt;&#xA;&lt;p&gt;How can I get around this error? And in general, are there any suggestions on how to get raw/normalized count data from RNA-seq data with such memory limitations? I'm rather new to this.&lt;/p&gt;&#xA;" OwnerUserId="18633" LastActivityDate="2023-11-30T08:19:28.203" Title="Kallisto RNA-seq errors because of incompatible indices when running &lt;kallisto quant&gt; using downloaded mouse index" Tags="|rna-seq|kallisto|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21911" PostTypeId="2" ParentId="21910" CreationDate="2023-11-30T08:19:28.203" Score="3" Body="&lt;p&gt;It means that the version of the tool used for indexing and your current version are different and not compatible. Either downgrade your version to the version used for indexing, or build a new index with your current version. As by the GitHub repo, these indices that you link &lt;code&gt;(supports kallisto versions &amp;gt;= 0.50.1)&lt;/code&gt;, so use this version if you want to use the index.&lt;/p&gt;&#xA;&lt;p&gt;If that does not work then I would open an issue at their GitHub.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2023-11-30T08:19:28.203" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21913" PostTypeId="1" AcceptedAnswerId="21919" CreationDate="2023-12-01T16:15:33.350" Score="1" ViewCount="33" Body="&lt;p&gt;I've been trying to run mmseqs2 on a few metagenomic assemblies and despite my best efforts in reading the wiki and playing with parameters, the process is taking over a day.&lt;/p&gt;&#xA;&lt;p&gt;In their paper they claim to be 2 - 18x faster than CAT. I ran CAT on the same datasets and they took at most a few hours. So I can only assume that the long run time for mmseqs is down to user error (i.e. me).&lt;/p&gt;&#xA;&lt;p&gt;I am currently running the commands (general format):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mmseqs createdb metagenome1_scaffolds.fasta metagenome1_query_DB&#xA;mmseqs taxonomy metagenome1_query_DB /home/mmseqs2/NR/NR metagenome1_mmseqs_taxonomy.txt metagenome1_temp --tax-lineage 1 --orf-filter 1 --threads 24&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Could someone please let me know where I'm going wrong or what parameters I can tweak to get this running in the expected time frame?&lt;/p&gt;&#xA;" OwnerUserId="18327" LastActivityDate="2023-12-04T13:52:55.380" Title="MMSeqs taxonomy running for over a day" Tags="|assembly|metagenome|taxonomy|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21914" PostTypeId="1" CreationDate="2023-12-01T21:02:28.397" Score="0" ViewCount="26" Body="&lt;p&gt;I am running searches for genes using &lt;strong&gt;Phaster&lt;/strong&gt; {&lt;a href=&quot;https://phaster.ca/&quot; rel=&quot;nofollow noreferrer&quot;&gt;PHASTER (PHAge Search Tool Enhanced Release) is a significant upgrade to the popular PHAST web server for the rapid identification and annotation of prophage sequences within bacterial genomes and plasmids. &lt;/a&gt;  } and would like to look up the phages that have them on &lt;strong&gt;PhageDB&lt;/strong&gt; &lt;a href=&quot;https://phagesdb.org/about/&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Actinobacteriophage Database &lt;/a&gt;to learn more about their distribution. The blast search returns a lot of information about the phages and I am unsure what part of the sequence it gives for phages would be helpful.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/GUVfz.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/GUVfz.png&quot; alt=&quot;Image&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18640" LastEditorUserId="10049" LastEditDate="2023-12-03T12:46:57.627" LastActivityDate="2023-12-03T12:46:57.627" Title="How to identify phages from Phaster in PhageDB" Tags="|genome-sequencing|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21915" PostTypeId="1" AcceptedAnswerId="21916" CreationDate="2023-12-02T18:44:19.307" Score="3" ViewCount="222" Body="&lt;p&gt;Given a .bed (BED12), how can I convert it to GTF/GFF formats with gene_id attributes? What is the fastest way or available tools to do it?&lt;/p&gt;&#xA;&lt;p&gt;For example, given an input like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr27 17266469 17281218 ENST00000541931.8 1000 + 17266469 17281218 0,0,200 2 103,74, 0,14675,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I convert it to:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr27 bed2gtf gene 17266470 17285418 . + . gene_id &amp;quot;ENSG00000151743&amp;quot;;&#xA;chr27 bed2gtf transcript 17266470 17281218 . + . gene_id &amp;quot;ENSG00000151743&amp;quot;; transcript_id &amp;quot;ENST00000541931.8&amp;quot;;&#xA;chr27 bed2gtf exon 17266470 17266572 . + . gene_id &amp;quot;ENSG00000151743&amp;quot;; transcript_id &amp;quot;ENST00000541931.8&amp;quot;; exon_number &amp;quot;1&amp;quot;; exon_id &amp;quot;ENST00000541931.8.1&amp;quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18305" LastEditorUserId="298" LastEditDate="2023-12-03T12:44:06.493" LastActivityDate="2023-12-05T09:57:38.570" Title="How can I convert a BED file to GTF/GFF with gene_ids?" Tags="|bed|format-conversion|gtf|gff|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21916" PostTypeId="2" ParentId="21915" CreationDate="2023-12-02T18:44:19.307" Score="6" Body="&lt;p&gt;Here I provide an ordered list of options (note that I am the author of &lt;code&gt;bed2gtf&lt;/code&gt; and &lt;code&gt;bed2gff&lt;/code&gt;):&lt;/p&gt;&#xA;&lt;h2&gt;bed2gtf&lt;/h2&gt;&#xA;&lt;p&gt;A high-performance BED-to-GTF converter written in Rust from &lt;a href=&quot;https://github.com/alejandrogzi/bed2gtf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/alejandrogzi/bed2gtf&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Usage: bed2gtf[EXE] --bed/-b &amp;lt;BED&amp;gt; --isoforms/-i &amp;lt;ISOFORMS&amp;gt; --output/-o &amp;lt;OUTPUT&amp;gt; --threads/-t &amp;lt;THREADS&amp;gt;&#xA;&#xA;where:&#xA;--bed &amp;lt;BED&amp;gt;: a .bed file&#xA;--isoforms &amp;lt;ISOFORMS&amp;gt;: a tab-delimited file&#xA;--output &amp;lt;OUTPUT&amp;gt;: path to output file (*.gtf)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The isoforms file specification:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;a tab-delimited .txt/.tsv/.csv/... file with genes/isoforms (all the transcripts in .bed file should appear in the isoforms file):&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; cat isoforms.txt&#xA;&#xA;ENSG00000198888 ENST00000361390&#xA;ENSG00000198763 ENST00000361453&#xA;ENSG00000198804 ENST00000361624&#xA;ENSG00000188868 ENST00000595977&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Converts&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Homo sapiens GRCh38 GENCODE 44 (252,835 transcripts) in 3.25 seconds.&lt;/li&gt;&#xA;&lt;li&gt;Mus musculus GRCm39 GENCODE 44 (149,547 transcritps) in 1.99 seconds.&lt;/li&gt;&#xA;&lt;li&gt;Canis lupus familiaris ROS_Cfam_1.0 Ensembl 110 (55,335 transcripts) in 1.20 seconds.&lt;/li&gt;&#xA;&lt;li&gt;Gallus galus bGalGal1 Ensembl 110 (72,689 transcripts) in 1.36 seconds.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;bed2gff&lt;/h2&gt;&#xA;&lt;p&gt;A Rust BED-to-GFF3 translator that runs in parallel from &lt;a href=&quot;https://github.com/alejandrogzi/bed2gff&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/alejandrogzi/bed2gff&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Usage: bed2gff[EXE] --bed/-b &amp;lt;BED&amp;gt; --isoforms/-i &amp;lt;ISOFORMS&amp;gt; --output/-o &amp;lt;OUTPUT&amp;gt; --threads/-t &amp;lt;THREADS&amp;gt;&#xA;&#xA;where:&#xA;--bed &amp;lt;BED&amp;gt;: a .bed file&#xA;--isoforms &amp;lt;ISOFORMS&amp;gt;: a tab-delimited file&#xA;--output &amp;lt;OUTPUT&amp;gt;: path to output file (*.gff)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The isoforms file specification:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;a tab-delimited .txt/.tsv/.csv/... file with genes/isoforms (all the transcripts in .bed file should appear in the isoforms file):&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt; cat isoforms.txt&#xA;&#xA;ENSG00000198888 ENST00000361390&#xA;ENSG00000198763 ENST00000361453&#xA;ENSG00000198804 ENST00000361624&#xA;ENSG00000188868 ENST00000595977&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Convert&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Homo sapiens GRCh38 GENCODE 44 (252,835 transcripts) in 4.16 seconds.&lt;/li&gt;&#xA;&lt;li&gt;Mus musculus GRCm39 GENCODE 44 (149,547 transcritps) in 2.15 seconds.&lt;/li&gt;&#xA;&lt;li&gt;Canis lupus familiaris ROS_Cfam_1.0 Ensembl 110 (55,335 transcripts) in 1.30 seconds.&lt;/li&gt;&#xA;&lt;li&gt;Gallus gallus bGalGal1 Ensembl 110 (72,689 transcripts) in 1.51 seconds.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;bedToGenePred + genePredToGtf + refTable&lt;/h2&gt;&#xA;&lt;p&gt;UCSC offers a fast way to convert BED into GTF files through KentUtils or specific binaries using:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bedToGenePred in.bed /dev/stdout | genePredToGtf file /dev/stdin out.gtf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can install these tools with bioconda, or download them &lt;a href=&quot;http://hgdownload.cse.ucsc.edu/admin/exe/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;. The gene_id is only achieved when using refTables (a format specified in UCSC's web browser), you can see a more elaborate answer here &lt;a href=&quot;https://www.biostars.org/p/54863/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Obtaining Ucsc Tables Via Ftp And Converting Them To Proper Gff3 Via Genepredtogtf?&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;h2&gt;Other options&lt;/h2&gt;&#xA;&lt;p&gt;Other scripts/tools &lt;strong&gt;That DO NOT produce a complete GTF file (lacking gene_id attributes)&lt;/strong&gt; are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;gtf2bed&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;gtf2bed &amp;lt; foo.gtf | sort-bed - &amp;gt; foo.bed &#xA;awk '{print $1&amp;quot;\t&amp;quot;$7&amp;quot;\t&amp;quot;$8&amp;quot;\t&amp;quot;($2+1)&amp;quot;\t&amp;quot;$3&amp;quot;\t&amp;quot;$5&amp;quot;\t&amp;quot;$6&amp;quot;\t&amp;quot;$9&amp;quot;\t&amp;quot;(substr($0, index($0,$10)))}' foo.bed &amp;gt; foo_from_gtf2bed.gtf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;-kscript&#xA;from &lt;a href=&quot;https://github.com/holgerbrandl/kscript&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/holgerbrandl/kscript&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;kscript https://git.io/vbJ4B my.bed &amp;gt; my.gtf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;pfurio/bed2gtf&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;from &lt;a href=&quot;https://github.com/pfurio/bed2gtf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/pfurio/bed2gtf&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;python bed2gtf [options] &amp;lt;mandatory&amp;gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;AGAT&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://agat.readthedocs.io/en/latest/tools/agat_convert_bed2gff.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;AGAT&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Considering only the options that produce gene_ids attributes, bed2gtf and bed2gff are faster by ~3-4 seconds than UCSC's C binaries. More detailed instructions of this tools are explained in the sources linked.&lt;/p&gt;&#xA;" OwnerUserId="18305" LastEditorUserId="298" LastEditDate="2023-12-05T09:57:38.570" LastActivityDate="2023-12-05T09:57:38.570" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21918" PostTypeId="1" AcceptedAnswerId="21935" CreationDate="2023-12-03T20:54:48.190" Score="2" ViewCount="66" Body="&lt;p&gt;I have a dataset that contains thousands of mixed multiple and single fast5 files in a non-homogenous folder structure.&#xA;The reads and fast5 files are coming from different labs and some labs gave single fast5 files and some multi fast5 files, without clear mappings. My final goal is to convert all the fast5 files to multi fast5 files.&lt;/p&gt;&#xA;&lt;p&gt;There are commands in ont_fast5_api which is an interface to HDF5 files of the Oxford Nanopore.&#xA;Using this API we can convert single fast5 files and multi fast5 files to each other.&#xA;this is the link to this API github page:&#xA;&lt;a href=&quot;https://github.com/nanoporetech/ont_fast5_api&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/nanoporetech/ont_fast5_api&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So I made a test mixed folder to check if I can use the API without missing reads as a result of conversion. The test folder structure is similar to my original folder structure:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;mixed&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Subfolder_A&#xA;&lt;ul&gt;&#xA;&lt;li&gt;single_fast5 x 4000&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;Subfolder_B&#xA;&lt;ul&gt;&#xA;&lt;li&gt;multiple_fast5 (contains 4000 single reads fast5)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;single_fast5 x 2&lt;/li&gt;&#xA;&lt;li&gt;multiple_fast5_1 (contains 4000 single reads fast5)&lt;/li&gt;&#xA;&lt;li&gt;multiple_fast5_2 (contains 4000 single reads fast5)&lt;/li&gt;&#xA;&lt;li&gt;multiple_fast5_3 (contains 100 single reads fast5)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Here is a screenshot of the structure with a few files from each folder:&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://cdn.discordapp.com/attachments/991841204088098977/1183239459714646037/mixed_tree.png?ex=65879c9e&amp;amp;is=6575279e&amp;amp;hm=0747962eff4b5167c6026c1d7a9612d66983bcab5fb3ce5a3315166b2c19dc5d&amp;amp;&quot; alt=&quot;alt text&quot; title=&quot;folder structure&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;and I created two other folders outside of the mixed folder as follows:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;single&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;multi&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Here my question is about &lt;em&gt;single_to_multi_fast5&lt;/em&gt; and &lt;em&gt;multi_to_single_fast5&lt;/em&gt; commands.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;test_path=mixed&#xA;multi_path=multi&#xA;single_path=single&#xA;multi_to_single_fast5 -i $test_path/ -s $single_path/ --recursive&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;in the $single_path, I expect to see, all of the reads in multi files converted to single files (12100 reads in 12100 single fast5 files), but I see 8100 single files.&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://cdn.discordapp.com/attachments/991841204088098977/1183088499252023406/single.png?ex=65871006&amp;amp;is=65749b06&amp;amp;hm=484c575feb7ddb4effbb6f05bbb91fe64290c7da4d1c25b4339aaffcccd77d50&amp;amp;&quot; alt=&quot;alt text&quot; title=&quot;single path&quot; /&gt;&#xA;So I have 4000 missing reads.&lt;/p&gt;&#xA;&lt;p&gt;Now I convert single read fast5 file to multi:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;single_to_multi_fast5 -i $test_path/ -s $multi_path/ --filename_base $output_name --batch_size 1000 --recursive&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;in the $multi_path folder, I expect to see that generated multi files contain all my single-read files (4000 + 2), but I see 3006 in the mapping summary.&lt;/p&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://cdn.discordapp.com/attachments/991841204088098977/1183089993523154984/multi.png?ex=6587116b&amp;amp;is=65749c6b&amp;amp;hm=6d78f50b7a5ede6581c7b0205d1db5e74d88397963907ebed457ce6f8f487592&amp;amp;&quot; alt=&quot;alt text&quot; title=&quot;mutli path&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;So again I missed 996 single reads in my multi files.&#xA;Since I made a small subsample of data, I am sure that the files are not overwritten.&lt;/p&gt;&#xA;&lt;p&gt;Because my final goal is &lt;strong&gt;converting everything to multi&lt;/strong&gt; I was going to do the following &lt;strong&gt;pipeline&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;orig_path=original_folder&#xA;intermediate_path=intermediate&#xA;final_path=final_multi&#xA;# converts/collects all multi files in the original folder&#xA;multi_to_single_fast5 -i &lt;span class=&quot;math-container&quot;&gt;$orig_path/ -s $&lt;/span&gt;intermediate_path/ --recursive&#xA;single_to_multi_fast5 -i &lt;span class=&quot;math-container&quot;&gt;$intermediate_path/ -s $&lt;/span&gt;final_path/ --filename_base $output_name --batch_size 1000 --recursive&#xA;&#xA;# converts/collects all single files in the original folder&#xA;single_to_multi_fast5 -i &lt;span class=&quot;math-container&quot;&gt;$orig_path/ -s $&lt;/span&gt;final_path/ --filename_base $output_name --batch_size 1000 --recursive&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However because it seems I get missing reads during the process, I cannot use this pipeline.&lt;/p&gt;&#xA;&lt;p&gt;At the same time, my original dataset is super huge, I cannot check if individual files are single or multi. It would take ages (I guess).&#xA;Is there a solution for solving this problem besides checking every file? Maybe my pipeline is not quite suitable for what I want to do.&lt;/p&gt;&#xA;" OwnerUserId="15341" LastEditorUserId="15341" LastEditDate="2023-12-10T02:54:57.607" LastActivityDate="2023-12-14T02:44:44.233" Title="single_to_multi_fast5 does not collect all the single files if the input folder contains mixed types of fast5 files" Tags="|nanopore|fast5|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21919" PostTypeId="2" ParentId="21913" CreationDate="2023-12-04T13:52:55.380" Score="1" Body="&lt;p&gt;After much googling about I realised that I was missing a step! Here are the commands I used to get it running:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mmseqs createdb metagenome1_scaffolds.fasta metagenome1_query_DB&#xA;mmseqs taxonomy metagenome1_query_DB /home/mmseqs2/NR/NR metagenome1_mmseqs_taxonomy.txt metagenome1_temp --tax-lineage 1 --orf-filter 1 --threads 24 --split-memory-limit 500G &#xA;mmseqs createtsv metagenome1_DB metagenome1_mmseqs_taxonomy.txt metagenome1_mmseqs_taxonomy_final.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The split-memory-limit flag sped up the process and the final line put all the split files back together&lt;/p&gt;&#xA;" OwnerUserId="18327" LastActivityDate="2023-12-04T13:52:55.380" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21920" PostTypeId="2" ParentId="5652" CreationDate="2023-12-04T22:44:19.577" Score="0" Body="&lt;p&gt;This question was posted and answered as a &lt;a href=&quot;https://github.com/NBISweden/MrBayes/issues/94&quot; rel=&quot;nofollow noreferrer&quot;&gt;Github Issue&lt;/a&gt; in the MrBayes repository; I am copying the answer here for better visibility.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Dear zillurbmb51,&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your question. You need to load the data matrix in&#xA;memory before issuing the &lt;code&gt;sumt&lt;/code&gt; command, or in other words: &amp;quot;A matrix&#xA;must be specified before sumt can be used&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Example 1&lt;/strong&gt; (where you haven't yet run a MCMC):&lt;/p&gt;&#xA;&lt;p&gt;Start mb, then type:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;exe infile.nex&#xA;mcmc&#xA;sumt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Example 2&lt;/strong&gt; (where you already did a MCMC, and have &lt;code&gt;.t&lt;/code&gt; files in your&#xA;current working directory, and they are named &lt;code&gt;infile.nex.run1.t&lt;/code&gt;,&#xA;&lt;code&gt;infile.nex.run2.t&lt;/code&gt;, etc):&lt;/p&gt;&#xA;&lt;p&gt;Start mb, then type:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;exe infile.nex &#xA;sumt filename=infile.nex &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Cheers&lt;/p&gt;&#xA;&lt;p&gt;Johan&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="17320" LastActivityDate="2023-12-04T22:44:19.577" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21921" PostTypeId="1" AcceptedAnswerId="21923" CreationDate="2023-12-05T13:26:10.400" Score="4" ViewCount="174" Body="&lt;p&gt;Here i will show you a minimal working example of code and as you can see the support values for the tree is always 100.&lt;/p&gt;&#xA;&lt;p&gt;I am using synthetic sequences of 100bp for 6 elements. The sequences have been generated at random choosing from ATCG for each position with equal probabilities.&lt;/p&gt;&#xA;&lt;p&gt;The input fasta file is&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;S1&#xA;CTCAGGCACTAGGAGCTTCTCCAGGGCAAAGTTGTCTACGAATATGCCGACTCAGAAGGTTATCAATACGGTTTACTTATCTGCACGCCATTTTCCTATG&#xA;&amp;gt;S2&#xA;TAACAAATGTTTCTCGCTCGAACCCCGTGGTGCGAGGGGTCACGATAAAAGGTCCCTCCTTCCACGGCATAATTGCTCCCTTTCTTTTCCGTGGGCAGGA&#xA;&amp;gt;S3&#xA;TTCCTAAAGGACGACTGGAAGCCGGGTACACGTCAACGGAGCTTCTAGCCCGAGGTCTATAGACCGGTTAATGAACTAGACTTAATGTGGAGCTCGTAGA&#xA;&amp;gt;S4&#xA;ACTTCGTGCCCAACCACTCTATGAGCAGAGTGCTCGAATGAACCTCAAAAGGATATCCGCATTTACTCTATATAACAAACGGCCGTCCCCCCATCTGTCA&#xA;&amp;gt;S5&#xA;TGACACTGGGTCATTTTACCCGCACTGATCGCTGGGCAGGTCGGCAATTTCGTCAGAATGCCGTGGCCGCACTGAAAATATCATTACCGGACTGAGTATG&#xA;&amp;gt;S6&#xA;ATAAGCCGAGGGGTAGCCCTCTATTTCGCACCGTATAGACGAAGTGATAAACTTTCTAACCACGTGTCGCCTATCTCTACCTAGCCACATTTGAGGTGCG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The following python code makes the multiple alignment using MUSCLE algorithm&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;### alignment&#xA;from Bio.Align.Applications import MuscleCommandline&#xA;from Bio import AlignIO, SeqIO&#xA;import os&#xA;&#xA;muscle_exe = &amp;quot;C:/Scripts/muscle3.exe&amp;quot;&#xA;path = os.getcwd()&#xA;with open('tmp_in.fa', 'w') as fp:&#xA;    fp.write(fasta)&#xA;&#xA;## replacing \\ with /&#xA;inp_fp = path.replace('\\', '/') + '/tmp_in.fa'&#xA;out_fp = path.replace('\\', '/') + '/tmp_out.afa'&#xA;&#xA;muscle_cline = MuscleCommandline(muscle_exe, input=inp_fp, out=out_fp)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The following code block takes the previous alignment &lt;code&gt;out_fp&lt;/code&gt; and creates a tree using UPGMA algorithm.&lt;br /&gt;&#xA;The number of bootstrap trees is set to 50 and the consensus is majority based.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio.Phylo.TreeConstruction import DistanceCalculator, DistanceTreeConstructor&#xA;from Bio.Phylo.Consensus import *&#xA;&#xA;# read the multiple sequence alignment (msa)&#xA;with open(out_fp,'r') as afa:&#xA;    alignment = AlignIO.read(afa, 'fasta')&#xA;&#xA;calculator = DistanceCalculator('identity')&#xA;constructor = DistanceTreeConstructor(method='upgma', distance_calculator=calculator)&#xA;&#xA;## get the list of bootstrap trees&#xA;consensus_tree = bootstrap_consensus(alignment=alignment, times=50, tree_constructor=constructor, consensus=majority_consensus)&#xA;&#xA;print(consensus_tree)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The final consensus tree looks like this&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Tree(rooted=True)&#xA;    Clade()&#xA;        Clade(branch_length=0.350925925925926, name='S5')&#xA;        Clade(branch_length=0.05833333333333341, confidence=100.0)&#xA;            Clade(branch_length=0.05879629629629617, confidence=100.0)&#xA;                Clade(branch_length=0.26481481481481484, name='S3')&#xA;                Clade(branch_length=0.020370370370370372, confidence=100.0)&#xA;                    Clade(branch_length=0.24444444444444446, name='S1')&#xA;                    Clade(branch_length=0.24444444444444446, name='S4')&#xA;            Clade(branch_length=0.031018518518518404, confidence=100.0)&#xA;                Clade(branch_length=0.2925925925925926, name='S6')&#xA;                Clade(branch_length=0.2925925925925926, name='S2')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My question is: why does the tree have all support values equal to 100? I am sure there must be an error either on my side or in the package module itself.&lt;/p&gt;&#xA;" OwnerUserId="18351" LastActivityDate="2023-12-05T17:54:21.330" Title="BioPython bootstrap is not reliable?" Tags="|sequence-alignment|multiple-sequence-alignment|biopython|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21923" PostTypeId="2" ParentId="21921" CreationDate="2023-12-05T17:23:08.750" Score="2" Body="&lt;p&gt;I think this is a bug.&lt;/p&gt;&#xA;&lt;p&gt;It seems to work if you do this, creating an equivalent Alignment object instead of a MultipleSeqAlignment to give the bootstrap step:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio.Align import Alignment&#xA;alignment2 = Alignment(list(alignment))&#xA;consensus_tree = bootstrap_consensus(alignment=alignment2, times=50, tree_constructor=constructor, consensus=majority_consensus)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;code&gt;bootstrap_consensus&lt;/code&gt; calls &lt;code&gt;bootstrap_trees&lt;/code&gt;, which makes however many randomly shuffled alignments you asked for.  But if the input is a MultipleSeqAlignment object like you're using it builds a tree over and over with &lt;em&gt;that&lt;/em&gt; &lt;a href=&quot;https://github.com/biopython/biopython/blob/10bf7ca8f7d4a9f2ec984c74c424c98fa1e31e00/Bio/Phylo/Consensus.py#L588&quot; rel=&quot;nofollow noreferrer&quot;&gt;same input to the function as the tree function input&lt;/a&gt;, so it always gets the same tree over and over.  The other branch of &lt;code&gt;bootstrap_trees&lt;/code&gt;, which runs if you have an Aignment object, builds trees based on the randomly-selected columns of the input, like you'd expect.  Pretty sure that's just a variable naming screw-up one of us should report to Biopython.&lt;/p&gt;&#xA;&lt;p&gt;Self-contained version of your tree part:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env python&#xA;from Bio.Phylo.TreeConstruction import DistanceCalculator, DistanceTreeConstructor&#xA;from Bio import AlignIO&#xA;from Bio.Phylo.Consensus import *&#xA;&#xA;out_fp = &amp;quot;tmp_out.fa&amp;quot;&#xA;&#xA;# read the multiple sequence alignment (msa)&#xA;with open(out_fp,'r') as afa:&#xA;    alignment = AlignIO.read(afa, 'fasta')&#xA;&#xA;calculator = DistanceCalculator('identity')&#xA;constructor = DistanceTreeConstructor(method='upgma', distance_calculator=calculator)&#xA;&#xA;## get the list of bootstrap trees&#xA;consensus_tree = bootstrap_consensus(alignment=alignment, times=50, tree_constructor=constructor, consensus=majority_consensus)&#xA;&#xA;print(consensus_tree)&#xA;&#xA;# Give bootstrap_consensus an Alignment object instead of MultipleSeqAlignment&#xA;# object&#xA;from Bio.Align import Alignment&#xA;alignment2 = Alignment(list(alignment))&#xA;consensus_tree = bootstrap_consensus(alignment=alignment2, times=50, tree_constructor=constructor, consensus=majority_consensus)&#xA;print(consensus_tree)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Example outputs for the two different input alignment objects:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Tree(rooted=True)&#xA;    Clade()&#xA;        Clade(branch_length=0.012239583333333349, confidence=100.0)&#xA;            Clade(branch_length=0.31666666666666665, name='S3')&#xA;            Clade(branch_length=0.033333333333333284, confidence=100.0)&#xA;                Clade(branch_length=0.2833333333333333, name='S2')&#xA;                Clade(branch_length=0.2833333333333333, name='S4')&#xA;        Clade(branch_length=0.033072916666666716, confidence=100.0)&#xA;            Clade(branch_length=0.29583333333333334, name='S5')&#xA;            Clade(branch_length=0.03750000000000007, confidence=100.0)&#xA;                Clade(branch_length=0.2583333333333333, name='S1')&#xA;                Clade(branch_length=0.2583333333333333, name='S6')&#xA;Tree(rooted=True)&#xA;    Clade()&#xA;        Clade(branch_length=0.2583333333333333, name='S2')&#xA;        Clade(branch_length=0.03405145202020203, confidence=66.0)&#xA;            Clade(branch_length=0.275, name='S5')&#xA;            Clade(branch_length=0.049892241379310344, confidence=58.0)&#xA;                Clade(branch_length=0.21250000000000002, name='S1')&#xA;                Clade(branch_length=0.21250000000000002, name='S6')&#xA;        Clade(branch_length=0.039416666666666676, confidence=50.0)&#xA;            Clade(branch_length=0.2583333333333333, name='S4')&#xA;            Clade(branch_length=0.321875, name='S3')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4980" LastActivityDate="2023-12-05T17:23:08.750" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21924" PostTypeId="2" ParentId="21921" CreationDate="2023-12-05T17:54:21.330" Score="0" Body="&lt;p&gt;Initially, 100 looks like a good value.  However, if you lower to bootstrap value, let's say maybe to 50 as you did, you will probably get an erroneous support value.  Bootstrapping (you probably already know this) is how many times a branch in a phylogenetic tree is observed.  A good bootstrapping value would be anywhere between 80 and 100.  50 is not a good bootstrapping value.  This means the branches are only being observed 50 times as opposed to 85, 90, etc...   If I were you, try changing the bootstrapping value of 50 to at least 80.  See what that gets you.  You may be satisfied with it, but you may want to increase it depending on your output.  By doing this, you will see a more realistic and phylogenetically accurate output of your tree.  Most bioinformaticians use 100 as a bootstrap value.  The only issue, I wouldn't really call it an issue, but it will take longer for an output to occur due to computing time and computing memory available.  BUT you will still get an output, you just may need to wait a bit.   My bioinformatics background is rooted in phylogenetic and horizontal gene transfer research involving phages and bacteria, so I have seen this issue a lot and have experienced myself a lot!  I hope this information provides some insight to your project and bioinformatics journey! :)&lt;/p&gt;&#xA;" OwnerUserId="18487" LastActivityDate="2023-12-05T17:54:21.330" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21925" PostTypeId="1" CreationDate="2023-12-05T23:24:32.530" Score="2" ViewCount="105" Body="&lt;p&gt;I am using MEGA11 &lt;a href=&quot;https://www.megasoftware.net/home&quot; rel=&quot;nofollow noreferrer&quot;&gt;MEGA11: Molecular Evolutionary Genetics Analysis Version 11&lt;/a&gt;    ,   &lt;a href=&quot;https://www.megasoftware.net/pdfs/TamuraStecherKumar21.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Mol. Biol. Evol. 38(7):3022–3027 &lt;/a&gt;  to align protein coding sequences that I downloaded from KEGG database. I am doing MUSCLE alignment and I choose the &amp;quot;align codons&amp;quot; option. The problem that occurs has to do with the error &amp;quot;stop codons are found in the translated sequences&amp;quot; that pops as an warning during the process. When I check the sequences I cannot find any stop codons except the one at the end, as expect. I don't know how to interpret the message. Does anyone has any idea what is going wrong? Thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="17573" LastEditorUserId="10049" LastEditDate="2023-12-07T21:14:15.733" LastActivityDate="2023-12-08T03:47:59.143" Title="MEGA11 gives the error &quot;stop codons are found in the translated sequences&quot;, even though there are not!" Tags="|errors|kegg|codon|megax|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21926" PostTypeId="1" CreationDate="2023-12-06T00:48:46.833" Score="0" ViewCount="102" Body="&lt;p&gt;I am doing a project on duplicated genes and I have a major difficulty on how to align sequences that use different genetic codes. I work with fasta files that contain sequences of protein coding genes, every fasta file includes genes from many species that are orthologs/paralogs. These genes even in the same species may use different genetic codes. My goal is to align the sequences, codon alignment [the process of codon alignment considers the reading frame of the translated protein and adjusts the nucleotide alignment so that the protein alignment stays in frame.] , and afterwards to create N-J trees. For the whole process I work on MEGA11. I cannot organize the species-genes into groups with common genetic codes because I need all of them in the tree. One option, to my mind at least, would be the alignment not in codon but in DNA,or another option is to align their protein sequences or lastly a third option could be the change of the sequences in order to get rid of the stop codons and afterwards to use one common code for everything, by doing that I know that I will lose some info but at least I can work.Do you have any idea how to overcome this obstacle? Do you agree with any of this options? Thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="17573" LastEditorUserId="10049" LastEditDate="2023-12-07T21:14:08.127" LastActivityDate="2024-02-24T11:01:05.170" Title="Aligning sequences with multiple genetic codes!" Tags="|phylogenetics|sequence-alignment|multiple-sequence-alignment|codon|megax|" AnswerCount="1" CommentCount="12" ContentLicense="CC BY-SA 4.0" />
  <row Id="21927" PostTypeId="2" ParentId="21905" CreationDate="2023-12-06T18:30:09.627" Score="1" Body="&lt;p&gt;(&lt;em&gt;Should be a comment not an answer, but I weary of SE comment formatting- feel free to express disapproval in whichever way&lt;/em&gt;)&lt;/p&gt;&#xA;&lt;p&gt;When in doubt, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3029072/&quot; rel=&quot;nofollow noreferrer&quot;&gt;google it&lt;/a&gt; and look at &lt;a href=&quot;https://en.wikipedia.org/wiki/Trie&quot; rel=&quot;nofollow noreferrer&quot;&gt;wikipedia&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Tries are used in Bioinformatics, notably in sequence alignment software applications such as BLAST, which indexes all the different substring of length k (called k-mers) of a text by storing the positions of their occurrences in a compressed trie sequence databases.[25]&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-12-06T18:30:09.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21928" PostTypeId="1" CreationDate="2023-12-07T10:16:56.150" Score="1" ViewCount="75" Body="&lt;p&gt;In a FastQ to BAM pipeline where only adapter trimming is performed, I've noticed a potential discrepancy in read counts between the initial FastQ files and their resulting BAM file. Specifically, I'm seeking clarification on whether the following statement holds true:&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;Total number of reads in R1 and R2 FastQ files = mapped and unmapped reads in the corresponding BAM file.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;I understand that adapter trimming is the sole preprocessing step in this pipeline. However, I'm uncertain about the impact of factors such as PCR duplicates, mapping efficiency, and any potential loss during processing. The BAM file does contain PCR duplicates, and I'm wondering if these factors contribute to the observed differences in read counts.&lt;/p&gt;&#xA;&lt;p&gt;Can someone shed light on the expected relationship between the read counts in the FastQ files and the BAM file under these conditions? Any insights into how PCR duplicates may affect the discrepancy would be particularly helpful. Additionally, if there are common reasons for the observed differences in read counts between FastQ and BAM files, I'd appreciate guidance on identifying and addressing them.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Additional information:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The pipeline used:&lt;/p&gt;&#xA;&lt;p&gt;pre-trimming: &lt;code&gt;fastqc R1.fastq.gz R2.fastq.gz -o pre_trimming&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;adapter trimming: &lt;code&gt;fastp -i R1.fastq.gz -o trimmed_R1.fastq.gz -I R2.fastq.gz -O trimmed_R2.fastq.gz -h file.html -j file.json&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;alignment: &lt;code&gt;bwa mem -M -t 4 -R “@RG\tID:FE_S211\tPL:ILLUMINA\tSM:A4” hg19.fa trimmed_R1.fastq.gz trimmed_R2.fastq.gz | samtools sort -o output.bam -O BAM -@ 4 -&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="17846" LastEditorUserId="17846" LastEditDate="2023-12-07T15:43:54.563" LastActivityDate="2023-12-15T17:45:34.750" Title="Discrepancy in Read Counts Between FastQ and BAM Files in Adapter-Trimmed Pipeline" Tags="|sam|fastq|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21929" PostTypeId="2" ParentId="21925" CreationDate="2023-12-07T20:18:16.740" Score="1" Body="&lt;p&gt;The solution is very simply, MegaX will not allow any stop codons within the &amp;quot;translated&amp;quot; sequence. The reason is simple&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;MEGAX's function is phylogenetic models with nucleotide or protein data&lt;/li&gt;&#xA;&lt;li&gt;All phylogenetic protein calculations require a 20x20 matrix of amino acid frequencies, e.g. BLOSUM, Dayhoff, mtREV24 etc ... No matrix has ever calculated stop codon frequency (well NCBI have and sort of obviously always '1' or transformed '1' [the definition of a gene is it has one stop codeon])&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Thus MEGAX will not permit a single stop codon, even the terminal codon of the gene, because it can't calculate phylogenetic information from them (doesn't fit the model). Thus the &amp;quot;getter&amp;quot; to pick up bugs occurs right at that beginning.&lt;/p&gt;&#xA;&lt;p&gt;Two solutions:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Don't translate the nucleotide sequence on import - just stick with nucleotides (there are no codon models in MEGAX).&lt;/li&gt;&#xA;&lt;li&gt;Remove the terminal 3 resides&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;pre&gt;&lt;code&gt;     perl -p -i -e 's/(^[AGCT-NRY]{3}.*)[AGCT-NRY]{3}$/$1/g' alignment.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note the above code is not tested and intended only for fasta sequence or MEGA input format - don't use it on Clustal or Phylip it will be disastrous, i.e. interleaved format. IN ADDITION make sure there are no carriage returns within the contiguous sequence (again the result is disastrous). Finally, note I forget the &lt;code&gt;{3}&lt;/code&gt; stuff (its ages since I wrote perl) and should include all the degenerate nucleotide code beyond NRY, e.g. MB etc ...&lt;/p&gt;&#xA;&lt;p&gt;Really trimming the 3' last 3 nucleotides is a good use of Biopython - no risk of errors regardless of alignment. Trimal can probably do it BTW or seqkit.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;I'm still interested in how you interrogated KEGG gene DB BTW&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-12-08T03:47:59.143" LastActivityDate="2023-12-08T03:47:59.143" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21931" PostTypeId="1" AcceptedAnswerId="21941" CreationDate="2023-12-08T16:43:50.353" Score="0" ViewCount="65" Body="&lt;p&gt;I have downloaded and installed the SRA toolkit, but there seems to be no online material on how to download SRR files using the SRA toolkit on Windows.&lt;/p&gt;&#xA;&lt;p&gt;All online materials are Ubuntu or Linux based.&lt;/p&gt;&#xA;&lt;p&gt;Is there some obvious resource that I am missing?&lt;/p&gt;&#xA;" OwnerUserId="18663" LastActivityDate="2023-12-11T06:32:34.120" Title="How to download Sequencing data on Windows using SRA toolkit?" Tags="|sratoolkit|sra|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21932" PostTypeId="1" AcceptedAnswerId="21936" CreationDate="2023-12-08T18:12:10.867" Score="0" ViewCount="47" Body="&lt;p&gt;I am trying to get a protein sequence using this command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;curl https://www.uniprot.org/uniprot/A2Z669.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I don't get any output:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ curl https://www.uniprot.org/uniprot/A2Z669.fasta&#xA;$ &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And no file is created.&lt;/p&gt;&#xA;" OwnerUserId="18665" LastEditorUserId="298" LastEditDate="2023-12-10T14:25:01.030" LastActivityDate="2023-12-11T10:52:52.567" Title="Fetching uniprot seq using bash command" Tags="|uniprot|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21933" PostTypeId="1" CreationDate="2023-12-08T23:38:54.957" Score="2" ViewCount="83" Body="&lt;p&gt;I have four vcf files, &lt;code&gt;SNPs_s1.vcf&lt;/code&gt;, &lt;code&gt;SNPs_s2.vcf&lt;/code&gt;, &lt;code&gt;SNPs_s3.vcf&lt;/code&gt;, and &lt;code&gt;SNPs_s4.vcf&lt;/code&gt;, which contain information about SNPs. These vcf files were obtained by using the following methods: the initial input files were short-paired reads&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;I did mapping with minimap2&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;./minimap2 -ax sr ref.fa read1.fq.gz read2.fq.gz &amp;gt; aln.sam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;converted to bam file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -Sb aln.sam &amp;gt; aln.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Sorted the bam file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools sort aln.bam &amp;gt; aln_sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;created an index file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools index aln_sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;created an index file of reference&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools faidx ref.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;extracted the raw variants using freebayes&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;freebayes -f ref.fa aln_sorted.bam &amp;gt; out_var_s1.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;extracted SNPs using vcf tools&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;vcftools --vcf out_var_s1.vcf --remove-indels --recode \&#xA;   --recode-INFO-all --out SNPs_s1.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I would like to calculate the fixation index (Fst) between &lt;code&gt;SNPs_s1.vcf&lt;/code&gt; and &lt;code&gt;SNPs_s2.vcf&lt;/code&gt;, &lt;code&gt;SNPs_s1.vcf&lt;/code&gt; and &lt;code&gt;SNPs_s3.vcf&lt;/code&gt; etc. I attempted to calculate Fst using &lt;code&gt;bcftools&lt;/code&gt;, but I have to provide the id list. I tried the code below for that.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools query -l snps_s1.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The output was &amp;quot;unknown&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;How do I calculate Fst using the above vcf files? Your suggestions would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="18667" LastEditorUserId="298" LastEditDate="2023-12-10T14:08:07.097" LastActivityDate="2023-12-10T14:08:07.097" Title="Fst calculation from VCF files" Tags="|phylogenetics|sequence-alignment|vcf|fst|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21935" PostTypeId="2" ParentId="21918" CreationDate="2023-12-10T05:36:26.217" Score="1" Body="&lt;p&gt;It looks like you're creating too much additional work for yourself. What I'm trying to lead to with my comments is that the file names should help in working out if a file represents single reads or multiple reads, in which case the single reads can be shifted into a separate folder.&lt;/p&gt;&#xA;&lt;p&gt;Based on the file names that I would expect, a search for files containing '_read_[0-9]' should be sufficient to fish out the single fast5 files:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;find . -name '*_read_[0-9]*' &amp;gt; single_reads.txt&#xA;mkdir -p single_reads&#xA;cat single_reads.txt | while read fname; do mv -i ${fname} single_reads; done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;[That &lt;code&gt;single_reads.txt&lt;/code&gt; file could be split into multiple files if directory entries get too large.]&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;After which you should have a single folder that contains thousands of single-read fast5 files, and other folders that contain multi-fast5 files, removing the problem of needing to deal with mixed file types.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2023-12-14T02:44:44.233" LastActivityDate="2023-12-14T02:44:44.233" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21936" PostTypeId="2" ParentId="21932" CreationDate="2023-12-10T14:23:33.790" Score="1" Body="&lt;p&gt;The URL &lt;a href=&quot;https://www.uniprot.org/uniprot/A2Z669.fasta&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.uniprot.org/uniprot/A2Z669.fasta&lt;/a&gt; redirects to &lt;a href=&quot;https://rest.uniprot.org/uniprotkb/A2Z669.fasta&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rest.uniprot.org/uniprotkb/A2Z669.fasta&lt;/a&gt;. You can see this if you open it in your browser: clicking on the first link will take you to the address of the second.&lt;/p&gt;&#xA;&lt;p&gt;Now, &lt;code&gt;curl&lt;/code&gt; &lt;a href=&quot;https://unix.stackexchange.com/q/763548/22222&quot;&gt;doesn't follow redirects by default&lt;/a&gt;, you need the &lt;code&gt;-L&lt;/code&gt; option for that (see &lt;code&gt;man curl&lt;/code&gt;):&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;-L, --location&lt;/p&gt;&#xA;&lt;p&gt;(HTTP)  If  the  server  reports  that  the requested page has moved to a different location (indicated with a Location:&#xA;header and a 3XX response code), this option makes curl redo the request on the new place. If  used  together  with  -i,&#xA;--include or -I, --head, headers from all requested pages are shown.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So you need &lt;code&gt;-L&lt;/code&gt; for &lt;code&gt;curl&lt;/code&gt; to work here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ curl -L https://www.uniprot.org/uniprot/A2Z669.fasta&#xA;&amp;gt;sp|A2Z669|CSPLT_ORYSI CASP-like protein 5A2 OS=Oryza sativa subsp. indica OX=39946 GN=OsI_33147 PE=3 SV=1&#xA;MRASRPVVHPVEAPPPAALAVAAAAVAVEAGVGAGGGAAAHGGENAQPRGVRMKDPPGAP&#xA;GTPGGLGLRLVQAFFAAAALAVMASTDDFPSVSAFCYLVAAAILQCLWSLSLAVVDIYAL&#xA;LVKRSLRNPQAVCIFTIGDGITGTLTLGAACASAGITVLIGNDLNICANNHCASFETATA&#xA;MAFISWFALAPSCVLNFWSMASR&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or, alternatively, you can use &lt;code&gt;wget&lt;/code&gt; which does follow redirects by default:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ wget &amp;quot;https://www.uniprot.org/uniprot/A2Z669.fasta&amp;quot; &#xA;--2023-12-10 16:21:46--  https://www.uniprot.org/uniprot/A2Z669.fasta&#xA;Loaded CA certificate '/etc/ssl/certs/ca-certificates.crt'&#xA;Resolving www.uniprot.org (www.uniprot.org)... 193.62.193.81&#xA;Connecting to www.uniprot.org (www.uniprot.org)|193.62.193.81|:443... connected.&#xA;HTTP request sent, awaiting response... 301 Moved Permanently&#xA;Location: https://rest.uniprot.org/uniprot/A2Z669.fasta [following]&#xA;--2023-12-10 16:21:46--  https://rest.uniprot.org/uniprot/A2Z669.fasta&#xA;Resolving rest.uniprot.org (rest.uniprot.org)... 193.62.193.81&#xA;Connecting to rest.uniprot.org (rest.uniprot.org)|193.62.193.81|:443... connected.&#xA;HTTP request sent, awaiting response... 301 Moved Permanently&#xA;Location: https://rest.uniprot.org/uniprotkb/A2Z669.fasta [following]&#xA;--2023-12-10 16:21:46--  https://rest.uniprot.org/uniprotkb/A2Z669.fasta&#xA;Reusing existing connection to rest.uniprot.org:443.&#xA;HTTP request sent, awaiting response... 200 OK&#xA;Length: 314 [text/plain]&#xA;Saving to: ‘A2Z669.fasta.1’&#xA;&#xA;A2Z669.fasta             100%[=======================================&amp;gt;]     314  --.-KB/s    in 0s      &#xA;&#xA;2023-12-10 16:21:47 (8.57 MB/s) - ‘A2Z669.fasta.1’ saved [314/314]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That creates a &lt;code&gt;A2Z669.fasta&lt;/code&gt; file in the current directory which will contain the sequence.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2023-12-11T10:52:52.567" LastActivityDate="2023-12-11T10:52:52.567" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21937" PostTypeId="1" CreationDate="2023-12-10T14:30:30.113" Score="2" ViewCount="186" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://github.com/theislab/destiny/issues/64&quot; rel=&quot;nofollow noreferrer&quot;&gt;GitHub&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;In the R package destiny, a Diffusion Map is &lt;a href=&quot;https://scholar.google.com/scholar?hl=en&amp;amp;as_sdt=0%2C5&amp;amp;q=%22n_pcs%3D50%22%20%20%20destiny&amp;amp;btnG=&quot; rel=&quot;nofollow noreferrer&quot;&gt;often&lt;/a&gt; performed on the first 50 PCs of the data.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Is there a theoretical reason for performing a Diffusion Map on PCs rather than on the data itself ( other than speed)?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Is it reasonable to do the diffusion map on the normalized data itself rather than on Principal Components of the data, if that differentiates the states better ?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Techinically, I am using the  R Bioconductor package destiny. The help page on destiny::DiffusionMap tells &amp;quot;Using e.g. 50 PCs results in more regular looking diffusion maps&amp;quot;, but it's not clear compared to what will they be more regularly looking.&lt;/p&gt;&#xA;" OwnerUserId="6317" LastEditorUserId="73" LastEditDate="2023-12-14T17:16:07.850" LastActivityDate="2024-01-10T15:15:31.820" Title="Using Diffusion Map on normalized - or PCA reduced data?" Tags="|dimensionality-reduction|diffusion-map|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21939" PostTypeId="2" ParentId="21928" CreationDate="2023-12-11T00:03:06.453" Score="2" Body="&lt;p&gt;My main question is whether you refer to the fastq files after adapter trimming or before. It seems you mean the original fastq files. In that case, any read filtering done by fastp will remove low quality or totally-adapter reads.&lt;/p&gt;&#xA;&lt;p&gt;Note from the &lt;a href=&quot;https://github.com/OpenGene/fastp#filtering&quot; rel=&quot;nofollow noreferrer&quot;&gt;fastp docs&lt;/a&gt; that length trimming is enabled by default, so trimmed reads will be removed if they are shorter than the length setting, even if they are otherwise high quality. And if they are low quality then they will be filtered out for sure. I refer you to the docs to see what all the various default settings are. Possibly you could turn them all off and still see the initial number of reads, but I very much doubt that they would be useful for you.&lt;/p&gt;&#xA;&lt;p&gt;UPDATE: see also @Yossi Farjoun's excellent point about alignments vs. reads- not at all the same thing! though you can learn the exact number of reads from a BAM (using e.g. &lt;code&gt;samtools stats&lt;/code&gt;), the BAM is in fact a record of alignments. So it's important to be looking at the right number when you're doing your comparison!&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2023-12-15T17:45:34.750" LastActivityDate="2023-12-15T17:45:34.750" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21940" PostTypeId="1" CreationDate="2023-12-11T06:27:51.997" Score="0" ViewCount="47" Body="&lt;p&gt;I am getting this error message after using the following code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;C:\sratoolkit.3.0.7-win64\sratoolkit.3.0.7-win64\bin&amp;gt;fastq-dump --fasta SRR1658345&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;2023-12-11T06:08:04 fastq-dump.3.0.7 err: timeout exhausted while waiting condition within process system module - failed SRR1658345&#xA;&#xA;=============================================================&#xA;&#xA;An error occurred during processing.&#xA;A report was generated into the file 'C:\Users\Hp/ncbi_error_report.txt'.&#xA;If the problem persists, you may consider sending the file&#xA;to 'sra-tools@ncbi.nlm.nih.gov' for assistance.&#xA;&#xA;=============================================================&#xA;&#xA;2023-12-11T06:09:46 fastq-dump.3.0.7 err: connection failed while opening file within cryptographic module - Cannot KStblHttpFileTimedReadChunked: to=0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What should be done?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://pdf.ac/22SpHj&quot; rel=&quot;nofollow noreferrer&quot;&gt;Link to ncbi error report&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Edit:&#xA;I am on Windows, using Command Prompt.&lt;/p&gt;&#xA;" OwnerUserId="18663" LastEditorUserId="18663" LastEditDate="2023-12-16T11:42:10.690" LastActivityDate="2023-12-16T11:42:10.690" Title="What is the troubleshoot for this error: conversion of .SRA to FASTA file on command prompt?" Tags="|fasta|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21941" PostTypeId="2" ParentId="21931" CreationDate="2023-12-11T06:32:34.120" Score="1" Body="&lt;p&gt;I found a solution to this problem.&lt;/p&gt;&#xA;&lt;p&gt;After having installed SRA toolkit:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;prefetch -v SRR9....1&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;this helps download the SRA file.&lt;/p&gt;&#xA;&lt;p&gt;Then to extract it as FASTAQ&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;fasterq-dump SRR9....1&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="18663" LastActivityDate="2023-12-11T06:32:34.120" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21944" PostTypeId="1" CreationDate="2023-12-12T02:22:17.070" Score="0" ViewCount="15" Body="&lt;p&gt;Thank you in advance for your help!&lt;/p&gt;&#xA;&lt;p&gt;I am trying to download data from the CCDI (Children Cancer Data Initiative) and I have a manifest csv file that I downloaded from the CCDI. I'm trying to upload this data to my CGC project, but there is not option to import data from manifest file from the CCDI specifically, and when I do try to upload it, only the manifest csv is added and not the files. What is the process for uploading manifest that are not from one of these databases?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/B4h0g.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/B4h0g.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Thank you so much!&lt;/p&gt;&#xA;" OwnerUserId="18685" LastEditorUserId="18685" LastEditDate="2023-12-12T06:19:56.047" LastActivityDate="2023-12-12T06:19:56.047" Title="Uploading manifest from CCDI to CGC project" Tags="|database|genomics|public-databases|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21945" PostTypeId="1" CreationDate="2023-12-12T10:26:09.523" Score="1" ViewCount="106" Body="&lt;p&gt;I need to build a phylogenetic tree using IQ-TREE, starting from a sequence alignment in CODON format of several invertebrate mitochondrial genes. These are my charsets:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;nexus  &#xA;BEGIN SETS;&#xA;&#xA;charset atp6.fasta = 1-651;  &#xA;charset atp8.fasta = 652-729;  &#xA;charset cob.fasta = 730-1749;  &#xA;charset cox1.fasta = 1750-3237;  &#xA;charset cox2.fasta = 3238-3858;  &#xA;charset cox3.fasta = 3859-4632;  &#xA;charset nad1.fasta = 4633-5484;  &#xA;charset nad2.fasta = 5485-6156;  &#xA;charset nad3.fasta = 6157-6459;  &#xA;charset nad4.fasta = 6460-7683;  &#xA;charset nad4L.fasta = 7684-7899;  &#xA;charset nad5.fasta = 7900-9294;  &#xA;charset nad6.fasta = 9295-9663;  &#xA;END;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would need to remove the third bases, how should I write them? These are the commands with which I would launch IQ-TREE:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;iqtree -s &amp;quot;fasta_file&amp;quot; -bb 1000 -alrt 1000 -m MFP+MERGE -spp &amp;quot;charset.nexus&amp;quot; -st CODON5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Please, any suggestions would be very helpful.&lt;/p&gt;&#xA;" OwnerUserId="18689" LastEditorUserId="3848" LastEditDate="2023-12-12T17:17:23.863" LastActivityDate="2024-01-14T17:06:40.893" Title="How to remove third codon positions from a charset in iqtree?" Tags="|phylogenetics|phylogeny|fasta|mtdna|iqtree|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21946" PostTypeId="1" AcceptedAnswerId="21957" CreationDate="2023-12-12T15:21:38.190" Score="4" ViewCount="168" Body="&lt;p&gt;I am using &lt;a href=&quot;https://github.com/lh3/minimap2&quot; rel=&quot;nofollow noreferrer&quot;&gt;Minimap2&lt;/a&gt; (v2.26-r1175) in Linux to generate a sequence alignment between the &lt;em&gt;Streptomyces coelicolor&lt;/em&gt; A3(2) chromosome (&lt;code&gt;ref.fa&lt;/code&gt;) and the &lt;em&gt;Mycobacterium tuberculosis&lt;/em&gt; chromosome (&lt;code&gt;query.fa&lt;/code&gt;). My desired output is a &lt;a href=&quot;https://github.com/lh3/miniasm/blob/master/PAF.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;PAF (Pairwise mApping Format)&lt;/a&gt; file.&lt;/p&gt;&#xA;&lt;p&gt;The general way to align reference and query sequences with Minimap2 is the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 ref.fa query.fa &amp;gt; approx_mapping.paf&#xA;# OR equivalently...&#xA;minimap2 -k15 -w10 ref.fa query.fa &amp;gt; approx_mapping.paf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can get Minimap2 to generate custom CIGAR tags (&lt;code&gt;cg:&lt;/code&gt;) in the PAF by adding the &lt;code&gt;-c&lt;/code&gt; flag:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;minimap2 -c ref.fa query.fa &amp;gt; approx_mapping_cigar.paf&#xA;# OR equivalently...&#xA;minimap2 -c -k15 -w10 ref.fa query.fa &amp;gt; approx_mapping_cigar.paf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Moving to an R environment [with the intention of downstream data visualization], you can quickly assess the number of observations in each object (a.k.a. rows in each dataframe) with &lt;code&gt;dim()&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;file1 &amp;lt;- &amp;quot;approx_mapping.paf&amp;quot;&#xA;paf_basic &amp;lt;- read.table(file1, sep = &amp;quot;\t&amp;quot;, fill = TRUE,&#xA;                        col.names = paste0(&amp;quot;V&amp;quot;,seq_len(max(count.fields(file1, sep = &amp;quot;\t&amp;quot;)))))&#xA;dim(paf_basic)&#xA;[1] 205  18&#xA;&#xA;file2 &amp;lt;- &amp;quot;approx_mapping_cigar.paf&amp;quot;&#xA;paf_cigar &amp;lt;- read.table(file2, sep = &amp;quot;\t&amp;quot;, fill = TRUE,&#xA;                        col.names = paste0(&amp;quot;V&amp;quot;,seq_len(max(count.fields(file2, sep = &amp;quot;\t&amp;quot;)))))&#xA;dim(paf_cigar)&#xA;[1] 200  24&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;Note the unqiue way we have to read the PAF files into R using &lt;code&gt;read.table()&lt;/code&gt; to &lt;a href=&quot;https://stackoverflow.com/a/18922914&quot;&gt;ensure no columns are cut off and added to new rows&lt;/a&gt;. This is because &lt;code&gt;read.table()&lt;/code&gt; is designed to examine only the first five lines of an input file to determine the number of columns in the dataframe. Any columns on lines beyond the fifth line that exceed the initially determined count are appended to a new row. In the case of PAF files generated by Minimap2 &lt;strong&gt;with CIGAR tags&lt;/strong&gt;, the number of columns can vary per line, which makes this necessary.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;With the &lt;strong&gt;default settings&lt;/strong&gt;, I get &lt;strong&gt;205 observations&lt;/strong&gt; and with the &lt;strong&gt;addition of the CIGAR flag&lt;/strong&gt;, I get &lt;strong&gt;200 observations&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Why do I get fewer computed alignments when the CIGAR tags are added?&lt;/strong&gt; I would expect the alignments between the reference and query to be identical between these runs.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Comparing the PAF files (directly generated by &lt;code&gt;minimap2&lt;/code&gt;) after sorting (&lt;code&gt;sort -k3,3 -k4,4&lt;/code&gt;) using &lt;code&gt;vimdiff&lt;/code&gt; reveals that the alignments computed between the default and CIGAR setting runs are largely &lt;em&gt;similar&lt;/em&gt;, but not identical:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bGQze.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bGQze.png&quot; alt=&quot;vimdiff_of_default_and_cigar_minimap2_output_files&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Note that the image above only shows the first 44 rows of both files.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Due to the small differences seen in alignment coordinates between these two dataframes (see fields &lt;code&gt;$3&lt;/code&gt; and &lt;code&gt;$4&lt;/code&gt; in the &lt;code&gt;vimdiff&lt;/code&gt; screenshot), it is &lt;strong&gt;NOT&lt;/strong&gt; trivial to determine which five observations (rows) are new alignments computed when using the CIGAR flag &lt;code&gt;-c&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Plotting these dataframes confirms that the &lt;code&gt;paf_basic&lt;/code&gt; and &lt;code&gt;paf_cigar&lt;/code&gt; objects do, in fact, contain some different sequence alignments from each other:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(&amp;quot;ggplot2&amp;quot;)&#xA;ggplot(data = paf) +&#xA;  geom_segment(aes(x = V8, xend = V9, y = V3, yend = V4), size = 2) +&#xA;  labs(title = &amp;quot;paf_name&amp;quot;, x = &amp;quot;Reference (Mb)&amp;quot;, y = &amp;quot;Query (Mb)&amp;quot;) +&#xA;  coord_fixed(ratio = 1) +&#xA;  theme_bw()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gu3ON.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gu3ON.png&quot; alt=&quot;paf_basic_dot_plot&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/dfB3N.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/dfB3N.png&quot; alt=&quot;paf_cigar_dot_plot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So, it appears as though &lt;code&gt;minimap2 -c&lt;/code&gt; produces a slightly different alignment than &lt;code&gt;minimap2&lt;/code&gt;, which is surprising to me and begs the question: &lt;strong&gt;Which alignment is better or preferable?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="9007" LastEditorUserId="9007" LastEditDate="2023-12-17T18:44:02.013" LastActivityDate="2023-12-20T05:34:38.300" Title="Why are Minimap2 alignments different with CIGAR generation flag?" Tags="|sequence-alignment|minimap2|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="21947" PostTypeId="2" ParentId="21945" CreationDate="2023-12-12T17:15:39.790" Score="0" Body="&lt;p&gt;Yes, it's easy.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#nexus  &#xA;BEGIN SETS;&#xA;&#xA;charset atp6.fasta = 1-651\3, 2-651\3;  &#xA;charset atp8.fasta = 652-729\3, 653-729\3;  &#xA;charset cob.fasta = 730-1749\3, 731-1749\3;  &#xA;# ... so on for every gene&#xA;END;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So for atp6 you will get 1,2,4,5,7,8,10,11 .... up to 651, and so on for each gene. Thus the residues at position 3, 6, 9, 12 are excluded.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-12-15T16:25:55.133" LastActivityDate="2023-12-15T16:25:55.133" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21950" PostTypeId="2" ParentId="18595" CreationDate="2023-12-13T09:11:12.080" Score="2" Body="&lt;p&gt;Very old question, but recently modified, which means that someone might still be looking into this.&lt;/p&gt;&#xA;&lt;p&gt;I usually prefer to use &lt;a href=&quot;https://github.com/carmonalab/UCell&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ucell&lt;/a&gt; as the signatures are calculated within cells, which makes comparisons a bit easier. In addition is allows the formulation of signatures with both &lt;code&gt;positive&lt;/code&gt; and &lt;code&gt;negative&lt;/code&gt; genes.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;hladr_genes &amp;lt;- grep(pattern = '^HLA-DR', x = rownames(x = pbmc_small), value = TRUE)&#xA;signatures_list &amp;lt;- list(MDSCs = c(&amp;quot;CD14+&amp;quot;,&amp;quot;CD33+&amp;quot;,&amp;quot;ITGAM+&amp;quot;,paste0(hladr_genes,&amp;quot;-&amp;quot;)))&#xA;&#xA;&amp;gt; signatures_list&#xA;$MDSCs&#xA;[1] &amp;quot;CD14+&amp;quot;     &amp;quot;CD33+&amp;quot;     &amp;quot;ITGAM+&amp;quot;    &amp;quot;HLA-DRA-&amp;quot;  &amp;quot;HLA-DRB5-&amp;quot; &amp;quot;HLA-DRB1-&amp;quot;&#xA;&#xA;# calculate signature&#xA;pbmc_small &amp;lt;- AddModuleScore_UCell(pbmc_small,signatures_list,ncores = 8,name = &amp;quot;&amp;quot;)&#xA;# perform smoothing&#xA;pbmc_small &amp;lt;- SmoothKNN(pbmc_small, signature.names = names(signatures_list), reduction=&amp;quot;pca&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3631" LastActivityDate="2023-12-13T09:11:12.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21951" PostTypeId="1" CreationDate="2023-12-14T10:07:12.080" Score="1" ViewCount="44" Body="&lt;p&gt;I am replicating differential expression analysis from a paper (&lt;a href=&quot;https://www.researchgate.net/publication/325322361_Overexpression_of_VaWRKY14_increases_drought_tolerance_in_Arabidopsis_by_modulating_the_expression_of_stress-related_genes&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;) on RNA-Seq data. I extracted the regulated genes using &lt;code&gt;DESeq2::results&lt;/code&gt; with a threshold of 1 for log2 fold change and 0.05 for the p-value. The results are very different from that of the paper. The paper does not mention much information about their statistical methods except for this paragraph:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Finally, the libraries were sequenced using the BGISEQ-500 platform. Differentially expressed genes were identified based on fold change ≥ 2 and diverge probability ≥ 0.8. The functional categories of the differently expressed genes and the pathway were obtained using MapMan as the classification source.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Now, I have no idea what they mean by &amp;quot;diverge probability&amp;quot; but it looks like another method to get the regulated genes. My questions are:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 1:&lt;/strong&gt; What could be the method they are using in their analysis?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 2:&lt;/strong&gt; Are the results from the &amp;quot;diverge probability&amp;quot; method different from the ones we get using &lt;code&gt;DESeq2::results&lt;/code&gt;?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Question 3:&lt;/strong&gt; If the answer to Question 2 is yes, how do we get from one to the other?&lt;/p&gt;&#xA;&lt;p&gt;I appreciate any insights or experiences you can share regarding this approach. Thank you!&lt;/p&gt;&#xA;" OwnerUserId="18703" LastEditorUserId="18703" LastEditDate="2023-12-15T11:14:02.413" LastActivityDate="2023-12-15T11:14:02.413" Title="What method uses &quot;diverge probability&quot; in RNA-Seq differential expression analysis?" Tags="|r|rna-seq|differential-expression|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21953" PostTypeId="1" AcceptedAnswerId="21954" CreationDate="2023-12-14T20:24:58.873" Score="1" ViewCount="30" Body="&lt;p&gt;My PI has asked me for advice on making a PCR screen to identify mice from a particular Cre line.  We have multiple Cre lines in the colony so it can't just be for mice with a copy of Cre in their genome the primer. The insertion site is known for the mutation,  is there a way to use blast or some other search tool to pull the sequences we should base our primers on?&lt;/p&gt;&#xA;&lt;p&gt;We also want to identify which mice are hetero vs homozygous.  Are there particular kinds of PCR we should use or will a well designed primer enable this?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Details&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The strain:  &lt;a href=&quot;https://www.jax.org/strain/004146&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.jax.org/strain/004146&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I was thinking of using this tool:  &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/tools/primer-blast/index.cgi?LINK_LOC=BlastHome&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/tools/primer-blast/index.cgi?LINK_LOC=BlastHome&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The strain reference from Jax, not sure the recommended &lt;strong&gt;cre&lt;/strong&gt; primers are going to be sufficient for what we want.  So I'm trying to find the FASTA file for this strain in the region of interest, then I can throw that into the BLAST tool for primer design I listed above.  Not sure where to search for the FASTA for this strain though. Strain information: &lt;a href=&quot;https://www.informatics.jax.org/reference/J:66884&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.informatics.jax.org/reference/J:66884&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Jax recommends this protocol so I suggested it:  &lt;a href=&quot;https://www.jax.org/Protocol?stockNumber=004146&amp;amp;protocolID=20627&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.jax.org/Protocol?stockNumber=004146&amp;amp;protocolID=20627&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2482" LastEditorUserId="3848" LastEditDate="2023-12-15T00:48:58.150" LastActivityDate="2023-12-15T00:48:58.150" Title="PCR primers for detecting hetero vs homozygous and WT mice in our colony" Tags="|primer|pcr|qpcr|mice|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21954" PostTypeId="2" ParentId="21953" CreationDate="2023-12-14T22:25:19.080" Score="1" Body="&lt;p&gt;Primer-blast is okay, but I presume these are lab mice so the variation is low, thus  you can just use Primer3. SNPs are detected via TaqMan &lt;a href=&quot;https://www.thermofisher.com/us/en/home/life-science/pcr/real-time-pcr/real-time-pcr-assays/snp-genotyping-taqman-assays.html?cid=gsd_pcr_sbu_r03_co_cp1491_pjt9621_gsd00000_0se_gaw_rs_pur_&amp;amp;ef_id=Cj0KCQiA7OqrBhD9ARIsAK3UXh1z5e06QTnP0pbSLGufHAvSHGh9vAvaNNLLNRoDiQVRnR9zgH6thosaAn1mEALw_wcB:G:s&amp;amp;s_kwcid=AL!3652!3!606132911456!e!!g!!taqman%20snp%20genotyping%20assay!17574808757!139287689898&amp;amp;gad_source=1&amp;amp;gclid=Cj0KCQiA7OqrBhD9ARIsAK3UXh1z5e06QTnP0pbSLGufHAvSHGh9vAvaNNLLNRoDiQVRnR9zgH6thosaAn1mEALw_wcB&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;, i.e. a separate probe that sits over the mutation site, it will detect heterozygosity via qPCR signal (half strength than homozygous).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2023-12-14T22:25:19.080" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21955" PostTypeId="2" ParentId="21937" CreationDate="2023-12-14T22:56:11.740" Score="2" Body="&lt;h3&gt;The empirical statistical point of view&lt;/h3&gt;&#xA;&lt;p&gt;The technique is  uniform manifold approximation and projection (UMAP) and is comparable to t-SNE. UMAP is considered a slight improvement, personally I think they have similar behaviour. They are dimensionality reduction methods.&lt;/p&gt;&#xA;&lt;p&gt;The reason UMAP or t-SNE is often not applied directly is it is weaker than PCA are capturing the multi-dimensional variance of the data. Both methods can be applied directly to the data, but will not necessarily deliver stronger clusters than PCA (which is the goal of the technique).&lt;/p&gt;&#xA;&lt;p&gt;However, the weakness of PCA is that the variance can be spread across 50 principle components, making it hard to recover the representative clusters in certain situations. These situations are commonly found in RNA expression analytics, notably where the first 3 or 4 PCs only comprise ~30% of the variance. PCA should be trapping &amp;gt;80% of the variance within the first 3 or 4 PCs. In expression data (RNAseq) the next 46 PCs might comprise a total of ~30% of the variance and omitting that is mistake, but manually comparing 50 PCs in bivariate plots is not practical. So at best a good unsupervised learning algorithm (UMAP/t-SNE) following PCA in RNA expression via RNAseq might trap 60% of the variance. It's not ideal but it's okay. Thus you can't ignore the 30% of the variance in the bottom 46 PCs and UMAP/t-SNE is recovering that. If 80% of the variance was in the top 4 PCs, then the remaining 20% of the variance in the lower 46 PCs can be ignored and t-SNE or UMAP is not needed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Resolution&lt;/strong&gt; The way you can capture the variance from all 50 PCs is to perform a clustering method over the top of PCA, this being t-SNE or UMAP.&lt;/p&gt;&#xA;&lt;p&gt;Summary,&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;t-SNE or UMAP without PCA might trap at best 20% of the variance.&lt;/li&gt;&#xA;&lt;li&gt;The first 4 PCs in a PCA might trap 30% of the variance (it will be a better result than t-SNE or UMAP).&lt;/li&gt;&#xA;&lt;li&gt;Combining PCA and then UMAP/t-SNE will capture 60% of the variance.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;60% of the variance, its just about acceptable. Thus thats why this approach  is done from an empirical point of view.&lt;/p&gt;&#xA;&lt;h3&gt;Practical point of view&lt;/h3&gt;&#xA;&lt;p&gt;PCA and then UMAP (or t-SNE) will give tighter, neater clusters (cleaner, less messy groups), but thats not the reason why you are using it.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;The publications in the link,&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10312943/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10312943/&lt;/a&gt;&#xA;and the PLoS Biology paper&#xA;&lt;a href=&quot;https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3001532&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3001532&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;... both use UMAP across PCA. Its a major approach to cluster resolution.&lt;/p&gt;&#xA;&lt;p&gt;Diffusion maps are an attempt to extrapolate a variance within Eigenvalues via a random walk, i.e. to defuse within Euclidean space. It is not a primary calculation in my personal opinion because you are sort of introducing more variance into a variance dimensionality reduction calculation. Once you have very tight clusters using PCA/UMAP or PCA/tSNE then you could check to see whether Diffusion map clusters are similar.&lt;/p&gt;&#xA;&lt;p&gt;This is my perspective only, not to replace PCA with Diffusion map nor to consider a &amp;quot;down-stream&amp;quot; calculation. Its separate calculation used to verify if PCA-UMAP is okay.&lt;/p&gt;&#xA;&lt;p&gt;If you doing graph theory a Diffusion map could be really important as a null hypothesis and a really important way to obtain a different understanding of the graph (i.e. this is a technical term for a network). Using a diffusion map without any directionality (the papers you cited)*, I struggle to see this as a primary analysis. In the PLOS Biology paper is part of the appendix.&lt;/p&gt;&#xA;&lt;p&gt;*, graph says A-&amp;gt;B-&amp;gt;C&#xA;The publications treat A,B, C as unconnected. If you can hypotheses how A, B and C are connected then diffusion map could be cool. Thats my opinion from the perspective of graph theory.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Summary&lt;/strong&gt; I would definitely choose PCA-UMAP over a diffusion map every time. This is what has happened in the two publications cited. The diffusion map is secondary analysis, so by all means do it ... but present PCA-UMAP first, and base the results around that.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;From the comments there's some sort of time-series. So if you are performing a time-series - it's not datamining - this would be the justification of using a Diffusion map over PCA. This therefore answers points 1 and 2. However, it would imply you are pooling your data.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2024-01-02T14:44:36.410" LastActivityDate="2024-01-02T14:44:36.410" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="21956" PostTypeId="1" CreationDate="2023-12-14T23:55:46.107" Score="0" ViewCount="25" Body="&lt;p&gt;I am trying to make a dataset for methylation analysis.&#xA;I called methylation using Methyldackel and there are various positions without methylation calls. This creates a problem because when I try to use for example, DMCHMM or MethylSig, to estimate differentially methylated regions, the initial step prompts an error:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;Building BSData object.&#xA;Warning message:&#xA;In .merge_two_Seqinfo_objects(x, y) :&#xA;  Each of the 2 combined objects has sequence levels not in the other:&#xA;  - in 'x': NW_022883351.1, NW_022883409.1&#xA;  - in 'y': NW_022883010.1&#xA;  Make sure to always combine/compare objects based on the same reference&#xA;  genome (use suppressWarnings() to suppress this warning).&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I used the same reference for all my samples, but when it tries to combine the files, it complains because some samples don't have data (missing) in some positions.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to use R or Linux to fix this, and check those lines positions with missing data and add those (lines) positions in the files?&lt;/p&gt;&#xA;&lt;p&gt;The files look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chrBase chr     base    strand  coverage        freqC   freqT&#xA;NW_022882922.1.4856     NW_022882922.1  4856    F       2         0.00  100.00&#xA;NW_022882922.1.4866     NW_022882922.1  4866    F       2         0.00  100.00&#xA;NW_022882922.1.4889     NW_022882922.1  4889    F       4         0.00  100.00&#xA;NW_022882922.1.4892     NW_022882922.1  4892    F       4         0.00  100.00&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;so that all the files have &amp;quot;information&amp;quot; in all the positions?&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;&lt;p&gt;Juan Pablo&lt;/p&gt;&#xA;" OwnerUserId="15593" LastEditorUserId="650" LastEditDate="2023-12-15T15:38:29.023" LastActivityDate="2023-12-15T15:38:29.023" Title="Making a matrix when samples that have missing data" Tags="|matrix|differential-methylation|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21957" PostTypeId="2" ParentId="21946" CreationDate="2023-12-15T01:15:09.790" Score="5" Body="&lt;p&gt;The program's author pointed me to the &lt;a href=&quot;https://github.com/lh3/minimap2/blob/master/FAQ.md&quot; rel=&quot;noreferrer&quot;&gt;Minimap2 FAQ&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;1. Alignment different with option &lt;code&gt;-a&lt;/code&gt; or &lt;code&gt;-c?&lt;/code&gt;&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Without &lt;code&gt;-a&lt;/code&gt;, &lt;code&gt;-c&lt;/code&gt; or &lt;code&gt;--cs&lt;/code&gt;, minimap2 only finds &lt;em&gt;approximate&lt;/em&gt;&#xA;mapping locations without detailed base alignment. In particular, the&#xA;start and end positions of the alignment are impricise. With one of&#xA;those options, minimap2 will perform base alignment, which is&#xA;generally more accurate but is much slower.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;So, to specifically address the main questions asked here:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;strong&gt;&lt;em&gt;Why are there fewer and non-identical computed alignments when the CIGAR tags are added?&lt;/em&gt;&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;&lt;code&gt;minimap2&lt;/code&gt; alignment accuracy differs when running &lt;code&gt;minimap2 -k15 -w10&lt;/code&gt; &lt;em&gt;vs.&lt;/em&gt; when running &lt;code&gt;minimap2 -c -k15 -w10&lt;/code&gt;. It is actually &lt;em&gt;not unexpected&lt;/em&gt; that the alignment coordinates in each PAF file are only &lt;em&gt;similar&lt;/em&gt;, not identical, and an increase or decrease in &lt;em&gt;the number of alignments&lt;/em&gt; between the outputs is simply an artefact of this accuracy discrepancy.&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;strong&gt;&lt;em&gt;Which alignment (default vs. with CIGAR tags) is better or preferable?&lt;/em&gt;&lt;/strong&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Although both alignments report very similar sequence relationships, adding the &lt;code&gt;-c&lt;/code&gt; flag for CIGAR tag generation is preferable to get more accurate base alignments.&lt;/p&gt;&#xA;" OwnerUserId="9007" LastActivityDate="2023-12-15T01:15:09.790" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21958" PostTypeId="2" ParentId="21928" CreationDate="2023-12-15T01:19:19.297" Score="2" Body="&lt;p&gt;&amp;quot;Total number of reads in R1 and R2 FastQ files = mapped and unmapped reads in the corresponding BAM file.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;there might be confusion regarding the word &amp;quot;read&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;record&amp;quot; refers to an entry in the BAM file (a row in SAM)&#xA;&amp;quot;template&amp;quot; refers to the original sequence from the fastq/sequencer.&lt;/p&gt;&#xA;&lt;p&gt;after mapping a single template may be the source of two different &lt;em&gt;records&lt;/em&gt; due to secondary and supplementary alignments.&lt;/p&gt;&#xA;&lt;p&gt;So the general answer to your question is: Not if you count &amp;quot;reads&amp;quot; as rows in the SAM file Yes, if you count carefully and make sure to not double-count records that have the same &amp;quot;query name&amp;quot; (the name from the fastq file)&lt;/p&gt;&#xA;&lt;p&gt;For more information about SAM/BAM/CRAM the &lt;a href=&quot;https://samtools.github.io/hts-specs/SAMv1.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;spec&lt;/a&gt; can provide hours of light reading... :-)&lt;/p&gt;&#xA;" OwnerUserId="9836" LastActivityDate="2023-12-15T01:19:19.297" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21959" PostTypeId="1" CreationDate="2023-12-16T15:47:17.670" Score="2" ViewCount="79" Body="&lt;p&gt;I am doing some coverage analysis on deep sequencing human genome. I first align to the whole hg38 genome, then split bam to each chrom,  and collect metrics on bam for each chom.&#xA;I first split using samtools. and then tried bamtools.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools: samtools view -bo ${pair_id}.\\\$chr.bam $bam \\\$chr&#xA;bamtools:bamtools split -in $bam -reference &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I collect metrics I am using the reference fasta from each chrom. The picard metrics complains the index dictionary size is different:  Sequence dictionaries are not the same size (25, 1). The picard method I used is: MarkDuplicates, CollectAlignmentSummaryMetrics, CollectGcBiasMetrics, CollectInsertSizeMetrics, CollectWgsMetrics.&lt;/p&gt;&#xA;&lt;p&gt;My data is really huge, 70G for a sample, that is why I would like to split when doing the picard metrics. Seems picard metrics collection use a lot of memory. So for the 70G human fastq(30X), how big the memory is needed for running picard wgs metrics?&lt;/p&gt;&#xA;&lt;p&gt;How can I deal with this? Can I modify the split bam header? How to do it?&#xA;Or not split it into chromosmes, rather, using an interval list when collecting the metrics? will that use less memory?&lt;/p&gt;&#xA;" OwnerUserId="14191" LastEditorUserId="14191" LastEditDate="2023-12-17T12:27:54.310" LastActivityDate="2024-01-24T02:19:09.843" Title="align to the whole hg38 genome, then split bam and collect metrics on each bam issue" Tags="|sequence-alignment|picard|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21960" PostTypeId="1" CreationDate="2023-12-16T19:45:41.960" Score="0" ViewCount="28" Body="&lt;p&gt;I am trying to do pathway analysis on a set of proteins associated with several outcomes (one is continuous, the others four are binary). I have run independent associations between the proteomics data and the outcomes in question and have ascertained betas, hazard ratios, p values, and associated multiple-testing correction p-values. Out of the set of 3000 proteins for which we have proteomics data, we have found significant FDR corrected associations with our outcomes in ~1800. I am trying to do pathway analysis with these proteins.&lt;/p&gt;&#xA;&lt;p&gt;I am trying several pathway analysis approaches, including &lt;code&gt;fgsea&lt;/code&gt;, online GSEA through GenePattern, and Ingenuity Pathway Analysis. For the GSEA approaches, I have been able to generate the ranked list and provide the ranked list to the software. However, it is my understanding that IPA is a form of overrepresentation analysis and therefore I should not provide a ranked list. However, when uploading my dataset to the IPA application, it asks for expression change values (my understanding is that all these approaches were originally designed for microarray-based  differential expression data).&lt;/p&gt;&#xA;&lt;p&gt;Since we are looking at the association between proteomics and several outcomes, expression change values are not relevant to these data. However, the directionality of the effect is relevant when considering pathways, and since we have the betas for the association it seems to me that I should include it. I looked through the IPA documentation and all of it seems specific to differential expression analysis. Can anyone point me in the correct direction to perform IPA on the association between my outcomes and proteomics data? Any help would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="16999" LastActivityDate="2023-12-16T19:45:41.960" Title="How can I use QIAGEN Ingenuity Pathway Analysis (IPA) on proteomics data associated with an outcome?" Tags="|gsea|pathway|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21962" PostTypeId="1" CreationDate="2023-12-18T13:03:53.080" Score="0" ViewCount="30" Body="&lt;p&gt;Is there a way to blast my proteins against all the TSA (Transcriptome Shotgun Assembly Sequence Database) belonging to a specific taxonomy using command line?&lt;/p&gt;&#xA;&lt;p&gt;Thanks,&lt;br /&gt;&#xA;Marco&lt;/p&gt;&#xA;" OwnerUserId="15596" LastEditorUserId="650" LastEditDate="2023-12-18T21:29:34.470" LastActivityDate="2024-01-17T22:03:20.390" Title="tBLASTn vs NCBI TSA" Tags="|proteins|taxonomy|blastn|tblastn|tsa|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21963" PostTypeId="1" CreationDate="2023-12-18T14:48:57.290" Score="0" ViewCount="18" Body="&lt;p&gt;Do you know if it is actually possible to predict with good accuracy a DNA binding motif from only the sequence/structure of the DNA-binding protein?&lt;/p&gt;&#xA;" OwnerUserId="18720" LastActivityDate="2023-12-18T14:48:57.290" Title="Prediction of DNA binding motif from sequence/structure of TF/regulator protein?" Tags="|machine-learning|docking|motifs|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21964" PostTypeId="2" ParentId="21962" CreationDate="2023-12-18T15:00:16.317" Score="0" Body="&lt;p&gt;Just go to Blast &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PROGRAM=blastn&amp;amp;PAGE_TYPE=BlastSearch&amp;amp;LINK_LOC=blasthome&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Click where is says Database go to Transcriptome Shotgun Assembly Sequence Database (TSA).&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Click where is says &amp;quot;Limited by&amp;quot; click organism and thereon specify the group of interest.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thats it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-12-18T21:16:29.837" LastActivityDate="2023-12-18T21:16:29.837" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21965" PostTypeId="1" CreationDate="2023-12-18T22:03:26.503" Score="0" ViewCount="73" Body="&lt;p&gt;I'm wondering how can I show unique gene names in the volcano plot.&lt;/p&gt;&#xA;&lt;p&gt;My data looks like,&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;head(DMCf1_huMetGNc)&#xA;     chr    start    end.x       pvalue       qvalue meth.diff        gene&#xA;152 chr1 24909805 24909805 1.915400e-03 0.0168087962 -52.00000 RUNX3;RUNX3&#xA;168 chr1 24931440 24931440 8.645625e-03 0.0489503201  28.98551 RUNX3;RUNX3&#xA;179 chr1 26862096 26862096 1.308307e-04 0.0023224596 -42.48366         SFN&#xA;187 chr1 26914178 26914178 4.508416e-06 0.0001786963 -57.97872       NR0B2&#xA;188 chr1 26914203 26914203 5.337425e-05 0.0011910452 -47.30424       NR0B2&#xA;194 chr1 35573100 35573100 1.433689e-03 0.0135780139 -42.04545      TFAP2E&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The original data file is quite large, here under the &lt;code&gt;gene&lt;/code&gt; column some of the rows have multiple same gene names separated by &lt;code&gt;;&lt;/code&gt;. How can I show only unique gene names.&lt;/p&gt;&#xA;&lt;p&gt;No pattern of gene names exists when multiple names are present. In certain cases, the same gene is repeated, while in other cases, distinct genes are present in the same cell. The majority of the time, a single gene exists in a cell.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(ggplot2)&#xA;library(ggrepel)&#xA;    v1&amp;lt;-ggplot(data=DMCf1_huMetGNc, aes(x=meth.diff, y=-log10(pvalue) )) + geom_point()&#xA;    v1+geom_text_repel(aes(label= gene))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/KuoIc.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/KuoIc.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here, you may see how the FGFR2 gene is repetitive in the plot that I want to remove and the same name from a single row should appear only once in the plot.&lt;/p&gt;&#xA;" OwnerUserId="12770" LastEditorUserId="73" LastEditDate="2023-12-21T09:06:32.647" LastActivityDate="2023-12-21T16:46:40.133" Title="Volcano Plot: How to show unique gene names in the plot?" Tags="|r|genomics|ggplot2|methylation|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="21967" PostTypeId="1" CreationDate="2023-12-19T17:21:00.963" Score="0" ViewCount="39" Body="&lt;p&gt;I was wondering if there is a way to provide pacbio lima demultiplexing only a barcode, and let  lima demux sample even if only one end has the barcode, not both?&lt;/p&gt;&#xA;" OwnerUserId="14191" LastActivityDate="2024-02-02T02:00:25.047" Title="pacbio demultiplexing issue" Tags="|pacbio|demultiplexing|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21968" PostTypeId="1" CreationDate="2023-12-19T22:16:38.227" Score="2" ViewCount="48" Body="&lt;p&gt;I installed &lt;a href=&quot;https://github.com/bcgsc/LongStitch&quot; rel=&quot;nofollow noreferrer&quot;&gt;Longstitch&lt;/a&gt; and ran the test script with no issues. The output files matched the expected output files. But when I am now trying to run Longstitch on my own data I am getting this error.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(medaka) operations@LabPC1:~/medaka/longstitch-1.0.5$ ./longstitch run draft=~/medaka/~medaka/medaka_results/consensus.fa reads=~/medaka/combined.fastq G=4.5e8&#xA;make: command: Command not found&#xA;make: command: Command not found&#xA;make: *** No rule to make target'/home/operations/medaka/~medaka/medaka_results/consensus.fa.fa', needed by'/home/operations/medaka/~medaka/medaka_results/consensus.fa.cut250.tigmint.fa'.  Stop.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I changed the file name from .fasta to .fa for my draft assembly because it requires a .fa file, but now I am getting &amp;quot;.fa.fa&amp;quot; What is wrong here?&lt;/p&gt;&#xA;" OwnerUserId="18726" LastEditorUserId="73" LastEditDate="2023-12-21T19:13:38.910" LastActivityDate="2023-12-21T19:13:38.910" Title="Longstitch error make: command: Command not found *** No rule to make target" Tags="|sequence-analysis|assembly|wgs|long-reads|scaffold|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21969" PostTypeId="2" ParentId="21968" CreationDate="2023-12-20T00:54:57.617" Score="1" Body="&lt;p&gt;Could you download it from &lt;code&gt;conda&lt;/code&gt; and try again? Its here, &lt;a href=&quot;https://docs.conda.io/en/main/miniconda.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://docs.conda.io/en/main/miniconda.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda create --name myenv&#xA;conda activate myenv&#xA;conda install -c bioconda longstitch&#xA;longstitch ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Conda avoids all the build stuff.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Notes&lt;/p&gt;&#xA;&lt;p&gt;If you find that &amp;quot;conda&amp;quot; is slow and doesn't ever load.&lt;/p&gt;&#xA;&lt;p&gt;There is an alternative to conda called &lt;strong&gt;mamba&lt;/strong&gt;, this is fast and designed to address this exact issue. I have never needed to use it so the following is a rough guide only.&lt;/p&gt;&#xA;&lt;p&gt;Download and install via&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda create --name myenv&#xA;conda activate myenv&#xA;conda install -c conda-forge mamba&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The documentation is here, &lt;a href=&quot;https://mamba.readthedocs.io/en/latest/user_guide/mamba.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://mamba.readthedocs.io/en/latest/user_guide/mamba.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I've never used it but the idea is ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mamba create -n mambaenv &amp;lt;list of packages&amp;gt;&#xA;mamba activate mambaenv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The list of packages could be left blank for now, then,&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mamba install &amp;quot;Python&amp;gt;=3.10.0&amp;quot; numpy pandas pathlib&#xA;mamba install longstitch &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can also do&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mamba install -n mambaenv longstitch&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;NOTE&lt;/strong&gt; longstitch exists strictly in -c bioconda. You do not seem to search and install via channels in mamba&lt;/p&gt;&#xA;&lt;p&gt;You search for a package and its dependencies via&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;mamba repoquery search longstitch&#xA;mamba repoquery depends -a longstitch&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It is important to place a virtual environment between the system Python and package Python (don't want a system corruption).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Finally ...&#xA;For &lt;code&gt;fa.fa&lt;/code&gt; ... hmmmm, try &lt;code&gt;ls *.fa&lt;/code&gt; and check for &lt;code&gt;.fa.fa&lt;/code&gt; files.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2023-12-20T01:30:58.637" LastActivityDate="2023-12-20T01:30:58.637" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21970" PostTypeId="2" ParentId="21946" CreationDate="2023-12-20T05:34:38.300" Score="0" Body="&lt;p&gt;Note also that the &amp;quot;score&amp;quot; can vary dramatically depending on whether -c is used or not. I made a short post about this here &lt;a href=&quot;https://gist.github.com/cmdcolin/dbb1c6ae291d0e8282cfd0455f908fea&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gist.github.com/cmdcolin/dbb1c6ae291d0e8282cfd0455f908fea&lt;/a&gt; it is also alluded to briefly in the PAF.md where score is a required field but &amp;quot;If alignment is not available, column 10 and 11 are still required but may be highly inaccurate.&amp;quot;&lt;/p&gt;&#xA;" OwnerUserId="60" LastActivityDate="2023-12-20T05:34:38.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21971" PostTypeId="1" CreationDate="2023-12-20T05:46:00.823" Score="1" ViewCount="80" Body="&lt;p&gt;I used FIMO to scan several of my promoter sequences for TF binding motifs. The output .tsv file gives me the motif name/sequence plus the start and stop location, how can I visualize these outputs to get a graph that shows each motif as differing length rectangles along each promoter sequence, along with a gradient that shows the p-value?&lt;/p&gt;&#xA;" OwnerUserId="18137" LastActivityDate="2024-01-22T22:03:00.317" Title="How to visualize FIMO outputs?" Tags="|bed|motifs|tsv|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21972" PostTypeId="1" AcceptedAnswerId="21979" CreationDate="2023-12-20T07:40:05.003" Score="1" ViewCount="368" Body="&lt;p&gt;a python beginner here.&#xA;I have a fasta file with 2500+ sequences, and after doing some analysis I want to remove around 200+ sequences based on the matching IDs. Now, I have one fasta file (as sample.fa) and a text file with a list of IDs for the sequences that I want to remove from this fasta file. Below is an example of the fasta sequences. For simplicity, I have shortened the sequences and their respective IDs. The ID in the text file (for the sequences to remove, and in the fast file will be exactly same). The sequences were originally downloaded form NCBI and had very long headers so I have them shortened as at this point I don't need all that info, usually just the sequence ID in the headers is enough.&lt;/p&gt;&#xA;&lt;p&gt;Sample fasta sequences in fasta file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;WP_017417394.1-Bacillus_amyloliquefaciens_gp.&#xA;MRGKKVWISLLFALALIFTMAFGSTSPAQAAGKSNGEKKYIVGFKQTMSTMSAAKKKDVISEKGGKVQKQFKYVDAASAT&#xA;&amp;gt;WP_268291751.1-Bacillus_inaquosorum&#xA;MRRKKLWISLLFALTLIFTMAFSNMSAQAAGKSSTEKKYIVGFKETMSAMSSAKKKDVISEKGGKVQKQFKYVNAATATL&#xA;&amp;gt;WP_003327717.1-Bacillus_atrophaeus&#xA;MRSRKLWIGLLFAFTLVFTMAFSSMSPAQAAGKSSEEKKYIVGFKQTMGAMSAAAKKDVIAENGGKLQKQFKYVDAATAT&#xA;&amp;gt;WP_206702175.1-Bacillus_vallismortis&#xA;MAFSNMSAQAAGKSSTEKKYIVGFKQTMSVMSSAKKKDVISEKGGKVQKQFKYVNAAAATLDAKAVKELKQDPSVAYVEE&#xA;&amp;gt;KAF1680298.1-Bacillus_sp.&#xA;MKSKKLWISLLFALTLVFTMAFSNMSAQASGKNSTEKKYIVGFKQTMSTAKKKDIISEKGGKVQKQFKYVNAASATLDQK&#xA;&amp;gt;WP_172293960.1-Bacillus_sp.&#xA;MGKKRLWYSLLTVLLLVFSMAFSHPANAVKSSAKDVKKDYIVGFKSSLKTTSAKKDVIKENGGKVDKHFRSINAAKVTLD&#xA;&amp;gt;WP_082194748.1-Bacillus_xiamenensis&#xA;MCVKKKNVMTSVLLAVPLLFSAGFGGSMANAETASKSESEKSYIVGFKASATTNSSKKQAVTQNGGTLEKQYRLINAAQV&#xA;&amp;gt;WP_048353666.1-Bacillus_glycinifermentans&#xA;MKKRSLWLSVLTALLLVFTMAFSNPASAEQPAKDVEKDYIVGFKSSVKTASVKKDVIKESGGKVDKQFKIINAAKATLDQ&#xA;&amp;gt;WP_206702858.1-Bacillus_sonorensis&#xA;MTALLLVLSTAFSSPASAAQPAKDVEKDYIVGFKSSVKTAAVKKDVIKENGGKVDKQFKIINAAKATLDQEEVKALKKD&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The list of IDs in a separate text file that need to be removed from the above fasta file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;WP_017417394.1-Bacillus_amyloliquefaciens_gp.&#xA;&amp;gt;WP_003327717.1-Bacillus_atrophaeus&#xA;&amp;gt;KAF1680298.1-Bacillus_sp.&#xA;&amp;gt;WP_048353666.1-Bacillus_glycinifermentans&#xA;&amp;gt;WP_206702858.1-Bacillus_sonorensis&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I searched and tried a few scripts shared elsewhere (e.g. from &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/15326/how-to-remove-sequences-from-a-fasta-file-using-a-sequence-id-list-which-contain&quot;&gt;here&lt;/a&gt;), but I couldn't make it work with my files. So all the help will be really appreciated. Thanks in advance.&lt;/p&gt;&#xA;&lt;p&gt;PS. I am using using python usually with its own IDLE on windows 10 PC.&lt;/p&gt;&#xA;&lt;p&gt;I did one thing with &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/15393/filtering-sequences-entries-by-headers-id-from-a-fasta-file-database&quot;&gt;this code&lt;/a&gt;. I removed the '&amp;gt;' character from the headers file and then this code works but it keeps those headers and their respective sequences. I think it works as intended in this code, but I want to remove those from the sequence file not to keep.&lt;/p&gt;&#xA;&lt;p&gt;Update: as asked below, I copied this code from another post.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;with open('headers.txt', 'r') as fin:&#xA;    to_keep = set([])&#xA;    for line in fin:&#xA;        to_keep.add(line.strip().split(' ')[0])&#xA;with open('output.fa', 'w') as fout:&#xA;    for record in SeqIO.parse('sample.fa','fasta'):&#xA;        if record.id in to_keep:&#xA;            SeqIO.write(record, fout, 'fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18012" LastEditorUserId="73" LastEditDate="2023-12-21T09:12:43.990" LastActivityDate="2023-12-21T09:16:21.220" Title="Remove sequences from a fasta file with IDs from a text file using Python" Tags="|python|fasta|sequence|" AnswerCount="4" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="21973" PostTypeId="2" ParentId="21972" CreationDate="2023-12-20T10:05:33.050" Score="2" Body="&lt;p&gt;This should work&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import argparse&#xA;&#xA;ap = argparse.ArgumentParser(description=__doc__, formatter_class=argparse.ArgumentDefaultsHelpFormatter)&#xA;ap.add_argument('--fasta', required=True, type=str, help='Fasta file to extract from')&#xA;ap.add_argument('--headers', required=True, type=str, help='Headers to remove, one per line.')&#xA;&#xA;conf = ap.parse_args()&#xA;f = conf.fasta&#xA;h = conf.headers&#xA;header_set = set()&#xA;&#xA;with open(h) as header_file:&#xA;    for line in header_file:&#xA;        line = line.rstrip()&#xA;        header_set.add(line)&#xA;&#xA;with open(f) as fasta_file:&#xA;    remove_sequence = False&#xA;    for line in fasta_file:&#xA;        if '&amp;gt;' in line:&#xA;            header_id = line.split()[0].replace(&amp;quot;&amp;gt;&amp;quot;, &amp;quot;&amp;quot;)&#xA;            if header_id in header_set:&#xA;                remove_sequence = True&#xA;            else:&#xA;                remove_sequence = False&#xA;                print(line.rstrip())&#xA;        elif not remove_sequence:&#xA;            print(line.rstrip())&#xA;&#xA;exit()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Save it as python script and then run&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;python remove_fasta.py --fasta sample.fa --headers listofheaders.txt &amp;gt; cleaned.fasta&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="15700" LastEditorUserId="15700" LastEditDate="2023-12-20T10:12:09.957" LastActivityDate="2023-12-20T10:12:09.957" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21974" PostTypeId="2" ParentId="21972" CreationDate="2023-12-20T13:53:21.690" Score="2" Body="&lt;p&gt;I would argue writing a whole python script for this is overkill. Assuming you are using a*nix based operating system like Linux or macOS, you can use the two scripts &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/2651/298&quot;&gt;I have posted here&lt;/a&gt;, &lt;code&gt;FastaToTbl&lt;/code&gt; and &lt;code&gt;TblToFasta&lt;/code&gt;, and just &lt;code&gt;grep&lt;/code&gt; out the ones you do not want:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;FastaToTbl sample.fa | grep -v -xFf listofheaders.txt | TblToFasta &amp;gt; cleaned.fa&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2023-12-20T13:53:21.690" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21975" PostTypeId="2" ParentId="21971" CreationDate="2023-12-20T15:43:47.147" Score="0" Body="&lt;p&gt;Potential workaround: use MAST.&lt;/p&gt;&#xA;&lt;p&gt;As far as I understand, 'under the hood', so to speak, FIMO and MAST work pretty much the same. The difference might be just that FIMO focuses on scoring motifs, while MAST is scoring sequences (&lt;a href=&quot;https://groups.google.com/g/meme-suite/c/qMSj6AdFxoM&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://groups.google.com/g/meme-suite/c/qMSj6AdFxoM&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;Example of FIMO output: &lt;a href=&quot;https://web.mit.edu/meme_v4.11.4/share/doc/examples/fimo_example_output_files/fimo.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://web.mit.edu/meme_v4.11.4/share/doc/examples/fimo_example_output_files/fimo.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Example of MAST output: &lt;a href=&quot;https://web.mit.edu/meme_v4.11.4/share/doc/examples/mast_example_output_files/mast.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://web.mit.edu/meme_v4.11.4/share/doc/examples/mast_example_output_files/mast.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There might be small differences in the output of the tools, but then it's up to you to see whether this is relevant for your data.&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2023-12-20T15:43:47.147" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21976" PostTypeId="1" CreationDate="2023-12-20T18:46:51.210" Score="1" ViewCount="42" Body="&lt;p&gt;I am currently trying to download all the requirements to make a COInr database, following this workflow; &lt;a href=&quot;https://mkcoinr.readthedocs.io/en/latest/content/overview.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://mkcoinr.readthedocs.io/en/latest/content/overview.html&lt;/a&gt;.&#xA;It requires a couple different third party programs to be able to run, one of them being nsdpy, which it says all it requires is cloning the repo from Github and then you're supposed to simply be able to use 'pip' to install nsdpy.&lt;/p&gt;&#xA;&lt;p&gt;However I continued to receive this error;&#xA;&amp;quot;&amp;quot;DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at &lt;a href=&quot;https://pip.pypa.io/en/latest/development/release-process/#python-2-support&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pip.pypa.io/en/latest/development/release-process/#python-2-support&lt;/a&gt;&#xA;ERROR: Could not find a version that satisfies the requirement nsdpy (from versions: none)&#xA;ERROR: No matching distribution found for nsdpy&amp;quot;&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Which makes it seem that my issue is with the version of our Python (I am using a linux system on a Mac). The version of Python that is says we should be using is 2.7, the current version I have downloaded is 3.7.7. But then I was reading through the nsdpy docs (&lt;a href=&quot;https://nsdpy.readthedocs.io/en/latest/main.html#overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://nsdpy.readthedocs.io/en/latest/main.html#overview&lt;/a&gt;) which says that it requires a Python version of 3.8+. So I am a little confused, which version of Python to I truly need to be able to install nsdpy?&#xA;Is there a more up to date version of nsdpy?&lt;/p&gt;&#xA;&lt;p&gt;If anybody else has had this problem and/or has a solution, please let me know!&lt;/p&gt;&#xA;" OwnerDisplayName="Grace Sheehan" LastEditorUserId="2108" LastEditDate="2023-12-21T00:43:03.103" LastActivityDate="2023-12-21T00:43:03.103" Title="What versions of pip and python are appropriate for nsdpy?" Tags="|python|software-installation|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21977" PostTypeId="2" ParentId="21972" CreationDate="2023-12-20T19:23:48.367" Score="5" Body="&lt;p&gt;You can use something like seqkit: &lt;a href=&quot;https://bioinf.shenwei.me/seqkit/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bioinf.shenwei.me/seqkit/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;seqkit has a &lt;code&gt;grep&lt;/code&gt; sub-command that is one of the most flexible I've ever seen. Your use case is quite simple, you can use &lt;code&gt;seqkit grep -v&lt;/code&gt; like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;seqkit grep -v -n -f list_of_IDs_to_exclude.txt in.fasta &amp;gt; out.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastEditorUserId="73" LastEditDate="2023-12-21T09:16:21.220" LastActivityDate="2023-12-21T09:16:21.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21978" PostTypeId="1" CreationDate="2023-12-20T22:08:30.217" Score="1" ViewCount="35" Body="&lt;p&gt;I've been struggling with constructing a PRS score. In the pics you can see how my summary stats and my target data look like.&lt;/p&gt;&#xA;&lt;p&gt;Target Data Summary Stats&lt;/p&gt;&#xA;&lt;p&gt;I've tried the following code which should be pretty straightforward:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;plink \&#xA;  --bfile target \&#xA;  --score summary_stats.txt 1 4 6 header \&#xA;  --out final&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I run it, I get this message:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;10499994 variants and 1198 people pass filters and QC. Note: No phenotypes present.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The data is binary and the fam file really doesn't have phenotypes (it's -9). Is there any way to still run a PRS?&lt;/p&gt;&#xA;&lt;p&gt;I'm unsure what I'm doing wrong and would really appreciate some insight! Also, the data has been harmonized already so it should technically work!&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WIFMj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WIFMj.png&quot; alt=&quot;Summary Statistics&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/hZrhp.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/hZrhp.png&quot; alt=&quot;My data&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18735" LastEditorUserId="650" LastEditDate="2023-12-29T21:50:22.080" LastActivityDate="2024-03-30T18:07:50.947" Title="Problem with Calculating Polygenic Risk Score (PRS) in Plink" Tags="|plink|polygenic-risk-score|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21979" PostTypeId="2" ParentId="21972" CreationDate="2023-12-21T05:41:14.237" Score="2" Body="&lt;p&gt;After modifying the code above in the original post, this one works the best for me. I got it working after putting 'not' in the second last line instead of &amp;quot;if record.id in to keep&amp;quot; to &amp;quot;if record.id not in to keep&amp;quot;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from Bio import SeqIO&#xA;with open('headers.txt', 'r') as fin:&#xA;    to_remove = set([])&#xA;    for line in fin:&#xA;        to_remove.add(line.strip().split(' ')[0])&#xA;with open('output.fa', 'w') as fout:&#xA;    for record in SeqIO.parse('sample.fa','fasta'):&#xA;        if record.id in to_remove:&#xA;            pass&#xA;        else:&#xA;            SeqIO.write(record, fout, 'fasta')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18012" LastEditorUserId="73" LastEditDate="2023-12-21T09:15:31.863" LastActivityDate="2023-12-21T09:15:31.863" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21980" PostTypeId="1" CreationDate="2023-12-21T07:43:20.753" Score="0" ViewCount="25" Body="&lt;p&gt;I have a background in Computer Science and have recently gotten very interested in the field of Protein Engineering. I am trying to build a web app which displays the 3D structure of a protein when I upload a PDB file. I also want to annotate subsequences with unique labels.&lt;/p&gt;&#xA;&lt;p&gt;As I am very new to the field I am confused about which library to use. Can I use BioPython for everything? Or should I use any other library for 3D structure visualization like 3Dmol.js, NGL Viewer etc?&lt;/p&gt;&#xA;&lt;p&gt;Any comments/suggestions/feedback would be really helpful guys as I am a beginner in the field of BioML.&lt;/p&gt;&#xA;" OwnerUserId="18741" LastActivityDate="2023-12-21T07:43:20.753" Title="Best Library for 3D Protein Structure Visualization and Annotation" Tags="|proteins|protein-structure|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21981" PostTypeId="1" AcceptedAnswerId="21983" CreationDate="2023-12-21T08:09:36.577" Score="3" ViewCount="77" Body="&lt;p&gt;We recently got some shotgun sequencing results from a soil bacterium. And we have obtained some contigs which have some genes of interest.&lt;/p&gt;&#xA;&lt;p&gt;Is there a way to automatically annotate the whole sequence (one single contig of around 25 kb size) and mark the genes on this stretch of DNA? I have never done this before so any help would be appreciated. The ultimate goal would be to know what genes are on this stretch of DNA and see if they work together in a pathway. But the first problem is:&lt;/p&gt;&#xA;&lt;p&gt;How to annotate a stretch of DNA with many genes on it that would code for a protein/enzyme? so that I get a map of genes on this stretch of DNA.&#xA;Thanks.&lt;/p&gt;&#xA;&lt;p&gt;Edit:&lt;/p&gt;&#xA;&lt;p&gt;This is a soil isolate and we don't have a full genome sequence. Presumably, it is a Rhodococcus species, which might be similar. From the one gene we know, there are no exact homologs in the database so most probably it is a new strain. The strain works symbiotically with another strain which we are also working on and later I want to predict what proteins maybe involved in the catalysis of a specific compound. It is not a one step process so possibly multiple enzymes/proteins are involved.&#xA;I tried UGENE and UGENE can identify multiple ORFs in this particular stretch, but there are hundreds of ORFS (on both strands) and I find it tedious to blast on NCBI one by one.&lt;/p&gt;&#xA;&lt;p&gt;I didn't want too many questions in one post, so at the moment just focus on the identification of all the putative genes on this one contig would be good enough to start.&lt;/p&gt;&#xA;&lt;p&gt;Thanks again.&lt;/p&gt;&#xA;" OwnerUserId="18012" LastEditorUserId="18012" LastEditDate="2023-12-22T01:09:02.553" LastActivityDate="2023-12-22T01:09:02.553" Title="How to annotate a bacterial genome automatically?" Tags="|genome|sequence-annotation|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21982" PostTypeId="2" ParentId="21968" CreationDate="2023-12-21T10:38:25.160" Score="1" Body="&lt;p&gt;The main issue I can see is the filenames, and should be easily solvable.&lt;/p&gt;&#xA;&lt;p&gt;Simply put, LongStitch is &lt;em&gt;very&lt;/em&gt; particular about filenames. It expects the input draft to have the extension .fa, and the input reads to have any from a range of extensions (your .fastq is perfectly fine, though).&lt;/p&gt;&#xA;&lt;p&gt;The main issue is that LongStitch expects the extensions to be &lt;em&gt;missing&lt;/em&gt; from the command. So if your draft is draft is called &lt;code&gt;consensus.fa&lt;/code&gt; and your reads are  &lt;code&gt;combined.fastq&lt;/code&gt;, then the command would include something like &lt;code&gt;draft=consensus reads=combined&lt;/code&gt;, not &lt;code&gt;draft=consensus.fa reads=combined.fastq&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;(unsure if it accepts input from different directories, or whether you'll need to move your input to the same directory as well)&lt;/p&gt;&#xA;&lt;p&gt;More generally, when having issues with a particular tool which is available via github, I would encourage you to have a look at the 'Issues' or 'Help' section. What I normally do is to delete the 'is:issue is:open' terms in the search bar, which allows me to search for my bug among older, potentially solved issues (e.g. &lt;a href=&quot;https://github.com/bcgsc/LongStitch/issues/54&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/bcgsc/LongStitch/issues/54&lt;/a&gt;), and also generally to check that the software is still supported.&lt;/p&gt;&#xA;&lt;p&gt;Good luck!&lt;/p&gt;&#xA;&lt;p&gt;(*that said, M_'s suggestion of using mamba is generally a good one, and has helped me with another LongStitch bug)&lt;/p&gt;&#xA;" OwnerUserId="4401" LastActivityDate="2023-12-21T10:38:25.160" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21983" PostTypeId="2" ParentId="21981" CreationDate="2023-12-21T13:40:29.890" Score="5" Body="&lt;p&gt;There are a number of annotation pipelines that you could use to get fairly standardized annotation - &lt;a href=&quot;https://github.com/tseemann/prokka&quot; rel=&quot;noreferrer&quot;&gt;Prokka&lt;/a&gt; has been around for a while (paper published in 2014) but is still used frequently. There's also &lt;a href=&quot;https://github.com/oschwengers/bakta&quot; rel=&quot;noreferrer&quot;&gt;Bakta&lt;/a&gt;, which may be better at catching short ORFs and providing more detailed functional annotation, but requires a fairly large database.&lt;/p&gt;&#xA;" OwnerUserId="12755" LastActivityDate="2023-12-21T13:40:29.890" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21984" PostTypeId="2" ParentId="21965" CreationDate="2023-12-21T16:41:37.647" Score="1" Body="&lt;p&gt;Based on your comments, you wish to obtain a semi-colon separated list of unique values in the gene column. You can create a new column containing these using:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;xyz &amp;lt;- data.frame(chr=&amp;quot;chr1&amp;quot;, gene = c(&amp;quot;RUNX3;RUNX3&amp;quot;, &amp;quot;RUNX3;RUNX3&amp;quot;, &amp;quot;SFN&amp;quot;, &amp;quot;NR0B2&amp;quot;, &amp;quot;NR0B2&amp;quot;, &amp;quot;TFAP2E&amp;quot;, &amp;quot;A;B;A&amp;quot;))&#xA;&#xA;xyz&#xA;   chr        gene&#xA;1 chr1 RUNX3;RUNX3&#xA;2 chr1 RUNX3;RUNX3&#xA;3 chr1         SFN&#xA;4 chr1       NR0B2&#xA;5 chr1       NR0B2&#xA;6 chr1      TFAP2E&#xA;7 chr1       A;B;A&#xA;&#xA;xyz&lt;span class=&quot;math-container&quot;&gt;$gene_uniq &amp;lt;- sapply(xyz$&lt;/span&gt;gene, function(x) paste(unique(unlist(strsplit(x, &amp;quot;;&amp;quot;))), collapse=&amp;quot;;&amp;quot;))&#xA;&#xA;xyz&#xA;   chr        gene gene_uniq&#xA;1 chr1 RUNX3;RUNX3     RUNX3&#xA;2 chr1 RUNX3;RUNX3     RUNX3&#xA;3 chr1         SFN       SFN&#xA;4 chr1       NR0B2     NR0B2&#xA;5 chr1       NR0B2     NR0B2&#xA;6 chr1      TFAP2E    TFAP2E&#xA;7 chr1       A;B;A       A;B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Or you can use dplyr. I'm adding a column called &lt;code&gt;entry_num&lt;/code&gt; below since my example has non-unique combinations of chr and gene that we need to retain.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(dplyr)&#xA;library(tidyr)&#xA;&#xA;xyz &amp;lt;- data.frame(chr=&amp;quot;chr1&amp;quot;, gene = c(&amp;quot;RUNX3;RUNX3&amp;quot;, &amp;quot;RUNX3;RUNX3&amp;quot;, &amp;quot;SFN&amp;quot;, &amp;quot;NR0B2&amp;quot;, &amp;quot;NR0B2&amp;quot;, &amp;quot;TFAP2E&amp;quot;, &amp;quot;A;B;A&amp;quot;))&#xA;xyz %&amp;gt;% mutate(entry_num = 1:nrow(xyz)) %&amp;gt;% separate_rows(gene, sep = &amp;quot;;&amp;quot;) %&amp;gt;% unique() %&amp;gt;% group_by(chr, entry_num) %&amp;gt;% summarize(gene = paste(gene, collapse=&amp;quot;;&amp;quot;)) %&amp;gt;% select(-entry_num) %&amp;gt;% as.data.frame()&#xA;&#xA;   chr   gene&#xA;1 chr1  RUNX3&#xA;2 chr1  RUNX3&#xA;3 chr1    SFN&#xA;4 chr1  NR0B2&#xA;5 chr1  NR0B2&#xA;6 chr1 TFAP2E&#xA;7 chr1    A;B&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastEditorUserId="650" LastEditDate="2023-12-21T16:46:40.133" LastActivityDate="2023-12-21T16:46:40.133" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21985" PostTypeId="1" CreationDate="2023-12-21T21:42:05.757" Score="1" ViewCount="43" Body="&lt;p&gt;I use the next versions:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gatk   4.4.0.0&#xA;minimap2   2.26-r1175&#xA;samtools   1.19&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have a fastq file. And try to implement gatk MergeBamAlignment to generate a new bam-file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar gatk.jar FastqToSam --FASTQ .fastq --OUTPUT u.bam&#xA;--SAMPLE_NAME sample1 -RG readgroup -PL ILLUMINA&#xA;minimap2 -a hg38.fa .fastq -o A.sam&#xA;java -jar gatk.jar SortSam -I A.sam -O A.sort.bam -SO queryname&#xA;java -jar gatk.jar SortSam -I u.bam -O u.sort.bam -SO queryname&#xA;java -jar gatk.jar MergeBamAlignment -O m.bam -R hg38.fa -UNMAPPED_BAM u.bam -ALIGNED_BAM A.sort.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But I receive error on the output of MergeBamAlignment:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;INFO    2023-12-21 19:20:41     SamAlignmentMerger      Processing SAM file(s): [./myprogs/data/NR.Aligned.sort.bam] &#xA;&#xA;[Thu Dec 21 19:20:42 2023] picard.sam.MergeBamAlignment done. &#xA;&#xA;Elapsed time: 0.02 minutes.&#xA;&#xA;Runtime.totalMemory()=486539264 &#xA;&#xA;To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp&#xA;&#xA;java.lang.ArrayIndexOutOfBoundsException: Index 86 out of bounds for length 86&#xA;at htsjdk.samtools.util.SequenceUtil.calculateMdAndNmTags(SequenceUtil.java:990)&#xA;at picard.sam.AbstractAlignmentMerger.fixNmMdAndUq(AbstractAlignmentMerger.java:611)&#xA;at picard.sam.AbstractAlignmentMerger.mergeAlignment(AbstractAlignmentMerger.java:586)&#xA;at picard.sam.SamAlignmentMerger.mergeAlignment(SamAlignmentMerger.java:181)&#xA;at picard.sam.MergeBamAlignment.doWork(MergeBamAlignment.java:368)&#xA;at picard.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:289)&#xA;at org.broadinstitute.hellbender.cmdline.PicardCommandLineProgramExecutor.instanceMain(PicardCommandLineProgramExecutor.java:37)&#xA;at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:160)&#xA;at org.broadinstitute.hellbender.Main.mainEntry(Main.java:203)&#xA;at org.broadinstitute.hellbender.Main.main(Main.java:289)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have diagnosed u.sort.bam and A.sam.bam with ValidateSamFile:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;java -jar gatk.jar ValidateSamFile -I u.sort.bam -M SUMMARY -R hg38.fa&#xA;No errors found&#xA;[Thu Dec 21 00:32:34 2023] picard.sam.ValidateSamFile done. Elapsed time: 0.00 minutes. &#xA;Runtime.totalMemory()=234881024 &#xA;&#xA;Tool returned: &#xA;0 &#xA;&#xA;java -jar gatk.jar ValidateSamFile -I A.sort.bam -M SUMMARY -R hg38.fa&#xA;ERROR    2023-12-21 01:17:28    ValidateSamFile    Requesting earlier reference sequence: 23 &amp;lt; 49 &#xA;[Thu Dec 21 01:17:28 2023] picard.sam.ValidateSamFile done. Elapsed time: 0.18 minutes.&#xA;Runtime.totalMemory()=2113929216&#xA;To get help, see http://broadinstitute.github.io/picard/index.html#GettingHelp  &#xA;&#xA;Tool returned:&#xA;-1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have done the same actions only with &lt;code&gt;--MD&lt;/code&gt; in minimap2. But the same errors occur. I would like to find out how to fix this problem. I use the latest versions of the programs. I have downloaded hg38.fa each one time, but it didn't change the situation. I acts in according with gatk's article 'Data pre-processing for variant discovery' : &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360035535912-Data-pre-processing-for-variant-discovery&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360035535912-Data-pre-processing-for-variant-discovery&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="16563" LastEditorUserId="650" LastEditDate="2023-12-22T22:30:06.397" LastActivityDate="2023-12-22T22:30:06.397" Title="java.lang.ArrayIndexOutOfBoundsException: Index 86 out of bounds for length 86" Tags="|sequence-alignment|gatk|minimap2|indexing|java|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="21986" PostTypeId="1" CreationDate="2023-12-22T07:51:08.977" Score="1" ViewCount="34" Body="&lt;p&gt;This is a BAM file from a 30x coverage Nebula Genomics test opened with IGV.&lt;/p&gt;&#xA;&lt;p&gt;I found an area (1:30169255-30169600) where the coverage is less than 15x. Can I interpret it as a part of one strand of Chromosome 1 is missing?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/glhOQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/glhOQ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3awb4.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3awb4.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18744" LastEditorUserId="18744" LastEditDate="2023-12-22T07:51:32.897" LastActivityDate="2023-12-22T19:17:48.797" Title="Can I interpret 15x coverage area as a part of one strand is missing?" Tags="|wgs|indel|igv|chromosomes|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21987" PostTypeId="1" AcceptedAnswerId="21988" CreationDate="2023-12-22T13:03:39.213" Score="2" ViewCount="76" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9583255/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have this script&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;# Defining the number of threads to be used&#xA;threads=20&#xA;&#xA;# Alignment of sequences to the reference genome using 'HISAT2'&#xA;echo -e &amp;quot;\nNow Running Classical Alignment-based with HISAT2...\n&amp;quot;&#xA;&#xA;mkdir -p hisat2&#xA;&#xA;# Download the human reference genome&#xA;wget -c ftp://ftp.ensembl.org/pub/release-100/fasta/homo_sapiens/dna/Homo_sapiens.GRCh38.dna.primary_assembly.fa.gz -O hisat2/Homo_sapiens.GRCh38.dna.primary_assembly.fa.gz&#xA;gunzip hisat2/Homo_sapiens.GRCh38.dna.primary_assembly.fa.gz&#xA;&#xA;# Download the human annotation file&#xA;wget -c ftp://ftp.ensembl.org/pub/release-100/gtf/homo_sapiens/Homo_sapiens.GRCh38.100.gtf.gz -O hisat2/Homo_sapiens.GRCh38.100.gtf.gz&#xA;gunzip hisat2/Homo_sapiens.GRCh38.100.gtf.gz&#xA;&#xA;# HISAT2 indexing to build the reference genome index&#xA;hisat2-build -p $threads hisat2/Homo_sapiens.GRCh38.dna.primary_assembly.fa hisat2/Homo_sapiens.GRCh38v3_hisat2.idx&#xA;&#xA;# HISAT2 Alignment&#xA;for sample in &lt;span class=&quot;math-container&quot;&gt;$(cat sample_ids.txt)&#xA;do&#xA;    hisat2 -p $&lt;/span&gt;threads -x hisat2/Homo_sapiens.GRCh38v3_hisat2.idx \&#xA;        -1 trimmed_data/&lt;span class=&quot;math-container&quot;&gt;${sample}_1_val_1.fq -2 trimmed_data/$&lt;/span&gt;{sample}_2_val_2.fq \&#xA;        -S hisat2/${sample}_hisat2.sam&#xA;&#xA;    # Compress the SAM files to binary format, sort them, and index them.&#xA;    samtools view -Sb hisat2/&lt;span class=&quot;math-container&quot;&gt;${sample}_hisat2.sam | samtools sort &amp;gt; hisat2/$&lt;/span&gt;{sample}_hisat2_sorted.bam&#xA;    samtools index hisat2/&lt;span class=&quot;math-container&quot;&gt;${sample}_hisat2_sorted.bam&#xA;rm hisat2/$&lt;/span&gt;{sample}_hisat2.sam  # Remove the .sam files for storage purposes&#xA;done&#xA;&#xA;# Quantifying Aligned reads using the Subread package's script 'featureCounts'&#xA;for sample in &lt;span class=&quot;math-container&quot;&gt;$(cat sample_ids.txt)&#xA;do&#xA;featureCounts -T $&lt;/span&gt;threads -a hisat2/Homo_sapiens.GRCh38.100.gtf \&#xA;        -o hisat2/&lt;span class=&quot;math-container&quot;&gt;${sample}_hisat2_counts.txt \&#xA;    hisat2/$&lt;/span&gt;{sample}_hisat2_sorted.bam&#xA;done&#xA;&#xA;# Generating analysis report with MultiQC&#xA;echo &amp;quot;Running MultiQC on HISAT2 alignment results .........&amp;quot;&#xA;multiqc -o multiQCreports hisat2&#xA;echo &amp;quot;MultiQC on HISAT2 alignment results completed successfully.&amp;quot;&#xA;&#xA;# Display message to show that the script is complete&#xA;echo &amp;quot;Script completed!&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, I have receieved an error below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Load annotation file Homo_sapiens.GRCh38.100.gtf ...                       ||&#xA;||    Features : 1378275                                                      ||&#xA;||    Meta-features : 60683                                                   ||&#xA;||    Chromosomes/contigs : 47                                                ||&#xA;||                                                                            ||&#xA;|| Process BAM file SRR19170514_hisat2_sorted.bam...                          ||&#xA;ERROR: Paired-end reads were detected in single-end read library : hisat2/SRR19170514_hisat2_sorted.bam&#xA;ERROR: Paired-end reads were detected in single-end read library : hisat2/SRR19170514_hisat2_sorted.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Question: What can I do to solve this?&lt;/p&gt;&#xA;" OwnerUserId="18661" LastEditorUserId="73" LastEditDate="2023-12-23T09:11:54.383" LastActivityDate="2023-12-23T09:11:54.383" Title="Classical alignment with HISAT2" Tags="|rna-seq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21988" PostTypeId="2" ParentId="21987" CreationDate="2023-12-22T13:28:26.453" Score="6" Body="&lt;p&gt;I believe the error is from &lt;code&gt;featureCounts&lt;/code&gt; and the solution would be to add the &lt;code&gt;-p&lt;/code&gt; flag to the &lt;code&gt;featureCounts&lt;/code&gt; call, stating that you are providing paired-end data.&lt;/p&gt;&#xA;&lt;p&gt;EDIT: You might want to check this &lt;a href=&quot;https://www.reddit.com/r/bioinformatics/comments/13m89bd/help_with_featurecounts_error_pairedend_reads/&quot; rel=&quot;noreferrer&quot;&gt;&lt;code&gt;r/bioinformatics&lt;/code&gt; (reddit) thread&lt;/a&gt; out.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2023-12-22T13:28:26.453" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="21989" PostTypeId="2" ParentId="21986" CreationDate="2023-12-22T19:07:08.987" Score="1" Body="&lt;h2&gt;UPDATE:&lt;/h2&gt;&#xA;&lt;p&gt;Seeing a different question where similar vocabulary is used: it would not be one &lt;em&gt;strand&lt;/em&gt; that would be lost. single stranded DNA is not very happy in the nucleus, and would either lead to something seriously bad or be gap-filled to make it dsDNA again.&lt;/p&gt;&#xA;&lt;p&gt;It would instead be that one of two chromosomes has a deletion at that position, which would be a classic CNV.&lt;/p&gt;&#xA;&lt;h2&gt;ORIGINAL ANSWER:&lt;/h2&gt;&#xA;&lt;p&gt;No.&lt;/p&gt;&#xA;&lt;p&gt;There are a variety of copy number variant (CNV) calling tools that put a lot of effort into statistically robust ways of asking this question, all of which are a lot more complex than just looking at raw coverage. &lt;a href=&quot;https://www.nature.com/articles/s41598-021-93878-2&quot; rel=&quot;nofollow noreferrer&quot;&gt;I'd recommend looking more into them&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In terms of explanations for lower coverage in some regions that aren't CNVs: sequenceability of DNA is not even, replication of DNA is not even, mappability of reads is not even, etc. You need to reject those hypotheses before you go to variants.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2023-12-22T19:17:48.797" LastActivityDate="2023-12-22T19:17:48.797" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21990" PostTypeId="1" CreationDate="2023-12-22T23:38:33.557" Score="-1" ViewCount="92" Body="&lt;p&gt;I am trying to adapt combinatorial index reads with some modifications for long read sequencing; it's a custom approach, not a commercial kit.&lt;/p&gt;&#xA;&lt;p&gt;I have a question about using Knee Plot for identifying Barcode white list for non-10X data. While generating the Knee Plot barcode which barcode frequencies should be used: without collapsing PCR replicates to after collapsing PCR replicates?&lt;/p&gt;&#xA;&lt;p&gt;I have used UMI-Tools (&lt;a href=&quot;https://umi-tools.readthedocs.io/en/latest/QUICK_START.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://umi-tools.readthedocs.io/en/latest/QUICK_START.html&lt;/a&gt;) and Pacbio groupdedup based approach. For the knee plot generation I used both collapsed and non-collapsed reads. Since its long reads and coverage is not great; bumps are not the perfect one but both inputs generate it. I am not sure which input I should consider to remove the backgrounds.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT:&lt;/strong&gt; &lt;em&gt;Can the down-voters please leave a note why they think this not a good query to ask?&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="4951" LastEditorUserId="73" LastEditDate="2024-01-02T21:11:02.907" LastActivityDate="2024-01-02T21:11:02.907" Title="Single Cell RNASeq Knee Plot - Collapsed vs Non-collapsed Reads" Tags="|scrnaseq|barcode|umi|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21991" PostTypeId="2" ParentId="21990" CreationDate="2023-12-23T09:10:19.370" Score="0" Body="&lt;p&gt;If you have UMIs, then cell/UMI/mapped transcript triplets should be collapsed into unique triplets.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2023-12-23T09:10:19.370" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="21992" PostTypeId="2" ParentId="21971" CreationDate="2023-12-23T21:13:23.240" Score="0" Body="&lt;p&gt;If the sequences you are scanning with FIMO are supported by the &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgGateway&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC &lt;/a&gt; or Ensembl genome browsers you can upload the GFF file generated by FIMO as a custom track: &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgCustom&quot; rel=&quot;nofollow noreferrer&quot;&gt;UCSC Custom Track&lt;/a&gt;, &lt;a href=&quot;https://useast.ensembl.org/Homo_sapiens/Info/Index&quot; rel=&quot;nofollow noreferrer&quot;&gt;ENSEMBL Custom Track&lt;/a&gt;. Note that for the Ensembl website the FIMO GFF file is in GFF3 format.&lt;/p&gt;&#xA;" OwnerUserId="313" LastActivityDate="2023-12-23T21:13:23.240" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21994" PostTypeId="2" ParentId="17916" CreationDate="2023-12-24T23:06:27.157" Score="0" Body="&lt;p&gt;&lt;code&gt;conda install -c conda-forge openbabel&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from openbabel import pybel as pb&#xA;babel_mol = pb.readfile(format=&amp;quot;pdb&amp;quot;, filename=path_to_pdb).__next__()&#xA;babel_mol.addh()&#xA;babel_mol.write(format='pdb', filename=path_to_babel_pdbh, overwrite=True)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18751" LastActivityDate="2023-12-24T23:06:27.157" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21995" PostTypeId="2" ParentId="11512" CreationDate="2023-12-27T22:39:05.647" Score="1" Body="&lt;p&gt;I would suggest plugging your reads into a fast k-mer matching tool like &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-019-1841-x&quot; rel=&quot;nofollow noreferrer&quot;&gt;mash screen&lt;/a&gt; to get a basic idea of where your reads come from, if you suspect contamination.&lt;/p&gt;&#xA;&lt;p&gt;An issue with just looking at GC content is that some organelles/plasmids/etc. such as mitochondria have quite divergent GC content, and the different populations you observe might just indicate random fluctuations of copy number of these DNAs due to different sample preps. Not sure that would explain your plot but it does give me pause.&lt;/p&gt;&#xA;&lt;p&gt;As suggested above, assembly is obviously a good idea if you are ok with the compute cost, and then you can use something like &lt;a href=&quot;https://github.com/ablab/spades&quot; rel=&quot;nofollow noreferrer&quot;&gt;SPAdes&lt;/a&gt; or blobtools to pull out contigs from different origins.&lt;/p&gt;&#xA;&lt;p&gt;KAT and similar tools as suggested are handy for hypothesis free evaluation of different populations in your reads, but they don't directly address the issue of contamination.&lt;/p&gt;&#xA;&lt;p&gt;And obviously measure the quality of the reads to establish that they are ok and nothing weird happened there, they aren't all Gs or something.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-12-27T22:39:05.647" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21997" PostTypeId="2" ParentId="18127" CreationDate="2023-12-30T05:33:31.300" Score="0" Body="&lt;p&gt;For PacBio HiFi reads (or other techs of comparable quality), it is not recommended to polish with Illumina at all. You run the risk of collapsing true heterozygosity/variation in your underlying genome and yielding a haploid assembly that does not represent either haplotype.&lt;/p&gt;&#xA;&lt;p&gt;If that underlying variation is an issue because you need a haploid assembly, you are free to use a deduplication technique such as purge_haplotigs or similar (&lt;a href=&quot;https://bmcbioinformatics.biomedcentral.com/articles/10.1186/s12859-021-04118-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;reviewed here&lt;/a&gt;) to obtain a primary set of contigs/scaffolds. Assigning p and q haplotypes is something that you can do with a tool like &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwiLufiHt7aDAxUwLTQIHYn0DAIQFnoECBAQAQ&amp;amp;url=https%3A%2F%2Fgithub.com%2Fphasegenomics%2FFALCON-Phase&amp;amp;usg=AOvVaw2ttek6UsNItYU1dATDadlW&amp;amp;opi=89978449&quot; rel=&quot;nofollow noreferrer&quot;&gt;falcon-phase&lt;/a&gt; (if you have Hi-C) or a similar technique (I haven't been following the literature for a couple years, possibly there's something a lot better now!).&lt;/p&gt;&#xA;&lt;p&gt;For a recent review of recommended strategies you can read &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9812399/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the T2T consortium benchmarking paper on the topic&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2023-12-30T05:33:31.300" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="21999" PostTypeId="2" ParentId="21959" CreationDate="2023-12-30T15:48:48.763" Score="1" Body="&lt;p&gt;This can be done, but you have to split the BAMs while keeping the full header (with the dictionary) in place for all the shards. This can be done with gatk PrintReads. One thing to remember is that you all need to have one shard with all the &amp;quot;unmapped&amp;quot; reads. Otherwise you will not have the complete information. You can get the unmapped reads by putting &lt;code&gt;unmapped&lt;/code&gt; into the &lt;code&gt;-L&lt;/code&gt; argument.&lt;/p&gt;&#xA;&lt;p&gt;That said, MarkDuplicates cannot be run on shards since it needs both reads from every pair and some of the read pairs might be split. So you'll need to mark-duplicates on the entire 70GB bam (sorry).&lt;/p&gt;&#xA;&lt;p&gt;You could split the BAM into shards but only the &amp;quot;proper pairs&amp;quot; ( filter after GATK with &lt;code&gt;samtools view&lt;/code&gt;, but only on the sam flag so you don't mess with the dictionary) which will make sure that your pairs are on the same contig. Then put all the not proper pairs into their own shard.&lt;/p&gt;&#xA;&lt;p&gt;You can also do the proper/improper filtering with GATK, but it involves read filters which are a bit more complex than samtools flag filters.&lt;/p&gt;&#xA;" OwnerUserId="9836" LastEditorUserId="9836" LastEditDate="2023-12-30T16:00:26.540" LastActivityDate="2023-12-30T16:00:26.540" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22000" PostTypeId="1" CreationDate="2023-12-31T05:42:52.167" Score="1" ViewCount="55" Body="&lt;p&gt;I have some eclipseq data, three IP and three input datasets, and have the alignments for all three. I used Bedtools &amp;quot;Intersect Interval&amp;quot; with the genome gtf file (hg19) I used to align them with, and that creates a bed file with gene names for each peak. I then wrote an R script to sum the clipper score for each gene within a sample, then do a simple student's t test across groups. I now have a gene list and p values for differential expression, but not sure if this is correct.&lt;/p&gt;&#xA;&lt;p&gt;I've tried looking at a lot of sources but cant really find any canonical way to generate a gene list from bed files.&lt;/p&gt;&#xA;&lt;p&gt;I've also tried IDR and it doesn't work on the clipper output, unfortunately.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;&lt;p&gt;Edit:&#xA;eCLIP-sequencing, it's essentially a variant of iCLIP-sequencing and data analysis methods are transferrable, to my knowledge.&lt;/p&gt;&#xA;&lt;p&gt;I have alignments (bam files) made with Star aligner against Hg19. After deduplication and sorting, I called peaks using Clipper (a peak-caller made by the creators of eCLIP). It outputs bed files with scores for each sample.&lt;/p&gt;&#xA;&lt;p&gt;My question is how exactly do I turn this bed file with scores into a gene list? Is my method above correct?&lt;/p&gt;&#xA;" OwnerUserId="18763" LastEditorUserId="18763" LastEditDate="2023-12-31T18:28:18.360" LastActivityDate="2024-02-19T20:03:23.913" Title="Gene list from eclip-seq .bed files" Tags="|peak-calling|fold-change|" AnswerCount="2" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22001" PostTypeId="1" CreationDate="2024-01-01T10:52:51.697" Score="0" ViewCount="18" Body="&lt;p&gt;I am a bioinformatics student doing my research on top-down mass spectrometry-based proteomics and seeking some guidance on acquiring very clean experimental data with known true positives and false positives. I want to assess my proposed FDR estimation method under controlled conditions and would appreciate your insights on obtaining high-quality experimental data.&#xA;My question is&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Any tips on acquiring clean experimental data with known true positives?&lt;/li&gt;&#xA;&lt;li&gt;Any suggestion on sample types or protocols you recommend for ensuring a reliable dataset for FDR estimation?&lt;/li&gt;&#xA;&lt;li&gt;How can one introduce controlled errors or variations, like false positives, while maintaining the overall data quality? If anyone has any idea or experience or any paper suggestion in which they used a similar approach please share thanks.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="18767" LastActivityDate="2024-01-01T10:52:51.697" Title="Seeking Advice on Acquiring Clean Experimental data comprises of proteins for FDR Estimation" Tags="|proteins|literature-search|mass-spectrometry|computational-biochemistry|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22002" PostTypeId="1" AcceptedAnswerId="22003" CreationDate="2024-01-02T03:27:51.387" Score="3" ViewCount="54" Body="&lt;p&gt;How to calculate the probability of finding two proteins that share a 5 amino acid long motif from a proteome of around 1067 proteins that have an average length of 65 residues.&#xA;The probability of a specific sequence that is completely the same is 1067/20^65&lt;/p&gt;&#xA;" OwnerUserId="18771" LastEditorUserId="73" LastEditDate="2024-01-04T21:13:36.307" LastActivityDate="2024-01-04T21:13:36.307" Title="probability of finding a 5 amino acids in a row within a proteome" Tags="|sequence-alignment|proteins|statistics|sequence-analysis|homework|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22003" PostTypeId="2" ParentId="22002" CreationDate="2024-01-02T04:10:50.577" Score="2" Body="&lt;p&gt;Its just significant at 2%&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;0.05&lt;sup&gt;5&lt;/sup&gt; . 65 . 1067 = 0.021 (&lt;em&gt;its actually 0.02 - see note&lt;/em&gt;)&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Where critical probability is 0.05, i.e. 5%&lt;/p&gt;&#xA;&lt;p&gt;There are 20 aa so 1/20 probability of random occurrence. The oligopeptide is 5 amino acids long, hence to the power 5. Obviously there are 1067 proteins and the probability is per protein, thus if these were independent its a straight multiplication because that is how independent events are measured.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; The 65 is really 60 because it's a sliding window moving 1 amino acid at a time and the proteins are only 65 in length. The actual value is 60 rather than 65 because at the 3' terminal of the sequence the sliding window can't capture the remaining 4 amino acids, because once the oligopeptide &amp;lt; 5 the calculation is no longer valid. I used 65 so you could see where the value is originating.&lt;/p&gt;&#xA;&lt;p&gt;Obviously the proteins in question are not independent so you need to state the assumption, i.e. assuming independence.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2024-01-02T04:15:51.257" LastActivityDate="2024-01-02T04:15:51.257" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22004" PostTypeId="2" ParentId="2081" CreationDate="2024-01-02T11:28:56.360" Score="1" Body="&lt;p&gt;You might want to check out this paper:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/pmid/31104630/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Ebbert MTW, &lt;em&gt;et al.&lt;/em&gt;, Systematic analysis of dark and camouflaged genes reveals disease-relevant genes hiding in plain sight. &lt;em&gt;Genome Biol.&lt;/em&gt; 2019 May 20;20(1):97. doi: 10.1186/s13059-019-1707-2. PMID: 31104630; PMCID: PMC6526621.&lt;/a&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;They have collected a list of &amp;quot;dark&amp;quot; (regions that have few reads aligning to them) and &amp;quot;camouflaged&amp;quot; (regions that have ambiguous alignment) regions of the human genome:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Here, we identify regions with few mappable reads that we call dark by depth, and others that have ambiguous alignment, called camouflaged.&lt;/p&gt;&#xA;&lt;p&gt;[. . .]&lt;/p&gt;&#xA;&lt;p&gt;Other dark regions arise, not because the sequencing is inherently problematic, but because of bioinformatic challenges. Specifically, many dark regions arise from duplicated genomic regions, where confidently aligning short reads to a unique location is not possible; we term these regions as “camouflaged”. These camouflaged regions are generally either large contiguous tandem repeats (e.g., centromeres, telomeres, and other short tandem repeats), or a larger specific DNA region that has been duplicated (e.g., a gene duplication) either in tandem or in a more distal genome region. In fact, many genes in the human genome were duplicated over evolutionary time and are still transcriptionally and translationally active (e.g., heat-shock proteins) [3–9], while others have been duplicated, but are considered inactive (i.e., pseudogenes). Regardless of whether the duplication is active, however, any genomic region that has been nearly identically duplicated and is large enough to prevent sequencing reads from aligning unambiguously will be “dark”, because the aligner cannot determine which genomic region the read originated from.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The authors have &lt;a href=&quot;https://github.com/mebbert/Dark_and_Camouflaged_genes&quot; rel=&quot;nofollow noreferrer&quot;&gt;made their scripts available&lt;/a&gt; on github, along with bed files for the relevant regions. The answer to your question, i.e. a specific percentage, is hard to give since it varies enormously with the genome build and sequencing technology, but this paper should have all the details you want. Have a look at Table 1:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/jJN36.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/jJN36.png&quot; alt=&quot;Dark and camouflaged regions vary by genome build. We identified dark and camouflaged regions throughout the genome for three different builds, including GRCh37, GRCh38, and GRCh38+alt, across five different sequencing technologies (or read lengths for Illumina). Specifically, we measured dark regions for Illumina based on 100-nucleotide read lengths, Illumina based on 250-nucleotide read lengths, 10x Genomics, PacBio, and Oxford Nanopore Technologies (ONT). Here, the counts for dark and camouflaged regions are combined. We found that the number of dark regions and nucleotides, both within gene bodies (represented as GB in the table) and outside gene bodies, varies dramatically by build and technology. Overall, each technology has its respective strengths. GRCh38 including alternate contigs has &amp;gt; 3x more dark nucleotides than GRCh37, and more than 2x more dark regions. Results presented throughout the manuscript are based on GRCh38 (in gray)&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2024-01-02T11:28:56.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22005" PostTypeId="1" AcceptedAnswerId="22006" CreationDate="2024-01-02T20:53:18.720" Score="2" ViewCount="44" Body="&lt;p&gt;I have a large tsv file (~15 GB) that is a kmer count matrix and I am trying to calculate jaccard distance of the counts but every time I try to run the following code the process ends before it's done with a &lt;code&gt;killed&lt;/code&gt; error message in the STDOUT:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;jaccard = pd.DataFrame(squareform(pdist(kmer_matrix, 'jaccard')))&#xA;jaccard.to_csv(matrix_path+kmer+&amp;quot;_&amp;quot;+method+&amp;quot;_jacDistance_noFilter.tsv&amp;quot;, sep='\t')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This process works really well with smaller tsv files so I know the code works but I wanted to know if anyone knew what would help run the process quicker/completely. I know the problem must be with my RAM/CPU but is there a way to maybe break up the matrix to calculate everything in batches to make it easier to work with?&lt;/p&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="650" LastEditDate="2024-01-02T21:51:57.640" LastActivityDate="2024-01-02T21:51:57.640" Title="Process gets killed before completing" Tags="|k-mer|errors|distance-matrix|troubleshooting|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22006" PostTypeId="2" ParentId="22005" CreationDate="2024-01-02T21:04:33.840" Score="3" Body="&lt;p&gt;You've overloaded the memory. One route is to deploy &amp;quot;chunking&amp;quot;. pandas will do this. This is complicated to implement code wise however.&lt;/p&gt;&#xA;&lt;p&gt;My suggested solution (because it simpler) would be to do this via &lt;code&gt;pandas2&lt;/code&gt; which uses Apache Arrow via PyArrow. You need to specify which method is being used at the onset of declaring the dataframe. Apache Arrow is for lifting large amounts of data. If this works there's no complicated coding you just &lt;code&gt;import pandas2 as pd&lt;/code&gt; and in the dataframe construction describe the precise heavy lift method (I think there's an additional approach besides PyArrow).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2024-01-02T21:10:01.900" LastActivityDate="2024-01-02T21:10:01.900" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22007" PostTypeId="2" ParentId="21967" CreationDate="2024-01-03T01:04:31.737" Score="0" Body="&lt;p&gt;PacBio lima need both 5'end and 3' end barcodes. You can have a unique barcode in one end and multiple on the other end; but it need two barcodes.&lt;/p&gt;&#xA;" OwnerUserId="4951" LastActivityDate="2024-01-03T01:04:31.737" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22008" PostTypeId="1" CreationDate="2024-01-03T08:56:09.150" Score="0" ViewCount="64" Body="&lt;p&gt;I have my data from NCBI. I am having difficulty making sense of and correlating my fastq file read 1 and read 2 and the fastQC report. As shown below, I have two paired-end read results each displaying 127 bp in length; however, my results on the fastQC report of per base sequence quality show 91 and it does not show the interquartile range (box-whisker plot). What could be the problem?&lt;/p&gt;&#xA;&lt;p&gt;Below is an excerpt display of my fastq file and then the fastQC report:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;@SRR20708369.1 NS500749:581:HGV7VBGXB:1:11101:17225:1048 length=127&#xA;AGCTATCACAGCANGGTACTTCCCAGCGTTACGGGANTGGCCGTACGCCTAACCGCTAACATTACTGCAGGCAACCTACTCATGCACCTAATTGGAAGCGCCACCCTAGCAATATCCACCATTAACC&#xA;+&#xA;A6AAAE6EA/AAA#EAEAEEEAEEEE/EAEEEEA&amp;lt;E#AAAAEEEE6E&amp;lt;E//EE6EAEEAEAEE/AAA6E//E/EE/A&amp;lt;AEEEE/AEEE//&amp;lt;EEA/EE/E/6&amp;lt;&amp;lt;EE&amp;lt;EEE/&amp;lt;EEEEE//&amp;lt;AE/AAEA/&#xA;&#xA;@SRR20708369.2 NS500749:581:HGV7VBGXB:1:11101:22488:1048 length=127&#xA;AGCTATCACGCCANAAGGCTTCCGAGTTAGTATAGGNGCAAAGTGAATACATCTCTGTAACCAGACTCAGATCAAGAAATGATACATTCTGGCCAGGTGTGGTGGCTCATGCCTGTAATCTCAGCAC&#xA;+&#xA;AAAAAE/EAAAAA#EEEEEEEEEEEEEEEEEEEEE6#AA/AEA/EEEEEEEEAEEEEAEA/EAE/AEAEEA6&amp;lt;/6E///EE//6A//AEAEE&amp;lt;E/EE&amp;lt;E&amp;lt;EEEEEEEEEAEAEAAE///EEE/EE6/&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;:&lt;a href=&quot;https://i.stack.imgur.com/ZSgZL.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZSgZL.jpg&quot; alt=&quot;fastq report results of the sample showing 91 bp and no box whisker plot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18775" LastEditorUserId="298" LastEditDate="2024-01-03T12:06:47.357" LastActivityDate="2024-01-04T11:55:02.223" Title="Fastq file with very high per base sequence quality and no box-whisker plot" Tags="|fastq|fastqc|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22010" PostTypeId="2" ParentId="22008" CreationDate="2024-01-03T17:29:50.683" Score="1" Body="&lt;p&gt;Cross-posted and answered on biostars: &lt;a href=&quot;https://www.biostars.org/p/9583849/#9583850&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.biostars.org/p/9583849/#9583850&lt;/a&gt;&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://trace.ncbi.nlm.nih.gov/Traces/?view=run_browser&amp;amp;acc=SRR20708369&amp;amp;display=metadata&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://trace.ncbi.nlm.nih.gov/Traces/?view=run_browser&amp;amp;acc=SRR20708369&amp;amp;display=metadata&lt;/a&gt; is a 10x single-cell sample.&lt;/p&gt;&#xA;&lt;p&gt;So FastQC plot you are showing us appears to be the R2 read from this sample which is 91 bp (split files, where as the example read you show above seems to be Illumina Index + R1 + R2). This is normal. Illumina index (8 bp), Read 1 is 28 bp (Cellbarcodes + UMI) and R2 is RNA (91 bp) = 127 bp.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="650" LastEditorUserId="298" LastEditDate="2024-01-04T11:55:02.223" LastActivityDate="2024-01-04T11:55:02.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22013" PostTypeId="2" ParentId="3931" CreationDate="2024-01-05T01:22:45.260" Score="0" Body="&lt;p&gt;I know you do not want a Python solution but nevertheless I would recommend it like others. Here is one way to do it using Biopython:&lt;/p&gt;&#xA;&#xA;&lt;pre class=&quot;lang-python prettyprint-override&quot;&gt;&lt;code&gt;from Bio import SeqIO&#xA;&#xA;ids_file = 'ids.txt'&#xA;input_f  = 'input_file.fasta'&#xA;output_f = 'result.fasta'&#xA;&#xA;my_dictionary = {}&#xA;with open(ids_file, 'r') as f:&#xA;    for line in f:&#xA;        my_dictionary[line[:-1]]='value'&#xA;&#xA;with open(input_f, 'r') as input_handle, open(output_f, 'w') as output_handle:&#xA;    for seq_record in SeqIO.parse(input_handle, &amp;quot;fasta&amp;quot;):&#xA;        if not seq_record.id in my_dictionary:&#xA;            SeqIO.write(seq_record, output_handle, &amp;quot;fasta&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2024-01-05T01:22:45.260" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22014" PostTypeId="1" CreationDate="2024-01-05T15:34:35.660" Score="1" ViewCount="39" Body="&lt;p&gt;I want to do LD local clumping.&#xA;The tutorial (&lt;a href=&quot;https://mrcieu.github.io/ieugwasr/articles/local_ld.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://mrcieu.github.io/ieugwasr/articles/local_ld.html&lt;/a&gt;) teaches how to, but the downloaded bfile is GRch37.&#xA;I followed this question to download GRch38.&#xA;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/21247/ld-clump-grch38-gwas-results&quot;&gt;LD clump GRCh38 GWAS results&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I downloaded and it is &amp;quot;all_hg38.pgen&amp;quot;. How can I transform it into bfile? Thank you.&lt;/p&gt;&#xA;" OwnerUserId="18782" LastEditorUserId="18782" LastEditDate="2024-01-05T15:37:29.223" LastActivityDate="2024-02-24T10:02:45.373" Title="LD local clump-GRCh38 bfile" Tags="|r|ld|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22017" PostTypeId="1" CreationDate="2024-01-05T17:15:07.837" Score="0" ViewCount="15" Body="&lt;p&gt;I used the sequence of a protein kinase to generate templates for another and build a homology model using these templates using swiss-model online tool. there are 2 templates that were generated using the same pdb ID and I cannot find a reason as to why this happened, they are identical in all aspects and the homology models generated were also identical with no variance&lt;/p&gt;&#xA;" OwnerUserId="18788" LastEditorUserId="650" LastEditDate="2024-01-06T00:21:29.410" LastActivityDate="2024-01-06T00:21:29.410" Title="why would a pdb ID generate 2 templates that are identical" Tags="|sequence-homology|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22018" PostTypeId="1" CreationDate="2024-01-06T21:35:46.353" Score="0" ViewCount="38" Body="&lt;p&gt;I did manage to bild index file (from NCBI transcriptome) and perform alingment in Rsubread in R with my fq files. I did get BAM files as a result and no error. However i am having trouble with next step as featurecount function does not produce any result( 0%). I have tried builtin annotation (hg38)&lt;/p&gt;&#xA;&lt;p&gt;i am wondering if BAM files are ok..&lt;/p&gt;&#xA;&lt;p&gt;this is the code i have used&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(Rsubread)&#xA;index_path &amp;lt;- &amp;quot;D:/FASTQ_Raw/hum_transcr&amp;quot;&#xA;all_files &amp;lt;- list.files(path = &amp;quot;D:/FASTQ_Raw&amp;quot;, pattern = &amp;quot;.fq&lt;span class=&quot;math-container&quot;&gt;$&quot;, full.names = TRUE)&#xA;common_prefixes &amp;lt;- unique(gsub(&quot;_[12]\\.fq$&lt;/span&gt;&amp;quot;, &amp;quot;&amp;quot;, basename(all_files)))&#xA;&#xA;output_directory &amp;lt;- &amp;quot;aligned_reads/&amp;quot;&#xA;&#xA;dir.create(output_directory, showWarnings = FALSE)&#xA;&#xA;for (prefix in common_prefixes) {&#xA; &#xA;  read1_file &amp;lt;- paste0(&amp;quot;D:/&amp;quot;, prefix, &amp;quot;_1.fq&amp;quot;)&#xA;  read2_file &amp;lt;- paste0(&amp;quot;D:/&amp;quot;, prefix, &amp;quot;_2.fq&amp;quot;)&#xA;  &#xA;  output_file &amp;lt;- paste0(output_directory, prefix, &amp;quot;_aligned.bam&amp;quot;)&#xA;  &#xA;&#xA;  align(index = index_path,&#xA;        readfile1 = read1_file,&#xA;        readfile2 = read2_file,&#xA;        output_file = output_file,&#xA;        nthreads = 4) &#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;so the results/stats of individual BAMs appear ok still when i try featurecount function on BAM files i get zero....i was also using built in annotation index hg38 for attempt of counting but it does not work. I had issues with getting  gtf file from NCBI. What do you recommend ?&lt;/p&gt;&#xA;" OwnerUserId="18791" LastActivityDate="2024-01-08T19:25:03.613" Title="problem with feature counts of RNA bulk seq paired data- in Rsubread" Tags="|r|rna-seq|bioconductor|featurecounts|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22019" PostTypeId="1" CreationDate="2024-01-06T22:15:45.173" Score="0" ViewCount="70" Body="&lt;p&gt;I am struggling in conducting at test on a board of 1000 genes in 50 individuals divided into to groups (A and B). I have a dataset of 1000 genes of two groups of people (total individuals=50)&#xA;group A = 30 (healthy) individuals &amp;amp; group B = 20 sick) individuals&lt;/p&gt;&#xA;&lt;p&gt;I created a matrix where rows are genes and ind are columns.&#xA;now I need to perform a t-test for all the genes and find the p values of my alternative hypothesis (not the null one): i)A&amp;lt;B, A&amp;gt;B and A=!B(not equal). threshold is 0.05 after FDR (which I must include)&lt;/p&gt;&#xA;&lt;p&gt;I am struggling with the code since t-test differ on sensitivity and each of my approach returns a different result. The dummy data is as follows:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pandas as pd&#xA;import numpy as np&#xA;&#xA;# Generate data for group A (healthy individuals) &amp;amp; group B (sick individuals)&#xA;A = np.random.rand(30, 1000)&#xA;B = np.random.rand(20, 1000)&#xA;# Create dataframe for group A &amp;amp; B with genes as columns&#xA;Adf = pd.DataFrame(A, columns=[f'Gene_{i}' for i in range(1, num_genes + 1)])&#xA;Bdf = pd.DataFrame(B, columns=[f'Gene_{i}' for i in range(1, num_genes + 1)])&#xA;&#xA;# Add a column indicating the group&#xA;Adf['Group'] = 'A'&#xA;Bdf['Group'] = 'B'&#xA;&#xA;df = pd.concat([Adf, Bdf], ignore_index=True)&#xA;df.head()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any ideas?&lt;/p&gt;&#xA;" OwnerUserId="18792" LastEditorUserId="650" LastEditDate="2024-02-10T22:34:07.620" LastActivityDate="2024-02-10T22:34:07.620" Title="conducting a ttest on a board of 1000 genes in 50 peope divided into to groups(A and B). pvalues are different each time" Tags="|python|gene|p-values|matrix|t-test|" AnswerCount="1" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="22020" PostTypeId="1" CreationDate="2024-01-07T12:33:55.283" Score="0" ViewCount="42" Body="&lt;p&gt;I have 193000 protein interactions in CSV named &lt;code&gt;all_proteininteractions.csv&lt;/code&gt; which have query protein name in the 2nd column and partner protein in the 3rd like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Query_ENSP,Query_Name,Partner_Name,Combined_Score&#xA;9606.ENSP00000000233,ARF5,RAC2,0.4&#xA;9606.ENSP00000000233,ARF5,RAC2,0.4&#xA;9606.ENSP00000000233,ARF5,PPP2CB,0.408&#xA;9606.ENSP00000000233,ARF5,PPP2CB,0.408&#xA;9606.ENSP00000000233,ARF5,R3HDML,0.413&#xA;9606.ENSP00000000233,ARF5,R3HDML,0.413&#xA;9606.ENSP00000000233,ARF5,PLEK,0.422&#xA;9606.ENSP00000000233,ARF5,PLEK,0.422&#xA;9606.ENSP00000000233,ARF5,RAB11FIP5,0.501&#xA;9606.ENSP00000000233,ARF5,RAB11FIP5,0.501&#xA;9606.ENSP00000000233,ARF5,AP3B1,0.553&#xA;9606.ENSP00000000233,ARF5,AP3B1,0.553&#xA;9606.ENSP00000000233,ARF5,CYTH4,0.579&#xA;9606.ENSP00000000233,ARF5,CYTH4,0.579&#xA;9606.ENSP00000000233,ARF5,RABAC1,0.658&#xA;9606.ENSP00000000233,ARF5,RABAC1,0.658&#xA;9606.ENSP00000000233,ARF5,YKT6,0.672&#xA;9606.ENSP00000000233,ARF5,YKT6,0.672&#xA;9606.ENSP00000000233,ARF5,BET1,0.679&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am using this Python code to fetch sequences of each protein and store them in a CSV file:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import pandas as pd&#xA;import requests&#xA;import math&#xA;&#xA;requests.packages.urllib3.disable_warnings()&#xA;&#xA;# Function to fetch protein sequence from UniProt&#xA;def fetch_protein_sequence_from_uniprot(protein_name):&#xA;    #Declaring UniProt API&#xA;    uniport_api_url = f&amp;quot;https://rest.uniprot.org/uniprotkb/search?query=gene_exact:{protein_name}+AND+organism_id:9606&amp;amp;format=fasta&amp;amp;size=1&amp;quot;&#xA;    response = requests.get(uniport_api_url,verify=False)&#xA;&#xA;    # Parse the response to extract sequence&#xA;    sequence = &amp;quot;&amp;quot;&#xA;    if response.ok:&#xA;        lines = response.text.split(&amp;quot;\n&amp;quot;)&#xA;        for line in lines:&#xA;            if not line.startswith(&amp;quot;&amp;gt;&amp;quot;):  # Exclude header lines&#xA;                sequence += line&#xA;    return sequence&#xA;&#xA;# Reading the input CSV file&#xA;input_csv = &amp;quot;all_protein_interactions.csv&amp;quot;&#xA;chunksize = 1000  # Batch size&#xA;&#xA;# Create an iterator to read the CSV in chunks&#xA;csv_reader = pd.read_csv(input_csv, chunksize=chunksize)&#xA;&#xA;# Initialize an empty list to store modified chunks&#xA;modified_chunks = []&#xA;&#xA;# Process each chunk&#xA;for chunk in csv_reader:&#xA;    # Fetch protein sequences for query and partner names&#xA;    query_sequences = [fetch_protein_sequence_from_uniprot(name) for name in chunk.iloc[:, 1]]&#xA;    partner_sequences = [fetch_protein_sequence_from_uniprot(name) for name in chunk.iloc[:, 2]]&#xA;&#xA;    # Add fetched sequences as new columns in the chunk&#xA;    chunk['Query_Sequence'] = query_sequences&#xA;    chunk['Partner_Sequence'] = partner_sequences&#xA;&#xA;    # Append the modified chunk to the list&#xA;    modified_chunks.append(chunk)&#xA;&#xA;# Concatenate all modified chunks into a single DataFrame&#xA;result_df = pd.concat(modified_chunks, ignore_index=True)&#xA;&#xA;# Save the updated DataFrame back to CSV&#xA;result_df.to_csv(&amp;quot;protein_interactionswithseqs.csv&amp;quot;, index=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;First, it was throwing SSLError, so I made &lt;code&gt;verify=False&lt;/code&gt;. But now, after running the script for 3 hours it's showing following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ConnectionError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How can I remove this connection aborted error to fetch 386000 sequences?&lt;/p&gt;&#xA;" OwnerUserId="9269" LastEditorUserId="650" LastEditDate="2024-01-08T17:14:34.020" LastActivityDate="2024-02-07T18:02:15.270" Title="Fetching Protein Sequences through UniPROT gives ConnectionError" Tags="|python|uniprot|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22021" PostTypeId="2" ParentId="22020" CreationDate="2024-01-07T14:04:46.817" Score="1" Body="&lt;p&gt;Your script works as expected on the example you gave. Running your script results in a &lt;code&gt;protein_interactionswithseqs.csv&lt;/code&gt; with lines like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ head -n2 protein_interactionswithseqs.csv &#xA;Query_ENSP,Query_Name,Partner_Name,Combined_Score,Query_Sequence,Partner_Sequence&#xA;9606.ENSP00000000233,ARF5,RAC2,0.4,MGLTVSALFSRIFGKKQMRILMVGLDAAGKTTILYKLKLGEIVTTIPTIGFNVETVEYKNICFTVWDVGGQDKIRPLWRHYFQNTQGLIFVVDSNDRERVQESADELQKMLQEDELRDAVLLVFANKQDMPNAMPVSELTDKLGLQHLRSRTWYVQATCATQGTGLYDGLDWLSHELSKR,MQAIKCVVVGDGAVGKTCLLISYTTNAFPGEYIPTVPGRPCALLQDAEAGLSSDVEGACWCQTQEGRMTGEAPAPPWPALRGPHHS&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, this is horribly, horribly inefficient! You are fetching the same sequence multiple times. With your example data, the protein ARF5 is actually downloaded 19 times, meaning this will take 18 times more time than needed. This is very likely the cause for the timeout error you describe. Now, it might be that your sequences are too many to be stored in RAM (but I would still try it), so one way around this is to download the sequences first, and then write the mappings. Something like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import pandas as pd&#xA;import requests&#xA;import sys&#xA;requests.packages.urllib3.disable_warnings()&#xA;&#xA;&#xA;def write_protein_seq_file(protein_name, protein_seq):&#xA;    file_name = &amp;quot;seqs/&amp;quot; + protein_name&#xA;    seq_file = open(file_name, &amp;quot;w&amp;quot;)&#xA;    data = f&amp;quot;{protein_seq}\n&amp;quot;&#xA;    seq_file.write(data)&#xA;    seq_file.close&#xA;&#xA;def read_protein_seq_file(protein_name):&#xA;    seq_file = open(&amp;quot;seqs/&amp;quot; + protein_name, &amp;quot;r&amp;quot;)&#xA;    data = seq_file.read()&#xA;    seq_file.close&#xA;    return data.rstrip()&#xA;&#xA;# Function to fetch protein sequence from UniProt&#xA;def fetch_protein_sequence_from_uniprot(protein_name):&#xA;    #Declaring UniProt API&#xA;    uniport_api_url = f&amp;quot;https://rest.uniprot.org/uniprotkb/search?query=gene_exact:{protein_name}+AND+organism_id:9606&amp;amp;format=fasta&amp;amp;size=1&amp;quot;&#xA;    response = requests.get(uniport_api_url,verify=False)&#xA;&#xA;    # Parse the response to extract sequence&#xA;    sequence = &amp;quot;&amp;quot;&#xA;    if response.ok:&#xA;        lines = response.text.split(&amp;quot;\n&amp;quot;)&#xA;        for line in lines:&#xA;            if not line.startswith(&amp;quot;&amp;gt;&amp;quot;):  # Exclude header lines&#xA;                sequence += line&#xA;    return sequence&#xA;&#xA;# Reading the input CSV file&#xA;input_csv = &amp;quot;all_protein_interactions.csv&amp;quot;&#xA;chunksize = 1000  # Batch size&#xA;&#xA;# Create an iterator to read the CSV in chunks&#xA;csv_reader = pd.read_csv(input_csv, chunksize=chunksize)&#xA;&#xA;# Initialize an empty list to store modified chunks&#xA;modified_chunks = []&#xA;&#xA;# Get all sequences&#xA;for chunk in csv_reader:&#xA;    for protein in set(list(chunk.iloc[:, 1]) + list(chunk.iloc[:, 2])):&#xA;        protein_seq = fetch_protein_sequence_from_uniprot(protein) &#xA;        write_protein_seq_file(protein,protein_seq)&#xA;&#xA;print(&amp;quot;Finished fetching sequences&amp;quot;, file=sys.stderr)     &#xA;## Now, process the file again and write the new file&#xA;csv_reader = pd.read_csv(input_csv, chunksize=chunksize)&#xA;&#xA;for chunk in csv_reader:&#xA;&#xA;    # Fetch protein sequences for query and partner names&#xA;    query_sequences = [read_protein_seq_file(name) for name in chunk.iloc[:, 1]]&#xA;    partner_sequences = [read_protein_seq_file(name) for name in chunk.iloc[:, 2]]&#xA;&#xA;    # Add fetched sequences as new columns in the chunk&#xA;    chunk['Query_Sequence'] = query_sequences&#xA;    chunk['Partner_Sequence'] = partner_sequences&#xA;&#xA;    # Append the modified chunk to the list&#xA;    modified_chunks.append(chunk)&#xA;&#xA;# Concatenate all modified chunks into a single DataFrame&#xA;result_df = pd.concat(modified_chunks, ignore_index=True)&#xA;&#xA;# Save the updated DataFrame back to CSV&#xA;result_df.to_csv(&amp;quot;protein_interactionswithseqs.csv&amp;quot;, index=False) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;First, create a directory named &lt;code&gt;seqs&lt;/code&gt;, then run the script above. It will process the csv file twice. Once to get all sequences mentioned in the file and save each one in a file called &lt;code&gt;seqs/$sequence_name&lt;/code&gt; (where &lt;code&gt;$sequence_name&lt;/code&gt; is replaced by the actual name of each protein. Then, it goes through the file a second time, but instead of the very time consuming download, it now fetches the sequence from the file stored on your computer. Once you are satisfied this works, you can delete the &lt;code&gt;seqs&lt;/code&gt; directory.&lt;/p&gt;&#xA;&lt;p&gt;Note that I don't have access to a Windows machine, so this was tested on Linux. Also, I am not a python coder, so I am sure this ins't very good python and there will be improvements to be made, but even on the toy example you gave, this speeds things up significantly:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ time your_script.py &#xA;&#xA;real    0m10.186s&#xA;user    0m0.998s&#xA;sys     0m0.076s&#xA;&#xA;$ time my_script.py&#xA;Finished fetching sequences&#xA;&#xA;real    0m3.180s&#xA;user    0m0.456s&#xA;sys     0m0.066s&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see above, even on the tiny example with only 19 data lines, your script took 10 seconds while mine only took 3. So try this approach (or, better, take my example and make it more pythonic, add some error checking, some try/except blocks, make sure your data are correct) and you might find the timeout error is no longer an issue. You should certainly find that the whole thing takes far less than three &lt;em&gt;hours&lt;/em&gt;(!) to run.&lt;/p&gt;&#xA;&lt;p&gt;First of all, that is a really, really inefficient way of doing it. Using your example data, you are getting the same protein, ARF5, 19 times. This makes your program 18 times slower than it needs to be. You should modify the script to download each protein &lt;em&gt;once&lt;/em&gt; and then write them all out.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2024-01-07T14:04:46.817" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22022" PostTypeId="1" CreationDate="2024-01-08T09:42:28.743" Score="0" ViewCount="25" Body="&lt;p&gt;I am working with single cell RNA data- control and treated. I analyzed and got a list of upregulated genes. Upon doing pathway analysis all the genes were of apoptotic pathway. But these genes can signify different meaning to my interpretation, because, these genes were upregulated even in diseased condition. let me take an example, NEAT1 is one of the gene that is seen upregulated in the cancer condition and is associated with cancer progression. the treated sample which is a therapy also has neat1 upregulated compared to control which was untreated. Just because pathway analysis showed apoptosis, I cannot conclude that neat1 gene is aiding in killing the cancer cell due to the therapy right/ It can also be like due to continued upregulation its delaying cancer cell death due to therpay.&lt;/p&gt;&#xA;&lt;p&gt;How to confirm computationally the above two statements and find the upregulated genes actual role in the therapy?&#xA;to be noted: there is no prior data or pattern available for the therapy based gene expression yet.&lt;/p&gt;&#xA;" OwnerUserId="18603" LastActivityDate="2024-01-08T09:42:28.743" Title="Suggestion for way foward with upregulated genes" Tags="|rna-seq|scrnaseq|ngs|cancer|pathway|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22024" PostTypeId="1" CreationDate="2024-01-08T15:12:16.403" Score="-1" ViewCount="34" Body="&lt;p&gt;I'm visualizing where mutations occur in a gene, and would like to add functional domains to the plots.&lt;/p&gt;&#xA;&lt;p&gt;I have some trouble locating sources to download data on functional domains.&lt;/p&gt;&#xA;&lt;p&gt;The dataset should contain:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Functional domain name&lt;/li&gt;&#xA;&lt;li&gt;Start position in gene&lt;/li&gt;&#xA;&lt;li&gt;End position in gene&lt;/li&gt;&#xA;&lt;li&gt;Gene symbol name&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Any tips kindly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="18798" LastActivityDate="2024-02-01T21:06:40.457" Title="Where can you download functional domain labels of the human genome?" Tags="|genome|database|data-download|public-dataset|functional-annotation|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22025" PostTypeId="1" CreationDate="2024-01-08T17:23:21.937" Score="0" ViewCount="53" Body="&lt;p&gt;Good afternoon,&lt;/p&gt;&#xA;&lt;p&gt;I just started working with Single cell RNA Seq and I am trying to understand what is an average / possible number of cells per sample.&lt;/p&gt;&#xA;&lt;p&gt;I am looking at a dataset of 7 patients with kidney tumour samples (please find attached). Would these number of cells be possible? I basically want to double check whether I didnt do any mistakes in my pre-processing.&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;img src=&quot;https://i.stack.imgur.com/mr2J2.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/p&gt;&#xA;" OwnerUserId="18799" LastActivityDate="2024-02-11T00:05:00.587" Title="What is a common number of cells per cell type in single cell RNA Seq?" Tags="|scrnaseq|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22026" PostTypeId="2" ParentId="22018" CreationDate="2024-01-08T19:25:03.613" Score="-1" Body="&lt;p&gt;I don't think FeatureCounts is meant to work on an alignment to the transcriptome.  It's not at all clear to me that the subread aligner, or any aligner, is meant to be used on a transcriptome.&lt;/p&gt;&#xA;&lt;p&gt;If you have the transcriptome, I'd just use pseudoaligners like Kallisto or Salmon.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2024-01-08T19:25:03.613" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22027" PostTypeId="2" ParentId="17486" CreationDate="2024-01-09T06:53:33.173" Score="0" Body="&lt;pre&gt;&lt;code&gt;library(dplyr)&#xA;#create the function to convert mouse to human genes&#xA;homologs = read.csv(&amp;quot;http://www.informatics.jax.org/downloads/reports/HOM_MouseHumanSequence.rpt&amp;quot;,sep=&amp;quot;\t&amp;quot;)&#xA;ms2hs &amp;lt;- function(gene_list){&#xA;  output = c()&#xA;  for(gene in gene_list){&#xA;    class_key = (homologs %&amp;gt;% filter(Symbol == gene &amp;amp; Common.Organism.Name==&amp;quot;mouse, laboratory&amp;quot;))[['DB.Class.Key']]&#xA;    if(!identical(class_key, integer(0)) ){&#xA;      human_genes = (homologs %&amp;gt;% filter(DB.Class.Key == class_key &amp;amp; Common.Organism.Name==&amp;quot;human&amp;quot;))[,&amp;quot;Symbol&amp;quot;]&#xA;      for(human_gene in human_genes){&#xA;        output = append(output,human_gene)&#xA;      }&#xA;    }&#xA;  }&#xA;  return (output)&#xA;}&#xA;&#xA;#apply the function&#xA;msgenes=c(&amp;quot;Zfp192&amp;quot;, &amp;quot;Ifitm2&amp;quot;, &amp;quot;Mettl8&amp;quot;, &amp;quot;Kcnh2&amp;quot;, &amp;quot;Npdc1&amp;quot;, &amp;quot;Pitpnc1&amp;quot;, &amp;quot;Ooep&amp;quot;, &amp;quot;Mettl8&amp;quot;)&#xA;ms2hs(msgenes)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18801" LastActivityDate="2024-01-09T06:53:33.173" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22028" PostTypeId="1" CreationDate="2024-01-09T14:44:45.663" Score="3" ViewCount="71" Body="&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/gErzk.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/gErzk.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;[Update] Thanks @terdon. To clarify my question:&lt;/p&gt;&#xA;&lt;p&gt;I have a bunch of protein isoforms sequences (produced by the transcripts in the figure) and I want to align protein products (e.g., proteins produced by the first 3 protein-coding transcripts against each other). See there are 3 blocks missing in 5' region in BRCA2-204 compared against BRCA2-206.&lt;/p&gt;&#xA;&lt;p&gt;This case might be very easy with any MSA softwares such as clustal omega, but there are complicated cases which may trick software into aligning single amino acids instead of being splicing aware. Instead of manually adjusting the alignments, I want to know whether I can directly download, or &amp;quot;translate&amp;quot;, this figure into alignment results (e.g., fasta format or clustal format)&lt;/p&gt;&#xA;&lt;p&gt;This figure should be visible with &lt;a href=&quot;https://www.ensembl.org/Homo_sapiens/Gene/Summary?db=core;g=ENSG00000139618;r=13:32315086-32400268&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ensembl.org/Homo_sapiens/Gene/Summary?db=core;g=ENSG00000139618;r=13:32315086-32400268&lt;/a&gt;&#xA;in the gene summary tab&lt;/p&gt;&#xA;&lt;p&gt;[Original] Is there a way to directly download the sequence alignment result from Ensembl? If not, what will be the best way to align exons and introns without parameter-tuning&lt;/p&gt;&#xA;" OwnerUserId="18802" LastEditorUserId="298" LastEditDate="2024-01-13T12:02:55.313" LastActivityDate="2024-01-13T12:02:55.313" Title="What is the best way to acquire protein isoform sequence alignment?" Tags="|sequence-alignment|proteins|ensembl|multiple-sequence-alignment|" AnswerCount="1" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="22030" PostTypeId="2" ParentId="22028" CreationDate="2024-01-09T17:05:49.313" Score="1" Body="&lt;p&gt;Here is a manual (or semi-manual, at best) approach. If you need to do this for very many sequences it won't be practical, but if dealing with one or two genes, it might be enough. The idea is to:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Get the protein coding transcripts&lt;/li&gt;&#xA;&lt;li&gt;Get the proteins they code for&lt;/li&gt;&#xA;&lt;li&gt;Align&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;In the page you gave, there is a &amp;quot;Show transcript table&amp;quot; button near the top:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/FYLSC.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/FYLSC.png&quot; alt=&quot;Screenshot of the EnsEmbl page showing the relevant button&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you click on that, you get the list of transcripts and their corresponding proteins:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/u25Ur.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/u25Ur.png&quot; alt=&quot;Screenshot of the Ensembl page showing the expanded transcript table&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can then click on the &amp;quot;Show/hide columns&amp;quot; and select &amp;quot;Translation ID&amp;quot; so you can get the Ensembl protein corresponding to each transcript:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ktoXx.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ktoXx.png&quot; alt=&quot;Screenshot of the Ensembl page with &amp;quot;Translation ID&amp;quot; selected&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you now deselct everything and only leave &amp;quot;Translation ID&amp;quot; selected:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/1UJwv.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/1UJwv.png&quot; alt=&quot;Screenshot of the Ensembl page with only &amp;quot;Translation ID&amp;quot; selected&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You are left with a nice list of Ensembl proteins. Copy/paste those into a text file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ENSP00000369497.3&#xA;ENSP00000499438.2&#xA;ENSP00000505508.1&#xA;ENSP00000439902.1&#xA;ENSP00000514856.2&#xA;ENSP00000434898.2&#xA;ENSP00000506251.1&#xA;ENSP00000499570.2&#xA;ENSP00000433168.2&#xA;ENSP00000499256.2&#xA;ENSP00000514855.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, go to &lt;a href=&quot;https://www.ensembl.org/biomart/martview/cc1e3843edd9549a4fb578f900d88d23&quot; rel=&quot;nofollow noreferrer&quot;&gt;BioMart&lt;/a&gt; and retrieve the sequences. Select &amp;quot;Ensembl Genes&amp;quot; as the database and then &amp;quot;Human Genes&amp;quot;:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/SgWpW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/SgWpW.png&quot; alt=&quot;biomart screenshot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then, click on &amp;quot;Filters&amp;quot; (1) and then &amp;quot;Gene&amp;quot; (2), select &amp;quot;Input external references ID list [Max 500 advised]&amp;quot; (3), set the ID to &amp;quot;Protein stable ID(s) with version&amp;quot; (4) and paste the protein list (5):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/rDGmI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/rDGmI.png&quot; alt=&quot;biomart screenshot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Next, go to &amp;quot;Attributes&amp;quot; (1) and select &amp;quot;Sequences&amp;quot; (2):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/iDwcj.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/iDwcj.png&quot; alt=&quot;biomart screenshot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, finally, click on &amp;quot;Results&amp;quot; (1) and you will get the sequences in fasta format (2):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/26Fx3.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/26Fx3.png&quot; alt=&quot;biomart screenshot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I'm afraid they are too long to fit in this answer, but I posted them here: &lt;a href=&quot;https://pastebin.com/NZycAmgD&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://pastebin.com/NZycAmgD&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You can now paste these sequences into your favorite aligner and get the alignment you were after. Here, I used the online &lt;a href=&quot;https://www.ebi.ac.uk/Tools/msa/clustalo/&quot; rel=&quot;nofollow noreferrer&quot;&gt;clustalΩ&lt;/a&gt; (the image only shows the first few lines):&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/mZap9.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/mZap9.png&quot; alt=&quot;clustalΩ alignment of the BRCA2 proteins&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can find the whole alignment in clustal format (text) &lt;a href=&quot;https://pastebin.com/GhHeG9CJ&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2024-01-09T17:05:49.313" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22032" PostTypeId="1" CreationDate="2024-01-09T21:26:46.597" Score="2" ViewCount="33" Body="&lt;p&gt;I'm working with RADseq data (288 compressed FASTQ files, &lt;code&gt;*.fq.gz&lt;/code&gt;) from 3 plates of plant tissue. I've done demultiplexing and preliminary QC (using the &lt;a href=&quot;https://catchenlab.life.illinois.edu/stacks/comp/process_radtags.php&quot; rel=&quot;nofollow noreferrer&quot;&gt;Stacks &lt;code&gt;process_radtags&lt;/code&gt; command&lt;/a&gt;), and before performing a reference alignment, I want make sure the sequences in each sample (roughly) correspond to the organism I expect them to be derived from, rather than potential contaminants (bacteria and/or fungi).&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have a straightforward, &amp;quot;canned&amp;quot; approach for doing this, either in R or a different environment? I believe the &lt;code&gt;blastn&lt;/code&gt; algorithm can be used to query the sequences in these files against a database (like NCBI), and is usable on the command line. However, I've not used it before, and I'm not sure if it makes sense to query &lt;em&gt;all&lt;/em&gt; the sequences in each file. Additionally, I don't know if the command can process this many files of this size (~600 MB per FASTQ file).&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/7473/testing-for-viral-bacterial-infection-fastq-files&quot;&gt;This post&lt;/a&gt; from ~4 years ago asks a similar question for a pair of human blood samples, but is concerned with bacterial/viral DNA. The answer linking &lt;code&gt;kmerfinder&lt;/code&gt; might have promise, but I'd appreciate any input from folks familiar with this issue.&lt;/p&gt;&#xA;&lt;p&gt;(&lt;em&gt;Background&lt;/em&gt;: I've gotten my Master's degree in Biology and have worked with NGS data for ~4 years now. I'm performing analyses on a 32-core Linux server. I'm less skilled in Python, but am willing to use it if there are no other options.)&lt;/p&gt;&#xA;" OwnerUserId="14041" LastActivityDate="2024-01-18T19:52:32.593" Title="Approaches for filtering bacterial/fungal contaminant sequences from RADseq results" Tags="|ngs|filtering|bacteria|blastn|radseq|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22033" PostTypeId="1" AcceptedAnswerId="22046" CreationDate="2024-01-10T09:07:52.080" Score="1" ViewCount="34" Body="&lt;p&gt;I'm currently performing some initial analyses on targeted Illumina sequencing data generated with DNA from diseased tissue. My hypothesis is that one particular gene might contain somatic mutations in this disease. I'm currently running Mutect2 in tumor-only mode, using default settings.&lt;/p&gt;&#xA;&lt;p&gt;As I have only found very few mutations (this result might well be plausible), I'd like to adjust the Mutect2 cutoffs to increase the sensitivity and see whether the result changes. For instance, I'd like to reduce VAF, depth of coverage and QUAL. Unfortunately, I can't work out which of the many arguments have to be modified and how: &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360037593851-Mutect2&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/360037593851-Mutect2&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can you help me out? Thanks!&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gatk Mutect2 \&#xA;-R $fastaFile \&#xA;-I $inputFile \&#xA;--germline-resource $germlineResource \&#xA;-O $resultDir/$sampleName.Mutect.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9025" LastEditorUserId="650" LastEditDate="2024-01-10T15:21:47.107" LastActivityDate="2024-01-11T23:48:17.677" Title="How to adjust cutoffs in Mutect2" Tags="|gatk|somatic|mutect2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22036" PostTypeId="1" AcceptedAnswerId="22037" CreationDate="2024-01-10T19:52:00.237" Score="6" ViewCount="78" Body="&lt;p&gt;In the popular &lt;code&gt;picard MarkDuplicates&lt;/code&gt; tool, a read is marked as a duplicate if it has the same position as another read starting from their first unclipped base in the 5'-direction. The implication is that if two reads have this property, it must be much more likely they arose from some &amp;quot;artificial&amp;quot; duplicates (PCR, optical, etc.) rather than due to sheer coincidence that two independent fragments of DNA happened to have this property and were sequenced in the same run.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have links to studies or other data that attempts to measure how often this mechanism for labeling duplicates fails? I.e. how frequently does a read get marked a duplicate in this way &amp;quot;incorrectly,&amp;quot; meaning it was actually just a coincidence and really did correspond to an independent fragment that should be considered in your analysis.&lt;/p&gt;&#xA;&lt;p&gt;I imagine this answer can vary quite a bit based on the sequencing conditions: library prep, machine, etc. I was hoping for at least a ballpark estimate in the case of some &amp;quot;standard&amp;quot; modern short read sequencing methods. But it should be possible to study this in particular cases, e.g. using UMIs, doing PCR vs non-PCR runs, etc.&lt;/p&gt;&#xA;" OwnerUserId="18808" LastActivityDate="2024-01-12T16:36:00.100" Title="How many false positive duplicates are marked using just the position of first unclipped base?" Tags="|sequence-alignment|reads|markduplicate|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22037" PostTypeId="2" ParentId="22036" CreationDate="2024-01-10T21:21:20.677" Score="6" Body="&lt;p&gt;There's a tool in Picard (full disclaimer: I wrote it...) called &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/13832626034203-CollectIndependentReplicateMetrics-Picard-EXPERIMENTAL-&quot; rel=&quot;nofollow noreferrer&quot;&gt;CollectIndependentReplicateMetrics&lt;/a&gt; which performs the test you are looking for, I think.&lt;/p&gt;&#xA;&lt;p&gt;It measures the percent of duplication that is &amp;quot;independent&amp;quot;, i.e. which arose from randomly sampled original fragments that &lt;em&gt;happen&lt;/em&gt; to be duplicates, rather than due to technical (optical, PCR, bridge-amp...) reasons.&lt;/p&gt;&#xA;&lt;p&gt;The way it works is by looking at duplicate-sets of size 2 (and separately, 3) over heterozygous sites in the bam (which is assumed to be a diploid sample) it classifies the sets into &amp;quot;homogeneous&amp;quot; and &amp;quot;heterogeneous&amp;quot; based on the alleles of the hetrozygous variant. From the relative sizes of these two classes, one can work out the percent of duplication that is &amp;quot;independent&amp;quot; since independent duplication will give rise to heterogeneous sets (sometimes), while dependent duplication will &lt;em&gt;never&lt;/em&gt; do that.&lt;/p&gt;&#xA;&lt;p&gt;The assumptions are quite simple actually:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a technical duplicate will have the same allele&lt;/li&gt;&#xA;&lt;li&gt;an independent replicate will have the same allele 50% of the time&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;So the rate of independence is 2x the rate of heterozygosity, &lt;em&gt;within&lt;/em&gt; duplicate sets of size 2.&lt;/p&gt;&#xA;&lt;p&gt;For sets of size 3 the math is a little more complex, so I wrote it up and docuemnted it.&lt;/p&gt;&#xA;&lt;p&gt;Here's the verbatim explanation I wrote at the time (in the source-code of the &lt;em&gt;Metric&lt;/em&gt; file...):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Explanation of calculation of independent replication rate from heterozygosity rate (within triplicate sets):&#xA;&#xA;We assume the there are two types of replication events:&#xA;&#xA;- those that are &amp;quot;independent&amp;quot;, such that we just happen to get 2 fragments from the exact same region&#xA;(These get a random allele so effectively change the allele with probability 0.5), and&#xA;- those that are &amp;quot;artifactual&amp;quot; (do not change the allele)&#xA;&#xA;We skip sets that have unexpected alleles as they do not fit our model. In the following we use the term &amp;quot;duplicates&amp;quot; to indicate that&#xA;the read-pairs would be marked as duplicates, not that they actually are technical duplicates.&#xA;&#xA;To reach a triplicate set, 2 replication events are required so there are the following options, assuming&#xA;that the independent replication rate is x (thus the artifactual is 1-x). We assume a diploid organism with no bias towards either allele&#xA;in heterozygous sites, so an idependent replication will result in the other alleles in half the cases:&#xA;&#xA;0 -&amp;gt; 0, 1 happens with probability x/2, therefore&#xA;0 -&amp;gt; 0, 0 happens with probability 1-x/2&#xA;&#xA;(This is the explanation that is required for calculating the independent replication rate from doubleton sets...quite simpler)&#xA;&#xA;Without loss of generality we assume that we &amp;quot;start&amp;quot; with allele 0 and that 1 is the other allele.&#xA;&#xA;Each of the resulting alleles (&amp;quot;First&amp;quot; or &amp;quot;second&amp;quot;) can replicate (each with probability 0.5) so we get:&#xA;&#xA;from first row:&#xA;0=&amp;gt;1 1 (0 replicate to a 1) with probability  x/2 * 0.5 * x/2&#xA;0=&amp;gt;0 1 (0 replicate to a 0) with probability  x/2 * 0.5 * (1-x/2)&#xA;=====subtotal = x/4&#xA;&#xA;0 1=&amp;gt;0 (1 replicate to a 0) with probability x/2 * 0.5 * x/2&#xA;0 1=&amp;gt;1 (1 replicate to a 1) with probability x/2 * 0.5 * (1-x/2)&#xA;=====subtotal = x/4&#xA;&#xA;from second row:&#xA;0=&amp;gt;1 0 (first 0 replicate to a 1) with probability (1-x/2) * 0.5 * x/2&#xA;0=&amp;gt;0 0 (first 0 replicate to a 0) with probability (1-x/2) * 0.5 * (1-x/2)   &amp;lt;======= Homogeneous set&#xA;=====subtotal = (1-x/2)/2&#xA;&#xA;0 0=&amp;gt;1 (second 0 replicate to a 1) with probability (1-x/2) * 0.5 * x/2&#xA;0 0=&amp;gt;0 (second 0 replicate to a 0) with probability (1-x/2) * 0.5 * (1-x/2)  &amp;lt;======= Homogeneous set&#xA;=====subtotal = (1-x/2)/2&#xA;&#xA;&#xA;total is x/2 (from first two sub-totals) + (1-x/2) (from last two sub-totals) = 1&#xA;&#xA;We differentiate between a heterogeneous result (with 0's and 1's in the set) and homogeneous results&#xA;(all zeros, since we assumed WLOG that we start with 0)&#xA;The probability of a homogeneous set is therefore the sum of the two only homogeneous results&#xA;&#xA;P(hom) = (1-x/2) * 0.5 * (1-x/2) + (1-x/2) * 0.5 * (1-x/2) = (1-x/2)^2 = y&#xA;&#xA;where y = P(hom) is the rate of homogeneity within triplicate sets.&#xA;&#xA;Solving for x we find that 2 * ( 1 - sqrt(y) ) = x.&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="9836" LastEditorUserId="9836" LastEditDate="2024-01-12T16:36:00.100" LastActivityDate="2024-01-12T16:36:00.100" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="22039" PostTypeId="2" ParentId="22019" CreationDate="2024-01-11T12:41:13.653" Score="1" Body="&lt;p&gt;Could it be that you are getting different results because you are not setting a seed on your random number generator?&lt;/p&gt;&#xA;&lt;p&gt;The recommended way to make sure you get the same results when using numpy to generate dummy data is:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np&#xA;# make random number generator with seed&#xA;rng = np.random.default_rng(seed=42)&#xA;rng.random([4,4])&#xA;#Out: &#xA;#array([[0.77395605, 0.43887844, 0.85859792, 0.69736803],&#xA;#       [0.09417735, 0.97562235, 0.7611397 , 0.78606431],&#xA;#       [0.12811363, 0.45038594, 0.37079802, 0.92676499],&#xA;#       [0.64386512, 0.82276161, 0.4434142 , 0.22723872]])&#xA;&#xA;&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18813" LastActivityDate="2024-01-11T12:41:13.653" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22040" PostTypeId="1" AcceptedAnswerId="22044" CreationDate="2024-01-11T12:49:07.467" Score="1" ViewCount="71" Body="&lt;p&gt;I asked this question already in the matter modeling forum (&lt;a href=&quot;https://mattermodeling.stackexchange.com/questions/12213/how-to-calculate-pharmacore-features&quot;&gt;here&lt;/a&gt;). But i hope this is a better place because the topic is more biological.&lt;/p&gt;&#xA;&lt;p&gt;I want to use the enzyme ß-Glucuronidase with the &lt;a href=&quot;https://pharmit.csb.pitt.edu/&quot; rel=&quot;nofollow noreferrer&quot;&gt;pharmit&lt;/a&gt; database. I uploaded my file for the receptor from pdb file. How can I get the pharmacophore features? So one thing is the enzyme is quiet popular, so is there any database with the features? I tried it with a &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3092953/&quot; rel=&quot;nofollow noreferrer&quot;&gt;publication&lt;/a&gt; but the given link is not working. It checked the site from the author but I did not found the database there.&lt;/p&gt;&#xA;&lt;p&gt;And if there is no database or maybe there is one but the enzyme is not included, how can I get the pharmacophore features for my calculations?&lt;/p&gt;&#xA;" OwnerUserId="18814" LastEditorUserId="10049" LastEditDate="2024-01-14T19:49:56.263" LastActivityDate="2024-01-14T19:49:56.263" Title="How to calculate pharmacophore features?" Tags="|docking|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22041" PostTypeId="1" AcceptedAnswerId="22049" CreationDate="2024-01-11T13:44:27.253" Score="1" ViewCount="79" Body="&lt;p&gt;In gnomAD related article&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9160216/#:%7E:text=Allele%20frequency%20is%20calculated%20by,can%20differ%20substantially%20between%20positions&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9160216/#:~:text=Allele%20frequency%20is%20calculated%20by,can%20differ%20substantially%20between%20positions&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I road this section&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Allele frequency is calculated by dividing allele count by allele&#xA;number, hence, allele frequency represents the frequency of&#xA;confidently sequenced haplotypes that carry the allele in question&#xA;(because coverage and sequencing quality varies across the genome, the&#xA;allele number can differ substantially between positions). Allele&#xA;frequency is not equivalent to the percentage of individuals that&#xA;carry the allele, but is a suitable value for expressing the frequency&#xA;of a variant in the general population. The number of individuals&#xA;carrying a variant will depend on the number of heterozygous and&#xA;homozygous individuals but can be calculated from the data provided in&#xA;the variant table.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Does this mean, for getting the frequency of each allele in gnomAD, one could use &lt;code&gt;Allele frequency (AF)&lt;/code&gt; column or must calculate population frequency?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2024-01-16T15:02:07.687" LastActivityDate="2024-01-16T15:02:07.687" Title="Variant allele frequency versus population frequency" Tags="|allele-frequency|gnomad|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22042" PostTypeId="1" CreationDate="2024-01-11T13:55:13.990" Score="0" ViewCount="18" Body="&lt;p&gt;I'm want to perform an GSEA of some RNA-seq data but when I upload the expression data set file this problem appears but I don't know how to solve it.&lt;/p&gt;&#xA;&lt;p&gt;Thanks for your help.&lt;/p&gt;&#xA;&lt;p&gt;Pol&lt;/p&gt;&#xA;&lt;p&gt;Error Details&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;---- Full Error Message ----&#xA;There were errors: ERROR(S) #:1&#xA;Parsing trouble&#xA;java.lang.NumberFormatException: ...&#xA;&#xA;---- Stack Trace ----&#xA;# of exceptions: 1&#xA;------For input string: &amp;quot;description&amp;quot;------&#xA;java.lang.NumberFormatException: For input string: &amp;quot;description&amp;quot;&#xA;    at java.base/jdk.internal.math.FloatingDecimal.readJavaFormatString(Unknown Source)&#xA;    at java.base/jdk.internal.math.FloatingDecimal.parseFloat(Unknown Source)&#xA;    at java.base/java.lang.Float.parseFloat(Unknown Source)&#xA;    at org.gsea_msigdb.gsea/edu.mit.broad.genome.parsers.AbstractParser.parseStringToFloat(AbstractParser.java:250)&#xA;    at org.gsea_msigdb.gsea/edu.mit.broad.genome.parsers.AbstractParser.parseFieldsIntoFloatArray(AbstractParser.java:361)&#xA;    at org.gsea_msigdb.gsea/edu.mit.broad.genome.parsers.AbstractParser.parseTextMatrixToDataset(AbstractParser.java:277)&#xA;    at org.gsea_msigdb.gsea/edu.mit.broad.genome.parsers.TxtDatasetParser.parse(TxtDatasetParser.java:108)&#xA;    at org.gsea_msigdb.gsea/edu.mit.broad.genome.parsers.ParserFactory.readDatasetTXT(ParserFactory.java:200)&#xA;    at org.gsea_msigdb.gsea/edu.mit.broad.genome.parsers.ParserFactory.read(ParserFactory.java:758)&#xA;    at org.gsea_msigdb.gsea/edu.mit.broad.genome.parsers.ParserFactory.read(ParserFactory.java:735)&#xA;    at org.gsea_msigdb.gsea/edu.mit.broad.genome.parsers.ParserWorker.doInBackground(ParserWorker.java:53)&#xA;    at java.desktop/javax.swing.SwingWorker&lt;span class=&quot;math-container&quot;&gt;$1.call(Unknown Source)&#xA;at java.base/java.util.concurrent.FutureTask.run(Unknown Source)&#xA;at java.desktop/javax.swing.SwingWorker.run(Unknown Source)&#xA;at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)&#xA;at java.base/java.util.concurrent.ThreadPoolExecutor$&lt;/span&gt;Worker.run(Unknown Source)&#xA;    at java.base/java.lang.Thread.run(Unknown Source)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18815" LastEditorUserId="650" LastEditDate="2024-01-11T14:05:22.383" LastActivityDate="2024-01-11T14:05:22.383" Title="Error in GSEA expresion data set file" Tags="|gsea|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22044" PostTypeId="2" ParentId="22040" CreationDate="2024-01-11T21:53:23.310" Score="1" Body="&lt;p&gt;Just put a know ligand inside the active site of your pdb , the webserver will calculate the Features of the ligand/pharmacophore then you can select the ones you want and the database where to search for molecules with the same Features. All the software used to crate a Features file seems to be not for free or really old (not sure if you can trust that code anymore) if you need some particular Features , you'll have to draw a FAKE LIGAND by yourself that carries them ... but thats its just my naive idea – –&lt;/p&gt;&#xA;&lt;p&gt;se my resul using E.coli &lt;a href=&quot;https://www.rcsb.org/structure/3LPF&quot; rel=&quot;nofollow noreferrer&quot;&gt;3LPF pdb&lt;/a&gt; carrying &lt;a href=&quot;https://www.rcsb.org/ligand/Z77&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;strong&gt;Z77&lt;/strong&gt;: 1-[(6,7-dimethyl-2-oxo-1,2-dihydroquinolin-3-yl)methyl]-1-(2-hydroxyethyl)-3-(3-methoxyphenyl)thiourea&lt;/a&gt; ligand and selecting first 4 Pharmacophire Features extracted by the webserver from it and running them against the CHEMBL32 database, first hit selected.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZzUn5.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZzUn5.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDITED&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The instructions above didnt work for me , loading &lt;code&gt;3lpf.pdb&lt;/code&gt; from my PC to the webserver for some reasons doesn't get the Z77 ligand. So the only option is to draw the Features by yourself using the Features menu, played a little bit with it but not well enough to get results , also the tricky part here would be to set the right coordinates for your ligand-Features, but overall the Tool looks very good, really fast and with a nice intuitive interface.&lt;/p&gt;&#xA;&lt;p&gt;Still would like to get ligands from PC uploaded pdb file too , so that I could test my Fake-Ligand theory.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YjrRk.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YjrRk.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;RE-EDITED&lt;/strong&gt; :&lt;/p&gt;&#xA;&lt;p&gt;found a third way :&lt;/p&gt;&#xA;&lt;p&gt;load your pdb from your pc , save the session , modify the &lt;code&gt;pharmit.json&lt;/code&gt; file you save adding the FAKE-LIGAND. To have an idea of how to put your FAKE-LIGAND inside the &lt;code&gt;json&lt;/code&gt; file , have a look at  a saved session from a session started suppliying a pdb code instead of a file. In my experiment I loaded the modified Z77 ligand stiched to a session started from a downloaded &lt;code&gt;3lpf.cif&lt;/code&gt; and saved to a &lt;code&gt;.json&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/IWkGQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/IWkGQ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;auchhh !!!!! it does load the ligand but doesn't create the Features , don't know if it is because the ligand is cut in 2 pieces or because it just doesnit work like when I uploaded a &lt;code&gt;.pdb&lt;/code&gt; containing a ligand.&lt;/p&gt;&#xA;&lt;p&gt;At least you have a way to position the Features you'll add by hands afterwards.&lt;/p&gt;&#xA;&lt;p&gt;***think that this one wont never works because the Features for the ligand are saved in the .json (first entry in the dictionary) the webserver calculates them once and then save them to be reloaded); yes it does do that , I can delete the ligand entry from .json and reload the session getting the Features but not the ligaand.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;FOURTH EDIT (last one I swear)&lt;/strong&gt; :&lt;/p&gt;&#xA;&lt;p&gt;Tried new approach: loading Features menu uploading Z77 from &lt;a href=&quot;https://www.rcsb.org/ligand/Z77&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.rcsb.org/ligand/Z77&lt;/a&gt; page, both model and ideal molecule as &lt;code&gt;.sdf&lt;/code&gt; file. The ligand gets uploaded and Features calculated by the webserver , obviously the model molecule doesn't get located in the active site of the molecule, so I guess you could draw yor FAKE ligand using something like &lt;strong&gt;PyMOL&lt;/strong&gt; exactly where you want it to be and then upload it with the load Features menu.&lt;/p&gt;&#xA;&lt;p&gt;My uploaded &lt;code&gt;Z77_ideal.sdf&lt;/code&gt; pic:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/X9DWX.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/X9DWX.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So that would provide ground for my FAKE-Ligand approach&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2024-01-11T23:02:30.440" LastActivityDate="2024-01-11T23:02:30.440" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22045" PostTypeId="2" ParentId="22025" CreationDate="2024-01-11T23:15:18.220" Score="0" Body="&lt;p&gt;It depends on many factors, both biological and technical, and it's impossible to answer in general.&lt;/p&gt;&#xA;&lt;p&gt;The starting question is which technology was used, and how many cells were used in input. For example, in a SMART-Seq experiment with one cell per well (in a multi-well plate), you should know how many cells you have (one per well). For a 10x experiment, the person running the experiment should have measured the cell concentration in the suspension before loading the chip, and targeted a particular number of cells (typically 10,000). In a combinatorial barcoding approach (e.g. Parse), you'd also have a measurement beforehand so as to load a particular number of cells per well. So, you should already have an idea of the total number of cells.&lt;/p&gt;&#xA;&lt;p&gt;Then, the first step of the analysis is typically some QC. In barcode-based approaches (including 10x), you'd make a knee-plot to estimate the actual number of sequenced cells in your sample.&lt;/p&gt;&#xA;&lt;p&gt;In the clustering and annotation steps, you'd start distinguishing and annotating groups of cells, which hopefully do correspond to cell types. The algorithms are not perfect, so the clusters don't necessarily match cell types perfectly. For example, a rare cell type will easily be lumped together with another, similar, cell type. For a well-represented cell type, the algorithms have more chance to capture subtypes or cell states.&lt;/p&gt;&#xA;&lt;p&gt;And of course there is the biology, and your knowledge of it. How many astrocytes are there in the kidney? I assume any number above 0 is worrisome. I'd expect the cell types in the data to roughly reflect the cell types in the tissue (but not exactly, there can be all kind of bias).&lt;/p&gt;&#xA;&lt;p&gt;There is a question of depth: if you have few cells in a cluster, and few reads per cell, the gene expression data in that cluster is not very reliable. The actual number depends on the technology and read depth. I would say for 10x I would prefer at least 100 cells, and would not trust a cluster with &amp;lt;10 cells.&lt;/p&gt;&#xA;&lt;p&gt;So for your question, you need to use your knowledge of the kidney and of the experimental design to decide. You can also compare to published scRNA-Seq experiments of the same tissue. You can also compare between patients to see if there is something troubling (for example, why does patient 1 have epithelial cells but no adipocytes, and opposite for patient 7? It might not be a problem, but only with context can you tell). And importantly, you do need to check that the annotation does match known gene expression patterns.&lt;/p&gt;&#xA;" OwnerUserId="9497" LastActivityDate="2024-01-11T23:15:18.220" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22046" PostTypeId="2" ParentId="22033" CreationDate="2024-01-11T23:48:17.677" Score="4" Body="&lt;p&gt;Mutect2 uses a statistical model based on the available reads. Unlike with some other tools, there is not threshold based on depth or VAF or QUAL. The main statistic that &lt;em&gt;does&lt;/em&gt; govern emission is the LOD (Log ODds) which is the log10 of the odds (&amp;quot;ratio&amp;quot;) that the variation being inspected is a tumor relative to the probability that it is a technical artifact. The default threshold for this is 3.0 (corresponding to a odds ratio of 1:1000).&lt;/p&gt;&#xA;&lt;p&gt;Perhaps you'd like to see lower probability results, in which case you should lower the threshold:&lt;/p&gt;&#xA;&lt;p&gt;Using &lt;code&gt;--tumor-lod-to-emit 2.0&lt;/code&gt; should give you many more results, many of them might be wrong. The LOD isn't well calibrated, so I'd take it with a grain of salt....&lt;strong&gt;validate&lt;/strong&gt; the results you get.&lt;/p&gt;&#xA;" OwnerUserId="9836" LastActivityDate="2024-01-11T23:48:17.677" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22047" PostTypeId="1" AcceptedAnswerId="22048" CreationDate="2024-01-12T00:00:08.783" Score="4" ViewCount="61" Body="&lt;p&gt;Does anyone know what conditions must be true for Picard to put a &lt;code&gt;?&lt;/code&gt; where I expect a number?&lt;/p&gt;&#xA;&lt;p&gt;Here is an example from the output of the tool &lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/360036804671-CollectWgsMetrics-Picard-&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;CollectWgsMetrics&lt;/code&gt;&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;❯ cat sample.wgs_metrics.txt | head -n8 | tail -n2 | verticalize | grep FOLD&#xA;FOLD_80_BASE_PENALTY    1.33476&#xA;FOLD_90_BASE_PENALTY    2.288159&#xA;FOLD_95_BASE_PENALTY    ?&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Code searching (&lt;a href=&quot;https://github.com/search?q=repo%3Abroadinstitute%2Fpicard%20%5C%22%3F%5C%22&amp;amp;type=code&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;) didn't reveal anything, and googling isn't returning anything obvious either.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;em&gt;[Verticalize is a little CLI tool I've forever been using by Pierre Lindenbaum: &lt;a href=&quot;https://github.com/lindenb/verticalize&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;verticalize&lt;/code&gt;&lt;/a&gt;. Easy to compile and I toss it in my &lt;code&gt;~/bin&lt;/code&gt; every time I get a new computer.]&lt;/em&gt;&lt;/p&gt;&#xA;" OwnerUserId="464" LastEditorUserId="73" LastEditDate="2024-01-17T09:22:54.597" LastActivityDate="2024-01-17T09:22:54.597" Title="What does a question mark (&quot;?&quot;) mean in Picard metrics files when I expect a number (integer, float, etc.)" Tags="|picard|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22048" PostTypeId="2" ParentId="22047" CreationDate="2024-01-12T00:32:13.627" Score="5" Body="&lt;p&gt;Picard (actually HtsJdk) uses a &lt;a href=&quot;https://github.com/samtools/htsjdk/blob/master/src/main/java/htsjdk/samtools/util/FormatUtil.java&quot; rel=&quot;noreferrer&quot;&gt;formatting class&lt;/a&gt; (via the &lt;a href=&quot;https://github.com/samtools/htsjdk/blob/master/src/main/java/htsjdk/samtools/metrics/MetricsFile.java&quot; rel=&quot;noreferrer&quot;&gt;MetricsFile&lt;/a&gt; class) that emits &amp;quot;?&amp;quot; for NaNs (and Infs).&lt;/p&gt;&#xA;&lt;p&gt;In your case, there seems to be a division-by-zero error during the calculation of the metric (likely due to zero coverage at some small percentage of bases in your input). This resulting Nan is emitted as a &amp;quot;?&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="9836" LastActivityDate="2024-01-12T00:32:13.627" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22049" PostTypeId="2" ParentId="22041" CreationDate="2024-01-12T15:39:30.277" Score="3" Body="&lt;p&gt;gnomAD provides a global Allele Frequency as well as AF values for most subsets. gnomAD does not provide individual genotypes (as far as I know), so calculating custom AF for non-precomputed subsets might not be possible.&lt;/p&gt;&#xA;&lt;p&gt;Since you're interested in the NFE (non-Finnish European) sub-population, the &lt;code&gt;nfe_*_AF&lt;/code&gt; fields might be of interest to you.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;As far as calculating the fraction of non-WT samples from &lt;code&gt;num_homozygous_samples&lt;/code&gt;, &lt;code&gt;AC&lt;/code&gt;, and &lt;code&gt;AN&lt;/code&gt; goes, your formula seems spot on. Here's my deconstruction of the formula:&lt;/p&gt;&#xA;&lt;p&gt;Your formula is: &lt;code&gt;(AC - num_homozygous_samples)/(AN / 2))&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;AN/2&lt;/code&gt; is the number of samples (assuming all samples are diploid). &lt;code&gt;AC - num_homo&lt;/code&gt; gives you &lt;code&gt;2 * num_hets&lt;/code&gt;. Dividing this by number of samples gives you frequency of samples with at least one ALT allele.&lt;/p&gt;&#xA;&lt;p&gt;For example, let's take a dataset of 200 diploid samples where 100 are WT, 75 are het and 25 are hom-alt for a biallelic variant.&lt;/p&gt;&#xA;&lt;p&gt;Since there are 200 diploid samples, &lt;code&gt;AN&lt;/code&gt; is 400.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;AC&lt;/code&gt; for the ALT allele is &lt;code&gt;(75 * 1) + (25 * 2) = 125&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Since 100 are WT, it is known that fraction of samples with mutation = &lt;code&gt;1 - (100/200) = 0.5&lt;/code&gt;. This number can be arrived using your formula:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(AC - num_homozygous_samples) / (AN/2)&#xA; = ((125 - 25) / (400/2))&#xA; = 100 / 200&#xA; = 0.5&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastActivityDate="2024-01-12T15:39:30.277" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22050" PostTypeId="1" CreationDate="2024-01-12T16:41:13.987" Score="1" ViewCount="49" Body="&lt;p&gt;This is an add-on to the previous question I asked &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/22005/process-gets-killed-before-completing/22006?noredirect=1#comment34017_22006&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I was able to deploy a &amp;quot;chunking&amp;quot; strategy to my kmer matrix and was able to get a file with the distance measurements! From what I could tell by using the &lt;code&gt;head&lt;/code&gt; command was that the data looks like what it did before when I was using this approach on a smaller dataset so now I'm going to use this data to generate a tree using MEGAX.&lt;/p&gt;&#xA;&lt;p&gt;Normally when I'd convert the &lt;code&gt;tsv&lt;/code&gt; file to a &lt;code&gt;meg&lt;/code&gt; file I'd manually add in the meg headers/requirements for conversion. However, I can't open the distance &lt;code&gt;tsv&lt;/code&gt; file (its ~23 GB) to make these changes.&lt;/p&gt;&#xA;&lt;p&gt;Is there an easy solution to this problem? How do you convert a &lt;code&gt;tsv&lt;/code&gt; file to a &lt;code&gt;meg&lt;/code&gt; file without opening the file?&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&lt;p&gt;For more context here is the format for each file.&lt;/p&gt;&#xA;&lt;p&gt;Currently, my &lt;code&gt;tsv&lt;/code&gt; file looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;0 1 2 3 4 5 6 7 8 9 10 ... 178&#xA;1 0 # # # # # # # # #  ... #&#xA;2 # 0 # # # # # # # #  ... #&#xA;3 # # 0 # # # # # # #  ... #&#xA;4 # # # 0 # # # # # #  ... #&#xA;5 # # # # 0 # # # # #  ... #&#xA;6 # # # # # 0 # # # #  ... #&#xA;7 # # # # # # 0 # # #  ... #&#xA;8 # # # # # # # 0 # #  ... #&#xA;9 # # # # # # # # 0 #  ... #&#xA;10 # # # # # # # # # 0 ... #&#xA;...&#xA;178 # # # # # # # # # #  ... 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is a jaccard distance matrix of 178 samples. The &lt;code&gt;#&lt;/code&gt; represent the distance values for each sample compared to the other. I was able to calcualte the distance using pandas and scipy: &lt;code&gt;matrix = pd.DataFrame(squareform(pdist(data, 'jaccard')))&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;And I would like it to look like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#mega&#xA;TITLE: Jaccard distance of 178 samples&#xA;&#xA;sample_1&#xA;sample_2&#xA;sample_3&#xA;...&#xA;sample_178&#xA;&#xA;distance_1&#xA;distance_1 distance_2&#xA;distance_1 distance_2 distance_3&#xA;distance_1 distance_2 distance_3 distance_4&#xA;...&#xA;distance_1 distance_2 distance_3 distance_4...distance_178&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16067" LastEditorUserId="16067" LastEditDate="2024-01-12T20:46:35.760" LastActivityDate="2024-01-15T10:41:51.923" Title="Converting .tsv to .meg for visualization" Tags="|format-conversion|k-mer|megax|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22051" PostTypeId="1" CreationDate="2024-01-12T17:57:36.983" Score="0" ViewCount="35" Body="&lt;p&gt;I'd like a tabix file that, for every position in the genome, gives me a list of every ENSP within 5kb of that position.&lt;/p&gt;&#xA;&lt;p&gt;This should be easy right... I know VEP /kinda/ gives me this, but I'd rather not run it for every postition in the genome &lt;/p&gt;&#xA;&lt;p&gt;Can I use biomart to get the start/end position for every ENSP, then bedtools to somehow expand those coords?&lt;/p&gt;&#xA;" OwnerUserId="18817" LastActivityDate="2024-02-15T15:34:03.310" Title="Chromosome position -&gt; ENSP" Tags="|ensembl|bed|bedtools|biomart|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22052" PostTypeId="2" ParentId="22050" CreationDate="2024-01-13T04:07:01.783" Score="1" Body="&lt;p&gt;Note, its a while since I've used &lt;code&gt;scipy&lt;/code&gt; and I don't fancy loading an deleting for this test ... so in the absence of &lt;code&gt;pdist()&lt;/code&gt; the solution would be ...&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np&#xA;import pandas as pd&#xA;from pathlib import Path&#xA;&#xA;data = [[1,2,3],[0,5,6],[7,0,9],[10,11,0]]&#xA;arr = np.tril([data]).astype('float')&#xA;arr[arr == 0] = 'nan' &#xA;arr = np.where(np.isnan(arr), '', arr)&#xA;columns = ['a','b','c', 'd']&#xA;file = Path('/Volumes/Data2/test/test1.meg')&#xA;if file.is_file():&#xA;    file.unlink()&#xA;with open(file, 'a') as fout:&#xA;    fout.write(&amp;quot;#MEGA\n&amp;quot;)&#xA;    for col in columns:&#xA;        fout.write(col + &amp;quot;\n&amp;quot;)&#xA;    fout.write(&amp;quot;\n0\n&amp;quot;)&#xA;    np.savetxt(fout, arr.reshape(3,-1), fmt='%s')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;output&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#MEGA&#xA;a&#xA;b&#xA;c&#xA;d&#xA;&#xA;0&#xA;1.0 0.0&#xA;5.0 7.0 0.0&#xA;9.0 10.0 11.0 0.0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That works. Keep in mind where it says &lt;code&gt;arr.reshape(3,-1)&lt;/code&gt; the &lt;code&gt;3&lt;/code&gt; should be the number of columns -1 in your matrix: &lt;strong&gt;in your case this appears to be 177 (178 - 1)&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;The gist is to use &lt;code&gt;np.tril&lt;/code&gt;, which doesn't exists in &lt;code&gt;pandas&lt;/code&gt;. Hence don't use &lt;code&gt;pandas&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;At a guess with &lt;code&gt;scipy&lt;/code&gt; then it would be ...&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np&#xA;import pandas as pd&#xA;from pathlib import Path&#xA;from scipy import pdist&#xA;&#xA;tmp = [[1,2,3],[4,5,6],[7,8,9],[10,11,12]]&#xA;&#xA;data = pdist(tmp, 'jaccard')&#xA;arr = np.tril([data]).astype('float')&#xA;arr[arr == 0] = 'nan' &#xA;arr = np.where(np.isnan(arr), '', arr)&#xA;columns = ['a','b','c', 'd']&#xA;file =Path('/Volumes/Data2/test/test1.meg')&#xA;if file.is_file():&#xA;    file.unlink()&#xA;with open(file, 'a') as fout:&#xA;    fout.write(&amp;quot;#MEGA\n&amp;quot;)&#xA;    for col in columns:&#xA;        fout.write(col + &amp;quot;\n&amp;quot;)&#xA;    fout.write(&amp;quot;\n0\n&amp;quot;)&#xA;    np.savetxt(fout, arr.reshape(3,-1), fmt='%s')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You might need a quick &lt;code&gt;perl -p -i -e 's/[&amp;quot; &amp;quot;]+/ /g' test1.meg&lt;/code&gt; or even &lt;code&gt;perl -p -i -e 's/[&amp;quot; &amp;quot;]+/\t/g' test1.meg&lt;/code&gt; to set the file for MEGA. I dunno why but &lt;code&gt;\s&lt;/code&gt; or &lt;code&gt;&amp;quot; &amp;quot;&lt;/code&gt; in the transform part of perl pie isn't working for me.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="298" LastEditDate="2024-01-15T10:41:51.923" LastActivityDate="2024-01-15T10:41:51.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22053" PostTypeId="1" CreationDate="2024-01-13T12:05:46.940" Score="1" ViewCount="18" Body="&lt;p&gt;Can you help me understand why I am getting different values when I calculate the segregating sites from tskit api versus manually counting? Thanks!&lt;/p&gt;&#xA;&lt;p&gt;Example data generation:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import sys, tskit, msprime&#xA;&#xA;ts = msprime.sim_ancestry(samples=3, ploidy=2, sequence_length=10, random_seed=2)&#xA;ts = msprime.sim_mutations(ts, rate=0.1, random_seed=2)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Example results: I get 5 from API and 6 from manual counting.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;&amp;gt;&amp;gt; ts.segregating_sites(span_normalise=False)&#xA;5.0&#xA;&amp;gt;&amp;gt;&amp;gt; n=0&#xA;&amp;gt;&amp;gt;&amp;gt; for tree in ts.trees():&#xA;...     for site in tree.sites():&#xA;...         for mutation in site.mutations:&#xA;...             node = ts.node(mutation.node)&#xA;...             pop_id = node.population&#xA;...             n+=1&#xA;... &#xA;&amp;gt;&amp;gt;&amp;gt; n&#xA;6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="11938" LastEditorUserId="11938" LastEditDate="2024-01-14T08:55:31.670" LastActivityDate="2024-01-14T08:55:31.670" Title="Calculating the segregating sites via tskit versus counting" Tags="|python|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22054" PostTypeId="1" CreationDate="2024-01-13T16:26:05.610" Score="3" ViewCount="22" Body="&lt;p&gt;I am working on Protein Secondary Structure Prediction using Recurrent Neural Network (GRU) model. I came across few open source projects which already worked on the similar problem. All of them are using Q8 Accuracy as evaluation metrics and the custom code they used for calculating Q8 accuracy is given below.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;def accuracy(y_true, y_pred):&#xA;  y = tf.argmax(y_true, axis =- 1)&#xA;  y_ = tf.argmax(y_pred, axis =- 1)&#xA;  mask = tf.greater(y, 0)&#xA;  return K.cast(K.equal(tf.boolean_mask(y, mask), tf.boolean_mask(y_, mask)), K.floatx())&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to know the mathematical expression of Q8 accuracy corresponding to the above code snippet. The above code source project reference &lt;a href=&quot;https://github.com/idrori/cu-ssp&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/idrori/cu-ssp&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18823" LastEditorUserId="650" LastEditDate="2024-01-14T20:04:49.850" LastActivityDate="2024-01-14T20:04:49.850" Title="Q8 Accuracy Evaluation metric mathematical expression used in Protein Secondary Structure Prediction" Tags="|python|proteins|protein-structure|differential-expression|deep-learning|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22057" PostTypeId="1" CreationDate="2024-01-16T14:46:49.897" Score="1" ViewCount="14" Body="&lt;p&gt;Hey fellow researchers,&lt;/p&gt;&#xA;&lt;p&gt;I recently conducted an analysis on the UniProt database, specifically focusing on entries with MlaD domains. After downloading the data and opening it in an Excel file, I noticed a significant discrepancy with the AlphaFold predictions.&lt;/p&gt;&#xA;&lt;p&gt;For instance, entry &amp;quot;A0A336N986&amp;quot; in UniProt indicates only 1 MlaD domain (link: &lt;a href=&quot;https://www.uniprot.org/uniprotkb/A0A336N986/entry#sequences&quot; rel=&quot;nofollow noreferrer&quot;&gt;UniProt Entry&lt;/a&gt;), but when cross-referencing with AlphaFold (link: &lt;a href=&quot;https://alphafold.ebi.ac.uk/entry/A0A336N986&quot; rel=&quot;nofollow noreferrer&quot;&gt;AlphaFold Entry&lt;/a&gt;), it reveals 5 MlaD domains with high confidence values.&lt;/p&gt;&#xA;&lt;p&gt;Considering the large dataset with over 100k entries, manually sorting them based on the accurate count of MlaD domains seems impractical. I'm reaching out to the community for guidance on automating this process. Has anyone encountered a similar issue, or does anyone have suggestions on tools, scripts, or strategies to efficiently sort UniProt entries based on the correct number of MlaD domains?&lt;/p&gt;&#xA;&lt;p&gt;Any insights or advice would be greatly appreciated! Thanks in advance.&lt;/p&gt;&#xA;" OwnerUserId="18832" LastActivityDate="2024-01-16T14:46:49.897" Title="Discrepancy between UniProt and AlphaFold data for MlaD domains – Seeking guidance on automating entry sorting based on accurate domain count" Tags="|uniprot|alphafold|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22058" PostTypeId="1" AcceptedAnswerId="22073" CreationDate="2024-01-17T11:08:43.600" Score="1" ViewCount="31" Body="&lt;p&gt;In UK biobank research environment&lt;/p&gt;&#xA;&lt;p&gt;In GENOMICS tab of UK BRAP, if I type &lt;code&gt;7:140476600-140480852&lt;/code&gt; coordinate in search bar and the same time after filtering my cohort for this coordinate, in &lt;code&gt;ALLELE TABLE&lt;/code&gt; I see columns like:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Cohort AF Cohort Allele Count Population AF Population Allele Count GnomAD AF&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/OH3UX.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/OH3UX.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I Googled a lot but no clue where population allele count and frequency come from&lt;/p&gt;&#xA;&lt;p&gt;Does anyone have any experience with this?&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2024-01-17T12:48:16.020" LastActivityDate="2024-01-20T21:33:23.680" Title="Population Allele Frequency" Tags="|allele-frequency|population-genetics|brap|dnanexus|biobank|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22059" PostTypeId="1" CreationDate="2024-01-17T15:41:47.670" Score="0" ViewCount="27" Body="&lt;p&gt;We are working with a large dataset of proteins for folding/docking tests across various tools (AlphaFold, ESM2, RoseTTAFold). In some of the FASTA files, there is an X for a non-standard residue. An example of this is in the 6SEN_1 FASTA file below. Is there a standard method of dealing with these non-standard residues prior to analysis to minimize impact on the folding/docking results?&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;6SEN_1|Chains A, B|Transcriptional enhancer factor TEF-3|Homo sapiens (9606)&#xA;GPRSVASSKLWMLEFSAFLEQQQDPDTYNKHLFVHIGQSSPSYSDPYLEAVDIRQIYDKFPEKKGGLKDLFERGPSNAFFLVKFWADLNTNIEDEGSSFYGVSSQYESPENMIITCSTKVCSFGKQVVE**X**VETEYARYENGHYSYRIHRSPLCEYMINFIHKLKHLPEKYMMNSVLENFTILQVVTNRDTQETLLCIAYVFEVSASEHGAQHHIYRLVKE&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18836" LastEditorUserId="650" LastEditDate="2024-01-17T16:30:40.237" LastActivityDate="2024-03-02T19:07:10.937" Title="Dealing with X Residues in FASTA files prior to folding analysis" Tags="|fasta|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22060" PostTypeId="2" ParentId="6694" CreationDate="2024-01-18T00:21:20.960" Score="1" Body="&lt;p&gt;I am unable to respond to other's comments as I never usually comment, so this is a new account. However, I spent the last couple days really diving into the BAM alignments when mapping scRNAseq reads to the human genome, but altering the GTF and FASTA using different annotations of my CAR construct (some very liberal allowing upstream and to the poly A tail sequence, and some less liberal)&lt;/p&gt;&#xA;&lt;p&gt;MY MAIN CONCLUSION!:&lt;/p&gt;&#xA;&lt;p&gt;DO NOT ADD THE BGH POLY A SIGNAL TO THE GENE ANNOTATION! it is highly homologous to the human beta globin subunit, and I am finding almost none of the reads actually mapping both to the adjacent CAR construct and the BGH POLY A subunit.&lt;/p&gt;&#xA;&lt;p&gt;I am still figuring out how to handle the CD3zeta, CD28, and the OX40 in the CAR construct being multi-mapped (I will test just using a small portion of the CD28 transmembrane domain and then using just the chimeric antigen portion for mapping)&lt;/p&gt;&#xA;&lt;p&gt;BUT, BIGEST TAKEAWAY, if you are going to attempt to add the poly A tail or any other genomic annotation not strictly relating to your gene of interest look at the bam and ensure that the reads mapped actually bridges into your gene. (as I also had residual genomic DNA from human beta globin upstream of my CAR construct that showed splicing over the CAR construct and aligning downstream of the CAR to the POLY A tail sequence)&lt;/p&gt;&#xA;&lt;p&gt;-for viewing the BAM i am using IGVtools, and for the creation of the GTF and the FASTAs that I concatenated to the human reference genome I used tears [not a software package] ;)&lt;/p&gt;&#xA;" OwnerUserId="18837" LastEditorUserId="18837" LastEditDate="2024-01-18T00:22:31.497" LastActivityDate="2024-01-18T00:22:31.497" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22061" PostTypeId="1" CreationDate="2024-01-18T07:40:31.047" Score="1" ViewCount="34" Body="&lt;p&gt;I want to find my copy number variants. I got DNA testing from the Nebula Genomics and received the raw datas in .cram, .fastq, .vcf formats. Are there any simple and easy method to call my copy number variants from one of these files? I tried GATK, but I was unable to collect Read Counts from 1000 genomes datas.&lt;/p&gt;&#xA;" OwnerUserId="18744" LastEditorUserId="3694" LastEditDate="2024-01-19T14:09:55.363" LastActivityDate="2024-02-18T15:01:25.197" Title="Are there any simple ways to call copy number variants from a Nebula Genomics WGS data?" Tags="|wgs|cnv|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22062" PostTypeId="1" AcceptedAnswerId="22063" CreationDate="2024-01-18T14:38:28.373" Score="2" ViewCount="44" Body="&lt;p&gt;I have an RNA-Seq and scRNA-Seq dataset from dog samples and pseudo-aligned them to the dog reference transcriptome (Ensembl release 109). When filtering for mitochondrial gene percentage in scRNA-Seq I couldn't find the &amp;quot;MT&amp;quot; annotation in this new release. Does anyone have any information about this? Did they not include it or am I doing something wrong somewhere (the MT annotations are present in the CanFam3.1 genome built but not in the newer ones).&#xA;I could not find anything on the Ensembl release details or blog.&lt;/p&gt;&#xA;&lt;p&gt;Thank you :)&lt;/p&gt;&#xA;&lt;p&gt;This is a part of the code that I used to search for the mitochondrial genes:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;searchFilterOptions(mart=ensembl_canis, filter=&amp;quot;chromosome_name&amp;quot;, pattern=&amp;quot;MT&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14575" LastActivityDate="2024-01-18T14:56:27.050" Title="Dog (Canis lupus familiaris) MT gene annotations" Tags="|sequence-annotation|ensembl|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22063" PostTypeId="2" ParentId="22062" CreationDate="2024-01-18T14:56:27.050" Score="3" Body="&lt;p&gt;Currently, there is no available mitochondrial sequence for the ROS_Cfam_1.0 assembly:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/datasets/genome/GCF_014441545.1/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/datasets/genome/GCF_014441545.1/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6500" LastActivityDate="2024-01-18T14:56:27.050" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22065" PostTypeId="1" CreationDate="2024-01-18T17:45:14.440" Score="0" ViewCount="17" Body="&lt;p&gt;I'm working with two sets of data. One of two sets of data, one contains the results of a RNAseq analysis and the other contains the result of TTseq analysis. I want to check which genes overlap in both data sets to check if the are affected at stabilization level or at a transcriptional level. If anyone knows how the coding pipeline it would help me. I work on linux.&lt;/p&gt;&#xA;" OwnerUserId="18842" LastActivityDate="2024-01-18T17:45:14.440" Title="RNAseq and TTseq results" Tags="|rna-seq|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22066" PostTypeId="2" ParentId="22032" CreationDate="2024-01-18T19:52:32.593" Score="1" Body="&lt;p&gt;I ultimately addressed this issue using the &lt;a href=&quot;https://github.com/DerrickWood/kraken2&quot; rel=&quot;nofollow noreferrer&quot;&gt;kraken2 software&lt;/a&gt;. I found this approach to rely on a good deal of my server's memory, but it was sufficient for doing a viral/bacterial classification of sequences.&lt;/p&gt;&#xA;" OwnerUserId="14041" LastActivityDate="2024-01-18T19:52:32.593" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22067" PostTypeId="2" ParentId="21143" CreationDate="2024-01-19T00:45:55.820" Score="2" Body="&lt;p&gt;If you truly &amp;quot;just need the end result to be readable by Histech readers&amp;quot; the simplest (and maybe even cheapest in the long run) solution might be a paid ($) license for CaseViewer (or whatever their newer viewer is called). I have no idea what that would cost but many of these vendor wsi viewers can read and even write other formats for a price.&lt;/p&gt;&#xA;" OwnerUserId="18844" LastEditorUserId="18844" LastEditDate="2024-01-19T00:46:18.323" LastActivityDate="2024-01-19T00:46:18.323" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22069" PostTypeId="2" ParentId="22061" CreationDate="2024-01-19T13:16:05.073" Score="0" Body="&lt;p&gt;Yes and no. Yes, there are tools that can do this, but no, they do not work very well. Short read NGS sequencing just isn't the right tool to find large structural variants like CNVs. As a rule of thumb, you can expect around 30% accuracy when compared to other experimental approaches such as &lt;a href=&quot;https://en.wikipedia.org/wiki/Multiplex_ligation-dependent_probe_amplification&quot; rel=&quot;nofollow noreferrer&quot;&gt;MLPA&lt;/a&gt; or &lt;a href=&quot;https://en.wikipedia.org/wiki/Comparative_genomic_hybridization#Array_comparative_genomic_hybridization&quot; rel=&quot;nofollow noreferrer&quot;&gt;array CHG&lt;/a&gt;. That said, yes, there are indeed various tools that can do this. My personal favorite is &lt;a href=&quot;https://github.com/dellytools/delly&quot; rel=&quot;nofollow noreferrer&quot;&gt;delly&lt;/a&gt; because it is accurate, actively maintained and since it is written in C it is quite fast.&lt;/p&gt;&#xA;&lt;p&gt;Once you have downloaded and installed it, you'll likely need to convert your cram to bam as I don't think it can deal with cram:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;samtools view -b foo.cram &amp;gt; foo.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then you can run delly on it with (assuming your cram was aligned against hg38):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;delly call -x path/to/delly/excludeTemplates/human.hg38.excl.tsv \&#xA;   -g path/to/hg38.fa -o out.bcf foo.bam&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You will also need to download a copy of the human genome fasta file that Nebula used to generate your cram. You can ask them where to get one, and change &lt;code&gt;path/to/hg38.fa&lt;/code&gt; with the path to that file.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2024-01-19T13:16:05.073" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22070" PostTypeId="1" CreationDate="2024-01-20T11:05:45.293" Score="1" ViewCount="24" Body="&lt;p&gt;The rule is written for calling genotypes using array analysis cli for illumina gsa chips.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import configparser&#xA;config=configparser.ConfigParser()&#xA;config.read(&amp;quot;.//config&amp;quot;)&#xA;homepath=config.get('Paths','home_dir')&#xA;inputpath=config.get('Paths','input_dir')&#xA;outputpath=config.get('Paths','output_dir')&#xA;Sentrix_ids=config.items('Sentrix_ids')&#xA;sentrixids=[]&#xA;positions=['R01C01','R01C02',&#xA;                        'R02C01','R02C02',&#xA;                        'R03C01','R03C02',&#xA;                        'R04C01','R04C02',&#xA;                        'R05C01','R05C02',&#xA;                        'R06C01','R06C02',&#xA;                        'R07C01','R07C02',&#xA;                        'R08C01','R08C02',&#xA;                        'R09C01','R09C02',&#xA;                        'R10C01','R10C02',&#xA;                        'R11C01','R11C02',&#xA;                        'R12C01','R12C02']&#xA;&#xA;rule all:&#xA;        input:&#xA;                expand(outputpath+&amp;quot;/{sids}_{pos}.gtc&amp;quot;,sids=sentrixids,pos=positions)&#xA;&#xA;rule idat2vcf:&#xA;        input:&#xA;                idatpath=homepath+&amp;quot;/ss.csv&amp;quot;,&#xA;                clusterfile=homepath+&amp;quot;/input/&amp;quot;+&amp;quot;GSA-24v3-0_A1_ClusterFile.egt&amp;quot;,&#xA;          manifestfile=homepath+&amp;quot;/input/&amp;quot;+&amp;quot;GSA-24v3-0_A1.bpm&amp;quot;&#xA;&#xA;        output:&#xA;                outputfol=outputpath+&amp;quot;/&amp;quot;+&amp;quot;gtcfiles&amp;quot;&#xA;        shell:&#xA;                &amp;quot;bash /scripts/idattogtc.sh {input.idatpath} {input.clusterfile} {input.manifestfile} {output.outputfol}&amp;quot;&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/bash&#xA;exec_path=&amp;quot;/home/user14/Downloads/array-analysis-cli-linux-x64-v2.1.0/array-&#xA;analysis-cli/array-analysis-cli&amp;quot;&#xA;idat_path=$1&#xA;cluster_file=$2&#xA;manifest_file=$3&#xA;output_fol=$4&#xA;${exec_path} genotype call \&#xA;  --bpm-manifest ${manifest_file} \&#xA;  --cluster-file ${cluster_file} \&#xA;  --idat-sample-sheet ${idat_path} \&#xA;  --output-folder ${output_fol}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I run snakemake, it gives the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Traceback (most recent call last):&#xA;  File &amp;quot;/home/user14/miniconda3/envs/snakemake/lib/python3.6/site-packages/snakemake/__init__.py&amp;quot;, line 469, in snakemake&#xA;    force_use_threads=use_threads)&#xA;  File &amp;quot;/home/user14/miniconda3/envs/snakemake/lib/python3.6/site-packages/snakemake/workflow.py&amp;quot;, line 447, in execute&#xA;    self.persistence.cleanup_shadow()&#xA;  File &amp;quot;/home/user14/miniconda3/envs/snakemake/lib/python3.6/site-packages/snakemake/persistence.py&amp;quot;, line 128, in cleanup_shadow&#xA;    shutil.rmtree(self.shadow_path)&#xA;  File &amp;quot;/home/user14/miniconda3/envs/snakemake/lib/python3.6/shutil.py&amp;quot;, line 490, in rmtree&#xA;    onerror(os.rmdir, path, sys.exc_info())&#xA;  File &amp;quot;/home/user14/miniconda3/envs/snakemake/lib/python3.6/shutil.py&amp;quot;, line 488, in rmtree&#xA;    os.rmdir(path)&#xA;OSError: [Errno 26] Text file busy: '/media/sf_vm_centfiles/workflow/.snakemake/shadow'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18579" LastEditorUserId="650" LastEditDate="2024-01-21T15:12:38.157" LastActivityDate="2024-01-21T15:12:38.157" Title="OsError in virtual box snakemake pipeline for array-analysis" Tags="|linux|snakemake|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22071" PostTypeId="2" ParentId="22000" CreationDate="2024-01-20T12:32:44.523" Score="0" Body="&lt;p&gt;As others have said it's not very clear what you're asking here, but I'm assuming you want to know how to determine relevant genes that were enriched in your IP.&lt;/p&gt;&#xA;&lt;p&gt;As per the group behind eCLIP, &lt;a href=&quot;https://genomebiology.biomedcentral.com/articles/10.1186/s13059-020-01982-9&quot; rel=&quot;nofollow noreferrer&quot;&gt;Van Nostrand et al. 2020&lt;/a&gt;&#xA;: &amp;quot;standard peak analysis used the set of peaks identified as irreproducible discovery rate (IDR) reproducible and meeting fold-enrichment (≥ 8-fold) and significance (p value ≤10−3) in immunoprecipitation versus paired size-matched input.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;If those cutoffs are too stringent you could change them to fit the context of your project. And what do you mean by differential expression? It sounds like you're addressing binding.&lt;/p&gt;&#xA;" OwnerUserId="4442" LastActivityDate="2024-01-20T12:32:44.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22072" PostTypeId="1" AcceptedAnswerId="22075" CreationDate="2024-01-20T20:44:54.850" Score="2" ViewCount="78" Body="&lt;p&gt;I was surprised as to how hard it was to find a reference on this among the docs.&lt;/p&gt;&#xA;&lt;p&gt;Does the Protein Data Bank &lt;a href=&quot;https://mmcif.wwpdb.org/&quot; rel=&quot;nofollow noreferrer&quot;&gt;mmcif/pdb-x&lt;/a&gt; archival format use Angstrom to specify cartesian x,y,z coordinates of atoms or something else or abstract units (perhaps defined per each &lt;code&gt;mmcif&lt;/code&gt; &lt;code&gt;_data&lt;/code&gt; frame?&lt;/p&gt;&#xA;&lt;p&gt;How is the origin of the grid calculated -- for crystallographers out there?&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="6511" LastEditorUserId="73" LastEditDate="2024-01-23T11:22:24.027" LastActivityDate="2024-01-23T11:22:24.027" Title="Does the `mmcif/pdb` format store atomic coordinates in units of Ångstrom, nanometers or something else?" Tags="|pdb|3d-structure|distance|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22073" PostTypeId="2" ParentId="22058" CreationDate="2024-01-20T21:33:23.680" Score="3" Body="&lt;p&gt;I don't know specifically about the UKBB RAP, however, Population Allele Count is the count of that specific allele genotyped in the population in question. In this case, it most likely refers to the population that your cohort is part of. The population allele frequency is the same thing but noted in terms of frequency, i.e. divided by 2 times the number of genotyped individuals in that population.&lt;/p&gt;&#xA;" OwnerUserId="9836" LastActivityDate="2024-01-20T21:33:23.680" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22074" PostTypeId="1" CreationDate="2024-01-21T01:28:26.430" Score="1" ViewCount="52" Body="&lt;p&gt;I am searching for specific transcription factor binding locations on DNA, and ranking them according the their scores.&lt;/p&gt;&#xA;&lt;p&gt;For this, I am in search of a tool or method that can generate motif scores for arbitrary-length DNA sequences (denoted as 'x') using a scoring function Motif_score(x, Motif_M_PWM). The purpose is to calculate motif scores in a moving window fashion along the entire DNA sequence. The 'Motif_M_PWM' parameter typically represents the probability weight matrix (PWM) associated with a specific motif 'M.'&lt;/p&gt;&#xA;&lt;p&gt;One common method to calculate the motif score for a given short sequence 'x' (which has the same length, n, as the motif itself) is by using the formula:&lt;/p&gt;&#xA;&lt;p&gt;&lt;span class=&quot;math-container&quot;&gt;$motif\_score = \sum_{k=1}^{n} \log\frac{PWM\_Probability(x(k), k)} {0.25}$&lt;/span&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this formula, 'x(k)' represents a specific nucleotide in sequence 'x,' and 'k' denotes the position of the nucleotide in the  PWM and x. The '0.25' term accounts for the random chance of any one of the four nucleotides (A, C, G, T) occurring. I think this scoring method is commonly used by tools like HOMER.&lt;/p&gt;&#xA;&lt;p&gt;I am curious to know if there are alternative scoring functions or methods (tools besides HOMER) available for calculating motif scores on DNA sequences.&lt;/p&gt;&#xA;" OwnerUserId="15741" LastEditorUserId="73" LastEditDate="2024-01-21T23:03:15.060" LastActivityDate="2024-01-21T23:03:15.060" Title="Recommendations on Motif scoring functions" Tags="|sequence-analysis|motifs|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22075" PostTypeId="2" ParentId="22072" CreationDate="2024-01-21T12:48:34.247" Score="4" Body="&lt;p&gt;From google I got this as first hit :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://cdn.rcsb.org/wwpdb/docs/documentation/file-format/PDB_format_1992.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB_format_1992.pdf&lt;/a&gt; :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nyNmb.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nyNmb.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Hopefully they keept the same Unit of measurement during the data/file format evolution/extension&lt;/p&gt;&#xA;&lt;p&gt;It seems the case for &lt;code&gt;mmcif&lt;/code&gt; &lt;a href=&quot;https://mmcif.wwpdb.org/dictionaries/mmcif_pdbx_v40.dic/Items/_atom_site.Cartn_x.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;Data Item _atom_site.Cartn_x&lt;/a&gt; :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/ZloIU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/ZloIU.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Some history here :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://journals.iucr.org/a/issues/2008/01/00/sc5004/sc5004.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Protein Data Bank: a historical perspective&lt;/a&gt; , &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC102472/pdf/gkd090.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;The Protein Data Bank&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;AND :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;PY  - 1971&#xA;DA  - 1971/10/01&#xA;TI  - Crystallography: Protein Data Bank&#xA;JO  - Nature New Biology&#xA;SP  - 223&#xA;EP  - 223&#xA;VL  - 233&#xA;IS  - 42&#xA;SN  - 2058-1092&#xA;UR  - https://doi.org/10.1038/newbio233223b0&#xA;DO  - 10.1038/newbio233223b0&#xA;ID  - ref1&#xA;ER  - &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/P8nmu.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/P8nmu.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;About the axis origin CHARMM-GUI has a nice introductory lesson to PDB format (see &lt;em&gt;&lt;strong&gt;CRYTST1 Record&lt;/strong&gt;&lt;/em&gt; Section) &lt;a href=&quot;https://charmm-gui.org/?doc=lecture&amp;amp;module=pdb&amp;amp;lesson=1&quot; rel=&quot;nofollow noreferrer&quot;&gt;Lesson 1: Introduction to PDB Format&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;and stolen from the web, free part of an old book :&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/4peth.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/4peth.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="10049" LastEditDate="2024-01-21T15:16:13.407" LastActivityDate="2024-01-21T15:16:13.407" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22076" PostTypeId="1" AcceptedAnswerId="22077" CreationDate="2024-01-22T16:30:20.410" Score="-1" ViewCount="25" Body="&lt;p&gt;I have several splitted fasta files like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TK273759.fastaaa&#xA;TK273759.fastaab&#xA;TK273759.fastaac&#xA;TK273759.fastaad&#xA;TK273759.fastaae&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I want to concatenate these fasta files into one file like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TK273759.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;What command should I use to make this happen? I appreciate your help.&lt;/p&gt;&#xA;" OwnerUserId="18863" LastEditorUserId="650" LastEditDate="2024-01-22T16:45:04.863" LastActivityDate="2024-01-22T16:45:04.863" Title="Concatenate multiple fasta files into individual files per unique entry" Tags="|genome|bash|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22077" PostTypeId="2" ParentId="22076" CreationDate="2024-01-22T16:44:50.210" Score="0" Body="&lt;p&gt;Please try searching online before asking for help. A simple &lt;code&gt;cat&lt;/code&gt; will do the trick here:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat TK273759.fasta[a-z][a-z] &amp;gt; TK273759.fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="650" LastActivityDate="2024-01-22T16:44:50.210" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22079" PostTypeId="1" CreationDate="2024-01-23T18:20:12.033" Score="0" ViewCount="17" Body="&lt;p&gt;I am using the MEGA software for trimming protein sequences and building phylogenetic trees.&lt;/p&gt;&#xA;&lt;p&gt;How can I define out-groups of phylogenetic tree in MEGA? Is there a way to calculate the distances of each species from the root?&lt;/p&gt;&#xA;" OwnerUserId="18868" LastEditorUserId="73" LastEditDate="2024-01-24T18:14:34.877" LastActivityDate="2024-02-23T19:09:36.227" Title="Defining out-groups in MEGA" Tags="|phylogenetics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22080" PostTypeId="2" ParentId="22079" CreationDate="2024-01-23T18:40:06.803" Score="0" Body="&lt;p&gt;Hmmm.... the issue is that the precise location of the root - unless it's a molecular clock - is arbitrarily assigned within limits. You can designate populations ('clades') and ask for the genetic distance between the clades, it's one of the menus header options. To do what you want you need to build the tree (neighbor-joining method) and assess it in the pop-up window. Switch the view of the tree to 'genetic distance' and then manually add the branch lengths for each root - tip distance. One of the Beast supporting packages use to calculate the root tip distance, I think this option was removed in the upgrades.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2024-01-23T18:40:06.803" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22082" PostTypeId="1" CreationDate="2024-01-23T21:58:01.160" Score="1" ViewCount="15" Body="&lt;p&gt;I am trying to get the ipdRatio and modified bases information from a PacBio bam file which is sorted and indexed. The command I'm running is:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ipdSummary sorted_chr21_0_100000.bam --reference $REF_path/$REF_FILE_NAME.fasta --referenceWindow 20:0-100000 --identify m6A,m4C,m5C_TET --methylFraction --outfile sorted_chr21_0_100000;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get the following warning message:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;RuntimeWarning: Non-unique read group integer IDs found - some features may be restricted warnings.warn(&amp;quot;Non-unique read group integer IDs found - some features may be restricted&amp;quot;, RuntimeWarning)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My question is which types of features might be restricted if my read groups are not unique?&lt;/p&gt;&#xA;&lt;p&gt;I am not sure how critical it could be.&lt;/p&gt;&#xA;" OwnerUserId="15341" LastEditorUserId="650" LastEditDate="2024-01-23T22:06:21.380" LastActivityDate="2024-01-23T22:06:21.380" Title="How critical is the &quot;Non-unique read group integer IDs found&quot; warning in running ipdSummary?" Tags="|pacbio|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22083" PostTypeId="1" CreationDate="2024-01-24T01:36:26.470" Score="2" ViewCount="25" Body="&lt;p&gt;Hello StackExchange Community,&lt;/p&gt;&#xA;&lt;p&gt;I am currently facing issues with running the Variant Effect Predictor (VEP) and vcf2maf in a Conda environment due to a Perl-related error.&lt;/p&gt;&#xA;&lt;p&gt;Installation Steps: I installed VEP and vcf2maf using Conda with the following commands, and everything installed without any problems:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda create -n vcf2maf -y python=3.8&#xA;conda activate vcf2maf &#xA;conda install -y  -c bioconda vcf2maf &#xA;conda install  -y  -c bioconda ensembl-vep &#xA;conda install  -y   -c bioconda samtools &#xA;vcf2maf.pl  --help &#xA;vep --help  &#xA;samtools -v &#xA;perl -e '{print join&amp;quot;\n&amp;quot;,@INC}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Cache Installation: I then installed the cache for VEP using these commands:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;export VEP_PATH=$HOME/vep &#xA;export VEP_DATA=$HOME/.vep &#xA;mkdir -p $VEP_PATH $VEP_DATA &#xA;cd $VEP_PATH&#xA;vep_install -a cf -s homo_sapiens -y GRCh38 -c  $VEP_PATH --CONVERT&#xA;#  downloading ftp://ftp.ensembl.org/pub/release-88/variation/VEP/homo_sapiens_vep_88_GRCh38.tar.gz&#xA;cd $HOME/.vep nohup wget&#xA;ftp://ftp.ensembl.org/pub/release-101/variation/indexed_vep_cache/homo_sapiens_vep_101_GRCh38.tar.gz &amp;amp; &#xA;tar zxvf homo_sapiens_vep_101_GRCh38.tar.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, when I tried to test run VEP with the following command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;vep --help&#xA;# By default the VEP uses $HOME/.vep/  &#xA;vep  -i test_varscan.vcf  -o test.vcf  \&#xA;--cache --dir_cache $HOME/vep --force_overwrite  --assembly GRCh38 --vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I encountered the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl: relocation error: /public2/qiuting/.conda/envs/vcf2maf/lib/perl5/site_perl/5.22.0/x86_64-linux-thread-multi/auto/DBD/mysql/mysql.so:symbol mysql_options4, version libmysqlclient_18 not defined in file libmysqlclient.so.18 with link time reference&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To resolve this, I tried:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cpanm DBD::mysql  &#xA;conda install -c conda-forge perl-dbd-mysql&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Unfortunately, these attempts did not resolve the issue, and the same error occurred.&lt;/p&gt;&#xA;&lt;p&gt;I am not very familiar with Perl and its modules. Could anyone please help me solve this issue? Any suggestions or insights would be greatly appreciated!&lt;/p&gt;&#xA;&lt;p&gt;Thank you in advance!&lt;/p&gt;&#xA;" OwnerUserId="18871" LastActivityDate="2024-01-24T01:36:26.470" Title="Trouble Running VEP in Conda Environment Due to Perl DBD::mysql Error" Tags="|perl|conda|vep|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22084" PostTypeId="1" AcceptedAnswerId="22095" CreationDate="2024-01-24T04:48:11.643" Score="1" ViewCount="16" Body="&lt;p&gt;I have installed the latest SMRT Tools (ipdSummary version 3.0). My goal is to extract ipdRatio and modified bases. In the ipdSummary --help command :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;--identify IDENTIFY   Specific modifications to identify (comma-separated list). Current options are m6A, m4C, m5C_TET. Using --control overrides this option. (default: m6A,m4C)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Accordingly, I am running this command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;ipdSummary sorted_chr21_000.bam --reference $REF_path/$REF_FILE_NAME.fasta --referenceWindow 20:1-200000 --identify m6A,m4C,m5C_TET -j 20 --methylFraction --outfile $save_path/$chrom/$name&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;My bam file is sorted and relatively small (&amp;lt;200 MB).&#xA;But this command never finishes and does not throw any errors. In different stages of working it just stops. I checked different thread numbers and smaller files, but still, the command doesn't work.&#xA;However, once I changed 'm5C_TET' to 'm5C' it worked perfectly fine and very fast on the same bam file, but in the output, I could not see any information on 'm5C' modifications. I checked the new command with many other bam files and still the same.&lt;/p&gt;&#xA;&lt;p&gt;My question is the following:&#xA;How can I get at least 'm5C' modifications? Should I change my SMRT Tools version?&lt;/p&gt;&#xA;" OwnerUserId="15341" LastActivityDate="2024-01-25T16:21:37.507" Title="ipdSummary command stops working if --identify m5C_TET is included in command, and does not give any m5C info either" Tags="|pacbio|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22087" PostTypeId="1" AcceptedAnswerId="22099" CreationDate="2024-01-24T16:22:13.197" Score="3" ViewCount="123" Body="&lt;p&gt;I have a for loop which prints available vcf files in the path for a list of patient IDs &lt;strong&gt;(using find dx data function)&lt;/strong&gt; into my zsh terminal:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in $(cat patient_id.txt); do&#xA; dx find data --property patient_id=$i --path &amp;quot;vcf/&amp;quot;  &#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How I can have a file contain the vcf files that the loop shows in terminal? I mean by using this command in the terminal, I am seeing the name and path of my vcf files for a given participant list and working directory; Instead of seeing the file name in the terminal, I want to have that file name in a txt file.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2024-01-29T22:30:36.513" LastActivityDate="2024-01-30T05:52:49.263" Title="Printing the results of for loop into a txt file" Tags="|bash|dnanexus|zsh|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22090" PostTypeId="1" AcceptedAnswerId="22098" CreationDate="2024-01-24T17:57:25.257" Score="0" ViewCount="97" Body="&lt;p&gt;After running this command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;featureCounts -p -s 2 -a &lt;span class=&quot;math-container&quot;&gt;$genome -o $&lt;/span&gt;dir&amp;quot;/&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$specie&quot;/count_table/&quot;$&lt;/span&gt;value&amp;quot;.txt&amp;quot; $output_loc -T 8&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;on the BAM output of HISAT (&lt;code&gt;output_loc&lt;/code&gt;), I got this output:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/3mbFg.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/3mbFg.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Now, does the last column of this file show the gene count value?&#xA;And if I merge the same column of different samples, how should I go through the normalization step?&lt;/p&gt;&#xA;&lt;p&gt;Thanks a lot for your help!&lt;/p&gt;&#xA;" OwnerUserId="18867" LastEditorUserId="18867" LastEditDate="2024-01-25T14:44:01.950" LastActivityDate="2024-01-28T12:46:15.673" Title="Featurecount output .txt file from bam file" Tags="|rna-seq|gene-expression|read-mapping|featurecounts|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22092" PostTypeId="2" ParentId="22014" CreationDate="2024-01-25T09:05:31.377" Score="0" Body="&lt;p&gt;I have the same problem when facing the analysis of eQTL data for clumping and I do not find any answer to solve it. So I try to find a method to transform 1000 Genomes phase 3 ailgned by hg38 data which is pfile format (pgen, pvar, psam). I found a method at: &lt;a href=&quot;https://cran.r-project.org/web/packages/snpsettest/vignettes/reference_1000Genomes.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://cran.r-project.org/web/packages/snpsettest/vignettes/reference_1000Genomes.html&lt;/a&gt;. This is a method about how to extract the EUR population and transfrom to bfile format using PLINK 2.0. After this transfrom we can use PLINK 1.9 to clump the GWAS data. I think this is the method which we want to use and I will try it. Hope this can help you.&lt;/p&gt;&#xA;" OwnerUserId="18878" LastEditorUserId="18878" LastEditDate="2024-01-25T09:06:56.663" LastActivityDate="2024-01-25T09:06:56.663" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22093" PostTypeId="2" ParentId="22087" CreationDate="2024-01-25T09:43:41.500" Score="-1" Body="&lt;p&gt;Things that write to standard output on the terminal can be wrapped up in brackets and redirected to a file. Assuming this is what you want to do, I added brackets around the existing code, then used &lt;code&gt;&amp;gt;&lt;/code&gt; to redirect the output to a file.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(for i in $(cat patient_id.txt); do&#xA; dx find data --property patient_id=$i --path 'vcf/'  &#xA;done) &amp;gt; vcf_files.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that the brackets go around the entire code, including the &lt;code&gt;for&lt;/code&gt; bit.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2024-01-30T05:52:49.263" LastActivityDate="2024-01-30T05:52:49.263" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="22094" PostTypeId="2" ParentId="21926" CreationDate="2024-01-25T10:18:55.917" Score="0" Body="&lt;p&gt;I am having a very similar issue, however mine are multiple sequences of the same specie. I have a mixture of DNA and RNA sequences so although it appears to aligning fine the proteins produced are not correct. I’m at a loss on how to fix this so if you managed to figure out what to do please let me know.&lt;/p&gt;&#xA;" OwnerUserId="18879" LastActivityDate="2024-01-25T10:18:55.917" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22095" PostTypeId="2" ParentId="22084" CreationDate="2024-01-25T16:21:37.507" Score="0" Body="&lt;p&gt;I asked the same question on the ipdSummary GitHub page and I got the following response:&lt;/p&gt;&#xA;&lt;p&gt;&amp;quot;I think that was an experimental option that was never fully working or validated, and it no longer works. We probably need to update the help text. Use the separate tool jasmine if you want to do m5C calling.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;jasmine also is a tool in the SMRT toolkit that analyzes the kinetic signatures of cytosine bases in CpG motifs to identify the presence of 5mC.&lt;/p&gt;&#xA;" OwnerUserId="15341" LastActivityDate="2024-01-25T16:21:37.507" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22098" PostTypeId="2" ParentId="22090" CreationDate="2024-01-28T12:46:15.673" Score="3" Body="&lt;p&gt;The first 6 column in standard featureCounts output represent what is in the column names. All columns after than (starting at 7) represent the counts for the sample(s). If you use a single bam file as input then it's one column, if you use many bams as input then it is one column per bam. If the counts are gene level, exon transcript or anything else depends on how you run the tool. Typically, with default settings, it is usually the raw gene level counts.&lt;/p&gt;&#xA;&lt;p&gt;Generally, you can sum up technical replicates (=lane/sequencing replicates to increase depth) prior to running normalization.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2024-01-28T12:46:15.673" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22099" PostTypeId="2" ParentId="22087" CreationDate="2024-01-29T11:51:49.983" Score="3" Body="&lt;p&gt;This command worked just by adding  &lt;code&gt;&amp;gt; file&lt;/code&gt; after &lt;code&gt;done&lt;/code&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for i in &lt;span class=&quot;math-container&quot;&gt;$(cat patient_id.txt); do &#xA;dx find data --property eid=$&lt;/span&gt;i --path 'vcf/'  &#xA;done &amp;gt; vcf_list_file.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2024-01-29T22:30:52.487" LastActivityDate="2024-01-29T22:30:52.487" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22101" PostTypeId="2" ParentId="17558" CreationDate="2024-01-29T18:10:38.973" Score="0" Body="&lt;p&gt;If you want the column names to be preserved, you can subset the column names as a vector:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;human_colnames = colnames(scData)[scData&lt;span class=&quot;math-container&quot;&gt;$cell_ann == &quot;Cancer_human&quot;]&#xA;mouse_colnames = colnames(scData)[!scData$&lt;/span&gt;cell_ann == &amp;quot;Cancer_human&amp;quot;]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="73" LastActivityDate="2024-01-29T18:10:38.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22104" PostTypeId="1" AcceptedAnswerId="22108" CreationDate="2024-01-30T08:39:40.123" Score="1" ViewCount="91" Body="&lt;p&gt;I have come across several papers that do a Fisher's exact test to show over represented genes in COG categories specifically in a pan-core-accessory-species specific genome analysis. The test is done to show over represented genes in the core, accessory and species specific datasets as compared to the pan genome. Here is one such paper - &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9101195/#:%7E:text=cincinnatiensis%2C%20represented%20as%20a%20species,septicemia%20and%20meningitis%20%5B6%5D.&quot; rel=&quot;nofollow noreferrer&quot;&gt;Vibrio cincinnatiensis - comparative genome analysis&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Usually, the way the pan and core genome analysis is represented in papers is by showing the proportion of genes in each of the COG categories in both. For example, the proportion of C (Energy production and conversion) could be 6.07 in the pan but 6.8 in the core genome. The Fisher's exact test is used to show that this particular category is over represented in the core genome as compared to the pan.&lt;/p&gt;&#xA;&lt;p&gt;I already have the csv files from eggNOG from which I filtered out the genes in major COG categories for the pan, core, accessory and species-specific genomes into a single Excel sheet for 8 closely related genomes. Could someone tell me how to perform this test on this dataset?&lt;/p&gt;&#xA;&lt;p&gt;My null hypothesis is that the proportion of genes in the pan, core, accessory and species specific genomes is the same. But, obviously this analysis is done to show how the proportions vary in some COG categories between the above mentioned 4 genome datasets.&lt;/p&gt;&#xA;&lt;p&gt;I did try using python and R as well, but the test requires 2 rows and columns and here I am just comparing two columns (pan, core) with respect to 1 row - the 'C' category. It seems impossible to apply the test to this problem and I am sure I am missing something and that's why I posted it here.&lt;/p&gt;&#xA;&lt;p&gt;I tried doing the Fisher's test on the proportions I mentioned previously (6.07 and 6.8) but I am getting an error : 'Fisher's test requires two rows and two columns'. Should I compare the absolute values, like '3745' genes in the pan and '4195' genes in the core?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/Y3XVJ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/Y3XVJ.png&quot; alt=&quot;An example dataset for COG categories in pan, core, accessory and species specific genomes&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17392" LastEditorUserId="17392" LastEditDate="2024-02-23T02:32:39.257" LastActivityDate="2024-02-23T02:32:39.257" Title="Fisher's exact test for COG categories in pan, core genome analysis" Tags="|statistics|eggnog|pangenome|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22107" PostTypeId="1" AcceptedAnswerId="22109" CreationDate="2024-01-30T21:51:31.420" Score="0" ViewCount="38" Body="&lt;p&gt;Hi&lt;/p&gt;&#xA;&lt;p&gt;I have been recently struggling with the gene annotations and changing gene ID to ensemble ID. While I was doing that there were some genes for the same species that had the same gene ID but different ensemble ID. Can someone let me know how this happens? for example, for these two ensemble IDs ENSCAFG00845015896, and ENSCAFG00000017519 which are both for canines there is the same symbol of &amp;quot;TNMD&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="18867" LastActivityDate="2024-01-31T07:18:06.530" Title="Gene ensemble vs Hugo id" Tags="|bioconductor|gene|ensembl|biomart|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22108" PostTypeId="2" ParentId="22104" CreationDate="2024-01-30T23:00:29.033" Score="1" Body="&lt;p&gt;Sorry I've got my head around your question. For Fisher's Exact you need 2 &amp;quot;independent&amp;quot; observations and for SciPy its a 2x2 table.&lt;/p&gt;&#xA;&lt;p&gt;So in your case ...&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;Genome&lt;/th&gt;&#xA;&lt;th&gt;Species A&lt;/th&gt;&#xA;&lt;th&gt;Species B&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Pan&lt;/td&gt;&#xA;&lt;td&gt;6.07&lt;/td&gt;&#xA;&lt;td&gt;6.10&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;Core&lt;/td&gt;&#xA;&lt;td&gt;6.08&lt;/td&gt;&#xA;&lt;td&gt;6.12&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&lt;/div&gt;&#xA;&lt;p&gt;If you think about it, thats a better way of doing it because thats comparing pan vs core between species. A single observation isn't sufficient. To really do this you need 2 x 10 type contingency tables but SciPy will not do that, but you can do it in RStudio. For example using 10 species rather than just 2 ... and thats a real test of pan/core. My advice is to have your test species and in the other column have your control species ... like E. coli that sort of thing.&lt;/p&gt;&#xA;&lt;p&gt;The SciPy solution is here. Place your data in a list of lists here its &lt;code&gt;mylist&lt;/code&gt;. Next place into a numpy array and place that in a scipy function, like this...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import numpy as np&#xA;from scipy.stats import fisher_exact&#xA;mylist = [[6.07, 6.08], [6.10, 6.12]]&#xA;mytable = np.array(mylist)&#xA;res = fisher_exact(mytable, alternative='two-sided')&#xA;print (res.pvalue)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It should print the p value where &amp;gt; 0.05 is not significant.&lt;/p&gt;&#xA;&lt;p&gt;To start doing 2 x &amp;gt;2 contingency tables in Python r2py is needed and from my experience (albeit about 5 years ago) I wouldn't advise it. r2py is a disaster when I tried to use it and just did the calculations in RStudio.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2024-02-14T13:47:54.240" LastActivityDate="2024-02-14T13:47:54.240" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22109" PostTypeId="2" ParentId="22107" CreationDate="2024-01-31T07:18:06.530" Score="2" Body="&lt;p&gt;If you search for those gene IDs in Ensembl, you will see that ENSCAFG00845015896 is in the primary assembly, but ENSCAFG00000017519 is deprecated, and apparently has never been in the primary assembly but originates from the genome annotation of one individual dog. I did not align the protein sequences, but on a quick look they seem identical, which is why both are connected to the same gene symbol. If your identifiers are not all from the same assembly (and sometimes even if they are), you may see several ids for the same gene symbol.&lt;/p&gt;&#xA;" OwnerUserId="15489" LastActivityDate="2024-01-31T07:18:06.530" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22111" PostTypeId="1" AcceptedAnswerId="22114" CreationDate="2024-01-31T20:49:27.240" Score="3" ViewCount="58" Body="&lt;p&gt;I was struggling to use AnnotationDbi to change my ensemble ID to gene name, for datasets of three different species (human, canine, mouse). Among gene names for all three species there are genes with names starting with &amp;quot;LOC&amp;quot; like ENSG00000228037:&#xA;LOC100996583, ENSCAFG00845012021: LOC613013 and ENSMUSG00000095523&#xA;LOC100038995.&lt;/p&gt;&#xA;&lt;p&gt;What type of genes are they? and is the naming unique through species, I mean if I have LOC100996583 in humans, is it possible to have it in canines or mice? if yes, are they doing the same thing?&lt;/p&gt;&#xA;&lt;p&gt;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="18867" LastActivityDate="2024-02-01T11:05:11.860" Title="What is it about gene names starting with &quot;LOC&quot;?" Tags="|bioconductor|gene|genomics|human-genome|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22113" PostTypeId="1" AcceptedAnswerId="22115" CreationDate="2024-02-01T09:22:44.573" Score="3" ViewCount="77" Body="&lt;p&gt;I have a &lt;code&gt;individual.snp.vcf.gz&lt;/code&gt; file of an individual genome and the &lt;code&gt;referencegenome.snp.vcf.gz&lt;/code&gt; file of the reference genome.&lt;/p&gt;&#xA;&lt;p&gt;When I run the following code on the individual genome&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gunzip -c individual.snp.vcf.gz | grep -v '^##' | awk 'BEGIN {OFS=&amp;quot;\t&amp;quot;} {print $1, $2, $3, $4, $5, $6, $7}' | awk 'BEGIN{OFS=&amp;quot;\t&amp;quot;} {print $0 , &amp;quot;SNP&amp;quot;}' | sed '1s/SNP/Type/'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT QUAL    FILTER  Type&#xA;chr1    10067   .   T   A   0   LowGQX  SNP&#xA;chr1    10070   .   C   A   0   LowGQX  SNP&#xA;chr1    10073   .   T   C   0   LowGQX  SNP&#xA;chr1    10105   .   A   C   0   LowGQX  SNP&#xA;chr1    10108   rs62651026  C   T   0   LowGQX  SNP&#xA;chr1    10157   .   T   C   0   LowGQX  SNP&#xA;chr1    10177   rs201752861 A   C   17  LowGQX  SNP&#xA;chr1    10180   rs201694901 T   C   0   LowGQX  SNP&#xA;chr1    10181   rs1246412344    A   T   0   LowGQX  SNP&#xA;chr1    10250   rs199706086 A   C   0   LowGQX  SNP&#xA;chr1    10257   rs111200574 A   C   0   LowGQX  SNP&#xA;chr1    10285   rs866375379 T   C   67  LowGQX  SNP&#xA;chr1    10321   rs1002315756    C   T   0   LowGQX  SNP&#xA;chr1    10327   rs112750067 T   C   36  LowGQX  SNP&#xA;chr1    10332   rs1175748383    C   A   0   LowGQX  SNP&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;When I run the equivalent code on the reference genome&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gunzip -c referencegenome.snp.vcf.gz | grep -v '^##' | awk 'BEGIN {OFS=&amp;quot;\t&amp;quot;} {print $1, $2, $3, $4, $5, $6, $7}' | head -n100 | awk 'BEGIN{OFS=&amp;quot;\t&amp;quot;} {print $0 , &amp;quot;SNP&amp;quot;}' | sed '1s/SNP/Type/'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I get this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT QUAL    FILTER  Type&#xA;1   10001   rs1570391677    T   A   .   .   SNP&#xA;1   10002   rs1570391692    A   C   .   .   SNP&#xA;1   10003   rs1570391694    A   C   .   .   SNP&#xA;1   10008   rs1570391698    A   G   .   .   SNP&#xA;1   10009   rs1570391702    A   G   .   .   SNP&#xA;1   10015   rs1570391706    A   G   .   .   SNP&#xA;1   10020   rs1570391708    A   C   .   .   SNP&#xA;1   10021   rs1570391710    A   G   .   .   SNP&#xA;1   10026   rs1570391712    A   C   .   .   SNP&#xA;1   10027   rs1570391716    A   C,G .   .   SNP&#xA;1   10032   rs1570391720    A   C   .   .   SNP&#xA;1   10033   rs1570391722    A   G   .   .   SNP&#xA;1   10039   rs978760828 A   C   .   .   SNP&#xA;1   10043   rs1008829651    T   A   .   .   SNP&#xA;1   10045   rs1570391729    A   C,G .   .   SNP&#xA;1   10051   rs1052373574    A   C,G .   .   SNP&#xA;1   10055   rs892501864 T   A   .   .   SNP&#xA;1   10056   rs1570391738    A   C   .   .   SNP&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now I want to get the genotype of the &lt;em&gt;individual genome&lt;/em&gt; at all the sites listed in the &lt;em&gt;reference genome&lt;/em&gt; table, so my question is: &lt;strong&gt;When I don't find a particular variant ID in the &lt;em&gt;individual genome&lt;/em&gt; table (for example, &lt;strong&gt;rs1570391716&lt;/strong&gt;), can I assume that the genotype of the individual is equal to the homozygous &lt;em&gt;REF&lt;/em&gt; (&lt;strong&gt;AA&lt;/strong&gt;, for &lt;strong&gt;rs1570391716&lt;/strong&gt;) in the &lt;em&gt;reference genome&lt;/em&gt; table?&lt;/strong&gt; How certain can I be (assuming a low sequence reading error)?&lt;/p&gt;&#xA;" OwnerUserId="18901" LastActivityDate="2024-03-02T00:21:20.183" Title="What does the absence of a variant in a VCF file mean?" Tags="|vcf|genomics|genotyping|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22114" PostTypeId="2" ParentId="22111" CreationDate="2024-02-01T11:05:11.860" Score="3" Body="&lt;p&gt;These are &amp;quot;genes&amp;quot; that don't have an official name. As explained in &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/books/NBK3840/#genefaq.Nomenclature&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/books/NBK3840/#genefaq.Nomenclature&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;&lt;strong&gt;Symbols beginning with LOC.&lt;/strong&gt; When a published symbol is not available, and orthologs have not yet been determined, Gene will provide a symbol that is constructed as 'LOC' + the GeneID. This is not retained when a replacement symbol has been identified, although queries by the LOC term are still supported. In other words, a record with the symbol LOC12345 is equivalent to GeneID = 12345. So if the symbol changes, the record can still be retrieved on the web using LOC12345 as a query, or from any file using GeneID = 12345.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;In other words, when a gene has no known function or homologs in other species, when all we know about it is that it is a &lt;em&gt;locus&lt;/em&gt; that seems to be actively transcribed, then it gets the name of &lt;code&gt;LOC&lt;/code&gt; plus a numerical gene ID. So these genes, the LOC&lt;em&gt;NNNN&lt;/em&gt; are essentially lesser known and studied genes. Some of them may graduate to a full gene name, others might not.&lt;/p&gt;&#xA;&lt;p&gt;As for uniqueness, no absolutely not. GenBank gene IDs are unique, they don't imply homology. Only one gene can have a given ID. For example, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene/1&quot; rel=&quot;nofollow noreferrer&quot;&gt;gene ID 1&lt;/a&gt; corresponds to human A1BG, alpha-1-B glycoprotein. The mouse homolog of human A1BG, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/gene/117586&quot; rel=&quot;nofollow noreferrer&quot;&gt;mouse A1bg, alpha-1-B glycoprotein&lt;/a&gt;, has the gene ID 117586. The numerical value of the ID tells you nothing at all. It is just a unique identifier for a specific gene in a specific species.&lt;/p&gt;&#xA;&lt;p&gt;Finally, a more general note, homology is rarely as straightforward as your question suggests you might think it is. Even actually homologous genes often have different functions with specific functions having been gained or lost in a given lineage. So it is very hard to know and never safe to assume that two homologous genes in different species &amp;quot;do the same thing&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2024-02-01T11:05:11.860" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22115" PostTypeId="2" ParentId="22113" CreationDate="2024-02-01T12:56:18.833" Score="3" Body="&lt;p&gt;The absence of a variant means the variant was not found. However, that does not mean you can assume the position in the sequenced individual was homozygous for what is in the reference. Perhaps the position was simply not sequenced. Or the coverage was too low. So your first step will have to be to focus only on the regions covered by whatever sequencing assay was used to sequence your data, and only those regions with sufficient coverage for a variant to be called.&lt;/p&gt;&#xA;&lt;p&gt;There is no magic number here, the threshold will depend on how the data were generated. For instance, a minimum coverage of 8 is a reasonable lower bound, but these days, we often have coverage values of 60x or more, and in the case of amplicon assays, we can have coverage in the thousands. So the threshold is something you will need to decide on.&lt;/p&gt;&#xA;&lt;p&gt;Next, you really need to take the quality and filters into account. You haven't shown us the full VCF line, but my bet is that all the variants you show in your individual file are most likely not genotyped at all. These are not actually present. They all have a score of 0 and are marked with &lt;code&gt;LowGQX&lt;/code&gt; which means:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The genotyping quality locus (GQX) is less than 30 or not present.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;And GQX is:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The minimum of genotype quality assuming variant position and genotype quality assuming nonvariant position.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Variants with a QUAL score of 0 and low genotype quality are almost certainly not variants at all and you really shouldn't consider them.&lt;/p&gt;&#xA;&lt;p&gt;So, since you apparently don't have the raw data, neither fastq nor bam, that means you cannot easily filter by coverage in the various positions. However, at the very least, you need to find out what sequencing assay and method was used to sequence your sample. Then, you can get a bed file with the regions of interest, and then limit your analysis to variants in those regions.&lt;/p&gt;&#xA;&lt;p&gt;Also, I urge you to have a look at tools like &lt;a href=&quot;https://bedtools.readthedocs.io/en/latest/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;bedtools&lt;/code&gt;&lt;/a&gt; and &lt;a href=&quot;https://samtools.github.io/bcftools/bcftools.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;code&gt;bcftools&lt;/code&gt;&lt;/a&gt; that will help you filter your data.&lt;/p&gt;&#xA;&lt;p&gt;Finally, while your command will work, it is a bit inefficient, and you could simplify&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gunzip -c individual.snp.vcf.gz | grep -v '^##' | &#xA;  awk 'BEGIN {OFS=&amp;quot;\t&amp;quot;} {print $1, $2, $3, $4, $5, $6, $7}' | &#xA;    awk 'BEGIN{OFS=&amp;quot;\t&amp;quot;} {print $0 , &amp;quot;SNP&amp;quot;}' | sed '1s/SNP/Type/'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;to&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zgrep -v '^##'  individual.snp.vcf.gz | &#xA;  awk 'BEGIN {OFS=&amp;quot;\t&amp;quot;} {print $1, $2, $3, $4, $5, $6, $7, NR==1 ? &amp;quot;Type&amp;quot; : &amp;quot;SNP&amp;quot;}'&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2024-03-02T00:21:20.183" LastActivityDate="2024-03-02T00:21:20.183" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22116" PostTypeId="1" CreationDate="2024-02-01T13:50:43.800" Score="1" ViewCount="43" Body="&lt;p&gt;I have samples from 45 individuals at 2 time points and these individuals were given a specific diet. The BMI of these individuals were recorded at both time points. The responders to the diet are the ones that show decrease in BMI from T1 to T2. I want to use continuous variable (BMI) to identify genes that are predictors of response to diet. I also have other covariates that need to be controlled like age &amp;amp; gender. The metadata table looks like following&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/y7znl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/y7znl.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am not sure the design that I should go for. The one that I think is appropriate is&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData=cts,colData=meta,design=~0+Time+Gender+AGE+Time:BMI)&#xA;dds &amp;lt;- DESeq(dds,test=&amp;quot;LRT&amp;quot;, reduced=~Gender+AGE)&#xA;res=results(dds, contrast=list(&amp;quot;T1.BMI&amp;quot;,&amp;quot;T2.BMI&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Is this the correct way to go ahead and to identify responder genes to the diet?&lt;/p&gt;&#xA;&lt;p&gt;All suggestions are very welcome.&lt;/p&gt;&#xA;&lt;p&gt;Thank you&lt;/p&gt;&#xA;" OwnerUserId="5062" LastEditorUserId="650" LastEditDate="2024-02-01T17:51:44.450" LastActivityDate="2024-02-01T17:51:44.450" Title="Continuous predictor variable to identify responder genes using DESeq2" Tags="|rna-seq|deseq2|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22117" PostTypeId="2" ParentId="22059" CreationDate="2024-02-01T18:24:52.263" Score="1" Body="&lt;p&gt;Similar to @pippo1980 comment: start with a related standard amino acid, fold it however you are doing that, and then mutate to the nonstandard in the folded structure. You can do this with a toolkit like &lt;a href=&quot;https://github.com/openmm/pdbfixer/blob/master/pdbfixer/pdbfixer.py#L729&quot; rel=&quot;nofollow noreferrer&quot;&gt;pdbfixer&lt;/a&gt; for standard AAs, you might need to do &lt;a href=&quot;https://stackoverflow.com/questions/47631064/create-a-polymer-chain-of-nonstandard-residues-from-a-single-residue-pdb&quot;&gt;something like this&lt;/a&gt; to directly add the atoms for nonstandards, using an example pdb structure of the nonstandard AA.&lt;/p&gt;&#xA;&lt;p&gt;You would then need to follow that with a round of energy minimization in the structure to resolve any large problems in the structure, using a toolkit like &lt;a href=&quot;http://docs.openmm.org/7.5.0/api-python/generated/simtk.openmm.openmm.LocalEnergyMinimizer.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;OpenMM&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;If &lt;code&gt;X&lt;/code&gt; truly means that you have no idea what's there, then... that's harder. In this specific case lysine may be a best bet, but it may be hard to generalize between e.g. methyllysine, phosphotyrosine, phosphoserine, etc. They're pretty different structurally, so having a good idea of what family of residue you would expect would be helpful.&lt;/p&gt;&#xA;&lt;p&gt;The safest thing to do would be to go back to the transcript and in vitro translate to get an idea of what that X is. Though that's getting a little complicated compared to just replacing it with a leucine or whatever.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2024-02-01T18:24:52.263" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22118" PostTypeId="2" ParentId="22024" CreationDate="2024-02-01T21:06:40.457" Score="0" Body="&lt;p&gt;I would suggest looking into &lt;a href=&quot;https://www.gencodegenes.org/human/&quot; rel=&quot;nofollow noreferrer&quot;&gt;GFF3 annotations&lt;/a&gt; of the human genome.&lt;/p&gt;&#xA;&lt;p&gt;If what you want is &lt;em&gt;protein domain annotations&lt;/em&gt;, you may be more interested in the &lt;a href=&quot;https://ftp.uniprot.org/pub/databases/uniprot/current_release/knowledgebase/genome_annotation_tracks/&quot; rel=&quot;nofollow noreferrer&quot;&gt;UniProt database&lt;/a&gt; which does seem to include &lt;a href=&quot;https://ftp.uniprot.org/pub/databases/uniprot/current_release/knowledgebase/genome_annotation_tracks/UP000000589_10090_beds/&quot; rel=&quot;nofollow noreferrer&quot;&gt;genome annotation tracks&lt;/a&gt; for different protein functional regions.&lt;/p&gt;&#xA;&lt;p&gt;I am unclear if uniprot directly has the gene names but this would be trivial to generate with a &lt;code&gt;bedtools intersect&lt;/code&gt; operation.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2024-02-01T21:06:40.457" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22119" PostTypeId="1" CreationDate="2024-02-01T21:26:59.743" Score="2" ViewCount="9" Body="&lt;p&gt;I have developed an ANN model to predict protein post-translational modification patterns from Elisa data. For simplicities sake, the how, feasibility and parameters aren't important for my question and some details were omitted.&lt;/p&gt;&#xA;&lt;p&gt;For a given protein, I will call it protein X, it has ubiquitin as a modification but no phosphorylation patterns.&lt;/p&gt;&#xA;&lt;p&gt;I trained the ANN with a variety of post-translational modification patterns but a KEY PIECE OF INFO: my training data did not contain any ubiquitin patterns (just assume there's a reason)&lt;/p&gt;&#xA;&lt;p&gt;So when I performed an ELISA with a set of antibodies and antibody a is specific for ubiquitin patterns and antibody b is specific for phosphorylated patterns. When I ran the prediction of protein x modification patterns using antibody a, antibody b (and others) we got pretty good accuracy.&lt;/p&gt;&#xA;&lt;p&gt;To explain how the model worked, I ran a SHAP analysis and a bipartite graph to show the feature importances (antibodies) and the modifications but got confounding results&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;For antibody a, there were positive and some negative SHAP values to other modification patterns except for ubiquitin, which it is specific for&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;For antibody b, we also found positive and negative SHAP values to modification patterns except phosphorylation, which protein x doesn't really have.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;So how do I explain why SHAP produced this pattern: 1)antibody a did not have any SHAP correlation to its target, ubiquitin but it did to others, 2) antibody b also did not have SHAP correlations to its target but to others instead.&lt;/p&gt;&#xA;&lt;p&gt;Again this is confounding because I expected it antibody a to have SHAP correlations to ubiquitin but not the others, and then antibody b should not have any SHAP correlations since it targets phosphorylation but protein x has no phosphorylation.&lt;/p&gt;&#xA;&lt;p&gt;Im not too convinced or can't relate some of the limitations with SHAP that it shows the hidden patterns/relationships for the model, not what we expect in &amp;quot;real life&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Can someone provide a more nuanced insight to this observed SHAP analysis?&lt;/p&gt;&#xA;" OwnerUserId="18908" LastActivityDate="2024-02-01T21:26:59.743" Title="SHAP correlation patterns for model prediction of protein are confusing" Tags="|computational-biochemistry|systems-biology|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22120" PostTypeId="1" CreationDate="2024-02-01T21:45:44.780" Score="2" ViewCount="16" Body="&lt;p&gt;I'm working with &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/refseq/MANE/&quot; rel=&quot;nofollow noreferrer&quot;&gt;the MANE v1.2 annotation&lt;/a&gt; (with RefSeq keys) and am puzzled about the ID naming convention being used. Specifically, there are several transcripts (18 by my count) that have hyphenated numbers attached to their IDs, which are otherwise just RefSeq IDs. For example, the entry &lt;strong&gt;NM_000937.5&lt;/strong&gt; (for POLR2A) is listed as &lt;code&gt;&amp;quot;NM_000937.5&amp;quot;&lt;/code&gt; in FASTA, but as &lt;code&gt;&amp;quot;NM_000937.5-2&amp;quot;&lt;/code&gt; in the GFF. In fact, everywhere in the GFF any reference to POLR2A is listed as &lt;code&gt;&amp;quot;POLR2A-2&amp;quot;&lt;/code&gt;. This perhaps could make sense if there were multiple transcripts included for the gene, but it is not the case - there are no plain &lt;code&gt;&amp;quot;POLR2A&amp;quot;&lt;/code&gt; entries nor any &lt;code&gt;&amp;quot;POLR2A-1&amp;quot;&lt;/code&gt; entries.&lt;/p&gt;&#xA;&lt;p&gt;What is going on here? Why do these entries have additional numbers attached?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;For reference, the 18 transcripts with hyphenated names are:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;MANE ID                 RefSeq ID&#xA;rna-NM_001405666.3-2    NM_001405666.3&#xA;rna-NM_001291281.3-2    NM_001291281.3&#xA;rna-NM_001137667.2-2    NM_001137667.2&#xA;rna-NM_017680.6-2       NM_017680.6&#xA;rna-NM_002457.5-2       NM_002457.5&#xA;rna-NM_001164277.2-2    NM_001164277.2&#xA;rna-NM_178167.5-2       NM_178167.5&#xA;rna-NM_000937.5-2       NM_000937.5&#xA;rna-NM_001324073.3-2    NM_001324073.3&#xA;rna-NM_001097639.3-2    NM_001097639.3&#xA;rna-NM_152503.8-2       NM_152503.8&#xA;rna-NM_001372044.2-2    NM_001372044.2&#xA;rna-NM_001417890.1-2    NM_001417890.1&#xA;rna-NM_022555.4-2       NM_022555.4&#xA;rna-NM_021983.5-2       NM_021983.5&#xA;rna-NM_014513.3-3       NM_014513.3&#xA;rna-NM_020535.3-5       NM_020535.3&#xA;rna-NM_014512.1-9       NM_014512.1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="900" LastActivityDate="2024-02-01T21:45:44.780" Title="Confusing naming in MANE GFF file" Tags="|sequence-annotation|gff3|refseq|mane|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22122" PostTypeId="1" CreationDate="2024-02-02T07:10:41.360" Score="1" ViewCount="12" Body="&lt;p&gt;I aligned multiple protein structures from the same protein family to a reference structure in pymol and now I want to calculate the Difference in Phi psi values for each 'aligned' residue pair between Reference Structure and each Target Structure, I have extracted the phi psi values using phi_psi command and I have the sequence alignment but I don't know how to extract Delta Phi Psi values for every aligned residue between Reference and Target.&lt;/p&gt;&#xA;" OwnerUserId="18909" LastActivityDate="2024-02-02T07:10:41.360" Title="How to Calculate Delta Phi Psi between aligned pair of residue between Reference and Target Protein Structure in Pymol" Tags="|pymol|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22123" PostTypeId="1" CreationDate="2024-02-02T11:25:03.833" Score="1" ViewCount="27" Body="&lt;p&gt;I am trying to run BLASTN on a 14GB Pseudomonas database using the command: blastn -db ~/pseudomonas-db/pseudomonas -query ~/desktop/inputfiles/poz176.fasta -out poz176-out -num_threads 8&lt;/p&gt;&#xA;&lt;p&gt;I get the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Error: NCBI C++ Exception:&#xA;    T0 &amp;quot;c++/include/corelib/ncbidiag.hpp&amp;quot;, line 980: Critical: (CCoreException::eNullPtr) ncbi::CObject::ThrowNullPointerException() - Attempt to access NULL pointer.&#xA;    T0 &amp;quot;c++/include/corelib/ncbidiag.hpp&amp;quot;, line 789: Critical: (CCoreException::eNullPtr) ncbi::BlastFormatter_PreFetchSequenceData() - Error pre-fetching sequence data &#xA;     Stack trace:&#xA;      /usr/lib/ncbi-blast+/libxncbi.so ???:0 ncbi::CObject::ThrowNullPointerException() offset=0xA6 addr=0x7fa1ea902f76&#xA;      /usr/lib/ncbi-blast+/libxobjmgr.so ???:0  offset=0x209609 addr=0x7fa1eb653609&#xA;      /usr/lib/ncbi-blast+/libxobjmgr.so ???:0 ncbi::objects::CBioseq_Handle::IsSetInst_Length() const offset=0x9 addr=0x7fa1eb653a09&#xA;      /usr/lib/ncbi-blast+/libxobjmgr.so ???:0 ncbi::objects::CBioseq_Handle::GetBioseqLength() const offset=0xE addr=0x7fa1eb653dce&#xA;      /usr/lib/ncbi-blast+/libxblast.so ???:0 ncbi::blast::LoadSequencesToScope(std::vector&amp;lt;ncbi::objects::CSeq_id_Handle, std::allocator&amp;lt;ncbi::objects::CSeq_id_Handle&amp;gt; &amp;gt;&amp;amp;, std::vector&amp;lt;ncbi::CRange&amp;lt;unsigned int&amp;gt;, std::allocator&amp;lt;ncbi::CRange&amp;lt;unsigned int&amp;gt; &amp;gt; &amp;gt;&amp;amp;, ncbi::CRef&amp;lt;ncbi::objects::CScope, ncbi::CObjectCounterLocker&amp;gt;&amp;amp;) offset=0x329 addr=0x7fa1ebd90e79&#xA;      blastn ???:0 ncbi::BlastFormatter_PreFetchSequenceData(ncbi::blast::CSearchResultSet const&amp;amp;, ncbi::CRef&amp;lt;ncbi::objects::CScope, ncbi::CObjectCounterLocker&amp;gt;, ncbi::blast::CFormattingArgs::EOutputFormat) offset=0x5FE addr=0x555af7fb31ee&#xA;      blastn ???:0 CBlastnApp::Run() offset=0xFDC addr=0x555af7fa99ac&#xA;      /usr/lib/ncbi-blast+/libxncbi.so ???:0 ncbi::CNcbiApplicationAPI::x_TryMain(ncbi::EAppDiagStream, char const*, int*, bool*) offset=0x193 addr=0x7fa1ea84edd3&#xA;      /usr/lib/ncbi-blast+/libxncbi.so ???:0 ncbi::CNcbiApplicationAPI::AppMain(int, char const* const*, char const* const*, ncbi::EAppDiagStream, char const*, std::__cxx11::basic_string&amp;lt;char, std::char_traits&amp;lt;char&amp;gt;, std::allocator&amp;lt;char&amp;gt; &amp;gt; const&amp;amp;) offset=0x61D addr=0x7fa1ea85032d&#xA;      blastn ???:0 main offset=0x6B addr=0x555af7fa7e5b&#xA;      /lib/x86_64-linux-gnu/libc.so.6 ???:0 __libc_start_main offset=0xEA addr=0x7fa1ea38ed0a&#xA;      blastn ???:0 _start offset=0x2A addr=0x555af7fa876a&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I tried changing memory allocation but that didn't seem to do anything. I'm not sure what this error is telling me. How do I get around this?&lt;/p&gt;&#xA;&lt;p&gt;(Had posted this on stackoverflow and was asked to post here)&lt;/p&gt;&#xA;" OwnerUserId="18910" LastActivityDate="2024-02-02T12:15:07.143" Title="What does this BLASTN mean: Error pre-fetching sequence data?" Tags="|blast|ncbi|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22124" PostTypeId="1" AcceptedAnswerId="22127" CreationDate="2024-02-02T11:43:10.040" Score="0" ViewCount="44" Body="&lt;p&gt;We want to compare some variant calling tools and their calls on Whole Exome Sequencing data. We will have to normalize the variants that are called (.vcf format) before comparison using for instance &lt;a href=&quot;https://genome.sph.umich.edu/wiki/Vt#Normalization&quot; rel=&quot;nofollow noreferrer&quot;&gt;vt normalize&lt;/a&gt; or &lt;a href=&quot;https://github.com/Illumina/hap.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;hap.py&lt;/a&gt;. We are comparing with a gold standard too and use hap.py for this now to get some benchmark information.&lt;/p&gt;&#xA;&lt;p&gt;Edit: with normalization I mean representing the variants in such a way, we are able to compare them: to ensure that the representation of a variant is “parsimonious” and “left-aligned.”&lt;/p&gt;&#xA;&lt;p&gt;I was just wondering, and I cannot find any real guidelines (yet), when we annotate using &lt;a href=&quot;https://www.ensembl.org/info/docs/tools/vep/script/vep_options.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;VEP&lt;/a&gt;, do we have to normalize before or after annotation to keep the annotation correct as well. I know VEP has some normalization functionality too &lt;a href=&quot;https://www.ensembl.info/2020/05/26/normalising-variants-to-standardise-ensembl-vep-output/&quot; rel=&quot;nofollow noreferrer&quot;&gt;see here&lt;/a&gt;. So might we even not need an additional normalization tool before and just let VEP do it for us?&lt;/p&gt;&#xA;&lt;p&gt;I am just wondering about the guidelines and what what others do.&#xA;Thanks in advance,&lt;/p&gt;&#xA;" OwnerUserId="16204" LastEditorUserId="16204" LastEditDate="2024-02-05T12:19:28.967" LastActivityDate="2024-02-05T12:19:28.967" Title="Variants from multiple tools: normalization before or after annotation with VEP?" Tags="|variant-calling|vep|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22125" PostTypeId="2" ParentId="22123" CreationDate="2024-02-02T12:15:07.143" Score="1" Body="&lt;p&gt;Based on other &lt;a href=&quot;https://www.biostars.org/p/9462173/&quot; rel=&quot;nofollow noreferrer&quot;&gt;posts elsewhere&lt;/a&gt; it seems likely that there is something funky about your database. How did you generate it?&lt;/p&gt;&#xA;&lt;p&gt;Could you try to re-generate the db without the &lt;code&gt;-parse-seqids&lt;/code&gt; flag?&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2024-02-02T12:15:07.143" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22126" PostTypeId="1" CreationDate="2024-02-02T12:54:58.047" Score="1" ViewCount="52" Body="&lt;p&gt;I have a vcf file like below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  2072184&#xA;chr1    10109   .   AACCCT  A   21.51   PASS    AC=2;AF=1.000;AN=2;DP=10;FS=8.451;MQ=13.01;MQRankSum=0.000;QD=2.15;ReadPosRankSum=-1.300;SOR=3.236;FractionInformativeReads=0.700;R2_5P_bias=0.000  GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    ./.:3,4:0.571:7:1,1:2,3:6:30,8,0:2.1511e+01,5.8560e+00,1.3473e+00:0.00,7.00,10.01:1,2,4,0:3,0,1,3&#xA;chr1    10230   .   AC  A   29.28   PASS    AC=1;AF=0.500;AN=2;DP=21;FS=12.601;MQ=20.22;MQRankSum=-0.715;QD=1.39;ReadPosRankSum=1.109;SOR=4.407;FractionInformativeReads=0.667;R2_5P_bias=-3.357    GT:AD:AF:DP:F1R2:F2R1:GQ:PL:GP:PRI:SB:MB    0/1:6,8:0.571:14:1,4:5,4:22:59,0,19:2.9283e+01,2.9815e-02,2.2470e+01:0.00,30.00,33.01:3,3,0,8:2,4,4,4&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am going to subset this VCF by the coordinates given in a bed file using tabix&lt;/p&gt;&#xA;&lt;p&gt;This is how my bed file looks&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr7    127471196  127472363&#xA;chr7    127472363  127473530&#xA;chr7    127473530  127474697&#xA;chr7    127474697  127475864&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is my command&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;tabix -h filevcf.gz -R XXX.bed &amp;gt; file.vcf.gz_extracted.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;But output file is empty after headers I can not find variants in the output&lt;/p&gt;&#xA;&lt;p&gt;I look at output by &lt;code&gt;bcftools view -H file.vcf.gz_extracted.vcf.gz &amp;gt; output.txt&lt;/code&gt; which is totally empty&lt;/p&gt;&#xA;&lt;p&gt;Where I am making a mistake here?&lt;/p&gt;&#xA;&lt;p&gt;I got results when I used some of positions in input vcf&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="73" LastEditDate="2024-02-04T19:14:58.457" LastActivityDate="2024-02-04T19:14:58.457" Title="tabix to subset each VCF by the coordinates given" Tags="|vcf|bash|bed|wgs|tabix|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22127" PostTypeId="2" ParentId="22124" CreationDate="2024-02-02T15:09:04.710" Score="1" Body="&lt;p&gt;If you're referring to left alignment + parsimonious representation, do it before any annotation. Most if not all annotation happens by comparing CHROM, POS and optionally REF and ALT and since vt normalize affects POS, you should do it before annotation.&lt;/p&gt;&#xA;&lt;p&gt;I'm not sure how smart VEP is, but most annotators are simple lookup tools at least for third party databases, so they're not built to normalize while looking up.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2024-02-02T15:09:04.710" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="22129" PostTypeId="2" ParentId="20938" CreationDate="2024-02-02T20:26:01.617" Score="1" Body="&lt;p&gt;It is not difficult to simulate every 10mer DNA oligo, here is some Python code that will do that:&lt;/p&gt;&#xA;&lt;p&gt;(there is probably a more elegant way to do this)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from itertools import product&#xA;&#xA;nts = [&amp;quot;A&amp;quot;, &amp;quot;T&amp;quot;, &amp;quot;C&amp;quot;, &amp;quot;G&amp;quot;]&#xA;# create a tuple with 10 copies of nts for the cartesian product&#xA;ten_tup = (nts,)*10&#xA;&#xA;decamers = []&#xA;# create and iterate over a stringified cartesian product,&#xA;# which consists of all 10mers.&#xA;for a,b,c,d,e,f,g,h,i,j in product(*ten_tup):&#xA;  decamers.append(&#xA;    a+b+c+d+e+f+g+h+i+j&#xA;  )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;At this point, you have some choices to make:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;how many 10mers do you want?&lt;/li&gt;&#xA;&lt;li&gt;what exactly is your criterion for GC content?&lt;/li&gt;&#xA;&lt;li&gt;what exactly is your criterion for thermodynamic interactions?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I'd suggest either &lt;a href=&quot;https://primer3.org/manual.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;primer3&lt;/a&gt; or &lt;a href=&quot;https://www.google.com/url?sa=t&amp;amp;rct=j&amp;amp;q=&amp;amp;esrc=s&amp;amp;source=web&amp;amp;cd=&amp;amp;cad=rja&amp;amp;uact=8&amp;amp;ved=2ahUKEwiY0NiLuY2EAxVkLzQIHegzBw8QFnoECA8QAQ&amp;amp;url=http%3A%2F%2Frna.tbi.univie.ac.at%2F&amp;amp;usg=AOvVaw10gHgHzOZkcSrePX6gBPnv&amp;amp;opi=89978449&quot; rel=&quot;nofollow noreferrer&quot;&gt;viennarna&lt;/a&gt; for evaluating thermodynamics. You can run each from the command line or from a web server, but they do have Python APIs, which I use below. ViennaRNA is definitely more trustworthy for sequences as short as 10, so I'll use it here. It will be a lot slower though as it's not cheating by using the nearest-neighbor approximations.&lt;/p&gt;&#xA;&lt;p&gt;I have made up some numbers for the thresholds below, replace with your own.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from random import shuffle&#xA;GC_MIN = 4&#xA;GC_MAX = 6&#xA;DDG = -5&#xA;NUM_WANTED = 10&#xA;&#xA;&#xA;def test_gc(seq, gc_min=GC_MIN, gc_max=GC_MAX):&#xA;  gc_count = seq.count(&amp;quot;G&amp;quot;) + seq.count(&amp;quot;C&amp;quot;)&#xA;  if gc_count &amp;gt;= gc_min and gc_count &amp;lt;= gc_max:&#xA;    return True&#xA;  else:&#xA;    return False&#xA;&#xA;&#xA;def test_thermo(seq, other_seqs, ddg=DDG):&#xA;  intra = RNA.fold_compound(seq).mfe()[1]&#xA;  print(intra)&#xA;  if other_seqs:&#xA;    # find the most favorable interaction with any other sequence&#xA;    inter = min(&#xA;      [RNA.fold_compound(seq+&amp;quot;&amp;amp;&amp;quot;+other_seq).mfe_dimer()[1] for other_seq in other_seqs]&#xA;    )&#xA;  else:&#xA;    inter = 0&#xA;  homo = RNA.fold_compound(seq+&amp;quot;&amp;amp;&amp;quot;+seq).mfe_dimer()[1]&#xA;  if intra &amp;gt; ddg and inter &amp;gt; ddg and homo &amp;gt; ddg:&#xA;    return True&#xA;  else:&#xA;    print(&amp;quot;thermo reject!!&amp;quot;)&#xA;    return False&#xA;&#xA;chosen_seqs = []&#xA;# shuffle the decamers to ensure that you get dissimilar sequences&#xA;shuffle(decamers)&#xA;i = 0&#xA;for seq in decamers:&#xA;  i+=1&#xA;  print(i)&#xA;  if test_gc(seq) and test_thermo(seq, chosen_seqs):&#xA;    chosen_seqs.append(seq)&#xA;  if len(chosen_seqs) &amp;gt;= NUM_WANTED:&#xA;    break&#xA;&#xA;# chosen_seqs has your decamers that pass the threshold&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In principle, you could even combine these 2 steps but I think it's cleaner to keep them apart unless storing 4^10 ~= 1 million decamers in memory is an issue, which it shouldn't be with modern computers.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Caveat emptor, this code may contain bugs and I haven't tested it other than making sure it runs.&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2024-02-02T20:39:08.923" LastActivityDate="2024-02-02T20:39:08.923" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22130" PostTypeId="1" CreationDate="2024-02-02T20:53:13.043" Score="1" ViewCount="34" Body="&lt;p&gt;I'm currently immersed in research involving the Rosa26-tdtomato (Ai9) mouse line. We performed scRNA seq and analysis focusing on tdtomato-expressing cells but faced a challenge with a lower-than-expected number of positive cells.&lt;/p&gt;&#xA;&lt;p&gt;Our alignment process, using the tdtomato sequence (Genbank accession: &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/AY678268&quot; rel=&quot;nofollow noreferrer&quot;&gt;AY678268&lt;/a&gt;), seems insufficient. Fellow researchers suggest success in similar cases by incorporating the full mRNA sequence, including the 3´UTR, along with tdtomato-WPRE mRNA.&lt;/p&gt;&#xA;&lt;p&gt;I searched everywhere for the full mRNA sequence but couldn't find it. Can anyone share insights or provide the complete mRNA sequence for tdtomato-WPRE? This would greatly enhance our alignment process and improve the accuracy of tdtomato positive cell detection.&lt;/p&gt;&#xA;&lt;p&gt;Appreciate your contributions to advancing our research!&lt;/p&gt;&#xA;" OwnerUserId="18911" LastEditorUserId="73" LastEditDate="2024-02-04T19:07:56.333" LastActivityDate="2024-02-04T19:12:13.970" Title="Can anyone share insights or provide the complete mRNA sequence for tdtomato-WPRE for scRNA seq analysis?" Tags="|scrnaseq|seurat|tdtomato|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22131" PostTypeId="2" ParentId="22130" CreationDate="2024-02-02T21:35:38.713" Score="2" Body="&lt;p&gt;I'd suggest looking at the mouse itself, which appears to be &lt;a href=&quot;https://www.informatics.jax.org/allele/MGI:3809523&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; on the JAX website.&lt;/p&gt;&#xA;&lt;p&gt;This link includes &lt;a href=&quot;https://www.informatics.jax.org/gxd/htexp_index/summary?mutantAlleleId=MGI:3809523&quot; rel=&quot;nofollow noreferrer&quot;&gt;RNA-seq experiments&lt;/a&gt;, so presumably you should be able to compare your pipeline and data to previous work to see whether there's anything weird there.&lt;/p&gt;&#xA;&lt;p&gt;Looking at the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2840225/&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt;, you can see the structure of the transgene which might be enough info to work from, there is also information in the supplement. You should in principle be able to figure out the sequence of each of those components and include them.&lt;/p&gt;&#xA;&lt;p&gt;If you are really having issues, you could do a transcriptome assembly to see whether you can recover the locus.&lt;/p&gt;&#xA;&lt;p&gt;You can always write to the senior author asking about this line, or try to figure out why your transcript may truly not be expressed.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/nktqZ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/nktqZ.png&quot; alt=&quot;ai9 transgene structure&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2024-02-02T21:35:38.713" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22132" PostTypeId="1" CreationDate="2024-02-03T12:40:06.220" Score="1" ViewCount="31" Body="&lt;p&gt;I'm currently working on a project where I'm comparing aggregate measurements (mean, median, etc.) of expression data (RNA-seq) from different groups of genes across various samples with different characteristics (tissue type, health status, etc.). Additionally, the raw counts were collected from several different labs using various techniques.&lt;/p&gt;&#xA;&lt;p&gt;Since I am conducting between-gene measurements, the data should be normalised to account for differences in transcript length and coverage depth (TPM, RPKM, FPKM). However, I am also interested in comparisons across samples based on tissue type and other factors. Therefore, the data should also be normalised to account for library size (TMM, quantile, etc.), and, as the data were collected from multiple sources, it should be corrected for batch effects.&lt;/p&gt;&#xA;&lt;p&gt;I have read through many papers but am unsure and confused about how to proceed with the normalisation procedure starting with the raw counts. Can I simply string the methods together, starting with batch effect correction, followed by library size normalisation, and then the within-sample normalisations?&lt;/p&gt;&#xA;&lt;p&gt;I would appreciate any insights or suggestions on this. Thanks&lt;/p&gt;&#xA;" OwnerUserId="18913" LastActivityDate="2024-02-04T19:03:46.140" Title="How to normalise Bulk RNA-seq data to account for transcript length, coverage depth, library size and batch effects?" Tags="|rna-seq|normalization|" AnswerCount="2" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22133" PostTypeId="1" CreationDate="2024-02-03T17:52:06.610" Score="1" ViewCount="18" Body="&lt;p&gt;I'm encountering a discrepancy in the depth of coverage assessment for specific coordinates when comparing results obtained from pileup files and samtools depth.&lt;/p&gt;&#xA;&lt;p&gt;I've processed pileup files and extracted coverage information for a given coordinate, resulting in a coverage depth of &lt;strong&gt;8028&lt;/strong&gt;. However, when using samtools depth on the same coordinate, the coverage depth is reported as &lt;strong&gt;88565&lt;/strong&gt;.&lt;/p&gt;&#xA;&lt;p&gt;My analysis involves the following steps:&lt;/p&gt;&#xA;&lt;p&gt;Alignment of paired-end reads using bwa mem.&#xA;Generation of pileup files and extraction of coverage information.&#xA;Utilizing samtools depth to calculate coverage depth for the same coordinates.&#xA;Visual inspection in IGV for validation, where I observe that the number of reads aligning to the region matches the count obtained from samtools depth.&#xA;I've noticed that in this region, there are no structural variations. The versions of samtools I'm using are samtools 1.19.2.&lt;/p&gt;&#xA;&lt;p&gt;Could you please help me understand the potential reasons for such big discrepancies? What parameters should I review, and could the differences be attributed to any specific factors?&lt;/p&gt;&#xA;&lt;p&gt;Thank you for your assistance!&lt;/p&gt;&#xA;" OwnerUserId="18915" LastActivityDate="2024-02-06T20:10:05.150" Title="Discrepancy in Depth of Coverage Estimation between Pileup and samtools depth" Tags="|samtools|coverage|mpileup|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22134" PostTypeId="2" ParentId="22132" CreationDate="2024-02-03T17:59:38.107" Score="2" Body="&lt;p&gt;All factors you mention bar the batch effects can be addressed by some sort of normalization (I use a modified version of &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5411036/&quot; rel=&quot;nofollow noreferrer&quot;&gt;UQpgQ2&lt;/a&gt;). Batch effects can be addressed using ComBat/SVA/ComBat-seq but my team's experience with them has been bad - they introduce artifacts and skew the data quite a bit especially if the batch effect is not pronounced.&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately, there is no way to do simple comparisons without huge caveats. You're better off using DESeq2 to perform statistically sound comparisons using the batch variable as a covariate.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2024-02-03T17:59:38.107" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22135" PostTypeId="1" CreationDate="2024-02-04T04:53:21.557" Score="1" ViewCount="33" Body="&lt;p&gt;I have a FIMO output, the input sequences were putative promoter sequences of several species. I want to graph the positions of these motifs along a horizontal graph, but I want to filter the data by clustering the motifs by family, e.g. all the zinc finger motifs being represented by one color, all the helix loop helix motifs being represented by one color, etc. How would I go about clustering the raw .tsv file?&lt;/p&gt;&#xA;" OwnerUserId="18137" LastActivityDate="2024-03-08T19:03:20.367" Title="How to classify TF motifs by family" Tags="|sequence-analysis|motifs|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22136" PostTypeId="2" ParentId="22132" CreationDate="2024-02-04T19:03:46.140" Score="1" Body="&lt;p&gt;Have a look through the DESeq2 manual. It goes into a lot of detail about how it works, and how you can use it to do all the things you're asking about:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The standard workflow already has all these things strung together, and works fairly well for the most common differential expression tasks:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;h1&gt;Quick start&lt;/h1&gt;&#xA;&lt;p&gt;Here we show the most basic steps for a differential expression analysis. There are a variety of steps upstream of DESeq2 that result in the generation of counts or estimated counts for each sample, which we will discuss in the sections below. This code chunk assumes that you have a count matrix called cts and a table of sample information called coldata. The design indicates how to model the samples, here, that we want to measure the effect of the condition, controlling for batch differences. The two factor variables batch and condition should be columns of coldata.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData = cts,&#xA;                             colData = coldata,&#xA;                             design= ~ batch + condition)&#xA;dds &amp;lt;- DESeq(dds)&#xA;resultsNames(dds) # lists the coefficients&#xA;res &amp;lt;- results(dds, name=&amp;quot;condition_trt_vs_untrt&amp;quot;)&#xA;# or to shrink log fold changes association with condition:&#xA;res &amp;lt;- lfcShrink(dds, coef=&amp;quot;condition_trt_vs_untrt&amp;quot;, type=&amp;quot;apeglm&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;/blockquote&gt;&#xA;" OwnerUserId="73" LastActivityDate="2024-02-04T19:03:46.140" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22137" PostTypeId="2" ParentId="22130" CreationDate="2024-02-04T19:12:13.970" Score="0" Body="&lt;p&gt;The Genbank sequence you've specified appears to be for RFP, not tdTomato.&lt;/p&gt;&#xA;&lt;p&gt;Have you tried one of the other &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/?term=tdtomato&quot; rel=&quot;nofollow noreferrer&quot;&gt;tdTomato sequences&lt;/a&gt;, for example a lentiviral sequence, or a sequence that is called tdTomato?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/KJ081243.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/nuccore/KJ081243.1&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2024-02-04T19:12:13.970" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22138" PostTypeId="2" ParentId="22135" CreationDate="2024-02-05T05:47:03.390" Score="0" Body="&lt;p&gt;Comment: I don't understand what you mean by &amp;quot;I want to graph the positions of these motifs along a horizontal graph&amp;quot;? Are you talking about a Manhatten plot? Or something else? And I'm not sure about &amp;quot;filter the data by clustering the motifs by family&amp;quot;. What type of clustering do you want to perform?&lt;/p&gt;&#xA;&lt;p&gt;Without further information/description it's difficult to understand what you're trying to do. If possible, please edit your question with an example of what your desired output. Also please indicate the language you're using for your visualisation, e.g. R, python, other.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Answer: I may have completely misunderstood, but perhaps this approach could be adapted to suit. Using the example FIMO output file from &lt;a href=&quot;https://meme-suite.org/meme/doc/examples/fimo_example_output_files/fimo.html?man_type=web&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://meme-suite.org/meme/doc/examples/fimo_example_output_files/fimo.html?man_type=web&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(tidyverse)&#xA;library(qqman)&#xA;#&amp;gt; &#xA;#&amp;gt; For example usage please run: vignette('qqman')&#xA;#&amp;gt; &#xA;#&amp;gt; Citation appreciated but not required:&#xA;#&amp;gt; Turner, (2018). qqman: an R package for visualizing GWAS results using Q-Q and manhattan plots. Journal of Open Source Software, 3(25), 731&#xA;#&amp;gt; &#xA;&#xA;fimo_df &amp;lt;- read_tsv(&amp;quot;~/Desktop/fimo.tsv&amp;quot;, comment = &amp;quot;#&amp;quot;) %&amp;gt;%&#xA;  rename(&amp;quot;SNP&amp;quot; = motif_id, &amp;quot;CHR&amp;quot; = sequence_name, &amp;quot;BP&amp;quot; = start,&#xA;         &amp;quot;P&amp;quot; = `p-value`, &amp;quot;zscore&amp;quot; = `q-value`) %&amp;gt;%&#xA;  mutate(CHR = case_when(str_detect(CHR, &amp;quot;X&amp;quot;) ~ &amp;quot;23&amp;quot;,&#xA;                         str_detect(CHR, &amp;quot;Y&amp;quot;) ~ &amp;quot;24&amp;quot;,&#xA;                         TRUE ~ CHR)) %&amp;gt;%&#xA;  mutate(CHR = parse_number(CHR)) %&amp;gt;%&#xA;  select(-c(stop, strand, matched_sequence))&#xA;#&amp;gt; Rows: 1494 Columns: 10&#xA;#&amp;gt; ── Column specification ────────────────────────────────────────────────────────&#xA;#&amp;gt; Delimiter: &amp;quot;\t&amp;quot;&#xA;#&amp;gt; chr (5): motif_id, motif_alt_id, sequence_name, strand, matched_sequence&#xA;#&amp;gt; dbl (5): start, stop, score, p-value, q-value&#xA;#&amp;gt; &#xA;#&amp;gt; ℹ Use `spec()` to retrieve the full column specification for this data.&#xA;#&amp;gt; ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message.&#xA;&#xA;manhattan(fimo_df, p = &amp;quot;zscore&amp;quot;, logp = TRUE, ylab = &amp;quot;q-score&amp;quot;,&#xA;          genomewideline = FALSE, suggestiveline = FALSE,&#xA;          main = &amp;quot;Manhattan plot of TF motifs&amp;quot;, &#xA;          chrlabs = c(1:19, &amp;quot;X&amp;quot;, &amp;quot;Y&amp;quot;))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.imgur.com/n1DHEK6.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you want to colour the points by TF motif (or by 'cluster'), perhaps ggplot would be appropriate, e.g.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(gghighlight)&#xA;axis_set &amp;lt;- fimo_df |&amp;gt;&#xA;  group_by(CHR) |&amp;gt;&#xA;  summarize(center = mean(BP, na.rm = TRUE))&#xA;&#xA;ylim &amp;lt;- fimo_df |&amp;gt;&#xA;  filter(P == min(P)) |&amp;gt;&#xA;  mutate(ylim = abs(floor(log10(P))) + 2) |&amp;gt;&#xA;  pull(ylim)&#xA;&#xA;ggplot(fimo_df, aes(x = BP, y = -log10(P),&#xA;                    color = as.factor(CHR), &#xA;                    size = -log10(P))) +&#xA;  geom_point(alpha = 0.75) +&#xA;  # scale_x_continuous(label = sort(axis_set&lt;span class=&quot;math-container&quot;&gt;$CHR), breaks = sort(axis_set$&lt;/span&gt;center)) +&#xA;  scale_y_continuous(expand = c(0, 0), limits = c(3.95, ylim)) +&#xA;  scale_color_manual(values = rep(&#xA;    c(&amp;quot;#276FBF&amp;quot;, &amp;quot;#183059&amp;quot;),&#xA;    unique(length(axis_set$CHR))&#xA;  )) +&#xA;  scale_size_continuous(range = c(0.5, 3)) +&#xA;  theme_minimal() +&#xA;  theme(&#xA;    legend.position = &amp;quot;none&amp;quot;,&#xA;    panel.grid.major.x = element_blank(),&#xA;    panel.grid.minor.x = element_blank(),&#xA;  ) +&#xA;  gghighlight(motif_alt_id == &amp;quot;TBP&amp;quot;)&#xA;#&amp;gt; Warning: Tried to calculate with group_by(), but the calculation failed.&#xA;#&amp;gt; Falling back to ungrouped filter operation...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;img src=&quot;https://i.imgur.com/oCDLI84.png&quot; alt=&quot;&quot; /&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;sup&gt;Created on 2024-02-05 with &lt;a href=&quot;https://reprex.tidyverse.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;reprex v2.1.0&lt;/a&gt;&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;p&gt;Does that help at all?&lt;/p&gt;&#xA;" OwnerUserId="9098" LastActivityDate="2024-02-05T05:47:03.390" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22139" PostTypeId="1" AcceptedAnswerId="22144" CreationDate="2024-02-05T15:53:03.773" Score="2" ViewCount="65" Body="&lt;p&gt;My inquiry pertains to WGCNA. After identifying the modules, I would like to extract genes within the different modules to create a dataframe with two columns: &lt;code&gt;Gene_a&lt;/code&gt; and &lt;code&gt;Gene_b&lt;/code&gt;. The values in these columns should represent co-expressed genes. An illustrative example is as follows:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WyULL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WyULL.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;In this example, the gene &lt;code&gt;XX62730x&lt;/code&gt; is co-expressed with &lt;code&gt;XCGR1Y&lt;/code&gt;. And &lt;code&gt;YYYYY&lt;/code&gt; is co-expressed with &lt;code&gt;ZZZZZZ&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;However, I am facing difficulties as the official WGCNA page is no longer accessible.&lt;/p&gt;&#xA;&lt;p&gt;I would greatly appreciate any help you can provide. Thank you in advance!&lt;/p&gt;&#xA;&lt;p&gt;Best,&lt;br /&gt;&#xA;Nana&lt;/p&gt;&#xA;" OwnerUserId="18917" LastEditorUserId="650" LastEditDate="2024-02-05T20:40:01.590" LastActivityDate="2024-02-06T10:25:10.110" Title="WGCNA: Extract genes within the different modules to create a dataframe with two columns" Tags="|r|rna-seq|wgcna|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22140" PostTypeId="1" CreationDate="2024-02-05T20:20:17.423" Score="0" ViewCount="26" Body="&lt;p&gt;I'm trying to run &lt;code&gt;hisatgenotype&lt;/code&gt; on a custom genome, originally specified in fasta format. I'm able to use &lt;code&gt;hisat2-build&lt;/code&gt; to build the &lt;code&gt;.1.ht2&lt;/code&gt;, ....&lt;code&gt;.8.ht2&lt;/code&gt; files, however I've not been able to figure out how to get the other files that &lt;code&gt;hisatgenotype&lt;/code&gt; requires. I've looked into &lt;code&gt;hisatgenotype_toolkit build-genome&lt;/code&gt; but it errors out due to problems with argument parsing. Is this even possible, or do I have to use the genomes/indices downloaded by the &lt;code&gt;setup.sh&lt;/code&gt; script?&lt;/p&gt;&#xA;&lt;p&gt;edit:&#xA;The error I get is the same as this issue: &lt;a href=&quot;https://github.com/DaehwanKimLab/hisat-genotype/issues/69&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/DaehwanKimLab/hisat-genotype/issues/69&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;% hisatgenotype_toolkit build-genome --base hla&#xA;Traceback (most recent call last):&#xA;  File &amp;quot;/Users/ted/miniforge-x86/envs/hisat/hisat-genotype/hisatgenotype_tools/hisatgenotype_build_genome.py&amp;quot;, line 546, in &amp;lt;module&amp;gt;&#xA;    typing_common.clone_hisatgenotype_database()&#xA;  File &amp;quot;/Users/ted/miniforge-x86/envs/hisat/hisat-genotype/hisatgenotype_modules/hisatgenotype_typing_common.py&amp;quot;, line 52, in wrapper&#xA;    func(*args, **kwargs)&#xA;TypeError: clone_hisatgenotype_database() missing 1 required positional argument: 'destination'&#xA;Script exited with error 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;If you try to run with an extra positional argument it fails too&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;% hisatgenotype_toolkit build-genome --base hla indices&#xA;usage: hisatgenotype_build_genome.py [-h] [--base BASE_FNAME]&#xA;                                     [--locus-list LOCUS_LIST]&#xA;                                     [--inter-gap INTER_GAP]&#xA;                                     [--intra-gap INTRA_GAP]&#xA;                                     [--aligner ALIGNER] [--linear-index]&#xA;                                     [--commonvar] [--clinvar] [-p THREADS]&#xA;                                     [-v]&#xA;hisatgenotype_build_genome.py: error: unrecognized arguments: indices&#xA;Script exited with error 2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And similar problems if you add &lt;code&gt;--destination indices&lt;/code&gt;&lt;/p&gt;&#xA;" OwnerUserId="18918" LastEditorUserId="18918" LastEditDate="2024-02-05T21:47:42.500" LastActivityDate="2024-02-05T21:47:42.500" Title="How to build hisatgenotype extra index files (.link, .locus, etc)" Tags="|genotyping|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22141" PostTypeId="1" CreationDate="2024-02-05T22:54:01.947" Score="0" ViewCount="23" Body="&lt;p&gt;I want to run some epigenome wide association studies based on city and pollutant type, could you please suggest if I can get DNA methylation sites (CpG) association based on pollutant type for each city. like below I am trying to find Pb toxic elements association site in four different cities which has varying level of Pb.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;#model matrix&#xA;var&amp;lt;-model.matrix(~log2Pb*City + sex + Age +CD8T +CD4T +NK + Bcell +Mono, data=targets2)&#xA;&#xA;fit&amp;lt;-lmFit(mval,var)&#xA;&#xA;fit2&amp;lt;-eBayes(fit,trend=TRUE, robust=TRUE)&#xA;&#xA;probe&amp;lt;-topTable(fit2,adjust=&amp;quot;BH&amp;quot;,coef=2,num=Inf)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;try to run interaction model using limma but not sure if this is the right approach&lt;/p&gt;&#xA;" OwnerUserId="1008" LastEditorUserId="650" LastEditDate="2024-02-05T23:00:49.897" LastActivityDate="2024-02-05T23:00:49.897" Title="limma linear model with interaction" Tags="|r|statistics|limma|epigenetics|biostatistics|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22142" PostTypeId="1" CreationDate="2024-02-06T00:41:11.423" Score="1" ViewCount="91" Body="&lt;p&gt;In certain cases, traditional significance test may not be able to identify that effect of a treatment or a drug is significant. However,clinical significance test may suggest a significant effect. Does it mean that this test is a better way of interpreting  effect-size or correlation coefficient when we are facing imperfect measurements or data. Imperfect measurements could produce a wrong conclusion based on traditional statistical test?&lt;/p&gt;&#xA;" OwnerUserId="7374" LastEditorUserId="7374" LastEditDate="2024-02-08T13:14:39.960" LastActivityDate="2024-02-08T13:14:39.960" Title="In medical and public health care, should we make use of &quot;clinical&quot; signficance testing (non-parametric tests) or parametric tests of significance?&quot;" Tags="|statistics|biostatistics|inference|non-parametric|" AnswerCount="0" CommentCount="5" ClosedDate="2024-02-06T14:42:41.760" ContentLicense="CC BY-SA 4.0" />
  <row Id="22144" PostTypeId="2" ParentId="22139" CreationDate="2024-02-06T10:25:10.110" Score="1" Body="&lt;p&gt;You can get this information, more or less, using the functions exportNetworkToVisANT and exportNetworkToCytoscape. You will find some extra columns that you can ignore or delete from the resulting table. We are still working with UCLA to restore the WGCNA page including the tutorials, but the tutorials can temporarily be accessed using &lt;a href=&quot;https://www.dropbox.com/scl/fo/4vqfiysan6rlurfo2pbnk/h?rlkey=3za224679kv8ulbpl5s9mwrkt&amp;amp;dl=0&quot; rel=&quot;nofollow noreferrer&quot;&gt;this dropbox link&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;To use either one of the two functions, you will need the network matrix for which we usually use TOM. You will probably want to set a threshold for connections to be included, otherwise you will get a fully connected network and the resulting table will (usually) be huge.&lt;/p&gt;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2024-02-06T10:25:10.110" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22146" PostTypeId="2" ParentId="22133" CreationDate="2024-02-06T20:10:05.150" Score="1" Body="&lt;p&gt;This is purely a guess based on what you write given that you are not showing the exact command for samtools etc.&lt;/p&gt;&#xA;&lt;p&gt;I suspect that you are allowing sequences with MAPQ==0 (supplementary alignments) into your depth calculation. A brief google search tells me that &lt;a href=&quot;https://www.biostars.org/p/350188/&quot; rel=&quot;nofollow noreferrer&quot;&gt;this issue has been previously observed&lt;/a&gt; with &lt;code&gt;samtools depth&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In principle, you should be able to see this directly in IGV, by controlling how supplementary alignments are displayed.&lt;/p&gt;&#xA;&lt;p&gt;This is something that &lt;a href=&quot;http://www.htslib.org/doc/samtools-depth.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;you can directly control with the &lt;code&gt;-Q&lt;/code&gt; flag&lt;/a&gt;, to restrict it to uniquely mapping reads (where you set the flag to e.g. &lt;code&gt;-Q 1&lt;/code&gt; or higher).&lt;/p&gt;&#xA;&lt;p&gt;I'd suggest trying to play with &lt;code&gt;samtools depth&lt;/code&gt; parameters and seeing if you can bring it more into alignment.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2024-02-06T20:10:05.150" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22147" PostTypeId="1" CreationDate="2024-02-06T21:34:39.323" Score="3" ViewCount="32" Body="&lt;p&gt;I have HiC reads from an HiC experiment and I am using HOMER to call loops and TADs. The numbers I am getting are like:&lt;/p&gt;&#xA;&lt;p&gt;Total TADs: 5024; Total Loops : 254&lt;/p&gt;&#xA;&lt;p&gt;As multiple loops are expected in a TAD so numbers of loops should be higher than the number of TADs. That's why I am not sure how to justify the numbers I am getting.&lt;/p&gt;&#xA;&lt;p&gt;I made sure I am not doing any mistake in the HOMER pipeline of loop/TAD calling and I followed all the steps in the tutorial here: &lt;a href=&quot;http://homer.ucsd.edu/homer/interactions2/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://homer.ucsd.edu/homer/interactions2/index.html&lt;/a&gt; Unfortunately I don't have any other loop calling program installed right now and HOMER is easy to install and run.&lt;/p&gt;&#xA;&lt;p&gt;If someone working with HiC data analysis and TAD/loop calling from HiC reads can please comment on this?&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;&lt;p&gt;Following is the set of commands I used (based on HOMER tutorial here &lt;a href=&quot;http://homer.ucsd.edu/homer/interactions2/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://homer.ucsd.edu/homer/interactions2/index.html&lt;/a&gt;) to get loop/TAD calls from HiC reads.&#xA;&lt;strong&gt;#Trimming/processing of HiC fastq files:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;homerTools trim -3 GATC -mis 0 -matchStart 20 -min 20 Hu_concat_1.fastq&lt;/p&gt;&#xA;&lt;p&gt;homerTools trim -3 GATC -mis 0 -matchStart 20 -min 20 Hu_concat_2.fastq&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;#Aligning the two paired-end fastq files to get sam:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;bowtie2 -p 20 -x hg38.fa -U Hu_concat_1.fastq.trimmed &amp;gt; Hu_concat_1.sam&lt;/p&gt;&#xA;&lt;p&gt;bowtie2 -p 20 -x hg38.fa -U Hu_concat_2.fastq.trimmed &amp;gt; Hu_concat_2.sam&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;#Making tag directory (the HOMER makeTagDirectory program is used to process the alignment file into a HOMER-style tag directory which is used for subsequent analysis steps)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;makeTagDirectory Hu_Hic_TagDir/ Hu_concat_1.sam,Hu_concat_2.sam -tbp 1&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;#Calling loops and TADs&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;findTADsAndLoops.pl find Hu_Hic_TagDir/ -cpu 10 -res 10000 -window 15000 -genome hg38.fa&lt;/p&gt;&#xA;&lt;p&gt;The whole process ran smoothly without errors. It is just that in the output, the number of loops I am getting are far fewer than no. of TADs (Total TADs: 5024; Total Loops : 254) which I think is not biologically correct. If someone using HOMER for calling loops and TADs from HiC reads can justify this output, that would be really helpful.&lt;/p&gt;&#xA;" OwnerUserId="18926" LastEditorUserId="18926" LastEditDate="2024-02-08T05:03:40.910" LastActivityDate="2024-04-04T15:58:56.077" Title="Loop and TAD calling from HiC reads using HOMER" Tags="|hi-c|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22148" PostTypeId="1" AcceptedAnswerId="22149" CreationDate="2024-02-06T22:09:42.633" Score="0" ViewCount="209" Body="&lt;p&gt;Given a GTF/GFF3 annotation file, what is the fastest way to convert it into a BED file?&lt;/p&gt;&#xA;" OwnerUserId="18305" LastActivityDate="2024-02-07T10:10:26.480" Title="Fastest way to convert GTF or GFF file to a BED file?" Tags="|bed|format-conversion|gtf|gff|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22149" PostTypeId="2" ParentId="22148" CreationDate="2024-02-06T22:09:42.633" Score="3" Body="&lt;h1&gt;gxf2bed&lt;/h1&gt;&#xA;&lt;p&gt;The fastest G{F,T}F-to-BED converter around the block!&lt;/p&gt;&#xA;&lt;p&gt;translates&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr27 gxf2bed gene 17266470 17285418 . + . gene_id &amp;quot;ENSG00000151743&amp;quot;;&#xA;chr27 gxf2bed transcript 17266470 17281218 . + . gene_id &amp;quot;ENSG00000151743&amp;quot;; transcript_id &amp;quot;ENST00000541931.8&amp;quot;;&#xA;chr27 gxf2bed exon 17266470 17266572 . + . gene_id &amp;quot;ENSG00000151743&amp;quot;; transcript_id &amp;quot;ENST00000541931.8&amp;quot;; exon_number &amp;quot;1&amp;quot;; exon_id &amp;quot;ENST00000541931.8.1&amp;quot;;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;into&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr27 17266469 17281218 ENST00000541931.8 1000 + 17266469 17281218 0,0,200 2 103,74, 0,14675,&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;before your eyes blink!&lt;/p&gt;&#xA;&lt;p&gt;Converts&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;em&gt;Homo sapiens&lt;/em&gt; GRCh38 GENCODE 44 (252,835 transcripts) in 3.34 seconds.&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Mus musculus&lt;/em&gt; GRCm39 GENCODE 44 (149,547 transcritps) in 2.11 seconds.&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Canis lupus familiaris&lt;/em&gt; ROS_Cfam_1.0 Ensembl 110 (55,335 transcripts) in 0.99 seconds.&lt;/li&gt;&#xA;&lt;li&gt;&lt;em&gt;Gallus galus&lt;/em&gt; bGalGal1 Ensembl 110 (72,689 transcripts) in 1.14 seconds.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;h2&gt;Usage&lt;/h2&gt;&#xA;&lt;pre class=&quot;lang-rust prettyprint-override&quot;&gt;&lt;code&gt;Usage: gxf2bed[EXE] --input/-i &amp;lt;GTF/GFF&amp;gt; --output/-o &amp;lt;BED&amp;gt;&#xA; &#xA;Arguments:&#xA;    --input/-i &amp;lt;GTF/GFF&amp;gt;: a .gtf/.gff file&#xA;    --output/-o &amp;lt;BED&amp;gt;: path to output .bed file&#xA;&#xA;Options:&#xA;    --help: print help&#xA;    --version: print version&#xA;    --threads/-t: number of threads (default: max ncpus)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;h2&gt;Installation&lt;/h2&gt;&#xA;&lt;p&gt;to install gxf2bed on your system follow this steps:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;get rust: &lt;code&gt;curl https://sh.rustup.rs -sSf | sh&lt;/code&gt; on unix, or go &lt;a href=&quot;https://www.rust-lang.org/tools/install&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt; for other options&lt;/li&gt;&#xA;&lt;li&gt;run &lt;code&gt;cargo install gxf2bed&lt;/code&gt; (make sure &lt;code&gt;~/.cargo/bin&lt;/code&gt; is in your &lt;code&gt;$PATH&lt;/code&gt; before running it)&lt;/li&gt;&#xA;&lt;li&gt;use &lt;code&gt;gxf2bed&lt;/code&gt; with the required arguments&lt;/li&gt;&#xA;&lt;li&gt;enjoy!&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h2&gt;Build&lt;/h2&gt;&#xA;&lt;p&gt;to build gxf2bed from this repo, do:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;get rust (as described above)&lt;/li&gt;&#xA;&lt;li&gt;run &lt;code&gt;git clone https://github.com/alejandrogzi/gxf2bed.git &amp;amp;&amp;amp; cd gxf2bed&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;run &lt;code&gt;cargo run --release -- -i &amp;lt;GTF/GFF&amp;gt; -o &amp;lt;BED&amp;gt;&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h2&gt;Container image&lt;/h2&gt;&#xA;&lt;p&gt;to build the development container image:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;run &lt;code&gt;git clone https://github.com/alejandrogzi/gxf2bed.git &amp;amp;&amp;amp; cd gxf2bed&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;initialize docker with &lt;code&gt;start docker&lt;/code&gt; or &lt;code&gt;systemctl start docker&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;build the image &lt;code&gt;docker image build --tag gxf2bed .&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;run &lt;code&gt;docker run --rm -v &amp;quot;[dir_where_your_gtf_is]:/dir&amp;quot; gxf2bed -i /dir/&amp;lt;GTF/GFF&amp;gt; -o /dir/&amp;lt;BED&amp;gt;&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h2&gt;Conda&lt;/h2&gt;&#xA;&lt;p&gt;to use gxf2bed through Conda just:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;code&gt;conda install gxf2bed -c bioconda&lt;/code&gt; or &lt;code&gt;conda create -n gxf2bed -c bioconda gxf2bed&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;h2&gt;Benchmark + FAQ&lt;/h2&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/vzjos.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/vzjos.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you google &amp;quot;gtf to bed&amp;quot; or &amp;quot;gff to bed&amp;quot; you'll find some posts about recommended tools. Part of these tools are already deprecated or too difficult to run (even in Linux), due to poor maintenance. This project was conceived to provide an easy way to convert GTF/GFF files to BED structures and finish a set of high-performance converters between gene model formats in Rust (bed2gtf, bed2gff and now gxf2bed).&lt;/p&gt;&#xA;&lt;p&gt;To test the efficiency of gxf2bed, I took 4 GTF/GFF-to-BED converters and ran them with the current GRCh38 GENCODE annotation (v.44), that has ~250,000 transcripts [the values showed here are the mean of 5 consecutive runs]. Briefly, the competitors are:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;UCSC's utils: &lt;code&gt;gtfToGenePred | genePredToBed&lt;/code&gt; &amp;amp; &lt;code&gt;gffToGenePred | genePredToBed&lt;/code&gt; (taken from &lt;a href=&quot;https://hgdownload.soe.ucsc.edu/admin/exe/linux.x86_64/&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;Signal &amp;amp; Brown: &lt;code&gt;gtf2bed.py&lt;/code&gt; &amp;amp; &lt;code&gt;gff32gtf.py &amp;amp;&amp;amp; gtf2bed.py&lt;/code&gt; (taken from &lt;a href=&quot;https://github.com/signalbash/how_are_we_stranded_here/tree/master/how_are_we_stranded_here&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;ea-utils: &lt;code&gt;gtf2bed.pl&lt;/code&gt; &amp;amp; &lt;code&gt;gff2gtf.pl | gtf2bed.pl&lt;/code&gt; (taken from &lt;a href=&quot;https://github.com/ExpressionAnalysis/ea-utils/tree/master/clipper&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;li&gt;BEDOPS: &lt;code&gt;gtf2bed&lt;/code&gt; &amp;amp;  &lt;code&gt;gtf2bed&lt;/code&gt; (taken from &lt;a href=&quot;https://github.com/bedops/bedops/tree/master/applications/bed/conversion/src/wrappers&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Besides the easy way to make &lt;code&gt;gxf2bed&lt;/code&gt; run and the GFF-GTF channel that allows centralize both in a single converter, this tool showed a significant difference in computation time against the other tools in each one of the two formats. Even using a combination of different number of threads, times were practically the same 3.5s +/- 0.4s and the differences were maintained. Is important to note that GFF3 files took a lot more time than its GTF counterpart. This could be due to the need of chaining two different programs 1) GFF-to-GTF and 2) GTF-to-BED.&lt;/p&gt;&#xA;&lt;p&gt;Taken together, &lt;code&gt;gxf2bed&lt;/code&gt; offers an easier and faster way to convert GTF/GFF files to BED files. This tool could save you at least x2-3 times the time you used to spend using other tools to convert GTF files and x5 times if you want to convert GFF3 files!&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Disclaimer: I am the author of the gxf2bed tool.&lt;/p&gt;&#xA;" OwnerUserId="18305" LastEditorUserId="298" LastEditDate="2024-02-07T10:10:26.480" LastActivityDate="2024-02-07T10:10:26.480" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="22152" PostTypeId="1" AcceptedAnswerId="22156" CreationDate="2024-02-07T08:56:53.883" Score="3" ViewCount="50" Body="&lt;p&gt;Right now I have six genomes that I want to compare and identify homologous regions in the genomes. I have run nucmer, show-coords and obtained the output files. An example is shown below with Genome 1 vs Genome 2. More files go through Genome 1 vs Genome 3 until all are compared.&lt;/p&gt;&#xA;&lt;p&gt;I was trying to use bedtools merge after concatenating the multiple output files from show-coords command to find the homologous regions. However, I am not quite sure on the command to use. Any help is appreciated.&lt;/p&gt;&#xA;&lt;p&gt;Edit: added commands. I just want to know what regions are homologous across the six genomes (99/100% IDY). I don't know what I am supposed to merge to find this out. I was reading and I believe bedtools should be but I don't know. I am usually a wetlab but I try to do a bit of bioinformatics.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt; # Loop through all pairs of genomes&#xA;for i in {1..6}; do&#xA;    for j in &lt;span class=&quot;math-container&quot;&gt;$(seq $&lt;/span&gt;((i + 1)) 6); do&#xA;        genome1=&amp;quot;genome&lt;span class=&quot;math-container&quot;&gt;${i}.fasta&quot;&#xA;        genome2=&quot;genome$&lt;/span&gt;{j}.fasta&amp;quot;&#xA;&#xA;        output_prefix=&amp;quot;output_&lt;span class=&quot;math-container&quot;&gt;${i}_$&lt;/span&gt;{j}&amp;quot;&#xA;&#xA;        # Run NUCmer&#xA;        nucmer -p &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${output_prefix}&quot; &quot;$&lt;/span&gt;genome1&amp;quot; &amp;quot;$genome2&amp;quot;&#xA;&#xA;        # Show coordinates&#xA;        show-coords -r -l -c -T &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${output_prefix}.delta&quot; &amp;gt; &quot;$&lt;/span&gt;{output_prefix}_coords.txt&amp;quot;&#xA;    done&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After using the above I then concatenate the files with&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;cat output_*_coords.txt &amp;gt; combined_coords.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This gives me something that looks like this below.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;[S1]    [E1]    [S2]    [E2]    [LEN 1] [LEN 2] [% IDY] [LEN R] [LEN Q] [COV R] [COV Q] [TAGS]&#xA;1       14347   1       14347   14347   14347   99.02   5064019 5004926 0.28    0.29    Genome1      Genome2&#xA;14474   32175   14350   32051   17702   17702   99.33   5064019 5004926 0.35    0.35    Genome1      Genome2&#xA;16541   22042   4698474 4692975 5502    5500    99.64   5064019 5004926 0.11    0.11    Genome1      Genome2&#xA;16548   22042   3728461 3722968 5495    5494    99.84   5064019 5004926 0.11    0.11    Genome1      Genome2&#xA;16583   22041   3904238 3898781 5459    5458    99.65   5064019 5004926 0.11    0.11    Genome1      Genome2&#xA;18727   22097   4426876 4423506 3371    3371    99.73   5064019 5004926 0.07    0.07    Genome1      Genome2&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18927" LastEditorUserId="18927" LastEditDate="2024-02-07T10:24:44.370" LastActivityDate="2024-02-08T11:17:49.313" Title="Finding homologous regions in multiple whole genomes" Tags="|sequence-alignment|bedtools|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22153" PostTypeId="1" AcceptedAnswerId="22154" CreationDate="2024-02-07T11:55:37.950" Score="4" ViewCount="88" Body="&lt;p&gt;I have some fastq files (obtained from nanopore sequencing) that contain reads that can be of either of these 5 forms:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;a known CDS with 3'UTR:&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;CDS----------------------- (seq1: original sequence) &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;the same cds with a block of 50bp represented by &lt;code&gt;|||||&lt;/code&gt; (that substitutes the 3'UTR at 4 different locations)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;CDS|||||------------------ (sub1: substitution 1)&#xA;CDS-----|||||------------- (sub2: substitution 2)&#xA;CDS----------|||||-------- (sub3: substitution 3)&#xA;CDS---------------|||||--- (sub4: substitution 4)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I need to split my fastq files in 5 smaller files clustering the different types of sequences (seq1, sub1, sub2, sub3 and sub4). The problem is that those sequences all share some segments and I don't know which method I should apply. I tried finding the best match by building a BLAST database with the 5 sequences but as it is local alignments, I get HSPs for each sequences and cannot discriminate between them.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone has an idea how I could achieve that?&lt;/p&gt;&#xA;&lt;p&gt;Any help would be amazing.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;edit&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;Here are 5 sequences as I expect them (i.e., fasta reference)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;seq1 (this one does not have the 50bp block)&#xA;ACCTGCCAGTTCCTGAAGGGTGTACTGATCCTGTGGCTGAAAACTTTGATCCAACGGCTAGAAGTGACGATGGAACCTGTGTCTACAACTTTTGAGCAATATTATCCTGCTTATTAATTTGCTGTTTTACTCCTATTGTCTCTTTTGGTTTATTTTTCTCCTTTGTGTAATTGTGGATTGGATCTTGTCCTCTTTTGTTCCCTTTTTTTTTTTTTATGATGTACAACACATTGGTAATTTAAAATTGCCTTGTCATAAA&#xA;&amp;gt;sub1&#xA;_GTAAGTGAGGGAGTCTCGGACTCAATTAGAGGCTCTTCTTTCACA_ TTGATCCAACGGCTAGAAGTGACGATGGAACCTGTGTCTACAACTTTTGAGCAATATTATCCTGCTTATTAATTTGCTGTTTTACTCCTATTGTCTCTTTTGGTTTATTTTTCTCCTTTGTGTAATTGTGGATTGGATCTTGTCCTCTTTTGTTCCCTTTTTTTTTTTTTATGATGTACAACACATTGGTAATTTAAAATTGCCTTGTCATAAA&#xA;&amp;gt;sub2&#xA;ACCTGCCAGTTCCTGAAGGGTGTACTGATCCTGTGGCTGAAAACT _GTAAGTGAGGGAGTCTCGGACTCAATTAGAGGCTCTTCTTTCACA_ TTTGAGCAATATTATCCTGCTTATTAATTTGCTGTTTTACTCCTATTGTCTCTTTTGGTTTATTTTTCTCCTTTGTGTAATTGTGGATTGGATCTTGTCCTCTTTTGTTCCCTTTTTTTTTTTTTATGATGTACAACACATTGGTAATTTAAAATTGCCTTGTCATAAA&#xA;&amp;gt;sub3&#xA;ACCTGCCAGTTCCTGAAGGGTGTACTGATCCTGTGGCTGAAAACTTTGATCCAACGGCTAGAAGTGACGATGGAACCTGTGTCTACAACT _GTAAGTGAGGGAGTCTCGGACTCAATTAGAGGCTCTTCTTTCACA_ TTGTCTCTTTTGGTTTATTTTTCTCCTTTGTGTAATTGTGGATTGGATCTTGTCCTCTTTTGTTCCCTTTTTTTTTTTTTATGATGTACAACACATTGGTAATTTAAAATTGCCTTGTCATAAA&#xA;&amp;gt;sub4&#xA;ACCTGCCAGTTCCTGAAGGGTGTACTGATCCTGTGGCTGAAAACTTTGATCCAACGGCTAGAAGTGACGATGGAACCTGTGTCTACAACTTTTGAGCAATATTATCCTGCTTATTAATTTGCTGTTTTACTCCTA _GTAAGTGAGGGAGTCTCGGACTCAATTAGAGGCTCTTCTTTCACA_ GATCTTGTCCTCTTTTGTTCCCTTTTTTTTTTTTTATGATGTACAACACATTGGTAATTTAAAATTGCCTTGTCATAAA&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;examples (50bp block if present is in lowercase here)&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;this read should be associated with sub3 because of the flanking sequences (...CTACAACT in 5' and TTGTCTCTTTT... in 3' of the 50 bp block)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;@e213fdd5-bbfb-4dfe-9b03-29eea256da0d &#xA;TGTGTACTTCGTTCAGTTACGTATTACTAGTTATTGAGTGTCTTTGTGTTTCTGTTGGTGCGTCTTCGCACAAGGCTAATCTTACTCTCCTCTCCAATGTACTGCTTCTATGTTCATCTCAGCCGAATTCAATAAGGAGAAGAACTTTCACTAGGTTTCCCCATTCTGTTCGATTAATTGGTGATGTTAGTAAATTTCAATTTTCTGTCGGTGGAAGGTGAAGGTGATTGCTGCTGGCTTTTCATTGTTTACTTTTGGCTTCTGTTTTGTAGTAACGCATCACTACTTTCTCTTATAGTGTTCAATGCTTTCCAAGTCTGGAATCATGAAGCAGCGGCTTCTTCAAGAGCGCCATGCCTGAGGGATACGTGCAGGAGAGGACCATCTTCTTCAAGGACGACGGGAACTACAAGACGTGCTGAAGTCAAGTTTGGGGAGACACCCTCGTCAATGAGATCGAGCTTAAGGGAATCGATTTCAAGGAGGACGGAAACATCCTCGGCCACAAGTTGGAATACAACTACAACTCCCACAACGTATACATCATAGCCGACAAAACCCAAAAGACGGCATCAAGAAACCAACTTCAGACCTTACCATCAGGCGGCGGCCATCACACAGTCATTGTCAACAAATGTAATTGGCGGTGGCTATTCTTTTTACCCAGACAACCATTACCTTGTCCACACAGTCACACTGTGAAAAGATCCCAACTTTGAAAGAGAGGCCACTGGTCTTCTTGAGTTTGTAACAACTGCTGGGATTACACATGGCATGGATGAACTATACAAACATGACAGACTCTAAAGCCTGCAGTTCCTGAAGAGATTGTACTGATCCTGTGGCTGAAAACTTTGATCCAACGGCTAGAAGTGACGATGGAACCTGTGTCTACAACTgtaagtgagggagtctcggactcaattagaggctcttctttcacaTTGTCTCTTTTGGTTTATTTTCTCCTTTGTGTAATTGTGGATTGGATCTTGTCCTCTTTTGTTTTCCTTTTTTTTTTTTATGATGTACAACACATTGGTAATTTAAAATTGCCTTGTCATGCAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACTGTAGGCACCATCAATGAAGATAGAGCGACAGGCAAGTCACAAAAACACCGACAACTTTCTTGTCACGGTAGGCGATAGCAATACGTAACTG&#xA;+&#xA;$&amp;amp;&amp;amp;&amp;amp;%234441124--585545;&amp;gt;B99968650-++*()'&amp;amp;'(;D540488?=888,++*+*)*$$$##&amp;amp;%$$$%)*****('%$$'1/+++-,('))'%$$%$$$&amp;amp;)&amp;amp;%%%&amp;amp;(*'&amp;amp;),/'''&amp;amp;&amp;amp;'06872..*&amp;amp;%)),,3/49:;&amp;lt;?A773)('()/066+'&amp;amp;&amp;amp;)('(+&amp;amp;%$&amp;amp;''$$&amp;amp;&amp;amp;()*--888)('()'')')'''++*+)&amp;amp;')&amp;amp;$$$%()'')*''&amp;amp;&amp;amp;))&amp;amp;'&amp;amp;$&amp;amp;(&amp;amp;$#$%&amp;amp;%&amp;amp;&amp;amp;&amp;amp;%%$%&amp;amp;..&amp;amp;%%%%&amp;amp;&amp;amp;%$%&amp;amp;'&amp;amp;$$'-)(%%%$&amp;amp;(*,+****&amp;amp;'''('+17&amp;lt;{..+*('((*+)*&amp;amp;&amp;amp;&amp;amp;'**.*%$&amp;amp;%&amp;amp;&amp;amp;%$$$%&amp;amp;&amp;amp;'&amp;amp;&amp;amp;(+'(&amp;amp;&amp;amp;&amp;amp;%%&amp;amp;&amp;amp;($$%&amp;amp;&amp;amp;'+55536988;;74632+(*)''%6&amp;lt;=(((&amp;amp;*''(68998===?ABBBB=B?&amp;lt;;;96667&amp;gt;&amp;gt;?&amp;gt;&amp;gt;?@?A=-+.7556&amp;lt;@&amp;gt;A&amp;lt;0//002(()-989::::;&amp;lt;&amp;lt;&amp;gt;&amp;gt;;:5++++-&amp;gt;;::;&amp;lt;?F:7&amp;lt;()988;;&amp;lt;?&amp;gt;9;9)36:5-963/002B@A@@?554543,,9&amp;lt;=&amp;lt;@?@&amp;gt;777461166.../B88&amp;lt;;&amp;lt;@=&amp;lt;:888;;&amp;gt;42(''%%(%%+*('&amp;amp;%%&amp;amp;&amp;amp;**)(()*+020*'('&amp;amp;$$$'('''&amp;amp;%'((%(&amp;amp;$##$&amp;amp;)*()&amp;amp;%%%$$#$%'&amp;amp;(&amp;amp;'%$$%%&amp;amp;'%%%$%%%$&amp;amp;('$##$&amp;amp;&amp;amp;&amp;amp;&amp;amp;$$$%$%$$%&amp;amp;%%&amp;amp;040(((&amp;amp;'.023266,,,++0)++-.-*,,0+&amp;amp;&amp;amp;'$&amp;amp;''''%%'&amp;amp;-+,0((+))*+*(%&amp;amp;)+00.3)''%'&amp;amp;'''(*(%%)++++,--+.....0'(()7:&amp;lt;=;=&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;0.&amp;lt;;9:;&amp;lt;@;;;:&amp;lt;&amp;lt;=;2+)++,1212611)())*:1++*'&amp;amp;&amp;amp;%((&amp;amp;&amp;amp;056/,454))*''''(3346;&amp;lt;;;;;;000.0/043546??=;&amp;lt;*)+64335663455665333788:78779&amp;lt;&amp;gt;9755446::&amp;lt;&amp;gt;&amp;lt;198662*+7899722248867&amp;gt;?8++4))+6779&amp;lt;&amp;lt;BA&amp;gt;&amp;lt;9;&amp;lt;;:;;{&amp;lt;?BB-,,-13444/154///933{3==@B:9989::::::;;&amp;lt;&amp;lt;=:3335=EBB;:*)..'-:69?@CKKKDC&amp;lt;&amp;lt;?&amp;gt;:)()((012.,,+,55;=?&amp;gt;@&amp;gt;:98:78544344*)%$$$&amp;amp;)))))))))(((((''''&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;&amp;amp;')+.1866677978:;;:::;;;99::988889653456:::9889;::8899.,+0668:;:::8436735//:632/-,+**+013459;9332/+($&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;this read should be associated with sub4 because of the flanking sequences (...TTTTACTCCTA in 5' even if a C is missing, and GATCTTGTCCT... in 3')&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;pre&gt;&lt;code&gt;@107dfd6d-2101-4fd8-aeb5-2c432e176d72 &#xA;ATGTACTTCGTTCAGTTACGTATTGCTATCGCCTACCGTGACAAGAAAGTTGTCGGTGTCTTTGTGTTTCTGTTGGTGCTGATATTGCATGAAGACTAATCTTTTCTCTTTCTCATCTTTTCACTTCTCCTATCATTATCCTCGACCGAATTCAGTAAAGGAGAAGAACTTTTCACTGGGTTGTCCCAATTCTTGTTGAATTAGATGGTGATGTTAATGGGTACAAATTTTCTGTCAGTGGAAGAGGGTGAAGGTGATGCAACATACGGAAAACTTACCCTTAAATTTATTTGCACTACTGGAAAACTACCTGTTCCATGGCCAACACTTGTCACTACTTTCTCTTATGGTGTTCAATGCTTTCAAGATACCAGATCATATGAAGCGGCACGACTTCTTCAAGAGCGCCATGCCTGAGGGATACGTGCGGGAGAGGACCATCTTCTTCAAGGACGACGGGAACTACAAGACACGTGCTGAAGTCAAGTTTGAGGAGACACCCTCGTCAACAGGATCGAGCTTAAGGGAATCGATTTCAAGGAGGACGGAAACATCCTCGGCCACAAGTTGGAATACAACCGCAACTCCCACAACGTATACATCATGGCCGACAAGCAAAAGAACGGCATCAAAGCCAACTTCAAGACCCGCCACAACATCGAAGACGGCGGCGTGCAACTCGCTGATCATTATCAACAAAATACTCCAATTGGCGATGGCCTGTCCTTTTACCAGACAACCATTACCTGTCCACACAATCTGCCCTTTCGAAAGATCCCAACGAAAAATTTTTGCATGGTCCTTCTTGAGTTTGTAACGGCTGCTGGGATTACATGGCATGGATGAAGAACTGCAACACAAACACTTGACGAACTCTAAACCTGCCAGTTCCTGAAGAAATTGTACTGATCCTGTGGCTGAAAACTTTGATCCAACGGCTAGAAAGTGACGATGGAACCTGTGTCACCAACTTTTGAGCAATATTATCCTGCTTATTAATTTGCTGTTTTACTCTAgtaagtgagggagtctcggactcaattagaggctcttctttcacaGATCTTGTCCTCTTTTGTTCCCTTTTTTTTTTTATGATGTACAACACATTGGTAATTTAAAATTGCCTTGTCATAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACTGTAGGCACCATCAATGAAGATAGAGCGACAGGCAAGTCACAAAGACACCGACAACTTTCTTGTCACGGTAGGCGATAGCAATACGTAACT&#xA;+&#xA;-0,+))),:::&amp;lt;??===&amp;gt;=&amp;lt;&amp;lt;&amp;lt;?@&amp;gt;=&amp;lt;&amp;lt;=?;:889:&amp;lt;;&amp;lt;&amp;lt;=;;;;CD@?:9:A&amp;gt;9999=999DBAA@&amp;lt;BE=3220111&amp;lt;B@@AA@A?&amp;gt;&amp;gt;&amp;gt;=&amp;gt;&amp;gt;;:9999:9;&amp;lt;E9A&amp;lt;;?????C8788;==@?;;;==9989@A6677?=7788+**+368&amp;gt;B87::==@&amp;lt;99;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;&amp;lt;??&amp;gt;@=:9/.))*698.*,/8A=&amp;lt;==&amp;gt;&amp;gt;=?BAB=::::;;;&amp;lt;?@&amp;gt;@B@@?:888:&amp;gt;?@CCCEDB&amp;gt;??&amp;gt;===::.+)()2157=7:99&amp;lt;B?&amp;gt;???@??=;::=DGC=&amp;lt;6?;84568&amp;gt;AB@CCBCBAA====AAACEI{AA=&amp;lt;::;&amp;lt;?@=&amp;lt;&amp;lt;&amp;gt;=&amp;gt;@AAA@&amp;gt;&amp;gt;&amp;gt;&amp;gt;?&amp;lt;?====A?==&amp;lt;=@&amp;gt;??@&amp;gt;?==??CA@AAA@AB=757:965(*.;&amp;lt;;&amp;lt;::?AAA@&amp;gt;=&amp;gt;::::&amp;lt;=?@BCC5336899=?&amp;gt;&amp;gt;?@A??42220;9:89.-***+77;97&amp;lt;;:;&amp;lt;&amp;lt;??BC{878&amp;lt;:9:&amp;lt;:2112&amp;lt;&amp;lt;&amp;gt;===@=&amp;lt;;;&amp;lt;&amp;gt;???877/,.-.-///0,++,)299:=;:;;=@@&amp;lt;;=7778?=::;;&amp;lt;;=&amp;lt;&amp;lt;C&amp;gt;AB&amp;lt;BBBCEDCDC?A&amp;gt;;;9:655:DAA@?&amp;gt;&amp;gt;&amp;gt;=?=&amp;gt;&amp;lt;;&amp;lt;&amp;lt;@=&amp;gt;&amp;gt;&amp;lt;=B=@=:870++,,;=;&amp;gt;&amp;gt;&amp;lt;&amp;lt;==CBA@?&amp;gt;=&amp;gt;&amp;gt;=667&amp;lt;;;7778B&amp;gt;B?B&amp;lt;&amp;lt;==&amp;lt;99:::4:659;6642212:.-&amp;lt;=??=6556;??@@&amp;gt;::;:==;8887899==;2143235&amp;lt;&amp;lt;&amp;lt;?&amp;gt;==?CEBD?&amp;gt;&amp;gt;&amp;gt;ADB=720/.02334&amp;lt;00//06568&amp;gt;:888:711*)-776'''&amp;amp;06:&amp;lt;=&amp;gt;&amp;lt;&amp;gt;&amp;lt;866;{C@A?&amp;gt;8897=&amp;gt;EK558568552-179664)(()),,--,()))3:;:&amp;lt;BAAA&amp;gt;=&amp;gt;&amp;gt;&amp;gt;&amp;gt;&amp;lt;&amp;lt;&amp;gt;&amp;gt;*))+/245999;&amp;gt;&amp;gt;=96;?B9888&amp;lt;&amp;lt;77-,,,,&amp;amp;&amp;amp;%%%&amp;amp;&amp;amp;'+6;A;2111--,,.:;&amp;gt;&amp;lt;&amp;lt;ACCA@@?=9;;;&amp;gt;=&amp;gt;==@930../+)(())07&amp;gt;&amp;gt;8&amp;lt;&amp;gt;&amp;lt;;:;9238&amp;lt;&amp;gt;=&amp;gt;998878:7700016545**1236434689&amp;lt;9&amp;lt;&amp;lt;;.--0-,,,&amp;amp;'():78862237@???&amp;gt;&amp;lt;;;;;&amp;lt;&amp;lt;=@&amp;gt;@??@@===;&amp;lt;=&amp;gt;&amp;gt;;:8;)(67511223/.---.2001023:;21&amp;lt;?=**6223;;&amp;gt;?AACGA=73366:;&amp;lt;=:766;ACC=&amp;lt;;86000?8=@DHEC@@=;;&amp;lt;9999&amp;lt;&amp;lt;==9&amp;gt;&amp;lt;==8889&amp;gt;A&amp;gt;=;30266777:972+***)))'&amp;amp;%%%%%$$$$$$######&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;&amp;quot;#&amp;amp;')+-1122/...46688&amp;lt;&amp;lt;=0('%%&amp;amp;258899;92136:999;:::9:;;--2-.343668=&amp;lt;?@EF????==:7668;6667;&amp;gt;&amp;lt;&amp;lt;;;&amp;gt;;;8851.*&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18928" LastEditorUserId="18928" LastEditDate="2024-02-07T15:31:05.727" LastActivityDate="2024-02-08T15:31:25.727" Title="split fastq file containing a sequence block at different locations" Tags="|sequence-alignment|ngs|clustering|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22154" PostTypeId="2" ParentId="22153" CreationDate="2024-02-07T17:09:12.453" Score="4" Body="&lt;p&gt;You can try &lt;a href=&quot;https://bioinf.shenwei.me/seqkit/usage/#locate&quot; rel=&quot;nofollow noreferrer&quot;&gt;seqkit locate&lt;/a&gt; with say start or end of CDS sequence. This will give you a tabulated output with locations of your CDS query in each read.&#xA;Since you expect that the CDS sequence should be located at one of the five different locations inside your reads you can figure out to which one of the five a given read belongs.&lt;/p&gt;&#xA;&lt;p&gt;You may also use &lt;a href=&quot;https://github.com/BioInfoTools/BBMap/blob/master/sh/clumpify.sh&quot; rel=&quot;nofollow noreferrer&quot;&gt;clumpify.sh from BBMap&lt;/a&gt; to cluster reads prior to searching for the CDS locations. The idea is that maybe the five groups will cluster in fairly well separated blocks, but this is fairly speculative.&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;edit&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;Since the alignment of your sequences looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;CLUSTAL 2.1 multiple sequence alignment&#xA;&#xA;&#xA;sub1            NNNNNNNNNNGTAAGTGAGGGAGTCTCGGACTCAAT---TAGAGGCTCTTCTTTCACANN&#xA;sub2            --ACCTGCCAGTTCCTGAAGGGTGTACTGATCCTGTGGCTGAAAACTNNNNNNNNNNGTA&#xA;seq1            --ACCTGCCAGTTCCTGAAGGGTGTACTGATCCTGTGGCTGAAAACTTTGATCCAACGG-&#xA;sub3            --ACCTGCCAGTTCCTGAAGGGTGTACTGATCCTGTGGCTGAAAACTTTGATCCAACGGC&#xA;sub4            --ACCTGCCAGTTCCTGAAGGGTGTACTGATCCTGTGGCTGAAAACTTTGATCCAACGG-&#xA;                          **   *** **     * **  *  *   *  *  **             &#xA;&#xA;sub1            NNNNNNNNTTGATCCAACGGCTAGAAGTGACGATGGAACCTG---TGTCTACAACTTTTG&#xA;sub2            AGTGAGGGAGTCTCGGACT-CAATTAGAGGCTCTTCTTTCAC---ANNNNNNNNNNTTTG&#xA;seq1            --------------------CTAGAAGTGACGATGGAACCTG---TGTCTACAACTTTTG&#xA;sub3            TAGAAGTGACGATGGAACCTGTGTCTACAACTNNNNNNNNNNGTAAGTGAGGGAGTCTCG&#xA;sub4            --------------------CTAGAAGTGACGATGGAACCTG---TGTCTACAACTTTTG&#xA;                                              *                          * *&#xA;&#xA;sub1            AGCAATATTATCCTGCTTATTAATTTGCTGTTTTACTCCTATTGTCTCTTT-----TGGT&#xA;sub2            AGCAATATTATCCTGCTTATTAATTTGCTGTTTTACTCCTATTGTCTCTTT-----TGGT&#xA;seq1            AGCAATATTATCCTGCTTATTAATTTGCTGTTTTACTCCTATTGTCTCTTT-----TGGT&#xA;sub3            GACTCAATTAGA--GGCTCTTCTTTCACANNNNNNNNNN--TTGTCTCTTT-----TGGT&#xA;sub4            AGCAATATTATCCTGCTTATTAATTTGCTGTTTTACTCCTANNNNNNNNNNGTAAGTGAG&#xA;                  *   ****    *  * **  **  *                            **  &#xA;&#xA;sub1            TTATTTTT---------------CTCCTTTGTGTAATTGTGGATTGGATCTTGTCCTCTT&#xA;sub2            TTATTTTT---------------CTCCTTTGTGTAATTGTGGATTGGATCTTGTCCTCTT&#xA;seq1            TTATTTTT---------------CTCCTTTGTGTAATTGTGGATTGGATCTTGTCCTCTT&#xA;sub3            TTATTTTT---------------CTCCTTTGTGTAATTGTGGATTGGATCTTGTCCTCTT&#xA;sub4            GGAGTCTCGGACTCAATTAGAGGCTCTTCTTTCACANNNNNNNNNNGATCTTGTCCTCTT&#xA;                  * * *                *** * * *   *          **************&#xA;&#xA;sub1            TTGTTCCCTTTTTTTTTTTTTATGATGTACAACACATTGGTAATTTAAAATTGCCTTGTC&#xA;sub2            TTGTTCCCTTTTTTTTTTTTTATGATGTACAACACATTGGTAATTTAAAATTGCCTTGTC&#xA;seq1            TTGTTCCCTTTTTTTTTTTTTATGATGTACAACACATTGGTAATTTAAAATTGCCTTGTC&#xA;sub3            TTGTTCCCTTTTTTTTTTTTTATGATGTACAACACATTGGTAATTTAAAATTGCCTTGTC&#xA;sub4            TTGTTCCCTTTTTTTTTTTTTATGATGTACAACACATTGGTAATTTAAAATTGCCTTGTC&#xA;                ************************************************************&#xA;&#xA;sub1            ATAAA&#xA;sub2            ATAAA&#xA;seq1            ATAAA&#xA;sub3            ATAAA&#xA;sub4            ATAAA&#xA;                *****&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;instead of CDS end/start you have to select regions around your deletions with say 15-20bp flanks on each side. Put these in fasta and run locate.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;caveat&lt;/strong&gt;&#xA;running with zero mismatches will miss some reads.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;edit 2&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;The command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zcat input.fq.gz | seqkit locate --threads 8 \&#xA;--max-mismatch 0 \&#xA;--pattern-file fasta_with_5_frags_extracted_from_algnt.fa &amp;gt; result.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Some trial and error is required to select good fragments matching exactly one of the five groups plus increasing &lt;code&gt;max-mismatch&lt;/code&gt; if due to sequencing errors a lot of reads were dropped.&#xA;To speed up IO one can compress &lt;code&gt;result.tsv&lt;/code&gt; on the fly, i.e. using &lt;code&gt;pigz&lt;/code&gt; or &lt;code&gt;zstd&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;In the next step we need to create five separate files listing read names containing a given fragment:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;grep frag_name_1 result.tsv | cut -f 1 &amp;gt; frag_name_1.read_names.txt&#xA;&#xA;seqkit grep --pattern-file frag_name_1.read_names.txt input.fq.gz -o frag_name_1.reads.fq.gz&#xA;&#xA;# repeat five times with different pattern file and changing the output&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6314" LastEditorUserId="6314" LastEditDate="2024-02-08T15:31:25.727" LastActivityDate="2024-02-08T15:31:25.727" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22155" PostTypeId="2" ParentId="22135" CreationDate="2024-02-07T18:37:25.983" Score="0" Body="&lt;p&gt;How do you know the &amp;quot;family&amp;quot; to which your motifs belong?  If the source for the motifs that FIMO matched is, say, &lt;a href=&quot;https://jaspar.elixir.no/search?q=&amp;amp;collection=CORE&amp;amp;tax_group=vertebrates&quot; rel=&quot;nofollow noreferrer&quot;&gt;JASPAR Vertebrata CORE&lt;/a&gt;, then you might reasonably decide to use JASPAR's &amp;quot;structural class&amp;quot; as provided by their pre-computed &amp;quot;Annotation table&amp;quot; in &lt;a href=&quot;https://jaspar.elixir.no/matrix-clusters/vertebrates/?detail=true&quot; rel=&quot;nofollow noreferrer&quot;&gt;JASPAR CORE Vertebrates clustering&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Assuming you are using JASPAR and you do make this decision...&lt;/p&gt;&#xA;&lt;p&gt;Your &amp;quot;raw .tsv file&amp;quot; is in &lt;a href=&quot;https://meme-suite.org/meme/doc/fimo-output-format.html#tsv_results&quot; rel=&quot;nofollow noreferrer&quot;&gt;FIMO Results TSV Format&lt;/a&gt;.  &amp;quot;Joining&amp;quot; it with a downloaded copy of the &amp;quot;Annotation table&amp;quot; serves to annotate it with additional columns, having the effect you seek of clustering the FIMO results on the &amp;quot;structural class&amp;quot; of each matched motif.  Joining can be implemented following &lt;a href=&quot;https://sqlite-utils.datasette.io/en/stable/cli.html#id16&quot; rel=&quot;nofollow noreferrer&quot;&gt;Running queries directly against CSV or JSON&lt;/a&gt; using the &lt;a href=&quot;https://sqlite-utils.datasette.io/en/stable/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;sqlite-utils&lt;/a&gt; command-line utility.&lt;/p&gt;&#xA;&lt;p&gt;How you would then plot these features and encode the class as a color requires more detailed description what the plot is supposed to represent or look like.  For instance, do you &amp;quot;want to graph the positions of these motifs along a horizontal graph&amp;quot; with coordinates taken with respect to the TSS of the transcript from which the promoter is taken?&lt;/p&gt;&#xA;" OwnerUserId="1692" LastEditorUserId="1692" LastEditDate="2024-02-07T18:49:21.880" LastActivityDate="2024-02-07T18:49:21.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22156" PostTypeId="2" ParentId="22152" CreationDate="2024-02-08T11:17:49.313" Score="2" Body="&lt;p&gt;According to &lt;a href=&quot;https://github.com/mummer4/mummer?tab=readme-ov-file#show-coords&quot; rel=&quot;nofollow noreferrer&quot;&gt;show cords&lt;/a&gt; the format you are getting contains columns which you can use to obtain &lt;code&gt;BED&lt;/code&gt; format with relevant information.&lt;br /&gt;&#xA;You need following columns:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;TAGS_genome1 &#xA;[S1]    Start of the alignment region in the reference sequence.&#xA;[E1]    End of the alignment region in the reference sequence.&#xA;TAGS_genome2&#xA;[% IDY]&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would start with not merging all coordinates but getting bed files for each pairwise genome_a__vs__genome_b.bed etc. and start from there, taking a subset of 3 genomes first (say a, b,c).&#xA;You can find the regions shared by all three &lt;code&gt;bedtools intersect&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bedtools intersect -a a_vs_b.bed -b a_vs_c.bed&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The issue is that you will have many to many relationships, since region_1 from b can have several similar regions in b, and each one of these (== regions in b) also can be similar to a number of regions in a. Depending on what you are looking for and how dis-similar the genomes are you may end up with reporting repetitive elements or if you filter hard missing i.e. exons.&lt;/p&gt;&#xA;" OwnerUserId="6314" LastActivityDate="2024-02-08T11:17:49.313" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22158" PostTypeId="1" CreationDate="2024-02-10T08:12:33.537" Score="3" ViewCount="76" Body="&lt;p&gt;I'm wondering how can I calculate the distance between two regions from the bed (&lt;code&gt;chr&lt;/code&gt;, &lt;code&gt;start&lt;/code&gt;, &lt;code&gt;end&lt;/code&gt;) alike dataframe in R. Precisely, I want to subtract &lt;code&gt;end&lt;/code&gt; of r from &lt;code&gt;start&lt;/code&gt; of r+1 (r=row). Also, the calculation will repeat similarly for each chromosome.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;head(G3_dat[1:3])&#xA;&#xA;# A tibble: 6 × 3&#xA;  chrom   start     end&#xA;  &amp;lt;chr&amp;gt;   &amp;lt;int&amp;gt;   &amp;lt;int&amp;gt;&#xA;1 chr1  1330426 1330628&#xA;2 chr1  2017424 2017750&#xA;3 chr1  2017424 2017750&#xA;4 chr1  2018462 2018871&#xA;5 chr1  3899411 3899468&#xA;6 chr1  4653431 4653724&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The expected outcome would be,&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;head(G3_dat[1:4])&#xA;&#xA;    # A tibble: 6 × 4&#xA;      chrom   start     end     dis&#xA;      &amp;lt;chr&amp;gt;   &amp;lt;int&amp;gt;   &amp;lt;int&amp;gt;     &amp;lt;int&amp;gt;&#xA;    1 chr1  1330426 1330628&#xA;    2 chr1  2017424 2017750     686796&#xA;    3 chr1  2017424 2017750       -326&#xA;    4 chr1  2018462 2018871        712&#xA;    5 chr1  3899411 3899468    1880540&#xA;    6 chr1  4653431 4653724     753963&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12770" LastEditorUserId="650" LastEditDate="2024-02-10T15:50:29.893" LastActivityDate="2024-03-22T08:53:44.920" Title="How to calculate distance between two regions from bed like dataframe in R?" Tags="|r|genomics|bed|dataframe|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22159" PostTypeId="2" ParentId="22158" CreationDate="2024-02-10T09:33:30.397" Score="3" Body="&lt;p&gt;There are two functions in the tidyverse suite (specifically &lt;code&gt;dplyr&lt;/code&gt;) called &lt;code&gt;lead&lt;/code&gt; and &lt;code&gt;lag&lt;/code&gt; that allow calculations like this to be done:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(tidyverse)&#xA;G3_dat %&amp;gt;%&#xA;   group_by(chrom) %&amp;gt;%&#xA;   arrange(start) %&amp;gt;%&#xA;   mutate(dis = start - lag(end))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm not sure if I've got the lead/lag function correct there (I've updated it based on a suggestion from @ram-rs), but it doesn't take too long to play round with all the combinations to get the right one.&lt;/p&gt;&#xA;&lt;p&gt;FWIW, &lt;code&gt;%&amp;gt;%&lt;/code&gt; (or &lt;code&gt;|&amp;gt;&lt;/code&gt; now in native R) is a &amp;quot;pipe&amp;quot; operator that means &amp;quot;take the result of the last function and feed it into the first argument of the next function&amp;quot;. It is used extensively in &lt;code&gt;tidyverse&lt;/code&gt; scripts. More details &lt;a href=&quot;https://stackoverflow.com/questions/67633022/what-are-the-differences-between-rs-native-pipe-and-the-magrittr-pipe&quot;&gt;here&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="131" LastEditDate="2024-03-22T08:53:44.920" LastActivityDate="2024-03-22T08:53:44.920" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22160" PostTypeId="1" CreationDate="2024-02-12T17:28:37.453" Score="0" ViewCount="35" Body="&lt;p&gt;I have 16 CPUs on my AWS EC2 instance and I am trying to use &lt;a href=&quot;https://sourceforge.net/p/ksnp/discussion/general/&quot; rel=&quot;nofollow noreferrer&quot;&gt;kSNP4&lt;/a&gt; to generate an SNP matrix. I was able to successfully run this with over 150 samples - while it took a while to run it did complete - and now I want to run it with over 170 samples but when I use the same command it dies at the second step of the process. I have reason to believe it is due to the processors on my server that I am using more than the tool itself.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know the best way to ensure maximum processing ability to run tools on an EC2 instance? Or what I can do to increase the power of the instance?&lt;/p&gt;&#xA;&lt;p&gt;I know this is more than a technical question than a specific bioinformatic question but I would appreciate any insight!&lt;/p&gt;&#xA;" OwnerUserId="16067" LastActivityDate="2024-02-14T14:23:47.307" Title="Increasing memory usage in AWS instance" Tags="|linux|aws|cpu|ksnp4|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22161" PostTypeId="1" AcceptedAnswerId="22162" CreationDate="2024-02-12T21:32:24.463" Score="3" ViewCount="67" Body="&lt;p&gt;I have written a pipeline that is composed of three processes. I seem to be misunderstanding how Nextflow passes the outputs from one process to the next or creates a queue of files. I am using a tumor and paired normal sample from SRA (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra/SRX21185781%5Baccn%5D&quot; rel=&quot;nofollow noreferrer&quot;&gt;SRX21185781&lt;/a&gt;) to prototype this pipeline.&lt;/p&gt;&#xA;&lt;p&gt;My first process, &lt;code&gt;trim_QC&lt;/code&gt; works fine and outputs the expected number of trimmed fastqs. However, the next process, &lt;code&gt;bwa_align&lt;/code&gt; only creates a single bam from one of the two output from the preceding process. My terminal looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;N E X T F L O W  ~  version 23.10.1&#xA;Launching `../main.nf` [sick_church] DSL2 - revision: beb03cc0c6&#xA;&#xA;[6e/48d4f6] process &amp;gt; trim_QC (SRR25452858_normal)                  [100%] 2 of 2, cached: 2 ✔&#xA;[39/1b8cb2] process &amp;gt; bwa_align (SRR25452858_normal)                [100%] 1 of 1, cached: 1 ✔&#xA;[5f/e598c1] process &amp;gt; mapping_stats (SRR25452858_normal_sorted.bam) [100%] 1 of 1, cached: 1 ✔&#xA;...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and when running with &lt;code&gt;-with-trace&lt;/code&gt; it is even more obvious only one bam file is being processed by the downstream processes:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;task_id hash    native_id   name    status  exit    submit  duration    realtime    %cpu    peak_rss    peak_vmem   rchar   wchar&#xA;2   d3/0bd651   71127   trim_QC (SRR25452863_tumor) CACHED  0   2024-02-07 16:28:37.449 1h 21m 10s  1h 21m 10s  -   -   -   -   -&#xA;1   6e/48d4f6   71128   trim_QC (SRR25452858_normal)    CACHED  0   2024-02-07 16:28:37.459 1h 16m 32s  1h 16m 32s  -   -   -   -   -&#xA;3   39/1b8cb2   2754    bwa_align (SRR25452858_normal)  CACHED  0   2024-02-08 10:08:46.033 4h 18m 10s  4h 18m 10s  -   -   -   -   -&#xA;4   5f/e598c1   9282    mapping_stats (SRR25452858_normal_sorted.bam)   CACHED  0   2024-02-08 14:26:55.823 53.9s   53.8s   -   -   -   -   -&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am expecting for the &lt;code&gt;bwa-align&lt;/code&gt; and &lt;code&gt;mapping_stats&lt;/code&gt; processes to also be n of 2.&lt;/p&gt;&#xA;&lt;p&gt;I have tried explicitly designating the &lt;code&gt;trim_QC.out&lt;/code&gt; as a channel (&lt;code&gt;Channel.of(trim_QC.out)&lt;/code&gt;) or using an operator to return a queue channel object (i.e., &lt;code&gt;.collect&lt;/code&gt;, &lt;code&gt;.collectFile()&lt;/code&gt;, etc.) but this either returns errors or fails.&lt;/p&gt;&#xA;&lt;p&gt;If I try to visualize the outputs from the &lt;code&gt;trim_QC&lt;/code&gt; process by adding this to my workflow: &lt;code&gt;trim_QC.out.trimmed_reads.view{it}&lt;/code&gt;, it does show both file pairs in my terminal, but only the first one is processed by &lt;code&gt;bwa_align&lt;/code&gt; and subsequent steps. Like so:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;...&#xA;[6e/48d4f6] process &amp;gt; trim_QC (SRR25452858_normal)            [100%] 2 of 2, cached: 2 ✔&#xA;[39/1b8cb2] process &amp;gt; bwa_align (SRR25452858_normal)          [100%] 1 of 1, cached: 1 ✔&#xA;[5f/e598c1] process &amp;gt; mapping_stats (SRR25452858_normal_so... [100%] 1 of 1, cached: 1 ✔&#xA;hello&#xA;&#xA;[SRR25452858_normal, /Users/me/Desktop/fastq2/work/6e/48d4f61cae83ff903abceaadf715e5/SRR25452858_normal_1_val_1.fq.gz, /Users/me/Desktop/fastq2/work/6e/48d4f61cae83ff903abceaadf715e5/SRR25452858_normal_2_val_2.fq.gz]&#xA;Analysis complete for SRR25452858_normal_1_val_1.fq.gz&#xA;Analysis complete for SRR25452858_normal_2_val_2.fq.gz&#xA;&#xA;[SRR25452863_tumor, /Users/me/Desktop/fastq2/work/d3/0bd6515fcedfc060596b75dd90c28a/SRR25452863_tumor_1_val_1.fq.gz, /Users/me/Desktop/fastq2/work/d3/0bd6515fcedfc060596b75dd90c28a/SRR25452863_tumor_2_val_2.fq.gz]&#xA;Analysis complete for SRR25452863_tumor_1_val_1.fq.gz&#xA;Analysis complete for SRR25452863_tumor_2_val_2.fq.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I am unsure if I am misunderstanding what Nextflow is capable of or how channels work. What I am trying to accomplish is trim adapters and QC result, align trimmed reads followed by indexing, and finally calculate mappings statistics. Any advice is appreciated.&lt;/p&gt;&#xA;&lt;p&gt;Here is my entire script:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/usr/bin/env nextflow&#xA;nextflow.enable.dsl = 2&#xA;&#xA;params.fastq = &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$projectDir/fastq2/*_{1,2}*&quot;&#xA;params.index_dir = &quot;$&lt;/span&gt;projectDir/RefGenome/BWAIndex&amp;quot;&#xA;params.ref = &amp;quot;hg38.fa&amp;quot;&#xA;params.out_dir = &amp;quot;$projectDir/Results&amp;quot;&#xA;&#xA;workflow {&#xA;&#xA;    fastq_ch = Channel.fromFilePairs(params.fastq)&#xA;    index_ch = Channel.fromPath(params.index_dir)&#xA;    ref_ch = Channel.of(params.ref)&#xA;&#xA;    trim_QC(fastq_ch)&#xA;    bwa_align(index_ch, ref_ch, trim_QC.out.trimmed_reads)&#xA;    mapping_stats(bwa_align.out.sorted_bam)&#xA;}&#xA;&#xA;process trim_QC {&#xA;    debug true&#xA;    tag &amp;quot;$sample_id&amp;quot;&#xA;&#xA;    publishDir = [&#xA;                    [&#xA;                        path: { &amp;quot;${params.out_dir}/trimmed/&amp;quot; },&#xA;                        mode: 'copy',&#xA;                        overwrite: false&#xA;                    ],&#xA;&#xA;                    [&#xA;                        path: { &amp;quot;${params.out_dir}/stats/&amp;quot; },&#xA;                        mode: 'copy',&#xA;                        pattern: &amp;quot;*fastqc*&amp;quot;,&#xA;                        overwrite: false&#xA;                    ]&#xA;                ]&#xA;&#xA;    input:&#xA;    tuple val(sample_id), path(fastq)&#xA;&#xA;    output:&#xA;    tuple val(&amp;quot;${sample_id}&amp;quot;), path(&amp;quot;*_val_1.fq.gz&amp;quot;), path(&amp;quot;*_val_2.fq.gz&amp;quot;), emit: trimmed_reads&#xA;    path(&amp;quot;*fastqc*&amp;quot;)&#xA;&#xA;    script:&#xA;    def (read1, read2) = fastq&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    trim_galore --paired --fastqc &lt;span class=&quot;math-container&quot;&gt;${read1} $&lt;/span&gt;{read2}&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process bwa_align {&#xA;    debug true&#xA;    tag &amp;quot;$sample_id&amp;quot;&#xA;&#xA;    publishDir &amp;quot;${params.out_dir}/bwa_aligned/&amp;quot;, mode: 'copy', overwrite: false&#xA;&#xA;    input:&#xA;    path index_dir&#xA;    val ref&#xA;    tuple val(sample_id), path(read1), path(read2)&#xA;&#xA;    output:&#xA;    path(&amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_id}_sorted.bam&quot;), emit: sorted_bam&#xA;path(&quot;$&lt;/span&gt;{sample_id}_sorted.bam.bai&amp;quot;), emit: bam_index&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    bwa mem -t 2 &lt;span class=&quot;math-container&quot;&gt;${index_dir}/$&lt;/span&gt;{ref} &lt;span class=&quot;math-container&quot;&gt;${read1} $&lt;/span&gt;{read2} | samtools sort -@2 -o &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${sample_id}_sorted.bam&quot; -&#xA;samtools index $&lt;/span&gt;{sample_id}_sorted.bam &amp;gt; &amp;quot;${sample_id}_sorted.bam.bai&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;process mapping_stats {&#xA;    debug true&#xA;    tag &amp;quot;$sorted_bam&amp;quot;&#xA;&#xA;    publishDir &amp;quot;${params.out_dir}/stats/&amp;quot;, mode: 'copy', overwrite: false&#xA;&#xA;    input:&#xA;    path(sorted_bam)&#xA;&#xA;    output:&#xA;    path(&amp;quot;${sorted_bam}_mapping_stats.txt&amp;quot;)&#xA;&#xA;    script:&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;    samtools flagstat &lt;span class=&quot;math-container&quot;&gt;${sorted_bam} &amp;gt; &quot;$&lt;/span&gt;{sorted_bam}_mapping_stats.txt&amp;quot;&#xA;    &amp;quot;&amp;quot;&amp;quot;&#xA;}&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17229" LastEditorUserId="17229" LastEditDate="2024-02-13T23:41:41.583" LastActivityDate="2024-02-13T23:41:41.583" Title="Nextflow: Properly chaining process outputs" Tags="|nextflow|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22162" PostTypeId="2" ParentId="22161" CreationDate="2024-02-13T08:56:39.110" Score="2" Body="&lt;p&gt;This is a result of nextflow channel types, &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#channel-types&quot; rel=&quot;nofollow noreferrer&quot;&gt;queue- and value-channels&lt;/a&gt;. Nextflow will make fifo combinations of your channels to run &lt;code&gt;bwa_align&lt;/code&gt; until either or both channel are empty. &lt;code&gt;index_ch&lt;/code&gt; contains only one item, so once it is used there are no more valid combinations to start new  &lt;code&gt;bwa_align&lt;/code&gt; processes.&lt;/p&gt;&#xA;&lt;p&gt;Either &lt;a href=&quot;https://www.nextflow.io/docs/latest/channel.html#value&quot; rel=&quot;nofollow noreferrer&quot;&gt;create&lt;/a&gt; your &lt;code&gt;index_ch&lt;/code&gt; as value channel, or keep the index out of a channel entirely and refer to it as a &lt;a href=&quot;https://www.nextflow.io/docs/latest/config.html#scope-params&quot; rel=&quot;nofollow noreferrer&quot;&gt;parameter&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="3818" LastActivityDate="2024-02-13T08:56:39.110" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22163" PostTypeId="1" AcceptedAnswerId="22164" CreationDate="2024-02-13T14:47:47.020" Score="2" ViewCount="55" Body="&lt;p&gt;I have a VCF file coming from a structural variant detection tool that doesn't add GT value. I can add GT values, for example by using this tool: &lt;a href=&quot;https://vatools.readthedocs.io/en/latest/vcf_genotype_annotator.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://vatools.readthedocs.io/en/latest/vcf_genotype_annotator.html&lt;/a&gt;. I know there exist the alternate allele (1), but I don't know if it's homozygous or heterozygous.&lt;/p&gt;&#xA;&lt;p&gt;My collaborator wants a VCF with genotype. If an SV is called but not genotyped, I was wondering if ./1 could mean unknown allele and alternate allele for a diploid genome (working with human DNA). Although, if it is not a standard format, it may be a problem.&lt;/p&gt;&#xA;&lt;p&gt;Is &lt;code&gt;./1&lt;/code&gt; a valid value to add or does it have to be &lt;code&gt;0/1&lt;/code&gt; or &lt;code&gt;1/1&lt;/code&gt;? Thanks.&lt;/p&gt;&#xA;" OwnerUserId="9699" LastEditorUserId="73" LastEditDate="2024-02-14T18:57:15.913" LastActivityDate="2024-02-14T18:57:15.913" Title="Is ./1 a valid GT value to add to VCF?" Tags="|vcf|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22164" PostTypeId="2" ParentId="22163" CreationDate="2024-02-13T19:03:37.227" Score="1" Body="&lt;p&gt;No, &lt;code&gt;./1&lt;/code&gt; isn't valid. You can find the VCF specifications at &lt;a href=&quot;https://samtools.github.io/hts-specs/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://samtools.github.io/hts-specs/&lt;/a&gt;. The current version is &lt;a href=&quot;https://samtools.github.io/hts-specs/VCFv4.4.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;VCFv4.4&lt;/a&gt; and this is the genotype section which hasn't really changed much from previous versions as far as I recall:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;GT (String): Genotype, encoded as allele value preceded by either of /&#xA;or | depending on whether that allele is considered phased. The first&#xA;phasing indicator may be omitted and is implicitly defined as / if any&#xA;phasing indicators are / and | otherwise. The allele values are 0 for&#xA;the reference allele (what is in the REF field), 1 for the first&#xA;allele listed in ALT, 2 for the second allele list in ALT and so on.&#xA;For diploid calls examples could be 0/1, 1 | 0, /0/1, or 1/2, etc.&#xA;Haploid calls, e.g. on Y, male non-pseudoautosomal X, or mitochondria,&#xA;should be indicated by having only one allele value. A triploid call&#xA;might look like 0/0/1, and a partially phased triploid call could be&#xA;|0/1/2 to indicate that the first allele is phased with another&#xA;variant in the VCF. If a call cannot be made for a sample at a given&#xA;locus, ‘.’ must be specified for each missing allele in the GT field&#xA;(for example ‘./.’ for a diploid genotype and ‘.’ for haploid&#xA;genotype). The meanings of the phasing indicators are as follows (see&#xA;the PS and PSL fields below for more details on incorporating phasing&#xA;information into the genotypes):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;/ : allele is unphased&lt;/li&gt;&#xA;&lt;li&gt;| : allele is phased (according to the phase-set indicated in PS or PSL)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;For symbolic structural variant alleles, GT=0 indicates the absence of any of the ALT symbolic structural&#xA;variants defined in the record. Implementer should note that merging a VCF record containing only symbolic&#xA;structural variant ALT alleles with a record containing other alleles will result a change of the meaning of the&#xA;GT=0 haplotypes from the record containing only symbolic SVs.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;This is the bit you are most interested in:&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;If a call cannot be made for a sample at a given locus, ‘.’ must be specified for each missing allele in the GT field (for example ‘./.’ for a diploid genotype and ‘.’ for haploid genotype).&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;So, &lt;code&gt;./1&lt;/code&gt; doesn't mean anything. You could use &lt;code&gt;./.&lt;/code&gt; which is the standard for &amp;quot;not genotyped&amp;quot;, or you could simply not have a GT field at all since that isn't actually required by the standard, so a VCF file with no genotype field at all can be a perfectly valid and standard compliant VCF.&lt;/p&gt;&#xA;&lt;p&gt;In other words, just think about what you really need. Do you absolutely need a GT field? If not, just leave it out and you're done. If you do, you can use &lt;code&gt;./.&lt;/code&gt; if you want to say &amp;quot;I don't know what the genotype here is&amp;quot; (but be aware that some tools interpret &lt;code&gt;./.&lt;/code&gt; as &amp;quot;this might not be a real call&amp;quot;), but don't use &lt;code&gt;./1&lt;/code&gt; since that won't be understood by anyone.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2024-02-13T19:03:37.227" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22165" PostTypeId="2" ParentId="19531" CreationDate="2024-02-14T13:25:34.367" Score="2" Body="&lt;p&gt;Regarding &lt;a href=&quot;https://bioinformatics.stackexchange.com/a/19551/650&quot;&gt;the other answer&lt;/a&gt; with the below code:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;data.df &amp;lt;- object@meta.data[, c(&amp;quot;celltypes&amp;quot;, &amp;quot;condition&amp;quot;)]&#xA;data.df[&amp;quot;value&amp;quot;] &amp;lt;- object&lt;span class=&quot;math-container&quot;&gt;$RNA@data[&quot;geneX&quot;, ]&#xA;data.df &amp;lt;- data.df %&amp;gt;% mutate(group = paste(celltypes, condition, sep=&quot;:&quot;))&#xA;data.df &amp;lt;- data.df %&amp;gt;% filter(value &amp;gt; 0)&#xA;data.df.2 &amp;lt;- data.df %&amp;gt;% group_by(group) %&amp;gt;% summarise(sd=sd(value),value=mean(value))&#xA;X &amp;lt;- str_split(data.df.2$&lt;/span&gt;'group', &amp;quot;:&amp;quot;, 2, simplify=TRUE)&#xA;&#xA;data.df.2&lt;span class=&quot;math-container&quot;&gt;$'celltypes' &amp;lt;- X[, 1]&#xA;data.df.2$&lt;/span&gt;'condition' &amp;lt;- X[, 2]&#xA;&#xA;ggplot(data.df.2, aes(fill=condition, y=value, x=celltypes)) + &#xA;    geom_bar(color=&amp;quot;black&amp;quot;, position=&amp;quot;dodge&amp;quot;, stat=&amp;quot;identity&amp;quot;) +&#xA;    geom_errorbar(&#xA;        aes(ymin=value, ymax=value+sd), width=.2,&#xA;        position=position_dodge(.9)&#xA;    )&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I find the plot matches more the results of Seurat differential expression if you DO NOT perform the step of removing the cells with 0 counts (&lt;code&gt;data.df &amp;lt;- data.df %&amp;gt;% filter(value &amp;gt; 0)&lt;/code&gt;). To avoid having huge error bars you can use standard error of the mean instead of sd.&lt;/p&gt;&#xA;" OwnerUserId="18954" LastEditorUserId="650" LastEditDate="2024-02-15T15:33:02.407" LastActivityDate="2024-02-15T15:33:02.407" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22166" PostTypeId="2" ParentId="22160" CreationDate="2024-02-14T14:23:47.307" Score="1" Body="&lt;p&gt;You need to diagnose for low memory and consider switching to an &lt;strong&gt;R5&lt;/strong&gt; processor that prioritises memory. Basically it's easily solvable by switching hardware. There are certainly alot of other approaches but this is specialist HPC.&lt;/p&gt;&#xA;&lt;p&gt;Personally, I would use Slurm ParallelCluster in AWS. This is very easy to set up from from the configuration file.&lt;/p&gt;&#xA;&lt;p&gt;The overview of instance types here.&#xA;&lt;a href=&quot;https://aws.amazon.com/ec2/instance-types/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://aws.amazon.com/ec2/instance-types/&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2024-02-14T14:23:47.307" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22167" PostTypeId="1" CreationDate="2024-02-14T15:52:30.390" Score="1" ViewCount="38" Body="&lt;p&gt;I need to plot the results of a GSEA analysis we performed on scRNA data. I am stumbling upon various possibilities for this and sometimes I am struggling a bit to understand how to interpret it. My source is this: &lt;a href=&quot;https://yulab-smu.top/biomedical-knowledge-mining-book/enrichplot.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://yulab-smu.top/biomedical-knowledge-mining-book/enrichplot.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Specifically, in the DotPlot the x axis is geneRatio, in my results table there is no metric called that way, how is it computed? Another question is regarding the ridge plot, the way it is defined is: &amp;quot;The ridgeplot will visualize expression distributions of core enriched genes&amp;quot; This is a bit oscure to me, and in the plot that I generated from my data following that tutoria the x axis is automatically labelled as &amp;quot;enrichment distribution&amp;quot; which in my understanding has a completely different meaning in this case.&lt;/p&gt;&#xA;&lt;p&gt;Ihave an object of this class: &lt;a href=&quot;https://rdrr.io/bioc/DOSE/man/gseaResult-class.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://rdrr.io/bioc/DOSE/man/gseaResult-class.html&lt;/a&gt;, that now I just fed to the functions as explained in that tutorial. The fields of the result table are:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;ID     Description setSize enrichmentScore NES pvalue  p.adjust    qvalue  rank    leading_edge    core_enrichment&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;I don't find any info on how these results are used, I don't know if showing values or plots may be useful as I have more of a methodological doubt.&lt;/p&gt;&#xA;&lt;p&gt;My question is, if I needed to build these plots on my own without relying on the library documented in the website, starting from the output of cluster profiler, how would I do it? I don't need R code but an understanding of what is done.&lt;/p&gt;&#xA;&lt;p&gt;Thank you very much in advance.&lt;/p&gt;&#xA;" OwnerUserId="18923" LastEditorUserId="73" LastEditDate="2024-02-15T09:11:05.890" LastActivityDate="2024-02-15T09:11:05.890" Title="GSEA results plot interpretation" Tags="|gsea|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22169" PostTypeId="1" CreationDate="2024-02-14T19:52:42.860" Score="3" ViewCount="21" Body="&lt;p&gt;I am new to using Alphafold 2 and have been primarily using ChimeraX and Colabfold to run some preliminary predictions.&#xA;I am particularly interested in using AF2 to predict the homo-oligomerization number of a protein. I have attempted this initially by manually running several predictions with different multiples of the protein end to end (&amp;quot;ABCD:ABCD&amp;quot;). This works but does not provide any direct comparison of plausibility between different oligomerization possibilities, and does not establish a firm prediction as to what the oligomerization state of the protein is.&lt;/p&gt;&#xA;&lt;p&gt;I have read a paper (&lt;a href=&quot;https://doi.org/10.1038/s41594-022-00849-w&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1038/s41594-022-00849-w&lt;/a&gt;) which appears to use Alphafold2 to make predictions about oligomer count. These researchers appear to use Colabfold for at least part of this calculation.&lt;/p&gt;&#xA;&lt;p&gt;Can Alphafold 2 (and Colabfold in particular) be used to make predictions about homo-oligomeric state in this way? If so, how?&lt;/p&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="18956" LastActivityDate="2024-02-14T19:52:42.860" Title="Oligomer status prediction with Alphafold 2" Tags="|protein-structure|alphafold|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22170" PostTypeId="1" CreationDate="2024-02-15T03:29:21.067" Score="2" ViewCount="59" Body="&lt;p&gt;I am using R to extract ClinVar classification of SNPs using rsid. Here's an example of my workflow:&lt;/p&gt;&#xA;&lt;p&gt;Patient data:&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Sample&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Func&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Gene&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Codon_Change&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;dbSNPv151_GRCh38&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Chr&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;Start&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: right;&quot;&gt;End&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Ref&lt;/th&gt;&#xA;&lt;th style=&quot;text-align: left;&quot;&gt;Obs&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Mary&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;missense_variant&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;SPINK5&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;Cga/Gga&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;rs199757347&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;chr5&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;148091214&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: right;&quot;&gt;148091214&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;C&lt;/td&gt;&#xA;&lt;td style=&quot;text-align: left;&quot;&gt;G&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&lt;/div&gt;&#xA;&lt;p&gt;With this rsid rs199757347, &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/clinvar/?term=rs199757347&quot; rel=&quot;nofollow noreferrer&quot;&gt;two ClinVar classifications&lt;/a&gt; were found with different codon changes, one is C &amp;gt; G (Uncertain significance, missense variant) and the other one C &amp;gt; T (Pathogenic, nonsense variant).&lt;/p&gt;&#xA;&lt;p&gt;With the &lt;code&gt;rentrez&lt;/code&gt; package in R, I can extract both ClinVar classifications:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(XML)&#xA;library(rentrez)&#xA;&#xA;extract_from_esummary(&#xA;  entrez_summary(db = &amp;quot;clinvar&amp;quot;, &#xA;                 id = entrez_search(db = &amp;quot;clinvar&amp;quot;, term = &amp;quot;rs199757347&amp;quot;)$ids, &#xA;                 rettype='xml', &#xA;                 parsed = F), &#xA;  &amp;quot;germline_classification&amp;quot;&#xA;  )&#xA;&#xA;               968176                               &#xA;description    &amp;quot;Uncertain significance&amp;quot;             &#xA;last_evaluated &amp;quot;2022/01/01 00:00&amp;quot;                   &#xA;review_status  &amp;quot;criteria provided, single submitter&amp;quot;&#xA;trait_set      data.frame,2                         &#xA;               351517                                                &#xA;description    &amp;quot;Pathogenic&amp;quot;                                          &#xA;last_evaluated &amp;quot;2019/01/01 00:00&amp;quot;                                    &#xA;review_status  &amp;quot;criteria provided, multiple submitters, no conflicts&amp;quot;&#xA;trait_set      data.frame,2 &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, only the classification &amp;quot;Uncertain significance&amp;quot; is relevant to my sample, how can I incorporate the alternative alleles in querying ClinVar with rsid? Or do you have a better way to query ClinVar with my sample data?&lt;/p&gt;&#xA;" OwnerUserId="14961" LastEditorUserId="14961" LastEditDate="2024-02-15T07:08:55.597" LastActivityDate="2024-02-15T07:08:55.597" Title="Different ClinVar classifications/SNPs are linked to the same rsid" Tags="|r|snp|wgs|entrez|rsid|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22171" PostTypeId="1" AcceptedAnswerId="22180" CreationDate="2024-02-15T12:05:58.967" Score="1" ViewCount="61" Body="&lt;p&gt;I have been running &lt;a href=&quot;https://nf-co.re/rnaseq/1.4.2/docs/output&quot; rel=&quot;nofollow noreferrer&quot;&gt;nf-core&lt;/a&gt; in Python and it works great! But I have a seemingly simple question that I'm struggling to find an answer for online. After running the nf-core pipeline on my RNA sequencing data, Salmon is able to provide gene counts in a file named &lt;code&gt;salmon.merged.gene_counts.tsv&lt;/code&gt;. This file has essentially all the information I am seeking with all counts associated to a unique gene_id. But, for a large number of these gene_ids where no specific gene_name is known, I would like to further find the corresponding DNA sequence (or corresponding transcript's RNA sequence) for each gene_id. Accordingly, my question is: where can I find the information to map the gene_id to the corresponding nucleotides of the gene's (or original transcript's) sequence?&lt;/p&gt;&#xA;&lt;p&gt;(Apologies in advance if my question is missing information or overly simply and I'm overlooking something obvious — I am quite new to this all.)&lt;/p&gt;&#xA;" OwnerUserId="18959" LastActivityDate="2024-02-16T10:45:21.553" Title="After running nf-core, is there a way to map a gene_id to a specific gene's DNA sequence?" Tags="|rna-seq|gene-expression|transcriptome|star|salmon|" AnswerCount="1" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="22172" PostTypeId="2" ParentId="22051" CreationDate="2024-02-15T15:13:16.753" Score="1" Body="&lt;p&gt;I used BioMart (and then tabix) to get the answer to this question:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;if (!requireNamespace(&amp;quot;BiocManager&amp;quot;, quietly = TRUE)) {&#xA;  install.packages(&amp;quot;BiocManager&amp;quot;)&#xA;}&#xA;&#xA;BiocManager::install(&amp;quot;biomaRt&amp;quot;)&#xA;&#xA;## F**kery:&#xA;install.packages(&amp;quot;devtools&amp;quot;)&#xA;devtools::install_version(&amp;quot;dbplyr&amp;quot;, version = &amp;quot;2.3.4&amp;quot;)&#xA;&#xA;&#xA;library(biomaRt)&#xA;&#xA;listEnsembl()&#xA;&#xA;ensembl &amp;lt;- useEnsembl(&#xA;  biomart = &amp;quot;ensembl&amp;quot;,&#xA;  dataset = &amp;quot;hsapiens_gene_ensembl&amp;quot;,&#xA;  GRCh = 37&#xA;)&#xA;&#xA;chrs &amp;lt;- c(1:22, &amp;quot;X&amp;quot;, &amp;quot;Y&amp;quot;, &amp;quot;MT&amp;quot;)&#xA;&#xA;genes &amp;lt;-&#xA;  getBM(&#xA;    attributes = c(&#xA;      &amp;quot;chromosome_name&amp;quot;,&#xA;      &amp;quot;start_position&amp;quot;,&#xA;      &amp;quot;end_position&amp;quot;,&#xA;      &amp;quot;ensembl_gene_id&amp;quot;,&#xA;      &amp;quot;ensembl_transcript_id&amp;quot;,&#xA;      &amp;quot;ensembl_peptide_id&amp;quot;,&#xA;      &amp;quot;hgnc_symbol&amp;quot;&#xA;    ),&#xA;    filters = &amp;quot;chromosome_name&amp;quot;,&#xA;    values = chrs,&#xA;    mart = ensembl&#xA;  )&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18817" LastEditorUserId="650" LastEditDate="2024-02-15T15:34:03.310" LastActivityDate="2024-02-15T15:34:03.310" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22173" PostTypeId="1" CreationDate="2024-02-15T17:06:42.467" Score="0" ViewCount="32" Body="&lt;p&gt;I was reading Jason Aten's paper &lt;a href=&quot;https://bmcsystbiol.biomedcentral.com/articles/10.1186/1752-0509-2-34&quot; rel=&quot;nofollow noreferrer&quot;&gt;Using genetic markers to orient the edges in quantitative trait networks: The NEO software&lt;/a&gt;, which states&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;The NEO software can be used to orient the edges of gene co-expression networks or quantitative trait networks if the edges can be anchored to genetic marker data. R software tutorials, data, and supplementary material can be downloaded from: &lt;a href=&quot;http://www.genetics.ucla.edu/labs/horvath/aten/NEO&quot; rel=&quot;nofollow noreferrer&quot;&gt;http://www.genetics.ucla.edu/labs/horvath/aten/NEO&lt;/a&gt;&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;But it seems the site is down, does anyone know where can I access these data?&lt;/p&gt;&#xA;" OwnerUserId="10365" LastActivityDate="2024-03-31T19:07:36.290" Title="NEO analysis tutorial and data" Tags="|wgcna|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22174" PostTypeId="1" CreationDate="2024-02-15T18:12:44.500" Score="1" ViewCount="23" Body="&lt;p&gt;I' am doing a sc-Seq analysis on a dataset from &lt;a href=&quot;https://www.10xgenomics.com/datasets/10-k-pbm-cs-from-a-healthy-donor-v-3-chemistry-3-standard-3-0-0&quot; rel=&quot;nofollow noreferrer&quot;&gt;10x genomics&lt;/a&gt;, using Seurat. I followed the standard workflow and I also did it with the SCTransform workflow. The results are very different when I plot a Feature Scatter with the same 2 genes.&lt;/p&gt;&#xA;&lt;p&gt;Standard workflow:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/PVcQg.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/PVcQg.jpg&quot; alt=&quot; CD3D and CD3E&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;SCTransform workflow:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/AH7at.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AH7at.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So I was wondering why they were so different. I know that the perfect lines in the first one happen because of the normalization and that it usually means that I need more data. But I don't understand why the SCTransform one. looks like it has less data and why it looks so perfectly arranged. My goal is to make a plot that looks normal and natural, and that may or may not have a visual correlation; instead of this weirdly looking and unnatural plots. If you also have a tip for this, I would be very grateful.  Thanks in advance.&lt;/p&gt;&#xA;&lt;p&gt;Here is my code for the normal workflow:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(Seurat)&#xA;library(tidyverse)&#xA;library(ggplot2)&#xA;library(gridExtra)&#xA;library(harmony)&#xA;library(future.apply)&#xA;library(cowplot)&#xA;library(patchwork)&#xA;library(&amp;quot;DESeq2&amp;quot;)&#xA;library(SeuratData)&#xA;library(SeuratDisk)&#xA;library(SeuratWrappers)&#xA;library(sctransform)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;data &amp;lt;- Read10X(data.dir = &amp;quot;/Users/rodrigohermoza/Desktop/UTEC/2024-0/PoliSia/rstudio/diferential_expression_2/DE2/filtered_gene_bc_matrices/raw_feature_bc_matrix/&amp;quot;)&#xA;&#xA;pbmc &amp;lt;- CreateSeuratObject(counts = data, project = &amp;quot;DE3&amp;quot;)&#xA;pbmc &amp;lt;- PercentageFeatureSet(pbmc, pattern = &amp;quot;^MT-&amp;quot;, col.name = &amp;quot;percent.mt&amp;quot;) &#xA;pbmc &amp;lt;- subset(pbmc, subset = nFeature_RNA &amp;gt; 200 &amp;amp; percent.mt &amp;lt;15 &amp;amp; nCount_RNA &amp;gt; 50)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;pbmc &amp;lt;- NormalizeData(pbmc)&#xA;pbmc &amp;lt;- FindVariableFeatures(pbmc, nfeatures = 3000)&#xA;pbmc &amp;lt;- ScaleData(pbmc, vars.to.regress = &amp;quot;percent.mt&amp;quot;)&#xA;pbmc &amp;lt;- RunPCA(pbmc)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;pbmc &amp;lt;- FindNeighbors(pbmc, dims = 1:21)&#xA;pbmc &amp;lt;- FindClusters(pbmc, resolution = 1)&#xA;pbmc &amp;lt;- RunUMAP(pbmc, dims = 1:21)&#xA;DimPlot(pbmc, label = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;sweep_res &amp;lt;- paramSweep(pbmc,PCs = 1:20, sct = FALSE)&#xA;sweep_stat &amp;lt;- summarizeSweep(sweep_res, GT = F)&#xA;bcmv &amp;lt;- find.pK(sweep_stat)&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;ggplot(bcmv, aes(pK, BCmetric, group = 1)) +&#xA;  geom_point()+&#xA;  geom_line()&#xA;&#xA;#0.005&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;pK&amp;lt;- bcmv %&amp;gt;%&#xA;  filter(BCmetric == max(BCmetric)) %&amp;gt;%&#xA;  select(pK)&#xA;pK&amp;lt;- as.numeric(as.character(pK[[1]]))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;annotations &amp;lt;- pbmc@meta.data$seurat_clusters&#xA;homotipic &amp;lt;- modelHomotypic(annotations)&#xA;exp &amp;lt;- round(0.08*nrow(pbmc@meta.data))&#xA;exp.adj &amp;lt;- round(exp*(1-homotipic))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;pbmc &amp;lt;- doubletFinder(pbmc,&#xA;                          PCs = 1:20,&#xA;                          pK = pK,&#xA;                          nExp = exp.adj,&#xA;                          reuse.pANN = F,&#xA;                          sct = F)&#xA;DimPlot(pbmc, reduction = &amp;quot;umap&amp;quot;, group.by = &amp;quot;DF.classifications_0.25_0.11_818&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;table(pbmc@meta.data$DF.classifications_0.25_0.11_818)&#xA;pbmc_filtered &amp;lt;- subset(pbmc, subset = DF.classifications_0.25_0.11_818 == &amp;quot;Singlet&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;pbmc.markers &amp;lt;- FindAllMarkers(pbmc_filtered)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;FeatureScatter(pbmc_filtered, feature1 = &amp;quot;CD3D&amp;quot;, feature2 = &amp;quot;CD3E&amp;quot;, slot = &amp;quot;data&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And here is my SCTransform workflow&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(Seurat)&#xA;library(tidyverse)&#xA;library(ggplot2)&#xA;library(gridExtra)&#xA;library(harmony)&#xA;library(future.apply)&#xA;library(cowplot)&#xA;library(patchwork)&#xA;library(&amp;quot;DESeq2&amp;quot;)&#xA;library(SeuratData)&#xA;library(SeuratDisk)&#xA;library(SeuratWrappers)&#xA;library(sctransform)&#xA;library(DoubletFinder)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;data &amp;lt;- Read10X(data.dir = &amp;quot;/Users/rodrigohermoza/Desktop/UTEC/2024-0/PoliSia/rstudio/diferential_expression_2/DE2/filtered_gene_bc_matrices/raw_feature_bc_matrix/&amp;quot;)&#xA;pbmc &amp;lt;- CreateSeuratObject(counts = data, project = &amp;quot;DE3&amp;quot;)&#xA;pbmc &amp;lt;- PercentageFeatureSet(pbmc, pattern = &amp;quot;^MT-&amp;quot;, col.name = &amp;quot;percent.mt&amp;quot;) &#xA;pbmc &amp;lt;- subset(pbmc, subset = nFeature_RNA &amp;gt; 200 &amp;amp; percent.mt &amp;lt;15 &amp;amp; nCount_RNA &amp;gt; 50)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;pbmc &amp;lt;- SCTransform(pbmc, vars.to.regress = &amp;quot;percent.mt&amp;quot;)&#xA;pbmc &amp;lt;- RunPCA(pbmc)&#xA;ElbowPlot(pbmc,ndims = 50)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;pbmc &amp;lt;- FindNeighbors(pbmc, dims = 1:20)&#xA;pbmc &amp;lt;- FindClusters(pbmc, resolution = 1)&#xA;pbmc &amp;lt;- RunUMAP(pbmc, dims = 1:20)&#xA;DimPlot(pbmc, label = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;pbmc.markers &amp;lt;- FindAllMarkers(pbmc)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;pre&gt;&lt;code&gt;FeatureScatter(pbmc, feature1 = &amp;quot;CD3D&amp;quot;, feature2 = &amp;quot;CD3E&amp;quot;, slot = &amp;quot;data&amp;quot;, smooth = T)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18962" LastActivityDate="2024-02-15T18:12:44.500" Title="Why my FeatureScatter looks so different, and what does that mean?" Tags="|r|scrnaseq|seurat|rstudio|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22175" PostTypeId="1" CreationDate="2024-02-15T18:28:08.513" Score="1" ViewCount="17" Body="&lt;p&gt;Hi I am using Salmon quantitation for multiple fastq paired files using following code&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;salmon_index=&amp;quot;human_salmon_index&amp;quot;&#xA;fastq_dir=&amp;quot;fastq/&amp;quot;&#xA;for dir in &amp;quot;&lt;span class=&quot;math-container&quot;&gt;${fastq_dir}&quot;/SRR*&#xA;do&#xA;r1_file=$&lt;/span&gt;(find &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$dir&quot; -name &quot;*_1.fastq&quot;)&#xA;r2_file=$&lt;/span&gt;(find &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$dir&quot; -name &quot;*_2.fastq&quot;)&#xA;samp=$&lt;/span&gt;(basename &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$dir&quot;)&#xA;echo &quot;Processing sample $&lt;/span&gt;{samp}&amp;quot;&#xA;salmon quant -i &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$salmon_index&quot; -l A -1 &quot;$&lt;/span&gt;r1_file&amp;quot; -2 &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$r2_file&quot; -p 10 -o &quot;salmon_out/$&lt;/span&gt;{samp}_quant&amp;quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and getting this error after it iterates over all files and start initial processing steps.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Processing sample SRR27756943_2.fastq&#xA;Version Info: This is the most recent version of salmon.&#xA;### salmon (selective-alignment-based) v1.10.0&#xA;&#xA;Exception : [&#xA;The following errors were detected with the read files&#xA;&#xA;======================================================&#xA;&#xA;ERROR: file [] does not appear to exist!&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any solution?&lt;/p&gt;&#xA;" OwnerUserId="15560" LastEditorUserId="650" LastEditDate="2024-02-15T20:07:45.293" LastActivityDate="2024-03-20T16:04:00.523" Title="Salmon Multiple File SCRIPT giving error" Tags="|rna-seq|sequence-alignment|bash|salmon|quantification|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22176" PostTypeId="1" AcceptedAnswerId="22181" CreationDate="2024-02-15T20:00:20.053" Score="1" ViewCount="59" Body="&lt;p&gt;Also posted &lt;a href=&quot;https://www.biostars.org/p/9587718/&quot; rel=&quot;nofollow noreferrer&quot;&gt;on biostars&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have bam files that I've obtained through STAR alignment. Here is the first 10 lines of one of my bam files:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;GWNJ-1013:671:GW2304233104th:2:1126:15501:17879 0       chromosome      1       255     11S139M *       0       0       ATTTCCATCACAACTGCTCCTCGGAAGTCGACCAACAAGTCAGCTATGACTTGGCATAATTTGTGCCGACAAAATGCGCGCAGAGTATAGGGGTGGATTAACCCCTATTCAACTCTTCGGTAGTGATTTCCGACTTCACGCTACAACAGG  FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFF  NH:i:1  HI:i:1  AS:i:137        nM:i:0&#xA;GWNJ-1013:671:GW2304233104th:2:1126:15130:18427 0       chromosome      1       255     11S139M *       0       0       ATTTCCATCACAACTGCTCCTCGGAAGTCGACCAACAAGTCAGCTATGACTTGGCATAATTTGTGCCGACAAAATGCGCGCAGAGTATAGGGGTGGATTAACCCCTATTCAACTCTTCGGTAGTGATTTCCGACTTCACGCTACAACAGG  FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF::FFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF  NH:i:1  HI:i:1  AS:i:137        nM:i:0&#xA;GWNJ-1013:671:GW2304233104th:2:1206:9272:29857  0       chromosome      1       255     28S108M14S      *       0       0       GGCGGTTTGGCGTGCGTATTTCCATCACAACTGCTCCTCGGAAGTCGACCAACAAGTCAGCTATGACTTGGCATAATTTGTGCCGACAAAATGCGCGCAGAGTATAGGGGTGGATTAACCCCTATTCAACTCTTCGAGATCGGAAGAGCG  FFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF  NH:i:1  HI:i:1  AS:i:106        nM:i:0&#xA;GWNJ-1013:671:GW2304233104th:2:1266:30734:10692 0       chromosome      1       255     24S126M *       0       0       GTTTGGCGTGCGTATTTCCATCACAACTGCTCCTCGGAAGTCGACCAACAAGTCAGCTATGACTTGGCATAATTTGTGCCGACAAAATGCGCGCAGAGTATAGGGGTGGATTAACCCCTATTCAACTCTTCGGTAGTGATTTCCGACTTC  FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFF  NH:i:1  HI:i:1  AS:i:124        nM:i:0&#xA;GWNJ-1013:671:GW2304233104th:2:1266:31412:33411 0       chromosome      1       255     24S126M *       0       0       GTTTGGCGTGCGTATTTCCATCACAACTGCTCCTCGGAAGTCGACCAACAAGTCAGCTATGACTTGGCATAATTTGTGCCGACAAAATGCGCGCAGAGTATAGGGGTGGATTAACCCCTATTCAACTCTTCGGTAGTGATTTCCGACTTC  FFFFFFFFFFFFFFFF:FFFFFFFFFFFF:FFFF::FFFFFFF:FFFFFFF:FFFFFFFFFFFFFFFF:FFFF,FFFFFFF:FFFFFFFFF:FFFFFFFFFFFFFFF:FFF::FFFFFFFF,FFFFFFFF:FFFFFFFFFFF:F,FFFFF  NH:i:1  HI:i:1  AS:i:124        nM:i:0&#xA;GWNJ-1013:671:GW2304233104th:2:1330:21938:10708 0       chromosome      1       255     16S134M *       0       0       TGCGTATTTCCATCACAACTGCTCCTCGGAAGTCGACCAACAAGTCAGCTATGACTTGGCATAATTTGTGCCGACAAAATGCGCGCAGAGTATAGGGGTGGATTAACCCCTATTCAACTCTTCGGTAGTGATTTCCGACTTCACGCTACA  FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFF,FFFFFFFFFFFFFFFFF:F  NH:i:1  HI:i:1  AS:i:132        nM:i:0&#xA;GWNJ-1013:671:GW2304233104th:2:1367:17074:4726  0       chromosome      1       255     11S82M  *       0       0       ATTTCCATCACAACTGCTCCTCGGAAGTCGACCAACAAGTCAGCTATGACTTGGCATAATTTGTGCCGACAAAATGCGCGCAGAGTATAGGGG   FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF  NH:i:1   HI:i:1  AS:i:80 nM:i:0&#xA;GWNJ-1013:671:GW2304233104th:2:1367:11939:11584 0       chromosome      1       255     11S82M  *       0       0       ATTTCCATCACAACTGCTCCTCGGAAGTCGACCAACAAGTCAGCTATGACTTGGCATAATTTGTGCCGACAAAATGCGCGCAGAGTATAGGGG   FFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFF:F:FFFFFFFFFFFFFFFFFFF  NH:i:1   HI:i:1  AS:i:80 nM:i:0&#xA;GWNJ-1013:671:GW2304233104th:2:1372:24740:30843 0       chromosome      1       255     16S48M  *       0       0       TGCGTATTTCCATCACAACTGCTCCTCGGAAGTCGACCAACAAGTCAGCTATGACTTGGCATAA        FFFFFFFFFFFFFF:FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF        NH:i:1  HI:i:1  AS:i:47 nM:i:0&#xA;GWNJ-1013:671:GW2304233104th:2:1401:11107:19319 0       chromosome      1       255     5S144M1S        *       0       0       ATCACAACTGCTCCTCGGAAGTCGACCAACAAGTCAGCTATGACTTGGCATAATTTGTGCCGACAAAATGCGCGCAGAGTATAGGGGTGGATTAACCCCTATTCAACTCTTCGGTAGTGATTTCCGACTTCACGCTACAACAGGAATTGA  FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFFFFFFFFFFFFFFF,FFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFFF:FFF  NH:i:1  HI:i:1  AS:i:142        nM:i:0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the script I'm using that contains the &lt;code&gt;genomecov -d -pc -ibam&lt;/code&gt; command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;BAM_DIR='./sorted_bam_files'&#xA;&#xA;# Loop over each BAM file in the directory&#xA;for file in &amp;quot;&lt;span class=&quot;math-container&quot;&gt;$BAM_DIR&quot;/*_high_mapq_reads.sorted.bam&#xA;do&#xA;    echo &quot;Calculating coverage for $&lt;/span&gt;file...&amp;quot;&#xA;    # Get the basename of the file&#xA;    base=&lt;span class=&quot;math-container&quot;&gt;$(basename &quot;$&lt;/span&gt;file&amp;quot;)&#xA;    # Calculate coverage&#xA;    bedtools genomecov -d -pc -ibam &lt;span class=&quot;math-container&quot;&gt;$file &amp;gt; &quot;$&lt;/span&gt;{base%.sorted.bam}.coverage&amp;quot;&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is the output I keep getting (first 10 lines only shown). I've tried this with at least 30 other bam files but the output is always the same.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chromosome      1       0&#xA;chromosome      2       0&#xA;chromosome      3       0&#xA;chromosome      4       0&#xA;chromosome      5       0&#xA;chromosome      6       0&#xA;chromosome      7       0&#xA;chromosome      8       0&#xA;chromosome      9       0&#xA;chromosome      10      0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Any ideas on why this keeps happening? I don't think there's something wrong with my bam files because I was able to use them for DGE analyses and also checked to see if most reads were getting unmapped but they weren't. Any help is appreciated.&lt;/p&gt;&#xA;" OwnerUserId="18963" LastEditorUserId="650" LastEditDate="2024-02-16T15:18:00.200" LastActivityDate="2024-02-16T15:18:22.347" Title="Bedtools genomecov gives me coverage files showing zeros at all nucleotide positions" Tags="|rna-seq|bedtools|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22177" PostTypeId="2" ParentId="22175" CreationDate="2024-02-15T20:29:26.773" Score="0" Body="&lt;p&gt;Are you sure the &lt;code&gt;find&lt;/code&gt; operations return exactly one file each? The error seems to state that there were no files found for some run. Or it could also be that you have a file that matched your glob pattern that's not a dir, so the &lt;code&gt;find&lt;/code&gt; operation fails.&lt;/p&gt;&#xA;&lt;p&gt;Try this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;salmon_index=&amp;quot;human_salmon_index&amp;quot;&#xA;fastq_dir=fastq&#xA;for dir in ${fastq_dir}/SRR*/&#xA;do&#xA;r1_file=$(find $dir -name &amp;quot;*_1.fastq&amp;quot;)&#xA;r2_file=$(find $dir -name &amp;quot;*_2.fastq&amp;quot;)&#xA;samp=$(basename $dir)&#xA;echo &amp;quot;Processing sample ${samp}&amp;quot;&#xA;echo salmon quant -i &amp;quot;$salmon_index&amp;quot; -l A -1 &amp;quot;$r1_file&amp;quot; -2 &amp;quot;$r2_file&amp;quot; -p 10 -o &amp;quot;salmon_out/${samp}_quant&amp;quot; | tee salmon_commands.cmd&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Verify the contents of &lt;code&gt;salmon_commands.cmd&lt;/code&gt; then use &lt;code&gt;bash&lt;/code&gt; or &lt;code&gt;parallel&lt;/code&gt; to run the actual commands.&lt;/p&gt;&#xA;" OwnerUserId="650" LastEditorUserId="650" LastEditDate="2024-02-19T15:41:09.870" LastActivityDate="2024-02-19T15:41:09.870" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22178" PostTypeId="1" CreationDate="2024-02-15T21:13:42.870" Score="1" ViewCount="20" Body="&lt;p&gt;I am working with an Illumina-generated final report file from human genotyping; the MEGA chip I believe. It's in a massive .csv file I've been 'exploring' via unix commands on a compute cluster, something I'm fairly new to.&lt;/p&gt;&#xA;&lt;p&gt;For a particular SNP (an APOE variant, rs429358, ancestral allele T with the C MAF presumed to be ~15% in this population of ~160 genotyped persons), though, I'm seeing seemingly non-sensical reads. (Table of first two sample IDs' results at bottom.)&lt;/p&gt;&#xA;&lt;p&gt;I've been trying to make sense of this, and I think the R 1-5 indicates 5 reverse reads and F1-15 indicate 15 forward reads. This also makes sense as to why the R reads would be A-G in the name since that would be the opposing strand of the T-C of the forward reads.&lt;/p&gt;&#xA;&lt;p&gt;But, as you can see, literally everything on both reads is T-T. This is true for every single person in the sample, everyone is T-T, which even if not for the forward/back mismatch, would still be mathematically virtually impossible.&lt;/p&gt;&#xA;&lt;p&gt;This is my first time working with raw final report files large enough that I can't simply open them and look at them, and so I'm having trouble determining if there's something wrong with the file, or something wrong with how I'm reading it. Is there any explanation that more experienced people know of for what I'm seeing here? (I will turn to the organization that did the genotyping soon, but I wanted to see if I could figure it out on my own or get as much info as possible to help &lt;em&gt;them&lt;/em&gt; troubleshoot first!)&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;SNP Name&lt;/th&gt;&#xA;&lt;th&gt;Sample ID&lt;/th&gt;&#xA;&lt;th&gt;Allele 1 - Forward&lt;/th&gt;&#xA;&lt;th&gt;Allele 2 - Forward&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-A-G&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-A-G-R1&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-A-G-R2&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-A-G-R3&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-A-G-R4&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-A-G-R5&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F1&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F10&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F11&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F12&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F13&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F15&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F2&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F3&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F5&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F6&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F7&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F8&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F9&lt;/td&gt;&#xA;&lt;td&gt;001&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-A-G&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-A-G-R1&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-A-G-R2&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-A-G-R3&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-A-G-R4&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-A-G-R5&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F1&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F10&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F11&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F12&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F13&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F15&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F2&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F3&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F5&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F6&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F7&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F8&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;19:45411941-T-C-F9&lt;/td&gt;&#xA;&lt;td&gt;002&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;td&gt;T&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&lt;/div&gt;&#xA;" OwnerUserId="18965" LastActivityDate="2024-02-15T21:13:42.870" Title="Illumina Final Report seemingly giving impossible genotypes; how do I make sense of this file?" Tags="|illumina|genotyping|" AnswerCount="0" CommentCount="3" ClosedDate="2024-03-07T15:56:58.677" ContentLicense="CC BY-SA 4.0" />
  <row Id="22180" PostTypeId="2" ParentId="22171" CreationDate="2024-02-16T10:45:21.553" Score="0" Body="&lt;h4&gt;fishing out tp53 RNA-Seq reads&lt;/h4&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;go to &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/Taxonomy/Browser/wwwtax.cgi&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI Taxonomy Browser&lt;/a&gt;&lt;/li&gt;&#xA;&lt;li&gt;put &lt;code&gt;Anas platyrhynchos&lt;/code&gt; as a search term&lt;/li&gt;&#xA;&lt;li&gt;click on &lt;code&gt;Anas platyrhynchos&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;on the right you will have a table with &lt;code&gt;SRA Experiments&lt;/code&gt;, click on &lt;code&gt;11,679&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;restrict to &lt;code&gt;RNA&lt;/code&gt; and &lt;code&gt;paired&lt;/code&gt; on the left&lt;/li&gt;&#xA;&lt;li&gt;send it to &lt;code&gt;Run Selector&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;order it by the &lt;code&gt;Bases&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;select few top runs, like &lt;code&gt;SRR16082819&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Now it depends what is your exact goal. You may (separate approaches):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;get mallard transcriptome as complete as possible, use &lt;code&gt;kallisto&lt;/code&gt; just to check that a given pair of FASTQs contains bunch of reads derived from tp53&lt;/li&gt;&#xA;&lt;li&gt;use &lt;code&gt;STAR&lt;/code&gt; to align it to the genome with the contig containing TP53 gene&lt;/li&gt;&#xA;&lt;li&gt;one can also try to fish out the reads out of FASTQ using tp53 cDNA sequence, but this will likely miss i.e. putative alternative 5- &amp;amp; 3-prime part of the gene&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="6314" LastActivityDate="2024-02-16T10:45:21.553" CommentCount="14" ContentLicense="CC BY-SA 4.0" />
  <row Id="22181" PostTypeId="2" ParentId="22176" CreationDate="2024-02-16T11:10:31.760" Score="2" Body="&lt;p&gt;option &lt;code&gt;-pc&lt;/code&gt; is for&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;-pc     Calculate coverage of pair-end fragments.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but your reads are single-end reads&lt;/p&gt;&#xA;" OwnerUserId="71" LastEditorUserId="650" LastEditDate="2024-02-16T15:18:22.347" LastActivityDate="2024-02-16T15:18:22.347" CommentCount="7" ContentLicense="CC BY-SA 4.0" />
  <row Id="22182" PostTypeId="1" AcceptedAnswerId="22184" CreationDate="2024-02-16T14:42:56.287" Score="3" ViewCount="63" Body="&lt;p&gt;I am new to bioinformatics, coming from a background in mathematical biology. I am currently participating in a project where the Principal Investigator (PI), who is a biologist, has tasked me with proving a correlation between the expression and functions of two or more elements. Specifically, one of them is a kinase enzyme and the other is mTOR. The PI has asked me to use omics data or databases for this purpose.&lt;/p&gt;&#xA;&lt;p&gt;As someone with limited experience in bioinformatics, I am struggling to understand how a biological pathway or correlation can be definitively proven, beyond basic statistical methods. Additionally, the ultimate goal of this research is to develop treatments using substances that our lab has discovered, which have shown promise in killing specific cancer cells in murine and cell lines (although our data on this is limited).&lt;/p&gt;&#xA;&lt;p&gt;My question goes as followes- is it possible to demonstrate a connection between these elements using only databases and omics methods? Why might a dynamic modeling approach, such as ordinary differential equations (ODEs) not be enough?&lt;/p&gt;&#xA;&lt;p&gt;I am feeling quite confused and would greatly appreciate any guidance or direction on how to approach this problem effectively.&lt;/p&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="18884" LastEditorUserId="18884" LastEditDate="2024-02-16T16:45:31.410" LastActivityDate="2024-02-16T21:01:01.853" Title="How to prove correlation between gene expression and functions using omics data in bioinformatics?" Tags="|genome|cancer|pathway|correlation|modelling|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22184" PostTypeId="2" ParentId="22182" CreationDate="2024-02-16T19:07:26.810" Score="2" Body="&lt;p&gt;In general terms data mining is needed, then supervised learning and finally ODE, because you are performing DE against enzyme kinetics and thats doable, but you'll need a lot of biochemical knowledge.&lt;/p&gt;&#xA;&lt;p&gt;I would first take the 'omics' data sets and perform a PCA analysis on it and see where your mTOR and kinase genes are located. PCA is known as unsupervised learning and can be accessed by Seurat or better using Scanpy packages. That will help establish the overall behaviour of your key enzymes pre- and post- treatment ... against the control.&lt;/p&gt;&#xA;&lt;p&gt;Basically you want:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Data sets of cancer mice - pre-treatement&lt;/li&gt;&#xA;&lt;li&gt;Data set of cancer mice under treatment (drugs targeting kinases)&lt;/li&gt;&#xA;&lt;li&gt;Healthy controls&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;See what unsupervised learning looks like between the 3 groups and where your enzymes are. The PCA plots then compared for the 3 groups and you want to see the targets you are interested in changing groups, especially the treatment group shifting between the control and the diseased unsupervised plots. This would provide a basis for exploring supervised learning (outlined below).&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Supervised learning&lt;/strong&gt; What you will do is look at gene expression and train it against your phenotype of interest (cancer and recovery from cancer). The training 'target' is the drug the mice are being treated with for each given data set, obviously they'll be a &amp;quot;untreated&amp;quot; and &amp;quot;control&amp;quot; categories. If your model trains correctly &amp;gt;0.85 to 0.9 accuracy ... post-back and I'll explain how to extract the answer. This approach will extract the biochemical pathway for how the cancer treatment works, but requires more work than described here and that provides the framework for the DE.&lt;/p&gt;&#xA;&lt;p&gt;Again, a lot of serious knowledge of biochemistry is needed to deploy ODE, unless you've got a brilliant &lt;em&gt;a priori&lt;/em&gt; - then you could dive straight in. To do that you'd need to know the exact mechanism of how these enzymes are involved in cancer and then parse that data from your data sets (totally different approach to what is described above). The problem with this approach is if someone knows both the maths and genomics (like a reviewer or examiner) and you've made an error, that will that will be a problem. If you've been through an orthodox approach (above), its a lot stronger justification for the ODE, thus difficult to criticise.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Note&lt;/strong&gt; The key is what data set types comprise 'omics', I'm assuming this is RNA-based (thats normal), but it might not be.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2024-02-16T21:01:01.853" LastActivityDate="2024-02-16T21:01:01.853" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22186" PostTypeId="1" AcceptedAnswerId="22187" CreationDate="2024-02-18T13:34:01.023" Score="1" ViewCount="41" Body="&lt;p&gt;I'm facing a challenge regarding the extraction of co-expressing genes from each module. After identifying modules, I've been using exportNetworkToCytoscape to visualize genes that co-express together. However, I solely seek the p-values of the co-expressed genes in each module, similar to the format below:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fromNode        toNode      corr-value      p-value&#xA;ENSG234        ENSG567       0.90           0.001&#xA;ENSG987        ENSG456       0.55           0.03&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;While the exported data provides useful information such as &amp;quot;fromNode,&amp;quot; &amp;quot;toNode,&amp;quot;  &amp;quot;weight&amp;quot;, it lacks the p-values (more important) and corr-value  I require for my analysis.&lt;/p&gt;&#xA;&lt;p&gt;I attempted to use tomSimilarityFromExpr with 'corType = &amp;quot;pearson&amp;quot;' to calculate pairwise correlations between genes, but I'm uncertain about the subsequent steps to obtain the desired p-values.&#xA;My goal is to export the co-expression genes of each module, as facilitated by exportNetworkToCytoscape, while also having the corresponding p-values for these co-expressed genes within each module. I'm not interested in linking the modules to external variables; rather, I solely seek the p-values of the co-expressed genes in each module.&lt;/p&gt;&#xA;&lt;p&gt;Even if there is another way to obtain what I want without going through tomSimilarityFromExpr and exportNetworkToCytoscape, this suggestion would be welcome!&lt;/p&gt;&#xA;&lt;p&gt;Could you kindly offer guidance on how to proceed in obtaining this information?&#xA;Thanks in advance!&lt;/p&gt;&#xA;" OwnerUserId="18977" LastActivityDate="2024-02-19T15:55:01.820" Title="Extracting P-Values for Co-Expressed Genes Within WGCNA Modules" Tags="|r|wgcna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22187" PostTypeId="2" ParentId="22186" CreationDate="2024-02-18T14:04:10.937" Score="2" Body="&lt;p&gt;The simplest way is to calculate the correlation matrix of genes in the module you want and use that as input to &lt;code&gt;exportNetworkToCytoscape&lt;/code&gt;. Something like this:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;inMod = moduleColors==&amp;quot;blue&amp;quot;  ## replace 'blue' with the color or module label you want&#xA;modExpr = datExpr[, inMod];&#xA;modCor = cor(modExpr)&#xA;cytoData = exportNetworkToCytoscape(modCor, threshold = 0, ...)&#xA;## Replace ... above with appropriate arguments&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then add the p-values to the edge data frame returned within &lt;code&gt;cytoData&lt;/code&gt; using function &lt;code&gt;corPvalueStudent&lt;/code&gt;, for example as&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;edgeDF = cytoData$edgeData&#xA;nSamples = nrow(datExpr);&#xA;edgeDF$pValue = corPvalueStudent(edgeDF$weight, nSamples)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can then manipulate the correlations and p-values further as necessary, e.g. export them into a csv file.&lt;/p&gt;&#xA;" OwnerUserId="851" LastEditorUserId="650" LastEditDate="2024-02-19T15:55:01.820" LastActivityDate="2024-02-19T15:55:01.820" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22188" PostTypeId="1" CreationDate="2024-02-19T17:11:06.477" Score="0" ViewCount="39" Body="&lt;p&gt;I'm new to bioinformatics. I have a problem in which I have a FASTA reference genome and lots of reads in FASTQ files. Some of them could be contaminants, so I'd like to filter them out and get only the ones that align to the reference genome. I also have access to the BAM/BAM.BAI files associated with each FASTQ, although I can't know how they were aligned.&lt;/p&gt;&#xA;&lt;p&gt;Thanks&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2024-02-19T20:17:37.720" Title="How to know if FASTQ/BAM is from reference genome (FASTA)?" Tags="|sequence-alignment|fasta|fastq|read-mapping|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22189" PostTypeId="2" ParentId="22188" CreationDate="2024-02-19T17:49:19.640" Score="3" Body="&lt;p&gt;Your post has multiple questions but one of them is not really a question so I'll address everything here:&lt;/p&gt;&#xA;&lt;p&gt;1. &amp;quot;Some of them could be contaminants, so I'd like to filter them out and get only the ones that align to the reference genome&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Just align the reads. Contaminants should not align well and should be discarded owing to poor mapping quality score.&lt;/p&gt;&#xA;&lt;p&gt;2. How to figure out if the FASTA file was used for the alignment&lt;/p&gt;&#xA;&lt;p&gt;This is not really a 100% perfect solution but the BAM header should contain the exact command used for the alignment, which should contain the ref FASTA with path. You can also compare contig names and lengths between the FASTA and the BAM &lt;code&gt;@SQ&lt;/code&gt; lines.&lt;/p&gt;&#xA;" OwnerUserId="650" LastActivityDate="2024-02-19T17:49:19.640" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22190" PostTypeId="2" ParentId="22188" CreationDate="2024-02-19T18:59:21.987" Score="2" Body="&lt;p&gt;In general, this answer wouldn't be more correct than @Ram RS' answer but here is how I would approach:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;p&gt;I would start everything from scratch, at the end of the day you will most probably need to document the exact genome version and its associated annotation file. It looks like you do not have this information at present.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I don't know the experimental set-up but if you are worried about &amp;quot;contaminants&amp;quot;, you can have an extra alignment step using &lt;code&gt;BBSplit&lt;/code&gt; (&lt;a href=&quot;https://www.seqanswers.com/forum/bioinformatics/bioinformatics-aa/35881-introducing-bbsplit-read-binning-tool-for-metagenomes-and-contaminated-libraries&quot; rel=&quot;nofollow noreferrer&quot;&gt;link&lt;/a&gt;), it is capable of mapping to multiple genomes simultaneously.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;I would give a try with &lt;code&gt;nfcore/rnaseq&lt;/code&gt;, it abstracts away a lot of complex bioinformatics steps so that you can focus on understanding what these tools do instead of dealing with installing and running these tools. For example you can just use the &lt;code&gt;--bbsplit-fasta-list&lt;/code&gt; argument to get rid of your contaminants.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="3541" LastEditorUserId="650" LastEditDate="2024-02-19T20:17:37.720" LastActivityDate="2024-02-19T20:17:37.720" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22191" PostTypeId="2" ParentId="22000" CreationDate="2024-02-19T20:03:23.913" Score="1" Body="&lt;p&gt;As someone who works with eCLIP data from ENCODE, I think I can help out.&lt;/p&gt;&#xA;&lt;p&gt;You need to think of this more similar to ATAC or RNA-Seq, whereby you actually get a counts matrix, normalize that, and then apply math.&lt;/p&gt;&#xA;&lt;p&gt;Another good idea would be to look through this eCLIP analysis document provided by Gene Yeo of ENCODE for more specific ideas.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.encodeproject.org/documents/1f171ac6-a36a-41ac-b632-741aeb47aad2/@@download/attachment/eCLIP_analysisSOP_v2.3.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.encodeproject.org/documents/1f171ac6-a36a-41ac-b632-741aeb47aad2/@@download/attachment/eCLIP_analysisSOP_v2.3.pdf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="17429" LastActivityDate="2024-02-19T20:03:23.913" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22192" PostTypeId="1" CreationDate="2024-02-20T08:24:00.213" Score="2" ViewCount="82" Body="&lt;p&gt;I am still a newbie in the field of bioinformatics. I have been attempting to visualize unique and common genes across six different tissues using the &lt;code&gt;UpSetR&lt;/code&gt; plot package. This script was used to generate the plot; however, out of the six tissues, only six could be on the plot.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(data.table)&#xA;library(UpSetR)&#xA;d &amp;lt;- data.table::fread(text = &amp;quot;Brainstem    Cerebellum  Olfactory Bulb  Heart   Kidney  Liver&#xA;Rac3    Coq9    Ndufs7  Slc25a10    Aacs    Acss3&#xA;Ndufb5  Mtnd5   Cotl1   Adhfe1  Aass    Cyp4v2&#xA;Clic6   Kpna4   Gabarapl2   Ddx5    Abcc4   Slc7a2&#xA;Rbmx    Snrpg   Rbm3    Pgd Acot12  Ehhadh&#xA;Ptcd3   Micos13 Atp5f1e Gss Acsf2   Selenbp2&#xA;Ndufa13 Ndufa5  Ndufb3  Arl3    Acsm3   Ugt2b17&#xA;Amt Ndufa10 Cstf3   Shmt2   Acy3    Gstp1&#xA;Ndufs1  Bcr Mtco2   Tpt1    Adk Pdk1&#xA;Acan    Mpc1    Bcap31  Me1 Ahcyl1  Bgn&#xA;Wasl    Cfdp1   Ndufaf2 Abcg2   Aif1l   Col4a2&#xA;Rbm3    Rps15   Gfap    H1-1    Akr1b1  Ttn&#xA;Mtnd5   Snu13   Atp6v0c Pcbp2   Amacr   Cox6b1&#xA;Ddc Lin7c   Tmed10  Ddx17   Atp2b1  Mup2&#xA;Tmc7    Ndufb5  S100a5  Slc25a1 Ca5b    Cyp2d9&#xA;Erc1    Ampd3   Ap3b1   H1-3    Cbr1    Cyp2b10&#xA;Nup35   Ipo5    Gja1    Cstb    Cbr3    Ftl1&#xA;Ndufb2  Ndufb10 Ndufb7  Tst Ces1f   Cyp8b1&#xA;Ndufb1  Ebp Ciapin1 Vat1    Cnpy2   Rbm3&#xA;Rexo2   Rabac1  Ndufb4  Glul    Col1a1  Ehd1&#xA;Cbln3   Wdfy2   Adk Rpl29   Col1a2  Cyp4a10&#xA;Arfgef3 Clcn6   Gfpt1   Pc  Col4a1  Ndufaf2&#xA;Dagla   Vbp1    Rpl28   Acy3    Col4a2  Cops2&#xA;Slc25a31    Nae1    Dgkz    Akr1a1  Cox7a1  Cyp2a4&#xA;Tph2    Gucy1a1 Tpbg    Pecr    Cpne5   Rbp1&#xA;Actbl2  Ap1s1   Gprc5b  Hnrnpd  Creg1   S100a13&#xA;Ndufb9  Ndufs1  Acyp2   Serpina1c   Crot    Gsta4&#xA;Timm10b Ndufa3  Kirrel2 Hsd17b11    Cryab   Ldhb&#xA;Syt12   Ndufs4  Ndufb11 Rpn1    Cyp2d26 Dhtkd1&#xA;Unc13c  Dnajb4  Ndufb10 Acly    Cyp2d9  Ndufb7&#xA;Serpina1d   Chmp4b  Wdr6    Atp6v1b2    Cyp2j6  Acot13&#xA;Yme1l1  Pdcd6   Mpst    Pxmp2   Cyp4a10 Fmo3&#xA;Aprt    Ndufaf2 Smdt1   Arpc3   Cyp4a14 Clta&#xA;Mtnd1   Igsf21  Sec61g  Kpna6   Cyp4b1  Eftud2&#xA;Jpt2    Stxbp3  C1qb    Atp6v1g1    Cyp7b1  Tex15&#xA;Ndufv3  Ndufb1  Ndufa2  Slc25a31    Ddc Ndufs1&#xA;Chordc1 Tpd52l2 Hddc2   Aldh3a2 Dpyd    Sult1c2&#xA;Cbln1   Sh3gl1  Agfg1   Gng12   Echdc3  Srl&#xA;Rbbp4   Pnn Manf    Tmpo    Eea1    Aimp1&#xA;Ndufaf2 Prodh   Kcnj10  Acox1   Ehhadh  Cyp2b9&#xA;Egfr    Exoc7   Ptgds   Rhoa    Folh1   Cyp4a14&#xA;Ca4 Ubxn1   Rpa3    Mapre1  Fth1    Csad&#xA;Vps33b  Nfu1    Eif3e   Gk  Ftl1    Gck&#xA;Ndufb11 Etnppl  Isg15   Rps2    Gsta2   Ss18l2&#xA;Gatad2b Spock1  Tmsb10  Atp6v1e1    Gsta3   Hebp1&#xA;Scn8a   Tagln2  Chmp1a  Sult1d1 Gsta4   Mup20&#xA;Ensa    Cbln1   Mobp    Plin1   Hacl1   Nudt7&#xA;Hkdc1   Sart3   Mtmr9   Idh1    Hmgb2   Tmem19&#xA;Ndufa5  Ndufs6  Ptprf   Cpne1   Hmgcs2  Mup18&#xA;Prpf6   Tagln   Hmox2   Prodh   Hmgn2   Selenbp1&#xA;Glipr2  Ndufs8  Ptprd   Asl Hpd Mpp1&#xA;Serpina3k   Drc7    Pmm1    Stmn1   Hsd3b2  Gstt2&#xA;Serpina1a   Hspg2   Pde6d   Abhd14b Iqgap2  Tmco1&#xA;Camk2g  Aamdc   Ipo7    Cryz    Krt18   Slc30a5&#xA;Nmnat1  Rpl26   Ndufc2  Amacr   Krt7    Hsd17b6&#xA;Tbcd    Nmral1  Csde1   Hmgb2   Krt8    Dusp3&#xA;Ckm Uck1    Itpa    Sfxn1   Ldhd    Aass&#xA;Mtif2   Lysmd2  Ndufa5  Acad11  Lmnb2   Sh3bgrl3&#xA;Hmgn5   Sars2   Actr3b  Alox12  Lsm6    Tex44&#xA;Tmem245 Cep170b Meis2   H2ac20  Mapre1  Cyp7b1&#xA;Cbx3    Bzw1    Bcas3   Csad    Mbp Tymp&#xA;Gjc3    Apex2   Ensa    Ucp1    Mpv17l  Fth1&#xA;Ugt8    Nucks1  Serpina1a   Gpd1    Mrpl22  Serpina3m&#xA;Lsmem1  Ndufb6  Serpina3k   Xab2    Mrpl23  Ces3a&#xA;Rbsn    Cp  Tnks1bp1    Adh1    Mrpl40  Trmt112&#xA;Cpped1  Npm1    Prnp    Dcps    Mrps26  Rdh16&#xA;Arpp19  Rhoa    Uap1l1  Dhx9    Myh6    Ndufs4&#xA;Ndufa6  Ndufa8  Ndufs3  Ezr Ndufa10 Hspa1b&#xA;Ndufs2  Ndufb2  Ndufv2  Sri Ndufa12 Ndufv1&#xA;Mug1    Ndufv2  Ndufs4  Ptrh2   Ndufa2  Sult1d1&#xA;Ndufs8  Nek9    Cltb    Mecr    Ndufa5  Acot3&#xA;Ndufb7  Ndufa12 Ndufa12 Sqor    Ndufa6  Ndufa12&#xA;Ndufv1  Ndufs5  Wdfy2   Echdc2  Ndufa7  Upb1&#xA;Ndufs3  Ndufa2  Ndufa10 Fabp1   Ndufa9  Ttc39c&#xA;Dnah8   Lin7b   Mtnd1   Atp2a3  Ndufaf2 Sult1a1&#xA;Ndufc2  Nek7    Ctsa    Gsr Ndufb7  Sqor&#xA;Ndufa10 Ndufs3  Ndufv3  Adrm1   Ndufs1  Aox3&#xA;Ndufs7  Ndufv1  Ndufs6  Egfr    Ndufs2  Sult2a8&#xA;Ndufs5  Ndufb4  Clybl   Gmpr    Ndufs3  Hyou1&#xA;Gcfc2   Ndufa9  Stat1   Gdap2   Ndufs4  Egfr&#xA;Ndufb3  Ndufa13 Rpl23   Ywhaq   Ndufs6  Ndufa7&#xA;Ndufb4  Ndufv3  Ndufv1  Serpina1e   Ndufv1  Ndufa6&#xA;Ndufa12 Ndufs7  Rif1    Lta4h   Ndufv2  Ndufa9&#xA;Ndufb8  Ndufb3  Ndufb8  Kifbp   Ndufv3  Srd5a1&#xA;Serpina1e   Pgam2   Capg    Psma4   Nqo1    Mpc1&#xA;Ndufb10 Ndufc2  Amt Mtr Nucks1  Keg1&#xA;Ndufs6  Ndufa6  Ndufs1  Cps1    Nudt19  Serpina3k&#xA;Fam136a Cltb    Cep170b Ehhadh  Pals2   Gsta2&#xA;Ndufv2  Ndufb11 Ndufa13 Flii    Pecr    Gstt3&#xA;Ndufa11 Pcdhga4 Vwa1    Aldob   Ppid    Metap1d&#xA;Ndufa1  Mif Tinagl1 H1-4    Ptprk   Igf2bp3&#xA;Ndufb6  Ndufb9  Ndufa11 Kyat3   Pyroxd2 Ndufs5&#xA;S100b   H1-0    Actl6b  Coasy   Rbm3    Nudt19&#xA;Acta1   Rpl29   Ndufa8  Snrpe   Reep6   Atp6v1g1&#xA;Ndufa2  Ndufs2  Lsmem1  Vta1    Retsat  Maoa&#xA;Nfia    Acad8   Ilk Osbpl8  Sephs2  Mgst3&#xA;Ndufa7  Ndufa11 Ndufa7  Agpat2  Serpina3m   Comt&#xA;Ndufa8  Ech1    Ndufs5  Clic1   Sh3bgrl3    Mat1a&#xA;Ndufs4  Chmp2a  Msn Macroh2a1   Slc12a3 Acot4&#xA;Alas1   Micos10 Nup85   Sars1   Slc22a27    Fuom&#xA;Phyhd1  S100b   Nucks1  Pabpc1  Slc22a6 Mrpl14&#xA;Mtnd4   Synpr   Tceal3  H2ac21  Slc6a19 Serpinb1a&#xA;Ndufa9  Ndufa7  Ndufa9  Nudt8   Stat1   Ces3b&#xA;Cdh10   Gnpda2  Cep152  Krt18   Tor1aip1    Prelp&#xA;Ring1   Rbfox2  Etnppl  Fuom    Ugt1a2  Mup3&#xA;Tmsb10  S1pr1   Acta1   Cyb5b       Cyp2c37&#xA;Mrpl43  mt-Nd2  Myh11   Slc25a35        Col12a1&#xA;Icmt    Rnps1   Ddx46   Rbm3        Endov&#xA;Ncln    Ap3s1   Slc44a2 Gpd1l       S100a9&#xA;Gh1 Rbx1    Ndufa6  Lrrc59      Slc25a4&#xA;Bola3   Cat Ifit1   Hmgn2       Lum&#xA;Ahsg    Atp5if1 Vps52   Tkt     Myl1&#xA;Pdlim1  Rpl18a  Rac2    Hmgb1       Col1a1&#xA;Ca3 Cyp51a1 Tmco5b  H1-2        Tpm2&#xA;Crim1   Pkn1    Bclaf1  H4c1        Myh7&#xA;Supt4h1a    Ppp3r1  Ube2v2  Stat1       Dcn&#xA;S100a9  Sltm    Atp5if1 Aldh9a1     Cyfip1&#xA;Shisa6  Chmp7   Mcm6    Elavl1      Phf11&#xA;Adcy5   Bcl7c   Vac14   H2bc7       Cse1l&#xA;Slc4a2  Lama5   Aqp1    H1-5        Col1a2&#xA;Rgs8    Prnp    Hnrnpul1    Fah     Fabp3&#xA;Asph    Nup85   Dnajc5  Aldh1l2     Acta1&#xA;Hsd17b2 Iah1    Fam136a H3-5        Myh8&#xA;Prx Mbnl2   Ublcp1  H2ac4       Ogn&#xA;Bzw2    Rbm3    Cmtm5   Ldhd        Soat2&#xA;Mboat4  Hdgfl3  Nipa1   Cryl1       Sf3b5&#xA;Cep76   Hacd2   Bzw1    Ncl     Btk&#xA;Cacna1e Asph    Tgm2    Gpd2        Cela1&#xA;Ldhd    Lpcat4  Akr1e2  Ndrg1       Uap1&#xA;    Iqsec3  Nek7    Acacb       Actn2&#xA;    Kyat3   Gabbr1  Rcc2        Tnnt3&#xA;    Hnrnpul1    Pld2    Fgb     Upp2&#xA;    Atp5f1e C1qa    Ces1f       Tgtp2&#xA;    Mrps21  Irgm1   Gclc        Tpm1&#xA;    Itpa    Try5    Pvalb       Myh1&#xA;    Sec61g  Alg2    Bdh1        Tex264&#xA;    Acads   Pik3c3  Sord        Amy2a5&#xA;    Poldip2 Dynll2  Ssb     Cela2a&#xA;    Rpl36a  Ckm Fabp4       Ckmt2&#xA;    Slc6a5  Gstm5   Apoc4       Fmod&#xA;    Fn1 Urod    Bpnt1       Des&#xA;    Usp19   Rbbp7   Fasn        Ckm&#xA;    Prtn3   Ociad1  Hcfc1       Myl3&#xA;    Sh3bgrl Rpl29   Cyp4b1      Try5&#xA;    Gclm    Ngef    Ganab       Nrde2&#xA;    Egfr    Cryab   Tkfc        Ctrb1&#xA;    Mobp    Rnps1   Acaca       Ldb3&#xA;    Abhd3   Ugdh    Cbr2&#xA;    Cirbp   Iglon5  Lipe&#xA;    Mast1   C1qc    Hnrnpa3&#xA;    Slc44a2 Eif2b4  Septin9&#xA;    Magi2   Ythdf3  Hnrnpf&#xA;    Spag7   Rps15   Hnrnpab&#xA;    C1qbp   Atg4b   Sfxn3&#xA;    Pllp    Pvalb   Ctsz&#xA;    Smad3   Lig1    Coro1c&#xA;    Exoc6b  Ehd1    S100a11&#xA;    Chtop   Fn3krp  Ssrp1&#xA;    Hmgn2   Cxadr   Ptbp1&#xA;    Abhd6   Tmem186 Rac2&#xA;    Cers1   Lsm4    Nup85&#xA;    Dsg2    Rpl10   Nherf1&#xA;    Arl8b   Ca1 Coro1a&#xA;    Rbck1   Mindy2  Atp6v1a&#xA;    Abhd4   Rcc1    Arpc1b&#xA;    Frrs1l  Sart1   Krt8&#xA;    Rnf214  H3-5    Ruvbl1&#xA;    Cdc42ep1    H1-0    Pck1&#xA;    Sf3a3   Clca3a1 Hnrnpc&#xA;    Metap2  Lpcat4  Rida&#xA;    Eif2b4  H2-K1   Ces1d&#xA;    Inpp5f  S100a13 Smarcc2&#xA;    Urod    Trappc4 Ewsr1&#xA;    Nmt2    Ca4 Akr1c21&#xA;    Alg11   S100a4  Serpina3m&#xA;    Rad23b  Hbb-b2  Chmp1a&#xA;    Drap1   Celf1   Actbl2&#xA;    Hddc2   Rap2a   Ndufa1&#xA;    App Ca8 Ndufs4&#xA;    Cpsf6   Fgg Rala&#xA;    Ppp2r5c Ube2fb  C8a&#xA;    Kiss1   Eif3k   Mug1&#xA;    Rgs6    Ctnnbl1 Rrp12&#xA;    Txndc9  Timm10  mt-Nd2&#xA;    Ehd4    Uvrag   Apoa2&#xA;    Slc25a31    Arid1a  Ndufaf2&#xA;    Cpped1  Krt8    Mtx1&#xA;    Rbm27   Bpifb9a N6amt1&#xA;    Cmtm5   Col1a2  Mrpl49&#xA;    Gh1 Actbl2  Ndufs6&#xA;    Elp3    Evl Cisd1&#xA;    Atp6v0c Rpl39   Ndufa12&#xA;    Ints3   Col6a1  Serpina3k&#xA;    Tmem68  Map4k1&#xA;    Tiam1   Atp6v1e2&#xA;    Cox7a2l Col1a1&#xA;    Arfgef3 Rpl35&#xA;    Higd1a  Cpsf6&#xA;    Ptgds   Lum&#xA;    Eny2&#xA;    Tmsb10&#xA;    Per2&#xA;    Tatdn1&#xA;    Rpl30&#xA;    Irag2&#xA;    Otud6b&#xA;    Myl1&#xA;    Akap10&#xA;    Cav1&#xA;    Fbxl20&#xA;    Adh1&#xA;    Ehhadh&#xA;    Atp4a&#xA;    Naga&#xA;    Tceal3&#xA;    Col1a1&#xA;    Celf1&#xA;    Cops6&#xA;    Vps11&#xA;    Dync1i2&#xA;    Hm13&#xA;    Eif2s2&#xA;    Fgg&#xA;    Ralgapa1&#xA;    Ipo7&#xA;    Tmod3&#xA;    Fgb&#xA;    Phpt1&#xA;    Eif3j2&#xA;    Lars2&#xA;    Snrpb2&#xA;    Mug1&#xA;    Slc4a1&#xA;    Arg1&#xA;    Celf2&#xA;    Actbl2&#xA;    Krt6a&#xA;    Fga&#xA;    Ufd1&#xA;    Fam136a&#xA;    Ntmt1&#xA;    Opalin&#xA;    Omp&#xA;    Acsf3&#xA;    Lig3&#xA;    Xpo5&#xA;    Tmem240&#xA;    Aldob&#xA;    Adprs&#xA;    Stx1a&#xA;    H2bc14&#xA;    Serpina1d&#xA;    Cldn11&#xA;    Cyp2j5&#xA;    Slc6a11&#xA;    Cpm&#xA;    Acp2&#xA;    Ca1&#xA;    Cd44&#xA;    Cplx2&amp;quot;)&#xA;fld &amp;lt;- fromList(as.list(d))&#xA;upset(fld)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As shown in the figure below, only five tissues could be seen, with the exception of the kidney, even in the presence of overlapping genes between the kidney and other tissues.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/pgvZL.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/pgvZL.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Can you kindly assist in this regard, I do not know if there is something wrong with my input data or the script itself.&lt;/p&gt;&#xA;" OwnerUserId="18984" LastEditorUserId="650" LastEditDate="2024-02-20T16:47:39.427" LastActivityDate="2024-02-21T08:05:56.940" Title="How to visualize unique and overlapping genes in upSet R plot" Tags="|r|upsetr|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22194" PostTypeId="1" AcceptedAnswerId="22196" CreationDate="2024-02-20T15:50:30.813" Score="1" ViewCount="41" Body="&lt;p&gt;I want to see how the gene expression differs in breast cancer between three species, and I am using &lt;code&gt;DESeqDataSetFromMatrix&lt;/code&gt; on my count table.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;dds &amp;lt;- DESeqDataSetFromMatrix(countData = whole_data, colData = whole_col_data, design = ~ species)&#xA;dds$species &amp;lt;- factor(dds$species, levels=c(&amp;quot;human&amp;quot;,&amp;quot;canine&amp;quot;,&amp;quot;mouse&amp;quot;))&#xA;dds$species &amp;lt;- relevel(dds$species, ref = &amp;quot;human&amp;quot;) &#xA;dds &amp;lt;- DESeq(dds)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Although I specified my levels for all three species my results are still as follows and only compare between two species:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;log2 fold change (MLE): species **mouse vs human** &#xA;Wald test p-value: species **mouse vs human** &#xA;DataFrame with 13556 rows and 6 columns&#xA;        baseMean log2FoldChange     lfcSE      stat      pvalue       &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Can someone please explain to me why that is happening?&lt;/p&gt;&#xA;" OwnerUserId="18867" LastEditorUserId="650" LastEditDate="2024-02-20T16:28:30.097" LastActivityDate="2024-02-20T16:28:30.097" Title="Multi Factor in Deseq2 Gene enrichment analysis" Tags="|sequence-analysis|deseq2|gene-expression|go-enrichment|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22195" PostTypeId="2" ParentId="22192" CreationDate="2024-02-20T15:58:09.027" Score="2" Body="&lt;p&gt;No idea how your code looks like before pasting into the web form, but in general it is not a great idea pasting +260 rows/lines of data in your script.&#xA;From what I can see in the form you have columns of data separated by different number of spaces even in the header (2 and 4 spaces).&#xA;Because of this it is impossible to figure out to which column is which.&lt;/p&gt;&#xA;&lt;p&gt;The easiest is to keep the data in the &lt;code&gt;TSV&lt;/code&gt; or &lt;code&gt;CSV&lt;/code&gt; format file and make sure there are no issues with column delimiters.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;EDIT&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have saved the Google Spreadsheet as &lt;code&gt;de_genes.csv&lt;/code&gt;, edited just in case &lt;code&gt;Olfactory Bulb&lt;/code&gt; to &lt;code&gt;Olfactory_Bulb&lt;/code&gt; then did:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;in_fn &amp;lt;- &amp;quot;de_genes.csv&amp;quot;&#xA;&#xA;df = fread(in_fn)&#xA;&#xA;de_lists &amp;lt;- fromList(as.list(df))&#xA;&#xA;&#xA;upset(de_lists, sets=c( &amp;quot;Brainstem&amp;quot;,    &amp;quot;Cerebellum&amp;quot;,   &amp;quot;Olfactory_Bulb&amp;quot;,   &amp;quot;Heart&amp;quot;, &amp;quot;Kidney&amp;quot;, &amp;quot;Liver&amp;quot;) , order.by = &amp;quot;freq&amp;quot;)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I got:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/f1YSF.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/f1YSF.png&quot; alt=&quot;upsetR plot&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6314" LastEditorUserId="6314" LastEditDate="2024-02-21T08:05:56.940" LastActivityDate="2024-02-21T08:05:56.940" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22196" PostTypeId="2" ParentId="22194" CreationDate="2024-02-20T16:04:56.950" Score="3" Body="&lt;p&gt;If you don't tell DESeq what contrast to make, it just does the default contrast.  Which is your reference versus the last other one alphabetically.&lt;/p&gt;&#xA;" OwnerUserId="2388" LastActivityDate="2024-02-20T16:04:56.950" CommentCount="10" ContentLicense="CC BY-SA 4.0" />
  <row Id="22197" PostTypeId="2" ParentId="15609" CreationDate="2024-02-20T18:39:31.733" Score="1" Body="&lt;p&gt;It seems like the &lt;code&gt;seq_start&lt;/code&gt; and &lt;code&gt;seq_end&lt;/code&gt; are sensitive to the direction. Try swapping the two.&lt;/p&gt;&#xA;" OwnerUserId="18987" LastEditorUserId="650" LastEditDate="2024-02-20T19:12:46.153" LastActivityDate="2024-02-20T19:12:46.153" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22198" PostTypeId="1" CreationDate="2024-02-20T18:58:25.173" Score="1" ViewCount="31" Body="&lt;p&gt;I am interested in using AlphaFold2 to identify residues within a homodimer that are important for the protein-protein interaction. I have been able to use ColabFold to produce a predicted dimer structure - however, as far as I can tell AlphaFold does not actually identify the residues or regions that are actually important for binding. Is there a way to manipulate the software to do this?&#xA;Thanks!&lt;/p&gt;&#xA;" OwnerUserId="18956" LastActivityDate="2024-04-04T05:13:12.167" Title="Can alphafold 2 be used to identify residues important for binding?" Tags="|alphafold|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22199" PostTypeId="2" ParentId="15609" CreationDate="2024-02-20T22:23:11.523" Score="1" Body="&lt;p&gt;Lets assume you can get the &lt;code&gt;blastn&lt;/code&gt; output in a &lt;code&gt;BED&lt;/code&gt; format:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;genomeA_contig001    hit_start    hit_end    query_transcript    score   strand&#xA;genomeB_contig021    hit_start    hit_end    query_transcript    score   strand&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;which will contain all the unknown hits to all genomes.&lt;/p&gt;&#xA;&lt;p&gt;All you need is a set of &lt;code&gt;GTF&lt;/code&gt; files specific for &lt;code&gt;genomeA&lt;/code&gt;, &lt;code&gt;genomeB&lt;/code&gt; and so on where you add the suitable prefix (&lt;code&gt;genomeA_&lt;/code&gt; for &lt;code&gt;GTF&lt;/code&gt; describing genes/exons in that genome and so on). For transcript hits you may start with just the annotated exons. Next run:&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;bedtools intersect -wao -a all.transctipts.hits.bed -b all_annotated_exons.gtf my_result.tsv&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;to get all the overlaps with number of overlapping bases.&#xA;Filter and sort the &lt;code&gt;my_result.tsv&lt;/code&gt; possibly importing it into a data frame using &lt;code&gt;pandas&lt;/code&gt; or &lt;code&gt;polars&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="6314" LastActivityDate="2024-02-20T22:23:11.523" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22200" PostTypeId="1" CreationDate="2024-02-21T06:30:51.273" Score="2" ViewCount="86" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://github.com/alexdobin/STAR/issues/2063&quot; rel=&quot;nofollow noreferrer&quot;&gt;github&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am using STAR 2.7.9a. In the last two days, I am getting an error when running alignments with STAR (independently from the data and genome index I am using). What could be causing this error, which is reported in the next paragraph? Notably, STAR returns this error even when performing alignments that used to complete successfully in the past, before this issue started. Thank you very much for your attention and help on this!&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;INFO: localdir at /localscratch/8889241.roberto.albanese&#xA;INFO: your job will run on local system.&#xA;STAR: bgzf.c:158: bgzf_open: Assertion `compressBound(0xff00) &amp;lt; 0x10000' failed.&#xA;*** Aborted&#xA;Register dump:&#xA;&#xA; RAX: 0000000000000000   RBX: 0000000000000006   RCX: 00007ffff684a37f&#xA; RDX: 0000000000000000   RSI: 00007ffffffdb580   RDI: 0000000000000002&#xA; RBP: 00007ffff699d5f8   R8 : 0000000000000000   R9 : 00007ffffffdb580&#xA; R10: 0000000000000008   R11: 0000000000000246   R12: 0000000000594eab&#xA; R13: 0000000000594f18   R14: 000000000000009e   R15: 00007ffffffeb540&#xA; RSP: 00007ffffffdb580&#xA;&#xA; RIP: 00007ffff684a37f   EFLAGS: 00000246&#xA;&#xA; CS: 0033   FS: 0000   GS: 0000&#xA;&#xA; Trap: 00000000   Error: 00000000   OldMask: 00000000   CR2: 00000000&#xA;&#xA; FPUCW: 0000037f   FPUSW: 00000000   TAG: 00007fff&#xA; RIP: f76a5f3b   RDP: 00000000&#xA;&#xA; ST(0) 0000 0000000000000000   ST(1) 0000 0000000000000000&#xA; ST(2) 0000 0000000000000000   ST(3) 0000 0000000000000000&#xA; ST(4) 0000 0000000000000000   ST(5) 0000 0000000000000000&#xA; ST(6) ffff b000000000000000   ST(7) b000 b000000000000000&#xA; mxcsr: 1fa0&#xA; XMM0:  00000000000000000000000000000000 XMM1:  00000000000000000000000000000000&#xA; XMM2:  00000000000000000000000000000000 XMM3:  00000000000000000000000000000000&#xA; XMM4:  00000000000000000000000000000000 XMM5:  00000000000000000000000000000000&#xA; XMM6:  00000000000000000000000000000000 XMM7:  00000000000000000000000000000000&#xA; XMM8:  00000000000000000000000000000000 XMM9:  00000000000000000000000000000000&#xA; XMM10: 00000000000000000000000000000000 XMM11: 00000000000000000000000000000000&#xA; XMM12: 00000000000000000000000000000000 XMM13: 00000000000000000000000000000000&#xA; XMM14: 00000000000000000000000000000000 XMM15: 00000000000000000000000000000000&#xA;&#xA;Backtrace:&#xA;/lib64/libc.so.6(gsignal+0x10f)[0x7ffff684a37f]&#xA;/lib64/libc.so.6(abort+0x127)[0x7ffff6834db5]&#xA;/lib64/libc.so.6(+0x21c89)[0x7ffff6834c89]&#xA;/lib64/libc.so.6(+0x2fa76)[0x7ffff6842a76]&#xA;/home/roberto.albanese/software/STAR-2.7.9a/bin/Linux_x86_64/STAR[0x53b3b0]&#xA;/home/roberto.albanese/software/STAR-2.7.9a/bin/Linux_x86_64/STAR[0x4af101]&#xA;/home/roberto.albanese/software/STAR-2.7.9a/bin/Linux_x86_64/STAR[0x40da52]&#xA;/lib64/libc.so.6(__libc_start_main+0xf3)[0x7ffff6836493]&#xA;/home/roberto.albanese/software/STAR-2.7.9a/bin/Linux_x86_64/STAR[0x419125]&#xA;&#xA;Memory map:&#xA;&#xA;00400000-005bc000 r-xp 00000000 00:48 4804921705                         /home/roberto.albanese/software/STAR-2.7.9a/bin/Linux_x86_64/STAR&#xA;007bc000-007bd000 r--p 001bc000 00:48 4804921705                         /home/roberto.albanese/software/STAR-2.7.9a/bin/Linux_x86_64/STAR&#xA;007bd000-007cb000 rw-p 001bd000 00:48 4804921705                         /home/roberto.albanese/software/STAR-2.7.9a/bin/Linux_x86_64/STAR&#xA;007cb000-00832000 rw-p 00000000 00:00 0                                  [heap]&#xA;7ffff660f000-7ffff6612000 r-xp 00000000 08:02 36793927                   /usr/lib64/libdl-2.28.so&#xA;7ffff6612000-7ffff6811000 ---p 00003000 08:02 36793927                   /usr/lib64/libdl-2.28.so&#xA;7ffff6811000-7ffff6812000 r--p 00002000 08:02 36793927                   /usr/lib64/libdl-2.28.so&#xA;7ffff6812000-7ffff6813000 rw-p 00003000 08:02 36793927                   /usr/lib64/libdl-2.28.so&#xA;7ffff6813000-7ffff69cf000 r-xp 00000000 08:02 36776071                   /usr/lib64/libc-2.28.so&#xA;7ffff69cf000-7ffff6bce000 ---p 001bc000 08:02 36776071                   /usr/lib64/libc-2.28.so&#xA;7ffff6bce000-7ffff6bd2000 r--p 001bb000 08:02 36776071                   /usr/lib64/libc-2.28.so&#xA;7ffff6bd2000-7ffff6bd4000 rw-p 001bf000 08:02 36776071                   /usr/lib64/libc-2.28.so&#xA;7ffff6bd4000-7ffff6bd8000 rw-p 00000000 00:00 0 &#xA;7ffff6bd8000-7ffff6bf3000 r-xp 00000000 08:02 37169965                   /usr/lib64/libpthread-2.28.so&#xA;7ffff6bf3000-7ffff6df2000 ---p 0001b000 08:02 37169965                   /usr/lib64/libpthread-2.28.so&#xA;7ffff6df2000-7ffff6df3000 r--p 0001a000 08:02 37169965                   /usr/lib64/libpthread-2.28.so&#xA;7ffff6df3000-7ffff6df4000 rw-p 0001b000 08:02 37169965                   /usr/lib64/libpthread-2.28.so&#xA;7ffff6df4000-7ffff6df8000 rw-p 00000000 00:00 0 &#xA;7ffff6df8000-7ffff6e0f000 r-xp 00000000 08:02 100684981                  /cm/local/apps/gcc/10.2.0/lib64/libgcc_s.so.1&#xA;7ffff6e0f000-7ffff700e000 ---p 00017000 08:02 100684981                  /cm/local/apps/gcc/10.2.0/lib64/libgcc_s.so.1&#xA;7ffff700e000-7ffff700f000 r--p 00016000 08:02 100684981                  /cm/local/apps/gcc/10.2.0/lib64/libgcc_s.so.1&#xA;7ffff700f000-7ffff7010000 rw-p 00017000 08:02 100684981                  /cm/local/apps/gcc/10.2.0/lib64/libgcc_s.so.1&#xA;7ffff7010000-7ffff704e000 r-xp 00000000 08:02 100684988                  /cm/local/apps/gcc/10.2.0/lib64/libgomp.so.1.0.0&#xA;7ffff704e000-7ffff724d000 ---p 0003e000 08:02 100684988                  /cm/local/apps/gcc/10.2.0/lib64/libgomp.so.1.0.0&#xA;7ffff724d000-7ffff724e000 r--p 0003d000 08:02 100684988                  /cm/local/apps/gcc/10.2.0/lib64/libgomp.so.1.0.0&#xA;7ffff724e000-7ffff724f000 rw-p 0003e000 08:02 100684988                  /cm/local/apps/gcc/10.2.0/lib64/libgomp.so.1.0.0&#xA;7ffff724f000-7ffff73d0000 r-xp 00000000 08:02 36888910                   /usr/lib64/libm-2.28.so&#xA;7ffff73d0000-7ffff75cf000 ---p 00181000 08:02 36888910                   /usr/lib64/libm-2.28.so&#xA;7ffff75cf000-7ffff75d0000 r--p 00180000 08:02 36888910                   /usr/lib64/libm-2.28.so&#xA;7ffff75d0000-7ffff75d1000 rw-p 00181000 08:02 36888910                   /usr/lib64/libm-2.28.so&#xA;7ffff75d1000-7ffff7794000 r-xp 00000000 08:02 100685649                  /cm/local/apps/gcc/10.2.0/lib64/libstdc++.so.6.0.28&#xA;7ffff7794000-7ffff7993000 ---p 001c3000 08:02 100685649                  /cm/local/apps/gcc/10.2.0/lib64/libstdc++.so.6.0.28&#xA;7ffff7993000-7ffff799e000 r--p 001c2000 08:02 100685649                  /cm/local/apps/gcc/10.2.0/lib64/libstdc++.so.6.0.28&#xA;7ffff799e000-7ffff79a1000 rw-p 001cd000 08:02 100685649                  /cm/local/apps/gcc/10.2.0/lib64/libstdc++.so.6.0.28&#xA;7ffff79a1000-7ffff79a4000 rw-p 00000000 00:00 0 &#xA;7ffff79a4000-7ffff79ca000 r-xp 00000000 00:89 7817491573                 /share/sciswrepo/spack/latest-2024-01-17/linux-centos8-skylake_avx512/gcc-8.4.1/zlib-ng-2.0.0-qvhoazdgq5hmo6s32psbvmz75vofgtbt/lib/libz.so.1.2.11.zlib-ng&#xA;7ffff79ca000-7ffff7bc9000 ---p 00026000 00:89 7817491573                 /share/sciswrepo/spack/latest-2024-01-17/linux-centos8-skylake_avx512/gcc-8.4.1/zlib-ng-2.0.0-qvhoazdgq5hmo6s32psbvmz75vofgtbt/lib/libz.so.1.2.11.zlib-ng&#xA;7ffff7bc9000-7ffff7bca000 r--p 00025000 00:89 7817491573                 /share/sciswrepo/spack/latest-2024-01-17/linux-centos8-skylake_avx512/gcc-8.4.1/zlib-ng-2.0.0-qvhoazdgq5hmo6s32psbvmz75vofgtbt/lib/libz.so.1.2.11.zlib-ng&#xA;7ffff7bca000-7ffff7bcb000 rw-p 00026000 00:89 7817491573                 /share/sciswrepo/spack/latest-2024-01-17/linux-centos8-skylake_avx512/gcc-8.4.1/zlib-ng-2.0.0-qvhoazdgq5hmo6s32psbvmz75vofgtbt/lib/libz.so.1.2.11.zlib-ng&#xA;7ffff7bcb000-7ffff7bcf000 r-xp 00000000 08:02 36760911                   /usr/lib64/libSegFault.so&#xA;7ffff7bcf000-7ffff7dce000 ---p 00004000 08:02 36760911                   /usr/lib64/libSegFault.so&#xA;7ffff7dce000-7ffff7dcf000 r--p 00003000 08:02 36760911                   /usr/lib64/libSegFault.so&#xA;7ffff7dcf000-7ffff7dd0000 rw-p 00004000 08:02 36760911                   /usr/lib64/libSegFault.so&#xA;7ffff7dd0000-7ffff7dfc000 r-xp 00000000 08:02 36760102                   /usr/lib64/ld-2.28.so&#xA;7ffff7fd7000-7ffff7fdd000 rw-p 00000000 00:00 0 &#xA;7ffff7fef000-7ffff7ff6000 rw-p 00000000 00:00 0 &#xA;7ffff7ff6000-7ffff7ffa000 r--p 00000000 00:00 0                          [vvar]&#xA;7ffff7ffa000-7ffff7ffc000 r-xp 00000000 00:00 0                          [vdso]&#xA;7ffff7ffc000-7ffff7ffd000 r--p 0002c000 08:02 36760102                   /usr/lib64/ld-2.28.so&#xA;7ffff7ffd000-7ffff7fff000 rw-p 0002d000 08:02 36760102                   /usr/lib64/ld-2.28.so&#xA;7ffffffcd000-7ffffffff000 rw-p 00000000 00:00 0                          [stack]&#xA;ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]&#xA;/cm/local/apps/slurm/var/spool/job8889241/slurm_script: line 18: 3742255 Aborted                 (core dumped)&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18988" LastEditorUserId="73" LastEditDate="2024-02-22T09:09:32.967" LastActivityDate="2024-02-22T09:09:32.967" Title="Assertion `compressBound(0xff00) &lt; 0x10000' failed when using STAR 2.7.9a" Tags="|rna-alignment|star|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22201" PostTypeId="1" CreationDate="2024-02-21T13:54:14.167" Score="0" ViewCount="23" Body="&lt;p&gt;I am comparing Illumina MiSeq (using MiSeq v3 kit) and MiniSeq (MiniSeq High Output kit) instruments for sequencing E.coli and Salmonella. I am curious if there is any way to bioinformatically distinguish between the sequences generated by these instruments without the header information? Would it simply be the read length difference (MiSeq=2 x 300bp vs MiniSeq=2 x 150 bp) and output (MiSeq=13.2-15Gb vs MiniSeq=6.6-7.5Gb). Quality wise they look nearly identical in my experience.&#xA;Thank you.&lt;/p&gt;&#xA;" OwnerUserId="18991" LastActivityDate="2024-02-21T13:54:14.167" Title="Aside from the header info, is there any way to distinguish between sequences generated by MiSeq and MiniSeq platforms?" Tags="|fastq|illumina|bacteria|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22202" PostTypeId="1" CreationDate="2024-02-21T18:34:42.493" Score="1" ViewCount="57" Body="&lt;p&gt;I did &lt;code&gt;hmmsearch&lt;/code&gt; with &lt;code&gt;--notextw&lt;/code&gt; and &lt;code&gt;--domtblout&lt;/code&gt; and I see several results for ids, I want to get the best result for each id. How I can parse it?&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/MK8KU.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/MK8KU.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I saw the tools in Biopython SearchIO but I really can't understand how I can do it. Please, can you explain it in detail?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;This is part of my file:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;NODE_1_length_959108_cov_67.987603_423 -            531 2-Hacid_dh           PF00389.34   134   1.6e-35  120.1   0.4   1   1   1.1e-38   3.3e-35  119.2   0.4     1   134     9   316     9   316 0.99 # 448882 # 450474 # -1 # ID=1_423;partial=00;start_type=GTG;rbs_motif=GGAG/GAGG;rbs_spacer=5-10bp;gc_cont=0.664&#xA;NODE_7_length_349797_cov_70.130335_175 -            319 2-Hacid_dh           PF00389.34   134   1.3e-13   49.2   0.0   1   1   5.1e-17   1.5e-13   49.0   0.0    21   129    28   314     6   318 0.94 # 187752 # 188708 # -1 # ID=7_175;partial=00;start_type=GTG;rbs_motif=AGGAG;rbs_spacer=5-10bp;gc_cont=0.637&#xA;NODE_1_length_959108_cov_67.987603_423 -            531 2-Hacid_dh_C         PF02826.23   178   6.6e-63  209.7   0.0   1   2   1.3e-65   1.9e-62  208.2   0.0     1   178   110   284   110   284 0.98 # 448882 # 450474 # -1 # ID=1_423;partial=00;start_type=GTG;rbs_motif=GGAG/GAGG;rbs_spacer=5-10bp;gc_cont=0.664&#xA;NODE_1_length_959108_cov_67.987603_423 -            531 2-Hacid_dh_C         PF02826.23   178   6.6e-63  209.7   0.0   2   2       1.1   1.6e+03   -3.3   0.0     6    44   331   369   328   371 0.66 # 448882 # 450474 # -1 # ID=1_423;partial=00;start_type=GTG;rbs_motif=GGAG/GAGG;rbs_spacer=5-10bp;gc_cont=0.664&#xA;NODE_7_length_349797_cov_70.130335_175 -            319 2-Hacid_dh_C         PF02826.23   178   7.2e-51  170.5   0.0   1   1   6.8e-54     1e-50  170.0   0.0     7   178   118   287   112   287 0.94 # 187752 # 188708 # -1 # ID=7_175;partial=00;start_type=GTG;rbs_motif=AGGAG;rbs_spacer=5-10bp;gc_cont=0.637&#xA;NODE_7_length_349797_cov_70.130335_232 -            309 2-Hacid_dh_C         PF02826.23   178   2.2e-49  165.7   0.0   1   1   1.9e-52   2.9e-49  165.3   0.0     4   178    97   271    95   271 0.92 # 243113 # 244039 # -1 # ID=7_232;partial=00;start_type=ATG;rbs_motif=None;rbs_spacer=None;gc_cont=0.653&#xA;NODE_9_length_318862_cov_66.931761_127 -            311 2-Hacid_dh_C         PF02826.23   178   3.8e-48  161.7   0.0   1   1   3.3e-51   4.9e-48  161.3   0.0     2   176   100   271    98   273 0.89 # 151217 # 152149 # -1 # ID=9_127;partial=00;start_type=ATG;rbs_motif=None;rbs_spacer=None;gc_cont=0.652&#xA;NODE_2_length_853960_cov_68.957247_395 -            506 2-oxoacid_dh         PF00198.27   232   1.7e-82  274.8   0.0   1   1   2.2e-85   2.3e-82  274.4   0.0     2   231   277   504   276   505 0.98 # 406642 # 408159 # 1 # ID=2_395;partial=00;start_type=ATG;rbs_motif=GGA/GAG/AGG;rbs_spacer=5-10bp;gc_cont=0.644&#xA;NODE_1_length_959108_cov_67.987603_423 -            531 PGDH_inter           PF19304.3    119   9.9e-19   66.1   1.5   1   3      0.76   4.6e+03   -3.8   0.0    68    81    87   100    86   123 0.54 # 448882 # 450474 # -1 # ID=1_423;partial=00;start_type=GTG;rbs_motif=GGAG/GAGG;rbs_spacer=5-10bp;gc_cont=0.664&#xA;NODE_1_length_959108_cov_67.987603_423 -            531 PGDH_inter           PF19304.3    119   9.9e-19   66.1   1.5   2   3       0.4   2.4e+03   -2.9   0.0    54    72   208   226   199   249 0.73 # 448882 # 450474 # -1 # ID=1_423;partial=00;start_type=GTG;rbs_motif=GGAG/GAGG;rbs_spacer=5-10bp;gc_cont=0.664&#xA;NODE_1_length_959108_cov_67.987603_423 -            531 PGDH_inter           PF19304.3    119   9.9e-19   66.1   1.5   3   3   2.6e-22   1.6e-18   65.4   0.4     2   119   326   446   325   446 0.91 # 448882 # 450474 # -1 # ID=1_423;partial=00;start_type=GTG;rbs_motif=GGAG/GAGG;rbs_spacer=5-10bp;gc_cont=0.664&#xA;NODE_2_length_853960_cov_68.957247_550 -            554 PGI                  PF00342.23   486  8.5e-202  669.5   0.0   1   1  1.6e-205  9.7e-202  669.3   0.0     1   486    52   543    52   543 0.97 # 568442 # 570103 # -1 # ID=2_550;partial=00;start_type=ATG;rbs_motif=None;rbs_spacer=None;gc_cont=0.626&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can see &lt;code&gt;NODE_1_length_959108_cov_67.987603_423&lt;/code&gt; several times with different annotations (&lt;code&gt;2-Hacid_dh&lt;/code&gt;, &lt;code&gt;PF00389.34&lt;/code&gt; and &lt;code&gt;2-Hacid_dh_C&lt;/code&gt;, &lt;code&gt;PF02826.23&lt;/code&gt; and &lt;code&gt;PGDH_inter&lt;/code&gt;, &lt;code&gt;PF19304.3&lt;/code&gt;).&lt;/p&gt;&#xA;&lt;p&gt;I need to get only best result, only ONE result. I need to see only one row with this id (&lt;code&gt;NODE_1_length_959108_cov_67.987603_423&lt;/code&gt;) with  only one annotation.&lt;/p&gt;&#xA;&lt;p&gt;I assumed that there were some programs with certain algorithms that could select one thing. For example, as in blastp there is --max_hsps 1. I need to make a pivot table for each ID. I did a search in the Swissprot database, in the Pataa database, and also in Pfam. But there are several search results in pfam and I don’t know how to put it in the pivot table&lt;/p&gt;&#xA;" OwnerUserId="18992" LastEditorUserId="73" LastEditDate="2024-02-22T09:06:20.917" LastActivityDate="2024-02-22T09:06:20.917" Title="How to parse hmmsearch output?" Tags="|linux|biopython|functional-annotation|parsing|hmmer|" AnswerCount="2" CommentCount="6" ContentLicense="CC BY-SA 4.0" />
  <row Id="22203" PostTypeId="2" ParentId="22202" CreationDate="2024-02-21T19:03:08.343" Score="2" Body="&lt;p&gt;The TSV output should be sorted by default in a way that for a given query the top hit comes first, followed by other hits. For a quick extraction of top hits you may use a rather simplistic script:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/Cancer-Genomics-TH/tRNA_methyl/blob/main/src/parse_top_hits_cmscan.py&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/Cancer-Genomics-TH/tRNA_methyl/blob/main/src/parse_top_hits_cmscan.py&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="6314" LastEditorUserId="650" LastEditDate="2024-02-21T19:13:44.973" LastActivityDate="2024-02-21T19:13:44.973" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22204" PostTypeId="2" ParentId="22202" CreationDate="2024-02-21T21:25:07.450" Score="1" Body="&lt;p&gt;Since we don't know what you are trying to maximize here, which of the various columns would define the &amp;quot;best&amp;quot; hit for what you are trying to do, here is a generic solution that will just find the highest score:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;$ awk '( ! best[$1] ||  $6 &amp;gt; best[$1] ){ best[$1]=$6; line[$1]=$0 } END{ for(name in best){ print line[name] }}' file &#xA;NODE_1_length_959108_cov_67.987603_423 -            531 2-Hacid_dh_C         PF02826.23   178   6.6e-63  209.7   0.0   1   2   1.3e-65   1.9e-62  208.2   0.0     1   178   110   284   110   284 0.98 # 448882 # 450474 # -1 # ID=1_423;partial=00;start_type=GTG;rbs_motif=GGAG/GAGG;rbs_spacer=5-10bp;gc_cont=0.664&#xA;NODE_7_length_349797_cov_70.130335_232 -            309 2-Hacid_dh_C         PF02826.23   178   2.2e-49  165.7   0.0   1   1   1.9e-52   2.9e-49  165.3   0.0     4   178    97   271    95   271 0.92 # 243113 # 244039 # -1 # ID=7_232;partial=00;start_type=ATG;rbs_motif=None;rbs_spacer=None;gc_cont=0.653&#xA;NODE_7_length_349797_cov_70.130335_175 -            319 2-Hacid_dh_C         PF02826.23   178   7.2e-51  170.5   0.0   1   1   6.8e-54     1e-50  170.0   0.0     7   178   118   287   112   287 0.94 # 187752 # 188708 # -1 # ID=7_175;partial=00;start_type=GTG;rbs_motif=AGGAG;rbs_spacer=5-10bp;gc_cont=0.637&#xA;NODE_9_length_318862_cov_66.931761_127 -            311 2-Hacid_dh_C         PF02826.23   178   3.8e-48  161.7   0.0   1   1   3.3e-51   4.9e-48  161.3   0.0     2   176   100   271    98   273 0.89 # 151217 # 152149 # -1 # ID=9_127;partial=00;start_type=ATG;rbs_motif=None;rbs_spacer=None;gc_cont=0.652&#xA;NODE_2_length_853960_cov_68.957247_395 -            506 2-oxoacid_dh         PF00198.27   232   1.7e-82  274.8   0.0   1   1   2.2e-85   2.3e-82  274.4   0.0     2   231   277   504   276   505 0.98 # 406642 # 408159 # 1 # ID=2_395;partial=00;start_type=ATG;rbs_motif=GGA/GAG/AGG;rbs_spacer=5-10bp;gc_cont=0.644&#xA;NODE_2_length_853960_cov_68.957247_550 -            554 PGI                  PF00342.23   486  8.5e-202  669.5   0.0   1   1  1.6e-205  9.7e-202  669.3   0.0     1   486    52   543    52   543 0.97 # 568442 # 570103 # -1 # ID=2_550;partial=00;start_type=ATG;rbs_motif=None;rbs_spacer=None;gc_cont=0.626&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;To get the maximum value of a different field, just change &lt;code&gt;$6&lt;/code&gt; to the corresponding field number.&lt;/p&gt;&#xA;&lt;p&gt;And here's a version that looks for the smallest value, in this case using the E-value (field 5):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-none prettyprint-override&quot;&gt;&lt;code&gt;$ awk '( ! best[$1] ||  $6 &amp;lt; best[$1] ){ best[$1]=$6; line[$1]=$0 } END{ for(name in best){ print line[name] }}' file &#xA;NODE_1_length_959108_cov_67.987603_423 -            531 PGDH_inter           PF19304.3    119   9.9e-19   66.1   1.5   1   3      0.76   4.6e+03   -3.8   0.0    68    81    87   100    86   123 0.54 # 448882 # 450474 # -1 # ID=1_423;partial=00;start_type=GTG;rbs_motif=GGAG/GAGG;rbs_spacer=5-10bp;gc_cont=0.664&#xA;NODE_7_length_349797_cov_70.130335_232 -            309 2-Hacid_dh_C         PF02826.23   178   2.2e-49  165.7   0.0   1   1   1.9e-52   2.9e-49  165.3   0.0     4   178    97   271    95   271 0.92 # 243113 # 244039 # -1 # ID=7_232;partial=00;start_type=ATG;rbs_motif=None;rbs_spacer=None;gc_cont=0.653&#xA;NODE_7_length_349797_cov_70.130335_175 -            319 2-Hacid_dh           PF00389.34   134   1.3e-13   49.2   0.0   1   1   5.1e-17   1.5e-13   49.0   0.0    21   129    28   314     6   318 0.94 # 187752 # 188708 # -1 # ID=7_175;partial=00;start_type=GTG;rbs_motif=AGGAG;rbs_spacer=5-10bp;gc_cont=0.637&#xA;NODE_9_length_318862_cov_66.931761_127 -            311 2-Hacid_dh_C         PF02826.23   178   3.8e-48  161.7   0.0   1   1   3.3e-51   4.9e-48  161.3   0.0     2   176   100   271    98   273 0.89 # 151217 # 152149 # -1 # ID=9_127;partial=00;start_type=ATG;rbs_motif=None;rbs_spacer=None;gc_cont=0.652&#xA;NODE_2_length_853960_cov_68.957247_395 -            506 2-oxoacid_dh         PF00198.27   232   1.7e-82  274.8   0.0   1   1   2.2e-85   2.3e-82  274.4   0.0     2   231   277   504   276   505 0.98 # 406642 # 408159 # 1 # ID=2_395;partial=00;start_type=ATG;rbs_motif=GGA/GAG/AGG;rbs_spacer=5-10bp;gc_cont=0.644&#xA;NODE_2_length_853960_cov_68.957247_550 -            554 PGI                  PF00342.23   486  8.5e-202  669.5   0.0   1   1  1.6e-205  9.7e-202  669.3   0.0     1   486    52   543    52   543 0.97 # 568442 # 570103 # -1 # ID=2_550;partial=00;start_type=ATG;rbs_motif=None;rbs_spacer=None;gc_cont=0.626&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And here is the same code, expanded, with an explanation (this will still execute correctly if you copy/paste it into a terminal):&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-shellsession prettyprint-override&quot;&gt;&lt;code&gt;awk '&#xA;  ## If there's no value for the first field in the 'best' array or, if the&#xA;  ## existing value is lower than the 6th field, then save the $6th  field &#xA;  ## as the new value for this name in the 'best' array, and save the current &#xA;  ## line as the value in the 'line' array.&#xA;  ( ! best[$1] ||  $6 &amp;lt; best[$1] ){ &#xA;     best[$1]=$6 &#xA;     line[$1]=$0&#xA;  } &#xA;  ## When we reach the end of the file, iterate over the 'line'&#xA;  ## array, and print each element.&#xA;  END{ &#xA;    for(name in best){ &#xA;      print line[name]&#xA;    }&#xA;   }' file &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastActivityDate="2024-02-21T21:25:07.450" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22206" PostTypeId="2" ParentId="22200" CreationDate="2024-02-22T01:36:46.323" Score="3" Body="&lt;pre&gt;&lt;code&gt;bgzf.c:158: bgzf_open: Assertion `compressBound(0xff00) &amp;lt; 0x10000' failed.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Upstream &lt;a href=&quot;https://github.com/samtools/htslib/issues/1257&quot; rel=&quot;nofollow noreferrer&quot;&gt;HTSlib users encountered this problem&lt;/a&gt; when HTSlib was used in conjunction with &lt;strong&gt;zlib-ng&lt;/strong&gt;, which has slightly worse worst-case compression behaviour than the original zlib.&lt;/p&gt;&#xA;&lt;p&gt;Is zlib-ng installed on your machine, and is STAR using it rather than vanilla zlib? (If so, presumably when these alignments completed successfully in the past, that was before zlib-ng was installed.)&lt;/p&gt;&#xA;&lt;p&gt;Whether the problem is due to zlib-ng or you have managed in some other way to have a slightly too-big compressed block for some particular input data, the same zlib-ng-supporting fix will resolve the issue because it removed these assertions and added code to handle any too-big blocks encountered.&lt;/p&gt;&#xA;&lt;p&gt;This problem was &lt;a href=&quot;https://github.com/samtools/htslib/pull/1258&quot; rel=&quot;nofollow noreferrer&quot;&gt;fixed in HTSlib three years ago&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Unfortunately STAR bundles a pre-historic copy of HTSlib that predates this fix. You have also reported this as a &lt;a href=&quot;https://github.com/alexdobin/STAR/issues/2063&quot; rel=&quot;nofollow noreferrer&quot;&gt;STAR issue&lt;/a&gt;, so hopefully the authors will address it in due course.&lt;/p&gt;&#xA;&lt;p&gt;In the meantime, you could for example use bioconda's STAR package, which is built to use bioconda's HTSlib package, which is up-to-date and does not exhibit this assertion failure. Or, if you are aware of having installed zlib-ng about two days ago, you could uninstall it.&lt;/p&gt;&#xA;" OwnerUserId="134" LastEditorUserId="134" LastEditDate="2024-02-22T01:44:17.220" LastActivityDate="2024-02-22T01:44:17.220" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22207" PostTypeId="1" CreationDate="2024-02-22T12:49:38.300" Score="2" ViewCount="29" Body="&lt;p&gt;Cheng and Church Algorithm is a biclustering algorithm or an unsupervised learning task with which both rows and columns in a data matrix can be placed in concurrent clusters.&lt;/p&gt;&#xA;&lt;p&gt;In the &lt;a href=&quot;https://www.researchgate.net/publication/12345321_Biclustering_of_Expression_Data&quot; rel=&quot;nofollow noreferrer&quot;&gt;original paper&lt;/a&gt;, authors explained that a row or column of an input matrix &lt;span class=&quot;math-container&quot;&gt;$A$&lt;/span&gt; with the largest &lt;span class=&quot;math-container&quot;&gt;$d(i)$&lt;/span&gt; or &lt;span class=&quot;math-container&quot;&gt;$d(j)$&lt;/span&gt; should be removed in single node deletion phase:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Find the row &lt;span class=&quot;math-container&quot;&gt;$i \in I$&lt;/span&gt; with largest: &lt;span class=&quot;math-container&quot;&gt;$d(i) = \frac{\sum_{j \in J} (a_{ij} - a_{iJ} - a_{Ij} + a_{IJ})}{\lvert J \rvert}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;li&gt;Find the column &lt;span class=&quot;math-container&quot;&gt;$j \in J$&lt;/span&gt; with largest: &lt;span class=&quot;math-container&quot;&gt;$d(j) = \frac{\sum{i \in I} (a_{ij} - a_{iJ} - a_{Ij} + a_{IJ})}{\lvert I \rvert}$&lt;/span&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I found one implementation of the algorithm trying to do the following operations for single node deletion:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;row_max_msr = np.argmax(row_msr) #finding the index of rows having largest d(i)&#xA;col_max_msr = np.argmax(col_msr) #finding the index of columns having largest d(j)&#xA;&#xA;#Continue with removing row where &lt;span class=&quot;math-container&quot;&gt;$d(i) &amp;gt;= d(j)$&lt;/span&gt; or column &lt;span class=&quot;math-container&quot;&gt;$d(j) &amp;gt; d(i)$&lt;/span&gt; &#xA;if row_msr[row_max_msr] &amp;gt;= col_msr[col_max_msr]: &#xA;    row2remove = row_indices[row_max_msr]&#xA;    rows[row2remove] = False&#xA;else:&#xA;    col2remove = col_indices[col_max_msr]&#xA;    cols[col2remove] = False&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I was wondering whether anyone has a suggestion for this matter. I don't mean that the code has installation or execution problem. Actually, I want to be sure of the logic to be aligned with the algorithm.&lt;/p&gt;&#xA;" OwnerUserId="17940" LastEditorUserId="73" LastEditDate="2024-02-23T10:28:38.903" LastActivityDate="2024-03-24T11:07:50.893" Title="Single Node Deletion in Cheng and Church Algorithm" Tags="|python|machine-learning|algorithms|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22208" PostTypeId="2" ParentId="19657" CreationDate="2024-02-22T12:59:54.740" Score="1" Body="&lt;p&gt;The missing values might be an issue indeed. You might want to use imputation methods, e.g.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from sklearn.impute import KNNImputer&#xA;from sklearn.experimental import enable_iterative_imputer&#xA;from sklearn.impute import IterativeImputer&#xA;&#xA;def impute_df(df, feature_columns_to_apply, strategy):&#xA;    if strategy == 'mean':&#xA;        for col in feature_columns_to_apply:    &#xA;            df.loc[:, col] = df.loc[:, col].fillna(df[col].mean())&#xA;    elif strategy == 'median':&#xA;        for col in feature_columns_to_apply:    &#xA;            df.loc[:, col] = df.loc[:, col].fillna(df[col].median())&#xA;    elif strategy == 'knn':&#xA;        knn_imputer = KNNImputer(n_neighbors=5)&#xA;        df_tmp = pd.DataFrame(knn_imputer.fit_transform(df[feature_columns_to_apply]), columns=feature_columns_to_apply)&#xA;        df.loc[:, feature_columns_to_apply] = df_tmp[feature_columns_to_apply]&#xA;    elif strategy == 'iterative':&#xA;        iterative_imputer = IterativeImputer()&#xA;        df_tmp = pd.DataFrame(iterative_imputer.fit_transform(df[feature_columns_to_apply]), columns=feature_columns_to_apply)&#xA;        df.loc[:, feature_columns_to_apply] = df_tmp[feature_columns_to_apply]&#xA;    return df&#xA;    &#xA;df_all = impute_df(df_all, feature_columns_to_apply, 'mean')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;(IterativeImputer is heavy and slow)&#xA;These are just sample strategies, you might want to use other methods, see e.g. &lt;a href=&quot;https://www.biorxiv.org/content/biorxiv/early/2023/04/08/2023.04.07.535980.full.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;.&lt;br /&gt;&#xA;Also consider other models such as random forest / XGBoost, they outperform linear regression in many cases.&lt;/p&gt;&#xA;" OwnerUserId="17705" LastEditorUserId="17705" LastEditDate="2024-03-17T13:39:17.793" LastActivityDate="2024-03-17T13:39:17.793" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22209" PostTypeId="2" ParentId="22207" CreationDate="2024-02-22T16:21:10.640" Score="0" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;Padilha, V. A. &amp;amp; Campello, R. J. G. B. (2017). A systematic comparative evaluation of biclustering techniques. BMC Bioinformatics, 18(1):55.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/padilha/biclustlib&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/padilha/biclustlib&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;It needs the r-bicluster library which is available on conda &lt;a href=&quot;https://anaconda.org/conda-forge/r-biclust&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://anaconda.org/conda-forge/r-biclust&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Citing the authors GitHub page for instructions&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import numpy as np&#xA;&#xA;from biclustlib.algorithms import ChengChurchAlgorithm&#xA;from biclustlib.datasets import load_yeast_tavazoie&#xA;&#xA;# load yeast data used in the original Cheng and Church's paper&#xA;data = load_yeast_tavazoie().values&#xA;&#xA;# missing value imputation suggested by Cheng and Church&#xA;missing = np.where(data &amp;lt; 0.0)&#xA;data[missing] = np.random.randint(low=0, high=800, size=len(missing[0]))&#xA;&#xA;# creating an instance of the ChengChurchAlgorithm class and running with the parameters of the original study&#xA;cca = ChengChurchAlgorithm(num_biclusters=100, msr_threshold=300.0, multiple_node_deletion_threshold=1.2)&#xA;biclustering = cca.run(data)&#xA;print(biclustering)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="650" LastEditDate="2024-02-22T20:09:59.087" LastActivityDate="2024-02-22T20:09:59.087" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22210" PostTypeId="1" AcceptedAnswerId="22241" CreationDate="2024-02-23T05:41:56.153" Score="2" ViewCount="42" Body="&lt;p&gt;This is a continuation of the following question -&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/22104/fishers-exact-test-for-cog-categories-in-pan-core-genome-analysis/22108&quot;&gt;Fisher&amp;#39;s exact test for COG categories in pan, core genome analysis&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Also, the dataset -&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/7dJra.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7dJra.png&quot; alt=&quot;COG categories dataset&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;How do I perform a Fisher's exact test on this data to show that a COG category is significantly more or less in the pan/core/accessory/species specific genomes?&lt;/p&gt;&#xA;" OwnerUserId="17392" LastEditorUserId="73" LastEditDate="2024-02-23T10:23:34.477" LastActivityDate="2024-03-05T03:26:13.437" Title="Calculating Fisher's exact test for COG categories" Tags="|statistics|eggnog|pangenome|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22211" PostTypeId="1" CreationDate="2024-02-23T10:43:55.400" Score="0" ViewCount="111" Body="&lt;p&gt;Please somebody debug my code&lt;/p&gt;&#xA;&lt;p&gt;In sub-folders there are &lt;code&gt;500000 vcf files&lt;/code&gt; from which I only need &lt;code&gt;500 vcf files&lt;/code&gt; listed in &lt;code&gt;temp.txt&lt;/code&gt; which randomly distributed in sub-folders 10 to 60. The code reads each of 500000 vcf files one by one, if a file was in temp.txt, the code runs&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dx run swiss-army-knife -icmd=&amp;quot;tabix -h &lt;span class=&quot;math-container&quot;&gt;$vcf -R $&lt;/span&gt;REGIONS_FILE &amp;gt; &lt;span class=&quot;math-container&quot;&gt;$output_file&quot; -iin=&quot;$&lt;/span&gt;filename_root&amp;quot; -iin=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$vcf.*&quot; -iin=&quot;/Bulk/DRAGEN\ WGS/Whole\ genome\ variant\ call\ files\ (VCFs)\ (DRAGEN)\ [500k release]/file.bed&quot; --destination &quot;$&lt;/span&gt;OUTPUT_FOLDER&amp;quot; --detach --priority high --yes&#xA;    done &amp;lt; &amp;quot;$VCF_LIST_FILE&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;on that if not, the code goes to next file such a process takes years. I want this code only locates files listed in temp.txt and run the main code on them by iteration.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#!/bin/bash&#xA;&#xA;# Set the working directory to the desired location within DNAnexus&#xA;&#xA;# Number of concurrent processes&#xA;N=4&#xA;&#xA;# Iterate from 10 to 60&#xA;for folder_number in {10..60}; do&#xA;  (&#xA;    # Define the input files and parameters for this iteration&#xA;    folder_path=&amp;quot;/Bulk/DRAGEN\ WGS/Whole\ genome\ variant\ call\ files\ (VCFs)\ (DRAGEN)\ [500k release]/$folder_number&amp;quot;&#xA;    VCF_LIST_FILE=&amp;quot;temp.txt&amp;quot;&#xA;    REGIONS_FILE='file.bed'&#xA;    OUTPUT_FOLDER='/out'  # Replace with your desired output folder&#xA;&#xA;    # Set the working directory to the folder for this iteration&#xA;    dx cd &amp;quot;$folder_path&amp;quot;&#xA;&#xA;    # Read each VCF file from the list&#xA;    while read -r vcf; do&#xA;      filename_root=&lt;span class=&quot;math-container&quot;&gt;$(basename &quot;$&lt;/span&gt;vcf&amp;quot;)&#xA;      output_file=&amp;quot;${filename_root}_extracted.vcf.gz&amp;quot;&#xA;&#xA;      # Use dx run swiss-army-knife to execute tabix on the VCF file&#xA;      dx run swiss-army-knife -icmd=&amp;quot;tabix -h &lt;span class=&quot;math-container&quot;&gt;$vcf -R $&lt;/span&gt;REGIONS_FILE &amp;gt; &lt;span class=&quot;math-container&quot;&gt;$output_file&quot; -iin=&quot;$&lt;/span&gt;filename_root&amp;quot; -iin=&amp;quot;&lt;span class=&quot;math-container&quot;&gt;$vcf.*&quot; -iin=&quot;/Bulk/DRAGEN\ WGS/Whole\ genome\ variant\ call\ files\ (VCFs)\ (DRAGEN)\ [500k release]/file.bed&quot; --destination &quot;$&lt;/span&gt;OUTPUT_FOLDER&amp;quot; --detach --priority high --yes&#xA;    done &amp;lt; &amp;quot;$VCF_LIST_FILE&amp;quot;&#xA;  ) &amp;amp;&#xA;&#xA;  # Limit the number of concurrent processes&#xA;  ((i=i%N)); ((i++==0)) &amp;amp;&amp;amp; wait&#xA;done&#xA;&#xA;# Wait for all background jobs to finish&#xA;wait&#xA;&#xA;# Finished message&#xA;echo &amp;quot;++ Finished extracting genomic coordinates from VCF files&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is how temp.txt looks. The first two digits of vcf file names defines which sub-folder they are located. For instance, the below vcf files distributed in sub-folders 32, 33 and 34&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head temp_new.txt&#xA;head: temp_new.txt: No such file or directory&#xA;izadia@LV19Y7325V Downloads % head new_temp.txt&#xA;11/1195100_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;12/1204363_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;12/1246629_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;13/1311312_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;14/1427973_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;14/1435043_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;15/1558824_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;16/1683316_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;16/1692707_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;17/1749825_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;izadia@LV19Y7325V Downloads % tail new_temp.txt&#xA;52/5230856_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;52/5257892_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;54/5453459_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;55/5514349_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;55/5526010_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;57/5718723_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;58/5868836_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;58/5875882_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;58/5889611_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;59/5917345_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;izadia@LV19Y7325V Downloads % &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After running the script I get this error&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;dxpy.exceptions.DXCLIError: Value provided for input field &amp;quot;in&amp;quot; could not be parsed as file: could not resolve &amp;quot;1195100_24053_0_0.dragen.hard-filtered.vcf.gz&amp;quot; to a name or ID&#xA;dxpy.exceptions.DXCLIError: Value provided for input field &amp;quot;in&amp;quot; could not be parsed as file: could not resolve &amp;quot;1204363_24053_0_0.dragen.hard-filtered.vcf.gz&amp;quot; to a name or ID&#xA;dxpy.exceptions.DXCLIError: Value provided for input field &amp;quot;in&amp;quot; could not be parsed as file: could not resolve &amp;quot;1246629_24053_0_0.dragen.hard-filtered.vcf.gz&amp;quot; to a name or ID&#xA;dxpy.exceptions.DXCLIError: Value provided for input field &amp;quot;in&amp;quot; could not be parsed as file: could not resolve &amp;quot;1311312_24053_0_0.dragen.hard-filtered.vcf.gz&amp;quot; to a name or ID&#xA;dxpy.exceptions.DXCLIError: Value provided for input field &amp;quot;in&amp;quot; could not be parsed as file: could not resolve &amp;quot;1427973_24053_0_0.dragen.hard-filtered.vcf.gz&amp;quot; to a name or ID&#xA;dxpy.exceptions.DXCLIError: Value provided for input field &amp;quot;in&amp;quot; could not be parsed as file: could not resolve &amp;quot;1435043_24053_0_0.dragen.hard-filtered.vcf.gz&amp;quot; to a name or ID&#xA;dxpy.exceptions.DXCLIError: Value provided for input field &amp;quot;in&amp;quot; could not be parsed as file: could not resolve &amp;quot;1558824_24053_0_0.dragen.hard-filtered.vcf.gz&amp;quot; to a name or ID&#xA;dxpy.exceptions.DXCLIError: Value provided for input field &amp;quot;in&amp;quot; could not be parsed as file: could not resolve &amp;quot;1683316_24053_0_0.dragen.hard-filtered.vcf.gz&amp;quot; to a name or ID&#xA;dxpy.exceptions.DXCLIError: Value provided for input field &amp;quot;in&amp;quot; could not be parsed as file: could not resolve &amp;quot;1692707_24053_0_0.dragen.hard-filtered.vcf.gz&amp;quot; to a name or ID&#xA;dxpy.exceptions.DXCLIError: Value provided for input field &amp;quot;in&amp;quot; could not be parsed as file: could not resolve &amp;quot;1749825_24053_0_0.dragen.hard-filtered.vcf.gz&amp;quot; to a name or ID&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;because the codes starts searching from sub-folder 10 for each of files in new_temp.txt one by one&lt;/p&gt;&#xA;" OwnerUserId="4595" LastEditorUserId="4595" LastEditDate="2024-02-23T15:02:04.873" LastActivityDate="2024-02-23T15:02:04.873" Title="How to debug a bash script" Tags="|vcf|bash|linux|wgs|for-loop|" AnswerCount="1" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="22212" PostTypeId="1" CreationDate="2024-02-23T11:10:45.340" Score="2" ViewCount="73" Body="&lt;p&gt;What tools can we use to compare two sets of peptide sequences?&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;The biological question is I have some kmer sequences of some proteins and would like to know how similar are these kmer sequences compared with known MHC classs I and II peptides.&lt;/li&gt;&#xA;&lt;li&gt;Therefore I need to compare two sets of peptides sequences.&lt;/li&gt;&#xA;&lt;li&gt;The kmer and peptide sequences lengths range from 5-80aa.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Alternatively, I am also considering to use the full length of the proteins for comparison.&lt;/p&gt;&#xA;" OwnerUserId="14169" LastEditorUserId="14169" LastEditDate="2024-02-24T02:28:26.280" LastActivityDate="2024-02-27T07:06:29.040" Title="Comparing sequence similarity of two sets of peptide sequences?" Tags="|blastp|peptide|" AnswerCount="3" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22213" PostTypeId="2" ParentId="22211" CreationDate="2024-02-23T12:50:59.920" Score="4" Body="&lt;blockquote&gt;&#xA;&lt;p&gt;The first two digits of vcf file names defines which sub-folder they are located.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;You could use this information to edit temp.txt so it contains the full path:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;grep -v &amp;quot;.tbi$&amp;quot; temp.txt | sed 's/^\(.\)\(.\)/\1\2\/\1\2/' &amp;gt; new_temp.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I'm assuming here that you don't actually want to run &lt;code&gt;dx run swiss-army-knife&lt;/code&gt; on the vcf indexes.&lt;/p&gt;&#xA;&lt;p&gt;This will turn the example of temp.txt you gave into this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;32/3295094_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;33/3309510_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;34/3468518_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;34/3471960_24053_0_0.dragen.hard-filtered.vcf.gz&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can then simply run a single for loop over that, instead of the nested one you are using now.&lt;/p&gt;&#xA;&lt;p&gt;eg.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-bash prettyprint-override&quot;&gt;&lt;code&gt;while read vcf&#xA;  do&#xA;    dx run swiss-army-knife [...]&#xA;  done &amp;lt;  new_temp.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="6955" LastEditorUserId="298" LastEditDate="2024-02-23T14:39:17.773" LastActivityDate="2024-02-23T14:39:17.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22214" PostTypeId="2" ParentId="22212" CreationDate="2024-02-23T13:23:14.907" Score="3" Body="&lt;p&gt;Not sure what is the shortest peptide one can fish out with &lt;code&gt;LAST&lt;/code&gt; out of small protein database (well, it starts with protein FASTA file), but you may check out &lt;a href=&quot;https://gitlab.com/mcfrith/last/-/blob/main/doc/last-cookbook.rst?ref_type=heads&quot; rel=&quot;nofollow noreferrer&quot;&gt;LAST cookbook&lt;/a&gt;. This can use different PAM matrices so you have the proper similar amino acid matching. If that does not suit your needs there can be some more desperate route using string fuzzy matching i.e. with &lt;a href=&quot;https://pypi.org/project/thefuzz/&quot; rel=&quot;nofollow noreferrer&quot;&gt;thefuzz python library&lt;/a&gt; and checking the best Levenshtein distance scores.&lt;/p&gt;&#xA;" OwnerUserId="6314" LastActivityDate="2024-02-23T13:23:14.907" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22215" PostTypeId="2" ParentId="22212" CreationDate="2024-02-23T16:08:58.413" Score="2" Body="&lt;p&gt;I would suggest focusing on short k-mers if you are using very small sequences. Simply quantify the similarity of the k-mer profile of each relevant pair of peptides. There are many tools for analyzing k-mer profiles, though many are targeted towards DNA so you should keep an eye out for that. &lt;a href=&quot;https://buildmedia.readthedocs.org/media/pdf/kmer/rtd/kmer.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here is one python tool&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;It is also &lt;a href=&quot;http://claresloggett.github.io/python_workshops/kmer_counting.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;not difficult to write a k-mer counting script&lt;/a&gt; if you decide that third party tools are not suitable.&lt;/p&gt;&#xA;&lt;p&gt;Granted, you will need to evaluate that k-mer profile similarity has some relevance to your intuitive notion of &amp;quot;similarity&amp;quot;, which it may not. It would clarify somewhat to know more about your desired application.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastActivityDate="2024-02-23T16:08:58.413" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22216" PostTypeId="1" AcceptedAnswerId="22219" CreationDate="2024-02-25T18:43:47.947" Score="3" ViewCount="34" Body="&lt;p&gt;For the two vcf files linked below, I cannot find any variants in the GT column other than &amp;quot;./.&amp;quot;. Is it possible to confirm if the GT column of vcf files have been annotated (i.e variants listed as &amp;quot;./1&amp;quot;, &amp;quot;1/.&amp;quot; or &amp;quot;1/1&amp;quot;)?&lt;/p&gt;&#xA;&lt;p&gt;vcf files: (&lt;a href=&quot;https://zenodo.org/records/6558593&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://zenodo.org/records/6558593&lt;/a&gt;, LUD.TH179.PASS.dbsnp_cosmic.vcf.gz and LUD.TH238.PASS.dbsnp_cosmic.vcf.gz).&lt;/p&gt;&#xA;&lt;p&gt;here are a few terminal commands I have tried. The output for the commands below are all 0, with no variant information under the columns of the outputted vcf data. I would assume, it means that the GT column was not annotated, but I am not sure.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;remove homozygous ref genotype from multi-sample vcf&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;bcftools view -i 'GT[*]=&amp;quot;alt&amp;quot;' LUD.TH179.PASS.dbsnp_cosmic.vcf.gz | less -SN&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;extracting heterozygous snp from a vcf file&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;vcftools --gzvcf LUD.TH179.PASS.dbsnp_cosmic.vcf.gz --extract-FORMAT-info GT | grep &amp;quot;0/1&amp;quot;&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;ol start=&quot;3&quot;&gt;&#xA;&lt;li&gt;ID heterozygous variants in VCF file using vcftools&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;vcftools --gzvcf LUD.TH179.PASS.dbsnp_cosmic.vcf.gz --het&lt;/code&gt;&lt;/li&gt;&#xA;&lt;li&gt;output (cat out.het)&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;code&gt;INDV O(HOM) E(HOM) N_SITES F (EMPTY)&lt;/code&gt;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20009" LastActivityDate="2024-02-26T13:35:41.360" Title="How to extract all variant alleles that do not match &quot;./.&quot; from the GT column of a vcf file?" Tags="|vcf|genomics|bcftools|vcftools|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22218" PostTypeId="1" CreationDate="2024-02-26T10:05:04.167" Score="5" ViewCount="89" Body="&lt;p&gt;I have some RNA-seq data where there are reads from the &amp;quot;host&amp;quot; as well as from several bacteria species. In this experimental context, I am interested in the host associated reads and the latter is regarded as &amp;quot;contaminants&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;I am using &lt;code&gt;BBSplit&lt;/code&gt; to split host- and bacteria-associated reads using my genome of interest as well as a &amp;quot;repo&amp;quot; containing thousands of bacterial genomes that are likely to be present in the host.&lt;/p&gt;&#xA;&lt;p&gt;This repo has multiple genome assemblies for the same species making the &lt;code&gt;BBSplit&lt;/code&gt; step computationally expensive. The total size of the repo is approximatly 25GB, more than 25X the human genome! I am wondering what would be the best criteria to pick one assembly of a particular genome over another assembly.&lt;/p&gt;&#xA;&lt;p&gt;All in all, different genome assemblies of a given species have more or less similar total lengths, %GC, number of contigs meaning that picking any would probably be fine but not being a microbiologist, I would like to hear expert's opinions on this. My current take is to take the assembly with the largest size for each species.&lt;/p&gt;&#xA;" OwnerUserId="3541" LastActivityDate="2024-04-01T15:02:05.277" Title="Multiple genome assemblies of the same bacterial species" Tags="|rna-seq|assembly|microbiology|" AnswerCount="2" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="22219" PostTypeId="2" ParentId="22216" CreationDate="2024-02-26T13:35:41.360" Score="2" Body="&lt;p&gt;I checked one of your files, &lt;code&gt;LUD.TH179.PASS.dbsnp_cosmic.vcf.gz&lt;/code&gt;, and can confirm that none of the samples listed in the VCF have a genotype for any of the variants. This isn't surprising, it is often hard if not impossible to genotype variants from somatic samples since you cannot know if the, for example, 10% allelic balance you see means that this is homozygous in 10% of the cells or heterozygous in 5% or any other combination of the two.&lt;/p&gt;&#xA;&lt;p&gt;In any case, this is how I confirmed that no variants were genotyped. First select all lines that are not headers (do not start with &lt;code&gt;#&lt;/code&gt;), then fields 10 to the end of the line, convert tabs to newline characters (this leaves me with the sample fields, e.g. &lt;code&gt;./.:.:.:.:.:.:.&lt;/code&gt;), then cut at the first &lt;code&gt;:&lt;/code&gt; to get the genotype, sort and count:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ zgrep -v '^#' LUD.TH179.PASS.dbsnp_cosmic.vcf.gz | cut -f 10- | &#xA;   tr '\t' '\n' | cut -d: -f1 | sort | uniq -c&#xA;537123250 ./.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;As you can see above, the result was 537123250 sample/variant combinations, all of whose genotype was &lt;code&gt;./.&lt;/code&gt;.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2024-02-26T13:35:41.360" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22220" PostTypeId="2" ParentId="22218" CreationDate="2024-02-26T22:07:25.110" Score="-1" Body="&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;What I would suggest, without knowing much biology about the system is it's a tree problem. The difficulty is knowing whether the repo is prokaryotic. To solve it I'd take the 16S gene and make a tree - each 16S sequence contains its genome ID. I'd then draw a tree, e.g. IQ-TREE is good and fast. This will describe the overall diversity and provides and informed basis of how to represent each species.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Your approach certainly has merit because again if these are bacteria they go through gene loss - which can cause problems. Selecting the largest genome ensures you minimise this loss.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;What I would do is combine 1 and 2. This is to label the taxa that has the largest genome and then produce the 16S tree. It will then be obvious by manual inspection if your approach is representative of the genetic diversity. If it isn't .... you can do the whole thing manually (could be painful), the other approach is to use a tree reading package/module/library. I use ETE3, you'd use ape (@haci's from memory is a big R codes). The issue is traversing the tree from the 'biggest genome' to the most distant genetically related member of the species. What you do is identify the MRCA (most recent common ancestor) for that species and pick a taxa that is in the opposite clade to the biggest genome and any member will do. Thus this is automating the process via Python or R code so thousands of genomes/species can be screened.&lt;/p&gt;&#xA;&lt;p&gt;To try and be clearer, what you are doing in the code is moving the tree one node at a time and checking if all the members below it are the same species. As soon as the answer is 'no', it's the previous node thats the MRCA for the species. In a bifurcating tree that divides the data into its two biggest groups of genetic diversity.&lt;/p&gt;&#xA;&lt;p&gt;This approach ensures you get a reasonable representation of the species, by selecting two members of it. One is the largest genome.&lt;/p&gt;&#xA;&lt;p&gt;It's a while since I've used ETE3, so I can't remember the methods, there's probably a MRCA method for a given species. You can do this empirically, doing a recursive climb up the tree from the biggest genome until your monophyly contains a difference species. The previous traversal is the MRCA, then you ask it to divide the species into two clades and any member of the clade without the biggest clade is what you want.&lt;/p&gt;&#xA;&lt;p&gt;I agree that it needs a bit of &amp;quot;tree-thinking&amp;quot; to do this ... but it's doable. You then use the genome IDs to filter your repo and produce a greatly slimmed down repo.&lt;/p&gt;&#xA;&lt;p&gt;You could alternative switch to an HPC method (e.g. AWS) using the entire repo&lt;/p&gt;&#xA;&lt;p&gt;The alternative approach is know a lot about the biology/ pathogenesis of the problem.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2024-02-26T22:38:09.807" LastActivityDate="2024-02-26T22:38:09.807" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22221" PostTypeId="2" ParentId="11238" CreationDate="2024-02-27T03:34:25.933" Score="2" Body="&lt;p&gt;Aside from the &lt;code&gt;ggrepel&lt;/code&gt; labels, another reason your volcano plot looks cluttered is the presence of the 4 very highly significant genes on the top right, which lead to the rest of the genes being squeezed into the x axis. For visualization purposes, you could try applying an inverse hyperbolic sine transformation (&lt;code&gt;asinh()&lt;/code&gt; in R) to both your x/y axes, which should alleviate this squeezing while not distorting too much the overall volcano shape. Just be sure to also apply this transformation to any cutoff lines you are including in the plot.&lt;/p&gt;&#xA;" OwnerUserId="16795" LastEditorUserId="16795" LastEditDate="2024-02-27T03:35:04.520" LastActivityDate="2024-02-27T03:35:04.520" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22222" PostTypeId="2" ParentId="22212" CreationDate="2024-02-27T07:06:29.040" Score="1" Body="&lt;p&gt;After more search, I go for &lt;code&gt;blastp-short&lt;/code&gt; with MHC peptides as query and my full-length proteins as database. Please share if you think it has problems.&lt;/p&gt;&#xA;" OwnerUserId="14169" LastActivityDate="2024-02-27T07:06:29.040" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22223" PostTypeId="1" AcceptedAnswerId="22225" CreationDate="2024-02-27T08:04:19.467" Score="4" ViewCount="42" Body="&lt;p&gt;Currently we are building a sequence based deep-learning model to predict binding affinity between antibody and antigen. For this we are training a sequence based model with &lt;strong&gt;AlphaSeq Antibody dataset&lt;/strong&gt; (&lt;a href=&quot;https://github.com/mit-ll/AlphaSeq_Antibody_Dataset&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/mit-ll/AlphaSeq_Antibody_Dataset&lt;/a&gt;). But in this dataset they haven't provided the sequence for SARS-CoV-2 virus.&lt;/p&gt;&#xA;&lt;p&gt;So I read the paper and there they have mentioned they have tested against &lt;strong&gt;peptide in the HR2 region of the SARS-CoV-2 spike protein&lt;/strong&gt; and they have given the sequence as &amp;quot;PDVDLGDISGINAS&amp;quot;. (&lt;a href=&quot;https://www.nature.com/articles/s41597-022-01779-4#Sec3&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41597-022-01779-4#Sec3&lt;/a&gt;)&lt;/p&gt;&#xA;&lt;p&gt;But this sequence is very small. I'd like a larger protein sequence that includes non-epitope portions so that the prediction is more meaningful (e.g. predicts binding to the actual epitope when provided a sequence which includes sequence beyond the epitope). I searched online and only found a covid genome sequence which isn't very helpful.&lt;/p&gt;&#xA;&lt;p&gt;Searching the RCSB PDB website for Spike protein HR2 gave two SARS-CoV-2 results (&lt;a href=&quot;https://www.rcsb.org/search?request=%7B%22query%22%3A%7B%22type%22%3A%22group%22%2C%22nodes%22%3A%5B%7B%22type%22%3A%22group%22%2C%22nodes%22%3A%5B%7B%22type%22%3A%22group%22%2C%22nodes%22%3A%5B%7B%22type%22%3A%22terminal%22%2C%22service%22%3A%22text%22%2C%22parameters%22%3A%7B%22attribute%22%3A%22rcsb_polymer_entity.pdbx_description%22%2C%22operator%22%3A%22exact_match%22%2C%22value%22%3A%22Spike%20protein%20S2%27%20HR2%22%7D%7D%5D%2C%22logical_operator%22%3A%22and%22%7D%5D%2C%22logical_operator%22%3A%22and%22%2C%22label%22%3A%22text%22%7D%5D%2C%22logical_operator%22%3A%22and%22%7D%2C%22return_type%22%3A%22entry%22%2C%22request_options%22%3A%7B%22paginate%22%3A%7B%22start%22%3A0%2C%22rows%22%3A25%7D%2C%22results_content_type%22%3A%5B%22experimental%22%5D%2C%22sort%22%3A%5B%7B%22sort_by%22%3A%22score%22%2C%22direction%22%3A%22desc%22%7D%5D%2C%22scoring_strategy%22%3A%22combined%22%7D%2C%22request_info%22%3A%7B%22query_id%22%3A%2298c7dd3dd558aef8b11a9237f910ac70%22%7D%7D&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;em&gt;RCSB PDB website&lt;/em&gt;&lt;/a&gt;).&lt;/p&gt;&#xA;&lt;p&gt;One of them had sequence like the following:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&amp;gt;8CZI_1|Chains A, B, C|Scaffolded Spike protein S2' HR1|Nostoc punctiforme (strain ATCC 29133 / PCC 73102) (63737)&#xA;MSHHHHHHGSQTLLRNFGNVYDNPVLLDRSVTAPVTEGFNVVLASFQALYLQYQKHHFVVEGSEFYSLHEFFNESYNQVQDHIHEIGERLDGLGGVPVATFSKLAELTCFEQESEGVYSSRQMVENDLAAEQAIIGVIRRQAAQAESLGDRGTRYLYEKILLKTEERAYHLSHFLAKDSLTLGFAYENQKLIANQFNSAIGKIQDSLSSTASALGKLQDVVNQNAQALNTLVKQLSSNFGAISSVLNDILSRLDKVE&#xA;&#xA;&amp;gt;8CZI_2|Chains D, E, F|Spike protein S2' HR2|Severe acute respiratory syndrome coronavirus 2 (2697049)&#xA;KNHTSPDVDLGDISGINASVVNIQKEIDRLNEVAKNLNESLIDLQ&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So I am not sure which one is the correct sequence for our task. Can someone provide some insights regarding this matter?&lt;/p&gt;&#xA;&lt;p&gt;Also why are there so many sequences popping up when I search for SARS-CoV-2? Is there a version that can be considered as the root from which other versions are derived?&lt;/p&gt;&#xA;" OwnerUserId="20014" LastEditorUserId="2108" LastEditDate="2024-02-27T20:07:14.630" LastActivityDate="2024-02-27T22:21:23.420" Title="SARS-CoV-2 sequence used in the AlphaSeq Antibody Datasets to predict binding affinity" Tags="|sequence-analysis|database|covid-19|sars-cov-2|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22224" PostTypeId="1" CreationDate="2024-02-27T18:55:06.510" Score="1" ViewCount="21" Body="&lt;p&gt;We are trying to convert a Phylogenetic tree to networkx previously calculated with&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from Bio.Phylo.TreeConstruction import DistanceMatrix&#xA;from Bio.Phylo.TreeConstruction import DistanceTreeConstructor&#xA;from Bio.Phylo import draw&#xA;&#xA;lower_triangle = []&#xA;&#xA;for i in range(0, len(distance_matrix)):&#xA;    lower_triangle.append(list(distance_matrix[i][:i+1]))&#xA;&#xA;lower_triangle    &#xA;&#xA;# Make a list of strings from '0' to '9'&#xA;labels = list(map(str, range(0, len(lower_triangle))))&#xA;&#xA;distance_matrix = DistanceMatrix(names=labels, matrix=lower_triangle)&#xA;&#xA;constructor = DistanceTreeConstructor()&#xA;upgma_tree = constructor.upgma(distance_matrix)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and convert with&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;from pyvis.network import Network&#xA;from Bio import Phylo&#xA;&#xA;g=Network(height=500, width=500)&#xA;g.toggle_physics(True)&#xA;network_x_graph = Phylo.to_networkx(upgma_tree)&#xA;g.from_nx(network_x_graph)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It fails with&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;assert isinstance(n_id, str) or isinstance(n_id, int)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;here&#xA;&lt;a href=&quot;https://github.com/WestHealth/pyvis/blob/ccb7ce745ee4159ce45eac70b9848ab965fc0906/pyvis/network.py#L236&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/WestHealth/pyvis/blob/ccb7ce745ee4159ce45eac70b9848ab965fc0906/pyvis/network.py#L236&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;so it seems the nodes are of wrong type? How can I check / adjust it?&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;print(upgma_tree)&#xA;&#xA;Tree(rooted=True)&#xA;    Clade(branch_length=0, name='Inner9')&#xA;        Clade(name='2')&#xA;        Clade(name='Inner8')&#xA;            Clade(name='7')&#xA;            Clade(name='Inner7')&#xA;                Clade(name='5')&#xA;                Clade(name='Inner6')&#xA;                    Clade(name='6')&#xA;                    Clade(name='Inner5')&#xA;                        Clade(name='4')&#xA;                        Clade(name='Inner4')&#xA;                            Clade(name='Inner1')&#xA;                                Clade(name='9')&#xA;                                Clade(name='1')&#xA;                            Clade(name='Inner3')&#xA;                                Clade(name='3')&#xA;                                Clade(name='Inner2')&#xA;                                    Clade(name='8')&#xA;                                    Clade(name='0')&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17804" LastEditorUserId="650" LastEditDate="2024-02-27T19:58:33.567" LastActivityDate="2024-02-27T19:58:33.567" Title="Biopython PhyloTree to PyVis Network" Tags="|python|phylogenetics|phylogeny|biopython|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22225" PostTypeId="2" ParentId="22223" CreationDate="2024-02-27T20:06:21.460" Score="1" Body="&lt;p&gt;&lt;a href=&quot;https://pdb101.rcsb.org/motm/246&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDB has a whole page&lt;/a&gt; talking about various slices of the covid spike protein. I'd strongly recommend reading that for a better understanding of why people choose different pieces to represent. In that page they include several different solved structures of the protein that may be helpful for your benchmarking.&lt;/p&gt;&#xA;&lt;p&gt;In terms of the &amp;quot;canonical&amp;quot; sequence, you could look at a reference genome sequence. &lt;a href=&quot;http://ftp.ensemblgenomes.org/pub/viruses/genbank/sars_cov_2/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Here&lt;/a&gt; for example is the EMBL reference genome for covid in GenBank format, which includes the protein sequences. I believe in that the spike protein has ID &lt;code&gt;QHD43416&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;You can additionally look at NCBI, which has &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/labs/virus/vssi/#/virus?VirusLineage_ss=Severe%20acute%20respiratory%20syndrome%20coronavirus%202%20(SARS-CoV-2),%20taxid:2697049&amp;amp;SeqType_s=Nucleotide&quot; rel=&quot;nofollow noreferrer&quot;&gt;a slightly different version&lt;/a&gt; of the reference (though they should be functionally identical, with just slightly different identifiers and annotations).&lt;/p&gt;&#xA;&lt;p&gt;Assuming that you want the ancestral initial isolate of covid, you can find a similar GenBank file &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/nuccore/NC_045512&quot; rel=&quot;nofollow noreferrer&quot;&gt;here from NCBI&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="2108" LastEditorUserId="2108" LastEditDate="2024-02-27T22:21:23.420" LastActivityDate="2024-02-27T22:21:23.420" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22226" PostTypeId="1" CreationDate="2024-02-28T16:25:26.173" Score="2" ViewCount="30" Body="&lt;p&gt;I created a phylogenetic tree of 328 species using RAxML. I run the ML algorithm 20 times and I chose the best tree (the one with the highest likelihood).&lt;/p&gt;&#xA;&lt;p&gt;The choice of 20 iterations was based on a tutorial that was carrying out the same analysis but on fewer species.&lt;/p&gt;&#xA;&lt;p&gt;Now I have a doubt that my search is actually not optimal, and could have got better by increasing the number of iterations.&lt;/p&gt;&#xA;&lt;p&gt;Do you know if there is a way to estimate the optimal number of iterations to find the best robust ML Tree?&lt;/p&gt;&#xA;" OwnerUserId="18351" LastEditorUserId="650" LastEditDate="2024-02-28T19:30:58.720" LastActivityDate="2024-02-29T01:11:36.673" Title="How many ML tree should I build?" Tags="|phylogenetics|phylogeny|taxonomy|dna|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22227" PostTypeId="2" ParentId="22226" CreationDate="2024-02-28T22:06:25.203" Score="2" Body="&lt;p&gt;Simply use the iterations option (e.g. &lt;code&gt;-n 1000&lt;/code&gt;) of IQTREE.&#xA;It's a lot quicker (several logs) and is strongly of identical performance to RAxML.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;iqtree -s alignment.phy -n 1000 -m GTR+I+G -nt 4&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It takes phylip, fasta, nexus  ... and no-one questions 1000 iterations, so you're safe.&lt;/p&gt;&#xA;&lt;p&gt;I personally I'd then go on to bootstrap the tree (IQTREE is extremely fast). If you were bootstrapping with iterations RAxML is unworkable, you'd need e.g. 10 iterations per bootstrap (1000 bootstraps) and would be time consuming. IQTREE could easily perform 10 iterations per bootstrap and would accommodate 100 iterations (1000 would be a bit excessive for bootstrapping).&lt;/p&gt;&#xA;&lt;p&gt;RAxML is an old program now, IQTREE is its successor and massively refactored for maximum likelihood phylogeny.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;If you don't understand the command line, drop a note and I'll explain the model it's using.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2024-02-29T01:11:36.673" LastActivityDate="2024-02-29T01:11:36.673" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22228" PostTypeId="2" ParentId="19163" CreationDate="2024-02-28T23:39:20.580" Score="1" Body="&lt;p&gt;Admixture requires a '12'-coded .ped file, rather than the allele-coding you're using. 'plink --file [file prefix] --recode 12' produced a .ped file that Admixture could use for me.&lt;/p&gt;&#xA;" OwnerUserId="20021" LastActivityDate="2024-02-28T23:39:20.580" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22229" PostTypeId="1" CreationDate="2024-02-29T09:49:57.703" Score="2" ViewCount="36" Body="&lt;p&gt;I was trying to install NCBI AMRFinderPLUS on my Nvidia Jetson using standard Bioconda based installtion. However, all the time it is giving error that the package is not available and I thought I am making some typo or channel is not proper.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;(amrfinder) jay@jetmimd-desktop:~$ conda install -y -c conda-forge -c bioconda ncbi-amrfinderplus&#xA;Channels:&#xA; - conda-forge&#xA; - bioconda&#xA; - defaults&#xA;Platform: linux-aarch64&#xA;Collecting package metadata (repodata.json): done&#xA;Solving environment: failed&#xA;&#xA;PackagesNotFoundError: The following packages are not available from current channels:&#xA;&#xA;  - ncbi-amrfinderplus&#xA;&#xA;Current channels:&#xA;&#xA;  - https://conda.anaconda.org/conda-forge&#xA;  - https://conda.anaconda.org/bioconda&#xA;  - defaults&#xA;&#xA;To search for alternate channels that may provide the conda package you're&#xA;looking for, navigate to&#xA;&#xA;    https://anaconda.org&#xA;&#xA;and use the search bar at the top of the page.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then I remembered about the different architecture. So, now here I am. Is there any way to install this on arm64 architecture?&lt;/p&gt;&#xA;&lt;p&gt;Thank you for the help in advance!!!!&lt;/p&gt;&#xA;" OwnerUserId="18524" LastActivityDate="2024-03-01T14:09:47.703" Title="arm64 installation of AMRFinderPlus" Tags="|software-installation|ncbi|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22230" PostTypeId="2" ParentId="21978" CreationDate="2024-02-29T17:30:56.033" Score="0" Body="&lt;p&gt;I'm a little confused about what you mean by there being no phenotype. To build the score you need a phenotype, although to apply it you might not. (Usually PRS is build with a base dataset and then applied using a target dataset). Here's a good tutorial about doing PRS in plink that might help you:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://choishingwan.github.io/PRS-Tutorial/plink/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://choishingwan.github.io/PRS-Tutorial/plink/&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Also this paper is a really good paper that helped me better understand how building PRS works:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.nature.com/articles/s41596-020-0353-1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nature.com/articles/s41596-020-0353-1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Hope this helps!&lt;/p&gt;&#xA;" OwnerUserId="1574" LastActivityDate="2024-02-29T17:30:56.033" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22231" PostTypeId="2" ParentId="22229" CreationDate="2024-02-29T20:32:27.243" Score="1" Body="&lt;p&gt;Try the following&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;conda install bioconda::ncbi-amrfinderplus&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The above does work I tested it (below). There could be an issue with you version of conda (24.1.2 OSX).&lt;/p&gt;&#xA;&lt;p&gt;Also has the package has been ported for GPU(?) ... &lt;em&gt;I assume it is, otherwise it's obviously not going to work and conda will not install it&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;However, if it ported for GPU and still fails use mamba (I don't use mamba BTW), because it is very slow and that could be where the error is (unusually slow).&lt;/p&gt;&#xA;&lt;p&gt;My connection initially was unsuccessful but resolved via&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Retrying with flexible solve.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;To demonstrate I got it working here's the reply ...&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;The following NEW packages will be INSTALLED:&#xA;&#xA;  blast              bioconda/osx-64::blast-2.12.0-pl5262h78c34c6_0&#xA;  c-ares             pkgs/main/osx-64::c-ares-1.19.1-h6c40b1e_0&#xA;  curl               pkgs/main/osx-64::curl-7.88.1-hdb2fb19_2&#xA;  entrez-direct      bioconda/osx-64::entrez-direct-16.2-h193322a_1&#xA;  hmmer              bioconda/osx-64::hmmer-3.3.2-hb19a6fa_3&#xA;  libcurl            pkgs/main/osx-64::libcurl-7.88.1-ha585b31_2&#xA;  libev              pkgs/main/osx-64::libev-4.33-h9ed2024_1&#xA;  libidn2            pkgs/main/osx-64::libidn2-2.3.4-h6c40b1e_0&#xA;  libnghttp2         pkgs/main/osx-64::libnghttp2-1.52.0-h1c88b7d_1&#xA;  libssh2            pkgs/main/osx-64::libssh2-1.10.0-hdb2fb19_2&#xA;  libunistring       pkgs/main/osx-64::libunistring-0.9.10-h9ed2024_0&#xA;  ncbi-amrfinderplus bioconda/osx-64::ncbi-amrfinderplus-3.11.14-hfee4e81_0&#xA;  perl               pkgs/main/osx-64::perl-5.26.2-h4e221da_0&#xA;  perl-archive-tar   bioconda/osx-64::perl-archive-tar-2.32-pl526_0&#xA;  perl-carp          bioconda/osx-64::perl-carp-1.38-pl526_3&#xA;  perl-common-sense  bioconda/osx-64::perl-common-sense-3.74-pl526_2&#xA;  perl-compress-raw~ bioconda/osx-64::perl-compress-raw-bzip2-2.087-pl526h6de7cb9_0&#xA;  perl-compress-raw~ bioconda/osx-64::perl-compress-raw-zlib-2.087-pl526h770b8ee_0&#xA;  perl-exporter      bioconda/osx-64::perl-exporter-5.72-pl526_1&#xA;  perl-exporter-tiny bioconda/osx-64::perl-exporter-tiny-1.002001-pl526_0&#xA;  perl-extutils-mak~ bioconda/osx-64::perl-extutils-makemaker-7.36-pl526_1&#xA;  perl-io-compress   bioconda/osx-64::perl-io-compress-2.087-pl526h6de7cb9_0&#xA;  perl-io-zlib       bioconda/osx-64::perl-io-zlib-1.10-pl526_2&#xA;  perl-json          bioconda/osx-64::perl-json-4.02-pl526_0&#xA;  perl-json-xs       bioconda/osx-64::perl-json-xs-2.34-pl526h04f5b5a_3&#xA;  perl-list-moreuti~ bioconda/osx-64::perl-list-moreutils-0.428-pl526_1&#xA;  perl-list-moreuti~ bioconda/osx-64::perl-list-moreutils-xs-0.428-pl526_0&#xA;  perl-pathtools     bioconda/osx-64::perl-pathtools-3.75-pl526h1de35cc_1&#xA;  perl-scalar-list-~ bioconda/osx-64::perl-scalar-list-utils-1.52-pl526h01d97ff_0&#xA;  perl-types-serial~ bioconda/osx-64::perl-types-serialiser-1.0-pl526_2&#xA;  perl-xsloader      bioconda/osx-64::perl-xsloader-0.24-pl526_0&#xA;  wget               pkgs/main/osx-64::wget-1.21.4-ha585b31_1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;There is a blast gpu and I'd assume the other library/packages have gpu versions. There's a lot of Perl code there BTW, so it's an old package, before NVIDIA was big perhaps?&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;If there is a GPU compliant channel on conda (nanoporetch) - that is impressive: especially given the amount of Perl code. Porting for GPU from CPU isn't trivial.&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;Few versions behind&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;The key thing is whether the upgrades are fixing bugs or refactoring/adding functionality. If it's the latter then I would go  for it because the GPU would compensate for the slower runtime (obviously given you don't need the extra functionality).&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2024-03-01T14:09:47.703" LastActivityDate="2024-03-01T14:09:47.703" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22232" PostTypeId="1" CreationDate="2024-03-01T10:35:12.770" Score="0" ViewCount="20" Body="&lt;p&gt;We are trying to download multiple UniRef clusters in one batch on the website. However, it seems the members are not downloaded... Could you guide me a way to the solution? Many thanks!&lt;/p&gt;&#xA;&lt;p&gt;For example, see &lt;a href=&quot;https://www.uniprot.org/uniref?query=membrane+protein&amp;amp;facets=identity%3A0.5&quot; rel=&quot;nofollow noreferrer&quot;&gt;this link&lt;/a&gt;. We would like to download all these clusters including their members. But when selected these clusters, only the representative sequences (one sequence per cluster) were downloaded.&lt;/p&gt;&#xA;" OwnerUserId="20028" LastEditorUserId="298" LastEditDate="2024-03-01T17:36:23.393" LastActivityDate="2024-03-31T21:07:45.947" Title="How to download multiple UniRef clusters with all their members?" Tags="|data-download|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22233" PostTypeId="2" ParentId="22173" CreationDate="2024-03-01T18:12:07.223" Score="0" Body="&lt;p&gt;Like mentioned in the comment above, I would suggest emailing the corresponding author. Scientific articles usually list the email of the corresponding author and this would be the most logical first step in addressing the missing materials.&lt;/p&gt;&#xA;" OwnerUserId="1574" LastActivityDate="2024-03-01T18:12:07.223" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22234" PostTypeId="2" ParentId="22232" CreationDate="2024-03-01T18:32:55.773" Score="0" Body="&lt;p&gt;The UniProt page apparently only lets you download fasta sequences for the representative protein in each UniRef group:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/QmOT6.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/QmOT6.png&quot; alt=&quot;screenshot of uniprot downlad page showing that only representative fasta is available&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;So you can't get all the fasta sequences from there. However, once you get the list of UniProt IDs, you can get the sequences from those. To get the IDs, first select TSV output:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/zPWsD.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/zPWsD.png&quot; alt=&quot;screenshot of the uniprot download page with TSV selected&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then remove everything and add &amp;quot;cluster members&amp;quot; from the &amp;quot;Miscellaneous&amp;quot; section:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/u37ha.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/u37ha.png&quot; alt=&quot;screenshot of the uniprot website with TSV selected and only &amp;quot;clustyer members&amp;quot;&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;This will give you a file that looks like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ zcat uniref_id_UniRef50_A0A0F6EQC6_OR_id_U_2024_03_01.tsv.gz &#xA;Cluster ID  Cluster members&#xA;UniRef50_A0A0F6EQC6 A0A0F6EQC6; E3E6D7; A0A3G8RAV1; A0A2K1EFR7; A0A2S6NYI6; A0A7H0Y5M1; A0A2N9YZ66; A0A853Q3J3; A0A8I2C241; A0A0K2FC93&#xA;UniRef50_A0A0T6A5I5 A0A0T6A5I5; A0A2V7BZB0; A0A2V6XWF2&#xA;UniRef50_A0A2H4TXU1 A0A2H4TXU1&#xA;UniRef50_A0A6C2YI31 A0A6C2YI31&#xA;UniRef50_B1KXG0 B1KXG0; UPI001F3F1379&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It's a tab separated file, and the second field is a &lt;code&gt;;&lt;/code&gt;-separated list of cluster members. Get those into a new file, one per line:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ zcat uniref_id_UniRef50_A0A0F6EQC6_OR_id_U_2024_03_01.tsv.gz | &#xA;    tail -n +2 | cut -f2 | sed 's/; /\n/g'&#xA;A0A0F6EQC6&#xA;E3E6D7&#xA;A0A3G8RAV1&#xA;A0A2K1EFR7&#xA;A0A2S6NYI6&#xA;A0A7H0Y5M1&#xA;A0A2N9YZ66&#xA;A0A853Q3J3&#xA;A0A8I2C241&#xA;A0A0K2FC93&#xA;A0A0T6A5I5&#xA;A0A2V7BZB0&#xA;A0A2V6XWF2&#xA;A0A2H4TXU1&#xA;A0A6C2YI31&#xA;B1KXG0&#xA;UPI001F3F1379&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Save to a file with&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ zcat uniref_id_UniRef50_A0A0F6EQC6_OR_id_U_2024_03_01.tsv.gz | &#xA;    tail -n +2 | cut -f2 | sed 's/; /\n/g &amp;gt; newfile&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It may be possible for the same protein to belong to multiple clusters, I don't know UniRef well enough to be sure if this is the case. So, to be on the safe side, make the IDs unique:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zcat uniref_id_UniRef50_A0A0F6EQC6_OR_id_U_2024_03_01.tsv.gz | tail -n +2 | cut -f2 | sed 's/; /\n/g' | sort -u&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That assumes GNU &lt;code&gt;sort&lt;/code&gt;, the default on Linux. If yours doesn't have a &lt;code&gt;-u&lt;/code&gt; option, use this instead:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zcat uniref_id_UniRef50_A0A0F6EQC6_OR_id_U_2024_03_01.tsv.gz | tail -n +2 | cut -f2 | sed 's/; /\n/g' | sort | uniq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, you can use these IDs and UniProt's ID mapping tool to get their sequences. Go to the &lt;a href=&quot;https://www.uniprot.org/id-mapping&quot; rel=&quot;nofollow noreferrer&quot;&gt;mapping tool&lt;/a&gt; and paste in your IDs (this assumes you have no more than 100000 IDs. If you have more, do this in multiple steps.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/bTTzO.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/bTTzO.png&quot; alt=&quot;screenshot of the ID mapping tool&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Click on &amp;quot;Map N IDs&amp;quot; and that will take you to &lt;a href=&quot;https://www.uniprot.org/id-mapping/uniprotkb/bb5c9c4ede74975062a9973483b64bca6c2f275c/overview&quot; rel=&quot;nofollow noreferrer&quot;&gt;a page&lt;/a&gt; like this, where you can now click on &amp;quot;Download&amp;quot; to retrieve the sequences:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YWNfW.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YWNfW.png&quot; alt=&quot;screenshot of the results page and download link&quot; /&gt;&lt;/a&gt;&#xA;&lt;a href=&quot;https://i.stack.imgur.com/sUuEo.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/sUuEo.png&quot; alt=&quot;this is what you see after clicking on download&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The result of all this is a fasta file like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ zcat foo.fa.gz | head&#xA;&amp;gt;tr|A0A0F6EQC6|A0A0F6EQC6_PAEPO Membrane protease regulatory membrane protein OS=Paenibacillus polymyxa OX=1406 GN=yuaF PE=4 SV=1&#xA;METLFWWLLAGGAIFTVVSVLIGDVLGSWLDGALELPGLDWFRPVVLLGAMTAFGGAGVL&#xA;LTKYTGLTMGWVVVLALGIALVIGVLVFFAFIKPMANSEVSSGFSMRELTGRIGEVTIPV&#xA;PEVGYGEVMIRLGAGNTIHTASSFDHKPLAAGTRIVVVEVAEGVVRVAYLDT&#xA;&amp;gt;tr|A0A0K2FC93|A0A0K2FC93_9BACL Membrane protease regulatory membrane protein OS=Paenibacillus peoriae OX=59893 GN=ABE82_18870 PE=4 SV=1&#xA;METIFWWLLAGGAIFTVVSVLIGDVLGGWLDGALELPGLDWFRPVVLLGAMTAFGGAGVL&#xA;LTKYTGLSMGWVVVLALGIAFVIGILVFFAFIKPMANSEVSSGFSMRELTGRIGEVTIPV&#xA;PEVGYGEVMIRLGAGNTIHTASSFDHKPLAAGTRIVVVEVAEGVVRVAYLDT&#xA;&amp;gt;tr|A0A0T6A5I5|A0A0T6A5I5_9BACT Membrane protein, putative membrane protein OS=candidate division NC10 bacterium CSP1-5 OX=1640516 GN=XU15_C0006G0101 PE=4 SV=1&#xA;MDSFVQLVVGTVVLRPYVVAFLLLYLFAAVADMGWRRTLLFTVVAWSVAFTAESLSIRFG&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However, I would urge you to think if this is really the best way for what you want to do. If you are trying to find orthologous groups, there are better ways.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2024-03-01T18:32:55.773" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22236" PostTypeId="2" ParentId="22218" CreationDate="2024-03-02T14:22:09.437" Score="1" Body="&lt;p&gt;A couple of ideas:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Use kraken2 to classify the reads. If you are able to convert your &amp;quot;repo&amp;quot; to kraken2 format, then the matching will be very fast and relatively accurate. Also, using bwa-mem on a joint database could work.&lt;/li&gt;&#xA;&lt;li&gt;I am not a big fan of custom and not well defined &amp;quot;repos&amp;quot;. Why don't you just use the official RefSeq repository of prokaryote genomes? (&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/refseq/about/prokaryotes/&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/refseq/about/prokaryotes/&lt;/a&gt;). I don't remember its size, but I remember using it without any problem.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="1140" LastActivityDate="2024-03-02T14:22:09.437" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22237" PostTypeId="1" AcceptedAnswerId="22238" CreationDate="2024-03-03T10:31:41.543" Score="2" ViewCount="80" Body="&lt;p&gt;I'm working with targeted Illumina sequencing data generated with DNA from diseased and healthy tissue (this is an age-related disease and is not cancer/neoplastic). My hypothesis is that the diseased tissue might contain specific somatic mutations in one particular gene (let's call it GENE), while the healthy tissue will not. The reasoning behind this: The same mutations cause autosomal-dominant versions of my disease of interest when present in the germline (my diseased tissue is from sporadic disease). My coverage varies, but is in the range of 1000x - 50'000x. My sequencing protocol employed UMIs.&lt;/p&gt;&#xA;&lt;p&gt;I have run Mutect2 in default settings and with lower thresholds (&lt;code&gt;--tumor-lod-to-emit&lt;/code&gt;) and haven't found any calls that correspond to these pathogenic mutations, so my results are negative. Nevertheless, I have reason to believe that my pathogenic mutations might be present at &lt;strong&gt;very&lt;/strong&gt; low VAF (around 0.05-0.1 %), based on other experiments.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;My question&lt;/strong&gt;: How do I quantify the number of reads that correspond to a particular variant of interest? Let's say that base 2000 in GENE is usually an A, whereas the mutation A&amp;gt;T would be pathogenic. How do I count the number of Ts in position 2000?&lt;/p&gt;&#xA;&lt;p&gt;As I'm somewhat new to bioinformatics, I'm also very open to any other ideas. Thanks!&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;: @terdon's approach worked perfectly in cases where I used a VCF for &lt;code&gt;--alleles&lt;/code&gt; that had been generated as an output of Mutect2, such as the following one:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;VCF produced by Mutect&lt;/em&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  &#xA;chr20   4699605 .   A   G   .   .   AS_SB_TABLE=881,932|920,847;DP=3791;ECNT=1;MBQ=20,17;MFRL=248,243;MMQ=60,60;MPOS=21;POPAF=7.30;TLOD=2014.57 GT:AD:AF:DP:F1R2:F2R1:FAD:SB    0/1:1813,1767:0.469:3580:559,218:384,191:1720,1679:881,932,920,847&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Curiously, if I used a manually produced file such as the following, no output was produced, unless the variant of interest was heterozygous:&lt;/p&gt;&#xA;&lt;p&gt;&lt;em&gt;Manually produced VCF&lt;/em&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT&#xA;chr20   4699605 rs1799990   A   G   .   .   .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;Output using the manual VCF, heterozygous sample&lt;/em&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  heterogyzous_sample&#xA;chr20   4699605 .   A   G   .   .   AS_SB_TABLE=881,932|920,847;DP=3791;ECNT=1;MBQ=20,17;MFRL=248,243;MMQ=60,60;MPOS=21;POPAF=7.30;TLOD=2014.57 GT:AD:AF:DP:F1R2:F2R1:FAD:SB    0/1:1813,1767:0.469:3580:559,218:384,191:1720,1679:881,932,920,847&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;em&gt;Output using the manual VCF, non-heterozygous sample&lt;/em&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  novariant_sample&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Manual inspection of this non-heterozygous sample revealed that it contained 1 read with the ALT variant, so this doesn't seem to depend on the variant being absent.&lt;/p&gt;&#xA;" OwnerUserId="9025" LastEditorUserId="9025" LastEditDate="2024-03-10T13:33:21.617" LastActivityDate="2024-03-10T13:33:21.617" Title="How do I quantify a specific somatic variant?" Tags="|variant-calling|illumina|gatk|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22238" PostTypeId="2" ParentId="22237" CreationDate="2024-03-03T17:45:10.007" Score="4" Body="&lt;p&gt;I have a little function that can report this by using samtools to look at the read pileup. Um. It isn't the most elegant piece of code, but it does sorta do the job:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-sh prettyprint-override&quot;&gt;&lt;code&gt;variantSupport () &#xA;{ &#xA;  genome=$1&#xA;  bam=$2;&#xA;  range=$3;&#xA;  samtools mpileup -f &amp;quot;$genome&amp;quot; -r &amp;quot;$range&amp;quot; &amp;quot;$bam&amp;quot; | cut -f 5 |&#xA;    tr '[a-z]' '[A-Z]' | fold -w 1 | sort | uniq -c |&#xA;    awk 'BEGIN{tot=1}$2~/,/{ $2=&amp;quot;.&amp;quot; }&#xA;         {&#xA;           tot+=$1;&#xA;           c[$2]+=$1&#xA;         }&#xA;         $2~/[a-zA-Z]/{ varReads+=$1; }&#xA;         END{&#xA;           for(i in c){&#xA;             print i,c[i]&#xA;           }&#xA;           printf &amp;quot;%s of %s reads (%.1f%%), support the variant.\n&amp;quot;,varReads,tot,varReads*100/tot&#xA;   }'&#xA;}&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Add the function to your &lt;code&gt;~/.bashrc&lt;/code&gt; or just paste it directly into a terminal and you can then do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ variantSupport /path/to/genome/hg19.fa foo.bam chr13:28607916-28607916&#xA;[mpileup] 1 samples in 1 input files&#xA;A 915&#xA;* 1&#xA;. 93&#xA;915 of 1010 reads (90.6%), support the variant.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This is telling me that at position 28607916 of chr13, I have 1 read supporting a deletion (&lt;code&gt;*&lt;/code&gt;, see &lt;a href=&quot;https://www.htslib.org/doc/samtools-mpileup.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.htslib.org/doc/samtools-mpileup.html&lt;/a&gt; for an explanation of the symbols), 93 reads supporting the reference residue (T, in this case) and 915 supporting an A, so a T=&amp;gt; A variant.&lt;/p&gt;&#xA;&lt;p&gt;However, a better approach is to use a variant caller and force it to call the variant you want. You can then investigate the support for it. With &lt;code&gt;mutect2&lt;/code&gt;, you can use the &lt;code&gt;--alleles&lt;/code&gt; option to force the calling and then use the &lt;code&gt;-L&lt;/code&gt; option to limit calling to only the specific region. If you give the same vcf file for both options, that will return results for the variants you want and only those. For example, given this file &lt;code&gt;foo.vcf.gz&lt;/code&gt; (which needs to be compressed using &lt;code&gt;bgzip&lt;/code&gt; and indexed using &lt;code&gt;tabix&lt;/code&gt;):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ zcat 1.vcf.gz &#xA;##fileformat=VCFv4.3&#xA;##reference=/genomes/hg19/hg19.fa&#xA;##FORMAT=&amp;lt;ID=AB,Number=A,Type=Float,Description=&amp;quot;Allelic Balance&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=DP,Number=1,Type=Integer,Description=&amp;quot;Read depth&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GQ,Number=1,Type=Integer,Description=&amp;quot;Net Genotype quality across all datasets, calculated from GQ scores of callsets supporting the consensus GT, using only one callset from each dataset&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&amp;quot;Genotype&amp;quot;&amp;gt;&#xA;##contig=&amp;lt;ID=chr13,assembly=hg19,length=115169878&amp;gt;&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  HG003&#xA;chr13   28607917    .   G   C   .   .   .   GT:AB:DP:GQ 1/1:0.6:395:318&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;You can then do:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gatk Mutect2 \&#xA;  -R /genomes/hg19/hg19.fa \&#xA;  --alleles 1.vcf.gz  \&#xA;  -L 1.vcf.gz \&#xA;  -I foo.bam \&#xA;  -O out.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will produce an &lt;code&gt;out.vcf&lt;/code&gt; like this (I am excluding most headers for clarity):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&#xA;##FORMAT=&amp;lt;ID=AD,Number=R,Type=Integer,Description=&amp;quot;Allelic depths for the ref and alt alleles in the order listed&amp;quot;&amp;gt;&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  331970&#xA;chr13   28607917    .   G   C   .   .   DP=1107;ECNT=1;MBQ=30,0;MFRL=353,0;MMQ=60,60;MPOS=50;POPAF=7.30;TLOD=-2.716e+00 GT:AD:AF:DP:F1R2:F2R1:SB    0/1:1043,0:9.616e-04:1043:518,0:496,0:989,54,0,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I can see this is not a real variant since the value for AD, the number of reads supporting the reference and the variant is &lt;code&gt;1043,0&lt;/code&gt;, meaning that 1043 reads support the reference and none support the variant.&lt;/p&gt;&#xA;&lt;p&gt;This way, you can get results for your variants of interest and see if there is any evidence supporting them.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2024-03-03T17:45:10.007" CommentCount="8" ContentLicense="CC BY-SA 4.0" />
  <row Id="22239" PostTypeId="1" CreationDate="2024-03-04T04:42:44.477" Score="2" ViewCount="80" Body="&lt;p&gt;Trying to understand when GSEA is more appropriate than GSVA and vice versa. I have seen cases when running GSEA and GSVA on the same task - compare enrichment of a geneset between two groups - gives different results. In this case one method will identify significant enrichment while the other will not. GSEA seems to detect more &amp;quot;hits&amp;quot; than GSVA.&lt;/p&gt;&#xA;&lt;p&gt;I understand that they are different methods operating on different levels (group/condition level vs single sample level) and GSVA is needed when you want to investigate the relationship between a geneset and a non-binary grouping (other continuous variable, more than 2 categories, survival analysis etc), but I cannot find much discussion on how to interpret GSEA/GSVA results when they differ or if there are some situations for the scenario I have described (comparison of geneset enrichment in 2 groups) where one method performs better than the other. Is one method better when there are fewer samples or does one method produce more false positives than the other?&lt;/p&gt;&#xA;&lt;p&gt;I also know there are other single sample enrichment scoring algorithms (singscore for bulk RNA-Seq, and a host of scRNA-seq methods), but I ask about GSVA as it still seems to be a popular choice in many papers.&lt;/p&gt;&#xA;&lt;p&gt;Can someone provide justification for when to use one method over the other or point me towards any papers that explore this topic?&lt;/p&gt;&#xA;" OwnerUserId="6674" LastEditorUserId="48" LastEditDate="2024-03-11T15:43:23.020" LastActivityDate="2024-03-11T15:43:23.020" Title="GSEA vs GSVA: Pros and cons of each method" Tags="|rna-seq|gene-expression|microarray|gsea|literature-search|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22241" PostTypeId="2" ParentId="22210" CreationDate="2024-03-05T03:20:46.687" Score="1" Body="&lt;p&gt;What I would personally do is run a Χ&lt;sup&gt;2&lt;/sup&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;from scipy.stats import chisquare&#xA;import numpy as np&#xA;&#xA;pan = [11, 39, 4 ....]&#xA;core = [29, 50, 7 ... ]&#xA;f_exp = np.array(pan)&#xA;f_obs = np. array(core)&#xA;print (chisquare(f_obs=f_obs, f_exp=f_exp))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This will output the total statistic and whether its above or below the critical probability.&lt;/p&gt;&#xA;&lt;p&gt;Fisher's Exact test is essentially Χ&lt;sup&gt;2&lt;/sup&gt; for a small number of observations, which is not the case with your data.&lt;/p&gt;&#xA;&lt;p&gt;Looking at your data Χ&lt;sup&gt;2&lt;/sup&gt; is not likely to be &amp;quot;significant&amp;quot; (which is the result you are looking for), i.e. the pan genome is the core genome across all COGs.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2024-03-05T03:26:13.437" LastActivityDate="2024-03-05T03:26:13.437" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22242" PostTypeId="1" AcceptedAnswerId="22243" CreationDate="2024-03-05T10:54:31.987" Score="3" ViewCount="62" Body="&lt;p&gt;I have got a number of VCF files that all contain data on one particular variant, with every VCF being created from an individual sample. So there is one entry per file (headers omitted):&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##source=Mutect2&#xA;##tumor_sample=312091_5-Ctrl1_G01_S6_R1_001.fastq.gz&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  312091_5-Ctrl1_G01_S6_R1_001.fastq.gz&#xA;chr20   4699818 .   G   A   .   .   AS_SB_TABLE=805,864|0,0;DP=1871;ECNT=1;MBQ=17,0;MFRL=273,0;MMQ=60,60;MPOS=50;POPAF=7.30;TLOD=-3.537e+00 GT:AD:AF:DP:F1R2:F2R1:FAD:SB    0/1:1669,0:6.794e-04:1669:252,0:233,0:1595,0:805,864,0,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I would like to create a summary table of the parameters given within the VCFs, including of the parameters within the &lt;code&gt;FORMAT&lt;/code&gt; field. Ideally this table would have the following as header:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Sample CHROM    POS ID  REF ALT QUAL    FILTER  GT INFO-DP FORMAT-DP AD-REF AD-ALT&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and every sample/variant combination would be a line. I would like to use such a table to create statistics for my samples (on zygosity, number of ALT variants etc).&lt;/p&gt;&#xA;&lt;p&gt;Is there a pre-made tool that is capable of doing this? I'm also open to any other suggestions. Thanks!&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;Edit&lt;/strong&gt;:&#xA;Here are two VCF files (I have about 30 in total) and the desired output&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;First VCF&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.2&#xA;##FORMAT=&amp;lt;ID=AD,Number=R,Type=Integer,Description=&amp;quot;Allelic depths for the ref and alt alleles in the order listed&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=AF,Number=A,Type=Float,Description=&amp;quot;Allele fractions of alternate alleles in the tumor&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=DP,Number=1,Type=Integer,Description=&amp;quot;Approximate read depth (reads with MQ=255 or with bad mates are filtered)&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=F1R2,Number=R,Type=Integer,Description=&amp;quot;Count of reads in F1R2 pair orientation supporting each allele&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=F2R1,Number=R,Type=Integer,Description=&amp;quot;Count of reads in F2R1 pair orientation supporting each allele&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=FAD,Number=R,Type=Integer,Description=&amp;quot;Count of fragments supporting each allele.&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GQ,Number=1,Type=Integer,Description=&amp;quot;Genotype Quality&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&amp;quot;Genotype&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=PGT,Number=1,Type=String,Description=&amp;quot;Physical phasing haplotype information, describing how the alternate alleles are phased in relation to one another; will always be heterozygous and is not intended to describe called alleles&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=PID,Number=1,Type=String,Description=&amp;quot;Physical phasing ID information, where each unique ID within a given sample (but not across samples) connects records within a phasing group&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=PL,Number=G,Type=Integer,Description=&amp;quot;Normalized, Phred-scaled likelihoods for genotypes as defined in the VCF specification&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=PS,Number=1,Type=Integer,Description=&amp;quot;Phasing set (typically the position of the first variant in the set)&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=SB,Number=4,Type=Integer,Description=&amp;quot;Per-sample component statistics which comprise the Fisher's Exact Test to detect strand bias.&amp;quot;&amp;gt;&#xA;##GATKCommandLine=&amp;lt;ID=Mutect2,CommandLine=&amp;quot;Mutect2 --alleles /add/2024-03-04_force_mutect/E200K.vcf.gz --output /add/2024-03-04_force_mutect/output//Ctrl5_E200K_frequency.vcf --intervals /add/2024-03-04_force_mutect/E200K.vcf.gz --input /add/2023-06-18_CJD_8_samples/312091_8-Ctrl5_H02_S3_R1_001.fastq.gz.picard.markedDup.recal.bam --reference /home/mcarta/databases/hg38.fa --f1r2-median-mq 50 --f1r2-min-bq 20 --f1r2-max-depth 200 --flow-likelihood-parallel-threads 0 --flow-likelihood-optimized-comp false --trim-to-haplotype true --exact-matching false --flow-use-t0-tag false --flow-probability-threshold 0.003 --flow-remove-non-single-base-pair-indels false --flow-remove-one-zero-probs false --flow-quantization-bins 121 --flow-fill-empty-bins-value 0.001 --flow-symmetric-indel-probs false --flow-report-insertion-or-deletion false --flow-disallow-probs-larger-than-call false --flow-lump-probs false --flow-retain-max-n-probs-base-format false --flow-probability-scaling-factor 10 --flow-order-cycle-length 4 --keep-boundary-flows false --genotype-pon-sites false --genotype-germline-sites false --af-of-alleles-not-in-resource -1.0 --mitochondria-mode false --mutect3-training-mode false --mutect3-ref-downsample 10 --mutect3-alt-downsample 20 --mutect3-non-artifact-ratio 20 --tumor-lod-to-emit 3.0 --initial-tumor-lod 2.0 --pcr-snv-qual 40 --pcr-indel-qual 40 --base-qual-correction-factor 5 --max-population-af 0.01 --downsampling-stride 1 --callable-depth 10 --max-suspicious-reads-per-alignment-start 0 --normal-lod 2.2 --ignore-itr-artifacts false --gvcf-lod-band -2.5 --gvcf-lod-band -2.0 --gvcf-lod-band -1.5 --gvcf-lod-band -1.0 --gvcf-lod-band -0.5 --gvcf-lod-band 0.0 --gvcf-lod-band 0.5 --gvcf-lod-band 1.0 --minimum-allele-fraction 0.0 --independent-mates false --flow-mode NONE --disable-adaptive-pruning false --kmer-size 10 --kmer-size 25 --dont-increase-kmer-sizes-for-cycles false --allow-non-unique-kmers-in-ref false --num-pruning-samples 1 --min-dangling-branch-length 4 --recover-all-dangling-branches false --max-num-haplotypes-in-population 128 --min-pruning 2 --adaptive-pruning-initial-error-rate 0.001 --pruning-lod-threshold 2.302585092994046 --pruning-seeding-lod-threshold 9.210340371976184 --max-unpruned-variants 100 --linked-de-bruijn-graph false --disable-artificial-haplotype-recovery false --enable-legacy-graph-cycle-detection false --debug-assembly false --debug-graph-transformations false --capture-assembly-failure-bam false --num-matching-bases-in-dangling-end-to-recover -1 --error-correction-log-odds -Infinity --error-correct-reads false --kmer-length-for-read-error-correction 25 --min-observations-for-kmer-to-be-solid 20 --likelihood-calculation-engine PairHMM --base-quality-score-threshold 18 --dragstr-het-hom-ratio 2 --dont-use-dragstr-pair-hmm-scores false --pair-hmm-gap-continuation-penalty 10 --expected-mismatch-rate-for-read-disqualification 0.02 --pair-hmm-implementation FASTEST_AVAILABLE --pcr-indel-model CONSERVATIVE --phred-scaled-global-read-mismapping-rate 45 --disable-symmetric-hmm-normalizing false --disable-cap-base-qualities-to-map-quality false --enable-dynamic-read-disqualification-for-genotyping false --dynamic-read-disqualification-threshold 1.0 --native-pair-hmm-threads 4 --native-pair-hmm-use-double-precision false --flow-hmm-engine-min-indel-adjust 6 --flow-hmm-engine-flat-insertion-penatly 45 --flow-hmm-engine-flat-deletion-penatly 45 --pileup-detection false --use-pdhmm false --use-pdhmm-overlap-optimization false --make-determined-haps-from-pd-code false --print-pileupcalling-status false --fallback-gga-if-pdhmm-fails true --pileup-detection-enable-indel-pileup-calling false --pileup-detection-active-region-phred-threshold 0.0 --num-artificial-haplotypes-to-add-per-allele 5 --artifical-haplotype-filtering-kmer-size 10 --pileup-detection-snp-alt-threshold 0.1 --pileup-detection-indel-alt-threshold 0.1 --pileup-detection-absolute-alt-depth 0.0 --pileup-detection-snp-adjacent-to-assembled-indel-range 5 --pileup-detection-snp-basequality-filter 12 --pileup-detection-bad-read-tolerance 0.0 --pileup-detection-proper-pair-read-badness true --pileup-detection-edit-distance-read-badness-threshold 0.08 --pileup-detection-chimeric-read-badness true --pileup-detection-template-mean-badness-threshold 0.0 --pileup-detection-template-std-badness-threshold 0.0 --pileup-detection-filter-assembly-alt-bad-read-tolerance 0.0 --pileup-detection-edit-distance-read-badness-for-assembly-filtering-threshold 0.12 --bam-writer-type CALLED_HAPLOTYPES --dont-use-soft-clipped-bases false --override-fragment-softclip-check false --min-base-quality-score 10 --smith-waterman FASTEST_AVAILABLE --emit-ref-confidence NONE --max-mnp-distance 1 --force-call-filtered-alleles false --reference-model-deletion-quality 30 --soft-clip-low-quality-ends false --allele-informative-reads-overlap-margin 2 --smith-waterman-dangling-end-match-value 25 --smith-waterman-dangling-end-mismatch-penalty -50 --smith-waterman-dangling-end-gap-open-penalty -110 --smith-waterman-dangling-end-gap-extend-penalty -6 --smith-waterman-haplotype-to-reference-match-value 200 --smith-waterman-haplotype-to-reference-mismatch-penalty -150 --smith-waterman-haplotype-to-reference-gap-open-penalty -260 --smith-waterman-haplotype-to-reference-gap-extend-penalty -11 --smith-waterman-read-to-haplotype-match-value 10 --smith-waterman-read-to-haplotype-mismatch-penalty -15 --smith-waterman-read-to-haplotype-gap-open-penalty -30 --smith-waterman-read-to-haplotype-gap-extend-penalty -5 --flow-assembly-collapse-hmer-size 0 --flow-assembly-collapse-partial-mode false --flow-filter-alleles false --flow-filter-alleles-qual-threshold 30.0 --flow-filter-alleles-sor-threshold 3.0 --flow-filter-lone-alleles false --flow-filter-alleles-debug-graphs false --min-assembly-region-size 50 --max-assembly-region-size 300 --active-probability-threshold 0.002 --max-prob-propagation-distance 50 --force-active false --assembly-region-padding 100 --padding-around-indels 75 --padding-around-snps 20 --padding-around-strs 75 --max-extension-into-assembly-region-padding-legacy 25 --max-reads-per-alignment-start 50 --enable-legacy-assembly-region-trimming false --interval-set-rule UNION --interval-padding 0 --interval-exclusion-padding 0 --interval-merging-rule ALL --read-validation-stringency SILENT --seconds-between-progress-updates 10.0 --disable-sequence-dictionary-validation false --create-output-bam-index true --create-output-bam-md5 false --create-output-variant-index true --create-output-variant-md5 false --max-variants-per-shard 0 --lenient false --add-output-sam-program-record true --add-output-vcf-command-line true --cloud-prefetch-buffer 40 --cloud-index-prefetch-buffer -1 --disable-bam-index-caching false --sites-only-vcf-output false --help false --version false --showHidden false --verbosity INFO --QUIET false --use-jdk-deflater false --use-jdk-inflater false --gcs-max-retries 20 --gcs-project-for-requester-pays  --disable-tool-default-read-filters false --max-read-length 2147483647 --min-read-length 30 --minimum-mapping-quality 20 --disable-tool-default-annotations false --enable-all-annotations false&amp;quot;,Version=&amp;quot;4.5.0.0&amp;quot;,Date=&amp;quot;March 5, 2024 at 10:20:03 AM UTC&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=AS_SB_TABLE,Number=1,Type=String,Description=&amp;quot;Allele-specific forward/reverse read counts for strand bias tests. Includes the reference and alleles separated by |.&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=AS_UNIQ_ALT_READ_COUNT,Number=A,Type=Integer,Description=&amp;quot;Number of reads with unique start and mate end positions for each alt at a variant site&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=CONTQ,Number=1,Type=Float,Description=&amp;quot;Phred-scaled qualities that alt allele are not due to contamination&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=DP,Number=1,Type=Integer,Description=&amp;quot;Approximate read depth; some reads may have been filtered&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=ECNT,Number=1,Type=Integer,Description=&amp;quot;Number of events in this haplotype&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=GERMQ,Number=1,Type=Integer,Description=&amp;quot;Phred-scaled quality that alt alleles are not germline variants&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=MBQ,Number=R,Type=Integer,Description=&amp;quot;median base quality by allele&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=MFRL,Number=R,Type=Integer,Description=&amp;quot;median fragment length by allele&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=MMQ,Number=R,Type=Integer,Description=&amp;quot;median mapping quality by allele&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=MPOS,Number=A,Type=Integer,Description=&amp;quot;median distance from end of read&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=NALOD,Number=A,Type=Float,Description=&amp;quot;Negative log 10 odds of artifact in normal with same allele fraction as tumor&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=NCount,Number=1,Type=Integer,Description=&amp;quot;Count of N bases in the pileup&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=NLOD,Number=A,Type=Float,Description=&amp;quot;Normal log 10 likelihood ratio of diploid het or hom alt genotypes&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=OCM,Number=1,Type=Integer,Description=&amp;quot;Number of alt reads whose original alignment doesn't match the current contig.&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=PON,Number=0,Type=Flag,Description=&amp;quot;site found in panel of normals&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=POPAF,Number=A,Type=Float,Description=&amp;quot;negative log 10 population allele frequencies of alt alleles&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=ROQ,Number=1,Type=Float,Description=&amp;quot;Phred-scaled qualities that alt allele are not due to read orientation artifact&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=RPA,Number=R,Type=Integer,Description=&amp;quot;Number of times tandem repeat unit is repeated, for each allele (including reference)&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=RU,Number=1,Type=String,Description=&amp;quot;Tandem repeat unit (bases)&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=SEQQ,Number=1,Type=Integer,Description=&amp;quot;Phred-scaled quality that alt alleles are not sequencing errors&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=STR,Number=0,Type=Flag,Description=&amp;quot;Variant is a short tandem repeat&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=STRANDQ,Number=1,Type=Integer,Description=&amp;quot;Phred-scaled quality of strand bias artifact&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=STRQ,Number=1,Type=Integer,Description=&amp;quot;Phred-scaled quality that alt alleles in STRs are not polymerase slippage errors&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=TLOD,Number=A,Type=Float,Description=&amp;quot;Log 10 likelihood ratio score of variant existing versus not existing&amp;quot;&amp;gt;&#xA;##MutectVersion=2.2&#xA;##contig=&amp;lt;ID=chr2,length=242193529&amp;gt;&#xA;##contig=&amp;lt;ID=chr20,length=64444167&amp;gt;&#xA;##contig=&amp;lt;ID=chr4,length=190214555&amp;gt;&#xA;##filtering_status=Warning: unfiltered Mutect 2 calls.  Please run FilterMutectCalls to remove false positives.&#xA;##source=Mutect2&#xA;##tumor_sample=312091_8-Ctrl5_H02_S3_R1_001.fastq.gz&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  312091_8-Ctrl5_H02_S3_R1_001.fastq.gz&#xA;chr20   4699818 .   G   A   .   .   AS_SB_TABLE=1376,1402|0,0;DP=2939;ECNT=1;MBQ=18,0;MFRL=257,0;MMQ=60,60;MPOS=50;POPAF=7.30;TLOD=-3.455e+00   GT:AD:AF:DP:F1R2:F2R1:FAD:SB    0/1:2778,0:4.100e-04:2778:472,0:398,0:2493,0:1376,1402,0,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Second VCF&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.2&#xA;##FORMAT=&amp;lt;ID=AD,Number=R,Type=Integer,Description=&amp;quot;Allelic depths for the ref and alt alleles in the order listed&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=AF,Number=A,Type=Float,Description=&amp;quot;Allele fractions of alternate alleles in the tumor&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=DP,Number=1,Type=Integer,Description=&amp;quot;Approximate read depth (reads with MQ=255 or with bad mates are filtered)&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=F1R2,Number=R,Type=Integer,Description=&amp;quot;Count of reads in F1R2 pair orientation supporting each allele&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=F2R1,Number=R,Type=Integer,Description=&amp;quot;Count of reads in F2R1 pair orientation supporting each allele&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=FAD,Number=R,Type=Integer,Description=&amp;quot;Count of fragments supporting each allele.&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GQ,Number=1,Type=Integer,Description=&amp;quot;Genotype Quality&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=GT,Number=1,Type=String,Description=&amp;quot;Genotype&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=PGT,Number=1,Type=String,Description=&amp;quot;Physical phasing haplotype information, describing how the alternate alleles are phased in relation to one another; will always be heterozygous and is not intended to describe called alleles&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=PID,Number=1,Type=String,Description=&amp;quot;Physical phasing ID information, where each unique ID within a given sample (but not across samples) connects records within a phasing group&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=PL,Number=G,Type=Integer,Description=&amp;quot;Normalized, Phred-scaled likelihoods for genotypes as defined in the VCF specification&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=PS,Number=1,Type=Integer,Description=&amp;quot;Phasing set (typically the position of the first variant in the set)&amp;quot;&amp;gt;&#xA;##FORMAT=&amp;lt;ID=SB,Number=4,Type=Integer,Description=&amp;quot;Per-sample component statistics which comprise the Fisher's Exact Test to detect strand bias.&amp;quot;&amp;gt;&#xA;##GATKCommandLine=&amp;lt;ID=Mutect2,CommandLine=&amp;quot;Mutect2 --alleles /add/2024-03-04_force_mutect/E200K.vcf.gz --output /add/2024-03-04_force_mutect/output//Ctrl4_E200K_frequency.vcf --intervals /add/2024-03-04_force_mutect/E200K.vcf.gz --input /add/2023-06-18_CJD_16_samples/312092_15-Ctrl4_H01_S11_R1_001.fastq.gz.picard.markedDup.recal.bam --reference /home/mcarta/databases/hg38.fa --f1r2-median-mq 50 --f1r2-min-bq 20 --f1r2-max-depth 200 --flow-likelihood-parallel-threads 0 --flow-likelihood-optimized-comp false --trim-to-haplotype true --exact-matching false --flow-use-t0-tag false --flow-probability-threshold 0.003 --flow-remove-non-single-base-pair-indels false --flow-remove-one-zero-probs false --flow-quantization-bins 121 --flow-fill-empty-bins-value 0.001 --flow-symmetric-indel-probs false --flow-report-insertion-or-deletion false --flow-disallow-probs-larger-than-call false --flow-lump-probs false --flow-retain-max-n-probs-base-format false --flow-probability-scaling-factor 10 --flow-order-cycle-length 4 --keep-boundary-flows false --genotype-pon-sites false --genotype-germline-sites false --af-of-alleles-not-in-resource -1.0 --mitochondria-mode false --mutect3-training-mode false --mutect3-ref-downsample 10 --mutect3-alt-downsample 20 --mutect3-non-artifact-ratio 20 --tumor-lod-to-emit 3.0 --initial-tumor-lod 2.0 --pcr-snv-qual 40 --pcr-indel-qual 40 --base-qual-correction-factor 5 --max-population-af 0.01 --downsampling-stride 1 --callable-depth 10 --max-suspicious-reads-per-alignment-start 0 --normal-lod 2.2 --ignore-itr-artifacts false --gvcf-lod-band -2.5 --gvcf-lod-band -2.0 --gvcf-lod-band -1.5 --gvcf-lod-band -1.0 --gvcf-lod-band -0.5 --gvcf-lod-band 0.0 --gvcf-lod-band 0.5 --gvcf-lod-band 1.0 --minimum-allele-fraction 0.0 --independent-mates false --flow-mode NONE --disable-adaptive-pruning false --kmer-size 10 --kmer-size 25 --dont-increase-kmer-sizes-for-cycles false --allow-non-unique-kmers-in-ref false --num-pruning-samples 1 --min-dangling-branch-length 4 --recover-all-dangling-branches false --max-num-haplotypes-in-population 128 --min-pruning 2 --adaptive-pruning-initial-error-rate 0.001 --pruning-lod-threshold 2.302585092994046 --pruning-seeding-lod-threshold 9.210340371976184 --max-unpruned-variants 100 --linked-de-bruijn-graph false --disable-artificial-haplotype-recovery false --enable-legacy-graph-cycle-detection false --debug-assembly false --debug-graph-transformations false --capture-assembly-failure-bam false --num-matching-bases-in-dangling-end-to-recover -1 --error-correction-log-odds -Infinity --error-correct-reads false --kmer-length-for-read-error-correction 25 --min-observations-for-kmer-to-be-solid 20 --likelihood-calculation-engine PairHMM --base-quality-score-threshold 18 --dragstr-het-hom-ratio 2 --dont-use-dragstr-pair-hmm-scores false --pair-hmm-gap-continuation-penalty 10 --expected-mismatch-rate-for-read-disqualification 0.02 --pair-hmm-implementation FASTEST_AVAILABLE --pcr-indel-model CONSERVATIVE --phred-scaled-global-read-mismapping-rate 45 --disable-symmetric-hmm-normalizing false --disable-cap-base-qualities-to-map-quality false --enable-dynamic-read-disqualification-for-genotyping false --dynamic-read-disqualification-threshold 1.0 --native-pair-hmm-threads 4 --native-pair-hmm-use-double-precision false --flow-hmm-engine-min-indel-adjust 6 --flow-hmm-engine-flat-insertion-penatly 45 --flow-hmm-engine-flat-deletion-penatly 45 --pileup-detection false --use-pdhmm false --use-pdhmm-overlap-optimization false --make-determined-haps-from-pd-code false --print-pileupcalling-status false --fallback-gga-if-pdhmm-fails true --pileup-detection-enable-indel-pileup-calling false --pileup-detection-active-region-phred-threshold 0.0 --num-artificial-haplotypes-to-add-per-allele 5 --artifical-haplotype-filtering-kmer-size 10 --pileup-detection-snp-alt-threshold 0.1 --pileup-detection-indel-alt-threshold 0.1 --pileup-detection-absolute-alt-depth 0.0 --pileup-detection-snp-adjacent-to-assembled-indel-range 5 --pileup-detection-snp-basequality-filter 12 --pileup-detection-bad-read-tolerance 0.0 --pileup-detection-proper-pair-read-badness true --pileup-detection-edit-distance-read-badness-threshold 0.08 --pileup-detection-chimeric-read-badness true --pileup-detection-template-mean-badness-threshold 0.0 --pileup-detection-template-std-badness-threshold 0.0 --pileup-detection-filter-assembly-alt-bad-read-tolerance 0.0 --pileup-detection-edit-distance-read-badness-for-assembly-filtering-threshold 0.12 --bam-writer-type CALLED_HAPLOTYPES --dont-use-soft-clipped-bases false --override-fragment-softclip-check false --min-base-quality-score 10 --smith-waterman FASTEST_AVAILABLE --emit-ref-confidence NONE --max-mnp-distance 1 --force-call-filtered-alleles false --reference-model-deletion-quality 30 --soft-clip-low-quality-ends false --allele-informative-reads-overlap-margin 2 --smith-waterman-dangling-end-match-value 25 --smith-waterman-dangling-end-mismatch-penalty -50 --smith-waterman-dangling-end-gap-open-penalty -110 --smith-waterman-dangling-end-gap-extend-penalty -6 --smith-waterman-haplotype-to-reference-match-value 200 --smith-waterman-haplotype-to-reference-mismatch-penalty -150 --smith-waterman-haplotype-to-reference-gap-open-penalty -260 --smith-waterman-haplotype-to-reference-gap-extend-penalty -11 --smith-waterman-read-to-haplotype-match-value 10 --smith-waterman-read-to-haplotype-mismatch-penalty -15 --smith-waterman-read-to-haplotype-gap-open-penalty -30 --smith-waterman-read-to-haplotype-gap-extend-penalty -5 --flow-assembly-collapse-hmer-size 0 --flow-assembly-collapse-partial-mode false --flow-filter-alleles false --flow-filter-alleles-qual-threshold 30.0 --flow-filter-alleles-sor-threshold 3.0 --flow-filter-lone-alleles false --flow-filter-alleles-debug-graphs false --min-assembly-region-size 50 --max-assembly-region-size 300 --active-probability-threshold 0.002 --max-prob-propagation-distance 50 --force-active false --assembly-region-padding 100 --padding-around-indels 75 --padding-around-snps 20 --padding-around-strs 75 --max-extension-into-assembly-region-padding-legacy 25 --max-reads-per-alignment-start 50 --enable-legacy-assembly-region-trimming false --interval-set-rule UNION --interval-padding 0 --interval-exclusion-padding 0 --interval-merging-rule ALL --read-validation-stringency SILENT --seconds-between-progress-updates 10.0 --disable-sequence-dictionary-validation false --create-output-bam-index true --create-output-bam-md5 false --create-output-variant-index true --create-output-variant-md5 false --max-variants-per-shard 0 --lenient false --add-output-sam-program-record true --add-output-vcf-command-line true --cloud-prefetch-buffer 40 --cloud-index-prefetch-buffer -1 --disable-bam-index-caching false --sites-only-vcf-output false --help false --version false --showHidden false --verbosity INFO --QUIET false --use-jdk-deflater false --use-jdk-inflater false --gcs-max-retries 20 --gcs-project-for-requester-pays  --disable-tool-default-read-filters false --max-read-length 2147483647 --min-read-length 30 --minimum-mapping-quality 20 --disable-tool-default-annotations false --enable-all-annotations false&amp;quot;,Version=&amp;quot;4.5.0.0&amp;quot;,Date=&amp;quot;March 5, 2024 at 10:21:02 AM UTC&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=AS_SB_TABLE,Number=1,Type=String,Description=&amp;quot;Allele-specific forward/reverse read counts for strand bias tests. Includes the reference and alleles separated by |.&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=AS_UNIQ_ALT_READ_COUNT,Number=A,Type=Integer,Description=&amp;quot;Number of reads with unique start and mate end positions for each alt at a variant site&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=CONTQ,Number=1,Type=Float,Description=&amp;quot;Phred-scaled qualities that alt allele are not due to contamination&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=DP,Number=1,Type=Integer,Description=&amp;quot;Approximate read depth; some reads may have been filtered&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=ECNT,Number=1,Type=Integer,Description=&amp;quot;Number of events in this haplotype&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=GERMQ,Number=1,Type=Integer,Description=&amp;quot;Phred-scaled quality that alt alleles are not germline variants&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=MBQ,Number=R,Type=Integer,Description=&amp;quot;median base quality by allele&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=MFRL,Number=R,Type=Integer,Description=&amp;quot;median fragment length by allele&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=MMQ,Number=R,Type=Integer,Description=&amp;quot;median mapping quality by allele&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=MPOS,Number=A,Type=Integer,Description=&amp;quot;median distance from end of read&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=NALOD,Number=A,Type=Float,Description=&amp;quot;Negative log 10 odds of artifact in normal with same allele fraction as tumor&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=NCount,Number=1,Type=Integer,Description=&amp;quot;Count of N bases in the pileup&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=NLOD,Number=A,Type=Float,Description=&amp;quot;Normal log 10 likelihood ratio of diploid het or hom alt genotypes&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=OCM,Number=1,Type=Integer,Description=&amp;quot;Number of alt reads whose original alignment doesn't match the current contig.&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=PON,Number=0,Type=Flag,Description=&amp;quot;site found in panel of normals&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=POPAF,Number=A,Type=Float,Description=&amp;quot;negative log 10 population allele frequencies of alt alleles&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=ROQ,Number=1,Type=Float,Description=&amp;quot;Phred-scaled qualities that alt allele are not due to read orientation artifact&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=RPA,Number=R,Type=Integer,Description=&amp;quot;Number of times tandem repeat unit is repeated, for each allele (including reference)&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=RU,Number=1,Type=String,Description=&amp;quot;Tandem repeat unit (bases)&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=SEQQ,Number=1,Type=Integer,Description=&amp;quot;Phred-scaled quality that alt alleles are not sequencing errors&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=STR,Number=0,Type=Flag,Description=&amp;quot;Variant is a short tandem repeat&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=STRANDQ,Number=1,Type=Integer,Description=&amp;quot;Phred-scaled quality of strand bias artifact&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=STRQ,Number=1,Type=Integer,Description=&amp;quot;Phred-scaled quality that alt alleles in STRs are not polymerase slippage errors&amp;quot;&amp;gt;&#xA;##INFO=&amp;lt;ID=TLOD,Number=A,Type=Float,Description=&amp;quot;Log 10 likelihood ratio score of variant existing versus not existing&amp;quot;&amp;gt;&#xA;##MutectVersion=2.2&#xA;##contig=&amp;lt;ID=chr2,length=242193529&amp;gt;&#xA;##contig=&amp;lt;ID=chr20,length=64444167&amp;gt;&#xA;##contig=&amp;lt;ID=chr4,length=190214555&amp;gt;&#xA;##filtering_status=Warning: unfiltered Mutect 2 calls.  Please run FilterMutectCalls to remove false positives.&#xA;##source=Mutect2&#xA;##tumor_sample=312092_15-Ctrl4_H01_S11_R1_001.fastq.gz&#xA;#CHROM  POS ID  REF ALT QUAL    FILTER  INFO    FORMAT  312092_15-Ctrl4_H01_S11_R1_001.fastq.gz&#xA;chr20   4699818 .   G   A   .   .   AS_SB_TABLE=22,12|0,0;DP=48;ECNT=1;MBQ=11,0;MFRL=243,0;MMQ=60,60;MPOS=50;POPAF=7.30;TLOD=-1.584e+00 GT:AD:AF:DP:F1R2:F2R1:FAD:SB    0/1:34,0:0.026:34:0,0:0,0:33,0:22,12,0,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Desired output&lt;/strong&gt; (values from the first VCF above, with tabs as delimiter)&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Sample CHROM    POS ID  REF ALT QUAL    FILTER  GT INFO-DP FORMAT-DP AD-REF AD-ALT&#xA;312091_8-Ctrl5_H02_S3_R1_001.fastq.gz chr20 4699818 . G A . . 0/1 2939 2778 2778 0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I suppose merging them would be an option. Is there a method to extract the information I'd like from a merged VCF?&lt;/p&gt;&#xA;" OwnerUserId="9025" LastEditorUserId="73" LastEditDate="2024-03-06T10:50:55.783" LastActivityDate="2024-03-07T19:30:27.497" Title="How do I summarise VCF files?" Tags="|vcf|genomics|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22243" PostTypeId="2" ParentId="22242" CreationDate="2024-03-05T18:41:26.467" Score="2" Body="&lt;p&gt;The first step is to merge your VCFs. For this, you need to compress them with &lt;code&gt;bgzip&lt;/code&gt; and then index with &lt;code&gt;tabix&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;for file in *vcf; do&#xA;  bgzip &amp;quot;$file&amp;quot;&#xA;  tabix &amp;quot;$file&amp;quot;.gz&#xA;done&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Now, feed the files to &lt;code&gt;bcftools merge&lt;/code&gt;. In this example, I have saved your two files as &lt;code&gt;sample1.vcf.gz&lt;/code&gt; and &lt;code&gt;sample2.vcf.gz&lt;/code&gt;, and they're the only vcf files in the directory, so I can catch them all with &lt;code&gt;*vcf.gz&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools merge *.vcf.gz &amp;gt; merged.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This results in a file containing a single variant line, with an entry for each of the two samples:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;chr20   4699818 .   G   A   .   .   AS_SB_TABLE=1376,1402|0,0;ECNT=1;DP=2987;MBQ=11,0;MFRL=243,0;MMQ=60,60;MPOS=50;POPAF=7.3;TLOD=-1.584    GT:AD:AF:DP:F1R2:F2R1:FAD:SB    0/1:2778,0:0.00041:2778:472,0:398,0:2493,0:1376,1402,0,0    0/1:34,0:0.026:34:0,0:0,0:33,0:22,12,0,0&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;From this, &lt;code&gt;bcftools query&lt;/code&gt; can give you &lt;em&gt;almost&lt;/em&gt; exactly what you need. The main problem is that when merging, the INFO fields are not combined. Only one is kept. In my test, it was the one from the last file processed, but this could be random, I don't know. This is the closest I could get to your desired format:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ { printf 'Sample\tCHROM\tPOS\tID\tREF\tALT\tQUAL\tFILTER\tGT\tFORMAT-DP\tAD\n'; &#xA;  bcftools query --format &amp;quot;[%SAMPLE ]\t%CHROM\t%POS0\t%ID\t%REF\t%ALT\t%QUAL\t%FILTER\t[%GT ]\t[%DP ]\t[%AD ] &amp;quot; merged.vcf ; } | column -s$'\t' -t&#xA;Sample                                                                          CHROM  POS      ID  REF  ALT  QUAL  FILTER  GT          FORMAT-DP  AD&#xA;312091_8-Ctrl5_H02_S3_R1_001.fastq.gz 312092_15-Ctrl4_H01_S11_R1_001.fastq.gz   chr20  4699817  .   G    A    .     .        0/1  0/1   2778 34    2778,0 34,0  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The main command is this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;bcftools query --format &amp;quot;[%SAMPLE ]\t%CHROM\t%POS0\t%ID\t%REF\t%ALT\t%QUAL\t%FILTER\t[%GT ]\t[%DP ]\t[%AD ] &amp;quot; merged.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The rest just add a header and pretty formatting for clarity. Please see &lt;a href=&quot;https://samtools.github.io/bcftools/bcftools.html#query&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://samtools.github.io/bcftools/bcftools.html#query&lt;/a&gt; for details on the options used.&lt;/p&gt;&#xA;&lt;p&gt;This is close, but not quite. You have the sample names listed on the same line, we're missing the DP value from the FORMAT field and we have the ADs shown in pairs, not in different columns as requested. A little post-processing can fix everything except the missing DP/FORMAT:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$ { &#xA;  printf 'Sample\tCHROM\tPOS\tID\tREF\tALT\tQUAL\tFILTER\tGT\tFORMAT-DP\tAD-REF\tAD-ALT\n'; &#xA;  bcftools query --format &quot;[%SAMPLE ]\t%CHROM\t%POS0\t%ID\t%REF\t%ALT\t%QUAL\t%FILTER\t[%GT ]\t[%DP ]\t[%AD ] &quot; merged.vcf | &#xA;   awk -F'\t' -v OFS=&quot;\t&quot; '&#xA;  { &#xA;    split($1, samples, &quot; &quot;); &#xA;    split($9, genotypes, &quot; &quot;); &#xA;    split($10, dps, &quot; &quot;);&#xA;    split($11, ads, &quot; &quot;); &#xA;    for(s in samples){ &#xA;       split(ads[s], sampleAd, &quot;,&quot;);  &#xA;       print samples[s], $2, $3, $4, $5, $6, $7, $8, genotypes[s], dps[s], sampleAd[1], sampleAd[2]&#xA;     }&#xA;  }'; &#xA;} | column -s$&lt;/span&gt;'\t' -t&#xA;Sample                                   CHROM  POS      ID       REF  ALT  QUAL  FILTER  GT   FORMAT-DP  AD-REF  AD-ALT&#xA;312091_8-Ctrl5_H02_S3_R1_001.fastq.gz    chr20  4699817  .  G    A    .     .       0/1  2778       2778    0&#xA;312092_15-Ctrl4_H01_S11_R1_001.fastq.gz  chr20  4699817 .  G    A    .     .       0/1  34         34      0&#xA;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Important&lt;/strong&gt;: the command above assumes there are no spaces in your sample names nor in any of the values returned. It also assumes only one ALT allele per variant. Please update your question if these assumptions are not valid.&lt;/p&gt;&#xA;&lt;p&gt;This is as close as I can get it using &amp;quot;standard&amp;quot; tools and relatively little post-processing. It is everything you asked for except the INFO DP. If you absolutely must have that too (which seems a bit odd, honestly, you usually want the one actually reported for the variant), I fear that will require just parsing the whole thing manually.&lt;/p&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2024-03-06T14:01:34.420" LastActivityDate="2024-03-06T14:01:34.420" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22245" PostTypeId="1" CreationDate="2024-03-06T22:17:21.157" Score="-1" ViewCount="45" Body="&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/9589317/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Also posted on biostars&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I would like to extract all the mutations in a single gene of interest from the VCF file. For example I would like to extract all the mutations in the NFKB1 gene from the whole genome VCF file.&lt;/p&gt;&#xA;" OwnerUserId="20046" LastEditorUserId="650" LastEditDate="2024-03-06T22:24:41.593" LastActivityDate="2024-03-28T21:55:01.117" Title="How to extract mutations in a particular gene from the VCF file of whole genome" Tags="|vcf|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22246" PostTypeId="1" CreationDate="2024-03-06T22:40:04.453" Score="1" ViewCount="23" Body="&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/9589320/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Also posted on biostars&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;While I was searching the genomes of some eukariotic organisms (fungi,plants,protists) even though I knew that they had mitochondrial DNA,when I searched their genome structure I couldn't find any &amp;quot;chromosome&amp;quot; labeled as &amp;quot;mtDNA&amp;quot;. Here I give some links of genomes that show this:&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genome/gdv/browser/genome/?id=GCF_016861865.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/genome/gdv/browser/genome/?id=GCF_016861865.1&lt;/a&gt;&#xA;&lt;a href=&quot;https://www.ncbi.nlm.nih.gov/genome/gdv/browser/genome/?id=GCF_009556855.1&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.ncbi.nlm.nih.gov/genome/gdv/browser/genome/?id=GCF_009556855.1&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Why does this happens? How can I find if the mtDNA sequence is inluded in these &amp;quot;chromosomes&amp;quot;?&lt;/p&gt;&#xA;" OwnerUserId="17573" LastEditorUserId="73" LastEditDate="2024-03-07T11:08:05.170" LastActivityDate="2024-03-07T11:08:05.170" Title="Why is mtDNA not labelled as such for many organisms?" Tags="|genome|sequence-annotation|reference-genome|mtdna|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22247" PostTypeId="2" ParentId="22246" CreationDate="2024-03-06T23:52:24.417" Score="1" Body="&lt;p&gt;The most obvious answer is wet-lab rather than bioinformatics: mtDNA can get removed in the DNA extraction process. As a control group, in bacterial drug-resistance studies the plasmids are frequently lost. mtDNA at ~15KB is smaller than a regular bacterial plasmid.&lt;/p&gt;&#xA;&lt;p&gt;On the bioinformatics front there would be an issue about reference genome assemblies as well, i.e. if mtDNA genome isn't present then it will not appear as a separate genome.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2024-03-07T01:04:31.960" LastActivityDate="2024-03-07T01:04:31.960" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22248" PostTypeId="1" CreationDate="2024-03-07T03:03:13.060" Score="1" ViewCount="41" Body="&lt;p&gt;I have a large set of protein sequences and would like to create a &amp;quot;similarity matrix&amp;quot; out of them (for phylo trees later). So I have to run &lt;code&gt;n(n-1)&lt;/code&gt; sequence alignments. Wondering what library I should use for that and also what sophisticated lib would provide this all against all without me writing a for loop and parsing results myself...&lt;/p&gt;&#xA;&lt;p&gt;Currently using &lt;code&gt;Bio.pairwise2.align.globalxx&lt;/code&gt; from BioPython as a development placeholder but I do not think the results look really useful? It returns a &lt;code&gt;score &amp;gt; 1&lt;/code&gt; and no similarity in &lt;code&gt;[0,1[&lt;/code&gt; or &lt;code&gt;evalue&lt;/code&gt;.&lt;/p&gt;&#xA;&lt;p&gt;I have been out of the Bioinfo game for a while, so any suggestions for libraries (python preferred) much appreciated! :)&lt;/p&gt;&#xA;" OwnerUserId="17804" LastActivityDate="2024-03-07T15:21:54.207" Title="All against all sequence alignment" Tags="|sequence-alignment|multiple-sequence-alignment|biopython|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22249" PostTypeId="2" ParentId="22248" CreationDate="2024-03-07T04:52:56.783" Score="1" Body="&lt;p&gt;I wouldn't use Python here. I've looked at using &lt;code&gt;Bio.pairwise2.align.&lt;/code&gt; and its has major shortcomings. In this context the problem is establishing homologous alignment positions ... if you want to make a tree.&lt;/p&gt;&#xA;&lt;p&gt;Whats the problems? This is you start introducing the gap character '-' as an additional state in constructing your tree and the tree will bias towards not the maximum number of substitutions but the maximum gap (indel) difference between the sequences.&lt;/p&gt;&#xA;&lt;p&gt;The second problem is changes in homologous sites between different pairwise alignments. This gets complicated fast.&lt;/p&gt;&#xA;&lt;p&gt;Is that bad? The problem is we have little understanding of the mutation model behind generating indels and there isn't a model to account for varying descriptions of homologous alignment positions. However, we have a very clear understanding of amino acid substitutions in a single global alignment, i.e. a singular description of homologous amino acid positions. A lot of work has gone into modelling amino acid mutation, with a tree a practical output of that work.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Thus, I would simply align using muscle5 under the &lt;strong&gt;-super5&lt;/strong&gt; option. This will produce a global alignment very quickly and for &lt;em&gt;vast&lt;/em&gt; amounts of data. This alignment will produce the best model of homologous amino acid sites.&lt;/p&gt;&#xA;&lt;p&gt;If you just want a distance matrix then import the entire alignment into MEGAXI and request the JTT model of amino acid substitution with the gamma distribution. This will output a distance matrix nicely and will then allow you to produce a tree.&lt;/p&gt;&#xA;&lt;p&gt;BTW estimating the gamma distribution without maximum likelihood can be difficult in distance methods (MEGA might have resolved this). This tree will be much better than the one you are proposing (you can estimate this parameter via e.g. IQTREE or ModelTest).&lt;/p&gt;&#xA;&lt;p&gt;Why? Again alot of work has been done on modelling amino acid substations and JTT is one of the better models (assuming this is nuclear sequence not mtDNA). The LG matrix is much better BTW, but I don't think thats in MEGA.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2024-03-07T15:21:54.207" LastActivityDate="2024-03-07T15:21:54.207" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22250" PostTypeId="1" CreationDate="2024-03-07T05:58:44.013" Score="0" ViewCount="16" Body="&lt;p&gt;&lt;a href=&quot;https://www.biostars.org/p/9589316/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Also asked on biostars&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I am looking for tools and software that could help me visualize moving cell compartments in 3D where I can go from the cell membrane to the ER, then out with the protein into the matrix. In short a 3D visualization of the cell environment.&lt;/p&gt;&#xA;&lt;p&gt;Since I am a newbie, I would appreciate any tool/software suggestion for me to try.&lt;/p&gt;&#xA;" OwnerUserId="20046" LastEditorUserId="650" LastEditDate="2024-03-07T15:21:41.107" LastActivityDate="2024-03-07T15:21:41.107" Title="Tools and software to visualize the moving cell compartments in 3D" Tags="|protein-structure|3d-structure|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22251" PostTypeId="1" CreationDate="2024-03-07T09:00:14.003" Score="2" ViewCount="61" Body="&lt;p&gt;I have multiple SRR files to download, and their accession is in a list:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SRR9004325&#xA;SRR9004326&#xA;SRR9004327&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I read the &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/sra/docs/sradownload/&quot; rel=&quot;nofollow noreferrer&quot;&gt;NCBI document&lt;/a&gt; but don't find how to achieve this using the command &amp;quot;fastq-dump&amp;quot;.&lt;/p&gt;&#xA;&lt;p&gt;And I noticed that if I use the command &lt;code&gt;prefetch&lt;/code&gt;, I can download from the list file:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;prefetch --option-file SraAccList.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;So is there any parameter in &amp;quot;fastq-dump&amp;quot; to download accessions stored in a file?&lt;/p&gt;&#xA;&lt;p&gt;As @terdon said, it is better to use &lt;code&gt;fasterq-dump&lt;/code&gt;, and the command:&#xA;&lt;code&gt;fasterq-dump --option-file SraAccList.txt&lt;/code&gt; doesn't work. When I try it, I just get the usage message:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ fasterq-dump --option-file SraAccList.txt&#xA;&#xA;Usage:&#xA;  fasterq-dump &amp;lt;path&amp;gt; [options]&#xA;  fasterq-dump &amp;lt;accession&amp;gt; [options]&#xA;&#xA;Options:&#xA;  -F|--format                      format (special, fastq, default=fastq) &#xA;  -o|--outfile                     output-file &#xA;  -O|--outdir                      output-dir &#xA;  -b|--bufsize                     size of file-buffer dflt=1MB &#xA;  -c|--curcache                    size of cursor-cache dflt=10MB &#xA;  -m|--mem                         memory limit for sorting dflt=100MB &#xA;  -t|--temp                        where to put temp. files dflt=curr dir &#xA;  -e|--threads                     how many thread dflt=6 &#xA;  -p|--progress                    show progress &#xA;  -x|--details                     print details &#xA;  -s|--split-spot                  split spots into reads &#xA;  -S|--split-files                 write reads into different files &#xA;  -3|--split-3                     writes single reads in special file &#xA;  --concatenate-reads              writes whole spots into one file &#xA;  -Z|--stdout                      print output to stdout &#xA;  -f|--force                       force to overwrite existing file(s) &#xA;  --skip-technical                 skip technical reads &#xA;  --include-technical              include technical reads &#xA;  -M|--min-read-len                filter by sequence-len &#xA;  --table                          which seq-table to use in case of pacbio &#xA;  -B|--bases                       filter by bases &#xA;  -A|--append                      append to output-file &#xA;  --fasta                          produce FASTA output &#xA;  --fasta-unsorted                 produce FASTA output, unsorted &#xA;  --fasta-ref-tbl                  produce FASTA output from REFERENCE tbl &#xA;  --fasta-concat-all               concatenate all rows and produce FASTA &#xA;  --internal-ref                   extract only internal REFERENCEs &#xA;  --external-ref                   extract only external REFERENCEs &#xA;  --ref-name                       extract only these REFERENCEs &#xA;  --ref-report                     enumerate references &#xA;  --use-name                       print name instead of seq-id &#xA;  --seq-defline                    custom defline for sequence:  &lt;span class=&quot;math-container&quot;&gt;$ac=accession, &#xA;                               $&lt;/span&gt;sn=spot-name,  &lt;span class=&quot;math-container&quot;&gt;$sg=spot-group, $&lt;/span&gt;si=spot-id,  &#xA;                                   &lt;span class=&quot;math-container&quot;&gt;$ri=read-id, $&lt;/span&gt;rl=read-length &#xA;  --qual-defline                   custom defline for qualities:  same as &#xA;                                   seq-defline &#xA;  -U|--only-unaligned              process only unaligned reads &#xA;  -a|--only-aligned                process only aligned reads &#xA;  --disk-limit                     explicitly set disk-limit &#xA;  --disk-limit-tmp                 explicitly set disk-limit for temp. files &#xA;  --size-check                     switch to control: on=perform size-check &#xA;                                   (default),  off=do not perform size-check,  &#xA;                                   only=perform size-check only &#xA;  --ngc &amp;lt;PATH&amp;gt;                     PATH to ngc file &#xA;&#xA;  -h|--help                        Output brief explanation for the program. &#xA;  -V|--version                     Display the version of the program then &#xA;                                   quit. &#xA;  -L|--log-level &amp;lt;level&amp;gt;           Logging level as number or enum string. One &#xA;                                   of (fatal|sys|int|err|warn|info|debug) or &#xA;                                   (0-6) Current/default is warn. &#xA;  -v|--verbose                     Increase the verbosity of the program &#xA;                                   status messages. Use multiple times for more &#xA;                                   verbosity. Negates quiet. &#xA;  -q|--quiet                       Turn off all status messages for the &#xA;                                   program. Negated by verbose. &#xA;  --option-file &amp;lt;file&amp;gt;             Read more options and parameters from the &#xA;                                   file. &#xA;for more information visit:&#xA;   https://github.com/ncbi/sra-tools/wiki/HowTo:-fasterq-dump&#xA;   https://github.com/ncbi/sra-tools/wiki/08.-prefetch-and-fasterq-dump&#xA;&#xA;fasterq-dump : 3.0.6&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="17928" LastEditorUserId="298" LastEditDate="2024-03-07T16:39:38.790" LastActivityDate="2024-03-07T16:39:38.790" Title="How can I download multiple SRR* files at a time when using sratoolkit's fastq-dump?" Tags="|data-download|ncbi|" AnswerCount="1" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22252" PostTypeId="2" ParentId="22251" CreationDate="2024-03-07T16:34:07.847" Score="2" Body="&lt;p&gt;If &lt;code&gt;fasterq-dump&lt;/code&gt; can't read from a file, you can simply loop over the file yourself:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;while read srr; do&#xA;  fasterq-dump &amp;quot;$srr&amp;quot; -o &amp;quot;$srr&amp;quot;.fastq&#xA;done &amp;lt; SraAccList.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;That will iterate over your &lt;code&gt;SraAccList.txt&lt;/code&gt; file and extract the fastqs for each accession. I tested with this command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;fasterq-dump SRR9004325 -o ff.fastq&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;And that created files &lt;code&gt;ff_1.fastq&lt;/code&gt; and &lt;code&gt;ff_2.fastq&lt;/code&gt;, so you can expect files named like &lt;code&gt;SRR9004325_1.fastq&lt;/code&gt; and &lt;code&gt;SRR9004325_1.fastq&lt;/code&gt; if you use the loop above.&lt;/p&gt;&#xA;&lt;p&gt;You could try and parallelize it but I doubt it is worth it since the bottleneck will always be the download speed.&lt;/p&gt;&#xA;" OwnerUserId="298" LastActivityDate="2024-03-07T16:34:07.847" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22253" PostTypeId="2" ParentId="22242" CreationDate="2024-03-07T19:30:27.497" Score="-1" Body="&lt;p&gt;The simplest alternative is to have a directory with all the vcf files you want to merge, and use bcftools merge. Example:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;vcf_dir=path_to_your_vcf_directory&#xA;bcftools merge $vcf_dir/*.vcf -o bcftools_merged.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="16686" LastActivityDate="2024-03-07T19:30:27.497" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22254" PostTypeId="1" CreationDate="2024-03-08T05:00:00.833" Score="1" ViewCount="14" Body="&lt;p&gt;In our recent research, we delved into the amino acid composition of protein sequences by applying n-gram analysis techniques. By examining the distribution of n-grams ranging from 1-gram to 11-gram, we aimed to discern underlying patterns that could be indicative of structural or functional significance.&lt;/p&gt;&#xA;&lt;p&gt;We started with an exploration of 1-grams to understand the basic composition of our protein dataset, followed by investigating the prevalence and distribution of larger n-grams. Our analysis included evaluating the fit of these distributions against well-known statistical laws—namely, Benford's, Pareto, and Zipf's.&lt;/p&gt;&#xA;&lt;p&gt;An intriguing pattern emerged as n-gram lengths increased. The frequency distributions began to approximate Benford's Law more closely, particularly at tetragram and pentagram levels. Beyond pentagrams, the distributions deviated, suggesting a higher level of sequence diversity and complexity.&lt;/p&gt;&#xA;&lt;p&gt;Furthermore, we assessed how these patterns aligned with the Pareto principle. While we found that the data did not strictly adhere to the &amp;quot;80/20 rule&amp;quot;, there was an interesting variation in the concentration of occurrences across different n-gram lengths.&lt;/p&gt;&#xA;&lt;p&gt;Our approach also involved identifying sequences with anomalies such as the presence of consecutive 'X' characters, which denotes unknown or unspecified amino acids, and sequences that were unusually short or long compared to the general protein population.&lt;/p&gt;&#xA;&lt;p&gt;As we continue to dissect these patterns, several questions arise, inviting further scrutiny and discussion within the scientific community:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;What biological insights can be inferred from the observed fit of amino acid patterns to Benford's Law, particularly for tetragrams and pentagrams?&lt;/li&gt;&#xA;&lt;li&gt;How might the deviations in longer n-grams inform our understanding of protein complexity and functionality?&lt;/li&gt;&#xA;&lt;li&gt;In what ways can the prevalence of specific n-grams guide the development of more accurate predictive models for protein function?&lt;/li&gt;&#xA;&lt;li&gt;What are the implications of the identified sequence anomalies for data quality and sequence annotation in protein databases?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;We are eager to engage with the bioinformatics community to explore these questions and welcome any insights or collaborative ideas that can drive this research forward.&lt;/p&gt;&#xA;&lt;p&gt;Our public notebook &lt;a href=&quot;https://www.kaggle.com/code/aakras/testing-pareto-benford-and-zipf&quot; rel=&quot;nofollow noreferrer&quot;&gt;Testing Pareto, Benford and Zipf&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20052" LastActivityDate="2024-03-08T05:00:00.833" Title="Exploring Amino Acid Patterns in Proteins Through N-gram Analysis" Tags="|proteins|sequence-analysis|amino-acids|" AnswerCount="0" CommentCount="1" ClosedDate="2024-03-08T17:26:02.460" ContentLicense="CC BY-SA 4.0" />
  <row Id="22255" PostTypeId="1" AcceptedAnswerId="22256" CreationDate="2024-03-08T13:01:05.030" Score="2" ViewCount="54" Body="&lt;p&gt;I'm stuck on what seems to be a banal problem, but I can't work out where the issue is. I need to create some customised VCF files that specify particular variants of interest.&lt;/p&gt;&#xA;&lt;p&gt;This is a custom VCF I created for this &lt;a href=&quot;http://www.ensembl.org/Homo_sapiens/Variation/Explore?r=20:4699252-4700252;v=rs74315403;vdb=variation;vf=1054935347&quot; rel=&quot;nofollow noreferrer&quot;&gt;variant&lt;/a&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;##fileformat=VCFv4.3&#xA;##FILTER=&amp;lt;ID=PASS,Description=&amp;quot;All filters passed&amp;quot;&amp;gt;&#xA;##fileDate=20240308&#xA;##source=Ensembl&#xA;##reference=GRCh38&#xA;#CHROM  POS ID  REF ALT QUAL  FILTER    INFO&#xA;20  4699752 rs74315403  G   A   .   .  .&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;The formatting seems correct to me; the values are separated by tabs, all relevant column headers are present, there is a value for every header. I try to use it in the Mutect2 &lt;code&gt;--alleles&lt;/code&gt; and &lt;code&gt;-L&lt;/code&gt; options, as follows:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gatk Mutect2 \&#xA;  -R ref.fa \&#xA;  --alleles my.vcf.gz  \&#xA;  -L my.vcf.gz\&#xA;  -I input.bam \&#xA;  -O output.vcf&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;This gives me the following error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;Caused by: htsjdk.tribble.TribbleException&lt;span class=&quot;math-container&quot;&gt;$MalformedFeatureFile: Unable to parse header with error: Your input file has a malformed header: there are not enough columns present in the header line: #CHROM     POS     ID      REF     ALT     QUAL  FILTER    INFO, for input source: path/to/my.vcf&#xA;        at htsjdk.tribble.TribbleIndexedFeatureReader.readHeader(TribbleIndexedFeatureReader.java:265)&#xA;        at htsjdk.tribble.TribbleIndexedFeatureReader.&amp;lt;init&amp;gt;(TribbleIndexedFeatureReader.java:104)&#xA;        at htsjdk.tribble.TribbleIndexedFeatureReader.&amp;lt;init&amp;gt;(TribbleIndexedFeatureReader.java:129)&#xA;        at htsjdk.tribble.AbstractFeatureReader.getFeatureReader(AbstractFeatureReader.java:121)&#xA;        at org.broadinstitute.hellbender.engine.FeatureDataSource.getTribbleFeatureReader(FeatureDataSource.java:433)&#xA;        ... 15 more&#xA;Caused by: htsjdk.tribble.TribbleException$&lt;/span&gt;InvalidHeader: Your input file has a malformed header: there are not enough columns present in the header line: #CHROM       POS     ID      REF    ALT      QUAL  FILTER    INFO&#xA;        at htsjdk.variant.vcf.AbstractVCFCodec.parseHeaderFromLines(AbstractVCFCodec.java:149)&#xA;        at htsjdk.variant.vcf.VCFCodec.readActualHeader(VCFCodec.java:117)&#xA;        at htsjdk.tribble.AsciiFeatureCodec.readHeader(AsciiFeatureCodec.java:79)&#xA;        at htsjdk.tribble.AsciiFeatureCodec.readHeader(AsciiFeatureCodec.java:37)&#xA;        at htsjdk.tribble.TribbleIndexedFeatureReader.readHeader(TribbleIndexedFeatureReader.java:263)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Supposedly there's something wrong with the header. Can you spot the issue? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="9025" LastActivityDate="2024-03-09T11:58:09.213" Title="Creating custom VCF file" Tags="|vcf|genomics|mutect2|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22256" PostTypeId="2" ParentId="22255" CreationDate="2024-03-08T14:44:24.027" Score="3" Body="&lt;p&gt;That sounds like your header isn't tab separated. You can check for the presence of tabs with &lt;code&gt;od -c&lt;/code&gt;. So, to check the header, do&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;zgrep '^#C' my.vcf.gz | od -c&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then check the output, it should show &lt;code&gt;\t&lt;/code&gt; between each header field, like this:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ zgrep '^#C'  variants.vcf.gz | od -c&#xA;0000000   #   C   H   R   O   M  \t   P   O   S  \t   I   D  \t   R   E&#xA;0000020   F  \t   A   L   T  \t   Q   U   A   L  \t   F   I   L   T   E&#xA;0000040   R  \t   I   N   F   O  \t   F   O   R   M   A   T  \t   E   D&#xA;0000060   2   0   0   1   3   9   6  \n&#xA;0000070&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="298" LastEditorUserId="298" LastEditDate="2024-03-09T11:58:09.213" LastActivityDate="2024-03-09T11:58:09.213" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22257" PostTypeId="1" CreationDate="2024-03-09T20:28:36.567" Score="0" ViewCount="8" Body="&lt;p&gt;I am working on a personal project for a better, more informed management of my own Diabetes Mellitus, Type 2 (aka &amp;quot;T2DM&amp;quot;, or &amp;quot;T2D&amp;quot;). With some difficulty I am able to access and parse my Interstitial Glucose Level (&amp;quot;IGL&amp;quot;) from my Continuous Glucose Monitor (&amp;quot;CGM&amp;quot;), and import the minute-by-minute data into a spreadsheet. I would like to be able to compare my IGL results to the activity profiles of both my prescribed insulin and my prescribed oral diabetes medications. I anticipate that such a comparison of the IGL and activity profiles of the treatments should give me an insight into the best times for administering my treatments. Later I will be adding further layers of comparison to this, such as my blood pressure, pulse, blood oxygen SpO₂ level, carbohydrate intake, exercise, etc to get a statistical view of which of these factors influence the IGL numbers the most, over time.&lt;/p&gt;&#xA;&lt;p&gt;What I need first to be able to do all of this is a copy of the dataset for the activity profile of the insulin that I have been prescribed, the Lantus SoloStar 100ɪᴜ/mL pen. I have seen the graph of this data available online quite often, but I need to get the actual figures that the graphs are based on, in order to use them in my own modelling. If anyone knows where I might obtain this information, as a non-clinician myself, please let me know. Similarly, for a source of such activity profile data for common diabetes drugs such as Metformin and Gliclazide.&lt;/p&gt;&#xA;&lt;p&gt;Also, if anyone knows of a better way to parse this information, including the raw data from the Abbott Freestyle Libre 2 with LibreView, please do let me know. Thanks.&lt;/p&gt;&#xA;" OwnerUserId="20057" LastActivityDate="2024-03-09T20:28:36.567" Title="Where would I get the activity profile dataset for Lantus Solostar insulin, and for other diabetes treatments such as Metformin?" Tags="|public-databases|visualization|data-download|modelling|public-dataset|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22258" PostTypeId="1" CreationDate="2024-03-11T12:47:51.093" Score="1" ViewCount="17" Body="&lt;p&gt;I have some doubts in how to proceed to compare ancient mtDNA with modern mtDNA. A little more context:&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;The problem:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have some FASTQs associated with a given FASTA. They contain mtDNA. My goal is to extract the mtDNA from those FASTQs and compare with a reference mtDNA to calculate the average and SD of that difference.&#xA;I currently download the samples FASTA and FASTQs from AMTDB (&lt;a href=&quot;https://amtdb.org/samples&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://amtdb.org/samples&lt;/a&gt;).&#xA;My final goal is to compare ancient mtDNA with modern mtDNA to analyze the deterioration based on the degradation patterns.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;What I'm lacking in knowledge:&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;Does it make sense if I align the FASTQ with the corresponded downloaded FASTA? I mean, if the FASTA was already generated with those reads, wouldn't that be redundant?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Could I use an arbitrary mtDNA FASTA to align and compare? That would assume mtDNA in humans is very similar (I have doubts if that would be a safe assumption, since my goal is to compare those ancient samples with a modern and not damaged sample)&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;In any case, is there a recommended program to compare the results of the processing of the FASTQs with the reference FASTA?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Thank you for your help.&lt;/p&gt;&#xA;" OwnerUserId="18981" LastActivityDate="2024-04-03T12:22:08.867" Title="Compare FASTQ reconstructions to mtDNA" Tags="|fasta|fastq|mtdna|" AnswerCount="1" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22259" PostTypeId="1" CreationDate="2024-03-11T13:56:32.500" Score="2" ViewCount="14" Body="&lt;p&gt;I made a phylogenetic tree for about 300 species using a region of the COX1 gene. The tree looks visually meaningful: species of the same genus are all clustered together.&lt;/p&gt;&#xA;&lt;p&gt;Then I performed a 350-replicate bootstrapping and I analyzed the support values. Surprisingly I found very low values for nodes that are more close to the root and high support values for nodes that are more close to the leaves.&lt;/p&gt;&#xA;&lt;p&gt;Almost all nodes that split into 2 leaves have Bootstrap &amp;gt;95% but the ones close to the root have Bootstrap&amp;lt;20%.&lt;/p&gt;&#xA;&lt;p&gt;Does that mean I chose a highly variable DNA region?&lt;br /&gt;&#xA;How would you re-build the tree to make it more consistent on more ancient nodes? I was thinking of using a concatenation of sequences with both high and low variability regions, but i am not sure it would work.&lt;/p&gt;&#xA;" OwnerUserId="18351" LastActivityDate="2024-04-02T04:37:45.197" Title="How to interpret high support values for terminal nodes and low for more ancient nodes?" Tags="|phylogenetics|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22261" PostTypeId="1" AcceptedAnswerId="22262" CreationDate="2024-03-11T23:32:30.280" Score="1" ViewCount="25" Body="&lt;p&gt;I was able to download all of the SRA files for a specific bioproject using the command:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;prefetch -O output_directory --option-file SRR_Acc_List.txt&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then tried the following command to download those SRA files into FASTQ files:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;find -name \*.sra -exec bsub fastq-dump --split-3 --gzip {} \;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It gave the error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;find: illegal option -- n&#xA;usage: find [-H | -L | -P] [-EXdsx] [-f path] path ... [expression]&#xA;       find [-H | -L | -P] [-EXdsx] -f path [path ...] [expression]&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then tried to add the path:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;find /Users/lucykelly/Desktop/Assel_paper/output_directory -name '*.sra' -exec bsub fastq-dump --split-3 --gzip {} \;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;It gave the error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;find: bsub: No such file or directory&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I then tried&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;find . -name '*.sra' -exec fasterq-dump --gzip {} \;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;and it gave the error:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;find: fasterq-dump: No such file or directory&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="20066" LastEditorUserId="73" LastEditDate="2024-03-12T11:12:09.667" LastActivityDate="2024-03-12T11:12:09.667" Title="Downloading multiple SRA files as FASTQ files using SRA Toolkit" Tags="|fastq|sratoolkit|sra|fastq-dump|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22262" PostTypeId="2" ParentId="22261" CreationDate="2024-03-12T08:33:48.343" Score="1" Body="&lt;p&gt;That is a classical problem where the folder containing the binary of the tool is not in PATH, see for example &lt;a href=&quot;https://askubuntu.com/questions/109381/how-to-add-path-of-a-program-to-path-environment-variable&quot;&gt;https://askubuntu.com/questions/109381/how-to-add-path-of-a-program-to-path-environment-variable&lt;/a&gt; or many other threads towards PATH in the SE network.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2024-03-12T08:33:48.343" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22263" PostTypeId="1" CreationDate="2024-03-12T08:50:49.610" Score="1" ViewCount="29" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.biostars.org/p/9589012/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Biostars&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I received a set of Nanopore long reads of a ~3kbp long known plasmid with an unknown insert between 1kbp and 7kbp. Could you suggest a pipeline to assemble the reads and find the insertion?&lt;/p&gt;&#xA;" OwnerUserId="6935" LastEditorUserId="73" LastEditDate="2024-03-12T10:49:19.253" LastActivityDate="2024-03-12T11:07:46.330" Title="Finding insertion in a plasmid" Tags="|nanopore|long-reads|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22264" PostTypeId="1" CreationDate="2024-03-12T10:25:27.770" Score="2" ViewCount="37" Body="&lt;p&gt;I am currently working on analyzing protein conservation across amino acid positions using Trident scores as output from the MstatX tool. I have 40,000 Trident score files, each containing amino acid positions and their respective scores.&lt;/p&gt;&#xA;&lt;p&gt;To streamline my analysis, I initially wrote a bash script to loop through each file and then implemented an R script for the calculations. The R script calculates moving averages with different spans to identify regions with high deviation in local protein conservation, referred to as &amp;quot;hotspots.&amp;quot; These hotspots are then analyzed to determine the strongest deviation, its position, and related metrics.&lt;/p&gt;&#xA;&lt;p&gt;Here is a snippet of my R script:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;# assign input files to a vector &#xA;args = commandArgs(trailingOnly = T)&#xA;&#xA;# put sample name in variable&#xA;sample&amp;lt;-sprintf(&amp;quot;%s&amp;quot;, args[1])&#xA;&#xA;# load sample and make data frame&#xA;raw.entropy&amp;lt;-read.table(sprintf(&amp;quot;%s&amp;quot;, args[1]))&#xA;col.names&amp;lt;-c(&amp;quot;AA.pos&amp;quot;, &amp;quot;Entropy.Score&amp;quot;)&#xA;colnames(raw.entropy)&amp;lt;-col.names&#xA;AA.pos&amp;lt;-raw.entropy&lt;span class=&quot;math-container&quot;&gt;$AA.pos&#xA;Entropy.Score&amp;lt;-raw.entropy$&lt;/span&gt;Entropy.Score&#xA;&#xA;# make sample name nicer&#xA;sub(&amp;quot;_entropyscores&amp;quot;, &amp;quot;&amp;quot;, sample)-&amp;gt;sample&#xA;&#xA;# load ggplot&#xA;library(ggplot2)&#xA;&#xA;# calculate short and long moving averages&#xA;mydata&amp;lt;-ggplot(data=raw.entropy, aes(x=AA.pos)) +&#xA;  geom_point(aes(y=Entropy.Score), size=2, color=&amp;quot;black&amp;quot;) +&#xA;  geom_smooth(mapping=aes(y=Entropy.Score), method = &amp;quot;loess&amp;quot;, color = &amp;quot;red&amp;quot;, span = 0.1) +&#xA;  geom_smooth(mapping=aes(y=Entropy.Score), method = &amp;quot;loess&amp;quot;, color = &amp;quot;blue&amp;quot;, span = 0.4) +&#xA;  theme_classic() +&#xA;  theme(plot.title = element_text(hjust = 0.5, face = &amp;quot;bold&amp;quot;)) +&#xA;  labs(title = sprintf(&amp;quot;%s&amp;quot;, args[1]), x = &amp;quot;Amino Acid Positions&amp;quot;, y = &amp;quot;Trident Score&amp;quot;)&#xA;&#xA;# infer ratios between short and long span moving averages&#xA;# ratios represent extent to which local conservation is adhered to&#xA;# ratio around 1 means perfect conservation, ratio around 0 means poor conservation&#xA;rolling_mean_10&amp;lt;-layer_data(mydata, 2)&lt;span class=&quot;math-container&quot;&gt;$y&#xA;rolling_mean_50&amp;lt;-layer_data(mydata, 3)$&lt;/span&gt;y&#xA;ratios&amp;lt;-rolling_mean_10/rolling_mean_50&#xA;&#xA;# find index value of 10th percentile&#xA;lowerbound&amp;lt;-NULL&#xA;for (i in (0.1*length(rolling_mean_10)):length(rolling_mean_10)){&#xA;if (rolling_mean_10[i] &amp;gt;= 0.9*median(raw.entropy$Entropy.Score)){&#xA;lowerbound&amp;lt;-i&#xA;break&#xA;}&#xA;}&#xA;&#xA;# find index value of 90th percentile&#xA;upperbound&amp;lt;-NULL&#xA;for (i in (0.9*length(rolling_mean_10)):1){&#xA;if (rolling_mean_10[i] &amp;gt;= 0.9*median(raw.entropy$Entropy.Score)){&#xA;upperbound&amp;lt;-i&#xA;break&#xA;}&#xA;}&#xA;&#xA;# find location in sequence with highest deviation in local protein conservation&#xA;which(ratios==min(ratios[lowerbound:upperbound]))-&amp;gt;hotspot&#xA;ratios[hotspot]-&amp;gt;cutoff&#xA;layer_data(mydata, 3)&lt;span class=&quot;math-container&quot;&gt;$x[hotspot]-&amp;gt;x&#xA;percentile&amp;lt;-x/length(raw.entropy$&lt;/span&gt;AA.pos)*100&#xA;&#xA;# write relevant items to standard output&#xA;# order is cluster name, strongest deviation, rounded amino acid position, percentile where deviation occurred&#xA;cat(sprintf(&amp;quot;%s %s %s %s %s\n&amp;quot;, sample, cutoff, round(x), round(percentile, 2)), file=&amp;quot;./output_deviation.txt&amp;quot;, append=TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;After writing the output file, I aim to create a density plot with the x-axis representing the strongest deviation of each input file. However, the current method does not capture the expected bimodal distribution. I am seeking advice on alternative ways to calculate small but strong deviations beyond using simple moving averages.&lt;/p&gt;&#xA;&lt;p&gt;Some options I've explored include EMA, z-scores, step detection, and anomaly detection. I would greatly appreciate any insights or suggestions.&lt;/p&gt;&#xA;" OwnerUserId="20069" LastEditorUserId="650" LastEditDate="2024-03-12T19:57:17.210" LastActivityDate="2024-03-12T19:57:17.210" Title="Protein conservation analysis with trident scores in R script" Tags="|r|proteins|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22265" PostTypeId="2" ParentId="22263" CreationDate="2024-03-12T10:59:34.430" Score="2" Body="&lt;p&gt;When I've needed to do this, I use canu to assemble the nanopore reads (choosing the highest-quality reads that are at least half the length of the target), then map the assembled consensus back to the reference sequence.&lt;/p&gt;&#xA;&lt;p&gt;The Canu assembly part of that is described here:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.protocols.io/view/plasmid-sequence-analysis-from-long-reads-36wgq4n5yvk5?step=12&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.protocols.io/view/plasmid-sequence-analysis-from-long-reads-36wgq4n5yvk5?step=12&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;There are numerous different ways to map the consensus assembled sequence to the reference sequence. &lt;a href=&quot;https://blast.ncbi.nlm.nih.gov/Blast.cgi?PROGRAM=blastn&amp;amp;PAGE_TYPE=BlastSearch&amp;amp;LINK_LOC=blasthome&quot; rel=&quot;nofollow noreferrer&quot;&gt;BLASTn&lt;/a&gt; in &amp;quot;align 2 or more sequences&amp;quot; mode could be used, for example. Another option, because the sequences are relatively small and you're just aligning 2, is &lt;a href=&quot;https://www.ebi.ac.uk/jdispatcher/psa/emboss_needle&quot; rel=&quot;nofollow noreferrer&quot;&gt;needle&lt;/a&gt; from the EMBOSS suite.&lt;/p&gt;&#xA;" OwnerUserId="73" LastEditorUserId="73" LastEditDate="2024-03-12T11:07:46.330" LastActivityDate="2024-03-12T11:07:46.330" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22266" PostTypeId="1" AcceptedAnswerId="22267" CreationDate="2024-03-12T19:02:29.887" Score="1" ViewCount="29" Body="&lt;p&gt;I am struggling to get my head around the correct design to use for my experiment.&#xA;I have RNAseq of 2 pure species (s1, s2) and of the hybrid (h). I aligned all three genotypes to the s1 genomes. I am interested in the comparison of hybrids vs s1 (maternal genome). But I want to remove a potential bias from reads from s2 chromosome being harder to align to s1. (The hybrid genome is composed of both s1 and s2.) I would like to correct my DE test by removing potential effect s2 vs s1. (I aligned s2 to s1 and got the count.)&#xA;But my issue is, I fail to have a full rank design.&#xA;I do not have any strong feelings about the library (deseq2, edgeR...).&lt;/p&gt;&#xA;&lt;p&gt;Following the tutorial&lt;/p&gt;&#xA;&lt;p&gt;below is the condition matrix, as you can see if I want to make the contrast ~ s2 + condition. The matrix is not full rank.&#xA;I believe I could write my design as ~s2 + h - s1 ?? But I don't know how to use deseq2 for this approach.&lt;/p&gt;&#xA;&lt;p&gt;Any approach/tips to achieve my goal here? Is it event possible? what&lt;/p&gt;&#xA;&lt;div class=&quot;s-table-container&quot;&gt;&lt;table class=&quot;s-table&quot;&gt;&#xA;&lt;thead&gt;&#xA;&lt;tr&gt;&#xA;&lt;th&gt;sample&lt;/th&gt;&#xA;&lt;th&gt;condition&lt;/th&gt;&#xA;&lt;th&gt;s2&lt;/th&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/thead&gt;&#xA;&lt;tbody&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;td&gt;s1&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;2&lt;/td&gt;&#xA;&lt;td&gt;s1&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;3&lt;/td&gt;&#xA;&lt;td&gt;s1&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;4&lt;/td&gt;&#xA;&lt;td&gt;s2&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;5&lt;/td&gt;&#xA;&lt;td&gt;s2&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;6&lt;/td&gt;&#xA;&lt;td&gt;s2&lt;/td&gt;&#xA;&lt;td&gt;1&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;7&lt;/td&gt;&#xA;&lt;td&gt;h&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;8&lt;/td&gt;&#xA;&lt;td&gt;h&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;tr&gt;&#xA;&lt;td&gt;9&lt;/td&gt;&#xA;&lt;td&gt;h&lt;/td&gt;&#xA;&lt;td&gt;0&lt;/td&gt;&#xA;&lt;/tr&gt;&#xA;&lt;/tbody&gt;&#xA;&lt;/table&gt;&lt;/div&gt;&#xA;&lt;p&gt;any help is warmly appreciated!&lt;/p&gt;&#xA;" OwnerUserId="6655" LastActivityDate="2024-03-12T23:39:10.087" Title="rna seq matrix design hybrids" Tags="|rna-seq|deseq2|edger|design|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22267" PostTypeId="2" ParentId="22266" CreationDate="2024-03-12T23:39:10.087" Score="1" Body="&lt;p&gt;I think you are over thinking the problem. You need to classify the genes of interest and to do that a classifier such as eggNOG is used. This will remove any bias. I understand eggNOG because it's based on protein phylogenetics and phylogenetics is what I do and I can assure you its very solid and the algorithm nicely constructed.&lt;/p&gt;&#xA;&lt;p&gt;DESeq2 (I ain't a huge fan BTW, but avoid controversy) will happily accept eggNOG classification, &lt;a href=&quot;https://microscope.readthedocs.io/en/3.14.3/content/transcriptomic/NGSDiffExprAnalysis.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;here&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Then its simply three tests&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;S1 vs S2&lt;/li&gt;&#xA;&lt;li&gt;S1 vs hybrid&lt;/li&gt;&#xA;&lt;li&gt;S2 vs hybrid&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I personally would perform unsupervised learning, particularly via &lt;a href=&quot;https://scanpy.readthedocs.io/en/stable/&quot; rel=&quot;nofollow noreferrer&quot;&gt;Scanpy&lt;/a&gt; first on S1, S2 and hybrid separately. This will give a rough indication of the key difference of expression profiles between S1 and S2 and see where the hydrid occurs.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2024-03-12T23:39:10.087" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22268" PostTypeId="1" CreationDate="2024-03-13T02:24:52.230" Score="1" ViewCount="32" Body="&lt;p&gt;I want to generate synthetic miRNA data for the performance assessment of the newly developed algorithm designed for miRNA data analysis. There are several synthetic sequence simulators available in the published literature, such as wgsim, dwgsim, ART, inSillicoseq, mason,  NEAT, BEER, polyester, RSEM, SimSeq, etc. All of these tools are either for DNA or transcriptome profiles. Can anybody please suggest the following:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Can synthetic DNA sequence simulators (ART, wgsim, dwgsim) generate synthetic miRNA data?&lt;/li&gt;&#xA;&lt;li&gt;Some tools, such as CAMPAREE, RSEM, and RNA-Seq simulator, generate RNA-seq data, but I think these tools generate mRNA data, and I am unsure whether I can use them for my miRNA data analysis.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Please suggest. I very much appreciate any help in this regard. Thanks&lt;/p&gt;&#xA;" OwnerUserId="2141" LastActivityDate="2024-03-22T11:52:28.047" Title="Generating synthetic RNA-Seq data for performance assessment of new algorithm" Tags="|rna-seq|simulated-data|dna|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22269" PostTypeId="1" CreationDate="2024-03-13T02:25:25.043" Score="1" ViewCount="20" Body="&lt;p&gt;I am working on a novel protein secondary structure detection algorithm using machine learning.&lt;/p&gt;&#xA;&lt;p&gt;As far as I understand, online protein databanks (e.g., RCSB, etc.) have a huge collection of proteins (several millions). All proteins are not useful for research purposes as they contain redundant, duplicate, or incomplete information.&lt;/p&gt;&#xA;&lt;p&gt;So, we have to choose the minimum number of proteins so that we have enough proteins to cover all the required information for our specific purpose.&lt;/p&gt;&#xA;&lt;p&gt;This is called &amp;quot;protein culling.&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Google search shows that there were some servers like PISCES, ProDaMa, etc. However, as usual, all of them are down.&lt;/p&gt;&#xA;&lt;p&gt;Now the question is, how do I do protein-culling myself in the context of protein secondary structure detection?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;N.B.&lt;/strong&gt; Proteins with similar folds or secondary structure elements can be considered redundant, even if their sequence identity is low.&lt;/p&gt;&#xA;" OwnerUserId="12686" LastEditorUserId="12686" LastEditDate="2024-03-13T16:09:39.703" LastActivityDate="2024-03-13T16:09:39.703" Title="How do I do &quot;protein culling&quot;?" Tags="|proteins|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="22270" PostTypeId="1" CreationDate="2024-03-13T12:27:30.977" Score="1" ViewCount="10" Body="&lt;p&gt;I'm doing a study in which I'm trying to analyze the effect of molecules (ligands) on binding sites of proteins.&lt;/p&gt;&#xA;&lt;p&gt;Therefore, I'm trying to find a database in which the proteins bind to multiple different molecules, and as a result have different binding sites. The database that I'm working with currently is from UniProt, and the problem with it is that it has proteins that bind to only a small class of molecules, and hence the binding site is similar for each protein-ligand pair.&lt;/p&gt;&#xA;&lt;p&gt;For reference, here is the database that I'm using right now: &lt;a href=&quot;https://bit.ly/ProteinLigandBindingSite&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://bit.ly/ProteinLigandBindingSite&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Can someone help me find a database which contains proteins that bind to different molecules, and hence have different binding sites?&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="20075" LastEditorUserId="20075" LastEditDate="2024-03-13T14:28:50.157" LastActivityDate="2024-03-13T14:28:50.157" Title="How to find a database for proteins that bind to different molecules and have different binding sites?" Tags="|proteins|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22271" PostTypeId="1" CreationDate="2024-03-13T14:08:32.230" Score="2" ViewCount="30" Body="&lt;p&gt;(For context: I'm somewhat new to bioinformatic analyses but am mostly comfortable with R)&#xA;I have identified transcriptomically and morphologically different cancer cell populations within a patient (scRNAseq, spatial transcriptomics, IHC). They are quite distinct, with hundreds of genes as differentially expressed genes (DEGs) even after high tresholds. It's quite interesting to describe, but I am looking to find better ways to represent their features. Of course I am attaching DEG tables as supplementary data etc. but I thought that gene set enrichment analysis could give broad terms that would a) help me understand which DEGs might contribute together to biological processes and b) describe the bigger picture.&lt;/p&gt;&#xA;&lt;p&gt;So, I tried running a few terms (GO:BP, GO:CC, GO:MF, ...) from the GSEA msigdb website on the DEGs from scRNAseq, but have come out a bit frustrated. A lot of terms make sense and reflect the changes, but have large overlap. For example, the 3 top GO:BP terms (Extracellular matrix organization&amp;quot;, &amp;quot;Extracellular structure organization&amp;quot;, &amp;quot;External encapsulating structure organziation&amp;quot; are scored identically, with identical DEGs as the genes that are part of their gene sets. This is repeated further down the list with subsets of this gene set. Now, reporting these doesn't actually do what I want, which is give a global overview (but the information is heavily redundant). I could just pick one of the 3 redundant ones and skip the others, but then I am worried my presentation would be biased.&lt;/p&gt;&#xA;&lt;p&gt;Any advice on how to properly do gene set enrichment analysis to find broader terms to describe and understand what's going on? Or alternative ways that lead to the goal of broader description and understanding? I would be very grateful.&lt;/p&gt;&#xA;&lt;p&gt;Thank you!&lt;/p&gt;&#xA;" OwnerUserId="20076" LastActivityDate="2024-03-14T09:46:51.880" Title="How to perform meaningful Gene set erichment analysis or otherwise find broader themes/functions in different cell populations?" Tags="|scrnaseq|single-cell|cancer|gsea|" AnswerCount="2" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22272" PostTypeId="2" ParentId="22271" CreationDate="2024-03-13T15:01:18.810" Score="3" Body="&lt;p&gt;It is totally acceptable in your main figure to display only a selected part of your GO term, as long as the reader is aware of this action (e.g., by including a legend specifying &amp;quot;selected pathways enriched&amp;quot;). Additionally, ensure that either readers have sufficient information to replicate your analyses or provide the full result table as supplementary information. By adhering to these points, you are transparent on what you did, allowing others to replicate the analyses independently. Purposely excluding top hit pathways that contradict your theory could be considered misleading.&lt;/p&gt;&#xA;&lt;p&gt;If you are looking for broader terms, have you tried to a use slim GO? Those are GO with only a subset of the term. You could also look at the GO graph for your term and select the one higher in the graph.&#xA;There are also plenty of tools out there to display GO results-graph. GOplot, GOnet are the first hit I got on google.&lt;/p&gt;&#xA;" OwnerUserId="6655" LastActivityDate="2024-03-13T15:01:18.810" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22273" PostTypeId="2" ParentId="10633" CreationDate="2024-03-14T02:52:03.013" Score="0" Body="&lt;p&gt;Here is how you can do it from command line version:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db nuccore -query &amp;quot;COB [GENE] AND &amp;quot;Acetes japonicus&amp;quot; [ORGN]&amp;quot;|efetch -format gb&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;OR&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;esearch -db nuccore -query &amp;quot;COB [GENE] AND &amp;quot;Acetes japonicus&amp;quot; [ORGN]&amp;quot;|efetch -format fasta&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Note that gene name is actually COB.&lt;/p&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2024-03-14T02:52:03.013" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22274" PostTypeId="1" CreationDate="2024-03-14T04:04:01.170" Score="1" ViewCount="27" Body="&lt;p&gt;I have currently made a tool that takes input sequences and maps them to a reference sequence highlighting it. You can then save the word document and it creates a table of the sequence name and the sequences.&lt;/p&gt;&#xA;&lt;p&gt;What I want is to&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Highlight the sequences in specific colours.&lt;/li&gt;&#xA;&lt;li&gt;Highlight those sequences in the reference sequence.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Below is the create a word function.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;def create_word_document(output_text, reference_sequence):&#xA;    document = Document()&#xA;    reverse_complemented_sequences = {}&#xA;&#xA;    # Add reference sequence&#xA;    document.add_heading(&amp;quot;Reference Sequence&amp;quot;, level=1)&#xA;    document.add_paragraph(reference_sequence, style='BodyText')&#xA;&#xA;    # Add a table for sequences&#xA;    table = document.add_table(rows=1, cols=2, style='Table Grid')&#xA;    table.autofit = True&#xA;    table.cell(0, 0).text = &amp;quot;Sequence Name&amp;quot;&#xA;    table.cell(0, 1).text = &amp;quot;Highlighted Sequence&amp;quot;&#xA;&#xA;    # Colors for highlighting sequences&#xA;    highlight_colors = {&#xA;        &amp;quot;F3&amp;quot;: &amp;quot;FF0000&amp;quot;,  # Red for F3&#xA;        &amp;quot;B3&amp;quot;: &amp;quot;0000FF&amp;quot;,  # Blue for B3&#xA;        &amp;quot;F2&amp;quot;: &amp;quot;FFC0CB&amp;quot;, # Pink for F2&#xA;        &amp;quot;F1c&amp;quot;: &amp;quot;0000FF&amp;quot;, # Blue for F1c&#xA;        &amp;quot;B2&amp;quot;: &amp;quot;FFFF00&amp;quot;, # Yellow for B2&#xA;        &amp;quot;B1c&amp;quot;: &amp;quot;00008B&amp;quot;, # Dark blue for B1c&#xA;        &amp;quot;LF&amp;quot;: &amp;quot;FFA500&amp;quot;, # Orange for LF&#xA;        &amp;quot;LB&amp;quot;: &amp;quot;800080&amp;quot;, # Purple for LB&#xA;        # Change colors as needed&#xA;    }&#xA;&#xA;    for seq_name, highlighted_sequence in output_text.highlighted_sequences.items():&#xA;        # Use the reverse complemented sequence for this section&#xA;        reversed_sequence = reverse_complement_if_needed(seq_name, highlighted_sequence)&#xA;        reverse_complemented_sequences[seq_name] = reversed_sequence&#xA;&#xA;        # Add sequence information to the table&#xA;        row_cells = table.add_row().cells&#xA;        row_cells[0].text = f&amp;quot;{seq_name} Sequence&amp;quot;&#xA;        row_cells[1].text = reversed_sequence&#xA;&#xA;        # Add shading to the cell containing the sequence text if available&#xA;        if seq_name in highlight_colors:&#xA;            color_code = highlight_colors[seq_name]&#xA;            apply_shading_to_cell(row_cells[1], reversed_sequence, color_code)&#xA;&#xA;        # Add FIP and BIP sequences after B3&#xA;        if seq_name == &amp;quot;B3&amp;quot;:&#xA;            f1c_sequence = reverse_complement_if_needed(&amp;quot;F1c&amp;quot;, output_text.highlighted_sequences[&amp;quot;F1c&amp;quot;])&#xA;            f2_sequence = output_text.highlighted_sequences[&amp;quot;F2&amp;quot;]&#xA;            bip_sequence = reverse_complement_if_needed(&amp;quot;B1c&amp;quot;, output_text.highlighted_sequences[&amp;quot;B1c&amp;quot;])&#xA;            b2_sequence = output_text.highlighted_sequences[&amp;quot;B2&amp;quot;]&#xA;&#xA;            # Add a row for FIP sequence&#xA;            row_cells_fip = table.add_row().cells&#xA;            row_cells_fip[0].text = &amp;quot;FIP Sequence&amp;quot;&#xA;            row_cells_fip[1].text = f&amp;quot;{f1c_sequence}-{f2_sequence}&amp;quot;&#xA;            # Apply shading to FIP sequence if available&#xA;            if &amp;quot;F1c&amp;quot; in highlight_colors:&#xA;                apply_shading_to_cell(row_cells_fip[1], row_cells_fip[1].text, highlight_colors[&amp;quot;F1c&amp;quot;])&#xA;&#xA;            # Add a row for BIP sequence&#xA;            row_cells_bip = table.add_row().cells&#xA;            row_cells_bip[0].text = &amp;quot;BIP Sequence&amp;quot;&#xA;            row_cells_bip[1].text = f&amp;quot;{bip_sequence}-{b2_sequence}&amp;quot;&#xA;            # Apply shading to BIP sequence if available&#xA;            if &amp;quot;B1c&amp;quot; in highlight_colors:&#xA;                apply_shading_to_cell(row_cells_bip[1], row_cells_bip[1].text, highlight_colors[&amp;quot;B1c&amp;quot;])&#xA;&#xA;    # Save the document&#xA;    document.save(&amp;quot;LAMP Primer mapping.docx&amp;quot;)&#xA;&#xA;def apply_shading_to_cell(cell, text, color_code):&#xA;    # Clear existing text&#xA;    cell.text = ''&#xA;    # Add the text to the cell&#xA;    paragraph = cell.paragraphs[0]&#xA;    run = paragraph.add_run(text)&#xA;    # Create shading element&#xA;    shading_elm = OxmlElement('w:shd')&#xA;    shading_elm.set(qn('w:val'), 'clear')&#xA;    shading_elm.set(qn('w:color'), 'auto')&#xA;    shading_elm.set(qn('w:fill'), color_code)  # Set fill color&#xA;    # Append shading element to run properties&#xA;    run._r.append(shading_elm)&#xA;    # Set font size for the entire paragraph&#xA;    paragraph.style.font.size = Pt(12)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Full code:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;import tkinter as tk&#xA;from Bio.Seq import Seq&#xA;from Bio import pairwise2&#xA;from docx import Document&#xA;from docx.shared import Inches&#xA;from docx.shared import Pt&#xA;from docx.oxml.shared import OxmlElement&#xA;from docx.oxml.ns import qn&#xA;&#xA;class HighlightedOutputText(tk.Text):&#xA;    def __init__(self, *args, **kwargs):&#xA;        tk.Text.__init__(self, *args, **kwargs)&#xA;        self.highlighted_sequences = {}&#xA;&#xA;    def map_and_highlight(self, seq_name, reference_sequence, input_sequence):&#xA;        best_alignment = self.find_best_alignment(reference_sequence, input_sequence)&#xA;&#xA;        if best_alignment is not None:&#xA;            reference_aligned, input_aligned, start_position, end_position = best_alignment&#xA;&#xA;            # Configure highlight tag based on sequence name&#xA;            highlight_tag = f&amp;quot;{seq_name.lower()}_highlight&amp;quot;&#xA;            self.tag_configure(highlight_tag, background=get_highlight_color(seq_name))&#xA;&#xA;            # Apply the color tag to the aligned region in the reference sequence&#xA;            self.insert(tk.END, f&amp;quot;\n{seq_name} Input sequence:\n{input_sequence}\n\n&amp;quot;)&#xA;            self.insert(tk.END, &amp;quot;\nHighlighted sequence:\n&amp;quot;)&#xA;            self.insert(tk.END, reference_sequence[:start_position], &amp;quot;default&amp;quot;)&#xA;            self.insert(tk.END, reference_sequence[start_position:end_position], highlight_tag)&#xA;            self.insert(tk.END, reference_sequence[end_position:], &amp;quot;default&amp;quot;)&#xA;&#xA;            # Save highlighted sequence for later use&#xA;            self.highlighted_sequences[seq_name] = reference_sequence[start_position:end_position]&#xA;        else:&#xA;            self.insert(tk.END, f&amp;quot;\n{seq_name} Input sequence:\n{input_sequence}\n\n&amp;quot;)&#xA;            self.insert(tk.END, &amp;quot;\nNo matching region found in the reference sequence.\n&amp;quot;, &amp;quot;default&amp;quot;)&#xA;&#xA;    def find_best_alignment(self, reference_sequence, input_sequence):&#xA;        alignments_forward = pairwise2.align.localxx(reference_sequence, input_sequence)&#xA;        alignments_reverse = pairwise2.align.localxx(reference_sequence, str(Seq(input_sequence).reverse_complement()))&#xA;&#xA;        if not alignments_forward and not alignments_reverse:&#xA;            print(f&amp;quot;No alignment found for input sequence: {input_sequence}&amp;quot;)&#xA;            return None&#xA;&#xA;        # Choose the best alignment based on the score&#xA;        best_alignment = max(alignments_forward + alignments_reverse, key=lambda a: a.score)&#xA;&#xA;        # Extract aligned sequences&#xA;        reference_aligned = best_alignment[0]&#xA;        input_aligned = best_alignment[1]&#xA;&#xA;        # Find the start and end positions of the input sequence in the reference&#xA;        start_position = reference_aligned.find(input_aligned.replace('-', '').lstrip())&#xA;        end_position = start_position + len(input_aligned.replace('-', '').rstrip())&#xA;&#xA;        return reference_aligned, input_aligned, start_position, end_position&#xA;&#xA;    def clear_output(self):&#xA;        self.delete(1.0, tk.END)&#xA;        self.highlighted_sequences = {}&#xA;&#xA;def reverse_complement_if_needed(seq_name, input_sequence):&#xA;    # Check if the sequence name indicates reverse complementing is needed&#xA;    if seq_name in [&amp;quot;B3&amp;quot;, &amp;quot;F1c&amp;quot;, &amp;quot;B2&amp;quot;, &amp;quot;LF&amp;quot;]:&#xA;        return str(Seq(input_sequence).reverse_complement())&#xA;    else:&#xA;        return input_sequence&#xA;&#xA;def get_highlight_color(seq_name):&#xA;    # Return RGB values based on the sequence name&#xA;    if seq_name == &amp;quot;F3&amp;quot;:&#xA;        return &amp;quot;#808080&amp;quot;  # Gray&#xA;    elif seq_name == &amp;quot;B3&amp;quot;:&#xA;        return &amp;quot;#404040&amp;quot;  # Dark Gray&#xA;    elif seq_name == &amp;quot;LF&amp;quot;:&#xA;        return &amp;quot;#008000&amp;quot;  # Green&#xA;    elif seq_name == &amp;quot;LB&amp;quot;:&#xA;        return &amp;quot;#FF0000&amp;quot;  # Red&#xA;    elif seq_name == &amp;quot;F2&amp;quot;:&#xA;        return &amp;quot;#40E0D0&amp;quot;  # Turquoise&#xA;    elif seq_name == &amp;quot;F1c&amp;quot;:&#xA;        return &amp;quot;#FFFF00&amp;quot;  # Yellow&#xA;    elif seq_name == &amp;quot;B2&amp;quot;:&#xA;        return &amp;quot;#FFC0CB&amp;quot;  # Pink&#xA;    elif seq_name == &amp;quot;B1c&amp;quot;:&#xA;        return &amp;quot;#00008B&amp;quot;  # Dark Blue&#xA;    else:&#xA;        return &amp;quot;#FFA500&amp;quot;  # Default color for other sequences&#xA;&#xA;def create_word_document(output_text, reference_sequence):&#xA;    document = Document()&#xA;    reverse_complemented_sequences = {}&#xA;&#xA;    # Add reference sequence&#xA;    document.add_heading(&amp;quot;Reference Sequence&amp;quot;, level=1)&#xA;    document.add_paragraph(reference_sequence, style='BodyText')&#xA;&#xA;    # Add a table for sequences&#xA;    table = document.add_table(rows=1, cols=2, style='Table Grid')&#xA;    table.autofit = True&#xA;    table.cell(0, 0).text = &amp;quot;Sequence Name&amp;quot;&#xA;    table.cell(0, 1).text = &amp;quot;Highlighted Sequence&amp;quot;&#xA;&#xA;    # Colors for highlighting sequences&#xA;    highlight_colors = {&#xA;        &amp;quot;F3&amp;quot;: &amp;quot;FF0000&amp;quot;,  # Red for F3&#xA;        &amp;quot;B3&amp;quot;: &amp;quot;0000FF&amp;quot;,  # Blue for B3&#xA;        &amp;quot;F2&amp;quot;: &amp;quot;FFC0CB&amp;quot;, # Pink for F2&#xA;        &amp;quot;F1c&amp;quot;: &amp;quot;0000FF&amp;quot;, # Blue for F1c&#xA;        &amp;quot;B2&amp;quot;: &amp;quot;FFFF00&amp;quot;, # Yellow for B2&#xA;        &amp;quot;B1c&amp;quot;: &amp;quot;00008B&amp;quot;, # Dark blue for B1c&#xA;        &amp;quot;LF&amp;quot;: &amp;quot;FFA500&amp;quot;, # Orange for LF&#xA;        &amp;quot;LB&amp;quot;: &amp;quot;800080&amp;quot;, # Purple for LB&#xA;        # Change colors as needed&#xA;    }&#xA;&#xA;    for seq_name, highlighted_sequence in output_text.highlighted_sequences.items():&#xA;        # Use the reverse complemented sequence for this section&#xA;        reversed_sequence = reverse_complement_if_needed(seq_name, highlighted_sequence)&#xA;        reverse_complemented_sequences[seq_name] = reversed_sequence&#xA;&#xA;        # Add sequence information to the table&#xA;        row_cells = table.add_row().cells&#xA;        row_cells[0].text = f&amp;quot;{seq_name} Sequence&amp;quot;&#xA;        row_cells[1].text = reversed_sequence&#xA;&#xA;        # Add shading to the cell containing the sequence text if available&#xA;        if seq_name in highlight_colors:&#xA;            color_code = highlight_colors[seq_name]&#xA;            apply_shading_to_cell(row_cells[1], reversed_sequence, color_code)&#xA;&#xA;        # Add FIP and BIP sequences after B3&#xA;        if seq_name == &amp;quot;B3&amp;quot;:&#xA;            f1c_sequence = reverse_complement_if_needed(&amp;quot;F1c&amp;quot;, output_text.highlighted_sequences[&amp;quot;F1c&amp;quot;])&#xA;            f2_sequence = output_text.highlighted_sequences[&amp;quot;F2&amp;quot;]&#xA;            bip_sequence = reverse_complement_if_needed(&amp;quot;B1c&amp;quot;, output_text.highlighted_sequences[&amp;quot;B1c&amp;quot;])&#xA;            b2_sequence = output_text.highlighted_sequences[&amp;quot;B2&amp;quot;]&#xA;&#xA;            # Add a row for FIP sequence&#xA;            row_cells_fip = table.add_row().cells&#xA;            row_cells_fip[0].text = &amp;quot;FIP Sequence&amp;quot;&#xA;            row_cells_fip[1].text = f&amp;quot;{f1c_sequence}-{f2_sequence}&amp;quot;&#xA;            # Apply shading to FIP sequence if available&#xA;            if &amp;quot;F1c&amp;quot; in highlight_colors:&#xA;                apply_shading_to_cell(row_cells_fip[1], row_cells_fip[1].text, highlight_colors[&amp;quot;F1c&amp;quot;])&#xA;&#xA;            # Add a row for BIP sequence&#xA;            row_cells_bip = table.add_row().cells&#xA;            row_cells_bip[0].text = &amp;quot;BIP Sequence&amp;quot;&#xA;            row_cells_bip[1].text = f&amp;quot;{bip_sequence}-{b2_sequence}&amp;quot;&#xA;            # Apply shading to BIP sequence if available&#xA;            if &amp;quot;B1c&amp;quot; in highlight_colors:&#xA;                apply_shading_to_cell(row_cells_bip[1], row_cells_bip[1].text, highlight_colors[&amp;quot;B1c&amp;quot;])&#xA;&#xA;    # Save the document&#xA;    document.save(&amp;quot;LAMP Primer mapping.docx&amp;quot;)&#xA;&#xA;def apply_shading_to_cell(cell, text, color_code):&#xA;    # Clear existing text&#xA;    cell.text = ''&#xA;    # Add the text to the cell&#xA;    paragraph = cell.paragraphs[0]&#xA;    run = paragraph.add_run(text)&#xA;    # Create shading element&#xA;    shading_elm = OxmlElement('w:shd')&#xA;    shading_elm.set(qn('w:val'), 'clear')&#xA;    shading_elm.set(qn('w:color'), 'auto')&#xA;    shading_elm.set(qn('w:fill'), color_code)  # Set fill color&#xA;    # Append shading element to run properties&#xA;    run._r.append(shading_elm)&#xA;    # Set font size for the entire paragraph&#xA;    paragraph.style.font.size = Pt(12)&#xA;&#xA;# GUI setup&#xA;root = tk.Tk()&#xA;root.title(&amp;quot;Sequence Alignment&amp;quot;)&#xA;&#xA;# Entry widgets for reference sequence&#xA;label_reference = tk.Label(root, text=&amp;quot;Reference Sequence:&amp;quot;)&#xA;entry_reference = tk.Entry(root, width=30)&#xA;&#xA;# Entry widgets for multiple sequences&#xA;entries = []&#xA;output_text = HighlightedOutputText(root, height=30, width=40)&#xA;output_text.configure(font=(&amp;quot;TkDefaultFont&amp;quot;, 10))&#xA;&#xA;for seq_name in [&amp;quot;F3&amp;quot;, &amp;quot;B3&amp;quot;, &amp;quot;F2&amp;quot;, &amp;quot;F1c&amp;quot;, &amp;quot;B2&amp;quot;, &amp;quot;B1c&amp;quot;, &amp;quot;LF&amp;quot;, &amp;quot;LB&amp;quot;]:&#xA;    label = tk.Label(root, text=f&amp;quot;{seq_name} Sequence:&amp;quot;)&#xA;    entry = tk.Entry(root, width=30)&#xA;    entries.append((seq_name, entry))&#xA;    label.grid(row=len(entries), column=0, padx=10, pady=5)&#xA;    entry.grid(row=len(entries), column=1, padx=10, pady=5)&#xA;&#xA;def on_align_button_click():&#xA;    reference_sequence = entry_reference.get()&#xA;&#xA;    output_text.clear_output()  # Clear previous output&#xA;&#xA;    for seq_name, entry in entries:&#xA;        input_sequence = entry.get()&#xA;        input_sequence = reverse_complement_if_needed(seq_name, input_sequence)&#xA;        output_text.map_and_highlight(seq_name, reference_sequence, input_sequence)&#xA;&#xA;    # Display all highlighted sequences in the output window&#xA;    for seq_name, highlighted_sequence in output_text.highlighted_sequences.items():&#xA;        output_text.insert(tk.END, f&amp;quot;\n{seq_name} Input sequence:\n{highlighted_sequence}\n&amp;quot;)&#xA;&#xA;# Button to trigger alignment&#xA;align_button = tk.Button(root, text=&amp;quot;Align&amp;quot;, command=on_align_button_click)&#xA;&#xA;# Button to create a Word document&#xA;document_button = tk.Button(root, text=&amp;quot;Create Word Document&amp;quot;, command=lambda: create_word_document(output_text,entry_reference.get()))&#xA;&#xA;# Grid layout&#xA;label_reference.grid(row=0, column=0, padx=10, pady=5)&#xA;entry_reference.grid(row=0, column=1, padx=10, pady=5)&#xA;&#xA;align_button.grid(row=len(entries) + 1, column=0, columnspan=2, pady=10)&#xA;document_button.grid(row=len(entries) + 2, column=0, columnspan=2, pady=10)&#xA;output_text.grid(row=len(entries) + 3, column=0, columnspan=2, padx=10, pady=5)&#xA;&#xA;root.mainloop()&#xA;&lt;span class=&quot;math-container&quot;&gt;```&lt;/span&gt;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="18927" LastEditorUserId="650" LastEditDate="2024-03-14T21:46:44.320" LastActivityDate="2024-03-14T21:46:44.320" Title="How to shade text in a table" Tags="|python|pcr|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22275" PostTypeId="2" ParentId="22271" CreationDate="2024-03-14T09:38:20.163" Score="4" Body="&lt;p&gt;Additionally there are plenty of tools to summarize long lists of GO terms by removing redundancy (which seems to be your concern), such as &lt;a href=&quot;http://revigo.irb.hr/&quot; rel=&quot;nofollow noreferrer&quot;&gt;REVIGO&lt;/a&gt; (web) or &lt;a href=&quot;https://bioconductor.org/packages/release/bioc/html/rrvgo.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;rrvgo&lt;/a&gt; (R+Bioconductor). Disclaimer: I'm the author of the latter.&lt;/p&gt;&#xA;&lt;p&gt;Such tools objectively pick the best representative term of each set of redundant terms, eg. by considering the GO term size (in genes), observed/expected ratio, p-value, etc.&lt;/p&gt;&#xA;" OwnerUserId="393" LastEditorUserId="393" LastEditDate="2024-03-14T09:46:51.880" LastActivityDate="2024-03-14T09:46:51.880" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22276" PostTypeId="1" CreationDate="2024-03-14T09:51:01.770" Score="0" ViewCount="5" Body="&lt;p&gt;I'm Sebastian, working on my MSc thesis, aiming to create a knowledge graph of the wastewater treatment microbiome. We're using &lt;a href=&quot;https://github.com/stefpeschel/NetCoMi&quot; rel=&quot;nofollow noreferrer&quot;&gt;Netcomi&lt;/a&gt; for constructing Microbial Association Networks (MANs) from &lt;a href=&quot;https://www.ebi.ac.uk/metagenomics&quot; rel=&quot;nofollow noreferrer&quot;&gt;MGnify database&lt;/a&gt; metagenomics and metatranscriptomics data. Specifically, we are using the &lt;a href=&quot;https://academic.oup.com/bioinformatics/article/31/19/3172/211784?login=true&quot; rel=&quot;nofollow noreferrer&quot;&gt;CCLasso&lt;/a&gt; method. However, we are facing these challenges and have some questions:&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;&lt;strong&gt;Insufficient Samples&lt;/strong&gt;: Many MGnify studies lack enough samples for MANs inference, leading us to combine data from various studies. Is combining samples from different studies a viable strategy for MANs inference in this context?&lt;/li&gt;&#xA;&lt;li&gt;&lt;strong&gt;Data Heterogeneity:&lt;/strong&gt; The data exhibits variability in sequencing technologies (e.g., Illumina Hiseq, Nextseq, Nanopore, etc), experiment types (metagenomics, assembly, or metatranscriptomics), sampling countries, and other variables. How best can we account for the data's heterogeneity in our analyses using Netcomi?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Your expertise and advice would be incredibly valuable. Thanks for your time and assistance.&lt;/p&gt;&#xA;&lt;p&gt;Sebastian&lt;/p&gt;&#xA;" OwnerUserId="20080" LastActivityDate="2024-03-14T09:51:01.770" Title="Looking for guidance on Microbial Association Networks (MANs) Inference with samples from different metagenomics/metatranscriptomics studies" Tags="|networks|microbial-genomics|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22278" PostTypeId="1" AcceptedAnswerId="22285" CreationDate="2024-03-14T11:29:24.197" Score="2" ViewCount="27" Body="&lt;p&gt;I used RAxML software on about 300 species mtDNA region (COI gene).&lt;br /&gt;&#xA;Unfortunately I used a highly variable region so my tree only has highly support values for branches close to the leaves.&lt;/p&gt;&#xA;&lt;p&gt;But I also noticed that all the nodes split into one leaf and one other node... Why is this happening?&lt;br /&gt;&#xA;Isn't it common that a node will simply split into 2 other nodes?&lt;/p&gt;&#xA;&lt;p&gt;I am using the &lt;code&gt;GTRGAMMA&lt;/code&gt; function in RAxML (I don't know if that may be the cause).&lt;/p&gt;&#xA;&lt;p&gt;Do you have any suggestions on this topic? Should I be suspicious of my tree if I don't see any node splitting into 2 other nodes?&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;More info&lt;/strong&gt;&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Alignment algorithm: MUSCLE&lt;/li&gt;&#xA;&lt;li&gt;Graphical example:  &lt;a href=&quot;https://i.stack.imgur.com/cl91c.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/cl91c.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;br /&gt;&#xA;My tree looks like the one on the left. Every bifurcation links to a tip/leaf (terminal node) of the tree and a non-terminal node of the tree.  It sounds suspicious to me that I never observe a bifurcation that splits into 2 non-terminal nodes of the tree (figure on the right)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;" OwnerUserId="20082" LastEditorUserId="20082" LastEditDate="2024-03-19T10:18:02.323" LastActivityDate="2024-03-19T12:14:38.350" Title="Why is my phylogenetic tree always splitting into one leaf and one node?" Tags="|phylogenetics|phylogeny|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22279" PostTypeId="1" CreationDate="2024-03-14T13:05:53.047" Score="-1" ViewCount="13" Body="&lt;p&gt;I have a list of RNA binding proteins from HyDRa prediction. I have a long list of mRNA, I'm interested in.&lt;/p&gt;&#xA;&lt;p&gt;For the list of RNA binding proteins, I want to find the potential RNA binding partners. Lets say each protein can bind to 5-20 RNAs, I want to find this list.&#xA;Now for each RNA-protein binding, I want to calculate the binding affinity or any equivalent measurement.&#xA;Is there any tools, I can use to do this? or any related insilico approach I can try?&lt;/p&gt;&#xA;" OwnerUserId="14290" LastActivityDate="2024-03-14T13:05:53.047" Title="Is there any tools to predict the potential RNA candidate that can bind to a protein?" Tags="|proteins|rna|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22280" PostTypeId="1" AcceptedAnswerId="22281" CreationDate="2024-03-14T15:21:00.943" Score="0" ViewCount="16" Body="&lt;p&gt;I will have to carry out a project of assembly using hifi reads for which I have already illumina reads and I am wondering which of the hybrid assembly or polishing would be the best option for this kind of data? I wanted to use hifiasm which seems the gold standard and then nextpolish2 but I am wondering if using for example spades to do an hybrid assembly would lead to something better or worse?&#xA;This question is only intended to see if someone did such benchmarking before or not.&lt;/p&gt;&#xA;&lt;p&gt;Best regards,&lt;/p&gt;&#xA;" OwnerUserId="20083" LastEditorUserId="20083" LastEditDate="2024-03-14T15:28:38.333" LastActivityDate="2024-03-15T12:07:31.503" Title="Hybrid assembly versus polishing for hifi and illumina reads" Tags="|assembly|illumina|pacbio|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22281" PostTypeId="2" ParentId="22280" CreationDate="2024-03-15T12:07:31.503" Score="1" Body="&lt;p&gt;Usually, if you have HiFi reads, there is no way you can improve the assembly with short reads. Not just that they don't have higher quality, but they also map a lot worse. People sometimes use short reads for genome profiling and QCing the whole thing using Merqury.&lt;/p&gt;&#xA;&lt;p&gt;Not sure if there is benchmarking published though.&lt;/p&gt;&#xA;" OwnerUserId="57" LastActivityDate="2024-03-15T12:07:31.503" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22284" PostTypeId="1" CreationDate="2024-03-17T13:59:08.967" Score="0" ViewCount="15" Body="&lt;p&gt;This is part of my result.I saw someone saying that Vina docking results with rmsd greater than 2 are unreliable, but I don’t know how to judge.Help,plz！&#xA;&lt;a href=&quot;https://i.stack.imgur.com/AZIiA.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/AZIiA.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20091" LastEditorUserId="20091" LastEditDate="2024-03-17T13:59:47.597" LastActivityDate="2024-03-17T13:59:47.597" Title="Does the value of RMSD have any impact on my vina results? Are results with RMSD greater than 2 unreliable?" Tags="|proteins|docking|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22285" PostTypeId="2" ParentId="22278" CreationDate="2024-03-17T18:25:12.407" Score="1" Body="&lt;p&gt;What I suggest is the description sounds like a polytomy (star tree). In a phylogenetic tree, a polytomy is a node that has more than two immediate descending branches.&lt;/p&gt;&#xA;&lt;p&gt;A polytomy can have many branches originate from one node. In contrast, a bifurcating tree only two branches stem from one node.&lt;/p&gt;&#xA;&lt;p&gt;A polygomy simply means that the evolutionary relationships can not be fully resolved to dichotomies, i.e. bifurcations. There's nothing wrong with this result and it makes reading the tree much easier. The model you describe looks fine.&lt;/p&gt;&#xA;&lt;p&gt;The only thing to watch if you're not looking at a dendrogram, but are looking at the phylogram. I doubt thats the explanation in any case.&lt;/p&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;From the update. The figure you presented is an asymmetry phylogeny and is perfectly normal, for example this is observed in molluscs, described in &lt;a href=&quot;https://www.pnas.org/doi/10.1073/pnas.93.25.14279&quot; rel=&quot;nofollow noreferrer&quot;&gt;Palmer 1995&lt;/a&gt;.  If a species is moving directionally e.g. east to west across a massive landmass and at uniform geographic intervals establishing an ecological niche and forming a new species this will result in an asymmetric tree and might form a &amp;quot;cline&amp;quot;.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2024-03-19T12:14:38.350" LastActivityDate="2024-03-19T12:14:38.350" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22286" PostTypeId="1" CreationDate="2024-03-17T21:07:13.263" Score="1" ViewCount="39" Body="&lt;p&gt;I have expression matrices for different cell types, representing the expression of individual cells of that type. They were learned through a generative model, so I am confident they represent the approximate expression patterns of specific cell types. Now, I want to implement a bulk sequencing deconvolution using the aforementioned expression patterns. The pseudo-bulk I used is the summation of a large number of single-cell sequencing data (sc-seq). That's the background.&lt;/p&gt;&#xA;&lt;p&gt;My first approach is to design an optimization process to optimize a series of weights, so that the product of the weights and the cell type expression approximates the pseudo-bulk. I was advised to use Poisson loss as the loss function because it aligns with the biological characteristics of RNA-seq. However, I couldn't add non-negativity constraints to the weights during optimization, resulting in negative values in the optimization results, which is meaningless. Then I found the optimiza.nnls method in the scipy package, which implements non-negativity constraints, but it uses Euclidean distance to compare the pseudo-bulk and the sc-seq combination. I obtained some good results using this method, but I have the following questions.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Can I use Euclidean distance to compare the differences between two sequencing methods? To me, this problem seems to become a linear regression problem, i.e., combining sc-seq to approximate pseudo-bulk. At this stage, there don't seem to be any biological distribution assumptions, so I guess it's feasible.&lt;/li&gt;&#xA;&lt;li&gt;If the answer to the previous question is no, what biological assumptions does using Poisson loss follow, and what am I ignoring when using Euclidean distance for comparison?&lt;/li&gt;&#xA;&lt;li&gt;If I want to continue using Poisson loss to optimize weights, how should I set the non-negativity constraints on the weights? I have tried methods such as ReLU and softmax in machine learning, but the results are not good.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;" OwnerUserId="20094" LastEditorUserId="650" LastEditDate="2024-03-18T16:09:03.720" LastActivityDate="2024-03-18T16:09:03.720" Title="Comparing sc-seq combination and bulk-sequencing" Tags="|scrnaseq|machine-learning|deconvolution|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22287" PostTypeId="1" CreationDate="2024-03-18T23:32:08.710" Score="1" ViewCount="33" Body="&lt;p&gt;I'm starting to study Bioinformatics, I already have basic knowledge of Biochemistry and Genetics, but I want to buy a reference book, to use when I forget a concept (I'm a Computer Science student, not a Biotechnologist).&lt;/p&gt;&#xA;&lt;p&gt;I have already had contact with the following two books (for basic concepts):&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Molecular Biology of the Cell, Alberts;&lt;/li&gt;&#xA;&lt;li&gt;Principles of Biochemistry, Lehninger.&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Which is better for my use or do you have another recommendation?&lt;/p&gt;&#xA;&lt;p&gt;I prefer a generalist book, which covers the basic concepts, and not a Bioinformatics book, which covers basic concepts in one chapter.&lt;/p&gt;&#xA;" OwnerUserId="20103" LastActivityDate="2024-03-19T01:55:43.513" Title="Best reference book for basic concepts when working with bioinformatics" Tags="|gene|computational-biochemistry|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22288" PostTypeId="2" ParentId="22287" CreationDate="2024-03-19T01:55:43.513" Score="0" Body="&lt;p&gt;For a reputable up to date text book on biochemistry I would only consider&lt;/p&gt;&#xA;&lt;blockquote&gt;&#xA;&lt;p&gt;'Biochemistry&amp;quot; Tenth Edition ©2023 by Jeremy Berg; Gregory Gatto Jr.; Justin Hines; John L. Tymoczko; Lubert Stryer published by MacMillan.&lt;/p&gt;&#xA;&lt;/blockquote&gt;&#xA;&lt;p&gt;Stryer has been in the text book biochemistry business for a long time. I agree it's not a cheap option and I stopped reading biochemistry text books some time ago (see below), but this should be  as good as it gets.&lt;/p&gt;&#xA;&lt;p&gt;Don't use Lehninger although admittedly the 7th Edition is 2017, but biochemistry is a very fast moving field and you need to be as up to date as possible if you are relying on text books as a primary source of information. To be relying on text books they need to have a &amp;quot;Grade A&amp;quot; reputation and it to be extremely to date. Few text books stand the test of time.&lt;/p&gt;&#xA;&lt;p&gt;I've taught text book stuff from courses that relied on &amp;quot;out of date editions&amp;quot;, i.e. the authors have not updated the edition for a while. It can be a nightmare because essentially you need to navigate material that you know is incorrect and insubstantial, then accommodate and moderate these deficiencies in the end of course exam. Thats okay at a certain level, at research grade it gets unstuck.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2024-03-19T01:55:43.513" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22289" PostTypeId="1" CreationDate="2024-03-19T08:55:22.260" Score="1" ViewCount="16" Body="&lt;p&gt;When downloading GSE data, often I encounter that the authors refer to additional visits in the data description, yet, they are not found in the data.&lt;/p&gt;&#xA;&lt;p&gt;For example, the following &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE72326&quot; rel=&quot;nofollow noreferrer&quot;&gt;study&lt;/a&gt;. They mention additional visits, however inspecting the phenotype data extracted using &lt;code&gt;getGEO&lt;/code&gt; function does not show them (only baseline). The same happens with different studies as well, for example this &lt;a href=&quot;https://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE88885&quot; rel=&quot;nofollow noreferrer&quot;&gt;study&lt;/a&gt;.&lt;/p&gt;&#xA;&lt;p&gt;Example to how I extract the phenotype data:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;gsm &amp;lt;- getGEO('GSE88885', GSEMatrix = TRUE)&#xA;gsm &amp;lt;- gsm[[1]]&#xA;pheno_dat &amp;lt;- gsm@phenoData@data %&amp;gt;% as.data.frame() &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Am I missing some additional data?&lt;/p&gt;&#xA;" OwnerUserId="3039" LastActivityDate="2024-03-19T09:35:12.037" Title="missing additional visits in GSE data" Tags="|rna-seq|differential-expression|public-databases|geoquery|gse|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22290" PostTypeId="2" ParentId="22289" CreationDate="2024-03-19T09:35:12.037" Score="1" Body="&lt;p&gt;If it's not in there then you can either email the authors or just live with it. It is fully on the authors what you upload to NCBI. Nobody at NCBI checks the actual content of the uploads in terms of whether it indeed contains what methods texts or the paper pretend that it contains. Nor do (usually) the reviewers. Published data are great in terms of having a large repository of resources and knowledge available, downside is that metadata are often a mess or absent. There is no general and easy answer to this.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2024-03-19T09:35:12.037" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22291" PostTypeId="1" CreationDate="2024-03-19T12:01:49.153" Score="1" ViewCount="26" Body="&lt;p&gt;I had isolated the total RNA from a sample of plant roots with colonized bacteria by excising the roots (harboring the bacterial cells) and crushing them into a powder using liquid nitrogen. The RNA isolation procedure was done following the kit's instructions specific for bacterial RNA.&lt;/p&gt;&#xA;&lt;p&gt;The RNA was sequenced and the raw reads obtained were checked for mapping percentage with the bacterial genome's GTF file using bowtie2, as I need to study the expression of the bacterial genes. While the mapping was close to 80% for samples coming from the planktonic state, it was quite less - 0.4 to 0.56%, for all the samples coming from root colonized cells, obviously because a major part of it is comprised of plant RNA. Firstly, is this acceptable? Can I use these reads for further analysis?&lt;/p&gt;&#xA;&lt;p&gt;Secondly, I use DESeq2 to analyze the data for upregulated and downregulated genes, comparing the colonized cells to the planktonic ones. Since the package includes a normalization step, is it safe to assume that although the mapping is less it does not influence the analysis significantly?&lt;/p&gt;&#xA;" OwnerUserId="17392" LastActivityDate="2024-03-19T12:01:49.153" Title="Is it normal to have low mapping to bacterial genome in a total RNA sample of plant root-colonized bacterial cells?" Tags="|rna-seq|deseq2|rna|read-mapping|bowtie2|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22292" PostTypeId="2" ParentId="6813" CreationDate="2024-03-19T16:49:07.870" Score="0" Body="&lt;p&gt;confusion: Why when I am putting each of 23 genes individually in glm model, 5 of them returns p-value &amp;lt; 0.05 but when summary of final model containing these 5 genes says only p-value of one of them is &amp;lt; 0.05.&lt;/p&gt;&#xA;&lt;p&gt;Ans. Pr(&amp;gt;|z|) simply suggests that whether z-value deviates beyond average z-value and if so,what is the extent of its probability. less than .05 indicates a 95% confidence in computed z value i.e. an acceptable significance level. The summary of 5 genes simply tells significance of over-all model.&lt;/p&gt;&#xA;" OwnerUserId="7374" LastActivityDate="2024-03-19T16:49:07.870" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22293" PostTypeId="1" CreationDate="2024-03-19T21:24:17.473" Score="0" ViewCount="10" Body="&lt;p&gt;currently, I am learning single-cell sequencing and microarray analysis. I am searching for somewhere (or any alternative way)to practice and get hands-on with these techniques. please help me with that.&#xA;Thank you in advance.&lt;/p&gt;&#xA;" OwnerUserId="20109" LastActivityDate="2024-03-19T21:24:17.473" Title="Practical challenges for microarray analysis and single cell" Tags="|single-cell|microarray|" AnswerCount="0" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22294" PostTypeId="1" CreationDate="2024-03-19T22:55:27.580" Score="-1" ViewCount="33" Body="&lt;p&gt;I am an apprentice-level python user who occasionally works in medical laboratories and mainly does business work. I am presented with the unenviable task of orchestrating a proficiency test of some sequencing teams who have the almost equally unenviable task of evaluating metagenomic FASTQ files of wastewater (sewage) data where the question is &amp;quot;could this have been engineered ?&amp;quot;&lt;/p&gt;&#xA;&lt;p&gt;Trick: how do you make some FASTQ files that look like wastewater (plenty of those online), definitely has some pathogen (SARS-CoV-2, Ebola, whatever), oh, and the pathogen has to look like it could have been engineered? And make an equal number that do not have the pathogen but otherwise look the same?&lt;/p&gt;&#xA;&lt;p&gt;Here is my initial thoughts:&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;&lt;p&gt;World's simplest version: cat a virus FASTQ onto the end of a&#xA;wastewater FASTQ.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;change all the identifiers in the virus file so&#xA;they look like a continuation of the wastewater, chop the virus file&#xA;with visual mode in Vim so the reads are the same length as the&#xA;wastewater.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;instead of cat, do some python to build a new&#xA;FASTQ by randomly selecting lines from the virus and wastewater&#xA;&amp;quot;feedstock&amp;quot; FASTQs.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Very fancy: grab a assembled virus FASTA, splice in an interesting&#xA;bit of pangolin, raccoon, pig, etc, then randomly select and subtly&#xA;mutate substrings as reads of appropriate length, and assemble a de&#xA;novo synthetic virus FASTQ. Mix that into the wastewater FASTQ as&#xA;above.&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Surely I am missing many, many steps and &amp;quot;very fancy&amp;quot; is probably not that fancy (or maybe the sun will grow cold before it succeeds, I do not know). Help and suggestions welcome, it could be fun.&lt;/p&gt;&#xA;" OwnerUserId="20110" LastEditorUserId="10049" LastEditDate="2024-03-20T21:59:05.753" LastActivityDate="2024-03-20T21:59:05.753" Title="How to design a synthetic wastewater FASTQ file that could be flagged as &quot;engineered&quot;?" Tags="|python|fastq|metagenome|quality-control|" AnswerCount="1" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22295" PostTypeId="1" CreationDate="2024-03-20T08:10:50.820" Score="0" ViewCount="20" Body="&lt;p&gt;Given the PDB or mmCIF information of a homomeric protein assembly whose global symmetry is unknown, how might one design an algorithm to find which point group symmetry class it belongs to (cyclic &lt;span class=&quot;math-container&quot;&gt;$C_n$&lt;/span&gt;, dihedral &lt;span class=&quot;math-container&quot;&gt;$D_n$&lt;/span&gt;, or neither) without requiring manual inspection?&lt;/p&gt;&#xA;&lt;p&gt;One idea I had for detecting cyclic symmetry was to calculate the centroid of each subunit in the homomer. If there indeed is some cyclic symmetry, the axis must pass through the global centroid, and it must be normal to the plane formed by the individual subunit centroids. To test if this axis is indeed valid, one can rotate the entire protein about the axis by 2π / n, and check that the atomic coordinates align closely with those of the original (using RMSD for example).&lt;/p&gt;&#xA;&lt;p&gt;But I'm having trouble thinking about dihedral symmetry, specifically for simple homomers. Any help on this would be appreciated.&lt;/p&gt;&#xA;" OwnerUserId="20114" LastEditorUserId="20114" LastEditDate="2024-03-20T08:27:24.553" LastActivityDate="2024-03-20T08:27:24.553" Title="Protein symmetry classification from PDB data" Tags="|protein-structure|pdb|" AnswerCount="0" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22296" PostTypeId="2" ParentId="22294" CreationDate="2024-03-20T08:19:21.230" Score="1" Body="&lt;p&gt;Use &lt;a href=&quot;https://www.nature.com/articles/s41437-022-00577-3&quot; rel=&quot;nofollow noreferrer&quot;&gt;existing&lt;/a&gt; software to generate simulated fastq files. The input for these tools is generally a set of reference organisms that you'd expect to find in your sample. Simply add your 'engineered' viral genome to the list of references. These tools have error models to simulate all the errors that real sequencing instruments make.&lt;/p&gt;&#xA;&lt;p&gt;How you'd go about making a viral sequence recognisable as genetically engineered, I have no clue. I'm sure the person who gave you this assignment has an idea on how to engineer viruses and what recognisable features that would result in, maybe consult with them.&lt;/p&gt;&#xA;" OwnerUserId="3818" LastActivityDate="2024-03-20T08:19:21.230" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22297" PostTypeId="1" CreationDate="2024-03-20T15:12:54.500" Score="1" ViewCount="28" Body="&lt;p&gt;I need 18S rRNA gene sequences for a wide variety of eukaryotic species.&lt;br /&gt;&#xA;I know that Ensembl has this gene sequence, but I don't understand what is the correct protocol to download the sequences. I already know  the basics of Ensembl's Rest API, but there are still a few things that I don't understand.&lt;/p&gt;&#xA;&lt;p&gt;I only know the gene name (18S rRNA) and the list of taxa I want to select it from. But I do not know the region of the gene.&lt;/p&gt;&#xA;&lt;p&gt;For example: I tried looking for '18srrna' on the Ensembl browser and I noticed that for the Fruit fly there are 2 sequences corresponding to this gene (Ensembl IDs: FBgn0085813 and FBgn0267521).&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;How do I know which sequence I should use to make an alignment with other species?&lt;/li&gt;&#xA;&lt;li&gt;How can I batch download the same region for all the species of interest?&lt;/li&gt;&#xA;&lt;li&gt;Do you know any protocol/guide that goes into detail of how to download and select sequences to make phylogenies?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;I don't have a solution for question (1) but for question (2) I am planning on requesting from the API the output of the endpoint &lt;code&gt;GET sequence/region/:species/:region&lt;/code&gt;. But then again the problem is that I would need the position of the region for each species, and it sounds unfeasible to do for more than a dozen of species.&lt;/p&gt;&#xA;" OwnerUserId="20082" LastEditorUserId="298" LastEditDate="2024-03-21T11:30:52.670" LastActivityDate="2024-03-24T16:04:14.987" Title="How to download the same 18S rRNA gene region for multiple species?" Tags="|sequence-alignment|ensembl|multiple-sequence-alignment|dna|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22298" PostTypeId="1" CreationDate="2024-03-20T16:21:21.673" Score="1" ViewCount="51" Body="&lt;p&gt;&lt;em&gt;This question was also asked on &lt;a href=&quot;https://www.reddit.com/r/bioinformatics/comments/1bjil16/genes_aligned_on_curve_in_volcano_plots_any_idea/&quot; rel=&quot;nofollow noreferrer&quot;&gt;reddit&lt;/a&gt;&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/BK9SI.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/BK9SI.png&quot; alt=&quot;Volcano plot of differentially expressed genes&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Does anyone know how to correct the abnormal genes that seemed to be located on the curves near the extremities?&#xA;The analysis was done using R package edgeR on samples measured once. However, we couldn't figure out what was causing the anomaly or how to fix it.&#xA;Does anyone have any advice on this matter?&#xA;The code used for analysis is from the following website:  &lt;a href=&quot;https://blog.csdn.net/u012110870/article/details/102804557&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://blog.csdn.net/u012110870/article/details/102804557&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="18125" LastEditorUserId="73" LastEditDate="2024-03-21T12:45:10.030" LastActivityDate="2024-03-21T12:45:59.997" Title="Abnormal representation of differentially expressed genes in volcano plots located at the extremities" Tags="|r|rna-seq|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22299" PostTypeId="2" ParentId="22298" CreationDate="2024-03-20T22:42:08.677" Score="1" Body="&lt;p&gt;In &amp;quot;normal&amp;quot; (that is replicated) experiments such structures usually are due to poor/insufficient/no prefiltering when there are unreliable count outliers (like two samples have zero counts for a gene and one has a lot).&lt;/p&gt;&#xA;&lt;p&gt;As you cannot really meaningfully do any prefiltering other than requiring that either one or both samples have a certain read count you should probably just see these &amp;quot;DE results&amp;quot; (it's not differential really as it is unreplicated) as a ranking. Fold changes and p-values are very unreliable at 1 vs 1 so just rank by something like signed p-value and go along with that. Don't overinterpret these sorts of plots which are mostly meaningless in such a setup.&lt;/p&gt;&#xA;&lt;p&gt;That having said, for prefilter I would probably require that at least one sample has a good number of reads, like 50 to at least dampen unreliably large fold changes a little bit in this setup.&lt;/p&gt;&#xA;" OwnerUserId="16676" LastActivityDate="2024-03-20T22:42:08.677" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22300" PostTypeId="1" CreationDate="2024-03-21T01:31:19.560" Score="2" ViewCount="25" Body="&lt;p&gt;I recently came across a preprint ( &lt;a href=&quot;https://doi.org/10.1101/2024.03.14.585103&quot; rel=&quot;nofollow noreferrer&quot;&gt;DOI&lt;/a&gt;, &lt;a href=&quot;https://www.biorxiv.org/content/10.1101/2024.03.14.585103v1.full.pdf&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDF link&lt;/a&gt; ) that has some figures of protein structures in a style that I have not seen before.  To me it appears easier on the eyes and less noisy than the more traditional 3D models with lighting that &amp;quot;reflects&amp;quot; on a glossy surface.  I am interested in creating similar plots of my own protein structures.&lt;/p&gt;&#xA;&lt;p&gt;Does anyone recognise this style / know how to generate these plots from a PDB file or similar?&lt;/p&gt;&#xA;&lt;p&gt;(Extended Data Figure 2, page 20)&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/WV8Fi.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/WV8Fi.png&quot; alt=&quot;Protein cartoon figure&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20120" LastActivityDate="2024-03-21T01:31:19.560" Title="Method for reproducing minimal, cartoon-style protein structure visualisation" Tags="|proteins|protein-structure|visualization|figure-reproduction|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22302" PostTypeId="2" ParentId="19569" CreationDate="2024-03-21T11:56:12.713" Score="1" Body="&lt;p&gt;The results you'd get are the same as with any other NCBI BLAST+ tools, python does not compute it.&lt;/p&gt;&#xA;&lt;p&gt;Biopython can run a blast query &lt;code&gt;qblast&lt;/code&gt; on the internet or locally. To make it run locally you need to install blast tools on your machine and biopython will launch it as a subprocess. Here you can find the pertinent information: &lt;a href=&quot;https://biopython.org/docs/dev/Tutorial/chapter_blast.html#running-blast-locally&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://biopython.org/docs/dev/Tutorial/chapter_blast.html#running-blast-locally&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Filtering data is as much a burden as it is important. Depending on your question you want to answer. Some people prefer a low e-value. The score is however affected by the length of the sequence since the number of matches is bigger, more details here: &lt;a href=&quot;https://www.nlm.nih.gov/ncbi/workshops/2023-08_BLAST_evol/blast_score.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.nlm.nih.gov/ncbi/workshops/2023-08_BLAST_evol/blast_score.html&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="5141" LastEditorUserId="5141" LastEditDate="2024-03-21T16:31:17.643" LastActivityDate="2024-03-21T16:31:17.643" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22303" PostTypeId="2" ParentId="22298" CreationDate="2024-03-21T12:45:59.997" Score="0" Body="&lt;p&gt;Might as well transfer my reddit answer here...&lt;/p&gt;&#xA;&lt;p&gt;If your p-value is smaller than the reciprocal of the number of atoms in the universe, the model used for calculating that differential expression is almost certainly wrong.&lt;/p&gt;&#xA;&lt;p&gt;Do an MA plot; that will give you a better idea of what's going on.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;X axis - mean of (log expression for all samples); should be in the range of 0~15&lt;/li&gt;&#xA;&lt;li&gt;Y axis - log2 fold change&lt;/li&gt;&#xA;&lt;li&gt;colour - p-value (linear scale)&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;I suspect that the curves you're seeing are likely to be low count genes, in other words, genes on the left hand side of the graph in an MA plot.&lt;/p&gt;&#xA;&lt;p&gt;I also suspect, given the super-astronomical p-values, that you've got an offset L2FC (i.e. the &amp;quot;spine&amp;quot; of the MA plot doesn't sit at L2FC = 0), suggesting that a covariate hasn't been appropriately accounted for. This is common when processing data from multiple samples from multiple different methods.&lt;/p&gt;&#xA;" OwnerUserId="73" LastActivityDate="2024-03-21T12:45:59.997" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22304" PostTypeId="1" CreationDate="2024-03-22T01:20:49.223" Score="-1" ViewCount="14" Body="&lt;p&gt;Hi, I want to use Circos to draw syntenic relationships among multiple species, like the Fig.1b in this article. How to define the ideograms for different species?&lt;/p&gt;&#xA;&lt;p&gt;Thank you.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://doi.org/10.1038/s41588-024-01683-&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://doi.org/10.1038/s41588-024-01683-&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://i.stack.imgur.com/YkmfS.jpg&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/YkmfS.jpg&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20123" LastEditorUserId="20123" LastEditDate="2024-03-22T01:44:59.910" LastActivityDate="2024-03-22T01:44:59.910" Title="How to use CIRCOS plot pairwise syteny blocks for multi species" Tags="|circos|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22305" PostTypeId="1" CreationDate="2024-03-22T13:14:50.303" Score="2" ViewCount="39" Body="&lt;p&gt;&lt;em&gt;Disclaimer: I am a machine learning researcher working on network science and want to use proteins to test an algorithm of mine using real-world data. My bioinformatics knowledge is minimal.&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;I need to create a dataset of &lt;a href=&quot;https://en.wikipedia.org/wiki/Protein_contact_map&quot; rel=&quot;nofollow noreferrer&quot;&gt;contact maps&lt;/a&gt;. Namely, for a given set of proteins, I want to find the set of binary matrices that represent each of their contact maps. I understand that this binary matrix is obtained by thresholding the distances between atoms with some criteria. Thus, it would also be helpful to find the set of distance matrices for these proteins (I can just threshold them afterward).&lt;/p&gt;&#xA;&lt;p&gt;For computational reasons, I would prefer proteins that are as small as possible. Namely, proteins that have the fewest amount of amino acids. Also, I need the proteins to be homologous since I want to be able to do a Multiple Sequence Alignment (MSA) to them and analyze the matrix returned by the MSA with my machine learning algorithm.&lt;/p&gt;&#xA;&lt;p&gt;&lt;strong&gt;So my questions is: given a reference protein (let's say &lt;a href=&quot;https://www.uniprot.org/uniprotkb/P07947/entry&quot; rel=&quot;nofollow noreferrer&quot;&gt;YES_HUMAN&lt;/a&gt; for example), where can I find a list of homologous proteins and the contact maps for all of them?&lt;/strong&gt;&lt;/p&gt;&#xA;" OwnerUserId="20119" LastEditorUserId="20119" LastEditDate="2024-03-22T17:41:53.320" LastActivityDate="2024-03-22T17:41:53.320" Title="How to create a dataset of contact maps of homologous proteins?" Tags="|sequence-alignment|proteins|protein-structure|machine-learning|multiple-sequence-alignment|" AnswerCount="0" CommentCount="11" ContentLicense="CC BY-SA 4.0" />
  <row Id="22306" PostTypeId="2" ParentId="22297" CreationDate="2024-03-23T22:46:18.313" Score="0" Body="&lt;ul&gt;&#xA;&lt;li&gt;How do I know which sequence I should use to make an alignment with other species?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Use both. For fruit flies there should thousands of 18S sequences.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;How can I batch download the same region for all the species of interest?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Using Blast (below)&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;Do you know any protocol/guide that goes into detail of how to download and select sequences to make phylogenies?&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Sure - Blast ... align via muscle -super5 and build via IQ-TREE&lt;/p&gt;&#xA;&lt;p&gt;EMBL I don't know - using REST API via a CLI is possible. The problem is that you could potentially 1000s of 18S genes but a simple search term can ignore everything that isn't part of a genome explicitly labelled &amp;quot;18S&amp;quot;. Thats where a Blast style approach is useful.&lt;/p&gt;&#xA;&lt;p&gt;However, for NCBI Blast they intentionally block access to CLI with &amp;gt;100 hits requested. This is a complete pain and really, really limits using Blast to extract out a gene remotely using a remote request script.&lt;/p&gt;&#xA;&lt;p&gt;Basically the way to do it is to ditch the API and run this via the Blast NCBI web interface, or download the entire eukaryotic database and perform local Blast, MMseqs2 or Minimap2.&lt;/p&gt;&#xA;&lt;p&gt;It's a bit bizarre I know, NCBI just doesn't like bots running its Blast service, but it gives more permission to use its web-service. The central advantage is that Blast will nicely remove the 18S gene from any given genome which can be downloaded as a single locus, without needing to download loads of genome stuff thats not needed. NCBI still does impose a limit on how much can be blasted via the web which sadly now is only 5000 (I'm sure it was much more like 50000) and its really not cool.&lt;/p&gt;&#xA;&lt;p&gt;What you can do is use the Blast search &amp;quot;exclude/include&amp;quot; parameters to target the taxa of interest within the 5000 limit and using a tree of relevant eukaryotic evolution navigate the data sets. I assume you are looking for around 100000 sequences. Basically, once you've blasted you have the option of downloading just the region blasted and nothing else ... or the whole genome (avoid this option)... you will then have 5000 nicely 18S genes excised &lt;em&gt;in silico&lt;/em&gt;&lt;/p&gt;&#xA;&lt;p&gt;You can also set a lot other blast parameters, in particular only considering full length hits.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastActivityDate="2024-03-23T22:46:18.313" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22307" PostTypeId="1" CreationDate="2024-03-24T03:29:10.537" Score="0" ViewCount="50" Body="&lt;p&gt;I have some bed alike files but don't have &lt;code&gt;chr&lt;/code&gt; term in the fisrt column, that I want to add &lt;code&gt;chr&lt;/code&gt; and its corresponding chromosome number, I'm wondering how can I do that in bash!&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;head dat.tsv&#xA;9901    10000   0.186130156985236&#xA;10001   10100   244.668091357092&#xA;10101   10200   367.420929888855&#xA;10201   10300   179.801731647738&#xA;10301   10400   141.551984387272&#xA;10401   10500   92.5066880216621&#xA;10501   10600   92.3205578646769&#xA;10601   10700   127.96448292735&#xA;10701   10800   74.6381929510795&#xA;10801   10900   62.6327978255318&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="12770" LastActivityDate="2024-03-26T04:56:06.043" Title="How to add corresponding &quot;chr&quot; before location (start, end) of bed alike file?" Tags="|bash|bed|tsv|" AnswerCount="2" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22308" PostTypeId="2" ParentId="22297" CreationDate="2024-03-24T16:04:14.987" Score="0" Body="&lt;p&gt;I can suggest two ways to batch-download aligned sequences for the species of interest. Both ways are integral to the Ensembl database and toolsets.&lt;/p&gt;&#xA;&lt;p&gt;But before that, I suggest FBgn0085802 (which is rRNA) instead of FBgn0085813 and FBgn0267521 (which are pseudogenes). You may check FlyBase database.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Since you are interested in Fruit fly 18S rRNA homologs, Metazoa Ensembl is preferable. You query Ensembl ID in the Ensembl search box (at &lt;a href=&quot;http://www.metazoa.ensebml.org&quot; rel=&quot;nofollow noreferrer&quot;&gt;www.metazoa.ensebml.org&lt;/a&gt;) and click the popping link. 18SrRNA:CR41548 comes for FBgn0085802. On the left-hand side, Gene-based displays, select Sequence. under the Marked-up sequence, pick &amp;quot;BLAST this sequence&amp;quot;.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Opening the BLAST/BLAT search automatically sets a DNA alignment query. You may search against the species of interest and add &amp;quot;18S&amp;quot; as Description. Later, run without changing any other parameter. It may take several minutes. Then, view results for hits found. You can investigate Sequence/s with the best alignment and closest genomic locations, then export data in FASTA or other formats.&lt;/p&gt;&#xA;&lt;p&gt;While this assures you of the alignment and lets you investigate in detail the homologs, you may not batch-download particularly in this way.&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;As mentioned, BioMart Restful API is a way to batch-download sequences that you want. The website does only allow one search at a moment but editing the Perl script, which is generated according to your query, you may get sequences for multiple datasets. For setup, you should follow BioMart Perl API guideline cautiosly.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;For your example query, set proper datasets, filters and attributes. Again for fruit fly, you may refer to:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://www.metazoa.ensembl.org/info/data/biomart/biomart_perl_api.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://www.metazoa.ensembl.org/info/data/biomart/biomart_perl_api.html&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;My example query seems like below in popping Perl script:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;&lt;span class=&quot;math-container&quot;&gt;$query-&amp;gt;setDataset(&quot;dyakuba_eg_gene&quot;); &#xA;$&lt;/span&gt;query-&amp;gt;addFilter(&amp;quot;with_dmelanogaster_eg_homolog&amp;quot;, [&amp;quot;Only&amp;quot;]);&#xA;&lt;span class=&quot;math-container&quot;&gt;$query-&amp;gt;addFilter(&quot;biotype&quot;, [&quot;rRNA&quot;]);&#xA;$&lt;/span&gt;query-&amp;gt;addFilter(&amp;quot;transcript_biotype&amp;quot;, [&amp;quot;rRNA&amp;quot;]);&#xA;&lt;span class=&quot;math-container&quot;&gt;$query-&amp;gt;addAttribute(&quot;ensembl_gene_id&quot;);&#xA;$&lt;/span&gt;query-&amp;gt;addAttribute(&amp;quot;ensembl_transcript_id&amp;quot;);&#xA;&lt;span class=&quot;math-container&quot;&gt;$query-&amp;gt;addAttribute(&quot;gene_biotype&quot;);&#xA;$&lt;/span&gt;query-&amp;gt;addAttribute(&amp;quot;description&amp;quot;);&#xA;$query-&amp;gt;addAttribute(&amp;quot;transcript_biotype&amp;quot;);&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="15695" LastActivityDate="2024-03-24T16:04:14.987" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22309" PostTypeId="1" CreationDate="2024-03-25T03:53:56.147" Score="2" ViewCount="34" Body="&lt;p&gt;I'd like to calculate per site Fst(fixation index) to investigate genomic differentiation between two populations(haploid).&lt;/p&gt;&#xA;&lt;p&gt;I did SNP calling and got vcf file.&#xA;In the vcf file, multi allelic sites exist:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;#CHROM  POS ID  REF ALT&#xA;1   154 .   G   &amp;quot;A, T&amp;quot;&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;For example this site is triallelic sites.&lt;/p&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;Could I calculate Fst in this site by normalizing vcf file?&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Normalization means dividing into 2 lows.&#xA;REF G ALT A(T would be convert G)&#xA;REF G ALT T(A would be convert G)&lt;/p&gt;&#xA;&lt;p&gt;In calculation using Popgenome package, we can specify missing sites(Missing sites would not be included in the calculation).&#xA;In this case,　which is better: replacing with 'REF' or replacing with missing sites?&lt;/p&gt;&#xA;&lt;ol start=&quot;2&quot;&gt;&#xA;&lt;li&gt;&lt;p&gt;Why most studies calculate only biallelic sites?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;li&gt;&lt;p&gt;Before analyzing, removing variants based on MAF is make sence?&#xA;For example, one sites include A(56%), T(44.5%), C(0.5%).&#xA;In this case, could I remove variant C(replace missing site(NA)) before calculation of Fst?&lt;/p&gt;&#xA;&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Best regards,&lt;/p&gt;&#xA;" OwnerUserId="20128" LastEditorUserId="73" LastEditDate="2024-03-26T07:16:51.703" LastActivityDate="2024-03-27T18:34:41.053" Title="How should multi allelic sites be handled in calculation per site Fst?" Tags="|vcf|snp|bcftools|fst|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22310" PostTypeId="1" CreationDate="2024-03-25T08:26:28.687" Score="1" ViewCount="16" Body="&lt;p&gt;Here I am conducting WGCNA analysis. When calculating the soft threshold, it is basically bel&lt;a href=&quot;https://i.stack.imgur.com/P3vHl.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/P3vHl.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;ow 0.8. I choose Power=6, which only has 0 and 7. Is it feasible?&lt;/p&gt;&#xA;" OwnerUserId="18036" LastActivityDate="2024-03-25T12:51:12.977" Title="WGCNA soft threshold selection" Tags="|wgcna|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22311" PostTypeId="2" ParentId="22310" CreationDate="2024-03-25T12:51:12.977" Score="0" Body="&lt;p&gt;I don't know the context (how many samples, what tissue, experimental design etc) but the mean and median connectivities are pretty high at power 6. I would first check the data for strong drivers of correlation and possibly adjust for them if they do not seem biologically relevant to the question you study. Assuming you find no issues, I would go with a higher power here, maybe 12.&lt;/p&gt;&#xA;" OwnerUserId="851" LastActivityDate="2024-03-25T12:51:12.977" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22313" PostTypeId="1" CreationDate="2024-03-25T19:27:10.040" Score="1" ViewCount="22" Body="&lt;p&gt;I have a filtered VCF file with indels (no SNPs). Does anyone know of a easy way to use bcftools/vcftools to filter out small or large indels (I'm interested in indels between 50-100bp in size)? Thanks!&lt;/p&gt;&#xA;" OwnerUserId="20134" LastActivityDate="2024-03-27T17:47:57.917" Title="How can I filter variants in a VCF file based on size?" Tags="|vcf|genome|bcftools|vcftools|" AnswerCount="1" CommentCount="2" ContentLicense="CC BY-SA 4.0" />
  <row Id="22314" PostTypeId="2" ParentId="22307" CreationDate="2024-03-26T02:54:04.333" Score="1" Body="&lt;p&gt;You could use the following perl one-liner, however if genome you are using contains more than one chromosome you need to figure out which lines get which chromosome chr1, chr2. If it's single chrosome like E.coli, usually name of chromosome is &amp;quot;chr&amp;quot;.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;perl -npe &amp;quot;s/(.*)/chr\t\1/&amp;quot; dat.tsv&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;" OwnerUserId="14180" LastActivityDate="2024-03-26T02:54:04.333" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22315" PostTypeId="2" ParentId="22307" CreationDate="2024-03-26T04:56:06.043" Score="-1" Body="&lt;p&gt;I appreciate your responses.&lt;/p&gt;&#xA;&lt;p&gt;Using the R &lt;code&gt;dplyr&lt;/code&gt; package after using &lt;code&gt;bedtools&lt;/code&gt;, I was able to deal with this problem.&#xA;Using the &lt;code&gt;makewindows&lt;/code&gt; function from &lt;code&gt;bedtool&lt;/code&gt;, I first obtain a 100 bp window.&lt;/p&gt;&#xA;&lt;p&gt;The &lt;code&gt;full_join&lt;/code&gt; function from the &lt;code&gt;dplyr&lt;/code&gt; R package then allows me to obtain a satisfactory result.&lt;/p&gt;&#xA;&lt;p&gt;I was looking for a script that only uses bash.&lt;/p&gt;&#xA;" OwnerUserId="12770" LastActivityDate="2024-03-26T04:56:06.043" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22316" PostTypeId="1" AcceptedAnswerId="22317" CreationDate="2024-03-26T11:09:54.597" Score="1" ViewCount="29" Body="&lt;p&gt;One of the requirements of simulating the protein chain is to check for missing atoms or residues. I.e., using software tools to identify (and, if necessary, model) any missing regions.&lt;/p&gt;&#xA;&lt;p&gt;How can I check if a PDB file is complete or incomplete using Python?&lt;/p&gt;&#xA;" OwnerUserId="12686" LastActivityDate="2024-03-26T16:35:28.963" Title="How can I check if a PDB file is incomplete?" Tags="|python|pdb|" AnswerCount="1" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22317" PostTypeId="2" ParentId="22316" CreationDate="2024-03-26T14:58:38.403" Score="3" Body="&lt;p&gt;OK, lets say you need that information for &lt;code&gt;2PE4.pdb&lt;/code&gt;, if you look at its webpage on RCSB PDB repository: &lt;a href=&quot;https://www.rcsb.org/structure/2pe4&quot; rel=&quot;nofollow noreferrer&quot;&gt;2PE4&lt;/a&gt; and display its .pdb format file at &lt;a href=&quot;https://files.rcsb.org/view/2PE4.pdb&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://files.rcsb.org/view/2PE4.pdb&lt;/a&gt;, you'll notice at &lt;code&gt;REMARK 465&lt;/code&gt;:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;REMARK 465                                                                      &#xA;REMARK 465 MISSING RESIDUES                                                     &#xA;REMARK 465 THE FOLLOWING RESIDUES WERE NOT LOCATED IN THE                       &#xA;REMARK 465 EXPERIMENT. (M=MODEL NUMBER; RES=RESIDUE NAME; C=CHAIN               &#xA;REMARK 465 IDENTIFIER; SSSEQ=SEQUENCE NUMBER; I=INSERTION CODE.)                &#xA;REMARK 465                                                                      &#xA;REMARK 465   M RES C SSSEQI                                                     &#xA;REMARK 465     GLY A   437                                                      &#xA;REMARK 465     HIS A   438                                                      &#xA;REMARK 465     HIS A   439                                                      &#xA;REMARK 465     HIS A   440                                                      &#xA;REMARK 465     HIS A   441                                                      &#xA;REMARK 465     HIS A   442                                                      &#xA;REMARK 465     HIS A   443            &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;this should be the missing residues from the full peptide sequence of the protein that is listed as SEQRES:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SEQRES   1 A  424  ARG SER PHE ARG GLY PRO LEU LEU PRO ASN ARG PRO PHE          &#xA;SEQRES   2 A  424  THR THR VAL TRP ASN ALA ASN THR GLN TRP CYS LEU GLU          &#xA;SEQRES   3 A  424  ARG HIS GLY VAL ASP VAL ASP VAL SER VAL PHE ASP VAL          &#xA;SEQRES   4 A  424  VAL ALA ASN PRO GLY GLN THR PHE ARG GLY PRO ASP MET          &#xA;SEQRES   5 A  424  THR ILE PHE TYR SER SER GLN LEU GLY THR TYR PRO TYR          &#xA;SEQRES   6 A  424  TYR THR PRO THR GLY GLU PRO VAL PHE GLY GLY LEU PRO          &#xA;SEQRES   7 A  424  GLN ASN ALA SER LEU ILE ALA HIS LEU ALA ARG THR PHE          &#xA;SEQRES   8 A  424  GLN ASP ILE LEU ALA ALA ILE PRO ALA PRO ASP PHE SER          &#xA;SEQRES   9 A  424  GLY LEU ALA VAL ILE ASP TRP GLU ALA TRP ARG PRO ARG          &#xA;SEQRES  10 A  424  TRP ALA PHE ASN TRP ASP THR LYS ASP ILE TYR ARG GLN          &#xA;SEQRES  11 A  424  ARG SER ARG ALA LEU VAL GLN ALA GLN HIS PRO ASP TRP          &#xA;SEQRES  12 A  424  PRO ALA PRO GLN VAL GLU ALA VAL ALA GLN ASP GLN PHE          &#xA;SEQRES  13 A  424  GLN GLY ALA ALA ARG ALA TRP MET ALA GLY THR LEU GLN          &#xA;SEQRES  14 A  424  LEU GLY ARG ALA LEU ARG PRO ARG GLY LEU TRP GLY PHE          &#xA;SEQRES  15 A  424  TYR GLY PHE PRO ASP CYS TYR ASN TYR ASP PHE LEU SER          &#xA;SEQRES  16 A  424  PRO ASN TYR THR GLY GLN CYS PRO SER GLY ILE ARG ALA          &#xA;SEQRES  17 A  424  GLN ASN ASP GLN LEU GLY TRP LEU TRP GLY GLN SER ARG          &#xA;SEQRES  18 A  424  ALA LEU TYR PRO SER ILE TYR MET PRO ALA VAL LEU GLU          &#xA;SEQRES  19 A  424  GLY THR GLY LYS SER GLN MET TYR VAL GLN HIS ARG VAL          &#xA;SEQRES  20 A  424  ALA GLU ALA PHE ARG VAL ALA VAL ALA ALA GLY ASP PRO          &#xA;SEQRES  21 A  424  ASN LEU PRO VAL LEU PRO TYR VAL GLN ILE PHE TYR ASP          &#xA;SEQRES  22 A  424  THR THR ASN HIS PHE LEU PRO LEU ASP GLU LEU GLU HIS          &#xA;SEQRES  23 A  424  SER LEU GLY GLU SER ALA ALA GLN GLY ALA ALA GLY VAL          &#xA;SEQRES  24 A  424  VAL LEU TRP VAL SER TRP GLU ASN THR ARG THR LYS GLU          &#xA;SEQRES  25 A  424  SER CYS GLN ALA ILE LYS GLU TYR MET ASP THR THR LEU          &#xA;SEQRES  26 A  424  GLY PRO PHE ILE LEU ASN VAL THR SER GLY ALA LEU LEU          &#xA;SEQRES  27 A  424  CYS SER GLN ALA LEU CYS SER GLY HIS GLY ARG CYS VAL          &#xA;SEQRES  28 A  424  ARG ARG THR SER HIS PRO LYS ALA LEU LEU LEU LEU ASN          &#xA;SEQRES  29 A  424  PRO ALA SER PHE SER ILE GLN LEU THR PRO GLY GLY GLY          &#xA;SEQRES  30 A  424  PRO LEU SER LEU ARG GLY ALA LEU SER LEU GLU ASP GLN          &#xA;SEQRES  31 A  424  ALA GLN MET ALA VAL GLU PHE LYS CYS ARG CYS TYR PRO          &#xA;SEQRES  32 A  424  GLY TRP GLN ALA PRO TRP CYS GLU ARG LYS SER MET TRP          &#xA;SEQRES  33 A  424  THR GLY HIS HIS HIS HIS HIS HIS  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Pay attention that the .pdb starts at residue position 20; that is a cloning artifact, as shown by:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;SEQADV 2PE4 ARG A   20  UNP  Q12794              CLONING ARTIFACT               &#xA;SEQADV 2PE4 SER A   21  UNP  Q12794              CLONING ARTIFACT               &#xA;SEQADV 2PE4 THR A  436  UNP  Q12794              CLONING ARTIFACT               &#xA;SEQADV 2PE4 GLY A  437  UNP  Q12794              CLONING ARTIFACT               &#xA;SEQADV 2PE4 HIS A  438  UNP  Q12794              EXPRESSION TAG                 &#xA;SEQADV 2PE4 HIS A  439  UNP  Q12794              EXPRESSION TAG                 &#xA;SEQADV 2PE4 HIS A  440  UNP  Q12794              EXPRESSION TAG                 &#xA;SEQADV 2PE4 HIS A  441  UNP  Q12794              EXPRESSION TAG                 &#xA;SEQADV 2PE4 HIS A  442  UNP  Q12794              EXPRESSION TAG                 &#xA;SEQADV 2PE4 HIS A  443  UNP  Q12794              EXPRESSION TAG              &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;See &lt;a href=&quot;https://bioinformatics.stackexchange.com/questions/14210/pdb-residue-numbering&quot;&gt;PDB residue numbering&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;Using &lt;strong&gt;Biopython&lt;/strong&gt; lib you could try using &lt;code&gt;2pe4.cif&lt;/code&gt; instead of .pdb file (&lt;a href=&quot;https://files.rcsb.org/view/2PE4.cif&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://files.rcsb.org/view/2PE4.cif&lt;/a&gt;) as input :&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-py prettyprint-override&quot;&gt;&lt;code&gt;# First import MMCIF2Dict function&#xA;from Bio.PDB.MMCIF2Dict import MMCIF2Dict&#xA;&#xA;# read in the CIF file.&#xA;pdb_info = MMCIF2Dict(&amp;quot;2pe4.cif&amp;quot;)&#xA;&#xA;# &#xA;print(pdb_info[&amp;quot;_pdbx_unobs_or_zero_occ_residues.label_seq_id&amp;quot;])&#xA;print(pdb_info[&amp;quot;_pdbx_unobs_or_zero_occ_residues.label_comp_id&amp;quot;])&#xA;print(list(zip(pdb_info[&amp;quot;_pdbx_unobs_or_zero_occ_residues.label_seq_id&amp;quot;],&#xA;pdb_info[&amp;quot;_pdbx_unobs_or_zero_occ_residues.label_comp_id&amp;quot;])))&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Result:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;['418', '419', '420', '421', '422', '423', '424']&#xA;['GLY', 'HIS', 'HIS', 'HIS', 'HIS', 'HIS', 'HIS']&#xA;[('418', 'GLY'), ('419', 'HIS'), ('420', 'HIS'), ('421', 'HIS'), ('422', 'HIS'), ('423', 'HIS'), ('424', 'HIS')]        &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;but for this kind of stuff it's probably better suited&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://mmcif.wwpdb.org/docs/sw-examples/python/html/index.html&quot; rel=&quot;nofollow noreferrer&quot;&gt;PDBx Python Parser &lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="10049" LastEditorUserId="650" LastEditDate="2024-03-26T16:35:28.963" LastActivityDate="2024-03-26T16:35:28.963" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22318" PostTypeId="1" CreationDate="2024-03-26T15:01:00.480" Score="-2" ViewCount="34" Body="&lt;p&gt;I wonder if there are any other more efficient long read aligners apart kma? I have heard there is some hisat2 long read application but it's not really efficient. Any suggestions?&lt;/p&gt;&#xA;" OwnerUserId="6948" LastEditorUserId="6948" LastEditDate="2024-03-28T09:10:45.553" LastActivityDate="2024-03-28T09:10:45.553" Title="Any other (good) long read aligners apart kma?" Tags="|sequence-alignment|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22319" PostTypeId="2" ParentId="22309" CreationDate="2024-03-27T00:39:29.697" Score="0" Body="&lt;ol&gt;&#xA;&lt;li&gt;Hmmmm .... I personally would not use a single VCF to calculate Fst in the way you are describing. It will also miss one of the nucleotides. I would aggregate across VCFs for a homologous position and use those frequencies. Its designed for heterozygotes, you are describing a haploid genome, like mtDNA (I think). Fst needs a genuine allelic frequency, whereas you've shoehorned it (possibly).&lt;/li&gt;&#xA;&lt;li&gt;Bi-allelic because that is a synonymous variation for most amino acids, i.e. without triggering an amino acid change.&lt;/li&gt;&#xA;&lt;li&gt;In terms of removing a variant ... unless you believe this represents the &lt;em&gt;in vitro&lt;/em&gt; error rates (which you can define from an invariant site) ... then okay. If it is above the &lt;em&gt;in vitro&lt;/em&gt; error rate then you can't remove it. I know the read-depth here, so I can't comment.&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;hr /&gt;&#xA;&lt;p&gt;Just to clarify Fst can be used for prokaryotic genomes (haploids), but unless I've misunderstood the question it seems to being derived from a single VCF ... single Metagenome (?). Thats not how it works. I can describe the what the calculation is measuring will artificially inflate Fst and you can't correct for it.&lt;/p&gt;&#xA;" OwnerUserId="3848" LastEditorUserId="3848" LastEditDate="2024-03-27T18:34:41.053" LastActivityDate="2024-03-27T18:34:41.053" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22321" PostTypeId="1" CreationDate="2024-03-27T09:13:47.567" Score="1" ViewCount="39" Body="&lt;p&gt;Spoiler: This is probably a theoretical question. But, of course, if you have any tool that does this I will be extremely happy.&lt;/p&gt;&#xA;&lt;p&gt;So, I have WGS data for an organism O, both wild type (WT-O) and modified (M-O). I was able to assemble the reads using the O reference genome. Then, I was able to compare WT-O against M-O. I finally have the variants files (vcf), which I annotated with SnpEff and VEP.&lt;/p&gt;&#xA;&lt;p&gt;Now what I need to do is to evaluate the effect of the variants at protein level. What I know from the vcf files is which nucleotides were affected, their position, which chromosome, gene, and whatnot (in most cases, although this is not always true for all the variants).&lt;/p&gt;&#xA;&lt;p&gt;Question is: how to translate these variants from nucleotides to amino acids (AAs) and then, eventually, compare (BLAST?) the AAs sequences containing the variants to check whether they match with any known protein?&lt;/p&gt;&#xA;&lt;p&gt;EDIT:&lt;/p&gt;&#xA;&lt;p&gt;so the final goal would be to know whether the variants are causing some weird protein to be produced which may have a deleterious effect. It can be something like a toxin, or an allergen that are normally not present but that can appear &amp;quot;by random chance&amp;quot;. I need to make sure that nothing like that happened and to be sure i need to know which proteins may result from the variants.&lt;/p&gt;&#xA;" OwnerUserId="3119" LastEditorUserId="3119" LastEditDate="2024-03-28T08:27:36.250" LastActivityDate="2024-03-28T08:27:36.250" Title="how to annotate the effect of a genomic variant to predict what the resulting protein is?" Tags="|proteins|vcf|wgs|variants|" AnswerCount="0" CommentCount="9" ContentLicense="CC BY-SA 4.0" />
  <row Id="22322" PostTypeId="1" CreationDate="2024-03-27T13:13:32.547" Score="0" ViewCount="16" Body="&lt;p&gt;Based on presence absence data, I'm attempting to find indicator species for different sample sites. I have 25 different groups or &amp;quot;clusters&amp;quot; as I've labelled them. Running the R code below, only results for 13 of the 25 groups was returned.&lt;/p&gt;&#xA;&lt;p&gt;Is the output truncated, i.e. is there a max output parameter in R I need to fix? Or does this imply that there simply aren't any statistically relevant indicator species in the groups not included in the output?&lt;/p&gt;&#xA;&lt;p&gt;Code used:&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;library(indicspecies)&#xA;&#xA;data = read.csv(&amp;quot;ISAI.txt&amp;quot;, header= TRUE, sep= &amp;quot;\t&amp;quot;) #Load up data from file&#xA;&#xA;abund = data[,3:ncol(data)] #Collect only numerical data, starts in 3rd column&#xA;cluster = data$Cluster #Groups are defined in the Cluster column of the imported data&#xA;&#xA;inv = multipatt(abund, cluster, func = &amp;quot;r.g&amp;quot;, control = how(nperm=1000),duleg = TRUE) #Run analysis.&#xA;&#xA;summary(inv) #print out results&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;&lt;strong&gt;Notes&lt;/strong&gt;:&lt;/p&gt;&#xA;&lt;p&gt;For the &lt;code&gt;func&lt;/code&gt; parameter, &amp;quot;r&amp;quot; was used due to data being in presence/absence format. The &amp;quot;g&amp;quot; corrects for groups with different numbers of members. The &lt;code&gt;duleg = True&lt;/code&gt; prevents group site combinations from being assessed as I'm not interested in those.&lt;/p&gt;&#xA;" OwnerUserId="18327" LastEditorUserId="650" LastEditDate="2024-03-27T17:19:33.393" LastActivityDate="2024-03-27T17:19:33.393" Title="multiplatt from indicspecies package not producing results for all groups" Tags="|r|features|" AnswerCount="1" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22323" PostTypeId="2" ParentId="22322" CreationDate="2024-03-27T14:52:21.740" Score="1" Body="&lt;p&gt;Got it! :)&lt;/p&gt;&#xA;&lt;p&gt;I switched the &lt;code&gt;summary(inv)&lt;/code&gt; to &lt;code&gt;summary(inv, alpha=1)&lt;/code&gt; to get all possible results and all my groups showed up.&lt;/p&gt;&#xA;&lt;p&gt;So, the answer to my own question is that the groups are missing from the output because they don't have any statistically significant OTUs correlated with those groups.&lt;/p&gt;&#xA;" OwnerUserId="18327" LastActivityDate="2024-03-27T14:52:21.740" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22325" PostTypeId="2" ParentId="22313" CreationDate="2024-03-27T17:47:57.917" Score="1" Body="&lt;p&gt;Vcftools allows you to filter to include or not include a list of sites.&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://vcftools.github.io/man_latest.html#SITE%20FILTERING%20OPTIONS&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://vcftools.github.io/man_latest.html#SITE%20FILTERING%20OPTIONS&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;If you can first grab the locations of indels of this size (for example you could write a python script  that goes through your vcf and grabs the location of anything with ref or alt of a length of at least 50 characters), then you can provide vcftools with a list of variants to filter on.&lt;/p&gt;&#xA;&lt;p&gt;If you are not comfortable writing your own script to look through the file, you could delete the vcf header lines, change the file extension to tsv, then open it in excel. Then you could use excel to filter for those sites and get their positions.&lt;/p&gt;&#xA;" OwnerUserId="1574" LastActivityDate="2024-03-27T17:47:57.917" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22326" PostTypeId="2" ParentId="22245" CreationDate="2024-03-27T18:04:24.883" Score="1" Body="&lt;p&gt;Find the location of the gene of interest and then filter the vcf for that location using vcftools, bcftools, plink, or some other software. There is a lot of good documentation available online for these tools.&lt;/p&gt;&#xA;" OwnerUserId="1574" LastActivityDate="2024-03-27T18:04:24.883" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22327" PostTypeId="1" CreationDate="2024-03-28T17:50:54.557" Score="0" ViewCount="21" Body="&lt;p&gt;I have been dealing with this problem for the past 48 hrs. I do not know what is it, which prevents the function from assessing the parameters it gets passed.&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;params &amp;lt;- GSEAGOHyperGParams(name = &amp;quot;My Custom GSEA based annot Params&amp;quot;,&#xA;                              geneSetCollection = new_df,&#xA;                              geneIds = new_df$GOIds,&#xA;                              universeGeneIds = gene_universe,&#xA;                              ontology = &amp;quot;BP&amp;quot;,&#xA;                              pvalueCutoff = 0.05,&#xA;                              conditional = FALSE,&#xA;                              testDirection = &amp;quot;over&amp;quot;,&#xA;                              organism = &amp;quot;human&amp;quot;)&#xA;Error in (function (classes, fdef, mtable)  : &#xA;  unable to find an inherited method for function ‘geneIdType’ for signature ‘&amp;quot;character&amp;quot;’ &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;In an example below (edited), you could see that it is character, but either way if I change it do integer, it does not work. Maybe character vector ?&lt;/p&gt;&#xA;&lt;pre class=&quot;lang-r prettyprint-override&quot;&gt;&lt;code&gt;&amp;gt; str(new_df)&#xA;'data.frame':   68 obs. of  4 variables:&#xA; $ GOIds   : chr  &amp;quot;GO:0001725&amp;quot; &amp;quot;GO:0001725&amp;quot; &amp;quot;GO:0003779&amp;quot; &amp;quot;GO:0005515&amp;quot; ...&#xA; $ evi1    : chr  &amp;quot;IBA&amp;quot; &amp;quot;IDA&amp;quot; &amp;quot;IEA&amp;quot; &amp;quot;IPI&amp;quot; ...&#xA; $ genes1  : chr  &amp;quot;23002&amp;quot; &amp;quot;23002&amp;quot; &amp;quot;23002&amp;quot; &amp;quot;23002&amp;quot; ...&#xA; $ ontology: chr  &amp;quot;CC&amp;quot; &amp;quot;CC&amp;quot; &amp;quot;MF&amp;quot; &amp;quot;MF&amp;quot; ...&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Same goes, when I try to cast it into an integer type by using &lt;code&gt;as.integer()&lt;/code&gt;.&#xA;Any suggestions for this function from an R package &lt;code&gt;clusterProfiler&lt;/code&gt;?&#xA;I believe this is from this package, but there could be one or two other packages, where this function maybe present.&lt;/p&gt;&#xA;" OwnerUserId="20151" LastEditorUserId="20151" LastEditDate="2024-03-28T23:00:58.810" LastActivityDate="2024-03-28T23:00:58.810" Title="Gene set enrichment ontology analysis issue" Tags="|r|kegg|gene-ontology|go|" AnswerCount="0" CommentCount="5" ContentLicense="CC BY-SA 4.0" />
  <row Id="22328" PostTypeId="1" CreationDate="2024-03-28T19:50:43.107" Score="3" ViewCount="24" Body="&lt;p&gt;I have a data set of several samples with their expression of proteins, in response to four different doses of a drug and two genotypes. I have been able to generate the two-way ANOVA using ezANOVA but I am having difficulty in generating multiple comparisons results. In GraphPad, I am able to compare means of each group to the control, the 0 dose, within each genotype and apply a Dunnett's correction. However, I am trying to implement this in R and my significant values are not lining up with GraphPad. I am using, emmeans and lme4 package to try generate these multiple comparisons. Any ideas or other readings to try? Thanks.&lt;/p&gt;&#xA;&lt;p&gt;The code for how I'm generating the anova results below&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(ez)&#xA;ezANOVA(data = data_long, &#xA;                         dv = .(Expression), &#xA;                         wid = .(Subject), &#xA;                         within = .(Dose), &#xA;                         between = .(Genotype),&#xA;                         type = 3, &#xA;                         detailed = TRUE,&#xA;                         return_aov = TRUE)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Then the code I used to generate the Dunnett's Test&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;library(DescTools)&#xA;&#xA;DunnettTest(data_long&lt;span class=&quot;math-container&quot;&gt;$Expression, data_long$&lt;/span&gt;Genotype)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;However the output of the Dunnett's test only compares from the Genotype and not the dosages within each Genotype at each dose.&lt;/p&gt;&#xA;&lt;p&gt;The output in R&#xA;&lt;a href=&quot;https://i.stack.imgur.com/5YABL.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/5YABL.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;The output in GraphPad&#xA;&lt;a href=&quot;https://i.stack.imgur.com/7DoMQ.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/7DoMQ.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I've tried switching from Genotype to Dose, but it doesn't separate the pairs by Genotype.&lt;/p&gt;&#xA;" OwnerUserId="20152" LastEditorUserId="20152" LastEditDate="2024-04-01T17:10:57.933" LastActivityDate="2024-04-01T17:10:57.933" Title="How to do post hoc comparisons after a repeated measures ANOVA in R" Tags="|statistics|modelling|graphs|lme4|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22329" PostTypeId="2" ParentId="22245" CreationDate="2024-03-28T21:55:01.117" Score="1" Body="&lt;p&gt;As mentioned, there are many options to go (eg. vcftools and bcftools). The Broad Institute provides a valuable toolset (GATK) for this very purpose and extended procedures for various variant selection tasks.&lt;/p&gt;&#xA;&lt;p&gt;SelectVariants function can be good for this purpose, though the location of a gene of interest should be addressed:&#xA;&lt;a href=&quot;https://gatk.broadinstitute.org/hc/en-us/articles/13832694334235-SelectVariants&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://gatk.broadinstitute.org/hc/en-us/articles/13832694334235-SelectVariants&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="15695" LastActivityDate="2024-03-28T21:55:01.117" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22330" PostTypeId="2" ParentId="21" CreationDate="2024-03-28T23:20:07.193" Score="-1" Body="&lt;p&gt;Note that the pDot of a certain mutation might differ based on which transcript you are using, and so Ensembl and RefSeq cannot be used interchangeably.&lt;/p&gt;&#xA;" OwnerUserId="1574" LastActivityDate="2024-03-28T23:20:07.193" CommentCount="3" ContentLicense="CC BY-SA 4.0" />
  <row Id="22331" PostTypeId="1" CreationDate="2024-03-29T08:32:58.733" Score="-1" ViewCount="28" Body="&lt;p&gt;Also &lt;a href=&quot;https://www.biostars.org/p/9591298/&quot; rel=&quot;nofollow noreferrer&quot;&gt;posted on biostars&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have a genbank file and I need the following details about the gene annotations in a text file&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# ID    name    start   stop    frame   product description function    cogcats status_region   status_function type    region_number   function_number&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;How do I go about extracting them? Please let me know if more information is needed from me! Thank you&lt;/p&gt;&#xA;" OwnerUserId="20155" LastEditorUserId="650" LastEditDate="2024-03-29T17:59:05.480" LastActivityDate="2024-03-29T17:59:05.480" Title="How do I get the gene annotations as a text file from a genbank file?" Tags="|gene|sequence-annotation|" AnswerCount="0" CommentCount="4" ContentLicense="CC BY-SA 4.0" />
  <row Id="22332" PostTypeId="1" CreationDate="2024-03-29T21:13:11.380" Score="2" ViewCount="33" Body="&lt;p&gt;Code:&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;$ python3 atram.py --max-processes=12 \&#xA;  --blast-db=/dgxb_home/se23plsc009/Project/results/aTRAM_db/T1.sqlite.db \&#xA;  --query-split=/dgxb_home/se23plsc009/Project/data/ref_orthologs/2395_query_file \&#xA;  --output-prefix=/dgxb_home/se23plsc009/Project/results/Temporary/Output.fasta-unitigs.fa \&#xA;  --assembler=abyss \&#xA;  --path=/dgxb_home/se23plsc009/softwares/aTRAM/lib/assemblers/velvet.py \&#xA;  --iterations=3 \&#xA;  --max-target-seqs=3000 \&#xA;  --log-file=/dgxb_home/se23plsc009/Project/results/aTRAM_assembly1/log_file \&#xA;  --temp-dir=/dgxb_home/se23plsc009/Project/results/aTRAM_assembly1/Temporary/ \&#xA;  --keep-temp-dir \&#xA;  --protein&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Output:&#xA;&lt;a href=&quot;https://i.stack.imgur.com/LSDeh.png&quot; rel=&quot;nofollow noreferrer&quot;&gt;&lt;img src=&quot;https://i.stack.imgur.com/LSDeh.png&quot; alt=&quot;enter image description here&quot; /&gt;&lt;/a&gt;&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;2024-03-29 08:21:29.100271 INFO : aTRAM blast DB = &amp;quot;/dgxb_home/se23plsc009/Thrips_Project/results/aTRAM_db/T1&amp;quot;, query = &amp;quot;2395_query_file_RPRC015459_PA_2395.fasta&amp;quot;, iteration 1&#xA;2024-03-29 08:21:29.101075 INFO : Blasting query against shards: iteration 1&#xA;2024-03-29 08:23:00.952156 INFO : All 93 blast results completed&#xA;2024-03-29 08:23:00.954933 INFO : 72 blast hits in iteration 1&#xA;2024-03-29 08:23:00.954965 INFO : Writing assembler input files: iteration 1&#xA;2024-03-29 08:23:01.258918 INFO : Assembling shards with abyss: iteration 1&#xA;2024-03-29 08:23:01.406976 ERROR: Exception: [Errno 2] No such file or directory: '/dgxb_home/se23plsc009/Thrips_Project/results/aTRAM_assembly/Temporary/atram_nowsbxk1/T1_2395_query_file_RPRC015459_PA_2395.fasta_01_clkdfnle/output.fasta-unitigs.fa'&#xA;2024-03-29 08:23:01.409907 INFO : Writing 0 filtered contigs after iteration 1&#xA;2024-03-29 08:23:01.412235 INFO : 0 total contigs after iteration 1&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I have tried running the job with different assembler, gave the path to &amp;quot;Output.fasta-unitigs.fa&amp;quot; as output folder but nothing is working.&lt;/p&gt;&#xA;&lt;p&gt;I've attached below the github link of the tool for reference: &lt;a href=&quot;https://github.com/juliema/aTRAM/blob/master/doc/atram_reference.md&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/juliema/aTRAM/blob/master/doc/atram_reference.md&lt;/a&gt;&lt;/p&gt;&#xA;&lt;p&gt;I have checked to make sure that each output directory exists.&lt;/p&gt;&#xA;&lt;p&gt;Here is the github issue that I think is related to my problem:&lt;/p&gt;&#xA;&lt;p&gt;&lt;a href=&quot;https://github.com/juliema/aTRAM/issues/308&quot; rel=&quot;nofollow noreferrer&quot;&gt;https://github.com/juliema/aTRAM/issues/308&lt;/a&gt;&lt;/p&gt;&#xA;" OwnerUserId="20156" LastEditorUserId="73" LastEditDate="2024-04-04T20:50:57.923" LastActivityDate="2024-04-04T20:50:57.923" Title="I'm trying to run aTRAM tool for assembly, but i'm stuck on this error:" Tags="|ngs|assembly|" AnswerCount="0" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
  <row Id="22333" PostTypeId="1" CreationDate="2024-03-30T13:00:15.930" Score="0" ViewCount="14" Body="&lt;p&gt;Has anyone tried to analyse the VCF files provided by Dante Labs? (Human WGS, Illumina)&lt;/p&gt;&#xA;&lt;p&gt;I'm looking into .raw.vcf and .filtered.vcf.&lt;/p&gt;&#xA;&lt;p&gt;For reference sequence, all I can see is 2 contigs:&#xA;GL000225.1 and GL000192.1.&#xA;All variants belong to one of these.&#xA;There's no conventional CHROM column.&lt;/p&gt;&#xA;&lt;p&gt;I don't know how those coordinates map to chromosomes in GRCh37 or GRCh38 assemblies.&lt;/p&gt;&#xA;&lt;ul&gt;&#xA;&lt;li&gt;GL000192 &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTracks?db=hg19&amp;amp;chromInfoPage=&quot; rel=&quot;nofollow noreferrer&quot;&gt;corresponds&lt;/a&gt; to chr1_gl000192_random&lt;/li&gt;&#xA;&lt;li&gt;GL000225 &lt;a href=&quot;https://genome.ucsc.edu/cgi-bin/hgTracks?db=hg19&amp;amp;chromInfoPage=&quot; rel=&quot;nofollow noreferrer&quot;&gt;corresponds&lt;/a&gt; to chrUn_gl000225&lt;/li&gt;&#xA;&lt;/ul&gt;&#xA;&lt;p&gt;Moreover, there's no VCF header with metainfo, all I can see is 1 top row&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;:11,27:0.711:38:4,11:7,16:25:64,0,24:2.8811e+01,1.4126e-02,2.7139e+01:0.00,34.77,37.77:6,5,12,15:7,4,18,9.&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;ol&gt;&#xA;&lt;li&gt;What's the use of those VCFs?&lt;/li&gt;&#xA;&lt;li&gt;What variants do they contain?&lt;/li&gt;&#xA;&lt;li&gt;What version of VCF it is and where can I get the standard metainfo?&lt;/li&gt;&#xA;&lt;li&gt;Is there a map of those coordinates to any core assembly&#xA;chromosomes?&lt;/li&gt;&#xA;&lt;li&gt;Am I right they contain not all genomic VCFs but only a small part&#xA;of the genome? (Size of GL000225.1 contig is 211,173 bp and size of&#xA;GL000192.1 is 547,496 bp, that obviously doesn't sum up to the whole&#xA;genome).&lt;/li&gt;&#xA;&lt;/ol&gt;&#xA;&lt;p&gt;Many thanks for your replies!&lt;/p&gt;&#xA;&lt;p&gt;UPD: I found a mistake, sorry. The file was incomplete, only tail was uploaded to Jupyterhub.&#xA;In local Jupyter, the file loaded completely, it actually has the  header and CHROM column.&lt;/p&gt;&#xA;" OwnerUserId="20162" LastEditorUserId="20162" LastEditDate="2024-03-30T15:55:38.210" LastActivityDate="2024-03-30T15:55:38.210" Title="Dante Labs VCF: UPDATE: SOLVED" Tags="|assembly|chromosomes|" AnswerCount="0" CommentCount="0" ClosedDate="2024-03-30T16:39:07.643" ContentLicense="CC BY-SA 4.0" />
  <row Id="22334" PostTypeId="1" CreationDate="2024-03-31T02:36:25.460" Score="1" ViewCount="17" Body="&lt;p&gt;I'm having some major chaos with the output table generated and then passing that to my plot function. New to this (1st time), could somebody review this piece of code and suggest some corrections. I saved the output of a data frame as a txt file and it does not look good at all. The numbers and columns are there, but they follow some sort of strange wa, like a staircase.&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;# Create DataFrame from significant results&#xA;df_go = pd.DataFrame(&#xA;    [(&#xA;        r.GO,&#xA;        r.goterm.name,&#xA;        r.goterm.namespace,&#xA;        r.p_uncorrected,&#xA;        len(r.study_items),  # Number of study genes&#xA;        GO_items.count(r.GO),&#xA;        ', '.join([rev_mapper[y] for y in r.study_items])  # Concatenate study genes into a single string&#xA;    ) for r in goea_results_sig],&#xA;    columns=['GO', 'term', 'p', 'n_genes', 'n_study', 'n_go', 'study_genes']&#xA;)&#xA;&#xA;print(df_go)&#xA;# Save DataFrame to a text file (CSV format)&#xA;df_go.to_csv('df_go_results.txt', sep='\t', index=False)&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;I can provide more code, because when I tried to commit and push from IDE onto github I got the following message.&lt;/p&gt;&#xA;&lt;p&gt;&lt;code&gt;Successfully created project 'GSEAplots' on GitHub, but initial commit failed: inflate: data stream error (incorrect data check) corrupt loose object 'a87848b96903648398f723df040fd0be540a025d' unable to read a87848b96903648398f723df040fd0be540a025d&lt;/code&gt;&lt;/p&gt;&#xA;&lt;p&gt;No idea why or what ? Git does not seem to interact with PyCharm and/ or Github really well ?&lt;/p&gt;&#xA;&lt;p&gt;The plots code :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;import matplotlib.pyplot as plt&#xA;import numpy as np&#xA;&#xA;plt.figure(figsize=(6,4),dpi=300)&#xA;ax = plt.gca()&#xA;ax.spines['top'].set_visible(False)&#xA;ax.spines['right'].set_visible(False)&#xA;&#xA;plt.rcParams[&amp;quot;font.size&amp;quot;] = 18&#xA;plt.rcParams[&amp;quot;font.family&amp;quot;] = &amp;quot;Helvetica&amp;quot;&#xA;&#xA;&#xA;plt.scatter(df_p['Z'],-np.log10(df_p['P(sgRNAs)']),s=18,color='grey',alpha=0.3,linewidths=0.3,edgecolors='grey')&#xA;&#xA;&#xA;label_ls1 = list(df_up['Gene'])&#xA;&#xA;df_show1 = df_p.loc[label_ls1,:]&#xA;lfc_ls1 = list(df_show1['Z']); p_ls1 = list(-np.log10(df_show1['P(sgRNAs)']))&#xA;&#xA;plt.scatter(lfc_ls1,p_ls1,s=20,color='red',linewidths=0.5,edgecolors='grey')&#xA;&#xA;for i, txt in enumerate(label_ls1[-4:]):&#xA;    plt.annotate(txt, (lfc_ls1[-4:][i]+0.03, p_ls1[-4:][i]-0.08),fontsize=8)&#xA;    &#xA;label_ls2 = list(df_down['Gene'])&#xA;&#xA;df_show2 = df_p.loc[label_ls2,:]&#xA;lfc_ls2 = list(df_show2['Z']); p_ls2 = list(-np.log10(df_show2['P(sgRNAs)']))&#xA;&#xA;plt.scatter(lfc_ls2,p_ls2,s=20,color='blue',linewidths=0.5,edgecolors='grey')&#xA;&#xA;for i, txt in enumerate(label_ls2[1:5]):&#xA;    plt.annotate(txt, (lfc_ls2[1:5][i]+0.03, p_ls2[1:5][i]-0.08),fontsize=8)&#xA;&#xA;plt.axhline(y=1.3, linestyle='--', color='lightgray') &#xA;plt.axvline(x=-2, linestyle='--', color='lightgray') &#xA;plt.axvline(x=2, linestyle='--', color='lightgray') &#xA;&#xA;plt.xlabel('Differential score (PRMT5/DMSO)')&#xA;plt.ylabel('-log10(p-value)')&#xA;&#xA;&#xA;#plt.savefig('DifferentialEssential_volcanoplot.png',dpi=300,facecolor='white')&#xA;plt.show()&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;Data frame that I am trying to plot has the following structure :&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;             total_samples  occurrence_total  occurrence_CRSsNP  \&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;ENSG00000100592             15                15                  8&lt;br /&gt;&#xA;ENSG00000101336             15                14                  8&lt;br /&gt;&#xA;ENSG00000134285             15                15                  8&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;             occurrence_CTL  sparsity  rlog_av_CTL  rlog_av_CRSsNP  \&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;ENSG00000100592               7  0.099193    11.709476       11.413235&lt;br /&gt;&#xA;ENSG00000101336               6  0.322330     4.084709        6.179892&lt;br /&gt;&#xA;ENSG00000134285               7  0.098323     9.169283        9.595925&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;            rlog_Regulation  vst_av_CTL  vst_av_CRSsNP vst_Regulation  \&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;ENSG00000100592             CTL   11.790292      11.367062            CTL&lt;br /&gt;&#xA;ENSG00000101336          CRSsNP    4.909465       7.051789         CRSsNP&lt;br /&gt;&#xA;ENSG00000134285          CRSsNP    9.142557       9.734598         CRSsNP&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;              raw_av_CTL  raw_av_CRSsNP Raw_Regulation structural_zero  \&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;ENSG00000100592  3973.285714       2428.500            CTL              No&lt;br /&gt;&#xA;ENSG00000101336    10.285714        119.750         CRSsNP              No&lt;br /&gt;&#xA;ENSG00000134285   602.857143        761.625         CRSsNP              No&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;             DESeq2Norm_av_CTL  DESeq2Norm_av_CRSsNP DESeq_Regulation  \&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;ENSG00000100592        3526.244455           2614.486202              CTL&lt;br /&gt;&#xA;ENSG00000101336           9.293777            131.796045           CRSsNP&lt;br /&gt;&#xA;ENSG00000134285         535.929110            828.762618           CRSsNP&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;                baseMean  log2FoldChange     lfcSE      stat  \&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;ENSG00000100592  3039.973387       -0.431423  0.076683 -5.626030&lt;br /&gt;&#xA;ENSG00000101336    74.628320        3.828859  0.605862  6.319689&lt;br /&gt;&#xA;ENSG00000134285   692.106981        0.628415  0.105894  5.934376&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;             pvalue_DESEq2  padj_DESEq2        pvalue      padj  \&#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;ENSG00000100592   1.844041e-08     0.000120  3.694635e-06  0.038127&lt;br /&gt;&#xA;ENSG00000101336   2.620902e-10     0.000005  2.009572e-07  0.006221&lt;br /&gt;&#xA;ENSG00000134285   2.949646e-09     0.000029  1.052822e-06  0.016297&lt;/p&gt;&#xA;&lt;pre&gt;&lt;code&gt;             errorBarUp  errorBarLow Final_Regulation external_gene_name  &#xA;&lt;/code&gt;&lt;/pre&gt;&#xA;&lt;p&gt;ENSG00000100592   -0.354739    -0.508106              CTL              DAAM1&lt;br /&gt;&#xA;ENSG00000101336    4.434721     3.222997           CRSsNP                HCK&lt;br /&gt;&#xA;ENSG00000134285    0.734309     0.522521           CRSsNP             FKBP11&lt;/p&gt;&#xA;" OwnerUserId="20151" LastActivityDate="2024-03-31T02:36:25.460" Title="Trying to plot the results of GSEA in python" Tags="|python|gsea|pandas|gene-ontology|go|" AnswerCount="0" CommentCount="1" ContentLicense="CC BY-SA 4.0" />
  <row Id="22335" PostTypeId="2" ParentId="4962" CreationDate="2024-03-31T15:57:33.467" Score="1" Body="&lt;p&gt;You could use the &lt;a href=&quot;https://api.bacdive.dsmz.de/&quot; rel=&quot;nofollow noreferrer&quot;&gt;BacDive API&lt;/a&gt;, or you could download the kgx formatted files from &lt;a href=&quot;https://github.com/Knowledge-Graph-Hub/kg-microbe&quot; rel=&quot;nofollow noreferrer&quot;&gt;KG Microbe&lt;/a&gt;.&lt;/p&gt;&#xA;" OwnerUserId="1322" LastActivityDate="2024-03-31T15:57:33.467" CommentCount="0" ContentLicense="CC BY-SA 4.0" />
</posts>